2024-04-25T13:45:13.070 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T13:45:13.070 DEBUG:teuthology.run:Teuthology command: teuthology --name pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi --archive /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673185 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/metrics} --owner scheduled_pdonnell@teuthology --verbose -- /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673185/orig.config.yaml 2024-04-25T13:45:13.118 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T13:45:13.207 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673185 branch: wip-pdonnell-testing-20240425.015853-debug description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/metrics} email: pdonnell@redhat.com first_in_suite: false job_id: '7673185' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-pdonnell-testing-20240425.015853-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 kclient: syntax: v1 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240425.015853-debug sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1880 sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 sleep_before_teardown: 0 subset: 178/512 suite: fs suite_branch: wip-pdonnell-testing-20240425.015853-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi086.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI/vyPCz35KI9WNWis9mr7nPzhgchQLOuGOS0aRaeuodF47ZVBEp1Nc94q0LWfJLamHLYt/NGpuniB3x65kAv+Y= smithi155.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJBmjCfRPUUkkwsMdWyBgBoqX7RlPMKPwyPeoefSpw0DFoNQnpoLioDSL4biIsefldSwVDMQ0l+7kbfLgBZVYQ4= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_mds_metrics teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_10:00:23 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T13:45:13.207 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa; will attempt to use it 2024-04-25T13:45:13.208 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks 2024-04-25T13:45:13.208 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T13:45:13.209 INFO:teuthology.task.internal:Checking packages... 2024-04-25T13:45:13.234 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '994e2f20102a20fa9a27966c14b625cc01884df8' 2024-04-25T13:45:13.235 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T13:45:13.235 INFO:teuthology.packaging:ref: None 2024-04-25T13:45:13.235 INFO:teuthology.packaging:tag: None 2024-04-25T13:45:13.235 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240425.015853-debug 2024-04-25T13:45:13.235 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:45:13.235 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-pdonnell-testing-20240425.015853-debug 2024-04-25T13:45:13.541 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3291-g994e2f20-1jammy 2024-04-25T13:45:13.545 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T13:45:13.555 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T13:45:13.555 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T13:45:13.568 INFO:teuthology.task.internal:Saving configuration 2024-04-25T13:45:13.587 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T13:45:13.597 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T13:45:13.623 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673185', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 13:39:15.316923', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDQxu/qHTsShNA9lE+QFg12FsljnGMqqoq1+FoUQvGB0'} 2024-04-25T13:45:13.648 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673185', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 13:39:15.315026', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEeBKmteSFMoHFbbc3yxpaZ4HfDtw6DOfd2DGRwROgyq'} 2024-04-25T13:45:13.648 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T13:45:13.659 INFO:teuthology.task.internal:roles: ubuntu@smithi086.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T13:45:13.659 INFO:teuthology.task.internal:roles: ubuntu@smithi155.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T13:45:13.659 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T13:45:13.729 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f10a0ad03a0>, signals=[15]) 2024-04-25T13:45:13.730 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T13:45:13.738 INFO:teuthology.task.internal:Opening connections... 2024-04-25T13:45:13.738 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T13:45:13.740 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:45:13.820 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T13:45:13.821 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:45:13.901 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T13:45:13.909 DEBUG:teuthology.orchestra.run.smithi086:> uname -m 2024-04-25T13:45:13.915 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64 2024-04-25T13:45:13.915 DEBUG:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2024-04-25T13:45:13.964 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T13:45:13.964 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="Ubuntu" 2024-04-25T13:45:13.964 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="22.04" 2024-04-25T13:45:13.964 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T13:45:13.964 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_CODENAME=jammy 2024-04-25T13:45:13.964 INFO:teuthology.orchestra.run.smithi086.stdout:ID=ubuntu 2024-04-25T13:45:13.965 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE=debian 2024-04-25T13:45:13.965 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T13:45:13.965 INFO:teuthology.orchestra.run.smithi086.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T13:45:13.965 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T13:45:13.965 INFO:teuthology.orchestra.run.smithi086.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T13:45:13.965 INFO:teuthology.orchestra.run.smithi086.stdout:UBUNTU_CODENAME=jammy 2024-04-25T13:45:13.965 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2024-04-25T13:45:13.992 DEBUG:teuthology.orchestra.run.smithi155:> uname -m 2024-04-25T13:45:13.998 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2024-04-25T13:45:13.998 DEBUG:teuthology.orchestra.run.smithi155:> cat /etc/os-release 2024-04-25T13:45:14.046 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T13:45:14.046 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="Ubuntu" 2024-04-25T13:45:14.046 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="22.04" 2024-04-25T13:45:14.046 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T13:45:14.046 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_CODENAME=jammy 2024-04-25T13:45:14.046 INFO:teuthology.orchestra.run.smithi155.stdout:ID=ubuntu 2024-04-25T13:45:14.046 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE=debian 2024-04-25T13:45:14.046 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T13:45:14.046 INFO:teuthology.orchestra.run.smithi155.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T13:45:14.046 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T13:45:14.047 INFO:teuthology.orchestra.run.smithi155.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T13:45:14.047 INFO:teuthology.orchestra.run.smithi155.stdout:UBUNTU_CODENAME=jammy 2024-04-25T13:45:14.047 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2024-04-25T13:45:14.075 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T13:45:14.089 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T13:45:14.104 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T13:45:14.104 DEBUG:teuthology.orchestra.run.smithi086:> test '!' -e /home/ubuntu/cephtest 2024-04-25T13:45:14.106 DEBUG:teuthology.orchestra.run.smithi155:> test '!' -e /home/ubuntu/cephtest 2024-04-25T13:45:14.111 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T13:45:14.120 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T13:45:14.120 DEBUG:teuthology.orchestra.run.smithi086:> test -z $(ls -A /var/lib/ceph) 2024-04-25T13:45:14.153 DEBUG:teuthology.orchestra.run.smithi155:> test -z $(ls -A /var/lib/ceph) 2024-04-25T13:45:14.251 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T13:45:14.308 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T13:45:14.324 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T13:45:14.325 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-04-25T13:45:14.325 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-04-25T13:45:14.325 DEBUG:teuthology.orchestra.run.smithi086:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T13:45:14.326 DEBUG:teuthology.orchestra.run.smithi155:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T13:45:14.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:45:14.331 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-04-25T13:45:14.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:45:14.334 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-04-25T13:45:14.687 DEBUG:teuthology.task.kernel:sha1 for client.1 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T13:45:14.687 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T13:45:14.688 DEBUG:teuthology.orchestra.run.smithi155:> uname -r 2024-04-25T13:45:14.693 INFO:teuthology.orchestra.run.smithi155.stdout:5.15.0-56-generic 2024-04-25T13:45:14.693 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T13:45:14.693 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T13:45:14.694 INFO:teuthology.task.kernel:Installing linux-firmware on client.1... 2024-04-25T13:45:14.694 DEBUG:teuthology.orchestra.run.smithi155:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-04-25T13:45:14.696 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T13:45:14.696 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T13:45:14.696 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-04-25T13:45:14.701 INFO:teuthology.orchestra.run.smithi086.stdout:5.15.0-56-generic 2024-04-25T13:45:14.701 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T13:45:14.701 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T13:45:14.702 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2024-04-25T13:45:14.702 DEBUG:teuthology.orchestra.run.smithi086:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-04-25T13:45:14.757 INFO:teuthology.orchestra.run.smithi155.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-04-25T13:45:14.764 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-04-25T13:45:14.898 INFO:teuthology.orchestra.run.smithi086.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-04-25T13:45:14.899 INFO:teuthology.orchestra.run.smithi086.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-04-25T13:45:14.899 INFO:teuthology.orchestra.run.smithi086.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-04-25T13:45:14.899 INFO:teuthology.orchestra.run.smithi086.stderr:hint: 2024-04-25T13:45:14.899 INFO:teuthology.orchestra.run.smithi086.stderr:hint: git config --global init.defaultBranch 2024-04-25T13:45:14.899 INFO:teuthology.orchestra.run.smithi086.stderr:hint: 2024-04-25T13:45:14.899 INFO:teuthology.orchestra.run.smithi086.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-04-25T13:45:14.899 INFO:teuthology.orchestra.run.smithi086.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-04-25T13:45:14.899 INFO:teuthology.orchestra.run.smithi086.stderr:hint: 2024-04-25T13:45:14.899 INFO:teuthology.orchestra.run.smithi086.stderr:hint: git branch -m 2024-04-25T13:45:14.900 INFO:teuthology.orchestra.run.smithi086.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-04-25T13:45:14.902 DEBUG:teuthology.orchestra.run.smithi086:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-04-25T13:45:14.968 INFO:teuthology.orchestra.run.smithi155.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-04-25T13:45:14.968 INFO:teuthology.orchestra.run.smithi155.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-04-25T13:45:14.968 INFO:teuthology.orchestra.run.smithi155.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-04-25T13:45:14.968 INFO:teuthology.orchestra.run.smithi155.stderr:hint: 2024-04-25T13:45:14.968 INFO:teuthology.orchestra.run.smithi155.stderr:hint: git config --global init.defaultBranch 2024-04-25T13:45:14.969 INFO:teuthology.orchestra.run.smithi155.stderr:hint: 2024-04-25T13:45:14.969 INFO:teuthology.orchestra.run.smithi155.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-04-25T13:45:14.969 INFO:teuthology.orchestra.run.smithi155.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-04-25T13:45:14.969 INFO:teuthology.orchestra.run.smithi155.stderr:hint: 2024-04-25T13:45:14.969 INFO:teuthology.orchestra.run.smithi155.stderr:hint: git branch -m 2024-04-25T13:45:14.970 DEBUG:teuthology.orchestra.run.smithi086:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-04-25T13:45:14.970 INFO:teuthology.orchestra.run.smithi155.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-04-25T13:45:14.971 DEBUG:teuthology.orchestra.run.smithi155:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-04-25T13:45:15.028 DEBUG:teuthology.orchestra.run.smithi086:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-04-25T13:45:15.037 DEBUG:teuthology.orchestra.run.smithi155:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-04-25T13:45:15.093 DEBUG:teuthology.orchestra.run.smithi155:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-04-25T13:48:29.132 INFO:teuthology.orchestra.run.smithi155.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-04-25T13:48:29.133 INFO:teuthology.orchestra.run.smithi155.stderr: * [new branch] main -> origin/main 2024-04-25T13:48:29.159 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20190312 -> 20190312 2024-04-25T13:48:29.159 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20190416 -> 20190416 2024-04-25T13:48:29.159 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20190514 -> 20190514 2024-04-25T13:48:29.159 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20190618 -> 20190618 2024-04-25T13:48:29.159 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20190717 -> 20190717 2024-04-25T13:48:29.159 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20190815 -> 20190815 2024-04-25T13:48:29.159 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20190923 -> 20190923 2024-04-25T13:48:29.159 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20191022 -> 20191022 2024-04-25T13:48:29.160 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20191215 -> 20191215 2024-04-25T13:48:29.160 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20200122 -> 20200122 2024-04-25T13:48:29.160 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20200316 -> 20200316 2024-04-25T13:48:29.160 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20200421 -> 20200421 2024-04-25T13:48:29.160 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20200519 -> 20200519 2024-04-25T13:48:29.164 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20200619 -> 20200619 2024-04-25T13:48:29.164 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20200721 -> 20200721 2024-04-25T13:48:29.164 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20200817 -> 20200817 2024-04-25T13:48:29.164 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20200918 -> 20200918 2024-04-25T13:48:29.164 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20201022 -> 20201022 2024-04-25T13:48:29.164 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20201118 -> 20201118 2024-04-25T13:48:29.164 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20201218 -> 20201218 2024-04-25T13:48:29.164 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20210208 -> 20210208 2024-04-25T13:48:29.164 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20210315 -> 20210315 2024-04-25T13:48:29.164 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20210511 -> 20210511 2024-04-25T13:48:29.165 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20210716 -> 20210716 2024-04-25T13:48:29.165 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20210818 -> 20210818 2024-04-25T13:48:29.165 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20210919 -> 20210919 2024-04-25T13:48:29.165 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20211027 -> 20211027 2024-04-25T13:48:29.165 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20211216 -> 20211216 2024-04-25T13:48:29.165 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20220209 -> 20220209 2024-04-25T13:48:29.165 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20220310 -> 20220310 2024-04-25T13:48:29.165 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20220411 -> 20220411 2024-04-25T13:48:29.165 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20220509 -> 20220509 2024-04-25T13:48:29.165 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20220610 -> 20220610 2024-04-25T13:48:29.165 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20220708 -> 20220708 2024-04-25T13:48:29.165 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20220815 -> 20220815 2024-04-25T13:48:29.166 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20220913 -> 20220913 2024-04-25T13:48:29.166 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20221012 -> 20221012 2024-04-25T13:48:29.166 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20221109 -> 20221109 2024-04-25T13:48:29.166 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20221214 -> 20221214 2024-04-25T13:48:29.166 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20230117 -> 20230117 2024-04-25T13:48:29.166 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20230210 -> 20230210 2024-04-25T13:48:29.166 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20230310 -> 20230310 2024-04-25T13:48:29.166 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20230404 -> 20230404 2024-04-25T13:48:29.166 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20230515 -> 20230515 2024-04-25T13:48:29.166 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20230625 -> 20230625 2024-04-25T13:48:29.166 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20230804 -> 20230804 2024-04-25T13:48:29.166 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20230919 -> 20230919 2024-04-25T13:48:29.167 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20231030 -> 20231030 2024-04-25T13:48:29.167 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20231111 -> 20231111 2024-04-25T13:48:29.168 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20231211 -> 20231211 2024-04-25T13:48:29.168 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20240115 -> 20240115 2024-04-25T13:48:29.168 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20240220 -> 20240220 2024-04-25T13:48:29.168 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20240312 -> 20240312 2024-04-25T13:48:29.168 INFO:teuthology.orchestra.run.smithi155.stderr: * [new tag] 20240410 -> 20240410 2024-04-25T13:48:33.445 INFO:teuthology.orchestra.run.smithi155.stderr:Updating files: 45% (1524/3381) Updating files: 46% (1556/3381) Updating files: 47% (1590/3381) Updating files: 48% (1623/3381) Updating files: 49% (1657/3381) Updating files: 50% (1691/3381) Updating files: 51% (1725/3381) Updating files: 52% (1759/3381) Updating files: 53% (1792/3381) Updating files: 54% (1826/3381) Updating files: 55% (1860/3381) Updating files: 56% (1894/3381) Updating files: 57% (1928/3381) Updating files: 58% (1961/3381) Updating files: 59% (1995/3381) Updating files: 59% (2008/3381) Updating files: 60% (2029/3381) Updating files: 61% (2063/3381) Updating files: 62% (2097/3381) Updating files: 63% (2131/3381) Updating files: 64% (2164/3381) Updating files: 65% (2198/3381) Updating files: 66% (2232/3381) Updating files: 67% (2266/3381) Updating files: 68% (2300/3381) Updating files: 69% (2333/3381) Updating files: 69% (2362/3381) Updating files: 70% (2367/3381) Updating files: 71% (2401/3381) Updating files: 72% (2435/3381) Updating files: 73% (2469/3381) Updating files: 74% (2502/3381) Updating files: 75% (2536/3381) Updating files: 76% (2570/3381) Updating files: 77% (2604/3381) Updating files: 78% (2638/3381) Updating files: 79% (2671/3381) Updating files: 80% (2705/3381) Updating files: 81% (2739/3381) Updating files: 82% (2773/3381) Updating files: 83% (2807/3381) Updating files: 83% (2811/3381) Updating files: 84% (2841/3381) Updating files: 85% (2874/3381) Updating files: 86% (2908/3381) Updating files: 87% (2942/3381) Updating files: 88% (2976/3381) Updating files: 89% (3010/3381) Updating files: 90% (3043/3381) Updating files: 91% (3077/3381) Updating files: 92% (3111/3381) Updating files: 93% (3145/3381) Updating files: 94% (3179/3381) Updating files: 95% (3212/3381) Updating files: 96% (3246/3381) Updating files: 97% (3280/3381) Updating files: 98% (3314/3381) Updating files: 99% (3348/3381) Updating files: 100% (3381/3381) Updating files: 100% (3381/3381), done. 2024-04-25T13:48:33.450 INFO:teuthology.orchestra.run.smithi155.stdout:HEAD is now at 63a66b66 Merge branch 'robot/pr-0-1713990671' into 'main' 2024-04-25T13:48:33.472 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-25T13:48:33.473 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T13:48:33.732 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/ 2024-04-25T13:48:33.733 DEBUG:teuthology.orchestra.run.smithi155:> rm -f /tmp/linux-image.deb && echo linux-image-6.9.0-rc3-g806ec36b2112_6.9.0-rc3-g806ec36b2112-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/ --input-file=- 2024-04-25T13:48:33.735 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi155.front.sepia.ceph.com to complete... 2024-04-25T13:48:35.874 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25 13:48:35 URL:https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/linux-image-6.9.0-rc3-g806ec36b2112_6.9.0-rc3-g806ec36b2112-1_amd64.deb [44805284/44805284] -> "/tmp/linux-image.deb" [1] 2024-04-25T13:48:35.874 INFO:teuthology.orchestra.run.smithi155.stderr:FINISHED --2024-04-25 13:48:35-- 2024-04-25T13:48:35.874 INFO:teuthology.orchestra.run.smithi155.stderr:Total wall clock time: 2.0s 2024-04-25T13:48:35.874 INFO:teuthology.orchestra.run.smithi155.stderr:Downloaded: 1 files, 43M in 1.9s (22.5 MB/s) 2024-04-25T13:48:35.879 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-25T13:48:35.879 DEBUG:teuthology.orchestra.run.smithi155:> sudo dpkg -i /tmp/linux-image.deb 2024-04-25T13:48:36.141 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package linux-image-6.9.0-rc3-g806ec36b2112. 2024-04-25T13:48:38.112 INFO:teuthology.orchestra.run.smithi155.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-04-25T13:48:38.116 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-04-25T13:48:38.227 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking linux-image-6.9.0-rc3-g806ec36b2112 (6.9.0-rc3-g806ec36b2112-1) ... 2024-04-25T13:48:42.188 INFO:teuthology.orchestra.run.smithi086.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-04-25T13:48:42.189 INFO:teuthology.orchestra.run.smithi086.stderr: * [new branch] main -> origin/main 2024-04-25T13:48:42.223 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20190312 -> 20190312 2024-04-25T13:48:42.223 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20190416 -> 20190416 2024-04-25T13:48:42.223 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20190514 -> 20190514 2024-04-25T13:48:42.223 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20190618 -> 20190618 2024-04-25T13:48:42.223 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20190717 -> 20190717 2024-04-25T13:48:42.223 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20190815 -> 20190815 2024-04-25T13:48:42.223 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20190923 -> 20190923 2024-04-25T13:48:42.223 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20191022 -> 20191022 2024-04-25T13:48:42.224 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20191215 -> 20191215 2024-04-25T13:48:42.224 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20200122 -> 20200122 2024-04-25T13:48:42.224 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20200316 -> 20200316 2024-04-25T13:48:42.228 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20200421 -> 20200421 2024-04-25T13:48:42.228 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20200519 -> 20200519 2024-04-25T13:48:42.228 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20200619 -> 20200619 2024-04-25T13:48:42.228 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20200721 -> 20200721 2024-04-25T13:48:42.228 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20200817 -> 20200817 2024-04-25T13:48:42.228 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20200918 -> 20200918 2024-04-25T13:48:42.228 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20201022 -> 20201022 2024-04-25T13:48:42.228 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20201118 -> 20201118 2024-04-25T13:48:42.229 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20201218 -> 20201218 2024-04-25T13:48:42.229 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20210208 -> 20210208 2024-04-25T13:48:42.229 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20210315 -> 20210315 2024-04-25T13:48:42.229 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20210511 -> 20210511 2024-04-25T13:48:42.229 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20210716 -> 20210716 2024-04-25T13:48:42.229 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20210818 -> 20210818 2024-04-25T13:48:42.229 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20210919 -> 20210919 2024-04-25T13:48:42.229 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20211027 -> 20211027 2024-04-25T13:48:42.229 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20211216 -> 20211216 2024-04-25T13:48:42.229 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20220209 -> 20220209 2024-04-25T13:48:42.229 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20220310 -> 20220310 2024-04-25T13:48:42.229 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20220411 -> 20220411 2024-04-25T13:48:42.229 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20220509 -> 20220509 2024-04-25T13:48:42.230 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20220610 -> 20220610 2024-04-25T13:48:42.230 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20220708 -> 20220708 2024-04-25T13:48:42.230 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20220815 -> 20220815 2024-04-25T13:48:42.230 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20220913 -> 20220913 2024-04-25T13:48:42.230 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20221012 -> 20221012 2024-04-25T13:48:42.230 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20221109 -> 20221109 2024-04-25T13:48:42.230 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20221214 -> 20221214 2024-04-25T13:48:42.230 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20230117 -> 20230117 2024-04-25T13:48:42.230 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20230210 -> 20230210 2024-04-25T13:48:42.230 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20230310 -> 20230310 2024-04-25T13:48:42.230 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20230404 -> 20230404 2024-04-25T13:48:42.230 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20230515 -> 20230515 2024-04-25T13:48:42.231 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20230625 -> 20230625 2024-04-25T13:48:42.231 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20230804 -> 20230804 2024-04-25T13:48:42.231 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20230919 -> 20230919 2024-04-25T13:48:42.231 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20231030 -> 20231030 2024-04-25T13:48:42.231 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20231111 -> 20231111 2024-04-25T13:48:42.231 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20231211 -> 20231211 2024-04-25T13:48:42.231 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20240115 -> 20240115 2024-04-25T13:48:42.231 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20240220 -> 20240220 2024-04-25T13:48:42.231 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20240312 -> 20240312 2024-04-25T13:48:42.231 INFO:teuthology.orchestra.run.smithi086.stderr: * [new tag] 20240410 -> 20240410 2024-04-25T13:48:42.352 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up linux-image-6.9.0-rc3-g806ec36b2112 (6.9.0-rc3-g806ec36b2112-1) ... 2024-04-25T13:48:42.441 INFO:teuthology.orchestra.run.smithi155.stderr:update-initramfs: Generating /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-25T13:48:46.485 INFO:teuthology.orchestra.run.smithi086.stderr:Updating files: 45% (1527/3381) Updating files: 46% (1556/3381) Updating files: 47% (1590/3381) Updating files: 48% (1623/3381) Updating files: 49% (1657/3381) Updating files: 50% (1691/3381) Updating files: 51% (1725/3381) Updating files: 52% (1759/3381) Updating files: 53% (1792/3381) Updating files: 54% (1826/3381) Updating files: 55% (1860/3381) Updating files: 56% (1894/3381) Updating files: 57% (1928/3381) Updating files: 58% (1961/3381) Updating files: 59% (1995/3381) Updating files: 59% (2011/3381) Updating files: 60% (2029/3381) Updating files: 61% (2063/3381) Updating files: 62% (2097/3381) Updating files: 63% (2131/3381) Updating files: 64% (2164/3381) Updating files: 65% (2198/3381) Updating files: 66% (2232/3381) Updating files: 67% (2266/3381) Updating files: 68% (2300/3381) Updating files: 69% (2333/3381) Updating files: 69% (2362/3381) Updating files: 70% (2367/3381) Updating files: 71% (2401/3381) Updating files: 72% (2435/3381) Updating files: 73% (2469/3381) Updating files: 74% (2502/3381) Updating files: 75% (2536/3381) Updating files: 76% (2570/3381) Updating files: 77% (2604/3381) Updating files: 78% (2638/3381) Updating files: 79% (2671/3381) Updating files: 80% (2705/3381) Updating files: 81% (2739/3381) Updating files: 82% (2773/3381) Updating files: 83% (2807/3381) Updating files: 83% (2813/3381) Updating files: 84% (2841/3381) Updating files: 85% (2874/3381) Updating files: 86% (2908/3381) Updating files: 87% (2942/3381) Updating files: 88% (2976/3381) Updating files: 89% (3010/3381) Updating files: 90% (3043/3381) Updating files: 91% (3077/3381) Updating files: 92% (3111/3381) Updating files: 93% (3145/3381) Updating files: 94% (3179/3381) Updating files: 95% (3212/3381) Updating files: 96% (3246/3381) Updating files: 97% (3280/3381) Updating files: 98% (3314/3381) Updating files: 99% (3348/3381) Updating files: 100% (3381/3381) Updating files: 100% (3381/3381), done. 2024-04-25T13:48:46.490 INFO:teuthology.orchestra.run.smithi086.stdout:HEAD is now at 63a66b66 Merge branch 'robot/pr-0-1713990671' into 'main' 2024-04-25T13:48:46.514 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T13:48:46.514 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T13:48:46.779 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/ 2024-04-25T13:48:46.779 DEBUG:teuthology.orchestra.run.smithi086:> rm -f /tmp/linux-image.deb && echo linux-image-6.9.0-rc3-g806ec36b2112_6.9.0-rc3-g806ec36b2112-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/ --input-file=- 2024-04-25T13:48:46.782 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi086.front.sepia.ceph.com to complete... 2024-04-25T13:48:49.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25 13:48:49 URL:https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/linux-image-6.9.0-rc3-g806ec36b2112_6.9.0-rc3-g806ec36b2112-1_amd64.deb [44805284/44805284] -> "/tmp/linux-image.deb" [1] 2024-04-25T13:48:49.920 INFO:teuthology.orchestra.run.smithi086.stderr:FINISHED --2024-04-25 13:48:49-- 2024-04-25T13:48:49.920 INFO:teuthology.orchestra.run.smithi086.stderr:Total wall clock time: 3.1s 2024-04-25T13:48:49.920 INFO:teuthology.orchestra.run.smithi086.stderr:Downloaded: 1 files, 43M in 2.8s (15.5 MB/s) 2024-04-25T13:48:49.924 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T13:48:49.924 DEBUG:teuthology.orchestra.run.smithi086:> sudo dpkg -i /tmp/linux-image.deb 2024-04-25T13:48:50.197 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package linux-image-6.9.0-rc3-g806ec36b2112. 2024-04-25T13:48:52.217 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-04-25T13:48:52.221 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-04-25T13:48:52.486 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking linux-image-6.9.0-rc3-g806ec36b2112 (6.9.0-rc3-g806ec36b2112-1) ... 2024-04-25T13:48:56.298 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up linux-image-6.9.0-rc3-g806ec36b2112 (6.9.0-rc3-g806ec36b2112-1) ... 2024-04-25T13:48:56.395 INFO:teuthology.orchestra.run.smithi086.stderr:update-initramfs: Generating /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-25T13:49:04.164 INFO:teuthology.orchestra.run.smithi155.stderr:Sourcing file `/etc/default/grub' 2024-04-25T13:49:04.177 INFO:teuthology.orchestra.run.smithi155.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T13:49:04.208 INFO:teuthology.orchestra.run.smithi155.stderr:Generating grub configuration file ... 2024-04-25T13:49:04.554 INFO:teuthology.orchestra.run.smithi155.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T13:49:04.568 INFO:teuthology.orchestra.run.smithi155.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-25T13:49:04.853 INFO:teuthology.orchestra.run.smithi155.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T13:49:04.870 INFO:teuthology.orchestra.run.smithi155.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T13:49:04.938 INFO:teuthology.orchestra.run.smithi155.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T13:49:04.944 INFO:teuthology.orchestra.run.smithi155.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T13:49:05.090 INFO:teuthology.orchestra.run.smithi155.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T13:49:05.090 INFO:teuthology.orchestra.run.smithi155.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T13:49:05.090 INFO:teuthology.orchestra.run.smithi155.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T13:49:05.132 INFO:teuthology.orchestra.run.smithi155.stderr:done 2024-04-25T13:49:05.312 DEBUG:teuthology.orchestra.run.smithi155:> dpkg-deb -c /tmp/linux-image.deb 2024-04-25T13:49:05.320 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./ 2024-04-25T13:49:05.320 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./boot/ 2024-04-25T13:49:05.320 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 7002012 2024-04-19 08:55 ./boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T13:49:05.334 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 172073 2024-04-19 08:55 ./boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T13:49:05.334 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13120512 2024-04-19 08:55 ./boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T13:49:05.354 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/ 2024-04-25T13:49:05.355 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/ 2024-04-25T13:49:05.355 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/postinst.d/ 2024-04-25T13:49:05.355 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/postrm.d/ 2024-04-25T13:49:05.355 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/preinst.d/ 2024-04-25T13:49:05.355 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/prerm.d/ 2024-04-25T13:49:05.355 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/ 2024-04-25T13:49:05.355 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/ 2024-04-25T13:49:05.355 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/ 2024-04-25T13:49:05.355 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/ 2024-04-25T13:49:05.355 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/ 2024-04-25T13:49:05.355 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/ 2024-04-25T13:49:05.355 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ 2024-04-25T13:49:05.356 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 507569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/aesni-intel.ko 2024-04-25T13:49:05.356 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T13:49:05.356 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 34313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T13:49:05.356 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 35009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T13:49:05.356 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 97953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T13:49:05.356 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 109905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T13:49:05.356 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 155249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T13:49:05.356 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T13:49:05.356 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T13:49:05.356 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 87009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T13:49:05.356 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T13:49:05.356 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 50305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T13:49:05.357 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 50377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T13:49:05.357 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 54513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T13:49:05.357 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 44657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T13:49:05.357 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 48049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T13:49:05.357 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 62873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T13:49:05.357 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 53569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T13:49:05.357 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 34761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T13:49:05.357 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T13:49:05.357 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/ 2024-04-25T13:49:05.357 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/ 2024-04-25T13:49:05.357 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/ 2024-04-25T13:49:05.357 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 49081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T13:49:05.357 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpuid.ko 2024-04-25T13:49:05.357 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/msr.ko 2024-04-25T13:49:05.358 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/ 2024-04-25T13:49:05.358 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 444425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T13:49:05.358 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 1035881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T13:49:05.358 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 2849609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T13:49:05.428 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ 2024-04-25T13:49:05.428 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 65025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/af_alg.ko 2024-04-25T13:49:05.429 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 27961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/algif_hash.ko 2024-04-25T13:49:05.429 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/algif_skcipher.ko 2024-04-25T13:49:05.429 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T13:49:05.429 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T13:49:05.429 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T13:49:05.429 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/asymmetric_keys/ 2024-04-25T13:49:05.429 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-04-25T13:49:05.429 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/ 2024-04-25T13:49:05.429 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T13:49:05.429 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T13:49:05.429 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T13:49:05.429 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T13:49:05.429 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T13:49:05.429 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T13:49:05.430 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T13:49:05.430 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 27953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T13:49:05.430 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T13:49:05.430 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T13:49:05.430 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T13:49:05.430 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 40889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T13:49:05.430 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T13:49:05.430 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T13:49:05.430 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T13:49:05.430 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 43281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T13:49:05.430 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T13:49:05.430 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T13:49:05.430 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 59129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cryptd.ko 2024-04-25T13:49:05.431 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 48113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T13:49:05.431 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 32169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_simd.ko 2024-04-25T13:49:05.431 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 30881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T13:49:05.431 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T13:49:05.431 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T13:49:05.431 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T13:49:05.431 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T13:49:05.431 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 30689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T13:49:05.431 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T13:49:05.431 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T13:49:05.431 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lrw.ko 2024-04-25T13:49:05.431 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T13:49:05.431 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T13:49:05.431 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T13:49:05.432 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T13:49:05.432 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T13:49:05.432 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T13:49:05.432 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T13:49:05.432 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 30673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T13:49:05.432 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T13:49:05.432 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 169065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T13:49:05.432 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T13:49:05.432 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T13:49:05.432 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T13:49:05.432 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T13:49:05.432 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 38209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T13:49:05.432 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T13:49:05.432 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 30345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T13:49:05.433 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xts.ko 2024-04-25T13:49:05.433 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T13:49:05.433 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ 2024-04-25T13:49:05.433 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ 2024-04-25T13:49:05.433 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_extlog.ko 2024-04-25T13:49:05.433 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 31921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T13:49:05.433 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 32281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T13:49:05.433 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/apei/ 2024-04-25T13:49:05.433 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 44705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/apei/einj.ko 2024-04-25T13:49:05.433 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T13:49:05.433 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T13:49:05.433 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T13:49:05.433 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T13:49:05.433 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 119033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T13:49:05.434 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ 2024-04-25T13:49:05.434 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T13:49:05.434 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 96825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci.ko 2024-04-25T13:49:05.434 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T13:49:05.434 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 107737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci.ko 2024-04-25T13:49:05.434 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 42001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T13:49:05.434 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T13:49:05.434 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 35705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T13:49:05.434 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 38329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T13:49:05.434 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T13:49:05.434 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T13:49:05.434 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T13:49:05.434 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T13:49:05.435 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T13:49:05.435 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T13:49:05.435 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T13:49:05.435 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T13:49:05.435 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 38913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T13:49:05.435 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 26969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T13:49:05.435 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T13:49:05.435 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T13:49:05.435 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 41289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T13:49:05.435 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T13:49:05.435 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 60257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_legacy.ko 2024-04-25T13:49:05.435 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T13:49:05.435 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T13:49:05.435 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T13:49:05.436 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T13:49:05.436 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T13:49:05.436 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T13:49:05.436 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T13:49:05.436 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T13:49:05.436 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T13:49:05.436 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 35145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T13:49:05.436 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 43009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T13:49:05.436 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T13:49:05.436 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T13:49:05.436 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_radisys.ko 2024-04-25T13:49:05.436 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T13:49:05.436 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rz1000.ko 2024-04-25T13:49:05.436 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T13:49:05.437 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 26793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T13:49:05.437 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 27305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T13:49:05.437 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T13:49:05.437 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T13:49:05.437 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 35025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T13:49:05.437 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 26337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T13:49:05.437 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 34889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T13:49:05.437 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 102121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T13:49:05.437 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 85409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T13:49:05.437 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 44313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T13:49:05.437 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 26673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T13:49:05.437 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 34025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T13:49:05.437 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 45905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T13:49:05.437 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T13:49:05.438 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T13:49:05.438 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 53353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T13:49:05.438 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T13:49:05.438 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 41049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T13:49:05.438 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T13:49:05.438 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ 2024-04-25T13:49:05.438 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T13:49:05.438 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/ 2024-04-25T13:49:05.438 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 163969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T13:49:05.438 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/ 2024-04-25T13:49:05.438 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/ 2024-04-25T13:49:05.438 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 140865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T13:49:05.438 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/ 2024-04-25T13:49:05.438 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 1241657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T13:49:05.439 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 203289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T13:49:05.439 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/ 2024-04-25T13:49:05.439 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 115265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T13:49:05.439 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 143433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T13:49:05.439 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/ 2024-04-25T13:49:05.439 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 126617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T13:49:05.439 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 119873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T13:49:05.439 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 313217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T13:49:05.439 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/ 2024-04-25T13:49:05.439 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 117233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T13:49:05.439 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/ 2024-04-25T13:49:05.439 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 70385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T13:49:05.439 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ 2024-04-25T13:49:05.439 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 35377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/applicom.ko 2024-04-25T13:49:05.440 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T13:49:05.440 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/ 2024-04-25T13:49:05.440 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T13:49:05.440 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T13:49:05.440 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T13:49:05.440 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T13:49:05.440 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T13:49:05.440 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ 2024-04-25T13:49:05.440 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 37257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T13:49:05.440 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 186921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T13:49:05.440 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 32521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T13:49:05.440 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 179561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T13:49:05.440 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 85441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T13:49:05.441 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 65313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T13:49:05.441 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 43857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T13:49:05.441 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/ 2024-04-25T13:49:05.441 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 103409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T13:49:05.441 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 31833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/nvram.ko 2024-04-25T13:49:05.441 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 38977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T13:49:05.441 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 59241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T13:49:05.441 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/ 2024-04-25T13:49:05.441 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T13:49:05.441 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-04-25T13:49:05.441 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-04-25T13:49:05.441 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 39193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-04-25T13:49:05.441 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 34169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T13:49:05.441 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T13:49:05.442 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-04-25T13:49:05.442 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/ 2024-04-25T13:49:05.442 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_class.ko 2024-04-25T13:49:05.442 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 62545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_core.ko 2024-04-25T13:49:05.442 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-04-25T13:49:05.442 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/ 2024-04-25T13:49:05.442 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-palmas.ko 2024-04-25T13:49:05.442 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 57289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-si5351.ko 2024-04-25T13:49:05.442 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-twl6040.ko 2024-04-25T13:49:05.442 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-wm831x.ko 2024-04-25T13:49:05.442 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/ 2024-04-25T13:49:05.442 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T13:49:05.442 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T13:49:05.443 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T13:49:05.443 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ 2024-04-25T13:49:05.443 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ 2024-04-25T13:49:05.443 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 89257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T13:49:05.443 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 333657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T13:49:05.443 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/ 2024-04-25T13:49:05.443 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/ 2024-04-25T13:49:05.443 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/ 2024-04-25T13:49:05.443 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 751713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T13:49:05.443 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-04-25T13:49:05.443 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T13:49:05.443 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T13:49:05.443 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 26721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T13:49:05.443 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/ 2024-04-25T13:49:05.444 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 90441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T13:49:05.444 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/ 2024-04-25T13:49:05.444 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 36105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T13:49:05.444 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ 2024-04-25T13:49:05.444 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/ 2024-04-25T13:49:05.444 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T13:49:05.444 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ 2024-04-25T13:49:05.444 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 146769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T13:49:05.444 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ 2024-04-25T13:49:05.444 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 96145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T13:49:05.444 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 41993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T13:49:05.444 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 51105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T13:49:05.444 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T13:49:05.444 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T13:49:05.445 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 26729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T13:49:05.445 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T13:49:05.445 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T13:49:05.445 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 64161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T13:49:05.445 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T13:49:05.445 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T13:49:05.445 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 68697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T13:49:05.445 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T13:49:05.445 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/ 2024-04-25T13:49:05.445 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-gpio.ko 2024-04-25T13:49:05.445 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 31369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max14577.ko 2024-04-25T13:49:05.445 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 35601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max77693.ko 2024-04-25T13:49:05.445 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max8997.ko 2024-04-25T13:49:05.445 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 30961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-palmas.ko 2024-04-25T13:49:05.446 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 27201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-rt8973a.ko 2024-04-25T13:49:05.446 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 31993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-sm5502.ko 2024-04-25T13:49:05.446 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/ 2024-04-25T13:49:05.446 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 42953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/dmi-sysfs.ko 2024-04-25T13:49:05.446 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/efi/ 2024-04-25T13:49:05.446 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/efi/efi-pstore.ko 2024-04-25T13:49:05.446 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 26361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T13:49:05.446 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/ 2024-04-25T13:49:05.446 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-adp5520.ko 2024-04-25T13:49:05.446 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-amd8111.ko 2024-04-25T13:49:05.446 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-arizona.ko 2024-04-25T13:49:05.446 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-da9052.ko 2024-04-25T13:49:05.446 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-da9055.ko 2024-04-25T13:49:05.446 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 26945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-dln2.ko 2024-04-25T13:49:05.447 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 81073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-f7188x.ko 2024-04-25T13:49:05.447 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 40761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-generic.ko 2024-04-25T13:49:05.447 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T13:49:05.447 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-04-25T13:49:05.447 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-kempld.ko 2024-04-25T13:49:05.447 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-lp3943.ko 2024-04-25T13:49:05.447 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max7300.ko 2024-04-25T13:49:05.447 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max730x.ko 2024-04-25T13:49:05.447 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max732x.ko 2024-04-25T13:49:05.447 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-04-25T13:49:05.447 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 55505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-pca953x.ko 2024-04-25T13:49:05.447 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 31881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-pcf857x.ko 2024-04-25T13:49:05.447 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-rdc321x.ko 2024-04-25T13:49:05.447 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-sch.ko 2024-04-25T13:49:05.448 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-sch311x.ko 2024-04-25T13:49:05.448 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-tps65912.ko 2024-04-25T13:49:05.448 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-twl4030.ko 2024-04-25T13:49:05.448 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-twl6040.ko 2024-04-25T13:49:05.448 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T13:49:05.448 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-vx855.ko 2024-04-25T13:49:05.448 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm831x.ko 2024-04-25T13:49:05.448 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm8350.ko 2024-04-25T13:49:05.448 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm8994.ko 2024-04-25T13:49:05.448 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/ 2024-04-25T13:49:05.448 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-a4tech.ko 2024-04-25T13:49:05.448 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 43553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-apple.ko 2024-04-25T13:49:05.448 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T13:49:05.448 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T13:49:05.449 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T13:49:05.449 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-belkin.ko 2024-04-25T13:49:05.449 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cherry.ko 2024-04-25T13:49:05.449 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-chicony.ko 2024-04-25T13:49:05.449 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 74393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cp2112.ko 2024-04-25T13:49:05.449 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cypress.ko 2024-04-25T13:49:05.449 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T13:49:05.449 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T13:49:05.449 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T13:49:05.449 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T13:49:05.449 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ezkey.ko 2024-04-25T13:49:05.449 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T13:49:05.449 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-generic.ko 2024-04-25T13:49:05.450 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T13:49:05.450 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T13:49:05.450 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T13:49:05.450 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T13:49:05.450 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T13:49:05.450 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T13:49:05.450 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T13:49:05.450 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kensington.ko 2024-04-25T13:49:05.450 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T13:49:05.450 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T13:49:05.450 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T13:49:05.450 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T13:49:05.450 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 54217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T13:49:05.450 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 37553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lg-g15.ko 2024-04-25T13:49:05.451 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 68081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T13:49:05.451 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 121369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T13:49:05.451 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 97953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech.ko 2024-04-25T13:49:05.451 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 36689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-magicmouse.ko 2024-04-25T13:49:05.451 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-microsoft.ko 2024-04-25T13:49:05.451 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-monterey.ko 2024-04-25T13:49:05.451 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 59001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T13:49:05.451 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 39697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ntrig.ko 2024-04-25T13:49:05.451 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T13:49:05.451 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T13:49:05.451 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T13:49:05.451 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 91241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T13:49:05.451 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T13:49:05.451 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T13:49:05.452 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T13:49:05.452 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 42889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T13:49:05.452 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T13:49:05.452 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T13:49:05.452 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 38865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T13:49:05.452 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 36305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T13:49:05.452 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 38073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T13:49:05.452 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 26097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T13:49:05.452 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 35625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T13:49:05.452 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T13:49:05.452 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 35425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T13:49:05.452 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 35561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T13:49:05.452 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T13:49:05.452 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T13:49:05.453 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T13:49:05.453 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 42305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T13:49:05.453 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 43609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T13:49:05.453 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T13:49:05.453 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 62129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T13:49:05.453 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T13:49:05.453 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 30017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T13:49:05.453 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T13:49:05.453 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T13:49:05.453 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T13:49:05.453 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T13:49:05.453 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T13:49:05.453 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T13:49:05.453 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 81113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T13:49:05.454 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T13:49:05.454 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 184745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T13:49:05.454 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T13:49:05.454 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T13:49:05.454 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T13:49:05.454 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 319921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid.ko 2024-04-25T13:49:05.454 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 39529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T13:49:05.454 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/ 2024-04-25T13:49:05.454 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 153633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbhid.ko 2024-04-25T13:49:05.454 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbkbd.ko 2024-04-25T13:49:05.454 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbmouse.ko 2024-04-25T13:49:05.454 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 261945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T13:49:05.454 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/ 2024-04-25T13:49:05.454 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 75065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T13:49:05.455 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 110681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T13:49:05.455 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 382033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T13:49:05.455 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ 2024-04-25T13:49:05.455 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 77153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T13:49:05.455 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 67009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T13:49:05.455 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 42249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T13:49:05.455 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T13:49:05.455 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T13:49:05.455 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T13:49:05.455 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 38097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T13:49:05.455 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 115057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T13:49:05.455 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T13:49:05.455 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 55225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T13:49:05.456 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 31441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T13:49:05.456 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T13:49:05.456 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T13:49:05.456 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 26505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T13:49:05.456 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 89025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T13:49:05.456 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 49153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T13:49:05.456 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 81473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T13:49:05.456 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T13:49:05.456 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 48177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T13:49:05.456 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 68865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T13:49:05.456 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 60193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T13:49:05.456 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 69009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T13:49:05.456 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 47977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T13:49:05.456 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T13:49:05.457 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 33641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T13:49:05.457 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/da9052-hwmon.ko 2024-04-25T13:49:05.457 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/da9055-hwmon.ko 2024-04-25T13:49:05.457 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 55713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T13:49:05.457 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 113201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T13:49:05.457 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T13:49:05.457 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T13:49:05.457 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 33665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T13:49:05.457 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 40177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc2103.ko 2024-04-25T13:49:05.457 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 34649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T13:49:05.457 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 78921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T13:49:05.457 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 108961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T13:49:05.457 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 50697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T13:49:05.457 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T13:49:05.457 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 65769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T13:49:05.458 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T13:49:05.458 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 30585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T13:49:05.458 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 59761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T13:49:05.458 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 56313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T13:49:05.458 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hih6130.ko 2024-04-25T13:49:05.458 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T13:49:05.458 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T13:49:05.458 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 27633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T13:49:05.458 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 42585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T13:49:05.458 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T13:49:05.458 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T13:49:05.458 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T13:49:05.458 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 156465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T13:49:05.459 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/jc42.ko 2024-04-25T13:49:05.459 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T13:49:05.459 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T13:49:05.459 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T13:49:05.459 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 57769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T13:49:05.459 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T13:49:05.459 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 44121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T13:49:05.459 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T13:49:05.459 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 41473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T13:49:05.459 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 37049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T13:49:05.459 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T13:49:05.459 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 86497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T13:49:05.459 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 55641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T13:49:05.459 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 42249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T13:49:05.460 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T13:49:05.460 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 131505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T13:49:05.460 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 40225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T13:49:05.460 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T13:49:05.460 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T13:49:05.460 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 26393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T13:49:05.460 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T13:49:05.460 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T13:49:05.460 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T13:49:05.460 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T13:49:05.460 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T13:49:05.460 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T13:49:05.460 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 36769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T13:49:05.460 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T13:49:05.460 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T13:49:05.461 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T13:49:05.461 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 32505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T13:49:05.461 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 27305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T13:49:05.461 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 39729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T13:49:05.461 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mc13783-adc.ko 2024-04-25T13:49:05.461 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T13:49:05.461 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-04-25T13:49:05.461 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 56505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T13:49:05.461 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 141353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T13:49:05.461 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 72737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T13:49:05.461 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 67841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T13:49:05.461 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 97777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T13:49:05.461 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 61921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T13:49:05.461 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T13:49:05.462 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ 2024-04-25T13:49:05.462 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T13:49:05.462 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T13:49:05.462 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 33265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T13:49:05.462 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T13:49:05.462 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T13:49:05.462 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T13:49:05.462 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T13:49:05.462 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 101233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T13:49:05.462 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T13:49:05.462 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 36177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T13:49:05.462 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T13:49:05.462 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T13:49:05.462 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T13:49:05.463 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T13:49:05.463 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 32945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T13:49:05.463 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 34561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T13:49:05.463 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 40849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T13:49:05.463 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T13:49:05.463 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T13:49:05.463 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 48713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T13:49:05.463 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T13:49:05.463 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 46473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T13:49:05.463 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 44313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T13:49:05.463 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 31417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T13:49:05.463 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T13:49:05.463 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T13:49:05.463 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T13:49:05.464 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T13:49:05.464 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T13:49:05.464 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 50217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T13:49:05.464 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 59665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T13:49:05.464 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 62737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T13:49:05.464 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 71169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T13:49:05.464 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 96577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T13:49:05.464 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 85913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T13:49:05.464 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 75129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T13:49:05.464 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 83425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T13:49:05.464 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 116777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T13:49:05.464 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 134441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T13:49:05.464 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T13:49:05.464 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 43385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T13:49:05.465 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-04-25T13:49:05.465 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-04-25T13:49:05.465 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/ 2024-04-25T13:49:05.465 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/ 2024-04-25T13:49:05.465 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 27353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T13:49:05.465 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T13:49:05.465 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/ 2024-04-25T13:49:05.465 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 33345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-04-25T13:49:05.465 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 35073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-04-25T13:49:05.465 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 32905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-04-25T13:49:05.465 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T13:49:05.465 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 30361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T13:49:05.465 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T13:49:05.465 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-04-25T13:49:05.466 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 7729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T13:49:05.466 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-04-25T13:49:05.466 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 64233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T13:49:05.466 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T13:49:05.466 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T13:49:05.466 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T13:49:05.466 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-04-25T13:49:05.466 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-04-25T13:49:05.466 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 76969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T13:49:05.466 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T13:49:05.466 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 52937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T13:49:05.466 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-04-25T13:49:05.466 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T13:49:05.466 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T13:49:05.467 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 33209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-04-25T13:49:05.467 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 30049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T13:49:05.467 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T13:49:05.467 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 57217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T13:49:05.467 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-04-25T13:49:05.467 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T13:49:05.467 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T13:49:05.467 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-04-25T13:49:05.467 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 32585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-04-25T13:49:05.467 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T13:49:05.467 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-04-25T13:49:05.467 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T13:49:05.467 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T13:49:05.467 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T13:49:05.468 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T13:49:05.468 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 59193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-04-25T13:49:05.468 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 27849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T13:49:05.468 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T13:49:05.468 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T13:49:05.468 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/ 2024-04-25T13:49:05.468 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-04-25T13:49:05.468 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-04-25T13:49:05.468 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 36569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-04-25T13:49:05.468 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ 2024-04-25T13:49:05.468 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ 2024-04-25T13:49:05.468 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 329697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T13:49:05.468 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 1119913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T13:49:05.468 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 83905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T13:49:05.469 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 408433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T13:49:05.469 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 120921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T13:49:05.469 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 347217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T13:49:05.469 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 84945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T13:49:05.469 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ 2024-04-25T13:49:05.469 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/ 2024-04-25T13:49:05.469 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 590929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T13:49:05.469 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/ 2024-04-25T13:49:05.469 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 514641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T13:49:05.469 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/ 2024-04-25T13:49:05.469 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 911665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T13:49:05.469 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ 2024-04-25T13:49:05.469 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 375545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T13:49:05.469 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ocrdma/ 2024-04-25T13:49:05.470 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 197121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-04-25T13:49:05.470 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/usnic/ 2024-04-25T13:49:05.470 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 212553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-04-25T13:49:05.470 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ 2024-04-25T13:49:05.470 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ 2024-04-25T13:49:05.470 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 334401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T13:49:05.470 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ 2024-04-25T13:49:05.470 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 120673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T13:49:05.470 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ 2024-04-25T13:49:05.470 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 132585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T13:49:05.470 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ 2024-04-25T13:49:05.470 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 157937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T13:49:05.470 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ 2024-04-25T13:49:05.470 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 156673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T13:49:05.471 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/ 2024-04-25T13:49:05.471 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/evbug.ko 2024-04-25T13:49:05.471 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 27609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/ff-memless.ko 2024-04-25T13:49:05.471 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/ 2024-04-25T13:49:05.471 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T13:49:05.471 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T13:49:05.471 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 50713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T13:49:05.471 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 49441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T13:49:05.471 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/ 2024-04-25T13:49:05.471 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-04-25T13:49:05.471 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-04-25T13:49:05.471 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 35665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-04-25T13:49:05.471 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-04-25T13:49:05.471 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 45473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T13:49:05.472 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-04-25T13:49:05.472 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lkkbd.ko 2024-04-25T13:49:05.472 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 31857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lm8323.ko 2024-04-25T13:49:05.472 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lm8333.ko 2024-04-25T13:49:05.472 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-04-25T13:49:05.472 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-04-25T13:49:05.472 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-04-25T13:49:05.472 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-04-25T13:49:05.472 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/newtonkbd.ko 2024-04-25T13:49:05.472 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-04-25T13:49:05.472 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/qt1070.ko 2024-04-25T13:49:05.472 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/qt2160.ko 2024-04-25T13:49:05.472 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-04-25T13:49:05.472 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/stowaway.ko 2024-04-25T13:49:05.473 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/sunkbd.ko 2024-04-25T13:49:05.473 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-04-25T13:49:05.473 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-04-25T13:49:05.473 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-04-25T13:49:05.473 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/xtkbd.ko 2024-04-25T13:49:05.473 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/matrix-keymap.ko 2024-04-25T13:49:05.473 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/ 2024-04-25T13:49:05.473 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 40617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T13:49:05.473 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 39553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T13:49:05.473 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 160017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T13:49:05.473 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 114273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T13:49:05.473 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/gpio_mouse.ko 2024-04-25T13:49:05.473 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 428185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/psmouse.ko 2024-04-25T13:49:05.473 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T13:49:05.474 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 27865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T13:49:05.474 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 27193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T13:49:05.474 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T13:49:05.474 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/ 2024-04-25T13:49:05.474 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 140969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T13:49:05.474 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ 2024-04-25T13:49:05.474 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T13:49:05.474 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T13:49:05.474 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ct82c710.ko 2024-04-25T13:49:05.474 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T13:49:05.474 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/parkbd.ko 2024-04-25T13:49:05.474 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/pcips2.ko 2024-04-25T13:49:05.474 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ps2mult.ko 2024-04-25T13:49:05.474 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T13:49:05.475 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serport.ko 2024-04-25T13:49:05.475 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T13:49:05.475 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/ 2024-04-25T13:49:05.475 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-88pm860x.ko 2024-04-25T13:49:05.475 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-adp5520.ko 2024-04-25T13:49:05.475 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 77265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-bd2802.ko 2024-04-25T13:49:05.475 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 31097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T13:49:05.475 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-da903x.ko 2024-04-25T13:49:05.475 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-da9052.ko 2024-04-25T13:49:05.475 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-gpio.ko 2024-04-25T13:49:05.475 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T13:49:05.475 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 36705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3533.ko 2024-04-25T13:49:05.475 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm355x.ko 2024-04-25T13:49:05.475 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3642.ko 2024-04-25T13:49:05.476 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T13:49:05.476 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp8788.ko 2024-04-25T13:49:05.476 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-max8997.ko 2024-04-25T13:49:05.476 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-mc13783.ko 2024-04-25T13:49:05.476 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-menf21bmc.ko 2024-04-25T13:49:05.476 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca9532.ko 2024-04-25T13:49:05.476 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca955x.ko 2024-04-25T13:49:05.476 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca963x.ko 2024-04-25T13:49:05.476 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 38113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T13:49:05.476 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-tca6507.ko 2024-04-25T13:49:05.476 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-wm831x-status.ko 2024-04-25T13:49:05.476 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-wm8350.ko 2024-04-25T13:49:05.476 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ 2024-04-25T13:49:05.476 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T13:49:05.476 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T13:49:05.477 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T13:49:05.477 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 8641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T13:49:05.477 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T13:49:05.477 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T13:49:05.477 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T13:49:05.477 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T13:49:05.477 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T13:49:05.477 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/ 2024-04-25T13:49:05.477 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/ 2024-04-25T13:49:05.477 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 790753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T13:49:05.477 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 49081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T13:49:05.477 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 116825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bufio.ko 2024-04-25T13:49:05.477 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 58281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T13:49:05.477 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 199041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T13:49:05.478 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 134945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T13:49:05.478 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T13:49:05.478 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 67497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-era.ko 2024-04-25T13:49:05.478 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 33017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T13:49:05.478 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 50457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T13:49:05.478 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 41145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log.ko 2024-04-25T13:49:05.478 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 60977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-mirror.ko 2024-04-25T13:49:05.478 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 97209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T13:49:05.478 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T13:49:05.478 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 85689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T13:49:05.478 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 44289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-region-hash.ko 2024-04-25T13:49:05.478 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T13:49:05.478 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T13:49:05.479 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 139817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-snapshot.ko 2024-04-25T13:49:05.479 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T13:49:05.479 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 234457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T13:49:05.479 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 66425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T13:49:05.479 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-zero.ko 2024-04-25T13:49:05.479 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/ 2024-04-25T13:49:05.479 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 264329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T13:49:05.479 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 43137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T13:49:05.479 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 125009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T13:49:05.479 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 157681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T13:49:05.479 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 444665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T13:49:05.479 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/ 2024-04-25T13:49:05.479 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/ 2024-04-25T13:49:05.479 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 214841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T13:49:05.479 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 77985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T13:49:05.480 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 54785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T13:49:05.480 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 44633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T13:49:05.480 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 150761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T13:49:05.480 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 97281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T13:49:05.480 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 63873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T13:49:05.480 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/ 2024-04-25T13:49:05.480 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm800.ko 2024-04-25T13:49:05.480 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm805.ko 2024-04-25T13:49:05.480 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm80x.ko 2024-04-25T13:49:05.480 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/arizona-i2c.ko 2024-04-25T13:49:05.480 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 148633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/arizona.ko 2024-04-25T13:49:05.480 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/bcm590xx.ko 2024-04-25T13:49:05.480 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/cros_ec_dev.ko 2024-04-25T13:49:05.480 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 41009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/dln2.ko 2024-04-25T13:49:05.481 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/janz-cmodio.ko 2024-04-25T13:49:05.481 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 59089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/kempld-core.ko 2024-04-25T13:49:05.481 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 31897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lm3533-core.ko 2024-04-25T13:49:05.481 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-04-25T13:49:05.481 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lp3943.ko 2024-04-25T13:49:05.481 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 60497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T13:49:05.481 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T13:49:05.481 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/max8907.ko 2024-04-25T13:49:05.481 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 35537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mc13xxx-core.ko 2024-04-25T13:49:05.481 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-04-25T13:49:05.481 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/menf21bmc.ko 2024-04-25T13:49:05.481 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633-adc.ko 2024-04-25T13:49:05.481 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633-gpio.ko 2024-04-25T13:49:05.481 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 42297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633.ko 2024-04-25T13:49:05.482 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-04-25T13:49:05.482 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/retu-mfd.ko 2024-04-25T13:49:05.482 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 99425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/si476x-core.ko 2024-04-25T13:49:05.482 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 58057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T13:49:05.482 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 60681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/tps65010.ko 2024-04-25T13:49:05.482 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/tps6507x.ko 2024-04-25T13:49:05.482 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T13:49:05.482 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T13:49:05.482 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T13:49:05.482 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 52361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wm8994.ko 2024-04-25T13:49:05.482 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ 2024-04-25T13:49:05.482 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-04-25T13:49:05.482 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 64265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ad525x_dpot.ko 2024-04-25T13:49:05.482 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/ 2024-04-25T13:49:05.483 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 57521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T13:49:05.483 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T13:49:05.483 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 47689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T13:49:05.483 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 58633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T13:49:05.483 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/ 2024-04-25T13:49:05.483 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-04-25T13:49:05.483 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 47449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/core.ko 2024-04-25T13:49:05.483 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/ 2024-04-25T13:49:05.483 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T13:49:05.483 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ds1682.ko 2024-04-25T13:49:05.483 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/dummy-irq.ko 2024-04-25T13:49:05.483 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/echo/ 2024-04-25T13:49:05.483 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/echo/echo.ko 2024-04-25T13:49:05.483 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/ 2024-04-25T13:49:05.483 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 43041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T13:49:05.484 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T13:49:05.484 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T13:49:05.484 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 41185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T13:49:05.484 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/genwqe/ 2024-04-25T13:49:05.484 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 191649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-04-25T13:49:05.484 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hmc6352.ko 2024-04-25T13:49:05.484 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 37761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T13:49:05.484 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ibmasm/ 2024-04-25T13:49:05.484 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 92345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-04-25T13:49:05.484 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ics932s401.ko 2024-04-25T13:49:05.484 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 26737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T13:49:05.484 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T13:49:05.484 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/ 2024-04-25T13:49:05.484 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 54865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T13:49:05.485 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T13:49:05.485 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/ 2024-04-25T13:49:05.485 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 154169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T13:49:05.485 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 69809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T13:49:05.485 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 389033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T13:49:05.485 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 37961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/phantom.ko 2024-04-25T13:49:05.485 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ti-st/ 2024-04-25T13:49:05.485 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 91145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ti-st/st_drv.ko 2024-04-25T13:49:05.485 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T13:49:05.485 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 34457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T13:49:05.485 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T13:49:05.485 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 55793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T13:49:05.485 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/ 2024-04-25T13:49:05.485 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 234905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T13:49:05.486 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ 2024-04-25T13:49:05.486 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/ 2024-04-25T13:49:05.486 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 620961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T13:49:05.486 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/ 2024-04-25T13:49:05.486 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/ 2024-04-25T13:49:05.486 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 139817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T13:49:05.486 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 49545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T13:49:05.486 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T13:49:05.486 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/ 2024-04-25T13:49:05.486 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 587065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T13:49:05.486 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T13:49:05.486 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T13:49:05.486 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ 2024-04-25T13:49:05.486 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/ 2024-04-25T13:49:05.487 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 53753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T13:49:05.487 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 49201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T13:49:05.487 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 158649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T13:49:05.553 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 85953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T13:49:05.553 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ 2024-04-25T13:49:05.553 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 44161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T13:49:05.553 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 73865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T13:49:05.553 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 33313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T13:49:05.553 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 109521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T13:49:05.553 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/ 2024-04-25T13:49:05.553 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 64705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T13:49:05.553 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/ 2024-04-25T13:49:05.553 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 84329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T13:49:05.554 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/ 2024-04-25T13:49:05.554 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 70737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T13:49:05.554 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/ 2024-04-25T13:49:05.554 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 52705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T13:49:05.554 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/ 2024-04-25T13:49:05.554 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 65857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T13:49:05.554 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 56057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T13:49:05.554 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 115881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T13:49:05.554 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/ 2024-04-25T13:49:05.554 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/ 2024-04-25T13:49:05.554 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 124497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T13:49:05.554 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-04-25T13:49:05.554 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 128409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T13:49:05.555 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-04-25T13:49:05.555 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 110345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T13:49:05.555 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/ 2024-04-25T13:49:05.555 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 101857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T13:49:05.555 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 88953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T13:49:05.555 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/ 2024-04-25T13:49:05.555 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 105633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T13:49:05.555 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 228985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T13:49:05.555 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-04-25T13:49:05.555 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 1733353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T13:49:05.555 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 167185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T13:49:05.555 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/ 2024-04-25T13:49:05.555 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 161417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T13:49:05.555 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 440625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T13:49:05.556 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/ 2024-04-25T13:49:05.556 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/ 2024-04-25T13:49:05.556 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 432201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T13:49:05.556 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/ 2024-04-25T13:49:05.556 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-04-25T13:49:05.556 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 257201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T13:49:05.556 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-04-25T13:49:05.556 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 446177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T13:49:05.556 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-04-25T13:49:05.556 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 1311433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T13:49:05.556 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-04-25T13:49:05.556 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 206689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T13:49:05.556 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-04-25T13:49:05.556 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 40161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T13:49:05.557 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/ 2024-04-25T13:49:05.557 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/ 2024-04-25T13:49:05.557 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 269313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T13:49:05.557 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/ 2024-04-25T13:49:05.557 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/ 2024-04-25T13:49:05.557 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 85385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T13:49:05.557 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 77457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T13:49:05.557 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 169089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T13:49:05.557 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 65913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T13:49:05.557 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 76209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T13:49:05.557 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 38065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T13:49:05.557 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/ 2024-04-25T13:49:05.557 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 75945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T13:49:05.557 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 78705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T13:49:05.558 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 52313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T13:49:05.558 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 30409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ec_bhf.ko 2024-04-25T13:49:05.558 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/ 2024-04-25T13:49:05.558 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/ 2024-04-25T13:49:05.558 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 356377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T13:49:05.558 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 61737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T13:49:05.558 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 63185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T13:49:05.558 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fujitsu/ 2024-04-25T13:49:05.558 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 51513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-04-25T13:49:05.558 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ 2024-04-25T13:49:05.558 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 115569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T13:49:05.558 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/ 2024-04-25T13:49:05.559 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 366217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T13:49:05.559 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/ 2024-04-25T13:49:05.559 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 755793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T13:49:05.559 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/ 2024-04-25T13:49:05.559 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 324577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T13:49:05.559 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/ 2024-04-25T13:49:05.559 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 1196297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T13:49:05.559 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/ 2024-04-25T13:49:05.559 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 362593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T13:49:05.559 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/ 2024-04-25T13:49:05.559 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 671801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T13:49:05.559 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/ 2024-04-25T13:49:05.559 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 121537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T13:49:05.559 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-04-25T13:49:05.560 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 966705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T13:49:05.560 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-04-25T13:49:05.560 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 193585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T13:49:05.560 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 108633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T13:49:05.560 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/ 2024-04-25T13:49:05.560 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T13:49:05.560 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 126881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T13:49:05.560 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 150777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T13:49:05.560 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/ 2024-04-25T13:49:05.560 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-04-25T13:49:05.560 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 905033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T13:49:05.560 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 344145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T13:49:05.560 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-04-25T13:49:05.560 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-04-25T13:49:05.560 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 1412417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T13:49:05.561 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ 2024-04-25T13:49:05.561 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 65913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-04-25T13:49:05.561 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 54497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-04-25T13:49:05.561 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-04-25T13:49:05.561 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 117353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T13:49:05.561 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/ 2024-04-25T13:49:05.561 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-04-25T13:49:05.561 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 131425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T13:49:05.561 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ 2024-04-25T13:49:05.561 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 91465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T13:49:05.561 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 56689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T13:49:05.561 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/ 2024-04-25T13:49:05.561 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 181081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T13:49:05.561 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/ 2024-04-25T13:49:05.562 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 171417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T13:49:05.562 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/ 2024-04-25T13:49:05.562 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 58249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T13:49:05.562 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 50121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T13:49:05.562 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/ 2024-04-25T13:49:05.562 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-04-25T13:49:05.562 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 300817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T13:49:05.562 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 105689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T13:49:05.562 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-04-25T13:49:05.562 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 885129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T13:49:05.562 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/ 2024-04-25T13:49:05.562 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 52305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T13:49:05.562 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/ 2024-04-25T13:49:05.563 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 81641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T13:49:05.563 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 100689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T13:49:05.563 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 34073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T13:49:05.563 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 264161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T13:49:05.563 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/ 2024-04-25T13:49:05.563 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-04-25T13:49:05.563 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 142449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-04-25T13:49:05.563 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/ 2024-04-25T13:49:05.563 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 1350385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T13:49:05.563 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/ 2024-04-25T13:49:05.563 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 57953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T13:49:05.563 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/ 2024-04-25T13:49:05.563 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 66505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T13:49:05.563 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 80065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T13:49:05.564 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/ 2024-04-25T13:49:05.564 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 74289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T13:49:05.564 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 79457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T13:49:05.564 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 100785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T13:49:05.564 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 83441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T13:49:05.564 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/ 2024-04-25T13:49:05.564 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-04-25T13:49:05.564 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-04-25T13:49:05.564 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 46985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T13:49:05.564 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-04-25T13:49:05.564 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 644537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T13:49:05.564 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/ 2024-04-25T13:49:05.564 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 128497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T13:49:05.564 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 207993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T13:49:05.565 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 82633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T13:49:05.565 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 91353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T13:49:05.565 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/ 2024-04-25T13:49:05.565 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 59865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T13:49:05.565 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/ 2024-04-25T13:49:05.565 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 110729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T13:49:05.565 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/ 2024-04-25T13:49:05.565 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 98689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T13:49:05.565 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 102513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T13:49:05.565 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/ 2024-04-25T13:49:05.565 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 61497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T13:49:05.565 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 40609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T13:49:05.565 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/ 2024-04-25T13:49:05.565 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 69913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T13:49:05.565 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/ 2024-04-25T13:49:05.566 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 49449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/defxx.ko 2024-04-25T13:49:05.566 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/skfp/ 2024-04-25T13:49:05.566 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 275801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/skfp/skfp.ko 2024-04-25T13:49:05.566 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 91105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T13:49:05.566 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/ 2024-04-25T13:49:05.566 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 290441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T13:49:05.566 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/ 2024-04-25T13:49:05.566 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T13:49:05.566 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 30001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T13:49:05.566 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ 2024-04-25T13:49:05.566 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 108329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T13:49:05.566 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 78065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T13:49:05.566 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T13:49:05.567 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/ 2024-04-25T13:49:05.567 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 26105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T13:49:05.567 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T13:49:05.567 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-gpio.ko 2024-04-25T13:49:05.567 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T13:49:05.567 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T13:49:05.567 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 26417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T13:49:05.567 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 62681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T13:49:05.567 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T13:49:05.567 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/ 2024-04-25T13:49:05.567 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T13:49:05.567 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 48313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T13:49:05.567 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ 2024-04-25T13:49:05.567 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T13:49:05.568 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 61985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T13:49:05.568 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 52865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T13:49:05.568 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T13:49:05.568 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 53409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T13:49:05.568 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T13:49:05.568 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T13:49:05.568 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T13:49:05.568 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 31313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T13:49:05.568 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T13:49:05.568 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 103377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T13:49:05.568 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 96025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T13:49:05.568 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T13:49:05.568 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 121105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T13:49:05.568 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T13:49:05.569 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 62113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T13:49:05.569 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 35513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T13:49:05.569 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T13:49:05.569 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T13:49:05.569 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/plip/ 2024-04-25T13:49:05.569 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 55353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/plip/plip.ko 2024-04-25T13:49:05.569 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ 2024-04-25T13:49:05.569 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T13:49:05.569 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 36273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T13:49:05.569 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T13:49:05.569 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T13:49:05.569 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 35369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T13:49:05.569 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 53233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T13:49:05.569 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T13:49:05.570 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 40081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T13:49:05.570 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 35937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sb1000.ko 2024-04-25T13:49:05.570 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/ 2024-04-25T13:49:05.570 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 52073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T13:49:05.570 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 42409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T13:49:05.570 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 80161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T13:49:05.570 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/ 2024-04-25T13:49:05.570 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 137321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T13:49:05.570 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T13:49:05.570 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T13:49:05.570 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 36209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T13:49:05.570 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T13:49:05.570 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T13:49:05.570 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 108089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T13:49:05.571 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/ 2024-04-25T13:49:05.571 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 176497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T13:49:05.571 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/ 2024-04-25T13:49:05.571 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 383593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T13:49:05.571 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/ 2024-04-25T13:49:05.571 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 175537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T13:49:05.571 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ 2024-04-25T13:49:05.571 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T13:49:05.571 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/ 2024-04-25T13:49:05.571 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/ 2024-04-25T13:49:05.571 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 335857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T13:49:05.571 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 126553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T13:49:05.571 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/ 2024-04-25T13:49:05.571 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 168145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T13:49:05.572 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T13:49:05.573 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 103889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T13:49:05.573 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 30313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T13:49:05.573 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/ 2024-04-25T13:49:05.573 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/ 2024-04-25T13:49:05.573 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T13:49:05.574 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-04-25T13:49:05.574 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-04-25T13:49:05.574 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pci-stub.ko 2024-04-25T13:49:05.574 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 55329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T13:49:05.574 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/ 2024-04-25T13:49:05.574 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 31529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T13:49:05.574 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 179961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia.ko 2024-04-25T13:49:05.574 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 78161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia_core.ko 2024-04-25T13:49:05.574 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 41945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-04-25T13:49:05.574 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 40457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T13:49:05.574 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 133753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T13:49:05.574 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/ 2024-04-25T13:49:05.574 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/broadcom/ 2024-04-25T13:49:05.575 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-04-25T13:49:05.575 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/samsung/ 2024-04-25T13:49:05.575 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-04-25T13:49:05.575 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/ 2024-04-25T13:49:05.575 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/ 2024-04-25T13:49:05.575 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 78193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T13:49:05.575 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/ 2024-04-25T13:49:05.575 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/ 2024-04-25T13:49:05.575 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 61041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-04-25T13:49:05.575 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec.ko 2024-04-25T13:49:05.575 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-04-25T13:49:05.575 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 30217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-04-25T13:49:05.575 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 31801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-04-25T13:49:05.575 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-04-25T13:49:05.576 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-04-25T13:49:05.576 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/ 2024-04-25T13:49:05.576 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 87481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T13:49:05.576 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 44105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T13:49:05.576 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 48257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T13:49:05.576 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 73865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T13:49:05.576 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 33321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T13:49:05.576 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 164769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T13:49:05.576 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 46185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T13:49:05.576 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 62073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T13:49:05.576 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T13:49:05.576 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 51881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T13:49:05.576 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T13:49:05.576 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 42761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T13:49:05.577 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/ibm_rtl.ko 2024-04-25T13:49:05.577 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/ 2024-04-25T13:49:05.577 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T13:49:05.577 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-04-25T13:49:05.577 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 54001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T13:49:05.577 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T13:49:05.577 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T13:49:05.577 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 43753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T13:49:05.577 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 51897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T13:49:05.577 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T13:49:05.577 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T13:49:05.577 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T13:49:05.577 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T13:49:05.577 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T13:49:05.578 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 61417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T13:49:05.578 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/ 2024-04-25T13:49:05.578 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ 2024-04-25T13:49:05.578 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 36513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/88pm860x_battery.ko 2024-04-25T13:49:05.578 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 40425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/88pm860x_charger.ko 2024-04-25T13:49:05.578 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 68833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq2415x_charger.ko 2024-04-25T13:49:05.578 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 73073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq24190_charger.ko 2024-04-25T13:49:05.578 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq24735-charger.ko 2024-04-25T13:49:05.578 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-04-25T13:49:05.578 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/da9030_battery.ko 2024-04-25T13:49:05.578 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/da9052-battery.ko 2024-04-25T13:49:05.578 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 34489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2760_battery.ko 2024-04-25T13:49:05.578 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2780_battery.ko 2024-04-25T13:49:05.578 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2781_battery.ko 2024-04-25T13:49:05.579 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2782_battery.ko 2024-04-25T13:49:05.579 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/gpio-charger.ko 2024-04-25T13:49:05.579 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/isp1704_charger.ko 2024-04-25T13:49:05.579 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/lp8727_charger.ko 2024-04-25T13:49:05.579 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max14577_charger.ko 2024-04-25T13:49:05.579 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 37833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max17042_battery.ko 2024-04-25T13:49:05.579 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max8903_charger.ko 2024-04-25T13:49:05.579 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 27689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max8925_power.ko 2024-04-25T13:49:05.579 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/pcf50633-charger.ko 2024-04-25T13:49:05.579 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 44841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/sbs-battery.ko 2024-04-25T13:49:05.579 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 34905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/test_power.ko 2024-04-25T13:49:05.579 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/tps65090-charger.ko 2024-04-25T13:49:05.579 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm831x_backup.ko 2024-04-25T13:49:05.579 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm831x_power.ko 2024-04-25T13:49:05.580 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 27441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm8350_power.ko 2024-04-25T13:49:05.580 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/ 2024-04-25T13:49:05.580 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 73297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T13:49:05.580 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T13:49:05.580 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/ 2024-04-25T13:49:05.580 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/ 2024-04-25T13:49:05.580 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T13:49:05.580 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T13:49:05.580 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T13:49:05.580 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 49817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T13:49:05.580 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ 2024-04-25T13:49:05.580 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 91393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T13:49:05.580 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T13:49:05.580 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/ 2024-04-25T13:49:05.581 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/ 2024-04-25T13:49:05.581 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/ 2024-04-25T13:49:05.581 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 67513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T13:49:05.581 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 47137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T13:49:05.581 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/ 2024-04-25T13:49:05.581 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 27593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-88pm80x.ko 2024-04-25T13:49:05.581 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-88pm860x.ko 2024-04-25T13:49:05.581 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T13:49:05.581 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9052.ko 2024-04-25T13:49:05.581 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9055.ko 2024-04-25T13:49:05.581 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9063.ko 2024-04-25T13:49:05.581 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T13:49:05.581 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 82225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T13:49:05.581 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T13:49:05.582 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T13:49:05.582 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T13:49:05.582 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T13:49:05.582 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T13:49:05.582 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T13:49:05.582 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T13:49:05.582 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T13:49:05.582 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T13:49:05.582 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T13:49:05.582 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 44129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T13:49:05.582 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-lp8788.ko 2024-04-25T13:49:05.582 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 46913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T13:49:05.582 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T13:49:05.582 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 33969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T13:49:05.583 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t86.ko 2024-04-25T13:49:05.583 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T13:49:05.583 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8907.ko 2024-04-25T13:49:05.583 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8925.ko 2024-04-25T13:49:05.583 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 31337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8997.ko 2024-04-25T13:49:05.583 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8998.ko 2024-04-25T13:49:05.583 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-04-25T13:49:05.583 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T13:49:05.583 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-palmas.ko 2024-04-25T13:49:05.583 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 56777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T13:49:05.583 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf50633.ko 2024-04-25T13:49:05.583 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 32657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T13:49:05.583 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T13:49:05.583 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 36065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T13:49:05.584 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T13:49:05.584 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rc5t583.ko 2024-04-25T13:49:05.584 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T13:49:05.584 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 40081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T13:49:05.584 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 34529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T13:49:05.584 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T13:49:05.584 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T13:49:05.584 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 26185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-s35390a.ko 2024-04-25T13:49:05.584 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T13:49:05.584 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-tps6586x.ko 2024-04-25T13:49:05.584 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-tps65910.ko 2024-04-25T13:49:05.584 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-wm831x.ko 2024-04-25T13:49:05.584 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-wm8350.ko 2024-04-25T13:49:05.584 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 33409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T13:49:05.585 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ 2024-04-25T13:49:05.585 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 90377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T13:49:05.585 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 69593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T13:49:05.585 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 65945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T13:49:05.585 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 164569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T13:49:05.585 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 31617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T13:49:05.585 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/ 2024-04-25T13:49:05.585 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 340721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T13:49:05.585 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 120337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T13:49:05.585 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/ 2024-04-25T13:49:05.585 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 468081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T13:49:05.585 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 380673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T13:49:05.585 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic94xx/ 2024-04-25T13:49:05.585 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 243705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-04-25T13:49:05.586 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T13:49:05.586 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/ 2024-04-25T13:49:05.586 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 135617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T13:49:05.586 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 44289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T13:49:05.586 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/ 2024-04-25T13:49:05.586 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 304993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T13:49:05.586 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/ 2024-04-25T13:49:05.586 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 1527113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T13:49:05.586 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/ 2024-04-25T13:49:05.586 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 297801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T13:49:05.586 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/ 2024-04-25T13:49:05.586 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 144401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T13:49:05.586 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 47457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T13:49:05.586 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/ 2024-04-25T13:49:05.587 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 390713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T13:49:05.587 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/ 2024-04-25T13:49:05.587 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-04-25T13:49:05.587 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 88881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T13:49:05.587 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-04-25T13:49:05.587 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 117481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T13:49:05.587 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 173017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T13:49:05.587 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 91569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T13:49:05.587 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 45401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T13:49:05.587 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/ 2024-04-25T13:49:05.587 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 267577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T13:49:05.587 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 78969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T13:49:05.587 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/ 2024-04-25T13:49:05.587 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 95329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T13:49:05.588 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 194017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T13:49:05.588 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fdomain.ko 2024-04-25T13:49:05.588 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/ 2024-04-25T13:49:05.588 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 366529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T13:49:05.588 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 251505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T13:49:05.588 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 65761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T13:49:05.588 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 57097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T13:49:05.588 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 44689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/imm.ko 2024-04-25T13:49:05.588 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 55153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T13:49:05.588 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 327841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T13:49:05.588 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 129785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T13:49:05.588 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/ 2024-04-25T13:49:05.588 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 397785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T13:49:05.588 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 32017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T13:49:05.589 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 63121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T13:49:05.589 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/ 2024-04-25T13:49:05.589 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 386849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T13:49:05.589 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 178177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T13:49:05.589 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 64809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T13:49:05.589 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/ 2024-04-25T13:49:05.589 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 274009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T13:49:05.589 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/ 2024-04-25T13:49:05.589 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 2736001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T13:49:05.589 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/ 2024-04-25T13:49:05.589 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 88529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T13:49:05.589 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 40145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T13:49:05.589 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 389177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T13:49:05.589 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 104769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T13:49:05.590 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/ 2024-04-25T13:49:05.590 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 836945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T13:49:05.590 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/ 2024-04-25T13:49:05.590 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 158729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T13:49:05.590 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 96537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T13:49:05.590 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/ 2024-04-25T13:49:05.590 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 101625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-04-25T13:49:05.590 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-04-25T13:49:05.590 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-04-25T13:49:05.590 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-04-25T13:49:05.590 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/ 2024-04-25T13:49:05.590 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 443745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T13:49:05.590 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 152033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T13:49:05.590 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 41625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ppa.ko 2024-04-25T13:49:05.591 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 66977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T13:49:05.591 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/ 2024-04-25T13:49:05.591 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 2324809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T13:49:05.591 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 103601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T13:49:05.591 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/ 2024-04-25T13:49:05.591 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 697337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T13:49:05.591 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qlogicfas408.ko 2024-04-25T13:49:05.591 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T13:49:05.591 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 322865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T13:49:05.591 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 219857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T13:49:05.591 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 357153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T13:49:05.591 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 121185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T13:49:05.591 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 93481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T13:49:05.591 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 55401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T13:49:05.592 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 33009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T13:49:05.592 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 138121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T13:49:05.592 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 57401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T13:49:05.592 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/ 2024-04-25T13:49:05.592 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 205025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T13:49:05.592 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 55425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T13:49:05.592 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 59185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T13:49:05.592 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 48777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T13:49:05.592 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 59417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/xen-scsifront.ko 2024-04-25T13:49:05.592 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ 2024-04-25T13:49:05.592 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 174913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T13:49:05.592 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/ 2024-04-25T13:49:05.592 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/ 2024-04-25T13:49:05.592 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 920169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T13:49:05.593 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/ 2024-04-25T13:49:05.593 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 60321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T13:49:05.593 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 48129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T13:49:05.593 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 54233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T13:49:05.593 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 1170033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T13:49:05.593 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 51897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T13:49:05.593 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 149721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T13:49:05.593 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/ 2024-04-25T13:49:05.593 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 90057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T13:49:05.593 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/ 2024-04-25T13:49:05.593 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/ 2024-04-25T13:49:05.593 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/ 2024-04-25T13:49:05.593 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T13:49:05.593 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 35209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T13:49:05.594 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T13:49:05.594 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T13:49:05.594 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T13:49:05.594 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T13:49:05.594 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 36729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T13:49:05.594 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 26289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T13:49:05.594 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T13:49:05.594 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T13:49:05.594 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T13:49:05.594 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T13:49:05.594 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 46065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T13:49:05.594 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T13:49:05.594 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T13:49:05.595 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 35913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-04-25T13:49:05.595 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T13:49:05.595 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T13:49:05.595 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 30577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T13:49:05.595 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ 2024-04-25T13:49:05.595 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ 2024-04-25T13:49:05.595 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 99313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T13:49:05.595 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 69729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/moxa.ko 2024-04-25T13:49:05.595 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 61649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/mxser.ko 2024-04-25T13:49:05.595 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 134401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T13:49:05.595 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 37905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T13:49:05.595 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 78969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T13:49:05.595 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/ 2024-04-25T13:49:05.595 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/ 2024-04-25T13:49:05.595 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 44057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-04-25T13:49:05.596 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 65121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T13:49:05.596 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-04-25T13:49:05.596 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/altera_uart.ko 2024-04-25T13:49:05.596 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T13:49:05.596 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 91665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-04-25T13:49:05.596 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/ 2024-04-25T13:49:05.596 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 120577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T13:49:05.596 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 35161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/rp2.ko 2024-04-25T13:49:05.596 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 65641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/sc16is7xx.ko 2024-04-25T13:49:05.596 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/ 2024-04-25T13:49:05.596 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ 2024-04-25T13:49:05.596 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 693081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T13:49:05.596 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ 2024-04-25T13:49:05.596 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 34449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T13:49:05.597 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-04-25T13:49:05.597 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/ 2024-04-25T13:49:05.597 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 55617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T13:49:05.597 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T13:49:05.597 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T13:49:05.597 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_dmem_genirq.ko 2024-04-25T13:49:05.597 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_mf624.ko 2024-04-25T13:49:05.597 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_netx.ko 2024-04-25T13:49:05.597 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T13:49:05.597 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-04-25T13:49:05.597 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T13:49:05.597 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/ 2024-04-25T13:49:05.597 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/c67x00/ 2024-04-25T13:49:05.597 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 90321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/c67x00/c67x00.ko 2024-04-25T13:49:05.598 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ 2024-04-25T13:49:05.598 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 189209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-04-25T13:49:05.598 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-04-25T13:49:05.598 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-04-25T13:49:05.598 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-04-25T13:49:05.598 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-04-25T13:49:05.598 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/ 2024-04-25T13:49:05.598 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 101545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T13:49:05.598 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 63073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T13:49:05.598 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 56721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T13:49:05.598 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 107417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T13:49:05.598 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/common/ 2024-04-25T13:49:05.598 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/common/ulpi.ko 2024-04-25T13:49:05.598 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc2/ 2024-04-25T13:49:05.599 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-04-25T13:49:05.599 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/ 2024-04-25T13:49:05.599 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-04-25T13:49:05.599 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 33465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-04-25T13:49:05.599 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 574593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3.ko 2024-04-25T13:49:05.599 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/ 2024-04-25T13:49:05.599 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/ 2024-04-25T13:49:05.599 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 72249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/u_ether.ko 2024-04-25T13:49:05.599 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 61953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/u_serial.ko 2024-04-25T13:49:05.599 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 33721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-04-25T13:49:05.599 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 49737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-04-25T13:49:05.599 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 33041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-04-25T13:49:05.599 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 43553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-04-25T13:49:05.600 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 131345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-04-25T13:49:05.600 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 62321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-04-25T13:49:05.600 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 146617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-04-25T13:49:05.600 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 70689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-04-25T13:49:05.600 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-04-25T13:49:05.600 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 37649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-04-25T13:49:05.600 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 70017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-04-25T13:49:05.600 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 101249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-04-25T13:49:05.600 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-04-25T13:49:05.600 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 71257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-04-25T13:49:05.600 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 84937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-04-25T13:49:05.600 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/ 2024-04-25T13:49:05.600 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-04-25T13:49:05.600 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-04-25T13:49:05.600 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-04-25T13:49:05.601 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-04-25T13:49:05.601 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 33041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-04-25T13:49:05.601 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-04-25T13:49:05.601 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-04-25T13:49:05.601 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-04-25T13:49:05.601 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 35689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-04-25T13:49:05.601 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-04-25T13:49:05.601 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-04-25T13:49:05.601 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 30345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-04-25T13:49:05.601 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 78073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-04-25T13:49:05.601 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-04-25T13:49:05.601 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 204049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/libcomposite.ko 2024-04-25T13:49:05.601 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/ 2024-04-25T13:49:05.601 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-04-25T13:49:05.602 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/bdc/ 2024-04-25T13:49:05.602 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 182561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-04-25T13:49:05.602 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 45545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-04-25T13:49:05.602 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 66513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-04-25T13:49:05.602 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 85217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-04-25T13:49:05.602 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 69313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-04-25T13:49:05.602 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 90313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/net2272.ko 2024-04-25T13:49:05.602 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 90361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/net2280.ko 2024-04-25T13:49:05.602 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 94841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-04-25T13:49:05.602 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 80521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-04-25T13:49:05.602 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 87465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-04-25T13:49:05.602 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 95441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-04-25T13:49:05.602 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 195969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-04-25T13:49:05.603 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/ 2024-04-25T13:49:05.603 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/bcma-hcd.ko 2024-04-25T13:49:05.603 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 108321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/isp116x-hcd.ko 2024-04-25T13:49:05.603 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 104377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-04-25T13:49:05.603 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 102273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-04-25T13:49:05.603 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 60105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T13:49:05.603 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811_cs.ko 2024-04-25T13:49:05.603 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/ssb-hcd.ko 2024-04-25T13:49:05.603 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 32553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-04-25T13:49:05.603 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/ 2024-04-25T13:49:05.603 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 40409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T13:49:05.603 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T13:49:05.603 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ 2024-04-25T13:49:05.603 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 47353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T13:49:05.604 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T13:49:05.604 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-04-25T13:49:05.604 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 26673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/cytherm.ko 2024-04-25T13:49:05.604 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ehset.ko 2024-04-25T13:49:05.604 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T13:49:05.604 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T13:49:05.604 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T13:49:05.604 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T13:49:05.604 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 42729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T13:49:05.604 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T13:49:05.604 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 42241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T13:49:05.604 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 42233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T13:49:05.604 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 33641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/lvstest.ko 2024-04-25T13:49:05.604 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/ 2024-04-25T13:49:05.604 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 85313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T13:49:05.605 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T13:49:05.605 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T13:49:05.605 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T13:49:05.605 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 27809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T13:49:05.605 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 84265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbtest.ko 2024-04-25T13:49:05.605 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 52065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T13:49:05.605 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 32345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T13:49:05.605 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/mon/ 2024-04-25T13:49:05.605 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 91529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/mon/usbmon.ko 2024-04-25T13:49:05.605 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/musb/ 2024-04-25T13:49:05.605 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 396209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/musb/musb_hdrc.ko 2024-04-25T13:49:05.605 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/ 2024-04-25T13:49:05.605 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T13:49:05.605 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-04-25T13:49:05.606 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-isp1301.ko 2024-04-25T13:49:05.606 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 26361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-tahvo.ko 2024-04-25T13:49:05.606 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ 2024-04-25T13:49:05.606 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T13:49:05.606 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 38209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T13:49:05.606 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 30681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T13:49:05.606 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 44329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T13:49:05.606 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 79681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T13:49:05.606 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 35137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T13:49:05.606 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 66873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T13:49:05.606 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 58329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T13:49:05.606 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T13:49:05.606 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 53009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/f81232.ko 2024-04-25T13:49:05.607 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 188609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T13:49:05.607 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 61489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T13:49:05.607 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 152305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T13:49:05.607 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 142833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T13:49:05.607 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 69217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T13:49:05.607 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T13:49:05.607 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T13:49:05.607 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 66361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T13:49:05.607 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 122329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T13:49:05.607 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 41297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T13:49:05.607 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T13:49:05.607 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 38049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T13:49:05.607 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 38041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T13:49:05.607 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/metro-usb.ko 2024-04-25T13:49:05.607 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 79681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T13:49:05.608 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 89969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T13:49:05.608 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 65073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mxuport.ko 2024-04-25T13:49:05.608 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T13:49:05.608 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T13:49:05.608 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T13:49:05.608 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 193857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T13:49:05.608 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 45337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T13:49:05.608 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 61761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T13:49:05.608 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T13:49:05.608 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 39985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T13:49:05.608 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 40529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T13:49:05.608 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T13:49:05.608 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 59345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T13:49:05.608 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T13:49:05.609 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 34065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T13:49:05.609 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T13:49:05.609 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 70497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T13:49:05.609 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T13:49:05.609 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T13:49:05.609 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 39321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T13:49:05.609 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 151137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usbserial.ko 2024-04-25T13:49:05.609 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 32505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T13:49:05.609 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 43577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T13:49:05.609 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/wishbone-serial.ko 2024-04-25T13:49:05.609 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T13:49:05.609 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ 2024-04-25T13:49:05.609 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 64049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T13:49:05.609 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 30209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T13:49:05.610 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T13:49:05.610 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T13:49:05.610 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 41073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T13:49:05.610 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T13:49:05.610 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T13:49:05.610 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T13:49:05.610 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T13:49:05.610 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T13:49:05.610 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 33873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T13:49:05.610 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 33849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T13:49:05.610 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T13:49:05.610 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 41785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T13:49:05.610 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 201193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T13:49:05.610 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/ 2024-04-25T13:49:05.611 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 76041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T13:49:05.611 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 85257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T13:49:05.611 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 122321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T13:49:05.611 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/ 2024-04-25T13:49:05.611 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/ 2024-04-25T13:49:05.611 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 206921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T13:49:05.611 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 33577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T13:49:05.611 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 134265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T13:49:05.611 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 93673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T13:49:05.611 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/ 2024-04-25T13:49:05.611 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 138721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T13:49:05.611 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T13:49:05.611 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 71281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T13:49:05.611 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 107905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T13:49:05.612 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/ 2024-04-25T13:49:05.612 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/ 2024-04-25T13:49:05.612 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-04-25T13:49:05.612 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/aat2870_bl.ko 2024-04-25T13:49:05.612 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp5520_bl.ko 2024-04-25T13:49:05.612 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 41169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp8860_bl.ko 2024-04-25T13:49:05.612 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 51745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp8870_bl.ko 2024-04-25T13:49:05.612 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T13:49:05.612 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/as3711_bl.ko 2024-04-25T13:49:05.612 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/bd6107.ko 2024-04-25T13:49:05.612 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/da903x_bl.ko 2024-04-25T13:49:05.612 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/da9052_bl.ko 2024-04-25T13:49:05.612 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/gpio_backlight.ko 2024-04-25T13:49:05.612 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/kb3886_bl.ko 2024-04-25T13:49:05.613 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 26457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T13:49:05.613 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lm3533_bl.ko 2024-04-25T13:49:05.613 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lm3639_bl.ko 2024-04-25T13:49:05.613 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lv5207lp.ko 2024-04-25T13:49:05.613 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/max8925_bl.ko 2024-04-25T13:49:05.613 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/pandora_bl.ko 2024-04-25T13:49:05.613 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-04-25T13:49:05.613 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T13:49:05.613 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/wm831x_bl.ko 2024-04-25T13:49:05.613 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/ 2024-04-25T13:49:05.613 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ 2024-04-25T13:49:05.613 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T13:49:05.613 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 46801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T13:49:05.613 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/matrox_w1.ko 2024-04-25T13:49:05.614 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/w1-gpio.ko 2024-04-25T13:49:05.614 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/ 2024-04-25T13:49:05.614 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T13:49:05.614 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 27257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T13:49:05.614 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T13:49:05.614 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T13:49:05.614 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T13:49:05.614 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T13:49:05.614 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T13:49:05.614 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T13:49:05.614 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T13:49:05.614 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 8313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T13:49:05.614 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 70369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T13:49:05.614 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 129433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T13:49:05.615 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ 2024-04-25T13:49:05.615 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/acquirewdt.ko 2024-04-25T13:49:05.615 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/advantechwdt.ko 2024-04-25T13:49:05.615 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T13:49:05.615 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 27401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T13:49:05.615 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/cpu5wdt.ko 2024-04-25T13:49:05.615 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9052_wdt.ko 2024-04-25T13:49:05.615 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9055_wdt.ko 2024-04-25T13:49:05.615 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9063_wdt.ko 2024-04-25T13:49:05.615 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/dw_wdt.ko 2024-04-25T13:49:05.615 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/eurotechwdt.ko 2024-04-25T13:49:05.615 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 26817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T13:49:05.615 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 32857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T13:49:05.615 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T13:49:05.616 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T13:49:05.616 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 26737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T13:49:05.616 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T13:49:05.616 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T13:49:05.616 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T13:49:05.616 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T13:49:05.616 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T13:49:05.616 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 26441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/kempld_wdt.ko 2024-04-25T13:49:05.616 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T13:49:05.616 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/mena21_wdt.ko 2024-04-25T13:49:05.616 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-04-25T13:49:05.617 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T13:49:05.617 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-04-25T13:49:05.617 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 27721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pc87413_wdt.ko 2024-04-25T13:49:05.617 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 46953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T13:49:05.617 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 37193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T13:49:05.617 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/retu_wdt.ko 2024-04-25T13:49:05.617 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-04-25T13:49:05.617 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-04-25T13:49:05.617 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T13:49:05.617 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sc1200wdt.ko 2024-04-25T13:49:05.617 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 27489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T13:49:05.617 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-04-25T13:49:05.617 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T13:49:05.617 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 26009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T13:49:05.618 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/twl4030_wdt.ko 2024-04-25T13:49:05.618 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T13:49:05.618 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T13:49:05.618 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T13:49:05.618 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 32481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T13:49:05.618 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wafer5823wdt.ko 2024-04-25T13:49:05.618 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 33465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T13:49:05.618 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wm831x_wdt.ko 2024-04-25T13:49:05.618 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wm8350_wdt.ko 2024-04-25T13:49:05.618 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T13:49:05.618 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/ 2024-04-25T13:49:05.618 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 30049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T13:49:05.618 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 30649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T13:49:05.619 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 55713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T13:49:05.619 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/ 2024-04-25T13:49:05.619 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 213913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T13:49:05.619 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 47689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T13:49:05.619 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 85713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-scsiback.ko 2024-04-25T13:49:05.619 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/ 2024-04-25T13:49:05.619 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T13:49:05.619 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ 2024-04-25T13:49:05.619 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/ 2024-04-25T13:49:05.619 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 192617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T13:49:05.619 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/adfs/ 2024-04-25T13:49:05.619 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 78153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/adfs/adfs.ko 2024-04-25T13:49:05.619 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/ 2024-04-25T13:49:05.619 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 197849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T13:49:05.619 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/afs/ 2024-04-25T13:49:05.620 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 1261113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/afs/kafs.ko 2024-04-25T13:49:05.620 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/autofs/ 2024-04-25T13:49:05.620 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 127377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/autofs/autofs4.ko 2024-04-25T13:49:05.620 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/ 2024-04-25T13:49:05.620 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 80521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T13:49:05.620 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/bfs/ 2024-04-25T13:49:05.620 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 50473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/bfs/bfs.ko 2024-04-25T13:49:05.620 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 45561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T13:49:05.620 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/ 2024-04-25T13:49:05.620 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 4489593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T13:49:05.661 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/ 2024-04-25T13:49:05.661 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 376857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T13:49:05.662 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ 2024-04-25T13:49:05.662 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 1586065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T13:49:05.662 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/ 2024-04-25T13:49:05.662 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 130073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T13:49:05.662 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/configfs/ 2024-04-25T13:49:05.662 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 139081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/configfs/configfs.ko 2024-04-25T13:49:05.662 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/ 2024-04-25T13:49:05.662 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 35665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T13:49:05.662 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/ 2024-04-25T13:49:05.662 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 813225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T13:49:05.662 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/efs/ 2024-04-25T13:49:05.662 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 46409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/efs/efs.ko 2024-04-25T13:49:05.662 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/ 2024-04-25T13:49:05.663 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 1906817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T13:49:05.663 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/ 2024-04-25T13:49:05.663 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 35137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T13:49:05.663 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/freevxfs/ 2024-04-25T13:49:05.663 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 52313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/freevxfs/freevxfs.ko 2024-04-25T13:49:05.663 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/ 2024-04-25T13:49:05.663 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T13:49:05.663 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/ 2024-04-25T13:49:05.663 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 1486521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T13:49:05.663 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/ 2024-04-25T13:49:05.663 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 183305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T13:49:05.663 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/ 2024-04-25T13:49:05.663 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 282097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T13:49:05.663 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hpfs/ 2024-04-25T13:49:05.664 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 249009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hpfs/hpfs.ko 2024-04-25T13:49:05.664 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/ 2024-04-25T13:49:05.664 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 93033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T13:49:05.664 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/ 2024-04-25T13:49:05.664 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 533073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T13:49:05.664 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/ 2024-04-25T13:49:05.664 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 361369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T13:49:05.664 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/ 2024-04-25T13:49:05.664 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 112609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T13:49:05.664 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/ 2024-04-25T13:49:05.664 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 591585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T13:49:05.664 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/ 2024-04-25T13:49:05.664 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/ 2024-04-25T13:49:05.664 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 102561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T13:49:05.665 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/ 2024-04-25T13:49:05.665 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 74353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T13:49:05.665 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/ 2024-04-25T13:49:05.665 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 147689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T13:49:05.665 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 1320249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T13:49:05.665 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 82105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv2.ko 2024-04-25T13:49:05.665 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 151641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T13:49:05.665 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 2518505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T13:49:05.665 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/ 2024-04-25T13:49:05.665 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T13:49:05.665 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T13:49:05.665 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/ 2024-04-25T13:49:05.665 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 2024049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T13:49:05.665 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/ 2024-04-25T13:49:05.666 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 734025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T13:49:05.666 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/ 2024-04-25T13:49:05.666 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T13:49:05.666 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T13:49:05.666 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T13:49:05.666 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T13:49:05.666 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T13:49:05.666 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T13:49:05.666 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T13:49:05.666 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T13:49:05.666 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T13:49:05.666 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T13:49:05.666 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T13:49:05.666 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ascii.ko 2024-04-25T13:49:05.667 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T13:49:05.667 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T13:49:05.667 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T13:49:05.667 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T13:49:05.667 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T13:49:05.667 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T13:49:05.667 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T13:49:05.667 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T13:49:05.667 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T13:49:05.667 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T13:49:05.667 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T13:49:05.667 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T13:49:05.667 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T13:49:05.667 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T13:49:05.668 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T13:49:05.668 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T13:49:05.668 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T13:49:05.668 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T13:49:05.668 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 95625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T13:49:05.668 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 144209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T13:49:05.668 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 167137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T13:49:05.668 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 119513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T13:49:05.668 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T13:49:05.668 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T13:49:05.668 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T13:49:05.668 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T13:49:05.668 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T13:49:05.668 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T13:49:05.669 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T13:49:05.669 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T13:49:05.669 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T13:49:05.669 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T13:49:05.669 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T13:49:05.669 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T13:49:05.669 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T13:49:05.669 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T13:49:05.669 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T13:49:05.669 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 7521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T13:49:05.669 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T13:49:05.669 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ 2024-04-25T13:49:05.669 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ 2024-04-25T13:49:05.669 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 302073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T13:49:05.670 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ 2024-04-25T13:49:05.670 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 548169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T13:49:05.670 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ 2024-04-25T13:49:05.670 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 66361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T13:49:05.670 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 3245617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T13:49:05.670 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 22113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T13:49:05.670 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 39313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T13:49:05.670 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 43073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T13:49:05.670 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/omfs/ 2024-04-25T13:49:05.670 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 67489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/omfs/omfs.ko 2024-04-25T13:49:05.670 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/ 2024-04-25T13:49:05.670 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 462873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T13:49:05.670 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ 2024-04-25T13:49:05.670 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 67977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T13:49:05.671 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx4/ 2024-04-25T13:49:05.671 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 33209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx4/qnx4.ko 2024-04-25T13:49:05.671 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx6/ 2024-04-25T13:49:05.671 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 56097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx6/qnx6.ko 2024-04-25T13:49:05.671 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/ 2024-04-25T13:49:05.671 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 40313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_tree.ko 2024-04-25T13:49:05.671 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_v1.ko 2024-04-25T13:49:05.671 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_v2.ko 2024-04-25T13:49:05.671 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/ 2024-04-25T13:49:05.671 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 684841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T13:49:05.671 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/ 2024-04-25T13:49:05.671 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 34673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T13:49:05.671 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/ 2024-04-25T13:49:05.671 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/ 2024-04-25T13:49:05.671 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 3319041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T13:49:05.672 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/ 2024-04-25T13:49:05.672 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 8073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T13:49:05.672 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T13:49:05.672 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/ 2024-04-25T13:49:05.672 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 164185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T13:49:05.672 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/ 2024-04-25T13:49:05.672 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 115977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T13:49:05.672 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/ 2024-04-25T13:49:05.672 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 310017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T13:49:05.672 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ 2024-04-25T13:49:05.672 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 198233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T13:49:05.672 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/ 2024-04-25T13:49:05.672 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 5192705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T13:49:05.672 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/ 2024-04-25T13:49:05.673 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/locking/ 2024-04-25T13:49:05.673 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 97625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/locking/locktorture.ko 2024-04-25T13:49:05.673 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/time/ 2024-04-25T13:49:05.673 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/time/test_udelay.ko 2024-04-25T13:49:05.673 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 72665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T13:49:05.673 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ 2024-04-25T13:49:05.673 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 7313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T13:49:05.673 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 7081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc7.ko 2024-04-25T13:49:05.673 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 7953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T13:49:05.673 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/ 2024-04-25T13:49:05.673 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 8009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T13:49:05.673 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 38985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T13:49:05.673 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/interval_tree_test.ko 2024-04-25T13:49:05.673 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T13:49:05.674 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T13:49:05.674 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/ 2024-04-25T13:49:05.674 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 36169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T13:49:05.674 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T13:49:05.674 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/ 2024-04-25T13:49:05.674 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 7281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T13:49:05.674 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/memory-notifier-error-inject.ko 2024-04-25T13:49:05.674 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/notifier-error-inject.ko 2024-04-25T13:49:05.674 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/percpu_test.ko 2024-04-25T13:49:05.674 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/pm-notifier-error-inject.ko 2024-04-25T13:49:05.674 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/ 2024-04-25T13:49:05.674 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 152025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T13:49:05.674 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/rbtree_test.ko 2024-04-25T13:49:05.674 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/ 2024-04-25T13:49:05.675 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T13:49:05.675 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 40969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test-kstrtox.ko 2024-04-25T13:49:05.675 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 4912441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_bpf.ko 2024-04-25T13:49:05.675 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 61169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_firmware.ko 2024-04-25T13:49:05.675 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 7121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_module.ko 2024-04-25T13:49:05.675 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_user_copy.ko 2024-04-25T13:49:05.675 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T13:49:05.675 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T13:49:05.675 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T13:49:05.675 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/xz/ 2024-04-25T13:49:05.675 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/xz/xz_dec_test.ko 2024-04-25T13:49:05.675 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/ 2024-04-25T13:49:05.675 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T13:49:05.675 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ 2024-04-25T13:49:05.676 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/ 2024-04-25T13:49:05.676 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 66969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T13:49:05.676 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 7577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T13:49:05.676 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 7529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T13:49:05.676 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 7513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T13:49:05.676 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 7513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T13:49:05.676 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 7529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T13:49:05.676 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 7521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T13:49:05.676 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T13:49:05.676 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/ 2024-04-25T13:49:05.676 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 36729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T13:49:05.676 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 42297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T13:49:05.676 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T13:49:05.676 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T13:49:05.676 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T13:49:05.677 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/ 2024-04-25T13:49:05.677 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 106793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T13:49:05.677 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/ 2024-04-25T13:49:05.677 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 236793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T13:49:05.677 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 44449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T13:49:05.677 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 32209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T13:49:05.677 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 39513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T13:49:05.677 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/ 2024-04-25T13:49:05.677 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 879697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T13:49:05.677 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/ 2024-04-25T13:49:05.677 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 84761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T13:49:05.677 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 1030873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T13:49:05.677 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ 2024-04-25T13:49:05.678 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 8905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T13:49:05.678 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T13:49:05.678 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T13:49:05.678 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T13:49:05.678 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T13:49:05.678 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T13:49:05.678 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T13:49:05.678 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T13:49:05.678 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T13:49:05.678 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 8841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T13:49:05.678 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 8777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T13:49:05.678 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T13:49:05.678 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T13:49:05.678 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T13:49:05.679 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T13:49:05.679 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T13:49:05.679 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T13:49:05.679 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T13:49:05.679 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T13:49:05.679 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T13:49:05.679 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 70529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T13:49:05.679 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T13:49:05.679 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T13:49:05.679 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/ 2024-04-25T13:49:05.679 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 1243505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T13:49:05.679 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/ 2024-04-25T13:49:05.679 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 153801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T13:49:05.679 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/ 2024-04-25T13:49:05.679 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 244889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T13:49:05.680 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T13:49:05.680 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 71033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T13:49:05.680 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 74993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T13:49:05.680 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/ 2024-04-25T13:49:05.680 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 507705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T13:49:05.680 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T13:49:05.680 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T13:49:05.680 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 8873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T13:49:05.680 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T13:49:05.680 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/hsr/ 2024-04-25T13:49:05.680 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 128873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/hsr/hsr.ko 2024-04-25T13:49:05.680 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ 2024-04-25T13:49:05.680 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ 2024-04-25T13:49:05.680 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 55505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T13:49:05.681 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 357425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T13:49:05.681 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 59393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T13:49:05.681 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ 2024-04-25T13:49:05.681 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T13:49:05.681 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 46417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T13:49:05.681 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 60233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T13:49:05.681 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T13:49:05.681 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 65665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T13:49:05.681 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 75489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T13:49:05.681 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 77721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T13:49:05.681 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 43705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T13:49:05.681 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T13:49:05.681 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 34745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T13:49:05.681 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ 2024-04-25T13:49:05.681 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 45521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T13:49:05.682 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T13:49:05.682 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T13:49:05.682 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 50169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ip_tables.ko 2024-04-25T13:49:05.682 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T13:49:05.682 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-04-25T13:49:05.682 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T13:49:05.682 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T13:49:05.682 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T13:49:05.682 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-04-25T13:49:05.682 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T13:49:05.682 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T13:49:05.682 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T13:49:05.682 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T13:49:05.683 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T13:49:05.683 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T13:49:05.683 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 32473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T13:49:05.683 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T13:49:05.683 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T13:49:05.683 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-04-25T13:49:05.683 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T13:49:05.683 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T13:49:05.683 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T13:49:05.683 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T13:49:05.683 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T13:49:05.683 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T13:49:05.683 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T13:49:05.683 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T13:49:05.684 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T13:49:05.684 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T13:49:05.684 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T13:49:05.684 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T13:49:05.684 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T13:49:05.684 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T13:49:05.684 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T13:49:05.684 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T13:49:05.684 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T13:49:05.684 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T13:49:05.684 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T13:49:05.684 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T13:49:05.684 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 54441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T13:49:05.684 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T13:49:05.684 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ 2024-04-25T13:49:05.685 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 31073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T13:49:05.685 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 47433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T13:49:05.685 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T13:49:05.685 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 103177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_gre.ko 2024-04-25T13:49:05.685 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 118881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T13:49:05.685 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T13:49:05.685 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 71089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T13:49:05.685 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T13:49:05.685 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/mip6.ko 2024-04-25T13:49:05.685 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ 2024-04-25T13:49:05.685 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 50673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T13:49:05.685 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-04-25T13:49:05.685 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T13:49:05.685 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T13:49:05.686 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T13:49:05.686 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 8721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T13:49:05.686 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T13:49:05.686 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T13:49:05.686 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T13:49:05.686 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T13:49:05.686 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T13:49:05.686 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T13:49:05.686 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T13:49:05.686 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T13:49:05.686 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T13:49:05.686 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T13:49:05.686 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T13:49:05.686 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 46313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T13:49:05.687 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T13:49:05.687 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 29817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T13:49:05.687 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T13:49:05.687 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T13:49:05.687 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T13:49:05.687 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T13:49:05.687 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T13:49:05.687 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 94089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T13:49:05.687 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 30297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T13:49:05.687 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T13:49:05.687 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/ 2024-04-25T13:49:05.687 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 106369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T13:49:05.687 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/ 2024-04-25T13:49:05.687 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 153217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T13:49:05.688 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T13:49:05.688 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 31985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T13:49:05.688 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 43641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T13:49:05.688 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 44993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T13:49:05.688 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 42049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T13:49:05.688 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 66241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T13:49:05.688 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/lapb/ 2024-04-25T13:49:05.688 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 63345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/lapb/lapb.ko 2024-04-25T13:49:05.688 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/ 2024-04-25T13:49:05.688 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 27385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T13:49:05.688 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 332889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc2.ko 2024-04-25T13:49:05.688 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/ 2024-04-25T13:49:05.688 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 388873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T13:49:05.688 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/ 2024-04-25T13:49:05.688 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T13:49:05.689 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ 2024-04-25T13:49:05.689 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ 2024-04-25T13:49:05.689 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 147745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T13:49:05.689 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 34473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T13:49:05.689 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 33073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T13:49:05.689 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 32553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T13:49:05.689 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 124041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T13:49:05.689 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 124345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T13:49:05.689 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 126681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T13:49:05.689 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 126473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T13:49:05.689 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 137833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T13:49:05.689 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 72353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T13:49:05.689 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 133681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T13:49:05.689 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 139073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T13:49:05.690 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 138265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T13:49:05.690 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 136177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T13:49:05.690 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 139897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T13:49:05.690 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 37129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T13:49:05.690 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ 2024-04-25T13:49:05.690 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 534113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T13:49:05.690 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T13:49:05.690 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 8801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T13:49:05.690 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T13:49:05.690 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T13:49:05.690 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T13:49:05.690 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 8801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T13:49:05.690 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 8865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T13:49:05.690 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T13:49:05.691 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T13:49:05.691 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 8921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T13:49:05.691 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T13:49:05.691 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 8921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T13:49:05.691 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T13:49:05.691 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 55217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T13:49:05.691 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 496817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T13:49:05.691 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T13:49:05.691 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T13:49:05.691 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 37865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T13:49:05.691 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 134377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T13:49:05.691 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T13:49:05.691 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T13:49:05.691 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 140801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T13:49:05.692 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 38305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T13:49:05.692 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T13:49:05.692 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 79873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T13:49:05.692 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T13:49:05.692 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T13:49:05.692 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T13:49:05.692 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 117201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T13:49:05.692 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T13:49:05.692 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 46105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T13:49:05.692 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 118369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T13:49:05.692 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T13:49:05.692 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T13:49:05.692 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T13:49:05.692 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 34609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T13:49:05.693 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T13:49:05.693 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 53393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T13:49:05.693 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 807985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T13:49:05.693 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 53049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T13:49:05.693 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 36481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T13:49:05.693 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 37209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-04-25T13:49:05.693 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 49297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T13:49:05.693 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 26473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T13:49:05.693 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 65145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T13:49:05.693 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T13:49:05.693 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 41769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T13:49:05.693 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 50657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T13:49:05.693 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T13:49:05.693 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T13:49:05.693 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T13:49:05.694 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T13:49:05.694 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T13:49:05.694 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T13:49:05.694 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T13:49:05.694 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T13:49:05.694 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T13:49:05.694 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_osf.ko 2024-04-25T13:49:05.694 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T13:49:05.694 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T13:49:05.694 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T13:49:05.694 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T13:49:05.694 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T13:49:05.694 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 24993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_synproxy.ko 2024-04-25T13:49:05.695 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tproxy.ko 2024-04-25T13:49:05.695 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 108393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/x_tables.ko 2024-04-25T13:49:05.695 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T13:49:05.695 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T13:49:05.695 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T13:49:05.695 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T13:49:05.695 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 21865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T13:49:05.695 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T13:49:05.695 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T13:49:05.695 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T13:49:05.695 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 45217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T13:49:05.695 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T13:49:05.695 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T13:49:05.695 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T13:49:05.696 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T13:49:05.696 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T13:49:05.696 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T13:49:05.696 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T13:49:05.696 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T13:49:05.696 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T13:49:05.696 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T13:49:05.696 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T13:49:05.696 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T13:49:05.696 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T13:49:05.696 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T13:49:05.696 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T13:49:05.696 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T13:49:05.696 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T13:49:05.697 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T13:49:05.697 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 8465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T13:49:05.697 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T13:49:05.697 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T13:49:05.697 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T13:49:05.697 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T13:49:05.697 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T13:49:05.697 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 8913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T13:49:05.697 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T13:49:05.697 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T13:49:05.697 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T13:49:05.697 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T13:49:05.697 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T13:49:05.697 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 52817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T13:49:05.697 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T13:49:05.698 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T13:49:05.698 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T13:49:05.698 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T13:49:05.698 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T13:49:05.698 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T13:49:05.698 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T13:49:05.698 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T13:49:05.698 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 8449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T13:49:05.698 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T13:49:05.698 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T13:49:05.698 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 16161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T13:49:05.698 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T13:49:05.698 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T13:49:05.698 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T13:49:05.699 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T13:49:05.699 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T13:49:05.699 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T13:49:05.699 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T13:49:05.699 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 11881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T13:49:05.699 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T13:49:05.699 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 43465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T13:49:05.699 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T13:49:05.699 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 36489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T13:49:05.699 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 19777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T13:49:05.699 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T13:49:05.699 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T13:49:05.699 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T13:49:05.699 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T13:49:05.700 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpudp.ko 2024-04-25T13:49:05.700 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T13:49:05.700 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T13:49:05.700 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/ 2024-04-25T13:49:05.700 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 17233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T13:49:05.700 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/ 2024-04-25T13:49:05.700 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 14073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T13:49:05.700 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/ 2024-04-25T13:49:05.700 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 488825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T13:49:05.700 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T13:49:05.700 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 10873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T13:49:05.700 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T13:49:05.700 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/ 2024-04-25T13:49:05.700 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T13:49:05.701 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/ 2024-04-25T13:49:05.701 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 142025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/phonet.ko 2024-04-25T13:49:05.701 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 68817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/pn_pep.ko 2024-04-25T13:49:05.701 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/ 2024-04-25T13:49:05.701 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 318617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T13:49:05.701 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 202305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T13:49:05.701 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 85105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T13:49:05.701 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rxrpc/ 2024-04-25T13:49:05.701 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 1027897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rxrpc/rxrpc.ko 2024-04-25T13:49:05.701 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/ 2024-04-25T13:49:05.701 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 39001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T13:49:05.701 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 28721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T13:49:05.701 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 45409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T13:49:05.701 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 27593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T13:49:05.702 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 53465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T13:49:05.702 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 38705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T13:49:05.702 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T13:49:05.702 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 43417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T13:49:05.702 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 45673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T13:49:05.702 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 32129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T13:49:05.702 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 41457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T13:49:05.702 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_cgroup.ko 2024-04-25T13:49:05.702 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 37305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T13:49:05.702 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 39601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T13:49:05.702 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 49321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T13:49:05.702 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 79145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T13:49:05.702 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 8713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T13:49:05.702 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 15385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T13:49:05.703 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 48785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T13:49:05.703 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T13:49:05.703 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T13:49:05.703 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 8377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T13:49:05.703 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T13:49:05.703 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 25097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T13:49:05.703 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 43481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T13:49:05.703 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 43153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T13:49:05.703 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 39177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq_codel.ko 2024-04-25T13:49:05.703 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 42889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T13:49:05.703 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 62817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T13:49:05.703 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 26585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T13:49:05.703 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 81833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T13:49:05.703 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 33545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T13:49:05.704 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 36625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T13:49:05.704 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T13:49:05.704 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 33065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T13:49:05.704 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 44793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T13:49:05.704 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 32833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T13:49:05.704 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 13505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T13:49:05.704 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 31801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T13:49:05.704 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 60249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T13:49:05.704 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 40889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T13:49:05.704 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 33433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T13:49:05.704 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 38329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T13:49:05.704 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 35777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T13:49:05.704 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 34161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T13:49:05.704 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/ 2024-04-25T13:49:05.704 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 1187313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T13:49:05.705 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T13:49:05.705 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/ 2024-04-25T13:49:05.705 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/ 2024-04-25T13:49:05.705 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 432409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T13:49:05.705 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 70113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T13:49:05.705 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 1940889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T13:49:05.705 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/ 2024-04-25T13:49:05.705 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 890729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T13:49:05.705 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/ 2024-04-25T13:49:05.705 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 1122337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T13:49:05.705 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T13:49:05.705 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/ 2024-04-25T13:49:05.705 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 18801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T13:49:05.705 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/ 2024-04-25T13:49:05.706 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 105609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T13:49:05.706 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 90265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T13:49:05.706 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 123769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T13:49:05.706 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 12905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T13:49:05.706 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T13:49:05.706 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/x25/ 2024-04-25T13:49:05.706 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 165841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/x25/x25.ko 2024-04-25T13:49:05.706 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/ 2024-04-25T13:49:05.706 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 26729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_algo.ko 2024-04-25T13:49:05.706 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T13:49:05.706 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 131865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_user.ko 2024-04-25T13:49:05.706 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 644769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T13:49:05.708 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 628542 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T13:49:05.709 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 7706 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T13:49:05.709 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 23609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T13:49:05.709 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 9941 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T13:49:05.709 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 70990 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T13:49:05.709 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 146959 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T13:49:05.709 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 232835 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T13:49:05.709 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 210 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T13:49:05.709 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 72602 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T13:49:05.709 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 879 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T13:49:05.709 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 258450 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T13:49:05.709 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 313019 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T13:49:05.709 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/ 2024-04-25T13:49:05.709 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/ 2024-04-25T13:49:05.710 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/doc/ 2024-04-25T13:49:05.710 INFO:teuthology.orchestra.run.smithi155.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/ 2024-04-25T13:49:05.710 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 172 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/changelog.Debian.gz 2024-04-25T13:49:05.710 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r-- root/root 691 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/copyright 2024-04-25T13:49:05.715 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T13:49:05.715 INFO:teuthology.task.kernel:searching for kernel 6.9.0-rc3-g806ec36b2112 2024-04-25T13:49:05.715 DEBUG:teuthology.orchestra.run.smithi155:> egrep '(submenu|menuentry.*6.9.0-rc3-g806ec36b2112).*{' /boot/grub/grub.cfg 2024-04-25T13:49:05.756 INFO:teuthology.orchestra.run.smithi155.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-25T13:49:05.757 INFO:teuthology.orchestra.run.smithi155.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g806ec36b2112-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-25T13:49:05.757 INFO:teuthology.orchestra.run.smithi155.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g806ec36b2112-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-25T13:49:05.757 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-04-25T13:49:05.757 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112 2024-04-25T13:49:05.757 DEBUG:teuthology.orchestra.run.smithi155:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-04-25T13:49:05.800 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi155.front.sepia.ceph.com to complete... 2024-04-25T13:49:06.158 INFO:teuthology.orchestra.run.smithi155.stderr:Sourcing file `/etc/default/grub' 2024-04-25T13:49:06.159 INFO:teuthology.orchestra.run.smithi155.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T13:49:06.189 INFO:teuthology.orchestra.run.smithi155.stderr:Generating grub configuration file ... 2024-04-25T13:49:06.400 INFO:teuthology.orchestra.run.smithi155.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T13:49:06.406 INFO:teuthology.orchestra.run.smithi155.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-25T13:49:06.669 INFO:teuthology.orchestra.run.smithi155.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T13:49:06.675 INFO:teuthology.orchestra.run.smithi155.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T13:49:06.718 INFO:teuthology.orchestra.run.smithi155.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T13:49:06.723 INFO:teuthology.orchestra.run.smithi155.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T13:49:06.864 INFO:teuthology.orchestra.run.smithi155.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T13:49:06.864 INFO:teuthology.orchestra.run.smithi155.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T13:49:06.864 INFO:teuthology.orchestra.run.smithi155.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T13:49:06.875 INFO:teuthology.orchestra.run.smithi155.stderr:done 2024-04-25T13:49:19.039 INFO:teuthology.orchestra.run.smithi086.stderr:Sourcing file `/etc/default/grub' 2024-04-25T13:49:19.056 INFO:teuthology.orchestra.run.smithi086.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T13:49:19.074 INFO:teuthology.orchestra.run.smithi086.stderr:Generating grub configuration file ... 2024-04-25T13:49:19.435 INFO:teuthology.orchestra.run.smithi086.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T13:49:19.452 INFO:teuthology.orchestra.run.smithi086.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-25T13:49:19.789 INFO:teuthology.orchestra.run.smithi086.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T13:49:19.803 INFO:teuthology.orchestra.run.smithi086.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T13:49:19.866 INFO:teuthology.orchestra.run.smithi086.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T13:49:19.871 INFO:teuthology.orchestra.run.smithi086.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T13:49:20.032 INFO:teuthology.orchestra.run.smithi086.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T13:49:20.033 INFO:teuthology.orchestra.run.smithi086.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T13:49:20.033 INFO:teuthology.orchestra.run.smithi086.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T13:49:20.080 INFO:teuthology.orchestra.run.smithi086.stderr:done 2024-04-25T13:49:20.263 DEBUG:teuthology.orchestra.run.smithi086:> dpkg-deb -c /tmp/linux-image.deb 2024-04-25T13:49:20.270 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./ 2024-04-25T13:49:20.270 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./boot/ 2024-04-25T13:49:20.270 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7002012 2024-04-19 08:55 ./boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T13:49:20.287 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 172073 2024-04-19 08:55 ./boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T13:49:20.288 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13120512 2024-04-19 08:55 ./boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T13:49:20.306 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/ 2024-04-25T13:49:20.306 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/ 2024-04-25T13:49:20.306 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/postinst.d/ 2024-04-25T13:49:20.306 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/postrm.d/ 2024-04-25T13:49:20.306 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/preinst.d/ 2024-04-25T13:49:20.306 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/prerm.d/ 2024-04-25T13:49:20.306 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/ 2024-04-25T13:49:20.306 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/ 2024-04-25T13:49:20.306 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/ 2024-04-25T13:49:20.307 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/ 2024-04-25T13:49:20.307 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/ 2024-04-25T13:49:20.307 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/ 2024-04-25T13:49:20.307 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ 2024-04-25T13:49:20.307 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 507569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/aesni-intel.ko 2024-04-25T13:49:20.310 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T13:49:20.310 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T13:49:20.310 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T13:49:20.310 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 97953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T13:49:20.310 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 109905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T13:49:20.311 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 155249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T13:49:20.311 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T13:49:20.311 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T13:49:20.311 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 87009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T13:49:20.311 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T13:49:20.311 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T13:49:20.311 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T13:49:20.311 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 54513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T13:49:20.311 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T13:49:20.311 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 48049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T13:49:20.311 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 62873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T13:49:20.311 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T13:49:20.312 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T13:49:20.312 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T13:49:20.312 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/ 2024-04-25T13:49:20.312 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/ 2024-04-25T13:49:20.312 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/ 2024-04-25T13:49:20.312 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T13:49:20.312 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpuid.ko 2024-04-25T13:49:20.312 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/msr.ko 2024-04-25T13:49:20.312 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/ 2024-04-25T13:49:20.312 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 444425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T13:49:20.312 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1035881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T13:49:20.313 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 2849609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T13:49:20.394 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ 2024-04-25T13:49:20.394 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 65025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/af_alg.ko 2024-04-25T13:49:20.394 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/algif_hash.ko 2024-04-25T13:49:20.394 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/algif_skcipher.ko 2024-04-25T13:49:20.395 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T13:49:20.395 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T13:49:20.395 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T13:49:20.395 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/asymmetric_keys/ 2024-04-25T13:49:20.395 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-04-25T13:49:20.395 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/ 2024-04-25T13:49:20.395 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T13:49:20.395 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T13:49:20.395 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T13:49:20.395 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T13:49:20.395 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T13:49:20.396 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T13:49:20.396 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T13:49:20.396 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T13:49:20.396 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T13:49:20.396 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T13:49:20.396 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T13:49:20.396 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T13:49:20.396 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T13:49:20.396 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T13:49:20.396 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T13:49:20.396 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T13:49:20.397 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T13:49:20.397 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T13:49:20.397 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cryptd.ko 2024-04-25T13:49:20.397 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 48113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T13:49:20.397 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_simd.ko 2024-04-25T13:49:20.397 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T13:49:20.397 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T13:49:20.397 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T13:49:20.397 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T13:49:20.397 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T13:49:20.397 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T13:49:20.397 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T13:49:20.398 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T13:49:20.398 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lrw.ko 2024-04-25T13:49:20.398 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T13:49:20.398 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T13:49:20.398 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T13:49:20.398 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T13:49:20.398 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T13:49:20.398 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T13:49:20.398 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T13:49:20.398 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T13:49:20.399 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T13:49:20.399 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 169065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T13:49:20.399 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T13:49:20.399 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T13:49:20.399 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T13:49:20.399 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T13:49:20.399 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T13:49:20.399 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T13:49:20.399 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T13:49:20.399 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xts.ko 2024-04-25T13:49:20.399 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T13:49:20.399 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ 2024-04-25T13:49:20.399 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ 2024-04-25T13:49:20.400 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_extlog.ko 2024-04-25T13:49:20.400 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T13:49:20.400 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T13:49:20.400 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/apei/ 2024-04-25T13:49:20.400 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/apei/einj.ko 2024-04-25T13:49:20.400 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T13:49:20.400 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T13:49:20.400 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T13:49:20.400 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T13:49:20.400 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 119033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T13:49:20.400 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ 2024-04-25T13:49:20.401 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T13:49:20.401 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 96825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci.ko 2024-04-25T13:49:20.401 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T13:49:20.401 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 107737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci.ko 2024-04-25T13:49:20.401 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T13:49:20.401 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T13:49:20.401 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T13:49:20.401 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T13:49:20.401 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T13:49:20.401 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T13:49:20.401 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T13:49:20.402 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T13:49:20.402 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T13:49:20.402 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T13:49:20.402 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T13:49:20.402 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T13:49:20.402 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T13:49:20.402 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T13:49:20.402 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T13:49:20.402 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T13:49:20.402 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T13:49:20.402 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T13:49:20.403 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 60257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_legacy.ko 2024-04-25T13:49:20.403 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T13:49:20.403 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T13:49:20.403 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T13:49:20.403 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T13:49:20.403 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T13:49:20.403 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T13:49:20.403 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T13:49:20.403 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T13:49:20.403 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T13:49:20.403 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T13:49:20.403 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T13:49:20.404 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T13:49:20.404 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T13:49:20.404 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_radisys.ko 2024-04-25T13:49:20.404 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T13:49:20.404 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rz1000.ko 2024-04-25T13:49:20.404 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T13:49:20.404 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T13:49:20.404 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T13:49:20.404 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T13:49:20.404 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T13:49:20.405 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T13:49:20.405 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T13:49:20.405 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T13:49:20.405 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 102121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T13:49:20.405 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 85409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T13:49:20.405 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T13:49:20.405 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T13:49:20.405 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T13:49:20.405 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T13:49:20.405 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T13:49:20.405 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T13:49:20.405 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T13:49:20.406 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T13:49:20.406 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T13:49:20.406 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T13:49:20.406 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ 2024-04-25T13:49:20.406 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T13:49:20.406 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/ 2024-04-25T13:49:20.406 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 163969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T13:49:20.406 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/ 2024-04-25T13:49:20.406 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/ 2024-04-25T13:49:20.406 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 140865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T13:49:20.406 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/ 2024-04-25T13:49:20.407 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1241657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T13:49:20.407 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 203289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T13:49:20.407 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/ 2024-04-25T13:49:20.407 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 115265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T13:49:20.407 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 143433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T13:49:20.407 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/ 2024-04-25T13:49:20.407 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 126617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T13:49:20.407 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 119873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T13:49:20.407 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 313217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T13:49:20.407 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/ 2024-04-25T13:49:20.407 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 117233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T13:49:20.408 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/ 2024-04-25T13:49:20.408 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 70385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T13:49:20.408 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ 2024-04-25T13:49:20.408 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/applicom.ko 2024-04-25T13:49:20.408 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T13:49:20.408 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/ 2024-04-25T13:49:20.408 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T13:49:20.408 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T13:49:20.408 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T13:49:20.408 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T13:49:20.408 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T13:49:20.408 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ 2024-04-25T13:49:20.409 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T13:49:20.409 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 186921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T13:49:20.409 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T13:49:20.409 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 179561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T13:49:20.409 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 85441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T13:49:20.409 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 65313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T13:49:20.409 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T13:49:20.409 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/ 2024-04-25T13:49:20.409 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 103409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T13:49:20.409 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/nvram.ko 2024-04-25T13:49:20.409 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T13:49:20.409 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T13:49:20.410 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/ 2024-04-25T13:49:20.410 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T13:49:20.410 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-04-25T13:49:20.410 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-04-25T13:49:20.410 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-04-25T13:49:20.410 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T13:49:20.410 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T13:49:20.410 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-04-25T13:49:20.410 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/ 2024-04-25T13:49:20.410 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_class.ko 2024-04-25T13:49:20.410 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 62545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_core.ko 2024-04-25T13:49:20.411 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-04-25T13:49:20.411 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/ 2024-04-25T13:49:20.411 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-palmas.ko 2024-04-25T13:49:20.411 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 57289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-si5351.ko 2024-04-25T13:49:20.411 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-twl6040.ko 2024-04-25T13:49:20.411 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-wm831x.ko 2024-04-25T13:49:20.411 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/ 2024-04-25T13:49:20.411 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T13:49:20.411 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T13:49:20.411 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T13:49:20.411 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ 2024-04-25T13:49:20.412 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ 2024-04-25T13:49:20.412 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 89257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T13:49:20.412 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 333657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T13:49:20.412 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/ 2024-04-25T13:49:20.412 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/ 2024-04-25T13:49:20.412 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/ 2024-04-25T13:49:20.412 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 751713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T13:49:20.412 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-04-25T13:49:20.412 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T13:49:20.412 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T13:49:20.412 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T13:49:20.413 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/ 2024-04-25T13:49:20.413 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 90441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T13:49:20.413 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/ 2024-04-25T13:49:20.413 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T13:49:20.413 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ 2024-04-25T13:49:20.413 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/ 2024-04-25T13:49:20.413 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T13:49:20.413 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ 2024-04-25T13:49:20.413 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 146769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T13:49:20.413 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ 2024-04-25T13:49:20.413 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 96145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T13:49:20.413 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T13:49:20.414 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 51105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T13:49:20.414 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T13:49:20.414 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T13:49:20.414 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T13:49:20.414 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T13:49:20.414 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T13:49:20.414 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 64161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T13:49:20.414 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T13:49:20.414 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T13:49:20.414 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 68697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T13:49:20.414 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T13:49:20.415 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/ 2024-04-25T13:49:20.415 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-gpio.ko 2024-04-25T13:49:20.415 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max14577.ko 2024-04-25T13:49:20.415 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max77693.ko 2024-04-25T13:49:20.415 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max8997.ko 2024-04-25T13:49:20.415 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-palmas.ko 2024-04-25T13:49:20.415 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-rt8973a.ko 2024-04-25T13:49:20.415 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-sm5502.ko 2024-04-25T13:49:20.415 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/ 2024-04-25T13:49:20.415 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/dmi-sysfs.ko 2024-04-25T13:49:20.415 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/efi/ 2024-04-25T13:49:20.416 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/efi/efi-pstore.ko 2024-04-25T13:49:20.416 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T13:49:20.416 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/ 2024-04-25T13:49:20.416 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-adp5520.ko 2024-04-25T13:49:20.416 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-amd8111.ko 2024-04-25T13:49:20.416 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-arizona.ko 2024-04-25T13:49:20.416 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-da9052.ko 2024-04-25T13:49:20.416 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-da9055.ko 2024-04-25T13:49:20.416 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-dln2.ko 2024-04-25T13:49:20.416 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 81073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-f7188x.ko 2024-04-25T13:49:20.416 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-generic.ko 2024-04-25T13:49:20.416 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T13:49:20.417 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-04-25T13:49:20.417 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-kempld.ko 2024-04-25T13:49:20.417 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-lp3943.ko 2024-04-25T13:49:20.417 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max7300.ko 2024-04-25T13:49:20.417 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max730x.ko 2024-04-25T13:49:20.417 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max732x.ko 2024-04-25T13:49:20.417 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-04-25T13:49:20.417 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-pca953x.ko 2024-04-25T13:49:20.417 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-pcf857x.ko 2024-04-25T13:49:20.417 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-rdc321x.ko 2024-04-25T13:49:20.417 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-sch.ko 2024-04-25T13:49:20.417 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-sch311x.ko 2024-04-25T13:49:20.417 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-tps65912.ko 2024-04-25T13:49:20.418 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-twl4030.ko 2024-04-25T13:49:20.418 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-twl6040.ko 2024-04-25T13:49:20.418 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T13:49:20.418 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-vx855.ko 2024-04-25T13:49:20.418 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm831x.ko 2024-04-25T13:49:20.418 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm8350.ko 2024-04-25T13:49:20.418 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm8994.ko 2024-04-25T13:49:20.418 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/ 2024-04-25T13:49:20.418 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-a4tech.ko 2024-04-25T13:49:20.418 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-apple.ko 2024-04-25T13:49:20.418 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T13:49:20.419 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T13:49:20.419 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T13:49:20.419 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-belkin.ko 2024-04-25T13:49:20.419 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cherry.ko 2024-04-25T13:49:20.419 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-chicony.ko 2024-04-25T13:49:20.419 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 74393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cp2112.ko 2024-04-25T13:49:20.419 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cypress.ko 2024-04-25T13:49:20.419 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T13:49:20.419 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T13:49:20.419 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T13:49:20.419 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T13:49:20.419 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ezkey.ko 2024-04-25T13:49:20.420 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T13:49:20.420 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-generic.ko 2024-04-25T13:49:20.420 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T13:49:20.420 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T13:49:20.420 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T13:49:20.420 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T13:49:20.420 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T13:49:20.420 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T13:49:20.420 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T13:49:20.420 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kensington.ko 2024-04-25T13:49:20.420 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T13:49:20.421 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T13:49:20.421 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T13:49:20.421 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T13:49:20.421 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 54217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T13:49:20.421 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lg-g15.ko 2024-04-25T13:49:20.421 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 68081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T13:49:20.421 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 121369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T13:49:20.421 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 97953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech.ko 2024-04-25T13:49:20.421 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-magicmouse.ko 2024-04-25T13:49:20.421 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-microsoft.ko 2024-04-25T13:49:20.421 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-monterey.ko 2024-04-25T13:49:20.421 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T13:49:20.422 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ntrig.ko 2024-04-25T13:49:20.422 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T13:49:20.422 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T13:49:20.422 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T13:49:20.422 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 91241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T13:49:20.422 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T13:49:20.422 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T13:49:20.422 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T13:49:20.422 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T13:49:20.422 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T13:49:20.422 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T13:49:20.423 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T13:49:20.423 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T13:49:20.423 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T13:49:20.423 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T13:49:20.423 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T13:49:20.423 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T13:49:20.423 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T13:49:20.423 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T13:49:20.423 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T13:49:20.423 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T13:49:20.423 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T13:49:20.423 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T13:49:20.424 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T13:49:20.424 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T13:49:20.424 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 62129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T13:49:20.424 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T13:49:20.424 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T13:49:20.424 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T13:49:20.424 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T13:49:20.424 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T13:49:20.424 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T13:49:20.424 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T13:49:20.424 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T13:49:20.425 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 81113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T13:49:20.425 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T13:49:20.425 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 184745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T13:49:20.425 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T13:49:20.425 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T13:49:20.425 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T13:49:20.425 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 319921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid.ko 2024-04-25T13:49:20.425 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T13:49:20.425 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/ 2024-04-25T13:49:20.425 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 153633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbhid.ko 2024-04-25T13:49:20.425 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbkbd.ko 2024-04-25T13:49:20.425 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbmouse.ko 2024-04-25T13:49:20.426 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 261945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T13:49:20.426 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/ 2024-04-25T13:49:20.426 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 75065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T13:49:20.426 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 110681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T13:49:20.426 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 382033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T13:49:20.426 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ 2024-04-25T13:49:20.426 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 77153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T13:49:20.426 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 67009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T13:49:20.426 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T13:49:20.426 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T13:49:20.426 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T13:49:20.426 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T13:49:20.427 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T13:49:20.427 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 115057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T13:49:20.427 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T13:49:20.427 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T13:49:20.427 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T13:49:20.427 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T13:49:20.427 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T13:49:20.427 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T13:49:20.427 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 89025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T13:49:20.427 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T13:49:20.427 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 81473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T13:49:20.428 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T13:49:20.428 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 48177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T13:49:20.428 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 68865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T13:49:20.428 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 60193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T13:49:20.428 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 69009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T13:49:20.428 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T13:49:20.428 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T13:49:20.428 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T13:49:20.428 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/da9052-hwmon.ko 2024-04-25T13:49:20.428 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/da9055-hwmon.ko 2024-04-25T13:49:20.428 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T13:49:20.429 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 113201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T13:49:20.429 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T13:49:20.429 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T13:49:20.429 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T13:49:20.429 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc2103.ko 2024-04-25T13:49:20.429 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T13:49:20.429 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 78921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T13:49:20.429 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 108961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T13:49:20.429 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T13:49:20.429 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T13:49:20.429 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 65769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T13:49:20.429 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T13:49:20.430 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T13:49:20.430 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T13:49:20.430 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 56313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T13:49:20.430 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hih6130.ko 2024-04-25T13:49:20.430 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T13:49:20.430 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T13:49:20.430 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T13:49:20.430 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T13:49:20.430 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T13:49:20.430 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T13:49:20.430 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T13:49:20.431 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 156465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T13:49:20.431 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/jc42.ko 2024-04-25T13:49:20.431 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T13:49:20.431 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T13:49:20.431 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T13:49:20.431 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 57769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T13:49:20.431 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T13:49:20.431 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T13:49:20.431 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T13:49:20.431 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T13:49:20.431 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T13:49:20.431 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T13:49:20.432 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 86497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T13:49:20.432 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T13:49:20.432 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T13:49:20.432 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T13:49:20.432 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 131505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T13:49:20.432 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T13:49:20.432 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T13:49:20.432 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T13:49:20.432 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T13:49:20.432 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T13:49:20.432 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T13:49:20.432 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T13:49:20.433 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T13:49:20.433 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T13:49:20.433 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T13:49:20.433 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T13:49:20.433 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T13:49:20.433 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T13:49:20.433 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T13:49:20.433 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T13:49:20.433 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T13:49:20.433 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T13:49:20.433 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mc13783-adc.ko 2024-04-25T13:49:20.433 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T13:49:20.434 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-04-25T13:49:20.434 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 56505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T13:49:20.434 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 141353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T13:49:20.434 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 72737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T13:49:20.434 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 67841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T13:49:20.434 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 97777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T13:49:20.434 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T13:49:20.434 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T13:49:20.434 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ 2024-04-25T13:49:20.434 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T13:49:20.434 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T13:49:20.435 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T13:49:20.435 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T13:49:20.435 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T13:49:20.435 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T13:49:20.435 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T13:49:20.435 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 101233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T13:49:20.435 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T13:49:20.435 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T13:49:20.435 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T13:49:20.435 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T13:49:20.435 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T13:49:20.435 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T13:49:20.436 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T13:49:20.436 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T13:49:20.436 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T13:49:20.436 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T13:49:20.436 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T13:49:20.436 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 48713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T13:49:20.436 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T13:49:20.436 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T13:49:20.436 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T13:49:20.436 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T13:49:20.436 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T13:49:20.436 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T13:49:20.437 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T13:49:20.437 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T13:49:20.437 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T13:49:20.437 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T13:49:20.437 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T13:49:20.437 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 62737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T13:49:20.437 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 71169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T13:49:20.437 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 96577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T13:49:20.437 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 85913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T13:49:20.437 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 75129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T13:49:20.437 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 83425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T13:49:20.438 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 116777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T13:49:20.438 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 134441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T13:49:20.438 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T13:49:20.438 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T13:49:20.438 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-04-25T13:49:20.438 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-04-25T13:49:20.438 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/ 2024-04-25T13:49:20.438 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/ 2024-04-25T13:49:20.438 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T13:49:20.438 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T13:49:20.438 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/ 2024-04-25T13:49:20.439 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-04-25T13:49:20.439 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-04-25T13:49:20.439 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-04-25T13:49:20.439 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T13:49:20.439 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T13:49:20.439 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T13:49:20.439 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-04-25T13:49:20.439 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T13:49:20.439 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-04-25T13:49:20.439 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 64233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T13:49:20.439 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T13:49:20.439 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T13:49:20.440 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T13:49:20.440 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-04-25T13:49:20.440 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-04-25T13:49:20.440 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 76969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T13:49:20.440 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T13:49:20.440 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 52937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T13:49:20.440 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-04-25T13:49:20.440 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T13:49:20.440 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T13:49:20.440 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-04-25T13:49:20.440 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T13:49:20.440 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T13:49:20.441 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 57217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T13:49:20.441 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-04-25T13:49:20.441 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T13:49:20.441 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T13:49:20.441 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-04-25T13:49:20.441 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-04-25T13:49:20.441 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T13:49:20.441 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-04-25T13:49:20.441 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T13:49:20.441 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T13:49:20.441 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T13:49:20.441 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T13:49:20.441 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-04-25T13:49:20.442 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T13:49:20.442 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T13:49:20.442 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T13:49:20.442 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/ 2024-04-25T13:49:20.442 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-04-25T13:49:20.442 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-04-25T13:49:20.442 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-04-25T13:49:20.442 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ 2024-04-25T13:49:20.442 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ 2024-04-25T13:49:20.442 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 329697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T13:49:20.443 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1119913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T13:49:20.443 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 83905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T13:49:20.443 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 408433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T13:49:20.443 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 120921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T13:49:20.443 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 347217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T13:49:20.443 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 84945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T13:49:20.443 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ 2024-04-25T13:49:20.443 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/ 2024-04-25T13:49:20.443 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 590929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T13:49:20.443 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/ 2024-04-25T13:49:20.443 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 514641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T13:49:20.444 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/ 2024-04-25T13:49:20.444 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 911665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T13:49:20.444 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ 2024-04-25T13:49:20.444 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 375545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T13:49:20.444 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ocrdma/ 2024-04-25T13:49:20.444 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 197121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-04-25T13:49:20.444 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/usnic/ 2024-04-25T13:49:20.444 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 212553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-04-25T13:49:20.444 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ 2024-04-25T13:49:20.444 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ 2024-04-25T13:49:20.444 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 334401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T13:49:20.444 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ 2024-04-25T13:49:20.445 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 120673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T13:49:20.445 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ 2024-04-25T13:49:20.445 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 132585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T13:49:20.445 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ 2024-04-25T13:49:20.445 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 157937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T13:49:20.445 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ 2024-04-25T13:49:20.445 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 156673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T13:49:20.445 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/ 2024-04-25T13:49:20.445 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/evbug.ko 2024-04-25T13:49:20.445 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/ff-memless.ko 2024-04-25T13:49:20.445 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/ 2024-04-25T13:49:20.446 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T13:49:20.446 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T13:49:20.446 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T13:49:20.446 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T13:49:20.446 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/ 2024-04-25T13:49:20.446 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-04-25T13:49:20.446 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-04-25T13:49:20.446 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-04-25T13:49:20.446 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-04-25T13:49:20.446 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T13:49:20.446 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-04-25T13:49:20.446 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lkkbd.ko 2024-04-25T13:49:20.447 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lm8323.ko 2024-04-25T13:49:20.447 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lm8333.ko 2024-04-25T13:49:20.447 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-04-25T13:49:20.447 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-04-25T13:49:20.447 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-04-25T13:49:20.447 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-04-25T13:49:20.447 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/newtonkbd.ko 2024-04-25T13:49:20.447 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-04-25T13:49:20.447 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/qt1070.ko 2024-04-25T13:49:20.447 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/qt2160.ko 2024-04-25T13:49:20.447 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-04-25T13:49:20.447 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/stowaway.ko 2024-04-25T13:49:20.448 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/sunkbd.ko 2024-04-25T13:49:20.448 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-04-25T13:49:20.448 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-04-25T13:49:20.448 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-04-25T13:49:20.448 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/xtkbd.ko 2024-04-25T13:49:20.448 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/matrix-keymap.ko 2024-04-25T13:49:20.448 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/ 2024-04-25T13:49:20.448 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T13:49:20.448 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T13:49:20.448 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 160017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T13:49:20.448 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 114273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T13:49:20.449 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/gpio_mouse.ko 2024-04-25T13:49:20.449 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 428185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/psmouse.ko 2024-04-25T13:49:20.449 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T13:49:20.449 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T13:49:20.449 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T13:49:20.449 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T13:49:20.449 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/ 2024-04-25T13:49:20.449 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 140969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T13:49:20.449 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ 2024-04-25T13:49:20.449 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T13:49:20.449 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T13:49:20.450 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ct82c710.ko 2024-04-25T13:49:20.450 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T13:49:20.450 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/parkbd.ko 2024-04-25T13:49:20.450 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/pcips2.ko 2024-04-25T13:49:20.450 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ps2mult.ko 2024-04-25T13:49:20.450 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T13:49:20.450 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serport.ko 2024-04-25T13:49:20.450 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T13:49:20.450 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/ 2024-04-25T13:49:20.450 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-88pm860x.ko 2024-04-25T13:49:20.450 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-adp5520.ko 2024-04-25T13:49:20.451 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 77265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-bd2802.ko 2024-04-25T13:49:20.451 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T13:49:20.451 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-da903x.ko 2024-04-25T13:49:20.451 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-da9052.ko 2024-04-25T13:49:20.451 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-gpio.ko 2024-04-25T13:49:20.451 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T13:49:20.451 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3533.ko 2024-04-25T13:49:20.451 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm355x.ko 2024-04-25T13:49:20.451 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3642.ko 2024-04-25T13:49:20.451 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T13:49:20.452 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp8788.ko 2024-04-25T13:49:20.452 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-max8997.ko 2024-04-25T13:49:20.452 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-mc13783.ko 2024-04-25T13:49:20.452 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-menf21bmc.ko 2024-04-25T13:49:20.452 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca9532.ko 2024-04-25T13:49:20.452 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca955x.ko 2024-04-25T13:49:20.452 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca963x.ko 2024-04-25T13:49:20.452 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T13:49:20.452 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-tca6507.ko 2024-04-25T13:49:20.452 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-wm831x-status.ko 2024-04-25T13:49:20.452 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-wm8350.ko 2024-04-25T13:49:20.452 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ 2024-04-25T13:49:20.453 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T13:49:20.453 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T13:49:20.453 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T13:49:20.453 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T13:49:20.453 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T13:49:20.453 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T13:49:20.453 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T13:49:20.453 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T13:49:20.453 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T13:49:20.453 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/ 2024-04-25T13:49:20.453 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/ 2024-04-25T13:49:20.454 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 790753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T13:49:20.454 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T13:49:20.454 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 116825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bufio.ko 2024-04-25T13:49:20.454 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 58281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T13:49:20.454 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 199041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T13:49:20.454 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 134945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T13:49:20.454 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T13:49:20.454 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 67497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-era.ko 2024-04-25T13:49:20.454 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T13:49:20.454 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T13:49:20.454 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log.ko 2024-04-25T13:49:20.455 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 60977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-mirror.ko 2024-04-25T13:49:20.455 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 97209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T13:49:20.455 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T13:49:20.455 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 85689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T13:49:20.455 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-region-hash.ko 2024-04-25T13:49:20.455 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T13:49:20.455 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T13:49:20.455 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 139817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-snapshot.ko 2024-04-25T13:49:20.455 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T13:49:20.455 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 234457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T13:49:20.455 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 66425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T13:49:20.456 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-zero.ko 2024-04-25T13:49:20.456 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/ 2024-04-25T13:49:20.456 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 264329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T13:49:20.456 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T13:49:20.456 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 125009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T13:49:20.456 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 157681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T13:49:20.456 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 444665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T13:49:20.456 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/ 2024-04-25T13:49:20.456 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/ 2024-04-25T13:49:20.456 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 214841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T13:49:20.456 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 77985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T13:49:20.457 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 54785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T13:49:20.457 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T13:49:20.457 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 150761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T13:49:20.457 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 97281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T13:49:20.457 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 63873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T13:49:20.457 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/ 2024-04-25T13:49:20.457 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm800.ko 2024-04-25T13:49:20.457 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm805.ko 2024-04-25T13:49:20.457 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm80x.ko 2024-04-25T13:49:20.457 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/arizona-i2c.ko 2024-04-25T13:49:20.457 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 148633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/arizona.ko 2024-04-25T13:49:20.458 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/bcm590xx.ko 2024-04-25T13:49:20.458 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/cros_ec_dev.ko 2024-04-25T13:49:20.458 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/dln2.ko 2024-04-25T13:49:20.458 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/janz-cmodio.ko 2024-04-25T13:49:20.458 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/kempld-core.ko 2024-04-25T13:49:20.458 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lm3533-core.ko 2024-04-25T13:49:20.458 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-04-25T13:49:20.458 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lp3943.ko 2024-04-25T13:49:20.458 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 60497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T13:49:20.458 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T13:49:20.458 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/max8907.ko 2024-04-25T13:49:20.458 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mc13xxx-core.ko 2024-04-25T13:49:20.459 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-04-25T13:49:20.459 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/menf21bmc.ko 2024-04-25T13:49:20.459 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633-adc.ko 2024-04-25T13:49:20.459 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633-gpio.ko 2024-04-25T13:49:20.459 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633.ko 2024-04-25T13:49:20.459 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-04-25T13:49:20.459 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/retu-mfd.ko 2024-04-25T13:49:20.459 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 99425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/si476x-core.ko 2024-04-25T13:49:20.459 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 58057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T13:49:20.459 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 60681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/tps65010.ko 2024-04-25T13:49:20.459 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/tps6507x.ko 2024-04-25T13:49:20.460 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T13:49:20.460 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T13:49:20.460 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T13:49:20.460 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 52361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wm8994.ko 2024-04-25T13:49:20.460 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ 2024-04-25T13:49:20.460 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-04-25T13:49:20.460 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 64265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ad525x_dpot.ko 2024-04-25T13:49:20.460 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/ 2024-04-25T13:49:20.460 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 57521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T13:49:20.460 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T13:49:20.460 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T13:49:20.461 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 58633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T13:49:20.461 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/ 2024-04-25T13:49:20.461 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-04-25T13:49:20.461 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/core.ko 2024-04-25T13:49:20.461 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/ 2024-04-25T13:49:20.461 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T13:49:20.461 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ds1682.ko 2024-04-25T13:49:20.461 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/dummy-irq.ko 2024-04-25T13:49:20.461 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/echo/ 2024-04-25T13:49:20.461 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/echo/echo.ko 2024-04-25T13:49:20.461 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/ 2024-04-25T13:49:20.461 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T13:49:20.462 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T13:49:20.462 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T13:49:20.462 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T13:49:20.462 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/genwqe/ 2024-04-25T13:49:20.462 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 191649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-04-25T13:49:20.462 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hmc6352.ko 2024-04-25T13:49:20.462 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T13:49:20.462 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ibmasm/ 2024-04-25T13:49:20.462 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 92345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-04-25T13:49:20.462 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ics932s401.ko 2024-04-25T13:49:20.462 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T13:49:20.463 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T13:49:20.463 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/ 2024-04-25T13:49:20.463 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 54865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T13:49:20.463 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T13:49:20.463 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/ 2024-04-25T13:49:20.463 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 154169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T13:49:20.463 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 69809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T13:49:20.463 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 389033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T13:49:20.463 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/phantom.ko 2024-04-25T13:49:20.463 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ti-st/ 2024-04-25T13:49:20.463 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 91145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ti-st/st_drv.ko 2024-04-25T13:49:20.464 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T13:49:20.464 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T13:49:20.464 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T13:49:20.464 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T13:49:20.464 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/ 2024-04-25T13:49:20.464 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 234905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T13:49:20.464 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ 2024-04-25T13:49:20.464 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/ 2024-04-25T13:49:20.464 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 620961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T13:49:20.464 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/ 2024-04-25T13:49:20.464 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/ 2024-04-25T13:49:20.464 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 139817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T13:49:20.465 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T13:49:20.465 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T13:49:20.465 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/ 2024-04-25T13:49:20.465 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 587065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T13:49:20.465 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T13:49:20.465 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T13:49:20.465 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ 2024-04-25T13:49:20.465 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/ 2024-04-25T13:49:20.465 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T13:49:20.465 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T13:49:20.465 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 158649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T13:49:20.465 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 85953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T13:49:20.466 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ 2024-04-25T13:49:20.466 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T13:49:20.466 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 73865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T13:49:20.466 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T13:49:20.466 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 109521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T13:49:20.466 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/ 2024-04-25T13:49:20.466 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 64705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T13:49:20.466 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/ 2024-04-25T13:49:20.466 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 84329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T13:49:20.466 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/ 2024-04-25T13:49:20.466 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 70737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T13:49:20.467 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/ 2024-04-25T13:49:20.467 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 52705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T13:49:20.467 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/ 2024-04-25T13:49:20.467 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 65857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T13:49:20.467 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 56057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T13:49:20.467 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 115881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T13:49:20.467 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/ 2024-04-25T13:49:20.467 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/ 2024-04-25T13:49:20.467 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 124497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T13:49:20.467 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-04-25T13:49:20.467 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 128409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T13:49:20.468 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-04-25T13:49:20.468 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 110345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T13:49:20.468 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/ 2024-04-25T13:49:20.468 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 101857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T13:49:20.468 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 88953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T13:49:20.468 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/ 2024-04-25T13:49:20.468 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 105633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T13:49:20.468 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 228985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T13:49:20.468 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-04-25T13:49:20.468 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1733353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T13:49:20.468 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 167185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T13:49:20.468 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/ 2024-04-25T13:49:20.469 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 161417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T13:49:20.469 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 440625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T13:49:20.469 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/ 2024-04-25T13:49:20.469 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/ 2024-04-25T13:49:20.469 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 432201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T13:49:20.469 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/ 2024-04-25T13:49:20.469 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-04-25T13:49:20.469 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 257201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T13:49:20.469 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-04-25T13:49:20.469 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 446177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T13:49:20.469 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-04-25T13:49:20.469 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1311433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T13:49:20.566 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-04-25T13:49:20.566 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 206689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T13:49:20.566 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-04-25T13:49:20.567 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T13:49:20.567 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/ 2024-04-25T13:49:20.567 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/ 2024-04-25T13:49:20.567 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 269313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T13:49:20.567 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/ 2024-04-25T13:49:20.567 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/ 2024-04-25T13:49:20.567 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 85385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T13:49:20.567 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 77457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T13:49:20.567 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 169089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T13:49:20.567 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 65913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T13:49:20.567 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 76209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T13:49:20.568 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T13:49:20.568 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/ 2024-04-25T13:49:20.568 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 75945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T13:49:20.568 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 78705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T13:49:20.568 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 52313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T13:49:20.568 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ec_bhf.ko 2024-04-25T13:49:20.568 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/ 2024-04-25T13:49:20.568 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/ 2024-04-25T13:49:20.568 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 356377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T13:49:20.568 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T13:49:20.568 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 63185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T13:49:20.569 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fujitsu/ 2024-04-25T13:49:20.569 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 51513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-04-25T13:49:20.569 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ 2024-04-25T13:49:20.569 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 115569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T13:49:20.569 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/ 2024-04-25T13:49:20.569 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 366217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T13:49:20.569 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/ 2024-04-25T13:49:20.569 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 755793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T13:49:20.569 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/ 2024-04-25T13:49:20.569 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 324577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T13:49:20.569 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/ 2024-04-25T13:49:20.569 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1196297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T13:49:20.570 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/ 2024-04-25T13:49:20.570 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 362593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T13:49:20.570 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/ 2024-04-25T13:49:20.570 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 671801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T13:49:20.570 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/ 2024-04-25T13:49:20.570 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 121537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T13:49:20.570 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-04-25T13:49:20.570 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 966705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T13:49:20.570 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-04-25T13:49:20.570 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 193585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T13:49:20.570 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 108633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T13:49:20.571 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/ 2024-04-25T13:49:20.571 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T13:49:20.571 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 126881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T13:49:20.571 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 150777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T13:49:20.571 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/ 2024-04-25T13:49:20.571 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-04-25T13:49:20.571 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 905033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T13:49:20.571 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 344145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T13:49:20.571 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-04-25T13:49:20.571 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-04-25T13:49:20.571 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1412417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T13:49:20.571 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ 2024-04-25T13:49:20.572 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 65913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-04-25T13:49:20.572 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 54497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-04-25T13:49:20.572 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-04-25T13:49:20.572 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 117353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T13:49:20.572 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/ 2024-04-25T13:49:20.572 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-04-25T13:49:20.572 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 131425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T13:49:20.572 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ 2024-04-25T13:49:20.572 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 91465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T13:49:20.572 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 56689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T13:49:20.572 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/ 2024-04-25T13:49:20.572 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 181081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T13:49:20.573 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/ 2024-04-25T13:49:20.573 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 171417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T13:49:20.573 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/ 2024-04-25T13:49:20.573 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 58249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T13:49:20.573 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T13:49:20.573 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/ 2024-04-25T13:49:20.573 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-04-25T13:49:20.573 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 300817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T13:49:20.573 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 105689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T13:49:20.573 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-04-25T13:49:20.573 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 885129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T13:49:20.573 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/ 2024-04-25T13:49:20.574 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 52305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T13:49:20.574 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/ 2024-04-25T13:49:20.574 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 81641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T13:49:20.574 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 100689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T13:49:20.574 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T13:49:20.574 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 264161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T13:49:20.574 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/ 2024-04-25T13:49:20.574 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-04-25T13:49:20.574 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 142449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-04-25T13:49:20.574 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/ 2024-04-25T13:49:20.575 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1350385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T13:49:20.575 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/ 2024-04-25T13:49:20.575 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 57953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T13:49:20.575 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/ 2024-04-25T13:49:20.575 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 66505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T13:49:20.575 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 80065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T13:49:20.575 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/ 2024-04-25T13:49:20.575 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 74289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T13:49:20.575 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 79457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T13:49:20.575 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 100785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T13:49:20.575 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 83441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T13:49:20.575 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/ 2024-04-25T13:49:20.576 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-04-25T13:49:20.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-04-25T13:49:20.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T13:49:20.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-04-25T13:49:20.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 644537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T13:49:20.576 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/ 2024-04-25T13:49:20.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 128497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T13:49:20.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 207993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T13:49:20.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 82633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T13:49:20.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 91353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T13:49:20.576 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/ 2024-04-25T13:49:20.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T13:49:20.577 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/ 2024-04-25T13:49:20.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 110729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T13:49:20.577 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/ 2024-04-25T13:49:20.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 98689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T13:49:20.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 102513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T13:49:20.577 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/ 2024-04-25T13:49:20.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T13:49:20.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T13:49:20.577 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/ 2024-04-25T13:49:20.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 69913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T13:49:20.577 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/ 2024-04-25T13:49:20.578 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/defxx.ko 2024-04-25T13:49:20.578 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/skfp/ 2024-04-25T13:49:20.578 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 275801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/skfp/skfp.ko 2024-04-25T13:49:20.578 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 91105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T13:49:20.578 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/ 2024-04-25T13:49:20.578 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 290441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T13:49:20.578 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/ 2024-04-25T13:49:20.578 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T13:49:20.578 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T13:49:20.578 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ 2024-04-25T13:49:20.578 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 108329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T13:49:20.579 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 78065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T13:49:20.579 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T13:49:20.579 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/ 2024-04-25T13:49:20.579 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T13:49:20.579 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T13:49:20.579 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-gpio.ko 2024-04-25T13:49:20.579 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T13:49:20.579 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T13:49:20.579 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T13:49:20.579 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 62681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T13:49:20.579 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T13:49:20.580 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/ 2024-04-25T13:49:20.580 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T13:49:20.580 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 48313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T13:49:20.580 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ 2024-04-25T13:49:20.580 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T13:49:20.580 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T13:49:20.580 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 52865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T13:49:20.580 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T13:49:20.580 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T13:49:20.580 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T13:49:20.580 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T13:49:20.580 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T13:49:20.581 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T13:49:20.581 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T13:49:20.581 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 103377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T13:49:20.581 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 96025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T13:49:20.581 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T13:49:20.581 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 121105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T13:49:20.581 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T13:49:20.581 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 62113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T13:49:20.581 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T13:49:20.581 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T13:49:20.581 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T13:49:20.581 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/plip/ 2024-04-25T13:49:20.582 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/plip/plip.ko 2024-04-25T13:49:20.582 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ 2024-04-25T13:49:20.582 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T13:49:20.582 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T13:49:20.582 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T13:49:20.582 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T13:49:20.582 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T13:49:20.582 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T13:49:20.582 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T13:49:20.582 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T13:49:20.582 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sb1000.ko 2024-04-25T13:49:20.582 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/ 2024-04-25T13:49:20.583 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 52073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T13:49:20.583 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T13:49:20.583 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 80161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T13:49:20.583 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/ 2024-04-25T13:49:20.583 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 137321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T13:49:20.583 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T13:49:20.583 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T13:49:20.583 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T13:49:20.583 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T13:49:20.583 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T13:49:20.583 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 108089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T13:49:20.584 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/ 2024-04-25T13:49:20.584 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 176497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T13:49:20.584 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/ 2024-04-25T13:49:20.584 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 383593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T13:49:20.584 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/ 2024-04-25T13:49:20.584 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 175537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T13:49:20.584 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ 2024-04-25T13:49:20.584 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T13:49:20.584 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/ 2024-04-25T13:49:20.584 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/ 2024-04-25T13:49:20.584 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 335857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T13:49:20.585 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 126553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T13:49:20.585 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/ 2024-04-25T13:49:20.585 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 168145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T13:49:20.585 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T13:49:20.585 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 103889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T13:49:20.585 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T13:49:20.585 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/ 2024-04-25T13:49:20.585 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/ 2024-04-25T13:49:20.585 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T13:49:20.585 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-04-25T13:49:20.585 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-04-25T13:49:20.585 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pci-stub.ko 2024-04-25T13:49:20.586 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T13:49:20.586 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/ 2024-04-25T13:49:20.586 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T13:49:20.586 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 179961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia.ko 2024-04-25T13:49:20.586 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 78161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia_core.ko 2024-04-25T13:49:20.586 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-04-25T13:49:20.586 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T13:49:20.586 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 133753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T13:49:20.586 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/ 2024-04-25T13:49:20.586 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/broadcom/ 2024-04-25T13:49:20.586 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-04-25T13:49:20.586 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/samsung/ 2024-04-25T13:49:20.587 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-04-25T13:49:20.587 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/ 2024-04-25T13:49:20.587 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/ 2024-04-25T13:49:20.587 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 78193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T13:49:20.587 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/ 2024-04-25T13:49:20.587 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/ 2024-04-25T13:49:20.587 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-04-25T13:49:20.587 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec.ko 2024-04-25T13:49:20.587 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-04-25T13:49:20.587 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-04-25T13:49:20.587 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-04-25T13:49:20.588 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-04-25T13:49:20.588 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-04-25T13:49:20.588 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/ 2024-04-25T13:49:20.588 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 87481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T13:49:20.588 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T13:49:20.588 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 48257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T13:49:20.588 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 73865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T13:49:20.588 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T13:49:20.588 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 164769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T13:49:20.588 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T13:49:20.588 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 62073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T13:49:20.588 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T13:49:20.589 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 51881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T13:49:20.589 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T13:49:20.589 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T13:49:20.589 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/ibm_rtl.ko 2024-04-25T13:49:20.589 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/ 2024-04-25T13:49:20.589 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T13:49:20.589 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-04-25T13:49:20.589 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 54001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T13:49:20.589 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T13:49:20.589 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T13:49:20.589 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T13:49:20.590 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 51897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T13:49:20.590 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T13:49:20.590 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T13:49:20.590 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T13:49:20.590 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T13:49:20.590 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T13:49:20.590 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T13:49:20.590 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/ 2024-04-25T13:49:20.590 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ 2024-04-25T13:49:20.590 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/88pm860x_battery.ko 2024-04-25T13:49:20.590 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/88pm860x_charger.ko 2024-04-25T13:49:20.591 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 68833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq2415x_charger.ko 2024-04-25T13:49:20.591 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 73073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq24190_charger.ko 2024-04-25T13:49:20.591 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq24735-charger.ko 2024-04-25T13:49:20.591 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-04-25T13:49:20.591 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/da9030_battery.ko 2024-04-25T13:49:20.591 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/da9052-battery.ko 2024-04-25T13:49:20.591 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2760_battery.ko 2024-04-25T13:49:20.591 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2780_battery.ko 2024-04-25T13:49:20.591 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2781_battery.ko 2024-04-25T13:49:20.591 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2782_battery.ko 2024-04-25T13:49:20.591 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/gpio-charger.ko 2024-04-25T13:49:20.592 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/isp1704_charger.ko 2024-04-25T13:49:20.592 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/lp8727_charger.ko 2024-04-25T13:49:20.592 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max14577_charger.ko 2024-04-25T13:49:20.592 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max17042_battery.ko 2024-04-25T13:49:20.592 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max8903_charger.ko 2024-04-25T13:49:20.592 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max8925_power.ko 2024-04-25T13:49:20.592 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/pcf50633-charger.ko 2024-04-25T13:49:20.592 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/sbs-battery.ko 2024-04-25T13:49:20.592 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/test_power.ko 2024-04-25T13:49:20.592 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/tps65090-charger.ko 2024-04-25T13:49:20.592 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm831x_backup.ko 2024-04-25T13:49:20.592 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm831x_power.ko 2024-04-25T13:49:20.593 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm8350_power.ko 2024-04-25T13:49:20.593 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/ 2024-04-25T13:49:20.593 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 73297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T13:49:20.593 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T13:49:20.593 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/ 2024-04-25T13:49:20.593 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/ 2024-04-25T13:49:20.593 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T13:49:20.593 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T13:49:20.593 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T13:49:20.593 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T13:49:20.593 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ 2024-04-25T13:49:20.594 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 91393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T13:49:20.594 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T13:49:20.594 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/ 2024-04-25T13:49:20.594 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/ 2024-04-25T13:49:20.594 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/ 2024-04-25T13:49:20.594 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 67513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T13:49:20.594 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T13:49:20.594 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/ 2024-04-25T13:49:20.594 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-88pm80x.ko 2024-04-25T13:49:20.594 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-88pm860x.ko 2024-04-25T13:49:20.595 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T13:49:20.595 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9052.ko 2024-04-25T13:49:20.595 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9055.ko 2024-04-25T13:49:20.595 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9063.ko 2024-04-25T13:49:20.595 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T13:49:20.595 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 82225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T13:49:20.595 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T13:49:20.595 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T13:49:20.595 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T13:49:20.595 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T13:49:20.595 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T13:49:20.595 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T13:49:20.596 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T13:49:20.596 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T13:49:20.596 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T13:49:20.596 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T13:49:20.596 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T13:49:20.596 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-lp8788.ko 2024-04-25T13:49:20.596 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T13:49:20.596 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T13:49:20.596 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T13:49:20.596 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t86.ko 2024-04-25T13:49:20.596 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T13:49:20.596 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8907.ko 2024-04-25T13:49:20.597 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8925.ko 2024-04-25T13:49:20.597 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8997.ko 2024-04-25T13:49:20.597 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8998.ko 2024-04-25T13:49:20.597 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-04-25T13:49:20.597 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T13:49:20.597 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-palmas.ko 2024-04-25T13:49:20.597 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 56777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T13:49:20.597 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf50633.ko 2024-04-25T13:49:20.597 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T13:49:20.597 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T13:49:20.597 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T13:49:20.598 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T13:49:20.598 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rc5t583.ko 2024-04-25T13:49:20.598 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T13:49:20.598 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T13:49:20.598 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T13:49:20.598 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T13:49:20.598 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T13:49:20.598 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-s35390a.ko 2024-04-25T13:49:20.598 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T13:49:20.598 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-tps6586x.ko 2024-04-25T13:49:20.598 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-tps65910.ko 2024-04-25T13:49:20.598 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-wm831x.ko 2024-04-25T13:49:20.599 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-wm8350.ko 2024-04-25T13:49:20.599 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T13:49:20.599 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ 2024-04-25T13:49:20.599 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 90377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T13:49:20.599 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 69593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T13:49:20.599 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 65945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T13:49:20.599 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 164569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T13:49:20.599 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T13:49:20.599 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/ 2024-04-25T13:49:20.599 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 340721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T13:49:20.599 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 120337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T13:49:20.599 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/ 2024-04-25T13:49:20.600 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 468081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T13:49:20.600 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 380673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T13:49:20.600 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic94xx/ 2024-04-25T13:49:20.600 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 243705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-04-25T13:49:20.600 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T13:49:20.600 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/ 2024-04-25T13:49:20.600 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 135617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T13:49:20.600 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T13:49:20.600 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/ 2024-04-25T13:49:20.600 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 304993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T13:49:20.600 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/ 2024-04-25T13:49:20.600 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1527113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T13:49:20.601 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/ 2024-04-25T13:49:20.601 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 297801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T13:49:20.601 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/ 2024-04-25T13:49:20.601 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 144401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T13:49:20.601 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T13:49:20.601 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/ 2024-04-25T13:49:20.601 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 390713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T13:49:20.601 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/ 2024-04-25T13:49:20.601 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-04-25T13:49:20.601 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 88881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T13:49:20.601 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-04-25T13:49:20.601 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 117481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T13:49:20.602 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 173017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T13:49:20.602 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 91569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T13:49:20.602 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T13:49:20.602 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/ 2024-04-25T13:49:20.602 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 267577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T13:49:20.602 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 78969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T13:49:20.602 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/ 2024-04-25T13:49:20.602 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 95329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T13:49:20.602 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 194017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T13:49:20.602 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fdomain.ko 2024-04-25T13:49:20.602 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/ 2024-04-25T13:49:20.603 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 366529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T13:49:20.603 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 251505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T13:49:20.603 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 65761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T13:49:20.603 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 57097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T13:49:20.603 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/imm.ko 2024-04-25T13:49:20.603 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T13:49:20.603 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 327841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T13:49:20.603 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 129785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T13:49:20.603 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/ 2024-04-25T13:49:20.603 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 397785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T13:49:20.603 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T13:49:20.603 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 63121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T13:49:20.604 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/ 2024-04-25T13:49:20.604 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 386849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T13:49:20.604 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 178177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T13:49:20.604 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 64809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T13:49:20.604 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/ 2024-04-25T13:49:20.604 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 274009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T13:49:20.604 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/ 2024-04-25T13:49:20.604 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 2736001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T13:49:20.604 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/ 2024-04-25T13:49:20.604 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 88529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T13:49:20.604 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T13:49:20.605 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 389177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T13:49:20.605 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 104769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T13:49:20.605 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/ 2024-04-25T13:49:20.605 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 836945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T13:49:20.605 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/ 2024-04-25T13:49:20.605 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 158729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T13:49:20.605 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 96537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T13:49:20.605 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/ 2024-04-25T13:49:20.605 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 101625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-04-25T13:49:20.605 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-04-25T13:49:20.605 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-04-25T13:49:20.605 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-04-25T13:49:20.606 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/ 2024-04-25T13:49:20.606 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 443745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T13:49:20.606 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 152033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T13:49:20.606 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ppa.ko 2024-04-25T13:49:20.606 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 66977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T13:49:20.606 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/ 2024-04-25T13:49:20.606 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 2324809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T13:49:20.606 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 103601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T13:49:20.606 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/ 2024-04-25T13:49:20.606 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 697337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T13:49:20.607 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qlogicfas408.ko 2024-04-25T13:49:20.607 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T13:49:20.607 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 322865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T13:49:20.607 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 219857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T13:49:20.607 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 357153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T13:49:20.607 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 121185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T13:49:20.607 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 93481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T13:49:20.607 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T13:49:20.607 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T13:49:20.607 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 138121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T13:49:20.607 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 57401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T13:49:20.607 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/ 2024-04-25T13:49:20.608 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 205025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T13:49:20.608 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T13:49:20.608 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T13:49:20.608 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 48777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T13:49:20.608 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/xen-scsifront.ko 2024-04-25T13:49:20.608 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ 2024-04-25T13:49:20.608 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 174913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T13:49:20.608 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/ 2024-04-25T13:49:20.608 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/ 2024-04-25T13:49:20.608 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 920169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T13:49:20.608 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/ 2024-04-25T13:49:20.608 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 60321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T13:49:20.609 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 48129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T13:49:20.609 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 54233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T13:49:20.609 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1170033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T13:49:20.609 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 51897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T13:49:20.609 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 149721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T13:49:20.609 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/ 2024-04-25T13:49:20.609 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 90057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T13:49:20.609 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/ 2024-04-25T13:49:20.609 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/ 2024-04-25T13:49:20.609 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/ 2024-04-25T13:49:20.609 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T13:49:20.609 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T13:49:20.609 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T13:49:20.610 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T13:49:20.610 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T13:49:20.610 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T13:49:20.610 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T13:49:20.610 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T13:49:20.610 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T13:49:20.610 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T13:49:20.610 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T13:49:20.610 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T13:49:20.610 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T13:49:20.610 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T13:49:20.611 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T13:49:20.611 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-04-25T13:49:20.611 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T13:49:20.611 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T13:49:20.611 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T13:49:20.611 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ 2024-04-25T13:49:20.611 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ 2024-04-25T13:49:20.611 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 99313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T13:49:20.611 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 69729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/moxa.ko 2024-04-25T13:49:20.611 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/mxser.ko 2024-04-25T13:49:20.611 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 134401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T13:49:20.611 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T13:49:20.612 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 78969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T13:49:20.612 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/ 2024-04-25T13:49:20.612 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/ 2024-04-25T13:49:20.612 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-04-25T13:49:20.612 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 65121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T13:49:20.612 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-04-25T13:49:20.612 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/altera_uart.ko 2024-04-25T13:49:20.612 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T13:49:20.612 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 91665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-04-25T13:49:20.612 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/ 2024-04-25T13:49:20.612 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 120577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T13:49:20.613 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/rp2.ko 2024-04-25T13:49:20.613 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 65641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/sc16is7xx.ko 2024-04-25T13:49:20.613 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/ 2024-04-25T13:49:20.613 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ 2024-04-25T13:49:20.613 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 693081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T13:49:20.613 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ 2024-04-25T13:49:20.613 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T13:49:20.613 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-04-25T13:49:20.613 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/ 2024-04-25T13:49:20.613 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T13:49:20.613 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T13:49:20.613 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T13:49:20.614 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_dmem_genirq.ko 2024-04-25T13:49:20.614 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_mf624.ko 2024-04-25T13:49:20.614 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_netx.ko 2024-04-25T13:49:20.614 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T13:49:20.614 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-04-25T13:49:20.614 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T13:49:20.614 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/ 2024-04-25T13:49:20.614 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/c67x00/ 2024-04-25T13:49:20.614 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 90321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/c67x00/c67x00.ko 2024-04-25T13:49:20.614 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ 2024-04-25T13:49:20.614 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 189209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-04-25T13:49:20.614 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-04-25T13:49:20.615 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-04-25T13:49:20.615 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-04-25T13:49:20.615 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-04-25T13:49:20.615 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/ 2024-04-25T13:49:20.615 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 101545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T13:49:20.615 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 63073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T13:49:20.615 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 56721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T13:49:20.615 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 107417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T13:49:20.615 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/common/ 2024-04-25T13:49:20.615 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/common/ulpi.ko 2024-04-25T13:49:20.615 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc2/ 2024-04-25T13:49:20.615 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-04-25T13:49:20.616 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/ 2024-04-25T13:49:20.616 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-04-25T13:49:20.616 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-04-25T13:49:20.616 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 574593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3.ko 2024-04-25T13:49:20.616 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/ 2024-04-25T13:49:20.616 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/ 2024-04-25T13:49:20.616 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 72249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/u_ether.ko 2024-04-25T13:49:20.616 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/u_serial.ko 2024-04-25T13:49:20.616 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-04-25T13:49:20.616 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-04-25T13:49:20.616 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-04-25T13:49:20.617 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-04-25T13:49:20.617 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 131345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-04-25T13:49:20.617 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 62321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-04-25T13:49:20.617 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 146617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-04-25T13:49:20.617 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 70689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-04-25T13:49:20.617 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-04-25T13:49:20.617 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-04-25T13:49:20.617 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 70017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-04-25T13:49:20.617 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 101249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-04-25T13:49:20.617 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-04-25T13:49:20.617 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 71257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-04-25T13:49:20.618 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 84937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-04-25T13:49:20.618 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/ 2024-04-25T13:49:20.618 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-04-25T13:49:20.618 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-04-25T13:49:20.618 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-04-25T13:49:20.618 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-04-25T13:49:20.618 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-04-25T13:49:20.618 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-04-25T13:49:20.618 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-04-25T13:49:20.618 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-04-25T13:49:20.618 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-04-25T13:49:20.619 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-04-25T13:49:20.619 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-04-25T13:49:20.619 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-04-25T13:49:20.619 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 78073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-04-25T13:49:20.619 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-04-25T13:49:20.619 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 204049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/libcomposite.ko 2024-04-25T13:49:20.619 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/ 2024-04-25T13:49:20.619 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-04-25T13:49:20.619 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/bdc/ 2024-04-25T13:49:20.619 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 182561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-04-25T13:49:20.619 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-04-25T13:49:20.620 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 66513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-04-25T13:49:20.620 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 85217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-04-25T13:49:20.620 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 69313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-04-25T13:49:20.620 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 90313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/net2272.ko 2024-04-25T13:49:20.620 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 90361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/net2280.ko 2024-04-25T13:49:20.620 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 94841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-04-25T13:49:20.620 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 80521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-04-25T13:49:20.620 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 87465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-04-25T13:49:20.620 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 95441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-04-25T13:49:20.620 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 195969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-04-25T13:49:20.620 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/ 2024-04-25T13:49:20.620 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/bcma-hcd.ko 2024-04-25T13:49:20.620 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 108321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/isp116x-hcd.ko 2024-04-25T13:49:20.621 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 104377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-04-25T13:49:20.621 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 102273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-04-25T13:49:20.621 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 60105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T13:49:20.621 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811_cs.ko 2024-04-25T13:49:20.621 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/ssb-hcd.ko 2024-04-25T13:49:20.621 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-04-25T13:49:20.621 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/ 2024-04-25T13:49:20.621 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T13:49:20.621 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T13:49:20.621 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ 2024-04-25T13:49:20.621 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T13:49:20.621 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T13:49:20.622 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-04-25T13:49:20.622 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/cytherm.ko 2024-04-25T13:49:20.622 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ehset.ko 2024-04-25T13:49:20.622 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T13:49:20.622 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T13:49:20.622 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T13:49:20.622 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T13:49:20.622 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T13:49:20.622 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T13:49:20.622 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T13:49:20.622 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T13:49:20.623 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/lvstest.ko 2024-04-25T13:49:20.623 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/ 2024-04-25T13:49:20.623 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 85313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T13:49:20.623 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T13:49:20.623 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T13:49:20.623 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T13:49:20.623 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T13:49:20.623 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 84265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbtest.ko 2024-04-25T13:49:20.623 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 52065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T13:49:20.623 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T13:49:20.623 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/mon/ 2024-04-25T13:49:20.623 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 91529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/mon/usbmon.ko 2024-04-25T13:49:20.624 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/musb/ 2024-04-25T13:49:20.624 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 396209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/musb/musb_hdrc.ko 2024-04-25T13:49:20.624 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/ 2024-04-25T13:49:20.624 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T13:49:20.624 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-04-25T13:49:20.624 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-isp1301.ko 2024-04-25T13:49:20.624 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-tahvo.ko 2024-04-25T13:49:20.624 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ 2024-04-25T13:49:20.624 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T13:49:20.624 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T13:49:20.624 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T13:49:20.625 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T13:49:20.625 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 79681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T13:49:20.625 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T13:49:20.625 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 66873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T13:49:20.625 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 58329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T13:49:20.625 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T13:49:20.625 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/f81232.ko 2024-04-25T13:49:20.625 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 188609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T13:49:20.625 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T13:49:20.625 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 152305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T13:49:20.625 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 142833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T13:49:20.626 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 69217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T13:49:20.626 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T13:49:20.626 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T13:49:20.626 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 66361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T13:49:20.626 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 122329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T13:49:20.626 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T13:49:20.626 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T13:49:20.626 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T13:49:20.626 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T13:49:20.626 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/metro-usb.ko 2024-04-25T13:49:20.626 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 79681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T13:49:20.626 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 89969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T13:49:20.627 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 65073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mxuport.ko 2024-04-25T13:49:20.627 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T13:49:20.627 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T13:49:20.627 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T13:49:20.627 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 193857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T13:49:20.627 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T13:49:20.627 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T13:49:20.627 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T13:49:20.627 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T13:49:20.627 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T13:49:20.627 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T13:49:20.628 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T13:49:20.628 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T13:49:20.628 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T13:49:20.628 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T13:49:20.628 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 70497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T13:49:20.628 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T13:49:20.628 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T13:49:20.628 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T13:49:20.628 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 151137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usbserial.ko 2024-04-25T13:49:20.628 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T13:49:20.628 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T13:49:20.628 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/wishbone-serial.ko 2024-04-25T13:49:20.629 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T13:49:20.629 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ 2024-04-25T13:49:20.629 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 64049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T13:49:20.629 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T13:49:20.629 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T13:49:20.629 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T13:49:20.629 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T13:49:20.629 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T13:49:20.629 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T13:49:20.629 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T13:49:20.629 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T13:49:20.629 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T13:49:20.630 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T13:49:20.630 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T13:49:20.630 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T13:49:20.630 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T13:49:20.630 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 201193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T13:49:20.630 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/ 2024-04-25T13:49:20.630 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 76041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T13:49:20.630 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 85257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T13:49:20.630 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 122321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T13:49:20.630 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/ 2024-04-25T13:49:20.630 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/ 2024-04-25T13:49:20.631 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 206921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T13:49:20.631 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T13:49:20.631 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 134265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T13:49:20.631 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 93673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T13:49:20.631 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/ 2024-04-25T13:49:20.631 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 138721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T13:49:20.631 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T13:49:20.631 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 71281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T13:49:20.631 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 107905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T13:49:20.631 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/ 2024-04-25T13:49:20.631 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/ 2024-04-25T13:49:20.631 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-04-25T13:49:20.631 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/aat2870_bl.ko 2024-04-25T13:49:20.632 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp5520_bl.ko 2024-04-25T13:49:20.632 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp8860_bl.ko 2024-04-25T13:49:20.632 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 51745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp8870_bl.ko 2024-04-25T13:49:20.632 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T13:49:20.632 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/as3711_bl.ko 2024-04-25T13:49:20.632 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/bd6107.ko 2024-04-25T13:49:20.632 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/da903x_bl.ko 2024-04-25T13:49:20.632 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/da9052_bl.ko 2024-04-25T13:49:20.632 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/gpio_backlight.ko 2024-04-25T13:49:20.632 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/kb3886_bl.ko 2024-04-25T13:49:20.632 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T13:49:20.633 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lm3533_bl.ko 2024-04-25T13:49:20.633 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lm3639_bl.ko 2024-04-25T13:49:20.633 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lv5207lp.ko 2024-04-25T13:49:20.633 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/max8925_bl.ko 2024-04-25T13:49:20.633 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/pandora_bl.ko 2024-04-25T13:49:20.633 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-04-25T13:49:20.633 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T13:49:20.633 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/wm831x_bl.ko 2024-04-25T13:49:20.633 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/ 2024-04-25T13:49:20.633 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ 2024-04-25T13:49:20.633 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T13:49:20.633 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T13:49:20.634 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/matrox_w1.ko 2024-04-25T13:49:20.634 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/w1-gpio.ko 2024-04-25T13:49:20.634 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/ 2024-04-25T13:49:20.634 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T13:49:20.634 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T13:49:20.634 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T13:49:20.634 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T13:49:20.634 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T13:49:20.634 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T13:49:20.634 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T13:49:20.635 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T13:49:20.635 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T13:49:20.635 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T13:49:20.635 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 70369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T13:49:20.635 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 129433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T13:49:20.635 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ 2024-04-25T13:49:20.635 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/acquirewdt.ko 2024-04-25T13:49:20.635 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/advantechwdt.ko 2024-04-25T13:49:20.635 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T13:49:20.636 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T13:49:20.636 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/cpu5wdt.ko 2024-04-25T13:49:20.636 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9052_wdt.ko 2024-04-25T13:49:20.636 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9055_wdt.ko 2024-04-25T13:49:20.636 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9063_wdt.ko 2024-04-25T13:49:20.636 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/dw_wdt.ko 2024-04-25T13:49:20.636 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/eurotechwdt.ko 2024-04-25T13:49:20.636 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T13:49:20.636 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T13:49:20.636 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T13:49:20.636 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T13:49:20.636 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T13:49:20.636 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T13:49:20.637 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T13:49:20.637 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T13:49:20.637 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T13:49:20.637 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T13:49:20.637 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/kempld_wdt.ko 2024-04-25T13:49:20.637 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T13:49:20.637 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/mena21_wdt.ko 2024-04-25T13:49:20.637 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-04-25T13:49:20.637 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T13:49:20.637 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-04-25T13:49:20.637 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pc87413_wdt.ko 2024-04-25T13:49:20.637 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T13:49:20.638 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T13:49:20.638 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/retu_wdt.ko 2024-04-25T13:49:20.638 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-04-25T13:49:20.638 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-04-25T13:49:20.638 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T13:49:20.638 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sc1200wdt.ko 2024-04-25T13:49:20.638 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T13:49:20.638 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-04-25T13:49:20.638 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T13:49:20.638 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T13:49:20.638 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/twl4030_wdt.ko 2024-04-25T13:49:20.639 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T13:49:20.639 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T13:49:20.639 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T13:49:20.639 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T13:49:20.639 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wafer5823wdt.ko 2024-04-25T13:49:20.639 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T13:49:20.639 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wm831x_wdt.ko 2024-04-25T13:49:20.639 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wm8350_wdt.ko 2024-04-25T13:49:20.639 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T13:49:20.639 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/ 2024-04-25T13:49:20.639 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T13:49:20.639 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T13:49:20.640 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T13:49:20.640 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/ 2024-04-25T13:49:20.640 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 213913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T13:49:20.640 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T13:49:20.640 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 85713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-scsiback.ko 2024-04-25T13:49:20.640 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/ 2024-04-25T13:49:20.640 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T13:49:20.640 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ 2024-04-25T13:49:20.640 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/ 2024-04-25T13:49:20.640 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 192617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T13:49:20.640 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/adfs/ 2024-04-25T13:49:20.640 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 78153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/adfs/adfs.ko 2024-04-25T13:49:20.640 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/ 2024-04-25T13:49:20.641 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 197849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T13:49:20.641 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/afs/ 2024-04-25T13:49:20.641 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1261113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/afs/kafs.ko 2024-04-25T13:49:20.641 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/autofs/ 2024-04-25T13:49:20.641 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 127377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/autofs/autofs4.ko 2024-04-25T13:49:20.641 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/ 2024-04-25T13:49:20.641 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 80521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T13:49:20.641 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/bfs/ 2024-04-25T13:49:20.641 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/bfs/bfs.ko 2024-04-25T13:49:20.641 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T13:49:20.641 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/ 2024-04-25T13:49:20.641 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 4489593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T13:49:20.642 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/ 2024-04-25T13:49:20.642 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 376857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T13:49:20.642 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ 2024-04-25T13:49:20.642 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1586065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T13:49:20.642 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/ 2024-04-25T13:49:20.642 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 130073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T13:49:20.642 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/configfs/ 2024-04-25T13:49:20.642 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 139081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/configfs/configfs.ko 2024-04-25T13:49:20.642 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/ 2024-04-25T13:49:20.642 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T13:49:20.642 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/ 2024-04-25T13:49:20.642 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 813225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T13:49:20.643 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/efs/ 2024-04-25T13:49:20.643 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/efs/efs.ko 2024-04-25T13:49:20.643 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/ 2024-04-25T13:49:20.643 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1906817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T13:49:20.643 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/ 2024-04-25T13:49:20.643 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T13:49:20.643 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/freevxfs/ 2024-04-25T13:49:20.643 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 52313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/freevxfs/freevxfs.ko 2024-04-25T13:49:20.643 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/ 2024-04-25T13:49:20.643 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T13:49:20.643 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/ 2024-04-25T13:49:20.643 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1486521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T13:49:20.644 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/ 2024-04-25T13:49:20.644 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 183305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T13:49:20.644 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/ 2024-04-25T13:49:20.644 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 282097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T13:49:20.644 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hpfs/ 2024-04-25T13:49:20.644 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 249009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hpfs/hpfs.ko 2024-04-25T13:49:20.644 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/ 2024-04-25T13:49:20.644 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 93033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T13:49:20.644 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/ 2024-04-25T13:49:20.644 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 533073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T13:49:20.644 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/ 2024-04-25T13:49:20.645 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 361369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T13:49:20.645 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/ 2024-04-25T13:49:20.645 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 112609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T13:49:20.645 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/ 2024-04-25T13:49:20.645 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 591585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T13:49:20.645 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/ 2024-04-25T13:49:20.645 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/ 2024-04-25T13:49:20.645 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 102561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T13:49:20.645 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/ 2024-04-25T13:49:20.645 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 74353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T13:49:20.645 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/ 2024-04-25T13:49:20.645 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 147689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T13:49:20.646 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1320249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T13:49:20.646 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 82105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv2.ko 2024-04-25T13:49:20.646 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 151641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T13:49:20.646 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 2518505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T13:49:20.646 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/ 2024-04-25T13:49:20.646 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T13:49:20.646 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T13:49:20.646 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/ 2024-04-25T13:49:20.646 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 2024049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T13:49:20.646 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/ 2024-04-25T13:49:20.646 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 734025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T13:49:20.646 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/ 2024-04-25T13:49:20.646 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T13:49:20.647 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T13:49:20.647 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T13:49:20.647 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T13:49:20.647 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T13:49:20.647 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T13:49:20.647 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T13:49:20.647 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T13:49:20.647 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T13:49:20.647 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T13:49:20.647 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T13:49:20.647 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ascii.ko 2024-04-25T13:49:20.647 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T13:49:20.647 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T13:49:20.648 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T13:49:20.648 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T13:49:20.648 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T13:49:20.648 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T13:49:20.648 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T13:49:20.648 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T13:49:20.648 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T13:49:20.648 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T13:49:20.648 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T13:49:20.648 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T13:49:20.648 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T13:49:20.648 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T13:49:20.648 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T13:49:20.649 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T13:49:20.649 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T13:49:20.649 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T13:49:20.649 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 95625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T13:49:20.649 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 144209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T13:49:20.649 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 167137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T13:49:20.649 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 119513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T13:49:20.649 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T13:49:20.649 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T13:49:20.649 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T13:49:20.649 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T13:49:20.649 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T13:49:20.650 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T13:49:20.650 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T13:49:20.650 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T13:49:20.650 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T13:49:20.650 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T13:49:20.650 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T13:49:20.650 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T13:49:20.650 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T13:49:20.650 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T13:49:20.650 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T13:49:20.650 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T13:49:20.650 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T13:49:20.651 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ 2024-04-25T13:49:20.651 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ 2024-04-25T13:49:20.651 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 302073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T13:49:20.651 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ 2024-04-25T13:49:20.651 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 548169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T13:49:20.651 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ 2024-04-25T13:49:20.651 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 66361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T13:49:20.651 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 3245617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T13:49:20.651 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 22113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T13:49:20.651 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T13:49:20.651 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T13:49:20.651 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/omfs/ 2024-04-25T13:49:20.651 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 67489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/omfs/omfs.ko 2024-04-25T13:49:20.652 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/ 2024-04-25T13:49:20.652 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 462873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T13:49:20.652 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ 2024-04-25T13:49:20.652 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 67977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T13:49:20.652 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx4/ 2024-04-25T13:49:20.652 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx4/qnx4.ko 2024-04-25T13:49:20.652 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx6/ 2024-04-25T13:49:20.652 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 56097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx6/qnx6.ko 2024-04-25T13:49:20.652 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/ 2024-04-25T13:49:20.652 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_tree.ko 2024-04-25T13:49:20.652 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_v1.ko 2024-04-25T13:49:20.652 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_v2.ko 2024-04-25T13:49:20.653 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/ 2024-04-25T13:49:20.653 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 684841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T13:49:20.653 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/ 2024-04-25T13:49:20.653 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T13:49:20.653 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/ 2024-04-25T13:49:20.653 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/ 2024-04-25T13:49:20.653 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 3319041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T13:49:20.653 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/ 2024-04-25T13:49:20.653 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T13:49:20.653 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T13:49:20.653 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/ 2024-04-25T13:49:20.654 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 164185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T13:49:20.654 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/ 2024-04-25T13:49:20.654 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 115977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T13:49:20.654 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/ 2024-04-25T13:49:20.654 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 310017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T13:49:20.654 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ 2024-04-25T13:49:20.654 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 198233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T13:49:20.654 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/ 2024-04-25T13:49:20.654 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 5192705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T13:49:20.654 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/ 2024-04-25T13:49:20.654 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/locking/ 2024-04-25T13:49:20.655 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 97625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/locking/locktorture.ko 2024-04-25T13:49:20.655 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/time/ 2024-04-25T13:49:20.655 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/time/test_udelay.ko 2024-04-25T13:49:20.655 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 72665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T13:49:20.655 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ 2024-04-25T13:49:20.655 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T13:49:20.655 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc7.ko 2024-04-25T13:49:20.655 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T13:49:20.655 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/ 2024-04-25T13:49:20.655 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T13:49:20.655 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T13:49:20.655 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/interval_tree_test.ko 2024-04-25T13:49:20.656 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T13:49:20.656 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T13:49:20.656 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/ 2024-04-25T13:49:20.656 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T13:49:20.656 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T13:49:20.656 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/ 2024-04-25T13:49:20.656 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T13:49:20.656 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/memory-notifier-error-inject.ko 2024-04-25T13:49:20.656 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/notifier-error-inject.ko 2024-04-25T13:49:20.656 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/percpu_test.ko 2024-04-25T13:49:20.656 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/pm-notifier-error-inject.ko 2024-04-25T13:49:20.657 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/ 2024-04-25T13:49:20.657 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 152025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T13:49:20.657 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/rbtree_test.ko 2024-04-25T13:49:20.657 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/ 2024-04-25T13:49:20.657 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T13:49:20.657 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test-kstrtox.ko 2024-04-25T13:49:20.657 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 4912441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_bpf.ko 2024-04-25T13:49:20.657 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 61169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_firmware.ko 2024-04-25T13:49:20.657 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_module.ko 2024-04-25T13:49:20.657 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_user_copy.ko 2024-04-25T13:49:20.657 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T13:49:20.657 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T13:49:20.658 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T13:49:20.658 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/xz/ 2024-04-25T13:49:20.658 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/xz/xz_dec_test.ko 2024-04-25T13:49:20.658 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/ 2024-04-25T13:49:20.658 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T13:49:20.658 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ 2024-04-25T13:49:20.658 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/ 2024-04-25T13:49:20.658 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 66969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T13:49:20.658 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T13:49:20.658 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T13:49:20.658 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T13:49:20.658 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T13:49:20.659 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T13:49:20.659 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T13:49:20.659 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T13:49:20.659 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/ 2024-04-25T13:49:20.659 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T13:49:20.659 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T13:49:20.659 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T13:49:20.659 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T13:49:20.659 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T13:49:20.659 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/ 2024-04-25T13:49:20.659 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 106793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T13:49:20.659 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/ 2024-04-25T13:49:20.660 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 236793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T13:49:20.660 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T13:49:20.660 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T13:49:20.660 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T13:49:20.660 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/ 2024-04-25T13:49:20.660 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 879697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T13:49:20.660 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/ 2024-04-25T13:49:20.660 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 84761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T13:49:20.660 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1030873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T13:49:20.660 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ 2024-04-25T13:49:20.660 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T13:49:20.660 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T13:49:20.660 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T13:49:20.661 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T13:49:20.661 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T13:49:20.661 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T13:49:20.661 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T13:49:20.661 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T13:49:20.661 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T13:49:20.661 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T13:49:20.661 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T13:49:20.661 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T13:49:20.661 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T13:49:20.661 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T13:49:20.662 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T13:49:20.662 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T13:49:20.662 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T13:49:20.662 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T13:49:20.662 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T13:49:20.662 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T13:49:20.662 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 70529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T13:49:20.662 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T13:49:20.662 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T13:49:20.662 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/ 2024-04-25T13:49:20.662 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1243505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T13:49:20.696 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/ 2024-04-25T13:49:20.697 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 153801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T13:49:20.697 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/ 2024-04-25T13:49:20.697 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 244889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T13:49:20.697 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T13:49:20.697 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 71033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T13:49:20.697 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 74993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T13:49:20.697 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/ 2024-04-25T13:49:20.697 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 507705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T13:49:20.697 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T13:49:20.697 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T13:49:20.698 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T13:49:20.698 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T13:49:20.698 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/hsr/ 2024-04-25T13:49:20.698 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 128873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/hsr/hsr.ko 2024-04-25T13:49:20.698 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ 2024-04-25T13:49:20.698 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ 2024-04-25T13:49:20.698 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T13:49:20.698 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 357425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T13:49:20.698 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 59393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T13:49:20.698 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ 2024-04-25T13:49:20.698 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T13:49:20.699 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T13:49:20.699 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 60233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T13:49:20.699 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T13:49:20.699 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 65665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T13:49:20.699 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 75489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T13:49:20.699 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 77721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T13:49:20.699 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T13:49:20.699 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T13:49:20.699 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T13:49:20.699 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ 2024-04-25T13:49:20.699 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T13:49:20.699 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T13:49:20.700 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T13:49:20.700 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ip_tables.ko 2024-04-25T13:49:20.700 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T13:49:20.700 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-04-25T13:49:20.700 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T13:49:20.700 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T13:49:20.700 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T13:49:20.700 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-04-25T13:49:20.700 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T13:49:20.700 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T13:49:20.700 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T13:49:20.701 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T13:49:20.701 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T13:49:20.701 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T13:49:20.701 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T13:49:20.701 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T13:49:20.701 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T13:49:20.701 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-04-25T13:49:20.701 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T13:49:20.701 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T13:49:20.701 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T13:49:20.701 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T13:49:20.702 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T13:49:20.702 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T13:49:20.702 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T13:49:20.702 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T13:49:20.702 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T13:49:20.702 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T13:49:20.702 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T13:49:20.702 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T13:49:20.702 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T13:49:20.702 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T13:49:20.702 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T13:49:20.703 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T13:49:20.703 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T13:49:20.703 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T13:49:20.703 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T13:49:20.703 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T13:49:20.703 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 54441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T13:49:20.703 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T13:49:20.703 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ 2024-04-25T13:49:20.703 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T13:49:20.703 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 47433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T13:49:20.703 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T13:49:20.704 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 103177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_gre.ko 2024-04-25T13:49:20.704 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 118881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T13:49:20.704 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T13:49:20.704 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 71089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T13:49:20.704 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T13:49:20.704 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/mip6.ko 2024-04-25T13:49:20.704 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ 2024-04-25T13:49:20.704 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T13:49:20.704 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-04-25T13:49:20.704 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T13:49:20.704 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T13:49:20.705 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T13:49:20.705 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T13:49:20.705 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T13:49:20.705 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T13:49:20.705 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T13:49:20.705 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T13:49:20.705 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T13:49:20.705 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T13:49:20.705 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T13:49:20.705 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T13:49:20.705 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T13:49:20.705 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T13:49:20.706 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T13:49:20.706 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T13:49:20.706 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T13:49:20.706 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 29817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T13:49:20.706 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T13:49:20.706 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T13:49:20.706 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T13:49:20.706 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T13:49:20.706 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T13:49:20.706 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 94089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T13:49:20.706 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 30297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T13:49:20.707 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T13:49:20.707 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/ 2024-04-25T13:49:20.707 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 106369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T13:49:20.707 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/ 2024-04-25T13:49:20.707 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 153217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T13:49:20.707 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T13:49:20.707 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T13:49:20.707 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T13:49:20.707 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T13:49:20.707 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T13:49:20.707 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 66241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T13:49:20.707 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/lapb/ 2024-04-25T13:49:20.708 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 63345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/lapb/lapb.ko 2024-04-25T13:49:20.708 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/ 2024-04-25T13:49:20.708 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T13:49:20.708 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 332889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc2.ko 2024-04-25T13:49:20.708 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/ 2024-04-25T13:49:20.708 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 388873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T13:49:20.708 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/ 2024-04-25T13:49:20.708 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T13:49:20.708 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ 2024-04-25T13:49:20.708 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ 2024-04-25T13:49:20.708 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 147745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T13:49:20.709 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T13:49:20.709 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T13:49:20.709 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T13:49:20.709 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 124041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T13:49:20.709 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 124345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T13:49:20.709 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 126681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T13:49:20.709 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 126473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T13:49:20.709 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 137833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T13:49:20.709 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 72353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T13:49:20.709 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 133681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T13:49:20.709 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 139073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T13:49:20.709 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 138265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T13:49:20.710 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 136177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T13:49:20.710 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 139897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T13:49:20.710 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T13:49:20.710 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ 2024-04-25T13:49:20.710 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 534113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T13:49:20.710 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T13:49:20.710 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T13:49:20.710 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T13:49:20.710 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T13:49:20.710 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T13:49:20.710 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T13:49:20.711 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T13:49:20.711 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T13:49:20.711 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T13:49:20.711 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T13:49:20.711 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T13:49:20.711 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T13:49:20.711 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T13:49:20.711 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 55217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T13:49:20.711 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 496817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T13:49:20.711 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T13:49:20.711 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T13:49:20.711 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T13:49:20.712 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 134377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T13:49:20.712 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T13:49:20.712 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T13:49:20.712 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 140801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T13:49:20.712 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T13:49:20.712 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T13:49:20.712 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 79873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T13:49:20.712 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T13:49:20.712 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T13:49:20.712 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T13:49:20.712 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 117201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T13:49:20.713 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T13:49:20.713 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 46105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T13:49:20.713 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 118369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T13:49:20.713 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T13:49:20.713 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T13:49:20.713 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T13:49:20.713 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T13:49:20.713 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T13:49:20.713 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T13:49:20.713 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 807985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T13:49:20.713 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T13:49:20.713 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T13:49:20.714 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-04-25T13:49:20.714 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T13:49:20.714 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T13:49:20.714 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 65145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T13:49:20.714 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T13:49:20.714 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T13:49:20.714 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 50657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T13:49:20.714 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T13:49:20.714 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T13:49:20.715 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T13:49:20.715 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T13:49:20.715 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T13:49:20.715 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T13:49:20.715 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T13:49:20.715 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T13:49:20.715 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T13:49:20.715 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_osf.ko 2024-04-25T13:49:20.715 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T13:49:20.715 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T13:49:20.715 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T13:49:20.715 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T13:49:20.716 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T13:49:20.716 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 24993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_synproxy.ko 2024-04-25T13:49:20.716 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tproxy.ko 2024-04-25T13:49:20.716 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 108393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/x_tables.ko 2024-04-25T13:49:20.716 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T13:49:20.716 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T13:49:20.716 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T13:49:20.716 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T13:49:20.716 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 21865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T13:49:20.716 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T13:49:20.716 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T13:49:20.716 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T13:49:20.717 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T13:49:20.717 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T13:49:20.717 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T13:49:20.717 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T13:49:20.717 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T13:49:20.717 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T13:49:20.717 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T13:49:20.717 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T13:49:20.717 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T13:49:20.717 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T13:49:20.717 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T13:49:20.717 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T13:49:20.718 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T13:49:20.718 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T13:49:20.718 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T13:49:20.718 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T13:49:20.718 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T13:49:20.718 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T13:49:20.721 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T13:49:20.722 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T13:49:20.722 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T13:49:20.722 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T13:49:20.722 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T13:49:20.722 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T13:49:20.722 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T13:49:20.722 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T13:49:20.722 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T13:49:20.722 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T13:49:20.722 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T13:49:20.723 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T13:49:20.723 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T13:49:20.723 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 52817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T13:49:20.723 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T13:49:20.723 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T13:49:20.723 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T13:49:20.723 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T13:49:20.723 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T13:49:20.723 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T13:49:20.723 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T13:49:20.723 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T13:49:20.723 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T13:49:20.724 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T13:49:20.724 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T13:49:20.724 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 16161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T13:49:20.724 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T13:49:20.724 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T13:49:20.724 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T13:49:20.724 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T13:49:20.724 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T13:49:20.724 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T13:49:20.724 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T13:49:20.724 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 11881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T13:49:20.725 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T13:49:20.725 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T13:49:20.725 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T13:49:20.725 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T13:49:20.725 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 19777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T13:49:20.725 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T13:49:20.725 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T13:49:20.725 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T13:49:20.725 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T13:49:20.725 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpudp.ko 2024-04-25T13:49:20.725 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T13:49:20.726 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T13:49:20.726 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/ 2024-04-25T13:49:20.726 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 17233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T13:49:20.726 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/ 2024-04-25T13:49:20.726 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 14073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T13:49:20.726 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/ 2024-04-25T13:49:20.726 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 488825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T13:49:20.726 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T13:49:20.726 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 10873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T13:49:20.726 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T13:49:20.726 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/ 2024-04-25T13:49:20.726 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T13:49:20.727 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/ 2024-04-25T13:49:20.727 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 142025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/phonet.ko 2024-04-25T13:49:20.727 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 68817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/pn_pep.ko 2024-04-25T13:49:20.727 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/ 2024-04-25T13:49:20.727 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 318617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T13:49:20.727 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 202305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T13:49:20.727 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 85105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T13:49:20.727 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rxrpc/ 2024-04-25T13:49:20.727 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1027897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rxrpc/rxrpc.ko 2024-04-25T13:49:20.727 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/ 2024-04-25T13:49:20.727 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T13:49:20.727 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 28721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T13:49:20.727 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T13:49:20.728 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 27593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T13:49:20.728 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 53465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T13:49:20.728 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T13:49:20.728 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T13:49:20.728 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T13:49:20.728 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 45673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T13:49:20.728 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T13:49:20.728 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 41457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T13:49:20.728 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_cgroup.ko 2024-04-25T13:49:20.728 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 37305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T13:49:20.728 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T13:49:20.728 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 49321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T13:49:20.729 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 79145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T13:49:20.729 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T13:49:20.729 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 15385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T13:49:20.729 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 48785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T13:49:20.729 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T13:49:20.729 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T13:49:20.729 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 8377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T13:49:20.729 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T13:49:20.729 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 25097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T13:49:20.729 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T13:49:20.729 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 43153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T13:49:20.729 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 39177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq_codel.ko 2024-04-25T13:49:20.730 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 42889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T13:49:20.730 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 62817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T13:49:20.730 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T13:49:20.730 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 81833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T13:49:20.730 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T13:49:20.730 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 36625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T13:49:20.730 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T13:49:20.730 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T13:49:20.730 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 44793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T13:49:20.730 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 32833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T13:49:20.730 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 13505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T13:49:20.731 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 31801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T13:49:20.731 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 60249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T13:49:20.731 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 40889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T13:49:20.731 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 33433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T13:49:20.731 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 38329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T13:49:20.731 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 35777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T13:49:20.731 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 34161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T13:49:20.731 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/ 2024-04-25T13:49:20.731 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1187313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T13:49:20.731 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T13:49:20.731 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/ 2024-04-25T13:49:20.731 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/ 2024-04-25T13:49:20.732 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 432409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T13:49:20.732 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 70113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T13:49:20.732 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1940889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T13:49:20.732 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/ 2024-04-25T13:49:20.732 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 890729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T13:49:20.732 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/ 2024-04-25T13:49:20.732 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 1122337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T13:49:20.732 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T13:49:20.732 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/ 2024-04-25T13:49:20.732 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 18801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T13:49:20.732 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/ 2024-04-25T13:49:20.732 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 105609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T13:49:20.733 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 90265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T13:49:20.733 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 123769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T13:49:20.733 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 12905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T13:49:20.733 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T13:49:20.733 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/x25/ 2024-04-25T13:49:20.733 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 165841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/x25/x25.ko 2024-04-25T13:49:20.733 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/ 2024-04-25T13:49:20.733 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 26729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_algo.ko 2024-04-25T13:49:20.733 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T13:49:20.733 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 131865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_user.ko 2024-04-25T13:49:20.733 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 644769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T13:49:20.733 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 628542 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T13:49:20.733 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 7706 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T13:49:20.734 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 23609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T13:49:20.734 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 9941 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T13:49:20.734 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 70990 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T13:49:20.734 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 146959 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T13:49:20.734 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 232835 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T13:49:20.734 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 210 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T13:49:20.734 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 72602 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T13:49:20.734 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 879 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T13:49:20.734 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 258450 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T13:49:20.734 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 313019 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T13:49:20.735 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/ 2024-04-25T13:49:20.735 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/ 2024-04-25T13:49:20.735 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/doc/ 2024-04-25T13:49:20.735 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/ 2024-04-25T13:49:20.735 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 172 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/changelog.Debian.gz 2024-04-25T13:49:20.735 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r-- root/root 691 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/copyright 2024-04-25T13:49:20.739 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T13:49:20.740 INFO:teuthology.task.kernel:searching for kernel 6.9.0-rc3-g806ec36b2112 2024-04-25T13:49:20.740 DEBUG:teuthology.orchestra.run.smithi086:> egrep '(submenu|menuentry.*6.9.0-rc3-g806ec36b2112).*{' /boot/grub/grub.cfg 2024-04-25T13:49:20.746 INFO:teuthology.orchestra.run.smithi086.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-25T13:49:20.747 INFO:teuthology.orchestra.run.smithi086.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g806ec36b2112-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-25T13:49:20.747 INFO:teuthology.orchestra.run.smithi086.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g806ec36b2112-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-25T13:49:20.747 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-04-25T13:49:20.747 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112 2024-04-25T13:49:20.747 DEBUG:teuthology.orchestra.run.smithi086:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-04-25T13:49:20.790 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi086.front.sepia.ceph.com to complete... 2024-04-25T13:49:21.145 INFO:teuthology.orchestra.run.smithi086.stderr:Sourcing file `/etc/default/grub' 2024-04-25T13:49:21.146 INFO:teuthology.orchestra.run.smithi086.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T13:49:21.171 INFO:teuthology.orchestra.run.smithi086.stderr:Generating grub configuration file ... 2024-04-25T13:49:21.365 INFO:teuthology.orchestra.run.smithi086.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T13:49:21.371 INFO:teuthology.orchestra.run.smithi086.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-25T13:49:21.615 INFO:teuthology.orchestra.run.smithi086.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T13:49:21.620 INFO:teuthology.orchestra.run.smithi086.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T13:49:21.659 INFO:teuthology.orchestra.run.smithi086.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T13:49:21.664 INFO:teuthology.orchestra.run.smithi086.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T13:49:21.819 INFO:teuthology.orchestra.run.smithi086.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T13:49:21.819 INFO:teuthology.orchestra.run.smithi086.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T13:49:21.819 INFO:teuthology.orchestra.run.smithi086.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T13:49:21.831 INFO:teuthology.orchestra.run.smithi086.stderr:done 2024-04-25T13:49:37.978 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-04-25T13:49:37.978 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-04-25T13:49:37.979 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:49:52.927 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-25T13:49:52.928 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-25T13:49:52.928 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:49:56.382 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-04-25T13:50:05.391 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-04-25T13:50:05.392 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:50:08.446 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-04-25T13:50:11.486 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-04-25T13:50:20.456 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-04-25T13:50:20.457 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:50:20.488 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-25T13:50:20.488 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:50:23.550 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-04-25T13:50:35.560 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-25T13:50:35.561 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:50:53.985 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-04-25T13:51:08.998 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-25T13:51:08.999 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:51:10.069 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-04-25T13:51:11.113 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-25T13:51:11.113 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T13:51:11.113 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-04-25T13:51:11.121 INFO:teuthology.orchestra.run.smithi086.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T13:51:11.121 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T13:51:11.121 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T13:51:11.121 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T13:51:12.122 DEBUG:teuthology.parallel:result is None 2024-04-25T13:51:20.459 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T13:51:35.460 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-04-25T13:51:35.461 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:51:35.682 DEBUG:teuthology.orchestra.run.smithi155:> true 2024-04-25T13:51:36.430 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-04-25T13:51:36.430 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T13:51:36.431 DEBUG:teuthology.orchestra.run.smithi155:> uname -r 2024-04-25T13:51:36.483 INFO:teuthology.orchestra.run.smithi155.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T13:51:36.483 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T13:51:36.483 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T13:51:36.483 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T13:51:37.484 DEBUG:teuthology.parallel:result is None 2024-04-25T13:51:37.484 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T13:51:37.500 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T13:51:37.500 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T13:51:37.504 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T13:51:37.512 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T13:51:37.518 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T13:51:37.526 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T13:51:37.527 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T13:51:37.554 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T13:51:37.578 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T13:51:37.585 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T13:51:37.585 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T13:51:37.622 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T13:51:37.693 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:51:37.715 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:51:37.724 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:51:37.757 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:51:37.759 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T13:51:37.767 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T13:51:37.767 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T13:51:37.771 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T13:51:37.792 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T13:51:37.830 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T13:51:37.831 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T13:51:37.834 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T13:51:37.844 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:51:37.895 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:51:37.947 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T13:51:37.947 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T13:51:38.030 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:51:38.039 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:51:38.088 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T13:51:38.088 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T13:51:38.169 DEBUG:teuthology.orchestra.run.smithi086:> sudo service rsyslog restart 2024-04-25T13:51:38.173 DEBUG:teuthology.orchestra.run.smithi155:> sudo service rsyslog restart 2024-04-25T13:51:38.294 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T13:51:38.301 INFO:teuthology.task.internal:Starting timer... 2024-04-25T13:51:38.301 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T13:51:38.322 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T13:51:38.330 DEBUG:teuthology.task.selinux:Excluding smithi086: OS 'ubuntu' does not support SELinux 2024-04-25T13:51:38.330 DEBUG:teuthology.task.selinux:Excluding smithi155: OS 'ubuntu' does not support SELinux 2024-04-25T13:51:38.330 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T13:51:38.331 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-25T13:51:38.331 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T13:51:38.331 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T13:51:38.339 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T13:51:38.345 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T13:51:38.489 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T13:51:38.505 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T13:51:38.507 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi086.front.sepia.ceph.com,smithi155.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T13:59:17.762 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi086.front.sepia.ceph.com'), Remote(name='ubuntu@smithi155.front.sepia.ceph.com')] 2024-04-25T13:59:17.763 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-25T13:59:17.764 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:59:17.848 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-04-25T13:59:17.973 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-25T13:59:17.974 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-04-25T13:59:17.975 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:59:18.061 DEBUG:teuthology.orchestra.run.smithi155:> true 2024-04-25T13:59:18.185 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-04-25T13:59:18.186 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T13:59:18.199 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T13:59:18.199 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T13:59:18.200 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:59:18.203 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T13:59:18.203 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:59:18.239 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 13:59:18 ntpd[17209]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T13:59:18.240 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 13:59:18 ntpd[17209]: Command line: ntpd -gq 2024-04-25T13:59:18.240 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 13:59:18 ntpd[17209]: ---------------------------------------------------- 2024-04-25T13:59:18.240 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 13:59:18 ntpd[17209]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T13:59:18.240 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 13:59:18 ntpd[17209]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T13:59:18.240 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 13:59:18 ntpd[17209]: corporation. Support and training for ntp-4 are 2024-04-25T13:59:18.240 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 13:59:18 ntpd[17209]: available at https://www.nwtime.org/support 2024-04-25T13:59:18.240 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 13:59:18 ntpd[17209]: ---------------------------------------------------- 2024-04-25T13:59:18.241 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 13:59:18 ntpd[17209]: proto: precision = 0.043 usec (-24) 2024-04-25T13:59:18.242 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 13:59:18 ntpd[17209]: basedate set to 2022-02-04 2024-04-25T13:59:18.242 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 13:59:18 ntpd[17209]: gps base set to 2022-02-06 (week 2196) 2024-04-25T13:59:18.242 INFO:teuthology.orchestra.run.smithi086.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T13:59:18.242 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 13:59:18 ntpd[17209]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T13:59:18.243 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 13:59:18 ntpd[17209]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T13:59:18.243 INFO:teuthology.orchestra.run.smithi086.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T13:59:18.244 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 13:59:18 ntpd[17209]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T13:59:18.244 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 13:59:18 ntpd[17209]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T13:59:18.244 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 13:59:18 ntpd[17209]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T13:59:18.245 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 13:59:18 ntpd[17209]: Listen normally on 3 enp3s0f1 172.21.15.86:123 2024-04-25T13:59:18.245 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 13:59:18 ntpd[17209]: Listen normally on 4 lo [::1]:123 2024-04-25T13:59:18.245 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 13:59:18 ntpd[17209]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d809%5]:123 2024-04-25T13:59:18.245 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 13:59:18 ntpd[17209]: Listening on routing socket on fd #22 for interface updates 2024-04-25T13:59:18.268 INFO:teuthology.orchestra.run.smithi155.stdout:25 Apr 13:59:18 ntpd[17207]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T13:59:18.268 INFO:teuthology.orchestra.run.smithi155.stdout:25 Apr 13:59:18 ntpd[17207]: Command line: ntpd -gq 2024-04-25T13:59:18.268 INFO:teuthology.orchestra.run.smithi155.stdout:25 Apr 13:59:18 ntpd[17207]: ---------------------------------------------------- 2024-04-25T13:59:18.268 INFO:teuthology.orchestra.run.smithi155.stdout:25 Apr 13:59:18 ntpd[17207]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T13:59:18.268 INFO:teuthology.orchestra.run.smithi155.stdout:25 Apr 13:59:18 ntpd[17207]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T13:59:18.268 INFO:teuthology.orchestra.run.smithi155.stdout:25 Apr 13:59:18 ntpd[17207]: corporation. Support and training for ntp-4 are 2024-04-25T13:59:18.268 INFO:teuthology.orchestra.run.smithi155.stdout:25 Apr 13:59:18 ntpd[17207]: available at https://www.nwtime.org/support 2024-04-25T13:59:18.268 INFO:teuthology.orchestra.run.smithi155.stdout:25 Apr 13:59:18 ntpd[17207]: ---------------------------------------------------- 2024-04-25T13:59:18.268 INFO:teuthology.orchestra.run.smithi155.stdout:25 Apr 13:59:18 ntpd[17207]: proto: precision = 0.041 usec (-24) 2024-04-25T13:59:18.268 INFO:teuthology.orchestra.run.smithi155.stdout:25 Apr 13:59:18 ntpd[17207]: basedate set to 2022-02-04 2024-04-25T13:59:18.268 INFO:teuthology.orchestra.run.smithi155.stdout:25 Apr 13:59:18 ntpd[17207]: gps base set to 2022-02-06 (week 2196) 2024-04-25T13:59:18.268 INFO:teuthology.orchestra.run.smithi155.stdout:25 Apr 13:59:18 ntpd[17207]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T13:59:18.269 INFO:teuthology.orchestra.run.smithi155.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T13:59:18.269 INFO:teuthology.orchestra.run.smithi155.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T13:59:18.270 INFO:teuthology.orchestra.run.smithi155.stdout:25 Apr 13:59:18 ntpd[17207]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T13:59:18.270 INFO:teuthology.orchestra.run.smithi155.stdout:25 Apr 13:59:18 ntpd[17207]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T13:59:18.270 INFO:teuthology.orchestra.run.smithi155.stdout:25 Apr 13:59:18 ntpd[17207]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T13:59:18.270 INFO:teuthology.orchestra.run.smithi155.stdout:25 Apr 13:59:18 ntpd[17207]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T13:59:18.270 INFO:teuthology.orchestra.run.smithi155.stdout:25 Apr 13:59:18 ntpd[17207]: Listen normally on 3 enp3s0f1 172.21.15.155:123 2024-04-25T13:59:18.270 INFO:teuthology.orchestra.run.smithi155.stdout:25 Apr 13:59:18 ntpd[17207]: Listen normally on 4 lo [::1]:123 2024-04-25T13:59:18.270 INFO:teuthology.orchestra.run.smithi155.stdout:25 Apr 13:59:18 ntpd[17207]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6ded%5]:123 2024-04-25T13:59:18.270 INFO:teuthology.orchestra.run.smithi155.stdout:25 Apr 13:59:18 ntpd[17207]: Listening on routing socket on fd #22 for interface updates 2024-04-25T13:59:19.240 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:19 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:19.240 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:19 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:19.240 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:19 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:19.240 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:19 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:19.240 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:19 ntpd[17209]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:59:19.240 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:19 ntpd[17209]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:59:19.267 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:19 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:19.267 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:19 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:19.267 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:19 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:19.267 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:19 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:19.267 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:19 ntpd[17207]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:59:19.267 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:19 ntpd[17207]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:59:20.239 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:20 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:20.240 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:20 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:21.240 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:21 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:21.240 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:21 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:21.240 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:21 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:21.240 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:21 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:21.240 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:21 ntpd[17209]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:59:21.240 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:21 ntpd[17209]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:59:21.268 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:21 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:21.268 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:21 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:21.268 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:21 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:21.268 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:21 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:21.268 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:21 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:21.269 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:21 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:21.269 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:21 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:21.269 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:21 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:21.269 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:21 ntpd[17207]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:59:21.269 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:21 ntpd[17207]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:59:21.269 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:21 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:21.269 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:21 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:21.269 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:21 ntpd[17207]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:59:21.269 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:21 ntpd[17207]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:59:22.240 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:22 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:22.240 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:22 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:22.240 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:22 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:22.240 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:22 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:22.240 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:22 ntpd[17209]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:59:22.240 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:22 ntpd[17209]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:59:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:23 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:23 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:23 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:23 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:23 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:23 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:23 ntpd[17209]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:59:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:23 ntpd[17209]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:59:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:23 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:23 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:23 ntpd[17209]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:59:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:23 ntpd[17209]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:59:23.268 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:23 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:23.268 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:23 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:23.268 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:23 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:23.268 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:23 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:23.268 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:23 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:23.268 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:23 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:23.268 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:23 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:23.269 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:23 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:23.269 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:23 ntpd[17207]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:59:23.269 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:23 ntpd[17207]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:59:23.269 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:23 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:23.269 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:23 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:23.269 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:23 ntpd[17207]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:59:23.269 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:23 ntpd[17207]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:59:23.269 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:23 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:23.269 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:23 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:23.269 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:23 ntpd[17207]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:59:23.269 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:23 ntpd[17207]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:59:24.240 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:24 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:24.240 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:24 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:24.240 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:24 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:24.240 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:24 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:24.241 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:24 ntpd[17209]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:59:24.241 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:24 ntpd[17209]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:59:25.240 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:25 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:25.240 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:25 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:25.240 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:25 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:25 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:25 ntpd[17209]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:25 ntpd[17209]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:25 ntpd[17209]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:59:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:25 ntpd[17209]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:59:25.241 INFO:teuthology.orchestra.run.smithi086.stdout:25 Apr 13:59:25 ntpd[17209]: ntpd: time slew -0.003304 s 2024-04-25T13:59:25.241 INFO:teuthology.orchestra.run.smithi086.stdout:ntpd: time slew -0.003304s 2024-04-25T13:59:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:25 ntpd[17209]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T13:59:25.242 INFO:teuthology.orchestra.run.smithi086.stderr:25 Apr 13:59:25 ntpd[17209]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T13:59:25.267 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:25 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:25.267 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:25 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:25.268 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:25 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:25.268 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:25 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:25.268 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:25 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:25.268 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:25 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:25.268 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:25 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:59:25.268 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:25 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:59:25.268 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:25 ntpd[17207]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:59:25.268 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:25 ntpd[17207]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:59:25.268 INFO:teuthology.orchestra.run.smithi155.stdout:25 Apr 13:59:25 ntpd[17207]: ntpd: time slew -0.000687 s 2024-04-25T13:59:25.268 INFO:teuthology.orchestra.run.smithi155.stdout:ntpd: time slew -0.000687s 2024-04-25T13:59:25.269 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:25 ntpd[17207]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T13:59:25.269 INFO:teuthology.orchestra.run.smithi155.stderr:25 Apr 13:59:25 ntpd[17207]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T13:59:25.374 INFO:teuthology.orchestra.run.smithi086.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T13:59:25.374 INFO:teuthology.orchestra.run.smithi086.stdout:============================================================================== 2024-04-25T13:59:25.375 INFO:teuthology.orchestra.run.smithi086.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T13:59:25.375 INFO:teuthology.orchestra.run.smithi086.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T13:59:25.375 INFO:teuthology.orchestra.run.smithi086.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T13:59:25.375 INFO:teuthology.orchestra.run.smithi086.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T13:59:25.385 INFO:teuthology.orchestra.run.smithi155.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T13:59:25.385 INFO:teuthology.orchestra.run.smithi155.stdout:============================================================================== 2024-04-25T13:59:25.385 INFO:teuthology.orchestra.run.smithi155.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T13:59:25.385 INFO:teuthology.orchestra.run.smithi155.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T13:59:25.385 INFO:teuthology.orchestra.run.smithi155.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T13:59:25.385 INFO:teuthology.orchestra.run.smithi155.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T13:59:25.386 INFO:teuthology.run_tasks:Running task install... 2024-04-25T13:59:25.395 DEBUG:teuthology.task.install:project ceph 2024-04-25T13:59:25.395 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}} 2024-04-25T13:59:25.395 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'} 2024-04-25T13:59:25.395 INFO:teuthology.task.install:Using flavor: default 2024-04-25T13:59:25.401 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T13:59:25.401 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T13:59:25.402 DEBUG:teuthology.orchestra.run.smithi086:> sudo apt-key list | grep Ceph 2024-04-25T13:59:25.402 DEBUG:teuthology.orchestra.run.smithi155:> sudo apt-key list | grep Ceph 2024-04-25T13:59:25.522 INFO:teuthology.orchestra.run.smithi086.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T13:59:25.529 INFO:teuthology.orchestra.run.smithi155.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T13:59:25.573 INFO:teuthology.orchestra.run.smithi086.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T13:59:25.574 INFO:teuthology.orchestra.run.smithi086.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T13:59:25.575 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T13:59:25.575 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T13:59:25.575 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:59:25.579 INFO:teuthology.orchestra.run.smithi155.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T13:59:25.580 INFO:teuthology.orchestra.run.smithi155.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T13:59:25.580 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T13:59:25.581 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T13:59:25.581 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:59:25.726 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default/ 2024-04-25T13:59:25.726 INFO:teuthology.task.install.deb:Package version is 19.0.0-3291-g994e2f20-1jammy 2024-04-25T13:59:25.797 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default/ 2024-04-25T13:59:25.797 INFO:teuthology.task.install.deb:Package version is 19.0.0-3291-g994e2f20-1jammy 2024-04-25T13:59:25.834 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T13:59:25.834 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T13:59:25.890 DEBUG:teuthology.orchestra.run.smithi155:> sudo apt-get update 2024-04-25T13:59:25.964 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T13:59:25.964 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T13:59:25.984 DEBUG:teuthology.orchestra.run.smithi086:> sudo apt-get update 2024-04-25T13:59:26.077 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T13:59:26.088 INFO:teuthology.orchestra.run.smithi155.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T13:59:26.176 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T13:59:26.176 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T13:59:26.183 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T13:59:26.186 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T13:59:26.188 INFO:teuthology.orchestra.run.smithi155.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T13:59:26.209 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T13:59:26.239 INFO:teuthology.orchestra.run.smithi155.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T13:59:26.305 INFO:teuthology.orchestra.run.smithi086.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T13:59:26.305 INFO:teuthology.orchestra.run.smithi155.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T13:59:26.346 INFO:teuthology.orchestra.run.smithi086.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T13:59:26.362 INFO:teuthology.orchestra.run.smithi155.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T13:59:26.398 INFO:teuthology.orchestra.run.smithi086.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T13:59:26.432 INFO:teuthology.orchestra.run.smithi155.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-04-25T13:59:26.439 INFO:teuthology.orchestra.run.smithi086.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,406 B] 2024-04-25T13:59:26.474 INFO:teuthology.orchestra.run.smithi086.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-04-25T13:59:26.488 INFO:teuthology.orchestra.run.smithi155.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,406 B] 2024-04-25T13:59:26.696 INFO:teuthology.orchestra.run.smithi155.stdout:Fetched 258 kB in 1s (440 kB/s) 2024-04-25T13:59:26.762 INFO:teuthology.orchestra.run.smithi086.stdout:Fetched 29.2 kB in 1s (52.7 kB/s) 2024-04-25T13:59:27.677 INFO:teuthology.orchestra.run.smithi155.stdout:Reading package lists... 2024-04-25T13:59:27.692 DEBUG:teuthology.orchestra.run.smithi155:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-3291-g994e2f20-1jammy cephadm=19.0.0-3291-g994e2f20-1jammy ceph-mds=19.0.0-3291-g994e2f20-1jammy ceph-mgr=19.0.0-3291-g994e2f20-1jammy ceph-common=19.0.0-3291-g994e2f20-1jammy ceph-fuse=19.0.0-3291-g994e2f20-1jammy ceph-test=19.0.0-3291-g994e2f20-1jammy ceph-volume=19.0.0-3291-g994e2f20-1jammy radosgw=19.0.0-3291-g994e2f20-1jammy python3-rados=19.0.0-3291-g994e2f20-1jammy python3-rgw=19.0.0-3291-g994e2f20-1jammy python3-cephfs=19.0.0-3291-g994e2f20-1jammy python3-rbd=19.0.0-3291-g994e2f20-1jammy libcephfs2=19.0.0-3291-g994e2f20-1jammy libcephfs-dev=19.0.0-3291-g994e2f20-1jammy librados2=19.0.0-3291-g994e2f20-1jammy librbd1=19.0.0-3291-g994e2f20-1jammy rbd-fuse=19.0.0-3291-g994e2f20-1jammy python3-cephfs=19.0.0-3291-g994e2f20-1jammy cephfs-shell=19.0.0-3291-g994e2f20-1jammy cephfs-top=19.0.0-3291-g994e2f20-1jammy cephfs-mirror=19.0.0-3291-g994e2f20-1jammy 2024-04-25T13:59:27.754 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2024-04-25T13:59:27.767 INFO:teuthology.orchestra.run.smithi155.stdout:Reading package lists... 2024-04-25T13:59:27.768 DEBUG:teuthology.orchestra.run.smithi086:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-3291-g994e2f20-1jammy cephadm=19.0.0-3291-g994e2f20-1jammy ceph-mds=19.0.0-3291-g994e2f20-1jammy ceph-mgr=19.0.0-3291-g994e2f20-1jammy ceph-common=19.0.0-3291-g994e2f20-1jammy ceph-fuse=19.0.0-3291-g994e2f20-1jammy ceph-test=19.0.0-3291-g994e2f20-1jammy ceph-volume=19.0.0-3291-g994e2f20-1jammy radosgw=19.0.0-3291-g994e2f20-1jammy python3-rados=19.0.0-3291-g994e2f20-1jammy python3-rgw=19.0.0-3291-g994e2f20-1jammy python3-cephfs=19.0.0-3291-g994e2f20-1jammy python3-rbd=19.0.0-3291-g994e2f20-1jammy libcephfs2=19.0.0-3291-g994e2f20-1jammy libcephfs-dev=19.0.0-3291-g994e2f20-1jammy librados2=19.0.0-3291-g994e2f20-1jammy librbd1=19.0.0-3291-g994e2f20-1jammy rbd-fuse=19.0.0-3291-g994e2f20-1jammy python3-cephfs=19.0.0-3291-g994e2f20-1jammy cephfs-shell=19.0.0-3291-g994e2f20-1jammy cephfs-top=19.0.0-3291-g994e2f20-1jammy cephfs-mirror=19.0.0-3291-g994e2f20-1jammy 2024-04-25T13:59:27.842 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2024-04-25T13:59:27.941 INFO:teuthology.orchestra.run.smithi155.stdout:Building dependency tree... 2024-04-25T13:59:27.941 INFO:teuthology.orchestra.run.smithi155.stdout:Reading state information... 2024-04-25T13:59:28.015 INFO:teuthology.orchestra.run.smithi086.stdout:Building dependency tree... 2024-04-25T13:59:28.016 INFO:teuthology.orchestra.run.smithi086.stdout:Reading state information... 2024-04-25T13:59:28.119 INFO:teuthology.orchestra.run.smithi155.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T13:59:28.119 INFO:teuthology.orchestra.run.smithi155.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T13:59:28.119 INFO:teuthology.orchestra.run.smithi155.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T13:59:28.119 INFO:teuthology.orchestra.run.smithi155.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T13:59:28.119 INFO:teuthology.orchestra.run.smithi155.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T13:59:28.120 INFO:teuthology.orchestra.run.smithi155.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T13:59:28.120 INFO:teuthology.orchestra.run.smithi155.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T13:59:28.120 INFO:teuthology.orchestra.run.smithi155.stdout:The following additional packages will be installed: 2024-04-25T13:59:28.120 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T13:59:28.120 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T13:59:28.121 INFO:teuthology.orchestra.run.smithi155.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T13:59:28.121 INFO:teuthology.orchestra.run.smithi155.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T13:59:28.122 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T13:59:28.122 INFO:teuthology.orchestra.run.smithi155.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T13:59:28.122 INFO:teuthology.orchestra.run.smithi155.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T13:59:28.122 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T13:59:28.122 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T13:59:28.122 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T13:59:28.122 INFO:teuthology.orchestra.run.smithi155.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T13:59:28.122 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T13:59:28.123 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T13:59:28.123 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T13:59:28.123 INFO:teuthology.orchestra.run.smithi155.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T13:59:28.123 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T13:59:28.123 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T13:59:28.123 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T13:59:28.123 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T13:59:28.123 INFO:teuthology.orchestra.run.smithi155.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T13:59:28.123 INFO:teuthology.orchestra.run.smithi155.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T13:59:28.123 INFO:teuthology.orchestra.run.smithi155.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T13:59:28.123 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T13:59:28.124 INFO:teuthology.orchestra.run.smithi155.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T13:59:28.124 INFO:teuthology.orchestra.run.smithi155.stdout:Suggested packages: 2024-04-25T13:59:28.124 INFO:teuthology.orchestra.run.smithi155.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T13:59:28.124 INFO:teuthology.orchestra.run.smithi155.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T13:59:28.124 INFO:teuthology.orchestra.run.smithi155.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T13:59:28.124 INFO:teuthology.orchestra.run.smithi155.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T13:59:28.124 INFO:teuthology.orchestra.run.smithi155.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T13:59:28.125 INFO:teuthology.orchestra.run.smithi155.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T13:59:28.125 INFO:teuthology.orchestra.run.smithi155.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T13:59:28.125 INFO:teuthology.orchestra.run.smithi155.stdout:Recommended packages: 2024-04-25T13:59:28.125 INFO:teuthology.orchestra.run.smithi155.stdout: btrfs-tools 2024-04-25T13:59:28.189 INFO:teuthology.orchestra.run.smithi086.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T13:59:28.190 INFO:teuthology.orchestra.run.smithi086.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T13:59:28.190 INFO:teuthology.orchestra.run.smithi086.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T13:59:28.190 INFO:teuthology.orchestra.run.smithi086.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T13:59:28.190 INFO:teuthology.orchestra.run.smithi086.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T13:59:28.190 INFO:teuthology.orchestra.run.smithi086.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T13:59:28.190 INFO:teuthology.orchestra.run.smithi086.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T13:59:28.191 INFO:teuthology.orchestra.run.smithi086.stdout:The following additional packages will be installed: 2024-04-25T13:59:28.191 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T13:59:28.191 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T13:59:28.191 INFO:teuthology.orchestra.run.smithi086.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T13:59:28.191 INFO:teuthology.orchestra.run.smithi086.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T13:59:28.193 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T13:59:28.193 INFO:teuthology.orchestra.run.smithi086.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T13:59:28.193 INFO:teuthology.orchestra.run.smithi086.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T13:59:28.193 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T13:59:28.193 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T13:59:28.193 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T13:59:28.193 INFO:teuthology.orchestra.run.smithi086.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T13:59:28.193 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T13:59:28.193 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T13:59:28.193 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T13:59:28.193 INFO:teuthology.orchestra.run.smithi086.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T13:59:28.193 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T13:59:28.194 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T13:59:28.194 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T13:59:28.194 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T13:59:28.194 INFO:teuthology.orchestra.run.smithi086.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T13:59:28.194 INFO:teuthology.orchestra.run.smithi086.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T13:59:28.194 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T13:59:28.194 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T13:59:28.194 INFO:teuthology.orchestra.run.smithi086.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T13:59:28.194 INFO:teuthology.orchestra.run.smithi086.stdout:Suggested packages: 2024-04-25T13:59:28.195 INFO:teuthology.orchestra.run.smithi086.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T13:59:28.195 INFO:teuthology.orchestra.run.smithi086.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T13:59:28.195 INFO:teuthology.orchestra.run.smithi086.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T13:59:28.195 INFO:teuthology.orchestra.run.smithi086.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T13:59:28.195 INFO:teuthology.orchestra.run.smithi086.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T13:59:28.195 INFO:teuthology.orchestra.run.smithi086.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T13:59:28.195 INFO:teuthology.orchestra.run.smithi086.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T13:59:28.195 INFO:teuthology.orchestra.run.smithi086.stdout:Recommended packages: 2024-04-25T13:59:28.195 INFO:teuthology.orchestra.run.smithi086.stdout: btrfs-tools 2024-04-25T13:59:28.275 INFO:teuthology.orchestra.run.smithi155.stdout:The following NEW packages will be installed: 2024-04-25T13:59:28.275 INFO:teuthology.orchestra.run.smithi155.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T13:59:28.275 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T13:59:28.276 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T13:59:28.276 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T13:59:28.276 INFO:teuthology.orchestra.run.smithi155.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T13:59:28.276 INFO:teuthology.orchestra.run.smithi155.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T13:59:28.276 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T13:59:28.276 INFO:teuthology.orchestra.run.smithi155.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T13:59:28.277 INFO:teuthology.orchestra.run.smithi155.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T13:59:28.277 INFO:teuthology.orchestra.run.smithi155.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T13:59:28.277 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T13:59:28.277 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T13:59:28.277 INFO:teuthology.orchestra.run.smithi155.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T13:59:28.278 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T13:59:28.278 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T13:59:28.278 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T13:59:28.278 INFO:teuthology.orchestra.run.smithi155.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T13:59:28.278 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T13:59:28.278 INFO:teuthology.orchestra.run.smithi155.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T13:59:28.278 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T13:59:28.278 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T13:59:28.278 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T13:59:28.278 INFO:teuthology.orchestra.run.smithi155.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T13:59:28.278 INFO:teuthology.orchestra.run.smithi155.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T13:59:28.278 INFO:teuthology.orchestra.run.smithi155.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T13:59:28.278 INFO:teuthology.orchestra.run.smithi155.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T13:59:28.279 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T13:59:28.279 INFO:teuthology.orchestra.run.smithi155.stdout: socat unzip xmlstarlet zip 2024-04-25T13:59:28.343 INFO:teuthology.orchestra.run.smithi086.stdout:The following NEW packages will be installed: 2024-04-25T13:59:28.343 INFO:teuthology.orchestra.run.smithi086.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T13:59:28.343 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T13:59:28.343 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T13:59:28.345 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T13:59:28.345 INFO:teuthology.orchestra.run.smithi086.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T13:59:28.345 INFO:teuthology.orchestra.run.smithi086.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T13:59:28.345 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T13:59:28.345 INFO:teuthology.orchestra.run.smithi086.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T13:59:28.345 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T13:59:28.345 INFO:teuthology.orchestra.run.smithi086.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T13:59:28.345 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T13:59:28.346 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T13:59:28.346 INFO:teuthology.orchestra.run.smithi086.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T13:59:28.346 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T13:59:28.346 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T13:59:28.346 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T13:59:28.346 INFO:teuthology.orchestra.run.smithi086.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T13:59:28.346 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T13:59:28.346 INFO:teuthology.orchestra.run.smithi086.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T13:59:28.346 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T13:59:28.346 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T13:59:28.346 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T13:59:28.346 INFO:teuthology.orchestra.run.smithi086.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T13:59:28.346 INFO:teuthology.orchestra.run.smithi086.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T13:59:28.346 INFO:teuthology.orchestra.run.smithi086.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T13:59:28.346 INFO:teuthology.orchestra.run.smithi086.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T13:59:28.347 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T13:59:28.347 INFO:teuthology.orchestra.run.smithi086.stdout: socat unzip xmlstarlet zip 2024-04-25T13:59:28.353 INFO:teuthology.orchestra.run.smithi155.stdout:0 upgraded, 124 newly installed, 0 to remove and 305 not upgraded. 2024-04-25T13:59:28.353 INFO:teuthology.orchestra.run.smithi155.stdout:Need to get 175 MB of archives. 2024-04-25T13:59:28.353 INFO:teuthology.orchestra.run.smithi155.stdout:After this operation, 887 MB of additional disk space will be used. 2024-04-25T13:59:28.354 INFO:teuthology.orchestra.run.smithi155.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T13:59:28.375 INFO:teuthology.orchestra.run.smithi155.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T13:59:28.388 INFO:teuthology.orchestra.run.smithi155.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T13:59:28.440 INFO:teuthology.orchestra.run.smithi155.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T13:59:28.465 INFO:teuthology.orchestra.run.smithi155.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T13:59:28.470 INFO:teuthology.orchestra.run.smithi155.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T13:59:28.484 INFO:teuthology.orchestra.run.smithi155.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T13:59:28.493 INFO:teuthology.orchestra.run.smithi086.stdout:0 upgraded, 124 newly installed, 0 to remove and 305 not upgraded. 2024-04-25T13:59:28.493 INFO:teuthology.orchestra.run.smithi086.stdout:Need to get 175 MB of archives. 2024-04-25T13:59:28.493 INFO:teuthology.orchestra.run.smithi086.stdout:After this operation, 887 MB of additional disk space will be used. 2024-04-25T13:59:28.493 INFO:teuthology.orchestra.run.smithi086.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-3291-g994e2f20-1jammy [3,525 kB] 2024-04-25T13:59:28.504 INFO:teuthology.orchestra.run.smithi155.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-3291-g994e2f20-1jammy [3,525 kB] 2024-04-25T13:59:28.536 INFO:teuthology.orchestra.run.smithi155.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T13:59:28.539 INFO:teuthology.orchestra.run.smithi155.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T13:59:28.541 INFO:teuthology.orchestra.run.smithi086.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T13:59:28.551 INFO:teuthology.orchestra.run.smithi155.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T13:59:28.556 INFO:teuthology.orchestra.run.smithi155.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T13:59:28.557 INFO:teuthology.orchestra.run.smithi155.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T13:59:28.565 INFO:teuthology.orchestra.run.smithi155.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T13:59:28.567 INFO:teuthology.orchestra.run.smithi155.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T13:59:28.569 INFO:teuthology.orchestra.run.smithi155.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T13:59:28.570 INFO:teuthology.orchestra.run.smithi155.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T13:59:28.573 INFO:teuthology.orchestra.run.smithi155.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T13:59:28.597 INFO:teuthology.orchestra.run.smithi155.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T13:59:28.599 INFO:teuthology.orchestra.run.smithi155.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T13:59:28.600 INFO:teuthology.orchestra.run.smithi155.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T13:59:28.601 INFO:teuthology.orchestra.run.smithi155.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T13:59:28.601 INFO:teuthology.orchestra.run.smithi155.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T13:59:28.602 INFO:teuthology.orchestra.run.smithi155.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T13:59:28.603 INFO:teuthology.orchestra.run.smithi155.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T13:59:28.603 INFO:teuthology.orchestra.run.smithi155.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T13:59:28.603 INFO:teuthology.orchestra.run.smithi155.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T13:59:28.619 INFO:teuthology.orchestra.run.smithi155.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T13:59:28.624 INFO:teuthology.orchestra.run.smithi086.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T13:59:28.626 INFO:teuthology.orchestra.run.smithi155.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T13:59:28.626 INFO:teuthology.orchestra.run.smithi155.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T13:59:28.629 INFO:teuthology.orchestra.run.smithi155.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T13:59:28.630 INFO:teuthology.orchestra.run.smithi155.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T13:59:28.631 INFO:teuthology.orchestra.run.smithi155.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T13:59:28.631 INFO:teuthology.orchestra.run.smithi155.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T13:59:28.632 INFO:teuthology.orchestra.run.smithi155.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T13:59:28.632 INFO:teuthology.orchestra.run.smithi155.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T13:59:28.639 INFO:teuthology.orchestra.run.smithi155.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T13:59:28.656 INFO:teuthology.orchestra.run.smithi155.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T13:59:28.657 INFO:teuthology.orchestra.run.smithi155.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T13:59:28.657 INFO:teuthology.orchestra.run.smithi155.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T13:59:28.668 INFO:teuthology.orchestra.run.smithi155.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T13:59:28.668 INFO:teuthology.orchestra.run.smithi155.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T13:59:28.669 INFO:teuthology.orchestra.run.smithi155.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T13:59:28.669 INFO:teuthology.orchestra.run.smithi155.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T13:59:28.670 INFO:teuthology.orchestra.run.smithi086.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T13:59:28.671 INFO:teuthology.orchestra.run.smithi155.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T13:59:28.682 INFO:teuthology.orchestra.run.smithi155.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T13:59:28.719 INFO:teuthology.orchestra.run.smithi155.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T13:59:28.721 INFO:teuthology.orchestra.run.smithi155.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T13:59:28.722 INFO:teuthology.orchestra.run.smithi155.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T13:59:28.725 INFO:teuthology.orchestra.run.smithi155.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T13:59:28.725 INFO:teuthology.orchestra.run.smithi155.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T13:59:28.726 INFO:teuthology.orchestra.run.smithi155.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T13:59:28.728 INFO:teuthology.orchestra.run.smithi155.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T13:59:28.755 INFO:teuthology.orchestra.run.smithi086.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-3291-g994e2f20-1jammy [3,287 kB] 2024-04-25T13:59:28.866 INFO:teuthology.orchestra.run.smithi155.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T13:59:28.867 INFO:teuthology.orchestra.run.smithi155.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T13:59:28.869 INFO:teuthology.orchestra.run.smithi155.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T13:59:28.869 INFO:teuthology.orchestra.run.smithi155.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T13:59:28.871 INFO:teuthology.orchestra.run.smithi155.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T13:59:28.876 INFO:teuthology.orchestra.run.smithi086.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T13:59:28.898 INFO:teuthology.orchestra.run.smithi086.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-3291-g994e2f20-1jammy [726 kB] 2024-04-25T13:59:28.911 INFO:teuthology.orchestra.run.smithi155.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T13:59:28.913 INFO:teuthology.orchestra.run.smithi155.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T13:59:28.914 INFO:teuthology.orchestra.run.smithi155.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T13:59:28.923 INFO:teuthology.orchestra.run.smithi086.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-3291-g994e2f20-1jammy [356 kB] 2024-04-25T13:59:28.935 INFO:teuthology.orchestra.run.smithi155.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T13:59:28.935 INFO:teuthology.orchestra.run.smithi155.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T13:59:28.944 INFO:teuthology.orchestra.run.smithi155.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T13:59:28.945 INFO:teuthology.orchestra.run.smithi155.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T13:59:28.945 INFO:teuthology.orchestra.run.smithi155.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T13:59:28.946 INFO:teuthology.orchestra.run.smithi155.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T13:59:28.946 INFO:teuthology.orchestra.run.smithi155.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T13:59:28.947 INFO:teuthology.orchestra.run.smithi155.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T13:59:28.948 INFO:teuthology.orchestra.run.smithi086.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-3291-g994e2f20-1jammy [32.8 kB] 2024-04-25T13:59:28.950 INFO:teuthology.orchestra.run.smithi086.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-3291-g994e2f20-1jammy [180 kB] 2024-04-25T13:59:28.953 INFO:teuthology.orchestra.run.smithi155.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T13:59:28.959 INFO:teuthology.orchestra.run.smithi086.stdout:Get:11 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-3291-g994e2f20-1jammy [67.1 kB] 2024-04-25T13:59:28.961 INFO:teuthology.orchestra.run.smithi086.stdout:Get:12 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-3291-g994e2f20-1jammy [330 kB] 2024-04-25T13:59:28.971 INFO:teuthology.orchestra.run.smithi155.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T13:59:28.972 INFO:teuthology.orchestra.run.smithi086.stdout:Get:13 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-3291-g994e2f20-1jammy [7,189 kB] 2024-04-25T13:59:28.973 INFO:teuthology.orchestra.run.smithi155.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T13:59:28.974 INFO:teuthology.orchestra.run.smithi155.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T13:59:28.976 INFO:teuthology.orchestra.run.smithi086.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T13:59:28.988 INFO:teuthology.orchestra.run.smithi086.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T13:59:28.993 INFO:teuthology.orchestra.run.smithi155.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T13:59:28.996 INFO:teuthology.orchestra.run.smithi155.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T13:59:28.997 INFO:teuthology.orchestra.run.smithi155.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T13:59:28.998 INFO:teuthology.orchestra.run.smithi155.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T13:59:28.999 INFO:teuthology.orchestra.run.smithi155.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T13:59:29.004 INFO:teuthology.orchestra.run.smithi155.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T13:59:29.005 INFO:teuthology.orchestra.run.smithi155.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T13:59:29.015 INFO:teuthology.orchestra.run.smithi155.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T13:59:29.015 INFO:teuthology.orchestra.run.smithi155.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T13:59:29.016 INFO:teuthology.orchestra.run.smithi155.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T13:59:29.040 INFO:teuthology.orchestra.run.smithi155.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T13:59:29.042 INFO:teuthology.orchestra.run.smithi086.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T13:59:29.042 INFO:teuthology.orchestra.run.smithi155.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T13:59:29.043 INFO:teuthology.orchestra.run.smithi155.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T13:59:29.055 INFO:teuthology.orchestra.run.smithi155.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T13:59:29.056 INFO:teuthology.orchestra.run.smithi155.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T13:59:29.057 INFO:teuthology.orchestra.run.smithi155.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T13:59:29.060 INFO:teuthology.orchestra.run.smithi155.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T13:59:29.083 INFO:teuthology.orchestra.run.smithi155.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T13:59:29.216 INFO:teuthology.orchestra.run.smithi086.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T13:59:29.225 INFO:teuthology.orchestra.run.smithi086.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T13:59:29.253 INFO:teuthology.orchestra.run.smithi086.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T13:59:29.261 INFO:teuthology.orchestra.run.smithi086.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T13:59:29.315 INFO:teuthology.orchestra.run.smithi086.stdout:Get:21 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-3291-g994e2f20-1jammy [112 kB] 2024-04-25T13:59:29.317 INFO:teuthology.orchestra.run.smithi086.stdout:Get:22 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-3291-g994e2f20-1jammy [471 kB] 2024-04-25T13:59:29.345 INFO:teuthology.orchestra.run.smithi086.stdout:Get:23 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-3291-g994e2f20-1jammy [25.6 MB] 2024-04-25T13:59:29.426 INFO:teuthology.orchestra.run.smithi086.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T13:59:30.015 INFO:teuthology.orchestra.run.smithi086.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T13:59:30.030 INFO:teuthology.orchestra.run.smithi086.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T13:59:30.046 INFO:teuthology.orchestra.run.smithi086.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T13:59:30.055 INFO:teuthology.orchestra.run.smithi086.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T13:59:30.055 INFO:teuthology.orchestra.run.smithi086.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T13:59:30.072 INFO:teuthology.orchestra.run.smithi086.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T13:59:30.109 INFO:teuthology.orchestra.run.smithi086.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T13:59:30.123 INFO:teuthology.orchestra.run.smithi086.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T13:59:30.127 INFO:teuthology.orchestra.run.smithi086.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T13:59:30.127 INFO:teuthology.orchestra.run.smithi086.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T13:59:30.298 INFO:teuthology.orchestra.run.smithi086.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T13:59:30.382 INFO:teuthology.orchestra.run.smithi086.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T13:59:30.388 INFO:teuthology.orchestra.run.smithi086.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T13:59:30.404 INFO:teuthology.orchestra.run.smithi155.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-3291-g994e2f20-1jammy [3,287 kB] 2024-04-25T13:59:30.463 INFO:teuthology.orchestra.run.smithi086.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T13:59:30.492 INFO:teuthology.orchestra.run.smithi086.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T13:59:30.503 INFO:teuthology.orchestra.run.smithi086.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T13:59:30.545 INFO:teuthology.orchestra.run.smithi086.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T13:59:30.686 INFO:teuthology.orchestra.run.smithi086.stdout:Get:42 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-3291-g994e2f20-1jammy [4,917 kB] 2024-04-25T13:59:30.723 INFO:teuthology.orchestra.run.smithi086.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T13:59:30.737 INFO:teuthology.orchestra.run.smithi086.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T13:59:30.744 INFO:teuthology.orchestra.run.smithi086.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T13:59:30.749 INFO:teuthology.orchestra.run.smithi086.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T13:59:30.934 INFO:teuthology.orchestra.run.smithi086.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T13:59:30.935 INFO:teuthology.orchestra.run.smithi086.stdout:Get:48 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-3291-g994e2f20-1jammy [244 kB] 2024-04-25T13:59:30.956 INFO:teuthology.orchestra.run.smithi086.stdout:Get:49 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-3291-g994e2f20-1jammy [125 kB] 2024-04-25T13:59:30.961 INFO:teuthology.orchestra.run.smithi086.stdout:Get:50 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-3291-g994e2f20-1jammy [1,542 kB] 2024-04-25T13:59:31.016 INFO:teuthology.orchestra.run.smithi086.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T13:59:31.032 INFO:teuthology.orchestra.run.smithi086.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T13:59:31.041 INFO:teuthology.orchestra.run.smithi086.stdout:Get:53 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-3291-g994e2f20-1jammy [6,148 kB] 2024-04-25T13:59:31.217 INFO:teuthology.orchestra.run.smithi086.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T13:59:31.267 INFO:teuthology.orchestra.run.smithi086.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T13:59:31.280 INFO:teuthology.orchestra.run.smithi086.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T13:59:31.409 INFO:teuthology.orchestra.run.smithi086.stdout:Get:57 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-3291-g994e2f20-1jammy [22.5 MB] 2024-04-25T13:59:31.438 INFO:teuthology.orchestra.run.smithi086.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T13:59:31.439 INFO:teuthology.orchestra.run.smithi086.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T13:59:31.443 INFO:teuthology.orchestra.run.smithi086.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T13:59:31.448 INFO:teuthology.orchestra.run.smithi086.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T13:59:31.462 INFO:teuthology.orchestra.run.smithi086.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T13:59:31.646 INFO:teuthology.orchestra.run.smithi086.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T13:59:31.833 INFO:teuthology.orchestra.run.smithi155.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-3291-g994e2f20-1jammy [726 kB] 2024-04-25T13:59:31.899 INFO:teuthology.orchestra.run.smithi086.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T13:59:32.032 INFO:teuthology.orchestra.run.smithi086.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T13:59:32.075 INFO:teuthology.orchestra.run.smithi155.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-3291-g994e2f20-1jammy [356 kB] 2024-04-25T13:59:32.075 INFO:teuthology.orchestra.run.smithi086.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T13:59:32.166 INFO:teuthology.orchestra.run.smithi086.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T13:59:32.175 INFO:teuthology.orchestra.run.smithi086.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T13:59:32.187 INFO:teuthology.orchestra.run.smithi086.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T13:59:32.215 INFO:teuthology.orchestra.run.smithi155.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-3291-g994e2f20-1jammy [32.8 kB] 2024-04-25T13:59:32.223 INFO:teuthology.orchestra.run.smithi155.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-3291-g994e2f20-1jammy [180 kB] 2024-04-25T13:59:32.231 INFO:teuthology.orchestra.run.smithi086.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T13:59:32.279 INFO:teuthology.orchestra.run.smithi155.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-3291-g994e2f20-1jammy [67.1 kB] 2024-04-25T13:59:32.311 INFO:teuthology.orchestra.run.smithi155.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-3291-g994e2f20-1jammy [330 kB] 2024-04-25T13:59:32.427 INFO:teuthology.orchestra.run.smithi155.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-3291-g994e2f20-1jammy [7,189 kB] 2024-04-25T13:59:32.493 INFO:teuthology.orchestra.run.smithi086.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T13:59:32.494 INFO:teuthology.orchestra.run.smithi086.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T13:59:32.495 INFO:teuthology.orchestra.run.smithi086.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T13:59:32.546 INFO:teuthology.orchestra.run.smithi086.stdout:Get:74 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-3291-g994e2f20-1jammy [14.1 kB] 2024-04-25T13:59:32.546 INFO:teuthology.orchestra.run.smithi086.stdout:Get:75 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-3291-g994e2f20-1jammy [856 kB] 2024-04-25T13:59:32.583 INFO:teuthology.orchestra.run.smithi086.stdout:Get:76 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-3291-g994e2f20-1jammy [2,367 kB] 2024-04-25T13:59:32.659 INFO:teuthology.orchestra.run.smithi086.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T13:59:32.888 INFO:teuthology.orchestra.run.smithi086.stdout:Get:78 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-3291-g994e2f20-1jammy [791 kB] 2024-04-25T13:59:32.894 INFO:teuthology.orchestra.run.smithi086.stdout:Get:79 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-3291-g994e2f20-1jammy [148 kB] 2024-04-25T13:59:32.895 INFO:teuthology.orchestra.run.smithi086.stdout:Get:80 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-3291-g994e2f20-1jammy [2,297 kB] 2024-04-25T13:59:33.003 INFO:teuthology.orchestra.run.smithi086.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T13:59:33.054 INFO:teuthology.orchestra.run.smithi086.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-3291-g994e2f20-1jammy [8,625 kB] 2024-04-25T13:59:33.404 INFO:teuthology.orchestra.run.smithi086.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T13:59:33.413 INFO:teuthology.orchestra.run.smithi086.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T13:59:33.414 INFO:teuthology.orchestra.run.smithi086.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T13:59:33.483 INFO:teuthology.orchestra.run.smithi086.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T13:59:33.484 INFO:teuthology.orchestra.run.smithi086.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T13:59:33.484 INFO:teuthology.orchestra.run.smithi086.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T13:59:33.485 INFO:teuthology.orchestra.run.smithi086.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T13:59:33.486 INFO:teuthology.orchestra.run.smithi086.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T13:59:33.487 INFO:teuthology.orchestra.run.smithi086.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T13:59:33.651 INFO:teuthology.orchestra.run.smithi086.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T13:59:33.693 INFO:teuthology.orchestra.run.smithi086.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-3291-g994e2f20-1jammy [14.2 kB] 2024-04-25T13:59:33.694 INFO:teuthology.orchestra.run.smithi086.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-3291-g994e2f20-1jammy [44.9 MB] 2024-04-25T13:59:33.860 INFO:teuthology.orchestra.run.smithi086.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T13:59:34.189 INFO:teuthology.orchestra.run.smithi086.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T13:59:34.236 INFO:teuthology.orchestra.run.smithi086.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T13:59:34.262 INFO:teuthology.orchestra.run.smithi086.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T13:59:34.271 INFO:teuthology.orchestra.run.smithi086.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T13:59:34.317 INFO:teuthology.orchestra.run.smithi086.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T13:59:34.343 INFO:teuthology.orchestra.run.smithi086.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T13:59:34.344 INFO:teuthology.orchestra.run.smithi086.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T13:59:34.351 INFO:teuthology.orchestra.run.smithi086.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T13:59:34.354 INFO:teuthology.orchestra.run.smithi086.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T13:59:34.551 INFO:teuthology.orchestra.run.smithi086.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T13:59:34.810 INFO:teuthology.orchestra.run.smithi086.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T13:59:35.081 INFO:teuthology.orchestra.run.smithi086.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T13:59:35.083 INFO:teuthology.orchestra.run.smithi086.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T13:59:35.099 INFO:teuthology.orchestra.run.smithi086.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T13:59:35.104 INFO:teuthology.orchestra.run.smithi086.stdout:Get:110 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T13:59:35.143 INFO:teuthology.orchestra.run.smithi086.stdout:Get:111 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T13:59:35.158 INFO:teuthology.orchestra.run.smithi086.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T13:59:35.248 INFO:teuthology.orchestra.run.smithi086.stdout:Get:113 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T13:59:35.250 INFO:teuthology.orchestra.run.smithi086.stdout:Get:114 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T13:59:35.251 INFO:teuthology.orchestra.run.smithi086.stdout:Get:115 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T13:59:35.434 INFO:teuthology.orchestra.run.smithi086.stdout:Get:116 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T13:59:35.479 INFO:teuthology.orchestra.run.smithi155.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-3291-g994e2f20-1jammy [112 kB] 2024-04-25T13:59:35.520 INFO:teuthology.orchestra.run.smithi155.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-3291-g994e2f20-1jammy [471 kB] 2024-04-25T13:59:35.640 INFO:teuthology.orchestra.run.smithi155.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-3291-g994e2f20-1jammy [25.6 MB] 2024-04-25T13:59:35.699 INFO:teuthology.orchestra.run.smithi086.stdout:Get:117 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T13:59:35.928 INFO:teuthology.orchestra.run.smithi086.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-3291-g994e2f20-1jammy [124 kB] 2024-04-25T13:59:35.929 INFO:teuthology.orchestra.run.smithi086.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-3291-g994e2f20-1jammy [236 kB] 2024-04-25T13:59:35.931 INFO:teuthology.orchestra.run.smithi086.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-3291-g994e2f20-1jammy [33.9 kB] 2024-04-25T13:59:35.931 INFO:teuthology.orchestra.run.smithi086.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-3291-g994e2f20-1jammy [23.4 kB] 2024-04-25T13:59:35.932 INFO:teuthology.orchestra.run.smithi086.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-3291-g994e2f20-1jammy [39.0 kB] 2024-04-25T13:59:35.932 INFO:teuthology.orchestra.run.smithi086.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-3291-g994e2f20-1jammy [14.2 MB] 2024-04-25T13:59:36.499 INFO:teuthology.orchestra.run.smithi086.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-3291-g994e2f20-1jammy [92.6 kB] 2024-04-25T13:59:37.135 INFO:teuthology.orchestra.run.smithi086.stdout:Fetched 175 MB in 8s (21.5 MB/s) 2024-04-25T13:59:37.273 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T13:59:37.339 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132922 files and directories currently installed.) 2024-04-25T13:59:37.342 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T13:59:37.380 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T13:59:37.685 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T13:59:37.696 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T13:59:37.733 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T13:59:38.061 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T13:59:38.074 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T13:59:38.157 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T13:59:38.522 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T13:59:38.536 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T13:59:38.648 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T13:59:39.001 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T13:59:39.015 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T13:59:39.075 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T13:59:39.362 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T13:59:39.377 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T13:59:39.420 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T13:59:40.039 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T13:59:40.054 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T13:59:40.133 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T13:59:40.502 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T13:59:40.517 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T13:59:40.571 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T13:59:40.922 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T13:59:40.937 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T13:59:40.980 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T13:59:41.408 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T13:59:41.423 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T13:59:41.475 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T13:59:41.852 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package librados2. 2024-04-25T13:59:41.867 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../010-librados2_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:59:41.919 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking librados2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:59:42.447 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package librbd1. 2024-04-25T13:59:42.461 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../011-librbd1_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:59:42.515 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking librbd1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:59:42.712 INFO:teuthology.orchestra.run.smithi155.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-3291-g994e2f20-1jammy [4,917 kB] 2024-04-25T13:59:43.051 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T13:59:43.066 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:59:43.118 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libcephfs2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:59:43.446 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-rados. 2024-04-25T13:59:43.460 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../013-python3-rados_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:59:43.545 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-rados (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:59:43.557 INFO:teuthology.orchestra.run.smithi155.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-3291-g994e2f20-1jammy [244 kB] 2024-04-25T13:59:43.594 INFO:teuthology.orchestra.run.smithi155.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-3291-g994e2f20-1jammy [125 kB] 2024-04-25T13:59:43.614 INFO:teuthology.orchestra.run.smithi155.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-3291-g994e2f20-1jammy [1,542 kB] 2024-04-25T13:59:43.866 INFO:teuthology.orchestra.run.smithi155.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-3291-g994e2f20-1jammy [6,148 kB] 2024-04-25T13:59:43.931 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T13:59:43.946 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T13:59:44.031 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-ceph-argparse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:59:44.292 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T13:59:44.307 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:59:44.350 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cephfs (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:59:44.745 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T13:59:44.759 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T13:59:44.823 INFO:teuthology.orchestra.run.smithi155.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-3291-g994e2f20-1jammy [22.5 MB] 2024-04-25T13:59:45.061 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:59:45.448 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T13:59:45.465 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T13:59:45.548 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T13:59:45.842 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T13:59:45.856 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:59:45.908 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-rbd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:59:46.295 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T13:59:46.310 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T13:59:46.353 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T13:59:46.672 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T13:59:46.686 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T13:59:46.738 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T13:59:47.108 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T13:59:47.123 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T13:59:47.174 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T13:59:47.578 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package lua5.1. 2024-04-25T13:59:47.592 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T13:59:47.644 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T13:59:47.989 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package lua-any. 2024-04-25T13:59:48.003 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T13:59:48.055 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T13:59:48.366 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package zip. 2024-04-25T13:59:48.381 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T13:59:48.449 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T13:59:48.827 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package unzip. 2024-04-25T13:59:48.842 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T13:59:48.919 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T13:59:49.415 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package luarocks. 2024-04-25T13:59:49.429 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T13:59:49.481 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T13:59:49.904 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package librgw2. 2024-04-25T13:59:49.918 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../027-librgw2_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:59:49.966 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking librgw2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:59:50.565 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T13:59:50.580 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:59:50.632 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-rgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:59:51.068 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T13:59:51.083 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T13:59:51.134 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T13:59:51.331 INFO:teuthology.orchestra.run.smithi155.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-3291-g994e2f20-1jammy [14.1 kB] 2024-04-25T13:59:51.342 INFO:teuthology.orchestra.run.smithi155.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-3291-g994e2f20-1jammy [856 kB] 2024-04-25T13:59:51.512 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T13:59:51.527 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:59:51.579 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libradosstriper1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:59:51.642 INFO:teuthology.orchestra.run.smithi155.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-3291-g994e2f20-1jammy [2,367 kB] 2024-04-25T13:59:52.074 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-common. 2024-04-25T13:59:52.088 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../031-ceph-common_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:59:52.132 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:59:52.872 INFO:teuthology.orchestra.run.smithi155.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-3291-g994e2f20-1jammy [791 kB] 2024-04-25T13:59:53.254 INFO:teuthology.orchestra.run.smithi155.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-3291-g994e2f20-1jammy [148 kB] 2024-04-25T13:59:53.321 INFO:teuthology.orchestra.run.smithi155.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-3291-g994e2f20-1jammy [2,297 kB] 2024-04-25T13:59:53.479 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-base. 2024-04-25T13:59:53.493 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../032-ceph-base_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:59:53.555 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-base (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:59:54.099 INFO:teuthology.orchestra.run.smithi155.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-3291-g994e2f20-1jammy [8,625 kB] 2024-04-25T13:59:54.116 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T13:59:54.129 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T13:59:54.174 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T13:59:54.510 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T13:59:54.524 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T13:59:54.569 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T13:59:54.921 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T13:59:54.936 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T13:59:54.979 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T13:59:55.599 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T13:59:55.614 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T13:59:55.665 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T13:59:56.069 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T13:59:56.084 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T13:59:56.135 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T13:59:56.463 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T13:59:56.478 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T13:59:56.522 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T13:59:56.915 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-portend. 2024-04-25T13:59:56.930 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T13:59:56.982 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T13:59:57.104 INFO:teuthology.orchestra.run.smithi155.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-3291-g994e2f20-1jammy [14.2 kB] 2024-04-25T13:59:57.115 INFO:teuthology.orchestra.run.smithi155.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-3291-g994e2f20-1jammy [44.9 MB] 2024-04-25T13:59:57.385 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T13:59:57.400 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T13:59:57.451 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T13:59:57.837 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T13:59:57.852 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T13:59:57.904 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T13:59:58.349 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T13:59:58.365 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T13:59:58.416 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T13:59:58.810 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T13:59:58.825 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T13:59:58.877 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T13:59:59.263 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T13:59:59.278 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T13:59:59.329 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T13:59:59.682 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-mako. 2024-04-25T13:59:59.697 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T13:59:59.740 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T14:00:00.100 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T14:00:00.115 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T14:00:00.160 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T14:00:00.463 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T14:00:00.478 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T14:00:00.521 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T14:00:00.832 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-webob. 2024-04-25T14:00:00.847 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T14:00:00.881 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T14:00:01.918 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T14:00:01.933 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T14:00:01.979 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T14:00:02.271 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T14:00:02.286 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T14:00:02.329 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T14:00:02.657 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-paste. 2024-04-25T14:00:02.672 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T14:00:02.715 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T14:00:03.043 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T14:00:03.057 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T14:00:03.093 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T14:00:03.421 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T14:00:03.436 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T14:00:03.479 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T14:00:03.781 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T14:00:03.797 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T14:00:03.865 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T14:00:04.185 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T14:00:04.200 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T14:00:04.243 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T14:00:04.622 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T14:00:04.637 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T14:00:04.680 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T14:00:05.025 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-idna. 2024-04-25T14:00:05.036 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T14:00:05.074 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T14:00:05.411 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T14:00:05.425 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T14:00:05.469 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T14:00:06.005 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-requests. 2024-04-25T14:00:06.020 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T14:00:06.063 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T14:00:06.424 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T14:00:06.437 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T14:00:06.482 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T14:00:06.811 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T14:00:06.824 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T14:00:06.860 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-modules-core (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:07.273 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T14:00:07.286 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:07.332 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:07.618 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T14:00:07.630 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T14:00:07.667 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T14:00:07.945 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T14:00:07.958 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T14:00:08.003 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T14:00:08.365 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T14:00:08.378 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:08.422 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:08.868 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T14:00:08.883 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:08.926 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mon (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:09.479 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T14:00:09.492 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:09.529 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-osd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:10.279 INFO:teuthology.orchestra.run.smithi155.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-3291-g994e2f20-1jammy [124 kB] 2024-04-25T14:00:10.300 INFO:teuthology.orchestra.run.smithi155.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-3291-g994e2f20-1jammy [236 kB] 2024-04-25T14:00:10.340 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph. 2024-04-25T14:00:10.353 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../068-ceph_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:10.364 INFO:teuthology.orchestra.run.smithi155.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-3291-g994e2f20-1jammy [33.9 kB] 2024-04-25T14:00:10.371 INFO:teuthology.orchestra.run.smithi155.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-3291-g994e2f20-1jammy [23.4 kB] 2024-04-25T14:00:10.372 INFO:teuthology.orchestra.run.smithi155.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-3291-g994e2f20-1jammy [39.0 kB] 2024-04-25T14:00:10.389 INFO:teuthology.orchestra.run.smithi155.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-3291-g994e2f20-1jammy [14.2 MB] 2024-04-25T14:00:10.398 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:10.759 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T14:00:10.773 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:10.817 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:11.187 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T14:00:11.201 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:11.237 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mds (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:11.584 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package cephadm. 2024-04-25T14:00:11.598 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../071-cephadm_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:11.648 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:11.950 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T14:00:11.964 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T14:00:12.008 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T14:00:12.319 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T14:00:12.334 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T14:00:12.377 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T14:00:13.376 INFO:teuthology.orchestra.run.smithi155.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-3291-g994e2f20-1jammy [92.6 kB] 2024-04-25T14:00:13.398 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-babel. 2024-04-25T14:00:13.411 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T14:00:13.448 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T14:00:13.784 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T14:00:13.798 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T14:00:13.834 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T14:00:14.024 INFO:teuthology.orchestra.run.smithi155.stdout:Fetched 175 MB in 45s (3,877 kB/s) 2024-04-25T14:00:14.192 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T14:00:14.204 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T14:00:14.217 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T14:00:14.254 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:14.258 INFO:teuthology.orchestra.run.smithi155.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132922 files and directories currently installed.) 2024-04-25T14:00:14.262 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T14:00:14.308 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T14:00:14.607 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T14:00:14.620 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T14:00:14.656 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T14:00:14.693 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T14:00:14.707 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T14:00:14.760 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T14:00:14.968 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-routes. 2024-04-25T14:00:14.986 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T14:00:15.068 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T14:00:15.138 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T14:00:15.152 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T14:00:15.230 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T14:00:15.387 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T14:00:15.398 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T14:00:15.437 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-dashboard (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:15.583 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T14:00:15.596 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T14:00:15.683 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T14:00:16.036 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T14:00:16.051 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T14:00:16.094 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T14:00:16.430 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T14:00:16.445 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T14:00:16.480 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T14:00:16.834 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T14:00:16.843 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T14:00:16.883 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T14:00:17.083 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T14:00:17.098 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T14:00:17.149 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T14:00:17.387 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T14:00:17.398 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T14:00:17.436 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T14:00:17.569 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T14:00:17.584 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T14:00:17.628 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T14:00:17.832 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T14:00:17.841 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T14:00:17.964 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T14:00:17.967 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T14:00:17.978 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T14:00:18.014 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T14:00:18.286 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T14:00:18.299 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T14:00:18.328 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T14:00:18.350 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T14:00:18.365 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T14:00:18.409 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T14:00:18.753 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package librados2. 2024-04-25T14:00:18.767 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../010-librados2_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:18.811 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking librados2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:18.978 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T14:00:18.989 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T14:00:19.028 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:19.272 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package librbd1. 2024-04-25T14:00:19.286 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../011-librbd1_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:19.330 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking librbd1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:19.733 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T14:00:19.747 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:19.783 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libcephfs2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:20.089 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T14:00:20.093 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-rados. 2024-04-25T14:00:20.101 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T14:00:20.108 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../013-python3-rados_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:20.139 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T14:00:20.152 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-rados (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:20.441 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T14:00:20.452 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T14:00:20.481 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T14:00:20.491 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T14:00:20.494 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T14:00:20.569 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-ceph-argparse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:20.832 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T14:00:20.847 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:20.853 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T14:00:20.865 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T14:00:20.890 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-cephfs (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:20.895 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T14:00:21.176 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T14:00:21.190 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T14:00:21.231 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T14:00:21.242 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T14:00:21.251 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:21.280 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T14:00:21.554 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T14:00:21.568 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T14:00:21.604 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T14:00:21.626 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T14:00:21.637 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T14:00:21.675 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T14:00:21.848 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T14:00:21.863 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:21.939 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-rbd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:22.038 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T14:00:22.050 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T14:00:22.095 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T14:00:22.267 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T14:00:22.282 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T14:00:22.325 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T14:00:22.373 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T14:00:22.385 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T14:00:22.414 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T14:00:22.628 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T14:00:22.642 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T14:00:22.686 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T14:00:22.834 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T14:00:22.846 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T14:00:22.884 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T14:00:22.981 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T14:00:22.995 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T14:00:23.030 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T14:00:23.367 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package lua5.1. 2024-04-25T14:00:23.381 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T14:00:23.425 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T14:00:23.556 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T14:00:23.567 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T14:00:23.606 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:23.728 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package lua-any. 2024-04-25T14:00:23.743 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T14:00:23.786 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T14:00:23.950 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T14:00:23.961 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T14:00:24.000 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T14:00:24.047 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package zip. 2024-04-25T14:00:24.062 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T14:00:24.097 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T14:00:24.305 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T14:00:24.316 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T14:00:24.371 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T14:00:24.408 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package unzip. 2024-04-25T14:00:24.423 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T14:00:24.467 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T14:00:24.640 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package jq. 2024-04-25T14:00:24.652 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T14:00:24.690 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T14:00:24.920 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package luarocks. 2024-04-25T14:00:24.935 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T14:00:24.968 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package socat. 2024-04-25T14:00:24.979 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T14:00:24.986 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T14:00:25.018 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T14:00:25.354 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T14:00:25.366 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T14:00:25.429 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T14:00:25.452 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package librgw2. 2024-04-25T14:00:25.466 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../027-librgw2_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:25.519 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking librgw2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:25.741 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-test. 2024-04-25T14:00:25.752 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../099-ceph-test_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:25.782 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-test (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:26.147 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T14:00:26.161 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:26.213 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-rgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:26.641 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T14:00:26.656 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T14:00:26.699 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T14:00:27.043 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T14:00:27.058 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:27.277 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libradosstriper1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:27.930 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-common. 2024-04-25T14:00:27.936 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T14:00:27.945 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../031-ceph-common_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:27.948 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T14:00:27.995 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-volume (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:28.021 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:28.357 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T14:00:28.369 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:28.399 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking cephfs-mirror (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:28.735 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T14:00:28.747 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T14:00:28.784 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T14:00:29.054 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T14:00:29.066 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T14:00:29.128 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking cephfs-shell (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:29.237 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-base. 2024-04-25T14:00:29.250 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../032-ceph-base_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:29.302 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-base (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:29.364 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T14:00:29.376 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T14:00:29.414 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking cephfs-top (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:29.691 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T14:00:29.703 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:29.741 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libcephfs-dev (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:29.947 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T14:00:29.961 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T14:00:30.014 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T14:00:30.085 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T14:00:30.097 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T14:00:30.127 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T14:00:30.408 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T14:00:30.422 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T14:00:30.456 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T14:00:30.468 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T14:00:30.474 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T14:00:30.540 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T14:00:30.868 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T14:00:30.882 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T14:00:30.894 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T14:00:30.909 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T14:00:30.918 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T14:00:30.961 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T14:00:31.297 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T14:00:31.312 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T14:00:31.348 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package pkg-config. 2024-04-25T14:00:31.355 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T14:00:31.359 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T14:00:31.398 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T14:00:31.692 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T14:00:31.704 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T14:00:31.733 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T14:00:31.742 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T14:00:31.748 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T14:00:31.799 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T14:00:32.152 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T14:00:32.166 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T14:00:32.210 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T14:00:32.211 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T14:00:32.223 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T14:00:32.261 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T14:00:32.523 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-portend. 2024-04-25T14:00:32.537 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T14:00:32.587 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T14:00:32.589 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T14:00:32.601 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T14:00:32.639 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T14:00:32.874 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T14:00:32.889 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T14:00:32.924 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T14:00:32.992 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T14:00:33.004 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T14:00:33.042 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T14:00:33.177 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T14:00:33.191 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T14:00:33.226 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T14:00:33.353 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T14:00:33.365 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T14:00:33.402 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T14:00:33.597 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T14:00:33.611 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T14:00:33.655 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T14:00:33.764 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-py. 2024-04-25T14:00:33.776 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T14:00:33.813 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T14:00:33.949 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T14:00:33.965 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T14:00:33.999 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T14:00:34.158 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T14:00:34.171 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T14:00:34.208 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T14:00:34.269 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T14:00:34.284 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T14:00:34.327 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T14:00:34.638 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-mako. 2024-04-25T14:00:34.653 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T14:00:34.679 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T14:00:34.690 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T14:00:34.696 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T14:00:34.728 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T14:00:35.039 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-toml. 2024-04-25T14:00:35.052 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T14:00:35.057 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T14:00:35.072 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T14:00:35.089 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T14:00:35.116 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T14:00:35.409 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T14:00:35.418 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T14:00:35.420 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T14:00:35.433 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T14:00:35.458 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T14:00:35.510 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T14:00:35.778 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T14:00:35.790 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T14:00:35.804 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-webob. 2024-04-25T14:00:35.819 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T14:00:35.853 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T14:00:35.871 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T14:00:36.131 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T14:00:36.143 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T14:00:36.180 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T14:00:36.241 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T14:00:36.255 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T14:00:36.301 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T14:00:36.610 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T14:00:36.625 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T14:00:36.668 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T14:00:36.917 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package radosgw. 2024-04-25T14:00:36.928 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../122-radosgw_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:36.967 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking radosgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:36.996 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-paste. 2024-04-25T14:00:37.011 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T14:00:37.054 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T14:00:37.374 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T14:00:37.388 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T14:00:37.432 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T14:00:37.710 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T14:00:37.712 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T14:00:37.724 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:37.724 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T14:00:37.759 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T14:00:37.761 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking rbd-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:38.062 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T14:00:38.077 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T14:00:38.087 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T14:00:38.120 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T14:00:38.366 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T14:00:38.449 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T14:00:38.463 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T14:00:38.476 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T14:00:38.507 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T14:00:38.706 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T14:00:38.886 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T14:00:38.900 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T14:00:38.944 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T14:00:39.115 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T14:00:39.272 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-idna. 2024-04-25T14:00:39.287 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T14:00:39.330 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T14:00:39.666 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T14:00:39.681 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T14:00:39.707 INFO:teuthology.orchestra.run.smithi086.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T14:00:39.724 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T14:00:39.742 INFO:teuthology.orchestra.run.smithi086.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-04-25T14:00:39.811 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:40.053 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-requests. 2024-04-25T14:00:40.067 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T14:00:40.110 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T14:00:40.371 INFO:teuthology.orchestra.run.smithi086.stdout:Adding system user cephadm....done 2024-04-25T14:00:40.440 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T14:00:40.455 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T14:00:40.499 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T14:00:40.525 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T14:00:40.760 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T14:00:40.774 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T14:00:40.809 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T14:00:40.810 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mgr-modules-core (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:41.060 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T14:00:41.170 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T14:00:41.189 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T14:00:41.203 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:41.247 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:41.463 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T14:00:41.592 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T14:00:41.606 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T14:00:41.650 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T14:00:41.730 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T14:00:41.856 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T14:00:41.919 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T14:00:41.934 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T14:00:41.969 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T14:00:42.191 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T14:00:42.330 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T14:00:42.345 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:42.388 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mgr (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:42.552 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T14:00:42.803 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T14:00:43.000 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T14:00:43.015 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:43.058 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mon (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:43.062 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T14:00:43.371 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T14:00:43.664 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T14:00:43.728 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T14:00:43.743 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:43.794 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-osd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:43.999 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T14:00:44.292 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T14:00:44.584 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-ceph-argparse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:44.614 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph. 2024-04-25T14:00:44.629 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../068-ceph_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:44.674 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:44.843 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T14:00:44.945 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T14:00:45.017 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T14:00:45.032 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:45.074 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:45.095 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T14:00:45.363 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T14:00:45.489 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T14:00:45.502 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T14:00:45.518 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:45.594 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mds (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:45.773 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T14:00:46.038 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package cephadm. 2024-04-25T14:00:46.053 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../071-cephadm_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:00:46.096 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:46.156 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T14:00:46.204 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T14:00:46.268 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T14:00:46.369 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T14:00:46.483 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T14:00:46.498 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T14:00:46.549 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T14:00:46.728 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T14:00:46.863 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T14:00:46.894 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T14:00:46.909 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T14:00:46.968 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T14:00:47.139 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T14:00:47.423 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T14:00:47.757 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T14:00:47.883 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T14:00:48.115 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-babel. 2024-04-25T14:00:48.128 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T14:00:48.165 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T14:00:48.467 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T14:00:48.481 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T14:00:48.518 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T14:00:48.593 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T14:00:48.845 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T14:00:48.853 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T14:00:48.859 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T14:00:48.895 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mgr-cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:49.029 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T14:00:49.155 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T14:00:49.256 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T14:00:49.270 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T14:00:49.315 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T14:00:49.472 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T14:00:49.601 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-routes. 2024-04-25T14:00:49.615 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T14:00:49.684 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T14:00:49.899 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T14:00:49.953 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T14:00:49.967 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T14:00:50.011 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mgr-dashboard (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:50.158 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T14:00:50.551 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T14:00:50.853 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T14:00:51.162 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T14:00:51.275 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T14:00:51.289 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T14:00:51.333 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T14:00:51.522 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T14:00:51.795 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T14:00:51.809 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T14:00:51.823 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T14:00:51.853 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T14:00:51.948 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:52.206 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T14:00:52.220 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T14:00:52.291 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T14:00:52.383 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T14:00:52.634 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T14:00:52.686 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T14:00:52.700 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T14:00:52.744 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T14:00:53.427 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T14:00:53.442 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T14:00:53.486 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:53.840 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T14:00:54.083 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T14:00:54.242 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T14:00:54.430 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T14:00:54.446 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T14:00:54.488 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T14:00:54.518 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T14:00:54.627 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T14:00:54.733 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T14:00:54.748 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T14:00:54.761 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T14:00:54.791 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T14:00:55.071 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T14:00:55.128 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T14:00:55.143 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T14:00:55.186 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T14:00:55.372 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T14:00:55.497 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T14:00:55.512 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T14:00:55.555 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T14:00:55.656 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T14:00:55.782 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T14:00:55.876 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T14:00:55.891 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T14:00:55.925 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T14:00:56.126 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T14:00:56.212 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T14:00:56.226 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T14:00:56.270 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T14:00:56.385 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T14:00:56.606 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T14:00:56.622 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T14:00:56.653 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T14:00:56.664 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T14:00:56.895 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T14:00:56.976 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T14:00:56.991 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T14:00:57.034 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T14:00:57.163 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T14:00:57.439 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T14:00:57.598 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T14:00:57.764 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T14:00:57.778 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T14:00:57.822 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:00:58.149 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T14:00:58.279 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T14:00:58.325 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T14:00:58.339 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T14:00:58.383 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T14:00:58.611 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T14:00:58.728 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T14:00:58.743 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T14:00:58.786 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T14:00:58.919 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T14:00:59.039 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package jq. 2024-04-25T14:00:59.054 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T14:00:59.122 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T14:00:59.161 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T14:00:59.262 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T14:00:59.349 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package socat. 2024-04-25T14:00:59.365 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T14:00:59.388 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T14:00:59.399 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T14:00:59.477 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T14:00:59.685 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T14:00:59.701 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T14:00:59.723 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T14:00:59.767 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T14:00:59.849 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T14:01:00.123 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-test. 2024-04-25T14:01:00.137 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../099-ceph-test_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:01:00.180 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-test (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:00.251 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T14:01:00.552 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T14:01:00.669 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T14:01:00.937 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T14:01:01.096 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T14:01:01.422 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T14:01:01.765 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T14:01:02.049 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T14:01:02.261 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T14:01:02.262 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T14:01:02.276 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T14:01:02.318 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-volume (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:02.520 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T14:01:02.705 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T14:01:02.721 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:01:02.755 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking cephfs-mirror (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:02.771 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T14:01:03.050 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T14:01:03.064 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T14:01:03.064 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T14:01:03.108 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T14:01:03.411 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T14:01:03.426 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T14:01:03.466 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T14:01:03.469 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking cephfs-shell (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:03.721 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T14:01:03.736 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T14:01:03.759 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T14:01:03.779 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking cephfs-top (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:04.040 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T14:01:04.056 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:01:04.098 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libcephfs-dev (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:04.111 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T14:01:04.237 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up librados2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:04.384 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libsqlite3-mod-ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:04.409 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T14:01:04.425 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T14:01:04.467 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T14:01:04.514 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T14:01:04.822 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T14:01:04.837 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T14:01:04.875 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libcephfs2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:04.879 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T14:01:05.001 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libradosstriper1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:05.127 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T14:01:05.200 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T14:01:05.216 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T14:01:05.283 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T14:01:05.403 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up librbd1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:05.538 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-modules-core (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:05.664 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:05.671 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package pkg-config. 2024-04-25T14:01:05.687 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T14:01:05.729 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T14:01:05.873 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T14:01:05.874 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T14:01:06.007 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T14:01:06.023 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T14:01:06.074 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T14:01:06.494 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T14:01:06.510 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T14:01:06.523 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libcephfs-dev (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:06.552 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T14:01:06.649 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-rados (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:06.800 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up librgw2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:06.871 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T14:01:06.887 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T14:01:06.920 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T14:01:06.926 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up cephfs-top (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:07.052 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T14:01:07.182 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T14:01:07.198 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T14:01:07.232 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T14:01:07.610 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T14:01:07.626 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T14:01:07.668 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T14:01:08.037 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-py. 2024-04-25T14:01:08.053 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T14:01:08.095 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T14:01:08.339 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-rbd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:08.424 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T14:01:08.440 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T14:01:08.465 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up rbd-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:08.482 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T14:01:08.591 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-rgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:08.750 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cephfs (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:08.876 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up cephfs-shell (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:08.944 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T14:01:08.959 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T14:01:08.994 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T14:01:09.002 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:09.263 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-toml. 2024-04-25T14:01:09.277 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T14:01:09.354 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T14:01:09.562 INFO:teuthology.orchestra.run.smithi086.stdout:Adding group ceph....done 2024-04-25T14:01:09.657 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T14:01:09.672 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T14:01:09.715 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T14:01:09.972 INFO:teuthology.orchestra.run.smithi086.stdout:Adding system user ceph....done 2024-04-25T14:01:10.052 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T14:01:10.067 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T14:01:10.102 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T14:01:10.124 INFO:teuthology.orchestra.run.smithi086.stdout:Setting system user ceph properties....done 2024-04-25T14:01:10.142 INFO:teuthology.orchestra.run.smithi086.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T14:01:10.285 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T14:01:10.329 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T14:01:10.344 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T14:01:10.379 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T14:01:10.681 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T14:01:11.116 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package radosgw. 2024-04-25T14:01:11.131 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../122-radosgw_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:01:11.166 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking radosgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:11.352 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up cephfs-mirror (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:11.544 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T14:01:11.544 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T14:01:11.902 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T14:01:11.918 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T14:01:11.960 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking rbd-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:12.153 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-test (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:12.279 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T14:01:12.280 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up radosgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:12.574 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T14:01:12.683 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T14:01:12.818 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T14:01:12.818 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T14:01:12.956 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T14:01:13.361 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T14:01:13.448 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-base (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:13.830 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T14:01:13.956 INFO:teuthology.orchestra.run.smithi155.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T14:01:13.997 INFO:teuthology.orchestra.run.smithi155.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-04-25T14:01:14.044 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:14.525 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mds (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:14.595 INFO:teuthology.orchestra.run.smithi155.stdout:Adding system user cephadm....done 2024-04-25T14:01:14.748 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T14:01:14.749 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T14:01:14.749 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T14:01:15.026 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T14:01:15.310 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T14:01:15.402 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:15.453 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T14:01:15.617 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T14:01:15.618 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T14:01:15.737 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T14:01:16.013 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T14:01:16.164 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T14:01:16.261 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-osd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:16.565 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T14:01:16.565 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T14:01:16.574 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T14:01:17.009 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T14:01:17.196 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-k8sevents (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:17.269 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T14:01:17.322 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:17.498 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mon (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:17.528 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T14:01:17.706 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T14:01:17.706 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T14:01:17.854 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T14:01:18.139 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T14:01:18.373 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:18.448 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T14:01:18.551 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:18.660 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-dashboard (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:18.724 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T14:01:18.778 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-volume (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:19.001 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-ceph-argparse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:19.157 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T14:01:19.274 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T14:01:19.276 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T14:01:19.402 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T14:01:19.460 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T14:01:19.528 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T14:01:19.838 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T14:01:19.972 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T14:01:20.281 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T14:01:20.578 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T14:01:20.704 INFO:teuthology.orchestra.run.smithi155.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T14:01:20.728 INFO:teuthology.orchestra.run.smithi155.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T14:01:20.801 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T14:01:20.977 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T14:01:21.284 INFO:teuthology.orchestra.run.smithi086.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T14:01:21.289 DEBUG:teuthology.orchestra.run.smithi086:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T14:01:21.345 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T14:01:21.363 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2024-04-25T14:01:21.471 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T14:01:21.544 INFO:teuthology.orchestra.run.smithi086.stdout:Building dependency tree... 2024-04-25T14:01:21.544 INFO:teuthology.orchestra.run.smithi086.stdout:Reading state information... 2024-04-25T14:01:21.719 INFO:teuthology.orchestra.run.smithi086.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T14:01:21.719 INFO:teuthology.orchestra.run.smithi086.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T14:01:21.719 INFO:teuthology.orchestra.run.smithi086.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T14:01:21.719 INFO:teuthology.orchestra.run.smithi086.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T14:01:21.719 INFO:teuthology.orchestra.run.smithi086.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T14:01:21.720 INFO:teuthology.orchestra.run.smithi086.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T14:01:21.720 INFO:teuthology.orchestra.run.smithi086.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T14:01:21.721 INFO:teuthology.orchestra.run.smithi086.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T14:01:21.721 INFO:teuthology.orchestra.run.smithi086.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T14:01:21.721 INFO:teuthology.orchestra.run.smithi086.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T14:01:21.721 INFO:teuthology.orchestra.run.smithi086.stdout:The following additional packages will be installed: 2024-04-25T14:01:21.721 INFO:teuthology.orchestra.run.smithi086.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T14:01:21.721 INFO:teuthology.orchestra.run.smithi086.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T14:01:21.722 INFO:teuthology.orchestra.run.smithi086.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T14:01:21.722 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T14:01:21.722 INFO:teuthology.orchestra.run.smithi086.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T14:01:21.723 INFO:teuthology.orchestra.run.smithi086.stdout:Suggested packages: 2024-04-25T14:01:21.723 INFO:teuthology.orchestra.run.smithi086.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T14:01:21.723 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T14:01:21.723 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T14:01:21.755 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T14:01:21.776 INFO:teuthology.orchestra.run.smithi086.stdout:The following NEW packages will be installed: 2024-04-25T14:01:21.777 INFO:teuthology.orchestra.run.smithi086.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T14:01:21.777 INFO:teuthology.orchestra.run.smithi086.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T14:01:21.778 INFO:teuthology.orchestra.run.smithi086.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T14:01:21.778 INFO:teuthology.orchestra.run.smithi086.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T14:01:21.778 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T14:01:21.778 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T14:01:21.778 INFO:teuthology.orchestra.run.smithi086.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T14:01:21.779 INFO:teuthology.orchestra.run.smithi086.stdout:The following packages will be upgraded: 2024-04-25T14:01:21.780 INFO:teuthology.orchestra.run.smithi086.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T14:01:21.843 INFO:teuthology.orchestra.run.smithi086.stdout:4 upgraded, 39 newly installed, 0 to remove and 301 not upgraded. 2024-04-25T14:01:21.844 INFO:teuthology.orchestra.run.smithi086.stdout:Need to get 55.7 MB of archives. 2024-04-25T14:01:21.844 INFO:teuthology.orchestra.run.smithi086.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T14:01:21.844 INFO:teuthology.orchestra.run.smithi086.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T14:01:22.013 INFO:teuthology.orchestra.run.smithi086.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T14:01:22.019 INFO:teuthology.orchestra.run.smithi086.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T14:01:22.048 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T14:01:22.065 INFO:teuthology.orchestra.run.smithi086.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T14:01:22.065 INFO:teuthology.orchestra.run.smithi086.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T14:01:22.066 INFO:teuthology.orchestra.run.smithi086.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T14:01:22.080 INFO:teuthology.orchestra.run.smithi086.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T14:01:22.081 INFO:teuthology.orchestra.run.smithi086.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T14:01:22.083 INFO:teuthology.orchestra.run.smithi086.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T14:01:22.084 INFO:teuthology.orchestra.run.smithi086.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T14:01:22.084 INFO:teuthology.orchestra.run.smithi086.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T14:01:22.086 INFO:teuthology.orchestra.run.smithi086.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T14:01:22.104 INFO:teuthology.orchestra.run.smithi086.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T14:01:22.104 INFO:teuthology.orchestra.run.smithi086.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T14:01:22.104 INFO:teuthology.orchestra.run.smithi086.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T14:01:22.126 INFO:teuthology.orchestra.run.smithi086.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T14:01:22.391 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T14:01:22.500 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T14:01:22.568 INFO:teuthology.orchestra.run.smithi086.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T14:01:22.570 INFO:teuthology.orchestra.run.smithi086.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T14:01:22.570 INFO:teuthology.orchestra.run.smithi086.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T14:01:22.573 INFO:teuthology.orchestra.run.smithi086.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T14:01:22.617 INFO:teuthology.orchestra.run.smithi086.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T14:01:22.751 INFO:teuthology.orchestra.run.smithi086.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T14:01:22.763 INFO:teuthology.orchestra.run.smithi086.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T14:01:22.797 INFO:teuthology.orchestra.run.smithi086.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T14:01:22.798 INFO:teuthology.orchestra.run.smithi086.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T14:01:22.851 INFO:teuthology.orchestra.run.smithi086.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T14:01:22.851 INFO:teuthology.orchestra.run.smithi086.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T14:01:22.867 INFO:teuthology.orchestra.run.smithi086.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T14:01:22.904 INFO:teuthology.orchestra.run.smithi086.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T14:01:22.907 INFO:teuthology.orchestra.run.smithi086.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T14:01:22.907 INFO:teuthology.orchestra.run.smithi086.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T14:01:22.908 INFO:teuthology.orchestra.run.smithi086.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T14:01:22.909 INFO:teuthology.orchestra.run.smithi086.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T14:01:22.930 INFO:teuthology.orchestra.run.smithi086.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T14:01:22.932 INFO:teuthology.orchestra.run.smithi086.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T14:01:23.185 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T14:01:23.227 INFO:teuthology.orchestra.run.smithi086.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T14:01:23.227 INFO:teuthology.orchestra.run.smithi086.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T14:01:23.227 INFO:teuthology.orchestra.run.smithi086.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T14:01:23.228 INFO:teuthology.orchestra.run.smithi086.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T14:01:23.228 INFO:teuthology.orchestra.run.smithi086.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T14:01:23.228 INFO:teuthology.orchestra.run.smithi086.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T14:01:23.243 INFO:teuthology.orchestra.run.smithi086.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T14:01:23.244 INFO:teuthology.orchestra.run.smithi086.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T14:01:23.453 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T14:01:23.579 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T14:01:23.638 INFO:teuthology.orchestra.run.smithi086.stdout:Preconfiguring packages ... 2024-04-25T14:01:23.721 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T14:01:23.854 INFO:teuthology.orchestra.run.smithi086.stdout:Fetched 55.7 MB in 1s (38.3 MB/s) 2024-04-25T14:01:24.039 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T14:01:24.106 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140922 files and directories currently installed.) 2024-04-25T14:01:24.110 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T14:01:24.285 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T14:01:24.448 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T14:01:24.718 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T14:01:24.910 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T14:01:25.103 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T14:01:25.413 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T14:01:25.747 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T14:01:26.057 INFO:teuthology.orchestra.run.smithi086.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T14:01:26.107 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T14:01:26.348 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package flex. 2024-04-25T14:01:26.391 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T14:01:26.418 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140922 files and directories currently installed.) 2024-04-25T14:01:26.422 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T14:01:26.471 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T14:01:26.518 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:26.835 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T14:01:27.016 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T14:01:27.170 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T14:01:27.219 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T14:01:27.627 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T14:01:28.173 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140995 files and directories currently installed.) 2024-04-25T14:01:28.177 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T14:01:28.310 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T14:01:28.409 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T14:01:28.644 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T14:01:28.778 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T14:01:28.789 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T14:01:29.046 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T14:01:29.147 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T14:01:29.273 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T14:01:29.309 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T14:01:29.608 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T14:01:29.816 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T14:01:29.925 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T14:01:30.193 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T14:01:30.276 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package bison. 2024-04-25T14:01:30.319 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T14:01:30.350 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140995 files and directories currently installed.) 2024-04-25T14:01:30.354 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T14:01:30.403 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T14:01:30.671 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T14:01:30.879 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T14:01:30.894 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T14:01:30.939 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T14:01:30.946 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T14:01:31.239 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T14:01:31.382 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package dump. 2024-04-25T14:01:31.398 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T14:01:31.449 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T14:01:31.524 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T14:01:31.783 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T14:01:31.794 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package indent. 2024-04-25T14:01:31.810 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T14:01:31.861 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T14:01:32.068 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T14:01:32.156 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T14:01:32.171 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T14:01:32.194 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T14:01:32.214 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T14:01:32.486 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T14:01:32.534 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T14:01:32.550 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T14:01:32.604 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T14:01:32.609 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T14:01:32.854 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T14:01:32.869 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T14:01:32.880 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T14:01:32.912 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T14:01:33.190 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T14:01:33.291 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T14:01:33.306 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T14:01:33.349 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T14:01:33.432 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T14:01:33.558 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T14:01:33.626 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T14:01:33.642 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T14:01:33.685 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T14:01:33.734 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T14:01:33.823 INFO:teuthology.orchestra.run.smithi155.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T14:01:33.887 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T14:01:33.903 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T14:01:33.945 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T14:01:34.070 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T14:01:34.195 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T14:01:34.350 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T14:01:34.365 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T14:01:34.408 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T14:01:34.589 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T14:01:34.856 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T14:01:34.965 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T14:01:35.242 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T14:01:35.376 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T14:01:35.702 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T14:01:35.761 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T14:01:35.777 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T14:01:35.862 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T14:01:36.028 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T14:01:36.105 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T14:01:36.121 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T14:01:36.155 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T14:01:36.313 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T14:01:36.515 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T14:01:36.525 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T14:01:36.540 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T14:01:36.583 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T14:01:36.800 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T14:01:36.986 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T14:01:37.002 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T14:01:37.044 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T14:01:37.101 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T14:01:37.378 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T14:01:37.414 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T14:01:37.430 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T14:01:37.480 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T14:01:37.788 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T14:01:37.892 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T14:01:37.908 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T14:01:37.950 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T14:01:38.064 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T14:01:38.320 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T14:01:38.335 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T14:01:38.378 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T14:01:38.441 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T14:01:38.576 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up librados2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:38.639 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T14:01:38.655 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T14:01:38.689 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T14:01:38.702 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libsqlite3-mod-ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:38.828 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T14:01:38.967 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T14:01:38.982 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T14:01:39.025 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T14:01:39.164 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libcephfs2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:39.290 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libradosstriper1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:39.416 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T14:01:39.631 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T14:01:39.647 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T14:01:39.710 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up librbd1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:39.714 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T14:01:39.886 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mgr-modules-core (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:40.012 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:40.166 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package modemmanager. 2024-04-25T14:01:40.181 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T14:01:40.219 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T14:01:40.219 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T14:01:40.233 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T14:01:40.905 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libcephfs-dev (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:40.931 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package network-manager. 2024-04-25T14:01:40.947 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T14:01:40.998 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T14:01:41.032 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-rados (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:41.491 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up librgw2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:41.589 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ppp. 2024-04-25T14:01:41.604 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T14:01:41.617 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up cephfs-top (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:41.744 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T14:01:41.982 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T14:01:42.394 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T14:01:42.408 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T14:01:42.452 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T14:01:42.730 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T14:01:42.744 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T14:01:42.788 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T14:01:43.035 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-rbd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:43.074 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T14:01:43.089 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T14:01:43.141 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T14:01:43.161 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up rbd-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:43.304 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-rgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:43.428 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T14:01:43.429 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-cephfs (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:43.443 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T14:01:43.486 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T14:01:43.530 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up cephfs-shell (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:43.673 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:44.132 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T14:01:44.148 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T14:01:44.209 INFO:teuthology.orchestra.run.smithi086.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T14:01:44.240 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-common (238) ... 2024-04-25T14:01:44.552 INFO:teuthology.orchestra.run.smithi155.stdout:Adding group ceph....done 2024-04-25T14:01:44.690 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T14:01:44.705 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T14:01:44.742 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T14:01:44.969 INFO:teuthology.orchestra.run.smithi155.stdout:Adding system user ceph....done 2024-04-25T14:01:45.130 INFO:teuthology.orchestra.run.smithi155.stdout:Setting system user ceph properties....done 2024-04-25T14:01:45.148 INFO:teuthology.orchestra.run.smithi155.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T14:01:45.291 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T14:01:45.693 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T14:01:45.843 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql. 2024-04-25T14:01:45.858 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T14:01:45.901 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql (14+238) ... 2024-04-25T14:01:46.145 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T14:01:46.159 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T14:01:46.203 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T14:01:46.366 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up cephfs-mirror (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:46.463 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T14:01:46.478 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T14:01:46.538 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T14:01:46.577 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T14:01:46.578 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T14:01:46.824 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T14:01:46.839 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T14:01:46.874 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T14:01:47.160 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package tcl. 2024-04-25T14:01:47.173 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T14:01:47.214 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T14:01:47.309 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-test (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:47.427 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up radosgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:47.588 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T14:01:47.606 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T14:01:47.646 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T14:01:47.948 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T14:01:47.949 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T14:01:48.017 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T14:01:48.032 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T14:01:48.076 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T14:01:48.445 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T14:01:48.461 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T14:01:48.511 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T14:01:48.562 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-base (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:48.796 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T14:01:48.925 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T14:01:49.043 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T14:01:49.169 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T14:01:49.286 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T14:01:49.421 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T14:01:49.547 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T14:01:49.598 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mds (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:49.673 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T14:01:49.827 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T14:01:49.827 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T14:01:50.474 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mgr (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:50.626 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T14:01:50.689 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T14:01:50.689 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T14:01:50.712 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T14:01:51.107 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T14:01:51.107 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T14:01:51.367 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-osd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:51.688 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T14:01:51.688 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T14:01:51.840 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T14:01:51.966 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T14:01:52.092 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T14:01:52.218 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T14:01:52.335 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mgr-k8sevents (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:52.369 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T14:01:52.461 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:52.486 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T14:01:52.588 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T14:01:52.604 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mon (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:52.705 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T14:01:52.837 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T14:01:52.838 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T14:01:53.419 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T14:01:53.472 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mgr-cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:53.536 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T14:01:53.615 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:53.631 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T14:01:53.688 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T14:01:53.740 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mgr-dashboard (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:53.856 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T14:01:53.883 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-volume (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T14:01:53.981 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T14:01:54.108 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T14:01:54.244 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T14:01:54.275 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T14:01:54.337 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T14:01:54.365 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T14:01:54.435 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T14:01:54.480 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T14:01:54.561 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T14:01:54.754 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T14:01:54.864 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T14:01:54.989 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T14:01:55.399 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T14:01:55.491 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T14:01:55.642 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T14:01:56.109 INFO:teuthology.orchestra.run.smithi155.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T14:01:56.113 DEBUG:teuthology.orchestra.run.smithi155:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T14:01:56.187 INFO:teuthology.orchestra.run.smithi155.stdout:Reading package lists... 2024-04-25T14:01:56.242 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:01:56.242 INFO:teuthology.orchestra.run.smithi086.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T14:01:56.242 INFO:teuthology.orchestra.run.smithi086.stdout:which means they are currently configured by ifupdown: 2024-04-25T14:01:56.242 INFO:teuthology.orchestra.run.smithi086.stdout:- enp3s0f1 2024-04-25T14:01:56.242 INFO:teuthology.orchestra.run.smithi086.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T14:01:56.242 INFO:teuthology.orchestra.run.smithi086.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T14:01:56.242 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:01:56.361 INFO:teuthology.orchestra.run.smithi155.stdout:Building dependency tree... 2024-04-25T14:01:56.362 INFO:teuthology.orchestra.run.smithi155.stdout:Reading state information... 2024-04-25T14:01:56.374 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T14:01:56.532 INFO:teuthology.orchestra.run.smithi155.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T14:01:56.532 INFO:teuthology.orchestra.run.smithi155.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T14:01:56.532 INFO:teuthology.orchestra.run.smithi155.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T14:01:56.532 INFO:teuthology.orchestra.run.smithi155.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T14:01:56.532 INFO:teuthology.orchestra.run.smithi155.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T14:01:56.532 INFO:teuthology.orchestra.run.smithi155.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T14:01:56.533 INFO:teuthology.orchestra.run.smithi155.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T14:01:56.534 INFO:teuthology.orchestra.run.smithi155.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T14:01:56.534 INFO:teuthology.orchestra.run.smithi155.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T14:01:56.534 INFO:teuthology.orchestra.run.smithi155.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T14:01:56.534 INFO:teuthology.orchestra.run.smithi155.stdout:The following additional packages will be installed: 2024-04-25T14:01:56.534 INFO:teuthology.orchestra.run.smithi155.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T14:01:56.534 INFO:teuthology.orchestra.run.smithi155.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T14:01:56.535 INFO:teuthology.orchestra.run.smithi155.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T14:01:56.535 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T14:01:56.535 INFO:teuthology.orchestra.run.smithi155.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T14:01:56.535 INFO:teuthology.orchestra.run.smithi155.stdout:Suggested packages: 2024-04-25T14:01:56.536 INFO:teuthology.orchestra.run.smithi155.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T14:01:56.536 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T14:01:56.536 INFO:teuthology.orchestra.run.smithi155.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T14:01:56.589 INFO:teuthology.orchestra.run.smithi155.stdout:The following NEW packages will be installed: 2024-04-25T14:01:56.589 INFO:teuthology.orchestra.run.smithi155.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T14:01:56.589 INFO:teuthology.orchestra.run.smithi155.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T14:01:56.590 INFO:teuthology.orchestra.run.smithi155.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T14:01:56.590 INFO:teuthology.orchestra.run.smithi155.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T14:01:56.590 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T14:01:56.590 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T14:01:56.590 INFO:teuthology.orchestra.run.smithi155.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T14:01:56.591 INFO:teuthology.orchestra.run.smithi155.stdout:The following packages will be upgraded: 2024-04-25T14:01:56.591 INFO:teuthology.orchestra.run.smithi155.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T14:01:56.776 INFO:teuthology.orchestra.run.smithi155.stdout:4 upgraded, 39 newly installed, 0 to remove and 301 not upgraded. 2024-04-25T14:01:56.776 INFO:teuthology.orchestra.run.smithi155.stdout:Need to get 55.7 MB of archives. 2024-04-25T14:01:56.776 INFO:teuthology.orchestra.run.smithi155.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T14:01:56.776 INFO:teuthology.orchestra.run.smithi155.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T14:01:56.799 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T14:01:57.233 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T14:01:57.417 INFO:teuthology.orchestra.run.smithi155.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T14:01:57.433 INFO:teuthology.orchestra.run.smithi155.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T14:01:57.558 INFO:teuthology.orchestra.run.smithi155.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T14:01:57.559 INFO:teuthology.orchestra.run.smithi155.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T14:01:57.560 INFO:teuthology.orchestra.run.smithi155.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T14:01:57.589 INFO:teuthology.orchestra.run.smithi155.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T14:01:57.590 INFO:teuthology.orchestra.run.smithi155.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T14:01:57.593 INFO:teuthology.orchestra.run.smithi155.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T14:01:57.596 INFO:teuthology.orchestra.run.smithi155.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T14:01:57.596 INFO:teuthology.orchestra.run.smithi155.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T14:01:57.763 INFO:teuthology.orchestra.run.smithi155.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T14:01:57.920 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T14:01:58.043 INFO:teuthology.orchestra.run.smithi155.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T14:01:58.053 INFO:teuthology.orchestra.run.smithi155.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T14:01:58.054 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T14:01:58.058 INFO:teuthology.orchestra.run.smithi155.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T14:01:58.146 INFO:teuthology.orchestra.run.smithi155.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T14:01:58.180 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T14:01:58.348 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-common (238) ... 2024-04-25T14:01:59.149 INFO:teuthology.orchestra.run.smithi155.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T14:01:59.150 INFO:teuthology.orchestra.run.smithi155.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T14:01:59.151 INFO:teuthology.orchestra.run.smithi155.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T14:01:59.153 INFO:teuthology.orchestra.run.smithi155.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T14:01:59.154 INFO:teuthology.orchestra.run.smithi155.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T14:01:59.158 INFO:teuthology.orchestra.run.smithi155.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T14:01:59.319 INFO:teuthology.orchestra.run.smithi086.stdout:Adding user postgres to group ssl-cert 2024-04-25T14:01:59.323 INFO:teuthology.orchestra.run.smithi155.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T14:01:59.565 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:01:59.565 INFO:teuthology.orchestra.run.smithi086.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T14:01:59.803 INFO:teuthology.orchestra.run.smithi086.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T14:01:59.803 INFO:teuthology.orchestra.run.smithi086.stdout:Removing obsolete dictionary files: 2024-04-25T14:01:59.929 INFO:teuthology.orchestra.run.smithi155.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T14:01:59.930 INFO:teuthology.orchestra.run.smithi155.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T14:02:00.109 INFO:teuthology.orchestra.run.smithi155.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T14:02:00.109 INFO:teuthology.orchestra.run.smithi155.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T14:02:00.148 INFO:teuthology.orchestra.run.smithi155.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T14:02:00.209 INFO:teuthology.orchestra.run.smithi155.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T14:02:00.215 INFO:teuthology.orchestra.run.smithi155.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T14:02:00.216 INFO:teuthology.orchestra.run.smithi155.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T14:02:00.216 INFO:teuthology.orchestra.run.smithi155.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T14:02:00.217 INFO:teuthology.orchestra.run.smithi155.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T14:02:00.424 INFO:teuthology.orchestra.run.smithi155.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T14:02:00.531 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T14:02:00.848 INFO:teuthology.orchestra.run.smithi155.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T14:02:00.990 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T14:02:01.320 INFO:teuthology.orchestra.run.smithi086.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T14:02:01.344 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T14:02:01.393 INFO:teuthology.orchestra.run.smithi086.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T14:02:01.393 INFO:teuthology.orchestra.run.smithi086.stdout:This user must also own the server process. 2024-04-25T14:02:01.393 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:01.393 INFO:teuthology.orchestra.run.smithi086.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T14:02:01.394 INFO:teuthology.orchestra.run.smithi086.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T14:02:01.394 INFO:teuthology.orchestra.run.smithi086.stdout:The default text search configuration will be set to "english". 2024-04-25T14:02:01.394 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:01.394 INFO:teuthology.orchestra.run.smithi086.stdout:Data page checksums are disabled. 2024-04-25T14:02:01.394 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:01.394 INFO:teuthology.orchestra.run.smithi086.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T14:02:01.395 INFO:teuthology.orchestra.run.smithi086.stdout:creating subdirectories ... ok 2024-04-25T14:02:01.395 INFO:teuthology.orchestra.run.smithi086.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T14:02:01.418 INFO:teuthology.orchestra.run.smithi086.stdout:selecting default max_connections ... 100 2024-04-25T14:02:01.452 INFO:teuthology.orchestra.run.smithi086.stdout:selecting default shared_buffers ... 128MB 2024-04-25T14:02:01.466 INFO:teuthology.orchestra.run.smithi086.stdout:selecting default time zone ... Etc/UTC 2024-04-25T14:02:01.467 INFO:teuthology.orchestra.run.smithi086.stdout:creating configuration files ... ok 2024-04-25T14:02:01.600 INFO:teuthology.orchestra.run.smithi155.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T14:02:01.600 INFO:teuthology.orchestra.run.smithi155.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T14:02:01.600 INFO:teuthology.orchestra.run.smithi155.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T14:02:01.600 INFO:teuthology.orchestra.run.smithi155.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T14:02:01.601 INFO:teuthology.orchestra.run.smithi155.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T14:02:01.601 INFO:teuthology.orchestra.run.smithi155.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T14:02:01.630 INFO:teuthology.orchestra.run.smithi155.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T14:02:01.630 INFO:teuthology.orchestra.run.smithi155.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T14:02:01.631 INFO:teuthology.orchestra.run.smithi086.stdout:running bootstrap script ... ok 2024-04-25T14:02:02.014 INFO:teuthology.orchestra.run.smithi155.stdout:Preconfiguring packages ... 2024-04-25T14:02:02.181 INFO:teuthology.orchestra.run.smithi155.stdout:Fetched 55.7 MB in 5s (11.1 MB/s) 2024-04-25T14:02:02.243 INFO:teuthology.orchestra.run.smithi086.stdout:performing post-bootstrap initialization ... ok 2024-04-25T14:02:02.468 INFO:teuthology.orchestra.run.smithi155.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140922 files and directories currently installed.) 2024-04-25T14:02:02.472 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T14:02:02.622 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T14:02:02.783 INFO:teuthology.orchestra.run.smithi086.stdout:syncing data to disk ... ok 2024-04-25T14:02:03.255 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T14:02:03.302 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T14:02:04.387 INFO:teuthology.orchestra.run.smithi155.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T14:02:04.652 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package flex. 2024-04-25T14:02:04.723 INFO:teuthology.orchestra.run.smithi155.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140922 files and directories currently installed.) 2024-04-25T14:02:04.727 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T14:02:04.800 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T14:02:05.286 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T14:02:05.489 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T14:02:05.871 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T14:02:06.258 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T14:02:06.384 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T14:02:06.443 INFO:teuthology.orchestra.run.smithi155.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140995 files and directories currently installed.) 2024-04-25T14:02:06.447 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T14:02:06.552 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql (14+238) ... 2024-04-25T14:02:06.572 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T14:02:06.678 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T14:02:06.796 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T14:02:06.968 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T14:02:07.030 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T14:02:07.030 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T14:02:07.087 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T14:02:07.402 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T14:02:07.804 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package bison. 2024-04-25T14:02:07.822 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T14:02:07.880 INFO:teuthology.orchestra.run.smithi155.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140995 files and directories currently installed.) 2024-04-25T14:02:07.884 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T14:02:07.931 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T14:02:07.939 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T14:02:08.267 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T14:02:08.357 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T14:02:08.371 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T14:02:08.415 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T14:02:08.452 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T14:02:08.735 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package dump. 2024-04-25T14:02:08.750 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T14:02:08.818 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T14:02:09.071 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package indent. 2024-04-25T14:02:09.087 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T14:02:09.122 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T14:02:09.374 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T14:02:09.390 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T14:02:09.452 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T14:02:09.786 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T14:02:09.802 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T14:02:09.845 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T14:02:09.863 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T14:02:09.963 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T14:02:10.106 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T14:02:10.122 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T14:02:10.180 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T14:02:10.492 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T14:02:10.494 INFO:teuthology.orchestra.run.smithi086.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T14:02:10.501 DEBUG:teuthology.parallel:result is None 2024-04-25T14:02:10.508 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T14:02:10.551 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T14:02:10.753 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T14:02:10.769 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T14:02:10.828 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T14:02:11.099 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T14:02:11.115 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T14:02:11.157 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T14:02:11.518 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T14:02:11.534 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T14:02:11.568 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T14:02:12.879 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T14:02:12.895 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T14:02:12.929 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T14:02:13.174 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T14:02:13.468 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T14:02:13.469 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T14:02:13.593 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T14:02:13.609 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T14:02:13.651 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T14:02:13.987 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T14:02:14.003 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T14:02:14.037 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T14:02:14.340 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T14:02:14.356 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T14:02:14.398 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T14:02:14.777 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T14:02:14.792 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T14:02:14.835 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T14:02:15.104 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T14:02:15.120 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T14:02:15.154 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T14:02:15.465 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T14:02:15.481 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T14:02:15.523 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T14:02:15.767 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T14:02:15.786 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T14:02:15.825 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T14:02:16.817 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T14:02:16.833 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T14:02:16.875 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T14:02:17.179 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package modemmanager. 2024-04-25T14:02:17.195 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T14:02:17.229 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T14:02:17.852 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package network-manager. 2024-04-25T14:02:17.867 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T14:02:17.924 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T14:02:18.535 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ppp. 2024-04-25T14:02:18.551 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T14:02:18.911 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T14:02:19.215 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T14:02:19.229 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T14:02:19.265 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T14:02:19.493 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T14:02:19.507 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T14:02:19.543 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T14:02:19.855 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T14:02:19.871 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T14:02:19.913 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T14:02:20.233 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T14:02:20.247 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T14:02:20.292 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T14:02:20.897 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T14:02:20.911 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T14:02:20.972 INFO:teuthology.orchestra.run.smithi155.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T14:02:21.004 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking postgresql-common (238) ... 2024-04-25T14:02:21.471 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T14:02:21.485 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T14:02:21.523 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T14:02:22.675 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package postgresql. 2024-04-25T14:02:22.689 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T14:02:22.766 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking postgresql (14+238) ... 2024-04-25T14:02:23.010 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T14:02:23.025 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T14:02:23.068 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T14:02:23.296 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T14:02:23.311 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T14:02:23.346 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T14:02:23.690 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T14:02:23.706 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T14:02:23.748 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T14:02:24.035 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package tcl. 2024-04-25T14:02:24.051 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T14:02:24.123 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T14:02:24.463 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T14:02:24.479 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T14:02:24.513 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T14:02:24.867 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T14:02:24.883 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T14:02:24.925 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T14:02:25.303 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T14:02:25.320 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T14:02:25.370 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T14:02:25.681 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T14:02:25.876 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T14:02:25.994 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T14:02:26.146 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T14:02:26.272 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T14:02:26.398 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T14:02:26.524 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T14:02:27.481 INFO:teuthology.orchestra.run.smithi155.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T14:02:27.572 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T14:02:27.933 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T14:02:27.933 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T14:02:28.625 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T14:02:28.751 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T14:02:28.893 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T14:02:29.020 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T14:02:29.154 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T14:02:29.264 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T14:02:29.373 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T14:02:29.533 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T14:02:30.213 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T14:02:30.323 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T14:02:30.401 INFO:teuthology.orchestra.run.smithi155.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T14:02:30.466 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T14:02:30.592 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T14:02:30.718 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T14:02:30.869 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T14:02:30.995 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T14:02:31.085 INFO:teuthology.orchestra.run.smithi155.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T14:02:31.155 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T14:02:31.265 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T14:02:31.433 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T14:02:31.584 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T14:02:31.710 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T14:02:31.878 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T14:02:32.004 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T14:02:32.739 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:32.739 INFO:teuthology.orchestra.run.smithi155.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T14:02:32.739 INFO:teuthology.orchestra.run.smithi155.stdout:which means they are currently configured by ifupdown: 2024-04-25T14:02:32.739 INFO:teuthology.orchestra.run.smithi155.stdout:- enp3s0f1 2024-04-25T14:02:32.739 INFO:teuthology.orchestra.run.smithi155.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T14:02:32.739 INFO:teuthology.orchestra.run.smithi155.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T14:02:32.739 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:32.870 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T14:02:33.287 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T14:02:33.696 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T14:02:34.399 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T14:02:34.542 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T14:02:34.667 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T14:02:34.794 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up postgresql-common (238) ... 2024-04-25T14:02:35.714 INFO:teuthology.orchestra.run.smithi155.stdout:Adding user postgres to group ssl-cert 2024-04-25T14:02:35.957 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:35.957 INFO:teuthology.orchestra.run.smithi155.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T14:02:36.218 INFO:teuthology.orchestra.run.smithi155.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T14:02:36.218 INFO:teuthology.orchestra.run.smithi155.stdout:Removing obsolete dictionary files: 2024-04-25T14:02:36.937 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T14:02:37.396 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T14:02:37.723 INFO:teuthology.orchestra.run.smithi155.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T14:02:37.737 INFO:teuthology.orchestra.run.smithi155.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T14:02:37.789 INFO:teuthology.orchestra.run.smithi155.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T14:02:37.789 INFO:teuthology.orchestra.run.smithi155.stdout:This user must also own the server process. 2024-04-25T14:02:37.789 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:37.789 INFO:teuthology.orchestra.run.smithi155.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T14:02:37.790 INFO:teuthology.orchestra.run.smithi155.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T14:02:37.790 INFO:teuthology.orchestra.run.smithi155.stdout:The default text search configuration will be set to "english". 2024-04-25T14:02:37.790 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:37.790 INFO:teuthology.orchestra.run.smithi155.stdout:Data page checksums are disabled. 2024-04-25T14:02:37.790 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:37.790 INFO:teuthology.orchestra.run.smithi155.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T14:02:37.791 INFO:teuthology.orchestra.run.smithi155.stdout:creating subdirectories ... ok 2024-04-25T14:02:37.791 INFO:teuthology.orchestra.run.smithi155.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T14:02:37.813 INFO:teuthology.orchestra.run.smithi155.stdout:selecting default max_connections ... 100 2024-04-25T14:02:37.847 INFO:teuthology.orchestra.run.smithi155.stdout:selecting default shared_buffers ... 128MB 2024-04-25T14:02:37.861 INFO:teuthology.orchestra.run.smithi155.stdout:selecting default time zone ... Etc/UTC 2024-04-25T14:02:37.862 INFO:teuthology.orchestra.run.smithi155.stdout:creating configuration files ... ok 2024-04-25T14:02:38.028 INFO:teuthology.orchestra.run.smithi155.stdout:running bootstrap script ... ok 2024-04-25T14:02:38.642 INFO:teuthology.orchestra.run.smithi155.stdout:performing post-bootstrap initialization ... ok 2024-04-25T14:02:39.144 INFO:teuthology.orchestra.run.smithi155.stdout:syncing data to disk ... ok 2024-04-25T14:02:39.649 INFO:teuthology.orchestra.run.smithi155.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T14:02:42.763 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T14:02:42.863 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T14:02:43.015 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up postgresql (14+238) ... 2024-04-25T14:02:43.141 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T14:02:43.267 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T14:02:43.527 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T14:02:43.527 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T14:02:44.361 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T14:02:44.453 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T14:02:44.764 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T14:02:44.890 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T14:02:46.278 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T14:02:46.411 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T14:02:46.962 INFO:teuthology.orchestra.run.smithi155.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T14:02:46.969 DEBUG:teuthology.parallel:result is None 2024-04-25T14:02:46.970 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T14:02:47.122 DEBUG:teuthology.orchestra.run.smithi086:> dpkg-query -W -f '${Version}' ceph 2024-04-25T14:02:47.146 INFO:teuthology.orchestra.run.smithi086.stdout:19.0.0-3291-g994e2f20-1jammy 2024-04-25T14:02:47.147 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291-g994e2f20-1jammy 2024-04-25T14:02:47.147 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291-g994e2f20-1jammy is installed. 2024-04-25T14:02:47.148 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T14:02:47.366 DEBUG:teuthology.orchestra.run.smithi155:> dpkg-query -W -f '${Version}' ceph 2024-04-25T14:02:47.388 INFO:teuthology.orchestra.run.smithi155.stdout:19.0.0-3291-g994e2f20-1jammy 2024-04-25T14:02:47.389 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291-g994e2f20-1jammy 2024-04-25T14:02:47.389 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291-g994e2f20-1jammy is installed. 2024-04-25T14:02:47.391 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T14:02:47.391 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:02:47.391 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T14:02:47.413 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:02:47.413 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T14:02:47.455 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T14:02:47.456 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:02:47.456 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T14:02:47.479 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T14:02:47.547 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:02:47.547 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T14:02:47.566 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T14:02:47.624 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T14:02:47.624 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:02:47.624 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T14:02:47.648 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T14:02:47.710 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:02:47.710 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T14:02:47.730 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T14:02:47.792 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T14:02:47.792 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:02:47.792 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T14:02:47.812 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T14:02:47.874 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:02:47.874 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T14:02:47.893 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T14:02:47.957 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T14:02:48.044 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T14:02:48.044 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 777 /var/log/ceph 2024-04-25T14:02:48.048 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 777 /var/log/ceph 2024-04-25T14:02:48.068 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T14:02:48.068 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T14:02:48.110 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T14:02:48.133 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T14:02:48.133 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T14:02:48.178 DEBUG:teuthology.orchestra.run.smithi155:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T14:02:48.224 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T14:02:48.225 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T14:02:48.225 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T14:02:48.225 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:02:48.225 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T14:02:48.247 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T14:02:48.338 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:02:48.339 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T14:02:48.347 DEBUG:teuthology.orchestra.run.smithi155:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T14:02:48.456 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T14:02:48.456 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T14:02:48.457 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673185', 'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/metrics}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7673185', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240425.015853-debug'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'kclient': {'syntax': 'v1'}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 85, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1880, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'sleep_before_teardown': 0, 'subset': '178/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240425.015853-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi086.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI/vyPCz35KI9WNWis9mr7nPzhgchQLOuGOS0aRaeuodF47ZVBEp1Nc94q0LWfJLamHLYt/NGpuniB3x65kAv+Y=', 'smithi155.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJBmjCfRPUUkkwsMdWyBgBoqX7RlPMKPwyPeoefSpw0DFoNQnpoLioDSL4biIsefldSwVDMQ0l+7kbfLgBZVYQ4='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_mds_metrics']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_10:00:23', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T14:02:48.457 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T14:02:48.469 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T14:02:48.522 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T14:02:48.532 DEBUG:teuthology.orchestra.run.smithi155:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T14:02:48.589 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:02:48.589 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T14:02:48.612 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T14:02:48.612 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_1 2024-04-25T14:02:48.671 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T14:02:48.671 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:02:48.671 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 888 Links: 1 2024-04-25T14:02:48.671 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:02:48.671 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-25 13:55:47.129086563 +0000 2024-04-25T14:02:48.671 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-25 13:55:46.849089064 +0000 2024-04-25T14:02:48.671 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-25 13:55:46.849089064 +0000 2024-04-25T14:02:48.672 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-25 13:55:46.849089064 +0000 2024-04-25T14:02:48.672 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T14:02:48.729 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:02:48.729 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:02:48.730 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000388035 s, 1.3 MB/s 2024-04-25T14:02:48.732 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T14:02:48.792 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_2 2024-04-25T14:02:48.845 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T14:02:48.845 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:02:48.845 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 904 Links: 1 2024-04-25T14:02:48.845 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:02:48.845 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-25 13:55:47.697081490 +0000 2024-04-25T14:02:48.845 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-25 13:55:47.433083848 +0000 2024-04-25T14:02:48.845 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-25 13:55:47.433083848 +0000 2024-04-25T14:02:48.845 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-25 13:55:47.433083848 +0000 2024-04-25T14:02:48.845 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T14:02:48.906 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:02:48.906 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:02:48.906 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.00037471 s, 1.4 MB/s 2024-04-25T14:02:48.908 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T14:02:48.961 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_3 2024-04-25T14:02:49.012 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T14:02:49.012 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:02:49.012 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 923 Links: 1 2024-04-25T14:02:49.012 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:02:49.013 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-25 13:55:48.317075952 +0000 2024-04-25T14:02:49.013 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-25 13:55:48.037078453 +0000 2024-04-25T14:02:49.013 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-25 13:55:48.037078453 +0000 2024-04-25T14:02:49.013 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-25 13:55:48.037078453 +0000 2024-04-25T14:02:49.013 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T14:02:49.078 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:02:49.078 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:02:49.078 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000387327 s, 1.3 MB/s 2024-04-25T14:02:49.080 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T14:02:49.133 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_4 2024-04-25T14:02:49.188 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T14:02:49.188 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:02:49.188 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 938 Links: 1 2024-04-25T14:02:49.189 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:02:49.189 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-25 13:55:48.865071057 +0000 2024-04-25T14:02:49.189 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-25 13:55:48.613073308 +0000 2024-04-25T14:02:49.189 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-25 13:55:48.613073308 +0000 2024-04-25T14:02:49.189 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-25 13:55:48.613073308 +0000 2024-04-25T14:02:49.189 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T14:02:49.255 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:02:49.255 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:02:49.255 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.00037807 s, 1.4 MB/s 2024-04-25T14:02:49.257 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T14:02:49.310 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T14:02:49.310 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:02:49.310 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T14:02:49.321 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T14:02:49.321 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_1 2024-04-25T14:02:49.376 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T14:02:49.376 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:02:49.376 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 899 Links: 1 2024-04-25T14:02:49.376 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:02:49.376 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-04-25 13:55:47.120480000 +0000 2024-04-25T14:02:49.376 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-04-25 13:55:46.852482393 +0000 2024-04-25T14:02:49.377 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-04-25 13:55:46.852482393 +0000 2024-04-25T14:02:49.377 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-04-25 13:55:46.852482393 +0000 2024-04-25T14:02:49.377 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T14:02:49.442 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-04-25T14:02:49.442 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-04-25T14:02:49.443 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000654923 s, 782 kB/s 2024-04-25T14:02:49.445 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T14:02:49.501 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_2 2024-04-25T14:02:49.555 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T14:02:49.555 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:02:49.556 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 913 Links: 1 2024-04-25T14:02:49.556 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:02:49.556 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-04-25 13:55:47.412477392 +0000 2024-04-25T14:02:49.556 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-04-25 13:55:47.408477428 +0000 2024-04-25T14:02:49.556 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-04-25 13:55:47.408477428 +0000 2024-04-25T14:02:49.556 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-04-25 13:55:47.408477428 +0000 2024-04-25T14:02:49.556 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T14:02:49.617 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-04-25T14:02:49.618 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-04-25T14:02:49.618 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000665112 s, 770 kB/s 2024-04-25T14:02:49.620 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T14:02:49.677 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_3 2024-04-25T14:02:49.731 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T14:02:49.731 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:02:49.731 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 930 Links: 1 2024-04-25T14:02:49.731 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:02:49.731 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-04-25 13:55:48.268469746 +0000 2024-04-25T14:02:49.731 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-04-25 13:55:48.004472104 +0000 2024-04-25T14:02:49.731 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-04-25 13:55:48.004472104 +0000 2024-04-25T14:02:49.731 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-04-25 13:55:48.004472104 +0000 2024-04-25T14:02:49.732 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T14:02:49.793 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-04-25T14:02:49.793 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-04-25T14:02:49.793 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000395964 s, 1.3 MB/s 2024-04-25T14:02:49.795 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T14:02:49.849 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_4 2024-04-25T14:02:49.900 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T14:02:49.900 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:02:49.900 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 949 Links: 1 2024-04-25T14:02:49.900 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:02:49.900 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-04-25 13:55:48.876464315 +0000 2024-04-25T14:02:49.900 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-04-25 13:55:48.596466816 +0000 2024-04-25T14:02:49.900 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-04-25 13:55:48.596466816 +0000 2024-04-25T14:02:49.901 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-04-25 13:55:48.596466816 +0000 2024-04-25T14:02:49.901 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T14:02:49.961 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-04-25T14:02:49.962 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-04-25T14:02:49.962 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000392116 s, 1.3 MB/s 2024-04-25T14:02:49.964 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T14:02:50.017 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T14:02:50.017 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi086.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi155.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T14:02:50.017 INFO:tasks.ceph:Generating config... 2024-04-25T14:02:50.018 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T14:02:50.018 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T14:02:50.018 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T14:02:50.019 INFO:tasks.ceph:[client] pid file = 2024-04-25T14:02:50.019 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T14:02:50.019 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T14:02:50.019 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-25T14:02:50.019 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T14:02:50.019 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T14:02:50.019 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T14:02:50.019 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T14:02:50.019 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T14:02:50.019 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T14:02:50.019 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T14:02:50.019 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T14:02:50.019 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T14:02:50.019 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T14:02:50.019 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-25T14:02:50.020 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T14:02:50.020 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T14:02:50.020 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T14:02:50.020 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T14:02:50.020 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T14:02:50.020 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-25T14:02:50.020 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T14:02:50.020 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T14:02:50.020 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T14:02:50.020 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-25T14:02:50.020 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-25T14:02:50.020 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T14:02:50.020 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T14:02:50.020 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-25T14:02:50.020 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T14:02:50.021 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T14:02:50.021 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T14:02:50.021 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T14:02:50.021 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T14:02:50.021 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T14:02:50.021 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T14:02:50.021 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T14:02:50.077 INFO:teuthology.orchestra.run.smithi086.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T14:02:50.098 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T14:02:50.155 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T14:02:50.175 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.86'), ('mon.b', '172.21.15.155'), ('mon.c', '[v2:172.21.15.155:3301,v1:172.21.15.155:6790]')] 2024-04-25T14:02:50.176 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.86,172.21.15.155,[v2:172.21.15.155:3301,v1:172.21.15.155:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T14:02:50.177 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:02:50.177 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T14:02:50.226 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.86 --add b 172.21.15.155 --addv c '[v2:172.21.15.155:3301,v1:172.21.15.155:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T14:02:50.324 INFO:teuthology.orchestra.run.smithi086.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T14:02:50.324 INFO:teuthology.orchestra.run.smithi086.stdout:monmaptool: generated fsid 8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb 2024-04-25T14:02:50.324 INFO:teuthology.orchestra.run.smithi086.stdout:setting min_mon_release = quincy 2024-04-25T14:02:50.324 INFO:teuthology.orchestra.run.smithi086.stdout:epoch 0 2024-04-25T14:02:50.324 INFO:teuthology.orchestra.run.smithi086.stdout:fsid 8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb 2024-04-25T14:02:50.324 INFO:teuthology.orchestra.run.smithi086.stdout:last_changed 2024-04-25T14:02:50.326741+0000 2024-04-25T14:02:50.324 INFO:teuthology.orchestra.run.smithi086.stdout:created 2024-04-25T14:02:50.326741+0000 2024-04-25T14:02:50.324 INFO:teuthology.orchestra.run.smithi086.stdout:min_mon_release 17 (quincy) 2024-04-25T14:02:50.324 INFO:teuthology.orchestra.run.smithi086.stdout:election_strategy: 1 2024-04-25T14:02:50.325 INFO:teuthology.orchestra.run.smithi086.stdout:0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-25T14:02:50.325 INFO:teuthology.orchestra.run.smithi086.stdout:1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.b 2024-04-25T14:02:50.325 INFO:teuthology.orchestra.run.smithi086.stdout:2: [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] mon.c 2024-04-25T14:02:50.325 INFO:teuthology.orchestra.run.smithi086.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T14:02:50.327 DEBUG:teuthology.orchestra.run.smithi086:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T14:02:50.335 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb... 2024-04-25T14:02:50.336 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T14:02:50.382 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T14:02:50.421 INFO:teuthology.orchestra.run.smithi086.stdout:[global] 2024-04-25T14:02:50.421 INFO:teuthology.orchestra.run.smithi086.stdout: chdir = "" 2024-04-25T14:02:50.421 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T14:02:50.421 INFO:teuthology.orchestra.run.smithi086.stdout: auth supported = cephx 2024-04-25T14:02:50.421 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.421 INFO:teuthology.orchestra.run.smithi086.stdout: filestore xattr use omap = true 2024-04-25T14:02:50.421 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.421 INFO:teuthology.orchestra.run.smithi086.stdout: mon clock drift allowed = 1.000 2024-04-25T14:02:50.421 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.422 INFO:teuthology.orchestra.run.smithi086.stdout: osd crush chooseleaf type = 0 2024-04-25T14:02:50.422 INFO:teuthology.orchestra.run.smithi086.stdout: auth debug = true 2024-04-25T14:02:50.422 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.422 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on old message = true 2024-04-25T14:02:50.422 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on bug = true 2024-04-25T14:02:50.422 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.422 INFO:teuthology.orchestra.run.smithi086.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T14:02:50.422 INFO:teuthology.orchestra.run.smithi086.stdout: mon pg warn max object skew = 0 2024-04-25T14:02:50.422 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.422 INFO:teuthology.orchestra.run.smithi086.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T14:02:50.422 INFO:teuthology.orchestra.run.smithi086.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T14:02:50.422 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.422 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default size = 2 2024-04-25T14:02:50.422 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.422 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow primary affinity = true 2024-04-25T14:02:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow pg remap = true 2024-04-25T14:02:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on legacy crush tunables = false 2024-04-25T14:02:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on crush straw calc version zero = false 2024-04-25T14:02:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on no sortbitwise = false 2024-04-25T14:02:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on osd down out interval zero = false 2024-04-25T14:02:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on too few osds = false 2024-04-25T14:02:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T14:02:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T14:02:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: mon_allow_pool_size_one = true 2024-04-25T14:02:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T14:02:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: osd default data pool replay window = 5 2024-04-25T14:02:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: mon allow pool delete = true 2024-04-25T14:02:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: mon cluster log file level = debug 2024-04-25T14:02:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: debug asserts on shutdown = true 2024-04-25T14:02:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: mon health detail to clog = false 2024-04-25T14:02:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: mon host = "172.21.15.86,172.21.15.155,[v2:172.21.15.155:3301,v1:172.21.15.155:6790]" 2024-04-25T14:02:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on skipped message = False 2024-04-25T14:02:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: fsid = 8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb 2024-04-25T14:02:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.424 INFO:teuthology.orchestra.run.smithi086.stdout:[osd] 2024-04-25T14:02:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: osd journal size = 100 2024-04-25T14:02:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub load threshold = 5.0 2024-04-25T14:02:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub max interval = 600 2024-04-25T14:02:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: osd mclock profile = high_recovery_ops 2024-04-25T14:02:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: osd recover clone overlap = true 2024-04-25T14:02:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: osd recovery max chunk = 1048576 2024-04-25T14:02:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug shutdown = true 2024-04-25T14:02:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug op order = true 2024-04-25T14:02:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug verify stray on activate = true 2024-04-25T14:02:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug trim objects = true 2024-04-25T14:02:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: osd open classes on start = true 2024-04-25T14:02:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug pg log writeout = true 2024-04-25T14:02:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: osd deep scrub update digest min age = 30 2024-04-25T14:02:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: osd map max advance = 10 2024-04-25T14:02:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: journal zero on create = true 2024-04-25T14:02:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: filestore ondisk finisher threads = 3 2024-04-25T14:02:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: filestore apply finisher threads = 3 2024-04-25T14:02:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: bdev debug aio = true 2024-04-25T14:02:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug misdirected ops = true 2024-04-25T14:02:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore block size = 96636764160 2024-04-25T14:02:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore fsck on mount = True 2024-04-25T14:02:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluefs = 20 2024-04-25T14:02:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluestore = 20 2024-04-25T14:02:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-25T14:02:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: debug osd = 20 2024-04-25T14:02:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: debug rocksdb = 10 2024-04-25T14:02:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T14:02:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd full ratio = 0.9 2024-04-25T14:02:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd nearfull ratio = 0.8 2024-04-25T14:02:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: osd failsafe full ratio = 0.95 2024-04-25T14:02:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: osd objectstore = bluestore 2024-04-25T14:02:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-04-25T14:02:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.427 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr] 2024-04-25T14:02:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-25T14:02:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: debug mgr = 20 2024-04-25T14:02:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-04-25T14:02:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-04-25T14:02:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-04-25T14:02:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-04-25T14:02:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: mgr/telemetry/nag = false 2024-04-25T14:02:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-04-25T14:02:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.428 INFO:teuthology.orchestra.run.smithi086.stdout:[mon] 2024-04-25T14:02:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-25T14:02:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-04-25T14:02:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: debug paxos = 20 2024-04-25T14:02:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-04-25T14:02:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: mon data avail warn = 5 2024-04-25T14:02:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: mon mgr mkfs grace = 240 2024-04-25T14:02:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-04-25T14:02:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd reporter subtree level = osd 2024-04-25T14:02:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd prime pg temp = true 2024-04-25T14:02:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-04-25T14:02:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T14:02:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T14:02:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T14:02:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T14:02:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T14:02:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T14:02:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: # 1m isn't quite enough 2024-04-25T14:02:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: mon_down_mkfs_grace = 2m 2024-04-25T14:02:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_filestore_osds = false 2024-04-25T14:02:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: mon down mkfs grace = 300 2024-04-25T14:02:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: mon op complaint time = 120 2024-04-25T14:02:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:02:50.429 INFO:teuthology.orchestra.run.smithi086.stdout:[client] 2024-04-25T14:02:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: rgw cache enabled = true 2024-04-25T14:02:50.430 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable ops log = true 2024-04-25T14:02:50.430 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable usage log = true 2024-04-25T14:02:50.430 INFO:teuthology.orchestra.run.smithi086.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T14:02:50.430 INFO:teuthology.orchestra.run.smithi086.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T14:02:50.430 INFO:teuthology.orchestra.run.smithi086.stdout: client mount timeout = 600 2024-04-25T14:02:50.430 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-04-25T14:02:50.430 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-25T14:02:50.430 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = "" 2024-04-25T14:02:50.430 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-04-25T14:02:50.430 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-04-25T14:02:50.430 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.a] 2024-04-25T14:02:50.430 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.b] 2024-04-25T14:02:50.430 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.c] 2024-04-25T14:02:50.430 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a] 2024-04-25T14:02:50.431 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c] 2024-04-25T14:02:50.431 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b] 2024-04-25T14:02:50.431 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d] 2024-04-25T14:02:50.431 INFO:teuthology.orchestra.run.smithi086.stdout:[mds] 2024-04-25T14:02:50.431 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds = 20 2024-04-25T14:02:50.431 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds balancer = 20 2024-04-25T14:02:50.431 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-25T14:02:50.431 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug frag = True 2024-04-25T14:02:50.431 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug scatterstat = True 2024-04-25T14:02:50.431 INFO:teuthology.orchestra.run.smithi086.stdout: mds op complaint time = 180 2024-04-25T14:02:50.431 INFO:teuthology.orchestra.run.smithi086.stdout: mds verify scatter = True 2024-04-25T14:02:50.431 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-04-25T14:02:50.431 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-04-25T14:02:50.431 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-04-25T14:02:50.432 INFO:teuthology.orchestra.run.smithi155.stdout:[global] 2024-04-25T14:02:50.432 INFO:teuthology.orchestra.run.smithi155.stdout: chdir = "" 2024-04-25T14:02:50.432 INFO:teuthology.orchestra.run.smithi155.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T14:02:50.432 INFO:teuthology.orchestra.run.smithi155.stdout: auth supported = cephx 2024-04-25T14:02:50.432 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.432 INFO:teuthology.orchestra.run.smithi155.stdout: filestore xattr use omap = true 2024-04-25T14:02:50.432 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.432 INFO:teuthology.orchestra.run.smithi155.stdout: mon clock drift allowed = 1.000 2024-04-25T14:02:50.432 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.432 INFO:teuthology.orchestra.run.smithi155.stdout: osd crush chooseleaf type = 0 2024-04-25T14:02:50.433 INFO:teuthology.orchestra.run.smithi155.stdout: auth debug = true 2024-04-25T14:02:50.433 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.433 INFO:teuthology.orchestra.run.smithi155.stdout: ms die on old message = true 2024-04-25T14:02:50.433 INFO:teuthology.orchestra.run.smithi155.stdout: ms die on bug = true 2024-04-25T14:02:50.433 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.433 INFO:teuthology.orchestra.run.smithi155.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T14:02:50.433 INFO:teuthology.orchestra.run.smithi155.stdout: mon pg warn max object skew = 0 2024-04-25T14:02:50.433 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.433 INFO:teuthology.orchestra.run.smithi155.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T14:02:50.433 INFO:teuthology.orchestra.run.smithi155.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T14:02:50.433 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.433 INFO:teuthology.orchestra.run.smithi155.stdout: osd pool default size = 2 2024-04-25T14:02:50.433 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.433 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd allow primary affinity = true 2024-04-25T14:02:50.433 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd allow pg remap = true 2024-04-25T14:02:50.434 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on legacy crush tunables = false 2024-04-25T14:02:50.434 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on crush straw calc version zero = false 2024-04-25T14:02:50.434 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on no sortbitwise = false 2024-04-25T14:02:50.434 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on osd down out interval zero = false 2024-04-25T14:02:50.434 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on too few osds = false 2024-04-25T14:02:50.434 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T14:02:50.434 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T14:02:50.434 INFO:teuthology.orchestra.run.smithi155.stdout: mon_allow_pool_size_one = true 2024-04-25T14:02:50.434 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.434 INFO:teuthology.orchestra.run.smithi155.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T14:02:50.434 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.434 INFO:teuthology.orchestra.run.smithi155.stdout: osd default data pool replay window = 5 2024-04-25T14:02:50.434 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.434 INFO:teuthology.orchestra.run.smithi155.stdout: mon allow pool delete = true 2024-04-25T14:02:50.434 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.435 INFO:teuthology.orchestra.run.smithi155.stdout: mon cluster log file level = debug 2024-04-25T14:02:50.435 INFO:teuthology.orchestra.run.smithi155.stdout: debug asserts on shutdown = true 2024-04-25T14:02:50.435 INFO:teuthology.orchestra.run.smithi155.stdout: mon health detail to clog = false 2024-04-25T14:02:50.435 INFO:teuthology.orchestra.run.smithi155.stdout: mon host = "172.21.15.86,172.21.15.155,[v2:172.21.15.155:3301,v1:172.21.15.155:6790]" 2024-04-25T14:02:50.435 INFO:teuthology.orchestra.run.smithi155.stdout: ms die on skipped message = False 2024-04-25T14:02:50.435 INFO:teuthology.orchestra.run.smithi155.stdout: fsid = 8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb 2024-04-25T14:02:50.435 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.435 INFO:teuthology.orchestra.run.smithi155.stdout:[osd] 2024-04-25T14:02:50.435 INFO:teuthology.orchestra.run.smithi155.stdout: osd journal size = 100 2024-04-25T14:02:50.435 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.435 INFO:teuthology.orchestra.run.smithi155.stdout: osd scrub load threshold = 5.0 2024-04-25T14:02:50.435 INFO:teuthology.orchestra.run.smithi155.stdout: osd scrub max interval = 600 2024-04-25T14:02:50.435 INFO:teuthology.orchestra.run.smithi155.stdout: osd mclock profile = high_recovery_ops 2024-04-25T14:02:50.435 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.435 INFO:teuthology.orchestra.run.smithi155.stdout: osd recover clone overlap = true 2024-04-25T14:02:50.436 INFO:teuthology.orchestra.run.smithi155.stdout: osd recovery max chunk = 1048576 2024-04-25T14:02:50.436 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.436 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug shutdown = true 2024-04-25T14:02:50.436 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug op order = true 2024-04-25T14:02:50.436 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug verify stray on activate = true 2024-04-25T14:02:50.436 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug trim objects = true 2024-04-25T14:02:50.436 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.436 INFO:teuthology.orchestra.run.smithi155.stdout: osd open classes on start = true 2024-04-25T14:02:50.436 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug pg log writeout = true 2024-04-25T14:02:50.436 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.436 INFO:teuthology.orchestra.run.smithi155.stdout: osd deep scrub update digest min age = 30 2024-04-25T14:02:50.436 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.436 INFO:teuthology.orchestra.run.smithi155.stdout: osd map max advance = 10 2024-04-25T14:02:50.436 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.436 INFO:teuthology.orchestra.run.smithi155.stdout: journal zero on create = true 2024-04-25T14:02:50.437 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.437 INFO:teuthology.orchestra.run.smithi155.stdout: filestore ondisk finisher threads = 3 2024-04-25T14:02:50.437 INFO:teuthology.orchestra.run.smithi155.stdout: filestore apply finisher threads = 3 2024-04-25T14:02:50.437 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.437 INFO:teuthology.orchestra.run.smithi155.stdout: bdev debug aio = true 2024-04-25T14:02:50.437 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug misdirected ops = true 2024-04-25T14:02:50.437 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore block size = 96636764160 2024-04-25T14:02:50.437 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore fsck on mount = True 2024-04-25T14:02:50.437 INFO:teuthology.orchestra.run.smithi155.stdout: debug bluefs = 20 2024-04-25T14:02:50.437 INFO:teuthology.orchestra.run.smithi155.stdout: debug bluestore = 20 2024-04-25T14:02:50.437 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-04-25T14:02:50.437 INFO:teuthology.orchestra.run.smithi155.stdout: debug osd = 20 2024-04-25T14:02:50.437 INFO:teuthology.orchestra.run.smithi155.stdout: debug rocksdb = 10 2024-04-25T14:02:50.437 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T14:02:50.437 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd full ratio = 0.9 2024-04-25T14:02:50.438 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd nearfull ratio = 0.8 2024-04-25T14:02:50.438 INFO:teuthology.orchestra.run.smithi155.stdout: osd failsafe full ratio = 0.95 2024-04-25T14:02:50.438 INFO:teuthology.orchestra.run.smithi155.stdout: osd objectstore = bluestore 2024-04-25T14:02:50.438 INFO:teuthology.orchestra.run.smithi155.stdout: osd op complaint time = 180 2024-04-25T14:02:50.438 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.438 INFO:teuthology.orchestra.run.smithi155.stdout:[mgr] 2024-04-25T14:02:50.438 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-04-25T14:02:50.438 INFO:teuthology.orchestra.run.smithi155.stdout: debug mgr = 20 2024-04-25T14:02:50.438 INFO:teuthology.orchestra.run.smithi155.stdout: debug mon = 20 2024-04-25T14:02:50.438 INFO:teuthology.orchestra.run.smithi155.stdout: debug auth = 20 2024-04-25T14:02:50.438 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min pgs per osd = 4 2024-04-25T14:02:50.438 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min bytes per osd = 10 2024-04-25T14:02:50.438 INFO:teuthology.orchestra.run.smithi155.stdout: mgr/telemetry/nag = false 2024-04-25T14:02:50.438 INFO:teuthology.orchestra.run.smithi155.stdout: debug client = 20 2024-04-25T14:02:50.438 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.439 INFO:teuthology.orchestra.run.smithi155.stdout:[mon] 2024-04-25T14:02:50.439 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-04-25T14:02:50.439 INFO:teuthology.orchestra.run.smithi155.stdout: debug mon = 20 2024-04-25T14:02:50.439 INFO:teuthology.orchestra.run.smithi155.stdout: debug paxos = 20 2024-04-25T14:02:50.439 INFO:teuthology.orchestra.run.smithi155.stdout: debug auth = 20 2024-04-25T14:02:50.439 INFO:teuthology.orchestra.run.smithi155.stdout: mon data avail warn = 5 2024-04-25T14:02:50.439 INFO:teuthology.orchestra.run.smithi155.stdout: mon mgr mkfs grace = 240 2024-04-25T14:02:50.439 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min pgs per osd = 4 2024-04-25T14:02:50.439 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd reporter subtree level = osd 2024-04-25T14:02:50.439 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd prime pg temp = true 2024-04-25T14:02:50.439 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min bytes per osd = 10 2024-04-25T14:02:50.439 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.439 INFO:teuthology.orchestra.run.smithi155.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T14:02:50.439 INFO:teuthology.orchestra.run.smithi155.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T14:02:50.439 INFO:teuthology.orchestra.run.smithi155.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T14:02:50.440 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.440 INFO:teuthology.orchestra.run.smithi155.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T14:02:50.440 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T14:02:50.440 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T14:02:50.440 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.440 INFO:teuthology.orchestra.run.smithi155.stdout: # 1m isn't quite enough 2024-04-25T14:02:50.440 INFO:teuthology.orchestra.run.smithi155.stdout: mon_down_mkfs_grace = 2m 2024-04-25T14:02:50.440 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.440 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_filestore_osds = false 2024-04-25T14:02:50.440 INFO:teuthology.orchestra.run.smithi155.stdout: mon down mkfs grace = 300 2024-04-25T14:02:50.440 INFO:teuthology.orchestra.run.smithi155.stdout: mon op complaint time = 120 2024-04-25T14:02:50.440 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-25T14:02:50.440 INFO:teuthology.orchestra.run.smithi155.stdout:[client] 2024-04-25T14:02:50.440 INFO:teuthology.orchestra.run.smithi155.stdout: rgw cache enabled = true 2024-04-25T14:02:50.440 INFO:teuthology.orchestra.run.smithi155.stdout: rgw enable ops log = true 2024-04-25T14:02:50.441 INFO:teuthology.orchestra.run.smithi155.stdout: rgw enable usage log = true 2024-04-25T14:02:50.441 INFO:teuthology.orchestra.run.smithi155.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T14:02:50.441 INFO:teuthology.orchestra.run.smithi155.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T14:02:50.441 INFO:teuthology.orchestra.run.smithi155.stdout: client mount timeout = 600 2024-04-25T14:02:50.441 INFO:teuthology.orchestra.run.smithi155.stdout: debug client = 20 2024-04-25T14:02:50.441 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-04-25T14:02:50.441 INFO:teuthology.orchestra.run.smithi155.stdout: pid file = "" 2024-04-25T14:02:50.441 INFO:teuthology.orchestra.run.smithi155.stdout: rados mon op timeout = 900 2024-04-25T14:02:50.441 INFO:teuthology.orchestra.run.smithi155.stdout: rados osd op timeout = 900 2024-04-25T14:02:50.441 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.a] 2024-04-25T14:02:50.441 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.b] 2024-04-25T14:02:50.441 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.c] 2024-04-25T14:02:50.441 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.a] 2024-04-25T14:02:50.441 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.c] 2024-04-25T14:02:50.441 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.b] 2024-04-25T14:02:50.441 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.d] 2024-04-25T14:02:50.442 INFO:teuthology.orchestra.run.smithi155.stdout:[mds] 2024-04-25T14:02:50.442 INFO:teuthology.orchestra.run.smithi155.stdout: debug mds = 20 2024-04-25T14:02:50.442 INFO:teuthology.orchestra.run.smithi155.stdout: debug mds balancer = 20 2024-04-25T14:02:50.442 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-04-25T14:02:50.442 INFO:teuthology.orchestra.run.smithi155.stdout: mds debug frag = True 2024-04-25T14:02:50.442 INFO:teuthology.orchestra.run.smithi155.stdout: mds debug scatterstat = True 2024-04-25T14:02:50.442 INFO:teuthology.orchestra.run.smithi155.stdout: mds op complaint time = 180 2024-04-25T14:02:50.442 INFO:teuthology.orchestra.run.smithi155.stdout: mds verify scatter = True 2024-04-25T14:02:50.442 INFO:teuthology.orchestra.run.smithi155.stdout: osd op complaint time = 180 2024-04-25T14:02:50.442 INFO:teuthology.orchestra.run.smithi155.stdout: rados mon op timeout = 900 2024-04-25T14:02:50.442 INFO:teuthology.orchestra.run.smithi155.stdout: rados osd op timeout = 900 2024-04-25T14:02:50.444 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T14:02:50.444 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T14:02:50.552 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T14:02:50.553 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:02:50.553 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T14:02:50.561 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:02:50.561 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T14:02:50.614 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T14:02:50.614 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:02:50.615 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T14:02:50.615 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T14:02:50.690 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:02:50.690 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T14:02:50.738 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T14:02:50.739 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:02:50.739 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T14:02:50.739 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T14:02:50.774 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:02:50.774 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T14:02:50.827 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T14:02:50.827 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T14:02:50.827 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T14:02:50.890 INFO:teuthology.orchestra.run.smithi086.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T14:02:50.896 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T14:02:50.965 INFO:teuthology.orchestra.run.smithi155.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T14:02:50.988 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T14:02:50.988 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T14:02:51.053 INFO:teuthology.orchestra.run.smithi086.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T14:02:51.058 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T14:02:51.078 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T14:02:51.186 INFO:teuthology.orchestra.run.smithi086.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T14:02:51.191 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T14:02:51.210 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T14:02:51.279 INFO:teuthology.orchestra.run.smithi155.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T14:02:51.284 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T14:02:51.304 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T14:02:51.409 INFO:teuthology.orchestra.run.smithi155.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T14:02:51.414 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T14:02:51.434 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T14:02:51.434 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T14:02:51.486 INFO:teuthology.orchestra.run.smithi086.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T14:02:51.504 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T14:02:51.555 INFO:teuthology.orchestra.run.smithi086.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T14:02:51.573 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T14:02:51.624 INFO:teuthology.orchestra.run.smithi155.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T14:02:51.641 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T14:02:51.693 INFO:teuthology.orchestra.run.smithi155.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T14:02:51.710 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T14:02:51.710 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi086.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi155.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T14:02:51.710 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T14:02:51.730 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T14:02:51.730 INFO:tasks.ceph:role: osd.0 2024-04-25T14:02:51.730 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T14:02:51.730 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T14:02:51.797 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T14:02:51.797 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T14:02:51.797 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T14:02:51.797 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T14:02:51.797 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T14:02:51.797 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2024-04-25T14:02:51.798 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T14:02:51.798 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T14:02:51.798 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T14:02:51.798 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T14:02:52.211 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2024-04-25T14:02:52.214 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2024-04-25T14:02:52.214 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T14:02:52.351 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T14:02:52.367 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T14:02:52.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:02:52.368 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T14:02:52.430 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T14:02:52.430 INFO:tasks.ceph:role: osd.1 2024-04-25T14:02:52.430 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T14:02:52.430 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T14:02:52.497 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T14:02:52.498 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T14:02:52.498 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T14:02:52.498 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T14:02:52.498 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T14:02:52.498 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2024-04-25T14:02:52.498 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T14:02:52.498 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T14:02:52.498 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T14:02:52.498 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T14:02:52.858 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2024-04-25T14:02:52.862 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2024-04-25T14:02:52.862 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T14:02:52.939 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T14:02:52.994 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T14:02:52.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:02:52.996 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T14:02:53.062 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T14:02:53.062 INFO:tasks.ceph:role: osd.2 2024-04-25T14:02:53.062 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T14:02:53.063 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T14:02:53.133 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T14:02:53.133 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T14:02:53.133 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T14:02:53.133 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T14:02:53.133 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T14:02:53.133 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2024-04-25T14:02:53.133 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T14:02:53.133 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T14:02:53.133 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T14:02:53.134 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T14:02:53.530 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2024-04-25T14:02:53.533 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2024-04-25T14:02:53.533 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T14:02:53.633 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T14:02:53.650 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T14:02:53.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:02:53.651 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T14:02:53.715 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T14:02:53.715 INFO:tasks.ceph:role: osd.3 2024-04-25T14:02:53.715 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T14:02:53.715 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T14:02:53.782 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T14:02:53.783 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T14:02:53.783 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T14:02:53.783 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T14:02:53.783 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T14:02:53.783 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2024-04-25T14:02:53.783 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T14:02:53.783 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T14:02:53.783 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T14:02:53.783 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T14:02:54.169 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2024-04-25T14:02:54.172 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2024-04-25T14:02:54.172 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T14:02:54.299 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T14:02:54.315 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T14:02:54.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:02:54.317 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T14:02:54.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:02:54.456+0000 7f3d3e6ff7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T14:02:54.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:02:54.460+0000 7f3d3e6ff7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T14:02:54.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:02:54.460+0000 7f3d3e6ff7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T14:02:54.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:02:54.460+0000 7f3d3e6ff7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T14:02:54.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:02:54.460+0000 7f3d3e6ff7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T14:02:54.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:02:54.468+0000 7f3d3e6ff7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T14:02:57.516 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T14:02:57.536 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T14:02:57.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:02:57.644+0000 7f5dbca2f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T14:02:57.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:02:57.644+0000 7f5dbca2f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T14:02:57.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:02:57.644+0000 7f5dbca2f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T14:02:57.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:02:57.644+0000 7f5dbca2f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T14:02:57.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:02:57.644+0000 7f5dbca2f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T14:02:57.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:02:57.644+0000 7f5dbca2f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T14:03:00.667 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T14:03:00.687 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T14:03:00.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:00.792+0000 7f44443967c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T14:03:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:00.796+0000 7f44443967c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T14:03:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:00.796+0000 7f44443967c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T14:03:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:00.796+0000 7f44443967c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T14:03:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:00.796+0000 7f44443967c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T14:03:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:00.796+0000 7f44443967c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T14:03:03.835 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T14:03:03.855 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T14:03:03.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:03.956+0000 7f81dd02b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T14:03:03.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:03.960+0000 7f81dd02b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T14:03:03.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:03.960+0000 7f81dd02b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T14:03:03.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:03.960+0000 7f81dd02b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T14:03:03.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:03.960+0000 7f81dd02b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T14:03:03.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:03.960+0000 7f81dd02b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T14:03:06.984 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T14:03:07.005 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T14:03:07.027 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T14:03:07.028 INFO:tasks.ceph:role: osd.4 2024-04-25T14:03:07.028 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T14:03:07.028 DEBUG:teuthology.orchestra.run.smithi155:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T14:03:07.099 INFO:teuthology.orchestra.run.smithi155.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T14:03:07.099 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T14:03:07.099 INFO:teuthology.orchestra.run.smithi155.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T14:03:07.099 INFO:teuthology.orchestra.run.smithi155.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T14:03:07.099 INFO:teuthology.orchestra.run.smithi155.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T14:03:07.099 INFO:teuthology.orchestra.run.smithi155.stdout: = sunit=0 swidth=0 blks 2024-04-25T14:03:07.099 INFO:teuthology.orchestra.run.smithi155.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T14:03:07.099 INFO:teuthology.orchestra.run.smithi155.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T14:03:07.099 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T14:03:07.099 INFO:teuthology.orchestra.run.smithi155.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T14:03:07.496 INFO:teuthology.orchestra.run.smithi155.stdout:Discarding blocks...Done. 2024-04-25T14:03:07.499 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi155.front.sepia.ceph.com -o noatime 2024-04-25T14:03:07.499 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T14:03:07.598 DEBUG:teuthology.orchestra.run.smithi155:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T14:03:07.615 INFO:teuthology.orchestra.run.smithi155.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T14:03:07.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:03:07.616 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T14:03:07.678 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T14:03:07.679 INFO:tasks.ceph:role: osd.5 2024-04-25T14:03:07.679 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T14:03:07.679 DEBUG:teuthology.orchestra.run.smithi155:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T14:03:07.747 INFO:teuthology.orchestra.run.smithi155.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T14:03:07.747 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T14:03:07.747 INFO:teuthology.orchestra.run.smithi155.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T14:03:07.747 INFO:teuthology.orchestra.run.smithi155.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T14:03:07.747 INFO:teuthology.orchestra.run.smithi155.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T14:03:07.747 INFO:teuthology.orchestra.run.smithi155.stdout: = sunit=0 swidth=0 blks 2024-04-25T14:03:07.747 INFO:teuthology.orchestra.run.smithi155.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T14:03:07.747 INFO:teuthology.orchestra.run.smithi155.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T14:03:07.747 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T14:03:07.748 INFO:teuthology.orchestra.run.smithi155.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T14:03:08.152 INFO:teuthology.orchestra.run.smithi155.stdout:Discarding blocks...Done. 2024-04-25T14:03:08.155 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi155.front.sepia.ceph.com -o noatime 2024-04-25T14:03:08.156 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T14:03:08.295 DEBUG:teuthology.orchestra.run.smithi155:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T14:03:08.312 INFO:teuthology.orchestra.run.smithi155.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T14:03:08.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:03:08.314 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T14:03:08.380 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T14:03:08.380 INFO:tasks.ceph:role: osd.6 2024-04-25T14:03:08.380 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T14:03:08.380 DEBUG:teuthology.orchestra.run.smithi155:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T14:03:08.451 INFO:teuthology.orchestra.run.smithi155.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T14:03:08.451 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T14:03:08.451 INFO:teuthology.orchestra.run.smithi155.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T14:03:08.451 INFO:teuthology.orchestra.run.smithi155.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T14:03:08.451 INFO:teuthology.orchestra.run.smithi155.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T14:03:08.451 INFO:teuthology.orchestra.run.smithi155.stdout: = sunit=0 swidth=0 blks 2024-04-25T14:03:08.451 INFO:teuthology.orchestra.run.smithi155.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T14:03:08.451 INFO:teuthology.orchestra.run.smithi155.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T14:03:08.451 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T14:03:08.452 INFO:teuthology.orchestra.run.smithi155.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T14:03:08.866 INFO:teuthology.orchestra.run.smithi155.stdout:Discarding blocks...Done. 2024-04-25T14:03:08.869 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi155.front.sepia.ceph.com -o noatime 2024-04-25T14:03:08.869 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T14:03:09.000 DEBUG:teuthology.orchestra.run.smithi155:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T14:03:09.017 INFO:teuthology.orchestra.run.smithi155.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T14:03:09.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:03:09.018 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T14:03:09.079 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T14:03:09.079 INFO:tasks.ceph:role: osd.7 2024-04-25T14:03:09.079 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T14:03:09.079 DEBUG:teuthology.orchestra.run.smithi155:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T14:03:09.151 INFO:teuthology.orchestra.run.smithi155.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T14:03:09.151 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T14:03:09.151 INFO:teuthology.orchestra.run.smithi155.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T14:03:09.151 INFO:teuthology.orchestra.run.smithi155.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T14:03:09.151 INFO:teuthology.orchestra.run.smithi155.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T14:03:09.151 INFO:teuthology.orchestra.run.smithi155.stdout: = sunit=0 swidth=0 blks 2024-04-25T14:03:09.152 INFO:teuthology.orchestra.run.smithi155.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T14:03:09.152 INFO:teuthology.orchestra.run.smithi155.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T14:03:09.152 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T14:03:09.152 INFO:teuthology.orchestra.run.smithi155.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T14:03:09.517 INFO:teuthology.orchestra.run.smithi155.stdout:Discarding blocks...Done. 2024-04-25T14:03:09.522 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi155.front.sepia.ceph.com -o noatime 2024-04-25T14:03:09.522 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T14:03:09.603 DEBUG:teuthology.orchestra.run.smithi155:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T14:03:09.668 INFO:teuthology.orchestra.run.smithi155.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T14:03:09.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:03:09.669 DEBUG:teuthology.orchestra.run.smithi155:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T14:03:09.815 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:09.817+0000 7fc52342b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T14:03:09.818 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:09.817+0000 7fc52342b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T14:03:09.818 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:09.817+0000 7fc52342b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T14:03:09.818 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:09.817+0000 7fc52342b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T14:03:09.819 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:09.817+0000 7fc52342b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T14:03:09.825 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:09.825+0000 7fc52342b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T14:03:12.874 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T14:03:12.894 DEBUG:teuthology.orchestra.run.smithi155:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T14:03:13.002 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:13.001+0000 7feab323d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T14:03:13.002 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:13.001+0000 7feab323d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T14:03:13.002 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:13.001+0000 7feab323d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T14:03:13.002 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:13.001+0000 7feab323d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T14:03:13.002 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:13.001+0000 7feab323d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T14:03:13.003 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:13.005+0000 7feab323d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T14:03:16.037 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T14:03:16.057 DEBUG:teuthology.orchestra.run.smithi155:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T14:03:16.162 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:16.161+0000 7f1a66b7c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T14:03:16.163 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:16.161+0000 7f1a66b7c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T14:03:16.163 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:16.161+0000 7f1a66b7c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T14:03:16.163 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:16.161+0000 7f1a66b7c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T14:03:16.163 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:16.161+0000 7f1a66b7c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T14:03:16.165 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:16.165+0000 7f1a66b7c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T14:03:18.458 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:03:18.462 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:03:19.223 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T14:03:19.247 DEBUG:teuthology.orchestra.run.smithi155:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T14:03:19.358 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:19.357+0000 7f8c4d1097c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T14:03:19.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:19.357+0000 7f8c4d1097c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T14:03:19.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:19.357+0000 7f8c4d1097c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T14:03:19.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:19.357+0000 7f8c4d1097c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T14:03:19.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:19.357+0000 7f8c4d1097c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T14:03:19.361 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:19.361+0000 7f8c4d1097c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T14:03:22.399 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T14:03:22.419 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T14:03:22.419 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:03:22.419 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T14:03:22.441 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:03:22.442 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T14:03:22.508 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:03:22.508 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T14:03:22.576 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:03:22.576 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T14:03:22.644 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:03:22.644 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T14:03:22.708 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:03:22.709 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T14:03:22.777 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:03:22.777 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T14:03:22.846 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:03:22.846 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T14:03:22.867 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:03:22.867 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T14:03:22.932 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:03:22.932 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T14:03:22.995 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:03:22.995 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T14:03:23.063 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:03:23.064 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T14:03:23.131 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:03:23.132 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T14:03:23.201 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:03:23.202 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T14:03:23.263 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:03:23.263 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T14:03:23.272 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:03:23.272 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T14:03:23.328 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:03:23.328 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T14:03:23.336 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:03:23.337 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T14:03:23.391 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T14:03:23.392 DEBUG:teuthology.orchestra.run.smithi086:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T14:03:23.395 DEBUG:teuthology.orchestra.run.smithi155:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T14:03:23.443 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr.y] 2024-04-25T14:03:23.443 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCKYipmC7A7NRAAEwVNA0yxt3fwdHXrsV0dvg== 2024-04-25T14:03:23.443 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a] 2024-04-25T14:03:23.443 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCLYipmWk5fAxAAKEBKhWewpl23MUcvMi8qDg== 2024-04-25T14:03:23.443 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c] 2024-04-25T14:03:23.443 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCLYipmA/hNCxAAclDGbIjLfjqyXn0MwX/IkA== 2024-04-25T14:03:23.443 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.0] 2024-04-25T14:03:23.444 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCOYipm9etgGxAAemg9CqXN0ShduDtDuKF2Zw== 2024-04-25T14:03:23.444 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.1] 2024-04-25T14:03:23.444 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCRYipmBCiNJhAABAWHk8NEGVvqieL2klGhmA== 2024-04-25T14:03:23.444 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.2] 2024-04-25T14:03:23.444 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCUYipm/XGYLxAAPXsw1svKseFzFiuDUjAsJQ== 2024-04-25T14:03:23.444 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.3] 2024-04-25T14:03:23.444 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCXYipmlSRaORAAwpKyzZDdVYHWl2pAsQjmpw== 2024-04-25T14:03:23.444 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr.x] 2024-04-25T14:03:23.444 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCKYipmKMzAORAAtRyTNJ5HWi2XwuamoqwB0Q== 2024-04-25T14:03:23.444 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b] 2024-04-25T14:03:23.444 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCLYipmDhzOEBAABNys/NAEhucQ7A+HzQlzQw== 2024-04-25T14:03:23.444 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d] 2024-04-25T14:03:23.444 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCLYipmemCcGBAAGrCx6C850qRJSWYjebxOBA== 2024-04-25T14:03:23.444 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.4] 2024-04-25T14:03:23.445 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCdYipmBI3FMBAAV2NdggfRSFnQvrcOC2YqiA== 2024-04-25T14:03:23.445 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.5] 2024-04-25T14:03:23.445 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQChYipmctYxABAABwmOMgyYuOfRWD9KdHILng== 2024-04-25T14:03:23.445 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.6] 2024-04-25T14:03:23.445 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCkYipmSL7TCRAAhFMWg1AQm54bB+x1GDcusA== 2024-04-25T14:03:23.445 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.7] 2024-04-25T14:03:23.445 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCnYipmZ4F+FRAAEGaZN+byv/qJUFZb7WxTFg== 2024-04-25T14:03:23.445 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-04-25T14:03:23.445 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCLYipm5u4xHRAACDFe3Eb4KLrsdLj4eFe7mA== 2024-04-25T14:03:23.445 INFO:teuthology.orchestra.run.smithi086.stdout:[client.2] 2024-04-25T14:03:23.445 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCLYipmhbhGIRAAxao/J0WY53IITLETFgwIVw== 2024-04-25T14:03:23.445 INFO:teuthology.orchestra.run.smithi086.stdout:[client.1] 2024-04-25T14:03:23.445 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCLYipmDp1hJRAAQkBc5PdQoMTT7W4HahkTyg== 2024-04-25T14:03:23.445 INFO:teuthology.orchestra.run.smithi086.stdout:[client.3] 2024-04-25T14:03:23.446 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCLYipmJsCBKRAAzXn2qDtblIpqt7U/TUUlyA== 2024-04-25T14:03:23.457 INFO:teuthology.orchestra.run.smithi155.stdout:[mgr.y] 2024-04-25T14:03:23.457 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCKYipmC7A7NRAAEwVNA0yxt3fwdHXrsV0dvg== 2024-04-25T14:03:23.457 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.a] 2024-04-25T14:03:23.457 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCLYipmWk5fAxAAKEBKhWewpl23MUcvMi8qDg== 2024-04-25T14:03:23.457 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.c] 2024-04-25T14:03:23.457 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCLYipmA/hNCxAAclDGbIjLfjqyXn0MwX/IkA== 2024-04-25T14:03:23.457 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.0] 2024-04-25T14:03:23.457 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCOYipm9etgGxAAemg9CqXN0ShduDtDuKF2Zw== 2024-04-25T14:03:23.457 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.1] 2024-04-25T14:03:23.457 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCRYipmBCiNJhAABAWHk8NEGVvqieL2klGhmA== 2024-04-25T14:03:23.457 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.2] 2024-04-25T14:03:23.458 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCUYipm/XGYLxAAPXsw1svKseFzFiuDUjAsJQ== 2024-04-25T14:03:23.458 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.3] 2024-04-25T14:03:23.458 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCXYipmlSRaORAAwpKyzZDdVYHWl2pAsQjmpw== 2024-04-25T14:03:23.458 INFO:teuthology.orchestra.run.smithi155.stdout:[mgr.x] 2024-04-25T14:03:23.458 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCKYipmKMzAORAAtRyTNJ5HWi2XwuamoqwB0Q== 2024-04-25T14:03:23.458 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.b] 2024-04-25T14:03:23.458 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCLYipmDhzOEBAABNys/NAEhucQ7A+HzQlzQw== 2024-04-25T14:03:23.458 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.d] 2024-04-25T14:03:23.458 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCLYipmemCcGBAAGrCx6C850qRJSWYjebxOBA== 2024-04-25T14:03:23.458 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.4] 2024-04-25T14:03:23.458 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCdYipmBI3FMBAAV2NdggfRSFnQvrcOC2YqiA== 2024-04-25T14:03:23.458 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.5] 2024-04-25T14:03:23.458 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQChYipmctYxABAABwmOMgyYuOfRWD9KdHILng== 2024-04-25T14:03:23.459 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.6] 2024-04-25T14:03:23.459 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCkYipmSL7TCRAAhFMWg1AQm54bB+x1GDcusA== 2024-04-25T14:03:23.459 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.7] 2024-04-25T14:03:23.459 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCnYipmZ4F+FRAAEGaZN+byv/qJUFZb7WxTFg== 2024-04-25T14:03:23.459 INFO:teuthology.orchestra.run.smithi155.stdout:[client.0] 2024-04-25T14:03:23.459 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCLYipm5u4xHRAACDFe3Eb4KLrsdLj4eFe7mA== 2024-04-25T14:03:23.459 INFO:teuthology.orchestra.run.smithi155.stdout:[client.2] 2024-04-25T14:03:23.459 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCLYipmhbhGIRAAxao/J0WY53IITLETFgwIVw== 2024-04-25T14:03:23.459 INFO:teuthology.orchestra.run.smithi155.stdout:[client.1] 2024-04-25T14:03:23.459 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCLYipmDp1hJRAAQkBc5PdQoMTT7W4HahkTyg== 2024-04-25T14:03:23.459 INFO:teuthology.orchestra.run.smithi155.stdout:[client.3] 2024-04-25T14:03:23.459 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCLYipmJsCBKRAAzXn2qDtblIpqt7U/TUUlyA== 2024-04-25T14:03:23.460 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T14:03:23.492 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T14:03:23.576 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T14:03:23.579 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T14:03:23.648 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T14:03:23.651 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T14:03:23.717 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T14:03:23.721 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T14:03:23.785 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T14:03:23.788 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T14:03:23.853 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T14:03:23.856 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T14:03:23.922 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T14:03:23.926 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T14:03:23.990 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T14:03:23.994 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T14:03:24.058 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T14:03:24.061 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T14:03:24.128 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T14:03:24.131 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T14:03:24.198 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T14:03:24.201 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T14:03:24.266 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T14:03:24.269 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T14:03:24.334 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T14:03:24.337 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T14:03:24.402 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T14:03:24.405 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T14:03:24.470 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T14:03:24.473 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T14:03:24.540 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T14:03:24.543 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T14:03:24.611 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T14:03:24.616 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T14:03:24.691 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T14:03:24.697 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T14:03:24.765 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T14:03:24.766 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T14:03:24.786 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T14:03:24.915 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T14:03:24.936 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T14:03:24.956 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T14:03:25.080 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T14:03:25.100 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T14:03:25.162 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T14:03:25.283 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T14:03:25.302 DEBUG:teuthology.orchestra.run.smithi086:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T14:03:25.305 DEBUG:teuthology.orchestra.run.smithi155:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T14:03:25.350 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T14:03:25.351 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T14:03:25.351 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T14:03:25.358 INFO:tasks.ceph.mon.a:Started 2024-04-25T14:03:25.358 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T14:03:25.358 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T14:03:25.394 INFO:tasks.ceph.mon.b:Started 2024-04-25T14:03:25.394 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T14:03:25.394 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T14:03:25.397 INFO:tasks.ceph.mon.c:Started 2024-04-25T14:03:25.397 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T14:03:25.397 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T14:03:25.397 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T14:03:25.400 INFO:tasks.ceph.mgr.y:Started 2024-04-25T14:03:25.400 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T14:03:25.400 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T14:03:25.403 INFO:tasks.ceph.mgr.x:Started 2024-04-25T14:03:25.403 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T14:03:25.403 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T14:03:25.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.551+0000 7fcd23400640 1 Processor -- start 2024-04-25T14:03:25.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.551+0000 7fcd23400640 1 -- start start 2024-04-25T14:03:25.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.551+0000 7fcd23400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd1c0b6ba0 0x7fcd1c0d2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:25.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.551+0000 7fcd23400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd1c162480 con 0x7fcd1c0d2420 2024-04-25T14:03:25.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.551+0000 7fcd23400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd1c0d1940 con 0x7fcd1c071530 2024-04-25T14:03:25.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.551+0000 7fcd23400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcd1c0d16f0 con 0x7fcd1c0b6ba0 2024-04-25T14:03:25.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.555+0000 7fcd20a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fcd1c0d2420 0x7fcd1c070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:43966/0 (socket says 172.21.15.86:43966) 2024-04-25T14:03:25.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.555+0000 7fcd20a00640 1 -- 172.21.15.86:0/729456948 learned_addr learned my addr 172.21.15.86:0/729456948 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:03:25.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.555+0000 7fcd21400640 1 --2- 172.21.15.86:0/729456948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd1c0b6ba0 0x7fcd1c0d2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:25.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.555+0000 7fcd21400640 1 -- 172.21.15.86:0/729456948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd1c0b6ba0 msgr2=0x7fcd1c0d2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T14:03:25.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.555+0000 7fcd21400640 1 -- 172.21.15.86:0/729456948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd1c0b6ba0 msgr2=0x7fcd1c0d2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T14:03:25.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.555+0000 7fcd21400640 1 --2- 172.21.15.86:0/729456948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd1c0b6ba0 0x7fcd1c0d2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T14:03:25.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.555+0000 7fcd21400640 1 --2- 172.21.15.86:0/729456948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd1c0b6ba0 0x7fcd1c0d2830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:03:25.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.707+0000 7fcd1be00640 1 -- 172.21.15.86:0/729456948 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 792240326 0 0) 0x7fcd1c162480 con 0x7fcd1c0d2420 2024-04-25T14:03:25.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.707+0000 7fcd1be00640 1 -- 172.21.15.86:0/729456948 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd00003530 con 0x7fcd1c0d2420 2024-04-25T14:03:25.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.711+0000 7fcd1be00640 1 -- 172.21.15.86:0/729456948 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2791160466 0 0) 0x7fcd1c0d1940 con 0x7fcd1c071530 2024-04-25T14:03:25.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.711+0000 7fcd1be00640 1 -- 172.21.15.86:0/729456948 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd1c162480 con 0x7fcd1c071530 2024-04-25T14:03:25.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.711+0000 7fcd1be00640 1 -- 172.21.15.86:0/729456948 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1791893440 0 0) 0x7fcd1c162480 con 0x7fcd1c071530 2024-04-25T14:03:25.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.711+0000 7fcd1be00640 1 -- 172.21.15.86:0/729456948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd1c0b6ba0 msgr2=0x7fcd1c0d2830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T14:03:25.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.711+0000 7fcd1be00640 1 --2- 172.21.15.86:0/729456948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd1c0b6ba0 0x7fcd1c0d2830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:25.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.711+0000 7fcd1be00640 1 -- 172.21.15.86:0/729456948 >> v1:172.21.15.86:6789/0 conn(0x7fcd1c0d2420 legacy=0x7fcd1c070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:25.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.711+0000 7fcd1be00640 1 -- 172.21.15.86:0/729456948 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1c0d1c20 con 0x7fcd1c071530 2024-04-25T14:03:25.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.711+0000 7fcd1be00640 1 -- 172.21.15.86:0/729456948 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fcd040016d0 con 0x7fcd1c071530 2024-04-25T14:03:25.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.711+0000 7fcd1be00640 1 -- 172.21.15.86:0/729456948 >> v1:172.21.15.155:6789/0 conn(0x7fcd1c071530 legacy=0x7fcd1c0b6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:25.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.711+0000 7fcd1be00640 1 --2- 172.21.15.86:0/729456948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd00003bc0 0x7fcd00004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:25.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.711+0000 7fcd1be00640 1 --2- 172.21.15.86:0/729456948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd00004810 0x7fcd00004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:25.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.711+0000 7fcd1be00640 1 --2- 172.21.15.86:0/729456948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd1c0b6ba0 0x7fcd0000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:25.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.711+0000 7fcd1be00640 1 -- 172.21.15.86:0/729456948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c0d16f0 con 0x7fcd00003bc0 2024-04-25T14:03:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.711+0000 7fcd1be00640 1 -- 172.21.15.86:0/729456948 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcd00005530 con 0x7fcd00004810 2024-04-25T14:03:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.711+0000 7fcd1be00640 1 -- 172.21.15.86:0/729456948 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcd0000bf10 con 0x7fcd1c0b6ba0 2024-04-25T14:03:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.711+0000 7fcd20a00640 1 --2- 172.21.15.86:0/729456948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd00003bc0 0x7fcd00004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.711+0000 7fcd13e00640 1 --2- 172.21.15.86:0/729456948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd00004810 0x7fcd00004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.711+0000 7fcd21400640 1 --2- 172.21.15.86:0/729456948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd1c0b6ba0 0x7fcd0000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.711+0000 7fcd20a00640 1 -- 172.21.15.86:0/729456948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd1c0b6ba0 msgr2=0x7fcd0000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.711+0000 7fcd20a00640 1 --2- 172.21.15.86:0/729456948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd1c0b6ba0 0x7fcd0000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.711+0000 7fcd20a00640 1 -- 172.21.15.86:0/729456948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd00004810 msgr2=0x7fcd00004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.711+0000 7fcd20a00640 1 --2- 172.21.15.86:0/729456948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd00004810 0x7fcd00004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.711+0000 7fcd20a00640 1 -- 172.21.15.86:0/729456948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1c162480 con 0x7fcd00003bc0 2024-04-25T14:03:25.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.715+0000 7fcd20a00640 1 --2- 172.21.15.86:0/729456948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd00003bc0 0x7fcd00004010 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c001a00 tx=0x7fcd0c01c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ff9ec500256a62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:25.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.719+0000 7fcd1be00640 1 -- 172.21.15.86:0/729456948 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c001620 con 0x7fcd00003bc0 2024-04-25T14:03:25.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.719+0000 7fcd1be00640 1 -- 172.21.15.86:0/729456948 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd0c004ab0 con 0x7fcd00003bc0 2024-04-25T14:03:25.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.719+0000 7fcd1be00640 1 -- 172.21.15.86:0/729456948 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c007070 con 0x7fcd00003bc0 2024-04-25T14:03:25.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.719+0000 7fcd23400640 1 -- 172.21.15.86:0/729456948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd00003bc0 msgr2=0x7fcd00004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:25.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.719+0000 7fcd23400640 1 --2- 172.21.15.86:0/729456948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd00003bc0 0x7fcd00004010 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c001a00 tx=0x7fcd0c01c6a0 comp rx=0 tx=0).stop 2024-04-25T14:03:25.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.719+0000 7fcd21400640 1 -- 172.21.15.86:0/729456948 reap_dead start 2024-04-25T14:03:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.719+0000 7fcd23400640 1 -- 172.21.15.86:0/729456948 shutdown_connections 2024-04-25T14:03:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.719+0000 7fcd23400640 1 -- 172.21.15.86:0/729456948 >> 172.21.15.86:0/729456948 conn(0x7fcd1c0cffc0 msgr2=0x7fcd1c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.719+0000 7fcd23400640 1 -- 172.21.15.86:0/729456948 shutdown_connections 2024-04-25T14:03:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.719+0000 7fcd23400640 1 -- 172.21.15.86:0/729456948 wait complete. 2024-04-25T14:03:25.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.723+0000 7fcd23400640 1 Processor -- start 2024-04-25T14:03:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.723+0000 7fcd23400640 1 -- start start 2024-04-25T14:03:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.723+0000 7fcd23400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd1c1524e0 0x7fcd1c1672b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.723+0000 7fcd23400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd1c15fbf0 0x7fcd1c160000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.723+0000 7fcd23400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd1c1fcf40 0x7fcd1c1fd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:25.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.723+0000 7fcd23400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c161310 con 0x7fcd1c15fbf0 2024-04-25T14:03:25.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.723+0000 7fcd23400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c1620a0 con 0x7fcd1c1fcf40 2024-04-25T14:03:25.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.723+0000 7fcd23400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcd1c1535a0 con 0x7fcd1c1524e0 2024-04-25T14:03:25.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.723+0000 7fcd13e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd1c15fbf0 0x7fcd1c160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:25.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.723+0000 7fcd20a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd1c1524e0 0x7fcd1c1672b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:25.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.723+0000 7fcd13e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd1c15fbf0 0x7fcd1c160000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48928/0 (socket says 172.21.15.86:48928) 2024-04-25T14:03:25.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.723+0000 7fcd13e00640 1 -- 172.21.15.86:0/1379696161 learned_addr learned my addr 172.21.15.86:0/1379696161 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:25.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.723+0000 7fcd21400640 1 --2- 172.21.15.86:0/1379696161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd1c1fcf40 0x7fcd1c1fd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:25.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.723+0000 7fcd13e00640 1 -- 172.21.15.86:0/1379696161 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd1c1524e0 msgr2=0x7fcd1c1672b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:25.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.723+0000 7fcd13e00640 1 --2- 172.21.15.86:0/1379696161 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd1c1524e0 0x7fcd1c1672b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:25.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.723+0000 7fcd13e00640 1 -- 172.21.15.86:0/1379696161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd1c1fcf40 msgr2=0x7fcd1c1fd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:25.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.723+0000 7fcd13e00640 1 --2- 172.21.15.86:0/1379696161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd1c1fcf40 0x7fcd1c1fd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:25.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.723+0000 7fcd13e00640 1 -- 172.21.15.86:0/1379696161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1c1528f0 con 0x7fcd1c15fbf0 2024-04-25T14:03:25.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.723+0000 7fcd20a00640 1 --2- 172.21.15.86:0/1379696161 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd1c1524e0 0x7fcd1c1672b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:25.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.723+0000 7fcd13e00640 1 --2- 172.21.15.86:0/1379696161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd1c15fbf0 0x7fcd1c160000 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcd04001010 tx=0x7fcd0400fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:25.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.727+0000 7fcd19600640 1 -- 172.21.15.86:0/1379696161 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0400fe50 con 0x7fcd1c15fbf0 2024-04-25T14:03:25.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.727+0000 7fcd19600640 1 -- 172.21.15.86:0/1379696161 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd04018070 con 0x7fcd1c15fbf0 2024-04-25T14:03:25.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.727+0000 7fcd19600640 1 -- 172.21.15.86:0/1379696161 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0400d730 con 0x7fcd1c15fbf0 2024-04-25T14:03:25.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.727+0000 7fcd23400640 1 -- 172.21.15.86:0/1379696161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd1c1677f0 con 0x7fcd1c15fbf0 2024-04-25T14:03:25.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.727+0000 7fcd23400640 1 -- 172.21.15.86:0/1379696161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd1c153810 con 0x7fcd1c15fbf0 2024-04-25T14:03:25.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.727+0000 7fcd19600640 1 -- 172.21.15.86:0/1379696161 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcd0400d8d0 con 0x7fcd1c15fbf0 2024-04-25T14:03:25.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.727+0000 7fcd19600640 1 -- 172.21.15.86:0/1379696161 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcd0401d460 con 0x7fcd1c15fbf0 2024-04-25T14:03:25.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.727+0000 7fcd23400640 1 -- 172.21.15.86:0/1379696161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd1c0720e0 con 0x7fcd1c15fbf0 2024-04-25T14:03:25.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.735+0000 7fcd19600640 1 -- 172.21.15.86:0/1379696161 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fcd1c1528f0 con 0x7fcd1c15fbf0 2024-04-25T14:03:25.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.799+0000 7fcd23400640 1 -- 172.21.15.86:0/1379696161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fcd1c0b2520 con 0x7fcd1c15fbf0 2024-04-25T14:03:25.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.803+0000 7fcd19600640 1 -- 172.21.15.86:0/1379696161 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fcd04003dd0 con 0x7fcd1c15fbf0 2024-04-25T14:03:25.805 INFO:teuthology.orchestra.run.smithi086.stderr:adjusted tunables profile to default 2024-04-25T14:03:25.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.803+0000 7fcd23400640 1 -- 172.21.15.86:0/1379696161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd1c15fbf0 msgr2=0x7fcd1c160000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:25.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.803+0000 7fcd23400640 1 --2- 172.21.15.86:0/1379696161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd1c15fbf0 0x7fcd1c160000 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcd04001010 tx=0x7fcd0400fc00 comp rx=0 tx=0).stop 2024-04-25T14:03:25.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.803+0000 7fcd23400640 1 -- 172.21.15.86:0/1379696161 shutdown_connections 2024-04-25T14:03:25.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.803+0000 7fcd23400640 1 --2- 172.21.15.86:0/1379696161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd1c1fcf40 0x7fcd1c1fd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:25.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.803+0000 7fcd23400640 1 --2- 172.21.15.86:0/1379696161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd1c15fbf0 0x7fcd1c160000 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:25.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.803+0000 7fcd23400640 1 --2- 172.21.15.86:0/1379696161 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd1c1524e0 0x7fcd1c1672b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:25.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.803+0000 7fcd23400640 1 -- 172.21.15.86:0/1379696161 >> 172.21.15.86:0/1379696161 conn(0x7fcd1c0cffc0 msgr2=0x7fcd1c15ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:25.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.803+0000 7fcd23400640 1 -- 172.21.15.86:0/1379696161 shutdown_connections 2024-04-25T14:03:25.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:25.803+0000 7fcd23400640 1 -- 172.21.15.86:0/1379696161 wait complete. 2024-04-25T14:03:25.825 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T14:03:25.825 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T14:03:25.826 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:03:25.826 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T14:03:25.845 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:03:25.845 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T14:03:25.907 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:03:25.907 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T14:03:25.922 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:03:25.919+0000 7f3d0228d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:03:25.934 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:03:25.932+0000 7fcdc204d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:03:25.940 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:03:25.941 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T14:03:26.003 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:03:26.003 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T14:03:26.023 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:03:26.023 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T14:03:26.082 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:03:26.083 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T14:03:26.143 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:03:26.143 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T14:03:26.202 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new bf938de9-4676-4229-a12f-aae77996ae9b 0 2024-04-25T14:03:26.396 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.392+0000 7f8a9b600640 1 Processor -- start 2024-04-25T14:03:26.396 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.396+0000 7f8a9b600640 1 -- start start 2024-04-25T14:03:26.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.396+0000 7f8a9b600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8a940afe10 0x7f8a9406e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:26.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.396+0000 7f8a9b600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8a94161610 0x7f8a94161a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:26.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.396+0000 7f8a9b600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a9415a940 con 0x7f8a94160690 2024-04-25T14:03:26.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.396+0000 7f8a9b600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8a9406dbc0 con 0x7f8a94161610 2024-04-25T14:03:26.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.396+0000 7f8a9b600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f8a9406d290 con 0x7f8a940afe10 2024-04-25T14:03:26.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.396+0000 7f8a98c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8a940afe10 0x7f8a9406e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:26.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.396+0000 7f8a98c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8a940afe10 0x7f8a9406e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:49994/0 (socket says 172.21.15.155:49994) 2024-04-25T14:03:26.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.396+0000 7f8a98c00640 1 -- 172.21.15.155:0/1506012478 learned_addr learned my addr 172.21.15.155:0/1506012478 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T14:03:26.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.396+0000 7f8a93e00640 1 --2- 172.21.15.155:0/1506012478 >> v2:172.21.15.86:3300/0 conn(0x7f8a94161610 0x7f8a94161a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:26.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.396+0000 7f8a98c00640 1 -- 172.21.15.155:0/1506012478 >> v2:172.21.15.86:3300/0 conn(0x7f8a94161610 msgr2=0x7f8a94161a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:26.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.396+0000 7f8a98c00640 1 --2- 172.21.15.155:0/1506012478 >> v2:172.21.15.86:3300/0 conn(0x7f8a94161610 0x7f8a94161a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:26.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.396+0000 7f8a98c00640 1 -- 172.21.15.155:0/1506012478 >> v1:172.21.15.155:6789/0 conn(0x7f8a94160690 legacy=0x7f8a94160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:26.399 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.396+0000 7f8a98c00640 1 -- 172.21.15.155:0/1506012478 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a9406d570 con 0x7f8a940afe10 2024-04-25T14:03:26.401 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.400+0000 7f8a98c00640 1 --2- 172.21.15.155:0/1506012478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a940afe10 0x7f8a9406e050 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8a88009870 tx=0x7f8a8801c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe55273f17b6ade0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:26.401 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.400+0000 7f8a93400640 1 -- 172.21.15.155:0/1506012478 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a880b0070 con 0x7f8a940afe10 2024-04-25T14:03:26.402 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.400+0000 7f8a93400640 1 -- 172.21.15.155:0/1506012478 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a880ab600 con 0x7f8a940afe10 2024-04-25T14:03:26.402 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.400+0000 7f8a93400640 1 -- 172.21.15.155:0/1506012478 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a880ab9d0 con 0x7f8a940afe10 2024-04-25T14:03:26.403 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.400+0000 7f8a9b600640 1 -- 172.21.15.155:0/1506012478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a940afe10 msgr2=0x7f8a9406e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:26.403 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.400+0000 7f8a9b600640 1 --2- 172.21.15.155:0/1506012478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a940afe10 0x7f8a9406e050 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8a88009870 tx=0x7f8a8801c520 comp rx=0 tx=0).stop 2024-04-25T14:03:26.403 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.400+0000 7f8a9b600640 1 -- 172.21.15.155:0/1506012478 shutdown_connections 2024-04-25T14:03:26.403 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.400+0000 7f8a9b600640 1 --2- 172.21.15.155:0/1506012478 >> v2:172.21.15.86:3300/0 conn(0x7f8a94161610 0x7f8a94161a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:26.403 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.400+0000 7f8a9b600640 1 --2- 172.21.15.155:0/1506012478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a940afe10 0x7f8a9406e050 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:26.403 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.400+0000 7f8a9b600640 1 -- 172.21.15.155:0/1506012478 >> 172.21.15.155:0/1506012478 conn(0x7f8a94073da0 msgr2=0x7f8a940b7420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:26.403 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.400+0000 7f8a9b600640 1 -- 172.21.15.155:0/1506012478 shutdown_connections 2024-04-25T14:03:26.403 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.400+0000 7f8a9b600640 1 -- 172.21.15.155:0/1506012478 wait complete. 2024-04-25T14:03:26.405 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.404+0000 7f8a9b600640 1 Processor -- start 2024-04-25T14:03:26.405 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.404+0000 7f8a9b600640 1 -- start start 2024-04-25T14:03:26.407 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.404+0000 7f8a9b600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a940afe10 0x7f8a9409d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:26.407 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.404+0000 7f8a9b600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a94160690 0x7f8a94145d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:26.407 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.404+0000 7f8a9b600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a94161610 0x7f8a9415c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:26.407 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.404+0000 7f8a9b600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8a940b03a0 con 0x7f8a940afe10 2024-04-25T14:03:26.407 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.404+0000 7f8a9b600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8a940b0220 con 0x7f8a94161610 2024-04-25T14:03:26.407 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.404+0000 7f8a9b600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8a94161f60 con 0x7f8a94160690 2024-04-25T14:03:26.407 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.404+0000 7f8a93e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a94160690 0x7f8a94145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:26.407 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.404+0000 7f8a93e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a94160690 0x7f8a94145d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:53878/0 (socket says 172.21.15.155:53878) 2024-04-25T14:03:26.407 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.404+0000 7f8a93e00640 1 -- 172.21.15.155:0/2438034877 learned_addr learned my addr 172.21.15.155:0/2438034877 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T14:03:26.408 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.404+0000 7f8a99600640 1 --2- 172.21.15.155:0/2438034877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a94161610 0x7f8a9415c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:26.408 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.404+0000 7f8a98c00640 1 --2- 172.21.15.155:0/2438034877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a940afe10 0x7f8a9409d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:26.408 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.404+0000 7f8a93e00640 1 -- 172.21.15.155:0/2438034877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a94161610 msgr2=0x7f8a9415c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:26.408 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.404+0000 7f8a93e00640 1 --2- 172.21.15.155:0/2438034877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a94161610 0x7f8a9415c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:26.408 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.404+0000 7f8a93e00640 1 -- 172.21.15.155:0/2438034877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a940afe10 msgr2=0x7f8a9409d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:26.408 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.404+0000 7f8a93e00640 1 --2- 172.21.15.155:0/2438034877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a940afe10 0x7f8a9409d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:26.408 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.404+0000 7f8a93e00640 1 -- 172.21.15.155:0/2438034877 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a9409d7f0 con 0x7f8a94160690 2024-04-25T14:03:26.409 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:03:26.403+0000 7f3d0228d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:03:26.409 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.404+0000 7f8a93e00640 1 --2- 172.21.15.155:0/2438034877 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a94160690 0x7f8a94145d10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8a8000e840 tx=0x7f8a8000ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:26.409 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.404+0000 7f8a90c00640 1 -- 172.21.15.155:0/2438034877 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a80017070 con 0x7f8a94160690 2024-04-25T14:03:26.410 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.404+0000 7f8a90c00640 1 -- 172.21.15.155:0/2438034877 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a8000ee90 con 0x7f8a94160690 2024-04-25T14:03:26.410 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.408+0000 7f8a90c00640 1 -- 172.21.15.155:0/2438034877 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8000cb80 con 0x7f8a94160690 2024-04-25T14:03:26.410 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.408+0000 7f8a9b600640 1 -- 172.21.15.155:0/2438034877 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a94155270 con 0x7f8a94160690 2024-04-25T14:03:26.410 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.408+0000 7f8a9b600640 1 -- 172.21.15.155:0/2438034877 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a9409f1d0 con 0x7f8a94160690 2024-04-25T14:03:26.410 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.408+0000 7f8a90c00640 1 -- 172.21.15.155:0/2438034877 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8a8001a070 con 0x7f8a94160690 2024-04-25T14:03:26.410 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.408+0000 7f8a90c00640 1 -- 172.21.15.155:0/2438034877 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f8a80006830 con 0x7f8a94160690 2024-04-25T14:03:26.410 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.408+0000 7f8a9b600640 1 -- 172.21.15.155:0/2438034877 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a94154fe0 con 0x7f8a94160690 2024-04-25T14:03:26.411 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.408+0000 7f8a90c00640 1 -- 172.21.15.155:0/2438034877 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f8a9409d7f0 con 0x7f8a94160690 2024-04-25T14:03:26.451 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:03:26.448+0000 7fcdc204d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:03:26.478 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.476+0000 7f8a9b600640 1 -- 172.21.15.155:0/2438034877 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd new", "uuid": "bf938de9-4676-4229-a12f-aae77996ae9b", "id": 0} v 0) -- 0x7f8a940a07f0 con 0x7f8a94160690 2024-04-25T14:03:26.486 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.480+0000 7f8a90c00640 1 -- 172.21.15.155:0/2438034877 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bf938de9-4676-4229-a12f-aae77996ae9b", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f8a800059a0 con 0x7f8a94160690 2024-04-25T14:03:26.486 INFO:teuthology.orchestra.run.smithi155.stdout:0 2024-04-25T14:03:26.488 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.484+0000 7f8a9b600640 1 -- 172.21.15.155:0/2438034877 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a94160690 msgr2=0x7f8a94145d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:26.488 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.484+0000 7f8a9b600640 1 --2- 172.21.15.155:0/2438034877 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a94160690 0x7f8a94145d10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8a8000e840 tx=0x7f8a8000ec40 comp rx=0 tx=0).stop 2024-04-25T14:03:26.488 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.484+0000 7f8a9b600640 1 -- 172.21.15.155:0/2438034877 shutdown_connections 2024-04-25T14:03:26.488 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.484+0000 7f8a9b600640 1 --2- 172.21.15.155:0/2438034877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a94161610 0x7f8a9415c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:26.488 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.484+0000 7f8a9b600640 1 --2- 172.21.15.155:0/2438034877 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a94160690 0x7f8a94145d10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:26.488 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.484+0000 7f8a9b600640 1 --2- 172.21.15.155:0/2438034877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a940afe10 0x7f8a9409d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:26.488 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.484+0000 7f8a9b600640 1 -- 172.21.15.155:0/2438034877 >> 172.21.15.155:0/2438034877 conn(0x7f8a94073da0 msgr2=0x7f8a94160aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:26.488 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.484+0000 7f8a9b600640 1 -- 172.21.15.155:0/2438034877 shutdown_connections 2024-04-25T14:03:26.488 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.484+0000 7f8a9b600640 1 -- 172.21.15.155:0/2438034877 wait complete. 2024-04-25T14:03:26.504 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new a3cc1c0b-9570-4687-826a-9a6ffadbeb9b 1 2024-04-25T14:03:26.629 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:03:26.627+0000 7f3d0228d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:03:26.650 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.648+0000 7f9a7c000640 1 Processor -- start 2024-04-25T14:03:26.651 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.648+0000 7f9a7c000640 1 -- start start 2024-04-25T14:03:26.653 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.648+0000 7f9a7c000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9a7415dc90 0x7f9a74161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:26.653 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.648+0000 7f9a7c000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a740cd8d0 con 0x7f9a74071530 2024-04-25T14:03:26.653 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.648+0000 7f9a7c000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a740ce970 con 0x7f9a740d2c80 2024-04-25T14:03:26.653 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.648+0000 7f9a7c000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9a740ce720 con 0x7f9a7415dc90 2024-04-25T14:03:26.654 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.652+0000 7f9a79600640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f9a740d2c80 0x7f9a74070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:36544/0 (socket says 172.21.15.155:36544) 2024-04-25T14:03:26.654 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.652+0000 7f9a79600640 1 -- 172.21.15.155:0/2904159973 learned_addr learned my addr 172.21.15.155:0/2904159973 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-25T14:03:26.654 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.652+0000 7f9a7a000640 1 --2- 172.21.15.155:0/2904159973 >> v2:172.21.15.86:3300/0 conn(0x7f9a7415dc90 0x7f9a74161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:26.654 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.652+0000 7f9a73e00640 1 -- 172.21.15.155:0/2904159973 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 605888933 0 0) 0x7f9a740ce970 con 0x7f9a740d2c80 2024-04-25T14:03:26.654 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.652+0000 7f9a73e00640 1 -- 172.21.15.155:0/2904159973 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a5c003530 con 0x7f9a740d2c80 2024-04-25T14:03:26.654 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.652+0000 7f9a7a000640 1 -- 172.21.15.155:0/2904159973 >> v1:172.21.15.155:6789/0 conn(0x7f9a740d2c80 legacy=0x7f9a74070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:26.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.652+0000 7f9a7a000640 1 -- 172.21.15.155:0/2904159973 >> v1:172.21.15.86:6789/0 conn(0x7f9a74071530 legacy=0x7f9a741616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:26.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.652+0000 7f9a73e00640 1 -- 172.21.15.155:0/2904159973 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3841876908 0 0) 0x7f9a740cd8d0 con 0x7f9a74071530 2024-04-25T14:03:26.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.652+0000 7f9a7a000640 1 -- 172.21.15.155:0/2904159973 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a740d1be0 con 0x7f9a7415dc90 2024-04-25T14:03:26.657 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.652+0000 7f9a7a000640 1 --2- 172.21.15.155:0/2904159973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a7415dc90 0x7f9a74161df0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c009d90 tx=0x7f9a6c01f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69aed38282c69fdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:26.657 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.656+0000 7f9a73e00640 1 -- 172.21.15.155:0/2904159973 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c0b4070 con 0x7f9a7415dc90 2024-04-25T14:03:26.657 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.656+0000 7f9a73e00640 1 -- 172.21.15.155:0/2904159973 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a6c0af600 con 0x7f9a7415dc90 2024-04-25T14:03:26.657 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.656+0000 7f9a73e00640 1 -- 172.21.15.155:0/2904159973 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c0af9d0 con 0x7f9a7415dc90 2024-04-25T14:03:26.658 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.656+0000 7f9a7c000640 1 -- 172.21.15.155:0/2904159973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a7415dc90 msgr2=0x7f9a74161df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:26.658 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.656+0000 7f9a7c000640 1 --2- 172.21.15.155:0/2904159973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a7415dc90 0x7f9a74161df0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c009d90 tx=0x7f9a6c01f150 comp rx=0 tx=0).stop 2024-04-25T14:03:26.658 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.656+0000 7f9a7c000640 1 -- 172.21.15.155:0/2904159973 shutdown_connections 2024-04-25T14:03:26.658 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.656+0000 7f9a7c000640 1 --2- 172.21.15.155:0/2904159973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a7415dc90 0x7f9a74161df0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:26.658 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.656+0000 7f9a7c000640 1 -- 172.21.15.155:0/2904159973 >> 172.21.15.155:0/2904159973 conn(0x7f9a740cffc0 msgr2=0x7f9a740d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:26.658 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.656+0000 7f9a7c000640 1 -- 172.21.15.155:0/2904159973 shutdown_connections 2024-04-25T14:03:26.658 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.656+0000 7f9a7c000640 1 -- 172.21.15.155:0/2904159973 wait complete. 2024-04-25T14:03:26.659 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.656+0000 7f9a7c000640 1 Processor -- start 2024-04-25T14:03:26.659 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.656+0000 7f9a7c000640 1 -- start start 2024-04-25T14:03:26.664 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.660+0000 7f9a7c000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a740d2c80 0x7f9a74160810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:26.664 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.660+0000 7f9a7c000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a7415dc90 0x7f9a741579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:26.665 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.660+0000 7f9a7c000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a741580f0 0x7f9a74070e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:26.665 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.660+0000 7f9a7c000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9a740d2480 con 0x7f9a7415dc90 2024-04-25T14:03:26.665 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.660+0000 7f9a7c000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9a740d21a0 con 0x7f9a740d2c80 2024-04-25T14:03:26.665 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.660+0000 7f9a7c000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9a740d29a0 con 0x7f9a741580f0 2024-04-25T14:03:26.665 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.660+0000 7f9a79600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a740d2c80 0x7f9a74160810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:26.665 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.660+0000 7f9a79600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a740d2c80 0x7f9a74160810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:50020/0 (socket says 172.21.15.155:50020) 2024-04-25T14:03:26.665 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.660+0000 7f9a79600640 1 -- 172.21.15.155:0/308874845 learned_addr learned my addr 172.21.15.155:0/308874845 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T14:03:26.666 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.660+0000 7f9a7a000640 1 --2- 172.21.15.155:0/308874845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a741580f0 0x7f9a74070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:26.666 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.660+0000 7f9a78c00640 1 --2- 172.21.15.155:0/308874845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a7415dc90 0x7f9a741579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:26.666 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.660+0000 7f9a79600640 1 -- 172.21.15.155:0/308874845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a741580f0 msgr2=0x7f9a74070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:26.666 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.660+0000 7f9a79600640 1 --2- 172.21.15.155:0/308874845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a741580f0 0x7f9a74070e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:26.666 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.660+0000 7f9a79600640 1 -- 172.21.15.155:0/308874845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a7415dc90 msgr2=0x7f9a741579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:26.666 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.660+0000 7f9a79600640 1 --2- 172.21.15.155:0/308874845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a7415dc90 0x7f9a741579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:26.666 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.660+0000 7f9a79600640 1 -- 172.21.15.155:0/308874845 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a7415ee30 con 0x7f9a740d2c80 2024-04-25T14:03:26.666 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.660+0000 7f9a7a000640 1 --2- 172.21.15.155:0/308874845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a741580f0 0x7f9a74070e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:26.667 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.660+0000 7f9a79600640 1 --2- 172.21.15.155:0/308874845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a740d2c80 0x7f9a74160810 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9a6800c420 tx=0x7f9a6800c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:26.667 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.660+0000 7f9a71600640 1 -- 172.21.15.155:0/308874845 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a68017070 con 0x7f9a740d2c80 2024-04-25T14:03:26.667 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.660+0000 7f9a71600640 1 -- 172.21.15.155:0/308874845 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a68005cf0 con 0x7f9a740d2c80 2024-04-25T14:03:26.667 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.660+0000 7f9a7c000640 1 -- 172.21.15.155:0/308874845 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a7415f110 con 0x7f9a740d2c80 2024-04-25T14:03:26.667 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.660+0000 7f9a71600640 1 -- 172.21.15.155:0/308874845 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a680072d0 con 0x7f9a740d2c80 2024-04-25T14:03:26.667 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.660+0000 7f9a7c000640 1 -- 172.21.15.155:0/308874845 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a7415f6d0 con 0x7f9a740d2c80 2024-04-25T14:03:26.667 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.660+0000 7f9a71600640 1 -- 172.21.15.155:0/308874845 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9a6801a070 con 0x7f9a740d2c80 2024-04-25T14:03:26.667 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.660+0000 7f9a71600640 1 -- 172.21.15.155:0/308874845 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f9a680127b0 con 0x7f9a740d2c80 2024-04-25T14:03:26.668 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.660+0000 7f9a7c000640 1 -- 172.21.15.155:0/308874845 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a740720e0 con 0x7f9a740d2c80 2024-04-25T14:03:26.668 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.664+0000 7f9a71600640 1 -- 172.21.15.155:0/308874845 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f9a6801cdc0 con 0x7f9a740d2c80 2024-04-25T14:03:26.684 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:03:26.680+0000 7fcdc204d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:03:26.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.732+0000 7f9a7c000640 1 -- 172.21.15.155:0/308874845 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a3cc1c0b-9570-4687-826a-9a6ffadbeb9b", "id": 1} v 0) -- 0x7f9a74076bf0 con 0x7f9a740d2c80 2024-04-25T14:03:26.741 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.740+0000 7f9a71600640 1 -- 172.21.15.155:0/308874845 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a3cc1c0b-9570-4687-826a-9a6ffadbeb9b", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f9a68007650 con 0x7f9a740d2c80 2024-04-25T14:03:26.741 INFO:teuthology.orchestra.run.smithi155.stdout:1 2024-04-25T14:03:26.743 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.740+0000 7f9a7c000640 1 -- 172.21.15.155:0/308874845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a740d2c80 msgr2=0x7f9a74160810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:26.743 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.740+0000 7f9a7c000640 1 --2- 172.21.15.155:0/308874845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a740d2c80 0x7f9a74160810 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9a6800c420 tx=0x7f9a6800c8e0 comp rx=0 tx=0).stop 2024-04-25T14:03:26.744 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.740+0000 7f9a7c000640 1 -- 172.21.15.155:0/308874845 shutdown_connections 2024-04-25T14:03:26.744 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.740+0000 7f9a7c000640 1 --2- 172.21.15.155:0/308874845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a741580f0 0x7f9a74070e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:26.744 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.740+0000 7f9a7c000640 1 --2- 172.21.15.155:0/308874845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a7415dc90 0x7f9a741579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:26.744 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.740+0000 7f9a7c000640 1 --2- 172.21.15.155:0/308874845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a740d2c80 0x7f9a74160810 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:26.744 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.740+0000 7f9a7c000640 1 -- 172.21.15.155:0/308874845 >> 172.21.15.155:0/308874845 conn(0x7f9a740cffc0 msgr2=0x7f9a7406ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:26.744 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.740+0000 7f9a7c000640 1 -- 172.21.15.155:0/308874845 shutdown_connections 2024-04-25T14:03:26.744 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.740+0000 7f9a7c000640 1 -- 172.21.15.155:0/308874845 wait complete. 2024-04-25T14:03:26.762 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 2f65f8e2-ab65-49ce-ae5c-080ca193d8b5 2 2024-04-25T14:03:26.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.904+0000 7f0ca0200640 1 Processor -- start 2024-04-25T14:03:26.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.904+0000 7f0ca0200640 1 -- start start 2024-04-25T14:03:26.910 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.908+0000 7f0ca0200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0c98071530 0x7f0c9815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:26.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.908+0000 7f0ca0200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c980cd8d0 con 0x7f0c980d2c80 2024-04-25T14:03:26.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.908+0000 7f0ca0200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c980ce970 con 0x7f0c981616e0 2024-04-25T14:03:26.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.908+0000 7f0ca0200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0c980ce720 con 0x7f0c98071530 2024-04-25T14:03:26.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.908+0000 7f0c9ce00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0c98071530 0x7f0c9815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:26.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.908+0000 7f0c9ce00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0c98071530 0x7f0c9815dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:50028/0 (socket says 172.21.15.155:50028) 2024-04-25T14:03:26.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.908+0000 7f0c9ce00640 1 -- 172.21.15.155:0/1319287974 learned_addr learned my addr 172.21.15.155:0/1319287974 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T14:03:26.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.908+0000 7f0c9ce00640 1 -- 172.21.15.155:0/1319287974 >> v1:172.21.15.155:6789/0 conn(0x7f0c981616e0 legacy=0x7f0c98161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:26.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.908+0000 7f0c9ce00640 1 -- 172.21.15.155:0/1319287974 >> v1:172.21.15.86:6789/0 conn(0x7f0c980d2c80 legacy=0x7f0c98070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:26.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.908+0000 7f0c9ce00640 1 -- 172.21.15.155:0/1319287974 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c980d1be0 con 0x7f0c98071530 2024-04-25T14:03:26.913 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.912+0000 7f0c9ce00640 1 --2- 172.21.15.155:0/1319287974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c98071530 0x7f0c9815dc90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0c84006fd0 tx=0x7f0c8401ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=972b33c7cdfeb66b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:26.913 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.912+0000 7f0c97e00640 1 -- 172.21.15.155:0/1319287974 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c840b5070 con 0x7f0c98071530 2024-04-25T14:03:26.914 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.912+0000 7f0c97e00640 1 -- 172.21.15.155:0/1319287974 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c840a8a30 con 0x7f0c98071530 2024-04-25T14:03:26.914 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.912+0000 7f0c97e00640 1 -- 172.21.15.155:0/1319287974 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c840a8e00 con 0x7f0c98071530 2024-04-25T14:03:26.914 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.912+0000 7f0ca0200640 1 -- 172.21.15.155:0/1319287974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c98071530 msgr2=0x7f0c9815dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:26.915 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.912+0000 7f0ca0200640 1 --2- 172.21.15.155:0/1319287974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c98071530 0x7f0c9815dc90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0c84006fd0 tx=0x7f0c8401ed30 comp rx=0 tx=0).stop 2024-04-25T14:03:26.915 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.912+0000 7f0ca0200640 1 -- 172.21.15.155:0/1319287974 shutdown_connections 2024-04-25T14:03:26.915 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.912+0000 7f0ca0200640 1 --2- 172.21.15.155:0/1319287974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c98071530 0x7f0c9815dc90 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:26.915 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.912+0000 7f0ca0200640 1 -- 172.21.15.155:0/1319287974 >> 172.21.15.155:0/1319287974 conn(0x7f0c980cffc0 msgr2=0x7f0c980d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:26.915 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.912+0000 7f0ca0200640 1 -- 172.21.15.155:0/1319287974 shutdown_connections 2024-04-25T14:03:26.915 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.912+0000 7f0ca0200640 1 -- 172.21.15.155:0/1319287974 wait complete. 2024-04-25T14:03:26.917 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.916+0000 7f0ca0200640 1 Processor -- start 2024-04-25T14:03:26.917 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.916+0000 7f0ca0200640 1 -- start start 2024-04-25T14:03:26.920 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.916+0000 7f0ca0200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c980d2c80 0x7f0c980c9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:26.920 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.916+0000 7f0ca0200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c981616e0 0x7f0c980ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:26.920 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.916+0000 7f0ca0200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c981515c0 0x7f0c9814f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:26.920 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.916+0000 7f0ca0200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0c980d2480 con 0x7f0c981616e0 2024-04-25T14:03:26.920 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.916+0000 7f0ca0200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0c980d21a0 con 0x7f0c980d2c80 2024-04-25T14:03:26.920 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.916+0000 7f0ca0200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0c980d29a0 con 0x7f0c981515c0 2024-04-25T14:03:26.920 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.916+0000 7f0c9d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c980d2c80 0x7f0c980c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:26.920 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.916+0000 7f0c9d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c980d2c80 0x7f0c980c9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:50034/0 (socket says 172.21.15.155:50034) 2024-04-25T14:03:26.920 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.916+0000 7f0c9d800640 1 -- 172.21.15.155:0/415630990 learned_addr learned my addr 172.21.15.155:0/415630990 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T14:03:26.920 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.916+0000 7f0c9e200640 1 --2- 172.21.15.155:0/415630990 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c981515c0 0x7f0c9814f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:26.921 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.916+0000 7f0c9ce00640 1 --2- 172.21.15.155:0/415630990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c981616e0 0x7f0c980ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:26.921 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.916+0000 7f0c9d800640 1 -- 172.21.15.155:0/415630990 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c981515c0 msgr2=0x7f0c9814f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:26.921 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.916+0000 7f0c9d800640 1 --2- 172.21.15.155:0/415630990 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c981515c0 0x7f0c9814f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:26.921 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.916+0000 7f0c9d800640 1 -- 172.21.15.155:0/415630990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c981616e0 msgr2=0x7f0c980ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:26.921 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.916+0000 7f0c9d800640 1 --2- 172.21.15.155:0/415630990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c981616e0 0x7f0c980ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:26.921 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.916+0000 7f0c9d800640 1 -- 172.21.15.155:0/415630990 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c98150520 con 0x7f0c980d2c80 2024-04-25T14:03:26.921 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.916+0000 7f0c9d800640 1 --2- 172.21.15.155:0/415630990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c980d2c80 0x7f0c980c9020 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c00c3a0 tx=0x7f0c8c00c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:26.921 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.916+0000 7f0c95600640 1 -- 172.21.15.155:0/415630990 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c0052d0 con 0x7f0c980d2c80 2024-04-25T14:03:26.921 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.916+0000 7f0c95600640 1 -- 172.21.15.155:0/415630990 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c8c005470 con 0x7f0c980d2c80 2024-04-25T14:03:26.921 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.916+0000 7f0ca0200640 1 -- 172.21.15.155:0/415630990 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c98150800 con 0x7f0c980d2c80 2024-04-25T14:03:26.921 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.920+0000 7f0c95600640 1 -- 172.21.15.155:0/415630990 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c007260 con 0x7f0c980d2c80 2024-04-25T14:03:26.921 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.920+0000 7f0ca0200640 1 -- 172.21.15.155:0/415630990 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c98150dc0 con 0x7f0c980d2c80 2024-04-25T14:03:26.921 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.920+0000 7f0c95600640 1 -- 172.21.15.155:0/415630990 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0c8c007400 con 0x7f0c980d2c80 2024-04-25T14:03:26.921 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.920+0000 7f0c95600640 1 -- 172.21.15.155:0/415630990 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f0c8c003de0 con 0x7f0c980d2c80 2024-04-25T14:03:26.922 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.920+0000 7f0ca0200640 1 -- 172.21.15.155:0/415630990 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c980720e0 con 0x7f0c980d2c80 2024-04-25T14:03:26.923 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.920+0000 7f0c95600640 1 -- 172.21.15.155:0/415630990 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0c8c0039c0 con 0x7f0c980d2c80 2024-04-25T14:03:26.990 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.988+0000 7f0ca0200640 1 -- 172.21.15.155:0/415630990 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2f65f8e2-ab65-49ce-ae5c-080ca193d8b5", "id": 2} v 0) -- 0x7f0c9815bb10 con 0x7f0c980d2c80 2024-04-25T14:03:26.996 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.996+0000 7f0c95600640 1 -- 172.21.15.155:0/415630990 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2f65f8e2-ab65-49ce-ae5c-080ca193d8b5", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f0c8c0067e0 con 0x7f0c980d2c80 2024-04-25T14:03:26.996 INFO:teuthology.orchestra.run.smithi155.stdout:2 2024-04-25T14:03:26.998 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.996+0000 7f0ca0200640 1 -- 172.21.15.155:0/415630990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c980d2c80 msgr2=0x7f0c980c9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:26.998 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.996+0000 7f0ca0200640 1 --2- 172.21.15.155:0/415630990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c980d2c80 0x7f0c980c9020 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c00c3a0 tx=0x7f0c8c00c860 comp rx=0 tx=0).stop 2024-04-25T14:03:26.998 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.996+0000 7f0ca0200640 1 -- 172.21.15.155:0/415630990 shutdown_connections 2024-04-25T14:03:26.998 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.996+0000 7f0ca0200640 1 --2- 172.21.15.155:0/415630990 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c981515c0 0x7f0c9814f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:26.998 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.996+0000 7f0ca0200640 1 --2- 172.21.15.155:0/415630990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c981616e0 0x7f0c980ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:26.998 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.996+0000 7f0ca0200640 1 --2- 172.21.15.155:0/415630990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c980d2c80 0x7f0c980c9020 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:26.998 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.996+0000 7f0ca0200640 1 -- 172.21.15.155:0/415630990 >> 172.21.15.155:0/415630990 conn(0x7f0c980cffc0 msgr2=0x7f0c980c9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:26.998 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.996+0000 7f0ca0200640 1 -- 172.21.15.155:0/415630990 shutdown_connections 2024-04-25T14:03:26.998 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:26.996+0000 7f0ca0200640 1 -- 172.21.15.155:0/415630990 wait complete. 2024-04-25T14:03:27.017 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 23909d38-59a9-41d9-af18-aec4a8cbfa2e 3 2024-04-25T14:03:27.163 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.160+0000 7f4454400640 1 Processor -- start 2024-04-25T14:03:27.163 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.160+0000 7f4454400640 1 -- start start 2024-04-25T14:03:27.164 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.164+0000 7f4454400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f444c0cdda0 0x7f444c0cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.165 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.164+0000 7f4454400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f444c0d18e0 0x7f444c0d1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.165 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.164+0000 7f4454400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f444c163940 con 0x7f444c0cec60 2024-04-25T14:03:27.165 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.164+0000 7f4454400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f444c0cfcc0 con 0x7f444c0cdda0 2024-04-25T14:03:27.165 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.164+0000 7f4454400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f444c0d2f40 con 0x7f444c0d18e0 2024-04-25T14:03:27.165 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.164+0000 7f4451000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f444c0cdda0 0x7f444c0cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.165 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.164+0000 7f4452400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f444c0d18e0 0x7f444c0d1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.165 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.164+0000 7f4451000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f444c0cdda0 0x7f444c0cf780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:50046/0 (socket says 172.21.15.155:50046) 2024-04-25T14:03:27.165 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.164+0000 7f4451000640 1 -- 172.21.15.155:0/414009097 learned_addr learned my addr 172.21.15.155:0/414009097 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T14:03:27.165 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.164+0000 7f4452400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f444c0d18e0 0x7f444c0d1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:53912/0 (socket says 172.21.15.155:53912) 2024-04-25T14:03:27.166 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.164+0000 7f4452400640 1 -- 172.21.15.155:0/414009097 >> v2:172.21.15.155:3300/0 conn(0x7f444c0cdda0 msgr2=0x7f444c0cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:27.166 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.164+0000 7f4452400640 1 --2- 172.21.15.155:0/414009097 >> v2:172.21.15.155:3300/0 conn(0x7f444c0cdda0 0x7f444c0cf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.166 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.164+0000 7f4452400640 1 -- 172.21.15.155:0/414009097 >> v1:172.21.15.155:6789/0 conn(0x7f444c0cec60 legacy=0x7f444c0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:27.166 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.164+0000 7f4452400640 1 -- 172.21.15.155:0/414009097 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f444c0b3720 con 0x7f444c0d18e0 2024-04-25T14:03:27.168 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.168+0000 7f4452400640 1 --2- 172.21.15.155:0/414009097 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f444c0d18e0 0x7f444c0d1cf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4444009d90 tx=0x7f444401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71852554f2990642 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:27.168 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.168+0000 7f444be00640 1 -- 172.21.15.155:0/414009097 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44440b2070 con 0x7f444c0d18e0 2024-04-25T14:03:27.168 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.168+0000 7f444be00640 1 -- 172.21.15.155:0/414009097 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44440a5830 con 0x7f444c0d18e0 2024-04-25T14:03:27.168 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.168+0000 7f444be00640 1 -- 172.21.15.155:0/414009097 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44440a5c00 con 0x7f444c0d18e0 2024-04-25T14:03:27.169 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.168+0000 7f4454400640 1 -- 172.21.15.155:0/414009097 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f444c0d18e0 msgr2=0x7f444c0d1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:27.169 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.168+0000 7f4454400640 1 --2- 172.21.15.155:0/414009097 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f444c0d18e0 0x7f444c0d1cf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4444009d90 tx=0x7f444401c970 comp rx=0 tx=0).stop 2024-04-25T14:03:27.169 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.168+0000 7f4454400640 1 -- 172.21.15.155:0/414009097 shutdown_connections 2024-04-25T14:03:27.169 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.168+0000 7f4454400640 1 --2- 172.21.15.155:0/414009097 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f444c0d18e0 0x7f444c0d1cf0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.169 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.168+0000 7f4454400640 1 --2- 172.21.15.155:0/414009097 >> v2:172.21.15.155:3300/0 conn(0x7f444c0cdda0 0x7f444c0cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.170 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.168+0000 7f4454400640 1 -- 172.21.15.155:0/414009097 >> 172.21.15.155:0/414009097 conn(0x7f444c09ba30 msgr2=0x7f444c0b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:27.170 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.168+0000 7f4454400640 1 -- 172.21.15.155:0/414009097 shutdown_connections 2024-04-25T14:03:27.170 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.168+0000 7f4454400640 1 -- 172.21.15.155:0/414009097 wait complete. 2024-04-25T14:03:27.171 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.168+0000 7f4454400640 1 Processor -- start 2024-04-25T14:03:27.171 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.168+0000 7f4454400640 1 -- start start 2024-04-25T14:03:27.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.172+0000 7f4454400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f444c0cdda0 0x7f444c0cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.172+0000 7f4454400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f444c0cec60 0x7f444c14aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.172+0000 7f4454400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f444c0d18e0 0x7f444c15df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.172+0000 7f4454400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f444c16cbf0 con 0x7f444c0cdda0 2024-04-25T14:03:27.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.172+0000 7f4454400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f444c0d2c50 con 0x7f444c0d18e0 2024-04-25T14:03:27.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.172+0000 7f4454400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f444c16cd70 con 0x7f444c0cec60 2024-04-25T14:03:27.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.172+0000 7f4451000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f444c0cec60 0x7f444c14aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.172+0000 7f4451000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f444c0cec60 0x7f444c14aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:53924/0 (socket says 172.21.15.155:53924) 2024-04-25T14:03:27.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.172+0000 7f4451000640 1 -- 172.21.15.155:0/660330997 learned_addr learned my addr 172.21.15.155:0/660330997 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T14:03:27.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.172+0000 7f4452400640 1 --2- 172.21.15.155:0/660330997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f444c0d18e0 0x7f444c15df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.172+0000 7f4451a00640 1 --2- 172.21.15.155:0/660330997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f444c0cdda0 0x7f444c0cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.172+0000 7f4451000640 1 -- 172.21.15.155:0/660330997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f444c0d18e0 msgr2=0x7f444c15df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:27.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.172+0000 7f4451000640 1 --2- 172.21.15.155:0/660330997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f444c0d18e0 0x7f444c15df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.172+0000 7f4451000640 1 -- 172.21.15.155:0/660330997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f444c0cdda0 msgr2=0x7f444c0cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:27.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.172+0000 7f4451000640 1 --2- 172.21.15.155:0/660330997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f444c0cdda0 0x7f444c0cb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.172+0000 7f4451000640 1 -- 172.21.15.155:0/660330997 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f444c14afb0 con 0x7f444c0cec60 2024-04-25T14:03:27.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.172+0000 7f4451a00640 1 --2- 172.21.15.155:0/660330997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f444c0cdda0 0x7f444c0cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:27.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.172+0000 7f4451000640 1 --2- 172.21.15.155:0/660330997 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f444c0cec60 0x7f444c14aa70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f443800e820 tx=0x7f443800ece0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:27.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.172+0000 7f4449600640 1 -- 172.21.15.155:0/660330997 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4438017070 con 0x7f444c0cec60 2024-04-25T14:03:27.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.172+0000 7f4449600640 1 -- 172.21.15.155:0/660330997 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f443800c940 con 0x7f444c0cec60 2024-04-25T14:03:27.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.172+0000 7f4454400640 1 -- 172.21.15.155:0/660330997 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f444c0ca740 con 0x7f444c0cec60 2024-04-25T14:03:27.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.172+0000 7f4449600640 1 -- 172.21.15.155:0/660330997 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443800cd30 con 0x7f444c0cec60 2024-04-25T14:03:27.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.172+0000 7f4454400640 1 -- 172.21.15.155:0/660330997 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f444c0cad00 con 0x7f444c0cec60 2024-04-25T14:03:27.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.172+0000 7f4449600640 1 -- 172.21.15.155:0/660330997 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f443801a070 con 0x7f444c0cec60 2024-04-25T14:03:27.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.172+0000 7f4449600640 1 -- 172.21.15.155:0/660330997 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f4438013900 con 0x7f444c0cec60 2024-04-25T14:03:27.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.172+0000 7f4454400640 1 -- 172.21.15.155:0/660330997 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f444c0a04c0 con 0x7f444c0cec60 2024-04-25T14:03:27.178 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.176+0000 7f4449600640 1 -- 172.21.15.155:0/660330997 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f4438013470 con 0x7f444c0cec60 2024-04-25T14:03:27.244 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.244+0000 7f4454400640 1 -- 172.21.15.155:0/660330997 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd new", "uuid": "23909d38-59a9-41d9-af18-aec4a8cbfa2e", "id": 3} v 0) -- 0x7f444c076bf0 con 0x7f444c0cec60 2024-04-25T14:03:27.249 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.248+0000 7f4449600640 1 -- 172.21.15.155:0/660330997 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "23909d38-59a9-41d9-af18-aec4a8cbfa2e", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f44380136f0 con 0x7f444c0cec60 2024-04-25T14:03:27.249 INFO:teuthology.orchestra.run.smithi155.stdout:3 2024-04-25T14:03:27.250 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.248+0000 7f4454400640 1 -- 172.21.15.155:0/660330997 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f444c0cec60 msgr2=0x7f444c14aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:27.251 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.248+0000 7f4454400640 1 --2- 172.21.15.155:0/660330997 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f444c0cec60 0x7f444c14aa70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f443800e820 tx=0x7f443800ece0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.251 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.248+0000 7f4454400640 1 -- 172.21.15.155:0/660330997 shutdown_connections 2024-04-25T14:03:27.251 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.248+0000 7f4454400640 1 --2- 172.21.15.155:0/660330997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f444c0d18e0 0x7f444c15df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.251 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.248+0000 7f4454400640 1 --2- 172.21.15.155:0/660330997 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f444c0cec60 0x7f444c14aa70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.251 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.248+0000 7f4454400640 1 --2- 172.21.15.155:0/660330997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f444c0cdda0 0x7f444c0cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.251 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.248+0000 7f4454400640 1 -- 172.21.15.155:0/660330997 >> 172.21.15.155:0/660330997 conn(0x7f444c09ba30 msgr2=0x7f444c162a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:27.251 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.248+0000 7f4454400640 1 -- 172.21.15.155:0/660330997 shutdown_connections 2024-04-25T14:03:27.251 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.248+0000 7f4454400640 1 -- 172.21.15.155:0/660330997 wait complete. 2024-04-25T14:03:27.253 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:03:27.251+0000 7f3d0228d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:03:27.269 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 57195638-5b04-4bc6-b52b-72d9f8808cd3 4 2024-04-25T14:03:27.326 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:03:27.323+0000 7f3d0228d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:03:27.333 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:03:27.332+0000 7fcdc204d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:03:27.386 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:03:27.383+0000 7f3d0228d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:03:27.398 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:03:27.396+0000 7fcdc204d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:03:27.416 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.416+0000 7f5b75c00640 1 Processor -- start 2024-04-25T14:03:27.417 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.416+0000 7f5b75c00640 1 -- start start 2024-04-25T14:03:27.419 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.416+0000 7f5b75c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b70071530 0x7f5b7015dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.419 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.416+0000 7f5b75c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5b701616e0 0x7f5b70161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.419 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.416+0000 7f5b75c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b700cd8d0 con 0x7f5b700d2c80 2024-04-25T14:03:27.420 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.416+0000 7f5b75c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f5b700ce720 con 0x7f5b701616e0 2024-04-25T14:03:27.420 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.416+0000 7f5b75c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5b700ce970 con 0x7f5b70071530 2024-04-25T14:03:27.420 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.416+0000 7f5b6e000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b70071530 0x7f5b7015dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.420 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.416+0000 7f5b6f400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5b701616e0 0x7f5b70161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.420 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.416+0000 7f5b6e000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b70071530 0x7f5b7015dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:53934/0 (socket says 172.21.15.155:53934) 2024-04-25T14:03:27.420 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.416+0000 7f5b6e000640 1 -- 172.21.15.155:0/943321502 learned_addr learned my addr 172.21.15.155:0/943321502 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T14:03:27.420 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.416+0000 7f5b6f400640 1 -- 172.21.15.155:0/943321502 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b70071530 msgr2=0x7f5b7015dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:27.420 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.416+0000 7f5b6f400640 1 --2- 172.21.15.155:0/943321502 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b70071530 0x7f5b7015dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.420 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.416+0000 7f5b6f400640 1 -- 172.21.15.155:0/943321502 >> v1:172.21.15.86:6789/0 conn(0x7f5b700d2c80 legacy=0x7f5b70070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:27.420 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.416+0000 7f5b6f400640 1 -- 172.21.15.155:0/943321502 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b700d1be0 con 0x7f5b701616e0 2024-04-25T14:03:27.420 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.416+0000 7f5b6e000640 1 --2- 172.21.15.155:0/943321502 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b70071530 0x7f5b7015dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:27.420 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.416+0000 7f5b6d600640 1 -- 172.21.15.155:0/943321502 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 218806550 0 0) 0x7f5b700cd8d0 con 0x7f5b700d2c80 2024-04-25T14:03:27.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.420+0000 7f5b6f400640 1 --2- 172.21.15.155:0/943321502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b701616e0 0x7f5b70161af0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009d90 tx=0x7f5b6801f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3a92dac0148fd32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:27.423 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.420+0000 7f5b6d600640 1 -- 172.21.15.155:0/943321502 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b680b4070 con 0x7f5b701616e0 2024-04-25T14:03:27.423 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.420+0000 7f5b6d600640 1 -- 172.21.15.155:0/943321502 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b680af600 con 0x7f5b701616e0 2024-04-25T14:03:27.423 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.420+0000 7f5b6d600640 1 -- 172.21.15.155:0/943321502 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b680af9d0 con 0x7f5b701616e0 2024-04-25T14:03:27.424 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.420+0000 7f5b75c00640 1 -- 172.21.15.155:0/943321502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b701616e0 msgr2=0x7f5b70161af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:27.424 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.420+0000 7f5b75c00640 1 --2- 172.21.15.155:0/943321502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b701616e0 0x7f5b70161af0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009d90 tx=0x7f5b6801f150 comp rx=0 tx=0).stop 2024-04-25T14:03:27.424 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.420+0000 7f5b75c00640 1 -- 172.21.15.155:0/943321502 shutdown_connections 2024-04-25T14:03:27.425 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.420+0000 7f5b75c00640 1 --2- 172.21.15.155:0/943321502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b701616e0 0x7f5b70161af0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.425 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.420+0000 7f5b75c00640 1 --2- 172.21.15.155:0/943321502 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b70071530 0x7f5b7015dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.425 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.420+0000 7f5b75c00640 1 -- 172.21.15.155:0/943321502 >> 172.21.15.155:0/943321502 conn(0x7f5b700cffc0 msgr2=0x7f5b700d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:27.425 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.420+0000 7f5b75c00640 1 -- 172.21.15.155:0/943321502 shutdown_connections 2024-04-25T14:03:27.425 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.420+0000 7f5b75c00640 1 -- 172.21.15.155:0/943321502 wait complete. 2024-04-25T14:03:27.426 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.424+0000 7f5b75c00640 1 Processor -- start 2024-04-25T14:03:27.426 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.424+0000 7f5b75c00640 1 -- start start 2024-04-25T14:03:27.430 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.424+0000 7f5b75c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b700d2c80 0x7f5b700c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.430 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.424+0000 7f5b75c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b701616e0 0x7f5b7014c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.430 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.424+0000 7f5b75c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b7014c8f0 0x7f5b7014e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.430 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.424+0000 7f5b75c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b700d2480 con 0x7f5b701616e0 2024-04-25T14:03:27.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.424+0000 7f5b75c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5b700d21a0 con 0x7f5b700d2c80 2024-04-25T14:03:27.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.424+0000 7f5b75c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5b700d29a0 con 0x7f5b7014c8f0 2024-04-25T14:03:27.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.424+0000 7f5b6ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b700d2c80 0x7f5b700c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.424+0000 7f5b6f400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b7014c8f0 0x7f5b7014e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.424+0000 7f5b6ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b700d2c80 0x7f5b700c8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:50086/0 (socket says 172.21.15.155:50086) 2024-04-25T14:03:27.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.424+0000 7f5b6ea00640 1 -- 172.21.15.155:0/2758154326 learned_addr learned my addr 172.21.15.155:0/2758154326 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T14:03:27.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.424+0000 7f5b6e000640 1 --2- 172.21.15.155:0/2758154326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b701616e0 0x7f5b7014c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.428+0000 7f5b6ea00640 1 -- 172.21.15.155:0/2758154326 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b7014c8f0 msgr2=0x7f5b7014e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:27.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.428+0000 7f5b6ea00640 1 --2- 172.21.15.155:0/2758154326 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b7014c8f0 0x7f5b7014e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.428+0000 7f5b6ea00640 1 -- 172.21.15.155:0/2758154326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b701616e0 msgr2=0x7f5b7014c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:27.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.428+0000 7f5b6ea00640 1 --2- 172.21.15.155:0/2758154326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b701616e0 0x7f5b7014c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.428+0000 7f5b6ea00640 1 -- 172.21.15.155:0/2758154326 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b7014d070 con 0x7f5b700d2c80 2024-04-25T14:03:27.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.428+0000 7f5b6e000640 1 --2- 172.21.15.155:0/2758154326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b701616e0 0x7f5b7014c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:27.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.428+0000 7f5b6ea00640 1 --2- 172.21.15.155:0/2758154326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b700d2c80 0x7f5b700c8e30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5b6000c480 tx=0x7f5b6000c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:27.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.428+0000 7f5b66a00640 1 -- 172.21.15.155:0/2758154326 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60017070 con 0x7f5b700d2c80 2024-04-25T14:03:27.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.428+0000 7f5b66a00640 1 -- 172.21.15.155:0/2758154326 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b60005480 con 0x7f5b700d2c80 2024-04-25T14:03:27.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.428+0000 7f5b75c00640 1 -- 172.21.15.155:0/2758154326 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b7014d350 con 0x7f5b700d2c80 2024-04-25T14:03:27.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.428+0000 7f5b66a00640 1 -- 172.21.15.155:0/2758154326 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60003840 con 0x7f5b700d2c80 2024-04-25T14:03:27.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.428+0000 7f5b75c00640 1 -- 172.21.15.155:0/2758154326 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b7014d910 con 0x7f5b700d2c80 2024-04-25T14:03:27.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.428+0000 7f5b66a00640 1 -- 172.21.15.155:0/2758154326 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5b6001a070 con 0x7f5b700d2c80 2024-04-25T14:03:27.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.428+0000 7f5b66a00640 1 -- 172.21.15.155:0/2758154326 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f5b60012910 con 0x7f5b700d2c80 2024-04-25T14:03:27.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.428+0000 7f5b75c00640 1 -- 172.21.15.155:0/2758154326 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b70076bf0 con 0x7f5b700d2c80 2024-04-25T14:03:27.442 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.440+0000 7f5b66a00640 1 -- 172.21.15.155:0/2758154326 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5b60021070 con 0x7f5b700d2c80 2024-04-25T14:03:27.464 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:03:27.464+0000 7fcdc204d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:03:27.474 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:03:27.471+0000 7f3d0228d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:03:27.510 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.508+0000 7f5b75c00640 1 -- 172.21.15.155:0/2758154326 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd new", "uuid": "57195638-5b04-4bc6-b52b-72d9f8808cd3", "id": 4} v 0) -- 0x7f5b700d1470 con 0x7f5b700d2c80 2024-04-25T14:03:27.515 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.512+0000 7f5b66a00640 1 -- 172.21.15.155:0/2758154326 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "57195638-5b04-4bc6-b52b-72d9f8808cd3", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f5b60012d80 con 0x7f5b700d2c80 2024-04-25T14:03:27.515 INFO:teuthology.orchestra.run.smithi155.stdout:4 2024-04-25T14:03:27.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.516+0000 7f5b75c00640 1 -- 172.21.15.155:0/2758154326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b700d2c80 msgr2=0x7f5b700c8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:27.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.516+0000 7f5b75c00640 1 --2- 172.21.15.155:0/2758154326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b700d2c80 0x7f5b700c8e30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5b6000c480 tx=0x7f5b6000c940 comp rx=0 tx=0).stop 2024-04-25T14:03:27.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.516+0000 7f5b75c00640 1 -- 172.21.15.155:0/2758154326 shutdown_connections 2024-04-25T14:03:27.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.516+0000 7f5b75c00640 1 --2- 172.21.15.155:0/2758154326 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b7014c8f0 0x7f5b7014e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.516+0000 7f5b75c00640 1 --2- 172.21.15.155:0/2758154326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b701616e0 0x7f5b7014c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.516+0000 7f5b75c00640 1 --2- 172.21.15.155:0/2758154326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b700d2c80 0x7f5b700c8e30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.516+0000 7f5b75c00640 1 -- 172.21.15.155:0/2758154326 >> 172.21.15.155:0/2758154326 conn(0x7f5b700cffc0 msgr2=0x7f5b700ca580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:27.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.516+0000 7f5b75c00640 1 -- 172.21.15.155:0/2758154326 shutdown_connections 2024-04-25T14:03:27.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.516+0000 7f5b75c00640 1 -- 172.21.15.155:0/2758154326 wait complete. 2024-04-25T14:03:27.536 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new f888702c-87b9-481e-9dd8-c3a97adea592 5 2024-04-25T14:03:27.537 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:03:27.531+0000 7f3d0228d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:03:27.556 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:03:27.556+0000 7fcdc204d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:03:27.619 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:03:27.616+0000 7fcdc204d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:03:27.623 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:03:27.619+0000 7f3d0228d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:03:27.683 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.680+0000 7fd0a1800640 1 Processor -- start 2024-04-25T14:03:27.683 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.680+0000 7fd0a1800640 1 -- start start 2024-04-25T14:03:27.685 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.684+0000 7fd0a1800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd09c072db0 0x7fd09c160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.685 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.684+0000 7fd0a1800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd09c0d1110 0x7fd09c0cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.685 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.684+0000 7fd0a1800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd09c0b7be0 con 0x7fd09c0b8c80 2024-04-25T14:03:27.686 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.684+0000 7fd0a1800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd09c0b87f0 con 0x7fd09c072db0 2024-04-25T14:03:27.686 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.684+0000 7fd0a1800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd09c0b7ec0 con 0x7fd09c0d1110 2024-04-25T14:03:27.686 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.684+0000 7fd09b400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd09c0d1110 0x7fd09c0cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.686 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.684+0000 7fd09b400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd09c0d1110 0x7fd09c0cdb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:50098/0 (socket says 172.21.15.155:50098) 2024-04-25T14:03:27.686 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.684+0000 7fd09b400640 1 -- 172.21.15.155:0/3807355189 learned_addr learned my addr 172.21.15.155:0/3807355189 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T14:03:27.686 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.684+0000 7fd09a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd09c072db0 0x7fd09c160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.686 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:03:27.683+0000 7f3d0228d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:03:27.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.684+0000 7fd09b400640 1 -- 172.21.15.155:0/3807355189 >> v2:172.21.15.86:3300/0 conn(0x7fd09c072db0 msgr2=0x7fd09c160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:27.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.684+0000 7fd09b400640 1 --2- 172.21.15.155:0/3807355189 >> v2:172.21.15.86:3300/0 conn(0x7fd09c072db0 0x7fd09c160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.684+0000 7fd09b400640 1 -- 172.21.15.155:0/3807355189 >> v1:172.21.15.86:6789/0 conn(0x7fd09c0b8c80 legacy=0x7fd09c0726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:27.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.684+0000 7fd09b400640 1 -- 172.21.15.155:0/3807355189 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd09c0b81a0 con 0x7fd09c0d1110 2024-04-25T14:03:27.688 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.688+0000 7fd09b400640 1 --2- 172.21.15.155:0/3807355189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd09c0d1110 0x7fd09c0cdb90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd09c074ac0 tx=0x7fd09401c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddc481cb293c1b4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:27.689 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.688+0000 7fd099600640 1 -- 172.21.15.155:0/3807355189 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0940a56a0 con 0x7fd09c0d1110 2024-04-25T14:03:27.689 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.688+0000 7fd099600640 1 -- 172.21.15.155:0/3807355189 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0940a5840 con 0x7fd09c0d1110 2024-04-25T14:03:27.689 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.688+0000 7fd099600640 1 -- 172.21.15.155:0/3807355189 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0940a5c10 con 0x7fd09c0d1110 2024-04-25T14:03:27.690 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.688+0000 7fd0a1800640 1 -- 172.21.15.155:0/3807355189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd09c0d1110 msgr2=0x7fd09c0cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:27.691 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.688+0000 7fd0a1800640 1 --2- 172.21.15.155:0/3807355189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd09c0d1110 0x7fd09c0cdb90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd09c074ac0 tx=0x7fd09401c8d0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.691 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.688+0000 7fd0a1800640 1 -- 172.21.15.155:0/3807355189 shutdown_connections 2024-04-25T14:03:27.691 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.688+0000 7fd0a1800640 1 --2- 172.21.15.155:0/3807355189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd09c0d1110 0x7fd09c0cdb90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.691 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.688+0000 7fd0a1800640 1 --2- 172.21.15.155:0/3807355189 >> v2:172.21.15.86:3300/0 conn(0x7fd09c072db0 0x7fd09c160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.691 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.688+0000 7fd0a1800640 1 -- 172.21.15.155:0/3807355189 >> 172.21.15.155:0/3807355189 conn(0x7fd09c0d3540 msgr2=0x7fd09c0d3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:27.691 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.688+0000 7fd0a1800640 1 -- 172.21.15.155:0/3807355189 shutdown_connections 2024-04-25T14:03:27.691 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.688+0000 7fd0a1800640 1 -- 172.21.15.155:0/3807355189 wait complete. 2024-04-25T14:03:27.694 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.692+0000 7fd0a1800640 1 Processor -- start 2024-04-25T14:03:27.694 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.692+0000 7fd0a1800640 1 -- start start 2024-04-25T14:03:27.697 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.696+0000 7fd0a1800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd09c0b8c80 0x7fd09c14ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.697 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.696+0000 7fd0a1800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd09c0d1110 0x7fd09c0ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.698 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.696+0000 7fd0a1800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd09c14baa0 0x7fd09c0ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.698 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.696+0000 7fd0a1800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c16d340 con 0x7fd09c14baa0 2024-04-25T14:03:27.698 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.696+0000 7fd0a1800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c16d1c0 con 0x7fd09c0b8c80 2024-04-25T14:03:27.698 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.696+0000 7fd0a1800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd09c16d4c0 con 0x7fd09c0d1110 2024-04-25T14:03:27.698 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.696+0000 7fd09aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd09c0b8c80 0x7fd09c14ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.698 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.696+0000 7fd09a000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd09c0d1110 0x7fd09c0ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.698 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.696+0000 7fd09aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd09c0b8c80 0x7fd09c14ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:50108/0 (socket says 172.21.15.155:50108) 2024-04-25T14:03:27.698 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.696+0000 7fd09aa00640 1 -- 172.21.15.155:0/4211638056 learned_addr learned my addr 172.21.15.155:0/4211638056 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T14:03:27.698 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.696+0000 7fd09b400640 1 --2- 172.21.15.155:0/4211638056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd09c14baa0 0x7fd09c0ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.698 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.696+0000 7fd09a000640 1 -- 172.21.15.155:0/4211638056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd09c0b8c80 msgr2=0x7fd09c14ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:27.698 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.696+0000 7fd09a000640 1 --2- 172.21.15.155:0/4211638056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd09c0b8c80 0x7fd09c14ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.698 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.696+0000 7fd09a000640 1 -- 172.21.15.155:0/4211638056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd09c14baa0 msgr2=0x7fd09c0ca9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:27.699 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.696+0000 7fd09a000640 1 --2- 172.21.15.155:0/4211638056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd09c14baa0 0x7fd09c0ca9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.699 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.696+0000 7fd09a000640 1 -- 172.21.15.155:0/4211638056 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd09c15f1f0 con 0x7fd09c0d1110 2024-04-25T14:03:27.699 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.696+0000 7fd09aa00640 1 --2- 172.21.15.155:0/4211638056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd09c0b8c80 0x7fd09c14ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:27.700 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.696+0000 7fd09a000640 1 --2- 172.21.15.155:0/4211638056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd09c0d1110 0x7fd09c0ca480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd08800e870 tx=0x7fd08800ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:27.700 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.696+0000 7fd092a00640 1 -- 172.21.15.155:0/4211638056 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd088017070 con 0x7fd09c0d1110 2024-04-25T14:03:27.700 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.696+0000 7fd092a00640 1 -- 172.21.15.155:0/4211638056 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd08800ee60 con 0x7fd09c0d1110 2024-04-25T14:03:27.700 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.696+0000 7fd0a1800640 1 -- 172.21.15.155:0/4211638056 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd09c15f4d0 con 0x7fd09c0d1110 2024-04-25T14:03:27.700 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.696+0000 7fd092a00640 1 -- 172.21.15.155:0/4211638056 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08800cb60 con 0x7fd09c0d1110 2024-04-25T14:03:27.700 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.696+0000 7fd0a1800640 1 -- 172.21.15.155:0/4211638056 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd09c0c9f60 con 0x7fd09c0d1110 2024-04-25T14:03:27.700 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.696+0000 7fd092a00640 1 -- 172.21.15.155:0/4211638056 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd08801a070 con 0x7fd09c0d1110 2024-04-25T14:03:27.700 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.696+0000 7fd092a00640 1 -- 172.21.15.155:0/4211638056 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fd0880139f0 con 0x7fd09c0d1110 2024-04-25T14:03:27.700 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.696+0000 7fd0a1800640 1 -- 172.21.15.155:0/4211638056 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd09c076bf0 con 0x7fd09c0d1110 2024-04-25T14:03:27.702 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.700+0000 7fd092a00640 1 -- 172.21.15.155:0/4211638056 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd08800cd00 con 0x7fd09c0d1110 2024-04-25T14:03:27.716 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:03:27.716+0000 7fcdc204d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:03:27.769 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.768+0000 7fd0a1800640 1 -- 172.21.15.155:0/4211638056 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f888702c-87b9-481e-9dd8-c3a97adea592", "id": 5} v 0) -- 0x7fd09c0d2970 con 0x7fd09c0d1110 2024-04-25T14:03:27.775 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.772+0000 7fd092a00640 1 -- 172.21.15.155:0/4211638056 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f888702c-87b9-481e-9dd8-c3a97adea592", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fd088013470 con 0x7fd09c0d1110 2024-04-25T14:03:27.776 INFO:teuthology.orchestra.run.smithi155.stdout:5 2024-04-25T14:03:27.777 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.776+0000 7fd0a1800640 1 -- 172.21.15.155:0/4211638056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd09c0d1110 msgr2=0x7fd09c0ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:27.777 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.776+0000 7fd0a1800640 1 --2- 172.21.15.155:0/4211638056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd09c0d1110 0x7fd09c0ca480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd08800e870 tx=0x7fd08800ed30 comp rx=0 tx=0).stop 2024-04-25T14:03:27.777 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.776+0000 7fd0a1800640 1 -- 172.21.15.155:0/4211638056 shutdown_connections 2024-04-25T14:03:27.777 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.776+0000 7fd0a1800640 1 --2- 172.21.15.155:0/4211638056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd09c14baa0 0x7fd09c0ca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.777 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.776+0000 7fd0a1800640 1 --2- 172.21.15.155:0/4211638056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd09c0d1110 0x7fd09c0ca480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.777 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.776+0000 7fd0a1800640 1 --2- 172.21.15.155:0/4211638056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd09c0b8c80 0x7fd09c14ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.777 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.776+0000 7fd0a1800640 1 -- 172.21.15.155:0/4211638056 >> 172.21.15.155:0/4211638056 conn(0x7fd09c0d3540 msgr2=0x7fd09c0cff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:27.777 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.776+0000 7fd0a1800640 1 -- 172.21.15.155:0/4211638056 shutdown_connections 2024-04-25T14:03:27.777 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.776+0000 7fd0a1800640 1 -- 172.21.15.155:0/4211638056 wait complete. 2024-04-25T14:03:27.782 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:03:27.780+0000 7fcdc204d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:03:27.796 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b 6 2024-04-25T14:03:27.853 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:03:27.851+0000 7f3d0228d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:03:27.923 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:03:27.919+0000 7f3d0228d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:03:27.953 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:03:27.952+0000 7fcdc204d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:03:27.987 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.984+0000 7f781a200640 1 Processor -- start 2024-04-25T14:03:27.987 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.984+0000 7f781a200640 1 -- start start 2024-04-25T14:03:27.989 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.988+0000 7f781a200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f78140d2c80 0x7f781415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.990 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.988+0000 7f781a200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7814070e20 0x7f7814071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.990 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.988+0000 7f781a200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f78140722f0 0x7f7814072700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.990 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.988+0000 7f781a200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f78140ce720 con 0x7f78140d2c80 2024-04-25T14:03:27.990 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.988+0000 7f781a200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f7814071770 con 0x7f78140722f0 2024-04-25T14:03:27.990 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.988+0000 7f781a200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f78140cd8d0 con 0x7f7814070e20 2024-04-25T14:03:27.990 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.988+0000 7f780aa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7814070e20 0x7f7814071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.990 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.988+0000 7f7813e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f78140722f0 0x7f7814072700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.990 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.988+0000 7f780aa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7814070e20 0x7f7814071230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:53956/0 (socket says 172.21.15.155:53956) 2024-04-25T14:03:27.990 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.988+0000 7f780aa00640 1 -- 172.21.15.155:0/3467871659 learned_addr learned my addr 172.21.15.155:0/3467871659 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T14:03:27.990 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.988+0000 7f7813400640 1 --2- 172.21.15.155:0/3467871659 >> v2:172.21.15.86:3300/0 conn(0x7f78140d2c80 0x7f781415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.990 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.988+0000 7f780aa00640 1 -- 172.21.15.155:0/3467871659 >> v2:172.21.15.155:3300/0 conn(0x7f78140722f0 msgr2=0x7f7814072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:27.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.988+0000 7f780aa00640 1 --2- 172.21.15.155:0/3467871659 >> v2:172.21.15.155:3300/0 conn(0x7f78140722f0 0x7f7814072700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.988+0000 7f780aa00640 1 -- 172.21.15.155:0/3467871659 >> v2:172.21.15.86:3300/0 conn(0x7f78140d2c80 msgr2=0x7f781415dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:27.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.988+0000 7f780aa00640 1 --2- 172.21.15.155:0/3467871659 >> v2:172.21.15.86:3300/0 conn(0x7f78140d2c80 0x7f781415dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.988+0000 7f780aa00640 1 -- 172.21.15.155:0/3467871659 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78140ce970 con 0x7f7814070e20 2024-04-25T14:03:27.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.988+0000 7f7813e00640 1 --2- 172.21.15.155:0/3467871659 >> v2:172.21.15.155:3300/0 conn(0x7f78140722f0 0x7f7814072700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:27.993 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.992+0000 7f780aa00640 1 --2- 172.21.15.155:0/3467871659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7814070e20 0x7f7814071230 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f78140736c0 tx=0x7f77fc01ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eb68dd94b5cb51f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:27.993 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.992+0000 7f7812a00640 1 -- 172.21.15.155:0/3467871659 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc0b5070 con 0x7f7814070e20 2024-04-25T14:03:27.993 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.992+0000 7f7812a00640 1 -- 172.21.15.155:0/3467871659 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77fc0a89d0 con 0x7f7814070e20 2024-04-25T14:03:27.993 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.992+0000 7f7812a00640 1 -- 172.21.15.155:0/3467871659 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc0a8da0 con 0x7f7814070e20 2024-04-25T14:03:27.994 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.992+0000 7f781a200640 1 -- 172.21.15.155:0/3467871659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7814070e20 msgr2=0x7f7814071230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:27.994 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.992+0000 7f781a200640 1 --2- 172.21.15.155:0/3467871659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7814070e20 0x7f7814071230 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f78140736c0 tx=0x7f77fc01ed10 comp rx=0 tx=0).stop 2024-04-25T14:03:27.994 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.992+0000 7f781a200640 1 -- 172.21.15.155:0/3467871659 shutdown_connections 2024-04-25T14:03:27.994 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.992+0000 7f781a200640 1 --2- 172.21.15.155:0/3467871659 >> v2:172.21.15.155:3300/0 conn(0x7f78140722f0 0x7f7814072700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.994 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.992+0000 7f781a200640 1 --2- 172.21.15.155:0/3467871659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7814070e20 0x7f7814071230 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.994 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.992+0000 7f781a200640 1 --2- 172.21.15.155:0/3467871659 >> v2:172.21.15.86:3300/0 conn(0x7f78140d2c80 0x7f781415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.995 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.992+0000 7f781a200640 1 -- 172.21.15.155:0/3467871659 >> 172.21.15.155:0/3467871659 conn(0x7f78140cffc0 msgr2=0x7f78140d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:27.995 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.992+0000 7f781a200640 1 -- 172.21.15.155:0/3467871659 shutdown_connections 2024-04-25T14:03:27.995 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.992+0000 7f781a200640 1 -- 172.21.15.155:0/3467871659 wait complete. 2024-04-25T14:03:27.996 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.996+0000 7f781a200640 1 Processor -- start 2024-04-25T14:03:27.996 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.996+0000 7f781a200640 1 -- start start 2024-04-25T14:03:28.000 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.996+0000 7f781a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7814070e20 0x7f78141f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:28.000 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.996+0000 7f781a200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78140722f0 0x7f78141f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:28.000 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.996+0000 7f781a200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78140d2c80 0x7f78140c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:28.000 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.996+0000 7f781a200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78140d21a0 con 0x7f7814070e20 2024-04-25T14:03:28.000 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.996+0000 7f781a200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f78140d1ec0 con 0x7f78140d2c80 2024-04-25T14:03:28.000 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.996+0000 7f781a200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f78140d2480 con 0x7f78140722f0 2024-04-25T14:03:28.000 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.996+0000 7f780aa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78140722f0 0x7f78141f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:28.000 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.996+0000 7f780aa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78140722f0 0x7f78141f3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:53970/0 (socket says 172.21.15.155:53970) 2024-04-25T14:03:28.000 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.996+0000 7f780aa00640 1 -- 172.21.15.155:0/194497256 learned_addr learned my addr 172.21.15.155:0/194497256 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T14:03:28.000 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.996+0000 7f7813e00640 1 --2- 172.21.15.155:0/194497256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78140d2c80 0x7f78140c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:28.001 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.996+0000 7f7813400640 1 --2- 172.21.15.155:0/194497256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7814070e20 0x7f78141f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:28.001 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.996+0000 7f780aa00640 1 -- 172.21.15.155:0/194497256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78140d2c80 msgr2=0x7f78140c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:28.001 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.996+0000 7f780aa00640 1 --2- 172.21.15.155:0/194497256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78140d2c80 0x7f78140c91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:28.001 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.996+0000 7f780aa00640 1 -- 172.21.15.155:0/194497256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7814070e20 msgr2=0x7f78141f4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:28.001 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.996+0000 7f780aa00640 1 --2- 172.21.15.155:0/194497256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7814070e20 0x7f78141f4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:28.001 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.996+0000 7f780aa00640 1 -- 172.21.15.155:0/194497256 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78141f5310 con 0x7f78140722f0 2024-04-25T14:03:28.001 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.996+0000 7f7813e00640 1 --2- 172.21.15.155:0/194497256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78140d2c80 0x7f78140c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:28.001 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.996+0000 7f780aa00640 1 --2- 172.21.15.155:0/194497256 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78140722f0 0x7f78141f3bf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f77fc01d3c0 tx=0x7f77fc01bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:28.001 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.996+0000 7f780be00640 1 -- 172.21.15.155:0/194497256 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc0b5040 con 0x7f78140722f0 2024-04-25T14:03:28.001 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.996+0000 7f780be00640 1 -- 172.21.15.155:0/194497256 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77fc004230 con 0x7f78140722f0 2024-04-25T14:03:28.001 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.996+0000 7f781a200640 1 -- 172.21.15.155:0/194497256 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78140ab770 con 0x7f78140722f0 2024-04-25T14:03:28.001 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.996+0000 7f780be00640 1 -- 172.21.15.155:0/194497256 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc003620 con 0x7f78140722f0 2024-04-25T14:03:28.001 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:27.996+0000 7f781a200640 1 -- 172.21.15.155:0/194497256 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f78140abd30 con 0x7f78140722f0 2024-04-25T14:03:28.001 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.000+0000 7f780be00640 1 -- 172.21.15.155:0/194497256 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f77fc0c0060 con 0x7f78140722f0 2024-04-25T14:03:28.002 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.000+0000 7f780be00640 1 -- 172.21.15.155:0/194497256 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f77fc0c2450 con 0x7f78140722f0 2024-04-25T14:03:28.002 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.000+0000 7f781a200640 1 -- 172.21.15.155:0/194497256 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78140720e0 con 0x7f78140722f0 2024-04-25T14:03:28.003 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.000+0000 7f780be00640 1 -- 172.21.15.155:0/194497256 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f78141f5310 con 0x7f78140722f0 2024-04-25T14:03:28.028 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:03:28.028+0000 7fcdc204d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:03:28.069 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.068+0000 7f781a200640 1 -- 172.21.15.155:0/194497256 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b", "id": 6} v 0) -- 0x7f7814076bf0 con 0x7f78140722f0 2024-04-25T14:03:28.081 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.080+0000 7f780be00640 1 -- 172.21.15.155:0/194497256 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f77fc018870 con 0x7f78140722f0 2024-04-25T14:03:28.081 INFO:teuthology.orchestra.run.smithi155.stdout:6 2024-04-25T14:03:28.082 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.080+0000 7f781a200640 1 -- 172.21.15.155:0/194497256 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78140722f0 msgr2=0x7f78141f3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:28.082 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.080+0000 7f781a200640 1 --2- 172.21.15.155:0/194497256 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78140722f0 0x7f78141f3bf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f77fc01d3c0 tx=0x7f77fc01bab0 comp rx=0 tx=0).stop 2024-04-25T14:03:28.082 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.080+0000 7f781a200640 1 -- 172.21.15.155:0/194497256 shutdown_connections 2024-04-25T14:03:28.082 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.080+0000 7f781a200640 1 --2- 172.21.15.155:0/194497256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78140d2c80 0x7f78140c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:28.082 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.080+0000 7f781a200640 1 --2- 172.21.15.155:0/194497256 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78140722f0 0x7f78141f3bf0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:28.083 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.080+0000 7f781a200640 1 --2- 172.21.15.155:0/194497256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7814070e20 0x7f78141f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:28.083 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.080+0000 7f781a200640 1 -- 172.21.15.155:0/194497256 >> 172.21.15.155:0/194497256 conn(0x7f78140cffc0 msgr2=0x7f78140b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:28.083 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.080+0000 7f781a200640 1 -- 172.21.15.155:0/194497256 shutdown_connections 2024-04-25T14:03:28.083 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.080+0000 7f781a200640 1 -- 172.21.15.155:0/194497256 wait complete. 2024-04-25T14:03:28.101 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 4a769485-72f3-4e7d-a04a-f8f1ae51d98f 7 2024-04-25T14:03:28.248 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.248+0000 7f7ee3600640 1 Processor -- start 2024-04-25T14:03:28.248 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.248+0000 7f7ee3600640 1 -- start start 2024-04-25T14:03:28.250 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.248+0000 7f7ee3600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7edc071530 0x7f7edc15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:28.250 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.248+0000 7f7ee3600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7edc1616e0 0x7f7edc161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:28.250 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.248+0000 7f7ee3600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7edc0cd8d0 con 0x7f7edc0d2c80 2024-04-25T14:03:28.250 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.248+0000 7f7ee3600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7edc0ce720 con 0x7f7edc1616e0 2024-04-25T14:03:28.250 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.248+0000 7f7ee3600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7edc0ce970 con 0x7f7edc071530 2024-04-25T14:03:28.250 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.248+0000 7f7edbe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7edc071530 0x7f7edc15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:28.250 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.248+0000 7f7edbe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7edc071530 0x7f7edc15dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:53980/0 (socket says 172.21.15.155:53980) 2024-04-25T14:03:28.250 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.248+0000 7f7edbe00640 1 -- 172.21.15.155:0/3113724121 learned_addr learned my addr 172.21.15.155:0/3113724121 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T14:03:28.251 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.248+0000 7f7ee1600640 1 --2- 172.21.15.155:0/3113724121 >> v2:172.21.15.86:3300/0 conn(0x7f7edc1616e0 0x7f7edc161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:28.251 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.248+0000 7f7edbe00640 1 -- 172.21.15.155:0/3113724121 >> v2:172.21.15.86:3300/0 conn(0x7f7edc1616e0 msgr2=0x7f7edc161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:28.251 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.248+0000 7f7edbe00640 1 --2- 172.21.15.155:0/3113724121 >> v2:172.21.15.86:3300/0 conn(0x7f7edc1616e0 0x7f7edc161af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:28.251 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.248+0000 7f7edbe00640 1 -- 172.21.15.155:0/3113724121 >> v1:172.21.15.86:6789/0 conn(0x7f7edc0d2c80 legacy=0x7f7edc070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:28.251 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.248+0000 7f7edbe00640 1 -- 172.21.15.155:0/3113724121 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7edc0d1be0 con 0x7f7edc071530 2024-04-25T14:03:28.253 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.252+0000 7f7edbe00640 1 --2- 172.21.15.155:0/3113724121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7edc071530 0x7f7edc15dc90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009860 tx=0x7f7ec801ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67d27f1939e43af7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:28.254 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.252+0000 7f7edb400640 1 -- 172.21.15.155:0/3113724121 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec80b5070 con 0x7f7edc071530 2024-04-25T14:03:28.254 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.252+0000 7f7edb400640 1 -- 172.21.15.155:0/3113724121 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec80a89b0 con 0x7f7edc071530 2024-04-25T14:03:28.254 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.252+0000 7f7edb400640 1 -- 172.21.15.155:0/3113724121 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec80a8d80 con 0x7f7edc071530 2024-04-25T14:03:28.255 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.252+0000 7f7ee3600640 1 -- 172.21.15.155:0/3113724121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7edc071530 msgr2=0x7f7edc15dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:28.255 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.252+0000 7f7ee3600640 1 --2- 172.21.15.155:0/3113724121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7edc071530 0x7f7edc15dc90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009860 tx=0x7f7ec801ecf0 comp rx=0 tx=0).stop 2024-04-25T14:03:28.255 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.252+0000 7f7ee3600640 1 -- 172.21.15.155:0/3113724121 shutdown_connections 2024-04-25T14:03:28.255 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.252+0000 7f7ee3600640 1 --2- 172.21.15.155:0/3113724121 >> v2:172.21.15.86:3300/0 conn(0x7f7edc1616e0 0x7f7edc161af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:28.255 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.252+0000 7f7ee3600640 1 --2- 172.21.15.155:0/3113724121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7edc071530 0x7f7edc15dc90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:28.255 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.252+0000 7f7ee3600640 1 -- 172.21.15.155:0/3113724121 >> 172.21.15.155:0/3113724121 conn(0x7f7edc0cffc0 msgr2=0x7f7edc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:28.255 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.252+0000 7f7ee3600640 1 -- 172.21.15.155:0/3113724121 shutdown_connections 2024-04-25T14:03:28.255 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.252+0000 7f7ee3600640 1 -- 172.21.15.155:0/3113724121 wait complete. 2024-04-25T14:03:28.256 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.256+0000 7f7ee3600640 1 Processor -- start 2024-04-25T14:03:28.257 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.256+0000 7f7ee3600640 1 -- start start 2024-04-25T14:03:28.258 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.256+0000 7f7ee3600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7edc0d2c80 0x7f7edc06e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:28.258 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.256+0000 7f7ee3600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7edc1616e0 0x7f7edc0b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:28.258 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.256+0000 7f7ee3600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7edc15dc90 0x7f7edc159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:28.259 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.256+0000 7f7ee3600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc0d2480 con 0x7f7edc15dc90 2024-04-25T14:03:28.259 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.256+0000 7f7ee3600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc0d21a0 con 0x7f7edc1616e0 2024-04-25T14:03:28.259 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.256+0000 7f7ee3600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7edc0d29a0 con 0x7f7edc0d2c80 2024-04-25T14:03:28.259 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.256+0000 7f7edbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7edc1616e0 0x7f7edc0b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:28.259 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.256+0000 7f7ee0c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7edc0d2c80 0x7f7edc06e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:28.259 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.256+0000 7f7edbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7edc1616e0 0x7f7edc0b1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:50140/0 (socket says 172.21.15.155:50140) 2024-04-25T14:03:28.259 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.256+0000 7f7edbe00640 1 -- 172.21.15.155:0/206538783 learned_addr learned my addr 172.21.15.155:0/206538783 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-25T14:03:28.260 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.256+0000 7f7ee0c00640 1 -- 172.21.15.155:0/206538783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7edc1616e0 msgr2=0x7f7edc0b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:28.260 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.256+0000 7f7ee0c00640 1 --2- 172.21.15.155:0/206538783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7edc1616e0 0x7f7edc0b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:28.260 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.256+0000 7f7ee0c00640 1 -- 172.21.15.155:0/206538783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7edc15dc90 msgr2=0x7f7edc159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:28.261 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.256+0000 7f7ee0c00640 1 --2- 172.21.15.155:0/206538783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7edc15dc90 0x7f7edc159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:28.261 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.256+0000 7f7ee0c00640 1 -- 172.21.15.155:0/206538783 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7edc0c7fb0 con 0x7f7edc0d2c80 2024-04-25T14:03:28.261 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.256+0000 7f7edbe00640 1 --2- 172.21.15.155:0/206538783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7edc1616e0 0x7f7edc0b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:28.261 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.256+0000 7f7ee0c00640 1 --2- 172.21.15.155:0/206538783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7edc0d2c80 0x7f7edc06e390 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000b440 tx=0x7f7ed000b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:28.261 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.256+0000 7f7ed8c00640 1 -- 172.21.15.155:0/206538783 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0011830 con 0x7f7edc0d2c80 2024-04-25T14:03:28.261 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.256+0000 7f7ed8c00640 1 -- 172.21.15.155:0/206538783 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ed0016070 con 0x7f7edc0d2c80 2024-04-25T14:03:28.261 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.256+0000 7f7ee3600640 1 -- 172.21.15.155:0/206538783 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7edc1607b0 con 0x7f7edc0d2c80 2024-04-25T14:03:28.261 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.256+0000 7f7ed8c00640 1 -- 172.21.15.155:0/206538783 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0011c10 con 0x7f7edc0d2c80 2024-04-25T14:03:28.261 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.256+0000 7f7ee3600640 1 -- 172.21.15.155:0/206538783 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7edc15e710 con 0x7f7edc0d2c80 2024-04-25T14:03:28.261 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.260+0000 7f7ed8c00640 1 -- 172.21.15.155:0/206538783 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7ed0004020 con 0x7f7edc0d2c80 2024-04-25T14:03:28.261 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.260+0000 7f7ed8c00640 1 -- 172.21.15.155:0/206538783 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f7ed0011db0 con 0x7f7edc0d2c80 2024-04-25T14:03:28.261 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.260+0000 7f7ee3600640 1 -- 172.21.15.155:0/206538783 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7edc0720e0 con 0x7f7edc0d2c80 2024-04-25T14:03:28.263 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.260+0000 7f7ed8c00640 1 -- 172.21.15.155:0/206538783 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7ed0020020 con 0x7f7edc0d2c80 2024-04-25T14:03:28.338 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.336+0000 7f7ee3600640 1 -- 172.21.15.155:0/206538783 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4a769485-72f3-4e7d-a04a-f8f1ae51d98f", "id": 7} v 0) -- 0x7f7edc076bf0 con 0x7f7edc0d2c80 2024-04-25T14:03:28.344 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.344+0000 7f7ed8c00640 1 -- 172.21.15.155:0/206538783 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4a769485-72f3-4e7d-a04a-f8f1ae51d98f", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f7ed00043e0 con 0x7f7edc0d2c80 2024-04-25T14:03:28.345 INFO:teuthology.orchestra.run.smithi155.stdout:7 2024-04-25T14:03:28.346 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.344+0000 7f7ee3600640 1 -- 172.21.15.155:0/206538783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7edc0d2c80 msgr2=0x7f7edc06e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:28.346 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.344+0000 7f7ee3600640 1 --2- 172.21.15.155:0/206538783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7edc0d2c80 0x7f7edc06e390 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000b440 tx=0x7f7ed000b900 comp rx=0 tx=0).stop 2024-04-25T14:03:28.346 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.344+0000 7f7ee3600640 1 -- 172.21.15.155:0/206538783 shutdown_connections 2024-04-25T14:03:28.346 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.344+0000 7f7ee3600640 1 --2- 172.21.15.155:0/206538783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7edc15dc90 0x7f7edc159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:28.346 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.344+0000 7f7ee3600640 1 --2- 172.21.15.155:0/206538783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7edc1616e0 0x7f7edc0b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:28.346 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.344+0000 7f7ee3600640 1 --2- 172.21.15.155:0/206538783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7edc0d2c80 0x7f7edc06e390 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:28.346 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.344+0000 7f7ee3600640 1 -- 172.21.15.155:0/206538783 >> 172.21.15.155:0/206538783 conn(0x7f7edc0cffc0 msgr2=0x7f7edc15ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:28.346 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.344+0000 7f7ee3600640 1 -- 172.21.15.155:0/206538783 shutdown_connections 2024-04-25T14:03:28.346 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:03:28.344+0000 7f7ee3600640 1 -- 172.21.15.155:0/206538783 wait complete. 2024-04-25T14:03:28.372 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T14:03:28.372 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T14:03:28.375 INFO:tasks.ceph.osd.0:Started 2024-04-25T14:03:28.375 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T14:03:28.375 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T14:03:28.378 INFO:tasks.ceph.osd.1:Started 2024-04-25T14:03:28.378 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T14:03:28.378 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T14:03:28.381 INFO:tasks.ceph.osd.2:Started 2024-04-25T14:03:28.381 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T14:03:28.381 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T14:03:28.384 INFO:tasks.ceph.osd.3:Started 2024-04-25T14:03:28.384 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T14:03:28.385 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T14:03:28.392 INFO:tasks.ceph.osd.4:Started 2024-04-25T14:03:28.392 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T14:03:28.392 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T14:03:28.400 INFO:tasks.ceph.osd.5:Started 2024-04-25T14:03:28.400 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T14:03:28.400 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T14:03:28.404 INFO:tasks.ceph.osd.6:Started 2024-04-25T14:03:28.404 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T14:03:28.404 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T14:03:28.414 INFO:tasks.ceph.osd.7:Started 2024-04-25T14:03:28.415 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:03:28.589 INFO:tasks.ceph.mgr.y.smithi086.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:03:28.589 INFO:tasks.ceph.mgr.y.smithi086.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:03:28.589 INFO:tasks.ceph.mgr.y.smithi086.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:03:28.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.595+0000 7f6289200640 1 Processor -- start 2024-04-25T14:03:28.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.595+0000 7f6289200640 1 -- start start 2024-04-25T14:03:28.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.595+0000 7f6289200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f62840cfca0 0x7f628406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:28.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.595+0000 7f6289200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f62840730b0 0x7f62840734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:28.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.595+0000 7f6289200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6284072bd0 con 0x7f62840d16e0 2024-04-25T14:03:28.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.595+0000 7f6289200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6284070470 con 0x7f62840730b0 2024-04-25T14:03:28.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.595+0000 7f6289200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f628406f930 con 0x7f62840cfca0 2024-04-25T14:03:28.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.595+0000 7f6282800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f62840730b0 0x7f62840734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:28.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.595+0000 7f6281400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f62840d16e0 0x7f62840d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:43986/0 (socket says 172.21.15.86:43986) 2024-04-25T14:03:28.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.595+0000 7f6282800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f62840730b0 0x7f62840734c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48952/0 (socket says 172.21.15.86:48952) 2024-04-25T14:03:28.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.595+0000 7f6281400640 1 -- 172.21.15.86:0/2181761957 learned_addr learned my addr 172.21.15.86:0/2181761957 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:03:28.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.599+0000 7f6281e00640 1 --2- 172.21.15.86:0/2181761957 >> v2:172.21.15.155:3300/0 conn(0x7f62840cfca0 0x7f628406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:28.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.599+0000 7f6282800640 1 -- 172.21.15.86:0/2181761957 >> v2:172.21.15.155:3300/0 conn(0x7f62840cfca0 msgr2=0x7f628406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:28.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.599+0000 7f6282800640 1 --2- 172.21.15.86:0/2181761957 >> v2:172.21.15.155:3300/0 conn(0x7f62840cfca0 0x7f628406f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:28.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.599+0000 7f6282800640 1 -- 172.21.15.86:0/2181761957 >> v1:172.21.15.86:6789/0 conn(0x7f62840d16e0 legacy=0x7f62840d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:28.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.599+0000 7f6282800640 1 -- 172.21.15.86:0/2181761957 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628406fbd0 con 0x7f62840730b0 2024-04-25T14:03:28.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.599+0000 7f6282800640 1 --2- 172.21.15.86:0/2181761957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62840730b0 0x7f62840734c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f62840d22a0 tx=0x7f627c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48c6f454d8c251a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:28.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.599+0000 7f6280a00640 1 -- 172.21.15.86:0/2181761957 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627c0b2070 con 0x7f62840730b0 2024-04-25T14:03:28.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.599+0000 7f6280a00640 1 -- 172.21.15.86:0/2181761957 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f627c0a5900 con 0x7f62840730b0 2024-04-25T14:03:28.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.599+0000 7f6280a00640 1 -- 172.21.15.86:0/2181761957 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627c0a5cd0 con 0x7f62840730b0 2024-04-25T14:03:28.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.603+0000 7f6289200640 1 -- 172.21.15.86:0/2181761957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62840730b0 msgr2=0x7f62840734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:28.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.603+0000 7f6289200640 1 --2- 172.21.15.86:0/2181761957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62840730b0 0x7f62840734c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f62840d22a0 tx=0x7f627c01c930 comp rx=0 tx=0).stop 2024-04-25T14:03:28.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.603+0000 7f6289200640 1 -- 172.21.15.86:0/2181761957 shutdown_connections 2024-04-25T14:03:28.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.603+0000 7f6289200640 1 --2- 172.21.15.86:0/2181761957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62840730b0 0x7f62840734c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:28.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.603+0000 7f6289200640 1 --2- 172.21.15.86:0/2181761957 >> v2:172.21.15.155:3300/0 conn(0x7f62840cfca0 0x7f628406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:28.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.603+0000 7f6289200640 1 -- 172.21.15.86:0/2181761957 >> 172.21.15.86:0/2181761957 conn(0x7f62840b4650 msgr2=0x7f62840b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:28.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.603+0000 7f6289200640 1 -- 172.21.15.86:0/2181761957 shutdown_connections 2024-04-25T14:03:28.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.603+0000 7f6289200640 1 -- 172.21.15.86:0/2181761957 wait complete. 2024-04-25T14:03:28.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.603+0000 7f6289200640 1 Processor -- start 2024-04-25T14:03:28.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.603+0000 7f6289200640 1 -- start start 2024-04-25T14:03:28.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.607+0000 7f6289200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62840730b0 0x7f628408cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:28.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.607+0000 7f6289200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62840cfca0 0x7f6284154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:28.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.607+0000 7f6289200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62840d16e0 0x7f62840caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:28.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.607+0000 7f6289200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6284169a70 con 0x7f62840cfca0 2024-04-25T14:03:28.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.607+0000 7f6289200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f62841698f0 con 0x7f62840730b0 2024-04-25T14:03:28.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.607+0000 7f6289200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6284169bf0 con 0x7f62840d16e0 2024-04-25T14:03:28.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.607+0000 7f6281400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62840cfca0 0x7f6284154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:28.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.607+0000 7f6281400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62840cfca0 0x7f6284154340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48956/0 (socket says 172.21.15.86:48956) 2024-04-25T14:03:28.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.607+0000 7f6281400640 1 -- 172.21.15.86:0/4162294251 learned_addr learned my addr 172.21.15.86:0/4162294251 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:28.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.607+0000 7f6281e00640 1 --2- 172.21.15.86:0/4162294251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62840730b0 0x7f628408cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:28.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.607+0000 7f6282800640 1 --2- 172.21.15.86:0/4162294251 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62840d16e0 0x7f62840caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:28.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.607+0000 7f6281400640 1 -- 172.21.15.86:0/4162294251 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62840d16e0 msgr2=0x7f62840caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:28.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.607+0000 7f6281400640 1 --2- 172.21.15.86:0/4162294251 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62840d16e0 0x7f62840caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:28.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.607+0000 7f6281400640 1 -- 172.21.15.86:0/4162294251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62840730b0 msgr2=0x7f628408cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:28.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.607+0000 7f6281400640 1 --2- 172.21.15.86:0/4162294251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62840730b0 0x7f628408cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:28.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.607+0000 7f6281400640 1 -- 172.21.15.86:0/4162294251 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628409ae70 con 0x7f62840cfca0 2024-04-25T14:03:28.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.607+0000 7f6281400640 1 --2- 172.21.15.86:0/4162294251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62840cfca0 0x7f6284154340 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f627000c340 tx=0x7f627000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:28.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.607+0000 7f627a000640 1 -- 172.21.15.86:0/4162294251 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6270017070 con 0x7f62840cfca0 2024-04-25T14:03:28.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.607+0000 7f627a000640 1 -- 172.21.15.86:0/4162294251 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6270005cf0 con 0x7f62840cfca0 2024-04-25T14:03:28.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.607+0000 7f627a000640 1 -- 172.21.15.86:0/4162294251 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62700072f0 con 0x7f62840cfca0 2024-04-25T14:03:28.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.607+0000 7f6289200640 1 -- 172.21.15.86:0/4162294251 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62840c9a30 con 0x7f62840cfca0 2024-04-25T14:03:28.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.607+0000 7f6289200640 1 -- 172.21.15.86:0/4162294251 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62840c9ff0 con 0x7f62840cfca0 2024-04-25T14:03:28.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.607+0000 7f627a000640 1 -- 172.21.15.86:0/4162294251 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f627001a070 con 0x7f62840cfca0 2024-04-25T14:03:28.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.607+0000 7f627a000640 1 -- 172.21.15.86:0/4162294251 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f6270007490 con 0x7f62840cfca0 2024-04-25T14:03:28.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.607+0000 7f6289200640 1 -- 172.21.15.86:0/4162294251 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f628409f4b0 con 0x7f62840cfca0 2024-04-25T14:03:28.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.611+0000 7f627a000640 1 -- 172.21.15.86:0/4162294251 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6270021020 con 0x7f62840cfca0 2024-04-25T14:03:28.651 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:03:28.647+0000 7f3d0228d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:03:28.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.667+0000 7f6289200640 1 -- 172.21.15.86:0/4162294251 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6284069d00 con 0x7f62840cfca0 2024-04-25T14:03:28.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.671+0000 7f627a000640 1 -- 172.21.15.86:0/4162294251 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f627001c440 con 0x7f62840cfca0 2024-04-25T14:03:28.672 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:03:28.673 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":10,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:03:28.341654+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:03:28.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.671+0000 7f6289200640 1 -- 172.21.15.86:0/4162294251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62840cfca0 msgr2=0x7f6284154340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:28.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.671+0000 7f6289200640 1 --2- 172.21.15.86:0/4162294251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62840cfca0 0x7f6284154340 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f627000c340 tx=0x7f627000c800 comp rx=0 tx=0).stop 2024-04-25T14:03:28.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.671+0000 7f6289200640 1 -- 172.21.15.86:0/4162294251 shutdown_connections 2024-04-25T14:03:28.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.671+0000 7f6289200640 1 --2- 172.21.15.86:0/4162294251 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62840d16e0 0x7f62840caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:28.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.671+0000 7f6289200640 1 --2- 172.21.15.86:0/4162294251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62840cfca0 0x7f6284154340 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:28.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.671+0000 7f6289200640 1 --2- 172.21.15.86:0/4162294251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62840730b0 0x7f628408cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:28.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.671+0000 7f6289200640 1 -- 172.21.15.86:0/4162294251 >> 172.21.15.86:0/4162294251 conn(0x7f62840b4650 msgr2=0x7f6284156290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:28.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.671+0000 7f6289200640 1 -- 172.21.15.86:0/4162294251 shutdown_connections 2024-04-25T14:03:28.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:28.671+0000 7f6289200640 1 -- 172.21.15.86:0/4162294251 wait complete. 2024-04-25T14:03:28.693 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T14:03:28.693 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T14:03:28.722 INFO:tasks.ceph.mgr.x.smithi155.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:03:28.722 INFO:tasks.ceph.mgr.x.smithi155.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:03:28.722 INFO:tasks.ceph.mgr.x.smithi155.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:03:28.774 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:03:28.772+0000 7fcdc204d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:03:28.803 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:03:28.799+0000 7f3d0228d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:03:28.922 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:03:28.920+0000 7fcdc204d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:03:28.980 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:03:28.979+0000 7f3d0228d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:03:29.053 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:03:29.051+0000 7f3d0228d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:03:29.073 INFO:tasks.ceph.osd.3.smithi086.stderr:2024-04-25T14:03:29.071+0000 7fcf91c4a7c0 -1 Falling back to public interface 2024-04-25T14:03:29.074 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-04-25T14:03:29.072+0000 7fb678a577c0 -1 Falling back to public interface 2024-04-25T14:03:29.074 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-04-25T14:03:29.072+0000 7fe1e9da77c0 -1 Falling back to public interface 2024-04-25T14:03:29.074 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-04-25T14:03:29.072+0000 7fe1e8800640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-04-25T14:03:29.075 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-04-25T14:03:29.072+0000 7fe1e8800640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6808/0: (98) Address already in use 2024-04-25T14:03:29.078 INFO:tasks.ceph.osd.1.smithi086.stderr:2024-04-25T14:03:29.075+0000 7f610a22f7c0 -1 Falling back to public interface 2024-04-25T14:03:29.082 INFO:tasks.ceph.osd.2.smithi086.stderr:2024-04-25T14:03:29.079+0000 7f9a1d9867c0 -1 Falling back to public interface 2024-04-25T14:03:29.086 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-25T14:03:29.083+0000 7f421322b7c0 -1 Falling back to public interface 2024-04-25T14:03:29.091 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:03:29.088+0000 7fcdc204d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:03:29.093 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-04-25T14:03:29.092+0000 7f277f42c7c0 -1 Falling back to public interface 2024-04-25T14:03:29.093 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-04-25T14:03:29.092+0000 7f4177f2d7c0 -1 Falling back to public interface 2024-04-25T14:03:29.169 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:03:29.168+0000 7fcdc204d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:03:29.212 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:03:29.207+0000 7f3d0228d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:03:29.319 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:03:29.316+0000 7fcdc204d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:03:29.499 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T14:03:29.637 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.635+0000 7fc2ff400640 1 Processor -- start 2024-04-25T14:03:29.637 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.635+0000 7fc2ff400640 1 -- start start 2024-04-25T14:03:29.640 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.635+0000 7fc2ff400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2f8071280 0x7fc2f8071690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:29.640 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.635+0000 7fc2ff400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2f8071bd0 con 0x7fc2f8160ab0 2024-04-25T14:03:29.640 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.635+0000 7fc2ff400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2f81605d0 con 0x7fc2f815cb30 2024-04-25T14:03:29.640 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.635+0000 7fc2ff400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc2f80d0730 con 0x7fc2f8071280 2024-04-25T14:03:29.640 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.635+0000 7fc2fd400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc2f8160ab0 0x7fc2f8160ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35738/0 (socket says 172.21.15.86:35738) 2024-04-25T14:03:29.640 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.635+0000 7fc2fd400640 1 -- 172.21.15.86:0/3584657686 learned_addr learned my addr 172.21.15.86:0/3584657686 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:03:29.641 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.635+0000 7fc2f7e00640 1 --2- 172.21.15.86:0/3584657686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2f8071280 0x7fc2f8071690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:29.641 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.635+0000 7fc2f7400640 1 -- 172.21.15.86:0/3584657686 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3043584296 0 0) 0x7fc2f00014a0 con 0x7fc2f8160ab0 2024-04-25T14:03:29.641 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.635+0000 7fc2f7400640 1 -- 172.21.15.86:0/3584657686 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc2dc003530 con 0x7fc2f8160ab0 2024-04-25T14:03:29.641 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.635+0000 7fc2f7400640 1 -- 172.21.15.86:0/3584657686 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3709112456 0 0) 0x7fc2f81605d0 con 0x7fc2f815cb30 2024-04-25T14:03:29.641 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.635+0000 7fc2f7400640 1 -- 172.21.15.86:0/3584657686 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc2f00014a0 con 0x7fc2f815cb30 2024-04-25T14:03:29.641 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.635+0000 7fc2f7e00640 1 -- 172.21.15.86:0/3584657686 >> v1:172.21.15.155:6789/0 conn(0x7fc2f815cb30 legacy=0x7fc2f80d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:29.641 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.635+0000 7fc2f7e00640 1 -- 172.21.15.86:0/3584657686 >> v1:172.21.15.86:6789/0 conn(0x7fc2f8160ab0 legacy=0x7fc2f8160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:29.641 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.635+0000 7fc2f7e00640 1 -- 172.21.15.86:0/3584657686 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f80b80b0 con 0x7fc2f8071280 2024-04-25T14:03:29.643 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.639+0000 7fc2f7e00640 1 --2- 172.21.15.86:0/3584657686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2f8071280 0x7fc2f8071690 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00097f0 tx=0x7fc2e001c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c86a24363f43365 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:29.643 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.639+0000 7fc2f7400640 1 -- 172.21.15.86:0/3584657686 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e00b0070 con 0x7fc2f8071280 2024-04-25T14:03:29.643 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.639+0000 7fc2f7400640 1 -- 172.21.15.86:0/3584657686 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2e00ab600 con 0x7fc2f8071280 2024-04-25T14:03:29.643 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.639+0000 7fc2f7400640 1 -- 172.21.15.86:0/3584657686 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e00ab9b0 con 0x7fc2f8071280 2024-04-25T14:03:29.644 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.639+0000 7fc2ff400640 1 -- 172.21.15.86:0/3584657686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2f8071280 msgr2=0x7fc2f8071690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:29.644 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.639+0000 7fc2ff400640 1 --2- 172.21.15.86:0/3584657686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2f8071280 0x7fc2f8071690 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00097f0 tx=0x7fc2e001c4e0 comp rx=0 tx=0).stop 2024-04-25T14:03:29.644 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.639+0000 7fc2ff400640 1 -- 172.21.15.86:0/3584657686 shutdown_connections 2024-04-25T14:03:29.644 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.639+0000 7fc2ff400640 1 --2- 172.21.15.86:0/3584657686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2f8071280 0x7fc2f8071690 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:29.644 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.639+0000 7fc2ff400640 1 -- 172.21.15.86:0/3584657686 >> 172.21.15.86:0/3584657686 conn(0x7fc2f80ceed0 msgr2=0x7fc2f80cf2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:29.644 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.639+0000 7fc2ff400640 1 -- 172.21.15.86:0/3584657686 shutdown_connections 2024-04-25T14:03:29.644 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.643+0000 7fc2ff400640 1 -- 172.21.15.86:0/3584657686 wait complete. 2024-04-25T14:03:29.646 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.643+0000 7fc2ff400640 1 Processor -- start 2024-04-25T14:03:29.646 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.643+0000 7fc2ff400640 1 -- start start 2024-04-25T14:03:29.650 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.643+0000 7fc2ff400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2f8071280 0x7fc2f814add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:29.650 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.643+0000 7fc2ff400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f815cb30 0x7fc2f815d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:29.650 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.643+0000 7fc2ff400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2f8160ab0 0x7fc2f80cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:29.650 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.643+0000 7fc2ff400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f80d2130 con 0x7fc2f815cb30 2024-04-25T14:03:29.650 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.643+0000 7fc2ff400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f80b1e80 con 0x7fc2f8071280 2024-04-25T14:03:29.650 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.643+0000 7fc2ff400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc2f80b84b0 con 0x7fc2f8160ab0 2024-04-25T14:03:29.650 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.647+0000 7fc2f7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f815cb30 0x7fc2f815d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:29.650 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.647+0000 7fc2f7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f815cb30 0x7fc2f815d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45390/0 (socket says 172.21.15.86:45390) 2024-04-25T14:03:29.650 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.647+0000 7fc2f7e00640 1 -- 172.21.15.86:0/1963322653 learned_addr learned my addr 172.21.15.86:0/1963322653 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:29.650 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.647+0000 7fc2fca00640 1 --2- 172.21.15.86:0/1963322653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2f8071280 0x7fc2f814add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:29.650 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.647+0000 7fc2fd400640 1 --2- 172.21.15.86:0/1963322653 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2f8160ab0 0x7fc2f80cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:29.650 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.647+0000 7fc2f7e00640 1 -- 172.21.15.86:0/1963322653 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2f8160ab0 msgr2=0x7fc2f80cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:29.650 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.647+0000 7fc2f7e00640 1 --2- 172.21.15.86:0/1963322653 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2f8160ab0 0x7fc2f80cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:29.651 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.647+0000 7fc2f7e00640 1 -- 172.21.15.86:0/1963322653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2f8071280 msgr2=0x7fc2f814add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:29.651 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.647+0000 7fc2f7e00640 1 --2- 172.21.15.86:0/1963322653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2f8071280 0x7fc2f814add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:29.651 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.647+0000 7fc2f7e00640 1 -- 172.21.15.86:0/1963322653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f815de60 con 0x7fc2f815cb30 2024-04-25T14:03:29.651 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.647+0000 7fc2f7e00640 1 --2- 172.21.15.86:0/1963322653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f815cb30 0x7fc2f815d1e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00ae040 tx=0x7fc2e0019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:29.651 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.647+0000 7fc2f4c00640 1 -- 172.21.15.86:0/1963322653 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e00ba070 con 0x7fc2f815cb30 2024-04-25T14:03:29.651 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.647+0000 7fc2f4c00640 1 -- 172.21.15.86:0/1963322653 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2e00ab7b0 con 0x7fc2f815cb30 2024-04-25T14:03:29.651 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.647+0000 7fc2f4c00640 1 -- 172.21.15.86:0/1963322653 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e00b0040 con 0x7fc2f815cb30 2024-04-25T14:03:29.651 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.647+0000 7fc2ff400640 1 -- 172.21.15.86:0/1963322653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f815e140 con 0x7fc2f815cb30 2024-04-25T14:03:29.651 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.647+0000 7fc2ff400640 1 -- 172.21.15.86:0/1963322653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f815e700 con 0x7fc2f815cb30 2024-04-25T14:03:29.651 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.647+0000 7fc2f4c00640 1 -- 172.21.15.86:0/1963322653 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc2e00abb60 con 0x7fc2f815cb30 2024-04-25T14:03:29.651 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.647+0000 7fc2f4c00640 1 -- 172.21.15.86:0/1963322653 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fc2e0003df0 con 0x7fc2f815cb30 2024-04-25T14:03:29.651 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.647+0000 7fc2ff400640 1 -- 172.21.15.86:0/1963322653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2f815dbd0 con 0x7fc2f815cb30 2024-04-25T14:03:29.653 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.651+0000 7fc2f4c00640 1 -- 172.21.15.86:0/1963322653 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc2e00d5e50 con 0x7fc2f815cb30 2024-04-25T14:03:29.712 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.707+0000 7fc2ff400640 1 -- 172.21.15.86:0/1963322653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc2f8050750 con 0x7fc2f815cb30 2024-04-25T14:03:29.712 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.707+0000 7fc2f4c00640 1 -- 172.21.15.86:0/1963322653 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fc2e00c4090 con 0x7fc2f815cb30 2024-04-25T14:03:29.712 INFO:teuthology.misc.health.smithi086.stdout: 2024-04-25T14:03:29.713 INFO:teuthology.misc.health.smithi086.stdout:{"epoch":10,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:03:28.341654+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:03:29.713 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.711+0000 7fc2ff400640 1 -- 172.21.15.86:0/1963322653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f815cb30 msgr2=0x7fc2f815d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:29.713 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.711+0000 7fc2ff400640 1 --2- 172.21.15.86:0/1963322653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f815cb30 0x7fc2f815d1e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00ae040 tx=0x7fc2e0019280 comp rx=0 tx=0).stop 2024-04-25T14:03:29.713 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.711+0000 7fc2ff400640 1 -- 172.21.15.86:0/1963322653 shutdown_connections 2024-04-25T14:03:29.714 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.711+0000 7fc2ff400640 1 --2- 172.21.15.86:0/1963322653 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2f8160ab0 0x7fc2f80cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:29.714 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.711+0000 7fc2ff400640 1 --2- 172.21.15.86:0/1963322653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f815cb30 0x7fc2f815d1e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:29.714 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.711+0000 7fc2ff400640 1 --2- 172.21.15.86:0/1963322653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2f8071280 0x7fc2f814add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:29.714 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.711+0000 7fc2ff400640 1 -- 172.21.15.86:0/1963322653 >> 172.21.15.86:0/1963322653 conn(0x7fc2f80ceed0 msgr2=0x7fc2f815bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:29.714 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.711+0000 7fc2ff400640 1 -- 172.21.15.86:0/1963322653 shutdown_connections 2024-04-25T14:03:29.714 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:29.711+0000 7fc2ff400640 1 -- 172.21.15.86:0/1963322653 wait complete. 2024-04-25T14:03:29.731 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T14:03:29.916 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:03:29.911+0000 7f3d0228d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:03:30.002 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:03:29.999+0000 7f3d0228d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:03:30.015 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:03:30.012+0000 7fcdc204d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:03:30.102 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:03:30.100+0000 7fcdc204d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:03:30.205 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:03:30.203+0000 7f3d0228d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:03:30.289 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:03:30.287+0000 7f3d0228d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:03:30.304 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:03:30.304+0000 7fcdc204d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:03:30.390 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:03:30.388+0000 7fcdc204d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:03:30.618 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:03:30.615+0000 7f3d0228d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:03:30.690 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:03:30.687+0000 7f3d0228d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:03:30.723 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:03:30.720+0000 7fcdc204d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:03:30.798 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:03:30.796+0000 7fcdc204d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:03:31.713 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:03:31.711+0000 7f3ca7600640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T14:03:31.713 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:03:31.711+0000 7f3ca7600640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T14:03:32.923 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-04-25T14:03:32.920+0000 7fe1e9da77c0 -1 osd.6 0 log_to_monitors true 2024-04-25T14:03:32.959 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-04-25T14:03:32.956+0000 7f4177f2d7c0 -1 osd.5 0 log_to_monitors true 2024-04-25T14:03:33.009 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-04-25T14:03:33.008+0000 7fb678a577c0 -1 osd.4 0 log_to_monitors true 2024-04-25T14:03:33.010 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-04-25T14:03:33.008+0000 7f277f42c7c0 -1 osd.7 0 log_to_monitors true 2024-04-25T14:03:33.061 INFO:tasks.ceph.osd.3.smithi086.stderr:2024-04-25T14:03:33.058+0000 7fcf91c4a7c0 -1 osd.3 0 log_to_monitors true 2024-04-25T14:03:33.071 INFO:tasks.ceph.osd.2.smithi086.stderr:2024-04-25T14:03:33.066+0000 7f9a1d9867c0 -1 osd.2 0 log_to_monitors true 2024-04-25T14:03:33.086 INFO:tasks.ceph.osd.1.smithi086.stderr:2024-04-25T14:03:33.082+0000 7f610a22f7c0 -1 osd.1 0 log_to_monitors true 2024-04-25T14:03:33.151 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-25T14:03:33.146+0000 7f421322b7c0 -1 osd.0 0 log_to_monitors true 2024-04-25T14:03:36.538 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T14:03:36.757 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.750+0000 7f3cd8200640 1 Processor -- start 2024-04-25T14:03:36.759 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.754+0000 7f3cd8200640 1 -- start start 2024-04-25T14:03:36.761 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.758+0000 7f3cd8200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3cd00b7880 0x7f3cd015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.762 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.758+0000 7f3cd8200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd00d1090 0x7f3cd00b6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.762 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.758+0000 7f3cd4e00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f3cd00b6510 0x7f3cd00d0980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:42040/0 (socket says 172.21.15.86:42040) 2024-04-25T14:03:36.762 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.758+0000 7f3cd4e00640 1 -- 172.21.15.86:0/1646752301 learned_addr learned my addr 172.21.15.86:0/1646752301 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:03:36.763 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.758+0000 7f3cd8200640 1 -- 172.21.15.86:0/1646752301 --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cd015cf50 con 0x7f3cd00b6510 2024-04-25T14:03:36.763 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.758+0000 7f3cd8200640 1 -- 172.21.15.86:0/1646752301 --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3cd006ea20 con 0x7f3cd00b7880 2024-04-25T14:03:36.763 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.758+0000 7f3cd8200640 1 -- 172.21.15.86:0/1646752301 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3cd006ec70 con 0x7f3cd00d1090 2024-04-25T14:03:36.763 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.758+0000 7f3cd5800640 1 --2- 172.21.15.86:0/1646752301 >> v2:172.21.15.155:3300/0 conn(0x7f3cd00b7880 0x7f3cd015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.764 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.758+0000 7f3ccfe00640 1 -- 172.21.15.86:0/1646752301 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 945796066 0 0) 0x7f3cd015cf50 con 0x7f3cd00b6510 2024-04-25T14:03:36.764 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.758+0000 7f3ccfe00640 1 -- 172.21.15.86:0/1646752301 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3cb8003530 con 0x7f3cd00b6510 2024-04-25T14:03:36.764 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.758+0000 7f3cd6200640 1 --2- 172.21.15.86:0/1646752301 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd00d1090 0x7f3cd00b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.764 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.758+0000 7f3ccfe00640 1 -- 172.21.15.86:0/1646752301 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1222847374 0 0) 0x7f3cb8003530 con 0x7f3cd00b6510 2024-04-25T14:03:36.764 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.758+0000 7f3ccfe00640 1 -- 172.21.15.86:0/1646752301 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd00d1090 msgr2=0x7f3cd00b6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:36.764 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.758+0000 7f3ccfe00640 1 --2- 172.21.15.86:0/1646752301 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd00d1090 0x7f3cd00b6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.764 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.758+0000 7f3ccfe00640 1 -- 172.21.15.86:0/1646752301 >> v2:172.21.15.155:3300/0 conn(0x7f3cd00b7880 msgr2=0x7f3cd015ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:36.764 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.758+0000 7f3ccfe00640 1 --2- 172.21.15.86:0/1646752301 >> v2:172.21.15.155:3300/0 conn(0x7f3cd00b7880 0x7f3cd015ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.765 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.758+0000 7f3ccfe00640 1 -- 172.21.15.86:0/1646752301 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd00b5d10 con 0x7f3cd00b6510 2024-04-25T14:03:36.765 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.758+0000 7f3ccfe00640 1 -- 172.21.15.86:0/1646752301 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f3cbc001680 con 0x7f3cd00b6510 2024-04-25T14:03:36.765 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.758+0000 7f3ccfe00640 1 -- 172.21.15.86:0/1646752301 >> v1:172.21.15.155:6789/0 conn(0x7f3cd00b6510 legacy=0x7f3cd00d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:36.765 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.758+0000 7f3ccfe00640 1 --2- 172.21.15.86:0/1646752301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cb80039c0 0x7f3cb8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.765 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.758+0000 7f3ccfe00640 1 --2- 172.21.15.86:0/1646752301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cb8004810 0x7f3cb8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.765 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.758+0000 7f3ccfe00640 1 --2- 172.21.15.86:0/1646752301 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cb8005360 0x7f3cb800fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.765 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.758+0000 7f3ccfe00640 1 -- 172.21.15.86:0/1646752301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd006ea20 con 0x7f3cb8004810 2024-04-25T14:03:36.765 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.758+0000 7f3ccfe00640 1 -- 172.21.15.86:0/1646752301 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd006ec70 con 0x7f3cb80039c0 2024-04-25T14:03:36.765 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.758+0000 7f3ccfe00640 1 -- 172.21.15.86:0/1646752301 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3cb8010680 con 0x7f3cb8005360 2024-04-25T14:03:36.765 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.758+0000 7f3cd4e00640 1 --2- 172.21.15.86:0/1646752301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cb8004810 0x7f3cb8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.765 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.762+0000 7f3cd4e00640 1 -- 172.21.15.86:0/1646752301 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cb8005360 msgr2=0x7f3cb800fde0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T14:03:36.766 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.762+0000 7f3cd6200640 1 --2- 172.21.15.86:0/1646752301 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cb8005360 0x7f3cb800fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.766 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.762+0000 7f3cd4e00640 1 --2- 172.21.15.86:0/1646752301 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cb8005360 0x7f3cb800fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.766 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.762+0000 7f3cd4e00640 1 -- 172.21.15.86:0/1646752301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cb80039c0 msgr2=0x7f3cb8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:36.766 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.762+0000 7f3cd5800640 1 --2- 172.21.15.86:0/1646752301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cb80039c0 0x7f3cb8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.768 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.762+0000 7f3cd4e00640 1 --2- 172.21.15.86:0/1646752301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cb80039c0 0x7f3cb8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.768 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.762+0000 7f3cd4e00640 1 -- 172.21.15.86:0/1646752301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb8003530 con 0x7f3cb8004810 2024-04-25T14:03:36.768 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.762+0000 7f3cd6200640 1 --2- 172.21.15.86:0/1646752301 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cb8005360 0x7f3cb800fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:36.768 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.762+0000 7f3cd5800640 1 --2- 172.21.15.86:0/1646752301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cb80039c0 0x7f3cb8003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:36.770 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.766+0000 7f3cd4e00640 1 --2- 172.21.15.86:0/1646752301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cb8004810 0x7f3cb8004c20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc00c2c0 tx=0x7f3cbc01cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a15caba1145509 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:36.771 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.766+0000 7f3ccfe00640 1 -- 172.21.15.86:0/1646752301 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc007d60 con 0x7f3cb8004810 2024-04-25T14:03:36.771 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.766+0000 7f3ccfe00640 1 -- 172.21.15.86:0/1646752301 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cbc0a9020 con 0x7f3cb8004810 2024-04-25T14:03:36.771 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.766+0000 7f3ccfe00640 1 -- 172.21.15.86:0/1646752301 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc005b00 con 0x7f3cb8004810 2024-04-25T14:03:36.772 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.770+0000 7f3cd8200640 1 -- 172.21.15.86:0/1646752301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cb8004810 msgr2=0x7f3cb8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:36.772 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.770+0000 7f3cd8200640 1 --2- 172.21.15.86:0/1646752301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cb8004810 0x7f3cb8004c20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc00c2c0 tx=0x7f3cbc01cd40 comp rx=0 tx=0).stop 2024-04-25T14:03:36.772 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.770+0000 7f3cd6200640 1 -- 172.21.15.86:0/1646752301 reap_dead start 2024-04-25T14:03:36.773 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.770+0000 7f3cd8200640 1 -- 172.21.15.86:0/1646752301 shutdown_connections 2024-04-25T14:03:36.773 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.770+0000 7f3cd8200640 1 -- 172.21.15.86:0/1646752301 >> 172.21.15.86:0/1646752301 conn(0x7f3cd00d2610 msgr2=0x7f3cd00d2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:36.774 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.770+0000 7f3cd8200640 1 -- 172.21.15.86:0/1646752301 shutdown_connections 2024-04-25T14:03:36.777 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.770+0000 7f3cd8200640 1 -- 172.21.15.86:0/1646752301 wait complete. 2024-04-25T14:03:36.781 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.774+0000 7f3cd8200640 1 Processor -- start 2024-04-25T14:03:36.781 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.774+0000 7f3cd8200640 1 -- start start 2024-04-25T14:03:36.781 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.778+0000 7f3cd8200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cb8004810 0x7f3cd0200880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.781 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.778+0000 7f3cd8200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd014b190 0x7f3cd02072b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.781 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.778+0000 7f3cd8200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cd014dc30 0x7f3cd014e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.782 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.778+0000 7f3cd8200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd006cf00 con 0x7f3cd014dc30 2024-04-25T14:03:36.782 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.778+0000 7f3cd8200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd00b6d30 con 0x7f3cb8004810 2024-04-25T14:03:36.783 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.778+0000 7f3cd4e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd014b190 0x7f3cd02072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.783 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.778+0000 7f3cd4e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd014b190 0x7f3cd02072b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:44394/0 (socket says 172.21.15.86:44394) 2024-04-25T14:03:36.783 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.778+0000 7f3cd4e00640 1 -- 172.21.15.86:0/3614816454 learned_addr learned my addr 172.21.15.86:0/3614816454 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:36.783 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.778+0000 7f3cd8200640 1 -- 172.21.15.86:0/3614816454 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3cd006d080 con 0x7f3cd014b190 2024-04-25T14:03:36.784 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.778+0000 7f3cd5800640 1 --2- 172.21.15.86:0/3614816454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cb8004810 0x7f3cd0200880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.784 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.778+0000 7f3cd6200640 1 --2- 172.21.15.86:0/3614816454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cd014dc30 0x7f3cd014e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.785 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.782+0000 7f3cd5800640 1 -- 172.21.15.86:0/3614816454 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd014b190 msgr2=0x7f3cd02072b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:36.785 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.782+0000 7f3cd5800640 1 --2- 172.21.15.86:0/3614816454 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd014b190 0x7f3cd02072b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.785 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.782+0000 7f3cd5800640 1 -- 172.21.15.86:0/3614816454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cd014dc30 msgr2=0x7f3cd014e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:36.785 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.782+0000 7f3cd5800640 1 --2- 172.21.15.86:0/3614816454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cd014dc30 0x7f3cd014e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.785 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.782+0000 7f3cd5800640 1 -- 172.21.15.86:0/3614816454 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd02077f0 con 0x7f3cb8004810 2024-04-25T14:03:36.786 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.782+0000 7f3cd6200640 1 --2- 172.21.15.86:0/3614816454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cd014dc30 0x7f3cd014e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:36.786 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.782+0000 7f3cd5800640 1 --2- 172.21.15.86:0/3614816454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cb8004810 0x7f3cd0200880 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3cc400e810 tx=0x7f3cc400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:36.790 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.786+0000 7f3ccd600640 1 -- 172.21.15.86:0/3614816454 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc400f040 con 0x7f3cb8004810 2024-04-25T14:03:36.790 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.786+0000 7f3cd8200640 1 -- 172.21.15.86:0/3614816454 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd014c7e0 con 0x7f3cb8004810 2024-04-25T14:03:36.791 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.786+0000 7f3cd8200640 1 -- 172.21.15.86:0/3614816454 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd014a6b0 con 0x7f3cb8004810 2024-04-25T14:03:36.797 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.790+0000 7f3cd8200640 1 -- 172.21.15.86:0/3614816454 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cd006f600 con 0x7f3cb8004810 2024-04-25T14:03:36.797 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.790+0000 7f3ccd600640 1 -- 172.21.15.86:0/3614816454 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cc4004030 con 0x7f3cb8004810 2024-04-25T14:03:36.797 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.790+0000 7f3ccd600640 1 -- 172.21.15.86:0/3614816454 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4004420 con 0x7f3cb8004810 2024-04-25T14:03:36.797 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.794+0000 7f3ccd600640 1 -- 172.21.15.86:0/3614816454 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 4) ==== 93151+0+0 (secure 0 0 0) 0x7f3cc40059a0 con 0x7f3cb8004810 2024-04-25T14:03:36.799 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.794+0000 7f3ccd600640 1 --2- 172.21.15.86:0/3614816454 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3cb006f3a0 0x7f3cb00718b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.799 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.794+0000 7f3ccd600640 1 -- 172.21.15.86:0/3614816454 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f3cc408e0c0 con 0x7f3cb8004810 2024-04-25T14:03:36.800 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.794+0000 7f3cd4e00640 1 --2- 172.21.15.86:0/3614816454 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3cb006f3a0 0x7f3cb00718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.805 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.802+0000 7f3cd4e00640 1 --2- 172.21.15.86:0/3614816454 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3cb006f3a0 0x7f3cb00718b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc01cb70 tx=0x7f3cbc001a50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:36.806 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:36.802+0000 7f3ccd600640 1 -- 172.21.15.86:0/3614816454 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3cc4017020 con 0x7f3cb8004810 2024-04-25T14:03:37.095 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:37.090+0000 7f3cd8200640 1 -- 172.21.15.86:0/3614816454 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3cd01544d0 con 0x7f3cb8004810 2024-04-25T14:03:37.096 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:37.090+0000 7f3ccd600640 1 -- 172.21.15.86:0/3614816454 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f3cc4017020 con 0x7f3cb8004810 2024-04-25T14:03:37.096 INFO:teuthology.misc.health.smithi086.stdout: 2024-04-25T14:03:37.096 INFO:teuthology.misc.health.smithi086.stdout:{"epoch":12,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:03:34.775234+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:03:37.100 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:37.094+0000 7f3cd8200640 1 -- 172.21.15.86:0/3614816454 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3cb006f3a0 msgr2=0x7f3cb00718b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:37.101 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:37.094+0000 7f3cd8200640 1 --2- 172.21.15.86:0/3614816454 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3cb006f3a0 0x7f3cb00718b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc01cb70 tx=0x7f3cbc001a50 comp rx=0 tx=0).stop 2024-04-25T14:03:37.101 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:37.094+0000 7f3cd8200640 1 -- 172.21.15.86:0/3614816454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cb8004810 msgr2=0x7f3cd0200880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:37.101 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:37.094+0000 7f3cd8200640 1 --2- 172.21.15.86:0/3614816454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cb8004810 0x7f3cd0200880 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3cc400e810 tx=0x7f3cc400ecd0 comp rx=0 tx=0).stop 2024-04-25T14:03:37.103 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:37.098+0000 7f3cd8200640 1 -- 172.21.15.86:0/3614816454 shutdown_connections 2024-04-25T14:03:37.103 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:37.098+0000 7f3cd8200640 1 --2- 172.21.15.86:0/3614816454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cd014dc30 0x7f3cd014e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:37.103 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:37.098+0000 7f3cd8200640 1 --2- 172.21.15.86:0/3614816454 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3cb006f3a0 0x7f3cb00718b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:37.103 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:37.098+0000 7f3cd8200640 1 --2- 172.21.15.86:0/3614816454 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd014b190 0x7f3cd02072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:37.103 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:37.098+0000 7f3cd8200640 1 --2- 172.21.15.86:0/3614816454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cb8004810 0x7f3cd0200880 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:37.103 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:37.098+0000 7f3cd8200640 1 -- 172.21.15.86:0/3614816454 >> 172.21.15.86:0/3614816454 conn(0x7f3cd00d2610 msgr2=0x7f3cd0157360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:37.103 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:37.098+0000 7f3cd8200640 1 -- 172.21.15.86:0/3614816454 shutdown_connections 2024-04-25T14:03:37.103 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:37.098+0000 7f3cd8200640 1 -- 172.21.15.86:0/3614816454 wait complete. 2024-04-25T14:03:37.132 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T14:03:37.139 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-04-25T14:03:37.136+0000 7fb673a00640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T14:03:37.167 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-04-25T14:03:37.164+0000 7f4172e00640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T14:03:37.174 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-04-25T14:03:37.172+0000 7f2779a00640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T14:03:37.174 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-04-25T14:03:37.172+0000 7fb669a00640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:03:37.193 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-04-25T14:03:37.192+0000 7f4168e00640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:03:37.209 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-04-25T14:03:37.208+0000 7f276fa00640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:03:37.224 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-04-25T14:03:37.220+0000 7fe1e5600640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T14:03:37.257 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-04-25T14:03:37.256+0000 7fe1dac00640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:03:37.262 INFO:tasks.ceph.osd.3.smithi086.stderr:2024-04-25T14:03:37.258+0000 7fcf8d600640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T14:03:37.288 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-25T14:03:37.282+0000 7f420d600640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T14:03:37.291 INFO:tasks.ceph.osd.3.smithi086.stderr:2024-04-25T14:03:37.286+0000 7fcf82c00640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:03:37.322 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-25T14:03:37.318+0000 7f4203600640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:03:37.359 INFO:tasks.ceph.osd.2.smithi086.stderr:2024-04-25T14:03:37.354+0000 7f9a18800640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T14:03:37.363 INFO:tasks.ceph.osd.1.smithi086.stderr:2024-04-25T14:03:37.358+0000 7f6105000640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T14:03:37.385 INFO:tasks.ceph.osd.2.smithi086.stderr:2024-04-25T14:03:37.382+0000 7f9a0e800640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:03:37.393 INFO:tasks.ceph.osd.1.smithi086.stderr:2024-04-25T14:03:37.390+0000 7f60fb000640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:03:43.940 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T14:03:44.086 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.082+0000 7fb975c00640 1 Processor -- start 2024-04-25T14:03:44.087 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.082+0000 7fb975c00640 1 -- start start 2024-04-25T14:03:44.089 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.082+0000 7fb975c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb9700cf850 0x7fb97006e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:44.089 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.082+0000 7fb975c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb9700cfc60 0x7fb970163a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:44.089 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.082+0000 7fb975c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb97006ef10 0x7fb970163f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:44.089 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.082+0000 7fb975c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb97006e010 con 0x7fb9700cf850 2024-04-25T14:03:44.090 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.082+0000 7fb975c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb9700b4d20 con 0x7fb9700cfc60 2024-04-25T14:03:44.090 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.082+0000 7fb975c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb97006e260 con 0x7fb97006ef10 2024-04-25T14:03:44.090 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.082+0000 7fb96ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb9700cf850 0x7fb97006e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:44.090 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.082+0000 7fb96ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb9700cf850 0x7fb97006e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41996/0 (socket says 172.21.15.86:41996) 2024-04-25T14:03:44.090 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.082+0000 7fb96ea00640 1 -- 172.21.15.86:0/2271782401 learned_addr learned my addr 172.21.15.86:0/2271782401 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:44.090 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.082+0000 7fb96e000640 1 --2- 172.21.15.86:0/2271782401 >> v2:172.21.15.155:3300/0 conn(0x7fb9700cfc60 0x7fb970163a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:44.090 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.086+0000 7fb96f400640 1 --2- 172.21.15.86:0/2271782401 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb97006ef10 0x7fb970163f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:44.091 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.086+0000 7fb96ea00640 1 -- 172.21.15.86:0/2271782401 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb97006ef10 msgr2=0x7fb970163f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:44.091 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.086+0000 7fb96ea00640 1 --2- 172.21.15.86:0/2271782401 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb97006ef10 0x7fb970163f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.091 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.086+0000 7fb96ea00640 1 -- 172.21.15.86:0/2271782401 >> v2:172.21.15.155:3300/0 conn(0x7fb9700cfc60 msgr2=0x7fb970163a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:44.091 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.086+0000 7fb96ea00640 1 --2- 172.21.15.86:0/2271782401 >> v2:172.21.15.155:3300/0 conn(0x7fb9700cfc60 0x7fb970163a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.091 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.086+0000 7fb96ea00640 1 -- 172.21.15.86:0/2271782401 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97006e540 con 0x7fb9700cf850 2024-04-25T14:03:44.091 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.086+0000 7fb96e000640 1 --2- 172.21.15.86:0/2271782401 >> v2:172.21.15.155:3300/0 conn(0x7fb9700cfc60 0x7fb970163a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:44.093 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.086+0000 7fb96ea00640 1 --2- 172.21.15.86:0/2271782401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9700cf850 0x7fb97006e9d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb968009e50 tx=0x7fb96801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ae3588b7c10830 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:44.093 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.086+0000 7fb96d600640 1 -- 172.21.15.86:0/2271782401 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9680a5840 con 0x7fb9700cf850 2024-04-25T14:03:44.093 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.086+0000 7fb96d600640 1 -- 172.21.15.86:0/2271782401 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9680a59e0 con 0x7fb9700cf850 2024-04-25T14:03:44.093 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.086+0000 7fb96d600640 1 -- 172.21.15.86:0/2271782401 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9680a5db0 con 0x7fb9700cf850 2024-04-25T14:03:44.094 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.086+0000 7fb975c00640 1 -- 172.21.15.86:0/2271782401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9700cf850 msgr2=0x7fb97006e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:44.094 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.086+0000 7fb975c00640 1 --2- 172.21.15.86:0/2271782401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9700cf850 0x7fb97006e9d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb968009e50 tx=0x7fb96801c9c0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.094 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.090+0000 7fb975c00640 1 -- 172.21.15.86:0/2271782401 shutdown_connections 2024-04-25T14:03:44.094 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.090+0000 7fb975c00640 1 --2- 172.21.15.86:0/2271782401 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb97006ef10 0x7fb970163f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.094 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.090+0000 7fb975c00640 1 --2- 172.21.15.86:0/2271782401 >> v2:172.21.15.155:3300/0 conn(0x7fb9700cfc60 0x7fb970163a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.094 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.090+0000 7fb975c00640 1 --2- 172.21.15.86:0/2271782401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9700cf850 0x7fb97006e9d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.094 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.090+0000 7fb975c00640 1 -- 172.21.15.86:0/2271782401 >> 172.21.15.86:0/2271782401 conn(0x7fb9700d3600 msgr2=0x7fb9700d3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:44.094 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.090+0000 7fb975c00640 1 -- 172.21.15.86:0/2271782401 shutdown_connections 2024-04-25T14:03:44.095 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.090+0000 7fb975c00640 1 -- 172.21.15.86:0/2271782401 wait complete. 2024-04-25T14:03:44.095 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.090+0000 7fb975c00640 1 Processor -- start 2024-04-25T14:03:44.095 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.090+0000 7fb975c00640 1 -- start start 2024-04-25T14:03:44.099 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.090+0000 7fb975c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb97006ef10 0x7fb9700cce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:44.099 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.090+0000 7fb975c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9700cf850 0x7fb970155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:44.099 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.090+0000 7fb975c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb970156680 0x7fb97015ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:44.099 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.094+0000 7fb975c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb9700cea00 con 0x7fb97006ef10 2024-04-25T14:03:44.099 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.094+0000 7fb975c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb9700ce880 con 0x7fb970156680 2024-04-25T14:03:44.099 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.094+0000 7fb975c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb9700d1c40 con 0x7fb9700cf850 2024-04-25T14:03:44.099 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.094+0000 7fb96ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb97006ef10 0x7fb9700cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:44.100 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.094+0000 7fb96ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb97006ef10 0x7fb9700cce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42004/0 (socket says 172.21.15.86:42004) 2024-04-25T14:03:44.100 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.094+0000 7fb96ea00640 1 -- 172.21.15.86:0/3716287773 learned_addr learned my addr 172.21.15.86:0/3716287773 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:44.100 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.094+0000 7fb96e000640 1 --2- 172.21.15.86:0/3716287773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9700cf850 0x7fb970155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:44.100 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.094+0000 7fb96f400640 1 --2- 172.21.15.86:0/3716287773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb970156680 0x7fb97015ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:44.100 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.094+0000 7fb96ea00640 1 -- 172.21.15.86:0/3716287773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9700cf850 msgr2=0x7fb970155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:44.100 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.094+0000 7fb96ea00640 1 --2- 172.21.15.86:0/3716287773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9700cf850 0x7fb970155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.100 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.094+0000 7fb96ea00640 1 -- 172.21.15.86:0/3716287773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb970156680 msgr2=0x7fb97015ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:44.100 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.094+0000 7fb96ea00640 1 --2- 172.21.15.86:0/3716287773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb970156680 0x7fb97015ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.100 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.094+0000 7fb96ea00640 1 -- 172.21.15.86:0/3716287773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97015b060 con 0x7fb97006ef10 2024-04-25T14:03:44.101 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.094+0000 7fb96f400640 1 --2- 172.21.15.86:0/3716287773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb970156680 0x7fb97015ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:44.101 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.094+0000 7fb96ea00640 1 --2- 172.21.15.86:0/3716287773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb97006ef10 0x7fb9700cce10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb96801b070 tx=0x7fb968019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:44.101 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.094+0000 7fb966a00640 1 -- 172.21.15.86:0/3716287773 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9680b8470 con 0x7fb97006ef10 2024-04-25T14:03:44.101 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.094+0000 7fb966a00640 1 -- 172.21.15.86:0/3716287773 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9680b8630 con 0x7fb97006ef10 2024-04-25T14:03:44.101 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.094+0000 7fb975c00640 1 -- 172.21.15.86:0/3716287773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb97015b340 con 0x7fb97006ef10 2024-04-25T14:03:44.101 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.094+0000 7fb966a00640 1 -- 172.21.15.86:0/3716287773 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9680b8a20 con 0x7fb97006ef10 2024-04-25T14:03:44.101 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.094+0000 7fb975c00640 1 -- 172.21.15.86:0/3716287773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb97015cea0 con 0x7fb97006ef10 2024-04-25T14:03:44.101 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.094+0000 7fb975c00640 1 -- 172.21.15.86:0/3716287773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9700c7020 con 0x7fb97006ef10 2024-04-25T14:03:44.103 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.094+0000 7fb966a00640 1 -- 172.21.15.86:0/3716287773 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb9680c0020 con 0x7fb97006ef10 2024-04-25T14:03:44.103 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.098+0000 7fb966a00640 1 --2- 172.21.15.86:0/3716287773 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb94c06f400 0x7fb94c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:44.103 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.098+0000 7fb966a00640 1 -- 172.21.15.86:0/3716287773 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fb968129c10 con 0x7fb97006ef10 2024-04-25T14:03:44.103 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.098+0000 7fb96e000640 1 --2- 172.21.15.86:0/3716287773 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb94c06f400 0x7fb94c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:44.103 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.098+0000 7fb96e000640 1 --2- 172.21.15.86:0/3716287773 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb94c06f400 0x7fb94c071910 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb970159e20 tx=0x7fb958008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:44.106 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.102+0000 7fb966a00640 1 -- 172.21.15.86:0/3716287773 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb9680165e0 con 0x7fb97006ef10 2024-04-25T14:03:44.263 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.258+0000 7fb975c00640 1 -- 172.21.15.86:0/3716287773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb97016ab70 con 0x7fb97006ef10 2024-04-25T14:03:44.264 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.258+0000 7fb966a00640 1 -- 172.21.15.86:0/3716287773 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12884 (secure 0 0 0) 0x7fb9680f6db0 con 0x7fb97006ef10 2024-04-25T14:03:44.264 INFO:teuthology.misc.health.smithi086.stdout: 2024-04-25T14:03:44.264 INFO:teuthology.misc.health.smithi086.stdout:{"epoch":16,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:03:41.164591+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:03:44.266 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.262+0000 7fb975c00640 1 -- 172.21.15.86:0/3716287773 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb94c06f400 msgr2=0x7fb94c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:44.266 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.262+0000 7fb975c00640 1 --2- 172.21.15.86:0/3716287773 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb94c06f400 0x7fb94c071910 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb970159e20 tx=0x7fb958008040 comp rx=0 tx=0).stop 2024-04-25T14:03:44.266 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.262+0000 7fb975c00640 1 -- 172.21.15.86:0/3716287773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb97006ef10 msgr2=0x7fb9700cce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:44.266 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.262+0000 7fb975c00640 1 --2- 172.21.15.86:0/3716287773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb97006ef10 0x7fb9700cce10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb96801b070 tx=0x7fb968019760 comp rx=0 tx=0).stop 2024-04-25T14:03:44.266 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.262+0000 7fb975c00640 1 -- 172.21.15.86:0/3716287773 shutdown_connections 2024-04-25T14:03:44.266 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.262+0000 7fb975c00640 1 --2- 172.21.15.86:0/3716287773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb970156680 0x7fb97015ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.266 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.262+0000 7fb975c00640 1 --2- 172.21.15.86:0/3716287773 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb94c06f400 0x7fb94c071910 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.266 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.262+0000 7fb975c00640 1 --2- 172.21.15.86:0/3716287773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9700cf850 0x7fb970155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.266 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.262+0000 7fb975c00640 1 --2- 172.21.15.86:0/3716287773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb97006ef10 0x7fb9700cce10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.267 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.262+0000 7fb975c00640 1 -- 172.21.15.86:0/3716287773 >> 172.21.15.86:0/3716287773 conn(0x7fb9700d3600 msgr2=0x7fb970159310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:44.267 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.262+0000 7fb975c00640 1 -- 172.21.15.86:0/3716287773 shutdown_connections 2024-04-25T14:03:44.267 INFO:teuthology.misc.health.smithi086.stderr:2024-04-25T14:03:44.262+0000 7fb975c00640 1 -- 172.21.15.86:0/3716287773 wait complete. 2024-04-25T14:03:44.284 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T14:03:44.285 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T14:03:44.285 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T14:03:44.285 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T14:03:44.288 INFO:tasks.ceph.mds.a:Started 2024-04-25T14:03:44.288 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T14:03:44.288 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T14:03:44.291 INFO:tasks.ceph.mds.c:Started 2024-04-25T14:03:44.291 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T14:03:44.292 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T14:03:44.295 INFO:tasks.ceph.mds.b:Started 2024-04-25T14:03:44.295 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T14:03:44.295 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T14:03:44.298 INFO:tasks.ceph.mds.d:Started 2024-04-25T14:03:44.299 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T14:03:44.299 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:03:44.400 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-04-25T14:03:44.400 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-25T14:03:44.407 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2024-04-25T14:03:44.411 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2024-04-25T14:03:44.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.450+0000 7f350c200640 1 Processor -- start 2024-04-25T14:03:44.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.450+0000 7f350c200640 1 -- start start 2024-04-25T14:03:44.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.454+0000 7f350c200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f35040d2c80 0x7f350415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:44.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.454+0000 7f350c200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3504070e20 0x7f3504071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:44.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.454+0000 7f350c200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3504071770 con 0x7f35040722f0 2024-04-25T14:03:44.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.454+0000 7f350c200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f35040ce720 con 0x7f3504070e20 2024-04-25T14:03:44.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.454+0000 7f350c200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f35040cd8d0 con 0x7f35040d2c80 2024-04-25T14:03:44.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.454+0000 7f3508e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3504070e20 0x7f3504071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:44.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.454+0000 7f350a200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f35040722f0 0x7f3504072700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36072/0 (socket says 172.21.15.86:36072) 2024-04-25T14:03:44.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.454+0000 7f350a200640 1 -- 172.21.15.86:0/3358196182 learned_addr learned my addr 172.21.15.86:0/3358196182 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:03:44.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.454+0000 7f3509800640 1 --2- 172.21.15.86:0/3358196182 >> v2:172.21.15.155:3300/0 conn(0x7f35040d2c80 0x7f350415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:44.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.454+0000 7f3508e00640 1 -- 172.21.15.86:0/3358196182 >> v2:172.21.15.155:3300/0 conn(0x7f35040d2c80 msgr2=0x7f350415dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:44.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.454+0000 7f3508e00640 1 --2- 172.21.15.86:0/3358196182 >> v2:172.21.15.155:3300/0 conn(0x7f35040d2c80 0x7f350415dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.454+0000 7f3508e00640 1 -- 172.21.15.86:0/3358196182 >> v1:172.21.15.86:6789/0 conn(0x7f35040722f0 legacy=0x7f3504072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:44.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.454+0000 7f3508e00640 1 -- 172.21.15.86:0/3358196182 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35040ce970 con 0x7f3504070e20 2024-04-25T14:03:44.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.454+0000 7f3503e00640 1 -- 172.21.15.86:0/3358196182 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2923910620 0 0) 0x7f34fc0014a0 con 0x7f35040722f0 2024-04-25T14:03:44.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.458+0000 7f3508e00640 1 --2- 172.21.15.86:0/3358196182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3504070e20 0x7f3504071230 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f35040d11a0 tx=0x7f34f001ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eecd234e006a62a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:44.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.458+0000 7f3503e00640 1 -- 172.21.15.86:0/3358196182 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f00b5070 con 0x7f3504070e20 2024-04-25T14:03:44.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.458+0000 7f3503e00640 1 -- 172.21.15.86:0/3358196182 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f00a8950 con 0x7f3504070e20 2024-04-25T14:03:44.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.458+0000 7f3503e00640 1 -- 172.21.15.86:0/3358196182 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f00a8d20 con 0x7f3504070e20 2024-04-25T14:03:44.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.458+0000 7f350c200640 1 -- 172.21.15.86:0/3358196182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3504070e20 msgr2=0x7f3504071230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:44.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.458+0000 7f350c200640 1 --2- 172.21.15.86:0/3358196182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3504070e20 0x7f3504071230 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f35040d11a0 tx=0x7f34f001ec50 comp rx=0 tx=0).stop 2024-04-25T14:03:44.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.458+0000 7f350c200640 1 -- 172.21.15.86:0/3358196182 shutdown_connections 2024-04-25T14:03:44.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.458+0000 7f350c200640 1 --2- 172.21.15.86:0/3358196182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3504070e20 0x7f3504071230 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.458+0000 7f350c200640 1 --2- 172.21.15.86:0/3358196182 >> v2:172.21.15.155:3300/0 conn(0x7f35040d2c80 0x7f350415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.458+0000 7f350c200640 1 -- 172.21.15.86:0/3358196182 >> 172.21.15.86:0/3358196182 conn(0x7f35040cffc0 msgr2=0x7f35040d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:44.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.458+0000 7f350c200640 1 -- 172.21.15.86:0/3358196182 shutdown_connections 2024-04-25T14:03:44.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.458+0000 7f350c200640 1 -- 172.21.15.86:0/3358196182 wait complete. 2024-04-25T14:03:44.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.462+0000 7f350c200640 1 Processor -- start 2024-04-25T14:03:44.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.462+0000 7f350c200640 1 -- start start 2024-04-25T14:03:44.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.462+0000 7f350c200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3504070e20 0x7f35041506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:44.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.462+0000 7f350c200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35040722f0 0x7f35041e6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:44.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.462+0000 7f350c200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35040d2c80 0x7f35041e7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:44.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.462+0000 7f350c200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f35040d21a0 con 0x7f35040722f0 2024-04-25T14:03:44.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.462+0000 7f350c200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f35040d1ec0 con 0x7f35040d2c80 2024-04-25T14:03:44.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.462+0000 7f350c200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f35040d2480 con 0x7f3504070e20 2024-04-25T14:03:44.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.462+0000 7f3508e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35040722f0 0x7f35041e6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:44.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.462+0000 7f3508e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35040722f0 0x7f35041e6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42050/0 (socket says 172.21.15.86:42050) 2024-04-25T14:03:44.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.462+0000 7f3508e00640 1 -- 172.21.15.86:0/947395242 learned_addr learned my addr 172.21.15.86:0/947395242 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:44.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.462+0000 7f3509800640 1 --2- 172.21.15.86:0/947395242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3504070e20 0x7f35041506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:44.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.462+0000 7f350a200640 1 --2- 172.21.15.86:0/947395242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35040d2c80 0x7f35041e7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:44.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.462+0000 7f3508e00640 1 -- 172.21.15.86:0/947395242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3504070e20 msgr2=0x7f35041506e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:44.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.462+0000 7f3508e00640 1 --2- 172.21.15.86:0/947395242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3504070e20 0x7f35041506e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.462+0000 7f3508e00640 1 -- 172.21.15.86:0/947395242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35040d2c80 msgr2=0x7f35041e7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:44.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.462+0000 7f3508e00640 1 --2- 172.21.15.86:0/947395242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35040d2c80 0x7f35041e7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.462+0000 7f3508e00640 1 -- 172.21.15.86:0/947395242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350415abf0 con 0x7f35040722f0 2024-04-25T14:03:44.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.462+0000 7f3508e00640 1 --2- 172.21.15.86:0/947395242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35040722f0 0x7f35041e6dd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f34f00b3040 tx=0x7f34f001b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:44.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.462+0000 7f3501600640 1 -- 172.21.15.86:0/947395242 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f00b5040 con 0x7f35040722f0 2024-04-25T14:03:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.462+0000 7f3501600640 1 -- 172.21.15.86:0/947395242 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f00059a0 con 0x7f35040722f0 2024-04-25T14:03:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.462+0000 7f350c200640 1 -- 172.21.15.86:0/947395242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f350415b120 con 0x7f35040722f0 2024-04-25T14:03:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.462+0000 7f3501600640 1 -- 172.21.15.86:0/947395242 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f0005d70 con 0x7f35040722f0 2024-04-25T14:03:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.462+0000 7f350c200640 1 -- 172.21.15.86:0/947395242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35041e65d0 con 0x7f35040722f0 2024-04-25T14:03:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.462+0000 7f350c200640 1 -- 172.21.15.86:0/947395242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35040720e0 con 0x7f35040722f0 2024-04-25T14:03:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.466+0000 7f3501600640 1 -- 172.21.15.86:0/947395242 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f34f00c1020 con 0x7f35040722f0 2024-04-25T14:03:44.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.466+0000 7f3501600640 1 --2- 172.21.15.86:0/947395242 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f34e406f400 0x7f34e4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:44.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.466+0000 7f3501600640 1 -- 172.21.15.86:0/947395242 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f34f012bcf0 con 0x7f35040722f0 2024-04-25T14:03:44.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.466+0000 7f3509800640 1 --2- 172.21.15.86:0/947395242 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f34e406f400 0x7f34e4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:44.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.466+0000 7f3509800640 1 --2- 172.21.15.86:0/947395242 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f34e406f400 0x7f34e4071910 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f34f80079c0 tx=0x7f34f8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.470+0000 7f3501600640 1 -- 172.21.15.86:0/947395242 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f34f00187b0 con 0x7f35040722f0 2024-04-25T14:03:44.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.626+0000 7f350c200640 1 -- 172.21.15.86:0/947395242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f350415aeb0 con 0x7f35040722f0 2024-04-25T14:03:44.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.626+0000 7f3501600640 1 -- 172.21.15.86:0/947395242 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12884 (secure 0 0 0) 0x7f34f00f88e0 con 0x7f35040722f0 2024-04-25T14:03:44.631 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:03:44.631 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":16,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:03:41.164591+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:03:44.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.630+0000 7f350c200640 1 -- 172.21.15.86:0/947395242 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f34e406f400 msgr2=0x7f34e4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:44.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.630+0000 7f350c200640 1 --2- 172.21.15.86:0/947395242 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f34e406f400 0x7f34e4071910 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f34f80079c0 tx=0x7f34f8008040 comp rx=0 tx=0).stop 2024-04-25T14:03:44.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.630+0000 7f350c200640 1 -- 172.21.15.86:0/947395242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35040722f0 msgr2=0x7f35041e6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:44.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.630+0000 7f350c200640 1 --2- 172.21.15.86:0/947395242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35040722f0 0x7f35041e6dd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f34f00b3040 tx=0x7f34f001b9f0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.630+0000 7f350c200640 1 -- 172.21.15.86:0/947395242 shutdown_connections 2024-04-25T14:03:44.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.630+0000 7f350c200640 1 --2- 172.21.15.86:0/947395242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35040d2c80 0x7f35041e7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.630+0000 7f350c200640 1 --2- 172.21.15.86:0/947395242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35040722f0 0x7f35041e6dd0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.630+0000 7f350c200640 1 --2- 172.21.15.86:0/947395242 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f34e406f400 0x7f34e4071910 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.630+0000 7f350c200640 1 --2- 172.21.15.86:0/947395242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3504070e20 0x7f35041506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.630+0000 7f350c200640 1 -- 172.21.15.86:0/947395242 >> 172.21.15.86:0/947395242 conn(0x7f35040cffc0 msgr2=0x7f35040c9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:44.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.630+0000 7f350c200640 1 -- 172.21.15.86:0/947395242 shutdown_connections 2024-04-25T14:03:44.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.630+0000 7f350c200640 1 -- 172.21.15.86:0/947395242 wait complete. 2024-04-25T14:03:44.653 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:03:44.654 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:03:44.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.798+0000 7f7691600640 1 Processor -- start 2024-04-25T14:03:44.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.798+0000 7f7691600640 1 -- start start 2024-04-25T14:03:44.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.802+0000 7f7691600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f768c0b59e0 0x7f768c0b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:44.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.802+0000 7f7691600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f768c0aeea0 0x7f768c0af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:44.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.802+0000 7f7691600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f768c0b6330 con 0x7f768c15faa0 2024-04-25T14:03:44.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.802+0000 7f7691600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f768c06d560 con 0x7f768c0aeea0 2024-04-25T14:03:44.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.802+0000 7f7691600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f768c06c670 con 0x7f768c0b59e0 2024-04-25T14:03:44.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.802+0000 7f7682000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f768c0b59e0 0x7f768c0b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:44.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.802+0000 7f768b400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f768c0aeea0 0x7f768c0af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:44.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.802+0000 7f7682000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f768c0b59e0 0x7f768c0b5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:51842/0 (socket says 172.21.15.86:51842) 2024-04-25T14:03:44.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.802+0000 7f7682000640 1 -- 172.21.15.86:0/15470922 learned_addr learned my addr 172.21.15.86:0/15470922 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:44.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.802+0000 7f768b400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f768c0aeea0 0x7f768c0af2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:45772/0 (socket says 172.21.15.86:45772) 2024-04-25T14:03:44.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.802+0000 7f7682000640 1 -- 172.21.15.86:0/15470922 >> v2:172.21.15.155:3300/0 conn(0x7f768c0aeea0 msgr2=0x7f768c0af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:44.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.802+0000 7f7682000640 1 --2- 172.21.15.86:0/15470922 >> v2:172.21.15.155:3300/0 conn(0x7f768c0aeea0 0x7f768c0af2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.802+0000 7f7682000640 1 -- 172.21.15.86:0/15470922 >> v1:172.21.15.155:6789/0 conn(0x7f768c15faa0 legacy=0x7f768c06d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:44.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.802+0000 7f7682000640 1 -- 172.21.15.86:0/15470922 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768c06c950 con 0x7f768c0b59e0 2024-04-25T14:03:44.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.806+0000 7f7682000640 1 --2- 172.21.15.86:0/15470922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f768c0b59e0 0x7f768c0b5df0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7674009870 tx=0x7f767401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20ebc412297f18d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:44.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.806+0000 7f768a000640 1 -- 172.21.15.86:0/15470922 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76740a9640 con 0x7f768c0b59e0 2024-04-25T14:03:44.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.806+0000 7f768a000640 1 -- 172.21.15.86:0/15470922 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76740a97e0 con 0x7f768c0b59e0 2024-04-25T14:03:44.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.806+0000 7f768a000640 1 -- 172.21.15.86:0/15470922 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76740a9bb0 con 0x7f768c0b59e0 2024-04-25T14:03:44.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.806+0000 7f7691600640 1 -- 172.21.15.86:0/15470922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f768c0b59e0 msgr2=0x7f768c0b5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:44.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.806+0000 7f7691600640 1 --2- 172.21.15.86:0/15470922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f768c0b59e0 0x7f768c0b5df0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7674009870 tx=0x7f767401c560 comp rx=0 tx=0).stop 2024-04-25T14:03:44.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.806+0000 7f7691600640 1 -- 172.21.15.86:0/15470922 shutdown_connections 2024-04-25T14:03:44.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.806+0000 7f7691600640 1 --2- 172.21.15.86:0/15470922 >> v2:172.21.15.155:3300/0 conn(0x7f768c0aeea0 0x7f768c0af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.806+0000 7f7691600640 1 --2- 172.21.15.86:0/15470922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f768c0b59e0 0x7f768c0b5df0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.806+0000 7f7691600640 1 -- 172.21.15.86:0/15470922 >> 172.21.15.86:0/15470922 conn(0x7f768c0d2230 msgr2=0x7f768c0d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:44.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.806+0000 7f7691600640 1 -- 172.21.15.86:0/15470922 shutdown_connections 2024-04-25T14:03:44.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.806+0000 7f7691600640 1 -- 172.21.15.86:0/15470922 wait complete. 2024-04-25T14:03:44.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.806+0000 7f7691600640 1 Processor -- start 2024-04-25T14:03:44.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.806+0000 7f7691600640 1 -- start start 2024-04-25T14:03:44.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.810+0000 7f7691600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f768c0aeea0 0x7f768c151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:44.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.810+0000 7f7691600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f768c0b59e0 0x7f768c152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:44.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.810+0000 7f7691600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c15faa0 0x7f768c161200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:44.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.810+0000 7f7691600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f768c160030 con 0x7f768c15faa0 2024-04-25T14:03:44.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.810+0000 7f7691600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f768c15feb0 con 0x7f768c0b59e0 2024-04-25T14:03:44.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.810+0000 7f7691600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f768c070f30 con 0x7f768c0aeea0 2024-04-25T14:03:44.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.810+0000 7f768b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c15faa0 0x7f768c161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:44.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.810+0000 7f7682000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f768c0b59e0 0x7f768c152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:44.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.810+0000 7f768b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c15faa0 0x7f768c161200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42058/0 (socket says 172.21.15.86:42058) 2024-04-25T14:03:44.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.810+0000 7f768b400640 1 -- 172.21.15.86:0/1525406960 learned_addr learned my addr 172.21.15.86:0/1525406960 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:44.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.810+0000 7f768aa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f768c0aeea0 0x7f768c151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:44.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.810+0000 7f7682000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f768c0b59e0 0x7f768c152310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:45780/0 (socket says 172.21.15.86:45780) 2024-04-25T14:03:44.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.810+0000 7f768b400640 1 -- 172.21.15.86:0/1525406960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f768c0aeea0 msgr2=0x7f768c151b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:44.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.810+0000 7f768b400640 1 --2- 172.21.15.86:0/1525406960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f768c0aeea0 0x7f768c151b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.810+0000 7f768b400640 1 -- 172.21.15.86:0/1525406960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f768c0b59e0 msgr2=0x7f768c152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:44.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.810+0000 7f768b400640 1 --2- 172.21.15.86:0/1525406960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f768c0b59e0 0x7f768c152310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.810+0000 7f768b400640 1 -- 172.21.15.86:0/1525406960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768c15d200 con 0x7f768c15faa0 2024-04-25T14:03:44.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.810+0000 7f768b400640 1 --2- 172.21.15.86:0/1525406960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c15faa0 0x7f768c161200 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f768400efc0 tx=0x7f768400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:44.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.810+0000 7f7683400640 1 -- 172.21.15.86:0/1525406960 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7684018020 con 0x7f768c15faa0 2024-04-25T14:03:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.810+0000 7f7683400640 1 -- 172.21.15.86:0/1525406960 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7684014480 con 0x7f768c15faa0 2024-04-25T14:03:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.810+0000 7f7691600640 1 -- 172.21.15.86:0/1525406960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f768c15d4e0 con 0x7f768c15faa0 2024-04-25T14:03:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.810+0000 7f7683400640 1 -- 172.21.15.86:0/1525406960 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7684014870 con 0x7f768c15faa0 2024-04-25T14:03:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.810+0000 7f7691600640 1 -- 172.21.15.86:0/1525406960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f768c15daa0 con 0x7f768c15faa0 2024-04-25T14:03:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.810+0000 7f7691600640 1 -- 172.21.15.86:0/1525406960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f768c074cf0 con 0x7f768c15faa0 2024-04-25T14:03:44.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.814+0000 7f7683400640 1 -- 172.21.15.86:0/1525406960 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f768401b020 con 0x7f768c15faa0 2024-04-25T14:03:44.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.814+0000 7f7683400640 1 --2- 172.21.15.86:0/1525406960 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f766806f400 0x7f7668071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:44.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.814+0000 7f7683400640 1 -- 172.21.15.86:0/1525406960 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f768408f840 con 0x7f768c15faa0 2024-04-25T14:03:44.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.814+0000 7f768aa00640 1 --2- 172.21.15.86:0/1525406960 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f766806f400 0x7f7668071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:44.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.814+0000 7f768aa00640 1 --2- 172.21.15.86:0/1525406960 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f766806f400 0x7f7668071910 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f767c001770 tx=0x7f767c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:44.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.818+0000 7f7683400640 1 -- 172.21.15.86:0/1525406960 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7684022070 con 0x7f768c15faa0 2024-04-25T14:03:44.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.974+0000 7f7691600640 1 -- 172.21.15.86:0/1525406960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f768c0b55b0 con 0x7f768c15faa0 2024-04-25T14:03:44.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.974+0000 7f7683400640 1 -- 172.21.15.86:0/1525406960 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f7684010040 con 0x7f768c15faa0 2024-04-25T14:03:44.978 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:03:44.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.974+0000 7f7691600640 1 -- 172.21.15.86:0/1525406960 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f766806f400 msgr2=0x7f7668071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:44.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.974+0000 7f7691600640 1 --2- 172.21.15.86:0/1525406960 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f766806f400 0x7f7668071910 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f767c001770 tx=0x7f767c009040 comp rx=0 tx=0).stop 2024-04-25T14:03:44.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.974+0000 7f7691600640 1 -- 172.21.15.86:0/1525406960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c15faa0 msgr2=0x7f768c161200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:44.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.974+0000 7f7691600640 1 --2- 172.21.15.86:0/1525406960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c15faa0 0x7f768c161200 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f768400efc0 tx=0x7f768400c490 comp rx=0 tx=0).stop 2024-04-25T14:03:44.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.974+0000 7f7691600640 1 -- 172.21.15.86:0/1525406960 shutdown_connections 2024-04-25T14:03:44.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.974+0000 7f7691600640 1 --2- 172.21.15.86:0/1525406960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c15faa0 0x7f768c161200 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.974+0000 7f7691600640 1 --2- 172.21.15.86:0/1525406960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f768c0b59e0 0x7f768c152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.974+0000 7f7691600640 1 --2- 172.21.15.86:0/1525406960 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f766806f400 0x7f7668071910 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.974+0000 7f7691600640 1 --2- 172.21.15.86:0/1525406960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f768c0aeea0 0x7f768c151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:44.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.974+0000 7f7691600640 1 -- 172.21.15.86:0/1525406960 >> 172.21.15.86:0/1525406960 conn(0x7f768c0d2230 msgr2=0x7f768c0b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:44.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.974+0000 7f7691600640 1 -- 172.21.15.86:0/1525406960 shutdown_connections 2024-04-25T14:03:44.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:44.974+0000 7f7691600640 1 -- 172.21.15.86:0/1525406960 wait complete. 2024-04-25T14:03:44.999 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:03:45.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.142+0000 7f9cc4000640 1 Processor -- start 2024-04-25T14:03:45.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.142+0000 7f9cc4000640 1 -- start start 2024-04-25T14:03:45.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.142+0000 7f9cc4000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9cbc072630 0x7f9cbc072a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:45.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.142+0000 7f9cc4000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9cbc06e340 0x7f9cbc06e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:45.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.142+0000 7f9cc4000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cbc072f80 con 0x7f9cbc163980 2024-04-25T14:03:45.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.142+0000 7f9cc4000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9cbc1648e0 con 0x7f9cbc072630 2024-04-25T14:03:45.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.142+0000 7f9cc4000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9cbc0d1520 con 0x7f9cbc06e340 2024-04-25T14:03:45.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.142+0000 7f9cc1600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9cbc072630 0x7f9cbc072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:45.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.142+0000 7f9cc0c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9cbc163980 0x7f9cbc163d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36074/0 (socket says 172.21.15.86:36074) 2024-04-25T14:03:45.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.142+0000 7f9cc0c00640 1 -- 172.21.15.86:0/3326091692 learned_addr learned my addr 172.21.15.86:0/3326091692 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:03:45.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.142+0000 7f9cc2000640 1 --2- 172.21.15.86:0/3326091692 >> v2:172.21.15.155:3300/0 conn(0x7f9cbc06e340 0x7f9cbc06e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:45.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.142+0000 7f9cc1600640 1 -- 172.21.15.86:0/3326091692 >> v2:172.21.15.155:3300/0 conn(0x7f9cbc06e340 msgr2=0x7f9cbc06e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:45.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.142+0000 7f9cc1600640 1 --2- 172.21.15.86:0/3326091692 >> v2:172.21.15.155:3300/0 conn(0x7f9cbc06e340 0x7f9cbc06e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:45.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.142+0000 7f9cc1600640 1 -- 172.21.15.86:0/3326091692 >> v1:172.21.15.86:6789/0 conn(0x7f9cbc163980 legacy=0x7f9cbc163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:45.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.142+0000 7f9cc1600640 1 -- 172.21.15.86:0/3326091692 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cbc0d23e0 con 0x7f9cbc072630 2024-04-25T14:03:45.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.142+0000 7f9cbbe00640 1 -- 172.21.15.86:0/3326091692 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1658940875 0 0) 0x7f9ca8001120 con 0x7f9cbc163980 2024-04-25T14:03:45.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.146+0000 7f9cc1600640 1 --2- 172.21.15.86:0/3326091692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cbc072630 0x7f9cbc072a40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc0c6b40 tx=0x7f9cb001c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d726879268869507 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:45.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.146+0000 7f9cbbe00640 1 -- 172.21.15.86:0/3326091692 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb00b0070 con 0x7f9cbc072630 2024-04-25T14:03:45.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.146+0000 7f9cbbe00640 1 -- 172.21.15.86:0/3326091692 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb00ab600 con 0x7f9cbc072630 2024-04-25T14:03:45.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.146+0000 7f9cbbe00640 1 -- 172.21.15.86:0/3326091692 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb00ab9d0 con 0x7f9cbc072630 2024-04-25T14:03:45.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.146+0000 7f9cc4000640 1 -- 172.21.15.86:0/3326091692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cbc072630 msgr2=0x7f9cbc072a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:45.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.146+0000 7f9cc4000640 1 --2- 172.21.15.86:0/3326091692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cbc072630 0x7f9cbc072a40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc0c6b40 tx=0x7f9cb001c360 comp rx=0 tx=0).stop 2024-04-25T14:03:45.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.146+0000 7f9cc4000640 1 -- 172.21.15.86:0/3326091692 shutdown_connections 2024-04-25T14:03:45.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.146+0000 7f9cc4000640 1 --2- 172.21.15.86:0/3326091692 >> v2:172.21.15.155:3300/0 conn(0x7f9cbc06e340 0x7f9cbc06e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:45.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.146+0000 7f9cc4000640 1 --2- 172.21.15.86:0/3326091692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cbc072630 0x7f9cbc072a40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:45.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.146+0000 7f9cc4000640 1 -- 172.21.15.86:0/3326091692 >> 172.21.15.86:0/3326091692 conn(0x7f9cbc0d3580 msgr2=0x7f9cbc0d3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:45.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.146+0000 7f9cc4000640 1 -- 172.21.15.86:0/3326091692 shutdown_connections 2024-04-25T14:03:45.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.146+0000 7f9cc4000640 1 -- 172.21.15.86:0/3326091692 wait complete. 2024-04-25T14:03:45.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.150+0000 7f9cc4000640 1 Processor -- start 2024-04-25T14:03:45.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.150+0000 7f9cc4000640 1 -- start start 2024-04-25T14:03:45.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.150+0000 7f9cc4000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cbc06e340 0x7f9cbc0c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:45.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.150+0000 7f9cc4000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cbc072630 0x7f9cbc1f9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:45.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.150+0000 7f9cc4000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cbc163980 0x7f9cbc1fa1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:45.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.150+0000 7f9cc4000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc16d530 con 0x7f9cbc072630 2024-04-25T14:03:45.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.150+0000 7f9cc4000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc06f0a0 con 0x7f9cbc06e340 2024-04-25T14:03:45.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.150+0000 7f9cc4000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9cbc16d6b0 con 0x7f9cbc163980 2024-04-25T14:03:45.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.150+0000 7f9cc0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cbc072630 0x7f9cbc1f9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:45.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.150+0000 7f9cc1600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cbc06e340 0x7f9cbc0c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:45.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.150+0000 7f9cc0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cbc072630 0x7f9cbc1f9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42078/0 (socket says 172.21.15.86:42078) 2024-04-25T14:03:45.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.150+0000 7f9cc0c00640 1 -- 172.21.15.86:0/2159098778 learned_addr learned my addr 172.21.15.86:0/2159098778 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:45.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.150+0000 7f9cc2000640 1 --2- 172.21.15.86:0/2159098778 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cbc163980 0x7f9cbc1fa1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:45.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.154+0000 7f9cc0c00640 1 -- 172.21.15.86:0/2159098778 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cbc163980 msgr2=0x7f9cbc1fa1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:45.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.154+0000 7f9cc0c00640 1 --2- 172.21.15.86:0/2159098778 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cbc163980 0x7f9cbc1fa1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:45.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.154+0000 7f9cc0c00640 1 -- 172.21.15.86:0/2159098778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cbc06e340 msgr2=0x7f9cbc0c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:45.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.154+0000 7f9cc0c00640 1 --2- 172.21.15.86:0/2159098778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cbc06e340 0x7f9cbc0c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:45.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.154+0000 7f9cc0c00640 1 -- 172.21.15.86:0/2159098778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cbc15aa90 con 0x7f9cbc072630 2024-04-25T14:03:45.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.154+0000 7f9cc0c00640 1 --2- 172.21.15.86:0/2159098778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cbc072630 0x7f9cbc1f9cb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8005890 tx=0x7f9ca800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:45.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.154+0000 7f9cb9600640 1 -- 172.21.15.86:0/2159098778 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8017070 con 0x7f9cbc072630 2024-04-25T14:03:45.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.154+0000 7f9cb9600640 1 -- 172.21.15.86:0/2159098778 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ca8005d10 con 0x7f9cbc072630 2024-04-25T14:03:45.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.154+0000 7f9cb9600640 1 -- 172.21.15.86:0/2159098778 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8007310 con 0x7f9cbc072630 2024-04-25T14:03:45.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.154+0000 7f9cc4000640 1 -- 172.21.15.86:0/2159098778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cbc0c8c70 con 0x7f9cbc072630 2024-04-25T14:03:45.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.154+0000 7f9cc4000640 1 -- 172.21.15.86:0/2159098778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cbc0c9230 con 0x7f9cbc072630 2024-04-25T14:03:45.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.154+0000 7f9cc4000640 1 -- 172.21.15.86:0/2159098778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cbc0c89c0 con 0x7f9cbc072630 2024-04-25T14:03:45.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.154+0000 7f9cb9600640 1 -- 172.21.15.86:0/2159098778 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9ca801a070 con 0x7f9cbc072630 2024-04-25T14:03:45.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.154+0000 7f9cb9600640 1 --2- 172.21.15.86:0/2159098778 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9c9c06f430 0x7f9c9c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:45.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.158+0000 7f9cb9600640 1 -- 172.21.15.86:0/2159098778 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f9ca808e2f0 con 0x7f9cbc072630 2024-04-25T14:03:45.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.158+0000 7f9cc1600640 1 --2- 172.21.15.86:0/2159098778 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9c9c06f430 0x7f9c9c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:45.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.158+0000 7f9cc1600640 1 --2- 172.21.15.86:0/2159098778 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9c9c06f430 0x7f9c9c071940 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9cb00ad000 tx=0x7f9cb0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:45.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.162+0000 7f9cb9600640 1 -- 172.21.15.86:0/2159098778 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9cbc15aa90 con 0x7f9cbc072630 2024-04-25T14:03:45.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.350+0000 7f9cc4000640 1 -- 172.21.15.86:0/2159098778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9cbc072170 con 0x7f9cbc072630 2024-04-25T14:03:45.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.350+0000 7f9cb9600640 1 -- 172.21.15.86:0/2159098778 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7f9cbc072f80 con 0x7f9cbc072630 2024-04-25T14:03:45.356 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:03:45.356 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2236737549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2236737549},{"type":"v1","addr":"172.21.15.86:6837","nonce":2236737549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T14:03:45.356 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 2 2024-04-25T14:03:45.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.354+0000 7f9cc4000640 1 -- 172.21.15.86:0/2159098778 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9c9c06f430 msgr2=0x7f9c9c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:45.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.354+0000 7f9cc4000640 1 --2- 172.21.15.86:0/2159098778 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9c9c06f430 0x7f9c9c071940 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9cb00ad000 tx=0x7f9cb0008040 comp rx=0 tx=0).stop 2024-04-25T14:03:45.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.354+0000 7f9cc4000640 1 -- 172.21.15.86:0/2159098778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cbc072630 msgr2=0x7f9cbc1f9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:45.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.354+0000 7f9cc4000640 1 --2- 172.21.15.86:0/2159098778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cbc072630 0x7f9cbc1f9cb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8005890 tx=0x7f9ca800c8e0 comp rx=0 tx=0).stop 2024-04-25T14:03:45.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.354+0000 7f9cc4000640 1 -- 172.21.15.86:0/2159098778 shutdown_connections 2024-04-25T14:03:45.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.354+0000 7f9cc4000640 1 --2- 172.21.15.86:0/2159098778 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9c9c06f430 0x7f9c9c071940 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:45.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.354+0000 7f9cc4000640 1 --2- 172.21.15.86:0/2159098778 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cbc163980 0x7f9cbc1fa1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:45.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.354+0000 7f9cc4000640 1 --2- 172.21.15.86:0/2159098778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cbc072630 0x7f9cbc1f9cb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:45.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.354+0000 7f9cc4000640 1 --2- 172.21.15.86:0/2159098778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cbc06e340 0x7f9cbc0c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:45.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.354+0000 7f9cc4000640 1 -- 172.21.15.86:0/2159098778 >> 172.21.15.86:0/2159098778 conn(0x7f9cbc0d3580 msgr2=0x7f9cbc163d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:45.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.354+0000 7f9cc4000640 1 -- 172.21.15.86:0/2159098778 shutdown_connections 2024-04-25T14:03:45.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:45.354+0000 7f9cc4000640 1 -- 172.21.15.86:0/2159098778 wait complete. 2024-04-25T14:03:47.379 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:03:47.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.538+0000 7fb4d2200640 1 Processor -- start 2024-04-25T14:03:47.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.538+0000 7fb4d2200640 1 -- start start 2024-04-25T14:03:47.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.538+0000 7fb4d2200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb4cc0d1df0 0x7fb4cc0b0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:47.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.538+0000 7fb4d2200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb4cc0b0690 0x7fb4cc0d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:47.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.538+0000 7fb4d2200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4cc0cde20 con 0x7fb4cc1642a0 2024-04-25T14:03:47.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.538+0000 7fb4d2200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb4cc0cea30 con 0x7fb4cc0b0690 2024-04-25T14:03:47.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.538+0000 7fb4d2200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb4cc0ce100 con 0x7fb4cc0d1df0 2024-04-25T14:03:47.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.538+0000 7fb4cbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb4cc0b0690 0x7fb4cc0d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:47.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.538+0000 7fb4cb400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb4cc1642a0 0x7fb4cc0d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36080/0 (socket says 172.21.15.86:36080) 2024-04-25T14:03:47.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.538+0000 7fb4cbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb4cc0b0690 0x7fb4cc0d0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42086/0 (socket says 172.21.15.86:42086) 2024-04-25T14:03:47.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.538+0000 7fb4cb400640 1 -- 172.21.15.86:0/1686057947 learned_addr learned my addr 172.21.15.86:0/1686057947 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:03:47.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.538+0000 7fb4c2a00640 1 --2- 172.21.15.86:0/1686057947 >> v2:172.21.15.155:3300/0 conn(0x7fb4cc0d1df0 0x7fb4cc0b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:47.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.538+0000 7fb4cbe00640 1 -- 172.21.15.86:0/1686057947 >> v2:172.21.15.155:3300/0 conn(0x7fb4cc0d1df0 msgr2=0x7fb4cc0b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:47.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.538+0000 7fb4cbe00640 1 --2- 172.21.15.86:0/1686057947 >> v2:172.21.15.155:3300/0 conn(0x7fb4cc0d1df0 0x7fb4cc0b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:47.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.538+0000 7fb4cbe00640 1 -- 172.21.15.86:0/1686057947 >> v1:172.21.15.86:6789/0 conn(0x7fb4cc1642a0 legacy=0x7fb4cc0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:47.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.538+0000 7fb4cbe00640 1 -- 172.21.15.86:0/1686057947 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4cc0ce3e0 con 0x7fb4cc0b0690 2024-04-25T14:03:47.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.538+0000 7fb4caa00640 1 -- 172.21.15.86:0/1686057947 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3894340069 0 0) 0x7fb4cc0cde20 con 0x7fb4cc1642a0 2024-04-25T14:03:47.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.542+0000 7fb4cbe00640 1 --2- 172.21.15.86:0/1686057947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4cc0b0690 0x7fb4cc0d0640 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc06df80 tx=0x7fb4c401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd862978700c6fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:47.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.542+0000 7fb4caa00640 1 -- 172.21.15.86:0/1686057947 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c40b2070 con 0x7fb4cc0b0690 2024-04-25T14:03:47.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.542+0000 7fb4caa00640 1 -- 172.21.15.86:0/1686057947 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4c40a5760 con 0x7fb4cc0b0690 2024-04-25T14:03:47.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.542+0000 7fb4caa00640 1 -- 172.21.15.86:0/1686057947 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c40a5b30 con 0x7fb4cc0b0690 2024-04-25T14:03:47.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.542+0000 7fb4d2200640 1 -- 172.21.15.86:0/1686057947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4cc0b0690 msgr2=0x7fb4cc0d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:47.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.542+0000 7fb4d2200640 1 --2- 172.21.15.86:0/1686057947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4cc0b0690 0x7fb4cc0d0640 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc06df80 tx=0x7fb4c401c8a0 comp rx=0 tx=0).stop 2024-04-25T14:03:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.542+0000 7fb4d2200640 1 -- 172.21.15.86:0/1686057947 shutdown_connections 2024-04-25T14:03:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.542+0000 7fb4d2200640 1 --2- 172.21.15.86:0/1686057947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4cc0b0690 0x7fb4cc0d0640 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.542+0000 7fb4d2200640 1 --2- 172.21.15.86:0/1686057947 >> v2:172.21.15.155:3300/0 conn(0x7fb4cc0d1df0 0x7fb4cc0b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.542+0000 7fb4d2200640 1 -- 172.21.15.86:0/1686057947 >> 172.21.15.86:0/1686057947 conn(0x7fb4cc0b4650 msgr2=0x7fb4cc0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.542+0000 7fb4d2200640 1 -- 172.21.15.86:0/1686057947 shutdown_connections 2024-04-25T14:03:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.542+0000 7fb4d2200640 1 -- 172.21.15.86:0/1686057947 wait complete. 2024-04-25T14:03:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.546+0000 7fb4d2200640 1 Processor -- start 2024-04-25T14:03:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.546+0000 7fb4d2200640 1 -- start start 2024-04-25T14:03:47.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.546+0000 7fb4d2200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4cc0b0690 0x7fb4cc09aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:47.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.546+0000 7fb4d2200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4cc1642a0 0x7fb4cc09b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:47.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.546+0000 7fb4d2200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4cc09dcf0 0x7fb4cc09e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:47.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.546+0000 7fb4d2200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc164830 con 0x7fb4cc0b0690 2024-04-25T14:03:47.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.546+0000 7fb4d2200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc1646b0 con 0x7fb4cc1642a0 2024-04-25T14:03:47.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.546+0000 7fb4d2200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb4cc160da0 con 0x7fb4cc09dcf0 2024-04-25T14:03:47.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.546+0000 7fb4cb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4cc0b0690 0x7fb4cc09aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:47.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.546+0000 7fb4cb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4cc0b0690 0x7fb4cc09aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42090/0 (socket says 172.21.15.86:42090) 2024-04-25T14:03:47.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.546+0000 7fb4cb400640 1 -- 172.21.15.86:0/3659219258 learned_addr learned my addr 172.21.15.86:0/3659219258 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:47.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.546+0000 7fb4cbe00640 1 --2- 172.21.15.86:0/3659219258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4cc09dcf0 0x7fb4cc09e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:47.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.550+0000 7fb4c2a00640 1 --2- 172.21.15.86:0/3659219258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4cc1642a0 0x7fb4cc09b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:47.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.550+0000 7fb4cb400640 1 -- 172.21.15.86:0/3659219258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4cc09dcf0 msgr2=0x7fb4cc09e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:47.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.550+0000 7fb4cb400640 1 --2- 172.21.15.86:0/3659219258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4cc09dcf0 0x7fb4cc09e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:47.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.550+0000 7fb4cb400640 1 -- 172.21.15.86:0/3659219258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4cc1642a0 msgr2=0x7fb4cc09b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:47.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.550+0000 7fb4cb400640 1 --2- 172.21.15.86:0/3659219258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4cc1642a0 0x7fb4cc09b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:47.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.550+0000 7fb4cb400640 1 -- 172.21.15.86:0/3659219258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4cc09b7f0 con 0x7fb4cc0b0690 2024-04-25T14:03:47.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.550+0000 7fb4c2a00640 1 --2- 172.21.15.86:0/3659219258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4cc1642a0 0x7fb4cc09b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:47.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.550+0000 7fb4cb400640 1 --2- 172.21.15.86:0/3659219258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4cc0b0690 0x7fb4cc09aa70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400c480 tx=0x7fb4b400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:47.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.550+0000 7fb4c3e00640 1 -- 172.21.15.86:0/3659219258 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4018070 con 0x7fb4cc0b0690 2024-04-25T14:03:47.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.550+0000 7fb4c3e00640 1 -- 172.21.15.86:0/3659219258 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4b40074e0 con 0x7fb4cc0b0690 2024-04-25T14:03:47.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.550+0000 7fb4c3e00640 1 -- 172.21.15.86:0/3659219258 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b40049f0 con 0x7fb4cc0b0690 2024-04-25T14:03:47.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.550+0000 7fb4d2200640 1 -- 172.21.15.86:0/3659219258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4cc09f150 con 0x7fb4cc0b0690 2024-04-25T14:03:47.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.550+0000 7fb4d2200640 1 -- 172.21.15.86:0/3659219258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4cc073d60 con 0x7fb4cc0b0690 2024-04-25T14:03:47.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.550+0000 7fb4d2200640 1 -- 172.21.15.86:0/3659219258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4cc069d00 con 0x7fb4cc0b0690 2024-04-25T14:03:47.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.550+0000 7fb4c3e00640 1 -- 172.21.15.86:0/3659219258 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb4b4004b90 con 0x7fb4cc0b0690 2024-04-25T14:03:47.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.550+0000 7fb4c3e00640 1 --2- 172.21.15.86:0/3659219258 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb4a806f400 0x7fb4a8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:47.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.550+0000 7fb4c3e00640 1 -- 172.21.15.86:0/3659219258 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fb4b408e2c0 con 0x7fb4cc0b0690 2024-04-25T14:03:47.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.554+0000 7fb4c2a00640 1 --2- 172.21.15.86:0/3659219258 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb4a806f400 0x7fb4a8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:47.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.554+0000 7fb4c2a00640 1 --2- 172.21.15.86:0/3659219258 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb4a806f400 0x7fb4a8071910 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc069120 tx=0x7fb4bc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:47.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.558+0000 7fb4c3e00640 1 -- 172.21.15.86:0/3659219258 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb4b405aea0 con 0x7fb4cc0b0690 2024-04-25T14:03:47.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.746+0000 7fb4d2200640 1 -- 172.21.15.86:0/3659219258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4cc076bf0 con 0x7fb4cc0b0690 2024-04-25T14:03:47.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.746+0000 7fb4c3e00640 1 -- 172.21.15.86:0/3659219258 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3864 (secure 0 0 0) 0x7fb4b405f1a0 con 0x7fb4cc0b0690 2024-04-25T14:03:47.751 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:03:47.751 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/347107112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":347107112},{"type":"v1","addr":"172.21.15.155:6835","nonce":347107112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2236737549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2236737549},{"type":"v1","addr":"172.21.15.86:6837","nonce":2236737549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/519560229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":519560229},{"type":"v1","addr":"172.21.15.86:6836","nonce":519560229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1944249570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1944249570},{"type":"v1","addr":"172.21.15.155:6833","nonce":1944249570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T14:03:47.751 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 3 2024-04-25T14:03:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.746+0000 7fb4d2200640 1 -- 172.21.15.86:0/3659219258 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb4a806f400 msgr2=0x7fb4a8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.746+0000 7fb4d2200640 1 --2- 172.21.15.86:0/3659219258 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb4a806f400 0x7fb4a8071910 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc069120 tx=0x7fb4bc008040 comp rx=0 tx=0).stop 2024-04-25T14:03:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.746+0000 7fb4d2200640 1 -- 172.21.15.86:0/3659219258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4cc0b0690 msgr2=0x7fb4cc09aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.746+0000 7fb4d2200640 1 --2- 172.21.15.86:0/3659219258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4cc0b0690 0x7fb4cc09aa70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400c480 tx=0x7fb4b400c940 comp rx=0 tx=0).stop 2024-04-25T14:03:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.746+0000 7fb4d2200640 1 -- 172.21.15.86:0/3659219258 shutdown_connections 2024-04-25T14:03:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.750+0000 7fb4d2200640 1 --2- 172.21.15.86:0/3659219258 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb4a806f400 0x7fb4a8071910 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.750+0000 7fb4d2200640 1 --2- 172.21.15.86:0/3659219258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4cc09dcf0 0x7fb4cc09e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.750+0000 7fb4d2200640 1 --2- 172.21.15.86:0/3659219258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4cc1642a0 0x7fb4cc09b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.750+0000 7fb4d2200640 1 --2- 172.21.15.86:0/3659219258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4cc0b0690 0x7fb4cc09aa70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.750+0000 7fb4d2200640 1 -- 172.21.15.86:0/3659219258 >> 172.21.15.86:0/3659219258 conn(0x7fb4cc0b4650 msgr2=0x7fb4cc0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.750+0000 7fb4d2200640 1 -- 172.21.15.86:0/3659219258 shutdown_connections 2024-04-25T14:03:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.750+0000 7fb4d2200640 1 -- 172.21.15.86:0/3659219258 wait complete. 2024-04-25T14:03:47.772 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:03:47.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.914+0000 7f91af600640 1 Processor -- start 2024-04-25T14:03:47.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.914+0000 7f91af600640 1 -- start start 2024-04-25T14:03:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.914+0000 7f91af600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f91a806e7a0 0x7f91a81602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.914+0000 7f91af600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91a8071530 0x7f91a8163e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.914+0000 7f91af600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91a80d1c10 con 0x7f91a81607e0 2024-04-25T14:03:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.914+0000 7f91af600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f91a80d1ef0 con 0x7f91a806e7a0 2024-04-25T14:03:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.914+0000 7f91af600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f91a80d17d0 con 0x7f91a8071530 2024-04-25T14:03:47.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.914+0000 7f91a7e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f91a81607e0 0x7f91a8070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36096/0 (socket says 172.21.15.86:36096) 2024-04-25T14:03:47.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.918+0000 7f91a7e00640 1 -- 172.21.15.86:0/3025872789 learned_addr learned my addr 172.21.15.86:0/3025872789 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:03:47.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.918+0000 7f91acc00640 1 --2- 172.21.15.86:0/3025872789 >> v2:172.21.15.155:3300/0 conn(0x7f91a806e7a0 0x7f91a81602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:47.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.918+0000 7f91ad600640 1 --2- 172.21.15.86:0/3025872789 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91a8071530 0x7f91a8163e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:47.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.918+0000 7f91a7400640 1 -- 172.21.15.86:0/3025872789 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 22436683 0 0) 0x7f91a80d1c10 con 0x7f91a81607e0 2024-04-25T14:03:47.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.918+0000 7f91a7400640 1 -- 172.21.15.86:0/3025872789 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9190003530 con 0x7f91a81607e0 2024-04-25T14:03:47.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.918+0000 7f91ad600640 1 -- 172.21.15.86:0/3025872789 >> v2:172.21.15.155:3300/0 conn(0x7f91a806e7a0 msgr2=0x7f91a81602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:47.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.918+0000 7f91ad600640 1 --2- 172.21.15.86:0/3025872789 >> v2:172.21.15.155:3300/0 conn(0x7f91a806e7a0 0x7f91a81602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:47.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.918+0000 7f91ad600640 1 -- 172.21.15.86:0/3025872789 >> v1:172.21.15.86:6789/0 conn(0x7f91a81607e0 legacy=0x7f91a8070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:47.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.918+0000 7f91ad600640 1 -- 172.21.15.86:0/3025872789 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a80d21d0 con 0x7f91a8071530 2024-04-25T14:03:47.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.918+0000 7f91ad600640 1 --2- 172.21.15.86:0/3025872789 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91a8071530 0x7f91a8163e10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009d90 tx=0x7f91a001c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23f95438fc2c2bad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:47.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.922+0000 7f91a7400640 1 -- 172.21.15.86:0/3025872789 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a00b2070 con 0x7f91a8071530 2024-04-25T14:03:47.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.922+0000 7f91a7400640 1 -- 172.21.15.86:0/3025872789 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91a00a57a0 con 0x7f91a8071530 2024-04-25T14:03:47.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.922+0000 7f91a7400640 1 -- 172.21.15.86:0/3025872789 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a00a5b70 con 0x7f91a8071530 2024-04-25T14:03:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.922+0000 7f91af600640 1 -- 172.21.15.86:0/3025872789 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91a8071530 msgr2=0x7f91a8163e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.922+0000 7f91af600640 1 --2- 172.21.15.86:0/3025872789 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91a8071530 0x7f91a8163e10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009d90 tx=0x7f91a001c8e0 comp rx=0 tx=0).stop 2024-04-25T14:03:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.922+0000 7f91af600640 1 -- 172.21.15.86:0/3025872789 shutdown_connections 2024-04-25T14:03:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.922+0000 7f91af600640 1 --2- 172.21.15.86:0/3025872789 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91a8071530 0x7f91a8163e10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.922+0000 7f91af600640 1 --2- 172.21.15.86:0/3025872789 >> v2:172.21.15.155:3300/0 conn(0x7f91a806e7a0 0x7f91a81602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:47.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.922+0000 7f91af600640 1 -- 172.21.15.86:0/3025872789 >> 172.21.15.86:0/3025872789 conn(0x7f91a80cffc0 msgr2=0x7f91a80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:47.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.922+0000 7f91af600640 1 -- 172.21.15.86:0/3025872789 shutdown_connections 2024-04-25T14:03:47.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.922+0000 7f91af600640 1 -- 172.21.15.86:0/3025872789 wait complete. 2024-04-25T14:03:47.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.922+0000 7f91af600640 1 Processor -- start 2024-04-25T14:03:47.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.922+0000 7f91af600640 1 -- start start 2024-04-25T14:03:47.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.926+0000 7f91af600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91a806e7a0 0x7f91a80b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:47.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.926+0000 7f91af600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91a81607e0 0x7f91a815c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.926+0000 7f91af600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91a80b80f0 0x7f91a80cc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.926+0000 7f91af600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f91a816d7d0 con 0x7f91a806e7a0 2024-04-25T14:03:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.926+0000 7f91af600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f91a816d650 con 0x7f91a80b80f0 2024-04-25T14:03:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.926+0000 7f91af600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f91a816d950 con 0x7f91a81607e0 2024-04-25T14:03:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.926+0000 7f91acc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91a806e7a0 0x7f91a80b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.926+0000 7f91acc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91a806e7a0 0x7f91a80b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42094/0 (socket says 172.21.15.86:42094) 2024-04-25T14:03:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.926+0000 7f91acc00640 1 -- 172.21.15.86:0/257476552 learned_addr learned my addr 172.21.15.86:0/257476552 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.926+0000 7f91a7e00640 1 --2- 172.21.15.86:0/257476552 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91a81607e0 0x7f91a815c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.926+0000 7f91ad600640 1 --2- 172.21.15.86:0/257476552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91a80b80f0 0x7f91a80cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.926+0000 7f91acc00640 1 -- 172.21.15.86:0/257476552 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91a81607e0 msgr2=0x7f91a815c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.926+0000 7f91acc00640 1 --2- 172.21.15.86:0/257476552 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91a81607e0 0x7f91a815c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.926+0000 7f91acc00640 1 -- 172.21.15.86:0/257476552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91a80b80f0 msgr2=0x7f91a80cc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.926+0000 7f91acc00640 1 --2- 172.21.15.86:0/257476552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91a80b80f0 0x7f91a80cc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.926+0000 7f91acc00640 1 -- 172.21.15.86:0/257476552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a80b70c0 con 0x7f91a806e7a0 2024-04-25T14:03:47.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.926+0000 7f91a7e00640 1 --2- 172.21.15.86:0/257476552 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91a81607e0 0x7f91a815c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:47.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.926+0000 7f91acc00640 1 --2- 172.21.15.86:0/257476552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91a806e7a0 0x7f91a80b45e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f919c00e8a0 tx=0x7f919c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:47.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.926+0000 7f91a4c00640 1 -- 172.21.15.86:0/257476552 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c017070 con 0x7f91a806e7a0 2024-04-25T14:03:47.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.926+0000 7f91a4c00640 1 -- 172.21.15.86:0/257476552 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f919c00c940 con 0x7f91a806e7a0 2024-04-25T14:03:47.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.926+0000 7f91a4c00640 1 -- 172.21.15.86:0/257476552 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c00ccf0 con 0x7f91a806e7a0 2024-04-25T14:03:47.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.926+0000 7f91af600640 1 -- 172.21.15.86:0/257476552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a80b7330 con 0x7f91a806e7a0 2024-04-25T14:03:47.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.926+0000 7f91af600640 1 -- 172.21.15.86:0/257476552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a80b78f0 con 0x7f91a806e7a0 2024-04-25T14:03:47.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.926+0000 7f91af600640 1 -- 172.21.15.86:0/257476552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91a80720e0 con 0x7f91a806e7a0 2024-04-25T14:03:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.930+0000 7f91a4c00640 1 -- 172.21.15.86:0/257476552 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f919c01a070 con 0x7f91a806e7a0 2024-04-25T14:03:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.930+0000 7f91a4c00640 1 --2- 172.21.15.86:0/257476552 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f918806f400 0x7f9188071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.930+0000 7f91a4c00640 1 -- 172.21.15.86:0/257476552 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f919c08f240 con 0x7f91a806e7a0 2024-04-25T14:03:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.930+0000 7f91a7e00640 1 --2- 172.21.15.86:0/257476552 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f918806f400 0x7f9188071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.930+0000 7f91a7e00640 1 --2- 172.21.15.86:0/257476552 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f918806f400 0x7f9188071910 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9194007fd0 tx=0x7f9194009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:47.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:47.934+0000 7f91a4c00640 1 -- 172.21.15.86:0/257476552 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f919c05bd90 con 0x7f91a806e7a0 2024-04-25T14:03:48.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.090+0000 7f91af600640 1 -- 172.21.15.86:0/257476552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91a8076bf0 con 0x7f91a806e7a0 2024-04-25T14:03:48.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.090+0000 7f91a4c00640 1 -- 172.21.15.86:0/257476552 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12884 (secure 0 0 0) 0x7f919c060090 con 0x7f91a806e7a0 2024-04-25T14:03:48.096 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:03:48.096 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":16,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:03:41.164591+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:03:48.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.094+0000 7f91af600640 1 -- 172.21.15.86:0/257476552 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f918806f400 msgr2=0x7f9188071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:48.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.094+0000 7f91af600640 1 --2- 172.21.15.86:0/257476552 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f918806f400 0x7f9188071910 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9194007fd0 tx=0x7f9194009040 comp rx=0 tx=0).stop 2024-04-25T14:03:48.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.094+0000 7f91af600640 1 -- 172.21.15.86:0/257476552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91a806e7a0 msgr2=0x7f91a80b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:48.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.094+0000 7f91af600640 1 --2- 172.21.15.86:0/257476552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91a806e7a0 0x7f91a80b45e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f919c00e8a0 tx=0x7f919c00eca0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.094+0000 7f91af600640 1 -- 172.21.15.86:0/257476552 shutdown_connections 2024-04-25T14:03:48.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.094+0000 7f91af600640 1 --2- 172.21.15.86:0/257476552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91a80b80f0 0x7f91a80cc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.094+0000 7f91af600640 1 --2- 172.21.15.86:0/257476552 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f918806f400 0x7f9188071910 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.094+0000 7f91af600640 1 --2- 172.21.15.86:0/257476552 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91a81607e0 0x7f91a815c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.094+0000 7f91af600640 1 --2- 172.21.15.86:0/257476552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91a806e7a0 0x7f91a80b45e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.094+0000 7f91af600640 1 -- 172.21.15.86:0/257476552 >> 172.21.15.86:0/257476552 conn(0x7f91a80cffc0 msgr2=0x7f91a80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:48.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.094+0000 7f91af600640 1 -- 172.21.15.86:0/257476552 shutdown_connections 2024-04-25T14:03:48.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.094+0000 7f91af600640 1 -- 172.21.15.86:0/257476552 wait complete. 2024-04-25T14:03:48.118 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:03:48.118 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:03:48.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.262+0000 7fa106400640 1 Processor -- start 2024-04-25T14:03:48.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.262+0000 7fa106400640 1 -- start start 2024-04-25T14:03:48.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.266+0000 7fa106400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa1000aeea0 0x7fa1000af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.266+0000 7fa106400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa10006c670 con 0x7fa1000b59e0 2024-04-25T14:03:48.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.266+0000 7fa106400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa10006c950 con 0x7fa10015faa0 2024-04-25T14:03:48.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.266+0000 7fa106400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa10006d560 con 0x7fa1000aeea0 2024-04-25T14:03:48.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.266+0000 7fa0ffe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa1000aeea0 0x7fa1000af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.266+0000 7fa0f6a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa1000b59e0 0x7fa1000b5df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36102/0 (socket says 172.21.15.86:36102) 2024-04-25T14:03:48.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.266+0000 7fa0f6a00640 1 -- 172.21.15.86:0/3896106071 learned_addr learned my addr 172.21.15.86:0/3896106071 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:03:48.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.266+0000 7fa0ffe00640 1 -- 172.21.15.86:0/3896106071 >> v1:172.21.15.155:6789/0 conn(0x7fa10015faa0 legacy=0x7fa10006d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:48.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.266+0000 7fa0ffe00640 1 -- 172.21.15.86:0/3896106071 >> v1:172.21.15.86:6789/0 conn(0x7fa1000b59e0 legacy=0x7fa1000b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:48.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.266+0000 7fa0ffe00640 1 -- 172.21.15.86:0/3896106071 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10006cc30 con 0x7fa1000aeea0 2024-04-25T14:03:48.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.266+0000 7fa0fea00640 1 -- 172.21.15.86:0/3896106071 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2227248715 0 0) 0x7fa10006c670 con 0x7fa1000b59e0 2024-04-25T14:03:48.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.270+0000 7fa0ffe00640 1 --2- 172.21.15.86:0/3896106071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1000aeea0 0x7fa1000af2b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009d90 tx=0x7fa0f801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f351e81e3f78800a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:48.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.270+0000 7fa0fea00640 1 -- 172.21.15.86:0/3896106071 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f80b2070 con 0x7fa1000aeea0 2024-04-25T14:03:48.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.270+0000 7fa0fea00640 1 -- 172.21.15.86:0/3896106071 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f80a5a50 con 0x7fa1000aeea0 2024-04-25T14:03:48.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.270+0000 7fa0fea00640 1 -- 172.21.15.86:0/3896106071 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f80a5e20 con 0x7fa1000aeea0 2024-04-25T14:03:48.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.270+0000 7fa106400640 1 -- 172.21.15.86:0/3896106071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1000aeea0 msgr2=0x7fa1000af2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:48.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.270+0000 7fa106400640 1 --2- 172.21.15.86:0/3896106071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1000aeea0 0x7fa1000af2b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009d90 tx=0x7fa0f801c970 comp rx=0 tx=0).stop 2024-04-25T14:03:48.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.270+0000 7fa106400640 1 -- 172.21.15.86:0/3896106071 shutdown_connections 2024-04-25T14:03:48.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.270+0000 7fa106400640 1 --2- 172.21.15.86:0/3896106071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1000aeea0 0x7fa1000af2b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.270+0000 7fa106400640 1 -- 172.21.15.86:0/3896106071 >> 172.21.15.86:0/3896106071 conn(0x7fa1000d2230 msgr2=0x7fa1000d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:48.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.270+0000 7fa106400640 1 -- 172.21.15.86:0/3896106071 shutdown_connections 2024-04-25T14:03:48.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.270+0000 7fa106400640 1 -- 172.21.15.86:0/3896106071 wait complete. 2024-04-25T14:03:48.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.270+0000 7fa106400640 1 Processor -- start 2024-04-25T14:03:48.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.270+0000 7fa106400640 1 -- start start 2024-04-25T14:03:48.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.274+0000 7fa106400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1000aeea0 0x7fa10015dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.274+0000 7fa106400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1000b59e0 0x7fa100154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.274+0000 7fa106400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa10015faa0 0x7fa10013df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.274+0000 7fa106400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa1000710b0 con 0x7fa10015faa0 2024-04-25T14:03:48.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.274+0000 7fa106400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa100070f30 con 0x7fa1000aeea0 2024-04-25T14:03:48.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.274+0000 7fa106400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa10016e780 con 0x7fa1000b59e0 2024-04-25T14:03:48.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.274+0000 7fa0ffe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa10015faa0 0x7fa10013df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.274+0000 7fa0ffe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa10015faa0 0x7fa10013df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42114/0 (socket says 172.21.15.86:42114) 2024-04-25T14:03:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.274+0000 7fa0ffe00640 1 -- 172.21.15.86:0/84046991 learned_addr learned my addr 172.21.15.86:0/84046991 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.274+0000 7fa0ff400640 1 --2- 172.21.15.86:0/84046991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1000aeea0 0x7fa10015dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.274+0000 7fa0f6a00640 1 --2- 172.21.15.86:0/84046991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1000b59e0 0x7fa100154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.274+0000 7fa0ffe00640 1 -- 172.21.15.86:0/84046991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1000b59e0 msgr2=0x7fa100154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.274+0000 7fa0ffe00640 1 --2- 172.21.15.86:0/84046991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1000b59e0 0x7fa100154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.274+0000 7fa0ffe00640 1 -- 172.21.15.86:0/84046991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1000aeea0 msgr2=0x7fa10015dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:48.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.274+0000 7fa0ffe00640 1 --2- 172.21.15.86:0/84046991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1000aeea0 0x7fa10015dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.274+0000 7fa0ffe00640 1 -- 172.21.15.86:0/84046991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa100155cb0 con 0x7fa10015faa0 2024-04-25T14:03:48.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.274+0000 7fa0f6a00640 1 --2- 172.21.15.86:0/84046991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1000b59e0 0x7fa100154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:48.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.274+0000 7fa0ff400640 1 --2- 172.21.15.86:0/84046991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1000aeea0 0x7fa10015dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:48.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.274+0000 7fa0ffe00640 1 --2- 172.21.15.86:0/84046991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa10015faa0 0x7fa10013df40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa0f801a5a0 tx=0x7fa0f8019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:48.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.274+0000 7fa0f7e00640 1 -- 172.21.15.86:0/84046991 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f80b2040 con 0x7fa10015faa0 2024-04-25T14:03:48.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.274+0000 7fa0f7e00640 1 -- 172.21.15.86:0/84046991 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f80bf070 con 0x7fa10015faa0 2024-04-25T14:03:48.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.274+0000 7fa0f7e00640 1 -- 172.21.15.86:0/84046991 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f80adde0 con 0x7fa10015faa0 2024-04-25T14:03:48.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.274+0000 7fa106400640 1 -- 172.21.15.86:0/84046991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa100155f90 con 0x7fa10015faa0 2024-04-25T14:03:48.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.274+0000 7fa106400640 1 -- 172.21.15.86:0/84046991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa100156550 con 0x7fa10015faa0 2024-04-25T14:03:48.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.274+0000 7fa106400640 1 -- 172.21.15.86:0/84046991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa100074cf0 con 0x7fa10015faa0 2024-04-25T14:03:48.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.278+0000 7fa0f7e00640 1 -- 172.21.15.86:0/84046991 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa0f80b8470 con 0x7fa10015faa0 2024-04-25T14:03:48.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.278+0000 7fa0f7e00640 1 --2- 172.21.15.86:0/84046991 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa0dc06f400 0x7fa0dc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.278+0000 7fa0f7e00640 1 -- 172.21.15.86:0/84046991 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fa0f812a4b0 con 0x7fa10015faa0 2024-04-25T14:03:48.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.278+0000 7fa0ff400640 1 --2- 172.21.15.86:0/84046991 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa0dc06f400 0x7fa0dc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.278+0000 7fa0ff400640 1 --2- 172.21.15.86:0/84046991 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa0dc06f400 0x7fa0dc071910 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0000f70 tx=0x7fa0f0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:48.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.282+0000 7fa0f7e00640 1 -- 172.21.15.86:0/84046991 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa0f8016560 con 0x7fa10015faa0 2024-04-25T14:03:48.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.438+0000 7fa106400640 1 -- 172.21.15.86:0/84046991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa1000b55b0 con 0x7fa10015faa0 2024-04-25T14:03:48.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.442+0000 7fa0f7e00640 1 -- 172.21.15.86:0/84046991 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fa0f80f7770 con 0x7fa10015faa0 2024-04-25T14:03:48.445 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:03:48.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.442+0000 7fa106400640 1 -- 172.21.15.86:0/84046991 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa0dc06f400 msgr2=0x7fa0dc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:48.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.442+0000 7fa106400640 1 --2- 172.21.15.86:0/84046991 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa0dc06f400 0x7fa0dc071910 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0000f70 tx=0x7fa0f0009040 comp rx=0 tx=0).stop 2024-04-25T14:03:48.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.442+0000 7fa106400640 1 -- 172.21.15.86:0/84046991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa10015faa0 msgr2=0x7fa10013df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:48.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.442+0000 7fa106400640 1 --2- 172.21.15.86:0/84046991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa10015faa0 0x7fa10013df40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa0f801a5a0 tx=0x7fa0f8019710 comp rx=0 tx=0).stop 2024-04-25T14:03:48.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.442+0000 7fa106400640 1 -- 172.21.15.86:0/84046991 shutdown_connections 2024-04-25T14:03:48.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.442+0000 7fa106400640 1 --2- 172.21.15.86:0/84046991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa10015faa0 0x7fa10013df40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.442+0000 7fa106400640 1 --2- 172.21.15.86:0/84046991 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa0dc06f400 0x7fa0dc071910 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.442+0000 7fa106400640 1 --2- 172.21.15.86:0/84046991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1000b59e0 0x7fa100154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.442+0000 7fa106400640 1 --2- 172.21.15.86:0/84046991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1000aeea0 0x7fa10015dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.442+0000 7fa106400640 1 -- 172.21.15.86:0/84046991 >> 172.21.15.86:0/84046991 conn(0x7fa1000d2230 msgr2=0x7fa100070520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:48.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.442+0000 7fa106400640 1 -- 172.21.15.86:0/84046991 shutdown_connections 2024-04-25T14:03:48.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.442+0000 7fa106400640 1 -- 172.21.15.86:0/84046991 wait complete. 2024-04-25T14:03:48.466 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T14:03:48.582 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:03:48.585 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:03:48.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.610+0000 7f550d000640 1 Processor -- start 2024-04-25T14:03:48.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.610+0000 7f550d000640 1 -- start start 2024-04-25T14:03:48.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.610+0000 7f550d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f55080d08c0 0x7f550815c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.610+0000 7f550d000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55080ae130 0x7f55080ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.610+0000 7f550d000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f550806d120 con 0x7f550806e830 2024-04-25T14:03:48.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.610+0000 7f550d000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f550806d400 con 0x7f55080d08c0 2024-04-25T14:03:48.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.610+0000 7f550d000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f550806d6e0 con 0x7f55080ae130 2024-04-25T14:03:48.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.610+0000 7f5507400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f55080d08c0 0x7f550815c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.610+0000 7f5507400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f55080d08c0 0x7f550815c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42120/0 (socket says 172.21.15.86:42120) 2024-04-25T14:03:48.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.610+0000 7f5507400640 1 -- 172.21.15.86:0/2696609267 learned_addr learned my addr 172.21.15.86:0/2696609267 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:48.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.610+0000 7f5507e00640 1 --2- 172.21.15.86:0/2696609267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55080ae130 0x7f55080ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.610+0000 7f5507400640 1 -- 172.21.15.86:0/2696609267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55080ae130 msgr2=0x7f55080ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:48.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.610+0000 7f5507400640 1 --2- 172.21.15.86:0/2696609267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55080ae130 0x7f55080ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.610+0000 7f5507400640 1 -- 172.21.15.86:0/2696609267 >> v1:172.21.15.155:6789/0 conn(0x7f550806e830 legacy=0x7f550806ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:48.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.610+0000 7f5507400640 1 -- 172.21.15.86:0/2696609267 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550806d9c0 con 0x7f55080d08c0 2024-04-25T14:03:48.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.614+0000 7f5507400640 1 --2- 172.21.15.86:0/2696609267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55080d08c0 0x7f550815c990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f54f800bb10 tx=0x7f54f801ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d955a7d1ab276f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:48.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.614+0000 7f5506000640 1 -- 172.21.15.86:0/2696609267 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f80b4070 con 0x7f55080d08c0 2024-04-25T14:03:48.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.614+0000 7f5506000640 1 -- 172.21.15.86:0/2696609267 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54f80a7b00 con 0x7f55080d08c0 2024-04-25T14:03:48.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.614+0000 7f5506000640 1 -- 172.21.15.86:0/2696609267 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f80af440 con 0x7f55080d08c0 2024-04-25T14:03:48.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.614+0000 7f550d000640 1 -- 172.21.15.86:0/2696609267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55080d08c0 msgr2=0x7f550815c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:48.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.614+0000 7f550d000640 1 --2- 172.21.15.86:0/2696609267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55080d08c0 0x7f550815c990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f54f800bb10 tx=0x7f54f801ec30 comp rx=0 tx=0).stop 2024-04-25T14:03:48.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.614+0000 7f550d000640 1 -- 172.21.15.86:0/2696609267 shutdown_connections 2024-04-25T14:03:48.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.614+0000 7f550d000640 1 --2- 172.21.15.86:0/2696609267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55080ae130 0x7f55080ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.614+0000 7f550d000640 1 --2- 172.21.15.86:0/2696609267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55080d08c0 0x7f550815c990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.614+0000 7f550d000640 1 -- 172.21.15.86:0/2696609267 >> 172.21.15.86:0/2696609267 conn(0x7f55080b4850 msgr2=0x7f55080b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:48.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.614+0000 7f550d000640 1 -- 172.21.15.86:0/2696609267 shutdown_connections 2024-04-25T14:03:48.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.614+0000 7f550d000640 1 -- 172.21.15.86:0/2696609267 wait complete. 2024-04-25T14:03:48.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.618+0000 7f550d000640 1 Processor -- start 2024-04-25T14:03:48.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.618+0000 7f550d000640 1 -- start start 2024-04-25T14:03:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.618+0000 7f550d000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f550806e830 0x7f5508148380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.618+0000 7f550d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55080ae130 0x7f5508147640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.618+0000 7f550d000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55080d08c0 0x7f550814aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.618+0000 7f550d000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55080cdde0 con 0x7f55080ae130 2024-04-25T14:03:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.618+0000 7f550d000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f55080cdc60 con 0x7f550806e830 2024-04-25T14:03:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.618+0000 7f550d000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f55080cdf60 con 0x7f55080d08c0 2024-04-25T14:03:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.618+0000 7f5506a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55080ae130 0x7f5508147640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.618+0000 7f5506a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55080ae130 0x7f5508147640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42132/0 (socket says 172.21.15.86:42132) 2024-04-25T14:03:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.618+0000 7f5506a00640 1 -- 172.21.15.86:0/357422391 learned_addr learned my addr 172.21.15.86:0/357422391 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.618+0000 7f5507e00640 1 --2- 172.21.15.86:0/357422391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55080d08c0 0x7f550814aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.618+0000 7f5507400640 1 --2- 172.21.15.86:0/357422391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f550806e830 0x7f5508148380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.618+0000 7f5506a00640 1 -- 172.21.15.86:0/357422391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55080d08c0 msgr2=0x7f550814aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.618+0000 7f5506a00640 1 --2- 172.21.15.86:0/357422391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55080d08c0 0x7f550814aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.618+0000 7f5506a00640 1 -- 172.21.15.86:0/357422391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f550806e830 msgr2=0x7f5508148380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.618+0000 7f5506a00640 1 --2- 172.21.15.86:0/357422391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f550806e830 0x7f5508148380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.618+0000 7f5506a00640 1 -- 172.21.15.86:0/357422391 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5508148cf0 con 0x7f55080ae130 2024-04-25T14:03:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.622+0000 7f5506a00640 1 --2- 172.21.15.86:0/357422391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55080ae130 0x7f5508147640 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f54f400c3a0 tx=0x7f54f400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.622+0000 7f54ff400640 1 -- 172.21.15.86:0/357422391 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f40052d0 con 0x7f55080ae130 2024-04-25T14:03:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.622+0000 7f54ff400640 1 -- 172.21.15.86:0/357422391 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54f4005470 con 0x7f55080ae130 2024-04-25T14:03:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.622+0000 7f54ff400640 1 -- 172.21.15.86:0/357422391 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f4007260 con 0x7f55080ae130 2024-04-25T14:03:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.622+0000 7f550d000640 1 -- 172.21.15.86:0/357422391 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5508148fd0 con 0x7f55080ae130 2024-04-25T14:03:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.622+0000 7f550d000640 1 -- 172.21.15.86:0/357422391 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5508149590 con 0x7f55080ae130 2024-04-25T14:03:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.622+0000 7f550d000640 1 -- 172.21.15.86:0/357422391 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5508148a60 con 0x7f55080ae130 2024-04-25T14:03:48.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.622+0000 7f54ff400640 1 -- 172.21.15.86:0/357422391 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f54f4007400 con 0x7f55080ae130 2024-04-25T14:03:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.622+0000 7f54ff400640 1 --2- 172.21.15.86:0/357422391 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f54e806f400 0x7f54e8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.622+0000 7f54ff400640 1 -- 172.21.15.86:0/357422391 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f54f408e4c0 con 0x7f55080ae130 2024-04-25T14:03:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.622+0000 7f5507400640 1 --2- 172.21.15.86:0/357422391 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f54e806f400 0x7f54e8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.626+0000 7f5507400640 1 --2- 172.21.15.86:0/357422391 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f54e806f400 0x7f54e8071910 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f54f80b1000 tx=0x7f54f80a9070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:48.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.626+0000 7f54ff400640 1 -- 172.21.15.86:0/357422391 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f54f405b660 con 0x7f55080ae130 2024-04-25T14:03:48.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.786+0000 7f550d000640 1 -- 172.21.15.86:0/357422391 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5508076b80 con 0x7f55080ae130 2024-04-25T14:03:48.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.790+0000 7f54ff400640 1 -- 172.21.15.86:0/357422391 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f54f405af60 con 0x7f55080ae130 2024-04-25T14:03:48.793 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:03:48.793 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-04-25T14:03:48.793 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:03:48.793 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-04-25T14:03:48.793 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-04-25T14:03:48.794 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:03:48.794 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-04-25T14:03:48.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.790+0000 7f550d000640 1 -- 172.21.15.86:0/357422391 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f54e806f400 msgr2=0x7f54e8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:48.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.790+0000 7f550d000640 1 --2- 172.21.15.86:0/357422391 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f54e806f400 0x7f54e8071910 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f54f80b1000 tx=0x7f54f80a9070 comp rx=0 tx=0).stop 2024-04-25T14:03:48.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.790+0000 7f550d000640 1 -- 172.21.15.86:0/357422391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55080ae130 msgr2=0x7f5508147640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:48.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.790+0000 7f550d000640 1 --2- 172.21.15.86:0/357422391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55080ae130 0x7f5508147640 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f54f400c3a0 tx=0x7f54f400c860 comp rx=0 tx=0).stop 2024-04-25T14:03:48.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.790+0000 7f550d000640 1 -- 172.21.15.86:0/357422391 shutdown_connections 2024-04-25T14:03:48.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.790+0000 7f550d000640 1 --2- 172.21.15.86:0/357422391 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f54e806f400 0x7f54e8071910 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.790+0000 7f550d000640 1 --2- 172.21.15.86:0/357422391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55080d08c0 0x7f550814aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.790+0000 7f550d000640 1 --2- 172.21.15.86:0/357422391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55080ae130 0x7f5508147640 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.790+0000 7f550d000640 1 --2- 172.21.15.86:0/357422391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f550806e830 0x7f5508148380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.790+0000 7f550d000640 1 -- 172.21.15.86:0/357422391 >> 172.21.15.86:0/357422391 conn(0x7f55080b4850 msgr2=0x7f5508152460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:48.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.790+0000 7f550d000640 1 -- 172.21.15.86:0/357422391 shutdown_connections 2024-04-25T14:03:48.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.790+0000 7f550d000640 1 -- 172.21.15.86:0/357422391 wait complete. 2024-04-25T14:03:48.815 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T14:03:48.815 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T14:03:48.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.962+0000 7fe0b4600640 1 Processor -- start 2024-04-25T14:03:48.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.962+0000 7fe0b4600640 1 -- start start 2024-04-25T14:03:48.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.962+0000 7fe0b4600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe0ac06ec00 0x7fe0ac06f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.962+0000 7fe0b4600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe0ac06fa20 0x7fe0ac0ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.962+0000 7fe0b4600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0ac06d120 con 0x7fe0ac06fe30 2024-04-25T14:03:48.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.962+0000 7fe0b4600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe0ac1600c0 con 0x7fe0ac06ec00 2024-04-25T14:03:48.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.962+0000 7fe0b4600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe0ac06d3c0 con 0x7fe0ac06fa20 2024-04-25T14:03:48.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.962+0000 7fe0b1c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe0ac06ec00 0x7fe0ac06f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.962+0000 7fe0b2600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe0ac06fe30 0x7fe0ac0ae7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35986/0 (socket says 172.21.15.86:35986) 2024-04-25T14:03:48.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.962+0000 7fe0b2600640 1 -- 172.21.15.86:0/4141417043 learned_addr learned my addr 172.21.15.86:0/4141417043 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:03:48.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.962+0000 7fe0b1200640 1 --2- 172.21.15.86:0/4141417043 >> v2:172.21.15.155:3300/0 conn(0x7fe0ac06fa20 0x7fe0ac0ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.962+0000 7fe0b1c00640 1 -- 172.21.15.86:0/4141417043 >> v2:172.21.15.155:3300/0 conn(0x7fe0ac06fa20 msgr2=0x7fe0ac0ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:48.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.962+0000 7fe0b1c00640 1 --2- 172.21.15.86:0/4141417043 >> v2:172.21.15.155:3300/0 conn(0x7fe0ac06fa20 0x7fe0ac0ae2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.962+0000 7fe0b1c00640 1 -- 172.21.15.86:0/4141417043 >> v1:172.21.15.86:6789/0 conn(0x7fe0ac06fe30 legacy=0x7fe0ac0ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:48.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.962+0000 7fe0b1c00640 1 -- 172.21.15.86:0/4141417043 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0ac06d6a0 con 0x7fe0ac06ec00 2024-04-25T14:03:48.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.962+0000 7fe0abe00640 1 -- 172.21.15.86:0/4141417043 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3741876136 0 0) 0x7fe0ac06d120 con 0x7fe0ac06fe30 2024-04-25T14:03:48.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.966+0000 7fe0b1c00640 1 --2- 172.21.15.86:0/4141417043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0ac06ec00 0x7fe0ac06f4e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac070f00 tx=0x7fe0a001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f517a862f4052328 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:48.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.966+0000 7fe0abe00640 1 -- 172.21.15.86:0/4141417043 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a00ab610 con 0x7fe0ac06ec00 2024-04-25T14:03:48.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.966+0000 7fe0abe00640 1 -- 172.21.15.86:0/4141417043 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0a00b0070 con 0x7fe0ac06ec00 2024-04-25T14:03:48.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.966+0000 7fe0abe00640 1 -- 172.21.15.86:0/4141417043 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a00ab9d0 con 0x7fe0ac06ec00 2024-04-25T14:03:48.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.966+0000 7fe0b4600640 1 -- 172.21.15.86:0/4141417043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0ac06ec00 msgr2=0x7fe0ac06f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:48.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.966+0000 7fe0b4600640 1 --2- 172.21.15.86:0/4141417043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0ac06ec00 0x7fe0ac06f4e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac070f00 tx=0x7fe0a001c450 comp rx=0 tx=0).stop 2024-04-25T14:03:48.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.966+0000 7fe0b4600640 1 -- 172.21.15.86:0/4141417043 shutdown_connections 2024-04-25T14:03:48.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.966+0000 7fe0b4600640 1 --2- 172.21.15.86:0/4141417043 >> v2:172.21.15.155:3300/0 conn(0x7fe0ac06fa20 0x7fe0ac0ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.966+0000 7fe0b4600640 1 --2- 172.21.15.86:0/4141417043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0ac06ec00 0x7fe0ac06f4e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.966+0000 7fe0b4600640 1 -- 172.21.15.86:0/4141417043 >> 172.21.15.86:0/4141417043 conn(0x7fe0ac0b4a40 msgr2=0x7fe0ac0b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:48.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.966+0000 7fe0b4600640 1 -- 172.21.15.86:0/4141417043 shutdown_connections 2024-04-25T14:03:48.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.966+0000 7fe0b4600640 1 -- 172.21.15.86:0/4141417043 wait complete. 2024-04-25T14:03:48.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.970+0000 7fe0b4600640 1 Processor -- start 2024-04-25T14:03:48.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.970+0000 7fe0b4600640 1 -- start start 2024-04-25T14:03:48.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.970+0000 7fe0b4600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0ac06ec00 0x7fe0ac149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.970+0000 7fe0b4600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0ac06fa20 0x7fe0ac14ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.970+0000 7fe0b4600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0ac06fe30 0x7fe0ac1530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.970+0000 7fe0b4600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac06dea0 con 0x7fe0ac06fe30 2024-04-25T14:03:48.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.970+0000 7fe0b4600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac0af300 con 0x7fe0ac06fa20 2024-04-25T14:03:48.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.970+0000 7fe0b4600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe0ac1f9620 con 0x7fe0ac06ec00 2024-04-25T14:03:48.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.970+0000 7fe0b2600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0ac06fe30 0x7fe0ac1530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.970+0000 7fe0b2600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0ac06fe30 0x7fe0ac1530c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59318/0 (socket says 172.21.15.86:59318) 2024-04-25T14:03:48.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.970+0000 7fe0b2600640 1 -- 172.21.15.86:0/3931569088 learned_addr learned my addr 172.21.15.86:0/3931569088 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:48.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.970+0000 7fe0b1200640 1 --2- 172.21.15.86:0/3931569088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0ac06fa20 0x7fe0ac14ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.970+0000 7fe0b1c00640 1 --2- 172.21.15.86:0/3931569088 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0ac06ec00 0x7fe0ac149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.970+0000 7fe0b2600640 1 -- 172.21.15.86:0/3931569088 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0ac06ec00 msgr2=0x7fe0ac149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:48.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.970+0000 7fe0b2600640 1 --2- 172.21.15.86:0/3931569088 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0ac06ec00 0x7fe0ac149240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.970+0000 7fe0b2600640 1 -- 172.21.15.86:0/3931569088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0ac06fa20 msgr2=0x7fe0ac14ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:48.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.970+0000 7fe0b2600640 1 --2- 172.21.15.86:0/3931569088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0ac06fa20 0x7fe0ac14ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.970+0000 7fe0b2600640 1 -- 172.21.15.86:0/3931569088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0ac149780 con 0x7fe0ac06fe30 2024-04-25T14:03:48.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.970+0000 7fe0b2600640 1 --2- 172.21.15.86:0/3931569088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0ac06fe30 0x7fe0ac1530c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe0a400caf0 tx=0x7fe0a40021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:48.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.974+0000 7fe0a9600640 1 -- 172.21.15.86:0/3931569088 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a400ac00 con 0x7fe0ac06fe30 2024-04-25T14:03:48.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.974+0000 7fe0a9600640 1 -- 172.21.15.86:0/3931569088 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0a400ada0 con 0x7fe0ac06fe30 2024-04-25T14:03:48.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.974+0000 7fe0a9600640 1 -- 172.21.15.86:0/3931569088 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4012640 con 0x7fe0ac06fe30 2024-04-25T14:03:48.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.974+0000 7fe0b4600640 1 -- 172.21.15.86:0/3931569088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0ac1579e0 con 0x7fe0ac06fe30 2024-04-25T14:03:48.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.974+0000 7fe0b4600640 1 -- 172.21.15.86:0/3931569088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0ac157cc0 con 0x7fe0ac06fe30 2024-04-25T14:03:48.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.974+0000 7fe0b4600640 1 -- 172.21.15.86:0/3931569088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0ac1fc470 con 0x7fe0ac06fe30 2024-04-25T14:03:48.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.974+0000 7fe0a9600640 1 -- 172.21.15.86:0/3931569088 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe0a40127e0 con 0x7fe0ac06fe30 2024-04-25T14:03:48.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.974+0000 7fe0a9600640 1 --2- 172.21.15.86:0/3931569088 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe08c06f430 0x7fe08c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.974+0000 7fe0a9600640 1 -- 172.21.15.86:0/3931569088 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fe0a408f5b0 con 0x7fe0ac06fe30 2024-04-25T14:03:48.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.974+0000 7fe0b1c00640 1 --2- 172.21.15.86:0/3931569088 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe08c06f430 0x7fe08c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.978+0000 7fe0b1c00640 1 --2- 172.21.15.86:0/3931569088 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe08c06f430 0x7fe08c071940 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe0a00af000 tx=0x7fe0a00a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:48.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:48.978+0000 7fe0a9600640 1 -- 172.21.15.86:0/3931569088 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe0ac1fc470 con 0x7fe0ac06fe30 2024-04-25T14:03:49.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.138+0000 7fe0b4600640 1 -- 172.21.15.86:0/3931569088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe0ac050750 con 0x7fe0ac06fe30 2024-04-25T14:03:49.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.430+0000 7fe0a9600640 1 -- 172.21.15.86:0/3931569088 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7fe0a405c750 con 0x7fe0ac06fe30 2024-04-25T14:03:49.433 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-04-25T14:03:49.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.430+0000 7fe0b4600640 1 -- 172.21.15.86:0/3931569088 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe08c06f430 msgr2=0x7fe08c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:49.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.430+0000 7fe0b4600640 1 --2- 172.21.15.86:0/3931569088 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe08c06f430 0x7fe08c071940 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe0a00af000 tx=0x7fe0a00a7000 comp rx=0 tx=0).stop 2024-04-25T14:03:49.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.430+0000 7fe0b4600640 1 -- 172.21.15.86:0/3931569088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0ac06fe30 msgr2=0x7fe0ac1530c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:49.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.430+0000 7fe0b4600640 1 --2- 172.21.15.86:0/3931569088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0ac06fe30 0x7fe0ac1530c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe0a400caf0 tx=0x7fe0a40021d0 comp rx=0 tx=0).stop 2024-04-25T14:03:49.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.430+0000 7fe0b4600640 1 -- 172.21.15.86:0/3931569088 shutdown_connections 2024-04-25T14:03:49.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.430+0000 7fe0b4600640 1 --2- 172.21.15.86:0/3931569088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0ac06fe30 0x7fe0ac1530c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:49.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.430+0000 7fe0b4600640 1 --2- 172.21.15.86:0/3931569088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0ac06fa20 0x7fe0ac14ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:49.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.430+0000 7fe0b4600640 1 --2- 172.21.15.86:0/3931569088 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe08c06f430 0x7fe08c071940 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:49.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.430+0000 7fe0b4600640 1 --2- 172.21.15.86:0/3931569088 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0ac06ec00 0x7fe0ac149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:49.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.430+0000 7fe0b4600640 1 -- 172.21.15.86:0/3931569088 >> 172.21.15.86:0/3931569088 conn(0x7fe0ac0b4a40 msgr2=0x7fe0ac156d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:49.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.430+0000 7fe0b4600640 1 -- 172.21.15.86:0/3931569088 shutdown_connections 2024-04-25T14:03:49.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.430+0000 7fe0b4600640 1 -- 172.21.15.86:0/3931569088 wait complete. 2024-04-25T14:03:49.464 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-25T14:03:49.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.614+0000 7f01c5800640 1 Processor -- start 2024-04-25T14:03:49.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.614+0000 7f01c5800640 1 -- start start 2024-04-25T14:03:49.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.614+0000 7f01c5800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f01c0163980 0x7f01c0163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:49.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.614+0000 7f01c5800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f01c006e340 0x7f01c006e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:49.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.614+0000 7f01c5800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01c01648e0 con 0x7f01c0072630 2024-04-25T14:03:49.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.614+0000 7f01c5800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f01c00d1520 con 0x7f01c0163980 2024-04-25T14:03:49.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.614+0000 7f01c5800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f01c00d23e0 con 0x7f01c006e340 2024-04-25T14:03:49.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.614+0000 7f01b6000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f01c0163980 0x7f01c0163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:49.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.614+0000 7f01b6000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f01c0163980 0x7f01c0163d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59328/0 (socket says 172.21.15.86:59328) 2024-04-25T14:03:49.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.614+0000 7f01b6000640 1 -- 172.21.15.86:0/4128133268 learned_addr learned my addr 172.21.15.86:0/4128133268 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:49.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.614+0000 7f01b6000640 1 -- 172.21.15.86:0/4128133268 >> v2:172.21.15.155:3300/0 conn(0x7f01c006e340 msgr2=0x7f01c006e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:49.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.614+0000 7f01b6000640 1 --2- 172.21.15.86:0/4128133268 >> v2:172.21.15.155:3300/0 conn(0x7f01c006e340 0x7f01c006e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:49.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.614+0000 7f01b6000640 1 -- 172.21.15.86:0/4128133268 >> v1:172.21.15.155:6789/0 conn(0x7f01c0072630 legacy=0x7f01c0072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:49.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.614+0000 7f01b6000640 1 -- 172.21.15.86:0/4128133268 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c0164600 con 0x7f01c0163980 2024-04-25T14:03:49.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.618+0000 7f01b6000640 1 --2- 172.21.15.86:0/4128133268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01c0163980 0x7f01c0163d90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f01c00c6b40 tx=0x7f01a801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a99442b7e69b4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:49.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.618+0000 7f01be000640 1 -- 172.21.15.86:0/4128133268 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a80b0070 con 0x7f01c0163980 2024-04-25T14:03:49.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.618+0000 7f01be000640 1 -- 172.21.15.86:0/4128133268 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a80a4e50 con 0x7f01c0163980 2024-04-25T14:03:49.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.618+0000 7f01be000640 1 -- 172.21.15.86:0/4128133268 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a80ab820 con 0x7f01c0163980 2024-04-25T14:03:49.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.618+0000 7f01c5800640 1 -- 172.21.15.86:0/4128133268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01c0163980 msgr2=0x7f01c0163d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:49.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.618+0000 7f01c5800640 1 --2- 172.21.15.86:0/4128133268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01c0163980 0x7f01c0163d90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f01c00c6b40 tx=0x7f01a801c3c0 comp rx=0 tx=0).stop 2024-04-25T14:03:49.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.618+0000 7f01c5800640 1 -- 172.21.15.86:0/4128133268 shutdown_connections 2024-04-25T14:03:49.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.618+0000 7f01c5800640 1 --2- 172.21.15.86:0/4128133268 >> v2:172.21.15.155:3300/0 conn(0x7f01c006e340 0x7f01c006e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:49.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.618+0000 7f01c5800640 1 --2- 172.21.15.86:0/4128133268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01c0163980 0x7f01c0163d90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:49.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.618+0000 7f01c5800640 1 -- 172.21.15.86:0/4128133268 >> 172.21.15.86:0/4128133268 conn(0x7f01c00d3580 msgr2=0x7f01c00d3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:49.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.618+0000 7f01c5800640 1 -- 172.21.15.86:0/4128133268 shutdown_connections 2024-04-25T14:03:49.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.618+0000 7f01c5800640 1 -- 172.21.15.86:0/4128133268 wait complete. 2024-04-25T14:03:49.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.622+0000 7f01c5800640 1 Processor -- start 2024-04-25T14:03:49.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.622+0000 7f01c5800640 1 -- start start 2024-04-25T14:03:49.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.622+0000 7f01c5800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01c006e340 0x7f01c014c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:49.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.622+0000 7f01c5800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01c0072630 0x7f01c014cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:49.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.622+0000 7f01c5800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01c0163980 0x7f01c013afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:49.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.622+0000 7f01c5800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01c016d530 con 0x7f01c0163980 2024-04-25T14:03:49.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.622+0000 7f01c5800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f01c006f0a0 con 0x7f01c0072630 2024-04-25T14:03:49.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.622+0000 7f01c5800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f01c016d6b0 con 0x7f01c006e340 2024-04-25T14:03:49.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.622+0000 7f01bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01c0163980 0x7f01c013afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:49.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.622+0000 7f01bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01c0163980 0x7f01c013afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59334/0 (socket says 172.21.15.86:59334) 2024-04-25T14:03:49.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.622+0000 7f01bf400640 1 -- 172.21.15.86:0/1497001439 learned_addr learned my addr 172.21.15.86:0/1497001439 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:49.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.622+0000 7f01b6000640 1 --2- 172.21.15.86:0/1497001439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01c0072630 0x7f01c014cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:49.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.622+0000 7f01bea00640 1 --2- 172.21.15.86:0/1497001439 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01c006e340 0x7f01c014c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:49.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.622+0000 7f01bf400640 1 -- 172.21.15.86:0/1497001439 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01c006e340 msgr2=0x7f01c014c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:49.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.622+0000 7f01bf400640 1 --2- 172.21.15.86:0/1497001439 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01c006e340 0x7f01c014c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:49.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.622+0000 7f01bf400640 1 -- 172.21.15.86:0/1497001439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01c0072630 msgr2=0x7f01c014cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:49.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.622+0000 7f01bf400640 1 --2- 172.21.15.86:0/1497001439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01c0072630 0x7f01c014cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:49.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.622+0000 7f01bf400640 1 -- 172.21.15.86:0/1497001439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c014dc60 con 0x7f01c0163980 2024-04-25T14:03:49.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.622+0000 7f01bea00640 1 --2- 172.21.15.86:0/1497001439 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01c006e340 0x7f01c014c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:49.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.622+0000 7f01bf400640 1 --2- 172.21.15.86:0/1497001439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01c0163980 0x7f01c013afb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f01b800b410 tx=0x7f01b800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:49.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.622+0000 7f01b7400640 1 -- 172.21.15.86:0/1497001439 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b8015020 con 0x7f01c0163980 2024-04-25T14:03:49.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.622+0000 7f01b7400640 1 -- 172.21.15.86:0/1497001439 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01b8011850 con 0x7f01c0163980 2024-04-25T14:03:49.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.622+0000 7f01b7400640 1 -- 172.21.15.86:0/1497001439 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b8011c40 con 0x7f01c0163980 2024-04-25T14:03:49.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.622+0000 7f01c5800640 1 -- 172.21.15.86:0/1497001439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c014df40 con 0x7f01c0163980 2024-04-25T14:03:49.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.622+0000 7f01c5800640 1 -- 172.21.15.86:0/1497001439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c0152b30 con 0x7f01c0163980 2024-04-25T14:03:49.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.626+0000 7f01c5800640 1 -- 172.21.15.86:0/1497001439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01c014d9d0 con 0x7f01c0163980 2024-04-25T14:03:49.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.626+0000 7f01b7400640 1 -- 172.21.15.86:0/1497001439 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f01b8018020 con 0x7f01c0163980 2024-04-25T14:03:49.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.626+0000 7f01b7400640 1 --2- 172.21.15.86:0/1497001439 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f019c06f400 0x7f019c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:49.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.626+0000 7f01b7400640 1 -- 172.21.15.86:0/1497001439 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f01b808f440 con 0x7f01c0163980 2024-04-25T14:03:49.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.626+0000 7f01bea00640 1 --2- 172.21.15.86:0/1497001439 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f019c06f400 0x7f019c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:49.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.626+0000 7f01bea00640 1 --2- 172.21.15.86:0/1497001439 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f019c06f400 0x7f019c071910 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f01b000c420 tx=0x7f01b0009570 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:49.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.630+0000 7f01b7400640 1 -- 172.21.15.86:0/1497001439 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f01b801f020 con 0x7f01c0163980 2024-04-25T14:03:49.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:49.786+0000 7f01c5800640 1 -- 172.21.15.86:0/1497001439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f01c0072170 con 0x7f01c0163980 2024-04-25T14:03:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.430+0000 7f01b7400640 1 -- 172.21.15.86:0/1497001439 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) ==== 171+0+0 (secure 0 0 0) 0x7f01b805c680 con 0x7f01c0163980 2024-04-25T14:03:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-04-25T14:03:50.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.434+0000 7f01c5800640 1 -- 172.21.15.86:0/1497001439 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f019c06f400 msgr2=0x7f019c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:50.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.434+0000 7f01c5800640 1 --2- 172.21.15.86:0/1497001439 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f019c06f400 0x7f019c071910 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f01b000c420 tx=0x7f01b0009570 comp rx=0 tx=0).stop 2024-04-25T14:03:50.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.434+0000 7f01c5800640 1 -- 172.21.15.86:0/1497001439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01c0163980 msgr2=0x7f01c013afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:50.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.434+0000 7f01c5800640 1 --2- 172.21.15.86:0/1497001439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01c0163980 0x7f01c013afb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f01b800b410 tx=0x7f01b800b8d0 comp rx=0 tx=0).stop 2024-04-25T14:03:50.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.434+0000 7f01c5800640 1 -- 172.21.15.86:0/1497001439 shutdown_connections 2024-04-25T14:03:50.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.434+0000 7f01c5800640 1 --2- 172.21.15.86:0/1497001439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01c0163980 0x7f01c013afb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:50.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.434+0000 7f01c5800640 1 --2- 172.21.15.86:0/1497001439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01c0072630 0x7f01c014cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:50.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.434+0000 7f01c5800640 1 --2- 172.21.15.86:0/1497001439 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f019c06f400 0x7f019c071910 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:50.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.434+0000 7f01c5800640 1 --2- 172.21.15.86:0/1497001439 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01c006e340 0x7f01c014c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:50.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.434+0000 7f01c5800640 1 -- 172.21.15.86:0/1497001439 >> 172.21.15.86:0/1497001439 conn(0x7f01c00d3580 msgr2=0x7f01c006f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:50.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.434+0000 7f01c5800640 1 -- 172.21.15.86:0/1497001439 shutdown_connections 2024-04-25T14:03:50.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.434+0000 7f01c5800640 1 -- 172.21.15.86:0/1497001439 wait complete. 2024-04-25T14:03:50.486 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T14:03:50.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.634+0000 7f1212000640 1 Processor -- start 2024-04-25T14:03:50.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.634+0000 7f1212000640 1 -- start start 2024-04-25T14:03:50.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.638+0000 7f1212000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f120c071530 0x7f120c15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:50.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.638+0000 7f1212000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f120c1616e0 0x7f120c161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:50.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.638+0000 7f1212000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f120c0cd8d0 con 0x7f120c0d2c80 2024-04-25T14:03:50.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.638+0000 7f1212000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f120c0ce720 con 0x7f120c071530 2024-04-25T14:03:50.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.638+0000 7f1212000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f120c0ce970 con 0x7f120c1616e0 2024-04-25T14:03:50.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.638+0000 7f1203e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f120c071530 0x7f120c15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:50.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.638+0000 7f1203e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f120c071530 0x7f120c15dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59342/0 (socket says 172.21.15.86:59342) 2024-04-25T14:03:50.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.638+0000 7f1203e00640 1 -- 172.21.15.86:0/1588469444 learned_addr learned my addr 172.21.15.86:0/1588469444 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:50.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.638+0000 7f120be00640 1 --2- 172.21.15.86:0/1588469444 >> v2:172.21.15.155:3300/0 conn(0x7f120c1616e0 0x7f120c161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:50.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.638+0000 7f1203e00640 1 -- 172.21.15.86:0/1588469444 >> v2:172.21.15.155:3300/0 conn(0x7f120c1616e0 msgr2=0x7f120c161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:50.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.638+0000 7f1203e00640 1 --2- 172.21.15.86:0/1588469444 >> v2:172.21.15.155:3300/0 conn(0x7f120c1616e0 0x7f120c161af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:50.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.638+0000 7f1203e00640 1 -- 172.21.15.86:0/1588469444 >> v1:172.21.15.155:6789/0 conn(0x7f120c0d2c80 legacy=0x7f120c070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:50.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.638+0000 7f1203e00640 1 -- 172.21.15.86:0/1588469444 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120c0d1be0 con 0x7f120c071530 2024-04-25T14:03:50.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.642+0000 7f1203e00640 1 --2- 172.21.15.86:0/1588469444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f120c071530 0x7f120c15dc90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f120c074af0 tx=0x7f11f401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c8c454a4802ef09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:50.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.642+0000 7f120aa00640 1 -- 172.21.15.86:0/1588469444 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f40b5070 con 0x7f120c071530 2024-04-25T14:03:50.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.642+0000 7f120aa00640 1 -- 172.21.15.86:0/1588469444 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11f40a8a10 con 0x7f120c071530 2024-04-25T14:03:50.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.642+0000 7f120aa00640 1 -- 172.21.15.86:0/1588469444 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f40a8de0 con 0x7f120c071530 2024-04-25T14:03:50.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.642+0000 7f1212000640 1 -- 172.21.15.86:0/1588469444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f120c071530 msgr2=0x7f120c15dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:50.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.642+0000 7f1212000640 1 --2- 172.21.15.86:0/1588469444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f120c071530 0x7f120c15dc90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f120c074af0 tx=0x7f11f401ed10 comp rx=0 tx=0).stop 2024-04-25T14:03:50.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.642+0000 7f1212000640 1 -- 172.21.15.86:0/1588469444 shutdown_connections 2024-04-25T14:03:50.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.642+0000 7f1212000640 1 --2- 172.21.15.86:0/1588469444 >> v2:172.21.15.155:3300/0 conn(0x7f120c1616e0 0x7f120c161af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:50.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.642+0000 7f1212000640 1 --2- 172.21.15.86:0/1588469444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f120c071530 0x7f120c15dc90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:50.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.642+0000 7f1212000640 1 -- 172.21.15.86:0/1588469444 >> 172.21.15.86:0/1588469444 conn(0x7f120c0cffc0 msgr2=0x7f120c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:50.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.642+0000 7f1212000640 1 -- 172.21.15.86:0/1588469444 shutdown_connections 2024-04-25T14:03:50.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.642+0000 7f1212000640 1 -- 172.21.15.86:0/1588469444 wait complete. 2024-04-25T14:03:50.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.646+0000 7f1212000640 1 Processor -- start 2024-04-25T14:03:50.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.646+0000 7f1212000640 1 -- start start 2024-04-25T14:03:50.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.646+0000 7f1212000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f120c0d2c80 0x7f120c15e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:50.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.646+0000 7f1212000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f120c1616e0 0x7f120c15ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:50.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.646+0000 7f1212000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f120c15ef70 0x7f120c0b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:50.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.646+0000 7f1212000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f120c0d2480 con 0x7f120c15ef70 2024-04-25T14:03:50.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.646+0000 7f1212000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f120c0d21a0 con 0x7f120c0d2c80 2024-04-25T14:03:50.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.646+0000 7f1212000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f120c0d29a0 con 0x7f120c1616e0 2024-04-25T14:03:50.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.646+0000 7f120b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f120c0d2c80 0x7f120c15e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:50.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.646+0000 7f1203e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f120c1616e0 0x7f120c15ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:50.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.646+0000 7f120be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f120c15ef70 0x7f120c0b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:50.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.646+0000 7f120b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f120c0d2c80 0x7f120c15e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:35016/0 (socket says 172.21.15.86:35016) 2024-04-25T14:03:50.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.646+0000 7f120b400640 1 -- 172.21.15.86:0/1115348559 learned_addr learned my addr 172.21.15.86:0/1115348559 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:50.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.646+0000 7f1203e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f120c1616e0 0x7f120c15ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:47116/0 (socket says 172.21.15.86:47116) 2024-04-25T14:03:50.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.646+0000 7f1203e00640 1 -- 172.21.15.86:0/1115348559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f120c0d2c80 msgr2=0x7f120c15e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:50.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.646+0000 7f1203e00640 1 --2- 172.21.15.86:0/1115348559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f120c0d2c80 0x7f120c15e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:50.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.646+0000 7f1203e00640 1 -- 172.21.15.86:0/1115348559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f120c15ef70 msgr2=0x7f120c0b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:50.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.646+0000 7f1203e00640 1 --2- 172.21.15.86:0/1115348559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f120c15ef70 0x7f120c0b0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:50.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.646+0000 7f1203e00640 1 -- 172.21.15.86:0/1115348559 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120c15f380 con 0x7f120c1616e0 2024-04-25T14:03:50.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.646+0000 7f1203e00640 1 --2- 172.21.15.86:0/1115348559 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f120c1616e0 0x7f120c15ea30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f11f40af000 tx=0x7f11f401bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:50.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.646+0000 7f1203400640 1 -- 172.21.15.86:0/1115348559 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f40b5040 con 0x7f120c1616e0 2024-04-25T14:03:50.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.646+0000 7f1203400640 1 -- 172.21.15.86:0/1115348559 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11f4004340 con 0x7f120c1616e0 2024-04-25T14:03:50.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.646+0000 7f1212000640 1 -- 172.21.15.86:0/1115348559 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f120c0b4c50 con 0x7f120c1616e0 2024-04-25T14:03:50.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.650+0000 7f1203400640 1 -- 172.21.15.86:0/1115348559 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f40044f0 con 0x7f120c1616e0 2024-04-25T14:03:50.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.650+0000 7f1212000640 1 -- 172.21.15.86:0/1115348559 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f120c0b5210 con 0x7f120c1616e0 2024-04-25T14:03:50.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.650+0000 7f1212000640 1 -- 172.21.15.86:0/1115348559 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f120c0b58d0 con 0x7f120c1616e0 2024-04-25T14:03:50.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.650+0000 7f1203400640 1 -- 172.21.15.86:0/1115348559 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f11f4003860 con 0x7f120c1616e0 2024-04-25T14:03:50.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.650+0000 7f1203400640 1 --2- 172.21.15.86:0/1115348559 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f11e806f400 0x7f11e8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:50.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.650+0000 7f1203400640 1 -- 172.21.15.86:0/1115348559 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f11f412bc00 con 0x7f120c1616e0 2024-04-25T14:03:50.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.650+0000 7f120b400640 1 --2- 172.21.15.86:0/1115348559 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f11e806f400 0x7f11e8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:50.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.654+0000 7f120b400640 1 --2- 172.21.15.86:0/1115348559 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f11e806f400 0x7f11e8071910 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f11fc007fd0 tx=0x7f11fc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:50.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.658+0000 7f1203400640 1 -- 172.21.15.86:0/1115348559 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f120c0b58d0 con 0x7f120c1616e0 2024-04-25T14:03:50.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:50.850+0000 7f1212000640 1 -- 172.21.15.86:0/1115348559 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f120c0720e0 con 0x7f120c1616e0 2024-04-25T14:03:51.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.474+0000 7f1203400640 1 -- 172.21.15.86:0/1115348559 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f11f40bf060 con 0x7f120c1616e0 2024-04-25T14:03:51.476 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T14:03:51.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.478+0000 7f1212000640 1 -- 172.21.15.86:0/1115348559 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f11e806f400 msgr2=0x7f11e8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:51.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.478+0000 7f1212000640 1 --2- 172.21.15.86:0/1115348559 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f11e806f400 0x7f11e8071910 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f11fc007fd0 tx=0x7f11fc009040 comp rx=0 tx=0).stop 2024-04-25T14:03:51.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.478+0000 7f1212000640 1 -- 172.21.15.86:0/1115348559 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f120c1616e0 msgr2=0x7f120c15ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:51.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.478+0000 7f1212000640 1 --2- 172.21.15.86:0/1115348559 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f120c1616e0 0x7f120c15ea30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f11f40af000 tx=0x7f11f401bab0 comp rx=0 tx=0).stop 2024-04-25T14:03:51.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.478+0000 7f1212000640 1 -- 172.21.15.86:0/1115348559 shutdown_connections 2024-04-25T14:03:51.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.478+0000 7f1212000640 1 --2- 172.21.15.86:0/1115348559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f120c15ef70 0x7f120c0b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:51.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.478+0000 7f1212000640 1 --2- 172.21.15.86:0/1115348559 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f11e806f400 0x7f11e8071910 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:51.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.478+0000 7f1212000640 1 --2- 172.21.15.86:0/1115348559 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f120c1616e0 0x7f120c15ea30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:51.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.478+0000 7f1212000640 1 --2- 172.21.15.86:0/1115348559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f120c0d2c80 0x7f120c15e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:51.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.478+0000 7f1212000640 1 -- 172.21.15.86:0/1115348559 >> 172.21.15.86:0/1115348559 conn(0x7f120c0cffc0 msgr2=0x7f120c15aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:51.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.478+0000 7f1212000640 1 -- 172.21.15.86:0/1115348559 shutdown_connections 2024-04-25T14:03:51.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.482+0000 7f1212000640 1 -- 172.21.15.86:0/1115348559 wait complete. 2024-04-25T14:03:51.510 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-25T14:03:51.511 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-25T14:03:51.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.662+0000 7fd27da00640 1 Processor -- start 2024-04-25T14:03:51.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.662+0000 7fd27da00640 1 -- start start 2024-04-25T14:03:51.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.666+0000 7fd27da00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd2780700a0 0x7fd2780b6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:51.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.666+0000 7fd27da00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd27815dbd0 0x7fd2781617b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:51.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.666+0000 7fd27da00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd27815dfe0 con 0x7fd2780d25c0 2024-04-25T14:03:51.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.666+0000 7fd27da00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd2780ce4e0 con 0x7fd2780700a0 2024-04-25T14:03:51.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.666+0000 7fd27da00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd2780ce730 con 0x7fd27815dbd0 2024-04-25T14:03:51.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.666+0000 7fd26fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd2780700a0 0x7fd2780b6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:51.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.666+0000 7fd26fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd2780700a0 0x7fd2780b6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59358/0 (socket says 172.21.15.86:59358) 2024-04-25T14:03:51.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.666+0000 7fd26fe00640 1 -- 172.21.15.86:0/3024749205 learned_addr learned my addr 172.21.15.86:0/3024749205 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:51.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.666+0000 7fd277400640 1 --2- 172.21.15.86:0/3024749205 >> v2:172.21.15.155:3300/0 conn(0x7fd27815dbd0 0x7fd2781617b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:51.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.666+0000 7fd26fe00640 1 -- 172.21.15.86:0/3024749205 >> v2:172.21.15.155:3300/0 conn(0x7fd27815dbd0 msgr2=0x7fd2781617b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:51.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.666+0000 7fd26fe00640 1 --2- 172.21.15.86:0/3024749205 >> v2:172.21.15.155:3300/0 conn(0x7fd27815dbd0 0x7fd2781617b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:51.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.666+0000 7fd26fe00640 1 -- 172.21.15.86:0/3024749205 >> v1:172.21.15.86:6789/0 conn(0x7fd2780d25c0 legacy=0x7fd27806f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:51.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.666+0000 7fd26fe00640 1 -- 172.21.15.86:0/3024749205 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2780cdd80 con 0x7fd2780700a0 2024-04-25T14:03:51.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.666+0000 7fd26fe00640 1 --2- 172.21.15.86:0/3024749205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2780700a0 0x7fd2780b6510 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd2780b9640 tx=0x7fd26001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da858d7982251a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:51.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.670+0000 7fd276000640 1 -- 172.21.15.86:0/3024749205 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2600b1070 con 0x7fd2780700a0 2024-04-25T14:03:51.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.670+0000 7fd276000640 1 -- 172.21.15.86:0/3024749205 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2600ac600 con 0x7fd2780700a0 2024-04-25T14:03:51.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.670+0000 7fd276000640 1 -- 172.21.15.86:0/3024749205 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2600ac9d0 con 0x7fd2780700a0 2024-04-25T14:03:51.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.670+0000 7fd27da00640 1 -- 172.21.15.86:0/3024749205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2780700a0 msgr2=0x7fd2780b6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:51.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.670+0000 7fd27da00640 1 --2- 172.21.15.86:0/3024749205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2780700a0 0x7fd2780b6510 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd2780b9640 tx=0x7fd26001c520 comp rx=0 tx=0).stop 2024-04-25T14:03:51.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.670+0000 7fd27da00640 1 -- 172.21.15.86:0/3024749205 shutdown_connections 2024-04-25T14:03:51.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.670+0000 7fd27da00640 1 --2- 172.21.15.86:0/3024749205 >> v2:172.21.15.155:3300/0 conn(0x7fd27815dbd0 0x7fd2781617b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:51.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.670+0000 7fd27da00640 1 --2- 172.21.15.86:0/3024749205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2780700a0 0x7fd2780b6510 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:51.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.670+0000 7fd27da00640 1 -- 172.21.15.86:0/3024749205 >> 172.21.15.86:0/3024749205 conn(0x7fd2780b5060 msgr2=0x7fd2780b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:51.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.670+0000 7fd27da00640 1 -- 172.21.15.86:0/3024749205 shutdown_connections 2024-04-25T14:03:51.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.670+0000 7fd27da00640 1 -- 172.21.15.86:0/3024749205 wait complete. 2024-04-25T14:03:51.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.670+0000 7fd27da00640 1 Processor -- start 2024-04-25T14:03:51.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.670+0000 7fd27da00640 1 -- start start 2024-04-25T14:03:51.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.674+0000 7fd27da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2780d25c0 0x7fd278152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:51.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.674+0000 7fd27da00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd27815dbd0 0x7fd278159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:51.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.674+0000 7fd27da00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd278159660 0x7fd278159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:51.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.674+0000 7fd27da00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd27816abb0 con 0x7fd2780d25c0 2024-04-25T14:03:51.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.674+0000 7fd27da00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd27816aa30 con 0x7fd278159660 2024-04-25T14:03:51.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.674+0000 7fd27da00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd27816ad30 con 0x7fd27815dbd0 2024-04-25T14:03:51.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.674+0000 7fd276a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2780d25c0 0x7fd278152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:51.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.674+0000 7fd276a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2780d25c0 0x7fd278152740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59362/0 (socket says 172.21.15.86:59362) 2024-04-25T14:03:51.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.674+0000 7fd276a00640 1 -- 172.21.15.86:0/1322638799 learned_addr learned my addr 172.21.15.86:0/1322638799 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:51.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.674+0000 7fd26fe00640 1 --2- 172.21.15.86:0/1322638799 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd27815dbd0 0x7fd278159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:51.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.674+0000 7fd277400640 1 --2- 172.21.15.86:0/1322638799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd278159660 0x7fd278159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:51.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.674+0000 7fd276a00640 1 -- 172.21.15.86:0/1322638799 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd27815dbd0 msgr2=0x7fd278159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:51.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.674+0000 7fd276a00640 1 --2- 172.21.15.86:0/1322638799 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd27815dbd0 0x7fd278159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:51.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.674+0000 7fd276a00640 1 -- 172.21.15.86:0/1322638799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd278159660 msgr2=0x7fd278159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:51.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.674+0000 7fd276a00640 1 --2- 172.21.15.86:0/1322638799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd278159660 0x7fd278159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:51.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.674+0000 7fd276a00640 1 -- 172.21.15.86:0/1322638799 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd278159fb0 con 0x7fd2780d25c0 2024-04-25T14:03:51.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.674+0000 7fd277400640 1 --2- 172.21.15.86:0/1322638799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd278159660 0x7fd278159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:51.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.674+0000 7fd276a00640 1 --2- 172.21.15.86:0/1322638799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2780d25c0 0x7fd278152740 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd26800c3c0 tx=0x7fd26800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:51.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.674+0000 7fd26ea00640 1 -- 172.21.15.86:0/1322638799 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd268017070 con 0x7fd2780d25c0 2024-04-25T14:03:51.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.674+0000 7fd26ea00640 1 -- 172.21.15.86:0/1322638799 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2680052f0 con 0x7fd2780d25c0 2024-04-25T14:03:51.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.674+0000 7fd27da00640 1 -- 172.21.15.86:0/1322638799 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2780cb940 con 0x7fd2780d25c0 2024-04-25T14:03:51.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.674+0000 7fd26ea00640 1 -- 172.21.15.86:0/1322638799 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd268003840 con 0x7fd2780d25c0 2024-04-25T14:03:51.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.674+0000 7fd27da00640 1 -- 172.21.15.86:0/1322638799 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd278158920 con 0x7fd2780d25c0 2024-04-25T14:03:51.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.674+0000 7fd27da00640 1 -- 172.21.15.86:0/1322638799 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd278050750 con 0x7fd2780d25c0 2024-04-25T14:03:51.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.678+0000 7fd26ea00640 1 -- 172.21.15.86:0/1322638799 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd26801a070 con 0x7fd2780d25c0 2024-04-25T14:03:51.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.678+0000 7fd26ea00640 1 --2- 172.21.15.86:0/1322638799 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd25406f430 0x7fd254071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:51.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.678+0000 7fd26ea00640 1 -- 172.21.15.86:0/1322638799 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fd26808e950 con 0x7fd2780d25c0 2024-04-25T14:03:51.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.678+0000 7fd26fe00640 1 --2- 172.21.15.86:0/1322638799 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd25406f430 0x7fd254071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:51.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.678+0000 7fd26fe00640 1 --2- 172.21.15.86:0/1322638799 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd25406f430 0x7fd254071940 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd2600ae000 tx=0x7fd260008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:51.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.682+0000 7fd26ea00640 1 -- 172.21.15.86:0/1322638799 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd268021020 con 0x7fd2780d25c0 2024-04-25T14:03:51.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:51.846+0000 7fd27da00640 1 -- 172.21.15.86:0/1322638799 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fd2780d1440 con 0x7fd2780d25c0 2024-04-25T14:03:52.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.450+0000 7fd26ea00640 1 -- 172.21.15.86:0/1322638799 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) ==== 157+0+0 (secure 0 0 0) 0x7fd26805b8b0 con 0x7fd2780d25c0 2024-04-25T14:03:52.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.454+0000 7fd27da00640 1 -- 172.21.15.86:0/1322638799 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd25406f430 msgr2=0x7fd254071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:52.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.454+0000 7fd27da00640 1 --2- 172.21.15.86:0/1322638799 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd25406f430 0x7fd254071940 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd2600ae000 tx=0x7fd260008040 comp rx=0 tx=0).stop 2024-04-25T14:03:52.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.454+0000 7fd27da00640 1 -- 172.21.15.86:0/1322638799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2780d25c0 msgr2=0x7fd278152740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:52.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.454+0000 7fd27da00640 1 --2- 172.21.15.86:0/1322638799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2780d25c0 0x7fd278152740 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd26800c3c0 tx=0x7fd26800c880 comp rx=0 tx=0).stop 2024-04-25T14:03:52.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.458+0000 7fd27da00640 1 -- 172.21.15.86:0/1322638799 shutdown_connections 2024-04-25T14:03:52.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.458+0000 7fd27da00640 1 --2- 172.21.15.86:0/1322638799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd278159660 0x7fd278159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:52.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.458+0000 7fd27da00640 1 --2- 172.21.15.86:0/1322638799 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd25406f430 0x7fd254071940 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:52.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.458+0000 7fd27da00640 1 --2- 172.21.15.86:0/1322638799 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd27815dbd0 0x7fd278159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:52.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.458+0000 7fd27da00640 1 --2- 172.21.15.86:0/1322638799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2780d25c0 0x7fd278152740 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd26800c3c0 tx=0x7fd26800c880 comp rx=0 tx=0).stop 2024-04-25T14:03:52.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.458+0000 7fd27da00640 1 -- 172.21.15.86:0/1322638799 >> 172.21.15.86:0/1322638799 conn(0x7fd2780b5060 msgr2=0x7fd2780700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:52.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.458+0000 7fd27da00640 1 -- 172.21.15.86:0/1322638799 shutdown_connections 2024-04-25T14:03:52.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.458+0000 7fd27da00640 1 -- 172.21.15.86:0/1322638799 wait complete. 2024-04-25T14:03:52.482 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T14:03:52.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.638+0000 7f9b2ae00640 1 Processor -- start 2024-04-25T14:03:52.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.642+0000 7f9b2ae00640 1 -- start start 2024-04-25T14:03:52.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.642+0000 7f9b2ae00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b240cfca0 0x7f9b2406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:52.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.642+0000 7f9b2ae00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9b240730b0 0x7f9b240734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:52.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.642+0000 7f9b2ae00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b24072bd0 con 0x7f9b240d16e0 2024-04-25T14:03:52.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.642+0000 7f9b2ae00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9b24070470 con 0x7f9b240730b0 2024-04-25T14:03:52.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.642+0000 7f9b2ae00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9b2406f930 con 0x7f9b240cfca0 2024-04-25T14:03:52.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.642+0000 7f9b28e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9b240730b0 0x7f9b240734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:52.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.642+0000 7f9b28e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9b240730b0 0x7f9b240734c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:35040/0 (socket says 172.21.15.86:35040) 2024-04-25T14:03:52.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.642+0000 7f9b28e00640 1 -- 172.21.15.86:0/4078141371 learned_addr learned my addr 172.21.15.86:0/4078141371 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:52.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.642+0000 7f9b23e00640 1 --2- 172.21.15.86:0/4078141371 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b240cfca0 0x7f9b2406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:52.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.642+0000 7f9b28e00640 1 -- 172.21.15.86:0/4078141371 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b240cfca0 msgr2=0x7f9b2406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:52.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.642+0000 7f9b28e00640 1 --2- 172.21.15.86:0/4078141371 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b240cfca0 0x7f9b2406f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:52.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.642+0000 7f9b28e00640 1 -- 172.21.15.86:0/4078141371 >> v1:172.21.15.155:6789/0 conn(0x7f9b240d16e0 legacy=0x7f9b240d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:52.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.642+0000 7f9b28e00640 1 -- 172.21.15.86:0/4078141371 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b2406fbd0 con 0x7f9b240730b0 2024-04-25T14:03:52.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.642+0000 7f9b23e00640 1 --2- 172.21.15.86:0/4078141371 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b240cfca0 0x7f9b2406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:52.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.646+0000 7f9b28e00640 1 --2- 172.21.15.86:0/4078141371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b240730b0 0x7f9b240734c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c009e50 tx=0x7f9b1c01ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ed32ffa9a9ac7d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:52.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.646+0000 7f9b23400640 1 -- 172.21.15.86:0/4078141371 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c0a5810 con 0x7f9b240730b0 2024-04-25T14:03:52.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.646+0000 7f9b23400640 1 -- 172.21.15.86:0/4078141371 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b1c0a59b0 con 0x7f9b240730b0 2024-04-25T14:03:52.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.646+0000 7f9b23400640 1 -- 172.21.15.86:0/4078141371 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c0a5d80 con 0x7f9b240730b0 2024-04-25T14:03:52.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.646+0000 7f9b2ae00640 1 -- 172.21.15.86:0/4078141371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b240730b0 msgr2=0x7f9b240734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:52.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.646+0000 7f9b2ae00640 1 --2- 172.21.15.86:0/4078141371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b240730b0 0x7f9b240734c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c009e50 tx=0x7f9b1c01ca30 comp rx=0 tx=0).stop 2024-04-25T14:03:52.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.646+0000 7f9b2ae00640 1 -- 172.21.15.86:0/4078141371 shutdown_connections 2024-04-25T14:03:52.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.646+0000 7f9b2ae00640 1 --2- 172.21.15.86:0/4078141371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b240730b0 0x7f9b240734c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:52.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.646+0000 7f9b2ae00640 1 --2- 172.21.15.86:0/4078141371 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b240cfca0 0x7f9b2406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:52.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.646+0000 7f9b2ae00640 1 -- 172.21.15.86:0/4078141371 >> 172.21.15.86:0/4078141371 conn(0x7f9b240b4650 msgr2=0x7f9b240b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:52.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.646+0000 7f9b2ae00640 1 -- 172.21.15.86:0/4078141371 shutdown_connections 2024-04-25T14:03:52.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.646+0000 7f9b2ae00640 1 -- 172.21.15.86:0/4078141371 wait complete. 2024-04-25T14:03:52.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.650+0000 7f9b2ae00640 1 Processor -- start 2024-04-25T14:03:52.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.650+0000 7f9b2ae00640 1 -- start start 2024-04-25T14:03:52.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.650+0000 7f9b2ae00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b240730b0 0x7f9b240ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:52.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.650+0000 7f9b2ae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b240cfca0 0x7f9b240c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:52.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.650+0000 7f9b2ae00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b240d16e0 0x7f9b240cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:52.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.650+0000 7f9b2ae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b24169a70 con 0x7f9b240cfca0 2024-04-25T14:03:52.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.650+0000 7f9b2ae00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9b241698f0 con 0x7f9b240730b0 2024-04-25T14:03:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.650+0000 7f9b2ae00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9b24169bf0 con 0x7f9b240d16e0 2024-04-25T14:03:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.650+0000 7f9b1b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b240cfca0 0x7f9b240c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.650+0000 7f9b1b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b240cfca0 0x7f9b240c3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59368/0 (socket says 172.21.15.86:59368) 2024-04-25T14:03:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.650+0000 7f9b1b400640 1 -- 172.21.15.86:0/589646649 learned_addr learned my addr 172.21.15.86:0/589646649 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.650+0000 7f9b28e00640 1 --2- 172.21.15.86:0/589646649 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b240d16e0 0x7f9b240cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.650+0000 7f9b23e00640 1 --2- 172.21.15.86:0/589646649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b240730b0 0x7f9b240ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.650+0000 7f9b1b400640 1 -- 172.21.15.86:0/589646649 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b240d16e0 msgr2=0x7f9b240cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.650+0000 7f9b1b400640 1 --2- 172.21.15.86:0/589646649 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b240d16e0 0x7f9b240cc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.650+0000 7f9b1b400640 1 -- 172.21.15.86:0/589646649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b240730b0 msgr2=0x7f9b240ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.650+0000 7f9b1b400640 1 --2- 172.21.15.86:0/589646649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b240730b0 0x7f9b240ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.650+0000 7f9b1b400640 1 -- 172.21.15.86:0/589646649 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b240c4460 con 0x7f9b240cfca0 2024-04-25T14:03:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.650+0000 7f9b1b400640 1 --2- 172.21.15.86:0/589646649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b240cfca0 0x7f9b240c3f20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c00c370 tx=0x7f9b0c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.650+0000 7f9b20c00640 1 -- 172.21.15.86:0/589646649 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c0052d0 con 0x7f9b240cfca0 2024-04-25T14:03:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.650+0000 7f9b20c00640 1 -- 172.21.15.86:0/589646649 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b0c005470 con 0x7f9b240cfca0 2024-04-25T14:03:52.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.650+0000 7f9b2ae00640 1 -- 172.21.15.86:0/589646649 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b24157f40 con 0x7f9b240cfca0 2024-04-25T14:03:52.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.650+0000 7f9b20c00640 1 -- 172.21.15.86:0/589646649 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c007260 con 0x7f9b240cfca0 2024-04-25T14:03:52.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.650+0000 7f9b2ae00640 1 -- 172.21.15.86:0/589646649 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b24158500 con 0x7f9b240cfca0 2024-04-25T14:03:52.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.654+0000 7f9b2ae00640 1 -- 172.21.15.86:0/589646649 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b241fa7d0 con 0x7f9b240cfca0 2024-04-25T14:03:52.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.654+0000 7f9b20c00640 1 -- 172.21.15.86:0/589646649 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9b0c007400 con 0x7f9b240cfca0 2024-04-25T14:03:52.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.654+0000 7f9b20c00640 1 --2- 172.21.15.86:0/589646649 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9b0006f400 0x7f9b00071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:52.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.654+0000 7f9b20c00640 1 -- 172.21.15.86:0/589646649 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 6007+0+0 (secure 0 0 0) 0x7f9b0c08e740 con 0x7f9b240cfca0 2024-04-25T14:03:52.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.654+0000 7f9b23e00640 1 --2- 172.21.15.86:0/589646649 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9b0006f400 0x7f9b00071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:52.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.654+0000 7f9b23e00640 1 --2- 172.21.15.86:0/589646649 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9b0006f400 0x7f9b00071910 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9b14009580 tx=0x7f9b14009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:52.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.658+0000 7f9b20c00640 1 -- 172.21.15.86:0/589646649 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9b0c05af40 con 0x7f9b240cfca0 2024-04-25T14:03:52.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:52.818+0000 7f9b2ae00640 1 -- 172.21.15.86:0/589646649 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9b2409f4b0 con 0x7f9b240cfca0 2024-04-25T14:03:54.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.482+0000 7f9b20c00640 1 -- 172.21.15.86:0/589646649 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) ==== 228+0+0 (secure 0 0 0) 0x7f9b0c05f240 con 0x7f9b240cfca0 2024-04-25T14:03:54.481 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data_ec' created 2024-04-25T14:03:54.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.486+0000 7f9b2ae00640 1 -- 172.21.15.86:0/589646649 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9b0006f400 msgr2=0x7f9b00071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:54.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.486+0000 7f9b2ae00640 1 --2- 172.21.15.86:0/589646649 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9b0006f400 0x7f9b00071910 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9b14009580 tx=0x7f9b14009190 comp rx=0 tx=0).stop 2024-04-25T14:03:54.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.486+0000 7f9b2ae00640 1 -- 172.21.15.86:0/589646649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b240cfca0 msgr2=0x7f9b240c3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:54.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.486+0000 7f9b2ae00640 1 --2- 172.21.15.86:0/589646649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b240cfca0 0x7f9b240c3f20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c00c370 tx=0x7f9b0c00c830 comp rx=0 tx=0).stop 2024-04-25T14:03:54.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.486+0000 7f9b2ae00640 1 -- 172.21.15.86:0/589646649 shutdown_connections 2024-04-25T14:03:54.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.486+0000 7f9b2ae00640 1 --2- 172.21.15.86:0/589646649 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9b0006f400 0x7f9b00071910 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:54.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.486+0000 7f9b2ae00640 1 --2- 172.21.15.86:0/589646649 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b240d16e0 0x7f9b240cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:54.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.486+0000 7f9b2ae00640 1 --2- 172.21.15.86:0/589646649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b240cfca0 0x7f9b240c3f20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:54.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.486+0000 7f9b2ae00640 1 --2- 172.21.15.86:0/589646649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b240730b0 0x7f9b240ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:54.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.486+0000 7f9b2ae00640 1 -- 172.21.15.86:0/589646649 >> 172.21.15.86:0/589646649 conn(0x7f9b240b4650 msgr2=0x7f9b241592a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:54.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.486+0000 7f9b2ae00640 1 -- 172.21.15.86:0/589646649 shutdown_connections 2024-04-25T14:03:54.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.486+0000 7f9b2ae00640 1 -- 172.21.15.86:0/589646649 wait complete. 2024-04-25T14:03:54.519 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-25T14:03:54.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.670+0000 7f0942c00640 1 Processor -- start 2024-04-25T14:03:54.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.670+0000 7f0942c00640 1 -- start start 2024-04-25T14:03:54.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.674+0000 7f0942c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f093c15fc10 0x7f093c160020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:54.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.674+0000 7f0942c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f093c06d680 0x7f093c06da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:54.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.674+0000 7f0942c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f093c06dfd0 con 0x7f093c06efc0 2024-04-25T14:03:54.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.674+0000 7f0942c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f093c1646d0 con 0x7f093c06d680 2024-04-25T14:03:54.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.674+0000 7f0942c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f093c163be0 con 0x7f093c15fc10 2024-04-25T14:03:54.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.674+0000 7f0933e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f093c06d680 0x7f093c06da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:54.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.674+0000 7f093be00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f093c15fc10 0x7f093c160020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:54.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.674+0000 7f0933e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f093c06d680 0x7f093c06da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:35060/0 (socket says 172.21.15.86:35060) 2024-04-25T14:03:54.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.674+0000 7f0933e00640 1 -- 172.21.15.86:0/4260005142 learned_addr learned my addr 172.21.15.86:0/4260005142 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:54.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.674+0000 7f0940c00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f093c06efc0 0x7f093c06f3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:60746/0 (socket says 172.21.15.86:60746) 2024-04-25T14:03:54.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.674+0000 7f093b400640 1 -- 172.21.15.86:0/4260005142 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1769265350 0 0) 0x7f09340014a0 con 0x7f093c06efc0 2024-04-25T14:03:54.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.674+0000 7f093b400640 1 -- 172.21.15.86:0/4260005142 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0920003530 con 0x7f093c06efc0 2024-04-25T14:03:54.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.674+0000 7f0933e00640 1 -- 172.21.15.86:0/4260005142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f093c15fc10 msgr2=0x7f093c160020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:54.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.674+0000 7f0933e00640 1 --2- 172.21.15.86:0/4260005142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f093c15fc10 0x7f093c160020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:54.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.674+0000 7f0933e00640 1 -- 172.21.15.86:0/4260005142 >> v1:172.21.15.155:6789/0 conn(0x7f093c06efc0 legacy=0x7f093c06f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:54.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.674+0000 7f0933e00640 1 -- 172.21.15.86:0/4260005142 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093c0705b0 con 0x7f093c06d680 2024-04-25T14:03:54.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.674+0000 7f093be00640 1 --2- 172.21.15.86:0/4260005142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f093c15fc10 0x7f093c160020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:54.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.678+0000 7f0933e00640 1 --2- 172.21.15.86:0/4260005142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f093c06d680 0x7f093c06da90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0924009870 tx=0x7f092401c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c3897589dd0b684 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:54.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.678+0000 7f093b400640 1 -- 172.21.15.86:0/4260005142 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09240b1070 con 0x7f093c06d680 2024-04-25T14:03:54.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.678+0000 7f093b400640 1 -- 172.21.15.86:0/4260005142 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09240ac620 con 0x7f093c06d680 2024-04-25T14:03:54.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.678+0000 7f093b400640 1 -- 172.21.15.86:0/4260005142 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09240ac9f0 con 0x7f093c06d680 2024-04-25T14:03:54.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.678+0000 7f0942c00640 1 -- 172.21.15.86:0/4260005142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f093c06d680 msgr2=0x7f093c06da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:54.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.678+0000 7f0942c00640 1 --2- 172.21.15.86:0/4260005142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f093c06d680 0x7f093c06da90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0924009870 tx=0x7f092401c4d0 comp rx=0 tx=0).stop 2024-04-25T14:03:54.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.678+0000 7f0942c00640 1 -- 172.21.15.86:0/4260005142 shutdown_connections 2024-04-25T14:03:54.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.678+0000 7f0942c00640 1 --2- 172.21.15.86:0/4260005142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f093c06d680 0x7f093c06da90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:54.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.678+0000 7f0942c00640 1 --2- 172.21.15.86:0/4260005142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f093c15fc10 0x7f093c160020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:54.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.678+0000 7f0942c00640 1 -- 172.21.15.86:0/4260005142 >> 172.21.15.86:0/4260005142 conn(0x7f093c073240 msgr2=0x7f093c073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:54.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.678+0000 7f0942c00640 1 -- 172.21.15.86:0/4260005142 shutdown_connections 2024-04-25T14:03:54.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.678+0000 7f0942c00640 1 -- 172.21.15.86:0/4260005142 wait complete. 2024-04-25T14:03:54.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.682+0000 7f0942c00640 1 Processor -- start 2024-04-25T14:03:54.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.682+0000 7f0942c00640 1 -- start start 2024-04-25T14:03:54.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.682+0000 7f0942c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f093c06d680 0x7f093c149820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:54.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.682+0000 7f0942c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f093c06efc0 0x7f093c162610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:54.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.682+0000 7f0942c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f093c15fc10 0x7f093c14a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:54.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.682+0000 7f0942c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f093c0b6170 con 0x7f093c06efc0 2024-04-25T14:03:54.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.682+0000 7f0942c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f093c0b4c40 con 0x7f093c06d680 2024-04-25T14:03:54.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.682+0000 7f0942c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f093c0b62f0 con 0x7f093c15fc10 2024-04-25T14:03:54.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.682+0000 7f0933e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f093c06efc0 0x7f093c162610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:54.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.682+0000 7f0933e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f093c06efc0 0x7f093c162610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59370/0 (socket says 172.21.15.86:59370) 2024-04-25T14:03:54.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.682+0000 7f0933e00640 1 -- 172.21.15.86:0/3926814936 learned_addr learned my addr 172.21.15.86:0/3926814936 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:54.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.682+0000 7f093be00640 1 --2- 172.21.15.86:0/3926814936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f093c06d680 0x7f093c149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:54.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.682+0000 7f0940c00640 1 --2- 172.21.15.86:0/3926814936 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f093c15fc10 0x7f093c14a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:54.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.682+0000 7f0933e00640 1 -- 172.21.15.86:0/3926814936 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f093c15fc10 msgr2=0x7f093c14a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:54.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.682+0000 7f0933e00640 1 --2- 172.21.15.86:0/3926814936 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f093c15fc10 0x7f093c14a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:54.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.682+0000 7f0933e00640 1 -- 172.21.15.86:0/3926814936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f093c06d680 msgr2=0x7f093c149820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:54.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.682+0000 7f0933e00640 1 --2- 172.21.15.86:0/3926814936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f093c06d680 0x7f093c149820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:54.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.682+0000 7f0933e00640 1 -- 172.21.15.86:0/3926814936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093c137f50 con 0x7f093c06efc0 2024-04-25T14:03:54.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.682+0000 7f0940c00640 1 --2- 172.21.15.86:0/3926814936 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f093c15fc10 0x7f093c14a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:54.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.682+0000 7f0933e00640 1 --2- 172.21.15.86:0/3926814936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f093c06efc0 0x7f093c162610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f09240af040 tx=0x7f0924019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:54.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.682+0000 7f0938c00640 1 -- 172.21.15.86:0/3926814936 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09240b1040 con 0x7f093c06efc0 2024-04-25T14:03:54.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.682+0000 7f0942c00640 1 -- 172.21.15.86:0/3926814936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f093c163590 con 0x7f093c06efc0 2024-04-25T14:03:54.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.682+0000 7f0938c00640 1 -- 172.21.15.86:0/3926814936 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09240b5420 con 0x7f093c06efc0 2024-04-25T14:03:54.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.682+0000 7f0938c00640 1 -- 172.21.15.86:0/3926814936 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09240b5810 con 0x7f093c06efc0 2024-04-25T14:03:54.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.682+0000 7f0942c00640 1 -- 172.21.15.86:0/3926814936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f093c162fd0 con 0x7f093c06efc0 2024-04-25T14:03:54.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.686+0000 7f0942c00640 1 -- 172.21.15.86:0/3926814936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f093c162d20 con 0x7f093c06efc0 2024-04-25T14:03:54.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.686+0000 7f0938c00640 1 -- 172.21.15.86:0/3926814936 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f09240b59b0 con 0x7f093c06efc0 2024-04-25T14:03:54.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.686+0000 7f0938c00640 1 --2- 172.21.15.86:0/3926814936 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f091806f400 0x7f0918071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:54.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.686+0000 7f0938c00640 1 -- 172.21.15.86:0/3926814936 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6500+0+0 (secure 0 0 0) 0x7f0924129d50 con 0x7f093c06efc0 2024-04-25T14:03:54.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.686+0000 7f093be00640 1 --2- 172.21.15.86:0/3926814936 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f091806f400 0x7f0918071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:54.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.686+0000 7f093be00640 1 --2- 172.21.15.86:0/3926814936 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f091806f400 0x7f0918071910 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f092c009580 tx=0x7f092c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:54.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.690+0000 7f0938c00640 1 -- 172.21.15.86:0/3926814936 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0924016150 con 0x7f093c06efc0 2024-04-25T14:03:54.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:54.850+0000 7f0942c00640 1 -- 172.21.15.86:0/3926814936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f093c050750 con 0x7f093c06efc0 2024-04-25T14:03:55.422 INFO:tasks.ceph.mds.d.smithi155.stderr:2024-04-25T14:03:55.420+0000 7f6e4d800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T14:03:55.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.486+0000 7f0938c00640 1 -- 172.21.15.86:0/3926814936 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) ==== 169+0+0 (secure 0 0 0) 0x7f09240f6910 con 0x7f093c06efc0 2024-04-25T14:03:55.487 INFO:teuthology.orchestra.run.smithi086.stderr:set pool 4 allow_ec_overwrites to true 2024-04-25T14:03:55.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.490+0000 7f0942c00640 1 -- 172.21.15.86:0/3926814936 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f091806f400 msgr2=0x7f0918071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:55.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.490+0000 7f0942c00640 1 --2- 172.21.15.86:0/3926814936 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f091806f400 0x7f0918071910 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f092c009580 tx=0x7f092c009190 comp rx=0 tx=0).stop 2024-04-25T14:03:55.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.490+0000 7f0942c00640 1 -- 172.21.15.86:0/3926814936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f093c06efc0 msgr2=0x7f093c162610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:55.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.490+0000 7f0942c00640 1 --2- 172.21.15.86:0/3926814936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f093c06efc0 0x7f093c162610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f09240af040 tx=0x7f0924019270 comp rx=0 tx=0).stop 2024-04-25T14:03:55.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.490+0000 7f0942c00640 1 -- 172.21.15.86:0/3926814936 shutdown_connections 2024-04-25T14:03:55.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.490+0000 7f0942c00640 1 --2- 172.21.15.86:0/3926814936 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f091806f400 0x7f0918071910 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:55.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.490+0000 7f0942c00640 1 --2- 172.21.15.86:0/3926814936 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f093c15fc10 0x7f093c14a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:55.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.490+0000 7f0942c00640 1 --2- 172.21.15.86:0/3926814936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f093c06efc0 0x7f093c162610 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:55.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.490+0000 7f0942c00640 1 --2- 172.21.15.86:0/3926814936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f093c06d680 0x7f093c149820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:55.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.490+0000 7f0942c00640 1 -- 172.21.15.86:0/3926814936 >> 172.21.15.86:0/3926814936 conn(0x7f093c073240 msgr2=0x7f093c14c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:55.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.490+0000 7f0942c00640 1 -- 172.21.15.86:0/3926814936 shutdown_connections 2024-04-25T14:03:55.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.494+0000 7f0942c00640 1 -- 172.21.15.86:0/3926814936 wait complete. 2024-04-25T14:03:55.533 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-25T14:03:55.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.686+0000 7f2f99000640 1 Processor -- start 2024-04-25T14:03:55.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.686+0000 7f2f99000640 1 -- start start 2024-04-25T14:03:55.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.690+0000 7f2f99000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f9406e970 0x7f2f9416abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:55.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.690+0000 7f2f99000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f9406ef90 con 0x7f2f94071530 2024-04-25T14:03:55.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.690+0000 7f2f99000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f94163a80 con 0x7f2f940cf970 2024-04-25T14:03:55.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.690+0000 7f2f99000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2f940cedf0 con 0x7f2f9406e970 2024-04-25T14:03:55.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.690+0000 7f2f91400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2f94071530 0x7f2f9406e260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35996/0 (socket says 172.21.15.86:35996) 2024-04-25T14:03:55.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.690+0000 7f2f91400640 1 -- 172.21.15.86:0/936479965 learned_addr learned my addr 172.21.15.86:0/936479965 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:03:55.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.690+0000 7f2f92800640 1 --2- 172.21.15.86:0/936479965 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f9406e970 0x7f2f9416abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:55.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.690+0000 7f2f90a00640 1 -- 172.21.15.86:0/936479965 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2888648180 0 0) 0x7f2f9406ef90 con 0x7f2f94071530 2024-04-25T14:03:55.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.690+0000 7f2f90a00640 1 -- 172.21.15.86:0/936479965 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f78003530 con 0x7f2f94071530 2024-04-25T14:03:55.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.690+0000 7f2f90a00640 1 -- 172.21.15.86:0/936479965 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3845049685 0 0) 0x7f2f94163a80 con 0x7f2f940cf970 2024-04-25T14:03:55.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.690+0000 7f2f90a00640 1 -- 172.21.15.86:0/936479965 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f9406ef90 con 0x7f2f940cf970 2024-04-25T14:03:55.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.690+0000 7f2f92800640 1 -- 172.21.15.86:0/936479965 >> v1:172.21.15.155:6789/0 conn(0x7f2f940cf970 legacy=0x7f2f94070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:55.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.690+0000 7f2f92800640 1 -- 172.21.15.86:0/936479965 >> v1:172.21.15.86:6789/0 conn(0x7f2f94071530 legacy=0x7f2f9406e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:55.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.690+0000 7f2f92800640 1 -- 172.21.15.86:0/936479965 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f94163d60 con 0x7f2f9406e970 2024-04-25T14:03:55.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.694+0000 7f2f92800640 1 --2- 172.21.15.86:0/936479965 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f9406e970 0x7f2f9416abc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c009dc0 tx=0x7f2f8c01c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b56a47f6d38ef9ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:55.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.694+0000 7f2f90a00640 1 -- 172.21.15.86:0/936479965 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c0b2070 con 0x7f2f9406e970 2024-04-25T14:03:55.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.694+0000 7f2f90a00640 1 -- 172.21.15.86:0/936479965 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f8c0a5850 con 0x7f2f9406e970 2024-04-25T14:03:55.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.694+0000 7f2f90a00640 1 -- 172.21.15.86:0/936479965 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c0a5c20 con 0x7f2f9406e970 2024-04-25T14:03:55.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.694+0000 7f2f99000640 1 -- 172.21.15.86:0/936479965 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f9406e970 msgr2=0x7f2f9416abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:55.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.694+0000 7f2f99000640 1 --2- 172.21.15.86:0/936479965 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f9406e970 0x7f2f9416abc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c009dc0 tx=0x7f2f8c01c990 comp rx=0 tx=0).stop 2024-04-25T14:03:55.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.694+0000 7f2f99000640 1 -- 172.21.15.86:0/936479965 shutdown_connections 2024-04-25T14:03:55.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.694+0000 7f2f99000640 1 --2- 172.21.15.86:0/936479965 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f9406e970 0x7f2f9416abc0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:55.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.694+0000 7f2f99000640 1 -- 172.21.15.86:0/936479965 >> 172.21.15.86:0/936479965 conn(0x7f2f940cffc0 msgr2=0x7f2f940d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:55.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.694+0000 7f2f99000640 1 -- 172.21.15.86:0/936479965 shutdown_connections 2024-04-25T14:03:55.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.694+0000 7f2f99000640 1 -- 172.21.15.86:0/936479965 wait complete. 2024-04-25T14:03:55.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.694+0000 7f2f99000640 1 Processor -- start 2024-04-25T14:03:55.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.694+0000 7f2f99000640 1 -- start start 2024-04-25T14:03:55.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.698+0000 7f2f99000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f9406e970 0x7f2f940b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:55.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.698+0000 7f2f99000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f940cf970 0x7f2f940b8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:55.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.698+0000 7f2f99000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f94153c30 0x7f2f94154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:55.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.698+0000 7f2f99000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9416dcb0 con 0x7f2f940cf970 2024-04-25T14:03:55.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.698+0000 7f2f99000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9416db30 con 0x7f2f94153c30 2024-04-25T14:03:55.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.698+0000 7f2f99000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2f9416de30 con 0x7f2f9406e970 2024-04-25T14:03:55.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.698+0000 7f2f91400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f940cf970 0x7f2f940b8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:55.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.698+0000 7f2f91400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f940cf970 0x7f2f940b8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59376/0 (socket says 172.21.15.86:59376) 2024-04-25T14:03:55.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.698+0000 7f2f91400640 1 -- 172.21.15.86:0/2420139386 learned_addr learned my addr 172.21.15.86:0/2420139386 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:55.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.698+0000 7f2f91e00640 1 --2- 172.21.15.86:0/2420139386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f9406e970 0x7f2f940b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:55.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.698+0000 7f2f92800640 1 --2- 172.21.15.86:0/2420139386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f94153c30 0x7f2f94154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:55.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.698+0000 7f2f91400640 1 -- 172.21.15.86:0/2420139386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f9406e970 msgr2=0x7f2f940b6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:55.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.698+0000 7f2f91400640 1 --2- 172.21.15.86:0/2420139386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f9406e970 0x7f2f940b6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:55.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.698+0000 7f2f91400640 1 -- 172.21.15.86:0/2420139386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f94153c30 msgr2=0x7f2f94154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:55.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.698+0000 7f2f91400640 1 --2- 172.21.15.86:0/2420139386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f94153c30 0x7f2f94154780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:55.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.698+0000 7f2f91400640 1 -- 172.21.15.86:0/2420139386 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f940cac90 con 0x7f2f940cf970 2024-04-25T14:03:55.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.698+0000 7f2f91400640 1 --2- 172.21.15.86:0/2420139386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f940cf970 0x7f2f940b8f30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000c480 tx=0x7f2f8000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:55.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.698+0000 7f2f8a000640 1 -- 172.21.15.86:0/2420139386 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f80017070 con 0x7f2f940cf970 2024-04-25T14:03:55.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.698+0000 7f2f8a000640 1 -- 172.21.15.86:0/2420139386 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f80007480 con 0x7f2f940cf970 2024-04-25T14:03:55.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.698+0000 7f2f99000640 1 -- 172.21.15.86:0/2420139386 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f940caf00 con 0x7f2f940cf970 2024-04-25T14:03:55.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.698+0000 7f2f8a000640 1 -- 172.21.15.86:0/2420139386 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f80007630 con 0x7f2f940cf970 2024-04-25T14:03:55.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.698+0000 7f2f99000640 1 -- 172.21.15.86:0/2420139386 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f940cb4c0 con 0x7f2f940cf970 2024-04-25T14:03:55.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.698+0000 7f2f99000640 1 -- 172.21.15.86:0/2420139386 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f940720e0 con 0x7f2f940cf970 2024-04-25T14:03:55.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.702+0000 7f2f8a000640 1 -- 172.21.15.86:0/2420139386 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2f8001a070 con 0x7f2f940cf970 2024-04-25T14:03:55.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.702+0000 7f2f8a000640 1 --2- 172.21.15.86:0/2420139386 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2f7406f400 0x7f2f74071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:55.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.702+0000 7f2f8a000640 1 -- 172.21.15.86:0/2420139386 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7f2f8008e890 con 0x7f2f940cf970 2024-04-25T14:03:55.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.702+0000 7f2f91e00640 1 --2- 172.21.15.86:0/2420139386 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2f7406f400 0x7f2f74071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:55.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.702+0000 7f2f91e00640 1 --2- 172.21.15.86:0/2420139386 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2f7406f400 0x7f2f74071910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2f840011f0 tx=0x7f2f84009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:55.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.706+0000 7f2f8a000640 1 -- 172.21.15.86:0/2420139386 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2f8005ae00 con 0x7f2f940cf970 2024-04-25T14:03:55.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:55.898+0000 7f2f99000640 1 -- 172.21.15.86:0/2420139386 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f2f940b6700 con 0x7f2f940cf970 2024-04-25T14:03:56.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.490+0000 7f2f8a000640 1 -- 172.21.15.86:0/2420139386 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f2f8008ce20 con 0x7f2f940cf970 2024-04-25T14:03:56.489 INFO:teuthology.orchestra.run.smithi086.stderr:added data pool 4 to fsmap 2024-04-25T14:03:56.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.490+0000 7f2f99000640 1 -- 172.21.15.86:0/2420139386 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2f7406f400 msgr2=0x7f2f74071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:56.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.490+0000 7f2f99000640 1 --2- 172.21.15.86:0/2420139386 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2f7406f400 0x7f2f74071910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2f840011f0 tx=0x7f2f84009040 comp rx=0 tx=0).stop 2024-04-25T14:03:56.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.490+0000 7f2f99000640 1 -- 172.21.15.86:0/2420139386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f940cf970 msgr2=0x7f2f940b8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:56.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.490+0000 7f2f99000640 1 --2- 172.21.15.86:0/2420139386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f940cf970 0x7f2f940b8f30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000c480 tx=0x7f2f8000c940 comp rx=0 tx=0).stop 2024-04-25T14:03:56.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.490+0000 7f2f99000640 1 -- 172.21.15.86:0/2420139386 shutdown_connections 2024-04-25T14:03:56.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.490+0000 7f2f99000640 1 --2- 172.21.15.86:0/2420139386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f94153c30 0x7f2f94154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:56.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.490+0000 7f2f99000640 1 --2- 172.21.15.86:0/2420139386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f940cf970 0x7f2f940b8f30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:56.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.490+0000 7f2f99000640 1 --2- 172.21.15.86:0/2420139386 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2f7406f400 0x7f2f74071910 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:56.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.490+0000 7f2f99000640 1 --2- 172.21.15.86:0/2420139386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f9406e970 0x7f2f940b6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:56.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.490+0000 7f2f99000640 1 -- 172.21.15.86:0/2420139386 >> 172.21.15.86:0/2420139386 conn(0x7f2f940cffc0 msgr2=0x7f2f940b4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:56.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.490+0000 7f2f99000640 1 -- 172.21.15.86:0/2420139386 shutdown_connections 2024-04-25T14:03:56.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.490+0000 7f2f99000640 1 -- 172.21.15.86:0/2420139386 wait complete. 2024-04-25T14:03:56.514 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:03:56.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.658+0000 7fe13f600640 1 Processor -- start 2024-04-25T14:03:56.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.658+0000 7fe13f600640 1 -- start start 2024-04-25T14:03:56.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.658+0000 7fe13f600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe138070330 0x7fe1380d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:56.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.658+0000 7fe13f600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe1380d1c20 0x7fe13806ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:56.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.658+0000 7fe13f600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1380d2030 con 0x7fe13806f270 2024-04-25T14:03:56.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.658+0000 7fe13f600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe13815e5c0 con 0x7fe1380d1c20 2024-04-25T14:03:56.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.658+0000 7fe13f600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe13815e8a0 con 0x7fe138070330 2024-04-25T14:03:56.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.662+0000 7fe12fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe1380d1c20 0x7fe13806ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:56.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.662+0000 7fe13d600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe13806f270 0x7fe13806f680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35998/0 (socket says 172.21.15.86:35998) 2024-04-25T14:03:56.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.662+0000 7fe13d600640 1 -- 172.21.15.86:0/3724622126 learned_addr learned my addr 172.21.15.86:0/3724622126 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:03:56.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.662+0000 7fe13cc00640 1 --2- 172.21.15.86:0/3724622126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe138070330 0x7fe1380d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:56.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.662+0000 7fe12fe00640 1 -- 172.21.15.86:0/3724622126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe138070330 msgr2=0x7fe1380d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:56.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.662+0000 7fe12fe00640 1 --2- 172.21.15.86:0/3724622126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe138070330 0x7fe1380d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:56.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.662+0000 7fe12fe00640 1 -- 172.21.15.86:0/3724622126 >> v1:172.21.15.86:6789/0 conn(0x7fe13806f270 legacy=0x7fe13806f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:56.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.662+0000 7fe12fe00640 1 -- 172.21.15.86:0/3724622126 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe13815eb80 con 0x7fe1380d1c20 2024-04-25T14:03:56.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.662+0000 7fe137e00640 1 -- 172.21.15.86:0/3724622126 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2595237113 0 0) 0x7fe130001120 con 0x7fe13806f270 2024-04-25T14:03:56.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.662+0000 7fe12fe00640 1 --2- 172.21.15.86:0/3724622126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1380d1c20 0x7fe13806ed30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe1200097f0 tx=0x7fe12001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=902cfde432ef6f56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:56.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.662+0000 7fe137e00640 1 -- 172.21.15.86:0/3724622126 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1200b0070 con 0x7fe1380d1c20 2024-04-25T14:03:56.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.662+0000 7fe137e00640 1 -- 172.21.15.86:0/3724622126 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1200ab600 con 0x7fe1380d1c20 2024-04-25T14:03:56.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.662+0000 7fe137e00640 1 -- 172.21.15.86:0/3724622126 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1200ab9d0 con 0x7fe1380d1c20 2024-04-25T14:03:56.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.666+0000 7fe13f600640 1 -- 172.21.15.86:0/3724622126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1380d1c20 msgr2=0x7fe13806ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:56.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.666+0000 7fe13f600640 1 --2- 172.21.15.86:0/3724622126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1380d1c20 0x7fe13806ed30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe1200097f0 tx=0x7fe12001c410 comp rx=0 tx=0).stop 2024-04-25T14:03:56.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.666+0000 7fe13f600640 1 -- 172.21.15.86:0/3724622126 shutdown_connections 2024-04-25T14:03:56.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.666+0000 7fe13f600640 1 --2- 172.21.15.86:0/3724622126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1380d1c20 0x7fe13806ed30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:56.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.666+0000 7fe13f600640 1 --2- 172.21.15.86:0/3724622126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe138070330 0x7fe1380d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:56.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.666+0000 7fe13f600640 1 -- 172.21.15.86:0/3724622126 >> 172.21.15.86:0/3724622126 conn(0x7fe1380b4650 msgr2=0x7fe1380b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:56.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.666+0000 7fe13f600640 1 -- 172.21.15.86:0/3724622126 shutdown_connections 2024-04-25T14:03:56.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.666+0000 7fe13f600640 1 -- 172.21.15.86:0/3724622126 wait complete. 2024-04-25T14:03:56.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.666+0000 7fe13f600640 1 Processor -- start 2024-04-25T14:03:56.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.666+0000 7fe13f600640 1 -- start start 2024-04-25T14:03:56.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.670+0000 7fe13f600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe13806f270 0x7fe138072880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:56.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.670+0000 7fe13f600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe138070330 0x7fe138072dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:56.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.670+0000 7fe13f600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1380d1c20 0x7fe138073300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:56.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.670+0000 7fe13f600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe1380708c0 con 0x7fe138070330 2024-04-25T14:03:56.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.670+0000 7fe13f600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe138070740 con 0x7fe13806f270 2024-04-25T14:03:56.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.670+0000 7fe13f600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe13816afc0 con 0x7fe1380d1c20 2024-04-25T14:03:56.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.670+0000 7fe12fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe138070330 0x7fe138072dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:56.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.670+0000 7fe12fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe138070330 0x7fe138072dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59384/0 (socket says 172.21.15.86:59384) 2024-04-25T14:03:56.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.670+0000 7fe12fe00640 1 -- 172.21.15.86:0/1033572053 learned_addr learned my addr 172.21.15.86:0/1033572053 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:56.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.670+0000 7fe13cc00640 1 --2- 172.21.15.86:0/1033572053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe13806f270 0x7fe138072880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:56.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.670+0000 7fe13d600640 1 --2- 172.21.15.86:0/1033572053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1380d1c20 0x7fe138073300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:56.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.670+0000 7fe12fe00640 1 -- 172.21.15.86:0/1033572053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1380d1c20 msgr2=0x7fe138073300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:56.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.670+0000 7fe12fe00640 1 --2- 172.21.15.86:0/1033572053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1380d1c20 0x7fe138073300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:56.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.670+0000 7fe12fe00640 1 -- 172.21.15.86:0/1033572053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe13806f270 msgr2=0x7fe138072880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:56.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.670+0000 7fe12fe00640 1 --2- 172.21.15.86:0/1033572053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe13806f270 0x7fe138072880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:56.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.670+0000 7fe12fe00640 1 -- 172.21.15.86:0/1033572053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1380cf530 con 0x7fe138070330 2024-04-25T14:03:56.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.670+0000 7fe13d600640 1 --2- 172.21.15.86:0/1033572053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1380d1c20 0x7fe138073300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:56.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.670+0000 7fe13cc00640 1 --2- 172.21.15.86:0/1033572053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe13806f270 0x7fe138072880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:56.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.670+0000 7fe12fe00640 1 --2- 172.21.15.86:0/1033572053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe138070330 0x7fe138072dc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe12001a040 tx=0x7fe1200191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:56.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.670+0000 7fe135600640 1 -- 172.21.15.86:0/1033572053 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1200ba020 con 0x7fe138070330 2024-04-25T14:03:56.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.670+0000 7fe135600640 1 -- 172.21.15.86:0/1033572053 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1200b4400 con 0x7fe138070330 2024-04-25T14:03:56.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.670+0000 7fe135600640 1 -- 172.21.15.86:0/1033572053 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1200b0040 con 0x7fe138070330 2024-04-25T14:03:56.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.670+0000 7fe13f600640 1 -- 172.21.15.86:0/1033572053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1380cf250 con 0x7fe138070330 2024-04-25T14:03:56.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.670+0000 7fe13f600640 1 -- 172.21.15.86:0/1033572053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe13815ab30 con 0x7fe138070330 2024-04-25T14:03:56.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.670+0000 7fe13f600640 1 -- 172.21.15.86:0/1033572053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe138069d00 con 0x7fe138070330 2024-04-25T14:03:56.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.674+0000 7fe135600640 1 -- 172.21.15.86:0/1033572053 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe1200bd020 con 0x7fe138070330 2024-04-25T14:03:56.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.674+0000 7fe135600640 1 --2- 172.21.15.86:0/1033572053 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe11406f400 0x7fe114071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:56.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.674+0000 7fe135600640 1 -- 172.21.15.86:0/1033572053 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe120129590 con 0x7fe138070330 2024-04-25T14:03:56.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.674+0000 7fe13cc00640 1 --2- 172.21.15.86:0/1033572053 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe11406f400 0x7fe114071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:56.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.674+0000 7fe13cc00640 1 --2- 172.21.15.86:0/1033572053 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe11406f400 0x7fe114071910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe128003fd0 tx=0x7fe1280039e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:56.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.678+0000 7fe135600640 1 -- 172.21.15.86:0/1033572053 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe1200c4030 con 0x7fe138070330 2024-04-25T14:03:56.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.870+0000 7fe13f600640 1 -- 172.21.15.86:0/1033572053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe138076bf0 con 0x7fe138070330 2024-04-25T14:03:56.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.874+0000 7fe135600640 1 -- 172.21.15.86:0/1033572053 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5177 (secure 0 0 0) 0x7fe120016090 con 0x7fe138070330 2024-04-25T14:03:56.873 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:03:56.873 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/347107112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":347107112},{"type":"v1","addr":"172.21.15.155:6835","nonce":347107112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2236737549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2236737549},{"type":"v1","addr":"172.21.15.86:6837","nonce":2236737549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/519560229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":519560229},{"type":"v1","addr":"172.21.15.86:6836","nonce":519560229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:03:50.852257+0000","modified":"2024-04-25T14:03:55.899313+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6833/1944249570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1944249570},{"type":"v1","addr":"172.21.15.155:6833","nonce":1944249570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4275,"qdb_cluster":[4275]},"id":1}]} 2024-04-25T14:03:56.873 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 7 2024-04-25T14:03:56.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.874+0000 7fe13f600640 1 -- 172.21.15.86:0/1033572053 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe11406f400 msgr2=0x7fe114071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:56.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.874+0000 7fe13f600640 1 --2- 172.21.15.86:0/1033572053 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe11406f400 0x7fe114071910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe128003fd0 tx=0x7fe1280039e0 comp rx=0 tx=0).stop 2024-04-25T14:03:56.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.874+0000 7fe13f600640 1 -- 172.21.15.86:0/1033572053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe138070330 msgr2=0x7fe138072dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:56.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.874+0000 7fe13f600640 1 --2- 172.21.15.86:0/1033572053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe138070330 0x7fe138072dc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe12001a040 tx=0x7fe1200191b0 comp rx=0 tx=0).stop 2024-04-25T14:03:56.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.874+0000 7fe13f600640 1 -- 172.21.15.86:0/1033572053 shutdown_connections 2024-04-25T14:03:56.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.874+0000 7fe13f600640 1 --2- 172.21.15.86:0/1033572053 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe11406f400 0x7fe114071910 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:56.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.874+0000 7fe13f600640 1 --2- 172.21.15.86:0/1033572053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1380d1c20 0x7fe138073300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:56.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.874+0000 7fe13f600640 1 --2- 172.21.15.86:0/1033572053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe138070330 0x7fe138072dc0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:56.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.874+0000 7fe13f600640 1 --2- 172.21.15.86:0/1033572053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe13806f270 0x7fe138072880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:56.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.874+0000 7fe13f600640 1 -- 172.21.15.86:0/1033572053 >> 172.21.15.86:0/1033572053 conn(0x7fe1380b4650 msgr2=0x7fe13815cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:56.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.874+0000 7fe13f600640 1 -- 172.21.15.86:0/1033572053 shutdown_connections 2024-04-25T14:03:56.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:56.874+0000 7fe13f600640 1 -- 172.21.15.86:0/1033572053 wait complete. 2024-04-25T14:03:56.894 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:03:57.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.038+0000 7f6ffa800640 1 Processor -- start 2024-04-25T14:03:57.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.042+0000 7f6ffa800640 1 -- start start 2024-04-25T14:03:57.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.042+0000 7f6ffa800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6ff40d1df0 0x7f6ff40cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:57.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.042+0000 7f6ffa800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ff415e5c0 con 0x7f6ff40cdcc0 2024-04-25T14:03:57.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.042+0000 7f6ffa800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ff415e8a0 con 0x7f6ff4070330 2024-04-25T14:03:57.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.042+0000 7f6ffa800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6ff406fd40 con 0x7f6ff40d1df0 2024-04-25T14:03:57.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.042+0000 7f6ff2a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6ff40d1df0 0x7f6ff40cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:57.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.042+0000 7f6ff3e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6ff40cdcc0 0x7f6ff40ce0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36010/0 (socket says 172.21.15.86:36010) 2024-04-25T14:03:57.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.042+0000 7f6ff3e00640 1 -- 172.21.15.86:0/1027186277 learned_addr learned my addr 172.21.15.86:0/1027186277 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:03:57.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.042+0000 7f6ff2a00640 1 -- 172.21.15.86:0/1027186277 >> v1:172.21.15.155:6789/0 conn(0x7f6ff4070330 legacy=0x7f6ff40d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:57.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.042+0000 7f6ff2a00640 1 -- 172.21.15.86:0/1027186277 >> v1:172.21.15.86:6789/0 conn(0x7f6ff40cdcc0 legacy=0x7f6ff40ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:57.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.042+0000 7f6ff2000640 1 -- 172.21.15.86:0/1027186277 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1750847300 0 0) 0x7f6ff415e5c0 con 0x7f6ff40cdcc0 2024-04-25T14:03:57.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.042+0000 7f6ff2a00640 1 -- 172.21.15.86:0/1027186277 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff415eb80 con 0x7f6ff40d1df0 2024-04-25T14:03:57.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.046+0000 7f6ff2a00640 1 --2- 172.21.15.86:0/1027186277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ff40d1df0 0x7f6ff40cd780 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6fe00097f0 tx=0x7f6fe001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b15ba3a52a26dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:57.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.046+0000 7f6ff2000640 1 -- 172.21.15.86:0/1027186277 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe00b0070 con 0x7f6ff40d1df0 2024-04-25T14:03:57.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.046+0000 7f6ff2000640 1 -- 172.21.15.86:0/1027186277 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fe00ab600 con 0x7f6ff40d1df0 2024-04-25T14:03:57.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.046+0000 7f6ff2000640 1 -- 172.21.15.86:0/1027186277 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe00ab9d0 con 0x7f6ff40d1df0 2024-04-25T14:03:57.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.046+0000 7f6ffa800640 1 -- 172.21.15.86:0/1027186277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ff40d1df0 msgr2=0x7f6ff40cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:57.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.046+0000 7f6ffa800640 1 --2- 172.21.15.86:0/1027186277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ff40d1df0 0x7f6ff40cd780 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6fe00097f0 tx=0x7f6fe001c3e0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.046+0000 7f6ffa800640 1 -- 172.21.15.86:0/1027186277 shutdown_connections 2024-04-25T14:03:57.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.046+0000 7f6ffa800640 1 --2- 172.21.15.86:0/1027186277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ff40d1df0 0x7f6ff40cd780 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.046+0000 7f6ffa800640 1 -- 172.21.15.86:0/1027186277 >> 172.21.15.86:0/1027186277 conn(0x7f6ff40b4650 msgr2=0x7f6ff40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:57.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.046+0000 7f6ffa800640 1 -- 172.21.15.86:0/1027186277 shutdown_connections 2024-04-25T14:03:57.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.046+0000 7f6ffa800640 1 -- 172.21.15.86:0/1027186277 wait complete. 2024-04-25T14:03:57.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.050+0000 7f6ffa800640 1 Processor -- start 2024-04-25T14:03:57.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.050+0000 7f6ffa800640 1 -- start start 2024-04-25T14:03:57.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.050+0000 7f6ffa800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ff4070330 0x7f6ff40ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:57.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.050+0000 7f6ffa800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ff40cdcc0 0x7f6ff40cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:57.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.050+0000 7f6ffa800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ff40aead0 0x7f6ff40cf930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:57.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.050+0000 7f6ffa800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff40708c0 con 0x7f6ff40aead0 2024-04-25T14:03:57.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.050+0000 7f6ffa800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff4070740 con 0x7f6ff40cdcc0 2024-04-25T14:03:57.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.050+0000 7f6ffa800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6ff416bfd0 con 0x7f6ff4070330 2024-04-25T14:03:57.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.050+0000 7f6ff3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ff40aead0 0x7f6ff40cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:57.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.050+0000 7f6ff3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ff40aead0 0x7f6ff40cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59408/0 (socket says 172.21.15.86:59408) 2024-04-25T14:03:57.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.050+0000 7f6ff3e00640 1 -- 172.21.15.86:0/592041933 learned_addr learned my addr 172.21.15.86:0/592041933 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:57.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.050+0000 7f6ff2a00640 1 --2- 172.21.15.86:0/592041933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ff40cdcc0 0x7f6ff40cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:57.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.050+0000 7f6ff3400640 1 --2- 172.21.15.86:0/592041933 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ff4070330 0x7f6ff40ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:57.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.050+0000 7f6ff3e00640 1 -- 172.21.15.86:0/592041933 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ff4070330 msgr2=0x7f6ff40ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:57.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.050+0000 7f6ff3e00640 1 --2- 172.21.15.86:0/592041933 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ff4070330 0x7f6ff40ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.050+0000 7f6ff3e00640 1 -- 172.21.15.86:0/592041933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ff40cdcc0 msgr2=0x7f6ff40cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:57.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.050+0000 7f6ff3e00640 1 --2- 172.21.15.86:0/592041933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ff40cdcc0 0x7f6ff40cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.050+0000 7f6ff3e00640 1 -- 172.21.15.86:0/592041933 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff4156bf0 con 0x7f6ff40aead0 2024-04-25T14:03:57.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.050+0000 7f6ff2a00640 1 --2- 172.21.15.86:0/592041933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ff40cdcc0 0x7f6ff40cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:57.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.050+0000 7f6ff3e00640 1 --2- 172.21.15.86:0/592041933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ff40aead0 0x7f6ff40cf930 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00c730 tx=0x7f6fec00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:57.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.050+0000 7f6feb400640 1 -- 172.21.15.86:0/592041933 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec018070 con 0x7f6ff40aead0 2024-04-25T14:03:57.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.050+0000 7f6feb400640 1 -- 172.21.15.86:0/592041933 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fec00ce40 con 0x7f6ff40aead0 2024-04-25T14:03:57.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.050+0000 7f6ffa800640 1 -- 172.21.15.86:0/592041933 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff409ad80 con 0x7f6ff40aead0 2024-04-25T14:03:57.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.050+0000 7f6feb400640 1 -- 172.21.15.86:0/592041933 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec00a880 con 0x7f6ff40aead0 2024-04-25T14:03:57.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.050+0000 7f6ffa800640 1 -- 172.21.15.86:0/592041933 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff4157170 con 0x7f6ff40aead0 2024-04-25T14:03:57.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.054+0000 7f6ffa800640 1 -- 172.21.15.86:0/592041933 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ff40c92d0 con 0x7f6ff40aead0 2024-04-25T14:03:57.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.054+0000 7f6feb400640 1 -- 172.21.15.86:0/592041933 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6fec01b070 con 0x7f6ff40aead0 2024-04-25T14:03:57.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.054+0000 7f6feb400640 1 --2- 172.21.15.86:0/592041933 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6fd406f430 0x7f6fd4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:57.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.054+0000 7f6feb400640 1 -- 172.21.15.86:0/592041933 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6fec08ebf0 con 0x7f6ff40aead0 2024-04-25T14:03:57.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.054+0000 7f6ff3400640 1 --2- 172.21.15.86:0/592041933 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6fd406f430 0x7f6fd4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:57.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.054+0000 7f6ff3400640 1 --2- 172.21.15.86:0/592041933 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6fd406f430 0x7f6fd4071940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4000f70 tx=0x7f6fe4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:57.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.058+0000 7f6feb400640 1 -- 172.21.15.86:0/592041933 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6ff40c92d0 con 0x7f6ff40aead0 2024-04-25T14:03:57.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.218+0000 7f6ffa800640 1 -- 172.21.15.86:0/592041933 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ff4069d00 con 0x7f6ff40aead0 2024-04-25T14:03:57.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.218+0000 7f6feb400640 1 -- 172.21.15.86:0/592041933 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7f6fec05b700 con 0x7f6ff40aead0 2024-04-25T14:03:57.219 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:03:57.219 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":24,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:03:55.899092+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:03:49.151473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:03:49.802597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T14:03:53.492008+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:03:57.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.222+0000 7f6ffa800640 1 -- 172.21.15.86:0/592041933 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6fd406f430 msgr2=0x7f6fd4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:57.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.222+0000 7f6ffa800640 1 --2- 172.21.15.86:0/592041933 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6fd406f430 0x7f6fd4071940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4000f70 tx=0x7f6fe4009040 comp rx=0 tx=0).stop 2024-04-25T14:03:57.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.222+0000 7f6ffa800640 1 -- 172.21.15.86:0/592041933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ff40aead0 msgr2=0x7f6ff40cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:57.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.222+0000 7f6ffa800640 1 --2- 172.21.15.86:0/592041933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ff40aead0 0x7f6ff40cf930 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00c730 tx=0x7f6fec00cbf0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.222+0000 7f6ffa800640 1 -- 172.21.15.86:0/592041933 shutdown_connections 2024-04-25T14:03:57.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.222+0000 7f6ffa800640 1 --2- 172.21.15.86:0/592041933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ff40aead0 0x7f6ff40cf930 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.222+0000 7f6ffa800640 1 --2- 172.21.15.86:0/592041933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ff40cdcc0 0x7f6ff40cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.222+0000 7f6ffa800640 1 --2- 172.21.15.86:0/592041933 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6fd406f430 0x7f6fd4071940 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.222+0000 7f6ffa800640 1 --2- 172.21.15.86:0/592041933 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ff4070330 0x7f6ff40ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.222+0000 7f6ffa800640 1 -- 172.21.15.86:0/592041933 >> 172.21.15.86:0/592041933 conn(0x7f6ff40b4650 msgr2=0x7f6ff4158e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:57.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.222+0000 7f6ffa800640 1 -- 172.21.15.86:0/592041933 shutdown_connections 2024-04-25T14:03:57.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.222+0000 7f6ffa800640 1 -- 172.21.15.86:0/592041933 wait complete. 2024-04-25T14:03:57.241 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:03:57.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.386+0000 7f6198200640 1 Processor -- start 2024-04-25T14:03:57.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.386+0000 7f6198200640 1 -- start start 2024-04-25T14:03:57.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.386+0000 7f6198200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f61900d1df0 0x7f61900cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:57.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.386+0000 7f6198200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f619015e5c0 con 0x7f61900cdcc0 2024-04-25T14:03:57.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.386+0000 7f6198200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f619015e8a0 con 0x7f6190070330 2024-04-25T14:03:57.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.386+0000 7f6198200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f619006fd40 con 0x7f61900d1df0 2024-04-25T14:03:57.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.386+0000 7f6196200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f61900cdcc0 0x7f61900ce0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36012/0 (socket says 172.21.15.86:36012) 2024-04-25T14:03:57.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.386+0000 7f6196200640 1 -- 172.21.15.86:0/1455561590 learned_addr learned my addr 172.21.15.86:0/1455561590 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:03:57.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.390+0000 7f6194e00640 1 --2- 172.21.15.86:0/1455561590 >> v2:172.21.15.155:3300/0 conn(0x7f61900d1df0 0x7f61900cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:57.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.390+0000 7f618fe00640 1 -- 172.21.15.86:0/1455561590 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 552366201 0 0) 0x7f619015e5c0 con 0x7f61900cdcc0 2024-04-25T14:03:57.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.390+0000 7f618fe00640 1 -- 172.21.15.86:0/1455561590 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6178003530 con 0x7f61900cdcc0 2024-04-25T14:03:57.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.390+0000 7f618fe00640 1 -- 172.21.15.86:0/1455561590 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2736474622 0 0) 0x7f6178003530 con 0x7f61900cdcc0 2024-04-25T14:03:57.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.390+0000 7f618fe00640 1 -- 172.21.15.86:0/1455561590 >> v2:172.21.15.155:3300/0 conn(0x7f61900d1df0 msgr2=0x7f61900cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:57.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.390+0000 7f618fe00640 1 --2- 172.21.15.86:0/1455561590 >> v2:172.21.15.155:3300/0 conn(0x7f61900d1df0 0x7f61900cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.390+0000 7f618fe00640 1 -- 172.21.15.86:0/1455561590 >> v1:172.21.15.155:6789/0 conn(0x7f6190070330 legacy=0x7f61900d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:57.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.390+0000 7f618fe00640 1 -- 172.21.15.86:0/1455561590 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f619015eb80 con 0x7f61900cdcc0 2024-04-25T14:03:57.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.390+0000 7f618fe00640 1 -- 172.21.15.86:0/1455561590 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f6188001680 con 0x7f61900cdcc0 2024-04-25T14:03:57.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.390+0000 7f618fe00640 1 -- 172.21.15.86:0/1455561590 >> v1:172.21.15.86:6789/0 conn(0x7f61900cdcc0 legacy=0x7f61900ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:57.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.390+0000 7f618fe00640 1 --2- 172.21.15.86:0/1455561590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6178003a30 0x7f6178003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:57.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.390+0000 7f618fe00640 1 --2- 172.21.15.86:0/1455561590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6178004810 0x7f6178004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:57.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.390+0000 7f618fe00640 1 --2- 172.21.15.86:0/1455561590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6178005360 0x7f617800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:57.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.390+0000 7f618fe00640 1 -- 172.21.15.86:0/1455561590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f617800c230 con 0x7f6178003a30 2024-04-25T14:03:57.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.390+0000 7f618fe00640 1 -- 172.21.15.86:0/1455561590 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f617800c3e0 con 0x7f6178005360 2024-04-25T14:03:57.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.390+0000 7f618fe00640 1 -- 172.21.15.86:0/1455561590 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f617800c5e0 con 0x7f6178004810 2024-04-25T14:03:57.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.390+0000 7f6195800640 1 --2- 172.21.15.86:0/1455561590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6178003a30 0x7f6178003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:57.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.390+0000 7f6194e00640 1 --2- 172.21.15.86:0/1455561590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6178004810 0x7f6178004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:57.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.390+0000 7f6196200640 1 --2- 172.21.15.86:0/1455561590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6178005360 0x7f617800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:57.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.390+0000 7f6195800640 1 -- 172.21.15.86:0/1455561590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6178004810 msgr2=0x7f6178004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:57.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.390+0000 7f6195800640 1 --2- 172.21.15.86:0/1455561590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6178004810 0x7f6178004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.390+0000 7f6195800640 1 -- 172.21.15.86:0/1455561590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6178005360 msgr2=0x7f617800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:57.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.390+0000 7f6195800640 1 --2- 172.21.15.86:0/1455561590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6178005360 0x7f617800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.390+0000 7f6195800640 1 -- 172.21.15.86:0/1455561590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6178003530 con 0x7f6178003a30 2024-04-25T14:03:57.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.390+0000 7f6196200640 1 -- 172.21.15.86:0/1455561590 reap_dead start 2024-04-25T14:03:57.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.394+0000 7f6195800640 1 --2- 172.21.15.86:0/1455561590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6178003a30 0x7f6178003ea0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6184001e40 tx=0x7f618401c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf728154ef4ab323 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:57.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.394+0000 7f618fe00640 1 -- 172.21.15.86:0/1455561590 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6184001a90 con 0x7f6178003a30 2024-04-25T14:03:57.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.394+0000 7f618fe00640 1 -- 172.21.15.86:0/1455561590 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6184003980 con 0x7f6178003a30 2024-04-25T14:03:57.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.394+0000 7f618fe00640 1 -- 172.21.15.86:0/1455561590 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6184003d70 con 0x7f6178003a30 2024-04-25T14:03:57.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.394+0000 7f6198200640 1 -- 172.21.15.86:0/1455561590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6178003a30 msgr2=0x7f6178003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:57.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.394+0000 7f6198200640 1 --2- 172.21.15.86:0/1455561590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6178003a30 0x7f6178003ea0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6184001e40 tx=0x7f618401c710 comp rx=0 tx=0).stop 2024-04-25T14:03:57.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.394+0000 7f6198200640 1 -- 172.21.15.86:0/1455561590 shutdown_connections 2024-04-25T14:03:57.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.394+0000 7f6198200640 1 --2- 172.21.15.86:0/1455561590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6178003a30 0x7f6178003ea0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.394+0000 7f6198200640 1 -- 172.21.15.86:0/1455561590 >> 172.21.15.86:0/1455561590 conn(0x7f61900b4650 msgr2=0x7f61900b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:57.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.394+0000 7f6198200640 1 -- 172.21.15.86:0/1455561590 shutdown_connections 2024-04-25T14:03:57.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.394+0000 7f6198200640 1 -- 172.21.15.86:0/1455561590 wait complete. 2024-04-25T14:03:57.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.394+0000 7f6198200640 1 Processor -- start 2024-04-25T14:03:57.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.394+0000 7f6198200640 1 -- start start 2024-04-25T14:03:57.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.398+0000 7f6198200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6178003a30 0x7f61901594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:57.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.398+0000 7f6198200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6190160590 0x7f61900af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:57.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.398+0000 7f6198200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61900af9e0 0x7f6190157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:57.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.398+0000 7f6198200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61900708c0 con 0x7f6190160590 2024-04-25T14:03:57.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.398+0000 7f6198200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6190070740 con 0x7f61900af9e0 2024-04-25T14:03:57.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.398+0000 7f6198200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f61900ce0d0 con 0x7f6178003a30 2024-04-25T14:03:57.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.398+0000 7f6194e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6190160590 0x7f61900af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:57.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.398+0000 7f6194e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6190160590 0x7f61900af4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59432/0 (socket says 172.21.15.86:59432) 2024-04-25T14:03:57.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.398+0000 7f6194e00640 1 -- 172.21.15.86:0/4026545401 learned_addr learned my addr 172.21.15.86:0/4026545401 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:57.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.398+0000 7f6195800640 1 --2- 172.21.15.86:0/4026545401 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6178003a30 0x7f61901594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:57.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.398+0000 7f6196200640 1 --2- 172.21.15.86:0/4026545401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61900af9e0 0x7f6190157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:57.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.398+0000 7f6194e00640 1 -- 172.21.15.86:0/4026545401 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6178003a30 msgr2=0x7f61901594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:57.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.398+0000 7f6194e00640 1 --2- 172.21.15.86:0/4026545401 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6178003a30 0x7f61901594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.398+0000 7f6194e00640 1 -- 172.21.15.86:0/4026545401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61900af9e0 msgr2=0x7f6190157a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:57.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.398+0000 7f6194e00640 1 --2- 172.21.15.86:0/4026545401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61900af9e0 0x7f6190157a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.398+0000 7f6194e00640 1 -- 172.21.15.86:0/4026545401 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6190162270 con 0x7f6190160590 2024-04-25T14:03:57.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.398+0000 7f6194e00640 1 --2- 172.21.15.86:0/4026545401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6190160590 0x7f61900af4a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f617c011490 tx=0x7f617c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:57.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.398+0000 7f618d600640 1 -- 172.21.15.86:0/4026545401 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c019020 con 0x7f6190160590 2024-04-25T14:03:57.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.398+0000 7f618d600640 1 -- 172.21.15.86:0/4026545401 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f617c003620 con 0x7f6190160590 2024-04-25T14:03:57.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.398+0000 7f618d600640 1 -- 172.21.15.86:0/4026545401 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c003a10 con 0x7f6190160590 2024-04-25T14:03:57.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.398+0000 7f6198200640 1 -- 172.21.15.86:0/4026545401 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61900afdf0 con 0x7f6190160590 2024-04-25T14:03:57.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.398+0000 7f6198200640 1 -- 172.21.15.86:0/4026545401 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61901f7dd0 con 0x7f6190160590 2024-04-25T14:03:57.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.398+0000 7f6198200640 1 -- 172.21.15.86:0/4026545401 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6190069d00 con 0x7f6190160590 2024-04-25T14:03:57.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.402+0000 7f618d600640 1 -- 172.21.15.86:0/4026545401 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f617c01c020 con 0x7f6190160590 2024-04-25T14:03:57.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.402+0000 7f618d600640 1 --2- 172.21.15.86:0/4026545401 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f617006f400 0x7f6170071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:57.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.402+0000 7f618d600640 1 -- 172.21.15.86:0/4026545401 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f617c08f6a0 con 0x7f6190160590 2024-04-25T14:03:57.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.402+0000 7f6195800640 1 --2- 172.21.15.86:0/4026545401 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f617006f400 0x7f6170071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:57.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.402+0000 7f6195800640 1 --2- 172.21.15.86:0/4026545401 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f617006f400 0x7f6170071910 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6184005000 tx=0x7f6184001520 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:57.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.406+0000 7f618d600640 1 -- 172.21.15.86:0/4026545401 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f617c023070 con 0x7f6190160590 2024-04-25T14:03:57.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.562+0000 7f6198200640 1 -- 172.21.15.86:0/4026545401 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f61901f7880 con 0x7f6190160590 2024-04-25T14:03:57.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.566+0000 7f618d600640 1 -- 172.21.15.86:0/4026545401 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7f61900afdf0 con 0x7f6190160590 2024-04-25T14:03:57.566 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:03:57.566 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":24,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:03:55.899092+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:03:49.151473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:03:49.802597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T14:03:53.492008+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:03:57.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.566+0000 7f6198200640 1 -- 172.21.15.86:0/4026545401 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f617006f400 msgr2=0x7f6170071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:57.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.566+0000 7f6198200640 1 --2- 172.21.15.86:0/4026545401 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f617006f400 0x7f6170071910 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6184005000 tx=0x7f6184001520 comp rx=0 tx=0).stop 2024-04-25T14:03:57.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.566+0000 7f6198200640 1 -- 172.21.15.86:0/4026545401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6190160590 msgr2=0x7f61900af4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:57.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.566+0000 7f6198200640 1 --2- 172.21.15.86:0/4026545401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6190160590 0x7f61900af4a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f617c011490 tx=0x7f617c011890 comp rx=0 tx=0).stop 2024-04-25T14:03:57.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.566+0000 7f6198200640 1 -- 172.21.15.86:0/4026545401 shutdown_connections 2024-04-25T14:03:57.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.566+0000 7f6198200640 1 --2- 172.21.15.86:0/4026545401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61900af9e0 0x7f6190157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.566+0000 7f6198200640 1 --2- 172.21.15.86:0/4026545401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6190160590 0x7f61900af4a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.566+0000 7f6198200640 1 --2- 172.21.15.86:0/4026545401 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f617006f400 0x7f6170071910 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.566+0000 7f6198200640 1 --2- 172.21.15.86:0/4026545401 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6178003a30 0x7f61901594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.566+0000 7f6198200640 1 -- 172.21.15.86:0/4026545401 >> 172.21.15.86:0/4026545401 conn(0x7f61900b4650 msgr2=0x7f61900b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:57.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.566+0000 7f6198200640 1 -- 172.21.15.86:0/4026545401 shutdown_connections 2024-04-25T14:03:57.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.566+0000 7f6198200640 1 -- 172.21.15.86:0/4026545401 wait complete. 2024-04-25T14:03:57.603 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:03:57.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.750+0000 7f7a2de00640 1 Processor -- start 2024-04-25T14:03:57.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.750+0000 7f7a2de00640 1 -- start start 2024-04-25T14:03:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.750+0000 7f7a2de00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7a28070330 0x7f7a280d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.750+0000 7f7a2de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7a280d1c20 0x7f7a2806ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.750+0000 7f7a2de00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a280d2030 con 0x7f7a2806f270 2024-04-25T14:03:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.750+0000 7f7a2de00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7a2815e5c0 con 0x7f7a280d1c20 2024-04-25T14:03:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.750+0000 7f7a2de00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f7a2815e8a0 con 0x7f7a28070330 2024-04-25T14:03:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.750+0000 7f7a1e000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7a280d1c20 0x7f7a2806ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.750+0000 7f7a1e000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7a280d1c20 0x7f7a2806ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59448/0 (socket says 172.21.15.86:59448) 2024-04-25T14:03:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.750+0000 7f7a1e000640 1 -- 172.21.15.86:0/1196327303 learned_addr learned my addr 172.21.15.86:0/1196327303 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.750+0000 7f7a26a00640 1 --2- 172.21.15.86:0/1196327303 >> v2:172.21.15.155:3300/0 conn(0x7f7a28070330 0x7f7a280d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:57.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.750+0000 7f7a1e000640 1 -- 172.21.15.86:0/1196327303 >> v2:172.21.15.155:3300/0 conn(0x7f7a28070330 msgr2=0x7f7a280d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:57.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.750+0000 7f7a1e000640 1 --2- 172.21.15.86:0/1196327303 >> v2:172.21.15.155:3300/0 conn(0x7f7a28070330 0x7f7a280d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.750+0000 7f7a1e000640 1 -- 172.21.15.86:0/1196327303 >> v1:172.21.15.86:6789/0 conn(0x7f7a2806f270 legacy=0x7f7a2806f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:57.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.750+0000 7f7a1e000640 1 -- 172.21.15.86:0/1196327303 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2815eb80 con 0x7f7a280d1c20 2024-04-25T14:03:57.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.754+0000 7f7a1e000640 1 --2- 172.21.15.86:0/1196327303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a280d1c20 0x7f7a2806ed30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009870 tx=0x7f7a1001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7c47784081bc57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:57.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.754+0000 7f7a26000640 1 -- 172.21.15.86:0/1196327303 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a100b0070 con 0x7f7a280d1c20 2024-04-25T14:03:57.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.754+0000 7f7a26000640 1 -- 172.21.15.86:0/1196327303 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a100ab600 con 0x7f7a280d1c20 2024-04-25T14:03:57.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.754+0000 7f7a26000640 1 -- 172.21.15.86:0/1196327303 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a100ab9d0 con 0x7f7a280d1c20 2024-04-25T14:03:57.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.754+0000 7f7a2de00640 1 -- 172.21.15.86:0/1196327303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a280d1c20 msgr2=0x7f7a2806ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:57.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.754+0000 7f7a2de00640 1 --2- 172.21.15.86:0/1196327303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a280d1c20 0x7f7a2806ed30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009870 tx=0x7f7a1001c490 comp rx=0 tx=0).stop 2024-04-25T14:03:57.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.754+0000 7f7a2de00640 1 -- 172.21.15.86:0/1196327303 shutdown_connections 2024-04-25T14:03:57.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.754+0000 7f7a2de00640 1 --2- 172.21.15.86:0/1196327303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a280d1c20 0x7f7a2806ed30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.754+0000 7f7a2de00640 1 --2- 172.21.15.86:0/1196327303 >> v2:172.21.15.155:3300/0 conn(0x7f7a28070330 0x7f7a280d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.754+0000 7f7a2de00640 1 -- 172.21.15.86:0/1196327303 >> 172.21.15.86:0/1196327303 conn(0x7f7a280b4650 msgr2=0x7f7a280b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:57.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.754+0000 7f7a2de00640 1 -- 172.21.15.86:0/1196327303 shutdown_connections 2024-04-25T14:03:57.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.754+0000 7f7a2de00640 1 -- 172.21.15.86:0/1196327303 wait complete. 2024-04-25T14:03:57.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.758+0000 7f7a2de00640 1 Processor -- start 2024-04-25T14:03:57.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.758+0000 7f7a2de00640 1 -- start start 2024-04-25T14:03:57.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.758+0000 7f7a2de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a2806f270 0x7f7a281fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:57.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.758+0000 7f7a2de00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a28070330 0x7f7a280c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:57.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.758+0000 7f7a2de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a280d1c20 0x7f7a280af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:57.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.758+0000 7f7a2de00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7a280708c0 con 0x7f7a2806f270 2024-04-25T14:03:57.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.758+0000 7f7a2de00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7a28070740 con 0x7f7a280d1c20 2024-04-25T14:03:57.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.758+0000 7f7a2de00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7a2816afc0 con 0x7f7a28070330 2024-04-25T14:03:57.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.758+0000 7f7a26a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a2806f270 0x7f7a281fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:57.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.758+0000 7f7a26a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a2806f270 0x7f7a281fc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59454/0 (socket says 172.21.15.86:59454) 2024-04-25T14:03:57.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.758+0000 7f7a26a00640 1 -- 172.21.15.86:0/3699726710 learned_addr learned my addr 172.21.15.86:0/3699726710 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:57.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.758+0000 7f7a1e000640 1 --2- 172.21.15.86:0/3699726710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a28070330 0x7f7a280c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:57.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.758+0000 7f7a27400640 1 --2- 172.21.15.86:0/3699726710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a280d1c20 0x7f7a280af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:57.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.758+0000 7f7a26a00640 1 -- 172.21.15.86:0/3699726710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a28070330 msgr2=0x7f7a280c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:57.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.758+0000 7f7a26a00640 1 --2- 172.21.15.86:0/3699726710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a28070330 0x7f7a280c88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.758+0000 7f7a26a00640 1 -- 172.21.15.86:0/3699726710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a280d1c20 msgr2=0x7f7a280af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:57.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.758+0000 7f7a26a00640 1 --2- 172.21.15.86:0/3699726710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a280d1c20 0x7f7a280af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.758+0000 7f7a26a00640 1 -- 172.21.15.86:0/3699726710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a28072bf0 con 0x7f7a2806f270 2024-04-25T14:03:57.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.758+0000 7f7a27400640 1 --2- 172.21.15.86:0/3699726710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a280d1c20 0x7f7a280af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:57.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.758+0000 7f7a26a00640 1 --2- 172.21.15.86:0/3699726710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a2806f270 0x7f7a281fc7e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800d600 tx=0x7f7a1800dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:57.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.758+0000 7f7a1fe00640 1 -- 172.21.15.86:0/3699726710 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18004240 con 0x7f7a2806f270 2024-04-25T14:03:57.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.758+0000 7f7a1fe00640 1 -- 172.21.15.86:0/3699726710 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a180043e0 con 0x7f7a2806f270 2024-04-25T14:03:57.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.758+0000 7f7a1fe00640 1 -- 172.21.15.86:0/3699726710 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18003840 con 0x7f7a2806f270 2024-04-25T14:03:57.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.758+0000 7f7a2de00640 1 -- 172.21.15.86:0/3699726710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a28072ed0 con 0x7f7a2806f270 2024-04-25T14:03:57.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.758+0000 7f7a2de00640 1 -- 172.21.15.86:0/3699726710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a28073490 con 0x7f7a2806f270 2024-04-25T14:03:57.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.762+0000 7f7a2de00640 1 -- 172.21.15.86:0/3699726710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a280736d0 con 0x7f7a2806f270 2024-04-25T14:03:57.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.762+0000 7f7a1fe00640 1 -- 172.21.15.86:0/3699726710 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7a180039e0 con 0x7f7a2806f270 2024-04-25T14:03:57.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.762+0000 7f7a1fe00640 1 --2- 172.21.15.86:0/3699726710 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7a0406f400 0x7f7a04071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:57.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.762+0000 7f7a1fe00640 1 -- 172.21.15.86:0/3699726710 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7a1808f890 con 0x7f7a2806f270 2024-04-25T14:03:57.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.762+0000 7f7a1e000640 1 --2- 172.21.15.86:0/3699726710 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7a0406f400 0x7f7a04071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:57.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.762+0000 7f7a1e000640 1 --2- 172.21.15.86:0/3699726710 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7a0406f400 0x7f7a04071910 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7a100ad000 tx=0x7f7a100a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:57.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.766+0000 7f7a1fe00640 1 -- 172.21.15.86:0/3699726710 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7a280736d0 con 0x7f7a2806f270 2024-04-25T14:03:57.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.954+0000 7f7a2de00640 1 -- 172.21.15.86:0/3699726710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a281fcf00 con 0x7f7a2806f270 2024-04-25T14:03:57.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.954+0000 7f7a1fe00640 1 -- 172.21.15.86:0/3699726710 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5177 (secure 0 0 0) 0x7f7a1805c3a0 con 0x7f7a2806f270 2024-04-25T14:03:57.956 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:03:57.956 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/347107112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":347107112},{"type":"v1","addr":"172.21.15.155:6835","nonce":347107112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2236737549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2236737549},{"type":"v1","addr":"172.21.15.86:6837","nonce":2236737549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/519560229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":519560229},{"type":"v1","addr":"172.21.15.86:6836","nonce":519560229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:03:50.852257+0000","modified":"2024-04-25T14:03:55.899313+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6833/1944249570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1944249570},{"type":"v1","addr":"172.21.15.155:6833","nonce":1944249570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4275,"qdb_cluster":[4275]},"id":1}]} 2024-04-25T14:03:57.956 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 7 2024-04-25T14:03:57.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.958+0000 7f7a2de00640 1 -- 172.21.15.86:0/3699726710 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7a0406f400 msgr2=0x7f7a04071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:57.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.958+0000 7f7a2de00640 1 --2- 172.21.15.86:0/3699726710 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7a0406f400 0x7f7a04071910 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7a100ad000 tx=0x7f7a100a6000 comp rx=0 tx=0).stop 2024-04-25T14:03:57.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.958+0000 7f7a2de00640 1 -- 172.21.15.86:0/3699726710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a2806f270 msgr2=0x7f7a281fc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:57.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.958+0000 7f7a2de00640 1 --2- 172.21.15.86:0/3699726710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a2806f270 0x7f7a281fc7e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800d600 tx=0x7f7a1800dac0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.958+0000 7f7a2de00640 1 -- 172.21.15.86:0/3699726710 shutdown_connections 2024-04-25T14:03:57.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.958+0000 7f7a2de00640 1 --2- 172.21.15.86:0/3699726710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a280d1c20 0x7f7a280af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.958+0000 7f7a2de00640 1 --2- 172.21.15.86:0/3699726710 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7a0406f400 0x7f7a04071910 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.958+0000 7f7a2de00640 1 --2- 172.21.15.86:0/3699726710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a28070330 0x7f7a280c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.958+0000 7f7a2de00640 1 --2- 172.21.15.86:0/3699726710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a2806f270 0x7f7a281fc7e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:57.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.958+0000 7f7a2de00640 1 -- 172.21.15.86:0/3699726710 >> 172.21.15.86:0/3699726710 conn(0x7f7a280b4650 msgr2=0x7f7a280b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:57.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.958+0000 7f7a2de00640 1 -- 172.21.15.86:0/3699726710 shutdown_connections 2024-04-25T14:03:57.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:57.958+0000 7f7a2de00640 1 -- 172.21.15.86:0/3699726710 wait complete. 2024-04-25T14:03:57.977 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:03:58.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.122+0000 7fb10e000640 1 Processor -- start 2024-04-25T14:03:58.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.122+0000 7fb10e000640 1 -- start start 2024-04-25T14:03:58.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.126+0000 7fb10e000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1080cfca0 0x7fb10806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.126+0000 7fb10e000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb1080d16e0 0x7fb1080d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.126+0000 7fb10e000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb10806c670 0x7fb10806ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.126+0000 7fb10e000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb108070470 con 0x7fb1080d16e0 2024-04-25T14:03:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.126+0000 7fb10e000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb1080d2030 con 0x7fb10806c670 2024-04-25T14:03:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.126+0000 7fb10e000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb108072bd0 con 0x7fb1080cfca0 2024-04-25T14:03:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.126+0000 7fb0ffe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb1080d16e0 0x7fb1080d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.126+0000 7fb0ffe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb1080d16e0 0x7fb1080d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59466/0 (socket says 172.21.15.86:59466) 2024-04-25T14:03:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.126+0000 7fb0ffe00640 1 -- 172.21.15.86:0/4166703753 learned_addr learned my addr 172.21.15.86:0/4166703753 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.126+0000 7fb107e00640 1 --2- 172.21.15.86:0/4166703753 >> v2:172.21.15.155:3300/0 conn(0x7fb10806c670 0x7fb10806ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.126+0000 7fb107400640 1 --2- 172.21.15.86:0/4166703753 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1080cfca0 0x7fb10806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.126+0000 7fb0ffe00640 1 -- 172.21.15.86:0/4166703753 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1080cfca0 msgr2=0x7fb10806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.126+0000 7fb0ffe00640 1 --2- 172.21.15.86:0/4166703753 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1080cfca0 0x7fb10806f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.126+0000 7fb0ffe00640 1 -- 172.21.15.86:0/4166703753 >> v2:172.21.15.155:3300/0 conn(0x7fb10806c670 msgr2=0x7fb10806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.126+0000 7fb0ffe00640 1 --2- 172.21.15.86:0/4166703753 >> v2:172.21.15.155:3300/0 conn(0x7fb10806c670 0x7fb10806ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.126+0000 7fb0ffe00640 1 -- 172.21.15.86:0/4166703753 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10806f930 con 0x7fb1080d16e0 2024-04-25T14:03:58.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.130+0000 7fb0ffe00640 1 --2- 172.21.15.86:0/4166703753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1080d16e0 0x7fb1080d1af0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb10806e060 tx=0x7fb0f001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2b47b116339ed7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:58.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.130+0000 7fb106a00640 1 -- 172.21.15.86:0/4166703753 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f00ab5f0 con 0x7fb1080d16e0 2024-04-25T14:03:58.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.130+0000 7fb106a00640 1 -- 172.21.15.86:0/4166703753 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0f00b0070 con 0x7fb1080d16e0 2024-04-25T14:03:58.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.130+0000 7fb106a00640 1 -- 172.21.15.86:0/4166703753 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f00ab9b0 con 0x7fb1080d16e0 2024-04-25T14:03:58.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.130+0000 7fb10e000640 1 -- 172.21.15.86:0/4166703753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1080d16e0 msgr2=0x7fb1080d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:58.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.130+0000 7fb10e000640 1 --2- 172.21.15.86:0/4166703753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1080d16e0 0x7fb1080d1af0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb10806e060 tx=0x7fb0f001c520 comp rx=0 tx=0).stop 2024-04-25T14:03:58.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.130+0000 7fb10e000640 1 -- 172.21.15.86:0/4166703753 shutdown_connections 2024-04-25T14:03:58.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.130+0000 7fb10e000640 1 --2- 172.21.15.86:0/4166703753 >> v2:172.21.15.155:3300/0 conn(0x7fb10806c670 0x7fb10806ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.130+0000 7fb10e000640 1 --2- 172.21.15.86:0/4166703753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1080d16e0 0x7fb1080d1af0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.130+0000 7fb10e000640 1 --2- 172.21.15.86:0/4166703753 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1080cfca0 0x7fb10806f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.130+0000 7fb10e000640 1 -- 172.21.15.86:0/4166703753 >> 172.21.15.86:0/4166703753 conn(0x7fb1080b4650 msgr2=0x7fb1080b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:58.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.130+0000 7fb10e000640 1 -- 172.21.15.86:0/4166703753 shutdown_connections 2024-04-25T14:03:58.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.130+0000 7fb10e000640 1 -- 172.21.15.86:0/4166703753 wait complete. 2024-04-25T14:03:58.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.134+0000 7fb10e000640 1 Processor -- start 2024-04-25T14:03:58.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.134+0000 7fb10e000640 1 -- start start 2024-04-25T14:03:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.134+0000 7fb10e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb10806c670 0x7fb10809b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.134+0000 7fb10e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1080cfca0 0x7fb1081f99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.134+0000 7fb10e000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1080d16e0 0x7fb10814aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.134+0000 7fb10e000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb1080738c0 con 0x7fb10806c670 2024-04-25T14:03:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.134+0000 7fb10e000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb10806d3d0 con 0x7fb1080cfca0 2024-04-25T14:03:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.134+0000 7fb10e000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb108073a40 con 0x7fb1080d16e0 2024-04-25T14:03:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.134+0000 7fb107400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb10806c670 0x7fb10809b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.134+0000 7fb107400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb10806c670 0x7fb10809b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59474/0 (socket says 172.21.15.86:59474) 2024-04-25T14:03:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.134+0000 7fb107400640 1 -- 172.21.15.86:0/5209343 learned_addr learned my addr 172.21.15.86:0/5209343 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.134+0000 7fb0ffe00640 1 --2- 172.21.15.86:0/5209343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1080cfca0 0x7fb1081f99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.134+0000 7fb107e00640 1 --2- 172.21.15.86:0/5209343 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1080d16e0 0x7fb10814aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.134+0000 7fb107400640 1 -- 172.21.15.86:0/5209343 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1080d16e0 msgr2=0x7fb10814aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.134+0000 7fb107400640 1 --2- 172.21.15.86:0/5209343 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1080d16e0 0x7fb10814aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.134+0000 7fb107400640 1 -- 172.21.15.86:0/5209343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1080cfca0 msgr2=0x7fb1081f99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.134+0000 7fb107400640 1 --2- 172.21.15.86:0/5209343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1080cfca0 0x7fb1081f99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.134+0000 7fb107400640 1 -- 172.21.15.86:0/5209343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10809d0c0 con 0x7fb10806c670 2024-04-25T14:03:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.134+0000 7fb107e00640 1 --2- 172.21.15.86:0/5209343 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1080d16e0 0x7fb10814aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.134+0000 7fb107400640 1 --2- 172.21.15.86:0/5209343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb10806c670 0x7fb10809b2b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800d600 tx=0x7fb0f800dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.134+0000 7fb0ff400640 1 -- 172.21.15.86:0/5209343 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8004240 con 0x7fb10806c670 2024-04-25T14:03:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.134+0000 7fb0ff400640 1 -- 172.21.15.86:0/5209343 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0f80043e0 con 0x7fb10806c670 2024-04-25T14:03:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.134+0000 7fb10e000640 1 -- 172.21.15.86:0/5209343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb10809d3a0 con 0x7fb10806c670 2024-04-25T14:03:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.134+0000 7fb0ff400640 1 -- 172.21.15.86:0/5209343 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8003840 con 0x7fb10806c670 2024-04-25T14:03:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.134+0000 7fb10e000640 1 -- 172.21.15.86:0/5209343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb10809d960 con 0x7fb10806c670 2024-04-25T14:03:58.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.138+0000 7fb10e000640 1 -- 172.21.15.86:0/5209343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb10809f4b0 con 0x7fb10806c670 2024-04-25T14:03:58.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.138+0000 7fb0ff400640 1 -- 172.21.15.86:0/5209343 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb0f80039e0 con 0x7fb10806c670 2024-04-25T14:03:58.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.138+0000 7fb0ff400640 1 --2- 172.21.15.86:0/5209343 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb0e406f430 0x7fb0e4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.138+0000 7fb0ff400640 1 -- 172.21.15.86:0/5209343 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb0f808f680 con 0x7fb10806c670 2024-04-25T14:03:58.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.138+0000 7fb0ffe00640 1 --2- 172.21.15.86:0/5209343 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb0e406f430 0x7fb0e4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.138+0000 7fb0ffe00640 1 --2- 172.21.15.86:0/5209343 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb0e406f430 0x7fb0e4071940 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00af000 tx=0x7fb0f00a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:58.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.142+0000 7fb0ff400640 1 -- 172.21.15.86:0/5209343 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb0f805c220 con 0x7fb10806c670 2024-04-25T14:03:58.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.306+0000 7fb10e000640 1 -- 172.21.15.86:0/5209343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb108069d00 con 0x7fb10806c670 2024-04-25T14:03:58.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.306+0000 7fb0ff400640 1 -- 172.21.15.86:0/5209343 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7fb0f805bb20 con 0x7fb10806c670 2024-04-25T14:03:58.307 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:03:58.308 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":24,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:03:55.899092+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:03:49.151473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:03:49.802597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T14:03:53.492008+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:03:58.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.310+0000 7fb10e000640 1 -- 172.21.15.86:0/5209343 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb0e406f430 msgr2=0x7fb0e4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:58.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.310+0000 7fb10e000640 1 --2- 172.21.15.86:0/5209343 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb0e406f430 0x7fb0e4071940 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00af000 tx=0x7fb0f00a7000 comp rx=0 tx=0).stop 2024-04-25T14:03:58.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.310+0000 7fb10e000640 1 -- 172.21.15.86:0/5209343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb10806c670 msgr2=0x7fb10809b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:58.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.310+0000 7fb10e000640 1 --2- 172.21.15.86:0/5209343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb10806c670 0x7fb10809b2b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800d600 tx=0x7fb0f800dac0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.310+0000 7fb10e000640 1 -- 172.21.15.86:0/5209343 shutdown_connections 2024-04-25T14:03:58.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.310+0000 7fb10e000640 1 --2- 172.21.15.86:0/5209343 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb0e406f430 0x7fb0e4071940 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.310+0000 7fb10e000640 1 --2- 172.21.15.86:0/5209343 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1080d16e0 0x7fb10814aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.310+0000 7fb10e000640 1 --2- 172.21.15.86:0/5209343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1080cfca0 0x7fb1081f99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.310+0000 7fb10e000640 1 --2- 172.21.15.86:0/5209343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb10806c670 0x7fb10809b2b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.310+0000 7fb10e000640 1 -- 172.21.15.86:0/5209343 >> 172.21.15.86:0/5209343 conn(0x7fb1080b4650 msgr2=0x7fb1080b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:58.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.310+0000 7fb10e000640 1 -- 172.21.15.86:0/5209343 shutdown_connections 2024-04-25T14:03:58.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.310+0000 7fb10e000640 1 -- 172.21.15.86:0/5209343 wait complete. 2024-04-25T14:03:58.731 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:03:50.852257+0000', 'modified': '2024-04-25T14:03:55.899313+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4275}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4275': {'gid': 4275, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.155:6833/1944249570', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6832', 'nonce': 1944249570}, {'type': 'v1', 'addr': '172.21.15.155:6833', 'nonce': 1944249570}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4275, 'qdb_cluster': [4275]} 2024-04-25T14:03:58.732 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T14:03:58.732 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T14:03:58.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.878+0000 7f3b14000640 1 Processor -- start 2024-04-25T14:03:58.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.878+0000 7f3b14000640 1 -- start start 2024-04-25T14:03:58.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.882+0000 7f3b14000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b0c15c2a0 0x7f3b0c0ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.882+0000 7f3b14000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3b0c06fd50 0x7f3b0c070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.882+0000 7f3b14000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3b0c06c670 0x7f3b0c06ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.882+0000 7f3b14000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3b0c0b0a60 con 0x7f3b0c06fd50 2024-04-25T14:03:58.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.882+0000 7f3b14000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3b0c06ef90 con 0x7f3b0c06c670 2024-04-25T14:03:58.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.882+0000 7f3b14000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3b0c06f270 con 0x7f3b0c15c2a0 2024-04-25T14:03:58.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.882+0000 7f3b10c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3b0c06fd50 0x7f3b0c070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.882+0000 7f3b10c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3b0c06fd50 0x7f3b0c070160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36182/0 (socket says 172.21.15.86:36182) 2024-04-25T14:03:58.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.882+0000 7f3b10c00640 1 -- 172.21.15.86:0/517936228 learned_addr learned my addr 172.21.15.86:0/517936228 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:58.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.882+0000 7f3b12000640 1 --2- 172.21.15.86:0/517936228 >> v2:172.21.15.155:3300/0 conn(0x7f3b0c06c670 0x7f3b0c06ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.882+0000 7f3b11600640 1 --2- 172.21.15.86:0/517936228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b0c15c2a0 0x7f3b0c0ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.882+0000 7f3b10c00640 1 -- 172.21.15.86:0/517936228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b0c15c2a0 msgr2=0x7f3b0c0ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:58.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.882+0000 7f3b10c00640 1 --2- 172.21.15.86:0/517936228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b0c15c2a0 0x7f3b0c0ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.882+0000 7f3b10c00640 1 -- 172.21.15.86:0/517936228 >> v2:172.21.15.155:3300/0 conn(0x7f3b0c06c670 msgr2=0x7f3b0c06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:58.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.882+0000 7f3b10c00640 1 --2- 172.21.15.86:0/517936228 >> v2:172.21.15.155:3300/0 conn(0x7f3b0c06c670 0x7f3b0c06ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.882+0000 7f3b10c00640 1 -- 172.21.15.86:0/517936228 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0c06f550 con 0x7f3b0c06fd50 2024-04-25T14:03:58.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.882+0000 7f3b11600640 1 --2- 172.21.15.86:0/517936228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b0c15c2a0 0x7f3b0c0ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:58.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.886+0000 7f3b10c00640 1 --2- 172.21.15.86:0/517936228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b0c06fd50 0x7f3b0c070160 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c074f10 tx=0x7f3af801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14fd12e711d402f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:58.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.886+0000 7f3b0be00640 1 -- 172.21.15.86:0/517936228 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af80b2070 con 0x7f3b0c06fd50 2024-04-25T14:03:58.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.886+0000 7f3b0be00640 1 -- 172.21.15.86:0/517936228 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af80a9070 con 0x7f3b0c06fd50 2024-04-25T14:03:58.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.886+0000 7f3b0be00640 1 -- 172.21.15.86:0/517936228 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af80ad880 con 0x7f3b0c06fd50 2024-04-25T14:03:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.886+0000 7f3b14000640 1 -- 172.21.15.86:0/517936228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b0c06fd50 msgr2=0x7f3b0c070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.886+0000 7f3b14000640 1 --2- 172.21.15.86:0/517936228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b0c06fd50 0x7f3b0c070160 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c074f10 tx=0x7f3af801c460 comp rx=0 tx=0).stop 2024-04-25T14:03:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.886+0000 7f3b14000640 1 -- 172.21.15.86:0/517936228 shutdown_connections 2024-04-25T14:03:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.886+0000 7f3b14000640 1 --2- 172.21.15.86:0/517936228 >> v2:172.21.15.155:3300/0 conn(0x7f3b0c06c670 0x7f3b0c06ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.886+0000 7f3b14000640 1 --2- 172.21.15.86:0/517936228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b0c06fd50 0x7f3b0c070160 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.886+0000 7f3b14000640 1 --2- 172.21.15.86:0/517936228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b0c15c2a0 0x7f3b0c0ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.886+0000 7f3b14000640 1 -- 172.21.15.86:0/517936228 >> 172.21.15.86:0/517936228 conn(0x7f3b0c0b4650 msgr2=0x7f3b0c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.886+0000 7f3b14000640 1 -- 172.21.15.86:0/517936228 shutdown_connections 2024-04-25T14:03:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.886+0000 7f3b14000640 1 -- 172.21.15.86:0/517936228 wait complete. 2024-04-25T14:03:58.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.890+0000 7f3b14000640 1 Processor -- start 2024-04-25T14:03:58.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.890+0000 7f3b14000640 1 -- start start 2024-04-25T14:03:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.890+0000 7f3b14000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b0c06c670 0x7f3b0c15ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.890+0000 7f3b14000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b0c06fd50 0x7f3b0c09a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.890+0000 7f3b14000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b0c15c2a0 0x7f3b0c164000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.890+0000 7f3b14000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c15c6b0 con 0x7f3b0c06fd50 2024-04-25T14:03:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.890+0000 7f3b14000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c06d3d0 con 0x7f3b0c15c2a0 2024-04-25T14:03:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.890+0000 7f3b14000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3b0c15c830 con 0x7f3b0c06c670 2024-04-25T14:03:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.890+0000 7f3b10c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b0c06fd50 0x7f3b0c09a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.890+0000 7f3b10c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b0c06fd50 0x7f3b0c09a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36196/0 (socket says 172.21.15.86:36196) 2024-04-25T14:03:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.890+0000 7f3b10c00640 1 -- 172.21.15.86:0/2507410991 learned_addr learned my addr 172.21.15.86:0/2507410991 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.890+0000 7f3b11600640 1 --2- 172.21.15.86:0/2507410991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b0c06c670 0x7f3b0c15ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.890+0000 7f3b12000640 1 --2- 172.21.15.86:0/2507410991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b0c15c2a0 0x7f3b0c164000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.890+0000 7f3b10c00640 1 -- 172.21.15.86:0/2507410991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b0c06c670 msgr2=0x7f3b0c15ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.890+0000 7f3b10c00640 1 --2- 172.21.15.86:0/2507410991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b0c06c670 0x7f3b0c15ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.890+0000 7f3b10c00640 1 -- 172.21.15.86:0/2507410991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b0c15c2a0 msgr2=0x7f3b0c164000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.890+0000 7f3b10c00640 1 --2- 172.21.15.86:0/2507410991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b0c15c2a0 0x7f3b0c164000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.890+0000 7f3b10c00640 1 -- 172.21.15.86:0/2507410991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0c09ae90 con 0x7f3b0c06fd50 2024-04-25T14:03:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.890+0000 7f3b12000640 1 --2- 172.21.15.86:0/2507410991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b0c15c2a0 0x7f3b0c164000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.890+0000 7f3b10c00640 1 --2- 172.21.15.86:0/2507410991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b0c06fd50 0x7f3b0c09a950 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3af80b0040 tx=0x7f3af8019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.890+0000 7f3b09600640 1 -- 172.21.15.86:0/2507410991 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af80bb070 con 0x7f3b0c06fd50 2024-04-25T14:03:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.890+0000 7f3b09600640 1 -- 172.21.15.86:0/2507410991 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af80addf0 con 0x7f3b0c06fd50 2024-04-25T14:03:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.890+0000 7f3b14000640 1 -- 172.21.15.86:0/2507410991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b0c166d30 con 0x7f3b0c06fd50 2024-04-25T14:03:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.890+0000 7f3b09600640 1 -- 172.21.15.86:0/2507410991 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af80b2040 con 0x7f3b0c06fd50 2024-04-25T14:03:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.890+0000 7f3b14000640 1 -- 172.21.15.86:0/2507410991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b0c1672f0 con 0x7f3b0c06fd50 2024-04-25T14:03:58.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.894+0000 7f3b09600640 1 -- 172.21.15.86:0/2507410991 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3af80be070 con 0x7f3b0c06fd50 2024-04-25T14:03:58.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.894+0000 7f3b09600640 1 --2- 172.21.15.86:0/2507410991 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3aec06f400 0x7f3aec071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.894+0000 7f3b09600640 1 -- 172.21.15.86:0/2507410991 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3af8128660 con 0x7f3b0c06fd50 2024-04-25T14:03:58.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.894+0000 7f3b11600640 1 --2- 172.21.15.86:0/2507410991 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3aec06f400 0x7f3aec071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.894+0000 7f3b1b641000 1 Processor -- start 2024-04-25T14:03:58.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.894+0000 7f3b1b641000 1 -- start start 2024-04-25T14:03:58.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.894+0000 7f3b11600640 1 --2- 172.21.15.86:0/2507410991 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3aec06f400 0x7f3aec071910 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c0b4200 tx=0x7f3b00008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b1b641000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ab3574c20 0x561ab36d2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b1b641000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ab3560ea0 0x561ab36d3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b1b641000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab3159280 0x561ab36d3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b1b641000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561ab3542160 con 0x561ab3159280 2024-04-25T14:03:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b1b641000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561ab34d7e20 con 0x561ab3574c20 2024-04-25T14:03:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b1b641000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x561ab3490a20 con 0x561ab3560ea0 2024-04-25T14:03:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b12000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab3159280 0x561ab36d3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b12000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab3159280 0x561ab36d3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36206/0 (socket says 172.21.15.86:36206) 2024-04-25T14:03:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b12000640 1 -- 172.21.15.86:0/1963156092 learned_addr learned my addr 172.21.15.86:0/1963156092 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b10c00640 1 --2- 172.21.15.86:0/1963156092 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ab3560ea0 0x561ab36d3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b11600640 1 --2- 172.21.15.86:0/1963156092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ab3574c20 0x561ab36d2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b12000640 1 -- 172.21.15.86:0/1963156092 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ab3560ea0 msgr2=0x561ab36d3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b12000640 1 --2- 172.21.15.86:0/1963156092 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ab3560ea0 0x561ab36d3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b12000640 1 -- 172.21.15.86:0/1963156092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ab3574c20 msgr2=0x561ab36d2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b12000640 1 --2- 172.21.15.86:0/1963156092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ab3574c20 0x561ab36d2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b12000640 1 -- 172.21.15.86:0/1963156092 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ab3495100 con 0x561ab3159280 2024-04-25T14:03:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b11600640 1 --2- 172.21.15.86:0/1963156092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ab3574c20 0x561ab36d2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b12000640 1 --2- 172.21.15.86:0/1963156092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab3159280 0x561ab36d3960 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3b04007360 tx=0x7f3b04007820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d11a870b04e4c33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3afdc00640 1 -- 172.21.15.86:0/1963156092 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b040147e0 con 0x561ab3159280 2024-04-25T14:03:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3afdc00640 1 -- 172.21.15.86:0/1963156092 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b0401a070 con 0x561ab3159280 2024-04-25T14:03:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3afdc00640 1 -- 172.21.15.86:0/1963156092 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b04014b80 con 0x561ab3159280 2024-04-25T14:03:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b1b641000 1 -- 172.21.15.86:0/1963156092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab3159280 msgr2=0x561ab36d3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b1b641000 1 --2- 172.21.15.86:0/1963156092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab3159280 0x561ab36d3960 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3b04007360 tx=0x7f3b04007820 comp rx=0 tx=0).stop 2024-04-25T14:03:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b1b641000 1 -- 172.21.15.86:0/1963156092 shutdown_connections 2024-04-25T14:03:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b1b641000 1 --2- 172.21.15.86:0/1963156092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab3159280 0x561ab36d3960 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b1b641000 1 --2- 172.21.15.86:0/1963156092 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ab3560ea0 0x561ab36d3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b1b641000 1 --2- 172.21.15.86:0/1963156092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ab3574c20 0x561ab36d2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b1b641000 1 -- 172.21.15.86:0/1963156092 >> 172.21.15.86:0/1963156092 conn(0x561ab3198ff0 msgr2=0x561ab36884c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b1b641000 1 -- 172.21.15.86:0/1963156092 shutdown_connections 2024-04-25T14:03:58.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b1b641000 1 -- 172.21.15.86:0/1963156092 wait complete. 2024-04-25T14:03:58.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b1b641000 1 -- start start 2024-04-25T14:03:58.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.898+0000 7f3b1b641000 1 Processor -- start 2024-04-25T14:03:58.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3b1b641000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab3159280 0x561ab36da070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3b1b641000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ab3560ea0 0x561ab36da5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3b1b641000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ab3574c20 0x561ab36950c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3b1b641000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561ab34e5220 con 0x561ab3159280 2024-04-25T14:03:58.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3b1b641000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561ab347bea0 con 0x561ab3574c20 2024-04-25T14:03:58.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3b1b641000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x561ab34d2b00 con 0x561ab3560ea0 2024-04-25T14:03:58.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3b1b641000 10 client.0 _ll_register_callbacks cb 0x561ab34e4dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:58.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3b11600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab3159280 0x561ab36da070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3b11600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab3159280 0x561ab36da070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36216/0 (socket says 172.21.15.86:36216) 2024-04-25T14:03:58.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3b11600640 1 -- 172.21.15.86:0/2465281865 learned_addr learned my addr 172.21.15.86:0/2465281865 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:58.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3b10c00640 1 --2- 172.21.15.86:0/2465281865 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ab3560ea0 0x561ab36da5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3b12000640 1 --2- 172.21.15.86:0/2465281865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ab3574c20 0x561ab36950c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3b11600640 1 -- 172.21.15.86:0/2465281865 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ab3560ea0 msgr2=0x561ab36da5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:58.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3b11600640 1 --2- 172.21.15.86:0/2465281865 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ab3560ea0 0x561ab36da5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3b11600640 1 -- 172.21.15.86:0/2465281865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ab3574c20 msgr2=0x561ab36950c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:58.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3b11600640 1 --2- 172.21.15.86:0/2465281865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ab3574c20 0x561ab36950c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3b11600640 1 -- 172.21.15.86:0/2465281865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ab3477500 con 0x561ab3159280 2024-04-25T14:03:58.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3b11600640 1 --2- 172.21.15.86:0/2465281865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab3159280 0x561ab36da070 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000b550 tx=0x7f3b000046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40561c6fa9728f52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:58.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3af2000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:03:58.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3af2000640 1 -- 172.21.15.86:0/2465281865 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b00026020 con 0x561ab3159280 2024-04-25T14:03:58.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3b1b641000 1 -- 172.21.15.86:0/2465281865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b0401a070 con 0x561ab3159280 2024-04-25T14:03:58.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3af2000640 1 -- 172.21.15.86:0/2465281865 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3b00014640 con 0x561ab3159280 2024-04-25T14:03:58.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3af2000640 1 -- 172.21.15.86:0/2465281865 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f3b00030050 con 0x561ab3159280 2024-04-25T14:03:58.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3b1b641000 10 client.4422 fetch_fsmap learned FSMap version 7 2024-04-25T14:03:58.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3b1b641000 1 -- 172.21.15.86:0/2465281865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x561ab345dea0 con 0x561ab3159280 2024-04-25T14:03:58.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3af2000640 1 -- 172.21.15.86:0/2465281865 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b0401a070 con 0x561ab3159280 2024-04-25T14:03:58.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3af2000640 1 -- 172.21.15.86:0/2465281865 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f3b00033020 con 0x561ab3159280 2024-04-25T14:03:58.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3b1b641000 10 client.4422 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T14:03:58.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3b1b641000 10 client.4422 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T14:03:58.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3b1b641000 20 client.4422 populate_metadata read hostname 'smithi086' 2024-04-25T14:03:58.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3b1b641000 1 --2- 172.21.15.86:0/2465281865 >> [v2:172.21.15.155:6832/1944249570,v1:172.21.15.155:6833/1944249570] conn(0x561ab317d7d0 0x561ab3695e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3b1b641000 4 client.4422 mds_command: new command op to 4275 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:58.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3b1b641000 1 -- 172.21.15.86:0/2465281865 --> [v2:172.21.15.155:6832/1944249570,v1:172.21.15.155:6833/1944249570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ab3470be0 con 0x561ab317d7d0 2024-04-25T14:03:58.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.902+0000 7f3b10c00640 1 --2- 172.21.15.86:0/2465281865 >> [v2:172.21.15.155:6832/1944249570,v1:172.21.15.155:6833/1944249570] conn(0x561ab317d7d0 0x561ab3695e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.906+0000 7f3b10c00640 1 --2- 172.21.15.86:0/2465281865 >> [v2:172.21.15.155:6832/1944249570,v1:172.21.15.155:6833/1944249570] conn(0x561ab317d7d0 0x561ab3695e30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70ee43321143be89 server_cookie=3f72a34b00b3590f in_seq=0 out_seq=0 2024-04-25T14:03:58.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.906+0000 7f3af2000640 10 client.4422 ms_handle_connect on v2:172.21.15.155:6832/1944249570 2024-04-25T14:03:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.906+0000 7f3af2000640 1 -- 172.21.15.86:0/2465281865 <== mds.0 v2:172.21.15.155:6832/1944249570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561ab3470be0 con 0x561ab317d7d0 2024-04-25T14:03:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.906+0000 7f3af2000640 10 client.4422 handle_command_reply: tid=0 2024-04-25T14:03:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.906+0000 7f3b1b641000 1 client.4422 shutdown 2024-04-25T14:03:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.906+0000 7f3b1b641000 1 -- 172.21.15.86:0/2465281865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab3159280 msgr2=0x561ab36da070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.906+0000 7f3b1b641000 1 --2- 172.21.15.86:0/2465281865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab3159280 0x561ab36da070 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000b550 tx=0x7f3b000046d0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.906+0000 7f3b1b641000 1 -- 172.21.15.86:0/2465281865 shutdown_connections 2024-04-25T14:03:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.906+0000 7f3b1b641000 1 --2- 172.21.15.86:0/2465281865 >> [v2:172.21.15.155:6832/1944249570,v1:172.21.15.155:6833/1944249570] conn(0x561ab317d7d0 0x561ab3695e30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.906+0000 7f3b1b641000 1 --2- 172.21.15.86:0/2465281865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ab3574c20 0x561ab36950c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.906+0000 7f3b1b641000 1 --2- 172.21.15.86:0/2465281865 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ab3560ea0 0x561ab36da5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.906+0000 7f3af2000640 0 client.4422 ms_handle_reset on v2:172.21.15.155:6832/1944249570 2024-04-25T14:03:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.906+0000 7f3b1b641000 1 --2- 172.21.15.86:0/2465281865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab3159280 0x561ab36da070 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.906+0000 7f3b1b641000 1 -- 172.21.15.86:0/2465281865 >> 172.21.15.86:0/2465281865 conn(0x561ab3198ff0 msgr2=0x561ab3688c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.906+0000 7f3b1b641000 1 -- 172.21.15.86:0/2465281865 shutdown_connections 2024-04-25T14:03:58.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.906+0000 7f3b1b641000 1 -- 172.21.15.86:0/2465281865 wait complete. 2024-04-25T14:03:58.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.906+0000 7f3b1b641000 20 client.4422 trim_cache size 0 max 16384 2024-04-25T14:03:58.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.922+0000 7f3b1b641000 1 Processor -- start 2024-04-25T14:03:58.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3b1b641000 1 -- start start 2024-04-25T14:03:58.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3b1b641000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ab3159280 0x561ab36df6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3b1b641000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab3560ea0 0x561ab36d9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3b1b641000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ab3574c20 0x561ab36d9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3b1b641000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c206420 con 0x561ab3560ea0 2024-04-25T14:03:58.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3b1b641000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c15a480 con 0x561ab3159280 2024-04-25T14:03:58.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3b1b641000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3b0c205e00 con 0x561ab3574c20 2024-04-25T14:03:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3b10c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab3560ea0 0x561ab36d9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3b10c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab3560ea0 0x561ab36d9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36226/0 (socket says 172.21.15.86:36226) 2024-04-25T14:03:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3b10c00640 1 -- 172.21.15.86:0/4042586690 learned_addr learned my addr 172.21.15.86:0/4042586690 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3b12000640 1 --2- 172.21.15.86:0/4042586690 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ab3574c20 0x561ab36d9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3b11600640 1 --2- 172.21.15.86:0/4042586690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ab3159280 0x561ab36df6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3b10c00640 1 -- 172.21.15.86:0/4042586690 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ab3574c20 msgr2=0x561ab36d9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3b10c00640 1 --2- 172.21.15.86:0/4042586690 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ab3574c20 0x561ab36d9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3b10c00640 1 -- 172.21.15.86:0/4042586690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ab3159280 msgr2=0x561ab36df6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3b10c00640 1 --2- 172.21.15.86:0/4042586690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ab3159280 0x561ab36df6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3b10c00640 1 -- 172.21.15.86:0/4042586690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ab368c840 con 0x561ab3560ea0 2024-04-25T14:03:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3b10c00640 1 --2- 172.21.15.86:0/4042586690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab3560ea0 0x561ab36d9050 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3af801c940 tx=0x7f3af8126680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a642ed224ebff8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3afd200640 1 -- 172.21.15.86:0/4042586690 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af80fd8d0 con 0x561ab3560ea0 2024-04-25T14:03:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3afd200640 1 -- 172.21.15.86:0/4042586690 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af80a9040 con 0x561ab3560ea0 2024-04-25T14:03:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3afd200640 1 -- 172.21.15.86:0/4042586690 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af80f3d80 con 0x561ab3560ea0 2024-04-25T14:03:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3b1b641000 1 -- 172.21.15.86:0/4042586690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab3560ea0 msgr2=0x561ab36d9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3b1b641000 1 --2- 172.21.15.86:0/4042586690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab3560ea0 0x561ab36d9050 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3af801c940 tx=0x7f3af8126680 comp rx=0 tx=0).stop 2024-04-25T14:03:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3b1b641000 1 -- 172.21.15.86:0/4042586690 shutdown_connections 2024-04-25T14:03:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3b1b641000 1 --2- 172.21.15.86:0/4042586690 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ab3574c20 0x561ab36d9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3b1b641000 1 --2- 172.21.15.86:0/4042586690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab3560ea0 0x561ab36d9050 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3b1b641000 1 --2- 172.21.15.86:0/4042586690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ab3159280 0x561ab36df6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3b1b641000 1 -- 172.21.15.86:0/4042586690 >> 172.21.15.86:0/4042586690 conn(0x561ab3198ff0 msgr2=0x561ab3690040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3b1b641000 1 -- 172.21.15.86:0/4042586690 shutdown_connections 2024-04-25T14:03:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.926+0000 7f3b1b641000 1 -- 172.21.15.86:0/4042586690 wait complete. 2024-04-25T14:03:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b1b641000 1 -- start start 2024-04-25T14:03:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b1b641000 1 Processor -- start 2024-04-25T14:03:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b1b641000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ab3159280 0x561ab37457a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b1b641000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ab3560ea0 0x561ab3745ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b1b641000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab3574c20 0x561ab36ddb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b1b641000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561ab34d7130 con 0x561ab3574c20 2024-04-25T14:03:58.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b1b641000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561ab346d7d0 con 0x561ab3159280 2024-04-25T14:03:58.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b1b641000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x561ab34dc450 con 0x561ab3560ea0 2024-04-25T14:03:58.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b1b641000 10 client.0 _ll_register_callbacks cb 0x7f3b00006ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:58.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b12000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab3574c20 0x561ab36ddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b12000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab3574c20 0x561ab36ddb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36238/0 (socket says 172.21.15.86:36238) 2024-04-25T14:03:58.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b12000640 1 -- 172.21.15.86:0/2907607677 learned_addr learned my addr 172.21.15.86:0/2907607677 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:58.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b10c00640 1 --2- 172.21.15.86:0/2907607677 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ab3560ea0 0x561ab3745ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b11600640 1 --2- 172.21.15.86:0/2907607677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ab3159280 0x561ab37457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b12000640 1 -- 172.21.15.86:0/2907607677 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ab3560ea0 msgr2=0x561ab3745ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:58.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b12000640 1 --2- 172.21.15.86:0/2907607677 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ab3560ea0 0x561ab3745ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b12000640 1 -- 172.21.15.86:0/2907607677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ab3159280 msgr2=0x561ab37457a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:58.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b12000640 1 --2- 172.21.15.86:0/2907607677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ab3159280 0x561ab37457a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b12000640 1 -- 172.21.15.86:0/2907607677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ab3695430 con 0x561ab3574c20 2024-04-25T14:03:58.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b12000640 1 --2- 172.21.15.86:0/2907607677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab3574c20 0x561ab36ddb80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3b04007860 tx=0x7f3b04012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f5584ab719dd685 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:58.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3af2000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:03:58.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3af2000640 1 -- 172.21.15.86:0/2907607677 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b04009990 con 0x561ab3574c20 2024-04-25T14:03:58.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3af2000640 1 -- 172.21.15.86:0/2907607677 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3b04014d10 con 0x561ab3574c20 2024-04-25T14:03:58.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b1b641000 1 -- 172.21.15.86:0/2907607677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3af80a9040 con 0x561ab3574c20 2024-04-25T14:03:58.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3af2000640 1 -- 172.21.15.86:0/2907607677 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f3b04033050 con 0x561ab3574c20 2024-04-25T14:03:58.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b1b641000 10 client.4428 fetch_fsmap learned FSMap version 7 2024-04-25T14:03:58.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b1b641000 1 -- 172.21.15.86:0/2907607677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x561ab3490c70 con 0x561ab3574c20 2024-04-25T14:03:58.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3af2000640 1 -- 172.21.15.86:0/2907607677 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af80a9040 con 0x561ab3574c20 2024-04-25T14:03:58.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3af2000640 1 -- 172.21.15.86:0/2907607677 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f3b04036020 con 0x561ab3574c20 2024-04-25T14:03:58.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b1b641000 10 client.4428 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T14:03:58.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b1b641000 10 client.4428 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T14:03:58.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b1b641000 20 client.4428 populate_metadata read hostname 'smithi086' 2024-04-25T14:03:58.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b1b641000 1 --2- 172.21.15.86:0/2907607677 >> [v2:172.21.15.155:6832/1944249570,v1:172.21.15.155:6833/1944249570] conn(0x561ab316b830 0x561ab36de4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:58.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b1b641000 4 client.4428 mds_command: new command op to 4275 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:03:58.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.930+0000 7f3b1b641000 1 -- 172.21.15.86:0/2907607677 --> [v2:172.21.15.155:6832/1944249570,v1:172.21.15.155:6833/1944249570] -- command(tid 0: {"prefix": "status"}) -- 0x561ab34e5a30 con 0x561ab316b830 2024-04-25T14:03:58.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3b11600640 1 --2- 172.21.15.86:0/2907607677 >> [v2:172.21.15.155:6832/1944249570,v1:172.21.15.155:6833/1944249570] conn(0x561ab316b830 0x561ab36de4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:58.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3af2000640 10 client.4428 ms_handle_connect on v2:172.21.15.155:6832/1944249570 2024-04-25T14:03:58.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3b11600640 1 --2- 172.21.15.86:0/2907607677 >> [v2:172.21.15.155:6832/1944249570,v1:172.21.15.155:6833/1944249570] conn(0x561ab316b830 0x561ab36de4e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aa4b04d79d2fbe2 server_cookie=7e4bcc2dcb665917 in_seq=0 out_seq=0 2024-04-25T14:03:58.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3af2000640 1 -- 172.21.15.86:0/2907607677 <== mds.0 v2:172.21.15.155:6832/1944249570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x561ab34e5a30 con 0x561ab316b830 2024-04-25T14:03:58.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3af2000640 10 client.4428 handle_command_reply: tid=0 2024-04-25T14:03:58.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3b1b641000 1 client.4428 shutdown 2024-04-25T14:03:58.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3b1b641000 1 -- 172.21.15.86:0/2907607677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab3574c20 msgr2=0x561ab36ddb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:58.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3b1b641000 1 --2- 172.21.15.86:0/2907607677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab3574c20 0x561ab36ddb80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3b04007860 tx=0x7f3b04012000 comp rx=0 tx=0).stop 2024-04-25T14:03:58.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3b1b641000 1 -- 172.21.15.86:0/2907607677 shutdown_connections 2024-04-25T14:03:58.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3b1b641000 1 --2- 172.21.15.86:0/2907607677 >> [v2:172.21.15.155:6832/1944249570,v1:172.21.15.155:6833/1944249570] conn(0x561ab316b830 0x561ab36de4e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3b1b641000 1 --2- 172.21.15.86:0/2907607677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ab3574c20 0x561ab36ddb80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3b1b641000 1 --2- 172.21.15.86:0/2907607677 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ab3560ea0 0x561ab3745ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3b1b641000 1 --2- 172.21.15.86:0/2907607677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ab3159280 0x561ab37457a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3b1b641000 1 -- 172.21.15.86:0/2907607677 >> 172.21.15.86:0/2907607677 conn(0x561ab3198ff0 msgr2=0x561ab36d8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:58.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3af2000640 0 client.4428 ms_handle_reset on v2:172.21.15.155:6832/1944249570 2024-04-25T14:03:58.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3b1b641000 1 -- 172.21.15.86:0/2907607677 shutdown_connections 2024-04-25T14:03:58.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3b1b641000 1 -- 172.21.15.86:0/2907607677 wait complete. 2024-04-25T14:03:58.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3b1b641000 20 client.4428 trim_cache size 0 max 16384 2024-04-25T14:03:58.938 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:03:58.939 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb", 2024-04-25T14:03:58.939 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-25T14:03:58.939 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 4275, 2024-04-25T14:03:58.939 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T14:03:58.939 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T14:03:58.939 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T14:03:58.939 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 7.4661202720000004, 2024-04-25T14:03:58.939 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 7, 2024-04-25T14:03:58.939 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 24, 2024-04-25T14:03:58.940 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 23, 2024-04-25T14:03:58.940 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 14.522077012 2024-04-25T14:03:58.940 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:03:58.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3b14000640 1 -- 172.21.15.86:0/2507410991 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3aec06f400 msgr2=0x7f3aec071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:58.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3b14000640 1 --2- 172.21.15.86:0/2507410991 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3aec06f400 0x7f3aec071910 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c0b4200 tx=0x7f3b00008040 comp rx=0 tx=0).stop 2024-04-25T14:03:58.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3b14000640 1 -- 172.21.15.86:0/2507410991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b0c06fd50 msgr2=0x7f3b0c09a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3b14000640 1 --2- 172.21.15.86:0/2507410991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b0c06fd50 0x7f3b0c09a950 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3af80b0040 tx=0x7f3af8019200 comp rx=0 tx=0).stop 2024-04-25T14:03:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3b14000640 1 -- 172.21.15.86:0/2507410991 shutdown_connections 2024-04-25T14:03:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3b14000640 1 --2- 172.21.15.86:0/2507410991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b0c15c2a0 0x7f3b0c164000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3b14000640 1 --2- 172.21.15.86:0/2507410991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b0c06fd50 0x7f3b0c09a950 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3b14000640 1 --2- 172.21.15.86:0/2507410991 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3aec06f400 0x7f3aec071910 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3b14000640 1 --2- 172.21.15.86:0/2507410991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b0c06c670 0x7f3b0c15ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3b14000640 1 -- 172.21.15.86:0/2507410991 >> 172.21.15.86:0/2507410991 conn(0x7f3b0c0b4650 msgr2=0x7f3b0c156c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3b14000640 1 -- 172.21.15.86:0/2507410991 shutdown_connections 2024-04-25T14:03:58.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:58.934+0000 7f3b14000640 1 -- 172.21.15.86:0/2507410991 wait complete. 2024-04-25T14:03:58.954 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T14:03:58.954 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T14:03:58.954 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T14:03:59.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.098+0000 7fcefbe00640 1 Processor -- start 2024-04-25T14:03:59.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.098+0000 7fcefbe00640 1 -- start start 2024-04-25T14:03:59.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.102+0000 7fcefbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcef41610d0 0x7fcef40b7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:59.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.102+0000 7fcefbe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcef40d15e0 0x7fcef40d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:59.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.102+0000 7fcefbe00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcef40b3750 con 0x7fcef40b2250 2024-04-25T14:03:59.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.102+0000 7fcefbe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcef40b7f70 con 0x7fcef41610d0 2024-04-25T14:03:59.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.102+0000 7fcefbe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcef4161620 con 0x7fcef40d15e0 2024-04-25T14:03:59.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.102+0000 7fcef9400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcef41610d0 0x7fcef40b7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:59.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.102+0000 7fcef9400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcef41610d0 0x7fcef40b7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36244/0 (socket says 172.21.15.86:36244) 2024-04-25T14:03:59.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.102+0000 7fcef9400640 1 -- 172.21.15.86:0/1720434381 learned_addr learned my addr 172.21.15.86:0/1720434381 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:59.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.102+0000 7fcef9e00640 1 --2- 172.21.15.86:0/1720434381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcef40d15e0 0x7fcef40d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:59.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.102+0000 7fcef9400640 1 -- 172.21.15.86:0/1720434381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcef40d15e0 msgr2=0x7fcef40d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:59.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.102+0000 7fcef9400640 1 --2- 172.21.15.86:0/1720434381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcef40d15e0 0x7fcef40d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:59.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.102+0000 7fcef9400640 1 -- 172.21.15.86:0/1720434381 >> v1:172.21.15.155:6789/0 conn(0x7fcef40b2250 legacy=0x7fcef40b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:59.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.102+0000 7fcef9400640 1 -- 172.21.15.86:0/1720434381 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef40ce7b0 con 0x7fcef41610d0 2024-04-25T14:03:59.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.106+0000 7fcef9400640 1 --2- 172.21.15.86:0/1720434381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcef41610d0 0x7fcef40b7a30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcee8009870 tx=0x7fcee801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fedb60b7edf3649 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:59.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.106+0000 7fcef3e00640 1 -- 172.21.15.86:0/1720434381 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee80b0070 con 0x7fcef41610d0 2024-04-25T14:03:59.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.106+0000 7fcef3e00640 1 -- 172.21.15.86:0/1720434381 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcee80ab600 con 0x7fcef41610d0 2024-04-25T14:03:59.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.106+0000 7fcef3e00640 1 -- 172.21.15.86:0/1720434381 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee80ab9d0 con 0x7fcef41610d0 2024-04-25T14:03:59.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.106+0000 7fcefbe00640 1 -- 172.21.15.86:0/1720434381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcef41610d0 msgr2=0x7fcef40b7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:59.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.106+0000 7fcefbe00640 1 --2- 172.21.15.86:0/1720434381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcef41610d0 0x7fcef40b7a30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcee8009870 tx=0x7fcee801c4a0 comp rx=0 tx=0).stop 2024-04-25T14:03:59.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.106+0000 7fcefbe00640 1 -- 172.21.15.86:0/1720434381 shutdown_connections 2024-04-25T14:03:59.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.106+0000 7fcefbe00640 1 --2- 172.21.15.86:0/1720434381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcef40d15e0 0x7fcef40d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:59.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.106+0000 7fcefbe00640 1 --2- 172.21.15.86:0/1720434381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcef41610d0 0x7fcef40b7a30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:59.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.106+0000 7fcefbe00640 1 -- 172.21.15.86:0/1720434381 >> 172.21.15.86:0/1720434381 conn(0x7fcef409ba30 msgr2=0x7fcef40b7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:59.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.106+0000 7fcefbe00640 1 -- 172.21.15.86:0/1720434381 shutdown_connections 2024-04-25T14:03:59.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.106+0000 7fcefbe00640 1 -- 172.21.15.86:0/1720434381 wait complete. 2024-04-25T14:03:59.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.106+0000 7fcefbe00640 1 Processor -- start 2024-04-25T14:03:59.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.110+0000 7fcefbe00640 1 -- start start 2024-04-25T14:03:59.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.110+0000 7fcefbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcef40b2250 0x7fcef415be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:59.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.110+0000 7fcefbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcef40d15e0 0x7fcef40b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:59.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.110+0000 7fcefbe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcef41610d0 0x7fcef415f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:59.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.110+0000 7fcefbe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcef40b2ef0 con 0x7fcef40d15e0 2024-04-25T14:03:59.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.110+0000 7fcefbe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcef40b2d70 con 0x7fcef40b2250 2024-04-25T14:03:59.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.110+0000 7fcefbe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcef40cf610 con 0x7fcef41610d0 2024-04-25T14:03:59.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.110+0000 7fcef8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcef40d15e0 0x7fcef40b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:59.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.110+0000 7fcef8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcef40d15e0 0x7fcef40b0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36256/0 (socket says 172.21.15.86:36256) 2024-04-25T14:03:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.110+0000 7fcef8a00640 1 -- 172.21.15.86:0/461417635 learned_addr learned my addr 172.21.15.86:0/461417635 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:03:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.110+0000 7fcef9400640 1 --2- 172.21.15.86:0/461417635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcef40b2250 0x7fcef415be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.110+0000 7fcef9e00640 1 --2- 172.21.15.86:0/461417635 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcef41610d0 0x7fcef415f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.110+0000 7fcef8a00640 1 -- 172.21.15.86:0/461417635 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcef41610d0 msgr2=0x7fcef415f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.110+0000 7fcef8a00640 1 --2- 172.21.15.86:0/461417635 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcef41610d0 0x7fcef415f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.110+0000 7fcef8a00640 1 -- 172.21.15.86:0/461417635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcef40b2250 msgr2=0x7fcef415be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.110+0000 7fcef8a00640 1 --2- 172.21.15.86:0/461417635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcef40b2250 0x7fcef415be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.110+0000 7fcef8a00640 1 -- 172.21.15.86:0/461417635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef415c360 con 0x7fcef40d15e0 2024-04-25T14:03:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.110+0000 7fcef8a00640 1 --2- 172.21.15.86:0/461417635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcef40d15e0 0x7fcef40b0b30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcee000c3c0 tx=0x7fcee000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.110+0000 7fcef1600640 1 -- 172.21.15.86:0/461417635 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee0017070 con 0x7fcef40d15e0 2024-04-25T14:03:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.110+0000 7fcef1600640 1 -- 172.21.15.86:0/461417635 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcee00052d0 con 0x7fcef40d15e0 2024-04-25T14:03:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.110+0000 7fcefbe00640 1 -- 172.21.15.86:0/461417635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcef40b2050 con 0x7fcef40d15e0 2024-04-25T14:03:59.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.110+0000 7fcef1600640 1 -- 172.21.15.86:0/461417635 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee0003840 con 0x7fcef40d15e0 2024-04-25T14:03:59.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.110+0000 7fcefbe00640 1 -- 172.21.15.86:0/461417635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcef40b1a90 con 0x7fcef40d15e0 2024-04-25T14:03:59.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.110+0000 7fcefbe00640 1 -- 172.21.15.86:0/461417635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcef41618e0 con 0x7fcef40d15e0 2024-04-25T14:03:59.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.114+0000 7fcef1600640 1 -- 172.21.15.86:0/461417635 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcee001a070 con 0x7fcef40d15e0 2024-04-25T14:03:59.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.114+0000 7fcef1600640 1 --2- 172.21.15.86:0/461417635 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fced406f400 0x7fced4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:59.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.114+0000 7fcef1600640 1 -- 172.21.15.86:0/461417635 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fcee008e630 con 0x7fcef40d15e0 2024-04-25T14:03:59.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.114+0000 7fcef9400640 1 --2- 172.21.15.86:0/461417635 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fced406f400 0x7fced4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:59.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.114+0000 7fcef9400640 1 --2- 172.21.15.86:0/461417635 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fced406f400 0x7fced4071910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcee80ad000 tx=0x7fcee80a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:59.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.118+0000 7fcef1600640 1 -- 172.21.15.86:0/461417635 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcef415c360 con 0x7fcef40d15e0 2024-04-25T14:03:59.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.310+0000 7fcefbe00640 1 -- 172.21.15.86:0/461417635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcef40b1540 con 0x7fcef40d15e0 2024-04-25T14:03:59.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.310+0000 7fcef1600640 1 -- 172.21.15.86:0/461417635 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7fcee0005480 con 0x7fcef40d15e0 2024-04-25T14:03:59.311 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T14:03:59.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.310+0000 7fcefbe00640 1 -- 172.21.15.86:0/461417635 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fced406f400 msgr2=0x7fced4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:59.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.310+0000 7fcefbe00640 1 --2- 172.21.15.86:0/461417635 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fced406f400 0x7fced4071910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcee80ad000 tx=0x7fcee80a6000 comp rx=0 tx=0).stop 2024-04-25T14:03:59.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.310+0000 7fcefbe00640 1 -- 172.21.15.86:0/461417635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcef40d15e0 msgr2=0x7fcef40b0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:59.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.310+0000 7fcefbe00640 1 --2- 172.21.15.86:0/461417635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcef40d15e0 0x7fcef40b0b30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcee000c3c0 tx=0x7fcee000c880 comp rx=0 tx=0).stop 2024-04-25T14:03:59.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.310+0000 7fcefbe00640 1 -- 172.21.15.86:0/461417635 shutdown_connections 2024-04-25T14:03:59.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.310+0000 7fcefbe00640 1 --2- 172.21.15.86:0/461417635 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fced406f400 0x7fced4071910 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:59.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.310+0000 7fcefbe00640 1 --2- 172.21.15.86:0/461417635 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcef41610d0 0x7fcef415f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:59.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.310+0000 7fcefbe00640 1 --2- 172.21.15.86:0/461417635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcef40d15e0 0x7fcef40b0b30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:59.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.310+0000 7fcefbe00640 1 --2- 172.21.15.86:0/461417635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcef40b2250 0x7fcef415be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:59.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.310+0000 7fcefbe00640 1 -- 172.21.15.86:0/461417635 >> 172.21.15.86:0/461417635 conn(0x7fcef409ba30 msgr2=0x7fcef406fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:59.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.310+0000 7fcefbe00640 1 -- 172.21.15.86:0/461417635 shutdown_connections 2024-04-25T14:03:59.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59.310+0000 7fcefbe00640 1 -- 172.21.15.86:0/461417635 wait complete. 2024-04-25T14:03:59.332 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T14:03:59.332 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T14:03:59.332 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-25T14:03:59.332 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-25T14:03:59.332 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T14:03:59.332 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T14:03:59.332 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T14:03:59.332 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T14:03:59.332 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T14:03:59.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:03:59.332 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-25T14:03:59.342 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:03:59.342 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:03:59.342 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:03:59.342 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:03:59.342 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-25T14:03:59.342 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:03:59.342 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:03:59.342 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:03:59.342 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-25T14:03:59.343 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:03:59.343 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:03:59.343 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-25T14:03:59.343 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:03:59.343 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:03:59.343 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T14:03:59.343 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:03:59.343 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T14:03:59.343 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42425sec preferred_lft 42425sec 2024-04-25T14:03:59.343 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-25T14:03:59.343 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:03:59.343 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T14:03:59.343 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:dc:87:9c:96 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:03:59.343 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T14:03:59.344 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:03:59.344 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T14:03:59.344 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:03:59.344 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T14:03:59.345 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add name ceph-brx type bridge 2024-04-25T14:03:59.345 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr flush dev ceph-brx 2024-04-25T14:03:59.345 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set ceph-brx up 2024-04-25T14:03:59.345 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T14:03:59.345 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T14:03:59.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:03:59.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:03:59.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:03:59.503 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T14:03:59.562 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-25T14:03:59.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:03:59.564 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-25T14:03:59.639 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-25T14:03:59.639 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T14:03:59.639 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T14:03:59.639 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T14:03:59.639 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T14:03:59.639 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T14:03:59.639 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T14:03:59.639 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:03:59.639 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:03:59.639 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:03:59.639 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T14:03:59.640 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:03:59.640 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T14:03:59.640 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T14:03:59.640 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T14:03:59.640 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T14:03:59.640 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T14:03:59.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:03:59.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:03:59.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:03:59.777 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T14:03:59.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:03:59.787 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list-id 2024-04-25T14:03:59.837 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:03:59.838 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T14:03:59.838 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T14:03:59.838 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-25T14:03:59.838 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T14:03:59.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:03:59.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:03:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:03:59.962 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-25T14:03:59.962 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:03:59.962 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T14:03:59.962 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-25T14:03:59.962 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T14:03:59.962 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-25T14:03:59.963 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-25T14:03:59.963 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-25T14:03:59.963 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T14:04:00.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:04:00.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:00.134 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:04:00.134 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T14:04:00.134 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-25T14:04:00.134 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T14:04:00.135 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T14:04:00.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:04:00.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:00.217 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-25T14:04:00.217 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:04:00.217 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-25T14:04:00.226 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T14:04:00.227 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:04:00.227 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-25T14:04:00.276 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-25T14:04:00.336 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T14:04:00.390 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-25T14:04:00.390 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-25T14:04:00.390 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-25T14:04:00.390 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-25T14:04:00.390 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-25T14:04:00.390 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-25T14:04:00.390 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-25T14:04:00.391 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-25T14:04:00.391 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2024-04-25T14:04:00.391 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-25T14:04:00.391 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-25T14:04:00.391 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-25T14:04:00.391 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T14:04:00.391 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-25T14:04:00.391 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-25T14:04:00.391 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-25T14:04:00.391 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-25T14:04:00.391 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2024-04-25T14:04:00.391 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-25T14:04:00.391 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-25T14:04:00.391 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T14:04:00.391 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2024-04-25T14:04:00.392 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-25T14:04:00.392 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-25T14:04:00.392 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T14:04:00.392 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T14:04:00.392 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T14:04:00.392 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T14:04:00.392 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-25T14:04:00.392 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T14:04:00.392 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T14:04:00.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:00.393 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-25T14:04:00.444 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T14:04:00.444 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-25T14:04:00.491 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-25T14:04:00.512 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T14:04:00.566 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-25T14:04:00.566 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-25T14:04:00.566 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-25T14:04:00.566 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-25T14:04:00.566 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-25T14:04:00.566 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-25T14:04:00.566 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-25T14:04:00.566 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-25T14:04:00.566 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2024-04-25T14:04:00.566 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-25T14:04:00.567 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-25T14:04:00.567 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-25T14:04:00.567 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T14:04:00.567 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-25T14:04:00.567 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-25T14:04:00.567 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-25T14:04:00.567 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-25T14:04:00.567 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2024-04-25T14:04:00.567 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-25T14:04:00.567 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-25T14:04:00.567 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T14:04:00.567 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2024-04-25T14:04:00.567 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-25T14:04:00.567 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-25T14:04:00.567 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T14:04:00.568 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T14:04:00.568 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T14:04:00.568 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T14:04:00.568 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-25T14:04:00.568 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T14:04:00.568 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T14:04:00.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:00.569 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-25T14:04:00.597 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi086.stderr:2024-04-25T14:04:00.598+0000 7f8356a4d580 -1 init, newargv = 0x55a80988b510 newargc=15 2024-04-25T14:04:00.597 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi086.stderr:ceph-fuse[33915]: starting ceph client 2024-04-25T14:04:00.608 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi086.stderr:ceph-fuse[33915]: starting fuse 2024-04-25T14:04:01.707 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-25T14:04:01.730 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T14:04:01.782 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-25T14:04:01.782 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-25T14:04:01.782 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-25T14:04:01.782 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-25T14:04:01.782 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-25T14:04:01.782 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-25T14:04:01.782 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-25T14:04:01.782 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-25T14:04:01.782 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2024-04-25T14:04:01.783 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-25T14:04:01.783 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-25T14:04:01.783 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-25T14:04:01.783 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T14:04:01.783 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-25T14:04:01.783 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-25T14:04:01.783 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-25T14:04:01.783 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-25T14:04:01.783 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2024-04-25T14:04:01.783 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-25T14:04:01.783 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-25T14:04:01.783 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T14:04:01.783 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2024-04-25T14:04:01.784 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-25T14:04:01.784 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-25T14:04:01.784 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T14:04:01.784 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T14:04:01.784 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T14:04:01.784 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T14:04:01.784 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-25T14:04:01.784 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T14:04:01.784 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T14:04:01.784 INFO:teuthology.orchestra.run.smithi086.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-25T14:04:01.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:01.785 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-25T14:04:01.840 INFO:teuthology.orchestra.run.smithi086.stdout:44 2024-04-25T14:04:01.840 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-25T14:04:01.841 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2024-04-25T14:04:01.841 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2024-04-25T14:04:01.841 DEBUG:teuthology.orchestra.run.smithi086:> import re 2024-04-25T14:04:01.841 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-25T14:04:01.841 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2024-04-25T14:04:01.841 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-25T14:04:01.841 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2024-04-25T14:04:01.841 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-25T14:04:01.841 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2024-04-25T14:04:01.841 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-25T14:04:01.841 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-25T14:04:01.841 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2024-04-25T14:04:01.841 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2024-04-25T14:04:01.841 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2024-04-25T14:04:01.842 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2024-04-25T14:04:01.842 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-25T14:04:01.842 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2024-04-25T14:04:01.842 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T14:04:01.842 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T14:04:01.842 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T14:04:01.842 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2024-04-25T14:04:01.842 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2024-04-25T14:04:01.842 DEBUG:teuthology.orchestra.run.smithi086:> return f 2024-04-25T14:04:01.842 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T14:04:01.842 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-25T14:04:01.842 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.admin")) 2024-04-25T14:04:01.842 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-25T14:04:01.973 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.admin.33915.asok 2024-04-25T14:04:01.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:01.985 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.33915.asok 2024-04-25T14:04:01.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:01.985 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.33915.asok status 2024-04-25T14:04:02.168 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:04:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2024-04-25T14:04:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T14:04:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T14:04:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "admin", 2024-04-25T14:04:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2024-04-25T14:04:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-25T14:04:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "33915", 2024-04-25T14:04:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2024-04-25T14:04:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:04:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2024-04-25T14:04:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2024-04-25T14:04:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 4443, 2024-04-25T14:04:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2024-04-25T14:04:02.169 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2024-04-25T14:04:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2024-04-25T14:04:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 4443 2024-04-25T14:04:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:04:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-25T14:04:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-25T14:04:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-25T14:04:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 40858636 2024-04-25T14:04:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:04:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:04:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-25T14:04:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-25T14:04:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-25T14:04:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 40858636 2024-04-25T14:04:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:04:02.170 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.4443 192.168.0.1:0/40858636", 2024-04-25T14:04:02.171 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/40858636", 2024-04-25T14:04:02.171 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2024-04-25T14:04:02.171 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 7, 2024-04-25T14:04:02.171 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 24, 2024-04-25T14:04:02.171 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2024-04-25T14:04:02.171 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2024-04-25T14:04:02.171 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2024-04-25T14:04:02.171 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:04:02.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:02.185 DEBUG:teuthology.orchestra.run.smithi086:> stat --file-system '--printf=%T 2024-04-25T14:04:02.185 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-25T14:04:02.196 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2024-04-25T14:04:02.196 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-25T14:04:02.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:02.196 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-25T14:04:02.257 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-25T14:04:02.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:04:02.386 INFO:teuthology.orchestra.run.smithi086.stdout:# file: . 2024-04-25T14:04:02.386 INFO:teuthology.orchestra.run.smithi086.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-25T14:04:02.386 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:02.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:02.392 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:04:02.393 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:04:02.401 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:04:02.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.590+0000 7fa210200640 1 Processor -- start 2024-04-25T14:04:02.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.590+0000 7fa210200640 1 -- start start 2024-04-25T14:04:02.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.590+0000 7fa210200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2080cfca0 0x7fa20806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.590+0000 7fa210200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa2080730b0 0x7fa2080734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.590+0000 7fa210200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa208072bd0 con 0x7fa2080d16e0 2024-04-25T14:04:02.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.590+0000 7fa210200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa208070470 con 0x7fa2080730b0 2024-04-25T14:04:02.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.590+0000 7fa210200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa20806f930 con 0x7fa2080cfca0 2024-04-25T14:04:02.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.590+0000 7fa20d800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2080cfca0 0x7fa20806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.590+0000 7fa20e200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa2080730b0 0x7fa2080734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.590+0000 7fa20d800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2080cfca0 0x7fa20806f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:47742/0 (socket says 172.21.15.86:47742) 2024-04-25T14:04:02.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.590+0000 7fa20d800640 1 -- 172.21.15.86:0/792101220 learned_addr learned my addr 172.21.15.86:0/792101220 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:02.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.590+0000 7fa20ce00640 1 --1- 172.21.15.86:0/792101220 >> v1:172.21.15.155:6789/0 conn(0x7fa2080d16e0 0x7fa2080d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:44902/0 (socket says 172.21.15.86:44902) 2024-04-25T14:04:02.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.590+0000 7fa20d800640 1 -- 172.21.15.86:0/792101220 >> v2:172.21.15.155:3300/0 conn(0x7fa2080730b0 msgr2=0x7fa2080734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:02.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.590+0000 7fa20d800640 1 --2- 172.21.15.86:0/792101220 >> v2:172.21.15.155:3300/0 conn(0x7fa2080730b0 0x7fa2080734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.590+0000 7fa20d800640 1 -- 172.21.15.86:0/792101220 >> v1:172.21.15.155:6789/0 conn(0x7fa2080d16e0 legacy=0x7fa2080d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:02.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.590+0000 7fa20d800640 1 -- 172.21.15.86:0/792101220 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20806fbd0 con 0x7fa2080cfca0 2024-04-25T14:04:02.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.590+0000 7fa20e200640 1 --2- 172.21.15.86:0/792101220 >> v2:172.21.15.155:3300/0 conn(0x7fa2080730b0 0x7fa2080734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:02.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.594+0000 7fa20d800640 1 --2- 172.21.15.86:0/792101220 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2080cfca0 0x7fa20806f3f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009870 tx=0x7fa1f801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ececd287ac33f6d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:02.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.594+0000 7fa207e00640 1 -- 172.21.15.86:0/792101220 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f80a7070 con 0x7fa2080cfca0 2024-04-25T14:04:02.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.594+0000 7fa207e00640 1 -- 172.21.15.86:0/792101220 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1f80b0070 con 0x7fa2080cfca0 2024-04-25T14:04:02.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.594+0000 7fa207e00640 1 -- 172.21.15.86:0/792101220 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f80ab890 con 0x7fa2080cfca0 2024-04-25T14:04:02.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.594+0000 7fa210200640 1 -- 172.21.15.86:0/792101220 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2080cfca0 msgr2=0x7fa20806f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:02.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.594+0000 7fa210200640 1 --2- 172.21.15.86:0/792101220 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2080cfca0 0x7fa20806f3f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009870 tx=0x7fa1f801c560 comp rx=0 tx=0).stop 2024-04-25T14:04:02.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.594+0000 7fa210200640 1 -- 172.21.15.86:0/792101220 shutdown_connections 2024-04-25T14:04:02.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.594+0000 7fa210200640 1 --2- 172.21.15.86:0/792101220 >> v2:172.21.15.155:3300/0 conn(0x7fa2080730b0 0x7fa2080734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.594+0000 7fa210200640 1 --2- 172.21.15.86:0/792101220 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2080cfca0 0x7fa20806f3f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.594+0000 7fa210200640 1 -- 172.21.15.86:0/792101220 >> 172.21.15.86:0/792101220 conn(0x7fa2080b4650 msgr2=0x7fa2080b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:02.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.594+0000 7fa210200640 1 -- 172.21.15.86:0/792101220 shutdown_connections 2024-04-25T14:04:02.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.594+0000 7fa210200640 1 -- 172.21.15.86:0/792101220 wait complete. 2024-04-25T14:04:02.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.598+0000 7fa210200640 1 Processor -- start 2024-04-25T14:04:02.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.598+0000 7fa210200640 1 -- start start 2024-04-25T14:04:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.598+0000 7fa210200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2080730b0 0x7fa20809ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.598+0000 7fa210200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2080cfca0 0x7fa20809a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.598+0000 7fa210200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2080d16e0 0x7fa2081583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.598+0000 7fa210200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2080ce170 con 0x7fa2080cfca0 2024-04-25T14:04:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.598+0000 7fa210200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa2080cdff0 con 0x7fa2080d16e0 2024-04-25T14:04:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.598+0000 7fa210200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa2080ce2f0 con 0x7fa2080730b0 2024-04-25T14:04:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.598+0000 7fa20ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2080cfca0 0x7fa20809a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.598+0000 7fa20ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2080cfca0 0x7fa20809a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36258/0 (socket says 172.21.15.86:36258) 2024-04-25T14:04:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.598+0000 7fa20ce00640 1 -- 172.21.15.86:0/1726226842 learned_addr learned my addr 172.21.15.86:0/1726226842 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.598+0000 7fa20e200640 1 --2- 172.21.15.86:0/1726226842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2080d16e0 0x7fa2081583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.598+0000 7fa20d800640 1 --2- 172.21.15.86:0/1726226842 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2080730b0 0x7fa20809ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.598+0000 7fa20ce00640 1 -- 172.21.15.86:0/1726226842 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2080730b0 msgr2=0x7fa20809ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:02.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.598+0000 7fa20ce00640 1 --2- 172.21.15.86:0/1726226842 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2080730b0 0x7fa20809ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.598+0000 7fa20ce00640 1 -- 172.21.15.86:0/1726226842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2080d16e0 msgr2=0x7fa2081583a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:02.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.598+0000 7fa20ce00640 1 --2- 172.21.15.86:0/1726226842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2080d16e0 0x7fa2081583a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.598+0000 7fa20ce00640 1 -- 172.21.15.86:0/1726226842 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20809d0c0 con 0x7fa2080cfca0 2024-04-25T14:04:02.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.598+0000 7fa20ce00640 1 --2- 172.21.15.86:0/1726226842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2080cfca0 0x7fa20809a870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00c3a0 tx=0x7fa1fc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:02.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.602+0000 7fa205600640 1 -- 172.21.15.86:0/1726226842 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc0074e0 con 0x7fa2080cfca0 2024-04-25T14:04:02.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.602+0000 7fa205600640 1 -- 172.21.15.86:0/1726226842 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1fc007680 con 0x7fa2080cfca0 2024-04-25T14:04:02.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.602+0000 7fa205600640 1 -- 172.21.15.86:0/1726226842 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc004a30 con 0x7fa2080cfca0 2024-04-25T14:04:02.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.602+0000 7fa210200640 1 -- 172.21.15.86:0/1726226842 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa20809d3a0 con 0x7fa2080cfca0 2024-04-25T14:04:02.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.602+0000 7fa210200640 1 -- 172.21.15.86:0/1726226842 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa20815f8f0 con 0x7fa2080cfca0 2024-04-25T14:04:02.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.602+0000 7fa210200640 1 -- 172.21.15.86:0/1726226842 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa20809d8c0 con 0x7fa2080cfca0 2024-04-25T14:04:02.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.602+0000 7fa205600640 1 -- 172.21.15.86:0/1726226842 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa1fc01a020 con 0x7fa2080cfca0 2024-04-25T14:04:02.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.602+0000 7fa205600640 1 --2- 172.21.15.86:0/1726226842 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa1ec06f430 0x7fa1ec071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.602+0000 7fa205600640 1 -- 172.21.15.86:0/1726226842 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa1fc08ecf0 con 0x7fa2080cfca0 2024-04-25T14:04:02.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.602+0000 7fa20d800640 1 --2- 172.21.15.86:0/1726226842 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa1ec06f430 0x7fa1ec071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.606+0000 7fa20d800640 1 --2- 172.21.15.86:0/1726226842 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa1ec06f430 0x7fa1ec071940 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80af000 tx=0x7fa1f80a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:02.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.610+0000 7fa205600640 1 -- 172.21.15.86:0/1726226842 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa1fc05b1b0 con 0x7fa2080cfca0 2024-04-25T14:04:02.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.766+0000 7fa210200640 1 -- 172.21.15.86:0/1726226842 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa20809f4b0 con 0x7fa2080cfca0 2024-04-25T14:04:02.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.770+0000 7fa205600640 1 -- 172.21.15.86:0/1726226842 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7fa1fc05f4b0 con 0x7fa2080cfca0 2024-04-25T14:04:02.769 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:02.769 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":24,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:03:55.899092+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:03:49.151473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:03:49.802597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T14:03:53.492008+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:04:02.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.770+0000 7fa210200640 1 -- 172.21.15.86:0/1726226842 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa1ec06f430 msgr2=0x7fa1ec071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:02.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.770+0000 7fa210200640 1 --2- 172.21.15.86:0/1726226842 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa1ec06f430 0x7fa1ec071940 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80af000 tx=0x7fa1f80a7000 comp rx=0 tx=0).stop 2024-04-25T14:04:02.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.770+0000 7fa210200640 1 -- 172.21.15.86:0/1726226842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2080cfca0 msgr2=0x7fa20809a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:02.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.770+0000 7fa210200640 1 --2- 172.21.15.86:0/1726226842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2080cfca0 0x7fa20809a870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00c3a0 tx=0x7fa1fc00c860 comp rx=0 tx=0).stop 2024-04-25T14:04:02.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.770+0000 7fa210200640 1 -- 172.21.15.86:0/1726226842 shutdown_connections 2024-04-25T14:04:02.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.770+0000 7fa210200640 1 --2- 172.21.15.86:0/1726226842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2080d16e0 0x7fa2081583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.770+0000 7fa210200640 1 --2- 172.21.15.86:0/1726226842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2080cfca0 0x7fa20809a870 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.770+0000 7fa210200640 1 --2- 172.21.15.86:0/1726226842 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa1ec06f430 0x7fa1ec071940 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.770+0000 7fa210200640 1 --2- 172.21.15.86:0/1726226842 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2080730b0 0x7fa20809ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.770+0000 7fa210200640 1 -- 172.21.15.86:0/1726226842 >> 172.21.15.86:0/1726226842 conn(0x7fa2080b4650 msgr2=0x7fa20806e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:02.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.770+0000 7fa210200640 1 -- 172.21.15.86:0/1726226842 shutdown_connections 2024-04-25T14:04:02.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.770+0000 7fa210200640 1 -- 172.21.15.86:0/1726226842 wait complete. 2024-04-25T14:04:02.791 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:03:49.151473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:03:49.802597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T14:03:53.492008+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T14:04:02.792 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:04:02.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.938+0000 7f65a6a00640 1 Processor -- start 2024-04-25T14:04:02.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f65a6a00640 1 -- start start 2024-04-25T14:04:02.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f65a6a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65a015e860 0x7f65a00ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f65a6a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65a00cf8f0 con 0x7f65a00cf3e0 2024-04-25T14:04:02.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f65a6a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65a006c670 con 0x7f65a015ec70 2024-04-25T14:04:02.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f65a6a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f65a006cfa0 con 0x7f65a015e860 2024-04-25T14:04:02.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f65a00cf3e0 0x7f65a00d0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51998/0 (socket says 172.21.15.86:51998) 2024-04-25T14:04:02.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659f400640 1 -- 172.21.15.86:0/2317289010 learned_addr learned my addr 172.21.15.86:0/2317289010 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:02.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659fe00640 1 --2- 172.21.15.86:0/2317289010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65a015e860 0x7f65a00ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659ea00640 1 -- 172.21.15.86:0/2317289010 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 519702340 0 0) 0x7f65a00cf8f0 con 0x7f65a00cf3e0 2024-04-25T14:04:02.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659ea00640 1 -- 172.21.15.86:0/2317289010 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6584003530 con 0x7f65a00cf3e0 2024-04-25T14:04:02.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659ea00640 1 -- 172.21.15.86:0/2317289010 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1091186051 0 0) 0x7f65a006c670 con 0x7f65a015ec70 2024-04-25T14:04:02.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659ea00640 1 -- 172.21.15.86:0/2317289010 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65a00cf8f0 con 0x7f65a015ec70 2024-04-25T14:04:02.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659ea00640 1 -- 172.21.15.86:0/2317289010 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4141884558 0 0) 0x7f6584003530 con 0x7f65a00cf3e0 2024-04-25T14:04:02.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659ea00640 1 -- 172.21.15.86:0/2317289010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65a015e860 msgr2=0x7f65a00ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:02.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659ea00640 1 --2- 172.21.15.86:0/2317289010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65a015e860 0x7f65a00ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659ea00640 1 -- 172.21.15.86:0/2317289010 >> v1:172.21.15.155:6789/0 conn(0x7f65a015ec70 legacy=0x7f65a006d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:02.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659ea00640 1 -- 172.21.15.86:0/2317289010 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a006c950 con 0x7f65a00cf3e0 2024-04-25T14:04:02.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659ea00640 1 -- 172.21.15.86:0/2317289010 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f658c001680 con 0x7f65a00cf3e0 2024-04-25T14:04:02.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659fe00640 1 --2- 172.21.15.86:0/2317289010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65a015e860 0x7f65a00ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:02.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659ea00640 1 -- 172.21.15.86:0/2317289010 >> v1:172.21.15.86:6789/0 conn(0x7f65a00cf3e0 legacy=0x7f65a00d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:02.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659ea00640 1 --2- 172.21.15.86:0/2317289010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6584003bf0 0x7f6584004040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659ea00640 1 --2- 172.21.15.86:0/2317289010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65a015e860 0x7f6584004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659ea00640 1 --2- 172.21.15.86:0/2317289010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6584004d50 0x7f6584005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659ea00640 1 -- 172.21.15.86:0/2317289010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f65a006cfa0 con 0x7f6584003bf0 2024-04-25T14:04:02.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659ea00640 1 -- 172.21.15.86:0/2317289010 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f658400bbb0 con 0x7f6584004d50 2024-04-25T14:04:02.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659ea00640 1 -- 172.21.15.86:0/2317289010 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f658400bdb0 con 0x7f65a015e860 2024-04-25T14:04:02.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659fe00640 1 --2- 172.21.15.86:0/2317289010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6584003bf0 0x7f6584004040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659f400640 1 --2- 172.21.15.86:0/2317289010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65a015e860 0x7f6584004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f65a4a00640 1 --2- 172.21.15.86:0/2317289010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6584004d50 0x7f6584005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659fe00640 1 -- 172.21.15.86:0/2317289010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65a015e860 msgr2=0x7f6584004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:02.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659fe00640 1 --2- 172.21.15.86:0/2317289010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65a015e860 0x7f6584004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659fe00640 1 -- 172.21.15.86:0/2317289010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6584004d50 msgr2=0x7f6584005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:02.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659fe00640 1 --2- 172.21.15.86:0/2317289010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6584004d50 0x7f6584005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659fe00640 1 -- 172.21.15.86:0/2317289010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6584003530 con 0x7f6584003bf0 2024-04-25T14:04:02.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.942+0000 7f659f400640 1 --2- 172.21.15.86:0/2317289010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65a015e860 0x7f6584004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:02.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.946+0000 7f659fe00640 1 --2- 172.21.15.86:0/2317289010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6584003bf0 0x7f6584004040 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f659000e350 tx=0x7f659001cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118268684b1ffb52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:02.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.946+0000 7f659ea00640 1 -- 172.21.15.86:0/2317289010 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65900b2070 con 0x7f6584003bf0 2024-04-25T14:04:02.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.946+0000 7f659ea00640 1 -- 172.21.15.86:0/2317289010 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65900a5e00 con 0x7f6584003bf0 2024-04-25T14:04:02.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.946+0000 7f659ea00640 1 -- 172.21.15.86:0/2317289010 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6590009de0 con 0x7f6584003bf0 2024-04-25T14:04:02.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.946+0000 7f65a6a00640 1 -- 172.21.15.86:0/2317289010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6584003bf0 msgr2=0x7f6584004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:02.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.946+0000 7f65a6a00640 1 --2- 172.21.15.86:0/2317289010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6584003bf0 0x7f6584004040 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f659000e350 tx=0x7f659001cdf0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.946+0000 7f65a4a00640 1 -- 172.21.15.86:0/2317289010 reap_dead start 2024-04-25T14:04:02.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.946+0000 7f65a6a00640 1 -- 172.21.15.86:0/2317289010 shutdown_connections 2024-04-25T14:04:02.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.946+0000 7f65a6a00640 1 -- 172.21.15.86:0/2317289010 >> 172.21.15.86:0/2317289010 conn(0x7f65a00d2230 msgr2=0x7f65a00d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:02.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.946+0000 7f65a6a00640 1 -- 172.21.15.86:0/2317289010 shutdown_connections 2024-04-25T14:04:02.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.946+0000 7f65a6a00640 1 -- 172.21.15.86:0/2317289010 wait complete. 2024-04-25T14:04:02.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.950+0000 7f65a6a00640 1 Processor -- start 2024-04-25T14:04:02.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.950+0000 7f65a6a00640 1 -- start start 2024-04-25T14:04:02.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.950+0000 7f65a6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6584003bf0 0x7f65a0167ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.950+0000 7f65a6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65a0168420 0x7f65a006ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.950+0000 7f65a6a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65a0070460 0x7f65a0153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.950+0000 7f65a6a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f65a006df50 con 0x7f6584003bf0 2024-04-25T14:04:02.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.950+0000 7f65a6a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f65a006ddd0 con 0x7f65a0168420 2024-04-25T14:04:02.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.950+0000 7f65a6a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f65a0169f30 con 0x7f65a0070460 2024-04-25T14:04:02.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.950+0000 7f659fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6584003bf0 0x7f65a0167ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.950+0000 7f659fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6584003bf0 0x7f65a0167ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36272/0 (socket says 172.21.15.86:36272) 2024-04-25T14:04:02.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.950+0000 7f659fe00640 1 -- 172.21.15.86:0/3240228413 learned_addr learned my addr 172.21.15.86:0/3240228413 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:02.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.950+0000 7f65a4a00640 1 --2- 172.21.15.86:0/3240228413 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65a0070460 0x7f65a0153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.950+0000 7f659f400640 1 --2- 172.21.15.86:0/3240228413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65a0168420 0x7f65a006ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.954+0000 7f659fe00640 1 -- 172.21.15.86:0/3240228413 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65a0070460 msgr2=0x7f65a0153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:02.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.954+0000 7f659fe00640 1 --2- 172.21.15.86:0/3240228413 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65a0070460 0x7f65a0153340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.954+0000 7f659fe00640 1 -- 172.21.15.86:0/3240228413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65a0168420 msgr2=0x7f65a006ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:02.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.954+0000 7f659fe00640 1 --2- 172.21.15.86:0/3240228413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65a0168420 0x7f65a006ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.954+0000 7f659fe00640 1 -- 172.21.15.86:0/3240228413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a0166760 con 0x7f6584003bf0 2024-04-25T14:04:02.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.954+0000 7f659fe00640 1 --2- 172.21.15.86:0/3240228413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6584003bf0 0x7f65a0167ee0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f65900b0040 tx=0x7f6590009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:02.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.954+0000 7f6597e00640 1 -- 172.21.15.86:0/3240228413 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65900bb020 con 0x7f6584003bf0 2024-04-25T14:04:02.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.954+0000 7f6597e00640 1 -- 172.21.15.86:0/3240228413 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6590009a50 con 0x7f6584003bf0 2024-04-25T14:04:02.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.954+0000 7f6597e00640 1 -- 172.21.15.86:0/3240228413 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65900b2040 con 0x7f6584003bf0 2024-04-25T14:04:02.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.954+0000 7f65a6a00640 1 -- 172.21.15.86:0/3240228413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65a0070c30 con 0x7f6584003bf0 2024-04-25T14:04:02.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.954+0000 7f65a6a00640 1 -- 172.21.15.86:0/3240228413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65a0164c20 con 0x7f6584003bf0 2024-04-25T14:04:02.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.954+0000 7f65a6a00640 1 -- 172.21.15.86:0/3240228413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65a0074cf0 con 0x7f6584003bf0 2024-04-25T14:04:02.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.954+0000 7f6597e00640 1 -- 172.21.15.86:0/3240228413 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f65900be070 con 0x7f6584003bf0 2024-04-25T14:04:02.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.954+0000 7f6597e00640 1 --2- 172.21.15.86:0/3240228413 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f658006f400 0x7f6580071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.958+0000 7f6597e00640 1 -- 172.21.15.86:0/3240228413 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6590129650 con 0x7f6584003bf0 2024-04-25T14:04:02.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.958+0000 7f659f400640 1 --2- 172.21.15.86:0/3240228413 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f658006f400 0x7f6580071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.958+0000 7f659f400640 1 --2- 172.21.15.86:0/3240228413 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f658006f400 0x7f6580071910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f65a006f620 tx=0x7f658c007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:02.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:02.962+0000 7f6597e00640 1 -- 172.21.15.86:0/3240228413 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f65900f0e70 con 0x7f6584003bf0 2024-04-25T14:04:03.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.118+0000 7f65a6a00640 1 -- 172.21.15.86:0/3240228413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f65a0076bf0 con 0x7f6584003bf0 2024-04-25T14:04:03.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.118+0000 7f6597e00640 1 -- 172.21.15.86:0/3240228413 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f6590016b30 con 0x7f6584003bf0 2024-04-25T14:04:03.119 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:04:03.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.118+0000 7f65a6a00640 1 -- 172.21.15.86:0/3240228413 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f658006f400 msgr2=0x7f6580071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:03.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.118+0000 7f65a6a00640 1 --2- 172.21.15.86:0/3240228413 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f658006f400 0x7f6580071910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f65a006f620 tx=0x7f658c007040 comp rx=0 tx=0).stop 2024-04-25T14:04:03.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.118+0000 7f65a6a00640 1 -- 172.21.15.86:0/3240228413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6584003bf0 msgr2=0x7f65a0167ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:03.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.118+0000 7f65a6a00640 1 --2- 172.21.15.86:0/3240228413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6584003bf0 0x7f65a0167ee0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f65900b0040 tx=0x7f6590009750 comp rx=0 tx=0).stop 2024-04-25T14:04:03.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.118+0000 7f65a6a00640 1 -- 172.21.15.86:0/3240228413 shutdown_connections 2024-04-25T14:04:03.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.118+0000 7f65a6a00640 1 --2- 172.21.15.86:0/3240228413 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f658006f400 0x7f6580071910 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.118+0000 7f65a6a00640 1 --2- 172.21.15.86:0/3240228413 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65a0070460 0x7f65a0153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.118+0000 7f65a6a00640 1 --2- 172.21.15.86:0/3240228413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65a0168420 0x7f65a006ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.118+0000 7f65a6a00640 1 --2- 172.21.15.86:0/3240228413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6584003bf0 0x7f65a0167ee0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.118+0000 7f65a6a00640 1 -- 172.21.15.86:0/3240228413 >> 172.21.15.86:0/3240228413 conn(0x7f65a00d2230 msgr2=0x7f65a015f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:03.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.118+0000 7f65a6a00640 1 -- 172.21.15.86:0/3240228413 shutdown_connections 2024-04-25T14:04:03.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.118+0000 7f65a6a00640 1 -- 172.21.15.86:0/3240228413 wait complete. 2024-04-25T14:04:03.140 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:04:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.290+0000 7f7402c00640 1 Processor -- start 2024-04-25T14:04:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.290+0000 7f7402c00640 1 -- start start 2024-04-25T14:04:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.290+0000 7f7402c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73fc0b7a30 0x7f73fc0b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.290+0000 7f7402c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f73fc0d15e0 0x7f73fc0d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.290+0000 7f7402c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73fc0b3750 con 0x7f73fc1610d0 2024-04-25T14:04:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.290+0000 7f7402c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f73fc161620 con 0x7f73fc0d15e0 2024-04-25T14:04:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.290+0000 7f7402c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f73fc0ce7b0 con 0x7f73fc0b7a30 2024-04-25T14:04:03.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.290+0000 7f7400c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f73fc0d15e0 0x7f73fc0d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:03.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.290+0000 7f73f3400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73fc0b7a30 0x7f73fc0b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:03.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.290+0000 7f7400c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f73fc0d15e0 0x7f73fc0d19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:58220/0 (socket says 172.21.15.86:58220) 2024-04-25T14:04:03.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.290+0000 7f7400c00640 1 -- 172.21.15.86:0/2401472834 learned_addr learned my addr 172.21.15.86:0/2401472834 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:03.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.290+0000 7f73fbe00640 1 --1- 172.21.15.86:0/2401472834 >> v1:172.21.15.155:6789/0 conn(0x7f73fc1610d0 0x7f73fc0b2250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:44922/0 (socket says 172.21.15.86:44922) 2024-04-25T14:04:03.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.290+0000 7f7400c00640 1 -- 172.21.15.86:0/2401472834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73fc0b7a30 msgr2=0x7f73fc0b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:03.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.290+0000 7f7400c00640 1 --2- 172.21.15.86:0/2401472834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73fc0b7a30 0x7f73fc0b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.290+0000 7f7400c00640 1 -- 172.21.15.86:0/2401472834 >> v1:172.21.15.155:6789/0 conn(0x7f73fc1610d0 legacy=0x7f73fc0b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:03.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.290+0000 7f7400c00640 1 -- 172.21.15.86:0/2401472834 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73fc0b39f0 con 0x7f73fc0d15e0 2024-04-25T14:04:03.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.294+0000 7f7400c00640 1 --2- 172.21.15.86:0/2401472834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73fc0d15e0 0x7f73fc0d19f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f73f4009e50 tx=0x7f73f401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eede0444a879b55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:03.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.294+0000 7f73fb400640 1 -- 172.21.15.86:0/2401472834 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f40a5630 con 0x7f73fc0d15e0 2024-04-25T14:04:03.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.294+0000 7f73fb400640 1 -- 172.21.15.86:0/2401472834 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73f40a57d0 con 0x7f73fc0d15e0 2024-04-25T14:04:03.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.294+0000 7f73fb400640 1 -- 172.21.15.86:0/2401472834 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f40a5ba0 con 0x7f73fc0d15e0 2024-04-25T14:04:03.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.294+0000 7f7402c00640 1 -- 172.21.15.86:0/2401472834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73fc0d15e0 msgr2=0x7f73fc0d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:03.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.294+0000 7f7402c00640 1 --2- 172.21.15.86:0/2401472834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73fc0d15e0 0x7f73fc0d19f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f73f4009e50 tx=0x7f73f401c970 comp rx=0 tx=0).stop 2024-04-25T14:04:03.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.294+0000 7f7402c00640 1 -- 172.21.15.86:0/2401472834 shutdown_connections 2024-04-25T14:04:03.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.294+0000 7f7402c00640 1 --2- 172.21.15.86:0/2401472834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73fc0d15e0 0x7f73fc0d19f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.294+0000 7f7402c00640 1 --2- 172.21.15.86:0/2401472834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73fc0b7a30 0x7f73fc0b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.294+0000 7f7402c00640 1 -- 172.21.15.86:0/2401472834 >> 172.21.15.86:0/2401472834 conn(0x7f73fc09ba30 msgr2=0x7f73fc0b7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:03.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.294+0000 7f7402c00640 1 -- 172.21.15.86:0/2401472834 shutdown_connections 2024-04-25T14:04:03.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.294+0000 7f7402c00640 1 -- 172.21.15.86:0/2401472834 wait complete. 2024-04-25T14:04:03.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.298+0000 7f7402c00640 1 Processor -- start 2024-04-25T14:04:03.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.298+0000 7f7402c00640 1 -- start start 2024-04-25T14:04:03.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.298+0000 7f7402c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73fc0b7a30 0x7f73fc0b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:03.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.298+0000 7f7402c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73fc0d15e0 0x7f73fc159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:03.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.298+0000 7f7402c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73fc1610d0 0x7f73fc158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.298+0000 7f7402c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73fc0cf610 con 0x7f73fc0d15e0 2024-04-25T14:04:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.298+0000 7f7402c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f73fc0b7e40 con 0x7f73fc0b7a30 2024-04-25T14:04:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.298+0000 7f7402c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f73fc0cf790 con 0x7f73fc1610d0 2024-04-25T14:04:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.298+0000 7f73f3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73fc0d15e0 0x7f73fc159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.298+0000 7f73f3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73fc0d15e0 0x7f73fc159990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36282/0 (socket says 172.21.15.86:36282) 2024-04-25T14:04:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.298+0000 7f73f3400640 1 -- 172.21.15.86:0/355475122 learned_addr learned my addr 172.21.15.86:0/355475122 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.298+0000 7f7400c00640 1 --2- 172.21.15.86:0/355475122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73fc1610d0 0x7f73fc158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.298+0000 7f73fbe00640 1 --2- 172.21.15.86:0/355475122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73fc0b7a30 0x7f73fc0b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.298+0000 7f73f3400640 1 -- 172.21.15.86:0/355475122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73fc1610d0 msgr2=0x7f73fc158690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.298+0000 7f73f3400640 1 --2- 172.21.15.86:0/355475122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73fc1610d0 0x7f73fc158690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.298+0000 7f73f3400640 1 -- 172.21.15.86:0/355475122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73fc0b7a30 msgr2=0x7f73fc0b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.298+0000 7f73f3400640 1 --2- 172.21.15.86:0/355475122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73fc0b7a30 0x7f73fc0b02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.298+0000 7f73f3400640 1 -- 172.21.15.86:0/355475122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73fc070ab0 con 0x7f73fc0d15e0 2024-04-25T14:04:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.302+0000 7f73f3400640 1 --2- 172.21.15.86:0/355475122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73fc0d15e0 0x7f73fc159990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f73e400e870 tx=0x7f73e400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.302+0000 7f73f8c00640 1 -- 172.21.15.86:0/355475122 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4017070 con 0x7f73fc0d15e0 2024-04-25T14:04:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.302+0000 7f73f8c00640 1 -- 172.21.15.86:0/355475122 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73e400ee60 con 0x7f73fc0d15e0 2024-04-25T14:04:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.302+0000 7f73f8c00640 1 -- 172.21.15.86:0/355475122 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e400cb40 con 0x7f73fc0d15e0 2024-04-25T14:04:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.302+0000 7f7402c00640 1 -- 172.21.15.86:0/355475122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73fc0b0800 con 0x7f73fc0d15e0 2024-04-25T14:04:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.302+0000 7f7402c00640 1 -- 172.21.15.86:0/355475122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73fc0af500 con 0x7f73fc0d15e0 2024-04-25T14:04:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.302+0000 7f7402c00640 1 -- 172.21.15.86:0/355475122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73fc076bf0 con 0x7f73fc0d15e0 2024-04-25T14:04:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.302+0000 7f73f8c00640 1 -- 172.21.15.86:0/355475122 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f73e401a070 con 0x7f73fc0d15e0 2024-04-25T14:04:03.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.302+0000 7f73f8c00640 1 --2- 172.21.15.86:0/355475122 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f73d806f400 0x7f73d8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:03.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.302+0000 7f73f8c00640 1 -- 172.21.15.86:0/355475122 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f73e408f410 con 0x7f73fc0d15e0 2024-04-25T14:04:03.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.302+0000 7f73fbe00640 1 --2- 172.21.15.86:0/355475122 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f73d806f400 0x7f73d8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.306+0000 7f73fbe00640 1 --2- 172.21.15.86:0/355475122 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f73d806f400 0x7f73d8071910 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f73ec000f70 tx=0x7f73ec007e30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:03.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.310+0000 7f73f8c00640 1 -- 172.21.15.86:0/355475122 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f73e405b960 con 0x7f73fc0d15e0 2024-04-25T14:04:03.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.462+0000 7f7402c00640 1 -- 172.21.15.86:0/355475122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f73fc074750 con 0x7f73fc0d15e0 2024-04-25T14:04:03.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.462+0000 7f73f8c00640 1 -- 172.21.15.86:0/355475122 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f73e405fc60 con 0x7f73fc0d15e0 2024-04-25T14:04:03.464 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:04:03.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.466+0000 7f7402c00640 1 -- 172.21.15.86:0/355475122 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f73d806f400 msgr2=0x7f73d8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:03.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.466+0000 7f7402c00640 1 --2- 172.21.15.86:0/355475122 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f73d806f400 0x7f73d8071910 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f73ec000f70 tx=0x7f73ec007e30 comp rx=0 tx=0).stop 2024-04-25T14:04:03.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.466+0000 7f7402c00640 1 -- 172.21.15.86:0/355475122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73fc0d15e0 msgr2=0x7f73fc159990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:03.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.466+0000 7f7402c00640 1 --2- 172.21.15.86:0/355475122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73fc0d15e0 0x7f73fc159990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f73e400e870 tx=0x7f73e400ed30 comp rx=0 tx=0).stop 2024-04-25T14:04:03.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.466+0000 7f7402c00640 1 -- 172.21.15.86:0/355475122 shutdown_connections 2024-04-25T14:04:03.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.466+0000 7f7402c00640 1 --2- 172.21.15.86:0/355475122 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f73d806f400 0x7f73d8071910 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.466+0000 7f7402c00640 1 --2- 172.21.15.86:0/355475122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73fc1610d0 0x7f73fc158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.466+0000 7f7402c00640 1 --2- 172.21.15.86:0/355475122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73fc0d15e0 0x7f73fc159990 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.466+0000 7f7402c00640 1 --2- 172.21.15.86:0/355475122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73fc0b7a30 0x7f73fc0b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.466+0000 7f7402c00640 1 -- 172.21.15.86:0/355475122 >> 172.21.15.86:0/355475122 conn(0x7f73fc09ba30 msgr2=0x7f73fc06fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:03.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.466+0000 7f7402c00640 1 -- 172.21.15.86:0/355475122 shutdown_connections 2024-04-25T14:04:03.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.466+0000 7f7402c00640 1 -- 172.21.15.86:0/355475122 wait complete. 2024-04-25T14:04:03.486 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:04:03.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.638+0000 7fe761600640 1 Processor -- start 2024-04-25T14:04:03.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.638+0000 7fe761600640 1 -- start start 2024-04-25T14:04:03.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.638+0000 7fe761600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe75c0b59e0 0x7fe75c0b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:03.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.638+0000 7fe761600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe75c0aeea0 0x7fe75c0af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:03.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.638+0000 7fe761600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe75c0b6330 con 0x7fe75c15faa0 2024-04-25T14:04:03.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.638+0000 7fe761600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe75c06d560 con 0x7fe75c0aeea0 2024-04-25T14:04:03.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.638+0000 7fe761600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe75c06c670 con 0x7fe75c0b59e0 2024-04-25T14:04:03.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.638+0000 7fe75aa00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe75c15faa0 0x7fe75c06d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52012/0 (socket says 172.21.15.86:52012) 2024-04-25T14:04:03.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.638+0000 7fe75aa00640 1 -- 172.21.15.86:0/2813257530 learned_addr learned my addr 172.21.15.86:0/2813257530 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:03.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.638+0000 7fe75b400640 1 --2- 172.21.15.86:0/2813257530 >> v2:172.21.15.86:3300/0 conn(0x7fe75c0aeea0 0x7fe75c0af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:03.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.638+0000 7fe75a000640 1 --2- 172.21.15.86:0/2813257530 >> v2:172.21.15.155:3300/0 conn(0x7fe75c0b59e0 0x7fe75c0b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:03.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.638+0000 7fe759600640 1 -- 172.21.15.86:0/2813257530 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3659533822 0 0) 0x7fe74c001120 con 0x7fe75c15faa0 2024-04-25T14:04:03.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.638+0000 7fe759600640 1 -- 172.21.15.86:0/2813257530 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe740003530 con 0x7fe75c15faa0 2024-04-25T14:04:03.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.638+0000 7fe75b400640 1 -- 172.21.15.86:0/2813257530 >> v2:172.21.15.155:3300/0 conn(0x7fe75c0b59e0 msgr2=0x7fe75c0b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:03.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.638+0000 7fe75b400640 1 --2- 172.21.15.86:0/2813257530 >> v2:172.21.15.155:3300/0 conn(0x7fe75c0b59e0 0x7fe75c0b5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.638+0000 7fe75b400640 1 -- 172.21.15.86:0/2813257530 >> v1:172.21.15.86:6789/0 conn(0x7fe75c15faa0 legacy=0x7fe75c06d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:03.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.638+0000 7fe75b400640 1 -- 172.21.15.86:0/2813257530 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75c06c950 con 0x7fe75c0aeea0 2024-04-25T14:04:03.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.638+0000 7fe75a000640 1 --2- 172.21.15.86:0/2813257530 >> v2:172.21.15.155:3300/0 conn(0x7fe75c0b59e0 0x7fe75c0b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:03.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.642+0000 7fe75b400640 1 --2- 172.21.15.86:0/2813257530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe75c0aeea0 0x7fe75c0af2b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0b7250 tx=0x7fe75401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=facf0ad3f26770af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:03.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.642+0000 7fe759600640 1 -- 172.21.15.86:0/2813257530 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7540b2070 con 0x7fe75c0aeea0 2024-04-25T14:04:03.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.642+0000 7fe759600640 1 -- 172.21.15.86:0/2813257530 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7540a5a10 con 0x7fe75c0aeea0 2024-04-25T14:04:03.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.642+0000 7fe759600640 1 -- 172.21.15.86:0/2813257530 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7540a5de0 con 0x7fe75c0aeea0 2024-04-25T14:04:03.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.642+0000 7fe761600640 1 -- 172.21.15.86:0/2813257530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe75c0aeea0 msgr2=0x7fe75c0af2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:03.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.642+0000 7fe761600640 1 --2- 172.21.15.86:0/2813257530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe75c0aeea0 0x7fe75c0af2b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0b7250 tx=0x7fe75401c930 comp rx=0 tx=0).stop 2024-04-25T14:04:03.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.642+0000 7fe761600640 1 -- 172.21.15.86:0/2813257530 shutdown_connections 2024-04-25T14:04:03.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.642+0000 7fe761600640 1 --2- 172.21.15.86:0/2813257530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe75c0aeea0 0x7fe75c0af2b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.642+0000 7fe761600640 1 --2- 172.21.15.86:0/2813257530 >> v2:172.21.15.155:3300/0 conn(0x7fe75c0b59e0 0x7fe75c0b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.642+0000 7fe761600640 1 -- 172.21.15.86:0/2813257530 >> 172.21.15.86:0/2813257530 conn(0x7fe75c0d2230 msgr2=0x7fe75c0d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:03.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.642+0000 7fe761600640 1 -- 172.21.15.86:0/2813257530 shutdown_connections 2024-04-25T14:04:03.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.642+0000 7fe761600640 1 -- 172.21.15.86:0/2813257530 wait complete. 2024-04-25T14:04:03.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.646+0000 7fe761600640 1 Processor -- start 2024-04-25T14:04:03.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.646+0000 7fe761600640 1 -- start start 2024-04-25T14:04:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.646+0000 7fe761600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe75c0aeea0 0x7fe75c13e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.646+0000 7fe761600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe75c0b59e0 0x7fe75c161960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.646+0000 7fe761600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe75c15faa0 0x7fe75c161ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.646+0000 7fe761600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c160030 con 0x7fe75c0b59e0 2024-04-25T14:04:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.646+0000 7fe761600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c15feb0 con 0x7fe75c0aeea0 2024-04-25T14:04:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.646+0000 7fe761600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe75c070f30 con 0x7fe75c15faa0 2024-04-25T14:04:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.646+0000 7fe75a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe75c0b59e0 0x7fe75c161960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.646+0000 7fe75a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe75c0b59e0 0x7fe75c161960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36298/0 (socket says 172.21.15.86:36298) 2024-04-25T14:04:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.646+0000 7fe75a000640 1 -- 172.21.15.86:0/132273854 learned_addr learned my addr 172.21.15.86:0/132273854 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.646+0000 7fe75a000640 1 -- 172.21.15.86:0/132273854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe75c15faa0 msgr2=0x7fe75c161ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.646+0000 7fe75a000640 1 --2- 172.21.15.86:0/132273854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe75c15faa0 0x7fe75c161ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.646+0000 7fe75aa00640 1 --2- 172.21.15.86:0/132273854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe75c0aeea0 0x7fe75c13e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:03.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.646+0000 7fe75a000640 1 -- 172.21.15.86:0/132273854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe75c0aeea0 msgr2=0x7fe75c13e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:03.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.646+0000 7fe75a000640 1 --2- 172.21.15.86:0/132273854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe75c0aeea0 0x7fe75c13e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.646+0000 7fe75a000640 1 -- 172.21.15.86:0/132273854 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75c150510 con 0x7fe75c0b59e0 2024-04-25T14:04:03.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.646+0000 7fe75aa00640 1 --2- 172.21.15.86:0/132273854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe75c0aeea0 0x7fe75c13e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:03.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.646+0000 7fe75a000640 1 --2- 172.21.15.86:0/132273854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe75c0b59e0 0x7fe75c161960 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe74800d830 tx=0x7fe74800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:03.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.646+0000 7fe752a00640 1 -- 172.21.15.86:0/132273854 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74800b6c0 con 0x7fe75c0b59e0 2024-04-25T14:04:03.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.650+0000 7fe752a00640 1 -- 172.21.15.86:0/132273854 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe748019070 con 0x7fe75c0b59e0 2024-04-25T14:04:03.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.650+0000 7fe761600640 1 -- 172.21.15.86:0/132273854 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe75c1529a0 con 0x7fe75c0b59e0 2024-04-25T14:04:03.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.650+0000 7fe752a00640 1 -- 172.21.15.86:0/132273854 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74800baa0 con 0x7fe75c0b59e0 2024-04-25T14:04:03.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.650+0000 7fe761600640 1 -- 172.21.15.86:0/132273854 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe75c152f60 con 0x7fe75c0b59e0 2024-04-25T14:04:03.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.650+0000 7fe761600640 1 -- 172.21.15.86:0/132273854 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe75c152710 con 0x7fe75c0b59e0 2024-04-25T14:04:03.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.650+0000 7fe752a00640 1 -- 172.21.15.86:0/132273854 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe74801e460 con 0x7fe75c0b59e0 2024-04-25T14:04:03.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.650+0000 7fe752a00640 1 --2- 172.21.15.86:0/132273854 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe73c06f430 0x7fe73c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:03.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.650+0000 7fe752a00640 1 -- 172.21.15.86:0/132273854 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe74808fc60 con 0x7fe75c0b59e0 2024-04-25T14:04:03.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.650+0000 7fe75aa00640 1 --2- 172.21.15.86:0/132273854 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe73c06f430 0x7fe73c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:03.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.654+0000 7fe75aa00640 1 --2- 172.21.15.86:0/132273854 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe73c06f430 0x7fe73c071940 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0012c0 tx=0x7fe74c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:03.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.654+0000 7fe752a00640 1 -- 172.21.15.86:0/132273854 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe75c150510 con 0x7fe75c0b59e0 2024-04-25T14:04:03.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.818+0000 7fe761600640 1 -- 172.21.15.86:0/132273854 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe75c074cf0 con 0x7fe75c0b59e0 2024-04-25T14:04:03.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.818+0000 7fe752a00640 1 -- 172.21.15.86:0/132273854 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fe74805c800 con 0x7fe75c0b59e0 2024-04-25T14:04:03.819 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:04:03.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.822+0000 7fe761600640 1 -- 172.21.15.86:0/132273854 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe73c06f430 msgr2=0x7fe73c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:03.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.822+0000 7fe761600640 1 --2- 172.21.15.86:0/132273854 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe73c06f430 0x7fe73c071940 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0012c0 tx=0x7fe74c009040 comp rx=0 tx=0).stop 2024-04-25T14:04:03.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.822+0000 7fe761600640 1 -- 172.21.15.86:0/132273854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe75c0b59e0 msgr2=0x7fe75c161960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:03.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.822+0000 7fe761600640 1 --2- 172.21.15.86:0/132273854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe75c0b59e0 0x7fe75c161960 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe74800d830 tx=0x7fe74800dcf0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.822+0000 7fe761600640 1 -- 172.21.15.86:0/132273854 shutdown_connections 2024-04-25T14:04:03.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.822+0000 7fe761600640 1 --2- 172.21.15.86:0/132273854 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe73c06f430 0x7fe73c071940 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.822+0000 7fe761600640 1 --2- 172.21.15.86:0/132273854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe75c15faa0 0x7fe75c161ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.822+0000 7fe761600640 1 --2- 172.21.15.86:0/132273854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe75c0b59e0 0x7fe75c161960 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.822+0000 7fe761600640 1 --2- 172.21.15.86:0/132273854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe75c0aeea0 0x7fe75c13e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.822+0000 7fe761600640 1 -- 172.21.15.86:0/132273854 >> 172.21.15.86:0/132273854 conn(0x7fe75c0d2230 msgr2=0x7fe75c068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:03.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.822+0000 7fe761600640 1 -- 172.21.15.86:0/132273854 shutdown_connections 2024-04-25T14:04:03.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.822+0000 7fe761600640 1 -- 172.21.15.86:0/132273854 wait complete. 2024-04-25T14:04:03.840 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T14:04:03.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.990+0000 7f1d6f800640 1 Processor -- start 2024-04-25T14:04:03.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.990+0000 7f1d6f800640 1 -- start start 2024-04-25T14:04:03.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.990+0000 7f1d6f800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d68070330 0x7f1d680d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:03.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.990+0000 7f1d6f800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1d680ce790 0x7f1d6806f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:03.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.990+0000 7f1d6f800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d680d2030 con 0x7f1d680d1c20 2024-04-25T14:04:03.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.990+0000 7f1d6f800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1d6815e5c0 con 0x7f1d680ce790 2024-04-25T14:04:03.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.990+0000 7f1d6f800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1d6815e8a0 con 0x7f1d68070330 2024-04-25T14:04:03.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.990+0000 7f1d6ce00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d68070330 0x7f1d680d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.990+0000 7f1d6d800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1d680ce790 0x7f1d6806f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.990+0000 7f1d5fe00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f1d680d1c20 0x7f1d6806ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:44934/0 (socket says 172.21.15.86:44934) 2024-04-25T14:04:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.990+0000 7f1d5fe00640 1 -- 172.21.15.86:0/800971173 learned_addr learned my addr 172.21.15.86:0/800971173 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.990+0000 7f1d67e00640 1 -- 172.21.15.86:0/800971173 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1025694651 0 0) 0x7f1d50001120 con 0x7f1d680d1c20 2024-04-25T14:04:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.990+0000 7f1d67e00640 1 -- 172.21.15.86:0/800971173 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d4c003530 con 0x7f1d680d1c20 2024-04-25T14:04:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.990+0000 7f1d6d800640 1 -- 172.21.15.86:0/800971173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d68070330 msgr2=0x7f1d680d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.990+0000 7f1d6d800640 1 --2- 172.21.15.86:0/800971173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d68070330 0x7f1d680d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.990+0000 7f1d6d800640 1 -- 172.21.15.86:0/800971173 >> v1:172.21.15.155:6789/0 conn(0x7f1d680d1c20 legacy=0x7f1d6806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.990+0000 7f1d6d800640 1 -- 172.21.15.86:0/800971173 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6815eb80 con 0x7f1d680ce790 2024-04-25T14:04:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.990+0000 7f1d6ce00640 1 --2- 172.21.15.86:0/800971173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d68070330 0x7f1d680d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:03.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.994+0000 7f1d6d800640 1 --2- 172.21.15.86:0/800971173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d680ce790 0x7f1d6806f440 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1d600097f0 tx=0x7f1d6001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=265dd96d1d644b29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:03.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.994+0000 7f1d67e00640 1 -- 172.21.15.86:0/800971173 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d600b0070 con 0x7f1d680ce790 2024-04-25T14:04:03.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.994+0000 7f1d67e00640 1 -- 172.21.15.86:0/800971173 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d600ab600 con 0x7f1d680ce790 2024-04-25T14:04:03.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.994+0000 7f1d67e00640 1 -- 172.21.15.86:0/800971173 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d600ab9d0 con 0x7f1d680ce790 2024-04-25T14:04:03.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.994+0000 7f1d6f800640 1 -- 172.21.15.86:0/800971173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d680ce790 msgr2=0x7f1d6806f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:03.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.994+0000 7f1d6f800640 1 --2- 172.21.15.86:0/800971173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d680ce790 0x7f1d6806f440 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1d600097f0 tx=0x7f1d6001c410 comp rx=0 tx=0).stop 2024-04-25T14:04:03.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.994+0000 7f1d6f800640 1 -- 172.21.15.86:0/800971173 shutdown_connections 2024-04-25T14:04:03.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.994+0000 7f1d6f800640 1 --2- 172.21.15.86:0/800971173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d680ce790 0x7f1d6806f440 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.994+0000 7f1d6f800640 1 --2- 172.21.15.86:0/800971173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d68070330 0x7f1d680d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.994+0000 7f1d6f800640 1 -- 172.21.15.86:0/800971173 >> 172.21.15.86:0/800971173 conn(0x7f1d680b4650 msgr2=0x7f1d680b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:03.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.994+0000 7f1d6f800640 1 -- 172.21.15.86:0/800971173 shutdown_connections 2024-04-25T14:04:03.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.998+0000 7f1d6f800640 1 -- 172.21.15.86:0/800971173 wait complete. 2024-04-25T14:04:03.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.998+0000 7f1d6f800640 1 Processor -- start 2024-04-25T14:04:03.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.998+0000 7f1d6f800640 1 -- start start 2024-04-25T14:04:04.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.998+0000 7f1d6f800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d68070330 0x7f1d681fd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:04.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.998+0000 7f1d6f800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d680ce790 0x7f1d6815b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:04.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.998+0000 7f1d6f800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d680d1c20 0x7f1d680cff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:04.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.998+0000 7f1d6f800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1d680708c0 con 0x7f1d680d1c20 2024-04-25T14:04:04.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.998+0000 7f1d6f800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1d68070740 con 0x7f1d68070330 2024-04-25T14:04:04.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:03.998+0000 7f1d6f800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1d6816afc0 con 0x7f1d680ce790 2024-04-25T14:04:04.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.002+0000 7f1d6d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d680d1c20 0x7f1d680cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:04.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.002+0000 7f1d6d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d680d1c20 0x7f1d680cff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36312/0 (socket says 172.21.15.86:36312) 2024-04-25T14:04:04.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.002+0000 7f1d6d800640 1 -- 172.21.15.86:0/3099142757 learned_addr learned my addr 172.21.15.86:0/3099142757 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:04.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.002+0000 7f1d6ce00640 1 --2- 172.21.15.86:0/3099142757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d68070330 0x7f1d681fd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:04.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.002+0000 7f1d5fe00640 1 --2- 172.21.15.86:0/3099142757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d680ce790 0x7f1d6815b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:04.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.002+0000 7f1d6d800640 1 -- 172.21.15.86:0/3099142757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d680ce790 msgr2=0x7f1d6815b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:04.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.002+0000 7f1d6d800640 1 --2- 172.21.15.86:0/3099142757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d680ce790 0x7f1d6815b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:04.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.002+0000 7f1d6d800640 1 -- 172.21.15.86:0/3099142757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d68070330 msgr2=0x7f1d681fd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:04.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.002+0000 7f1d6d800640 1 --2- 172.21.15.86:0/3099142757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d68070330 0x7f1d681fd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:04.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.002+0000 7f1d6d800640 1 -- 172.21.15.86:0/3099142757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d68164060 con 0x7f1d680d1c20 2024-04-25T14:04:04.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.002+0000 7f1d5fe00640 1 --2- 172.21.15.86:0/3099142757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d680ce790 0x7f1d6815b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:04.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.002+0000 7f1d6ce00640 1 --2- 172.21.15.86:0/3099142757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d68070330 0x7f1d681fd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:04.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.002+0000 7f1d6d800640 1 --2- 172.21.15.86:0/3099142757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d680d1c20 0x7f1d680cff00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1d6001aac0 tx=0x7f1d600191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:04.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.002+0000 7f1d65600640 1 -- 172.21.15.86:0/3099142757 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d600b0040 con 0x7f1d680d1c20 2024-04-25T14:04:04.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.002+0000 7f1d65600640 1 -- 172.21.15.86:0/3099142757 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d600b9070 con 0x7f1d680d1c20 2024-04-25T14:04:04.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.002+0000 7f1d6f800640 1 -- 172.21.15.86:0/3099142757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d68163d80 con 0x7f1d680d1c20 2024-04-25T14:04:04.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.002+0000 7f1d65600640 1 -- 172.21.15.86:0/3099142757 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d600b4650 con 0x7f1d680d1c20 2024-04-25T14:04:04.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.002+0000 7f1d6f800640 1 -- 172.21.15.86:0/3099142757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d6809d2f0 con 0x7f1d680d1c20 2024-04-25T14:04:04.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.002+0000 7f1d6f800640 1 -- 172.21.15.86:0/3099142757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d68163ae0 con 0x7f1d680d1c20 2024-04-25T14:04:04.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.002+0000 7f1d65600640 1 -- 172.21.15.86:0/3099142757 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1d600b47f0 con 0x7f1d680d1c20 2024-04-25T14:04:04.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.006+0000 7f1d65600640 1 --2- 172.21.15.86:0/3099142757 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1d4406f430 0x7f1d44071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:04.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.006+0000 7f1d65600640 1 -- 172.21.15.86:0/3099142757 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1d601291e0 con 0x7f1d680d1c20 2024-04-25T14:04:04.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.006+0000 7f1d6ce00640 1 --2- 172.21.15.86:0/3099142757 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1d4406f430 0x7f1d44071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:04.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.006+0000 7f1d6ce00640 1 --2- 172.21.15.86:0/3099142757 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1d4406f430 0x7f1d44071940 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1d68072200 tx=0x7f1d5800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:04.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.010+0000 7f1d65600640 1 -- 172.21.15.86:0/3099142757 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1d68163ae0 con 0x7f1d680d1c20 2024-04-25T14:04:04.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.166+0000 7f1d6f800640 1 -- 172.21.15.86:0/3099142757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f1d6806a340 con 0x7f1d680d1c20 2024-04-25T14:04:04.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.166+0000 7f1d65600640 1 -- 172.21.15.86:0/3099142757 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f1d60016090 con 0x7f1d680d1c20 2024-04-25T14:04:04.167 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:04:04.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.166+0000 7f1d6f800640 1 -- 172.21.15.86:0/3099142757 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1d4406f430 msgr2=0x7f1d44071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:04.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.166+0000 7f1d6f800640 1 --2- 172.21.15.86:0/3099142757 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1d4406f430 0x7f1d44071940 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1d68072200 tx=0x7f1d5800b040 comp rx=0 tx=0).stop 2024-04-25T14:04:04.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.166+0000 7f1d6f800640 1 -- 172.21.15.86:0/3099142757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d680d1c20 msgr2=0x7f1d680cff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:04.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.166+0000 7f1d6f800640 1 --2- 172.21.15.86:0/3099142757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d680d1c20 0x7f1d680cff00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1d6001aac0 tx=0x7f1d600191b0 comp rx=0 tx=0).stop 2024-04-25T14:04:04.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.166+0000 7f1d6f800640 1 -- 172.21.15.86:0/3099142757 shutdown_connections 2024-04-25T14:04:04.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.166+0000 7f1d6f800640 1 --2- 172.21.15.86:0/3099142757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d680d1c20 0x7f1d680cff00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:04.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.166+0000 7f1d6f800640 1 --2- 172.21.15.86:0/3099142757 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1d4406f430 0x7f1d44071940 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:04.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.166+0000 7f1d6f800640 1 --2- 172.21.15.86:0/3099142757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d680ce790 0x7f1d6815b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.166+0000 7f1d6f800640 1 --2- 172.21.15.86:0/3099142757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d68070330 0x7f1d681fd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.166+0000 7f1d6f800640 1 -- 172.21.15.86:0/3099142757 >> 172.21.15.86:0/3099142757 conn(0x7f1d680b4650 msgr2=0x7f1d680b1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.170+0000 7f1d6f800640 1 -- 172.21.15.86:0/3099142757 shutdown_connections 2024-04-25T14:04:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.170+0000 7f1d6f800640 1 -- 172.21.15.86:0/3099142757 wait complete. 2024-04-25T14:04:04.191 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T14:04:04.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.337+0000 7f61e7000640 1 Processor -- start 2024-04-25T14:04:04.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.337+0000 7f61e7000640 1 -- start start 2024-04-25T14:04:04.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.337+0000 7f61e7000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f61e00cfca0 0x7f61e006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:04.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.337+0000 7f61e7000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61e00730b0 0x7f61e00734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:04.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.337+0000 7f61e7000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61e0072bd0 con 0x7f61e00d16e0 2024-04-25T14:04:04.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.337+0000 7f61e7000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f61e0070470 con 0x7f61e00cfca0 2024-04-25T14:04:04.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.337+0000 7f61e7000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f61e006f930 con 0x7f61e00730b0 2024-04-25T14:04:04.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.337+0000 7f61df400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f61e00d16e0 0x7f61e00d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52028/0 (socket says 172.21.15.86:52028) 2024-04-25T14:04:04.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.337+0000 7f61df400640 1 -- 172.21.15.86:0/394177199 learned_addr learned my addr 172.21.15.86:0/394177199 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:04.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.337+0000 7f61dfe00640 1 --2- 172.21.15.86:0/394177199 >> v2:172.21.15.155:3300/0 conn(0x7f61e00cfca0 0x7f61e006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:04.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.337+0000 7f61e5000640 1 --2- 172.21.15.86:0/394177199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61e00730b0 0x7f61e00734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:04.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.337+0000 7f61dea00640 1 -- 172.21.15.86:0/394177199 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 95333699 0 0) 0x7f61e0072bd0 con 0x7f61e00d16e0 2024-04-25T14:04:04.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.337+0000 7f61dea00640 1 -- 172.21.15.86:0/394177199 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61c4003530 con 0x7f61e00d16e0 2024-04-25T14:04:04.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.337+0000 7f61e5000640 1 -- 172.21.15.86:0/394177199 >> v2:172.21.15.155:3300/0 conn(0x7f61e00cfca0 msgr2=0x7f61e006f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:04.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.337+0000 7f61e5000640 1 --2- 172.21.15.86:0/394177199 >> v2:172.21.15.155:3300/0 conn(0x7f61e00cfca0 0x7f61e006f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:04.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.337+0000 7f61e5000640 1 -- 172.21.15.86:0/394177199 >> v1:172.21.15.86:6789/0 conn(0x7f61e00d16e0 legacy=0x7f61e00d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:04.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.337+0000 7f61e5000640 1 -- 172.21.15.86:0/394177199 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e006fbd0 con 0x7f61e00730b0 2024-04-25T14:04:04.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.337+0000 7f61dfe00640 1 --2- 172.21.15.86:0/394177199 >> v2:172.21.15.155:3300/0 conn(0x7f61e00cfca0 0x7f61e006f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:04.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.341+0000 7f61e5000640 1 --2- 172.21.15.86:0/394177199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61e00730b0 0x7f61e00734c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009d90 tx=0x7f61d801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dfe78602329f6e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:04.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.341+0000 7f61dea00640 1 -- 172.21.15.86:0/394177199 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d80b2070 con 0x7f61e00730b0 2024-04-25T14:04:04.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.341+0000 7f61dea00640 1 -- 172.21.15.86:0/394177199 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d80a5940 con 0x7f61e00730b0 2024-04-25T14:04:04.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.341+0000 7f61dea00640 1 -- 172.21.15.86:0/394177199 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d80a5d10 con 0x7f61e00730b0 2024-04-25T14:04:04.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.341+0000 7f61e7000640 1 -- 172.21.15.86:0/394177199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61e00730b0 msgr2=0x7f61e00734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:04.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.341+0000 7f61e7000640 1 --2- 172.21.15.86:0/394177199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61e00730b0 0x7f61e00734c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009d90 tx=0x7f61d801c970 comp rx=0 tx=0).stop 2024-04-25T14:04:04.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.341+0000 7f61e7000640 1 -- 172.21.15.86:0/394177199 shutdown_connections 2024-04-25T14:04:04.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.341+0000 7f61e7000640 1 --2- 172.21.15.86:0/394177199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61e00730b0 0x7f61e00734c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:04.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.341+0000 7f61e7000640 1 --2- 172.21.15.86:0/394177199 >> v2:172.21.15.155:3300/0 conn(0x7f61e00cfca0 0x7f61e006f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:04.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.341+0000 7f61e7000640 1 -- 172.21.15.86:0/394177199 >> 172.21.15.86:0/394177199 conn(0x7f61e00b4650 msgr2=0x7f61e00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:04.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.341+0000 7f61e7000640 1 -- 172.21.15.86:0/394177199 shutdown_connections 2024-04-25T14:04:04.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.341+0000 7f61e7000640 1 -- 172.21.15.86:0/394177199 wait complete. 2024-04-25T14:04:04.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.345+0000 7f61e7000640 1 Processor -- start 2024-04-25T14:04:04.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.345+0000 7f61e7000640 1 -- start start 2024-04-25T14:04:04.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.345+0000 7f61e7000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61e00730b0 0x7f61e009d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:04.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.345+0000 7f61e7000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61e00cfca0 0x7f61e01f7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:04.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.345+0000 7f61e7000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61e00d16e0 0x7f61e0159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:04.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.345+0000 7f61e7000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61e0169a70 con 0x7f61e00730b0 2024-04-25T14:04:04.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.345+0000 7f61e7000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01698f0 con 0x7f61e00d16e0 2024-04-25T14:04:04.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.345+0000 7f61e7000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f61e0169bf0 con 0x7f61e00cfca0 2024-04-25T14:04:04.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.345+0000 7f61dfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61e00730b0 0x7f61e009d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:04.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.345+0000 7f61dfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61e00730b0 0x7f61e009d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36318/0 (socket says 172.21.15.86:36318) 2024-04-25T14:04:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.345+0000 7f61dfe00640 1 -- 172.21.15.86:0/3643690072 learned_addr learned my addr 172.21.15.86:0/3643690072 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.345+0000 7f61e5000640 1 --2- 172.21.15.86:0/3643690072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61e00d16e0 0x7f61e0159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.345+0000 7f61df400640 1 --2- 172.21.15.86:0/3643690072 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61e00cfca0 0x7f61e01f7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.345+0000 7f61dfe00640 1 -- 172.21.15.86:0/3643690072 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61e00cfca0 msgr2=0x7f61e01f7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.345+0000 7f61dfe00640 1 --2- 172.21.15.86:0/3643690072 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61e00cfca0 0x7f61e01f7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.345+0000 7f61dfe00640 1 -- 172.21.15.86:0/3643690072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61e00d16e0 msgr2=0x7f61e0159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.345+0000 7f61dfe00640 1 --2- 172.21.15.86:0/3643690072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61e00d16e0 0x7f61e0159f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.345+0000 7f61dfe00640 1 -- 172.21.15.86:0/3643690072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e006a3f0 con 0x7f61e00730b0 2024-04-25T14:04:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.345+0000 7f61e5000640 1 --2- 172.21.15.86:0/3643690072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61e00d16e0 0x7f61e0159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:04.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.345+0000 7f61dfe00640 1 --2- 172.21.15.86:0/3643690072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61e00730b0 0x7f61e009d3e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f61d000e960 tx=0x7f61d000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:04.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.349+0000 7f61d7e00640 1 -- 172.21.15.86:0/3643690072 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0017070 con 0x7f61e00730b0 2024-04-25T14:04:04.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.349+0000 7f61d7e00640 1 -- 172.21.15.86:0/3643690072 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d000ca00 con 0x7f61e00730b0 2024-04-25T14:04:04.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.349+0000 7f61d7e00640 1 -- 172.21.15.86:0/3643690072 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d000cdd0 con 0x7f61e00730b0 2024-04-25T14:04:04.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.349+0000 7f61e7000640 1 -- 172.21.15.86:0/3643690072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e01597d0 con 0x7f61e00730b0 2024-04-25T14:04:04.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.349+0000 7f61e7000640 1 -- 172.21.15.86:0/3643690072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e009cb10 con 0x7f61e00730b0 2024-04-25T14:04:04.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.349+0000 7f61e7000640 1 -- 172.21.15.86:0/3643690072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61e009f4b0 con 0x7f61e00730b0 2024-04-25T14:04:04.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.349+0000 7f61d7e00640 1 -- 172.21.15.86:0/3643690072 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f61d001a070 con 0x7f61e00730b0 2024-04-25T14:04:04.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.349+0000 7f61d7e00640 1 --2- 172.21.15.86:0/3643690072 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f61c006f400 0x7f61c0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:04.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.349+0000 7f61d7e00640 1 -- 172.21.15.86:0/3643690072 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f61d008f5c0 con 0x7f61e00730b0 2024-04-25T14:04:04.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.349+0000 7f61df400640 1 --2- 172.21.15.86:0/3643690072 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f61c006f400 0x7f61c0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:04.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.353+0000 7f61df400640 1 --2- 172.21.15.86:0/3643690072 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f61c006f400 0x7f61c0071910 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f61e00ce290 tx=0x7f61cc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:04.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.353+0000 7f61d7e00640 1 -- 172.21.15.86:0/3643690072 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f61d005c0d0 con 0x7f61e00730b0 2024-04-25T14:04:04.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.521+0000 7f61e7000640 1 -- 172.21.15.86:0/3643690072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f61e0069d00 con 0x7f61e00730b0 2024-04-25T14:04:04.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.521+0000 7f61d7e00640 1 -- 172.21.15.86:0/3643690072 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f61d005b9d0 con 0x7f61e00730b0 2024-04-25T14:04:04.524 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-25T14:04:04.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.525+0000 7f61e7000640 1 -- 172.21.15.86:0/3643690072 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f61c006f400 msgr2=0x7f61c0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:04.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.525+0000 7f61e7000640 1 --2- 172.21.15.86:0/3643690072 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f61c006f400 0x7f61c0071910 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f61e00ce290 tx=0x7f61cc009040 comp rx=0 tx=0).stop 2024-04-25T14:04:04.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.525+0000 7f61e7000640 1 -- 172.21.15.86:0/3643690072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61e00730b0 msgr2=0x7f61e009d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:04.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.525+0000 7f61e7000640 1 --2- 172.21.15.86:0/3643690072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61e00730b0 0x7f61e009d3e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f61d000e960 tx=0x7f61d000ee20 comp rx=0 tx=0).stop 2024-04-25T14:04:04.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.525+0000 7f61e7000640 1 -- 172.21.15.86:0/3643690072 shutdown_connections 2024-04-25T14:04:04.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.525+0000 7f61e7000640 1 --2- 172.21.15.86:0/3643690072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61e00d16e0 0x7f61e0159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:04.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.525+0000 7f61e7000640 1 --2- 172.21.15.86:0/3643690072 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f61c006f400 0x7f61c0071910 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:04.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.525+0000 7f61e7000640 1 --2- 172.21.15.86:0/3643690072 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61e00cfca0 0x7f61e01f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:04.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.525+0000 7f61e7000640 1 --2- 172.21.15.86:0/3643690072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61e00730b0 0x7f61e009d3e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:04.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.525+0000 7f61e7000640 1 -- 172.21.15.86:0/3643690072 >> 172.21.15.86:0/3643690072 conn(0x7f61e00b4650 msgr2=0x7f61e00b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:04.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.525+0000 7f61e7000640 1 -- 172.21.15.86:0/3643690072 shutdown_connections 2024-04-25T14:04:04.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.525+0000 7f61e7000640 1 -- 172.21.15.86:0/3643690072 wait complete. 2024-04-25T14:04:04.548 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:04:04.548 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:04:04.556 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:04:04.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.745+0000 7f5ee9e00640 1 Processor -- start 2024-04-25T14:04:04.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.745+0000 7f5ee9e00640 1 -- start start 2024-04-25T14:04:04.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.745+0000 7f5ee9e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5ee40cdd70 0x7f5ee40ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:04.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.745+0000 7f5ee9e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ee4070e20 0x7f5ee4071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:04.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.745+0000 7f5ee9e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ee4071770 con 0x7f5ee406e700 2024-04-25T14:04:04.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.745+0000 7f5ee9e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5ee406d940 con 0x7f5ee40cdd70 2024-04-25T14:04:04.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.745+0000 7f5ee9e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5ee406dc20 con 0x7f5ee4070e20 2024-04-25T14:04:04.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.745+0000 7f5ee2a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5ee40cdd70 0x7f5ee40ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:04.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.745+0000 7f5ee2a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5ee40cdd70 0x7f5ee40ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36334/0 (socket says 172.21.15.86:36334) 2024-04-25T14:04:04.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.745+0000 7f5ee2a00640 1 -- 172.21.15.86:0/2789436407 learned_addr learned my addr 172.21.15.86:0/2789436407 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:04.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.745+0000 7f5ee2000640 1 --2- 172.21.15.86:0/2789436407 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ee4070e20 0x7f5ee4071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:04.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.745+0000 7f5ee2a00640 1 -- 172.21.15.86:0/2789436407 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ee4070e20 msgr2=0x7f5ee4071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:04.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.745+0000 7f5ee2a00640 1 --2- 172.21.15.86:0/2789436407 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ee4070e20 0x7f5ee4071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:04.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.745+0000 7f5ee2a00640 1 -- 172.21.15.86:0/2789436407 >> v1:172.21.15.155:6789/0 conn(0x7f5ee406e700 legacy=0x7f5ee406eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:04.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.745+0000 7f5ee2a00640 1 -- 172.21.15.86:0/2789436407 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee406df00 con 0x7f5ee40cdd70 2024-04-25T14:04:04.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.749+0000 7f5ee2a00640 1 --2- 172.21.15.86:0/2789436407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee40cdd70 0x7f5ee40ce180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009870 tx=0x7f5ed401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2ef5496118366b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:04.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.749+0000 7f5ee1600640 1 -- 172.21.15.86:0/2789436407 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed40b0070 con 0x7f5ee40cdd70 2024-04-25T14:04:04.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.749+0000 7f5ee1600640 1 -- 172.21.15.86:0/2789436407 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ed40ab600 con 0x7f5ee40cdd70 2024-04-25T14:04:04.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.749+0000 7f5ee1600640 1 -- 172.21.15.86:0/2789436407 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed40ab9d0 con 0x7f5ee40cdd70 2024-04-25T14:04:04.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.749+0000 7f5ee9e00640 1 -- 172.21.15.86:0/2789436407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee40cdd70 msgr2=0x7f5ee40ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:04.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.749+0000 7f5ee9e00640 1 --2- 172.21.15.86:0/2789436407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee40cdd70 0x7f5ee40ce180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009870 tx=0x7f5ed401c4d0 comp rx=0 tx=0).stop 2024-04-25T14:04:04.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.749+0000 7f5ee9e00640 1 -- 172.21.15.86:0/2789436407 shutdown_connections 2024-04-25T14:04:04.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.749+0000 7f5ee9e00640 1 --2- 172.21.15.86:0/2789436407 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ee4070e20 0x7f5ee4071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:04.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.749+0000 7f5ee9e00640 1 --2- 172.21.15.86:0/2789436407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee40cdd70 0x7f5ee40ce180 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:04.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.749+0000 7f5ee9e00640 1 -- 172.21.15.86:0/2789436407 >> 172.21.15.86:0/2789436407 conn(0x7f5ee40cffc0 msgr2=0x7f5ee40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:04.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.749+0000 7f5ee9e00640 1 -- 172.21.15.86:0/2789436407 shutdown_connections 2024-04-25T14:04:04.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.749+0000 7f5ee9e00640 1 -- 172.21.15.86:0/2789436407 wait complete. 2024-04-25T14:04:04.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.753+0000 7f5ee9e00640 1 Processor -- start 2024-04-25T14:04:04.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.753+0000 7f5ee9e00640 1 -- start start 2024-04-25T14:04:04.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.753+0000 7f5ee9e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ee406e700 0x7f5ee40b86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:04.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.753+0000 7f5ee9e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ee4070e20 0x7f5ee40cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:04.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.753+0000 7f5ee9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee40cdd70 0x7f5ee40ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:04.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.753+0000 7f5ee9e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee416cc00 con 0x7f5ee40cdd70 2024-04-25T14:04:04.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.753+0000 7f5ee9e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee4164380 con 0x7f5ee4070e20 2024-04-25T14:04:04.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.753+0000 7f5ee9e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5ee416cd80 con 0x7f5ee406e700 2024-04-25T14:04:04.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.753+0000 7f5ee3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee40cdd70 0x7f5ee40ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:04.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.753+0000 7f5ee3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee40cdd70 0x7f5ee40ca580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36344/0 (socket says 172.21.15.86:36344) 2024-04-25T14:04:04.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.753+0000 7f5ee3400640 1 -- 172.21.15.86:0/3789349972 learned_addr learned my addr 172.21.15.86:0/3789349972 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:04.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.753+0000 7f5ee2000640 1 --2- 172.21.15.86:0/3789349972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ee4070e20 0x7f5ee40cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:04.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.753+0000 7f5ee2a00640 1 --2- 172.21.15.86:0/3789349972 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ee406e700 0x7f5ee40b86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:04.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.757+0000 7f5ee3400640 1 -- 172.21.15.86:0/3789349972 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ee406e700 msgr2=0x7f5ee40b86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:04.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.757+0000 7f5ee3400640 1 --2- 172.21.15.86:0/3789349972 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ee406e700 0x7f5ee40b86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:04.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.757+0000 7f5ee3400640 1 -- 172.21.15.86:0/3789349972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ee4070e20 msgr2=0x7f5ee40cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:04.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.757+0000 7f5ee3400640 1 --2- 172.21.15.86:0/3789349972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ee4070e20 0x7f5ee40cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:04.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.757+0000 7f5ee3400640 1 -- 172.21.15.86:0/3789349972 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee40caac0 con 0x7f5ee40cdd70 2024-04-25T14:04:04.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.757+0000 7f5ee3400640 1 --2- 172.21.15.86:0/3789349972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee40cdd70 0x7f5ee40ca580 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5edc00c730 tx=0x7f5edc00cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:04.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.757+0000 7f5edb400640 1 -- 172.21.15.86:0/3789349972 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5edc016020 con 0x7f5ee40cdd70 2024-04-25T14:04:04.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.757+0000 7f5edb400640 1 -- 172.21.15.86:0/3789349972 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5edc006d60 con 0x7f5ee40cdd70 2024-04-25T14:04:04.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.757+0000 7f5edb400640 1 -- 172.21.15.86:0/3789349972 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5edc003bc0 con 0x7f5ee40cdd70 2024-04-25T14:04:04.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.757+0000 7f5ee9e00640 1 -- 172.21.15.86:0/3789349972 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ee41fcac0 con 0x7f5ee40cdd70 2024-04-25T14:04:04.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.757+0000 7f5ee9e00640 1 -- 172.21.15.86:0/3789349972 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ee41fa000 con 0x7f5ee40cdd70 2024-04-25T14:04:04.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.757+0000 7f5ee9e00640 1 -- 172.21.15.86:0/3789349972 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ee40720e0 con 0x7f5ee40cdd70 2024-04-25T14:04:04.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.757+0000 7f5edb400640 1 -- 172.21.15.86:0/3789349972 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5edc0050d0 con 0x7f5ee40cdd70 2024-04-25T14:04:04.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.757+0000 7f5edb400640 1 --2- 172.21.15.86:0/3789349972 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f5ec406f400 0x7f5ec4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:04.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.757+0000 7f5edb400640 1 -- 172.21.15.86:0/3789349972 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5edc08f0e0 con 0x7f5ee40cdd70 2024-04-25T14:04:04.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.761+0000 7f5ee2a00640 1 --2- 172.21.15.86:0/3789349972 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f5ec406f400 0x7f5ec4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:04.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.761+0000 7f5ee2a00640 1 --2- 172.21.15.86:0/3789349972 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f5ec406f400 0x7f5ec4071910 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5ed40ad000 tx=0x7f5ed4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:04.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.765+0000 7f5edb400640 1 -- 172.21.15.86:0/3789349972 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5edc020020 con 0x7f5ee40cdd70 2024-04-25T14:04:04.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.921+0000 7f5ee9e00640 1 -- 172.21.15.86:0/3789349972 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ee41f9ab0 con 0x7f5ee40cdd70 2024-04-25T14:04:04.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.921+0000 7f5edb400640 1 -- 172.21.15.86:0/3789349972 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7f5ee4071770 con 0x7f5ee40cdd70 2024-04-25T14:04:04.924 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:04.924 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":24,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:03:55.899092+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:03:49.151473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:03:49.802597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T14:03:53.492008+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:04:04.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.925+0000 7f5ee9e00640 1 -- 172.21.15.86:0/3789349972 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f5ec406f400 msgr2=0x7f5ec4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:04.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.925+0000 7f5ee9e00640 1 --2- 172.21.15.86:0/3789349972 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f5ec406f400 0x7f5ec4071910 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5ed40ad000 tx=0x7f5ed4008040 comp rx=0 tx=0).stop 2024-04-25T14:04:04.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.925+0000 7f5ee9e00640 1 -- 172.21.15.86:0/3789349972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee40cdd70 msgr2=0x7f5ee40ca580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:04.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.925+0000 7f5ee9e00640 1 --2- 172.21.15.86:0/3789349972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee40cdd70 0x7f5ee40ca580 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5edc00c730 tx=0x7f5edc00cb30 comp rx=0 tx=0).stop 2024-04-25T14:04:04.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.925+0000 7f5ee9e00640 1 -- 172.21.15.86:0/3789349972 shutdown_connections 2024-04-25T14:04:04.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.925+0000 7f5ee9e00640 1 --2- 172.21.15.86:0/3789349972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee40cdd70 0x7f5ee40ca580 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:04.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.925+0000 7f5ee9e00640 1 --2- 172.21.15.86:0/3789349972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ee4070e20 0x7f5ee40cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:04.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.925+0000 7f5ee9e00640 1 --2- 172.21.15.86:0/3789349972 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f5ec406f400 0x7f5ec4071910 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:04.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.925+0000 7f5ee9e00640 1 --2- 172.21.15.86:0/3789349972 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ee406e700 0x7f5ee40b86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:04.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.925+0000 7f5ee9e00640 1 -- 172.21.15.86:0/3789349972 >> 172.21.15.86:0/3789349972 conn(0x7f5ee40cffc0 msgr2=0x7f5ee415f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:04.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.925+0000 7f5ee9e00640 1 -- 172.21.15.86:0/3789349972 shutdown_connections 2024-04-25T14:04:04.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:04.925+0000 7f5ee9e00640 1 -- 172.21.15.86:0/3789349972 wait complete. 2024-04-25T14:04:04.946 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:03:49.151473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:03:49.802597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T14:03:53.492008+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T14:04:04.946 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:04:05.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.093+0000 7f5937400640 1 Processor -- start 2024-04-25T14:04:05.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.093+0000 7f5937400640 1 -- start start 2024-04-25T14:04:05.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.097+0000 7f5937400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f59300b59e0 0x7f59300b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:05.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.097+0000 7f5937400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f59300aeea0 0x7f59300af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:05.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.097+0000 7f5937400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59300b6330 con 0x7f593015faa0 2024-04-25T14:04:05.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.097+0000 7f5937400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f593006d560 con 0x7f59300b59e0 2024-04-25T14:04:05.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.097+0000 7f5937400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f593006c670 con 0x7f59300aeea0 2024-04-25T14:04:05.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.097+0000 7f5934a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f593015faa0 0x7f593006d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52038/0 (socket says 172.21.15.86:52038) 2024-04-25T14:04:05.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.097+0000 7f5934a00640 1 -- 172.21.15.86:0/1995609930 learned_addr learned my addr 172.21.15.86:0/1995609930 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:05.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.097+0000 7f5927e00640 1 --2- 172.21.15.86:0/1995609930 >> v2:172.21.15.86:3300/0 conn(0x7f59300b59e0 0x7f59300b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:05.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.097+0000 7f5935400640 1 --2- 172.21.15.86:0/1995609930 >> v2:172.21.15.155:3300/0 conn(0x7f59300aeea0 0x7f59300af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:05.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.097+0000 7f592fe00640 1 -- 172.21.15.86:0/1995609930 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 960850106 0 0) 0x7f5920001120 con 0x7f593015faa0 2024-04-25T14:04:05.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.097+0000 7f592fe00640 1 -- 172.21.15.86:0/1995609930 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5914003530 con 0x7f593015faa0 2024-04-25T14:04:05.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.097+0000 7f5927e00640 1 -- 172.21.15.86:0/1995609930 >> v2:172.21.15.155:3300/0 conn(0x7f59300aeea0 msgr2=0x7f59300af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:05.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.097+0000 7f5927e00640 1 --2- 172.21.15.86:0/1995609930 >> v2:172.21.15.155:3300/0 conn(0x7f59300aeea0 0x7f59300af2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:05.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.097+0000 7f5927e00640 1 -- 172.21.15.86:0/1995609930 >> v1:172.21.15.86:6789/0 conn(0x7f593015faa0 legacy=0x7f593006d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:05.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.097+0000 7f5927e00640 1 -- 172.21.15.86:0/1995609930 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593006c950 con 0x7f59300b59e0 2024-04-25T14:04:05.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.097+0000 7f5927e00640 1 --2- 172.21.15.86:0/1995609930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59300b59e0 0x7f59300b5df0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f59300765a0 tx=0x7f591801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce96bd44da7483c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:05.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.101+0000 7f592fe00640 1 -- 172.21.15.86:0/1995609930 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59180aa600 con 0x7f59300b59e0 2024-04-25T14:04:05.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.101+0000 7f592fe00640 1 -- 172.21.15.86:0/1995609930 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59180af070 con 0x7f59300b59e0 2024-04-25T14:04:05.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.101+0000 7f592fe00640 1 -- 172.21.15.86:0/1995609930 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59180aa9c0 con 0x7f59300b59e0 2024-04-25T14:04:05.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.101+0000 7f5937400640 1 -- 172.21.15.86:0/1995609930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59300b59e0 msgr2=0x7f59300b5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:05.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.101+0000 7f5937400640 1 --2- 172.21.15.86:0/1995609930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59300b59e0 0x7f59300b5df0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f59300765a0 tx=0x7f591801c480 comp rx=0 tx=0).stop 2024-04-25T14:04:05.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.101+0000 7f5937400640 1 -- 172.21.15.86:0/1995609930 shutdown_connections 2024-04-25T14:04:05.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.101+0000 7f5937400640 1 --2- 172.21.15.86:0/1995609930 >> v2:172.21.15.155:3300/0 conn(0x7f59300aeea0 0x7f59300af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:05.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.101+0000 7f5937400640 1 --2- 172.21.15.86:0/1995609930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59300b59e0 0x7f59300b5df0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:05.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.101+0000 7f5937400640 1 -- 172.21.15.86:0/1995609930 >> 172.21.15.86:0/1995609930 conn(0x7f59300d2230 msgr2=0x7f59300d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:05.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.101+0000 7f5937400640 1 -- 172.21.15.86:0/1995609930 shutdown_connections 2024-04-25T14:04:05.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.101+0000 7f5937400640 1 -- 172.21.15.86:0/1995609930 wait complete. 2024-04-25T14:04:05.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.101+0000 7f5937400640 1 Processor -- start 2024-04-25T14:04:05.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.101+0000 7f5937400640 1 -- start start 2024-04-25T14:04:05.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.105+0000 7f5937400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59300aeea0 0x7f5930155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:05.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.105+0000 7f5937400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59300b59e0 0x7f59300b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:05.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.105+0000 7f5937400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f593015faa0 0x7f59300b2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:05.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.105+0000 7f5937400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5930160030 con 0x7f59300aeea0 2024-04-25T14:04:05.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.105+0000 7f5937400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f593015feb0 con 0x7f593015faa0 2024-04-25T14:04:05.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.105+0000 7f5937400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5930070f30 con 0x7f59300b59e0 2024-04-25T14:04:05.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.105+0000 7f5934a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59300aeea0 0x7f5930155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:05.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.105+0000 7f5934a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59300aeea0 0x7f5930155530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36352/0 (socket says 172.21.15.86:36352) 2024-04-25T14:04:05.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.105+0000 7f5934a00640 1 -- 172.21.15.86:0/3863705061 learned_addr learned my addr 172.21.15.86:0/3863705061 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:05.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.105+0000 7f5927e00640 1 --2- 172.21.15.86:0/3863705061 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59300b59e0 0x7f59300b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:05.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.105+0000 7f5935400640 1 --2- 172.21.15.86:0/3863705061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f593015faa0 0x7f59300b2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:05.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.105+0000 7f5934a00640 1 -- 172.21.15.86:0/3863705061 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59300b59e0 msgr2=0x7f59300b25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:05.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.105+0000 7f5934a00640 1 --2- 172.21.15.86:0/3863705061 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59300b59e0 0x7f59300b25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:05.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.105+0000 7f5934a00640 1 -- 172.21.15.86:0/3863705061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f593015faa0 msgr2=0x7f59300b2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:05.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.105+0000 7f5934a00640 1 --2- 172.21.15.86:0/3863705061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f593015faa0 0x7f59300b2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:05.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.105+0000 7f5934a00640 1 -- 172.21.15.86:0/3863705061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59300b1fe0 con 0x7f59300aeea0 2024-04-25T14:04:05.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.105+0000 7f5935400640 1 --2- 172.21.15.86:0/3863705061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f593015faa0 0x7f59300b2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:05.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.105+0000 7f5934a00640 1 --2- 172.21.15.86:0/3863705061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59300aeea0 0x7f5930155530 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f592000c3f0 tx=0x7f592000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:05.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.105+0000 7f592d600640 1 -- 172.21.15.86:0/3863705061 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920017070 con 0x7f59300aeea0 2024-04-25T14:04:05.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.105+0000 7f592d600640 1 -- 172.21.15.86:0/3863705061 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5920005cf0 con 0x7f59300aeea0 2024-04-25T14:04:05.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.105+0000 7f592d600640 1 -- 172.21.15.86:0/3863705061 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920007310 con 0x7f59300aeea0 2024-04-25T14:04:05.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.105+0000 7f5937400640 1 -- 172.21.15.86:0/3863705061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59301638e0 con 0x7f59300aeea0 2024-04-25T14:04:05.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.105+0000 7f5937400640 1 -- 172.21.15.86:0/3863705061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59301668e0 con 0x7f59300aeea0 2024-04-25T14:04:05.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.105+0000 7f5937400640 1 -- 172.21.15.86:0/3863705061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5930074cf0 con 0x7f59300aeea0 2024-04-25T14:04:05.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.109+0000 7f592d600640 1 -- 172.21.15.86:0/3863705061 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f592001a070 con 0x7f59300aeea0 2024-04-25T14:04:05.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.109+0000 7f592d600640 1 --2- 172.21.15.86:0/3863705061 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f590c06f430 0x7f590c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:05.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.109+0000 7f592d600640 1 -- 172.21.15.86:0/3863705061 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f592008ead0 con 0x7f59300aeea0 2024-04-25T14:04:05.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.109+0000 7f5927e00640 1 --2- 172.21.15.86:0/3863705061 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f590c06f430 0x7f590c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:05.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.109+0000 7f5927e00640 1 --2- 172.21.15.86:0/3863705061 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f590c06f430 0x7f590c071940 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f59180ae000 tx=0x7f5918008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:05.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.113+0000 7f592d600640 1 -- 172.21.15.86:0/3863705061 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f59300b6330 con 0x7f59300aeea0 2024-04-25T14:04:05.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.269+0000 7f5937400640 1 -- 172.21.15.86:0/3863705061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f59300b55b0 con 0x7f59300aeea0 2024-04-25T14:04:05.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.269+0000 7f592d600640 1 -- 172.21.15.86:0/3863705061 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f592005b5e0 con 0x7f59300aeea0 2024-04-25T14:04:05.272 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:04:05.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.273+0000 7f5937400640 1 -- 172.21.15.86:0/3863705061 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f590c06f430 msgr2=0x7f590c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:05.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.273+0000 7f5937400640 1 --2- 172.21.15.86:0/3863705061 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f590c06f430 0x7f590c071940 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f59180ae000 tx=0x7f5918008040 comp rx=0 tx=0).stop 2024-04-25T14:04:05.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.273+0000 7f5937400640 1 -- 172.21.15.86:0/3863705061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59300aeea0 msgr2=0x7f5930155530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:05.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.273+0000 7f5937400640 1 --2- 172.21.15.86:0/3863705061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59300aeea0 0x7f5930155530 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f592000c3f0 tx=0x7f592000c8b0 comp rx=0 tx=0).stop 2024-04-25T14:04:05.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.273+0000 7f5937400640 1 -- 172.21.15.86:0/3863705061 shutdown_connections 2024-04-25T14:04:05.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.273+0000 7f5937400640 1 --2- 172.21.15.86:0/3863705061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f593015faa0 0x7f59300b2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:05.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.273+0000 7f5937400640 1 --2- 172.21.15.86:0/3863705061 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f590c06f430 0x7f590c071940 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:05.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.273+0000 7f5937400640 1 --2- 172.21.15.86:0/3863705061 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59300b59e0 0x7f59300b25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:05.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.273+0000 7f5937400640 1 --2- 172.21.15.86:0/3863705061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59300aeea0 0x7f5930155530 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:05.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.273+0000 7f5937400640 1 -- 172.21.15.86:0/3863705061 >> 172.21.15.86:0/3863705061 conn(0x7f59300d2230 msgr2=0x7f59301629b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:05.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.273+0000 7f5937400640 1 -- 172.21.15.86:0/3863705061 shutdown_connections 2024-04-25T14:04:05.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.273+0000 7f5937400640 1 -- 172.21.15.86:0/3863705061 wait complete. 2024-04-25T14:04:05.294 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:04:05.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.441+0000 7ff06f200640 1 Processor -- start 2024-04-25T14:04:05.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.441+0000 7ff06f200640 1 -- start start 2024-04-25T14:04:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.441+0000 7ff06f200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff0680740c0 0x7ff0681676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.441+0000 7ff06f200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff068072bd0 con 0x7ff0680d16e0 2024-04-25T14:04:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.441+0000 7ff06f200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff06806fbd0 con 0x7ff0680cfca0 2024-04-25T14:04:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.441+0000 7ff06f200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff068070470 con 0x7ff0680740c0 2024-04-25T14:04:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.445+0000 7ff06d200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff0680740c0 0x7ff0681676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.445+0000 7ff05f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff0680d16e0 0x7ff0680d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52050/0 (socket says 172.21.15.86:52050) 2024-04-25T14:04:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.445+0000 7ff05f400640 1 -- 172.21.15.86:0/3342036211 learned_addr learned my addr 172.21.15.86:0/3342036211 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:05.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.445+0000 7ff06d200640 1 -- 172.21.15.86:0/3342036211 >> v1:172.21.15.155:6789/0 conn(0x7ff0680cfca0 legacy=0x7ff06806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:05.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.445+0000 7ff06d200640 1 -- 172.21.15.86:0/3342036211 >> v1:172.21.15.86:6789/0 conn(0x7ff0680d16e0 legacy=0x7ff0680d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:05.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.445+0000 7ff06d200640 1 -- 172.21.15.86:0/3342036211 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06806feb0 con 0x7ff0680740c0 2024-04-25T14:04:05.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.445+0000 7ff06d200640 1 --2- 172.21.15.86:0/3342036211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0680740c0 0x7ff0681676d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff060009d90 tx=0x7ff06001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=284f8147d623a1d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:05.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.445+0000 7ff067400640 1 -- 172.21.15.86:0/3342036211 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0600b2070 con 0x7ff0680740c0 2024-04-25T14:04:05.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.445+0000 7ff067400640 1 -- 172.21.15.86:0/3342036211 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0600a5940 con 0x7ff0680740c0 2024-04-25T14:04:05.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.445+0000 7ff067400640 1 -- 172.21.15.86:0/3342036211 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0600a5d10 con 0x7ff0680740c0 2024-04-25T14:04:05.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.449+0000 7ff06f200640 1 -- 172.21.15.86:0/3342036211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0680740c0 msgr2=0x7ff0681676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:05.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.449+0000 7ff06f200640 1 --2- 172.21.15.86:0/3342036211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0680740c0 0x7ff0681676d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff060009d90 tx=0x7ff06001c970 comp rx=0 tx=0).stop 2024-04-25T14:04:05.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.449+0000 7ff06f200640 1 -- 172.21.15.86:0/3342036211 shutdown_connections 2024-04-25T14:04:05.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.449+0000 7ff06f200640 1 --2- 172.21.15.86:0/3342036211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0680740c0 0x7ff0681676d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:05.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.449+0000 7ff06f200640 1 -- 172.21.15.86:0/3342036211 >> 172.21.15.86:0/3342036211 conn(0x7ff0680b4650 msgr2=0x7ff0680b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:05.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.449+0000 7ff06f200640 1 -- 172.21.15.86:0/3342036211 shutdown_connections 2024-04-25T14:04:05.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.449+0000 7ff06f200640 1 -- 172.21.15.86:0/3342036211 wait complete. 2024-04-25T14:04:05.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.449+0000 7ff06f200640 1 Processor -- start 2024-04-25T14:04:05.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.449+0000 7ff06f200640 1 -- start start 2024-04-25T14:04:05.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.453+0000 7ff06f200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0680740c0 0x7ff06809a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:05.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.453+0000 7ff06f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0680cfca0 0x7ff068159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:05.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.453+0000 7ff06f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0680d16e0 0x7ff06815a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:05.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.453+0000 7ff06f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff06816ade0 con 0x7ff0680cfca0 2024-04-25T14:04:05.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.453+0000 7ff06f200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff06816ac60 con 0x7ff0680d16e0 2024-04-25T14:04:05.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.453+0000 7ff06f200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff06816af60 con 0x7ff0680740c0 2024-04-25T14:04:05.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.453+0000 7ff05f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0680cfca0 0x7ff068159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:05.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.453+0000 7ff05f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0680cfca0 0x7ff068159d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36378/0 (socket says 172.21.15.86:36378) 2024-04-25T14:04:05.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.453+0000 7ff05f400640 1 -- 172.21.15.86:0/1574764983 learned_addr learned my addr 172.21.15.86:0/1574764983 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:05.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.453+0000 7ff06d200640 1 --2- 172.21.15.86:0/1574764983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0680d16e0 0x7ff06815a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:05.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.453+0000 7ff05f400640 1 -- 172.21.15.86:0/1574764983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0680740c0 msgr2=0x7ff06809a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:05.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.453+0000 7ff067e00640 1 --2- 172.21.15.86:0/1574764983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0680740c0 0x7ff06809a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:05.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.453+0000 7ff05f400640 1 --2- 172.21.15.86:0/1574764983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0680740c0 0x7ff06809a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:05.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.453+0000 7ff05f400640 1 -- 172.21.15.86:0/1574764983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0680d16e0 msgr2=0x7ff06815a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:05.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.453+0000 7ff05f400640 1 --2- 172.21.15.86:0/1574764983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0680d16e0 0x7ff06815a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:05.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.453+0000 7ff05f400640 1 -- 172.21.15.86:0/1574764983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06809a680 con 0x7ff0680cfca0 2024-04-25T14:04:05.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.453+0000 7ff06d200640 1 --2- 172.21.15.86:0/1574764983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0680d16e0 0x7ff06815a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:05.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.453+0000 7ff067e00640 1 --2- 172.21.15.86:0/1574764983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0680740c0 0x7ff06809a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:05.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.453+0000 7ff05f400640 1 --2- 172.21.15.86:0/1574764983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0680cfca0 0x7ff068159d90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff058005890 tx=0x7ff05800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:05.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.453+0000 7ff064c00640 1 -- 172.21.15.86:0/1574764983 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff058017070 con 0x7ff0680cfca0 2024-04-25T14:04:05.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.453+0000 7ff064c00640 1 -- 172.21.15.86:0/1574764983 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff058005d10 con 0x7ff0680cfca0 2024-04-25T14:04:05.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.453+0000 7ff064c00640 1 -- 172.21.15.86:0/1574764983 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff058007310 con 0x7ff0680cfca0 2024-04-25T14:04:05.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.453+0000 7ff06f200640 1 -- 172.21.15.86:0/1574764983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff06815f2b0 con 0x7ff0680cfca0 2024-04-25T14:04:05.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.453+0000 7ff06f200640 1 -- 172.21.15.86:0/1574764983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff06815f870 con 0x7ff0680cfca0 2024-04-25T14:04:05.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.453+0000 7ff06f200640 1 -- 172.21.15.86:0/1574764983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff06809f4b0 con 0x7ff0680cfca0 2024-04-25T14:04:05.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.453+0000 7ff064c00640 1 -- 172.21.15.86:0/1574764983 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff05801a070 con 0x7ff0680cfca0 2024-04-25T14:04:05.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.457+0000 7ff064c00640 1 --2- 172.21.15.86:0/1574764983 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff04406f430 0x7ff044071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:05.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.457+0000 7ff064c00640 1 -- 172.21.15.86:0/1574764983 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff05808e980 con 0x7ff0680cfca0 2024-04-25T14:04:05.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.457+0000 7ff067e00640 1 --2- 172.21.15.86:0/1574764983 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff04406f430 0x7ff044071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:05.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.457+0000 7ff067e00640 1 --2- 172.21.15.86:0/1574764983 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff04406f430 0x7ff044071940 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff068070710 tx=0x7ff050009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:05.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.461+0000 7ff064c00640 1 -- 172.21.15.86:0/1574764983 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff05805b520 con 0x7ff0680cfca0 2024-04-25T14:04:05.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.617+0000 7ff06f200640 1 -- 172.21.15.86:0/1574764983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff068069d00 con 0x7ff0680cfca0 2024-04-25T14:04:05.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.617+0000 7ff064c00640 1 -- 172.21.15.86:0/1574764983 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7ff05805ae20 con 0x7ff0680cfca0 2024-04-25T14:04:05.619 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:04:05.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.617+0000 7ff06f200640 1 -- 172.21.15.86:0/1574764983 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff04406f430 msgr2=0x7ff044071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:05.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.617+0000 7ff06f200640 1 --2- 172.21.15.86:0/1574764983 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff04406f430 0x7ff044071940 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff068070710 tx=0x7ff050009040 comp rx=0 tx=0).stop 2024-04-25T14:04:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.617+0000 7ff06f200640 1 -- 172.21.15.86:0/1574764983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0680cfca0 msgr2=0x7ff068159d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.617+0000 7ff06f200640 1 --2- 172.21.15.86:0/1574764983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0680cfca0 0x7ff068159d90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff058005890 tx=0x7ff05800c890 comp rx=0 tx=0).stop 2024-04-25T14:04:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.617+0000 7ff06f200640 1 -- 172.21.15.86:0/1574764983 shutdown_connections 2024-04-25T14:04:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.617+0000 7ff06f200640 1 --2- 172.21.15.86:0/1574764983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0680d16e0 0x7ff06815a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.617+0000 7ff06f200640 1 --2- 172.21.15.86:0/1574764983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0680cfca0 0x7ff068159d90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.617+0000 7ff06f200640 1 --2- 172.21.15.86:0/1574764983 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff04406f430 0x7ff044071940 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.617+0000 7ff06f200640 1 --2- 172.21.15.86:0/1574764983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0680740c0 0x7ff06809a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.617+0000 7ff06f200640 1 -- 172.21.15.86:0/1574764983 >> 172.21.15.86:0/1574764983 conn(0x7ff0680b4650 msgr2=0x7ff06815bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.617+0000 7ff06f200640 1 -- 172.21.15.86:0/1574764983 shutdown_connections 2024-04-25T14:04:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.617+0000 7ff06f200640 1 -- 172.21.15.86:0/1574764983 wait complete. 2024-04-25T14:04:05.640 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:04:05.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.789+0000 7fdb63a00640 1 Processor -- start 2024-04-25T14:04:05.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.789+0000 7fdb63a00640 1 -- start start 2024-04-25T14:04:05.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.789+0000 7fdb63a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdb5c071530 0x7fdb5c15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:05.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.789+0000 7fdb63a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb5c1616e0 0x7fdb5c161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:05.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.789+0000 7fdb63a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb5c0cd8d0 con 0x7fdb5c0d2c80 2024-04-25T14:04:05.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.789+0000 7fdb63a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fdb5c0ce720 con 0x7fdb5c071530 2024-04-25T14:04:05.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.789+0000 7fdb63a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdb5c0ce970 con 0x7fdb5c1616e0 2024-04-25T14:04:05.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.793+0000 7fdb53e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdb5c071530 0x7fdb5c15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:05.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.793+0000 7fdb61a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb5c1616e0 0x7fdb5c161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:05.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.793+0000 7fdb61000640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fdb5c0d2c80 0x7fdb5c070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:44954/0 (socket says 172.21.15.86:44954) 2024-04-25T14:04:05.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.793+0000 7fdb61000640 1 -- 172.21.15.86:0/948411543 learned_addr learned my addr 172.21.15.86:0/948411543 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:05.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.793+0000 7fdb53e00640 1 -- 172.21.15.86:0/948411543 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb5c1616e0 msgr2=0x7fdb5c161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:05.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.793+0000 7fdb53e00640 1 --2- 172.21.15.86:0/948411543 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb5c1616e0 0x7fdb5c161af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:05.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.793+0000 7fdb53e00640 1 -- 172.21.15.86:0/948411543 >> v1:172.21.15.155:6789/0 conn(0x7fdb5c0d2c80 legacy=0x7fdb5c070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:05.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.793+0000 7fdb53e00640 1 -- 172.21.15.86:0/948411543 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5c0d1be0 con 0x7fdb5c071530 2024-04-25T14:04:05.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.793+0000 7fdb61a00640 1 --2- 172.21.15.86:0/948411543 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb5c1616e0 0x7fdb5c161af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:05.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.793+0000 7fdb53e00640 1 --2- 172.21.15.86:0/948411543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb5c071530 0x7fdb5c15dc90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdb440097a0 tx=0x7fdb4401ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7428e6128191423 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:05.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.797+0000 7fdb5be00640 1 -- 172.21.15.86:0/948411543 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb440b5070 con 0x7fdb5c071530 2024-04-25T14:04:05.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.797+0000 7fdb5be00640 1 -- 172.21.15.86:0/948411543 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb440a8970 con 0x7fdb5c071530 2024-04-25T14:04:05.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.797+0000 7fdb5be00640 1 -- 172.21.15.86:0/948411543 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb440a8d40 con 0x7fdb5c071530 2024-04-25T14:04:05.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.797+0000 7fdb63a00640 1 -- 172.21.15.86:0/948411543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb5c071530 msgr2=0x7fdb5c15dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:05.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.797+0000 7fdb63a00640 1 --2- 172.21.15.86:0/948411543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb5c071530 0x7fdb5c15dc90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdb440097a0 tx=0x7fdb4401ec70 comp rx=0 tx=0).stop 2024-04-25T14:04:05.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.797+0000 7fdb63a00640 1 -- 172.21.15.86:0/948411543 shutdown_connections 2024-04-25T14:04:05.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.797+0000 7fdb63a00640 1 --2- 172.21.15.86:0/948411543 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb5c1616e0 0x7fdb5c161af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:05.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.797+0000 7fdb63a00640 1 --2- 172.21.15.86:0/948411543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb5c071530 0x7fdb5c15dc90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:05.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.797+0000 7fdb63a00640 1 -- 172.21.15.86:0/948411543 >> 172.21.15.86:0/948411543 conn(0x7fdb5c0cffc0 msgr2=0x7fdb5c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:05.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.797+0000 7fdb63a00640 1 -- 172.21.15.86:0/948411543 shutdown_connections 2024-04-25T14:04:05.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.797+0000 7fdb63a00640 1 -- 172.21.15.86:0/948411543 wait complete. 2024-04-25T14:04:05.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.797+0000 7fdb63a00640 1 Processor -- start 2024-04-25T14:04:05.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.797+0000 7fdb63a00640 1 -- start start 2024-04-25T14:04:05.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.801+0000 7fdb63a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb5c0d2c80 0x7fdb5c14fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:05.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.801+0000 7fdb63a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb5c1616e0 0x7fdb5c150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:05.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.801+0000 7fdb63a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb5c0b5820 0x7fdb5c0b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:05.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.801+0000 7fdb63a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c0d2480 con 0x7fdb5c0d2c80 2024-04-25T14:04:05.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.801+0000 7fdb63a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c0d21a0 con 0x7fdb5c0b5820 2024-04-25T14:04:05.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.801+0000 7fdb63a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdb5c0d29a0 con 0x7fdb5c1616e0 2024-04-25T14:04:05.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.801+0000 7fdb61000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb5c0d2c80 0x7fdb5c14fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:05.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.801+0000 7fdb61000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb5c0d2c80 0x7fdb5c14fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36380/0 (socket says 172.21.15.86:36380) 2024-04-25T14:04:05.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.801+0000 7fdb61000640 1 -- 172.21.15.86:0/2493562462 learned_addr learned my addr 172.21.15.86:0/2493562462 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:05.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.801+0000 7fdb53e00640 1 --2- 172.21.15.86:0/2493562462 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb5c1616e0 0x7fdb5c150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:05.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.801+0000 7fdb61a00640 1 --2- 172.21.15.86:0/2493562462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb5c0b5820 0x7fdb5c0b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:05.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.801+0000 7fdb61000640 1 -- 172.21.15.86:0/2493562462 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb5c1616e0 msgr2=0x7fdb5c150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:05.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.801+0000 7fdb61000640 1 --2- 172.21.15.86:0/2493562462 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb5c1616e0 0x7fdb5c150580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:05.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.801+0000 7fdb61000640 1 -- 172.21.15.86:0/2493562462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb5c0b5820 msgr2=0x7fdb5c0b1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:05.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.801+0000 7fdb61000640 1 --2- 172.21.15.86:0/2493562462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb5c0b5820 0x7fdb5c0b1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:05.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.801+0000 7fdb61000640 1 -- 172.21.15.86:0/2493562462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5c0b4780 con 0x7fdb5c0d2c80 2024-04-25T14:04:05.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.801+0000 7fdb61a00640 1 --2- 172.21.15.86:0/2493562462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb5c0b5820 0x7fdb5c0b1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:05.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.801+0000 7fdb61000640 1 --2- 172.21.15.86:0/2493562462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb5c0d2c80 0x7fdb5c14fa00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00c3a0 tx=0x7fdb4c00c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:05.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.801+0000 7fdb59600640 1 -- 172.21.15.86:0/2493562462 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c017070 con 0x7fdb5c0d2c80 2024-04-25T14:04:05.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.801+0000 7fdb59600640 1 -- 172.21.15.86:0/2493562462 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb4c005cf0 con 0x7fdb5c0d2c80 2024-04-25T14:04:05.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.801+0000 7fdb59600640 1 -- 172.21.15.86:0/2493562462 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c0072f0 con 0x7fdb5c0d2c80 2024-04-25T14:04:05.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.801+0000 7fdb63a00640 1 -- 172.21.15.86:0/2493562462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb5c0b4a60 con 0x7fdb5c0d2c80 2024-04-25T14:04:05.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.801+0000 7fdb63a00640 1 -- 172.21.15.86:0/2493562462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb5c0b5020 con 0x7fdb5c0d2c80 2024-04-25T14:04:05.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.801+0000 7fdb63a00640 1 -- 172.21.15.86:0/2493562462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb5c0720e0 con 0x7fdb5c0d2c80 2024-04-25T14:04:05.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.805+0000 7fdb59600640 1 -- 172.21.15.86:0/2493562462 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdb4c01a070 con 0x7fdb5c0d2c80 2024-04-25T14:04:05.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.805+0000 7fdb59600640 1 --2- 172.21.15.86:0/2493562462 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdb3806f430 0x7fdb38071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:05.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.805+0000 7fdb59600640 1 -- 172.21.15.86:0/2493562462 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdb4c08e8a0 con 0x7fdb5c0d2c80 2024-04-25T14:04:05.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.805+0000 7fdb53e00640 1 --2- 172.21.15.86:0/2493562462 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdb3806f430 0x7fdb38071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:05.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.805+0000 7fdb53e00640 1 --2- 172.21.15.86:0/2493562462 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdb3806f430 0x7fdb38071940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdb440b2000 tx=0x7fdb44008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:05.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.809+0000 7fdb59600640 1 -- 172.21.15.86:0/2493562462 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdb4c05adf0 con 0x7fdb5c0d2c80 2024-04-25T14:04:05.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.969+0000 7fdb63a00640 1 -- 172.21.15.86:0/2493562462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdb5c076bf0 con 0x7fdb5c0d2c80 2024-04-25T14:04:05.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.969+0000 7fdb59600640 1 -- 172.21.15.86:0/2493562462 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fdb4c05f0f0 con 0x7fdb5c0d2c80 2024-04-25T14:04:05.970 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:04:05.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.969+0000 7fdb63a00640 1 -- 172.21.15.86:0/2493562462 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdb3806f430 msgr2=0x7fdb38071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:05.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.969+0000 7fdb63a00640 1 --2- 172.21.15.86:0/2493562462 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdb3806f430 0x7fdb38071940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdb440b2000 tx=0x7fdb44008040 comp rx=0 tx=0).stop 2024-04-25T14:04:05.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.969+0000 7fdb63a00640 1 -- 172.21.15.86:0/2493562462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb5c0d2c80 msgr2=0x7fdb5c14fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:05.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.969+0000 7fdb63a00640 1 --2- 172.21.15.86:0/2493562462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb5c0d2c80 0x7fdb5c14fa00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00c3a0 tx=0x7fdb4c00c7a0 comp rx=0 tx=0).stop 2024-04-25T14:04:05.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.969+0000 7fdb63a00640 1 -- 172.21.15.86:0/2493562462 shutdown_connections 2024-04-25T14:04:05.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.969+0000 7fdb63a00640 1 --2- 172.21.15.86:0/2493562462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb5c0b5820 0x7fdb5c0b1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:05.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.969+0000 7fdb63a00640 1 --2- 172.21.15.86:0/2493562462 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdb3806f430 0x7fdb38071940 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:05.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.969+0000 7fdb63a00640 1 --2- 172.21.15.86:0/2493562462 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb5c1616e0 0x7fdb5c150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:05.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.969+0000 7fdb63a00640 1 --2- 172.21.15.86:0/2493562462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb5c0d2c80 0x7fdb5c14fa00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:05.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.969+0000 7fdb63a00640 1 -- 172.21.15.86:0/2493562462 >> 172.21.15.86:0/2493562462 conn(0x7fdb5c0cffc0 msgr2=0x7fdb5c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:05.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.969+0000 7fdb63a00640 1 -- 172.21.15.86:0/2493562462 shutdown_connections 2024-04-25T14:04:05.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:05.969+0000 7fdb63a00640 1 -- 172.21.15.86:0/2493562462 wait complete. 2024-04-25T14:04:05.992 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T14:04:06.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.141+0000 7fdb23800640 1 Processor -- start 2024-04-25T14:04:06.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.141+0000 7fdb23800640 1 -- start start 2024-04-25T14:04:06.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.141+0000 7fdb23800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb1c1642a0 0x7fdb1c0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.141+0000 7fdb23800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb1c0d2030 con 0x7fdb1c0b0860 2024-04-25T14:04:06.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.141+0000 7fdb23800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb1c0cde20 con 0x7fdb1c0d1c20 2024-04-25T14:04:06.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.141+0000 7fdb23800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdb1c0cea30 con 0x7fdb1c1642a0 2024-04-25T14:04:06.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.141+0000 7fdb21800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fdb1c0b0860 0x7fdb1c0d0640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52062/0 (socket says 172.21.15.86:52062) 2024-04-25T14:04:06.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.141+0000 7fdb21800640 1 -- 172.21.15.86:0/3670268263 learned_addr learned my addr 172.21.15.86:0/3670268263 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:06.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.141+0000 7fdb20e00640 1 --2- 172.21.15.86:0/3670268263 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb1c1642a0 0x7fdb1c0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.141+0000 7fdb1b400640 1 -- 172.21.15.86:0/3670268263 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3449814613 0 0) 0x7fdb140014a0 con 0x7fdb1c0b0860 2024-04-25T14:04:06.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.141+0000 7fdb1b400640 1 -- 172.21.15.86:0/3670268263 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb04003530 con 0x7fdb1c0b0860 2024-04-25T14:04:06.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.141+0000 7fdb1b400640 1 -- 172.21.15.86:0/3670268263 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 325615197 0 0) 0x7fdb1c0cde20 con 0x7fdb1c0d1c20 2024-04-25T14:04:06.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.141+0000 7fdb1b400640 1 -- 172.21.15.86:0/3670268263 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb140014a0 con 0x7fdb1c0d1c20 2024-04-25T14:04:06.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.141+0000 7fdb1b400640 1 -- 172.21.15.86:0/3670268263 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 665377937 0 0) 0x7fdb04003530 con 0x7fdb1c0b0860 2024-04-25T14:04:06.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.141+0000 7fdb1b400640 1 -- 172.21.15.86:0/3670268263 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb1c1642a0 msgr2=0x7fdb1c0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:06.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.141+0000 7fdb1b400640 1 --2- 172.21.15.86:0/3670268263 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb1c1642a0 0x7fdb1c0d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.141+0000 7fdb1b400640 1 -- 172.21.15.86:0/3670268263 >> v1:172.21.15.155:6789/0 conn(0x7fdb1c0d1c20 legacy=0x7fdb1c0b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.141+0000 7fdb1b400640 1 -- 172.21.15.86:0/3670268263 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1c0ce100 con 0x7fdb1c0b0860 2024-04-25T14:04:06.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.141+0000 7fdb1b400640 1 -- 172.21.15.86:0/3670268263 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fdb14001bd0 con 0x7fdb1c0b0860 2024-04-25T14:04:06.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.141+0000 7fdb1b400640 1 -- 172.21.15.86:0/3670268263 >> v1:172.21.15.86:6789/0 conn(0x7fdb1c0b0860 legacy=0x7fdb1c0d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.141+0000 7fdb1b400640 1 --2- 172.21.15.86:0/3670268263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb04003bf0 0x7fdb04004040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.141+0000 7fdb1b400640 1 --2- 172.21.15.86:0/3670268263 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb1c1642a0 0x7fdb04004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.141+0000 7fdb1b400640 1 --2- 172.21.15.86:0/3670268263 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb04004d50 0x7fdb040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.141+0000 7fdb1b400640 1 -- 172.21.15.86:0/3670268263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c0cea30 con 0x7fdb04003bf0 2024-04-25T14:04:06.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.141+0000 7fdb1b400640 1 -- 172.21.15.86:0/3670268263 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0400bc00 con 0x7fdb04004d50 2024-04-25T14:04:06.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.141+0000 7fdb1b400640 1 -- 172.21.15.86:0/3670268263 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdb0400be00 con 0x7fdb1c1642a0 2024-04-25T14:04:06.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.141+0000 7fdb20e00640 1 --2- 172.21.15.86:0/3670268263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb04003bf0 0x7fdb04004040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.145+0000 7fdb1be00640 1 --2- 172.21.15.86:0/3670268263 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb1c1642a0 0x7fdb04004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.145+0000 7fdb20e00640 1 -- 172.21.15.86:0/3670268263 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb1c1642a0 msgr2=0x7fdb04004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:06.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.145+0000 7fdb20e00640 1 --2- 172.21.15.86:0/3670268263 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb1c1642a0 0x7fdb04004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.145+0000 7fdb20e00640 1 -- 172.21.15.86:0/3670268263 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb04004d50 msgr2=0x7fdb040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:06.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.145+0000 7fdb20e00640 1 --2- 172.21.15.86:0/3670268263 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb04004d50 0x7fdb040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.145+0000 7fdb20e00640 1 -- 172.21.15.86:0/3670268263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb04003530 con 0x7fdb04003bf0 2024-04-25T14:04:06.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.145+0000 7fdb1be00640 1 --2- 172.21.15.86:0/3670268263 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb1c1642a0 0x7fdb04004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:06.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.145+0000 7fdb20e00640 1 --2- 172.21.15.86:0/3670268263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb04003bf0 0x7fdb04004040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdb10009ab0 tx=0x7fdb1001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b042d840a7fe4622 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:06.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.145+0000 7fdb1b400640 1 -- 172.21.15.86:0/3670268263 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb100096b0 con 0x7fdb04003bf0 2024-04-25T14:04:06.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.145+0000 7fdb1b400640 1 -- 172.21.15.86:0/3670268263 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb100a6450 con 0x7fdb04003bf0 2024-04-25T14:04:06.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.145+0000 7fdb1b400640 1 -- 172.21.15.86:0/3670268263 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb100a6800 con 0x7fdb04003bf0 2024-04-25T14:04:06.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.145+0000 7fdb23800640 1 -- 172.21.15.86:0/3670268263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb04003bf0 msgr2=0x7fdb04004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.145+0000 7fdb23800640 1 --2- 172.21.15.86:0/3670268263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb04003bf0 0x7fdb04004040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdb10009ab0 tx=0x7fdb1001ca70 comp rx=0 tx=0).stop 2024-04-25T14:04:06.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.145+0000 7fdb21800640 1 -- 172.21.15.86:0/3670268263 reap_dead start 2024-04-25T14:04:06.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.145+0000 7fdb23800640 1 -- 172.21.15.86:0/3670268263 shutdown_connections 2024-04-25T14:04:06.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.145+0000 7fdb23800640 1 -- 172.21.15.86:0/3670268263 >> 172.21.15.86:0/3670268263 conn(0x7fdb1c0b4650 msgr2=0x7fdb1c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:06.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.149+0000 7fdb23800640 1 -- 172.21.15.86:0/3670268263 shutdown_connections 2024-04-25T14:04:06.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.149+0000 7fdb23800640 1 -- 172.21.15.86:0/3670268263 wait complete. 2024-04-25T14:04:06.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.149+0000 7fdb23800640 1 Processor -- start 2024-04-25T14:04:06.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.149+0000 7fdb23800640 1 -- start start 2024-04-25T14:04:06.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.149+0000 7fdb23800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb04003bf0 0x7fdb1c074470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.149+0000 7fdb23800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb1c0b0c70 0x7fdb1c15bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.149+0000 7fdb23800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb1c09de50 0x7fdb1c0af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.149+0000 7fdb23800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c1648c0 con 0x7fdb1c0b0c70 2024-04-25T14:04:06.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.149+0000 7fdb23800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c0d1160 con 0x7fdb1c09de50 2024-04-25T14:04:06.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.149+0000 7fdb23800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdb1c160da0 con 0x7fdb04003bf0 2024-04-25T14:04:06.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.153+0000 7fdb1be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb1c0b0c70 0x7fdb1c15bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.153+0000 7fdb1be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb1c0b0c70 0x7fdb1c15bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36408/0 (socket says 172.21.15.86:36408) 2024-04-25T14:04:06.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.153+0000 7fdb1be00640 1 -- 172.21.15.86:0/2398916834 learned_addr learned my addr 172.21.15.86:0/2398916834 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:06.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.153+0000 7fdb20e00640 1 --2- 172.21.15.86:0/2398916834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb04003bf0 0x7fdb1c074470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.153+0000 7fdb21800640 1 --2- 172.21.15.86:0/2398916834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb1c09de50 0x7fdb1c0af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.153+0000 7fdb1be00640 1 -- 172.21.15.86:0/2398916834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb04003bf0 msgr2=0x7fdb1c074470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.153+0000 7fdb1be00640 1 --2- 172.21.15.86:0/2398916834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb04003bf0 0x7fdb1c074470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.153+0000 7fdb1be00640 1 -- 172.21.15.86:0/2398916834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb1c09de50 msgr2=0x7fdb1c0af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.153+0000 7fdb1be00640 1 --2- 172.21.15.86:0/2398916834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb1c09de50 0x7fdb1c0af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.153+0000 7fdb1be00640 1 -- 172.21.15.86:0/2398916834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1c09b9b0 con 0x7fdb1c0b0c70 2024-04-25T14:04:06.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.153+0000 7fdb20e00640 1 --2- 172.21.15.86:0/2398916834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb04003bf0 0x7fdb1c074470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:06.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.153+0000 7fdb21800640 1 --2- 172.21.15.86:0/2398916834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb1c09de50 0x7fdb1c0af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:06.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.153+0000 7fdb1be00640 1 --2- 172.21.15.86:0/2398916834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb1c0b0c70 0x7fdb1c15bcc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800d7a0 tx=0x7fdb0800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:06.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.153+0000 7fdb18c00640 1 -- 172.21.15.86:0/2398916834 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb080069e0 con 0x7fdb1c0b0c70 2024-04-25T14:04:06.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.153+0000 7fdb18c00640 1 -- 172.21.15.86:0/2398916834 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb08006b80 con 0x7fdb1c0b0c70 2024-04-25T14:04:06.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.153+0000 7fdb18c00640 1 -- 172.21.15.86:0/2398916834 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb08007030 con 0x7fdb1c0b0c70 2024-04-25T14:04:06.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.153+0000 7fdb23800640 1 -- 172.21.15.86:0/2398916834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb1c09d370 con 0x7fdb1c0b0c70 2024-04-25T14:04:06.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.153+0000 7fdb23800640 1 -- 172.21.15.86:0/2398916834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb1c09d930 con 0x7fdb1c0b0c70 2024-04-25T14:04:06.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.153+0000 7fdb23800640 1 -- 172.21.15.86:0/2398916834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb1c069d00 con 0x7fdb1c0b0c70 2024-04-25T14:04:06.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.153+0000 7fdb18c00640 1 -- 172.21.15.86:0/2398916834 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdb080071d0 con 0x7fdb1c0b0c70 2024-04-25T14:04:06.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.157+0000 7fdb18c00640 1 --2- 172.21.15.86:0/2398916834 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdafc06f400 0x7fdafc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.157+0000 7fdb18c00640 1 -- 172.21.15.86:0/2398916834 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdb0808f7a0 con 0x7fdb1c0b0c70 2024-04-25T14:04:06.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.157+0000 7fdb20e00640 1 --2- 172.21.15.86:0/2398916834 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdafc06f400 0x7fdafc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.157+0000 7fdb20e00640 1 --2- 172.21.15.86:0/2398916834 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdafc06f400 0x7fdafc071910 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdb100ad000 tx=0x7fdb100046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:06.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.161+0000 7fdb18c00640 1 -- 172.21.15.86:0/2398916834 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdb08016020 con 0x7fdb1c0b0c70 2024-04-25T14:04:06.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.325+0000 7fdb23800640 1 -- 172.21.15.86:0/2398916834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fdb1c09b740 con 0x7fdb1c0b0c70 2024-04-25T14:04:06.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.325+0000 7fdb18c00640 1 -- 172.21.15.86:0/2398916834 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7fdb0805c2b0 con 0x7fdb1c0b0c70 2024-04-25T14:04:06.325 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:04:06.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.325+0000 7fdb23800640 1 -- 172.21.15.86:0/2398916834 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdafc06f400 msgr2=0x7fdafc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.325+0000 7fdb23800640 1 --2- 172.21.15.86:0/2398916834 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdafc06f400 0x7fdafc071910 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdb100ad000 tx=0x7fdb100046d0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.325+0000 7fdb23800640 1 -- 172.21.15.86:0/2398916834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb1c0b0c70 msgr2=0x7fdb1c15bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.325+0000 7fdb23800640 1 --2- 172.21.15.86:0/2398916834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb1c0b0c70 0x7fdb1c15bcc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800d7a0 tx=0x7fdb0800dc60 comp rx=0 tx=0).stop 2024-04-25T14:04:06.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.325+0000 7fdb23800640 1 -- 172.21.15.86:0/2398916834 shutdown_connections 2024-04-25T14:04:06.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.325+0000 7fdb23800640 1 --2- 172.21.15.86:0/2398916834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb1c09de50 0x7fdb1c0af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.325+0000 7fdb23800640 1 --2- 172.21.15.86:0/2398916834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb1c0b0c70 0x7fdb1c15bcc0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.325+0000 7fdb23800640 1 --2- 172.21.15.86:0/2398916834 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdafc06f400 0x7fdafc071910 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.325+0000 7fdb23800640 1 --2- 172.21.15.86:0/2398916834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb04003bf0 0x7fdb1c074470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.325+0000 7fdb23800640 1 -- 172.21.15.86:0/2398916834 >> 172.21.15.86:0/2398916834 conn(0x7fdb1c0b4650 msgr2=0x7fdb1c15d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:06.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.325+0000 7fdb23800640 1 -- 172.21.15.86:0/2398916834 shutdown_connections 2024-04-25T14:04:06.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.325+0000 7fdb23800640 1 -- 172.21.15.86:0/2398916834 wait complete. 2024-04-25T14:04:06.347 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T14:04:06.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.493+0000 7fd731a00640 1 Processor -- start 2024-04-25T14:04:06.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.493+0000 7fd731a00640 1 -- start start 2024-04-25T14:04:06.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.497+0000 7fd731a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd72c15c2a0 0x7fd72c0ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.497+0000 7fd731a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd72c06fd50 0x7fd72c070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.497+0000 7fd731a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd72c0b0a60 con 0x7fd72c06c670 2024-04-25T14:04:06.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.497+0000 7fd731a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd72c06ef90 con 0x7fd72c15c2a0 2024-04-25T14:04:06.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.497+0000 7fd731a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd72c06f270 con 0x7fd72c06fd50 2024-04-25T14:04:06.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.497+0000 7fd72b400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd72c06c670 0x7fd72c06ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52078/0 (socket says 172.21.15.86:52078) 2024-04-25T14:04:06.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.497+0000 7fd72aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd72c15c2a0 0x7fd72c0ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.497+0000 7fd72b400640 1 -- 172.21.15.86:0/3966909695 learned_addr learned my addr 172.21.15.86:0/3966909695 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:06.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.497+0000 7fd722000640 1 --2- 172.21.15.86:0/3966909695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd72c06fd50 0x7fd72c070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.497+0000 7fd72aa00640 1 -- 172.21.15.86:0/3966909695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd72c06fd50 msgr2=0x7fd72c070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:06.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.497+0000 7fd72aa00640 1 --2- 172.21.15.86:0/3966909695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd72c06fd50 0x7fd72c070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.497+0000 7fd72aa00640 1 -- 172.21.15.86:0/3966909695 >> v1:172.21.15.86:6789/0 conn(0x7fd72c06c670 legacy=0x7fd72c06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.497+0000 7fd72a000640 1 -- 172.21.15.86:0/3966909695 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4273224441 0 0) 0x7fd72c0b0a60 con 0x7fd72c06c670 2024-04-25T14:04:06.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.497+0000 7fd72aa00640 1 -- 172.21.15.86:0/3966909695 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72c06f550 con 0x7fd72c15c2a0 2024-04-25T14:04:06.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.501+0000 7fd72aa00640 1 --2- 172.21.15.86:0/3966909695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd72c15c2a0 0x7fd72c0ae130 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009d90 tx=0x7fd71c01c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1cbc20b8935b392 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:06.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.501+0000 7fd72a000640 1 -- 172.21.15.86:0/3966909695 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c0b2070 con 0x7fd72c15c2a0 2024-04-25T14:04:06.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.501+0000 7fd72a000640 1 -- 172.21.15.86:0/3966909695 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd71c0a5930 con 0x7fd72c15c2a0 2024-04-25T14:04:06.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.501+0000 7fd72a000640 1 -- 172.21.15.86:0/3966909695 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c0a5d00 con 0x7fd72c15c2a0 2024-04-25T14:04:06.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.501+0000 7fd731a00640 1 -- 172.21.15.86:0/3966909695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd72c15c2a0 msgr2=0x7fd72c0ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.501+0000 7fd731a00640 1 --2- 172.21.15.86:0/3966909695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd72c15c2a0 0x7fd72c0ae130 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009d90 tx=0x7fd71c01c8b0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.501+0000 7fd731a00640 1 -- 172.21.15.86:0/3966909695 shutdown_connections 2024-04-25T14:04:06.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.501+0000 7fd731a00640 1 --2- 172.21.15.86:0/3966909695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd72c06fd50 0x7fd72c070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.501+0000 7fd731a00640 1 --2- 172.21.15.86:0/3966909695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd72c15c2a0 0x7fd72c0ae130 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.501+0000 7fd731a00640 1 -- 172.21.15.86:0/3966909695 >> 172.21.15.86:0/3966909695 conn(0x7fd72c0b4650 msgr2=0x7fd72c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:06.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.501+0000 7fd731a00640 1 -- 172.21.15.86:0/3966909695 shutdown_connections 2024-04-25T14:04:06.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.501+0000 7fd731a00640 1 -- 172.21.15.86:0/3966909695 wait complete. 2024-04-25T14:04:06.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.501+0000 7fd731a00640 1 Processor -- start 2024-04-25T14:04:06.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.505+0000 7fd731a00640 1 -- start start 2024-04-25T14:04:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.505+0000 7fd731a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd72c06c670 0x7fd72c1f9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.505+0000 7fd731a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd72c06fd50 0x7fd72c155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.505+0000 7fd731a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd72c15c2a0 0x7fd72c0b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.505+0000 7fd731a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c15c830 con 0x7fd72c15c2a0 2024-04-25T14:04:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.505+0000 7fd731a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c15c6b0 con 0x7fd72c06fd50 2024-04-25T14:04:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.505+0000 7fd731a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd72c0ae670 con 0x7fd72c06c670 2024-04-25T14:04:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.505+0000 7fd72b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd72c15c2a0 0x7fd72c0b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.505+0000 7fd72b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd72c15c2a0 0x7fd72c0b35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36426/0 (socket says 172.21.15.86:36426) 2024-04-25T14:04:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.505+0000 7fd72b400640 1 -- 172.21.15.86:0/4074672942 learned_addr learned my addr 172.21.15.86:0/4074672942 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.505+0000 7fd72aa00640 1 --2- 172.21.15.86:0/4074672942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd72c06c670 0x7fd72c1f9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.505+0000 7fd722000640 1 --2- 172.21.15.86:0/4074672942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd72c06fd50 0x7fd72c155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.505+0000 7fd72b400640 1 -- 172.21.15.86:0/4074672942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd72c06c670 msgr2=0x7fd72c1f9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.505+0000 7fd72b400640 1 --2- 172.21.15.86:0/4074672942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd72c06c670 0x7fd72c1f9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.505+0000 7fd72b400640 1 -- 172.21.15.86:0/4074672942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd72c06fd50 msgr2=0x7fd72c155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.505+0000 7fd72b400640 1 --2- 172.21.15.86:0/4074672942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd72c06fd50 0x7fd72c155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.505+0000 7fd72b400640 1 -- 172.21.15.86:0/4074672942 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72c166ed0 con 0x7fd72c15c2a0 2024-04-25T14:04:06.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.505+0000 7fd72aa00640 1 --2- 172.21.15.86:0/4074672942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd72c06c670 0x7fd72c1f9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:06.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.505+0000 7fd722000640 1 --2- 172.21.15.86:0/4074672942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd72c06fd50 0x7fd72c155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:06.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.505+0000 7fd72b400640 1 --2- 172.21.15.86:0/4074672942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd72c15c2a0 0x7fd72c0b35b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd72400c410 tx=0x7fd72400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:06.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.505+0000 7fd723400640 1 -- 172.21.15.86:0/4074672942 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7240074e0 con 0x7fd72c15c2a0 2024-04-25T14:04:06.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.505+0000 7fd723400640 1 -- 172.21.15.86:0/4074672942 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7240047f0 con 0x7fd72c15c2a0 2024-04-25T14:04:06.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.505+0000 7fd723400640 1 -- 172.21.15.86:0/4074672942 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724007680 con 0x7fd72c15c2a0 2024-04-25T14:04:06.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.505+0000 7fd731a00640 1 -- 172.21.15.86:0/4074672942 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd72c157bc0 con 0x7fd72c15c2a0 2024-04-25T14:04:06.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.505+0000 7fd731a00640 1 -- 172.21.15.86:0/4074672942 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd72c166bf0 con 0x7fd72c15c2a0 2024-04-25T14:04:06.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.505+0000 7fd731a00640 1 -- 172.21.15.86:0/4074672942 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd72c069d00 con 0x7fd72c15c2a0 2024-04-25T14:04:06.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.509+0000 7fd723400640 1 -- 172.21.15.86:0/4074672942 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd724011070 con 0x7fd72c15c2a0 2024-04-25T14:04:06.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.509+0000 7fd723400640 1 --2- 172.21.15.86:0/4074672942 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd70806f430 0x7fd708071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.509+0000 7fd723400640 1 -- 172.21.15.86:0/4074672942 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd72408f140 con 0x7fd72c15c2a0 2024-04-25T14:04:06.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.509+0000 7fd72aa00640 1 --2- 172.21.15.86:0/4074672942 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd70806f430 0x7fd708071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.509+0000 7fd72aa00640 1 --2- 172.21.15.86:0/4074672942 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd70806f430 0x7fd708071940 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd71c0af000 tx=0x7fd71c0a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:06.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.513+0000 7fd723400640 1 -- 172.21.15.86:0/4074672942 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd72405bce0 con 0x7fd72c15c2a0 2024-04-25T14:04:06.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.681+0000 7fd731a00640 1 -- 172.21.15.86:0/4074672942 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd72c076bf0 con 0x7fd72c15c2a0 2024-04-25T14:04:06.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.681+0000 7fd723400640 1 -- 172.21.15.86:0/4074672942 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7fd72405b5e0 con 0x7fd72c15c2a0 2024-04-25T14:04:06.684 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-25T14:04:06.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.685+0000 7fd731a00640 1 -- 172.21.15.86:0/4074672942 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd70806f430 msgr2=0x7fd708071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.685+0000 7fd731a00640 1 --2- 172.21.15.86:0/4074672942 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd70806f430 0x7fd708071940 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd71c0af000 tx=0x7fd71c0a7070 comp rx=0 tx=0).stop 2024-04-25T14:04:06.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.685+0000 7fd731a00640 1 -- 172.21.15.86:0/4074672942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd72c15c2a0 msgr2=0x7fd72c0b35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.685+0000 7fd731a00640 1 --2- 172.21.15.86:0/4074672942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd72c15c2a0 0x7fd72c0b35b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd72400c410 tx=0x7fd72400c8d0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.685+0000 7fd731a00640 1 -- 172.21.15.86:0/4074672942 shutdown_connections 2024-04-25T14:04:06.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.685+0000 7fd731a00640 1 --2- 172.21.15.86:0/4074672942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd72c15c2a0 0x7fd72c0b35b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.685+0000 7fd731a00640 1 --2- 172.21.15.86:0/4074672942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd72c06fd50 0x7fd72c155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.685+0000 7fd731a00640 1 --2- 172.21.15.86:0/4074672942 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd70806f430 0x7fd708071940 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.685+0000 7fd731a00640 1 --2- 172.21.15.86:0/4074672942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd72c06c670 0x7fd72c1f9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.685+0000 7fd731a00640 1 -- 172.21.15.86:0/4074672942 >> 172.21.15.86:0/4074672942 conn(0x7fd72c0b4650 msgr2=0x7fd72c0729f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:06.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.685+0000 7fd731a00640 1 -- 172.21.15.86:0/4074672942 shutdown_connections 2024-04-25T14:04:06.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:06.685+0000 7fd731a00640 1 -- 172.21.15.86:0/4074672942 wait complete. 2024-04-25T14:04:06.706 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2024-04-25T14:04:06.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:06.706 DEBUG:teuthology.orchestra.run.smithi086:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-25T14:04:06.743 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi086.stderr:ceph-fuse[33915]: fuse finished with error 0 and tester_r 0 2024-04-25T14:04:06.746 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:04:06.772 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T14:04:06.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:06.772 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-25T14:04:06.824 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:04:07.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.013+0000 7fbb1fe00640 1 Processor -- start 2024-04-25T14:04:07.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.013+0000 7fbb1fe00640 1 -- start start 2024-04-25T14:04:07.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.013+0000 7fbb1fe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbb201635a0 0x7fbb2006eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.013+0000 7fbb1fe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb200d08c0 0x7fbb200722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.013+0000 7fbb1fe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb20163d80 con 0x7fbb2006f060 2024-04-25T14:04:07.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.013+0000 7fbb1fe00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fbb2006f470 con 0x7fbb201635a0 2024-04-25T14:04:07.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.013+0000 7fbb1fe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbb20164060 con 0x7fbb200d08c0 2024-04-25T14:04:07.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.013+0000 7fbb16000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fbb2006f060 0x7fbb200d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52094/0 (socket says 172.21.15.86:52094) 2024-04-25T14:04:07.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.013+0000 7fbb16000640 1 -- 172.21.15.86:0/2950062916 learned_addr learned my addr 172.21.15.86:0/2950062916 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:07.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.013+0000 7fbb1f400640 1 --2- 172.21.15.86:0/2950062916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb200d08c0 0x7fbb200722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.013+0000 7fbb1ea00640 1 --2- 172.21.15.86:0/2950062916 >> v2:172.21.15.155:3300/0 conn(0x7fbb201635a0 0x7fbb2006eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.013+0000 7fbb1e000640 1 -- 172.21.15.86:0/2950062916 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2565131078 0 0) 0x7fbb20163d80 con 0x7fbb2006f060 2024-04-25T14:04:07.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.013+0000 7fbb1e000640 1 -- 172.21.15.86:0/2950062916 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb04003530 con 0x7fbb2006f060 2024-04-25T14:04:07.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.013+0000 7fbb1e000640 1 -- 172.21.15.86:0/2950062916 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3653094749 0 0) 0x7fbb04003530 con 0x7fbb2006f060 2024-04-25T14:04:07.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.013+0000 7fbb1e000640 1 -- 172.21.15.86:0/2950062916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb200d08c0 msgr2=0x7fbb200722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.013+0000 7fbb1e000640 1 --2- 172.21.15.86:0/2950062916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb200d08c0 0x7fbb200722a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.013+0000 7fbb1e000640 1 -- 172.21.15.86:0/2950062916 >> v2:172.21.15.155:3300/0 conn(0x7fbb201635a0 msgr2=0x7fbb2006eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.013+0000 7fbb1e000640 1 --2- 172.21.15.86:0/2950062916 >> v2:172.21.15.155:3300/0 conn(0x7fbb201635a0 0x7fbb2006eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.013+0000 7fbb1e000640 1 -- 172.21.15.86:0/2950062916 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb20164340 con 0x7fbb2006f060 2024-04-25T14:04:07.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.013+0000 7fbb1e000640 1 -- 172.21.15.86:0/2950062916 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fbb08001680 con 0x7fbb2006f060 2024-04-25T14:04:07.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.013+0000 7fbb1ea00640 1 --2- 172.21.15.86:0/2950062916 >> v2:172.21.15.155:3300/0 conn(0x7fbb201635a0 0x7fbb2006eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:07.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.013+0000 7fbb1e000640 1 -- 172.21.15.86:0/2950062916 >> v1:172.21.15.86:6789/0 conn(0x7fbb2006f060 legacy=0x7fbb200d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.013+0000 7fbb1e000640 1 --2- 172.21.15.86:0/2950062916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb200d08c0 0x7fbb040039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.013+0000 7fbb1e000640 1 --2- 172.21.15.86:0/2950062916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb04003f40 0x7fbb04004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.013+0000 7fbb1e000640 1 --2- 172.21.15.86:0/2950062916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb04004d50 0x7fbb04005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.013+0000 7fbb1e000640 1 -- 172.21.15.86:0/2950062916 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb20164060 con 0x7fbb04004d50 2024-04-25T14:04:07.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.013+0000 7fbb1e000640 1 -- 172.21.15.86:0/2950062916 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0400b9d0 con 0x7fbb04003f40 2024-04-25T14:04:07.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.013+0000 7fbb1e000640 1 -- 172.21.15.86:0/2950062916 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbb0400bba0 con 0x7fbb200d08c0 2024-04-25T14:04:07.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.017+0000 7fbb1f400640 1 --2- 172.21.15.86:0/2950062916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb04004d50 0x7fbb04005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.017+0000 7fbb1ea00640 1 --2- 172.21.15.86:0/2950062916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb200d08c0 0x7fbb040039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.017+0000 7fbb16000640 1 --2- 172.21.15.86:0/2950062916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb04003f40 0x7fbb04004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.017+0000 7fbb1f400640 1 -- 172.21.15.86:0/2950062916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb200d08c0 msgr2=0x7fbb040039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.017+0000 7fbb1f400640 1 --2- 172.21.15.86:0/2950062916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb200d08c0 0x7fbb040039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.017+0000 7fbb1f400640 1 -- 172.21.15.86:0/2950062916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb04003f40 msgr2=0x7fbb04004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.017+0000 7fbb1f400640 1 --2- 172.21.15.86:0/2950062916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb04003f40 0x7fbb04004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.017+0000 7fbb1f400640 1 -- 172.21.15.86:0/2950062916 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb04003530 con 0x7fbb04004d50 2024-04-25T14:04:07.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.017+0000 7fbb16000640 1 --2- 172.21.15.86:0/2950062916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb04003f40 0x7fbb04004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.017+0000 7fbb1f400640 1 --2- 172.21.15.86:0/2950062916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb04004d50 0x7fbb04005160 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbb200d43d0 tx=0x7fbb1801cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d23e4bfa26e64c69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.017+0000 7fbb1e000640 1 -- 172.21.15.86:0/2950062916 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb180b2070 con 0x7fbb04004d50 2024-04-25T14:04:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.017+0000 7fbb1e000640 1 -- 172.21.15.86:0/2950062916 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb180a5bf0 con 0x7fbb04004d50 2024-04-25T14:04:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.017+0000 7fbb1e000640 1 -- 172.21.15.86:0/2950062916 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18009c10 con 0x7fbb04004d50 2024-04-25T14:04:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.021+0000 7fbb1fe00640 1 -- 172.21.15.86:0/2950062916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb04004d50 msgr2=0x7fbb04005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.021+0000 7fbb1fe00640 1 --2- 172.21.15.86:0/2950062916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb04004d50 0x7fbb04005160 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbb200d43d0 tx=0x7fbb1801cbe0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.021+0000 7fbb1f400640 1 -- 172.21.15.86:0/2950062916 reap_dead start 2024-04-25T14:04:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.021+0000 7fbb1fe00640 1 -- 172.21.15.86:0/2950062916 shutdown_connections 2024-04-25T14:04:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.021+0000 7fbb1fe00640 1 -- 172.21.15.86:0/2950062916 >> 172.21.15.86:0/2950062916 conn(0x7fbb200b4f80 msgr2=0x7fbb200b5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.021+0000 7fbb1fe00640 1 -- 172.21.15.86:0/2950062916 shutdown_connections 2024-04-25T14:04:07.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.021+0000 7fbb1fe00640 1 -- 172.21.15.86:0/2950062916 wait complete. 2024-04-25T14:04:07.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.021+0000 7fbb1fe00640 1 Processor -- start 2024-04-25T14:04:07.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.021+0000 7fbb1fe00640 1 -- start start 2024-04-25T14:04:07.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.025+0000 7fbb1fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb04004d50 0x7fbb2015a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.025+0000 7fbb1fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb200cd650 0x7fbb200b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.025+0000 7fbb1fe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb200cda60 0x7fbb200b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.025+0000 7fbb1fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201639b0 con 0x7fbb200cd650 2024-04-25T14:04:07.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.025+0000 7fbb1fe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2015fd50 con 0x7fbb04004d50 2024-04-25T14:04:07.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.025+0000 7fbb1fe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbb20163b30 con 0x7fbb200cda60 2024-04-25T14:04:07.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.025+0000 7fbb16000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb200cd650 0x7fbb200b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.025+0000 7fbb16000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb200cd650 0x7fbb200b64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36452/0 (socket says 172.21.15.86:36452) 2024-04-25T14:04:07.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.025+0000 7fbb16000640 1 -- 172.21.15.86:0/1205435684 learned_addr learned my addr 172.21.15.86:0/1205435684 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:07.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.025+0000 7fbb1ea00640 1 --2- 172.21.15.86:0/1205435684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb04004d50 0x7fbb2015a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.025+0000 7fbb1f400640 1 --2- 172.21.15.86:0/1205435684 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb200cda60 0x7fbb200b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.025+0000 7fbb16000640 1 -- 172.21.15.86:0/1205435684 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb200cda60 msgr2=0x7fbb200b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.025+0000 7fbb16000640 1 --2- 172.21.15.86:0/1205435684 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb200cda60 0x7fbb200b69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.025+0000 7fbb16000640 1 -- 172.21.15.86:0/1205435684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb04004d50 msgr2=0x7fbb2015a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.025+0000 7fbb16000640 1 --2- 172.21.15.86:0/1205435684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb04004d50 0x7fbb2015a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.025+0000 7fbb16000640 1 -- 172.21.15.86:0/1205435684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb20161c60 con 0x7fbb200cd650 2024-04-25T14:04:07.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.025+0000 7fbb1f400640 1 --2- 172.21.15.86:0/1205435684 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb200cda60 0x7fbb200b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:07.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.025+0000 7fbb16000640 1 --2- 172.21.15.86:0/1205435684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb200cd650 0x7fbb200b64a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800e7c0 tx=0x7fbb0800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:07.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.025+0000 7fbb17400640 1 -- 172.21.15.86:0/1205435684 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0800c8d0 con 0x7fbb200cd650 2024-04-25T14:04:07.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.025+0000 7fbb17400640 1 -- 172.21.15.86:0/1205435684 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb08018070 con 0x7fbb200cd650 2024-04-25T14:04:07.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.025+0000 7fbb1fe00640 1 -- 172.21.15.86:0/1205435684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb2015ad70 con 0x7fbb200cd650 2024-04-25T14:04:07.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.025+0000 7fbb17400640 1 -- 172.21.15.86:0/1205435684 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0800cc70 con 0x7fbb200cd650 2024-04-25T14:04:07.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.025+0000 7fbb1fe00640 1 -- 172.21.15.86:0/1205435684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb200ce540 con 0x7fbb200cd650 2024-04-25T14:04:07.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.025+0000 7fbb1fe00640 1 -- 172.21.15.86:0/1205435684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb200cdfd0 con 0x7fbb200cd650 2024-04-25T14:04:07.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.025+0000 7fbb17400640 1 -- 172.21.15.86:0/1205435684 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbb08013420 con 0x7fbb200cd650 2024-04-25T14:04:07.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.029+0000 7fbb17400640 1 --2- 172.21.15.86:0/1205435684 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fbafc06f430 0x7fbafc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.029+0000 7fbb17400640 1 -- 172.21.15.86:0/1205435684 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fbb0808f2e0 con 0x7fbb200cd650 2024-04-25T14:04:07.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.029+0000 7fbb1ea00640 1 --2- 172.21.15.86:0/1205435684 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fbafc06f430 0x7fbafc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.029+0000 7fbb1ea00640 1 --2- 172.21.15.86:0/1205435684 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fbafc06f430 0x7fbafc071940 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbb200cfc00 tx=0x7fbb10008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:07.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.033+0000 7fbb17400640 1 -- 172.21.15.86:0/1205435684 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbb20161c60 con 0x7fbb200cd650 2024-04-25T14:04:07.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.193+0000 7fbb1fe00640 1 -- 172.21.15.86:0/1205435684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbb20076b80 con 0x7fbb200cd650 2024-04-25T14:04:07.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.193+0000 7fbb17400640 1 -- 172.21.15.86:0/1205435684 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7fbb08062e20 con 0x7fbb200cd650 2024-04-25T14:04:07.195 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:07.195 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":24,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:03:55.899092+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:03:49.151473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:03:49.802597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T14:03:53.492008+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:04:07.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.193+0000 7fbb1fe00640 1 -- 172.21.15.86:0/1205435684 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fbafc06f430 msgr2=0x7fbafc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.193+0000 7fbb1fe00640 1 --2- 172.21.15.86:0/1205435684 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fbafc06f430 0x7fbafc071940 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbb200cfc00 tx=0x7fbb10008040 comp rx=0 tx=0).stop 2024-04-25T14:04:07.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.193+0000 7fbb1fe00640 1 -- 172.21.15.86:0/1205435684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb200cd650 msgr2=0x7fbb200b64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.193+0000 7fbb1fe00640 1 --2- 172.21.15.86:0/1205435684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb200cd650 0x7fbb200b64a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800e7c0 tx=0x7fbb0800ec80 comp rx=0 tx=0).stop 2024-04-25T14:04:07.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.197+0000 7fbb1fe00640 1 -- 172.21.15.86:0/1205435684 shutdown_connections 2024-04-25T14:04:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.197+0000 7fbb1fe00640 1 --2- 172.21.15.86:0/1205435684 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fbafc06f430 0x7fbafc071940 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.197+0000 7fbb1fe00640 1 --2- 172.21.15.86:0/1205435684 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb200cda60 0x7fbb200b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.197+0000 7fbb1fe00640 1 --2- 172.21.15.86:0/1205435684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb200cd650 0x7fbb200b64a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.197+0000 7fbb1fe00640 1 --2- 172.21.15.86:0/1205435684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb04004d50 0x7fbb2015a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.197+0000 7fbb1fe00640 1 -- 172.21.15.86:0/1205435684 >> 172.21.15.86:0/1205435684 conn(0x7fbb200b4f80 msgr2=0x7fbb200b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.197+0000 7fbb1fe00640 1 -- 172.21.15.86:0/1205435684 shutdown_connections 2024-04-25T14:04:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.197+0000 7fbb1fe00640 1 -- 172.21.15.86:0/1205435684 wait complete. 2024-04-25T14:04:07.217 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:04:07.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.361+0000 7f25e8c00640 1 Processor -- start 2024-04-25T14:04:07.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.361+0000 7f25e8c00640 1 -- start start 2024-04-25T14:04:07.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.365+0000 7f25e8c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25e40d1df0 0x7f25e40cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.365+0000 7f25e8c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f25e40cdcc0 0x7f25e40ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.365+0000 7f25e8c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25e415e5c0 con 0x7f25e4070330 2024-04-25T14:04:07.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.365+0000 7f25e8c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f25e406fd40 con 0x7f25e40cdcc0 2024-04-25T14:04:07.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.365+0000 7f25e8c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f25e415e8a0 con 0x7f25e40d1df0 2024-04-25T14:04:07.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.365+0000 7f25e2800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f25e40cdcc0 0x7f25e40ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.365+0000 7f25e2800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f25e40cdcc0 0x7f25e40ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36466/0 (socket says 172.21.15.86:36466) 2024-04-25T14:04:07.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.365+0000 7f25e2800640 1 -- 172.21.15.86:0/576060960 learned_addr learned my addr 172.21.15.86:0/576060960 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:07.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.365+0000 7f25e1e00640 1 --1- 172.21.15.86:0/576060960 >> v1:172.21.15.86:6789/0 conn(0x7f25e4070330 0x7f25e40d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52108/0 (socket says 172.21.15.86:52108) 2024-04-25T14:04:07.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.365+0000 7f25dbe00640 1 --2- 172.21.15.86:0/576060960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25e40d1df0 0x7f25e40cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.365+0000 7f25e2800640 1 -- 172.21.15.86:0/576060960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25e40d1df0 msgr2=0x7f25e40cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.365+0000 7f25e2800640 1 --2- 172.21.15.86:0/576060960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25e40d1df0 0x7f25e40cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.365+0000 7f25e2800640 1 -- 172.21.15.86:0/576060960 >> v1:172.21.15.86:6789/0 conn(0x7f25e4070330 legacy=0x7f25e40d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.365+0000 7f25e2800640 1 -- 172.21.15.86:0/576060960 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e415eb80 con 0x7f25e40cdcc0 2024-04-25T14:04:07.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.365+0000 7f25e2800640 1 --2- 172.21.15.86:0/576060960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25e40cdcc0 0x7f25e40ce0d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f25dc009e50 tx=0x7f25dc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ecfa42289383df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:07.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.369+0000 7f25e1400640 1 -- 172.21.15.86:0/576060960 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc0a5810 con 0x7f25e40cdcc0 2024-04-25T14:04:07.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.369+0000 7f25e1400640 1 -- 172.21.15.86:0/576060960 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25dc0a59b0 con 0x7f25e40cdcc0 2024-04-25T14:04:07.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.369+0000 7f25e1400640 1 -- 172.21.15.86:0/576060960 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc0a5d80 con 0x7f25e40cdcc0 2024-04-25T14:04:07.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.369+0000 7f25e8c00640 1 -- 172.21.15.86:0/576060960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25e40cdcc0 msgr2=0x7f25e40ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.369+0000 7f25e8c00640 1 --2- 172.21.15.86:0/576060960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25e40cdcc0 0x7f25e40ce0d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f25dc009e50 tx=0x7f25dc01c970 comp rx=0 tx=0).stop 2024-04-25T14:04:07.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.369+0000 7f25e8c00640 1 -- 172.21.15.86:0/576060960 shutdown_connections 2024-04-25T14:04:07.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.369+0000 7f25e8c00640 1 --2- 172.21.15.86:0/576060960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25e40cdcc0 0x7f25e40ce0d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.369+0000 7f25e8c00640 1 --2- 172.21.15.86:0/576060960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25e40d1df0 0x7f25e40cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.369+0000 7f25e8c00640 1 -- 172.21.15.86:0/576060960 >> 172.21.15.86:0/576060960 conn(0x7f25e40b4650 msgr2=0x7f25e40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:07.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.369+0000 7f25e8c00640 1 -- 172.21.15.86:0/576060960 shutdown_connections 2024-04-25T14:04:07.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.369+0000 7f25e8c00640 1 -- 172.21.15.86:0/576060960 wait complete. 2024-04-25T14:04:07.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.369+0000 7f25e8c00640 1 Processor -- start 2024-04-25T14:04:07.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.369+0000 7f25e8c00640 1 -- start start 2024-04-25T14:04:07.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.373+0000 7f25e8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25e4070330 0x7f25e41554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.373+0000 7f25e8c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25e40cdcc0 0x7f25e40b0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.373+0000 7f25e8c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25e415cd90 0x7f25e415d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.373+0000 7f25e8c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f25e415f4b0 con 0x7f25e4070330 2024-04-25T14:04:07.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.373+0000 7f25e8c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f25e415f330 con 0x7f25e415cd90 2024-04-25T14:04:07.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.373+0000 7f25e8c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f25e4070740 con 0x7f25e40cdcc0 2024-04-25T14:04:07.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.373+0000 7f25e1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25e4070330 0x7f25e41554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.373+0000 7f25e1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25e4070330 0x7f25e41554c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36476/0 (socket says 172.21.15.86:36476) 2024-04-25T14:04:07.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.373+0000 7f25e1e00640 1 -- 172.21.15.86:0/1136394295 learned_addr learned my addr 172.21.15.86:0/1136394295 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:07.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.373+0000 7f25dbe00640 1 --2- 172.21.15.86:0/1136394295 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25e40cdcc0 0x7f25e40b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.373+0000 7f25e2800640 1 --2- 172.21.15.86:0/1136394295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25e415cd90 0x7f25e415d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.373+0000 7f25e1e00640 1 -- 172.21.15.86:0/1136394295 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25e40cdcc0 msgr2=0x7f25e40b0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.373+0000 7f25e1e00640 1 --2- 172.21.15.86:0/1136394295 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25e40cdcc0 0x7f25e40b0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.373+0000 7f25e1e00640 1 -- 172.21.15.86:0/1136394295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25e415cd90 msgr2=0x7f25e415d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.373+0000 7f25e1e00640 1 --2- 172.21.15.86:0/1136394295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25e415cd90 0x7f25e415d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.373+0000 7f25e1e00640 1 -- 172.21.15.86:0/1136394295 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e415d6e0 con 0x7f25e4070330 2024-04-25T14:04:07.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.373+0000 7f25dbe00640 1 --2- 172.21.15.86:0/1136394295 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25e40cdcc0 0x7f25e40b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:07.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.373+0000 7f25e2800640 1 --2- 172.21.15.86:0/1136394295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25e415cd90 0x7f25e415d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:07.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.373+0000 7f25e1e00640 1 --2- 172.21.15.86:0/1136394295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25e4070330 0x7f25e41554c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f25d400c340 tx=0x7f25d400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:07.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.373+0000 7f25da000640 1 -- 172.21.15.86:0/1136394295 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d40074e0 con 0x7f25e4070330 2024-04-25T14:04:07.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.373+0000 7f25da000640 1 -- 172.21.15.86:0/1136394295 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25d4007680 con 0x7f25e4070330 2024-04-25T14:04:07.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.373+0000 7f25da000640 1 -- 172.21.15.86:0/1136394295 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d4004a30 con 0x7f25e4070330 2024-04-25T14:04:07.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.373+0000 7f25e8c00640 1 -- 172.21.15.86:0/1136394295 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25e4156a10 con 0x7f25e4070330 2024-04-25T14:04:07.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.373+0000 7f25e8c00640 1 -- 172.21.15.86:0/1136394295 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25e4156fd0 con 0x7f25e4070330 2024-04-25T14:04:07.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.373+0000 7f25e8c00640 1 -- 172.21.15.86:0/1136394295 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25e4069d00 con 0x7f25e4070330 2024-04-25T14:04:07.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.377+0000 7f25da000640 1 -- 172.21.15.86:0/1136394295 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f25d401a020 con 0x7f25e4070330 2024-04-25T14:04:07.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.377+0000 7f25da000640 1 --2- 172.21.15.86:0/1136394295 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f25c006f430 0x7f25c0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.377+0000 7f25da000640 1 -- 172.21.15.86:0/1136394295 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f25d408ed80 con 0x7f25e4070330 2024-04-25T14:04:07.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.377+0000 7f25dbe00640 1 --2- 172.21.15.86:0/1136394295 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f25c006f430 0x7f25c0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.377+0000 7f25dbe00640 1 --2- 172.21.15.86:0/1136394295 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f25c006f430 0x7f25c0071940 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f25e40ce2f0 tx=0x7f25cc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:07.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.381+0000 7f25da000640 1 -- 172.21.15.86:0/1136394295 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f25e415d6e0 con 0x7f25e4070330 2024-04-25T14:04:07.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.545+0000 7f25e8c00640 1 -- 172.21.15.86:0/1136394295 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25e4076bf0 con 0x7f25e4070330 2024-04-25T14:04:07.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.545+0000 7f25da000640 1 -- 172.21.15.86:0/1136394295 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7f25d405b920 con 0x7f25e4070330 2024-04-25T14:04:07.548 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:07.548 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":24,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:03:55.899092+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:03:49.151473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:03:49.802597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T14:03:53.492008+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:04:07.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.549+0000 7f25e8c00640 1 -- 172.21.15.86:0/1136394295 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f25c006f430 msgr2=0x7f25c0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.549+0000 7f25e8c00640 1 --2- 172.21.15.86:0/1136394295 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f25c006f430 0x7f25c0071940 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f25e40ce2f0 tx=0x7f25cc008040 comp rx=0 tx=0).stop 2024-04-25T14:04:07.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.549+0000 7f25e8c00640 1 -- 172.21.15.86:0/1136394295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25e4070330 msgr2=0x7f25e41554c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.549+0000 7f25e8c00640 1 --2- 172.21.15.86:0/1136394295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25e4070330 0x7f25e41554c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f25d400c340 tx=0x7f25d400c800 comp rx=0 tx=0).stop 2024-04-25T14:04:07.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.549+0000 7f25e8c00640 1 -- 172.21.15.86:0/1136394295 shutdown_connections 2024-04-25T14:04:07.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.549+0000 7f25e8c00640 1 --2- 172.21.15.86:0/1136394295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25e415cd90 0x7f25e415d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.549+0000 7f25e8c00640 1 --2- 172.21.15.86:0/1136394295 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f25c006f430 0x7f25c0071940 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.549+0000 7f25e8c00640 1 --2- 172.21.15.86:0/1136394295 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25e40cdcc0 0x7f25e40b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.549+0000 7f25e8c00640 1 --2- 172.21.15.86:0/1136394295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25e4070330 0x7f25e41554c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.549+0000 7f25e8c00640 1 -- 172.21.15.86:0/1136394295 >> 172.21.15.86:0/1136394295 conn(0x7f25e40b4650 msgr2=0x7f25e4073dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:07.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.549+0000 7f25e8c00640 1 -- 172.21.15.86:0/1136394295 shutdown_connections 2024-04-25T14:04:07.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.549+0000 7f25e8c00640 1 -- 172.21.15.86:0/1136394295 wait complete. 2024-04-25T14:04:07.571 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T14:04:07.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.717+0000 7f4c2c400640 1 Processor -- start 2024-04-25T14:04:07.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.717+0000 7f4c2c400640 1 -- start start 2024-04-25T14:04:07.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.721+0000 7f4c2c400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c2415faa0 0x7f4c2406d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.721+0000 7f4c2c400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4c240b59e0 0x7f4c240b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.721+0000 7f4c2c400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c240b6330 con 0x7f4c240aeea0 2024-04-25T14:04:07.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.721+0000 7f4c2c400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f4c2406d560 con 0x7f4c240b59e0 2024-04-25T14:04:07.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.721+0000 7f4c2c400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4c2406c670 con 0x7f4c2415faa0 2024-04-25T14:04:07.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.721+0000 7f4c29a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c2415faa0 0x7f4c2406d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.721+0000 7f4c29000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4c240b59e0 0x7f4c240b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.721+0000 7f4c2a400640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f4c240aeea0 0x7f4c240af2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:44970/0 (socket says 172.21.15.86:44970) 2024-04-25T14:04:07.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.721+0000 7f4c2a400640 1 -- 172.21.15.86:0/1852348993 learned_addr learned my addr 172.21.15.86:0/1852348993 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:07.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.721+0000 7f4c29000640 1 -- 172.21.15.86:0/1852348993 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c2415faa0 msgr2=0x7f4c2406d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.721+0000 7f4c29000640 1 --2- 172.21.15.86:0/1852348993 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c2415faa0 0x7f4c2406d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.721+0000 7f4c29000640 1 -- 172.21.15.86:0/1852348993 >> v1:172.21.15.155:6789/0 conn(0x7f4c240aeea0 legacy=0x7f4c240af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.721+0000 7f4c29000640 1 -- 172.21.15.86:0/1852348993 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c2406c950 con 0x7f4c240b59e0 2024-04-25T14:04:07.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.725+0000 7f4c29000640 1 --2- 172.21.15.86:0/1852348993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c240b59e0 0x7f4c240b5df0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4c100097c0 tx=0x7f4c1001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f4592eea452073 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:07.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.725+0000 7f4c23e00640 1 -- 172.21.15.86:0/1852348993 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c100aa600 con 0x7f4c240b59e0 2024-04-25T14:04:07.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.725+0000 7f4c23e00640 1 -- 172.21.15.86:0/1852348993 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c100af070 con 0x7f4c240b59e0 2024-04-25T14:04:07.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.725+0000 7f4c23e00640 1 -- 172.21.15.86:0/1852348993 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c100aa9c0 con 0x7f4c240b59e0 2024-04-25T14:04:07.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.725+0000 7f4c2c400640 1 -- 172.21.15.86:0/1852348993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c240b59e0 msgr2=0x7f4c240b5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.725+0000 7f4c2c400640 1 --2- 172.21.15.86:0/1852348993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c240b59e0 0x7f4c240b5df0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4c100097c0 tx=0x7f4c1001c4a0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.725+0000 7f4c2c400640 1 -- 172.21.15.86:0/1852348993 shutdown_connections 2024-04-25T14:04:07.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.725+0000 7f4c2c400640 1 --2- 172.21.15.86:0/1852348993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c240b59e0 0x7f4c240b5df0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.725+0000 7f4c2c400640 1 --2- 172.21.15.86:0/1852348993 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c2415faa0 0x7f4c2406d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.725+0000 7f4c2c400640 1 -- 172.21.15.86:0/1852348993 >> 172.21.15.86:0/1852348993 conn(0x7f4c240d2230 msgr2=0x7f4c240d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:07.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.725+0000 7f4c2c400640 1 -- 172.21.15.86:0/1852348993 shutdown_connections 2024-04-25T14:04:07.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.725+0000 7f4c2c400640 1 -- 172.21.15.86:0/1852348993 wait complete. 2024-04-25T14:04:07.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.729+0000 7f4c2c400640 1 Processor -- start 2024-04-25T14:04:07.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.729+0000 7f4c2c400640 1 -- start start 2024-04-25T14:04:07.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.729+0000 7f4c2c400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c240aeea0 0x7f4c24154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.729+0000 7f4c2c400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c240b59e0 0x7f4c2415ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.729+0000 7f4c2c400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c2415faa0 0x7f4c240b2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.729+0000 7f4c2c400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2406df30 con 0x7f4c2415faa0 2024-04-25T14:04:07.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.729+0000 7f4c2c400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4c240af9c0 con 0x7f4c240b59e0 2024-04-25T14:04:07.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.729+0000 7f4c2c400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4c2415feb0 con 0x7f4c240aeea0 2024-04-25T14:04:07.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.729+0000 7f4c2a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c2415faa0 0x7f4c240b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.729+0000 7f4c2a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c2415faa0 0x7f4c240b2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36488/0 (socket says 172.21.15.86:36488) 2024-04-25T14:04:07.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.729+0000 7f4c2a400640 1 -- 172.21.15.86:0/728706742 learned_addr learned my addr 172.21.15.86:0/728706742 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:07.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.729+0000 7f4c29000640 1 --2- 172.21.15.86:0/728706742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c240b59e0 0x7f4c2415ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.729+0000 7f4c29a00640 1 --2- 172.21.15.86:0/728706742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c240aeea0 0x7f4c24154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.729+0000 7f4c2a400640 1 -- 172.21.15.86:0/728706742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c240aeea0 msgr2=0x7f4c24154630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.729+0000 7f4c2a400640 1 --2- 172.21.15.86:0/728706742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c240aeea0 0x7f4c24154630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.729+0000 7f4c2a400640 1 -- 172.21.15.86:0/728706742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c240b59e0 msgr2=0x7f4c2415ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.729+0000 7f4c2a400640 1 --2- 172.21.15.86:0/728706742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c240b59e0 0x7f4c2415ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.729+0000 7f4c2a400640 1 -- 172.21.15.86:0/728706742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c24154b70 con 0x7f4c2415faa0 2024-04-25T14:04:07.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.729+0000 7f4c29a00640 1 --2- 172.21.15.86:0/728706742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c240aeea0 0x7f4c24154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:07.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.729+0000 7f4c2a400640 1 --2- 172.21.15.86:0/728706742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c2415faa0 0x7f4c240b2340 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c00ca80 tx=0x7f4c1c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:07.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.729+0000 7f4c21600640 1 -- 172.21.15.86:0/728706742 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c017070 con 0x7f4c2415faa0 2024-04-25T14:04:07.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.729+0000 7f4c21600640 1 -- 172.21.15.86:0/728706742 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c1c00ac30 con 0x7f4c2415faa0 2024-04-25T14:04:07.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.729+0000 7f4c2c400640 1 -- 172.21.15.86:0/728706742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c2415da20 con 0x7f4c2415faa0 2024-04-25T14:04:07.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.733+0000 7f4c21600640 1 -- 172.21.15.86:0/728706742 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c012400 con 0x7f4c2415faa0 2024-04-25T14:04:07.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.733+0000 7f4c2c400640 1 -- 172.21.15.86:0/728706742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c2415dfe0 con 0x7f4c2415faa0 2024-04-25T14:04:07.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.733+0000 7f4c2c400640 1 -- 172.21.15.86:0/728706742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c2415d770 con 0x7f4c2415faa0 2024-04-25T14:04:07.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.733+0000 7f4c21600640 1 -- 172.21.15.86:0/728706742 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4c1c01a070 con 0x7f4c2415faa0 2024-04-25T14:04:07.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.733+0000 7f4c21600640 1 --2- 172.21.15.86:0/728706742 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f4c0406f430 0x7f4c04071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.733+0000 7f4c21600640 1 -- 172.21.15.86:0/728706742 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4c1c08f150 con 0x7f4c2415faa0 2024-04-25T14:04:07.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.733+0000 7f4c29a00640 1 --2- 172.21.15.86:0/728706742 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f4c0406f430 0x7f4c04071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.737+0000 7f4c29a00640 1 --2- 172.21.15.86:0/728706742 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f4c0406f430 0x7f4c04071940 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4c18009b10 tx=0x7f4c18008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:07.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.737+0000 7f4c21600640 1 -- 172.21.15.86:0/728706742 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4c240b6330 con 0x7f4c2415faa0 2024-04-25T14:04:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:07.929+0000 7f4c2c400640 1 -- 172.21.15.86:0/728706742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f4c24074cf0 con 0x7f4c2415faa0 2024-04-25T14:04:08.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.805+0000 7f4c21600640 1 -- 172.21.15.86:0/728706742 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7f4c1c05b740 con 0x7f4c2415faa0 2024-04-25T14:04:08.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.809+0000 7f4c2c400640 1 -- 172.21.15.86:0/728706742 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f4c0406f430 msgr2=0x7f4c04071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:08.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.809+0000 7f4c2c400640 1 --2- 172.21.15.86:0/728706742 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f4c0406f430 0x7f4c04071940 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4c18009b10 tx=0x7f4c18008040 comp rx=0 tx=0).stop 2024-04-25T14:04:08.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.809+0000 7f4c2c400640 1 -- 172.21.15.86:0/728706742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c2415faa0 msgr2=0x7f4c240b2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:08.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.809+0000 7f4c2c400640 1 --2- 172.21.15.86:0/728706742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c2415faa0 0x7f4c240b2340 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c00ca80 tx=0x7f4c1c00cf40 comp rx=0 tx=0).stop 2024-04-25T14:04:08.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.809+0000 7f4c2c400640 1 -- 172.21.15.86:0/728706742 shutdown_connections 2024-04-25T14:04:08.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.809+0000 7f4c2c400640 1 --2- 172.21.15.86:0/728706742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c2415faa0 0x7f4c240b2340 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.809+0000 7f4c2c400640 1 --2- 172.21.15.86:0/728706742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c240b59e0 0x7f4c2415ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.809+0000 7f4c2c400640 1 --2- 172.21.15.86:0/728706742 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f4c0406f430 0x7f4c04071940 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.809+0000 7f4c2c400640 1 --2- 172.21.15.86:0/728706742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c240aeea0 0x7f4c24154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.809+0000 7f4c2c400640 1 -- 172.21.15.86:0/728706742 >> 172.21.15.86:0/728706742 conn(0x7f4c240d2230 msgr2=0x7f4c24161c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:08.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.809+0000 7f4c2c400640 1 -- 172.21.15.86:0/728706742 shutdown_connections 2024-04-25T14:04:08.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.809+0000 7f4c2c400640 1 -- 172.21.15.86:0/728706742 wait complete. 2024-04-25T14:04:08.834 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-25T14:04:08.835 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T14:04:08.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.989+0000 7f10ec200640 1 Processor -- start 2024-04-25T14:04:08.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.989+0000 7f10ec200640 1 -- start start 2024-04-25T14:04:08.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.989+0000 7f10ec200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10e40cfca0 0x7f10e406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:08.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.989+0000 7f10ec200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f10e40d16e0 0x7f10e40d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:08.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.989+0000 7f10ec200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f10e406c670 0x7f10e406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:08.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.989+0000 7f10ec200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f10e4070470 con 0x7f10e406c670 2024-04-25T14:04:08.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.989+0000 7f10ec200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f10e40d2030 con 0x7f10e40d16e0 2024-04-25T14:04:08.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.989+0000 7f10ec200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f10e4072bd0 con 0x7f10e40cfca0 2024-04-25T14:04:08.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.989+0000 7f10ea200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f10e406c670 0x7f10e406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:08.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.989+0000 7f10ea200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f10e406c670 0x7f10e406ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37778/0 (socket says 172.21.15.86:37778) 2024-04-25T14:04:08.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.989+0000 7f10ea200640 1 -- 172.21.15.86:0/4224289463 learned_addr learned my addr 172.21.15.86:0/4224289463 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:08.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.989+0000 7f10e9800640 1 --2- 172.21.15.86:0/4224289463 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10e40cfca0 0x7f10e406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:08.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.989+0000 7f10e8e00640 1 --2- 172.21.15.86:0/4224289463 >> v2:172.21.15.155:3300/0 conn(0x7f10e40d16e0 0x7f10e40d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:08.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.989+0000 7f10ea200640 1 -- 172.21.15.86:0/4224289463 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10e40cfca0 msgr2=0x7f10e406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:08.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.989+0000 7f10ea200640 1 --2- 172.21.15.86:0/4224289463 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10e40cfca0 0x7f10e406f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.989+0000 7f10ea200640 1 -- 172.21.15.86:0/4224289463 >> v2:172.21.15.155:3300/0 conn(0x7f10e40d16e0 msgr2=0x7f10e40d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:08.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.989+0000 7f10ea200640 1 --2- 172.21.15.86:0/4224289463 >> v2:172.21.15.155:3300/0 conn(0x7f10e40d16e0 0x7f10e40d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.989+0000 7f10ea200640 1 -- 172.21.15.86:0/4224289463 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e406f930 con 0x7f10e406c670 2024-04-25T14:04:08.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.989+0000 7f10e8e00640 1 --2- 172.21.15.86:0/4224289463 >> v2:172.21.15.155:3300/0 conn(0x7f10e40d16e0 0x7f10e40d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:08.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.993+0000 7f10ea200640 1 --2- 172.21.15.86:0/4224289463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e406c670 0x7f10e406ca80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f10e4070f40 tx=0x7f10dc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ef512b35ba8b95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:08.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.993+0000 7f10e3e00640 1 -- 172.21.15.86:0/4224289463 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc0a5780 con 0x7f10e406c670 2024-04-25T14:04:08.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.993+0000 7f10e3e00640 1 -- 172.21.15.86:0/4224289463 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10dc0a5920 con 0x7f10e406c670 2024-04-25T14:04:08.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.993+0000 7f10e3e00640 1 -- 172.21.15.86:0/4224289463 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc0a5cf0 con 0x7f10e406c670 2024-04-25T14:04:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.993+0000 7f10ec200640 1 -- 172.21.15.86:0/4224289463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e406c670 msgr2=0x7f10e406ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.993+0000 7f10ec200640 1 --2- 172.21.15.86:0/4224289463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e406c670 0x7f10e406ca80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f10e4070f40 tx=0x7f10dc01c9f0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.993+0000 7f10ec200640 1 -- 172.21.15.86:0/4224289463 shutdown_connections 2024-04-25T14:04:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.993+0000 7f10ec200640 1 --2- 172.21.15.86:0/4224289463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e406c670 0x7f10e406ca80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.993+0000 7f10ec200640 1 --2- 172.21.15.86:0/4224289463 >> v2:172.21.15.155:3300/0 conn(0x7f10e40d16e0 0x7f10e40d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.993+0000 7f10ec200640 1 --2- 172.21.15.86:0/4224289463 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10e40cfca0 0x7f10e406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.993+0000 7f10ec200640 1 -- 172.21.15.86:0/4224289463 >> 172.21.15.86:0/4224289463 conn(0x7f10e40b4650 msgr2=0x7f10e40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.993+0000 7f10ec200640 1 -- 172.21.15.86:0/4224289463 shutdown_connections 2024-04-25T14:04:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.993+0000 7f10ec200640 1 -- 172.21.15.86:0/4224289463 wait complete. 2024-04-25T14:04:08.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.997+0000 7f10ec200640 1 Processor -- start 2024-04-25T14:04:08.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.997+0000 7f10ec200640 1 -- start start 2024-04-25T14:04:09.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.997+0000 7f10ec200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10e406c670 0x7f10e4158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:09.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.997+0000 7f10ec200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10e40cfca0 0x7f10e4159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:09.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.997+0000 7f10ec200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e40d16e0 0x7f10e41597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:09.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.997+0000 7f10ec200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f10e40738c0 con 0x7f10e40d16e0 2024-04-25T14:04:09.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.997+0000 7f10ec200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f10e406d3d0 con 0x7f10e406c670 2024-04-25T14:04:09.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.997+0000 7f10ec200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f10e4073a40 con 0x7f10e40cfca0 2024-04-25T14:04:09.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.997+0000 7f10ea200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e40d16e0 0x7f10e41597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:09.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.997+0000 7f10ea200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e40d16e0 0x7f10e41597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37788/0 (socket says 172.21.15.86:37788) 2024-04-25T14:04:09.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.997+0000 7f10ea200640 1 -- 172.21.15.86:0/1434379549 learned_addr learned my addr 172.21.15.86:0/1434379549 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:09.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.997+0000 7f10e9800640 1 --2- 172.21.15.86:0/1434379549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10e406c670 0x7f10e4158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:09.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.997+0000 7f10e8e00640 1 --2- 172.21.15.86:0/1434379549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10e40cfca0 0x7f10e4159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:09.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.997+0000 7f10ea200640 1 -- 172.21.15.86:0/1434379549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10e40cfca0 msgr2=0x7f10e4159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:09.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.997+0000 7f10ea200640 1 --2- 172.21.15.86:0/1434379549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10e40cfca0 0x7f10e4159260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.997+0000 7f10ea200640 1 -- 172.21.15.86:0/1434379549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10e406c670 msgr2=0x7f10e4158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:09.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.997+0000 7f10ea200640 1 --2- 172.21.15.86:0/1434379549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10e406c670 0x7f10e4158d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.997+0000 7f10ea200640 1 -- 172.21.15.86:0/1434379549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e40b4080 con 0x7f10e40d16e0 2024-04-25T14:04:09.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:08.997+0000 7f10e8e00640 1 --2- 172.21.15.86:0/1434379549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10e40cfca0 0x7f10e4159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:09.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.001+0000 7f10ea200640 1 --2- 172.21.15.86:0/1434379549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e40d16e0 0x7f10e41597a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0af040 tx=0x7f10dc019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:09.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.001+0000 7f10e1600640 1 -- 172.21.15.86:0/1434379549 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc009510 con 0x7f10e40d16e0 2024-04-25T14:04:09.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.001+0000 7f10e1600640 1 -- 172.21.15.86:0/1434379549 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10dc0be070 con 0x7f10e40d16e0 2024-04-25T14:04:09.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.001+0000 7f10ec200640 1 -- 172.21.15.86:0/1434379549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e4159fd0 con 0x7f10e40d16e0 2024-04-25T14:04:09.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.001+0000 7f10e1600640 1 -- 172.21.15.86:0/1434379549 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc0b88b0 con 0x7f10e40d16e0 2024-04-25T14:04:09.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.001+0000 7f10ec200640 1 -- 172.21.15.86:0/1434379549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e40ca040 con 0x7f10e40d16e0 2024-04-25T14:04:09.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.001+0000 7f10ec200640 1 -- 172.21.15.86:0/1434379549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e409f4b0 con 0x7f10e40d16e0 2024-04-25T14:04:09.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.001+0000 7f10e1600640 1 -- 172.21.15.86:0/1434379549 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f10dc0b8a50 con 0x7f10e40d16e0 2024-04-25T14:04:09.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.001+0000 7f10e1600640 1 --2- 172.21.15.86:0/1434379549 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f10c406f430 0x7f10c4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:09.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.001+0000 7f10e1600640 1 -- 172.21.15.86:0/1434379549 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f10dc12a550 con 0x7f10e40d16e0 2024-04-25T14:04:09.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.005+0000 7f10e9800640 1 --2- 172.21.15.86:0/1434379549 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f10c406f430 0x7f10c4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:09.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.005+0000 7f10e9800640 1 --2- 172.21.15.86:0/1434379549 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f10c406f430 0x7f10c4071940 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f10e4070710 tx=0x7f10d8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:09.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.009+0000 7f10e1600640 1 -- 172.21.15.86:0/1434379549 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f10dc0165e0 con 0x7f10e40d16e0 2024-04-25T14:04:09.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.197+0000 7f10ec200640 1 -- 172.21.15.86:0/1434379549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f10e4076bf0 con 0x7f10e40d16e0 2024-04-25T14:04:09.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.809+0000 7f10e1600640 1 -- 172.21.15.86:0/1434379549 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f10dc0f7060 con 0x7f10e40d16e0 2024-04-25T14:04:09.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.813+0000 7f10ec200640 1 -- 172.21.15.86:0/1434379549 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f10c406f430 msgr2=0x7f10c4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:09.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.813+0000 7f10ec200640 1 --2- 172.21.15.86:0/1434379549 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f10c406f430 0x7f10c4071940 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f10e4070710 tx=0x7f10d8009190 comp rx=0 tx=0).stop 2024-04-25T14:04:09.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.813+0000 7f10ec200640 1 -- 172.21.15.86:0/1434379549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e40d16e0 msgr2=0x7f10e41597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:09.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.813+0000 7f10ec200640 1 --2- 172.21.15.86:0/1434379549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e40d16e0 0x7f10e41597a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0af040 tx=0x7f10dc019790 comp rx=0 tx=0).stop 2024-04-25T14:04:09.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.813+0000 7f10ec200640 1 -- 172.21.15.86:0/1434379549 shutdown_connections 2024-04-25T14:04:09.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.813+0000 7f10ec200640 1 --2- 172.21.15.86:0/1434379549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10e40d16e0 0x7f10e41597a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.813+0000 7f10ec200640 1 --2- 172.21.15.86:0/1434379549 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f10c406f430 0x7f10c4071940 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.813+0000 7f10ec200640 1 --2- 172.21.15.86:0/1434379549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10e40cfca0 0x7f10e4159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.813+0000 7f10ec200640 1 --2- 172.21.15.86:0/1434379549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10e406c670 0x7f10e4158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.813+0000 7f10ec200640 1 -- 172.21.15.86:0/1434379549 >> 172.21.15.86:0/1434379549 conn(0x7f10e40b4650 msgr2=0x7f10e40b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:09.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.813+0000 7f10ec200640 1 -- 172.21.15.86:0/1434379549 shutdown_connections 2024-04-25T14:04:09.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.813+0000 7f10ec200640 1 -- 172.21.15.86:0/1434379549 wait complete. 2024-04-25T14:04:09.838 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T14:04:09.838 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:04:09.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.985+0000 7f66b7e00640 1 Processor -- start 2024-04-25T14:04:09.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.985+0000 7f66b7e00640 1 -- start start 2024-04-25T14:04:09.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.989+0000 7f66b7e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66b8160120 0x7f66b80af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:09.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.989+0000 7f66b7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f66b80d22d0 0x7f66b80d26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:09.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.989+0000 7f66b7e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66b80af780 con 0x7f66b80d0f50 2024-04-25T14:04:09.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.989+0000 7f66b7e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f66b80d0240 con 0x7f66b80d22d0 2024-04-25T14:04:09.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.989+0000 7f66b7e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f66b80b7fd0 con 0x7f66b8160120 2024-04-25T14:04:09.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.989+0000 7f66b7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f66b80d22d0 0x7f66b80d26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:09.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.989+0000 7f66b6000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f66b80d0f50 0x7f66b80d1360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33318/0 (socket says 172.21.15.86:33318) 2024-04-25T14:04:09.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.989+0000 7f66b6000640 1 -- 172.21.15.86:0/1988959241 learned_addr learned my addr 172.21.15.86:0/1988959241 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:09.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.989+0000 7f66b6a00640 1 --2- 172.21.15.86:0/1988959241 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66b8160120 0x7f66b80af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:09.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.989+0000 7f66b7400640 1 -- 172.21.15.86:0/1988959241 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66b8160120 msgr2=0x7f66b80af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:09.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.989+0000 7f66b7400640 1 --2- 172.21.15.86:0/1988959241 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66b8160120 0x7f66b80af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.989+0000 7f66b7400640 1 -- 172.21.15.86:0/1988959241 >> v1:172.21.15.86:6789/0 conn(0x7f66b80d0f50 legacy=0x7f66b80d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:09.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.989+0000 7f66b7400640 1 -- 172.21.15.86:0/1988959241 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b80b7cf0 con 0x7f66b80d22d0 2024-04-25T14:04:09.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.989+0000 7f66b7400640 1 --2- 172.21.15.86:0/1988959241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66b80d22d0 0x7f66b80d26e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009dc0 tx=0x7f66b001c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f0e5304aec8226 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:09.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.993+0000 7f66b5600640 1 -- 172.21.15.86:0/1988959241 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b00b2070 con 0x7f66b80d22d0 2024-04-25T14:04:09.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.993+0000 7f66b5600640 1 -- 172.21.15.86:0/1988959241 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b00a57b0 con 0x7f66b80d22d0 2024-04-25T14:04:09.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.993+0000 7f66b5600640 1 -- 172.21.15.86:0/1988959241 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b00a5b80 con 0x7f66b80d22d0 2024-04-25T14:04:09.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.993+0000 7f66b7e00640 1 -- 172.21.15.86:0/1988959241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66b80d22d0 msgr2=0x7f66b80d26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:09.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.993+0000 7f66b7e00640 1 --2- 172.21.15.86:0/1988959241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66b80d22d0 0x7f66b80d26e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009dc0 tx=0x7f66b001c750 comp rx=0 tx=0).stop 2024-04-25T14:04:09.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.993+0000 7f66b7e00640 1 -- 172.21.15.86:0/1988959241 shutdown_connections 2024-04-25T14:04:09.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.993+0000 7f66b7e00640 1 --2- 172.21.15.86:0/1988959241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66b80d22d0 0x7f66b80d26e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.993+0000 7f66b7e00640 1 --2- 172.21.15.86:0/1988959241 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66b8160120 0x7f66b80af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.993+0000 7f66b7e00640 1 -- 172.21.15.86:0/1988959241 >> 172.21.15.86:0/1988959241 conn(0x7f66b809ba30 msgr2=0x7f66b80b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:09.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.993+0000 7f66b7e00640 1 -- 172.21.15.86:0/1988959241 shutdown_connections 2024-04-25T14:04:09.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.993+0000 7f66b7e00640 1 -- 172.21.15.86:0/1988959241 wait complete. 2024-04-25T14:04:09.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.993+0000 7f66b7e00640 1 Processor -- start 2024-04-25T14:04:09.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.993+0000 7f66b7e00640 1 -- start start 2024-04-25T14:04:09.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.997+0000 7f66b7e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66b80d0f50 0x7f66b8149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:09.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.997+0000 7f66b7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66b80d22d0 0x7f66b81f9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:09.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.997+0000 7f66b7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66b8160120 0x7f66b806e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:09.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.997+0000 7f66b7e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f66b80b79c0 con 0x7f66b8160120 2024-04-25T14:04:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.997+0000 7f66b7e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f66b80d0750 con 0x7f66b80d22d0 2024-04-25T14:04:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.997+0000 7f66b7e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f66b80d1a70 con 0x7f66b80d0f50 2024-04-25T14:04:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.997+0000 7f66b7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66b8160120 0x7f66b806e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.997+0000 7f66b7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66b8160120 0x7f66b806e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37806/0 (socket says 172.21.15.86:37806) 2024-04-25T14:04:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.997+0000 7f66b7400640 1 -- 172.21.15.86:0/2335260531 learned_addr learned my addr 172.21.15.86:0/2335260531 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.997+0000 7f66b6a00640 1 --2- 172.21.15.86:0/2335260531 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66b80d0f50 0x7f66b8149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.997+0000 7f66b6000640 1 --2- 172.21.15.86:0/2335260531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66b80d22d0 0x7f66b81f9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.997+0000 7f66b7400640 1 -- 172.21.15.86:0/2335260531 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66b80d0f50 msgr2=0x7f66b8149dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.997+0000 7f66b7400640 1 --2- 172.21.15.86:0/2335260531 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66b80d0f50 0x7f66b8149dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.997+0000 7f66b7400640 1 -- 172.21.15.86:0/2335260531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66b80d22d0 msgr2=0x7f66b81f9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.997+0000 7f66b7400640 1 --2- 172.21.15.86:0/2335260531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66b80d22d0 0x7f66b81f9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.997+0000 7f66b7400640 1 -- 172.21.15.86:0/2335260531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b814a300 con 0x7f66b8160120 2024-04-25T14:04:10.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.997+0000 7f66b7400640 1 --2- 172.21.15.86:0/2335260531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66b8160120 0x7f66b806e550 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f66b001cff0 tx=0x7f66b00194f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:10.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.997+0000 7f66af400640 1 -- 172.21.15.86:0/2335260531 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b00b2050 con 0x7f66b8160120 2024-04-25T14:04:10.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.997+0000 7f66af400640 1 -- 172.21.15.86:0/2335260531 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b00be070 con 0x7f66b8160120 2024-04-25T14:04:10.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.997+0000 7f66af400640 1 -- 172.21.15.86:0/2335260531 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0009450 con 0x7f66b8160120 2024-04-25T14:04:10.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.997+0000 7f66b7e00640 1 -- 172.21.15.86:0/2335260531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b81fa790 con 0x7f66b8160120 2024-04-25T14:04:10.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.997+0000 7f66b7e00640 1 -- 172.21.15.86:0/2335260531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b8160780 con 0x7f66b8160120 2024-04-25T14:04:10.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:09.997+0000 7f66b7e00640 1 -- 172.21.15.86:0/2335260531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66b8076b80 con 0x7f66b8160120 2024-04-25T14:04:10.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.001+0000 7f66af400640 1 -- 172.21.15.86:0/2335260531 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f66b00c3460 con 0x7f66b8160120 2024-04-25T14:04:10.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.001+0000 7f66af400640 1 --2- 172.21.15.86:0/2335260531 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f669806f400 0x7f6698071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:10.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.001+0000 7f66af400640 1 -- 172.21.15.86:0/2335260531 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f66b012b470 con 0x7f66b8160120 2024-04-25T14:04:10.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.001+0000 7f66b6a00640 1 --2- 172.21.15.86:0/2335260531 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f669806f400 0x7f6698071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:10.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.001+0000 7f66b6a00640 1 --2- 172.21.15.86:0/2335260531 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f669806f400 0x7f6698071910 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f66a80079c0 tx=0x7f66a8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:10.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.005+0000 7f66af400640 1 -- 172.21.15.86:0/2335260531 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f66b00164a0 con 0x7f66b8160120 2024-04-25T14:04:10.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.193+0000 7f66b7e00640 1 -- 172.21.15.86:0/2335260531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66b81fa520 con 0x7f66b8160120 2024-04-25T14:04:10.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.197+0000 7f66af400640 1 -- 172.21.15.86:0/2335260531 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5177 (secure 0 0 0) 0x7f66b00f7a40 con 0x7f66b8160120 2024-04-25T14:04:10.197 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:10.197 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/347107112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":347107112},{"type":"v1","addr":"172.21.15.155:6835","nonce":347107112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2236737549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2236737549},{"type":"v1","addr":"172.21.15.86:6837","nonce":2236737549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/519560229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":519560229},{"type":"v1","addr":"172.21.15.86:6836","nonce":519560229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:03:50.852257+0000","modified":"2024-04-25T14:04:09.202137+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6833/1944249570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1944249570},{"type":"v1","addr":"172.21.15.155:6833","nonce":1944249570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":1}]} 2024-04-25T14:04:10.197 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 9 2024-04-25T14:04:10.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.197+0000 7f66b7e00640 1 -- 172.21.15.86:0/2335260531 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f669806f400 msgr2=0x7f6698071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:10.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.197+0000 7f66b7e00640 1 --2- 172.21.15.86:0/2335260531 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f669806f400 0x7f6698071910 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f66a80079c0 tx=0x7f66a8008040 comp rx=0 tx=0).stop 2024-04-25T14:04:10.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.197+0000 7f66b7e00640 1 -- 172.21.15.86:0/2335260531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66b8160120 msgr2=0x7f66b806e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:10.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.197+0000 7f66b7e00640 1 --2- 172.21.15.86:0/2335260531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66b8160120 0x7f66b806e550 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f66b001cff0 tx=0x7f66b00194f0 comp rx=0 tx=0).stop 2024-04-25T14:04:10.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.197+0000 7f66b7e00640 1 -- 172.21.15.86:0/2335260531 shutdown_connections 2024-04-25T14:04:10.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.197+0000 7f66b7e00640 1 --2- 172.21.15.86:0/2335260531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66b8160120 0x7f66b806e550 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:10.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.197+0000 7f66b7e00640 1 --2- 172.21.15.86:0/2335260531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66b80d22d0 0x7f66b81f9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:10.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.197+0000 7f66b7e00640 1 --2- 172.21.15.86:0/2335260531 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f669806f400 0x7f6698071910 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:10.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.197+0000 7f66b7e00640 1 --2- 172.21.15.86:0/2335260531 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66b80d0f50 0x7f66b8149dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:10.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.197+0000 7f66b7e00640 1 -- 172.21.15.86:0/2335260531 >> 172.21.15.86:0/2335260531 conn(0x7f66b809ba30 msgr2=0x7f66b80af950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:10.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.197+0000 7f66b7e00640 1 -- 172.21.15.86:0/2335260531 shutdown_connections 2024-04-25T14:04:10.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.197+0000 7f66b7e00640 1 -- 172.21.15.86:0/2335260531 wait complete. 2024-04-25T14:04:10.218 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:04:10.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.361+0000 7f5f0d200640 1 Processor -- start 2024-04-25T14:04:10.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.365+0000 7f5f0d200640 1 -- start start 2024-04-25T14:04:10.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.365+0000 7f5f0d200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5f080cfca0 0x7f5f0806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:10.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.365+0000 7f5f0d200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5f080d16e0 0x7f5f080d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:10.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.365+0000 7f5f0d200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f080d2030 con 0x7f5f0806c670 2024-04-25T14:04:10.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.365+0000 7f5f0d200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5f08070470 con 0x7f5f080d16e0 2024-04-25T14:04:10.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.365+0000 7f5f0d200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f5f08072bd0 con 0x7f5f080cfca0 2024-04-25T14:04:10.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.365+0000 7f5efd400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5f080d16e0 0x7f5f080d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:10.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.365+0000 7f5f06800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5f0806c670 0x7f5f0806ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33332/0 (socket says 172.21.15.86:33332) 2024-04-25T14:04:10.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.365+0000 7f5f06800640 1 -- 172.21.15.86:0/2065714289 learned_addr learned my addr 172.21.15.86:0/2065714289 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:10.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.365+0000 7f5f05e00640 1 --2- 172.21.15.86:0/2065714289 >> v2:172.21.15.155:3300/0 conn(0x7f5f080cfca0 0x7f5f0806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:10.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.365+0000 7f5f05400640 1 -- 172.21.15.86:0/2065714289 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1845632700 0 0) 0x7f5f000014a0 con 0x7f5f0806c670 2024-04-25T14:04:10.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.365+0000 7f5f05400640 1 -- 172.21.15.86:0/2065714289 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5eec003530 con 0x7f5f0806c670 2024-04-25T14:04:10.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.365+0000 7f5efd400640 1 -- 172.21.15.86:0/2065714289 >> v2:172.21.15.155:3300/0 conn(0x7f5f080cfca0 msgr2=0x7f5f0806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:10.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.365+0000 7f5efd400640 1 --2- 172.21.15.86:0/2065714289 >> v2:172.21.15.155:3300/0 conn(0x7f5f080cfca0 0x7f5f0806f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:10.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.365+0000 7f5efd400640 1 -- 172.21.15.86:0/2065714289 >> v1:172.21.15.86:6789/0 conn(0x7f5f0806c670 legacy=0x7f5f0806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:10.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.365+0000 7f5efd400640 1 -- 172.21.15.86:0/2065714289 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f0806f930 con 0x7f5f080d16e0 2024-04-25T14:04:10.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.369+0000 7f5efd400640 1 --2- 172.21.15.86:0/2065714289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f080d16e0 0x7f5f080d1af0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5f080d22a0 tx=0x7f5ef001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c68e3bf22dd951f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:10.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.369+0000 7f5f05400640 1 -- 172.21.15.86:0/2065714289 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef00b1070 con 0x7f5f080d16e0 2024-04-25T14:04:10.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.369+0000 7f5f05400640 1 -- 172.21.15.86:0/2065714289 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ef00ac5f0 con 0x7f5f080d16e0 2024-04-25T14:04:10.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.369+0000 7f5f05400640 1 -- 172.21.15.86:0/2065714289 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef00ac9c0 con 0x7f5f080d16e0 2024-04-25T14:04:10.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.369+0000 7f5f0d200640 1 -- 172.21.15.86:0/2065714289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f080d16e0 msgr2=0x7f5f080d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:10.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.369+0000 7f5f0d200640 1 --2- 172.21.15.86:0/2065714289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f080d16e0 0x7f5f080d1af0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5f080d22a0 tx=0x7f5ef001c480 comp rx=0 tx=0).stop 2024-04-25T14:04:10.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.369+0000 7f5f0d200640 1 -- 172.21.15.86:0/2065714289 shutdown_connections 2024-04-25T14:04:10.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.369+0000 7f5f0d200640 1 --2- 172.21.15.86:0/2065714289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f080d16e0 0x7f5f080d1af0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:10.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.369+0000 7f5f0d200640 1 --2- 172.21.15.86:0/2065714289 >> v2:172.21.15.155:3300/0 conn(0x7f5f080cfca0 0x7f5f0806f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:10.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.369+0000 7f5f0d200640 1 -- 172.21.15.86:0/2065714289 >> 172.21.15.86:0/2065714289 conn(0x7f5f080b4650 msgr2=0x7f5f080b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:10.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.369+0000 7f5f0d200640 1 -- 172.21.15.86:0/2065714289 shutdown_connections 2024-04-25T14:04:10.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.369+0000 7f5f0d200640 1 -- 172.21.15.86:0/2065714289 wait complete. 2024-04-25T14:04:10.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.373+0000 7f5f0d200640 1 Processor -- start 2024-04-25T14:04:10.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.373+0000 7f5f0d200640 1 -- start start 2024-04-25T14:04:10.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.373+0000 7f5f0d200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f0806c670 0x7f5f0809a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:10.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.373+0000 7f5f0d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f080cfca0 0x7f5f080cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:10.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.373+0000 7f5f0d200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f080d16e0 0x7f5f080cd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:10.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.373+0000 7f5f0d200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0816a070 con 0x7f5f080cfca0 2024-04-25T14:04:10.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.373+0000 7f5f0d200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5f08169ef0 con 0x7f5f0806c670 2024-04-25T14:04:10.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.373+0000 7f5f0d200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5f0816a1f0 con 0x7f5f080d16e0 2024-04-25T14:04:10.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.373+0000 7f5efd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f080cfca0 0x7f5f080cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:10.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.373+0000 7f5efd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f080cfca0 0x7f5f080cd1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37824/0 (socket says 172.21.15.86:37824) 2024-04-25T14:04:10.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.373+0000 7f5efd400640 1 -- 172.21.15.86:0/3368648400 learned_addr learned my addr 172.21.15.86:0/3368648400 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:10.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.373+0000 7f5f05e00640 1 --2- 172.21.15.86:0/3368648400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f0806c670 0x7f5f0809a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:10.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.373+0000 7f5f06800640 1 --2- 172.21.15.86:0/3368648400 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f080d16e0 0x7f5f080cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:10.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.373+0000 7f5efd400640 1 -- 172.21.15.86:0/3368648400 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f080d16e0 msgr2=0x7f5f080cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:10.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.373+0000 7f5efd400640 1 --2- 172.21.15.86:0/3368648400 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f080d16e0 0x7f5f080cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:10.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.373+0000 7f5efd400640 1 -- 172.21.15.86:0/3368648400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f0806c670 msgr2=0x7f5f0809a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:10.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.373+0000 7f5efd400640 1 --2- 172.21.15.86:0/3368648400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f0806c670 0x7f5f0809a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:10.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.373+0000 7f5efd400640 1 -- 172.21.15.86:0/3368648400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f080cbf40 con 0x7f5f080cfca0 2024-04-25T14:04:10.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.373+0000 7f5efd400640 1 --2- 172.21.15.86:0/3368648400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f080cfca0 0x7f5f080cd1d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5ef001ab30 tx=0x7f5ef0019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:10.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.373+0000 7f5efea00640 1 -- 172.21.15.86:0/3368648400 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef00b1040 con 0x7f5f080cfca0 2024-04-25T14:04:10.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.373+0000 7f5efea00640 1 -- 172.21.15.86:0/3368648400 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ef00bb070 con 0x7f5f080cfca0 2024-04-25T14:04:10.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.373+0000 7f5efea00640 1 -- 172.21.15.86:0/3368648400 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0006e20 con 0x7f5f080cfca0 2024-04-25T14:04:10.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.373+0000 7f5f0d200640 1 -- 172.21.15.86:0/3368648400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f080cc220 con 0x7f5f080cfca0 2024-04-25T14:04:10.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.373+0000 7f5f0d200640 1 -- 172.21.15.86:0/3368648400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f080cc7e0 con 0x7f5f080cfca0 2024-04-25T14:04:10.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.377+0000 7f5f0d200640 1 -- 172.21.15.86:0/3368648400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f0809f4b0 con 0x7f5f080cfca0 2024-04-25T14:04:10.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.377+0000 7f5efea00640 1 -- 172.21.15.86:0/3368648400 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5ef00b5650 con 0x7f5f080cfca0 2024-04-25T14:04:10.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.377+0000 7f5efea00640 1 --2- 172.21.15.86:0/3368648400 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f5ee406f430 0x7f5ee4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:10.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.377+0000 7f5efea00640 1 -- 172.21.15.86:0/3368648400 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5ef0128fb0 con 0x7f5f080cfca0 2024-04-25T14:04:10.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.377+0000 7f5f05e00640 1 --2- 172.21.15.86:0/3368648400 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f5ee406f430 0x7f5ee4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:10.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.377+0000 7f5f05e00640 1 --2- 172.21.15.86:0/3368648400 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f5ee406f430 0x7f5ee4071940 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5f08070710 tx=0x7f5ef8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:10.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.381+0000 7f5efea00640 1 -- 172.21.15.86:0/3368648400 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5ef00a9070 con 0x7f5f080cfca0 2024-04-25T14:04:10.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.541+0000 7f5f0d200640 1 -- 172.21.15.86:0/3368648400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f08069d00 con 0x7f5f080cfca0 2024-04-25T14:04:10.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.541+0000 7f5efea00640 1 -- 172.21.15.86:0/3368648400 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7f5ef0016070 con 0x7f5f080cfca0 2024-04-25T14:04:10.543 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:10.544 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":24,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:03:55.899092+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:03:49.151473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:03:49.802597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T14:03:53.492008+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:04:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.545+0000 7f5f0d200640 1 -- 172.21.15.86:0/3368648400 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f5ee406f430 msgr2=0x7f5ee4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.545+0000 7f5f0d200640 1 --2- 172.21.15.86:0/3368648400 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f5ee406f430 0x7f5ee4071940 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5f08070710 tx=0x7f5ef8008040 comp rx=0 tx=0).stop 2024-04-25T14:04:10.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.545+0000 7f5f0d200640 1 -- 172.21.15.86:0/3368648400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f080cfca0 msgr2=0x7f5f080cd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:10.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.545+0000 7f5f0d200640 1 --2- 172.21.15.86:0/3368648400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f080cfca0 0x7f5f080cd1d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5ef001ab30 tx=0x7f5ef0019220 comp rx=0 tx=0).stop 2024-04-25T14:04:10.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.545+0000 7f5f0d200640 1 -- 172.21.15.86:0/3368648400 shutdown_connections 2024-04-25T14:04:10.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.545+0000 7f5f0d200640 1 --2- 172.21.15.86:0/3368648400 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f5ee406f430 0x7f5ee4071940 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:10.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.545+0000 7f5f0d200640 1 --2- 172.21.15.86:0/3368648400 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f080d16e0 0x7f5f080cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:10.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.545+0000 7f5f0d200640 1 --2- 172.21.15.86:0/3368648400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f080cfca0 0x7f5f080cd1d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:10.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.545+0000 7f5f0d200640 1 --2- 172.21.15.86:0/3368648400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f0806c670 0x7f5f0809a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:10.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.545+0000 7f5f0d200640 1 -- 172.21.15.86:0/3368648400 >> 172.21.15.86:0/3368648400 conn(0x7f5f080b4650 msgr2=0x7f5f08156420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:10.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.545+0000 7f5f0d200640 1 -- 172.21.15.86:0/3368648400 shutdown_connections 2024-04-25T14:04:10.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.545+0000 7f5f0d200640 1 -- 172.21.15.86:0/3368648400 wait complete. 2024-04-25T14:04:10.565 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T14:04:10.566 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:04:10.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.709+0000 7f59f3e00640 1 Processor -- start 2024-04-25T14:04:10.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.709+0000 7f59f3e00640 1 -- start start 2024-04-25T14:04:10.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.713+0000 7f59f3e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59ec15c2a0 0x7f59ec0ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:10.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.713+0000 7f59f3e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59ec0b0a60 con 0x7f59ec15feb0 2024-04-25T14:04:10.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.713+0000 7f59f3e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59ec06ef90 con 0x7f59ec06fd50 2024-04-25T14:04:10.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.713+0000 7f59f3e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f59ec06f270 con 0x7f59ec15c2a0 2024-04-25T14:04:10.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.713+0000 7f59f1e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f59ec15feb0 0x7f59ec1602c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33342/0 (socket says 172.21.15.86:33342) 2024-04-25T14:04:10.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.713+0000 7f59f1e00640 1 -- 172.21.15.86:0/3504965915 learned_addr learned my addr 172.21.15.86:0/3504965915 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:10.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.713+0000 7f59f1400640 1 --2- 172.21.15.86:0/3504965915 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59ec15c2a0 0x7f59ec0ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:10.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.713+0000 7f59ebe00640 1 -- 172.21.15.86:0/3504965915 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1343621661 0 0) 0x7f59ec0b0a60 con 0x7f59ec15feb0 2024-04-25T14:04:10.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.713+0000 7f59ebe00640 1 -- 172.21.15.86:0/3504965915 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59d4003450 con 0x7f59ec15feb0 2024-04-25T14:04:10.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.713+0000 7f59ebe00640 1 -- 172.21.15.86:0/3504965915 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3769046675 0 0) 0x7f59ec06ef90 con 0x7f59ec06fd50 2024-04-25T14:04:10.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.713+0000 7f59ebe00640 1 -- 172.21.15.86:0/3504965915 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59ec0b0a60 con 0x7f59ec06fd50 2024-04-25T14:04:10.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.713+0000 7f59f1400640 1 -- 172.21.15.86:0/3504965915 >> v1:172.21.15.155:6789/0 conn(0x7f59ec06fd50 legacy=0x7f59ec070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:10.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.713+0000 7f59f1400640 1 -- 172.21.15.86:0/3504965915 >> v1:172.21.15.86:6789/0 conn(0x7f59ec15feb0 legacy=0x7f59ec1602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:10.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.713+0000 7f59ebe00640 1 -- 172.21.15.86:0/3504965915 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1297970479 0 0) 0x7f59d4003450 con 0x7f59ec15feb0 2024-04-25T14:04:10.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.713+0000 7f59f1400640 1 -- 172.21.15.86:0/3504965915 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59ec06f550 con 0x7f59ec15c2a0 2024-04-25T14:04:10.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.717+0000 7f59f1400640 1 --2- 172.21.15.86:0/3504965915 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59ec15c2a0 0x7f59ec0ae130 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f59e0009dc0 tx=0x7f59e001c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc7e36cad2892a57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:10.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.717+0000 7f59ebe00640 1 -- 172.21.15.86:0/3504965915 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e4001680 con 0x7f59ec15c2a0 2024-04-25T14:04:10.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.717+0000 7f59ebe00640 1 -- 172.21.15.86:0/3504965915 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59e00a64d0 con 0x7f59ec15c2a0 2024-04-25T14:04:10.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.717+0000 7f59ebe00640 1 -- 172.21.15.86:0/3504965915 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e00a6880 con 0x7f59ec15c2a0 2024-04-25T14:04:10.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.717+0000 7f59f3e00640 1 -- 172.21.15.86:0/3504965915 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59ec15c2a0 msgr2=0x7f59ec0ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:10.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.717+0000 7f59f3e00640 1 --2- 172.21.15.86:0/3504965915 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59ec15c2a0 0x7f59ec0ae130 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f59e0009dc0 tx=0x7f59e001c7f0 comp rx=0 tx=0).stop 2024-04-25T14:04:10.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.717+0000 7f59f3e00640 1 -- 172.21.15.86:0/3504965915 shutdown_connections 2024-04-25T14:04:10.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.717+0000 7f59f3e00640 1 --2- 172.21.15.86:0/3504965915 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59ec15c2a0 0x7f59ec0ae130 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:10.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.717+0000 7f59f3e00640 1 -- 172.21.15.86:0/3504965915 >> 172.21.15.86:0/3504965915 conn(0x7f59ec0b4650 msgr2=0x7f59ec0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:10.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.717+0000 7f59f3e00640 1 -- 172.21.15.86:0/3504965915 shutdown_connections 2024-04-25T14:04:10.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.717+0000 7f59f3e00640 1 -- 172.21.15.86:0/3504965915 wait complete. 2024-04-25T14:04:10.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.721+0000 7f59f3e00640 1 Processor -- start 2024-04-25T14:04:10.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.721+0000 7f59f3e00640 1 -- start start 2024-04-25T14:04:10.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.721+0000 7f59f3e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59ec06fd50 0x7f59ec15c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:10.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.721+0000 7f59f3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59ec15c2a0 0x7f59ec154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:10.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.721+0000 7f59f3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59ec15feb0 0x7f59ec09a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:10.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.721+0000 7f59f3e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f59ec0ae7f0 con 0x7f59ec15c2a0 2024-04-25T14:04:10.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.721+0000 7f59f3e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f59ec0ae670 con 0x7f59ec15feb0 2024-04-25T14:04:10.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.721+0000 7f59f3e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f59ec16a200 con 0x7f59ec06fd50 2024-04-25T14:04:10.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.721+0000 7f59f0a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59ec15c2a0 0x7f59ec154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:10.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.721+0000 7f59f0a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59ec15c2a0 0x7f59ec154340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37830/0 (socket says 172.21.15.86:37830) 2024-04-25T14:04:10.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.721+0000 7f59f0a00640 1 -- 172.21.15.86:0/2570424516 learned_addr learned my addr 172.21.15.86:0/2570424516 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:10.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.721+0000 7f59f1400640 1 --2- 172.21.15.86:0/2570424516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59ec06fd50 0x7f59ec15c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:10.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.721+0000 7f59f1e00640 1 --2- 172.21.15.86:0/2570424516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59ec15feb0 0x7f59ec09a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:10.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.721+0000 7f59f0a00640 1 -- 172.21.15.86:0/2570424516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59ec06fd50 msgr2=0x7f59ec15c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:10.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.721+0000 7f59f0a00640 1 --2- 172.21.15.86:0/2570424516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59ec06fd50 0x7f59ec15c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:10.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.721+0000 7f59f0a00640 1 -- 172.21.15.86:0/2570424516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59ec15feb0 msgr2=0x7f59ec09a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:10.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.721+0000 7f59f0a00640 1 --2- 172.21.15.86:0/2570424516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59ec15feb0 0x7f59ec09a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:10.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.721+0000 7f59f0a00640 1 -- 172.21.15.86:0/2570424516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59ec0b4fd0 con 0x7f59ec15c2a0 2024-04-25T14:04:10.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.721+0000 7f59f0a00640 1 --2- 172.21.15.86:0/2570424516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59ec15c2a0 0x7f59ec154340 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f59d800c460 tx=0x7f59d800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:10.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.721+0000 7f59e9600640 1 -- 172.21.15.86:0/2570424516 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d80052d0 con 0x7f59ec15c2a0 2024-04-25T14:04:10.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.721+0000 7f59e9600640 1 -- 172.21.15.86:0/2570424516 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59d8005490 con 0x7f59ec15c2a0 2024-04-25T14:04:10.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.721+0000 7f59f3e00640 1 -- 172.21.15.86:0/2570424516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59ec0b24c0 con 0x7f59ec15c2a0 2024-04-25T14:04:10.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.721+0000 7f59e9600640 1 -- 172.21.15.86:0/2570424516 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d8012850 con 0x7f59ec15c2a0 2024-04-25T14:04:10.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.721+0000 7f59f3e00640 1 -- 172.21.15.86:0/2570424516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59ec0b2a80 con 0x7f59ec15c2a0 2024-04-25T14:04:10.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.725+0000 7f59f3e00640 1 -- 172.21.15.86:0/2570424516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59ec069d00 con 0x7f59ec15c2a0 2024-04-25T14:04:10.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.725+0000 7f59e9600640 1 -- 172.21.15.86:0/2570424516 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f59d80129f0 con 0x7f59ec15c2a0 2024-04-25T14:04:10.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.725+0000 7f59e9600640 1 --2- 172.21.15.86:0/2570424516 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f59cc06f400 0x7f59cc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:10.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.725+0000 7f59e9600640 1 -- 172.21.15.86:0/2570424516 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f59d808f180 con 0x7f59ec15c2a0 2024-04-25T14:04:10.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.725+0000 7f59f1400640 1 --2- 172.21.15.86:0/2570424516 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f59cc06f400 0x7f59cc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:10.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.729+0000 7f59f1400640 1 --2- 172.21.15.86:0/2570424516 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f59cc06f400 0x7f59cc071910 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f59e00af000 tx=0x7f59e00a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:10.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.733+0000 7f59e9600640 1 -- 172.21.15.86:0/2570424516 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f59d805b640 con 0x7f59ec15c2a0 2024-04-25T14:04:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.889+0000 7f59f3e00640 1 -- 172.21.15.86:0/2570424516 --> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f59ec1549d0 con 0x7f59cc06f400 2024-04-25T14:04:10.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.901+0000 7f59e9600640 1 -- 172.21.15.86:0/2570424516 <== mgr.4103 v2:172.21.15.86:6832/2968869755 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332527 (secure 0 0 0) 0x7f59ec1549d0 con 0x7f59cc06f400 2024-04-25T14:04:10.905 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:10.907 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:04:10.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.905+0000 7f59f3e00640 1 -- 172.21.15.86:0/2570424516 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f59cc06f400 msgr2=0x7f59cc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:10.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.905+0000 7f59f3e00640 1 --2- 172.21.15.86:0/2570424516 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f59cc06f400 0x7f59cc071910 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f59e00af000 tx=0x7f59e00a7000 comp rx=0 tx=0).stop 2024-04-25T14:04:10.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.905+0000 7f59f3e00640 1 -- 172.21.15.86:0/2570424516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59ec15c2a0 msgr2=0x7f59ec154340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:10.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.905+0000 7f59f3e00640 1 --2- 172.21.15.86:0/2570424516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59ec15c2a0 0x7f59ec154340 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f59d800c460 tx=0x7f59d800c920 comp rx=0 tx=0).stop 2024-04-25T14:04:10.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.905+0000 7f59f3e00640 1 -- 172.21.15.86:0/2570424516 shutdown_connections 2024-04-25T14:04:10.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.905+0000 7f59f3e00640 1 --2- 172.21.15.86:0/2570424516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59ec15feb0 0x7f59ec09a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:10.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.905+0000 7f59f3e00640 1 --2- 172.21.15.86:0/2570424516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59ec15c2a0 0x7f59ec154340 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:10.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.905+0000 7f59f3e00640 1 --2- 172.21.15.86:0/2570424516 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f59cc06f400 0x7f59cc071910 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:10.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.905+0000 7f59f3e00640 1 --2- 172.21.15.86:0/2570424516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59ec06fd50 0x7f59ec15c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:10.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.905+0000 7f59f3e00640 1 -- 172.21.15.86:0/2570424516 >> 172.21.15.86:0/2570424516 conn(0x7f59ec0b4650 msgr2=0x7f59ec0ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:10.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.905+0000 7f59f3e00640 1 -- 172.21.15.86:0/2570424516 shutdown_connections 2024-04-25T14:04:10.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:10.905+0000 7f59f3e00640 1 -- 172.21.15.86:0/2570424516 wait complete. 2024-04-25T14:04:10.926 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-04-25T14:04:10.714677+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":150,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219400,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755320,"statfs":{"total":773094113280,"available":772869447680,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003039"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774780+0000","last_change":"2024-04-25T14:03:50.477862+0000","last_active":"2024-04-25T14:03:56.774780+0000","last_peered":"2024-04-25T14:03:56.774780+0000","last_clean":"2024-04-25T14:03:56.774780+0000","last_became_active":"2024-04-25T14:03:50.477375+0000","last_became_peered":"2024-04-25T14:03:50.477375+0000","last_unstale":"2024-04-25T14:03:56.774780+0000","last_undegraded":"2024-04-25T14:03:56.774780+0000","last_fullsized":"2024-04-25T14:03:56.774780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:01.852459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494933+0000","last_change":"2024-04-25T14:03:51.479650+0000","last_active":"2024-04-25T14:03:56.494933+0000","last_peered":"2024-04-25T14:03:56.494933+0000","last_clean":"2024-04-25T14:03:56.494933+0000","last_became_active":"2024-04-25T14:03:51.479295+0000","last_became_peered":"2024-04-25T14:03:51.479295+0000","last_unstale":"2024-04-25T14:03:56.494933+0000","last_undegraded":"2024-04-25T14:03:56.494933+0000","last_fullsized":"2024-04-25T14:03:56.494933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:17.381249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493593+0000","last_change":"2024-04-25T14:03:55.681212+0000","last_active":"2024-04-25T14:03:56.493593+0000","last_peered":"2024-04-25T14:03:56.493593+0000","last_clean":"2024-04-25T14:03:56.493593+0000","last_became_active":"2024-04-25T14:03:55.680579+0000","last_became_peered":"2024-04-25T14:03:55.680579+0000","last_unstale":"2024-04-25T14:03:56.493593+0000","last_undegraded":"2024-04-25T14:03:56.493593+0000","last_fullsized":"2024-04-25T14:03:56.493593+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:33:32.909419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495038+0000","last_change":"2024-04-25T14:03:50.468056+0000","last_active":"2024-04-25T14:03:56.495038+0000","last_peered":"2024-04-25T14:03:56.495038+0000","last_clean":"2024-04-25T14:03:56.495038+0000","last_became_active":"2024-04-25T14:03:50.467531+0000","last_became_peered":"2024-04-25T14:03:50.467531+0000","last_unstale":"2024-04-25T14:03:56.495038+0000","last_undegraded":"2024-04-25T14:03:56.495038+0000","last_fullsized":"2024-04-25T14:03:56.495038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:29.264084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774540+0000","last_change":"2024-04-25T14:03:51.485995+0000","last_active":"2024-04-25T14:03:56.774540+0000","last_peered":"2024-04-25T14:03:56.774540+0000","last_clean":"2024-04-25T14:03:56.774540+0000","last_became_active":"2024-04-25T14:03:51.485364+0000","last_became_peered":"2024-04-25T14:03:51.485364+0000","last_unstale":"2024-04-25T14:03:56.774540+0000","last_undegraded":"2024-04-25T14:03:56.774540+0000","last_fullsized":"2024-04-25T14:03:56.774540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:10.318378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772007+0000","last_change":"2024-04-25T14:03:55.514585+0000","last_active":"2024-04-25T14:03:56.772007+0000","last_peered":"2024-04-25T14:03:56.772007+0000","last_clean":"2024-04-25T14:03:56.772007+0000","last_became_active":"2024-04-25T14:03:55.514106+0000","last_became_peered":"2024-04-25T14:03:55.514106+0000","last_unstale":"2024-04-25T14:03:56.772007+0000","last_undegraded":"2024-04-25T14:03:56.772007+0000","last_fullsized":"2024-04-25T14:03:56.772007+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:14.363355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773034+0000","last_change":"2024-04-25T14:03:50.476493+0000","last_active":"2024-04-25T14:03:56.773034+0000","last_peered":"2024-04-25T14:03:56.773034+0000","last_clean":"2024-04-25T14:03:56.773034+0000","last_became_active":"2024-04-25T14:03:50.476154+0000","last_became_peered":"2024-04-25T14:03:50.476154+0000","last_unstale":"2024-04-25T14:03:56.773034+0000","last_undegraded":"2024-04-25T14:03:56.773034+0000","last_fullsized":"2024-04-25T14:03:56.773034+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:45.080718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496310+0000","last_change":"2024-04-25T14:03:51.494560+0000","last_active":"2024-04-25T14:03:56.496310+0000","last_peered":"2024-04-25T14:03:56.496310+0000","last_clean":"2024-04-25T14:03:56.496310+0000","last_became_active":"2024-04-25T14:03:51.494211+0000","last_became_peered":"2024-04-25T14:03:51.494211+0000","last_unstale":"2024-04-25T14:03:56.496310+0000","last_undegraded":"2024-04-25T14:03:56.496310+0000","last_fullsized":"2024-04-25T14:03:56.496310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:17.381249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772740+0000","last_change":"2024-04-25T14:03:55.518340+0000","last_active":"2024-04-25T14:03:56.772740+0000","last_peered":"2024-04-25T14:03:56.772740+0000","last_clean":"2024-04-25T14:03:56.772740+0000","last_became_active":"2024-04-25T14:03:55.515787+0000","last_became_peered":"2024-04-25T14:03:55.515787+0000","last_unstale":"2024-04-25T14:03:56.772740+0000","last_undegraded":"2024-04-25T14:03:56.772740+0000","last_fullsized":"2024-04-25T14:03:56.772740+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:24.136045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772400+0000","last_change":"2024-04-25T14:03:50.472667+0000","last_active":"2024-04-25T14:03:56.772400+0000","last_peered":"2024-04-25T14:03:56.772400+0000","last_clean":"2024-04-25T14:03:56.772400+0000","last_became_active":"2024-04-25T14:03:50.472286+0000","last_became_peered":"2024-04-25T14:03:50.472286+0000","last_unstale":"2024-04-25T14:03:56.772400+0000","last_undegraded":"2024-04-25T14:03:56.772400+0000","last_fullsized":"2024-04-25T14:03:56.772400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:02.117945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496569+0000","last_change":"2024-04-25T14:03:51.494545+0000","last_active":"2024-04-25T14:03:56.496569+0000","last_peered":"2024-04-25T14:03:56.496569+0000","last_clean":"2024-04-25T14:03:56.496569+0000","last_became_active":"2024-04-25T14:03:51.494238+0000","last_became_peered":"2024-04-25T14:03:51.494238+0000","last_unstale":"2024-04-25T14:03:56.496569+0000","last_undegraded":"2024-04-25T14:03:56.496569+0000","last_fullsized":"2024-04-25T14:03:56.496569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:08.845346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772125+0000","last_change":"2024-04-25T14:03:55.512835+0000","last_active":"2024-04-25T14:03:56.772125+0000","last_peered":"2024-04-25T14:03:56.772125+0000","last_clean":"2024-04-25T14:03:56.772125+0000","last_became_active":"2024-04-25T14:03:55.512330+0000","last_became_peered":"2024-04-25T14:03:55.512330+0000","last_unstale":"2024-04-25T14:03:56.772125+0000","last_undegraded":"2024-04-25T14:03:56.772125+0000","last_fullsized":"2024-04-25T14:03:56.772125+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:47.781818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494962+0000","last_change":"2024-04-25T14:03:51.494496+0000","last_active":"2024-04-25T14:03:56.494962+0000","last_peered":"2024-04-25T14:03:56.494962+0000","last_clean":"2024-04-25T14:03:56.494962+0000","last_became_active":"2024-04-25T14:03:51.494201+0000","last_became_peered":"2024-04-25T14:03:51.494201+0000","last_unstale":"2024-04-25T14:03:56.494962+0000","last_undegraded":"2024-04-25T14:03:56.494962+0000","last_fullsized":"2024-04-25T14:03:56.494962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.599283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773644+0000","last_change":"2024-04-25T14:03:50.472026+0000","last_active":"2024-04-25T14:03:56.773644+0000","last_peered":"2024-04-25T14:03:56.773644+0000","last_clean":"2024-04-25T14:03:56.773644+0000","last_became_active":"2024-04-25T14:03:50.468476+0000","last_became_peered":"2024-04-25T14:03:50.468476+0000","last_unstale":"2024-04-25T14:03:56.773644+0000","last_undegraded":"2024-04-25T14:03:56.773644+0000","last_fullsized":"2024-04-25T14:03:56.773644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:39.630327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772916+0000","last_change":"2024-04-25T14:03:55.524347+0000","last_active":"2024-04-25T14:03:56.772916+0000","last_peered":"2024-04-25T14:03:56.772916+0000","last_clean":"2024-04-25T14:03:56.772916+0000","last_became_active":"2024-04-25T14:03:55.523890+0000","last_became_peered":"2024-04-25T14:03:55.523890+0000","last_unstale":"2024-04-25T14:03:56.772916+0000","last_undegraded":"2024-04-25T14:03:56.772916+0000","last_fullsized":"2024-04-25T14:03:56.772916+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:20.858506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493611+0000","last_change":"2024-04-25T14:03:51.471282+0000","last_active":"2024-04-25T14:03:56.493611+0000","last_peered":"2024-04-25T14:03:56.493611+0000","last_clean":"2024-04-25T14:03:56.493611+0000","last_became_active":"2024-04-25T14:03:51.470871+0000","last_became_peered":"2024-04-25T14:03:51.470871+0000","last_unstale":"2024-04-25T14:03:56.493611+0000","last_undegraded":"2024-04-25T14:03:56.493611+0000","last_fullsized":"2024-04-25T14:03:56.493611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:34.707119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773462+0000","last_change":"2024-04-25T14:03:50.467312+0000","last_active":"2024-04-25T14:03:56.773462+0000","last_peered":"2024-04-25T14:03:56.773462+0000","last_clean":"2024-04-25T14:03:56.773462+0000","last_became_active":"2024-04-25T14:03:50.467020+0000","last_became_peered":"2024-04-25T14:03:50.467020+0000","last_unstale":"2024-04-25T14:03:56.773462+0000","last_undegraded":"2024-04-25T14:03:56.773462+0000","last_fullsized":"2024-04-25T14:03:56.773462+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:45.080718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772970+0000","last_change":"2024-04-25T14:03:55.520580+0000","last_active":"2024-04-25T14:03:56.772970+0000","last_peered":"2024-04-25T14:03:56.772970+0000","last_clean":"2024-04-25T14:03:56.772970+0000","last_became_active":"2024-04-25T14:03:55.520013+0000","last_became_peered":"2024-04-25T14:03:55.520013+0000","last_unstale":"2024-04-25T14:03:56.772970+0000","last_undegraded":"2024-04-25T14:03:56.772970+0000","last_fullsized":"2024-04-25T14:03:56.772970+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:38.752096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494042+0000","last_change":"2024-04-25T14:03:51.474463+0000","last_active":"2024-04-25T14:03:56.494042+0000","last_peered":"2024-04-25T14:03:56.494042+0000","last_clean":"2024-04-25T14:03:56.494042+0000","last_became_active":"2024-04-25T14:03:51.474154+0000","last_became_peered":"2024-04-25T14:03:51.474154+0000","last_unstale":"2024-04-25T14:03:56.494042+0000","last_undegraded":"2024-04-25T14:03:56.494042+0000","last_fullsized":"2024-04-25T14:03:56.494042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:10.318378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772985+0000","last_change":"2024-04-25T14:03:50.476410+0000","last_active":"2024-04-25T14:03:56.772985+0000","last_peered":"2024-04-25T14:03:56.772985+0000","last_clean":"2024-04-25T14:03:56.772985+0000","last_became_active":"2024-04-25T14:03:50.476083+0000","last_became_peered":"2024-04-25T14:03:50.476083+0000","last_unstale":"2024-04-25T14:03:56.772985+0000","last_undegraded":"2024-04-25T14:03:56.772985+0000","last_fullsized":"2024-04-25T14:03:56.772985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:29.264084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774278+0000","last_change":"2024-04-25T14:03:55.674875+0000","last_active":"2024-04-25T14:03:56.774278+0000","last_peered":"2024-04-25T14:03:56.774278+0000","last_clean":"2024-04-25T14:03:56.774278+0000","last_became_active":"2024-04-25T14:03:55.674408+0000","last_became_peered":"2024-04-25T14:03:55.674408+0000","last_unstale":"2024-04-25T14:03:56.774278+0000","last_undegraded":"2024-04-25T14:03:56.774278+0000","last_fullsized":"2024-04-25T14:03:56.774278+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:50.234674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773825+0000","last_change":"2024-04-25T14:03:51.487560+0000","last_active":"2024-04-25T14:03:56.773825+0000","last_peered":"2024-04-25T14:03:56.773825+0000","last_clean":"2024-04-25T14:03:56.773825+0000","last_became_active":"2024-04-25T14:03:51.486822+0000","last_became_peered":"2024-04-25T14:03:51.486822+0000","last_unstale":"2024-04-25T14:03:56.773825+0000","last_undegraded":"2024-04-25T14:03:56.773825+0000","last_fullsized":"2024-04-25T14:03:56.773825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:16.370061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.771874+0000","last_change":"2024-04-25T14:03:50.479489+0000","last_active":"2024-04-25T14:03:56.771874+0000","last_peered":"2024-04-25T14:03:56.771874+0000","last_clean":"2024-04-25T14:03:56.771874+0000","last_became_active":"2024-04-25T14:03:50.479212+0000","last_became_peered":"2024-04-25T14:03:50.479212+0000","last_unstale":"2024-04-25T14:03:56.771874+0000","last_undegraded":"2024-04-25T14:03:56.771874+0000","last_fullsized":"2024-04-25T14:03:56.771874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:10.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773707+0000","last_change":"2024-04-25T14:03:55.519638+0000","last_active":"2024-04-25T14:03:56.773707+0000","last_peered":"2024-04-25T14:03:56.773707+0000","last_clean":"2024-04-25T14:03:56.773707+0000","last_became_active":"2024-04-25T14:03:55.519191+0000","last_became_peered":"2024-04-25T14:03:55.519191+0000","last_unstale":"2024-04-25T14:03:56.773707+0000","last_undegraded":"2024-04-25T14:03:56.773707+0000","last_fullsized":"2024-04-25T14:03:56.773707+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:25.451897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774673+0000","last_change":"2024-04-25T14:03:51.477687+0000","last_active":"2024-04-25T14:03:56.774673+0000","last_peered":"2024-04-25T14:03:56.774673+0000","last_clean":"2024-04-25T14:03:56.774673+0000","last_became_active":"2024-04-25T14:03:51.477339+0000","last_became_peered":"2024-04-25T14:03:51.477339+0000","last_unstale":"2024-04-25T14:03:56.774673+0000","last_undegraded":"2024-04-25T14:03:56.774673+0000","last_fullsized":"2024-04-25T14:03:56.774673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:43.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496742+0000","last_change":"2024-04-25T14:03:50.470614+0000","last_active":"2024-04-25T14:03:56.496742+0000","last_peered":"2024-04-25T14:03:56.496742+0000","last_clean":"2024-04-25T14:03:56.496742+0000","last_became_active":"2024-04-25T14:03:50.470405+0000","last_became_peered":"2024-04-25T14:03:50.470405+0000","last_unstale":"2024-04-25T14:03:56.496742+0000","last_undegraded":"2024-04-25T14:03:56.496742+0000","last_fullsized":"2024-04-25T14:03:56.496742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:02.117945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773539+0000","last_change":"2024-04-25T14:03:55.681096+0000","last_active":"2024-04-25T14:03:56.773539+0000","last_peered":"2024-04-25T14:03:56.773539+0000","last_clean":"2024-04-25T14:03:56.773539+0000","last_became_active":"2024-04-25T14:03:55.680556+0000","last_became_peered":"2024-04-25T14:03:55.680556+0000","last_unstale":"2024-04-25T14:03:56.773539+0000","last_undegraded":"2024-04-25T14:03:56.773539+0000","last_fullsized":"2024-04-25T14:03:56.773539+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:09.534866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495184+0000","last_change":"2024-04-25T14:03:51.473595+0000","last_active":"2024-04-25T14:03:56.495184+0000","last_peered":"2024-04-25T14:03:56.495184+0000","last_clean":"2024-04-25T14:03:56.495184+0000","last_became_active":"2024-04-25T14:03:51.473273+0000","last_became_peered":"2024-04-25T14:03:51.473273+0000","last_unstale":"2024-04-25T14:03:56.495184+0000","last_undegraded":"2024-04-25T14:03:56.495184+0000","last_fullsized":"2024-04-25T14:03:56.495184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:05.489889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774166+0000","last_change":"2024-04-25T14:03:50.476274+0000","last_active":"2024-04-25T14:03:56.774166+0000","last_peered":"2024-04-25T14:03:56.774166+0000","last_clean":"2024-04-25T14:03:56.774166+0000","last_became_active":"2024-04-25T14:03:50.475935+0000","last_became_peered":"2024-04-25T14:03:50.475935+0000","last_unstale":"2024-04-25T14:03:56.774166+0000","last_undegraded":"2024-04-25T14:03:56.774166+0000","last_fullsized":"2024-04-25T14:03:56.774166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:39.630327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493917+0000","last_change":"2024-04-25T14:03:55.675251+0000","last_active":"2024-04-25T14:03:56.493917+0000","last_peered":"2024-04-25T14:03:56.493917+0000","last_clean":"2024-04-25T14:03:56.493917+0000","last_became_active":"2024-04-25T14:03:55.674772+0000","last_became_peered":"2024-04-25T14:03:55.674772+0000","last_unstale":"2024-04-25T14:03:56.493917+0000","last_undegraded":"2024-04-25T14:03:56.493917+0000","last_fullsized":"2024-04-25T14:03:56.493917+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:58.844877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773967+0000","last_change":"2024-04-25T14:03:51.476624+0000","last_active":"2024-04-25T14:03:56.773967+0000","last_peered":"2024-04-25T14:03:56.773967+0000","last_clean":"2024-04-25T14:03:56.773967+0000","last_became_active":"2024-04-25T14:03:51.476320+0000","last_became_peered":"2024-04-25T14:03:51.476320+0000","last_unstale":"2024-04-25T14:03:56.773967+0000","last_undegraded":"2024-04-25T14:03:56.773967+0000","last_fullsized":"2024-04-25T14:03:56.773967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:20.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773171+0000","last_change":"2024-04-25T14:03:50.482353+0000","last_active":"2024-04-25T14:03:56.773171+0000","last_peered":"2024-04-25T14:03:56.773171+0000","last_clean":"2024-04-25T14:03:56.773171+0000","last_became_active":"2024-04-25T14:03:50.482063+0000","last_became_peered":"2024-04-25T14:03:50.482063+0000","last_unstale":"2024-04-25T14:03:56.773171+0000","last_undegraded":"2024-04-25T14:03:56.773171+0000","last_fullsized":"2024-04-25T14:03:56.773171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:37.406291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773151+0000","last_change":"2024-04-25T14:03:55.676872+0000","last_active":"2024-04-25T14:03:56.773151+0000","last_peered":"2024-04-25T14:03:56.773151+0000","last_clean":"2024-04-25T14:03:56.773151+0000","last_became_active":"2024-04-25T14:03:55.676444+0000","last_became_peered":"2024-04-25T14:03:55.676444+0000","last_unstale":"2024-04-25T14:03:56.773151+0000","last_undegraded":"2024-04-25T14:03:56.773151+0000","last_fullsized":"2024-04-25T14:03:56.773151+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:37.811376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496691+0000","last_change":"2024-04-25T14:03:51.480200+0000","last_active":"2024-04-25T14:03:56.496691+0000","last_peered":"2024-04-25T14:03:56.496691+0000","last_clean":"2024-04-25T14:03:56.496691+0000","last_became_active":"2024-04-25T14:03:51.479798+0000","last_became_peered":"2024-04-25T14:03:51.479798+0000","last_unstale":"2024-04-25T14:03:56.496691+0000","last_undegraded":"2024-04-25T14:03:56.496691+0000","last_fullsized":"2024-04-25T14:03:56.496691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:46.189697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774003+0000","last_change":"2024-04-25T14:03:50.479761+0000","last_active":"2024-04-25T14:03:56.774003+0000","last_peered":"2024-04-25T14:03:56.774003+0000","last_clean":"2024-04-25T14:03:56.774003+0000","last_became_active":"2024-04-25T14:03:50.479175+0000","last_became_peered":"2024-04-25T14:03:50.479175+0000","last_unstale":"2024-04-25T14:03:56.774003+0000","last_undegraded":"2024-04-25T14:03:56.774003+0000","last_fullsized":"2024-04-25T14:03:56.774003+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:02.117945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773256+0000","last_change":"2024-04-25T14:03:55.515692+0000","last_active":"2024-04-25T14:03:56.773256+0000","last_peered":"2024-04-25T14:03:56.773256+0000","last_clean":"2024-04-25T14:03:56.773256+0000","last_became_active":"2024-04-25T14:03:55.515238+0000","last_became_peered":"2024-04-25T14:03:55.515238+0000","last_unstale":"2024-04-25T14:03:56.773256+0000","last_undegraded":"2024-04-25T14:03:56.773256+0000","last_fullsized":"2024-04-25T14:03:56.773256+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:51.371333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773096+0000","last_change":"2024-04-25T14:03:51.495207+0000","last_active":"2024-04-25T14:03:56.773096+0000","last_peered":"2024-04-25T14:03:56.773096+0000","last_clean":"2024-04-25T14:03:56.773096+0000","last_became_active":"2024-04-25T14:03:51.492531+0000","last_became_peered":"2024-04-25T14:03:51.492531+0000","last_unstale":"2024-04-25T14:03:56.773096+0000","last_undegraded":"2024-04-25T14:03:56.773096+0000","last_fullsized":"2024-04-25T14:03:56.773096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:03.123407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495711+0000","last_change":"2024-04-25T14:03:50.459698+0000","last_active":"2024-04-25T14:03:56.495711+0000","last_peered":"2024-04-25T14:03:56.495711+0000","last_clean":"2024-04-25T14:03:56.495711+0000","last_became_active":"2024-04-25T14:03:50.459451+0000","last_became_peered":"2024-04-25T14:03:50.459451+0000","last_unstale":"2024-04-25T14:03:56.495711+0000","last_undegraded":"2024-04-25T14:03:56.495711+0000","last_fullsized":"2024-04-25T14:03:56.495711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:15.364599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773837+0000","last_change":"2024-04-25T14:03:55.519334+0000","last_active":"2024-04-25T14:03:56.773837+0000","last_peered":"2024-04-25T14:03:56.773837+0000","last_clean":"2024-04-25T14:03:56.773837+0000","last_became_active":"2024-04-25T14:03:55.518940+0000","last_became_peered":"2024-04-25T14:03:55.518940+0000","last_unstale":"2024-04-25T14:03:56.773837+0000","last_undegraded":"2024-04-25T14:03:56.773837+0000","last_fullsized":"2024-04-25T14:03:56.773837+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:12.890323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772902+0000","last_change":"2024-04-25T14:03:51.476141+0000","last_active":"2024-04-25T14:03:56.772902+0000","last_peered":"2024-04-25T14:03:56.772902+0000","last_clean":"2024-04-25T14:03:56.772902+0000","last_became_active":"2024-04-25T14:03:51.475819+0000","last_became_peered":"2024-04-25T14:03:51.475819+0000","last_unstale":"2024-04-25T14:03:56.772902+0000","last_undegraded":"2024-04-25T14:03:56.772902+0000","last_fullsized":"2024-04-25T14:03:56.772902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:31.707048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772028+0000","last_change":"2024-04-25T14:03:50.458612+0000","last_active":"2024-04-25T14:03:56.772028+0000","last_peered":"2024-04-25T14:03:56.772028+0000","last_clean":"2024-04-25T14:03:56.772028+0000","last_became_active":"2024-04-25T14:03:50.458254+0000","last_became_peered":"2024-04-25T14:03:50.458254+0000","last_unstale":"2024-04-25T14:03:56.772028+0000","last_undegraded":"2024-04-25T14:03:56.772028+0000","last_fullsized":"2024-04-25T14:03:56.772028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:29.264084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774279+0000","last_change":"2024-04-25T14:03:55.513023+0000","last_active":"2024-04-25T14:03:56.774279+0000","last_peered":"2024-04-25T14:03:56.774279+0000","last_clean":"2024-04-25T14:03:56.774279+0000","last_became_active":"2024-04-25T14:03:55.512550+0000","last_became_peered":"2024-04-25T14:03:55.512550+0000","last_unstale":"2024-04-25T14:03:56.774279+0000","last_undegraded":"2024-04-25T14:03:56.774279+0000","last_fullsized":"2024-04-25T14:03:56.774279+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:06.818399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.771972+0000","last_change":"2024-04-25T14:03:51.487355+0000","last_active":"2024-04-25T14:03:56.771972+0000","last_peered":"2024-04-25T14:03:56.771972+0000","last_clean":"2024-04-25T14:03:56.771972+0000","last_became_active":"2024-04-25T14:03:51.487050+0000","last_became_peered":"2024-04-25T14:03:51.487050+0000","last_unstale":"2024-04-25T14:03:56.771972+0000","last_undegraded":"2024-04-25T14:03:56.771972+0000","last_fullsized":"2024-04-25T14:03:56.771972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:33.766258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772008+0000","last_change":"2024-04-25T14:03:50.476776+0000","last_active":"2024-04-25T14:03:56.772008+0000","last_peered":"2024-04-25T14:03:56.772008+0000","last_clean":"2024-04-25T14:03:56.772008+0000","last_became_active":"2024-04-25T14:03:50.476470+0000","last_became_peered":"2024-04-25T14:03:50.476470+0000","last_unstale":"2024-04-25T14:03:56.772008+0000","last_undegraded":"2024-04-25T14:03:56.772008+0000","last_fullsized":"2024-04-25T14:03:56.772008+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:15.364599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774604+0000","last_change":"2024-04-25T14:03:55.674908+0000","last_active":"2024-04-25T14:03:56.774604+0000","last_peered":"2024-04-25T14:03:56.774604+0000","last_clean":"2024-04-25T14:03:56.774604+0000","last_became_active":"2024-04-25T14:03:55.674383+0000","last_became_peered":"2024-04-25T14:03:55.674383+0000","last_unstale":"2024-04-25T14:03:56.774604+0000","last_undegraded":"2024-04-25T14:03:56.774604+0000","last_fullsized":"2024-04-25T14:03:56.774604+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:06.742929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493557+0000","last_change":"2024-04-25T14:03:51.489259+0000","last_active":"2024-04-25T14:03:56.493557+0000","last_peered":"2024-04-25T14:03:56.493557+0000","last_clean":"2024-04-25T14:03:56.493557+0000","last_became_active":"2024-04-25T14:03:51.488822+0000","last_became_peered":"2024-04-25T14:03:51.488822+0000","last_unstale":"2024-04-25T14:03:56.493557+0000","last_undegraded":"2024-04-25T14:03:56.493557+0000","last_fullsized":"2024-04-25T14:03:56.493557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:43.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495950+0000","last_change":"2024-04-25T14:03:50.466483+0000","last_active":"2024-04-25T14:03:56.495950+0000","last_peered":"2024-04-25T14:03:56.495950+0000","last_clean":"2024-04-25T14:03:56.495950+0000","last_became_active":"2024-04-25T14:03:50.463125+0000","last_became_peered":"2024-04-25T14:03:50.463125+0000","last_unstale":"2024-04-25T14:03:56.495950+0000","last_undegraded":"2024-04-25T14:03:56.495950+0000","last_fullsized":"2024-04-25T14:03:56.495950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:29.264084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":117,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772125+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:03:56.772125+0000","last_peered":"2024-04-25T14:03:56.772125+0000","last_clean":"2024-04-25T14:03:56.772125+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:03:56.772125+0000","last_undegraded":"2024-04-25T14:03:56.772125+0000","last_fullsized":"2024-04-25T14:03:56.772125+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496238+0000","last_change":"2024-04-25T14:03:55.678084+0000","last_active":"2024-04-25T14:03:56.496238+0000","last_peered":"2024-04-25T14:03:56.496238+0000","last_clean":"2024-04-25T14:03:56.496238+0000","last_became_active":"2024-04-25T14:03:55.677526+0000","last_became_peered":"2024-04-25T14:03:55.677526+0000","last_unstale":"2024-04-25T14:03:56.496238+0000","last_undegraded":"2024-04-25T14:03:56.496238+0000","last_fullsized":"2024-04-25T14:03:56.496238+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:25.451897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772807+0000","last_change":"2024-04-25T14:03:51.482608+0000","last_active":"2024-04-25T14:03:56.772807+0000","last_peered":"2024-04-25T14:03:56.772807+0000","last_clean":"2024-04-25T14:03:56.772807+0000","last_became_active":"2024-04-25T14:03:51.482022+0000","last_became_peered":"2024-04-25T14:03:51.482022+0000","last_unstale":"2024-04-25T14:03:56.772807+0000","last_undegraded":"2024-04-25T14:03:56.772807+0000","last_fullsized":"2024-04-25T14:03:56.772807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:33.766399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772584+0000","last_change":"2024-04-25T14:03:50.469170+0000","last_active":"2024-04-25T14:03:56.772584+0000","last_peered":"2024-04-25T14:03:56.772584+0000","last_clean":"2024-04-25T14:03:56.772584+0000","last_became_active":"2024-04-25T14:03:50.468742+0000","last_became_peered":"2024-04-25T14:03:50.468742+0000","last_unstale":"2024-04-25T14:03:56.772584+0000","last_undegraded":"2024-04-25T14:03:56.772584+0000","last_fullsized":"2024-04-25T14:03:56.772584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:10.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494752+0000","last_change":"2024-04-25T14:03:55.676560+0000","last_active":"2024-04-25T14:03:56.494752+0000","last_peered":"2024-04-25T14:03:56.494752+0000","last_clean":"2024-04-25T14:03:56.494752+0000","last_became_active":"2024-04-25T14:03:55.676021+0000","last_became_peered":"2024-04-25T14:03:55.676021+0000","last_unstale":"2024-04-25T14:03:56.494752+0000","last_undegraded":"2024-04-25T14:03:56.494752+0000","last_fullsized":"2024-04-25T14:03:56.494752+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:09.534866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496565+0000","last_change":"2024-04-25T14:03:51.466314+0000","last_active":"2024-04-25T14:03:56.496565+0000","last_peered":"2024-04-25T14:03:56.496565+0000","last_clean":"2024-04-25T14:03:56.496565+0000","last_became_active":"2024-04-25T14:03:51.465982+0000","last_became_peered":"2024-04-25T14:03:51.465982+0000","last_unstale":"2024-04-25T14:03:56.496565+0000","last_undegraded":"2024-04-25T14:03:56.496565+0000","last_fullsized":"2024-04-25T14:03:56.496565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:47.326356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774109+0000","last_change":"2024-04-25T14:03:50.465115+0000","last_active":"2024-04-25T14:03:56.774109+0000","last_peered":"2024-04-25T14:03:56.774109+0000","last_clean":"2024-04-25T14:03:56.774109+0000","last_became_active":"2024-04-25T14:03:50.464561+0000","last_became_peered":"2024-04-25T14:03:50.464561+0000","last_unstale":"2024-04-25T14:03:56.774109+0000","last_undegraded":"2024-04-25T14:03:56.774109+0000","last_fullsized":"2024-04-25T14:03:56.774109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:10.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773457+0000","last_change":"2024-04-25T14:03:55.526344+0000","last_active":"2024-04-25T14:03:56.773457+0000","last_peered":"2024-04-25T14:03:56.773457+0000","last_clean":"2024-04-25T14:03:56.773457+0000","last_became_active":"2024-04-25T14:03:55.525889+0000","last_became_peered":"2024-04-25T14:03:55.525889+0000","last_unstale":"2024-04-25T14:03:56.773457+0000","last_undegraded":"2024-04-25T14:03:56.773457+0000","last_fullsized":"2024-04-25T14:03:56.773457+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:06.818399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773492+0000","last_change":"2024-04-25T14:03:51.494453+0000","last_active":"2024-04-25T14:03:56.773492+0000","last_peered":"2024-04-25T14:03:56.773492+0000","last_clean":"2024-04-25T14:03:56.773492+0000","last_became_active":"2024-04-25T14:03:51.494113+0000","last_became_peered":"2024-04-25T14:03:51.494113+0000","last_unstale":"2024-04-25T14:03:56.773492+0000","last_undegraded":"2024-04-25T14:03:56.773492+0000","last_fullsized":"2024-04-25T14:03:56.773492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:08.845346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772486+0000","last_change":"2024-04-25T14:03:50.463824+0000","last_active":"2024-04-25T14:03:56.772486+0000","last_peered":"2024-04-25T14:03:56.772486+0000","last_clean":"2024-04-25T14:03:56.772486+0000","last_became_active":"2024-04-25T14:03:50.463507+0000","last_became_peered":"2024-04-25T14:03:50.463507+0000","last_unstale":"2024-04-25T14:03:56.772486+0000","last_undegraded":"2024-04-25T14:03:56.772486+0000","last_fullsized":"2024-04-25T14:03:56.772486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:15.364599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494579+0000","last_change":"2024-04-25T14:03:55.676329+0000","last_active":"2024-04-25T14:03:56.494579+0000","last_peered":"2024-04-25T14:03:56.494579+0000","last_clean":"2024-04-25T14:03:56.494579+0000","last_became_active":"2024-04-25T14:03:55.675752+0000","last_became_peered":"2024-04-25T14:03:55.675752+0000","last_unstale":"2024-04-25T14:03:56.494579+0000","last_undegraded":"2024-04-25T14:03:56.494579+0000","last_fullsized":"2024-04-25T14:03:56.494579+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:28.167574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772572+0000","last_change":"2024-04-25T14:03:51.495282+0000","last_active":"2024-04-25T14:03:56.772572+0000","last_peered":"2024-04-25T14:03:56.772572+0000","last_clean":"2024-04-25T14:03:56.772572+0000","last_became_active":"2024-04-25T14:03:51.494965+0000","last_became_peered":"2024-04-25T14:03:51.494965+0000","last_unstale":"2024-04-25T14:03:56.772572+0000","last_undegraded":"2024-04-25T14:03:56.772572+0000","last_fullsized":"2024-04-25T14:03:56.772572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:02.773422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773963+0000","last_change":"2024-04-25T14:03:50.465925+0000","last_active":"2024-04-25T14:03:56.773963+0000","last_peered":"2024-04-25T14:03:56.773963+0000","last_clean":"2024-04-25T14:03:56.773963+0000","last_became_active":"2024-04-25T14:03:50.465495+0000","last_became_peered":"2024-04-25T14:03:50.465495+0000","last_unstale":"2024-04-25T14:03:56.773963+0000","last_undegraded":"2024-04-25T14:03:56.773963+0000","last_fullsized":"2024-04-25T14:03:56.773963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:15.520202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773183+0000","last_change":"2024-04-25T14:03:55.520108+0000","last_active":"2024-04-25T14:03:56.773183+0000","last_peered":"2024-04-25T14:03:56.773183+0000","last_clean":"2024-04-25T14:03:56.773183+0000","last_became_active":"2024-04-25T14:03:55.519374+0000","last_became_peered":"2024-04-25T14:03:55.519374+0000","last_unstale":"2024-04-25T14:03:56.773183+0000","last_undegraded":"2024-04-25T14:03:56.773183+0000","last_fullsized":"2024-04-25T14:03:56.773183+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:21.426226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772332+0000","last_change":"2024-04-25T14:03:51.488205+0000","last_active":"2024-04-25T14:03:56.772332+0000","last_peered":"2024-04-25T14:03:56.772332+0000","last_clean":"2024-04-25T14:03:56.772332+0000","last_became_active":"2024-04-25T14:03:51.487872+0000","last_became_peered":"2024-04-25T14:03:51.487872+0000","last_unstale":"2024-04-25T14:03:56.772332+0000","last_undegraded":"2024-04-25T14:03:56.772332+0000","last_fullsized":"2024-04-25T14:03:56.772332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:55.876775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.771697+0000","last_change":"2024-04-25T14:03:50.475566+0000","last_active":"2024-04-25T14:03:56.771697+0000","last_peered":"2024-04-25T14:03:56.771697+0000","last_clean":"2024-04-25T14:03:56.771697+0000","last_became_active":"2024-04-25T14:03:50.475304+0000","last_became_peered":"2024-04-25T14:03:50.475304+0000","last_unstale":"2024-04-25T14:03:56.771697+0000","last_undegraded":"2024-04-25T14:03:56.771697+0000","last_fullsized":"2024-04-25T14:03:56.771697+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:15.520202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774574+0000","last_change":"2024-04-25T14:03:55.523592+0000","last_active":"2024-04-25T14:03:56.774574+0000","last_peered":"2024-04-25T14:03:56.774574+0000","last_clean":"2024-04-25T14:03:56.774574+0000","last_became_active":"2024-04-25T14:03:55.521767+0000","last_became_peered":"2024-04-25T14:03:55.521767+0000","last_unstale":"2024-04-25T14:03:56.774574+0000","last_undegraded":"2024-04-25T14:03:56.774574+0000","last_fullsized":"2024-04-25T14:03:56.774574+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:50.234674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494779+0000","last_change":"2024-04-25T14:03:51.480142+0000","last_active":"2024-04-25T14:03:56.494779+0000","last_peered":"2024-04-25T14:03:56.494779+0000","last_clean":"2024-04-25T14:03:56.494779+0000","last_became_active":"2024-04-25T14:03:51.479798+0000","last_became_peered":"2024-04-25T14:03:51.479798+0000","last_unstale":"2024-04-25T14:03:56.494779+0000","last_undegraded":"2024-04-25T14:03:56.494779+0000","last_fullsized":"2024-04-25T14:03:56.494779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:31.840842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494679+0000","last_change":"2024-04-25T14:03:50.459894+0000","last_active":"2024-04-25T14:03:56.494679+0000","last_peered":"2024-04-25T14:03:56.494679+0000","last_clean":"2024-04-25T14:03:56.494679+0000","last_became_active":"2024-04-25T14:03:50.459580+0000","last_became_peered":"2024-04-25T14:03:50.459580+0000","last_unstale":"2024-04-25T14:03:56.494679+0000","last_undegraded":"2024-04-25T14:03:56.494679+0000","last_fullsized":"2024-04-25T14:03:56.494679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:29.264084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495170+0000","last_change":"2024-04-25T14:03:55.681298+0000","last_active":"2024-04-25T14:03:56.495170+0000","last_peered":"2024-04-25T14:03:56.495170+0000","last_clean":"2024-04-25T14:03:56.495170+0000","last_became_active":"2024-04-25T14:03:55.680746+0000","last_became_peered":"2024-04-25T14:03:55.680746+0000","last_unstale":"2024-04-25T14:03:56.495170+0000","last_undegraded":"2024-04-25T14:03:56.495170+0000","last_fullsized":"2024-04-25T14:03:56.495170+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:54.496413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772452+0000","last_change":"2024-04-25T14:03:51.499753+0000","last_active":"2024-04-25T14:03:56.772452+0000","last_peered":"2024-04-25T14:03:56.772452+0000","last_clean":"2024-04-25T14:03:56.772452+0000","last_became_active":"2024-04-25T14:03:51.499427+0000","last_became_peered":"2024-04-25T14:03:51.499427+0000","last_unstale":"2024-04-25T14:03:56.772452+0000","last_undegraded":"2024-04-25T14:03:56.772452+0000","last_fullsized":"2024-04-25T14:03:56.772452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:05.489889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495577+0000","last_change":"2024-04-25T14:03:50.467395+0000","last_active":"2024-04-25T14:03:56.495577+0000","last_peered":"2024-04-25T14:03:56.495577+0000","last_clean":"2024-04-25T14:03:56.495577+0000","last_became_active":"2024-04-25T14:03:50.465757+0000","last_became_peered":"2024-04-25T14:03:50.465757+0000","last_unstale":"2024-04-25T14:03:56.495577+0000","last_undegraded":"2024-04-25T14:03:56.495577+0000","last_fullsized":"2024-04-25T14:03:56.495577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:39.630327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494300+0000","last_change":"2024-04-25T14:03:55.676664+0000","last_active":"2024-04-25T14:03:56.494300+0000","last_peered":"2024-04-25T14:03:56.494300+0000","last_clean":"2024-04-25T14:03:56.494300+0000","last_became_active":"2024-04-25T14:03:55.674357+0000","last_became_peered":"2024-04-25T14:03:55.674357+0000","last_unstale":"2024-04-25T14:03:56.494300+0000","last_undegraded":"2024-04-25T14:03:56.494300+0000","last_fullsized":"2024-04-25T14:03:56.494300+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:55:42.758380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772783+0000","last_change":"2024-04-25T14:03:51.469041+0000","last_active":"2024-04-25T14:03:56.772783+0000","last_peered":"2024-04-25T14:03:56.772783+0000","last_clean":"2024-04-25T14:03:56.772783+0000","last_became_active":"2024-04-25T14:03:51.468704+0000","last_became_peered":"2024-04-25T14:03:51.468704+0000","last_unstale":"2024-04-25T14:03:56.772783+0000","last_undegraded":"2024-04-25T14:03:56.772783+0000","last_fullsized":"2024-04-25T14:03:56.772783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:31.840842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773670+0000","last_change":"2024-04-25T14:03:50.465211+0000","last_active":"2024-04-25T14:03:56.773670+0000","last_peered":"2024-04-25T14:03:56.773670+0000","last_clean":"2024-04-25T14:03:56.773670+0000","last_became_active":"2024-04-25T14:03:50.464836+0000","last_became_peered":"2024-04-25T14:03:50.464836+0000","last_unstale":"2024-04-25T14:03:56.773670+0000","last_undegraded":"2024-04-25T14:03:56.773670+0000","last_fullsized":"2024-04-25T14:03:56.773670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:04.876207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496500+0000","last_change":"2024-04-25T14:03:55.681339+0000","last_active":"2024-04-25T14:03:56.496500+0000","last_peered":"2024-04-25T14:03:56.496500+0000","last_clean":"2024-04-25T14:03:56.496500+0000","last_became_active":"2024-04-25T14:03:55.680779+0000","last_became_peered":"2024-04-25T14:03:55.680779+0000","last_unstale":"2024-04-25T14:03:56.496500+0000","last_undegraded":"2024-04-25T14:03:56.496500+0000","last_fullsized":"2024-04-25T14:03:56.496500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:58.844877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772688+0000","last_change":"2024-04-25T14:03:51.476195+0000","last_active":"2024-04-25T14:03:56.772688+0000","last_peered":"2024-04-25T14:03:56.772688+0000","last_clean":"2024-04-25T14:03:56.772688+0000","last_became_active":"2024-04-25T14:03:51.475372+0000","last_became_peered":"2024-04-25T14:03:51.475372+0000","last_unstale":"2024-04-25T14:03:56.772688+0000","last_undegraded":"2024-04-25T14:03:56.772688+0000","last_fullsized":"2024-04-25T14:03:56.772688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:47.326356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:04:06.748698+0000","last_change":"2024-04-25T14:03:50.470612+0000","last_active":"2024-04-25T14:04:06.748698+0000","last_peered":"2024-04-25T14:04:06.748698+0000","last_clean":"2024-04-25T14:04:06.748698+0000","last_became_active":"2024-04-25T14:03:50.470312+0000","last_became_peered":"2024-04-25T14:03:50.470312+0000","last_unstale":"2024-04-25T14:04:06.748698+0000","last_undegraded":"2024-04-25T14:04:06.748698+0000","last_fullsized":"2024-04-25T14:04:06.748698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:10.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773458+0000","last_change":"2024-04-25T14:03:55.526774+0000","last_active":"2024-04-25T14:03:56.773458+0000","last_peered":"2024-04-25T14:03:56.773458+0000","last_clean":"2024-04-25T14:03:56.773458+0000","last_became_active":"2024-04-25T14:03:55.524255+0000","last_became_peered":"2024-04-25T14:03:55.524255+0000","last_unstale":"2024-04-25T14:03:56.773458+0000","last_undegraded":"2024-04-25T14:03:56.773458+0000","last_fullsized":"2024-04-25T14:03:56.773458+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:37.811376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773340+0000","last_change":"2024-04-25T14:03:51.493162+0000","last_active":"2024-04-25T14:03:56.773340+0000","last_peered":"2024-04-25T14:03:56.773340+0000","last_clean":"2024-04-25T14:03:56.773340+0000","last_became_active":"2024-04-25T14:03:51.488560+0000","last_became_peered":"2024-04-25T14:03:51.488560+0000","last_unstale":"2024-04-25T14:03:56.773340+0000","last_undegraded":"2024-04-25T14:03:56.773340+0000","last_fullsized":"2024-04-25T14:03:56.773340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:40.635789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774009+0000","last_change":"2024-04-25T14:03:50.469594+0000","last_active":"2024-04-25T14:03:56.774009+0000","last_peered":"2024-04-25T14:03:56.774009+0000","last_clean":"2024-04-25T14:03:56.774009+0000","last_became_active":"2024-04-25T14:03:50.469196+0000","last_became_peered":"2024-04-25T14:03:50.469196+0000","last_unstale":"2024-04-25T14:03:56.774009+0000","last_undegraded":"2024-04-25T14:03:56.774009+0000","last_fullsized":"2024-04-25T14:03:56.774009+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:15.364599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772969+0000","last_change":"2024-04-25T14:03:55.518510+0000","last_active":"2024-04-25T14:03:56.772969+0000","last_peered":"2024-04-25T14:03:56.772969+0000","last_clean":"2024-04-25T14:03:56.772969+0000","last_became_active":"2024-04-25T14:03:55.515796+0000","last_became_peered":"2024-04-25T14:03:55.515796+0000","last_unstale":"2024-04-25T14:03:56.772969+0000","last_undegraded":"2024-04-25T14:03:56.772969+0000","last_fullsized":"2024-04-25T14:03:56.772969+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:59.921752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774672+0000","last_change":"2024-04-25T14:03:51.486474+0000","last_active":"2024-04-25T14:03:56.774672+0000","last_peered":"2024-04-25T14:03:56.774672+0000","last_clean":"2024-04-25T14:03:56.774672+0000","last_became_active":"2024-04-25T14:03:51.485742+0000","last_became_peered":"2024-04-25T14:03:51.485742+0000","last_unstale":"2024-04-25T14:03:56.774672+0000","last_undegraded":"2024-04-25T14:03:56.774672+0000","last_fullsized":"2024-04-25T14:03:56.774672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:16.813529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496428+0000","last_change":"2024-04-25T14:03:50.475187+0000","last_active":"2024-04-25T14:03:56.496428+0000","last_peered":"2024-04-25T14:03:56.496428+0000","last_clean":"2024-04-25T14:03:56.496428+0000","last_became_active":"2024-04-25T14:03:50.474761+0000","last_became_peered":"2024-04-25T14:03:50.474761+0000","last_unstale":"2024-04-25T14:03:56.496428+0000","last_undegraded":"2024-04-25T14:03:56.496428+0000","last_fullsized":"2024-04-25T14:03:56.496428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:02.117945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493593+0000","last_change":"2024-04-25T14:03:55.524048+0000","last_active":"2024-04-25T14:03:56.493593+0000","last_peered":"2024-04-25T14:03:56.493593+0000","last_clean":"2024-04-25T14:03:56.493593+0000","last_became_active":"2024-04-25T14:03:55.520419+0000","last_became_peered":"2024-04-25T14:03:55.520419+0000","last_unstale":"2024-04-25T14:03:56.493593+0000","last_undegraded":"2024-04-25T14:03:56.493593+0000","last_fullsized":"2024-04-25T14:03:56.493593+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:30.069597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496533+0000","last_change":"2024-04-25T14:03:51.467640+0000","last_active":"2024-04-25T14:03:56.496533+0000","last_peered":"2024-04-25T14:03:56.496533+0000","last_clean":"2024-04-25T14:03:56.496533+0000","last_became_active":"2024-04-25T14:03:51.467347+0000","last_became_peered":"2024-04-25T14:03:51.467347+0000","last_unstale":"2024-04-25T14:03:56.496533+0000","last_undegraded":"2024-04-25T14:03:56.496533+0000","last_fullsized":"2024-04-25T14:03:56.496533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:33.766399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496540+0000","last_change":"2024-04-25T14:03:50.462889+0000","last_active":"2024-04-25T14:03:56.496540+0000","last_peered":"2024-04-25T14:03:56.496540+0000","last_clean":"2024-04-25T14:03:56.496540+0000","last_became_active":"2024-04-25T14:03:50.462529+0000","last_became_peered":"2024-04-25T14:03:50.462529+0000","last_unstale":"2024-04-25T14:03:56.496540+0000","last_undegraded":"2024-04-25T14:03:56.496540+0000","last_fullsized":"2024-04-25T14:03:56.496540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:15.364599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493822+0000","last_change":"2024-04-25T14:03:55.675236+0000","last_active":"2024-04-25T14:03:56.493822+0000","last_peered":"2024-04-25T14:03:56.493822+0000","last_clean":"2024-04-25T14:03:56.493822+0000","last_became_active":"2024-04-25T14:03:55.674759+0000","last_became_peered":"2024-04-25T14:03:55.674759+0000","last_unstale":"2024-04-25T14:03:56.493822+0000","last_undegraded":"2024-04-25T14:03:56.493822+0000","last_fullsized":"2024-04-25T14:03:56.493822+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:28.167574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773362+0000","last_change":"2024-04-25T14:03:55.677958+0000","last_active":"2024-04-25T14:03:56.773362+0000","last_peered":"2024-04-25T14:03:56.773362+0000","last_clean":"2024-04-25T14:03:56.773362+0000","last_became_active":"2024-04-25T14:03:55.677471+0000","last_became_peered":"2024-04-25T14:03:55.677471+0000","last_unstale":"2024-04-25T14:03:56.773362+0000","last_undegraded":"2024-04-25T14:03:56.773362+0000","last_fullsized":"2024-04-25T14:03:56.773362+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:12.890323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772866+0000","last_change":"2024-04-25T14:03:51.477601+0000","last_active":"2024-04-25T14:03:56.772866+0000","last_peered":"2024-04-25T14:03:56.772866+0000","last_clean":"2024-04-25T14:03:56.772866+0000","last_became_active":"2024-04-25T14:03:51.477318+0000","last_became_peered":"2024-04-25T14:03:51.477318+0000","last_unstale":"2024-04-25T14:03:56.772866+0000","last_undegraded":"2024-04-25T14:03:56.772866+0000","last_fullsized":"2024-04-25T14:03:56.772866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.599283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494934+0000","last_change":"2024-04-25T14:03:50.456493+0000","last_active":"2024-04-25T14:03:56.494934+0000","last_peered":"2024-04-25T14:03:56.494934+0000","last_clean":"2024-04-25T14:03:56.494934+0000","last_became_active":"2024-04-25T14:03:50.454242+0000","last_became_peered":"2024-04-25T14:03:50.454242+0000","last_unstale":"2024-04-25T14:03:56.494934+0000","last_undegraded":"2024-04-25T14:03:56.494934+0000","last_fullsized":"2024-04-25T14:03:56.494934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:10.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496401+0000","last_change":"2024-04-25T14:03:55.516548+0000","last_active":"2024-04-25T14:03:56.496401+0000","last_peered":"2024-04-25T14:03:56.496401+0000","last_clean":"2024-04-25T14:03:56.496401+0000","last_became_active":"2024-04-25T14:03:55.515311+0000","last_became_peered":"2024-04-25T14:03:55.515311+0000","last_unstale":"2024-04-25T14:03:56.496401+0000","last_undegraded":"2024-04-25T14:03:56.496401+0000","last_fullsized":"2024-04-25T14:03:56.496401+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:51.371333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493692+0000","last_change":"2024-04-25T14:03:51.468740+0000","last_active":"2024-04-25T14:03:56.493692+0000","last_peered":"2024-04-25T14:03:56.493692+0000","last_clean":"2024-04-25T14:03:56.493692+0000","last_became_active":"2024-04-25T14:03:51.468366+0000","last_became_peered":"2024-04-25T14:03:51.468366+0000","last_unstale":"2024-04-25T14:03:56.493692+0000","last_undegraded":"2024-04-25T14:03:56.493692+0000","last_fullsized":"2024-04-25T14:03:56.493692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:31.707048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493992+0000","last_change":"2024-04-25T14:03:50.466457+0000","last_active":"2024-04-25T14:03:56.493992+0000","last_peered":"2024-04-25T14:03:56.493992+0000","last_clean":"2024-04-25T14:03:56.493992+0000","last_became_active":"2024-04-25T14:03:50.466070+0000","last_became_peered":"2024-04-25T14:03:50.466070+0000","last_unstale":"2024-04-25T14:03:56.493992+0000","last_undegraded":"2024-04-25T14:03:56.493992+0000","last_fullsized":"2024-04-25T14:03:56.493992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:15.520202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495294+0000","last_change":"2024-04-25T14:03:50.469128+0000","last_active":"2024-04-25T14:03:56.495294+0000","last_peered":"2024-04-25T14:03:56.495294+0000","last_clean":"2024-04-25T14:03:56.495294+0000","last_became_active":"2024-04-25T14:03:50.468409+0000","last_became_peered":"2024-04-25T14:03:50.468409+0000","last_unstale":"2024-04-25T14:03:56.495294+0000","last_undegraded":"2024-04-25T14:03:56.495294+0000","last_fullsized":"2024-04-25T14:03:56.495294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:15.520202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773570+0000","last_change":"2024-04-25T14:03:51.502539+0000","last_active":"2024-04-25T14:03:56.773570+0000","last_peered":"2024-04-25T14:03:56.773570+0000","last_clean":"2024-04-25T14:03:56.773570+0000","last_became_active":"2024-04-25T14:03:51.502229+0000","last_became_peered":"2024-04-25T14:03:51.502229+0000","last_unstale":"2024-04-25T14:03:56.773570+0000","last_undegraded":"2024-04-25T14:03:56.773570+0000","last_fullsized":"2024-04-25T14:03:56.773570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:20.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772371+0000","last_change":"2024-04-25T14:03:55.524446+0000","last_active":"2024-04-25T14:03:56.772371+0000","last_peered":"2024-04-25T14:03:56.772371+0000","last_clean":"2024-04-25T14:03:56.772371+0000","last_became_active":"2024-04-25T14:03:55.523886+0000","last_became_peered":"2024-04-25T14:03:55.523886+0000","last_unstale":"2024-04-25T14:03:56.772371+0000","last_undegraded":"2024-04-25T14:03:56.772371+0000","last_fullsized":"2024-04-25T14:03:56.772371+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:37.811235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493654+0000","last_change":"2024-04-25T14:03:50.472564+0000","last_active":"2024-04-25T14:03:56.493654+0000","last_peered":"2024-04-25T14:03:56.493654+0000","last_clean":"2024-04-25T14:03:56.493654+0000","last_became_active":"2024-04-25T14:03:50.472297+0000","last_became_peered":"2024-04-25T14:03:50.472297+0000","last_unstale":"2024-04-25T14:03:56.493654+0000","last_undegraded":"2024-04-25T14:03:56.493654+0000","last_fullsized":"2024-04-25T14:03:56.493654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:43.281768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772419+0000","last_change":"2024-04-25T14:03:51.486031+0000","last_active":"2024-04-25T14:03:56.772419+0000","last_peered":"2024-04-25T14:03:56.772419+0000","last_clean":"2024-04-25T14:03:56.772419+0000","last_became_active":"2024-04-25T14:03:51.477708+0000","last_became_peered":"2024-04-25T14:03:51.477708+0000","last_unstale":"2024-04-25T14:03:56.772419+0000","last_undegraded":"2024-04-25T14:03:56.772419+0000","last_fullsized":"2024-04-25T14:03:56.772419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:11.859288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494673+0000","last_change":"2024-04-25T14:03:55.519123+0000","last_active":"2024-04-25T14:03:56.494673+0000","last_peered":"2024-04-25T14:03:56.494673+0000","last_clean":"2024-04-25T14:03:56.494673+0000","last_became_active":"2024-04-25T14:03:55.518743+0000","last_became_peered":"2024-04-25T14:03:55.518743+0000","last_unstale":"2024-04-25T14:03:56.494673+0000","last_undegraded":"2024-04-25T14:03:56.494673+0000","last_fullsized":"2024-04-25T14:03:56.494673+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:18.368759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494540+0000","last_change":"2024-04-25T14:03:51.480209+0000","last_active":"2024-04-25T14:03:56.494540+0000","last_peered":"2024-04-25T14:03:56.494540+0000","last_clean":"2024-04-25T14:03:56.494540+0000","last_became_active":"2024-04-25T14:03:51.479824+0000","last_became_peered":"2024-04-25T14:03:51.479824+0000","last_unstale":"2024-04-25T14:03:56.494540+0000","last_undegraded":"2024-04-25T14:03:56.494540+0000","last_fullsized":"2024-04-25T14:03:56.494540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:31.707048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.771783+0000","last_change":"2024-04-25T14:03:51.486748+0000","last_active":"2024-04-25T14:03:56.771783+0000","last_peered":"2024-04-25T14:03:56.771783+0000","last_clean":"2024-04-25T14:03:56.771783+0000","last_became_active":"2024-04-25T14:03:51.486343+0000","last_became_peered":"2024-04-25T14:03:51.486343+0000","last_unstale":"2024-04-25T14:03:56.771783+0000","last_undegraded":"2024-04-25T14:03:56.771783+0000","last_fullsized":"2024-04-25T14:03:56.771783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:46.086179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495470+0000","last_change":"2024-04-25T14:03:51.465637+0000","last_active":"2024-04-25T14:03:56.495470+0000","last_peered":"2024-04-25T14:03:56.495470+0000","last_clean":"2024-04-25T14:03:56.495470+0000","last_became_active":"2024-04-25T14:03:51.465339+0000","last_became_peered":"2024-04-25T14:03:51.465339+0000","last_unstale":"2024-04-25T14:03:56.495470+0000","last_undegraded":"2024-04-25T14:03:56.495470+0000","last_fullsized":"2024-04-25T14:03:56.495470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:21.406920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773508+0000","last_change":"2024-04-25T14:03:51.475134+0000","last_active":"2024-04-25T14:03:56.773508+0000","last_peered":"2024-04-25T14:03:56.773508+0000","last_clean":"2024-04-25T14:03:56.773508+0000","last_became_active":"2024-04-25T14:03:51.474796+0000","last_became_peered":"2024-04-25T14:03:51.474796+0000","last_unstale":"2024-04-25T14:03:56.773508+0000","last_undegraded":"2024-04-25T14:03:56.773508+0000","last_fullsized":"2024-04-25T14:03:56.773508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:31.707048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494247+0000","last_change":"2024-04-25T14:03:51.494845+0000","last_active":"2024-04-25T14:03:56.494247+0000","last_peered":"2024-04-25T14:03:56.494247+0000","last_clean":"2024-04-25T14:03:56.494247+0000","last_became_active":"2024-04-25T14:03:51.494547+0000","last_became_peered":"2024-04-25T14:03:51.494547+0000","last_unstale":"2024-04-25T14:03:56.494247+0000","last_undegraded":"2024-04-25T14:03:56.494247+0000","last_fullsized":"2024-04-25T14:03:56.494247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:16.813529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773722+0000","last_change":"2024-04-25T14:03:51.478241+0000","last_active":"2024-04-25T14:03:56.773722+0000","last_peered":"2024-04-25T14:03:56.773722+0000","last_clean":"2024-04-25T14:03:56.773722+0000","last_became_active":"2024-04-25T14:03:51.477955+0000","last_became_peered":"2024-04-25T14:03:51.477955+0000","last_unstale":"2024-04-25T14:03:56.773722+0000","last_undegraded":"2024-04-25T14:03:56.773722+0000","last_fullsized":"2024-04-25T14:03:56.773722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:43.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496093+0000","last_change":"2024-04-25T14:03:51.478644+0000","last_active":"2024-04-25T14:03:56.496093+0000","last_peered":"2024-04-25T14:03:56.496093+0000","last_clean":"2024-04-25T14:03:56.496093+0000","last_became_active":"2024-04-25T14:03:51.478326+0000","last_became_peered":"2024-04-25T14:03:51.478326+0000","last_unstale":"2024-04-25T14:03:56.496093+0000","last_undegraded":"2024-04-25T14:03:56.496093+0000","last_fullsized":"2024-04-25T14:03:56.496093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:20.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495116+0000","last_change":"2024-04-25T14:03:51.506061+0000","last_active":"2024-04-25T14:03:56.495116+0000","last_peered":"2024-04-25T14:03:56.495116+0000","last_clean":"2024-04-25T14:03:56.495116+0000","last_became_active":"2024-04-25T14:03:51.505804+0000","last_became_peered":"2024-04-25T14:03:51.505804+0000","last_unstale":"2024-04-25T14:03:56.495116+0000","last_undegraded":"2024-04-25T14:03:56.495116+0000","last_fullsized":"2024-04-25T14:03:56.495116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:55.876775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494831+0000","last_change":"2024-04-25T14:03:51.465574+0000","last_active":"2024-04-25T14:03:56.494831+0000","last_peered":"2024-04-25T14:03:56.494831+0000","last_clean":"2024-04-25T14:03:56.494831+0000","last_became_active":"2024-04-25T14:03:51.465206+0000","last_became_peered":"2024-04-25T14:03:51.465206+0000","last_unstale":"2024-04-25T14:03:56.494831+0000","last_undegraded":"2024-04-25T14:03:56.494831+0000","last_fullsized":"2024-04-25T14:03:56.494831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:16.813529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493905+0000","last_change":"2024-04-25T14:03:51.486782+0000","last_active":"2024-04-25T14:03:56.493905+0000","last_peered":"2024-04-25T14:03:56.493905+0000","last_clean":"2024-04-25T14:03:56.493905+0000","last_became_active":"2024-04-25T14:03:51.486465+0000","last_became_peered":"2024-04-25T14:03:51.486465+0000","last_unstale":"2024-04-25T14:03:56.493905+0000","last_undegraded":"2024-04-25T14:03:56.493905+0000","last_fullsized":"2024-04-25T14:03:56.493905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:39.949870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495357+0000","last_change":"2024-04-25T14:03:51.468022+0000","last_active":"2024-04-25T14:03:56.495357+0000","last_peered":"2024-04-25T14:03:56.495357+0000","last_clean":"2024-04-25T14:03:56.495357+0000","last_became_active":"2024-04-25T14:03:51.467637+0000","last_became_peered":"2024-04-25T14:03:51.467637+0000","last_unstale":"2024-04-25T14:03:56.495357+0000","last_undegraded":"2024-04-25T14:03:56.495357+0000","last_fullsized":"2024-04-25T14:03:56.495357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:55.876775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773216+0000","last_change":"2024-04-25T14:03:51.469881+0000","last_active":"2024-04-25T14:03:56.773216+0000","last_peered":"2024-04-25T14:03:56.773216+0000","last_clean":"2024-04-25T14:03:56.773216+0000","last_became_active":"2024-04-25T14:03:51.469572+0000","last_became_peered":"2024-04-25T14:03:51.469572+0000","last_unstale":"2024-04-25T14:03:56.773216+0000","last_undegraded":"2024-04-25T14:03:56.773216+0000","last_fullsized":"2024-04-25T14:03:56.773216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.599283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495680+0000","last_change":"2024-04-25T14:03:51.494605+0000","last_active":"2024-04-25T14:03:56.495680+0000","last_peered":"2024-04-25T14:03:56.495680+0000","last_clean":"2024-04-25T14:03:56.495680+0000","last_became_active":"2024-04-25T14:03:51.494257+0000","last_became_peered":"2024-04-25T14:03:51.494257+0000","last_unstale":"2024-04-25T14:03:56.495680+0000","last_undegraded":"2024-04-25T14:03:56.495680+0000","last_fullsized":"2024-04-25T14:03:56.495680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:39.949870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773065+0000","last_change":"2024-04-25T14:03:51.476473+0000","last_active":"2024-04-25T14:03:56.773065+0000","last_peered":"2024-04-25T14:03:56.773065+0000","last_clean":"2024-04-25T14:03:56.773065+0000","last_became_active":"2024-04-25T14:03:51.475857+0000","last_became_peered":"2024-04-25T14:03:51.475857+0000","last_unstale":"2024-04-25T14:03:56.773065+0000","last_undegraded":"2024-04-25T14:03:56.773065+0000","last_fullsized":"2024-04-25T14:03:56.773065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:46.189697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772375+0000","last_change":"2024-04-25T14:03:51.479889+0000","last_active":"2024-04-25T14:03:56.772375+0000","last_peered":"2024-04-25T14:03:56.772375+0000","last_clean":"2024-04-25T14:03:56.772375+0000","last_became_active":"2024-04-25T14:03:51.478793+0000","last_became_peered":"2024-04-25T14:03:51.478793+0000","last_unstale":"2024-04-25T14:03:56.772375+0000","last_undegraded":"2024-04-25T14:03:56.772375+0000","last_fullsized":"2024-04-25T14:03:56.772375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:38.411753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496637+0000","last_change":"2024-04-25T14:03:51.480313+0000","last_active":"2024-04-25T14:03:56.496637+0000","last_peered":"2024-04-25T14:03:56.496637+0000","last_clean":"2024-04-25T14:03:56.496637+0000","last_became_active":"2024-04-25T14:03:51.479875+0000","last_became_peered":"2024-04-25T14:03:51.479875+0000","last_unstale":"2024-04-25T14:03:56.496637+0000","last_undegraded":"2024-04-25T14:03:56.496637+0000","last_fullsized":"2024-04-25T14:03:56.496637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:02.773422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494729+0000","last_change":"2024-04-25T14:03:51.472514+0000","last_active":"2024-04-25T14:03:56.494729+0000","last_peered":"2024-04-25T14:03:56.494729+0000","last_clean":"2024-04-25T14:03:56.494729+0000","last_became_active":"2024-04-25T14:03:51.472148+0000","last_became_peered":"2024-04-25T14:03:51.472148+0000","last_unstale":"2024-04-25T14:03:56.494729+0000","last_undegraded":"2024-04-25T14:03:56.494729+0000","last_fullsized":"2024-04-25T14:03:56.494729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:20.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772899+0000","last_change":"2024-04-25T14:03:51.489086+0000","last_active":"2024-04-25T14:03:56.772899+0000","last_peered":"2024-04-25T14:03:56.772899+0000","last_clean":"2024-04-25T14:03:56.772899+0000","last_became_active":"2024-04-25T14:03:51.488810+0000","last_became_peered":"2024-04-25T14:03:51.488810+0000","last_unstale":"2024-04-25T14:03:56.772899+0000","last_undegraded":"2024-04-25T14:03:56.772899+0000","last_fullsized":"2024-04-25T14:03:56.772899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:39.949870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772674+0000","last_change":"2024-04-25T14:03:51.475508+0000","last_active":"2024-04-25T14:03:56.772674+0000","last_peered":"2024-04-25T14:03:56.772674+0000","last_clean":"2024-04-25T14:03:56.772674+0000","last_became_active":"2024-04-25T14:03:51.475146+0000","last_became_peered":"2024-04-25T14:03:51.475146+0000","last_unstale":"2024-04-25T14:03:56.772674+0000","last_undegraded":"2024-04-25T14:03:56.772674+0000","last_fullsized":"2024-04-25T14:03:56.772674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:34.374126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493554+0000","last_change":"2024-04-25T14:03:51.479243+0000","last_active":"2024-04-25T14:03:56.493554+0000","last_peered":"2024-04-25T14:03:56.493554+0000","last_clean":"2024-04-25T14:03:56.493554+0000","last_became_active":"2024-04-25T14:03:51.478899+0000","last_became_peered":"2024-04-25T14:03:51.478899+0000","last_unstale":"2024-04-25T14:03:56.493554+0000","last_undegraded":"2024-04-25T14:03:56.493554+0000","last_fullsized":"2024-04-25T14:03:56.493554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:20.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773258+0000","last_change":"2024-04-25T14:03:51.493878+0000","last_active":"2024-04-25T14:03:56.773258+0000","last_peered":"2024-04-25T14:03:56.773258+0000","last_clean":"2024-04-25T14:03:56.773258+0000","last_became_active":"2024-04-25T14:03:51.493501+0000","last_became_peered":"2024-04-25T14:03:51.493501+0000","last_unstale":"2024-04-25T14:03:56.773258+0000","last_undegraded":"2024-04-25T14:03:56.773258+0000","last_fullsized":"2024-04-25T14:03:56.773258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:46.086179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496241+0000","last_change":"2024-04-25T14:03:51.489072+0000","last_active":"2024-04-25T14:03:56.496241+0000","last_peered":"2024-04-25T14:03:56.496241+0000","last_clean":"2024-04-25T14:03:56.496241+0000","last_became_active":"2024-04-25T14:03:51.488895+0000","last_became_peered":"2024-04-25T14:03:51.488895+0000","last_unstale":"2024-04-25T14:03:56.496241+0000","last_undegraded":"2024-04-25T14:03:56.496241+0000","last_fullsized":"2024-04-25T14:03:56.496241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:10.318378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496156+0000","last_change":"2024-04-25T14:03:51.488196+0000","last_active":"2024-04-25T14:03:56.496156+0000","last_peered":"2024-04-25T14:03:56.496156+0000","last_clean":"2024-04-25T14:03:56.496156+0000","last_became_active":"2024-04-25T14:03:51.487833+0000","last_became_peered":"2024-04-25T14:03:51.487833+0000","last_unstale":"2024-04-25T14:03:56.496156+0000","last_undegraded":"2024-04-25T14:03:56.496156+0000","last_fullsized":"2024-04-25T14:03:56.496156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:43.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773432+0000","last_change":"2024-04-25T14:03:51.482441+0000","last_active":"2024-04-25T14:03:56.773432+0000","last_peered":"2024-04-25T14:03:56.773432+0000","last_clean":"2024-04-25T14:03:56.773432+0000","last_became_active":"2024-04-25T14:03:51.482089+0000","last_became_peered":"2024-04-25T14:03:51.482089+0000","last_unstale":"2024-04-25T14:03:56.773432+0000","last_undegraded":"2024-04-25T14:03:56.773432+0000","last_fullsized":"2024-04-25T14:03:56.773432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:02.697952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493326+0000","last_change":"2024-04-25T14:03:51.487289+0000","last_active":"2024-04-25T14:03:56.493326+0000","last_peered":"2024-04-25T14:03:56.493326+0000","last_clean":"2024-04-25T14:03:56.493326+0000","last_became_active":"2024-04-25T14:03:51.486948+0000","last_became_peered":"2024-04-25T14:03:51.486948+0000","last_unstale":"2024-04-25T14:03:56.493326+0000","last_undegraded":"2024-04-25T14:03:56.493326+0000","last_fullsized":"2024-04-25T14:03:56.493326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.599283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772227+0000","last_change":"2024-04-25T14:03:51.473228+0000","last_active":"2024-04-25T14:03:56.772227+0000","last_peered":"2024-04-25T14:03:56.772227+0000","last_clean":"2024-04-25T14:03:56.772227+0000","last_became_active":"2024-04-25T14:03:51.472850+0000","last_became_peered":"2024-04-25T14:03:51.472850+0000","last_unstale":"2024-04-25T14:03:56.772227+0000","last_undegraded":"2024-04-25T14:03:56.772227+0000","last_fullsized":"2024-04-25T14:03:56.772227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:34.707119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495222+0000","last_change":"2024-04-25T14:03:51.467162+0000","last_active":"2024-04-25T14:03:56.495222+0000","last_peered":"2024-04-25T14:03:56.495222+0000","last_clean":"2024-04-25T14:03:56.495222+0000","last_became_active":"2024-04-25T14:03:51.466616+0000","last_became_peered":"2024-04-25T14:03:51.466616+0000","last_unstale":"2024-04-25T14:03:56.495222+0000","last_undegraded":"2024-04-25T14:03:56.495222+0000","last_fullsized":"2024-04-25T14:03:56.495222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:34.707119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772199+0000","last_change":"2024-04-25T14:03:51.476149+0000","last_active":"2024-04-25T14:03:56.772199+0000","last_peered":"2024-04-25T14:03:56.772199+0000","last_clean":"2024-04-25T14:03:56.772199+0000","last_became_active":"2024-04-25T14:03:51.475934+0000","last_became_peered":"2024-04-25T14:03:51.475934+0000","last_unstale":"2024-04-25T14:03:56.772199+0000","last_undegraded":"2024-04-25T14:03:56.772199+0000","last_fullsized":"2024-04-25T14:03:56.772199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:17.381249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496219+0000","last_change":"2024-04-25T14:03:51.475153+0000","last_active":"2024-04-25T14:03:56.496219+0000","last_peered":"2024-04-25T14:03:56.496219+0000","last_clean":"2024-04-25T14:03:56.496219+0000","last_became_active":"2024-04-25T14:03:51.474862+0000","last_became_peered":"2024-04-25T14:03:51.474862+0000","last_unstale":"2024-04-25T14:03:56.496219+0000","last_undegraded":"2024-04-25T14:03:56.496219+0000","last_fullsized":"2024-04-25T14:03:56.496219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:05.489889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773218+0000","last_change":"2024-04-25T14:03:51.460567+0000","last_active":"2024-04-25T14:03:56.773218+0000","last_peered":"2024-04-25T14:03:56.773218+0000","last_clean":"2024-04-25T14:03:56.773218+0000","last_became_active":"2024-04-25T14:03:51.460193+0000","last_became_peered":"2024-04-25T14:03:51.460193+0000","last_unstale":"2024-04-25T14:03:56.773218+0000","last_undegraded":"2024-04-25T14:03:56.773218+0000","last_fullsized":"2024-04-25T14:03:56.773218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:39.949870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774153+0000","last_change":"2024-04-25T14:03:51.480094+0000","last_active":"2024-04-25T14:03:56.774153+0000","last_peered":"2024-04-25T14:03:56.774153+0000","last_clean":"2024-04-25T14:03:56.774153+0000","last_became_active":"2024-04-25T14:03:51.479529+0000","last_became_peered":"2024-04-25T14:03:51.479529+0000","last_unstale":"2024-04-25T14:03:56.774153+0000","last_undegraded":"2024-04-25T14:03:56.774153+0000","last_fullsized":"2024-04-25T14:03:56.774153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:10.318378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773366+0000","last_change":"2024-04-25T14:03:51.481397+0000","last_active":"2024-04-25T14:03:56.773366+0000","last_peered":"2024-04-25T14:03:56.773366+0000","last_clean":"2024-04-25T14:03:56.773366+0000","last_became_active":"2024-04-25T14:03:51.481153+0000","last_became_peered":"2024-04-25T14:03:51.481153+0000","last_unstale":"2024-04-25T14:03:56.773366+0000","last_undegraded":"2024-04-25T14:03:56.773366+0000","last_fullsized":"2024-04-25T14:03:56.773366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:21.406920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:04:10.931 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:04:11.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.113+0000 7f58a1200640 1 Processor -- start 2024-04-25T14:04:11.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.113+0000 7f58a1200640 1 -- start start 2024-04-25T14:04:11.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.113+0000 7f58a1200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f589c0b6ee0 0x7f589c0d06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:11.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.113+0000 7f58a1200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f589c0d0c00 0x7f589c0722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:11.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.113+0000 7f58a1200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f589c072bf0 con 0x7f589c0727e0 2024-04-25T14:04:11.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.113+0000 7f58a1200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f589c15df00 con 0x7f589c0b6ee0 2024-04-25T14:04:11.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.113+0000 7f58a1200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f589c15e1e0 con 0x7f589c0d0c00 2024-04-25T14:04:11.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.113+0000 7f589b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f589c0b6ee0 0x7f589c0d06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:11.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.113+0000 7f589be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f589c0727e0 0x7f589c1614c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33348/0 (socket says 172.21.15.86:33348) 2024-04-25T14:04:11.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.113+0000 7f589be00640 1 -- 172.21.15.86:0/3654027770 learned_addr learned my addr 172.21.15.86:0/3654027770 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:11.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.113+0000 7f589aa00640 1 --2- 172.21.15.86:0/3654027770 >> v2:172.21.15.155:3300/0 conn(0x7f589c0d0c00 0x7f589c0722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:11.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.113+0000 7f589b400640 1 -- 172.21.15.86:0/3654027770 >> v2:172.21.15.155:3300/0 conn(0x7f589c0d0c00 msgr2=0x7f589c0722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:11.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.113+0000 7f589b400640 1 --2- 172.21.15.86:0/3654027770 >> v2:172.21.15.155:3300/0 conn(0x7f589c0d0c00 0x7f589c0722a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:11.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.113+0000 7f589b400640 1 -- 172.21.15.86:0/3654027770 >> v1:172.21.15.86:6789/0 conn(0x7f589c0727e0 legacy=0x7f589c1614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:11.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.113+0000 7f589b400640 1 -- 172.21.15.86:0/3654027770 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f589c15e4c0 con 0x7f589c0b6ee0 2024-04-25T14:04:11.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.113+0000 7f589aa00640 1 --2- 172.21.15.86:0/3654027770 >> v2:172.21.15.155:3300/0 conn(0x7f589c0d0c00 0x7f589c0722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:11.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.113+0000 7f589a000640 1 -- 172.21.15.86:0/3654027770 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3238837391 0 0) 0x7f58940014a0 con 0x7f589c0727e0 2024-04-25T14:04:11.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.117+0000 7f589b400640 1 --2- 172.21.15.86:0/3654027770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f589c0b6ee0 0x7f589c0d06c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f589c0d1e40 tx=0x7f588c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b57da0d1bfbb72b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:11.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.117+0000 7f589a000640 1 -- 172.21.15.86:0/3654027770 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c0b0070 con 0x7f589c0b6ee0 2024-04-25T14:04:11.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.117+0000 7f589a000640 1 -- 172.21.15.86:0/3654027770 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f588c0ab600 con 0x7f589c0b6ee0 2024-04-25T14:04:11.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.117+0000 7f589a000640 1 -- 172.21.15.86:0/3654027770 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c0ab9d0 con 0x7f589c0b6ee0 2024-04-25T14:04:11.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.117+0000 7f58a1200640 1 -- 172.21.15.86:0/3654027770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f589c0b6ee0 msgr2=0x7f589c0d06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:11.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.117+0000 7f58a1200640 1 --2- 172.21.15.86:0/3654027770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f589c0b6ee0 0x7f589c0d06c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f589c0d1e40 tx=0x7f588c01c3f0 comp rx=0 tx=0).stop 2024-04-25T14:04:11.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.117+0000 7f58a1200640 1 -- 172.21.15.86:0/3654027770 shutdown_connections 2024-04-25T14:04:11.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.117+0000 7f58a1200640 1 --2- 172.21.15.86:0/3654027770 >> v2:172.21.15.155:3300/0 conn(0x7f589c0d0c00 0x7f589c0722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:11.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.117+0000 7f58a1200640 1 --2- 172.21.15.86:0/3654027770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f589c0b6ee0 0x7f589c0d06c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:11.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.117+0000 7f58a1200640 1 -- 172.21.15.86:0/3654027770 >> 172.21.15.86:0/3654027770 conn(0x7f589c0b4f80 msgr2=0x7f589c0b5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:11.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.117+0000 7f58a1200640 1 -- 172.21.15.86:0/3654027770 shutdown_connections 2024-04-25T14:04:11.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.117+0000 7f58a1200640 1 -- 172.21.15.86:0/3654027770 wait complete. 2024-04-25T14:04:11.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.121+0000 7f58a1200640 1 Processor -- start 2024-04-25T14:04:11.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.121+0000 7f58a1200640 1 -- start start 2024-04-25T14:04:11.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.121+0000 7f58a1200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f589c0727e0 0x7f589c1575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:11.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.121+0000 7f58a1200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f589c0b6ee0 0x7f589c0b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:11.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.121+0000 7f58a1200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f589c0d0c00 0x7f589c0b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:11.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.121+0000 7f58a1200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f589c161fe0 con 0x7f589c0727e0 2024-04-25T14:04:11.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.121+0000 7f58a1200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f589c0cff50 con 0x7f589c0b6ee0 2024-04-25T14:04:11.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.121+0000 7f58a1200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f589c162160 con 0x7f589c0d0c00 2024-04-25T14:04:11.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.121+0000 7f589b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f589c0727e0 0x7f589c1575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:11.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.121+0000 7f589b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f589c0727e0 0x7f589c1575b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37844/0 (socket says 172.21.15.86:37844) 2024-04-25T14:04:11.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.121+0000 7f589b400640 1 -- 172.21.15.86:0/2418858872 learned_addr learned my addr 172.21.15.86:0/2418858872 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:11.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.121+0000 7f589aa00640 1 --2- 172.21.15.86:0/2418858872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f589c0b6ee0 0x7f589c0b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:11.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.121+0000 7f589b400640 1 -- 172.21.15.86:0/2418858872 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f589c0d0c00 msgr2=0x7f589c0b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:11.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.121+0000 7f589be00640 1 --2- 172.21.15.86:0/2418858872 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f589c0d0c00 0x7f589c0b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:11.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.121+0000 7f589b400640 1 --2- 172.21.15.86:0/2418858872 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f589c0d0c00 0x7f589c0b45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:11.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.121+0000 7f589b400640 1 -- 172.21.15.86:0/2418858872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f589c0b6ee0 msgr2=0x7f589c0b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:11.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.121+0000 7f589b400640 1 --2- 172.21.15.86:0/2418858872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f589c0b6ee0 0x7f589c0b4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:11.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.121+0000 7f589b400640 1 -- 172.21.15.86:0/2418858872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f589c15ca20 con 0x7f589c0727e0 2024-04-25T14:04:11.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.121+0000 7f589be00640 1 --2- 172.21.15.86:0/2418858872 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f589c0d0c00 0x7f589c0b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:11.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.121+0000 7f589b400640 1 --2- 172.21.15.86:0/2418858872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f589c0727e0 0x7f589c1575b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f588c0ae040 tx=0x7f588c01ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:11.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.125+0000 7f5893400640 1 -- 172.21.15.86:0/2418858872 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c0b0050 con 0x7f589c0727e0 2024-04-25T14:04:11.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.125+0000 7f5893400640 1 -- 172.21.15.86:0/2418858872 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f588c0bc070 con 0x7f589c0727e0 2024-04-25T14:04:11.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.125+0000 7f58a1200640 1 -- 172.21.15.86:0/2418858872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f589c06d9d0 con 0x7f589c0727e0 2024-04-25T14:04:11.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.125+0000 7f5893400640 1 -- 172.21.15.86:0/2418858872 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c0b5a70 con 0x7f589c0727e0 2024-04-25T14:04:11.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.125+0000 7f58a1200640 1 -- 172.21.15.86:0/2418858872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f589c15c4b0 con 0x7f589c0727e0 2024-04-25T14:04:11.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.125+0000 7f58a1200640 1 -- 172.21.15.86:0/2418858872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f589c076b80 con 0x7f589c0727e0 2024-04-25T14:04:11.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.125+0000 7f5893400640 1 -- 172.21.15.86:0/2418858872 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f588c0ab480 con 0x7f589c0727e0 2024-04-25T14:04:11.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.125+0000 7f5893400640 1 --2- 172.21.15.86:0/2418858872 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f587c06f430 0x7f587c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:11.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.125+0000 7f5893400640 1 -- 172.21.15.86:0/2418858872 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f588c129da0 con 0x7f589c0727e0 2024-04-25T14:04:11.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.125+0000 7f589aa00640 1 --2- 172.21.15.86:0/2418858872 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f587c06f430 0x7f587c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:11.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.129+0000 7f589aa00640 1 --2- 172.21.15.86:0/2418858872 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f587c06f430 0x7f587c071940 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5888006fd0 tx=0x7f5888008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:11.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.129+0000 7f5893400640 1 -- 172.21.15.86:0/2418858872 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f588c0c6020 con 0x7f589c0727e0 2024-04-25T14:04:11.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.285+0000 7f58a1200640 1 -- 172.21.15.86:0/2418858872 --> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f589c06d780 con 0x7f587c06f430 2024-04-25T14:04:11.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.293+0000 7f5893400640 1 -- 172.21.15.86:0/2418858872 <== mgr.4103 v2:172.21.15.86:6832/2968869755 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332527 (secure 0 0 0) 0x7f589c06d780 con 0x7f587c06f430 2024-04-25T14:04:11.295 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:11.305 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:04:11.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.297+0000 7f58a1200640 1 -- 172.21.15.86:0/2418858872 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f587c06f430 msgr2=0x7f587c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:11.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.297+0000 7f58a1200640 1 --2- 172.21.15.86:0/2418858872 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f587c06f430 0x7f587c071940 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5888006fd0 tx=0x7f5888008040 comp rx=0 tx=0).stop 2024-04-25T14:04:11.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.297+0000 7f58a1200640 1 -- 172.21.15.86:0/2418858872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f589c0727e0 msgr2=0x7f589c1575b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:11.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.297+0000 7f58a1200640 1 --2- 172.21.15.86:0/2418858872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f589c0727e0 0x7f589c1575b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f588c0ae040 tx=0x7f588c01ac30 comp rx=0 tx=0).stop 2024-04-25T14:04:11.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.297+0000 7f58a1200640 1 -- 172.21.15.86:0/2418858872 shutdown_connections 2024-04-25T14:04:11.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.297+0000 7f58a1200640 1 --2- 172.21.15.86:0/2418858872 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f587c06f430 0x7f587c071940 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:11.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.297+0000 7f58a1200640 1 --2- 172.21.15.86:0/2418858872 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f589c0d0c00 0x7f589c0b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:11.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.297+0000 7f58a1200640 1 --2- 172.21.15.86:0/2418858872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f589c0b6ee0 0x7f589c0b4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:11.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.297+0000 7f58a1200640 1 --2- 172.21.15.86:0/2418858872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f589c0727e0 0x7f589c1575b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:11.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.297+0000 7f58a1200640 1 -- 172.21.15.86:0/2418858872 >> 172.21.15.86:0/2418858872 conn(0x7f589c0b4f80 msgr2=0x7f589c1614c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:11.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.297+0000 7f58a1200640 1 -- 172.21.15.86:0/2418858872 shutdown_connections 2024-04-25T14:04:11.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.297+0000 7f58a1200640 1 -- 172.21.15.86:0/2418858872 wait complete. 2024-04-25T14:04:11.318 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-04-25T14:04:10.714677+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":150,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219400,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755320,"statfs":{"total":773094113280,"available":772869447680,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003039"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774780+0000","last_change":"2024-04-25T14:03:50.477862+0000","last_active":"2024-04-25T14:03:56.774780+0000","last_peered":"2024-04-25T14:03:56.774780+0000","last_clean":"2024-04-25T14:03:56.774780+0000","last_became_active":"2024-04-25T14:03:50.477375+0000","last_became_peered":"2024-04-25T14:03:50.477375+0000","last_unstale":"2024-04-25T14:03:56.774780+0000","last_undegraded":"2024-04-25T14:03:56.774780+0000","last_fullsized":"2024-04-25T14:03:56.774780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:01.852459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494933+0000","last_change":"2024-04-25T14:03:51.479650+0000","last_active":"2024-04-25T14:03:56.494933+0000","last_peered":"2024-04-25T14:03:56.494933+0000","last_clean":"2024-04-25T14:03:56.494933+0000","last_became_active":"2024-04-25T14:03:51.479295+0000","last_became_peered":"2024-04-25T14:03:51.479295+0000","last_unstale":"2024-04-25T14:03:56.494933+0000","last_undegraded":"2024-04-25T14:03:56.494933+0000","last_fullsized":"2024-04-25T14:03:56.494933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:17.381249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493593+0000","last_change":"2024-04-25T14:03:55.681212+0000","last_active":"2024-04-25T14:03:56.493593+0000","last_peered":"2024-04-25T14:03:56.493593+0000","last_clean":"2024-04-25T14:03:56.493593+0000","last_became_active":"2024-04-25T14:03:55.680579+0000","last_became_peered":"2024-04-25T14:03:55.680579+0000","last_unstale":"2024-04-25T14:03:56.493593+0000","last_undegraded":"2024-04-25T14:03:56.493593+0000","last_fullsized":"2024-04-25T14:03:56.493593+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:33:32.909419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495038+0000","last_change":"2024-04-25T14:03:50.468056+0000","last_active":"2024-04-25T14:03:56.495038+0000","last_peered":"2024-04-25T14:03:56.495038+0000","last_clean":"2024-04-25T14:03:56.495038+0000","last_became_active":"2024-04-25T14:03:50.467531+0000","last_became_peered":"2024-04-25T14:03:50.467531+0000","last_unstale":"2024-04-25T14:03:56.495038+0000","last_undegraded":"2024-04-25T14:03:56.495038+0000","last_fullsized":"2024-04-25T14:03:56.495038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:29.264084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774540+0000","last_change":"2024-04-25T14:03:51.485995+0000","last_active":"2024-04-25T14:03:56.774540+0000","last_peered":"2024-04-25T14:03:56.774540+0000","last_clean":"2024-04-25T14:03:56.774540+0000","last_became_active":"2024-04-25T14:03:51.485364+0000","last_became_peered":"2024-04-25T14:03:51.485364+0000","last_unstale":"2024-04-25T14:03:56.774540+0000","last_undegraded":"2024-04-25T14:03:56.774540+0000","last_fullsized":"2024-04-25T14:03:56.774540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:10.318378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772007+0000","last_change":"2024-04-25T14:03:55.514585+0000","last_active":"2024-04-25T14:03:56.772007+0000","last_peered":"2024-04-25T14:03:56.772007+0000","last_clean":"2024-04-25T14:03:56.772007+0000","last_became_active":"2024-04-25T14:03:55.514106+0000","last_became_peered":"2024-04-25T14:03:55.514106+0000","last_unstale":"2024-04-25T14:03:56.772007+0000","last_undegraded":"2024-04-25T14:03:56.772007+0000","last_fullsized":"2024-04-25T14:03:56.772007+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:14.363355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773034+0000","last_change":"2024-04-25T14:03:50.476493+0000","last_active":"2024-04-25T14:03:56.773034+0000","last_peered":"2024-04-25T14:03:56.773034+0000","last_clean":"2024-04-25T14:03:56.773034+0000","last_became_active":"2024-04-25T14:03:50.476154+0000","last_became_peered":"2024-04-25T14:03:50.476154+0000","last_unstale":"2024-04-25T14:03:56.773034+0000","last_undegraded":"2024-04-25T14:03:56.773034+0000","last_fullsized":"2024-04-25T14:03:56.773034+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:45.080718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496310+0000","last_change":"2024-04-25T14:03:51.494560+0000","last_active":"2024-04-25T14:03:56.496310+0000","last_peered":"2024-04-25T14:03:56.496310+0000","last_clean":"2024-04-25T14:03:56.496310+0000","last_became_active":"2024-04-25T14:03:51.494211+0000","last_became_peered":"2024-04-25T14:03:51.494211+0000","last_unstale":"2024-04-25T14:03:56.496310+0000","last_undegraded":"2024-04-25T14:03:56.496310+0000","last_fullsized":"2024-04-25T14:03:56.496310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:17.381249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772740+0000","last_change":"2024-04-25T14:03:55.518340+0000","last_active":"2024-04-25T14:03:56.772740+0000","last_peered":"2024-04-25T14:03:56.772740+0000","last_clean":"2024-04-25T14:03:56.772740+0000","last_became_active":"2024-04-25T14:03:55.515787+0000","last_became_peered":"2024-04-25T14:03:55.515787+0000","last_unstale":"2024-04-25T14:03:56.772740+0000","last_undegraded":"2024-04-25T14:03:56.772740+0000","last_fullsized":"2024-04-25T14:03:56.772740+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:24.136045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772400+0000","last_change":"2024-04-25T14:03:50.472667+0000","last_active":"2024-04-25T14:03:56.772400+0000","last_peered":"2024-04-25T14:03:56.772400+0000","last_clean":"2024-04-25T14:03:56.772400+0000","last_became_active":"2024-04-25T14:03:50.472286+0000","last_became_peered":"2024-04-25T14:03:50.472286+0000","last_unstale":"2024-04-25T14:03:56.772400+0000","last_undegraded":"2024-04-25T14:03:56.772400+0000","last_fullsized":"2024-04-25T14:03:56.772400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:02.117945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496569+0000","last_change":"2024-04-25T14:03:51.494545+0000","last_active":"2024-04-25T14:03:56.496569+0000","last_peered":"2024-04-25T14:03:56.496569+0000","last_clean":"2024-04-25T14:03:56.496569+0000","last_became_active":"2024-04-25T14:03:51.494238+0000","last_became_peered":"2024-04-25T14:03:51.494238+0000","last_unstale":"2024-04-25T14:03:56.496569+0000","last_undegraded":"2024-04-25T14:03:56.496569+0000","last_fullsized":"2024-04-25T14:03:56.496569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:08.845346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772125+0000","last_change":"2024-04-25T14:03:55.512835+0000","last_active":"2024-04-25T14:03:56.772125+0000","last_peered":"2024-04-25T14:03:56.772125+0000","last_clean":"2024-04-25T14:03:56.772125+0000","last_became_active":"2024-04-25T14:03:55.512330+0000","last_became_peered":"2024-04-25T14:03:55.512330+0000","last_unstale":"2024-04-25T14:03:56.772125+0000","last_undegraded":"2024-04-25T14:03:56.772125+0000","last_fullsized":"2024-04-25T14:03:56.772125+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:47.781818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494962+0000","last_change":"2024-04-25T14:03:51.494496+0000","last_active":"2024-04-25T14:03:56.494962+0000","last_peered":"2024-04-25T14:03:56.494962+0000","last_clean":"2024-04-25T14:03:56.494962+0000","last_became_active":"2024-04-25T14:03:51.494201+0000","last_became_peered":"2024-04-25T14:03:51.494201+0000","last_unstale":"2024-04-25T14:03:56.494962+0000","last_undegraded":"2024-04-25T14:03:56.494962+0000","last_fullsized":"2024-04-25T14:03:56.494962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.599283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773644+0000","last_change":"2024-04-25T14:03:50.472026+0000","last_active":"2024-04-25T14:03:56.773644+0000","last_peered":"2024-04-25T14:03:56.773644+0000","last_clean":"2024-04-25T14:03:56.773644+0000","last_became_active":"2024-04-25T14:03:50.468476+0000","last_became_peered":"2024-04-25T14:03:50.468476+0000","last_unstale":"2024-04-25T14:03:56.773644+0000","last_undegraded":"2024-04-25T14:03:56.773644+0000","last_fullsized":"2024-04-25T14:03:56.773644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:39.630327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772916+0000","last_change":"2024-04-25T14:03:55.524347+0000","last_active":"2024-04-25T14:03:56.772916+0000","last_peered":"2024-04-25T14:03:56.772916+0000","last_clean":"2024-04-25T14:03:56.772916+0000","last_became_active":"2024-04-25T14:03:55.523890+0000","last_became_peered":"2024-04-25T14:03:55.523890+0000","last_unstale":"2024-04-25T14:03:56.772916+0000","last_undegraded":"2024-04-25T14:03:56.772916+0000","last_fullsized":"2024-04-25T14:03:56.772916+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:20.858506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493611+0000","last_change":"2024-04-25T14:03:51.471282+0000","last_active":"2024-04-25T14:03:56.493611+0000","last_peered":"2024-04-25T14:03:56.493611+0000","last_clean":"2024-04-25T14:03:56.493611+0000","last_became_active":"2024-04-25T14:03:51.470871+0000","last_became_peered":"2024-04-25T14:03:51.470871+0000","last_unstale":"2024-04-25T14:03:56.493611+0000","last_undegraded":"2024-04-25T14:03:56.493611+0000","last_fullsized":"2024-04-25T14:03:56.493611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:34.707119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773462+0000","last_change":"2024-04-25T14:03:50.467312+0000","last_active":"2024-04-25T14:03:56.773462+0000","last_peered":"2024-04-25T14:03:56.773462+0000","last_clean":"2024-04-25T14:03:56.773462+0000","last_became_active":"2024-04-25T14:03:50.467020+0000","last_became_peered":"2024-04-25T14:03:50.467020+0000","last_unstale":"2024-04-25T14:03:56.773462+0000","last_undegraded":"2024-04-25T14:03:56.773462+0000","last_fullsized":"2024-04-25T14:03:56.773462+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:45.080718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772970+0000","last_change":"2024-04-25T14:03:55.520580+0000","last_active":"2024-04-25T14:03:56.772970+0000","last_peered":"2024-04-25T14:03:56.772970+0000","last_clean":"2024-04-25T14:03:56.772970+0000","last_became_active":"2024-04-25T14:03:55.520013+0000","last_became_peered":"2024-04-25T14:03:55.520013+0000","last_unstale":"2024-04-25T14:03:56.772970+0000","last_undegraded":"2024-04-25T14:03:56.772970+0000","last_fullsized":"2024-04-25T14:03:56.772970+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:38.752096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494042+0000","last_change":"2024-04-25T14:03:51.474463+0000","last_active":"2024-04-25T14:03:56.494042+0000","last_peered":"2024-04-25T14:03:56.494042+0000","last_clean":"2024-04-25T14:03:56.494042+0000","last_became_active":"2024-04-25T14:03:51.474154+0000","last_became_peered":"2024-04-25T14:03:51.474154+0000","last_unstale":"2024-04-25T14:03:56.494042+0000","last_undegraded":"2024-04-25T14:03:56.494042+0000","last_fullsized":"2024-04-25T14:03:56.494042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:10.318378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772985+0000","last_change":"2024-04-25T14:03:50.476410+0000","last_active":"2024-04-25T14:03:56.772985+0000","last_peered":"2024-04-25T14:03:56.772985+0000","last_clean":"2024-04-25T14:03:56.772985+0000","last_became_active":"2024-04-25T14:03:50.476083+0000","last_became_peered":"2024-04-25T14:03:50.476083+0000","last_unstale":"2024-04-25T14:03:56.772985+0000","last_undegraded":"2024-04-25T14:03:56.772985+0000","last_fullsized":"2024-04-25T14:03:56.772985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:29.264084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774278+0000","last_change":"2024-04-25T14:03:55.674875+0000","last_active":"2024-04-25T14:03:56.774278+0000","last_peered":"2024-04-25T14:03:56.774278+0000","last_clean":"2024-04-25T14:03:56.774278+0000","last_became_active":"2024-04-25T14:03:55.674408+0000","last_became_peered":"2024-04-25T14:03:55.674408+0000","last_unstale":"2024-04-25T14:03:56.774278+0000","last_undegraded":"2024-04-25T14:03:56.774278+0000","last_fullsized":"2024-04-25T14:03:56.774278+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:50.234674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773825+0000","last_change":"2024-04-25T14:03:51.487560+0000","last_active":"2024-04-25T14:03:56.773825+0000","last_peered":"2024-04-25T14:03:56.773825+0000","last_clean":"2024-04-25T14:03:56.773825+0000","last_became_active":"2024-04-25T14:03:51.486822+0000","last_became_peered":"2024-04-25T14:03:51.486822+0000","last_unstale":"2024-04-25T14:03:56.773825+0000","last_undegraded":"2024-04-25T14:03:56.773825+0000","last_fullsized":"2024-04-25T14:03:56.773825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:16.370061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.771874+0000","last_change":"2024-04-25T14:03:50.479489+0000","last_active":"2024-04-25T14:03:56.771874+0000","last_peered":"2024-04-25T14:03:56.771874+0000","last_clean":"2024-04-25T14:03:56.771874+0000","last_became_active":"2024-04-25T14:03:50.479212+0000","last_became_peered":"2024-04-25T14:03:50.479212+0000","last_unstale":"2024-04-25T14:03:56.771874+0000","last_undegraded":"2024-04-25T14:03:56.771874+0000","last_fullsized":"2024-04-25T14:03:56.771874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:10.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773707+0000","last_change":"2024-04-25T14:03:55.519638+0000","last_active":"2024-04-25T14:03:56.773707+0000","last_peered":"2024-04-25T14:03:56.773707+0000","last_clean":"2024-04-25T14:03:56.773707+0000","last_became_active":"2024-04-25T14:03:55.519191+0000","last_became_peered":"2024-04-25T14:03:55.519191+0000","last_unstale":"2024-04-25T14:03:56.773707+0000","last_undegraded":"2024-04-25T14:03:56.773707+0000","last_fullsized":"2024-04-25T14:03:56.773707+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:25.451897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774673+0000","last_change":"2024-04-25T14:03:51.477687+0000","last_active":"2024-04-25T14:03:56.774673+0000","last_peered":"2024-04-25T14:03:56.774673+0000","last_clean":"2024-04-25T14:03:56.774673+0000","last_became_active":"2024-04-25T14:03:51.477339+0000","last_became_peered":"2024-04-25T14:03:51.477339+0000","last_unstale":"2024-04-25T14:03:56.774673+0000","last_undegraded":"2024-04-25T14:03:56.774673+0000","last_fullsized":"2024-04-25T14:03:56.774673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:43.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496742+0000","last_change":"2024-04-25T14:03:50.470614+0000","last_active":"2024-04-25T14:03:56.496742+0000","last_peered":"2024-04-25T14:03:56.496742+0000","last_clean":"2024-04-25T14:03:56.496742+0000","last_became_active":"2024-04-25T14:03:50.470405+0000","last_became_peered":"2024-04-25T14:03:50.470405+0000","last_unstale":"2024-04-25T14:03:56.496742+0000","last_undegraded":"2024-04-25T14:03:56.496742+0000","last_fullsized":"2024-04-25T14:03:56.496742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:02.117945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773539+0000","last_change":"2024-04-25T14:03:55.681096+0000","last_active":"2024-04-25T14:03:56.773539+0000","last_peered":"2024-04-25T14:03:56.773539+0000","last_clean":"2024-04-25T14:03:56.773539+0000","last_became_active":"2024-04-25T14:03:55.680556+0000","last_became_peered":"2024-04-25T14:03:55.680556+0000","last_unstale":"2024-04-25T14:03:56.773539+0000","last_undegraded":"2024-04-25T14:03:56.773539+0000","last_fullsized":"2024-04-25T14:03:56.773539+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:09.534866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495184+0000","last_change":"2024-04-25T14:03:51.473595+0000","last_active":"2024-04-25T14:03:56.495184+0000","last_peered":"2024-04-25T14:03:56.495184+0000","last_clean":"2024-04-25T14:03:56.495184+0000","last_became_active":"2024-04-25T14:03:51.473273+0000","last_became_peered":"2024-04-25T14:03:51.473273+0000","last_unstale":"2024-04-25T14:03:56.495184+0000","last_undegraded":"2024-04-25T14:03:56.495184+0000","last_fullsized":"2024-04-25T14:03:56.495184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:05.489889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774166+0000","last_change":"2024-04-25T14:03:50.476274+0000","last_active":"2024-04-25T14:03:56.774166+0000","last_peered":"2024-04-25T14:03:56.774166+0000","last_clean":"2024-04-25T14:03:56.774166+0000","last_became_active":"2024-04-25T14:03:50.475935+0000","last_became_peered":"2024-04-25T14:03:50.475935+0000","last_unstale":"2024-04-25T14:03:56.774166+0000","last_undegraded":"2024-04-25T14:03:56.774166+0000","last_fullsized":"2024-04-25T14:03:56.774166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:39.630327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493917+0000","last_change":"2024-04-25T14:03:55.675251+0000","last_active":"2024-04-25T14:03:56.493917+0000","last_peered":"2024-04-25T14:03:56.493917+0000","last_clean":"2024-04-25T14:03:56.493917+0000","last_became_active":"2024-04-25T14:03:55.674772+0000","last_became_peered":"2024-04-25T14:03:55.674772+0000","last_unstale":"2024-04-25T14:03:56.493917+0000","last_undegraded":"2024-04-25T14:03:56.493917+0000","last_fullsized":"2024-04-25T14:03:56.493917+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:58.844877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773967+0000","last_change":"2024-04-25T14:03:51.476624+0000","last_active":"2024-04-25T14:03:56.773967+0000","last_peered":"2024-04-25T14:03:56.773967+0000","last_clean":"2024-04-25T14:03:56.773967+0000","last_became_active":"2024-04-25T14:03:51.476320+0000","last_became_peered":"2024-04-25T14:03:51.476320+0000","last_unstale":"2024-04-25T14:03:56.773967+0000","last_undegraded":"2024-04-25T14:03:56.773967+0000","last_fullsized":"2024-04-25T14:03:56.773967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:20.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773171+0000","last_change":"2024-04-25T14:03:50.482353+0000","last_active":"2024-04-25T14:03:56.773171+0000","last_peered":"2024-04-25T14:03:56.773171+0000","last_clean":"2024-04-25T14:03:56.773171+0000","last_became_active":"2024-04-25T14:03:50.482063+0000","last_became_peered":"2024-04-25T14:03:50.482063+0000","last_unstale":"2024-04-25T14:03:56.773171+0000","last_undegraded":"2024-04-25T14:03:56.773171+0000","last_fullsized":"2024-04-25T14:03:56.773171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:37.406291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773151+0000","last_change":"2024-04-25T14:03:55.676872+0000","last_active":"2024-04-25T14:03:56.773151+0000","last_peered":"2024-04-25T14:03:56.773151+0000","last_clean":"2024-04-25T14:03:56.773151+0000","last_became_active":"2024-04-25T14:03:55.676444+0000","last_became_peered":"2024-04-25T14:03:55.676444+0000","last_unstale":"2024-04-25T14:03:56.773151+0000","last_undegraded":"2024-04-25T14:03:56.773151+0000","last_fullsized":"2024-04-25T14:03:56.773151+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:37.811376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496691+0000","last_change":"2024-04-25T14:03:51.480200+0000","last_active":"2024-04-25T14:03:56.496691+0000","last_peered":"2024-04-25T14:03:56.496691+0000","last_clean":"2024-04-25T14:03:56.496691+0000","last_became_active":"2024-04-25T14:03:51.479798+0000","last_became_peered":"2024-04-25T14:03:51.479798+0000","last_unstale":"2024-04-25T14:03:56.496691+0000","last_undegraded":"2024-04-25T14:03:56.496691+0000","last_fullsized":"2024-04-25T14:03:56.496691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:46.189697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774003+0000","last_change":"2024-04-25T14:03:50.479761+0000","last_active":"2024-04-25T14:03:56.774003+0000","last_peered":"2024-04-25T14:03:56.774003+0000","last_clean":"2024-04-25T14:03:56.774003+0000","last_became_active":"2024-04-25T14:03:50.479175+0000","last_became_peered":"2024-04-25T14:03:50.479175+0000","last_unstale":"2024-04-25T14:03:56.774003+0000","last_undegraded":"2024-04-25T14:03:56.774003+0000","last_fullsized":"2024-04-25T14:03:56.774003+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:02.117945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773256+0000","last_change":"2024-04-25T14:03:55.515692+0000","last_active":"2024-04-25T14:03:56.773256+0000","last_peered":"2024-04-25T14:03:56.773256+0000","last_clean":"2024-04-25T14:03:56.773256+0000","last_became_active":"2024-04-25T14:03:55.515238+0000","last_became_peered":"2024-04-25T14:03:55.515238+0000","last_unstale":"2024-04-25T14:03:56.773256+0000","last_undegraded":"2024-04-25T14:03:56.773256+0000","last_fullsized":"2024-04-25T14:03:56.773256+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:51.371333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773096+0000","last_change":"2024-04-25T14:03:51.495207+0000","last_active":"2024-04-25T14:03:56.773096+0000","last_peered":"2024-04-25T14:03:56.773096+0000","last_clean":"2024-04-25T14:03:56.773096+0000","last_became_active":"2024-04-25T14:03:51.492531+0000","last_became_peered":"2024-04-25T14:03:51.492531+0000","last_unstale":"2024-04-25T14:03:56.773096+0000","last_undegraded":"2024-04-25T14:03:56.773096+0000","last_fullsized":"2024-04-25T14:03:56.773096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:03.123407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495711+0000","last_change":"2024-04-25T14:03:50.459698+0000","last_active":"2024-04-25T14:03:56.495711+0000","last_peered":"2024-04-25T14:03:56.495711+0000","last_clean":"2024-04-25T14:03:56.495711+0000","last_became_active":"2024-04-25T14:03:50.459451+0000","last_became_peered":"2024-04-25T14:03:50.459451+0000","last_unstale":"2024-04-25T14:03:56.495711+0000","last_undegraded":"2024-04-25T14:03:56.495711+0000","last_fullsized":"2024-04-25T14:03:56.495711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:15.364599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773837+0000","last_change":"2024-04-25T14:03:55.519334+0000","last_active":"2024-04-25T14:03:56.773837+0000","last_peered":"2024-04-25T14:03:56.773837+0000","last_clean":"2024-04-25T14:03:56.773837+0000","last_became_active":"2024-04-25T14:03:55.518940+0000","last_became_peered":"2024-04-25T14:03:55.518940+0000","last_unstale":"2024-04-25T14:03:56.773837+0000","last_undegraded":"2024-04-25T14:03:56.773837+0000","last_fullsized":"2024-04-25T14:03:56.773837+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:12.890323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772902+0000","last_change":"2024-04-25T14:03:51.476141+0000","last_active":"2024-04-25T14:03:56.772902+0000","last_peered":"2024-04-25T14:03:56.772902+0000","last_clean":"2024-04-25T14:03:56.772902+0000","last_became_active":"2024-04-25T14:03:51.475819+0000","last_became_peered":"2024-04-25T14:03:51.475819+0000","last_unstale":"2024-04-25T14:03:56.772902+0000","last_undegraded":"2024-04-25T14:03:56.772902+0000","last_fullsized":"2024-04-25T14:03:56.772902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:31.707048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772028+0000","last_change":"2024-04-25T14:03:50.458612+0000","last_active":"2024-04-25T14:03:56.772028+0000","last_peered":"2024-04-25T14:03:56.772028+0000","last_clean":"2024-04-25T14:03:56.772028+0000","last_became_active":"2024-04-25T14:03:50.458254+0000","last_became_peered":"2024-04-25T14:03:50.458254+0000","last_unstale":"2024-04-25T14:03:56.772028+0000","last_undegraded":"2024-04-25T14:03:56.772028+0000","last_fullsized":"2024-04-25T14:03:56.772028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:29.264084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774279+0000","last_change":"2024-04-25T14:03:55.513023+0000","last_active":"2024-04-25T14:03:56.774279+0000","last_peered":"2024-04-25T14:03:56.774279+0000","last_clean":"2024-04-25T14:03:56.774279+0000","last_became_active":"2024-04-25T14:03:55.512550+0000","last_became_peered":"2024-04-25T14:03:55.512550+0000","last_unstale":"2024-04-25T14:03:56.774279+0000","last_undegraded":"2024-04-25T14:03:56.774279+0000","last_fullsized":"2024-04-25T14:03:56.774279+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:06.818399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.771972+0000","last_change":"2024-04-25T14:03:51.487355+0000","last_active":"2024-04-25T14:03:56.771972+0000","last_peered":"2024-04-25T14:03:56.771972+0000","last_clean":"2024-04-25T14:03:56.771972+0000","last_became_active":"2024-04-25T14:03:51.487050+0000","last_became_peered":"2024-04-25T14:03:51.487050+0000","last_unstale":"2024-04-25T14:03:56.771972+0000","last_undegraded":"2024-04-25T14:03:56.771972+0000","last_fullsized":"2024-04-25T14:03:56.771972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:33.766258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772008+0000","last_change":"2024-04-25T14:03:50.476776+0000","last_active":"2024-04-25T14:03:56.772008+0000","last_peered":"2024-04-25T14:03:56.772008+0000","last_clean":"2024-04-25T14:03:56.772008+0000","last_became_active":"2024-04-25T14:03:50.476470+0000","last_became_peered":"2024-04-25T14:03:50.476470+0000","last_unstale":"2024-04-25T14:03:56.772008+0000","last_undegraded":"2024-04-25T14:03:56.772008+0000","last_fullsized":"2024-04-25T14:03:56.772008+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:15.364599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774604+0000","last_change":"2024-04-25T14:03:55.674908+0000","last_active":"2024-04-25T14:03:56.774604+0000","last_peered":"2024-04-25T14:03:56.774604+0000","last_clean":"2024-04-25T14:03:56.774604+0000","last_became_active":"2024-04-25T14:03:55.674383+0000","last_became_peered":"2024-04-25T14:03:55.674383+0000","last_unstale":"2024-04-25T14:03:56.774604+0000","last_undegraded":"2024-04-25T14:03:56.774604+0000","last_fullsized":"2024-04-25T14:03:56.774604+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:06.742929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493557+0000","last_change":"2024-04-25T14:03:51.489259+0000","last_active":"2024-04-25T14:03:56.493557+0000","last_peered":"2024-04-25T14:03:56.493557+0000","last_clean":"2024-04-25T14:03:56.493557+0000","last_became_active":"2024-04-25T14:03:51.488822+0000","last_became_peered":"2024-04-25T14:03:51.488822+0000","last_unstale":"2024-04-25T14:03:56.493557+0000","last_undegraded":"2024-04-25T14:03:56.493557+0000","last_fullsized":"2024-04-25T14:03:56.493557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:43.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495950+0000","last_change":"2024-04-25T14:03:50.466483+0000","last_active":"2024-04-25T14:03:56.495950+0000","last_peered":"2024-04-25T14:03:56.495950+0000","last_clean":"2024-04-25T14:03:56.495950+0000","last_became_active":"2024-04-25T14:03:50.463125+0000","last_became_peered":"2024-04-25T14:03:50.463125+0000","last_unstale":"2024-04-25T14:03:56.495950+0000","last_undegraded":"2024-04-25T14:03:56.495950+0000","last_fullsized":"2024-04-25T14:03:56.495950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:29.264084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":117,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772125+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:03:56.772125+0000","last_peered":"2024-04-25T14:03:56.772125+0000","last_clean":"2024-04-25T14:03:56.772125+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:03:56.772125+0000","last_undegraded":"2024-04-25T14:03:56.772125+0000","last_fullsized":"2024-04-25T14:03:56.772125+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496238+0000","last_change":"2024-04-25T14:03:55.678084+0000","last_active":"2024-04-25T14:03:56.496238+0000","last_peered":"2024-04-25T14:03:56.496238+0000","last_clean":"2024-04-25T14:03:56.496238+0000","last_became_active":"2024-04-25T14:03:55.677526+0000","last_became_peered":"2024-04-25T14:03:55.677526+0000","last_unstale":"2024-04-25T14:03:56.496238+0000","last_undegraded":"2024-04-25T14:03:56.496238+0000","last_fullsized":"2024-04-25T14:03:56.496238+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:25.451897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772807+0000","last_change":"2024-04-25T14:03:51.482608+0000","last_active":"2024-04-25T14:03:56.772807+0000","last_peered":"2024-04-25T14:03:56.772807+0000","last_clean":"2024-04-25T14:03:56.772807+0000","last_became_active":"2024-04-25T14:03:51.482022+0000","last_became_peered":"2024-04-25T14:03:51.482022+0000","last_unstale":"2024-04-25T14:03:56.772807+0000","last_undegraded":"2024-04-25T14:03:56.772807+0000","last_fullsized":"2024-04-25T14:03:56.772807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:33.766399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772584+0000","last_change":"2024-04-25T14:03:50.469170+0000","last_active":"2024-04-25T14:03:56.772584+0000","last_peered":"2024-04-25T14:03:56.772584+0000","last_clean":"2024-04-25T14:03:56.772584+0000","last_became_active":"2024-04-25T14:03:50.468742+0000","last_became_peered":"2024-04-25T14:03:50.468742+0000","last_unstale":"2024-04-25T14:03:56.772584+0000","last_undegraded":"2024-04-25T14:03:56.772584+0000","last_fullsized":"2024-04-25T14:03:56.772584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:10.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494752+0000","last_change":"2024-04-25T14:03:55.676560+0000","last_active":"2024-04-25T14:03:56.494752+0000","last_peered":"2024-04-25T14:03:56.494752+0000","last_clean":"2024-04-25T14:03:56.494752+0000","last_became_active":"2024-04-25T14:03:55.676021+0000","last_became_peered":"2024-04-25T14:03:55.676021+0000","last_unstale":"2024-04-25T14:03:56.494752+0000","last_undegraded":"2024-04-25T14:03:56.494752+0000","last_fullsized":"2024-04-25T14:03:56.494752+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:09.534866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496565+0000","last_change":"2024-04-25T14:03:51.466314+0000","last_active":"2024-04-25T14:03:56.496565+0000","last_peered":"2024-04-25T14:03:56.496565+0000","last_clean":"2024-04-25T14:03:56.496565+0000","last_became_active":"2024-04-25T14:03:51.465982+0000","last_became_peered":"2024-04-25T14:03:51.465982+0000","last_unstale":"2024-04-25T14:03:56.496565+0000","last_undegraded":"2024-04-25T14:03:56.496565+0000","last_fullsized":"2024-04-25T14:03:56.496565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:47.326356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774109+0000","last_change":"2024-04-25T14:03:50.465115+0000","last_active":"2024-04-25T14:03:56.774109+0000","last_peered":"2024-04-25T14:03:56.774109+0000","last_clean":"2024-04-25T14:03:56.774109+0000","last_became_active":"2024-04-25T14:03:50.464561+0000","last_became_peered":"2024-04-25T14:03:50.464561+0000","last_unstale":"2024-04-25T14:03:56.774109+0000","last_undegraded":"2024-04-25T14:03:56.774109+0000","last_fullsized":"2024-04-25T14:03:56.774109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:10.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773457+0000","last_change":"2024-04-25T14:03:55.526344+0000","last_active":"2024-04-25T14:03:56.773457+0000","last_peered":"2024-04-25T14:03:56.773457+0000","last_clean":"2024-04-25T14:03:56.773457+0000","last_became_active":"2024-04-25T14:03:55.525889+0000","last_became_peered":"2024-04-25T14:03:55.525889+0000","last_unstale":"2024-04-25T14:03:56.773457+0000","last_undegraded":"2024-04-25T14:03:56.773457+0000","last_fullsized":"2024-04-25T14:03:56.773457+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:06.818399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773492+0000","last_change":"2024-04-25T14:03:51.494453+0000","last_active":"2024-04-25T14:03:56.773492+0000","last_peered":"2024-04-25T14:03:56.773492+0000","last_clean":"2024-04-25T14:03:56.773492+0000","last_became_active":"2024-04-25T14:03:51.494113+0000","last_became_peered":"2024-04-25T14:03:51.494113+0000","last_unstale":"2024-04-25T14:03:56.773492+0000","last_undegraded":"2024-04-25T14:03:56.773492+0000","last_fullsized":"2024-04-25T14:03:56.773492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:08.845346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772486+0000","last_change":"2024-04-25T14:03:50.463824+0000","last_active":"2024-04-25T14:03:56.772486+0000","last_peered":"2024-04-25T14:03:56.772486+0000","last_clean":"2024-04-25T14:03:56.772486+0000","last_became_active":"2024-04-25T14:03:50.463507+0000","last_became_peered":"2024-04-25T14:03:50.463507+0000","last_unstale":"2024-04-25T14:03:56.772486+0000","last_undegraded":"2024-04-25T14:03:56.772486+0000","last_fullsized":"2024-04-25T14:03:56.772486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:15.364599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494579+0000","last_change":"2024-04-25T14:03:55.676329+0000","last_active":"2024-04-25T14:03:56.494579+0000","last_peered":"2024-04-25T14:03:56.494579+0000","last_clean":"2024-04-25T14:03:56.494579+0000","last_became_active":"2024-04-25T14:03:55.675752+0000","last_became_peered":"2024-04-25T14:03:55.675752+0000","last_unstale":"2024-04-25T14:03:56.494579+0000","last_undegraded":"2024-04-25T14:03:56.494579+0000","last_fullsized":"2024-04-25T14:03:56.494579+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:28.167574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772572+0000","last_change":"2024-04-25T14:03:51.495282+0000","last_active":"2024-04-25T14:03:56.772572+0000","last_peered":"2024-04-25T14:03:56.772572+0000","last_clean":"2024-04-25T14:03:56.772572+0000","last_became_active":"2024-04-25T14:03:51.494965+0000","last_became_peered":"2024-04-25T14:03:51.494965+0000","last_unstale":"2024-04-25T14:03:56.772572+0000","last_undegraded":"2024-04-25T14:03:56.772572+0000","last_fullsized":"2024-04-25T14:03:56.772572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:02.773422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773963+0000","last_change":"2024-04-25T14:03:50.465925+0000","last_active":"2024-04-25T14:03:56.773963+0000","last_peered":"2024-04-25T14:03:56.773963+0000","last_clean":"2024-04-25T14:03:56.773963+0000","last_became_active":"2024-04-25T14:03:50.465495+0000","last_became_peered":"2024-04-25T14:03:50.465495+0000","last_unstale":"2024-04-25T14:03:56.773963+0000","last_undegraded":"2024-04-25T14:03:56.773963+0000","last_fullsized":"2024-04-25T14:03:56.773963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:15.520202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773183+0000","last_change":"2024-04-25T14:03:55.520108+0000","last_active":"2024-04-25T14:03:56.773183+0000","last_peered":"2024-04-25T14:03:56.773183+0000","last_clean":"2024-04-25T14:03:56.773183+0000","last_became_active":"2024-04-25T14:03:55.519374+0000","last_became_peered":"2024-04-25T14:03:55.519374+0000","last_unstale":"2024-04-25T14:03:56.773183+0000","last_undegraded":"2024-04-25T14:03:56.773183+0000","last_fullsized":"2024-04-25T14:03:56.773183+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:21.426226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772332+0000","last_change":"2024-04-25T14:03:51.488205+0000","last_active":"2024-04-25T14:03:56.772332+0000","last_peered":"2024-04-25T14:03:56.772332+0000","last_clean":"2024-04-25T14:03:56.772332+0000","last_became_active":"2024-04-25T14:03:51.487872+0000","last_became_peered":"2024-04-25T14:03:51.487872+0000","last_unstale":"2024-04-25T14:03:56.772332+0000","last_undegraded":"2024-04-25T14:03:56.772332+0000","last_fullsized":"2024-04-25T14:03:56.772332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:55.876775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.771697+0000","last_change":"2024-04-25T14:03:50.475566+0000","last_active":"2024-04-25T14:03:56.771697+0000","last_peered":"2024-04-25T14:03:56.771697+0000","last_clean":"2024-04-25T14:03:56.771697+0000","last_became_active":"2024-04-25T14:03:50.475304+0000","last_became_peered":"2024-04-25T14:03:50.475304+0000","last_unstale":"2024-04-25T14:03:56.771697+0000","last_undegraded":"2024-04-25T14:03:56.771697+0000","last_fullsized":"2024-04-25T14:03:56.771697+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:15.520202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774574+0000","last_change":"2024-04-25T14:03:55.523592+0000","last_active":"2024-04-25T14:03:56.774574+0000","last_peered":"2024-04-25T14:03:56.774574+0000","last_clean":"2024-04-25T14:03:56.774574+0000","last_became_active":"2024-04-25T14:03:55.521767+0000","last_became_peered":"2024-04-25T14:03:55.521767+0000","last_unstale":"2024-04-25T14:03:56.774574+0000","last_undegraded":"2024-04-25T14:03:56.774574+0000","last_fullsized":"2024-04-25T14:03:56.774574+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:50.234674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494779+0000","last_change":"2024-04-25T14:03:51.480142+0000","last_active":"2024-04-25T14:03:56.494779+0000","last_peered":"2024-04-25T14:03:56.494779+0000","last_clean":"2024-04-25T14:03:56.494779+0000","last_became_active":"2024-04-25T14:03:51.479798+0000","last_became_peered":"2024-04-25T14:03:51.479798+0000","last_unstale":"2024-04-25T14:03:56.494779+0000","last_undegraded":"2024-04-25T14:03:56.494779+0000","last_fullsized":"2024-04-25T14:03:56.494779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:31.840842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494679+0000","last_change":"2024-04-25T14:03:50.459894+0000","last_active":"2024-04-25T14:03:56.494679+0000","last_peered":"2024-04-25T14:03:56.494679+0000","last_clean":"2024-04-25T14:03:56.494679+0000","last_became_active":"2024-04-25T14:03:50.459580+0000","last_became_peered":"2024-04-25T14:03:50.459580+0000","last_unstale":"2024-04-25T14:03:56.494679+0000","last_undegraded":"2024-04-25T14:03:56.494679+0000","last_fullsized":"2024-04-25T14:03:56.494679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:29.264084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495170+0000","last_change":"2024-04-25T14:03:55.681298+0000","last_active":"2024-04-25T14:03:56.495170+0000","last_peered":"2024-04-25T14:03:56.495170+0000","last_clean":"2024-04-25T14:03:56.495170+0000","last_became_active":"2024-04-25T14:03:55.680746+0000","last_became_peered":"2024-04-25T14:03:55.680746+0000","last_unstale":"2024-04-25T14:03:56.495170+0000","last_undegraded":"2024-04-25T14:03:56.495170+0000","last_fullsized":"2024-04-25T14:03:56.495170+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:54.496413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772452+0000","last_change":"2024-04-25T14:03:51.499753+0000","last_active":"2024-04-25T14:03:56.772452+0000","last_peered":"2024-04-25T14:03:56.772452+0000","last_clean":"2024-04-25T14:03:56.772452+0000","last_became_active":"2024-04-25T14:03:51.499427+0000","last_became_peered":"2024-04-25T14:03:51.499427+0000","last_unstale":"2024-04-25T14:03:56.772452+0000","last_undegraded":"2024-04-25T14:03:56.772452+0000","last_fullsized":"2024-04-25T14:03:56.772452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:05.489889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495577+0000","last_change":"2024-04-25T14:03:50.467395+0000","last_active":"2024-04-25T14:03:56.495577+0000","last_peered":"2024-04-25T14:03:56.495577+0000","last_clean":"2024-04-25T14:03:56.495577+0000","last_became_active":"2024-04-25T14:03:50.465757+0000","last_became_peered":"2024-04-25T14:03:50.465757+0000","last_unstale":"2024-04-25T14:03:56.495577+0000","last_undegraded":"2024-04-25T14:03:56.495577+0000","last_fullsized":"2024-04-25T14:03:56.495577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:39.630327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494300+0000","last_change":"2024-04-25T14:03:55.676664+0000","last_active":"2024-04-25T14:03:56.494300+0000","last_peered":"2024-04-25T14:03:56.494300+0000","last_clean":"2024-04-25T14:03:56.494300+0000","last_became_active":"2024-04-25T14:03:55.674357+0000","last_became_peered":"2024-04-25T14:03:55.674357+0000","last_unstale":"2024-04-25T14:03:56.494300+0000","last_undegraded":"2024-04-25T14:03:56.494300+0000","last_fullsized":"2024-04-25T14:03:56.494300+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:55:42.758380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772783+0000","last_change":"2024-04-25T14:03:51.469041+0000","last_active":"2024-04-25T14:03:56.772783+0000","last_peered":"2024-04-25T14:03:56.772783+0000","last_clean":"2024-04-25T14:03:56.772783+0000","last_became_active":"2024-04-25T14:03:51.468704+0000","last_became_peered":"2024-04-25T14:03:51.468704+0000","last_unstale":"2024-04-25T14:03:56.772783+0000","last_undegraded":"2024-04-25T14:03:56.772783+0000","last_fullsized":"2024-04-25T14:03:56.772783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:31.840842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773670+0000","last_change":"2024-04-25T14:03:50.465211+0000","last_active":"2024-04-25T14:03:56.773670+0000","last_peered":"2024-04-25T14:03:56.773670+0000","last_clean":"2024-04-25T14:03:56.773670+0000","last_became_active":"2024-04-25T14:03:50.464836+0000","last_became_peered":"2024-04-25T14:03:50.464836+0000","last_unstale":"2024-04-25T14:03:56.773670+0000","last_undegraded":"2024-04-25T14:03:56.773670+0000","last_fullsized":"2024-04-25T14:03:56.773670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:04.876207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496500+0000","last_change":"2024-04-25T14:03:55.681339+0000","last_active":"2024-04-25T14:03:56.496500+0000","last_peered":"2024-04-25T14:03:56.496500+0000","last_clean":"2024-04-25T14:03:56.496500+0000","last_became_active":"2024-04-25T14:03:55.680779+0000","last_became_peered":"2024-04-25T14:03:55.680779+0000","last_unstale":"2024-04-25T14:03:56.496500+0000","last_undegraded":"2024-04-25T14:03:56.496500+0000","last_fullsized":"2024-04-25T14:03:56.496500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:58.844877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772688+0000","last_change":"2024-04-25T14:03:51.476195+0000","last_active":"2024-04-25T14:03:56.772688+0000","last_peered":"2024-04-25T14:03:56.772688+0000","last_clean":"2024-04-25T14:03:56.772688+0000","last_became_active":"2024-04-25T14:03:51.475372+0000","last_became_peered":"2024-04-25T14:03:51.475372+0000","last_unstale":"2024-04-25T14:03:56.772688+0000","last_undegraded":"2024-04-25T14:03:56.772688+0000","last_fullsized":"2024-04-25T14:03:56.772688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:47.326356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:04:06.748698+0000","last_change":"2024-04-25T14:03:50.470612+0000","last_active":"2024-04-25T14:04:06.748698+0000","last_peered":"2024-04-25T14:04:06.748698+0000","last_clean":"2024-04-25T14:04:06.748698+0000","last_became_active":"2024-04-25T14:03:50.470312+0000","last_became_peered":"2024-04-25T14:03:50.470312+0000","last_unstale":"2024-04-25T14:04:06.748698+0000","last_undegraded":"2024-04-25T14:04:06.748698+0000","last_fullsized":"2024-04-25T14:04:06.748698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:10.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773458+0000","last_change":"2024-04-25T14:03:55.526774+0000","last_active":"2024-04-25T14:03:56.773458+0000","last_peered":"2024-04-25T14:03:56.773458+0000","last_clean":"2024-04-25T14:03:56.773458+0000","last_became_active":"2024-04-25T14:03:55.524255+0000","last_became_peered":"2024-04-25T14:03:55.524255+0000","last_unstale":"2024-04-25T14:03:56.773458+0000","last_undegraded":"2024-04-25T14:03:56.773458+0000","last_fullsized":"2024-04-25T14:03:56.773458+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:37.811376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773340+0000","last_change":"2024-04-25T14:03:51.493162+0000","last_active":"2024-04-25T14:03:56.773340+0000","last_peered":"2024-04-25T14:03:56.773340+0000","last_clean":"2024-04-25T14:03:56.773340+0000","last_became_active":"2024-04-25T14:03:51.488560+0000","last_became_peered":"2024-04-25T14:03:51.488560+0000","last_unstale":"2024-04-25T14:03:56.773340+0000","last_undegraded":"2024-04-25T14:03:56.773340+0000","last_fullsized":"2024-04-25T14:03:56.773340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:40.635789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774009+0000","last_change":"2024-04-25T14:03:50.469594+0000","last_active":"2024-04-25T14:03:56.774009+0000","last_peered":"2024-04-25T14:03:56.774009+0000","last_clean":"2024-04-25T14:03:56.774009+0000","last_became_active":"2024-04-25T14:03:50.469196+0000","last_became_peered":"2024-04-25T14:03:50.469196+0000","last_unstale":"2024-04-25T14:03:56.774009+0000","last_undegraded":"2024-04-25T14:03:56.774009+0000","last_fullsized":"2024-04-25T14:03:56.774009+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:15.364599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772969+0000","last_change":"2024-04-25T14:03:55.518510+0000","last_active":"2024-04-25T14:03:56.772969+0000","last_peered":"2024-04-25T14:03:56.772969+0000","last_clean":"2024-04-25T14:03:56.772969+0000","last_became_active":"2024-04-25T14:03:55.515796+0000","last_became_peered":"2024-04-25T14:03:55.515796+0000","last_unstale":"2024-04-25T14:03:56.772969+0000","last_undegraded":"2024-04-25T14:03:56.772969+0000","last_fullsized":"2024-04-25T14:03:56.772969+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:59.921752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774672+0000","last_change":"2024-04-25T14:03:51.486474+0000","last_active":"2024-04-25T14:03:56.774672+0000","last_peered":"2024-04-25T14:03:56.774672+0000","last_clean":"2024-04-25T14:03:56.774672+0000","last_became_active":"2024-04-25T14:03:51.485742+0000","last_became_peered":"2024-04-25T14:03:51.485742+0000","last_unstale":"2024-04-25T14:03:56.774672+0000","last_undegraded":"2024-04-25T14:03:56.774672+0000","last_fullsized":"2024-04-25T14:03:56.774672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:16.813529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496428+0000","last_change":"2024-04-25T14:03:50.475187+0000","last_active":"2024-04-25T14:03:56.496428+0000","last_peered":"2024-04-25T14:03:56.496428+0000","last_clean":"2024-04-25T14:03:56.496428+0000","last_became_active":"2024-04-25T14:03:50.474761+0000","last_became_peered":"2024-04-25T14:03:50.474761+0000","last_unstale":"2024-04-25T14:03:56.496428+0000","last_undegraded":"2024-04-25T14:03:56.496428+0000","last_fullsized":"2024-04-25T14:03:56.496428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:02.117945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493593+0000","last_change":"2024-04-25T14:03:55.524048+0000","last_active":"2024-04-25T14:03:56.493593+0000","last_peered":"2024-04-25T14:03:56.493593+0000","last_clean":"2024-04-25T14:03:56.493593+0000","last_became_active":"2024-04-25T14:03:55.520419+0000","last_became_peered":"2024-04-25T14:03:55.520419+0000","last_unstale":"2024-04-25T14:03:56.493593+0000","last_undegraded":"2024-04-25T14:03:56.493593+0000","last_fullsized":"2024-04-25T14:03:56.493593+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:30.069597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496533+0000","last_change":"2024-04-25T14:03:51.467640+0000","last_active":"2024-04-25T14:03:56.496533+0000","last_peered":"2024-04-25T14:03:56.496533+0000","last_clean":"2024-04-25T14:03:56.496533+0000","last_became_active":"2024-04-25T14:03:51.467347+0000","last_became_peered":"2024-04-25T14:03:51.467347+0000","last_unstale":"2024-04-25T14:03:56.496533+0000","last_undegraded":"2024-04-25T14:03:56.496533+0000","last_fullsized":"2024-04-25T14:03:56.496533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:33.766399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496540+0000","last_change":"2024-04-25T14:03:50.462889+0000","last_active":"2024-04-25T14:03:56.496540+0000","last_peered":"2024-04-25T14:03:56.496540+0000","last_clean":"2024-04-25T14:03:56.496540+0000","last_became_active":"2024-04-25T14:03:50.462529+0000","last_became_peered":"2024-04-25T14:03:50.462529+0000","last_unstale":"2024-04-25T14:03:56.496540+0000","last_undegraded":"2024-04-25T14:03:56.496540+0000","last_fullsized":"2024-04-25T14:03:56.496540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:15.364599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493822+0000","last_change":"2024-04-25T14:03:55.675236+0000","last_active":"2024-04-25T14:03:56.493822+0000","last_peered":"2024-04-25T14:03:56.493822+0000","last_clean":"2024-04-25T14:03:56.493822+0000","last_became_active":"2024-04-25T14:03:55.674759+0000","last_became_peered":"2024-04-25T14:03:55.674759+0000","last_unstale":"2024-04-25T14:03:56.493822+0000","last_undegraded":"2024-04-25T14:03:56.493822+0000","last_fullsized":"2024-04-25T14:03:56.493822+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:28.167574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773362+0000","last_change":"2024-04-25T14:03:55.677958+0000","last_active":"2024-04-25T14:03:56.773362+0000","last_peered":"2024-04-25T14:03:56.773362+0000","last_clean":"2024-04-25T14:03:56.773362+0000","last_became_active":"2024-04-25T14:03:55.677471+0000","last_became_peered":"2024-04-25T14:03:55.677471+0000","last_unstale":"2024-04-25T14:03:56.773362+0000","last_undegraded":"2024-04-25T14:03:56.773362+0000","last_fullsized":"2024-04-25T14:03:56.773362+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:12.890323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772866+0000","last_change":"2024-04-25T14:03:51.477601+0000","last_active":"2024-04-25T14:03:56.772866+0000","last_peered":"2024-04-25T14:03:56.772866+0000","last_clean":"2024-04-25T14:03:56.772866+0000","last_became_active":"2024-04-25T14:03:51.477318+0000","last_became_peered":"2024-04-25T14:03:51.477318+0000","last_unstale":"2024-04-25T14:03:56.772866+0000","last_undegraded":"2024-04-25T14:03:56.772866+0000","last_fullsized":"2024-04-25T14:03:56.772866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.599283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494934+0000","last_change":"2024-04-25T14:03:50.456493+0000","last_active":"2024-04-25T14:03:56.494934+0000","last_peered":"2024-04-25T14:03:56.494934+0000","last_clean":"2024-04-25T14:03:56.494934+0000","last_became_active":"2024-04-25T14:03:50.454242+0000","last_became_peered":"2024-04-25T14:03:50.454242+0000","last_unstale":"2024-04-25T14:03:56.494934+0000","last_undegraded":"2024-04-25T14:03:56.494934+0000","last_fullsized":"2024-04-25T14:03:56.494934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:10.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496401+0000","last_change":"2024-04-25T14:03:55.516548+0000","last_active":"2024-04-25T14:03:56.496401+0000","last_peered":"2024-04-25T14:03:56.496401+0000","last_clean":"2024-04-25T14:03:56.496401+0000","last_became_active":"2024-04-25T14:03:55.515311+0000","last_became_peered":"2024-04-25T14:03:55.515311+0000","last_unstale":"2024-04-25T14:03:56.496401+0000","last_undegraded":"2024-04-25T14:03:56.496401+0000","last_fullsized":"2024-04-25T14:03:56.496401+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:51.371333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493692+0000","last_change":"2024-04-25T14:03:51.468740+0000","last_active":"2024-04-25T14:03:56.493692+0000","last_peered":"2024-04-25T14:03:56.493692+0000","last_clean":"2024-04-25T14:03:56.493692+0000","last_became_active":"2024-04-25T14:03:51.468366+0000","last_became_peered":"2024-04-25T14:03:51.468366+0000","last_unstale":"2024-04-25T14:03:56.493692+0000","last_undegraded":"2024-04-25T14:03:56.493692+0000","last_fullsized":"2024-04-25T14:03:56.493692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:31.707048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493992+0000","last_change":"2024-04-25T14:03:50.466457+0000","last_active":"2024-04-25T14:03:56.493992+0000","last_peered":"2024-04-25T14:03:56.493992+0000","last_clean":"2024-04-25T14:03:56.493992+0000","last_became_active":"2024-04-25T14:03:50.466070+0000","last_became_peered":"2024-04-25T14:03:50.466070+0000","last_unstale":"2024-04-25T14:03:56.493992+0000","last_undegraded":"2024-04-25T14:03:56.493992+0000","last_fullsized":"2024-04-25T14:03:56.493992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:15.520202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495294+0000","last_change":"2024-04-25T14:03:50.469128+0000","last_active":"2024-04-25T14:03:56.495294+0000","last_peered":"2024-04-25T14:03:56.495294+0000","last_clean":"2024-04-25T14:03:56.495294+0000","last_became_active":"2024-04-25T14:03:50.468409+0000","last_became_peered":"2024-04-25T14:03:50.468409+0000","last_unstale":"2024-04-25T14:03:56.495294+0000","last_undegraded":"2024-04-25T14:03:56.495294+0000","last_fullsized":"2024-04-25T14:03:56.495294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:15.520202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773570+0000","last_change":"2024-04-25T14:03:51.502539+0000","last_active":"2024-04-25T14:03:56.773570+0000","last_peered":"2024-04-25T14:03:56.773570+0000","last_clean":"2024-04-25T14:03:56.773570+0000","last_became_active":"2024-04-25T14:03:51.502229+0000","last_became_peered":"2024-04-25T14:03:51.502229+0000","last_unstale":"2024-04-25T14:03:56.773570+0000","last_undegraded":"2024-04-25T14:03:56.773570+0000","last_fullsized":"2024-04-25T14:03:56.773570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:20.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772371+0000","last_change":"2024-04-25T14:03:55.524446+0000","last_active":"2024-04-25T14:03:56.772371+0000","last_peered":"2024-04-25T14:03:56.772371+0000","last_clean":"2024-04-25T14:03:56.772371+0000","last_became_active":"2024-04-25T14:03:55.523886+0000","last_became_peered":"2024-04-25T14:03:55.523886+0000","last_unstale":"2024-04-25T14:03:56.772371+0000","last_undegraded":"2024-04-25T14:03:56.772371+0000","last_fullsized":"2024-04-25T14:03:56.772371+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:37.811235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493654+0000","last_change":"2024-04-25T14:03:50.472564+0000","last_active":"2024-04-25T14:03:56.493654+0000","last_peered":"2024-04-25T14:03:56.493654+0000","last_clean":"2024-04-25T14:03:56.493654+0000","last_became_active":"2024-04-25T14:03:50.472297+0000","last_became_peered":"2024-04-25T14:03:50.472297+0000","last_unstale":"2024-04-25T14:03:56.493654+0000","last_undegraded":"2024-04-25T14:03:56.493654+0000","last_fullsized":"2024-04-25T14:03:56.493654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:43.281768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772419+0000","last_change":"2024-04-25T14:03:51.486031+0000","last_active":"2024-04-25T14:03:56.772419+0000","last_peered":"2024-04-25T14:03:56.772419+0000","last_clean":"2024-04-25T14:03:56.772419+0000","last_became_active":"2024-04-25T14:03:51.477708+0000","last_became_peered":"2024-04-25T14:03:51.477708+0000","last_unstale":"2024-04-25T14:03:56.772419+0000","last_undegraded":"2024-04-25T14:03:56.772419+0000","last_fullsized":"2024-04-25T14:03:56.772419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:11.859288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494673+0000","last_change":"2024-04-25T14:03:55.519123+0000","last_active":"2024-04-25T14:03:56.494673+0000","last_peered":"2024-04-25T14:03:56.494673+0000","last_clean":"2024-04-25T14:03:56.494673+0000","last_became_active":"2024-04-25T14:03:55.518743+0000","last_became_peered":"2024-04-25T14:03:55.518743+0000","last_unstale":"2024-04-25T14:03:56.494673+0000","last_undegraded":"2024-04-25T14:03:56.494673+0000","last_fullsized":"2024-04-25T14:03:56.494673+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:18.368759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494540+0000","last_change":"2024-04-25T14:03:51.480209+0000","last_active":"2024-04-25T14:03:56.494540+0000","last_peered":"2024-04-25T14:03:56.494540+0000","last_clean":"2024-04-25T14:03:56.494540+0000","last_became_active":"2024-04-25T14:03:51.479824+0000","last_became_peered":"2024-04-25T14:03:51.479824+0000","last_unstale":"2024-04-25T14:03:56.494540+0000","last_undegraded":"2024-04-25T14:03:56.494540+0000","last_fullsized":"2024-04-25T14:03:56.494540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:31.707048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.771783+0000","last_change":"2024-04-25T14:03:51.486748+0000","last_active":"2024-04-25T14:03:56.771783+0000","last_peered":"2024-04-25T14:03:56.771783+0000","last_clean":"2024-04-25T14:03:56.771783+0000","last_became_active":"2024-04-25T14:03:51.486343+0000","last_became_peered":"2024-04-25T14:03:51.486343+0000","last_unstale":"2024-04-25T14:03:56.771783+0000","last_undegraded":"2024-04-25T14:03:56.771783+0000","last_fullsized":"2024-04-25T14:03:56.771783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:46.086179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495470+0000","last_change":"2024-04-25T14:03:51.465637+0000","last_active":"2024-04-25T14:03:56.495470+0000","last_peered":"2024-04-25T14:03:56.495470+0000","last_clean":"2024-04-25T14:03:56.495470+0000","last_became_active":"2024-04-25T14:03:51.465339+0000","last_became_peered":"2024-04-25T14:03:51.465339+0000","last_unstale":"2024-04-25T14:03:56.495470+0000","last_undegraded":"2024-04-25T14:03:56.495470+0000","last_fullsized":"2024-04-25T14:03:56.495470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:21.406920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773508+0000","last_change":"2024-04-25T14:03:51.475134+0000","last_active":"2024-04-25T14:03:56.773508+0000","last_peered":"2024-04-25T14:03:56.773508+0000","last_clean":"2024-04-25T14:03:56.773508+0000","last_became_active":"2024-04-25T14:03:51.474796+0000","last_became_peered":"2024-04-25T14:03:51.474796+0000","last_unstale":"2024-04-25T14:03:56.773508+0000","last_undegraded":"2024-04-25T14:03:56.773508+0000","last_fullsized":"2024-04-25T14:03:56.773508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:31.707048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494247+0000","last_change":"2024-04-25T14:03:51.494845+0000","last_active":"2024-04-25T14:03:56.494247+0000","last_peered":"2024-04-25T14:03:56.494247+0000","last_clean":"2024-04-25T14:03:56.494247+0000","last_became_active":"2024-04-25T14:03:51.494547+0000","last_became_peered":"2024-04-25T14:03:51.494547+0000","last_unstale":"2024-04-25T14:03:56.494247+0000","last_undegraded":"2024-04-25T14:03:56.494247+0000","last_fullsized":"2024-04-25T14:03:56.494247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:16.813529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773722+0000","last_change":"2024-04-25T14:03:51.478241+0000","last_active":"2024-04-25T14:03:56.773722+0000","last_peered":"2024-04-25T14:03:56.773722+0000","last_clean":"2024-04-25T14:03:56.773722+0000","last_became_active":"2024-04-25T14:03:51.477955+0000","last_became_peered":"2024-04-25T14:03:51.477955+0000","last_unstale":"2024-04-25T14:03:56.773722+0000","last_undegraded":"2024-04-25T14:03:56.773722+0000","last_fullsized":"2024-04-25T14:03:56.773722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:43.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496093+0000","last_change":"2024-04-25T14:03:51.478644+0000","last_active":"2024-04-25T14:03:56.496093+0000","last_peered":"2024-04-25T14:03:56.496093+0000","last_clean":"2024-04-25T14:03:56.496093+0000","last_became_active":"2024-04-25T14:03:51.478326+0000","last_became_peered":"2024-04-25T14:03:51.478326+0000","last_unstale":"2024-04-25T14:03:56.496093+0000","last_undegraded":"2024-04-25T14:03:56.496093+0000","last_fullsized":"2024-04-25T14:03:56.496093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:20.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495116+0000","last_change":"2024-04-25T14:03:51.506061+0000","last_active":"2024-04-25T14:03:56.495116+0000","last_peered":"2024-04-25T14:03:56.495116+0000","last_clean":"2024-04-25T14:03:56.495116+0000","last_became_active":"2024-04-25T14:03:51.505804+0000","last_became_peered":"2024-04-25T14:03:51.505804+0000","last_unstale":"2024-04-25T14:03:56.495116+0000","last_undegraded":"2024-04-25T14:03:56.495116+0000","last_fullsized":"2024-04-25T14:03:56.495116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:55.876775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494831+0000","last_change":"2024-04-25T14:03:51.465574+0000","last_active":"2024-04-25T14:03:56.494831+0000","last_peered":"2024-04-25T14:03:56.494831+0000","last_clean":"2024-04-25T14:03:56.494831+0000","last_became_active":"2024-04-25T14:03:51.465206+0000","last_became_peered":"2024-04-25T14:03:51.465206+0000","last_unstale":"2024-04-25T14:03:56.494831+0000","last_undegraded":"2024-04-25T14:03:56.494831+0000","last_fullsized":"2024-04-25T14:03:56.494831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:16.813529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493905+0000","last_change":"2024-04-25T14:03:51.486782+0000","last_active":"2024-04-25T14:03:56.493905+0000","last_peered":"2024-04-25T14:03:56.493905+0000","last_clean":"2024-04-25T14:03:56.493905+0000","last_became_active":"2024-04-25T14:03:51.486465+0000","last_became_peered":"2024-04-25T14:03:51.486465+0000","last_unstale":"2024-04-25T14:03:56.493905+0000","last_undegraded":"2024-04-25T14:03:56.493905+0000","last_fullsized":"2024-04-25T14:03:56.493905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:39.949870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495357+0000","last_change":"2024-04-25T14:03:51.468022+0000","last_active":"2024-04-25T14:03:56.495357+0000","last_peered":"2024-04-25T14:03:56.495357+0000","last_clean":"2024-04-25T14:03:56.495357+0000","last_became_active":"2024-04-25T14:03:51.467637+0000","last_became_peered":"2024-04-25T14:03:51.467637+0000","last_unstale":"2024-04-25T14:03:56.495357+0000","last_undegraded":"2024-04-25T14:03:56.495357+0000","last_fullsized":"2024-04-25T14:03:56.495357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:55.876775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773216+0000","last_change":"2024-04-25T14:03:51.469881+0000","last_active":"2024-04-25T14:03:56.773216+0000","last_peered":"2024-04-25T14:03:56.773216+0000","last_clean":"2024-04-25T14:03:56.773216+0000","last_became_active":"2024-04-25T14:03:51.469572+0000","last_became_peered":"2024-04-25T14:03:51.469572+0000","last_unstale":"2024-04-25T14:03:56.773216+0000","last_undegraded":"2024-04-25T14:03:56.773216+0000","last_fullsized":"2024-04-25T14:03:56.773216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.599283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495680+0000","last_change":"2024-04-25T14:03:51.494605+0000","last_active":"2024-04-25T14:03:56.495680+0000","last_peered":"2024-04-25T14:03:56.495680+0000","last_clean":"2024-04-25T14:03:56.495680+0000","last_became_active":"2024-04-25T14:03:51.494257+0000","last_became_peered":"2024-04-25T14:03:51.494257+0000","last_unstale":"2024-04-25T14:03:56.495680+0000","last_undegraded":"2024-04-25T14:03:56.495680+0000","last_fullsized":"2024-04-25T14:03:56.495680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:39.949870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773065+0000","last_change":"2024-04-25T14:03:51.476473+0000","last_active":"2024-04-25T14:03:56.773065+0000","last_peered":"2024-04-25T14:03:56.773065+0000","last_clean":"2024-04-25T14:03:56.773065+0000","last_became_active":"2024-04-25T14:03:51.475857+0000","last_became_peered":"2024-04-25T14:03:51.475857+0000","last_unstale":"2024-04-25T14:03:56.773065+0000","last_undegraded":"2024-04-25T14:03:56.773065+0000","last_fullsized":"2024-04-25T14:03:56.773065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:46.189697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772375+0000","last_change":"2024-04-25T14:03:51.479889+0000","last_active":"2024-04-25T14:03:56.772375+0000","last_peered":"2024-04-25T14:03:56.772375+0000","last_clean":"2024-04-25T14:03:56.772375+0000","last_became_active":"2024-04-25T14:03:51.478793+0000","last_became_peered":"2024-04-25T14:03:51.478793+0000","last_unstale":"2024-04-25T14:03:56.772375+0000","last_undegraded":"2024-04-25T14:03:56.772375+0000","last_fullsized":"2024-04-25T14:03:56.772375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:38.411753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496637+0000","last_change":"2024-04-25T14:03:51.480313+0000","last_active":"2024-04-25T14:03:56.496637+0000","last_peered":"2024-04-25T14:03:56.496637+0000","last_clean":"2024-04-25T14:03:56.496637+0000","last_became_active":"2024-04-25T14:03:51.479875+0000","last_became_peered":"2024-04-25T14:03:51.479875+0000","last_unstale":"2024-04-25T14:03:56.496637+0000","last_undegraded":"2024-04-25T14:03:56.496637+0000","last_fullsized":"2024-04-25T14:03:56.496637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:02.773422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494729+0000","last_change":"2024-04-25T14:03:51.472514+0000","last_active":"2024-04-25T14:03:56.494729+0000","last_peered":"2024-04-25T14:03:56.494729+0000","last_clean":"2024-04-25T14:03:56.494729+0000","last_became_active":"2024-04-25T14:03:51.472148+0000","last_became_peered":"2024-04-25T14:03:51.472148+0000","last_unstale":"2024-04-25T14:03:56.494729+0000","last_undegraded":"2024-04-25T14:03:56.494729+0000","last_fullsized":"2024-04-25T14:03:56.494729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:20.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772899+0000","last_change":"2024-04-25T14:03:51.489086+0000","last_active":"2024-04-25T14:03:56.772899+0000","last_peered":"2024-04-25T14:03:56.772899+0000","last_clean":"2024-04-25T14:03:56.772899+0000","last_became_active":"2024-04-25T14:03:51.488810+0000","last_became_peered":"2024-04-25T14:03:51.488810+0000","last_unstale":"2024-04-25T14:03:56.772899+0000","last_undegraded":"2024-04-25T14:03:56.772899+0000","last_fullsized":"2024-04-25T14:03:56.772899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:39.949870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772674+0000","last_change":"2024-04-25T14:03:51.475508+0000","last_active":"2024-04-25T14:03:56.772674+0000","last_peered":"2024-04-25T14:03:56.772674+0000","last_clean":"2024-04-25T14:03:56.772674+0000","last_became_active":"2024-04-25T14:03:51.475146+0000","last_became_peered":"2024-04-25T14:03:51.475146+0000","last_unstale":"2024-04-25T14:03:56.772674+0000","last_undegraded":"2024-04-25T14:03:56.772674+0000","last_fullsized":"2024-04-25T14:03:56.772674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:34.374126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493554+0000","last_change":"2024-04-25T14:03:51.479243+0000","last_active":"2024-04-25T14:03:56.493554+0000","last_peered":"2024-04-25T14:03:56.493554+0000","last_clean":"2024-04-25T14:03:56.493554+0000","last_became_active":"2024-04-25T14:03:51.478899+0000","last_became_peered":"2024-04-25T14:03:51.478899+0000","last_unstale":"2024-04-25T14:03:56.493554+0000","last_undegraded":"2024-04-25T14:03:56.493554+0000","last_fullsized":"2024-04-25T14:03:56.493554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:20.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773258+0000","last_change":"2024-04-25T14:03:51.493878+0000","last_active":"2024-04-25T14:03:56.773258+0000","last_peered":"2024-04-25T14:03:56.773258+0000","last_clean":"2024-04-25T14:03:56.773258+0000","last_became_active":"2024-04-25T14:03:51.493501+0000","last_became_peered":"2024-04-25T14:03:51.493501+0000","last_unstale":"2024-04-25T14:03:56.773258+0000","last_undegraded":"2024-04-25T14:03:56.773258+0000","last_fullsized":"2024-04-25T14:03:56.773258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:46.086179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496241+0000","last_change":"2024-04-25T14:03:51.489072+0000","last_active":"2024-04-25T14:03:56.496241+0000","last_peered":"2024-04-25T14:03:56.496241+0000","last_clean":"2024-04-25T14:03:56.496241+0000","last_became_active":"2024-04-25T14:03:51.488895+0000","last_became_peered":"2024-04-25T14:03:51.488895+0000","last_unstale":"2024-04-25T14:03:56.496241+0000","last_undegraded":"2024-04-25T14:03:56.496241+0000","last_fullsized":"2024-04-25T14:03:56.496241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:10.318378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496156+0000","last_change":"2024-04-25T14:03:51.488196+0000","last_active":"2024-04-25T14:03:56.496156+0000","last_peered":"2024-04-25T14:03:56.496156+0000","last_clean":"2024-04-25T14:03:56.496156+0000","last_became_active":"2024-04-25T14:03:51.487833+0000","last_became_peered":"2024-04-25T14:03:51.487833+0000","last_unstale":"2024-04-25T14:03:56.496156+0000","last_undegraded":"2024-04-25T14:03:56.496156+0000","last_fullsized":"2024-04-25T14:03:56.496156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:43.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773432+0000","last_change":"2024-04-25T14:03:51.482441+0000","last_active":"2024-04-25T14:03:56.773432+0000","last_peered":"2024-04-25T14:03:56.773432+0000","last_clean":"2024-04-25T14:03:56.773432+0000","last_became_active":"2024-04-25T14:03:51.482089+0000","last_became_peered":"2024-04-25T14:03:51.482089+0000","last_unstale":"2024-04-25T14:03:56.773432+0000","last_undegraded":"2024-04-25T14:03:56.773432+0000","last_fullsized":"2024-04-25T14:03:56.773432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:02.697952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493326+0000","last_change":"2024-04-25T14:03:51.487289+0000","last_active":"2024-04-25T14:03:56.493326+0000","last_peered":"2024-04-25T14:03:56.493326+0000","last_clean":"2024-04-25T14:03:56.493326+0000","last_became_active":"2024-04-25T14:03:51.486948+0000","last_became_peered":"2024-04-25T14:03:51.486948+0000","last_unstale":"2024-04-25T14:03:56.493326+0000","last_undegraded":"2024-04-25T14:03:56.493326+0000","last_fullsized":"2024-04-25T14:03:56.493326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.599283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772227+0000","last_change":"2024-04-25T14:03:51.473228+0000","last_active":"2024-04-25T14:03:56.772227+0000","last_peered":"2024-04-25T14:03:56.772227+0000","last_clean":"2024-04-25T14:03:56.772227+0000","last_became_active":"2024-04-25T14:03:51.472850+0000","last_became_peered":"2024-04-25T14:03:51.472850+0000","last_unstale":"2024-04-25T14:03:56.772227+0000","last_undegraded":"2024-04-25T14:03:56.772227+0000","last_fullsized":"2024-04-25T14:03:56.772227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:34.707119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495222+0000","last_change":"2024-04-25T14:03:51.467162+0000","last_active":"2024-04-25T14:03:56.495222+0000","last_peered":"2024-04-25T14:03:56.495222+0000","last_clean":"2024-04-25T14:03:56.495222+0000","last_became_active":"2024-04-25T14:03:51.466616+0000","last_became_peered":"2024-04-25T14:03:51.466616+0000","last_unstale":"2024-04-25T14:03:56.495222+0000","last_undegraded":"2024-04-25T14:03:56.495222+0000","last_fullsized":"2024-04-25T14:03:56.495222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:34.707119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772199+0000","last_change":"2024-04-25T14:03:51.476149+0000","last_active":"2024-04-25T14:03:56.772199+0000","last_peered":"2024-04-25T14:03:56.772199+0000","last_clean":"2024-04-25T14:03:56.772199+0000","last_became_active":"2024-04-25T14:03:51.475934+0000","last_became_peered":"2024-04-25T14:03:51.475934+0000","last_unstale":"2024-04-25T14:03:56.772199+0000","last_undegraded":"2024-04-25T14:03:56.772199+0000","last_fullsized":"2024-04-25T14:03:56.772199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:17.381249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496219+0000","last_change":"2024-04-25T14:03:51.475153+0000","last_active":"2024-04-25T14:03:56.496219+0000","last_peered":"2024-04-25T14:03:56.496219+0000","last_clean":"2024-04-25T14:03:56.496219+0000","last_became_active":"2024-04-25T14:03:51.474862+0000","last_became_peered":"2024-04-25T14:03:51.474862+0000","last_unstale":"2024-04-25T14:03:56.496219+0000","last_undegraded":"2024-04-25T14:03:56.496219+0000","last_fullsized":"2024-04-25T14:03:56.496219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:05.489889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773218+0000","last_change":"2024-04-25T14:03:51.460567+0000","last_active":"2024-04-25T14:03:56.773218+0000","last_peered":"2024-04-25T14:03:56.773218+0000","last_clean":"2024-04-25T14:03:56.773218+0000","last_became_active":"2024-04-25T14:03:51.460193+0000","last_became_peered":"2024-04-25T14:03:51.460193+0000","last_unstale":"2024-04-25T14:03:56.773218+0000","last_undegraded":"2024-04-25T14:03:56.773218+0000","last_fullsized":"2024-04-25T14:03:56.773218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:39.949870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774153+0000","last_change":"2024-04-25T14:03:51.480094+0000","last_active":"2024-04-25T14:03:56.774153+0000","last_peered":"2024-04-25T14:03:56.774153+0000","last_clean":"2024-04-25T14:03:56.774153+0000","last_became_active":"2024-04-25T14:03:51.479529+0000","last_became_peered":"2024-04-25T14:03:51.479529+0000","last_unstale":"2024-04-25T14:03:56.774153+0000","last_undegraded":"2024-04-25T14:03:56.774153+0000","last_fullsized":"2024-04-25T14:03:56.774153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:10.318378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773366+0000","last_change":"2024-04-25T14:03:51.481397+0000","last_active":"2024-04-25T14:03:56.773366+0000","last_peered":"2024-04-25T14:03:56.773366+0000","last_clean":"2024-04-25T14:03:56.773366+0000","last_became_active":"2024-04-25T14:03:51.481153+0000","last_became_peered":"2024-04-25T14:03:51.481153+0000","last_unstale":"2024-04-25T14:03:56.773366+0000","last_undegraded":"2024-04-25T14:03:56.773366+0000","last_fullsized":"2024-04-25T14:03:56.773366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:21.406920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:04:11.323 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T14:04:11.323 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T14:04:11.323 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T14:04:11.324 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T14:04:11.324 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T14:04:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.505+0000 7fcb92800640 1 Processor -- start 2024-04-25T14:04:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.505+0000 7fcb92800640 1 -- start start 2024-04-25T14:04:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.505+0000 7fcb92800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcb8c071530 0x7fcb8c06e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.505+0000 7fcb92800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcb8c06e7a0 0x7fcb8c0cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.505+0000 7fcb92800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb8c06ef90 con 0x7fcb8c0cf970 2024-04-25T14:04:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.505+0000 7fcb92800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcb8c163830 con 0x7fcb8c06e7a0 2024-04-25T14:04:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.505+0000 7fcb92800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fcb8c163a80 con 0x7fcb8c071530 2024-04-25T14:04:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.505+0000 7fcb8be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcb8c06e7a0 0x7fcb8c0cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.505+0000 7fcb8be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcb8c06e7a0 0x7fcb8c0cdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37860/0 (socket says 172.21.15.86:37860) 2024-04-25T14:04:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.505+0000 7fcb8be00640 1 -- 172.21.15.86:0/304790714 learned_addr learned my addr 172.21.15.86:0/304790714 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.505+0000 7fcb82a00640 1 --2- 172.21.15.86:0/304790714 >> v2:172.21.15.155:3300/0 conn(0x7fcb8c071530 0x7fcb8c06e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.509+0000 7fcb8be00640 1 -- 172.21.15.86:0/304790714 >> v2:172.21.15.155:3300/0 conn(0x7fcb8c071530 msgr2=0x7fcb8c06e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.509+0000 7fcb8be00640 1 --2- 172.21.15.86:0/304790714 >> v2:172.21.15.155:3300/0 conn(0x7fcb8c071530 0x7fcb8c06e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:11.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.509+0000 7fcb8be00640 1 -- 172.21.15.86:0/304790714 >> v1:172.21.15.155:6789/0 conn(0x7fcb8c0cf970 legacy=0x7fcb8c070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:11.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.509+0000 7fcb8be00640 1 -- 172.21.15.86:0/304790714 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8c163d60 con 0x7fcb8c06e7a0 2024-04-25T14:04:11.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.509+0000 7fcb8be00640 1 --2- 172.21.15.86:0/304790714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8c06e7a0 0x7fcb8c0cdde0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c1639b0 tx=0x7fcb8401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb81165f21e976c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:11.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.509+0000 7fcb8aa00640 1 -- 172.21.15.86:0/304790714 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb840aa670 con 0x7fcb8c06e7a0 2024-04-25T14:04:11.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.509+0000 7fcb8aa00640 1 -- 172.21.15.86:0/304790714 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb840af070 con 0x7fcb8c06e7a0 2024-04-25T14:04:11.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.509+0000 7fcb8aa00640 1 -- 172.21.15.86:0/304790714 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb840aaa30 con 0x7fcb8c06e7a0 2024-04-25T14:04:11.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.513+0000 7fcb92800640 1 -- 172.21.15.86:0/304790714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8c06e7a0 msgr2=0x7fcb8c0cdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:11.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.513+0000 7fcb92800640 1 --2- 172.21.15.86:0/304790714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8c06e7a0 0x7fcb8c0cdde0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c1639b0 tx=0x7fcb8401c4f0 comp rx=0 tx=0).stop 2024-04-25T14:04:11.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.513+0000 7fcb92800640 1 -- 172.21.15.86:0/304790714 shutdown_connections 2024-04-25T14:04:11.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.513+0000 7fcb92800640 1 --2- 172.21.15.86:0/304790714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8c06e7a0 0x7fcb8c0cdde0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:11.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.513+0000 7fcb92800640 1 --2- 172.21.15.86:0/304790714 >> v2:172.21.15.155:3300/0 conn(0x7fcb8c071530 0x7fcb8c06e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:11.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.513+0000 7fcb92800640 1 -- 172.21.15.86:0/304790714 >> 172.21.15.86:0/304790714 conn(0x7fcb8c0cffc0 msgr2=0x7fcb8c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:11.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.513+0000 7fcb92800640 1 -- 172.21.15.86:0/304790714 shutdown_connections 2024-04-25T14:04:11.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.513+0000 7fcb92800640 1 -- 172.21.15.86:0/304790714 wait complete. 2024-04-25T14:04:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.513+0000 7fcb92800640 1 Processor -- start 2024-04-25T14:04:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.513+0000 7fcb92800640 1 -- start start 2024-04-25T14:04:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.517+0000 7fcb92800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb8c06e7a0 0x7fcb8c0cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.517+0000 7fcb92800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8c0cf970 0x7fcb8c15e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.517+0000 7fcb92800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb8c0ca710 0x7fcb8c15ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.517+0000 7fcb92800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c16cbd0 con 0x7fcb8c0cf970 2024-04-25T14:04:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.517+0000 7fcb92800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c0ced40 con 0x7fcb8c0ca710 2024-04-25T14:04:11.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.517+0000 7fcb92800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcb8c16cd50 con 0x7fcb8c06e7a0 2024-04-25T14:04:11.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.517+0000 7fcb8b400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb8c06e7a0 0x7fcb8c0cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:11.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.517+0000 7fcb8b400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb8c06e7a0 0x7fcb8c0cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:48174/0 (socket says 172.21.15.86:48174) 2024-04-25T14:04:11.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.517+0000 7fcb82a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8c0cf970 0x7fcb8c15e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:11.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.517+0000 7fcb8b400640 1 -- 172.21.15.86:0/1929339908 learned_addr learned my addr 172.21.15.86:0/1929339908 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:11.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.517+0000 7fcb8be00640 1 --2- 172.21.15.86:0/1929339908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb8c0ca710 0x7fcb8c15ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:11.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.517+0000 7fcb82a00640 1 -- 172.21.15.86:0/1929339908 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb8c06e7a0 msgr2=0x7fcb8c0cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:11.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.517+0000 7fcb82a00640 1 --2- 172.21.15.86:0/1929339908 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb8c06e7a0 0x7fcb8c0cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:11.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.517+0000 7fcb82a00640 1 -- 172.21.15.86:0/1929339908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb8c0ca710 msgr2=0x7fcb8c15ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:11.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.517+0000 7fcb82a00640 1 --2- 172.21.15.86:0/1929339908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb8c0ca710 0x7fcb8c15ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:11.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.517+0000 7fcb82a00640 1 -- 172.21.15.86:0/1929339908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8c159a30 con 0x7fcb8c0cf970 2024-04-25T14:04:11.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.517+0000 7fcb82a00640 1 --2- 172.21.15.86:0/1929339908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8c0cf970 0x7fcb8c15e7d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcb7400e870 tx=0x7fcb7400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:11.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.517+0000 7fcb83e00640 1 -- 172.21.15.86:0/1929339908 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb74017070 con 0x7fcb8c0cf970 2024-04-25T14:04:11.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.517+0000 7fcb83e00640 1 -- 172.21.15.86:0/1929339908 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb7400c940 con 0x7fcb8c0cf970 2024-04-25T14:04:11.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.517+0000 7fcb83e00640 1 -- 172.21.15.86:0/1929339908 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7400ccf0 con 0x7fcb8c0cf970 2024-04-25T14:04:11.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.517+0000 7fcb92800640 1 -- 172.21.15.86:0/1929339908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb8c159d10 con 0x7fcb8c0cf970 2024-04-25T14:04:11.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.517+0000 7fcb92800640 1 -- 172.21.15.86:0/1929339908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb8c0cc990 con 0x7fcb8c0cf970 2024-04-25T14:04:11.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.517+0000 7fcb92800640 1 -- 172.21.15.86:0/1929339908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb8c0720e0 con 0x7fcb8c0cf970 2024-04-25T14:04:11.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.521+0000 7fcb83e00640 1 -- 172.21.15.86:0/1929339908 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcb7401a070 con 0x7fcb8c0cf970 2024-04-25T14:04:11.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.521+0000 7fcb83e00640 1 --2- 172.21.15.86:0/1929339908 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fcb6806f400 0x7fcb68071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:11.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.521+0000 7fcb83e00640 1 -- 172.21.15.86:0/1929339908 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fcb7408f460 con 0x7fcb8c0cf970 2024-04-25T14:04:11.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.521+0000 7fcb8b400640 1 --2- 172.21.15.86:0/1929339908 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fcb6806f400 0x7fcb68071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:11.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.521+0000 7fcb8b400640 1 --2- 172.21.15.86:0/1929339908 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fcb6806f400 0x7fcb68071910 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c001ec0 tx=0x7fcb7c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:11.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.525+0000 7fcb83e00640 1 -- 172.21.15.86:0/1929339908 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcb7405ba50 con 0x7fcb8c0cf970 2024-04-25T14:04:11.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.725+0000 7fcb92800640 1 -- 172.21.15.86:0/1929339908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fcb8c076bf0 con 0x7fcb8c0cf970 2024-04-25T14:04:11.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.729+0000 7fcb83e00640 1 -- 172.21.15.86:0/1929339908 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+179545 (secure 0 0 0) 0x7fcb7405fd50 con 0x7fcb8c0cf970 2024-04-25T14:04:11.731 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:11.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.729+0000 7fcb92800640 1 -- 172.21.15.86:0/1929339908 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fcb6806f400 msgr2=0x7fcb68071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:11.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.729+0000 7fcb92800640 1 --2- 172.21.15.86:0/1929339908 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fcb6806f400 0x7fcb68071910 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c001ec0 tx=0x7fcb7c009040 comp rx=0 tx=0).stop 2024-04-25T14:04:11.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.729+0000 7fcb92800640 1 -- 172.21.15.86:0/1929339908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8c0cf970 msgr2=0x7fcb8c15e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:11.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.729+0000 7fcb92800640 1 --2- 172.21.15.86:0/1929339908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8c0cf970 0x7fcb8c15e7d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcb7400e870 tx=0x7fcb7400ec70 comp rx=0 tx=0).stop 2024-04-25T14:04:11.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.729+0000 7fcb92800640 1 -- 172.21.15.86:0/1929339908 shutdown_connections 2024-04-25T14:04:11.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.729+0000 7fcb92800640 1 --2- 172.21.15.86:0/1929339908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb8c0ca710 0x7fcb8c15ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:11.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.729+0000 7fcb92800640 1 --2- 172.21.15.86:0/1929339908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8c0cf970 0x7fcb8c15e7d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:11.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.729+0000 7fcb92800640 1 --2- 172.21.15.86:0/1929339908 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fcb6806f400 0x7fcb68071910 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:11.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.729+0000 7fcb92800640 1 --2- 172.21.15.86:0/1929339908 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb8c06e7a0 0x7fcb8c0cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:11.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.729+0000 7fcb92800640 1 -- 172.21.15.86:0/1929339908 >> 172.21.15.86:0/1929339908 conn(0x7fcb8c0cffc0 msgr2=0x7fcb8c0b5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:11.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.729+0000 7fcb92800640 1 -- 172.21.15.86:0/1929339908 shutdown_connections 2024-04-25T14:04:11.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.729+0000 7fcb92800640 1 -- 172.21.15.86:0/1929339908 wait complete. 2024-04-25T14:04:11.752 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":5,"flags":0,"active_gid":4103,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2968869755},{"type":"v1","addr":"172.21.15.86:6833","nonce":2968869755}]},"active_addr":"172.21.15.86:6833/2968869755","active_change":"2024-04-25T14:03:30.695012+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4101,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":2602032945}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":1843278167}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":2073988773}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":3558294881}]}]} 2024-04-25T14:04:11.756 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T14:04:11.756 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T14:04:11.756 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:04:11.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.901+0000 7f7f74a00640 1 Processor -- start 2024-04-25T14:04:11.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.901+0000 7f7f74a00640 1 -- start start 2024-04-25T14:04:11.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f74a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7f70072d40 0x7f7f700d1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:11.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f74a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f700b6840 0x7f7f700b6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:11.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f74a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f700d2440 con 0x7f7f7015de40 2024-04-25T14:04:11.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f74a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f7f700d2720 con 0x7f7f70072d40 2024-04-25T14:04:11.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f74a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7f700d2a00 con 0x7f7f700b6840 2024-04-25T14:04:11.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f6de00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7f7015de40 0x7f7f70072630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33358/0 (socket says 172.21.15.86:33358) 2024-04-25T14:04:11.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f6de00640 1 -- 172.21.15.86:0/3077052943 learned_addr learned my addr 172.21.15.86:0/3077052943 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:11.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f6e800640 1 --2- 172.21.15.86:0/3077052943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f700b6840 0x7f7f700b6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:11.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f6d400640 1 -- 172.21.15.86:0/3077052943 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1403872908 0 0) 0x7f7f700d2440 con 0x7f7f7015de40 2024-04-25T14:04:11.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f67e00640 1 --2- 172.21.15.86:0/3077052943 >> v2:172.21.15.155:3300/0 conn(0x7f7f70072d40 0x7f7f700d1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:11.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f6d400640 1 -- 172.21.15.86:0/3077052943 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f54003530 con 0x7f7f7015de40 2024-04-25T14:04:11.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f6d400640 1 -- 172.21.15.86:0/3077052943 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4062925544 0 0) 0x7f7f54003530 con 0x7f7f7015de40 2024-04-25T14:04:11.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f6d400640 1 -- 172.21.15.86:0/3077052943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f700b6840 msgr2=0x7f7f700b6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:11.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f6d400640 1 --2- 172.21.15.86:0/3077052943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f700b6840 0x7f7f700b6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:11.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f6d400640 1 -- 172.21.15.86:0/3077052943 >> v2:172.21.15.155:3300/0 conn(0x7f7f70072d40 msgr2=0x7f7f700d1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:11.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f6d400640 1 --2- 172.21.15.86:0/3077052943 >> v2:172.21.15.155:3300/0 conn(0x7f7f70072d40 0x7f7f700d1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:11.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f6d400640 1 -- 172.21.15.86:0/3077052943 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f700d2ce0 con 0x7f7f7015de40 2024-04-25T14:04:11.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f6d400640 1 -- 172.21.15.86:0/3077052943 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f7f60001680 con 0x7f7f7015de40 2024-04-25T14:04:11.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f6d400640 1 -- 172.21.15.86:0/3077052943 >> v1:172.21.15.86:6789/0 conn(0x7f7f7015de40 legacy=0x7f7f70072630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:11.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f6d400640 1 --2- 172.21.15.86:0/3077052943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f54003a30 0x7f7f54003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:11.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f6d400640 1 --2- 172.21.15.86:0/3077052943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f700b6840 0x7f7f54004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:11.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f6d400640 1 --2- 172.21.15.86:0/3077052943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f54004d50 0x7f7f54005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:11.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f6d400640 1 -- 172.21.15.86:0/3077052943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f700d2720 con 0x7f7f54004d50 2024-04-25T14:04:11.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f6d400640 1 -- 172.21.15.86:0/3077052943 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7f700d2a00 con 0x7f7f54003a30 2024-04-25T14:04:11.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f6d400640 1 -- 172.21.15.86:0/3077052943 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7f5400ba80 con 0x7f7f700b6840 2024-04-25T14:04:11.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f6e800640 1 --2- 172.21.15.86:0/3077052943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f54004d50 0x7f7f54005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:11.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f6de00640 1 --2- 172.21.15.86:0/3077052943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f54003a30 0x7f7f54003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:11.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f67e00640 1 --2- 172.21.15.86:0/3077052943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f700b6840 0x7f7f54004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:11.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f6e800640 1 -- 172.21.15.86:0/3077052943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f700b6840 msgr2=0x7f7f54004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:11.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f6e800640 1 --2- 172.21.15.86:0/3077052943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f700b6840 0x7f7f54004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:11.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f6e800640 1 -- 172.21.15.86:0/3077052943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f54003a30 msgr2=0x7f7f54003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:11.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f6e800640 1 --2- 172.21.15.86:0/3077052943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f54003a30 0x7f7f54003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:11.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.905+0000 7f7f6e800640 1 -- 172.21.15.86:0/3077052943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f54003530 con 0x7f7f54004d50 2024-04-25T14:04:11.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.909+0000 7f7f6e800640 1 --2- 172.21.15.86:0/3077052943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f54004d50 0x7f7f54005160 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7f68009fc0 tx=0x7f7f6801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2eaee55eff737 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:11.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.909+0000 7f7f6d400640 1 -- 172.21.15.86:0/3077052943 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f680b2070 con 0x7f7f54004d50 2024-04-25T14:04:11.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.909+0000 7f7f6d400640 1 -- 172.21.15.86:0/3077052943 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f680a5c70 con 0x7f7f54004d50 2024-04-25T14:04:11.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.909+0000 7f7f6d400640 1 -- 172.21.15.86:0/3077052943 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f68009c10 con 0x7f7f54004d50 2024-04-25T14:04:11.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.909+0000 7f7f74a00640 1 -- 172.21.15.86:0/3077052943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f54004d50 msgr2=0x7f7f54005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:11.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.909+0000 7f7f74a00640 1 --2- 172.21.15.86:0/3077052943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f54004d50 0x7f7f54005160 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7f68009fc0 tx=0x7f7f6801cc60 comp rx=0 tx=0).stop 2024-04-25T14:04:11.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.909+0000 7f7f6e800640 1 -- 172.21.15.86:0/3077052943 reap_dead start 2024-04-25T14:04:11.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.909+0000 7f7f74a00640 1 -- 172.21.15.86:0/3077052943 shutdown_connections 2024-04-25T14:04:11.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.909+0000 7f7f74a00640 1 -- 172.21.15.86:0/3077052943 >> 172.21.15.86:0/3077052943 conn(0x7f7f700d3580 msgr2=0x7f7f700d3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:11.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.909+0000 7f7f74a00640 1 -- 172.21.15.86:0/3077052943 shutdown_connections 2024-04-25T14:04:11.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.909+0000 7f7f74a00640 1 -- 172.21.15.86:0/3077052943 wait complete. 2024-04-25T14:04:11.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.913+0000 7f7f74a00640 1 Processor -- start 2024-04-25T14:04:11.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.913+0000 7f7f74a00640 1 -- start start 2024-04-25T14:04:11.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.913+0000 7f7f74a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f7014d010 0x7f7f701f6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:11.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.913+0000 7f7f74a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f700b3c20 0x7f7f700b4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:11.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.913+0000 7f7f74a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f700cb1e0 0x7f7f700cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:11.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.913+0000 7f7f74a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f700b6dd0 con 0x7f7f700b3c20 2024-04-25T14:04:11.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.913+0000 7f7f74a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7f700b6c50 con 0x7f7f700cb1e0 2024-04-25T14:04:11.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.913+0000 7f7f74a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7f700b6f50 con 0x7f7f7014d010 2024-04-25T14:04:11.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.913+0000 7f7f67e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f700b3c20 0x7f7f700b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:11.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.913+0000 7f7f67e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f700b3c20 0x7f7f700b4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37898/0 (socket says 172.21.15.86:37898) 2024-04-25T14:04:11.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.913+0000 7f7f67e00640 1 -- 172.21.15.86:0/2399711953 learned_addr learned my addr 172.21.15.86:0/2399711953 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:11.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.913+0000 7f7f6de00640 1 --2- 172.21.15.86:0/2399711953 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f7014d010 0x7f7f701f6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:11.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.913+0000 7f7f6e800640 1 --2- 172.21.15.86:0/2399711953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f700cb1e0 0x7f7f700cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:11.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.913+0000 7f7f67e00640 1 -- 172.21.15.86:0/2399711953 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f7014d010 msgr2=0x7f7f701f6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:11.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.913+0000 7f7f67e00640 1 --2- 172.21.15.86:0/2399711953 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f7014d010 0x7f7f701f6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:11.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.913+0000 7f7f67e00640 1 -- 172.21.15.86:0/2399711953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f700cb1e0 msgr2=0x7f7f700cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:11.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.913+0000 7f7f67e00640 1 --2- 172.21.15.86:0/2399711953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f700cb1e0 0x7f7f700cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:11.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.913+0000 7f7f67e00640 1 -- 172.21.15.86:0/2399711953 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f700b2b80 con 0x7f7f700b3c20 2024-04-25T14:04:11.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.913+0000 7f7f67e00640 1 --2- 172.21.15.86:0/2399711953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f700b3c20 0x7f7f700b4030 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7f58011980 tx=0x7f7f58011d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:11.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.913+0000 7f7f66000640 1 -- 172.21.15.86:0/2399711953 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5800fa30 con 0x7f7f700b3c20 2024-04-25T14:04:11.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.913+0000 7f7f66000640 1 -- 172.21.15.86:0/2399711953 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f5800fbd0 con 0x7f7f700b3c20 2024-04-25T14:04:11.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.913+0000 7f7f66000640 1 -- 172.21.15.86:0/2399711953 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f58014440 con 0x7f7f700b3c20 2024-04-25T14:04:11.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.913+0000 7f7f74a00640 1 -- 172.21.15.86:0/2399711953 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f700b2e60 con 0x7f7f700b3c20 2024-04-25T14:04:11.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.913+0000 7f7f74a00640 1 -- 172.21.15.86:0/2399711953 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f700b3420 con 0x7f7f700b3c20 2024-04-25T14:04:11.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.917+0000 7f7f74a00640 1 -- 172.21.15.86:0/2399711953 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f70072170 con 0x7f7f700b3c20 2024-04-25T14:04:11.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.917+0000 7f7f66000640 1 -- 172.21.15.86:0/2399711953 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7f58012040 con 0x7f7f700b3c20 2024-04-25T14:04:11.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.917+0000 7f7f66000640 1 --2- 172.21.15.86:0/2399711953 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7f4c06f400 0x7f7f4c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:11.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.917+0000 7f7f66000640 1 -- 172.21.15.86:0/2399711953 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7f5808fa80 con 0x7f7f700b3c20 2024-04-25T14:04:11.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.917+0000 7f7f6de00640 1 --2- 172.21.15.86:0/2399711953 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7f4c06f400 0x7f7f4c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:11.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.917+0000 7f7f6de00640 1 --2- 172.21.15.86:0/2399711953 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7f4c06f400 0x7f7f4c071910 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7f60000fc0 tx=0x7f7f60007980 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:11.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:11.921+0000 7f7f66000640 1 -- 172.21.15.86:0/2399711953 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7f58019030 con 0x7f7f700b3c20 2024-04-25T14:04:12.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.081+0000 7f7f74a00640 1 -- 172.21.15.86:0/2399711953 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f70076bf0 con 0x7f7f700b3c20 2024-04-25T14:04:12.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.081+0000 7f7f66000640 1 -- 172.21.15.86:0/2399711953 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7f7f5805c750 con 0x7f7f700b3c20 2024-04-25T14:04:12.082 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:12.082 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":24,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:03:55.899092+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:03:49.151473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:03:49.802597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T14:03:53.492008+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:04:12.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.081+0000 7f7f74a00640 1 -- 172.21.15.86:0/2399711953 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7f4c06f400 msgr2=0x7f7f4c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.081+0000 7f7f74a00640 1 --2- 172.21.15.86:0/2399711953 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7f4c06f400 0x7f7f4c071910 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7f60000fc0 tx=0x7f7f60007980 comp rx=0 tx=0).stop 2024-04-25T14:04:12.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.081+0000 7f7f74a00640 1 -- 172.21.15.86:0/2399711953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f700b3c20 msgr2=0x7f7f700b4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.081+0000 7f7f74a00640 1 --2- 172.21.15.86:0/2399711953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f700b3c20 0x7f7f700b4030 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7f58011980 tx=0x7f7f58011d80 comp rx=0 tx=0).stop 2024-04-25T14:04:12.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.081+0000 7f7f74a00640 1 -- 172.21.15.86:0/2399711953 shutdown_connections 2024-04-25T14:04:12.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.081+0000 7f7f74a00640 1 --2- 172.21.15.86:0/2399711953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f700cb1e0 0x7f7f700cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.081+0000 7f7f74a00640 1 --2- 172.21.15.86:0/2399711953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f700b3c20 0x7f7f700b4030 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.081+0000 7f7f74a00640 1 --2- 172.21.15.86:0/2399711953 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7f4c06f400 0x7f7f4c071910 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.081+0000 7f7f74a00640 1 --2- 172.21.15.86:0/2399711953 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f7014d010 0x7f7f701f6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.081+0000 7f7f74a00640 1 -- 172.21.15.86:0/2399711953 >> 172.21.15.86:0/2399711953 conn(0x7f7f700d3580 msgr2=0x7f7f700b7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:12.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.081+0000 7f7f74a00640 1 -- 172.21.15.86:0/2399711953 shutdown_connections 2024-04-25T14:04:12.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.081+0000 7f7f74a00640 1 -- 172.21.15.86:0/2399711953 wait complete. 2024-04-25T14:04:12.104 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T14:04:12.104 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:04:12.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.249+0000 7f8c37e00640 1 Processor -- start 2024-04-25T14:04:12.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.249+0000 7f8c37e00640 1 -- start start 2024-04-25T14:04:12.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.249+0000 7f8c37e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8c30070330 0x7f8c300d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.249+0000 7f8c37e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c300ce790 0x7f8c3006f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.249+0000 7f8c37e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c300d2030 con 0x7f8c300d1c20 2024-04-25T14:04:12.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.249+0000 7f8c37e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f8c3015e5c0 con 0x7f8c30070330 2024-04-25T14:04:12.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.249+0000 7f8c37e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8c3015e8a0 con 0x7f8c300ce790 2024-04-25T14:04:12.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.249+0000 7f8c34a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8c300d1c20 0x7f8c3006ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33374/0 (socket says 172.21.15.86:33374) 2024-04-25T14:04:12.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.249+0000 7f8c34a00640 1 -- 172.21.15.86:0/1052546633 learned_addr learned my addr 172.21.15.86:0/1052546633 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:12.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.249+0000 7f8c35e00640 1 --2- 172.21.15.86:0/1052546633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c300ce790 0x7f8c3006f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.249+0000 7f8c35400640 1 --2- 172.21.15.86:0/1052546633 >> v2:172.21.15.155:3300/0 conn(0x7f8c30070330 0x7f8c300d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.249+0000 7f8c2fe00640 1 -- 172.21.15.86:0/1052546633 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1858771676 0 0) 0x7f8c1c001120 con 0x7f8c300d1c20 2024-04-25T14:04:12.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.249+0000 7f8c2fe00640 1 -- 172.21.15.86:0/1052546633 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c18003530 con 0x7f8c300d1c20 2024-04-25T14:04:12.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.249+0000 7f8c2fe00640 1 -- 172.21.15.86:0/1052546633 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1874118055 0 0) 0x7f8c18003530 con 0x7f8c300d1c20 2024-04-25T14:04:12.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.249+0000 7f8c35e00640 1 -- 172.21.15.86:0/1052546633 >> v2:172.21.15.155:3300/0 conn(0x7f8c30070330 msgr2=0x7f8c300d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:12.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.249+0000 7f8c35e00640 1 --2- 172.21.15.86:0/1052546633 >> v2:172.21.15.155:3300/0 conn(0x7f8c30070330 0x7f8c300d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.249+0000 7f8c35e00640 1 -- 172.21.15.86:0/1052546633 >> v1:172.21.15.86:6789/0 conn(0x7f8c300d1c20 legacy=0x7f8c3006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.249+0000 7f8c35e00640 1 -- 172.21.15.86:0/1052546633 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c3015eb80 con 0x7f8c300ce790 2024-04-25T14:04:12.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.253+0000 7f8c35e00640 1 --2- 172.21.15.86:0/1052546633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c300ce790 0x7f8c3006f440 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8c280097c0 tx=0x7f8c2801c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d1927c808a5729 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.253+0000 7f8c2fe00640 1 -- 172.21.15.86:0/1052546633 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c280b0070 con 0x7f8c300ce790 2024-04-25T14:04:12.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.253+0000 7f8c2fe00640 1 -- 172.21.15.86:0/1052546633 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c280ab400 con 0x7f8c300ce790 2024-04-25T14:04:12.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.253+0000 7f8c2fe00640 1 -- 172.21.15.86:0/1052546633 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c001850 con 0x7f8c300ce790 2024-04-25T14:04:12.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.253+0000 7f8c37e00640 1 -- 172.21.15.86:0/1052546633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c300ce790 msgr2=0x7f8c3006f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.253+0000 7f8c37e00640 1 --2- 172.21.15.86:0/1052546633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c300ce790 0x7f8c3006f440 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8c280097c0 tx=0x7f8c2801c330 comp rx=0 tx=0).stop 2024-04-25T14:04:12.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.253+0000 7f8c37e00640 1 -- 172.21.15.86:0/1052546633 shutdown_connections 2024-04-25T14:04:12.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.253+0000 7f8c37e00640 1 --2- 172.21.15.86:0/1052546633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c300ce790 0x7f8c3006f440 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.253+0000 7f8c37e00640 1 --2- 172.21.15.86:0/1052546633 >> v2:172.21.15.155:3300/0 conn(0x7f8c30070330 0x7f8c300d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.253+0000 7f8c37e00640 1 -- 172.21.15.86:0/1052546633 >> 172.21.15.86:0/1052546633 conn(0x7f8c300b4650 msgr2=0x7f8c300b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:12.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.253+0000 7f8c37e00640 1 -- 172.21.15.86:0/1052546633 shutdown_connections 2024-04-25T14:04:12.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.253+0000 7f8c37e00640 1 -- 172.21.15.86:0/1052546633 wait complete. 2024-04-25T14:04:12.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.257+0000 7f8c37e00640 1 Processor -- start 2024-04-25T14:04:12.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.257+0000 7f8c37e00640 1 -- start start 2024-04-25T14:04:12.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.257+0000 7f8c37e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c30070330 0x7f8c3015b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.257+0000 7f8c37e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c300ce790 0x7f8c300c5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.257+0000 7f8c37e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c300d1c20 0x7f8c300c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.257+0000 7f8c37e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c300708c0 con 0x7f8c300d1c20 2024-04-25T14:04:12.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.257+0000 7f8c37e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8c30070740 con 0x7f8c30070330 2024-04-25T14:04:12.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.257+0000 7f8c37e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8c3016afc0 con 0x7f8c300ce790 2024-04-25T14:04:12.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.257+0000 7f8c35e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c300d1c20 0x7f8c300c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.257+0000 7f8c35e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c300d1c20 0x7f8c300c6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37904/0 (socket says 172.21.15.86:37904) 2024-04-25T14:04:12.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.257+0000 7f8c35e00640 1 -- 172.21.15.86:0/3047671354 learned_addr learned my addr 172.21.15.86:0/3047671354 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:12.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.261+0000 7f8c35400640 1 --2- 172.21.15.86:0/3047671354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c30070330 0x7f8c3015b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.261+0000 7f8c34a00640 1 --2- 172.21.15.86:0/3047671354 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c300ce790 0x7f8c300c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.261+0000 7f8c35e00640 1 -- 172.21.15.86:0/3047671354 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c300ce790 msgr2=0x7f8c300c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.261+0000 7f8c35e00640 1 --2- 172.21.15.86:0/3047671354 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c300ce790 0x7f8c300c5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.261+0000 7f8c35e00640 1 -- 172.21.15.86:0/3047671354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c30070330 msgr2=0x7f8c3015b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.261+0000 7f8c35e00640 1 --2- 172.21.15.86:0/3047671354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c30070330 0x7f8c3015b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.261+0000 7f8c35e00640 1 -- 172.21.15.86:0/3047671354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c300c48a0 con 0x7f8c300d1c20 2024-04-25T14:04:12.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.261+0000 7f8c34a00640 1 --2- 172.21.15.86:0/3047671354 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c300ce790 0x7f8c300c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:12.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.261+0000 7f8c35e00640 1 --2- 172.21.15.86:0/3047671354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c300d1c20 0x7f8c300c6c30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8c280ae040 tx=0x7f8c280190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.261+0000 7f8c2d600640 1 -- 172.21.15.86:0/3047671354 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c280b8020 con 0x7f8c300d1c20 2024-04-25T14:04:12.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.261+0000 7f8c2d600640 1 -- 172.21.15.86:0/3047671354 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c280abe30 con 0x7f8c300d1c20 2024-04-25T14:04:12.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.261+0000 7f8c2d600640 1 -- 172.21.15.86:0/3047671354 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c280b0040 con 0x7f8c300d1c20 2024-04-25T14:04:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.261+0000 7f8c37e00640 1 -- 172.21.15.86:0/3047671354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c301fa6c0 con 0x7f8c300d1c20 2024-04-25T14:04:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.261+0000 7f8c37e00640 1 -- 172.21.15.86:0/3047671354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c300c7eb0 con 0x7f8c300d1c20 2024-04-25T14:04:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.261+0000 7f8c37e00640 1 -- 172.21.15.86:0/3047671354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c301fa890 con 0x7f8c300d1c20 2024-04-25T14:04:12.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.261+0000 7f8c2d600640 1 -- 172.21.15.86:0/3047671354 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8c280bb020 con 0x7f8c300d1c20 2024-04-25T14:04:12.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.265+0000 7f8c2d600640 1 --2- 172.21.15.86:0/3047671354 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8c1006f430 0x7f8c10071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.265+0000 7f8c2d600640 1 -- 172.21.15.86:0/3047671354 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f8c28128a10 con 0x7f8c300d1c20 2024-04-25T14:04:12.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.265+0000 7f8c35400640 1 --2- 172.21.15.86:0/3047671354 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8c1006f430 0x7f8c10071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.265+0000 7f8c35400640 1 --2- 172.21.15.86:0/3047671354 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8c1006f430 0x7f8c10071940 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8c30072200 tx=0x7f8c2400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.269+0000 7f8c2d600640 1 -- 172.21.15.86:0/3047671354 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8c301fa890 con 0x7f8c300d1c20 2024-04-25T14:04:12.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.425+0000 7f8c37e00640 1 -- 172.21.15.86:0/3047671354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8c30069d00 con 0x7f8c300d1c20 2024-04-25T14:04:12.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.429+0000 7f8c2d600640 1 -- 172.21.15.86:0/3047671354 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7f8c280c3020 con 0x7f8c300d1c20 2024-04-25T14:04:12.430 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:12.430 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":24,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:03:55.899092+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:03:49.151473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:03:49.802597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T14:03:53.492008+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:04:12.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.429+0000 7f8c37e00640 1 -- 172.21.15.86:0/3047671354 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8c1006f430 msgr2=0x7f8c10071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.429+0000 7f8c37e00640 1 --2- 172.21.15.86:0/3047671354 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8c1006f430 0x7f8c10071940 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8c30072200 tx=0x7f8c2400b040 comp rx=0 tx=0).stop 2024-04-25T14:04:12.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.429+0000 7f8c37e00640 1 -- 172.21.15.86:0/3047671354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c300d1c20 msgr2=0x7f8c300c6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.429+0000 7f8c37e00640 1 --2- 172.21.15.86:0/3047671354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c300d1c20 0x7f8c300c6c30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8c280ae040 tx=0x7f8c280190d0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.429+0000 7f8c37e00640 1 -- 172.21.15.86:0/3047671354 shutdown_connections 2024-04-25T14:04:12.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.429+0000 7f8c37e00640 1 --2- 172.21.15.86:0/3047671354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c300d1c20 0x7f8c300c6c30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.429+0000 7f8c37e00640 1 --2- 172.21.15.86:0/3047671354 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8c1006f430 0x7f8c10071940 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.429+0000 7f8c37e00640 1 --2- 172.21.15.86:0/3047671354 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c300ce790 0x7f8c300c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.429+0000 7f8c37e00640 1 --2- 172.21.15.86:0/3047671354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c30070330 0x7f8c3015b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.429+0000 7f8c37e00640 1 -- 172.21.15.86:0/3047671354 >> 172.21.15.86:0/3047671354 conn(0x7f8c300b4650 msgr2=0x7f8c3015d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:12.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.429+0000 7f8c37e00640 1 -- 172.21.15.86:0/3047671354 shutdown_connections 2024-04-25T14:04:12.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.429+0000 7f8c37e00640 1 -- 172.21.15.86:0/3047671354 wait complete. 2024-04-25T14:04:12.452 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T14:04:12.452 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T14:04:12.453 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T14:04:12.453 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T14:04:12.453 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T14:04:12.454 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T14:04:12.454 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T14:04:12.454 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T14:04:12.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.713+0000 7faf2ac00640 1 Processor -- start 2024-04-25T14:04:12.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.713+0000 7faf2ac00640 1 -- start start 2024-04-25T14:04:12.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.717+0000 7faf2ac00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7faf24074550 0x7faf24074960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.717+0000 7feafa000640 1 Processor -- start 2024-04-25T14:04:12.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.717+0000 7faf2ac00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf240b27c0 con 0x7faf2406d9d0 2024-04-25T14:04:12.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.717+0000 7faf2ac00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf240b2aa0 con 0x7faf240cdb60 2024-04-25T14:04:12.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.717+0000 7faf2ac00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7faf240740c0 con 0x7faf24074550 2024-04-25T14:04:12.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.717+0000 7feafa000640 1 -- start start 2024-04-25T14:04:12.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.717+0000 7faf23e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7faf2406d9d0 0x7faf2415bcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33376/0 (socket says 172.21.15.86:33376) 2024-04-25T14:04:12.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.717+0000 7faf23e00640 1 -- 172.21.15.86:0/18775397 learned_addr learned my addr 172.21.15.86:0/18775397 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:12.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.717+0000 7faf1b400640 1 --2- 172.21.15.86:0/18775397 >> v2:172.21.15.155:3300/0 conn(0x7faf24074550 0x7faf24074960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.717+0000 7faf1b400640 1 -- 172.21.15.86:0/18775397 >> v1:172.21.15.155:6789/0 conn(0x7faf240cdb60 legacy=0x7faf240d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:12.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.717+0000 7faf1b400640 1 -- 172.21.15.86:0/18775397 >> v1:172.21.15.86:6789/0 conn(0x7faf2406d9d0 legacy=0x7faf2415bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.717+0000 7faf1b400640 1 -- 172.21.15.86:0/18775397 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf2406dde0 con 0x7faf24074550 2024-04-25T14:04:12.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.717+0000 7febb1800640 1 Processor -- start 2024-04-25T14:04:12.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.717+0000 7febb1800640 1 -- start start 2024-04-25T14:04:12.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.721+0000 7febb1800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7febac0d17c0 0x7febac0d2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.721+0000 7febb1800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7febac160690 0x7febac160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.721+0000 7febabe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7febac0d17c0 0x7febac0d2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.721+0000 7febb1800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febac09be50 con 0x7febac15d110 2024-04-25T14:04:12.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.721+0000 7febb1800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7febac0b1c40 con 0x7febac0d17c0 2024-04-25T14:04:12.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.721+0000 7febb1800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7febac160fe0 con 0x7febac160690 2024-04-25T14:04:12.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.721+0000 7febabe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7febac0d17c0 0x7febac0d2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37920/0 (socket says 172.21.15.86:37920) 2024-04-25T14:04:12.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.721+0000 7febabe00640 1 -- 172.21.15.86:0/913853703 learned_addr learned my addr 172.21.15.86:0/913853703 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:12.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.721+0000 7faf1b400640 1 --2- 172.21.15.86:0/18775397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf24074550 0x7faf24074960 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00a8c0 tx=0x7faf0c01d560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61c733ab33071cf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.721+0000 7febab400640 1 -- 172.21.15.86:0/913853703 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1816113263 0 0) 0x7feba4001120 con 0x7febac15d110 2024-04-25T14:04:12.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.721+0000 7febab400640 1 -- 172.21.15.86:0/913853703 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb90003530 con 0x7febac15d110 2024-04-25T14:04:12.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.721+0000 7feba3400640 1 --2- 172.21.15.86:0/913853703 >> v2:172.21.15.155:3300/0 conn(0x7febac160690 0x7febac160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.721+0000 7febabe00640 1 -- 172.21.15.86:0/913853703 >> v2:172.21.15.155:3300/0 conn(0x7febac160690 msgr2=0x7febac160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:12.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.721+0000 7feafa000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feaec055d00 0x7feaec056110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7febabe00640 1 --2- 172.21.15.86:0/913853703 >> v2:172.21.15.155:3300/0 conn(0x7febac160690 0x7febac160aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7febabe00640 1 -- 172.21.15.86:0/913853703 >> v1:172.21.15.155:6789/0 conn(0x7febac15d110 legacy=0x7febac15d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7febabe00640 1 -- 172.21.15.86:0/913853703 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febac0b1680 con 0x7febac0d17c0 2024-04-25T14:04:12.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7faf23400640 1 -- 172.21.15.86:0/18775397 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c0b1070 con 0x7faf24074550 2024-04-25T14:04:12.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7feafa000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7feaec0092d0 0x7feaec0096e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7feafa000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaec009c20 con 0x7feaec0e75b0 2024-04-25T14:04:12.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7feafa000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7feaec0e4f40 con 0x7feaec0092d0 2024-04-25T14:04:12.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7feafa000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7feaec055ad0 con 0x7feaec055d00 2024-04-25T14:04:12.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7feaf3400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feaec055d00 0x7feaec056110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7feaf3400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feaec055d00 0x7feaec056110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:48224/0 (socket says 172.21.15.86:48224) 2024-04-25T14:04:12.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7feaf3400640 1 -- 172.21.15.86:0/858174113 learned_addr learned my addr 172.21.15.86:0/858174113 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:12.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7faf23400640 1 -- 172.21.15.86:0/18775397 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf0c0ac600 con 0x7faf24074550 2024-04-25T14:04:12.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7faf23400640 1 -- 172.21.15.86:0/18775397 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c0ac9d0 con 0x7faf24074550 2024-04-25T14:04:12.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7f0347e00640 1 Processor -- start 2024-04-25T14:04:12.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7feaf2a00640 1 --2- 172.21.15.86:0/858174113 >> v2:172.21.15.155:3300/0 conn(0x7feaec0092d0 0x7feaec0096e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7feaf3400640 1 -- 172.21.15.86:0/858174113 >> v2:172.21.15.155:3300/0 conn(0x7feaec0092d0 msgr2=0x7feaec0096e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:12.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7feaf3400640 1 --2- 172.21.15.86:0/858174113 >> v2:172.21.15.155:3300/0 conn(0x7feaec0092d0 0x7feaec0096e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7feaf3400640 1 -- 172.21.15.86:0/858174113 >> v1:172.21.15.155:6789/0 conn(0x7feaec0e75b0 legacy=0x7feaec0e79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7faf2ac00640 1 -- 172.21.15.86:0/18775397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf24074550 msgr2=0x7faf24074960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7faf2ac00640 1 --2- 172.21.15.86:0/18775397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf24074550 0x7faf24074960 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00a8c0 tx=0x7faf0c01d560 comp rx=0 tx=0).stop 2024-04-25T14:04:12.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7feaf2000640 1 -- 172.21.15.86:0/858174113 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3792158701 0 0) 0x7feae0001120 con 0x7feaec0e75b0 2024-04-25T14:04:12.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7f0347e00640 1 -- start start 2024-04-25T14:04:12.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7faf2ac00640 1 -- 172.21.15.86:0/18775397 shutdown_connections 2024-04-25T14:04:12.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7faf2ac00640 1 --2- 172.21.15.86:0/18775397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf24074550 0x7faf24074960 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7faf2ac00640 1 -- 172.21.15.86:0/18775397 >> 172.21.15.86:0/18775397 conn(0x7faf240b6130 msgr2=0x7faf240b6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:12.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7feaf3400640 1 -- 172.21.15.86:0/858174113 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaec038b00 con 0x7feaec055d00 2024-04-25T14:04:12.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7feaf2a00640 1 --2- 172.21.15.86:0/858174113 >> v2:172.21.15.155:3300/0 conn(0x7feaec0092d0 0x7feaec0096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:12.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7faf2ac00640 1 -- 172.21.15.86:0/18775397 shutdown_connections 2024-04-25T14:04:12.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7faf2ac00640 1 -- 172.21.15.86:0/18775397 wait complete. 2024-04-25T14:04:12.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7fe7efc00640 1 Processor -- start 2024-04-25T14:04:12.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7f6c13000640 1 Processor -- start 2024-04-25T14:04:12.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7f6c13000640 1 -- start start 2024-04-25T14:04:12.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7fe7efc00640 1 -- start start 2024-04-25T14:04:12.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7f0347e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f034015dc90 0x7f0340161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7f0347e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03400cd8d0 con 0x7f03400d2c80 2024-04-25T14:04:12.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7f0347e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03400ce970 con 0x7f0340071530 2024-04-25T14:04:12.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.725+0000 7f0347e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f03400ce720 con 0x7f034015dc90 2024-04-25T14:04:12.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7febabe00640 1 --2- 172.21.15.86:0/913853703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febac0d17c0 0x7febac0d2b10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7febac0722d0 tx=0x7feb9401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6390df9fea50f0fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7f0345400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f03400d2c80 0x7f0340070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33390/0 (socket says 172.21.15.86:33390) 2024-04-25T14:04:12.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7f0345400640 1 -- 172.21.15.86:0/3703140515 learned_addr learned my addr 172.21.15.86:0/3703140515 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:12.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7f033fe00640 1 -- 172.21.15.86:0/3703140515 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4188246979 0 0) 0x7f03400ce970 con 0x7f0340071530 2024-04-25T14:04:12.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7f033fe00640 1 -- 172.21.15.86:0/3703140515 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0328003530 con 0x7f0340071530 2024-04-25T14:04:12.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7febab400640 1 -- 172.21.15.86:0/913853703 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb940b0070 con 0x7febac0d17c0 2024-04-25T14:04:12.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7febab400640 1 -- 172.21.15.86:0/913853703 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb940ab600 con 0x7febac0d17c0 2024-04-25T14:04:12.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7febab400640 1 -- 172.21.15.86:0/913853703 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb940b0070 con 0x7febac0d17c0 2024-04-25T14:04:12.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7f0345e00640 1 --2- 172.21.15.86:0/3703140515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f034015dc90 0x7f0340161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7f033fe00640 1 -- 172.21.15.86:0/3703140515 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1059295452 0 0) 0x7f03400cd8d0 con 0x7f03400d2c80 2024-04-25T14:04:12.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7f033fe00640 1 -- 172.21.15.86:0/3703140515 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03400ce970 con 0x7f03400d2c80 2024-04-25T14:04:12.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7f033fe00640 1 -- 172.21.15.86:0/3703140515 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 462893317 0 0) 0x7f0328003530 con 0x7f0340071530 2024-04-25T14:04:12.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7f033fe00640 1 -- 172.21.15.86:0/3703140515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f034015dc90 msgr2=0x7f0340161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:12.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7f033fe00640 1 --2- 172.21.15.86:0/3703140515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f034015dc90 0x7f0340161df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7f033fe00640 1 -- 172.21.15.86:0/3703140515 >> v1:172.21.15.86:6789/0 conn(0x7f03400d2c80 legacy=0x7f0340070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7f033fe00640 1 -- 172.21.15.86:0/3703140515 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03400d1be0 con 0x7f0340071530 2024-04-25T14:04:12.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7f033fe00640 1 -- 172.21.15.86:0/3703140515 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f03300016d0 con 0x7f0340071530 2024-04-25T14:04:12.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7f033fe00640 1 -- 172.21.15.86:0/3703140515 >> v1:172.21.15.155:6789/0 conn(0x7f0340071530 legacy=0x7f03401616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7feaf3400640 1 --2- 172.21.15.86:0/858174113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feaec055d00 0x7feaec056110 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feaec0e50c0 tx=0x7feae801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf35120c8d970225 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7f033fe00640 1 --2- 172.21.15.86:0/3703140515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0328003bc0 0x7f0328004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7f6c13000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6c0c1614e0 0x7f6c0c06eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7feaf2000640 1 -- 172.21.15.86:0/858174113 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae80b2070 con 0x7feaec055d00 2024-04-25T14:04:12.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7feaf2000640 1 -- 172.21.15.86:0/858174113 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feae80ad640 con 0x7feaec055d00 2024-04-25T14:04:12.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7feaf2000640 1 -- 172.21.15.86:0/858174113 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae80ada10 con 0x7feaec055d00 2024-04-25T14:04:12.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7faf2ac00640 1 Processor -- start 2024-04-25T14:04:12.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7faf2ac00640 1 -- start start 2024-04-25T14:04:12.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7fe7efc00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe7e80ce570 0x7fe7e80c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7fe7efc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe7e806f9f0 0x7fe7e8168530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7fe7efc00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7e80c4c00 con 0x7fe7e80c47f0 2024-04-25T14:04:12.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7fe7efc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe7e806dcb0 con 0x7fe7e806f9f0 2024-04-25T14:04:12.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7fe7efc00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe7e806d0a0 con 0x7fe7e80ce570 2024-04-25T14:04:12.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7fe7edc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe7e806f9f0 0x7fe7e8168530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7fe7edc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe7e806f9f0 0x7fe7e8168530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37928/0 (socket says 172.21.15.86:37928) 2024-04-25T14:04:12.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7fe7edc00640 1 -- 172.21.15.86:0/641274332 learned_addr learned my addr 172.21.15.86:0/641274332 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:12.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7fe7e7e00640 1 --1- 172.21.15.86:0/641274332 >> v1:172.21.15.155:6789/0 conn(0x7fe7e80c47f0 0x7fe7e8167e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:50788/0 (socket says 172.21.15.86:50788) 2024-04-25T14:04:12.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7fe7ed200640 1 --2- 172.21.15.86:0/641274332 >> v2:172.21.15.155:3300/0 conn(0x7fe7e80ce570 0x7fe7e80c42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7fe7e7400640 1 -- 172.21.15.86:0/641274332 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 480171568 0 0) 0x7fe7d8001120 con 0x7fe7e80c47f0 2024-04-25T14:04:12.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7fe7e7400640 1 -- 172.21.15.86:0/641274332 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7cc003530 con 0x7fe7e80c47f0 2024-04-25T14:04:12.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7fe7edc00640 1 -- 172.21.15.86:0/641274332 >> v2:172.21.15.155:3300/0 conn(0x7fe7e80ce570 msgr2=0x7fe7e80c42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:12.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7fe7edc00640 1 --2- 172.21.15.86:0/641274332 >> v2:172.21.15.155:3300/0 conn(0x7fe7e80ce570 0x7fe7e80c42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7fe7edc00640 1 -- 172.21.15.86:0/641274332 >> v1:172.21.15.155:6789/0 conn(0x7fe7e80c47f0 legacy=0x7fe7e8167e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7fe7e7400640 1 -- 172.21.15.86:0/641274332 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 402236228 0 0) 0x7fe7cc003530 con 0x7fe7e80c47f0 2024-04-25T14:04:12.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7fe7edc00640 1 -- 172.21.15.86:0/641274332 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e806d380 con 0x7fe7e806f9f0 2024-04-25T14:04:12.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7f033fe00640 1 --2- 172.21.15.86:0/3703140515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f034015dc90 0x7f0328004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7f033fe00640 1 --2- 172.21.15.86:0/3703140515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0328004dc0 0x7f0328005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7f033fe00640 1 -- 172.21.15.86:0/3703140515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03400ce720 con 0x7f0328003bc0 2024-04-25T14:04:12.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7f033fe00640 1 -- 172.21.15.86:0/3703140515 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f032800bc30 con 0x7f0328004dc0 2024-04-25T14:04:12.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7f033fe00640 1 -- 172.21.15.86:0/3703140515 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f032800be30 con 0x7f034015dc90 2024-04-25T14:04:12.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7f0344a00640 1 --2- 172.21.15.86:0/3703140515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f034015dc90 0x7f0328004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7f0345400640 1 --2- 172.21.15.86:0/3703140515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0328003bc0 0x7f0328004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7f0345e00640 1 --2- 172.21.15.86:0/3703140515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0328004dc0 0x7f0328005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7f0344a00640 1 -- 172.21.15.86:0/3703140515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0328004dc0 msgr2=0x7f0328005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:12.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7f6c13000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c0c163770 con 0x7f6c0c06d740 2024-04-25T14:04:12.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7f6c13000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c0c1618f0 con 0x7f6c0c06e260 2024-04-25T14:04:12.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7f6c0be00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6c0c1614e0 0x7f6c0c06eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7f6c13000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f6c0c09bd80 con 0x7f6c0c1614e0 2024-04-25T14:04:12.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.729+0000 7f6c0be00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6c0c1614e0 0x7f6c0c06eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:54266/0 (socket says 172.21.15.86:54266) 2024-04-25T14:04:12.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7f6c0be00640 1 -- 172.21.15.86:0/854019594 learned_addr learned my addr 172.21.15.86:0/854019594 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:12.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7f6c0be00640 1 -- 172.21.15.86:0/854019594 >> v1:172.21.15.155:6789/0 conn(0x7f6c0c06e260 legacy=0x7f6c0c1603d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:12.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7f6c0be00640 1 -- 172.21.15.86:0/854019594 >> v1:172.21.15.86:6789/0 conn(0x7f6c0c06d740 legacy=0x7f6c0c06db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:12.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7f6c0be00640 1 -- 172.21.15.86:0/854019594 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c0c0b0020 con 0x7f6c0c1614e0 2024-04-25T14:04:12.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7febb1800640 1 -- 172.21.15.86:0/913853703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febac0d17c0 msgr2=0x7febac0d2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7febb1800640 1 --2- 172.21.15.86:0/913853703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febac0d17c0 0x7febac0d2b10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7febac0722d0 tx=0x7feb9401c490 comp rx=0 tx=0).stop 2024-04-25T14:04:12.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7feafa000640 1 -- 172.21.15.86:0/858174113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feaec055d00 msgr2=0x7feaec056110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7feafa000640 1 --2- 172.21.15.86:0/858174113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feaec055d00 0x7feaec056110 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feaec0e50c0 tx=0x7feae801c480 comp rx=0 tx=0).stop 2024-04-25T14:04:12.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7faf2ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf2406d9d0 0x7faf24157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7faf2ac00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf24074550 0x7faf2415ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7faf2ac00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf240cdb60 0x7faf2415b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7faf2ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faf240d1fc0 con 0x7faf2406d9d0 2024-04-25T14:04:12.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7faf2ac00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faf240d1e40 con 0x7faf240cdb60 2024-04-25T14:04:12.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7faf2ac00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7faf240d12e0 con 0x7faf24074550 2024-04-25T14:04:12.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7febb1800640 1 -- 172.21.15.86:0/913853703 shutdown_connections 2024-04-25T14:04:12.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7febb1800640 1 --2- 172.21.15.86:0/913853703 >> v2:172.21.15.155:3300/0 conn(0x7febac160690 0x7febac160aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7febb1800640 1 --2- 172.21.15.86:0/913853703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febac0d17c0 0x7febac0d2b10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7febb1800640 1 -- 172.21.15.86:0/913853703 >> 172.21.15.86:0/913853703 conn(0x7febac073c00 msgr2=0x7febac0b77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:12.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7faf28c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf240cdb60 0x7faf2415b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7faf28c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf240cdb60 0x7faf2415b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:54288/0 (socket says 172.21.15.86:54288) 2024-04-25T14:04:12.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7faf28c00640 1 -- 172.21.15.86:0/3990538937 learned_addr learned my addr 172.21.15.86:0/3990538937 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:12.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7faf1b400640 1 --2- 172.21.15.86:0/3990538937 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf24074550 0x7faf2415ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7febb1800640 1 -- 172.21.15.86:0/913853703 shutdown_connections 2024-04-25T14:04:12.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7febb1800640 1 -- 172.21.15.86:0/913853703 wait complete. 2024-04-25T14:04:12.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7f0344a00640 1 --2- 172.21.15.86:0/3703140515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0328004dc0 0x7f0328005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7f0344a00640 1 -- 172.21.15.86:0/3703140515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0328003bc0 msgr2=0x7f0328004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:12.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7f0344a00640 1 --2- 172.21.15.86:0/3703140515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0328003bc0 0x7f0328004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7f0344a00640 1 -- 172.21.15.86:0/3703140515 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0328003530 con 0x7f034015dc90 2024-04-25T14:04:12.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.733+0000 7f7abb200640 1 Processor -- start 2024-04-25T14:04:12.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7faf28c00640 1 -- 172.21.15.86:0/3990538937 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf24074550 msgr2=0x7faf2415ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7faf28c00640 1 --2- 172.21.15.86:0/3990538937 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf24074550 0x7faf2415ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7faf28c00640 1 -- 172.21.15.86:0/3990538937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf2406d9d0 msgr2=0x7faf24157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7faf28c00640 1 --2- 172.21.15.86:0/3990538937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf2406d9d0 0x7faf24157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7feafa000640 1 -- 172.21.15.86:0/858174113 shutdown_connections 2024-04-25T14:04:12.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7feafa000640 1 --2- 172.21.15.86:0/858174113 >> v2:172.21.15.155:3300/0 conn(0x7feaec0092d0 0x7feaec0096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7feafa000640 1 --2- 172.21.15.86:0/858174113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feaec055d00 0x7feaec056110 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7faf28c00640 1 -- 172.21.15.86:0/3990538937 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf24068c00 con 0x7faf240cdb60 2024-04-25T14:04:12.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7feafa000640 1 -- 172.21.15.86:0/858174113 >> 172.21.15.86:0/858174113 conn(0x7feaec053990 msgr2=0x7feaec053da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:12.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7f7abb200640 1 -- start start 2024-04-25T14:04:12.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7feafa000640 1 -- 172.21.15.86:0/858174113 shutdown_connections 2024-04-25T14:04:12.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7feafa000640 1 -- 172.21.15.86:0/858174113 wait complete. 2024-04-25T14:04:12.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7fde80a00640 1 Processor -- start 2024-04-25T14:04:12.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7fde80a00640 1 -- start start 2024-04-25T14:04:12.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7f6c0be00640 1 --2- 172.21.15.86:0/854019594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c0c1614e0 0x7f6c0c06eb00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc0098a0 tx=0x7f6bfc01c1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92dcef6882620fe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7faf28c00640 1 --2- 172.21.15.86:0/3990538937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf240cdb60 0x7faf2415b2c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00b440 tx=0x7faf1c00b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7f0344a00640 1 --2- 172.21.15.86:0/3703140515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f034015dc90 0x7f0328004810 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0330001ec0 tx=0x7f033001ca40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d793946ed5b3997 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7faf20c00640 1 -- 172.21.15.86:0/3990538937 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1c015020 con 0x7faf240cdb60 2024-04-25T14:04:12.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7faf20c00640 1 -- 172.21.15.86:0/3990538937 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf1c011850 con 0x7faf240cdb60 2024-04-25T14:04:12.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7faf20c00640 1 -- 172.21.15.86:0/3990538937 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1c011c40 con 0x7faf240cdb60 2024-04-25T14:04:12.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7f6c0aa00640 1 -- 172.21.15.86:0/854019594 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc0b0070 con 0x7f6c0c1614e0 2024-04-25T14:04:12.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7f6c0aa00640 1 -- 172.21.15.86:0/854019594 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bfc0ab600 con 0x7f6c0c1614e0 2024-04-25T14:04:12.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7f6c0aa00640 1 -- 172.21.15.86:0/854019594 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc0ab9d0 con 0x7f6c0c1614e0 2024-04-25T14:04:12.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7f7abb200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7ab40e79f0 0x7f7ab4006930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7f7abb200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7ab40e7e00 0x7f7ab4006e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7f7abb200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ab4050970 con 0x7f7ab40e3760 2024-04-25T14:04:12.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7f7abb200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7ab40064a0 con 0x7f7ab40e7e00 2024-04-25T14:04:12.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7f7abb200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f7ab40345b0 con 0x7f7ab40e79f0 2024-04-25T14:04:12.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7fe7edc00640 1 --2- 172.21.15.86:0/641274332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7e806f9f0 0x7fe7e8168530 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe7e80d1b70 tx=0x7fe7e001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f9302205a97ae1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7f033fe00640 1 -- 172.21.15.86:0/3703140515 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03300b2070 con 0x7f034015dc90 2024-04-25T14:04:12.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7f033fe00640 1 -- 172.21.15.86:0/3703140515 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03300a5930 con 0x7f034015dc90 2024-04-25T14:04:12.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7f033fe00640 1 -- 172.21.15.86:0/3703140515 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330007b20 con 0x7f034015dc90 2024-04-25T14:04:12.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7faf2ac00640 1 -- 172.21.15.86:0/3990538937 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf24158250 con 0x7faf240cdb60 2024-04-25T14:04:12.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.737+0000 7faf2ac00640 1 -- 172.21.15.86:0/3990538937 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf2415f3e0 con 0x7faf240cdb60 2024-04-25T14:04:12.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7fde80a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fde7c0cfca0 0x7fde7c06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7fde80a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde7c0730b0 0x7fde7c0734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7fde80a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde7c072bd0 con 0x7fde7c0d16e0 2024-04-25T14:04:12.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7fde80a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fde7c070470 con 0x7fde7c0cfca0 2024-04-25T14:04:12.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7fde80a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fde7c06f930 con 0x7fde7c0730b0 2024-04-25T14:04:12.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7f0347e00640 1 -- 172.21.15.86:0/3703140515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f034015dc90 msgr2=0x7f0328004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7f0347e00640 1 --2- 172.21.15.86:0/3703140515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f034015dc90 0x7f0328004810 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0330001ec0 tx=0x7f033001ca40 comp rx=0 tx=0).stop 2024-04-25T14:04:12.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7f0345e00640 1 -- 172.21.15.86:0/3703140515 reap_dead start 2024-04-25T14:04:12.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7f6c13000640 1 -- 172.21.15.86:0/854019594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c0c1614e0 msgr2=0x7f6c0c06eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7f6c13000640 1 --2- 172.21.15.86:0/854019594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c0c1614e0 0x7f6c0c06eb00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc0098a0 tx=0x7f6bfc01c1c0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7f0347e00640 1 -- 172.21.15.86:0/3703140515 shutdown_connections 2024-04-25T14:04:12.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7f0347e00640 1 -- 172.21.15.86:0/3703140515 >> 172.21.15.86:0/3703140515 conn(0x7f03400cffc0 msgr2=0x7f03400d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:12.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7fe7e7400640 1 -- 172.21.15.86:0/641274332 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8001850 con 0x7fe7e806f9f0 2024-04-25T14:04:12.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7f0347e00640 1 -- 172.21.15.86:0/3703140515 shutdown_connections 2024-04-25T14:04:12.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7febb1800640 1 Processor -- start 2024-04-25T14:04:12.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7febb1800640 1 -- start start 2024-04-25T14:04:12.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7f7ab9400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7ab40e79f0 0x7f7ab4006930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7f0347e00640 1 -- 172.21.15.86:0/3703140515 wait complete. 2024-04-25T14:04:12.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7fde79400640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fde7c0d16e0 0x7fde7c0d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:50808/0 (socket says 172.21.15.86:50808) 2024-04-25T14:04:12.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7fde79400640 1 -- 172.21.15.86:0/1146252500 learned_addr learned my addr 172.21.15.86:0/1146252500 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:12.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7f6c13000640 1 -- 172.21.15.86:0/854019594 shutdown_connections 2024-04-25T14:04:12.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7f6c13000640 1 --2- 172.21.15.86:0/854019594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c0c1614e0 0x7f6c0c06eb00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7f6c13000640 1 -- 172.21.15.86:0/854019594 >> 172.21.15.86:0/854019594 conn(0x7f6c0c073da0 msgr2=0x7f6c0c0b7420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:12.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7f7ab9400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7ab40e79f0 0x7f7ab4006930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:54296/0 (socket says 172.21.15.86:54296) 2024-04-25T14:04:12.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7f7ab9400640 1 -- 172.21.15.86:0/3192647560 learned_addr learned my addr 172.21.15.86:0/3192647560 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:12.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7feafa000640 1 Processor -- start 2024-04-25T14:04:12.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7fde79e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fde7c0cfca0 0x7fde7c06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7fde7a800640 1 --2- 172.21.15.86:0/1146252500 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde7c0730b0 0x7fde7c0734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7faf2ac00640 1 -- 172.21.15.86:0/3990538937 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faf2409f4b0 con 0x7faf240cdb60 2024-04-25T14:04:12.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7feafa000640 1 -- start start 2024-04-25T14:04:12.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7f6c13000640 1 -- 172.21.15.86:0/854019594 shutdown_connections 2024-04-25T14:04:12.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7fe7e7400640 1 -- 172.21.15.86:0/641274332 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e00090d0 con 0x7fe7e806f9f0 2024-04-25T14:04:12.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7fe7e7400640 1 -- 172.21.15.86:0/641274332 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8001850 con 0x7fe7e806f9f0 2024-04-25T14:04:12.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7f7ab9400640 1 -- 172.21.15.86:0/3192647560 >> v2:172.21.15.86:3300/0 conn(0x7f7ab40e7e00 msgr2=0x7f7ab4006e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:12.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7f7ab9400640 1 --2- 172.21.15.86:0/3192647560 >> v2:172.21.15.86:3300/0 conn(0x7f7ab40e7e00 0x7f7ab4006e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7f6c13000640 1 -- 172.21.15.86:0/854019594 wait complete. 2024-04-25T14:04:12.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7f7ab9400640 1 -- 172.21.15.86:0/3192647560 >> v1:172.21.15.86:6789/0 conn(0x7f7ab40e3760 legacy=0x7f7ab40e72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:12.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7f7ab9400640 1 -- 172.21.15.86:0/3192647560 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab40500d0 con 0x7f7ab40e79f0 2024-04-25T14:04:12.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7fe7efc00640 1 -- 172.21.15.86:0/641274332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7e806f9f0 msgr2=0x7fe7e8168530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7fe7efc00640 1 --2- 172.21.15.86:0/641274332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7e806f9f0 0x7fe7e8168530 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe7e80d1b70 tx=0x7fe7e001c8a0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7fde79e00640 1 -- 172.21.15.86:0/1146252500 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde7c0730b0 msgr2=0x7fde7c0734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:12.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7febb1800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febac0d17c0 0x7febac0cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7febb1800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febac15d110 0x7febac1f8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7febb1800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febac160690 0x7febac15ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7febb1800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7febac0d1d50 con 0x7febac15d110 2024-04-25T14:04:12.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7febb1800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7febac0d1bd0 con 0x7febac0d17c0 2024-04-25T14:04:12.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7febb1800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7febac06ebf0 con 0x7febac160690 2024-04-25T14:04:12.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7fde79e00640 1 --2- 172.21.15.86:0/1146252500 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde7c0730b0 0x7fde7c0734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7fe7efc00640 1 -- 172.21.15.86:0/641274332 shutdown_connections 2024-04-25T14:04:12.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7fe7efc00640 1 --2- 172.21.15.86:0/641274332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7e806f9f0 0x7fe7e8168530 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7fe7efc00640 1 --2- 172.21.15.86:0/641274332 >> v2:172.21.15.155:3300/0 conn(0x7fe7e80ce570 0x7fe7e80c42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7febb0e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febac160690 0x7febac15ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7febabe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febac0d17c0 0x7febac0cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7febabe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febac0d17c0 0x7febac0cc030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:54312/0 (socket says 172.21.15.86:54312) 2024-04-25T14:04:12.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7febabe00640 1 -- 172.21.15.86:0/3065797565 learned_addr learned my addr 172.21.15.86:0/3065797565 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:12.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7fe7efc00640 1 -- 172.21.15.86:0/641274332 >> 172.21.15.86:0/641274332 conn(0x7fe7e80b4650 msgr2=0x7fe7e80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:12.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7fde79e00640 1 -- 172.21.15.86:0/1146252500 >> v1:172.21.15.155:6789/0 conn(0x7fde7c0d16e0 legacy=0x7fde7c0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.741+0000 7fde79e00640 1 -- 172.21.15.86:0/1146252500 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7c06fbd0 con 0x7fde7c0cfca0 2024-04-25T14:04:12.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7febabe00640 1 -- 172.21.15.86:0/3065797565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febac160690 msgr2=0x7febac15ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7febabe00640 1 --2- 172.21.15.86:0/3065797565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febac160690 0x7febac15ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7febabe00640 1 -- 172.21.15.86:0/3065797565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febac15d110 msgr2=0x7febac1f8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7febabe00640 1 --2- 172.21.15.86:0/3065797565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febac15d110 0x7febac1f8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7febabe00640 1 -- 172.21.15.86:0/3065797565 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febac0cd480 con 0x7febac0d17c0 2024-04-25T14:04:12.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7febb0e00640 1 --2- 172.21.15.86:0/3065797565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febac160690 0x7febac15ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:12.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7f0347e00640 1 Processor -- start 2024-04-25T14:04:12.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7faf20c00640 1 -- 172.21.15.86:0/3990538937 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7faf1c018020 con 0x7faf240cdb60 2024-04-25T14:04:12.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7f0347e00640 1 -- start start 2024-04-25T14:04:12.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7fe7efc00640 1 -- 172.21.15.86:0/641274332 shutdown_connections 2024-04-25T14:04:12.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7fe7efc00640 1 -- 172.21.15.86:0/641274332 wait complete. 2024-04-25T14:04:12.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7febabe00640 1 --2- 172.21.15.86:0/3065797565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febac0d17c0 0x7febac0cc030 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7feb94000c10 tx=0x7feb94019230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7f6c13000640 1 Processor -- start 2024-04-25T14:04:12.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7f6c13000640 1 -- start start 2024-04-25T14:04:12.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7faf20c00640 1 --2- 172.21.15.86:0/3990538937 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7faf0006f450 0x7faf00071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7feba8c00640 1 -- 172.21.15.86:0/3065797565 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb940b0040 con 0x7febac0d17c0 2024-04-25T14:04:12.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7feba8c00640 1 -- 172.21.15.86:0/3065797565 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb940bb070 con 0x7febac0d17c0 2024-04-25T14:04:12.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7feba8c00640 1 -- 172.21.15.86:0/3065797565 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb940abbb0 con 0x7febac0d17c0 2024-04-25T14:04:12.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7febb1800640 1 -- 172.21.15.86:0/3065797565 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febac0cc870 con 0x7febac0d17c0 2024-04-25T14:04:12.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7faf20c00640 1 -- 172.21.15.86:0/3990538937 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7faf1c08f390 con 0x7faf240cdb60 2024-04-25T14:04:12.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7faf20c00640 1 --2- 172.21.15.86:0/3990538937 >> [v2:172.21.15.86:6816/2851379769,v1:172.21.15.86:6817/2851379769] conn(0x7faf00078c10 0x7faf0007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7faf20c00640 1 -- 172.21.15.86:0/3990538937 --> [v2:172.21.15.86:6816/2851379769,v1:172.21.15.86:6817/2851379769] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faf1c01ee70 con 0x7faf00078c10 2024-04-25T14:04:12.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7faf20c00640 1 -- 172.21.15.86:0/3990538937 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7faf1c096050 con 0x7faf240cdb60 2024-04-25T14:04:12.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7feafa000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feaec0092d0 0x7feaec0d3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7feafa000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feaec055d00 0x7feaec04a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7feafa000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaec0e75b0 0x7feaec04acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7feafa000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feaec007df0 con 0x7feaec0e75b0 2024-04-25T14:04:12.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7feafa000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7feaec039680 con 0x7feaec0092d0 2024-04-25T14:04:12.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7feafa000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7feaec04bad0 con 0x7feaec055d00 2024-04-25T14:04:12.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7faf1b400640 1 --2- 172.21.15.86:0/3990538937 >> [v2:172.21.15.86:6816/2851379769,v1:172.21.15.86:6817/2851379769] conn(0x7faf00078c10 0x7faf0007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7f0347e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f034015dc90 0x7f034014ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7f0347e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0340150940 0x7f0340150d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7f0347e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03401f55e0 0x7f03401f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7f0347e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03400d2480 con 0x7f0340150940 2024-04-25T14:04:12.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7f0347e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f03400d21a0 con 0x7f03401f55e0 2024-04-25T14:04:12.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7f0347e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f03400d29a0 con 0x7f034015dc90 2024-04-25T14:04:12.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7feaf3400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feaec0092d0 0x7feaec0d3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7febb1800640 1 -- 172.21.15.86:0/3065797565 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febac0cce30 con 0x7febac0d17c0 2024-04-25T14:04:12.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7feaf2a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feaec055d00 0x7feaec04a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.745+0000 7f0345400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f034015dc90 0x7f034014ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f0345400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f034015dc90 0x7f034014ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:48274/0 (socket says 172.21.15.86:48274) 2024-04-25T14:04:12.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f0345400640 1 -- 172.21.15.86:0/4134364944 learned_addr learned my addr 172.21.15.86:0/4134364944 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:12.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f0345e00640 1 --2- 172.21.15.86:0/4134364944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03401f55e0 0x7f03401f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f0344a00640 1 --2- 172.21.15.86:0/4134364944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0340150940 0x7f0340150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7feaf2a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feaec055d00 0x7feaec04a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:48266/0 (socket says 172.21.15.86:48266) 2024-04-25T14:04:12.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7feaf2a00640 1 -- 172.21.15.86:0/2735467408 learned_addr learned my addr 172.21.15.86:0/2735467408 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:12.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7feaf3e00640 1 --2- 172.21.15.86:0/2735467408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaec0e75b0 0x7feaec04acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f0345400640 1 -- 172.21.15.86:0/4134364944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03401f55e0 msgr2=0x7f03401f59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f0345400640 1 --2- 172.21.15.86:0/4134364944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03401f55e0 0x7f03401f59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f0345400640 1 -- 172.21.15.86:0/4134364944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0340150940 msgr2=0x7f0340150d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f0345400640 1 --2- 172.21.15.86:0/4134364944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0340150940 0x7f0340150d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f0345400640 1 -- 172.21.15.86:0/4134364944 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03401f5f30 con 0x7f034015dc90 2024-04-25T14:04:12.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f6c13000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c0c06d740 0x7f6c0c1560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f6c13000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c0c1614e0 0x7f6c0c1565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f6c13000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c0c0b3370 0x7f6c0c0b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f6c13000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c16e6c0 con 0x7f6c0c0b3370 2024-04-25T14:04:12.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f6c13000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c16e540 con 0x7f6c0c1614e0 2024-04-25T14:04:12.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f6c13000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6c0c16e840 con 0x7f6c0c06d740 2024-04-25T14:04:12.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7fde79e00640 1 --2- 172.21.15.86:0/1146252500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde7c0cfca0 0x7fde7c06f3f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fde6800ba60 tx=0x7fde6801ea00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=227a7b90a4a34fef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f0345400640 1 --2- 172.21.15.86:0/4134364944 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f034015dc90 0x7f034014ec10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0338014730 tx=0x7f0338014bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f7ab9400640 1 --2- 172.21.15.86:0/3192647560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ab40e79f0 0x7f7ab4006930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400c750 tx=0x7f7aa801c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bee3872c9bf3eb01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f6c11000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c0c0b3370 0x7f6c0c0b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f6c0be00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c0c06d740 0x7f6c0c1560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f6c0be00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c0c06d740 0x7f6c0c1560b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:48290/0 (socket says 172.21.15.86:48290) 2024-04-25T14:04:12.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f6c0be00640 1 -- 172.21.15.86:0/3569917292 learned_addr learned my addr 172.21.15.86:0/3569917292 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:12.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7faf23e00640 1 --2- 172.21.15.86:0/3990538937 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7faf0006f450 0x7faf00071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f033d600640 1 -- 172.21.15.86:0/4134364944 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033800e2d0 con 0x7f034015dc90 2024-04-25T14:04:12.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f033d600640 1 -- 172.21.15.86:0/4134364944 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f033801f070 con 0x7f034015dc90 2024-04-25T14:04:12.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f0347e00640 1 -- 172.21.15.86:0/4134364944 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f034014f150 con 0x7f034015dc90 2024-04-25T14:04:12.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7fde78a00640 1 -- 172.21.15.86:0/1146252500 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde680b4070 con 0x7fde7c0cfca0 2024-04-25T14:04:12.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7fde78a00640 1 -- 172.21.15.86:0/1146252500 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde680a7a30 con 0x7fde7c0cfca0 2024-04-25T14:04:12.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7fde78a00640 1 -- 172.21.15.86:0/1146252500 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde680a7e00 con 0x7fde7c0cfca0 2024-04-25T14:04:12.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f6c0b400640 1 --2- 172.21.15.86:0/3569917292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c0c1614e0 0x7f6c0c1565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f0347e00640 1 -- 172.21.15.86:0/4134364944 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0340151690 con 0x7f034015dc90 2024-04-25T14:04:12.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7feaf3e00640 1 -- 172.21.15.86:0/2735467408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feaec055d00 msgr2=0x7feaec04a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7feaf3e00640 1 --2- 172.21.15.86:0/2735467408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feaec055d00 0x7feaec04a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7feaf3e00640 1 -- 172.21.15.86:0/2735467408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feaec0092d0 msgr2=0x7feaec0d3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7feaf3e00640 1 --2- 172.21.15.86:0/2735467408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feaec0092d0 0x7feaec0d3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7feaf3e00640 1 -- 172.21.15.86:0/2735467408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaec04fc90 con 0x7feaec0e75b0 2024-04-25T14:04:12.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7feaf3400640 1 --2- 172.21.15.86:0/2735467408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feaec0092d0 0x7feaec0d3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:12.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7febb1800640 1 -- 172.21.15.86:0/3065797565 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7febac076b80 con 0x7febac0d17c0 2024-04-25T14:04:12.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f033d600640 1 -- 172.21.15.86:0/4134364944 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0338012800 con 0x7f034015dc90 2024-04-25T14:04:12.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f6c0be00640 1 -- 172.21.15.86:0/3569917292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c0c1614e0 msgr2=0x7f6c0c1565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f6c0be00640 1 --2- 172.21.15.86:0/3569917292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c0c1614e0 0x7f6c0c1565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f6c0be00640 1 -- 172.21.15.86:0/3569917292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c0c0b3370 msgr2=0x7f6c0c0b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f6c0be00640 1 --2- 172.21.15.86:0/3569917292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c0c0b3370 0x7f6c0c0b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f6c0be00640 1 -- 172.21.15.86:0/3569917292 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c0c153070 con 0x7f6c0c06d740 2024-04-25T14:04:12.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f6c11000640 1 --2- 172.21.15.86:0/3569917292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c0c0b3370 0x7f6c0c0b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:12.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7fde80a00640 1 -- 172.21.15.86:0/1146252500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde7c0cfca0 msgr2=0x7fde7c06f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7fde80a00640 1 --2- 172.21.15.86:0/1146252500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde7c0cfca0 0x7fde7c06f3f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fde6800ba60 tx=0x7fde6801ea00 comp rx=0 tx=0).stop 2024-04-25T14:04:12.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7feaf3e00640 1 --2- 172.21.15.86:0/2735467408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaec0e75b0 0x7feaec04acd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7feae000c410 tx=0x7feae000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f7ab8a00640 1 -- 172.21.15.86:0/3192647560 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa80ab430 con 0x7f7ab40e79f0 2024-04-25T14:04:12.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f7ab8a00640 1 -- 172.21.15.86:0/3192647560 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa80b0070 con 0x7f7ab40e79f0 2024-04-25T14:04:12.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f7ab8a00640 1 -- 172.21.15.86:0/3192647560 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa80ab7f0 con 0x7f7ab40e79f0 2024-04-25T14:04:12.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f0347e00640 1 -- 172.21.15.86:0/4134364944 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f031c001040 con 0x7f034015dc90 2024-04-25T14:04:12.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7faf1b400640 1 --2- 172.21.15.86:0/3990538937 >> [v2:172.21.15.86:6816/2851379769,v1:172.21.15.86:6817/2851379769] conn(0x7faf00078c10 0x7faf0007b0a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7f6c0be00640 1 --2- 172.21.15.86:0/3569917292 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c0c06d740 0x7f6c0c1560b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc019df0 tx=0x7f6bfc01aa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7feae7400640 1 -- 172.21.15.86:0/2735467408 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0007030 con 0x7feaec0e75b0 2024-04-25T14:04:12.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7feba8c00640 1 -- 172.21.15.86:0/3065797565 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7feb94003620 con 0x7febac0d17c0 2024-04-25T14:04:12.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7feae7400640 1 -- 172.21.15.86:0/2735467408 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feae00071f0 con 0x7feaec0e75b0 2024-04-25T14:04:12.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7feae7400640 1 -- 172.21.15.86:0/2735467408 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae00075e0 con 0x7feaec0e75b0 2024-04-25T14:04:12.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7fe7efc00640 1 Processor -- start 2024-04-25T14:04:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.749+0000 7feafa000640 1 -- 172.21.15.86:0/2735467408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaec04ff70 con 0x7feaec0e75b0 2024-04-25T14:04:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7feafa000640 1 -- 172.21.15.86:0/2735467408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaec050530 con 0x7feaec0e75b0 2024-04-25T14:04:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7fe7efc00640 1 -- start start 2024-04-25T14:04:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7f6c03e00640 1 -- 172.21.15.86:0/3569917292 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc0ba020 con 0x7f6c0c06d740 2024-04-25T14:04:12.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7f6c03e00640 1 -- 172.21.15.86:0/3569917292 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bfc0b6620 con 0x7f6c0c06d740 2024-04-25T14:04:12.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7f6c03e00640 1 -- 172.21.15.86:0/3569917292 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc0b0040 con 0x7f6c0c06d740 2024-04-25T14:04:12.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7f6c13000640 1 -- 172.21.15.86:0/3569917292 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c0c153570 con 0x7f6c0c06d740 2024-04-25T14:04:12.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7f7abb200640 1 -- 172.21.15.86:0/3192647560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ab40e79f0 msgr2=0x7f7ab4006930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7f7abb200640 1 --2- 172.21.15.86:0/3192647560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ab40e79f0 0x7f7ab4006930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400c750 tx=0x7f7aa801c540 comp rx=0 tx=0).stop 2024-04-25T14:04:12.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7feafa000640 1 -- 172.21.15.86:0/2735467408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fead0001040 con 0x7feaec0e75b0 2024-04-25T14:04:12.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7faf20c00640 1 -- 172.21.15.86:0/3990538937 <== osd.2 v2:172.21.15.86:6816/2851379769 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faf1c01ee70 con 0x7faf00078c10 2024-04-25T14:04:12.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7faf23e00640 1 --2- 172.21.15.86:0/3990538937 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7faf0006f450 0x7faf00071960 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7faf240d1280 tx=0x7faf14005f50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7fde80a00640 1 -- 172.21.15.86:0/1146252500 shutdown_connections 2024-04-25T14:04:12.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7fde80a00640 1 --2- 172.21.15.86:0/1146252500 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde7c0730b0 0x7fde7c0734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7fde80a00640 1 --2- 172.21.15.86:0/1146252500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde7c0cfca0 0x7fde7c06f3f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7fde80a00640 1 -- 172.21.15.86:0/1146252500 >> 172.21.15.86:0/1146252500 conn(0x7fde7c0b4650 msgr2=0x7fde7c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:12.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7f7abb200640 1 -- 172.21.15.86:0/3192647560 shutdown_connections 2024-04-25T14:04:12.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7f7abb200640 1 --2- 172.21.15.86:0/3192647560 >> v2:172.21.15.86:3300/0 conn(0x7f7ab40e7e00 0x7f7ab4006e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7f7abb200640 1 --2- 172.21.15.86:0/3192647560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ab40e79f0 0x7f7ab4006930 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7f7abb200640 1 -- 172.21.15.86:0/3192647560 >> 172.21.15.86:0/3192647560 conn(0x7f7ab4054ed0 msgr2=0x7f7ab40552e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:12.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7f7abb200640 1 -- 172.21.15.86:0/3192647560 shutdown_connections 2024-04-25T14:04:12.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7f7abb200640 1 -- 172.21.15.86:0/3192647560 wait complete. 2024-04-25T14:04:12.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7f033d600640 1 -- 172.21.15.86:0/4134364944 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f03380129a0 con 0x7f034015dc90 2024-04-25T14:04:12.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7fde80a00640 1 -- 172.21.15.86:0/1146252500 shutdown_connections 2024-04-25T14:04:12.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7feba8c00640 1 --2- 172.21.15.86:0/3065797565 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7feb8806f480 0x7feb88071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7f6c13000640 1 -- 172.21.15.86:0/3569917292 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c0c153b30 con 0x7f6c0c06d740 2024-04-25T14:04:12.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7feba8c00640 1 -- 172.21.15.86:0/3065797565 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7feb94129b20 con 0x7febac0d17c0 2024-04-25T14:04:12.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7feba8c00640 1 --2- 172.21.15.86:0/3065797565 >> [v2:172.21.15.86:6808/1043507731,v1:172.21.15.86:6809/1043507731] conn(0x7feb88078c50 0x7feb8807b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7feba3400640 1 --2- 172.21.15.86:0/3065797565 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7feb8806f480 0x7feb88071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7feba8c00640 1 -- 172.21.15.86:0/3065797565 --> [v2:172.21.15.86:6808/1043507731,v1:172.21.15.86:6809/1043507731] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feb94008ea0 con 0x7feb88078c50 2024-04-25T14:04:12.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7feba8c00640 1 -- 172.21.15.86:0/3065797565 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7feb9412a000 con 0x7febac0d17c0 2024-04-25T14:04:12.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7fde80a00640 1 -- 172.21.15.86:0/1146252500 wait complete. 2024-04-25T14:04:12.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7f033d600640 1 --2- 172.21.15.86:0/4134364944 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f032406f480 0x7f0324071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7f033d600640 1 -- 172.21.15.86:0/4134364944 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f0338097150 con 0x7f034015dc90 2024-04-25T14:04:12.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7f6c13000640 1 -- 172.21.15.86:0/3569917292 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6c0c153d70 con 0x7f6c0c06d740 2024-04-25T14:04:12.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7f033d600640 1 --2- 172.21.15.86:0/4134364944 >> [v2:172.21.15.155:6800/1655306076,v1:172.21.15.155:6802/1655306076] conn(0x7f0324078c50 0x7f032407b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7f033d600640 1 -- 172.21.15.86:0/4134364944 --> [v2:172.21.15.155:6800/1655306076,v1:172.21.15.155:6802/1655306076] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f032407b7b0 con 0x7f0324078c50 2024-04-25T14:04:12.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7f033d600640 1 -- 172.21.15.86:0/4134364944 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0338097500 con 0x7f034015dc90 2024-04-25T14:04:12.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7feae7400640 1 -- 172.21.15.86:0/2735467408 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7feae001a020 con 0x7feaec0e75b0 2024-04-25T14:04:12.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.753+0000 7febb0e00640 1 --2- 172.21.15.86:0/3065797565 >> [v2:172.21.15.86:6808/1043507731,v1:172.21.15.86:6809/1043507731] conn(0x7feb88078c50 0x7feb8807b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7febb0e00640 1 --2- 172.21.15.86:0/3065797565 >> [v2:172.21.15.86:6808/1043507731,v1:172.21.15.86:6809/1043507731] conn(0x7feb88078c50 0x7feb8807b0e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7f0344a00640 1 --2- 172.21.15.86:0/4134364944 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f032406f480 0x7f0324071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7f0345e00640 1 --2- 172.21.15.86:0/4134364944 >> [v2:172.21.15.155:6800/1655306076,v1:172.21.15.155:6802/1655306076] conn(0x7f0324078c50 0x7f032407b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7feba3400640 1 --2- 172.21.15.86:0/3065797565 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7feb8806f480 0x7feb88071990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0070b0 tx=0x7feb9c007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7f0344a00640 1 --2- 172.21.15.86:0/4134364944 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f032406f480 0x7f0324071990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f03300af000 tx=0x7f03300a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7fe7efc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7e806f9f0 0x7fe7e815a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7fe7efc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7e80c47f0 0x7fe7e80c5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7f0345e00640 1 --2- 172.21.15.86:0/4134364944 >> [v2:172.21.15.155:6800/1655306076,v1:172.21.15.155:6802/1655306076] conn(0x7f0324078c50 0x7f032407b0e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7f7abb200640 1 Processor -- start 2024-04-25T14:04:12.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7fe7efc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7e80ce570 0x7fe7e80c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7fe7efc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e8167b80 con 0x7fe7e806f9f0 2024-04-25T14:04:12.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7fe7efc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e8167a00 con 0x7fe7e80ce570 2024-04-25T14:04:12.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7fe7efc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe7e8168a70 con 0x7fe7e80c47f0 2024-04-25T14:04:12.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7feae7400640 1 --2- 172.21.15.86:0/2735467408 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fead406f4d0 0x7fead40719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7feae7400640 1 -- 172.21.15.86:0/2735467408 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7feae008f5a0 con 0x7feaec0e75b0 2024-04-25T14:04:12.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7feae7400640 1 --2- 172.21.15.86:0/2735467408 >> [v2:172.21.15.86:6822/2230428952,v1:172.21.15.86:6824/2230428952] conn(0x7fead4078c90 0x7fead407b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7feae7400640 1 -- 172.21.15.86:0/2735467408 --> [v2:172.21.15.86:6822/2230428952,v1:172.21.15.86:6824/2230428952] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feae0016ec0 con 0x7fead4078c90 2024-04-25T14:04:12.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7feae7400640 1 -- 172.21.15.86:0/2735467408 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7feae008f990 con 0x7feaec0e75b0 2024-04-25T14:04:12.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7fe7edc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7e80ce570 0x7fe7e80c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7fe7edc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7e80ce570 0x7fe7e80c5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:54358/0 (socket says 172.21.15.86:54358) 2024-04-25T14:04:12.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7fe7edc00640 1 -- 172.21.15.86:0/1578516589 learned_addr learned my addr 172.21.15.86:0/1578516589 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:12.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7fe7ed200640 1 --2- 172.21.15.86:0/1578516589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7e806f9f0 0x7fe7e815a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7f7abb200640 1 -- start start 2024-04-25T14:04:12.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7feaf2a00640 1 --2- 172.21.15.86:0/2735467408 >> [v2:172.21.15.86:6822/2230428952,v1:172.21.15.86:6824/2230428952] conn(0x7fead4078c90 0x7fead407b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7f033d600640 1 -- 172.21.15.86:0/4134364944 <== osd.4 v2:172.21.15.155:6800/1655306076 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f032407b7b0 con 0x7f0324078c50 2024-04-25T14:04:12.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7feba8c00640 1 -- 172.21.15.86:0/3065797565 <== osd.1 v2:172.21.15.86:6808/1043507731 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feb94008ea0 con 0x7feb88078c50 2024-04-25T14:04:12.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7fe7ed200640 1 -- 172.21.15.86:0/1578516589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7e80c47f0 msgr2=0x7fe7e80c5800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:04:12.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7fe7ed200640 1 --2- 172.21.15.86:0/1578516589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7e80c47f0 0x7fe7e80c5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7fe7ed200640 1 -- 172.21.15.86:0/1578516589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7e80ce570 msgr2=0x7fe7e80c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7fe7ed200640 1 --2- 172.21.15.86:0/1578516589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7e80ce570 0x7fe7e80c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7fe7ed200640 1 -- 172.21.15.86:0/1578516589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e80b1860 con 0x7fe7e806f9f0 2024-04-25T14:04:12.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7fde80a00640 1 Processor -- start 2024-04-25T14:04:12.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.757+0000 7fde80a00640 1 -- start start 2024-04-25T14:04:12.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7feaf3400640 1 --2- 172.21.15.86:0/2735467408 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fead406f4d0 0x7fead40719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7f7abb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ab40e3760 0x7f7ab40e6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7f7abb200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ab40e79f0 0x7f7ab404e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7f7abb200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ab40e7e00 0x7f7ab4184e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7f7abb200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab40f0800 con 0x7f7ab40e3760 2024-04-25T14:04:12.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7f7abb200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab40f0680 con 0x7f7ab40e7e00 2024-04-25T14:04:12.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7f7abb200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7ab40f0980 con 0x7f7ab40e79f0 2024-04-25T14:04:12.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7feaf2a00640 1 --2- 172.21.15.86:0/2735467408 >> [v2:172.21.15.86:6822/2230428952,v1:172.21.15.86:6824/2230428952] conn(0x7fead4078c90 0x7fead407b120 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7f7ab9400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ab40e79f0 0x7f7ab404e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7f7ac0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ab40e7e00 0x7f7ab4184e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7f7ab9400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ab40e79f0 0x7f7ab404e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:48304/0 (socket says 172.21.15.86:48304) 2024-04-25T14:04:12.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7f7ab9400640 1 -- 172.21.15.86:0/1396964250 learned_addr learned my addr 172.21.15.86:0/1396964250 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:12.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7feaf3400640 1 --2- 172.21.15.86:0/2735467408 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fead406f4d0 0x7fead40719e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7feae80af000 tx=0x7feae80a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7fde80a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde7c0730b0 0x7fde7c15d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7fde80a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c0cfca0 0x7fde7c1fab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7fde80a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde7c0d16e0 0x7fde7c0af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7fde80a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c073a00 con 0x7fde7c0cfca0 2024-04-25T14:04:12.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7fde80a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c070190 con 0x7fde7c0d16e0 2024-04-25T14:04:12.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7fde80a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fde7c073b80 con 0x7fde7c0730b0 2024-04-25T14:04:12.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7f7ab9400640 1 -- 172.21.15.86:0/1396964250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ab40e7e00 msgr2=0x7f7ab4184e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7f7ab9400640 1 --2- 172.21.15.86:0/1396964250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ab40e7e00 0x7f7ab4184e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7f7ab9400640 1 -- 172.21.15.86:0/1396964250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ab40e3760 msgr2=0x7f7ab40e6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7f7ab9400640 1 --2- 172.21.15.86:0/1396964250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ab40e3760 0x7f7ab40e6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7f7ab9400640 1 -- 172.21.15.86:0/1396964250 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab404b7a0 con 0x7f7ab40e79f0 2024-04-25T14:04:12.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7fde79e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde7c0730b0 0x7fde7c15d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7f6c03e00640 1 -- 172.21.15.86:0/3569917292 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6bfc0bd020 con 0x7f6c0c06d740 2024-04-25T14:04:12.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7f6c03e00640 1 --2- 172.21.15.86:0/3569917292 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6bec06f4d0 0x7f6bec0719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7f6c03e00640 1 -- 172.21.15.86:0/3569917292 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6bfc1296c0 con 0x7f6c0c06d740 2024-04-25T14:04:12.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7f6c03e00640 1 --2- 172.21.15.86:0/3569917292 >> [v2:172.21.15.155:6817/288803543,v1:172.21.15.155:6818/288803543] conn(0x7f6bec078ca0 0x7f6bec07b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7f6c03e00640 1 -- 172.21.15.86:0/3569917292 --> [v2:172.21.15.155:6817/288803543,v1:172.21.15.155:6818/288803543] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6bfc00af30 con 0x7f6bec078ca0 2024-04-25T14:04:12.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7f6c03e00640 1 -- 172.21.15.86:0/3569917292 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6bfc130050 con 0x7f6c0c06d740 2024-04-25T14:04:12.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7fde79e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde7c0730b0 0x7fde7c15d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:48312/0 (socket says 172.21.15.86:48312) 2024-04-25T14:04:12.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7fde79e00640 1 -- 172.21.15.86:0/574889499 learned_addr learned my addr 172.21.15.86:0/574889499 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:12.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7fde7a800640 1 --2- 172.21.15.86:0/574889499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde7c0d16e0 0x7fde7c0af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7fde79400640 1 --2- 172.21.15.86:0/574889499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c0cfca0 0x7fde7c1fab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7fde79e00640 1 -- 172.21.15.86:0/574889499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde7c0d16e0 msgr2=0x7fde7c0af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7feae7400640 1 -- 172.21.15.86:0/2735467408 <== osd.0 v2:172.21.15.86:6822/2230428952 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feae0016ec0 con 0x7fead4078c90 2024-04-25T14:04:12.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7fe7ed200640 1 --2- 172.21.15.86:0/1578516589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7e806f9f0 0x7fe7e815a640 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400c540 tx=0x7fe7d400ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.761+0000 7f7ab9400640 1 --2- 172.21.15.86:0/1396964250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ab40e79f0 0x7f7ab404e3a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8019150 tx=0x7f7aa801ad80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.765+0000 7fde79e00640 1 --2- 172.21.15.86:0/574889499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde7c0d16e0 0x7fde7c0af4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.765+0000 7fde79e00640 1 -- 172.21.15.86:0/574889499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c0cfca0 msgr2=0x7fde7c1fab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.765+0000 7fde79e00640 1 --2- 172.21.15.86:0/574889499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c0cfca0 0x7fde7c1fab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.765+0000 7fde79e00640 1 -- 172.21.15.86:0/574889499 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7c15dc30 con 0x7fde7c0730b0 2024-04-25T14:04:12.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.765+0000 7fde7a800640 1 --2- 172.21.15.86:0/574889499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde7c0d16e0 0x7fde7c0af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:12.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.765+0000 7fde79400640 1 --2- 172.21.15.86:0/574889499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c0cfca0 0x7fde7c1fab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:12.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.765+0000 7f6c11000640 1 --2- 172.21.15.86:0/3569917292 >> [v2:172.21.15.155:6817/288803543,v1:172.21.15.155:6818/288803543] conn(0x7f6bec078ca0 0x7f6bec07b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.765+0000 7fde79e00640 1 --2- 172.21.15.86:0/574889499 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde7c0730b0 0x7fde7c15d170 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fde6801f2a0 tx=0x7fde6801b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.765+0000 7f7ab2000640 1 -- 172.21.15.86:0/1396964250 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa80b4800 con 0x7f7ab40e79f0 2024-04-25T14:04:12.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.765+0000 7f7ab2000640 1 -- 172.21.15.86:0/1396964250 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa80b49a0 con 0x7f7ab40e79f0 2024-04-25T14:04:12.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.765+0000 7f7ab2000640 1 -- 172.21.15.86:0/1396964250 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa80b4d90 con 0x7f7ab40e79f0 2024-04-25T14:04:12.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.765+0000 7f6c0b400640 1 --2- 172.21.15.86:0/3569917292 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6bec06f4d0 0x7f6bec0719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.765+0000 7fde72000640 1 -- 172.21.15.86:0/574889499 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde680bf070 con 0x7fde7c0730b0 2024-04-25T14:04:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.765+0000 7fde72000640 1 -- 172.21.15.86:0/574889499 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde680b0c00 con 0x7fde7c0730b0 2024-04-25T14:04:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.765+0000 7fde72000640 1 -- 172.21.15.86:0/574889499 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde680b4040 con 0x7fde7c0730b0 2024-04-25T14:04:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.765+0000 7fde80a00640 1 -- 172.21.15.86:0/574889499 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde7c15df10 con 0x7fde7c0730b0 2024-04-25T14:04:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.765+0000 7fe7e4c00640 1 -- 172.21.15.86:0/1578516589 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4015070 con 0x7fe7e806f9f0 2024-04-25T14:04:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.765+0000 7fde80a00640 1 -- 172.21.15.86:0/574889499 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fde7c15e9f0 con 0x7fde7c0730b0 2024-04-25T14:04:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.765+0000 7fe7e4c00640 1 -- 172.21.15.86:0/1578516589 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7d4011800 con 0x7fe7e806f9f0 2024-04-25T14:04:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.765+0000 7fe7e4c00640 1 -- 172.21.15.86:0/1578516589 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4011bf0 con 0x7fe7e806f9f0 2024-04-25T14:04:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.765+0000 7fe7efc00640 1 -- 172.21.15.86:0/1578516589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e80b1b40 con 0x7fe7e806f9f0 2024-04-25T14:04:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.765+0000 7fe7efc00640 1 -- 172.21.15.86:0/1578516589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e815c410 con 0x7fe7e806f9f0 2024-04-25T14:04:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.765+0000 7fe7efc00640 1 -- 172.21.15.86:0/1578516589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe7c0001040 con 0x7fe7e806f9f0 2024-04-25T14:04:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.765+0000 7fde80a00640 1 -- 172.21.15.86:0/574889499 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fde54001040 con 0x7fde7c0730b0 2024-04-25T14:04:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.765+0000 7f7abb200640 1 -- 172.21.15.86:0/1396964250 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab40e5f00 con 0x7f7ab40e79f0 2024-04-25T14:04:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.765+0000 7f7abb200640 1 -- 172.21.15.86:0/1396964250 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab40db0e0 con 0x7f7ab40e79f0 2024-04-25T14:04:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.765+0000 7f7aa7400640 1 -- 172.21.15.86:0/1396964250 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7a90001040 con 0x7f7ab40e79f0 2024-04-25T14:04:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.769+0000 7f6c11000640 1 --2- 172.21.15.86:0/3569917292 >> [v2:172.21.15.155:6817/288803543,v1:172.21.15.155:6818/288803543] conn(0x7f6bec078ca0 0x7f6bec07b130 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.769+0000 7f6c0b400640 1 --2- 172.21.15.86:0/3569917292 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6bec06f4d0 0x7f6bec0719e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c06e050 tx=0x7f6bf8009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.769+0000 7f6c03e00640 1 -- 172.21.15.86:0/3569917292 <== osd.5 v2:172.21.15.155:6817/288803543 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6bfc00af30 con 0x7f6bec078ca0 2024-04-25T14:04:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.769+0000 7f7ab2000640 1 -- 172.21.15.86:0/1396964250 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7aa80be3f0 con 0x7f7ab40e79f0 2024-04-25T14:04:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.769+0000 7f7ab2000640 1 --2- 172.21.15.86:0/1396964250 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7a9c06f450 0x7f7a9c071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.769+0000 7f7ab2000640 1 -- 172.21.15.86:0/1396964250 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7aa812a050 con 0x7f7ab40e79f0 2024-04-25T14:04:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.769+0000 7f7ab9e00640 1 --2- 172.21.15.86:0/1396964250 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7a9c06f450 0x7f7a9c071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.773+0000 7f7ab2000640 1 --2- 172.21.15.86:0/1396964250 >> [v2:172.21.15.155:6816/2704073457,v1:172.21.15.155:6819/2704073457] conn(0x7f7a9c078c10 0x7f7a9c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.773+0000 7f7ab2000640 1 -- 172.21.15.86:0/1396964250 --> [v2:172.21.15.155:6816/2704073457,v1:172.21.15.155:6819/2704073457] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7aa80b2e90 con 0x7f7a9c078c10 2024-04-25T14:04:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.773+0000 7f7ab2000640 1 -- 172.21.15.86:0/1396964250 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7aa812a4b0 con 0x7f7ab40e79f0 2024-04-25T14:04:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.769+0000 7fe7e4c00640 1 -- 172.21.15.86:0/1578516589 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe7d4018070 con 0x7fe7e806f9f0 2024-04-25T14:04:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.773+0000 7fde72000640 1 -- 172.21.15.86:0/574889499 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fde680ba430 con 0x7fde7c0730b0 2024-04-25T14:04:12.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.773+0000 7f7ab9e00640 1 --2- 172.21.15.86:0/1396964250 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7a9c06f450 0x7f7a9c071960 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7aac000f70 tx=0x7f7aac0079c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.773+0000 7fe7e4c00640 1 --2- 172.21.15.86:0/1578516589 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe7c806f480 0x7fe7c8071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.773+0000 7fe7e4c00640 1 -- 172.21.15.86:0/1578516589 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe7d408f730 con 0x7fe7e806f9f0 2024-04-25T14:04:12.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.773+0000 7fe7e4c00640 1 --2- 172.21.15.86:0/1578516589 >> [v2:172.21.15.86:6800/2257199129,v1:172.21.15.86:6801/2257199129] conn(0x7fe7c8078c50 0x7fe7c807b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.773+0000 7fe7e4c00640 1 -- 172.21.15.86:0/1578516589 --> [v2:172.21.15.86:6800/2257199129,v1:172.21.15.86:6801/2257199129] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe7d4014e50 con 0x7fe7c8078c50 2024-04-25T14:04:12.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.773+0000 7fe7e4c00640 1 -- 172.21.15.86:0/1578516589 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe7d408fc10 con 0x7fe7e806f9f0 2024-04-25T14:04:12.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.773+0000 7fe7e7e00640 1 --2- 172.21.15.86:0/1578516589 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe7c806f480 0x7fe7c8071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.773+0000 7f7ac0c00640 1 --2- 172.21.15.86:0/1396964250 >> [v2:172.21.15.155:6816/2704073457,v1:172.21.15.155:6819/2704073457] conn(0x7f7a9c078c10 0x7f7a9c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.773+0000 7fe7edc00640 1 --2- 172.21.15.86:0/1578516589 >> [v2:172.21.15.86:6800/2257199129,v1:172.21.15.86:6801/2257199129] conn(0x7fe7c8078c50 0x7fe7c807b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.773+0000 7fe7e7e00640 1 --2- 172.21.15.86:0/1578516589 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe7c806f480 0x7fe7c8071990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe7e815c1a0 tx=0x7fe7d80010b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.773+0000 7f7ac0c00640 1 --2- 172.21.15.86:0/1396964250 >> [v2:172.21.15.155:6816/2704073457,v1:172.21.15.155:6819/2704073457] conn(0x7f7a9c078c10 0x7f7a9c07b0a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.773+0000 7fde72000640 1 --2- 172.21.15.86:0/574889499 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fde5c06f500 0x7fde5c071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.773+0000 7fde72000640 1 -- 172.21.15.86:0/574889499 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fde6812a040 con 0x7fde7c0730b0 2024-04-25T14:04:12.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.773+0000 7fde72000640 1 --2- 172.21.15.86:0/574889499 >> [v2:172.21.15.155:6801/2597410484,v1:172.21.15.155:6803/2597410484] conn(0x7fde5c078cc0 0x7fde5c07b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:12.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.773+0000 7fde72000640 1 -- 172.21.15.86:0/574889499 --> [v2:172.21.15.155:6801/2597410484,v1:172.21.15.155:6803/2597410484] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fde68007ea0 con 0x7fde5c078cc0 2024-04-25T14:04:12.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.773+0000 7fde72000640 1 -- 172.21.15.86:0/574889499 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fde6812a430 con 0x7fde7c0730b0 2024-04-25T14:04:12.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.773+0000 7fde79400640 1 --2- 172.21.15.86:0/574889499 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fde5c06f500 0x7fde5c071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.773+0000 7fde7a800640 1 --2- 172.21.15.86:0/574889499 >> [v2:172.21.15.155:6801/2597410484,v1:172.21.15.155:6803/2597410484] conn(0x7fde5c078cc0 0x7fde5c07b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:12.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.777+0000 7f7ab2000640 1 -- 172.21.15.86:0/1396964250 <== osd.7 v2:172.21.15.155:6816/2704073457 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7aa80b2e90 con 0x7f7a9c078c10 2024-04-25T14:04:12.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.777+0000 7fe7edc00640 1 --2- 172.21.15.86:0/1578516589 >> [v2:172.21.15.86:6800/2257199129,v1:172.21.15.86:6801/2257199129] conn(0x7fe7c8078c50 0x7fe7c807b0e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.777+0000 7fde79400640 1 --2- 172.21.15.86:0/574889499 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fde5c06f500 0x7fde5c071a10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fde6c000c10 tx=0x7fde6c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.777+0000 7fde7a800640 1 --2- 172.21.15.86:0/574889499 >> [v2:172.21.15.155:6801/2597410484,v1:172.21.15.155:6803/2597410484] conn(0x7fde5c078cc0 0x7fde5c07b150 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.777+0000 7fde72000640 1 -- 172.21.15.86:0/574889499 <== osd.6 v2:172.21.15.155:6801/2597410484 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fde68007ea0 con 0x7fde5c078cc0 2024-04-25T14:04:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.777+0000 7fe7e4c00640 1 -- 172.21.15.86:0/1578516589 <== osd.3 v2:172.21.15.86:6800/2257199129 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe7d4014e50 con 0x7fe7c8078c50 2024-04-25T14:04:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.781+0000 7feafa000640 1 -- 172.21.15.86:0/2735467408 --> [v2:172.21.15.86:6822/2230428952,v1:172.21.15.86:6824/2230428952] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fead0002de0 con 0x7fead4078c90 2024-04-25T14:04:12.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.785+0000 7f0347e00640 1 -- 172.21.15.86:0/4134364944 --> [v2:172.21.15.155:6800/1655306076,v1:172.21.15.155:6802/1655306076] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f031c002de0 con 0x7f0324078c50 2024-04-25T14:04:12.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.785+0000 7feae7400640 1 -- 172.21.15.86:0/2735467408 <== osd.0 v2:172.21.15.86:6822/2230428952 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fead0002de0 con 0x7fead4078c90 2024-04-25T14:04:12.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.785+0000 7feafa000640 1 -- 172.21.15.86:0/2735467408 >> [v2:172.21.15.86:6822/2230428952,v1:172.21.15.86:6824/2230428952] conn(0x7fead4078c90 msgr2=0x7fead407b120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.785+0000 7feafa000640 1 --2- 172.21.15.86:0/2735467408 >> [v2:172.21.15.86:6822/2230428952,v1:172.21.15.86:6824/2230428952] conn(0x7fead4078c90 0x7fead407b120 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.785+0000 7f033d600640 1 -- 172.21.15.86:0/4134364944 <== osd.4 v2:172.21.15.155:6800/1655306076 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f031c002de0 con 0x7f0324078c50 2024-04-25T14:04:12.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.785+0000 7f0347e00640 1 -- 172.21.15.86:0/4134364944 >> [v2:172.21.15.155:6800/1655306076,v1:172.21.15.155:6802/1655306076] conn(0x7f0324078c50 msgr2=0x7f032407b0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.785+0000 7f0347e00640 1 --2- 172.21.15.86:0/4134364944 >> [v2:172.21.15.155:6800/1655306076,v1:172.21.15.155:6802/1655306076] conn(0x7f0324078c50 0x7f032407b0e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.785+0000 7feafa000640 1 -- 172.21.15.86:0/2735467408 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fead406f4d0 msgr2=0x7fead40719e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.785+0000 7feafa000640 1 --2- 172.21.15.86:0/2735467408 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fead406f4d0 0x7fead40719e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7feae80af000 tx=0x7feae80a7070 comp rx=0 tx=0).stop 2024-04-25T14:04:12.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.785+0000 7feafa000640 1 -- 172.21.15.86:0/2735467408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaec0e75b0 msgr2=0x7feaec04acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.785+0000 7feafa000640 1 --2- 172.21.15.86:0/2735467408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaec0e75b0 0x7feaec04acd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7feae000c410 tx=0x7feae000c8d0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.785+0000 7f0347e00640 1 -- 172.21.15.86:0/4134364944 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f032406f480 msgr2=0x7f0324071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.785+0000 7f0347e00640 1 --2- 172.21.15.86:0/4134364944 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f032406f480 0x7f0324071990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f03300af000 tx=0x7f03300a7070 comp rx=0 tx=0).stop 2024-04-25T14:04:12.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.785+0000 7f0347e00640 1 -- 172.21.15.86:0/4134364944 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f034015dc90 msgr2=0x7f034014ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.785+0000 7f0347e00640 1 --2- 172.21.15.86:0/4134364944 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f034015dc90 0x7f034014ec10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0338014730 tx=0x7f0338014bf0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.785+0000 7f0345e00640 1 -- 172.21.15.86:0/4134364944 reap_dead start 2024-04-25T14:04:12.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.785+0000 7f0347e00640 1 -- 172.21.15.86:0/4134364944 shutdown_connections 2024-04-25T14:04:12.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.785+0000 7f0347e00640 1 -- 172.21.15.86:0/4134364944 >> 172.21.15.86:0/4134364944 conn(0x7f03400cffc0 msgr2=0x7f034015c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:12.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.785+0000 7feaf3e00640 1 -- 172.21.15.86:0/2735467408 reap_dead start 2024-04-25T14:04:12.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.789+0000 7feafa000640 1 -- 172.21.15.86:0/2735467408 shutdown_connections 2024-04-25T14:04:12.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.789+0000 7feafa000640 1 -- 172.21.15.86:0/2735467408 >> 172.21.15.86:0/2735467408 conn(0x7feaec053990 msgr2=0x7feaec0381b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:12.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.789+0000 7feafa000640 1 -- 172.21.15.86:0/2735467408 shutdown_connections 2024-04-25T14:04:12.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.789+0000 7feafa000640 1 -- 172.21.15.86:0/2735467408 wait complete. 2024-04-25T14:04:12.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.789+0000 7febb1800640 1 -- 172.21.15.86:0/3065797565 --> [v2:172.21.15.86:6808/1043507731,v1:172.21.15.86:6809/1043507731] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7febac0742d0 con 0x7feb88078c50 2024-04-25T14:04:12.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.789+0000 7f0347e00640 1 -- 172.21.15.86:0/4134364944 shutdown_connections 2024-04-25T14:04:12.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.789+0000 7f0347e00640 1 -- 172.21.15.86:0/4134364944 wait complete. 2024-04-25T14:04:12.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.789+0000 7feba8c00640 1 -- 172.21.15.86:0/3065797565 <== osd.1 v2:172.21.15.86:6808/1043507731 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7febac0742d0 con 0x7feb88078c50 2024-04-25T14:04:12.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.789+0000 7febb1800640 1 -- 172.21.15.86:0/3065797565 >> [v2:172.21.15.86:6808/1043507731,v1:172.21.15.86:6809/1043507731] conn(0x7feb88078c50 msgr2=0x7feb8807b0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.789+0000 7febb1800640 1 --2- 172.21.15.86:0/3065797565 >> [v2:172.21.15.86:6808/1043507731,v1:172.21.15.86:6809/1043507731] conn(0x7feb88078c50 0x7feb8807b0e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.789+0000 7febb1800640 1 -- 172.21.15.86:0/3065797565 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7feb8806f480 msgr2=0x7feb88071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.789+0000 7faf2ac00640 1 -- 172.21.15.86:0/3990538937 --> [v2:172.21.15.86:6816/2851379769,v1:172.21.15.86:6817/2851379769] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faf24069d00 con 0x7faf00078c10 2024-04-25T14:04:12.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.789+0000 7febb1800640 1 --2- 172.21.15.86:0/3065797565 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7feb8806f480 0x7feb88071990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0070b0 tx=0x7feb9c007040 comp rx=0 tx=0).stop 2024-04-25T14:04:12.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.789+0000 7febb1800640 1 -- 172.21.15.86:0/3065797565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febac0d17c0 msgr2=0x7febac0cc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.789+0000 7febb1800640 1 --2- 172.21.15.86:0/3065797565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febac0d17c0 0x7febac0cc030 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7feb94000c10 tx=0x7feb94019230 comp rx=0 tx=0).stop 2024-04-25T14:04:12.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.793+0000 7febb0e00640 1 -- 172.21.15.86:0/3065797565 reap_dead start 2024-04-25T14:04:12.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.793+0000 7febb1800640 1 -- 172.21.15.86:0/3065797565 shutdown_connections 2024-04-25T14:04:12.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.793+0000 7febb1800640 1 -- 172.21.15.86:0/3065797565 >> 172.21.15.86:0/3065797565 conn(0x7febac073c00 msgr2=0x7febac15d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:12.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.793+0000 7febb1800640 1 -- 172.21.15.86:0/3065797565 shutdown_connections 2024-04-25T14:04:12.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.793+0000 7febb1800640 1 -- 172.21.15.86:0/3065797565 wait complete. 2024-04-25T14:04:12.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.793+0000 7faf20c00640 1 -- 172.21.15.86:0/3990538937 <== osd.2 v2:172.21.15.86:6816/2851379769 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7faf24069d00 con 0x7faf00078c10 2024-04-25T14:04:12.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.793+0000 7faf2ac00640 1 -- 172.21.15.86:0/3990538937 >> [v2:172.21.15.86:6816/2851379769,v1:172.21.15.86:6817/2851379769] conn(0x7faf00078c10 msgr2=0x7faf0007b0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.793+0000 7faf2ac00640 1 --2- 172.21.15.86:0/3990538937 >> [v2:172.21.15.86:6816/2851379769,v1:172.21.15.86:6817/2851379769] conn(0x7faf00078c10 0x7faf0007b0a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.797+0000 7faf2ac00640 1 -- 172.21.15.86:0/3990538937 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7faf0006f450 msgr2=0x7faf00071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.797+0000 7faf2ac00640 1 --2- 172.21.15.86:0/3990538937 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7faf0006f450 0x7faf00071960 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7faf240d1280 tx=0x7faf14005f50 comp rx=0 tx=0).stop 2024-04-25T14:04:12.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.797+0000 7faf2ac00640 1 -- 172.21.15.86:0/3990538937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf240cdb60 msgr2=0x7faf2415b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.797+0000 7faf2ac00640 1 --2- 172.21.15.86:0/3990538937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf240cdb60 0x7faf2415b2c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00b440 tx=0x7faf1c00b900 comp rx=0 tx=0).stop 2024-04-25T14:04:12.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.797+0000 7faf28c00640 1 -- 172.21.15.86:0/3990538937 reap_dead start 2024-04-25T14:04:12.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.797+0000 7faf2ac00640 1 -- 172.21.15.86:0/3990538937 shutdown_connections 2024-04-25T14:04:12.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.797+0000 7faf2ac00640 1 -- 172.21.15.86:0/3990538937 >> 172.21.15.86:0/3990538937 conn(0x7faf240b6130 msgr2=0x7faf24070270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:12.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.797+0000 7faf2ac00640 1 -- 172.21.15.86:0/3990538937 shutdown_connections 2024-04-25T14:04:12.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.797+0000 7faf2ac00640 1 -- 172.21.15.86:0/3990538937 wait complete. 2024-04-25T14:04:12.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.801+0000 7f6c13000640 1 -- 172.21.15.86:0/3569917292 --> [v2:172.21.15.155:6817/288803543,v1:172.21.15.155:6818/288803543] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6c0c0a07f0 con 0x7f6bec078ca0 2024-04-25T14:04:12.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.801+0000 7f6c03e00640 1 -- 172.21.15.86:0/3569917292 <== osd.5 v2:172.21.15.155:6817/288803543 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6c0c0a07f0 con 0x7f6bec078ca0 2024-04-25T14:04:12.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.805+0000 7f6c01600640 1 -- 172.21.15.86:0/3569917292 >> [v2:172.21.15.155:6817/288803543,v1:172.21.15.155:6818/288803543] conn(0x7f6bec078ca0 msgr2=0x7f6bec07b130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.805+0000 7f6c01600640 1 --2- 172.21.15.86:0/3569917292 >> [v2:172.21.15.155:6817/288803543,v1:172.21.15.155:6818/288803543] conn(0x7f6bec078ca0 0x7f6bec07b130 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.805+0000 7f6c01600640 1 -- 172.21.15.86:0/3569917292 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6bec06f4d0 msgr2=0x7f6bec0719e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.805+0000 7f6c01600640 1 --2- 172.21.15.86:0/3569917292 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6bec06f4d0 0x7f6bec0719e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c06e050 tx=0x7f6bf8009060 comp rx=0 tx=0).stop 2024-04-25T14:04:12.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.805+0000 7f6c01600640 1 -- 172.21.15.86:0/3569917292 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c0c06d740 msgr2=0x7f6c0c1560b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.805+0000 7f6c01600640 1 --2- 172.21.15.86:0/3569917292 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c0c06d740 0x7f6c0c1560b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc019df0 tx=0x7f6bfc01aa00 comp rx=0 tx=0).stop 2024-04-25T14:04:12.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.805+0000 7f6c11000640 1 -- 172.21.15.86:0/3569917292 reap_dead start 2024-04-25T14:04:12.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.805+0000 7f6c01600640 1 -- 172.21.15.86:0/3569917292 shutdown_connections 2024-04-25T14:04:12.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.805+0000 7f6c01600640 1 -- 172.21.15.86:0/3569917292 >> 172.21.15.86:0/3569917292 conn(0x7f6c0c073da0 msgr2=0x7f6c0c071160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:12.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.805+0000 7f6c01600640 1 -- 172.21.15.86:0/3569917292 shutdown_connections 2024-04-25T14:04:12.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.805+0000 7f6c01600640 1 -- 172.21.15.86:0/3569917292 wait complete. 2024-04-25T14:04:12.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.805+0000 7f7aa7400640 1 -- 172.21.15.86:0/1396964250 --> [v2:172.21.15.155:6816/2704073457,v1:172.21.15.155:6819/2704073457] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7a90002de0 con 0x7f7a9c078c10 2024-04-25T14:04:12.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.805+0000 7f7ab2000640 1 -- 172.21.15.86:0/1396964250 <== osd.7 v2:172.21.15.155:6816/2704073457 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7a90002de0 con 0x7f7a9c078c10 2024-04-25T14:04:12.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.809+0000 7f7aa7400640 1 -- 172.21.15.86:0/1396964250 >> [v2:172.21.15.155:6816/2704073457,v1:172.21.15.155:6819/2704073457] conn(0x7f7a9c078c10 msgr2=0x7f7a9c07b0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.809+0000 7f7aa7400640 1 --2- 172.21.15.86:0/1396964250 >> [v2:172.21.15.155:6816/2704073457,v1:172.21.15.155:6819/2704073457] conn(0x7f7a9c078c10 0x7f7a9c07b0a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.809+0000 7fde80a00640 1 -- 172.21.15.86:0/574889499 --> [v2:172.21.15.155:6801/2597410484,v1:172.21.15.155:6803/2597410484] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fde54002de0 con 0x7fde5c078cc0 2024-04-25T14:04:12.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.809+0000 7f7aa7400640 1 -- 172.21.15.86:0/1396964250 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7a9c06f450 msgr2=0x7f7a9c071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.809+0000 7f7aa7400640 1 --2- 172.21.15.86:0/1396964250 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7a9c06f450 0x7f7a9c071960 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7aac000f70 tx=0x7f7aac0079c0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.809+0000 7f7aa7400640 1 -- 172.21.15.86:0/1396964250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ab40e79f0 msgr2=0x7f7ab404e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.809+0000 7f7aa7400640 1 --2- 172.21.15.86:0/1396964250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ab40e79f0 0x7f7ab404e3a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8019150 tx=0x7f7aa801ad80 comp rx=0 tx=0).stop 2024-04-25T14:04:12.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.809+0000 7f7ac0c00640 1 -- 172.21.15.86:0/1396964250 reap_dead start 2024-04-25T14:04:12.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.809+0000 7f7aa7400640 1 -- 172.21.15.86:0/1396964250 shutdown_connections 2024-04-25T14:04:12.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.809+0000 7f7aa7400640 1 -- 172.21.15.86:0/1396964250 >> 172.21.15.86:0/1396964250 conn(0x7f7ab4054ed0 msgr2=0x7f7ab4006bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:12.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.809+0000 7f7aa7400640 1 -- 172.21.15.86:0/1396964250 shutdown_connections 2024-04-25T14:04:12.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.809+0000 7f7aa7400640 1 -- 172.21.15.86:0/1396964250 wait complete. 2024-04-25T14:04:12.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.809+0000 7fde72000640 1 -- 172.21.15.86:0/574889499 <== osd.6 v2:172.21.15.155:6801/2597410484 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fde54002de0 con 0x7fde5c078cc0 2024-04-25T14:04:12.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.809+0000 7fe7efc00640 1 -- 172.21.15.86:0/1578516589 --> [v2:172.21.15.86:6800/2257199129,v1:172.21.15.86:6801/2257199129] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe7c0002de0 con 0x7fe7c8078c50 2024-04-25T14:04:12.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.809+0000 7fde67400640 1 -- 172.21.15.86:0/574889499 >> [v2:172.21.15.155:6801/2597410484,v1:172.21.15.155:6803/2597410484] conn(0x7fde5c078cc0 msgr2=0x7fde5c07b150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.809+0000 7fde67400640 1 --2- 172.21.15.86:0/574889499 >> [v2:172.21.15.155:6801/2597410484,v1:172.21.15.155:6803/2597410484] conn(0x7fde5c078cc0 0x7fde5c07b150 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.813+0000 7fde67400640 1 -- 172.21.15.86:0/574889499 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fde5c06f500 msgr2=0x7fde5c071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.813+0000 7fde67400640 1 --2- 172.21.15.86:0/574889499 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fde5c06f500 0x7fde5c071a10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fde6c000c10 tx=0x7fde6c00b040 comp rx=0 tx=0).stop 2024-04-25T14:04:12.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.813+0000 7fde67400640 1 -- 172.21.15.86:0/574889499 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde7c0730b0 msgr2=0x7fde7c15d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.813+0000 7fde67400640 1 --2- 172.21.15.86:0/574889499 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde7c0730b0 0x7fde7c15d170 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fde6801f2a0 tx=0x7fde6801b7a0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.813+0000 7fde7a800640 1 -- 172.21.15.86:0/574889499 reap_dead start 2024-04-25T14:04:12.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.813+0000 7fe7e4c00640 1 -- 172.21.15.86:0/1578516589 <== osd.3 v2:172.21.15.86:6800/2257199129 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe7c0002de0 con 0x7fe7c8078c50 2024-04-25T14:04:12.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.813+0000 7fde67400640 1 -- 172.21.15.86:0/574889499 shutdown_connections 2024-04-25T14:04:12.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.813+0000 7fde67400640 1 -- 172.21.15.86:0/574889499 >> 172.21.15.86:0/574889499 conn(0x7fde7c0b4650 msgr2=0x7fde7c15c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:12.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.813+0000 7fde67400640 1 -- 172.21.15.86:0/574889499 shutdown_connections 2024-04-25T14:04:12.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.813+0000 7fde67400640 1 -- 172.21.15.86:0/574889499 wait complete. 2024-04-25T14:04:12.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.813+0000 7fe7efc00640 1 -- 172.21.15.86:0/1578516589 >> [v2:172.21.15.86:6800/2257199129,v1:172.21.15.86:6801/2257199129] conn(0x7fe7c8078c50 msgr2=0x7fe7c807b0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.813+0000 7fe7efc00640 1 --2- 172.21.15.86:0/1578516589 >> [v2:172.21.15.86:6800/2257199129,v1:172.21.15.86:6801/2257199129] conn(0x7fe7c8078c50 0x7fe7c807b0e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.813+0000 7fe7efc00640 1 -- 172.21.15.86:0/1578516589 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe7c806f480 msgr2=0x7fe7c8071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.813+0000 7fe7efc00640 1 --2- 172.21.15.86:0/1578516589 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe7c806f480 0x7fe7c8071990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe7e815c1a0 tx=0x7fe7d80010b0 comp rx=0 tx=0).stop 2024-04-25T14:04:12.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.813+0000 7fe7efc00640 1 -- 172.21.15.86:0/1578516589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7e806f9f0 msgr2=0x7fe7e815a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:12.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.813+0000 7fe7efc00640 1 --2- 172.21.15.86:0/1578516589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7e806f9f0 0x7fe7e815a640 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400c540 tx=0x7fe7d400ca00 comp rx=0 tx=0).stop 2024-04-25T14:04:12.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.813+0000 7fe7edc00640 1 -- 172.21.15.86:0/1578516589 reap_dead start 2024-04-25T14:04:12.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.813+0000 7fe7efc00640 1 -- 172.21.15.86:0/1578516589 shutdown_connections 2024-04-25T14:04:12.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.821+0000 7fe7efc00640 1 -- 172.21.15.86:0/1578516589 >> 172.21.15.86:0/1578516589 conn(0x7fe7e80b4650 msgr2=0x7fe7e80b2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:12.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.821+0000 7fe7efc00640 1 -- 172.21.15.86:0/1578516589 shutdown_connections 2024-04-25T14:04:12.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:12.821+0000 7fe7efc00640 1 -- 172.21.15.86:0/1578516589 wait complete. 2024-04-25T14:04:12.833 INFO:teuthology.orchestra.run.smithi086.stdout:55834574857 2024-04-25T14:04:12.833 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T14:04:12.834 INFO:teuthology.orchestra.run.smithi086.stdout:55834574857 2024-04-25T14:04:12.834 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T14:04:12.840 INFO:teuthology.orchestra.run.smithi086.stdout:55834574858 2024-04-25T14:04:12.840 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T14:04:12.843 INFO:teuthology.orchestra.run.smithi086.stdout:55834574856 2024-04-25T14:04:12.843 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T14:04:12.851 INFO:teuthology.orchestra.run.smithi086.stdout:55834574858 2024-04-25T14:04:12.851 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T14:04:12.855 INFO:teuthology.orchestra.run.smithi086.stdout:55834574856 2024-04-25T14:04:12.855 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T14:04:12.858 INFO:teuthology.orchestra.run.smithi086.stdout:55834574857 2024-04-25T14:04:12.858 INFO:teuthology.orchestra.run.smithi086.stdout:55834574857 2024-04-25T14:04:12.858 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T14:04:12.859 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T14:04:13.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.105+0000 7f964a400640 1 Processor -- start 2024-04-25T14:04:13.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.105+0000 7f964a400640 1 -- start start 2024-04-25T14:04:13.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.105+0000 7f964a400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96440d0540 0x7f9644163f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.105+0000 7f964a400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96441644a0 con 0x7f96440ba0c0 2024-04-25T14:04:13.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.105+0000 7f964a400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96440d4810 con 0x7f96440b6f90 2024-04-25T14:04:13.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.105+0000 7f964a400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f96440d4af0 con 0x7f96440d0540 2024-04-25T14:04:13.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.109+0000 7f9649000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96440d0540 0x7f9644163f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.109+0000 7f9649000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96440d0540 0x7f9644163f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:48326/0 (socket says 172.21.15.86:48326) 2024-04-25T14:04:13.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.109+0000 7f9649000640 1 -- 172.21.15.86:0/1706044541 learned_addr learned my addr 172.21.15.86:0/1706044541 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:13.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.109+0000 7f9649000640 1 -- 172.21.15.86:0/1706044541 >> v1:172.21.15.155:6789/0 conn(0x7f96440b6f90 legacy=0x7f96440b73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.109+0000 7f9649000640 1 -- 172.21.15.86:0/1706044541 >> v1:172.21.15.86:6789/0 conn(0x7f96440ba0c0 legacy=0x7f96440ba4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:13.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.109+0000 7f9649000640 1 -- 172.21.15.86:0/1706044541 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96440d4dd0 con 0x7f96440d0540 2024-04-25T14:04:13.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.109+0000 7f44f5400640 1 Processor -- start 2024-04-25T14:04:13.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.109+0000 7f44f5400640 1 -- start start 2024-04-25T14:04:13.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.113+0000 7f31eca00640 1 Processor -- start 2024-04-25T14:04:13.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.113+0000 7f31eca00640 1 -- start start 2024-04-25T14:04:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.113+0000 7f44f5400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f44f00d1200 0x7f44f00d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.113+0000 7f44f5400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44f00d51a0 con 0x7f44f00b4440 2024-04-25T14:04:13.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.113+0000 7f44f5400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44f00b4a30 con 0x7f44f00d3e70 2024-04-25T14:04:13.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.113+0000 7f44f5400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f44f00b4d10 con 0x7f44f00d1200 2024-04-25T14:04:13.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.113+0000 7f9649000640 1 --2- 172.21.15.86:0/1706044541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96440d0540 0x7f9644163f60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f96380098a0 tx=0x7f963801ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77ab33e9330f40fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.113+0000 7f44ee800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f44f00d1200 0x7f44f00d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.113+0000 7f44ee800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f44f00d1200 0x7f44f00d1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:54400/0 (socket says 172.21.15.86:54400) 2024-04-25T14:04:13.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.113+0000 7f44ee800640 1 -- 172.21.15.86:0/2734457453 learned_addr learned my addr 172.21.15.86:0/2734457453 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:13.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.117+0000 7f44ed400640 1 -- 172.21.15.86:0/2734457453 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1889103506 0 0) 0x7f44f00d51a0 con 0x7f44f00b4440 2024-04-25T14:04:13.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.117+0000 7f44ed400640 1 -- 172.21.15.86:0/2734457453 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44d4003530 con 0x7f44f00b4440 2024-04-25T14:04:13.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.117+0000 7f44ee800640 1 -- 172.21.15.86:0/2734457453 >> v1:172.21.15.155:6789/0 conn(0x7f44f00d3e70 legacy=0x7f44f00d4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:13.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.117+0000 7f44ee800640 1 -- 172.21.15.86:0/2734457453 >> v1:172.21.15.86:6789/0 conn(0x7f44f00b4440 legacy=0x7f44f00d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.117+0000 7f44ee800640 1 -- 172.21.15.86:0/2734457453 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f006d9c0 con 0x7f44f00d1200 2024-04-25T14:04:13.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.117+0000 7f9643400640 1 -- 172.21.15.86:0/1706044541 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96380b5070 con 0x7f96440d0540 2024-04-25T14:04:13.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.117+0000 7f9643400640 1 -- 172.21.15.86:0/1706044541 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96380a8a70 con 0x7f96440d0540 2024-04-25T14:04:13.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.117+0000 7f9643400640 1 -- 172.21.15.86:0/1706044541 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96380a8e40 con 0x7f96440d0540 2024-04-25T14:04:13.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.117+0000 7f9525800640 1 Processor -- start 2024-04-25T14:04:13.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.117+0000 7f31eca00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f31e80d2a50 0x7f31e80b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.117+0000 7f31eca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f31e80b3a90 0x7f31e80c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.117+0000 7f31eca00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31e80b1c20 con 0x7f31e815cc80 2024-04-25T14:04:13.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.117+0000 7f31eca00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f31e815d090 con 0x7f31e80b3a90 2024-04-25T14:04:13.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.117+0000 7f31eca00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f31e80b1940 con 0x7f31e80d2a50 2024-04-25T14:04:13.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.117+0000 7f964a400640 1 -- 172.21.15.86:0/1706044541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96440d0540 msgr2=0x7f9644163f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.117+0000 7f964a400640 1 --2- 172.21.15.86:0/1706044541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96440d0540 0x7f9644163f60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f96380098a0 tx=0x7f963801ed70 comp rx=0 tx=0).stop 2024-04-25T14:04:13.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.117+0000 7f964a400640 1 -- 172.21.15.86:0/1706044541 shutdown_connections 2024-04-25T14:04:13.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.117+0000 7f964a400640 1 --2- 172.21.15.86:0/1706044541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96440d0540 0x7f9644163f60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.117+0000 7f964a400640 1 -- 172.21.15.86:0/1706044541 >> 172.21.15.86:0/1706044541 conn(0x7f96440d60e0 msgr2=0x7f964406db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.117+0000 7f31e5400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f31e80d2a50 0x7f31e80b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.117+0000 7f31e5400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f31e80d2a50 0x7f31e80b21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:54412/0 (socket says 172.21.15.86:54412) 2024-04-25T14:04:13.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.117+0000 7f31e5400640 1 -- 172.21.15.86:0/525368937 learned_addr learned my addr 172.21.15.86:0/525368937 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:13.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.117+0000 7f31e6800640 1 --2- 172.21.15.86:0/525368937 >> v2:172.21.15.86:3300/0 conn(0x7f31e80b3a90 0x7f31e80c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.117+0000 7f9525800640 1 -- start start 2024-04-25T14:04:13.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7f9525800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9520074550 0x7f9520074960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7f9525800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95200cdb60 0x7f95200cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7f9525800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95200b27c0 con 0x7f952006d9d0 2024-04-25T14:04:13.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7f9525800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f95200740c0 con 0x7f9520074550 2024-04-25T14:04:13.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7f9525800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f95200ce4b0 con 0x7f95200cdb60 2024-04-25T14:04:13.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7f31e5400640 1 -- 172.21.15.86:0/525368937 >> v2:172.21.15.86:3300/0 conn(0x7f31e80b3a90 msgr2=0x7f31e80c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:13.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7f951ea00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f952006d9d0 0x7f952015bcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:50840/0 (socket says 172.21.15.86:50840) 2024-04-25T14:04:13.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7f951ea00640 1 -- 172.21.15.86:0/3209458391 learned_addr learned my addr 172.21.15.86:0/3209458391 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:13.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7f951e000640 1 --2- 172.21.15.86:0/3209458391 >> v2:172.21.15.86:3300/0 conn(0x7f9520074550 0x7f9520074960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7f951f400640 1 --2- 172.21.15.86:0/3209458391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95200cdb60 0x7f95200cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7f964a400640 1 -- 172.21.15.86:0/1706044541 shutdown_connections 2024-04-25T14:04:13.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7f951d600640 1 -- 172.21.15.86:0/3209458391 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2731860380 0 0) 0x7f95200b27c0 con 0x7f952006d9d0 2024-04-25T14:04:13.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7f951d600640 1 -- 172.21.15.86:0/3209458391 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9504003530 con 0x7f952006d9d0 2024-04-25T14:04:13.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7f289ca00640 1 Processor -- start 2024-04-25T14:04:13.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7f951e000640 1 -- 172.21.15.86:0/3209458391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95200cdb60 msgr2=0x7f95200cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:13.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7f951e000640 1 --2- 172.21.15.86:0/3209458391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95200cdb60 0x7f95200cdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7f951e000640 1 -- 172.21.15.86:0/3209458391 >> v1:172.21.15.155:6789/0 conn(0x7f952006d9d0 legacy=0x7f952015bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7f951e000640 1 -- 172.21.15.86:0/3209458391 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95200b2aa0 con 0x7f9520074550 2024-04-25T14:04:13.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7f289ca00640 1 -- start start 2024-04-25T14:04:13.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7fe090e00640 1 Processor -- start 2024-04-25T14:04:13.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7f964a400640 1 -- 172.21.15.86:0/1706044541 wait complete. 2024-04-25T14:04:13.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7f44ee800640 1 --2- 172.21.15.86:0/2734457453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44f00d1200 0x7f44f00d1dd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009870 tx=0x7f44e001c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a03451aa14ec4aa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7fe090e00640 1 -- start start 2024-04-25T14:04:13.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7f44ed400640 1 -- 172.21.15.86:0/2734457453 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e00b0070 con 0x7f44f00d1200 2024-04-25T14:04:13.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7f44ed400640 1 -- 172.21.15.86:0/2734457453 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44e00ab600 con 0x7f44f00d1200 2024-04-25T14:04:13.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7f44ed400640 1 -- 172.21.15.86:0/2734457453 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e00ab9d0 con 0x7f44f00d1200 2024-04-25T14:04:13.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7f31e5400640 1 --2- 172.21.15.86:0/525368937 >> v2:172.21.15.86:3300/0 conn(0x7f31e80b3a90 0x7f31e80c7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7f31e5400640 1 -- 172.21.15.86:0/525368937 >> v1:172.21.15.86:6789/0 conn(0x7f31e815cc80 legacy=0x7f31e80d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7f31e5400640 1 -- 172.21.15.86:0/525368937 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e80d2f80 con 0x7f31e80d2a50 2024-04-25T14:04:13.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.121+0000 7f31e6800640 1 --2- 172.21.15.86:0/525368937 >> v2:172.21.15.86:3300/0 conn(0x7f31e80b3a90 0x7f31e80c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:13.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f289ca00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28980bb140 0x7f28980d09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f289ca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f289815f160 0x7f289815f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f289ca00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28980b8ad0 con 0x7f28980d1de0 2024-04-25T14:04:13.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f289ca00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f28980d11f0 con 0x7f289815f160 2024-04-25T14:04:13.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f289ca00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f28980d14d0 con 0x7f28980bb140 2024-04-25T14:04:13.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f288fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f28980d1de0 0x7f28980d21f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33448/0 (socket says 172.21.15.86:33448) 2024-04-25T14:04:13.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f288fe00640 1 -- 172.21.15.86:0/1321188820 learned_addr learned my addr 172.21.15.86:0/1321188820 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:13.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f964a400640 1 Processor -- start 2024-04-25T14:04:13.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f964a400640 1 -- start start 2024-04-25T14:04:13.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f44f5400640 1 -- 172.21.15.86:0/2734457453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44f00d1200 msgr2=0x7f44f00d1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f44f5400640 1 --2- 172.21.15.86:0/2734457453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44f00d1200 0x7f44f00d1dd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009870 tx=0x7f44e001c4d0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7fe090e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe08c0b4440 0x7fe08c163920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7fe090e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe08c0b4a30 con 0x7fe08c0d3e70 2024-04-25T14:04:13.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7fe090e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe08c06d9c0 con 0x7fe08c0d1200 2024-04-25T14:04:13.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7fe090e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe08c0b4d10 con 0x7fe08c0b4440 2024-04-25T14:04:13.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f44f5400640 1 -- 172.21.15.86:0/2734457453 shutdown_connections 2024-04-25T14:04:13.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f44f5400640 1 --2- 172.21.15.86:0/2734457453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44f00d1200 0x7f44f00d1dd0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f44f5400640 1 -- 172.21.15.86:0/2734457453 >> 172.21.15.86:0/2734457453 conn(0x7f44f00b72d0 msgr2=0x7f44f00b76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f44f5400640 1 -- 172.21.15.86:0/2734457453 shutdown_connections 2024-04-25T14:04:13.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f44f5400640 1 -- 172.21.15.86:0/2734457453 wait complete. 2024-04-25T14:04:13.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f951e000640 1 --2- 172.21.15.86:0/3209458391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9520074550 0x7f9520074960 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f951800aea0 tx=0x7f951801da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee8cf18514a033 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f951d600640 1 -- 172.21.15.86:0/3209458391 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95180a67e0 con 0x7f9520074550 2024-04-25T14:04:13.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f951d600640 1 -- 172.21.15.86:0/3209458391 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95180a6980 con 0x7f9520074550 2024-04-25T14:04:13.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f951d600640 1 -- 172.21.15.86:0/3209458391 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95180a6d30 con 0x7f9520074550 2024-04-25T14:04:13.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f31e5400640 1 --2- 172.21.15.86:0/525368937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31e80d2a50 0x7f31e80b21f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f31e806f190 tx=0x7f31d401c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3674a0816bd96517 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7fe08a800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe08c0d3e70 0x7fe08c163e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33462/0 (socket says 172.21.15.86:33462) 2024-04-25T14:04:13.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7fe08a800640 1 -- 172.21.15.86:0/2836801258 learned_addr learned my addr 172.21.15.86:0/2836801258 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:13.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f2897200640 1 --2- 172.21.15.86:0/1321188820 >> v2:172.21.15.86:3300/0 conn(0x7f289815f160 0x7f289815f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f2895e00640 1 -- 172.21.15.86:0/1321188820 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1496111523 0 0) 0x7f28980b8ad0 con 0x7f28980d1de0 2024-04-25T14:04:13.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f2895e00640 1 -- 172.21.15.86:0/1321188820 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2878003530 con 0x7f28980d1de0 2024-04-25T14:04:13.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f2896800640 1 --2- 172.21.15.86:0/1321188820 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28980bb140 0x7f28980d09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f2897200640 1 -- 172.21.15.86:0/1321188820 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28980bb140 msgr2=0x7f28980d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:13.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f2897200640 1 --2- 172.21.15.86:0/1321188820 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28980bb140 0x7f28980d09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f2897200640 1 -- 172.21.15.86:0/1321188820 >> v1:172.21.15.86:6789/0 conn(0x7f28980d1de0 legacy=0x7f28980d21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f2897200640 1 -- 172.21.15.86:0/1321188820 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28980d17b0 con 0x7f289815f160 2024-04-25T14:04:13.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f31e4a00640 1 -- 172.21.15.86:0/525368937 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d40b0070 con 0x7f31e80d2a50 2024-04-25T14:04:13.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f31e4a00640 1 -- 172.21.15.86:0/525368937 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31d40ab400 con 0x7f31e80d2a50 2024-04-25T14:04:13.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7f31e4a00640 1 -- 172.21.15.86:0/525368937 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d40ab7d0 con 0x7f31e80d2a50 2024-04-25T14:04:13.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f9525800640 1 -- 172.21.15.86:0/3209458391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9520074550 msgr2=0x7f9520074960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f9525800640 1 --2- 172.21.15.86:0/3209458391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9520074550 0x7f9520074960 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f951800aea0 tx=0x7f951801da30 comp rx=0 tx=0).stop 2024-04-25T14:04:13.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7fe089400640 1 --2- 172.21.15.86:0/2836801258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe08c0b4440 0x7fe08c163920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7fe088a00640 1 -- 172.21.15.86:0/2836801258 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3341257346 0 0) 0x7fe08c0b4a30 con 0x7fe08c0d3e70 2024-04-25T14:04:13.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7fe088a00640 1 -- 172.21.15.86:0/2836801258 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe070003530 con 0x7fe08c0d3e70 2024-04-25T14:04:13.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7fe088a00640 1 -- 172.21.15.86:0/2836801258 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3687756340 0 0) 0x7fe070003530 con 0x7fe08c0d3e70 2024-04-25T14:04:13.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7fe088a00640 1 -- 172.21.15.86:0/2836801258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe08c0b4440 msgr2=0x7fe08c163920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:13.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7fe088a00640 1 --2- 172.21.15.86:0/2836801258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe08c0b4440 0x7fe08c163920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7fe088a00640 1 -- 172.21.15.86:0/2836801258 >> v1:172.21.15.155:6789/0 conn(0x7fe08c0d1200 legacy=0x7fe08c0d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:13.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7fe088a00640 1 -- 172.21.15.86:0/2836801258 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe08c06dca0 con 0x7fe08c0d3e70 2024-04-25T14:04:13.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7fe088a00640 1 -- 172.21.15.86:0/2836801258 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fe084001a00 con 0x7fe08c0d3e70 2024-04-25T14:04:13.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7fe088a00640 1 -- 172.21.15.86:0/2836801258 >> v1:172.21.15.86:6789/0 conn(0x7fe08c0d3e70 legacy=0x7fe08c163e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7fe088a00640 1 --2- 172.21.15.86:0/2836801258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe070003a60 0x7fe070003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7fe088a00640 1 --2- 172.21.15.86:0/2836801258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe070004810 0x7fe070004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7fe088a00640 1 --2- 172.21.15.86:0/2836801258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe070005340 0x7fe07000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7fe088a00640 1 -- 172.21.15.86:0/2836801258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe08c0b4d10 con 0x7fe070004810 2024-04-25T14:04:13.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7fe088a00640 1 -- 172.21.15.86:0/2836801258 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe07000c260 con 0x7fe070003a60 2024-04-25T14:04:13.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.125+0000 7fe088a00640 1 -- 172.21.15.86:0/2836801258 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe07000c460 con 0x7fe070005340 2024-04-25T14:04:13.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f9525800640 1 -- 172.21.15.86:0/3209458391 shutdown_connections 2024-04-25T14:04:13.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f9525800640 1 --2- 172.21.15.86:0/3209458391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95200cdb60 0x7f95200cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f9525800640 1 --2- 172.21.15.86:0/3209458391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9520074550 0x7f9520074960 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f9525800640 1 -- 172.21.15.86:0/3209458391 >> 172.21.15.86:0/3209458391 conn(0x7f95200b6130 msgr2=0x7f95200b6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7fe08a800640 1 --2- 172.21.15.86:0/2836801258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe070005340 0x7fe07000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f964a400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96440b6f90 0x7f9644162620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f964a400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96440ba0c0 0x7f96440ce780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f964a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96440d0540 0x7f96440cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f964a400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f96440b9b30 con 0x7f96440d0540 2024-04-25T14:04:13.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f964a400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f96440d0950 con 0x7f96440ba0c0 2024-04-25T14:04:13.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f964a400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f96440babe0 con 0x7f96440b6f90 2024-04-25T14:04:13.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f31eca00640 1 -- 172.21.15.86:0/525368937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31e80d2a50 msgr2=0x7f31e80b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f31eca00640 1 --2- 172.21.15.86:0/525368937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31e80d2a50 0x7f31e80b21f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f31e806f190 tx=0x7f31d401c520 comp rx=0 tx=0).stop 2024-04-25T14:04:13.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f9525800640 1 -- 172.21.15.86:0/3209458391 shutdown_connections 2024-04-25T14:04:13.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7fe08a800640 1 -- 172.21.15.86:0/2836801258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe070003a60 msgr2=0x7fe070003eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T14:04:13.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7fe08a800640 1 --2- 172.21.15.86:0/2836801258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe070003a60 0x7fe070003eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7fe08a800640 1 -- 172.21.15.86:0/2836801258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe070004810 msgr2=0x7fe070004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:13.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7fe08a800640 1 --2- 172.21.15.86:0/2836801258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe070004810 0x7fe070004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7fe08a800640 1 -- 172.21.15.86:0/2836801258 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe070003530 con 0x7fe070005340 2024-04-25T14:04:13.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f44f5400640 1 Processor -- start 2024-04-25T14:04:13.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f31eca00640 1 -- 172.21.15.86:0/525368937 shutdown_connections 2024-04-25T14:04:13.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f31eca00640 1 --2- 172.21.15.86:0/525368937 >> v2:172.21.15.86:3300/0 conn(0x7f31e80b3a90 0x7f31e80c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f9649000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96440b6f90 0x7f9644162620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f31eca00640 1 --2- 172.21.15.86:0/525368937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31e80d2a50 0x7f31e80b21f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f44f5400640 1 -- start start 2024-04-25T14:04:13.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f31eca00640 1 -- 172.21.15.86:0/525368937 >> 172.21.15.86:0/525368937 conn(0x7f31e809ba30 msgr2=0x7f31e80b7420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f31eca00640 1 -- 172.21.15.86:0/525368937 shutdown_connections 2024-04-25T14:04:13.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f31eca00640 1 -- 172.21.15.86:0/525368937 wait complete. 2024-04-25T14:04:13.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f2897200640 1 --2- 172.21.15.86:0/1321188820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f289815f160 0x7f289815f570 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2890009d90 tx=0x7f289001f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed19f9d3545d36fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f9525800640 1 -- 172.21.15.86:0/3209458391 wait complete. 2024-04-25T14:04:13.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f2895e00640 1 -- 172.21.15.86:0/1321188820 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28900b4070 con 0x7f289815f160 2024-04-25T14:04:13.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f2895e00640 1 -- 172.21.15.86:0/1321188820 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28900af600 con 0x7f289815f160 2024-04-25T14:04:13.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f2895e00640 1 -- 172.21.15.86:0/1321188820 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28900af9d0 con 0x7f289815f160 2024-04-25T14:04:13.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f9649000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96440b6f90 0x7f9644162620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:48358/0 (socket says 172.21.15.86:48358) 2024-04-25T14:04:13.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f9649000640 1 -- 172.21.15.86:0/1176285558 learned_addr learned my addr 172.21.15.86:0/1176285558 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:13.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f9643e00640 1 --2- 172.21.15.86:0/1176285558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96440ba0c0 0x7f96440ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f9649000640 1 -- 172.21.15.86:0/1176285558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96440ba0c0 msgr2=0x7f96440ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f9649000640 1 --2- 172.21.15.86:0/1176285558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96440ba0c0 0x7f96440ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f9649000640 1 -- 172.21.15.86:0/1176285558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96440d0540 msgr2=0x7f96440cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f9649000640 1 --2- 172.21.15.86:0/1176285558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96440d0540 0x7f96440cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f9649000640 1 -- 172.21.15.86:0/1176285558 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9644070200 con 0x7f96440b6f90 2024-04-25T14:04:13.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.129+0000 7f9649000640 1 --2- 172.21.15.86:0/1176285558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96440b6f90 0x7f9644162620 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f96380098a0 tx=0x7f963801bb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f9640c00640 1 -- 172.21.15.86:0/1176285558 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96380b5040 con 0x7f96440b6f90 2024-04-25T14:04:13.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f9640c00640 1 -- 172.21.15.86:0/1176285558 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96380c1070 con 0x7f96440b6f90 2024-04-25T14:04:13.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f9640c00640 1 -- 172.21.15.86:0/1176285558 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96380bb660 con 0x7f96440b6f90 2024-04-25T14:04:13.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f44f5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44f00b4440 0x7f44f0157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f44f5400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44f00d1200 0x7f44f0148de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f289ca00640 1 -- 172.21.15.86:0/1321188820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f289815f160 msgr2=0x7f289815f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f289ca00640 1 --2- 172.21.15.86:0/1321188820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f289815f160 0x7f289815f570 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2890009d90 tx=0x7f289001f150 comp rx=0 tx=0).stop 2024-04-25T14:04:13.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f964a400640 1 -- 172.21.15.86:0/1176285558 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96440704e0 con 0x7f96440b6f90 2024-04-25T14:04:13.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f964a400640 1 -- 172.21.15.86:0/1176285558 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9644070aa0 con 0x7f96440b6f90 2024-04-25T14:04:13.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f44ee800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44f00b4440 0x7f44f0157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7fe08a800640 1 --2- 172.21.15.86:0/2836801258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe070005340 0x7fe07000b9d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe08c0d3210 tx=0x7fe08401d2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c84d02e94ec476f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f934ce00640 1 Processor -- start 2024-04-25T14:04:13.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f44ee800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44f00b4440 0x7f44f0157d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38074/0 (socket says 172.21.15.86:38074) 2024-04-25T14:04:13.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f31eca00640 1 Processor -- start 2024-04-25T14:04:13.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f289ca00640 1 -- 172.21.15.86:0/1321188820 shutdown_connections 2024-04-25T14:04:13.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f9525800640 1 Processor -- start 2024-04-25T14:04:13.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7fe088a00640 1 -- 172.21.15.86:0/2836801258 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08400a480 con 0x7fe070005340 2024-04-25T14:04:13.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f289ca00640 1 --2- 172.21.15.86:0/1321188820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f289815f160 0x7f289815f570 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f289ca00640 1 --2- 172.21.15.86:0/1321188820 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28980bb140 0x7f28980d09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f289ca00640 1 -- 172.21.15.86:0/1321188820 >> 172.21.15.86:0/1321188820 conn(0x7f289806d160 msgr2=0x7f28980d5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7fe088a00640 1 -- 172.21.15.86:0/2836801258 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0840ad650 con 0x7fe070005340 2024-04-25T14:04:13.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f9525800640 1 -- start start 2024-04-25T14:04:13.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f31eca00640 1 -- start start 2024-04-25T14:04:13.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f289ca00640 1 -- 172.21.15.86:0/1321188820 shutdown_connections 2024-04-25T14:04:13.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f44ee800640 1 -- 172.21.15.86:0/577765441 learned_addr learned my addr 172.21.15.86:0/577765441 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:13.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f44f5400640 1 --2- 172.21.15.86:0/577765441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44f00d3e70 0x7f44f00ced70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7fe088a00640 1 -- 172.21.15.86:0/2836801258 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0840ada40 con 0x7fe070005340 2024-04-25T14:04:13.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f44f5400640 1 -- 172.21.15.86:0/577765441 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44f016e870 con 0x7f44f00b4440 2024-04-25T14:04:13.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7fe090e00640 1 -- 172.21.15.86:0/2836801258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe070005340 msgr2=0x7fe07000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7fe090e00640 1 --2- 172.21.15.86:0/2836801258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe070005340 0x7fe07000b9d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe08c0d3210 tx=0x7fe08401d2d0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7fe08a800640 1 -- 172.21.15.86:0/2836801258 reap_dead start 2024-04-25T14:04:13.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f289ca00640 1 -- 172.21.15.86:0/1321188820 wait complete. 2024-04-25T14:04:13.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f934ce00640 1 -- start start 2024-04-25T14:04:13.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7fe090e00640 1 -- 172.21.15.86:0/2836801258 shutdown_connections 2024-04-25T14:04:13.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7fe090e00640 1 -- 172.21.15.86:0/2836801258 >> 172.21.15.86:0/2836801258 conn(0x7fe08c0b72d0 msgr2=0x7fe08c0b76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f44ef200640 1 --2- 172.21.15.86:0/577765441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44f00d3e70 0x7f44f00ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f44ede00640 1 --2- 172.21.15.86:0/577765441 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44f00d1200 0x7f44f0148de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f44f5400640 1 -- 172.21.15.86:0/577765441 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f44f016e6f0 con 0x7f44f00d3e70 2024-04-25T14:04:13.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.133+0000 7f6911000640 1 Processor -- start 2024-04-25T14:04:13.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7fe090e00640 1 -- 172.21.15.86:0/2836801258 shutdown_connections 2024-04-25T14:04:13.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7fe090e00640 1 -- 172.21.15.86:0/2836801258 wait complete. 2024-04-25T14:04:13.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f44f5400640 1 -- 172.21.15.86:0/577765441 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f44f016e9f0 con 0x7f44f00d1200 2024-04-25T14:04:13.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f6911000640 1 -- start start 2024-04-25T14:04:13.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f964a400640 1 -- 172.21.15.86:0/1176285558 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9644076b80 con 0x7f96440b6f90 2024-04-25T14:04:13.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f9525800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f952006d9d0 0x7f9520155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f9525800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9520074550 0x7f9520099d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f9525800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95200cdb60 0x7f9520153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f9525800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95200d16b0 con 0x7f95200cdb60 2024-04-25T14:04:13.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f9525800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f95200d1530 con 0x7f952006d9d0 2024-04-25T14:04:13.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f9525800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f95200d1830 con 0x7f9520074550 2024-04-25T14:04:13.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f31eca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31e80b3a90 0x7f31e8147580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f951e000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9520074550 0x7f9520099d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f951e000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9520074550 0x7f9520099d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:48378/0 (socket says 172.21.15.86:48378) 2024-04-25T14:04:13.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f951e000640 1 -- 172.21.15.86:0/3394498723 learned_addr learned my addr 172.21.15.86:0/3394498723 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:13.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f934ce00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f934806d9d0 0x7f934815bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f934ce00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9348074960 0x7f93480cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f934ce00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9348074d70 con 0x7f9348074550 2024-04-25T14:04:13.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f951f400640 1 --2- 172.21.15.86:0/3394498723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95200cdb60 0x7f9520153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f31eca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31e80d2a50 0x7f31e8147ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f934ce00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f93480740c0 con 0x7f9348074960 2024-04-25T14:04:13.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f934ce00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f93480b27c0 con 0x7f934806d9d0 2024-04-25T14:04:13.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f9346800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9348074960 0x7f93480cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f9346800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9348074960 0x7f93480cfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:54460/0 (socket says 172.21.15.86:54460) 2024-04-25T14:04:13.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f9346800640 1 -- 172.21.15.86:0/2783121928 learned_addr learned my addr 172.21.15.86:0/2783121928 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:13.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f31eca00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31e815cc80 0x7f31e8150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f44ede00640 1 -- 172.21.15.86:0/577765441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44f00d3e70 msgr2=0x7f44f00ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f31e5400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31e80d2a50 0x7f31e8147ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f31e5400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31e80d2a50 0x7f31e8147ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:54470/0 (socket says 172.21.15.86:54470) 2024-04-25T14:04:13.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f31e5400640 1 -- 172.21.15.86:0/1406367161 learned_addr learned my addr 172.21.15.86:0/1406367161 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:13.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f31eca00640 1 -- 172.21.15.86:0/1406367161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f31e806ca80 con 0x7f31e80b3a90 2024-04-25T14:04:13.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f44ede00640 1 --2- 172.21.15.86:0/577765441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44f00d3e70 0x7f44f00ced70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f44ede00640 1 -- 172.21.15.86:0/577765441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44f00b4440 msgr2=0x7f44f0157d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f44ede00640 1 --2- 172.21.15.86:0/577765441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44f00b4440 0x7f44f0157d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f44ede00640 1 -- 172.21.15.86:0/577765441 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f00cc720 con 0x7f44f00d1200 2024-04-25T14:04:13.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f44ee800640 1 --2- 172.21.15.86:0/577765441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44f00b4440 0x7f44f0157d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:13.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f951ea00640 1 --2- 172.21.15.86:0/3394498723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f952006d9d0 0x7f9520155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f951f400640 1 -- 172.21.15.86:0/3394498723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9520074550 msgr2=0x7f9520099d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f951f400640 1 --2- 172.21.15.86:0/3394498723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9520074550 0x7f9520099d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f951f400640 1 -- 172.21.15.86:0/3394498723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f952006d9d0 msgr2=0x7f9520155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f951f400640 1 --2- 172.21.15.86:0/3394498723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f952006d9d0 0x7f9520155840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f951f400640 1 -- 172.21.15.86:0/3394498723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f952015b270 con 0x7f95200cdb60 2024-04-25T14:04:13.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f951ea00640 1 --2- 172.21.15.86:0/3394498723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f952006d9d0 0x7f9520155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:13.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f31eca00640 1 -- 172.21.15.86:0/1406367161 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f31e806c900 con 0x7f31e80d2a50 2024-04-25T14:04:13.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f31eca00640 1 -- 172.21.15.86:0/1406367161 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f31e806cc00 con 0x7f31e815cc80 2024-04-25T14:04:13.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f31e5e00640 1 --2- 172.21.15.86:0/1406367161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31e80b3a90 0x7f31e8147580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f31e6800640 1 --2- 172.21.15.86:0/1406367161 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31e815cc80 0x7f31e8150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f31e5400640 1 -- 172.21.15.86:0/1406367161 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31e815cc80 msgr2=0x7f31e8150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f31e5400640 1 --2- 172.21.15.86:0/1406367161 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31e815cc80 0x7f31e8150fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f31e5400640 1 -- 172.21.15.86:0/1406367161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31e80b3a90 msgr2=0x7f31e8147580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f31e5400640 1 --2- 172.21.15.86:0/1406367161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31e80b3a90 0x7f31e8147580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f31e5400640 1 -- 172.21.15.86:0/1406367161 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e81496a0 con 0x7f31e80d2a50 2024-04-25T14:04:13.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f9640c00640 1 -- 172.21.15.86:0/1176285558 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f96380bb800 con 0x7f96440b6f90 2024-04-25T14:04:13.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f6911000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f690404fe70 0x7f6904006860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f289ca00640 1 Processor -- start 2024-04-25T14:04:13.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f289ca00640 1 -- start start 2024-04-25T14:04:13.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f31e5400640 1 --2- 172.21.15.86:0/1406367161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31e80d2a50 0x7f31e8147ac0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f31d401ca00 tx=0x7f31d40192c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f6911000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6904006da0 0x7f69040071b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f44ede00640 1 --2- 172.21.15.86:0/577765441 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44f00d1200 0x7f44f0148de0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f44dc00c390 tx=0x7f44dc00c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f951f400640 1 --2- 172.21.15.86:0/3394498723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95200cdb60 0x7f9520153570 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f952006d940 tx=0x7f951000d910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f690aa00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f690404fe70 0x7f6904006860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f690aa00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f690404fe70 0x7f6904006860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:54482/0 (socket says 172.21.15.86:54482) 2024-04-25T14:04:13.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f9346800640 1 -- 172.21.15.86:0/2783121928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f934806d9d0 msgr2=0x7f934815bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:13.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f9346800640 1 --2- 172.21.15.86:0/2783121928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f934806d9d0 0x7f934815bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f9346800640 1 -- 172.21.15.86:0/2783121928 >> v1:172.21.15.155:6789/0 conn(0x7f9348074550 legacy=0x7f93480cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:13.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.137+0000 7f9346800640 1 -- 172.21.15.86:0/2783121928 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93480b2aa0 con 0x7f9348074960 2024-04-25T14:04:13.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f690b400640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f690404f350 0x7f690404f760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:50874/0 (socket says 172.21.15.86:50874) 2024-04-25T14:04:13.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f690aa00640 1 -- 172.21.15.86:0/3926230271 learned_addr learned my addr 172.21.15.86:0/3926230271 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:13.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f31de000640 1 -- 172.21.15.86:0/1406367161 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d40b0040 con 0x7f31e80d2a50 2024-04-25T14:04:13.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f31de000640 1 -- 172.21.15.86:0/1406367161 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31d40b9070 con 0x7f31e80d2a50 2024-04-25T14:04:13.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f31de000640 1 -- 172.21.15.86:0/1406367161 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d40b48b0 con 0x7f31e80d2a50 2024-04-25T14:04:13.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f31eca00640 1 -- 172.21.15.86:0/1406367161 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e8154670 con 0x7f31e80d2a50 2024-04-25T14:04:13.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f31eca00640 1 -- 172.21.15.86:0/1406367161 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e8150500 con 0x7f31e80d2a50 2024-04-25T14:04:13.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f6911000640 1 -- 172.21.15.86:0/3926230271 --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6904034460 con 0x7f690404f350 2024-04-25T14:04:13.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f6911000640 1 -- 172.21.15.86:0/3926230271 --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f69040060a0 con 0x7f6904006da0 2024-04-25T14:04:13.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f6911000640 1 -- 172.21.15.86:0/3926230271 --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f69040e0620 con 0x7f690404fe70 2024-04-25T14:04:13.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f9516a00640 1 -- 172.21.15.86:0/3394498723 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9510004020 con 0x7f95200cdb60 2024-04-25T14:04:13.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f9516a00640 1 -- 172.21.15.86:0/3394498723 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95100041e0 con 0x7f95200cdb60 2024-04-25T14:04:13.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f9516a00640 1 -- 172.21.15.86:0/3394498723 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95100043d0 con 0x7f95200cdb60 2024-04-25T14:04:13.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f690be00640 1 --2- 172.21.15.86:0/3926230271 >> v2:172.21.15.86:3300/0 conn(0x7f6904006da0 0x7f69040071b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f9640c00640 1 --2- 172.21.15.86:0/1176285558 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f962406f500 0x7f9624071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f9525800640 1 -- 172.21.15.86:0/3394498723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f952015b550 con 0x7f95200cdb60 2024-04-25T14:04:13.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f9640c00640 1 -- 172.21.15.86:0/1176285558 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f963812d950 con 0x7f96440b6f90 2024-04-25T14:04:13.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f690aa00640 1 -- 172.21.15.86:0/3926230271 >> v2:172.21.15.86:3300/0 conn(0x7f6904006da0 msgr2=0x7f69040071b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:13.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f690aa00640 1 --2- 172.21.15.86:0/3926230271 >> v2:172.21.15.86:3300/0 conn(0x7f6904006da0 0x7f69040071b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f690aa00640 1 -- 172.21.15.86:0/3926230271 >> v1:172.21.15.155:6789/0 conn(0x7f690404f350 legacy=0x7f690404f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f690a000640 1 -- 172.21.15.86:0/3926230271 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3496823747 0 0) 0x7f6904034460 con 0x7f690404f350 2024-04-25T14:04:13.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f9525800640 1 -- 172.21.15.86:0/3394498723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f952015b830 con 0x7f95200cdb60 2024-04-25T14:04:13.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f690aa00640 1 -- 172.21.15.86:0/3926230271 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69040e0900 con 0x7f690404fe70 2024-04-25T14:04:13.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f690be00640 1 --2- 172.21.15.86:0/3926230271 >> v2:172.21.15.86:3300/0 conn(0x7f6904006da0 0x7f69040071b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:13.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f9525800640 1 -- 172.21.15.86:0/3394498723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f952009f4b0 con 0x7f95200cdb60 2024-04-25T14:04:13.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f9643e00640 1 --2- 172.21.15.86:0/1176285558 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f962406f500 0x7f9624071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f289ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28980bb140 0x7f2898160060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7fe090e00640 1 Processor -- start 2024-04-25T14:04:13.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7fe090e00640 1 -- start start 2024-04-25T14:04:13.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f289ca00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28980d1de0 0x7f28980ca250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.141+0000 7f44e6a00640 1 -- 172.21.15.86:0/577765441 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc017070 con 0x7f44f00d1200 2024-04-25T14:04:13.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f9643e00640 1 --2- 172.21.15.86:0/1176285558 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f962406f500 0x7f9624071a10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f96440d3c10 tx=0x7f9630009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f31eca00640 1 -- 172.21.15.86:0/1406367161 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31c00052a0 con 0x7f31e80d2a50 2024-04-25T14:04:13.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f289ca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f289815f160 0x7f2898154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f44f5400640 1 -- 172.21.15.86:0/577765441 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44f00cd650 con 0x7f44f00d1200 2024-04-25T14:04:13.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f44f5400640 1 -- 172.21.15.86:0/577765441 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f44f00cccc0 con 0x7f44f00d1200 2024-04-25T14:04:13.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f44e6a00640 1 -- 172.21.15.86:0/577765441 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44dc003600 con 0x7f44f00d1200 2024-04-25T14:04:13.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f44e6a00640 1 -- 172.21.15.86:0/577765441 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc017070 con 0x7f44f00d1200 2024-04-25T14:04:13.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f289ca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f289815fab0 con 0x7f28980bb140 2024-04-25T14:04:13.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f289ca00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f28980d0f00 con 0x7f289815f160 2024-04-25T14:04:13.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f289ca00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f289815fc30 con 0x7f28980d1de0 2024-04-25T14:04:13.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f31de000640 1 -- 172.21.15.86:0/1406367161 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f31d40b4a50 con 0x7f31e80d2a50 2024-04-25T14:04:13.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f690aa00640 1 --2- 172.21.15.86:0/3926230271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f690404fe70 0x7f6904006860 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f69040e32b0 tx=0x7f690001d4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=352b5c288eb2170b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7fe090e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe070005340 0x7fe08c155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7fe090e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe08c157480 0x7fe08c159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7fe090e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe08c1603a0 0x7fe08c0cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7fe090e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe08c163920 con 0x7fe070005340 2024-04-25T14:04:13.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7fe090e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe08c0d2730 con 0x7fe08c1603a0 2024-04-25T14:04:13.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7fe090e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe08c163aa0 con 0x7fe08c157480 2024-04-25T14:04:13.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f9346800640 1 --2- 172.21.15.86:0/2783121928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9348074960 0x7f93480cfec0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9334009870 tx=0x7f933401c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=787a7461fcf58e27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f9344a00640 1 -- 172.21.15.86:0/2783121928 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93340b0070 con 0x7f9348074960 2024-04-25T14:04:13.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f9344a00640 1 -- 172.21.15.86:0/2783121928 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93340ab600 con 0x7f9348074960 2024-04-25T14:04:13.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f9344a00640 1 -- 172.21.15.86:0/2783121928 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93340ab9d0 con 0x7f9348074960 2024-04-25T14:04:13.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f934ce00640 1 -- 172.21.15.86:0/2783121928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9348074960 msgr2=0x7f93480cfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f934ce00640 1 --2- 172.21.15.86:0/2783121928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9348074960 0x7f93480cfec0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9334009870 tx=0x7f933401c520 comp rx=0 tx=0).stop 2024-04-25T14:04:13.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f9640c00640 1 -- 172.21.15.86:0/1176285558 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f96380188d0 con 0x7f96440b6f90 2024-04-25T14:04:13.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f44f5400640 1 -- 172.21.15.86:0/577765441 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44f006ef80 con 0x7f44f00d1200 2024-04-25T14:04:13.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f31de000640 1 --2- 172.21.15.86:0/1406367161 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f31c806f4d0 0x7f31c80719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f31de000640 1 -- 172.21.15.86:0/1406367161 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f31d41290f0 con 0x7f31e80d2a50 2024-04-25T14:04:13.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f44e6a00640 1 -- 172.21.15.86:0/577765441 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f44dc01a070 con 0x7f44f00d1200 2024-04-25T14:04:13.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f2896800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28980bb140 0x7f2898160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f2896800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28980bb140 0x7f2898160060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38100/0 (socket says 172.21.15.86:38100) 2024-04-25T14:04:13.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f2896800640 1 -- 172.21.15.86:0/1085263331 learned_addr learned my addr 172.21.15.86:0/1085263331 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:13.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f2896800640 1 -- 172.21.15.86:0/1085263331 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28980d1de0 msgr2=0x7f28980ca250 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T14:04:13.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f2896800640 1 --2- 172.21.15.86:0/1085263331 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28980d1de0 0x7f28980ca250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f2896800640 1 -- 172.21.15.86:0/1085263331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f289815f160 msgr2=0x7f2898154b90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T14:04:13.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f2896800640 1 --2- 172.21.15.86:0/1085263331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f289815f160 0x7f2898154b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f2896800640 1 -- 172.21.15.86:0/1085263331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2898160850 con 0x7f28980bb140 2024-04-25T14:04:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.145+0000 7f31e5e00640 1 --2- 172.21.15.86:0/1406367161 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f31c806f4d0 0x7f31c80719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.149+0000 7f9516a00640 1 -- 172.21.15.86:0/3394498723 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f951001b020 con 0x7f95200cdb60 2024-04-25T14:04:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.149+0000 7f31e5e00640 1 --2- 172.21.15.86:0/1406367161 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f31c806f4d0 0x7f31c80719e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f31d8000f70 tx=0x7f31d8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.149+0000 7f2896800640 1 --2- 172.21.15.86:0/1085263331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28980bb140 0x7f2898160060 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f288000c820 tx=0x7f288000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.149+0000 7fe089e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe070005340 0x7fe08c155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.149+0000 7fe089400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe08c157480 0x7fe08c159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.149+0000 7f44e6a00640 1 --2- 172.21.15.86:0/577765441 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f44d006f600 0x7f44d0071b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.149+0000 7fe089e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe070005340 0x7fe08c155c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38110/0 (socket says 172.21.15.86:38110) 2024-04-25T14:04:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.149+0000 7fe089e00640 1 -- 172.21.15.86:0/2192008817 learned_addr learned my addr 172.21.15.86:0/2192008817 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.149+0000 7fe08a800640 1 --2- 172.21.15.86:0/2192008817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe08c1603a0 0x7fe08c0cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.149+0000 7f44e6a00640 1 -- 172.21.15.86:0/577765441 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f44dc08e490 con 0x7f44f00d1200 2024-04-25T14:04:13.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.149+0000 7f934ce00640 1 -- 172.21.15.86:0/2783121928 shutdown_connections 2024-04-25T14:04:13.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.149+0000 7f934ce00640 1 --2- 172.21.15.86:0/2783121928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9348074960 0x7f93480cfec0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.149+0000 7f44ee800640 1 --2- 172.21.15.86:0/577765441 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f44d006f600 0x7f44d0071b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.149+0000 7fe089e00640 1 -- 172.21.15.86:0/2192008817 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe08c157480 msgr2=0x7fe08c159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.149+0000 7fe089e00640 1 --2- 172.21.15.86:0/2192008817 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe08c157480 0x7fe08c159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.149+0000 7fe089e00640 1 -- 172.21.15.86:0/2192008817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe08c1603a0 msgr2=0x7fe08c0cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.149+0000 7fe089e00640 1 --2- 172.21.15.86:0/2192008817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe08c1603a0 0x7fe08c0cebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.149+0000 7fe089e00640 1 -- 172.21.15.86:0/2192008817 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe08c157f80 con 0x7fe070005340 2024-04-25T14:04:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.149+0000 7f934ce00640 1 --2- 172.21.15.86:0/2783121928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f934806d9d0 0x7f934815bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.149+0000 7f934ce00640 1 -- 172.21.15.86:0/2783121928 >> 172.21.15.86:0/2783121928 conn(0x7f93480b6130 msgr2=0x7f93480b6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.149+0000 7fe089e00640 1 --2- 172.21.15.86:0/2192008817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe070005340 0x7fe08c155c00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00b3e0 tx=0x7fe07c00b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.149+0000 7f934ce00640 1 -- 172.21.15.86:0/2783121928 shutdown_connections 2024-04-25T14:04:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.149+0000 7f934ce00640 1 -- 172.21.15.86:0/2783121928 wait complete. 2024-04-25T14:04:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.149+0000 7f9516a00640 1 --2- 172.21.15.86:0/3394498723 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f950006f450 0x7f9500071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.149+0000 7f9516a00640 1 -- 172.21.15.86:0/3394498723 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f951008fc90 con 0x7f95200cdb60 2024-04-25T14:04:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.149+0000 7f44ee800640 1 --2- 172.21.15.86:0/577765441 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f44d006f600 0x7f44d0071b10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f44e001c510 tx=0x7f44e0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.149+0000 7f951ea00640 1 --2- 172.21.15.86:0/3394498723 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f950006f450 0x7f9500071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.149+0000 7f951ea00640 1 --2- 172.21.15.86:0/3394498723 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f950006f450 0x7f9500071960 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f950c0048e0 tx=0x7f950c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.153+0000 7f31de000640 1 -- 172.21.15.86:0/1406367161 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f31d4016110 con 0x7f31e80d2a50 2024-04-25T14:04:13.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.153+0000 7f44e6a00640 1 -- 172.21.15.86:0/577765441 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f44dc05a950 con 0x7f44f00d1200 2024-04-25T14:04:13.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.153+0000 7f934ce00640 1 Processor -- start 2024-04-25T14:04:13.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.153+0000 7f934ce00640 1 -- start start 2024-04-25T14:04:13.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.153+0000 7f690a000640 1 -- 172.21.15.86:0/3926230271 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6900008be0 con 0x7f690404fe70 2024-04-25T14:04:13.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.153+0000 7fe082000640 1 -- 172.21.15.86:0/2192008817 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c015020 con 0x7fe070005340 2024-04-25T14:04:13.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.153+0000 7f288ea00640 1 -- 172.21.15.86:0/1085263331 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880015070 con 0x7f28980bb140 2024-04-25T14:04:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.153+0000 7f289ca00640 1 -- 172.21.15.86:0/1085263331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2898160b30 con 0x7f28980bb140 2024-04-25T14:04:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.153+0000 7f690a000640 1 -- 172.21.15.86:0/3926230271 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69000b0070 con 0x7f690404fe70 2024-04-25T14:04:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.153+0000 7f289ca00640 1 -- 172.21.15.86:0/1085263331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28981615e0 con 0x7f28980bb140 2024-04-25T14:04:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.153+0000 7f288ea00640 1 -- 172.21.15.86:0/1085263331 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2880007a30 con 0x7f28980bb140 2024-04-25T14:04:13.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.153+0000 7f9516a00640 1 -- 172.21.15.86:0/3394498723 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f951005c920 con 0x7f95200cdb60 2024-04-25T14:04:13.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.153+0000 7f288ea00640 1 -- 172.21.15.86:0/1085263331 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880007e20 con 0x7f28980bb140 2024-04-25T14:04:13.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.153+0000 7fe082000640 1 -- 172.21.15.86:0/2192008817 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe07c011850 con 0x7fe070005340 2024-04-25T14:04:13.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.153+0000 7fe082000640 1 -- 172.21.15.86:0/2192008817 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c011c40 con 0x7fe070005340 2024-04-25T14:04:13.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.153+0000 7fe090e00640 1 -- 172.21.15.86:0/2192008817 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe08c158260 con 0x7fe070005340 2024-04-25T14:04:13.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.153+0000 7fe090e00640 1 -- 172.21.15.86:0/2192008817 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe08c158820 con 0x7fe070005340 2024-04-25T14:04:13.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.157+0000 7f690a000640 1 -- 172.21.15.86:0/3926230271 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6900008be0 con 0x7f690404fe70 2024-04-25T14:04:13.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.157+0000 7f6911000640 1 -- 172.21.15.86:0/3926230271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f690404fe70 msgr2=0x7f6904006860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.157+0000 7f6911000640 1 --2- 172.21.15.86:0/3926230271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f690404fe70 0x7f6904006860 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f69040e32b0 tx=0x7f690001d4a0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.157+0000 7f6911000640 1 -- 172.21.15.86:0/3926230271 shutdown_connections 2024-04-25T14:04:13.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.157+0000 7f6911000640 1 --2- 172.21.15.86:0/3926230271 >> v2:172.21.15.86:3300/0 conn(0x7f6904006da0 0x7f69040071b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.157+0000 7f6911000640 1 --2- 172.21.15.86:0/3926230271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f690404fe70 0x7f6904006860 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.157+0000 7f6911000640 1 -- 172.21.15.86:0/3926230271 >> 172.21.15.86:0/3926230271 conn(0x7f6904054d00 msgr2=0x7f6904055110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.157+0000 7f6911000640 1 -- 172.21.15.86:0/3926230271 shutdown_connections 2024-04-25T14:04:13.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.157+0000 7f289ca00640 1 -- 172.21.15.86:0/1085263331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f286c0052a0 con 0x7f28980bb140 2024-04-25T14:04:13.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.157+0000 7fe090e00640 1 -- 172.21.15.86:0/2192008817 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0640052a0 con 0x7fe070005340 2024-04-25T14:04:13.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.157+0000 7f934ce00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f934806d9d0 0x7f934815b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.157+0000 7f934ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9348074550 0x7f934809a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.157+0000 7f934ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9348074960 0x7f934815dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.157+0000 7f9345e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f934806d9d0 0x7f934815b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.157+0000 7f9345e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f934806d9d0 0x7f934815b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:48414/0 (socket says 172.21.15.86:48414) 2024-04-25T14:04:13.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.157+0000 7f934ce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93480b23e0 con 0x7f9348074960 2024-04-25T14:04:13.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.157+0000 7f9346800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9348074960 0x7f934815dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.161+0000 7f9346800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9348074960 0x7f934815dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38112/0 (socket says 172.21.15.86:38112) 2024-04-25T14:04:13.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.161+0000 7f9346800640 1 -- 172.21.15.86:0/3871576992 learned_addr learned my addr 172.21.15.86:0/3871576992 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:13.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.157+0000 7f6911000640 1 -- 172.21.15.86:0/3926230271 wait complete. 2024-04-25T14:04:13.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.161+0000 7f6911000640 1 Processor -- start 2024-04-25T14:04:13.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.161+0000 7f6911000640 1 -- start start 2024-04-25T14:04:13.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.161+0000 7f9345400640 1 --2- 172.21.15.86:0/3871576992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9348074550 0x7f934809a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.161+0000 7f934ce00640 1 -- 172.21.15.86:0/3871576992 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9348166e20 con 0x7f9348074550 2024-04-25T14:04:13.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.161+0000 7f934ce00640 1 -- 172.21.15.86:0/3871576992 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f93480b2560 con 0x7f934806d9d0 2024-04-25T14:04:13.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.161+0000 7fe082000640 1 -- 172.21.15.86:0/2192008817 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe07c018020 con 0x7fe070005340 2024-04-25T14:04:13.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.161+0000 7fe082000640 1 --2- 172.21.15.86:0/2192008817 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe06c06f500 0x7fe06c071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.161+0000 7fe082000640 1 -- 172.21.15.86:0/2192008817 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe07c08f2e0 con 0x7fe070005340 2024-04-25T14:04:13.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.165+0000 7f9345400640 1 -- 172.21.15.86:0/3871576992 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f934806d9d0 msgr2=0x7f934815b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.165+0000 7f9345400640 1 --2- 172.21.15.86:0/3871576992 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f934806d9d0 0x7f934815b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.165+0000 7f9345400640 1 -- 172.21.15.86:0/3871576992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9348074960 msgr2=0x7f934815dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.165+0000 7f9345400640 1 --2- 172.21.15.86:0/3871576992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9348074960 0x7f934815dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.165+0000 7f9345400640 1 -- 172.21.15.86:0/3871576992 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934809ae20 con 0x7f9348074550 2024-04-25T14:04:13.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.161+0000 7f288ea00640 1 -- 172.21.15.86:0/1085263331 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2880018070 con 0x7f28980bb140 2024-04-25T14:04:13.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.165+0000 7f288ea00640 1 --2- 172.21.15.86:0/1085263331 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f287406f430 0x7f2874071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.165+0000 7f288ea00640 1 -- 172.21.15.86:0/1085263331 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f288008f970 con 0x7f28980bb140 2024-04-25T14:04:13.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.165+0000 7f9345400640 1 --2- 172.21.15.86:0/3871576992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9348074550 0x7f934809a8e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f93480b15d0 tx=0x7f933800b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.165+0000 7fe089400640 1 --2- 172.21.15.86:0/2192008817 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe06c06f500 0x7fe06c071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.165+0000 7f288fe00640 1 --2- 172.21.15.86:0/1085263331 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f287406f430 0x7f2874071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.165+0000 7f933e000640 1 -- 172.21.15.86:0/3871576992 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9338015020 con 0x7f9348074550 2024-04-25T14:04:13.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.165+0000 7f933e000640 1 -- 172.21.15.86:0/3871576992 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9338011830 con 0x7f9348074550 2024-04-25T14:04:13.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.165+0000 7f934ce00640 1 -- 172.21.15.86:0/3871576992 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9348200af0 con 0x7f9348074550 2024-04-25T14:04:13.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.165+0000 7f934ce00640 1 -- 172.21.15.86:0/3871576992 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93482010b0 con 0x7f9348074550 2024-04-25T14:04:13.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.165+0000 7f933e000640 1 -- 172.21.15.86:0/3871576992 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9338011c60 con 0x7f9348074550 2024-04-25T14:04:13.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.165+0000 7f6911000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6904006da0 0x7f69040de3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.165+0000 7f6911000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f690404f350 0x7f69040dcf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.165+0000 7f6911000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f690404fe70 0x7f69040dd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.165+0000 7f6911000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f69040ecf00 con 0x7f690404fe70 2024-04-25T14:04:13.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.165+0000 7f6911000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f69040e0390 con 0x7f690404f350 2024-04-25T14:04:13.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.165+0000 7f6911000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f69040ed080 con 0x7f6904006da0 2024-04-25T14:04:13.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.165+0000 7f288fe00640 1 --2- 172.21.15.86:0/1085263331 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f287406f430 0x7f2874071940 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2888007fd0 tx=0x7f2888007900 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.169+0000 7f690aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f690404f350 0x7f69040dcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.169+0000 7f690aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f690404f350 0x7f69040dcf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:54510/0 (socket says 172.21.15.86:54510) 2024-04-25T14:04:13.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.169+0000 7f690aa00640 1 -- 172.21.15.86:0/1950830698 learned_addr learned my addr 172.21.15.86:0/1950830698 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:13.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.169+0000 7fe089400640 1 --2- 172.21.15.86:0/2192008817 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe06c06f500 0x7fe06c071a10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe08c1632e0 tx=0x7fe078008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.169+0000 7f690b400640 1 --2- 172.21.15.86:0/1950830698 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6904006da0 0x7f69040de3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.169+0000 7f690be00640 1 --2- 172.21.15.86:0/1950830698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f690404fe70 0x7f69040dd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.169+0000 7f690b400640 1 -- 172.21.15.86:0/1950830698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f690404f350 msgr2=0x7f69040dcf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.169+0000 7f934ce00640 1 -- 172.21.15.86:0/3871576992 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93200052a0 con 0x7f9348074550 2024-04-25T14:04:13.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.169+0000 7f933e000640 1 -- 172.21.15.86:0/3871576992 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9338018020 con 0x7f9348074550 2024-04-25T14:04:13.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.169+0000 7f690b400640 1 --2- 172.21.15.86:0/1950830698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f690404f350 0x7f69040dcf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.169+0000 7f690b400640 1 -- 172.21.15.86:0/1950830698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f690404fe70 msgr2=0x7f69040dd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.169+0000 7f690b400640 1 --2- 172.21.15.86:0/1950830698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f690404fe70 0x7f69040dd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.169+0000 7f690b400640 1 -- 172.21.15.86:0/1950830698 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f690404d810 con 0x7f6904006da0 2024-04-25T14:04:13.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.169+0000 7f690be00640 1 --2- 172.21.15.86:0/1950830698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f690404fe70 0x7f69040dd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:13.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.169+0000 7f690aa00640 1 --2- 172.21.15.86:0/1950830698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f690404f350 0x7f69040dcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:13.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.169+0000 7f690b400640 1 --2- 172.21.15.86:0/1950830698 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6904006da0 0x7f69040de3d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f690400d6e0 tx=0x7f68f800c870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.169+0000 7f68ff400640 1 -- 172.21.15.86:0/1950830698 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f80074e0 con 0x7f6904006da0 2024-04-25T14:04:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.169+0000 7f68ff400640 1 -- 172.21.15.86:0/1950830698 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68f80047f0 con 0x7f6904006da0 2024-04-25T14:04:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.169+0000 7f68ff400640 1 -- 172.21.15.86:0/1950830698 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8004be0 con 0x7f6904006da0 2024-04-25T14:04:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.169+0000 7f288ea00640 1 -- 172.21.15.86:0/1085263331 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f288005c500 con 0x7f28980bb140 2024-04-25T14:04:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.173+0000 7f6911000640 1 -- 172.21.15.86:0/1950830698 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6904189940 con 0x7f6904006da0 2024-04-25T14:04:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.173+0000 7f6911000640 1 -- 172.21.15.86:0/1950830698 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f690404dff0 con 0x7f6904006da0 2024-04-25T14:04:13.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.173+0000 7f933e000640 1 --2- 172.21.15.86:0/3871576992 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f932806f4d0 0x7f93280719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.173+0000 7f933e000640 1 -- 172.21.15.86:0/3871576992 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f933808f1d0 con 0x7f9348074550 2024-04-25T14:04:13.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.173+0000 7f9345e00640 1 --2- 172.21.15.86:0/3871576992 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f932806f4d0 0x7f93280719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.173+0000 7f9345e00640 1 --2- 172.21.15.86:0/3871576992 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f932806f4d0 0x7f93280719e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f93480d1280 tx=0x7f934000d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.173+0000 7f6911000640 1 -- 172.21.15.86:0/1950830698 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6904005580 con 0x7f6904006da0 2024-04-25T14:04:13.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.177+0000 7f933e000640 1 -- 172.21.15.86:0/3871576992 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f933801f020 con 0x7f9348074550 2024-04-25T14:04:13.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.177+0000 7f68ff400640 1 -- 172.21.15.86:0/1950830698 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f68f8011070 con 0x7f6904006da0 2024-04-25T14:04:13.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.177+0000 7f68ff400640 1 --2- 172.21.15.86:0/1950830698 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f68ec06f480 0x7f68ec071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.177+0000 7f690aa00640 1 --2- 172.21.15.86:0/1950830698 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f68ec06f480 0x7f68ec071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.177+0000 7f690aa00640 1 --2- 172.21.15.86:0/1950830698 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f68ec06f480 0x7f68ec071990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f69000ad040 tx=0x7f69000a8000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.181+0000 7f68ff400640 1 -- 172.21.15.86:0/1950830698 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f68f8057af0 con 0x7f6904006da0 2024-04-25T14:04:13.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.181+0000 7fe082000640 1 -- 172.21.15.86:0/2192008817 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe07c01f020 con 0x7fe070005340 2024-04-25T14:04:13.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.185+0000 7f68ff400640 1 -- 172.21.15.86:0/1950830698 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f68f805bc00 con 0x7f6904006da0 2024-04-25T14:04:13.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.489+0000 7f31eca00640 1 -- 172.21.15.86:0/1406367161 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f31c0005590 con 0x7f31e80d2a50 2024-04-25T14:04:13.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.489+0000 7f9525800640 1 -- 172.21.15.86:0/3394498723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f9520076bf0 con 0x7f95200cdb60 2024-04-25T14:04:13.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.489+0000 7f964a400640 1 -- 172.21.15.86:0/1176285558 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f9644070ce0 con 0x7f96440b6f90 2024-04-25T14:04:13.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.489+0000 7f9640c00640 1 -- 172.21.15.86:0/1176285558 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f96380fa570 con 0x7f96440b6f90 2024-04-25T14:04:13.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.489+0000 7f31de000640 1 -- 172.21.15.86:0/1406367161 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f31d40f5c80 con 0x7f31e80d2a50 2024-04-25T14:04:13.492 INFO:teuthology.orchestra.run.smithi086.stdout:55834574856 2024-04-25T14:04:13.494 INFO:teuthology.orchestra.run.smithi086.stdout:55834574855 2024-04-25T14:04:13.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.493+0000 7f9516a00640 1 -- 172.21.15.86:0/3394498723 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f951005c220 con 0x7f95200cdb60 2024-04-25T14:04:13.495 INFO:teuthology.orchestra.run.smithi086.stdout:55834574857 2024-04-25T14:04:13.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.493+0000 7f964a400640 1 -- 172.21.15.86:0/1176285558 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f962406f500 msgr2=0x7f9624071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.493+0000 7f964a400640 1 --2- 172.21.15.86:0/1176285558 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f962406f500 0x7f9624071a10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f96440d3c10 tx=0x7f9630009040 comp rx=0 tx=0).stop 2024-04-25T14:04:13.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.493+0000 7f964a400640 1 -- 172.21.15.86:0/1176285558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96440b6f90 msgr2=0x7f9644162620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.493+0000 7f964a400640 1 --2- 172.21.15.86:0/1176285558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96440b6f90 0x7f9644162620 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f96380098a0 tx=0x7f963801bb10 comp rx=0 tx=0).stop 2024-04-25T14:04:13.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.493+0000 7f964a400640 1 -- 172.21.15.86:0/1176285558 shutdown_connections 2024-04-25T14:04:13.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.493+0000 7f964a400640 1 --2- 172.21.15.86:0/1176285558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96440d0540 0x7f96440cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.493+0000 7f964a400640 1 --2- 172.21.15.86:0/1176285558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96440ba0c0 0x7f96440ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.493+0000 7f964a400640 1 --2- 172.21.15.86:0/1176285558 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f962406f500 0x7f9624071a10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.493+0000 7f964a400640 1 --2- 172.21.15.86:0/1176285558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96440b6f90 0x7f9644162620 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.493+0000 7f964a400640 1 -- 172.21.15.86:0/1176285558 >> 172.21.15.86:0/1176285558 conn(0x7f96440d60e0 msgr2=0x7f96440ba4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.493+0000 7f9525800640 1 -- 172.21.15.86:0/3394498723 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f950006f450 msgr2=0x7f9500071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.493+0000 7f9525800640 1 --2- 172.21.15.86:0/3394498723 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f950006f450 0x7f9500071960 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f950c0048e0 tx=0x7f950c009040 comp rx=0 tx=0).stop 2024-04-25T14:04:13.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.493+0000 7f964a400640 1 -- 172.21.15.86:0/1176285558 shutdown_connections 2024-04-25T14:04:13.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.493+0000 7f964a400640 1 -- 172.21.15.86:0/1176285558 wait complete. 2024-04-25T14:04:13.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.493+0000 7f9525800640 1 -- 172.21.15.86:0/3394498723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95200cdb60 msgr2=0x7f9520153570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.493+0000 7f9525800640 1 --2- 172.21.15.86:0/3394498723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95200cdb60 0x7f9520153570 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f952006d940 tx=0x7f951000d910 comp rx=0 tx=0).stop 2024-04-25T14:04:13.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.497+0000 7f9525800640 1 -- 172.21.15.86:0/3394498723 shutdown_connections 2024-04-25T14:04:13.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.497+0000 7f9525800640 1 --2- 172.21.15.86:0/3394498723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95200cdb60 0x7f9520153570 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.497+0000 7f9525800640 1 --2- 172.21.15.86:0/3394498723 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f950006f450 0x7f9500071960 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.497+0000 7f9525800640 1 --2- 172.21.15.86:0/3394498723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9520074550 0x7f9520099d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.497+0000 7f9525800640 1 --2- 172.21.15.86:0/3394498723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f952006d9d0 0x7f9520155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.497+0000 7f9525800640 1 -- 172.21.15.86:0/3394498723 >> 172.21.15.86:0/3394498723 conn(0x7f95200b6130 msgr2=0x7f95200cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.497+0000 7f31eca00640 1 -- 172.21.15.86:0/1406367161 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f31c806f4d0 msgr2=0x7f31c80719e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.497+0000 7f31eca00640 1 --2- 172.21.15.86:0/1406367161 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f31c806f4d0 0x7f31c80719e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f31d8000f70 tx=0x7f31d8009040 comp rx=0 tx=0).stop 2024-04-25T14:04:13.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.497+0000 7f31eca00640 1 -- 172.21.15.86:0/1406367161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31e80d2a50 msgr2=0x7f31e8147ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.497+0000 7f31eca00640 1 --2- 172.21.15.86:0/1406367161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31e80d2a50 0x7f31e8147ac0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f31d401ca00 tx=0x7f31d40192c0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.497+0000 7f9525800640 1 -- 172.21.15.86:0/3394498723 shutdown_connections 2024-04-25T14:04:13.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.497+0000 7f31eca00640 1 -- 172.21.15.86:0/1406367161 shutdown_connections 2024-04-25T14:04:13.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.497+0000 7f31eca00640 1 --2- 172.21.15.86:0/1406367161 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f31c806f4d0 0x7f31c80719e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.497+0000 7f31eca00640 1 --2- 172.21.15.86:0/1406367161 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31e815cc80 0x7f31e8150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.497+0000 7f31eca00640 1 --2- 172.21.15.86:0/1406367161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31e80d2a50 0x7f31e8147ac0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.497+0000 7f31eca00640 1 --2- 172.21.15.86:0/1406367161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31e80b3a90 0x7f31e8147580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.497+0000 7f31eca00640 1 -- 172.21.15.86:0/1406367161 >> 172.21.15.86:0/1406367161 conn(0x7f31e809ba30 msgr2=0x7f31e80d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.497+0000 7f31eca00640 1 -- 172.21.15.86:0/1406367161 shutdown_connections 2024-04-25T14:04:13.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.497+0000 7f31eca00640 1 -- 172.21.15.86:0/1406367161 wait complete. 2024-04-25T14:04:13.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.497+0000 7f9525800640 1 -- 172.21.15.86:0/3394498723 wait complete. 2024-04-25T14:04:13.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.497+0000 7f44f5400640 1 -- 172.21.15.86:0/577765441 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f44f00cc140 con 0x7f44f00d1200 2024-04-25T14:04:13.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.497+0000 7f44e6a00640 1 -- 172.21.15.86:0/577765441 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f44dc05ec50 con 0x7f44f00d1200 2024-04-25T14:04:13.508 INFO:teuthology.orchestra.run.smithi086.stdout:55834574856 2024-04-25T14:04:13.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.509+0000 7f44f5400640 1 -- 172.21.15.86:0/577765441 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f44d006f600 msgr2=0x7f44d0071b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.509+0000 7f44f5400640 1 --2- 172.21.15.86:0/577765441 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f44d006f600 0x7f44d0071b10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f44e001c510 tx=0x7f44e0008040 comp rx=0 tx=0).stop 2024-04-25T14:04:13.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.509+0000 7f44f5400640 1 -- 172.21.15.86:0/577765441 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44f00d1200 msgr2=0x7f44f0148de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.509+0000 7f44f5400640 1 --2- 172.21.15.86:0/577765441 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44f00d1200 0x7f44f0148de0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f44dc00c390 tx=0x7f44dc00c850 comp rx=0 tx=0).stop 2024-04-25T14:04:13.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.509+0000 7f44f5400640 1 -- 172.21.15.86:0/577765441 shutdown_connections 2024-04-25T14:04:13.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.509+0000 7f44f5400640 1 --2- 172.21.15.86:0/577765441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44f00d3e70 0x7f44f00ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.509+0000 7f44f5400640 1 --2- 172.21.15.86:0/577765441 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f44d006f600 0x7f44d0071b10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.509+0000 7f44f5400640 1 --2- 172.21.15.86:0/577765441 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44f00d1200 0x7f44f0148de0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.509+0000 7f44f5400640 1 --2- 172.21.15.86:0/577765441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44f00b4440 0x7f44f0157d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.509+0000 7f44f5400640 1 -- 172.21.15.86:0/577765441 >> 172.21.15.86:0/577765441 conn(0x7f44f00b72d0 msgr2=0x7f44f00b9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.509+0000 7f289ca00640 1 -- 172.21.15.86:0/1085263331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f286c005860 con 0x7f28980bb140 2024-04-25T14:04:13.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.509+0000 7f44f5400640 1 -- 172.21.15.86:0/577765441 shutdown_connections 2024-04-25T14:04:13.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.509+0000 7f44f5400640 1 -- 172.21.15.86:0/577765441 wait complete. 2024-04-25T14:04:13.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.509+0000 7f934ce00640 1 -- 172.21.15.86:0/3871576992 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f9320005590 con 0x7f9348074550 2024-04-25T14:04:13.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.509+0000 7f288ea00640 1 -- 172.21.15.86:0/1085263331 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f288005be00 con 0x7f28980bb140 2024-04-25T14:04:13.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.513+0000 7f933e000640 1 -- 172.21.15.86:0/3871576992 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f933805bdf0 con 0x7f9348074550 2024-04-25T14:04:13.513 INFO:teuthology.orchestra.run.smithi086.stdout:55834574856 2024-04-25T14:04:13.514 INFO:teuthology.orchestra.run.smithi086.stdout:55834574857 2024-04-25T14:04:13.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.517+0000 7f934ce00640 1 -- 172.21.15.86:0/3871576992 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f932806f4d0 msgr2=0x7f93280719e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.517+0000 7f934ce00640 1 --2- 172.21.15.86:0/3871576992 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f932806f4d0 0x7f93280719e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f93480d1280 tx=0x7f934000d040 comp rx=0 tx=0).stop 2024-04-25T14:04:13.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.517+0000 7f934ce00640 1 -- 172.21.15.86:0/3871576992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9348074550 msgr2=0x7f934809a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.517+0000 7f934ce00640 1 --2- 172.21.15.86:0/3871576992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9348074550 0x7f934809a8e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f93480b15d0 tx=0x7f933800b870 comp rx=0 tx=0).stop 2024-04-25T14:04:13.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.517+0000 7f934ce00640 1 -- 172.21.15.86:0/3871576992 shutdown_connections 2024-04-25T14:04:13.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.517+0000 7f934ce00640 1 --2- 172.21.15.86:0/3871576992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9348074960 0x7f934815dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.517+0000 7f934ce00640 1 --2- 172.21.15.86:0/3871576992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9348074550 0x7f934809a8e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.517+0000 7f934ce00640 1 --2- 172.21.15.86:0/3871576992 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f932806f4d0 0x7f93280719e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.517+0000 7f934ce00640 1 --2- 172.21.15.86:0/3871576992 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f934806d9d0 0x7f934815b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.517+0000 7f934ce00640 1 -- 172.21.15.86:0/3871576992 >> 172.21.15.86:0/3871576992 conn(0x7f93480b6130 msgr2=0x7f93480cc560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.517+0000 7fe090e00640 1 -- 172.21.15.86:0/2192008817 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fe064005590 con 0x7fe070005340 2024-04-25T14:04:13.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.517+0000 7f289ca00640 1 -- 172.21.15.86:0/1085263331 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f287406f430 msgr2=0x7f2874071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.517+0000 7f289ca00640 1 --2- 172.21.15.86:0/1085263331 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f287406f430 0x7f2874071940 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2888007fd0 tx=0x7f2888007900 comp rx=0 tx=0).stop 2024-04-25T14:04:13.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.517+0000 7f289ca00640 1 -- 172.21.15.86:0/1085263331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28980bb140 msgr2=0x7f2898160060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.517+0000 7f289ca00640 1 --2- 172.21.15.86:0/1085263331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28980bb140 0x7f2898160060 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f288000c820 tx=0x7f288000cce0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.517+0000 7f289ca00640 1 -- 172.21.15.86:0/1085263331 shutdown_connections 2024-04-25T14:04:13.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.517+0000 7f289ca00640 1 --2- 172.21.15.86:0/1085263331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f289815f160 0x7f2898154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.517+0000 7f289ca00640 1 --2- 172.21.15.86:0/1085263331 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f287406f430 0x7f2874071940 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.517+0000 7f289ca00640 1 --2- 172.21.15.86:0/1085263331 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28980d1de0 0x7f28980ca250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.517+0000 7fe082000640 1 -- 172.21.15.86:0/2192008817 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe07c05bf00 con 0x7fe070005340 2024-04-25T14:04:13.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.517+0000 7f289ca00640 1 --2- 172.21.15.86:0/1085263331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28980bb140 0x7f2898160060 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.517+0000 7f289ca00640 1 -- 172.21.15.86:0/1085263331 >> 172.21.15.86:0/1085263331 conn(0x7f289806d160 msgr2=0x7f28980b47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.522 INFO:teuthology.orchestra.run.smithi086.stdout:55834574856 2024-04-25T14:04:13.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.521+0000 7f289ca00640 1 -- 172.21.15.86:0/1085263331 shutdown_connections 2024-04-25T14:04:13.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.521+0000 7f934ce00640 1 -- 172.21.15.86:0/3871576992 shutdown_connections 2024-04-25T14:04:13.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.521+0000 7f934ce00640 1 -- 172.21.15.86:0/3871576992 wait complete. 2024-04-25T14:04:13.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.521+0000 7f6911000640 1 -- 172.21.15.86:0/1950830698 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f6904053630 con 0x7f6904006da0 2024-04-25T14:04:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.521+0000 7f289ca00640 1 -- 172.21.15.86:0/1085263331 wait complete. 2024-04-25T14:04:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.521+0000 7f68ff400640 1 -- 172.21.15.86:0/1950830698 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f68f805b500 con 0x7f6904006da0 2024-04-25T14:04:13.524 INFO:teuthology.orchestra.run.smithi086.stdout:55834574855 2024-04-25T14:04:13.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.525+0000 7fe090e00640 1 -- 172.21.15.86:0/2192008817 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe06c06f500 msgr2=0x7fe06c071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.525+0000 7fe090e00640 1 --2- 172.21.15.86:0/2192008817 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe06c06f500 0x7fe06c071a10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe08c1632e0 tx=0x7fe078008040 comp rx=0 tx=0).stop 2024-04-25T14:04:13.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.525+0000 7fe090e00640 1 -- 172.21.15.86:0/2192008817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe070005340 msgr2=0x7fe08c155c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.525+0000 7fe090e00640 1 --2- 172.21.15.86:0/2192008817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe070005340 0x7fe08c155c00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00b3e0 tx=0x7fe07c00b7e0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.525+0000 7f6911000640 1 -- 172.21.15.86:0/1950830698 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f68ec06f480 msgr2=0x7f68ec071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.525+0000 7f6911000640 1 --2- 172.21.15.86:0/1950830698 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f68ec06f480 0x7f68ec071990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f69000ad040 tx=0x7f69000a8000 comp rx=0 tx=0).stop 2024-04-25T14:04:13.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.525+0000 7f6911000640 1 -- 172.21.15.86:0/1950830698 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6904006da0 msgr2=0x7f69040de3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.525+0000 7f6911000640 1 --2- 172.21.15.86:0/1950830698 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6904006da0 0x7f69040de3d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f690400d6e0 tx=0x7f68f800c870 comp rx=0 tx=0).stop 2024-04-25T14:04:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.525+0000 7f6911000640 1 -- 172.21.15.86:0/1950830698 shutdown_connections 2024-04-25T14:04:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.525+0000 7f6911000640 1 --2- 172.21.15.86:0/1950830698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f690404fe70 0x7f69040dd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.525+0000 7f6911000640 1 --2- 172.21.15.86:0/1950830698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f690404f350 0x7f69040dcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.525+0000 7f6911000640 1 --2- 172.21.15.86:0/1950830698 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f68ec06f480 0x7f68ec071990 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.525+0000 7f6911000640 1 --2- 172.21.15.86:0/1950830698 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6904006da0 0x7f69040de3d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.525+0000 7f6911000640 1 -- 172.21.15.86:0/1950830698 >> 172.21.15.86:0/1950830698 conn(0x7f6904054d00 msgr2=0x7f690404f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.525+0000 7fe090e00640 1 -- 172.21.15.86:0/2192008817 shutdown_connections 2024-04-25T14:04:13.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.525+0000 7fe090e00640 1 --2- 172.21.15.86:0/2192008817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe08c1603a0 0x7fe08c0cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.525+0000 7fe090e00640 1 --2- 172.21.15.86:0/2192008817 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe06c06f500 0x7fe06c071a10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.525+0000 7fe090e00640 1 --2- 172.21.15.86:0/2192008817 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe08c157480 0x7fe08c159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.525+0000 7fe090e00640 1 --2- 172.21.15.86:0/2192008817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe070005340 0x7fe08c155c00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.525+0000 7fe090e00640 1 -- 172.21.15.86:0/2192008817 >> 172.21.15.86:0/2192008817 conn(0x7fe08c0b72d0 msgr2=0x7fe08c0b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.525+0000 7f6911000640 1 -- 172.21.15.86:0/1950830698 shutdown_connections 2024-04-25T14:04:13.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.525+0000 7f6911000640 1 -- 172.21.15.86:0/1950830698 wait complete. 2024-04-25T14:04:13.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.529+0000 7fe090e00640 1 -- 172.21.15.86:0/2192008817 shutdown_connections 2024-04-25T14:04:13.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:13.529+0000 7fe090e00640 1 -- 172.21.15.86:0/2192008817 wait complete. 2024-04-25T14:04:13.533 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.4 2024-04-25T14:04:13.541 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.5 2024-04-25T14:04:13.542 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.2 2024-04-25T14:04:13.554 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.0 2024-04-25T14:04:13.555 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.6 2024-04-25T14:04:13.561 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.7 2024-04-25T14:04:13.563 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.1 2024-04-25T14:04:13.567 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.3 2024-04-25T14:04:14.534 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T14:04:14.542 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T14:04:14.543 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T14:04:14.555 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T14:04:14.556 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T14:04:14.562 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T14:04:14.564 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T14:04:14.569 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T14:04:14.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.781+0000 7fd604a00640 1 Processor -- start 2024-04-25T14:04:14.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.781+0000 7fd604a00640 1 -- start start 2024-04-25T14:04:14.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.785+0000 7fd604a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd600160670 0x7fd60015f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.785+0000 7fd604a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd60015f980 0x7fd60015fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.785+0000 7fd604a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6001602d0 con 0x7fd6000ce100 2024-04-25T14:04:14.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.785+0000 7fd604a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd6000b3a90 con 0x7fd600160670 2024-04-25T14:04:14.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.785+0000 7fd604a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd6000b22e0 con 0x7fd60015f980 2024-04-25T14:04:14.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.785+0000 7fd5fe800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd60015f980 0x7fd60015fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.789+0000 7fd5fe800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd60015f980 0x7fd60015fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:54512/0 (socket says 172.21.15.86:54512) 2024-04-25T14:04:14.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.789+0000 7fd5fe800640 1 -- 172.21.15.86:0/2779918553 learned_addr learned my addr 172.21.15.86:0/2779918553 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:14.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.789+0000 7fd5fde00640 1 --2- 172.21.15.86:0/2779918553 >> v2:172.21.15.86:3300/0 conn(0x7fd600160670 0x7fd60015f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.789+0000 7fd5fe800640 1 -- 172.21.15.86:0/2779918553 >> v2:172.21.15.86:3300/0 conn(0x7fd600160670 msgr2=0x7fd60015f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.789+0000 7fd5fe800640 1 --2- 172.21.15.86:0/2779918553 >> v2:172.21.15.86:3300/0 conn(0x7fd600160670 0x7fd60015f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.789+0000 7fd5fe800640 1 -- 172.21.15.86:0/2779918553 >> v1:172.21.15.86:6789/0 conn(0x7fd6000ce100 legacy=0x7fd6000b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.789+0000 7fd5fe800640 1 -- 172.21.15.86:0/2779918553 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6000b25c0 con 0x7fd60015f980 2024-04-25T14:04:14.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.789+0000 7fd5fde00640 1 --2- 172.21.15.86:0/2779918553 >> v2:172.21.15.86:3300/0 conn(0x7fd600160670 0x7fd60015f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:14.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.793+0000 7fd5fe800640 1 --2- 172.21.15.86:0/2779918553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd60015f980 0x7fd60015fd90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd600072810 tx=0x7fd5f801c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5132fe0490ed163 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.793+0000 7fd5fca00640 1 -- 172.21.15.86:0/2779918553 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f80a5790 con 0x7fd60015f980 2024-04-25T14:04:14.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.793+0000 7fd5fca00640 1 -- 172.21.15.86:0/2779918553 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5f80a5930 con 0x7fd60015f980 2024-04-25T14:04:14.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.793+0000 7fd5fca00640 1 -- 172.21.15.86:0/2779918553 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f80a5d00 con 0x7fd60015f980 2024-04-25T14:04:14.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.797+0000 7fd604a00640 1 -- 172.21.15.86:0/2779918553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd60015f980 msgr2=0x7fd60015fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.797+0000 7fd604a00640 1 --2- 172.21.15.86:0/2779918553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd60015f980 0x7fd60015fd90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd600072810 tx=0x7fd5f801c9c0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.797+0000 7fd604a00640 1 -- 172.21.15.86:0/2779918553 shutdown_connections 2024-04-25T14:04:14.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.797+0000 7fd604a00640 1 --2- 172.21.15.86:0/2779918553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd60015f980 0x7fd60015fd90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.797+0000 7fd604a00640 1 --2- 172.21.15.86:0/2779918553 >> v2:172.21.15.86:3300/0 conn(0x7fd600160670 0x7fd60015f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.797+0000 7fd604a00640 1 -- 172.21.15.86:0/2779918553 >> 172.21.15.86:0/2779918553 conn(0x7fd6000b6890 msgr2=0x7fd6000b6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.797+0000 7fd604a00640 1 -- 172.21.15.86:0/2779918553 shutdown_connections 2024-04-25T14:04:14.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.797+0000 7fd604a00640 1 -- 172.21.15.86:0/2779918553 wait complete. 2024-04-25T14:04:14.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.801+0000 7fd604a00640 1 Processor -- start 2024-04-25T14:04:14.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.801+0000 7fd604a00640 1 -- start start 2024-04-25T14:04:14.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.805+0000 7fd604a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6000ce100 0x7fd6001f9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.805+0000 7fd604a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd60015f980 0x7fd6001fc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.805+0000 7fd604a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd600160670 0x7fd6001620c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.805+0000 7fd604a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6000ce510 con 0x7fd600160670 2024-04-25T14:04:14.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.805+0000 7fd604a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd6000b1fb0 con 0x7fd60015f980 2024-04-25T14:04:14.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.805+0000 7fd604a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd600160ce0 con 0x7fd6000ce100 2024-04-25T14:04:14.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.805+0000 7fd5fd400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd60015f980 0x7fd6001fc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.805+0000 7fd5fd400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd60015f980 0x7fd6001fc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:54516/0 (socket says 172.21.15.86:54516) 2024-04-25T14:04:14.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.805+0000 7fd5fd400640 1 -- 172.21.15.86:0/771939484 learned_addr learned my addr 172.21.15.86:0/771939484 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:14.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.805+0000 7fd5fe800640 1 --2- 172.21.15.86:0/771939484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd600160670 0x7fd6001620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.805+0000 7fd5fde00640 1 --2- 172.21.15.86:0/771939484 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6000ce100 0x7fd6001f9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.805+0000 7fd5fd400640 1 -- 172.21.15.86:0/771939484 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6000ce100 msgr2=0x7fd6001f9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.805+0000 7fd5fd400640 1 --2- 172.21.15.86:0/771939484 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6000ce100 0x7fd6001f9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.805+0000 7fd5fd400640 1 -- 172.21.15.86:0/771939484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd600160670 msgr2=0x7fd6001620c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.805+0000 7fd5fd400640 1 --2- 172.21.15.86:0/771939484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd600160670 0x7fd6001620c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.805+0000 7fd5fd400640 1 -- 172.21.15.86:0/771939484 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6001fa360 con 0x7fd60015f980 2024-04-25T14:04:14.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.805+0000 7fd5fd400640 1 --2- 172.21.15.86:0/771939484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd60015f980 0x7fd6001fc560 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00c3c0 tx=0x7fd5ec00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.813+0000 7fd5f6000640 1 -- 172.21.15.86:0/771939484 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec017070 con 0x7fd60015f980 2024-04-25T14:04:14.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.813+0000 7fd5f6000640 1 -- 172.21.15.86:0/771939484 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5ec0052d0 con 0x7fd60015f980 2024-04-25T14:04:14.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.813+0000 7fd604a00640 1 -- 172.21.15.86:0/771939484 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6001fa640 con 0x7fd60015f980 2024-04-25T14:04:14.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.813+0000 7fd604a00640 1 -- 172.21.15.86:0/771939484 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd600168560 con 0x7fd60015f980 2024-04-25T14:04:14.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.813+0000 7fd5f6000640 1 -- 172.21.15.86:0/771939484 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec003840 con 0x7fd60015f980 2024-04-25T14:04:14.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.817+0000 7fd5f6000640 1 -- 172.21.15.86:0/771939484 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd5ec01a070 con 0x7fd60015f980 2024-04-25T14:04:14.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.817+0000 7fd5f6000640 1 --2- 172.21.15.86:0/771939484 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd5e006f500 0x7fd5e0071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.817+0000 7fd5f6000640 1 -- 172.21.15.86:0/771939484 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd5ec08e8b0 con 0x7fd60015f980 2024-04-25T14:04:14.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.817+0000 7fd604a00640 1 -- 172.21.15.86:0/771939484 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd600168070 con 0x7fd60015f980 2024-04-25T14:04:14.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.825+0000 7fd5fde00640 1 --2- 172.21.15.86:0/771939484 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd5e006f500 0x7fd5e0071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.829+0000 7fd5fde00640 1 --2- 172.21.15.86:0/771939484 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd5e006f500 0x7fd5e0071a10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009580 tx=0x7fd5f0009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.829+0000 7fd5f6000640 1 -- 172.21.15.86:0/771939484 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd6001602d0 con 0x7fd60015f980 2024-04-25T14:04:14.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.845+0000 7f4a02a00640 1 Processor -- start 2024-04-25T14:04:14.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.849+0000 7f5cfa400640 1 Processor -- start 2024-04-25T14:04:14.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.849+0000 7f5cfa400640 1 -- start start 2024-04-25T14:04:14.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.849+0000 7f4a02a00640 1 -- start start 2024-04-25T14:04:14.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.853+0000 7f4a02a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49fc160da0 0x7f49fc168a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.853+0000 7f4a02a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49fc09be50 con 0x7f49fc06d780 2024-04-25T14:04:14.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.853+0000 7f4a02a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49fc0d3220 con 0x7f49fc0d17c0 2024-04-25T14:04:14.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.853+0000 7f4a02a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f49fc0b1c40 con 0x7f49fc160da0 2024-04-25T14:04:14.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.853+0000 7f4a01600640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f49fc0d17c0 0x7f49fc0d2b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:50882/0 (socket says 172.21.15.86:50882) 2024-04-25T14:04:14.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.853+0000 7f4a01600640 1 -- 172.21.15.86:0/174911994 learned_addr learned my addr 172.21.15.86:0/174911994 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:14.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.853+0000 7f4a02000640 1 --2- 172.21.15.86:0/174911994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49fc160da0 0x7f49fc168a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.853+0000 7f4a02000640 1 -- 172.21.15.86:0/174911994 >> v1:172.21.15.155:6789/0 conn(0x7f49fc0d17c0 legacy=0x7f49fc0d2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.853+0000 7f4a02000640 1 -- 172.21.15.86:0/174911994 >> v1:172.21.15.86:6789/0 conn(0x7f49fc06d780 legacy=0x7f49fc160690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T14:04:14.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.853+0000 7f4a02000640 1 -- 172.21.15.86:0/174911994 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49fc0b1680 con 0x7f49fc160da0 2024-04-25T14:04:14.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.853+0000 7f5cfa400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cf40b3950 0x7f5cf415cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.857+0000 7f5cfa400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5cf41607a0 0x7f5cf4160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.857+0000 7f5cfa400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5cf41610f0 0x7f5cf406d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.857+0000 7f5cfa400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5cf4161500 con 0x7f5cf41607a0 2024-04-25T14:04:14.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.857+0000 7f5cfa400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f5cf40af140 con 0x7f5cf41610f0 2024-04-25T14:04:14.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.857+0000 7f5cfa400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5cf40af420 con 0x7f5cf40b3950 2024-04-25T14:04:14.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.857+0000 7f5cf3400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cf40b3950 0x7f5cf415cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.857+0000 7f5cf3400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cf40b3950 0x7f5cf415cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:48460/0 (socket says 172.21.15.86:48460) 2024-04-25T14:04:14.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.857+0000 7f5cf3400640 1 -- 172.21.15.86:0/975825327 learned_addr learned my addr 172.21.15.86:0/975825327 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:14.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.857+0000 7f5cf3e00640 1 --2- 172.21.15.86:0/975825327 >> v2:172.21.15.155:3300/0 conn(0x7f5cf41610f0 0x7f5cf406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.857+0000 7f5cf3e00640 1 -- 172.21.15.86:0/975825327 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cf40b3950 msgr2=0x7f5cf415cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.857+0000 7f5cf3e00640 1 --2- 172.21.15.86:0/975825327 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cf40b3950 0x7f5cf415cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.857+0000 7f5cf3e00640 1 -- 172.21.15.86:0/975825327 >> v2:172.21.15.86:3300/0 conn(0x7f5cf41607a0 msgr2=0x7f5cf4160bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T14:04:14.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.857+0000 7f5cf3e00640 1 --2- 172.21.15.86:0/975825327 >> v2:172.21.15.86:3300/0 conn(0x7f5cf41607a0 0x7f5cf4160bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.857+0000 7f5cf3e00640 1 -- 172.21.15.86:0/975825327 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf40b3d60 con 0x7f5cf41610f0 2024-04-25T14:04:14.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.857+0000 7f5cf3400640 1 --2- 172.21.15.86:0/975825327 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cf40b3950 0x7f5cf415cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:14.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.857+0000 7f4a02000640 1 --2- 172.21.15.86:0/174911994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49fc160da0 0x7f49fc168a30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f49f400bfd0 tx=0x7f49f401e8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b656b82115559cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.857+0000 7f49fbe00640 1 -- 172.21.15.86:0/174911994 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f400b2d0 con 0x7f49fc160da0 2024-04-25T14:04:14.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.857+0000 7f49fbe00640 1 -- 172.21.15.86:0/174911994 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49f4009540 con 0x7f49fc160da0 2024-04-25T14:04:14.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.857+0000 7f49fbe00640 1 -- 172.21.15.86:0/174911994 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f4009910 con 0x7f49fc160da0 2024-04-25T14:04:14.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.857+0000 7fa72f200640 1 Processor -- start 2024-04-25T14:04:14.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.857+0000 7fa72f200640 1 -- start start 2024-04-25T14:04:14.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.861+0000 7f4a02a00640 1 -- 172.21.15.86:0/174911994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49fc160da0 msgr2=0x7f49fc168a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.861+0000 7f4a02a00640 1 --2- 172.21.15.86:0/174911994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49fc160da0 0x7f49fc168a30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f49f400bfd0 tx=0x7f49f401e8e0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.861+0000 7f4a02a00640 1 -- 172.21.15.86:0/174911994 shutdown_connections 2024-04-25T14:04:14.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.861+0000 7f4a02a00640 1 --2- 172.21.15.86:0/174911994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49fc160da0 0x7f49fc168a30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.861+0000 7f4a02a00640 1 -- 172.21.15.86:0/174911994 >> 172.21.15.86:0/174911994 conn(0x7f49fc073c00 msgr2=0x7f49fc0b77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.861+0000 7f5cf3e00640 1 --2- 172.21.15.86:0/975825327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cf41610f0 0x7f5cf406d780 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5cf40cff90 tx=0x7f5ce401c150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=287437e3aa323f0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.861+0000 7f4a02a00640 1 -- 172.21.15.86:0/174911994 shutdown_connections 2024-04-25T14:04:14.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.861+0000 7f4a02a00640 1 -- 172.21.15.86:0/174911994 wait complete. 2024-04-25T14:04:14.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.861+0000 7f5cf2000640 1 -- 172.21.15.86:0/975825327 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce40b0070 con 0x7f5cf41610f0 2024-04-25T14:04:14.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.861+0000 7f5cf2000640 1 -- 172.21.15.86:0/975825327 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ce40ab600 con 0x7f5cf41610f0 2024-04-25T14:04:14.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.861+0000 7f5cf2000640 1 -- 172.21.15.86:0/975825327 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce40ab9d0 con 0x7f5cf41610f0 2024-04-25T14:04:14.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.861+0000 7fa72f200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa724054190 0x7fa7240075a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.861+0000 7fa72f200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa724053280 con 0x7fa724038310 2024-04-25T14:04:14.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.861+0000 7fa72f200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7240e7fc0 con 0x7fa724053d80 2024-04-25T14:04:14.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.861+0000 7fa72f200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa724034840 con 0x7fa724054190 2024-04-25T14:04:14.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.861+0000 7fa72e800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa724054190 0x7fa7240075a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.861+0000 7fa72e800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa724054190 0x7fa7240075a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:48474/0 (socket says 172.21.15.86:48474) 2024-04-25T14:04:14.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.861+0000 7fa72e800640 1 -- 172.21.15.86:0/3910134931 learned_addr learned my addr 172.21.15.86:0/3910134931 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:14.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.861+0000 7fa72de00640 1 --1- 172.21.15.86:0/3910134931 >> v1:172.21.15.86:6789/0 conn(0x7fa724038310 0x7fa724053670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33482/0 (socket says 172.21.15.86:33482) 2024-04-25T14:04:14.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.865+0000 7f5cfa400640 1 -- 172.21.15.86:0/975825327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cf41610f0 msgr2=0x7f5cf406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.865+0000 7f5cfa400640 1 --2- 172.21.15.86:0/975825327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cf41610f0 0x7f5cf406d780 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5cf40cff90 tx=0x7f5ce401c150 comp rx=0 tx=0).stop 2024-04-25T14:04:14.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.865+0000 7fa72e800640 1 -- 172.21.15.86:0/3910134931 >> v1:172.21.15.155:6789/0 conn(0x7fa724053d80 legacy=0x7fa724006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.865+0000 7fa72e800640 1 -- 172.21.15.86:0/3910134931 >> v1:172.21.15.86:6789/0 conn(0x7fa724038310 legacy=0x7fa724053670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.865+0000 7fa72e800640 1 -- 172.21.15.86:0/3910134931 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa724050350 con 0x7fa724054190 2024-04-25T14:04:14.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.865+0000 7f5cfa400640 1 -- 172.21.15.86:0/975825327 shutdown_connections 2024-04-25T14:04:14.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.865+0000 7f5cfa400640 1 --2- 172.21.15.86:0/975825327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cf41610f0 0x7f5cf406d780 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.865+0000 7f5cfa400640 1 --2- 172.21.15.86:0/975825327 >> v2:172.21.15.86:3300/0 conn(0x7f5cf41607a0 0x7f5cf4160bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.865+0000 7f5cfa400640 1 --2- 172.21.15.86:0/975825327 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cf40b3950 0x7f5cf415cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.865+0000 7f5cfa400640 1 -- 172.21.15.86:0/975825327 >> 172.21.15.86:0/975825327 conn(0x7f5cf409ba30 msgr2=0x7f5cf40b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.865+0000 7f5cfa400640 1 -- 172.21.15.86:0/975825327 shutdown_connections 2024-04-25T14:04:14.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.865+0000 7f5cfa400640 1 -- 172.21.15.86:0/975825327 wait complete. 2024-04-25T14:04:14.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.865+0000 7f71ab200640 1 Processor -- start 2024-04-25T14:04:14.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.865+0000 7f71ab200640 1 -- start start 2024-04-25T14:04:14.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.865+0000 7f3d58a00640 1 Processor -- start 2024-04-25T14:04:14.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.865+0000 7f3d58a00640 1 -- start start 2024-04-25T14:04:14.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.865+0000 7f4a02a00640 1 Processor -- start 2024-04-25T14:04:14.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.865+0000 7f4a02a00640 1 -- start start 2024-04-25T14:04:14.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f3d58a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3d540ceeb0 0x7f3d540d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f3d58a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d540d20e0 con 0x7f3d540cd050 2024-04-25T14:04:14.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f3d58a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d540ce5d0 con 0x7f3d540d1cd0 2024-04-25T14:04:14.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f3d58a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3d540d0a50 con 0x7f3d540ceeb0 2024-04-25T14:04:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f3d52800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3d540cd050 0x7f3d5415baf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33492/0 (socket says 172.21.15.86:33492) 2024-04-25T14:04:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f3d52800640 1 -- 172.21.15.86:0/2907694920 learned_addr learned my addr 172.21.15.86:0/2907694920 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f71ab200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f71a415d830 0x7f71a40d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f71ab200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71a40cf0b0 con 0x7f71a40d1380 2024-04-25T14:04:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f71ab200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71a40cf350 con 0x7f71a4160ab0 2024-04-25T14:04:14.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f71ab200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f71a40cf630 con 0x7f71a415d830 2024-04-25T14:04:14.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f4a02a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49fc06d780 0x7f49fc1f92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f4154e00640 1 Processor -- start 2024-04-25T14:04:14.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f4154e00640 1 -- start start 2024-04-25T14:04:14.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f3d51e00640 1 --2- 172.21.15.86:0/2907694920 >> v2:172.21.15.155:3300/0 conn(0x7f3d540ceeb0 0x7f3d540d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7fa72e800640 1 --2- 172.21.15.86:0/3910134931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa724054190 0x7fa7240075a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa7180098a0 tx=0x7fa71801c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2e8aaf5a6c0c65a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f71a9e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f71a415d830 0x7f71a40d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f4a01600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49fc06d780 0x7f49fc1f92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f4a01600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49fc06d780 0x7f49fc1f92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:48478/0 (socket says 172.21.15.86:48478) 2024-04-25T14:04:14.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f4a02a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49fc0d17c0 0x7f49fc15b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f5cfa400640 1 Processor -- start 2024-04-25T14:04:14.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f71a9e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f71a415d830 0x7f71a40d0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38152/0 (socket says 172.21.15.86:38152) 2024-04-25T14:04:14.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7fa72ca00640 1 -- 172.21.15.86:0/3910134931 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7180b0070 con 0x7fa724054190 2024-04-25T14:04:14.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f71aa800640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f71a4160ab0 0x7f71a415e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:50892/0 (socket says 172.21.15.86:50892) 2024-04-25T14:04:14.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f71aa800640 1 -- 172.21.15.86:0/331578962 learned_addr learned my addr 172.21.15.86:0/331578962 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:14.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f3d51e00640 1 -- 172.21.15.86:0/2907694920 >> v1:172.21.15.155:6789/0 conn(0x7f3d540d1cd0 legacy=0x7f3d540cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f3d51e00640 1 -- 172.21.15.86:0/2907694920 >> v1:172.21.15.86:6789/0 conn(0x7f3d540cd050 legacy=0x7f3d5415baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f3d51e00640 1 -- 172.21.15.86:0/2907694920 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d541639e0 con 0x7f3d540ceeb0 2024-04-25T14:04:14.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7fa72ca00640 1 -- 172.21.15.86:0/3910134931 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7180ab600 con 0x7fa724054190 2024-04-25T14:04:14.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7fa72ca00640 1 -- 172.21.15.86:0/3910134931 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7180b0070 con 0x7fa724054190 2024-04-25T14:04:14.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f5cfa400640 1 -- start start 2024-04-25T14:04:14.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f4a00c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49fc0d17c0 0x7f49fc15b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f4a00c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49fc0d17c0 0x7f49fc15b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:54544/0 (socket says 172.21.15.86:54544) 2024-04-25T14:04:14.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f4a01600640 1 -- 172.21.15.86:0/1649480996 learned_addr learned my addr 172.21.15.86:0/1649480996 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:14.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f4a02a00640 1 --2- 172.21.15.86:0/1649480996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49fc15bbc0 0x7f49fc15bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f4a02a00640 1 -- 172.21.15.86:0/1649480996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc0d1d50 con 0x7f49fc15bbc0 2024-04-25T14:04:14.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f4a02a00640 1 -- 172.21.15.86:0/1649480996 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc0d1bd0 con 0x7f49fc0d17c0 2024-04-25T14:04:14.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.869+0000 7f4a02a00640 1 -- 172.21.15.86:0/1649480996 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f49fc06ebf0 con 0x7f49fc06d780 2024-04-25T14:04:14.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7f4a00c00640 1 -- 172.21.15.86:0/1649480996 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49fc06d780 msgr2=0x7f49fc1f92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7f4a00c00640 1 --2- 172.21.15.86:0/1649480996 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49fc06d780 0x7f49fc1f92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7f4a00c00640 1 -- 172.21.15.86:0/1649480996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49fc15bbc0 msgr2=0x7f49fc15bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7f4a00c00640 1 --2- 172.21.15.86:0/1649480996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49fc15bbc0 0x7f49fc15bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7f4a00c00640 1 -- 172.21.15.86:0/1649480996 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49fc150ab0 con 0x7f49fc0d17c0 2024-04-25T14:04:14.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7fa72f200640 1 -- 172.21.15.86:0/3910134931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa724054190 msgr2=0x7fa7240075a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7fa72f200640 1 --2- 172.21.15.86:0/3910134931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa724054190 0x7fa7240075a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa7180098a0 tx=0x7fa71801c4b0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7fa72f200640 1 -- 172.21.15.86:0/3910134931 shutdown_connections 2024-04-25T14:04:14.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7f4a00c00640 1 --2- 172.21.15.86:0/1649480996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49fc0d17c0 0x7f49fc15b680 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f49e800b410 tx=0x7f49e800b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7fa72f200640 1 --2- 172.21.15.86:0/3910134931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa724054190 0x7fa7240075a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7fa72f200640 1 -- 172.21.15.86:0/3910134931 >> 172.21.15.86:0/3910134931 conn(0x7fa724055150 msgr2=0x7fa724055560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7fa72f200640 1 -- 172.21.15.86:0/3910134931 shutdown_connections 2024-04-25T14:04:14.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7fa72f200640 1 -- 172.21.15.86:0/3910134931 wait complete. 2024-04-25T14:04:14.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7ff166a00640 1 Processor -- start 2024-04-25T14:04:14.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7f71a8a00640 1 -- 172.21.15.86:0/331578962 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2936385629 0 0) 0x7f71a40cf350 con 0x7f71a4160ab0 2024-04-25T14:04:14.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7f71a8a00640 1 -- 172.21.15.86:0/331578962 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f718c003530 con 0x7f71a4160ab0 2024-04-25T14:04:14.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7f49f9600640 1 -- 172.21.15.86:0/1649480996 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e8015020 con 0x7f49fc0d17c0 2024-04-25T14:04:14.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7f4a02a00640 1 -- 172.21.15.86:0/1649480996 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49fc150d90 con 0x7f49fc0d17c0 2024-04-25T14:04:14.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7f4a02a00640 1 -- 172.21.15.86:0/1649480996 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49fc15b160 con 0x7f49fc0d17c0 2024-04-25T14:04:14.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7f71a8a00640 1 -- 172.21.15.86:0/331578962 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3172308815 0 0) 0x7f71a40cf0b0 con 0x7f71a40d1380 2024-04-25T14:04:14.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7ff166a00640 1 -- start start 2024-04-25T14:04:14.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7f71a9e00640 1 -- 172.21.15.86:0/331578962 >> v1:172.21.15.155:6789/0 conn(0x7f71a4160ab0 legacy=0x7f71a415e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7f71a9e00640 1 -- 172.21.15.86:0/331578962 >> v1:172.21.15.86:6789/0 conn(0x7f71a40d1380 legacy=0x7f71a41603a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7f49f9600640 1 -- 172.21.15.86:0/1649480996 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e8011830 con 0x7f49fc0d17c0 2024-04-25T14:04:14.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7f49f9600640 1 -- 172.21.15.86:0/1649480996 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e8011c00 con 0x7f49fc0d17c0 2024-04-25T14:04:14.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7f4a02a00640 1 -- 172.21.15.86:0/1649480996 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d80052a0 con 0x7f49fc0d17c0 2024-04-25T14:04:14.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7f4154e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f41500cf4f0 0x7f4150070890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7f4154e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4150070dd0 0x7f415015fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7f4154e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41501605d0 con 0x7f41501601c0 2024-04-25T14:04:14.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7f4154e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f41500b4c20 con 0x7f41500cf4f0 2024-04-25T14:04:14.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7f4154e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f41500b6c90 con 0x7f4150070dd0 2024-04-25T14:04:14.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7f414de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f41500cf4f0 0x7f4150070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f5cfa400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cf40b3950 0x7f5cf415f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f5cfa400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cf41607a0 0x7f5cf40d24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f5cfa400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cf41610f0 0x7f5cf40d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f5cf2a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cf41607a0 0x7f5cf40d24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f5cf2a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cf41607a0 0x7f5cf40d24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:48492/0 (socket says 172.21.15.86:48492) 2024-04-25T14:04:14.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f5cf2a00640 1 -- 172.21.15.86:0/3990676025 learned_addr learned my addr 172.21.15.86:0/3990676025 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:14.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f5cfa400640 1 -- 172.21.15.86:0/3990676025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf406ca00 con 0x7f5cf41610f0 2024-04-25T14:04:14.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f5cfa400640 1 -- 172.21.15.86:0/3990676025 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf406c880 con 0x7f5cf40b3950 2024-04-25T14:04:14.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f5cfa400640 1 -- 172.21.15.86:0/3990676025 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5cf406cb80 con 0x7f5cf41607a0 2024-04-25T14:04:14.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f5cf3400640 1 --2- 172.21.15.86:0/3990676025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cf40b3950 0x7f5cf415f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f414d400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4150070dd0 0x7f415015fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f414d400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4150070dd0 0x7f415015fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:48486/0 (socket says 172.21.15.86:48486) 2024-04-25T14:04:14.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f414d400640 1 -- 172.21.15.86:0/4106146798 learned_addr learned my addr 172.21.15.86:0/4106146798 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:14.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f414d400640 1 -- 172.21.15.86:0/4106146798 >> v2:172.21.15.86:3300/0 conn(0x7f41500cf4f0 msgr2=0x7f4150070890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f414d400640 1 --2- 172.21.15.86:0/4106146798 >> v2:172.21.15.86:3300/0 conn(0x7f41500cf4f0 0x7f4150070890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f414d400640 1 -- 172.21.15.86:0/4106146798 >> v1:172.21.15.86:6789/0 conn(0x7f41501601c0 legacy=0x7f415006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f414d400640 1 -- 172.21.15.86:0/4106146798 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41500b4ec0 con 0x7f4150070dd0 2024-04-25T14:04:14.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.873+0000 7f71a9e00640 1 -- 172.21.15.86:0/331578962 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a415e0a0 con 0x7f71a415d830 2024-04-25T14:04:14.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f3d51e00640 1 --2- 172.21.15.86:0/2907694920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d540ceeb0 0x7f3d540d1790 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009d90 tx=0x7f3d4c01c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=772c7a8db593d989 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7ff166a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff158057620 0x7ff158057da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7ff166a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff1580582e0 0x7ff1580063b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7ff166a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1580586f0 con 0x7ff1580068f0 2024-04-25T14:04:14.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7ff15f400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff1580582e0 0x7ff1580063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7ff166a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff158007910 con 0x7ff158057620 2024-04-25T14:04:14.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7ff15f400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff1580582e0 0x7ff1580063b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:54568/0 (socket says 172.21.15.86:54568) 2024-04-25T14:04:14.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7ff166a00640 1 -- 172.21.15.86:0/3781596702 --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff158007bf0 con 0x7ff1580582e0 2024-04-25T14:04:14.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7ff15f400640 1 -- 172.21.15.86:0/3781596702 learned_addr learned my addr 172.21.15.86:0/3781596702 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:14.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f5cf2a00640 1 -- 172.21.15.86:0/3990676025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cf40b3950 msgr2=0x7f5cf415f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f5cf2a00640 1 --2- 172.21.15.86:0/3990676025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cf40b3950 0x7f5cf415f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f5cf2a00640 1 -- 172.21.15.86:0/3990676025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cf41610f0 msgr2=0x7f5cf40d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f5cf2a00640 1 --2- 172.21.15.86:0/3990676025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cf41610f0 0x7f5cf40d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f5cf2a00640 1 -- 172.21.15.86:0/3990676025 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf40d1310 con 0x7f5cf41607a0 2024-04-25T14:04:14.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f5cf2a00640 1 --2- 172.21.15.86:0/3990676025 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cf41607a0 0x7f5cf40d24c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5cec00ba80 tx=0x7f5cec00bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f5ceb400640 1 -- 172.21.15.86:0/3990676025 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cec019070 con 0x7f5cf41607a0 2024-04-25T14:04:14.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f5ceb400640 1 -- 172.21.15.86:0/3990676025 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cec00cc90 con 0x7f5cf41607a0 2024-04-25T14:04:14.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f5ceb400640 1 -- 172.21.15.86:0/3990676025 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cec014680 con 0x7f5cf41607a0 2024-04-25T14:04:14.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f5cfa400640 1 -- 172.21.15.86:0/3990676025 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf40d15f0 con 0x7f5cf41607a0 2024-04-25T14:04:14.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f5cfa400640 1 -- 172.21.15.86:0/3990676025 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf40d1fa0 con 0x7f5cf41607a0 2024-04-25T14:04:14.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7fa72f200640 1 Processor -- start 2024-04-25T14:04:14.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f49f9600640 1 -- 172.21.15.86:0/1649480996 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f49e8018020 con 0x7f49fc0d17c0 2024-04-25T14:04:14.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7fa72f200640 1 -- start start 2024-04-25T14:04:14.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7ff15fe00640 1 --2- 172.21.15.86:0/3781596702 >> v2:172.21.15.86:3300/0 conn(0x7ff158057620 0x7ff158057da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7ff15f400640 1 -- 172.21.15.86:0/3781596702 >> v2:172.21.15.86:3300/0 conn(0x7ff158057620 msgr2=0x7ff158057da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.877+0000 7f71a9e00640 1 --2- 172.21.15.86:0/331578962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71a415d830 0x7f71a40d0e40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f71a0009d90 tx=0x7f71a001c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525066c3efd55bb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7f49f9600640 1 --2- 172.21.15.86:0/1649480996 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f49dc06f500 0x7f49dc071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7f49f9600640 1 -- 172.21.15.86:0/1649480996 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f49e808f640 con 0x7f49fc0d17c0 2024-04-25T14:04:14.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7f71a8a00640 1 -- 172.21.15.86:0/331578962 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a00b2070 con 0x7f71a415d830 2024-04-25T14:04:14.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7f71a8a00640 1 -- 172.21.15.86:0/331578962 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71a00a5870 con 0x7f71a415d830 2024-04-25T14:04:14.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7f71a8a00640 1 -- 172.21.15.86:0/331578962 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a00b2070 con 0x7f71a415d830 2024-04-25T14:04:14.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7ff15f400640 1 --2- 172.21.15.86:0/3781596702 >> v2:172.21.15.86:3300/0 conn(0x7ff158057620 0x7ff158057da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7ff15f400640 1 -- 172.21.15.86:0/3781596702 >> v1:172.21.15.155:6789/0 conn(0x7ff1580068f0 legacy=0x7ff158006d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7ff15f400640 1 -- 172.21.15.86:0/3781596702 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff15803cc20 con 0x7ff1580582e0 2024-04-25T14:04:14.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7ff15fe00640 1 --2- 172.21.15.86:0/3781596702 >> v2:172.21.15.86:3300/0 conn(0x7ff158057620 0x7ff158057da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:14.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7f3d50a00640 1 -- 172.21.15.86:0/2907694920 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c0b2070 con 0x7f3d540ceeb0 2024-04-25T14:04:14.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7fa72f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa724038310 0x7fa72404b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7fa72f200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa724053d80 0x7fa7240db410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7fa72f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa724054190 0x7fa724048c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7fa72f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7240f6470 con 0x7fa724038310 2024-04-25T14:04:14.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7fa72f200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa724006970 con 0x7fa724054190 2024-04-25T14:04:14.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7fa72f200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa7240f65f0 con 0x7fa724053d80 2024-04-25T14:04:14.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7f3d50a00640 1 -- 172.21.15.86:0/2907694920 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d4c0a5880 con 0x7f3d540ceeb0 2024-04-25T14:04:14.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7f3d50a00640 1 -- 172.21.15.86:0/2907694920 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c0a5c50 con 0x7f3d540ceeb0 2024-04-25T14:04:14.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7f71ab200640 1 -- 172.21.15.86:0/331578962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71a415d830 msgr2=0x7f71a40d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7f71ab200640 1 --2- 172.21.15.86:0/331578962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71a415d830 0x7f71a40d0e40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f71a0009d90 tx=0x7f71a001c8c0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7f4a01600640 1 --2- 172.21.15.86:0/1649480996 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f49dc06f500 0x7f49dc071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7f49f9600640 1 -- 172.21.15.86:0/1649480996 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f49e801f020 con 0x7f49fc0d17c0 2024-04-25T14:04:14.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7fa72de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa724038310 0x7fa72404b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7f414d400640 1 --2- 172.21.15.86:0/4106146798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4150070dd0 0x7f415015fc80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f413c006fd0 tx=0x7f413c0209c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=417f0342ed634280 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7fa72e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa724054190 0x7fa724048c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7fa72e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa724054190 0x7fa724048c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:54580/0 (socket says 172.21.15.86:54580) 2024-04-25T14:04:14.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7fa72e800640 1 -- 172.21.15.86:0/2356848379 learned_addr learned my addr 172.21.15.86:0/2356848379 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:14.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7f71ab200640 1 -- 172.21.15.86:0/331578962 shutdown_connections 2024-04-25T14:04:14.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7f71ab200640 1 --2- 172.21.15.86:0/331578962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71a415d830 0x7f71a40d0e40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7f71ab200640 1 -- 172.21.15.86:0/331578962 >> 172.21.15.86:0/331578962 conn(0x7f71a406c840 msgr2=0x7f71a406cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7fa72d400640 1 --2- 172.21.15.86:0/2356848379 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa724053d80 0x7fa7240db410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7f5cfa400640 1 -- 172.21.15.86:0/3990676025 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cf40a04c0 con 0x7f5cf41607a0 2024-04-25T14:04:14.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7f414ca00640 1 -- 172.21.15.86:0/4106146798 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c0af070 con 0x7f4150070dd0 2024-04-25T14:04:14.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7f414ca00640 1 -- 172.21.15.86:0/4106146798 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f413c004230 con 0x7f4150070dd0 2024-04-25T14:04:14.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7f414ca00640 1 -- 172.21.15.86:0/4106146798 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c0a97d0 con 0x7f4150070dd0 2024-04-25T14:04:14.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7f71ab200640 1 -- 172.21.15.86:0/331578962 shutdown_connections 2024-04-25T14:04:14.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7f71ab200640 1 -- 172.21.15.86:0/331578962 wait complete. 2024-04-25T14:04:14.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7f3d58a00640 1 -- 172.21.15.86:0/2907694920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d540ceeb0 msgr2=0x7f3d540d1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7f3d58a00640 1 --2- 172.21.15.86:0/2907694920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d540ceeb0 0x7f3d540d1790 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009d90 tx=0x7f3d4c01c820 comp rx=0 tx=0).stop 2024-04-25T14:04:14.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7f3d58a00640 1 -- 172.21.15.86:0/2907694920 shutdown_connections 2024-04-25T14:04:14.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7f3d58a00640 1 --2- 172.21.15.86:0/2907694920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d540ceeb0 0x7f3d540d1790 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009d90 tx=0x7f3d4c01c820 comp rx=0 tx=0).stop 2024-04-25T14:04:14.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7f3d58a00640 1 -- 172.21.15.86:0/2907694920 >> 172.21.15.86:0/2907694920 conn(0x7f3d540b4650 msgr2=0x7f3d540b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7fa72d400640 1 -- 172.21.15.86:0/2356848379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa724054190 msgr2=0x7fa724048c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7fa72d400640 1 --2- 172.21.15.86:0/2356848379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa724054190 0x7fa724048c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7fa72d400640 1 -- 172.21.15.86:0/2356848379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa724038310 msgr2=0x7fa72404b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7fa72d400640 1 --2- 172.21.15.86:0/2356848379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa724038310 0x7fa72404b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7f4a01600640 1 --2- 172.21.15.86:0/1649480996 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f49dc06f500 0x7f49dc071a10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f49f0007fd0 tx=0x7f49f0007f20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7f3d58a00640 1 -- 172.21.15.86:0/2907694920 shutdown_connections 2024-04-25T14:04:14.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7fa72d400640 1 -- 172.21.15.86:0/2356848379 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa724034eb0 con 0x7fa724053d80 2024-04-25T14:04:14.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7fa72de00640 1 --2- 172.21.15.86:0/2356848379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa724038310 0x7fa72404b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:14.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7f3d58a00640 1 -- 172.21.15.86:0/2907694920 wait complete. 2024-04-25T14:04:14.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7f4154e00640 1 -- 172.21.15.86:0/4106146798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4150070dd0 msgr2=0x7f415015fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7f4154e00640 1 --2- 172.21.15.86:0/4106146798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4150070dd0 0x7f415015fc80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f413c006fd0 tx=0x7f413c0209c0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.881+0000 7f5ceb400640 1 -- 172.21.15.86:0/3990676025 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5cec007370 con 0x7f5cf41607a0 2024-04-25T14:04:14.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7ff15f400640 1 --2- 172.21.15.86:0/3781596702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1580582e0 0x7ff1580063b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff1580573a0 tx=0x7ff14c01c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdac4b84fe75b331 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7fa72d400640 1 --2- 172.21.15.86:0/2356848379 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa724053d80 0x7fa7240db410 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa728069f20 tx=0x7fa728069c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7f5ceb400640 1 --2- 172.21.15.86:0/3990676025 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f5cd406f500 0x7f5cd4071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7f5ceb400640 1 -- 172.21.15.86:0/3990676025 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5cec08f1e0 con 0x7f5cf41607a0 2024-04-25T14:04:14.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7f4154e00640 1 -- 172.21.15.86:0/4106146798 shutdown_connections 2024-04-25T14:04:14.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7f4154e00640 1 --2- 172.21.15.86:0/4106146798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4150070dd0 0x7f415015fc80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7f4154e00640 1 --2- 172.21.15.86:0/4106146798 >> v2:172.21.15.86:3300/0 conn(0x7f41500cf4f0 0x7f4150070890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7f4154e00640 1 -- 172.21.15.86:0/4106146798 >> 172.21.15.86:0/4106146798 conn(0x7f41500d2230 msgr2=0x7f41500d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7ff15ea00640 1 -- 172.21.15.86:0/3781596702 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c0b0070 con 0x7ff1580582e0 2024-04-25T14:04:14.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7f4154e00640 1 -- 172.21.15.86:0/4106146798 shutdown_connections 2024-04-25T14:04:14.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7f4154e00640 1 -- 172.21.15.86:0/4106146798 wait complete. 2024-04-25T14:04:14.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7ff15ea00640 1 -- 172.21.15.86:0/3781596702 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff14c0ab600 con 0x7ff1580582e0 2024-04-25T14:04:14.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7ff15ea00640 1 -- 172.21.15.86:0/3781596702 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c0ab9d0 con 0x7ff1580582e0 2024-04-25T14:04:14.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7f5cf3400640 1 --2- 172.21.15.86:0/3990676025 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f5cd406f500 0x7f5cd4071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7f71ab200640 1 Processor -- start 2024-04-25T14:04:14.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7f71ab200640 1 -- start start 2024-04-25T14:04:14.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7f5cf3400640 1 --2- 172.21.15.86:0/3990676025 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f5cd406f500 0x7f5cd4071a10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5cf40cfec0 tx=0x7f5ce0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7ff166a00640 1 -- 172.21.15.86:0/3781596702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1580582e0 msgr2=0x7ff1580063b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7ff166a00640 1 --2- 172.21.15.86:0/3781596702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1580582e0 0x7ff1580063b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff1580573a0 tx=0x7ff14c01c520 comp rx=0 tx=0).stop 2024-04-25T14:04:14.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7ff166a00640 1 -- 172.21.15.86:0/3781596702 shutdown_connections 2024-04-25T14:04:14.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7ff166a00640 1 --2- 172.21.15.86:0/3781596702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1580582e0 0x7ff1580063b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7ff166a00640 1 --2- 172.21.15.86:0/3781596702 >> v2:172.21.15.86:3300/0 conn(0x7ff158057620 0x7ff158057da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7ff166a00640 1 -- 172.21.15.86:0/3781596702 >> 172.21.15.86:0/3781596702 conn(0x7ff15803ac60 msgr2=0x7ff15803b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7ff166a00640 1 -- 172.21.15.86:0/3781596702 shutdown_connections 2024-04-25T14:04:14.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.885+0000 7ff166a00640 1 -- 172.21.15.86:0/3781596702 wait complete. 2024-04-25T14:04:14.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7f3d58a00640 1 Processor -- start 2024-04-25T14:04:14.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7fa722000640 1 -- 172.21.15.86:0/2356848379 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa728069440 con 0x7fa724053d80 2024-04-25T14:04:14.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7fa722000640 1 -- 172.21.15.86:0/2356848379 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa728090030 con 0x7fa724053d80 2024-04-25T14:04:14.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7fa722000640 1 -- 172.21.15.86:0/2356848379 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa728075560 con 0x7fa724053d80 2024-04-25T14:04:14.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7f71ab200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71a40d1380 0x7f71a40af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7f71ab200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71a415d830 0x7f71a41fd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7f71ab200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71a4160ab0 0x7f71a41fd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7f71ab200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71a415dc40 con 0x7f71a40d1380 2024-04-25T14:04:14.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7f71ab200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f71a40b7c40 con 0x7f71a415d830 2024-04-25T14:04:14.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7f71ab200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f71a41f2d60 con 0x7f71a4160ab0 2024-04-25T14:04:14.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7f3d58a00640 1 -- start start 2024-04-25T14:04:14.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7f3d58a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d540cd050 0x7f3d540c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7f3d58a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d540d1cd0 0x7f3d540b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7f3d58a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d5409a240 0x7f3d540b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7f3d58a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3d54070820 con 0x7f3d540d1cd0 2024-04-25T14:04:14.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7f3d58a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5415d210 con 0x7f3d540cd050 2024-04-25T14:04:14.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7f3d58a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3d540cf2c0 con 0x7f3d5409a240 2024-04-25T14:04:14.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7f4154e00640 1 Processor -- start 2024-04-25T14:04:14.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7f4154e00640 1 -- start start 2024-04-25T14:04:14.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7ff166a00640 1 Processor -- start 2024-04-25T14:04:14.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7fa72f200640 1 -- 172.21.15.86:0/2356848379 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa724035190 con 0x7fa724053d80 2024-04-25T14:04:14.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7ff166a00640 1 -- start start 2024-04-25T14:04:14.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7f3d52800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d5409a240 0x7f3d540b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7fa72f200640 1 -- 172.21.15.86:0/2356848379 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa724035750 con 0x7fa724053d80 2024-04-25T14:04:14.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7f3d52800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d5409a240 0x7f3d540b2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:48518/0 (socket says 172.21.15.86:48518) 2024-04-25T14:04:14.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7f3d52800640 1 -- 172.21.15.86:0/725483188 learned_addr learned my addr 172.21.15.86:0/725483188 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:14.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7f5ceb400640 1 -- 172.21.15.86:0/3990676025 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5cec05be00 con 0x7f5cf41607a0 2024-04-25T14:04:14.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7f3d51e00640 1 --2- 172.21.15.86:0/725483188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d540cd050 0x7f3d540c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7f3d51e00640 1 -- 172.21.15.86:0/725483188 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d5409a240 msgr2=0x7f3d540b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7f3d51e00640 1 --2- 172.21.15.86:0/725483188 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d5409a240 0x7f3d540b2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7f3d51e00640 1 -- 172.21.15.86:0/725483188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d540d1cd0 msgr2=0x7f3d540b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7f3d51e00640 1 --2- 172.21.15.86:0/725483188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d540d1cd0 0x7f3d540b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.889+0000 7f3d51e00640 1 -- 172.21.15.86:0/725483188 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d540b1550 con 0x7f3d540cd050 2024-04-25T14:04:14.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7fa72f200640 1 -- 172.21.15.86:0/2356848379 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa724034c20 con 0x7fa724053d80 2024-04-25T14:04:14.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7ff166a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1580068f0 0x7ff1580e67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f71a9400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71a415d830 0x7f71a41fd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f71a9400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71a415d830 0x7f71a41fd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:54600/0 (socket says 172.21.15.86:54600) 2024-04-25T14:04:14.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f71a9400640 1 -- 172.21.15.86:0/3944133268 learned_addr learned my addr 172.21.15.86:0/3944133268 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:14.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f71a9e00640 1 --2- 172.21.15.86:0/3944133268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71a40d1380 0x7f71a40af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f71a9400640 1 -- 172.21.15.86:0/3944133268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71a4160ab0 msgr2=0x7f71a41fd740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:04:14.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f71a9400640 1 --2- 172.21.15.86:0/3944133268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71a4160ab0 0x7f71a41fd740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f71a9400640 1 -- 172.21.15.86:0/3944133268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71a40d1380 msgr2=0x7f71a40af540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f71a9400640 1 --2- 172.21.15.86:0/3944133268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71a40d1380 0x7f71a40af540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f71a9400640 1 -- 172.21.15.86:0/3944133268 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a40b0b50 con 0x7f71a415d830 2024-04-25T14:04:14.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f71a9400640 1 --2- 172.21.15.86:0/3944133268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71a415d830 0x7f71a41fd200 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f719800c410 tx=0x7f719800c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7ff166a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff158057620 0x7ff1580dbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7ff166a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1580582e0 0x7ff1580dc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f719e000640 1 -- 172.21.15.86:0/3944133268 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7198017070 con 0x7f71a415d830 2024-04-25T14:04:14.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f719e000640 1 -- 172.21.15.86:0/3944133268 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71980052d0 con 0x7f71a415d830 2024-04-25T14:04:14.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f719e000640 1 -- 172.21.15.86:0/3944133268 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7198003860 con 0x7f71a415d830 2024-04-25T14:04:14.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f4154e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4150070dd0 0x7f41501f24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f4154e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41500cf4f0 0x7f41501f1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f414de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4150070dd0 0x7f41501f24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f3d51e00640 1 --2- 172.21.15.86:0/725483188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d540cd050 0x7f3d540c7690 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0ac000 tx=0x7f3d4c0195c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f414de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4150070dd0 0x7f41501f24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38236/0 (socket says 172.21.15.86:38236) 2024-04-25T14:04:14.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7fa722000640 1 -- 172.21.15.86:0/2356848379 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa728075700 con 0x7fa724053d80 2024-04-25T14:04:14.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7ff15f400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff158057620 0x7ff1580dbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7ff166a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff1580f0d90 con 0x7ff1580068f0 2024-04-25T14:04:14.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7ff166a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff1580f0c10 con 0x7ff1580582e0 2024-04-25T14:04:14.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7ff166a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff1580f0f10 con 0x7ff158057620 2024-04-25T14:04:14.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7ff15fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1580068f0 0x7ff1580e67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7ff15fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1580068f0 0x7ff1580e67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38220/0 (socket says 172.21.15.86:38220) 2024-04-25T14:04:14.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7ff15fe00640 1 -- 172.21.15.86:0/2345707604 learned_addr learned my addr 172.21.15.86:0/2345707604 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:14.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f4154e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41501601c0 0x7f415013ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f414de00640 1 -- 172.21.15.86:0/4185046603 learned_addr learned my addr 172.21.15.86:0/4185046603 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:14.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f4154e00640 1 -- 172.21.15.86:0/4185046603 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4150145ce0 con 0x7f4150070dd0 2024-04-25T14:04:14.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f4154e00640 1 -- 172.21.15.86:0/4185046603 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4150145b60 con 0x7f41501601c0 2024-04-25T14:04:14.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f4154e00640 1 -- 172.21.15.86:0/4185046603 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f415006cd80 con 0x7f41500cf4f0 2024-04-25T14:04:14.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f414e800640 1 --2- 172.21.15.86:0/4185046603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41501601c0 0x7f415013ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f414d400640 1 --2- 172.21.15.86:0/4185046603 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41500cf4f0 0x7f41501f1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.897+0000 7f414de00640 1 -- 172.21.15.86:0/4185046603 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41500cf4f0 msgr2=0x7f41501f1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.897+0000 7f414de00640 1 --2- 172.21.15.86:0/4185046603 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41500cf4f0 0x7f41501f1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.897+0000 7f414de00640 1 -- 172.21.15.86:0/4185046603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41501601c0 msgr2=0x7f415013ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.897+0000 7f414de00640 1 --2- 172.21.15.86:0/4185046603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41501601c0 0x7f415013ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.897+0000 7f414de00640 1 -- 172.21.15.86:0/4185046603 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41501f1e50 con 0x7f4150070dd0 2024-04-25T14:04:14.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.897+0000 7f414d400640 1 --2- 172.21.15.86:0/4185046603 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41500cf4f0 0x7f41501f1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:14.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.897+0000 7f414e800640 1 --2- 172.21.15.86:0/4185046603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41501601c0 0x7f415013ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:14.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f71ab200640 1 -- 172.21.15.86:0/3944133268 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a40b02b0 con 0x7f71a415d830 2024-04-25T14:04:14.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f71ab200640 1 -- 172.21.15.86:0/3944133268 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a40b0870 con 0x7f71a415d830 2024-04-25T14:04:14.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f3d4a000640 1 -- 172.21.15.86:0/725483188 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c0b2040 con 0x7f3d540cd050 2024-04-25T14:04:14.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f3d4a000640 1 -- 172.21.15.86:0/725483188 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d4c0aeb00 con 0x7f3d540cd050 2024-04-25T14:04:14.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f3d4a000640 1 -- 172.21.15.86:0/725483188 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c0b8470 con 0x7f3d540cd050 2024-04-25T14:04:14.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f3d58a00640 1 -- 172.21.15.86:0/725483188 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d540b3a60 con 0x7f3d540cd050 2024-04-25T14:04:14.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7f3d58a00640 1 -- 172.21.15.86:0/725483188 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d540b11e0 con 0x7f3d540cd050 2024-04-25T14:04:14.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7ff164a00640 1 --2- 172.21.15.86:0/2345707604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1580582e0 0x7ff1580dc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7ff15f400640 1 -- 172.21.15.86:0/2345707604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1580582e0 msgr2=0x7ff1580dc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7ff15f400640 1 --2- 172.21.15.86:0/2345707604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1580582e0 0x7ff1580dc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7ff15f400640 1 -- 172.21.15.86:0/2345707604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1580068f0 msgr2=0x7ff1580e67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7ff15f400640 1 --2- 172.21.15.86:0/2345707604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1580068f0 0x7ff1580e67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.897+0000 7ff15f400640 1 -- 172.21.15.86:0/2345707604 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff158053390 con 0x7ff158057620 2024-04-25T14:04:14.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.897+0000 7ff15fe00640 1 --2- 172.21.15.86:0/2345707604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1580068f0 0x7ff1580e67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:14.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.897+0000 7ff15f400640 1 --2- 172.21.15.86:0/2345707604 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff158057620 0x7ff1580dbfb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff14c0ab000 tx=0x7ff14c0192e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.897+0000 7f414de00640 1 --2- 172.21.15.86:0/4185046603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4150070dd0 0x7f41501f24f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f414000c7f0 tx=0x7f414000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7fa722000640 1 --2- 172.21.15.86:0/2356848379 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa70c06f5d0 0x7fa70c071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.893+0000 7fa722000640 1 -- 172.21.15.86:0/2356848379 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa72810b430 con 0x7fa724053d80 2024-04-25T14:04:14.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.897+0000 7fa72de00640 1 --2- 172.21.15.86:0/2356848379 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa70c06f5d0 0x7fa70c071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.897+0000 7fa72de00640 1 --2- 172.21.15.86:0/2356848379 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa70c06f5d0 0x7fa70c071ae0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa71c000f70 tx=0x7fa71c007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.897+0000 7f719e000640 1 -- 172.21.15.86:0/3944133268 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f719801a070 con 0x7f71a415d830 2024-04-25T14:04:14.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.897+0000 7f4146000640 1 -- 172.21.15.86:0/4185046603 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4140004230 con 0x7f4150070dd0 2024-04-25T14:04:14.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.897+0000 7f3d4a000640 1 -- 172.21.15.86:0/725483188 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3d4c0c1020 con 0x7f3d540cd050 2024-04-25T14:04:14.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.897+0000 7f4146000640 1 -- 172.21.15.86:0/4185046603 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41400043d0 con 0x7f4150070dd0 2024-04-25T14:04:14.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.897+0000 7f4154e00640 1 -- 172.21.15.86:0/4185046603 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41501f0870 con 0x7f4150070dd0 2024-04-25T14:04:14.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.897+0000 7f4154e00640 1 -- 172.21.15.86:0/4185046603 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41501f0e30 con 0x7f4150070dd0 2024-04-25T14:04:14.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.897+0000 7f71ab200640 1 -- 172.21.15.86:0/3944133268 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71a4076b80 con 0x7f71a415d830 2024-04-25T14:04:14.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.901+0000 7f4146000640 1 -- 172.21.15.86:0/4185046603 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4140003860 con 0x7f4150070dd0 2024-04-25T14:04:14.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.901+0000 7f719e000640 1 --2- 172.21.15.86:0/3944133268 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f718806f500 0x7f7188071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.901+0000 7ff153e00640 1 -- 172.21.15.86:0/2345707604 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c0b0050 con 0x7ff158057620 2024-04-25T14:04:14.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.901+0000 7ff153e00640 1 -- 172.21.15.86:0/2345707604 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff14c0ba070 con 0x7ff158057620 2024-04-25T14:04:14.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.901+0000 7ff153e00640 1 -- 172.21.15.86:0/2345707604 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c0b4a10 con 0x7ff158057620 2024-04-25T14:04:14.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.901+0000 7fa722000640 1 -- 172.21.15.86:0/2356848379 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa7280d7980 con 0x7fa724053d80 2024-04-25T14:04:14.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.901+0000 7ff166a00640 1 -- 172.21.15.86:0/2345707604 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff158050350 con 0x7ff158057620 2024-04-25T14:04:14.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.901+0000 7ff166a00640 1 -- 172.21.15.86:0/2345707604 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1580508d0 con 0x7ff158057620 2024-04-25T14:04:14.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.901+0000 7f3d58a00640 1 -- 172.21.15.86:0/725483188 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d5409f4b0 con 0x7f3d540cd050 2024-04-25T14:04:14.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.901+0000 7f3d4a000640 1 --2- 172.21.15.86:0/725483188 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3d3406f400 0x7f3d34071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.901+0000 7f3d4a000640 1 -- 172.21.15.86:0/725483188 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3d4c12a170 con 0x7f3d540cd050 2024-04-25T14:04:14.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.901+0000 7f719e000640 1 -- 172.21.15.86:0/3944133268 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f719808e8b0 con 0x7f71a415d830 2024-04-25T14:04:14.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.901+0000 7f3d51400640 1 --2- 172.21.15.86:0/725483188 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3d3406f400 0x7f3d34071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.901+0000 7f71a9e00640 1 --2- 172.21.15.86:0/3944133268 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f718806f500 0x7f7188071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.901+0000 7ff166a00640 1 -- 172.21.15.86:0/2345707604 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff13c0052a0 con 0x7ff158057620 2024-04-25T14:04:14.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.901+0000 7f413b400640 1 -- 172.21.15.86:0/4185046603 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41280052a0 con 0x7f4150070dd0 2024-04-25T14:04:14.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.905+0000 7f3d51400640 1 --2- 172.21.15.86:0/725483188 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3d3406f400 0x7f3d34071910 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3d541573b0 tx=0x7f3d40001ca0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.905+0000 7f4146000640 1 -- 172.21.15.86:0/4185046603 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4140003c10 con 0x7f4150070dd0 2024-04-25T14:04:14.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.905+0000 7f4146000640 1 --2- 172.21.15.86:0/4185046603 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f413006f4d0 0x7f41300719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.905+0000 7f4146000640 1 -- 172.21.15.86:0/4185046603 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f414008fb30 con 0x7f4150070dd0 2024-04-25T14:04:14.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.905+0000 7ff153e00640 1 -- 172.21.15.86:0/2345707604 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff14c0ab4c0 con 0x7ff158057620 2024-04-25T14:04:14.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.905+0000 7f71a9e00640 1 --2- 172.21.15.86:0/3944133268 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f718806f500 0x7f7188071a10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f71a00af000 tx=0x7f71a00a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.905+0000 7f414d400640 1 --2- 172.21.15.86:0/4185046603 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f413006f4d0 0x7f41300719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.905+0000 7f719e000640 1 -- 172.21.15.86:0/3944133268 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7198005480 con 0x7f71a415d830 2024-04-25T14:04:14.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.905+0000 7ff153e00640 1 --2- 172.21.15.86:0/2345707604 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff14006f400 0x7ff140071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.905+0000 7ff153e00640 1 -- 172.21.15.86:0/2345707604 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff14c129d40 con 0x7ff158057620 2024-04-25T14:04:14.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.909+0000 7f414d400640 1 --2- 172.21.15.86:0/4185046603 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f413006f4d0 0x7f41300719e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f413c0b3000 tx=0x7f413c0ab070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.909+0000 7ff15fe00640 1 --2- 172.21.15.86:0/2345707604 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff14006f400 0x7ff140071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.909+0000 7ff153e00640 1 -- 172.21.15.86:0/2345707604 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff14c016130 con 0x7ff158057620 2024-04-25T14:04:14.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.909+0000 7ff15fe00640 1 --2- 172.21.15.86:0/2345707604 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff14006f400 0x7ff140071910 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff154006fd0 tx=0x7ff154008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.909+0000 7f3d4a000640 1 -- 172.21.15.86:0/725483188 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3d4c0c8020 con 0x7f3d540cd050 2024-04-25T14:04:14.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:14.917+0000 7f4146000640 1 -- 172.21.15.86:0/4185046603 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4140015020 con 0x7f4150070dd0 2024-04-25T14:04:15.165 INFO:teuthology.orchestra.run.smithi086.stdout:55834574858 2024-04-25T14:04:15.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.161+0000 7fd604a00640 1 -- 172.21.15.86:0/771939484 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fd6001fa0d0 con 0x7fd60015f980 2024-04-25T14:04:15.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.161+0000 7fd5f6000640 1 -- 172.21.15.86:0/771939484 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd5ec005480 con 0x7fd60015f980 2024-04-25T14:04:15.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.165+0000 7fd604a00640 1 -- 172.21.15.86:0/771939484 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd5e006f500 msgr2=0x7fd5e0071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.165+0000 7fd604a00640 1 --2- 172.21.15.86:0/771939484 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd5e006f500 0x7fd5e0071a10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009580 tx=0x7fd5f0009190 comp rx=0 tx=0).stop 2024-04-25T14:04:15.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.165+0000 7fd604a00640 1 -- 172.21.15.86:0/771939484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd60015f980 msgr2=0x7fd6001fc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.165+0000 7fd604a00640 1 --2- 172.21.15.86:0/771939484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd60015f980 0x7fd6001fc560 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00c3c0 tx=0x7fd5ec00c880 comp rx=0 tx=0).stop 2024-04-25T14:04:15.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.165+0000 7fd604a00640 1 -- 172.21.15.86:0/771939484 shutdown_connections 2024-04-25T14:04:15.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.165+0000 7fd604a00640 1 --2- 172.21.15.86:0/771939484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd600160670 0x7fd6001620c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.165+0000 7fd604a00640 1 --2- 172.21.15.86:0/771939484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd60015f980 0x7fd6001fc560 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.165+0000 7fd604a00640 1 --2- 172.21.15.86:0/771939484 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd5e006f500 0x7fd5e0071a10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.165+0000 7fd604a00640 1 --2- 172.21.15.86:0/771939484 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6000ce100 0x7fd6001f9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.165+0000 7fd604a00640 1 -- 172.21.15.86:0/771939484 >> 172.21.15.86:0/771939484 conn(0x7fd6000b6890 msgr2=0x7fd60006e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:15.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.165+0000 7fd604a00640 1 -- 172.21.15.86:0/771939484 shutdown_connections 2024-04-25T14:04:15.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.165+0000 7fd604a00640 1 -- 172.21.15.86:0/771939484 wait complete. 2024-04-25T14:04:15.200 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.4 2024-04-25T14:04:15.200 DEBUG:teuthology.parallel:result is None 2024-04-25T14:04:15.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.225+0000 7f4a02a00640 1 -- 172.21.15.86:0/1649480996 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f49d8005860 con 0x7f49fc0d17c0 2024-04-25T14:04:15.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.225+0000 7ff166a00640 1 -- 172.21.15.86:0/2345707604 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ff13c005860 con 0x7ff158057620 2024-04-25T14:04:15.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.225+0000 7f49f9600640 1 -- 172.21.15.86:0/1649480996 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f49e805c1d0 con 0x7f49fc0d17c0 2024-04-25T14:04:15.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.225+0000 7ff153e00640 1 -- 172.21.15.86:0/2345707604 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff14c0f63a0 con 0x7ff158057620 2024-04-25T14:04:15.226 INFO:teuthology.orchestra.run.smithi086.stdout:55834574859 2024-04-25T14:04:15.228 INFO:teuthology.orchestra.run.smithi086.stdout:55834574859 2024-04-25T14:04:15.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.225+0000 7ff166a00640 1 -- 172.21.15.86:0/2345707604 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff14006f400 msgr2=0x7ff140071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.225+0000 7ff166a00640 1 --2- 172.21.15.86:0/2345707604 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff14006f400 0x7ff140071910 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff154006fd0 tx=0x7ff154008040 comp rx=0 tx=0).stop 2024-04-25T14:04:15.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.225+0000 7ff166a00640 1 -- 172.21.15.86:0/2345707604 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff158057620 msgr2=0x7ff1580dbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.225+0000 7ff166a00640 1 --2- 172.21.15.86:0/2345707604 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff158057620 0x7ff1580dbfb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff14c0ab000 tx=0x7ff14c0192e0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.225+0000 7ff166a00640 1 -- 172.21.15.86:0/2345707604 shutdown_connections 2024-04-25T14:04:15.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.229+0000 7ff166a00640 1 --2- 172.21.15.86:0/2345707604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1580582e0 0x7ff1580dc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.229+0000 7ff166a00640 1 --2- 172.21.15.86:0/2345707604 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff14006f400 0x7ff140071910 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.229+0000 7ff166a00640 1 --2- 172.21.15.86:0/2345707604 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff158057620 0x7ff1580dbfb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.229+0000 7ff166a00640 1 --2- 172.21.15.86:0/2345707604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1580068f0 0x7ff1580e67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.229+0000 7ff166a00640 1 -- 172.21.15.86:0/2345707604 >> 172.21.15.86:0/2345707604 conn(0x7ff15803ac60 msgr2=0x7ff158054490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:15.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.229+0000 7ff166a00640 1 -- 172.21.15.86:0/2345707604 shutdown_connections 2024-04-25T14:04:15.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.229+0000 7ff166a00640 1 -- 172.21.15.86:0/2345707604 wait complete. 2024-04-25T14:04:15.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.229+0000 7f4a02a00640 1 -- 172.21.15.86:0/1649480996 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f49dc06f500 msgr2=0x7f49dc071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.229+0000 7f4a02a00640 1 --2- 172.21.15.86:0/1649480996 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f49dc06f500 0x7f49dc071a10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f49f0007fd0 tx=0x7f49f0007f20 comp rx=0 tx=0).stop 2024-04-25T14:04:15.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.229+0000 7f4a02a00640 1 -- 172.21.15.86:0/1649480996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49fc0d17c0 msgr2=0x7f49fc15b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.229+0000 7f4a02a00640 1 --2- 172.21.15.86:0/1649480996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49fc0d17c0 0x7f49fc15b680 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f49e800b410 tx=0x7f49e800b8d0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.229+0000 7f4a02a00640 1 -- 172.21.15.86:0/1649480996 shutdown_connections 2024-04-25T14:04:15.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.229+0000 7f4a02a00640 1 --2- 172.21.15.86:0/1649480996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49fc15bbc0 0x7f49fc15bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.229+0000 7f4a02a00640 1 --2- 172.21.15.86:0/1649480996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49fc0d17c0 0x7f49fc15b680 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.229+0000 7f4a02a00640 1 --2- 172.21.15.86:0/1649480996 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f49dc06f500 0x7f49dc071a10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.229+0000 7f4a02a00640 1 --2- 172.21.15.86:0/1649480996 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49fc06d780 0x7f49fc1f92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.229+0000 7f4a02a00640 1 -- 172.21.15.86:0/1649480996 >> 172.21.15.86:0/1649480996 conn(0x7f49fc073c00 msgr2=0x7f49fc0b77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:15.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.229+0000 7f4a02a00640 1 -- 172.21.15.86:0/1649480996 shutdown_connections 2024-04-25T14:04:15.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.229+0000 7f4a02a00640 1 -- 172.21.15.86:0/1649480996 wait complete. 2024-04-25T14:04:15.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.237+0000 7f5cfa400640 1 -- 172.21.15.86:0/3990676025 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f5cf4076bf0 con 0x7f5cf41607a0 2024-04-25T14:04:15.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.241+0000 7f5ceb400640 1 -- 172.21.15.86:0/3990676025 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5cec05b700 con 0x7f5cf41607a0 2024-04-25T14:04:15.245 INFO:teuthology.orchestra.run.smithi086.stdout:55834574858 2024-04-25T14:04:15.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.245+0000 7f5cfa400640 1 -- 172.21.15.86:0/3990676025 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f5cd406f500 msgr2=0x7f5cd4071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.245+0000 7f5cfa400640 1 --2- 172.21.15.86:0/3990676025 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f5cd406f500 0x7f5cd4071a10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5cf40cfec0 tx=0x7f5ce0008040 comp rx=0 tx=0).stop 2024-04-25T14:04:15.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.245+0000 7f5cfa400640 1 -- 172.21.15.86:0/3990676025 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cf41607a0 msgr2=0x7f5cf40d24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.245+0000 7f5cfa400640 1 --2- 172.21.15.86:0/3990676025 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cf41607a0 0x7f5cf40d24c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5cec00ba80 tx=0x7f5cec00bf40 comp rx=0 tx=0).stop 2024-04-25T14:04:15.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.245+0000 7f5cfa400640 1 -- 172.21.15.86:0/3990676025 shutdown_connections 2024-04-25T14:04:15.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.245+0000 7f5cfa400640 1 --2- 172.21.15.86:0/3990676025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cf41610f0 0x7f5cf40d2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.245+0000 7f5cfa400640 1 --2- 172.21.15.86:0/3990676025 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f5cd406f500 0x7f5cd4071a10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.245+0000 7f5cfa400640 1 --2- 172.21.15.86:0/3990676025 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cf41607a0 0x7f5cf40d24c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.245+0000 7f5cfa400640 1 --2- 172.21.15.86:0/3990676025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cf40b3950 0x7f5cf415f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.245+0000 7f5cfa400640 1 -- 172.21.15.86:0/3990676025 >> 172.21.15.86:0/3990676025 conn(0x7f5cf409ba30 msgr2=0x7f5cf415c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:15.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.245+0000 7f71ab200640 1 -- 172.21.15.86:0/3944133268 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f71a40b7250 con 0x7f71a415d830 2024-04-25T14:04:15.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.245+0000 7f5cfa400640 1 -- 172.21.15.86:0/3990676025 shutdown_connections 2024-04-25T14:04:15.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.245+0000 7f5cfa400640 1 -- 172.21.15.86:0/3990676025 wait complete. 2024-04-25T14:04:15.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.245+0000 7f719e000640 1 -- 172.21.15.86:0/3944133268 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f719805adf0 con 0x7f71a415d830 2024-04-25T14:04:15.248 INFO:teuthology.orchestra.run.smithi086.stdout:55834574858 2024-04-25T14:04:15.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.245+0000 7f3d58a00640 1 -- 172.21.15.86:0/725483188 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f3d54069d00 con 0x7f3d540cd050 2024-04-25T14:04:15.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.249+0000 7f3d4a000640 1 -- 172.21.15.86:0/725483188 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3d4c0164a0 con 0x7f3d540cd050 2024-04-25T14:04:15.251 INFO:teuthology.orchestra.run.smithi086.stdout:55834574857 2024-04-25T14:04:15.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.249+0000 7f71ab200640 1 -- 172.21.15.86:0/3944133268 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f718806f500 msgr2=0x7f7188071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.249+0000 7f71ab200640 1 --2- 172.21.15.86:0/3944133268 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f718806f500 0x7f7188071a10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f71a00af000 tx=0x7f71a00a7070 comp rx=0 tx=0).stop 2024-04-25T14:04:15.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.249+0000 7f71ab200640 1 -- 172.21.15.86:0/3944133268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71a415d830 msgr2=0x7f71a41fd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.249+0000 7f71ab200640 1 --2- 172.21.15.86:0/3944133268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71a415d830 0x7f71a41fd200 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f719800c410 tx=0x7f719800c8d0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.249+0000 7f71ab200640 1 -- 172.21.15.86:0/3944133268 shutdown_connections 2024-04-25T14:04:15.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.249+0000 7f71ab200640 1 --2- 172.21.15.86:0/3944133268 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f718806f500 0x7f7188071a10 secure :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f71a00af000 tx=0x7f71a00a7070 comp rx=0 tx=0).stop 2024-04-25T14:04:15.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.249+0000 7f71ab200640 1 --2- 172.21.15.86:0/3944133268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71a4160ab0 0x7f71a41fd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.249+0000 7f71ab200640 1 --2- 172.21.15.86:0/3944133268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71a415d830 0x7f71a41fd200 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.249+0000 7f71ab200640 1 --2- 172.21.15.86:0/3944133268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71a40d1380 0x7f71a40af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.249+0000 7f71ab200640 1 -- 172.21.15.86:0/3944133268 >> 172.21.15.86:0/3944133268 conn(0x7f71a406c840 msgr2=0x7f71a40b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:15.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.249+0000 7f71ab200640 1 -- 172.21.15.86:0/3944133268 shutdown_connections 2024-04-25T14:04:15.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.249+0000 7f71ab200640 1 -- 172.21.15.86:0/3944133268 wait complete. 2024-04-25T14:04:15.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.253+0000 7fa72f200640 1 -- 172.21.15.86:0/2356848379 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fa7240059d0 con 0x7fa724053d80 2024-04-25T14:04:15.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.253+0000 7fa722000640 1 -- 172.21.15.86:0/2356848379 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa7280dbc80 con 0x7fa724053d80 2024-04-25T14:04:15.254 INFO:teuthology.orchestra.run.smithi086.stdout:55834574857 2024-04-25T14:04:15.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.253+0000 7f3d58a00640 1 -- 172.21.15.86:0/725483188 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3d3406f400 msgr2=0x7f3d34071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.253+0000 7f3d58a00640 1 --2- 172.21.15.86:0/725483188 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3d3406f400 0x7f3d34071910 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3d541573b0 tx=0x7f3d40001ca0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.253+0000 7f3d58a00640 1 -- 172.21.15.86:0/725483188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d540cd050 msgr2=0x7f3d540c7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.253+0000 7f3d58a00640 1 --2- 172.21.15.86:0/725483188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d540cd050 0x7f3d540c7690 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0ac000 tx=0x7f3d4c0195c0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.253+0000 7f3d58a00640 1 -- 172.21.15.86:0/725483188 shutdown_connections 2024-04-25T14:04:15.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.253+0000 7f3d58a00640 1 --2- 172.21.15.86:0/725483188 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3d3406f400 0x7f3d34071910 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.253+0000 7f3d58a00640 1 --2- 172.21.15.86:0/725483188 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d5409a240 0x7f3d540b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.253+0000 7f3d58a00640 1 --2- 172.21.15.86:0/725483188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d540d1cd0 0x7f3d540b2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.253+0000 7f3d58a00640 1 --2- 172.21.15.86:0/725483188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d540cd050 0x7f3d540c7690 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.253+0000 7f3d58a00640 1 -- 172.21.15.86:0/725483188 >> 172.21.15.86:0/725483188 conn(0x7f3d540b4650 msgr2=0x7f3d540afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:15.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.253+0000 7f3d58a00640 1 -- 172.21.15.86:0/725483188 shutdown_connections 2024-04-25T14:04:15.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.253+0000 7f3d58a00640 1 -- 172.21.15.86:0/725483188 wait complete. 2024-04-25T14:04:15.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.257+0000 7f413b400640 1 -- 172.21.15.86:0/4185046603 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f4128005860 con 0x7f4150070dd0 2024-04-25T14:04:15.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.257+0000 7fa72f200640 1 -- 172.21.15.86:0/2356848379 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa70c06f5d0 msgr2=0x7fa70c071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.257+0000 7fa72f200640 1 --2- 172.21.15.86:0/2356848379 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa70c06f5d0 0x7fa70c071ae0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa71c000f70 tx=0x7fa71c007d50 comp rx=0 tx=0).stop 2024-04-25T14:04:15.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.257+0000 7fa72f200640 1 -- 172.21.15.86:0/2356848379 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa724053d80 msgr2=0x7fa7240db410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.257+0000 7fa72f200640 1 --2- 172.21.15.86:0/2356848379 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa724053d80 0x7fa7240db410 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa728069f20 tx=0x7fa728069c40 comp rx=0 tx=0).stop 2024-04-25T14:04:15.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.257+0000 7f4146000640 1 -- 172.21.15.86:0/4185046603 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f414005c100 con 0x7f4150070dd0 2024-04-25T14:04:15.259 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.1 2024-04-25T14:04:15.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.257+0000 7fa72f200640 1 -- 172.21.15.86:0/2356848379 shutdown_connections 2024-04-25T14:04:15.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.257+0000 7fa72f200640 1 --2- 172.21.15.86:0/2356848379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa724054190 0x7fa724048c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.257+0000 7fa72f200640 1 --2- 172.21.15.86:0/2356848379 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa70c06f5d0 0x7fa70c071ae0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.257+0000 7fa72f200640 1 --2- 172.21.15.86:0/2356848379 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa724053d80 0x7fa7240db410 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.257+0000 7fa72f200640 1 --2- 172.21.15.86:0/2356848379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa724038310 0x7fa72404b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.257+0000 7fa72f200640 1 -- 172.21.15.86:0/2356848379 >> 172.21.15.86:0/2356848379 conn(0x7fa724055150 msgr2=0x7fa7240f2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:15.260 INFO:teuthology.orchestra.run.smithi086.stdout:55834574858 2024-04-25T14:04:15.260 DEBUG:teuthology.parallel:result is None 2024-04-25T14:04:15.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.257+0000 7fa72f200640 1 -- 172.21.15.86:0/2356848379 shutdown_connections 2024-04-25T14:04:15.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.257+0000 7fa72f200640 1 -- 172.21.15.86:0/2356848379 wait complete. 2024-04-25T14:04:15.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.261+0000 7f413b400640 1 -- 172.21.15.86:0/4185046603 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f413006f4d0 msgr2=0x7f41300719e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.261+0000 7f413b400640 1 --2- 172.21.15.86:0/4185046603 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f413006f4d0 0x7f41300719e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f413c0b3000 tx=0x7f413c0ab070 comp rx=0 tx=0).stop 2024-04-25T14:04:15.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.261+0000 7f413b400640 1 -- 172.21.15.86:0/4185046603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4150070dd0 msgr2=0x7f41501f24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.261+0000 7f413b400640 1 --2- 172.21.15.86:0/4185046603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4150070dd0 0x7f41501f24f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f414000c7f0 tx=0x7f414000ccb0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.261+0000 7f413b400640 1 -- 172.21.15.86:0/4185046603 shutdown_connections 2024-04-25T14:04:15.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.261+0000 7f413b400640 1 --2- 172.21.15.86:0/4185046603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41501601c0 0x7f415013ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.261+0000 7f413b400640 1 --2- 172.21.15.86:0/4185046603 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f413006f4d0 0x7f41300719e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.261+0000 7f413b400640 1 --2- 172.21.15.86:0/4185046603 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41500cf4f0 0x7f41501f1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.261+0000 7f413b400640 1 --2- 172.21.15.86:0/4185046603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4150070dd0 0x7f41501f24f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.261+0000 7f413b400640 1 -- 172.21.15.86:0/4185046603 >> 172.21.15.86:0/4185046603 conn(0x7f41500d2230 msgr2=0x7f41500d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:15.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.261+0000 7f413b400640 1 -- 172.21.15.86:0/4185046603 shutdown_connections 2024-04-25T14:04:15.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.261+0000 7f413b400640 1 -- 172.21.15.86:0/4185046603 wait complete. 2024-04-25T14:04:15.265 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.5 2024-04-25T14:04:15.265 DEBUG:teuthology.parallel:result is None 2024-04-25T14:04:15.284 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.6 2024-04-25T14:04:15.284 DEBUG:teuthology.parallel:result is None 2024-04-25T14:04:15.286 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.3 2024-04-25T14:04:15.286 DEBUG:teuthology.parallel:result is None 2024-04-25T14:04:15.291 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574857 for osd.7 2024-04-25T14:04:15.292 DEBUG:teuthology.parallel:result is None 2024-04-25T14:04:15.299 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574857 for osd.2 2024-04-25T14:04:15.299 DEBUG:teuthology.parallel:result is None 2024-04-25T14:04:15.300 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.0 2024-04-25T14:04:15.300 DEBUG:teuthology.parallel:result is None 2024-04-25T14:04:15.300 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T14:04:15.300 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:04:15.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.493+0000 7f0583200640 1 Processor -- start 2024-04-25T14:04:15.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.493+0000 7f0583200640 1 -- start start 2024-04-25T14:04:15.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.493+0000 7f0583200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f058406d780 0x7f0584160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.493+0000 7f0583200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f058406db90 0x7f0584160bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.493+0000 7f0583200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f058409be50 con 0x7f05840d17c0 2024-04-25T14:04:15.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.493+0000 7f0583200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f05840b1c40 con 0x7f058406db90 2024-04-25T14:04:15.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.493+0000 7f0583200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f05840d3220 con 0x7f058406d780 2024-04-25T14:04:15.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.493+0000 7f0582800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f058406db90 0x7f0584160bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.493+0000 7f0582800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f058406db90 0x7f0584160bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38252/0 (socket says 172.21.15.86:38252) 2024-04-25T14:04:15.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.493+0000 7f0582800640 1 -- 172.21.15.86:0/3668128118 learned_addr learned my addr 172.21.15.86:0/3668128118 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:15.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.493+0000 7f0581e00640 1 --1- 172.21.15.86:0/3668128118 >> v1:172.21.15.155:6789/0 conn(0x7f05840d17c0 0x7f05840d2b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:50902/0 (socket says 172.21.15.86:50902) 2024-04-25T14:04:15.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.493+0000 7f0581400640 1 --2- 172.21.15.86:0/3668128118 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f058406d780 0x7f0584160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.493+0000 7f0582800640 1 -- 172.21.15.86:0/3668128118 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f058406d780 msgr2=0x7f0584160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:15.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.493+0000 7f0582800640 1 --2- 172.21.15.86:0/3668128118 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f058406d780 0x7f0584160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.493+0000 7f0582800640 1 -- 172.21.15.86:0/3668128118 >> v1:172.21.15.155:6789/0 conn(0x7f05840d17c0 legacy=0x7f05840d2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.493+0000 7f0582800640 1 -- 172.21.15.86:0/3668128118 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05840b1680 con 0x7f058406db90 2024-04-25T14:04:15.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.497+0000 7f0582800640 1 --2- 172.21.15.86:0/3668128118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f058406db90 0x7f0584160bd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f057c009e50 tx=0x7f057c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1f967d8b278d87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:15.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.497+0000 7f0580a00640 1 -- 172.21.15.86:0/3668128118 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c0a5550 con 0x7f058406db90 2024-04-25T14:04:15.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.497+0000 7f0580a00640 1 -- 172.21.15.86:0/3668128118 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f057c0a56f0 con 0x7f058406db90 2024-04-25T14:04:15.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.497+0000 7f0580a00640 1 -- 172.21.15.86:0/3668128118 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c0a5ac0 con 0x7f058406db90 2024-04-25T14:04:15.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.497+0000 7f0583200640 1 -- 172.21.15.86:0/3668128118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f058406db90 msgr2=0x7f0584160bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.497+0000 7f0583200640 1 --2- 172.21.15.86:0/3668128118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f058406db90 0x7f0584160bd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f057c009e50 tx=0x7f057c01c9a0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.497+0000 7f0583200640 1 -- 172.21.15.86:0/3668128118 shutdown_connections 2024-04-25T14:04:15.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.497+0000 7f0583200640 1 --2- 172.21.15.86:0/3668128118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f058406db90 0x7f0584160bd0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.497+0000 7f0583200640 1 --2- 172.21.15.86:0/3668128118 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f058406d780 0x7f0584160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.497+0000 7f0583200640 1 -- 172.21.15.86:0/3668128118 >> 172.21.15.86:0/3668128118 conn(0x7f0584073c00 msgr2=0x7f05840b77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:15.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.497+0000 7f0583200640 1 -- 172.21.15.86:0/3668128118 shutdown_connections 2024-04-25T14:04:15.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.497+0000 7f0583200640 1 -- 172.21.15.86:0/3668128118 wait complete. 2024-04-25T14:04:15.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.501+0000 7f0583200640 1 Processor -- start 2024-04-25T14:04:15.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.501+0000 7f0583200640 1 -- start start 2024-04-25T14:04:15.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.501+0000 7f0583200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f058406d780 0x7f058414a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.501+0000 7f0583200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f058406db90 0x7f0584153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.501+0000 7f0583200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05840d17c0 0x7f058414da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.501+0000 7f0583200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05840d1d50 con 0x7f058406db90 2024-04-25T14:04:15.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.501+0000 7f0583200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f05840d1bd0 con 0x7f05840d17c0 2024-04-25T14:04:15.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.501+0000 7f0583200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f058406ebf0 con 0x7f058406d780 2024-04-25T14:04:15.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.501+0000 7f0581400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f058406db90 0x7f0584153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.501+0000 7f0581400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f058406db90 0x7f0584153260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38268/0 (socket says 172.21.15.86:38268) 2024-04-25T14:04:15.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.501+0000 7f0581400640 1 -- 172.21.15.86:0/878768105 learned_addr learned my addr 172.21.15.86:0/878768105 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:15.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.501+0000 7f0582800640 1 --2- 172.21.15.86:0/878768105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05840d17c0 0x7f058414da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.501+0000 7f0581e00640 1 --2- 172.21.15.86:0/878768105 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f058406d780 0x7f058414a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.505+0000 7f0581400640 1 -- 172.21.15.86:0/878768105 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f058406d780 msgr2=0x7f058414a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.505+0000 7f0581400640 1 --2- 172.21.15.86:0/878768105 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f058406d780 0x7f058414a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.505+0000 7f0581400640 1 -- 172.21.15.86:0/878768105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05840d17c0 msgr2=0x7f058414da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.505+0000 7f0581400640 1 --2- 172.21.15.86:0/878768105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05840d17c0 0x7f058414da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.505+0000 7f0581400640 1 -- 172.21.15.86:0/878768105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f058414aa50 con 0x7f058406db90 2024-04-25T14:04:15.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.505+0000 7f0581400640 1 --2- 172.21.15.86:0/878768105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f058406db90 0x7f0584153260 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f057000e870 tx=0x7f057000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:15.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.505+0000 7f057a000640 1 -- 172.21.15.86:0/878768105 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570017070 con 0x7f058406db90 2024-04-25T14:04:15.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.505+0000 7f057a000640 1 -- 172.21.15.86:0/878768105 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f057000c940 con 0x7f058406db90 2024-04-25T14:04:15.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.505+0000 7f0583200640 1 -- 172.21.15.86:0/878768105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0584153f20 con 0x7f058406db90 2024-04-25T14:04:15.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.505+0000 7f057a000640 1 -- 172.21.15.86:0/878768105 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057000ccf0 con 0x7f058406db90 2024-04-25T14:04:15.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.505+0000 7f0583200640 1 -- 172.21.15.86:0/878768105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05841566d0 con 0x7f058406db90 2024-04-25T14:04:15.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.505+0000 7f0583200640 1 -- 172.21.15.86:0/878768105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0584153c90 con 0x7f058406db90 2024-04-25T14:04:15.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.505+0000 7f057a000640 1 -- 172.21.15.86:0/878768105 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f057001a070 con 0x7f058406db90 2024-04-25T14:04:15.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.509+0000 7f057a000640 1 --2- 172.21.15.86:0/878768105 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f056406f400 0x7f0564071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.509+0000 7f057a000640 1 -- 172.21.15.86:0/878768105 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f057008f5b0 con 0x7f058406db90 2024-04-25T14:04:15.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.509+0000 7f0581e00640 1 --2- 172.21.15.86:0/878768105 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f056406f400 0x7f0564071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.509+0000 7f0581e00640 1 --2- 172.21.15.86:0/878768105 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f056406f400 0x7f0564071910 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0574000f70 tx=0x7f0574007c90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:15.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.513+0000 7f057a000640 1 -- 172.21.15.86:0/878768105 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f057005c0c0 con 0x7f058406db90 2024-04-25T14:04:15.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.665+0000 7f0583200640 1 -- 172.21.15.86:0/878768105 --> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0584076b80 con 0x7f056406f400 2024-04-25T14:04:15.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.677+0000 7f057a000640 1 -- 172.21.15.86:0/878768105 <== mgr.4103 v2:172.21.15.86:6832/2968869755 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332527 (secure 0 0 0) 0x7f0584076b80 con 0x7f056406f400 2024-04-25T14:04:15.681 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:15.683 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:04:15.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.681+0000 7f0583200640 1 -- 172.21.15.86:0/878768105 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f056406f400 msgr2=0x7f0564071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.681+0000 7f0583200640 1 --2- 172.21.15.86:0/878768105 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f056406f400 0x7f0564071910 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0574000f70 tx=0x7f0574007c90 comp rx=0 tx=0).stop 2024-04-25T14:04:15.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.681+0000 7f0583200640 1 -- 172.21.15.86:0/878768105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f058406db90 msgr2=0x7f0584153260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.681+0000 7f0583200640 1 --2- 172.21.15.86:0/878768105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f058406db90 0x7f0584153260 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f057000e870 tx=0x7f057000ec70 comp rx=0 tx=0).stop 2024-04-25T14:04:15.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.681+0000 7f0583200640 1 -- 172.21.15.86:0/878768105 shutdown_connections 2024-04-25T14:04:15.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.681+0000 7f0583200640 1 --2- 172.21.15.86:0/878768105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05840d17c0 0x7f058414da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.681+0000 7f0583200640 1 --2- 172.21.15.86:0/878768105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f058406db90 0x7f0584153260 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.681+0000 7f0583200640 1 --2- 172.21.15.86:0/878768105 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f056406f400 0x7f0564071910 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.681+0000 7f0583200640 1 --2- 172.21.15.86:0/878768105 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f058406d780 0x7f058414a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.681+0000 7f0583200640 1 -- 172.21.15.86:0/878768105 >> 172.21.15.86:0/878768105 conn(0x7f0584073c00 msgr2=0x7f0584159c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:15.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.681+0000 7f0583200640 1 -- 172.21.15.86:0/878768105 shutdown_connections 2024-04-25T14:04:15.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.681+0000 7f0583200640 1 -- 172.21.15.86:0/878768105 wait complete. 2024-04-25T14:04:15.703 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-04-25T14:04:14.715700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":150,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219400,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755320,"statfs":{"total":773094113280,"available":772869447680,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002893"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774780+0000","last_change":"2024-04-25T14:03:50.477862+0000","last_active":"2024-04-25T14:03:56.774780+0000","last_peered":"2024-04-25T14:03:56.774780+0000","last_clean":"2024-04-25T14:03:56.774780+0000","last_became_active":"2024-04-25T14:03:50.477375+0000","last_became_peered":"2024-04-25T14:03:50.477375+0000","last_unstale":"2024-04-25T14:03:56.774780+0000","last_undegraded":"2024-04-25T14:03:56.774780+0000","last_fullsized":"2024-04-25T14:03:56.774780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:01.852459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494933+0000","last_change":"2024-04-25T14:03:51.479650+0000","last_active":"2024-04-25T14:03:56.494933+0000","last_peered":"2024-04-25T14:03:56.494933+0000","last_clean":"2024-04-25T14:03:56.494933+0000","last_became_active":"2024-04-25T14:03:51.479295+0000","last_became_peered":"2024-04-25T14:03:51.479295+0000","last_unstale":"2024-04-25T14:03:56.494933+0000","last_undegraded":"2024-04-25T14:03:56.494933+0000","last_fullsized":"2024-04-25T14:03:56.494933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:17.381249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493593+0000","last_change":"2024-04-25T14:03:55.681212+0000","last_active":"2024-04-25T14:03:56.493593+0000","last_peered":"2024-04-25T14:03:56.493593+0000","last_clean":"2024-04-25T14:03:56.493593+0000","last_became_active":"2024-04-25T14:03:55.680579+0000","last_became_peered":"2024-04-25T14:03:55.680579+0000","last_unstale":"2024-04-25T14:03:56.493593+0000","last_undegraded":"2024-04-25T14:03:56.493593+0000","last_fullsized":"2024-04-25T14:03:56.493593+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:33:32.909419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495038+0000","last_change":"2024-04-25T14:03:50.468056+0000","last_active":"2024-04-25T14:03:56.495038+0000","last_peered":"2024-04-25T14:03:56.495038+0000","last_clean":"2024-04-25T14:03:56.495038+0000","last_became_active":"2024-04-25T14:03:50.467531+0000","last_became_peered":"2024-04-25T14:03:50.467531+0000","last_unstale":"2024-04-25T14:03:56.495038+0000","last_undegraded":"2024-04-25T14:03:56.495038+0000","last_fullsized":"2024-04-25T14:03:56.495038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:29.264084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774540+0000","last_change":"2024-04-25T14:03:51.485995+0000","last_active":"2024-04-25T14:03:56.774540+0000","last_peered":"2024-04-25T14:03:56.774540+0000","last_clean":"2024-04-25T14:03:56.774540+0000","last_became_active":"2024-04-25T14:03:51.485364+0000","last_became_peered":"2024-04-25T14:03:51.485364+0000","last_unstale":"2024-04-25T14:03:56.774540+0000","last_undegraded":"2024-04-25T14:03:56.774540+0000","last_fullsized":"2024-04-25T14:03:56.774540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:10.318378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772007+0000","last_change":"2024-04-25T14:03:55.514585+0000","last_active":"2024-04-25T14:03:56.772007+0000","last_peered":"2024-04-25T14:03:56.772007+0000","last_clean":"2024-04-25T14:03:56.772007+0000","last_became_active":"2024-04-25T14:03:55.514106+0000","last_became_peered":"2024-04-25T14:03:55.514106+0000","last_unstale":"2024-04-25T14:03:56.772007+0000","last_undegraded":"2024-04-25T14:03:56.772007+0000","last_fullsized":"2024-04-25T14:03:56.772007+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:14.363355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773034+0000","last_change":"2024-04-25T14:03:50.476493+0000","last_active":"2024-04-25T14:03:56.773034+0000","last_peered":"2024-04-25T14:03:56.773034+0000","last_clean":"2024-04-25T14:03:56.773034+0000","last_became_active":"2024-04-25T14:03:50.476154+0000","last_became_peered":"2024-04-25T14:03:50.476154+0000","last_unstale":"2024-04-25T14:03:56.773034+0000","last_undegraded":"2024-04-25T14:03:56.773034+0000","last_fullsized":"2024-04-25T14:03:56.773034+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:45.080718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496310+0000","last_change":"2024-04-25T14:03:51.494560+0000","last_active":"2024-04-25T14:03:56.496310+0000","last_peered":"2024-04-25T14:03:56.496310+0000","last_clean":"2024-04-25T14:03:56.496310+0000","last_became_active":"2024-04-25T14:03:51.494211+0000","last_became_peered":"2024-04-25T14:03:51.494211+0000","last_unstale":"2024-04-25T14:03:56.496310+0000","last_undegraded":"2024-04-25T14:03:56.496310+0000","last_fullsized":"2024-04-25T14:03:56.496310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:17.381249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772740+0000","last_change":"2024-04-25T14:03:55.518340+0000","last_active":"2024-04-25T14:03:56.772740+0000","last_peered":"2024-04-25T14:03:56.772740+0000","last_clean":"2024-04-25T14:03:56.772740+0000","last_became_active":"2024-04-25T14:03:55.515787+0000","last_became_peered":"2024-04-25T14:03:55.515787+0000","last_unstale":"2024-04-25T14:03:56.772740+0000","last_undegraded":"2024-04-25T14:03:56.772740+0000","last_fullsized":"2024-04-25T14:03:56.772740+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:24.136045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772400+0000","last_change":"2024-04-25T14:03:50.472667+0000","last_active":"2024-04-25T14:03:56.772400+0000","last_peered":"2024-04-25T14:03:56.772400+0000","last_clean":"2024-04-25T14:03:56.772400+0000","last_became_active":"2024-04-25T14:03:50.472286+0000","last_became_peered":"2024-04-25T14:03:50.472286+0000","last_unstale":"2024-04-25T14:03:56.772400+0000","last_undegraded":"2024-04-25T14:03:56.772400+0000","last_fullsized":"2024-04-25T14:03:56.772400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:02.117945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496569+0000","last_change":"2024-04-25T14:03:51.494545+0000","last_active":"2024-04-25T14:03:56.496569+0000","last_peered":"2024-04-25T14:03:56.496569+0000","last_clean":"2024-04-25T14:03:56.496569+0000","last_became_active":"2024-04-25T14:03:51.494238+0000","last_became_peered":"2024-04-25T14:03:51.494238+0000","last_unstale":"2024-04-25T14:03:56.496569+0000","last_undegraded":"2024-04-25T14:03:56.496569+0000","last_fullsized":"2024-04-25T14:03:56.496569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:08.845346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772125+0000","last_change":"2024-04-25T14:03:55.512835+0000","last_active":"2024-04-25T14:03:56.772125+0000","last_peered":"2024-04-25T14:03:56.772125+0000","last_clean":"2024-04-25T14:03:56.772125+0000","last_became_active":"2024-04-25T14:03:55.512330+0000","last_became_peered":"2024-04-25T14:03:55.512330+0000","last_unstale":"2024-04-25T14:03:56.772125+0000","last_undegraded":"2024-04-25T14:03:56.772125+0000","last_fullsized":"2024-04-25T14:03:56.772125+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:47.781818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494962+0000","last_change":"2024-04-25T14:03:51.494496+0000","last_active":"2024-04-25T14:03:56.494962+0000","last_peered":"2024-04-25T14:03:56.494962+0000","last_clean":"2024-04-25T14:03:56.494962+0000","last_became_active":"2024-04-25T14:03:51.494201+0000","last_became_peered":"2024-04-25T14:03:51.494201+0000","last_unstale":"2024-04-25T14:03:56.494962+0000","last_undegraded":"2024-04-25T14:03:56.494962+0000","last_fullsized":"2024-04-25T14:03:56.494962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.599283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773644+0000","last_change":"2024-04-25T14:03:50.472026+0000","last_active":"2024-04-25T14:03:56.773644+0000","last_peered":"2024-04-25T14:03:56.773644+0000","last_clean":"2024-04-25T14:03:56.773644+0000","last_became_active":"2024-04-25T14:03:50.468476+0000","last_became_peered":"2024-04-25T14:03:50.468476+0000","last_unstale":"2024-04-25T14:03:56.773644+0000","last_undegraded":"2024-04-25T14:03:56.773644+0000","last_fullsized":"2024-04-25T14:03:56.773644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:39.630327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772916+0000","last_change":"2024-04-25T14:03:55.524347+0000","last_active":"2024-04-25T14:03:56.772916+0000","last_peered":"2024-04-25T14:03:56.772916+0000","last_clean":"2024-04-25T14:03:56.772916+0000","last_became_active":"2024-04-25T14:03:55.523890+0000","last_became_peered":"2024-04-25T14:03:55.523890+0000","last_unstale":"2024-04-25T14:03:56.772916+0000","last_undegraded":"2024-04-25T14:03:56.772916+0000","last_fullsized":"2024-04-25T14:03:56.772916+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:20.858506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493611+0000","last_change":"2024-04-25T14:03:51.471282+0000","last_active":"2024-04-25T14:03:56.493611+0000","last_peered":"2024-04-25T14:03:56.493611+0000","last_clean":"2024-04-25T14:03:56.493611+0000","last_became_active":"2024-04-25T14:03:51.470871+0000","last_became_peered":"2024-04-25T14:03:51.470871+0000","last_unstale":"2024-04-25T14:03:56.493611+0000","last_undegraded":"2024-04-25T14:03:56.493611+0000","last_fullsized":"2024-04-25T14:03:56.493611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:34.707119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773462+0000","last_change":"2024-04-25T14:03:50.467312+0000","last_active":"2024-04-25T14:03:56.773462+0000","last_peered":"2024-04-25T14:03:56.773462+0000","last_clean":"2024-04-25T14:03:56.773462+0000","last_became_active":"2024-04-25T14:03:50.467020+0000","last_became_peered":"2024-04-25T14:03:50.467020+0000","last_unstale":"2024-04-25T14:03:56.773462+0000","last_undegraded":"2024-04-25T14:03:56.773462+0000","last_fullsized":"2024-04-25T14:03:56.773462+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:45.080718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772970+0000","last_change":"2024-04-25T14:03:55.520580+0000","last_active":"2024-04-25T14:03:56.772970+0000","last_peered":"2024-04-25T14:03:56.772970+0000","last_clean":"2024-04-25T14:03:56.772970+0000","last_became_active":"2024-04-25T14:03:55.520013+0000","last_became_peered":"2024-04-25T14:03:55.520013+0000","last_unstale":"2024-04-25T14:03:56.772970+0000","last_undegraded":"2024-04-25T14:03:56.772970+0000","last_fullsized":"2024-04-25T14:03:56.772970+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:38.752096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494042+0000","last_change":"2024-04-25T14:03:51.474463+0000","last_active":"2024-04-25T14:03:56.494042+0000","last_peered":"2024-04-25T14:03:56.494042+0000","last_clean":"2024-04-25T14:03:56.494042+0000","last_became_active":"2024-04-25T14:03:51.474154+0000","last_became_peered":"2024-04-25T14:03:51.474154+0000","last_unstale":"2024-04-25T14:03:56.494042+0000","last_undegraded":"2024-04-25T14:03:56.494042+0000","last_fullsized":"2024-04-25T14:03:56.494042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:10.318378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772985+0000","last_change":"2024-04-25T14:03:50.476410+0000","last_active":"2024-04-25T14:03:56.772985+0000","last_peered":"2024-04-25T14:03:56.772985+0000","last_clean":"2024-04-25T14:03:56.772985+0000","last_became_active":"2024-04-25T14:03:50.476083+0000","last_became_peered":"2024-04-25T14:03:50.476083+0000","last_unstale":"2024-04-25T14:03:56.772985+0000","last_undegraded":"2024-04-25T14:03:56.772985+0000","last_fullsized":"2024-04-25T14:03:56.772985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:29.264084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774278+0000","last_change":"2024-04-25T14:03:55.674875+0000","last_active":"2024-04-25T14:03:56.774278+0000","last_peered":"2024-04-25T14:03:56.774278+0000","last_clean":"2024-04-25T14:03:56.774278+0000","last_became_active":"2024-04-25T14:03:55.674408+0000","last_became_peered":"2024-04-25T14:03:55.674408+0000","last_unstale":"2024-04-25T14:03:56.774278+0000","last_undegraded":"2024-04-25T14:03:56.774278+0000","last_fullsized":"2024-04-25T14:03:56.774278+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:50.234674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773825+0000","last_change":"2024-04-25T14:03:51.487560+0000","last_active":"2024-04-25T14:03:56.773825+0000","last_peered":"2024-04-25T14:03:56.773825+0000","last_clean":"2024-04-25T14:03:56.773825+0000","last_became_active":"2024-04-25T14:03:51.486822+0000","last_became_peered":"2024-04-25T14:03:51.486822+0000","last_unstale":"2024-04-25T14:03:56.773825+0000","last_undegraded":"2024-04-25T14:03:56.773825+0000","last_fullsized":"2024-04-25T14:03:56.773825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:16.370061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.771874+0000","last_change":"2024-04-25T14:03:50.479489+0000","last_active":"2024-04-25T14:03:56.771874+0000","last_peered":"2024-04-25T14:03:56.771874+0000","last_clean":"2024-04-25T14:03:56.771874+0000","last_became_active":"2024-04-25T14:03:50.479212+0000","last_became_peered":"2024-04-25T14:03:50.479212+0000","last_unstale":"2024-04-25T14:03:56.771874+0000","last_undegraded":"2024-04-25T14:03:56.771874+0000","last_fullsized":"2024-04-25T14:03:56.771874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:10.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773707+0000","last_change":"2024-04-25T14:03:55.519638+0000","last_active":"2024-04-25T14:03:56.773707+0000","last_peered":"2024-04-25T14:03:56.773707+0000","last_clean":"2024-04-25T14:03:56.773707+0000","last_became_active":"2024-04-25T14:03:55.519191+0000","last_became_peered":"2024-04-25T14:03:55.519191+0000","last_unstale":"2024-04-25T14:03:56.773707+0000","last_undegraded":"2024-04-25T14:03:56.773707+0000","last_fullsized":"2024-04-25T14:03:56.773707+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:25.451897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774673+0000","last_change":"2024-04-25T14:03:51.477687+0000","last_active":"2024-04-25T14:03:56.774673+0000","last_peered":"2024-04-25T14:03:56.774673+0000","last_clean":"2024-04-25T14:03:56.774673+0000","last_became_active":"2024-04-25T14:03:51.477339+0000","last_became_peered":"2024-04-25T14:03:51.477339+0000","last_unstale":"2024-04-25T14:03:56.774673+0000","last_undegraded":"2024-04-25T14:03:56.774673+0000","last_fullsized":"2024-04-25T14:03:56.774673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:43.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496742+0000","last_change":"2024-04-25T14:03:50.470614+0000","last_active":"2024-04-25T14:03:56.496742+0000","last_peered":"2024-04-25T14:03:56.496742+0000","last_clean":"2024-04-25T14:03:56.496742+0000","last_became_active":"2024-04-25T14:03:50.470405+0000","last_became_peered":"2024-04-25T14:03:50.470405+0000","last_unstale":"2024-04-25T14:03:56.496742+0000","last_undegraded":"2024-04-25T14:03:56.496742+0000","last_fullsized":"2024-04-25T14:03:56.496742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:02.117945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773539+0000","last_change":"2024-04-25T14:03:55.681096+0000","last_active":"2024-04-25T14:03:56.773539+0000","last_peered":"2024-04-25T14:03:56.773539+0000","last_clean":"2024-04-25T14:03:56.773539+0000","last_became_active":"2024-04-25T14:03:55.680556+0000","last_became_peered":"2024-04-25T14:03:55.680556+0000","last_unstale":"2024-04-25T14:03:56.773539+0000","last_undegraded":"2024-04-25T14:03:56.773539+0000","last_fullsized":"2024-04-25T14:03:56.773539+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:09.534866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495184+0000","last_change":"2024-04-25T14:03:51.473595+0000","last_active":"2024-04-25T14:03:56.495184+0000","last_peered":"2024-04-25T14:03:56.495184+0000","last_clean":"2024-04-25T14:03:56.495184+0000","last_became_active":"2024-04-25T14:03:51.473273+0000","last_became_peered":"2024-04-25T14:03:51.473273+0000","last_unstale":"2024-04-25T14:03:56.495184+0000","last_undegraded":"2024-04-25T14:03:56.495184+0000","last_fullsized":"2024-04-25T14:03:56.495184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:05.489889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774166+0000","last_change":"2024-04-25T14:03:50.476274+0000","last_active":"2024-04-25T14:03:56.774166+0000","last_peered":"2024-04-25T14:03:56.774166+0000","last_clean":"2024-04-25T14:03:56.774166+0000","last_became_active":"2024-04-25T14:03:50.475935+0000","last_became_peered":"2024-04-25T14:03:50.475935+0000","last_unstale":"2024-04-25T14:03:56.774166+0000","last_undegraded":"2024-04-25T14:03:56.774166+0000","last_fullsized":"2024-04-25T14:03:56.774166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:39.630327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493917+0000","last_change":"2024-04-25T14:03:55.675251+0000","last_active":"2024-04-25T14:03:56.493917+0000","last_peered":"2024-04-25T14:03:56.493917+0000","last_clean":"2024-04-25T14:03:56.493917+0000","last_became_active":"2024-04-25T14:03:55.674772+0000","last_became_peered":"2024-04-25T14:03:55.674772+0000","last_unstale":"2024-04-25T14:03:56.493917+0000","last_undegraded":"2024-04-25T14:03:56.493917+0000","last_fullsized":"2024-04-25T14:03:56.493917+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:58.844877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773967+0000","last_change":"2024-04-25T14:03:51.476624+0000","last_active":"2024-04-25T14:03:56.773967+0000","last_peered":"2024-04-25T14:03:56.773967+0000","last_clean":"2024-04-25T14:03:56.773967+0000","last_became_active":"2024-04-25T14:03:51.476320+0000","last_became_peered":"2024-04-25T14:03:51.476320+0000","last_unstale":"2024-04-25T14:03:56.773967+0000","last_undegraded":"2024-04-25T14:03:56.773967+0000","last_fullsized":"2024-04-25T14:03:56.773967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:20.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773171+0000","last_change":"2024-04-25T14:03:50.482353+0000","last_active":"2024-04-25T14:03:56.773171+0000","last_peered":"2024-04-25T14:03:56.773171+0000","last_clean":"2024-04-25T14:03:56.773171+0000","last_became_active":"2024-04-25T14:03:50.482063+0000","last_became_peered":"2024-04-25T14:03:50.482063+0000","last_unstale":"2024-04-25T14:03:56.773171+0000","last_undegraded":"2024-04-25T14:03:56.773171+0000","last_fullsized":"2024-04-25T14:03:56.773171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:37.406291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773151+0000","last_change":"2024-04-25T14:03:55.676872+0000","last_active":"2024-04-25T14:03:56.773151+0000","last_peered":"2024-04-25T14:03:56.773151+0000","last_clean":"2024-04-25T14:03:56.773151+0000","last_became_active":"2024-04-25T14:03:55.676444+0000","last_became_peered":"2024-04-25T14:03:55.676444+0000","last_unstale":"2024-04-25T14:03:56.773151+0000","last_undegraded":"2024-04-25T14:03:56.773151+0000","last_fullsized":"2024-04-25T14:03:56.773151+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:37.811376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496691+0000","last_change":"2024-04-25T14:03:51.480200+0000","last_active":"2024-04-25T14:03:56.496691+0000","last_peered":"2024-04-25T14:03:56.496691+0000","last_clean":"2024-04-25T14:03:56.496691+0000","last_became_active":"2024-04-25T14:03:51.479798+0000","last_became_peered":"2024-04-25T14:03:51.479798+0000","last_unstale":"2024-04-25T14:03:56.496691+0000","last_undegraded":"2024-04-25T14:03:56.496691+0000","last_fullsized":"2024-04-25T14:03:56.496691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:46.189697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774003+0000","last_change":"2024-04-25T14:03:50.479761+0000","last_active":"2024-04-25T14:03:56.774003+0000","last_peered":"2024-04-25T14:03:56.774003+0000","last_clean":"2024-04-25T14:03:56.774003+0000","last_became_active":"2024-04-25T14:03:50.479175+0000","last_became_peered":"2024-04-25T14:03:50.479175+0000","last_unstale":"2024-04-25T14:03:56.774003+0000","last_undegraded":"2024-04-25T14:03:56.774003+0000","last_fullsized":"2024-04-25T14:03:56.774003+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:02.117945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773256+0000","last_change":"2024-04-25T14:03:55.515692+0000","last_active":"2024-04-25T14:03:56.773256+0000","last_peered":"2024-04-25T14:03:56.773256+0000","last_clean":"2024-04-25T14:03:56.773256+0000","last_became_active":"2024-04-25T14:03:55.515238+0000","last_became_peered":"2024-04-25T14:03:55.515238+0000","last_unstale":"2024-04-25T14:03:56.773256+0000","last_undegraded":"2024-04-25T14:03:56.773256+0000","last_fullsized":"2024-04-25T14:03:56.773256+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:51.371333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773096+0000","last_change":"2024-04-25T14:03:51.495207+0000","last_active":"2024-04-25T14:03:56.773096+0000","last_peered":"2024-04-25T14:03:56.773096+0000","last_clean":"2024-04-25T14:03:56.773096+0000","last_became_active":"2024-04-25T14:03:51.492531+0000","last_became_peered":"2024-04-25T14:03:51.492531+0000","last_unstale":"2024-04-25T14:03:56.773096+0000","last_undegraded":"2024-04-25T14:03:56.773096+0000","last_fullsized":"2024-04-25T14:03:56.773096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:03.123407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495711+0000","last_change":"2024-04-25T14:03:50.459698+0000","last_active":"2024-04-25T14:03:56.495711+0000","last_peered":"2024-04-25T14:03:56.495711+0000","last_clean":"2024-04-25T14:03:56.495711+0000","last_became_active":"2024-04-25T14:03:50.459451+0000","last_became_peered":"2024-04-25T14:03:50.459451+0000","last_unstale":"2024-04-25T14:03:56.495711+0000","last_undegraded":"2024-04-25T14:03:56.495711+0000","last_fullsized":"2024-04-25T14:03:56.495711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:15.364599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773837+0000","last_change":"2024-04-25T14:03:55.519334+0000","last_active":"2024-04-25T14:03:56.773837+0000","last_peered":"2024-04-25T14:03:56.773837+0000","last_clean":"2024-04-25T14:03:56.773837+0000","last_became_active":"2024-04-25T14:03:55.518940+0000","last_became_peered":"2024-04-25T14:03:55.518940+0000","last_unstale":"2024-04-25T14:03:56.773837+0000","last_undegraded":"2024-04-25T14:03:56.773837+0000","last_fullsized":"2024-04-25T14:03:56.773837+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:12.890323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772902+0000","last_change":"2024-04-25T14:03:51.476141+0000","last_active":"2024-04-25T14:03:56.772902+0000","last_peered":"2024-04-25T14:03:56.772902+0000","last_clean":"2024-04-25T14:03:56.772902+0000","last_became_active":"2024-04-25T14:03:51.475819+0000","last_became_peered":"2024-04-25T14:03:51.475819+0000","last_unstale":"2024-04-25T14:03:56.772902+0000","last_undegraded":"2024-04-25T14:03:56.772902+0000","last_fullsized":"2024-04-25T14:03:56.772902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:31.707048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772028+0000","last_change":"2024-04-25T14:03:50.458612+0000","last_active":"2024-04-25T14:03:56.772028+0000","last_peered":"2024-04-25T14:03:56.772028+0000","last_clean":"2024-04-25T14:03:56.772028+0000","last_became_active":"2024-04-25T14:03:50.458254+0000","last_became_peered":"2024-04-25T14:03:50.458254+0000","last_unstale":"2024-04-25T14:03:56.772028+0000","last_undegraded":"2024-04-25T14:03:56.772028+0000","last_fullsized":"2024-04-25T14:03:56.772028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:29.264084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774279+0000","last_change":"2024-04-25T14:03:55.513023+0000","last_active":"2024-04-25T14:03:56.774279+0000","last_peered":"2024-04-25T14:03:56.774279+0000","last_clean":"2024-04-25T14:03:56.774279+0000","last_became_active":"2024-04-25T14:03:55.512550+0000","last_became_peered":"2024-04-25T14:03:55.512550+0000","last_unstale":"2024-04-25T14:03:56.774279+0000","last_undegraded":"2024-04-25T14:03:56.774279+0000","last_fullsized":"2024-04-25T14:03:56.774279+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:06.818399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.771972+0000","last_change":"2024-04-25T14:03:51.487355+0000","last_active":"2024-04-25T14:03:56.771972+0000","last_peered":"2024-04-25T14:03:56.771972+0000","last_clean":"2024-04-25T14:03:56.771972+0000","last_became_active":"2024-04-25T14:03:51.487050+0000","last_became_peered":"2024-04-25T14:03:51.487050+0000","last_unstale":"2024-04-25T14:03:56.771972+0000","last_undegraded":"2024-04-25T14:03:56.771972+0000","last_fullsized":"2024-04-25T14:03:56.771972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:33.766258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772008+0000","last_change":"2024-04-25T14:03:50.476776+0000","last_active":"2024-04-25T14:03:56.772008+0000","last_peered":"2024-04-25T14:03:56.772008+0000","last_clean":"2024-04-25T14:03:56.772008+0000","last_became_active":"2024-04-25T14:03:50.476470+0000","last_became_peered":"2024-04-25T14:03:50.476470+0000","last_unstale":"2024-04-25T14:03:56.772008+0000","last_undegraded":"2024-04-25T14:03:56.772008+0000","last_fullsized":"2024-04-25T14:03:56.772008+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:15.364599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774604+0000","last_change":"2024-04-25T14:03:55.674908+0000","last_active":"2024-04-25T14:03:56.774604+0000","last_peered":"2024-04-25T14:03:56.774604+0000","last_clean":"2024-04-25T14:03:56.774604+0000","last_became_active":"2024-04-25T14:03:55.674383+0000","last_became_peered":"2024-04-25T14:03:55.674383+0000","last_unstale":"2024-04-25T14:03:56.774604+0000","last_undegraded":"2024-04-25T14:03:56.774604+0000","last_fullsized":"2024-04-25T14:03:56.774604+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:06.742929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493557+0000","last_change":"2024-04-25T14:03:51.489259+0000","last_active":"2024-04-25T14:03:56.493557+0000","last_peered":"2024-04-25T14:03:56.493557+0000","last_clean":"2024-04-25T14:03:56.493557+0000","last_became_active":"2024-04-25T14:03:51.488822+0000","last_became_peered":"2024-04-25T14:03:51.488822+0000","last_unstale":"2024-04-25T14:03:56.493557+0000","last_undegraded":"2024-04-25T14:03:56.493557+0000","last_fullsized":"2024-04-25T14:03:56.493557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:43.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495950+0000","last_change":"2024-04-25T14:03:50.466483+0000","last_active":"2024-04-25T14:03:56.495950+0000","last_peered":"2024-04-25T14:03:56.495950+0000","last_clean":"2024-04-25T14:03:56.495950+0000","last_became_active":"2024-04-25T14:03:50.463125+0000","last_became_peered":"2024-04-25T14:03:50.463125+0000","last_unstale":"2024-04-25T14:03:56.495950+0000","last_undegraded":"2024-04-25T14:03:56.495950+0000","last_fullsized":"2024-04-25T14:03:56.495950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:29.264084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":117,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772125+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:03:56.772125+0000","last_peered":"2024-04-25T14:03:56.772125+0000","last_clean":"2024-04-25T14:03:56.772125+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:03:56.772125+0000","last_undegraded":"2024-04-25T14:03:56.772125+0000","last_fullsized":"2024-04-25T14:03:56.772125+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496238+0000","last_change":"2024-04-25T14:03:55.678084+0000","last_active":"2024-04-25T14:03:56.496238+0000","last_peered":"2024-04-25T14:03:56.496238+0000","last_clean":"2024-04-25T14:03:56.496238+0000","last_became_active":"2024-04-25T14:03:55.677526+0000","last_became_peered":"2024-04-25T14:03:55.677526+0000","last_unstale":"2024-04-25T14:03:56.496238+0000","last_undegraded":"2024-04-25T14:03:56.496238+0000","last_fullsized":"2024-04-25T14:03:56.496238+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:25.451897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772807+0000","last_change":"2024-04-25T14:03:51.482608+0000","last_active":"2024-04-25T14:03:56.772807+0000","last_peered":"2024-04-25T14:03:56.772807+0000","last_clean":"2024-04-25T14:03:56.772807+0000","last_became_active":"2024-04-25T14:03:51.482022+0000","last_became_peered":"2024-04-25T14:03:51.482022+0000","last_unstale":"2024-04-25T14:03:56.772807+0000","last_undegraded":"2024-04-25T14:03:56.772807+0000","last_fullsized":"2024-04-25T14:03:56.772807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:33.766399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772584+0000","last_change":"2024-04-25T14:03:50.469170+0000","last_active":"2024-04-25T14:03:56.772584+0000","last_peered":"2024-04-25T14:03:56.772584+0000","last_clean":"2024-04-25T14:03:56.772584+0000","last_became_active":"2024-04-25T14:03:50.468742+0000","last_became_peered":"2024-04-25T14:03:50.468742+0000","last_unstale":"2024-04-25T14:03:56.772584+0000","last_undegraded":"2024-04-25T14:03:56.772584+0000","last_fullsized":"2024-04-25T14:03:56.772584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:10.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494752+0000","last_change":"2024-04-25T14:03:55.676560+0000","last_active":"2024-04-25T14:03:56.494752+0000","last_peered":"2024-04-25T14:03:56.494752+0000","last_clean":"2024-04-25T14:03:56.494752+0000","last_became_active":"2024-04-25T14:03:55.676021+0000","last_became_peered":"2024-04-25T14:03:55.676021+0000","last_unstale":"2024-04-25T14:03:56.494752+0000","last_undegraded":"2024-04-25T14:03:56.494752+0000","last_fullsized":"2024-04-25T14:03:56.494752+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:09.534866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496565+0000","last_change":"2024-04-25T14:03:51.466314+0000","last_active":"2024-04-25T14:03:56.496565+0000","last_peered":"2024-04-25T14:03:56.496565+0000","last_clean":"2024-04-25T14:03:56.496565+0000","last_became_active":"2024-04-25T14:03:51.465982+0000","last_became_peered":"2024-04-25T14:03:51.465982+0000","last_unstale":"2024-04-25T14:03:56.496565+0000","last_undegraded":"2024-04-25T14:03:56.496565+0000","last_fullsized":"2024-04-25T14:03:56.496565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:47.326356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774109+0000","last_change":"2024-04-25T14:03:50.465115+0000","last_active":"2024-04-25T14:03:56.774109+0000","last_peered":"2024-04-25T14:03:56.774109+0000","last_clean":"2024-04-25T14:03:56.774109+0000","last_became_active":"2024-04-25T14:03:50.464561+0000","last_became_peered":"2024-04-25T14:03:50.464561+0000","last_unstale":"2024-04-25T14:03:56.774109+0000","last_undegraded":"2024-04-25T14:03:56.774109+0000","last_fullsized":"2024-04-25T14:03:56.774109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:10.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773457+0000","last_change":"2024-04-25T14:03:55.526344+0000","last_active":"2024-04-25T14:03:56.773457+0000","last_peered":"2024-04-25T14:03:56.773457+0000","last_clean":"2024-04-25T14:03:56.773457+0000","last_became_active":"2024-04-25T14:03:55.525889+0000","last_became_peered":"2024-04-25T14:03:55.525889+0000","last_unstale":"2024-04-25T14:03:56.773457+0000","last_undegraded":"2024-04-25T14:03:56.773457+0000","last_fullsized":"2024-04-25T14:03:56.773457+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:06.818399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773492+0000","last_change":"2024-04-25T14:03:51.494453+0000","last_active":"2024-04-25T14:03:56.773492+0000","last_peered":"2024-04-25T14:03:56.773492+0000","last_clean":"2024-04-25T14:03:56.773492+0000","last_became_active":"2024-04-25T14:03:51.494113+0000","last_became_peered":"2024-04-25T14:03:51.494113+0000","last_unstale":"2024-04-25T14:03:56.773492+0000","last_undegraded":"2024-04-25T14:03:56.773492+0000","last_fullsized":"2024-04-25T14:03:56.773492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:08.845346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772486+0000","last_change":"2024-04-25T14:03:50.463824+0000","last_active":"2024-04-25T14:03:56.772486+0000","last_peered":"2024-04-25T14:03:56.772486+0000","last_clean":"2024-04-25T14:03:56.772486+0000","last_became_active":"2024-04-25T14:03:50.463507+0000","last_became_peered":"2024-04-25T14:03:50.463507+0000","last_unstale":"2024-04-25T14:03:56.772486+0000","last_undegraded":"2024-04-25T14:03:56.772486+0000","last_fullsized":"2024-04-25T14:03:56.772486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:15.364599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494579+0000","last_change":"2024-04-25T14:03:55.676329+0000","last_active":"2024-04-25T14:03:56.494579+0000","last_peered":"2024-04-25T14:03:56.494579+0000","last_clean":"2024-04-25T14:03:56.494579+0000","last_became_active":"2024-04-25T14:03:55.675752+0000","last_became_peered":"2024-04-25T14:03:55.675752+0000","last_unstale":"2024-04-25T14:03:56.494579+0000","last_undegraded":"2024-04-25T14:03:56.494579+0000","last_fullsized":"2024-04-25T14:03:56.494579+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:28.167574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772572+0000","last_change":"2024-04-25T14:03:51.495282+0000","last_active":"2024-04-25T14:03:56.772572+0000","last_peered":"2024-04-25T14:03:56.772572+0000","last_clean":"2024-04-25T14:03:56.772572+0000","last_became_active":"2024-04-25T14:03:51.494965+0000","last_became_peered":"2024-04-25T14:03:51.494965+0000","last_unstale":"2024-04-25T14:03:56.772572+0000","last_undegraded":"2024-04-25T14:03:56.772572+0000","last_fullsized":"2024-04-25T14:03:56.772572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:02.773422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773963+0000","last_change":"2024-04-25T14:03:50.465925+0000","last_active":"2024-04-25T14:03:56.773963+0000","last_peered":"2024-04-25T14:03:56.773963+0000","last_clean":"2024-04-25T14:03:56.773963+0000","last_became_active":"2024-04-25T14:03:50.465495+0000","last_became_peered":"2024-04-25T14:03:50.465495+0000","last_unstale":"2024-04-25T14:03:56.773963+0000","last_undegraded":"2024-04-25T14:03:56.773963+0000","last_fullsized":"2024-04-25T14:03:56.773963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:15.520202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773183+0000","last_change":"2024-04-25T14:03:55.520108+0000","last_active":"2024-04-25T14:03:56.773183+0000","last_peered":"2024-04-25T14:03:56.773183+0000","last_clean":"2024-04-25T14:03:56.773183+0000","last_became_active":"2024-04-25T14:03:55.519374+0000","last_became_peered":"2024-04-25T14:03:55.519374+0000","last_unstale":"2024-04-25T14:03:56.773183+0000","last_undegraded":"2024-04-25T14:03:56.773183+0000","last_fullsized":"2024-04-25T14:03:56.773183+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:21.426226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772332+0000","last_change":"2024-04-25T14:03:51.488205+0000","last_active":"2024-04-25T14:03:56.772332+0000","last_peered":"2024-04-25T14:03:56.772332+0000","last_clean":"2024-04-25T14:03:56.772332+0000","last_became_active":"2024-04-25T14:03:51.487872+0000","last_became_peered":"2024-04-25T14:03:51.487872+0000","last_unstale":"2024-04-25T14:03:56.772332+0000","last_undegraded":"2024-04-25T14:03:56.772332+0000","last_fullsized":"2024-04-25T14:03:56.772332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:55.876775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.771697+0000","last_change":"2024-04-25T14:03:50.475566+0000","last_active":"2024-04-25T14:03:56.771697+0000","last_peered":"2024-04-25T14:03:56.771697+0000","last_clean":"2024-04-25T14:03:56.771697+0000","last_became_active":"2024-04-25T14:03:50.475304+0000","last_became_peered":"2024-04-25T14:03:50.475304+0000","last_unstale":"2024-04-25T14:03:56.771697+0000","last_undegraded":"2024-04-25T14:03:56.771697+0000","last_fullsized":"2024-04-25T14:03:56.771697+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:15.520202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774574+0000","last_change":"2024-04-25T14:03:55.523592+0000","last_active":"2024-04-25T14:03:56.774574+0000","last_peered":"2024-04-25T14:03:56.774574+0000","last_clean":"2024-04-25T14:03:56.774574+0000","last_became_active":"2024-04-25T14:03:55.521767+0000","last_became_peered":"2024-04-25T14:03:55.521767+0000","last_unstale":"2024-04-25T14:03:56.774574+0000","last_undegraded":"2024-04-25T14:03:56.774574+0000","last_fullsized":"2024-04-25T14:03:56.774574+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:50.234674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494779+0000","last_change":"2024-04-25T14:03:51.480142+0000","last_active":"2024-04-25T14:03:56.494779+0000","last_peered":"2024-04-25T14:03:56.494779+0000","last_clean":"2024-04-25T14:03:56.494779+0000","last_became_active":"2024-04-25T14:03:51.479798+0000","last_became_peered":"2024-04-25T14:03:51.479798+0000","last_unstale":"2024-04-25T14:03:56.494779+0000","last_undegraded":"2024-04-25T14:03:56.494779+0000","last_fullsized":"2024-04-25T14:03:56.494779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:31.840842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494679+0000","last_change":"2024-04-25T14:03:50.459894+0000","last_active":"2024-04-25T14:03:56.494679+0000","last_peered":"2024-04-25T14:03:56.494679+0000","last_clean":"2024-04-25T14:03:56.494679+0000","last_became_active":"2024-04-25T14:03:50.459580+0000","last_became_peered":"2024-04-25T14:03:50.459580+0000","last_unstale":"2024-04-25T14:03:56.494679+0000","last_undegraded":"2024-04-25T14:03:56.494679+0000","last_fullsized":"2024-04-25T14:03:56.494679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:29.264084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495170+0000","last_change":"2024-04-25T14:03:55.681298+0000","last_active":"2024-04-25T14:03:56.495170+0000","last_peered":"2024-04-25T14:03:56.495170+0000","last_clean":"2024-04-25T14:03:56.495170+0000","last_became_active":"2024-04-25T14:03:55.680746+0000","last_became_peered":"2024-04-25T14:03:55.680746+0000","last_unstale":"2024-04-25T14:03:56.495170+0000","last_undegraded":"2024-04-25T14:03:56.495170+0000","last_fullsized":"2024-04-25T14:03:56.495170+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:54.496413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772452+0000","last_change":"2024-04-25T14:03:51.499753+0000","last_active":"2024-04-25T14:03:56.772452+0000","last_peered":"2024-04-25T14:03:56.772452+0000","last_clean":"2024-04-25T14:03:56.772452+0000","last_became_active":"2024-04-25T14:03:51.499427+0000","last_became_peered":"2024-04-25T14:03:51.499427+0000","last_unstale":"2024-04-25T14:03:56.772452+0000","last_undegraded":"2024-04-25T14:03:56.772452+0000","last_fullsized":"2024-04-25T14:03:56.772452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:05.489889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495577+0000","last_change":"2024-04-25T14:03:50.467395+0000","last_active":"2024-04-25T14:03:56.495577+0000","last_peered":"2024-04-25T14:03:56.495577+0000","last_clean":"2024-04-25T14:03:56.495577+0000","last_became_active":"2024-04-25T14:03:50.465757+0000","last_became_peered":"2024-04-25T14:03:50.465757+0000","last_unstale":"2024-04-25T14:03:56.495577+0000","last_undegraded":"2024-04-25T14:03:56.495577+0000","last_fullsized":"2024-04-25T14:03:56.495577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:39.630327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494300+0000","last_change":"2024-04-25T14:03:55.676664+0000","last_active":"2024-04-25T14:03:56.494300+0000","last_peered":"2024-04-25T14:03:56.494300+0000","last_clean":"2024-04-25T14:03:56.494300+0000","last_became_active":"2024-04-25T14:03:55.674357+0000","last_became_peered":"2024-04-25T14:03:55.674357+0000","last_unstale":"2024-04-25T14:03:56.494300+0000","last_undegraded":"2024-04-25T14:03:56.494300+0000","last_fullsized":"2024-04-25T14:03:56.494300+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:55:42.758380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772783+0000","last_change":"2024-04-25T14:03:51.469041+0000","last_active":"2024-04-25T14:03:56.772783+0000","last_peered":"2024-04-25T14:03:56.772783+0000","last_clean":"2024-04-25T14:03:56.772783+0000","last_became_active":"2024-04-25T14:03:51.468704+0000","last_became_peered":"2024-04-25T14:03:51.468704+0000","last_unstale":"2024-04-25T14:03:56.772783+0000","last_undegraded":"2024-04-25T14:03:56.772783+0000","last_fullsized":"2024-04-25T14:03:56.772783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:31.840842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773670+0000","last_change":"2024-04-25T14:03:50.465211+0000","last_active":"2024-04-25T14:03:56.773670+0000","last_peered":"2024-04-25T14:03:56.773670+0000","last_clean":"2024-04-25T14:03:56.773670+0000","last_became_active":"2024-04-25T14:03:50.464836+0000","last_became_peered":"2024-04-25T14:03:50.464836+0000","last_unstale":"2024-04-25T14:03:56.773670+0000","last_undegraded":"2024-04-25T14:03:56.773670+0000","last_fullsized":"2024-04-25T14:03:56.773670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:04.876207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496500+0000","last_change":"2024-04-25T14:03:55.681339+0000","last_active":"2024-04-25T14:03:56.496500+0000","last_peered":"2024-04-25T14:03:56.496500+0000","last_clean":"2024-04-25T14:03:56.496500+0000","last_became_active":"2024-04-25T14:03:55.680779+0000","last_became_peered":"2024-04-25T14:03:55.680779+0000","last_unstale":"2024-04-25T14:03:56.496500+0000","last_undegraded":"2024-04-25T14:03:56.496500+0000","last_fullsized":"2024-04-25T14:03:56.496500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:58.844877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772688+0000","last_change":"2024-04-25T14:03:51.476195+0000","last_active":"2024-04-25T14:03:56.772688+0000","last_peered":"2024-04-25T14:03:56.772688+0000","last_clean":"2024-04-25T14:03:56.772688+0000","last_became_active":"2024-04-25T14:03:51.475372+0000","last_became_peered":"2024-04-25T14:03:51.475372+0000","last_unstale":"2024-04-25T14:03:56.772688+0000","last_undegraded":"2024-04-25T14:03:56.772688+0000","last_fullsized":"2024-04-25T14:03:56.772688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:47.326356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:04:06.748698+0000","last_change":"2024-04-25T14:03:50.470612+0000","last_active":"2024-04-25T14:04:06.748698+0000","last_peered":"2024-04-25T14:04:06.748698+0000","last_clean":"2024-04-25T14:04:06.748698+0000","last_became_active":"2024-04-25T14:03:50.470312+0000","last_became_peered":"2024-04-25T14:03:50.470312+0000","last_unstale":"2024-04-25T14:04:06.748698+0000","last_undegraded":"2024-04-25T14:04:06.748698+0000","last_fullsized":"2024-04-25T14:04:06.748698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:10.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773458+0000","last_change":"2024-04-25T14:03:55.526774+0000","last_active":"2024-04-25T14:03:56.773458+0000","last_peered":"2024-04-25T14:03:56.773458+0000","last_clean":"2024-04-25T14:03:56.773458+0000","last_became_active":"2024-04-25T14:03:55.524255+0000","last_became_peered":"2024-04-25T14:03:55.524255+0000","last_unstale":"2024-04-25T14:03:56.773458+0000","last_undegraded":"2024-04-25T14:03:56.773458+0000","last_fullsized":"2024-04-25T14:03:56.773458+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:37.811376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773340+0000","last_change":"2024-04-25T14:03:51.493162+0000","last_active":"2024-04-25T14:03:56.773340+0000","last_peered":"2024-04-25T14:03:56.773340+0000","last_clean":"2024-04-25T14:03:56.773340+0000","last_became_active":"2024-04-25T14:03:51.488560+0000","last_became_peered":"2024-04-25T14:03:51.488560+0000","last_unstale":"2024-04-25T14:03:56.773340+0000","last_undegraded":"2024-04-25T14:03:56.773340+0000","last_fullsized":"2024-04-25T14:03:56.773340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:40.635789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774009+0000","last_change":"2024-04-25T14:03:50.469594+0000","last_active":"2024-04-25T14:03:56.774009+0000","last_peered":"2024-04-25T14:03:56.774009+0000","last_clean":"2024-04-25T14:03:56.774009+0000","last_became_active":"2024-04-25T14:03:50.469196+0000","last_became_peered":"2024-04-25T14:03:50.469196+0000","last_unstale":"2024-04-25T14:03:56.774009+0000","last_undegraded":"2024-04-25T14:03:56.774009+0000","last_fullsized":"2024-04-25T14:03:56.774009+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:15.364599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772969+0000","last_change":"2024-04-25T14:03:55.518510+0000","last_active":"2024-04-25T14:03:56.772969+0000","last_peered":"2024-04-25T14:03:56.772969+0000","last_clean":"2024-04-25T14:03:56.772969+0000","last_became_active":"2024-04-25T14:03:55.515796+0000","last_became_peered":"2024-04-25T14:03:55.515796+0000","last_unstale":"2024-04-25T14:03:56.772969+0000","last_undegraded":"2024-04-25T14:03:56.772969+0000","last_fullsized":"2024-04-25T14:03:56.772969+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:59.921752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774672+0000","last_change":"2024-04-25T14:03:51.486474+0000","last_active":"2024-04-25T14:03:56.774672+0000","last_peered":"2024-04-25T14:03:56.774672+0000","last_clean":"2024-04-25T14:03:56.774672+0000","last_became_active":"2024-04-25T14:03:51.485742+0000","last_became_peered":"2024-04-25T14:03:51.485742+0000","last_unstale":"2024-04-25T14:03:56.774672+0000","last_undegraded":"2024-04-25T14:03:56.774672+0000","last_fullsized":"2024-04-25T14:03:56.774672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:16.813529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496428+0000","last_change":"2024-04-25T14:03:50.475187+0000","last_active":"2024-04-25T14:03:56.496428+0000","last_peered":"2024-04-25T14:03:56.496428+0000","last_clean":"2024-04-25T14:03:56.496428+0000","last_became_active":"2024-04-25T14:03:50.474761+0000","last_became_peered":"2024-04-25T14:03:50.474761+0000","last_unstale":"2024-04-25T14:03:56.496428+0000","last_undegraded":"2024-04-25T14:03:56.496428+0000","last_fullsized":"2024-04-25T14:03:56.496428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:02.117945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493593+0000","last_change":"2024-04-25T14:03:55.524048+0000","last_active":"2024-04-25T14:03:56.493593+0000","last_peered":"2024-04-25T14:03:56.493593+0000","last_clean":"2024-04-25T14:03:56.493593+0000","last_became_active":"2024-04-25T14:03:55.520419+0000","last_became_peered":"2024-04-25T14:03:55.520419+0000","last_unstale":"2024-04-25T14:03:56.493593+0000","last_undegraded":"2024-04-25T14:03:56.493593+0000","last_fullsized":"2024-04-25T14:03:56.493593+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:30.069597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496533+0000","last_change":"2024-04-25T14:03:51.467640+0000","last_active":"2024-04-25T14:03:56.496533+0000","last_peered":"2024-04-25T14:03:56.496533+0000","last_clean":"2024-04-25T14:03:56.496533+0000","last_became_active":"2024-04-25T14:03:51.467347+0000","last_became_peered":"2024-04-25T14:03:51.467347+0000","last_unstale":"2024-04-25T14:03:56.496533+0000","last_undegraded":"2024-04-25T14:03:56.496533+0000","last_fullsized":"2024-04-25T14:03:56.496533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:33.766399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496540+0000","last_change":"2024-04-25T14:03:50.462889+0000","last_active":"2024-04-25T14:03:56.496540+0000","last_peered":"2024-04-25T14:03:56.496540+0000","last_clean":"2024-04-25T14:03:56.496540+0000","last_became_active":"2024-04-25T14:03:50.462529+0000","last_became_peered":"2024-04-25T14:03:50.462529+0000","last_unstale":"2024-04-25T14:03:56.496540+0000","last_undegraded":"2024-04-25T14:03:56.496540+0000","last_fullsized":"2024-04-25T14:03:56.496540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:15.364599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493822+0000","last_change":"2024-04-25T14:03:55.675236+0000","last_active":"2024-04-25T14:03:56.493822+0000","last_peered":"2024-04-25T14:03:56.493822+0000","last_clean":"2024-04-25T14:03:56.493822+0000","last_became_active":"2024-04-25T14:03:55.674759+0000","last_became_peered":"2024-04-25T14:03:55.674759+0000","last_unstale":"2024-04-25T14:03:56.493822+0000","last_undegraded":"2024-04-25T14:03:56.493822+0000","last_fullsized":"2024-04-25T14:03:56.493822+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:28.167574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773362+0000","last_change":"2024-04-25T14:03:55.677958+0000","last_active":"2024-04-25T14:03:56.773362+0000","last_peered":"2024-04-25T14:03:56.773362+0000","last_clean":"2024-04-25T14:03:56.773362+0000","last_became_active":"2024-04-25T14:03:55.677471+0000","last_became_peered":"2024-04-25T14:03:55.677471+0000","last_unstale":"2024-04-25T14:03:56.773362+0000","last_undegraded":"2024-04-25T14:03:56.773362+0000","last_fullsized":"2024-04-25T14:03:56.773362+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:12.890323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772866+0000","last_change":"2024-04-25T14:03:51.477601+0000","last_active":"2024-04-25T14:03:56.772866+0000","last_peered":"2024-04-25T14:03:56.772866+0000","last_clean":"2024-04-25T14:03:56.772866+0000","last_became_active":"2024-04-25T14:03:51.477318+0000","last_became_peered":"2024-04-25T14:03:51.477318+0000","last_unstale":"2024-04-25T14:03:56.772866+0000","last_undegraded":"2024-04-25T14:03:56.772866+0000","last_fullsized":"2024-04-25T14:03:56.772866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.599283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494934+0000","last_change":"2024-04-25T14:03:50.456493+0000","last_active":"2024-04-25T14:03:56.494934+0000","last_peered":"2024-04-25T14:03:56.494934+0000","last_clean":"2024-04-25T14:03:56.494934+0000","last_became_active":"2024-04-25T14:03:50.454242+0000","last_became_peered":"2024-04-25T14:03:50.454242+0000","last_unstale":"2024-04-25T14:03:56.494934+0000","last_undegraded":"2024-04-25T14:03:56.494934+0000","last_fullsized":"2024-04-25T14:03:56.494934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:10.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496401+0000","last_change":"2024-04-25T14:03:55.516548+0000","last_active":"2024-04-25T14:03:56.496401+0000","last_peered":"2024-04-25T14:03:56.496401+0000","last_clean":"2024-04-25T14:03:56.496401+0000","last_became_active":"2024-04-25T14:03:55.515311+0000","last_became_peered":"2024-04-25T14:03:55.515311+0000","last_unstale":"2024-04-25T14:03:56.496401+0000","last_undegraded":"2024-04-25T14:03:56.496401+0000","last_fullsized":"2024-04-25T14:03:56.496401+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:51.371333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493692+0000","last_change":"2024-04-25T14:03:51.468740+0000","last_active":"2024-04-25T14:03:56.493692+0000","last_peered":"2024-04-25T14:03:56.493692+0000","last_clean":"2024-04-25T14:03:56.493692+0000","last_became_active":"2024-04-25T14:03:51.468366+0000","last_became_peered":"2024-04-25T14:03:51.468366+0000","last_unstale":"2024-04-25T14:03:56.493692+0000","last_undegraded":"2024-04-25T14:03:56.493692+0000","last_fullsized":"2024-04-25T14:03:56.493692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:31.707048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493992+0000","last_change":"2024-04-25T14:03:50.466457+0000","last_active":"2024-04-25T14:03:56.493992+0000","last_peered":"2024-04-25T14:03:56.493992+0000","last_clean":"2024-04-25T14:03:56.493992+0000","last_became_active":"2024-04-25T14:03:50.466070+0000","last_became_peered":"2024-04-25T14:03:50.466070+0000","last_unstale":"2024-04-25T14:03:56.493992+0000","last_undegraded":"2024-04-25T14:03:56.493992+0000","last_fullsized":"2024-04-25T14:03:56.493992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:15.520202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495294+0000","last_change":"2024-04-25T14:03:50.469128+0000","last_active":"2024-04-25T14:03:56.495294+0000","last_peered":"2024-04-25T14:03:56.495294+0000","last_clean":"2024-04-25T14:03:56.495294+0000","last_became_active":"2024-04-25T14:03:50.468409+0000","last_became_peered":"2024-04-25T14:03:50.468409+0000","last_unstale":"2024-04-25T14:03:56.495294+0000","last_undegraded":"2024-04-25T14:03:56.495294+0000","last_fullsized":"2024-04-25T14:03:56.495294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:15.520202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773570+0000","last_change":"2024-04-25T14:03:51.502539+0000","last_active":"2024-04-25T14:03:56.773570+0000","last_peered":"2024-04-25T14:03:56.773570+0000","last_clean":"2024-04-25T14:03:56.773570+0000","last_became_active":"2024-04-25T14:03:51.502229+0000","last_became_peered":"2024-04-25T14:03:51.502229+0000","last_unstale":"2024-04-25T14:03:56.773570+0000","last_undegraded":"2024-04-25T14:03:56.773570+0000","last_fullsized":"2024-04-25T14:03:56.773570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:20.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772371+0000","last_change":"2024-04-25T14:03:55.524446+0000","last_active":"2024-04-25T14:03:56.772371+0000","last_peered":"2024-04-25T14:03:56.772371+0000","last_clean":"2024-04-25T14:03:56.772371+0000","last_became_active":"2024-04-25T14:03:55.523886+0000","last_became_peered":"2024-04-25T14:03:55.523886+0000","last_unstale":"2024-04-25T14:03:56.772371+0000","last_undegraded":"2024-04-25T14:03:56.772371+0000","last_fullsized":"2024-04-25T14:03:56.772371+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:37.811235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493654+0000","last_change":"2024-04-25T14:03:50.472564+0000","last_active":"2024-04-25T14:03:56.493654+0000","last_peered":"2024-04-25T14:03:56.493654+0000","last_clean":"2024-04-25T14:03:56.493654+0000","last_became_active":"2024-04-25T14:03:50.472297+0000","last_became_peered":"2024-04-25T14:03:50.472297+0000","last_unstale":"2024-04-25T14:03:56.493654+0000","last_undegraded":"2024-04-25T14:03:56.493654+0000","last_fullsized":"2024-04-25T14:03:56.493654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:43.281768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772419+0000","last_change":"2024-04-25T14:03:51.486031+0000","last_active":"2024-04-25T14:03:56.772419+0000","last_peered":"2024-04-25T14:03:56.772419+0000","last_clean":"2024-04-25T14:03:56.772419+0000","last_became_active":"2024-04-25T14:03:51.477708+0000","last_became_peered":"2024-04-25T14:03:51.477708+0000","last_unstale":"2024-04-25T14:03:56.772419+0000","last_undegraded":"2024-04-25T14:03:56.772419+0000","last_fullsized":"2024-04-25T14:03:56.772419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:11.859288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494673+0000","last_change":"2024-04-25T14:03:55.519123+0000","last_active":"2024-04-25T14:03:56.494673+0000","last_peered":"2024-04-25T14:03:56.494673+0000","last_clean":"2024-04-25T14:03:56.494673+0000","last_became_active":"2024-04-25T14:03:55.518743+0000","last_became_peered":"2024-04-25T14:03:55.518743+0000","last_unstale":"2024-04-25T14:03:56.494673+0000","last_undegraded":"2024-04-25T14:03:56.494673+0000","last_fullsized":"2024-04-25T14:03:56.494673+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:18.368759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494540+0000","last_change":"2024-04-25T14:03:51.480209+0000","last_active":"2024-04-25T14:03:56.494540+0000","last_peered":"2024-04-25T14:03:56.494540+0000","last_clean":"2024-04-25T14:03:56.494540+0000","last_became_active":"2024-04-25T14:03:51.479824+0000","last_became_peered":"2024-04-25T14:03:51.479824+0000","last_unstale":"2024-04-25T14:03:56.494540+0000","last_undegraded":"2024-04-25T14:03:56.494540+0000","last_fullsized":"2024-04-25T14:03:56.494540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:31.707048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.771783+0000","last_change":"2024-04-25T14:03:51.486748+0000","last_active":"2024-04-25T14:03:56.771783+0000","last_peered":"2024-04-25T14:03:56.771783+0000","last_clean":"2024-04-25T14:03:56.771783+0000","last_became_active":"2024-04-25T14:03:51.486343+0000","last_became_peered":"2024-04-25T14:03:51.486343+0000","last_unstale":"2024-04-25T14:03:56.771783+0000","last_undegraded":"2024-04-25T14:03:56.771783+0000","last_fullsized":"2024-04-25T14:03:56.771783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:46.086179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495470+0000","last_change":"2024-04-25T14:03:51.465637+0000","last_active":"2024-04-25T14:03:56.495470+0000","last_peered":"2024-04-25T14:03:56.495470+0000","last_clean":"2024-04-25T14:03:56.495470+0000","last_became_active":"2024-04-25T14:03:51.465339+0000","last_became_peered":"2024-04-25T14:03:51.465339+0000","last_unstale":"2024-04-25T14:03:56.495470+0000","last_undegraded":"2024-04-25T14:03:56.495470+0000","last_fullsized":"2024-04-25T14:03:56.495470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:21.406920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773508+0000","last_change":"2024-04-25T14:03:51.475134+0000","last_active":"2024-04-25T14:03:56.773508+0000","last_peered":"2024-04-25T14:03:56.773508+0000","last_clean":"2024-04-25T14:03:56.773508+0000","last_became_active":"2024-04-25T14:03:51.474796+0000","last_became_peered":"2024-04-25T14:03:51.474796+0000","last_unstale":"2024-04-25T14:03:56.773508+0000","last_undegraded":"2024-04-25T14:03:56.773508+0000","last_fullsized":"2024-04-25T14:03:56.773508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:31.707048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494247+0000","last_change":"2024-04-25T14:03:51.494845+0000","last_active":"2024-04-25T14:03:56.494247+0000","last_peered":"2024-04-25T14:03:56.494247+0000","last_clean":"2024-04-25T14:03:56.494247+0000","last_became_active":"2024-04-25T14:03:51.494547+0000","last_became_peered":"2024-04-25T14:03:51.494547+0000","last_unstale":"2024-04-25T14:03:56.494247+0000","last_undegraded":"2024-04-25T14:03:56.494247+0000","last_fullsized":"2024-04-25T14:03:56.494247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:16.813529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773722+0000","last_change":"2024-04-25T14:03:51.478241+0000","last_active":"2024-04-25T14:03:56.773722+0000","last_peered":"2024-04-25T14:03:56.773722+0000","last_clean":"2024-04-25T14:03:56.773722+0000","last_became_active":"2024-04-25T14:03:51.477955+0000","last_became_peered":"2024-04-25T14:03:51.477955+0000","last_unstale":"2024-04-25T14:03:56.773722+0000","last_undegraded":"2024-04-25T14:03:56.773722+0000","last_fullsized":"2024-04-25T14:03:56.773722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:43.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496093+0000","last_change":"2024-04-25T14:03:51.478644+0000","last_active":"2024-04-25T14:03:56.496093+0000","last_peered":"2024-04-25T14:03:56.496093+0000","last_clean":"2024-04-25T14:03:56.496093+0000","last_became_active":"2024-04-25T14:03:51.478326+0000","last_became_peered":"2024-04-25T14:03:51.478326+0000","last_unstale":"2024-04-25T14:03:56.496093+0000","last_undegraded":"2024-04-25T14:03:56.496093+0000","last_fullsized":"2024-04-25T14:03:56.496093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:20.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495116+0000","last_change":"2024-04-25T14:03:51.506061+0000","last_active":"2024-04-25T14:03:56.495116+0000","last_peered":"2024-04-25T14:03:56.495116+0000","last_clean":"2024-04-25T14:03:56.495116+0000","last_became_active":"2024-04-25T14:03:51.505804+0000","last_became_peered":"2024-04-25T14:03:51.505804+0000","last_unstale":"2024-04-25T14:03:56.495116+0000","last_undegraded":"2024-04-25T14:03:56.495116+0000","last_fullsized":"2024-04-25T14:03:56.495116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:55.876775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494831+0000","last_change":"2024-04-25T14:03:51.465574+0000","last_active":"2024-04-25T14:03:56.494831+0000","last_peered":"2024-04-25T14:03:56.494831+0000","last_clean":"2024-04-25T14:03:56.494831+0000","last_became_active":"2024-04-25T14:03:51.465206+0000","last_became_peered":"2024-04-25T14:03:51.465206+0000","last_unstale":"2024-04-25T14:03:56.494831+0000","last_undegraded":"2024-04-25T14:03:56.494831+0000","last_fullsized":"2024-04-25T14:03:56.494831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:16.813529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493905+0000","last_change":"2024-04-25T14:03:51.486782+0000","last_active":"2024-04-25T14:03:56.493905+0000","last_peered":"2024-04-25T14:03:56.493905+0000","last_clean":"2024-04-25T14:03:56.493905+0000","last_became_active":"2024-04-25T14:03:51.486465+0000","last_became_peered":"2024-04-25T14:03:51.486465+0000","last_unstale":"2024-04-25T14:03:56.493905+0000","last_undegraded":"2024-04-25T14:03:56.493905+0000","last_fullsized":"2024-04-25T14:03:56.493905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:39.949870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495357+0000","last_change":"2024-04-25T14:03:51.468022+0000","last_active":"2024-04-25T14:03:56.495357+0000","last_peered":"2024-04-25T14:03:56.495357+0000","last_clean":"2024-04-25T14:03:56.495357+0000","last_became_active":"2024-04-25T14:03:51.467637+0000","last_became_peered":"2024-04-25T14:03:51.467637+0000","last_unstale":"2024-04-25T14:03:56.495357+0000","last_undegraded":"2024-04-25T14:03:56.495357+0000","last_fullsized":"2024-04-25T14:03:56.495357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:55.876775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773216+0000","last_change":"2024-04-25T14:03:51.469881+0000","last_active":"2024-04-25T14:03:56.773216+0000","last_peered":"2024-04-25T14:03:56.773216+0000","last_clean":"2024-04-25T14:03:56.773216+0000","last_became_active":"2024-04-25T14:03:51.469572+0000","last_became_peered":"2024-04-25T14:03:51.469572+0000","last_unstale":"2024-04-25T14:03:56.773216+0000","last_undegraded":"2024-04-25T14:03:56.773216+0000","last_fullsized":"2024-04-25T14:03:56.773216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.599283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495680+0000","last_change":"2024-04-25T14:03:51.494605+0000","last_active":"2024-04-25T14:03:56.495680+0000","last_peered":"2024-04-25T14:03:56.495680+0000","last_clean":"2024-04-25T14:03:56.495680+0000","last_became_active":"2024-04-25T14:03:51.494257+0000","last_became_peered":"2024-04-25T14:03:51.494257+0000","last_unstale":"2024-04-25T14:03:56.495680+0000","last_undegraded":"2024-04-25T14:03:56.495680+0000","last_fullsized":"2024-04-25T14:03:56.495680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:39.949870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773065+0000","last_change":"2024-04-25T14:03:51.476473+0000","last_active":"2024-04-25T14:03:56.773065+0000","last_peered":"2024-04-25T14:03:56.773065+0000","last_clean":"2024-04-25T14:03:56.773065+0000","last_became_active":"2024-04-25T14:03:51.475857+0000","last_became_peered":"2024-04-25T14:03:51.475857+0000","last_unstale":"2024-04-25T14:03:56.773065+0000","last_undegraded":"2024-04-25T14:03:56.773065+0000","last_fullsized":"2024-04-25T14:03:56.773065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:46.189697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772375+0000","last_change":"2024-04-25T14:03:51.479889+0000","last_active":"2024-04-25T14:03:56.772375+0000","last_peered":"2024-04-25T14:03:56.772375+0000","last_clean":"2024-04-25T14:03:56.772375+0000","last_became_active":"2024-04-25T14:03:51.478793+0000","last_became_peered":"2024-04-25T14:03:51.478793+0000","last_unstale":"2024-04-25T14:03:56.772375+0000","last_undegraded":"2024-04-25T14:03:56.772375+0000","last_fullsized":"2024-04-25T14:03:56.772375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:38.411753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496637+0000","last_change":"2024-04-25T14:03:51.480313+0000","last_active":"2024-04-25T14:03:56.496637+0000","last_peered":"2024-04-25T14:03:56.496637+0000","last_clean":"2024-04-25T14:03:56.496637+0000","last_became_active":"2024-04-25T14:03:51.479875+0000","last_became_peered":"2024-04-25T14:03:51.479875+0000","last_unstale":"2024-04-25T14:03:56.496637+0000","last_undegraded":"2024-04-25T14:03:56.496637+0000","last_fullsized":"2024-04-25T14:03:56.496637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:02.773422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494729+0000","last_change":"2024-04-25T14:03:51.472514+0000","last_active":"2024-04-25T14:03:56.494729+0000","last_peered":"2024-04-25T14:03:56.494729+0000","last_clean":"2024-04-25T14:03:56.494729+0000","last_became_active":"2024-04-25T14:03:51.472148+0000","last_became_peered":"2024-04-25T14:03:51.472148+0000","last_unstale":"2024-04-25T14:03:56.494729+0000","last_undegraded":"2024-04-25T14:03:56.494729+0000","last_fullsized":"2024-04-25T14:03:56.494729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:20.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772899+0000","last_change":"2024-04-25T14:03:51.489086+0000","last_active":"2024-04-25T14:03:56.772899+0000","last_peered":"2024-04-25T14:03:56.772899+0000","last_clean":"2024-04-25T14:03:56.772899+0000","last_became_active":"2024-04-25T14:03:51.488810+0000","last_became_peered":"2024-04-25T14:03:51.488810+0000","last_unstale":"2024-04-25T14:03:56.772899+0000","last_undegraded":"2024-04-25T14:03:56.772899+0000","last_fullsized":"2024-04-25T14:03:56.772899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:39.949870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772674+0000","last_change":"2024-04-25T14:03:51.475508+0000","last_active":"2024-04-25T14:03:56.772674+0000","last_peered":"2024-04-25T14:03:56.772674+0000","last_clean":"2024-04-25T14:03:56.772674+0000","last_became_active":"2024-04-25T14:03:51.475146+0000","last_became_peered":"2024-04-25T14:03:51.475146+0000","last_unstale":"2024-04-25T14:03:56.772674+0000","last_undegraded":"2024-04-25T14:03:56.772674+0000","last_fullsized":"2024-04-25T14:03:56.772674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:34.374126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493554+0000","last_change":"2024-04-25T14:03:51.479243+0000","last_active":"2024-04-25T14:03:56.493554+0000","last_peered":"2024-04-25T14:03:56.493554+0000","last_clean":"2024-04-25T14:03:56.493554+0000","last_became_active":"2024-04-25T14:03:51.478899+0000","last_became_peered":"2024-04-25T14:03:51.478899+0000","last_unstale":"2024-04-25T14:03:56.493554+0000","last_undegraded":"2024-04-25T14:03:56.493554+0000","last_fullsized":"2024-04-25T14:03:56.493554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:20.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773258+0000","last_change":"2024-04-25T14:03:51.493878+0000","last_active":"2024-04-25T14:03:56.773258+0000","last_peered":"2024-04-25T14:03:56.773258+0000","last_clean":"2024-04-25T14:03:56.773258+0000","last_became_active":"2024-04-25T14:03:51.493501+0000","last_became_peered":"2024-04-25T14:03:51.493501+0000","last_unstale":"2024-04-25T14:03:56.773258+0000","last_undegraded":"2024-04-25T14:03:56.773258+0000","last_fullsized":"2024-04-25T14:03:56.773258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:46.086179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496241+0000","last_change":"2024-04-25T14:03:51.489072+0000","last_active":"2024-04-25T14:03:56.496241+0000","last_peered":"2024-04-25T14:03:56.496241+0000","last_clean":"2024-04-25T14:03:56.496241+0000","last_became_active":"2024-04-25T14:03:51.488895+0000","last_became_peered":"2024-04-25T14:03:51.488895+0000","last_unstale":"2024-04-25T14:03:56.496241+0000","last_undegraded":"2024-04-25T14:03:56.496241+0000","last_fullsized":"2024-04-25T14:03:56.496241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:10.318378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496156+0000","last_change":"2024-04-25T14:03:51.488196+0000","last_active":"2024-04-25T14:03:56.496156+0000","last_peered":"2024-04-25T14:03:56.496156+0000","last_clean":"2024-04-25T14:03:56.496156+0000","last_became_active":"2024-04-25T14:03:51.487833+0000","last_became_peered":"2024-04-25T14:03:51.487833+0000","last_unstale":"2024-04-25T14:03:56.496156+0000","last_undegraded":"2024-04-25T14:03:56.496156+0000","last_fullsized":"2024-04-25T14:03:56.496156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:43.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773432+0000","last_change":"2024-04-25T14:03:51.482441+0000","last_active":"2024-04-25T14:03:56.773432+0000","last_peered":"2024-04-25T14:03:56.773432+0000","last_clean":"2024-04-25T14:03:56.773432+0000","last_became_active":"2024-04-25T14:03:51.482089+0000","last_became_peered":"2024-04-25T14:03:51.482089+0000","last_unstale":"2024-04-25T14:03:56.773432+0000","last_undegraded":"2024-04-25T14:03:56.773432+0000","last_fullsized":"2024-04-25T14:03:56.773432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:02.697952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493326+0000","last_change":"2024-04-25T14:03:51.487289+0000","last_active":"2024-04-25T14:03:56.493326+0000","last_peered":"2024-04-25T14:03:56.493326+0000","last_clean":"2024-04-25T14:03:56.493326+0000","last_became_active":"2024-04-25T14:03:51.486948+0000","last_became_peered":"2024-04-25T14:03:51.486948+0000","last_unstale":"2024-04-25T14:03:56.493326+0000","last_undegraded":"2024-04-25T14:03:56.493326+0000","last_fullsized":"2024-04-25T14:03:56.493326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.599283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772227+0000","last_change":"2024-04-25T14:03:51.473228+0000","last_active":"2024-04-25T14:03:56.772227+0000","last_peered":"2024-04-25T14:03:56.772227+0000","last_clean":"2024-04-25T14:03:56.772227+0000","last_became_active":"2024-04-25T14:03:51.472850+0000","last_became_peered":"2024-04-25T14:03:51.472850+0000","last_unstale":"2024-04-25T14:03:56.772227+0000","last_undegraded":"2024-04-25T14:03:56.772227+0000","last_fullsized":"2024-04-25T14:03:56.772227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:34.707119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495222+0000","last_change":"2024-04-25T14:03:51.467162+0000","last_active":"2024-04-25T14:03:56.495222+0000","last_peered":"2024-04-25T14:03:56.495222+0000","last_clean":"2024-04-25T14:03:56.495222+0000","last_became_active":"2024-04-25T14:03:51.466616+0000","last_became_peered":"2024-04-25T14:03:51.466616+0000","last_unstale":"2024-04-25T14:03:56.495222+0000","last_undegraded":"2024-04-25T14:03:56.495222+0000","last_fullsized":"2024-04-25T14:03:56.495222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:34.707119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772199+0000","last_change":"2024-04-25T14:03:51.476149+0000","last_active":"2024-04-25T14:03:56.772199+0000","last_peered":"2024-04-25T14:03:56.772199+0000","last_clean":"2024-04-25T14:03:56.772199+0000","last_became_active":"2024-04-25T14:03:51.475934+0000","last_became_peered":"2024-04-25T14:03:51.475934+0000","last_unstale":"2024-04-25T14:03:56.772199+0000","last_undegraded":"2024-04-25T14:03:56.772199+0000","last_fullsized":"2024-04-25T14:03:56.772199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:17.381249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496219+0000","last_change":"2024-04-25T14:03:51.475153+0000","last_active":"2024-04-25T14:03:56.496219+0000","last_peered":"2024-04-25T14:03:56.496219+0000","last_clean":"2024-04-25T14:03:56.496219+0000","last_became_active":"2024-04-25T14:03:51.474862+0000","last_became_peered":"2024-04-25T14:03:51.474862+0000","last_unstale":"2024-04-25T14:03:56.496219+0000","last_undegraded":"2024-04-25T14:03:56.496219+0000","last_fullsized":"2024-04-25T14:03:56.496219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:05.489889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773218+0000","last_change":"2024-04-25T14:03:51.460567+0000","last_active":"2024-04-25T14:03:56.773218+0000","last_peered":"2024-04-25T14:03:56.773218+0000","last_clean":"2024-04-25T14:03:56.773218+0000","last_became_active":"2024-04-25T14:03:51.460193+0000","last_became_peered":"2024-04-25T14:03:51.460193+0000","last_unstale":"2024-04-25T14:03:56.773218+0000","last_undegraded":"2024-04-25T14:03:56.773218+0000","last_fullsized":"2024-04-25T14:03:56.773218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:39.949870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774153+0000","last_change":"2024-04-25T14:03:51.480094+0000","last_active":"2024-04-25T14:03:56.774153+0000","last_peered":"2024-04-25T14:03:56.774153+0000","last_clean":"2024-04-25T14:03:56.774153+0000","last_became_active":"2024-04-25T14:03:51.479529+0000","last_became_peered":"2024-04-25T14:03:51.479529+0000","last_unstale":"2024-04-25T14:03:56.774153+0000","last_undegraded":"2024-04-25T14:03:56.774153+0000","last_fullsized":"2024-04-25T14:03:56.774153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:10.318378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773366+0000","last_change":"2024-04-25T14:03:51.481397+0000","last_active":"2024-04-25T14:03:56.773366+0000","last_peered":"2024-04-25T14:03:56.773366+0000","last_clean":"2024-04-25T14:03:56.773366+0000","last_became_active":"2024-04-25T14:03:51.481153+0000","last_became_peered":"2024-04-25T14:03:51.481153+0000","last_unstale":"2024-04-25T14:03:56.773366+0000","last_undegraded":"2024-04-25T14:03:56.773366+0000","last_fullsized":"2024-04-25T14:03:56.773366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:21.406920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:04:15.708 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:04:15.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.889+0000 7f9099000640 1 Processor -- start 2024-04-25T14:04:15.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.889+0000 7f9099000640 1 -- start start 2024-04-25T14:04:15.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.893+0000 7f9099000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9094072630 0x7f9094072a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.893+0000 7f9099000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9094163980 0x7f9094163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.893+0000 7f9099000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9094072f80 con 0x7f909406e340 2024-04-25T14:04:15.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.893+0000 7f9099000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f90941648e0 con 0x7f9094163980 2024-04-25T14:04:15.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.893+0000 7f9099000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f90940d1520 con 0x7f9094072630 2024-04-25T14:04:15.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.893+0000 7f9089400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9094163980 0x7f9094163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.893+0000 7f9091e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9094072630 0x7f9094072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.893+0000 7f9092800640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f909406e340 0x7f909406e750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:50904/0 (socket says 172.21.15.86:50904) 2024-04-25T14:04:15.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.893+0000 7f9089400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9094163980 0x7f9094163d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:54630/0 (socket says 172.21.15.86:54630) 2024-04-25T14:04:15.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.893+0000 7f9092800640 1 -- 172.21.15.86:0/1378724074 learned_addr learned my addr 172.21.15.86:0/1378724074 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:15.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.893+0000 7f9091400640 1 -- 172.21.15.86:0/1378724074 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1226192757 0 0) 0x7f908c001120 con 0x7f909406e340 2024-04-25T14:04:15.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.893+0000 7f9091e00640 1 -- 172.21.15.86:0/1378724074 >> v2:172.21.15.155:3300/0 conn(0x7f9094163980 msgr2=0x7f9094163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:15.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.893+0000 7f9091e00640 1 --2- 172.21.15.86:0/1378724074 >> v2:172.21.15.155:3300/0 conn(0x7f9094163980 0x7f9094163d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.893+0000 7f9091e00640 1 -- 172.21.15.86:0/1378724074 >> v1:172.21.15.155:6789/0 conn(0x7f909406e340 legacy=0x7f909406e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.893+0000 7f9091e00640 1 -- 172.21.15.86:0/1378724074 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90940d23e0 con 0x7f9094072630 2024-04-25T14:04:15.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.893+0000 7f9089400640 1 --2- 172.21.15.86:0/1378724074 >> v2:172.21.15.155:3300/0 conn(0x7f9094163980 0x7f9094163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:15.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.897+0000 7f9091e00640 1 --2- 172.21.15.86:0/1378724074 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9094072630 0x7f9094072a40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9084009d90 tx=0x7f908401c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7874f9bdad9e2eba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:15.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.897+0000 7f9091400640 1 -- 172.21.15.86:0/1378724074 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90840b2070 con 0x7f9094072630 2024-04-25T14:04:15.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.897+0000 7f9091400640 1 -- 172.21.15.86:0/1378724074 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90840a5710 con 0x7f9094072630 2024-04-25T14:04:15.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.897+0000 7f9091400640 1 -- 172.21.15.86:0/1378724074 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90840a5ae0 con 0x7f9094072630 2024-04-25T14:04:15.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.897+0000 7f9099000640 1 -- 172.21.15.86:0/1378724074 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9094072630 msgr2=0x7f9094072a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.897+0000 7f9099000640 1 --2- 172.21.15.86:0/1378724074 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9094072630 0x7f9094072a40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9084009d90 tx=0x7f908401c850 comp rx=0 tx=0).stop 2024-04-25T14:04:15.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.897+0000 7f9099000640 1 -- 172.21.15.86:0/1378724074 shutdown_connections 2024-04-25T14:04:15.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.897+0000 7f9099000640 1 --2- 172.21.15.86:0/1378724074 >> v2:172.21.15.155:3300/0 conn(0x7f9094163980 0x7f9094163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.897+0000 7f9099000640 1 --2- 172.21.15.86:0/1378724074 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9094072630 0x7f9094072a40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.897+0000 7f9099000640 1 -- 172.21.15.86:0/1378724074 >> 172.21.15.86:0/1378724074 conn(0x7f90940d3580 msgr2=0x7f90940d3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:15.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.897+0000 7f9099000640 1 -- 172.21.15.86:0/1378724074 shutdown_connections 2024-04-25T14:04:15.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.897+0000 7f9099000640 1 -- 172.21.15.86:0/1378724074 wait complete. 2024-04-25T14:04:15.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.897+0000 7f9099000640 1 Processor -- start 2024-04-25T14:04:15.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.897+0000 7f9099000640 1 -- start start 2024-04-25T14:04:15.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.901+0000 7f9099000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f909406e340 0x7f9094159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.901+0000 7f9099000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9094072630 0x7f90940c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.901+0000 7f9099000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9094163980 0x7f90940c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.901+0000 7f9099000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f90940cfe30 con 0x7f9094163980 2024-04-25T14:04:15.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.901+0000 7f9099000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f90940cfcb0 con 0x7f909406e340 2024-04-25T14:04:15.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.901+0000 7f9099000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f909416d530 con 0x7f9094072630 2024-04-25T14:04:15.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.901+0000 7f9092800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9094163980 0x7f90940c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.901+0000 7f9092800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9094163980 0x7f90940c83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38278/0 (socket says 172.21.15.86:38278) 2024-04-25T14:04:15.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.901+0000 7f9092800640 1 -- 172.21.15.86:0/1854527324 learned_addr learned my addr 172.21.15.86:0/1854527324 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:15.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.901+0000 7f9091e00640 1 --2- 172.21.15.86:0/1854527324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f909406e340 0x7f9094159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.901+0000 7f9089400640 1 --2- 172.21.15.86:0/1854527324 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9094072630 0x7f90940c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.901+0000 7f9092800640 1 -- 172.21.15.86:0/1854527324 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9094072630 msgr2=0x7f90940c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.901+0000 7f9092800640 1 --2- 172.21.15.86:0/1854527324 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9094072630 0x7f90940c7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.901+0000 7f9092800640 1 -- 172.21.15.86:0/1854527324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f909406e340 msgr2=0x7f9094159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.901+0000 7f9092800640 1 --2- 172.21.15.86:0/1854527324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f909406e340 0x7f9094159400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.901+0000 7f9092800640 1 -- 172.21.15.86:0/1854527324 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90940b78f0 con 0x7f9094163980 2024-04-25T14:04:15.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.901+0000 7f9089400640 1 --2- 172.21.15.86:0/1854527324 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9094072630 0x7f90940c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:15.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.901+0000 7f9092800640 1 --2- 172.21.15.86:0/1854527324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9094163980 0x7f90940c83a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f908c00c440 tx=0x7f908c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:15.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.901+0000 7f908aa00640 1 -- 172.21.15.86:0/1854527324 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c017070 con 0x7f9094163980 2024-04-25T14:04:15.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.901+0000 7f908aa00640 1 -- 172.21.15.86:0/1854527324 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f908c005cf0 con 0x7f9094163980 2024-04-25T14:04:15.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.901+0000 7f9099000640 1 -- 172.21.15.86:0/1854527324 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90940b7bd0 con 0x7f9094163980 2024-04-25T14:04:15.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.901+0000 7f908aa00640 1 -- 172.21.15.86:0/1854527324 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c007310 con 0x7f9094163980 2024-04-25T14:04:15.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.901+0000 7f9099000640 1 -- 172.21.15.86:0/1854527324 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90940b8190 con 0x7f9094163980 2024-04-25T14:04:15.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.901+0000 7f9099000640 1 -- 172.21.15.86:0/1854527324 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90940ca050 con 0x7f9094163980 2024-04-25T14:04:15.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.905+0000 7f908aa00640 1 -- 172.21.15.86:0/1854527324 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f908c01a070 con 0x7f9094163980 2024-04-25T14:04:15.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.905+0000 7f908aa00640 1 --2- 172.21.15.86:0/1854527324 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f907006f400 0x7f9070071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.905+0000 7f908aa00640 1 -- 172.21.15.86:0/1854527324 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f908c08e940 con 0x7f9094163980 2024-04-25T14:04:15.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.905+0000 7f9091e00640 1 --2- 172.21.15.86:0/1854527324 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f907006f400 0x7f9070071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.905+0000 7f9091e00640 1 --2- 172.21.15.86:0/1854527324 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f907006f400 0x7f9070071910 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f90840af000 tx=0x7f90840a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:15.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:15.909+0000 7f908aa00640 1 -- 172.21.15.86:0/1854527324 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9094072f80 con 0x7f9094163980 2024-04-25T14:04:16.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.061+0000 7f9099000640 1 -- 172.21.15.86:0/1854527324 --> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9094072170 con 0x7f907006f400 2024-04-25T14:04:16.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.073+0000 7f908aa00640 1 -- 172.21.15.86:0/1854527324 <== mgr.4103 v2:172.21.15.86:6832/2968869755 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332527 (secure 0 0 0) 0x7f9094072170 con 0x7f907006f400 2024-04-25T14:04:16.076 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:16.078 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:04:16.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.077+0000 7f9099000640 1 -- 172.21.15.86:0/1854527324 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f907006f400 msgr2=0x7f9070071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:16.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.077+0000 7f9099000640 1 --2- 172.21.15.86:0/1854527324 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f907006f400 0x7f9070071910 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f90840af000 tx=0x7f90840a7070 comp rx=0 tx=0).stop 2024-04-25T14:04:16.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.077+0000 7f9099000640 1 -- 172.21.15.86:0/1854527324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9094163980 msgr2=0x7f90940c83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:16.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.077+0000 7f9099000640 1 --2- 172.21.15.86:0/1854527324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9094163980 0x7f90940c83a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f908c00c440 tx=0x7f908c00c900 comp rx=0 tx=0).stop 2024-04-25T14:04:16.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.077+0000 7f9099000640 1 -- 172.21.15.86:0/1854527324 shutdown_connections 2024-04-25T14:04:16.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.077+0000 7f9099000640 1 --2- 172.21.15.86:0/1854527324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9094163980 0x7f90940c83a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.077+0000 7f9099000640 1 --2- 172.21.15.86:0/1854527324 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f907006f400 0x7f9070071910 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.077+0000 7f9099000640 1 --2- 172.21.15.86:0/1854527324 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9094072630 0x7f90940c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.077+0000 7f9099000640 1 --2- 172.21.15.86:0/1854527324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f909406e340 0x7f9094159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.077+0000 7f9099000640 1 -- 172.21.15.86:0/1854527324 >> 172.21.15.86:0/1854527324 conn(0x7f90940d3580 msgr2=0x7f909415a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:16.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.077+0000 7f9099000640 1 -- 172.21.15.86:0/1854527324 shutdown_connections 2024-04-25T14:04:16.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.077+0000 7f9099000640 1 -- 172.21.15.86:0/1854527324 wait complete. 2024-04-25T14:04:16.097 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-04-25T14:04:14.715700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":150,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219400,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755320,"statfs":{"total":773094113280,"available":772869447680,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002893"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774780+0000","last_change":"2024-04-25T14:03:50.477862+0000","last_active":"2024-04-25T14:03:56.774780+0000","last_peered":"2024-04-25T14:03:56.774780+0000","last_clean":"2024-04-25T14:03:56.774780+0000","last_became_active":"2024-04-25T14:03:50.477375+0000","last_became_peered":"2024-04-25T14:03:50.477375+0000","last_unstale":"2024-04-25T14:03:56.774780+0000","last_undegraded":"2024-04-25T14:03:56.774780+0000","last_fullsized":"2024-04-25T14:03:56.774780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:01.852459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494933+0000","last_change":"2024-04-25T14:03:51.479650+0000","last_active":"2024-04-25T14:03:56.494933+0000","last_peered":"2024-04-25T14:03:56.494933+0000","last_clean":"2024-04-25T14:03:56.494933+0000","last_became_active":"2024-04-25T14:03:51.479295+0000","last_became_peered":"2024-04-25T14:03:51.479295+0000","last_unstale":"2024-04-25T14:03:56.494933+0000","last_undegraded":"2024-04-25T14:03:56.494933+0000","last_fullsized":"2024-04-25T14:03:56.494933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:17.381249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493593+0000","last_change":"2024-04-25T14:03:55.681212+0000","last_active":"2024-04-25T14:03:56.493593+0000","last_peered":"2024-04-25T14:03:56.493593+0000","last_clean":"2024-04-25T14:03:56.493593+0000","last_became_active":"2024-04-25T14:03:55.680579+0000","last_became_peered":"2024-04-25T14:03:55.680579+0000","last_unstale":"2024-04-25T14:03:56.493593+0000","last_undegraded":"2024-04-25T14:03:56.493593+0000","last_fullsized":"2024-04-25T14:03:56.493593+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:33:32.909419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495038+0000","last_change":"2024-04-25T14:03:50.468056+0000","last_active":"2024-04-25T14:03:56.495038+0000","last_peered":"2024-04-25T14:03:56.495038+0000","last_clean":"2024-04-25T14:03:56.495038+0000","last_became_active":"2024-04-25T14:03:50.467531+0000","last_became_peered":"2024-04-25T14:03:50.467531+0000","last_unstale":"2024-04-25T14:03:56.495038+0000","last_undegraded":"2024-04-25T14:03:56.495038+0000","last_fullsized":"2024-04-25T14:03:56.495038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:29.264084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774540+0000","last_change":"2024-04-25T14:03:51.485995+0000","last_active":"2024-04-25T14:03:56.774540+0000","last_peered":"2024-04-25T14:03:56.774540+0000","last_clean":"2024-04-25T14:03:56.774540+0000","last_became_active":"2024-04-25T14:03:51.485364+0000","last_became_peered":"2024-04-25T14:03:51.485364+0000","last_unstale":"2024-04-25T14:03:56.774540+0000","last_undegraded":"2024-04-25T14:03:56.774540+0000","last_fullsized":"2024-04-25T14:03:56.774540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:10.318378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772007+0000","last_change":"2024-04-25T14:03:55.514585+0000","last_active":"2024-04-25T14:03:56.772007+0000","last_peered":"2024-04-25T14:03:56.772007+0000","last_clean":"2024-04-25T14:03:56.772007+0000","last_became_active":"2024-04-25T14:03:55.514106+0000","last_became_peered":"2024-04-25T14:03:55.514106+0000","last_unstale":"2024-04-25T14:03:56.772007+0000","last_undegraded":"2024-04-25T14:03:56.772007+0000","last_fullsized":"2024-04-25T14:03:56.772007+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:14.363355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773034+0000","last_change":"2024-04-25T14:03:50.476493+0000","last_active":"2024-04-25T14:03:56.773034+0000","last_peered":"2024-04-25T14:03:56.773034+0000","last_clean":"2024-04-25T14:03:56.773034+0000","last_became_active":"2024-04-25T14:03:50.476154+0000","last_became_peered":"2024-04-25T14:03:50.476154+0000","last_unstale":"2024-04-25T14:03:56.773034+0000","last_undegraded":"2024-04-25T14:03:56.773034+0000","last_fullsized":"2024-04-25T14:03:56.773034+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:45.080718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496310+0000","last_change":"2024-04-25T14:03:51.494560+0000","last_active":"2024-04-25T14:03:56.496310+0000","last_peered":"2024-04-25T14:03:56.496310+0000","last_clean":"2024-04-25T14:03:56.496310+0000","last_became_active":"2024-04-25T14:03:51.494211+0000","last_became_peered":"2024-04-25T14:03:51.494211+0000","last_unstale":"2024-04-25T14:03:56.496310+0000","last_undegraded":"2024-04-25T14:03:56.496310+0000","last_fullsized":"2024-04-25T14:03:56.496310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:17.381249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772740+0000","last_change":"2024-04-25T14:03:55.518340+0000","last_active":"2024-04-25T14:03:56.772740+0000","last_peered":"2024-04-25T14:03:56.772740+0000","last_clean":"2024-04-25T14:03:56.772740+0000","last_became_active":"2024-04-25T14:03:55.515787+0000","last_became_peered":"2024-04-25T14:03:55.515787+0000","last_unstale":"2024-04-25T14:03:56.772740+0000","last_undegraded":"2024-04-25T14:03:56.772740+0000","last_fullsized":"2024-04-25T14:03:56.772740+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:24.136045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772400+0000","last_change":"2024-04-25T14:03:50.472667+0000","last_active":"2024-04-25T14:03:56.772400+0000","last_peered":"2024-04-25T14:03:56.772400+0000","last_clean":"2024-04-25T14:03:56.772400+0000","last_became_active":"2024-04-25T14:03:50.472286+0000","last_became_peered":"2024-04-25T14:03:50.472286+0000","last_unstale":"2024-04-25T14:03:56.772400+0000","last_undegraded":"2024-04-25T14:03:56.772400+0000","last_fullsized":"2024-04-25T14:03:56.772400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:02.117945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496569+0000","last_change":"2024-04-25T14:03:51.494545+0000","last_active":"2024-04-25T14:03:56.496569+0000","last_peered":"2024-04-25T14:03:56.496569+0000","last_clean":"2024-04-25T14:03:56.496569+0000","last_became_active":"2024-04-25T14:03:51.494238+0000","last_became_peered":"2024-04-25T14:03:51.494238+0000","last_unstale":"2024-04-25T14:03:56.496569+0000","last_undegraded":"2024-04-25T14:03:56.496569+0000","last_fullsized":"2024-04-25T14:03:56.496569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:08.845346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772125+0000","last_change":"2024-04-25T14:03:55.512835+0000","last_active":"2024-04-25T14:03:56.772125+0000","last_peered":"2024-04-25T14:03:56.772125+0000","last_clean":"2024-04-25T14:03:56.772125+0000","last_became_active":"2024-04-25T14:03:55.512330+0000","last_became_peered":"2024-04-25T14:03:55.512330+0000","last_unstale":"2024-04-25T14:03:56.772125+0000","last_undegraded":"2024-04-25T14:03:56.772125+0000","last_fullsized":"2024-04-25T14:03:56.772125+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:47.781818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494962+0000","last_change":"2024-04-25T14:03:51.494496+0000","last_active":"2024-04-25T14:03:56.494962+0000","last_peered":"2024-04-25T14:03:56.494962+0000","last_clean":"2024-04-25T14:03:56.494962+0000","last_became_active":"2024-04-25T14:03:51.494201+0000","last_became_peered":"2024-04-25T14:03:51.494201+0000","last_unstale":"2024-04-25T14:03:56.494962+0000","last_undegraded":"2024-04-25T14:03:56.494962+0000","last_fullsized":"2024-04-25T14:03:56.494962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.599283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773644+0000","last_change":"2024-04-25T14:03:50.472026+0000","last_active":"2024-04-25T14:03:56.773644+0000","last_peered":"2024-04-25T14:03:56.773644+0000","last_clean":"2024-04-25T14:03:56.773644+0000","last_became_active":"2024-04-25T14:03:50.468476+0000","last_became_peered":"2024-04-25T14:03:50.468476+0000","last_unstale":"2024-04-25T14:03:56.773644+0000","last_undegraded":"2024-04-25T14:03:56.773644+0000","last_fullsized":"2024-04-25T14:03:56.773644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:39.630327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772916+0000","last_change":"2024-04-25T14:03:55.524347+0000","last_active":"2024-04-25T14:03:56.772916+0000","last_peered":"2024-04-25T14:03:56.772916+0000","last_clean":"2024-04-25T14:03:56.772916+0000","last_became_active":"2024-04-25T14:03:55.523890+0000","last_became_peered":"2024-04-25T14:03:55.523890+0000","last_unstale":"2024-04-25T14:03:56.772916+0000","last_undegraded":"2024-04-25T14:03:56.772916+0000","last_fullsized":"2024-04-25T14:03:56.772916+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:20.858506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493611+0000","last_change":"2024-04-25T14:03:51.471282+0000","last_active":"2024-04-25T14:03:56.493611+0000","last_peered":"2024-04-25T14:03:56.493611+0000","last_clean":"2024-04-25T14:03:56.493611+0000","last_became_active":"2024-04-25T14:03:51.470871+0000","last_became_peered":"2024-04-25T14:03:51.470871+0000","last_unstale":"2024-04-25T14:03:56.493611+0000","last_undegraded":"2024-04-25T14:03:56.493611+0000","last_fullsized":"2024-04-25T14:03:56.493611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:34.707119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773462+0000","last_change":"2024-04-25T14:03:50.467312+0000","last_active":"2024-04-25T14:03:56.773462+0000","last_peered":"2024-04-25T14:03:56.773462+0000","last_clean":"2024-04-25T14:03:56.773462+0000","last_became_active":"2024-04-25T14:03:50.467020+0000","last_became_peered":"2024-04-25T14:03:50.467020+0000","last_unstale":"2024-04-25T14:03:56.773462+0000","last_undegraded":"2024-04-25T14:03:56.773462+0000","last_fullsized":"2024-04-25T14:03:56.773462+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:45.080718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772970+0000","last_change":"2024-04-25T14:03:55.520580+0000","last_active":"2024-04-25T14:03:56.772970+0000","last_peered":"2024-04-25T14:03:56.772970+0000","last_clean":"2024-04-25T14:03:56.772970+0000","last_became_active":"2024-04-25T14:03:55.520013+0000","last_became_peered":"2024-04-25T14:03:55.520013+0000","last_unstale":"2024-04-25T14:03:56.772970+0000","last_undegraded":"2024-04-25T14:03:56.772970+0000","last_fullsized":"2024-04-25T14:03:56.772970+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:38.752096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494042+0000","last_change":"2024-04-25T14:03:51.474463+0000","last_active":"2024-04-25T14:03:56.494042+0000","last_peered":"2024-04-25T14:03:56.494042+0000","last_clean":"2024-04-25T14:03:56.494042+0000","last_became_active":"2024-04-25T14:03:51.474154+0000","last_became_peered":"2024-04-25T14:03:51.474154+0000","last_unstale":"2024-04-25T14:03:56.494042+0000","last_undegraded":"2024-04-25T14:03:56.494042+0000","last_fullsized":"2024-04-25T14:03:56.494042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:10.318378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772985+0000","last_change":"2024-04-25T14:03:50.476410+0000","last_active":"2024-04-25T14:03:56.772985+0000","last_peered":"2024-04-25T14:03:56.772985+0000","last_clean":"2024-04-25T14:03:56.772985+0000","last_became_active":"2024-04-25T14:03:50.476083+0000","last_became_peered":"2024-04-25T14:03:50.476083+0000","last_unstale":"2024-04-25T14:03:56.772985+0000","last_undegraded":"2024-04-25T14:03:56.772985+0000","last_fullsized":"2024-04-25T14:03:56.772985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:29.264084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774278+0000","last_change":"2024-04-25T14:03:55.674875+0000","last_active":"2024-04-25T14:03:56.774278+0000","last_peered":"2024-04-25T14:03:56.774278+0000","last_clean":"2024-04-25T14:03:56.774278+0000","last_became_active":"2024-04-25T14:03:55.674408+0000","last_became_peered":"2024-04-25T14:03:55.674408+0000","last_unstale":"2024-04-25T14:03:56.774278+0000","last_undegraded":"2024-04-25T14:03:56.774278+0000","last_fullsized":"2024-04-25T14:03:56.774278+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:50.234674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773825+0000","last_change":"2024-04-25T14:03:51.487560+0000","last_active":"2024-04-25T14:03:56.773825+0000","last_peered":"2024-04-25T14:03:56.773825+0000","last_clean":"2024-04-25T14:03:56.773825+0000","last_became_active":"2024-04-25T14:03:51.486822+0000","last_became_peered":"2024-04-25T14:03:51.486822+0000","last_unstale":"2024-04-25T14:03:56.773825+0000","last_undegraded":"2024-04-25T14:03:56.773825+0000","last_fullsized":"2024-04-25T14:03:56.773825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:16.370061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.771874+0000","last_change":"2024-04-25T14:03:50.479489+0000","last_active":"2024-04-25T14:03:56.771874+0000","last_peered":"2024-04-25T14:03:56.771874+0000","last_clean":"2024-04-25T14:03:56.771874+0000","last_became_active":"2024-04-25T14:03:50.479212+0000","last_became_peered":"2024-04-25T14:03:50.479212+0000","last_unstale":"2024-04-25T14:03:56.771874+0000","last_undegraded":"2024-04-25T14:03:56.771874+0000","last_fullsized":"2024-04-25T14:03:56.771874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:10.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773707+0000","last_change":"2024-04-25T14:03:55.519638+0000","last_active":"2024-04-25T14:03:56.773707+0000","last_peered":"2024-04-25T14:03:56.773707+0000","last_clean":"2024-04-25T14:03:56.773707+0000","last_became_active":"2024-04-25T14:03:55.519191+0000","last_became_peered":"2024-04-25T14:03:55.519191+0000","last_unstale":"2024-04-25T14:03:56.773707+0000","last_undegraded":"2024-04-25T14:03:56.773707+0000","last_fullsized":"2024-04-25T14:03:56.773707+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:25.451897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774673+0000","last_change":"2024-04-25T14:03:51.477687+0000","last_active":"2024-04-25T14:03:56.774673+0000","last_peered":"2024-04-25T14:03:56.774673+0000","last_clean":"2024-04-25T14:03:56.774673+0000","last_became_active":"2024-04-25T14:03:51.477339+0000","last_became_peered":"2024-04-25T14:03:51.477339+0000","last_unstale":"2024-04-25T14:03:56.774673+0000","last_undegraded":"2024-04-25T14:03:56.774673+0000","last_fullsized":"2024-04-25T14:03:56.774673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:43.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496742+0000","last_change":"2024-04-25T14:03:50.470614+0000","last_active":"2024-04-25T14:03:56.496742+0000","last_peered":"2024-04-25T14:03:56.496742+0000","last_clean":"2024-04-25T14:03:56.496742+0000","last_became_active":"2024-04-25T14:03:50.470405+0000","last_became_peered":"2024-04-25T14:03:50.470405+0000","last_unstale":"2024-04-25T14:03:56.496742+0000","last_undegraded":"2024-04-25T14:03:56.496742+0000","last_fullsized":"2024-04-25T14:03:56.496742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:02.117945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773539+0000","last_change":"2024-04-25T14:03:55.681096+0000","last_active":"2024-04-25T14:03:56.773539+0000","last_peered":"2024-04-25T14:03:56.773539+0000","last_clean":"2024-04-25T14:03:56.773539+0000","last_became_active":"2024-04-25T14:03:55.680556+0000","last_became_peered":"2024-04-25T14:03:55.680556+0000","last_unstale":"2024-04-25T14:03:56.773539+0000","last_undegraded":"2024-04-25T14:03:56.773539+0000","last_fullsized":"2024-04-25T14:03:56.773539+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:09.534866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495184+0000","last_change":"2024-04-25T14:03:51.473595+0000","last_active":"2024-04-25T14:03:56.495184+0000","last_peered":"2024-04-25T14:03:56.495184+0000","last_clean":"2024-04-25T14:03:56.495184+0000","last_became_active":"2024-04-25T14:03:51.473273+0000","last_became_peered":"2024-04-25T14:03:51.473273+0000","last_unstale":"2024-04-25T14:03:56.495184+0000","last_undegraded":"2024-04-25T14:03:56.495184+0000","last_fullsized":"2024-04-25T14:03:56.495184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:05.489889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774166+0000","last_change":"2024-04-25T14:03:50.476274+0000","last_active":"2024-04-25T14:03:56.774166+0000","last_peered":"2024-04-25T14:03:56.774166+0000","last_clean":"2024-04-25T14:03:56.774166+0000","last_became_active":"2024-04-25T14:03:50.475935+0000","last_became_peered":"2024-04-25T14:03:50.475935+0000","last_unstale":"2024-04-25T14:03:56.774166+0000","last_undegraded":"2024-04-25T14:03:56.774166+0000","last_fullsized":"2024-04-25T14:03:56.774166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:39.630327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493917+0000","last_change":"2024-04-25T14:03:55.675251+0000","last_active":"2024-04-25T14:03:56.493917+0000","last_peered":"2024-04-25T14:03:56.493917+0000","last_clean":"2024-04-25T14:03:56.493917+0000","last_became_active":"2024-04-25T14:03:55.674772+0000","last_became_peered":"2024-04-25T14:03:55.674772+0000","last_unstale":"2024-04-25T14:03:56.493917+0000","last_undegraded":"2024-04-25T14:03:56.493917+0000","last_fullsized":"2024-04-25T14:03:56.493917+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:58.844877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773967+0000","last_change":"2024-04-25T14:03:51.476624+0000","last_active":"2024-04-25T14:03:56.773967+0000","last_peered":"2024-04-25T14:03:56.773967+0000","last_clean":"2024-04-25T14:03:56.773967+0000","last_became_active":"2024-04-25T14:03:51.476320+0000","last_became_peered":"2024-04-25T14:03:51.476320+0000","last_unstale":"2024-04-25T14:03:56.773967+0000","last_undegraded":"2024-04-25T14:03:56.773967+0000","last_fullsized":"2024-04-25T14:03:56.773967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:20.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773171+0000","last_change":"2024-04-25T14:03:50.482353+0000","last_active":"2024-04-25T14:03:56.773171+0000","last_peered":"2024-04-25T14:03:56.773171+0000","last_clean":"2024-04-25T14:03:56.773171+0000","last_became_active":"2024-04-25T14:03:50.482063+0000","last_became_peered":"2024-04-25T14:03:50.482063+0000","last_unstale":"2024-04-25T14:03:56.773171+0000","last_undegraded":"2024-04-25T14:03:56.773171+0000","last_fullsized":"2024-04-25T14:03:56.773171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:37.406291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773151+0000","last_change":"2024-04-25T14:03:55.676872+0000","last_active":"2024-04-25T14:03:56.773151+0000","last_peered":"2024-04-25T14:03:56.773151+0000","last_clean":"2024-04-25T14:03:56.773151+0000","last_became_active":"2024-04-25T14:03:55.676444+0000","last_became_peered":"2024-04-25T14:03:55.676444+0000","last_unstale":"2024-04-25T14:03:56.773151+0000","last_undegraded":"2024-04-25T14:03:56.773151+0000","last_fullsized":"2024-04-25T14:03:56.773151+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:37.811376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496691+0000","last_change":"2024-04-25T14:03:51.480200+0000","last_active":"2024-04-25T14:03:56.496691+0000","last_peered":"2024-04-25T14:03:56.496691+0000","last_clean":"2024-04-25T14:03:56.496691+0000","last_became_active":"2024-04-25T14:03:51.479798+0000","last_became_peered":"2024-04-25T14:03:51.479798+0000","last_unstale":"2024-04-25T14:03:56.496691+0000","last_undegraded":"2024-04-25T14:03:56.496691+0000","last_fullsized":"2024-04-25T14:03:56.496691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:46.189697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774003+0000","last_change":"2024-04-25T14:03:50.479761+0000","last_active":"2024-04-25T14:03:56.774003+0000","last_peered":"2024-04-25T14:03:56.774003+0000","last_clean":"2024-04-25T14:03:56.774003+0000","last_became_active":"2024-04-25T14:03:50.479175+0000","last_became_peered":"2024-04-25T14:03:50.479175+0000","last_unstale":"2024-04-25T14:03:56.774003+0000","last_undegraded":"2024-04-25T14:03:56.774003+0000","last_fullsized":"2024-04-25T14:03:56.774003+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:02.117945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773256+0000","last_change":"2024-04-25T14:03:55.515692+0000","last_active":"2024-04-25T14:03:56.773256+0000","last_peered":"2024-04-25T14:03:56.773256+0000","last_clean":"2024-04-25T14:03:56.773256+0000","last_became_active":"2024-04-25T14:03:55.515238+0000","last_became_peered":"2024-04-25T14:03:55.515238+0000","last_unstale":"2024-04-25T14:03:56.773256+0000","last_undegraded":"2024-04-25T14:03:56.773256+0000","last_fullsized":"2024-04-25T14:03:56.773256+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:51.371333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773096+0000","last_change":"2024-04-25T14:03:51.495207+0000","last_active":"2024-04-25T14:03:56.773096+0000","last_peered":"2024-04-25T14:03:56.773096+0000","last_clean":"2024-04-25T14:03:56.773096+0000","last_became_active":"2024-04-25T14:03:51.492531+0000","last_became_peered":"2024-04-25T14:03:51.492531+0000","last_unstale":"2024-04-25T14:03:56.773096+0000","last_undegraded":"2024-04-25T14:03:56.773096+0000","last_fullsized":"2024-04-25T14:03:56.773096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:03.123407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495711+0000","last_change":"2024-04-25T14:03:50.459698+0000","last_active":"2024-04-25T14:03:56.495711+0000","last_peered":"2024-04-25T14:03:56.495711+0000","last_clean":"2024-04-25T14:03:56.495711+0000","last_became_active":"2024-04-25T14:03:50.459451+0000","last_became_peered":"2024-04-25T14:03:50.459451+0000","last_unstale":"2024-04-25T14:03:56.495711+0000","last_undegraded":"2024-04-25T14:03:56.495711+0000","last_fullsized":"2024-04-25T14:03:56.495711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:15.364599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773837+0000","last_change":"2024-04-25T14:03:55.519334+0000","last_active":"2024-04-25T14:03:56.773837+0000","last_peered":"2024-04-25T14:03:56.773837+0000","last_clean":"2024-04-25T14:03:56.773837+0000","last_became_active":"2024-04-25T14:03:55.518940+0000","last_became_peered":"2024-04-25T14:03:55.518940+0000","last_unstale":"2024-04-25T14:03:56.773837+0000","last_undegraded":"2024-04-25T14:03:56.773837+0000","last_fullsized":"2024-04-25T14:03:56.773837+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:12.890323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772902+0000","last_change":"2024-04-25T14:03:51.476141+0000","last_active":"2024-04-25T14:03:56.772902+0000","last_peered":"2024-04-25T14:03:56.772902+0000","last_clean":"2024-04-25T14:03:56.772902+0000","last_became_active":"2024-04-25T14:03:51.475819+0000","last_became_peered":"2024-04-25T14:03:51.475819+0000","last_unstale":"2024-04-25T14:03:56.772902+0000","last_undegraded":"2024-04-25T14:03:56.772902+0000","last_fullsized":"2024-04-25T14:03:56.772902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:31.707048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772028+0000","last_change":"2024-04-25T14:03:50.458612+0000","last_active":"2024-04-25T14:03:56.772028+0000","last_peered":"2024-04-25T14:03:56.772028+0000","last_clean":"2024-04-25T14:03:56.772028+0000","last_became_active":"2024-04-25T14:03:50.458254+0000","last_became_peered":"2024-04-25T14:03:50.458254+0000","last_unstale":"2024-04-25T14:03:56.772028+0000","last_undegraded":"2024-04-25T14:03:56.772028+0000","last_fullsized":"2024-04-25T14:03:56.772028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:29.264084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774279+0000","last_change":"2024-04-25T14:03:55.513023+0000","last_active":"2024-04-25T14:03:56.774279+0000","last_peered":"2024-04-25T14:03:56.774279+0000","last_clean":"2024-04-25T14:03:56.774279+0000","last_became_active":"2024-04-25T14:03:55.512550+0000","last_became_peered":"2024-04-25T14:03:55.512550+0000","last_unstale":"2024-04-25T14:03:56.774279+0000","last_undegraded":"2024-04-25T14:03:56.774279+0000","last_fullsized":"2024-04-25T14:03:56.774279+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:06.818399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.771972+0000","last_change":"2024-04-25T14:03:51.487355+0000","last_active":"2024-04-25T14:03:56.771972+0000","last_peered":"2024-04-25T14:03:56.771972+0000","last_clean":"2024-04-25T14:03:56.771972+0000","last_became_active":"2024-04-25T14:03:51.487050+0000","last_became_peered":"2024-04-25T14:03:51.487050+0000","last_unstale":"2024-04-25T14:03:56.771972+0000","last_undegraded":"2024-04-25T14:03:56.771972+0000","last_fullsized":"2024-04-25T14:03:56.771972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:33.766258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772008+0000","last_change":"2024-04-25T14:03:50.476776+0000","last_active":"2024-04-25T14:03:56.772008+0000","last_peered":"2024-04-25T14:03:56.772008+0000","last_clean":"2024-04-25T14:03:56.772008+0000","last_became_active":"2024-04-25T14:03:50.476470+0000","last_became_peered":"2024-04-25T14:03:50.476470+0000","last_unstale":"2024-04-25T14:03:56.772008+0000","last_undegraded":"2024-04-25T14:03:56.772008+0000","last_fullsized":"2024-04-25T14:03:56.772008+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:15.364599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774604+0000","last_change":"2024-04-25T14:03:55.674908+0000","last_active":"2024-04-25T14:03:56.774604+0000","last_peered":"2024-04-25T14:03:56.774604+0000","last_clean":"2024-04-25T14:03:56.774604+0000","last_became_active":"2024-04-25T14:03:55.674383+0000","last_became_peered":"2024-04-25T14:03:55.674383+0000","last_unstale":"2024-04-25T14:03:56.774604+0000","last_undegraded":"2024-04-25T14:03:56.774604+0000","last_fullsized":"2024-04-25T14:03:56.774604+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:06.742929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493557+0000","last_change":"2024-04-25T14:03:51.489259+0000","last_active":"2024-04-25T14:03:56.493557+0000","last_peered":"2024-04-25T14:03:56.493557+0000","last_clean":"2024-04-25T14:03:56.493557+0000","last_became_active":"2024-04-25T14:03:51.488822+0000","last_became_peered":"2024-04-25T14:03:51.488822+0000","last_unstale":"2024-04-25T14:03:56.493557+0000","last_undegraded":"2024-04-25T14:03:56.493557+0000","last_fullsized":"2024-04-25T14:03:56.493557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:43.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495950+0000","last_change":"2024-04-25T14:03:50.466483+0000","last_active":"2024-04-25T14:03:56.495950+0000","last_peered":"2024-04-25T14:03:56.495950+0000","last_clean":"2024-04-25T14:03:56.495950+0000","last_became_active":"2024-04-25T14:03:50.463125+0000","last_became_peered":"2024-04-25T14:03:50.463125+0000","last_unstale":"2024-04-25T14:03:56.495950+0000","last_undegraded":"2024-04-25T14:03:56.495950+0000","last_fullsized":"2024-04-25T14:03:56.495950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:29.264084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":117,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772125+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:03:56.772125+0000","last_peered":"2024-04-25T14:03:56.772125+0000","last_clean":"2024-04-25T14:03:56.772125+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:03:56.772125+0000","last_undegraded":"2024-04-25T14:03:56.772125+0000","last_fullsized":"2024-04-25T14:03:56.772125+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496238+0000","last_change":"2024-04-25T14:03:55.678084+0000","last_active":"2024-04-25T14:03:56.496238+0000","last_peered":"2024-04-25T14:03:56.496238+0000","last_clean":"2024-04-25T14:03:56.496238+0000","last_became_active":"2024-04-25T14:03:55.677526+0000","last_became_peered":"2024-04-25T14:03:55.677526+0000","last_unstale":"2024-04-25T14:03:56.496238+0000","last_undegraded":"2024-04-25T14:03:56.496238+0000","last_fullsized":"2024-04-25T14:03:56.496238+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:25.451897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772807+0000","last_change":"2024-04-25T14:03:51.482608+0000","last_active":"2024-04-25T14:03:56.772807+0000","last_peered":"2024-04-25T14:03:56.772807+0000","last_clean":"2024-04-25T14:03:56.772807+0000","last_became_active":"2024-04-25T14:03:51.482022+0000","last_became_peered":"2024-04-25T14:03:51.482022+0000","last_unstale":"2024-04-25T14:03:56.772807+0000","last_undegraded":"2024-04-25T14:03:56.772807+0000","last_fullsized":"2024-04-25T14:03:56.772807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:33.766399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772584+0000","last_change":"2024-04-25T14:03:50.469170+0000","last_active":"2024-04-25T14:03:56.772584+0000","last_peered":"2024-04-25T14:03:56.772584+0000","last_clean":"2024-04-25T14:03:56.772584+0000","last_became_active":"2024-04-25T14:03:50.468742+0000","last_became_peered":"2024-04-25T14:03:50.468742+0000","last_unstale":"2024-04-25T14:03:56.772584+0000","last_undegraded":"2024-04-25T14:03:56.772584+0000","last_fullsized":"2024-04-25T14:03:56.772584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:10.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494752+0000","last_change":"2024-04-25T14:03:55.676560+0000","last_active":"2024-04-25T14:03:56.494752+0000","last_peered":"2024-04-25T14:03:56.494752+0000","last_clean":"2024-04-25T14:03:56.494752+0000","last_became_active":"2024-04-25T14:03:55.676021+0000","last_became_peered":"2024-04-25T14:03:55.676021+0000","last_unstale":"2024-04-25T14:03:56.494752+0000","last_undegraded":"2024-04-25T14:03:56.494752+0000","last_fullsized":"2024-04-25T14:03:56.494752+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:09.534866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496565+0000","last_change":"2024-04-25T14:03:51.466314+0000","last_active":"2024-04-25T14:03:56.496565+0000","last_peered":"2024-04-25T14:03:56.496565+0000","last_clean":"2024-04-25T14:03:56.496565+0000","last_became_active":"2024-04-25T14:03:51.465982+0000","last_became_peered":"2024-04-25T14:03:51.465982+0000","last_unstale":"2024-04-25T14:03:56.496565+0000","last_undegraded":"2024-04-25T14:03:56.496565+0000","last_fullsized":"2024-04-25T14:03:56.496565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:47.326356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774109+0000","last_change":"2024-04-25T14:03:50.465115+0000","last_active":"2024-04-25T14:03:56.774109+0000","last_peered":"2024-04-25T14:03:56.774109+0000","last_clean":"2024-04-25T14:03:56.774109+0000","last_became_active":"2024-04-25T14:03:50.464561+0000","last_became_peered":"2024-04-25T14:03:50.464561+0000","last_unstale":"2024-04-25T14:03:56.774109+0000","last_undegraded":"2024-04-25T14:03:56.774109+0000","last_fullsized":"2024-04-25T14:03:56.774109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:10.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773457+0000","last_change":"2024-04-25T14:03:55.526344+0000","last_active":"2024-04-25T14:03:56.773457+0000","last_peered":"2024-04-25T14:03:56.773457+0000","last_clean":"2024-04-25T14:03:56.773457+0000","last_became_active":"2024-04-25T14:03:55.525889+0000","last_became_peered":"2024-04-25T14:03:55.525889+0000","last_unstale":"2024-04-25T14:03:56.773457+0000","last_undegraded":"2024-04-25T14:03:56.773457+0000","last_fullsized":"2024-04-25T14:03:56.773457+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:06.818399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773492+0000","last_change":"2024-04-25T14:03:51.494453+0000","last_active":"2024-04-25T14:03:56.773492+0000","last_peered":"2024-04-25T14:03:56.773492+0000","last_clean":"2024-04-25T14:03:56.773492+0000","last_became_active":"2024-04-25T14:03:51.494113+0000","last_became_peered":"2024-04-25T14:03:51.494113+0000","last_unstale":"2024-04-25T14:03:56.773492+0000","last_undegraded":"2024-04-25T14:03:56.773492+0000","last_fullsized":"2024-04-25T14:03:56.773492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:08.845346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772486+0000","last_change":"2024-04-25T14:03:50.463824+0000","last_active":"2024-04-25T14:03:56.772486+0000","last_peered":"2024-04-25T14:03:56.772486+0000","last_clean":"2024-04-25T14:03:56.772486+0000","last_became_active":"2024-04-25T14:03:50.463507+0000","last_became_peered":"2024-04-25T14:03:50.463507+0000","last_unstale":"2024-04-25T14:03:56.772486+0000","last_undegraded":"2024-04-25T14:03:56.772486+0000","last_fullsized":"2024-04-25T14:03:56.772486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:15.364599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494579+0000","last_change":"2024-04-25T14:03:55.676329+0000","last_active":"2024-04-25T14:03:56.494579+0000","last_peered":"2024-04-25T14:03:56.494579+0000","last_clean":"2024-04-25T14:03:56.494579+0000","last_became_active":"2024-04-25T14:03:55.675752+0000","last_became_peered":"2024-04-25T14:03:55.675752+0000","last_unstale":"2024-04-25T14:03:56.494579+0000","last_undegraded":"2024-04-25T14:03:56.494579+0000","last_fullsized":"2024-04-25T14:03:56.494579+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:28.167574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772572+0000","last_change":"2024-04-25T14:03:51.495282+0000","last_active":"2024-04-25T14:03:56.772572+0000","last_peered":"2024-04-25T14:03:56.772572+0000","last_clean":"2024-04-25T14:03:56.772572+0000","last_became_active":"2024-04-25T14:03:51.494965+0000","last_became_peered":"2024-04-25T14:03:51.494965+0000","last_unstale":"2024-04-25T14:03:56.772572+0000","last_undegraded":"2024-04-25T14:03:56.772572+0000","last_fullsized":"2024-04-25T14:03:56.772572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:02.773422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773963+0000","last_change":"2024-04-25T14:03:50.465925+0000","last_active":"2024-04-25T14:03:56.773963+0000","last_peered":"2024-04-25T14:03:56.773963+0000","last_clean":"2024-04-25T14:03:56.773963+0000","last_became_active":"2024-04-25T14:03:50.465495+0000","last_became_peered":"2024-04-25T14:03:50.465495+0000","last_unstale":"2024-04-25T14:03:56.773963+0000","last_undegraded":"2024-04-25T14:03:56.773963+0000","last_fullsized":"2024-04-25T14:03:56.773963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:15.520202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773183+0000","last_change":"2024-04-25T14:03:55.520108+0000","last_active":"2024-04-25T14:03:56.773183+0000","last_peered":"2024-04-25T14:03:56.773183+0000","last_clean":"2024-04-25T14:03:56.773183+0000","last_became_active":"2024-04-25T14:03:55.519374+0000","last_became_peered":"2024-04-25T14:03:55.519374+0000","last_unstale":"2024-04-25T14:03:56.773183+0000","last_undegraded":"2024-04-25T14:03:56.773183+0000","last_fullsized":"2024-04-25T14:03:56.773183+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:21.426226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772332+0000","last_change":"2024-04-25T14:03:51.488205+0000","last_active":"2024-04-25T14:03:56.772332+0000","last_peered":"2024-04-25T14:03:56.772332+0000","last_clean":"2024-04-25T14:03:56.772332+0000","last_became_active":"2024-04-25T14:03:51.487872+0000","last_became_peered":"2024-04-25T14:03:51.487872+0000","last_unstale":"2024-04-25T14:03:56.772332+0000","last_undegraded":"2024-04-25T14:03:56.772332+0000","last_fullsized":"2024-04-25T14:03:56.772332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:55.876775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.771697+0000","last_change":"2024-04-25T14:03:50.475566+0000","last_active":"2024-04-25T14:03:56.771697+0000","last_peered":"2024-04-25T14:03:56.771697+0000","last_clean":"2024-04-25T14:03:56.771697+0000","last_became_active":"2024-04-25T14:03:50.475304+0000","last_became_peered":"2024-04-25T14:03:50.475304+0000","last_unstale":"2024-04-25T14:03:56.771697+0000","last_undegraded":"2024-04-25T14:03:56.771697+0000","last_fullsized":"2024-04-25T14:03:56.771697+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:15.520202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774574+0000","last_change":"2024-04-25T14:03:55.523592+0000","last_active":"2024-04-25T14:03:56.774574+0000","last_peered":"2024-04-25T14:03:56.774574+0000","last_clean":"2024-04-25T14:03:56.774574+0000","last_became_active":"2024-04-25T14:03:55.521767+0000","last_became_peered":"2024-04-25T14:03:55.521767+0000","last_unstale":"2024-04-25T14:03:56.774574+0000","last_undegraded":"2024-04-25T14:03:56.774574+0000","last_fullsized":"2024-04-25T14:03:56.774574+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:50.234674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494779+0000","last_change":"2024-04-25T14:03:51.480142+0000","last_active":"2024-04-25T14:03:56.494779+0000","last_peered":"2024-04-25T14:03:56.494779+0000","last_clean":"2024-04-25T14:03:56.494779+0000","last_became_active":"2024-04-25T14:03:51.479798+0000","last_became_peered":"2024-04-25T14:03:51.479798+0000","last_unstale":"2024-04-25T14:03:56.494779+0000","last_undegraded":"2024-04-25T14:03:56.494779+0000","last_fullsized":"2024-04-25T14:03:56.494779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:31.840842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494679+0000","last_change":"2024-04-25T14:03:50.459894+0000","last_active":"2024-04-25T14:03:56.494679+0000","last_peered":"2024-04-25T14:03:56.494679+0000","last_clean":"2024-04-25T14:03:56.494679+0000","last_became_active":"2024-04-25T14:03:50.459580+0000","last_became_peered":"2024-04-25T14:03:50.459580+0000","last_unstale":"2024-04-25T14:03:56.494679+0000","last_undegraded":"2024-04-25T14:03:56.494679+0000","last_fullsized":"2024-04-25T14:03:56.494679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:29.264084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495170+0000","last_change":"2024-04-25T14:03:55.681298+0000","last_active":"2024-04-25T14:03:56.495170+0000","last_peered":"2024-04-25T14:03:56.495170+0000","last_clean":"2024-04-25T14:03:56.495170+0000","last_became_active":"2024-04-25T14:03:55.680746+0000","last_became_peered":"2024-04-25T14:03:55.680746+0000","last_unstale":"2024-04-25T14:03:56.495170+0000","last_undegraded":"2024-04-25T14:03:56.495170+0000","last_fullsized":"2024-04-25T14:03:56.495170+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:54.496413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772452+0000","last_change":"2024-04-25T14:03:51.499753+0000","last_active":"2024-04-25T14:03:56.772452+0000","last_peered":"2024-04-25T14:03:56.772452+0000","last_clean":"2024-04-25T14:03:56.772452+0000","last_became_active":"2024-04-25T14:03:51.499427+0000","last_became_peered":"2024-04-25T14:03:51.499427+0000","last_unstale":"2024-04-25T14:03:56.772452+0000","last_undegraded":"2024-04-25T14:03:56.772452+0000","last_fullsized":"2024-04-25T14:03:56.772452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:05.489889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495577+0000","last_change":"2024-04-25T14:03:50.467395+0000","last_active":"2024-04-25T14:03:56.495577+0000","last_peered":"2024-04-25T14:03:56.495577+0000","last_clean":"2024-04-25T14:03:56.495577+0000","last_became_active":"2024-04-25T14:03:50.465757+0000","last_became_peered":"2024-04-25T14:03:50.465757+0000","last_unstale":"2024-04-25T14:03:56.495577+0000","last_undegraded":"2024-04-25T14:03:56.495577+0000","last_fullsized":"2024-04-25T14:03:56.495577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:39.630327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494300+0000","last_change":"2024-04-25T14:03:55.676664+0000","last_active":"2024-04-25T14:03:56.494300+0000","last_peered":"2024-04-25T14:03:56.494300+0000","last_clean":"2024-04-25T14:03:56.494300+0000","last_became_active":"2024-04-25T14:03:55.674357+0000","last_became_peered":"2024-04-25T14:03:55.674357+0000","last_unstale":"2024-04-25T14:03:56.494300+0000","last_undegraded":"2024-04-25T14:03:56.494300+0000","last_fullsized":"2024-04-25T14:03:56.494300+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:55:42.758380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772783+0000","last_change":"2024-04-25T14:03:51.469041+0000","last_active":"2024-04-25T14:03:56.772783+0000","last_peered":"2024-04-25T14:03:56.772783+0000","last_clean":"2024-04-25T14:03:56.772783+0000","last_became_active":"2024-04-25T14:03:51.468704+0000","last_became_peered":"2024-04-25T14:03:51.468704+0000","last_unstale":"2024-04-25T14:03:56.772783+0000","last_undegraded":"2024-04-25T14:03:56.772783+0000","last_fullsized":"2024-04-25T14:03:56.772783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:31.840842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773670+0000","last_change":"2024-04-25T14:03:50.465211+0000","last_active":"2024-04-25T14:03:56.773670+0000","last_peered":"2024-04-25T14:03:56.773670+0000","last_clean":"2024-04-25T14:03:56.773670+0000","last_became_active":"2024-04-25T14:03:50.464836+0000","last_became_peered":"2024-04-25T14:03:50.464836+0000","last_unstale":"2024-04-25T14:03:56.773670+0000","last_undegraded":"2024-04-25T14:03:56.773670+0000","last_fullsized":"2024-04-25T14:03:56.773670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:04.876207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496500+0000","last_change":"2024-04-25T14:03:55.681339+0000","last_active":"2024-04-25T14:03:56.496500+0000","last_peered":"2024-04-25T14:03:56.496500+0000","last_clean":"2024-04-25T14:03:56.496500+0000","last_became_active":"2024-04-25T14:03:55.680779+0000","last_became_peered":"2024-04-25T14:03:55.680779+0000","last_unstale":"2024-04-25T14:03:56.496500+0000","last_undegraded":"2024-04-25T14:03:56.496500+0000","last_fullsized":"2024-04-25T14:03:56.496500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:58.844877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772688+0000","last_change":"2024-04-25T14:03:51.476195+0000","last_active":"2024-04-25T14:03:56.772688+0000","last_peered":"2024-04-25T14:03:56.772688+0000","last_clean":"2024-04-25T14:03:56.772688+0000","last_became_active":"2024-04-25T14:03:51.475372+0000","last_became_peered":"2024-04-25T14:03:51.475372+0000","last_unstale":"2024-04-25T14:03:56.772688+0000","last_undegraded":"2024-04-25T14:03:56.772688+0000","last_fullsized":"2024-04-25T14:03:56.772688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:47.326356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:04:06.748698+0000","last_change":"2024-04-25T14:03:50.470612+0000","last_active":"2024-04-25T14:04:06.748698+0000","last_peered":"2024-04-25T14:04:06.748698+0000","last_clean":"2024-04-25T14:04:06.748698+0000","last_became_active":"2024-04-25T14:03:50.470312+0000","last_became_peered":"2024-04-25T14:03:50.470312+0000","last_unstale":"2024-04-25T14:04:06.748698+0000","last_undegraded":"2024-04-25T14:04:06.748698+0000","last_fullsized":"2024-04-25T14:04:06.748698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:10.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773458+0000","last_change":"2024-04-25T14:03:55.526774+0000","last_active":"2024-04-25T14:03:56.773458+0000","last_peered":"2024-04-25T14:03:56.773458+0000","last_clean":"2024-04-25T14:03:56.773458+0000","last_became_active":"2024-04-25T14:03:55.524255+0000","last_became_peered":"2024-04-25T14:03:55.524255+0000","last_unstale":"2024-04-25T14:03:56.773458+0000","last_undegraded":"2024-04-25T14:03:56.773458+0000","last_fullsized":"2024-04-25T14:03:56.773458+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:37.811376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773340+0000","last_change":"2024-04-25T14:03:51.493162+0000","last_active":"2024-04-25T14:03:56.773340+0000","last_peered":"2024-04-25T14:03:56.773340+0000","last_clean":"2024-04-25T14:03:56.773340+0000","last_became_active":"2024-04-25T14:03:51.488560+0000","last_became_peered":"2024-04-25T14:03:51.488560+0000","last_unstale":"2024-04-25T14:03:56.773340+0000","last_undegraded":"2024-04-25T14:03:56.773340+0000","last_fullsized":"2024-04-25T14:03:56.773340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:40.635789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774009+0000","last_change":"2024-04-25T14:03:50.469594+0000","last_active":"2024-04-25T14:03:56.774009+0000","last_peered":"2024-04-25T14:03:56.774009+0000","last_clean":"2024-04-25T14:03:56.774009+0000","last_became_active":"2024-04-25T14:03:50.469196+0000","last_became_peered":"2024-04-25T14:03:50.469196+0000","last_unstale":"2024-04-25T14:03:56.774009+0000","last_undegraded":"2024-04-25T14:03:56.774009+0000","last_fullsized":"2024-04-25T14:03:56.774009+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:15.364599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772969+0000","last_change":"2024-04-25T14:03:55.518510+0000","last_active":"2024-04-25T14:03:56.772969+0000","last_peered":"2024-04-25T14:03:56.772969+0000","last_clean":"2024-04-25T14:03:56.772969+0000","last_became_active":"2024-04-25T14:03:55.515796+0000","last_became_peered":"2024-04-25T14:03:55.515796+0000","last_unstale":"2024-04-25T14:03:56.772969+0000","last_undegraded":"2024-04-25T14:03:56.772969+0000","last_fullsized":"2024-04-25T14:03:56.772969+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:59.921752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774672+0000","last_change":"2024-04-25T14:03:51.486474+0000","last_active":"2024-04-25T14:03:56.774672+0000","last_peered":"2024-04-25T14:03:56.774672+0000","last_clean":"2024-04-25T14:03:56.774672+0000","last_became_active":"2024-04-25T14:03:51.485742+0000","last_became_peered":"2024-04-25T14:03:51.485742+0000","last_unstale":"2024-04-25T14:03:56.774672+0000","last_undegraded":"2024-04-25T14:03:56.774672+0000","last_fullsized":"2024-04-25T14:03:56.774672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:16.813529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496428+0000","last_change":"2024-04-25T14:03:50.475187+0000","last_active":"2024-04-25T14:03:56.496428+0000","last_peered":"2024-04-25T14:03:56.496428+0000","last_clean":"2024-04-25T14:03:56.496428+0000","last_became_active":"2024-04-25T14:03:50.474761+0000","last_became_peered":"2024-04-25T14:03:50.474761+0000","last_unstale":"2024-04-25T14:03:56.496428+0000","last_undegraded":"2024-04-25T14:03:56.496428+0000","last_fullsized":"2024-04-25T14:03:56.496428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:02.117945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493593+0000","last_change":"2024-04-25T14:03:55.524048+0000","last_active":"2024-04-25T14:03:56.493593+0000","last_peered":"2024-04-25T14:03:56.493593+0000","last_clean":"2024-04-25T14:03:56.493593+0000","last_became_active":"2024-04-25T14:03:55.520419+0000","last_became_peered":"2024-04-25T14:03:55.520419+0000","last_unstale":"2024-04-25T14:03:56.493593+0000","last_undegraded":"2024-04-25T14:03:56.493593+0000","last_fullsized":"2024-04-25T14:03:56.493593+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:30.069597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496533+0000","last_change":"2024-04-25T14:03:51.467640+0000","last_active":"2024-04-25T14:03:56.496533+0000","last_peered":"2024-04-25T14:03:56.496533+0000","last_clean":"2024-04-25T14:03:56.496533+0000","last_became_active":"2024-04-25T14:03:51.467347+0000","last_became_peered":"2024-04-25T14:03:51.467347+0000","last_unstale":"2024-04-25T14:03:56.496533+0000","last_undegraded":"2024-04-25T14:03:56.496533+0000","last_fullsized":"2024-04-25T14:03:56.496533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:33.766399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496540+0000","last_change":"2024-04-25T14:03:50.462889+0000","last_active":"2024-04-25T14:03:56.496540+0000","last_peered":"2024-04-25T14:03:56.496540+0000","last_clean":"2024-04-25T14:03:56.496540+0000","last_became_active":"2024-04-25T14:03:50.462529+0000","last_became_peered":"2024-04-25T14:03:50.462529+0000","last_unstale":"2024-04-25T14:03:56.496540+0000","last_undegraded":"2024-04-25T14:03:56.496540+0000","last_fullsized":"2024-04-25T14:03:56.496540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:15.364599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493822+0000","last_change":"2024-04-25T14:03:55.675236+0000","last_active":"2024-04-25T14:03:56.493822+0000","last_peered":"2024-04-25T14:03:56.493822+0000","last_clean":"2024-04-25T14:03:56.493822+0000","last_became_active":"2024-04-25T14:03:55.674759+0000","last_became_peered":"2024-04-25T14:03:55.674759+0000","last_unstale":"2024-04-25T14:03:56.493822+0000","last_undegraded":"2024-04-25T14:03:56.493822+0000","last_fullsized":"2024-04-25T14:03:56.493822+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:28.167574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773362+0000","last_change":"2024-04-25T14:03:55.677958+0000","last_active":"2024-04-25T14:03:56.773362+0000","last_peered":"2024-04-25T14:03:56.773362+0000","last_clean":"2024-04-25T14:03:56.773362+0000","last_became_active":"2024-04-25T14:03:55.677471+0000","last_became_peered":"2024-04-25T14:03:55.677471+0000","last_unstale":"2024-04-25T14:03:56.773362+0000","last_undegraded":"2024-04-25T14:03:56.773362+0000","last_fullsized":"2024-04-25T14:03:56.773362+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:12.890323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772866+0000","last_change":"2024-04-25T14:03:51.477601+0000","last_active":"2024-04-25T14:03:56.772866+0000","last_peered":"2024-04-25T14:03:56.772866+0000","last_clean":"2024-04-25T14:03:56.772866+0000","last_became_active":"2024-04-25T14:03:51.477318+0000","last_became_peered":"2024-04-25T14:03:51.477318+0000","last_unstale":"2024-04-25T14:03:56.772866+0000","last_undegraded":"2024-04-25T14:03:56.772866+0000","last_fullsized":"2024-04-25T14:03:56.772866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.599283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494934+0000","last_change":"2024-04-25T14:03:50.456493+0000","last_active":"2024-04-25T14:03:56.494934+0000","last_peered":"2024-04-25T14:03:56.494934+0000","last_clean":"2024-04-25T14:03:56.494934+0000","last_became_active":"2024-04-25T14:03:50.454242+0000","last_became_peered":"2024-04-25T14:03:50.454242+0000","last_unstale":"2024-04-25T14:03:56.494934+0000","last_undegraded":"2024-04-25T14:03:56.494934+0000","last_fullsized":"2024-04-25T14:03:56.494934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:10.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496401+0000","last_change":"2024-04-25T14:03:55.516548+0000","last_active":"2024-04-25T14:03:56.496401+0000","last_peered":"2024-04-25T14:03:56.496401+0000","last_clean":"2024-04-25T14:03:56.496401+0000","last_became_active":"2024-04-25T14:03:55.515311+0000","last_became_peered":"2024-04-25T14:03:55.515311+0000","last_unstale":"2024-04-25T14:03:56.496401+0000","last_undegraded":"2024-04-25T14:03:56.496401+0000","last_fullsized":"2024-04-25T14:03:56.496401+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:51.371333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493692+0000","last_change":"2024-04-25T14:03:51.468740+0000","last_active":"2024-04-25T14:03:56.493692+0000","last_peered":"2024-04-25T14:03:56.493692+0000","last_clean":"2024-04-25T14:03:56.493692+0000","last_became_active":"2024-04-25T14:03:51.468366+0000","last_became_peered":"2024-04-25T14:03:51.468366+0000","last_unstale":"2024-04-25T14:03:56.493692+0000","last_undegraded":"2024-04-25T14:03:56.493692+0000","last_fullsized":"2024-04-25T14:03:56.493692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:31.707048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493992+0000","last_change":"2024-04-25T14:03:50.466457+0000","last_active":"2024-04-25T14:03:56.493992+0000","last_peered":"2024-04-25T14:03:56.493992+0000","last_clean":"2024-04-25T14:03:56.493992+0000","last_became_active":"2024-04-25T14:03:50.466070+0000","last_became_peered":"2024-04-25T14:03:50.466070+0000","last_unstale":"2024-04-25T14:03:56.493992+0000","last_undegraded":"2024-04-25T14:03:56.493992+0000","last_fullsized":"2024-04-25T14:03:56.493992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:15.520202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495294+0000","last_change":"2024-04-25T14:03:50.469128+0000","last_active":"2024-04-25T14:03:56.495294+0000","last_peered":"2024-04-25T14:03:56.495294+0000","last_clean":"2024-04-25T14:03:56.495294+0000","last_became_active":"2024-04-25T14:03:50.468409+0000","last_became_peered":"2024-04-25T14:03:50.468409+0000","last_unstale":"2024-04-25T14:03:56.495294+0000","last_undegraded":"2024-04-25T14:03:56.495294+0000","last_fullsized":"2024-04-25T14:03:56.495294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:15.520202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773570+0000","last_change":"2024-04-25T14:03:51.502539+0000","last_active":"2024-04-25T14:03:56.773570+0000","last_peered":"2024-04-25T14:03:56.773570+0000","last_clean":"2024-04-25T14:03:56.773570+0000","last_became_active":"2024-04-25T14:03:51.502229+0000","last_became_peered":"2024-04-25T14:03:51.502229+0000","last_unstale":"2024-04-25T14:03:56.773570+0000","last_undegraded":"2024-04-25T14:03:56.773570+0000","last_fullsized":"2024-04-25T14:03:56.773570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:20.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772371+0000","last_change":"2024-04-25T14:03:55.524446+0000","last_active":"2024-04-25T14:03:56.772371+0000","last_peered":"2024-04-25T14:03:56.772371+0000","last_clean":"2024-04-25T14:03:56.772371+0000","last_became_active":"2024-04-25T14:03:55.523886+0000","last_became_peered":"2024-04-25T14:03:55.523886+0000","last_unstale":"2024-04-25T14:03:56.772371+0000","last_undegraded":"2024-04-25T14:03:56.772371+0000","last_fullsized":"2024-04-25T14:03:56.772371+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:37.811235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493654+0000","last_change":"2024-04-25T14:03:50.472564+0000","last_active":"2024-04-25T14:03:56.493654+0000","last_peered":"2024-04-25T14:03:56.493654+0000","last_clean":"2024-04-25T14:03:56.493654+0000","last_became_active":"2024-04-25T14:03:50.472297+0000","last_became_peered":"2024-04-25T14:03:50.472297+0000","last_unstale":"2024-04-25T14:03:56.493654+0000","last_undegraded":"2024-04-25T14:03:56.493654+0000","last_fullsized":"2024-04-25T14:03:56.493654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:49.429161+0000","last_clean_scrub_stamp":"2024-04-25T14:03:49.429161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:43.281768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772419+0000","last_change":"2024-04-25T14:03:51.486031+0000","last_active":"2024-04-25T14:03:56.772419+0000","last_peered":"2024-04-25T14:03:56.772419+0000","last_clean":"2024-04-25T14:03:56.772419+0000","last_became_active":"2024-04-25T14:03:51.477708+0000","last_became_peered":"2024-04-25T14:03:51.477708+0000","last_unstale":"2024-04-25T14:03:56.772419+0000","last_undegraded":"2024-04-25T14:03:56.772419+0000","last_fullsized":"2024-04-25T14:03:56.772419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:11.859288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494673+0000","last_change":"2024-04-25T14:03:55.519123+0000","last_active":"2024-04-25T14:03:56.494673+0000","last_peered":"2024-04-25T14:03:56.494673+0000","last_clean":"2024-04-25T14:03:56.494673+0000","last_became_active":"2024-04-25T14:03:55.518743+0000","last_became_peered":"2024-04-25T14:03:55.518743+0000","last_unstale":"2024-04-25T14:03:56.494673+0000","last_undegraded":"2024-04-25T14:03:56.494673+0000","last_fullsized":"2024-04-25T14:03:56.494673+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:54.479600+0000","last_clean_scrub_stamp":"2024-04-25T14:03:54.479600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:18.368759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494540+0000","last_change":"2024-04-25T14:03:51.480209+0000","last_active":"2024-04-25T14:03:56.494540+0000","last_peered":"2024-04-25T14:03:56.494540+0000","last_clean":"2024-04-25T14:03:56.494540+0000","last_became_active":"2024-04-25T14:03:51.479824+0000","last_became_peered":"2024-04-25T14:03:51.479824+0000","last_unstale":"2024-04-25T14:03:56.494540+0000","last_undegraded":"2024-04-25T14:03:56.494540+0000","last_fullsized":"2024-04-25T14:03:56.494540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:31.707048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.771783+0000","last_change":"2024-04-25T14:03:51.486748+0000","last_active":"2024-04-25T14:03:56.771783+0000","last_peered":"2024-04-25T14:03:56.771783+0000","last_clean":"2024-04-25T14:03:56.771783+0000","last_became_active":"2024-04-25T14:03:51.486343+0000","last_became_peered":"2024-04-25T14:03:51.486343+0000","last_unstale":"2024-04-25T14:03:56.771783+0000","last_undegraded":"2024-04-25T14:03:56.771783+0000","last_fullsized":"2024-04-25T14:03:56.771783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:46.086179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495470+0000","last_change":"2024-04-25T14:03:51.465637+0000","last_active":"2024-04-25T14:03:56.495470+0000","last_peered":"2024-04-25T14:03:56.495470+0000","last_clean":"2024-04-25T14:03:56.495470+0000","last_became_active":"2024-04-25T14:03:51.465339+0000","last_became_peered":"2024-04-25T14:03:51.465339+0000","last_unstale":"2024-04-25T14:03:56.495470+0000","last_undegraded":"2024-04-25T14:03:56.495470+0000","last_fullsized":"2024-04-25T14:03:56.495470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:21.406920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773508+0000","last_change":"2024-04-25T14:03:51.475134+0000","last_active":"2024-04-25T14:03:56.773508+0000","last_peered":"2024-04-25T14:03:56.773508+0000","last_clean":"2024-04-25T14:03:56.773508+0000","last_became_active":"2024-04-25T14:03:51.474796+0000","last_became_peered":"2024-04-25T14:03:51.474796+0000","last_unstale":"2024-04-25T14:03:56.773508+0000","last_undegraded":"2024-04-25T14:03:56.773508+0000","last_fullsized":"2024-04-25T14:03:56.773508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:31.707048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494247+0000","last_change":"2024-04-25T14:03:51.494845+0000","last_active":"2024-04-25T14:03:56.494247+0000","last_peered":"2024-04-25T14:03:56.494247+0000","last_clean":"2024-04-25T14:03:56.494247+0000","last_became_active":"2024-04-25T14:03:51.494547+0000","last_became_peered":"2024-04-25T14:03:51.494547+0000","last_unstale":"2024-04-25T14:03:56.494247+0000","last_undegraded":"2024-04-25T14:03:56.494247+0000","last_fullsized":"2024-04-25T14:03:56.494247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:16.813529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773722+0000","last_change":"2024-04-25T14:03:51.478241+0000","last_active":"2024-04-25T14:03:56.773722+0000","last_peered":"2024-04-25T14:03:56.773722+0000","last_clean":"2024-04-25T14:03:56.773722+0000","last_became_active":"2024-04-25T14:03:51.477955+0000","last_became_peered":"2024-04-25T14:03:51.477955+0000","last_unstale":"2024-04-25T14:03:56.773722+0000","last_undegraded":"2024-04-25T14:03:56.773722+0000","last_fullsized":"2024-04-25T14:03:56.773722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:43.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496093+0000","last_change":"2024-04-25T14:03:51.478644+0000","last_active":"2024-04-25T14:03:56.496093+0000","last_peered":"2024-04-25T14:03:56.496093+0000","last_clean":"2024-04-25T14:03:56.496093+0000","last_became_active":"2024-04-25T14:03:51.478326+0000","last_became_peered":"2024-04-25T14:03:51.478326+0000","last_unstale":"2024-04-25T14:03:56.496093+0000","last_undegraded":"2024-04-25T14:03:56.496093+0000","last_fullsized":"2024-04-25T14:03:56.496093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:20.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495116+0000","last_change":"2024-04-25T14:03:51.506061+0000","last_active":"2024-04-25T14:03:56.495116+0000","last_peered":"2024-04-25T14:03:56.495116+0000","last_clean":"2024-04-25T14:03:56.495116+0000","last_became_active":"2024-04-25T14:03:51.505804+0000","last_became_peered":"2024-04-25T14:03:51.505804+0000","last_unstale":"2024-04-25T14:03:56.495116+0000","last_undegraded":"2024-04-25T14:03:56.495116+0000","last_fullsized":"2024-04-25T14:03:56.495116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:55.876775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494831+0000","last_change":"2024-04-25T14:03:51.465574+0000","last_active":"2024-04-25T14:03:56.494831+0000","last_peered":"2024-04-25T14:03:56.494831+0000","last_clean":"2024-04-25T14:03:56.494831+0000","last_became_active":"2024-04-25T14:03:51.465206+0000","last_became_peered":"2024-04-25T14:03:51.465206+0000","last_unstale":"2024-04-25T14:03:56.494831+0000","last_undegraded":"2024-04-25T14:03:56.494831+0000","last_fullsized":"2024-04-25T14:03:56.494831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:16.813529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493905+0000","last_change":"2024-04-25T14:03:51.486782+0000","last_active":"2024-04-25T14:03:56.493905+0000","last_peered":"2024-04-25T14:03:56.493905+0000","last_clean":"2024-04-25T14:03:56.493905+0000","last_became_active":"2024-04-25T14:03:51.486465+0000","last_became_peered":"2024-04-25T14:03:51.486465+0000","last_unstale":"2024-04-25T14:03:56.493905+0000","last_undegraded":"2024-04-25T14:03:56.493905+0000","last_fullsized":"2024-04-25T14:03:56.493905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:39.949870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495357+0000","last_change":"2024-04-25T14:03:51.468022+0000","last_active":"2024-04-25T14:03:56.495357+0000","last_peered":"2024-04-25T14:03:56.495357+0000","last_clean":"2024-04-25T14:03:56.495357+0000","last_became_active":"2024-04-25T14:03:51.467637+0000","last_became_peered":"2024-04-25T14:03:51.467637+0000","last_unstale":"2024-04-25T14:03:56.495357+0000","last_undegraded":"2024-04-25T14:03:56.495357+0000","last_fullsized":"2024-04-25T14:03:56.495357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:55.876775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773216+0000","last_change":"2024-04-25T14:03:51.469881+0000","last_active":"2024-04-25T14:03:56.773216+0000","last_peered":"2024-04-25T14:03:56.773216+0000","last_clean":"2024-04-25T14:03:56.773216+0000","last_became_active":"2024-04-25T14:03:51.469572+0000","last_became_peered":"2024-04-25T14:03:51.469572+0000","last_unstale":"2024-04-25T14:03:56.773216+0000","last_undegraded":"2024-04-25T14:03:56.773216+0000","last_fullsized":"2024-04-25T14:03:56.773216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.599283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495680+0000","last_change":"2024-04-25T14:03:51.494605+0000","last_active":"2024-04-25T14:03:56.495680+0000","last_peered":"2024-04-25T14:03:56.495680+0000","last_clean":"2024-04-25T14:03:56.495680+0000","last_became_active":"2024-04-25T14:03:51.494257+0000","last_became_peered":"2024-04-25T14:03:51.494257+0000","last_unstale":"2024-04-25T14:03:56.495680+0000","last_undegraded":"2024-04-25T14:03:56.495680+0000","last_fullsized":"2024-04-25T14:03:56.495680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:39.949870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773065+0000","last_change":"2024-04-25T14:03:51.476473+0000","last_active":"2024-04-25T14:03:56.773065+0000","last_peered":"2024-04-25T14:03:56.773065+0000","last_clean":"2024-04-25T14:03:56.773065+0000","last_became_active":"2024-04-25T14:03:51.475857+0000","last_became_peered":"2024-04-25T14:03:51.475857+0000","last_unstale":"2024-04-25T14:03:56.773065+0000","last_undegraded":"2024-04-25T14:03:56.773065+0000","last_fullsized":"2024-04-25T14:03:56.773065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:46.189697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772375+0000","last_change":"2024-04-25T14:03:51.479889+0000","last_active":"2024-04-25T14:03:56.772375+0000","last_peered":"2024-04-25T14:03:56.772375+0000","last_clean":"2024-04-25T14:03:56.772375+0000","last_became_active":"2024-04-25T14:03:51.478793+0000","last_became_peered":"2024-04-25T14:03:51.478793+0000","last_unstale":"2024-04-25T14:03:56.772375+0000","last_undegraded":"2024-04-25T14:03:56.772375+0000","last_fullsized":"2024-04-25T14:03:56.772375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:38.411753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496637+0000","last_change":"2024-04-25T14:03:51.480313+0000","last_active":"2024-04-25T14:03:56.496637+0000","last_peered":"2024-04-25T14:03:56.496637+0000","last_clean":"2024-04-25T14:03:56.496637+0000","last_became_active":"2024-04-25T14:03:51.479875+0000","last_became_peered":"2024-04-25T14:03:51.479875+0000","last_unstale":"2024-04-25T14:03:56.496637+0000","last_undegraded":"2024-04-25T14:03:56.496637+0000","last_fullsized":"2024-04-25T14:03:56.496637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:02.773422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.494729+0000","last_change":"2024-04-25T14:03:51.472514+0000","last_active":"2024-04-25T14:03:56.494729+0000","last_peered":"2024-04-25T14:03:56.494729+0000","last_clean":"2024-04-25T14:03:56.494729+0000","last_became_active":"2024-04-25T14:03:51.472148+0000","last_became_peered":"2024-04-25T14:03:51.472148+0000","last_unstale":"2024-04-25T14:03:56.494729+0000","last_undegraded":"2024-04-25T14:03:56.494729+0000","last_fullsized":"2024-04-25T14:03:56.494729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:20.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772899+0000","last_change":"2024-04-25T14:03:51.489086+0000","last_active":"2024-04-25T14:03:56.772899+0000","last_peered":"2024-04-25T14:03:56.772899+0000","last_clean":"2024-04-25T14:03:56.772899+0000","last_became_active":"2024-04-25T14:03:51.488810+0000","last_became_peered":"2024-04-25T14:03:51.488810+0000","last_unstale":"2024-04-25T14:03:56.772899+0000","last_undegraded":"2024-04-25T14:03:56.772899+0000","last_fullsized":"2024-04-25T14:03:56.772899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:39.949870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772674+0000","last_change":"2024-04-25T14:03:51.475508+0000","last_active":"2024-04-25T14:03:56.772674+0000","last_peered":"2024-04-25T14:03:56.772674+0000","last_clean":"2024-04-25T14:03:56.772674+0000","last_became_active":"2024-04-25T14:03:51.475146+0000","last_became_peered":"2024-04-25T14:03:51.475146+0000","last_unstale":"2024-04-25T14:03:56.772674+0000","last_undegraded":"2024-04-25T14:03:56.772674+0000","last_fullsized":"2024-04-25T14:03:56.772674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:34.374126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493554+0000","last_change":"2024-04-25T14:03:51.479243+0000","last_active":"2024-04-25T14:03:56.493554+0000","last_peered":"2024-04-25T14:03:56.493554+0000","last_clean":"2024-04-25T14:03:56.493554+0000","last_became_active":"2024-04-25T14:03:51.478899+0000","last_became_peered":"2024-04-25T14:03:51.478899+0000","last_unstale":"2024-04-25T14:03:56.493554+0000","last_undegraded":"2024-04-25T14:03:56.493554+0000","last_fullsized":"2024-04-25T14:03:56.493554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:20.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773258+0000","last_change":"2024-04-25T14:03:51.493878+0000","last_active":"2024-04-25T14:03:56.773258+0000","last_peered":"2024-04-25T14:03:56.773258+0000","last_clean":"2024-04-25T14:03:56.773258+0000","last_became_active":"2024-04-25T14:03:51.493501+0000","last_became_peered":"2024-04-25T14:03:51.493501+0000","last_unstale":"2024-04-25T14:03:56.773258+0000","last_undegraded":"2024-04-25T14:03:56.773258+0000","last_fullsized":"2024-04-25T14:03:56.773258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:46.086179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496241+0000","last_change":"2024-04-25T14:03:51.489072+0000","last_active":"2024-04-25T14:03:56.496241+0000","last_peered":"2024-04-25T14:03:56.496241+0000","last_clean":"2024-04-25T14:03:56.496241+0000","last_became_active":"2024-04-25T14:03:51.488895+0000","last_became_peered":"2024-04-25T14:03:51.488895+0000","last_unstale":"2024-04-25T14:03:56.496241+0000","last_undegraded":"2024-04-25T14:03:56.496241+0000","last_fullsized":"2024-04-25T14:03:56.496241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:10.318378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496156+0000","last_change":"2024-04-25T14:03:51.488196+0000","last_active":"2024-04-25T14:03:56.496156+0000","last_peered":"2024-04-25T14:03:56.496156+0000","last_clean":"2024-04-25T14:03:56.496156+0000","last_became_active":"2024-04-25T14:03:51.487833+0000","last_became_peered":"2024-04-25T14:03:51.487833+0000","last_unstale":"2024-04-25T14:03:56.496156+0000","last_undegraded":"2024-04-25T14:03:56.496156+0000","last_fullsized":"2024-04-25T14:03:56.496156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:43.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773432+0000","last_change":"2024-04-25T14:03:51.482441+0000","last_active":"2024-04-25T14:03:56.773432+0000","last_peered":"2024-04-25T14:03:56.773432+0000","last_clean":"2024-04-25T14:03:56.773432+0000","last_became_active":"2024-04-25T14:03:51.482089+0000","last_became_peered":"2024-04-25T14:03:51.482089+0000","last_unstale":"2024-04-25T14:03:56.773432+0000","last_undegraded":"2024-04-25T14:03:56.773432+0000","last_fullsized":"2024-04-25T14:03:56.773432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:02.697952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.493326+0000","last_change":"2024-04-25T14:03:51.487289+0000","last_active":"2024-04-25T14:03:56.493326+0000","last_peered":"2024-04-25T14:03:56.493326+0000","last_clean":"2024-04-25T14:03:56.493326+0000","last_became_active":"2024-04-25T14:03:51.486948+0000","last_became_peered":"2024-04-25T14:03:51.486948+0000","last_unstale":"2024-04-25T14:03:56.493326+0000","last_undegraded":"2024-04-25T14:03:56.493326+0000","last_fullsized":"2024-04-25T14:03:56.493326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.599283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772227+0000","last_change":"2024-04-25T14:03:51.473228+0000","last_active":"2024-04-25T14:03:56.772227+0000","last_peered":"2024-04-25T14:03:56.772227+0000","last_clean":"2024-04-25T14:03:56.772227+0000","last_became_active":"2024-04-25T14:03:51.472850+0000","last_became_peered":"2024-04-25T14:03:51.472850+0000","last_unstale":"2024-04-25T14:03:56.772227+0000","last_undegraded":"2024-04-25T14:03:56.772227+0000","last_fullsized":"2024-04-25T14:03:56.772227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:34.707119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.495222+0000","last_change":"2024-04-25T14:03:51.467162+0000","last_active":"2024-04-25T14:03:56.495222+0000","last_peered":"2024-04-25T14:03:56.495222+0000","last_clean":"2024-04-25T14:03:56.495222+0000","last_became_active":"2024-04-25T14:03:51.466616+0000","last_became_peered":"2024-04-25T14:03:51.466616+0000","last_unstale":"2024-04-25T14:03:56.495222+0000","last_undegraded":"2024-04-25T14:03:56.495222+0000","last_fullsized":"2024-04-25T14:03:56.495222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:34.707119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.772199+0000","last_change":"2024-04-25T14:03:51.476149+0000","last_active":"2024-04-25T14:03:56.772199+0000","last_peered":"2024-04-25T14:03:56.772199+0000","last_clean":"2024-04-25T14:03:56.772199+0000","last_became_active":"2024-04-25T14:03:51.475934+0000","last_became_peered":"2024-04-25T14:03:51.475934+0000","last_unstale":"2024-04-25T14:03:56.772199+0000","last_undegraded":"2024-04-25T14:03:56.772199+0000","last_fullsized":"2024-04-25T14:03:56.772199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:17.381249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.496219+0000","last_change":"2024-04-25T14:03:51.475153+0000","last_active":"2024-04-25T14:03:56.496219+0000","last_peered":"2024-04-25T14:03:56.496219+0000","last_clean":"2024-04-25T14:03:56.496219+0000","last_became_active":"2024-04-25T14:03:51.474862+0000","last_became_peered":"2024-04-25T14:03:51.474862+0000","last_unstale":"2024-04-25T14:03:56.496219+0000","last_undegraded":"2024-04-25T14:03:56.496219+0000","last_fullsized":"2024-04-25T14:03:56.496219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:05.489889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773218+0000","last_change":"2024-04-25T14:03:51.460567+0000","last_active":"2024-04-25T14:03:56.773218+0000","last_peered":"2024-04-25T14:03:56.773218+0000","last_clean":"2024-04-25T14:03:56.773218+0000","last_became_active":"2024-04-25T14:03:51.460193+0000","last_became_peered":"2024-04-25T14:03:51.460193+0000","last_unstale":"2024-04-25T14:03:56.773218+0000","last_undegraded":"2024-04-25T14:03:56.773218+0000","last_fullsized":"2024-04-25T14:03:56.773218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:39.949870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.774153+0000","last_change":"2024-04-25T14:03:51.480094+0000","last_active":"2024-04-25T14:03:56.774153+0000","last_peered":"2024-04-25T14:03:56.774153+0000","last_clean":"2024-04-25T14:03:56.774153+0000","last_became_active":"2024-04-25T14:03:51.479529+0000","last_became_peered":"2024-04-25T14:03:51.479529+0000","last_unstale":"2024-04-25T14:03:56.774153+0000","last_undegraded":"2024-04-25T14:03:56.774153+0000","last_fullsized":"2024-04-25T14:03:56.774153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:10.318378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T14:03:56.773366+0000","last_change":"2024-04-25T14:03:51.481397+0000","last_active":"2024-04-25T14:03:56.773366+0000","last_peered":"2024-04-25T14:03:56.773366+0000","last_clean":"2024-04-25T14:03:56.773366+0000","last_became_active":"2024-04-25T14:03:51.481153+0000","last_became_peered":"2024-04-25T14:03:51.481153+0000","last_unstale":"2024-04-25T14:03:56.773366+0000","last_undegraded":"2024-04-25T14:03:56.773366+0000","last_fullsized":"2024-04-25T14:03:56.773366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:50.434623+0000","last_clean_scrub_stamp":"2024-04-25T14:03:50.434623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:21.406920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:04:16.102 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T14:04:16.102 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T14:04:16.102 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T14:04:16.102 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T14:04:16.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.285+0000 7f640c000640 1 Processor -- start 2024-04-25T14:04:16.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.285+0000 7f640c000640 1 -- start start 2024-04-25T14:04:16.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.285+0000 7f640c000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f64040cec60 0x7f64040cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.285+0000 7f640c000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64040cf5b0 0x7f64040ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.285+0000 7f640c000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6404163940 con 0x7f64040cf9c0 2024-04-25T14:04:16.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.285+0000 7f640c000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f64040d2f40 con 0x7f64040cec60 2024-04-25T14:04:16.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.285+0000 7f640c000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f64040b3720 con 0x7f64040cf5b0 2024-04-25T14:04:16.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.285+0000 7f6409600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f64040cec60 0x7f64040cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.285+0000 7f640a000640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f64040cf9c0 0x7f64040cd2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:50918/0 (socket says 172.21.15.86:50918) 2024-04-25T14:04:16.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.285+0000 7f640a000640 1 -- 172.21.15.86:0/3388383086 learned_addr learned my addr 172.21.15.86:0/3388383086 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:16.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.285+0000 7f6408c00640 1 --2- 172.21.15.86:0/3388383086 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64040cf5b0 0x7f64040ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.285+0000 7f6409600640 1 -- 172.21.15.86:0/3388383086 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64040cf5b0 msgr2=0x7f64040ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:16.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.285+0000 7f6409600640 1 --2- 172.21.15.86:0/3388383086 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64040cf5b0 0x7f64040ccd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.285+0000 7f6409600640 1 -- 172.21.15.86:0/3388383086 >> v1:172.21.15.155:6789/0 conn(0x7f64040cf9c0 legacy=0x7f64040cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:16.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.285+0000 7f6409600640 1 -- 172.21.15.86:0/3388383086 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64040b3a00 con 0x7f64040cec60 2024-04-25T14:04:16.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.285+0000 7f6403e00640 1 -- 172.21.15.86:0/3388383086 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4098213202 0 0) 0x7f6404163940 con 0x7f64040cf9c0 2024-04-25T14:04:16.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.285+0000 7f6408c00640 1 --2- 172.21.15.86:0/3388383086 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64040cf5b0 0x7f64040ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:16.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.289+0000 7f6409600640 1 --2- 172.21.15.86:0/3388383086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64040cec60 0x7f64040cf070 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f63f80097a0 tx=0x7f64040ce1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a96a1e0c4635827 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:16.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.289+0000 7f6403e00640 1 -- 172.21.15.86:0/3388383086 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f80b2070 con 0x7f64040cec60 2024-04-25T14:04:16.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.289+0000 7f6403e00640 1 -- 172.21.15.86:0/3388383086 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63f80ad640 con 0x7f64040cec60 2024-04-25T14:04:16.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.289+0000 7f6403e00640 1 -- 172.21.15.86:0/3388383086 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f80ada10 con 0x7f64040cec60 2024-04-25T14:04:16.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.289+0000 7f640c000640 1 -- 172.21.15.86:0/3388383086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64040cec60 msgr2=0x7f64040cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:16.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.289+0000 7f640c000640 1 --2- 172.21.15.86:0/3388383086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64040cec60 0x7f64040cf070 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f63f80097a0 tx=0x7f64040ce1f0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.289+0000 7f640c000640 1 -- 172.21.15.86:0/3388383086 shutdown_connections 2024-04-25T14:04:16.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.289+0000 7f640c000640 1 --2- 172.21.15.86:0/3388383086 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64040cf5b0 0x7f64040ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.289+0000 7f640c000640 1 --2- 172.21.15.86:0/3388383086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64040cec60 0x7f64040cf070 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.289+0000 7f640c000640 1 -- 172.21.15.86:0/3388383086 >> 172.21.15.86:0/3388383086 conn(0x7f640409ba30 msgr2=0x7f64040b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:16.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.289+0000 7f640c000640 1 -- 172.21.15.86:0/3388383086 shutdown_connections 2024-04-25T14:04:16.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.289+0000 7f640c000640 1 -- 172.21.15.86:0/3388383086 wait complete. 2024-04-25T14:04:16.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.293+0000 7f640c000640 1 Processor -- start 2024-04-25T14:04:16.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.293+0000 7f640c000640 1 -- start start 2024-04-25T14:04:16.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.293+0000 7f640c000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64040cec60 0x7f64040b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.293+0000 7f640c000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64040cf5b0 0x7f640414a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.293+0000 7f640c000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64040cf9c0 0x7f64040afc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.293+0000 7f640c000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f640416ce00 con 0x7f64040cf5b0 2024-04-25T14:04:16.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.293+0000 7f640c000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f64040d2d00 con 0x7f64040cf9c0 2024-04-25T14:04:16.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.293+0000 7f640c000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f640416cf80 con 0x7f64040cec60 2024-04-25T14:04:16.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.293+0000 7f6408c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64040cf5b0 0x7f640414a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.293+0000 7f6408c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64040cf5b0 0x7f640414a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38288/0 (socket says 172.21.15.86:38288) 2024-04-25T14:04:16.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.293+0000 7f6408c00640 1 -- 172.21.15.86:0/1897514507 learned_addr learned my addr 172.21.15.86:0/1897514507 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:16.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.293+0000 7f640a000640 1 --2- 172.21.15.86:0/1897514507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64040cf9c0 0x7f64040afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.293+0000 7f6409600640 1 --2- 172.21.15.86:0/1897514507 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64040cec60 0x7f64040b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.297+0000 7f6408c00640 1 -- 172.21.15.86:0/1897514507 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64040cec60 msgr2=0x7f64040b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:16.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.297+0000 7f6408c00640 1 --2- 172.21.15.86:0/1897514507 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64040cec60 0x7f64040b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.297+0000 7f6408c00640 1 -- 172.21.15.86:0/1897514507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64040cf9c0 msgr2=0x7f64040afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:16.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.297+0000 7f6408c00640 1 --2- 172.21.15.86:0/1897514507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64040cf9c0 0x7f64040afc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.297+0000 7f6408c00640 1 -- 172.21.15.86:0/1897514507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640415d2e0 con 0x7f64040cf5b0 2024-04-25T14:04:16.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.297+0000 7f6408c00640 1 --2- 172.21.15.86:0/1897514507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64040cf5b0 0x7f640414a970 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f63f000e980 tx=0x7f63f000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:16.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.297+0000 7f6401600640 1 -- 172.21.15.86:0/1897514507 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f0017070 con 0x7f64040cf5b0 2024-04-25T14:04:16.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.297+0000 7f6401600640 1 -- 172.21.15.86:0/1897514507 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63f000ca00 con 0x7f64040cf5b0 2024-04-25T14:04:16.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.297+0000 7f640c000640 1 -- 172.21.15.86:0/1897514507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64040b2100 con 0x7f64040cf5b0 2024-04-25T14:04:16.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.297+0000 7f6401600640 1 -- 172.21.15.86:0/1897514507 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f000cdd0 con 0x7f64040cf5b0 2024-04-25T14:04:16.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.297+0000 7f640c000640 1 -- 172.21.15.86:0/1897514507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f640415dac0 con 0x7f64040cf5b0 2024-04-25T14:04:16.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.297+0000 7f640c000640 1 -- 172.21.15.86:0/1897514507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f640415d550 con 0x7f64040cf5b0 2024-04-25T14:04:16.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.297+0000 7f6401600640 1 -- 172.21.15.86:0/1897514507 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f63f001a070 con 0x7f64040cf5b0 2024-04-25T14:04:16.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.297+0000 7f6401600640 1 --2- 172.21.15.86:0/1897514507 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f63e406f400 0x7f63e4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.301+0000 7f6401600640 1 -- 172.21.15.86:0/1897514507 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f63f008f490 con 0x7f64040cf5b0 2024-04-25T14:04:16.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.301+0000 7f6409600640 1 --2- 172.21.15.86:0/1897514507 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f63e406f400 0x7f63e4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.301+0000 7f6409600640 1 --2- 172.21.15.86:0/1897514507 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f63e406f400 0x7f63e4071910 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f63f80af000 tx=0x7f63f8004740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:16.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.305+0000 7f6401600640 1 -- 172.21.15.86:0/1897514507 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f63f005b9e0 con 0x7f64040cf5b0 2024-04-25T14:04:16.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.493+0000 7f640c000640 1 -- 172.21.15.86:0/1897514507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f64040a04c0 con 0x7f64040cf5b0 2024-04-25T14:04:16.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.493+0000 7f6401600640 1 -- 172.21.15.86:0/1897514507 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f63f005fce0 con 0x7f64040cf5b0 2024-04-25T14:04:16.496 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:16.496 INFO:teuthology.orchestra.run.smithi086.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T14:04:16.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.497+0000 7f640c000640 1 -- 172.21.15.86:0/1897514507 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f63e406f400 msgr2=0x7f63e4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:16.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.497+0000 7f640c000640 1 --2- 172.21.15.86:0/1897514507 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f63e406f400 0x7f63e4071910 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f63f80af000 tx=0x7f63f8004740 comp rx=0 tx=0).stop 2024-04-25T14:04:16.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.497+0000 7f640c000640 1 -- 172.21.15.86:0/1897514507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64040cf5b0 msgr2=0x7f640414a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:16.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.497+0000 7f640c000640 1 --2- 172.21.15.86:0/1897514507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64040cf5b0 0x7f640414a970 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f63f000e980 tx=0x7f63f000ee40 comp rx=0 tx=0).stop 2024-04-25T14:04:16.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.497+0000 7f640c000640 1 -- 172.21.15.86:0/1897514507 shutdown_connections 2024-04-25T14:04:16.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.497+0000 7f640c000640 1 --2- 172.21.15.86:0/1897514507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64040cf9c0 0x7f64040afc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.497+0000 7f640c000640 1 --2- 172.21.15.86:0/1897514507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64040cf5b0 0x7f640414a970 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.497+0000 7f640c000640 1 --2- 172.21.15.86:0/1897514507 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f63e406f400 0x7f63e4071910 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.497+0000 7f640c000640 1 --2- 172.21.15.86:0/1897514507 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64040cec60 0x7f64040b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.497+0000 7f640c000640 1 -- 172.21.15.86:0/1897514507 >> 172.21.15.86:0/1897514507 conn(0x7f640409ba30 msgr2=0x7f640406e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:16.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.497+0000 7f640c000640 1 -- 172.21.15.86:0/1897514507 shutdown_connections 2024-04-25T14:04:16.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.497+0000 7f640c000640 1 -- 172.21.15.86:0/1897514507 wait complete. 2024-04-25T14:04:16.518 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T14:04:16.518 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:04:16.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.661+0000 7f89f4c00640 1 Processor -- start 2024-04-25T14:04:16.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.661+0000 7f89f4c00640 1 -- start start 2024-04-25T14:04:16.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.661+0000 7f89f4c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f89f00b59e0 0x7f89f00b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.661+0000 7f89f4c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89f00b6330 con 0x7f89f015faa0 2024-04-25T14:04:16.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.661+0000 7f89f4c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89f006c670 con 0x7f89f00aeea0 2024-04-25T14:04:16.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.661+0000 7f89f4c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f89f006d560 con 0x7f89f00b59e0 2024-04-25T14:04:16.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.665+0000 7f89e5400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f89f00b59e0 0x7f89f00b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.665+0000 7f89ede00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f89f015faa0 0x7f89f006d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33506/0 (socket says 172.21.15.86:33506) 2024-04-25T14:04:16.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.665+0000 7f89ede00640 1 -- 172.21.15.86:0/1055291831 learned_addr learned my addr 172.21.15.86:0/1055291831 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:16.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.665+0000 7f89e5400640 1 -- 172.21.15.86:0/1055291831 >> v1:172.21.15.155:6789/0 conn(0x7f89f00aeea0 legacy=0x7f89f00af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:16.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.665+0000 7f89e5400640 1 -- 172.21.15.86:0/1055291831 >> v1:172.21.15.86:6789/0 conn(0x7f89f015faa0 legacy=0x7f89f006d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:16.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.665+0000 7f89ed400640 1 -- 172.21.15.86:0/1055291831 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 907338913 0 0) 0x7f89e0001120 con 0x7f89f015faa0 2024-04-25T14:04:16.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.665+0000 7f89e5400640 1 -- 172.21.15.86:0/1055291831 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f006c950 con 0x7f89f00b59e0 2024-04-25T14:04:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.665+0000 7f89e5400640 1 --2- 172.21.15.86:0/1055291831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89f00b59e0 0x7f89f00b5df0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f89d80097c0 tx=0x7f89d801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4014dfc4348b42fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.665+0000 7f89ed400640 1 -- 172.21.15.86:0/1055291831 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d80aa620 con 0x7f89f00b59e0 2024-04-25T14:04:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.665+0000 7f89ed400640 1 -- 172.21.15.86:0/1055291831 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89d80af070 con 0x7f89f00b59e0 2024-04-25T14:04:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.665+0000 7f89ed400640 1 -- 172.21.15.86:0/1055291831 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d80aa9e0 con 0x7f89f00b59e0 2024-04-25T14:04:16.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.669+0000 7f89f4c00640 1 -- 172.21.15.86:0/1055291831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89f00b59e0 msgr2=0x7f89f00b5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:16.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.669+0000 7f89f4c00640 1 --2- 172.21.15.86:0/1055291831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89f00b59e0 0x7f89f00b5df0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f89d80097c0 tx=0x7f89d801c4a0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.669+0000 7f89f4c00640 1 -- 172.21.15.86:0/1055291831 shutdown_connections 2024-04-25T14:04:16.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.669+0000 7f89f4c00640 1 --2- 172.21.15.86:0/1055291831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89f00b59e0 0x7f89f00b5df0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.669+0000 7f89f4c00640 1 -- 172.21.15.86:0/1055291831 >> 172.21.15.86:0/1055291831 conn(0x7f89f00d2230 msgr2=0x7f89f00d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:16.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.669+0000 7f89f4c00640 1 -- 172.21.15.86:0/1055291831 shutdown_connections 2024-04-25T14:04:16.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.669+0000 7f89f4c00640 1 -- 172.21.15.86:0/1055291831 wait complete. 2024-04-25T14:04:16.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.669+0000 7f89f4c00640 1 Processor -- start 2024-04-25T14:04:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.669+0000 7f89f4c00640 1 -- start start 2024-04-25T14:04:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.673+0000 7f89f4c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89f00aeea0 0x7f89f015bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.673+0000 7f89f4c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89f00b59e0 0x7f89f015ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.673+0000 7f89f4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89f015faa0 0x7f89f0141120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.673+0000 7f89f4c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f89f015feb0 con 0x7f89f015faa0 2024-04-25T14:04:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.673+0000 7f89f4c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f89f00af9c0 con 0x7f89f00b59e0 2024-04-25T14:04:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.673+0000 7f89f4c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f89f0160030 con 0x7f89f00aeea0 2024-04-25T14:04:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.673+0000 7f89ee800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89f015faa0 0x7f89f0141120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.673+0000 7f89ee800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89f015faa0 0x7f89f0141120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38316/0 (socket says 172.21.15.86:38316) 2024-04-25T14:04:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.673+0000 7f89ee800640 1 -- 172.21.15.86:0/990700589 learned_addr learned my addr 172.21.15.86:0/990700589 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.673+0000 7f89e5400640 1 --2- 172.21.15.86:0/990700589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89f00b59e0 0x7f89f015ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.673+0000 7f89ede00640 1 --2- 172.21.15.86:0/990700589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89f00aeea0 0x7f89f015bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.673+0000 7f89ee800640 1 -- 172.21.15.86:0/990700589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89f00aeea0 msgr2=0x7f89f015bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.673+0000 7f89ee800640 1 --2- 172.21.15.86:0/990700589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89f00aeea0 0x7f89f015bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.673+0000 7f89ee800640 1 -- 172.21.15.86:0/990700589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89f00b59e0 msgr2=0x7f89f015ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.673+0000 7f89ee800640 1 --2- 172.21.15.86:0/990700589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89f00b59e0 0x7f89f015ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.673+0000 7f89ee800640 1 -- 172.21.15.86:0/990700589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f015e810 con 0x7f89f015faa0 2024-04-25T14:04:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.673+0000 7f89ee800640 1 --2- 172.21.15.86:0/990700589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89f015faa0 0x7f89f0141120 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f89e800c730 tx=0x7f89e800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.673+0000 7f89e6a00640 1 -- 172.21.15.86:0/990700589 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e8003b80 con 0x7f89f015faa0 2024-04-25T14:04:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.673+0000 7f89e6a00640 1 -- 172.21.15.86:0/990700589 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89e8003d20 con 0x7f89f015faa0 2024-04-25T14:04:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.673+0000 7f89f4c00640 1 -- 172.21.15.86:0/990700589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89f015eaf0 con 0x7f89f015faa0 2024-04-25T14:04:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.673+0000 7f89e6a00640 1 -- 172.21.15.86:0/990700589 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e800aa30 con 0x7f89f015faa0 2024-04-25T14:04:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.673+0000 7f89f4c00640 1 -- 172.21.15.86:0/990700589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89f015f0b0 con 0x7f89f015faa0 2024-04-25T14:04:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.673+0000 7f89f4c00640 1 -- 172.21.15.86:0/990700589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89f0074cf0 con 0x7f89f015faa0 2024-04-25T14:04:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.677+0000 7f89e6a00640 1 -- 172.21.15.86:0/990700589 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f89e800abd0 con 0x7f89f015faa0 2024-04-25T14:04:16.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.677+0000 7f89e6a00640 1 --2- 172.21.15.86:0/990700589 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f89cc06f430 0x7f89cc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.677+0000 7f89e6a00640 1 -- 172.21.15.86:0/990700589 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f89e808f3e0 con 0x7f89f015faa0 2024-04-25T14:04:16.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.677+0000 7f89ede00640 1 --2- 172.21.15.86:0/990700589 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f89cc06f430 0x7f89cc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.677+0000 7f89ede00640 1 --2- 172.21.15.86:0/990700589 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f89cc06f430 0x7f89cc071940 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f89e0000f70 tx=0x7f89e0007240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:16.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.681+0000 7f89e6a00640 1 -- 172.21.15.86:0/990700589 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f89e805b8a0 con 0x7f89f015faa0 2024-04-25T14:04:16.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.837+0000 7f89f4c00640 1 -- 172.21.15.86:0/990700589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f89f00b55b0 con 0x7f89f015faa0 2024-04-25T14:04:16.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.841+0000 7f89e6a00640 1 -- 172.21.15.86:0/990700589 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7f89e805fba0 con 0x7f89f015faa0 2024-04-25T14:04:16.841 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:16.841 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":24,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:03:55.899092+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:03:49.151473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:03:49.802597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T14:03:53.492008+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:04:16.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.841+0000 7f89f4c00640 1 -- 172.21.15.86:0/990700589 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f89cc06f430 msgr2=0x7f89cc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:16.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.841+0000 7f89f4c00640 1 --2- 172.21.15.86:0/990700589 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f89cc06f430 0x7f89cc071940 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f89e0000f70 tx=0x7f89e0007240 comp rx=0 tx=0).stop 2024-04-25T14:04:16.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.841+0000 7f89f4c00640 1 -- 172.21.15.86:0/990700589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89f015faa0 msgr2=0x7f89f0141120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:16.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.841+0000 7f89f4c00640 1 --2- 172.21.15.86:0/990700589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89f015faa0 0x7f89f0141120 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f89e800c730 tx=0x7f89e800cbf0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.841+0000 7f89f4c00640 1 -- 172.21.15.86:0/990700589 shutdown_connections 2024-04-25T14:04:16.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.841+0000 7f89f4c00640 1 --2- 172.21.15.86:0/990700589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89f015faa0 0x7f89f0141120 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.841+0000 7f89f4c00640 1 --2- 172.21.15.86:0/990700589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89f00b59e0 0x7f89f015ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.841+0000 7f89f4c00640 1 --2- 172.21.15.86:0/990700589 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f89cc06f430 0x7f89cc071940 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.841+0000 7f89f4c00640 1 --2- 172.21.15.86:0/990700589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89f00aeea0 0x7f89f015bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.841+0000 7f89f4c00640 1 -- 172.21.15.86:0/990700589 >> 172.21.15.86:0/990700589 conn(0x7f89f00d2230 msgr2=0x7f89f01625b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:16.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.841+0000 7f89f4c00640 1 -- 172.21.15.86:0/990700589 shutdown_connections 2024-04-25T14:04:16.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:16.841+0000 7f89f4c00640 1 -- 172.21.15.86:0/990700589 wait complete. 2024-04-25T14:04:16.863 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:03:49.151473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:03:49.802597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T14:03:53.492008+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T14:04:16.864 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:04:17.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.013+0000 7fcee1800640 1 Processor -- start 2024-04-25T14:04:17.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.013+0000 7fcee1800640 1 -- start start 2024-04-25T14:04:17.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.013+0000 7fcee1800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcedc0b4440 0x7fcedc163920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:17.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.013+0000 7fcee1800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcedc0d3e70 0x7fcedc163e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:17.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.013+0000 7fcee1800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcedc0b4a30 con 0x7fcedc0d1200 2024-04-25T14:04:17.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.013+0000 7fcee1800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fcedc0b4d10 con 0x7fcedc0d3e70 2024-04-25T14:04:17.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.013+0000 7fcee1800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcedc06e010 con 0x7fcedc0b4440 2024-04-25T14:04:17.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.013+0000 7fcedb400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcedc0d3e70 0x7fcedc163e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:17.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.013+0000 7fcedaa00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fcedc0d1200 0x7fcedc0d1dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:50938/0 (socket says 172.21.15.86:50938) 2024-04-25T14:04:17.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.013+0000 7fced2000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcedc0b4440 0x7fcedc163920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:17.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.013+0000 7fcedaa00640 1 -- 172.21.15.86:0/1052855633 learned_addr learned my addr 172.21.15.86:0/1052855633 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:17.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.013+0000 7fced2000640 1 -- 172.21.15.86:0/1052855633 >> v2:172.21.15.155:3300/0 conn(0x7fcedc0d3e70 msgr2=0x7fcedc163e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:17.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.013+0000 7fced2000640 1 --2- 172.21.15.86:0/1052855633 >> v2:172.21.15.155:3300/0 conn(0x7fcedc0d3e70 0x7fcedc163e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:17.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.013+0000 7fced2000640 1 -- 172.21.15.86:0/1052855633 >> v1:172.21.15.155:6789/0 conn(0x7fcedc0d1200 legacy=0x7fcedc0d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:17.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.013+0000 7fced2000640 1 -- 172.21.15.86:0/1052855633 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcedc06d9c0 con 0x7fcedc0b4440 2024-04-25T14:04:17.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.013+0000 7fcedb400640 1 --2- 172.21.15.86:0/1052855633 >> v2:172.21.15.155:3300/0 conn(0x7fcedc0d3e70 0x7fcedc163e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:17.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.017+0000 7fced2000640 1 --2- 172.21.15.86:0/1052855633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcedc0b4440 0x7fcedc163920 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcec40097c0 tx=0x7fcec401c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9af8da4c23dd07a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:17.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.017+0000 7fceda000640 1 -- 172.21.15.86:0/1052855633 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec40b0070 con 0x7fcedc0b4440 2024-04-25T14:04:17.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.017+0000 7fceda000640 1 -- 172.21.15.86:0/1052855633 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcec40ab600 con 0x7fcedc0b4440 2024-04-25T14:04:17.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.017+0000 7fceda000640 1 -- 172.21.15.86:0/1052855633 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec40ab9d0 con 0x7fcedc0b4440 2024-04-25T14:04:17.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.017+0000 7fcee1800640 1 -- 172.21.15.86:0/1052855633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcedc0b4440 msgr2=0x7fcedc163920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:17.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.017+0000 7fcee1800640 1 --2- 172.21.15.86:0/1052855633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcedc0b4440 0x7fcedc163920 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcec40097c0 tx=0x7fcec401c410 comp rx=0 tx=0).stop 2024-04-25T14:04:17.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.017+0000 7fcee1800640 1 -- 172.21.15.86:0/1052855633 shutdown_connections 2024-04-25T14:04:17.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.017+0000 7fcee1800640 1 --2- 172.21.15.86:0/1052855633 >> v2:172.21.15.155:3300/0 conn(0x7fcedc0d3e70 0x7fcedc163e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:17.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.017+0000 7fcee1800640 1 --2- 172.21.15.86:0/1052855633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcedc0b4440 0x7fcedc163920 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:17.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.017+0000 7fcee1800640 1 -- 172.21.15.86:0/1052855633 >> 172.21.15.86:0/1052855633 conn(0x7fcedc0b72d0 msgr2=0x7fcedc0b76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:17.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.017+0000 7fcee1800640 1 -- 172.21.15.86:0/1052855633 shutdown_connections 2024-04-25T14:04:17.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.017+0000 7fcee1800640 1 -- 172.21.15.86:0/1052855633 wait complete. 2024-04-25T14:04:17.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.021+0000 7fcee1800640 1 Processor -- start 2024-04-25T14:04:17.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.021+0000 7fcee1800640 1 -- start start 2024-04-25T14:04:17.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.021+0000 7fcee1800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcedc0b4440 0x7fcedc0d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:17.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.021+0000 7fcee1800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcedc0d1200 0x7fcedc14b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:17.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.021+0000 7fcee1800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcedc0d3e70 0x7fcedc0cb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:17.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.021+0000 7fcee1800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcedc16d650 con 0x7fcedc0d3e70 2024-04-25T14:04:17.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.021+0000 7fcee1800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcedc16d4d0 con 0x7fcedc0d1200 2024-04-25T14:04:17.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.021+0000 7fcee1800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcedc16d7d0 con 0x7fcedc0b4440 2024-04-25T14:04:17.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.021+0000 7fcedb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcedc0d3e70 0x7fcedc0cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:17.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.021+0000 7fcedaa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcedc0b4440 0x7fcedc0d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:17.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.021+0000 7fced2000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcedc0d1200 0x7fcedc14b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:17.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.021+0000 7fcedb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcedc0d3e70 0x7fcedc0cb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38318/0 (socket says 172.21.15.86:38318) 2024-04-25T14:04:17.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.025+0000 7fced2000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcedc0d1200 0x7fcedc14b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:54690/0 (socket says 172.21.15.86:54690) 2024-04-25T14:04:17.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.025+0000 7fcedb400640 1 -- 172.21.15.86:0/2141841690 learned_addr learned my addr 172.21.15.86:0/2141841690 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:17.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.025+0000 7fcedb400640 1 -- 172.21.15.86:0/2141841690 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcedc0b4440 msgr2=0x7fcedc0d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:17.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.025+0000 7fcedb400640 1 --2- 172.21.15.86:0/2141841690 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcedc0b4440 0x7fcedc0d1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:17.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.025+0000 7fcedb400640 1 -- 172.21.15.86:0/2141841690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcedc0d1200 msgr2=0x7fcedc14b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:17.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.025+0000 7fcedb400640 1 --2- 172.21.15.86:0/2141841690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcedc0d1200 0x7fcedc14b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:17.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.025+0000 7fcedb400640 1 -- 172.21.15.86:0/2141841690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcedc161720 con 0x7fcedc0d3e70 2024-04-25T14:04:17.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.025+0000 7fcedb400640 1 --2- 172.21.15.86:0/2141841690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcedc0d3e70 0x7fcedc0cb680 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fced400efc0 tx=0x7fced400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:17.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.025+0000 7fced3400640 1 -- 172.21.15.86:0/2141841690 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced4018020 con 0x7fcedc0d3e70 2024-04-25T14:04:17.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.025+0000 7fced3400640 1 -- 172.21.15.86:0/2141841690 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fced4014480 con 0x7fcedc0d3e70 2024-04-25T14:04:17.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.025+0000 7fcee1800640 1 -- 172.21.15.86:0/2141841690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcedc161a00 con 0x7fcedc0d3e70 2024-04-25T14:04:17.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.025+0000 7fced3400640 1 -- 172.21.15.86:0/2141841690 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced4014870 con 0x7fcedc0d3e70 2024-04-25T14:04:17.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.025+0000 7fcee1800640 1 -- 172.21.15.86:0/2141841690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcedc161fc0 con 0x7fcedc0d3e70 2024-04-25T14:04:17.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.025+0000 7fcee1800640 1 -- 172.21.15.86:0/2141841690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcedc161470 con 0x7fcedc0d3e70 2024-04-25T14:04:17.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.025+0000 7fced3400640 1 -- 172.21.15.86:0/2141841690 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fced401b020 con 0x7fcedc0d3e70 2024-04-25T14:04:17.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.025+0000 7fced3400640 1 --2- 172.21.15.86:0/2141841690 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fceb806f430 0x7fceb8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:17.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.029+0000 7fced3400640 1 -- 172.21.15.86:0/2141841690 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fced4090100 con 0x7fcedc0d3e70 2024-04-25T14:04:17.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.029+0000 7fcedaa00640 1 --2- 172.21.15.86:0/2141841690 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fceb806f430 0x7fceb8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:17.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.029+0000 7fcedaa00640 1 --2- 172.21.15.86:0/2141841690 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fceb806f430 0x7fceb8071940 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcecc000f70 tx=0x7fcecc007e90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:17.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.033+0000 7fced3400640 1 -- 172.21.15.86:0/2141841690 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fced4022030 con 0x7fcedc0d3e70 2024-04-25T14:04:17.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.185+0000 7fcee1800640 1 -- 172.21.15.86:0/2141841690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcedc06ef80 con 0x7fcedc0d3e70 2024-04-25T14:04:17.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.189+0000 7fced3400640 1 -- 172.21.15.86:0/2141841690 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fced4010040 con 0x7fcedc0d3e70 2024-04-25T14:04:17.189 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:04:17.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.189+0000 7fcee1800640 1 -- 172.21.15.86:0/2141841690 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fceb806f430 msgr2=0x7fceb8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:17.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.189+0000 7fcee1800640 1 --2- 172.21.15.86:0/2141841690 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fceb806f430 0x7fceb8071940 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcecc000f70 tx=0x7fcecc007e90 comp rx=0 tx=0).stop 2024-04-25T14:04:17.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.189+0000 7fcee1800640 1 -- 172.21.15.86:0/2141841690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcedc0d3e70 msgr2=0x7fcedc0cb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:17.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.189+0000 7fcee1800640 1 --2- 172.21.15.86:0/2141841690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcedc0d3e70 0x7fcedc0cb680 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fced400efc0 tx=0x7fced400c490 comp rx=0 tx=0).stop 2024-04-25T14:04:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.189+0000 7fcee1800640 1 -- 172.21.15.86:0/2141841690 shutdown_connections 2024-04-25T14:04:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.189+0000 7fcee1800640 1 --2- 172.21.15.86:0/2141841690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcedc0d3e70 0x7fcedc0cb680 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.189+0000 7fcee1800640 1 --2- 172.21.15.86:0/2141841690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcedc0d1200 0x7fcedc14b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.189+0000 7fcee1800640 1 --2- 172.21.15.86:0/2141841690 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fceb806f430 0x7fceb8071940 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.189+0000 7fcee1800640 1 --2- 172.21.15.86:0/2141841690 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcedc0b4440 0x7fcedc0d1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.189+0000 7fcee1800640 1 -- 172.21.15.86:0/2141841690 >> 172.21.15.86:0/2141841690 conn(0x7fcedc0b72d0 msgr2=0x7fcedc158ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.189+0000 7fcee1800640 1 -- 172.21.15.86:0/2141841690 shutdown_connections 2024-04-25T14:04:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.189+0000 7fcee1800640 1 -- 172.21.15.86:0/2141841690 wait complete. 2024-04-25T14:04:17.211 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:04:17.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.357+0000 7ffaf8a00640 1 Processor -- start 2024-04-25T14:04:17.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.357+0000 7ffaf8a00640 1 -- start start 2024-04-25T14:04:17.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.361+0000 7ffaf8a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffaf4160e90 0x7ffaf40d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:17.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.361+0000 7ffaf8a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ffaf4072630 0x7ffaf4072a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:17.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.361+0000 7ffaf8a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaf406f0f0 con 0x7ffaf4070e70 2024-04-25T14:04:17.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.361+0000 7ffaf8a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ffaf4072f80 con 0x7ffaf4160e90 2024-04-25T14:04:17.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.361+0000 7ffaf8a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ffaf4163a80 con 0x7ffaf4072630 2024-04-25T14:04:17.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.361+0000 7ffaf1e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffaf4160e90 0x7ffaf40d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:17.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.361+0000 7ffaf1e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffaf4160e90 0x7ffaf40d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38334/0 (socket says 172.21.15.86:38334) 2024-04-25T14:04:17.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.361+0000 7ffaf1e00640 1 -- 172.21.15.86:0/3279732609 learned_addr learned my addr 172.21.15.86:0/3279732609 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:17.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.361+0000 7ffae9400640 1 --2- 172.21.15.86:0/3279732609 >> v2:172.21.15.155:3300/0 conn(0x7ffaf4072630 0x7ffaf4072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:17.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.361+0000 7ffaf1e00640 1 -- 172.21.15.86:0/3279732609 >> v2:172.21.15.155:3300/0 conn(0x7ffaf4072630 msgr2=0x7ffaf4072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:17.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.361+0000 7ffaf1e00640 1 --2- 172.21.15.86:0/3279732609 >> v2:172.21.15.155:3300/0 conn(0x7ffaf4072630 0x7ffaf4072a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:17.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.361+0000 7ffaf1e00640 1 -- 172.21.15.86:0/3279732609 >> v1:172.21.15.86:6789/0 conn(0x7ffaf4070e70 legacy=0x7ffaf4071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:17.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.361+0000 7ffaf1e00640 1 -- 172.21.15.86:0/3279732609 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaf4163d60 con 0x7ffaf4160e90 2024-04-25T14:04:17.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.361+0000 7ffae9400640 1 --2- 172.21.15.86:0/3279732609 >> v2:172.21.15.155:3300/0 conn(0x7ffaf4072630 0x7ffaf4072a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:17.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.365+0000 7ffaf1e00640 1 --2- 172.21.15.86:0/3279732609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaf4160e90 0x7ffaf40d1b30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffaf40749d0 tx=0x7ffae401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1401f8a6cb0752 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:17.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.365+0000 7ffaf1400640 1 -- 172.21.15.86:0/3279732609 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae40b0070 con 0x7ffaf4160e90 2024-04-25T14:04:17.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.365+0000 7ffaf1400640 1 -- 172.21.15.86:0/3279732609 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffae40ab600 con 0x7ffaf4160e90 2024-04-25T14:04:17.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.365+0000 7ffaf1400640 1 -- 172.21.15.86:0/3279732609 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae40ab9d0 con 0x7ffaf4160e90 2024-04-25T14:04:17.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.365+0000 7ffaf8a00640 1 -- 172.21.15.86:0/3279732609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaf4160e90 msgr2=0x7ffaf40d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:17.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.365+0000 7ffaf8a00640 1 --2- 172.21.15.86:0/3279732609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaf4160e90 0x7ffaf40d1b30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffaf40749d0 tx=0x7ffae401c520 comp rx=0 tx=0).stop 2024-04-25T14:04:17.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.365+0000 7ffaf8a00640 1 -- 172.21.15.86:0/3279732609 shutdown_connections 2024-04-25T14:04:17.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.365+0000 7ffaf8a00640 1 --2- 172.21.15.86:0/3279732609 >> v2:172.21.15.155:3300/0 conn(0x7ffaf4072630 0x7ffaf4072a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:17.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.365+0000 7ffaf8a00640 1 --2- 172.21.15.86:0/3279732609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaf4160e90 0x7ffaf40d1b30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:17.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.365+0000 7ffaf8a00640 1 -- 172.21.15.86:0/3279732609 >> 172.21.15.86:0/3279732609 conn(0x7ffaf40d3570 msgr2=0x7ffaf40d3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:17.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.365+0000 7ffaf8a00640 1 -- 172.21.15.86:0/3279732609 shutdown_connections 2024-04-25T14:04:17.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.365+0000 7ffaf8a00640 1 -- 172.21.15.86:0/3279732609 wait complete. 2024-04-25T14:04:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.365+0000 7ffaf8a00640 1 Processor -- start 2024-04-25T14:04:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.365+0000 7ffaf8a00640 1 -- start start 2024-04-25T14:04:17.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.369+0000 7ffaf8a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffaf4070e70 0x7ffaf4159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:17.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.369+0000 7ffaf8a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffaf4072630 0x7ffaf4154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:17.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.369+0000 7ffaf8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaf4160e90 0x7ffaf406f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:17.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.369+0000 7ffaf8a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf40cdf90 con 0x7ffaf4160e90 2024-04-25T14:04:17.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.369+0000 7ffaf8a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf40cde10 con 0x7ffaf4070e70 2024-04-25T14:04:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.369+0000 7ffaf8a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ffaf416d420 con 0x7ffaf4072630 2024-04-25T14:04:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.369+0000 7ffaf2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaf4160e90 0x7ffaf406f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.369+0000 7ffaf2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaf4160e90 0x7ffaf406f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38348/0 (socket says 172.21.15.86:38348) 2024-04-25T14:04:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.369+0000 7ffaf2800640 1 -- 172.21.15.86:0/1009716679 learned_addr learned my addr 172.21.15.86:0/1009716679 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.369+0000 7ffae9400640 1 --2- 172.21.15.86:0/1009716679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffaf4072630 0x7ffaf4154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.369+0000 7ffaf1e00640 1 --2- 172.21.15.86:0/1009716679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffaf4070e70 0x7ffaf4159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.369+0000 7ffaf2800640 1 -- 172.21.15.86:0/1009716679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffaf4072630 msgr2=0x7ffaf4154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.369+0000 7ffaf2800640 1 --2- 172.21.15.86:0/1009716679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffaf4072630 0x7ffaf4154da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.369+0000 7ffaf2800640 1 -- 172.21.15.86:0/1009716679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffaf4070e70 msgr2=0x7ffaf4159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.369+0000 7ffaf2800640 1 --2- 172.21.15.86:0/1009716679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffaf4070e70 0x7ffaf4159490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.369+0000 7ffaf2800640 1 -- 172.21.15.86:0/1009716679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaf41552e0 con 0x7ffaf4160e90 2024-04-25T14:04:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.369+0000 7ffaf2800640 1 --2- 172.21.15.86:0/1009716679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaf4160e90 0x7ffaf406f990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffaec00c760 tx=0x7ffaec00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.369+0000 7ffaeaa00640 1 -- 172.21.15.86:0/1009716679 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec003b80 con 0x7ffaf4160e90 2024-04-25T14:04:17.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.369+0000 7ffaeaa00640 1 -- 172.21.15.86:0/1009716679 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaec003d20 con 0x7ffaf4160e90 2024-04-25T14:04:17.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.369+0000 7ffaeaa00640 1 -- 172.21.15.86:0/1009716679 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec00aa90 con 0x7ffaf4160e90 2024-04-25T14:04:17.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.369+0000 7ffaf8a00640 1 -- 172.21.15.86:0/1009716679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaf40b61b0 con 0x7ffaf4160e90 2024-04-25T14:04:17.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.369+0000 7ffaf8a00640 1 -- 172.21.15.86:0/1009716679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaf40b6770 con 0x7ffaf4160e90 2024-04-25T14:04:17.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.369+0000 7ffaf8a00640 1 -- 172.21.15.86:0/1009716679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaf4159a30 con 0x7ffaf4160e90 2024-04-25T14:04:17.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.373+0000 7ffaeaa00640 1 -- 172.21.15.86:0/1009716679 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ffaec00ac30 con 0x7ffaf4160e90 2024-04-25T14:04:17.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.373+0000 7ffaeaa00640 1 --2- 172.21.15.86:0/1009716679 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ffad006f430 0x7ffad0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:17.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.373+0000 7ffaeaa00640 1 -- 172.21.15.86:0/1009716679 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ffaec08f320 con 0x7ffaf4160e90 2024-04-25T14:04:17.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.373+0000 7ffaf1e00640 1 --2- 172.21.15.86:0/1009716679 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ffad006f430 0x7ffad0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:17.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.373+0000 7ffaf1e00640 1 --2- 172.21.15.86:0/1009716679 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ffad006f430 0x7ffad0071940 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffae40ad000 tx=0x7ffae40a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:17.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.377+0000 7ffaeaa00640 1 -- 172.21.15.86:0/1009716679 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffaec05b880 con 0x7ffaf4160e90 2024-04-25T14:04:17.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.533+0000 7ffaf8a00640 1 -- 172.21.15.86:0/1009716679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ffaf4072170 con 0x7ffaf4160e90 2024-04-25T14:04:17.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.533+0000 7ffaeaa00640 1 -- 172.21.15.86:0/1009716679 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7ffaec05fb80 con 0x7ffaf4160e90 2024-04-25T14:04:17.535 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:04:17.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.533+0000 7ffaf8a00640 1 -- 172.21.15.86:0/1009716679 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ffad006f430 msgr2=0x7ffad0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:17.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.533+0000 7ffaf8a00640 1 --2- 172.21.15.86:0/1009716679 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ffad006f430 0x7ffad0071940 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffae40ad000 tx=0x7ffae40a6000 comp rx=0 tx=0).stop 2024-04-25T14:04:17.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.533+0000 7ffaf8a00640 1 -- 172.21.15.86:0/1009716679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaf4160e90 msgr2=0x7ffaf406f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:17.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.533+0000 7ffaf8a00640 1 --2- 172.21.15.86:0/1009716679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaf4160e90 0x7ffaf406f990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffaec00c760 tx=0x7ffaec00cc20 comp rx=0 tx=0).stop 2024-04-25T14:04:17.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.533+0000 7ffaf8a00640 1 -- 172.21.15.86:0/1009716679 shutdown_connections 2024-04-25T14:04:17.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.533+0000 7ffaf8a00640 1 --2- 172.21.15.86:0/1009716679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaf4160e90 0x7ffaf406f990 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:17.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.533+0000 7ffaf8a00640 1 --2- 172.21.15.86:0/1009716679 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ffad006f430 0x7ffad0071940 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:17.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.533+0000 7ffaf8a00640 1 --2- 172.21.15.86:0/1009716679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffaf4072630 0x7ffaf4154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:17.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.533+0000 7ffaf8a00640 1 --2- 172.21.15.86:0/1009716679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffaf4070e70 0x7ffaf4159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:17.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.537+0000 7ffaf8a00640 1 -- 172.21.15.86:0/1009716679 >> 172.21.15.86:0/1009716679 conn(0x7ffaf40d3570 msgr2=0x7ffaf4071280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:17.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.537+0000 7ffaf8a00640 1 -- 172.21.15.86:0/1009716679 shutdown_connections 2024-04-25T14:04:17.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.537+0000 7ffaf8a00640 1 -- 172.21.15.86:0/1009716679 wait complete. 2024-04-25T14:04:17.557 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:04:17.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.705+0000 7f8b45000640 1 Processor -- start 2024-04-25T14:04:17.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.705+0000 7f8b45000640 1 -- start start 2024-04-25T14:04:17.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.705+0000 7f8b45000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8b400ce360 0x7f8b40162090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:17.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.705+0000 7f8b45000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b400d2380 0x7f8b4006f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:17.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.705+0000 7f8b45000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b401625d0 con 0x7f8b400d2790 2024-04-25T14:04:17.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.705+0000 7f8b45000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8b400d15c0 con 0x7f8b400ce360 2024-04-25T14:04:17.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.705+0000 7f8b45000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8b400d18a0 con 0x7f8b400d2380 2024-04-25T14:04:17.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.705+0000 7f8b3f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8b400ce360 0x7f8b40162090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:17.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.705+0000 7f8b3f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8b400ce360 0x7f8b40162090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38358/0 (socket says 172.21.15.86:38358) 2024-04-25T14:04:17.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.705+0000 7f8b3f400640 1 -- 172.21.15.86:0/244444511 learned_addr learned my addr 172.21.15.86:0/244444511 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:17.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.705+0000 7f8b3fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8b400d2790 0x7f8b400b63f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33528/0 (socket says 172.21.15.86:33528) 2024-04-25T14:04:17.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.705+0000 7f8b3f400640 1 -- 172.21.15.86:0/244444511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b400d2380 msgr2=0x7f8b4006f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:17.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.705+0000 7f8b3f400640 1 --2- 172.21.15.86:0/244444511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b400d2380 0x7f8b4006f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:17.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.705+0000 7f8b3f400640 1 -- 172.21.15.86:0/244444511 >> v1:172.21.15.86:6789/0 conn(0x7f8b400d2790 legacy=0x7f8b400b63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:17.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.705+0000 7f8b3f400640 1 -- 172.21.15.86:0/244444511 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b400d1b80 con 0x7f8b400ce360 2024-04-25T14:04:17.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.705+0000 7f8b3e000640 1 -- 172.21.15.86:0/244444511 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2128666940 0 0) 0x7f8b401625d0 con 0x7f8b400d2790 2024-04-25T14:04:17.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.709+0000 7f8b3f400640 1 --2- 172.21.15.86:0/244444511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b400ce360 0x7f8b40162090 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8b30009870 tx=0x7f8b3001ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a1953bdc153a52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:17.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.709+0000 7f8b3e000640 1 -- 172.21.15.86:0/244444511 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b300b5070 con 0x7f8b400ce360 2024-04-25T14:04:17.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.709+0000 7f8b3e000640 1 -- 172.21.15.86:0/244444511 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b300a8a50 con 0x7f8b400ce360 2024-04-25T14:04:17.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.709+0000 7f8b3e000640 1 -- 172.21.15.86:0/244444511 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b300a8e20 con 0x7f8b400ce360 2024-04-25T14:04:17.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.709+0000 7f8b45000640 1 -- 172.21.15.86:0/244444511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b400ce360 msgr2=0x7f8b40162090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:17.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.709+0000 7f8b45000640 1 --2- 172.21.15.86:0/244444511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b400ce360 0x7f8b40162090 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8b30009870 tx=0x7f8b3001ed50 comp rx=0 tx=0).stop 2024-04-25T14:04:17.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.709+0000 7f8b45000640 1 -- 172.21.15.86:0/244444511 shutdown_connections 2024-04-25T14:04:17.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.709+0000 7f8b45000640 1 --2- 172.21.15.86:0/244444511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b400d2380 0x7f8b4006f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:17.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.709+0000 7f8b45000640 1 --2- 172.21.15.86:0/244444511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b400ce360 0x7f8b40162090 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:17.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.709+0000 7f8b45000640 1 -- 172.21.15.86:0/244444511 >> 172.21.15.86:0/244444511 conn(0x7f8b400cfea0 msgr2=0x7f8b400d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:17.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.709+0000 7f8b45000640 1 -- 172.21.15.86:0/244444511 shutdown_connections 2024-04-25T14:04:17.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.709+0000 7f8b45000640 1 -- 172.21.15.86:0/244444511 wait complete. 2024-04-25T14:04:17.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.713+0000 7f8b45000640 1 Processor -- start 2024-04-25T14:04:17.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.713+0000 7f8b45000640 1 -- start start 2024-04-25T14:04:17.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.713+0000 7f8b45000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b400ce360 0x7f8b4014f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.713+0000 7f8b45000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b400d2380 0x7f8b400c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.713+0000 7f8b45000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b400d2790 0x7f8b400b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.713+0000 7f8b45000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4006fed0 con 0x7f8b400d2790 2024-04-25T14:04:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.713+0000 7f8b45000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8b400ce960 con 0x7f8b400ce360 2024-04-25T14:04:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.713+0000 7f8b45000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8b40070050 con 0x7f8b400d2380 2024-04-25T14:04:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.713+0000 7f8b3fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b400d2790 0x7f8b400b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.713+0000 7f8b3fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b400d2790 0x7f8b400b76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38372/0 (socket says 172.21.15.86:38372) 2024-04-25T14:04:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.713+0000 7f8b3fe00640 1 -- 172.21.15.86:0/341197596 learned_addr learned my addr 172.21.15.86:0/341197596 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.713+0000 7f8b3f400640 1 --2- 172.21.15.86:0/341197596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b400ce360 0x7f8b4014f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.713+0000 7f8b3ea00640 1 --2- 172.21.15.86:0/341197596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b400d2380 0x7f8b400c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.713+0000 7f8b3fe00640 1 -- 172.21.15.86:0/341197596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b400d2380 msgr2=0x7f8b400c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.713+0000 7f8b3fe00640 1 --2- 172.21.15.86:0/341197596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b400d2380 0x7f8b400c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.713+0000 7f8b3fe00640 1 -- 172.21.15.86:0/341197596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b400ce360 msgr2=0x7f8b4014f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.713+0000 7f8b3fe00640 1 --2- 172.21.15.86:0/341197596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b400ce360 0x7f8b4014f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.713+0000 7f8b3fe00640 1 -- 172.21.15.86:0/341197596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b401f4c30 con 0x7f8b400d2790 2024-04-25T14:04:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.713+0000 7f8b3ea00640 1 --2- 172.21.15.86:0/341197596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b400d2380 0x7f8b400c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.713+0000 7f8b3fe00640 1 --2- 172.21.15.86:0/341197596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b400d2790 0x7f8b400b76b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8b3800c730 tx=0x7f8b3800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.713+0000 7f8b37400640 1 -- 172.21.15.86:0/341197596 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b38018070 con 0x7f8b400d2790 2024-04-25T14:04:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.713+0000 7f8b37400640 1 -- 172.21.15.86:0/341197596 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b3800ce40 con 0x7f8b400d2790 2024-04-25T14:04:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.713+0000 7f8b37400640 1 -- 172.21.15.86:0/341197596 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3800a860 con 0x7f8b400d2790 2024-04-25T14:04:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.717+0000 7f8b45000640 1 -- 172.21.15.86:0/341197596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b401f4f10 con 0x7f8b400d2790 2024-04-25T14:04:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.717+0000 7f8b45000640 1 -- 172.21.15.86:0/341197596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b401f54d0 con 0x7f8b400d2790 2024-04-25T14:04:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.717+0000 7f8b45000640 1 -- 172.21.15.86:0/341197596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b400cda60 con 0x7f8b400d2790 2024-04-25T14:04:17.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.717+0000 7f8b37400640 1 -- 172.21.15.86:0/341197596 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8b3800aa00 con 0x7f8b400d2790 2024-04-25T14:04:17.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.717+0000 7f8b37400640 1 --2- 172.21.15.86:0/341197596 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8b2006f400 0x7f8b20071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:17.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.717+0000 7f8b37400640 1 -- 172.21.15.86:0/341197596 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f8b3808ebe0 con 0x7f8b400d2790 2024-04-25T14:04:17.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.717+0000 7f8b3f400640 1 --2- 172.21.15.86:0/341197596 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8b2006f400 0x7f8b20071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:17.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.721+0000 7f8b3f400640 1 --2- 172.21.15.86:0/341197596 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8b2006f400 0x7f8b20071910 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8b300b2000 tx=0x7f8b30008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.721+0000 7f8b37400640 1 -- 172.21.15.86:0/341197596 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8b3805b130 con 0x7f8b400d2790 2024-04-25T14:04:17.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.881+0000 7f8b45000640 1 -- 172.21.15.86:0/341197596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8b40076b80 con 0x7f8b400d2790 2024-04-25T14:04:17.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.881+0000 7f8b37400640 1 -- 172.21.15.86:0/341197596 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f8b3805f430 con 0x7f8b400d2790 2024-04-25T14:04:17.884 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:04:17.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.885+0000 7f8b45000640 1 -- 172.21.15.86:0/341197596 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8b2006f400 msgr2=0x7f8b20071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:17.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.885+0000 7f8b45000640 1 --2- 172.21.15.86:0/341197596 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8b2006f400 0x7f8b20071910 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8b300b2000 tx=0x7f8b30008040 comp rx=0 tx=0).stop 2024-04-25T14:04:17.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.885+0000 7f8b45000640 1 -- 172.21.15.86:0/341197596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b400d2790 msgr2=0x7f8b400b76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:17.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.885+0000 7f8b45000640 1 --2- 172.21.15.86:0/341197596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b400d2790 0x7f8b400b76b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8b3800c730 tx=0x7f8b3800cbf0 comp rx=0 tx=0).stop 2024-04-25T14:04:17.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.885+0000 7f8b45000640 1 -- 172.21.15.86:0/341197596 shutdown_connections 2024-04-25T14:04:17.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.885+0000 7f8b45000640 1 --2- 172.21.15.86:0/341197596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b400d2790 0x7f8b400b76b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:17.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.885+0000 7f8b45000640 1 --2- 172.21.15.86:0/341197596 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8b2006f400 0x7f8b20071910 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:17.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.885+0000 7f8b45000640 1 --2- 172.21.15.86:0/341197596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b400d2380 0x7f8b400c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:17.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.885+0000 7f8b45000640 1 --2- 172.21.15.86:0/341197596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b400ce360 0x7f8b4014f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:17.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.885+0000 7f8b45000640 1 -- 172.21.15.86:0/341197596 >> 172.21.15.86:0/341197596 conn(0x7f8b400cfea0 msgr2=0x7f8b4015c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:17.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.885+0000 7f8b45000640 1 -- 172.21.15.86:0/341197596 shutdown_connections 2024-04-25T14:04:17.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:17.885+0000 7f8b45000640 1 -- 172.21.15.86:0/341197596 wait complete. 2024-04-25T14:04:17.906 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T14:04:18.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.053+0000 7f231b600640 1 Processor -- start 2024-04-25T14:04:18.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.053+0000 7f231b600640 1 -- start start 2024-04-25T14:04:18.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.057+0000 7f231b600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2314160e90 0x7f23140d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:18.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.057+0000 7f231b600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2314072630 0x7f2314072a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:18.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.057+0000 7f231b600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f231406f0f0 con 0x7f2314070e70 2024-04-25T14:04:18.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.057+0000 7f231b600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2314072f80 con 0x7f2314160e90 2024-04-25T14:04:18.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.057+0000 7f231b600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f2314163a80 con 0x7f2314072630 2024-04-25T14:04:18.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.057+0000 7f2318c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2314160e90 0x7f23140d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:18.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.057+0000 7f2318c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2314160e90 0x7f23140d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38374/0 (socket says 172.21.15.86:38374) 2024-04-25T14:04:18.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.057+0000 7f2318c00640 1 -- 172.21.15.86:0/1754232882 learned_addr learned my addr 172.21.15.86:0/1754232882 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:18.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.057+0000 7f2319600640 1 --1- 172.21.15.86:0/1754232882 >> v1:172.21.15.86:6789/0 conn(0x7f2314070e70 0x7f2314071280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33532/0 (socket says 172.21.15.86:33532) 2024-04-25T14:04:18.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.057+0000 7f230be00640 1 --2- 172.21.15.86:0/1754232882 >> v2:172.21.15.155:3300/0 conn(0x7f2314072630 0x7f2314072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:18.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.057+0000 7f2318c00640 1 -- 172.21.15.86:0/1754232882 >> v2:172.21.15.155:3300/0 conn(0x7f2314072630 msgr2=0x7f2314072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:18.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.057+0000 7f2318c00640 1 --2- 172.21.15.86:0/1754232882 >> v2:172.21.15.155:3300/0 conn(0x7f2314072630 0x7f2314072a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.057+0000 7f2318c00640 1 -- 172.21.15.86:0/1754232882 >> v1:172.21.15.86:6789/0 conn(0x7f2314070e70 legacy=0x7f2314071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:18.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.057+0000 7f2318c00640 1 -- 172.21.15.86:0/1754232882 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2314163d60 con 0x7f2314160e90 2024-04-25T14:04:18.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.057+0000 7f2313e00640 1 -- 172.21.15.86:0/1754232882 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3632402868 0 0) 0x7f231406f0f0 con 0x7f2314070e70 2024-04-25T14:04:18.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.061+0000 7f2318c00640 1 --2- 172.21.15.86:0/1754232882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2314160e90 0x7f23140d1b30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f23140749d0 tx=0x7f230401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d647ace29e3ed77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:18.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.061+0000 7f2313e00640 1 -- 172.21.15.86:0/1754232882 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23040b0070 con 0x7f2314160e90 2024-04-25T14:04:18.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.061+0000 7f2313e00640 1 -- 172.21.15.86:0/1754232882 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23040ab600 con 0x7f2314160e90 2024-04-25T14:04:18.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.061+0000 7f2313e00640 1 -- 172.21.15.86:0/1754232882 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23040ab9d0 con 0x7f2314160e90 2024-04-25T14:04:18.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.061+0000 7f231b600640 1 -- 172.21.15.86:0/1754232882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2314160e90 msgr2=0x7f23140d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:18.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.061+0000 7f231b600640 1 --2- 172.21.15.86:0/1754232882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2314160e90 0x7f23140d1b30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f23140749d0 tx=0x7f230401c520 comp rx=0 tx=0).stop 2024-04-25T14:04:18.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.061+0000 7f231b600640 1 -- 172.21.15.86:0/1754232882 shutdown_connections 2024-04-25T14:04:18.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.061+0000 7f231b600640 1 --2- 172.21.15.86:0/1754232882 >> v2:172.21.15.155:3300/0 conn(0x7f2314072630 0x7f2314072a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.061+0000 7f231b600640 1 --2- 172.21.15.86:0/1754232882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2314160e90 0x7f23140d1b30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.061+0000 7f231b600640 1 -- 172.21.15.86:0/1754232882 >> 172.21.15.86:0/1754232882 conn(0x7f23140d3570 msgr2=0x7f23140d3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:18.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.061+0000 7f231b600640 1 -- 172.21.15.86:0/1754232882 shutdown_connections 2024-04-25T14:04:18.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.061+0000 7f231b600640 1 -- 172.21.15.86:0/1754232882 wait complete. 2024-04-25T14:04:18.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.061+0000 7f231b600640 1 Processor -- start 2024-04-25T14:04:18.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.061+0000 7f231b600640 1 -- start start 2024-04-25T14:04:18.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.065+0000 7f231b600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2314070e70 0x7f2314155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:18.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.065+0000 7f231b600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2314072630 0x7f23140af210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:18.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.065+0000 7f231b600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2314160e90 0x7f231406e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:18.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.065+0000 7f231b600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f23140cdf90 con 0x7f2314160e90 2024-04-25T14:04:18.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.065+0000 7f231b600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f23140cde10 con 0x7f2314072630 2024-04-25T14:04:18.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.065+0000 7f231b600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f231416d420 con 0x7f2314070e70 2024-04-25T14:04:18.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.065+0000 7f2319600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2314160e90 0x7f231406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:18.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.065+0000 7f2318c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2314070e70 0x7f2314155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:18.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.065+0000 7f2319600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2314160e90 0x7f231406e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38380/0 (socket says 172.21.15.86:38380) 2024-04-25T14:04:18.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.065+0000 7f2319600640 1 -- 172.21.15.86:0/3596287519 learned_addr learned my addr 172.21.15.86:0/3596287519 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:18.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.065+0000 7f230be00640 1 --2- 172.21.15.86:0/3596287519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2314072630 0x7f23140af210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:18.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.065+0000 7f2319600640 1 -- 172.21.15.86:0/3596287519 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2314070e70 msgr2=0x7f2314155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:18.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.065+0000 7f2319600640 1 --2- 172.21.15.86:0/3596287519 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2314070e70 0x7f2314155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.065+0000 7f2319600640 1 -- 172.21.15.86:0/3596287519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2314072630 msgr2=0x7f23140af210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:18.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.065+0000 7f2319600640 1 --2- 172.21.15.86:0/3596287519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2314072630 0x7f23140af210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.065+0000 7f2319600640 1 -- 172.21.15.86:0/3596287519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2314155ca0 con 0x7f2314160e90 2024-04-25T14:04:18.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.065+0000 7f2319600640 1 --2- 172.21.15.86:0/3596287519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2314160e90 0x7f231406e330 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f230c00c700 tx=0x7f230c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:18.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.065+0000 7f2311600640 1 -- 172.21.15.86:0/3596287519 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c018070 con 0x7f2314160e90 2024-04-25T14:04:18.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.065+0000 7f2311600640 1 -- 172.21.15.86:0/3596287519 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f230c00ce10 con 0x7f2314160e90 2024-04-25T14:04:18.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.065+0000 7f231b600640 1 -- 172.21.15.86:0/3596287519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23140b1690 con 0x7f2314160e90 2024-04-25T14:04:18.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.065+0000 7f2311600640 1 -- 172.21.15.86:0/3596287519 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c00a880 con 0x7f2314160e90 2024-04-25T14:04:18.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.065+0000 7f231b600640 1 -- 172.21.15.86:0/3596287519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23140b1c50 con 0x7f2314160e90 2024-04-25T14:04:18.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.065+0000 7f231b600640 1 -- 172.21.15.86:0/3596287519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23140b1400 con 0x7f2314160e90 2024-04-25T14:04:18.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.069+0000 7f2311600640 1 -- 172.21.15.86:0/3596287519 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f230c01b070 con 0x7f2314160e90 2024-04-25T14:04:18.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.069+0000 7f2311600640 1 --2- 172.21.15.86:0/3596287519 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f22f006f400 0x7f22f0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:18.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.069+0000 7f2311600640 1 -- 172.21.15.86:0/3596287519 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f230c08eaf0 con 0x7f2314160e90 2024-04-25T14:04:18.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.069+0000 7f2318c00640 1 --2- 172.21.15.86:0/3596287519 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f22f006f400 0x7f22f0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:18.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.069+0000 7f2318c00640 1 --2- 172.21.15.86:0/3596287519 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f22f006f400 0x7f22f0071910 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f23040ad000 tx=0x7f23040a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:18.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.073+0000 7f2311600640 1 -- 172.21.15.86:0/3596287519 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f230c05b040 con 0x7f2314160e90 2024-04-25T14:04:18.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.229+0000 7f231b600640 1 -- 172.21.15.86:0/3596287519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2314072170 con 0x7f2314160e90 2024-04-25T14:04:18.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.229+0000 7f2311600640 1 -- 172.21.15.86:0/3596287519 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f230c05f340 con 0x7f2314160e90 2024-04-25T14:04:18.230 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:04:18.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.229+0000 7f231b600640 1 -- 172.21.15.86:0/3596287519 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f22f006f400 msgr2=0x7f22f0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:18.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.229+0000 7f231b600640 1 --2- 172.21.15.86:0/3596287519 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f22f006f400 0x7f22f0071910 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f23040ad000 tx=0x7f23040a6000 comp rx=0 tx=0).stop 2024-04-25T14:04:18.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.229+0000 7f231b600640 1 -- 172.21.15.86:0/3596287519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2314160e90 msgr2=0x7f231406e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:18.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.229+0000 7f231b600640 1 --2- 172.21.15.86:0/3596287519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2314160e90 0x7f231406e330 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f230c00c700 tx=0x7f230c00cbc0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.229+0000 7f231b600640 1 -- 172.21.15.86:0/3596287519 shutdown_connections 2024-04-25T14:04:18.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.229+0000 7f231b600640 1 --2- 172.21.15.86:0/3596287519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2314160e90 0x7f231406e330 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.229+0000 7f231b600640 1 --2- 172.21.15.86:0/3596287519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2314072630 0x7f23140af210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.229+0000 7f231b600640 1 --2- 172.21.15.86:0/3596287519 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f22f006f400 0x7f22f0071910 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.229+0000 7f231b600640 1 --2- 172.21.15.86:0/3596287519 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2314070e70 0x7f2314155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.229+0000 7f231b600640 1 -- 172.21.15.86:0/3596287519 >> 172.21.15.86:0/3596287519 conn(0x7f23140d3570 msgr2=0x7f23140b2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:18.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.229+0000 7f231b600640 1 -- 172.21.15.86:0/3596287519 shutdown_connections 2024-04-25T14:04:18.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.229+0000 7f231b600640 1 -- 172.21.15.86:0/3596287519 wait complete. 2024-04-25T14:04:18.252 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:04:18.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.397+0000 7f8eb9a00640 1 Processor -- start 2024-04-25T14:04:18.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.397+0000 7f8eb9a00640 1 -- start start 2024-04-25T14:04:18.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.397+0000 7f8eb9a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8eb40d2c80 0x7f8eb415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:18.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.397+0000 7f8eb9a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8eb4070e20 0x7f8eb4071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:18.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.397+0000 7f8eb9a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8eb4071770 con 0x7f8eb40722f0 2024-04-25T14:04:18.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.397+0000 7f8eb9a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f8eb40ce720 con 0x7f8eb4070e20 2024-04-25T14:04:18.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.397+0000 7f8eb9a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8eb40cd8d0 con 0x7f8eb40d2c80 2024-04-25T14:04:18.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.397+0000 7f8eb2000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8eb4070e20 0x7f8eb4071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:18.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.397+0000 7f8eb2a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8eb40d2c80 0x7f8eb415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:18.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.397+0000 7f8eb2000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8eb4070e20 0x7f8eb4071230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:54774/0 (socket says 172.21.15.86:54774) 2024-04-25T14:04:18.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.397+0000 7f8eb2000640 1 -- 172.21.15.86:0/2564541508 learned_addr learned my addr 172.21.15.86:0/2564541508 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:18.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.401+0000 7f8eb2000640 1 -- 172.21.15.86:0/2564541508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8eb40d2c80 msgr2=0x7f8eb415dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:18.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.401+0000 7f8eb2000640 1 --2- 172.21.15.86:0/2564541508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8eb40d2c80 0x7f8eb415dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.401+0000 7f8eb2000640 1 -- 172.21.15.86:0/2564541508 >> v1:172.21.15.155:6789/0 conn(0x7f8eb40722f0 legacy=0x7f8eb4072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:18.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.401+0000 7f8eb2000640 1 -- 172.21.15.86:0/2564541508 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb40ce970 con 0x7f8eb4070e20 2024-04-25T14:04:18.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.401+0000 7f8eb2000640 1 --2- 172.21.15.86:0/2564541508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8eb4070e20 0x7f8eb4071230 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0006fd0 tx=0x7f8ea001ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e029409fe814feb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:18.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.401+0000 7f8eb1600640 1 -- 172.21.15.86:0/2564541508 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea00b5070 con 0x7f8eb4070e20 2024-04-25T14:04:18.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.401+0000 7f8eb1600640 1 -- 172.21.15.86:0/2564541508 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea00a8a30 con 0x7f8eb4070e20 2024-04-25T14:04:18.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.401+0000 7f8eb1600640 1 -- 172.21.15.86:0/2564541508 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea00a8e00 con 0x7f8eb4070e20 2024-04-25T14:04:18.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.405+0000 7f8eb9a00640 1 -- 172.21.15.86:0/2564541508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8eb4070e20 msgr2=0x7f8eb4071230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:18.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.405+0000 7f8eb9a00640 1 --2- 172.21.15.86:0/2564541508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8eb4070e20 0x7f8eb4071230 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0006fd0 tx=0x7f8ea001ed30 comp rx=0 tx=0).stop 2024-04-25T14:04:18.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.405+0000 7f8eb9a00640 1 -- 172.21.15.86:0/2564541508 shutdown_connections 2024-04-25T14:04:18.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.405+0000 7f8eb9a00640 1 --2- 172.21.15.86:0/2564541508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8eb4070e20 0x7f8eb4071230 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.405+0000 7f8eb9a00640 1 --2- 172.21.15.86:0/2564541508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8eb40d2c80 0x7f8eb415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.405+0000 7f8eb9a00640 1 -- 172.21.15.86:0/2564541508 >> 172.21.15.86:0/2564541508 conn(0x7f8eb40cffc0 msgr2=0x7f8eb40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:18.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.405+0000 7f8eb9a00640 1 -- 172.21.15.86:0/2564541508 shutdown_connections 2024-04-25T14:04:18.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.405+0000 7f8eb9a00640 1 -- 172.21.15.86:0/2564541508 wait complete. 2024-04-25T14:04:18.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.405+0000 7f8eb9a00640 1 Processor -- start 2024-04-25T14:04:18.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.405+0000 7f8eb9a00640 1 -- start start 2024-04-25T14:04:18.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.409+0000 7f8eb9a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8eb4070e20 0x7f8eb41504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:18.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.409+0000 7f8eb9a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8eb40722f0 0x7f8eb40c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:18.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.409+0000 7f8eb9a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8eb40d2c80 0x7f8eb40c7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:18.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.409+0000 7f8eb9a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb40d21a0 con 0x7f8eb40722f0 2024-04-25T14:04:18.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.409+0000 7f8eb9a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb40d1ec0 con 0x7f8eb40d2c80 2024-04-25T14:04:18.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.409+0000 7f8eb9a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8eb40d2480 con 0x7f8eb4070e20 2024-04-25T14:04:18.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.409+0000 7f8eb2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8eb40722f0 0x7f8eb40c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:18.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.409+0000 7f8eb2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8eb40722f0 0x7f8eb40c6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38388/0 (socket says 172.21.15.86:38388) 2024-04-25T14:04:18.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.409+0000 7f8eb2000640 1 -- 172.21.15.86:0/4131040731 learned_addr learned my addr 172.21.15.86:0/4131040731 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:18.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.409+0000 7f8eb2a00640 1 --2- 172.21.15.86:0/4131040731 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8eb4070e20 0x7f8eb41504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:18.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.409+0000 7f8eb3400640 1 --2- 172.21.15.86:0/4131040731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8eb40d2c80 0x7f8eb40c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:18.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.409+0000 7f8eb2000640 1 -- 172.21.15.86:0/4131040731 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8eb4070e20 msgr2=0x7f8eb41504c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:18.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.409+0000 7f8eb2000640 1 --2- 172.21.15.86:0/4131040731 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8eb4070e20 0x7f8eb41504c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.409+0000 7f8eb2000640 1 -- 172.21.15.86:0/4131040731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8eb40d2c80 msgr2=0x7f8eb40c7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:18.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.409+0000 7f8eb2000640 1 --2- 172.21.15.86:0/4131040731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8eb40d2c80 0x7f8eb40c7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.409+0000 7f8eb2000640 1 -- 172.21.15.86:0/4131040731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb40c7980 con 0x7f8eb40722f0 2024-04-25T14:04:18.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.409+0000 7f8eb3400640 1 --2- 172.21.15.86:0/4131040731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8eb40d2c80 0x7f8eb40c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:18.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.409+0000 7f8eb2000640 1 --2- 172.21.15.86:0/4131040731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8eb40722f0 0x7f8eb40c6c30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0009930 tx=0x7f8ea001bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:18.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.409+0000 7f8eaaa00640 1 -- 172.21.15.86:0/4131040731 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea00b5070 con 0x7f8eb40722f0 2024-04-25T14:04:18.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.409+0000 7f8eaaa00640 1 -- 172.21.15.86:0/4131040731 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea00059a0 con 0x7f8eb40722f0 2024-04-25T14:04:18.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.409+0000 7f8eb9a00640 1 -- 172.21.15.86:0/4131040731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb40c83d0 con 0x7f8eb40722f0 2024-04-25T14:04:18.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.409+0000 7f8eaaa00640 1 -- 172.21.15.86:0/4131040731 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0005d70 con 0x7f8eb40722f0 2024-04-25T14:04:18.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.409+0000 7f8eb9a00640 1 -- 172.21.15.86:0/4131040731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb40c9f30 con 0x7f8eb40722f0 2024-04-25T14:04:18.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.409+0000 7f8eb9a00640 1 -- 172.21.15.86:0/4131040731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eb40720e0 con 0x7f8eb40722f0 2024-04-25T14:04:18.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.409+0000 7f8eaaa00640 1 -- 172.21.15.86:0/4131040731 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8ea0003620 con 0x7f8eb40722f0 2024-04-25T14:04:18.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.413+0000 7f8eaaa00640 1 --2- 172.21.15.86:0/4131040731 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8e9406f400 0x7f8e94071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:18.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.413+0000 7f8eaaa00640 1 -- 172.21.15.86:0/4131040731 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f8ea012c300 con 0x7f8eb40722f0 2024-04-25T14:04:18.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.413+0000 7f8eb2a00640 1 --2- 172.21.15.86:0/4131040731 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8e9406f400 0x7f8e94071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:18.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.413+0000 7f8eb2a00640 1 --2- 172.21.15.86:0/4131040731 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8e9406f400 0x7f8e94071910 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4011340 tx=0x7f8ea400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:18.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.417+0000 7f8eaaa00640 1 -- 172.21.15.86:0/4131040731 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8ea00bf060 con 0x7f8eb40722f0 2024-04-25T14:04:18.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.605+0000 7f8eb9a00640 1 -- 172.21.15.86:0/4131040731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8eb4076bf0 con 0x7f8eb40722f0 2024-04-25T14:04:18.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.605+0000 7f8eaaa00640 1 -- 172.21.15.86:0/4131040731 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5177 (secure 0 0 0) 0x7f8ea0018890 con 0x7f8eb40722f0 2024-04-25T14:04:18.608 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:18.608 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/347107112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":347107112},{"type":"v1","addr":"172.21.15.155:6835","nonce":347107112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2236737549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2236737549},{"type":"v1","addr":"172.21.15.86:6837","nonce":2236737549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/519560229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":519560229},{"type":"v1","addr":"172.21.15.86:6836","nonce":519560229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:03:50.852257+0000","modified":"2024-04-25T14:04:09.202137+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6833/1944249570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1944249570},{"type":"v1","addr":"172.21.15.155:6833","nonce":1944249570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":1}]} 2024-04-25T14:04:18.608 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 9 2024-04-25T14:04:18.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.609+0000 7f8eb9a00640 1 -- 172.21.15.86:0/4131040731 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8e9406f400 msgr2=0x7f8e94071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:18.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.609+0000 7f8eb9a00640 1 --2- 172.21.15.86:0/4131040731 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8e9406f400 0x7f8e94071910 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4011340 tx=0x7f8ea400b040 comp rx=0 tx=0).stop 2024-04-25T14:04:18.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.609+0000 7f8eb9a00640 1 -- 172.21.15.86:0/4131040731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8eb40722f0 msgr2=0x7f8eb40c6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:18.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.609+0000 7f8eb9a00640 1 --2- 172.21.15.86:0/4131040731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8eb40722f0 0x7f8eb40c6c30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0009930 tx=0x7f8ea001bad0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.609+0000 7f8eb9a00640 1 -- 172.21.15.86:0/4131040731 shutdown_connections 2024-04-25T14:04:18.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.609+0000 7f8eb9a00640 1 --2- 172.21.15.86:0/4131040731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8eb40d2c80 0x7f8eb40c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.609+0000 7f8eb9a00640 1 --2- 172.21.15.86:0/4131040731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8eb40722f0 0x7f8eb40c6c30 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.609+0000 7f8eb9a00640 1 --2- 172.21.15.86:0/4131040731 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8e9406f400 0x7f8e94071910 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.609+0000 7f8eb9a00640 1 --2- 172.21.15.86:0/4131040731 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8eb4070e20 0x7f8eb41504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.609+0000 7f8eb9a00640 1 -- 172.21.15.86:0/4131040731 >> 172.21.15.86:0/4131040731 conn(0x7f8eb40cffc0 msgr2=0x7f8eb415b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:18.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.609+0000 7f8eb9a00640 1 -- 172.21.15.86:0/4131040731 shutdown_connections 2024-04-25T14:04:18.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.609+0000 7f8eb9a00640 1 -- 172.21.15.86:0/4131040731 wait complete. 2024-04-25T14:04:18.630 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:04:18.665 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:04:18.683 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:04:18.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.777+0000 7fc3fec00640 1 Processor -- start 2024-04-25T14:04:18.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.777+0000 7fc3fec00640 1 -- start start 2024-04-25T14:04:18.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.777+0000 7fc3fec00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc3f8071530 0x7fc3f815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:18.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.777+0000 7fc3fec00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3f81616e0 0x7fc3f8161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:18.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.777+0000 7fc3fec00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3f80cd8d0 con 0x7fc3f80d2c80 2024-04-25T14:04:18.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.777+0000 7fc3fec00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc3f80ce720 con 0x7fc3f8071530 2024-04-25T14:04:18.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.777+0000 7fc3fec00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc3f80ce970 con 0x7fc3f81616e0 2024-04-25T14:04:18.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.777+0000 7fc3f7e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc3f80d2c80 0x7fc3f8070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33538/0 (socket says 172.21.15.86:33538) 2024-04-25T14:04:18.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.777+0000 7fc3f7e00640 1 -- 172.21.15.86:0/3000317417 learned_addr learned my addr 172.21.15.86:0/3000317417 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:18.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.781+0000 7fc3fcc00640 1 --2- 172.21.15.86:0/3000317417 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3f81616e0 0x7fc3f8161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:18.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.781+0000 7fc3ef400640 1 --2- 172.21.15.86:0/3000317417 >> v2:172.21.15.155:3300/0 conn(0x7fc3f8071530 0x7fc3f815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:18.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.781+0000 7fc3f7400640 1 -- 172.21.15.86:0/3000317417 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1256900013 0 0) 0x7fc3f80cd8d0 con 0x7fc3f80d2c80 2024-04-25T14:04:18.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.781+0000 7fc3f7400640 1 -- 172.21.15.86:0/3000317417 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3dc003530 con 0x7fc3f80d2c80 2024-04-25T14:04:18.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.781+0000 7fc3fcc00640 1 -- 172.21.15.86:0/3000317417 >> v2:172.21.15.155:3300/0 conn(0x7fc3f8071530 msgr2=0x7fc3f815dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:18.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.781+0000 7fc3fcc00640 1 --2- 172.21.15.86:0/3000317417 >> v2:172.21.15.155:3300/0 conn(0x7fc3f8071530 0x7fc3f815dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.781+0000 7fc3fcc00640 1 -- 172.21.15.86:0/3000317417 >> v1:172.21.15.86:6789/0 conn(0x7fc3f80d2c80 legacy=0x7fc3f8070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:18.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.781+0000 7fc3fcc00640 1 -- 172.21.15.86:0/3000317417 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f80d1be0 con 0x7fc3f81616e0 2024-04-25T14:04:18.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.781+0000 7fc3fcc00640 1 --2- 172.21.15.86:0/3000317417 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3f81616e0 0x7fc3f8161af0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009d90 tx=0x7fc3f001f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cee4fabdfa8319a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:18.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.781+0000 7fc3f7400640 1 -- 172.21.15.86:0/3000317417 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f00b4070 con 0x7fc3f81616e0 2024-04-25T14:04:18.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.781+0000 7fc3f7400640 1 -- 172.21.15.86:0/3000317417 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3f00af600 con 0x7fc3f81616e0 2024-04-25T14:04:18.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.781+0000 7fc3f7400640 1 -- 172.21.15.86:0/3000317417 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f00af9d0 con 0x7fc3f81616e0 2024-04-25T14:04:18.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.785+0000 7fc3fec00640 1 -- 172.21.15.86:0/3000317417 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3f81616e0 msgr2=0x7fc3f8161af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:18.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.785+0000 7fc3fec00640 1 --2- 172.21.15.86:0/3000317417 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3f81616e0 0x7fc3f8161af0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009d90 tx=0x7fc3f001f150 comp rx=0 tx=0).stop 2024-04-25T14:04:18.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.785+0000 7fc3fec00640 1 -- 172.21.15.86:0/3000317417 shutdown_connections 2024-04-25T14:04:18.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.785+0000 7fc3fec00640 1 --2- 172.21.15.86:0/3000317417 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3f81616e0 0x7fc3f8161af0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.785+0000 7fc3fec00640 1 --2- 172.21.15.86:0/3000317417 >> v2:172.21.15.155:3300/0 conn(0x7fc3f8071530 0x7fc3f815dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.785+0000 7fc3fec00640 1 -- 172.21.15.86:0/3000317417 >> 172.21.15.86:0/3000317417 conn(0x7fc3f80cffc0 msgr2=0x7fc3f80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:18.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.785+0000 7fc3fec00640 1 -- 172.21.15.86:0/3000317417 shutdown_connections 2024-04-25T14:04:18.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.785+0000 7fc3fec00640 1 -- 172.21.15.86:0/3000317417 wait complete. 2024-04-25T14:04:18.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.785+0000 7fc3fec00640 1 Processor -- start 2024-04-25T14:04:18.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.785+0000 7fc3fec00640 1 -- start start 2024-04-25T14:04:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.789+0000 7fc3fec00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3f80d2c80 0x7fc3f806f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.789+0000 7fc3fec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3f81616e0 0x7fc3f8153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.789+0000 7fc3fec00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3f815c360 0x7fc3f815c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.789+0000 7fc3fec00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f80d2480 con 0x7fc3f81616e0 2024-04-25T14:04:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.789+0000 7fc3fec00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f80d21a0 con 0x7fc3f80d2c80 2024-04-25T14:04:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.789+0000 7fc3fec00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc3f80d29a0 con 0x7fc3f815c360 2024-04-25T14:04:18.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.789+0000 7fc3ef400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3f81616e0 0x7fc3f8153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:18.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.789+0000 7fc3ef400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3f81616e0 0x7fc3f8153ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58664/0 (socket says 172.21.15.86:58664) 2024-04-25T14:04:18.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.789+0000 7fc3ef400640 1 -- 172.21.15.86:0/4254552422 learned_addr learned my addr 172.21.15.86:0/4254552422 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:18.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.789+0000 7fc3f7e00640 1 --2- 172.21.15.86:0/4254552422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3f80d2c80 0x7fc3f806f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:18.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.789+0000 7fc3fcc00640 1 --2- 172.21.15.86:0/4254552422 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3f815c360 0x7fc3f815c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:18.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.789+0000 7fc3ef400640 1 -- 172.21.15.86:0/4254552422 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3f815c360 msgr2=0x7fc3f815c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:18.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.789+0000 7fc3ef400640 1 --2- 172.21.15.86:0/4254552422 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3f815c360 0x7fc3f815c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.789+0000 7fc3ef400640 1 -- 172.21.15.86:0/4254552422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3f80d2c80 msgr2=0x7fc3f806f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:18.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.789+0000 7fc3ef400640 1 --2- 172.21.15.86:0/4254552422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3f80d2c80 0x7fc3f806f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.789+0000 7fc3ef400640 1 -- 172.21.15.86:0/4254552422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f8154010 con 0x7fc3f81616e0 2024-04-25T14:04:18.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.789+0000 7fc3fcc00640 1 --2- 172.21.15.86:0/4254552422 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3f815c360 0x7fc3f815c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:18.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.789+0000 7fc3ef400640 1 --2- 172.21.15.86:0/4254552422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3f81616e0 0x7fc3f8153ad0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000e8a0 tx=0x7fc3e000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:18.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.789+0000 7fc3f4c00640 1 -- 172.21.15.86:0/4254552422 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0017070 con 0x7fc3f81616e0 2024-04-25T14:04:18.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.789+0000 7fc3f4c00640 1 -- 172.21.15.86:0/4254552422 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3e000c940 con 0x7fc3f81616e0 2024-04-25T14:04:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.789+0000 7fc3fec00640 1 -- 172.21.15.86:0/4254552422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f815ccb0 con 0x7fc3f81616e0 2024-04-25T14:04:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.789+0000 7fc3f4c00640 1 -- 172.21.15.86:0/4254552422 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e000ccf0 con 0x7fc3f81616e0 2024-04-25T14:04:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.789+0000 7fc3fec00640 1 -- 172.21.15.86:0/4254552422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f806dfb0 con 0x7fc3f81616e0 2024-04-25T14:04:18.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.789+0000 7fc3fec00640 1 -- 172.21.15.86:0/4254552422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3f80720e0 con 0x7fc3f81616e0 2024-04-25T14:04:18.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.793+0000 7fc3f4c00640 1 -- 172.21.15.86:0/4254552422 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc3e001a070 con 0x7fc3f81616e0 2024-04-25T14:04:18.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.793+0000 7fc3f4c00640 1 --2- 172.21.15.86:0/4254552422 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fc3d406f430 0x7fc3d4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:18.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.793+0000 7fc3f4c00640 1 -- 172.21.15.86:0/4254552422 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc3e008f8d0 con 0x7fc3f81616e0 2024-04-25T14:04:18.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.793+0000 7fc3f7e00640 1 --2- 172.21.15.86:0/4254552422 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fc3d406f430 0x7fc3d4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:18.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.793+0000 7fc3f7e00640 1 --2- 172.21.15.86:0/4254552422 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fc3d406f430 0x7fc3d4071940 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8007fd0 tx=0x7fc3e8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:18.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.797+0000 7fc3f4c00640 1 -- 172.21.15.86:0/4254552422 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc3f815ccb0 con 0x7fc3f81616e0 2024-04-25T14:04:18.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.957+0000 7fc3fec00640 1 -- 172.21.15.86:0/4254552422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3f815dc90 con 0x7fc3f81616e0 2024-04-25T14:04:18.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.957+0000 7fc3f4c00640 1 -- 172.21.15.86:0/4254552422 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7fc3e005bd90 con 0x7fc3f81616e0 2024-04-25T14:04:18.961 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:18.961 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":24,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:03:55.899092+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:03:49.151473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:03:49.802597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T14:03:53.492008+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:04:18.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.961+0000 7fc3fec00640 1 -- 172.21.15.86:0/4254552422 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fc3d406f430 msgr2=0x7fc3d4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:18.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.961+0000 7fc3fec00640 1 --2- 172.21.15.86:0/4254552422 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fc3d406f430 0x7fc3d4071940 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8007fd0 tx=0x7fc3e8009040 comp rx=0 tx=0).stop 2024-04-25T14:04:18.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.961+0000 7fc3fec00640 1 -- 172.21.15.86:0/4254552422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3f81616e0 msgr2=0x7fc3f8153ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:18.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.961+0000 7fc3fec00640 1 --2- 172.21.15.86:0/4254552422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3f81616e0 0x7fc3f8153ad0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000e8a0 tx=0x7fc3e000ed60 comp rx=0 tx=0).stop 2024-04-25T14:04:18.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.961+0000 7fc3fec00640 1 -- 172.21.15.86:0/4254552422 shutdown_connections 2024-04-25T14:04:18.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.961+0000 7fc3fec00640 1 --2- 172.21.15.86:0/4254552422 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fc3d406f430 0x7fc3d4071940 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.961+0000 7fc3fec00640 1 --2- 172.21.15.86:0/4254552422 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3f815c360 0x7fc3f815c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.961+0000 7fc3fec00640 1 --2- 172.21.15.86:0/4254552422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3f81616e0 0x7fc3f8153ad0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.961+0000 7fc3fec00640 1 --2- 172.21.15.86:0/4254552422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3f80d2c80 0x7fc3f806f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:18.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.961+0000 7fc3fec00640 1 -- 172.21.15.86:0/4254552422 >> 172.21.15.86:0/4254552422 conn(0x7fc3f80cffc0 msgr2=0x7fc3f815a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:18.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.961+0000 7fc3fec00640 1 -- 172.21.15.86:0/4254552422 shutdown_connections 2024-04-25T14:04:18.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:18.961+0000 7fc3fec00640 1 -- 172.21.15.86:0/4254552422 wait complete. 2024-04-25T14:04:18.982 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:03:49.151473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:03:49.802597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T14:03:53.492008+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T14:04:18.983 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:04:19.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.129+0000 7ff27b000640 1 Processor -- start 2024-04-25T14:04:19.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.129+0000 7ff27b000640 1 -- start start 2024-04-25T14:04:19.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff27b000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff2740d1df0 0x7ff2740cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:19.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff27b000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff27415e5c0 con 0x7ff2740cdcc0 2024-04-25T14:04:19.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff27b000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff27415e8a0 con 0x7ff274070330 2024-04-25T14:04:19.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff27b000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff27406fd40 con 0x7ff2740d1df0 2024-04-25T14:04:19.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff279000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff2740cdcc0 0x7ff2740ce0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:38818/0 (socket says 172.21.15.86:38818) 2024-04-25T14:04:19.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff279000640 1 -- 172.21.15.86:0/2572211212 learned_addr learned my addr 172.21.15.86:0/2572211212 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:19.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff273400640 1 -- 172.21.15.86:0/2572211212 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4189675600 0 0) 0x7ff27415e5c0 con 0x7ff2740cdcc0 2024-04-25T14:04:19.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff26b400640 1 --2- 172.21.15.86:0/2572211212 >> v2:172.21.15.155:3300/0 conn(0x7ff2740d1df0 0x7ff2740cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:19.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff273400640 1 -- 172.21.15.86:0/2572211212 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff258003530 con 0x7ff2740cdcc0 2024-04-25T14:04:19.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff273400640 1 -- 172.21.15.86:0/2572211212 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 806023551 0 0) 0x7ff258003530 con 0x7ff2740cdcc0 2024-04-25T14:04:19.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff273400640 1 -- 172.21.15.86:0/2572211212 >> v2:172.21.15.155:3300/0 conn(0x7ff2740d1df0 msgr2=0x7ff2740cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:19.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff273400640 1 --2- 172.21.15.86:0/2572211212 >> v2:172.21.15.155:3300/0 conn(0x7ff2740d1df0 0x7ff2740cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff273400640 1 -- 172.21.15.86:0/2572211212 >> v1:172.21.15.155:6789/0 conn(0x7ff274070330 legacy=0x7ff2740d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:19.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff273400640 1 -- 172.21.15.86:0/2572211212 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff27415eb80 con 0x7ff2740cdcc0 2024-04-25T14:04:19.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff273400640 1 -- 172.21.15.86:0/2572211212 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7ff26c001a00 con 0x7ff2740cdcc0 2024-04-25T14:04:19.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff273400640 1 -- 172.21.15.86:0/2572211212 >> v1:172.21.15.86:6789/0 conn(0x7ff2740cdcc0 legacy=0x7ff2740ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:19.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff273400640 1 --2- 172.21.15.86:0/2572211212 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff258003a30 0x7ff258003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:19.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff273400640 1 --2- 172.21.15.86:0/2572211212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff258004810 0x7ff258004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:19.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff273400640 1 --2- 172.21.15.86:0/2572211212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff258005360 0x7ff25800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:19.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff273400640 1 -- 172.21.15.86:0/2572211212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff25800c230 con 0x7ff258004810 2024-04-25T14:04:19.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff273400640 1 -- 172.21.15.86:0/2572211212 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff25800c3e0 con 0x7ff258005360 2024-04-25T14:04:19.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff273400640 1 -- 172.21.15.86:0/2572211212 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff25800c5e0 con 0x7ff258003a30 2024-04-25T14:04:19.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff26b400640 1 --2- 172.21.15.86:0/2572211212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff258004810 0x7ff258004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:19.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff279000640 1 --2- 172.21.15.86:0/2572211212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff258005360 0x7ff25800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:19.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff273e00640 1 --2- 172.21.15.86:0/2572211212 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff258003a30 0x7ff258003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:19.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff26b400640 1 -- 172.21.15.86:0/2572211212 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff258003a30 msgr2=0x7ff258003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:19.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff26b400640 1 --2- 172.21.15.86:0/2572211212 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff258003a30 0x7ff258003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff26b400640 1 -- 172.21.15.86:0/2572211212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff258005360 msgr2=0x7ff25800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:19.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff26b400640 1 --2- 172.21.15.86:0/2572211212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff258005360 0x7ff25800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff26b400640 1 -- 172.21.15.86:0/2572211212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff258003530 con 0x7ff258004810 2024-04-25T14:04:19.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.133+0000 7ff279000640 1 -- 172.21.15.86:0/2572211212 reap_dead start 2024-04-25T14:04:19.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.137+0000 7ff26b400640 1 --2- 172.21.15.86:0/2572211212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff258004810 0x7ff258004c20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff274074ed0 tx=0x7ff25c01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23bddf0cfe06c8b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:19.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.137+0000 7ff273400640 1 -- 172.21.15.86:0/2572211212 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c0096b0 con 0x7ff258004810 2024-04-25T14:04:19.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.137+0000 7ff273400640 1 -- 172.21.15.86:0/2572211212 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff25c0a64d0 con 0x7ff258004810 2024-04-25T14:04:19.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.137+0000 7ff273400640 1 -- 172.21.15.86:0/2572211212 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c0a68c0 con 0x7ff258004810 2024-04-25T14:04:19.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.137+0000 7ff27b000640 1 -- 172.21.15.86:0/2572211212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff258004810 msgr2=0x7ff258004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:19.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.137+0000 7ff27b000640 1 --2- 172.21.15.86:0/2572211212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff258004810 0x7ff258004c20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff274074ed0 tx=0x7ff25c01c9e0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.137+0000 7ff27b000640 1 -- 172.21.15.86:0/2572211212 shutdown_connections 2024-04-25T14:04:19.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.137+0000 7ff27b000640 1 --2- 172.21.15.86:0/2572211212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff258004810 0x7ff258004c20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.137+0000 7ff27b000640 1 -- 172.21.15.86:0/2572211212 >> 172.21.15.86:0/2572211212 conn(0x7ff2740b4650 msgr2=0x7ff2740b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:19.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.137+0000 7ff27b000640 1 -- 172.21.15.86:0/2572211212 shutdown_connections 2024-04-25T14:04:19.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.137+0000 7ff27b000640 1 -- 172.21.15.86:0/2572211212 wait complete. 2024-04-25T14:04:19.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.141+0000 7ff27b000640 1 Processor -- start 2024-04-25T14:04:19.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.141+0000 7ff27b000640 1 -- start start 2024-04-25T14:04:19.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.141+0000 7ff27b000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff258004810 0x7ff27415af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:19.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.141+0000 7ff27b000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2740c9220 0x7ff2740cff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:19.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.141+0000 7ff27b000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2741fc740 0x7ff2740d0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:19.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.141+0000 7ff27b000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff2740708c0 con 0x7ff258004810 2024-04-25T14:04:19.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.141+0000 7ff27b000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff274070740 con 0x7ff2741fc740 2024-04-25T14:04:19.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.141+0000 7ff27b000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff2740ce0d0 con 0x7ff2740c9220 2024-04-25T14:04:19.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.141+0000 7ff273e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff258004810 0x7ff27415af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:19.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.141+0000 7ff273e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff258004810 0x7ff27415af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58686/0 (socket says 172.21.15.86:58686) 2024-04-25T14:04:19.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.141+0000 7ff273e00640 1 -- 172.21.15.86:0/1845860612 learned_addr learned my addr 172.21.15.86:0/1845860612 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:19.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.141+0000 7ff26b400640 1 --2- 172.21.15.86:0/1845860612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2740c9220 0x7ff2740cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:19.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.141+0000 7ff279000640 1 --2- 172.21.15.86:0/1845860612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2741fc740 0x7ff2740d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:19.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.141+0000 7ff273e00640 1 -- 172.21.15.86:0/1845860612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2740c9220 msgr2=0x7ff2740cff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:19.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.141+0000 7ff273e00640 1 --2- 172.21.15.86:0/1845860612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2740c9220 0x7ff2740cff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.141+0000 7ff273e00640 1 -- 172.21.15.86:0/1845860612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2741fc740 msgr2=0x7ff2740d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:19.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.141+0000 7ff273e00640 1 --2- 172.21.15.86:0/1845860612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2741fc740 0x7ff2740d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.141+0000 7ff273e00640 1 -- 172.21.15.86:0/1845860612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2740c9990 con 0x7ff258004810 2024-04-25T14:04:19.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.141+0000 7ff279000640 1 --2- 172.21.15.86:0/1845860612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2741fc740 0x7ff2740d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:19.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.141+0000 7ff273e00640 1 --2- 172.21.15.86:0/1845860612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff258004810 0x7ff27415af60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff26400f5c0 tx=0x7ff26400fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:19.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.141+0000 7ff270c00640 1 -- 172.21.15.86:0/1845860612 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2640052e0 con 0x7ff258004810 2024-04-25T14:04:19.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.141+0000 7ff270c00640 1 -- 172.21.15.86:0/1845860612 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff264018070 con 0x7ff258004810 2024-04-25T14:04:19.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.141+0000 7ff270c00640 1 -- 172.21.15.86:0/1845860612 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff264003840 con 0x7ff258004810 2024-04-25T14:04:19.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.141+0000 7ff27b000640 1 -- 172.21.15.86:0/1845860612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2740c9c70 con 0x7ff258004810 2024-04-25T14:04:19.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.145+0000 7ff27b000640 1 -- 172.21.15.86:0/1845860612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff274160fc0 con 0x7ff258004810 2024-04-25T14:04:19.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.145+0000 7ff27b000640 1 -- 172.21.15.86:0/1845860612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff274069d00 con 0x7ff258004810 2024-04-25T14:04:19.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.145+0000 7ff270c00640 1 -- 172.21.15.86:0/1845860612 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff2640039e0 con 0x7ff258004810 2024-04-25T14:04:19.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.145+0000 7ff270c00640 1 --2- 172.21.15.86:0/1845860612 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff25006f430 0x7ff250071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:19.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.145+0000 7ff270c00640 1 -- 172.21.15.86:0/1845860612 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff26408f2d0 con 0x7ff258004810 2024-04-25T14:04:19.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.145+0000 7ff26b400640 1 --2- 172.21.15.86:0/1845860612 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff25006f430 0x7ff250071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:19.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.145+0000 7ff26b400640 1 --2- 172.21.15.86:0/1845860612 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff25006f430 0x7ff250071940 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff25c0af000 tx=0x7ff25c0046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:19.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.149+0000 7ff270c00640 1 -- 172.21.15.86:0/1845860612 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff26405bf10 con 0x7ff258004810 2024-04-25T14:04:19.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.309+0000 7ff27b000640 1 -- 172.21.15.86:0/1845860612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff274076bf0 con 0x7ff258004810 2024-04-25T14:04:19.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.309+0000 7ff270c00640 1 -- 172.21.15.86:0/1845860612 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7ff26405b810 con 0x7ff258004810 2024-04-25T14:04:19.312 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:04:19.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.313+0000 7ff27b000640 1 -- 172.21.15.86:0/1845860612 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff25006f430 msgr2=0x7ff250071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:19.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.313+0000 7ff27b000640 1 --2- 172.21.15.86:0/1845860612 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff25006f430 0x7ff250071940 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff25c0af000 tx=0x7ff25c0046d0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.313+0000 7ff27b000640 1 -- 172.21.15.86:0/1845860612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff258004810 msgr2=0x7ff27415af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:19.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.313+0000 7ff27b000640 1 --2- 172.21.15.86:0/1845860612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff258004810 0x7ff27415af60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff26400f5c0 tx=0x7ff26400fa80 comp rx=0 tx=0).stop 2024-04-25T14:04:19.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.313+0000 7ff27b000640 1 -- 172.21.15.86:0/1845860612 shutdown_connections 2024-04-25T14:04:19.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.313+0000 7ff27b000640 1 --2- 172.21.15.86:0/1845860612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2741fc740 0x7ff2740d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.313+0000 7ff27b000640 1 --2- 172.21.15.86:0/1845860612 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff25006f430 0x7ff250071940 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.313+0000 7ff27b000640 1 --2- 172.21.15.86:0/1845860612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2740c9220 0x7ff2740cff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.313+0000 7ff27b000640 1 --2- 172.21.15.86:0/1845860612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff258004810 0x7ff27415af60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.313+0000 7ff27b000640 1 -- 172.21.15.86:0/1845860612 >> 172.21.15.86:0/1845860612 conn(0x7ff2740b4650 msgr2=0x7ff2740ae130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:19.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.313+0000 7ff27b000640 1 -- 172.21.15.86:0/1845860612 shutdown_connections 2024-04-25T14:04:19.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.313+0000 7ff27b000640 1 -- 172.21.15.86:0/1845860612 wait complete. 2024-04-25T14:04:19.334 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:04:19.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.481+0000 7ff2ea200640 1 Processor -- start 2024-04-25T14:04:19.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.481+0000 7ff2ea200640 1 -- start start 2024-04-25T14:04:19.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.485+0000 7ff2ea200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2e41610d0 0x7ff2e40b7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:19.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.485+0000 7ff2ea200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff2e40b2250 0x7ff2e40b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:19.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.485+0000 7ff2ea200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff2e40b2ba0 0x7ff2e40d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:19.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.485+0000 7ff2ea200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff2e40b7f70 con 0x7ff2e40b2250 2024-04-25T14:04:19.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.485+0000 7ff2ea200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff2e4161620 con 0x7ff2e40b2ba0 2024-04-25T14:04:19.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.485+0000 7ff2ea200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff2e40b3750 con 0x7ff2e41610d0 2024-04-25T14:04:19.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.485+0000 7ff2daa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff2e40b2250 0x7ff2e40b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:19.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.485+0000 7ff2daa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff2e40b2250 0x7ff2e40b2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58690/0 (socket says 172.21.15.86:58690) 2024-04-25T14:04:19.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.485+0000 7ff2daa00640 1 -- 172.21.15.86:0/2597094545 learned_addr learned my addr 172.21.15.86:0/2597094545 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:19.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.485+0000 7ff2e3e00640 1 --2- 172.21.15.86:0/2597094545 >> v2:172.21.15.155:3300/0 conn(0x7ff2e40b2ba0 0x7ff2e40d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:19.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.485+0000 7ff2daa00640 1 -- 172.21.15.86:0/2597094545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2e41610d0 msgr2=0x7ff2e40b7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:19.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.485+0000 7ff2e3400640 1 --2- 172.21.15.86:0/2597094545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2e41610d0 0x7ff2e40b7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:19.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.485+0000 7ff2daa00640 1 --2- 172.21.15.86:0/2597094545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2e41610d0 0x7ff2e40b7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.485+0000 7ff2daa00640 1 -- 172.21.15.86:0/2597094545 >> v2:172.21.15.155:3300/0 conn(0x7ff2e40b2ba0 msgr2=0x7ff2e40d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:19.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.485+0000 7ff2daa00640 1 --2- 172.21.15.86:0/2597094545 >> v2:172.21.15.155:3300/0 conn(0x7ff2e40b2ba0 0x7ff2e40d15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.485+0000 7ff2daa00640 1 -- 172.21.15.86:0/2597094545 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e40ce7b0 con 0x7ff2e40b2250 2024-04-25T14:04:19.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.485+0000 7ff2e3400640 1 --2- 172.21.15.86:0/2597094545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2e41610d0 0x7ff2e40b7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:19.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.489+0000 7ff2daa00640 1 --2- 172.21.15.86:0/2597094545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e40b2250 0x7ff2e40b2660 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4163b00 tx=0x7ff2cc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497ac94a12805f5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:19.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.489+0000 7ff2e2a00640 1 -- 172.21.15.86:0/2597094545 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc0b0070 con 0x7ff2e40b2250 2024-04-25T14:04:19.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.489+0000 7ff2e2a00640 1 -- 172.21.15.86:0/2597094545 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2cc0ab600 con 0x7ff2e40b2250 2024-04-25T14:04:19.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.489+0000 7ff2e2a00640 1 -- 172.21.15.86:0/2597094545 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc0ab9d0 con 0x7ff2e40b2250 2024-04-25T14:04:19.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.489+0000 7ff2ea200640 1 -- 172.21.15.86:0/2597094545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e40b2250 msgr2=0x7ff2e40b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:19.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.489+0000 7ff2ea200640 1 --2- 172.21.15.86:0/2597094545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e40b2250 0x7ff2e40b2660 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4163b00 tx=0x7ff2cc01c520 comp rx=0 tx=0).stop 2024-04-25T14:04:19.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.489+0000 7ff2ea200640 1 -- 172.21.15.86:0/2597094545 shutdown_connections 2024-04-25T14:04:19.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.489+0000 7ff2ea200640 1 --2- 172.21.15.86:0/2597094545 >> v2:172.21.15.155:3300/0 conn(0x7ff2e40b2ba0 0x7ff2e40d15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.489+0000 7ff2ea200640 1 --2- 172.21.15.86:0/2597094545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e40b2250 0x7ff2e40b2660 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.489+0000 7ff2ea200640 1 --2- 172.21.15.86:0/2597094545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2e41610d0 0x7ff2e40b7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.489+0000 7ff2ea200640 1 -- 172.21.15.86:0/2597094545 >> 172.21.15.86:0/2597094545 conn(0x7ff2e409ba30 msgr2=0x7ff2e40b7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:19.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.489+0000 7ff2ea200640 1 -- 172.21.15.86:0/2597094545 shutdown_connections 2024-04-25T14:04:19.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.489+0000 7ff2ea200640 1 -- 172.21.15.86:0/2597094545 wait complete. 2024-04-25T14:04:19.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.489+0000 7ff2ea200640 1 Processor -- start 2024-04-25T14:04:19.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.489+0000 7ff2ea200640 1 -- start start 2024-04-25T14:04:19.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.493+0000 7ff2ea200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e40b2250 0x7ff2e406c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:19.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.493+0000 7ff2ea200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2e40b2ba0 0x7ff2e40b1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:19.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.493+0000 7ff2ea200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2e41610d0 0x7ff2e41523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:19.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.493+0000 7ff2ea200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e4163d40 con 0x7ff2e40b2250 2024-04-25T14:04:19.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.493+0000 7ff2ea200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e4163bc0 con 0x7ff2e40b2ba0 2024-04-25T14:04:19.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.493+0000 7ff2ea200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff2e4163ec0 con 0x7ff2e41610d0 2024-04-25T14:04:19.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.493+0000 7ff2e3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e40b2250 0x7ff2e406c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:19.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.493+0000 7ff2e3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e40b2250 0x7ff2e406c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58704/0 (socket says 172.21.15.86:58704) 2024-04-25T14:04:19.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.493+0000 7ff2e3400640 1 -- 172.21.15.86:0/384125250 learned_addr learned my addr 172.21.15.86:0/384125250 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:19.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.493+0000 7ff2e3e00640 1 --2- 172.21.15.86:0/384125250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2e41610d0 0x7ff2e41523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:19.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.493+0000 7ff2daa00640 1 --2- 172.21.15.86:0/384125250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2e40b2ba0 0x7ff2e40b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:19.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.493+0000 7ff2e3400640 1 -- 172.21.15.86:0/384125250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2e41610d0 msgr2=0x7ff2e41523a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:19.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.493+0000 7ff2e3400640 1 --2- 172.21.15.86:0/384125250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2e41610d0 0x7ff2e41523a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.493+0000 7ff2e3400640 1 -- 172.21.15.86:0/384125250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2e40b2ba0 msgr2=0x7ff2e40b1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:19.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.493+0000 7ff2e3400640 1 --2- 172.21.15.86:0/384125250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2e40b2ba0 0x7ff2e40b1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.493+0000 7ff2e3400640 1 -- 172.21.15.86:0/384125250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e4152c90 con 0x7ff2e40b2250 2024-04-25T14:04:19.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.493+0000 7ff2e3400640 1 --2- 172.21.15.86:0/384125250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e40b2250 0x7ff2e406c670 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400c7f0 tx=0x7ff2d400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:19.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.493+0000 7ff2dbe00640 1 -- 172.21.15.86:0/384125250 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4015070 con 0x7ff2e40b2250 2024-04-25T14:04:19.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.493+0000 7ff2dbe00640 1 -- 172.21.15.86:0/384125250 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2d4007820 con 0x7ff2e40b2250 2024-04-25T14:04:19.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.493+0000 7ff2ea200640 1 -- 172.21.15.86:0/384125250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e40b19b0 con 0x7ff2e40b2250 2024-04-25T14:04:19.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.493+0000 7ff2dbe00640 1 -- 172.21.15.86:0/384125250 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4007c10 con 0x7ff2e40b2250 2024-04-25T14:04:19.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.493+0000 7ff2ea200640 1 -- 172.21.15.86:0/384125250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e4152f00 con 0x7ff2e40b2250 2024-04-25T14:04:19.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.493+0000 7ff2ea200640 1 -- 172.21.15.86:0/384125250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2e40d0340 con 0x7ff2e40b2250 2024-04-25T14:04:19.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.497+0000 7ff2dbe00640 1 -- 172.21.15.86:0/384125250 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff2d4018070 con 0x7ff2e40b2250 2024-04-25T14:04:19.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.497+0000 7ff2dbe00640 1 --2- 172.21.15.86:0/384125250 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff2c006f400 0x7ff2c0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:19.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.497+0000 7ff2dbe00640 1 -- 172.21.15.86:0/384125250 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff2d408f830 con 0x7ff2e40b2250 2024-04-25T14:04:19.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.497+0000 7ff2daa00640 1 --2- 172.21.15.86:0/384125250 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff2c006f400 0x7ff2c0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:19.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.497+0000 7ff2daa00640 1 --2- 172.21.15.86:0/384125250 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff2c006f400 0x7ff2c0071910 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc0ad000 tx=0x7ff2cc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:19.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.501+0000 7ff2dbe00640 1 -- 172.21.15.86:0/384125250 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff2d405c340 con 0x7ff2e40b2250 2024-04-25T14:04:19.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.661+0000 7ff2ea200640 1 -- 172.21.15.86:0/384125250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff2e4076bf0 con 0x7ff2e40b2250 2024-04-25T14:04:19.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.661+0000 7ff2dbe00640 1 -- 172.21.15.86:0/384125250 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7ff2d405bc40 con 0x7ff2e40b2250 2024-04-25T14:04:19.662 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:04:19.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.661+0000 7ff2ea200640 1 -- 172.21.15.86:0/384125250 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff2c006f400 msgr2=0x7ff2c0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:19.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.661+0000 7ff2ea200640 1 --2- 172.21.15.86:0/384125250 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff2c006f400 0x7ff2c0071910 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc0ad000 tx=0x7ff2cc008040 comp rx=0 tx=0).stop 2024-04-25T14:04:19.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.661+0000 7ff2ea200640 1 -- 172.21.15.86:0/384125250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e40b2250 msgr2=0x7ff2e406c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:19.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.661+0000 7ff2ea200640 1 --2- 172.21.15.86:0/384125250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e40b2250 0x7ff2e406c670 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400c7f0 tx=0x7ff2d400ccb0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.661+0000 7ff2ea200640 1 -- 172.21.15.86:0/384125250 shutdown_connections 2024-04-25T14:04:19.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.661+0000 7ff2ea200640 1 --2- 172.21.15.86:0/384125250 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff2c006f400 0x7ff2c0071910 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.661+0000 7ff2ea200640 1 --2- 172.21.15.86:0/384125250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2e41610d0 0x7ff2e41523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.661+0000 7ff2ea200640 1 --2- 172.21.15.86:0/384125250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2e40b2ba0 0x7ff2e40b1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.661+0000 7ff2ea200640 1 --2- 172.21.15.86:0/384125250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2e40b2250 0x7ff2e406c670 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.661+0000 7ff2ea200640 1 -- 172.21.15.86:0/384125250 >> 172.21.15.86:0/384125250 conn(0x7ff2e409ba30 msgr2=0x7ff2e4161840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:19.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.661+0000 7ff2ea200640 1 -- 172.21.15.86:0/384125250 shutdown_connections 2024-04-25T14:04:19.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.661+0000 7ff2ea200640 1 -- 172.21.15.86:0/384125250 wait complete. 2024-04-25T14:04:19.685 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:04:19.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.833+0000 7fb70b200640 1 Processor -- start 2024-04-25T14:04:19.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.833+0000 7fb70b200640 1 -- start start 2024-04-25T14:04:19.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.833+0000 7fb70b200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb7040d2c80 0x7fb70415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:19.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.833+0000 7fb70b200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb704070e20 0x7fb704071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:19.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.833+0000 7fb70b200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb704071770 con 0x7fb7040722f0 2024-04-25T14:04:19.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.833+0000 7fb70b200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb7040ce720 con 0x7fb704070e20 2024-04-25T14:04:19.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.833+0000 7fb70b200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb7040cd8d0 con 0x7fb7040d2c80 2024-04-25T14:04:19.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.833+0000 7fb703400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb704070e20 0x7fb704071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:19.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.833+0000 7fb709200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb7040722f0 0x7fb704072700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:38822/0 (socket says 172.21.15.86:38822) 2024-04-25T14:04:19.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.833+0000 7fb709200640 1 -- 172.21.15.86:0/1149574657 learned_addr learned my addr 172.21.15.86:0/1149574657 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:19.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.833+0000 7fb703e00640 1 --2- 172.21.15.86:0/1149574657 >> v2:172.21.15.155:3300/0 conn(0x7fb7040d2c80 0x7fb70415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:19.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.837+0000 7fb702a00640 1 -- 172.21.15.86:0/1149574657 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2871019731 0 0) 0x7fb6fc0014a0 con 0x7fb7040722f0 2024-04-25T14:04:19.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.837+0000 7fb702a00640 1 -- 172.21.15.86:0/1149574657 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6e8003530 con 0x7fb7040722f0 2024-04-25T14:04:19.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.837+0000 7fb703400640 1 -- 172.21.15.86:0/1149574657 >> v2:172.21.15.155:3300/0 conn(0x7fb7040d2c80 msgr2=0x7fb70415dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:19.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.837+0000 7fb703400640 1 --2- 172.21.15.86:0/1149574657 >> v2:172.21.15.155:3300/0 conn(0x7fb7040d2c80 0x7fb70415dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.837+0000 7fb703400640 1 -- 172.21.15.86:0/1149574657 >> v1:172.21.15.86:6789/0 conn(0x7fb7040722f0 legacy=0x7fb704072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:19.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.837+0000 7fb703400640 1 -- 172.21.15.86:0/1149574657 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7040ce970 con 0x7fb704070e20 2024-04-25T14:04:19.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.837+0000 7fb703400640 1 --2- 172.21.15.86:0/1149574657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb704070e20 0x7fb704071230 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb7040d11a0 tx=0x7fb6f001ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe944d617a80be4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:19.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.837+0000 7fb702a00640 1 -- 172.21.15.86:0/1149574657 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f00b5070 con 0x7fb704070e20 2024-04-25T14:04:19.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.837+0000 7fb702a00640 1 -- 172.21.15.86:0/1149574657 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6f00a8950 con 0x7fb704070e20 2024-04-25T14:04:19.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.837+0000 7fb702a00640 1 -- 172.21.15.86:0/1149574657 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f00a8d20 con 0x7fb704070e20 2024-04-25T14:04:19.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.837+0000 7fb70b200640 1 -- 172.21.15.86:0/1149574657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb704070e20 msgr2=0x7fb704071230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:19.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.837+0000 7fb70b200640 1 --2- 172.21.15.86:0/1149574657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb704070e20 0x7fb704071230 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb7040d11a0 tx=0x7fb6f001ec50 comp rx=0 tx=0).stop 2024-04-25T14:04:19.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.841+0000 7fb70b200640 1 -- 172.21.15.86:0/1149574657 shutdown_connections 2024-04-25T14:04:19.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.841+0000 7fb70b200640 1 --2- 172.21.15.86:0/1149574657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb704070e20 0x7fb704071230 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.841+0000 7fb70b200640 1 --2- 172.21.15.86:0/1149574657 >> v2:172.21.15.155:3300/0 conn(0x7fb7040d2c80 0x7fb70415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.841+0000 7fb70b200640 1 -- 172.21.15.86:0/1149574657 >> 172.21.15.86:0/1149574657 conn(0x7fb7040cffc0 msgr2=0x7fb7040d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:19.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.841+0000 7fb70b200640 1 -- 172.21.15.86:0/1149574657 shutdown_connections 2024-04-25T14:04:19.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.841+0000 7fb70b200640 1 -- 172.21.15.86:0/1149574657 wait complete. 2024-04-25T14:04:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.841+0000 7fb70b200640 1 Processor -- start 2024-04-25T14:04:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.841+0000 7fb70b200640 1 -- start start 2024-04-25T14:04:19.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.841+0000 7fb70b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb704070e20 0x7fb7041f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:19.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.841+0000 7fb70b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7040722f0 0x7fb7040b6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:19.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.841+0000 7fb70b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7040d2c80 0x7fb7040b7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:19.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.841+0000 7fb70b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb7040d21a0 con 0x7fb704070e20 2024-04-25T14:04:19.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.841+0000 7fb70b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb7040d1ec0 con 0x7fb7040722f0 2024-04-25T14:04:19.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.841+0000 7fb70b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb7040d2480 con 0x7fb7040d2c80 2024-04-25T14:04:19.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.845+0000 7fb703e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb704070e20 0x7fb7041f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:19.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.845+0000 7fb703e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb704070e20 0x7fb7041f5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58718/0 (socket says 172.21.15.86:58718) 2024-04-25T14:04:19.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.845+0000 7fb703e00640 1 -- 172.21.15.86:0/607647493 learned_addr learned my addr 172.21.15.86:0/607647493 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:19.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.845+0000 7fb703e00640 1 -- 172.21.15.86:0/607647493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7040d2c80 msgr2=0x7fb7040b7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:19.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.845+0000 7fb703e00640 1 --2- 172.21.15.86:0/607647493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7040d2c80 0x7fb7040b7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.845+0000 7fb703e00640 1 -- 172.21.15.86:0/607647493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7040722f0 msgr2=0x7fb7040b6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:19.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.845+0000 7fb703e00640 1 --2- 172.21.15.86:0/607647493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7040722f0 0x7fb7040b6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:19.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.845+0000 7fb703e00640 1 -- 172.21.15.86:0/607647493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7040b0640 con 0x7fb704070e20 2024-04-25T14:04:19.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.845+0000 7fb703e00640 1 --2- 172.21.15.86:0/607647493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb704070e20 0x7fb7041f5cc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400d620 tx=0x7fb6f400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:19.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.845+0000 7fb6fbe00640 1 -- 172.21.15.86:0/607647493 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f400b620 con 0x7fb704070e20 2024-04-25T14:04:19.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.845+0000 7fb6fbe00640 1 -- 172.21.15.86:0/607647493 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6f4019070 con 0x7fb704070e20 2024-04-25T14:04:19.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.845+0000 7fb6fbe00640 1 -- 172.21.15.86:0/607647493 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f400b9e0 con 0x7fb704070e20 2024-04-25T14:04:19.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.845+0000 7fb70b200640 1 -- 172.21.15.86:0/607647493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7040b7da0 con 0x7fb704070e20 2024-04-25T14:04:19.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.845+0000 7fb70b200640 1 -- 172.21.15.86:0/607647493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7040aeb00 con 0x7fb704070e20 2024-04-25T14:04:19.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.845+0000 7fb70b200640 1 -- 172.21.15.86:0/607647493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7040720e0 con 0x7fb704070e20 2024-04-25T14:04:19.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.845+0000 7fb6fbe00640 1 -- 172.21.15.86:0/607647493 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb6f400bb80 con 0x7fb704070e20 2024-04-25T14:04:19.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.849+0000 7fb6fbe00640 1 --2- 172.21.15.86:0/607647493 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb6e406f430 0x7fb6e4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:19.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.849+0000 7fb6fbe00640 1 -- 172.21.15.86:0/607647493 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb6f408fa70 con 0x7fb704070e20 2024-04-25T14:04:19.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.849+0000 7fb703400640 1 --2- 172.21.15.86:0/607647493 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb6e406f430 0x7fb6e4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:19.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.849+0000 7fb703400640 1 --2- 172.21.15.86:0/607647493 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb6e406f430 0x7fb6e4071940 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00b2000 tx=0x7fb6f0004740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:19.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:19.853+0000 7fb6fbe00640 1 -- 172.21.15.86:0/607647493 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb6f405c600 con 0x7fb704070e20 2024-04-25T14:04:20.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.009+0000 7fb70b200640 1 -- 172.21.15.86:0/607647493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb7040d03d0 con 0x7fb704070e20 2024-04-25T14:04:20.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.013+0000 7fb6fbe00640 1 -- 172.21.15.86:0/607647493 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fb6f408fe20 con 0x7fb704070e20 2024-04-25T14:04:20.013 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:04:20.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.013+0000 7fb70b200640 1 -- 172.21.15.86:0/607647493 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb6e406f430 msgr2=0x7fb6e4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.013+0000 7fb70b200640 1 --2- 172.21.15.86:0/607647493 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb6e406f430 0x7fb6e4071940 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00b2000 tx=0x7fb6f0004740 comp rx=0 tx=0).stop 2024-04-25T14:04:20.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.013+0000 7fb70b200640 1 -- 172.21.15.86:0/607647493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb704070e20 msgr2=0x7fb7041f5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.013+0000 7fb70b200640 1 --2- 172.21.15.86:0/607647493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb704070e20 0x7fb7041f5cc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400d620 tx=0x7fb6f400dae0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.013+0000 7fb70b200640 1 -- 172.21.15.86:0/607647493 shutdown_connections 2024-04-25T14:04:20.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.013+0000 7fb70b200640 1 --2- 172.21.15.86:0/607647493 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb6e406f430 0x7fb6e4071940 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.013+0000 7fb70b200640 1 --2- 172.21.15.86:0/607647493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7040d2c80 0x7fb7040b7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.013+0000 7fb70b200640 1 --2- 172.21.15.86:0/607647493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7040722f0 0x7fb7040b6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.013+0000 7fb70b200640 1 --2- 172.21.15.86:0/607647493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb704070e20 0x7fb7041f5cc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.013+0000 7fb70b200640 1 -- 172.21.15.86:0/607647493 >> 172.21.15.86:0/607647493 conn(0x7fb7040cffc0 msgr2=0x7fb70406eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:20.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.013+0000 7fb70b200640 1 -- 172.21.15.86:0/607647493 shutdown_connections 2024-04-25T14:04:20.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.013+0000 7fb70b200640 1 -- 172.21.15.86:0/607647493 wait complete. 2024-04-25T14:04:20.035 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T14:04:20.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.181+0000 7f4eb7800640 1 Processor -- start 2024-04-25T14:04:20.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.181+0000 7f4eb7800640 1 -- start start 2024-04-25T14:04:20.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.185+0000 7f4eb7800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4eb00d16e0 0x7f4eb00d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.185+0000 7f4eb7800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eb00d2030 con 0x7f4eb00cfca0 2024-04-25T14:04:20.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.185+0000 7f4eb7800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eb0072bd0 con 0x7f4eb00730b0 2024-04-25T14:04:20.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.185+0000 7f4eb7800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f4eb0070470 con 0x7f4eb00d16e0 2024-04-25T14:04:20.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.185+0000 7f4eb4e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f4eb00cfca0 0x7f4eb006f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:38838/0 (socket says 172.21.15.86:38838) 2024-04-25T14:04:20.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.185+0000 7f4eb4e00640 1 -- 172.21.15.86:0/1214146904 learned_addr learned my addr 172.21.15.86:0/1214146904 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:20.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.185+0000 7f4eafe00640 1 --2- 172.21.15.86:0/1214146904 >> v2:172.21.15.155:3300/0 conn(0x7f4eb00d16e0 0x7f4eb00d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.185+0000 7f4eaf400640 1 -- 172.21.15.86:0/1214146904 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 775898254 0 0) 0x7f4ea4001120 con 0x7f4eb00cfca0 2024-04-25T14:04:20.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.185+0000 7f4eaf400640 1 -- 172.21.15.86:0/1214146904 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e98003530 con 0x7f4eb00cfca0 2024-04-25T14:04:20.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.185+0000 7f4eafe00640 1 -- 172.21.15.86:0/1214146904 >> v1:172.21.15.155:6789/0 conn(0x7f4eb00730b0 legacy=0x7f4eb00734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.185+0000 7f4eafe00640 1 -- 172.21.15.86:0/1214146904 >> v1:172.21.15.86:6789/0 conn(0x7f4eb00cfca0 legacy=0x7f4eb006f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.185+0000 7f4eafe00640 1 -- 172.21.15.86:0/1214146904 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb006fbd0 con 0x7f4eb00d16e0 2024-04-25T14:04:20.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.189+0000 7f4eafe00640 1 --2- 172.21.15.86:0/1214146904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4eb00d16e0 0x7f4eb00d1af0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0097c0 tx=0x7f4e9c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87affd2e47701005 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:20.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.189+0000 7f4eaf400640 1 -- 172.21.15.86:0/1214146904 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c0b1070 con 0x7f4eb00d16e0 2024-04-25T14:04:20.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.189+0000 7f4eaf400640 1 -- 172.21.15.86:0/1214146904 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e9c0ac410 con 0x7f4eb00d16e0 2024-04-25T14:04:20.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.189+0000 7f4eaf400640 1 -- 172.21.15.86:0/1214146904 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb00743e0 con 0x7f4eb00d16e0 2024-04-25T14:04:20.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.189+0000 7f4eb7800640 1 -- 172.21.15.86:0/1214146904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4eb00d16e0 msgr2=0x7f4eb00d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.189+0000 7f4eb7800640 1 --2- 172.21.15.86:0/1214146904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4eb00d16e0 0x7f4eb00d1af0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0097c0 tx=0x7f4e9c01c4a0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.189+0000 7f4eb7800640 1 -- 172.21.15.86:0/1214146904 shutdown_connections 2024-04-25T14:04:20.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.189+0000 7f4eb7800640 1 --2- 172.21.15.86:0/1214146904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4eb00d16e0 0x7f4eb00d1af0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.189+0000 7f4eb7800640 1 -- 172.21.15.86:0/1214146904 >> 172.21.15.86:0/1214146904 conn(0x7f4eb00b4650 msgr2=0x7f4eb00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:20.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.189+0000 7f4eb7800640 1 -- 172.21.15.86:0/1214146904 shutdown_connections 2024-04-25T14:04:20.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.189+0000 7f4eb7800640 1 -- 172.21.15.86:0/1214146904 wait complete. 2024-04-25T14:04:20.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.189+0000 7f4eb7800640 1 Processor -- start 2024-04-25T14:04:20.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.189+0000 7f4eb7800640 1 -- start start 2024-04-25T14:04:20.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.193+0000 7f4eb7800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb00730b0 0x7f4eb009aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.193+0000 7f4eb7800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4eb00cfca0 0x7f4eb00c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.193+0000 7f4eb7800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4eb00d16e0 0x7f4eb00ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.193+0000 7f4eb7800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb016aa80 con 0x7f4eb00730b0 2024-04-25T14:04:20.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.193+0000 7f4eb7800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb016a900 con 0x7f4eb00d16e0 2024-04-25T14:04:20.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.193+0000 7f4eb7800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4eb016ac00 con 0x7f4eb00cfca0 2024-04-25T14:04:20.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.193+0000 7f4eb4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb00730b0 0x7f4eb009aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.193+0000 7f4eb4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb00730b0 0x7f4eb009aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58724/0 (socket says 172.21.15.86:58724) 2024-04-25T14:04:20.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.193+0000 7f4eb4e00640 1 -- 172.21.15.86:0/2694967957 learned_addr learned my addr 172.21.15.86:0/2694967957 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:20.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.193+0000 7f4eafe00640 1 --2- 172.21.15.86:0/2694967957 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4eb00cfca0 0x7f4eb00c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.193+0000 7f4eb5800640 1 --2- 172.21.15.86:0/2694967957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4eb00d16e0 0x7f4eb00ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.193+0000 7f4eb4e00640 1 -- 172.21.15.86:0/2694967957 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4eb00cfca0 msgr2=0x7f4eb00c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.193+0000 7f4eb4e00640 1 --2- 172.21.15.86:0/2694967957 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4eb00cfca0 0x7f4eb00c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.193+0000 7f4eb4e00640 1 -- 172.21.15.86:0/2694967957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4eb00d16e0 msgr2=0x7f4eb00ae4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.193+0000 7f4eb4e00640 1 --2- 172.21.15.86:0/2694967957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4eb00d16e0 0x7f4eb00ae4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.193+0000 7f4eb4e00640 1 -- 172.21.15.86:0/2694967957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb00ca3b0 con 0x7f4eb00730b0 2024-04-25T14:04:20.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.193+0000 7f4eb4e00640 1 --2- 172.21.15.86:0/2694967957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb00730b0 0x7f4eb009aa50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4ea400c790 tx=0x7f4ea400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:20.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.193+0000 7f4eacc00640 1 -- 172.21.15.86:0/2694967957 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4017070 con 0x7f4eb00730b0 2024-04-25T14:04:20.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.193+0000 7f4eacc00640 1 -- 172.21.15.86:0/2694967957 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea4005ac0 con 0x7f4eb00730b0 2024-04-25T14:04:20.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.193+0000 7f4eacc00640 1 -- 172.21.15.86:0/2694967957 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4001d00 con 0x7f4eb00730b0 2024-04-25T14:04:20.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.193+0000 7f4eb7800640 1 -- 172.21.15.86:0/2694967957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb00ca670 con 0x7f4eb00730b0 2024-04-25T14:04:20.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.193+0000 7f4eb7800640 1 -- 172.21.15.86:0/2694967957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb00b0f40 con 0x7f4eb00730b0 2024-04-25T14:04:20.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.193+0000 7f4eb7800640 1 -- 172.21.15.86:0/2694967957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eb009f4b0 con 0x7f4eb00730b0 2024-04-25T14:04:20.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.197+0000 7f4eacc00640 1 -- 172.21.15.86:0/2694967957 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4ea4007070 con 0x7f4eb00730b0 2024-04-25T14:04:20.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.197+0000 7f4eacc00640 1 --2- 172.21.15.86:0/2694967957 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f4e9006f400 0x7f4e90071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.197+0000 7f4eacc00640 1 -- 172.21.15.86:0/2694967957 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4ea408e790 con 0x7f4eb00730b0 2024-04-25T14:04:20.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.197+0000 7f4eafe00640 1 --2- 172.21.15.86:0/2694967957 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f4e9006f400 0x7f4e90071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.197+0000 7f4eafe00640 1 --2- 172.21.15.86:0/2694967957 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f4e9006f400 0x7f4e90071910 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0ae000 tx=0x7f4e9c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:20.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.201+0000 7f4eacc00640 1 -- 172.21.15.86:0/2694967957 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4eb00d2030 con 0x7f4eb00730b0 2024-04-25T14:04:20.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.361+0000 7f4eb7800640 1 -- 172.21.15.86:0/2694967957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f4eb0069d00 con 0x7f4eb00730b0 2024-04-25T14:04:20.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.361+0000 7f4eacc00640 1 -- 172.21.15.86:0/2694967957 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f4ea405ace0 con 0x7f4eb00730b0 2024-04-25T14:04:20.362 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:04:20.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.361+0000 7f4eb7800640 1 -- 172.21.15.86:0/2694967957 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f4e9006f400 msgr2=0x7f4e90071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.361+0000 7f4eb7800640 1 --2- 172.21.15.86:0/2694967957 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f4e9006f400 0x7f4e90071910 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0ae000 tx=0x7f4e9c008040 comp rx=0 tx=0).stop 2024-04-25T14:04:20.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.361+0000 7f4eb7800640 1 -- 172.21.15.86:0/2694967957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb00730b0 msgr2=0x7f4eb009aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.361+0000 7f4eb7800640 1 --2- 172.21.15.86:0/2694967957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb00730b0 0x7f4eb009aa50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4ea400c790 tx=0x7f4ea400cc50 comp rx=0 tx=0).stop 2024-04-25T14:04:20.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.361+0000 7f4eb7800640 1 -- 172.21.15.86:0/2694967957 shutdown_connections 2024-04-25T14:04:20.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.361+0000 7f4eb7800640 1 --2- 172.21.15.86:0/2694967957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4eb00d16e0 0x7f4eb00ae4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.361+0000 7f4eb7800640 1 --2- 172.21.15.86:0/2694967957 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f4e9006f400 0x7f4e90071910 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.361+0000 7f4eb7800640 1 --2- 172.21.15.86:0/2694967957 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4eb00cfca0 0x7f4eb00c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.361+0000 7f4eb7800640 1 --2- 172.21.15.86:0/2694967957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb00730b0 0x7f4eb009aa50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.361+0000 7f4eb7800640 1 -- 172.21.15.86:0/2694967957 >> 172.21.15.86:0/2694967957 conn(0x7f4eb00b4650 msgr2=0x7f4eb00cdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:20.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.361+0000 7f4eb7800640 1 -- 172.21.15.86:0/2694967957 shutdown_connections 2024-04-25T14:04:20.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.361+0000 7f4eb7800640 1 -- 172.21.15.86:0/2694967957 wait complete. 2024-04-25T14:04:20.383 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:04:20.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.529+0000 7f5665600640 1 Processor -- start 2024-04-25T14:04:20.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.529+0000 7f5665600640 1 -- start start 2024-04-25T14:04:20.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.529+0000 7f5665600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f56600cfca0 0x7f566006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.529+0000 7f5665600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56600d16e0 0x7f56600d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.533+0000 7f5665600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56600d2030 con 0x7f566006c670 2024-04-25T14:04:20.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.533+0000 7f5665600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f5660070470 con 0x7f56600cfca0 2024-04-25T14:04:20.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.533+0000 7f5665600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5660072bd0 con 0x7f56600d16e0 2024-04-25T14:04:20.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.533+0000 7f5656000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56600d16e0 0x7f56600d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.533+0000 7f565ea00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f56600cfca0 0x7f566006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.533+0000 7f565f400640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f566006c670 0x7f566006ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:52784/0 (socket says 172.21.15.86:52784) 2024-04-25T14:04:20.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.533+0000 7f565f400640 1 -- 172.21.15.86:0/1729968204 learned_addr learned my addr 172.21.15.86:0/1729968204 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:20.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.533+0000 7f565ea00640 1 -- 172.21.15.86:0/1729968204 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56600d16e0 msgr2=0x7f56600d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:20.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.533+0000 7f565ea00640 1 --2- 172.21.15.86:0/1729968204 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56600d16e0 0x7f56600d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.533+0000 7f565ea00640 1 -- 172.21.15.86:0/1729968204 >> v1:172.21.15.155:6789/0 conn(0x7f566006c670 legacy=0x7f566006ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.533+0000 7f565ea00640 1 -- 172.21.15.86:0/1729968204 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566006f930 con 0x7f56600cfca0 2024-04-25T14:04:20.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.537+0000 7f565ea00640 1 --2- 172.21.15.86:0/1729968204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56600cfca0 0x7f566006f3f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f56500097c0 tx=0x7f565001c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=817e23e6fad7e625 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:20.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.537+0000 7f565e000640 1 -- 172.21.15.86:0/1729968204 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56500b0070 con 0x7f56600cfca0 2024-04-25T14:04:20.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.537+0000 7f565e000640 1 -- 172.21.15.86:0/1729968204 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56500ab600 con 0x7f56600cfca0 2024-04-25T14:04:20.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.537+0000 7f565e000640 1 -- 172.21.15.86:0/1729968204 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56500ab9d0 con 0x7f56600cfca0 2024-04-25T14:04:20.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.537+0000 7f5665600640 1 -- 172.21.15.86:0/1729968204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56600cfca0 msgr2=0x7f566006f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.537+0000 7f5665600640 1 --2- 172.21.15.86:0/1729968204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56600cfca0 0x7f566006f3f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f56500097c0 tx=0x7f565001c330 comp rx=0 tx=0).stop 2024-04-25T14:04:20.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.537+0000 7f5665600640 1 -- 172.21.15.86:0/1729968204 shutdown_connections 2024-04-25T14:04:20.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.537+0000 7f5665600640 1 --2- 172.21.15.86:0/1729968204 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56600d16e0 0x7f56600d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.537+0000 7f5665600640 1 --2- 172.21.15.86:0/1729968204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56600cfca0 0x7f566006f3f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.537+0000 7f5665600640 1 -- 172.21.15.86:0/1729968204 >> 172.21.15.86:0/1729968204 conn(0x7f56600b4650 msgr2=0x7f56600b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:20.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.537+0000 7f5665600640 1 -- 172.21.15.86:0/1729968204 shutdown_connections 2024-04-25T14:04:20.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.537+0000 7f5665600640 1 -- 172.21.15.86:0/1729968204 wait complete. 2024-04-25T14:04:20.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.537+0000 7f5665600640 1 Processor -- start 2024-04-25T14:04:20.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.537+0000 7f5665600640 1 -- start start 2024-04-25T14:04:20.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.541+0000 7f5665600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f566006c670 0x7f566009ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.541+0000 7f5665600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56600cfca0 0x7f56601f7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.541+0000 7f5665600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56600d16e0 0x7f566009b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.541+0000 7f5665600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5660070190 con 0x7f56600d16e0 2024-04-25T14:04:20.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.541+0000 7f5665600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f566006feb0 con 0x7f566006c670 2024-04-25T14:04:20.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.541+0000 7f5665600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f56600740c0 con 0x7f56600cfca0 2024-04-25T14:04:20.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.541+0000 7f565f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56600d16e0 0x7f566009b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.541+0000 7f565f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56600d16e0 0x7f566009b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58736/0 (socket says 172.21.15.86:58736) 2024-04-25T14:04:20.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.541+0000 7f565ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f566006c670 0x7f566009ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.541+0000 7f565f400640 1 -- 172.21.15.86:0/1892675516 learned_addr learned my addr 172.21.15.86:0/1892675516 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:20.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.541+0000 7f5656000640 1 --2- 172.21.15.86:0/1892675516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56600cfca0 0x7f56601f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.541+0000 7f565f400640 1 -- 172.21.15.86:0/1892675516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56600cfca0 msgr2=0x7f56601f7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.541+0000 7f565f400640 1 --2- 172.21.15.86:0/1892675516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56600cfca0 0x7f56601f7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.541+0000 7f565f400640 1 -- 172.21.15.86:0/1892675516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f566006c670 msgr2=0x7f566009ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.541+0000 7f565f400640 1 --2- 172.21.15.86:0/1892675516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f566006c670 0x7f566009ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.541+0000 7f565f400640 1 -- 172.21.15.86:0/1892675516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566015bac0 con 0x7f56600d16e0 2024-04-25T14:04:20.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.541+0000 7f565f400640 1 --2- 172.21.15.86:0/1892675516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56600d16e0 0x7f566009b2b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f565800d110 tx=0x7f565800d5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:20.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.541+0000 7f5657400640 1 -- 172.21.15.86:0/1892675516 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565801b070 con 0x7f56600d16e0 2024-04-25T14:04:20.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.541+0000 7f5657400640 1 -- 172.21.15.86:0/1892675516 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f565800a2d0 con 0x7f56600d16e0 2024-04-25T14:04:20.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.541+0000 7f5665600640 1 -- 172.21.15.86:0/1892675516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f566015bda0 con 0x7f56600d16e0 2024-04-25T14:04:20.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.541+0000 7f5657400640 1 -- 172.21.15.86:0/1892675516 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565800a700 con 0x7f56600d16e0 2024-04-25T14:04:20.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.541+0000 7f5665600640 1 -- 172.21.15.86:0/1892675516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56601f7100 con 0x7f56600d16e0 2024-04-25T14:04:20.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.541+0000 7f5665600640 1 -- 172.21.15.86:0/1892675516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f566009f4b0 con 0x7f56600d16e0 2024-04-25T14:04:20.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.545+0000 7f5657400640 1 -- 172.21.15.86:0/1892675516 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f565800a8a0 con 0x7f56600d16e0 2024-04-25T14:04:20.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.545+0000 7f5657400640 1 --2- 172.21.15.86:0/1892675516 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f563c06f430 0x7f563c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.545+0000 7f5657400640 1 -- 172.21.15.86:0/1892675516 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f56580926e0 con 0x7f56600d16e0 2024-04-25T14:04:20.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.545+0000 7f565ea00640 1 --2- 172.21.15.86:0/1892675516 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f563c06f430 0x7f563c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.545+0000 7f565ea00640 1 --2- 172.21.15.86:0/1892675516 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f563c06f430 0x7f563c071940 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f56500ad000 tx=0x7f5650008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:20.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.549+0000 7f5657400640 1 -- 172.21.15.86:0/1892675516 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f56600d2030 con 0x7f56600d16e0 2024-04-25T14:04:20.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.741+0000 7f5665600640 1 -- 172.21.15.86:0/1892675516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5660069d00 con 0x7f56600d16e0 2024-04-25T14:04:20.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.741+0000 7f5657400640 1 -- 172.21.15.86:0/1892675516 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5177 (secure 0 0 0) 0x7f565805ec70 con 0x7f56600d16e0 2024-04-25T14:04:20.743 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:20.743 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/347107112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":347107112},{"type":"v1","addr":"172.21.15.155:6835","nonce":347107112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2236737549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2236737549},{"type":"v1","addr":"172.21.15.86:6837","nonce":2236737549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/519560229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":519560229},{"type":"v1","addr":"172.21.15.86:6836","nonce":519560229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:03:50.852257+0000","modified":"2024-04-25T14:04:09.202137+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6833/1944249570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1944249570},{"type":"v1","addr":"172.21.15.155:6833","nonce":1944249570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":1}]} 2024-04-25T14:04:20.743 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 9 2024-04-25T14:04:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.741+0000 7f5665600640 1 -- 172.21.15.86:0/1892675516 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f563c06f430 msgr2=0x7f563c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.741+0000 7f5665600640 1 --2- 172.21.15.86:0/1892675516 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f563c06f430 0x7f563c071940 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f56500ad000 tx=0x7f5650008040 comp rx=0 tx=0).stop 2024-04-25T14:04:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.741+0000 7f5665600640 1 -- 172.21.15.86:0/1892675516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56600d16e0 msgr2=0x7f566009b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.741+0000 7f5665600640 1 --2- 172.21.15.86:0/1892675516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56600d16e0 0x7f566009b2b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f565800d110 tx=0x7f565800d5d0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.745+0000 7f5665600640 1 -- 172.21.15.86:0/1892675516 shutdown_connections 2024-04-25T14:04:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.745+0000 7f5665600640 1 --2- 172.21.15.86:0/1892675516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56600d16e0 0x7f566009b2b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.745+0000 7f5665600640 1 --2- 172.21.15.86:0/1892675516 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f563c06f430 0x7f563c071940 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.745+0000 7f5665600640 1 --2- 172.21.15.86:0/1892675516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56600cfca0 0x7f56601f7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.745+0000 7f5665600640 1 --2- 172.21.15.86:0/1892675516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f566006c670 0x7f566009ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.745+0000 7f5665600640 1 -- 172.21.15.86:0/1892675516 >> 172.21.15.86:0/1892675516 conn(0x7f56600b4650 msgr2=0x7f56600b2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.745+0000 7f5665600640 1 -- 172.21.15.86:0/1892675516 shutdown_connections 2024-04-25T14:04:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.745+0000 7f5665600640 1 -- 172.21.15.86:0/1892675516 wait complete. 2024-04-25T14:04:20.765 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:04:20.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.909+0000 7fed1b000640 1 Processor -- start 2024-04-25T14:04:20.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.909+0000 7fed1b000640 1 -- start start 2024-04-25T14:04:20.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.913+0000 7fed1b000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fed140d16e0 0x7fed140d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.913+0000 7fed1b000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed140d2030 con 0x7fed140cfca0 2024-04-25T14:04:20.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.913+0000 7fed1b000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed14072bd0 con 0x7fed140730b0 2024-04-25T14:04:20.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.913+0000 7fed1b000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fed14070470 con 0x7fed140d16e0 2024-04-25T14:04:20.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.913+0000 7fed0b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fed140d16e0 0x7fed140d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.913+0000 7fed0b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fed140d16e0 0x7fed140d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58746/0 (socket says 172.21.15.86:58746) 2024-04-25T14:04:20.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.913+0000 7fed0b400640 1 -- 172.21.15.86:0/3200942561 learned_addr learned my addr 172.21.15.86:0/3200942561 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:20.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.913+0000 7fed0b400640 1 -- 172.21.15.86:0/3200942561 >> v1:172.21.15.155:6789/0 conn(0x7fed140730b0 legacy=0x7fed140734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.913+0000 7fed0b400640 1 -- 172.21.15.86:0/3200942561 >> v1:172.21.15.86:6789/0 conn(0x7fed140cfca0 legacy=0x7fed1406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.913+0000 7fed0b400640 1 -- 172.21.15.86:0/3200942561 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1406fbd0 con 0x7fed140d16e0 2024-04-25T14:04:20.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.913+0000 7fed13400640 1 -- 172.21.15.86:0/3200942561 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 45148589 0 0) 0x7fed04001120 con 0x7fed140cfca0 2024-04-25T14:04:20.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.917+0000 7fed0b400640 1 --2- 172.21.15.86:0/3200942561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed140d16e0 0x7fed140d1af0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fecfc009870 tx=0x7fecfc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec439d06e4e9bd82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:20.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.917+0000 7fed13400640 1 -- 172.21.15.86:0/3200942561 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc0a7070 con 0x7fed140d16e0 2024-04-25T14:04:20.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.917+0000 7fed13400640 1 -- 172.21.15.86:0/3200942561 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecfc0b0070 con 0x7fed140d16e0 2024-04-25T14:04:20.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.917+0000 7fed13400640 1 -- 172.21.15.86:0/3200942561 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed140743e0 con 0x7fed140d16e0 2024-04-25T14:04:20.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.917+0000 7fed1b000640 1 -- 172.21.15.86:0/3200942561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed140d16e0 msgr2=0x7fed140d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.917+0000 7fed1b000640 1 --2- 172.21.15.86:0/3200942561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed140d16e0 0x7fed140d1af0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fecfc009870 tx=0x7fecfc01c560 comp rx=0 tx=0).stop 2024-04-25T14:04:20.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.917+0000 7fed1b000640 1 -- 172.21.15.86:0/3200942561 shutdown_connections 2024-04-25T14:04:20.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.917+0000 7fed1b000640 1 --2- 172.21.15.86:0/3200942561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed140d16e0 0x7fed140d1af0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.917+0000 7fed1b000640 1 -- 172.21.15.86:0/3200942561 >> 172.21.15.86:0/3200942561 conn(0x7fed140b4650 msgr2=0x7fed140b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:20.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.917+0000 7fed1b000640 1 -- 172.21.15.86:0/3200942561 shutdown_connections 2024-04-25T14:04:20.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.917+0000 7fed1b000640 1 -- 172.21.15.86:0/3200942561 wait complete. 2024-04-25T14:04:20.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.921+0000 7fed1b000640 1 Processor -- start 2024-04-25T14:04:20.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.921+0000 7fed1b000640 1 -- start start 2024-04-25T14:04:20.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.921+0000 7fed1b000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed140730b0 0x7fed140b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.921+0000 7fed1b000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed140cfca0 0x7fed140c33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.921+0000 7fed1b000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed140d16e0 0x7fed140999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.921+0000 7fed1b000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed1416aa80 con 0x7fed140730b0 2024-04-25T14:04:20.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.921+0000 7fed1b000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fed1416a900 con 0x7fed140cfca0 2024-04-25T14:04:20.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.921+0000 7fed1b000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fed1416ac00 con 0x7fed140d16e0 2024-04-25T14:04:20.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.921+0000 7fed13e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed140730b0 0x7fed140b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.921+0000 7fed13e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed140730b0 0x7fed140b1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58752/0 (socket says 172.21.15.86:58752) 2024-04-25T14:04:20.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.921+0000 7fed13e00640 1 -- 172.21.15.86:0/1590645974 learned_addr learned my addr 172.21.15.86:0/1590645974 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:20.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.921+0000 7fed0b400640 1 --2- 172.21.15.86:0/1590645974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed140cfca0 0x7fed140c33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.921+0000 7fed19000640 1 --2- 172.21.15.86:0/1590645974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed140d16e0 0x7fed140999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.921+0000 7fed13e00640 1 -- 172.21.15.86:0/1590645974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed140d16e0 msgr2=0x7fed140999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.921+0000 7fed13e00640 1 --2- 172.21.15.86:0/1590645974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed140d16e0 0x7fed140999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.921+0000 7fed13e00640 1 -- 172.21.15.86:0/1590645974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed140cfca0 msgr2=0x7fed140c33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.921+0000 7fed13e00640 1 --2- 172.21.15.86:0/1590645974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed140cfca0 0x7fed140c33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.921+0000 7fed13e00640 1 -- 172.21.15.86:0/1590645974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed14154de0 con 0x7fed140730b0 2024-04-25T14:04:20.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.921+0000 7fed13e00640 1 --2- 172.21.15.86:0/1590645974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed140730b0 0x7fed140b1f10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fed0400c410 tx=0x7fed0400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:20.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.921+0000 7fed10c00640 1 -- 172.21.15.86:0/1590645974 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed04017070 con 0x7fed140730b0 2024-04-25T14:04:20.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.921+0000 7fed10c00640 1 -- 172.21.15.86:0/1590645974 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed040052d0 con 0x7fed140730b0 2024-04-25T14:04:20.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.921+0000 7fed10c00640 1 -- 172.21.15.86:0/1590645974 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed04003860 con 0x7fed140730b0 2024-04-25T14:04:20.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.921+0000 7fed1b000640 1 -- 172.21.15.86:0/1590645974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed1415a6e0 con 0x7fed140730b0 2024-04-25T14:04:20.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.921+0000 7fed1b000640 1 -- 172.21.15.86:0/1590645974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed1415b1c0 con 0x7fed140730b0 2024-04-25T14:04:20.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.925+0000 7fed1b000640 1 -- 172.21.15.86:0/1590645974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed1409f4b0 con 0x7fed140730b0 2024-04-25T14:04:20.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.925+0000 7fed10c00640 1 -- 172.21.15.86:0/1590645974 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fed0401a070 con 0x7fed140730b0 2024-04-25T14:04:20.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.925+0000 7fed10c00640 1 --2- 172.21.15.86:0/1590645974 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fecf006f430 0x7fecf0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.925+0000 7fed10c00640 1 -- 172.21.15.86:0/1590645974 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fed0408e7d0 con 0x7fed140730b0 2024-04-25T14:04:20.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.925+0000 7fed0b400640 1 --2- 172.21.15.86:0/1590645974 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fecf006f430 0x7fecf0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.925+0000 7fed0b400640 1 --2- 172.21.15.86:0/1590645974 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fecf006f430 0x7fecf0071940 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fecfc0af000 tx=0x7fecfc0a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:20.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:20.929+0000 7fed10c00640 1 -- 172.21.15.86:0/1590645974 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fed140d2030 con 0x7fed140730b0 2024-04-25T14:04:21.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.085+0000 7fed1b000640 1 -- 172.21.15.86:0/1590645974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed14069d00 con 0x7fed140730b0 2024-04-25T14:04:21.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.089+0000 7fed10c00640 1 -- 172.21.15.86:0/1590645974 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18514 (secure 0 0 0) 0x7fed04005480 con 0x7fed140730b0 2024-04-25T14:04:21.090 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:21.090 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":24,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:03:55.899092+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:03:49.151473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:03:49.802597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T14:03:53.492008+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:04:21.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.089+0000 7fed1b000640 1 -- 172.21.15.86:0/1590645974 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fecf006f430 msgr2=0x7fecf0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:21.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.089+0000 7fed1b000640 1 --2- 172.21.15.86:0/1590645974 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fecf006f430 0x7fecf0071940 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fecfc0af000 tx=0x7fecfc0a7000 comp rx=0 tx=0).stop 2024-04-25T14:04:21.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.089+0000 7fed1b000640 1 -- 172.21.15.86:0/1590645974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed140730b0 msgr2=0x7fed140b1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:21.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.089+0000 7fed1b000640 1 --2- 172.21.15.86:0/1590645974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed140730b0 0x7fed140b1f10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fed0400c410 tx=0x7fed0400c8d0 comp rx=0 tx=0).stop 2024-04-25T14:04:21.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.089+0000 7fed1b000640 1 -- 172.21.15.86:0/1590645974 shutdown_connections 2024-04-25T14:04:21.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.089+0000 7fed1b000640 1 --2- 172.21.15.86:0/1590645974 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fecf006f430 0x7fecf0071940 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:21.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.089+0000 7fed1b000640 1 --2- 172.21.15.86:0/1590645974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed140d16e0 0x7fed140999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:21.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.089+0000 7fed1b000640 1 --2- 172.21.15.86:0/1590645974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed140cfca0 0x7fed140c33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:21.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.089+0000 7fed1b000640 1 --2- 172.21.15.86:0/1590645974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed140730b0 0x7fed140b1f10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:21.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.089+0000 7fed1b000640 1 -- 172.21.15.86:0/1590645974 >> 172.21.15.86:0/1590645974 conn(0x7fed140b4650 msgr2=0x7fed14155d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:21.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.089+0000 7fed1b000640 1 -- 172.21.15.86:0/1590645974 shutdown_connections 2024-04-25T14:04:21.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.089+0000 7fed1b000640 1 -- 172.21.15.86:0/1590645974 wait complete. 2024-04-25T14:04:21.113 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:04:21.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.257+0000 7f6692a00640 1 Processor -- start 2024-04-25T14:04:21.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.257+0000 7f6692a00640 1 -- start start 2024-04-25T14:04:21.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f6692a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f668c0b59e0 0x7f668c0b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:21.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f6692a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f668c0b6330 con 0x7f668c0aeea0 2024-04-25T14:04:21.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f6692a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f668c06c670 con 0x7f668c15faa0 2024-04-25T14:04:21.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f6692a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f668c06d560 con 0x7f668c0b59e0 2024-04-25T14:04:21.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f6690a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f668c0aeea0 0x7f668c0af2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:38858/0 (socket says 172.21.15.86:38858) 2024-04-25T14:04:21.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f6690a00640 1 -- 172.21.15.86:0/1782951106 learned_addr learned my addr 172.21.15.86:0/1782951106 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:21.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f6683400640 1 --2- 172.21.15.86:0/1782951106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f668c0b59e0 0x7f668c0b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:21.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f668b400640 1 -- 172.21.15.86:0/1782951106 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2140554312 0 0) 0x7f66840014a0 con 0x7f668c0aeea0 2024-04-25T14:04:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f668b400640 1 -- 172.21.15.86:0/1782951106 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6670003530 con 0x7f668c0aeea0 2024-04-25T14:04:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f668b400640 1 -- 172.21.15.86:0/1782951106 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2240608637 0 0) 0x7f6670003530 con 0x7f668c0aeea0 2024-04-25T14:04:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f668b400640 1 -- 172.21.15.86:0/1782951106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f668c0b59e0 msgr2=0x7f668c0b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f668b400640 1 --2- 172.21.15.86:0/1782951106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f668c0b59e0 0x7f668c0b5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f668b400640 1 -- 172.21.15.86:0/1782951106 >> v1:172.21.15.155:6789/0 conn(0x7f668c15faa0 legacy=0x7f668c06d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f668b400640 1 -- 172.21.15.86:0/1782951106 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f668c06c950 con 0x7f668c0aeea0 2024-04-25T14:04:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f668b400640 1 -- 172.21.15.86:0/1782951106 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f6684001bd0 con 0x7f668c0aeea0 2024-04-25T14:04:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f668b400640 1 -- 172.21.15.86:0/1782951106 >> v1:172.21.15.86:6789/0 conn(0x7f668c0aeea0 legacy=0x7f668c0af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f668b400640 1 --2- 172.21.15.86:0/1782951106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6670003a60 0x7f6670003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f668b400640 1 --2- 172.21.15.86:0/1782951106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6670004810 0x7f6670004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f668b400640 1 --2- 172.21.15.86:0/1782951106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f668c0b59e0 0x7f6670005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f668b400640 1 -- 172.21.15.86:0/1782951106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f668c06d560 con 0x7f6670004810 2024-04-25T14:04:21.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f668b400640 1 -- 172.21.15.86:0/1782951106 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f667000bae0 con 0x7f6670003a60 2024-04-25T14:04:21.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f668b400640 1 -- 172.21.15.86:0/1782951106 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f667000bce0 con 0x7f668c0b59e0 2024-04-25T14:04:21.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f6683400640 1 --2- 172.21.15.86:0/1782951106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6670004810 0x7f6670004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:21.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f668be00640 1 --2- 172.21.15.86:0/1782951106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6670003a60 0x7f6670003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:21.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f6690a00640 1 --2- 172.21.15.86:0/1782951106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f668c0b59e0 0x7f6670005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:21.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f6683400640 1 -- 172.21.15.86:0/1782951106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f668c0b59e0 msgr2=0x7f6670005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:21.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f6683400640 1 --2- 172.21.15.86:0/1782951106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f668c0b59e0 0x7f6670005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:21.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f6683400640 1 -- 172.21.15.86:0/1782951106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6670003a60 msgr2=0x7f6670003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:21.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f6683400640 1 --2- 172.21.15.86:0/1782951106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6670003a60 0x7f6670003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:21.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.261+0000 7f6683400640 1 -- 172.21.15.86:0/1782951106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6670003530 con 0x7f6670004810 2024-04-25T14:04:21.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.265+0000 7f6683400640 1 --2- 172.21.15.86:0/1782951106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6670004810 0x7f6670004c20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6674009ae0 tx=0x7f667401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d14e3ad58c37d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:21.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.265+0000 7f668b400640 1 -- 172.21.15.86:0/1782951106 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66740b2070 con 0x7f6670004810 2024-04-25T14:04:21.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.265+0000 7f668b400640 1 -- 172.21.15.86:0/1782951106 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66740a5680 con 0x7f6670004810 2024-04-25T14:04:21.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.265+0000 7f668b400640 1 -- 172.21.15.86:0/1782951106 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66740096b0 con 0x7f6670004810 2024-04-25T14:04:21.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.265+0000 7f6692a00640 1 -- 172.21.15.86:0/1782951106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6670004810 msgr2=0x7f6670004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:21.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.265+0000 7f6692a00640 1 --2- 172.21.15.86:0/1782951106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6670004810 0x7f6670004c20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6674009ae0 tx=0x7f667401ca90 comp rx=0 tx=0).stop 2024-04-25T14:04:21.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.265+0000 7f6690a00640 1 -- 172.21.15.86:0/1782951106 reap_dead start 2024-04-25T14:04:21.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.265+0000 7f6692a00640 1 -- 172.21.15.86:0/1782951106 shutdown_connections 2024-04-25T14:04:21.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.265+0000 7f6692a00640 1 -- 172.21.15.86:0/1782951106 >> 172.21.15.86:0/1782951106 conn(0x7f668c0d2230 msgr2=0x7f668c0d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:21.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.265+0000 7f6692a00640 1 -- 172.21.15.86:0/1782951106 shutdown_connections 2024-04-25T14:04:21.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.265+0000 7f6692a00640 1 -- 172.21.15.86:0/1782951106 wait complete. 2024-04-25T14:04:21.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.269+0000 7f6692a00640 1 Processor -- start 2024-04-25T14:04:21.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.269+0000 7f6692a00640 1 -- start start 2024-04-25T14:04:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.269+0000 7f6692a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6670004810 0x7f668c0b1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.269+0000 7f6692a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f668c149fa0 0x7f668c156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.269+0000 7f6692a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f668c1556f0 0x7f668c155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.269+0000 7f6692a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f668c06d1f0 con 0x7f668c149fa0 2024-04-25T14:04:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.269+0000 7f6692a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f668c06cf10 con 0x7f668c1556f0 2024-04-25T14:04:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.269+0000 7f6692a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f668c0af9c0 con 0x7f6670004810 2024-04-25T14:04:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.269+0000 7f6683400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f668c149fa0 0x7f668c156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.269+0000 7f6683400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f668c149fa0 0x7f668c156ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58770/0 (socket says 172.21.15.86:58770) 2024-04-25T14:04:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.269+0000 7f6683400640 1 -- 172.21.15.86:0/2984329149 learned_addr learned my addr 172.21.15.86:0/2984329149 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.269+0000 7f668be00640 1 --2- 172.21.15.86:0/2984329149 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6670004810 0x7f668c0b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.269+0000 7f6690a00640 1 --2- 172.21.15.86:0/2984329149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f668c1556f0 0x7f668c155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.269+0000 7f6683400640 1 -- 172.21.15.86:0/2984329149 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6670004810 msgr2=0x7f668c0b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.269+0000 7f6683400640 1 --2- 172.21.15.86:0/2984329149 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6670004810 0x7f668c0b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.269+0000 7f6683400640 1 -- 172.21.15.86:0/2984329149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f668c1556f0 msgr2=0x7f668c155b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.269+0000 7f6683400640 1 --2- 172.21.15.86:0/2984329149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f668c1556f0 0x7f668c155b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.269+0000 7f6683400640 1 -- 172.21.15.86:0/2984329149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f668c156040 con 0x7f668c149fa0 2024-04-25T14:04:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.269+0000 7f6683400640 1 --2- 172.21.15.86:0/2984329149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f668c149fa0 0x7f668c156ff0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6674009ae0 tx=0x7f6670005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:21.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.269+0000 7f6688c00640 1 -- 172.21.15.86:0/2984329149 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66740b2040 con 0x7f668c149fa0 2024-04-25T14:04:21.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.269+0000 7f6688c00640 1 -- 172.21.15.86:0/2984329149 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f667400f000 con 0x7f668c149fa0 2024-04-25T14:04:21.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.269+0000 7f6692a00640 1 -- 172.21.15.86:0/2984329149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f668c15a6d0 con 0x7f668c149fa0 2024-04-25T14:04:21.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.269+0000 7f6688c00640 1 -- 172.21.15.86:0/2984329149 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6674004240 con 0x7f668c149fa0 2024-04-25T14:04:21.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.269+0000 7f6692a00640 1 -- 172.21.15.86:0/2984329149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f668c0b1190 con 0x7f668c149fa0 2024-04-25T14:04:21.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.273+0000 7f6692a00640 1 -- 172.21.15.86:0/2984329149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f668c0b0ce0 con 0x7f668c149fa0 2024-04-25T14:04:21.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.273+0000 7f6688c00640 1 -- 172.21.15.86:0/2984329149 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f66740c0030 con 0x7f668c149fa0 2024-04-25T14:04:21.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.273+0000 7f6688c00640 1 --2- 172.21.15.86:0/2984329149 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f666806f430 0x7f6668071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:21.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.273+0000 7f6688c00640 1 -- 172.21.15.86:0/2984329149 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6674129620 con 0x7f668c149fa0 2024-04-25T14:04:21.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.273+0000 7f668be00640 1 --2- 172.21.15.86:0/2984329149 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f666806f430 0x7f6668071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.273+0000 7f668be00640 1 --2- 172.21.15.86:0/2984329149 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f666806f430 0x7f6668071940 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f667c007fd0 tx=0x7f667c007cd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:21.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.277+0000 7f6688c00640 1 -- 172.21.15.86:0/2984329149 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f66740c7060 con 0x7f668c149fa0 2024-04-25T14:04:21.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.469+0000 7f6692a00640 1 -- 172.21.15.86:0/2984329149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f668c074cf0 con 0x7f668c149fa0 2024-04-25T14:04:21.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.469+0000 7f6688c00640 1 -- 172.21.15.86:0/2984329149 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5177 (secure 0 0 0) 0x7f66740167a0 con 0x7f668c149fa0 2024-04-25T14:04:21.470 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:21.471 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/347107112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":347107112},{"type":"v1","addr":"172.21.15.155:6835","nonce":347107112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2236737549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2236737549},{"type":"v1","addr":"172.21.15.86:6837","nonce":2236737549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/519560229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":519560229},{"type":"v1","addr":"172.21.15.86:6836","nonce":519560229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:03:50.852257+0000","modified":"2024-04-25T14:04:09.202137+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6833/1944249570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1944249570},{"type":"v1","addr":"172.21.15.155:6833","nonce":1944249570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":1}]} 2024-04-25T14:04:21.471 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 9 2024-04-25T14:04:21.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.469+0000 7f6692a00640 1 -- 172.21.15.86:0/2984329149 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f666806f430 msgr2=0x7f6668071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:21.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.469+0000 7f6692a00640 1 --2- 172.21.15.86:0/2984329149 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f666806f430 0x7f6668071940 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f667c007fd0 tx=0x7f667c007cd0 comp rx=0 tx=0).stop 2024-04-25T14:04:21.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.469+0000 7f6692a00640 1 -- 172.21.15.86:0/2984329149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f668c149fa0 msgr2=0x7f668c156ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:21.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.469+0000 7f6692a00640 1 --2- 172.21.15.86:0/2984329149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f668c149fa0 0x7f668c156ff0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6674009ae0 tx=0x7f6670005840 comp rx=0 tx=0).stop 2024-04-25T14:04:21.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.469+0000 7f6692a00640 1 -- 172.21.15.86:0/2984329149 shutdown_connections 2024-04-25T14:04:21.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.469+0000 7f6692a00640 1 --2- 172.21.15.86:0/2984329149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f668c1556f0 0x7f668c155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:21.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.469+0000 7f6692a00640 1 --2- 172.21.15.86:0/2984329149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f668c149fa0 0x7f668c156ff0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:21.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.469+0000 7f6692a00640 1 --2- 172.21.15.86:0/2984329149 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f666806f430 0x7f6668071940 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:21.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.469+0000 7f6692a00640 1 --2- 172.21.15.86:0/2984329149 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6670004810 0x7f668c0b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:21.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.469+0000 7f6692a00640 1 -- 172.21.15.86:0/2984329149 >> 172.21.15.86:0/2984329149 conn(0x7f668c0d2230 msgr2=0x7f668c15e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:21.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.469+0000 7f6692a00640 1 -- 172.21.15.86:0/2984329149 shutdown_connections 2024-04-25T14:04:21.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:21.469+0000 7f6692a00640 1 -- 172.21.15.86:0/2984329149 wait complete. 2024-04-25T14:04:21.894 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:03:50.852257+0000', 'modified': '2024-04-25T14:04:09.202137+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4275}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4275': {'gid': 4275, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.155:6833/1944249570', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6832', 'nonce': 1944249570}, {'type': 'v1', 'addr': '172.21.15.155:6833', 'nonce': 1944249570}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4275, 'qdb_cluster': [4275]} 2024-04-25T14:04:21.895 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T14:04:21.895 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T14:04:22.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.041+0000 7faf4a800640 1 Processor -- start 2024-04-25T14:04:22.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.041+0000 7faf4a800640 1 -- start start 2024-04-25T14:04:22.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.045+0000 7faf4a800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7faf4406d780 0x7faf44160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:22.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.045+0000 7faf4a800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf4409be50 con 0x7faf4406db90 2024-04-25T14:04:22.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.045+0000 7faf4a800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf440d3220 con 0x7faf440d17c0 2024-04-25T14:04:22.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.045+0000 7faf4a800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7faf440b1c40 con 0x7faf4406d780 2024-04-25T14:04:22.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.045+0000 7faf48a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7faf4406d780 0x7faf44160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:22.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.045+0000 7faf49e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7faf4406db90 0x7faf44167a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:38872/0 (socket says 172.21.15.86:38872) 2024-04-25T14:04:22.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.045+0000 7faf48a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7faf4406d780 0x7faf44160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58786/0 (socket says 172.21.15.86:58786) 2024-04-25T14:04:22.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.045+0000 7faf49e00640 1 -- 172.21.15.86:0/2234197877 learned_addr learned my addr 172.21.15.86:0/2234197877 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:22.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.045+0000 7faf48a00640 1 -- 172.21.15.86:0/2234197877 >> v1:172.21.15.155:6789/0 conn(0x7faf440d17c0 legacy=0x7faf440d2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:22.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.045+0000 7faf48a00640 1 -- 172.21.15.86:0/2234197877 >> v1:172.21.15.86:6789/0 conn(0x7faf4406db90 legacy=0x7faf44167a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:22.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.045+0000 7faf48a00640 1 -- 172.21.15.86:0/2234197877 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf440b1680 con 0x7faf4406d780 2024-04-25T14:04:22.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.045+0000 7faf43e00640 1 -- 172.21.15.86:0/2234197877 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2116954676 0 0) 0x7faf3c0014a0 con 0x7faf4406db90 2024-04-25T14:04:22.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.049+0000 7faf48a00640 1 --2- 172.21.15.86:0/2234197877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf4406d780 0x7faf44160690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7faf3000ba60 tx=0x7faf3001eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba154994af0d51e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:22.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.049+0000 7faf43e00640 1 -- 172.21.15.86:0/2234197877 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf300b4070 con 0x7faf4406d780 2024-04-25T14:04:22.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.049+0000 7faf43e00640 1 -- 172.21.15.86:0/2234197877 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf300a77a0 con 0x7faf4406d780 2024-04-25T14:04:22.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.049+0000 7faf43e00640 1 -- 172.21.15.86:0/2234197877 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf300a7b70 con 0x7faf4406d780 2024-04-25T14:04:22.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.049+0000 7faf4a800640 1 -- 172.21.15.86:0/2234197877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf4406d780 msgr2=0x7faf44160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:22.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.049+0000 7faf4a800640 1 --2- 172.21.15.86:0/2234197877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf4406d780 0x7faf44160690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7faf3000ba60 tx=0x7faf3001eae0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.049+0000 7faf4a800640 1 -- 172.21.15.86:0/2234197877 shutdown_connections 2024-04-25T14:04:22.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.049+0000 7faf4a800640 1 --2- 172.21.15.86:0/2234197877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf4406d780 0x7faf44160690 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.049+0000 7faf4a800640 1 -- 172.21.15.86:0/2234197877 >> 172.21.15.86:0/2234197877 conn(0x7faf44073c00 msgr2=0x7faf440b77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:22.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.049+0000 7faf4a800640 1 -- 172.21.15.86:0/2234197877 shutdown_connections 2024-04-25T14:04:22.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.049+0000 7faf4a800640 1 -- 172.21.15.86:0/2234197877 wait complete. 2024-04-25T14:04:22.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.049+0000 7faf4a800640 1 Processor -- start 2024-04-25T14:04:22.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.049+0000 7faf4a800640 1 -- start start 2024-04-25T14:04:22.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.053+0000 7faf4a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf4406d780 0x7faf4414e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:22.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.053+0000 7faf4a800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf4406db90 0x7faf440b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:22.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.053+0000 7faf4a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf440d17c0 0x7faf440b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:22.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.053+0000 7faf4a800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faf440d1bd0 con 0x7faf4406d780 2024-04-25T14:04:22.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.053+0000 7faf4a800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faf44160fe0 con 0x7faf440d17c0 2024-04-25T14:04:22.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.053+0000 7faf4a800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7faf440d1d50 con 0x7faf4406db90 2024-04-25T14:04:22.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.053+0000 7faf49400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf4406d780 0x7faf4414e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:22.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.053+0000 7faf49400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf4406d780 0x7faf4414e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58790/0 (socket says 172.21.15.86:58790) 2024-04-25T14:04:22.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.053+0000 7faf49400640 1 -- 172.21.15.86:0/2497784907 learned_addr learned my addr 172.21.15.86:0/2497784907 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.053+0000 7faf48a00640 1 --2- 172.21.15.86:0/2497784907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf4406db90 0x7faf440b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.053+0000 7faf49400640 1 -- 172.21.15.86:0/2497784907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf4406db90 msgr2=0x7faf440b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.053+0000 7faf49400640 1 --2- 172.21.15.86:0/2497784907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf4406db90 0x7faf440b0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.053+0000 7faf49400640 1 -- 172.21.15.86:0/2497784907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf440d17c0 msgr2=0x7faf440b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.053+0000 7faf49e00640 1 --2- 172.21.15.86:0/2497784907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf440d17c0 0x7faf440b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.053+0000 7faf49400640 1 --2- 172.21.15.86:0/2497784907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf440d17c0 0x7faf440b0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.053+0000 7faf49400640 1 -- 172.21.15.86:0/2497784907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf441f73b0 con 0x7faf4406d780 2024-04-25T14:04:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.053+0000 7faf49e00640 1 --2- 172.21.15.86:0/2497784907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf440d17c0 0x7faf440b0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.053+0000 7faf49400640 1 --2- 172.21.15.86:0/2497784907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf4406d780 0x7faf4414e070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7faf3800c3f0 tx=0x7faf3800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.053+0000 7faf41600640 1 -- 172.21.15.86:0/2497784907 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38017070 con 0x7faf4406d780 2024-04-25T14:04:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.053+0000 7faf41600640 1 -- 172.21.15.86:0/2497784907 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf380052d0 con 0x7faf4406d780 2024-04-25T14:04:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.053+0000 7faf4a800640 1 -- 172.21.15.86:0/2497784907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf441f7690 con 0x7faf4406d780 2024-04-25T14:04:22.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.053+0000 7faf41600640 1 -- 172.21.15.86:0/2497784907 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38003840 con 0x7faf4406d780 2024-04-25T14:04:22.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.053+0000 7faf4a800640 1 -- 172.21.15.86:0/2497784907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf441f7c50 con 0x7faf4406d780 2024-04-25T14:04:22.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.057+0000 7faf41600640 1 -- 172.21.15.86:0/2497784907 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7faf3801a070 con 0x7faf4406d780 2024-04-25T14:04:22.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.057+0000 7faf41600640 1 --2- 172.21.15.86:0/2497784907 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7faf2406f430 0x7faf24071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:22.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.057+0000 7faf53422000 1 Processor -- start 2024-04-25T14:04:22.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.057+0000 7faf41600640 1 -- 172.21.15.86:0/2497784907 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7faf3808e8e0 con 0x7faf4406d780 2024-04-25T14:04:22.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.057+0000 7faf53422000 1 -- start start 2024-04-25T14:04:22.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.057+0000 7faf48a00640 1 --2- 172.21.15.86:0/2497784907 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7faf2406f430 0x7faf24071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:22.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.057+0000 7faf48a00640 1 --2- 172.21.15.86:0/2497784907 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7faf2406f430 0x7faf24071940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7faf300b1000 tx=0x7faf300a9070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:22.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf53422000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ed930c20 0x55c4eda8eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:22.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf53422000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c4ed91cea0 0x55c4eda8f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:22.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf53422000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ed515280 0x55c4eda8f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:22.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf53422000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c4ed8fe160 con 0x55c4ed930c20 2024-04-25T14:04:22.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf53422000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c4ed893e20 con 0x55c4ed515280 2024-04-25T14:04:22.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf53422000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55c4ed84ca20 con 0x55c4ed91cea0 2024-04-25T14:04:22.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf49400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ed930c20 0x55c4eda8eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:22.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf49400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ed930c20 0x55c4eda8eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58794/0 (socket says 172.21.15.86:58794) 2024-04-25T14:04:22.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf49400640 1 -- 172.21.15.86:0/793800685 learned_addr learned my addr 172.21.15.86:0/793800685 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:22.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf49e00640 1 --2- 172.21.15.86:0/793800685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ed515280 0x55c4eda8f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:22.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf48a00640 1 --2- 172.21.15.86:0/793800685 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c4ed91cea0 0x55c4eda8f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:22.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf49400640 1 -- 172.21.15.86:0/793800685 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c4ed91cea0 msgr2=0x55c4eda8f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:22.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf49400640 1 --2- 172.21.15.86:0/793800685 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c4ed91cea0 0x55c4eda8f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf49400640 1 -- 172.21.15.86:0/793800685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ed515280 msgr2=0x55c4eda8f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:22.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf49400640 1 --2- 172.21.15.86:0/793800685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ed515280 0x55c4eda8f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf49400640 1 -- 172.21.15.86:0/793800685 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4ed851100 con 0x55c4ed930c20 2024-04-25T14:04:22.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf49400640 1 --2- 172.21.15.86:0/793800685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ed930c20 0x55c4eda8eee0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7faf3800a040 tx=0x7faf3805a8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e440e80cb1853561 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:22.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf35c00640 1 -- 172.21.15.86:0/793800685 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38074380 con 0x55c4ed930c20 2024-04-25T14:04:22.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf35c00640 1 -- 172.21.15.86:0/793800685 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf3805f1d0 con 0x55c4ed930c20 2024-04-25T14:04:22.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf35c00640 1 -- 172.21.15.86:0/793800685 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3805a280 con 0x55c4ed930c20 2024-04-25T14:04:22.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf53422000 1 -- 172.21.15.86:0/793800685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ed930c20 msgr2=0x55c4eda8eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:22.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf53422000 1 --2- 172.21.15.86:0/793800685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ed930c20 0x55c4eda8eee0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7faf3800a040 tx=0x7faf3805a8a0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf53422000 1 -- 172.21.15.86:0/793800685 shutdown_connections 2024-04-25T14:04:22.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf53422000 1 --2- 172.21.15.86:0/793800685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ed515280 0x55c4eda8f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf53422000 1 --2- 172.21.15.86:0/793800685 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c4ed91cea0 0x55c4eda8f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf53422000 1 --2- 172.21.15.86:0/793800685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ed930c20 0x55c4eda8eee0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf53422000 1 -- 172.21.15.86:0/793800685 >> 172.21.15.86:0/793800685 conn(0x55c4ed554ff0 msgr2=0x55c4eda444c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:22.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf53422000 1 -- 172.21.15.86:0/793800685 shutdown_connections 2024-04-25T14:04:22.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf53422000 1 -- 172.21.15.86:0/793800685 wait complete. 2024-04-25T14:04:22.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf53422000 1 -- start start 2024-04-25T14:04:22.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.061+0000 7faf53422000 1 Processor -- start 2024-04-25T14:04:22.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf53422000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ed515280 0x55c4eda95c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:22.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf53422000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ed91cea0 0x55c4eda96190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:22.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf53422000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c4ed930c20 0x55c4eda510c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:22.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf53422000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c4ed8a1220 con 0x55c4ed91cea0 2024-04-25T14:04:22.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf53422000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c4ed837ea0 con 0x55c4ed515280 2024-04-25T14:04:22.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf53422000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55c4ed88eb00 con 0x55c4ed930c20 2024-04-25T14:04:22.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf53422000 10 client.0 _ll_register_callbacks cb 0x55c4ed8a0dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:22.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf48a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ed91cea0 0x55c4eda96190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:22.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf48a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ed91cea0 0x55c4eda96190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58808/0 (socket says 172.21.15.86:58808) 2024-04-25T14:04:22.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf48a00640 1 -- 172.21.15.86:0/1127845297 learned_addr learned my addr 172.21.15.86:0/1127845297 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:22.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf49400640 1 --2- 172.21.15.86:0/1127845297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ed515280 0x55c4eda95c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:22.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf49e00640 1 --2- 172.21.15.86:0/1127845297 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c4ed930c20 0x55c4eda510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:22.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf48a00640 1 -- 172.21.15.86:0/1127845297 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c4ed930c20 msgr2=0x55c4eda510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:22.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf48a00640 1 --2- 172.21.15.86:0/1127845297 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c4ed930c20 0x55c4eda510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf48a00640 1 -- 172.21.15.86:0/1127845297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ed515280 msgr2=0x55c4eda95c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:22.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf48a00640 1 --2- 172.21.15.86:0/1127845297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ed515280 0x55c4eda95c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf48a00640 1 -- 172.21.15.86:0/1127845297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4ed833500 con 0x55c4ed91cea0 2024-04-25T14:04:22.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf49e00640 1 --2- 172.21.15.86:0/1127845297 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c4ed930c20 0x55c4eda510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:22.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf48a00640 1 --2- 172.21.15.86:0/1127845297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ed91cea0 0x55c4eda96190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7faf300b2040 tx=0x7faf3001d320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee75ba392c83a6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:22.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf2a000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:04:22.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf2a000640 1 -- 172.21.15.86:0/1127845297 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf300bbb20 con 0x55c4ed91cea0 2024-04-25T14:04:22.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf2a000640 1 -- 172.21.15.86:0/1127845297 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7faf300bbcc0 con 0x55c4ed91cea0 2024-04-25T14:04:22.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf53422000 1 -- 172.21.15.86:0/1127845297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf3805f1d0 con 0x55c4ed91cea0 2024-04-25T14:04:22.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf2a000640 1 -- 172.21.15.86:0/1127845297 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7faf300af440 con 0x55c4ed91cea0 2024-04-25T14:04:22.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf53422000 10 client.4806 fetch_fsmap learned FSMap version 9 2024-04-25T14:04:22.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf53422000 1 -- 172.21.15.86:0/1127845297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55c4ed819ea0 con 0x55c4ed91cea0 2024-04-25T14:04:22.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf2a000640 1 -- 172.21.15.86:0/1127845297 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf3805f1d0 con 0x55c4ed91cea0 2024-04-25T14:04:22.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf2a000640 1 -- 172.21.15.86:0/1127845297 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7faf3000e040 con 0x55c4ed91cea0 2024-04-25T14:04:22.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf53422000 10 client.4806 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T14:04:22.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf53422000 10 client.4806 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T14:04:22.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf53422000 20 client.4806 populate_metadata read hostname 'smithi086' 2024-04-25T14:04:22.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf53422000 1 --2- 172.21.15.86:0/1127845297 >> [v2:172.21.15.155:6832/1944249570,v1:172.21.15.155:6833/1944249570] conn(0x55c4ed5397d0 0x55c4eda51e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:22.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf53422000 4 client.4806 mds_command: new command op to 4275 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:22.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf53422000 1 -- 172.21.15.86:0/1127845297 --> [v2:172.21.15.155:6832/1944249570,v1:172.21.15.155:6833/1944249570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4ed82cbe0 con 0x55c4ed5397d0 2024-04-25T14:04:22.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.065+0000 7faf49400640 1 --2- 172.21.15.86:0/1127845297 >> [v2:172.21.15.155:6832/1944249570,v1:172.21.15.155:6833/1944249570] conn(0x55c4ed5397d0 0x55c4eda51e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:22.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.069+0000 7faf49400640 1 --2- 172.21.15.86:0/1127845297 >> [v2:172.21.15.155:6832/1944249570,v1:172.21.15.155:6833/1944249570] conn(0x55c4ed5397d0 0x55c4eda51e60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f9cedb63102e566 server_cookie=29e32af95aa8c6e3 in_seq=0 out_seq=0 2024-04-25T14:04:22.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.069+0000 7faf2a000640 10 client.4806 ms_handle_connect on v2:172.21.15.155:6832/1944249570 2024-04-25T14:04:22.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.069+0000 7faf2a000640 1 -- 172.21.15.86:0/1127845297 <== mds.0 v2:172.21.15.155:6832/1944249570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c4ed82cbe0 con 0x55c4ed5397d0 2024-04-25T14:04:22.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.069+0000 7faf2a000640 10 client.4806 handle_command_reply: tid=0 2024-04-25T14:04:22.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.069+0000 7faf53422000 1 client.4806 shutdown 2024-04-25T14:04:22.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.069+0000 7faf53422000 1 -- 172.21.15.86:0/1127845297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ed91cea0 msgr2=0x55c4eda96190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:22.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.069+0000 7faf53422000 1 --2- 172.21.15.86:0/1127845297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ed91cea0 0x55c4eda96190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7faf300b2040 tx=0x7faf3001d320 comp rx=0 tx=0).stop 2024-04-25T14:04:22.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.069+0000 7faf53422000 1 -- 172.21.15.86:0/1127845297 shutdown_connections 2024-04-25T14:04:22.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.069+0000 7faf53422000 1 --2- 172.21.15.86:0/1127845297 >> [v2:172.21.15.155:6832/1944249570,v1:172.21.15.155:6833/1944249570] conn(0x55c4ed5397d0 0x55c4eda51e60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.069+0000 7faf53422000 1 --2- 172.21.15.86:0/1127845297 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c4ed930c20 0x55c4eda510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.069+0000 7faf53422000 1 --2- 172.21.15.86:0/1127845297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ed91cea0 0x55c4eda96190 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.069+0000 7faf53422000 1 --2- 172.21.15.86:0/1127845297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ed515280 0x55c4eda95c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.069+0000 7faf2a000640 0 client.4806 ms_handle_reset on v2:172.21.15.155:6832/1944249570 2024-04-25T14:04:22.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.069+0000 7faf53422000 1 -- 172.21.15.86:0/1127845297 >> 172.21.15.86:0/1127845297 conn(0x55c4ed554ff0 msgr2=0x55c4eda44c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:22.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.069+0000 7faf53422000 1 -- 172.21.15.86:0/1127845297 shutdown_connections 2024-04-25T14:04:22.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.069+0000 7faf53422000 1 -- 172.21.15.86:0/1127845297 wait complete. 2024-04-25T14:04:22.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.069+0000 7faf53422000 20 client.4806 trim_cache size 0 max 16384 2024-04-25T14:04:22.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.085+0000 7faf53422000 1 Processor -- start 2024-04-25T14:04:22.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.085+0000 7faf53422000 1 -- start start 2024-04-25T14:04:22.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf53422000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ed515280 0x55c4edaa6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:22.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf53422000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c4ed91cea0 0x55c4edaa4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:22.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf53422000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ed930c20 0x55c4edaa5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:22.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf53422000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faf44202380 con 0x55c4ed515280 2024-04-25T14:04:22.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf53422000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faf4415b990 con 0x55c4ed930c20 2024-04-25T14:04:22.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf53422000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7faf44201d30 con 0x55c4ed91cea0 2024-04-25T14:04:22.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf49400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ed515280 0x55c4edaa6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:22.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf49400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ed515280 0x55c4edaa6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58818/0 (socket says 172.21.15.86:58818) 2024-04-25T14:04:22.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf49400640 1 -- 172.21.15.86:0/532170782 learned_addr learned my addr 172.21.15.86:0/532170782 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:22.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf48a00640 1 --2- 172.21.15.86:0/532170782 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c4ed91cea0 0x55c4edaa4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:22.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf49e00640 1 --2- 172.21.15.86:0/532170782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ed930c20 0x55c4edaa5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:22.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf49400640 1 -- 172.21.15.86:0/532170782 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c4ed91cea0 msgr2=0x55c4edaa4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:22.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf49400640 1 --2- 172.21.15.86:0/532170782 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c4ed91cea0 0x55c4edaa4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf49400640 1 -- 172.21.15.86:0/532170782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ed930c20 msgr2=0x55c4edaa5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:22.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf49400640 1 --2- 172.21.15.86:0/532170782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ed930c20 0x55c4edaa5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf49400640 1 -- 172.21.15.86:0/532170782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4eda48840 con 0x55c4ed515280 2024-04-25T14:04:22.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf49e00640 1 --2- 172.21.15.86:0/532170782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ed930c20 0x55c4edaa5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:22.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf49400640 1 --2- 172.21.15.86:0/532170782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ed515280 0x55c4edaa6420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faf3800cdb0 tx=0x7faf38004000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf506bfcf4847f90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:22.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf35200640 1 -- 172.21.15.86:0/532170782 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3808c040 con 0x55c4ed515280 2024-04-25T14:04:22.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf35200640 1 -- 172.21.15.86:0/532170782 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf38063bf0 con 0x55c4ed515280 2024-04-25T14:04:22.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf35200640 1 -- 172.21.15.86:0/532170782 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3807bdf0 con 0x55c4ed515280 2024-04-25T14:04:22.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf53422000 1 -- 172.21.15.86:0/532170782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ed515280 msgr2=0x55c4edaa6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:22.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf53422000 1 --2- 172.21.15.86:0/532170782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ed515280 0x55c4edaa6420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faf3800cdb0 tx=0x7faf38004000 comp rx=0 tx=0).stop 2024-04-25T14:04:22.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf53422000 1 -- 172.21.15.86:0/532170782 shutdown_connections 2024-04-25T14:04:22.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf53422000 1 --2- 172.21.15.86:0/532170782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ed930c20 0x55c4edaa5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf53422000 1 --2- 172.21.15.86:0/532170782 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c4ed91cea0 0x55c4edaa4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf53422000 1 --2- 172.21.15.86:0/532170782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ed515280 0x55c4edaa6420 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf53422000 1 -- 172.21.15.86:0/532170782 >> 172.21.15.86:0/532170782 conn(0x55c4ed554ff0 msgr2=0x55c4eda94aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:22.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf53422000 1 -- 172.21.15.86:0/532170782 shutdown_connections 2024-04-25T14:04:22.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.089+0000 7faf53422000 1 -- 172.21.15.86:0/532170782 wait complete. 2024-04-25T14:04:22.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf53422000 1 -- start start 2024-04-25T14:04:22.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf53422000 1 Processor -- start 2024-04-25T14:04:22.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf53422000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c4ed515280 0x55c4edaa6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:22.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf53422000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ed91cea0 0x55c4edac17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:22.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf53422000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ed930c20 0x55c4edac1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:22.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf53422000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c4ed893130 con 0x55c4ed930c20 2024-04-25T14:04:22.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf53422000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c4ed8297d0 con 0x55c4ed91cea0 2024-04-25T14:04:22.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf53422000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55c4ed898450 con 0x55c4ed515280 2024-04-25T14:04:22.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf53422000 10 client.0 _ll_register_callbacks cb 0x7faf300b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:22.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf49e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ed930c20 0x55c4edac1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:22.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf49e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ed930c20 0x55c4edac1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58824/0 (socket says 172.21.15.86:58824) 2024-04-25T14:04:22.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf49e00640 1 -- 172.21.15.86:0/327938681 learned_addr learned my addr 172.21.15.86:0/327938681 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:22.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf48a00640 1 --2- 172.21.15.86:0/327938681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ed91cea0 0x55c4edac17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:22.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf49400640 1 --2- 172.21.15.86:0/327938681 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c4ed515280 0x55c4edaa6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:22.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf49e00640 1 -- 172.21.15.86:0/327938681 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c4ed515280 msgr2=0x55c4edaa6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:22.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf49e00640 1 --2- 172.21.15.86:0/327938681 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c4ed515280 0x55c4edaa6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf49e00640 1 -- 172.21.15.86:0/327938681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ed91cea0 msgr2=0x55c4edac17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:22.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf49e00640 1 --2- 172.21.15.86:0/327938681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ed91cea0 0x55c4edac17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf49e00640 1 -- 172.21.15.86:0/327938681 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4eda51430 con 0x55c4ed930c20 2024-04-25T14:04:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf49e00640 1 --2- 172.21.15.86:0/327938681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ed930c20 0x55c4edac1cf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faf440b3860 tx=0x7faf3c012630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d3a360a7cb5739 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf2a000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:04:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf2a000640 1 -- 172.21.15.86:0/327938681 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3c0157f0 con 0x55c4ed930c20 2024-04-25T14:04:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf2a000640 1 -- 172.21.15.86:0/327938681 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7faf3c015990 con 0x55c4ed930c20 2024-04-25T14:04:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf53422000 1 -- 172.21.15.86:0/327938681 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf38063bf0 con 0x55c4ed930c20 2024-04-25T14:04:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf2a000640 1 -- 172.21.15.86:0/327938681 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7faf3c023050 con 0x55c4ed930c20 2024-04-25T14:04:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf53422000 10 client.4812 fetch_fsmap learned FSMap version 9 2024-04-25T14:04:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf53422000 1 -- 172.21.15.86:0/327938681 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55c4ed84cc70 con 0x55c4ed930c20 2024-04-25T14:04:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf2a000640 1 -- 172.21.15.86:0/327938681 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf38063bf0 con 0x55c4ed930c20 2024-04-25T14:04:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf2a000640 1 -- 172.21.15.86:0/327938681 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7faf3c026020 con 0x55c4ed930c20 2024-04-25T14:04:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.093+0000 7faf53422000 10 client.4812 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T14:04:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf53422000 10 client.4812 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T14:04:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf53422000 20 client.4812 populate_metadata read hostname 'smithi086' 2024-04-25T14:04:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf53422000 1 --2- 172.21.15.86:0/327938681 >> [v2:172.21.15.155:6832/1944249570,v1:172.21.15.155:6833/1944249570] conn(0x55c4ed527830 0x55c4edb05420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf53422000 4 client.4812 mds_command: new command op to 4275 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:04:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf53422000 1 -- 172.21.15.86:0/327938681 --> [v2:172.21.15.155:6832/1944249570,v1:172.21.15.155:6833/1944249570] -- command(tid 0: {"prefix": "status"}) -- 0x55c4ed8a1a30 con 0x55c4ed527830 2024-04-25T14:04:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf49400640 1 --2- 172.21.15.86:0/327938681 >> [v2:172.21.15.155:6832/1944249570,v1:172.21.15.155:6833/1944249570] conn(0x55c4ed527830 0x55c4edb05420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf49400640 1 --2- 172.21.15.86:0/327938681 >> [v2:172.21.15.155:6832/1944249570,v1:172.21.15.155:6833/1944249570] conn(0x55c4ed527830 0x55c4edb05420 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da5c8f597c1facd8 server_cookie=2b27b7e94c9813ea in_seq=0 out_seq=0 2024-04-25T14:04:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf2a000640 10 client.4812 ms_handle_connect on v2:172.21.15.155:6832/1944249570 2024-04-25T14:04:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf2a000640 1 -- 172.21.15.86:0/327938681 <== mds.0 v2:172.21.15.155:6832/1944249570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55c4ed8a1a30 con 0x55c4ed527830 2024-04-25T14:04:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf2a000640 10 client.4812 handle_command_reply: tid=0 2024-04-25T14:04:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf53422000 1 client.4812 shutdown 2024-04-25T14:04:22.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf53422000 1 -- 172.21.15.86:0/327938681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ed930c20 msgr2=0x55c4edac1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:22.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf53422000 1 --2- 172.21.15.86:0/327938681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ed930c20 0x55c4edac1cf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faf440b3860 tx=0x7faf3c012630 comp rx=0 tx=0).stop 2024-04-25T14:04:22.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf53422000 1 -- 172.21.15.86:0/327938681 shutdown_connections 2024-04-25T14:04:22.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf53422000 1 --2- 172.21.15.86:0/327938681 >> [v2:172.21.15.155:6832/1944249570,v1:172.21.15.155:6833/1944249570] conn(0x55c4ed527830 0x55c4edb05420 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf53422000 1 --2- 172.21.15.86:0/327938681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ed930c20 0x55c4edac1cf0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf53422000 1 --2- 172.21.15.86:0/327938681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ed91cea0 0x55c4edac17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf53422000 1 --2- 172.21.15.86:0/327938681 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c4ed515280 0x55c4edaa6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf2a000640 0 client.4812 ms_handle_reset on v2:172.21.15.155:6832/1944249570 2024-04-25T14:04:22.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf53422000 1 -- 172.21.15.86:0/327938681 >> 172.21.15.86:0/327938681 conn(0x55c4ed554ff0 msgr2=0x55c4eda9a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:22.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf53422000 1 -- 172.21.15.86:0/327938681 shutdown_connections 2024-04-25T14:04:22.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf53422000 1 -- 172.21.15.86:0/327938681 wait complete. 2024-04-25T14:04:22.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf53422000 20 client.4812 trim_cache size 0 max 16384 2024-04-25T14:04:22.103 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:04:22.103 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb", 2024-04-25T14:04:22.103 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-25T14:04:22.103 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 4275, 2024-04-25T14:04:22.103 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T14:04:22.103 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T14:04:22.103 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T14:04:22.103 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 30.630464132, 2024-04-25T14:04:22.103 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 9, 2024-04-25T14:04:22.103 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 24, 2024-04-25T14:04:22.103 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 24, 2024-04-25T14:04:22.103 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 37.686426419999997 2024-04-25T14:04:22.104 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:04:22.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf4a800640 1 -- 172.21.15.86:0/2497784907 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7faf2406f430 msgr2=0x7faf24071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:22.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf4a800640 1 --2- 172.21.15.86:0/2497784907 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7faf2406f430 0x7faf24071940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7faf300b1000 tx=0x7faf300a9070 comp rx=0 tx=0).stop 2024-04-25T14:04:22.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf4a800640 1 -- 172.21.15.86:0/2497784907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf4406d780 msgr2=0x7faf4414e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:22.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf4a800640 1 --2- 172.21.15.86:0/2497784907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf4406d780 0x7faf4414e070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7faf3800c3f0 tx=0x7faf3800c8b0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf4a800640 1 -- 172.21.15.86:0/2497784907 shutdown_connections 2024-04-25T14:04:22.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf4a800640 1 --2- 172.21.15.86:0/2497784907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf440d17c0 0x7faf440b0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf4a800640 1 --2- 172.21.15.86:0/2497784907 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7faf2406f430 0x7faf24071940 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf4a800640 1 --2- 172.21.15.86:0/2497784907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf4406db90 0x7faf440b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf4a800640 1 --2- 172.21.15.86:0/2497784907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf4406d780 0x7faf4414e070 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf4a800640 1 -- 172.21.15.86:0/2497784907 >> 172.21.15.86:0/2497784907 conn(0x7faf44073c00 msgr2=0x7faf4415eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:22.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf4a800640 1 -- 172.21.15.86:0/2497784907 shutdown_connections 2024-04-25T14:04:22.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.097+0000 7faf4a800640 1 -- 172.21.15.86:0/2497784907 wait complete. 2024-04-25T14:04:22.119 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T14:04:22.130 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T14:04:22.130 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T14:04:22.131 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T14:04:22.131 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T14:04:22.143 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T14:04:22.143 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T14:04:22.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:22.144 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T14:04:22.153 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-25T14:04:22.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:22.154 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T14:04:22.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:22.216 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-04-25T14:04:22.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:22.375 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T14:04:22.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:22.384 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-04-25T14:04:22.449 INFO:teuthology.orchestra.run.smithi086.stderr:Cannot find device "ceph-brx" 2024-04-25T14:04:22.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:04:22.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:22.452 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2024-04-25T14:04:22.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:22.463 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link delete ceph-brx 2024-04-25T14:04:22.532 INFO:teuthology.orchestra.run.smithi155.stderr:Cannot find device "ceph-brx" 2024-04-25T14:04:22.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:04:22.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:22.535 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2024-04-25T14:04:22.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:22.585 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link delete ceph-brx 2024-04-25T14:04:22.651 INFO:teuthology.orchestra.run.smithi155.stderr:Cannot find device "ceph-brx" 2024-04-25T14:04:22.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:04:22.653 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-25T14:04:22.653 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T14:04:22.653 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T14:04:22.654 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T14:04:22.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.797+0000 7fa00b200640 1 Processor -- start 2024-04-25T14:04:22.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.801+0000 7fa00b200640 1 -- start start 2024-04-25T14:04:22.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.801+0000 7fa00b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0040d1df0 0x7fa0040cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:22.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.801+0000 7fa00b200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa0040cdcc0 0x7fa0040ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:22.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.801+0000 7fa00b200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa00415e5c0 con 0x7fa004070330 2024-04-25T14:04:22.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.801+0000 7fa00b200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa00406fd40 con 0x7fa0040cdcc0 2024-04-25T14:04:22.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.801+0000 7fa00b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa00415e8a0 con 0x7fa0040d1df0 2024-04-25T14:04:22.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.801+0000 7fa003e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa004070330 0x7fa0040d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:38884/0 (socket says 172.21.15.86:38884) 2024-04-25T14:04:22.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.801+0000 7fa003e00640 1 -- 172.21.15.86:0/3249729929 learned_addr learned my addr 172.21.15.86:0/3249729929 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:22.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.801+0000 7f9ffb400640 1 --2- 172.21.15.86:0/3249729929 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0040d1df0 0x7fa0040cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:22.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.801+0000 7fa009200640 1 --2- 172.21.15.86:0/3249729929 >> v2:172.21.15.155:3300/0 conn(0x7fa0040cdcc0 0x7fa0040ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:22.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.801+0000 7fa003400640 1 -- 172.21.15.86:0/3249729929 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 531558943 0 0) 0x7fa00415e5c0 con 0x7fa004070330 2024-04-25T14:04:22.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.801+0000 7fa003400640 1 -- 172.21.15.86:0/3249729929 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fe8003530 con 0x7fa004070330 2024-04-25T14:04:22.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.801+0000 7fa009200640 1 -- 172.21.15.86:0/3249729929 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0040d1df0 msgr2=0x7fa0040cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:22.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.801+0000 7fa009200640 1 --2- 172.21.15.86:0/3249729929 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0040d1df0 0x7fa0040cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.801+0000 7fa009200640 1 -- 172.21.15.86:0/3249729929 >> v1:172.21.15.86:6789/0 conn(0x7fa004070330 legacy=0x7fa0040d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:22.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.801+0000 7fa009200640 1 -- 172.21.15.86:0/3249729929 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa00415eb80 con 0x7fa0040cdcc0 2024-04-25T14:04:22.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.801+0000 7f9ffb400640 1 --2- 172.21.15.86:0/3249729929 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0040d1df0 0x7fa0040cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:22.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.805+0000 7fa009200640 1 --2- 172.21.15.86:0/3249729929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0040cdcc0 0x7fa0040ce0d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc009d90 tx=0x7f9ffc01c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daf0146123d739c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:22.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.805+0000 7fa003400640 1 -- 172.21.15.86:0/3249729929 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc0b2070 con 0x7fa0040cdcc0 2024-04-25T14:04:22.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.805+0000 7fa003400640 1 -- 172.21.15.86:0/3249729929 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ffc0a5950 con 0x7fa0040cdcc0 2024-04-25T14:04:22.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.805+0000 7fa003400640 1 -- 172.21.15.86:0/3249729929 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc0a5d20 con 0x7fa0040cdcc0 2024-04-25T14:04:22.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.805+0000 7fa00b200640 1 -- 172.21.15.86:0/3249729929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0040cdcc0 msgr2=0x7fa0040ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:22.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.805+0000 7fa00b200640 1 --2- 172.21.15.86:0/3249729929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0040cdcc0 0x7fa0040ce0d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc009d90 tx=0x7f9ffc01c8b0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.805+0000 7fa00b200640 1 -- 172.21.15.86:0/3249729929 shutdown_connections 2024-04-25T14:04:22.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.805+0000 7fa00b200640 1 --2- 172.21.15.86:0/3249729929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0040cdcc0 0x7fa0040ce0d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.805+0000 7fa00b200640 1 --2- 172.21.15.86:0/3249729929 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0040d1df0 0x7fa0040cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.805+0000 7fa00b200640 1 -- 172.21.15.86:0/3249729929 >> 172.21.15.86:0/3249729929 conn(0x7fa0040b4650 msgr2=0x7fa0040b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:22.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.805+0000 7fa00b200640 1 -- 172.21.15.86:0/3249729929 shutdown_connections 2024-04-25T14:04:22.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.805+0000 7fa00b200640 1 -- 172.21.15.86:0/3249729929 wait complete. 2024-04-25T14:04:22.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.809+0000 7fa00b200640 1 Processor -- start 2024-04-25T14:04:22.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.809+0000 7fa00b200640 1 -- start start 2024-04-25T14:04:22.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.809+0000 7fa00b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa004070330 0x7fa0040c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:22.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.809+0000 7fa00b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0040cdcc0 0x7fa0040c7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:22.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.809+0000 7fa00b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0040d0cb0 0x7fa0040c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:22.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.809+0000 7fa00b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa00415f4b0 con 0x7fa0040cdcc0 2024-04-25T14:04:22.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.809+0000 7fa00b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa00415f330 con 0x7fa004070330 2024-04-25T14:04:22.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.809+0000 7fa00b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa004070740 con 0x7fa0040d0cb0 2024-04-25T14:04:22.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.809+0000 7f9ffb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0040cdcc0 0x7fa0040c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:22.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.809+0000 7f9ffb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0040cdcc0 0x7fa0040c7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58830/0 (socket says 172.21.15.86:58830) 2024-04-25T14:04:22.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.809+0000 7f9ffb400640 1 -- 172.21.15.86:0/1010204108 learned_addr learned my addr 172.21.15.86:0/1010204108 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:22.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.809+0000 7fa009200640 1 --2- 172.21.15.86:0/1010204108 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0040d0cb0 0x7fa0040c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:22.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.809+0000 7fa003e00640 1 --2- 172.21.15.86:0/1010204108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa004070330 0x7fa0040c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:22.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.809+0000 7f9ffb400640 1 -- 172.21.15.86:0/1010204108 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0040d0cb0 msgr2=0x7fa0040c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:22.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.809+0000 7f9ffb400640 1 --2- 172.21.15.86:0/1010204108 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0040d0cb0 0x7fa0040c7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.809+0000 7f9ffb400640 1 -- 172.21.15.86:0/1010204108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa004070330 msgr2=0x7fa0040c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:22.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.809+0000 7f9ffb400640 1 --2- 172.21.15.86:0/1010204108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa004070330 0x7fa0040c9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:22.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.809+0000 7f9ffb400640 1 -- 172.21.15.86:0/1010204108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0040cfc10 con 0x7fa0040cdcc0 2024-04-25T14:04:22.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.809+0000 7fa003e00640 1 --2- 172.21.15.86:0/1010204108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa004070330 0x7fa0040c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:22.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.809+0000 7f9ffb400640 1 --2- 172.21.15.86:0/1010204108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0040cdcc0 0x7fa0040c7800 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00e980 tx=0x7f9fec00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:22.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.809+0000 7fa000c00640 1 -- 172.21.15.86:0/1010204108 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec017070 con 0x7fa0040cdcc0 2024-04-25T14:04:22.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.813+0000 7fa000c00640 1 -- 172.21.15.86:0/1010204108 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fec00ca00 con 0x7fa0040cdcc0 2024-04-25T14:04:22.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.813+0000 7fa00b200640 1 -- 172.21.15.86:0/1010204108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0040cfef0 con 0x7fa0040cdcc0 2024-04-25T14:04:22.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.813+0000 7fa000c00640 1 -- 172.21.15.86:0/1010204108 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec00cdd0 con 0x7fa0040cdcc0 2024-04-25T14:04:22.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.813+0000 7fa00b200640 1 -- 172.21.15.86:0/1010204108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0040d04b0 con 0x7fa0040cdcc0 2024-04-25T14:04:22.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.813+0000 7fa00b200640 1 -- 172.21.15.86:0/1010204108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa004069d00 con 0x7fa0040cdcc0 2024-04-25T14:04:22.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.813+0000 7fa000c00640 1 -- 172.21.15.86:0/1010204108 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9fec01a070 con 0x7fa0040cdcc0 2024-04-25T14:04:22.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.813+0000 7fa000c00640 1 --2- 172.21.15.86:0/1010204108 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9fe006f430 0x7f9fe0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:22.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.813+0000 7fa000c00640 1 -- 172.21.15.86:0/1010204108 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9fec08f820 con 0x7fa0040cdcc0 2024-04-25T14:04:22.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.813+0000 7fa003e00640 1 --2- 172.21.15.86:0/1010204108 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9fe006f430 0x7f9fe0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:22.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.817+0000 7fa003e00640 1 --2- 172.21.15.86:0/1010204108 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9fe006f430 0x7f9fe0071940 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4005890 tx=0x7f9ff4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:22.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:22.817+0000 7fa000c00640 1 -- 172.21.15.86:0/1010204108 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9fec05bce0 con 0x7fa0040cdcc0 2024-04-25T14:04:23.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:23.009+0000 7fa00b200640 1 -- 172.21.15.86:0/1010204108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa004076bf0 con 0x7fa0040cdcc0 2024-04-25T14:04:23.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:23.013+0000 7fa000c00640 1 -- 172.21.15.86:0/1010204108 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f9fec05ffe0 con 0x7fa0040cdcc0 2024-04-25T14:04:23.013 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T14:04:23.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:23.013+0000 7fa00b200640 1 -- 172.21.15.86:0/1010204108 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9fe006f430 msgr2=0x7f9fe0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:23.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:23.013+0000 7fa00b200640 1 --2- 172.21.15.86:0/1010204108 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9fe006f430 0x7f9fe0071940 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4005890 tx=0x7f9ff4009040 comp rx=0 tx=0).stop 2024-04-25T14:04:23.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:23.013+0000 7fa00b200640 1 -- 172.21.15.86:0/1010204108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0040cdcc0 msgr2=0x7fa0040c7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:23.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:23.013+0000 7fa00b200640 1 --2- 172.21.15.86:0/1010204108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0040cdcc0 0x7fa0040c7800 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00e980 tx=0x7f9fec00ee40 comp rx=0 tx=0).stop 2024-04-25T14:04:23.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:23.013+0000 7fa00b200640 1 -- 172.21.15.86:0/1010204108 shutdown_connections 2024-04-25T14:04:23.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:23.013+0000 7fa00b200640 1 --2- 172.21.15.86:0/1010204108 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9fe006f430 0x7f9fe0071940 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:23.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:23.013+0000 7fa00b200640 1 --2- 172.21.15.86:0/1010204108 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0040d0cb0 0x7fa0040c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:23.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:23.013+0000 7fa00b200640 1 --2- 172.21.15.86:0/1010204108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0040cdcc0 0x7fa0040c7800 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:23.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:23.013+0000 7fa00b200640 1 --2- 172.21.15.86:0/1010204108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa004070330 0x7fa0040c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:23.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:23.013+0000 7fa00b200640 1 -- 172.21.15.86:0/1010204108 >> 172.21.15.86:0/1010204108 conn(0x7fa0040b4650 msgr2=0x7fa0040ae340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:23.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:23.013+0000 7fa00b200640 1 -- 172.21.15.86:0/1010204108 shutdown_connections 2024-04-25T14:04:23.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:23.013+0000 7fa00b200640 1 -- 172.21.15.86:0/1010204108 wait complete. 2024-04-25T14:04:23.035 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T14:04:23.036 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T14:04:23.036 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-25T14:04:23.036 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T14:04:23.036 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T14:04:23.036 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T14:04:23.036 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T14:04:23.036 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T14:04:23.036 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:04:23.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:23.036 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-25T14:04:23.046 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:04:23.046 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:04:23.046 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:04:23.046 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:23.047 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-25T14:04:23.047 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:23.047 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:04:23.047 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:04:23.047 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-25T14:04:23.047 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:04:23.047 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:04:23.047 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-25T14:04:23.047 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:04:23.047 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:04:23.047 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T14:04:23.047 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:04:23.047 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T14:04:23.048 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42401sec preferred_lft 42401sec 2024-04-25T14:04:23.048 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-25T14:04:23.048 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:23.048 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T14:04:23.048 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:dc:87:9c:96 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:04:23.048 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T14:04:23.048 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:23.048 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T14:04:23.048 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:04:23.049 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T14:04:23.049 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add name ceph-brx type bridge 2024-04-25T14:04:23.049 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr flush dev ceph-brx 2024-04-25T14:04:23.049 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set ceph-brx up 2024-04-25T14:04:23.049 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T14:04:23.049 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T14:04:23.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:04:23.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:23.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:23.199 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T14:04:23.217 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-25T14:04:23.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:23.219 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-25T14:04:23.275 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-25T14:04:23.276 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T14:04:23.276 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T14:04:23.276 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T14:04:23.276 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T14:04:23.276 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T14:04:23.276 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T14:04:23.276 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:04:23.276 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:04:23.276 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:04:23.276 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T14:04:23.277 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:04:23.277 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T14:04:23.277 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T14:04:23.277 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T14:04:23.277 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T14:04:23.277 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T14:04:23.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:04:23.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:23.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:23.409 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T14:04:23.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:23.419 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list-id 2024-04-25T14:04:23.473 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:04:23.473 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T14:04:23.473 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T14:04:23.473 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T14:04:23.473 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T14:04:23.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:04:23.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:23.602 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T14:04:23.603 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:04:23.603 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T14:04:23.603 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T14:04:23.603 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T14:04:23.603 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T14:04:23.603 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T14:04:23.603 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T14:04:23.603 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T14:04:23.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:04:23.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:23.740 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:04:23.740 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T14:04:23.740 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-25T14:04:23.741 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T14:04:23.741 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T14:04:23.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:04:23.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:23.825 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:04:23.825 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T14:04:23.877 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:04:23.877 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:04:23.877 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T14:04:23.928 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T14:04:23.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:23.928 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:04:24.027 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:04:24.027 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T14:04:24.027 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:04:24.642 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:04:24.643 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T14:04:24.643 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:04:24.643 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:04:24.643 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:04:24.656 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:04:24.656 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T14:04:24.656 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:04:24.656 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:04:24.656 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:04:24.656 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790" 2024-04-25T14:04:24.656 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790:/ 2024-04-25T14:04:24.656 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb" will pass to kernel 2024-04-25T14:04:24.660 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T14:04:24.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:24.660 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T14:04:24.680 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:04:24.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:04:24.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:24.778 INFO:teuthology.orchestra.run.smithi086.stdout:client4821 2024-04-25T14:04:24.779 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:04:24.779 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:04:24.829 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:04:24.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:04:24.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:24.929 INFO:teuthology.orchestra.run.smithi086.stdout:client4821 2024-04-25T14:04:24.929 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:04:24.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:04:24.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:24.985 INFO:teuthology.orchestra.run.smithi086.stdout:client4821 2024-04-25T14:04:24.985 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb.client4821/status') 2024-04-25T14:04:25.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:04:25.086 INFO:teuthology.orchestra.run.smithi086.stdout:instance: client.4821 (3)192.168.0.1:0/316790280 2024-04-25T14:04:25.087 INFO:teuthology.orchestra.run.smithi086.stdout:blocklisted: false 2024-04-25T14:04:25.087 INFO:teuthology.orchestra.run.smithi086.stderr:0+1 records in 2024-04-25T14:04:25.087 INFO:teuthology.orchestra.run.smithi086.stderr:0+1 records out 2024-04-25T14:04:25.087 INFO:teuthology.orchestra.run.smithi086.stderr:68 bytes copied, 6.0293e-05 s, 1.1 MB/s 2024-04-25T14:04:25.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:25.095 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-04-25T14:04:25.095 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T14:04:25.095 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T14:04:25.095 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T14:04:25.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.241+0000 7f5590c00640 1 Processor -- start 2024-04-25T14:04:25.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.241+0000 7f5590c00640 1 -- start start 2024-04-25T14:04:25.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.241+0000 7f5590c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f558c0d2c80 0x7f558c15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:25.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.241+0000 7f5590c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f558c1616e0 0x7f558c161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:25.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.241+0000 7f5590c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f558c0cd8d0 con 0x7f558c070e20 2024-04-25T14:04:25.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.241+0000 7f5590c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f558c0ce720 con 0x7f558c1616e0 2024-04-25T14:04:25.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.241+0000 7f5590c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f558c0ce970 con 0x7f558c0d2c80 2024-04-25T14:04:25.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.241+0000 7f558a800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f558c1616e0 0x7f558c161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:25.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.241+0000 7f5589e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f558c0d2c80 0x7f558c15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:25.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.241+0000 7f5581400640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f558c070e20 0x7f558c071230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:52822/0 (socket says 172.21.15.86:52822) 2024-04-25T14:04:25.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.241+0000 7f5581400640 1 -- 172.21.15.86:0/1608394193 learned_addr learned my addr 172.21.15.86:0/1608394193 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:25.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.245+0000 7f5589e00640 1 -- 172.21.15.86:0/1608394193 >> v2:172.21.15.155:3300/0 conn(0x7f558c1616e0 msgr2=0x7f558c161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:25.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.245+0000 7f5589e00640 1 --2- 172.21.15.86:0/1608394193 >> v2:172.21.15.155:3300/0 conn(0x7f558c1616e0 0x7f558c161af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:25.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.245+0000 7f5589e00640 1 -- 172.21.15.86:0/1608394193 >> v1:172.21.15.155:6789/0 conn(0x7f558c070e20 legacy=0x7f558c071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:25.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.245+0000 7f5589e00640 1 -- 172.21.15.86:0/1608394193 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558c0d1be0 con 0x7f558c0d2c80 2024-04-25T14:04:25.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.245+0000 7f558a800640 1 --2- 172.21.15.86:0/1608394193 >> v2:172.21.15.155:3300/0 conn(0x7f558c1616e0 0x7f558c161af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.245+0000 7f5589e00640 1 --2- 172.21.15.86:0/1608394193 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f558c0d2c80 0x7f558c15dc90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f557c0097c0 tx=0x7f557c01c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c36617bdb9af006 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.245+0000 7f5589400640 1 -- 172.21.15.86:0/1608394193 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c0b0070 con 0x7f558c0d2c80 2024-04-25T14:04:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.245+0000 7f5589400640 1 -- 172.21.15.86:0/1608394193 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f557c0ab600 con 0x7f558c0d2c80 2024-04-25T14:04:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.245+0000 7f5589400640 1 -- 172.21.15.86:0/1608394193 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c0ab9d0 con 0x7f558c0d2c80 2024-04-25T14:04:25.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.249+0000 7f5590c00640 1 -- 172.21.15.86:0/1608394193 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f558c0d2c80 msgr2=0x7f558c15dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:25.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.249+0000 7f5590c00640 1 --2- 172.21.15.86:0/1608394193 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f558c0d2c80 0x7f558c15dc90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f557c0097c0 tx=0x7f557c01c3c0 comp rx=0 tx=0).stop 2024-04-25T14:04:25.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.249+0000 7f5590c00640 1 -- 172.21.15.86:0/1608394193 shutdown_connections 2024-04-25T14:04:25.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.249+0000 7f5590c00640 1 --2- 172.21.15.86:0/1608394193 >> v2:172.21.15.155:3300/0 conn(0x7f558c1616e0 0x7f558c161af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:25.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.249+0000 7f5590c00640 1 --2- 172.21.15.86:0/1608394193 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f558c0d2c80 0x7f558c15dc90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:25.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.249+0000 7f5590c00640 1 -- 172.21.15.86:0/1608394193 >> 172.21.15.86:0/1608394193 conn(0x7f558c0cffc0 msgr2=0x7f558c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:25.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.249+0000 7f5590c00640 1 -- 172.21.15.86:0/1608394193 shutdown_connections 2024-04-25T14:04:25.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.249+0000 7f5590c00640 1 -- 172.21.15.86:0/1608394193 wait complete. 2024-04-25T14:04:25.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.249+0000 7f5590c00640 1 Processor -- start 2024-04-25T14:04:25.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.249+0000 7f5590c00640 1 -- start start 2024-04-25T14:04:25.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.249+0000 7f5590c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f558c070e20 0x7f558c0c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:25.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.249+0000 7f5590c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f558c0d2c80 0x7f558c157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:25.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.253+0000 7f5590c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f558c1616e0 0x7f558c1f6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.253+0000 7f5590c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f558c0d2480 con 0x7f558c0d2c80 2024-04-25T14:04:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.253+0000 7f5590c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f558c0d21a0 con 0x7f558c1616e0 2024-04-25T14:04:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.253+0000 7f5590c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f558c0d29a0 con 0x7f558c070e20 2024-04-25T14:04:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.253+0000 7f5581400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f558c0d2c80 0x7f558c157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.253+0000 7f5581400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f558c0d2c80 0x7f558c157230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58844/0 (socket says 172.21.15.86:58844) 2024-04-25T14:04:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.253+0000 7f5581400640 1 -- 172.21.15.86:0/181044952 learned_addr learned my addr 172.21.15.86:0/181044952 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.253+0000 7f558a800640 1 --2- 172.21.15.86:0/181044952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f558c1616e0 0x7f558c1f6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.253+0000 7f5589e00640 1 --2- 172.21.15.86:0/181044952 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f558c070e20 0x7f558c0c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.253+0000 7f5581400640 1 -- 172.21.15.86:0/181044952 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f558c070e20 msgr2=0x7f558c0c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.253+0000 7f5581400640 1 --2- 172.21.15.86:0/181044952 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f558c070e20 0x7f558c0c6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.253+0000 7f5581400640 1 -- 172.21.15.86:0/181044952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f558c1616e0 msgr2=0x7f558c1f6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.253+0000 7f5581400640 1 --2- 172.21.15.86:0/181044952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f558c1616e0 0x7f558c1f6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.253+0000 7f5581400640 1 -- 172.21.15.86:0/181044952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558c156470 con 0x7f558c0d2c80 2024-04-25T14:04:25.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.253+0000 7f558a800640 1 --2- 172.21.15.86:0/181044952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f558c1616e0 0x7f558c1f6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:25.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.253+0000 7f5589e00640 1 --2- 172.21.15.86:0/181044952 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f558c070e20 0x7f558c0c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:25.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.253+0000 7f5581400640 1 --2- 172.21.15.86:0/181044952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f558c0d2c80 0x7f558c157230 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5574005900 tx=0x7f557400c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:25.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.253+0000 7f5582a00640 1 -- 172.21.15.86:0/181044952 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574019070 con 0x7f558c0d2c80 2024-04-25T14:04:25.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.253+0000 7f5582a00640 1 -- 172.21.15.86:0/181044952 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5574007940 con 0x7f558c0d2c80 2024-04-25T14:04:25.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.253+0000 7f5582a00640 1 -- 172.21.15.86:0/181044952 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574007d30 con 0x7f558c0d2c80 2024-04-25T14:04:25.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.253+0000 7f5590c00640 1 -- 172.21.15.86:0/181044952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558c156750 con 0x7f558c0d2c80 2024-04-25T14:04:25.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.253+0000 7f5590c00640 1 -- 172.21.15.86:0/181044952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f558c156d10 con 0x7f558c0d2c80 2024-04-25T14:04:25.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.253+0000 7f5590c00640 1 -- 172.21.15.86:0/181044952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f558c0720e0 con 0x7f558c0d2c80 2024-04-25T14:04:25.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.253+0000 7f5582a00640 1 -- 172.21.15.86:0/181044952 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f557401c070 con 0x7f558c0d2c80 2024-04-25T14:04:25.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.257+0000 7f5582a00640 1 --2- 172.21.15.86:0/181044952 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f556806f400 0x7f5568071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:25.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.257+0000 7f5582a00640 1 -- 172.21.15.86:0/181044952 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5574091370 con 0x7f558c0d2c80 2024-04-25T14:04:25.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.257+0000 7f5589e00640 1 --2- 172.21.15.86:0/181044952 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f556806f400 0x7f5568071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.257+0000 7f5589e00640 1 --2- 172.21.15.86:0/181044952 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f556806f400 0x7f5568071910 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f557c0ad000 tx=0x7f557c0a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.261+0000 7f5582a00640 1 -- 172.21.15.86:0/181044952 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f557405de80 con 0x7f558c0d2c80 2024-04-25T14:04:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.453+0000 7f5590c00640 1 -- 172.21.15.86:0/181044952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f558c1f7600 con 0x7f558c0d2c80 2024-04-25T14:04:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.453+0000 7f5582a00640 1 -- 172.21.15.86:0/181044952 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f557405d780 con 0x7f558c0d2c80 2024-04-25T14:04:25.455 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T14:04:25.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.453+0000 7f5590c00640 1 -- 172.21.15.86:0/181044952 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f556806f400 msgr2=0x7f5568071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.453+0000 7f5590c00640 1 --2- 172.21.15.86:0/181044952 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f556806f400 0x7f5568071910 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f557c0ad000 tx=0x7f557c0a6000 comp rx=0 tx=0).stop 2024-04-25T14:04:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.453+0000 7f5590c00640 1 -- 172.21.15.86:0/181044952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f558c0d2c80 msgr2=0x7f558c157230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.453+0000 7f5590c00640 1 --2- 172.21.15.86:0/181044952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f558c0d2c80 0x7f558c157230 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5574005900 tx=0x7f557400c430 comp rx=0 tx=0).stop 2024-04-25T14:04:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.453+0000 7f5590c00640 1 -- 172.21.15.86:0/181044952 shutdown_connections 2024-04-25T14:04:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.453+0000 7f5590c00640 1 --2- 172.21.15.86:0/181044952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f558c1616e0 0x7f558c1f6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.453+0000 7f5590c00640 1 --2- 172.21.15.86:0/181044952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f558c0d2c80 0x7f558c157230 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.453+0000 7f5590c00640 1 --2- 172.21.15.86:0/181044952 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f556806f400 0x7f5568071910 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.453+0000 7f5590c00640 1 --2- 172.21.15.86:0/181044952 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f558c070e20 0x7f558c0c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.453+0000 7f5590c00640 1 -- 172.21.15.86:0/181044952 >> 172.21.15.86:0/181044952 conn(0x7f558c0cffc0 msgr2=0x7f558c06e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.453+0000 7f5590c00640 1 -- 172.21.15.86:0/181044952 shutdown_connections 2024-04-25T14:04:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25.453+0000 7f5590c00640 1 -- 172.21.15.86:0/181044952 wait complete. 2024-04-25T14:04:25.476 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T14:04:25.476 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T14:04:25.476 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-25T14:04:25.476 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T14:04:25.476 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T14:04:25.476 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T14:04:25.476 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T14:04:25.476 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T14:04:25.476 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T14:04:25.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:25.477 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-25T14:04:25.486 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:04:25.486 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:04:25.486 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:04:25.486 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:25.486 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-25T14:04:25.486 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:25.486 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:04:25.486 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:04:25.486 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-25T14:04:25.486 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:04:25.486 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:04:25.487 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-25T14:04:25.487 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:04:25.487 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:04:25.487 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T14:04:25.487 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:04:25.487 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T14:04:25.487 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42399sec preferred_lft 42399sec 2024-04-25T14:04:25.487 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-25T14:04:25.487 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:25.487 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T14:04:25.487 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:dc:87:9c:96 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:04:25.487 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T14:04:25.487 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:25.487 INFO:teuthology.orchestra.run.smithi086.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T14:04:25.487 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T14:04:25.488 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T14:04:25.488 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:25.488 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T14:04:25.488 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:25.488 INFO:teuthology.orchestra.run.smithi086.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:04:25.488 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T14:04:25.488 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T14:04:25.488 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:25.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:25.488 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T14:04:25.546 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-25T14:04:25.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:25.548 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-25T14:04:25.600 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-25T14:04:25.600 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T14:04:25.600 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T14:04:25.600 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T14:04:25.600 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T14:04:25.600 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T14:04:25.601 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T14:04:25.601 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:04:25.601 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:04:25.601 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:04:25.601 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T14:04:25.601 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:04:25.601 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T14:04:25.601 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T14:04:25.601 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T14:04:25.601 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T14:04:25.601 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T14:04:25.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:04:25.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:25.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:25.738 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T14:04:25.747 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T14:04:25.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:25.748 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list-id 2024-04-25T14:04:25.796 INFO:teuthology.orchestra.run.smithi086.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T14:04:25.797 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:04:25.797 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T14:04:25.797 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T14:04:25.797 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T14:04:25.797 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T14:04:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:04:25.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:25.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:25.923 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T14:04:25.944 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:04:25.944 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:04:25.944 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:04:25.944 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:25.944 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-25T14:04:25.944 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:25.945 INFO:teuthology.orchestra.run.smithi086.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T14:04:25.945 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 16:e5:c3:f4:80:cd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T14:04:25.945 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T14:04:25.945 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:25.945 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::14e5:c3ff:fef4:80cd/64 scope link 2024-04-25T14:04:25.945 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:25.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:25.946 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T14:04:26.008 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:04:26.008 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:04:26.008 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:04:26.008 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:26.008 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-25T14:04:26.008 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:26.008 INFO:teuthology.orchestra.run.smithi086.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T14:04:26.009 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 16:e5:c3:f4:80:cd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T14:04:26.009 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T14:04:26.009 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:26.009 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::14e5:c3ff:fef4:80cd/64 scope link 2024-04-25T14:04:26.009 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:26.010 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T14:04:26.010 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:04:26.010 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T14:04:26.010 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T14:04:26.011 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T14:04:26.011 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T14:04:26.011 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T14:04:26.011 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T14:04:26.011 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T14:04:26.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:04:26.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:26.193 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:04:26.194 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T14:04:26.194 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.1 up 2024-04-25T14:04:26.194 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T14:04:26.194 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T14:04:26.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:04:26.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:26.276 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:04:26.276 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T14:04:26.325 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T14:04:26.325 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:04:26.325 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T14:04:26.380 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T14:04:26.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:26.380 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T14:04:26.483 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T14:04:26.483 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T14:04:26.483 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:04:26.501 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T14:04:26.501 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T14:04:26.501 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:04:26.501 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:04:26.501 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:04:26.512 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T14:04:26.513 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T14:04:26.513 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:04:26.513 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:04:26.513 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:04:26.513 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790" 2024-04-25T14:04:26.513 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790:/ 2024-04-25T14:04:26.513 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb" will pass to kernel 2024-04-25T14:04:26.516 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T14:04:26.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:26.517 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T14:04:26.536 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:04:26.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:04:26.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:26.633 INFO:teuthology.orchestra.run.smithi086.stdout:client4628 2024-04-25T14:04:26.633 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:04:26.633 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:04:26.689 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:04:26.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:04:26.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:26.788 INFO:teuthology.orchestra.run.smithi086.stdout:client4628 2024-04-25T14:04:26.788 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:04:26.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:04:26.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:26.846 INFO:teuthology.orchestra.run.smithi086.stdout:client4628 2024-04-25T14:04:26.846 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb.client4628/status') 2024-04-25T14:04:26.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:04:26.951 INFO:teuthology.orchestra.run.smithi086.stdout:instance: client.4628 (3)172.21.15.86:0/2860594241 2024-04-25T14:04:26.952 INFO:teuthology.orchestra.run.smithi086.stdout:blocklisted: false 2024-04-25T14:04:26.952 INFO:teuthology.orchestra.run.smithi086.stderr:0+1 records in 2024-04-25T14:04:26.952 INFO:teuthology.orchestra.run.smithi086.stderr:0+1 records out 2024-04-25T14:04:26.952 INFO:teuthology.orchestra.run.smithi086.stderr:70 bytes copied, 6.7675e-05 s, 1.0 MB/s 2024-04-25T14:04:26.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:26.959 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-25T14:04:26.960 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T14:04:26.960 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T14:04:26.960 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T14:04:27.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.105+0000 7f671fc00640 1 Processor -- start 2024-04-25T14:04:27.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.105+0000 7f671fc00640 1 -- start start 2024-04-25T14:04:27.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.105+0000 7f671fc00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f67180d2360 0x7f67180d2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:27.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.105+0000 7f671fc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f671806d7b0 0x7f671806dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:27.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.105+0000 7f671fc00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67180d2cb0 con 0x7f671806e9b0 2024-04-25T14:04:27.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.105+0000 7f671fc00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f671815e500 con 0x7f67180d2360 2024-04-25T14:04:27.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.105+0000 7f671fc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f671806c710 con 0x7f671806d7b0 2024-04-25T14:04:27.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.105+0000 7f670fe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f67180d2360 0x7f67180d2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:27.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.105+0000 7f671d200640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f671806e9b0 0x7f671815ddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:52834/0 (socket says 172.21.15.86:52834) 2024-04-25T14:04:27.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.105+0000 7f671d200640 1 -- 172.21.15.86:0/367089411 learned_addr learned my addr 172.21.15.86:0/367089411 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:27.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.105+0000 7f671dc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f671806d7b0 0x7f671806dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:27.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.105+0000 7f670fe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f67180d2360 0x7f67180d2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:40736/0 (socket says 172.21.15.86:40736) 2024-04-25T14:04:27.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.105+0000 7f670fe00640 1 -- 172.21.15.86:0/367089411 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f671806d7b0 msgr2=0x7f671806dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:27.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.105+0000 7f670fe00640 1 --2- 172.21.15.86:0/367089411 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f671806d7b0 0x7f671806dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:27.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.105+0000 7f670fe00640 1 -- 172.21.15.86:0/367089411 >> v1:172.21.15.155:6789/0 conn(0x7f671806e9b0 legacy=0x7f671815ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:27.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.105+0000 7f670fe00640 1 -- 172.21.15.86:0/367089411 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671806c9f0 con 0x7f67180d2360 2024-04-25T14:04:27.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.109+0000 7f670fe00640 1 --2- 172.21.15.86:0/367089411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67180d2360 0x7f67180d2770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f67000097c0 tx=0x7f670001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96cd7e953f2f7df2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:27.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.109+0000 7f6717e00640 1 -- 172.21.15.86:0/367089411 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67000ab620 con 0x7f67180d2360 2024-04-25T14:04:27.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.109+0000 7f6717e00640 1 -- 172.21.15.86:0/367089411 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67000b0070 con 0x7f67180d2360 2024-04-25T14:04:27.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.109+0000 7f6717e00640 1 -- 172.21.15.86:0/367089411 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67000ab9e0 con 0x7f67180d2360 2024-04-25T14:04:27.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.109+0000 7f671fc00640 1 -- 172.21.15.86:0/367089411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67180d2360 msgr2=0x7f67180d2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:27.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.109+0000 7f671fc00640 1 --2- 172.21.15.86:0/367089411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67180d2360 0x7f67180d2770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f67000097c0 tx=0x7f670001c4a0 comp rx=0 tx=0).stop 2024-04-25T14:04:27.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.109+0000 7f671fc00640 1 -- 172.21.15.86:0/367089411 shutdown_connections 2024-04-25T14:04:27.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.109+0000 7f671fc00640 1 --2- 172.21.15.86:0/367089411 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f671806d7b0 0x7f671806dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:27.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.109+0000 7f671fc00640 1 --2- 172.21.15.86:0/367089411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67180d2360 0x7f67180d2770 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:27.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.109+0000 7f671fc00640 1 -- 172.21.15.86:0/367089411 >> 172.21.15.86:0/367089411 conn(0x7f671809ba30 msgr2=0x7f67180b7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:27.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.113+0000 7f671fc00640 1 -- 172.21.15.86:0/367089411 shutdown_connections 2024-04-25T14:04:27.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.113+0000 7f671fc00640 1 -- 172.21.15.86:0/367089411 wait complete. 2024-04-25T14:04:27.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.113+0000 7f671fc00640 1 Processor -- start 2024-04-25T14:04:27.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.113+0000 7f671fc00640 1 -- start start 2024-04-25T14:04:27.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.113+0000 7f671fc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f671806d7b0 0x7f67180caf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:27.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.113+0000 7f671fc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f671806e9b0 0x7f6718156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:27.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.113+0000 7f671fc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67180d2360 0x7f6718157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:27.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.113+0000 7f671fc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f671806f110 con 0x7f671806e9b0 2024-04-25T14:04:27.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.113+0000 7f671fc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f671806ef90 con 0x7f67180d2360 2024-04-25T14:04:27.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.113+0000 7f671fc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f671816a950 con 0x7f671806d7b0 2024-04-25T14:04:27.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.117+0000 7f671dc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67180d2360 0x7f6718157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:27.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.117+0000 7f671d200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f671806d7b0 0x7f67180caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:27.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.117+0000 7f671dc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67180d2360 0x7f6718157520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:40746/0 (socket says 172.21.15.86:40746) 2024-04-25T14:04:27.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.117+0000 7f671d200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f671806d7b0 0x7f67180caf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:40084/0 (socket says 172.21.15.86:40084) 2024-04-25T14:04:27.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.117+0000 7f671dc00640 1 -- 172.21.15.86:0/824210069 learned_addr learned my addr 172.21.15.86:0/824210069 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:27.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.117+0000 7f670fe00640 1 --2- 172.21.15.86:0/824210069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f671806e9b0 0x7f6718156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:27.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.117+0000 7f671dc00640 1 -- 172.21.15.86:0/824210069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f671806d7b0 msgr2=0x7f67180caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:27.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.117+0000 7f671dc00640 1 --2- 172.21.15.86:0/824210069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f671806d7b0 0x7f67180caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:27.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.117+0000 7f671dc00640 1 -- 172.21.15.86:0/824210069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f671806e9b0 msgr2=0x7f6718156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:27.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.117+0000 7f671dc00640 1 --2- 172.21.15.86:0/824210069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f671806e9b0 0x7f6718156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:27.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.117+0000 7f671dc00640 1 -- 172.21.15.86:0/824210069 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67180cb750 con 0x7f67180d2360 2024-04-25T14:04:27.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.117+0000 7f670fe00640 1 --2- 172.21.15.86:0/824210069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f671806e9b0 0x7f6718156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:27.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.117+0000 7f671dc00640 1 --2- 172.21.15.86:0/824210069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67180d2360 0x7f6718157520 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6710009cb0 tx=0x7f671000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:27.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.117+0000 7f6715600640 1 -- 172.21.15.86:0/824210069 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710018020 con 0x7f67180d2360 2024-04-25T14:04:27.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.117+0000 7f6715600640 1 -- 172.21.15.86:0/824210069 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6710014480 con 0x7f67180d2360 2024-04-25T14:04:27.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.117+0000 7f671fc00640 1 -- 172.21.15.86:0/824210069 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67180cba30 con 0x7f67180d2360 2024-04-25T14:04:27.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.117+0000 7f6715600640 1 -- 172.21.15.86:0/824210069 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710014870 con 0x7f67180d2360 2024-04-25T14:04:27.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.117+0000 7f671fc00640 1 -- 172.21.15.86:0/824210069 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67180c89c0 con 0x7f67180d2360 2024-04-25T14:04:27.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.117+0000 7f671fc00640 1 -- 172.21.15.86:0/824210069 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67180c8c20 con 0x7f67180d2360 2024-04-25T14:04:27.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.117+0000 7f6715600640 1 -- 172.21.15.86:0/824210069 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f671001b020 con 0x7f67180d2360 2024-04-25T14:04:27.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.121+0000 7f6715600640 1 --2- 172.21.15.86:0/824210069 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f66f406f430 0x7f66f4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:27.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.121+0000 7f6715600640 1 -- 172.21.15.86:0/824210069 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f67100905c0 con 0x7f67180d2360 2024-04-25T14:04:27.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.121+0000 7f671d200640 1 --2- 172.21.15.86:0/824210069 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f66f406f430 0x7f66f4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:27.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.121+0000 7f671d200640 1 --2- 172.21.15.86:0/824210069 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f66f406f430 0x7f66f4071940 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6708000f70 tx=0x7f6708007e90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:27.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.125+0000 7f6715600640 1 -- 172.21.15.86:0/824210069 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6710022030 con 0x7f67180d2360 2024-04-25T14:04:27.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.317+0000 7f671fc00640 1 -- 172.21.15.86:0/824210069 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f67180a05a0 con 0x7f67180d2360 2024-04-25T14:04:27.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.321+0000 7f6715600640 1 -- 172.21.15.86:0/824210069 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f6710010040 con 0x7f67180d2360 2024-04-25T14:04:27.321 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T14:04:27.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.321+0000 7f671fc00640 1 -- 172.21.15.86:0/824210069 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f66f406f430 msgr2=0x7f66f4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:27.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.321+0000 7f671fc00640 1 --2- 172.21.15.86:0/824210069 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f66f406f430 0x7f66f4071940 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6708000f70 tx=0x7f6708007e90 comp rx=0 tx=0).stop 2024-04-25T14:04:27.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.321+0000 7f671fc00640 1 -- 172.21.15.86:0/824210069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67180d2360 msgr2=0x7f6718157520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:27.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.321+0000 7f671fc00640 1 --2- 172.21.15.86:0/824210069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67180d2360 0x7f6718157520 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6710009cb0 tx=0x7f671000c490 comp rx=0 tx=0).stop 2024-04-25T14:04:27.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.321+0000 7f671fc00640 1 -- 172.21.15.86:0/824210069 shutdown_connections 2024-04-25T14:04:27.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.321+0000 7f671fc00640 1 --2- 172.21.15.86:0/824210069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67180d2360 0x7f6718157520 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:27.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.321+0000 7f671fc00640 1 --2- 172.21.15.86:0/824210069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f671806e9b0 0x7f6718156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:27.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.321+0000 7f671fc00640 1 --2- 172.21.15.86:0/824210069 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f66f406f430 0x7f66f4071940 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:27.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.321+0000 7f671fc00640 1 --2- 172.21.15.86:0/824210069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f671806d7b0 0x7f67180caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:27.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.321+0000 7f671fc00640 1 -- 172.21.15.86:0/824210069 >> 172.21.15.86:0/824210069 conn(0x7f671809ba30 msgr2=0x7f671806fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:27.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.321+0000 7f671fc00640 1 -- 172.21.15.86:0/824210069 shutdown_connections 2024-04-25T14:04:27.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:27.321+0000 7f671fc00640 1 -- 172.21.15.86:0/824210069 wait complete. 2024-04-25T14:04:27.342 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T14:04:27.342 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T14:04:27.342 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2024-04-25T14:04:27.342 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T14:04:27.343 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T14:04:27.343 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T14:04:27.343 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T14:04:27.343 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T14:04:27.343 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:04:27.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:27.343 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2024-04-25T14:04:27.353 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:04:27.353 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:04:27.353 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:04:27.353 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:27.353 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2024-04-25T14:04:27.353 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:27.353 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:04:27.353 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2024-04-25T14:04:27.353 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f0 2024-04-25T14:04:27.354 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:04:27.354 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2024-04-25T14:04:27.354 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f1 2024-04-25T14:04:27.354 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:04:27.354 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2024-04-25T14:04:27.354 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T14:04:27.354 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2024-04-25T14:04:27.354 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T14:04:27.354 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 42390sec preferred_lft 42390sec 2024-04-25T14:04:27.354 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::ec4:7aff:fe88:6ded/64 scope link 2024-04-25T14:04:27.354 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:27.354 INFO:teuthology.orchestra.run.smithi155.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T14:04:27.354 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 02:42:b1:ff:60:50 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:04:27.355 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T14:04:27.355 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:27.355 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T14:04:27.356 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:04:27.356 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-04-25T14:04:27.356 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link add name ceph-brx type bridge 2024-04-25T14:04:27.356 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip addr flush dev ceph-brx 2024-04-25T14:04:27.356 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set ceph-brx up 2024-04-25T14:04:27.356 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T14:04:27.356 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-04-25T14:04:27.452 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:04:27.516 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:27.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:27.523 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T14:04:27.541 INFO:teuthology.orchestra.run.smithi155.stdout:1 2024-04-25T14:04:27.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:27.544 DEBUG:teuthology.orchestra.run.smithi155:> route 2024-04-25T14:04:27.626 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2024-04-25T14:04:27.626 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T14:04:27.627 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T14:04:27.627 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T14:04:27.627 INFO:teuthology.orchestra.run.smithi155.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T14:04:27.627 INFO:teuthology.orchestra.run.smithi155.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T14:04:27.627 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T14:04:27.627 INFO:teuthology.orchestra.run.smithi155.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:04:27.627 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:04:27.627 INFO:teuthology.orchestra.run.smithi155.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:04:27.627 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T14:04:27.628 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:04:27.628 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-04-25T14:04:27.628 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T14:04:27.628 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T14:04:27.628 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T14:04:27.628 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-04-25T14:04:27.715 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:04:27.756 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:27.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:27.763 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2024-04-25T14:04:27.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:27.817 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list-id 2024-04-25T14:04:27.868 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:04:27.868 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-04-25T14:04:27.868 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T14:04:27.868 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T14:04:27.868 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-04-25T14:04:27.958 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:04:27.990 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:27.997 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T14:04:27.997 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:04:27.997 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-04-25T14:04:27.997 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T14:04:27.997 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T14:04:27.997 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T14:04:27.997 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T14:04:27.997 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T14:04:27.998 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-04-25T14:04:28.045 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:04:28.161 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:28.167 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:04:28.167 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-04-25T14:04:28.167 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.0 up 2024-04-25T14:04:28.168 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T14:04:28.168 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-04-25T14:04:28.214 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:04:28.246 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:28.252 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:04:28.252 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T14:04:28.308 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:04:28.308 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:04:28.308 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T14:04:28.359 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T14:04:28.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:28.359 DEBUG:teuthology.orchestra.run.smithi155:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:04:28.460 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:04:28.460 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T14:04:28.460 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:04:29.051 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:04:29.051 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T14:04:29.051 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:04:29.051 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:04:29.051 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:04:29.065 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:04:29.065 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T14:04:29.065 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:04:29.065 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:04:29.065 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:04:29.065 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790" 2024-04-25T14:04:29.066 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790:/ 2024-04-25T14:04:29.066 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb" will pass to kernel 2024-04-25T14:04:29.069 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T14:04:29.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:29.070 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T14:04:29.089 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:04:29.180 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:04:29.192 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:29.199 INFO:teuthology.orchestra.run.smithi155.stdout:client4830 2024-04-25T14:04:29.199 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:04:29.199 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:04:29.208 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:04:29.297 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:04:29.300 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:29.307 INFO:teuthology.orchestra.run.smithi155.stdout:client4830 2024-04-25T14:04:29.307 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:04:29.353 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:04:29.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:29.362 INFO:teuthology.orchestra.run.smithi155.stdout:client4830 2024-04-25T14:04:29.363 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb.client4830/status') 2024-04-25T14:04:29.408 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:04:29.420 INFO:teuthology.orchestra.run.smithi155.stdout:instance: client.4830 (3)172.21.15.155:0/1971324219 2024-04-25T14:04:29.420 INFO:teuthology.orchestra.run.smithi155.stdout:blocklisted: false 2024-04-25T14:04:29.421 INFO:teuthology.orchestra.run.smithi155.stderr:0+1 records in 2024-04-25T14:04:29.421 INFO:teuthology.orchestra.run.smithi155.stderr:0+1 records out 2024-04-25T14:04:29.421 INFO:teuthology.orchestra.run.smithi155.stderr:71 bytes copied, 6.3131e-05 s, 1.1 MB/s 2024-04-25T14:04:29.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:29.428 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-04-25T14:04:29.428 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T14:04:29.429 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T14:04:29.429 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T14:04:29.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.589+0000 7f4031600640 1 Processor -- start 2024-04-25T14:04:29.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.589+0000 7f4031600640 1 -- start start 2024-04-25T14:04:29.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.589+0000 7f4031600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f402c15dc90 0x7f402c164950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:29.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.589+0000 7f4031600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f402c076070 con 0x7f402c164e90 2024-04-25T14:04:29.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.589+0000 7f4031600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f402c0714e0 con 0x7f402c070de0 2024-04-25T14:04:29.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.589+0000 7f4031600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f402c0cfaa0 con 0x7f402c15dc90 2024-04-25T14:04:29.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.589+0000 7f402b400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f402c164e90 0x7f402c1672b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60616/0 (socket says 172.21.15.86:60616) 2024-04-25T14:04:29.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.589+0000 7f402b400640 1 -- 172.21.15.86:0/573997192 learned_addr learned my addr 172.21.15.86:0/573997192 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:29.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.589+0000 7f402a000640 1 --2- 172.21.15.86:0/573997192 >> v2:172.21.15.155:3300/0 conn(0x7f402c15dc90 0x7f402c164950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:29.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.589+0000 7f4029600640 1 -- 172.21.15.86:0/573997192 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2925580330 0 0) 0x7f402c076070 con 0x7f402c164e90 2024-04-25T14:04:29.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.593+0000 7f4029600640 1 -- 172.21.15.86:0/573997192 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4010003530 con 0x7f402c164e90 2024-04-25T14:04:29.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.593+0000 7f4029600640 1 -- 172.21.15.86:0/573997192 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3601814583 0 0) 0x7f402c0714e0 con 0x7f402c070de0 2024-04-25T14:04:29.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.593+0000 7f4029600640 1 -- 172.21.15.86:0/573997192 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f402c076070 con 0x7f402c070de0 2024-04-25T14:04:29.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.593+0000 7f402a000640 1 -- 172.21.15.86:0/573997192 >> v1:172.21.15.155:6789/0 conn(0x7f402c070de0 legacy=0x7f402c15d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:29.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.593+0000 7f402a000640 1 -- 172.21.15.86:0/573997192 >> v1:172.21.15.86:6789/0 conn(0x7f402c164e90 legacy=0x7f402c1672b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:29.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.593+0000 7f402a000640 1 -- 172.21.15.86:0/573997192 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402c0717c0 con 0x7f402c15dc90 2024-04-25T14:04:29.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.593+0000 7f402a000640 1 --2- 172.21.15.86:0/573997192 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f402c15dc90 0x7f402c164950 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f402c0cfc20 tx=0x7f401801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efd03aec605bd5fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:29.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.593+0000 7f4029600640 1 -- 172.21.15.86:0/573997192 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40180aa620 con 0x7f402c15dc90 2024-04-25T14:04:29.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.593+0000 7f4029600640 1 -- 172.21.15.86:0/573997192 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40180af070 con 0x7f402c15dc90 2024-04-25T14:04:29.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.593+0000 7f4029600640 1 -- 172.21.15.86:0/573997192 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40180aa9c0 con 0x7f402c15dc90 2024-04-25T14:04:29.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.597+0000 7f4031600640 1 -- 172.21.15.86:0/573997192 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f402c15dc90 msgr2=0x7f402c164950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:29.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.597+0000 7f4031600640 1 --2- 172.21.15.86:0/573997192 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f402c15dc90 0x7f402c164950 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f402c0cfc20 tx=0x7f401801c4a0 comp rx=0 tx=0).stop 2024-04-25T14:04:29.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.597+0000 7f4031600640 1 -- 172.21.15.86:0/573997192 shutdown_connections 2024-04-25T14:04:29.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.597+0000 7f4031600640 1 --2- 172.21.15.86:0/573997192 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f402c15dc90 0x7f402c164950 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:29.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.597+0000 7f4031600640 1 -- 172.21.15.86:0/573997192 >> 172.21.15.86:0/573997192 conn(0x7f402c0ce620 msgr2=0x7f402c0cea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:29.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.597+0000 7f4031600640 1 -- 172.21.15.86:0/573997192 shutdown_connections 2024-04-25T14:04:29.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.597+0000 7f4031600640 1 -- 172.21.15.86:0/573997192 wait complete. 2024-04-25T14:04:29.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.597+0000 7f4031600640 1 Processor -- start 2024-04-25T14:04:29.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.597+0000 7f4031600640 1 -- start start 2024-04-25T14:04:29.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.597+0000 7f4031600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f402c070de0 0x7f402c141080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:29.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.597+0000 7f4031600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f402c15dc90 0x7f402c13ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:29.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.597+0000 7f4031600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f402c164e90 0x7f402c0b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:29.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.597+0000 7f4031600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f402c15e220 con 0x7f402c15dc90 2024-04-25T14:04:29.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.597+0000 7f4031600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f402c15e0a0 con 0x7f402c164e90 2024-04-25T14:04:29.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.597+0000 7f4031600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f402c0d1220 con 0x7f402c070de0 2024-04-25T14:04:29.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.601+0000 7f402a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f402c15dc90 0x7f402c13ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:29.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.601+0000 7f402a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f402c15dc90 0x7f402c13ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45074/0 (socket says 172.21.15.86:45074) 2024-04-25T14:04:29.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.601+0000 7f402a000640 1 -- 172.21.15.86:0/290740402 learned_addr learned my addr 172.21.15.86:0/290740402 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.601+0000 7f402b400640 1 --2- 172.21.15.86:0/290740402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f402c164e90 0x7f402c0b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.601+0000 7f402aa00640 1 --2- 172.21.15.86:0/290740402 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f402c070de0 0x7f402c141080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.601+0000 7f402a000640 1 -- 172.21.15.86:0/290740402 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f402c070de0 msgr2=0x7f402c141080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.601+0000 7f402a000640 1 --2- 172.21.15.86:0/290740402 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f402c070de0 0x7f402c141080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.601+0000 7f402a000640 1 -- 172.21.15.86:0/290740402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f402c164e90 msgr2=0x7f402c0b4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.601+0000 7f402a000640 1 --2- 172.21.15.86:0/290740402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f402c164e90 0x7f402c0b4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.601+0000 7f402a000640 1 -- 172.21.15.86:0/290740402 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402c0b5bd0 con 0x7f402c15dc90 2024-04-25T14:04:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.601+0000 7f402b400640 1 --2- 172.21.15.86:0/290740402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f402c164e90 0x7f402c0b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.601+0000 7f402a000640 1 --2- 172.21.15.86:0/290740402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f402c15dc90 0x7f402c13ab00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f40180ac040 tx=0x7f4018019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.601+0000 7f4022a00640 1 -- 172.21.15.86:0/290740402 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4018003820 con 0x7f402c15dc90 2024-04-25T14:04:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.601+0000 7f4022a00640 1 -- 172.21.15.86:0/290740402 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40180039c0 con 0x7f402c15dc90 2024-04-25T14:04:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.601+0000 7f4031600640 1 -- 172.21.15.86:0/290740402 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f402c0b5eb0 con 0x7f402c15dc90 2024-04-25T14:04:29.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.601+0000 7f4022a00640 1 -- 172.21.15.86:0/290740402 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4018003d90 con 0x7f402c15dc90 2024-04-25T14:04:29.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.601+0000 7f4031600640 1 -- 172.21.15.86:0/290740402 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f402c15e7b0 con 0x7f402c15dc90 2024-04-25T14:04:29.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.601+0000 7f4031600640 1 -- 172.21.15.86:0/290740402 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f402c0b5940 con 0x7f402c15dc90 2024-04-25T14:04:29.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.601+0000 7f4022a00640 1 -- 172.21.15.86:0/290740402 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f40180c2060 con 0x7f402c15dc90 2024-04-25T14:04:29.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.605+0000 7f4022a00640 1 --2- 172.21.15.86:0/290740402 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f400c06f430 0x7f400c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:29.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.605+0000 7f4022a00640 1 -- 172.21.15.86:0/290740402 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f40181293b0 con 0x7f402c15dc90 2024-04-25T14:04:29.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.605+0000 7f402aa00640 1 --2- 172.21.15.86:0/290740402 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f400c06f430 0x7f400c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:29.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.605+0000 7f402aa00640 1 --2- 172.21.15.86:0/290740402 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f400c06f430 0x7f400c071940 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f401c0011f0 tx=0x7f401c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:29.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.609+0000 7f4022a00640 1 -- 172.21.15.86:0/290740402 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4018016090 con 0x7f402c15dc90 2024-04-25T14:04:29.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.801+0000 7f4031600640 1 -- 172.21.15.86:0/290740402 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f402c15fd50 con 0x7f402c15dc90 2024-04-25T14:04:29.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.801+0000 7f4022a00640 1 -- 172.21.15.86:0/290740402 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f40180f60f0 con 0x7f402c15dc90 2024-04-25T14:04:29.804 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T14:04:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.805+0000 7f4031600640 1 -- 172.21.15.86:0/290740402 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f400c06f430 msgr2=0x7f400c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.805+0000 7f4031600640 1 --2- 172.21.15.86:0/290740402 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f400c06f430 0x7f400c071940 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f401c0011f0 tx=0x7f401c009040 comp rx=0 tx=0).stop 2024-04-25T14:04:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.805+0000 7f4031600640 1 -- 172.21.15.86:0/290740402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f402c15dc90 msgr2=0x7f402c13ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.805+0000 7f4031600640 1 --2- 172.21.15.86:0/290740402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f402c15dc90 0x7f402c13ab00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f40180ac040 tx=0x7f4018019240 comp rx=0 tx=0).stop 2024-04-25T14:04:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.805+0000 7f4031600640 1 -- 172.21.15.86:0/290740402 shutdown_connections 2024-04-25T14:04:29.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.805+0000 7f4031600640 1 --2- 172.21.15.86:0/290740402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f402c164e90 0x7f402c0b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:29.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.805+0000 7f4031600640 1 --2- 172.21.15.86:0/290740402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f402c15dc90 0x7f402c13ab00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:29.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.805+0000 7f4031600640 1 --2- 172.21.15.86:0/290740402 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f400c06f430 0x7f400c071940 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:29.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.805+0000 7f4031600640 1 --2- 172.21.15.86:0/290740402 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f402c070de0 0x7f402c141080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:29.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.805+0000 7f4031600640 1 -- 172.21.15.86:0/290740402 >> 172.21.15.86:0/290740402 conn(0x7f402c0ce620 msgr2=0x7f402c1421f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:29.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.805+0000 7f4031600640 1 -- 172.21.15.86:0/290740402 shutdown_connections 2024-04-25T14:04:29.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:29.805+0000 7f4031600640 1 -- 172.21.15.86:0/290740402 wait complete. 2024-04-25T14:04:29.826 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T14:04:29.826 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T14:04:29.826 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2024-04-25T14:04:29.826 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T14:04:29.827 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T14:04:29.827 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T14:04:29.827 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T14:04:29.827 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T14:04:29.827 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T14:04:29.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:29.827 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2024-04-25T14:04:29.836 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:04:29.836 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:04:29.836 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:04:29.836 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:29.836 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2024-04-25T14:04:29.836 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:29.836 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:04:29.836 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2024-04-25T14:04:29.836 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f0 2024-04-25T14:04:29.836 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:04:29.836 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2024-04-25T14:04:29.837 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f1 2024-04-25T14:04:29.837 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:04:29.837 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2024-04-25T14:04:29.837 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T14:04:29.837 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2024-04-25T14:04:29.837 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T14:04:29.837 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 42388sec preferred_lft 42388sec 2024-04-25T14:04:29.837 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::ec4:7aff:fe88:6ded/64 scope link 2024-04-25T14:04:29.837 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:29.837 INFO:teuthology.orchestra.run.smithi155.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T14:04:29.837 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 02:42:b1:ff:60:50 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:04:29.837 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T14:04:29.837 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:29.838 INFO:teuthology.orchestra.run.smithi155.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T14:04:29.838 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T14:04:29.838 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T14:04:29.838 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:29.838 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link tentative 2024-04-25T14:04:29.838 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:29.838 INFO:teuthology.orchestra.run.smithi155.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:04:29.838 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T14:04:29.838 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2024-04-25T14:04:29.838 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:29.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:29.839 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T14:04:29.897 INFO:teuthology.orchestra.run.smithi155.stdout:1 2024-04-25T14:04:29.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:29.899 DEBUG:teuthology.orchestra.run.smithi155:> route 2024-04-25T14:04:29.952 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2024-04-25T14:04:29.952 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T14:04:29.952 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T14:04:29.952 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T14:04:29.952 INFO:teuthology.orchestra.run.smithi155.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T14:04:29.952 INFO:teuthology.orchestra.run.smithi155.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T14:04:29.952 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T14:04:29.952 INFO:teuthology.orchestra.run.smithi155.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:04:29.953 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:04:29.953 INFO:teuthology.orchestra.run.smithi155.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:04:29.953 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T14:04:29.953 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:04:29.953 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-04-25T14:04:29.953 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T14:04:29.953 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T14:04:29.953 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T14:04:29.953 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-04-25T14:04:30.042 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:04:30.083 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:30.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:30.090 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2024-04-25T14:04:30.099 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T14:04:30.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:30.101 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list-id 2024-04-25T14:04:30.152 INFO:teuthology.orchestra.run.smithi155.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T14:04:30.152 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:04:30.152 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-04-25T14:04:30.152 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T14:04:30.152 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T14:04:30.152 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-04-25T14:04:30.241 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:04:30.271 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:30.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:30.278 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T14:04:30.298 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:04:30.298 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:04:30.298 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:04:30.298 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:30.298 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2024-04-25T14:04:30.298 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:30.299 INFO:teuthology.orchestra.run.smithi155.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T14:04:30.299 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether aa:d1:58:ed:ba:e6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T14:04:30.299 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T14:04:30.299 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:30.299 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::a8d1:58ff:feed:bae6/64 scope link 2024-04-25T14:04:30.299 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:30.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:30.300 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T14:04:30.364 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:04:30.365 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:04:30.365 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:04:30.365 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:30.365 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2024-04-25T14:04:30.365 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:30.365 INFO:teuthology.orchestra.run.smithi155.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T14:04:30.365 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether aa:d1:58:ed:ba:e6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T14:04:30.365 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T14:04:30.365 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:30.365 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::a8d1:58ff:feed:bae6/64 scope link 2024-04-25T14:04:30.365 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:04:30.366 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T14:04:30.367 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:04:30.367 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-04-25T14:04:30.367 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T14:04:30.367 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T14:04:30.367 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T14:04:30.367 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T14:04:30.367 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T14:04:30.367 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-04-25T14:04:30.454 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:04:30.539 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:30.546 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:04:30.546 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-04-25T14:04:30.546 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.1 up 2024-04-25T14:04:30.546 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T14:04:30.547 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-04-25T14:04:30.596 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:04:30.625 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:30.632 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:04:30.632 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T14:04:30.683 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T14:04:30.684 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:04:30.684 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T14:04:30.735 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T14:04:30.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:30.735 DEBUG:teuthology.orchestra.run.smithi155:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T14:04:30.839 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T14:04:30.839 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T14:04:30.839 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:04:30.858 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T14:04:30.858 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T14:04:30.858 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:04:30.858 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:04:30.858 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:04:30.868 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T14:04:30.868 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T14:04:30.868 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:04:30.869 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:04:30.869 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:04:30.869 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790" 2024-04-25T14:04:30.869 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790:/ 2024-04-25T14:04:30.869 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb" will pass to kernel 2024-04-25T14:04:30.872 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T14:04:30.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:30.873 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T14:04:30.891 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:04:30.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:04:30.986 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:30.992 INFO:teuthology.orchestra.run.smithi155.stdout:client4640 2024-04-25T14:04:30.992 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:04:30.992 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:04:31.001 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:04:31.088 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:04:31.091 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:31.098 INFO:teuthology.orchestra.run.smithi155.stdout:client4640 2024-04-25T14:04:31.098 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:04:31.144 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:04:31.148 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:31.154 INFO:teuthology.orchestra.run.smithi155.stdout:client4640 2024-04-25T14:04:31.155 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb.client4640/status') 2024-04-25T14:04:31.241 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:04:31.254 INFO:teuthology.orchestra.run.smithi155.stdout:instance: client.4640 (3)192.168.0.2:0/2640183059 2024-04-25T14:04:31.254 INFO:teuthology.orchestra.run.smithi155.stdout:blocklisted: false 2024-04-25T14:04:31.254 INFO:teuthology.orchestra.run.smithi155.stderr:0+1 records in 2024-04-25T14:04:31.254 INFO:teuthology.orchestra.run.smithi155.stderr:0+1 records out 2024-04-25T14:04:31.254 INFO:teuthology.orchestra.run.smithi155.stderr:69 bytes copied, 4.8046e-05 s, 1.4 MB/s 2024-04-25T14:04:31.255 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:04:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:04:31.263 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T14:04:31.280 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:04:31.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.437+0000 7f0dbca00640 1 Processor -- start 2024-04-25T14:04:31.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.437+0000 7f0dbca00640 1 -- start start 2024-04-25T14:04:31.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.437+0000 7f0dbca00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0db80cfca0 0x7f0db806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:31.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.437+0000 7f0dbca00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0db80d16e0 0x7f0db80d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:31.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.437+0000 7f0dbca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0db806c670 0x7f0db806ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:31.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.437+0000 7f0dbca00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0db8070470 con 0x7f0db806c670 2024-04-25T14:04:31.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.437+0000 7f0dbca00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0db80d2030 con 0x7f0db80d16e0 2024-04-25T14:04:31.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.437+0000 7f0dbca00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0db8072bd0 con 0x7f0db80cfca0 2024-04-25T14:04:31.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.437+0000 7f0db6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0db806c670 0x7f0db806ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:31.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.437+0000 7f0db6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0db806c670 0x7f0db806ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45084/0 (socket says 172.21.15.86:45084) 2024-04-25T14:04:31.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.437+0000 7f0db6800640 1 -- 172.21.15.86:0/3196069237 learned_addr learned my addr 172.21.15.86:0/3196069237 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:31.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.437+0000 7f0db5e00640 1 --2- 172.21.15.86:0/3196069237 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0db80cfca0 0x7f0db806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:31.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.437+0000 7f0db5400640 1 --2- 172.21.15.86:0/3196069237 >> v2:172.21.15.155:3300/0 conn(0x7f0db80d16e0 0x7f0db80d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:31.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.437+0000 7f0db6800640 1 -- 172.21.15.86:0/3196069237 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0db80cfca0 msgr2=0x7f0db806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:31.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.437+0000 7f0db6800640 1 --2- 172.21.15.86:0/3196069237 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0db80cfca0 0x7f0db806f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:31.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.437+0000 7f0db6800640 1 -- 172.21.15.86:0/3196069237 >> v2:172.21.15.155:3300/0 conn(0x7f0db80d16e0 msgr2=0x7f0db80d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:31.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.437+0000 7f0db6800640 1 --2- 172.21.15.86:0/3196069237 >> v2:172.21.15.155:3300/0 conn(0x7f0db80d16e0 0x7f0db80d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:31.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.437+0000 7f0db6800640 1 -- 172.21.15.86:0/3196069237 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db806f930 con 0x7f0db806c670 2024-04-25T14:04:31.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.441+0000 7f0db6800640 1 --2- 172.21.15.86:0/3196069237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0db806c670 0x7f0db806ca80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0db8070f40 tx=0x7f0db001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d22dbc7228f496 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:31.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.441+0000 7f0db4a00640 1 -- 172.21.15.86:0/3196069237 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db00a5780 con 0x7f0db806c670 2024-04-25T14:04:31.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.441+0000 7f0db4a00640 1 -- 172.21.15.86:0/3196069237 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0db00a5920 con 0x7f0db806c670 2024-04-25T14:04:31.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.441+0000 7f0db4a00640 1 -- 172.21.15.86:0/3196069237 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db00a5cf0 con 0x7f0db806c670 2024-04-25T14:04:31.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.441+0000 7f0dbca00640 1 -- 172.21.15.86:0/3196069237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0db806c670 msgr2=0x7f0db806ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:31.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.441+0000 7f0dbca00640 1 --2- 172.21.15.86:0/3196069237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0db806c670 0x7f0db806ca80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0db8070f40 tx=0x7f0db001c9f0 comp rx=0 tx=0).stop 2024-04-25T14:04:31.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.441+0000 7f0dbca00640 1 -- 172.21.15.86:0/3196069237 shutdown_connections 2024-04-25T14:04:31.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.441+0000 7f0dbca00640 1 --2- 172.21.15.86:0/3196069237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0db806c670 0x7f0db806ca80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:31.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.441+0000 7f0dbca00640 1 --2- 172.21.15.86:0/3196069237 >> v2:172.21.15.155:3300/0 conn(0x7f0db80d16e0 0x7f0db80d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:31.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.441+0000 7f0dbca00640 1 --2- 172.21.15.86:0/3196069237 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0db80cfca0 0x7f0db806f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:31.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.441+0000 7f0dbca00640 1 -- 172.21.15.86:0/3196069237 >> 172.21.15.86:0/3196069237 conn(0x7f0db80b4650 msgr2=0x7f0db80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:31.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.441+0000 7f0dbca00640 1 -- 172.21.15.86:0/3196069237 shutdown_connections 2024-04-25T14:04:31.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.441+0000 7f0dbca00640 1 -- 172.21.15.86:0/3196069237 wait complete. 2024-04-25T14:04:31.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.445+0000 7f0dbca00640 1 Processor -- start 2024-04-25T14:04:31.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.445+0000 7f0dbca00640 1 -- start start 2024-04-25T14:04:31.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.445+0000 7f0dbca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0db806c670 0x7f0db815f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:31.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.445+0000 7f0dbca00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0db80cfca0 0x7f0db8201990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:31.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.445+0000 7f0dbca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0db80d16e0 0x7f0db8201ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:31.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.445+0000 7f0dbca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0db80738c0 con 0x7f0db806c670 2024-04-25T14:04:31.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.445+0000 7f0dbca00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0db806d3d0 con 0x7f0db80d16e0 2024-04-25T14:04:31.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.445+0000 7f0dbca00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0db8073a40 con 0x7f0db80cfca0 2024-04-25T14:04:31.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.445+0000 7f0db5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0db806c670 0x7f0db815f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:31.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.445+0000 7f0db5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0db806c670 0x7f0db815f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45094/0 (socket says 172.21.15.86:45094) 2024-04-25T14:04:31.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.445+0000 7f0db5e00640 1 -- 172.21.15.86:0/53613638 learned_addr learned my addr 172.21.15.86:0/53613638 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:31.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.445+0000 7f0db5400640 1 --2- 172.21.15.86:0/53613638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0db80cfca0 0x7f0db8201990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:31.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.445+0000 7f0db6800640 1 --2- 172.21.15.86:0/53613638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0db80d16e0 0x7f0db8201ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:31.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.445+0000 7f0db5e00640 1 -- 172.21.15.86:0/53613638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0db80cfca0 msgr2=0x7f0db8201990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:31.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.445+0000 7f0db5e00640 1 --2- 172.21.15.86:0/53613638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0db80cfca0 0x7f0db8201990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:31.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.445+0000 7f0db5e00640 1 -- 172.21.15.86:0/53613638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0db80d16e0 msgr2=0x7f0db8201ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:31.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.445+0000 7f0db5e00640 1 --2- 172.21.15.86:0/53613638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0db80d16e0 0x7f0db8201ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:31.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.445+0000 7f0db5e00640 1 -- 172.21.15.86:0/53613638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db80cbf10 con 0x7f0db806c670 2024-04-25T14:04:31.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.445+0000 7f0db5e00640 1 --2- 172.21.15.86:0/53613638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0db806c670 0x7f0db815f9f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0da800d5d0 tx=0x7f0da800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:31.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.449+0000 7f0dae000640 1 -- 172.21.15.86:0/53613638 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da8004240 con 0x7f0db806c670 2024-04-25T14:04:31.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.449+0000 7f0dae000640 1 -- 172.21.15.86:0/53613638 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da80043e0 con 0x7f0db806c670 2024-04-25T14:04:31.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.449+0000 7f0dae000640 1 -- 172.21.15.86:0/53613638 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da8003840 con 0x7f0db806c670 2024-04-25T14:04:31.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.449+0000 7f0dbca00640 1 -- 172.21.15.86:0/53613638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db815d0f0 con 0x7f0db806c670 2024-04-25T14:04:31.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.449+0000 7f0dbca00640 1 -- 172.21.15.86:0/53613638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db815d840 con 0x7f0db806c670 2024-04-25T14:04:31.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.449+0000 7f0dbca00640 1 -- 172.21.15.86:0/53613638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0db8069d00 con 0x7f0db806c670 2024-04-25T14:04:31.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.449+0000 7f0dae000640 1 -- 172.21.15.86:0/53613638 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0da80039e0 con 0x7f0db806c670 2024-04-25T14:04:31.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.453+0000 7f0dae000640 1 --2- 172.21.15.86:0/53613638 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0d9806f400 0x7f0d98071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:31.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.453+0000 7f0dae000640 1 -- 172.21.15.86:0/53613638 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6735+0+0 (secure 0 0 0) 0x7f0da8090190 con 0x7f0db806c670 2024-04-25T14:04:31.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.453+0000 7f0db5400640 1 --2- 172.21.15.86:0/53613638 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0d9806f400 0x7f0d98071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:31.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.453+0000 7f0db5400640 1 --2- 172.21.15.86:0/53613638 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0d9806f400 0x7f0d98071910 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0db80b0310 tx=0x7f0da4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:31.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.457+0000 7f0dae000640 1 -- 172.21.15.86:0/53613638 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0da805c7a0 con 0x7f0db806c670 2024-04-25T14:04:31.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.617+0000 7f0dbca00640 1 -- 172.21.15.86:0/53613638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0db8076bf0 con 0x7f0db806c670 2024-04-25T14:04:31.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.617+0000 7f0dae000640 1 -- 172.21.15.86:0/53613638 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18902 (secure 0 0 0) 0x7f0da8060aa0 con 0x7f0db806c670 2024-04-25T14:04:31.619 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:31.619 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":25,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:04:30.925427+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:03:49.151473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:03:49.802597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T14:03:53.492008+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:04:31.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.621+0000 7f0dbca00640 1 -- 172.21.15.86:0/53613638 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0d9806f400 msgr2=0x7f0d98071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:31.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.621+0000 7f0dbca00640 1 --2- 172.21.15.86:0/53613638 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0d9806f400 0x7f0d98071910 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0db80b0310 tx=0x7f0da4008040 comp rx=0 tx=0).stop 2024-04-25T14:04:31.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.621+0000 7f0dbca00640 1 -- 172.21.15.86:0/53613638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0db806c670 msgr2=0x7f0db815f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:31.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.621+0000 7f0dbca00640 1 --2- 172.21.15.86:0/53613638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0db806c670 0x7f0db815f9f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0da800d5d0 tx=0x7f0da800da90 comp rx=0 tx=0).stop 2024-04-25T14:04:31.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.621+0000 7f0dbca00640 1 -- 172.21.15.86:0/53613638 shutdown_connections 2024-04-25T14:04:31.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.621+0000 7f0dbca00640 1 --2- 172.21.15.86:0/53613638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0db80d16e0 0x7f0db8201ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:31.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.621+0000 7f0dbca00640 1 --2- 172.21.15.86:0/53613638 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0d9806f400 0x7f0d98071910 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:31.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.621+0000 7f0dbca00640 1 --2- 172.21.15.86:0/53613638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0db80cfca0 0x7f0db8201990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:31.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.621+0000 7f0dbca00640 1 --2- 172.21.15.86:0/53613638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0db806c670 0x7f0db815f9f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:31.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.621+0000 7f0dbca00640 1 -- 172.21.15.86:0/53613638 >> 172.21.15.86:0/53613638 conn(0x7f0db80b4650 msgr2=0x7f0db815a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:31.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.621+0000 7f0dbca00640 1 -- 172.21.15.86:0/53613638 shutdown_connections 2024-04-25T14:04:31.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.621+0000 7f0dbca00640 1 -- 172.21.15.86:0/53613638 wait complete. 2024-04-25T14:04:31.641 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:03:49.151473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:03:49.802597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T14:03:53.492008+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T14:04:31.642 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:04:31.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.789+0000 7fe783200640 1 Processor -- start 2024-04-25T14:04:31.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.789+0000 7fe783200640 1 -- start start 2024-04-25T14:04:31.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.793+0000 7fe783200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe7840d1c70 0x7fe7840d2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:31.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.793+0000 7fe783200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe784160ec0 con 0x7fe7840b7480 2024-04-25T14:04:31.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.793+0000 7fe783200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7840d1190 con 0x7fe784070e70 2024-04-25T14:04:31.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.793+0000 7fe783200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe7841611a0 con 0x7fe7840d1c70 2024-04-25T14:04:31.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.793+0000 7fe782800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe7840b7480 0x7fe7840b7890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60622/0 (socket says 172.21.15.86:60622) 2024-04-25T14:04:31.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.793+0000 7fe782800640 1 -- 172.21.15.86:0/4094753675 learned_addr learned my addr 172.21.15.86:0/4094753675 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:31.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.793+0000 7fe781e00640 1 --2- 172.21.15.86:0/4094753675 >> v2:172.21.15.155:3300/0 conn(0x7fe7840d1c70 0x7fe7840d2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:31.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.793+0000 7fe781400640 1 -- 172.21.15.86:0/4094753675 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 302248143 0 0) 0x7fe784160ec0 con 0x7fe7840b7480 2024-04-25T14:04:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.793+0000 7fe781400640 1 -- 172.21.15.86:0/4094753675 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe768003530 con 0x7fe7840b7480 2024-04-25T14:04:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.793+0000 7fe781400640 1 -- 172.21.15.86:0/4094753675 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4122043973 0 0) 0x7fe7840d1190 con 0x7fe784070e70 2024-04-25T14:04:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.793+0000 7fe781400640 1 -- 172.21.15.86:0/4094753675 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe784160ec0 con 0x7fe784070e70 2024-04-25T14:04:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.793+0000 7fe781e00640 1 -- 172.21.15.86:0/4094753675 >> v1:172.21.15.155:6789/0 conn(0x7fe784070e70 legacy=0x7fe784071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.793+0000 7fe781e00640 1 -- 172.21.15.86:0/4094753675 >> v1:172.21.15.86:6789/0 conn(0x7fe7840b7480 legacy=0x7fe7840b7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.793+0000 7fe781e00640 1 -- 172.21.15.86:0/4094753675 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7840d1470 con 0x7fe7840d1c70 2024-04-25T14:04:31.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.797+0000 7fe781e00640 1 --2- 172.21.15.86:0/4094753675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7840d1c70 0x7fe7840d2080 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe774009d90 tx=0x7fe77401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dc758563dd543b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:31.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.797+0000 7fe781400640 1 -- 172.21.15.86:0/4094753675 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7740b2070 con 0x7fe7840d1c70 2024-04-25T14:04:31.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.797+0000 7fe781400640 1 -- 172.21.15.86:0/4094753675 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7740a5940 con 0x7fe7840d1c70 2024-04-25T14:04:31.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.797+0000 7fe781400640 1 -- 172.21.15.86:0/4094753675 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7740a5cf0 con 0x7fe7840d1c70 2024-04-25T14:04:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.797+0000 7fe783200640 1 -- 172.21.15.86:0/4094753675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7840d1c70 msgr2=0x7fe7840d2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.797+0000 7fe783200640 1 --2- 172.21.15.86:0/4094753675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7840d1c70 0x7fe7840d2080 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe774009d90 tx=0x7fe77401c970 comp rx=0 tx=0).stop 2024-04-25T14:04:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.797+0000 7fe783200640 1 -- 172.21.15.86:0/4094753675 shutdown_connections 2024-04-25T14:04:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.797+0000 7fe783200640 1 --2- 172.21.15.86:0/4094753675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7840d1c70 0x7fe7840d2080 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.797+0000 7fe783200640 1 -- 172.21.15.86:0/4094753675 >> 172.21.15.86:0/4094753675 conn(0x7fe7840cfea0 msgr2=0x7fe7840d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.797+0000 7fe783200640 1 -- 172.21.15.86:0/4094753675 shutdown_connections 2024-04-25T14:04:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.797+0000 7fe783200640 1 -- 172.21.15.86:0/4094753675 wait complete. 2024-04-25T14:04:31.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.801+0000 7fe783200640 1 Processor -- start 2024-04-25T14:04:31.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.801+0000 7fe783200640 1 -- start start 2024-04-25T14:04:31.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.801+0000 7fe783200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe784070e70 0x7fe78414efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:31.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.801+0000 7fe783200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7840b7480 0x7fe7840c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.801+0000 7fe783200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7840d1c70 0x7fe784156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.801+0000 7fe783200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe78416d7e0 con 0x7fe784070e70 2024-04-25T14:04:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.801+0000 7fe783200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe784160bb0 con 0x7fe7840b7480 2024-04-25T14:04:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.801+0000 7fe783200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe78416d960 con 0x7fe7840d1c70 2024-04-25T14:04:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.801+0000 7fe781e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe784070e70 0x7fe78414efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.801+0000 7fe781e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe784070e70 0x7fe78414efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45100/0 (socket says 172.21.15.86:45100) 2024-04-25T14:04:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.801+0000 7fe781e00640 1 -- 172.21.15.86:0/3952245040 learned_addr learned my addr 172.21.15.86:0/3952245040 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.801+0000 7fe782800640 1 --2- 172.21.15.86:0/3952245040 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7840d1c70 0x7fe784156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.801+0000 7fe779400640 1 --2- 172.21.15.86:0/3952245040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7840b7480 0x7fe7840c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.801+0000 7fe781e00640 1 -- 172.21.15.86:0/3952245040 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7840d1c70 msgr2=0x7fe784156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.801+0000 7fe781e00640 1 --2- 172.21.15.86:0/3952245040 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7840d1c70 0x7fe784156cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.801+0000 7fe781e00640 1 -- 172.21.15.86:0/3952245040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7840b7480 msgr2=0x7fe7840c77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.801+0000 7fe781e00640 1 --2- 172.21.15.86:0/3952245040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7840b7480 0x7fe7840c77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.801+0000 7fe781e00640 1 -- 172.21.15.86:0/3952245040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe784158340 con 0x7fe784070e70 2024-04-25T14:04:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.801+0000 7fe779400640 1 --2- 172.21.15.86:0/3952245040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7840b7480 0x7fe7840c77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.805+0000 7fe781e00640 1 --2- 172.21.15.86:0/3952245040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe784070e70 0x7fe78414efc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe77401a5a0 tx=0x7fe774019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.805+0000 7fe77aa00640 1 -- 172.21.15.86:0/3952245040 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7740b2040 con 0x7fe784070e70 2024-04-25T14:04:31.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.805+0000 7fe77aa00640 1 -- 172.21.15.86:0/3952245040 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7740bf070 con 0x7fe784070e70 2024-04-25T14:04:31.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.805+0000 7fe77aa00640 1 -- 172.21.15.86:0/3952245040 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe774009450 con 0x7fe784070e70 2024-04-25T14:04:31.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.805+0000 7fe783200640 1 -- 172.21.15.86:0/3952245040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe784157d80 con 0x7fe784070e70 2024-04-25T14:04:31.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.805+0000 7fe783200640 1 -- 172.21.15.86:0/3952245040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7840b4750 con 0x7fe784070e70 2024-04-25T14:04:31.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.805+0000 7fe783200640 1 -- 172.21.15.86:0/3952245040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe784076b80 con 0x7fe784070e70 2024-04-25T14:04:31.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.805+0000 7fe77aa00640 1 -- 172.21.15.86:0/3952245040 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe774004040 con 0x7fe784070e70 2024-04-25T14:04:31.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.805+0000 7fe77aa00640 1 --2- 172.21.15.86:0/3952245040 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe76006f400 0x7fe760071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:31.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.805+0000 7fe77aa00640 1 -- 172.21.15.86:0/3952245040 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6735+0+0 (secure 0 0 0) 0x7fe77412b290 con 0x7fe784070e70 2024-04-25T14:04:31.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.805+0000 7fe779400640 1 --2- 172.21.15.86:0/3952245040 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe76006f400 0x7fe760071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:31.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.809+0000 7fe779400640 1 --2- 172.21.15.86:0/3952245040 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe76006f400 0x7fe760071910 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe7840ca230 tx=0x7fe76c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:31.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.809+0000 7fe77aa00640 1 -- 172.21.15.86:0/3952245040 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe774016560 con 0x7fe784070e70 2024-04-25T14:04:31.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.977+0000 7fe783200640 1 -- 172.21.15.86:0/3952245040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe784069d00 con 0x7fe784070e70 2024-04-25T14:04:31.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.977+0000 7fe77aa00640 1 -- 172.21.15.86:0/3952245040 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fe7740f7df0 con 0x7fe784070e70 2024-04-25T14:04:31.978 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:04:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.977+0000 7fe783200640 1 -- 172.21.15.86:0/3952245040 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe76006f400 msgr2=0x7fe760071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.977+0000 7fe783200640 1 --2- 172.21.15.86:0/3952245040 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe76006f400 0x7fe760071910 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe7840ca230 tx=0x7fe76c009040 comp rx=0 tx=0).stop 2024-04-25T14:04:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.977+0000 7fe783200640 1 -- 172.21.15.86:0/3952245040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe784070e70 msgr2=0x7fe78414efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.977+0000 7fe783200640 1 --2- 172.21.15.86:0/3952245040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe784070e70 0x7fe78414efc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe77401a5a0 tx=0x7fe774019710 comp rx=0 tx=0).stop 2024-04-25T14:04:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.977+0000 7fe783200640 1 -- 172.21.15.86:0/3952245040 shutdown_connections 2024-04-25T14:04:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.977+0000 7fe783200640 1 --2- 172.21.15.86:0/3952245040 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe76006f400 0x7fe760071910 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:31.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.977+0000 7fe783200640 1 --2- 172.21.15.86:0/3952245040 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7840d1c70 0x7fe784156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:31.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.977+0000 7fe783200640 1 --2- 172.21.15.86:0/3952245040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7840b7480 0x7fe7840c77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:31.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.977+0000 7fe783200640 1 --2- 172.21.15.86:0/3952245040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe784070e70 0x7fe78414efc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:31.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.977+0000 7fe783200640 1 -- 172.21.15.86:0/3952245040 >> 172.21.15.86:0/3952245040 conn(0x7fe7840cfea0 msgr2=0x7fe784152f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:31.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.977+0000 7fe783200640 1 -- 172.21.15.86:0/3952245040 shutdown_connections 2024-04-25T14:04:31.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:31.977+0000 7fe783200640 1 -- 172.21.15.86:0/3952245040 wait complete. 2024-04-25T14:04:32.000 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:04:32.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.149+0000 7f0a91000640 1 Processor -- start 2024-04-25T14:04:32.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.149+0000 7f0a91000640 1 -- start start 2024-04-25T14:04:32.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.153+0000 7f0a91000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a8c0b59e0 0x7f0a8c0b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:32.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.153+0000 7f0a91000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0a8c0aeea0 0x7f0a8c0af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:32.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.153+0000 7f0a91000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a8c0b6330 con 0x7f0a8c15faa0 2024-04-25T14:04:32.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.153+0000 7f0a91000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0a8c06d560 con 0x7f0a8c0aeea0 2024-04-25T14:04:32.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.153+0000 7f0a91000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0a8c06c670 con 0x7f0a8c0b59e0 2024-04-25T14:04:32.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.153+0000 7f0a8a800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0a8c0aeea0 0x7f0a8c0af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:32.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.153+0000 7f0a89400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a8c0b59e0 0x7f0a8c0b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:32.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.153+0000 7f0a89e00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f0a8c15faa0 0x7f0a8c06d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:38074/0 (socket says 172.21.15.86:38074) 2024-04-25T14:04:32.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.153+0000 7f0a8a800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0a8c0aeea0 0x7f0a8c0af2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:54064/0 (socket says 172.21.15.86:54064) 2024-04-25T14:04:32.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.153+0000 7f0a89e00640 1 -- 172.21.15.86:0/4025246666 learned_addr learned my addr 172.21.15.86:0/4025246666 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:32.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.153+0000 7f0a8a800640 1 -- 172.21.15.86:0/4025246666 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a8c0b59e0 msgr2=0x7f0a8c0b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:32.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.153+0000 7f0a8a800640 1 --2- 172.21.15.86:0/4025246666 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a8c0b59e0 0x7f0a8c0b5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:32.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.153+0000 7f0a8a800640 1 -- 172.21.15.86:0/4025246666 >> v1:172.21.15.155:6789/0 conn(0x7f0a8c15faa0 legacy=0x7f0a8c06d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:32.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.153+0000 7f0a8a800640 1 -- 172.21.15.86:0/4025246666 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8c06c950 con 0x7f0a8c0aeea0 2024-04-25T14:04:32.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.157+0000 7f0a8a800640 1 --2- 172.21.15.86:0/4025246666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a8c0aeea0 0x7f0a8c0af2b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0a840097c0 tx=0x7f0a8401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecb4f5c98618edaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:32.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.157+0000 7f0a88a00640 1 -- 172.21.15.86:0/4025246666 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a840aa620 con 0x7f0a8c0aeea0 2024-04-25T14:04:32.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.157+0000 7f0a88a00640 1 -- 172.21.15.86:0/4025246666 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a840af070 con 0x7f0a8c0aeea0 2024-04-25T14:04:32.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.157+0000 7f0a88a00640 1 -- 172.21.15.86:0/4025246666 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a840aa9e0 con 0x7f0a8c0aeea0 2024-04-25T14:04:32.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.157+0000 7f0a91000640 1 -- 172.21.15.86:0/4025246666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a8c0aeea0 msgr2=0x7f0a8c0af2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:32.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.157+0000 7f0a91000640 1 --2- 172.21.15.86:0/4025246666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a8c0aeea0 0x7f0a8c0af2b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0a840097c0 tx=0x7f0a8401c4a0 comp rx=0 tx=0).stop 2024-04-25T14:04:32.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.157+0000 7f0a91000640 1 -- 172.21.15.86:0/4025246666 shutdown_connections 2024-04-25T14:04:32.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.157+0000 7f0a91000640 1 --2- 172.21.15.86:0/4025246666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a8c0aeea0 0x7f0a8c0af2b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:32.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.157+0000 7f0a91000640 1 --2- 172.21.15.86:0/4025246666 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a8c0b59e0 0x7f0a8c0b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:32.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.157+0000 7f0a91000640 1 -- 172.21.15.86:0/4025246666 >> 172.21.15.86:0/4025246666 conn(0x7f0a8c0d2230 msgr2=0x7f0a8c0d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:32.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.157+0000 7f0a91000640 1 -- 172.21.15.86:0/4025246666 shutdown_connections 2024-04-25T14:04:32.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.157+0000 7f0a91000640 1 -- 172.21.15.86:0/4025246666 wait complete. 2024-04-25T14:04:32.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.157+0000 7f0a91000640 1 Processor -- start 2024-04-25T14:04:32.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.157+0000 7f0a91000640 1 -- start start 2024-04-25T14:04:32.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.161+0000 7f0a91000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a8c0aeea0 0x7f0a8c159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:32.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.161+0000 7f0a91000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a8c0b59e0 0x7f0a8c160c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:32.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.161+0000 7f0a91000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a8c15faa0 0x7f0a8c140510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:32.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.161+0000 7f0a91000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c160030 con 0x7f0a8c15faa0 2024-04-25T14:04:32.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.161+0000 7f0a91000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c15feb0 con 0x7f0a8c0b59e0 2024-04-25T14:04:32.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.161+0000 7f0a91000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0a8c070f30 con 0x7f0a8c0aeea0 2024-04-25T14:04:32.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.161+0000 7f0a8a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a8c15faa0 0x7f0a8c140510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:32.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.161+0000 7f0a8a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a8c15faa0 0x7f0a8c140510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45110/0 (socket says 172.21.15.86:45110) 2024-04-25T14:04:32.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.161+0000 7f0a8a800640 1 -- 172.21.15.86:0/4004929393 learned_addr learned my addr 172.21.15.86:0/4004929393 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:32.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.161+0000 7f0a89e00640 1 --2- 172.21.15.86:0/4004929393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a8c0aeea0 0x7f0a8c159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:32.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.161+0000 7f0a89400640 1 --2- 172.21.15.86:0/4004929393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a8c0b59e0 0x7f0a8c160c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:32.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.161+0000 7f0a8a800640 1 -- 172.21.15.86:0/4004929393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a8c0aeea0 msgr2=0x7f0a8c159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:32.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.161+0000 7f0a8a800640 1 --2- 172.21.15.86:0/4004929393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a8c0aeea0 0x7f0a8c159b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:32.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.161+0000 7f0a8a800640 1 -- 172.21.15.86:0/4004929393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a8c0b59e0 msgr2=0x7f0a8c160c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:32.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.161+0000 7f0a8a800640 1 --2- 172.21.15.86:0/4004929393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a8c0b59e0 0x7f0a8c160c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:32.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.161+0000 7f0a8a800640 1 -- 172.21.15.86:0/4004929393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8c161740 con 0x7f0a8c15faa0 2024-04-25T14:04:32.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.161+0000 7f0a8a800640 1 --2- 172.21.15.86:0/4004929393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a8c15faa0 0x7f0a8c140510 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0a8401ab50 tx=0x7f0a8401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:32.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.161+0000 7f0a82000640 1 -- 172.21.15.86:0/4004929393 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a840aac50 con 0x7f0a8c15faa0 2024-04-25T14:04:32.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.161+0000 7f0a82000640 1 -- 172.21.15.86:0/4004929393 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a840aadf0 con 0x7f0a8c15faa0 2024-04-25T14:04:32.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.161+0000 7f0a91000640 1 -- 172.21.15.86:0/4004929393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a8c161a20 con 0x7f0a8c15faa0 2024-04-25T14:04:32.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.161+0000 7f0a82000640 1 -- 172.21.15.86:0/4004929393 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a840be630 con 0x7f0a8c15faa0 2024-04-25T14:04:32.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.161+0000 7f0a91000640 1 -- 172.21.15.86:0/4004929393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a8c1611c0 con 0x7f0a8c15faa0 2024-04-25T14:04:32.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.161+0000 7f0a91000640 1 -- 172.21.15.86:0/4004929393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a8c15a0c0 con 0x7f0a8c15faa0 2024-04-25T14:04:32.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.165+0000 7f0a82000640 1 -- 172.21.15.86:0/4004929393 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0a840bc020 con 0x7f0a8c15faa0 2024-04-25T14:04:32.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.165+0000 7f0a82000640 1 --2- 172.21.15.86:0/4004929393 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0a6c06f430 0x7f0a6c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:32.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.165+0000 7f0a82000640 1 -- 172.21.15.86:0/4004929393 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f0a84129270 con 0x7f0a8c15faa0 2024-04-25T14:04:32.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.165+0000 7f0a89e00640 1 --2- 172.21.15.86:0/4004929393 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0a6c06f430 0x7f0a6c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:32.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.165+0000 7f0a89e00640 1 --2- 172.21.15.86:0/4004929393 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0a6c06f430 0x7f0a6c071940 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0012f0 tx=0x7f0a7c0019a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:32.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.169+0000 7f0a82000640 1 -- 172.21.15.86:0/4004929393 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0a840c5030 con 0x7f0a8c15faa0 2024-04-25T14:04:32.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.325+0000 7f0a91000640 1 -- 172.21.15.86:0/4004929393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0a8c1614b0 con 0x7f0a8c15faa0 2024-04-25T14:04:32.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.329+0000 7f0a82000640 1 -- 172.21.15.86:0/4004929393 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f0a84016090 con 0x7f0a8c15faa0 2024-04-25T14:04:32.329 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:04:32.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.329+0000 7f0a91000640 1 -- 172.21.15.86:0/4004929393 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0a6c06f430 msgr2=0x7f0a6c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:32.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.329+0000 7f0a91000640 1 --2- 172.21.15.86:0/4004929393 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0a6c06f430 0x7f0a6c071940 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0012f0 tx=0x7f0a7c0019a0 comp rx=0 tx=0).stop 2024-04-25T14:04:32.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.329+0000 7f0a91000640 1 -- 172.21.15.86:0/4004929393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a8c15faa0 msgr2=0x7f0a8c140510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:32.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.329+0000 7f0a91000640 1 --2- 172.21.15.86:0/4004929393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a8c15faa0 0x7f0a8c140510 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0a8401ab50 tx=0x7f0a8401ace0 comp rx=0 tx=0).stop 2024-04-25T14:04:32.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.329+0000 7f0a91000640 1 -- 172.21.15.86:0/4004929393 shutdown_connections 2024-04-25T14:04:32.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.329+0000 7f0a91000640 1 --2- 172.21.15.86:0/4004929393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a8c15faa0 0x7f0a8c140510 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:32.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.329+0000 7f0a91000640 1 --2- 172.21.15.86:0/4004929393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a8c0b59e0 0x7f0a8c160c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:32.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.329+0000 7f0a91000640 1 --2- 172.21.15.86:0/4004929393 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0a6c06f430 0x7f0a6c071940 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:32.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.329+0000 7f0a91000640 1 --2- 172.21.15.86:0/4004929393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a8c0aeea0 0x7f0a8c159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:32.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.329+0000 7f0a91000640 1 -- 172.21.15.86:0/4004929393 >> 172.21.15.86:0/4004929393 conn(0x7f0a8c0d2230 msgr2=0x7f0a8c0d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:32.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.329+0000 7f0a91000640 1 -- 172.21.15.86:0/4004929393 shutdown_connections 2024-04-25T14:04:32.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.329+0000 7f0a91000640 1 -- 172.21.15.86:0/4004929393 wait complete. 2024-04-25T14:04:32.351 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:04:32.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.501+0000 7f3b56a00640 1 Processor -- start 2024-04-25T14:04:32.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.501+0000 7f3b56a00640 1 -- start start 2024-04-25T14:04:32.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.501+0000 7f3b56a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3b500d16e0 0x7f3b500d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:32.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.501+0000 7f3b56a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b500d2030 con 0x7f3b500730b0 2024-04-25T14:04:32.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.501+0000 7f3b56a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b50072bd0 con 0x7f3b500cfca0 2024-04-25T14:04:32.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.501+0000 7f3b56a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3b50070470 con 0x7f3b500d16e0 2024-04-25T14:04:32.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.501+0000 7f3b54a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3b500730b0 0x7f3b500734c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60634/0 (socket says 172.21.15.86:60634) 2024-04-25T14:04:32.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.501+0000 7f3b54a00640 1 -- 172.21.15.86:0/1987656594 learned_addr learned my addr 172.21.15.86:0/1987656594 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:32.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.501+0000 7f3b4f400640 1 --2- 172.21.15.86:0/1987656594 >> v2:172.21.15.155:3300/0 conn(0x7f3b500d16e0 0x7f3b500d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:32.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.501+0000 7f3b4ea00640 1 -- 172.21.15.86:0/1987656594 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3999765386 0 0) 0x7f3b48001120 con 0x7f3b500730b0 2024-04-25T14:04:32.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.501+0000 7f3b4ea00640 1 -- 172.21.15.86:0/1987656594 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b34003530 con 0x7f3b500730b0 2024-04-25T14:04:32.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.501+0000 7f3b4f400640 1 -- 172.21.15.86:0/1987656594 >> v1:172.21.15.155:6789/0 conn(0x7f3b500cfca0 legacy=0x7f3b5006f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:32.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.501+0000 7f3b4f400640 1 -- 172.21.15.86:0/1987656594 >> v1:172.21.15.86:6789/0 conn(0x7f3b500730b0 legacy=0x7f3b500734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:32.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.501+0000 7f3b4f400640 1 -- 172.21.15.86:0/1987656594 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b5006fbd0 con 0x7f3b500d16e0 2024-04-25T14:04:32.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.505+0000 7f3b4f400640 1 --2- 172.21.15.86:0/1987656594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b500d16e0 0x7f3b500d1af0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c0097c0 tx=0x7f3b3c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c72ce54d8c77f2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:32.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.505+0000 7f3b4ea00640 1 -- 172.21.15.86:0/1987656594 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c0b1070 con 0x7f3b500d16e0 2024-04-25T14:04:32.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.505+0000 7f3b4ea00640 1 -- 172.21.15.86:0/1987656594 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b3c0ac410 con 0x7f3b500d16e0 2024-04-25T14:04:32.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.505+0000 7f3b4ea00640 1 -- 172.21.15.86:0/1987656594 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b500743e0 con 0x7f3b500d16e0 2024-04-25T14:04:32.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.505+0000 7f3b56a00640 1 -- 172.21.15.86:0/1987656594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b500d16e0 msgr2=0x7f3b500d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:32.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.505+0000 7f3b56a00640 1 --2- 172.21.15.86:0/1987656594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b500d16e0 0x7f3b500d1af0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c0097c0 tx=0x7f3b3c01c4a0 comp rx=0 tx=0).stop 2024-04-25T14:04:32.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.505+0000 7f3b56a00640 1 -- 172.21.15.86:0/1987656594 shutdown_connections 2024-04-25T14:04:32.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.505+0000 7f3b56a00640 1 --2- 172.21.15.86:0/1987656594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b500d16e0 0x7f3b500d1af0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:32.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.505+0000 7f3b56a00640 1 -- 172.21.15.86:0/1987656594 >> 172.21.15.86:0/1987656594 conn(0x7f3b500b4650 msgr2=0x7f3b500b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:32.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.505+0000 7f3b56a00640 1 -- 172.21.15.86:0/1987656594 shutdown_connections 2024-04-25T14:04:32.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.505+0000 7f3b56a00640 1 -- 172.21.15.86:0/1987656594 wait complete. 2024-04-25T14:04:32.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.509+0000 7f3b56a00640 1 Processor -- start 2024-04-25T14:04:32.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.509+0000 7f3b56a00640 1 -- start start 2024-04-25T14:04:32.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.509+0000 7f3b56a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b500730b0 0x7f3b500cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:32.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.509+0000 7f3b56a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b500cfca0 0x7f3b5009a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:32.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.509+0000 7f3b56a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b500d16e0 0x7f3b501f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:32.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.509+0000 7f3b56a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5016aa80 con 0x7f3b500cfca0 2024-04-25T14:04:32.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.509+0000 7f3b56a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5016a900 con 0x7f3b500d16e0 2024-04-25T14:04:32.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.509+0000 7f3b56a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3b5016ac00 con 0x7f3b500730b0 2024-04-25T14:04:32.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.509+0000 7f3b4f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b500cfca0 0x7f3b5009a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:32.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.509+0000 7f3b4fe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b500730b0 0x7f3b500cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:32.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.509+0000 7f3b54a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b500d16e0 0x7f3b501f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:32.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.509+0000 7f3b4f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b500cfca0 0x7f3b5009a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45112/0 (socket says 172.21.15.86:45112) 2024-04-25T14:04:32.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.509+0000 7f3b4fe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b500730b0 0x7f3b500cac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:50750/0 (socket says 172.21.15.86:50750) 2024-04-25T14:04:32.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.509+0000 7f3b4f400640 1 -- 172.21.15.86:0/1312631573 learned_addr learned my addr 172.21.15.86:0/1312631573 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:32.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.513+0000 7f3b4f400640 1 -- 172.21.15.86:0/1312631573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b500730b0 msgr2=0x7f3b500cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:32.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.513+0000 7f3b4f400640 1 --2- 172.21.15.86:0/1312631573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b500730b0 0x7f3b500cac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:32.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.513+0000 7f3b4f400640 1 -- 172.21.15.86:0/1312631573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b500d16e0 msgr2=0x7f3b501f9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:32.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.513+0000 7f3b4f400640 1 --2- 172.21.15.86:0/1312631573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b500d16e0 0x7f3b501f9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:32.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.513+0000 7f3b4f400640 1 -- 172.21.15.86:0/1312631573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b500ce720 con 0x7f3b500cfca0 2024-04-25T14:04:32.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.513+0000 7f3b4f400640 1 --2- 172.21.15.86:0/1312631573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b500cfca0 0x7f3b5009a830 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c0af040 tx=0x7f3b3c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:32.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.513+0000 7f3b47e00640 1 -- 172.21.15.86:0/1312631573 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c0b1040 con 0x7f3b500cfca0 2024-04-25T14:04:32.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.513+0000 7f3b47e00640 1 -- 172.21.15.86:0/1312631573 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b3c0bb070 con 0x7f3b500cfca0 2024-04-25T14:04:32.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.513+0000 7f3b47e00640 1 -- 172.21.15.86:0/1312631573 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c0b5410 con 0x7f3b500cfca0 2024-04-25T14:04:32.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.513+0000 7f3b56a00640 1 -- 172.21.15.86:0/1312631573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b500afb80 con 0x7f3b500cfca0 2024-04-25T14:04:32.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.513+0000 7f3b56a00640 1 -- 172.21.15.86:0/1312631573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b501fa800 con 0x7f3b500cfca0 2024-04-25T14:04:32.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.513+0000 7f3b45600640 1 -- 172.21.15.86:0/1312631573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b500ce490 con 0x7f3b500cfca0 2024-04-25T14:04:32.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.513+0000 7f3b47e00640 1 -- 172.21.15.86:0/1312631573 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3b3c0b55b0 con 0x7f3b500cfca0 2024-04-25T14:04:32.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.517+0000 7f3b47e00640 1 --2- 172.21.15.86:0/1312631573 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3b3006f400 0x7f3b30071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:32.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.517+0000 7f3b47e00640 1 -- 172.21.15.86:0/1312631573 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f3b3c128c00 con 0x7f3b500cfca0 2024-04-25T14:04:32.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.517+0000 7f3b4fe00640 1 --2- 172.21.15.86:0/1312631573 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3b3006f400 0x7f3b30071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:32.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.517+0000 7f3b4fe00640 1 --2- 172.21.15.86:0/1312631573 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3b3006f400 0x7f3b30071910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3b50070710 tx=0x7f3b40009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:32.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.521+0000 7f3b47e00640 1 -- 172.21.15.86:0/1312631573 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3b3c12f080 con 0x7f3b500cfca0 2024-04-25T14:04:32.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.677+0000 7f3b45600640 1 -- 172.21.15.86:0/1312631573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3b500cb140 con 0x7f3b500cfca0 2024-04-25T14:04:32.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.677+0000 7f3b47e00640 1 -- 172.21.15.86:0/1312631573 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f3b3c016090 con 0x7f3b500cfca0 2024-04-25T14:04:32.679 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:04:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.681+0000 7f3b45600640 1 -- 172.21.15.86:0/1312631573 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3b3006f400 msgr2=0x7f3b30071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.681+0000 7f3b45600640 1 --2- 172.21.15.86:0/1312631573 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3b3006f400 0x7f3b30071910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3b50070710 tx=0x7f3b40009040 comp rx=0 tx=0).stop 2024-04-25T14:04:32.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.681+0000 7f3b45600640 1 -- 172.21.15.86:0/1312631573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b500cfca0 msgr2=0x7f3b5009a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.681+0000 7f3b45600640 1 --2- 172.21.15.86:0/1312631573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b500cfca0 0x7f3b5009a830 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c0af040 tx=0x7f3b3c019240 comp rx=0 tx=0).stop 2024-04-25T14:04:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.681+0000 7f3b45600640 1 -- 172.21.15.86:0/1312631573 shutdown_connections 2024-04-25T14:04:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.681+0000 7f3b45600640 1 --2- 172.21.15.86:0/1312631573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b500d16e0 0x7f3b501f9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.681+0000 7f3b45600640 1 --2- 172.21.15.86:0/1312631573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b500cfca0 0x7f3b5009a830 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.681+0000 7f3b45600640 1 --2- 172.21.15.86:0/1312631573 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3b3006f400 0x7f3b30071910 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.681+0000 7f3b45600640 1 --2- 172.21.15.86:0/1312631573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b500730b0 0x7f3b500cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.681+0000 7f3b45600640 1 -- 172.21.15.86:0/1312631573 >> 172.21.15.86:0/1312631573 conn(0x7f3b500b4650 msgr2=0x7f3b50154fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.681+0000 7f3b45600640 1 -- 172.21.15.86:0/1312631573 shutdown_connections 2024-04-25T14:04:32.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.681+0000 7f3b45600640 1 -- 172.21.15.86:0/1312631573 wait complete. 2024-04-25T14:04:32.701 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T14:04:32.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.849+0000 7feac6000640 1 Processor -- start 2024-04-25T14:04:32.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.849+0000 7feac6000640 1 -- start start 2024-04-25T14:04:32.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feac6000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7feac01651d0 0x7feac01675f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:32.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feac6000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feac00b0a60 con 0x7feac006fd50 2024-04-25T14:04:32.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feac6000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feac00ae840 con 0x7feac015c2a0 2024-04-25T14:04:32.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feac6000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7feac006ef90 con 0x7feac01651d0 2024-04-25T14:04:32.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feab6a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7feac006fd50 0x7feac0070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60638/0 (socket says 172.21.15.86:60638) 2024-04-25T14:04:32.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feab6a00640 1 -- 172.21.15.86:0/2989560052 learned_addr learned my addr 172.21.15.86:0/2989560052 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:32.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feabea00640 1 -- 172.21.15.86:0/2989560052 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1184287225 0 0) 0x7feac00b0a60 con 0x7feac006fd50 2024-04-25T14:04:32.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feabea00640 1 -- 172.21.15.86:0/2989560052 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feaa4003450 con 0x7feac006fd50 2024-04-25T14:04:32.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feabfe00640 1 --2- 172.21.15.86:0/2989560052 >> v2:172.21.15.155:3300/0 conn(0x7feac01651d0 0x7feac01675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:32.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feabea00640 1 -- 172.21.15.86:0/2989560052 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3910282389 0 0) 0x7feaa4003450 con 0x7feac006fd50 2024-04-25T14:04:32.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feabea00640 1 -- 172.21.15.86:0/2989560052 >> v2:172.21.15.155:3300/0 conn(0x7feac01651d0 msgr2=0x7feac01675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:32.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feabea00640 1 --2- 172.21.15.86:0/2989560052 >> v2:172.21.15.155:3300/0 conn(0x7feac01651d0 0x7feac01675f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:32.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feabea00640 1 -- 172.21.15.86:0/2989560052 >> v1:172.21.15.155:6789/0 conn(0x7feac015c2a0 legacy=0x7feac00ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:32.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feabea00640 1 -- 172.21.15.86:0/2989560052 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac006f270 con 0x7feac006fd50 2024-04-25T14:04:32.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feabfe00640 1 --2- 172.21.15.86:0/2989560052 >> v2:172.21.15.155:3300/0 conn(0x7feac01651d0 0x7feac01675f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:32.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feabea00640 1 -- 172.21.15.86:0/2989560052 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7feaa8001680 con 0x7feac006fd50 2024-04-25T14:04:32.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feabea00640 1 -- 172.21.15.86:0/2989560052 >> v1:172.21.15.86:6789/0 conn(0x7feac006fd50 legacy=0x7feac0070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:32.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feabea00640 1 --2- 172.21.15.86:0/2989560052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa4003b20 0x7feaa4003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:32.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feabea00640 1 --2- 172.21.15.86:0/2989560052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feaa40089b0 0x7feaa400ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:32.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feabea00640 1 --2- 172.21.15.86:0/2989560052 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feaa400b730 0x7feaa400dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:32.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feabea00640 1 -- 172.21.15.86:0/2989560052 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feac006ef90 con 0x7feaa4003b20 2024-04-25T14:04:32.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feabea00640 1 -- 172.21.15.86:0/2989560052 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7feaa400e520 con 0x7feaa40089b0 2024-04-25T14:04:32.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feabea00640 1 -- 172.21.15.86:0/2989560052 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7feaa400e720 con 0x7feaa400b730 2024-04-25T14:04:32.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feabf400640 1 --2- 172.21.15.86:0/2989560052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa4003b20 0x7feaa4003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:32.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feab6a00640 1 --2- 172.21.15.86:0/2989560052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feaa40089b0 0x7feaa400ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:32.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feabfe00640 1 --2- 172.21.15.86:0/2989560052 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feaa400b730 0x7feaa400dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:32.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feabf400640 1 -- 172.21.15.86:0/2989560052 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feaa400b730 msgr2=0x7feaa400dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:32.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feabf400640 1 --2- 172.21.15.86:0/2989560052 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feaa400b730 0x7feaa400dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feabf400640 1 -- 172.21.15.86:0/2989560052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feaa40089b0 msgr2=0x7feaa400ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feabf400640 1 --2- 172.21.15.86:0/2989560052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feaa40089b0 0x7feaa400ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feabf400640 1 -- 172.21.15.86:0/2989560052 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa4003450 con 0x7feaa4003b20 2024-04-25T14:04:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feab6a00640 1 --2- 172.21.15.86:0/2989560052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feaa40089b0 0x7feaa400ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.853+0000 7feabfe00640 1 -- 172.21.15.86:0/2989560052 reap_dead start 2024-04-25T14:04:32.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.857+0000 7feabf400640 1 --2- 172.21.15.86:0/2989560052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa4003b20 0x7feaa4003f90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7feac0070bb0 tx=0x7feab001c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d480574f99ca55f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:32.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.857+0000 7feabea00640 1 -- 172.21.15.86:0/2989560052 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab00af070 con 0x7feaa4003b20 2024-04-25T14:04:32.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.857+0000 7feabea00640 1 -- 172.21.15.86:0/2989560052 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feab0003b80 con 0x7feaa4003b20 2024-04-25T14:04:32.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.857+0000 7feabea00640 1 -- 172.21.15.86:0/2989560052 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab0001a90 con 0x7feaa4003b20 2024-04-25T14:04:32.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.857+0000 7feac6000640 1 -- 172.21.15.86:0/2989560052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa4003b20 msgr2=0x7feaa4003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:32.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.857+0000 7feac6000640 1 --2- 172.21.15.86:0/2989560052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa4003b20 0x7feaa4003f90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7feac0070bb0 tx=0x7feab001c690 comp rx=0 tx=0).stop 2024-04-25T14:04:32.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.857+0000 7feac6000640 1 -- 172.21.15.86:0/2989560052 shutdown_connections 2024-04-25T14:04:32.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.857+0000 7feac6000640 1 --2- 172.21.15.86:0/2989560052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa4003b20 0x7feaa4003f90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:32.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.857+0000 7feac6000640 1 -- 172.21.15.86:0/2989560052 >> 172.21.15.86:0/2989560052 conn(0x7feac00b4650 msgr2=0x7feac00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:32.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.857+0000 7feac6000640 1 -- 172.21.15.86:0/2989560052 shutdown_connections 2024-04-25T14:04:32.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.857+0000 7feac6000640 1 -- 172.21.15.86:0/2989560052 wait complete. 2024-04-25T14:04:32.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.861+0000 7feac6000640 1 Processor -- start 2024-04-25T14:04:32.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.861+0000 7feac6000640 1 -- start start 2024-04-25T14:04:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.861+0000 7feac6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa4003b20 0x7feac014bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.861+0000 7feac6000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feac014d220 0x7feac01f9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.861+0000 7feac6000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feac0073e70 0x7feac0074280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.861+0000 7feac6000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feac015c830 con 0x7feaa4003b20 2024-04-25T14:04:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.861+0000 7feac6000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7feac015c6b0 con 0x7feac0073e70 2024-04-25T14:04:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.861+0000 7feac6000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7feac0070570 con 0x7feac014d220 2024-04-25T14:04:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.861+0000 7feabf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa4003b20 0x7feac014bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.861+0000 7feabf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa4003b20 0x7feac014bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45130/0 (socket says 172.21.15.86:45130) 2024-04-25T14:04:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.861+0000 7feabf400640 1 -- 172.21.15.86:0/2723047302 learned_addr learned my addr 172.21.15.86:0/2723047302 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.861+0000 7feab6a00640 1 --2- 172.21.15.86:0/2723047302 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feac014d220 0x7feac01f9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.861+0000 7feabfe00640 1 --2- 172.21.15.86:0/2723047302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feac0073e70 0x7feac0074280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.861+0000 7feabf400640 1 -- 172.21.15.86:0/2723047302 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feac014d220 msgr2=0x7feac01f9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.861+0000 7feabf400640 1 --2- 172.21.15.86:0/2723047302 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feac014d220 0x7feac01f9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.861+0000 7feabf400640 1 -- 172.21.15.86:0/2723047302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feac0073e70 msgr2=0x7feac0074280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.861+0000 7feabf400640 1 --2- 172.21.15.86:0/2723047302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feac0073e70 0x7feac0074280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.861+0000 7feabf400640 1 -- 172.21.15.86:0/2723047302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac014cc80 con 0x7feaa4003b20 2024-04-25T14:04:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.861+0000 7feabfe00640 1 --2- 172.21.15.86:0/2723047302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feac0073e70 0x7feac0074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.861+0000 7feabf400640 1 --2- 172.21.15.86:0/2723047302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa4003b20 0x7feac014bb90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7feab001c4c0 tx=0x7feaa4003740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.861+0000 7feab7e00640 1 -- 172.21.15.86:0/2723047302 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab00af040 con 0x7feaa4003b20 2024-04-25T14:04:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.861+0000 7feab7e00640 1 -- 172.21.15.86:0/2723047302 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feab00050d0 con 0x7feaa4003b20 2024-04-25T14:04:32.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.861+0000 7feac6000640 1 -- 172.21.15.86:0/2723047302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac00747c0 con 0x7feaa4003b20 2024-04-25T14:04:32.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.861+0000 7feab7e00640 1 -- 172.21.15.86:0/2723047302 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab0005480 con 0x7feaa4003b20 2024-04-25T14:04:32.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.861+0000 7feac6000640 1 -- 172.21.15.86:0/2723047302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac014c6c0 con 0x7feaa4003b20 2024-04-25T14:04:32.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.865+0000 7feac6000640 1 -- 172.21.15.86:0/2723047302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feac0069d00 con 0x7feaa4003b20 2024-04-25T14:04:32.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.865+0000 7feab7e00640 1 -- 172.21.15.86:0/2723047302 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7feab00bd060 con 0x7feaa4003b20 2024-04-25T14:04:32.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.865+0000 7feab7e00640 1 --2- 172.21.15.86:0/2723047302 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fea9c06f430 0x7fea9c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:32.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.865+0000 7feab7e00640 1 -- 172.21.15.86:0/2723047302 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7feab0128870 con 0x7feaa4003b20 2024-04-25T14:04:32.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.865+0000 7feab6a00640 1 --2- 172.21.15.86:0/2723047302 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fea9c06f430 0x7fea9c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:32.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.865+0000 7feab6a00640 1 --2- 172.21.15.86:0/2723047302 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fea9c06f430 0x7fea9c071940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7feac00b0190 tx=0x7feaa8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:32.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:32.869+0000 7feab7e00640 1 -- 172.21.15.86:0/2723047302 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7feab00163d0 con 0x7feaa4003b20 2024-04-25T14:04:33.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.029+0000 7feac6000640 1 -- 172.21.15.86:0/2723047302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7feac0076bf0 con 0x7feaa4003b20 2024-04-25T14:04:33.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.033+0000 7feab7e00640 1 -- 172.21.15.86:0/2723047302 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7feab00f52c0 con 0x7feaa4003b20 2024-04-25T14:04:33.033 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:04:33.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.033+0000 7feac6000640 1 -- 172.21.15.86:0/2723047302 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fea9c06f430 msgr2=0x7fea9c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:33.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.033+0000 7feac6000640 1 --2- 172.21.15.86:0/2723047302 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fea9c06f430 0x7fea9c071940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7feac00b0190 tx=0x7feaa8009040 comp rx=0 tx=0).stop 2024-04-25T14:04:33.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.033+0000 7feac6000640 1 -- 172.21.15.86:0/2723047302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa4003b20 msgr2=0x7feac014bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:33.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.033+0000 7feac6000640 1 --2- 172.21.15.86:0/2723047302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa4003b20 0x7feac014bb90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7feab001c4c0 tx=0x7feaa4003740 comp rx=0 tx=0).stop 2024-04-25T14:04:33.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.033+0000 7feac6000640 1 -- 172.21.15.86:0/2723047302 shutdown_connections 2024-04-25T14:04:33.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.033+0000 7feac6000640 1 --2- 172.21.15.86:0/2723047302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feac0073e70 0x7feac0074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.033+0000 7feac6000640 1 --2- 172.21.15.86:0/2723047302 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fea9c06f430 0x7fea9c071940 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.033+0000 7feac6000640 1 --2- 172.21.15.86:0/2723047302 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feac014d220 0x7feac01f9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.033+0000 7feac6000640 1 --2- 172.21.15.86:0/2723047302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa4003b20 0x7feac014bb90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.033+0000 7feac6000640 1 -- 172.21.15.86:0/2723047302 >> 172.21.15.86:0/2723047302 conn(0x7feac00b4650 msgr2=0x7feac015b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:33.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.033+0000 7feac6000640 1 -- 172.21.15.86:0/2723047302 shutdown_connections 2024-04-25T14:04:33.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.033+0000 7feac6000640 1 -- 172.21.15.86:0/2723047302 wait complete. 2024-04-25T14:04:33.055 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:04:33.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.201+0000 7f954f600640 1 Processor -- start 2024-04-25T14:04:33.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.201+0000 7f954f600640 1 -- start start 2024-04-25T14:04:33.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.201+0000 7f954f600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9548070330 0x7f95480d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.201+0000 7f954f600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f95480d1c20 0x7f954806ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.201+0000 7f954f600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95480d2030 con 0x7f954806f270 2024-04-25T14:04:33.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.201+0000 7f954f600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f954815e5c0 con 0x7f95480d1c20 2024-04-25T14:04:33.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.201+0000 7f954f600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f954815e8a0 con 0x7f9548070330 2024-04-25T14:04:33.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.201+0000 7f954d600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f954806f270 0x7f954806f680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60652/0 (socket says 172.21.15.86:60652) 2024-04-25T14:04:33.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.201+0000 7f954d600640 1 -- 172.21.15.86:0/1734586876 learned_addr learned my addr 172.21.15.86:0/1734586876 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:33.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.201+0000 7f9547e00640 1 --2- 172.21.15.86:0/1734586876 >> v2:172.21.15.155:3300/0 conn(0x7f95480d1c20 0x7f954806ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.201+0000 7f954cc00640 1 --2- 172.21.15.86:0/1734586876 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9548070330 0x7f95480d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.201+0000 7f9547400640 1 -- 172.21.15.86:0/1734586876 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4015631687 0 0) 0x7f95400014a0 con 0x7f954806f270 2024-04-25T14:04:33.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.201+0000 7f9547400640 1 -- 172.21.15.86:0/1734586876 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9530003530 con 0x7f954806f270 2024-04-25T14:04:33.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.201+0000 7f954cc00640 1 -- 172.21.15.86:0/1734586876 >> v2:172.21.15.155:3300/0 conn(0x7f95480d1c20 msgr2=0x7f954806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:33.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.201+0000 7f954cc00640 1 --2- 172.21.15.86:0/1734586876 >> v2:172.21.15.155:3300/0 conn(0x7f95480d1c20 0x7f954806ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.201+0000 7f954cc00640 1 -- 172.21.15.86:0/1734586876 >> v1:172.21.15.86:6789/0 conn(0x7f954806f270 legacy=0x7f954806f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:33.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.201+0000 7f9547400640 1 -- 172.21.15.86:0/1734586876 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4004313536 0 0) 0x7f9530003530 con 0x7f954806f270 2024-04-25T14:04:33.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.201+0000 7f954cc00640 1 -- 172.21.15.86:0/1734586876 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f954815eb80 con 0x7f9548070330 2024-04-25T14:04:33.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.201+0000 7f9547e00640 1 --2- 172.21.15.86:0/1734586876 >> v2:172.21.15.155:3300/0 conn(0x7f95480d1c20 0x7f954806ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:33.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.205+0000 7f954cc00640 1 --2- 172.21.15.86:0/1734586876 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9548070330 0x7f95480d16e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f953c0097f0 tx=0x7f953c01c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1076755ee04ddd00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:33.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.205+0000 7f9547400640 1 -- 172.21.15.86:0/1734586876 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c0b0070 con 0x7f9548070330 2024-04-25T14:04:33.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.205+0000 7f9547400640 1 -- 172.21.15.86:0/1734586876 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f953c0ab400 con 0x7f9548070330 2024-04-25T14:04:33.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.205+0000 7f9547400640 1 -- 172.21.15.86:0/1734586876 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9540001bd0 con 0x7f9548070330 2024-04-25T14:04:33.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.205+0000 7f954f600640 1 -- 172.21.15.86:0/1734586876 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9548070330 msgr2=0x7f95480d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:33.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.205+0000 7f954f600640 1 --2- 172.21.15.86:0/1734586876 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9548070330 0x7f95480d16e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f953c0097f0 tx=0x7f953c01c330 comp rx=0 tx=0).stop 2024-04-25T14:04:33.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.205+0000 7f954f600640 1 -- 172.21.15.86:0/1734586876 shutdown_connections 2024-04-25T14:04:33.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.205+0000 7f954f600640 1 --2- 172.21.15.86:0/1734586876 >> v2:172.21.15.155:3300/0 conn(0x7f95480d1c20 0x7f954806ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.205+0000 7f954f600640 1 --2- 172.21.15.86:0/1734586876 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9548070330 0x7f95480d16e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.205+0000 7f954f600640 1 -- 172.21.15.86:0/1734586876 >> 172.21.15.86:0/1734586876 conn(0x7f95480b4650 msgr2=0x7f95480b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:33.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.205+0000 7f954f600640 1 -- 172.21.15.86:0/1734586876 shutdown_connections 2024-04-25T14:04:33.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.205+0000 7f954f600640 1 -- 172.21.15.86:0/1734586876 wait complete. 2024-04-25T14:04:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.209+0000 7f954f600640 1 Processor -- start 2024-04-25T14:04:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.209+0000 7f954f600640 1 -- start start 2024-04-25T14:04:33.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.209+0000 7f954f600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f954806f270 0x7f95481f8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.209+0000 7f954f600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9548070330 0x7f9548159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.209+0000 7f954f600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95480d1c20 0x7f95480ae130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.209+0000 7f954f600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95480708c0 con 0x7f95480d1c20 2024-04-25T14:04:33.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.209+0000 7f954f600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9548070740 con 0x7f9548070330 2024-04-25T14:04:33.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.209+0000 7f954f600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f954816afc0 con 0x7f954806f270 2024-04-25T14:04:33.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.209+0000 7f954d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95480d1c20 0x7f95480ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.209+0000 7f954d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95480d1c20 0x7f95480ae130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45138/0 (socket says 172.21.15.86:45138) 2024-04-25T14:04:33.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.209+0000 7f954d600640 1 -- 172.21.15.86:0/4254543751 learned_addr learned my addr 172.21.15.86:0/4254543751 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:33.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.209+0000 7f954cc00640 1 --2- 172.21.15.86:0/4254543751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f954806f270 0x7f95481f8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.209+0000 7f9547e00640 1 --2- 172.21.15.86:0/4254543751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9548070330 0x7f9548159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.209+0000 7f954d600640 1 -- 172.21.15.86:0/4254543751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f954806f270 msgr2=0x7f95481f8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:33.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.209+0000 7f954d600640 1 --2- 172.21.15.86:0/4254543751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f954806f270 0x7f95481f8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.209+0000 7f954d600640 1 -- 172.21.15.86:0/4254543751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9548070330 msgr2=0x7f9548159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:33.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.209+0000 7f954d600640 1 --2- 172.21.15.86:0/4254543751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9548070330 0x7f9548159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.209+0000 7f954d600640 1 -- 172.21.15.86:0/4254543751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95481f88b0 con 0x7f95480d1c20 2024-04-25T14:04:33.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.213+0000 7f954d600640 1 --2- 172.21.15.86:0/4254543751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95480d1c20 0x7f95480ae130 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f954000cb90 tx=0x7f954000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:33.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.213+0000 7f9544c00640 1 -- 172.21.15.86:0/4254543751 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9540017070 con 0x7f95480d1c20 2024-04-25T14:04:33.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.213+0000 7f9544c00640 1 -- 172.21.15.86:0/4254543751 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f954000ad40 con 0x7f95480d1c20 2024-04-25T14:04:33.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.213+0000 7f954f600640 1 -- 172.21.15.86:0/4254543751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9548072920 con 0x7f95480d1c20 2024-04-25T14:04:33.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.213+0000 7f9544c00640 1 -- 172.21.15.86:0/4254543751 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9540012620 con 0x7f95480d1c20 2024-04-25T14:04:33.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.213+0000 7f954f600640 1 -- 172.21.15.86:0/4254543751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9548072ee0 con 0x7f95480d1c20 2024-04-25T14:04:33.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.213+0000 7f954f600640 1 -- 172.21.15.86:0/4254543751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9548069d00 con 0x7f95480d1c20 2024-04-25T14:04:33.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.213+0000 7f9544c00640 1 -- 172.21.15.86:0/4254543751 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f954001a070 con 0x7f95480d1c20 2024-04-25T14:04:33.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.213+0000 7f9544c00640 1 --2- 172.21.15.86:0/4254543751 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f952806f400 0x7f9528071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.213+0000 7f9544c00640 1 -- 172.21.15.86:0/4254543751 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f954008f510 con 0x7f95480d1c20 2024-04-25T14:04:33.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.217+0000 7f954cc00640 1 --2- 172.21.15.86:0/4254543751 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f952806f400 0x7f9528071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.217+0000 7f954cc00640 1 --2- 172.21.15.86:0/4254543751 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f952806f400 0x7f9528071910 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f953c0ad000 tx=0x7f953c0a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:33.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.221+0000 7f9544c00640 1 -- 172.21.15.86:0/4254543751 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f95481f88b0 con 0x7f95480d1c20 2024-04-25T14:04:33.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.377+0000 7f954f600640 1 -- 172.21.15.86:0/4254543751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f954806f890 con 0x7f95480d1c20 2024-04-25T14:04:33.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.377+0000 7f9544c00640 1 -- 172.21.15.86:0/4254543751 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18902 (secure 0 0 0) 0x7f95480d2030 con 0x7f95480d1c20 2024-04-25T14:04:33.379 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:33.379 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":26,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:04:31.930803+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:03:49.151473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:03:49.802597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T14:03:53.492008+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:04:33.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.381+0000 7f954f600640 1 -- 172.21.15.86:0/4254543751 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f952806f400 msgr2=0x7f9528071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:33.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.381+0000 7f954f600640 1 --2- 172.21.15.86:0/4254543751 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f952806f400 0x7f9528071910 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f953c0ad000 tx=0x7f953c0a6000 comp rx=0 tx=0).stop 2024-04-25T14:04:33.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.381+0000 7f954f600640 1 -- 172.21.15.86:0/4254543751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95480d1c20 msgr2=0x7f95480ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:33.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.381+0000 7f954f600640 1 --2- 172.21.15.86:0/4254543751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95480d1c20 0x7f95480ae130 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f954000cb90 tx=0x7f954000a430 comp rx=0 tx=0).stop 2024-04-25T14:04:33.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.381+0000 7f954f600640 1 -- 172.21.15.86:0/4254543751 shutdown_connections 2024-04-25T14:04:33.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.381+0000 7f954f600640 1 --2- 172.21.15.86:0/4254543751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95480d1c20 0x7f95480ae130 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.381+0000 7f954f600640 1 --2- 172.21.15.86:0/4254543751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9548070330 0x7f9548159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.381+0000 7f954f600640 1 --2- 172.21.15.86:0/4254543751 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f952806f400 0x7f9528071910 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.381+0000 7f954f600640 1 --2- 172.21.15.86:0/4254543751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f954806f270 0x7f95481f8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.381+0000 7f954f600640 1 -- 172.21.15.86:0/4254543751 >> 172.21.15.86:0/4254543751 conn(0x7f95480b4650 msgr2=0x7f954815bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:33.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.381+0000 7f954f600640 1 -- 172.21.15.86:0/4254543751 shutdown_connections 2024-04-25T14:04:33.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.381+0000 7f954f600640 1 -- 172.21.15.86:0/4254543751 wait complete. 2024-04-25T14:04:33.401 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:03:49.151473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:03:49.802597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T14:03:53.492008+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T14:04:33.402 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:04:33.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.549+0000 7f04fe600640 1 Processor -- start 2024-04-25T14:04:33.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.549+0000 7f04fe600640 1 -- start start 2024-04-25T14:04:33.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.549+0000 7f04fe600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f04f80cde90 0x7f04f80ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.549+0000 7f04fe600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04f815e5c0 con 0x7f04f8070330 2024-04-25T14:04:33.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.549+0000 7f04fe600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04f815e8a0 con 0x7f04f80d1df0 2024-04-25T14:04:33.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.549+0000 7f04fe600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f04f806fd40 con 0x7f04f80cde90 2024-04-25T14:04:33.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.553+0000 7f04f7400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f04f8070330 0x7f04f80d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60660/0 (socket says 172.21.15.86:60660) 2024-04-25T14:04:33.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.553+0000 7f04f7400640 1 -- 172.21.15.86:0/1537996763 learned_addr learned my addr 172.21.15.86:0/1537996763 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:33.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.553+0000 7f04f7e00640 1 --2- 172.21.15.86:0/1537996763 >> v2:172.21.15.86:3300/0 conn(0x7f04f80cde90 0x7f04f80ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.553+0000 7f04f6a00640 1 -- 172.21.15.86:0/1537996763 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3119402467 0 0) 0x7f04f815e5c0 con 0x7f04f8070330 2024-04-25T14:04:33.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.553+0000 7f04f6a00640 1 -- 172.21.15.86:0/1537996763 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04dc003530 con 0x7f04f8070330 2024-04-25T14:04:33.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.553+0000 7f04f7e00640 1 -- 172.21.15.86:0/1537996763 >> v1:172.21.15.155:6789/0 conn(0x7f04f80d1df0 legacy=0x7f04f80cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:33.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.553+0000 7f04f7e00640 1 -- 172.21.15.86:0/1537996763 >> v1:172.21.15.86:6789/0 conn(0x7f04f8070330 legacy=0x7f04f80d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:33.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.553+0000 7f04f7e00640 1 -- 172.21.15.86:0/1537996763 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f815eb80 con 0x7f04f80cde90 2024-04-25T14:04:33.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.553+0000 7f04f7e00640 1 --2- 172.21.15.86:0/1537996763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04f80cde90 0x7f04f80ce2a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f04f00097f0 tx=0x7f04f001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74901604ececac41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:33.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.553+0000 7f04f6a00640 1 -- 172.21.15.86:0/1537996763 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f00b0070 con 0x7f04f80cde90 2024-04-25T14:04:33.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.553+0000 7f04f6a00640 1 -- 172.21.15.86:0/1537996763 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f00ab600 con 0x7f04f80cde90 2024-04-25T14:04:33.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.553+0000 7f04f6a00640 1 -- 172.21.15.86:0/1537996763 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f00ab9d0 con 0x7f04f80cde90 2024-04-25T14:04:33.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.557+0000 7f04fe600640 1 -- 172.21.15.86:0/1537996763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04f80cde90 msgr2=0x7f04f80ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:33.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.557+0000 7f04fe600640 1 --2- 172.21.15.86:0/1537996763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04f80cde90 0x7f04f80ce2a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f04f00097f0 tx=0x7f04f001c3e0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.557+0000 7f04fe600640 1 -- 172.21.15.86:0/1537996763 shutdown_connections 2024-04-25T14:04:33.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.557+0000 7f04fe600640 1 --2- 172.21.15.86:0/1537996763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04f80cde90 0x7f04f80ce2a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.557+0000 7f04fe600640 1 -- 172.21.15.86:0/1537996763 >> 172.21.15.86:0/1537996763 conn(0x7f04f80b4650 msgr2=0x7f04f80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:33.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.557+0000 7f04fe600640 1 -- 172.21.15.86:0/1537996763 shutdown_connections 2024-04-25T14:04:33.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.557+0000 7f04fe600640 1 -- 172.21.15.86:0/1537996763 wait complete. 2024-04-25T14:04:33.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.557+0000 7f04fe600640 1 Processor -- start 2024-04-25T14:04:33.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.557+0000 7f04fe600640 1 -- start start 2024-04-25T14:04:33.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.561+0000 7f04fe600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04f8070330 0x7f04f809a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.561+0000 7f04fe600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04f80cde90 0x7f04f80ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.561+0000 7f04fe600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04f815ad90 0x7f04f80ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.561+0000 7f04fe600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f04f80708c0 con 0x7f04f8070330 2024-04-25T14:04:33.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.561+0000 7f04fe600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f04f8070740 con 0x7f04f80cde90 2024-04-25T14:04:33.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.561+0000 7f04fe600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f04f816bfd0 con 0x7f04f815ad90 2024-04-25T14:04:33.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.561+0000 7f04f7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04f8070330 0x7f04f809a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.561+0000 7f04f7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04f8070330 0x7f04f809a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45154/0 (socket says 172.21.15.86:45154) 2024-04-25T14:04:33.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.561+0000 7f04f7400640 1 -- 172.21.15.86:0/660250990 learned_addr learned my addr 172.21.15.86:0/660250990 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:33.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.561+0000 7f04eea00640 1 --2- 172.21.15.86:0/660250990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04f80cde90 0x7f04f80ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.561+0000 7f04f7e00640 1 --2- 172.21.15.86:0/660250990 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04f815ad90 0x7f04f80ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.561+0000 7f04f7400640 1 -- 172.21.15.86:0/660250990 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04f815ad90 msgr2=0x7f04f80ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:33.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.561+0000 7f04f7400640 1 --2- 172.21.15.86:0/660250990 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04f815ad90 0x7f04f80ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.561+0000 7f04f7400640 1 -- 172.21.15.86:0/660250990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04f80cde90 msgr2=0x7f04f80ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.561+0000 7f04f7400640 1 --2- 172.21.15.86:0/660250990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04f80cde90 0x7f04f80ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.561+0000 7f04f7400640 1 -- 172.21.15.86:0/660250990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f81551c0 con 0x7f04f8070330 2024-04-25T14:04:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.561+0000 7f04f7400640 1 --2- 172.21.15.86:0/660250990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04f8070330 0x7f04f809a8e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f04e800cbe0 tx=0x7f04e800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.561+0000 7f04efe00640 1 -- 172.21.15.86:0/660250990 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e8017070 con 0x7f04f8070330 2024-04-25T14:04:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.561+0000 7f04efe00640 1 -- 172.21.15.86:0/660250990 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04e800aca0 con 0x7f04f8070330 2024-04-25T14:04:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.561+0000 7f04fe600640 1 -- 172.21.15.86:0/660250990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f8159b50 con 0x7f04f8070330 2024-04-25T14:04:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.561+0000 7f04efe00640 1 -- 172.21.15.86:0/660250990 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e8012600 con 0x7f04f8070330 2024-04-25T14:04:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.561+0000 7f04fe600640 1 -- 172.21.15.86:0/660250990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f815a110 con 0x7f04f8070330 2024-04-25T14:04:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.561+0000 7f04fe600640 1 -- 172.21.15.86:0/660250990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04f8069d00 con 0x7f04f8070330 2024-04-25T14:04:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.561+0000 7f04efe00640 1 -- 172.21.15.86:0/660250990 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f04e801a070 con 0x7f04f8070330 2024-04-25T14:04:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.565+0000 7f04efe00640 1 --2- 172.21.15.86:0/660250990 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f04d406f400 0x7f04d4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.565+0000 7f04efe00640 1 -- 172.21.15.86:0/660250990 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f04e808f410 con 0x7f04f8070330 2024-04-25T14:04:33.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.565+0000 7f04eea00640 1 --2- 172.21.15.86:0/660250990 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f04d406f400 0x7f04d4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.565+0000 7f04eea00640 1 --2- 172.21.15.86:0/660250990 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f04d406f400 0x7f04d4071910 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f04f80ce470 tx=0x7f04e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:33.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.569+0000 7f04efe00640 1 -- 172.21.15.86:0/660250990 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f04f81551c0 con 0x7f04f8070330 2024-04-25T14:04:33.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.725+0000 7f04fe600640 1 -- 172.21.15.86:0/660250990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f04f80ce560 con 0x7f04f8070330 2024-04-25T14:04:33.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.725+0000 7f04efe00640 1 -- 172.21.15.86:0/660250990 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f04e805b810 con 0x7f04f8070330 2024-04-25T14:04:33.728 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:04:33.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.729+0000 7f04fe600640 1 -- 172.21.15.86:0/660250990 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f04d406f400 msgr2=0x7f04d4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:33.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.729+0000 7f04fe600640 1 --2- 172.21.15.86:0/660250990 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f04d406f400 0x7f04d4071910 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f04f80ce470 tx=0x7f04e0009040 comp rx=0 tx=0).stop 2024-04-25T14:04:33.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.729+0000 7f04fe600640 1 -- 172.21.15.86:0/660250990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04f8070330 msgr2=0x7f04f809a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:33.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.729+0000 7f04fe600640 1 --2- 172.21.15.86:0/660250990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04f8070330 0x7f04f809a8e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f04e800cbe0 tx=0x7f04e800a060 comp rx=0 tx=0).stop 2024-04-25T14:04:33.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.729+0000 7f04fe600640 1 -- 172.21.15.86:0/660250990 shutdown_connections 2024-04-25T14:04:33.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.729+0000 7f04fe600640 1 --2- 172.21.15.86:0/660250990 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f04d406f400 0x7f04d4071910 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.729+0000 7f04fe600640 1 --2- 172.21.15.86:0/660250990 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04f815ad90 0x7f04f80ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.729+0000 7f04fe600640 1 --2- 172.21.15.86:0/660250990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04f80cde90 0x7f04f80ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.729+0000 7f04fe600640 1 --2- 172.21.15.86:0/660250990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04f8070330 0x7f04f809a8e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.729+0000 7f04fe600640 1 -- 172.21.15.86:0/660250990 >> 172.21.15.86:0/660250990 conn(0x7f04f80b4650 msgr2=0x7f04f815cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:33.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.729+0000 7f04fe600640 1 -- 172.21.15.86:0/660250990 shutdown_connections 2024-04-25T14:04:33.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.729+0000 7f04fe600640 1 -- 172.21.15.86:0/660250990 wait complete. 2024-04-25T14:04:33.750 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:04:33.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.897+0000 7f3e6fe00640 1 Processor -- start 2024-04-25T14:04:33.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.897+0000 7f3e6fe00640 1 -- start start 2024-04-25T14:04:33.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.901+0000 7f3e6fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3e6815faa0 0x7f3e6806d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.901+0000 7f3e6fe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e680b59e0 0x7f3e680b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.901+0000 7f3e6fe00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e680b6330 con 0x7f3e680aeea0 2024-04-25T14:04:33.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.901+0000 7f3e6fe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3e6806d560 con 0x7f3e6815faa0 2024-04-25T14:04:33.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.901+0000 7f3e6fe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3e6806c670 con 0x7f3e680b59e0 2024-04-25T14:04:33.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.901+0000 7f3e6d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3e6815faa0 0x7f3e6806d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.901+0000 7f3e6d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3e6815faa0 0x7f3e6806d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45166/0 (socket says 172.21.15.86:45166) 2024-04-25T14:04:33.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.901+0000 7f3e6d400640 1 -- 172.21.15.86:0/3873493158 learned_addr learned my addr 172.21.15.86:0/3873493158 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:33.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.901+0000 7f3e6ca00640 1 --2- 172.21.15.86:0/3873493158 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e680b59e0 0x7f3e680b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.901+0000 7f3e6d400640 1 -- 172.21.15.86:0/3873493158 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e680b59e0 msgr2=0x7f3e680b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:33.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.901+0000 7f3e6d400640 1 --2- 172.21.15.86:0/3873493158 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e680b59e0 0x7f3e680b5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.901+0000 7f3e6d400640 1 -- 172.21.15.86:0/3873493158 >> v1:172.21.15.155:6789/0 conn(0x7f3e680aeea0 legacy=0x7f3e680af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:33.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.901+0000 7f3e6d400640 1 -- 172.21.15.86:0/3873493158 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e6806c950 con 0x7f3e6815faa0 2024-04-25T14:04:33.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.905+0000 7f3e6d400640 1 --2- 172.21.15.86:0/3873493158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e6815faa0 0x7f3e6806d9f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c009870 tx=0x7f3e5c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b63d4a280e5327 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:33.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.905+0000 7f3e67e00640 1 -- 172.21.15.86:0/3873493158 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c0a9620 con 0x7f3e6815faa0 2024-04-25T14:04:33.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.905+0000 7f3e67e00640 1 -- 172.21.15.86:0/3873493158 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e5c0a97c0 con 0x7f3e6815faa0 2024-04-25T14:04:33.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.905+0000 7f3e67e00640 1 -- 172.21.15.86:0/3873493158 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c0a9b90 con 0x7f3e6815faa0 2024-04-25T14:04:33.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.905+0000 7f3e6fe00640 1 -- 172.21.15.86:0/3873493158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e6815faa0 msgr2=0x7f3e6806d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:33.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.905+0000 7f3e6fe00640 1 --2- 172.21.15.86:0/3873493158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e6815faa0 0x7f3e6806d9f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c009870 tx=0x7f3e5c01c560 comp rx=0 tx=0).stop 2024-04-25T14:04:33.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.905+0000 7f3e6fe00640 1 -- 172.21.15.86:0/3873493158 shutdown_connections 2024-04-25T14:04:33.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.905+0000 7f3e6fe00640 1 --2- 172.21.15.86:0/3873493158 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e680b59e0 0x7f3e680b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.905+0000 7f3e6fe00640 1 --2- 172.21.15.86:0/3873493158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e6815faa0 0x7f3e6806d9f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.905+0000 7f3e6fe00640 1 -- 172.21.15.86:0/3873493158 >> 172.21.15.86:0/3873493158 conn(0x7f3e680d2230 msgr2=0x7f3e680d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:33.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.905+0000 7f3e6fe00640 1 -- 172.21.15.86:0/3873493158 shutdown_connections 2024-04-25T14:04:33.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.905+0000 7f3e6fe00640 1 -- 172.21.15.86:0/3873493158 wait complete. 2024-04-25T14:04:33.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.905+0000 7f3e6fe00640 1 Processor -- start 2024-04-25T14:04:33.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.905+0000 7f3e6fe00640 1 -- start start 2024-04-25T14:04:33.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.909+0000 7f3e6fe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e680aeea0 0x7f3e681547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.909+0000 7f3e6fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e680b59e0 0x7f3e6815dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.909+0000 7f3e6fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e6815faa0 0x7f3e6815e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.909+0000 7f3e6fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6806df30 con 0x7f3e680b59e0 2024-04-25T14:04:33.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.909+0000 7f3e6fe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3e680af9c0 con 0x7f3e6815faa0 2024-04-25T14:04:33.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.909+0000 7f3e6fe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3e6815feb0 con 0x7f3e680aeea0 2024-04-25T14:04:33.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.909+0000 7f3e6ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e680b59e0 0x7f3e6815dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.909+0000 7f3e6ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e680b59e0 0x7f3e6815dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45172/0 (socket says 172.21.15.86:45172) 2024-04-25T14:04:33.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.909+0000 7f3e6ca00640 1 -- 172.21.15.86:0/3491817367 learned_addr learned my addr 172.21.15.86:0/3491817367 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:33.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.909+0000 7f3e6d400640 1 --2- 172.21.15.86:0/3491817367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e680aeea0 0x7f3e681547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.909+0000 7f3e6de00640 1 --2- 172.21.15.86:0/3491817367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e6815faa0 0x7f3e6815e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.909+0000 7f3e6ca00640 1 -- 172.21.15.86:0/3491817367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e680aeea0 msgr2=0x7f3e681547e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:33.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.909+0000 7f3e6ca00640 1 --2- 172.21.15.86:0/3491817367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e680aeea0 0x7f3e681547e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.909+0000 7f3e6ca00640 1 -- 172.21.15.86:0/3491817367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e6815faa0 msgr2=0x7f3e6815e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:33.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.909+0000 7f3e6ca00640 1 --2- 172.21.15.86:0/3491817367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e6815faa0 0x7f3e6815e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.909+0000 7f3e6ca00640 1 -- 172.21.15.86:0/3491817367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e68154dd0 con 0x7f3e680b59e0 2024-04-25T14:04:33.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.909+0000 7f3e6ca00640 1 --2- 172.21.15.86:0/3491817367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e680b59e0 0x7f3e6815dc10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400d5f0 tx=0x7f3e5400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:33.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.909+0000 7f3e65600640 1 -- 172.21.15.86:0/3491817367 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e540125f0 con 0x7f3e680b59e0 2024-04-25T14:04:33.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.909+0000 7f3e65600640 1 -- 172.21.15.86:0/3491817367 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e54019070 con 0x7f3e680b59e0 2024-04-25T14:04:33.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.909+0000 7f3e6fe00640 1 -- 172.21.15.86:0/3491817367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e6815c370 con 0x7f3e680b59e0 2024-04-25T14:04:33.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.909+0000 7f3e65600640 1 -- 172.21.15.86:0/3491817367 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e540129b0 con 0x7f3e680b59e0 2024-04-25T14:04:33.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.909+0000 7f3e6fe00640 1 -- 172.21.15.86:0/3491817367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e6815c930 con 0x7f3e680b59e0 2024-04-25T14:04:33.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.909+0000 7f3e6fe00640 1 -- 172.21.15.86:0/3491817367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e68074cf0 con 0x7f3e680b59e0 2024-04-25T14:04:33.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.913+0000 7f3e65600640 1 -- 172.21.15.86:0/3491817367 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3e54012b50 con 0x7f3e680b59e0 2024-04-25T14:04:33.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.913+0000 7f3e65600640 1 --2- 172.21.15.86:0/3491817367 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3e4806f430 0x7f3e48071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.913+0000 7f3e65600640 1 -- 172.21.15.86:0/3491817367 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f3e5408f900 con 0x7f3e680b59e0 2024-04-25T14:04:33.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.913+0000 7f3e6d400640 1 --2- 172.21.15.86:0/3491817367 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3e4806f430 0x7f3e48071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.913+0000 7f3e6d400640 1 --2- 172.21.15.86:0/3491817367 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3e4806f430 0x7f3e48071940 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c0ae000 tx=0x7f3e5c0a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:33.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:33.917+0000 7f3e65600640 1 -- 172.21.15.86:0/3491817367 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3e5405be20 con 0x7f3e680b59e0 2024-04-25T14:04:34.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.073+0000 7f3e6fe00640 1 -- 172.21.15.86:0/3491817367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3e680b55b0 con 0x7f3e680b59e0 2024-04-25T14:04:34.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.077+0000 7f3e65600640 1 -- 172.21.15.86:0/3491817367 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f3e54060120 con 0x7f3e680b59e0 2024-04-25T14:04:34.077 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:04:34.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.077+0000 7f3e6fe00640 1 -- 172.21.15.86:0/3491817367 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3e4806f430 msgr2=0x7f3e48071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:34.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.077+0000 7f3e6fe00640 1 --2- 172.21.15.86:0/3491817367 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3e4806f430 0x7f3e48071940 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c0ae000 tx=0x7f3e5c0a7000 comp rx=0 tx=0).stop 2024-04-25T14:04:34.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.077+0000 7f3e6fe00640 1 -- 172.21.15.86:0/3491817367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e680b59e0 msgr2=0x7f3e6815dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:34.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.077+0000 7f3e6fe00640 1 --2- 172.21.15.86:0/3491817367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e680b59e0 0x7f3e6815dc10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400d5f0 tx=0x7f3e5400dab0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.077+0000 7f3e6fe00640 1 -- 172.21.15.86:0/3491817367 shutdown_connections 2024-04-25T14:04:34.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.077+0000 7f3e6fe00640 1 --2- 172.21.15.86:0/3491817367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e6815faa0 0x7f3e6815e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.077+0000 7f3e6fe00640 1 --2- 172.21.15.86:0/3491817367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e680b59e0 0x7f3e6815dc10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.077+0000 7f3e6fe00640 1 --2- 172.21.15.86:0/3491817367 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3e4806f430 0x7f3e48071940 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.077+0000 7f3e6fe00640 1 --2- 172.21.15.86:0/3491817367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e680aeea0 0x7f3e681547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.077+0000 7f3e6fe00640 1 -- 172.21.15.86:0/3491817367 >> 172.21.15.86:0/3491817367 conn(0x7f3e680d2230 msgr2=0x7f3e68157e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:34.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.077+0000 7f3e6fe00640 1 -- 172.21.15.86:0/3491817367 shutdown_connections 2024-04-25T14:04:34.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.077+0000 7f3e6fe00640 1 -- 172.21.15.86:0/3491817367 wait complete. 2024-04-25T14:04:34.099 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:04:34.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.245+0000 7fee47e00640 1 Processor -- start 2024-04-25T14:04:34.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.245+0000 7fee47e00640 1 -- start start 2024-04-25T14:04:34.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.249+0000 7fee47e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fee480cc950 0x7fee4806eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:34.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.249+0000 7fee47e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fee480b3530 0x7fee480d2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:34.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.249+0000 7fee47e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee480ccd60 con 0x7fee480d27d0 2024-04-25T14:04:34.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.249+0000 7fee47e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fee480b20d0 con 0x7fee480cc950 2024-04-25T14:04:34.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.249+0000 7fee47e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fee480d2be0 con 0x7fee480b3530 2024-04-25T14:04:34.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.249+0000 7fee46a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fee480cc950 0x7fee4806eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:34.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.249+0000 7fee47400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fee480d27d0 0x7fee480b2560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60676/0 (socket says 172.21.15.86:60676) 2024-04-25T14:04:34.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.249+0000 7fee47400640 1 -- 172.21.15.86:0/3332078553 learned_addr learned my addr 172.21.15.86:0/3332078553 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:34.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.249+0000 7fee46a00640 1 -- 172.21.15.86:0/3332078553 >> v2:172.21.15.155:3300/0 conn(0x7fee480b3530 msgr2=0x7fee480d2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:34.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.249+0000 7fee3e000640 1 --2- 172.21.15.86:0/3332078553 >> v2:172.21.15.155:3300/0 conn(0x7fee480b3530 0x7fee480d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:34.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.249+0000 7fee46a00640 1 --2- 172.21.15.86:0/3332078553 >> v2:172.21.15.155:3300/0 conn(0x7fee480b3530 0x7fee480d2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.249+0000 7fee46a00640 1 -- 172.21.15.86:0/3332078553 >> v1:172.21.15.86:6789/0 conn(0x7fee480d27d0 legacy=0x7fee480b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:34.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.249+0000 7fee46000640 1 -- 172.21.15.86:0/3332078553 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2827319818 0 0) 0x7fee40001120 con 0x7fee480d27d0 2024-04-25T14:04:34.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.249+0000 7fee46a00640 1 -- 172.21.15.86:0/3332078553 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee4806f0d0 con 0x7fee480cc950 2024-04-25T14:04:34.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.249+0000 7fee3e000640 1 --2- 172.21.15.86:0/3332078553 >> v2:172.21.15.155:3300/0 conn(0x7fee480b3530 0x7fee480d2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:34.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.253+0000 7fee46a00640 1 --2- 172.21.15.86:0/3332078553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee480cc950 0x7fee4806eb90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fee480cfdb0 tx=0x7fee3801c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fddc01d6f68d63ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:34.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.253+0000 7fee46000640 1 -- 172.21.15.86:0/3332078553 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee380b2070 con 0x7fee480cc950 2024-04-25T14:04:34.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.253+0000 7fee46000640 1 -- 172.21.15.86:0/3332078553 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee380a58d0 con 0x7fee480cc950 2024-04-25T14:04:34.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.253+0000 7fee46000640 1 -- 172.21.15.86:0/3332078553 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee380a5ca0 con 0x7fee480cc950 2024-04-25T14:04:34.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.253+0000 7fee47e00640 1 -- 172.21.15.86:0/3332078553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee480cc950 msgr2=0x7fee4806eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:34.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.253+0000 7fee47e00640 1 --2- 172.21.15.86:0/3332078553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee480cc950 0x7fee4806eb90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fee480cfdb0 tx=0x7fee3801c810 comp rx=0 tx=0).stop 2024-04-25T14:04:34.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.253+0000 7fee47e00640 1 -- 172.21.15.86:0/3332078553 shutdown_connections 2024-04-25T14:04:34.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.253+0000 7fee47e00640 1 --2- 172.21.15.86:0/3332078553 >> v2:172.21.15.155:3300/0 conn(0x7fee480b3530 0x7fee480d2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.253+0000 7fee47e00640 1 --2- 172.21.15.86:0/3332078553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee480cc950 0x7fee4806eb90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.253+0000 7fee47e00640 1 -- 172.21.15.86:0/3332078553 >> 172.21.15.86:0/3332078553 conn(0x7fee4809ba30 msgr2=0x7fee480b71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:34.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.253+0000 7fee47e00640 1 -- 172.21.15.86:0/3332078553 shutdown_connections 2024-04-25T14:04:34.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.253+0000 7fee47e00640 1 -- 172.21.15.86:0/3332078553 wait complete. 2024-04-25T14:04:34.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.253+0000 7fee47e00640 1 Processor -- start 2024-04-25T14:04:34.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.253+0000 7fee47e00640 1 -- start start 2024-04-25T14:04:34.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.257+0000 7fee47e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee480b3530 0x7fee4815c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:34.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.257+0000 7fee47e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee480cc950 0x7fee4814f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:34.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.257+0000 7fee47e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee480d27d0 0x7fee4814ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:34.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.257+0000 7fee47e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee480b3ac0 con 0x7fee480d27d0 2024-04-25T14:04:34.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.257+0000 7fee47e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fee480b3940 con 0x7fee480cc950 2024-04-25T14:04:34.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.257+0000 7fee47e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fee480b3080 con 0x7fee480b3530 2024-04-25T14:04:34.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.257+0000 7fee47400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee480d27d0 0x7fee4814ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:34.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.257+0000 7fee47400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee480d27d0 0x7fee4814ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45192/0 (socket says 172.21.15.86:45192) 2024-04-25T14:04:34.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.257+0000 7fee47400640 1 -- 172.21.15.86:0/2330219841 learned_addr learned my addr 172.21.15.86:0/2330219841 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:34.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.257+0000 7fee3e000640 1 --2- 172.21.15.86:0/2330219841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee480cc950 0x7fee4814f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:34.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.257+0000 7fee46a00640 1 --2- 172.21.15.86:0/2330219841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee480b3530 0x7fee4815c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:34.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.257+0000 7fee47400640 1 -- 172.21.15.86:0/2330219841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee480b3530 msgr2=0x7fee4815c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:34.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.257+0000 7fee47400640 1 --2- 172.21.15.86:0/2330219841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee480b3530 0x7fee4815c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.257+0000 7fee47400640 1 -- 172.21.15.86:0/2330219841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee480cc950 msgr2=0x7fee4814f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:34.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.257+0000 7fee47400640 1 --2- 172.21.15.86:0/2330219841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee480cc950 0x7fee4814f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.257+0000 7fee47400640 1 -- 172.21.15.86:0/2330219841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee48156f90 con 0x7fee480d27d0 2024-04-25T14:04:34.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.257+0000 7fee46a00640 1 --2- 172.21.15.86:0/2330219841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee480b3530 0x7fee4815c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:34.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.257+0000 7fee47400640 1 --2- 172.21.15.86:0/2330219841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee480d27d0 0x7fee4814ea20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fee4000c3e0 tx=0x7fee4000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:34.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.257+0000 7fee3f400640 1 -- 172.21.15.86:0/2330219841 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee400074e0 con 0x7fee480d27d0 2024-04-25T14:04:34.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.257+0000 7fee3f400640 1 -- 172.21.15.86:0/2330219841 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee400047f0 con 0x7fee480d27d0 2024-04-25T14:04:34.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.257+0000 7fee3f400640 1 -- 172.21.15.86:0/2330219841 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee40004be0 con 0x7fee480d27d0 2024-04-25T14:04:34.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.257+0000 7fee47e00640 1 -- 172.21.15.86:0/2330219841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee48157270 con 0x7fee480d27d0 2024-04-25T14:04:34.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.257+0000 7fee47e00640 1 -- 172.21.15.86:0/2330219841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee48157830 con 0x7fee480d27d0 2024-04-25T14:04:34.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.257+0000 7fee47e00640 1 -- 172.21.15.86:0/2330219841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee48156d00 con 0x7fee480d27d0 2024-04-25T14:04:34.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.261+0000 7fee3f400640 1 -- 172.21.15.86:0/2330219841 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fee40011070 con 0x7fee480d27d0 2024-04-25T14:04:34.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.261+0000 7fee3f400640 1 --2- 172.21.15.86:0/2330219841 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fee2406f400 0x7fee24071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:34.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.261+0000 7fee3f400640 1 -- 172.21.15.86:0/2330219841 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fee4008f320 con 0x7fee480d27d0 2024-04-25T14:04:34.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.261+0000 7fee46a00640 1 --2- 172.21.15.86:0/2330219841 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fee2406f400 0x7fee24071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:34.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.261+0000 7fee46a00640 1 --2- 172.21.15.86:0/2330219841 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fee2406f400 0x7fee24071910 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fee380af000 tx=0x7fee380a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:34.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.265+0000 7fee3f400640 1 -- 172.21.15.86:0/2330219841 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fee480ccd60 con 0x7fee480d27d0 2024-04-25T14:04:34.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.421+0000 7fee47e00640 1 -- 172.21.15.86:0/2330219841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fee48076b80 con 0x7fee480d27d0 2024-04-25T14:04:34.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.421+0000 7fee3f400640 1 -- 172.21.15.86:0/2330219841 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fee4005bd70 con 0x7fee480d27d0 2024-04-25T14:04:34.426 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:04:34.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.425+0000 7fee47e00640 1 -- 172.21.15.86:0/2330219841 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fee2406f400 msgr2=0x7fee24071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:34.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.425+0000 7fee47e00640 1 --2- 172.21.15.86:0/2330219841 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fee2406f400 0x7fee24071910 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fee380af000 tx=0x7fee380a7070 comp rx=0 tx=0).stop 2024-04-25T14:04:34.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.425+0000 7fee47e00640 1 -- 172.21.15.86:0/2330219841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee480d27d0 msgr2=0x7fee4814ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:34.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.425+0000 7fee47e00640 1 --2- 172.21.15.86:0/2330219841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee480d27d0 0x7fee4814ea20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fee4000c3e0 tx=0x7fee4000c8a0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.425+0000 7fee47e00640 1 -- 172.21.15.86:0/2330219841 shutdown_connections 2024-04-25T14:04:34.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.425+0000 7fee47e00640 1 --2- 172.21.15.86:0/2330219841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee480d27d0 0x7fee4814ea20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.425+0000 7fee47e00640 1 --2- 172.21.15.86:0/2330219841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee480cc950 0x7fee4814f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.425+0000 7fee47e00640 1 --2- 172.21.15.86:0/2330219841 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fee2406f400 0x7fee24071910 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.425+0000 7fee47e00640 1 --2- 172.21.15.86:0/2330219841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee480b3530 0x7fee4815c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.425+0000 7fee47e00640 1 -- 172.21.15.86:0/2330219841 >> 172.21.15.86:0/2330219841 conn(0x7fee4809ba30 msgr2=0x7fee48159670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:34.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.425+0000 7fee47e00640 1 -- 172.21.15.86:0/2330219841 shutdown_connections 2024-04-25T14:04:34.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.425+0000 7fee47e00640 1 -- 172.21.15.86:0/2330219841 wait complete. 2024-04-25T14:04:34.446 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T14:04:34.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.593+0000 7f4513200640 1 Processor -- start 2024-04-25T14:04:34.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.593+0000 7f4513200640 1 -- start start 2024-04-25T14:04:34.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.593+0000 7f4513200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f450c0cfca0 0x7f450c06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:34.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.593+0000 7f4513200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f450c0730b0 0x7f450c0734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:34.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.593+0000 7f4513200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f450c072bd0 con 0x7f450c0d16e0 2024-04-25T14:04:34.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.593+0000 7f4513200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f450c070470 con 0x7f450c0730b0 2024-04-25T14:04:34.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.593+0000 7f4513200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f450c06f930 con 0x7f450c0cfca0 2024-04-25T14:04:34.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.593+0000 7f4511200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f450c0730b0 0x7f450c0734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:34.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.593+0000 7f4511200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f450c0730b0 0x7f450c0734c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45202/0 (socket says 172.21.15.86:45202) 2024-04-25T14:04:34.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.593+0000 7f4511200640 1 -- 172.21.15.86:0/3523165664 learned_addr learned my addr 172.21.15.86:0/3523165664 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:34.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.593+0000 7f450be00640 1 --2- 172.21.15.86:0/3523165664 >> v2:172.21.15.155:3300/0 conn(0x7f450c0cfca0 0x7f450c06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:34.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.597+0000 7f4511200640 1 -- 172.21.15.86:0/3523165664 >> v2:172.21.15.155:3300/0 conn(0x7f450c0cfca0 msgr2=0x7f450c06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:34.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.597+0000 7f4511200640 1 --2- 172.21.15.86:0/3523165664 >> v2:172.21.15.155:3300/0 conn(0x7f450c0cfca0 0x7f450c06f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.597+0000 7f4511200640 1 -- 172.21.15.86:0/3523165664 >> v1:172.21.15.155:6789/0 conn(0x7f450c0d16e0 legacy=0x7f450c0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:34.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.597+0000 7f4511200640 1 -- 172.21.15.86:0/3523165664 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f450c06fbd0 con 0x7f450c0730b0 2024-04-25T14:04:34.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.597+0000 7f4511200640 1 --2- 172.21.15.86:0/3523165664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f450c0730b0 0x7f450c0734c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f450c0d22a0 tx=0x7f450401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e2a4c42bd4897b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:34.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.597+0000 7f450aa00640 1 -- 172.21.15.86:0/3523165664 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45040a5780 con 0x7f450c0730b0 2024-04-25T14:04:34.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.597+0000 7f450aa00640 1 -- 172.21.15.86:0/3523165664 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45040a5920 con 0x7f450c0730b0 2024-04-25T14:04:34.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.597+0000 7f450aa00640 1 -- 172.21.15.86:0/3523165664 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45040a5cf0 con 0x7f450c0730b0 2024-04-25T14:04:34.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.597+0000 7f4513200640 1 -- 172.21.15.86:0/3523165664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f450c0730b0 msgr2=0x7f450c0734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:34.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.597+0000 7f4513200640 1 --2- 172.21.15.86:0/3523165664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f450c0730b0 0x7f450c0734c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f450c0d22a0 tx=0x7f450401c9b0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.601+0000 7f4513200640 1 -- 172.21.15.86:0/3523165664 shutdown_connections 2024-04-25T14:04:34.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.601+0000 7f4513200640 1 --2- 172.21.15.86:0/3523165664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f450c0730b0 0x7f450c0734c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.601+0000 7f4513200640 1 --2- 172.21.15.86:0/3523165664 >> v2:172.21.15.155:3300/0 conn(0x7f450c0cfca0 0x7f450c06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.601+0000 7f4513200640 1 -- 172.21.15.86:0/3523165664 >> 172.21.15.86:0/3523165664 conn(0x7f450c0b4650 msgr2=0x7f450c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:34.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.601+0000 7f4513200640 1 -- 172.21.15.86:0/3523165664 shutdown_connections 2024-04-25T14:04:34.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.601+0000 7f4513200640 1 -- 172.21.15.86:0/3523165664 wait complete. 2024-04-25T14:04:34.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.601+0000 7f4513200640 1 Processor -- start 2024-04-25T14:04:34.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.601+0000 7f4513200640 1 -- start start 2024-04-25T14:04:34.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.601+0000 7f4513200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f450c0730b0 0x7f450c09a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:34.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.601+0000 7f4513200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f450c0cfca0 0x7f450c0cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:34.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.601+0000 7f4513200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f450c0d16e0 0x7f450c0ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:34.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.601+0000 7f4513200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f450c169a70 con 0x7f450c0730b0 2024-04-25T14:04:34.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.601+0000 7f4513200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f450c1698f0 con 0x7f450c0cfca0 2024-04-25T14:04:34.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.601+0000 7f4513200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f450c169bf0 con 0x7f450c0d16e0 2024-04-25T14:04:34.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.601+0000 7f450be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f450c0730b0 0x7f450c09a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:34.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.605+0000 7f450be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f450c0730b0 0x7f450c09a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45216/0 (socket says 172.21.15.86:45216) 2024-04-25T14:04:34.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.605+0000 7f450be00640 1 -- 172.21.15.86:0/1392333054 learned_addr learned my addr 172.21.15.86:0/1392333054 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:34.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.605+0000 7f450be00640 1 -- 172.21.15.86:0/1392333054 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f450c0d16e0 msgr2=0x7f450c0ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:34.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.605+0000 7f450be00640 1 --2- 172.21.15.86:0/1392333054 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f450c0d16e0 0x7f450c0ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.605+0000 7f450be00640 1 -- 172.21.15.86:0/1392333054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f450c0cfca0 msgr2=0x7f450c0cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:34.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.605+0000 7f450be00640 1 --2- 172.21.15.86:0/1392333054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f450c0cfca0 0x7f450c0cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.605+0000 7f450be00640 1 -- 172.21.15.86:0/1392333054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f450c0ae1d0 con 0x7f450c0730b0 2024-04-25T14:04:34.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.605+0000 7f450be00640 1 --2- 172.21.15.86:0/1392333054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f450c0730b0 0x7f450c09a7b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00d600 tx=0x7f44fc00dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:34.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.605+0000 7f4503e00640 1 -- 172.21.15.86:0/1392333054 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc004240 con 0x7f450c0730b0 2024-04-25T14:04:34.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.605+0000 7f4503e00640 1 -- 172.21.15.86:0/1392333054 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44fc0043e0 con 0x7f450c0730b0 2024-04-25T14:04:34.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.605+0000 7f4513200640 1 -- 172.21.15.86:0/1392333054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f450c0ae470 con 0x7f450c0730b0 2024-04-25T14:04:34.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.605+0000 7f4503e00640 1 -- 172.21.15.86:0/1392333054 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc003840 con 0x7f450c0730b0 2024-04-25T14:04:34.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.605+0000 7f4513200640 1 -- 172.21.15.86:0/1392333054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f450c0aea30 con 0x7f450c0730b0 2024-04-25T14:04:34.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.605+0000 7f4513200640 1 -- 172.21.15.86:0/1392333054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f450c09f4b0 con 0x7f450c0730b0 2024-04-25T14:04:34.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.605+0000 7f4503e00640 1 -- 172.21.15.86:0/1392333054 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f44fc0039e0 con 0x7f450c0730b0 2024-04-25T14:04:34.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.609+0000 7f4503e00640 1 --2- 172.21.15.86:0/1392333054 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f44ec06f430 0x7f44ec071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:34.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.609+0000 7f4503e00640 1 -- 172.21.15.86:0/1392333054 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f44fc08f940 con 0x7f450c0730b0 2024-04-25T14:04:34.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.609+0000 7f450b400640 1 --2- 172.21.15.86:0/1392333054 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f44ec06f430 0x7f44ec071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:34.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.609+0000 7f450b400640 1 --2- 172.21.15.86:0/1392333054 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f44ec06f430 0x7f44ec071940 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f450c15fba0 tx=0x7f44f8006bd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:34.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.613+0000 7f4503e00640 1 -- 172.21.15.86:0/1392333054 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f44fc05c420 con 0x7f450c0730b0 2024-04-25T14:04:34.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.769+0000 7f4513200640 1 -- 172.21.15.86:0/1392333054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f450c069d00 con 0x7f450c0730b0 2024-04-25T14:04:34.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.769+0000 7f4503e00640 1 -- 172.21.15.86:0/1392333054 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f44fc05bd20 con 0x7f450c0730b0 2024-04-25T14:04:34.771 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:04:34.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.769+0000 7f4513200640 1 -- 172.21.15.86:0/1392333054 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f44ec06f430 msgr2=0x7f44ec071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:34.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.769+0000 7f4513200640 1 --2- 172.21.15.86:0/1392333054 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f44ec06f430 0x7f44ec071940 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f450c15fba0 tx=0x7f44f8006bd0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.769+0000 7f4513200640 1 -- 172.21.15.86:0/1392333054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f450c0730b0 msgr2=0x7f450c09a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:34.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.769+0000 7f4513200640 1 --2- 172.21.15.86:0/1392333054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f450c0730b0 0x7f450c09a7b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00d600 tx=0x7f44fc00dac0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.769+0000 7f4513200640 1 -- 172.21.15.86:0/1392333054 shutdown_connections 2024-04-25T14:04:34.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.769+0000 7f4513200640 1 --2- 172.21.15.86:0/1392333054 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f44ec06f430 0x7f44ec071940 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.769+0000 7f4513200640 1 --2- 172.21.15.86:0/1392333054 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f450c0d16e0 0x7f450c0ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.769+0000 7f4513200640 1 --2- 172.21.15.86:0/1392333054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f450c0cfca0 0x7f450c0cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.769+0000 7f4513200640 1 --2- 172.21.15.86:0/1392333054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f450c0730b0 0x7f450c09a7b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.769+0000 7f4513200640 1 -- 172.21.15.86:0/1392333054 >> 172.21.15.86:0/1392333054 conn(0x7f450c0b4650 msgr2=0x7f450c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:34.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.773+0000 7f4513200640 1 -- 172.21.15.86:0/1392333054 shutdown_connections 2024-04-25T14:04:34.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.773+0000 7f4513200640 1 -- 172.21.15.86:0/1392333054 wait complete. 2024-04-25T14:04:34.792 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:04:34.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.937+0000 7fe4e2e00640 1 Processor -- start 2024-04-25T14:04:34.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.937+0000 7fe4e2e00640 1 -- start start 2024-04-25T14:04:34.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4e2e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4dc15faa0 0x7fe4dc06d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:34.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4e2e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe4dc0aeea0 0x7fe4dc0af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:34.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4e2e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4dc06c670 con 0x7fe4dc0b59e0 2024-04-25T14:04:34.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4e2e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe4dc06d560 con 0x7fe4dc0aeea0 2024-04-25T14:04:34.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4e2e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe4dc06c950 con 0x7fe4dc15faa0 2024-04-25T14:04:34.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4d3400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe4dc0b59e0 0x7fe4dc0b5df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60692/0 (socket says 172.21.15.86:60692) 2024-04-25T14:04:34.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4d3400640 1 -- 172.21.15.86:0/1984701949 learned_addr learned my addr 172.21.15.86:0/1984701949 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:34.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4e0e00640 1 --2- 172.21.15.86:0/1984701949 >> v2:172.21.15.155:3300/0 conn(0x7fe4dc0aeea0 0x7fe4dc0af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:34.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4dbe00640 1 --2- 172.21.15.86:0/1984701949 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4dc15faa0 0x7fe4dc06d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:34.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4db400640 1 -- 172.21.15.86:0/1984701949 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 645748666 0 0) 0x7fe4dc06c670 con 0x7fe4dc0b59e0 2024-04-25T14:04:34.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4db400640 1 -- 172.21.15.86:0/1984701949 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4c0003530 con 0x7fe4dc0b59e0 2024-04-25T14:04:34.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4db400640 1 -- 172.21.15.86:0/1984701949 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 206526396 0 0) 0x7fe4c0003530 con 0x7fe4dc0b59e0 2024-04-25T14:04:34.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4db400640 1 -- 172.21.15.86:0/1984701949 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4dc15faa0 msgr2=0x7fe4dc06d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:34.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4db400640 1 --2- 172.21.15.86:0/1984701949 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4dc15faa0 0x7fe4dc06d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4db400640 1 -- 172.21.15.86:0/1984701949 >> v2:172.21.15.155:3300/0 conn(0x7fe4dc0aeea0 msgr2=0x7fe4dc0af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:34.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4db400640 1 --2- 172.21.15.86:0/1984701949 >> v2:172.21.15.155:3300/0 conn(0x7fe4dc0aeea0 0x7fe4dc0af2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4db400640 1 -- 172.21.15.86:0/1984701949 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4dc06cc30 con 0x7fe4dc0b59e0 2024-04-25T14:04:34.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4db400640 1 -- 172.21.15.86:0/1984701949 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fe4c4001680 con 0x7fe4dc0b59e0 2024-04-25T14:04:34.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4db400640 1 -- 172.21.15.86:0/1984701949 >> v1:172.21.15.86:6789/0 conn(0x7fe4dc0b59e0 legacy=0x7fe4dc0b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:34.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4db400640 1 --2- 172.21.15.86:0/1984701949 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4c0003a30 0x7fe4c0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:34.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4db400640 1 --2- 172.21.15.86:0/1984701949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4c0004810 0x7fe4c0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:34.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4db400640 1 --2- 172.21.15.86:0/1984701949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4c0005160 0x7fe4c000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:34.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4db400640 1 -- 172.21.15.86:0/1984701949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc06d560 con 0x7fe4c0005160 2024-04-25T14:04:34.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4db400640 1 -- 172.21.15.86:0/1984701949 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc06c950 con 0x7fe4c0004810 2024-04-25T14:04:34.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4db400640 1 -- 172.21.15.86:0/1984701949 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe4c000bff0 con 0x7fe4c0003a30 2024-04-25T14:04:34.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4e0e00640 1 --2- 172.21.15.86:0/1984701949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4c0005160 0x7fe4c000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:34.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4dbe00640 1 --2- 172.21.15.86:0/1984701949 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4c0003a30 0x7fe4c0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:34.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4d3400640 1 --2- 172.21.15.86:0/1984701949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4c0004810 0x7fe4c0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:34.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4e0e00640 1 -- 172.21.15.86:0/1984701949 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4c0003a30 msgr2=0x7fe4c0003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:34.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4e0e00640 1 --2- 172.21.15.86:0/1984701949 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4c0003a30 0x7fe4c0003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4e0e00640 1 -- 172.21.15.86:0/1984701949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4c0004810 msgr2=0x7fe4c0004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:34.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4e0e00640 1 --2- 172.21.15.86:0/1984701949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4c0004810 0x7fe4c0004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.941+0000 7fe4e0e00640 1 -- 172.21.15.86:0/1984701949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c0003530 con 0x7fe4c0005160 2024-04-25T14:04:34.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.945+0000 7fe4e0e00640 1 --2- 172.21.15.86:0/1984701949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4c0005160 0x7fe4c000b9d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009f90 tx=0x7fe4d401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b65e380ef2f9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:34.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.945+0000 7fe4db400640 1 -- 172.21.15.86:0/1984701949 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40b2070 con 0x7fe4c0005160 2024-04-25T14:04:34.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.945+0000 7fe4db400640 1 -- 172.21.15.86:0/1984701949 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4d40a5c90 con 0x7fe4c0005160 2024-04-25T14:04:34.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.945+0000 7fe4db400640 1 -- 172.21.15.86:0/1984701949 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4009c10 con 0x7fe4c0005160 2024-04-25T14:04:34.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.945+0000 7fe4e2e00640 1 -- 172.21.15.86:0/1984701949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4c0005160 msgr2=0x7fe4c000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:34.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.945+0000 7fe4e2e00640 1 --2- 172.21.15.86:0/1984701949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4c0005160 0x7fe4c000b9d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009f90 tx=0x7fe4d401cc80 comp rx=0 tx=0).stop 2024-04-25T14:04:34.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.945+0000 7fe4e0e00640 1 -- 172.21.15.86:0/1984701949 reap_dead start 2024-04-25T14:04:34.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.945+0000 7fe4e2e00640 1 -- 172.21.15.86:0/1984701949 shutdown_connections 2024-04-25T14:04:34.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.945+0000 7fe4e2e00640 1 -- 172.21.15.86:0/1984701949 >> 172.21.15.86:0/1984701949 conn(0x7fe4dc0d2230 msgr2=0x7fe4dc0d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:34.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.945+0000 7fe4e2e00640 1 -- 172.21.15.86:0/1984701949 shutdown_connections 2024-04-25T14:04:34.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.945+0000 7fe4e2e00640 1 -- 172.21.15.86:0/1984701949 wait complete. 2024-04-25T14:04:34.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.949+0000 7fe4e2e00640 1 Processor -- start 2024-04-25T14:04:34.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.949+0000 7fe4e2e00640 1 -- start start 2024-04-25T14:04:34.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.949+0000 7fe4e2e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4c0005160 0x7fe4dc1fa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:34.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.949+0000 7fe4e2e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4dc150dd0 0x7fe4dc0b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:34.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.949+0000 7fe4e2e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4dc0b2f10 0x7fe4dc14e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:34.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.949+0000 7fe4e2e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc15feb0 con 0x7fe4dc0b2f10 2024-04-25T14:04:34.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.949+0000 7fe4e2e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc06d1f0 con 0x7fe4c0005160 2024-04-25T14:04:34.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.949+0000 7fe4e2e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe4dc160030 con 0x7fe4dc150dd0 2024-04-25T14:04:34.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.949+0000 7fe4e0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4dc0b2f10 0x7fe4dc14e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:34.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.949+0000 7fe4e0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4dc0b2f10 0x7fe4dc14e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45244/0 (socket says 172.21.15.86:45244) 2024-04-25T14:04:34.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.949+0000 7fe4e0e00640 1 -- 172.21.15.86:0/2365718400 learned_addr learned my addr 172.21.15.86:0/2365718400 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:34.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.949+0000 7fe4dbe00640 1 --2- 172.21.15.86:0/2365718400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4c0005160 0x7fe4dc1fa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:34.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.949+0000 7fe4d3400640 1 --2- 172.21.15.86:0/2365718400 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4dc150dd0 0x7fe4dc0b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:34.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.953+0000 7fe4e0e00640 1 -- 172.21.15.86:0/2365718400 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4dc150dd0 msgr2=0x7fe4dc0b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:34.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.953+0000 7fe4e0e00640 1 --2- 172.21.15.86:0/2365718400 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4dc150dd0 0x7fe4dc0b29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.953+0000 7fe4e0e00640 1 -- 172.21.15.86:0/2365718400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4c0005160 msgr2=0x7fe4dc1fa7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:34.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.953+0000 7fe4e0e00640 1 --2- 172.21.15.86:0/2365718400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4c0005160 0x7fe4dc1fa7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:34.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.953+0000 7fe4e0e00640 1 -- 172.21.15.86:0/2365718400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4dc06dfa0 con 0x7fe4dc0b2f10 2024-04-25T14:04:34.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.953+0000 7fe4e0e00640 1 --2- 172.21.15.86:0/2365718400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4dc0b2f10 0x7fe4dc14e0f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40b0040 tx=0x7fe4d4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:34.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.953+0000 7fe4d8c00640 1 -- 172.21.15.86:0/2365718400 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40bd070 con 0x7fe4dc0b2f10 2024-04-25T14:04:34.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.953+0000 7fe4d8c00640 1 -- 172.21.15.86:0/2365718400 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4d40a5a90 con 0x7fe4dc0b2f10 2024-04-25T14:04:34.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.953+0000 7fe4e2e00640 1 -- 172.21.15.86:0/2365718400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4dc151600 con 0x7fe4dc0b2f10 2024-04-25T14:04:34.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.953+0000 7fe4d8c00640 1 -- 172.21.15.86:0/2365718400 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40b2040 con 0x7fe4dc0b2f10 2024-04-25T14:04:34.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.953+0000 7fe4e2e00640 1 -- 172.21.15.86:0/2365718400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4dc151bc0 con 0x7fe4dc0b2f10 2024-04-25T14:04:34.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.953+0000 7fe4e2e00640 1 -- 172.21.15.86:0/2365718400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4dc074cf0 con 0x7fe4dc0b2f10 2024-04-25T14:04:34.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.953+0000 7fe4d8c00640 1 -- 172.21.15.86:0/2365718400 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe4d40c0070 con 0x7fe4dc0b2f10 2024-04-25T14:04:34.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.957+0000 7fe4d8c00640 1 --2- 172.21.15.86:0/2365718400 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe4b806f400 0x7fe4b8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:34.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.957+0000 7fe4d8c00640 1 -- 172.21.15.86:0/2365718400 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe4d41294c0 con 0x7fe4dc0b2f10 2024-04-25T14:04:34.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.957+0000 7fe4dbe00640 1 --2- 172.21.15.86:0/2365718400 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe4b806f400 0x7fe4b8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:34.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.957+0000 7fe4dbe00640 1 --2- 172.21.15.86:0/2365718400 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe4b806f400 0x7fe4b8071910 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc072730 tx=0x7fe4cc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:34.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:34.961+0000 7fe4d8c00640 1 -- 172.21.15.86:0/2365718400 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe4d4016990 con 0x7fe4dc0b2f10 2024-04-25T14:04:35.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.121+0000 7fe4e2e00640 1 -- 172.21.15.86:0/2365718400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4dc0b55b0 con 0x7fe4dc0b2f10 2024-04-25T14:04:35.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.121+0000 7fe4d8c00640 1 -- 172.21.15.86:0/2365718400 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18902 (secure 0 0 0) 0x7fe4d40f5900 con 0x7fe4dc0b2f10 2024-04-25T14:04:35.124 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:35.124 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":26,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:04:31.930803+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:03:49.151473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:03:49.802597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T14:03:53.492008+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:04:35.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.125+0000 7fe4e2e00640 1 -- 172.21.15.86:0/2365718400 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe4b806f400 msgr2=0x7fe4b8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:35.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.125+0000 7fe4e2e00640 1 --2- 172.21.15.86:0/2365718400 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe4b806f400 0x7fe4b8071910 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc072730 tx=0x7fe4cc009190 comp rx=0 tx=0).stop 2024-04-25T14:04:35.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.125+0000 7fe4e2e00640 1 -- 172.21.15.86:0/2365718400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4dc0b2f10 msgr2=0x7fe4dc14e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:35.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.125+0000 7fe4e2e00640 1 --2- 172.21.15.86:0/2365718400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4dc0b2f10 0x7fe4dc14e0f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40b0040 tx=0x7fe4d4009750 comp rx=0 tx=0).stop 2024-04-25T14:04:35.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.125+0000 7fe4e2e00640 1 -- 172.21.15.86:0/2365718400 shutdown_connections 2024-04-25T14:04:35.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.125+0000 7fe4e2e00640 1 --2- 172.21.15.86:0/2365718400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4dc0b2f10 0x7fe4dc14e0f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:35.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.125+0000 7fe4e2e00640 1 --2- 172.21.15.86:0/2365718400 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe4b806f400 0x7fe4b8071910 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:35.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.125+0000 7fe4e2e00640 1 --2- 172.21.15.86:0/2365718400 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4dc150dd0 0x7fe4dc0b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:35.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.125+0000 7fe4e2e00640 1 --2- 172.21.15.86:0/2365718400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4c0005160 0x7fe4dc1fa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:35.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.125+0000 7fe4e2e00640 1 -- 172.21.15.86:0/2365718400 >> 172.21.15.86:0/2365718400 conn(0x7fe4dc0d2230 msgr2=0x7fe4dc15ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:35.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.125+0000 7fe4e2e00640 1 -- 172.21.15.86:0/2365718400 shutdown_connections 2024-04-25T14:04:35.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.125+0000 7fe4e2e00640 1 -- 172.21.15.86:0/2365718400 wait complete. 2024-04-25T14:04:35.146 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:03:49.151473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:03:49.802597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T14:03:53.492008+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T14:04:35.146 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:04:35.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.293+0000 7f8c7b200640 1 Processor -- start 2024-04-25T14:04:35.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.293+0000 7f8c7b200640 1 -- start start 2024-04-25T14:04:35.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.293+0000 7f8c7b200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8c740d16e0 0x7f8c740d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:35.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.293+0000 7f8c7b200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8c740730b0 0x7f8c740734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:35.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.293+0000 7f8c7b200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c740d2030 con 0x7f8c740cfca0 2024-04-25T14:04:35.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.293+0000 7f8c7b200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8c74070470 con 0x7f8c740d16e0 2024-04-25T14:04:35.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.293+0000 7f8c7b200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f8c74072bd0 con 0x7f8c740730b0 2024-04-25T14:04:35.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.297+0000 7f8c73400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8c740d16e0 0x7f8c740d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:35.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.297+0000 7f8c73400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8c740d16e0 0x7f8c740d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45254/0 (socket says 172.21.15.86:45254) 2024-04-25T14:04:35.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.297+0000 7f8c73400640 1 -- 172.21.15.86:0/1023879773 learned_addr learned my addr 172.21.15.86:0/1023879773 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:35.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.297+0000 7f8c79200640 1 --2- 172.21.15.86:0/1023879773 >> v2:172.21.15.155:3300/0 conn(0x7f8c740730b0 0x7f8c740734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:35.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.297+0000 7f8c73400640 1 -- 172.21.15.86:0/1023879773 >> v2:172.21.15.155:3300/0 conn(0x7f8c740730b0 msgr2=0x7f8c740734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:35.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.297+0000 7f8c73400640 1 --2- 172.21.15.86:0/1023879773 >> v2:172.21.15.155:3300/0 conn(0x7f8c740730b0 0x7f8c740734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:35.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.297+0000 7f8c73400640 1 -- 172.21.15.86:0/1023879773 >> v1:172.21.15.155:6789/0 conn(0x7f8c740cfca0 legacy=0x7f8c7406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:35.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.297+0000 7f8c73400640 1 -- 172.21.15.86:0/1023879773 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7406fbd0 con 0x7f8c740d16e0 2024-04-25T14:04:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.297+0000 7f8c73400640 1 --2- 172.21.15.86:0/1023879773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c740d16e0 0x7f8c740d1af0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8c740729c0 tx=0x7f8c6001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26e7a65fc94474a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.297+0000 7f8c72a00640 1 -- 172.21.15.86:0/1023879773 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c600ab430 con 0x7f8c740d16e0 2024-04-25T14:04:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.297+0000 7f8c72a00640 1 -- 172.21.15.86:0/1023879773 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c600b0070 con 0x7f8c740d16e0 2024-04-25T14:04:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.297+0000 7f8c72a00640 1 -- 172.21.15.86:0/1023879773 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c600ab7f0 con 0x7f8c740d16e0 2024-04-25T14:04:35.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.301+0000 7f8c7b200640 1 -- 172.21.15.86:0/1023879773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c740d16e0 msgr2=0x7f8c740d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:35.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.301+0000 7f8c7b200640 1 --2- 172.21.15.86:0/1023879773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c740d16e0 0x7f8c740d1af0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8c740729c0 tx=0x7f8c6001c520 comp rx=0 tx=0).stop 2024-04-25T14:04:35.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.301+0000 7f8c7b200640 1 -- 172.21.15.86:0/1023879773 shutdown_connections 2024-04-25T14:04:35.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.301+0000 7f8c7b200640 1 --2- 172.21.15.86:0/1023879773 >> v2:172.21.15.155:3300/0 conn(0x7f8c740730b0 0x7f8c740734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:35.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.301+0000 7f8c7b200640 1 --2- 172.21.15.86:0/1023879773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c740d16e0 0x7f8c740d1af0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:35.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.301+0000 7f8c7b200640 1 -- 172.21.15.86:0/1023879773 >> 172.21.15.86:0/1023879773 conn(0x7f8c740b4650 msgr2=0x7f8c740b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:35.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.301+0000 7f8c7b200640 1 -- 172.21.15.86:0/1023879773 shutdown_connections 2024-04-25T14:04:35.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.301+0000 7f8c7b200640 1 -- 172.21.15.86:0/1023879773 wait complete. 2024-04-25T14:04:35.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.301+0000 7f8c7b200640 1 Processor -- start 2024-04-25T14:04:35.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.301+0000 7f8c7b200640 1 -- start start 2024-04-25T14:04:35.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.305+0000 7f8c7b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c740730b0 0x7f8c741f9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:35.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.305+0000 7f8c7b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c740cfca0 0x7f8c740c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:35.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.305+0000 7f8c7b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c740d16e0 0x7f8c740c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:35.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.305+0000 7f8c7b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c74169a70 con 0x7f8c740cfca0 2024-04-25T14:04:35.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.305+0000 7f8c7b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8c741698f0 con 0x7f8c740730b0 2024-04-25T14:04:35.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.305+0000 7f8c7b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8c74169bf0 con 0x7f8c740d16e0 2024-04-25T14:04:35.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.305+0000 7f8c73400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c740cfca0 0x7f8c740c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:35.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.305+0000 7f8c73400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c740cfca0 0x7f8c740c9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45262/0 (socket says 172.21.15.86:45262) 2024-04-25T14:04:35.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.305+0000 7f8c73400640 1 -- 172.21.15.86:0/23702123 learned_addr learned my addr 172.21.15.86:0/23702123 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:35.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.305+0000 7f8c73e00640 1 --2- 172.21.15.86:0/23702123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c740730b0 0x7f8c741f9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:35.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.305+0000 7f8c79200640 1 --2- 172.21.15.86:0/23702123 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c740d16e0 0x7f8c740c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:35.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.305+0000 7f8c73400640 1 -- 172.21.15.86:0/23702123 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c740d16e0 msgr2=0x7f8c740c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:35.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.305+0000 7f8c73400640 1 --2- 172.21.15.86:0/23702123 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c740d16e0 0x7f8c740c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:35.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.305+0000 7f8c73400640 1 -- 172.21.15.86:0/23702123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c740730b0 msgr2=0x7f8c741f9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:35.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.305+0000 7f8c73400640 1 --2- 172.21.15.86:0/23702123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c740730b0 0x7f8c741f9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:35.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.305+0000 7f8c73400640 1 -- 172.21.15.86:0/23702123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c740aed90 con 0x7f8c740cfca0 2024-04-25T14:04:35.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.305+0000 7f8c73400640 1 --2- 172.21.15.86:0/23702123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c740cfca0 0x7f8c740c9220 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8c600ad040 tx=0x7f8c6001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:35.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.305+0000 7f8c6be00640 1 -- 172.21.15.86:0/23702123 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c600be5f0 con 0x7f8c740cfca0 2024-04-25T14:04:35.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.305+0000 7f8c6be00640 1 -- 172.21.15.86:0/23702123 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c600be790 con 0x7f8c740cfca0 2024-04-25T14:04:35.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.305+0000 7f8c7b200640 1 -- 172.21.15.86:0/23702123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c740af070 con 0x7f8c740cfca0 2024-04-25T14:04:35.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.305+0000 7f8c6be00640 1 -- 172.21.15.86:0/23702123 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c600beb80 con 0x7f8c740cfca0 2024-04-25T14:04:35.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.305+0000 7f8c7b200640 1 -- 172.21.15.86:0/23702123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c740af630 con 0x7f8c740cfca0 2024-04-25T14:04:35.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.305+0000 7f8c7b200640 1 -- 172.21.15.86:0/23702123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c7409f4b0 con 0x7f8c740cfca0 2024-04-25T14:04:35.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.309+0000 7f8c6be00640 1 -- 172.21.15.86:0/23702123 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8c600b43f0 con 0x7f8c740cfca0 2024-04-25T14:04:35.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.309+0000 7f8c6be00640 1 --2- 172.21.15.86:0/23702123 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8c5406f430 0x7f8c54071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:35.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.309+0000 7f8c6be00640 1 -- 172.21.15.86:0/23702123 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f8c600a8070 con 0x7f8c740cfca0 2024-04-25T14:04:35.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.309+0000 7f8c73e00640 1 --2- 172.21.15.86:0/23702123 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8c5406f430 0x7f8c54071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:35.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.309+0000 7f8c73e00640 1 --2- 172.21.15.86:0/23702123 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8c5406f430 0x7f8c54071940 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8c64007fd0 tx=0x7f8c64009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:35.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.313+0000 7f8c6be00640 1 -- 172.21.15.86:0/23702123 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8c60016110 con 0x7f8c740cfca0 2024-04-25T14:04:35.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.473+0000 7f8c7b200640 1 -- 172.21.15.86:0/23702123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8c74069d00 con 0x7f8c740cfca0 2024-04-25T14:04:35.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.473+0000 7f8c6be00640 1 -- 172.21.15.86:0/23702123 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f8c600f6290 con 0x7f8c740cfca0 2024-04-25T14:04:35.474 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:04:35.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.473+0000 7f8c7b200640 1 -- 172.21.15.86:0/23702123 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8c5406f430 msgr2=0x7f8c54071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:35.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.473+0000 7f8c7b200640 1 --2- 172.21.15.86:0/23702123 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8c5406f430 0x7f8c54071940 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8c64007fd0 tx=0x7f8c64009040 comp rx=0 tx=0).stop 2024-04-25T14:04:35.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.473+0000 7f8c7b200640 1 -- 172.21.15.86:0/23702123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c740cfca0 msgr2=0x7f8c740c9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:35.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.473+0000 7f8c7b200640 1 --2- 172.21.15.86:0/23702123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c740cfca0 0x7f8c740c9220 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8c600ad040 tx=0x7f8c6001ad60 comp rx=0 tx=0).stop 2024-04-25T14:04:35.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.473+0000 7f8c7b200640 1 -- 172.21.15.86:0/23702123 shutdown_connections 2024-04-25T14:04:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.473+0000 7f8c7b200640 1 --2- 172.21.15.86:0/23702123 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8c5406f430 0x7f8c54071940 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.473+0000 7f8c7b200640 1 --2- 172.21.15.86:0/23702123 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c740d16e0 0x7f8c740c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.473+0000 7f8c7b200640 1 --2- 172.21.15.86:0/23702123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c740cfca0 0x7f8c740c9220 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.473+0000 7f8c7b200640 1 --2- 172.21.15.86:0/23702123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c740730b0 0x7f8c741f9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.473+0000 7f8c7b200640 1 -- 172.21.15.86:0/23702123 >> 172.21.15.86:0/23702123 conn(0x7f8c740b4650 msgr2=0x7f8c740cdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.473+0000 7f8c7b200640 1 -- 172.21.15.86:0/23702123 shutdown_connections 2024-04-25T14:04:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.473+0000 7f8c7b200640 1 -- 172.21.15.86:0/23702123 wait complete. 2024-04-25T14:04:35.496 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:04:35.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.645+0000 7fe543a00640 1 Processor -- start 2024-04-25T14:04:35.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.645+0000 7fe543a00640 1 -- start start 2024-04-25T14:04:35.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.649+0000 7fe543a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe53c15c2a0 0x7fe53c0ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:35.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.649+0000 7fe543a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe53c15feb0 0x7fe53c1602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:35.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.649+0000 7fe543a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe53c0b0a60 con 0x7fe53c06fd50 2024-04-25T14:04:35.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.649+0000 7fe543a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe53c06ef90 con 0x7fe53c15feb0 2024-04-25T14:04:35.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.649+0000 7fe543a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe53c06f270 con 0x7fe53c15c2a0 2024-04-25T14:04:35.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.649+0000 7fe541a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe53c15feb0 0x7fe53c1602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:35.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.649+0000 7fe53be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe53c06fd50 0x7fe53c070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60702/0 (socket says 172.21.15.86:60702) 2024-04-25T14:04:35.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.649+0000 7fe53be00640 1 -- 172.21.15.86:0/2056284492 learned_addr learned my addr 172.21.15.86:0/2056284492 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:35.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.649+0000 7fe541000640 1 --2- 172.21.15.86:0/2056284492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe53c15c2a0 0x7fe53c0ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:35.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.649+0000 7fe541a00640 1 -- 172.21.15.86:0/2056284492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe53c15c2a0 msgr2=0x7fe53c0ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:35.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.649+0000 7fe541a00640 1 --2- 172.21.15.86:0/2056284492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe53c15c2a0 0x7fe53c0ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:35.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.649+0000 7fe541a00640 1 -- 172.21.15.86:0/2056284492 >> v1:172.21.15.86:6789/0 conn(0x7fe53c06fd50 legacy=0x7fe53c070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:35.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.649+0000 7fe541a00640 1 -- 172.21.15.86:0/2056284492 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53c06f550 con 0x7fe53c15feb0 2024-04-25T14:04:35.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.649+0000 7fe53b400640 1 -- 172.21.15.86:0/2056284492 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 429201048 0 0) 0x7fe53c0b0a60 con 0x7fe53c06fd50 2024-04-25T14:04:35.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.653+0000 7fe541a00640 1 --2- 172.21.15.86:0/2056284492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe53c15feb0 0x7fe53c1602c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe534009d90 tx=0x7fe53401d310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb54703c3bdae8de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:35.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.653+0000 7fe53b400640 1 -- 172.21.15.86:0/2056284492 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5340b2070 con 0x7fe53c15feb0 2024-04-25T14:04:35.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.653+0000 7fe53b400640 1 -- 172.21.15.86:0/2056284492 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5340a5990 con 0x7fe53c15feb0 2024-04-25T14:04:35.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.653+0000 7fe53b400640 1 -- 172.21.15.86:0/2056284492 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5340a5d60 con 0x7fe53c15feb0 2024-04-25T14:04:35.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.653+0000 7fe543a00640 1 -- 172.21.15.86:0/2056284492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe53c15feb0 msgr2=0x7fe53c1602c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:35.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.653+0000 7fe543a00640 1 --2- 172.21.15.86:0/2056284492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe53c15feb0 0x7fe53c1602c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe534009d90 tx=0x7fe53401d310 comp rx=0 tx=0).stop 2024-04-25T14:04:35.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.653+0000 7fe543a00640 1 -- 172.21.15.86:0/2056284492 shutdown_connections 2024-04-25T14:04:35.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.653+0000 7fe543a00640 1 --2- 172.21.15.86:0/2056284492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe53c15feb0 0x7fe53c1602c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:35.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.653+0000 7fe543a00640 1 --2- 172.21.15.86:0/2056284492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe53c15c2a0 0x7fe53c0ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:35.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.653+0000 7fe543a00640 1 -- 172.21.15.86:0/2056284492 >> 172.21.15.86:0/2056284492 conn(0x7fe53c0b4650 msgr2=0x7fe53c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:35.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.653+0000 7fe543a00640 1 -- 172.21.15.86:0/2056284492 shutdown_connections 2024-04-25T14:04:35.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.653+0000 7fe543a00640 1 -- 172.21.15.86:0/2056284492 wait complete. 2024-04-25T14:04:35.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.657+0000 7fe543a00640 1 Processor -- start 2024-04-25T14:04:35.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.657+0000 7fe543a00640 1 -- start start 2024-04-25T14:04:35.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.657+0000 7fe543a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe53c06fd50 0x7fe53c155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:35.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.657+0000 7fe543a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe53c15c2a0 0x7fe53c158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:35.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.657+0000 7fe543a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe53c15feb0 0x7fe53c159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:35.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.657+0000 7fe543a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c160d90 con 0x7fe53c06fd50 2024-04-25T14:04:35.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.657+0000 7fe543a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c160c10 con 0x7fe53c15c2a0 2024-04-25T14:04:35.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.657+0000 7fe543a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe53c0ae670 con 0x7fe53c15feb0 2024-04-25T14:04:35.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.657+0000 7fe541000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe53c06fd50 0x7fe53c155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:35.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.657+0000 7fe541000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe53c06fd50 0x7fe53c155ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45282/0 (socket says 172.21.15.86:45282) 2024-04-25T14:04:35.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.657+0000 7fe541000640 1 -- 172.21.15.86:0/2299506833 learned_addr learned my addr 172.21.15.86:0/2299506833 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:35.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.657+0000 7fe53be00640 1 --2- 172.21.15.86:0/2299506833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe53c15c2a0 0x7fe53c158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:35.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.657+0000 7fe541a00640 1 --2- 172.21.15.86:0/2299506833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe53c15feb0 0x7fe53c159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:35.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.657+0000 7fe541000640 1 -- 172.21.15.86:0/2299506833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe53c15feb0 msgr2=0x7fe53c159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:35.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.657+0000 7fe541000640 1 --2- 172.21.15.86:0/2299506833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe53c15feb0 0x7fe53c159340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:35.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.657+0000 7fe541000640 1 -- 172.21.15.86:0/2299506833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe53c15c2a0 msgr2=0x7fe53c158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:35.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.657+0000 7fe541000640 1 --2- 172.21.15.86:0/2299506833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe53c15c2a0 0x7fe53c158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.657+0000 7fe541000640 1 -- 172.21.15.86:0/2299506833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53c0b2f90 con 0x7fe53c06fd50 2024-04-25T14:04:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.657+0000 7fe541000640 1 --2- 172.21.15.86:0/2299506833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe53c06fd50 0x7fe53c155ee0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe53000e870 tx=0x7fe53000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.657+0000 7fe538c00640 1 -- 172.21.15.86:0/2299506833 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530017070 con 0x7fe53c06fd50 2024-04-25T14:04:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.657+0000 7fe538c00640 1 -- 172.21.15.86:0/2299506833 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe53000ee60 con 0x7fe53c06fd50 2024-04-25T14:04:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.657+0000 7fe543a00640 1 -- 172.21.15.86:0/2299506833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe53c0b3270 con 0x7fe53c06fd50 2024-04-25T14:04:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.657+0000 7fe538c00640 1 -- 172.21.15.86:0/2299506833 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53000cb40 con 0x7fe53c06fd50 2024-04-25T14:04:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.657+0000 7fe543a00640 1 -- 172.21.15.86:0/2299506833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe53c0b3830 con 0x7fe53c06fd50 2024-04-25T14:04:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.657+0000 7fe543a00640 1 -- 172.21.15.86:0/2299506833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe53c069d00 con 0x7fe53c06fd50 2024-04-25T14:04:35.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.661+0000 7fe538c00640 1 -- 172.21.15.86:0/2299506833 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe53001a070 con 0x7fe53c06fd50 2024-04-25T14:04:35.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.661+0000 7fe538c00640 1 --2- 172.21.15.86:0/2299506833 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe51c06f400 0x7fe51c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:35.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.661+0000 7fe538c00640 1 -- 172.21.15.86:0/2299506833 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe53008f9b0 con 0x7fe53c06fd50 2024-04-25T14:04:35.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.661+0000 7fe53be00640 1 --2- 172.21.15.86:0/2299506833 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe51c06f400 0x7fe51c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:35.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.661+0000 7fe53be00640 1 --2- 172.21.15.86:0/2299506833 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe51c06f400 0x7fe51c071910 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe53c156c50 tx=0x7fe528007c10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:35.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.665+0000 7fe538c00640 1 -- 172.21.15.86:0/2299506833 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe53005c400 con 0x7fe53c06fd50 2024-04-25T14:04:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.825+0000 7fe543a00640 1 -- 172.21.15.86:0/2299506833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe53c076bf0 con 0x7fe53c06fd50 2024-04-25T14:04:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.825+0000 7fe538c00640 1 -- 172.21.15.86:0/2299506833 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fe53005bd00 con 0x7fe53c06fd50 2024-04-25T14:04:35.828 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:04:35.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.829+0000 7fe543a00640 1 -- 172.21.15.86:0/2299506833 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe51c06f400 msgr2=0x7fe51c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:35.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.829+0000 7fe543a00640 1 --2- 172.21.15.86:0/2299506833 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe51c06f400 0x7fe51c071910 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe53c156c50 tx=0x7fe528007c10 comp rx=0 tx=0).stop 2024-04-25T14:04:35.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.829+0000 7fe543a00640 1 -- 172.21.15.86:0/2299506833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe53c06fd50 msgr2=0x7fe53c155ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:35.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.829+0000 7fe543a00640 1 --2- 172.21.15.86:0/2299506833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe53c06fd50 0x7fe53c155ee0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe53000e870 tx=0x7fe53000ed30 comp rx=0 tx=0).stop 2024-04-25T14:04:35.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.829+0000 7fe543a00640 1 -- 172.21.15.86:0/2299506833 shutdown_connections 2024-04-25T14:04:35.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.829+0000 7fe543a00640 1 --2- 172.21.15.86:0/2299506833 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe51c06f400 0x7fe51c071910 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:35.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.829+0000 7fe543a00640 1 --2- 172.21.15.86:0/2299506833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe53c15feb0 0x7fe53c159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:35.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.829+0000 7fe543a00640 1 --2- 172.21.15.86:0/2299506833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe53c15c2a0 0x7fe53c158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:35.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.829+0000 7fe543a00640 1 --2- 172.21.15.86:0/2299506833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe53c06fd50 0x7fe53c155ee0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:35.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.829+0000 7fe543a00640 1 -- 172.21.15.86:0/2299506833 >> 172.21.15.86:0/2299506833 conn(0x7fe53c0b4650 msgr2=0x7fe53c15cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:35.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.829+0000 7fe543a00640 1 -- 172.21.15.86:0/2299506833 shutdown_connections 2024-04-25T14:04:35.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.829+0000 7fe543a00640 1 -- 172.21.15.86:0/2299506833 wait complete. 2024-04-25T14:04:35.850 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:04:36.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.997+0000 7f6e0a000640 1 Processor -- start 2024-04-25T14:04:36.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:35.997+0000 7f6e0a000640 1 -- start start 2024-04-25T14:04:36.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.001+0000 7f6e0a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6e04070330 0x7f6e040d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:36.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.001+0000 7f6e0a000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6e040d1c20 0x7f6e0406ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:36.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.001+0000 7f6e0a000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e040d2030 con 0x7f6e0406f270 2024-04-25T14:04:36.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.001+0000 7f6e0a000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6e0415e5c0 con 0x7f6e04070330 2024-04-25T14:04:36.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.001+0000 7f6e0a000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f6e0415e8a0 con 0x7f6e040d1c20 2024-04-25T14:04:36.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.001+0000 7f6e03400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6e04070330 0x7f6e040d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:36.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.001+0000 7f6e03400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6e04070330 0x7f6e040d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45286/0 (socket says 172.21.15.86:45286) 2024-04-25T14:04:36.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.001+0000 7f6e03400640 1 -- 172.21.15.86:0/4221480944 learned_addr learned my addr 172.21.15.86:0/4221480944 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:36.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.001+0000 7f6dfaa00640 1 --2- 172.21.15.86:0/4221480944 >> v2:172.21.15.155:3300/0 conn(0x7f6e040d1c20 0x7f6e0406ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:36.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.001+0000 7f6e03400640 1 -- 172.21.15.86:0/4221480944 >> v2:172.21.15.155:3300/0 conn(0x7f6e040d1c20 msgr2=0x7f6e0406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:36.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.001+0000 7f6e03400640 1 --2- 172.21.15.86:0/4221480944 >> v2:172.21.15.155:3300/0 conn(0x7f6e040d1c20 0x7f6e0406ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.001+0000 7f6e03400640 1 -- 172.21.15.86:0/4221480944 >> v1:172.21.15.155:6789/0 conn(0x7f6e0406f270 legacy=0x7f6e0406f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:36.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.001+0000 7f6e03400640 1 -- 172.21.15.86:0/4221480944 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e0415eb80 con 0x7f6e04070330 2024-04-25T14:04:36.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.005+0000 7f6e03400640 1 --2- 172.21.15.86:0/4221480944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e04070330 0x7f6e040d16e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6df4009870 tx=0x7f6df401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c640ded500cac46b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:36.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.005+0000 7f6e02a00640 1 -- 172.21.15.86:0/4221480944 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df40b0070 con 0x7f6e04070330 2024-04-25T14:04:36.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.005+0000 7f6e02a00640 1 -- 172.21.15.86:0/4221480944 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6df40ab600 con 0x7f6e04070330 2024-04-25T14:04:36.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.005+0000 7f6e02a00640 1 -- 172.21.15.86:0/4221480944 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df40ab9d0 con 0x7f6e04070330 2024-04-25T14:04:36.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.005+0000 7f6e0a000640 1 -- 172.21.15.86:0/4221480944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e04070330 msgr2=0x7f6e040d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:36.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.005+0000 7f6e0a000640 1 --2- 172.21.15.86:0/4221480944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e04070330 0x7f6e040d16e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6df4009870 tx=0x7f6df401c490 comp rx=0 tx=0).stop 2024-04-25T14:04:36.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.005+0000 7f6e0a000640 1 -- 172.21.15.86:0/4221480944 shutdown_connections 2024-04-25T14:04:36.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.005+0000 7f6e0a000640 1 --2- 172.21.15.86:0/4221480944 >> v2:172.21.15.155:3300/0 conn(0x7f6e040d1c20 0x7f6e0406ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.005+0000 7f6e0a000640 1 --2- 172.21.15.86:0/4221480944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e04070330 0x7f6e040d16e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.005+0000 7f6e0a000640 1 -- 172.21.15.86:0/4221480944 >> 172.21.15.86:0/4221480944 conn(0x7f6e040b4650 msgr2=0x7f6e040b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:36.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.005+0000 7f6e0a000640 1 -- 172.21.15.86:0/4221480944 shutdown_connections 2024-04-25T14:04:36.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.005+0000 7f6e0a000640 1 -- 172.21.15.86:0/4221480944 wait complete. 2024-04-25T14:04:36.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.005+0000 7f6e0a000640 1 Processor -- start 2024-04-25T14:04:36.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.009+0000 7f6e0a000640 1 -- start start 2024-04-25T14:04:36.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.009+0000 7f6e0a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e0406f270 0x7f6e040cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:36.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.009+0000 7f6e0a000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6e04070330 0x7f6e040cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:36.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.009+0000 7f6e0a000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e040d1c20 0x7f6e040c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:36.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.009+0000 7f6e0a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e040708c0 con 0x7f6e0406f270 2024-04-25T14:04:36.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.009+0000 7f6e0a000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6e04070740 con 0x7f6e040d1c20 2024-04-25T14:04:36.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.009+0000 7f6e0a000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6e0416afc0 con 0x7f6e04070330 2024-04-25T14:04:36.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.009+0000 7f6e03400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e0406f270 0x7f6e040cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:36.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.009+0000 7f6e03400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e0406f270 0x7f6e040cace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45300/0 (socket says 172.21.15.86:45300) 2024-04-25T14:04:36.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.009+0000 7f6e03400640 1 -- 172.21.15.86:0/1115949492 learned_addr learned my addr 172.21.15.86:0/1115949492 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:36.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.009+0000 7f6dfaa00640 1 --2- 172.21.15.86:0/1115949492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6e04070330 0x7f6e040cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:36.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.009+0000 7f6e03e00640 1 --2- 172.21.15.86:0/1115949492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e040d1c20 0x7f6e040c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:36.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.009+0000 7f6e03400640 1 -- 172.21.15.86:0/1115949492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6e04070330 msgr2=0x7f6e040cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:36.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.009+0000 7f6e03400640 1 --2- 172.21.15.86:0/1115949492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6e04070330 0x7f6e040cb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.009+0000 7f6e03400640 1 -- 172.21.15.86:0/1115949492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e040d1c20 msgr2=0x7f6e040c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:36.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.009+0000 7f6e03400640 1 --2- 172.21.15.86:0/1115949492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e040d1c20 0x7f6e040c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.009+0000 7f6e03400640 1 -- 172.21.15.86:0/1115949492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e0409ba50 con 0x7f6e0406f270 2024-04-25T14:04:36.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.009+0000 7f6e03e00640 1 --2- 172.21.15.86:0/1115949492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e040d1c20 0x7f6e040c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:36.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.009+0000 7f6e03400640 1 --2- 172.21.15.86:0/1115949492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e0406f270 0x7f6e040cace0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6df40ae040 tx=0x7f6df401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:36.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.009+0000 7f6dfbe00640 1 -- 172.21.15.86:0/1115949492 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df40ba020 con 0x7f6e0406f270 2024-04-25T14:04:36.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.009+0000 7f6dfbe00640 1 -- 172.21.15.86:0/1115949492 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6df40b6420 con 0x7f6e0406f270 2024-04-25T14:04:36.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.009+0000 7f6e0a000640 1 -- 172.21.15.86:0/1115949492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e0409bd30 con 0x7f6e0406f270 2024-04-25T14:04:36.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.009+0000 7f6dfbe00640 1 -- 172.21.15.86:0/1115949492 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df40b0040 con 0x7f6e0406f270 2024-04-25T14:04:36.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.009+0000 7f6e0a000640 1 -- 172.21.15.86:0/1115949492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e0409d9f0 con 0x7f6e0406f270 2024-04-25T14:04:36.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.009+0000 7f6e0a000640 1 -- 172.21.15.86:0/1115949492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e04069d00 con 0x7f6e0406f270 2024-04-25T14:04:36.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.013+0000 7f6dfbe00640 1 -- 172.21.15.86:0/1115949492 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6df40bd020 con 0x7f6e0406f270 2024-04-25T14:04:36.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.013+0000 7f6dfbe00640 1 --2- 172.21.15.86:0/1115949492 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6de006f430 0x7f6de0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:36.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.013+0000 7f6dfbe00640 1 -- 172.21.15.86:0/1115949492 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f6df41297c0 con 0x7f6e0406f270 2024-04-25T14:04:36.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.013+0000 7f6dfaa00640 1 --2- 172.21.15.86:0/1115949492 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6de006f430 0x7f6de0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:36.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.013+0000 7f6dfaa00640 1 --2- 172.21.15.86:0/1115949492 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6de006f430 0x7f6de0071940 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6e040cd560 tx=0x7f6dec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:36.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.017+0000 7f6dfbe00640 1 -- 172.21.15.86:0/1115949492 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6df40c4030 con 0x7f6e0406f270 2024-04-25T14:04:36.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.177+0000 7f6e0a000640 1 -- 172.21.15.86:0/1115949492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6e04076bf0 con 0x7f6e0406f270 2024-04-25T14:04:36.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.177+0000 7f6dfbe00640 1 -- 172.21.15.86:0/1115949492 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f6df4016110 con 0x7f6e0406f270 2024-04-25T14:04:36.178 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:04:36.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.177+0000 7f6e0a000640 1 -- 172.21.15.86:0/1115949492 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6de006f430 msgr2=0x7f6de0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:36.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.177+0000 7f6e0a000640 1 --2- 172.21.15.86:0/1115949492 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6de006f430 0x7f6de0071940 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6e040cd560 tx=0x7f6dec008040 comp rx=0 tx=0).stop 2024-04-25T14:04:36.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.177+0000 7f6e0a000640 1 -- 172.21.15.86:0/1115949492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e0406f270 msgr2=0x7f6e040cace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:36.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.177+0000 7f6e0a000640 1 --2- 172.21.15.86:0/1115949492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e0406f270 0x7f6e040cace0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6df40ae040 tx=0x7f6df401acd0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.177+0000 7f6e0a000640 1 -- 172.21.15.86:0/1115949492 shutdown_connections 2024-04-25T14:04:36.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.177+0000 7f6e0a000640 1 --2- 172.21.15.86:0/1115949492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e040d1c20 0x7f6e040c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.177+0000 7f6e0a000640 1 --2- 172.21.15.86:0/1115949492 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6de006f430 0x7f6de0071940 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.177+0000 7f6e0a000640 1 --2- 172.21.15.86:0/1115949492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6e04070330 0x7f6e040cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.177+0000 7f6e0a000640 1 --2- 172.21.15.86:0/1115949492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e0406f270 0x7f6e040cace0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.177+0000 7f6e0a000640 1 -- 172.21.15.86:0/1115949492 >> 172.21.15.86:0/1115949492 conn(0x7f6e040b4650 msgr2=0x7f6e0415ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:36.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.177+0000 7f6e0a000640 1 -- 172.21.15.86:0/1115949492 shutdown_connections 2024-04-25T14:04:36.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.177+0000 7f6e0a000640 1 -- 172.21.15.86:0/1115949492 wait complete. 2024-04-25T14:04:36.200 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T14:04:36.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.345+0000 7f7769a00640 1 Processor -- start 2024-04-25T14:04:36.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.345+0000 7f7769a00640 1 -- start start 2024-04-25T14:04:36.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.349+0000 7f7769a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77640d16e0 0x7f77640d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.349+0000 7f7769a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f77640730b0 0x7f77640734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.349+0000 7f7769a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77640d2030 con 0x7f77640cfca0 2024-04-25T14:04:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.349+0000 7f7769a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f7764070470 con 0x7f77640730b0 2024-04-25T14:04:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.349+0000 7f7769a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7764072bd0 con 0x7f77640d16e0 2024-04-25T14:04:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.349+0000 7f7763400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f77640730b0 0x7f77640734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.349+0000 7f7762000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77640d16e0 0x7f77640d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.349+0000 7f7762000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77640d16e0 0x7f77640d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:50902/0 (socket says 172.21.15.86:50902) 2024-04-25T14:04:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.349+0000 7f7763400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f77640730b0 0x7f77640734c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:54324/0 (socket says 172.21.15.86:54324) 2024-04-25T14:04:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.349+0000 7f7762000640 1 -- 172.21.15.86:0/2019118243 learned_addr learned my addr 172.21.15.86:0/2019118243 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:36.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.349+0000 7f7762000640 1 -- 172.21.15.86:0/2019118243 >> v2:172.21.15.155:3300/0 conn(0x7f77640730b0 msgr2=0x7f77640734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:36.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.349+0000 7f7762000640 1 --2- 172.21.15.86:0/2019118243 >> v2:172.21.15.155:3300/0 conn(0x7f77640730b0 0x7f77640734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.349+0000 7f7762000640 1 -- 172.21.15.86:0/2019118243 >> v1:172.21.15.155:6789/0 conn(0x7f77640cfca0 legacy=0x7f776406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:36.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.349+0000 7f7762000640 1 -- 172.21.15.86:0/2019118243 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776406fbd0 con 0x7f77640d16e0 2024-04-25T14:04:36.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.349+0000 7f7763400640 1 --2- 172.21.15.86:0/2019118243 >> v2:172.21.15.155:3300/0 conn(0x7f77640730b0 0x7f77640734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:36.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.349+0000 7f7761600640 1 -- 172.21.15.86:0/2019118243 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1011333443 0 0) 0x7f774c001120 con 0x7f77640cfca0 2024-04-25T14:04:36.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.353+0000 7f7762000640 1 --2- 172.21.15.86:0/2019118243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77640d16e0 0x7f77640d1af0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7754009870 tx=0x7f775401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=990ac68ac0b8edb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:36.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.353+0000 7f7761600640 1 -- 172.21.15.86:0/2019118243 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77540a7070 con 0x7f77640d16e0 2024-04-25T14:04:36.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.353+0000 7f7761600640 1 -- 172.21.15.86:0/2019118243 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77540b0070 con 0x7f77640d16e0 2024-04-25T14:04:36.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.353+0000 7f7761600640 1 -- 172.21.15.86:0/2019118243 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77540ab890 con 0x7f77640d16e0 2024-04-25T14:04:36.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.353+0000 7f7769a00640 1 -- 172.21.15.86:0/2019118243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77640d16e0 msgr2=0x7f77640d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:36.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.353+0000 7f7769a00640 1 --2- 172.21.15.86:0/2019118243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77640d16e0 0x7f77640d1af0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7754009870 tx=0x7f775401c560 comp rx=0 tx=0).stop 2024-04-25T14:04:36.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.353+0000 7f7769a00640 1 -- 172.21.15.86:0/2019118243 shutdown_connections 2024-04-25T14:04:36.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.353+0000 7f7769a00640 1 --2- 172.21.15.86:0/2019118243 >> v2:172.21.15.155:3300/0 conn(0x7f77640730b0 0x7f77640734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.353+0000 7f7769a00640 1 --2- 172.21.15.86:0/2019118243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77640d16e0 0x7f77640d1af0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.353+0000 7f7769a00640 1 -- 172.21.15.86:0/2019118243 >> 172.21.15.86:0/2019118243 conn(0x7f77640b4650 msgr2=0x7f77640b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:36.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.353+0000 7f7769a00640 1 -- 172.21.15.86:0/2019118243 shutdown_connections 2024-04-25T14:04:36.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.353+0000 7f7769a00640 1 -- 172.21.15.86:0/2019118243 wait complete. 2024-04-25T14:04:36.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.357+0000 7f7769a00640 1 Processor -- start 2024-04-25T14:04:36.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.357+0000 7f7769a00640 1 -- start start 2024-04-25T14:04:36.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.357+0000 7f7769a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77640730b0 0x7f77640cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:36.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.357+0000 7f7769a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77640cfca0 0x7f77640cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:36.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.357+0000 7f7769a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77640d16e0 0x7f776415d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:36.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.357+0000 7f7769a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7764169a70 con 0x7f77640d16e0 2024-04-25T14:04:36.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.357+0000 7f7769a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f77641698f0 con 0x7f77640cfca0 2024-04-25T14:04:36.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.357+0000 7f7769a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7764169bf0 con 0x7f77640730b0 2024-04-25T14:04:36.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.357+0000 7f7763400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77640d16e0 0x7f776415d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:36.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.357+0000 7f7763400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77640d16e0 0x7f776415d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45306/0 (socket says 172.21.15.86:45306) 2024-04-25T14:04:36.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.357+0000 7f7763400640 1 -- 172.21.15.86:0/564030434 learned_addr learned my addr 172.21.15.86:0/564030434 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:36.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.357+0000 7f7762a00640 1 --2- 172.21.15.86:0/564030434 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77640730b0 0x7f77640cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:36.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.357+0000 7f7762000640 1 --2- 172.21.15.86:0/564030434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77640cfca0 0x7f77640cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:36.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.357+0000 7f7763400640 1 -- 172.21.15.86:0/564030434 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77640730b0 msgr2=0x7f77640cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:36.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.357+0000 7f7763400640 1 --2- 172.21.15.86:0/564030434 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77640730b0 0x7f77640cbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.357+0000 7f7763400640 1 -- 172.21.15.86:0/564030434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77640cfca0 msgr2=0x7f77640cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:36.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.357+0000 7f7763400640 1 --2- 172.21.15.86:0/564030434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77640cfca0 0x7f77640cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.357+0000 7f7763400640 1 -- 172.21.15.86:0/564030434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77640c4600 con 0x7f77640d16e0 2024-04-25T14:04:36.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.357+0000 7f7763400640 1 --2- 172.21.15.86:0/564030434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77640d16e0 0x7f776415d500 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f775c00ef90 tx=0x7f775c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:36.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.357+0000 7f775aa00640 1 -- 172.21.15.86:0/564030434 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c018020 con 0x7f77640d16e0 2024-04-25T14:04:36.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.357+0000 7f775aa00640 1 -- 172.21.15.86:0/564030434 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f775c014480 con 0x7f77640d16e0 2024-04-25T14:04:36.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.357+0000 7f7769a00640 1 -- 172.21.15.86:0/564030434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77640cc610 con 0x7f77640d16e0 2024-04-25T14:04:36.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.357+0000 7f775aa00640 1 -- 172.21.15.86:0/564030434 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c014870 con 0x7f77640d16e0 2024-04-25T14:04:36.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.361+0000 7f7769a00640 1 -- 172.21.15.86:0/564030434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77640ccbd0 con 0x7f77640d16e0 2024-04-25T14:04:36.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.361+0000 7f7769a00640 1 -- 172.21.15.86:0/564030434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f776409f4b0 con 0x7f77640d16e0 2024-04-25T14:04:36.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.361+0000 7f775aa00640 1 -- 172.21.15.86:0/564030434 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f775c01b020 con 0x7f77640d16e0 2024-04-25T14:04:36.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.361+0000 7f775aa00640 1 --2- 172.21.15.86:0/564030434 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f774006f430 0x7f7740071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:36.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.361+0000 7f775aa00640 1 -- 172.21.15.86:0/564030434 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f775c0901d0 con 0x7f77640d16e0 2024-04-25T14:04:36.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.361+0000 7f7762a00640 1 --2- 172.21.15.86:0/564030434 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f774006f430 0x7f7740071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:36.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.365+0000 7f7762a00640 1 --2- 172.21.15.86:0/564030434 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f774006f430 0x7f7740071940 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f774c007fd0 tx=0x7f774c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:36.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.369+0000 7f775aa00640 1 -- 172.21.15.86:0/564030434 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f775c022030 con 0x7f77640d16e0 2024-04-25T14:04:36.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.525+0000 7f7769a00640 1 -- 172.21.15.86:0/564030434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7764069d00 con 0x7f77640d16e0 2024-04-25T14:04:36.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.525+0000 7f775aa00640 1 -- 172.21.15.86:0/564030434 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f775c010040 con 0x7f77640d16e0 2024-04-25T14:04:36.525 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:04:36.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.525+0000 7f7769a00640 1 -- 172.21.15.86:0/564030434 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f774006f430 msgr2=0x7f7740071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:36.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.525+0000 7f7769a00640 1 --2- 172.21.15.86:0/564030434 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f774006f430 0x7f7740071940 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f774c007fd0 tx=0x7f774c009040 comp rx=0 tx=0).stop 2024-04-25T14:04:36.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.525+0000 7f7769a00640 1 -- 172.21.15.86:0/564030434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77640d16e0 msgr2=0x7f776415d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:36.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.525+0000 7f7769a00640 1 --2- 172.21.15.86:0/564030434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77640d16e0 0x7f776415d500 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f775c00ef90 tx=0x7f775c00c490 comp rx=0 tx=0).stop 2024-04-25T14:04:36.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.525+0000 7f7769a00640 1 -- 172.21.15.86:0/564030434 shutdown_connections 2024-04-25T14:04:36.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.525+0000 7f7769a00640 1 --2- 172.21.15.86:0/564030434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77640d16e0 0x7f776415d500 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.525+0000 7f7769a00640 1 --2- 172.21.15.86:0/564030434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77640cfca0 0x7f77640cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.525+0000 7f7769a00640 1 --2- 172.21.15.86:0/564030434 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f774006f430 0x7f7740071940 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.525+0000 7f7769a00640 1 --2- 172.21.15.86:0/564030434 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77640730b0 0x7f77640cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.525+0000 7f7769a00640 1 -- 172.21.15.86:0/564030434 >> 172.21.15.86:0/564030434 conn(0x7f77640b4650 msgr2=0x7f776406e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:36.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.525+0000 7f7769a00640 1 -- 172.21.15.86:0/564030434 shutdown_connections 2024-04-25T14:04:36.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.525+0000 7f7769a00640 1 -- 172.21.15.86:0/564030434 wait complete. 2024-04-25T14:04:36.547 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:04:36.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.693+0000 7fa8ca200640 1 Processor -- start 2024-04-25T14:04:36.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.693+0000 7fa8ca200640 1 -- start start 2024-04-25T14:04:36.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.693+0000 7fa8ca200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8c40cfca0 0x7fa8c406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:36.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.693+0000 7fa8ca200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa8c40d16e0 0x7fa8c40d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:36.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.693+0000 7fa8ca200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa8c406c670 0x7fa8c406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:36.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.693+0000 7fa8ca200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa8c4070470 con 0x7fa8c40d16e0 2024-04-25T14:04:36.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.693+0000 7fa8ca200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa8c40d2030 con 0x7fa8c406c670 2024-04-25T14:04:36.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.693+0000 7fa8ca200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa8c4072bd0 con 0x7fa8c40cfca0 2024-04-25T14:04:36.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.693+0000 7fa8baa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa8c40d16e0 0x7fa8c40d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:36.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.693+0000 7fa8baa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa8c40d16e0 0x7fa8c40d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45312/0 (socket says 172.21.15.86:45312) 2024-04-25T14:04:36.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.693+0000 7fa8baa00640 1 -- 172.21.15.86:0/3191686492 learned_addr learned my addr 172.21.15.86:0/3191686492 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:36.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.693+0000 7fa8c3400640 1 --2- 172.21.15.86:0/3191686492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8c40cfca0 0x7fa8c406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:36.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.693+0000 7fa8c3e00640 1 --2- 172.21.15.86:0/3191686492 >> v2:172.21.15.155:3300/0 conn(0x7fa8c406c670 0x7fa8c406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:36.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.693+0000 7fa8baa00640 1 -- 172.21.15.86:0/3191686492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8c40cfca0 msgr2=0x7fa8c406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:36.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.693+0000 7fa8baa00640 1 --2- 172.21.15.86:0/3191686492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8c40cfca0 0x7fa8c406f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.693+0000 7fa8baa00640 1 -- 172.21.15.86:0/3191686492 >> v2:172.21.15.155:3300/0 conn(0x7fa8c406c670 msgr2=0x7fa8c406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:36.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.693+0000 7fa8baa00640 1 --2- 172.21.15.86:0/3191686492 >> v2:172.21.15.155:3300/0 conn(0x7fa8c406c670 0x7fa8c406ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.693+0000 7fa8baa00640 1 -- 172.21.15.86:0/3191686492 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c406f930 con 0x7fa8c40d16e0 2024-04-25T14:04:36.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.697+0000 7fa8baa00640 1 --2- 172.21.15.86:0/3191686492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8c40d16e0 0x7fa8c40d1af0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa8c406e060 tx=0x7fa8ac01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d7c13e8642938f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:36.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.697+0000 7fa8c2a00640 1 -- 172.21.15.86:0/3191686492 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac0ab5f0 con 0x7fa8c40d16e0 2024-04-25T14:04:36.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.697+0000 7fa8c2a00640 1 -- 172.21.15.86:0/3191686492 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8ac0b0070 con 0x7fa8c40d16e0 2024-04-25T14:04:36.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.697+0000 7fa8c2a00640 1 -- 172.21.15.86:0/3191686492 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac0ab9b0 con 0x7fa8c40d16e0 2024-04-25T14:04:36.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.697+0000 7fa8ca200640 1 -- 172.21.15.86:0/3191686492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8c40d16e0 msgr2=0x7fa8c40d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:36.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.697+0000 7fa8ca200640 1 --2- 172.21.15.86:0/3191686492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8c40d16e0 0x7fa8c40d1af0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa8c406e060 tx=0x7fa8ac01c520 comp rx=0 tx=0).stop 2024-04-25T14:04:36.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.697+0000 7fa8ca200640 1 -- 172.21.15.86:0/3191686492 shutdown_connections 2024-04-25T14:04:36.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.697+0000 7fa8ca200640 1 --2- 172.21.15.86:0/3191686492 >> v2:172.21.15.155:3300/0 conn(0x7fa8c406c670 0x7fa8c406ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.697+0000 7fa8ca200640 1 --2- 172.21.15.86:0/3191686492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8c40d16e0 0x7fa8c40d1af0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.697+0000 7fa8ca200640 1 --2- 172.21.15.86:0/3191686492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8c40cfca0 0x7fa8c406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.697+0000 7fa8ca200640 1 -- 172.21.15.86:0/3191686492 >> 172.21.15.86:0/3191686492 conn(0x7fa8c40b4650 msgr2=0x7fa8c40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:36.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.697+0000 7fa8ca200640 1 -- 172.21.15.86:0/3191686492 shutdown_connections 2024-04-25T14:04:36.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.697+0000 7fa8ca200640 1 -- 172.21.15.86:0/3191686492 wait complete. 2024-04-25T14:04:36.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.701+0000 7fa8ca200640 1 Processor -- start 2024-04-25T14:04:36.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.701+0000 7fa8ca200640 1 -- start start 2024-04-25T14:04:36.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.701+0000 7fa8ca200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8c406c670 0x7fa8c40b2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:36.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.701+0000 7fa8ca200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8c40cfca0 0x7fa8c41fce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:36.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.701+0000 7fa8ca200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8c40d16e0 0x7fa8c409bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:36.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.701+0000 7fa8ca200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c40738c0 con 0x7fa8c40cfca0 2024-04-25T14:04:36.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.701+0000 7fa8ca200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c406d3d0 con 0x7fa8c40d16e0 2024-04-25T14:04:36.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.701+0000 7fa8ca200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa8c4073a40 con 0x7fa8c406c670 2024-04-25T14:04:36.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.701+0000 7fa8baa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8c40cfca0 0x7fa8c41fce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:36.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.701+0000 7fa8baa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8c40cfca0 0x7fa8c41fce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45328/0 (socket says 172.21.15.86:45328) 2024-04-25T14:04:36.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.701+0000 7fa8baa00640 1 -- 172.21.15.86:0/3934134462 learned_addr learned my addr 172.21.15.86:0/3934134462 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:36.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.701+0000 7fa8c3400640 1 --2- 172.21.15.86:0/3934134462 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8c406c670 0x7fa8c40b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:36.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.701+0000 7fa8c3e00640 1 --2- 172.21.15.86:0/3934134462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8c40d16e0 0x7fa8c409bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:36.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.701+0000 7fa8baa00640 1 -- 172.21.15.86:0/3934134462 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8c406c670 msgr2=0x7fa8c40b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:36.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.701+0000 7fa8baa00640 1 --2- 172.21.15.86:0/3934134462 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8c406c670 0x7fa8c40b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.701+0000 7fa8baa00640 1 -- 172.21.15.86:0/3934134462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8c40d16e0 msgr2=0x7fa8c409bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:36.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.701+0000 7fa8baa00640 1 --2- 172.21.15.86:0/3934134462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8c40d16e0 0x7fa8c409bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.701+0000 7fa8baa00640 1 -- 172.21.15.86:0/3934134462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c41fd3d0 con 0x7fa8c40cfca0 2024-04-25T14:04:36.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.701+0000 7fa8baa00640 1 --2- 172.21.15.86:0/3934134462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8c40cfca0 0x7fa8c41fce90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac097ec0 tx=0x7fa8ac01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:36.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.705+0000 7fa8bbe00640 1 -- 172.21.15.86:0/3934134462 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac0be5f0 con 0x7fa8c40cfca0 2024-04-25T14:04:36.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.705+0000 7fa8bbe00640 1 -- 172.21.15.86:0/3934134462 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8ac0be790 con 0x7fa8c40cfca0 2024-04-25T14:04:36.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.705+0000 7fa8ca200640 1 -- 172.21.15.86:0/3934134462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c40cc780 con 0x7fa8c40cfca0 2024-04-25T14:04:36.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.705+0000 7fa8bbe00640 1 -- 172.21.15.86:0/3934134462 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac0beb80 con 0x7fa8c40cfca0 2024-04-25T14:04:36.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.705+0000 7fa8ca200640 1 -- 172.21.15.86:0/3934134462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c40ccd40 con 0x7fa8c40cfca0 2024-04-25T14:04:36.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.705+0000 7fa8ca200640 1 -- 172.21.15.86:0/3934134462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8c409f4b0 con 0x7fa8c40cfca0 2024-04-25T14:04:36.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.705+0000 7fa8bbe00640 1 -- 172.21.15.86:0/3934134462 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa8ac0b43f0 con 0x7fa8c40cfca0 2024-04-25T14:04:36.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.705+0000 7fa8bbe00640 1 --2- 172.21.15.86:0/3934134462 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa8a006f430 0x7fa8a0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:36.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.705+0000 7fa8bbe00640 1 -- 172.21.15.86:0/3934134462 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa8ac129d00 con 0x7fa8c40cfca0 2024-04-25T14:04:36.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.705+0000 7fa8c3400640 1 --2- 172.21.15.86:0/3934134462 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa8a006f430 0x7fa8a0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:36.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.709+0000 7fa8c3400640 1 --2- 172.21.15.86:0/3934134462 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa8a006f430 0x7fa8a0071940 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4070710 tx=0x7fa8b4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:36.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.709+0000 7fa8bbe00640 1 -- 172.21.15.86:0/3934134462 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa8ac0c4060 con 0x7fa8c40cfca0 2024-04-25T14:04:36.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.869+0000 7fa8ca200640 1 -- 172.21.15.86:0/3934134462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8c4069d00 con 0x7fa8c40cfca0 2024-04-25T14:04:36.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.869+0000 7fa8bbe00640 1 -- 172.21.15.86:0/3934134462 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18902 (secure 0 0 0) 0x7fa8ac016110 con 0x7fa8c40cfca0 2024-04-25T14:04:36.872 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:36.872 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":26,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:04:31.930803+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:03:49.151473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:03:49.802597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T14:03:53.492008+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:04:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.873+0000 7fa8ca200640 1 -- 172.21.15.86:0/3934134462 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa8a006f430 msgr2=0x7fa8a0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.873+0000 7fa8ca200640 1 --2- 172.21.15.86:0/3934134462 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa8a006f430 0x7fa8a0071940 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4070710 tx=0x7fa8b4008040 comp rx=0 tx=0).stop 2024-04-25T14:04:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.873+0000 7fa8ca200640 1 -- 172.21.15.86:0/3934134462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8c40cfca0 msgr2=0x7fa8c41fce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.873+0000 7fa8ca200640 1 --2- 172.21.15.86:0/3934134462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8c40cfca0 0x7fa8c41fce90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac097ec0 tx=0x7fa8ac01ad60 comp rx=0 tx=0).stop 2024-04-25T14:04:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.873+0000 7fa8ca200640 1 -- 172.21.15.86:0/3934134462 shutdown_connections 2024-04-25T14:04:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.873+0000 7fa8ca200640 1 --2- 172.21.15.86:0/3934134462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8c40d16e0 0x7fa8c409bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.873+0000 7fa8ca200640 1 --2- 172.21.15.86:0/3934134462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8c40cfca0 0x7fa8c41fce90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.873+0000 7fa8ca200640 1 --2- 172.21.15.86:0/3934134462 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa8a006f430 0x7fa8a0071940 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.873+0000 7fa8ca200640 1 --2- 172.21.15.86:0/3934134462 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8c406c670 0x7fa8c40b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.873+0000 7fa8ca200640 1 -- 172.21.15.86:0/3934134462 >> 172.21.15.86:0/3934134462 conn(0x7fa8c40b4650 msgr2=0x7fa8c415be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.873+0000 7fa8ca200640 1 -- 172.21.15.86:0/3934134462 shutdown_connections 2024-04-25T14:04:36.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:36.873+0000 7fa8ca200640 1 -- 172.21.15.86:0/3934134462 wait complete. 2024-04-25T14:04:36.895 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:03:49.151473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:03:49.802597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T14:03:53.492008+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T14:04:36.895 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:04:37.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.045+0000 7f2652400640 1 Processor -- start 2024-04-25T14:04:37.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.045+0000 7f2652400640 1 -- start start 2024-04-25T14:04:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.045+0000 7f2652400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f264c070330 0x7f264c0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.045+0000 7f2652400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f264c0d1c20 0x7f264c06ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.045+0000 7f2652400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f264c0d2030 con 0x7f264c06f270 2024-04-25T14:04:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.045+0000 7f2652400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f264c15e5c0 con 0x7f264c0d1c20 2024-04-25T14:04:37.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.045+0000 7f2652400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f264c15e8a0 con 0x7f264c070330 2024-04-25T14:04:37.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.045+0000 7f2642a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f264c0d1c20 0x7f264c06ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:37.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.045+0000 7f2642a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f264c0d1c20 0x7f264c06ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45342/0 (socket says 172.21.15.86:45342) 2024-04-25T14:04:37.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.045+0000 7f2642a00640 1 -- 172.21.15.86:0/3602937995 learned_addr learned my addr 172.21.15.86:0/3602937995 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:37.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.045+0000 7f264b400640 1 --2- 172.21.15.86:0/3602937995 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f264c070330 0x7f264c0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:37.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.045+0000 7f2642a00640 1 -- 172.21.15.86:0/3602937995 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f264c070330 msgr2=0x7f264c0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:37.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.045+0000 7f2642a00640 1 --2- 172.21.15.86:0/3602937995 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f264c070330 0x7f264c0d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.045+0000 7f2642a00640 1 -- 172.21.15.86:0/3602937995 >> v1:172.21.15.155:6789/0 conn(0x7f264c06f270 legacy=0x7f264c06f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:37.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.045+0000 7f2642a00640 1 -- 172.21.15.86:0/3602937995 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264c15eb80 con 0x7f264c0d1c20 2024-04-25T14:04:37.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.049+0000 7f2642a00640 1 --2- 172.21.15.86:0/3602937995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f264c0d1c20 0x7f264c06ed30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f26340098a0 tx=0x7f263401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426ce4cdec9096a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:37.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.049+0000 7f264aa00640 1 -- 172.21.15.86:0/3602937995 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26340b0070 con 0x7f264c0d1c20 2024-04-25T14:04:37.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.049+0000 7f264aa00640 1 -- 172.21.15.86:0/3602937995 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26340ab600 con 0x7f264c0d1c20 2024-04-25T14:04:37.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.049+0000 7f264aa00640 1 -- 172.21.15.86:0/3602937995 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26340ab9d0 con 0x7f264c0d1c20 2024-04-25T14:04:37.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.049+0000 7f2652400640 1 -- 172.21.15.86:0/3602937995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f264c0d1c20 msgr2=0x7f264c06ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:37.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.049+0000 7f2652400640 1 --2- 172.21.15.86:0/3602937995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f264c0d1c20 0x7f264c06ed30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f26340098a0 tx=0x7f263401c4b0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.049+0000 7f2652400640 1 -- 172.21.15.86:0/3602937995 shutdown_connections 2024-04-25T14:04:37.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.049+0000 7f2652400640 1 --2- 172.21.15.86:0/3602937995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f264c0d1c20 0x7f264c06ed30 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.049+0000 7f2652400640 1 --2- 172.21.15.86:0/3602937995 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f264c070330 0x7f264c0d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.049+0000 7f2652400640 1 -- 172.21.15.86:0/3602937995 >> 172.21.15.86:0/3602937995 conn(0x7f264c0b4650 msgr2=0x7f264c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:37.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.049+0000 7f2652400640 1 -- 172.21.15.86:0/3602937995 shutdown_connections 2024-04-25T14:04:37.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.049+0000 7f2652400640 1 -- 172.21.15.86:0/3602937995 wait complete. 2024-04-25T14:04:37.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.053+0000 7f2652400640 1 Processor -- start 2024-04-25T14:04:37.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.053+0000 7f2652400640 1 -- start start 2024-04-25T14:04:37.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.053+0000 7f2652400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f264c06f270 0x7f264c0d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:37.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.053+0000 7f2652400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f264c070330 0x7f264c0ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:37.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.053+0000 7f2652400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f264c0d1c20 0x7f264c0c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:37.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.053+0000 7f2652400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f264c0708c0 con 0x7f264c070330 2024-04-25T14:04:37.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.053+0000 7f2652400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f264c070740 con 0x7f264c0d1c20 2024-04-25T14:04:37.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.053+0000 7f2652400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f264c16afc0 con 0x7f264c06f270 2024-04-25T14:04:37.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.053+0000 7f2642a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f264c070330 0x7f264c0ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:37.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.053+0000 7f2642a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f264c070330 0x7f264c0ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45356/0 (socket says 172.21.15.86:45356) 2024-04-25T14:04:37.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.053+0000 7f2642a00640 1 -- 172.21.15.86:0/2253765561 learned_addr learned my addr 172.21.15.86:0/2253765561 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:37.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.053+0000 7f264b400640 1 --2- 172.21.15.86:0/2253765561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f264c06f270 0x7f264c0d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:37.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.053+0000 7f264be00640 1 --2- 172.21.15.86:0/2253765561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f264c0d1c20 0x7f264c0c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:37.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.053+0000 7f2642a00640 1 -- 172.21.15.86:0/2253765561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f264c06f270 msgr2=0x7f264c0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:37.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.053+0000 7f2642a00640 1 --2- 172.21.15.86:0/2253765561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f264c06f270 0x7f264c0d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.053+0000 7f2642a00640 1 -- 172.21.15.86:0/2253765561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f264c0d1c20 msgr2=0x7f264c0c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:37.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.053+0000 7f2642a00640 1 --2- 172.21.15.86:0/2253765561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f264c0d1c20 0x7f264c0c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.053+0000 7f2642a00640 1 -- 172.21.15.86:0/2253765561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264c072a20 con 0x7f264c070330 2024-04-25T14:04:37.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.053+0000 7f264b400640 1 --2- 172.21.15.86:0/2253765561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f264c06f270 0x7f264c0d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:37.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.057+0000 7f2642a00640 1 --2- 172.21.15.86:0/2253765561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f264c070330 0x7f264c0ceeb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f263401ab60 tx=0x7f2634019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:37.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.057+0000 7f2643e00640 1 -- 172.21.15.86:0/2253765561 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26340b0040 con 0x7f264c070330 2024-04-25T14:04:37.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.057+0000 7f2643e00640 1 -- 172.21.15.86:0/2253765561 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26340ab400 con 0x7f264c070330 2024-04-25T14:04:37.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.057+0000 7f2643e00640 1 -- 172.21.15.86:0/2253765561 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26340b6640 con 0x7f264c070330 2024-04-25T14:04:37.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.057+0000 7f2652400640 1 -- 172.21.15.86:0/2253765561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f264c072d00 con 0x7f264c070330 2024-04-25T14:04:37.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.057+0000 7f2652400640 1 -- 172.21.15.86:0/2253765561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f264c0ae270 con 0x7f264c070330 2024-04-25T14:04:37.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.057+0000 7f2652400640 1 -- 172.21.15.86:0/2253765561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f264c069d00 con 0x7f264c070330 2024-04-25T14:04:37.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.057+0000 7f2643e00640 1 -- 172.21.15.86:0/2253765561 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f26340be020 con 0x7f264c070330 2024-04-25T14:04:37.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.057+0000 7f2643e00640 1 --2- 172.21.15.86:0/2253765561 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f262806f430 0x7f2628071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:37.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.057+0000 7f2643e00640 1 -- 172.21.15.86:0/2253765561 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f2634129200 con 0x7f264c070330 2024-04-25T14:04:37.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.061+0000 7f264b400640 1 --2- 172.21.15.86:0/2253765561 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f262806f430 0x7f2628071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:37.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.061+0000 7f264b400640 1 --2- 172.21.15.86:0/2253765561 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f262806f430 0x7f2628071940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f264c072200 tx=0x7f263c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:37.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.065+0000 7f2643e00640 1 -- 172.21.15.86:0/2253765561 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2634016130 con 0x7f264c070330 2024-04-25T14:04:37.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.221+0000 7f2652400640 1 -- 172.21.15.86:0/2253765561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f264c0b0050 con 0x7f264c070330 2024-04-25T14:04:37.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.221+0000 7f2643e00640 1 -- 172.21.15.86:0/2253765561 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f26340f5cf0 con 0x7f264c070330 2024-04-25T14:04:37.222 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:04:37.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.221+0000 7f2652400640 1 -- 172.21.15.86:0/2253765561 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f262806f430 msgr2=0x7f2628071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:37.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.221+0000 7f2652400640 1 --2- 172.21.15.86:0/2253765561 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f262806f430 0x7f2628071940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f264c072200 tx=0x7f263c008040 comp rx=0 tx=0).stop 2024-04-25T14:04:37.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.221+0000 7f2652400640 1 -- 172.21.15.86:0/2253765561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f264c070330 msgr2=0x7f264c0ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:37.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.221+0000 7f2652400640 1 --2- 172.21.15.86:0/2253765561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f264c070330 0x7f264c0ceeb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f263401ab60 tx=0x7f2634019250 comp rx=0 tx=0).stop 2024-04-25T14:04:37.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.221+0000 7f2652400640 1 -- 172.21.15.86:0/2253765561 shutdown_connections 2024-04-25T14:04:37.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.221+0000 7f2652400640 1 --2- 172.21.15.86:0/2253765561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f264c0d1c20 0x7f264c0c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.221+0000 7f2652400640 1 --2- 172.21.15.86:0/2253765561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f264c070330 0x7f264c0ceeb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.221+0000 7f2652400640 1 --2- 172.21.15.86:0/2253765561 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f262806f430 0x7f2628071940 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.221+0000 7f2652400640 1 --2- 172.21.15.86:0/2253765561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f264c06f270 0x7f264c0d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.221+0000 7f2652400640 1 -- 172.21.15.86:0/2253765561 >> 172.21.15.86:0/2253765561 conn(0x7f264c0b4650 msgr2=0x7f264c15d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:37.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.221+0000 7f2652400640 1 -- 172.21.15.86:0/2253765561 shutdown_connections 2024-04-25T14:04:37.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.221+0000 7f2652400640 1 -- 172.21.15.86:0/2253765561 wait complete. 2024-04-25T14:04:37.243 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:04:37.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.389+0000 7ff154200640 1 Processor -- start 2024-04-25T14:04:37.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.389+0000 7ff154200640 1 -- start start 2024-04-25T14:04:37.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.393+0000 7ff154200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff14c15faa0 0x7ff14c06d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:37.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.393+0000 7ff154200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff14c0aeea0 0x7ff14c0af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:37.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.393+0000 7ff154200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff14c06c670 con 0x7ff14c0b59e0 2024-04-25T14:04:37.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.393+0000 7ff154200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff14c06d560 con 0x7ff14c0aeea0 2024-04-25T14:04:37.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.393+0000 7ff154200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff14c06c950 con 0x7ff14c15faa0 2024-04-25T14:04:37.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.393+0000 7ff151800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff14c15faa0 0x7ff14c06d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:37.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.393+0000 7ff152200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff14c0aeea0 0x7ff14c0af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:37.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.393+0000 7ff151800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff14c15faa0 0x7ff14c06d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:50954/0 (socket says 172.21.15.86:50954) 2024-04-25T14:04:37.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.393+0000 7ff150e00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7ff14c0b59e0 0x7ff14c0b5df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:38170/0 (socket says 172.21.15.86:38170) 2024-04-25T14:04:37.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.393+0000 7ff151800640 1 -- 172.21.15.86:0/1875880388 learned_addr learned my addr 172.21.15.86:0/1875880388 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:37.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.393+0000 7ff151800640 1 -- 172.21.15.86:0/1875880388 >> v2:172.21.15.155:3300/0 conn(0x7ff14c0aeea0 msgr2=0x7ff14c0af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:37.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.393+0000 7ff151800640 1 --2- 172.21.15.86:0/1875880388 >> v2:172.21.15.155:3300/0 conn(0x7ff14c0aeea0 0x7ff14c0af2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.393+0000 7ff151800640 1 -- 172.21.15.86:0/1875880388 >> v1:172.21.15.155:6789/0 conn(0x7ff14c0b59e0 legacy=0x7ff14c0b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:37.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.393+0000 7ff151800640 1 -- 172.21.15.86:0/1875880388 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff14c06cc30 con 0x7ff14c15faa0 2024-04-25T14:04:37.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.397+0000 7ff151800640 1 --2- 172.21.15.86:0/1875880388 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff14c15faa0 0x7ff14c06d9f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff140009e50 tx=0x7ff14001ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f7715ea48a52047 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:37.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.397+0000 7ff14be00640 1 -- 172.21.15.86:0/1875880388 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1400a5900 con 0x7ff14c15faa0 2024-04-25T14:04:37.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.397+0000 7ff14be00640 1 -- 172.21.15.86:0/1875880388 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1400a5aa0 con 0x7ff14c15faa0 2024-04-25T14:04:37.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.397+0000 7ff14be00640 1 -- 172.21.15.86:0/1875880388 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1400ad3f0 con 0x7ff14c15faa0 2024-04-25T14:04:37.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.397+0000 7ff154200640 1 -- 172.21.15.86:0/1875880388 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff14c15faa0 msgr2=0x7ff14c06d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:37.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.397+0000 7ff154200640 1 --2- 172.21.15.86:0/1875880388 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff14c15faa0 0x7ff14c06d9f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff140009e50 tx=0x7ff14001ca30 comp rx=0 tx=0).stop 2024-04-25T14:04:37.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.397+0000 7ff154200640 1 -- 172.21.15.86:0/1875880388 shutdown_connections 2024-04-25T14:04:37.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.397+0000 7ff154200640 1 --2- 172.21.15.86:0/1875880388 >> v2:172.21.15.155:3300/0 conn(0x7ff14c0aeea0 0x7ff14c0af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.397+0000 7ff154200640 1 --2- 172.21.15.86:0/1875880388 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff14c15faa0 0x7ff14c06d9f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.397+0000 7ff154200640 1 -- 172.21.15.86:0/1875880388 >> 172.21.15.86:0/1875880388 conn(0x7ff14c0d2230 msgr2=0x7ff14c0d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:37.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.397+0000 7ff154200640 1 -- 172.21.15.86:0/1875880388 shutdown_connections 2024-04-25T14:04:37.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.397+0000 7ff154200640 1 -- 172.21.15.86:0/1875880388 wait complete. 2024-04-25T14:04:37.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.401+0000 7ff154200640 1 Processor -- start 2024-04-25T14:04:37.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.401+0000 7ff154200640 1 -- start start 2024-04-25T14:04:37.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.401+0000 7ff154200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff14c0aeea0 0x7ff14c14f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:37.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.401+0000 7ff154200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff14c0b59e0 0x7ff14c0b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:37.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.401+0000 7ff154200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff14c15faa0 0x7ff14c0b1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:37.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.401+0000 7ff154200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff14c160030 con 0x7ff14c15faa0 2024-04-25T14:04:37.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.401+0000 7ff154200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff14c15feb0 con 0x7ff14c0aeea0 2024-04-25T14:04:37.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.401+0000 7ff154200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff14c070f30 con 0x7ff14c0b59e0 2024-04-25T14:04:37.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.401+0000 7ff152200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff14c15faa0 0x7ff14c0b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:37.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.401+0000 7ff152200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff14c15faa0 0x7ff14c0b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45364/0 (socket says 172.21.15.86:45364) 2024-04-25T14:04:37.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.401+0000 7ff152200640 1 -- 172.21.15.86:0/3093347144 learned_addr learned my addr 172.21.15.86:0/3093347144 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:37.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.401+0000 7ff151800640 1 --2- 172.21.15.86:0/3093347144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff14c0aeea0 0x7ff14c14f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:37.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.401+0000 7ff150e00640 1 --2- 172.21.15.86:0/3093347144 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff14c0b59e0 0x7ff14c0b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:37.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.401+0000 7ff152200640 1 -- 172.21.15.86:0/3093347144 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff14c0b59e0 msgr2=0x7ff14c0b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:37.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.401+0000 7ff152200640 1 --2- 172.21.15.86:0/3093347144 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff14c0b59e0 0x7ff14c0b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.401+0000 7ff152200640 1 -- 172.21.15.86:0/3093347144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff14c0aeea0 msgr2=0x7ff14c14f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:37.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.401+0000 7ff152200640 1 --2- 172.21.15.86:0/3093347144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff14c0aeea0 0x7ff14c14f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.401+0000 7ff152200640 1 -- 172.21.15.86:0/3093347144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff14c0b2140 con 0x7ff14c15faa0 2024-04-25T14:04:37.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.401+0000 7ff150e00640 1 --2- 172.21.15.86:0/3093347144 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff14c0b59e0 0x7ff14c0b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:37.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.401+0000 7ff152200640 1 --2- 172.21.15.86:0/3093347144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff14c15faa0 0x7ff14c0b1820 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff14400e820 tx=0x7ff14400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:37.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.401+0000 7ff149600640 1 -- 172.21.15.86:0/3093347144 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff144017070 con 0x7ff14c15faa0 2024-04-25T14:04:37.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.401+0000 7ff149600640 1 -- 172.21.15.86:0/3093347144 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff14400c940 con 0x7ff14c15faa0 2024-04-25T14:04:37.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.401+0000 7ff149600640 1 -- 172.21.15.86:0/3093347144 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14400cd10 con 0x7ff14c15faa0 2024-04-25T14:04:37.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.401+0000 7ff154200640 1 -- 172.21.15.86:0/3093347144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff14c1552f0 con 0x7ff14c15faa0 2024-04-25T14:04:37.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.401+0000 7ff154200640 1 -- 172.21.15.86:0/3093347144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff14c155da0 con 0x7ff14c15faa0 2024-04-25T14:04:37.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.405+0000 7ff154200640 1 -- 172.21.15.86:0/3093347144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff14c074cf0 con 0x7ff14c15faa0 2024-04-25T14:04:37.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.405+0000 7ff149600640 1 -- 172.21.15.86:0/3093347144 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff14401a070 con 0x7ff14c15faa0 2024-04-25T14:04:37.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.405+0000 7ff149600640 1 --2- 172.21.15.86:0/3093347144 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff12c06f400 0x7ff12c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:37.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.405+0000 7ff149600640 1 -- 172.21.15.86:0/3093347144 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff14408f600 con 0x7ff14c15faa0 2024-04-25T14:04:37.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.405+0000 7ff151800640 1 --2- 172.21.15.86:0/3093347144 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff12c06f400 0x7ff12c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:37.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.405+0000 7ff151800640 1 --2- 172.21.15.86:0/3093347144 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff12c06f400 0x7ff12c071910 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff1400ae000 tx=0x7ff1400a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:37.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.409+0000 7ff149600640 1 -- 172.21.15.86:0/3093347144 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff14405c050 con 0x7ff14c15faa0 2024-04-25T14:04:37.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.565+0000 7ff154200640 1 -- 172.21.15.86:0/3093347144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff14c0b55b0 con 0x7ff14c15faa0 2024-04-25T14:04:37.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.565+0000 7ff149600640 1 -- 172.21.15.86:0/3093347144 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7ff14405b950 con 0x7ff14c15faa0 2024-04-25T14:04:37.568 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:04:37.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.569+0000 7ff154200640 1 -- 172.21.15.86:0/3093347144 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff12c06f400 msgr2=0x7ff12c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:37.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.569+0000 7ff154200640 1 --2- 172.21.15.86:0/3093347144 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff12c06f400 0x7ff12c071910 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff1400ae000 tx=0x7ff1400a7070 comp rx=0 tx=0).stop 2024-04-25T14:04:37.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.569+0000 7ff154200640 1 -- 172.21.15.86:0/3093347144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff14c15faa0 msgr2=0x7ff14c0b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:37.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.569+0000 7ff154200640 1 --2- 172.21.15.86:0/3093347144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff14c15faa0 0x7ff14c0b1820 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff14400e820 tx=0x7ff14400ece0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.569+0000 7ff154200640 1 -- 172.21.15.86:0/3093347144 shutdown_connections 2024-04-25T14:04:37.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.569+0000 7ff154200640 1 --2- 172.21.15.86:0/3093347144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff14c15faa0 0x7ff14c0b1820 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.569+0000 7ff154200640 1 --2- 172.21.15.86:0/3093347144 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff12c06f400 0x7ff12c071910 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.569+0000 7ff154200640 1 --2- 172.21.15.86:0/3093347144 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff14c0b59e0 0x7ff14c0b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.569+0000 7ff154200640 1 --2- 172.21.15.86:0/3093347144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff14c0aeea0 0x7ff14c14f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.569+0000 7ff154200640 1 -- 172.21.15.86:0/3093347144 >> 172.21.15.86:0/3093347144 conn(0x7ff14c0d2230 msgr2=0x7ff14c15c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:37.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.569+0000 7ff154200640 1 -- 172.21.15.86:0/3093347144 shutdown_connections 2024-04-25T14:04:37.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.569+0000 7ff154200640 1 -- 172.21.15.86:0/3093347144 wait complete. 2024-04-25T14:04:37.589 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:04:37.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.737+0000 7fdbeae00640 1 Processor -- start 2024-04-25T14:04:37.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.737+0000 7fdbeae00640 1 -- start start 2024-04-25T14:04:37.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.737+0000 7fdbeae00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbe40cfca0 0x7fdbe406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:37.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbeae00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbe4072bd0 con 0x7fdbe40d16e0 2024-04-25T14:04:37.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbeae00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbe406f930 con 0x7fdbe40730b0 2024-04-25T14:04:37.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbeae00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdbe4070470 con 0x7fdbe40cfca0 2024-04-25T14:04:37.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbdb400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fdbe40d16e0 0x7fdbe40d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60706/0 (socket says 172.21.15.86:60706) 2024-04-25T14:04:37.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbdb400640 1 -- 172.21.15.86:0/1659770991 learned_addr learned my addr 172.21.15.86:0/1659770991 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbe3e00640 1 --2- 172.21.15.86:0/1659770991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbe40cfca0 0x7fdbe406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbe3400640 1 -- 172.21.15.86:0/1659770991 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1063909125 0 0) 0x7fdbe4072bd0 con 0x7fdbe40d16e0 2024-04-25T14:04:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbe3400640 1 -- 172.21.15.86:0/1659770991 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdbc8003530 con 0x7fdbe40d16e0 2024-04-25T14:04:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbe3400640 1 -- 172.21.15.86:0/1659770991 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3991682240 0 0) 0x7fdbc8003530 con 0x7fdbe40d16e0 2024-04-25T14:04:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbe3400640 1 -- 172.21.15.86:0/1659770991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbe40cfca0 msgr2=0x7fdbe406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbe3400640 1 --2- 172.21.15.86:0/1659770991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbe40cfca0 0x7fdbe406f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbe3400640 1 -- 172.21.15.86:0/1659770991 >> v1:172.21.15.155:6789/0 conn(0x7fdbe40730b0 legacy=0x7fdbe40734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbe3400640 1 -- 172.21.15.86:0/1659770991 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe406fbd0 con 0x7fdbe40d16e0 2024-04-25T14:04:37.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbe3400640 1 -- 172.21.15.86:0/1659770991 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fdbcc001680 con 0x7fdbe40d16e0 2024-04-25T14:04:37.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbe3400640 1 -- 172.21.15.86:0/1659770991 >> v1:172.21.15.86:6789/0 conn(0x7fdbe40d16e0 legacy=0x7fdbe40d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:37.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbe3400640 1 --2- 172.21.15.86:0/1659770991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbe40cfca0 0x7fdbc8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:37.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbe3400640 1 --2- 172.21.15.86:0/1659770991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbc8003fc0 0x7fdbc8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:37.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbe3400640 1 --2- 172.21.15.86:0/1659770991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbc8004d50 0x7fdbc8005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:37.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbe3400640 1 -- 172.21.15.86:0/1659770991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe4070470 con 0x7fdbc8004d50 2024-04-25T14:04:37.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbe3400640 1 -- 172.21.15.86:0/1659770991 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc800bb60 con 0x7fdbc8003fc0 2024-04-25T14:04:37.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbe3400640 1 -- 172.21.15.86:0/1659770991 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdbc800bd60 con 0x7fdbe40cfca0 2024-04-25T14:04:37.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbe8e00640 1 --2- 172.21.15.86:0/1659770991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbc8004d50 0x7fdbc8005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:37.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbe3e00640 1 --2- 172.21.15.86:0/1659770991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbe40cfca0 0x7fdbc8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:37.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbdb400640 1 --2- 172.21.15.86:0/1659770991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbc8003fc0 0x7fdbc8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:37.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbe8e00640 1 -- 172.21.15.86:0/1659770991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbe40cfca0 msgr2=0x7fdbc8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:37.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbe8e00640 1 --2- 172.21.15.86:0/1659770991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbe40cfca0 0x7fdbc8003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbe8e00640 1 -- 172.21.15.86:0/1659770991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbc8003fc0 msgr2=0x7fdbc8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:37.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbe8e00640 1 --2- 172.21.15.86:0/1659770991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbc8003fc0 0x7fdbc8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.741+0000 7fdbe8e00640 1 -- 172.21.15.86:0/1659770991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc8003530 con 0x7fdbc8004d50 2024-04-25T14:04:37.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.745+0000 7fdbe8e00640 1 --2- 172.21.15.86:0/1659770991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbc8004d50 0x7fdbc8005160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdbe40b4220 tx=0x7fdbdc01c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4affdb799b45f126 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:37.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.745+0000 7fdbe3400640 1 -- 172.21.15.86:0/1659770991 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc001c50 con 0x7fdbc8004d50 2024-04-25T14:04:37.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.745+0000 7fdbe3400640 1 -- 172.21.15.86:0/1659770991 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbdc003ba0 con 0x7fdbc8004d50 2024-04-25T14:04:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.745+0000 7fdbe3400640 1 -- 172.21.15.86:0/1659770991 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc005080 con 0x7fdbc8004d50 2024-04-25T14:04:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.745+0000 7fdbeae00640 1 -- 172.21.15.86:0/1659770991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbc8004d50 msgr2=0x7fdbc8005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.745+0000 7fdbeae00640 1 --2- 172.21.15.86:0/1659770991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbc8004d50 0x7fdbc8005160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdbe40b4220 tx=0x7fdbdc01c880 comp rx=0 tx=0).stop 2024-04-25T14:04:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.745+0000 7fdbe8e00640 1 -- 172.21.15.86:0/1659770991 reap_dead start 2024-04-25T14:04:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.745+0000 7fdbeae00640 1 -- 172.21.15.86:0/1659770991 shutdown_connections 2024-04-25T14:04:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.745+0000 7fdbeae00640 1 -- 172.21.15.86:0/1659770991 >> 172.21.15.86:0/1659770991 conn(0x7fdbe40b4650 msgr2=0x7fdbe40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.745+0000 7fdbeae00640 1 -- 172.21.15.86:0/1659770991 shutdown_connections 2024-04-25T14:04:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.745+0000 7fdbeae00640 1 -- 172.21.15.86:0/1659770991 wait complete. 2024-04-25T14:04:37.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.749+0000 7fdbeae00640 1 Processor -- start 2024-04-25T14:04:37.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.749+0000 7fdbeae00640 1 -- start start 2024-04-25T14:04:37.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.749+0000 7fdbeae00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbc8004d50 0x7fdbe409d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:37.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.749+0000 7fdbeae00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbe409c300 0x7fdbe415be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:37.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.749+0000 7fdbeae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbe41e62b0 0x7fdbe41e66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:37.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.749+0000 7fdbeae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe406f3f0 con 0x7fdbe41e62b0 2024-04-25T14:04:37.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.749+0000 7fdbeae00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe4070190 con 0x7fdbc8004d50 2024-04-25T14:04:37.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.749+0000 7fdbeae00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdbe406f570 con 0x7fdbe409c300 2024-04-25T14:04:37.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.749+0000 7fdbe8e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbe41e62b0 0x7fdbe41e66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:37.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.749+0000 7fdbe8e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbe41e62b0 0x7fdbe41e66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45374/0 (socket says 172.21.15.86:45374) 2024-04-25T14:04:37.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.749+0000 7fdbe8e00640 1 -- 172.21.15.86:0/2476450354 learned_addr learned my addr 172.21.15.86:0/2476450354 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:37.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.749+0000 7fdbe3e00640 1 --2- 172.21.15.86:0/2476450354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbc8004d50 0x7fdbe409d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:37.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.749+0000 7fdbdb400640 1 --2- 172.21.15.86:0/2476450354 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbe409c300 0x7fdbe415be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:37.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.749+0000 7fdbe8e00640 1 -- 172.21.15.86:0/2476450354 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbe409c300 msgr2=0x7fdbe415be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:37.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.749+0000 7fdbe8e00640 1 --2- 172.21.15.86:0/2476450354 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbe409c300 0x7fdbe415be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.749+0000 7fdbe8e00640 1 -- 172.21.15.86:0/2476450354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbc8004d50 msgr2=0x7fdbe409d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:37.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.749+0000 7fdbe8e00640 1 --2- 172.21.15.86:0/2476450354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbc8004d50 0x7fdbe409d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.749+0000 7fdbe8e00640 1 -- 172.21.15.86:0/2476450354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe41e6c00 con 0x7fdbe41e62b0 2024-04-25T14:04:37.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.749+0000 7fdbdb400640 1 --2- 172.21.15.86:0/2476450354 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbe409c300 0x7fdbe415be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:37.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.749+0000 7fdbe8e00640 1 --2- 172.21.15.86:0/2476450354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbe41e62b0 0x7fdbe41e66c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc01d120 tx=0x7fdbc8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:37.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.749+0000 7fdbe0c00640 1 -- 172.21.15.86:0/2476450354 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc0ba070 con 0x7fdbe41e62b0 2024-04-25T14:04:37.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.749+0000 7fdbe0c00640 1 -- 172.21.15.86:0/2476450354 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbdc001c30 con 0x7fdbe41e62b0 2024-04-25T14:04:37.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.749+0000 7fdbeae00640 1 -- 172.21.15.86:0/2476450354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe40ae260 con 0x7fdbe41e62b0 2024-04-25T14:04:37.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.749+0000 7fdbe0c00640 1 -- 172.21.15.86:0/2476450354 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc007e00 con 0x7fdbe41e62b0 2024-04-25T14:04:37.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.749+0000 7fdbeae00640 1 -- 172.21.15.86:0/2476450354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe409dd90 con 0x7fdbe41e62b0 2024-04-25T14:04:37.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.753+0000 7fdbeae00640 1 -- 172.21.15.86:0/2476450354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbe409d800 con 0x7fdbe41e62b0 2024-04-25T14:04:37.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.753+0000 7fdbe0c00640 1 -- 172.21.15.86:0/2476450354 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdbdc0bd070 con 0x7fdbe41e62b0 2024-04-25T14:04:37.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.753+0000 7fdbe0c00640 1 --2- 172.21.15.86:0/2476450354 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdbc006f400 0x7fdbc0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:37.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.753+0000 7fdbe0c00640 1 -- 172.21.15.86:0/2476450354 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fdbdc129860 con 0x7fdbe41e62b0 2024-04-25T14:04:37.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.753+0000 7fdbe3e00640 1 --2- 172.21.15.86:0/2476450354 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdbc006f400 0x7fdbc0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:37.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.753+0000 7fdbe3e00640 1 --2- 172.21.15.86:0/2476450354 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdbc006f400 0x7fdbc0071910 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4070710 tx=0x7fdbd4009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:37.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.757+0000 7fdbe0c00640 1 -- 172.21.15.86:0/2476450354 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdbdc0c4020 con 0x7fdbe41e62b0 2024-04-25T14:04:37.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.913+0000 7fdbeae00640 1 -- 172.21.15.86:0/2476450354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdbe4069d00 con 0x7fdbe41e62b0 2024-04-25T14:04:37.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.913+0000 7fdbe0c00640 1 -- 172.21.15.86:0/2476450354 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fdbe41e6c00 con 0x7fdbe41e62b0 2024-04-25T14:04:37.917 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:04:37.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.917+0000 7fdbeae00640 1 -- 172.21.15.86:0/2476450354 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdbc006f400 msgr2=0x7fdbc0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:37.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.917+0000 7fdbeae00640 1 --2- 172.21.15.86:0/2476450354 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdbc006f400 0x7fdbc0071910 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4070710 tx=0x7fdbd4009190 comp rx=0 tx=0).stop 2024-04-25T14:04:37.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.917+0000 7fdbeae00640 1 -- 172.21.15.86:0/2476450354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbe41e62b0 msgr2=0x7fdbe41e66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:37.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.917+0000 7fdbeae00640 1 --2- 172.21.15.86:0/2476450354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbe41e62b0 0x7fdbe41e66c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc01d120 tx=0x7fdbc8005840 comp rx=0 tx=0).stop 2024-04-25T14:04:37.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.917+0000 7fdbeae00640 1 -- 172.21.15.86:0/2476450354 shutdown_connections 2024-04-25T14:04:37.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.917+0000 7fdbeae00640 1 --2- 172.21.15.86:0/2476450354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbe41e62b0 0x7fdbe41e66c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.917+0000 7fdbeae00640 1 --2- 172.21.15.86:0/2476450354 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdbc006f400 0x7fdbc0071910 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.917+0000 7fdbeae00640 1 --2- 172.21.15.86:0/2476450354 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbe409c300 0x7fdbe415be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.917+0000 7fdbeae00640 1 --2- 172.21.15.86:0/2476450354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbc8004d50 0x7fdbe409d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:37.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.917+0000 7fdbeae00640 1 -- 172.21.15.86:0/2476450354 >> 172.21.15.86:0/2476450354 conn(0x7fdbe40b4650 msgr2=0x7fdbe40b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:37.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.917+0000 7fdbeae00640 1 -- 172.21.15.86:0/2476450354 shutdown_connections 2024-04-25T14:04:37.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:37.917+0000 7fdbeae00640 1 -- 172.21.15.86:0/2476450354 wait complete. 2024-04-25T14:04:37.938 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T14:04:38.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.089+0000 7f9dbea00640 1 Processor -- start 2024-04-25T14:04:38.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.089+0000 7f9dbea00640 1 -- start start 2024-04-25T14:04:38.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.089+0000 7f9dbea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9db815faa0 0x7f9db806d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:38.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.089+0000 7f9dbea00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9db80aeea0 0x7f9db80af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:38.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.089+0000 7f9dbea00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9db806c670 con 0x7f9db80b59e0 2024-04-25T14:04:38.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.089+0000 7f9dbea00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9db806d560 con 0x7f9db815faa0 2024-04-25T14:04:38.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.089+0000 7f9dbea00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9db806c950 con 0x7f9db80aeea0 2024-04-25T14:04:38.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.089+0000 7f9db7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9db815faa0 0x7f9db806d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:38.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.089+0000 7f9db7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9db815faa0 0x7f9db806d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45382/0 (socket says 172.21.15.86:45382) 2024-04-25T14:04:38.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.089+0000 7f9db7e00640 1 -- 172.21.15.86:0/158934638 learned_addr learned my addr 172.21.15.86:0/158934638 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:38.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.089+0000 7f9dbca00640 1 --2- 172.21.15.86:0/158934638 >> v2:172.21.15.155:3300/0 conn(0x7f9db80aeea0 0x7f9db80af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:38.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.089+0000 7f9db7e00640 1 -- 172.21.15.86:0/158934638 >> v2:172.21.15.155:3300/0 conn(0x7f9db80aeea0 msgr2=0x7f9db80af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:38.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.089+0000 7f9db7e00640 1 --2- 172.21.15.86:0/158934638 >> v2:172.21.15.155:3300/0 conn(0x7f9db80aeea0 0x7f9db80af2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:38.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.089+0000 7f9db7e00640 1 -- 172.21.15.86:0/158934638 >> v1:172.21.15.155:6789/0 conn(0x7f9db80b59e0 legacy=0x7f9db80b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:38.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.089+0000 7f9db7e00640 1 -- 172.21.15.86:0/158934638 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db806cc30 con 0x7f9db815faa0 2024-04-25T14:04:38.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.089+0000 7f9dbca00640 1 --2- 172.21.15.86:0/158934638 >> v2:172.21.15.155:3300/0 conn(0x7f9db80aeea0 0x7f9db80af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:38.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.093+0000 7f9db7e00640 1 --2- 172.21.15.86:0/158934638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9db815faa0 0x7f9db806d9f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9db80765a0 tx=0x7f9da801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2224294a5f98044 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:38.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.093+0000 7f9db7400640 1 -- 172.21.15.86:0/158934638 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da80a6040 con 0x7f9db815faa0 2024-04-25T14:04:38.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.093+0000 7f9db7400640 1 -- 172.21.15.86:0/158934638 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da80af070 con 0x7f9db815faa0 2024-04-25T14:04:38.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.093+0000 7f9db7400640 1 -- 172.21.15.86:0/158934638 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da80aa860 con 0x7f9db815faa0 2024-04-25T14:04:38.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.093+0000 7f9dbea00640 1 -- 172.21.15.86:0/158934638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9db815faa0 msgr2=0x7f9db806d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:38.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.093+0000 7f9dbea00640 1 --2- 172.21.15.86:0/158934638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9db815faa0 0x7f9db806d9f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9db80765a0 tx=0x7f9da801c4e0 comp rx=0 tx=0).stop 2024-04-25T14:04:38.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.093+0000 7f9dbea00640 1 -- 172.21.15.86:0/158934638 shutdown_connections 2024-04-25T14:04:38.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.093+0000 7f9dbea00640 1 --2- 172.21.15.86:0/158934638 >> v2:172.21.15.155:3300/0 conn(0x7f9db80aeea0 0x7f9db80af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:38.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.093+0000 7f9dbea00640 1 --2- 172.21.15.86:0/158934638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9db815faa0 0x7f9db806d9f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:38.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.093+0000 7f9dbea00640 1 -- 172.21.15.86:0/158934638 >> 172.21.15.86:0/158934638 conn(0x7f9db80d2230 msgr2=0x7f9db80d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:38.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.093+0000 7f9dbea00640 1 -- 172.21.15.86:0/158934638 shutdown_connections 2024-04-25T14:04:38.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.093+0000 7f9dbea00640 1 -- 172.21.15.86:0/158934638 wait complete. 2024-04-25T14:04:38.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.097+0000 7f9dbea00640 1 Processor -- start 2024-04-25T14:04:38.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.097+0000 7f9dbea00640 1 -- start start 2024-04-25T14:04:38.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.097+0000 7f9dbea00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9db80aeea0 0x7f9db814e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:38.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.097+0000 7f9dbea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9db80b59e0 0x7f9db8154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:38.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.097+0000 7f9dbea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9db815faa0 0x7f9db8154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:38.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.097+0000 7f9dbea00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9db8160030 con 0x7f9db80b59e0 2024-04-25T14:04:38.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.097+0000 7f9dbea00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9db815feb0 con 0x7f9db815faa0 2024-04-25T14:04:38.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.097+0000 7f9dbea00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9db8070f30 con 0x7f9db80aeea0 2024-04-25T14:04:38.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.097+0000 7f9daf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9db80b59e0 0x7f9db8154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:38.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.097+0000 7f9daf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9db80b59e0 0x7f9db8154670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45390/0 (socket says 172.21.15.86:45390) 2024-04-25T14:04:38.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.097+0000 7f9daf400640 1 -- 172.21.15.86:0/2041889084 learned_addr learned my addr 172.21.15.86:0/2041889084 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:38.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.097+0000 7f9db7e00640 1 --2- 172.21.15.86:0/2041889084 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9db80aeea0 0x7f9db814e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:38.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.097+0000 7f9dbca00640 1 --2- 172.21.15.86:0/2041889084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9db815faa0 0x7f9db8154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:38.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.097+0000 7f9daf400640 1 -- 172.21.15.86:0/2041889084 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9db80aeea0 msgr2=0x7f9db814e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:38.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.097+0000 7f9daf400640 1 --2- 172.21.15.86:0/2041889084 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9db80aeea0 0x7f9db814e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:38.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.097+0000 7f9daf400640 1 -- 172.21.15.86:0/2041889084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9db815faa0 msgr2=0x7f9db8154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:38.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.097+0000 7f9daf400640 1 --2- 172.21.15.86:0/2041889084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9db815faa0 0x7f9db8154bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:38.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.097+0000 7f9daf400640 1 -- 172.21.15.86:0/2041889084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db814f600 con 0x7f9db80b59e0 2024-04-25T14:04:38.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.097+0000 7f9dbca00640 1 --2- 172.21.15.86:0/2041889084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9db815faa0 0x7f9db8154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:38.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.097+0000 7f9daf400640 1 --2- 172.21.15.86:0/2041889084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9db80b59e0 0x7f9db8154670 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9da000b440 tx=0x7f9da000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:38.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.097+0000 7f9db4c00640 1 -- 172.21.15.86:0/2041889084 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0015020 con 0x7f9db80b59e0 2024-04-25T14:04:38.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.097+0000 7f9db4c00640 1 -- 172.21.15.86:0/2041889084 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da0011890 con 0x7f9db80b59e0 2024-04-25T14:04:38.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.097+0000 7f9dbea00640 1 -- 172.21.15.86:0/2041889084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9db8155740 con 0x7f9db80b59e0 2024-04-25T14:04:38.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.097+0000 7f9db4c00640 1 -- 172.21.15.86:0/2041889084 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0011c80 con 0x7f9db80b59e0 2024-04-25T14:04:38.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.097+0000 7f9dbea00640 1 -- 172.21.15.86:0/2041889084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9db8155d00 con 0x7f9db80b59e0 2024-04-25T14:04:38.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.101+0000 7f9dbea00640 1 -- 172.21.15.86:0/2041889084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9db8158610 con 0x7f9db80b59e0 2024-04-25T14:04:38.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.101+0000 7f9db4c00640 1 -- 172.21.15.86:0/2041889084 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9da0018020 con 0x7f9db80b59e0 2024-04-25T14:04:38.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.101+0000 7f9db4c00640 1 --2- 172.21.15.86:0/2041889084 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9d9406f430 0x7f9d94071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:38.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.101+0000 7f9db4c00640 1 -- 172.21.15.86:0/2041889084 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f9da008f380 con 0x7f9db80b59e0 2024-04-25T14:04:38.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.101+0000 7f9db7e00640 1 --2- 172.21.15.86:0/2041889084 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9d9406f430 0x7f9d94071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:38.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.105+0000 7f9db7e00640 1 --2- 172.21.15.86:0/2041889084 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9d9406f430 0x7f9d94071940 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9da80ae000 tx=0x7f9da80a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:38.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.105+0000 7f9db4c00640 1 -- 172.21.15.86:0/2041889084 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9da001f030 con 0x7f9db80b59e0 2024-04-25T14:04:38.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.265+0000 7f9dbea00640 1 -- 172.21.15.86:0/2041889084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9db8074cf0 con 0x7f9db80b59e0 2024-04-25T14:04:38.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.265+0000 7f9db4c00640 1 -- 172.21.15.86:0/2041889084 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f9da000d040 con 0x7f9db80b59e0 2024-04-25T14:04:38.269 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:04:38.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.265+0000 7f9dbea00640 1 -- 172.21.15.86:0/2041889084 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9d9406f430 msgr2=0x7f9d94071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:38.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.265+0000 7f9dbea00640 1 --2- 172.21.15.86:0/2041889084 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9d9406f430 0x7f9d94071940 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9da80ae000 tx=0x7f9da80a7000 comp rx=0 tx=0).stop 2024-04-25T14:04:38.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.265+0000 7f9dbea00640 1 -- 172.21.15.86:0/2041889084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9db80b59e0 msgr2=0x7f9db8154670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:38.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.265+0000 7f9dbea00640 1 --2- 172.21.15.86:0/2041889084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9db80b59e0 0x7f9db8154670 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9da000b440 tx=0x7f9da000b900 comp rx=0 tx=0).stop 2024-04-25T14:04:38.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.265+0000 7f9dbea00640 1 -- 172.21.15.86:0/2041889084 shutdown_connections 2024-04-25T14:04:38.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.265+0000 7f9dbea00640 1 --2- 172.21.15.86:0/2041889084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9db815faa0 0x7f9db8154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:38.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.265+0000 7f9dbea00640 1 --2- 172.21.15.86:0/2041889084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9db80b59e0 0x7f9db8154670 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:38.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.265+0000 7f9dbea00640 1 --2- 172.21.15.86:0/2041889084 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9d9406f430 0x7f9d94071940 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:38.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.265+0000 7f9dbea00640 1 --2- 172.21.15.86:0/2041889084 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9db80aeea0 0x7f9db814e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:38.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.265+0000 7f9dbea00640 1 -- 172.21.15.86:0/2041889084 >> 172.21.15.86:0/2041889084 conn(0x7f9db80d2230 msgr2=0x7f9db80afc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:38.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.265+0000 7f9dbea00640 1 -- 172.21.15.86:0/2041889084 shutdown_connections 2024-04-25T14:04:38.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.265+0000 7f9dbea00640 1 -- 172.21.15.86:0/2041889084 wait complete. 2024-04-25T14:04:38.323 INFO:tasks.cephfs_test_runner:Starting test: test_client_metrics_and_metadata (tasks.cephfs.test_mds_metrics.TestMDSMetrics) 2024-04-25T14:04:38.324 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:04:38.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.469+0000 7f2f78800640 1 Processor -- start 2024-04-25T14:04:38.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.469+0000 7f2f78800640 1 -- start start 2024-04-25T14:04:38.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.469+0000 7f2f78800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2f700cfca0 0x7f2f7006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:38.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.469+0000 7f2f78800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f700d16e0 0x7f2f700d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:38.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.469+0000 7f2f78800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2f7006c670 0x7f2f7006ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:38.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.469+0000 7f2f78800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2f70070470 con 0x7f2f700cfca0 2024-04-25T14:04:38.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.469+0000 7f2f78800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f2f700d2030 con 0x7f2f7006c670 2024-04-25T14:04:38.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.469+0000 7f2f78800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2f70072bd0 con 0x7f2f700d16e0 2024-04-25T14:04:38.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.469+0000 7f2f75e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2f700cfca0 0x7f2f7006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:38.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.469+0000 7f2f75e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2f700cfca0 0x7f2f7006f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45394/0 (socket says 172.21.15.86:45394) 2024-04-25T14:04:38.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.469+0000 7f2f75e00640 1 -- 172.21.15.86:0/1868603551 learned_addr learned my addr 172.21.15.86:0/1868603551 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:38.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.469+0000 7f2f75400640 1 --2- 172.21.15.86:0/1868603551 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f700d16e0 0x7f2f700d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:38.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.469+0000 7f2f76800640 1 --2- 172.21.15.86:0/1868603551 >> v2:172.21.15.155:3300/0 conn(0x7f2f7006c670 0x7f2f7006ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:38.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.469+0000 7f2f75e00640 1 -- 172.21.15.86:0/1868603551 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f700d16e0 msgr2=0x7f2f700d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:38.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.469+0000 7f2f75e00640 1 --2- 172.21.15.86:0/1868603551 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f700d16e0 0x7f2f700d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:38.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.469+0000 7f2f75e00640 1 -- 172.21.15.86:0/1868603551 >> v2:172.21.15.155:3300/0 conn(0x7f2f7006c670 msgr2=0x7f2f7006ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:38.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.469+0000 7f2f75e00640 1 --2- 172.21.15.86:0/1868603551 >> v2:172.21.15.155:3300/0 conn(0x7f2f7006c670 0x7f2f7006ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:38.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.469+0000 7f2f75e00640 1 -- 172.21.15.86:0/1868603551 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f7006f930 con 0x7f2f700cfca0 2024-04-25T14:04:38.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.473+0000 7f2f75e00640 1 --2- 172.21.15.86:0/1868603551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f700cfca0 0x7f2f7006f3f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2f7006e060 tx=0x7f2f6401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c1cd85ec3114214 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:38.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.473+0000 7f2f74a00640 1 -- 172.21.15.86:0/1868603551 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f640ab5f0 con 0x7f2f700cfca0 2024-04-25T14:04:38.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.473+0000 7f2f74a00640 1 -- 172.21.15.86:0/1868603551 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f640b0070 con 0x7f2f700cfca0 2024-04-25T14:04:38.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.473+0000 7f2f74a00640 1 -- 172.21.15.86:0/1868603551 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f640ab9b0 con 0x7f2f700cfca0 2024-04-25T14:04:38.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.473+0000 7f2f78800640 1 -- 172.21.15.86:0/1868603551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f700cfca0 msgr2=0x7f2f7006f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:38.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.473+0000 7f2f78800640 1 --2- 172.21.15.86:0/1868603551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f700cfca0 0x7f2f7006f3f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2f7006e060 tx=0x7f2f6401c520 comp rx=0 tx=0).stop 2024-04-25T14:04:38.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.473+0000 7f2f78800640 1 -- 172.21.15.86:0/1868603551 shutdown_connections 2024-04-25T14:04:38.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.473+0000 7f2f78800640 1 --2- 172.21.15.86:0/1868603551 >> v2:172.21.15.155:3300/0 conn(0x7f2f7006c670 0x7f2f7006ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:38.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.473+0000 7f2f78800640 1 --2- 172.21.15.86:0/1868603551 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f700d16e0 0x7f2f700d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:38.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.473+0000 7f2f78800640 1 --2- 172.21.15.86:0/1868603551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f700cfca0 0x7f2f7006f3f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:38.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.473+0000 7f2f78800640 1 -- 172.21.15.86:0/1868603551 >> 172.21.15.86:0/1868603551 conn(0x7f2f700b4650 msgr2=0x7f2f700b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:38.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.473+0000 7f2f78800640 1 -- 172.21.15.86:0/1868603551 shutdown_connections 2024-04-25T14:04:38.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.473+0000 7f2f78800640 1 -- 172.21.15.86:0/1868603551 wait complete. 2024-04-25T14:04:38.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.477+0000 7f2f78800640 1 Processor -- start 2024-04-25T14:04:38.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.477+0000 7f2f78800640 1 -- start start 2024-04-25T14:04:38.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.477+0000 7f2f78800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f7006c670 0x7f2f7009a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:38.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.477+0000 7f2f78800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f700cfca0 0x7f2f701547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:38.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.477+0000 7f2f78800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f700d16e0 0x7f2f70154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:38.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.477+0000 7f2f78800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2f700738c0 con 0x7f2f700cfca0 2024-04-25T14:04:38.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.477+0000 7f2f78800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7006d3d0 con 0x7f2f7006c670 2024-04-25T14:04:38.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.477+0000 7f2f78800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2f70073a40 con 0x7f2f700d16e0 2024-04-25T14:04:38.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.477+0000 7f2f75400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f700cfca0 0x7f2f701547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:38.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.477+0000 7f2f75400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f700cfca0 0x7f2f701547a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45404/0 (socket says 172.21.15.86:45404) 2024-04-25T14:04:38.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.477+0000 7f2f75400640 1 -- 172.21.15.86:0/1400341839 learned_addr learned my addr 172.21.15.86:0/1400341839 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:38.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.477+0000 7f2f75e00640 1 --2- 172.21.15.86:0/1400341839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f7006c670 0x7f2f7009a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:38.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.477+0000 7f2f76800640 1 --2- 172.21.15.86:0/1400341839 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f700d16e0 0x7f2f70154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:38.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.481+0000 7f2f75400640 1 -- 172.21.15.86:0/1400341839 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f700d16e0 msgr2=0x7f2f70154ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:38.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.481+0000 7f2f75400640 1 --2- 172.21.15.86:0/1400341839 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f700d16e0 0x7f2f70154ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:38.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.481+0000 7f2f75400640 1 -- 172.21.15.86:0/1400341839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f7006c670 msgr2=0x7f2f7009a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:38.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.481+0000 7f2f75400640 1 --2- 172.21.15.86:0/1400341839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f7006c670 0x7f2f7009a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:38.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.481+0000 7f2f75400640 1 -- 172.21.15.86:0/1400341839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f700c9380 con 0x7f2f700cfca0 2024-04-25T14:04:38.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.481+0000 7f2f75400640 1 --2- 172.21.15.86:0/1400341839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f700cfca0 0x7f2f701547a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2f6000e870 tx=0x7f2f6000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:38.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.481+0000 7f2f6a000640 1 -- 172.21.15.86:0/1400341839 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f60017070 con 0x7f2f700cfca0 2024-04-25T14:04:38.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.481+0000 7f2f6a000640 1 -- 172.21.15.86:0/1400341839 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f6000c940 con 0x7f2f700cfca0 2024-04-25T14:04:38.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.481+0000 7f2f6a000640 1 -- 172.21.15.86:0/1400341839 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6000ccf0 con 0x7f2f700cfca0 2024-04-25T14:04:38.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.481+0000 7f2f78800640 1 -- 172.21.15.86:0/1400341839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f700c9660 con 0x7f2f700cfca0 2024-04-25T14:04:38.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.481+0000 7f2f78800640 1 -- 172.21.15.86:0/1400341839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f700c9c20 con 0x7f2f700cfca0 2024-04-25T14:04:38.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.481+0000 7f2f78800640 1 -- 172.21.15.86:0/1400341839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f7009f4b0 con 0x7f2f700cfca0 2024-04-25T14:04:38.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.481+0000 7f2f6a000640 1 -- 172.21.15.86:0/1400341839 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2f6001a070 con 0x7f2f700cfca0 2024-04-25T14:04:38.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.481+0000 7f2f6a000640 1 --2- 172.21.15.86:0/1400341839 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2f5406f430 0x7f2f54071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:38.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.485+0000 7f2f6a000640 1 -- 172.21.15.86:0/1400341839 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f2f6008f7b0 con 0x7f2f700cfca0 2024-04-25T14:04:38.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.485+0000 7f2f75e00640 1 --2- 172.21.15.86:0/1400341839 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2f5406f430 0x7f2f54071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:38.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.485+0000 7f2f75e00640 1 --2- 172.21.15.86:0/1400341839 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2f5406f430 0x7f2f54071940 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2f640af000 tx=0x7f2f640a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:38.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.489+0000 7f2f6a000640 1 -- 172.21.15.86:0/1400341839 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2f6005bcc0 con 0x7f2f700cfca0 2024-04-25T14:04:38.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.645+0000 7f2f78800640 1 -- 172.21.15.86:0/1400341839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f70069d00 con 0x7f2f700cfca0 2024-04-25T14:04:38.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.645+0000 7f2f6a000640 1 -- 172.21.15.86:0/1400341839 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18902 (secure 0 0 0) 0x7f2f6005ffc0 con 0x7f2f700cfca0 2024-04-25T14:04:38.647 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:38.647 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":26,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:04:31.930803+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:03:49.151473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:03:49.802597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T14:03:53.492008+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:04:38.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.645+0000 7f2f78800640 1 -- 172.21.15.86:0/1400341839 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2f5406f430 msgr2=0x7f2f54071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:38.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.645+0000 7f2f78800640 1 --2- 172.21.15.86:0/1400341839 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2f5406f430 0x7f2f54071940 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2f640af000 tx=0x7f2f640a7000 comp rx=0 tx=0).stop 2024-04-25T14:04:38.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.645+0000 7f2f78800640 1 -- 172.21.15.86:0/1400341839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f700cfca0 msgr2=0x7f2f701547a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:38.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.645+0000 7f2f78800640 1 --2- 172.21.15.86:0/1400341839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f700cfca0 0x7f2f701547a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2f6000e870 tx=0x7f2f6000ec70 comp rx=0 tx=0).stop 2024-04-25T14:04:38.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.649+0000 7f2f78800640 1 -- 172.21.15.86:0/1400341839 shutdown_connections 2024-04-25T14:04:38.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.649+0000 7f2f78800640 1 --2- 172.21.15.86:0/1400341839 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2f5406f430 0x7f2f54071940 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:38.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.649+0000 7f2f78800640 1 --2- 172.21.15.86:0/1400341839 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f700d16e0 0x7f2f70154ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:38.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.649+0000 7f2f78800640 1 --2- 172.21.15.86:0/1400341839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f700cfca0 0x7f2f701547a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:38.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.649+0000 7f2f78800640 1 --2- 172.21.15.86:0/1400341839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f7006c670 0x7f2f7009a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:38.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.649+0000 7f2f78800640 1 -- 172.21.15.86:0/1400341839 >> 172.21.15.86:0/1400341839 conn(0x7f2f700b4650 msgr2=0x7f2f7015d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:38.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.649+0000 7f2f78800640 1 -- 172.21.15.86:0/1400341839 shutdown_connections 2024-04-25T14:04:38.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.649+0000 7f2f78800640 1 -- 172.21.15.86:0/1400341839 wait complete. 2024-04-25T14:04:38.668 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:03:49.151473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:03:49.802597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T14:03:53.492008+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T14:04:38.669 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:04:38.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.821+0000 7fe8ec800640 1 Processor -- start 2024-04-25T14:04:38.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.821+0000 7fe8ec800640 1 -- start start 2024-04-25T14:04:38.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.821+0000 7fe8ec800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8e4163980 0x7fe8e4163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:38.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.821+0000 7fe8ec800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe8e406e340 0x7fe8e406e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:38.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.821+0000 7fe8ec800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8e41648e0 con 0x7fe8e4072630 2024-04-25T14:04:38.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.821+0000 7fe8ec800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe8e40d1520 con 0x7fe8e406e340 2024-04-25T14:04:38.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.821+0000 7fe8ec800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e40d23e0 con 0x7fe8e4163980 2024-04-25T14:04:38.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.821+0000 7fe8ea800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe8e406e340 0x7fe8e406e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:38.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.821+0000 7fe8e9400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8e4163980 0x7fe8e4163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:38.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.821+0000 7fe8ea800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe8e406e340 0x7fe8e406e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:38544/0 (socket says 172.21.15.86:38544) 2024-04-25T14:04:38.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.821+0000 7fe8ea800640 1 -- 172.21.15.86:0/2853934097 learned_addr learned my addr 172.21.15.86:0/2853934097 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:38.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.821+0000 7fe8e9400640 1 -- 172.21.15.86:0/2853934097 >> v2:172.21.15.155:3300/0 conn(0x7fe8e406e340 msgr2=0x7fe8e406e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:38.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.821+0000 7fe8e9400640 1 --2- 172.21.15.86:0/2853934097 >> v2:172.21.15.155:3300/0 conn(0x7fe8e406e340 0x7fe8e406e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:38.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.821+0000 7fe8e9400640 1 -- 172.21.15.86:0/2853934097 >> v1:172.21.15.155:6789/0 conn(0x7fe8e4072630 legacy=0x7fe8e4072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:38.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.821+0000 7fe8e9400640 1 -- 172.21.15.86:0/2853934097 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e4164600 con 0x7fe8e4163980 2024-04-25T14:04:38.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.821+0000 7fe8ea800640 1 --2- 172.21.15.86:0/2853934097 >> v2:172.21.15.155:3300/0 conn(0x7fe8e406e340 0x7fe8e406e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:38.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.825+0000 7fe8e9400640 1 --2- 172.21.15.86:0/2853934097 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8e4163980 0x7fe8e4163d90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe8d40097c0 tx=0x7fe8d401c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96ee4ec49333b8a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:38.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.825+0000 7fe8e8a00640 1 -- 172.21.15.86:0/2853934097 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d40b0070 con 0x7fe8e4163980 2024-04-25T14:04:38.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.825+0000 7fe8e8a00640 1 -- 172.21.15.86:0/2853934097 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d40ab600 con 0x7fe8e4163980 2024-04-25T14:04:38.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.825+0000 7fe8e8a00640 1 -- 172.21.15.86:0/2853934097 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d40ab9d0 con 0x7fe8e4163980 2024-04-25T14:04:38.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.825+0000 7fe8ec800640 1 -- 172.21.15.86:0/2853934097 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8e4163980 msgr2=0x7fe8e4163d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:38.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.825+0000 7fe8ec800640 1 --2- 172.21.15.86:0/2853934097 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8e4163980 0x7fe8e4163d90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe8d40097c0 tx=0x7fe8d401c380 comp rx=0 tx=0).stop 2024-04-25T14:04:38.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.825+0000 7fe8ec800640 1 -- 172.21.15.86:0/2853934097 shutdown_connections 2024-04-25T14:04:38.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.825+0000 7fe8ec800640 1 --2- 172.21.15.86:0/2853934097 >> v2:172.21.15.155:3300/0 conn(0x7fe8e406e340 0x7fe8e406e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:38.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.825+0000 7fe8ec800640 1 --2- 172.21.15.86:0/2853934097 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8e4163980 0x7fe8e4163d90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:38.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.825+0000 7fe8ec800640 1 -- 172.21.15.86:0/2853934097 >> 172.21.15.86:0/2853934097 conn(0x7fe8e40d3580 msgr2=0x7fe8e40d3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:38.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.825+0000 7fe8ec800640 1 -- 172.21.15.86:0/2853934097 shutdown_connections 2024-04-25T14:04:38.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.825+0000 7fe8ec800640 1 -- 172.21.15.86:0/2853934097 wait complete. 2024-04-25T14:04:38.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.829+0000 7fe8ec800640 1 Processor -- start 2024-04-25T14:04:38.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.829+0000 7fe8ec800640 1 -- start start 2024-04-25T14:04:38.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.829+0000 7fe8ec800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8e406e340 0x7fe8e414fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:38.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.829+0000 7fe8ec800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8e4072630 0x7fe8e415ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:38.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.829+0000 7fe8ec800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8e4163980 0x7fe8e415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:38.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.829+0000 7fe8ec800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e416d530 con 0x7fe8e4163980 2024-04-25T14:04:38.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.829+0000 7fe8ec800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e406f0a0 con 0x7fe8e4072630 2024-04-25T14:04:38.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.829+0000 7fe8ec800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e416d6b0 con 0x7fe8e406e340 2024-04-25T14:04:38.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.829+0000 7fe8ea800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8e4163980 0x7fe8e415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:38.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.829+0000 7fe8ea800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8e4163980 0x7fe8e415f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60556/0 (socket says 172.21.15.86:60556) 2024-04-25T14:04:38.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.829+0000 7fe8ea800640 1 -- 172.21.15.86:0/1331998741 learned_addr learned my addr 172.21.15.86:0/1331998741 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:38.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.829+0000 7fe8e9400640 1 --2- 172.21.15.86:0/1331998741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8e4072630 0x7fe8e415ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:38.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.829+0000 7fe8e9e00640 1 --2- 172.21.15.86:0/1331998741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8e406e340 0x7fe8e414fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:38.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.833+0000 7fe8ea800640 1 -- 172.21.15.86:0/1331998741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8e406e340 msgr2=0x7fe8e414fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:38.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.833+0000 7fe8ea800640 1 --2- 172.21.15.86:0/1331998741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8e406e340 0x7fe8e414fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:38.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.833+0000 7fe8ea800640 1 -- 172.21.15.86:0/1331998741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8e4072630 msgr2=0x7fe8e415ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:38.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.833+0000 7fe8ea800640 1 --2- 172.21.15.86:0/1331998741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8e4072630 0x7fe8e415ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:38.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.833+0000 7fe8ea800640 1 -- 172.21.15.86:0/1331998741 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e415db50 con 0x7fe8e4163980 2024-04-25T14:04:38.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.833+0000 7fe8ea800640 1 --2- 172.21.15.86:0/1331998741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8e4163980 0x7fe8e415f3c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000ef90 tx=0x7fe8e000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:38.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.833+0000 7fe8de000640 1 -- 172.21.15.86:0/1331998741 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e000cac0 con 0x7fe8e4163980 2024-04-25T14:04:38.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.833+0000 7fe8de000640 1 -- 172.21.15.86:0/1331998741 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8e000cc60 con 0x7fe8e4163980 2024-04-25T14:04:38.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.833+0000 7fe8ec800640 1 -- 172.21.15.86:0/1331998741 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e415de30 con 0x7fe8e4163980 2024-04-25T14:04:38.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.833+0000 7fe8de000640 1 -- 172.21.15.86:0/1331998741 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0003bc0 con 0x7fe8e4163980 2024-04-25T14:04:38.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.833+0000 7fe8ec800640 1 -- 172.21.15.86:0/1331998741 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e415e3f0 con 0x7fe8e4163980 2024-04-25T14:04:38.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.833+0000 7fe8ec800640 1 -- 172.21.15.86:0/1331998741 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8e4072170 con 0x7fe8e4163980 2024-04-25T14:04:38.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.833+0000 7fe8de000640 1 -- 172.21.15.86:0/1331998741 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe8e001b030 con 0x7fe8e4163980 2024-04-25T14:04:38.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.837+0000 7fe8de000640 1 --2- 172.21.15.86:0/1331998741 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe8c806f430 0x7fe8c8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:38.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.837+0000 7fe8de000640 1 -- 172.21.15.86:0/1331998741 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe8e008f7a0 con 0x7fe8e4163980 2024-04-25T14:04:38.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.837+0000 7fe8e9e00640 1 --2- 172.21.15.86:0/1331998741 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe8c806f430 0x7fe8c8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:38.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.837+0000 7fe8e9e00640 1 --2- 172.21.15.86:0/1331998741 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe8c806f430 0x7fe8c8071940 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8001770 tx=0x7fe8d8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:38.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.841+0000 7fe8de000640 1 -- 172.21.15.86:0/1331998741 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe8e0022020 con 0x7fe8e4163980 2024-04-25T14:04:38.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.997+0000 7fe8ec800640 1 -- 172.21.15.86:0/1331998741 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe8e4076bf0 con 0x7fe8e4163980 2024-04-25T14:04:38.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:38.997+0000 7fe8de000640 1 -- 172.21.15.86:0/1331998741 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fe8e0010040 con 0x7fe8e4163980 2024-04-25T14:04:38.999 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:04:39.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.001+0000 7fe8ec800640 1 -- 172.21.15.86:0/1331998741 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe8c806f430 msgr2=0x7fe8c8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.001+0000 7fe8ec800640 1 --2- 172.21.15.86:0/1331998741 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe8c806f430 0x7fe8c8071940 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8001770 tx=0x7fe8d8009040 comp rx=0 tx=0).stop 2024-04-25T14:04:39.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.001+0000 7fe8ec800640 1 -- 172.21.15.86:0/1331998741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8e4163980 msgr2=0x7fe8e415f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.001+0000 7fe8ec800640 1 --2- 172.21.15.86:0/1331998741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8e4163980 0x7fe8e415f3c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000ef90 tx=0x7fe8e000c490 comp rx=0 tx=0).stop 2024-04-25T14:04:39.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.001+0000 7fe8ec800640 1 -- 172.21.15.86:0/1331998741 shutdown_connections 2024-04-25T14:04:39.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.001+0000 7fe8ec800640 1 --2- 172.21.15.86:0/1331998741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8e4163980 0x7fe8e415f3c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.001+0000 7fe8ec800640 1 --2- 172.21.15.86:0/1331998741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8e4072630 0x7fe8e415ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.001+0000 7fe8ec800640 1 --2- 172.21.15.86:0/1331998741 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe8c806f430 0x7fe8c8071940 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.001+0000 7fe8ec800640 1 --2- 172.21.15.86:0/1331998741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8e406e340 0x7fe8e414fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.001+0000 7fe8ec800640 1 -- 172.21.15.86:0/1331998741 >> 172.21.15.86:0/1331998741 conn(0x7fe8e40d3580 msgr2=0x7fe8e4161a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:39.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.001+0000 7fe8ec800640 1 -- 172.21.15.86:0/1331998741 shutdown_connections 2024-04-25T14:04:39.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.001+0000 7fe8ec800640 1 -- 172.21.15.86:0/1331998741 wait complete. 2024-04-25T14:04:39.021 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:04:39.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.169+0000 7f724dc00640 1 Processor -- start 2024-04-25T14:04:39.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.169+0000 7f724dc00640 1 -- start start 2024-04-25T14:04:39.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.169+0000 7f724dc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7248163980 0x7f7248163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.169+0000 7f724dc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72481648e0 con 0x7f7248072630 2024-04-25T14:04:39.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.169+0000 7f724dc00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72480d1520 con 0x7f724806e340 2024-04-25T14:04:39.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.169+0000 7f724dc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f72480d23e0 con 0x7f7248163980 2024-04-25T14:04:39.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.169+0000 7f7246a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7248072630 0x7f7248072a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54818/0 (socket says 172.21.15.86:54818) 2024-04-25T14:04:39.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.169+0000 7f7246a00640 1 -- 172.21.15.86:0/2124479458 learned_addr learned my addr 172.21.15.86:0/2124479458 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:39.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.169+0000 7f7245600640 1 -- 172.21.15.86:0/2124479458 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4119302577 0 0) 0x7f72481648e0 con 0x7f7248072630 2024-04-25T14:04:39.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.169+0000 7f7245600640 1 -- 172.21.15.86:0/2124479458 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f722c003530 con 0x7f7248072630 2024-04-25T14:04:39.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.169+0000 7f7246000640 1 --2- 172.21.15.86:0/2124479458 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7248163980 0x7f7248163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.173+0000 7f7245600640 1 -- 172.21.15.86:0/2124479458 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3066798318 0 0) 0x7f722c003530 con 0x7f7248072630 2024-04-25T14:04:39.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.173+0000 7f7245600640 1 -- 172.21.15.86:0/2124479458 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7248163980 msgr2=0x7f7248163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.173+0000 7f7245600640 1 --2- 172.21.15.86:0/2124479458 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7248163980 0x7f7248163d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.173+0000 7f7245600640 1 -- 172.21.15.86:0/2124479458 >> v1:172.21.15.155:6789/0 conn(0x7f724806e340 legacy=0x7f724806e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.173+0000 7f7245600640 1 -- 172.21.15.86:0/2124479458 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7248164600 con 0x7f7248072630 2024-04-25T14:04:39.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.173+0000 7f7245600640 1 -- 172.21.15.86:0/2124479458 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f7238001a00 con 0x7f7248072630 2024-04-25T14:04:39.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.173+0000 7f7245600640 1 -- 172.21.15.86:0/2124479458 >> v1:172.21.15.86:6789/0 conn(0x7f7248072630 legacy=0x7f7248072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.173+0000 7f7245600640 1 --2- 172.21.15.86:0/2124479458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f722c003a60 0x7f722c003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.173+0000 7f7245600640 1 --2- 172.21.15.86:0/2124479458 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7248163980 0x7f722c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.173+0000 7f7245600640 1 --2- 172.21.15.86:0/2124479458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f722c004dc0 0x7f722c005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.173+0000 7f7245600640 1 -- 172.21.15.86:0/2124479458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72480d23e0 con 0x7f722c004dc0 2024-04-25T14:04:39.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.173+0000 7f7245600640 1 -- 172.21.15.86:0/2124479458 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f722c00bcd0 con 0x7f722c003a60 2024-04-25T14:04:39.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.173+0000 7f7245600640 1 -- 172.21.15.86:0/2124479458 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f722c00bed0 con 0x7f7248163980 2024-04-25T14:04:39.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.173+0000 7f7247400640 1 --2- 172.21.15.86:0/2124479458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f722c004dc0 0x7f722c005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.173+0000 7f7246a00640 1 --2- 172.21.15.86:0/2124479458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f722c003a60 0x7f722c003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.173+0000 7f7246000640 1 --2- 172.21.15.86:0/2124479458 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7248163980 0x7f722c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.173+0000 7f7247400640 1 -- 172.21.15.86:0/2124479458 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7248163980 msgr2=0x7f722c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.173+0000 7f7247400640 1 --2- 172.21.15.86:0/2124479458 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7248163980 0x7f722c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.173+0000 7f7247400640 1 -- 172.21.15.86:0/2124479458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f722c003a60 msgr2=0x7f722c003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.173+0000 7f7247400640 1 --2- 172.21.15.86:0/2124479458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f722c003a60 0x7f722c003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.173+0000 7f7247400640 1 -- 172.21.15.86:0/2124479458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f722c003530 con 0x7f722c004dc0 2024-04-25T14:04:39.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.177+0000 7f7247400640 1 --2- 172.21.15.86:0/2124479458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f722c004dc0 0x7f722c005290 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f72480c6b60 tx=0x7f724001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e88ca48f30f8dba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.177+0000 7f7245600640 1 -- 172.21.15.86:0/2124479458 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72400013a0 con 0x7f722c004dc0 2024-04-25T14:04:39.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.177+0000 7f7245600640 1 -- 172.21.15.86:0/2124479458 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72400052f0 con 0x7f722c004dc0 2024-04-25T14:04:39.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.177+0000 7f7245600640 1 -- 172.21.15.86:0/2124479458 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7240003840 con 0x7f722c004dc0 2024-04-25T14:04:39.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.177+0000 7f724dc00640 1 -- 172.21.15.86:0/2124479458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f722c004dc0 msgr2=0x7f722c005290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.177+0000 7f724dc00640 1 --2- 172.21.15.86:0/2124479458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f722c004dc0 0x7f722c005290 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f72480c6b60 tx=0x7f724001c380 comp rx=0 tx=0).stop 2024-04-25T14:04:39.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.177+0000 7f7247400640 1 -- 172.21.15.86:0/2124479458 reap_dead start 2024-04-25T14:04:39.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.177+0000 7f724dc00640 1 -- 172.21.15.86:0/2124479458 shutdown_connections 2024-04-25T14:04:39.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.177+0000 7f724dc00640 1 -- 172.21.15.86:0/2124479458 >> 172.21.15.86:0/2124479458 conn(0x7f72480d3580 msgr2=0x7f72480d3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:39.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.177+0000 7f724dc00640 1 -- 172.21.15.86:0/2124479458 shutdown_connections 2024-04-25T14:04:39.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.177+0000 7f724dc00640 1 -- 172.21.15.86:0/2124479458 wait complete. 2024-04-25T14:04:39.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.177+0000 7f724dc00640 1 Processor -- start 2024-04-25T14:04:39.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.177+0000 7f724dc00640 1 -- start start 2024-04-25T14:04:39.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.181+0000 7f724dc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f722c004dc0 0x7f724815b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.181+0000 7f724dc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7248166690 0x7f72480c9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.181+0000 7f724dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72480ca600 0x7f72480caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.181+0000 7f724dc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f724816c530 con 0x7f72480ca600 2024-04-25T14:04:39.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.181+0000 7f724dc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f72481643b0 con 0x7f7248166690 2024-04-25T14:04:39.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.181+0000 7f724dc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f724816c6b0 con 0x7f722c004dc0 2024-04-25T14:04:39.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.181+0000 7f7247400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72480ca600 0x7f72480caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.181+0000 7f7247400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72480ca600 0x7f72480caa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60580/0 (socket says 172.21.15.86:60580) 2024-04-25T14:04:39.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.181+0000 7f7247400640 1 -- 172.21.15.86:0/4116990249 learned_addr learned my addr 172.21.15.86:0/4116990249 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:39.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.181+0000 7f7246000640 1 --2- 172.21.15.86:0/4116990249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7248166690 0x7f72480c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.181+0000 7f7246a00640 1 --2- 172.21.15.86:0/4116990249 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f722c004dc0 0x7f724815b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.181+0000 7f7247400640 1 -- 172.21.15.86:0/4116990249 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f722c004dc0 msgr2=0x7f724815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.181+0000 7f7247400640 1 --2- 172.21.15.86:0/4116990249 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f722c004dc0 0x7f724815b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.181+0000 7f7247400640 1 -- 172.21.15.86:0/4116990249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7248166690 msgr2=0x7f72480c9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.181+0000 7f7247400640 1 --2- 172.21.15.86:0/4116990249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7248166690 0x7f72480c9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.181+0000 7f7247400640 1 -- 172.21.15.86:0/4116990249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72480ce770 con 0x7f72480ca600 2024-04-25T14:04:39.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.181+0000 7f7247400640 1 --2- 172.21.15.86:0/4116990249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72480ca600 0x7f72480caa10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f724001cc20 tx=0x7f722c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.181+0000 7f723ea00640 1 -- 172.21.15.86:0/4116990249 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72400013a0 con 0x7f72480ca600 2024-04-25T14:04:39.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.181+0000 7f723ea00640 1 -- 172.21.15.86:0/4116990249 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72400bb070 con 0x7f72480ca600 2024-04-25T14:04:39.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.181+0000 7f723ea00640 1 -- 172.21.15.86:0/4116990249 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7240007a40 con 0x7f72480ca600 2024-04-25T14:04:39.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.181+0000 7f724dc00640 1 -- 172.21.15.86:0/4116990249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72480cea50 con 0x7f72480ca600 2024-04-25T14:04:39.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.181+0000 7f724dc00640 1 -- 172.21.15.86:0/4116990249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72480b6da0 con 0x7f72480ca600 2024-04-25T14:04:39.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.181+0000 7f724dc00640 1 -- 172.21.15.86:0/4116990249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72480c9ca0 con 0x7f72480ca600 2024-04-25T14:04:39.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.185+0000 7f723ea00640 1 -- 172.21.15.86:0/4116990249 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7240005270 con 0x7f72480ca600 2024-04-25T14:04:39.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.185+0000 7f723ea00640 1 --2- 172.21.15.86:0/4116990249 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f722806f430 0x7f7228071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.185+0000 7f723ea00640 1 -- 172.21.15.86:0/4116990249 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f7240129180 con 0x7f72480ca600 2024-04-25T14:04:39.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.185+0000 7f7246a00640 1 --2- 172.21.15.86:0/4116990249 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f722806f430 0x7f7228071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.185+0000 7f7246a00640 1 --2- 172.21.15.86:0/4116990249 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f722806f430 0x7f7228071940 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7238000f90 tx=0x7f7238006040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.189+0000 7f723ea00640 1 -- 172.21.15.86:0/4116990249 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7240016090 con 0x7f72480ca600 2024-04-25T14:04:39.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.345+0000 7f724dc00640 1 -- 172.21.15.86:0/4116990249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7248072170 con 0x7f72480ca600 2024-04-25T14:04:39.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.345+0000 7f723ea00640 1 -- 172.21.15.86:0/4116990249 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f72400f5bd0 con 0x7f72480ca600 2024-04-25T14:04:39.346 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:04:39.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.345+0000 7f724dc00640 1 -- 172.21.15.86:0/4116990249 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f722806f430 msgr2=0x7f7228071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.345+0000 7f724dc00640 1 --2- 172.21.15.86:0/4116990249 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f722806f430 0x7f7228071940 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7238000f90 tx=0x7f7238006040 comp rx=0 tx=0).stop 2024-04-25T14:04:39.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.345+0000 7f724dc00640 1 -- 172.21.15.86:0/4116990249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72480ca600 msgr2=0x7f72480caa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.345+0000 7f724dc00640 1 --2- 172.21.15.86:0/4116990249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72480ca600 0x7f72480caa10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f724001cc20 tx=0x7f722c005840 comp rx=0 tx=0).stop 2024-04-25T14:04:39.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.345+0000 7f724dc00640 1 -- 172.21.15.86:0/4116990249 shutdown_connections 2024-04-25T14:04:39.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.345+0000 7f724dc00640 1 --2- 172.21.15.86:0/4116990249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72480ca600 0x7f72480caa10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.345+0000 7f724dc00640 1 --2- 172.21.15.86:0/4116990249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7248166690 0x7f72480c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.345+0000 7f724dc00640 1 --2- 172.21.15.86:0/4116990249 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f722806f430 0x7f7228071940 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.345+0000 7f724dc00640 1 --2- 172.21.15.86:0/4116990249 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f722c004dc0 0x7f724815b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.345+0000 7f724dc00640 1 -- 172.21.15.86:0/4116990249 >> 172.21.15.86:0/4116990249 conn(0x7f72480d3580 msgr2=0x7f7248162240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:39.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.345+0000 7f724dc00640 1 -- 172.21.15.86:0/4116990249 shutdown_connections 2024-04-25T14:04:39.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.345+0000 7f724dc00640 1 -- 172.21.15.86:0/4116990249 wait complete. 2024-04-25T14:04:39.368 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:04:39.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.517+0000 7f9d47200640 1 Processor -- start 2024-04-25T14:04:39.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.517+0000 7f9d47200640 1 -- start start 2024-04-25T14:04:39.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d47200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d4806f990 0x7f9d4806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d47200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9d480b6530 0x7f9d480b6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d47200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d480702e0 con 0x7f9d481615c0 2024-04-25T14:04:39.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d47200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9d481622f0 con 0x7f9d480b6530 2024-04-25T14:04:39.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d47200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9d480d1730 con 0x7f9d4806f990 2024-04-25T14:04:39.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d3fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9d481615c0 0x7f9d481619d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54826/0 (socket says 172.21.15.86:54826) 2024-04-25T14:04:39.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d3fe00640 1 -- 172.21.15.86:0/1898443501 learned_addr learned my addr 172.21.15.86:0/1898443501 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:39.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d46800640 1 --2- 172.21.15.86:0/1898443501 >> v2:172.21.15.155:3300/0 conn(0x7f9d480b6530 0x7f9d480b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d45400640 1 -- 172.21.15.86:0/1898443501 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 264262803 0 0) 0x7f9d30001120 con 0x7f9d481615c0 2024-04-25T14:04:39.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d45e00640 1 --2- 172.21.15.86:0/1898443501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d4806f990 0x7f9d4806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d45400640 1 -- 172.21.15.86:0/1898443501 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d2c003530 con 0x7f9d481615c0 2024-04-25T14:04:39.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d45400640 1 -- 172.21.15.86:0/1898443501 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4045727218 0 0) 0x7f9d2c003530 con 0x7f9d481615c0 2024-04-25T14:04:39.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d45400640 1 -- 172.21.15.86:0/1898443501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d4806f990 msgr2=0x7f9d4806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d45400640 1 --2- 172.21.15.86:0/1898443501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d4806f990 0x7f9d4806fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d45400640 1 -- 172.21.15.86:0/1898443501 >> v2:172.21.15.155:3300/0 conn(0x7f9d480b6530 msgr2=0x7f9d480b6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d45400640 1 --2- 172.21.15.86:0/1898443501 >> v2:172.21.15.155:3300/0 conn(0x7f9d480b6530 0x7f9d480b6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d45400640 1 -- 172.21.15.86:0/1898443501 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d480d1a10 con 0x7f9d481615c0 2024-04-25T14:04:39.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d45400640 1 -- 172.21.15.86:0/1898443501 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f9d30001850 con 0x7f9d481615c0 2024-04-25T14:04:39.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d45400640 1 -- 172.21.15.86:0/1898443501 >> v1:172.21.15.86:6789/0 conn(0x7f9d481615c0 legacy=0x7f9d481619d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d45400640 1 --2- 172.21.15.86:0/1898443501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d2c0039c0 0x7f9d2c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d45400640 1 --2- 172.21.15.86:0/1898443501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d2c004810 0x7f9d2c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d45400640 1 --2- 172.21.15.86:0/1898443501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d4806f990 0x7f9d2c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d45400640 1 -- 172.21.15.86:0/1898443501 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d481622f0 con 0x7f9d2c0039c0 2024-04-25T14:04:39.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d45400640 1 -- 172.21.15.86:0/1898443501 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9d480d1730 con 0x7f9d2c004810 2024-04-25T14:04:39.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d45400640 1 -- 172.21.15.86:0/1898443501 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9d2c00ba80 con 0x7f9d4806f990 2024-04-25T14:04:39.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d45e00640 1 --2- 172.21.15.86:0/1898443501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d2c0039c0 0x7f9d2c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d3fe00640 1 --2- 172.21.15.86:0/1898443501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d2c004810 0x7f9d2c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d46800640 1 --2- 172.21.15.86:0/1898443501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d4806f990 0x7f9d2c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d45e00640 1 -- 172.21.15.86:0/1898443501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d4806f990 msgr2=0x7f9d2c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d45e00640 1 --2- 172.21.15.86:0/1898443501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d4806f990 0x7f9d2c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d45e00640 1 -- 172.21.15.86:0/1898443501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d2c004810 msgr2=0x7f9d2c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d45e00640 1 --2- 172.21.15.86:0/1898443501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d2c004810 0x7f9d2c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d45e00640 1 -- 172.21.15.86:0/1898443501 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2c003530 con 0x7f9d2c0039c0 2024-04-25T14:04:39.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.521+0000 7f9d3fe00640 1 --2- 172.21.15.86:0/1898443501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d2c004810 0x7f9d2c004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:39.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.525+0000 7f9d45e00640 1 --2- 172.21.15.86:0/1898443501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d2c0039c0 0x7f9d2c003e30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009ae0 tx=0x7f9d3801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ae2ebe90475442 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.525+0000 7f9d45400640 1 -- 172.21.15.86:0/1898443501 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d380096b0 con 0x7f9d2c0039c0 2024-04-25T14:04:39.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.525+0000 7f9d45400640 1 -- 172.21.15.86:0/1898443501 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d380a6470 con 0x7f9d2c0039c0 2024-04-25T14:04:39.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.525+0000 7f9d45400640 1 -- 172.21.15.86:0/1898443501 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d380a6860 con 0x7f9d2c0039c0 2024-04-25T14:04:39.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.525+0000 7f9d47200640 1 -- 172.21.15.86:0/1898443501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d2c0039c0 msgr2=0x7f9d2c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.525+0000 7f9d47200640 1 --2- 172.21.15.86:0/1898443501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d2c0039c0 0x7f9d2c003e30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009ae0 tx=0x7f9d3801ca90 comp rx=0 tx=0).stop 2024-04-25T14:04:39.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.525+0000 7f9d46800640 1 -- 172.21.15.86:0/1898443501 reap_dead start 2024-04-25T14:04:39.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.525+0000 7f9d47200640 1 -- 172.21.15.86:0/1898443501 shutdown_connections 2024-04-25T14:04:39.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.525+0000 7f9d47200640 1 -- 172.21.15.86:0/1898443501 >> 172.21.15.86:0/1898443501 conn(0x7f9d480cfea0 msgr2=0x7f9d480d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:39.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.525+0000 7f9d47200640 1 -- 172.21.15.86:0/1898443501 shutdown_connections 2024-04-25T14:04:39.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.525+0000 7f9d47200640 1 -- 172.21.15.86:0/1898443501 wait complete. 2024-04-25T14:04:39.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.529+0000 7f9d47200640 1 Processor -- start 2024-04-25T14:04:39.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.529+0000 7f9d47200640 1 -- start start 2024-04-25T14:04:39.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.529+0000 7f9d47200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d2c0039c0 0x7f9d480cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.529+0000 7f9d47200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d480ca380 0x7f9d481655b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.529+0000 7f9d47200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d4815c5f0 0x7f9d481f9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.529+0000 7f9d47200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d48169160 con 0x7f9d2c0039c0 2024-04-25T14:04:39.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.529+0000 7f9d47200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4815df20 con 0x7f9d480ca380 2024-04-25T14:04:39.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.529+0000 7f9d47200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9d481692e0 con 0x7f9d4815c5f0 2024-04-25T14:04:39.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.529+0000 7f9d45e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d2c0039c0 0x7f9d480cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.529+0000 7f9d45e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d2c0039c0 0x7f9d480cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60606/0 (socket says 172.21.15.86:60606) 2024-04-25T14:04:39.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.529+0000 7f9d45e00640 1 -- 172.21.15.86:0/3751698027 learned_addr learned my addr 172.21.15.86:0/3751698027 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:39.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.529+0000 7f9d3fe00640 1 --2- 172.21.15.86:0/3751698027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d480ca380 0x7f9d481655b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.529+0000 7f9d46800640 1 --2- 172.21.15.86:0/3751698027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d4815c5f0 0x7f9d481f9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.529+0000 7f9d45e00640 1 -- 172.21.15.86:0/3751698027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d4815c5f0 msgr2=0x7f9d481f9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.529+0000 7f9d45e00640 1 --2- 172.21.15.86:0/3751698027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d4815c5f0 0x7f9d481f9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.529+0000 7f9d45e00640 1 -- 172.21.15.86:0/3751698027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d480ca380 msgr2=0x7f9d481655b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.529+0000 7f9d45e00640 1 --2- 172.21.15.86:0/3751698027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d480ca380 0x7f9d481655b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.529+0000 7f9d45e00640 1 -- 172.21.15.86:0/3751698027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d48167470 con 0x7f9d2c0039c0 2024-04-25T14:04:39.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.529+0000 7f9d46800640 1 --2- 172.21.15.86:0/3751698027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d4815c5f0 0x7f9d481f9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:39.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.529+0000 7f9d3fe00640 1 --2- 172.21.15.86:0/3751698027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d480ca380 0x7f9d481655b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:39.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.529+0000 7f9d45e00640 1 --2- 172.21.15.86:0/3751698027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d2c0039c0 0x7f9d480cccb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9d3801c8c0 tx=0x7f9d2c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.529+0000 7f9d3e000640 1 -- 172.21.15.86:0/3751698027 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d380be070 con 0x7f9d2c0039c0 2024-04-25T14:04:39.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.533+0000 7f9d3e000640 1 -- 172.21.15.86:0/3751698027 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d3800f000 con 0x7f9d2c0039c0 2024-04-25T14:04:39.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.533+0000 7f9d47200640 1 -- 172.21.15.86:0/3751698027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d48166d20 con 0x7f9d2c0039c0 2024-04-25T14:04:39.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.533+0000 7f9d3e000640 1 -- 172.21.15.86:0/3751698027 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d380a7450 con 0x7f9d2c0039c0 2024-04-25T14:04:39.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.533+0000 7f9d47200640 1 -- 172.21.15.86:0/3751698027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d48167190 con 0x7f9d2c0039c0 2024-04-25T14:04:39.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.533+0000 7f9d47200640 1 -- 172.21.15.86:0/3751698027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d48076b80 con 0x7f9d2c0039c0 2024-04-25T14:04:39.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.533+0000 7f9d3e000640 1 -- 172.21.15.86:0/3751698027 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9d380c1070 con 0x7f9d2c0039c0 2024-04-25T14:04:39.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.533+0000 7f9d3e000640 1 --2- 172.21.15.86:0/3751698027 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9d2406f430 0x7f9d24071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.533+0000 7f9d3e000640 1 -- 172.21.15.86:0/3751698027 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f9d38129090 con 0x7f9d2c0039c0 2024-04-25T14:04:39.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.533+0000 7f9d3fe00640 1 --2- 172.21.15.86:0/3751698027 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9d2406f430 0x7f9d24071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.533+0000 7f9d3fe00640 1 --2- 172.21.15.86:0/3751698027 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9d2406f430 0x7f9d24071940 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9d30001300 tx=0x7f9d30009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.537+0000 7f9d3e000640 1 -- 172.21.15.86:0/3751698027 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9d380ac020 con 0x7f9d2c0039c0 2024-04-25T14:04:39.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.697+0000 7f9d47200640 1 -- 172.21.15.86:0/3751698027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9d48069d00 con 0x7f9d2c0039c0 2024-04-25T14:04:39.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.697+0000 7f9d3e000640 1 -- 172.21.15.86:0/3751698027 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f9d380167a0 con 0x7f9d2c0039c0 2024-04-25T14:04:39.698 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:04:39.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.697+0000 7f9d47200640 1 -- 172.21.15.86:0/3751698027 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9d2406f430 msgr2=0x7f9d24071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.697+0000 7f9d47200640 1 --2- 172.21.15.86:0/3751698027 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9d2406f430 0x7f9d24071940 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9d30001300 tx=0x7f9d30009040 comp rx=0 tx=0).stop 2024-04-25T14:04:39.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.697+0000 7f9d47200640 1 -- 172.21.15.86:0/3751698027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d2c0039c0 msgr2=0x7f9d480cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.697+0000 7f9d47200640 1 --2- 172.21.15.86:0/3751698027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d2c0039c0 0x7f9d480cccb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9d3801c8c0 tx=0x7f9d2c005840 comp rx=0 tx=0).stop 2024-04-25T14:04:39.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.697+0000 7f9d47200640 1 -- 172.21.15.86:0/3751698027 shutdown_connections 2024-04-25T14:04:39.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.697+0000 7f9d47200640 1 --2- 172.21.15.86:0/3751698027 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9d2406f430 0x7f9d24071940 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.697+0000 7f9d47200640 1 --2- 172.21.15.86:0/3751698027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d4815c5f0 0x7f9d481f9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.697+0000 7f9d47200640 1 --2- 172.21.15.86:0/3751698027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d480ca380 0x7f9d481655b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.697+0000 7f9d47200640 1 --2- 172.21.15.86:0/3751698027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d2c0039c0 0x7f9d480cccb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.697+0000 7f9d47200640 1 -- 172.21.15.86:0/3751698027 >> 172.21.15.86:0/3751698027 conn(0x7f9d480cfea0 msgr2=0x7f9d480b4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:39.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.697+0000 7f9d47200640 1 -- 172.21.15.86:0/3751698027 shutdown_connections 2024-04-25T14:04:39.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.697+0000 7f9d47200640 1 -- 172.21.15.86:0/3751698027 wait complete. 2024-04-25T14:04:39.720 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T14:04:39.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.869+0000 7f2e7dc00640 1 Processor -- start 2024-04-25T14:04:39.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.869+0000 7f2e7dc00640 1 -- start start 2024-04-25T14:04:39.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.869+0000 7f2e7dc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2e780cde90 0x7f2e780ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.869+0000 7f2e7dc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e7815e5c0 con 0x7f2e78070330 2024-04-25T14:04:39.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.869+0000 7f2e7dc00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e7815e8a0 con 0x7f2e780d1df0 2024-04-25T14:04:39.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.869+0000 7f2e7dc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2e7806fd40 con 0x7f2e780cde90 2024-04-25T14:04:39.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.869+0000 7f2e77400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2e780cde90 0x7f2e780ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.869+0000 7f2e76a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2e78070330 0x7f2e780d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54836/0 (socket says 172.21.15.86:54836) 2024-04-25T14:04:39.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.869+0000 7f2e76a00640 1 -- 172.21.15.86:0/1112550091 learned_addr learned my addr 172.21.15.86:0/1112550091 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:39.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.869+0000 7f2e77400640 1 -- 172.21.15.86:0/1112550091 >> v1:172.21.15.155:6789/0 conn(0x7f2e780d1df0 legacy=0x7f2e780cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.869+0000 7f2e77400640 1 -- 172.21.15.86:0/1112550091 >> v1:172.21.15.86:6789/0 conn(0x7f2e78070330 legacy=0x7f2e780d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.869+0000 7f2e77400640 1 -- 172.21.15.86:0/1112550091 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e7815eb80 con 0x7f2e780cde90 2024-04-25T14:04:39.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.869+0000 7f2e75600640 1 -- 172.21.15.86:0/1112550091 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3597733612 0 0) 0x7f2e7815e5c0 con 0x7f2e78070330 2024-04-25T14:04:39.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.873+0000 7f2e77400640 1 --2- 172.21.15.86:0/1112550091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e780cde90 0x7f2e780ce2a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2e70009dc0 tx=0x7f2e7001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dafc01b241ba4636 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.873+0000 7f2e75600640 1 -- 172.21.15.86:0/1112550091 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e700b2070 con 0x7f2e780cde90 2024-04-25T14:04:39.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.873+0000 7f2e75600640 1 -- 172.21.15.86:0/1112550091 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e700a5860 con 0x7f2e780cde90 2024-04-25T14:04:39.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.873+0000 7f2e75600640 1 -- 172.21.15.86:0/1112550091 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e700a5c30 con 0x7f2e780cde90 2024-04-25T14:04:39.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.873+0000 7f2e7dc00640 1 -- 172.21.15.86:0/1112550091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e780cde90 msgr2=0x7f2e780ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.873+0000 7f2e7dc00640 1 --2- 172.21.15.86:0/1112550091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e780cde90 0x7f2e780ce2a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2e70009dc0 tx=0x7f2e7001c890 comp rx=0 tx=0).stop 2024-04-25T14:04:39.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.873+0000 7f2e7dc00640 1 -- 172.21.15.86:0/1112550091 shutdown_connections 2024-04-25T14:04:39.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.873+0000 7f2e7dc00640 1 --2- 172.21.15.86:0/1112550091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e780cde90 0x7f2e780ce2a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.873+0000 7f2e7dc00640 1 -- 172.21.15.86:0/1112550091 >> 172.21.15.86:0/1112550091 conn(0x7f2e780b4650 msgr2=0x7f2e780b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:39.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.873+0000 7f2e7dc00640 1 -- 172.21.15.86:0/1112550091 shutdown_connections 2024-04-25T14:04:39.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.873+0000 7f2e7dc00640 1 -- 172.21.15.86:0/1112550091 wait complete. 2024-04-25T14:04:39.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.877+0000 7f2e7dc00640 1 Processor -- start 2024-04-25T14:04:39.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.877+0000 7f2e7dc00640 1 -- start start 2024-04-25T14:04:39.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.877+0000 7f2e7dc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e78070330 0x7f2e7815a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.877+0000 7f2e7dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e780cde90 0x7f2e780ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.877+0000 7f2e7dc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e7815ace0 0x7f2e780cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.877+0000 7f2e7dc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e780708c0 con 0x7f2e780cde90 2024-04-25T14:04:39.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.877+0000 7f2e7dc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2e78070740 con 0x7f2e78070330 2024-04-25T14:04:39.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.877+0000 7f2e7dc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2e7816bfd0 con 0x7f2e7815ace0 2024-04-25T14:04:39.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.877+0000 7f2e76000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e780cde90 0x7f2e780ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.877+0000 7f2e76000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e780cde90 0x7f2e780ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60626/0 (socket says 172.21.15.86:60626) 2024-04-25T14:04:39.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.877+0000 7f2e76000640 1 -- 172.21.15.86:0/3032494558 learned_addr learned my addr 172.21.15.86:0/3032494558 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:39.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.877+0000 7f2e76a00640 1 --2- 172.21.15.86:0/3032494558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e78070330 0x7f2e7815a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.877+0000 7f2e77400640 1 --2- 172.21.15.86:0/3032494558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e7815ace0 0x7f2e780cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.881+0000 7f2e76000640 1 -- 172.21.15.86:0/3032494558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e7815ace0 msgr2=0x7f2e780cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.881+0000 7f2e76000640 1 --2- 172.21.15.86:0/3032494558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e7815ace0 0x7f2e780cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.881+0000 7f2e76000640 1 -- 172.21.15.86:0/3032494558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e78070330 msgr2=0x7f2e7815a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.881+0000 7f2e76000640 1 --2- 172.21.15.86:0/3032494558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e78070330 0x7f2e7815a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.881+0000 7f2e76000640 1 -- 172.21.15.86:0/3032494558 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e78074280 con 0x7f2e780cde90 2024-04-25T14:04:39.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.881+0000 7f2e76000640 1 --2- 172.21.15.86:0/3032494558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e780cde90 0x7f2e780ceeb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2e6400c3f0 tx=0x7f2e6400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.881+0000 7f2e6ea00640 1 -- 172.21.15.86:0/3032494558 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e64017070 con 0x7f2e780cde90 2024-04-25T14:04:39.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.881+0000 7f2e6ea00640 1 -- 172.21.15.86:0/3032494558 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e640052d0 con 0x7f2e780cde90 2024-04-25T14:04:39.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.881+0000 7f2e6ea00640 1 -- 172.21.15.86:0/3032494558 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e64003840 con 0x7f2e780cde90 2024-04-25T14:04:39.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.881+0000 7f2e7dc00640 1 -- 172.21.15.86:0/3032494558 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e78074560 con 0x7f2e780cde90 2024-04-25T14:04:39.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.881+0000 7f2e7dc00640 1 -- 172.21.15.86:0/3032494558 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e780c9500 con 0x7f2e780cde90 2024-04-25T14:04:39.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.881+0000 7f2e7dc00640 1 -- 172.21.15.86:0/3032494558 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e78069d00 con 0x7f2e780cde90 2024-04-25T14:04:39.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.881+0000 7f2e6ea00640 1 -- 172.21.15.86:0/3032494558 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2e6401a070 con 0x7f2e780cde90 2024-04-25T14:04:39.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.881+0000 7f2e6ea00640 1 --2- 172.21.15.86:0/3032494558 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2e5806f400 0x7f2e58071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.885+0000 7f2e6ea00640 1 -- 172.21.15.86:0/3032494558 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f2e6408e8f0 con 0x7f2e780cde90 2024-04-25T14:04:39.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.885+0000 7f2e76a00640 1 --2- 172.21.15.86:0/3032494558 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2e5806f400 0x7f2e58071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.885+0000 7f2e76a00640 1 --2- 172.21.15.86:0/3032494558 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2e5806f400 0x7f2e58071910 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2e68000f70 tx=0x7f2e68007240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:39.889+0000 7f2e6ea00640 1 -- 172.21.15.86:0/3032494558 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2e64005480 con 0x7f2e780cde90 2024-04-25T14:04:40.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.049+0000 7f2e7dc00640 1 -- 172.21.15.86:0/3032494558 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2e78076bf0 con 0x7f2e780cde90 2024-04-25T14:04:40.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.049+0000 7f2e6ea00640 1 -- 172.21.15.86:0/3032494558 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f2e6405ae20 con 0x7f2e780cde90 2024-04-25T14:04:40.050 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:04:40.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.049+0000 7f2e7dc00640 1 -- 172.21.15.86:0/3032494558 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2e5806f400 msgr2=0x7f2e58071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:40.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.049+0000 7f2e7dc00640 1 --2- 172.21.15.86:0/3032494558 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2e5806f400 0x7f2e58071910 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2e68000f70 tx=0x7f2e68007240 comp rx=0 tx=0).stop 2024-04-25T14:04:40.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.049+0000 7f2e7dc00640 1 -- 172.21.15.86:0/3032494558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e780cde90 msgr2=0x7f2e780ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:40.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.049+0000 7f2e7dc00640 1 --2- 172.21.15.86:0/3032494558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e780cde90 0x7f2e780ceeb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2e6400c3f0 tx=0x7f2e6400c8b0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.049+0000 7f2e7dc00640 1 -- 172.21.15.86:0/3032494558 shutdown_connections 2024-04-25T14:04:40.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.049+0000 7f2e7dc00640 1 --2- 172.21.15.86:0/3032494558 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2e5806f400 0x7f2e58071910 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.049+0000 7f2e7dc00640 1 --2- 172.21.15.86:0/3032494558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e7815ace0 0x7f2e780cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.049+0000 7f2e7dc00640 1 --2- 172.21.15.86:0/3032494558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e780cde90 0x7f2e780ceeb0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.049+0000 7f2e7dc00640 1 --2- 172.21.15.86:0/3032494558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e78070330 0x7f2e7815a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.049+0000 7f2e7dc00640 1 -- 172.21.15.86:0/3032494558 >> 172.21.15.86:0/3032494558 conn(0x7f2e780b4650 msgr2=0x7f2e7815c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:40.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.049+0000 7f2e7dc00640 1 -- 172.21.15.86:0/3032494558 shutdown_connections 2024-04-25T14:04:40.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.049+0000 7f2e7dc00640 1 -- 172.21.15.86:0/3032494558 wait complete. 2024-04-25T14:04:40.071 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_client_metrics_and_metadata' 2024-04-25T14:04:40.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.217+0000 7fe3c6400640 1 Processor -- start 2024-04-25T14:04:40.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.217+0000 7fe3c6400640 1 -- start start 2024-04-25T14:04:40.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.217+0000 7fe3c6400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe3c0070330 0x7fe3c00d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:40.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.217+0000 7fe3c6400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe3c00d1c20 0x7fe3c006ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:40.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.217+0000 7fe3c6400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3c00d2030 con 0x7fe3c006f270 2024-04-25T14:04:40.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.217+0000 7fe3c6400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe3c015e5c0 con 0x7fe3c0070330 2024-04-25T14:04:40.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.217+0000 7fe3c6400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe3c015e8a0 con 0x7fe3c00d1c20 2024-04-25T14:04:40.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.217+0000 7fe3bf400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe3c0070330 0x7fe3c00d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:40.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.217+0000 7fe3bfe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe3c006f270 0x7fe3c006f680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54850/0 (socket says 172.21.15.86:54850) 2024-04-25T14:04:40.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.217+0000 7fe3bfe00640 1 -- 172.21.15.86:0/2844949249 learned_addr learned my addr 172.21.15.86:0/2844949249 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:40.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.217+0000 7fe3bea00640 1 --2- 172.21.15.86:0/2844949249 >> v2:172.21.15.155:3300/0 conn(0x7fe3c00d1c20 0x7fe3c006ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:40.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.217+0000 7fe3bf400640 1 -- 172.21.15.86:0/2844949249 >> v2:172.21.15.155:3300/0 conn(0x7fe3c00d1c20 msgr2=0x7fe3c006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:40.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.217+0000 7fe3bf400640 1 --2- 172.21.15.86:0/2844949249 >> v2:172.21.15.155:3300/0 conn(0x7fe3c00d1c20 0x7fe3c006ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.217+0000 7fe3bf400640 1 -- 172.21.15.86:0/2844949249 >> v1:172.21.15.86:6789/0 conn(0x7fe3c006f270 legacy=0x7fe3c006f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:40.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.217+0000 7fe3bf400640 1 -- 172.21.15.86:0/2844949249 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c015eb80 con 0x7fe3c0070330 2024-04-25T14:04:40.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.221+0000 7fe3bf400640 1 --2- 172.21.15.86:0/2844949249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c0070330 0x7fe3c00d16e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe3b00097c0 tx=0x7fe3b001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232a07d02c26597e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:40.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.221+0000 7fe3be000640 1 -- 172.21.15.86:0/2844949249 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b00b0070 con 0x7fe3c0070330 2024-04-25T14:04:40.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.221+0000 7fe3be000640 1 -- 172.21.15.86:0/2844949249 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3b00ab600 con 0x7fe3c0070330 2024-04-25T14:04:40.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.221+0000 7fe3be000640 1 -- 172.21.15.86:0/2844949249 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b00a4e60 con 0x7fe3c0070330 2024-04-25T14:04:40.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.221+0000 7fe3c6400640 1 -- 172.21.15.86:0/2844949249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c0070330 msgr2=0x7fe3c00d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:40.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.221+0000 7fe3c6400640 1 --2- 172.21.15.86:0/2844949249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c0070330 0x7fe3c00d16e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe3b00097c0 tx=0x7fe3b001c3d0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.221+0000 7fe3c6400640 1 -- 172.21.15.86:0/2844949249 shutdown_connections 2024-04-25T14:04:40.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.221+0000 7fe3c6400640 1 --2- 172.21.15.86:0/2844949249 >> v2:172.21.15.155:3300/0 conn(0x7fe3c00d1c20 0x7fe3c006ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.221+0000 7fe3c6400640 1 --2- 172.21.15.86:0/2844949249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c0070330 0x7fe3c00d16e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.221+0000 7fe3c6400640 1 -- 172.21.15.86:0/2844949249 >> 172.21.15.86:0/2844949249 conn(0x7fe3c00b4650 msgr2=0x7fe3c00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:40.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.221+0000 7fe3c6400640 1 -- 172.21.15.86:0/2844949249 shutdown_connections 2024-04-25T14:04:40.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.221+0000 7fe3c6400640 1 -- 172.21.15.86:0/2844949249 wait complete. 2024-04-25T14:04:40.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.225+0000 7fe3c6400640 1 Processor -- start 2024-04-25T14:04:40.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.225+0000 7fe3c6400640 1 -- start start 2024-04-25T14:04:40.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.225+0000 7fe3c6400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3c006f270 0x7fe3c00ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:40.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.225+0000 7fe3c6400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c0070330 0x7fe3c01578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:40.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.225+0000 7fe3c6400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3c00d1c20 0x7fe3c015c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:40.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.225+0000 7fe3c6400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c00708c0 con 0x7fe3c0070330 2024-04-25T14:04:40.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.225+0000 7fe3c6400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c0070740 con 0x7fe3c00d1c20 2024-04-25T14:04:40.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.225+0000 7fe3c6400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe3c016afc0 con 0x7fe3c006f270 2024-04-25T14:04:40.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.225+0000 7fe3bea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c0070330 0x7fe3c01578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:40.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.225+0000 7fe3bea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c0070330 0x7fe3c01578a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60642/0 (socket says 172.21.15.86:60642) 2024-04-25T14:04:40.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.225+0000 7fe3bea00640 1 -- 172.21.15.86:0/3943824901 learned_addr learned my addr 172.21.15.86:0/3943824901 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:40.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.225+0000 7fe3bfe00640 1 --2- 172.21.15.86:0/3943824901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3c00d1c20 0x7fe3c015c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:40.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.225+0000 7fe3bf400640 1 --2- 172.21.15.86:0/3943824901 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3c006f270 0x7fe3c00ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:40.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.225+0000 7fe3bea00640 1 -- 172.21.15.86:0/3943824901 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3c006f270 msgr2=0x7fe3c00ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:40.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.225+0000 7fe3bea00640 1 --2- 172.21.15.86:0/3943824901 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3c006f270 0x7fe3c00ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.225+0000 7fe3bea00640 1 -- 172.21.15.86:0/3943824901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3c00d1c20 msgr2=0x7fe3c015c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:40.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.225+0000 7fe3bea00640 1 --2- 172.21.15.86:0/3943824901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3c00d1c20 0x7fe3c015c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.225+0000 7fe3bea00640 1 -- 172.21.15.86:0/3943824901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c00ae1d0 con 0x7fe3c0070330 2024-04-25T14:04:40.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.229+0000 7fe3bea00640 1 --2- 172.21.15.86:0/3943824901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c0070330 0x7fe3c01578a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00d5c0 tx=0x7fe3ac00da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:40.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.229+0000 7fe3b7400640 1 -- 172.21.15.86:0/3943824901 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac004240 con 0x7fe3c0070330 2024-04-25T14:04:40.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.229+0000 7fe3b7400640 1 -- 172.21.15.86:0/3943824901 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3ac0043e0 con 0x7fe3c0070330 2024-04-25T14:04:40.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.229+0000 7fe3c6400640 1 -- 172.21.15.86:0/3943824901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c00741b0 con 0x7fe3c0070330 2024-04-25T14:04:40.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.229+0000 7fe3b7400640 1 -- 172.21.15.86:0/3943824901 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac003840 con 0x7fe3c0070330 2024-04-25T14:04:40.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.229+0000 7fe3c6400640 1 -- 172.21.15.86:0/3943824901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c00d0800 con 0x7fe3c0070330 2024-04-25T14:04:40.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.229+0000 7fe3c6400640 1 -- 172.21.15.86:0/3943824901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3c0069d00 con 0x7fe3c0070330 2024-04-25T14:04:40.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.229+0000 7fe3b7400640 1 -- 172.21.15.86:0/3943824901 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe3ac0039e0 con 0x7fe3c0070330 2024-04-25T14:04:40.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.229+0000 7fe3b7400640 1 --2- 172.21.15.86:0/3943824901 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe3a006f400 0x7fe3a0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:40.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.229+0000 7fe3b7400640 1 -- 172.21.15.86:0/3943824901 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe3ac08f750 con 0x7fe3c0070330 2024-04-25T14:04:40.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.229+0000 7fe3bf400640 1 --2- 172.21.15.86:0/3943824901 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe3a006f400 0x7fe3a0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:40.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.233+0000 7fe3bf400640 1 --2- 172.21.15.86:0/3943824901 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe3a006f400 0x7fe3a0071910 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe3b00ad000 tx=0x7fe3b00a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:40.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.233+0000 7fe3b7400640 1 -- 172.21.15.86:0/3943824901 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe3ac05bc80 con 0x7fe3c0070330 2024-04-25T14:04:40.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.449+0000 7fe3c6400640 1 -- 172.21.15.86:0/3943824901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_client_metrics_and_metadata"]} v 0) -- 0x7fe3c00d0b00 con 0x7fe3c0070330 2024-04-25T14:04:40.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.977+0000 7fe3b7400640 1 -- 172.21.15.86:0/3943824901 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_client_metrics_and_metadata"]}]=0 v74) ==== 159+0+0 (secure 0 0 0) 0x7fe3ac05ff80 con 0x7fe3c0070330 2024-04-25T14:04:40.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.981+0000 7fe3c6400640 1 -- 172.21.15.86:0/3943824901 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe3a006f400 msgr2=0x7fe3a0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:40.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.981+0000 7fe3c6400640 1 --2- 172.21.15.86:0/3943824901 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe3a006f400 0x7fe3a0071910 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe3b00ad000 tx=0x7fe3b00a6000 comp rx=0 tx=0).stop 2024-04-25T14:04:40.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.981+0000 7fe3c6400640 1 -- 172.21.15.86:0/3943824901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c0070330 msgr2=0x7fe3c01578a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:40.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.981+0000 7fe3c6400640 1 --2- 172.21.15.86:0/3943824901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c0070330 0x7fe3c01578a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00d5c0 tx=0x7fe3ac00da80 comp rx=0 tx=0).stop 2024-04-25T14:04:40.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.981+0000 7fe3c6400640 1 -- 172.21.15.86:0/3943824901 shutdown_connections 2024-04-25T14:04:40.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.981+0000 7fe3c6400640 1 --2- 172.21.15.86:0/3943824901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3c00d1c20 0x7fe3c015c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.981+0000 7fe3c6400640 1 --2- 172.21.15.86:0/3943824901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c0070330 0x7fe3c01578a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.981+0000 7fe3c6400640 1 --2- 172.21.15.86:0/3943824901 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe3a006f400 0x7fe3a0071910 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.981+0000 7fe3c6400640 1 --2- 172.21.15.86:0/3943824901 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3c006f270 0x7fe3c00ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.981+0000 7fe3c6400640 1 -- 172.21.15.86:0/3943824901 >> 172.21.15.86:0/3943824901 conn(0x7fe3c00b4650 msgr2=0x7fe3c006f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:40.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.981+0000 7fe3c6400640 1 -- 172.21.15.86:0/3943824901 shutdown_connections 2024-04-25T14:04:40.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:40.981+0000 7fe3c6400640 1 -- 172.21.15.86:0/3943824901 wait complete. 2024-04-25T14:04:41.003 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T14:04:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.149+0000 7f781ee00640 1 Processor -- start 2024-04-25T14:04:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.149+0000 7f781ee00640 1 -- start start 2024-04-25T14:04:41.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.153+0000 7f781ee00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f78180cfca0 0x7f781806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.153+0000 7f781ee00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f78180730b0 0x7f78180734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.153+0000 7f781ee00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7818072bd0 con 0x7f78180d16e0 2024-04-25T14:04:41.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.153+0000 7f781ee00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7818070470 con 0x7f78180cfca0 2024-04-25T14:04:41.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.153+0000 7f781ee00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f781806f930 con 0x7f78180730b0 2024-04-25T14:04:41.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.153+0000 7f7817e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f78180cfca0 0x7f781806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.153+0000 7f7817e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f78180cfca0 0x7f781806f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60650/0 (socket says 172.21.15.86:60650) 2024-04-25T14:04:41.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.153+0000 7f7817e00640 1 -- 172.21.15.86:0/2977074802 learned_addr learned my addr 172.21.15.86:0/2977074802 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:41.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.153+0000 7f781ce00640 1 --2- 172.21.15.86:0/2977074802 >> v2:172.21.15.155:3300/0 conn(0x7f78180730b0 0x7f78180734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.153+0000 7f7817e00640 1 -- 172.21.15.86:0/2977074802 >> v2:172.21.15.155:3300/0 conn(0x7f78180730b0 msgr2=0x7f78180734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:41.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.153+0000 7f7817e00640 1 --2- 172.21.15.86:0/2977074802 >> v2:172.21.15.155:3300/0 conn(0x7f78180730b0 0x7f78180734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.153+0000 7f7817e00640 1 -- 172.21.15.86:0/2977074802 >> v1:172.21.15.155:6789/0 conn(0x7f78180d16e0 legacy=0x7f78180d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:41.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.153+0000 7f7817e00640 1 -- 172.21.15.86:0/2977074802 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781806fbd0 con 0x7f78180cfca0 2024-04-25T14:04:41.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.157+0000 7f7817e00640 1 --2- 172.21.15.86:0/2977074802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78180cfca0 0x7f781806f3f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f78180cb8c0 tx=0x7f780801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec19e4cb7faf386 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:41.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.157+0000 7f7817400640 1 -- 172.21.15.86:0/2977074802 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78080a5780 con 0x7f78180cfca0 2024-04-25T14:04:41.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.157+0000 7f7817400640 1 -- 172.21.15.86:0/2977074802 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78080a5920 con 0x7f78180cfca0 2024-04-25T14:04:41.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.157+0000 7f7817400640 1 -- 172.21.15.86:0/2977074802 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78080a5cf0 con 0x7f78180cfca0 2024-04-25T14:04:41.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.157+0000 7f781ee00640 1 -- 172.21.15.86:0/2977074802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78180cfca0 msgr2=0x7f781806f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:41.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.157+0000 7f781ee00640 1 --2- 172.21.15.86:0/2977074802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78180cfca0 0x7f781806f3f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f78180cb8c0 tx=0x7f780801c9b0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.157+0000 7f781ee00640 1 -- 172.21.15.86:0/2977074802 shutdown_connections 2024-04-25T14:04:41.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.157+0000 7f781ee00640 1 --2- 172.21.15.86:0/2977074802 >> v2:172.21.15.155:3300/0 conn(0x7f78180730b0 0x7f78180734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.157+0000 7f781ee00640 1 --2- 172.21.15.86:0/2977074802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78180cfca0 0x7f781806f3f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.157+0000 7f781ee00640 1 -- 172.21.15.86:0/2977074802 >> 172.21.15.86:0/2977074802 conn(0x7f78180b4650 msgr2=0x7f78180b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:41.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.157+0000 7f781ee00640 1 -- 172.21.15.86:0/2977074802 shutdown_connections 2024-04-25T14:04:41.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.157+0000 7f781ee00640 1 -- 172.21.15.86:0/2977074802 wait complete. 2024-04-25T14:04:41.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.161+0000 7f781ee00640 1 Processor -- start 2024-04-25T14:04:41.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.161+0000 7f781ee00640 1 -- start start 2024-04-25T14:04:41.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.161+0000 7f781ee00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78180730b0 0x7f78180aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.161+0000 7f781ee00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78180cfca0 0x7f78180af120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.161+0000 7f781ee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78180d16e0 0x7f781815da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.161+0000 7f781ee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7818169a70 con 0x7f78180d16e0 2024-04-25T14:04:41.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.161+0000 7f781ee00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f78181698f0 con 0x7f78180cfca0 2024-04-25T14:04:41.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.161+0000 7f781ee00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7818169bf0 con 0x7f78180730b0 2024-04-25T14:04:41.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.161+0000 7f781ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78180d16e0 0x7f781815da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.161+0000 7f781ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78180d16e0 0x7f781815da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60658/0 (socket says 172.21.15.86:60658) 2024-04-25T14:04:41.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.161+0000 7f781ce00640 1 -- 172.21.15.86:0/3548214669 learned_addr learned my addr 172.21.15.86:0/3548214669 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:41.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.161+0000 7f7817e00640 1 --2- 172.21.15.86:0/3548214669 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78180730b0 0x7f78180aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.161+0000 7f780f400640 1 --2- 172.21.15.86:0/3548214669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78180cfca0 0x7f78180af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.161+0000 7f781ce00640 1 -- 172.21.15.86:0/3548214669 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78180730b0 msgr2=0x7f78180aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:41.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.161+0000 7f781ce00640 1 --2- 172.21.15.86:0/3548214669 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78180730b0 0x7f78180aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.161+0000 7f781ce00640 1 -- 172.21.15.86:0/3548214669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78180cfca0 msgr2=0x7f78180af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:41.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.161+0000 7f781ce00640 1 --2- 172.21.15.86:0/3548214669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78180cfca0 0x7f78180af120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.161+0000 7f781ce00640 1 -- 172.21.15.86:0/3548214669 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781809aec0 con 0x7f78180d16e0 2024-04-25T14:04:41.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.161+0000 7f781ce00640 1 --2- 172.21.15.86:0/3548214669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78180d16e0 0x7f781815da30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f781000d550 tx=0x7f781000da10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:41.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.161+0000 7f7814c00640 1 -- 172.21.15.86:0/3548214669 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78100125f0 con 0x7f78180d16e0 2024-04-25T14:04:41.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.161+0000 7f7814c00640 1 -- 172.21.15.86:0/3548214669 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7810019070 con 0x7f78180d16e0 2024-04-25T14:04:41.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.161+0000 7f7814c00640 1 -- 172.21.15.86:0/3548214669 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78100129d0 con 0x7f78180d16e0 2024-04-25T14:04:41.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.161+0000 7f781ee00640 1 -- 172.21.15.86:0/3548214669 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78180c9c30 con 0x7f78180d16e0 2024-04-25T14:04:41.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.161+0000 7f781ee00640 1 -- 172.21.15.86:0/3548214669 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78180ca1b0 con 0x7f78180d16e0 2024-04-25T14:04:41.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.165+0000 7f781ee00640 1 -- 172.21.15.86:0/3548214669 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f781809f4b0 con 0x7f78180d16e0 2024-04-25T14:04:41.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.165+0000 7f7814c00640 1 -- 172.21.15.86:0/3548214669 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7810012b70 con 0x7f78180d16e0 2024-04-25T14:04:41.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.165+0000 7f7814c00640 1 --2- 172.21.15.86:0/3548214669 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f77f406f400 0x7f77f4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.165+0000 7f7814c00640 1 -- 172.21.15.86:0/3548214669 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f781008f9b0 con 0x7f78180d16e0 2024-04-25T14:04:41.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.165+0000 7f7817e00640 1 --2- 172.21.15.86:0/3548214669 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f77f406f400 0x7f77f4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.165+0000 7f7817e00640 1 --2- 172.21.15.86:0/3548214669 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f77f406f400 0x7f77f4071910 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f78080ae000 tx=0x7f78080a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:41.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.169+0000 7f7814c00640 1 -- 172.21.15.86:0/3548214669 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f781809aec0 con 0x7f78180d16e0 2024-04-25T14:04:41.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.333+0000 7f781ee00640 1 -- 172.21.15.86:0/3548214669 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f7818069d00 con 0x7f78180d16e0 2024-04-25T14:04:41.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.337+0000 7f7814c00640 1 -- 172.21.15.86:0/3548214669 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f781005bdb0 con 0x7f78180d16e0 2024-04-25T14:04:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.337+0000 7f781ee00640 1 -- 172.21.15.86:0/3548214669 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f77f406f400 msgr2=0x7f77f4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.337+0000 7f781ee00640 1 --2- 172.21.15.86:0/3548214669 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f77f406f400 0x7f77f4071910 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f78080ae000 tx=0x7f78080a7070 comp rx=0 tx=0).stop 2024-04-25T14:04:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.337+0000 7f781ee00640 1 -- 172.21.15.86:0/3548214669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78180d16e0 msgr2=0x7f781815da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.337+0000 7f781ee00640 1 --2- 172.21.15.86:0/3548214669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78180d16e0 0x7f781815da30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f781000d550 tx=0x7f781000da10 comp rx=0 tx=0).stop 2024-04-25T14:04:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.337+0000 7f781ee00640 1 -- 172.21.15.86:0/3548214669 shutdown_connections 2024-04-25T14:04:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.337+0000 7f781ee00640 1 --2- 172.21.15.86:0/3548214669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78180d16e0 0x7f781815da30 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.337+0000 7f781ee00640 1 --2- 172.21.15.86:0/3548214669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78180cfca0 0x7f78180af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.337+0000 7f781ee00640 1 --2- 172.21.15.86:0/3548214669 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f77f406f400 0x7f77f4071910 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.337+0000 7f781ee00640 1 --2- 172.21.15.86:0/3548214669 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78180730b0 0x7f78180aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.337+0000 7f781ee00640 1 -- 172.21.15.86:0/3548214669 >> 172.21.15.86:0/3548214669 conn(0x7f78180b4650 msgr2=0x7f78180b2430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.341+0000 7f781ee00640 1 -- 172.21.15.86:0/3548214669 shutdown_connections 2024-04-25T14:04:41.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.341+0000 7f781ee00640 1 -- 172.21.15.86:0/3548214669 wait complete. 2024-04-25T14:04:41.360 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T14:04:41.360 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:04:41.459 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:04:41.551 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T14:04:41.551 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:04:41.551 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:04:41.560 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:04:41.560 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:04:41.608 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:04:41.608 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:04:41.656 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:04:41.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.845+0000 7f8d4e800640 1 Processor -- start 2024-04-25T14:04:41.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.845+0000 7f8d4e800640 1 -- start start 2024-04-25T14:04:41.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.845+0000 7f8d4e800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8d48072d40 0x7f8d480d1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.845+0000 7f8d4e800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8d480b6840 0x7f8d480b6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.845+0000 7f8d4e800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d480d2440 con 0x7f8d4815de40 2024-04-25T14:04:41.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.845+0000 7f8d4e800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8d480d2720 con 0x7f8d480b6840 2024-04-25T14:04:41.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.845+0000 7f8d4e800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f8d480d2a00 con 0x7f8d48072d40 2024-04-25T14:04:41.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.845+0000 7f8d47e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8d480b6840 0x7f8d480b6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.845+0000 7f8d47e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8d480b6840 0x7f8d480b6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60664/0 (socket says 172.21.15.86:60664) 2024-04-25T14:04:41.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.845+0000 7f8d47400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8d4815de40 0x7f8d48072630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54866/0 (socket says 172.21.15.86:54866) 2024-04-25T14:04:41.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.845+0000 7f8d47e00640 1 -- 172.21.15.86:0/636767344 learned_addr learned my addr 172.21.15.86:0/636767344 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:41.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.845+0000 7f8d3ea00640 1 --2- 172.21.15.86:0/636767344 >> v2:172.21.15.155:3300/0 conn(0x7f8d48072d40 0x7f8d480d1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.849+0000 7f8d47e00640 1 -- 172.21.15.86:0/636767344 >> v2:172.21.15.155:3300/0 conn(0x7f8d48072d40 msgr2=0x7f8d480d1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:41.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.849+0000 7f8d47e00640 1 --2- 172.21.15.86:0/636767344 >> v2:172.21.15.155:3300/0 conn(0x7f8d48072d40 0x7f8d480d1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.849+0000 7f8d47e00640 1 -- 172.21.15.86:0/636767344 >> v1:172.21.15.86:6789/0 conn(0x7f8d4815de40 legacy=0x7f8d48072630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:41.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.849+0000 7f8d47e00640 1 -- 172.21.15.86:0/636767344 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d480d2ce0 con 0x7f8d480b6840 2024-04-25T14:04:41.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.849+0000 7f8d46a00640 1 -- 172.21.15.86:0/636767344 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 168615025 0 0) 0x7f8d480d2440 con 0x7f8d4815de40 2024-04-25T14:04:41.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.849+0000 7f8d47e00640 1 --2- 172.21.15.86:0/636767344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d480b6840 0x7f8d480b6c50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8d48071eb0 tx=0x7f8d4001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c542c5aa30a008 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:41.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.849+0000 7f8d46a00640 1 -- 172.21.15.86:0/636767344 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d400a57d0 con 0x7f8d480b6840 2024-04-25T14:04:41.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.849+0000 7f8d46a00640 1 -- 172.21.15.86:0/636767344 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d400a5970 con 0x7f8d480b6840 2024-04-25T14:04:41.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.849+0000 7f8d46a00640 1 -- 172.21.15.86:0/636767344 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d400a5d40 con 0x7f8d480b6840 2024-04-25T14:04:41.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.849+0000 7f8d4e800640 1 -- 172.21.15.86:0/636767344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d480b6840 msgr2=0x7f8d480b6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:41.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.849+0000 7f8d4e800640 1 --2- 172.21.15.86:0/636767344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d480b6840 0x7f8d480b6c50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8d48071eb0 tx=0x7f8d4001c9f0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.853+0000 7f8d4e800640 1 -- 172.21.15.86:0/636767344 shutdown_connections 2024-04-25T14:04:41.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.853+0000 7f8d4e800640 1 --2- 172.21.15.86:0/636767344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d480b6840 0x7f8d480b6c50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.853+0000 7f8d4e800640 1 --2- 172.21.15.86:0/636767344 >> v2:172.21.15.155:3300/0 conn(0x7f8d48072d40 0x7f8d480d1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.853+0000 7f8d4e800640 1 -- 172.21.15.86:0/636767344 >> 172.21.15.86:0/636767344 conn(0x7f8d480d3580 msgr2=0x7f8d480d3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:41.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.853+0000 7f8d4e800640 1 -- 172.21.15.86:0/636767344 shutdown_connections 2024-04-25T14:04:41.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.853+0000 7f8d4e800640 1 -- 172.21.15.86:0/636767344 wait complete. 2024-04-25T14:04:41.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.853+0000 7f8d4e800640 1 Processor -- start 2024-04-25T14:04:41.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.853+0000 7f8d4e800640 1 -- start start 2024-04-25T14:04:41.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.853+0000 7f8d4e800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d480b6840 0x7f8d4806f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.853+0000 7f8d4e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d4815de40 0x7f8d48159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.853+0000 7f8d4e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d481f8e20 0x7f8d4815a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.853+0000 7f8d4e800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4816b4b0 con 0x7f8d481f8e20 2024-04-25T14:04:41.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.853+0000 7f8d4e800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4816b330 con 0x7f8d4815de40 2024-04-25T14:04:41.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.853+0000 7f8d4e800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8d4816b630 con 0x7f8d480b6840 2024-04-25T14:04:41.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.857+0000 7f8d47e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d481f8e20 0x7f8d4815a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.857+0000 7f8d47e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d481f8e20 0x7f8d4815a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60674/0 (socket says 172.21.15.86:60674) 2024-04-25T14:04:41.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.857+0000 7f8d47e00640 1 -- 172.21.15.86:0/2445689069 learned_addr learned my addr 172.21.15.86:0/2445689069 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:41.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.857+0000 7f8d47400640 1 --2- 172.21.15.86:0/2445689069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d480b6840 0x7f8d4806f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.857+0000 7f8d3ea00640 1 --2- 172.21.15.86:0/2445689069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d4815de40 0x7f8d48159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.857+0000 7f8d47e00640 1 -- 172.21.15.86:0/2445689069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d480b6840 msgr2=0x7f8d4806f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:41.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.857+0000 7f8d47e00640 1 --2- 172.21.15.86:0/2445689069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d480b6840 0x7f8d4806f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.857+0000 7f8d47e00640 1 -- 172.21.15.86:0/2445689069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d4815de40 msgr2=0x7f8d48159ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:41.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.857+0000 7f8d47e00640 1 --2- 172.21.15.86:0/2445689069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d4815de40 0x7f8d48159ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.857+0000 7f8d47e00640 1 -- 172.21.15.86:0/2445689069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d480cb400 con 0x7f8d481f8e20 2024-04-25T14:04:41.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.857+0000 7f8d3ea00640 1 --2- 172.21.15.86:0/2445689069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d4815de40 0x7f8d48159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:41.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.857+0000 7f8d47e00640 1 --2- 172.21.15.86:0/2445689069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d481f8e20 0x7f8d4815a020 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8d400af040 tx=0x7f8d40019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:41.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.857+0000 7f8d3fe00640 1 -- 172.21.15.86:0/2445689069 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d400b8880 con 0x7f8d481f8e20 2024-04-25T14:04:41.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.857+0000 7f8d3fe00640 1 -- 172.21.15.86:0/2445689069 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d400be070 con 0x7f8d481f8e20 2024-04-25T14:04:41.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.857+0000 7f8d3fe00640 1 -- 172.21.15.86:0/2445689069 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d400b8c60 con 0x7f8d481f8e20 2024-04-25T14:04:41.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.857+0000 7f8d4e800640 1 -- 172.21.15.86:0/2445689069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d480cb6e0 con 0x7f8d481f8e20 2024-04-25T14:04:41.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.857+0000 7f8d4e800640 1 -- 172.21.15.86:0/2445689069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d480cbca0 con 0x7f8d481f8e20 2024-04-25T14:04:41.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.857+0000 7f8d4e800640 1 -- 172.21.15.86:0/2445689069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d48072170 con 0x7f8d481f8e20 2024-04-25T14:04:41.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.857+0000 7f8d3fe00640 1 -- 172.21.15.86:0/2445689069 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8d400c2410 con 0x7f8d481f8e20 2024-04-25T14:04:41.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.861+0000 7f8d3fe00640 1 --2- 172.21.15.86:0/2445689069 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8d2406f430 0x7f8d24071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.861+0000 7f8d3fe00640 1 -- 172.21.15.86:0/2445689069 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f8d4012a370 con 0x7f8d481f8e20 2024-04-25T14:04:41.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.861+0000 7f8d47400640 1 --2- 172.21.15.86:0/2445689069 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8d2406f430 0x7f8d24071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.861+0000 7f8d47400640 1 --2- 172.21.15.86:0/2445689069 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8d2406f430 0x7f8d24071940 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8d38000f70 tx=0x7f8d38007c30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:41.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:41.865+0000 7f8d3fe00640 1 -- 172.21.15.86:0/2445689069 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8d400c7020 con 0x7f8d481f8e20 2024-04-25T14:04:42.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.025+0000 7f8d4e800640 1 -- 172.21.15.86:0/2445689069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8d48076bd0 con 0x7f8d481f8e20 2024-04-25T14:04:42.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.025+0000 7f8d3fe00640 1 -- 172.21.15.86:0/2445689069 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18902 (secure 0 0 0) 0x7f8d400165e0 con 0x7f8d481f8e20 2024-04-25T14:04:42.028 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:42.028 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":26,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:04:31.930803+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:03:49.151473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:03:49.802597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T14:03:53.492008+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:04:42.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.029+0000 7f8d4e800640 1 -- 172.21.15.86:0/2445689069 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8d2406f430 msgr2=0x7f8d24071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.029+0000 7f8d4e800640 1 --2- 172.21.15.86:0/2445689069 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8d2406f430 0x7f8d24071940 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8d38000f70 tx=0x7f8d38007c30 comp rx=0 tx=0).stop 2024-04-25T14:04:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.029+0000 7f8d4e800640 1 -- 172.21.15.86:0/2445689069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d481f8e20 msgr2=0x7f8d4815a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.029+0000 7f8d4e800640 1 --2- 172.21.15.86:0/2445689069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d481f8e20 0x7f8d4815a020 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8d400af040 tx=0x7f8d40019790 comp rx=0 tx=0).stop 2024-04-25T14:04:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.029+0000 7f8d4e800640 1 -- 172.21.15.86:0/2445689069 shutdown_connections 2024-04-25T14:04:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.029+0000 7f8d4e800640 1 --2- 172.21.15.86:0/2445689069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d481f8e20 0x7f8d4815a020 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.029+0000 7f8d4e800640 1 --2- 172.21.15.86:0/2445689069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d4815de40 0x7f8d48159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.029+0000 7f8d4e800640 1 --2- 172.21.15.86:0/2445689069 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8d2406f430 0x7f8d24071940 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.029+0000 7f8d4e800640 1 --2- 172.21.15.86:0/2445689069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d480b6840 0x7f8d4806f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.029+0000 7f8d4e800640 1 -- 172.21.15.86:0/2445689069 >> 172.21.15.86:0/2445689069 conn(0x7f8d480d3580 msgr2=0x7f8d48071250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.029+0000 7f8d4e800640 1 -- 172.21.15.86:0/2445689069 shutdown_connections 2024-04-25T14:04:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.029+0000 7f8d4e800640 1 -- 172.21.15.86:0/2445689069 wait complete. 2024-04-25T14:04:42.050 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:03:49.151473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:03:49.802597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T14:03:53.492008+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T14:04:42.051 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:04:42.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.197+0000 7ff77ea00640 1 Processor -- start 2024-04-25T14:04:42.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.197+0000 7ff77ea00640 1 -- start start 2024-04-25T14:04:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.201+0000 7ff77ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff7780ceef0 0x7ff7780702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.201+0000 7ff77ea00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff778070830 0x7ff7780b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.201+0000 7ff77ea00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7780b6b30 0x7ff7780b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.201+0000 7ff77ea00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff77815dc10 con 0x7ff7780ceef0 2024-04-25T14:04:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.201+0000 7ff77ea00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff7780b7480 con 0x7ff778070830 2024-04-25T14:04:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.201+0000 7ff77ea00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff77815d780 con 0x7ff7780b6b30 2024-04-25T14:04:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.201+0000 7ff77d600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff7780ceef0 0x7ff7780702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.201+0000 7ff77d600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff7780ceef0 0x7ff7780702f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60676/0 (socket says 172.21.15.86:60676) 2024-04-25T14:04:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.201+0000 7ff77d600640 1 -- 172.21.15.86:0/105492210 learned_addr learned my addr 172.21.15.86:0/105492210 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.201+0000 7ff77cc00640 1 --2- 172.21.15.86:0/105492210 >> v2:172.21.15.155:3300/0 conn(0x7ff778070830 0x7ff7780b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.201+0000 7ff77e000640 1 --2- 172.21.15.86:0/105492210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7780b6b30 0x7ff7780b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.201+0000 7ff77d600640 1 -- 172.21.15.86:0/105492210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7780b6b30 msgr2=0x7ff7780b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.201+0000 7ff77d600640 1 --2- 172.21.15.86:0/105492210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7780b6b30 0x7ff7780b6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.201+0000 7ff77d600640 1 -- 172.21.15.86:0/105492210 >> v2:172.21.15.155:3300/0 conn(0x7ff778070830 msgr2=0x7ff7780b65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.201+0000 7ff77d600640 1 --2- 172.21.15.86:0/105492210 >> v2:172.21.15.155:3300/0 conn(0x7ff778070830 0x7ff7780b65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.201+0000 7ff77d600640 1 -- 172.21.15.86:0/105492210 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7780b13e0 con 0x7ff7780ceef0 2024-04-25T14:04:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.205+0000 7ff77d600640 1 --2- 172.21.15.86:0/105492210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7780ceef0 0x7ff7780702f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff778074010 tx=0x7ff76c01eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51fed9621ce95708 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:42.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.205+0000 7ff777e00640 1 -- 172.21.15.86:0/105492210 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c0b4070 con 0x7ff7780ceef0 2024-04-25T14:04:42.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.205+0000 7ff777e00640 1 -- 172.21.15.86:0/105492210 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff76c0a7970 con 0x7ff7780ceef0 2024-04-25T14:04:42.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.205+0000 7ff777e00640 1 -- 172.21.15.86:0/105492210 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c0a7d40 con 0x7ff7780ceef0 2024-04-25T14:04:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.205+0000 7ff77ea00640 1 -- 172.21.15.86:0/105492210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7780ceef0 msgr2=0x7ff7780702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.205+0000 7ff77ea00640 1 --2- 172.21.15.86:0/105492210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7780ceef0 0x7ff7780702f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff778074010 tx=0x7ff76c01eb60 comp rx=0 tx=0).stop 2024-04-25T14:04:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.205+0000 7ff77ea00640 1 -- 172.21.15.86:0/105492210 shutdown_connections 2024-04-25T14:04:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.205+0000 7ff77ea00640 1 --2- 172.21.15.86:0/105492210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7780b6b30 0x7ff7780b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.205+0000 7ff77ea00640 1 --2- 172.21.15.86:0/105492210 >> v2:172.21.15.155:3300/0 conn(0x7ff778070830 0x7ff7780b65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.205+0000 7ff77ea00640 1 --2- 172.21.15.86:0/105492210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7780ceef0 0x7ff7780702f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.205+0000 7ff77ea00640 1 -- 172.21.15.86:0/105492210 >> 172.21.15.86:0/105492210 conn(0x7ff778073ab0 msgr2=0x7ff77806c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.205+0000 7ff77ea00640 1 -- 172.21.15.86:0/105492210 shutdown_connections 2024-04-25T14:04:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.205+0000 7ff77ea00640 1 -- 172.21.15.86:0/105492210 wait complete. 2024-04-25T14:04:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.209+0000 7ff77ea00640 1 Processor -- start 2024-04-25T14:04:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.209+0000 7ff77ea00640 1 -- start start 2024-04-25T14:04:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.209+0000 7ff77ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff778070830 0x7ff7781444b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.209+0000 7ff77ea00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7780b6b30 0x7ff778156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.209+0000 7ff77ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7780ceef0 0x7ff778156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.209+0000 7ff77ea00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7780cf5b0 con 0x7ff778070830 2024-04-25T14:04:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.209+0000 7ff77ea00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff77815d410 con 0x7ff7780ceef0 2024-04-25T14:04:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.209+0000 7ff77ea00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff7780b4ad0 con 0x7ff7780b6b30 2024-04-25T14:04:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.209+0000 7ff77d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff778070830 0x7ff7781444b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.209+0000 7ff77d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff778070830 0x7ff7781444b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60678/0 (socket says 172.21.15.86:60678) 2024-04-25T14:04:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.209+0000 7ff77d600640 1 -- 172.21.15.86:0/3849330452 learned_addr learned my addr 172.21.15.86:0/3849330452 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.209+0000 7ff77e000640 1 --2- 172.21.15.86:0/3849330452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7780ceef0 0x7ff778156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.209+0000 7ff77cc00640 1 --2- 172.21.15.86:0/3849330452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7780b6b30 0x7ff778156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.209+0000 7ff77d600640 1 -- 172.21.15.86:0/3849330452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7780b6b30 msgr2=0x7ff778156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.209+0000 7ff77d600640 1 --2- 172.21.15.86:0/3849330452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7780b6b30 0x7ff778156220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.209+0000 7ff77d600640 1 -- 172.21.15.86:0/3849330452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7780ceef0 msgr2=0x7ff778156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.209+0000 7ff77d600640 1 --2- 172.21.15.86:0/3849330452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7780ceef0 0x7ff778156760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.209+0000 7ff77d600640 1 -- 172.21.15.86:0/3849330452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff778146a30 con 0x7ff778070830 2024-04-25T14:04:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.209+0000 7ff77cc00640 1 --2- 172.21.15.86:0/3849330452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7780b6b30 0x7ff778156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.209+0000 7ff77d600640 1 --2- 172.21.15.86:0/3849330452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff778070830 0x7ff7781444b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff76c01f400 tx=0x7ff76c01b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.209+0000 7ff775600640 1 -- 172.21.15.86:0/3849330452 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c0b4040 con 0x7ff778070830 2024-04-25T14:04:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.209+0000 7ff775600640 1 -- 172.21.15.86:0/3849330452 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff76c0c1070 con 0x7ff778070830 2024-04-25T14:04:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.209+0000 7ff775600640 1 -- 172.21.15.86:0/3849330452 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c0af980 con 0x7ff778070830 2024-04-25T14:04:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.209+0000 7ff77ea00640 1 -- 172.21.15.86:0/3849330452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff778145450 con 0x7ff778070830 2024-04-25T14:04:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.209+0000 7ff77ea00640 1 -- 172.21.15.86:0/3849330452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff778145a10 con 0x7ff778070830 2024-04-25T14:04:42.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.213+0000 7ff77ea00640 1 -- 172.21.15.86:0/3849330452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff778076b80 con 0x7ff778070830 2024-04-25T14:04:42.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.213+0000 7ff775600640 1 -- 172.21.15.86:0/3849330452 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff76c0b7070 con 0x7ff778070830 2024-04-25T14:04:42.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.213+0000 7ff775600640 1 --2- 172.21.15.86:0/3849330452 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff75806f400 0x7ff758071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:42.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.213+0000 7ff775600640 1 -- 172.21.15.86:0/3849330452 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff76c12c300 con 0x7ff778070830 2024-04-25T14:04:42.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.213+0000 7ff77cc00640 1 --2- 172.21.15.86:0/3849330452 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff75806f400 0x7ff758071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:42.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.213+0000 7ff77cc00640 1 --2- 172.21.15.86:0/3849330452 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff75806f400 0x7ff758071910 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff7780b3730 tx=0x7ff764008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.217+0000 7ff775600640 1 -- 172.21.15.86:0/3849330452 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff7780b13e0 con 0x7ff778070830 2024-04-25T14:04:42.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.373+0000 7ff77ea00640 1 -- 172.21.15.86:0/3849330452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff778167410 con 0x7ff778070830 2024-04-25T14:04:42.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.373+0000 7ff775600640 1 -- 172.21.15.86:0/3849330452 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7ff76c0187e0 con 0x7ff778070830 2024-04-25T14:04:42.376 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:04:42.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.377+0000 7ff77ea00640 1 -- 172.21.15.86:0/3849330452 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff75806f400 msgr2=0x7ff758071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:42.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.377+0000 7ff77ea00640 1 --2- 172.21.15.86:0/3849330452 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff75806f400 0x7ff758071910 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff7780b3730 tx=0x7ff764008040 comp rx=0 tx=0).stop 2024-04-25T14:04:42.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.377+0000 7ff77ea00640 1 -- 172.21.15.86:0/3849330452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff778070830 msgr2=0x7ff7781444b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:42.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.377+0000 7ff77ea00640 1 --2- 172.21.15.86:0/3849330452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff778070830 0x7ff7781444b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff76c01f400 tx=0x7ff76c01b900 comp rx=0 tx=0).stop 2024-04-25T14:04:42.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.377+0000 7ff77ea00640 1 -- 172.21.15.86:0/3849330452 shutdown_connections 2024-04-25T14:04:42.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.377+0000 7ff77ea00640 1 --2- 172.21.15.86:0/3849330452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7780ceef0 0x7ff778156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.377+0000 7ff77ea00640 1 --2- 172.21.15.86:0/3849330452 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff75806f400 0x7ff758071910 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.377+0000 7ff77ea00640 1 --2- 172.21.15.86:0/3849330452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7780b6b30 0x7ff778156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.377+0000 7ff77ea00640 1 --2- 172.21.15.86:0/3849330452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff778070830 0x7ff7781444b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.377+0000 7ff77ea00640 1 -- 172.21.15.86:0/3849330452 >> 172.21.15.86:0/3849330452 conn(0x7ff778073ab0 msgr2=0x7ff7780d0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:42.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.377+0000 7ff77ea00640 1 -- 172.21.15.86:0/3849330452 shutdown_connections 2024-04-25T14:04:42.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.377+0000 7ff77ea00640 1 -- 172.21.15.86:0/3849330452 wait complete. 2024-04-25T14:04:42.397 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:04:42.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.545+0000 7f14ce200640 1 Processor -- start 2024-04-25T14:04:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.545+0000 7f14ce200640 1 -- start start 2024-04-25T14:04:42.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.545+0000 7f14ce200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f14c80740c0 0x7f14c81676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:42.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.545+0000 7f14ce200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14c8072bd0 con 0x7f14c80cfca0 2024-04-25T14:04:42.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.545+0000 7f14ce200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14c806fbd0 con 0x7f14c80d16e0 2024-04-25T14:04:42.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.545+0000 7f14ce200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f14c8070470 con 0x7f14c80740c0 2024-04-25T14:04:42.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.545+0000 7f14c7400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f14c80cfca0 0x7f14c806f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54880/0 (socket says 172.21.15.86:54880) 2024-04-25T14:04:42.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.545+0000 7f14c7400640 1 -- 172.21.15.86:0/2819809715 learned_addr learned my addr 172.21.15.86:0/2819809715 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:42.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.545+0000 7f14c7e00640 1 --2- 172.21.15.86:0/2819809715 >> v2:172.21.15.86:3300/0 conn(0x7f14c80740c0 0x7f14c81676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:42.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.549+0000 7f14c6000640 1 -- 172.21.15.86:0/2819809715 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 315352558 0 0) 0x7f14c8072bd0 con 0x7f14c80cfca0 2024-04-25T14:04:42.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.549+0000 7f14c6000640 1 -- 172.21.15.86:0/2819809715 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14ac005740 con 0x7f14c80cfca0 2024-04-25T14:04:42.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.549+0000 7f14c7e00640 1 -- 172.21.15.86:0/2819809715 >> v1:172.21.15.155:6789/0 conn(0x7f14c80d16e0 legacy=0x7f14c80d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:42.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.549+0000 7f14c7e00640 1 -- 172.21.15.86:0/2819809715 >> v1:172.21.15.86:6789/0 conn(0x7f14c80cfca0 legacy=0x7f14c806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:42.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.549+0000 7f14c7e00640 1 -- 172.21.15.86:0/2819809715 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c806feb0 con 0x7f14c80740c0 2024-04-25T14:04:42.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.549+0000 7f14c7e00640 1 --2- 172.21.15.86:0/2819809715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14c80740c0 0x7f14c81676d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f14c00097c0 tx=0x7f14c001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468dbc306c46867d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:42.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.549+0000 7f14c6000640 1 -- 172.21.15.86:0/2819809715 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c00b0070 con 0x7f14c80740c0 2024-04-25T14:04:42.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.549+0000 7f14c6000640 1 -- 172.21.15.86:0/2819809715 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14c00ab600 con 0x7f14c80740c0 2024-04-25T14:04:42.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.549+0000 7f14c6000640 1 -- 172.21.15.86:0/2819809715 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c00ab9d0 con 0x7f14c80740c0 2024-04-25T14:04:42.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.549+0000 7f14ce200640 1 -- 172.21.15.86:0/2819809715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14c80740c0 msgr2=0x7f14c81676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:42.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.549+0000 7f14ce200640 1 --2- 172.21.15.86:0/2819809715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14c80740c0 0x7f14c81676d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f14c00097c0 tx=0x7f14c001c4a0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.553+0000 7f14ce200640 1 -- 172.21.15.86:0/2819809715 shutdown_connections 2024-04-25T14:04:42.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.553+0000 7f14ce200640 1 --2- 172.21.15.86:0/2819809715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14c80740c0 0x7f14c81676d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.553+0000 7f14ce200640 1 -- 172.21.15.86:0/2819809715 >> 172.21.15.86:0/2819809715 conn(0x7f14c80b4650 msgr2=0x7f14c80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:42.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.553+0000 7f14ce200640 1 -- 172.21.15.86:0/2819809715 shutdown_connections 2024-04-25T14:04:42.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.553+0000 7f14ce200640 1 -- 172.21.15.86:0/2819809715 wait complete. 2024-04-25T14:04:42.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.553+0000 7f14ce200640 1 Processor -- start 2024-04-25T14:04:42.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.553+0000 7f14ce200640 1 -- start start 2024-04-25T14:04:42.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.553+0000 7f14ce200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14c80740c0 0x7f14c8158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:42.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.553+0000 7f14ce200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14c80cfca0 0x7f14c815b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:42.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.553+0000 7f14ce200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14c80d16e0 0x7f14c815b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:42.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.553+0000 7f14ce200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14c80ce900 con 0x7f14c80740c0 2024-04-25T14:04:42.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.553+0000 7f14ce200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f14c80ce780 con 0x7f14c80d16e0 2024-04-25T14:04:42.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.553+0000 7f14ce200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f14c80cea80 con 0x7f14c80cfca0 2024-04-25T14:04:42.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.557+0000 7f14c7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14c80740c0 0x7f14c8158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:42.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.557+0000 7f14c7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14c80740c0 0x7f14c8158c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60696/0 (socket says 172.21.15.86:60696) 2024-04-25T14:04:42.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.557+0000 7f14c7400640 1 -- 172.21.15.86:0/2022978056 learned_addr learned my addr 172.21.15.86:0/2022978056 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:42.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.557+0000 7f14c7e00640 1 --2- 172.21.15.86:0/2022978056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14c80d16e0 0x7f14c815b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:42.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.557+0000 7f14c7400640 1 -- 172.21.15.86:0/2022978056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14c80cfca0 msgr2=0x7f14c815b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:42.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.557+0000 7f14c6a00640 1 --2- 172.21.15.86:0/2022978056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14c80cfca0 0x7f14c815b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:42.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.557+0000 7f14c7400640 1 --2- 172.21.15.86:0/2022978056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14c80cfca0 0x7f14c815b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.557+0000 7f14c7400640 1 -- 172.21.15.86:0/2022978056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14c80d16e0 msgr2=0x7f14c815b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:42.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.557+0000 7f14c7400640 1 --2- 172.21.15.86:0/2022978056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14c80d16e0 0x7f14c815b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.557+0000 7f14c7400640 1 -- 172.21.15.86:0/2022978056 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c815c450 con 0x7f14c80740c0 2024-04-25T14:04:42.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.557+0000 7f14c7e00640 1 --2- 172.21.15.86:0/2022978056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14c80d16e0 0x7f14c815b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:42.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.557+0000 7f14c6a00640 1 --2- 172.21.15.86:0/2022978056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14c80cfca0 0x7f14c815b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:42.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.557+0000 7f14c7400640 1 --2- 172.21.15.86:0/2022978056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14c80740c0 0x7f14c8158c00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f14b800cae0 tx=0x7f14b800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:42.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.557+0000 7f14bf400640 1 -- 172.21.15.86:0/2022978056 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b8017070 con 0x7f14c80740c0 2024-04-25T14:04:42.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.557+0000 7f14bf400640 1 -- 172.21.15.86:0/2022978056 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14b800ac40 con 0x7f14c80740c0 2024-04-25T14:04:42.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.557+0000 7f14ce200640 1 -- 172.21.15.86:0/2022978056 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c815d290 con 0x7f14c80740c0 2024-04-25T14:04:42.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.557+0000 7f14bf400640 1 -- 172.21.15.86:0/2022978056 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b8012620 con 0x7f14c80740c0 2024-04-25T14:04:42.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.557+0000 7f14ce200640 1 -- 172.21.15.86:0/2022978056 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c815eb10 con 0x7f14c80740c0 2024-04-25T14:04:42.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.557+0000 7f14ce200640 1 -- 172.21.15.86:0/2022978056 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14c815ed70 con 0x7f14c80740c0 2024-04-25T14:04:42.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.557+0000 7f14bf400640 1 -- 172.21.15.86:0/2022978056 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f14b801a070 con 0x7f14c80740c0 2024-04-25T14:04:42.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.561+0000 7f14bf400640 1 --2- 172.21.15.86:0/2022978056 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f14a806f400 0x7f14a8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:42.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.561+0000 7f14bf400640 1 -- 172.21.15.86:0/2022978056 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f14b808f480 con 0x7f14c80740c0 2024-04-25T14:04:42.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.561+0000 7f14c6a00640 1 --2- 172.21.15.86:0/2022978056 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f14a806f400 0x7f14a8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:42.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.561+0000 7f14c6a00640 1 --2- 172.21.15.86:0/2022978056 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f14a806f400 0x7f14a8071910 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f14c80cbcd0 tx=0x7f14b4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:42.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.565+0000 7f14bf400640 1 -- 172.21.15.86:0/2022978056 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f14b805bf60 con 0x7f14c80740c0 2024-04-25T14:04:42.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.721+0000 7f14ce200640 1 -- 172.21.15.86:0/2022978056 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f14c815c620 con 0x7f14c80740c0 2024-04-25T14:04:42.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.721+0000 7f14bf400640 1 -- 172.21.15.86:0/2022978056 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f14b805b860 con 0x7f14c80740c0 2024-04-25T14:04:42.722 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:04:42.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.721+0000 7f14ce200640 1 -- 172.21.15.86:0/2022978056 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f14a806f400 msgr2=0x7f14a8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:42.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.721+0000 7f14ce200640 1 --2- 172.21.15.86:0/2022978056 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f14a806f400 0x7f14a8071910 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f14c80cbcd0 tx=0x7f14b4009040 comp rx=0 tx=0).stop 2024-04-25T14:04:42.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.721+0000 7f14ce200640 1 -- 172.21.15.86:0/2022978056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14c80740c0 msgr2=0x7f14c8158c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:42.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.721+0000 7f14ce200640 1 --2- 172.21.15.86:0/2022978056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14c80740c0 0x7f14c8158c00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f14b800cae0 tx=0x7f14b800a060 comp rx=0 tx=0).stop 2024-04-25T14:04:42.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.721+0000 7f14ce200640 1 -- 172.21.15.86:0/2022978056 shutdown_connections 2024-04-25T14:04:42.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.721+0000 7f14ce200640 1 --2- 172.21.15.86:0/2022978056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14c80d16e0 0x7f14c815b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.721+0000 7f14ce200640 1 --2- 172.21.15.86:0/2022978056 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f14a806f400 0x7f14a8071910 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.721+0000 7f14ce200640 1 --2- 172.21.15.86:0/2022978056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14c80cfca0 0x7f14c815b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.721+0000 7f14ce200640 1 --2- 172.21.15.86:0/2022978056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14c80740c0 0x7f14c8158c00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.721+0000 7f14ce200640 1 -- 172.21.15.86:0/2022978056 >> 172.21.15.86:0/2022978056 conn(0x7f14c80b4650 msgr2=0x7f14c80b0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:42.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.725+0000 7f14ce200640 1 -- 172.21.15.86:0/2022978056 shutdown_connections 2024-04-25T14:04:42.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.725+0000 7f14ce200640 1 -- 172.21.15.86:0/2022978056 wait complete. 2024-04-25T14:04:42.745 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:04:42.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.889+0000 7fd069000640 1 Processor -- start 2024-04-25T14:04:42.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.893+0000 7fd069000640 1 -- start start 2024-04-25T14:04:42.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.893+0000 7fd069000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd064070d60 0x7fd064071170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:42.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.893+0000 7fd069000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd064161450 0x7fd064161860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:42.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.893+0000 7fd069000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd0640cdac0 0x7fd0640cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:42.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.893+0000 7fd069000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd064162420 con 0x7fd064070d60 2024-04-25T14:04:42.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.893+0000 7fd069000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd0640ca220 con 0x7fd0640cdac0 2024-04-25T14:04:42.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.893+0000 7fd069000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd0640716b0 con 0x7fd064161450 2024-04-25T14:04:42.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.893+0000 7fd063400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd064070d60 0x7fd064071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:42.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.893+0000 7fd063400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd064070d60 0x7fd064071170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60702/0 (socket says 172.21.15.86:60702) 2024-04-25T14:04:42.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.893+0000 7fd063400640 1 -- 172.21.15.86:0/1174889529 learned_addr learned my addr 172.21.15.86:0/1174889529 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:42.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.893+0000 7fd062a00640 1 --2- 172.21.15.86:0/1174889529 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd064161450 0x7fd064161860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:42.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.893+0000 7fd063e00640 1 --2- 172.21.15.86:0/1174889529 >> v2:172.21.15.155:3300/0 conn(0x7fd0640cdac0 0x7fd0640cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:42.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.893+0000 7fd063400640 1 -- 172.21.15.86:0/1174889529 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd064161450 msgr2=0x7fd064161860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:42.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.893+0000 7fd063400640 1 --2- 172.21.15.86:0/1174889529 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd064161450 0x7fd064161860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.893+0000 7fd063400640 1 -- 172.21.15.86:0/1174889529 >> v2:172.21.15.155:3300/0 conn(0x7fd0640cdac0 msgr2=0x7fd0640cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:42.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.893+0000 7fd063400640 1 --2- 172.21.15.86:0/1174889529 >> v2:172.21.15.155:3300/0 conn(0x7fd0640cdac0 0x7fd0640cded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.893+0000 7fd063400640 1 -- 172.21.15.86:0/1174889529 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd064072440 con 0x7fd064070d60 2024-04-25T14:04:42.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.893+0000 7fd062a00640 1 --2- 172.21.15.86:0/1174889529 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd064161450 0x7fd064161860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:42.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.897+0000 7fd063400640 1 --2- 172.21.15.86:0/1174889529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd064070d60 0x7fd064071170 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd0640742a0 tx=0x7fd05401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e4801e979f5aa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:42.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.897+0000 7fd062000640 1 -- 172.21.15.86:0/1174889529 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0540a57e0 con 0x7fd064070d60 2024-04-25T14:04:42.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.897+0000 7fd062000640 1 -- 172.21.15.86:0/1174889529 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0540a5980 con 0x7fd064070d60 2024-04-25T14:04:42.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.897+0000 7fd062000640 1 -- 172.21.15.86:0/1174889529 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0540a5d50 con 0x7fd064070d60 2024-04-25T14:04:42.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.897+0000 7fd069000640 1 -- 172.21.15.86:0/1174889529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd064070d60 msgr2=0x7fd064071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:42.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.897+0000 7fd069000640 1 --2- 172.21.15.86:0/1174889529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd064070d60 0x7fd064071170 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd0640742a0 tx=0x7fd05401c960 comp rx=0 tx=0).stop 2024-04-25T14:04:42.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.897+0000 7fd069000640 1 -- 172.21.15.86:0/1174889529 shutdown_connections 2024-04-25T14:04:42.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.897+0000 7fd069000640 1 --2- 172.21.15.86:0/1174889529 >> v2:172.21.15.155:3300/0 conn(0x7fd0640cdac0 0x7fd0640cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.897+0000 7fd069000640 1 --2- 172.21.15.86:0/1174889529 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd064161450 0x7fd064161860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.897+0000 7fd069000640 1 --2- 172.21.15.86:0/1174889529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd064070d60 0x7fd064071170 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.897+0000 7fd069000640 1 -- 172.21.15.86:0/1174889529 >> 172.21.15.86:0/1174889529 conn(0x7fd0640d3330 msgr2=0x7fd0640d3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:42.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.897+0000 7fd069000640 1 -- 172.21.15.86:0/1174889529 shutdown_connections 2024-04-25T14:04:42.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.897+0000 7fd069000640 1 -- 172.21.15.86:0/1174889529 wait complete. 2024-04-25T14:04:42.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.901+0000 7fd069000640 1 Processor -- start 2024-04-25T14:04:42.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.901+0000 7fd069000640 1 -- start start 2024-04-25T14:04:42.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.901+0000 7fd069000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd064070d60 0x7fd0640b7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:42.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.901+0000 7fd069000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0640cdac0 0x7fd0641f8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:42.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.901+0000 7fd069000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd064161450 0x7fd0641fd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:42.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.901+0000 7fd069000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd064161f20 con 0x7fd0640cdac0 2024-04-25T14:04:42.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.901+0000 7fd069000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd064161da0 con 0x7fd064161450 2024-04-25T14:04:42.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.901+0000 7fd069000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd0640ce820 con 0x7fd064070d60 2024-04-25T14:04:42.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.901+0000 7fd062a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0640cdac0 0x7fd0641f8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:42.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.901+0000 7fd062a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0640cdac0 0x7fd0641f8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60708/0 (socket says 172.21.15.86:60708) 2024-04-25T14:04:42.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.901+0000 7fd062a00640 1 -- 172.21.15.86:0/602898840 learned_addr learned my addr 172.21.15.86:0/602898840 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:42.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.901+0000 7fd063400640 1 --2- 172.21.15.86:0/602898840 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd064070d60 0x7fd0640b7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:42.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.901+0000 7fd063e00640 1 --2- 172.21.15.86:0/602898840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd064161450 0x7fd0641fd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:42.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.901+0000 7fd062a00640 1 -- 172.21.15.86:0/602898840 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd064070d60 msgr2=0x7fd0640b7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:42.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.901+0000 7fd062a00640 1 --2- 172.21.15.86:0/602898840 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd064070d60 0x7fd0640b7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.901+0000 7fd062a00640 1 -- 172.21.15.86:0/602898840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd064161450 msgr2=0x7fd0641fd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:42.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.901+0000 7fd062a00640 1 --2- 172.21.15.86:0/602898840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd064161450 0x7fd0641fd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:42.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.901+0000 7fd062a00640 1 -- 172.21.15.86:0/602898840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0641fc970 con 0x7fd0640cdac0 2024-04-25T14:04:42.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.901+0000 7fd063e00640 1 --2- 172.21.15.86:0/602898840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd064161450 0x7fd0641fd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:42.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.901+0000 7fd062a00640 1 --2- 172.21.15.86:0/602898840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0640cdac0 0x7fd0641f8c40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd05000d860 tx=0x7fd05000dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:42.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.901+0000 7fd05b400640 1 -- 172.21.15.86:0/602898840 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05000de50 con 0x7fd0640cdac0 2024-04-25T14:04:42.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.901+0000 7fd05b400640 1 -- 172.21.15.86:0/602898840 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd050019070 con 0x7fd0640cdac0 2024-04-25T14:04:42.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.901+0000 7fd069000640 1 -- 172.21.15.86:0/602898840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0640b7ce0 con 0x7fd0640cdac0 2024-04-25T14:04:42.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.901+0000 7fd05b400640 1 -- 172.21.15.86:0/602898840 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05000bab0 con 0x7fd0640cdac0 2024-04-25T14:04:42.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.901+0000 7fd069000640 1 -- 172.21.15.86:0/602898840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0640b82a0 con 0x7fd0640cdac0 2024-04-25T14:04:42.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.905+0000 7fd069000640 1 -- 172.21.15.86:0/602898840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd064076b80 con 0x7fd0640cdac0 2024-04-25T14:04:42.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.905+0000 7fd05b400640 1 -- 172.21.15.86:0/602898840 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd05000bc50 con 0x7fd0640cdac0 2024-04-25T14:04:42.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.905+0000 7fd05b400640 1 --2- 172.21.15.86:0/602898840 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd04406f430 0x7fd044071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:42.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.905+0000 7fd05b400640 1 -- 172.21.15.86:0/602898840 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd05008f8e0 con 0x7fd0640cdac0 2024-04-25T14:04:42.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.905+0000 7fd063400640 1 --2- 172.21.15.86:0/602898840 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd04406f430 0x7fd044071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:42.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.905+0000 7fd063400640 1 --2- 172.21.15.86:0/602898840 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd04406f430 0x7fd044071940 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd0540ae000 tx=0x7fd0540a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:42.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:42.909+0000 7fd05b400640 1 -- 172.21.15.86:0/602898840 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd05005c460 con 0x7fd0640cdac0 2024-04-25T14:04:43.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.069+0000 7fd069000640 1 -- 172.21.15.86:0/602898840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd0640c83a0 con 0x7fd0640cdac0 2024-04-25T14:04:43.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.069+0000 7fd05b400640 1 -- 172.21.15.86:0/602898840 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fd05005bd60 con 0x7fd0640cdac0 2024-04-25T14:04:43.069 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:04:43.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.069+0000 7fd069000640 1 -- 172.21.15.86:0/602898840 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd04406f430 msgr2=0x7fd044071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:43.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.069+0000 7fd069000640 1 --2- 172.21.15.86:0/602898840 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd04406f430 0x7fd044071940 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd0540ae000 tx=0x7fd0540a7070 comp rx=0 tx=0).stop 2024-04-25T14:04:43.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.069+0000 7fd069000640 1 -- 172.21.15.86:0/602898840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0640cdac0 msgr2=0x7fd0641f8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:43.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.069+0000 7fd069000640 1 --2- 172.21.15.86:0/602898840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0640cdac0 0x7fd0641f8c40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd05000d860 tx=0x7fd05000dd20 comp rx=0 tx=0).stop 2024-04-25T14:04:43.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.069+0000 7fd069000640 1 -- 172.21.15.86:0/602898840 shutdown_connections 2024-04-25T14:04:43.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.069+0000 7fd069000640 1 --2- 172.21.15.86:0/602898840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd064161450 0x7fd0641fd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.069+0000 7fd069000640 1 --2- 172.21.15.86:0/602898840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0640cdac0 0x7fd0641f8c40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.069+0000 7fd069000640 1 --2- 172.21.15.86:0/602898840 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd04406f430 0x7fd044071940 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.069+0000 7fd069000640 1 --2- 172.21.15.86:0/602898840 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd064070d60 0x7fd0640b7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.069+0000 7fd069000640 1 -- 172.21.15.86:0/602898840 >> 172.21.15.86:0/602898840 conn(0x7fd0640d3330 msgr2=0x7fd0640b3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:43.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.069+0000 7fd069000640 1 -- 172.21.15.86:0/602898840 shutdown_connections 2024-04-25T14:04:43.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.069+0000 7fd069000640 1 -- 172.21.15.86:0/602898840 wait complete. 2024-04-25T14:04:43.091 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T14:04:43.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.237+0000 7f9ea5200640 1 Processor -- start 2024-04-25T14:04:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.237+0000 7f9ea5200640 1 -- start start 2024-04-25T14:04:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.237+0000 7f9ea5200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9ea015faa0 0x7f9ea006d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.237+0000 7f9ea5200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ea00aeea0 0x7f9ea00af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.237+0000 7f9ea5200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ea006c670 con 0x7f9ea00b59e0 2024-04-25T14:04:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.237+0000 7f9ea5200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9ea006d560 con 0x7f9ea015faa0 2024-04-25T14:04:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.237+0000 7f9ea5200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9ea006c950 con 0x7f9ea00aeea0 2024-04-25T14:04:43.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.241+0000 7f9e9e800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ea00aeea0 0x7f9ea00af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.241+0000 7f9e9de00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9ea015faa0 0x7f9ea006d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.241+0000 7f9e9e800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ea00aeea0 0x7f9ea00af2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:39176/0 (socket says 172.21.15.86:39176) 2024-04-25T14:04:43.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.241+0000 7f9e9e800640 1 -- 172.21.15.86:0/2910049352 learned_addr learned my addr 172.21.15.86:0/2910049352 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:43.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.241+0000 7f9e9d400640 1 -- 172.21.15.86:0/2910049352 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3775869774 0 0) 0x7f9ea006c670 con 0x7f9ea00b59e0 2024-04-25T14:04:43.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.241+0000 7f9e9d400640 1 -- 172.21.15.86:0/2910049352 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e84003530 con 0x7f9ea00b59e0 2024-04-25T14:04:43.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.241+0000 7f9e9e800640 1 -- 172.21.15.86:0/2910049352 >> v2:172.21.15.155:3300/0 conn(0x7f9ea015faa0 msgr2=0x7f9ea006d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:43.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.241+0000 7f9e9e800640 1 --2- 172.21.15.86:0/2910049352 >> v2:172.21.15.155:3300/0 conn(0x7f9ea015faa0 0x7f9ea006d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.241+0000 7f9e9e800640 1 -- 172.21.15.86:0/2910049352 >> v1:172.21.15.155:6789/0 conn(0x7f9ea00b59e0 legacy=0x7f9ea00b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:43.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.241+0000 7f9e9e800640 1 -- 172.21.15.86:0/2910049352 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea006cc30 con 0x7f9ea00aeea0 2024-04-25T14:04:43.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.241+0000 7f9e9de00640 1 --2- 172.21.15.86:0/2910049352 >> v2:172.21.15.155:3300/0 conn(0x7f9ea015faa0 0x7f9ea006d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:43.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.241+0000 7f9e9e800640 1 --2- 172.21.15.86:0/2910049352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ea00aeea0 0x7f9ea00af2b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009e50 tx=0x7f9e9801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12f9bb0cc6e80de0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:43.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.241+0000 7f9e9d400640 1 -- 172.21.15.86:0/2910049352 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e980a5900 con 0x7f9ea00aeea0 2024-04-25T14:04:43.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.241+0000 7f9e9d400640 1 -- 172.21.15.86:0/2910049352 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e980a5aa0 con 0x7f9ea00aeea0 2024-04-25T14:04:43.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.241+0000 7f9e9d400640 1 -- 172.21.15.86:0/2910049352 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e980a5e50 con 0x7f9ea00aeea0 2024-04-25T14:04:43.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.245+0000 7f9ea5200640 1 -- 172.21.15.86:0/2910049352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ea00aeea0 msgr2=0x7f9ea00af2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:43.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.245+0000 7f9ea5200640 1 --2- 172.21.15.86:0/2910049352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ea00aeea0 0x7f9ea00af2b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009e50 tx=0x7f9e9801ca30 comp rx=0 tx=0).stop 2024-04-25T14:04:43.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.245+0000 7f9ea5200640 1 -- 172.21.15.86:0/2910049352 shutdown_connections 2024-04-25T14:04:43.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.245+0000 7f9ea5200640 1 --2- 172.21.15.86:0/2910049352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ea00aeea0 0x7f9ea00af2b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.245+0000 7f9ea5200640 1 --2- 172.21.15.86:0/2910049352 >> v2:172.21.15.155:3300/0 conn(0x7f9ea015faa0 0x7f9ea006d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.245+0000 7f9ea5200640 1 -- 172.21.15.86:0/2910049352 >> 172.21.15.86:0/2910049352 conn(0x7f9ea00d2230 msgr2=0x7f9ea00d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:43.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.245+0000 7f9ea5200640 1 -- 172.21.15.86:0/2910049352 shutdown_connections 2024-04-25T14:04:43.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.245+0000 7f9ea5200640 1 -- 172.21.15.86:0/2910049352 wait complete. 2024-04-25T14:04:43.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.245+0000 7f9ea5200640 1 Processor -- start 2024-04-25T14:04:43.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.245+0000 7f9ea5200640 1 -- start start 2024-04-25T14:04:43.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.249+0000 7f9ea5200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ea00aeea0 0x7f9ea013e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.249+0000 7f9ea5200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ea00b59e0 0x7f9ea013e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.249+0000 7f9ea5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ea015faa0 0x7f9ea01410a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.249+0000 7f9ea5200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea0160030 con 0x7f9ea015faa0 2024-04-25T14:04:43.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.249+0000 7f9ea5200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea015feb0 con 0x7f9ea00aeea0 2024-04-25T14:04:43.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.249+0000 7f9ea5200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9ea0070f30 con 0x7f9ea00b59e0 2024-04-25T14:04:43.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.249+0000 7f9e9e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ea015faa0 0x7f9ea01410a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.249+0000 7f9e9e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ea015faa0 0x7f9ea01410a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60710/0 (socket says 172.21.15.86:60710) 2024-04-25T14:04:43.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.249+0000 7f9e9e800640 1 -- 172.21.15.86:0/2049970055 learned_addr learned my addr 172.21.15.86:0/2049970055 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:43.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.249+0000 7f9e9de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ea00aeea0 0x7f9ea013e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.249+0000 7f9e97e00640 1 --2- 172.21.15.86:0/2049970055 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ea00b59e0 0x7f9ea013e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.249+0000 7f9e9de00640 1 -- 172.21.15.86:0/2049970055 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ea00b59e0 msgr2=0x7f9ea013e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:43.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.249+0000 7f9e9de00640 1 --2- 172.21.15.86:0/2049970055 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ea00b59e0 0x7f9ea013e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.249+0000 7f9e9de00640 1 -- 172.21.15.86:0/2049970055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ea015faa0 msgr2=0x7f9ea01410a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:43.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.249+0000 7f9e9de00640 1 --2- 172.21.15.86:0/2049970055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ea015faa0 0x7f9ea01410a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.249+0000 7f9e9de00640 1 -- 172.21.15.86:0/2049970055 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea0160250 con 0x7f9ea00aeea0 2024-04-25T14:04:43.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.249+0000 7f9e9e800640 1 --2- 172.21.15.86:0/2049970055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ea015faa0 0x7f9ea01410a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:43.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.249+0000 7f9e9de00640 1 --2- 172.21.15.86:0/2049970055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ea00aeea0 0x7f9ea013e180 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000e8e0 tx=0x7f9e9000eda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.249+0000 7f9e96000640 1 -- 172.21.15.86:0/2049970055 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90017070 con 0x7f9ea00aeea0 2024-04-25T14:04:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.249+0000 7f9e96000640 1 -- 172.21.15.86:0/2049970055 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e9000c8e0 con 0x7f9ea00aeea0 2024-04-25T14:04:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.249+0000 7f9ea5200640 1 -- 172.21.15.86:0/2049970055 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea0162020 con 0x7f9ea00aeea0 2024-04-25T14:04:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.249+0000 7f9e96000640 1 -- 172.21.15.86:0/2049970055 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9000ccd0 con 0x7f9ea00aeea0 2024-04-25T14:04:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.249+0000 7f9ea5200640 1 -- 172.21.15.86:0/2049970055 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea01604f0 con 0x7f9ea00aeea0 2024-04-25T14:04:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.249+0000 7f9ea5200640 1 -- 172.21.15.86:0/2049970055 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ea0074cf0 con 0x7f9ea00aeea0 2024-04-25T14:04:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.253+0000 7f9e96000640 1 -- 172.21.15.86:0/2049970055 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9e9001a070 con 0x7f9ea00aeea0 2024-04-25T14:04:43.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.253+0000 7f9e96000640 1 --2- 172.21.15.86:0/2049970055 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9e7c06f400 0x7f9e7c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.253+0000 7f9e96000640 1 -- 172.21.15.86:0/2049970055 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f9e9008fdb0 con 0x7f9ea00aeea0 2024-04-25T14:04:43.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.253+0000 7f9e97e00640 1 --2- 172.21.15.86:0/2049970055 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9e7c06f400 0x7f9e7c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.253+0000 7f9e97e00640 1 --2- 172.21.15.86:0/2049970055 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9e7c06f400 0x7f9e7c071910 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9ea006da70 tx=0x7f9e88009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:43.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.257+0000 7f9e96000640 1 -- 172.21.15.86:0/2049970055 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9e9005c230 con 0x7f9ea00aeea0 2024-04-25T14:04:43.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.413+0000 7f9ea5200640 1 -- 172.21.15.86:0/2049970055 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9ea015b6c0 con 0x7f9ea00aeea0 2024-04-25T14:04:43.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.413+0000 7f9e96000640 1 -- 172.21.15.86:0/2049970055 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f9e90060530 con 0x7f9ea00aeea0 2024-04-25T14:04:43.416 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:04:43.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.417+0000 7f9ea5200640 1 -- 172.21.15.86:0/2049970055 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9e7c06f400 msgr2=0x7f9e7c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:43.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.417+0000 7f9ea5200640 1 --2- 172.21.15.86:0/2049970055 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9e7c06f400 0x7f9e7c071910 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9ea006da70 tx=0x7f9e88009040 comp rx=0 tx=0).stop 2024-04-25T14:04:43.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.417+0000 7f9ea5200640 1 -- 172.21.15.86:0/2049970055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ea00aeea0 msgr2=0x7f9ea013e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:43.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.417+0000 7f9ea5200640 1 --2- 172.21.15.86:0/2049970055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ea00aeea0 0x7f9ea013e180 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000e8e0 tx=0x7f9e9000eda0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.417+0000 7f9ea5200640 1 -- 172.21.15.86:0/2049970055 shutdown_connections 2024-04-25T14:04:43.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.417+0000 7f9ea5200640 1 --2- 172.21.15.86:0/2049970055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ea015faa0 0x7f9ea01410a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.417+0000 7f9ea5200640 1 --2- 172.21.15.86:0/2049970055 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f9e7c06f400 0x7f9e7c071910 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.417+0000 7f9ea5200640 1 --2- 172.21.15.86:0/2049970055 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ea00b59e0 0x7f9ea013e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.417+0000 7f9ea5200640 1 --2- 172.21.15.86:0/2049970055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ea00aeea0 0x7f9ea013e180 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.417+0000 7f9ea5200640 1 -- 172.21.15.86:0/2049970055 >> 172.21.15.86:0/2049970055 conn(0x7f9ea00d2230 msgr2=0x7f9ea00b2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:43.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.417+0000 7f9ea5200640 1 -- 172.21.15.86:0/2049970055 shutdown_connections 2024-04-25T14:04:43.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.417+0000 7f9ea5200640 1 -- 172.21.15.86:0/2049970055 wait complete. 2024-04-25T14:04:43.437 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T14:04:43.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.585+0000 7f675d200640 1 Processor -- start 2024-04-25T14:04:43.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.585+0000 7f675d200640 1 -- start start 2024-04-25T14:04:43.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.589+0000 7f675d200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67580d1df0 0x7f67580b0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.589+0000 7f675d200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f67580b0690 0x7f67580d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.589+0000 7f675d200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67580cde20 con 0x7f67581642a0 2024-04-25T14:04:43.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.589+0000 7f675d200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f67580cea30 con 0x7f67580b0690 2024-04-25T14:04:43.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.589+0000 7f675d200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f67580ce100 con 0x7f67580d1df0 2024-04-25T14:04:43.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.589+0000 7f6756800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f67580b0690 0x7f67580d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.589+0000 7f6755e00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f67581642a0 0x7f67580d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:58052/0 (socket says 172.21.15.86:58052) 2024-04-25T14:04:43.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.589+0000 7f674d400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67580d1df0 0x7f67580b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.589+0000 7f6755e00640 1 -- 172.21.15.86:0/1847857571 learned_addr learned my addr 172.21.15.86:0/1847857571 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:43.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.589+0000 7f6755400640 1 -- 172.21.15.86:0/1847857571 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1766954742 0 0) 0x7f67580cde20 con 0x7f67581642a0 2024-04-25T14:04:43.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.589+0000 7f6755400640 1 -- 172.21.15.86:0/1847857571 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f673c003530 con 0x7f67581642a0 2024-04-25T14:04:43.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.589+0000 7f6756800640 1 -- 172.21.15.86:0/1847857571 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67580d1df0 msgr2=0x7f67580b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:43.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.589+0000 7f6756800640 1 --2- 172.21.15.86:0/1847857571 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67580d1df0 0x7f67580b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.589+0000 7f6756800640 1 -- 172.21.15.86:0/1847857571 >> v1:172.21.15.155:6789/0 conn(0x7f67581642a0 legacy=0x7f67580d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:43.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.589+0000 7f6756800640 1 -- 172.21.15.86:0/1847857571 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67580ce3e0 con 0x7f67580b0690 2024-04-25T14:04:43.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.589+0000 7f674d400640 1 --2- 172.21.15.86:0/1847857571 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67580d1df0 0x7f67580b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:43.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.593+0000 7f6756800640 1 --2- 172.21.15.86:0/1847857571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67580b0690 0x7f67580d0640 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6750009d90 tx=0x7f675001c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54b2dd680301958a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:43.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.593+0000 7f6755400640 1 -- 172.21.15.86:0/1847857571 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67500b2070 con 0x7f67580b0690 2024-04-25T14:04:43.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.593+0000 7f6755400640 1 -- 172.21.15.86:0/1847857571 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67500a57a0 con 0x7f67580b0690 2024-04-25T14:04:43.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.593+0000 7f6755400640 1 -- 172.21.15.86:0/1847857571 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67500a5b70 con 0x7f67580b0690 2024-04-25T14:04:43.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.593+0000 7f675d200640 1 -- 172.21.15.86:0/1847857571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67580b0690 msgr2=0x7f67580d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:43.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.593+0000 7f675d200640 1 --2- 172.21.15.86:0/1847857571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67580b0690 0x7f67580d0640 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6750009d90 tx=0x7f675001c8e0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.593+0000 7f675d200640 1 -- 172.21.15.86:0/1847857571 shutdown_connections 2024-04-25T14:04:43.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.593+0000 7f675d200640 1 --2- 172.21.15.86:0/1847857571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67580b0690 0x7f67580d0640 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.593+0000 7f675d200640 1 --2- 172.21.15.86:0/1847857571 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67580d1df0 0x7f67580b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.593+0000 7f675d200640 1 -- 172.21.15.86:0/1847857571 >> 172.21.15.86:0/1847857571 conn(0x7f67580b4650 msgr2=0x7f67580b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:43.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.593+0000 7f675d200640 1 -- 172.21.15.86:0/1847857571 shutdown_connections 2024-04-25T14:04:43.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.593+0000 7f675d200640 1 -- 172.21.15.86:0/1847857571 wait complete. 2024-04-25T14:04:43.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.593+0000 7f675d200640 1 Processor -- start 2024-04-25T14:04:43.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.597+0000 7f675d200640 1 -- start start 2024-04-25T14:04:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.597+0000 7f675d200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67580b0690 0x7f6758159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.597+0000 7f675d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67581642a0 0x7f675815b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.597+0000 7f675d200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6758159c50 0x7f675815b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.597+0000 7f675d200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6758164830 con 0x7f67581642a0 2024-04-25T14:04:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.597+0000 7f675d200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f67581646b0 con 0x7f67580b0690 2024-04-25T14:04:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.597+0000 7f675d200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6758160da0 con 0x7f6758159c50 2024-04-25T14:04:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.597+0000 7f674d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67581642a0 0x7f675815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.597+0000 7f674d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67581642a0 0x7f675815b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60720/0 (socket says 172.21.15.86:60720) 2024-04-25T14:04:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.597+0000 7f674d400640 1 -- 172.21.15.86:0/2087153650 learned_addr learned my addr 172.21.15.86:0/2087153650 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.597+0000 7f6755e00640 1 --2- 172.21.15.86:0/2087153650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67580b0690 0x7f6758159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.597+0000 7f6756800640 1 --2- 172.21.15.86:0/2087153650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6758159c50 0x7f675815b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.597+0000 7f674d400640 1 -- 172.21.15.86:0/2087153650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6758159c50 msgr2=0x7f675815b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.597+0000 7f674d400640 1 --2- 172.21.15.86:0/2087153650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6758159c50 0x7f675815b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.597+0000 7f674d400640 1 -- 172.21.15.86:0/2087153650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67580b0690 msgr2=0x7f6758159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.597+0000 7f674d400640 1 --2- 172.21.15.86:0/2087153650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67580b0690 0x7f6758159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.597+0000 7f674d400640 1 -- 172.21.15.86:0/2087153650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6758073480 con 0x7f67581642a0 2024-04-25T14:04:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.597+0000 7f674d400640 1 --2- 172.21.15.86:0/2087153650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67581642a0 0x7f675815b280 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f674000e980 tx=0x7f674000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.597+0000 7f674ea00640 1 -- 172.21.15.86:0/2087153650 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6740017070 con 0x7f67581642a0 2024-04-25T14:04:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.597+0000 7f674ea00640 1 -- 172.21.15.86:0/2087153650 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f674000ca00 con 0x7f67581642a0 2024-04-25T14:04:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.597+0000 7f675d200640 1 -- 172.21.15.86:0/2087153650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6758072a90 con 0x7f67581642a0 2024-04-25T14:04:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.597+0000 7f674ea00640 1 -- 172.21.15.86:0/2087153650 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674000cdd0 con 0x7f67581642a0 2024-04-25T14:04:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.597+0000 7f675d200640 1 -- 172.21.15.86:0/2087153650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f675815fd90 con 0x7f67581642a0 2024-04-25T14:04:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.597+0000 7f675d200640 1 -- 172.21.15.86:0/2087153650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6758069d00 con 0x7f67581642a0 2024-04-25T14:04:43.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.601+0000 7f674ea00640 1 -- 172.21.15.86:0/2087153650 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f674001a070 con 0x7f67581642a0 2024-04-25T14:04:43.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.601+0000 7f674ea00640 1 --2- 172.21.15.86:0/2087153650 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f673406f400 0x7f6734071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.601+0000 7f674ea00640 1 -- 172.21.15.86:0/2087153650 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f674008f470 con 0x7f67581642a0 2024-04-25T14:04:43.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.601+0000 7f6755e00640 1 --2- 172.21.15.86:0/2087153650 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f673406f400 0x7f6734071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.601+0000 7f6755e00640 1 --2- 172.21.15.86:0/2087153650 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f673406f400 0x7f6734071910 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6748007fd0 tx=0x7f6748009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:43.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.605+0000 7f674ea00640 1 -- 172.21.15.86:0/2087153650 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f674005b990 con 0x7f67581642a0 2024-04-25T14:04:43.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.777+0000 7f675d200640 1 -- 172.21.15.86:0/2087153650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f67580cafd0 con 0x7f67581642a0 2024-04-25T14:04:43.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.777+0000 7f674ea00640 1 -- 172.21.15.86:0/2087153650 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f6740060e20 con 0x7f67581642a0 2024-04-25T14:04:43.779 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-25T14:04:43.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.777+0000 7f675d200640 1 -- 172.21.15.86:0/2087153650 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f673406f400 msgr2=0x7f6734071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:43.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.777+0000 7f675d200640 1 --2- 172.21.15.86:0/2087153650 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f673406f400 0x7f6734071910 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6748007fd0 tx=0x7f6748009040 comp rx=0 tx=0).stop 2024-04-25T14:04:43.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.777+0000 7f675d200640 1 -- 172.21.15.86:0/2087153650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67581642a0 msgr2=0x7f675815b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:43.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.777+0000 7f675d200640 1 --2- 172.21.15.86:0/2087153650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67581642a0 0x7f675815b280 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f674000e980 tx=0x7f674000ee40 comp rx=0 tx=0).stop 2024-04-25T14:04:43.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.777+0000 7f675d200640 1 -- 172.21.15.86:0/2087153650 shutdown_connections 2024-04-25T14:04:43.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.777+0000 7f675d200640 1 --2- 172.21.15.86:0/2087153650 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f673406f400 0x7f6734071910 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.777+0000 7f675d200640 1 --2- 172.21.15.86:0/2087153650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6758159c50 0x7f675815b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.777+0000 7f675d200640 1 --2- 172.21.15.86:0/2087153650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67581642a0 0x7f675815b280 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.777+0000 7f675d200640 1 --2- 172.21.15.86:0/2087153650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67580b0690 0x7f6758159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.777+0000 7f675d200640 1 -- 172.21.15.86:0/2087153650 >> 172.21.15.86:0/2087153650 conn(0x7f67580b4650 msgr2=0x7f67580ccd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:43.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.781+0000 7f675d200640 1 -- 172.21.15.86:0/2087153650 shutdown_connections 2024-04-25T14:04:43.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:43.781+0000 7f675d200640 1 -- 172.21.15.86:0/2087153650 wait complete. 2024-04-25T14:04:43.800 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T14:04:43.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:43.801 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-25T14:04:43.836 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T14:04:43.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:43.836 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T14:04:43.885 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:04:43.885 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:04:43.895 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:04:43.895 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:04:43.944 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:04:43.945 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:04:43.997 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:04:44.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.141+0000 7f7763c00640 1 Processor -- start 2024-04-25T14:04:44.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.141+0000 7f7763c00640 1 -- start start 2024-04-25T14:04:44.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.145+0000 7f7763c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f775c0cfca0 0x7f775c06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:44.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.145+0000 7f7763c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f775c0730b0 0x7f775c0734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:44.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.145+0000 7f7763c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f775c072bd0 con 0x7f775c0d16e0 2024-04-25T14:04:44.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.145+0000 7f7763c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f775c070470 con 0x7f775c0cfca0 2024-04-25T14:04:44.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.145+0000 7f7763c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f775c06f930 con 0x7f775c0730b0 2024-04-25T14:04:44.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.145+0000 7f7761200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f775c0cfca0 0x7f775c06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:44.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.145+0000 7f7761200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f775c0cfca0 0x7f775c06f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60734/0 (socket says 172.21.15.86:60734) 2024-04-25T14:04:44.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.145+0000 7f7761200640 1 -- 172.21.15.86:0/174490399 learned_addr learned my addr 172.21.15.86:0/174490399 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:44.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.145+0000 7f7761c00640 1 --2- 172.21.15.86:0/174490399 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f775c0730b0 0x7f775c0734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:44.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.145+0000 7f7761200640 1 -- 172.21.15.86:0/174490399 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f775c0730b0 msgr2=0x7f775c0734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:44.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.145+0000 7f7761200640 1 --2- 172.21.15.86:0/174490399 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f775c0730b0 0x7f775c0734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.145+0000 7f7761200640 1 -- 172.21.15.86:0/174490399 >> v1:172.21.15.155:6789/0 conn(0x7f775c0d16e0 legacy=0x7f775c0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:44.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.145+0000 7f7761200640 1 -- 172.21.15.86:0/174490399 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f775c06fbd0 con 0x7f775c0cfca0 2024-04-25T14:04:44.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.149+0000 7f7761200640 1 --2- 172.21.15.86:0/174490399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f775c0cfca0 0x7f775c06f3f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f775c0d22a0 tx=0x7f774c01ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d068d050e0e910ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:44.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.149+0000 7f775be00640 1 -- 172.21.15.86:0/174490399 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c0b4070 con 0x7f775c0cfca0 2024-04-25T14:04:44.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.149+0000 7f775be00640 1 -- 172.21.15.86:0/174490399 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f774c0a7b70 con 0x7f775c0cfca0 2024-04-25T14:04:44.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.149+0000 7f775be00640 1 -- 172.21.15.86:0/174490399 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c0af440 con 0x7f775c0cfca0 2024-04-25T14:04:44.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.149+0000 7f7763c00640 1 -- 172.21.15.86:0/174490399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f775c0cfca0 msgr2=0x7f775c06f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:44.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.149+0000 7f7763c00640 1 --2- 172.21.15.86:0/174490399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f775c0cfca0 0x7f775c06f3f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f775c0d22a0 tx=0x7f774c01ebd0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.149+0000 7f7763c00640 1 -- 172.21.15.86:0/174490399 shutdown_connections 2024-04-25T14:04:44.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.149+0000 7f7763c00640 1 --2- 172.21.15.86:0/174490399 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f775c0730b0 0x7f775c0734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.149+0000 7f7763c00640 1 --2- 172.21.15.86:0/174490399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f775c0cfca0 0x7f775c06f3f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.149+0000 7f7763c00640 1 -- 172.21.15.86:0/174490399 >> 172.21.15.86:0/174490399 conn(0x7f775c0b4650 msgr2=0x7f775c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:44.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.149+0000 7f7763c00640 1 -- 172.21.15.86:0/174490399 shutdown_connections 2024-04-25T14:04:44.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.149+0000 7f7763c00640 1 -- 172.21.15.86:0/174490399 wait complete. 2024-04-25T14:04:44.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.149+0000 7f7763c00640 1 Processor -- start 2024-04-25T14:04:44.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.153+0000 7f7763c00640 1 -- start start 2024-04-25T14:04:44.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.153+0000 7f7763c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f775c0730b0 0x7f775c1fcd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:44.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.153+0000 7f7763c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f775c0cfca0 0x7f775c15eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:44.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.153+0000 7f7763c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f775c0d16e0 0x7f775c15f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:44.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.153+0000 7f7763c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f775c0ce130 con 0x7f775c0d16e0 2024-04-25T14:04:44.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.153+0000 7f7763c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f775c0cdfb0 con 0x7f775c0cfca0 2024-04-25T14:04:44.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.153+0000 7f7763c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f775c0ce2b0 con 0x7f775c0730b0 2024-04-25T14:04:44.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.153+0000 7f7761c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f775c0d16e0 0x7f775c15f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:44.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.153+0000 7f7761c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f775c0d16e0 0x7f775c15f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60746/0 (socket says 172.21.15.86:60746) 2024-04-25T14:04:44.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.153+0000 7f7761c00640 1 -- 172.21.15.86:0/2959894530 learned_addr learned my addr 172.21.15.86:0/2959894530 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:44.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.153+0000 7f7761200640 1 --2- 172.21.15.86:0/2959894530 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f775c0730b0 0x7f775c1fcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:44.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.153+0000 7f7753e00640 1 --2- 172.21.15.86:0/2959894530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f775c0cfca0 0x7f775c15eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:44.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.153+0000 7f7761c00640 1 -- 172.21.15.86:0/2959894530 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f775c0730b0 msgr2=0x7f775c1fcd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:44.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.153+0000 7f7761c00640 1 --2- 172.21.15.86:0/2959894530 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f775c0730b0 0x7f775c1fcd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.153+0000 7f7761c00640 1 -- 172.21.15.86:0/2959894530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f775c0cfca0 msgr2=0x7f775c15eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:44.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.153+0000 7f7761c00640 1 --2- 172.21.15.86:0/2959894530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f775c0cfca0 0x7f775c15eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.153+0000 7f7761c00640 1 -- 172.21.15.86:0/2959894530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f775c1fd290 con 0x7f775c0d16e0 2024-04-25T14:04:44.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.153+0000 7f7761200640 1 --2- 172.21.15.86:0/2959894530 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f775c0730b0 0x7f775c1fcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:44.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.153+0000 7f7761c00640 1 --2- 172.21.15.86:0/2959894530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f775c0d16e0 0x7f775c15f070 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7754009fd0 tx=0x7f775400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:44.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.153+0000 7f7759600640 1 -- 172.21.15.86:0/2959894530 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7754014420 con 0x7f775c0d16e0 2024-04-25T14:04:44.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.153+0000 7f7759600640 1 -- 172.21.15.86:0/2959894530 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7754019070 con 0x7f775c0d16e0 2024-04-25T14:04:44.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.153+0000 7f7763c00640 1 -- 172.21.15.86:0/2959894530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f775c0ca5d0 con 0x7f775c0d16e0 2024-04-25T14:04:44.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.153+0000 7f7759600640 1 -- 172.21.15.86:0/2959894530 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7754014800 con 0x7f775c0d16e0 2024-04-25T14:04:44.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.153+0000 7f7763c00640 1 -- 172.21.15.86:0/2959894530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f775c0cab90 con 0x7f775c0d16e0 2024-04-25T14:04:44.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.153+0000 7f7763c00640 1 -- 172.21.15.86:0/2959894530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f775c09f4b0 con 0x7f775c0d16e0 2024-04-25T14:04:44.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.157+0000 7f7759600640 1 -- 172.21.15.86:0/2959894530 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f77540149a0 con 0x7f775c0d16e0 2024-04-25T14:04:44.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.157+0000 7f7759600640 1 --2- 172.21.15.86:0/2959894530 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f773806f400 0x7f7738071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:44.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.157+0000 7f7759600640 1 -- 172.21.15.86:0/2959894530 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f775408fe90 con 0x7f775c0d16e0 2024-04-25T14:04:44.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.157+0000 7f7761200640 1 --2- 172.21.15.86:0/2959894530 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f773806f400 0x7f7738071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:44.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.157+0000 7f7761200640 1 --2- 172.21.15.86:0/2959894530 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f773806f400 0x7f7738071910 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f774c0b1000 tx=0x7f774c0a9070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:44.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.161+0000 7f7759600640 1 -- 172.21.15.86:0/2959894530 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f775405c320 con 0x7f775c0d16e0 2024-04-25T14:04:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.317+0000 7f7763c00640 1 -- 172.21.15.86:0/2959894530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f775c0ca320 con 0x7f775c0d16e0 2024-04-25T14:04:44.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.321+0000 7f7759600640 1 -- 172.21.15.86:0/2959894530 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18902 (secure 0 0 0) 0x7f7754091090 con 0x7f775c0d16e0 2024-04-25T14:04:44.322 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:44.322 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":26,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:04:31.930803+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:03:49.151473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:03:49.802597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T14:03:53.492008+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:04:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.321+0000 7f7763c00640 1 -- 172.21.15.86:0/2959894530 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f773806f400 msgr2=0x7f7738071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.321+0000 7f7763c00640 1 --2- 172.21.15.86:0/2959894530 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f773806f400 0x7f7738071910 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f774c0b1000 tx=0x7f774c0a9070 comp rx=0 tx=0).stop 2024-04-25T14:04:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.321+0000 7f7763c00640 1 -- 172.21.15.86:0/2959894530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f775c0d16e0 msgr2=0x7f775c15f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.321+0000 7f7763c00640 1 --2- 172.21.15.86:0/2959894530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f775c0d16e0 0x7f775c15f070 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7754009fd0 tx=0x7f775400e4b0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.321+0000 7f7763c00640 1 -- 172.21.15.86:0/2959894530 shutdown_connections 2024-04-25T14:04:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.321+0000 7f7763c00640 1 --2- 172.21.15.86:0/2959894530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f775c0d16e0 0x7f775c15f070 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.321+0000 7f7763c00640 1 --2- 172.21.15.86:0/2959894530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f775c0cfca0 0x7f775c15eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.321+0000 7f7763c00640 1 --2- 172.21.15.86:0/2959894530 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f773806f400 0x7f7738071910 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.321+0000 7f7763c00640 1 --2- 172.21.15.86:0/2959894530 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f775c0730b0 0x7f775c1fcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.321+0000 7f7763c00640 1 -- 172.21.15.86:0/2959894530 >> 172.21.15.86:0/2959894530 conn(0x7f775c0b4650 msgr2=0x7f775c159b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.321+0000 7f7763c00640 1 -- 172.21.15.86:0/2959894530 shutdown_connections 2024-04-25T14:04:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.321+0000 7f7763c00640 1 -- 172.21.15.86:0/2959894530 wait complete. 2024-04-25T14:04:44.345 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:03:49.151473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:03:49.802597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T14:03:53.492008+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T14:04:44.345 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:04:44.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.493+0000 7f62cf800640 1 Processor -- start 2024-04-25T14:04:44.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.493+0000 7f62cf800640 1 -- start start 2024-04-25T14:04:44.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.493+0000 7f62cf800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f62c81642a0 0x7f62c80d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:44.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.493+0000 7f62cf800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62c80d2030 con 0x7f62c80b0860 2024-04-25T14:04:44.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.493+0000 7f62cf800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62c80cde20 con 0x7f62c80d1c20 2024-04-25T14:04:44.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.493+0000 7f62cf800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f62c80cea30 con 0x7f62c81642a0 2024-04-25T14:04:44.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.493+0000 7f62cce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f62c81642a0 0x7f62c80d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:44.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.493+0000 7f62cce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f62c81642a0 0x7f62c80d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60750/0 (socket says 172.21.15.86:60750) 2024-04-25T14:04:44.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.493+0000 7f62cd800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f62c80b0860 0x7f62c80d0640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54882/0 (socket says 172.21.15.86:54882) 2024-04-25T14:04:44.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.493+0000 7f62cce00640 1 -- 172.21.15.86:0/2331844965 learned_addr learned my addr 172.21.15.86:0/2331844965 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:44.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.493+0000 7f62cce00640 1 -- 172.21.15.86:0/2331844965 >> v1:172.21.15.155:6789/0 conn(0x7f62c80d1c20 legacy=0x7f62c80b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:44.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.493+0000 7f62cce00640 1 -- 172.21.15.86:0/2331844965 >> v1:172.21.15.86:6789/0 conn(0x7f62c80b0860 legacy=0x7f62c80d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:44.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.493+0000 7f62cce00640 1 -- 172.21.15.86:0/2331844965 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c80ce100 con 0x7f62c81642a0 2024-04-25T14:04:44.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.497+0000 7f62cce00640 1 --2- 172.21.15.86:0/2331844965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c81642a0 0x7f62c80d16e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f62b8009870 tx=0x7f62b801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a778989723967 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:44.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.497+0000 7f62c7e00640 1 -- 172.21.15.86:0/2331844965 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b80b0070 con 0x7f62c81642a0 2024-04-25T14:04:44.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.497+0000 7f62c7e00640 1 -- 172.21.15.86:0/2331844965 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b80ab600 con 0x7f62c81642a0 2024-04-25T14:04:44.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.497+0000 7f62c7e00640 1 -- 172.21.15.86:0/2331844965 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b80ab9d0 con 0x7f62c81642a0 2024-04-25T14:04:44.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.497+0000 7f62cf800640 1 -- 172.21.15.86:0/2331844965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c81642a0 msgr2=0x7f62c80d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:44.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.497+0000 7f62cf800640 1 --2- 172.21.15.86:0/2331844965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c81642a0 0x7f62c80d16e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f62b8009870 tx=0x7f62b801c490 comp rx=0 tx=0).stop 2024-04-25T14:04:44.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.497+0000 7f62cf800640 1 -- 172.21.15.86:0/2331844965 shutdown_connections 2024-04-25T14:04:44.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.497+0000 7f62cf800640 1 --2- 172.21.15.86:0/2331844965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c81642a0 0x7f62c80d16e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.497+0000 7f62cf800640 1 -- 172.21.15.86:0/2331844965 >> 172.21.15.86:0/2331844965 conn(0x7f62c80b4650 msgr2=0x7f62c80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:44.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.497+0000 7f62cf800640 1 -- 172.21.15.86:0/2331844965 shutdown_connections 2024-04-25T14:04:44.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.497+0000 7f62cf800640 1 -- 172.21.15.86:0/2331844965 wait complete. 2024-04-25T14:04:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.501+0000 7f62cf800640 1 Processor -- start 2024-04-25T14:04:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.501+0000 7f62cf800640 1 -- start start 2024-04-25T14:04:44.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.501+0000 7f62cf800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62c80b0860 0x7f62c80999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:44.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.501+0000 7f62cf800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62c80d1c20 0x7f62c8158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:44.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.501+0000 7f62cf800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c81642a0 0x7f62c81591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:44.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.501+0000 7f62cf800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62c81648c0 con 0x7f62c81642a0 2024-04-25T14:04:44.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.501+0000 7f62cf800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f62c80d1160 con 0x7f62c80d1c20 2024-04-25T14:04:44.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.501+0000 7f62cf800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f62c8160da0 con 0x7f62c80b0860 2024-04-25T14:04:44.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.501+0000 7f62cd800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c81642a0 0x7f62c81591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:44.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.501+0000 7f62cd800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c81642a0 0x7f62c81591e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60756/0 (socket says 172.21.15.86:60756) 2024-04-25T14:04:44.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.501+0000 7f62cd800640 1 -- 172.21.15.86:0/3765851262 learned_addr learned my addr 172.21.15.86:0/3765851262 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:44.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.505+0000 7f62cd800640 1 -- 172.21.15.86:0/3765851262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62c80b0860 msgr2=0x7f62c80999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:44.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.505+0000 7f62cce00640 1 --2- 172.21.15.86:0/3765851262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62c80b0860 0x7f62c80999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:44.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.505+0000 7f62cd800640 1 --2- 172.21.15.86:0/3765851262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62c80b0860 0x7f62c80999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.505+0000 7f62cd800640 1 -- 172.21.15.86:0/3765851262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62c80d1c20 msgr2=0x7f62c8158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:44.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.505+0000 7f62bfe00640 1 --2- 172.21.15.86:0/3765851262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62c80d1c20 0x7f62c8158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:44.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.505+0000 7f62cd800640 1 --2- 172.21.15.86:0/3765851262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62c80d1c20 0x7f62c8158ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.505+0000 7f62cd800640 1 -- 172.21.15.86:0/3765851262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c8163160 con 0x7f62c81642a0 2024-04-25T14:04:44.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.505+0000 7f62bfe00640 1 --2- 172.21.15.86:0/3765851262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62c80d1c20 0x7f62c8158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:44.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.505+0000 7f62cd800640 1 --2- 172.21.15.86:0/3765851262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c81642a0 0x7f62c81591e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f62c000c6c0 tx=0x7f62c000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:44.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.505+0000 7f62c5600640 1 -- 172.21.15.86:0/3765851262 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0018070 con 0x7f62c81642a0 2024-04-25T14:04:44.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.505+0000 7f62c5600640 1 -- 172.21.15.86:0/3765851262 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62c0004d70 con 0x7f62c81642a0 2024-04-25T14:04:44.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.505+0000 7f62c5600640 1 -- 172.21.15.86:0/3765851262 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c000a840 con 0x7f62c81642a0 2024-04-25T14:04:44.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.505+0000 7f62cf800640 1 -- 172.21.15.86:0/3765851262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62c8162e80 con 0x7f62c81642a0 2024-04-25T14:04:44.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.505+0000 7f62cf800640 1 -- 172.21.15.86:0/3765851262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62c809a6a0 con 0x7f62c81642a0 2024-04-25T14:04:44.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.505+0000 7f62cf800640 1 -- 172.21.15.86:0/3765851262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62c8069d00 con 0x7f62c81642a0 2024-04-25T14:04:44.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.505+0000 7f62c5600640 1 -- 172.21.15.86:0/3765851262 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f62c000a9e0 con 0x7f62c81642a0 2024-04-25T14:04:44.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.505+0000 7f62c5600640 1 --2- 172.21.15.86:0/3765851262 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f62a406f400 0x7f62a4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:44.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.509+0000 7f62c5600640 1 -- 172.21.15.86:0/3765851262 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f62c008ea60 con 0x7f62c81642a0 2024-04-25T14:04:44.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.509+0000 7f62cce00640 1 --2- 172.21.15.86:0/3765851262 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f62a406f400 0x7f62a4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:44.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.509+0000 7f62cce00640 1 --2- 172.21.15.86:0/3765851262 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f62a406f400 0x7f62a4071910 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f62b80ad000 tx=0x7f62b8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:44.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.513+0000 7f62c5600640 1 -- 172.21.15.86:0/3765851262 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f62c80d2030 con 0x7f62c81642a0 2024-04-25T14:04:44.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.669+0000 7f62cf800640 1 -- 172.21.15.86:0/3765851262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f62c8076bf0 con 0x7f62c81642a0 2024-04-25T14:04:44.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.669+0000 7f62c5600640 1 -- 172.21.15.86:0/3765851262 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f62c005aef0 con 0x7f62c81642a0 2024-04-25T14:04:44.670 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:04:44.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.669+0000 7f62cf800640 1 -- 172.21.15.86:0/3765851262 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f62a406f400 msgr2=0x7f62a4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:44.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.669+0000 7f62cf800640 1 --2- 172.21.15.86:0/3765851262 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f62a406f400 0x7f62a4071910 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f62b80ad000 tx=0x7f62b8008040 comp rx=0 tx=0).stop 2024-04-25T14:04:44.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.669+0000 7f62cf800640 1 -- 172.21.15.86:0/3765851262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c81642a0 msgr2=0x7f62c81591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:44.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.669+0000 7f62cf800640 1 --2- 172.21.15.86:0/3765851262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c81642a0 0x7f62c81591e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f62c000c6c0 tx=0x7f62c000cb80 comp rx=0 tx=0).stop 2024-04-25T14:04:44.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.669+0000 7f62cf800640 1 -- 172.21.15.86:0/3765851262 shutdown_connections 2024-04-25T14:04:44.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.669+0000 7f62cf800640 1 --2- 172.21.15.86:0/3765851262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c81642a0 0x7f62c81591e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.669+0000 7f62cf800640 1 --2- 172.21.15.86:0/3765851262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62c80d1c20 0x7f62c8158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.669+0000 7f62cf800640 1 --2- 172.21.15.86:0/3765851262 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f62a406f400 0x7f62a4071910 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.669+0000 7f62cf800640 1 --2- 172.21.15.86:0/3765851262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62c80b0860 0x7f62c80999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.669+0000 7f62cf800640 1 -- 172.21.15.86:0/3765851262 >> 172.21.15.86:0/3765851262 conn(0x7f62c80b4650 msgr2=0x7f62c80b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:44.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.669+0000 7f62cf800640 1 -- 172.21.15.86:0/3765851262 shutdown_connections 2024-04-25T14:04:44.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.669+0000 7f62cf800640 1 -- 172.21.15.86:0/3765851262 wait complete. 2024-04-25T14:04:44.692 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:04:44.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.841+0000 7f7dd9c00640 1 Processor -- start 2024-04-25T14:04:44.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.841+0000 7f7dd9c00640 1 -- start start 2024-04-25T14:04:44.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.841+0000 7f7dd9c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7dd41615c0 0x7f7dd41619d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:44.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.841+0000 7f7dd9c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dd41622f0 con 0x7f7dd40b6530 2024-04-25T14:04:44.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.841+0000 7f7dd9c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dd40d1730 con 0x7f7dd406f990 2024-04-25T14:04:44.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.841+0000 7f7dd9c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f7dd40d1a10 con 0x7f7dd41615c0 2024-04-25T14:04:44.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.841+0000 7f7dd9200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7dd40b6530 0x7f7dd40b6940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54894/0 (socket says 172.21.15.86:54894) 2024-04-25T14:04:44.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.841+0000 7f7dd9200640 1 -- 172.21.15.86:0/2240446964 learned_addr learned my addr 172.21.15.86:0/2240446964 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:44.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.841+0000 7f7dd3400640 1 --2- 172.21.15.86:0/2240446964 >> v2:172.21.15.155:3300/0 conn(0x7f7dd41615c0 0x7f7dd41619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:44.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.841+0000 7f7dd2a00640 1 -- 172.21.15.86:0/2240446964 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 805975126 0 0) 0x7f7dd41622f0 con 0x7f7dd40b6530 2024-04-25T14:04:44.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.841+0000 7f7dd2a00640 1 -- 172.21.15.86:0/2240446964 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7db8003530 con 0x7f7dd40b6530 2024-04-25T14:04:44.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.841+0000 7f7dd2a00640 1 -- 172.21.15.86:0/2240446964 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 475995209 0 0) 0x7f7db8003530 con 0x7f7dd40b6530 2024-04-25T14:04:44.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.841+0000 7f7dd2a00640 1 -- 172.21.15.86:0/2240446964 >> v2:172.21.15.155:3300/0 conn(0x7f7dd41615c0 msgr2=0x7f7dd41619d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:44.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.841+0000 7f7dd2a00640 1 --2- 172.21.15.86:0/2240446964 >> v2:172.21.15.155:3300/0 conn(0x7f7dd41615c0 0x7f7dd41619d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.841+0000 7f7dd2a00640 1 -- 172.21.15.86:0/2240446964 >> v1:172.21.15.155:6789/0 conn(0x7f7dd406f990 legacy=0x7f7dd406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:44.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.841+0000 7f7dd2a00640 1 -- 172.21.15.86:0/2240446964 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd40d1cf0 con 0x7f7dd40b6530 2024-04-25T14:04:44.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.841+0000 7f7dd2a00640 1 -- 172.21.15.86:0/2240446964 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f7dcc001a00 con 0x7f7dd40b6530 2024-04-25T14:04:44.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.841+0000 7f7dd2a00640 1 -- 172.21.15.86:0/2240446964 >> v1:172.21.15.86:6789/0 conn(0x7f7dd40b6530 legacy=0x7f7dd40b6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:44.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.841+0000 7f7dd2a00640 1 --2- 172.21.15.86:0/2240446964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7db80039c0 0x7f7db8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:44.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.845+0000 7f7dd2a00640 1 --2- 172.21.15.86:0/2240446964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7db8004810 0x7f7db8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:44.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.845+0000 7f7dd2a00640 1 --2- 172.21.15.86:0/2240446964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7db8005260 0x7f7db800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:44.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.845+0000 7f7dd2a00640 1 -- 172.21.15.86:0/2240446964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd40d1a10 con 0x7f7db8004810 2024-04-25T14:04:44.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.845+0000 7f7dd2a00640 1 -- 172.21.15.86:0/2240446964 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7db800c1f0 con 0x7f7db8005260 2024-04-25T14:04:44.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.845+0000 7f7dd2a00640 1 -- 172.21.15.86:0/2240446964 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7db800c3f0 con 0x7f7db80039c0 2024-04-25T14:04:44.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.845+0000 7f7dd3400640 1 --2- 172.21.15.86:0/2240446964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7db8004810 0x7f7db8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:44.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.845+0000 7f7dd3e00640 1 --2- 172.21.15.86:0/2240446964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7db80039c0 0x7f7db8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:44.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.845+0000 7f7dd9200640 1 --2- 172.21.15.86:0/2240446964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7db8005260 0x7f7db800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:44.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.845+0000 7f7dd3400640 1 -- 172.21.15.86:0/2240446964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7db80039c0 msgr2=0x7f7db8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:44.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.845+0000 7f7dd3400640 1 --2- 172.21.15.86:0/2240446964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7db80039c0 0x7f7db8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.845+0000 7f7dd3400640 1 -- 172.21.15.86:0/2240446964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7db8005260 msgr2=0x7f7db800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:44.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.845+0000 7f7dd3400640 1 --2- 172.21.15.86:0/2240446964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7db8005260 0x7f7db800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.845+0000 7f7dd9200640 1 -- 172.21.15.86:0/2240446964 reap_dead start 2024-04-25T14:04:44.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.845+0000 7f7dd3400640 1 -- 172.21.15.86:0/2240446964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db8003530 con 0x7f7db8004810 2024-04-25T14:04:44.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.845+0000 7f7dd3400640 1 --2- 172.21.15.86:0/2240446964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7db8004810 0x7f7db8004c20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0009ae0 tx=0x7f7dd41621e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f864b5e9d6372491 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:44.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.845+0000 7f7dd2a00640 1 -- 172.21.15.86:0/2240446964 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc00096b0 con 0x7f7db8004810 2024-04-25T14:04:44.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.845+0000 7f7dd2a00640 1 -- 172.21.15.86:0/2240446964 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dc00a6450 con 0x7f7db8004810 2024-04-25T14:04:44.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.845+0000 7f7dd2a00640 1 -- 172.21.15.86:0/2240446964 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc00a6840 con 0x7f7db8004810 2024-04-25T14:04:44.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.849+0000 7f7dd9c00640 1 -- 172.21.15.86:0/2240446964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7db8004810 msgr2=0x7f7db8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:44.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.849+0000 7f7dd9c00640 1 --2- 172.21.15.86:0/2240446964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7db8004810 0x7f7db8004c20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0009ae0 tx=0x7f7dd41621e0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.849+0000 7f7dd9c00640 1 -- 172.21.15.86:0/2240446964 shutdown_connections 2024-04-25T14:04:44.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.849+0000 7f7dd9c00640 1 --2- 172.21.15.86:0/2240446964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7db8004810 0x7f7db8004c20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.849+0000 7f7dd9c00640 1 -- 172.21.15.86:0/2240446964 >> 172.21.15.86:0/2240446964 conn(0x7f7dd40cfea0 msgr2=0x7f7dd40d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:44.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.849+0000 7f7dd9c00640 1 -- 172.21.15.86:0/2240446964 shutdown_connections 2024-04-25T14:04:44.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.849+0000 7f7dd9c00640 1 -- 172.21.15.86:0/2240446964 wait complete. 2024-04-25T14:04:44.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.849+0000 7f7dd9c00640 1 Processor -- start 2024-04-25T14:04:44.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.849+0000 7f7dd9c00640 1 -- start start 2024-04-25T14:04:44.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.853+0000 7f7dd9c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7db8004810 0x7f7dd41ea050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:44.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.853+0000 7f7dd9c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dd40aa630 0x7f7dd40c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:44.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.853+0000 7f7dd9c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7dd40cc830 0x7f7dd40ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:44.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.853+0000 7f7dd9c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd40b6940 con 0x7f7dd40aa630 2024-04-25T14:04:44.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.853+0000 7f7dd9c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd40d2840 con 0x7f7dd40cc830 2024-04-25T14:04:44.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.853+0000 7f7dd9c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7dd40b6ac0 con 0x7f7db8004810 2024-04-25T14:04:44.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.853+0000 7f7dd3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dd40aa630 0x7f7dd40c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:44.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.853+0000 7f7dd3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dd40aa630 0x7f7dd40c8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60776/0 (socket says 172.21.15.86:60776) 2024-04-25T14:04:44.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.853+0000 7f7dd3400640 1 -- 172.21.15.86:0/1680288845 learned_addr learned my addr 172.21.15.86:0/1680288845 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:44.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.853+0000 7f7dd3e00640 1 --2- 172.21.15.86:0/1680288845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7db8004810 0x7f7dd41ea050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:44.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.853+0000 7f7dd9200640 1 --2- 172.21.15.86:0/1680288845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7dd40cc830 0x7f7dd40ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:44.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.853+0000 7f7dd3400640 1 -- 172.21.15.86:0/1680288845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7db8004810 msgr2=0x7f7dd41ea050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:44.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.853+0000 7f7dd3400640 1 --2- 172.21.15.86:0/1680288845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7db8004810 0x7f7dd41ea050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.853+0000 7f7dd3400640 1 -- 172.21.15.86:0/1680288845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7dd40cc830 msgr2=0x7f7dd40ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:44.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.853+0000 7f7dd3400640 1 --2- 172.21.15.86:0/1680288845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7dd40cc830 0x7f7dd40ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.853+0000 7f7dd3400640 1 -- 172.21.15.86:0/1680288845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd40cd180 con 0x7f7dd40aa630 2024-04-25T14:04:44.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.853+0000 7f7dd9200640 1 --2- 172.21.15.86:0/1680288845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7dd40cc830 0x7f7dd40ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:44.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.853+0000 7f7dd3400640 1 --2- 172.21.15.86:0/1680288845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dd40aa630 0x7f7dd40c8980 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0009ae0 tx=0x7f7dc001b2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:44.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.853+0000 7f7dcbe00640 1 -- 172.21.15.86:0/1680288845 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc00bc070 con 0x7f7dd40aa630 2024-04-25T14:04:44.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.853+0000 7f7dcbe00640 1 -- 172.21.15.86:0/1680288845 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dc00a6ce0 con 0x7f7dd40aa630 2024-04-25T14:04:44.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.853+0000 7f7dd9c00640 1 -- 172.21.15.86:0/1680288845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd414d560 con 0x7f7dd40aa630 2024-04-25T14:04:44.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.853+0000 7f7dcbe00640 1 -- 172.21.15.86:0/1680288845 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0004270 con 0x7f7dd40aa630 2024-04-25T14:04:44.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.853+0000 7f7dd9c00640 1 -- 172.21.15.86:0/1680288845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd414db20 con 0x7f7dd40aa630 2024-04-25T14:04:44.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.853+0000 7f7dd9c00640 1 -- 172.21.15.86:0/1680288845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dd4076b80 con 0x7f7dd40aa630 2024-04-25T14:04:44.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.857+0000 7f7dcbe00640 1 -- 172.21.15.86:0/1680288845 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7dc00bf070 con 0x7f7dd40aa630 2024-04-25T14:04:44.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.857+0000 7f7dcbe00640 1 --2- 172.21.15.86:0/1680288845 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7db406f400 0x7f7db4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:44.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.857+0000 7f7dcbe00640 1 -- 172.21.15.86:0/1680288845 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f7dc01291a0 con 0x7f7dd40aa630 2024-04-25T14:04:44.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.857+0000 7f7dd3e00640 1 --2- 172.21.15.86:0/1680288845 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7db406f400 0x7f7db4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:44.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.857+0000 7f7dd3e00640 1 --2- 172.21.15.86:0/1680288845 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7db406f400 0x7f7db4071910 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4000c10 tx=0x7f7dc4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:44.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:44.861+0000 7f7dcbe00640 1 -- 172.21.15.86:0/1680288845 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7dd40cd180 con 0x7f7dd40aa630 2024-04-25T14:04:45.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.017+0000 7f7dd9c00640 1 -- 172.21.15.86:0/1680288845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7dd41ea5f0 con 0x7f7dd40aa630 2024-04-25T14:04:45.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.017+0000 7f7dcbe00640 1 -- 172.21.15.86:0/1680288845 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f7dc00ab020 con 0x7f7dd40aa630 2024-04-25T14:04:45.019 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:04:45.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.021+0000 7f7dd9c00640 1 -- 172.21.15.86:0/1680288845 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7db406f400 msgr2=0x7f7db4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:45.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.021+0000 7f7dd9c00640 1 --2- 172.21.15.86:0/1680288845 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7db406f400 0x7f7db4071910 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4000c10 tx=0x7f7dc4009040 comp rx=0 tx=0).stop 2024-04-25T14:04:45.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.021+0000 7f7dd9c00640 1 -- 172.21.15.86:0/1680288845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dd40aa630 msgr2=0x7f7dd40c8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:45.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.021+0000 7f7dd9c00640 1 --2- 172.21.15.86:0/1680288845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dd40aa630 0x7f7dd40c8980 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0009ae0 tx=0x7f7dc001b2b0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.021+0000 7f7dd9c00640 1 -- 172.21.15.86:0/1680288845 shutdown_connections 2024-04-25T14:04:45.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.021+0000 7f7dd9c00640 1 --2- 172.21.15.86:0/1680288845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7dd40cc830 0x7f7dd40ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.021+0000 7f7dd9c00640 1 --2- 172.21.15.86:0/1680288845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dd40aa630 0x7f7dd40c8980 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.021+0000 7f7dd9c00640 1 --2- 172.21.15.86:0/1680288845 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7db406f400 0x7f7db4071910 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.021+0000 7f7dd9c00640 1 --2- 172.21.15.86:0/1680288845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7db8004810 0x7f7dd41ea050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.021+0000 7f7dd9c00640 1 -- 172.21.15.86:0/1680288845 >> 172.21.15.86:0/1680288845 conn(0x7f7dd40cfea0 msgr2=0x7f7dd40b44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:45.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.021+0000 7f7dd9c00640 1 -- 172.21.15.86:0/1680288845 shutdown_connections 2024-04-25T14:04:45.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.021+0000 7f7dd9c00640 1 -- 172.21.15.86:0/1680288845 wait complete. 2024-04-25T14:04:45.041 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:04:45.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.189+0000 7f81afc00640 1 Processor -- start 2024-04-25T14:04:45.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.189+0000 7f81afc00640 1 -- start start 2024-04-25T14:04:45.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.189+0000 7f81afc00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f81a815e860 0x7f81a80ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:45.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.189+0000 7f81afc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81a80cf3e0 0x7f81a80d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:45.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.189+0000 7f81afc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81a80cf8f0 con 0x7f81a815ec70 2024-04-25T14:04:45.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.189+0000 7f81afc00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f81a806cfa0 con 0x7f81a815e860 2024-04-25T14:04:45.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.189+0000 7f81afc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f81a806c670 con 0x7f81a80cf3e0 2024-04-25T14:04:45.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.189+0000 7f81adc00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f81a815ec70 0x7f81a806d430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54906/0 (socket says 172.21.15.86:54906) 2024-04-25T14:04:45.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.189+0000 7f81adc00640 1 -- 172.21.15.86:0/2287158314 learned_addr learned my addr 172.21.15.86:0/2287158314 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:45.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.189+0000 7f81ad200640 1 --2- 172.21.15.86:0/2287158314 >> v2:172.21.15.155:3300/0 conn(0x7f81a815e860 0x7f81a80ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:45.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.189+0000 7f819fe00640 1 --2- 172.21.15.86:0/2287158314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81a80cf3e0 0x7f81a80d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:45.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.193+0000 7f81a7e00640 1 -- 172.21.15.86:0/2287158314 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3793670241 0 0) 0x7f81a80cf8f0 con 0x7f81a815ec70 2024-04-25T14:04:45.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.193+0000 7f81a7e00640 1 -- 172.21.15.86:0/2287158314 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f818c003530 con 0x7f81a815ec70 2024-04-25T14:04:45.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.193+0000 7f81a7e00640 1 -- 172.21.15.86:0/2287158314 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 133671560 0 0) 0x7f818c003530 con 0x7f81a815ec70 2024-04-25T14:04:45.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.193+0000 7f81a7e00640 1 -- 172.21.15.86:0/2287158314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81a80cf3e0 msgr2=0x7f81a80d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:45.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.193+0000 7f81a7e00640 1 --2- 172.21.15.86:0/2287158314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81a80cf3e0 0x7f81a80d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.193+0000 7f81a7e00640 1 -- 172.21.15.86:0/2287158314 >> v2:172.21.15.155:3300/0 conn(0x7f81a815e860 msgr2=0x7f81a80ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:45.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.193+0000 7f81a7e00640 1 --2- 172.21.15.86:0/2287158314 >> v2:172.21.15.155:3300/0 conn(0x7f81a815e860 0x7f81a80ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.193+0000 7f81a7e00640 1 -- 172.21.15.86:0/2287158314 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a806c950 con 0x7f81a815ec70 2024-04-25T14:04:45.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.193+0000 7f81a7e00640 1 -- 172.21.15.86:0/2287158314 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f81a0001a00 con 0x7f81a815ec70 2024-04-25T14:04:45.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.193+0000 7f81a7e00640 1 -- 172.21.15.86:0/2287158314 >> v1:172.21.15.86:6789/0 conn(0x7f81a815ec70 legacy=0x7f81a806d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:45.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.193+0000 7f81a7e00640 1 --2- 172.21.15.86:0/2287158314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f818c003a30 0x7f818c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:45.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.193+0000 7f81a7e00640 1 --2- 172.21.15.86:0/2287158314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f818c004810 0x7f818c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:45.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.193+0000 7f81a7e00640 1 --2- 172.21.15.86:0/2287158314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81a80cf3e0 0x7f818c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:45.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.193+0000 7f81a7e00640 1 -- 172.21.15.86:0/2287158314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f81a806cfa0 con 0x7f818c004810 2024-04-25T14:04:45.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.193+0000 7f81a7e00640 1 -- 172.21.15.86:0/2287158314 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f81a806c670 con 0x7f818c003a30 2024-04-25T14:04:45.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.193+0000 7f81a7e00640 1 -- 172.21.15.86:0/2287158314 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f818c00bad0 con 0x7f81a80cf3e0 2024-04-25T14:04:45.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.193+0000 7f819fe00640 1 --2- 172.21.15.86:0/2287158314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f818c004810 0x7f818c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:45.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.193+0000 7f81ad200640 1 --2- 172.21.15.86:0/2287158314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f818c003a30 0x7f818c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:45.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.193+0000 7f81adc00640 1 --2- 172.21.15.86:0/2287158314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81a80cf3e0 0x7f818c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:45.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.193+0000 7f819fe00640 1 -- 172.21.15.86:0/2287158314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81a80cf3e0 msgr2=0x7f818c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:45.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.193+0000 7f819fe00640 1 --2- 172.21.15.86:0/2287158314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81a80cf3e0 0x7f818c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.193+0000 7f819fe00640 1 -- 172.21.15.86:0/2287158314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f818c003a30 msgr2=0x7f818c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:45.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.193+0000 7f819fe00640 1 --2- 172.21.15.86:0/2287158314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f818c003a30 0x7f818c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.193+0000 7f819fe00640 1 -- 172.21.15.86:0/2287158314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f818c003530 con 0x7f818c004810 2024-04-25T14:04:45.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.193+0000 7f81adc00640 1 --2- 172.21.15.86:0/2287158314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81a80cf3e0 0x7f818c005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:45.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.197+0000 7f819fe00640 1 --2- 172.21.15.86:0/2287158314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f818c004810 0x7f818c004c20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8190009ae0 tx=0x7f819001ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70792d6405430685 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:45.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.197+0000 7f81a7e00640 1 -- 172.21.15.86:0/2287158314 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81900b2070 con 0x7f818c004810 2024-04-25T14:04:45.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.197+0000 7f81a7e00640 1 -- 172.21.15.86:0/2287158314 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81900a55c0 con 0x7f818c004810 2024-04-25T14:04:45.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.197+0000 7f81a7e00640 1 -- 172.21.15.86:0/2287158314 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81900096b0 con 0x7f818c004810 2024-04-25T14:04:45.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.197+0000 7f81afc00640 1 -- 172.21.15.86:0/2287158314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f818c004810 msgr2=0x7f818c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:45.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.197+0000 7f81afc00640 1 --2- 172.21.15.86:0/2287158314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f818c004810 0x7f818c004c20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8190009ae0 tx=0x7f819001ca60 comp rx=0 tx=0).stop 2024-04-25T14:04:45.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.197+0000 7f81adc00640 1 -- 172.21.15.86:0/2287158314 reap_dead start 2024-04-25T14:04:45.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.197+0000 7f81afc00640 1 -- 172.21.15.86:0/2287158314 shutdown_connections 2024-04-25T14:04:45.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.197+0000 7f81afc00640 1 -- 172.21.15.86:0/2287158314 >> 172.21.15.86:0/2287158314 conn(0x7f81a80d2230 msgr2=0x7f81a80d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:45.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.197+0000 7f81afc00640 1 -- 172.21.15.86:0/2287158314 shutdown_connections 2024-04-25T14:04:45.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.197+0000 7f81afc00640 1 -- 172.21.15.86:0/2287158314 wait complete. 2024-04-25T14:04:45.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.197+0000 7f81afc00640 1 Processor -- start 2024-04-25T14:04:45.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.197+0000 7f81afc00640 1 -- start start 2024-04-25T14:04:45.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.201+0000 7f81afc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f818c004810 0x7f81a8150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:45.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.201+0000 7f81afc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81a8156d90 0x7f81a8157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:45.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.201+0000 7f81afc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81a81579a0 0x7f81a8154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:45.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.201+0000 7f81afc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f81a80d18e0 con 0x7f81a81579a0 2024-04-25T14:04:45.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.201+0000 7f81afc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f81a80d1760 con 0x7f81a8156d90 2024-04-25T14:04:45.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.201+0000 7f81afc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f81a8168710 con 0x7f818c004810 2024-04-25T14:04:45.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.201+0000 7f81adc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81a81579a0 0x7f81a8154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:45.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.201+0000 7f81adc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81a81579a0 0x7f81a8154120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60804/0 (socket says 172.21.15.86:60804) 2024-04-25T14:04:45.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.201+0000 7f81adc00640 1 -- 172.21.15.86:0/440160618 learned_addr learned my addr 172.21.15.86:0/440160618 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:45.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.201+0000 7f819fe00640 1 --2- 172.21.15.86:0/440160618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81a8156d90 0x7f81a8157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:45.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.201+0000 7f81ad200640 1 --2- 172.21.15.86:0/440160618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f818c004810 0x7f81a8150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:45.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.201+0000 7f81adc00640 1 -- 172.21.15.86:0/440160618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f818c004810 msgr2=0x7f81a8150370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:45.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.201+0000 7f81adc00640 1 --2- 172.21.15.86:0/440160618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f818c004810 0x7f81a8150370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.201+0000 7f81adc00640 1 -- 172.21.15.86:0/440160618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81a8156d90 msgr2=0x7f81a8157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:45.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.201+0000 7f81adc00640 1 --2- 172.21.15.86:0/440160618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81a8156d90 0x7f81a8157460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.201+0000 7f81adc00640 1 -- 172.21.15.86:0/440160618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a8155fd0 con 0x7f81a81579a0 2024-04-25T14:04:45.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.201+0000 7f81adc00640 1 --2- 172.21.15.86:0/440160618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81a81579a0 0x7f81a8154120 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f81a80712b0 tx=0x7f81a000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:45.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.201+0000 7f81a5600640 1 -- 172.21.15.86:0/440160618 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0004d70 con 0x7f81a81579a0 2024-04-25T14:04:45.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.201+0000 7f81a5600640 1 -- 172.21.15.86:0/440160618 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81a000bb80 con 0x7f81a81579a0 2024-04-25T14:04:45.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.201+0000 7f81afc00640 1 -- 172.21.15.86:0/440160618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a81562b0 con 0x7f81a81579a0 2024-04-25T14:04:45.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.201+0000 7f81a5600640 1 -- 172.21.15.86:0/440160618 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0007070 con 0x7f81a81579a0 2024-04-25T14:04:45.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.201+0000 7f81afc00640 1 -- 172.21.15.86:0/440160618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a8151be0 con 0x7f81a81579a0 2024-04-25T14:04:45.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.201+0000 7f81afc00640 1 -- 172.21.15.86:0/440160618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81a8074cf0 con 0x7f81a81579a0 2024-04-25T14:04:45.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.205+0000 7f81a5600640 1 -- 172.21.15.86:0/440160618 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f81a001b020 con 0x7f81a81579a0 2024-04-25T14:04:45.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.205+0000 7f81a5600640 1 --2- 172.21.15.86:0/440160618 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f818406f400 0x7f8184071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:45.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.205+0000 7f81a5600640 1 -- 172.21.15.86:0/440160618 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f81a008f8c0 con 0x7f81a81579a0 2024-04-25T14:04:45.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.205+0000 7f81ad200640 1 --2- 172.21.15.86:0/440160618 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f818406f400 0x7f8184071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:45.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.205+0000 7f81ad200640 1 --2- 172.21.15.86:0/440160618 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f818406f400 0x7f8184071910 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8198009ba0 tx=0x7f8198008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:45.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.209+0000 7f81a5600640 1 -- 172.21.15.86:0/440160618 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f81a005c310 con 0x7f81a81579a0 2024-04-25T14:04:45.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.365+0000 7f81afc00640 1 -- 172.21.15.86:0/440160618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f81a8076bf0 con 0x7f81a81579a0 2024-04-25T14:04:45.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.369+0000 7f81a5600640 1 -- 172.21.15.86:0/440160618 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f81a005bc10 con 0x7f81a81579a0 2024-04-25T14:04:45.369 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:04:45.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.369+0000 7f81afc00640 1 -- 172.21.15.86:0/440160618 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f818406f400 msgr2=0x7f8184071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:45.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.369+0000 7f81afc00640 1 --2- 172.21.15.86:0/440160618 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f818406f400 0x7f8184071910 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8198009ba0 tx=0x7f8198008040 comp rx=0 tx=0).stop 2024-04-25T14:04:45.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.369+0000 7f81afc00640 1 -- 172.21.15.86:0/440160618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81a81579a0 msgr2=0x7f81a8154120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:45.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.369+0000 7f81afc00640 1 --2- 172.21.15.86:0/440160618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81a81579a0 0x7f81a8154120 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f81a80712b0 tx=0x7f81a000b4a0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.369+0000 7f81afc00640 1 -- 172.21.15.86:0/440160618 shutdown_connections 2024-04-25T14:04:45.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.369+0000 7f81afc00640 1 --2- 172.21.15.86:0/440160618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81a81579a0 0x7f81a8154120 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.369+0000 7f81afc00640 1 --2- 172.21.15.86:0/440160618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81a8156d90 0x7f81a8157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.369+0000 7f81afc00640 1 --2- 172.21.15.86:0/440160618 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f818406f400 0x7f8184071910 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.369+0000 7f81afc00640 1 --2- 172.21.15.86:0/440160618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f818c004810 0x7f81a8150370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.369+0000 7f81afc00640 1 -- 172.21.15.86:0/440160618 >> 172.21.15.86:0/440160618 conn(0x7f81a80d2230 msgr2=0x7f81a8166ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:45.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.369+0000 7f81afc00640 1 -- 172.21.15.86:0/440160618 shutdown_connections 2024-04-25T14:04:45.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.369+0000 7f81afc00640 1 -- 172.21.15.86:0/440160618 wait complete. 2024-04-25T14:04:45.391 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T14:04:45.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.537+0000 7f3e3aa00640 1 Processor -- start 2024-04-25T14:04:45.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.537+0000 7f3e3aa00640 1 -- start start 2024-04-25T14:04:45.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.541+0000 7f3e3aa00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3e340d2d70 0x7f3e34163780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:45.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.541+0000 7f3e3aa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e340d16b0 0x7f3e340d1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:45.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.541+0000 7f3e3aa00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e340cf9e0 con 0x7f3e34070e70 2024-04-25T14:04:45.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.541+0000 7f3e3aa00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3e34163e80 con 0x7f3e340d2d70 2024-04-25T14:04:45.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.541+0000 7f3e3aa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3e34164160 con 0x7f3e340d16b0 2024-04-25T14:04:45.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.541+0000 7f3e39600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3e340d2d70 0x7f3e34163780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:45.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.541+0000 7f3e38c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e340d16b0 0x7f3e340d1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:45.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.541+0000 7f3e39600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3e340d2d70 0x7f3e34163780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:38828/0 (socket says 172.21.15.86:38828) 2024-04-25T14:04:45.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.541+0000 7f3e39600640 1 -- 172.21.15.86:0/287738412 learned_addr learned my addr 172.21.15.86:0/287738412 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:45.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.541+0000 7f3e39600640 1 -- 172.21.15.86:0/287738412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e340d16b0 msgr2=0x7f3e340d1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:45.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.541+0000 7f3e39600640 1 --2- 172.21.15.86:0/287738412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e340d16b0 0x7f3e340d1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.541+0000 7f3e39600640 1 -- 172.21.15.86:0/287738412 >> v1:172.21.15.155:6789/0 conn(0x7f3e34070e70 legacy=0x7f3e34071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:45.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.541+0000 7f3e39600640 1 -- 172.21.15.86:0/287738412 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e34164440 con 0x7f3e340d2d70 2024-04-25T14:04:45.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.541+0000 7f3e33e00640 1 -- 172.21.15.86:0/287738412 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3580818983 0 0) 0x7f3e2c0014a0 con 0x7f3e34070e70 2024-04-25T14:04:45.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.545+0000 7f3e39600640 1 --2- 172.21.15.86:0/287738412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e340d2d70 0x7f3e34163780 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009870 tx=0x7f3e2801c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da5c5917e04c7969 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:45.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.545+0000 7f3e33e00640 1 -- 172.21.15.86:0/287738412 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e280b1070 con 0x7f3e340d2d70 2024-04-25T14:04:45.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.545+0000 7f3e33e00640 1 -- 172.21.15.86:0/287738412 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e280a8050 con 0x7f3e340d2d70 2024-04-25T14:04:45.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.545+0000 7f3e33e00640 1 -- 172.21.15.86:0/287738412 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e280ac860 con 0x7f3e340d2d70 2024-04-25T14:04:45.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.545+0000 7f3e3aa00640 1 -- 172.21.15.86:0/287738412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e340d2d70 msgr2=0x7f3e34163780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:45.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.545+0000 7f3e3aa00640 1 --2- 172.21.15.86:0/287738412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e340d2d70 0x7f3e34163780 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009870 tx=0x7f3e2801c4d0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.545+0000 7f3e3aa00640 1 -- 172.21.15.86:0/287738412 shutdown_connections 2024-04-25T14:04:45.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.545+0000 7f3e3aa00640 1 --2- 172.21.15.86:0/287738412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e340d16b0 0x7f3e340d1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.545+0000 7f3e3aa00640 1 --2- 172.21.15.86:0/287738412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e340d2d70 0x7f3e34163780 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.545+0000 7f3e3aa00640 1 -- 172.21.15.86:0/287738412 >> 172.21.15.86:0/287738412 conn(0x7f3e340cfea0 msgr2=0x7f3e340d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:45.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.545+0000 7f3e3aa00640 1 -- 172.21.15.86:0/287738412 shutdown_connections 2024-04-25T14:04:45.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.545+0000 7f3e3aa00640 1 -- 172.21.15.86:0/287738412 wait complete. 2024-04-25T14:04:45.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.545+0000 7f3e3aa00640 1 Processor -- start 2024-04-25T14:04:45.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.549+0000 7f3e3aa00640 1 -- start start 2024-04-25T14:04:45.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.549+0000 7f3e3aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e34070e70 0x7f3e3414f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:45.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.549+0000 7f3e3aa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e340d16b0 0x7f3e3406dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:45.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.549+0000 7f3e3aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e340d2d70 0x7f3e3406e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:45.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.549+0000 7f3e3aa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3416d660 con 0x7f3e340d2d70 2024-04-25T14:04:45.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.549+0000 7f3e3aa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3416d4e0 con 0x7f3e34070e70 2024-04-25T14:04:45.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.549+0000 7f3e3aa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3e3416d7e0 con 0x7f3e340d16b0 2024-04-25T14:04:45.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.549+0000 7f3e3a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e340d2d70 0x7f3e3406e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:45.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.549+0000 7f3e3a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e340d2d70 0x7f3e3406e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60818/0 (socket says 172.21.15.86:60818) 2024-04-25T14:04:45.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.549+0000 7f3e3a000640 1 -- 172.21.15.86:0/418261904 learned_addr learned my addr 172.21.15.86:0/418261904 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:45.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.549+0000 7f3e39600640 1 --2- 172.21.15.86:0/418261904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e34070e70 0x7f3e3414f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:45.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.549+0000 7f3e38c00640 1 --2- 172.21.15.86:0/418261904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e340d16b0 0x7f3e3406dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:45.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.549+0000 7f3e3a000640 1 -- 172.21.15.86:0/418261904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e340d16b0 msgr2=0x7f3e3406dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:45.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.549+0000 7f3e3a000640 1 --2- 172.21.15.86:0/418261904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e340d16b0 0x7f3e3406dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.549+0000 7f3e3a000640 1 -- 172.21.15.86:0/418261904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e34070e70 msgr2=0x7f3e3414f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:45.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.549+0000 7f3e3a000640 1 --2- 172.21.15.86:0/418261904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e34070e70 0x7f3e3414f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.549+0000 7f3e3a000640 1 -- 172.21.15.86:0/418261904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3414fe20 con 0x7f3e340d2d70 2024-04-25T14:04:45.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.549+0000 7f3e39600640 1 --2- 172.21.15.86:0/418261904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e34070e70 0x7f3e3414f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:45.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.549+0000 7f3e3a000640 1 --2- 172.21.15.86:0/418261904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e340d2d70 0x7f3e3406e2d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00c7b0 tx=0x7f3e2c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:45.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.549+0000 7f3e31600640 1 -- 172.21.15.86:0/418261904 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c00ce60 con 0x7f3e340d2d70 2024-04-25T14:04:45.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.549+0000 7f3e31600640 1 -- 172.21.15.86:0/418261904 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e2c003db0 con 0x7f3e340d2d70 2024-04-25T14:04:45.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.549+0000 7f3e31600640 1 -- 172.21.15.86:0/418261904 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c0052b0 con 0x7f3e340d2d70 2024-04-25T14:04:45.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.549+0000 7f3e3aa00640 1 -- 172.21.15.86:0/418261904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e340704a0 con 0x7f3e340d2d70 2024-04-25T14:04:45.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.549+0000 7f3e3aa00640 1 -- 172.21.15.86:0/418261904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e34155370 con 0x7f3e340d2d70 2024-04-25T14:04:45.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.549+0000 7f3e3aa00640 1 -- 172.21.15.86:0/418261904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e34076b80 con 0x7f3e340d2d70 2024-04-25T14:04:45.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.553+0000 7f3e31600640 1 -- 172.21.15.86:0/418261904 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3e2c005450 con 0x7f3e340d2d70 2024-04-25T14:04:45.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.553+0000 7f3e31600640 1 --2- 172.21.15.86:0/418261904 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3e1406f400 0x7f3e14071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:45.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.553+0000 7f3e31600640 1 -- 172.21.15.86:0/418261904 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f3e2c08fe00 con 0x7f3e340d2d70 2024-04-25T14:04:45.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.553+0000 7f3e39600640 1 --2- 172.21.15.86:0/418261904 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3e1406f400 0x7f3e14071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:45.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.553+0000 7f3e39600640 1 --2- 172.21.15.86:0/418261904 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3e1406f400 0x7f3e14071910 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3e280ae000 tx=0x7f3e280a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:45.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.557+0000 7f3e31600640 1 -- 172.21.15.86:0/418261904 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3e340cf9e0 con 0x7f3e340d2d70 2024-04-25T14:04:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.713+0000 7f3e3aa00640 1 -- 172.21.15.86:0/418261904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f3e340d23f0 con 0x7f3e340d2d70 2024-04-25T14:04:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.713+0000 7f3e31600640 1 -- 172.21.15.86:0/418261904 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f3e2c05c380 con 0x7f3e340d2d70 2024-04-25T14:04:45.714 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:04:45.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.713+0000 7f3e3aa00640 1 -- 172.21.15.86:0/418261904 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3e1406f400 msgr2=0x7f3e14071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.713+0000 7f3e3aa00640 1 --2- 172.21.15.86:0/418261904 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3e1406f400 0x7f3e14071910 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3e280ae000 tx=0x7f3e280a7000 comp rx=0 tx=0).stop 2024-04-25T14:04:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.713+0000 7f3e3aa00640 1 -- 172.21.15.86:0/418261904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e340d2d70 msgr2=0x7f3e3406e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.713+0000 7f3e3aa00640 1 --2- 172.21.15.86:0/418261904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e340d2d70 0x7f3e3406e2d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00c7b0 tx=0x7f3e2c00cc70 comp rx=0 tx=0).stop 2024-04-25T14:04:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.713+0000 7f3e3aa00640 1 -- 172.21.15.86:0/418261904 shutdown_connections 2024-04-25T14:04:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.713+0000 7f3e3aa00640 1 --2- 172.21.15.86:0/418261904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e340d2d70 0x7f3e3406e2d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.713+0000 7f3e3aa00640 1 --2- 172.21.15.86:0/418261904 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3e1406f400 0x7f3e14071910 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.713+0000 7f3e3aa00640 1 --2- 172.21.15.86:0/418261904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e340d16b0 0x7f3e3406dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.713+0000 7f3e3aa00640 1 --2- 172.21.15.86:0/418261904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e34070e70 0x7f3e3414f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.713+0000 7f3e3aa00640 1 -- 172.21.15.86:0/418261904 >> 172.21.15.86:0/418261904 conn(0x7f3e340cfea0 msgr2=0x7f3e3415f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.713+0000 7f3e3aa00640 1 -- 172.21.15.86:0/418261904 shutdown_connections 2024-04-25T14:04:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.713+0000 7f3e3aa00640 1 -- 172.21.15.86:0/418261904 wait complete. 2024-04-25T14:04:45.735 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T14:04:45.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.881+0000 7fea09000640 1 Processor -- start 2024-04-25T14:04:45.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.881+0000 7fea09000640 1 -- start start 2024-04-25T14:04:45.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.881+0000 7fea09000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fea04070330 0x7fea040d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:45.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.881+0000 7fea09000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fea040d1c20 0x7fea0406ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:45.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.881+0000 7fea09000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea040d2030 con 0x7fea0406f270 2024-04-25T14:04:45.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.881+0000 7fea09000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fea0415e5c0 con 0x7fea04070330 2024-04-25T14:04:45.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.881+0000 7fea09000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fea0415e8a0 con 0x7fea040d1c20 2024-04-25T14:04:45.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.881+0000 7fea01e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fea04070330 0x7fea040d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:45.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.881+0000 7fea01e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fea04070330 0x7fea040d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60826/0 (socket says 172.21.15.86:60826) 2024-04-25T14:04:45.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.881+0000 7fea01e00640 1 -- 172.21.15.86:0/2018646973 learned_addr learned my addr 172.21.15.86:0/2018646973 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:45.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.881+0000 7fe9fbe00640 1 --2- 172.21.15.86:0/2018646973 >> v2:172.21.15.155:3300/0 conn(0x7fea040d1c20 0x7fea0406ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:45.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.885+0000 7fea01e00640 1 -- 172.21.15.86:0/2018646973 >> v2:172.21.15.155:3300/0 conn(0x7fea040d1c20 msgr2=0x7fea0406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:45.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.885+0000 7fea01e00640 1 --2- 172.21.15.86:0/2018646973 >> v2:172.21.15.155:3300/0 conn(0x7fea040d1c20 0x7fea0406ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.885+0000 7fea01e00640 1 -- 172.21.15.86:0/2018646973 >> v1:172.21.15.155:6789/0 conn(0x7fea0406f270 legacy=0x7fea0406f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:45.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.885+0000 7fea01e00640 1 -- 172.21.15.86:0/2018646973 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea0415eb80 con 0x7fea04070330 2024-04-25T14:04:45.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.885+0000 7fea01e00640 1 --2- 172.21.15.86:0/2018646973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea04070330 0x7fea040d16e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009870 tx=0x7fe9f401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d1cf7323d0836b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:45.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.885+0000 7fea01400640 1 -- 172.21.15.86:0/2018646973 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f40b0070 con 0x7fea04070330 2024-04-25T14:04:45.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.885+0000 7fea01400640 1 -- 172.21.15.86:0/2018646973 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f40ab600 con 0x7fea04070330 2024-04-25T14:04:45.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.885+0000 7fea01400640 1 -- 172.21.15.86:0/2018646973 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f40ab9d0 con 0x7fea04070330 2024-04-25T14:04:45.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.889+0000 7fea09000640 1 -- 172.21.15.86:0/2018646973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea04070330 msgr2=0x7fea040d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:45.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.889+0000 7fea09000640 1 --2- 172.21.15.86:0/2018646973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea04070330 0x7fea040d16e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009870 tx=0x7fe9f401c490 comp rx=0 tx=0).stop 2024-04-25T14:04:45.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.889+0000 7fea09000640 1 -- 172.21.15.86:0/2018646973 shutdown_connections 2024-04-25T14:04:45.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.889+0000 7fea09000640 1 --2- 172.21.15.86:0/2018646973 >> v2:172.21.15.155:3300/0 conn(0x7fea040d1c20 0x7fea0406ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.889+0000 7fea09000640 1 --2- 172.21.15.86:0/2018646973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea04070330 0x7fea040d16e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.889+0000 7fea09000640 1 -- 172.21.15.86:0/2018646973 >> 172.21.15.86:0/2018646973 conn(0x7fea040b4650 msgr2=0x7fea040b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:45.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.889+0000 7fea09000640 1 -- 172.21.15.86:0/2018646973 shutdown_connections 2024-04-25T14:04:45.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.889+0000 7fea09000640 1 -- 172.21.15.86:0/2018646973 wait complete. 2024-04-25T14:04:45.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.889+0000 7fea09000640 1 Processor -- start 2024-04-25T14:04:45.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.889+0000 7fea09000640 1 -- start start 2024-04-25T14:04:45.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.893+0000 7fea09000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea0406f270 0x7fea040d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:45.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.893+0000 7fea09000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea04070330 0x7fea04156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:45.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.893+0000 7fea09000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea040d1c20 0x7fea040c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:45.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.893+0000 7fea09000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea040708c0 con 0x7fea04070330 2024-04-25T14:04:45.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.893+0000 7fea09000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fea04070740 con 0x7fea0406f270 2024-04-25T14:04:45.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.893+0000 7fea09000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fea0416afc0 con 0x7fea040d1c20 2024-04-25T14:04:45.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.893+0000 7fe9fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea04070330 0x7fea04156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:45.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.893+0000 7fe9fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea04070330 0x7fea04156700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60828/0 (socket says 172.21.15.86:60828) 2024-04-25T14:04:45.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.893+0000 7fe9fbe00640 1 -- 172.21.15.86:0/2405256848 learned_addr learned my addr 172.21.15.86:0/2405256848 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:45.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.893+0000 7fea01e00640 1 --2- 172.21.15.86:0/2405256848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea0406f270 0x7fea040d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:45.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.893+0000 7fea02800640 1 --2- 172.21.15.86:0/2405256848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea040d1c20 0x7fea040c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:45.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.893+0000 7fe9fbe00640 1 -- 172.21.15.86:0/2405256848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea040d1c20 msgr2=0x7fea040c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:45.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.893+0000 7fe9fbe00640 1 --2- 172.21.15.86:0/2405256848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea040d1c20 0x7fea040c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.893+0000 7fe9fbe00640 1 -- 172.21.15.86:0/2405256848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea0406f270 msgr2=0x7fea040d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:45.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.893+0000 7fe9fbe00640 1 --2- 172.21.15.86:0/2405256848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea0406f270 0x7fea040d02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:45.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.893+0000 7fe9fbe00640 1 -- 172.21.15.86:0/2405256848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea040d07e0 con 0x7fea04070330 2024-04-25T14:04:45.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.893+0000 7fea01e00640 1 --2- 172.21.15.86:0/2405256848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea0406f270 0x7fea040d02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:45.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.893+0000 7fe9fbe00640 1 --2- 172.21.15.86:0/2405256848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea04070330 0x7fea04156700 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00e8a0 tx=0x7fe9ec00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:45.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.893+0000 7fe9fa000640 1 -- 172.21.15.86:0/2405256848 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec017070 con 0x7fea04070330 2024-04-25T14:04:45.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.893+0000 7fe9fa000640 1 -- 172.21.15.86:0/2405256848 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9ec00ee90 con 0x7fea04070330 2024-04-25T14:04:45.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.893+0000 7fe9fa000640 1 -- 172.21.15.86:0/2405256848 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec00cb40 con 0x7fea04070330 2024-04-25T14:04:45.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.893+0000 7fea09000640 1 -- 172.21.15.86:0/2405256848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea040cb1d0 con 0x7fea04070330 2024-04-25T14:04:45.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.893+0000 7fea09000640 1 -- 172.21.15.86:0/2405256848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea040ca0e0 con 0x7fea04070330 2024-04-25T14:04:45.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.893+0000 7fea09000640 1 -- 172.21.15.86:0/2405256848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea040b0120 con 0x7fea04070330 2024-04-25T14:04:45.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.893+0000 7fe9fa000640 1 -- 172.21.15.86:0/2405256848 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe9ec01a070 con 0x7fea04070330 2024-04-25T14:04:45.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.897+0000 7fe9fa000640 1 --2- 172.21.15.86:0/2405256848 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe9e006f400 0x7fe9e0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:45.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.897+0000 7fe9fa000640 1 -- 172.21.15.86:0/2405256848 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe9ec08f650 con 0x7fea04070330 2024-04-25T14:04:45.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.897+0000 7fea01e00640 1 --2- 172.21.15.86:0/2405256848 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe9e006f400 0x7fe9e0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:45.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.897+0000 7fea01e00640 1 --2- 172.21.15.86:0/2405256848 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe9e006f400 0x7fe9e0071910 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe9f40ad000 tx=0x7fe9f40a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:45.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:45.901+0000 7fe9fa000640 1 -- 172.21.15.86:0/2405256848 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fea040b0120 con 0x7fea04070330 2024-04-25T14:04:46.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.069+0000 7fea09000640 1 -- 172.21.15.86:0/2405256848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fea04069d00 con 0x7fea04070330 2024-04-25T14:04:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.069+0000 7fe9fa000640 1 -- 172.21.15.86:0/2405256848 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fe9ec05c0a0 con 0x7fea04070330 2024-04-25T14:04:46.072 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-25T14:04:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.073+0000 7fea09000640 1 -- 172.21.15.86:0/2405256848 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe9e006f400 msgr2=0x7fe9e0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.073+0000 7fea09000640 1 --2- 172.21.15.86:0/2405256848 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe9e006f400 0x7fe9e0071910 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe9f40ad000 tx=0x7fe9f40a6000 comp rx=0 tx=0).stop 2024-04-25T14:04:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.073+0000 7fea09000640 1 -- 172.21.15.86:0/2405256848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea04070330 msgr2=0x7fea04156700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.073+0000 7fea09000640 1 --2- 172.21.15.86:0/2405256848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea04070330 0x7fea04156700 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00e8a0 tx=0x7fe9ec00ed60 comp rx=0 tx=0).stop 2024-04-25T14:04:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.073+0000 7fea09000640 1 -- 172.21.15.86:0/2405256848 shutdown_connections 2024-04-25T14:04:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.073+0000 7fea09000640 1 --2- 172.21.15.86:0/2405256848 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe9e006f400 0x7fe9e0071910 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.073+0000 7fea09000640 1 --2- 172.21.15.86:0/2405256848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea040d1c20 0x7fea040c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.073+0000 7fea09000640 1 --2- 172.21.15.86:0/2405256848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea04070330 0x7fea04156700 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.073+0000 7fea09000640 1 --2- 172.21.15.86:0/2405256848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea0406f270 0x7fea040d02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.073+0000 7fea09000640 1 -- 172.21.15.86:0/2405256848 >> 172.21.15.86:0/2405256848 conn(0x7fea040b4650 msgr2=0x7fea0415abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.073+0000 7fea09000640 1 -- 172.21.15.86:0/2405256848 shutdown_connections 2024-04-25T14:04:46.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.073+0000 7fea09000640 1 -- 172.21.15.86:0/2405256848 wait complete. 2024-04-25T14:04:46.093 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T14:04:46.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:46.093 DEBUG:teuthology.orchestra.run.smithi155:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-25T14:04:46.134 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T14:04:46.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:46.135 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T14:04:46.191 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:04:46.191 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:04:46.200 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:04:46.200 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:04:46.248 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:04:46.248 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:04:46.296 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:04:46.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.481+0000 7f5adb200640 1 Processor -- start 2024-04-25T14:04:46.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.481+0000 7f5adb200640 1 -- start start 2024-04-25T14:04:46.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.485+0000 7f5adb200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5ad41610d0 0x7f5ad40b7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.485+0000 7f5adb200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5ad40b2250 0x7f5ad40b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.485+0000 7f5adb200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ad40b3750 con 0x7f5ad40b2ba0 2024-04-25T14:04:46.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.485+0000 7f5adb200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5ad40b7f70 con 0x7f5ad41610d0 2024-04-25T14:04:46.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.485+0000 7f5adb200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f5ad4161620 con 0x7f5ad40b2250 2024-04-25T14:04:46.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.485+0000 7f5ad3e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5ad41610d0 0x7f5ad40b7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.485+0000 7f5ad3e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5ad41610d0 0x7f5ad40b7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60832/0 (socket says 172.21.15.86:60832) 2024-04-25T14:04:46.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.485+0000 7f5ad9200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5ad40b2ba0 0x7f5ad40d15e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54908/0 (socket says 172.21.15.86:54908) 2024-04-25T14:04:46.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.485+0000 7f5ad3e00640 1 -- 172.21.15.86:0/1969526659 learned_addr learned my addr 172.21.15.86:0/1969526659 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:46.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.485+0000 7f5acb400640 1 --2- 172.21.15.86:0/1969526659 >> v2:172.21.15.155:3300/0 conn(0x7f5ad40b2250 0x7f5ad40b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.485+0000 7f5ad3e00640 1 -- 172.21.15.86:0/1969526659 >> v2:172.21.15.155:3300/0 conn(0x7f5ad40b2250 msgr2=0x7f5ad40b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.485+0000 7f5ad3e00640 1 --2- 172.21.15.86:0/1969526659 >> v2:172.21.15.155:3300/0 conn(0x7f5ad40b2250 0x7f5ad40b2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.485+0000 7f5ad3e00640 1 -- 172.21.15.86:0/1969526659 >> v1:172.21.15.86:6789/0 conn(0x7f5ad40b2ba0 legacy=0x7f5ad40d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.485+0000 7f5ad3e00640 1 -- 172.21.15.86:0/1969526659 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad40ce7b0 con 0x7f5ad41610d0 2024-04-25T14:04:46.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.485+0000 7f5ad3400640 1 -- 172.21.15.86:0/1969526659 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3887366034 0 0) 0x7f5ad40b3750 con 0x7f5ad40b2ba0 2024-04-25T14:04:46.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.485+0000 7f5ad3e00640 1 --2- 172.21.15.86:0/1969526659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ad41610d0 0x7f5ad40b7a30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5ad40b52f0 tx=0x7f5ac401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353c6bcbcea146e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.489+0000 7f5ad3400640 1 -- 172.21.15.86:0/1969526659 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac40a55f0 con 0x7f5ad41610d0 2024-04-25T14:04:46.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.489+0000 7f5ad3400640 1 -- 172.21.15.86:0/1969526659 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac40a5790 con 0x7f5ad41610d0 2024-04-25T14:04:46.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.489+0000 7f5ad3400640 1 -- 172.21.15.86:0/1969526659 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac40a5b60 con 0x7f5ad41610d0 2024-04-25T14:04:46.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.489+0000 7f5adb200640 1 -- 172.21.15.86:0/1969526659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ad41610d0 msgr2=0x7f5ad40b7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.489+0000 7f5adb200640 1 --2- 172.21.15.86:0/1969526659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ad41610d0 0x7f5ad40b7a30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5ad40b52f0 tx=0x7f5ac401c930 comp rx=0 tx=0).stop 2024-04-25T14:04:46.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.489+0000 7f5adb200640 1 -- 172.21.15.86:0/1969526659 shutdown_connections 2024-04-25T14:04:46.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.489+0000 7f5adb200640 1 --2- 172.21.15.86:0/1969526659 >> v2:172.21.15.155:3300/0 conn(0x7f5ad40b2250 0x7f5ad40b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.489+0000 7f5adb200640 1 --2- 172.21.15.86:0/1969526659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ad41610d0 0x7f5ad40b7a30 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.489+0000 7f5adb200640 1 -- 172.21.15.86:0/1969526659 >> 172.21.15.86:0/1969526659 conn(0x7f5ad409ba30 msgr2=0x7f5ad40b7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.489+0000 7f5adb200640 1 -- 172.21.15.86:0/1969526659 shutdown_connections 2024-04-25T14:04:46.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.489+0000 7f5adb200640 1 -- 172.21.15.86:0/1969526659 wait complete. 2024-04-25T14:04:46.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.489+0000 7f5adb200640 1 Processor -- start 2024-04-25T14:04:46.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.489+0000 7f5adb200640 1 -- start start 2024-04-25T14:04:46.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.493+0000 7f5adb200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ad40b2250 0x7f5ad4151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.493+0000 7f5adb200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ad40b2ba0 0x7f5ad4153330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.493+0000 7f5adb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ad41610d0 0x7f5ad4156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.493+0000 7f5adb200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad40d2280 con 0x7f5ad41610d0 2024-04-25T14:04:46.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.493+0000 7f5adb200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad40d2100 con 0x7f5ad40b2250 2024-04-25T14:04:46.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.493+0000 7f5adb200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5ad40cf610 con 0x7f5ad40b2ba0 2024-04-25T14:04:46.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.493+0000 7f5ad9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ad41610d0 0x7f5ad4156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.493+0000 7f5ad9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ad41610d0 0x7f5ad4156a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60838/0 (socket says 172.21.15.86:60838) 2024-04-25T14:04:46.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.493+0000 7f5ad9200640 1 -- 172.21.15.86:0/883308356 learned_addr learned my addr 172.21.15.86:0/883308356 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:46.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.493+0000 7f5ad3e00640 1 --2- 172.21.15.86:0/883308356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ad40b2250 0x7f5ad4151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.493+0000 7f5acb400640 1 --2- 172.21.15.86:0/883308356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ad40b2ba0 0x7f5ad4153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.493+0000 7f5ad9200640 1 -- 172.21.15.86:0/883308356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ad40b2ba0 msgr2=0x7f5ad4153330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.493+0000 7f5ad9200640 1 --2- 172.21.15.86:0/883308356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ad40b2ba0 0x7f5ad4153330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.493+0000 7f5ad9200640 1 -- 172.21.15.86:0/883308356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ad40b2250 msgr2=0x7f5ad4151e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.493+0000 7f5ad9200640 1 --2- 172.21.15.86:0/883308356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ad40b2250 0x7f5ad4151e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.493+0000 7f5ad9200640 1 -- 172.21.15.86:0/883308356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad415acf0 con 0x7f5ad41610d0 2024-04-25T14:04:46.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.493+0000 7f5acb400640 1 --2- 172.21.15.86:0/883308356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ad40b2ba0 0x7f5ad4153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:46.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.493+0000 7f5ad9200640 1 --2- 172.21.15.86:0/883308356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ad41610d0 0x7f5ad4156a40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00c410 tx=0x7f5acc00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.493+0000 7f5ad0c00640 1 -- 172.21.15.86:0/883308356 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc0074e0 con 0x7f5ad41610d0 2024-04-25T14:04:46.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.493+0000 7f5ad0c00640 1 -- 172.21.15.86:0/883308356 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5acc007680 con 0x7f5ad41610d0 2024-04-25T14:04:46.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.493+0000 7f5ad0c00640 1 -- 172.21.15.86:0/883308356 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc004a10 con 0x7f5ad41610d0 2024-04-25T14:04:46.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.493+0000 7f5adb200640 1 -- 172.21.15.86:0/883308356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad415e3e0 con 0x7f5ad41610d0 2024-04-25T14:04:46.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.493+0000 7f5adb200640 1 -- 172.21.15.86:0/883308356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad4152390 con 0x7f5ad41610d0 2024-04-25T14:04:46.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.493+0000 7f5adb200640 1 -- 172.21.15.86:0/883308356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ad415e900 con 0x7f5ad41610d0 2024-04-25T14:04:46.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.497+0000 7f5ad0c00640 1 -- 172.21.15.86:0/883308356 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5acc01a020 con 0x7f5ad41610d0 2024-04-25T14:04:46.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.497+0000 7f5ad0c00640 1 --2- 172.21.15.86:0/883308356 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f5ab006f430 0x7f5ab0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.497+0000 7f5ad0c00640 1 -- 172.21.15.86:0/883308356 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f5acc08eca0 con 0x7f5ad41610d0 2024-04-25T14:04:46.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.497+0000 7f5ad3e00640 1 --2- 172.21.15.86:0/883308356 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f5ab006f430 0x7f5ab0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.497+0000 7f5ad3e00640 1 --2- 172.21.15.86:0/883308356 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f5ab006f430 0x7f5ab0071940 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5ac40ae000 tx=0x7f5ac40a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.501+0000 7f5ad0c00640 1 -- 172.21.15.86:0/883308356 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5acc05b810 con 0x7f5ad41610d0 2024-04-25T14:04:46.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.657+0000 7f5adb200640 1 -- 172.21.15.86:0/883308356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ad415e170 con 0x7f5ad41610d0 2024-04-25T14:04:46.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.661+0000 7f5ad0c00640 1 -- 172.21.15.86:0/883308356 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18902 (secure 0 0 0) 0x7f5acc05b110 con 0x7f5ad41610d0 2024-04-25T14:04:46.661 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:46.661 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":26,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:04:31.930803+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:03:49.151473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:03:49.802597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T14:03:53.492008+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:04:46.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.661+0000 7f5adb200640 1 -- 172.21.15.86:0/883308356 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f5ab006f430 msgr2=0x7f5ab0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.661+0000 7f5adb200640 1 --2- 172.21.15.86:0/883308356 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f5ab006f430 0x7f5ab0071940 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5ac40ae000 tx=0x7f5ac40a7070 comp rx=0 tx=0).stop 2024-04-25T14:04:46.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.661+0000 7f5adb200640 1 -- 172.21.15.86:0/883308356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ad41610d0 msgr2=0x7f5ad4156a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.661+0000 7f5adb200640 1 --2- 172.21.15.86:0/883308356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ad41610d0 0x7f5ad4156a40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00c410 tx=0x7f5acc00c810 comp rx=0 tx=0).stop 2024-04-25T14:04:46.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.661+0000 7f5adb200640 1 -- 172.21.15.86:0/883308356 shutdown_connections 2024-04-25T14:04:46.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.661+0000 7f5adb200640 1 --2- 172.21.15.86:0/883308356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ad41610d0 0x7f5ad4156a40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.661+0000 7f5adb200640 1 --2- 172.21.15.86:0/883308356 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f5ab006f430 0x7f5ab0071940 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.661+0000 7f5adb200640 1 --2- 172.21.15.86:0/883308356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ad40b2ba0 0x7f5ad4153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.661+0000 7f5adb200640 1 --2- 172.21.15.86:0/883308356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ad40b2250 0x7f5ad4151e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.661+0000 7f5adb200640 1 -- 172.21.15.86:0/883308356 >> 172.21.15.86:0/883308356 conn(0x7f5ad409ba30 msgr2=0x7f5ad40d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.661+0000 7f5adb200640 1 -- 172.21.15.86:0/883308356 shutdown_connections 2024-04-25T14:04:46.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.661+0000 7f5adb200640 1 -- 172.21.15.86:0/883308356 wait complete. 2024-04-25T14:04:46.682 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:03:49.151473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:03:49.802597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T14:03:53.492008+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T14:04:46.683 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:04:46.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.833+0000 7f0579400640 1 Processor -- start 2024-04-25T14:04:46.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.833+0000 7f0579400640 1 -- start start 2024-04-25T14:04:46.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.833+0000 7f0579400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05740aeea0 0x7f05740af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.833+0000 7f0579400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f057406c670 con 0x7f057415faa0 2024-04-25T14:04:46.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.833+0000 7f0579400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f057406c950 con 0x7f05740b59e0 2024-04-25T14:04:46.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.833+0000 7f0579400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f057406d560 con 0x7f05740aeea0 2024-04-25T14:04:46.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.833+0000 7f0572800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f057415faa0 0x7f057406d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54922/0 (socket says 172.21.15.86:54922) 2024-04-25T14:04:46.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.833+0000 7f0572800640 1 -- 172.21.15.86:0/3418875631 learned_addr learned my addr 172.21.15.86:0/3418875631 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:46.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.833+0000 7f0573200640 1 --2- 172.21.15.86:0/3418875631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05740aeea0 0x7f05740af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.833+0000 7f0571400640 1 -- 172.21.15.86:0/3418875631 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 815778505 0 0) 0x7f057406c670 con 0x7f057415faa0 2024-04-25T14:04:46.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.833+0000 7f0571400640 1 -- 172.21.15.86:0/3418875631 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0558003530 con 0x7f057415faa0 2024-04-25T14:04:46.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.833+0000 7f0571400640 1 -- 172.21.15.86:0/3418875631 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 378280549 0 0) 0x7f057406c950 con 0x7f05740b59e0 2024-04-25T14:04:46.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.833+0000 7f0571400640 1 -- 172.21.15.86:0/3418875631 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f057406c670 con 0x7f05740b59e0 2024-04-25T14:04:46.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.833+0000 7f0573200640 1 -- 172.21.15.86:0/3418875631 >> v1:172.21.15.155:6789/0 conn(0x7f05740b59e0 legacy=0x7f05740b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.833+0000 7f0573200640 1 -- 172.21.15.86:0/3418875631 >> v1:172.21.15.86:6789/0 conn(0x7f057415faa0 legacy=0x7f057406d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.833+0000 7f0573200640 1 -- 172.21.15.86:0/3418875631 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f057406cc30 con 0x7f05740aeea0 2024-04-25T14:04:46.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.837+0000 7f0573200640 1 --2- 172.21.15.86:0/3418875631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05740aeea0 0x7f05740af2b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f056c009dc0 tx=0x7f056c01c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b965f1b6422b0d9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.837+0000 7f0571400640 1 -- 172.21.15.86:0/3418875631 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c0b2070 con 0x7f05740aeea0 2024-04-25T14:04:46.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.837+0000 7f0571400640 1 -- 172.21.15.86:0/3418875631 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f056c0a5a70 con 0x7f05740aeea0 2024-04-25T14:04:46.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.837+0000 7f0571400640 1 -- 172.21.15.86:0/3418875631 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c0a5e20 con 0x7f05740aeea0 2024-04-25T14:04:46.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.837+0000 7f0579400640 1 -- 172.21.15.86:0/3418875631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05740aeea0 msgr2=0x7f05740af2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.837+0000 7f0579400640 1 --2- 172.21.15.86:0/3418875631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05740aeea0 0x7f05740af2b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f056c009dc0 tx=0x7f056c01c990 comp rx=0 tx=0).stop 2024-04-25T14:04:46.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.841+0000 7f0579400640 1 -- 172.21.15.86:0/3418875631 shutdown_connections 2024-04-25T14:04:46.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.841+0000 7f0579400640 1 --2- 172.21.15.86:0/3418875631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05740aeea0 0x7f05740af2b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.841+0000 7f0579400640 1 -- 172.21.15.86:0/3418875631 >> 172.21.15.86:0/3418875631 conn(0x7f05740d2230 msgr2=0x7f05740d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.841+0000 7f0579400640 1 -- 172.21.15.86:0/3418875631 shutdown_connections 2024-04-25T14:04:46.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.841+0000 7f0579400640 1 -- 172.21.15.86:0/3418875631 wait complete. 2024-04-25T14:04:46.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.841+0000 7f0579400640 1 Processor -- start 2024-04-25T14:04:46.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.841+0000 7f0579400640 1 -- start start 2024-04-25T14:04:46.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.841+0000 7f0579400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05740aeea0 0x7f0574140630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.841+0000 7f0579400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05740b59e0 0x7f057415ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.841+0000 7f0579400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f057415faa0 0x7f057415bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.841+0000 7f0579400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05740710b0 con 0x7f05740b59e0 2024-04-25T14:04:46.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.841+0000 7f0579400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0574070f30 con 0x7f05740aeea0 2024-04-25T14:04:46.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.841+0000 7f0579400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f057416e780 con 0x7f057415faa0 2024-04-25T14:04:46.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.845+0000 7f0571e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05740b59e0 0x7f057415ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.845+0000 7f0571e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05740b59e0 0x7f057415ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60844/0 (socket says 172.21.15.86:60844) 2024-04-25T14:04:46.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.845+0000 7f0571e00640 1 -- 172.21.15.86:0/3568387582 learned_addr learned my addr 172.21.15.86:0/3568387582 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:46.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.845+0000 7f0572800640 1 --2- 172.21.15.86:0/3568387582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05740aeea0 0x7f0574140630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.845+0000 7f0573200640 1 --2- 172.21.15.86:0/3568387582 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f057415faa0 0x7f057415bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.845+0000 7f0571e00640 1 -- 172.21.15.86:0/3568387582 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f057415faa0 msgr2=0x7f057415bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.845+0000 7f0571e00640 1 --2- 172.21.15.86:0/3568387582 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f057415faa0 0x7f057415bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.845+0000 7f0571e00640 1 -- 172.21.15.86:0/3568387582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05740aeea0 msgr2=0x7f0574140630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.845+0000 7f0571e00640 1 --2- 172.21.15.86:0/3568387582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05740aeea0 0x7f0574140630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.845+0000 7f0571e00640 1 -- 172.21.15.86:0/3568387582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0574140b70 con 0x7f05740b59e0 2024-04-25T14:04:46.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.845+0000 7f0571e00640 1 --2- 172.21.15.86:0/3568387582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05740b59e0 0x7f057415ba50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f056000c460 tx=0x7f056000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.845+0000 7f056aa00640 1 -- 172.21.15.86:0/3568387582 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05600052d0 con 0x7f05740b59e0 2024-04-25T14:04:46.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.845+0000 7f056aa00640 1 -- 172.21.15.86:0/3568387582 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0560005490 con 0x7f05740b59e0 2024-04-25T14:04:46.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.845+0000 7f056aa00640 1 -- 172.21.15.86:0/3568387582 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0560012850 con 0x7f05740b59e0 2024-04-25T14:04:46.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.845+0000 7f0579400640 1 -- 172.21.15.86:0/3568387582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0574161b40 con 0x7f05740b59e0 2024-04-25T14:04:46.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.845+0000 7f0579400640 1 -- 172.21.15.86:0/3568387582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0574162100 con 0x7f05740b59e0 2024-04-25T14:04:46.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.845+0000 7f0579400640 1 -- 172.21.15.86:0/3568387582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0574159990 con 0x7f05740b59e0 2024-04-25T14:04:46.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.845+0000 7f056aa00640 1 -- 172.21.15.86:0/3568387582 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f05600129f0 con 0x7f05740b59e0 2024-04-25T14:04:46.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.849+0000 7f056aa00640 1 --2- 172.21.15.86:0/3568387582 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f055406f400 0x7f0554071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.849+0000 7f056aa00640 1 -- 172.21.15.86:0/3568387582 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f056008f110 con 0x7f05740b59e0 2024-04-25T14:04:46.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.849+0000 7f0572800640 1 --2- 172.21.15.86:0/3568387582 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f055406f400 0x7f0554071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.849+0000 7f0572800640 1 --2- 172.21.15.86:0/3568387582 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f055406f400 0x7f0554071910 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f05640011f0 tx=0x7f0564007d40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:46.853+0000 7f056aa00640 1 -- 172.21.15.86:0/3568387582 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f056005b630 con 0x7f05740b59e0 2024-04-25T14:04:47.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.009+0000 7f0579400640 1 -- 172.21.15.86:0/3568387582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0574074cf0 con 0x7f05740b59e0 2024-04-25T14:04:47.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.013+0000 7f056aa00640 1 -- 172.21.15.86:0/3568387582 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f056005f930 con 0x7f05740b59e0 2024-04-25T14:04:47.075 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:04:47.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.013+0000 7f0579400640 1 -- 172.21.15.86:0/3568387582 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f055406f400 msgr2=0x7f0554071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.013+0000 7f0579400640 1 --2- 172.21.15.86:0/3568387582 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f055406f400 0x7f0554071910 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f05640011f0 tx=0x7f0564007d40 comp rx=0 tx=0).stop 2024-04-25T14:04:47.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.013+0000 7f0579400640 1 -- 172.21.15.86:0/3568387582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05740b59e0 msgr2=0x7f057415ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.013+0000 7f0579400640 1 --2- 172.21.15.86:0/3568387582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05740b59e0 0x7f057415ba50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f056000c460 tx=0x7f056000c920 comp rx=0 tx=0).stop 2024-04-25T14:04:47.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.013+0000 7f0579400640 1 -- 172.21.15.86:0/3568387582 shutdown_connections 2024-04-25T14:04:47.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.013+0000 7f0579400640 1 --2- 172.21.15.86:0/3568387582 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f055406f400 0x7f0554071910 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.013+0000 7f0579400640 1 --2- 172.21.15.86:0/3568387582 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f057415faa0 0x7f057415bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.013+0000 7f0579400640 1 --2- 172.21.15.86:0/3568387582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05740b59e0 0x7f057415ba50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.013+0000 7f0579400640 1 --2- 172.21.15.86:0/3568387582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05740aeea0 0x7f0574140630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.013+0000 7f0579400640 1 -- 172.21.15.86:0/3568387582 >> 172.21.15.86:0/3568387582 conn(0x7f05740d2230 msgr2=0x7f05740b2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:47.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.013+0000 7f0579400640 1 -- 172.21.15.86:0/3568387582 shutdown_connections 2024-04-25T14:04:47.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.017+0000 7f0579400640 1 -- 172.21.15.86:0/3568387582 wait complete. 2024-04-25T14:04:47.078 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:04:47.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.225+0000 7f7492800640 1 Processor -- start 2024-04-25T14:04:47.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.225+0000 7f7492800640 1 -- start start 2024-04-25T14:04:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.225+0000 7f7492800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f748c0b4440 0x7f748c163920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.225+0000 7f7492800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f748c0b4a30 con 0x7f748c0d3e70 2024-04-25T14:04:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.225+0000 7f7492800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f748c06d9c0 con 0x7f748c0d1200 2024-04-25T14:04:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.225+0000 7f7492800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f748c0b4d10 con 0x7f748c0b4440 2024-04-25T14:04:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.225+0000 7f748be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f748c0d3e70 0x7f748c163e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54936/0 (socket says 172.21.15.86:54936) 2024-04-25T14:04:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.225+0000 7f748be00640 1 -- 172.21.15.86:0/2130083207 learned_addr learned my addr 172.21.15.86:0/2130083207 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.225+0000 7f7482a00640 1 --2- 172.21.15.86:0/2130083207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f748c0b4440 0x7f748c163920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.229+0000 7f748aa00640 1 -- 172.21.15.86:0/2130083207 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1777539826 0 0) 0x7f748c0b4a30 con 0x7f748c0d3e70 2024-04-25T14:04:47.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.229+0000 7f748aa00640 1 -- 172.21.15.86:0/2130083207 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7470003530 con 0x7f748c0d3e70 2024-04-25T14:04:47.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.229+0000 7f7482a00640 1 -- 172.21.15.86:0/2130083207 >> v1:172.21.15.155:6789/0 conn(0x7f748c0d1200 legacy=0x7f748c0d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.229+0000 7f748aa00640 1 -- 172.21.15.86:0/2130083207 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4027976856 0 0) 0x7f748c06d9c0 con 0x7f748c0d1200 2024-04-25T14:04:47.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.229+0000 7f7482a00640 1 -- 172.21.15.86:0/2130083207 >> v1:172.21.15.86:6789/0 conn(0x7f748c0d3e70 legacy=0x7f748c163e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.229+0000 7f7482a00640 1 -- 172.21.15.86:0/2130083207 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f748c06dca0 con 0x7f748c0b4440 2024-04-25T14:04:47.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.229+0000 7f7482a00640 1 --2- 172.21.15.86:0/2130083207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f748c0b4440 0x7f748c163920 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f74740097f0 tx=0x7f747401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5aaacd889c8de834 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.229+0000 7f748aa00640 1 -- 172.21.15.86:0/2130083207 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74740b0070 con 0x7f748c0b4440 2024-04-25T14:04:47.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.229+0000 7f748aa00640 1 -- 172.21.15.86:0/2130083207 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74740ab600 con 0x7f748c0b4440 2024-04-25T14:04:47.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.229+0000 7f748aa00640 1 -- 172.21.15.86:0/2130083207 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74740ab9d0 con 0x7f748c0b4440 2024-04-25T14:04:47.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.233+0000 7f7492800640 1 -- 172.21.15.86:0/2130083207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f748c0b4440 msgr2=0x7f748c163920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.233+0000 7f7492800640 1 --2- 172.21.15.86:0/2130083207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f748c0b4440 0x7f748c163920 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f74740097f0 tx=0x7f747401c450 comp rx=0 tx=0).stop 2024-04-25T14:04:47.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.233+0000 7f7492800640 1 -- 172.21.15.86:0/2130083207 shutdown_connections 2024-04-25T14:04:47.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.233+0000 7f7492800640 1 --2- 172.21.15.86:0/2130083207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f748c0b4440 0x7f748c163920 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.233+0000 7f7492800640 1 -- 172.21.15.86:0/2130083207 >> 172.21.15.86:0/2130083207 conn(0x7f748c0b72d0 msgr2=0x7f748c0b76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:47.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.233+0000 7f7492800640 1 -- 172.21.15.86:0/2130083207 shutdown_connections 2024-04-25T14:04:47.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.233+0000 7f7492800640 1 -- 172.21.15.86:0/2130083207 wait complete. 2024-04-25T14:04:47.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.233+0000 7f7492800640 1 Processor -- start 2024-04-25T14:04:47.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.233+0000 7f7492800640 1 -- start start 2024-04-25T14:04:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.237+0000 7f7492800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f748c0b4440 0x7f748c14b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.237+0000 7f7492800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f748c0d1200 0x7f748c14d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.237+0000 7f7492800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f748c0d3e70 0x7f748c160300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.237+0000 7f7492800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f748c16e4e0 con 0x7f748c0d1200 2024-04-25T14:04:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.237+0000 7f7492800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f748c0d27b0 con 0x7f748c0d3e70 2024-04-25T14:04:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.237+0000 7f7492800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f748c16e660 con 0x7f748c0b4440 2024-04-25T14:04:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.237+0000 7f7482a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f748c0d1200 0x7f748c14d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.237+0000 7f7482a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f748c0d1200 0x7f748c14d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60852/0 (socket says 172.21.15.86:60852) 2024-04-25T14:04:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.237+0000 7f7482a00640 1 -- 172.21.15.86:0/3518066564 learned_addr learned my addr 172.21.15.86:0/3518066564 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.237+0000 7f748b400640 1 --2- 172.21.15.86:0/3518066564 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f748c0b4440 0x7f748c14b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.237+0000 7f748be00640 1 --2- 172.21.15.86:0/3518066564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f748c0d3e70 0x7f748c160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.237+0000 7f7482a00640 1 -- 172.21.15.86:0/3518066564 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f748c0b4440 msgr2=0x7f748c14b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.237+0000 7f7482a00640 1 --2- 172.21.15.86:0/3518066564 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f748c0b4440 0x7f748c14b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.237+0000 7f7482a00640 1 -- 172.21.15.86:0/3518066564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f748c0d3e70 msgr2=0x7f748c160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.237+0000 7f7482a00640 1 --2- 172.21.15.86:0/3518066564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f748c0d3e70 0x7f748c160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.237+0000 7f7482a00640 1 -- 172.21.15.86:0/3518066564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f748c0c9330 con 0x7f748c0d1200 2024-04-25T14:04:47.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.237+0000 7f748b400640 1 --2- 172.21.15.86:0/3518066564 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f748c0b4440 0x7f748c14b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:47.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.237+0000 7f748be00640 1 --2- 172.21.15.86:0/3518066564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f748c0d3e70 0x7f748c160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:47.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.237+0000 7f7482a00640 1 --2- 172.21.15.86:0/3518066564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f748c0d1200 0x7f748c14d960 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f74740097f0 tx=0x7f74740191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.237+0000 7f7483e00640 1 -- 172.21.15.86:0/3518066564 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74740b0040 con 0x7f748c0d1200 2024-04-25T14:04:47.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.237+0000 7f7483e00640 1 -- 172.21.15.86:0/3518066564 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7474004030 con 0x7f748c0d1200 2024-04-25T14:04:47.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.237+0000 7f7483e00640 1 -- 172.21.15.86:0/3518066564 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7474004400 con 0x7f748c0d1200 2024-04-25T14:04:47.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.237+0000 7f7492800640 1 -- 172.21.15.86:0/3518066564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f748c0c9830 con 0x7f748c0d1200 2024-04-25T14:04:47.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.237+0000 7f7492800640 1 -- 172.21.15.86:0/3518066564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f748c14d160 con 0x7f748c0d1200 2024-04-25T14:04:47.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.237+0000 7f7492800640 1 -- 172.21.15.86:0/3518066564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f748c06ef80 con 0x7f748c0d1200 2024-04-25T14:04:47.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.241+0000 7f7483e00640 1 -- 172.21.15.86:0/3518066564 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f74740aa7d0 con 0x7f748c0d1200 2024-04-25T14:04:47.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.241+0000 7f7483e00640 1 --2- 172.21.15.86:0/3518066564 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f746806f430 0x7f7468071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.241+0000 7f7483e00640 1 -- 172.21.15.86:0/3518066564 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f7474129760 con 0x7f748c0d1200 2024-04-25T14:04:47.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.241+0000 7f748b400640 1 --2- 172.21.15.86:0/3518066564 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f746806f430 0x7f7468071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.241+0000 7f748b400640 1 --2- 172.21.15.86:0/3518066564 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f746806f430 0x7f7468071940 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f747c000f70 tx=0x7f747c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.245+0000 7f7483e00640 1 -- 172.21.15.86:0/3518066564 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f74740b8020 con 0x7f748c0d1200 2024-04-25T14:04:47.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.401+0000 7f7492800640 1 -- 172.21.15.86:0/3518066564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f748c076bf0 con 0x7f748c0d1200 2024-04-25T14:04:47.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.401+0000 7f7483e00640 1 -- 172.21.15.86:0/3518066564 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f74740160d0 con 0x7f748c0d1200 2024-04-25T14:04:47.402 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:04:47.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.401+0000 7f7492800640 1 -- 172.21.15.86:0/3518066564 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f746806f430 msgr2=0x7f7468071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.401+0000 7f7492800640 1 --2- 172.21.15.86:0/3518066564 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f746806f430 0x7f7468071940 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f747c000f70 tx=0x7f747c009040 comp rx=0 tx=0).stop 2024-04-25T14:04:47.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.401+0000 7f7492800640 1 -- 172.21.15.86:0/3518066564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f748c0d1200 msgr2=0x7f748c14d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.401+0000 7f7492800640 1 --2- 172.21.15.86:0/3518066564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f748c0d1200 0x7f748c14d960 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f74740097f0 tx=0x7f74740191f0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.401+0000 7f7492800640 1 -- 172.21.15.86:0/3518066564 shutdown_connections 2024-04-25T14:04:47.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.401+0000 7f7492800640 1 --2- 172.21.15.86:0/3518066564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f748c0d3e70 0x7f748c160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.401+0000 7f7492800640 1 --2- 172.21.15.86:0/3518066564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f748c0d1200 0x7f748c14d960 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.401+0000 7f7492800640 1 --2- 172.21.15.86:0/3518066564 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f746806f430 0x7f7468071940 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.401+0000 7f7492800640 1 --2- 172.21.15.86:0/3518066564 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f748c0b4440 0x7f748c14b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.401+0000 7f7492800640 1 -- 172.21.15.86:0/3518066564 >> 172.21.15.86:0/3518066564 conn(0x7f748c0b72d0 msgr2=0x7f748c0b8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:47.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.401+0000 7f7492800640 1 -- 172.21.15.86:0/3518066564 shutdown_connections 2024-04-25T14:04:47.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.401+0000 7f7492800640 1 -- 172.21.15.86:0/3518066564 wait complete. 2024-04-25T14:04:47.424 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:04:47.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.573+0000 7f0be2600640 1 Processor -- start 2024-04-25T14:04:47.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.573+0000 7f0be2600640 1 -- start start 2024-04-25T14:04:47.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.577+0000 7f0be2600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0bdc0cfca0 0x7f0bdc06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.577+0000 7f0be2600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0bdc0730b0 0x7f0bdc0734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.577+0000 7f0be2600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bdc072bd0 con 0x7f0bdc0d16e0 2024-04-25T14:04:47.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.577+0000 7f0be2600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0bdc070470 con 0x7f0bdc0cfca0 2024-04-25T14:04:47.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.577+0000 7f0be2600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0bdc06f930 con 0x7f0bdc0730b0 2024-04-25T14:04:47.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.577+0000 7f0bdb400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0bdc0cfca0 0x7f0bdc06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.577+0000 7f0bdb400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0bdc0cfca0 0x7f0bdc06f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60862/0 (socket says 172.21.15.86:60862) 2024-04-25T14:04:47.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.577+0000 7f0bdb400640 1 -- 172.21.15.86:0/3148403551 learned_addr learned my addr 172.21.15.86:0/3148403551 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:47.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.577+0000 7f0bd2a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0bdc0d16e0 0x7f0bdc0d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54946/0 (socket says 172.21.15.86:54946) 2024-04-25T14:04:47.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.577+0000 7f0bdbe00640 1 --2- 172.21.15.86:0/3148403551 >> v2:172.21.15.155:3300/0 conn(0x7f0bdc0730b0 0x7f0bdc0734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.577+0000 7f0bdb400640 1 -- 172.21.15.86:0/3148403551 >> v2:172.21.15.155:3300/0 conn(0x7f0bdc0730b0 msgr2=0x7f0bdc0734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:47.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.577+0000 7f0bdb400640 1 --2- 172.21.15.86:0/3148403551 >> v2:172.21.15.155:3300/0 conn(0x7f0bdc0730b0 0x7f0bdc0734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.577+0000 7f0bdb400640 1 -- 172.21.15.86:0/3148403551 >> v1:172.21.15.86:6789/0 conn(0x7f0bdc0d16e0 legacy=0x7f0bdc0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.577+0000 7f0bdb400640 1 -- 172.21.15.86:0/3148403551 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bdc06fbd0 con 0x7f0bdc0cfca0 2024-04-25T14:04:47.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.581+0000 7f0bdb400640 1 --2- 172.21.15.86:0/3148403551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bdc0cfca0 0x7f0bdc06f3f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc0cb8c0 tx=0x7f0bcc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8eeb6e81cdb35d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.581+0000 7f0bdaa00640 1 -- 172.21.15.86:0/3148403551 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc0ab430 con 0x7f0bdc0cfca0 2024-04-25T14:04:47.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.581+0000 7f0bdaa00640 1 -- 172.21.15.86:0/3148403551 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bcc0b0070 con 0x7f0bdc0cfca0 2024-04-25T14:04:47.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.581+0000 7f0bdaa00640 1 -- 172.21.15.86:0/3148403551 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc0ab7f0 con 0x7f0bdc0cfca0 2024-04-25T14:04:47.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.581+0000 7f0be2600640 1 -- 172.21.15.86:0/3148403551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bdc0cfca0 msgr2=0x7f0bdc06f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.581+0000 7f0be2600640 1 --2- 172.21.15.86:0/3148403551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bdc0cfca0 0x7f0bdc06f3f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc0cb8c0 tx=0x7f0bcc01c520 comp rx=0 tx=0).stop 2024-04-25T14:04:47.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.581+0000 7f0be2600640 1 -- 172.21.15.86:0/3148403551 shutdown_connections 2024-04-25T14:04:47.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.581+0000 7f0be2600640 1 --2- 172.21.15.86:0/3148403551 >> v2:172.21.15.155:3300/0 conn(0x7f0bdc0730b0 0x7f0bdc0734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.581+0000 7f0be2600640 1 --2- 172.21.15.86:0/3148403551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bdc0cfca0 0x7f0bdc06f3f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.581+0000 7f0be2600640 1 -- 172.21.15.86:0/3148403551 >> 172.21.15.86:0/3148403551 conn(0x7f0bdc0b4650 msgr2=0x7f0bdc0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:47.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.581+0000 7f0be2600640 1 -- 172.21.15.86:0/3148403551 shutdown_connections 2024-04-25T14:04:47.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.581+0000 7f0be2600640 1 -- 172.21.15.86:0/3148403551 wait complete. 2024-04-25T14:04:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.585+0000 7f0be2600640 1 Processor -- start 2024-04-25T14:04:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.585+0000 7f0be2600640 1 -- start start 2024-04-25T14:04:47.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.585+0000 7f0be2600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bdc0730b0 0x7f0bdc0be8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.585+0000 7f0be2600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bdc0cfca0 0x7f0bdc099b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.585+0000 7f0be2600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0bdc0d16e0 0x7f0bdc155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.585+0000 7f0be2600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc169a70 con 0x7f0bdc0cfca0 2024-04-25T14:04:47.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.585+0000 7f0be2600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc1698f0 con 0x7f0bdc0730b0 2024-04-25T14:04:47.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.585+0000 7f0be2600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0bdc169bf0 con 0x7f0bdc0d16e0 2024-04-25T14:04:47.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.585+0000 7f0bd2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bdc0cfca0 0x7f0bdc099b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.585+0000 7f0bd2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bdc0cfca0 0x7f0bdc099b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60870/0 (socket says 172.21.15.86:60870) 2024-04-25T14:04:47.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.585+0000 7f0bd2a00640 1 -- 172.21.15.86:0/2141660767 learned_addr learned my addr 172.21.15.86:0/2141660767 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:47.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.585+0000 7f0bdb400640 1 --2- 172.21.15.86:0/2141660767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bdc0730b0 0x7f0bdc0be8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.585+0000 7f0bdbe00640 1 --2- 172.21.15.86:0/2141660767 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0bdc0d16e0 0x7f0bdc155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.585+0000 7f0bd2a00640 1 -- 172.21.15.86:0/2141660767 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0bdc0d16e0 msgr2=0x7f0bdc155a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.585+0000 7f0bd2a00640 1 --2- 172.21.15.86:0/2141660767 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0bdc0d16e0 0x7f0bdc155a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.585+0000 7f0bd2a00640 1 -- 172.21.15.86:0/2141660767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bdc0730b0 msgr2=0x7f0bdc0be8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.585+0000 7f0bd2a00640 1 --2- 172.21.15.86:0/2141660767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bdc0730b0 0x7f0bdc0be8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.585+0000 7f0bd2a00640 1 -- 172.21.15.86:0/2141660767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bdc0c16c0 con 0x7f0bdc0cfca0 2024-04-25T14:04:47.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.585+0000 7f0bd2a00640 1 --2- 172.21.15.86:0/2141660767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bdc0cfca0 0x7f0bdc099b80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400c730 tx=0x7f0bc400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.585+0000 7f0bd3e00640 1 -- 172.21.15.86:0/2141660767 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4017070 con 0x7f0bdc0cfca0 2024-04-25T14:04:47.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.585+0000 7f0bd3e00640 1 -- 172.21.15.86:0/2141660767 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bc400ce40 con 0x7f0bdc0cfca0 2024-04-25T14:04:47.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.585+0000 7f0bd3e00640 1 -- 172.21.15.86:0/2141660767 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc400a7e0 con 0x7f0bdc0cfca0 2024-04-25T14:04:47.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.585+0000 7f0be2600640 1 -- 172.21.15.86:0/2141660767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bdc153120 con 0x7f0bdc0cfca0 2024-04-25T14:04:47.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.585+0000 7f0be2600640 1 -- 172.21.15.86:0/2141660767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bdc153400 con 0x7f0bdc0cfca0 2024-04-25T14:04:47.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.589+0000 7f0be2600640 1 -- 172.21.15.86:0/2141660767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bdc152f10 con 0x7f0bdc0cfca0 2024-04-25T14:04:47.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.589+0000 7f0bd3e00640 1 -- 172.21.15.86:0/2141660767 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0bc401a070 con 0x7f0bdc0cfca0 2024-04-25T14:04:47.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.589+0000 7f0bd3e00640 1 --2- 172.21.15.86:0/2141660767 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0bb806f430 0x7f0bb8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.589+0000 7f0bd3e00640 1 -- 172.21.15.86:0/2141660767 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f0bc408eb60 con 0x7f0bdc0cfca0 2024-04-25T14:04:47.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.589+0000 7f0bdb400640 1 --2- 172.21.15.86:0/2141660767 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0bb806f430 0x7f0bb8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.589+0000 7f0bdb400640 1 --2- 172.21.15.86:0/2141660767 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0bb806f430 0x7f0bb8071940 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0af000 tx=0x7f0bcc0a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.593+0000 7f0bd3e00640 1 -- 172.21.15.86:0/2141660767 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0bc405aff0 con 0x7f0bdc0cfca0 2024-04-25T14:04:47.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.749+0000 7f0be2600640 1 -- 172.21.15.86:0/2141660767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0bdc09f4b0 con 0x7f0bdc0cfca0 2024-04-25T14:04:47.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.749+0000 7f0bd3e00640 1 -- 172.21.15.86:0/2141660767 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f0bc405f2f0 con 0x7f0bdc0cfca0 2024-04-25T14:04:47.753 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:04:47.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.753+0000 7f0be2600640 1 -- 172.21.15.86:0/2141660767 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0bb806f430 msgr2=0x7f0bb8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.753+0000 7f0be2600640 1 --2- 172.21.15.86:0/2141660767 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0bb806f430 0x7f0bb8071940 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0af000 tx=0x7f0bcc0a7000 comp rx=0 tx=0).stop 2024-04-25T14:04:47.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.753+0000 7f0be2600640 1 -- 172.21.15.86:0/2141660767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bdc0cfca0 msgr2=0x7f0bdc099b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.753+0000 7f0be2600640 1 --2- 172.21.15.86:0/2141660767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bdc0cfca0 0x7f0bdc099b80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400c730 tx=0x7f0bc400cbf0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.753+0000 7f0be2600640 1 -- 172.21.15.86:0/2141660767 shutdown_connections 2024-04-25T14:04:47.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.753+0000 7f0be2600640 1 --2- 172.21.15.86:0/2141660767 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0bb806f430 0x7f0bb8071940 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.753+0000 7f0be2600640 1 --2- 172.21.15.86:0/2141660767 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0bdc0d16e0 0x7f0bdc155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.753+0000 7f0be2600640 1 --2- 172.21.15.86:0/2141660767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bdc0cfca0 0x7f0bdc099b80 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.753+0000 7f0be2600640 1 --2- 172.21.15.86:0/2141660767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bdc0730b0 0x7f0bdc0be8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.753+0000 7f0be2600640 1 -- 172.21.15.86:0/2141660767 >> 172.21.15.86:0/2141660767 conn(0x7f0bdc0b4650 msgr2=0x7f0bdc0b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:47.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.753+0000 7f0be2600640 1 -- 172.21.15.86:0/2141660767 shutdown_connections 2024-04-25T14:04:47.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.753+0000 7f0be2600640 1 -- 172.21.15.86:0/2141660767 wait complete. 2024-04-25T14:04:47.774 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T14:04:47.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.921+0000 7f2feaa00640 1 Processor -- start 2024-04-25T14:04:47.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.921+0000 7f2feaa00640 1 -- start start 2024-04-25T14:04:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.925+0000 7f2feaa00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2fe40d1df0 0x7f2fe40cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.925+0000 7f2feaa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2fe40cdcc0 0x7f2fe40ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.925+0000 7f2feaa00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fe415e5c0 con 0x7f2fe4070330 2024-04-25T14:04:47.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.925+0000 7f2feaa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2fe406fd40 con 0x7f2fe40cdcc0 2024-04-25T14:04:47.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.925+0000 7f2feaa00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f2fe415e8a0 con 0x7f2fe40d1df0 2024-04-25T14:04:47.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.925+0000 7f2fe8a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2fe40cdcc0 0x7f2fe40ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.925+0000 7f2fe8a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2fe40cdcc0 0x7f2fe40ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60876/0 (socket says 172.21.15.86:60876) 2024-04-25T14:04:47.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.925+0000 7f2fe8a00640 1 -- 172.21.15.86:0/1808916183 learned_addr learned my addr 172.21.15.86:0/1808916183 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:47.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.925+0000 7f2fdb400640 1 --2- 172.21.15.86:0/1808916183 >> v2:172.21.15.155:3300/0 conn(0x7f2fe40d1df0 0x7f2fe40cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.925+0000 7f2fe8a00640 1 -- 172.21.15.86:0/1808916183 >> v2:172.21.15.155:3300/0 conn(0x7f2fe40d1df0 msgr2=0x7f2fe40cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:47.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.925+0000 7f2fe8a00640 1 --2- 172.21.15.86:0/1808916183 >> v2:172.21.15.155:3300/0 conn(0x7f2fe40d1df0 0x7f2fe40cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.925+0000 7f2fe8a00640 1 -- 172.21.15.86:0/1808916183 >> v1:172.21.15.155:6789/0 conn(0x7f2fe4070330 legacy=0x7f2fe40d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:47.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.925+0000 7f2fe8a00640 1 -- 172.21.15.86:0/1808916183 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe415eb80 con 0x7f2fe40cdcc0 2024-04-25T14:04:47.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.925+0000 7f2fe8a00640 1 --2- 172.21.15.86:0/1808916183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fe40cdcc0 0x7f2fe40ce0d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc009e50 tx=0x7f2fdc01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f83dd02b636de42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.925+0000 7f2fe3400640 1 -- 172.21.15.86:0/1808916183 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc0a57d0 con 0x7f2fe40cdcc0 2024-04-25T14:04:47.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.925+0000 7f2fe3400640 1 -- 172.21.15.86:0/1808916183 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fdc0a5970 con 0x7f2fe40cdcc0 2024-04-25T14:04:47.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.925+0000 7f2fe3400640 1 -- 172.21.15.86:0/1808916183 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc0a5d40 con 0x7f2fe40cdcc0 2024-04-25T14:04:47.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.929+0000 7f2feaa00640 1 -- 172.21.15.86:0/1808916183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fe40cdcc0 msgr2=0x7f2fe40ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.929+0000 7f2feaa00640 1 --2- 172.21.15.86:0/1808916183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fe40cdcc0 0x7f2fe40ce0d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc009e50 tx=0x7f2fdc01c930 comp rx=0 tx=0).stop 2024-04-25T14:04:47.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.929+0000 7f2feaa00640 1 -- 172.21.15.86:0/1808916183 shutdown_connections 2024-04-25T14:04:47.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.929+0000 7f2feaa00640 1 --2- 172.21.15.86:0/1808916183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fe40cdcc0 0x7f2fe40ce0d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.929+0000 7f2feaa00640 1 --2- 172.21.15.86:0/1808916183 >> v2:172.21.15.155:3300/0 conn(0x7f2fe40d1df0 0x7f2fe40cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.929+0000 7f2feaa00640 1 -- 172.21.15.86:0/1808916183 >> 172.21.15.86:0/1808916183 conn(0x7f2fe40b4650 msgr2=0x7f2fe40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:47.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.929+0000 7f2feaa00640 1 -- 172.21.15.86:0/1808916183 shutdown_connections 2024-04-25T14:04:47.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.929+0000 7f2feaa00640 1 -- 172.21.15.86:0/1808916183 wait complete. 2024-04-25T14:04:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.929+0000 7f2feaa00640 1 Processor -- start 2024-04-25T14:04:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.929+0000 7f2feaa00640 1 -- start start 2024-04-25T14:04:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.933+0000 7f2feaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fe4070330 0x7f2fe415a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.933+0000 7f2feaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fe40cdcc0 0x7f2fe40afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.933+0000 7f2feaa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fe4157d30 0x7f2fe4159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.933+0000 7f2feaa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe415f4b0 con 0x7f2fe40cdcc0 2024-04-25T14:04:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.933+0000 7f2feaa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe415f330 con 0x7f2fe4070330 2024-04-25T14:04:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.933+0000 7f2feaa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2fe4070740 con 0x7f2fe4157d30 2024-04-25T14:04:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.933+0000 7f2fdb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fe40cdcc0 0x7f2fe40afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.933+0000 7f2fdb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fe40cdcc0 0x7f2fe40afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60884/0 (socket says 172.21.15.86:60884) 2024-04-25T14:04:47.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.933+0000 7f2fdb400640 1 -- 172.21.15.86:0/2174775423 learned_addr learned my addr 172.21.15.86:0/2174775423 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:47.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.933+0000 7f2fe3e00640 1 --2- 172.21.15.86:0/2174775423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fe4070330 0x7f2fe415a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.933+0000 7f2fe8a00640 1 --2- 172.21.15.86:0/2174775423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fe4157d30 0x7f2fe4159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.933+0000 7f2fdb400640 1 -- 172.21.15.86:0/2174775423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fe4157d30 msgr2=0x7f2fe4159560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.933+0000 7f2fdb400640 1 --2- 172.21.15.86:0/2174775423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fe4157d30 0x7f2fe4159560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.933+0000 7f2fdb400640 1 -- 172.21.15.86:0/2174775423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fe4070330 msgr2=0x7f2fe415a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.933+0000 7f2fdb400640 1 --2- 172.21.15.86:0/2174775423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fe4070330 0x7f2fe415a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.933+0000 7f2fdb400640 1 -- 172.21.15.86:0/2174775423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe4156c90 con 0x7f2fe40cdcc0 2024-04-25T14:04:47.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.933+0000 7f2fdb400640 1 --2- 172.21.15.86:0/2174775423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fe40cdcc0 0x7f2fe40afab0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400d5c0 tx=0x7f2fd400da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.933+0000 7f2fe0c00640 1 -- 172.21.15.86:0/2174775423 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4004240 con 0x7f2fe40cdcc0 2024-04-25T14:04:47.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.933+0000 7f2fe0c00640 1 -- 172.21.15.86:0/2174775423 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fd40043e0 con 0x7f2fe40cdcc0 2024-04-25T14:04:47.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.933+0000 7f2fe0c00640 1 -- 172.21.15.86:0/2174775423 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4003840 con 0x7f2fe40cdcc0 2024-04-25T14:04:47.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.933+0000 7f2feaa00640 1 -- 172.21.15.86:0/2174775423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe4156f70 con 0x7f2fe40cdcc0 2024-04-25T14:04:47.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.933+0000 7f2feaa00640 1 -- 172.21.15.86:0/2174775423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe4157530 con 0x7f2fe40cdcc0 2024-04-25T14:04:47.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.933+0000 7f2feaa00640 1 -- 172.21.15.86:0/2174775423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fe4069d00 con 0x7f2fe40cdcc0 2024-04-25T14:04:47.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.937+0000 7f2fe0c00640 1 -- 172.21.15.86:0/2174775423 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2fd40039e0 con 0x7f2fe40cdcc0 2024-04-25T14:04:47.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.937+0000 7f2fe0c00640 1 --2- 172.21.15.86:0/2174775423 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2fc006f400 0x7f2fc0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.937+0000 7f2fe0c00640 1 -- 172.21.15.86:0/2174775423 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f2fd408f810 con 0x7f2fe40cdcc0 2024-04-25T14:04:47.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.937+0000 7f2fe3e00640 1 --2- 172.21.15.86:0/2174775423 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2fc006f400 0x7f2fc0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.937+0000 7f2fe3e00640 1 --2- 172.21.15.86:0/2174775423 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2fc006f400 0x7f2fc0071910 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4072200 tx=0x7f2fcc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:47.941+0000 7f2fe0c00640 1 -- 172.21.15.86:0/2174775423 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2fd405bcb0 con 0x7f2fe40cdcc0 2024-04-25T14:04:48.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.101+0000 7f2feaa00640 1 -- 172.21.15.86:0/2174775423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2fe4076bf0 con 0x7f2fe40cdcc0 2024-04-25T14:04:48.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.101+0000 7f2fe0c00640 1 -- 172.21.15.86:0/2174775423 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f2fd405ffb0 con 0x7f2fe40cdcc0 2024-04-25T14:04:48.102 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:04:48.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.101+0000 7f2feaa00640 1 -- 172.21.15.86:0/2174775423 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2fc006f400 msgr2=0x7f2fc0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.101+0000 7f2feaa00640 1 --2- 172.21.15.86:0/2174775423 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2fc006f400 0x7f2fc0071910 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4072200 tx=0x7f2fcc009040 comp rx=0 tx=0).stop 2024-04-25T14:04:48.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.101+0000 7f2feaa00640 1 -- 172.21.15.86:0/2174775423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fe40cdcc0 msgr2=0x7f2fe40afab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.101+0000 7f2feaa00640 1 --2- 172.21.15.86:0/2174775423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fe40cdcc0 0x7f2fe40afab0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400d5c0 tx=0x7f2fd400da80 comp rx=0 tx=0).stop 2024-04-25T14:04:48.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.101+0000 7f2feaa00640 1 -- 172.21.15.86:0/2174775423 shutdown_connections 2024-04-25T14:04:48.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.101+0000 7f2feaa00640 1 --2- 172.21.15.86:0/2174775423 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2fc006f400 0x7f2fc0071910 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.101+0000 7f2feaa00640 1 --2- 172.21.15.86:0/2174775423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fe4157d30 0x7f2fe4159560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.101+0000 7f2feaa00640 1 --2- 172.21.15.86:0/2174775423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fe40cdcc0 0x7f2fe40afab0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.101+0000 7f2feaa00640 1 --2- 172.21.15.86:0/2174775423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fe4070330 0x7f2fe415a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.101+0000 7f2feaa00640 1 -- 172.21.15.86:0/2174775423 >> 172.21.15.86:0/2174775423 conn(0x7f2fe40b4650 msgr2=0x7f2fe40b24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:48.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.101+0000 7f2feaa00640 1 -- 172.21.15.86:0/2174775423 shutdown_connections 2024-04-25T14:04:48.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.101+0000 7f2feaa00640 1 -- 172.21.15.86:0/2174775423 wait complete. 2024-04-25T14:04:48.124 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T14:04:48.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.269+0000 7fdfa9200640 1 Processor -- start 2024-04-25T14:04:48.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.269+0000 7fdfa9200640 1 -- start start 2024-04-25T14:04:48.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.273+0000 7fdfa9200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfa40cde90 0x7fdfa40ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.273+0000 7fdfa9200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfa415e5c0 con 0x7fdfa40d1df0 2024-04-25T14:04:48.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.273+0000 7fdfa9200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfa415e8a0 con 0x7fdfa4070330 2024-04-25T14:04:48.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.273+0000 7fdfa9200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdfa406fd40 con 0x7fdfa40cde90 2024-04-25T14:04:48.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.273+0000 7fdfa1400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fdfa40d1df0 0x7fdfa40cd780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54956/0 (socket says 172.21.15.86:54956) 2024-04-25T14:04:48.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.273+0000 7fdfa1400640 1 -- 172.21.15.86:0/200249075 learned_addr learned my addr 172.21.15.86:0/200249075 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:48.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.273+0000 7fdfa2800640 1 --2- 172.21.15.86:0/200249075 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfa40cde90 0x7fdfa40ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.273+0000 7fdfa0a00640 1 -- 172.21.15.86:0/200249075 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2993530597 0 0) 0x7fdfa415e5c0 con 0x7fdfa40d1df0 2024-04-25T14:04:48.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.273+0000 7fdfa0a00640 1 -- 172.21.15.86:0/200249075 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf88003530 con 0x7fdfa40d1df0 2024-04-25T14:04:48.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.273+0000 7fdfa0a00640 1 -- 172.21.15.86:0/200249075 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 902283997 0 0) 0x7fdfa415e8a0 con 0x7fdfa4070330 2024-04-25T14:04:48.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.273+0000 7fdfa0a00640 1 -- 172.21.15.86:0/200249075 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdfa415e5c0 con 0x7fdfa4070330 2024-04-25T14:04:48.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.273+0000 7fdfa0a00640 1 -- 172.21.15.86:0/200249075 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 642176384 0 0) 0x7fdf88003530 con 0x7fdfa40d1df0 2024-04-25T14:04:48.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.273+0000 7fdfa2800640 1 -- 172.21.15.86:0/200249075 >> v1:172.21.15.155:6789/0 conn(0x7fdfa4070330 legacy=0x7fdfa40d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.273+0000 7fdfa2800640 1 -- 172.21.15.86:0/200249075 >> v1:172.21.15.86:6789/0 conn(0x7fdfa40d1df0 legacy=0x7fdfa40cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.273+0000 7fdfa2800640 1 -- 172.21.15.86:0/200249075 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa415eb80 con 0x7fdfa40cde90 2024-04-25T14:04:48.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.277+0000 7fdfa2800640 1 --2- 172.21.15.86:0/200249075 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfa40cde90 0x7fdfa40ce2a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009dc0 tx=0x7fdf9c01c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75aa56942d8cc55e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:48.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.277+0000 7fdfa0a00640 1 -- 172.21.15.86:0/200249075 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c0b2070 con 0x7fdfa40cde90 2024-04-25T14:04:48.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.277+0000 7fdfa0a00640 1 -- 172.21.15.86:0/200249075 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf9c0a5540 con 0x7fdfa40cde90 2024-04-25T14:04:48.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.277+0000 7fdfa0a00640 1 -- 172.21.15.86:0/200249075 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90001680 con 0x7fdfa40cde90 2024-04-25T14:04:48.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.277+0000 7fdfa9200640 1 -- 172.21.15.86:0/200249075 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfa40cde90 msgr2=0x7fdfa40ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.277+0000 7fdfa9200640 1 --2- 172.21.15.86:0/200249075 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfa40cde90 0x7fdfa40ce2a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009dc0 tx=0x7fdf9c01c7f0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.277+0000 7fdfa9200640 1 -- 172.21.15.86:0/200249075 shutdown_connections 2024-04-25T14:04:48.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.277+0000 7fdfa9200640 1 --2- 172.21.15.86:0/200249075 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfa40cde90 0x7fdfa40ce2a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.277+0000 7fdfa9200640 1 -- 172.21.15.86:0/200249075 >> 172.21.15.86:0/200249075 conn(0x7fdfa40b4650 msgr2=0x7fdfa40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:48.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.277+0000 7fdfa9200640 1 -- 172.21.15.86:0/200249075 shutdown_connections 2024-04-25T14:04:48.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.277+0000 7fdfa9200640 1 -- 172.21.15.86:0/200249075 wait complete. 2024-04-25T14:04:48.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.277+0000 7fdfa9200640 1 Processor -- start 2024-04-25T14:04:48.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.281+0000 7fdfa9200640 1 -- start start 2024-04-25T14:04:48.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.281+0000 7fdfa9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfa4070330 0x7fdfa409a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.281+0000 7fdfa9200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfa40cde90 0x7fdfa4148190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.281+0000 7fdfa9200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfa40c4e10 0x7fdfa40c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.281+0000 7fdfa9200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa40708c0 con 0x7fdfa4070330 2024-04-25T14:04:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.281+0000 7fdfa9200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa4070740 con 0x7fdfa40cde90 2024-04-25T14:04:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.281+0000 7fdfa9200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdfa416bfd0 con 0x7fdfa40c4e10 2024-04-25T14:04:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.281+0000 7fdfa1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfa4070330 0x7fdfa409a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.281+0000 7fdfa1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfa4070330 0x7fdfa409a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60892/0 (socket says 172.21.15.86:60892) 2024-04-25T14:04:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.281+0000 7fdfa1e00640 1 -- 172.21.15.86:0/2793877181 learned_addr learned my addr 172.21.15.86:0/2793877181 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.281+0000 7fdfa2800640 1 --2- 172.21.15.86:0/2793877181 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfa40c4e10 0x7fdfa40c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.281+0000 7fdfa1400640 1 --2- 172.21.15.86:0/2793877181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfa40cde90 0x7fdfa4148190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.281+0000 7fdfa1e00640 1 -- 172.21.15.86:0/2793877181 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfa40c4e10 msgr2=0x7fdfa40c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.281+0000 7fdfa1e00640 1 --2- 172.21.15.86:0/2793877181 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfa40c4e10 0x7fdfa40c5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.281+0000 7fdfa1e00640 1 -- 172.21.15.86:0/2793877181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfa40cde90 msgr2=0x7fdfa4148190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.281+0000 7fdfa1e00640 1 --2- 172.21.15.86:0/2793877181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfa40cde90 0x7fdfa4148190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.281+0000 7fdfa1e00640 1 -- 172.21.15.86:0/2793877181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa41596e0 con 0x7fdfa4070330 2024-04-25T14:04:48.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.281+0000 7fdfa2800640 1 --2- 172.21.15.86:0/2793877181 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfa40c4e10 0x7fdfa40c5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:48.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.281+0000 7fdfa1e00640 1 --2- 172.21.15.86:0/2793877181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfa4070330 0x7fdfa409a9e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdf9400c460 tx=0x7fdf9400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:48.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.281+0000 7fdf9a000640 1 -- 172.21.15.86:0/2793877181 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf94017070 con 0x7fdfa4070330 2024-04-25T14:04:48.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.281+0000 7fdf9a000640 1 -- 172.21.15.86:0/2793877181 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf940052f0 con 0x7fdfa4070330 2024-04-25T14:04:48.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.281+0000 7fdfa9200640 1 -- 172.21.15.86:0/2793877181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa40c5760 con 0x7fdfa4070330 2024-04-25T14:04:48.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.281+0000 7fdf9a000640 1 -- 172.21.15.86:0/2793877181 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf94003840 con 0x7fdfa4070330 2024-04-25T14:04:48.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.281+0000 7fdfa9200640 1 -- 172.21.15.86:0/2793877181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa4148c10 con 0x7fdfa4070330 2024-04-25T14:04:48.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.281+0000 7fdfa9200640 1 -- 172.21.15.86:0/2793877181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfa4069d00 con 0x7fdfa4070330 2024-04-25T14:04:48.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.285+0000 7fdf9a000640 1 -- 172.21.15.86:0/2793877181 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdf9401a070 con 0x7fdfa4070330 2024-04-25T14:04:48.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.285+0000 7fdf9a000640 1 --2- 172.21.15.86:0/2793877181 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdf8406f430 0x7fdf84071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.285+0000 7fdf9a000640 1 -- 172.21.15.86:0/2793877181 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fdf9408eaf0 con 0x7fdfa4070330 2024-04-25T14:04:48.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.285+0000 7fdfa1400640 1 --2- 172.21.15.86:0/2793877181 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdf8406f430 0x7fdf84071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.285+0000 7fdfa1400640 1 --2- 172.21.15.86:0/2793877181 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdf8406f430 0x7fdf84071940 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdfa41571d0 tx=0x7fdf90009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:48.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.289+0000 7fdf9a000640 1 -- 172.21.15.86:0/2793877181 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdfa40c5760 con 0x7fdfa4070330 2024-04-25T14:04:48.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.457+0000 7fdfa9200640 1 -- 172.21.15.86:0/2793877181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdfa4076bf0 con 0x7fdfa4070330 2024-04-25T14:04:48.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.457+0000 7fdf9a000640 1 -- 172.21.15.86:0/2793877181 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fdf9405b5e0 con 0x7fdfa4070330 2024-04-25T14:04:48.460 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-25T14:04:48.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.461+0000 7fdfa9200640 1 -- 172.21.15.86:0/2793877181 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdf8406f430 msgr2=0x7fdf84071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.461+0000 7fdfa9200640 1 --2- 172.21.15.86:0/2793877181 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdf8406f430 0x7fdf84071940 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdfa41571d0 tx=0x7fdf90009040 comp rx=0 tx=0).stop 2024-04-25T14:04:48.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.461+0000 7fdfa9200640 1 -- 172.21.15.86:0/2793877181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfa4070330 msgr2=0x7fdfa409a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.461+0000 7fdfa9200640 1 --2- 172.21.15.86:0/2793877181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfa4070330 0x7fdfa409a9e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdf9400c460 tx=0x7fdf9400c920 comp rx=0 tx=0).stop 2024-04-25T14:04:48.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.461+0000 7fdfa9200640 1 -- 172.21.15.86:0/2793877181 shutdown_connections 2024-04-25T14:04:48.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.461+0000 7fdfa9200640 1 --2- 172.21.15.86:0/2793877181 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdf8406f430 0x7fdf84071940 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.461+0000 7fdfa9200640 1 --2- 172.21.15.86:0/2793877181 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfa40c4e10 0x7fdfa40c5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.461+0000 7fdfa9200640 1 --2- 172.21.15.86:0/2793877181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfa40cde90 0x7fdfa4148190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.461+0000 7fdfa9200640 1 --2- 172.21.15.86:0/2793877181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfa4070330 0x7fdfa409a9e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.461+0000 7fdfa9200640 1 -- 172.21.15.86:0/2793877181 >> 172.21.15.86:0/2793877181 conn(0x7fdfa40b4650 msgr2=0x7fdfa415d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:48.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.461+0000 7fdfa9200640 1 -- 172.21.15.86:0/2793877181 shutdown_connections 2024-04-25T14:04:48.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.461+0000 7fdfa9200640 1 -- 172.21.15.86:0/2793877181 wait complete. 2024-04-25T14:04:48.482 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-25T14:04:48.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:48.482 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-25T14:04:48.505 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T14:04:48.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:48.505 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T14:04:48.558 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:04:48.559 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:04:48.568 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:04:48.568 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:04:48.620 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:04:48.620 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:04:48.672 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:04:48.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.817+0000 7fe2b0400640 1 Processor -- start 2024-04-25T14:04:48.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.817+0000 7fe2b0400640 1 -- start start 2024-04-25T14:04:48.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.821+0000 7fe2b0400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe2a80d16e0 0x7fe2a80d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.821+0000 7fe2b0400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe2a80730b0 0x7fe2a80734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.821+0000 7fe2b0400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2a80d2030 con 0x7fe2a80cfca0 2024-04-25T14:04:48.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.821+0000 7fe2b0400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe2a8070470 con 0x7fe2a80730b0 2024-04-25T14:04:48.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.821+0000 7fe2b0400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe2a8072bd0 con 0x7fe2a80d16e0 2024-04-25T14:04:48.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.821+0000 7fe2ae400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe2a80730b0 0x7fe2a80734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.821+0000 7fe2ae400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe2a80730b0 0x7fe2a80734c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52154/0 (socket says 172.21.15.86:52154) 2024-04-25T14:04:48.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.821+0000 7fe2ae400640 1 -- 172.21.15.86:0/3314655666 learned_addr learned my addr 172.21.15.86:0/3314655666 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:48.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.821+0000 7fe2ad000640 1 --2- 172.21.15.86:0/3314655666 >> v2:172.21.15.155:3300/0 conn(0x7fe2a80d16e0 0x7fe2a80d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.821+0000 7fe2ae400640 1 -- 172.21.15.86:0/3314655666 >> v2:172.21.15.155:3300/0 conn(0x7fe2a80d16e0 msgr2=0x7fe2a80d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:48.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.821+0000 7fe2ae400640 1 --2- 172.21.15.86:0/3314655666 >> v2:172.21.15.155:3300/0 conn(0x7fe2a80d16e0 0x7fe2a80d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.821+0000 7fe2ae400640 1 -- 172.21.15.86:0/3314655666 >> v1:172.21.15.155:6789/0 conn(0x7fe2a80cfca0 legacy=0x7fe2a806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.821+0000 7fe2ae400640 1 -- 172.21.15.86:0/3314655666 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a806fbd0 con 0x7fe2a80730b0 2024-04-25T14:04:48.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.825+0000 7fe2ae400640 1 --2- 172.21.15.86:0/3314655666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a80730b0 0x7fe2a80734c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe2a809f480 tx=0x7fe2a001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e6d5f76a3b4d63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:48.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.825+0000 7fe2a7e00640 1 -- 172.21.15.86:0/3314655666 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a00a57d0 con 0x7fe2a80730b0 2024-04-25T14:04:48.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.825+0000 7fe2a7e00640 1 -- 172.21.15.86:0/3314655666 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2a00a5970 con 0x7fe2a80730b0 2024-04-25T14:04:48.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.825+0000 7fe2a7e00640 1 -- 172.21.15.86:0/3314655666 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a00a5d40 con 0x7fe2a80730b0 2024-04-25T14:04:48.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.825+0000 7fe2b0400640 1 -- 172.21.15.86:0/3314655666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a80730b0 msgr2=0x7fe2a80734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.825+0000 7fe2b0400640 1 --2- 172.21.15.86:0/3314655666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a80730b0 0x7fe2a80734c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe2a809f480 tx=0x7fe2a001c9f0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.825+0000 7fe2b0400640 1 -- 172.21.15.86:0/3314655666 shutdown_connections 2024-04-25T14:04:48.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.825+0000 7fe2b0400640 1 --2- 172.21.15.86:0/3314655666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a80730b0 0x7fe2a80734c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.825+0000 7fe2b0400640 1 --2- 172.21.15.86:0/3314655666 >> v2:172.21.15.155:3300/0 conn(0x7fe2a80d16e0 0x7fe2a80d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.825+0000 7fe2b0400640 1 -- 172.21.15.86:0/3314655666 >> 172.21.15.86:0/3314655666 conn(0x7fe2a80b4650 msgr2=0x7fe2a80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:48.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.825+0000 7fe2b0400640 1 -- 172.21.15.86:0/3314655666 shutdown_connections 2024-04-25T14:04:48.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.825+0000 7fe2b0400640 1 -- 172.21.15.86:0/3314655666 wait complete. 2024-04-25T14:04:48.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.829+0000 7fe2b0400640 1 Processor -- start 2024-04-25T14:04:48.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.829+0000 7fe2b0400640 1 -- start start 2024-04-25T14:04:48.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.829+0000 7fe2b0400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2a80730b0 0x7fe2a80cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.829+0000 7fe2b0400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2a80cfca0 0x7fe2a80cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.829+0000 7fe2b0400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a80d16e0 0x7fe2a80ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.829+0000 7fe2b0400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a8169a70 con 0x7fe2a80d16e0 2024-04-25T14:04:48.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.829+0000 7fe2b0400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a81698f0 con 0x7fe2a80730b0 2024-04-25T14:04:48.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.829+0000 7fe2b0400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe2a8169bf0 con 0x7fe2a80cfca0 2024-04-25T14:04:48.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.829+0000 7fe2ae400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a80d16e0 0x7fe2a80ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.829+0000 7fe2ae400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a80d16e0 0x7fe2a80ca610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52162/0 (socket says 172.21.15.86:52162) 2024-04-25T14:04:48.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.829+0000 7fe2ae400640 1 -- 172.21.15.86:0/968732355 learned_addr learned my addr 172.21.15.86:0/968732355 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:48.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.829+0000 7fe2ada00640 1 --2- 172.21.15.86:0/968732355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2a80730b0 0x7fe2a80cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.829+0000 7fe2ae400640 1 -- 172.21.15.86:0/968732355 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2a80cfca0 msgr2=0x7fe2a80cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.829+0000 7fe2ad000640 1 --2- 172.21.15.86:0/968732355 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2a80cfca0 0x7fe2a80cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.829+0000 7fe2ae400640 1 --2- 172.21.15.86:0/968732355 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2a80cfca0 0x7fe2a80cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.829+0000 7fe2ae400640 1 -- 172.21.15.86:0/968732355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2a80730b0 msgr2=0x7fe2a80cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.829+0000 7fe2ae400640 1 --2- 172.21.15.86:0/968732355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2a80730b0 0x7fe2a80cc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.829+0000 7fe2ae400640 1 -- 172.21.15.86:0/968732355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a8157d20 con 0x7fe2a80d16e0 2024-04-25T14:04:48.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.829+0000 7fe2ada00640 1 --2- 172.21.15.86:0/968732355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2a80730b0 0x7fe2a80cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:48.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.829+0000 7fe2ad000640 1 --2- 172.21.15.86:0/968732355 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2a80cfca0 0x7fe2a80cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:48.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.829+0000 7fe2ae400640 1 --2- 172.21.15.86:0/968732355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a80d16e0 0x7fe2a80ca610 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe2a00af040 tx=0x7fe2a0019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:48.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.829+0000 7fe2a5600640 1 -- 172.21.15.86:0/968732355 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a00b6640 con 0x7fe2a80d16e0 2024-04-25T14:04:48.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.829+0000 7fe2a5600640 1 -- 172.21.15.86:0/968732355 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2a00bd070 con 0x7fe2a80d16e0 2024-04-25T14:04:48.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.829+0000 7fe2b0400640 1 -- 172.21.15.86:0/968732355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a8157f90 con 0x7fe2a80d16e0 2024-04-25T14:04:48.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.829+0000 7fe2a5600640 1 -- 172.21.15.86:0/968732355 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a00b6a20 con 0x7fe2a80d16e0 2024-04-25T14:04:48.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.829+0000 7fe2b0400640 1 -- 172.21.15.86:0/968732355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a8158550 con 0x7fe2a80d16e0 2024-04-25T14:04:48.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.833+0000 7fe2b0400640 1 -- 172.21.15.86:0/968732355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2a815d150 con 0x7fe2a80d16e0 2024-04-25T14:04:48.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.833+0000 7fe2a5600640 1 -- 172.21.15.86:0/968732355 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe2a00b6bc0 con 0x7fe2a80d16e0 2024-04-25T14:04:48.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.833+0000 7fe2a5600640 1 --2- 172.21.15.86:0/968732355 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe28806f430 0x7fe288071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.833+0000 7fe2a5600640 1 -- 172.21.15.86:0/968732355 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe2a012a670 con 0x7fe2a80d16e0 2024-04-25T14:04:48.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.833+0000 7fe2ada00640 1 --2- 172.21.15.86:0/968732355 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe28806f430 0x7fe288071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.833+0000 7fe2ada00640 1 --2- 172.21.15.86:0/968732355 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe28806f430 0x7fe288071940 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe29c000f70 tx=0x7fe29c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:48.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:48.837+0000 7fe2a5600640 1 -- 172.21.15.86:0/968732355 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe2a815d150 con 0x7fe2a80d16e0 2024-04-25T14:04:48.888 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:04:48.892 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:04:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.001+0000 7fe2b0400640 1 -- 172.21.15.86:0/968732355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2a8069d00 con 0x7fe2a80d16e0 2024-04-25T14:04:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.001+0000 7fe2a5600640 1 -- 172.21.15.86:0/968732355 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18902 (secure 0 0 0) 0x7fe2a00165e0 con 0x7fe2a80d16e0 2024-04-25T14:04:49.003 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:49.003 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":26,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:04:31.930803+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:03:49.151473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:03:49.802597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T14:03:53.492008+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:04:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.005+0000 7fe2b0400640 1 -- 172.21.15.86:0/968732355 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe28806f430 msgr2=0x7fe288071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.005+0000 7fe2b0400640 1 --2- 172.21.15.86:0/968732355 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe28806f430 0x7fe288071940 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe29c000f70 tx=0x7fe29c009040 comp rx=0 tx=0).stop 2024-04-25T14:04:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.005+0000 7fe2b0400640 1 -- 172.21.15.86:0/968732355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a80d16e0 msgr2=0x7fe2a80ca610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.005+0000 7fe2b0400640 1 --2- 172.21.15.86:0/968732355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a80d16e0 0x7fe2a80ca610 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe2a00af040 tx=0x7fe2a0019790 comp rx=0 tx=0).stop 2024-04-25T14:04:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.005+0000 7fe2b0400640 1 -- 172.21.15.86:0/968732355 shutdown_connections 2024-04-25T14:04:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.005+0000 7fe2b0400640 1 --2- 172.21.15.86:0/968732355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a80d16e0 0x7fe2a80ca610 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.005+0000 7fe2b0400640 1 --2- 172.21.15.86:0/968732355 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe28806f430 0x7fe288071940 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.005+0000 7fe2b0400640 1 --2- 172.21.15.86:0/968732355 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2a80cfca0 0x7fe2a80cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.005+0000 7fe2b0400640 1 --2- 172.21.15.86:0/968732355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2a80730b0 0x7fe2a80cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.005+0000 7fe2b0400640 1 -- 172.21.15.86:0/968732355 >> 172.21.15.86:0/968732355 conn(0x7fe2a80b4650 msgr2=0x7fe2a806eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.005+0000 7fe2b0400640 1 -- 172.21.15.86:0/968732355 shutdown_connections 2024-04-25T14:04:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.005+0000 7fe2b0400640 1 -- 172.21.15.86:0/968732355 wait complete. 2024-04-25T14:04:49.026 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:03:49.151473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:03:49.802597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T14:03:53.492008+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T14:04:49.026 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:04:49.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.173+0000 7fb064600640 1 Processor -- start 2024-04-25T14:04:49.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.173+0000 7fb064600640 1 -- start start 2024-04-25T14:04:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.173+0000 7fb064600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb05c0d1df0 0x7fb05c0b0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.173+0000 7fb064600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb05c0b0690 0x7fb05c0d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.177+0000 7fb064600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb05c0cde20 con 0x7fb05c1642a0 2024-04-25T14:04:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.177+0000 7fb064600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb05c0cea30 con 0x7fb05c0b0690 2024-04-25T14:04:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.177+0000 7fb064600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb05c0ce100 con 0x7fb05c0d1df0 2024-04-25T14:04:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.177+0000 7fb062600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb05c0b0690 0x7fb05c0d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.177+0000 7fb062600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb05c0b0690 0x7fb05c0d0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52166/0 (socket says 172.21.15.86:52166) 2024-04-25T14:04:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.177+0000 7fb062600640 1 -- 172.21.15.86:0/2526814739 learned_addr learned my addr 172.21.15.86:0/2526814739 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.177+0000 7fb061200640 1 --2- 172.21.15.86:0/2526814739 >> v2:172.21.15.155:3300/0 conn(0x7fb05c0d1df0 0x7fb05c0b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.177+0000 7fb062600640 1 -- 172.21.15.86:0/2526814739 >> v2:172.21.15.155:3300/0 conn(0x7fb05c0d1df0 msgr2=0x7fb05c0b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.177+0000 7fb062600640 1 --2- 172.21.15.86:0/2526814739 >> v2:172.21.15.155:3300/0 conn(0x7fb05c0d1df0 0x7fb05c0b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.177+0000 7fb062600640 1 -- 172.21.15.86:0/2526814739 >> v1:172.21.15.155:6789/0 conn(0x7fb05c1642a0 legacy=0x7fb05c0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.177+0000 7fb062600640 1 -- 172.21.15.86:0/2526814739 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05c0ce3e0 con 0x7fb05c0b0690 2024-04-25T14:04:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.177+0000 7fb061200640 1 --2- 172.21.15.86:0/2526814739 >> v2:172.21.15.155:3300/0 conn(0x7fb05c0d1df0 0x7fb05c0b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:49.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.177+0000 7fb062600640 1 --2- 172.21.15.86:0/2526814739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb05c0b0690 0x7fb05c0d0640 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb05c06df80 tx=0x7fb05401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f4c846713fa7ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:49.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.177+0000 7fb05be00640 1 -- 172.21.15.86:0/2526814739 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0540a55e0 con 0x7fb05c0b0690 2024-04-25T14:04:49.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.177+0000 7fb05be00640 1 -- 172.21.15.86:0/2526814739 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0540a5780 con 0x7fb05c0b0690 2024-04-25T14:04:49.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.177+0000 7fb05be00640 1 -- 172.21.15.86:0/2526814739 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0540a5b50 con 0x7fb05c0b0690 2024-04-25T14:04:49.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.181+0000 7fb064600640 1 -- 172.21.15.86:0/2526814739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb05c0b0690 msgr2=0x7fb05c0d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:49.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.181+0000 7fb064600640 1 --2- 172.21.15.86:0/2526814739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb05c0b0690 0x7fb05c0d0640 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb05c06df80 tx=0x7fb05401c920 comp rx=0 tx=0).stop 2024-04-25T14:04:49.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.181+0000 7fb064600640 1 -- 172.21.15.86:0/2526814739 shutdown_connections 2024-04-25T14:04:49.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.181+0000 7fb064600640 1 --2- 172.21.15.86:0/2526814739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb05c0b0690 0x7fb05c0d0640 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:49.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.181+0000 7fb064600640 1 --2- 172.21.15.86:0/2526814739 >> v2:172.21.15.155:3300/0 conn(0x7fb05c0d1df0 0x7fb05c0b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:49.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.181+0000 7fb064600640 1 -- 172.21.15.86:0/2526814739 >> 172.21.15.86:0/2526814739 conn(0x7fb05c0b4650 msgr2=0x7fb05c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:49.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.181+0000 7fb064600640 1 -- 172.21.15.86:0/2526814739 shutdown_connections 2024-04-25T14:04:49.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.181+0000 7fb064600640 1 -- 172.21.15.86:0/2526814739 wait complete. 2024-04-25T14:04:49.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.181+0000 7fb064600640 1 Processor -- start 2024-04-25T14:04:49.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.181+0000 7fb064600640 1 -- start start 2024-04-25T14:04:49.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.185+0000 7fb064600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb05c0b0690 0x7fb05c15a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:49.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.185+0000 7fb064600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb05c1642a0 0x7fb05c09a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:49.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.185+0000 7fb064600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb05c156c80 0x7fb05c155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:49.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.185+0000 7fb064600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c164830 con 0x7fb05c156c80 2024-04-25T14:04:49.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.185+0000 7fb064600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c1646b0 con 0x7fb05c1642a0 2024-04-25T14:04:49.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.185+0000 7fb064600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb05c160da0 con 0x7fb05c0b0690 2024-04-25T14:04:49.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.185+0000 7fb062600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb05c156c80 0x7fb05c155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.185+0000 7fb062600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb05c156c80 0x7fb05c155720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52178/0 (socket says 172.21.15.86:52178) 2024-04-25T14:04:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.185+0000 7fb062600640 1 -- 172.21.15.86:0/3554459020 learned_addr learned my addr 172.21.15.86:0/3554459020 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.185+0000 7fb061200640 1 --2- 172.21.15.86:0/3554459020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb05c1642a0 0x7fb05c09a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.185+0000 7fb061c00640 1 --2- 172.21.15.86:0/3554459020 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb05c0b0690 0x7fb05c15a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.185+0000 7fb062600640 1 -- 172.21.15.86:0/3554459020 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb05c0b0690 msgr2=0x7fb05c15a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.185+0000 7fb062600640 1 --2- 172.21.15.86:0/3554459020 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb05c0b0690 0x7fb05c15a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.185+0000 7fb062600640 1 -- 172.21.15.86:0/3554459020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb05c1642a0 msgr2=0x7fb05c09a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.185+0000 7fb062600640 1 --2- 172.21.15.86:0/3554459020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb05c1642a0 0x7fb05c09a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.185+0000 7fb062600640 1 -- 172.21.15.86:0/3554459020 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05c09ae90 con 0x7fb05c156c80 2024-04-25T14:04:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.185+0000 7fb062600640 1 --2- 172.21.15.86:0/3554459020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb05c156c80 0x7fb05c155720 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb05401a550 tx=0x7fb0540196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.185+0000 7fb059600640 1 -- 172.21.15.86:0/3554459020 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0540bc020 con 0x7fb05c156c80 2024-04-25T14:04:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.185+0000 7fb059600640 1 -- 172.21.15.86:0/3554459020 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0540b8690 con 0x7fb05c156c80 2024-04-25T14:04:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.185+0000 7fb064600640 1 -- 172.21.15.86:0/3554459020 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb05c159960 con 0x7fb05c156c80 2024-04-25T14:04:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.185+0000 7fb059600640 1 -- 172.21.15.86:0/3554459020 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb054009510 con 0x7fb05c156c80 2024-04-25T14:04:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.185+0000 7fb064600640 1 -- 172.21.15.86:0/3554459020 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb05c159f20 con 0x7fb05c156c80 2024-04-25T14:04:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.185+0000 7fb064600640 1 -- 172.21.15.86:0/3554459020 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb05c069d00 con 0x7fb05c156c80 2024-04-25T14:04:49.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.189+0000 7fb059600640 1 -- 172.21.15.86:0/3554459020 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb0540bf020 con 0x7fb05c156c80 2024-04-25T14:04:49.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.189+0000 7fb059600640 1 --2- 172.21.15.86:0/3554459020 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb03c06f400 0x7fb03c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:49.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.189+0000 7fb059600640 1 -- 172.21.15.86:0/3554459020 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb05412a560 con 0x7fb05c156c80 2024-04-25T14:04:49.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.189+0000 7fb061c00640 1 --2- 172.21.15.86:0/3554459020 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb03c06f400 0x7fb03c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:49.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.189+0000 7fb061c00640 1 --2- 172.21.15.86:0/3554459020 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb03c06f400 0x7fb03c071910 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb05c070ac0 tx=0x7fb050009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:49.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.193+0000 7fb059600640 1 -- 172.21.15.86:0/3554459020 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb0540c6030 con 0x7fb05c156c80 2024-04-25T14:04:49.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.349+0000 7fb064600640 1 -- 172.21.15.86:0/3554459020 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb05c076bf0 con 0x7fb05c156c80 2024-04-25T14:04:49.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.349+0000 7fb059600640 1 -- 172.21.15.86:0/3554459020 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fb0540165a0 con 0x7fb05c156c80 2024-04-25T14:04:49.351 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:04:49.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.353+0000 7fb064600640 1 -- 172.21.15.86:0/3554459020 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb03c06f400 msgr2=0x7fb03c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:49.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.353+0000 7fb064600640 1 --2- 172.21.15.86:0/3554459020 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb03c06f400 0x7fb03c071910 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb05c070ac0 tx=0x7fb050009060 comp rx=0 tx=0).stop 2024-04-25T14:04:49.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.353+0000 7fb064600640 1 -- 172.21.15.86:0/3554459020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb05c156c80 msgr2=0x7fb05c155720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:49.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.353+0000 7fb064600640 1 --2- 172.21.15.86:0/3554459020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb05c156c80 0x7fb05c155720 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb05401a550 tx=0x7fb0540196c0 comp rx=0 tx=0).stop 2024-04-25T14:04:49.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.353+0000 7fb064600640 1 -- 172.21.15.86:0/3554459020 shutdown_connections 2024-04-25T14:04:49.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.353+0000 7fb064600640 1 --2- 172.21.15.86:0/3554459020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb05c156c80 0x7fb05c155720 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:49.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.353+0000 7fb064600640 1 --2- 172.21.15.86:0/3554459020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb05c1642a0 0x7fb05c09a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:49.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.353+0000 7fb064600640 1 --2- 172.21.15.86:0/3554459020 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb03c06f400 0x7fb03c071910 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:49.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.353+0000 7fb064600640 1 --2- 172.21.15.86:0/3554459020 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb05c0b0690 0x7fb05c15a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:49.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.353+0000 7fb064600640 1 -- 172.21.15.86:0/3554459020 >> 172.21.15.86:0/3554459020 conn(0x7fb05c0b4650 msgr2=0x7fb05c06e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:49.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.353+0000 7fb064600640 1 -- 172.21.15.86:0/3554459020 shutdown_connections 2024-04-25T14:04:49.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.353+0000 7fb064600640 1 -- 172.21.15.86:0/3554459020 wait complete. 2024-04-25T14:04:49.372 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:04:49.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.521+0000 7f8823c00640 1 Processor -- start 2024-04-25T14:04:49.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.521+0000 7f8823c00640 1 -- start start 2024-04-25T14:04:49.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.521+0000 7f8823c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f881c15faa0 0x7f881c06d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:49.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.521+0000 7f8823c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f881c06c670 con 0x7f881c0aeea0 2024-04-25T14:04:49.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.521+0000 7f8823c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f881c06c950 con 0x7f881c0b59e0 2024-04-25T14:04:49.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.521+0000 7f8823c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f881c06d560 con 0x7f881c15faa0 2024-04-25T14:04:49.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.521+0000 7f8821200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f881c15faa0 0x7f881c06d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:49.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.521+0000 7f8821200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f881c15faa0 0x7f881c06d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52180/0 (socket says 172.21.15.86:52180) 2024-04-25T14:04:49.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.521+0000 7f8821200640 1 -- 172.21.15.86:0/666623813 learned_addr learned my addr 172.21.15.86:0/666623813 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:49.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.521+0000 7f8821c00640 1 --1- 172.21.15.86:0/666623813 >> v1:172.21.15.86:6789/0 conn(0x7f881c0aeea0 0x7f881c0af2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54386/0 (socket says 172.21.15.86:54386) 2024-04-25T14:04:49.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.521+0000 7f8821200640 1 -- 172.21.15.86:0/666623813 >> v1:172.21.15.155:6789/0 conn(0x7f881c0b59e0 legacy=0x7f881c0b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:49.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.521+0000 7f8821200640 1 -- 172.21.15.86:0/666623813 >> v1:172.21.15.86:6789/0 conn(0x7f881c0aeea0 legacy=0x7f881c0af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:49.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.521+0000 7f8821200640 1 -- 172.21.15.86:0/666623813 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881c06cc30 con 0x7f881c15faa0 2024-04-25T14:04:49.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.521+0000 7f881b400640 1 -- 172.21.15.86:0/666623813 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 632124386 0 0) 0x7f881c06c670 con 0x7f881c0aeea0 2024-04-25T14:04:49.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.525+0000 7f8821200640 1 --2- 172.21.15.86:0/666623813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881c15faa0 0x7f881c06d9f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8810009870 tx=0x7f881001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8f80261c122f4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:49.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.525+0000 7f881b400640 1 -- 172.21.15.86:0/666623813 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88100a9620 con 0x7f881c15faa0 2024-04-25T14:04:49.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.525+0000 7f881b400640 1 -- 172.21.15.86:0/666623813 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88100a97c0 con 0x7f881c15faa0 2024-04-25T14:04:49.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.525+0000 7f881b400640 1 -- 172.21.15.86:0/666623813 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88100a9b90 con 0x7f881c15faa0 2024-04-25T14:04:49.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.525+0000 7f8823c00640 1 -- 172.21.15.86:0/666623813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881c15faa0 msgr2=0x7f881c06d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:49.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.525+0000 7f8823c00640 1 --2- 172.21.15.86:0/666623813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881c15faa0 0x7f881c06d9f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8810009870 tx=0x7f881001c560 comp rx=0 tx=0).stop 2024-04-25T14:04:49.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.525+0000 7f8823c00640 1 -- 172.21.15.86:0/666623813 shutdown_connections 2024-04-25T14:04:49.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.525+0000 7f8823c00640 1 --2- 172.21.15.86:0/666623813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881c15faa0 0x7f881c06d9f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:49.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.525+0000 7f8823c00640 1 -- 172.21.15.86:0/666623813 >> 172.21.15.86:0/666623813 conn(0x7f881c0d2230 msgr2=0x7f881c0d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:49.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.525+0000 7f8823c00640 1 -- 172.21.15.86:0/666623813 shutdown_connections 2024-04-25T14:04:49.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.525+0000 7f8823c00640 1 -- 172.21.15.86:0/666623813 wait complete. 2024-04-25T14:04:49.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.529+0000 7f8823c00640 1 Processor -- start 2024-04-25T14:04:49.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.529+0000 7f8823c00640 1 -- start start 2024-04-25T14:04:49.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.529+0000 7f8823c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f881c0aeea0 0x7f881c1434a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:49.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.529+0000 7f8823c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f881c0b59e0 0x7f881c156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:49.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.529+0000 7f8823c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881c15faa0 0x7f881c15b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:49.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.529+0000 7f8823c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f881c15feb0 con 0x7f881c15faa0 2024-04-25T14:04:49.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.529+0000 7f8823c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f881c06df30 con 0x7f881c0b59e0 2024-04-25T14:04:49.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.529+0000 7f8823c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f881c160030 con 0x7f881c0aeea0 2024-04-25T14:04:49.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.529+0000 7f8821c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881c15faa0 0x7f881c15b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:49.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.529+0000 7f8821c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881c15faa0 0x7f881c15b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52184/0 (socket says 172.21.15.86:52184) 2024-04-25T14:04:49.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.529+0000 7f8821c00640 1 -- 172.21.15.86:0/486877426 learned_addr learned my addr 172.21.15.86:0/486877426 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:49.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.529+0000 7f8821200640 1 --2- 172.21.15.86:0/486877426 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f881c0aeea0 0x7f881c1434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:49.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.529+0000 7f881be00640 1 --2- 172.21.15.86:0/486877426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f881c0b59e0 0x7f881c156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:49.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.529+0000 7f8821c00640 1 -- 172.21.15.86:0/486877426 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f881c0aeea0 msgr2=0x7f881c1434a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:49.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.529+0000 7f8821c00640 1 --2- 172.21.15.86:0/486877426 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f881c0aeea0 0x7f881c1434a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:49.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.529+0000 7f8821c00640 1 -- 172.21.15.86:0/486877426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f881c0b59e0 msgr2=0x7f881c156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:49.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.529+0000 7f8821c00640 1 --2- 172.21.15.86:0/486877426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f881c0b59e0 0x7f881c156c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:49.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.529+0000 7f8821c00640 1 -- 172.21.15.86:0/486877426 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881c157790 con 0x7f881c15faa0 2024-04-25T14:04:49.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.533+0000 7f8821c00640 1 --2- 172.21.15.86:0/486877426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881c15faa0 0x7f881c15b860 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f881400c730 tx=0x7f881400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:49.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.533+0000 7f8818c00640 1 -- 172.21.15.86:0/486877426 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8814018070 con 0x7f881c15faa0 2024-04-25T14:04:49.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.533+0000 7f8818c00640 1 -- 172.21.15.86:0/486877426 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f881400ce40 con 0x7f881c15faa0 2024-04-25T14:04:49.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.533+0000 7f8818c00640 1 -- 172.21.15.86:0/486877426 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881400a860 con 0x7f881c15faa0 2024-04-25T14:04:49.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.533+0000 7f8823c00640 1 -- 172.21.15.86:0/486877426 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f881c157a70 con 0x7f881c15faa0 2024-04-25T14:04:49.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.533+0000 7f8823c00640 1 -- 172.21.15.86:0/486877426 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f881c15ecb0 con 0x7f881c15faa0 2024-04-25T14:04:49.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.533+0000 7f8823c00640 1 -- 172.21.15.86:0/486877426 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f881c074cf0 con 0x7f881c15faa0 2024-04-25T14:04:49.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.533+0000 7f8818c00640 1 -- 172.21.15.86:0/486877426 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f881401b070 con 0x7f881c15faa0 2024-04-25T14:04:49.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.533+0000 7f8818c00640 1 --2- 172.21.15.86:0/486877426 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f87fc06f430 0x7f87fc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:49.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.533+0000 7f8818c00640 1 -- 172.21.15.86:0/486877426 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f881408eba0 con 0x7f881c15faa0 2024-04-25T14:04:49.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.533+0000 7f8821200640 1 --2- 172.21.15.86:0/486877426 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f87fc06f430 0x7f87fc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:49.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.537+0000 7f8821200640 1 --2- 172.21.15.86:0/486877426 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f87fc06f430 0x7f87fc071940 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f88100ae000 tx=0x7f88100b3040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:49.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.537+0000 7f8818c00640 1 -- 172.21.15.86:0/486877426 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f881405b030 con 0x7f881c15faa0 2024-04-25T14:04:49.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.693+0000 7f8823c00640 1 -- 172.21.15.86:0/486877426 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f881c157520 con 0x7f881c15faa0 2024-04-25T14:04:49.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.693+0000 7f8818c00640 1 -- 172.21.15.86:0/486877426 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f881405f330 con 0x7f881c15faa0 2024-04-25T14:04:49.697 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:04:49.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.697+0000 7f8823c00640 1 -- 172.21.15.86:0/486877426 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f87fc06f430 msgr2=0x7f87fc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:49.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.697+0000 7f8823c00640 1 --2- 172.21.15.86:0/486877426 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f87fc06f430 0x7f87fc071940 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f88100ae000 tx=0x7f88100b3040 comp rx=0 tx=0).stop 2024-04-25T14:04:49.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.697+0000 7f8823c00640 1 -- 172.21.15.86:0/486877426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881c15faa0 msgr2=0x7f881c15b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:49.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.697+0000 7f8823c00640 1 --2- 172.21.15.86:0/486877426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881c15faa0 0x7f881c15b860 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f881400c730 tx=0x7f881400cbf0 comp rx=0 tx=0).stop 2024-04-25T14:04:49.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.697+0000 7f8823c00640 1 -- 172.21.15.86:0/486877426 shutdown_connections 2024-04-25T14:04:49.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.697+0000 7f8823c00640 1 --2- 172.21.15.86:0/486877426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881c15faa0 0x7f881c15b860 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:49.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.697+0000 7f8823c00640 1 --2- 172.21.15.86:0/486877426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f881c0b59e0 0x7f881c156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:49.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.697+0000 7f8823c00640 1 --2- 172.21.15.86:0/486877426 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f87fc06f430 0x7f87fc071940 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:49.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.697+0000 7f8823c00640 1 --2- 172.21.15.86:0/486877426 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f881c0aeea0 0x7f881c1434a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:49.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.697+0000 7f8823c00640 1 -- 172.21.15.86:0/486877426 >> 172.21.15.86:0/486877426 conn(0x7f881c0d2230 msgr2=0x7f881c0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:49.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.697+0000 7f8823c00640 1 -- 172.21.15.86:0/486877426 shutdown_connections 2024-04-25T14:04:49.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.697+0000 7f8823c00640 1 -- 172.21.15.86:0/486877426 wait complete. 2024-04-25T14:04:49.719 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:04:49.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.865+0000 7f0d43c00640 1 Processor -- start 2024-04-25T14:04:49.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.865+0000 7f0d43c00640 1 -- start start 2024-04-25T14:04:49.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.869+0000 7f0d43c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0d3c0cfca0 0x7f0d3c06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:49.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.869+0000 7f0d43c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0d3c0730b0 0x7f0d3c0734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:49.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.869+0000 7f0d43c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d3c072bd0 con 0x7f0d3c0d16e0 2024-04-25T14:04:49.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.869+0000 7f0d43c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0d3c070470 con 0x7f0d3c0730b0 2024-04-25T14:04:49.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.869+0000 7f0d43c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0d3c06f930 con 0x7f0d3c0cfca0 2024-04-25T14:04:49.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.869+0000 7f0d3be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0d3c0d16e0 0x7f0d3c0d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54388/0 (socket says 172.21.15.86:54388) 2024-04-25T14:04:49.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.869+0000 7f0d3be00640 1 -- 172.21.15.86:0/1651870523 learned_addr learned my addr 172.21.15.86:0/1651870523 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:49.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.869+0000 7f0d41c00640 1 --2- 172.21.15.86:0/1651870523 >> v2:172.21.15.86:3300/0 conn(0x7f0d3c0730b0 0x7f0d3c0734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:49.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.869+0000 7f0d41200640 1 --2- 172.21.15.86:0/1651870523 >> v2:172.21.15.155:3300/0 conn(0x7f0d3c0cfca0 0x7f0d3c06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:49.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.869+0000 7f0d3b400640 1 -- 172.21.15.86:0/1651870523 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2654382861 0 0) 0x7f0d3c072bd0 con 0x7f0d3c0d16e0 2024-04-25T14:04:49.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.869+0000 7f0d3b400640 1 -- 172.21.15.86:0/1651870523 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d20005740 con 0x7f0d3c0d16e0 2024-04-25T14:04:49.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.869+0000 7f0d41c00640 1 -- 172.21.15.86:0/1651870523 >> v2:172.21.15.155:3300/0 conn(0x7f0d3c0cfca0 msgr2=0x7f0d3c06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:49.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.869+0000 7f0d41c00640 1 --2- 172.21.15.86:0/1651870523 >> v2:172.21.15.155:3300/0 conn(0x7f0d3c0cfca0 0x7f0d3c06f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:49.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.869+0000 7f0d41c00640 1 -- 172.21.15.86:0/1651870523 >> v1:172.21.15.86:6789/0 conn(0x7f0d3c0d16e0 legacy=0x7f0d3c0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:49.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.869+0000 7f0d41c00640 1 -- 172.21.15.86:0/1651870523 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3c06fbd0 con 0x7f0d3c0730b0 2024-04-25T14:04:49.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.873+0000 7f0d41c00640 1 --2- 172.21.15.86:0/1651870523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c0730b0 0x7f0d3c0734c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c0d22a0 tx=0x7f0d3401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11515bfc2396de42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:49.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.873+0000 7f0d3b400640 1 -- 172.21.15.86:0/1651870523 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d340b2070 con 0x7f0d3c0730b0 2024-04-25T14:04:49.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.873+0000 7f0d3b400640 1 -- 172.21.15.86:0/1651870523 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d340a5900 con 0x7f0d3c0730b0 2024-04-25T14:04:49.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.873+0000 7f0d3b400640 1 -- 172.21.15.86:0/1651870523 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d340a5cd0 con 0x7f0d3c0730b0 2024-04-25T14:04:49.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.873+0000 7f0d43c00640 1 -- 172.21.15.86:0/1651870523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c0730b0 msgr2=0x7f0d3c0734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:49.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.873+0000 7f0d43c00640 1 --2- 172.21.15.86:0/1651870523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c0730b0 0x7f0d3c0734c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c0d22a0 tx=0x7f0d3401c930 comp rx=0 tx=0).stop 2024-04-25T14:04:49.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.873+0000 7f0d43c00640 1 -- 172.21.15.86:0/1651870523 shutdown_connections 2024-04-25T14:04:49.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.873+0000 7f0d43c00640 1 --2- 172.21.15.86:0/1651870523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c0730b0 0x7f0d3c0734c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:49.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.873+0000 7f0d43c00640 1 --2- 172.21.15.86:0/1651870523 >> v2:172.21.15.155:3300/0 conn(0x7f0d3c0cfca0 0x7f0d3c06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:49.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.873+0000 7f0d43c00640 1 -- 172.21.15.86:0/1651870523 >> 172.21.15.86:0/1651870523 conn(0x7f0d3c0b4650 msgr2=0x7f0d3c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:49.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.873+0000 7f0d43c00640 1 -- 172.21.15.86:0/1651870523 shutdown_connections 2024-04-25T14:04:49.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.873+0000 7f0d43c00640 1 -- 172.21.15.86:0/1651870523 wait complete. 2024-04-25T14:04:49.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.877+0000 7f0d43c00640 1 Processor -- start 2024-04-25T14:04:49.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.877+0000 7f0d43c00640 1 -- start start 2024-04-25T14:04:49.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.877+0000 7f0d43c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d3c0730b0 0x7f0d3c15d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:49.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.877+0000 7f0d43c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d3c0cfca0 0x7f0d3c159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:49.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.877+0000 7f0d43c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c0d16e0 0x7f0d3c1547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:49.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.877+0000 7f0d43c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c0ce0f0 con 0x7f0d3c0d16e0 2024-04-25T14:04:49.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.877+0000 7f0d43c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c0cdf70 con 0x7f0d3c0730b0 2024-04-25T14:04:49.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.877+0000 7f0d43c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0d3c0ce270 con 0x7f0d3c0cfca0 2024-04-25T14:04:49.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.877+0000 7f0d41c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c0d16e0 0x7f0d3c1547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:49.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.877+0000 7f0d41c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c0d16e0 0x7f0d3c1547a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52204/0 (socket says 172.21.15.86:52204) 2024-04-25T14:04:49.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.877+0000 7f0d41c00640 1 -- 172.21.15.86:0/3101469979 learned_addr learned my addr 172.21.15.86:0/3101469979 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:49.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.877+0000 7f0d41200640 1 --2- 172.21.15.86:0/3101469979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d3c0730b0 0x7f0d3c15d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:49.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.877+0000 7f0d3be00640 1 --2- 172.21.15.86:0/3101469979 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d3c0cfca0 0x7f0d3c159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:49.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.877+0000 7f0d41c00640 1 -- 172.21.15.86:0/3101469979 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d3c0cfca0 msgr2=0x7f0d3c159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:49.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.877+0000 7f0d41c00640 1 --2- 172.21.15.86:0/3101469979 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d3c0cfca0 0x7f0d3c159580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:49.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.877+0000 7f0d41c00640 1 -- 172.21.15.86:0/3101469979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d3c0730b0 msgr2=0x7f0d3c15d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:49.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.877+0000 7f0d41c00640 1 --2- 172.21.15.86:0/3101469979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d3c0730b0 0x7f0d3c15d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:49.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.877+0000 7f0d41c00640 1 -- 172.21.15.86:0/3101469979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3c159ac0 con 0x7f0d3c0d16e0 2024-04-25T14:04:49.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.877+0000 7f0d3be00640 1 --2- 172.21.15.86:0/3101469979 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d3c0cfca0 0x7f0d3c159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:49.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.877+0000 7f0d41c00640 1 --2- 172.21.15.86:0/3101469979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c0d16e0 0x7f0d3c1547a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0d34019540 tx=0x7f0d340196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:49.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.877+0000 7f0d38c00640 1 -- 172.21.15.86:0/3101469979 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d340b2050 con 0x7f0d3c0d16e0 2024-04-25T14:04:49.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.877+0000 7f0d38c00640 1 -- 172.21.15.86:0/3101469979 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d340be070 con 0x7f0d3c0d16e0 2024-04-25T14:04:49.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.877+0000 7f0d43c00640 1 -- 172.21.15.86:0/3101469979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d3c0cd4b0 con 0x7f0d3c0d16e0 2024-04-25T14:04:49.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.877+0000 7f0d38c00640 1 -- 172.21.15.86:0/3101469979 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d340ad890 con 0x7f0d3c0d16e0 2024-04-25T14:04:49.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.877+0000 7f0d43c00640 1 -- 172.21.15.86:0/3101469979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d3c09a780 con 0x7f0d3c0d16e0 2024-04-25T14:04:49.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.881+0000 7f0d43c00640 1 -- 172.21.15.86:0/3101469979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d3c09f4b0 con 0x7f0d3c0d16e0 2024-04-25T14:04:49.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.881+0000 7f0d38c00640 1 -- 172.21.15.86:0/3101469979 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0d340ada30 con 0x7f0d3c0d16e0 2024-04-25T14:04:49.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.881+0000 7f0d38c00640 1 --2- 172.21.15.86:0/3101469979 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0d1c06f430 0x7f0d1c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:49.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.881+0000 7f0d38c00640 1 -- 172.21.15.86:0/3101469979 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f0d3412a5c0 con 0x7f0d3c0d16e0 2024-04-25T14:04:49.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.881+0000 7f0d41200640 1 --2- 172.21.15.86:0/3101469979 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0d1c06f430 0x7f0d1c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:49.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.881+0000 7f0d41200640 1 --2- 172.21.15.86:0/3101469979 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0d1c06f430 0x7f0d1c071940 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c070710 tx=0x7f0d2c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:49.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:49.885+0000 7f0d38c00640 1 -- 172.21.15.86:0/3101469979 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0d34016520 con 0x7f0d3c0d16e0 2024-04-25T14:04:50.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.041+0000 7f0d43c00640 1 -- 172.21.15.86:0/3101469979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0d3c069d00 con 0x7f0d3c0d16e0 2024-04-25T14:04:50.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.045+0000 7f0d38c00640 1 -- 172.21.15.86:0/3101469979 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f0d340f70b0 con 0x7f0d3c0d16e0 2024-04-25T14:04:50.045 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:04:50.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.045+0000 7f0d43c00640 1 -- 172.21.15.86:0/3101469979 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0d1c06f430 msgr2=0x7f0d1c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:50.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.045+0000 7f0d43c00640 1 --2- 172.21.15.86:0/3101469979 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0d1c06f430 0x7f0d1c071940 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c070710 tx=0x7f0d2c008040 comp rx=0 tx=0).stop 2024-04-25T14:04:50.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.045+0000 7f0d43c00640 1 -- 172.21.15.86:0/3101469979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c0d16e0 msgr2=0x7f0d3c1547a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:50.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.045+0000 7f0d43c00640 1 --2- 172.21.15.86:0/3101469979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c0d16e0 0x7f0d3c1547a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0d34019540 tx=0x7f0d340196d0 comp rx=0 tx=0).stop 2024-04-25T14:04:50.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.045+0000 7f0d43c00640 1 -- 172.21.15.86:0/3101469979 shutdown_connections 2024-04-25T14:04:50.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.045+0000 7f0d43c00640 1 --2- 172.21.15.86:0/3101469979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c0d16e0 0x7f0d3c1547a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:50.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.045+0000 7f0d43c00640 1 --2- 172.21.15.86:0/3101469979 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0d1c06f430 0x7f0d1c071940 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:50.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.045+0000 7f0d43c00640 1 --2- 172.21.15.86:0/3101469979 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d3c0cfca0 0x7f0d3c159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:50.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.045+0000 7f0d43c00640 1 --2- 172.21.15.86:0/3101469979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d3c0730b0 0x7f0d3c15d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:50.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.045+0000 7f0d43c00640 1 -- 172.21.15.86:0/3101469979 >> 172.21.15.86:0/3101469979 conn(0x7f0d3c0b4650 msgr2=0x7f0d3c15a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:50.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.045+0000 7f0d43c00640 1 -- 172.21.15.86:0/3101469979 shutdown_connections 2024-04-25T14:04:50.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.045+0000 7f0d43c00640 1 -- 172.21.15.86:0/3101469979 wait complete. 2024-04-25T14:04:50.068 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T14:04:50.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.217+0000 7f94df000640 1 Processor -- start 2024-04-25T14:04:50.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.217+0000 7f94df000640 1 -- start start 2024-04-25T14:04:50.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.217+0000 7f94df000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94d8071530 0x7f94d80d15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:50.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.217+0000 7f94df000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94d8161770 con 0x7f94d80d1af0 2024-04-25T14:04:50.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.217+0000 7f94df000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94d8161a50 con 0x7f94d80ce5a0 2024-04-25T14:04:50.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.217+0000 7f94df000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f94d8162380 con 0x7f94d8071530 2024-04-25T14:04:50.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.217+0000 7f94dd000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f94d80d1af0 0x7f94d80d1f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54400/0 (socket says 172.21.15.86:54400) 2024-04-25T14:04:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.217+0000 7f94dd000640 1 -- 172.21.15.86:0/1806648362 learned_addr learned my addr 172.21.15.86:0/1806648362 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.217+0000 7f94d7400640 1 --2- 172.21.15.86:0/1806648362 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94d8071530 0x7f94d80d15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.221+0000 7f94d6a00640 1 -- 172.21.15.86:0/1806648362 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3549279873 0 0) 0x7f94d8161770 con 0x7f94d80d1af0 2024-04-25T14:04:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.221+0000 7f94d6a00640 1 -- 172.21.15.86:0/1806648362 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94bc003530 con 0x7f94d80d1af0 2024-04-25T14:04:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.221+0000 7f94d6a00640 1 -- 172.21.15.86:0/1806648362 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1915870707 0 0) 0x7f94d8161a50 con 0x7f94d80ce5a0 2024-04-25T14:04:50.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.221+0000 7f94d6a00640 1 -- 172.21.15.86:0/1806648362 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94d8161770 con 0x7f94d80ce5a0 2024-04-25T14:04:50.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.221+0000 7f94d7400640 1 -- 172.21.15.86:0/1806648362 >> v1:172.21.15.155:6789/0 conn(0x7f94d80ce5a0 legacy=0x7f94d8070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:50.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.221+0000 7f94d7400640 1 -- 172.21.15.86:0/1806648362 >> v1:172.21.15.86:6789/0 conn(0x7f94d80d1af0 legacy=0x7f94d80d1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:50.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.221+0000 7f94d7400640 1 -- 172.21.15.86:0/1806648362 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d8161d30 con 0x7f94d8071530 2024-04-25T14:04:50.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.221+0000 7f94d7400640 1 --2- 172.21.15.86:0/1806648362 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94d8071530 0x7f94d80d15b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f94d8162500 tx=0x7f94c401c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf065874f71c41f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:50.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.221+0000 7f94d6a00640 1 -- 172.21.15.86:0/1806648362 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c40ab650 con 0x7f94d8071530 2024-04-25T14:04:50.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.221+0000 7f94d6a00640 1 -- 172.21.15.86:0/1806648362 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94c40b0070 con 0x7f94d8071530 2024-04-25T14:04:50.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.221+0000 7f94d6a00640 1 -- 172.21.15.86:0/1806648362 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c40ab9f0 con 0x7f94d8071530 2024-04-25T14:04:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.225+0000 7f94df000640 1 -- 172.21.15.86:0/1806648362 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94d8071530 msgr2=0x7f94d80d15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.225+0000 7f94df000640 1 --2- 172.21.15.86:0/1806648362 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94d8071530 0x7f94d80d15b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f94d8162500 tx=0x7f94c401c490 comp rx=0 tx=0).stop 2024-04-25T14:04:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.225+0000 7f94df000640 1 -- 172.21.15.86:0/1806648362 shutdown_connections 2024-04-25T14:04:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.225+0000 7f94df000640 1 --2- 172.21.15.86:0/1806648362 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94d8071530 0x7f94d80d15b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:50.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.225+0000 7f94df000640 1 -- 172.21.15.86:0/1806648362 >> 172.21.15.86:0/1806648362 conn(0x7f94d80cffc0 msgr2=0x7f94d80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:50.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.225+0000 7f94df000640 1 -- 172.21.15.86:0/1806648362 shutdown_connections 2024-04-25T14:04:50.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.225+0000 7f94df000640 1 -- 172.21.15.86:0/1806648362 wait complete. 2024-04-25T14:04:50.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.225+0000 7f94df000640 1 Processor -- start 2024-04-25T14:04:50.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.225+0000 7f94df000640 1 -- start start 2024-04-25T14:04:50.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.229+0000 7f94df000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94d80ce5a0 0x7f94d814f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:50.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.229+0000 7f94df000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94d80d1af0 0x7f94d80c5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:50.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.229+0000 7f94df000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94d815cb70 0x7f94d815cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:50.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.229+0000 7f94df000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f94d8158550 con 0x7f94d815cb70 2024-04-25T14:04:50.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.229+0000 7f94df000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f94d81583d0 con 0x7f94d80ce5a0 2024-04-25T14:04:50.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.229+0000 7f94df000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f94d816c6a0 con 0x7f94d80d1af0 2024-04-25T14:04:50.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.229+0000 7f94dd000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94d815cb70 0x7f94d815cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:50.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.229+0000 7f94dd000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94d815cb70 0x7f94d815cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52208/0 (socket says 172.21.15.86:52208) 2024-04-25T14:04:50.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.229+0000 7f94dd000640 1 -- 172.21.15.86:0/3211578307 learned_addr learned my addr 172.21.15.86:0/3211578307 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:50.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.229+0000 7f94d7e00640 1 --2- 172.21.15.86:0/3211578307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94d80ce5a0 0x7f94d814f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:50.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.229+0000 7f94d7400640 1 --2- 172.21.15.86:0/3211578307 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94d80d1af0 0x7f94d80c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:50.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.229+0000 7f94dd000640 1 -- 172.21.15.86:0/3211578307 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94d80d1af0 msgr2=0x7f94d80c5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:50.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.229+0000 7f94dd000640 1 --2- 172.21.15.86:0/3211578307 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94d80d1af0 0x7f94d80c5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:50.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.229+0000 7f94dd000640 1 -- 172.21.15.86:0/3211578307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94d80ce5a0 msgr2=0x7f94d814f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:50.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.229+0000 7f94dd000640 1 --2- 172.21.15.86:0/3211578307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94d80ce5a0 0x7f94d814f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:50.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.229+0000 7f94dd000640 1 -- 172.21.15.86:0/3211578307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d815d4c0 con 0x7f94d815cb70 2024-04-25T14:04:50.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.229+0000 7f94dd000640 1 --2- 172.21.15.86:0/3211578307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94d815cb70 0x7f94d815cf80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f94d000cad0 tx=0x7f94d000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:50.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.229+0000 7f94cfe00640 1 -- 172.21.15.86:0/3211578307 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d0017070 con 0x7f94d815cb70 2024-04-25T14:04:50.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.229+0000 7f94cfe00640 1 -- 172.21.15.86:0/3211578307 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94d000abf0 con 0x7f94d815cb70 2024-04-25T14:04:50.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.229+0000 7f94cfe00640 1 -- 172.21.15.86:0/3211578307 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d0012400 con 0x7f94d815cb70 2024-04-25T14:04:50.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.229+0000 7f94df000640 1 -- 172.21.15.86:0/3211578307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94d80c8c70 con 0x7f94d815cb70 2024-04-25T14:04:50.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.229+0000 7f94df000640 1 -- 172.21.15.86:0/3211578307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94d80c9230 con 0x7f94d815cb70 2024-04-25T14:04:50.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.229+0000 7f94df000640 1 -- 172.21.15.86:0/3211578307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94d80720e0 con 0x7f94d815cb70 2024-04-25T14:04:50.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.233+0000 7f94cfe00640 1 -- 172.21.15.86:0/3211578307 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f94d001a070 con 0x7f94d815cb70 2024-04-25T14:04:50.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.233+0000 7f94cfe00640 1 --2- 172.21.15.86:0/3211578307 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f94b806f430 0x7f94b8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:50.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.233+0000 7f94cfe00640 1 -- 172.21.15.86:0/3211578307 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f94d008f790 con 0x7f94d815cb70 2024-04-25T14:04:50.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.233+0000 7f94d7e00640 1 --2- 172.21.15.86:0/3211578307 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f94b806f430 0x7f94b8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:50.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.233+0000 7f94d7e00640 1 --2- 172.21.15.86:0/3211578307 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f94b806f430 0x7f94b8071940 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f94c80011f0 tx=0x7f94c8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:50.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.237+0000 7f94cfe00640 1 -- 172.21.15.86:0/3211578307 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f94d0021020 con 0x7f94d815cb70 2024-04-25T14:04:50.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.393+0000 7f94df000640 1 -- 172.21.15.86:0/3211578307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f94d8076bf0 con 0x7f94d815cb70 2024-04-25T14:04:50.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.393+0000 7f94cfe00640 1 -- 172.21.15.86:0/3211578307 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f94d815d4c0 con 0x7f94d815cb70 2024-04-25T14:04:50.397 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:04:50.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.397+0000 7f94df000640 1 -- 172.21.15.86:0/3211578307 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f94b806f430 msgr2=0x7f94b8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:50.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.397+0000 7f94df000640 1 --2- 172.21.15.86:0/3211578307 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f94b806f430 0x7f94b8071940 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f94c80011f0 tx=0x7f94c8009040 comp rx=0 tx=0).stop 2024-04-25T14:04:50.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.397+0000 7f94df000640 1 -- 172.21.15.86:0/3211578307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94d815cb70 msgr2=0x7f94d815cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:50.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.397+0000 7f94df000640 1 --2- 172.21.15.86:0/3211578307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94d815cb70 0x7f94d815cf80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f94d000cad0 tx=0x7f94d000cf90 comp rx=0 tx=0).stop 2024-04-25T14:04:50.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.397+0000 7f94df000640 1 -- 172.21.15.86:0/3211578307 shutdown_connections 2024-04-25T14:04:50.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.397+0000 7f94df000640 1 --2- 172.21.15.86:0/3211578307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94d815cb70 0x7f94d815cf80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:50.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.397+0000 7f94df000640 1 --2- 172.21.15.86:0/3211578307 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f94b806f430 0x7f94b8071940 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:50.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.397+0000 7f94df000640 1 --2- 172.21.15.86:0/3211578307 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94d80d1af0 0x7f94d80c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:50.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.397+0000 7f94df000640 1 --2- 172.21.15.86:0/3211578307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94d80ce5a0 0x7f94d814f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:50.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.397+0000 7f94df000640 1 -- 172.21.15.86:0/3211578307 >> 172.21.15.86:0/3211578307 conn(0x7f94d80cffc0 msgr2=0x7f94d80d1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:50.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.397+0000 7f94df000640 1 -- 172.21.15.86:0/3211578307 shutdown_connections 2024-04-25T14:04:50.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.397+0000 7f94df000640 1 -- 172.21.15.86:0/3211578307 wait complete. 2024-04-25T14:04:50.419 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T14:04:50.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.565+0000 7f980c600640 1 Processor -- start 2024-04-25T14:04:50.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.565+0000 7f980c600640 1 -- start start 2024-04-25T14:04:50.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.569+0000 7f980c600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f980415dc90 0x7f9804161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:50.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.569+0000 7f980c600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98040cd8d0 con 0x7f98040d2c80 2024-04-25T14:04:50.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.569+0000 7f980c600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98040ce970 con 0x7f9804071530 2024-04-25T14:04:50.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.569+0000 7f980c600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f98040ce720 con 0x7f980415dc90 2024-04-25T14:04:50.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.569+0000 7f9809c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f98040d2c80 0x7f9804070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54408/0 (socket says 172.21.15.86:54408) 2024-04-25T14:04:50.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.569+0000 7f9809c00640 1 -- 172.21.15.86:0/715665327 learned_addr learned my addr 172.21.15.86:0/715665327 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:50.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.569+0000 7f980a600640 1 --2- 172.21.15.86:0/715665327 >> v2:172.21.15.86:3300/0 conn(0x7f980415dc90 0x7f9804161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:50.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.569+0000 7f980a600640 1 -- 172.21.15.86:0/715665327 >> v1:172.21.15.155:6789/0 conn(0x7f9804071530 legacy=0x7f98041616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:50.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.569+0000 7f980a600640 1 -- 172.21.15.86:0/715665327 >> v1:172.21.15.86:6789/0 conn(0x7f98040d2c80 legacy=0x7f9804070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:50.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.569+0000 7f980a600640 1 -- 172.21.15.86:0/715665327 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98040d1be0 con 0x7f980415dc90 2024-04-25T14:04:50.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.573+0000 7f980a600640 1 --2- 172.21.15.86:0/715665327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f980415dc90 0x7f9804161df0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f97fc009d90 tx=0x7f97fc01f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d64ecd0c45cde39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:50.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.573+0000 7f9803e00640 1 -- 172.21.15.86:0/715665327 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc0b4070 con 0x7f980415dc90 2024-04-25T14:04:50.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.573+0000 7f9803e00640 1 -- 172.21.15.86:0/715665327 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97fc0af600 con 0x7f980415dc90 2024-04-25T14:04:50.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.573+0000 7f9803e00640 1 -- 172.21.15.86:0/715665327 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc0af9d0 con 0x7f980415dc90 2024-04-25T14:04:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.573+0000 7f980c600640 1 -- 172.21.15.86:0/715665327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f980415dc90 msgr2=0x7f9804161df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.573+0000 7f980c600640 1 --2- 172.21.15.86:0/715665327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f980415dc90 0x7f9804161df0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f97fc009d90 tx=0x7f97fc01f150 comp rx=0 tx=0).stop 2024-04-25T14:04:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.573+0000 7f980c600640 1 -- 172.21.15.86:0/715665327 shutdown_connections 2024-04-25T14:04:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.573+0000 7f980c600640 1 --2- 172.21.15.86:0/715665327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f980415dc90 0x7f9804161df0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.573+0000 7f980c600640 1 -- 172.21.15.86:0/715665327 >> 172.21.15.86:0/715665327 conn(0x7f98040cffc0 msgr2=0x7f98040d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.573+0000 7f980c600640 1 -- 172.21.15.86:0/715665327 shutdown_connections 2024-04-25T14:04:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.573+0000 7f980c600640 1 -- 172.21.15.86:0/715665327 wait complete. 2024-04-25T14:04:50.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.577+0000 7f980c600640 1 Processor -- start 2024-04-25T14:04:50.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.577+0000 7f980c600640 1 -- start start 2024-04-25T14:04:50.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.577+0000 7f980c600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98040d2c80 0x7f9804153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:50.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.577+0000 7f980c600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f980415dc90 0x7f98040b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:50.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.577+0000 7f980c600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98040b2fa0 0x7f98040b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:50.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.577+0000 7f980c600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98040d2480 con 0x7f98040d2c80 2024-04-25T14:04:50.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.577+0000 7f980c600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f98040d21a0 con 0x7f98040b2fa0 2024-04-25T14:04:50.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.577+0000 7f980c600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f98040d29a0 con 0x7f980415dc90 2024-04-25T14:04:50.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.577+0000 7f9809c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98040d2c80 0x7f9804153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:50.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.577+0000 7f9809c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98040d2c80 0x7f9804153e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52216/0 (socket says 172.21.15.86:52216) 2024-04-25T14:04:50.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.577+0000 7f9809c00640 1 -- 172.21.15.86:0/3704050342 learned_addr learned my addr 172.21.15.86:0/3704050342 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:50.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.577+0000 7f980a600640 1 --2- 172.21.15.86:0/3704050342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98040b2fa0 0x7f98040b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:50.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.577+0000 7f9809200640 1 --2- 172.21.15.86:0/3704050342 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f980415dc90 0x7f98040b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:50.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.577+0000 7f9809c00640 1 -- 172.21.15.86:0/3704050342 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f980415dc90 msgr2=0x7f98040b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.577+0000 7f9809c00640 1 --2- 172.21.15.86:0/3704050342 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f980415dc90 0x7f98040b5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.577+0000 7f9809c00640 1 -- 172.21.15.86:0/3704050342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98040b2fa0 msgr2=0x7f98040b55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.577+0000 7f9809c00640 1 --2- 172.21.15.86:0/3704050342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98040b2fa0 0x7f98040b55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.577+0000 7f9809c00640 1 -- 172.21.15.86:0/3704050342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98040af950 con 0x7f98040d2c80 2024-04-25T14:04:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.577+0000 7f9809200640 1 --2- 172.21.15.86:0/3704050342 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f980415dc90 0x7f98040b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.577+0000 7f9809c00640 1 --2- 172.21.15.86:0/3704050342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98040d2c80 0x7f9804153e90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f97f800c430 tx=0x7f97f800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.577+0000 7f9801600640 1 -- 172.21.15.86:0/3704050342 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f8017070 con 0x7f98040d2c80 2024-04-25T14:04:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.577+0000 7f9801600640 1 -- 172.21.15.86:0/3704050342 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97f8005cf0 con 0x7f98040d2c80 2024-04-25T14:04:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.577+0000 7f980c600640 1 -- 172.21.15.86:0/3704050342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98041543d0 con 0x7f98040d2c80 2024-04-25T14:04:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.577+0000 7f9801600640 1 -- 172.21.15.86:0/3704050342 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f80072d0 con 0x7f98040d2c80 2024-04-25T14:04:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.577+0000 7f980c600640 1 -- 172.21.15.86:0/3704050342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f980415e280 con 0x7f98040d2c80 2024-04-25T14:04:50.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.581+0000 7f980c600640 1 -- 172.21.15.86:0/3704050342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f980415e0a0 con 0x7f98040d2c80 2024-04-25T14:04:50.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.581+0000 7f9801600640 1 -- 172.21.15.86:0/3704050342 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f97f801a070 con 0x7f98040d2c80 2024-04-25T14:04:50.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.581+0000 7f9801600640 1 --2- 172.21.15.86:0/3704050342 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f97e406f430 0x7f97e4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:50.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.581+0000 7f9801600640 1 -- 172.21.15.86:0/3704050342 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f97f808ea70 con 0x7f98040d2c80 2024-04-25T14:04:50.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.581+0000 7f9809200640 1 --2- 172.21.15.86:0/3704050342 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f97e406f430 0x7f97e4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:50.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.581+0000 7f9809200640 1 --2- 172.21.15.86:0/3704050342 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f97e406f430 0x7f97e4071940 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f98040b5020 tx=0x7f97f0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:50.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.585+0000 7f9801600640 1 -- 172.21.15.86:0/3704050342 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f980415e0a0 con 0x7f98040d2c80 2024-04-25T14:04:50.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.753+0000 7f980c600640 1 -- 172.21.15.86:0/3704050342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f98040720e0 con 0x7f98040d2c80 2024-04-25T14:04:50.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.753+0000 7f9801600640 1 -- 172.21.15.86:0/3704050342 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f97f805ae70 con 0x7f98040d2c80 2024-04-25T14:04:50.755 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-25T14:04:50.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.757+0000 7f980c600640 1 -- 172.21.15.86:0/3704050342 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f97e406f430 msgr2=0x7f97e4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:50.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.757+0000 7f980c600640 1 --2- 172.21.15.86:0/3704050342 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f97e406f430 0x7f97e4071940 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f98040b5020 tx=0x7f97f0009040 comp rx=0 tx=0).stop 2024-04-25T14:04:50.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.757+0000 7f980c600640 1 -- 172.21.15.86:0/3704050342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98040d2c80 msgr2=0x7f9804153e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:50.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.757+0000 7f980c600640 1 --2- 172.21.15.86:0/3704050342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98040d2c80 0x7f9804153e90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f97f800c430 tx=0x7f97f800c8f0 comp rx=0 tx=0).stop 2024-04-25T14:04:50.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.757+0000 7f980c600640 1 -- 172.21.15.86:0/3704050342 shutdown_connections 2024-04-25T14:04:50.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.757+0000 7f980c600640 1 --2- 172.21.15.86:0/3704050342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98040b2fa0 0x7f98040b55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:50.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.757+0000 7f980c600640 1 --2- 172.21.15.86:0/3704050342 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f97e406f430 0x7f97e4071940 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:50.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.757+0000 7f980c600640 1 --2- 172.21.15.86:0/3704050342 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f980415dc90 0x7f98040b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:50.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.757+0000 7f980c600640 1 --2- 172.21.15.86:0/3704050342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98040d2c80 0x7f9804153e90 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:50.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.757+0000 7f980c600640 1 -- 172.21.15.86:0/3704050342 >> 172.21.15.86:0/3704050342 conn(0x7f98040cffc0 msgr2=0x7f980406e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:50.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.757+0000 7f980c600640 1 -- 172.21.15.86:0/3704050342 shutdown_connections 2024-04-25T14:04:50.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.757+0000 7f980c600640 1 -- 172.21.15.86:0/3704050342 wait complete. 2024-04-25T14:04:50.776 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-25T14:04:50.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:50.777 DEBUG:teuthology.orchestra.run.smithi155:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-25T14:04:50.799 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T14:04:50.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:50.800 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T14:04:50.852 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T14:04:50.852 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:04:50.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.997+0000 7f6a4e000640 1 Processor -- start 2024-04-25T14:04:50.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:50.997+0000 7f6a4e000640 1 -- start start 2024-04-25T14:04:51.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.001+0000 7f6a4e000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a4815ec70 0x7f6a480d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:51.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.001+0000 7f6a4e000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a480cf8f0 con 0x7f6a480d1490 2024-04-25T14:04:51.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.001+0000 7f6a4e000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a4806c670 con 0x7f6a4815e860 2024-04-25T14:04:51.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.001+0000 7f6a4e000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6a4806cfa0 con 0x7f6a4815ec70 2024-04-25T14:04:51.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.001+0000 7f6a47e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6a480d1490 0x7f6a4806d430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54418/0 (socket says 172.21.15.86:54418) 2024-04-25T14:04:51.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.001+0000 7f6a47e00640 1 -- 172.21.15.86:0/3691946324 learned_addr learned my addr 172.21.15.86:0/3691946324 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:51.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.001+0000 7f6a3ea00640 1 --2- 172.21.15.86:0/3691946324 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a4815ec70 0x7f6a480d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:51.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.001+0000 7f6a46a00640 1 -- 172.21.15.86:0/3691946324 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1627325047 0 0) 0x7f6a480cf8f0 con 0x7f6a480d1490 2024-04-25T14:04:51.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.001+0000 7f6a46a00640 1 -- 172.21.15.86:0/3691946324 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a2c003530 con 0x7f6a480d1490 2024-04-25T14:04:51.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.001+0000 7f6a46a00640 1 -- 172.21.15.86:0/3691946324 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 15587304 0 0) 0x7f6a4806c670 con 0x7f6a4815e860 2024-04-25T14:04:51.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.001+0000 7f6a46a00640 1 -- 172.21.15.86:0/3691946324 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a480cf8f0 con 0x7f6a4815e860 2024-04-25T14:04:51.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.001+0000 7f6a3ea00640 1 -- 172.21.15.86:0/3691946324 >> v1:172.21.15.155:6789/0 conn(0x7f6a4815e860 legacy=0x7f6a480ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:51.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.001+0000 7f6a3ea00640 1 -- 172.21.15.86:0/3691946324 >> v1:172.21.15.86:6789/0 conn(0x7f6a480d1490 legacy=0x7f6a4806d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:51.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.001+0000 7f6a3ea00640 1 -- 172.21.15.86:0/3691946324 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a4806c950 con 0x7f6a4815ec70 2024-04-25T14:04:51.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.005+0000 7f6a3ea00640 1 --2- 172.21.15.86:0/3691946324 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a4815ec70 0x7f6a480d0f50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6a4806d120 tx=0x7f6a3001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=396660affb12c10d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:51.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.005+0000 7f6a46a00640 1 -- 172.21.15.86:0/3691946324 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a300b1070 con 0x7f6a4815ec70 2024-04-25T14:04:51.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.005+0000 7f6a46a00640 1 -- 172.21.15.86:0/3691946324 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a300ac5f0 con 0x7f6a4815ec70 2024-04-25T14:04:51.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.005+0000 7f6a46a00640 1 -- 172.21.15.86:0/3691946324 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a300ac9a0 con 0x7f6a4815ec70 2024-04-25T14:04:51.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.005+0000 7f6a4e000640 1 -- 172.21.15.86:0/3691946324 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a4815ec70 msgr2=0x7f6a480d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:51.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.005+0000 7f6a4e000640 1 --2- 172.21.15.86:0/3691946324 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a4815ec70 0x7f6a480d0f50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6a4806d120 tx=0x7f6a3001c400 comp rx=0 tx=0).stop 2024-04-25T14:04:51.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.005+0000 7f6a4e000640 1 -- 172.21.15.86:0/3691946324 shutdown_connections 2024-04-25T14:04:51.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.005+0000 7f6a4e000640 1 --2- 172.21.15.86:0/3691946324 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a4815ec70 0x7f6a480d0f50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:51.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.005+0000 7f6a4e000640 1 -- 172.21.15.86:0/3691946324 >> 172.21.15.86:0/3691946324 conn(0x7f6a480d2230 msgr2=0x7f6a480d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:51.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.005+0000 7f6a4e000640 1 -- 172.21.15.86:0/3691946324 shutdown_connections 2024-04-25T14:04:51.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.005+0000 7f6a4e000640 1 -- 172.21.15.86:0/3691946324 wait complete. 2024-04-25T14:04:51.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.005+0000 7f6a4e000640 1 Processor -- start 2024-04-25T14:04:51.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.005+0000 7f6a4e000640 1 -- start start 2024-04-25T14:04:51.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.009+0000 7f6a4e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a480d1490 0x7f6a481425c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:51.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.009+0000 7f6a4e000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a4815e860 0x7f6a481655e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:51.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.009+0000 7f6a4e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a4815ec70 0x7f6a481456f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:51.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.009+0000 7f6a4e000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6a480d18a0 con 0x7f6a480d1490 2024-04-25T14:04:51.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.009+0000 7f6a4e000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6a480b5770 con 0x7f6a4815ec70 2024-04-25T14:04:51.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.009+0000 7f6a4e000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6a4816bf40 con 0x7f6a4815e860 2024-04-25T14:04:51.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.009+0000 7f6a47400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a480d1490 0x7f6a481425c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:51.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.009+0000 7f6a47400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a480d1490 0x7f6a481425c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52220/0 (socket says 172.21.15.86:52220) 2024-04-25T14:04:51.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.009+0000 7f6a47400640 1 -- 172.21.15.86:0/823670109 learned_addr learned my addr 172.21.15.86:0/823670109 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:51.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.009+0000 7f6a3ea00640 1 --2- 172.21.15.86:0/823670109 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a4815e860 0x7f6a481655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:51.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.009+0000 7f6a47e00640 1 --2- 172.21.15.86:0/823670109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a4815ec70 0x7f6a481456f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:51.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.009+0000 7f6a47400640 1 -- 172.21.15.86:0/823670109 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a4815e860 msgr2=0x7f6a481655e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:51.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.009+0000 7f6a47400640 1 --2- 172.21.15.86:0/823670109 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a4815e860 0x7f6a481655e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:51.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.009+0000 7f6a47400640 1 -- 172.21.15.86:0/823670109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a4815ec70 msgr2=0x7f6a481456f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:51.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.009+0000 7f6a47400640 1 --2- 172.21.15.86:0/823670109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a4815ec70 0x7f6a481456f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:51.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.009+0000 7f6a47400640 1 -- 172.21.15.86:0/823670109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a48144650 con 0x7f6a480d1490 2024-04-25T14:04:51.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.009+0000 7f6a47400640 1 --2- 172.21.15.86:0/823670109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a480d1490 0x7f6a481425c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800c430 tx=0x7f6a3800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:51.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.009+0000 7f6a3fe00640 1 -- 172.21.15.86:0/823670109 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38017070 con 0x7f6a480d1490 2024-04-25T14:04:51.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.009+0000 7f6a3fe00640 1 -- 172.21.15.86:0/823670109 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a380052f0 con 0x7f6a480d1490 2024-04-25T14:04:51.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.009+0000 7f6a4e000640 1 -- 172.21.15.86:0/823670109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a48144930 con 0x7f6a480d1490 2024-04-25T14:04:51.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.009+0000 7f6a3fe00640 1 -- 172.21.15.86:0/823670109 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38003840 con 0x7f6a480d1490 2024-04-25T14:04:51.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.009+0000 7f6a4e000640 1 -- 172.21.15.86:0/823670109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a48144ef0 con 0x7f6a480d1490 2024-04-25T14:04:51.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.009+0000 7f6a4e000640 1 -- 172.21.15.86:0/823670109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a48074cf0 con 0x7f6a480d1490 2024-04-25T14:04:51.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.013+0000 7f6a3fe00640 1 -- 172.21.15.86:0/823670109 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6a3801a070 con 0x7f6a480d1490 2024-04-25T14:04:51.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.013+0000 7f6a3fe00640 1 --2- 172.21.15.86:0/823670109 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6a2406f400 0x7f6a24071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:51.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.013+0000 7f6a3fe00640 1 -- 172.21.15.86:0/823670109 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f6a3808eae0 con 0x7f6a480d1490 2024-04-25T14:04:51.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.013+0000 7f6a3ea00640 1 --2- 172.21.15.86:0/823670109 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6a2406f400 0x7f6a24071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:51.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.013+0000 7f6a3ea00640 1 --2- 172.21.15.86:0/823670109 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6a2406f400 0x7f6a24071910 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6a300ae000 tx=0x7f6a30008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:51.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.017+0000 7f6a3fe00640 1 -- 172.21.15.86:0/823670109 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6a3805aee0 con 0x7f6a480d1490 2024-04-25T14:04:51.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.209+0000 7f6a4e000640 1 -- 172.21.15.86:0/823670109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a48076bf0 con 0x7f6a480d1490 2024-04-25T14:04:51.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.209+0000 7f6a3fe00640 1 -- 172.21.15.86:0/823670109 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5177 (secure 0 0 0) 0x7f6a3805f1e0 con 0x7f6a480d1490 2024-04-25T14:04:51.210 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:51.210 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/347107112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":347107112},{"type":"v1","addr":"172.21.15.155:6835","nonce":347107112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2236737549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2236737549},{"type":"v1","addr":"172.21.15.86:6837","nonce":2236737549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/519560229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":519560229},{"type":"v1","addr":"172.21.15.86:6836","nonce":519560229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:03:50.852257+0000","modified":"2024-04-25T14:04:09.202137+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6833/1944249570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1944249570},{"type":"v1","addr":"172.21.15.155:6833","nonce":1944249570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":1}]} 2024-04-25T14:04:51.210 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 9 2024-04-25T14:04:51.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.209+0000 7f6a4e000640 1 -- 172.21.15.86:0/823670109 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6a2406f400 msgr2=0x7f6a24071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:51.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.209+0000 7f6a4e000640 1 --2- 172.21.15.86:0/823670109 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6a2406f400 0x7f6a24071910 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6a300ae000 tx=0x7f6a30008040 comp rx=0 tx=0).stop 2024-04-25T14:04:51.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.209+0000 7f6a4e000640 1 -- 172.21.15.86:0/823670109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a480d1490 msgr2=0x7f6a481425c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:51.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.209+0000 7f6a4e000640 1 --2- 172.21.15.86:0/823670109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a480d1490 0x7f6a481425c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800c430 tx=0x7f6a3800c8f0 comp rx=0 tx=0).stop 2024-04-25T14:04:51.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.209+0000 7f6a4e000640 1 -- 172.21.15.86:0/823670109 shutdown_connections 2024-04-25T14:04:51.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.209+0000 7f6a4e000640 1 --2- 172.21.15.86:0/823670109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a4815ec70 0x7f6a481456f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:51.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.209+0000 7f6a4e000640 1 --2- 172.21.15.86:0/823670109 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6a2406f400 0x7f6a24071910 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:51.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.209+0000 7f6a4e000640 1 --2- 172.21.15.86:0/823670109 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a4815e860 0x7f6a481655e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:51.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.209+0000 7f6a4e000640 1 --2- 172.21.15.86:0/823670109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a480d1490 0x7f6a481425c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:51.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.209+0000 7f6a4e000640 1 -- 172.21.15.86:0/823670109 >> 172.21.15.86:0/823670109 conn(0x7f6a480d2230 msgr2=0x7f6a480b58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:51.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.209+0000 7f6a4e000640 1 -- 172.21.15.86:0/823670109 shutdown_connections 2024-04-25T14:04:51.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.209+0000 7f6a4e000640 1 -- 172.21.15.86:0/823670109 wait complete. 2024-04-25T14:04:51.232 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:04:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.377+0000 7fd667200640 1 Processor -- start 2024-04-25T14:04:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.377+0000 7fd667200640 1 -- start start 2024-04-25T14:04:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.377+0000 7fd667200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6680b6ff0 0x7fd6680b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.377+0000 7fd667200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd6680ce320 0x7fd6680ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.377+0000 7fd667200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd668071180 con 0x7fd6680b11b0 2024-04-25T14:04:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.377+0000 7fd667200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd6680d0ab0 con 0x7fd6680ce320 2024-04-25T14:04:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.377+0000 7fd667200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd6680d0850 con 0x7fd6680b6ff0 2024-04-25T14:04:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.377+0000 7fd65fe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6680b6ff0 0x7fd6680b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:51.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.377+0000 7fd666800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd6680ce320 0x7fd6680ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:51.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.377+0000 7fd65fe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6680b6ff0 0x7fd6680b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:49796/0 (socket says 172.21.15.86:49796) 2024-04-25T14:04:51.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.377+0000 7fd65fe00640 1 -- 172.21.15.86:0/1194646982 learned_addr learned my addr 172.21.15.86:0/1194646982 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:51.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.377+0000 7fd65fe00640 1 -- 172.21.15.86:0/1194646982 >> v2:172.21.15.155:3300/0 conn(0x7fd6680ce320 msgr2=0x7fd6680ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:51.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.377+0000 7fd65fe00640 1 --2- 172.21.15.86:0/1194646982 >> v2:172.21.15.155:3300/0 conn(0x7fd6680ce320 0x7fd6680ce730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:51.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.377+0000 7fd65fe00640 1 -- 172.21.15.86:0/1194646982 >> v1:172.21.15.155:6789/0 conn(0x7fd6680b11b0 legacy=0x7fd6680d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:51.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.377+0000 7fd65fe00640 1 -- 172.21.15.86:0/1194646982 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6680b7940 con 0x7fd6680b6ff0 2024-04-25T14:04:51.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.381+0000 7fd65fe00640 1 --2- 172.21.15.86:0/1194646982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6680b6ff0 0x7fd6680b7400 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd650009870 tx=0x7fd65001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9755edb4b7adb70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:51.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.381+0000 7fd665400640 1 -- 172.21.15.86:0/1194646982 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6500ab650 con 0x7fd6680b6ff0 2024-04-25T14:04:51.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.381+0000 7fd665400640 1 -- 172.21.15.86:0/1194646982 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6500b0070 con 0x7fd6680b6ff0 2024-04-25T14:04:51.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.381+0000 7fd665400640 1 -- 172.21.15.86:0/1194646982 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6500aba10 con 0x7fd6680b6ff0 2024-04-25T14:04:51.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.381+0000 7fd667200640 1 -- 172.21.15.86:0/1194646982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6680b6ff0 msgr2=0x7fd6680b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:51.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.381+0000 7fd667200640 1 --2- 172.21.15.86:0/1194646982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6680b6ff0 0x7fd6680b7400 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd650009870 tx=0x7fd65001c560 comp rx=0 tx=0).stop 2024-04-25T14:04:51.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.381+0000 7fd667200640 1 -- 172.21.15.86:0/1194646982 shutdown_connections 2024-04-25T14:04:51.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.381+0000 7fd667200640 1 --2- 172.21.15.86:0/1194646982 >> v2:172.21.15.155:3300/0 conn(0x7fd6680ce320 0x7fd6680ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:51.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.381+0000 7fd667200640 1 --2- 172.21.15.86:0/1194646982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6680b6ff0 0x7fd6680b7400 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:51.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.381+0000 7fd667200640 1 -- 172.21.15.86:0/1194646982 >> 172.21.15.86:0/1194646982 conn(0x7fd66806c840 msgr2=0x7fd66806cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:51.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.381+0000 7fd667200640 1 -- 172.21.15.86:0/1194646982 shutdown_connections 2024-04-25T14:04:51.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.381+0000 7fd667200640 1 -- 172.21.15.86:0/1194646982 wait complete. 2024-04-25T14:04:51.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.385+0000 7fd667200640 1 Processor -- start 2024-04-25T14:04:51.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.385+0000 7fd667200640 1 -- start start 2024-04-25T14:04:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.385+0000 7fd667200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6680b11b0 0x7fd668147180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.385+0000 7fd667200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6680b6ff0 0x7fd668156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.385+0000 7fd667200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6680ce320 0x7fd66814d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.385+0000 7fd667200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd66816a8a0 con 0x7fd6680b11b0 2024-04-25T14:04:51.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.385+0000 7fd667200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd668070c60 con 0x7fd6680b6ff0 2024-04-25T14:04:51.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.385+0000 7fd667200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd66816aa20 con 0x7fd6680ce320 2024-04-25T14:04:51.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.385+0000 7fd665e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6680b11b0 0x7fd668147180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:51.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.385+0000 7fd65fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6680b6ff0 0x7fd668156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:51.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.385+0000 7fd665e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6680b11b0 0x7fd668147180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52224/0 (socket says 172.21.15.86:52224) 2024-04-25T14:04:51.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.385+0000 7fd665e00640 1 -- 172.21.15.86:0/2501731588 learned_addr learned my addr 172.21.15.86:0/2501731588 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:51.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.385+0000 7fd666800640 1 --2- 172.21.15.86:0/2501731588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6680ce320 0x7fd66814d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:51.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.385+0000 7fd665e00640 1 -- 172.21.15.86:0/2501731588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6680ce320 msgr2=0x7fd66814d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:51.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.385+0000 7fd665e00640 1 --2- 172.21.15.86:0/2501731588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6680ce320 0x7fd66814d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:51.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.385+0000 7fd665e00640 1 -- 172.21.15.86:0/2501731588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6680b6ff0 msgr2=0x7fd668156270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:51.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.385+0000 7fd665e00640 1 --2- 172.21.15.86:0/2501731588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6680b6ff0 0x7fd668156270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.385+0000 7fd665e00640 1 -- 172.21.15.86:0/2501731588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd668156ec0 con 0x7fd6680b11b0 2024-04-25T14:04:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.385+0000 7fd65fe00640 1 --2- 172.21.15.86:0/2501731588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6680b6ff0 0x7fd668156270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.389+0000 7fd665e00640 1 --2- 172.21.15.86:0/2501731588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6680b11b0 0x7fd668147180 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd65800c370 tx=0x7fd65800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.389+0000 7fd65e000640 1 -- 172.21.15.86:0/2501731588 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6580052d0 con 0x7fd6680b11b0 2024-04-25T14:04:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.389+0000 7fd65e000640 1 -- 172.21.15.86:0/2501731588 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd658005470 con 0x7fd6680b11b0 2024-04-25T14:04:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.389+0000 7fd667200640 1 -- 172.21.15.86:0/2501731588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6681571a0 con 0x7fd6680b11b0 2024-04-25T14:04:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.389+0000 7fd65e000640 1 -- 172.21.15.86:0/2501731588 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd658007260 con 0x7fd6680b11b0 2024-04-25T14:04:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.389+0000 7fd667200640 1 -- 172.21.15.86:0/2501731588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd66815f1e0 con 0x7fd6680b11b0 2024-04-25T14:04:51.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.389+0000 7fd667200640 1 -- 172.21.15.86:0/2501731588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd66815f440 con 0x7fd6680b11b0 2024-04-25T14:04:51.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.389+0000 7fd65e000640 1 -- 172.21.15.86:0/2501731588 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd658007400 con 0x7fd6680b11b0 2024-04-25T14:04:51.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.389+0000 7fd65e000640 1 --2- 172.21.15.86:0/2501731588 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd64406f430 0x7fd644071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:51.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.389+0000 7fd65e000640 1 -- 172.21.15.86:0/2501731588 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd65808eab0 con 0x7fd6680b11b0 2024-04-25T14:04:51.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.389+0000 7fd65fe00640 1 --2- 172.21.15.86:0/2501731588 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd64406f430 0x7fd644071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:51.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.393+0000 7fd65fe00640 1 --2- 172.21.15.86:0/2501731588 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd64406f430 0x7fd644071940 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd668071870 tx=0x7fd6500a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:51.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.393+0000 7fd65e000640 1 -- 172.21.15.86:0/2501731588 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd65805af40 con 0x7fd6680b11b0 2024-04-25T14:04:51.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.557+0000 7fd667200640 1 -- 172.21.15.86:0/2501731588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd668076b80 con 0x7fd6680b11b0 2024-04-25T14:04:51.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.561+0000 7fd65e000640 1 -- 172.21.15.86:0/2501731588 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18902 (secure 0 0 0) 0x7fd65805f240 con 0x7fd6680b11b0 2024-04-25T14:04:51.562 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:51.562 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":26,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:04:31.930803+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:03:49.151473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:03:49.802597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T14:03:53.492008+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:04:51.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.561+0000 7fd667200640 1 -- 172.21.15.86:0/2501731588 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd64406f430 msgr2=0x7fd644071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:51.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.561+0000 7fd667200640 1 --2- 172.21.15.86:0/2501731588 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd64406f430 0x7fd644071940 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd668071870 tx=0x7fd6500a7000 comp rx=0 tx=0).stop 2024-04-25T14:04:51.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.561+0000 7fd667200640 1 -- 172.21.15.86:0/2501731588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6680b11b0 msgr2=0x7fd668147180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:51.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.561+0000 7fd667200640 1 --2- 172.21.15.86:0/2501731588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6680b11b0 0x7fd668147180 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd65800c370 tx=0x7fd65800c830 comp rx=0 tx=0).stop 2024-04-25T14:04:51.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.561+0000 7fd667200640 1 -- 172.21.15.86:0/2501731588 shutdown_connections 2024-04-25T14:04:51.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.561+0000 7fd667200640 1 --2- 172.21.15.86:0/2501731588 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd64406f430 0x7fd644071940 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:51.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.561+0000 7fd667200640 1 --2- 172.21.15.86:0/2501731588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6680ce320 0x7fd66814d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:51.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.561+0000 7fd667200640 1 --2- 172.21.15.86:0/2501731588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6680b6ff0 0x7fd668156270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:51.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.561+0000 7fd667200640 1 --2- 172.21.15.86:0/2501731588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6680b11b0 0x7fd668147180 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:51.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.561+0000 7fd667200640 1 -- 172.21.15.86:0/2501731588 >> 172.21.15.86:0/2501731588 conn(0x7fd66806c840 msgr2=0x7fd66806cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:51.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.561+0000 7fd667200640 1 -- 172.21.15.86:0/2501731588 shutdown_connections 2024-04-25T14:04:51.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.561+0000 7fd667200640 1 -- 172.21.15.86:0/2501731588 wait complete. 2024-04-25T14:04:51.583 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:03:49.151473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:03:49.802597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T14:03:53.492008+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T14:04:51.584 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:04:51.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.733+0000 7fa7ac600640 1 Processor -- start 2024-04-25T14:04:51.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.733+0000 7fa7ac600640 1 -- start start 2024-04-25T14:04:51.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.737+0000 7fa7ac600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7a40b59e0 0x7fa7a40b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:51.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.737+0000 7fa7ac600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa7a40aeea0 0x7fa7a40af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:51.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.737+0000 7fa7ac600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7a40b6330 con 0x7fa7a415faa0 2024-04-25T14:04:51.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.737+0000 7fa7ac600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa7a406d560 con 0x7fa7a40aeea0 2024-04-25T14:04:51.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.737+0000 7fa7ac600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa7a406c670 con 0x7fa7a40b59e0 2024-04-25T14:04:51.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.737+0000 7fa7aa600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa7a40aeea0 0x7fa7a40af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:51.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.737+0000 7fa7aa600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa7a40aeea0 0x7fa7a40af2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52236/0 (socket says 172.21.15.86:52236) 2024-04-25T14:04:51.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.737+0000 7fa7aa600640 1 -- 172.21.15.86:0/1867390161 learned_addr learned my addr 172.21.15.86:0/1867390161 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:51.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.737+0000 7fa7a9200640 1 --2- 172.21.15.86:0/1867390161 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7a40b59e0 0x7fa7a40b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:51.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.737+0000 7fa7aa600640 1 -- 172.21.15.86:0/1867390161 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7a40b59e0 msgr2=0x7fa7a40b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:51.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.737+0000 7fa7aa600640 1 --2- 172.21.15.86:0/1867390161 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7a40b59e0 0x7fa7a40b5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:51.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.737+0000 7fa7aa600640 1 -- 172.21.15.86:0/1867390161 >> v1:172.21.15.155:6789/0 conn(0x7fa7a415faa0 legacy=0x7fa7a406d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.737+0000 7fa7aa600640 1 -- 172.21.15.86:0/1867390161 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a406c950 con 0x7fa7a40aeea0 2024-04-25T14:04:51.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.737+0000 7fa7aa600640 1 --2- 172.21.15.86:0/1867390161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7a40aeea0 0x7fa7a40af2b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa79c009e50 tx=0x7fa79c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65a133f4de5d718 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:51.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.741+0000 7fa7a3e00640 1 -- 172.21.15.86:0/1867390161 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c0a5920 con 0x7fa7a40aeea0 2024-04-25T14:04:51.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.741+0000 7fa7a3e00640 1 -- 172.21.15.86:0/1867390161 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa79c0a5ac0 con 0x7fa7a40aeea0 2024-04-25T14:04:51.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.741+0000 7fa7a3e00640 1 -- 172.21.15.86:0/1867390161 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c0ad3f0 con 0x7fa7a40aeea0 2024-04-25T14:04:51.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.741+0000 7fa7ac600640 1 -- 172.21.15.86:0/1867390161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7a40aeea0 msgr2=0x7fa7a40af2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:51.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.741+0000 7fa7ac600640 1 --2- 172.21.15.86:0/1867390161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7a40aeea0 0x7fa7a40af2b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa79c009e50 tx=0x7fa79c01ca30 comp rx=0 tx=0).stop 2024-04-25T14:04:51.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.741+0000 7fa7ac600640 1 -- 172.21.15.86:0/1867390161 shutdown_connections 2024-04-25T14:04:51.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.741+0000 7fa7ac600640 1 --2- 172.21.15.86:0/1867390161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7a40aeea0 0x7fa7a40af2b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:51.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.741+0000 7fa7ac600640 1 --2- 172.21.15.86:0/1867390161 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7a40b59e0 0x7fa7a40b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:51.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.741+0000 7fa7ac600640 1 -- 172.21.15.86:0/1867390161 >> 172.21.15.86:0/1867390161 conn(0x7fa7a40d2230 msgr2=0x7fa7a40d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:51.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.741+0000 7fa7ac600640 1 -- 172.21.15.86:0/1867390161 shutdown_connections 2024-04-25T14:04:51.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.741+0000 7fa7ac600640 1 -- 172.21.15.86:0/1867390161 wait complete. 2024-04-25T14:04:51.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.741+0000 7fa7ac600640 1 Processor -- start 2024-04-25T14:04:51.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.741+0000 7fa7ac600640 1 -- start start 2024-04-25T14:04:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.745+0000 7fa7ac600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7a40aeea0 0x7fa7a415d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.745+0000 7fa7ac600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7a40b59e0 0x7fa7a4151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.745+0000 7fa7ac600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7a415faa0 0x7fa7a4158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.745+0000 7fa7ac600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a4160030 con 0x7fa7a40b59e0 2024-04-25T14:04:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.745+0000 7fa7ac600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a415feb0 con 0x7fa7a415faa0 2024-04-25T14:04:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.745+0000 7fa7ac600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa7a4070f30 con 0x7fa7a40aeea0 2024-04-25T14:04:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.745+0000 7fa7a9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7a40b59e0 0x7fa7a4151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.745+0000 7fa7a9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7a40b59e0 0x7fa7a4151960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52240/0 (socket says 172.21.15.86:52240) 2024-04-25T14:04:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.745+0000 7fa7a9200640 1 -- 172.21.15.86:0/2049948007 learned_addr learned my addr 172.21.15.86:0/2049948007 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.745+0000 7fa7a9c00640 1 --2- 172.21.15.86:0/2049948007 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7a40aeea0 0x7fa7a415d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.745+0000 7fa7aa600640 1 --2- 172.21.15.86:0/2049948007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7a415faa0 0x7fa7a4158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.745+0000 7fa7a9200640 1 -- 172.21.15.86:0/2049948007 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7a40aeea0 msgr2=0x7fa7a415d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.745+0000 7fa7a9200640 1 --2- 172.21.15.86:0/2049948007 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7a40aeea0 0x7fa7a415d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.745+0000 7fa7a9200640 1 -- 172.21.15.86:0/2049948007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7a415faa0 msgr2=0x7fa7a4158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.745+0000 7fa7a9200640 1 --2- 172.21.15.86:0/2049948007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7a415faa0 0x7fa7a4158c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.745+0000 7fa7a9200640 1 -- 172.21.15.86:0/2049948007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a4152a80 con 0x7fa7a40b59e0 2024-04-25T14:04:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.745+0000 7fa7a9200640 1 --2- 172.21.15.86:0/2049948007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7a40b59e0 0x7fa7a4151960 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa79000e870 tx=0x7fa79000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.745+0000 7fa7a1600640 1 -- 172.21.15.86:0/2049948007 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa790017070 con 0x7fa7a40b59e0 2024-04-25T14:04:51.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.745+0000 7fa7a1600640 1 -- 172.21.15.86:0/2049948007 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa79000c940 con 0x7fa7a40b59e0 2024-04-25T14:04:51.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.745+0000 7fa7ac600640 1 -- 172.21.15.86:0/2049948007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7a4156db0 con 0x7fa7a40b59e0 2024-04-25T14:04:51.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.745+0000 7fa7a1600640 1 -- 172.21.15.86:0/2049948007 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79000ccf0 con 0x7fa7a40b59e0 2024-04-25T14:04:51.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.745+0000 7fa7ac600640 1 -- 172.21.15.86:0/2049948007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7a4157370 con 0x7fa7a40b59e0 2024-04-25T14:04:51.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.745+0000 7fa7ac600640 1 -- 172.21.15.86:0/2049948007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7a4156b20 con 0x7fa7a40b59e0 2024-04-25T14:04:51.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.749+0000 7fa7a1600640 1 -- 172.21.15.86:0/2049948007 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa79001a070 con 0x7fa7a40b59e0 2024-04-25T14:04:51.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.749+0000 7fa7a1600640 1 --2- 172.21.15.86:0/2049948007 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa78406f400 0x7fa784071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:51.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.749+0000 7fa7a1600640 1 -- 172.21.15.86:0/2049948007 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa79008f520 con 0x7fa7a40b59e0 2024-04-25T14:04:51.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.749+0000 7fa7a9c00640 1 --2- 172.21.15.86:0/2049948007 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa78406f400 0x7fa784071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:51.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.749+0000 7fa7a9c00640 1 --2- 172.21.15.86:0/2049948007 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa78406f400 0x7fa784071910 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa798001770 tx=0x7fa798009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:51.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.753+0000 7fa7a1600640 1 -- 172.21.15.86:0/2049948007 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa7a4152a80 con 0x7fa7a40b59e0 2024-04-25T14:04:51.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.909+0000 7fa7ac600640 1 -- 172.21.15.86:0/2049948007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa7a4074cf0 con 0x7fa7a40b59e0 2024-04-25T14:04:51.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.909+0000 7fa7a1600640 1 -- 172.21.15.86:0/2049948007 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fa79005b9b0 con 0x7fa7a40b59e0 2024-04-25T14:04:51.911 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:04:51.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.909+0000 7fa7ac600640 1 -- 172.21.15.86:0/2049948007 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa78406f400 msgr2=0x7fa784071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:51.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.909+0000 7fa7ac600640 1 --2- 172.21.15.86:0/2049948007 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa78406f400 0x7fa784071910 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa798001770 tx=0x7fa798009040 comp rx=0 tx=0).stop 2024-04-25T14:04:51.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.909+0000 7fa7ac600640 1 -- 172.21.15.86:0/2049948007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7a40b59e0 msgr2=0x7fa7a4151960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:51.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.909+0000 7fa7ac600640 1 --2- 172.21.15.86:0/2049948007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7a40b59e0 0x7fa7a4151960 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa79000e870 tx=0x7fa79000ec70 comp rx=0 tx=0).stop 2024-04-25T14:04:51.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.909+0000 7fa7ac600640 1 -- 172.21.15.86:0/2049948007 shutdown_connections 2024-04-25T14:04:51.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.909+0000 7fa7ac600640 1 --2- 172.21.15.86:0/2049948007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7a415faa0 0x7fa7a4158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:51.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.909+0000 7fa7ac600640 1 --2- 172.21.15.86:0/2049948007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7a40b59e0 0x7fa7a4151960 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:51.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.909+0000 7fa7ac600640 1 --2- 172.21.15.86:0/2049948007 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa78406f400 0x7fa784071910 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:51.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.909+0000 7fa7ac600640 1 --2- 172.21.15.86:0/2049948007 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7a40aeea0 0x7fa7a415d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:51.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.909+0000 7fa7ac600640 1 -- 172.21.15.86:0/2049948007 >> 172.21.15.86:0/2049948007 conn(0x7fa7a40d2230 msgr2=0x7fa7a4163470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:51.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.909+0000 7fa7ac600640 1 -- 172.21.15.86:0/2049948007 shutdown_connections 2024-04-25T14:04:51.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:51.909+0000 7fa7ac600640 1 -- 172.21.15.86:0/2049948007 wait complete. 2024-04-25T14:04:51.932 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:04:52.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.081+0000 7f7b87400640 1 Processor -- start 2024-04-25T14:04:52.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.081+0000 7f7b87400640 1 -- start start 2024-04-25T14:04:52.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.081+0000 7f7b87400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7b8015ec70 0x7f7b800d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:52.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.081+0000 7f7b87400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b800d1490 0x7f7b8006d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:52.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.081+0000 7f7b87400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b800cf8f0 con 0x7f7b8015e860 2024-04-25T14:04:52.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.081+0000 7f7b87400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7b8006cfa0 con 0x7f7b8015ec70 2024-04-25T14:04:52.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.081+0000 7f7b87400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7b8006c670 con 0x7f7b800d1490 2024-04-25T14:04:52.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.081+0000 7f7b7fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7b8015ec70 0x7f7b800d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:52.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.081+0000 7f7b7fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7b8015ec70 0x7f7b800d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52242/0 (socket says 172.21.15.86:52242) 2024-04-25T14:04:52.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.081+0000 7f7b7fe00640 1 -- 172.21.15.86:0/3123627457 learned_addr learned my addr 172.21.15.86:0/3123627457 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:52.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.081+0000 7f7b84a00640 1 --1- 172.21.15.86:0/3123627457 >> v1:172.21.15.86:6789/0 conn(0x7f7b8015e860 0x7f7b800ceea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54424/0 (socket says 172.21.15.86:54424) 2024-04-25T14:04:52.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.081+0000 7f7b85400640 1 --2- 172.21.15.86:0/3123627457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b800d1490 0x7f7b8006d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:52.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.081+0000 7f7b7fe00640 1 -- 172.21.15.86:0/3123627457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b800d1490 msgr2=0x7f7b8006d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:52.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.081+0000 7f7b7fe00640 1 --2- 172.21.15.86:0/3123627457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b800d1490 0x7f7b8006d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.081+0000 7f7b7fe00640 1 -- 172.21.15.86:0/3123627457 >> v1:172.21.15.86:6789/0 conn(0x7f7b8015e860 legacy=0x7f7b800ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:52.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.081+0000 7f7b7fe00640 1 -- 172.21.15.86:0/3123627457 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b8006c950 con 0x7f7b8015ec70 2024-04-25T14:04:52.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.081+0000 7f7b7f400640 1 -- 172.21.15.86:0/3123627457 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3646939901 0 0) 0x7f7b800cf8f0 con 0x7f7b8015e860 2024-04-25T14:04:52.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.085+0000 7f7b7fe00640 1 --2- 172.21.15.86:0/3123627457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b8015ec70 0x7f7b800d0f50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009870 tx=0x7f7b6c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e106af2977bb0ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:52.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.085+0000 7f7b7f400640 1 -- 172.21.15.86:0/3123627457 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c0ab430 con 0x7f7b8015ec70 2024-04-25T14:04:52.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.085+0000 7f7b7f400640 1 -- 172.21.15.86:0/3123627457 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b6c0b0070 con 0x7f7b8015ec70 2024-04-25T14:04:52.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.085+0000 7f7b7f400640 1 -- 172.21.15.86:0/3123627457 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c0ab7f0 con 0x7f7b8015ec70 2024-04-25T14:04:52.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.085+0000 7f7b87400640 1 -- 172.21.15.86:0/3123627457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b8015ec70 msgr2=0x7f7b800d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:52.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.085+0000 7f7b87400640 1 --2- 172.21.15.86:0/3123627457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b8015ec70 0x7f7b800d0f50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009870 tx=0x7f7b6c01c4a0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.085+0000 7f7b87400640 1 -- 172.21.15.86:0/3123627457 shutdown_connections 2024-04-25T14:04:52.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.085+0000 7f7b87400640 1 --2- 172.21.15.86:0/3123627457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b800d1490 0x7f7b8006d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.085+0000 7f7b87400640 1 --2- 172.21.15.86:0/3123627457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b8015ec70 0x7f7b800d0f50 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.085+0000 7f7b87400640 1 -- 172.21.15.86:0/3123627457 >> 172.21.15.86:0/3123627457 conn(0x7f7b800d2230 msgr2=0x7f7b800d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:52.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.085+0000 7f7b87400640 1 -- 172.21.15.86:0/3123627457 shutdown_connections 2024-04-25T14:04:52.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.085+0000 7f7b87400640 1 -- 172.21.15.86:0/3123627457 wait complete. 2024-04-25T14:04:52.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.089+0000 7f7b87400640 1 Processor -- start 2024-04-25T14:04:52.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.089+0000 7f7b87400640 1 -- start start 2024-04-25T14:04:52.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.089+0000 7f7b87400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b800d1490 0x7f7b801626c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:52.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.089+0000 7f7b87400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b8015e860 0x7f7b801536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:52.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.089+0000 7f7b87400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b8015ec70 0x7f7b8015ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:52.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.089+0000 7f7b87400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8016a720 con 0x7f7b8015ec70 2024-04-25T14:04:52.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.089+0000 7f7b87400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7b800d18a0 con 0x7f7b800d1490 2024-04-25T14:04:52.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.089+0000 7f7b87400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7b8016a8a0 con 0x7f7b8015e860 2024-04-25T14:04:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.089+0000 7f7b85400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b8015ec70 0x7f7b8015ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.089+0000 7f7b85400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b8015ec70 0x7f7b8015ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52250/0 (socket says 172.21.15.86:52250) 2024-04-25T14:04:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.089+0000 7f7b85400640 1 -- 172.21.15.86:0/1553527897 learned_addr learned my addr 172.21.15.86:0/1553527897 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.089+0000 7f7b7fe00640 1 --2- 172.21.15.86:0/1553527897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b8015e860 0x7f7b801536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.089+0000 7f7b84a00640 1 --2- 172.21.15.86:0/1553527897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b800d1490 0x7f7b801626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.089+0000 7f7b85400640 1 -- 172.21.15.86:0/1553527897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b8015e860 msgr2=0x7f7b801536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.089+0000 7f7b85400640 1 --2- 172.21.15.86:0/1553527897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b8015e860 0x7f7b801536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.089+0000 7f7b85400640 1 -- 172.21.15.86:0/1553527897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b800d1490 msgr2=0x7f7b801626c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.089+0000 7f7b85400640 1 --2- 172.21.15.86:0/1553527897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b800d1490 0x7f7b801626c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.089+0000 7f7b85400640 1 -- 172.21.15.86:0/1553527897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b80162f10 con 0x7f7b8015ec70 2024-04-25T14:04:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.089+0000 7f7b84a00640 1 --2- 172.21.15.86:0/1553527897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b800d1490 0x7f7b801626c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.093+0000 7f7b85400640 1 --2- 172.21.15.86:0/1553527897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b8015ec70 0x7f7b8015ca70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7b7800ef90 tx=0x7f7b7800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.093+0000 7f7b7cc00640 1 -- 172.21.15.86:0/1553527897 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b78018020 con 0x7f7b8015ec70 2024-04-25T14:04:52.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.093+0000 7f7b7cc00640 1 -- 172.21.15.86:0/1553527897 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b78014480 con 0x7f7b8015ec70 2024-04-25T14:04:52.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.093+0000 7f7b87400640 1 -- 172.21.15.86:0/1553527897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b801631f0 con 0x7f7b8015ec70 2024-04-25T14:04:52.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.093+0000 7f7b7cc00640 1 -- 172.21.15.86:0/1553527897 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b78009e60 con 0x7f7b8015ec70 2024-04-25T14:04:52.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.093+0000 7f7b87400640 1 -- 172.21.15.86:0/1553527897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b801637b0 con 0x7f7b8015ec70 2024-04-25T14:04:52.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.093+0000 7f7b87400640 1 -- 172.21.15.86:0/1553527897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b80074cf0 con 0x7f7b8015ec70 2024-04-25T14:04:52.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.093+0000 7f7b7cc00640 1 -- 172.21.15.86:0/1553527897 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7b7801b020 con 0x7f7b8015ec70 2024-04-25T14:04:52.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.093+0000 7f7b7cc00640 1 --2- 172.21.15.86:0/1553527897 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7b6006f400 0x7f7b60071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:52.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.093+0000 7f7b7cc00640 1 -- 172.21.15.86:0/1553527897 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f7b780904c0 con 0x7f7b8015ec70 2024-04-25T14:04:52.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.093+0000 7f7b84a00640 1 --2- 172.21.15.86:0/1553527897 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7b6006f400 0x7f7b60071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:52.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.097+0000 7f7b84a00640 1 --2- 172.21.15.86:0/1553527897 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7b6006f400 0x7f7b60071910 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7b74000f70 tx=0x7f7b74007cb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:52.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.097+0000 7f7b7cc00640 1 -- 172.21.15.86:0/1553527897 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7b78022020 con 0x7f7b8015ec70 2024-04-25T14:04:52.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.253+0000 7f7b87400640 1 -- 172.21.15.86:0/1553527897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7b80076bf0 con 0x7f7b8015ec70 2024-04-25T14:04:52.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.253+0000 7f7b7cc00640 1 -- 172.21.15.86:0/1553527897 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f7b78010040 con 0x7f7b8015ec70 2024-04-25T14:04:52.256 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:04:52.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.257+0000 7f7b87400640 1 -- 172.21.15.86:0/1553527897 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7b6006f400 msgr2=0x7f7b60071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:52.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.257+0000 7f7b87400640 1 --2- 172.21.15.86:0/1553527897 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7b6006f400 0x7f7b60071910 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7b74000f70 tx=0x7f7b74007cb0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.257+0000 7f7b87400640 1 -- 172.21.15.86:0/1553527897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b8015ec70 msgr2=0x7f7b8015ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:52.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.257+0000 7f7b87400640 1 --2- 172.21.15.86:0/1553527897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b8015ec70 0x7f7b8015ca70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7b7800ef90 tx=0x7f7b7800c490 comp rx=0 tx=0).stop 2024-04-25T14:04:52.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.257+0000 7f7b87400640 1 -- 172.21.15.86:0/1553527897 shutdown_connections 2024-04-25T14:04:52.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.257+0000 7f7b87400640 1 --2- 172.21.15.86:0/1553527897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b8015ec70 0x7f7b8015ca70 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.257+0000 7f7b87400640 1 --2- 172.21.15.86:0/1553527897 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f7b6006f400 0x7f7b60071910 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.257+0000 7f7b87400640 1 --2- 172.21.15.86:0/1553527897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b8015e860 0x7f7b801536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.257+0000 7f7b87400640 1 --2- 172.21.15.86:0/1553527897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b800d1490 0x7f7b801626c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.257+0000 7f7b87400640 1 -- 172.21.15.86:0/1553527897 >> 172.21.15.86:0/1553527897 conn(0x7f7b800d2230 msgr2=0x7f7b800b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:52.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.257+0000 7f7b87400640 1 -- 172.21.15.86:0/1553527897 shutdown_connections 2024-04-25T14:04:52.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.257+0000 7f7b87400640 1 -- 172.21.15.86:0/1553527897 wait complete. 2024-04-25T14:04:52.277 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:04:52.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.425+0000 7ff889c00640 1 Processor -- start 2024-04-25T14:04:52.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.425+0000 7ff889c00640 1 -- start start 2024-04-25T14:04:52.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.429+0000 7ff889c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff884070330 0x7ff8840d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:52.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.429+0000 7ff889c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8840ce790 0x7ff88406f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:52.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.429+0000 7ff889c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8840d2030 con 0x7ff8840d1c20 2024-04-25T14:04:52.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.429+0000 7ff889c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff88415e5c0 con 0x7ff884070330 2024-04-25T14:04:52.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.429+0000 7ff889c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff88415e8a0 con 0x7ff8840ce790 2024-04-25T14:04:52.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.429+0000 7ff882a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff884070330 0x7ff8840d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:52.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.429+0000 7ff87a000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff8840d1c20 0x7ff88406ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54436/0 (socket says 172.21.15.86:54436) 2024-04-25T14:04:52.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.429+0000 7ff87a000640 1 -- 172.21.15.86:0/958863063 learned_addr learned my addr 172.21.15.86:0/958863063 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:52.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.429+0000 7ff883400640 1 --2- 172.21.15.86:0/958863063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8840ce790 0x7ff88406f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:52.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.429+0000 7ff882a00640 1 -- 172.21.15.86:0/958863063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8840ce790 msgr2=0x7ff88406f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:52.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.429+0000 7ff882a00640 1 --2- 172.21.15.86:0/958863063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8840ce790 0x7ff88406f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.429+0000 7ff882a00640 1 -- 172.21.15.86:0/958863063 >> v1:172.21.15.86:6789/0 conn(0x7ff8840d1c20 legacy=0x7ff88406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:52.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.429+0000 7ff882a00640 1 -- 172.21.15.86:0/958863063 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88415eb80 con 0x7ff884070330 2024-04-25T14:04:52.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.429+0000 7ff883400640 1 --2- 172.21.15.86:0/958863063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8840ce790 0x7ff88406f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:52.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.433+0000 7ff882a00640 1 --2- 172.21.15.86:0/958863063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff884070330 0x7ff8840d16e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff86c0097c0 tx=0x7ff86c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6b908d6010f5c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:52.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.433+0000 7ff882000640 1 -- 172.21.15.86:0/958863063 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c0b0070 con 0x7ff884070330 2024-04-25T14:04:52.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.433+0000 7ff882000640 1 -- 172.21.15.86:0/958863063 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff86c0ab600 con 0x7ff884070330 2024-04-25T14:04:52.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.433+0000 7ff882000640 1 -- 172.21.15.86:0/958863063 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c0ab9d0 con 0x7ff884070330 2024-04-25T14:04:52.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.433+0000 7ff889c00640 1 -- 172.21.15.86:0/958863063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff884070330 msgr2=0x7ff8840d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:52.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.433+0000 7ff889c00640 1 --2- 172.21.15.86:0/958863063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff884070330 0x7ff8840d16e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff86c0097c0 tx=0x7ff86c01c410 comp rx=0 tx=0).stop 2024-04-25T14:04:52.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.433+0000 7ff889c00640 1 -- 172.21.15.86:0/958863063 shutdown_connections 2024-04-25T14:04:52.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.433+0000 7ff889c00640 1 --2- 172.21.15.86:0/958863063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8840ce790 0x7ff88406f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.433+0000 7ff889c00640 1 --2- 172.21.15.86:0/958863063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff884070330 0x7ff8840d16e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.433+0000 7ff889c00640 1 -- 172.21.15.86:0/958863063 >> 172.21.15.86:0/958863063 conn(0x7ff8840b4650 msgr2=0x7ff8840b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:52.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.433+0000 7ff889c00640 1 -- 172.21.15.86:0/958863063 shutdown_connections 2024-04-25T14:04:52.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.433+0000 7ff889c00640 1 -- 172.21.15.86:0/958863063 wait complete. 2024-04-25T14:04:52.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.433+0000 7ff889c00640 1 Processor -- start 2024-04-25T14:04:52.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.433+0000 7ff889c00640 1 -- start start 2024-04-25T14:04:52.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.437+0000 7ff889c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff884070330 0x7ff88415afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:52.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.437+0000 7ff889c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8840ce790 0x7ff8840c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:52.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.437+0000 7ff889c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8840d1c20 0x7ff8840b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:52.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.437+0000 7ff889c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff8840708c0 con 0x7ff884070330 2024-04-25T14:04:52.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.437+0000 7ff889c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff884070740 con 0x7ff8840d1c20 2024-04-25T14:04:52.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.437+0000 7ff889c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff88416afc0 con 0x7ff8840ce790 2024-04-25T14:04:52.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.437+0000 7ff882a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff884070330 0x7ff88415afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:52.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.437+0000 7ff882a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff884070330 0x7ff88415afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52260/0 (socket says 172.21.15.86:52260) 2024-04-25T14:04:52.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.437+0000 7ff882a00640 1 -- 172.21.15.86:0/1843794134 learned_addr learned my addr 172.21.15.86:0/1843794134 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:52.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.437+0000 7ff883400640 1 --2- 172.21.15.86:0/1843794134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8840d1c20 0x7ff8840b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:52.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.437+0000 7ff87a000640 1 --2- 172.21.15.86:0/1843794134 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8840ce790 0x7ff8840c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:52.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.437+0000 7ff882a00640 1 -- 172.21.15.86:0/1843794134 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8840ce790 msgr2=0x7ff8840c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:52.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.437+0000 7ff882a00640 1 --2- 172.21.15.86:0/1843794134 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8840ce790 0x7ff8840c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.437+0000 7ff882a00640 1 -- 172.21.15.86:0/1843794134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8840d1c20 msgr2=0x7ff8840b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:52.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.437+0000 7ff882a00640 1 --2- 172.21.15.86:0/1843794134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8840d1c20 0x7ff8840b0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.437+0000 7ff882a00640 1 -- 172.21.15.86:0/1843794134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8840cff20 con 0x7ff884070330 2024-04-25T14:04:52.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.437+0000 7ff883400640 1 --2- 172.21.15.86:0/1843794134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8840d1c20 0x7ff8840b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:52.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.437+0000 7ff87a000640 1 --2- 172.21.15.86:0/1843794134 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8840ce790 0x7ff8840c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:52.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.437+0000 7ff882a00640 1 --2- 172.21.15.86:0/1843794134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff884070330 0x7ff88415afd0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff86c0ae040 tx=0x7ff86c01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:52.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.437+0000 7ff87b400640 1 -- 172.21.15.86:0/1843794134 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c0b0050 con 0x7ff884070330 2024-04-25T14:04:52.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.437+0000 7ff87b400640 1 -- 172.21.15.86:0/1843794134 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff86c0bc070 con 0x7ff884070330 2024-04-25T14:04:52.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.437+0000 7ff87b400640 1 -- 172.21.15.86:0/1843794134 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c0b5a70 con 0x7ff884070330 2024-04-25T14:04:52.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.437+0000 7ff889c00640 1 -- 172.21.15.86:0/1843794134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8840d0200 con 0x7ff884070330 2024-04-25T14:04:52.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.437+0000 7ff889c00640 1 -- 172.21.15.86:0/1843794134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8840d07c0 con 0x7ff884070330 2024-04-25T14:04:52.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.437+0000 7ff889c00640 1 -- 172.21.15.86:0/1843794134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff884069d00 con 0x7ff884070330 2024-04-25T14:04:52.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.441+0000 7ff87b400640 1 -- 172.21.15.86:0/1843794134 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff86c0ab4c0 con 0x7ff884070330 2024-04-25T14:04:52.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.441+0000 7ff87b400640 1 --2- 172.21.15.86:0/1843794134 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff86006f400 0x7ff860071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:52.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.441+0000 7ff87b400640 1 -- 172.21.15.86:0/1843794134 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff86c12aab0 con 0x7ff884070330 2024-04-25T14:04:52.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.441+0000 7ff87a000640 1 --2- 172.21.15.86:0/1843794134 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff86006f400 0x7ff860071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:52.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.441+0000 7ff87a000640 1 --2- 172.21.15.86:0/1843794134 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff86006f400 0x7ff860071910 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff8840becf0 tx=0x7ff874009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:52.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.445+0000 7ff87b400640 1 -- 172.21.15.86:0/1843794134 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff86c016090 con 0x7ff884070330 2024-04-25T14:04:52.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.601+0000 7ff889c00640 1 -- 172.21.15.86:0/1843794134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff884076bf0 con 0x7ff884070330 2024-04-25T14:04:52.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.601+0000 7ff87b400640 1 -- 172.21.15.86:0/1843794134 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7ff86c0f76a0 con 0x7ff884070330 2024-04-25T14:04:52.603 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:04:52.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.605+0000 7ff889c00640 1 -- 172.21.15.86:0/1843794134 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff86006f400 msgr2=0x7ff860071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:52.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.605+0000 7ff889c00640 1 --2- 172.21.15.86:0/1843794134 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff86006f400 0x7ff860071910 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff8840becf0 tx=0x7ff874009040 comp rx=0 tx=0).stop 2024-04-25T14:04:52.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.605+0000 7ff889c00640 1 -- 172.21.15.86:0/1843794134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff884070330 msgr2=0x7ff88415afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:52.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.605+0000 7ff889c00640 1 --2- 172.21.15.86:0/1843794134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff884070330 0x7ff88415afd0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff86c0ae040 tx=0x7ff86c01ac50 comp rx=0 tx=0).stop 2024-04-25T14:04:52.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.605+0000 7ff889c00640 1 -- 172.21.15.86:0/1843794134 shutdown_connections 2024-04-25T14:04:52.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.605+0000 7ff889c00640 1 --2- 172.21.15.86:0/1843794134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8840d1c20 0x7ff8840b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.605+0000 7ff889c00640 1 --2- 172.21.15.86:0/1843794134 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff86006f400 0x7ff860071910 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.605+0000 7ff889c00640 1 --2- 172.21.15.86:0/1843794134 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8840ce790 0x7ff8840c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.605+0000 7ff889c00640 1 --2- 172.21.15.86:0/1843794134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff884070330 0x7ff88415afd0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.605+0000 7ff889c00640 1 -- 172.21.15.86:0/1843794134 >> 172.21.15.86:0/1843794134 conn(0x7ff8840b4650 msgr2=0x7ff88415d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:52.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.605+0000 7ff889c00640 1 -- 172.21.15.86:0/1843794134 shutdown_connections 2024-04-25T14:04:52.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.605+0000 7ff889c00640 1 -- 172.21.15.86:0/1843794134 wait complete. 2024-04-25T14:04:52.625 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T14:04:52.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.773+0000 7f49f5200640 1 Processor -- start 2024-04-25T14:04:52.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.773+0000 7f49f5200640 1 -- start start 2024-04-25T14:04:52.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.773+0000 7f49f5200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f49f0070330 0x7f49f00d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:52.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.773+0000 7f49f5200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49f00ce790 0x7f49f006f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:52.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.773+0000 7f49f5200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49f00d2030 con 0x7f49f00d1c20 2024-04-25T14:04:52.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.773+0000 7f49f5200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f49f015e5c0 con 0x7f49f0070330 2024-04-25T14:04:52.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.773+0000 7f49f5200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f49f015e8a0 con 0x7f49f00ce790 2024-04-25T14:04:52.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.773+0000 7f49ee800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49f00ce790 0x7f49f006f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:52.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.773+0000 7f49ede00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f49f0070330 0x7f49f00d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:52.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.773+0000 7f49ee800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49f00ce790 0x7f49f006f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:49872/0 (socket says 172.21.15.86:49872) 2024-04-25T14:04:52.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.773+0000 7f49ee800640 1 -- 172.21.15.86:0/744889727 learned_addr learned my addr 172.21.15.86:0/744889727 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:52.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.777+0000 7f49ee800640 1 -- 172.21.15.86:0/744889727 >> v2:172.21.15.155:3300/0 conn(0x7f49f0070330 msgr2=0x7f49f00d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:52.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.777+0000 7f49ee800640 1 --2- 172.21.15.86:0/744889727 >> v2:172.21.15.155:3300/0 conn(0x7f49f0070330 0x7f49f00d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.777+0000 7f49ee800640 1 -- 172.21.15.86:0/744889727 >> v1:172.21.15.155:6789/0 conn(0x7f49f00d1c20 legacy=0x7f49f006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:52.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.777+0000 7f49ee800640 1 -- 172.21.15.86:0/744889727 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f015eb80 con 0x7f49f00ce790 2024-04-25T14:04:52.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.777+0000 7f49ee800640 1 --2- 172.21.15.86:0/744889727 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49f00ce790 0x7f49f006f440 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009870 tx=0x7f49e801c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f488b0cea209cf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:52.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.777+0000 7f49eca00640 1 -- 172.21.15.86:0/744889727 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e80b0070 con 0x7f49f00ce790 2024-04-25T14:04:52.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.777+0000 7f49eca00640 1 -- 172.21.15.86:0/744889727 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e80ab600 con 0x7f49f00ce790 2024-04-25T14:04:52.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.777+0000 7f49eca00640 1 -- 172.21.15.86:0/744889727 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e80ab9d0 con 0x7f49f00ce790 2024-04-25T14:04:52.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.781+0000 7f49f5200640 1 -- 172.21.15.86:0/744889727 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49f00ce790 msgr2=0x7f49f006f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:52.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.781+0000 7f49f5200640 1 --2- 172.21.15.86:0/744889727 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49f00ce790 0x7f49f006f440 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009870 tx=0x7f49e801c4d0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.781+0000 7f49f5200640 1 -- 172.21.15.86:0/744889727 shutdown_connections 2024-04-25T14:04:52.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.781+0000 7f49f5200640 1 --2- 172.21.15.86:0/744889727 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49f00ce790 0x7f49f006f440 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.781+0000 7f49f5200640 1 --2- 172.21.15.86:0/744889727 >> v2:172.21.15.155:3300/0 conn(0x7f49f0070330 0x7f49f00d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.781+0000 7f49f5200640 1 -- 172.21.15.86:0/744889727 >> 172.21.15.86:0/744889727 conn(0x7f49f00b4650 msgr2=0x7f49f00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:52.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.781+0000 7f49f5200640 1 -- 172.21.15.86:0/744889727 shutdown_connections 2024-04-25T14:04:52.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.781+0000 7f49f5200640 1 -- 172.21.15.86:0/744889727 wait complete. 2024-04-25T14:04:52.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.781+0000 7f49f5200640 1 Processor -- start 2024-04-25T14:04:52.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.781+0000 7f49f5200640 1 -- start start 2024-04-25T14:04:52.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.781+0000 7f49f5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49f0070330 0x7f49f0159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:52.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.781+0000 7f49f5200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49f00ce790 0x7f49f00738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:52.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.781+0000 7f49f5200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49f00d1c20 0x7f49f0073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:52.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.785+0000 7f49f5200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f49f00708c0 con 0x7f49f0070330 2024-04-25T14:04:52.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.785+0000 7f49f5200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f49f0070740 con 0x7f49f00ce790 2024-04-25T14:04:52.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.785+0000 7f49f5200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f49f016afc0 con 0x7f49f00d1c20 2024-04-25T14:04:52.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.785+0000 7f49ede00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49f0070330 0x7f49f0159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:52.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.785+0000 7f49ede00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49f0070330 0x7f49f0159fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52262/0 (socket says 172.21.15.86:52262) 2024-04-25T14:04:52.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.785+0000 7f49ede00640 1 -- 172.21.15.86:0/3513226157 learned_addr learned my addr 172.21.15.86:0/3513226157 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:52.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.785+0000 7f49ee800640 1 --2- 172.21.15.86:0/3513226157 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49f00d1c20 0x7f49f0073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:52.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.785+0000 7f49ed400640 1 --2- 172.21.15.86:0/3513226157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49f00ce790 0x7f49f00738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:52.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.785+0000 7f49ede00640 1 -- 172.21.15.86:0/3513226157 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49f00d1c20 msgr2=0x7f49f0073e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:52.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.785+0000 7f49ede00640 1 --2- 172.21.15.86:0/3513226157 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49f00d1c20 0x7f49f0073e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.785+0000 7f49ede00640 1 -- 172.21.15.86:0/3513226157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49f00ce790 msgr2=0x7f49f00738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:52.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.785+0000 7f49ede00640 1 --2- 172.21.15.86:0/3513226157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49f00ce790 0x7f49f00738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.785+0000 7f49ede00640 1 -- 172.21.15.86:0/3513226157 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f015b080 con 0x7f49f0070330 2024-04-25T14:04:52.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.785+0000 7f49ede00640 1 --2- 172.21.15.86:0/3513226157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49f0070330 0x7f49f0159fc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f49e000efc0 tx=0x7f49e000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:52.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.785+0000 7f49e6000640 1 -- 172.21.15.86:0/3513226157 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0018020 con 0x7f49f0070330 2024-04-25T14:04:52.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.785+0000 7f49e6000640 1 -- 172.21.15.86:0/3513226157 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e0014480 con 0x7f49f0070330 2024-04-25T14:04:52.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.785+0000 7f49e6000640 1 -- 172.21.15.86:0/3513226157 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0014850 con 0x7f49f0070330 2024-04-25T14:04:52.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.785+0000 7f49f5200640 1 -- 172.21.15.86:0/3513226157 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f00c4230 con 0x7f49f0070330 2024-04-25T14:04:52.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.785+0000 7f49f5200640 1 -- 172.21.15.86:0/3513226157 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f00c9240 con 0x7f49f0070330 2024-04-25T14:04:52.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.785+0000 7f49f5200640 1 -- 172.21.15.86:0/3513226157 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49f0069d00 con 0x7f49f0070330 2024-04-25T14:04:52.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.785+0000 7f49e6000640 1 -- 172.21.15.86:0/3513226157 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f49e001b020 con 0x7f49f0070330 2024-04-25T14:04:52.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.789+0000 7f49e6000640 1 --2- 172.21.15.86:0/3513226157 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f49d006f400 0x7f49d0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:52.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.789+0000 7f49e6000640 1 -- 172.21.15.86:0/3513226157 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f49e00909d0 con 0x7f49f0070330 2024-04-25T14:04:52.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.789+0000 7f49ed400640 1 --2- 172.21.15.86:0/3513226157 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f49d006f400 0x7f49d0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:52.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.789+0000 7f49ed400640 1 --2- 172.21.15.86:0/3513226157 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f49d006f400 0x7f49d0071910 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f49f00b1ae0 tx=0x7f49dc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:52.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.789+0000 7f49e6000640 1 -- 172.21.15.86:0/3513226157 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f49e0022030 con 0x7f49f0070330 2024-04-25T14:04:52.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.945+0000 7f49f5200640 1 -- 172.21.15.86:0/3513226157 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f49f0076bf0 con 0x7f49f0070330 2024-04-25T14:04:52.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.945+0000 7f49e6000640 1 -- 172.21.15.86:0/3513226157 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f49f015b080 con 0x7f49f0070330 2024-04-25T14:04:52.948 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:04:52.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.949+0000 7f49f5200640 1 -- 172.21.15.86:0/3513226157 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f49d006f400 msgr2=0x7f49d0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:52.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.949+0000 7f49f5200640 1 --2- 172.21.15.86:0/3513226157 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f49d006f400 0x7f49d0071910 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f49f00b1ae0 tx=0x7f49dc009040 comp rx=0 tx=0).stop 2024-04-25T14:04:52.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.949+0000 7f49f5200640 1 -- 172.21.15.86:0/3513226157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49f0070330 msgr2=0x7f49f0159fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:52.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.949+0000 7f49f5200640 1 --2- 172.21.15.86:0/3513226157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49f0070330 0x7f49f0159fc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f49e000efc0 tx=0x7f49e000c490 comp rx=0 tx=0).stop 2024-04-25T14:04:52.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.949+0000 7f49f5200640 1 -- 172.21.15.86:0/3513226157 shutdown_connections 2024-04-25T14:04:52.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.949+0000 7f49f5200640 1 --2- 172.21.15.86:0/3513226157 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f49d006f400 0x7f49d0071910 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.949+0000 7f49f5200640 1 --2- 172.21.15.86:0/3513226157 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49f00d1c20 0x7f49f0073e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.949+0000 7f49f5200640 1 --2- 172.21.15.86:0/3513226157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49f00ce790 0x7f49f00738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.949+0000 7f49f5200640 1 --2- 172.21.15.86:0/3513226157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49f0070330 0x7f49f0159fc0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:52.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.949+0000 7f49f5200640 1 -- 172.21.15.86:0/3513226157 >> 172.21.15.86:0/3513226157 conn(0x7f49f00b4650 msgr2=0x7f49f015d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:52.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.949+0000 7f49f5200640 1 -- 172.21.15.86:0/3513226157 shutdown_connections 2024-04-25T14:04:52.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:52.949+0000 7f49f5200640 1 -- 172.21.15.86:0/3513226157 wait complete. 2024-04-25T14:04:52.970 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:04:53.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2680c00640 1 Processor -- start 2024-04-25T14:04:53.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2680c00640 1 -- start start 2024-04-25T14:04:53.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2680c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f267c0cde90 0x7f267c0ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:53.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2680c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f267c15e5c0 con 0x7f267c0d1df0 2024-04-25T14:04:53.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2680c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f267c15e8a0 con 0x7f267c070330 2024-04-25T14:04:53.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2680c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f267c06fd40 con 0x7f267c0cde90 2024-04-25T14:04:53.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2673e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f267c0d1df0 0x7f267c0cd780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54438/0 (socket says 172.21.15.86:54438) 2024-04-25T14:04:53.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2673e00640 1 -- 172.21.15.86:0/715674994 learned_addr learned my addr 172.21.15.86:0/715674994 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:53.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f267a800640 1 --2- 172.21.15.86:0/715674994 >> v2:172.21.15.155:3300/0 conn(0x7f267c0cde90 0x7f267c0ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:53.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2679400640 1 -- 172.21.15.86:0/715674994 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1014248913 0 0) 0x7f267c15e5c0 con 0x7f267c0d1df0 2024-04-25T14:04:53.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2679400640 1 -- 172.21.15.86:0/715674994 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2660003530 con 0x7f267c0d1df0 2024-04-25T14:04:53.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2679400640 1 -- 172.21.15.86:0/715674994 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3935334214 0 0) 0x7f2660003530 con 0x7f267c0d1df0 2024-04-25T14:04:53.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2679400640 1 -- 172.21.15.86:0/715674994 >> v2:172.21.15.155:3300/0 conn(0x7f267c0cde90 msgr2=0x7f267c0ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:53.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2679400640 1 --2- 172.21.15.86:0/715674994 >> v2:172.21.15.155:3300/0 conn(0x7f267c0cde90 0x7f267c0ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:53.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2679400640 1 -- 172.21.15.86:0/715674994 >> v1:172.21.15.155:6789/0 conn(0x7f267c070330 legacy=0x7f267c0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:53.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2679400640 1 -- 172.21.15.86:0/715674994 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267c15eb80 con 0x7f267c0d1df0 2024-04-25T14:04:53.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2679400640 1 -- 172.21.15.86:0/715674994 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f2664001680 con 0x7f267c0d1df0 2024-04-25T14:04:53.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f267a800640 1 --2- 172.21.15.86:0/715674994 >> v2:172.21.15.155:3300/0 conn(0x7f267c0cde90 0x7f267c0ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:53.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2679400640 1 -- 172.21.15.86:0/715674994 >> v1:172.21.15.86:6789/0 conn(0x7f267c0d1df0 legacy=0x7f267c0cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:53.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2679400640 1 --2- 172.21.15.86:0/715674994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2660003a60 0x7f2660003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:53.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2679400640 1 --2- 172.21.15.86:0/715674994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2660004810 0x7f2660004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:53.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2679400640 1 --2- 172.21.15.86:0/715674994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2660005280 0x7f266000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:53.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2679400640 1 -- 172.21.15.86:0/715674994 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f266000c1e0 con 0x7f2660004810 2024-04-25T14:04:53.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2679400640 1 -- 172.21.15.86:0/715674994 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f266000c390 con 0x7f2660003a60 2024-04-25T14:04:53.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2679400640 1 -- 172.21.15.86:0/715674994 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f266000c590 con 0x7f2660005280 2024-04-25T14:04:53.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2673e00640 1 --2- 172.21.15.86:0/715674994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2660004810 0x7f2660004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:53.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2679e00640 1 --2- 172.21.15.86:0/715674994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2660003a60 0x7f2660003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:53.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f267a800640 1 --2- 172.21.15.86:0/715674994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2660005280 0x7f266000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:53.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2673e00640 1 -- 172.21.15.86:0/715674994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2660005280 msgr2=0x7f266000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:53.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2673e00640 1 --2- 172.21.15.86:0/715674994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2660005280 0x7f266000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:53.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2673e00640 1 -- 172.21.15.86:0/715674994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2660003a60 msgr2=0x7f2660003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:53.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2673e00640 1 --2- 172.21.15.86:0/715674994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2660003a60 0x7f2660003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:53.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.117+0000 7f2673e00640 1 -- 172.21.15.86:0/715674994 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2660003530 con 0x7f2660004810 2024-04-25T14:04:53.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.121+0000 7f267a800640 1 -- 172.21.15.86:0/715674994 reap_dead start 2024-04-25T14:04:53.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.121+0000 7f2673e00640 1 --2- 172.21.15.86:0/715674994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2660004810 0x7f2660004c20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f267c074ed0 tx=0x7f266401cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9345404501ea458d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:53.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.121+0000 7f2679400640 1 -- 172.21.15.86:0/715674994 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2664007d40 con 0x7f2660004810 2024-04-25T14:04:53.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.121+0000 7f2679400640 1 -- 172.21.15.86:0/715674994 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26640a9070 con 0x7f2660004810 2024-04-25T14:04:53.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.121+0000 7f2679400640 1 -- 172.21.15.86:0/715674994 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2664005d80 con 0x7f2660004810 2024-04-25T14:04:53.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.121+0000 7f2680c00640 1 -- 172.21.15.86:0/715674994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2660004810 msgr2=0x7f2660004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:53.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.121+0000 7f2680c00640 1 --2- 172.21.15.86:0/715674994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2660004810 0x7f2660004c20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f267c074ed0 tx=0x7f266401cb70 comp rx=0 tx=0).stop 2024-04-25T14:04:53.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.121+0000 7f2680c00640 1 -- 172.21.15.86:0/715674994 shutdown_connections 2024-04-25T14:04:53.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.121+0000 7f2680c00640 1 --2- 172.21.15.86:0/715674994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2660004810 0x7f2660004c20 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:53.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.121+0000 7f2680c00640 1 -- 172.21.15.86:0/715674994 >> 172.21.15.86:0/715674994 conn(0x7f267c0b4650 msgr2=0x7f267c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:53.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.121+0000 7f2680c00640 1 -- 172.21.15.86:0/715674994 shutdown_connections 2024-04-25T14:04:53.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.121+0000 7f2680c00640 1 -- 172.21.15.86:0/715674994 wait complete. 2024-04-25T14:04:53.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.125+0000 7f2680c00640 1 Processor -- start 2024-04-25T14:04:53.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.125+0000 7f2680c00640 1 -- start start 2024-04-25T14:04:53.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.125+0000 7f2680c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2660004810 0x7f267c159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:53.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.125+0000 7f2680c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f267c09e150 0x7f267c09b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:53.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.125+0000 7f2680c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f267c1f4750 0x7f267c1f4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:53.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.125+0000 7f2680c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f267c0cdd10 con 0x7f267c09e150 2024-04-25T14:04:53.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.125+0000 7f2680c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f267c0cdb90 con 0x7f2660004810 2024-04-25T14:04:53.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.125+0000 7f2680c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f267c15f330 con 0x7f267c1f4750 2024-04-25T14:04:53.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.125+0000 7f2673e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f267c09e150 0x7f267c09b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:53.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.129+0000 7f2673e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f267c09e150 0x7f267c09b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52270/0 (socket says 172.21.15.86:52270) 2024-04-25T14:04:53.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.129+0000 7f2673e00640 1 -- 172.21.15.86:0/1824983065 learned_addr learned my addr 172.21.15.86:0/1824983065 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:53.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.129+0000 7f2679e00640 1 --2- 172.21.15.86:0/1824983065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2660004810 0x7f267c159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:53.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.129+0000 7f267a800640 1 --2- 172.21.15.86:0/1824983065 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f267c1f4750 0x7f267c1f4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:53.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.129+0000 7f2673e00640 1 -- 172.21.15.86:0/1824983065 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f267c1f4750 msgr2=0x7f267c1f4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:53.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.129+0000 7f2673e00640 1 --2- 172.21.15.86:0/1824983065 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f267c1f4750 0x7f267c1f4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:53.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.129+0000 7f2673e00640 1 -- 172.21.15.86:0/1824983065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2660004810 msgr2=0x7f267c159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:53.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.129+0000 7f2673e00640 1 --2- 172.21.15.86:0/1824983065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2660004810 0x7f267c159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:53.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.129+0000 7f2673e00640 1 -- 172.21.15.86:0/1824983065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267c1e5480 con 0x7f267c09e150 2024-04-25T14:04:53.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.129+0000 7f2673e00640 1 --2- 172.21.15.86:0/1824983065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f267c09e150 0x7f267c09b2b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f266401c5e0 tx=0x7f2660005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:53.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.129+0000 7f2672000640 1 -- 172.21.15.86:0/1824983065 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2664007d40 con 0x7f267c09e150 2024-04-25T14:04:53.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.129+0000 7f2672000640 1 -- 172.21.15.86:0/1824983065 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26640a6e40 con 0x7f267c09e150 2024-04-25T14:04:53.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.129+0000 7f2672000640 1 -- 172.21.15.86:0/1824983065 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26640047f0 con 0x7f267c09e150 2024-04-25T14:04:53.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.129+0000 7f2680c00640 1 -- 172.21.15.86:0/1824983065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f267c09d670 con 0x7f267c09e150 2024-04-25T14:04:53.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.129+0000 7f2680c00640 1 -- 172.21.15.86:0/1824983065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f267c09dc30 con 0x7f267c09e150 2024-04-25T14:04:53.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.129+0000 7f2680c00640 1 -- 172.21.15.86:0/1824983065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f267c09d460 con 0x7f267c09e150 2024-04-25T14:04:53.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.129+0000 7f2672000640 1 -- 172.21.15.86:0/1824983065 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2664004990 con 0x7f267c09e150 2024-04-25T14:04:53.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.133+0000 7f2672000640 1 --2- 172.21.15.86:0/1824983065 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f265806f400 0x7f2658071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:53.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.133+0000 7f2672000640 1 -- 172.21.15.86:0/1824983065 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f2664128d70 con 0x7f267c09e150 2024-04-25T14:04:53.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.133+0000 7f2679e00640 1 --2- 172.21.15.86:0/1824983065 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f265806f400 0x7f2658071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:53.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.133+0000 7f2679e00640 1 --2- 172.21.15.86:0/1824983065 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f265806f400 0x7f2658071910 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f267c06b910 tx=0x7f266c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:53.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.137+0000 7f2672000640 1 -- 172.21.15.86:0/1824983065 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f26640168b0 con 0x7f267c09e150 2024-04-25T14:04:53.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.325+0000 7f2680c00640 1 -- 172.21.15.86:0/1824983065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f267c1e51d0 con 0x7f267c09e150 2024-04-25T14:04:53.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.325+0000 7f2672000640 1 -- 172.21.15.86:0/1824983065 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5177 (secure 0 0 0) 0x7f26640f5210 con 0x7f267c09e150 2024-04-25T14:04:53.326 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:53.327 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/347107112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":347107112},{"type":"v1","addr":"172.21.15.155:6835","nonce":347107112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2236737549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2236737549},{"type":"v1","addr":"172.21.15.86:6837","nonce":2236737549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/519560229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":519560229},{"type":"v1","addr":"172.21.15.86:6836","nonce":519560229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:03:50.852257+0000","modified":"2024-04-25T14:04:09.202137+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6833/1944249570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1944249570},{"type":"v1","addr":"172.21.15.155:6833","nonce":1944249570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":1}]} 2024-04-25T14:04:53.327 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 9 2024-04-25T14:04:53.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.325+0000 7f2680c00640 1 -- 172.21.15.86:0/1824983065 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f265806f400 msgr2=0x7f2658071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:53.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.325+0000 7f2680c00640 1 --2- 172.21.15.86:0/1824983065 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f265806f400 0x7f2658071910 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f267c06b910 tx=0x7f266c009040 comp rx=0 tx=0).stop 2024-04-25T14:04:53.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.325+0000 7f2680c00640 1 -- 172.21.15.86:0/1824983065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f267c09e150 msgr2=0x7f267c09b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:53.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.325+0000 7f2680c00640 1 --2- 172.21.15.86:0/1824983065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f267c09e150 0x7f267c09b2b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f266401c5e0 tx=0x7f2660005840 comp rx=0 tx=0).stop 2024-04-25T14:04:53.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.325+0000 7f2680c00640 1 -- 172.21.15.86:0/1824983065 shutdown_connections 2024-04-25T14:04:53.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.325+0000 7f2680c00640 1 --2- 172.21.15.86:0/1824983065 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f265806f400 0x7f2658071910 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:53.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.325+0000 7f2680c00640 1 --2- 172.21.15.86:0/1824983065 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f267c1f4750 0x7f267c1f4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:53.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.325+0000 7f2680c00640 1 --2- 172.21.15.86:0/1824983065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f267c09e150 0x7f267c09b2b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:53.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.325+0000 7f2680c00640 1 --2- 172.21.15.86:0/1824983065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2660004810 0x7f267c159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:53.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.325+0000 7f2680c00640 1 -- 172.21.15.86:0/1824983065 >> 172.21.15.86:0/1824983065 conn(0x7f267c0b4650 msgr2=0x7f267c0af3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:53.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.325+0000 7f2680c00640 1 -- 172.21.15.86:0/1824983065 shutdown_connections 2024-04-25T14:04:53.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.325+0000 7f2680c00640 1 -- 172.21.15.86:0/1824983065 wait complete. 2024-04-25T14:04:53.348 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:04:53.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.493+0000 7efec8000640 1 Processor -- start 2024-04-25T14:04:53.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.493+0000 7efec8000640 1 -- start start 2024-04-25T14:04:53.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.493+0000 7efec8000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efec00cdda0 0x7efec00cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:53.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.493+0000 7efec8000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efec0163940 con 0x7efec00cec60 2024-04-25T14:04:53.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.493+0000 7efec8000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efec00cfcc0 con 0x7efec00d18e0 2024-04-25T14:04:53.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.493+0000 7efec8000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7efec00d2f40 con 0x7efec00cdda0 2024-04-25T14:04:53.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.497+0000 7efec4c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efec00cdda0 0x7efec00cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:53.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.497+0000 7efec5600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7efec00cec60 0x7efec00cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54450/0 (socket says 172.21.15.86:54450) 2024-04-25T14:04:53.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.497+0000 7efec5600640 1 -- 172.21.15.86:0/2263738750 learned_addr learned my addr 172.21.15.86:0/2263738750 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:53.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.497+0000 7efec4c00640 1 -- 172.21.15.86:0/2263738750 >> v1:172.21.15.155:6789/0 conn(0x7efec00d18e0 legacy=0x7efec00d1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:53.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.497+0000 7efec4c00640 1 -- 172.21.15.86:0/2263738750 >> v1:172.21.15.86:6789/0 conn(0x7efec00cec60 legacy=0x7efec00cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:53.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.497+0000 7efec4c00640 1 -- 172.21.15.86:0/2263738750 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec00b3720 con 0x7efec00cdda0 2024-04-25T14:04:53.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.497+0000 7efebfe00640 1 -- 172.21.15.86:0/2263738750 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1831409293 0 0) 0x7efec0163940 con 0x7efec00cec60 2024-04-25T14:04:53.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.497+0000 7efec4c00640 1 --2- 172.21.15.86:0/2263738750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efec00cdda0 0x7efec00cf780 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7efeac0097c0 tx=0x7efeac01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5221faed21a13b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:53.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.497+0000 7efebfe00640 1 -- 172.21.15.86:0/2263738750 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac0b0070 con 0x7efec00cdda0 2024-04-25T14:04:53.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.497+0000 7efebfe00640 1 -- 172.21.15.86:0/2263738750 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeac0ab600 con 0x7efec00cdda0 2024-04-25T14:04:53.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.497+0000 7efebfe00640 1 -- 172.21.15.86:0/2263738750 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac0ab9d0 con 0x7efec00cdda0 2024-04-25T14:04:53.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.501+0000 7efec8000640 1 -- 172.21.15.86:0/2263738750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efec00cdda0 msgr2=0x7efec00cf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:53.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.501+0000 7efec8000640 1 --2- 172.21.15.86:0/2263738750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efec00cdda0 0x7efec00cf780 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7efeac0097c0 tx=0x7efeac01c4a0 comp rx=0 tx=0).stop 2024-04-25T14:04:53.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.501+0000 7efec8000640 1 -- 172.21.15.86:0/2263738750 shutdown_connections 2024-04-25T14:04:53.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.501+0000 7efec8000640 1 --2- 172.21.15.86:0/2263738750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efec00cdda0 0x7efec00cf780 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:53.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.501+0000 7efec8000640 1 -- 172.21.15.86:0/2263738750 >> 172.21.15.86:0/2263738750 conn(0x7efec009ba30 msgr2=0x7efec00b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:53.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.501+0000 7efec8000640 1 -- 172.21.15.86:0/2263738750 shutdown_connections 2024-04-25T14:04:53.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.501+0000 7efec8000640 1 -- 172.21.15.86:0/2263738750 wait complete. 2024-04-25T14:04:53.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.501+0000 7efec8000640 1 Processor -- start 2024-04-25T14:04:53.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.501+0000 7efec8000640 1 -- start start 2024-04-25T14:04:53.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.505+0000 7efec8000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efec00cdda0 0x7efec0155890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:53.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.505+0000 7efec8000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efec00cec60 0x7efec0159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:53.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.505+0000 7efec8000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efec00d18e0 0x7efec0157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:53.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.505+0000 7efec8000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efec016df90 con 0x7efec00cec60 2024-04-25T14:04:53.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.505+0000 7efec8000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efec016de10 con 0x7efec00cdda0 2024-04-25T14:04:53.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.505+0000 7efec8000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efec016e110 con 0x7efec00d18e0 2024-04-25T14:04:53.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.505+0000 7efec4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efec00cec60 0x7efec0159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:53.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.505+0000 7efec4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efec00cec60 0x7efec0159e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52288/0 (socket says 172.21.15.86:52288) 2024-04-25T14:04:53.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.505+0000 7efec4c00640 1 -- 172.21.15.86:0/579594067 learned_addr learned my addr 172.21.15.86:0/579594067 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:53.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.505+0000 7efec5600640 1 --2- 172.21.15.86:0/579594067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efec00cdda0 0x7efec0155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:53.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.505+0000 7efec6000640 1 --2- 172.21.15.86:0/579594067 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efec00d18e0 0x7efec0157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:53.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.505+0000 7efec4c00640 1 -- 172.21.15.86:0/579594067 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efec00d18e0 msgr2=0x7efec0157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:53.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.505+0000 7efec4c00640 1 --2- 172.21.15.86:0/579594067 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efec00d18e0 0x7efec0157fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:53.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.505+0000 7efec4c00640 1 -- 172.21.15.86:0/579594067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efec00cdda0 msgr2=0x7efec0155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:53.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.505+0000 7efec4c00640 1 --2- 172.21.15.86:0/579594067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efec00cdda0 0x7efec0155890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:53.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.505+0000 7efec4c00640 1 -- 172.21.15.86:0/579594067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec0155dd0 con 0x7efec00cec60 2024-04-25T14:04:53.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.505+0000 7efec6000640 1 --2- 172.21.15.86:0/579594067 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efec00d18e0 0x7efec0157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:53.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.505+0000 7efec4c00640 1 --2- 172.21.15.86:0/579594067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efec00cec60 0x7efec0159e50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7efeac0190b0 tx=0x7efeac019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:53.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.505+0000 7efebd600640 1 -- 172.21.15.86:0/579594067 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac0b0040 con 0x7efec00cec60 2024-04-25T14:04:53.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.505+0000 7efebd600640 1 -- 172.21.15.86:0/579594067 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeac0ba070 con 0x7efec00cec60 2024-04-25T14:04:53.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.505+0000 7efec8000640 1 -- 172.21.15.86:0/579594067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec015a390 con 0x7efec00cec60 2024-04-25T14:04:53.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.505+0000 7efebd600640 1 -- 172.21.15.86:0/579594067 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac0b4830 con 0x7efec00cec60 2024-04-25T14:04:53.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.505+0000 7efec8000640 1 -- 172.21.15.86:0/579594067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec0156440 con 0x7efec00cec60 2024-04-25T14:04:53.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.505+0000 7efec8000640 1 -- 172.21.15.86:0/579594067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efec00a04c0 con 0x7efec00cec60 2024-04-25T14:04:53.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.505+0000 7efebd600640 1 -- 172.21.15.86:0/579594067 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efeac0ab4c0 con 0x7efec00cec60 2024-04-25T14:04:53.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.509+0000 7efebd600640 1 --2- 172.21.15.86:0/579594067 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7efea006f430 0x7efea0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:53.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.509+0000 7efebd600640 1 -- 172.21.15.86:0/579594067 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7efeac1292b0 con 0x7efec00cec60 2024-04-25T14:04:53.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.509+0000 7efec5600640 1 --2- 172.21.15.86:0/579594067 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7efea006f430 0x7efea0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:53.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.509+0000 7efec5600640 1 --2- 172.21.15.86:0/579594067 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7efea006f430 0x7efea0071940 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efeb4001470 tx=0x7efeb4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:53.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.513+0000 7efebd600640 1 -- 172.21.15.86:0/579594067 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efeac016090 con 0x7efec00cec60 2024-04-25T14:04:53.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.669+0000 7efec8000640 1 -- 172.21.15.86:0/579594067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efec0076bf0 con 0x7efec00cec60 2024-04-25T14:04:53.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.669+0000 7efebd600640 1 -- 172.21.15.86:0/579594067 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18902 (secure 0 0 0) 0x7efeac0f5e30 con 0x7efec00cec60 2024-04-25T14:04:53.672 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:53.672 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":26,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:04:31.930803+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:03:49.151473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:03:49.802597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T14:03:53.492008+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:04:53.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.673+0000 7efec8000640 1 -- 172.21.15.86:0/579594067 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7efea006f430 msgr2=0x7efea0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:53.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.673+0000 7efec8000640 1 --2- 172.21.15.86:0/579594067 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7efea006f430 0x7efea0071940 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efeb4001470 tx=0x7efeb4009040 comp rx=0 tx=0).stop 2024-04-25T14:04:53.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.673+0000 7efec8000640 1 -- 172.21.15.86:0/579594067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efec00cec60 msgr2=0x7efec0159e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:53.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.673+0000 7efec8000640 1 --2- 172.21.15.86:0/579594067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efec00cec60 0x7efec0159e50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7efeac0190b0 tx=0x7efeac019240 comp rx=0 tx=0).stop 2024-04-25T14:04:53.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.673+0000 7efec8000640 1 -- 172.21.15.86:0/579594067 shutdown_connections 2024-04-25T14:04:53.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.673+0000 7efec8000640 1 --2- 172.21.15.86:0/579594067 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7efea006f430 0x7efea0071940 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:53.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.673+0000 7efec8000640 1 --2- 172.21.15.86:0/579594067 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efec00d18e0 0x7efec0157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:53.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.673+0000 7efec8000640 1 --2- 172.21.15.86:0/579594067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efec00cec60 0x7efec0159e50 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:53.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.673+0000 7efec8000640 1 --2- 172.21.15.86:0/579594067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efec00cdda0 0x7efec0155890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:53.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.673+0000 7efec8000640 1 -- 172.21.15.86:0/579594067 >> 172.21.15.86:0/579594067 conn(0x7efec009ba30 msgr2=0x7efec00711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:53.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.673+0000 7efec8000640 1 -- 172.21.15.86:0/579594067 shutdown_connections 2024-04-25T14:04:53.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.673+0000 7efec8000640 1 -- 172.21.15.86:0/579594067 wait complete. 2024-04-25T14:04:53.693 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T14:04:53.694 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:04:53.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.841+0000 7fd7e7200640 1 Processor -- start 2024-04-25T14:04:53.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.841+0000 7fd7e7200640 1 -- start start 2024-04-25T14:04:53.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.841+0000 7fd7e7200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd7e80cc8c0 0x7fd7e80cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:53.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.841+0000 7fd7e7200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7e80d14c0 con 0x7fd7e80ce7a0 2024-04-25T14:04:53.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.841+0000 7fd7e7200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7e80ce220 con 0x7fd7e80cd210 2024-04-25T14:04:53.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.841+0000 7fd7e7200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd7e80d1760 con 0x7fd7e80cc8c0 2024-04-25T14:04:53.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.841+0000 7fd7e5e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd7e80ce7a0 0x7fd7e80cebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54458/0 (socket says 172.21.15.86:54458) 2024-04-25T14:04:53.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.841+0000 7fd7e5e00640 1 -- 172.21.15.86:0/3498684332 learned_addr learned my addr 172.21.15.86:0/3498684332 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:53.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.841+0000 7fd7e5400640 1 --2- 172.21.15.86:0/3498684332 >> v2:172.21.15.86:3300/0 conn(0x7fd7e80cc8c0 0x7fd7e80cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:53.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.841+0000 7fd7e4a00640 1 -- 172.21.15.86:0/3498684332 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2934177603 0 0) 0x7fd7e80d14c0 con 0x7fd7e80ce7a0 2024-04-25T14:04:53.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.841+0000 7fd7e4a00640 1 -- 172.21.15.86:0/3498684332 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7cc003530 con 0x7fd7e80ce7a0 2024-04-25T14:04:53.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.845+0000 7fd7e5400640 1 -- 172.21.15.86:0/3498684332 >> v1:172.21.15.155:6789/0 conn(0x7fd7e80cd210 legacy=0x7fd7e80cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:53.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.845+0000 7fd7e5400640 1 -- 172.21.15.86:0/3498684332 >> v1:172.21.15.86:6789/0 conn(0x7fd7e80ce7a0 legacy=0x7fd7e80cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:53.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.845+0000 7fd7e5400640 1 -- 172.21.15.86:0/3498684332 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7e80d1a40 con 0x7fd7e80cc8c0 2024-04-25T14:04:53.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.845+0000 7fd7e5400640 1 --2- 172.21.15.86:0/3498684332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7e80cc8c0 0x7fd7e80cccd0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd7d40097c0 tx=0x7fd7d401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8436937a25790ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:53.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.845+0000 7fd7e4a00640 1 -- 172.21.15.86:0/3498684332 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d40aa620 con 0x7fd7e80cc8c0 2024-04-25T14:04:53.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.845+0000 7fd7e4a00640 1 -- 172.21.15.86:0/3498684332 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7d40af070 con 0x7fd7e80cc8c0 2024-04-25T14:04:53.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.845+0000 7fd7e4a00640 1 -- 172.21.15.86:0/3498684332 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d40aa9e0 con 0x7fd7e80cc8c0 2024-04-25T14:04:53.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.845+0000 7fd7e7200640 1 -- 172.21.15.86:0/3498684332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7e80cc8c0 msgr2=0x7fd7e80cccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.845+0000 7fd7e7200640 1 --2- 172.21.15.86:0/3498684332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7e80cc8c0 0x7fd7e80cccd0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd7d40097c0 tx=0x7fd7d401c4a0 comp rx=0 tx=0).stop 2024-04-25T14:04:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.849+0000 7fd7e7200640 1 -- 172.21.15.86:0/3498684332 shutdown_connections 2024-04-25T14:04:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.849+0000 7fd7e7200640 1 --2- 172.21.15.86:0/3498684332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7e80cc8c0 0x7fd7e80cccd0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.849+0000 7fd7e7200640 1 -- 172.21.15.86:0/3498684332 >> 172.21.15.86:0/3498684332 conn(0x7fd7e809ba30 msgr2=0x7fd7e80b70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.849+0000 7fd7e7200640 1 -- 172.21.15.86:0/3498684332 shutdown_connections 2024-04-25T14:04:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.849+0000 7fd7e7200640 1 -- 172.21.15.86:0/3498684332 wait complete. 2024-04-25T14:04:53.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.849+0000 7fd7e7200640 1 Processor -- start 2024-04-25T14:04:53.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.849+0000 7fd7e7200640 1 -- start start 2024-04-25T14:04:53.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.849+0000 7fd7e7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7e80cc8c0 0x7fd7e8158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:53.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.849+0000 7fd7e7200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7e80cd210 0x7fd7e80b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:53.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.849+0000 7fd7e7200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e80ce7a0 0x7fd7e80b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:53.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.849+0000 7fd7e7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e816e4e0 con 0x7fd7e80cc8c0 2024-04-25T14:04:53.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.849+0000 7fd7e7200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e80b3c10 con 0x7fd7e80cd210 2024-04-25T14:04:53.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.849+0000 7fd7e7200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd7e816e660 con 0x7fd7e80ce7a0 2024-04-25T14:04:53.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.849+0000 7fd7e5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7e80cc8c0 0x7fd7e8158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:53.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.853+0000 7fd7e5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7e80cc8c0 0x7fd7e8158a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52308/0 (socket says 172.21.15.86:52308) 2024-04-25T14:04:53.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.853+0000 7fd7e5e00640 1 -- 172.21.15.86:0/1921422390 learned_addr learned my addr 172.21.15.86:0/1921422390 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:53.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.853+0000 7fd7e5400640 1 --2- 172.21.15.86:0/1921422390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7e80cd210 0x7fd7e80b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:53.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.853+0000 7fd7e6800640 1 --2- 172.21.15.86:0/1921422390 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e80ce7a0 0x7fd7e80b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:53.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.853+0000 7fd7e5e00640 1 -- 172.21.15.86:0/1921422390 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e80ce7a0 msgr2=0x7fd7e80b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:53.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.853+0000 7fd7e5e00640 1 --2- 172.21.15.86:0/1921422390 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e80ce7a0 0x7fd7e80b0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:53.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.853+0000 7fd7e5e00640 1 -- 172.21.15.86:0/1921422390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7e80cd210 msgr2=0x7fd7e80b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:53.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.853+0000 7fd7e5e00640 1 --2- 172.21.15.86:0/1921422390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7e80cd210 0x7fd7e80b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:53.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.853+0000 7fd7e5e00640 1 -- 172.21.15.86:0/1921422390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7e815ed60 con 0x7fd7e80cc8c0 2024-04-25T14:04:53.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.853+0000 7fd7e5e00640 1 --2- 172.21.15.86:0/1921422390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7e80cc8c0 0x7fd7e8158a60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd7d800cae0 tx=0x7fd7d800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:53.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.853+0000 7fd7de000640 1 -- 172.21.15.86:0/1921422390 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8017070 con 0x7fd7e80cc8c0 2024-04-25T14:04:53.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.853+0000 7fd7de000640 1 -- 172.21.15.86:0/1921422390 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7d800ac40 con 0x7fd7e80cc8c0 2024-04-25T14:04:53.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.853+0000 7fd7e7200640 1 -- 172.21.15.86:0/1921422390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7e815c1f0 con 0x7fd7e80cc8c0 2024-04-25T14:04:53.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.853+0000 7fd7de000640 1 -- 172.21.15.86:0/1921422390 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8012620 con 0x7fd7e80cc8c0 2024-04-25T14:04:53.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.853+0000 7fd7e7200640 1 -- 172.21.15.86:0/1921422390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7e815c7b0 con 0x7fd7e80cc8c0 2024-04-25T14:04:53.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.853+0000 7fd7e7200640 1 -- 172.21.15.86:0/1921422390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7e8076b80 con 0x7fd7e80cc8c0 2024-04-25T14:04:53.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.853+0000 7fd7de000640 1 -- 172.21.15.86:0/1921422390 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd7d801a070 con 0x7fd7e80cc8c0 2024-04-25T14:04:53.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.857+0000 7fd7de000640 1 --2- 172.21.15.86:0/1921422390 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd7c806f400 0x7fd7c8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:53.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.857+0000 7fd7de000640 1 -- 172.21.15.86:0/1921422390 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd7d808f330 con 0x7fd7e80cc8c0 2024-04-25T14:04:53.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.857+0000 7fd7e5400640 1 --2- 172.21.15.86:0/1921422390 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd7c806f400 0x7fd7c8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:53.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.857+0000 7fd7e5400640 1 --2- 172.21.15.86:0/1921422390 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd7c806f400 0x7fd7c8071910 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd7d40ae000 tx=0x7fd7d40a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:53.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:53.861+0000 7fd7de000640 1 -- 172.21.15.86:0/1921422390 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd7d805b860 con 0x7fd7e80cc8c0 2024-04-25T14:04:54.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.053+0000 7fd7e7200640 1 -- 172.21.15.86:0/1921422390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7e80b47f0 con 0x7fd7e80cc8c0 2024-04-25T14:04:54.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.053+0000 7fd7de000640 1 -- 172.21.15.86:0/1921422390 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5177 (secure 0 0 0) 0x7fd7d805fb60 con 0x7fd7e80cc8c0 2024-04-25T14:04:54.055 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:54.055 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/347107112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":347107112},{"type":"v1","addr":"172.21.15.155:6835","nonce":347107112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2236737549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2236737549},{"type":"v1","addr":"172.21.15.86:6837","nonce":2236737549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/519560229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":519560229},{"type":"v1","addr":"172.21.15.86:6836","nonce":519560229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:03:50.852257+0000","modified":"2024-04-25T14:04:09.202137+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6833/1944249570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1944249570},{"type":"v1","addr":"172.21.15.155:6833","nonce":1944249570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":1}]} 2024-04-25T14:04:54.055 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 9 2024-04-25T14:04:54.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.053+0000 7fd7e7200640 1 -- 172.21.15.86:0/1921422390 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd7c806f400 msgr2=0x7fd7c8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:54.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.053+0000 7fd7e7200640 1 --2- 172.21.15.86:0/1921422390 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd7c806f400 0x7fd7c8071910 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd7d40ae000 tx=0x7fd7d40a7000 comp rx=0 tx=0).stop 2024-04-25T14:04:54.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.053+0000 7fd7e7200640 1 -- 172.21.15.86:0/1921422390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7e80cc8c0 msgr2=0x7fd7e8158a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:54.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.053+0000 7fd7e7200640 1 --2- 172.21.15.86:0/1921422390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7e80cc8c0 0x7fd7e8158a60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd7d800cae0 tx=0x7fd7d800a060 comp rx=0 tx=0).stop 2024-04-25T14:04:54.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.053+0000 7fd7e7200640 1 -- 172.21.15.86:0/1921422390 shutdown_connections 2024-04-25T14:04:54.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.053+0000 7fd7e7200640 1 --2- 172.21.15.86:0/1921422390 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd7c806f400 0x7fd7c8071910 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:54.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.053+0000 7fd7e7200640 1 --2- 172.21.15.86:0/1921422390 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e80ce7a0 0x7fd7e80b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:54.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.053+0000 7fd7e7200640 1 --2- 172.21.15.86:0/1921422390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7e80cd210 0x7fd7e80b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:54.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.053+0000 7fd7e7200640 1 --2- 172.21.15.86:0/1921422390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7e80cc8c0 0x7fd7e8158a60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:54.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.053+0000 7fd7e7200640 1 -- 172.21.15.86:0/1921422390 >> 172.21.15.86:0/1921422390 conn(0x7fd7e809ba30 msgr2=0x7fd7e80b18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:54.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.057+0000 7fd7e7200640 1 -- 172.21.15.86:0/1921422390 shutdown_connections 2024-04-25T14:04:54.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.057+0000 7fd7e7200640 1 -- 172.21.15.86:0/1921422390 wait complete. 2024-04-25T14:04:54.077 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:04:54.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.221+0000 7fe42da00640 1 Processor -- start 2024-04-25T14:04:54.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.221+0000 7fe42da00640 1 -- start start 2024-04-25T14:04:54.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.225+0000 7fe42da00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe4280cde90 0x7fe4280ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:54.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.225+0000 7fe42da00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe42815e5c0 con 0x7fe4280d1df0 2024-04-25T14:04:54.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.225+0000 7fe42da00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe42815e8a0 con 0x7fe428070330 2024-04-25T14:04:54.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.225+0000 7fe42da00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe42806fd40 con 0x7fe4280cde90 2024-04-25T14:04:54.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.225+0000 7fe427400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe4280cde90 0x7fe4280ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:54.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.225+0000 7fe427400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe4280cde90 0x7fe4280ce2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52320/0 (socket says 172.21.15.86:52320) 2024-04-25T14:04:54.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.225+0000 7fe427400640 1 -- 172.21.15.86:0/1786590223 learned_addr learned my addr 172.21.15.86:0/1786590223 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:54.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.225+0000 7fe427400640 1 -- 172.21.15.86:0/1786590223 >> v1:172.21.15.155:6789/0 conn(0x7fe428070330 legacy=0x7fe4280d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:54.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.225+0000 7fe427400640 1 -- 172.21.15.86:0/1786590223 >> v1:172.21.15.86:6789/0 conn(0x7fe4280d1df0 legacy=0x7fe4280cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:54.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.225+0000 7fe427400640 1 -- 172.21.15.86:0/1786590223 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe42815eb80 con 0x7fe4280cde90 2024-04-25T14:04:54.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.225+0000 7fe425600640 1 -- 172.21.15.86:0/1786590223 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 89803304 0 0) 0x7fe42815e5c0 con 0x7fe4280d1df0 2024-04-25T14:04:54.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.229+0000 7fe427400640 1 --2- 172.21.15.86:0/1786590223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4280cde90 0x7fe4280ce2a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe4200098a0 tx=0x7fe42001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda1a775a96eebe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:54.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.229+0000 7fe425600640 1 -- 172.21.15.86:0/1786590223 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4200b1070 con 0x7fe4280cde90 2024-04-25T14:04:54.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.229+0000 7fe425600640 1 -- 172.21.15.86:0/1786590223 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4200ac5f0 con 0x7fe4280cde90 2024-04-25T14:04:54.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.229+0000 7fe425600640 1 -- 172.21.15.86:0/1786590223 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4200ac9c0 con 0x7fe4280cde90 2024-04-25T14:04:54.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.229+0000 7fe42da00640 1 -- 172.21.15.86:0/1786590223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4280cde90 msgr2=0x7fe4280ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:54.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.229+0000 7fe42da00640 1 --2- 172.21.15.86:0/1786590223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4280cde90 0x7fe4280ce2a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe4200098a0 tx=0x7fe42001c480 comp rx=0 tx=0).stop 2024-04-25T14:04:54.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.229+0000 7fe42da00640 1 -- 172.21.15.86:0/1786590223 shutdown_connections 2024-04-25T14:04:54.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.229+0000 7fe42da00640 1 --2- 172.21.15.86:0/1786590223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4280cde90 0x7fe4280ce2a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:54.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.229+0000 7fe42da00640 1 -- 172.21.15.86:0/1786590223 >> 172.21.15.86:0/1786590223 conn(0x7fe4280b4650 msgr2=0x7fe4280b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:54.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.229+0000 7fe42da00640 1 -- 172.21.15.86:0/1786590223 shutdown_connections 2024-04-25T14:04:54.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.229+0000 7fe42da00640 1 -- 172.21.15.86:0/1786590223 wait complete. 2024-04-25T14:04:54.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.229+0000 7fe42da00640 1 Processor -- start 2024-04-25T14:04:54.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.229+0000 7fe42da00640 1 -- start start 2024-04-25T14:04:54.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.233+0000 7fe42da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe428070330 0x7fe42809a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:54.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.233+0000 7fe42da00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4280cde90 0x7fe428157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:54.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.233+0000 7fe42da00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe42815ad70 0x7fe42815b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:54.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.233+0000 7fe42da00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4280708c0 con 0x7fe428070330 2024-04-25T14:04:54.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.233+0000 7fe42da00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe428070740 con 0x7fe42815ad70 2024-04-25T14:04:54.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.233+0000 7fe42da00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe42816bfd0 con 0x7fe4280cde90 2024-04-25T14:04:54.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.233+0000 7fe426a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe428070330 0x7fe42809a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:54.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.233+0000 7fe426a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe428070330 0x7fe42809a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52336/0 (socket says 172.21.15.86:52336) 2024-04-25T14:04:54.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.233+0000 7fe426a00640 1 -- 172.21.15.86:0/3321683331 learned_addr learned my addr 172.21.15.86:0/3321683331 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:54.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.233+0000 7fe427400640 1 --2- 172.21.15.86:0/3321683331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe42815ad70 0x7fe42815b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:54.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.233+0000 7fe426000640 1 --2- 172.21.15.86:0/3321683331 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4280cde90 0x7fe428157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:54.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.233+0000 7fe426a00640 1 -- 172.21.15.86:0/3321683331 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4280cde90 msgr2=0x7fe428157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:54.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.233+0000 7fe426a00640 1 --2- 172.21.15.86:0/3321683331 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4280cde90 0x7fe428157100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:54.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.233+0000 7fe426a00640 1 -- 172.21.15.86:0/3321683331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe42815ad70 msgr2=0x7fe42815b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:54.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.233+0000 7fe426a00640 1 --2- 172.21.15.86:0/3321683331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe42815ad70 0x7fe42815b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:54.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.233+0000 7fe426a00640 1 -- 172.21.15.86:0/3321683331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe42815b6c0 con 0x7fe428070330 2024-04-25T14:04:54.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.233+0000 7fe426000640 1 --2- 172.21.15.86:0/3321683331 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4280cde90 0x7fe428157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:54.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.233+0000 7fe426a00640 1 --2- 172.21.15.86:0/3321683331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe428070330 0x7fe42809a9e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe41800c790 tx=0x7fe41800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:54.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.233+0000 7fe41ea00640 1 -- 172.21.15.86:0/3321683331 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe418003b80 con 0x7fe428070330 2024-04-25T14:04:54.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.233+0000 7fe41ea00640 1 -- 172.21.15.86:0/3321683331 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe418003d20 con 0x7fe428070330 2024-04-25T14:04:54.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.233+0000 7fe41ea00640 1 -- 172.21.15.86:0/3321683331 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41800aa30 con 0x7fe428070330 2024-04-25T14:04:54.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.233+0000 7fe42da00640 1 -- 172.21.15.86:0/3321683331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe428157ae0 con 0x7fe428070330 2024-04-25T14:04:54.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.233+0000 7fe42da00640 1 -- 172.21.15.86:0/3321683331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4281580a0 con 0x7fe428070330 2024-04-25T14:04:54.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.233+0000 7fe42da00640 1 -- 172.21.15.86:0/3321683331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe428069d00 con 0x7fe428070330 2024-04-25T14:04:54.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.237+0000 7fe41ea00640 1 -- 172.21.15.86:0/3321683331 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe41800abd0 con 0x7fe428070330 2024-04-25T14:04:54.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.237+0000 7fe41ea00640 1 --2- 172.21.15.86:0/3321683331 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe40806f430 0x7fe408071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:54.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.237+0000 7fe41ea00640 1 -- 172.21.15.86:0/3321683331 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe41808f4d0 con 0x7fe428070330 2024-04-25T14:04:54.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.237+0000 7fe426000640 1 --2- 172.21.15.86:0/3321683331 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe40806f430 0x7fe408071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:54.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.237+0000 7fe426000640 1 --2- 172.21.15.86:0/3321683331 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe40806f430 0x7fe408071940 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe4280c4d60 tx=0x7fe414009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.241+0000 7fe41ea00640 1 -- 172.21.15.86:0/3321683331 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe42815b6c0 con 0x7fe428070330 2024-04-25T14:04:54.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.433+0000 7fe42da00640 1 -- 172.21.15.86:0/3321683331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4280b0b60 con 0x7fe428070330 2024-04-25T14:04:54.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.433+0000 7fe41ea00640 1 -- 172.21.15.86:0/3321683331 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5177 (secure 0 0 0) 0x7fe41805bfc0 con 0x7fe428070330 2024-04-25T14:04:54.433 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:54.434 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/347107112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":347107112},{"type":"v1","addr":"172.21.15.155:6835","nonce":347107112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2236737549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2236737549},{"type":"v1","addr":"172.21.15.86:6837","nonce":2236737549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/519560229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":519560229},{"type":"v1","addr":"172.21.15.86:6836","nonce":519560229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:03:50.852257+0000","modified":"2024-04-25T14:04:09.202137+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6833/1944249570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1944249570},{"type":"v1","addr":"172.21.15.155:6833","nonce":1944249570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4275,"qdb_cluster":[4275]},"id":1}]} 2024-04-25T14:04:54.434 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 9 2024-04-25T14:04:54.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.433+0000 7fe42da00640 1 -- 172.21.15.86:0/3321683331 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe40806f430 msgr2=0x7fe408071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:54.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.433+0000 7fe42da00640 1 --2- 172.21.15.86:0/3321683331 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe40806f430 0x7fe408071940 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe4280c4d60 tx=0x7fe414009040 comp rx=0 tx=0).stop 2024-04-25T14:04:54.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.433+0000 7fe42da00640 1 -- 172.21.15.86:0/3321683331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe428070330 msgr2=0x7fe42809a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:54.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.433+0000 7fe42da00640 1 --2- 172.21.15.86:0/3321683331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe428070330 0x7fe42809a9e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe41800c790 tx=0x7fe41800cc50 comp rx=0 tx=0).stop 2024-04-25T14:04:54.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.433+0000 7fe42da00640 1 -- 172.21.15.86:0/3321683331 shutdown_connections 2024-04-25T14:04:54.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.433+0000 7fe42da00640 1 --2- 172.21.15.86:0/3321683331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe42815ad70 0x7fe42815b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:54.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.433+0000 7fe42da00640 1 --2- 172.21.15.86:0/3321683331 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe40806f430 0x7fe408071940 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:54.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.433+0000 7fe42da00640 1 --2- 172.21.15.86:0/3321683331 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4280cde90 0x7fe428157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:54.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.433+0000 7fe42da00640 1 --2- 172.21.15.86:0/3321683331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe428070330 0x7fe42809a9e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:54.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.433+0000 7fe42da00640 1 -- 172.21.15.86:0/3321683331 >> 172.21.15.86:0/3321683331 conn(0x7fe4280b4650 msgr2=0x7fe4280742b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:54.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.433+0000 7fe42da00640 1 -- 172.21.15.86:0/3321683331 shutdown_connections 2024-04-25T14:04:54.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.433+0000 7fe42da00640 1 -- 172.21.15.86:0/3321683331 wait complete. 2024-04-25T14:04:54.455 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:04:54.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.601+0000 7f479a000640 1 Processor -- start 2024-04-25T14:04:54.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.601+0000 7f479a000640 1 -- start start 2024-04-25T14:04:54.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.605+0000 7f479a000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47940cfca0 0x7f479406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:54.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.605+0000 7f479a000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f47940730b0 0x7f47940734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:54.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.605+0000 7f479a000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4794072bd0 con 0x7f47940d16e0 2024-04-25T14:04:54.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.605+0000 7f479a000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f4794070470 con 0x7f47940730b0 2024-04-25T14:04:54.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.605+0000 7f479a000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f479406f930 con 0x7f47940cfca0 2024-04-25T14:04:54.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.605+0000 7f4792a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f47940d16e0 0x7f47940d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54466/0 (socket says 172.21.15.86:54466) 2024-04-25T14:04:54.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.605+0000 7f4792a00640 1 -- 172.21.15.86:0/1424595724 learned_addr learned my addr 172.21.15.86:0/1424595724 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:54.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.605+0000 7f4793400640 1 --2- 172.21.15.86:0/1424595724 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47940cfca0 0x7f479406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:54.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.605+0000 7f4793e00640 1 --2- 172.21.15.86:0/1424595724 >> v2:172.21.15.155:3300/0 conn(0x7f47940730b0 0x7f47940734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:54.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.605+0000 7f4792000640 1 -- 172.21.15.86:0/1424595724 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3625055889 0 0) 0x7f4794072bd0 con 0x7f47940d16e0 2024-04-25T14:04:54.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.605+0000 7f4792000640 1 -- 172.21.15.86:0/1424595724 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4778003530 con 0x7f47940d16e0 2024-04-25T14:04:54.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.605+0000 7f4793400640 1 -- 172.21.15.86:0/1424595724 >> v2:172.21.15.155:3300/0 conn(0x7f47940730b0 msgr2=0x7f47940734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:54.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.605+0000 7f4793400640 1 --2- 172.21.15.86:0/1424595724 >> v2:172.21.15.155:3300/0 conn(0x7f47940730b0 0x7f47940734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:54.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.605+0000 7f4793400640 1 -- 172.21.15.86:0/1424595724 >> v1:172.21.15.86:6789/0 conn(0x7f47940d16e0 legacy=0x7f47940d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:54.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.605+0000 7f4793400640 1 -- 172.21.15.86:0/1424595724 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479406fbd0 con 0x7f47940cfca0 2024-04-25T14:04:54.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.605+0000 7f4793e00640 1 --2- 172.21.15.86:0/1424595724 >> v2:172.21.15.155:3300/0 conn(0x7f47940730b0 0x7f47940734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:54.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.609+0000 7f4793400640 1 --2- 172.21.15.86:0/1424595724 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47940cfca0 0x7f479406f3f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f478400ba60 tx=0x7f478401eb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b4c52761242e7d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:54.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.609+0000 7f4792000640 1 -- 172.21.15.86:0/1424595724 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47840b4070 con 0x7f47940cfca0 2024-04-25T14:04:54.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.609+0000 7f4792000640 1 -- 172.21.15.86:0/1424595724 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47840a7b50 con 0x7f47940cfca0 2024-04-25T14:04:54.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.609+0000 7f4792000640 1 -- 172.21.15.86:0/1424595724 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47840af440 con 0x7f47940cfca0 2024-04-25T14:04:54.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.609+0000 7f479a000640 1 -- 172.21.15.86:0/1424595724 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47940cfca0 msgr2=0x7f479406f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:54.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.609+0000 7f479a000640 1 --2- 172.21.15.86:0/1424595724 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47940cfca0 0x7f479406f3f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f478400ba60 tx=0x7f478401eb70 comp rx=0 tx=0).stop 2024-04-25T14:04:54.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.609+0000 7f479a000640 1 -- 172.21.15.86:0/1424595724 shutdown_connections 2024-04-25T14:04:54.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.609+0000 7f479a000640 1 --2- 172.21.15.86:0/1424595724 >> v2:172.21.15.155:3300/0 conn(0x7f47940730b0 0x7f47940734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:54.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.609+0000 7f479a000640 1 --2- 172.21.15.86:0/1424595724 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47940cfca0 0x7f479406f3f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:54.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.609+0000 7f479a000640 1 -- 172.21.15.86:0/1424595724 >> 172.21.15.86:0/1424595724 conn(0x7f47940b4650 msgr2=0x7f47940b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:54.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.609+0000 7f479a000640 1 -- 172.21.15.86:0/1424595724 shutdown_connections 2024-04-25T14:04:54.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.609+0000 7f479a000640 1 -- 172.21.15.86:0/1424595724 wait complete. 2024-04-25T14:04:54.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.613+0000 7f479a000640 1 Processor -- start 2024-04-25T14:04:54.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.613+0000 7f479a000640 1 -- start start 2024-04-25T14:04:54.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.613+0000 7f479a000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47940730b0 0x7f4794159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:54.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.613+0000 7f479a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47940cfca0 0x7f4794157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:54.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.613+0000 7f479a000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47940d16e0 0x7f47940cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:54.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.613+0000 7f479a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47940ce170 con 0x7f47940cfca0 2024-04-25T14:04:54.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.613+0000 7f479a000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f47940cdff0 con 0x7f47940730b0 2024-04-25T14:04:54.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.613+0000 7f479a000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f47940ce2f0 con 0x7f47940d16e0 2024-04-25T14:04:54.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.613+0000 7f4792a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47940cfca0 0x7f4794157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:54.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.613+0000 7f4792a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47940cfca0 0x7f4794157700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52346/0 (socket says 172.21.15.86:52346) 2024-04-25T14:04:54.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.613+0000 7f4792a00640 1 -- 172.21.15.86:0/1067416838 learned_addr learned my addr 172.21.15.86:0/1067416838 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:54.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.613+0000 7f4793400640 1 --2- 172.21.15.86:0/1067416838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47940730b0 0x7f4794159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:54.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.613+0000 7f4793e00640 1 --2- 172.21.15.86:0/1067416838 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47940d16e0 0x7f47940cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:54.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.613+0000 7f4792a00640 1 -- 172.21.15.86:0/1067416838 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47940d16e0 msgr2=0x7f47940cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:54.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.613+0000 7f4792a00640 1 --2- 172.21.15.86:0/1067416838 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47940d16e0 0x7f47940cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:54.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.613+0000 7f4792a00640 1 -- 172.21.15.86:0/1067416838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47940730b0 msgr2=0x7f4794159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:54.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.613+0000 7f4792a00640 1 --2- 172.21.15.86:0/1067416838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47940730b0 0x7f4794159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:54.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.613+0000 7f4792a00640 1 -- 172.21.15.86:0/1067416838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4794158e90 con 0x7f47940cfca0 2024-04-25T14:04:54.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.613+0000 7f4792a00640 1 --2- 172.21.15.86:0/1067416838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47940cfca0 0x7f4794157700 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f478000c3c0 tx=0x7f478000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:54.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.613+0000 7f478b400640 1 -- 172.21.15.86:0/1067416838 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4780017070 con 0x7f47940cfca0 2024-04-25T14:04:54.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.613+0000 7f478b400640 1 -- 172.21.15.86:0/1067416838 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4780005cf0 con 0x7f47940cfca0 2024-04-25T14:04:54.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.613+0000 7f479a000640 1 -- 172.21.15.86:0/1067416838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4794159170 con 0x7f47940cfca0 2024-04-25T14:04:54.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.613+0000 7f478b400640 1 -- 172.21.15.86:0/1067416838 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47800072f0 con 0x7f47940cfca0 2024-04-25T14:04:54.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.613+0000 7f479a000640 1 -- 172.21.15.86:0/1067416838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4794159730 con 0x7f47940cfca0 2024-04-25T14:04:54.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.617+0000 7f479a000640 1 -- 172.21.15.86:0/1067416838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f479409f4b0 con 0x7f47940cfca0 2024-04-25T14:04:54.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.617+0000 7f478b400640 1 -- 172.21.15.86:0/1067416838 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f478001a070 con 0x7f47940cfca0 2024-04-25T14:04:54.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.617+0000 7f478b400640 1 --2- 172.21.15.86:0/1067416838 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f477406f400 0x7f4774071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:54.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.617+0000 7f478b400640 1 -- 172.21.15.86:0/1067416838 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f478008e980 con 0x7f47940cfca0 2024-04-25T14:04:54.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.617+0000 7f4793400640 1 --2- 172.21.15.86:0/1067416838 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f477406f400 0x7f4774071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:54.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.617+0000 7f4793400640 1 --2- 172.21.15.86:0/1067416838 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f477406f400 0x7f4774071910 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f47840b1000 tx=0x7f47840a9070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:54.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.621+0000 7f478b400640 1 -- 172.21.15.86:0/1067416838 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f478005aeb0 con 0x7f47940cfca0 2024-04-25T14:04:54.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.780+0000 7f479a000640 1 -- 172.21.15.86:0/1067416838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4794158c20 con 0x7f47940cfca0 2024-04-25T14:04:54.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.784+0000 7f478b400640 1 -- 172.21.15.86:0/1067416838 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18902 (secure 0 0 0) 0x7f478005f1b0 con 0x7f47940cfca0 2024-04-25T14:04:54.786 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:04:54.786 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":26,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:04:31.930803+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:03:49.151473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:03:49.802597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T14:03:53.492008+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:04:54.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.784+0000 7f479a000640 1 -- 172.21.15.86:0/1067416838 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f477406f400 msgr2=0x7f4774071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:54.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.784+0000 7f479a000640 1 --2- 172.21.15.86:0/1067416838 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f477406f400 0x7f4774071910 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f47840b1000 tx=0x7f47840a9070 comp rx=0 tx=0).stop 2024-04-25T14:04:54.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.784+0000 7f479a000640 1 -- 172.21.15.86:0/1067416838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47940cfca0 msgr2=0x7f4794157700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:54.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.784+0000 7f479a000640 1 --2- 172.21.15.86:0/1067416838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47940cfca0 0x7f4794157700 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f478000c3c0 tx=0x7f478000c880 comp rx=0 tx=0).stop 2024-04-25T14:04:54.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.784+0000 7f479a000640 1 -- 172.21.15.86:0/1067416838 shutdown_connections 2024-04-25T14:04:54.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.784+0000 7f479a000640 1 --2- 172.21.15.86:0/1067416838 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f477406f400 0x7f4774071910 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:54.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.784+0000 7f479a000640 1 --2- 172.21.15.86:0/1067416838 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47940d16e0 0x7f47940cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:54.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.784+0000 7f479a000640 1 --2- 172.21.15.86:0/1067416838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47940cfca0 0x7f4794157700 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:54.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.784+0000 7f479a000640 1 --2- 172.21.15.86:0/1067416838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47940730b0 0x7f4794159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:54.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.784+0000 7f479a000640 1 -- 172.21.15.86:0/1067416838 >> 172.21.15.86:0/1067416838 conn(0x7f47940b4650 msgr2=0x7f479406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:54.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.784+0000 7f479a000640 1 -- 172.21.15.86:0/1067416838 shutdown_connections 2024-04-25T14:04:54.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.784+0000 7f479a000640 1 -- 172.21.15.86:0/1067416838 wait complete. 2024-04-25T14:04:54.807 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T14:04:54.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.952+0000 7f5624000640 1 Processor -- start 2024-04-25T14:04:54.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.952+0000 7f5624000640 1 -- start start 2024-04-25T14:04:54.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.952+0000 7f5624000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f561c0aeea0 0x7f561c0af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:54.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.952+0000 7f5624000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f561c06c670 con 0x7f561c15faa0 2024-04-25T14:04:54.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.956+0000 7f5624000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f561c06c950 con 0x7f561c0b59e0 2024-04-25T14:04:54.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.956+0000 7f5624000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f561c06d560 con 0x7f561c0aeea0 2024-04-25T14:04:54.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.956+0000 7f5621600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f561c15faa0 0x7f561c06d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54480/0 (socket says 172.21.15.86:54480) 2024-04-25T14:04:54.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.956+0000 7f5621600640 1 -- 172.21.15.86:0/596253715 learned_addr learned my addr 172.21.15.86:0/596253715 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:54.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.956+0000 7f5622000640 1 --2- 172.21.15.86:0/596253715 >> v2:172.21.15.86:3300/0 conn(0x7f561c0aeea0 0x7f561c0af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:54.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.956+0000 7f561be00640 1 -- 172.21.15.86:0/596253715 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 444097113 0 0) 0x7f561c06c670 con 0x7f561c15faa0 2024-04-25T14:04:54.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.956+0000 7f561be00640 1 -- 172.21.15.86:0/596253715 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5604003530 con 0x7f561c15faa0 2024-04-25T14:04:54.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.956+0000 7f561be00640 1 -- 172.21.15.86:0/596253715 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 11476979 0 0) 0x7f561c06c950 con 0x7f561c0b59e0 2024-04-25T14:04:54.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.956+0000 7f5622000640 1 -- 172.21.15.86:0/596253715 >> v1:172.21.15.155:6789/0 conn(0x7f561c0b59e0 legacy=0x7f561c0b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:54.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.956+0000 7f5622000640 1 -- 172.21.15.86:0/596253715 >> v1:172.21.15.86:6789/0 conn(0x7f561c15faa0 legacy=0x7f561c06d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:54.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.956+0000 7f5622000640 1 -- 172.21.15.86:0/596253715 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561c06cc30 con 0x7f561c0aeea0 2024-04-25T14:04:54.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.956+0000 7f5622000640 1 --2- 172.21.15.86:0/596253715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f561c0aeea0 0x7f561c0af2b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5614009d90 tx=0x7f561401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0cb1452861de07a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:54.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.956+0000 7f561be00640 1 -- 172.21.15.86:0/596253715 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56140b2070 con 0x7f561c0aeea0 2024-04-25T14:04:54.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.956+0000 7f561be00640 1 -- 172.21.15.86:0/596253715 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56140a5a50 con 0x7f561c0aeea0 2024-04-25T14:04:54.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.956+0000 7f561be00640 1 -- 172.21.15.86:0/596253715 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56140a5e20 con 0x7f561c0aeea0 2024-04-25T14:04:54.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.960+0000 7f5624000640 1 -- 172.21.15.86:0/596253715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f561c0aeea0 msgr2=0x7f561c0af2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:54.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.960+0000 7f5624000640 1 --2- 172.21.15.86:0/596253715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f561c0aeea0 0x7f561c0af2b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5614009d90 tx=0x7f561401c970 comp rx=0 tx=0).stop 2024-04-25T14:04:54.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.960+0000 7f5624000640 1 -- 172.21.15.86:0/596253715 shutdown_connections 2024-04-25T14:04:54.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.960+0000 7f5624000640 1 --2- 172.21.15.86:0/596253715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f561c0aeea0 0x7f561c0af2b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:54.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.960+0000 7f5624000640 1 -- 172.21.15.86:0/596253715 >> 172.21.15.86:0/596253715 conn(0x7f561c0d2230 msgr2=0x7f561c0d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:54.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.960+0000 7f5624000640 1 -- 172.21.15.86:0/596253715 shutdown_connections 2024-04-25T14:04:54.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.960+0000 7f5624000640 1 -- 172.21.15.86:0/596253715 wait complete. 2024-04-25T14:04:54.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.960+0000 7f5624000640 1 Processor -- start 2024-04-25T14:04:54.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.960+0000 7f5624000640 1 -- start start 2024-04-25T14:04:54.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.964+0000 7f5624000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f561c0aeea0 0x7f561c1583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:54.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.964+0000 7f5624000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f561c0b59e0 0x7f561c15ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:54.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.964+0000 7f5624000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f561c15faa0 0x7f561c13cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:54.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.964+0000 7f5624000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f561c0710b0 con 0x7f561c0b59e0 2024-04-25T14:04:54.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.964+0000 7f5624000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f561c070f30 con 0x7f561c0aeea0 2024-04-25T14:04:54.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.964+0000 7f5624000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f561c16e780 con 0x7f561c15faa0 2024-04-25T14:04:54.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.964+0000 7f5620c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f561c0b59e0 0x7f561c15ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:54.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.964+0000 7f5620c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f561c0b59e0 0x7f561c15ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52358/0 (socket says 172.21.15.86:52358) 2024-04-25T14:04:54.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.964+0000 7f5620c00640 1 -- 172.21.15.86:0/1594265561 learned_addr learned my addr 172.21.15.86:0/1594265561 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:54.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.964+0000 7f5622000640 1 --2- 172.21.15.86:0/1594265561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f561c15faa0 0x7f561c13cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:54.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.964+0000 7f5621600640 1 --2- 172.21.15.86:0/1594265561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f561c0aeea0 0x7f561c1583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:54.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.964+0000 7f5620c00640 1 -- 172.21.15.86:0/1594265561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f561c15faa0 msgr2=0x7f561c13cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:54.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.964+0000 7f5620c00640 1 --2- 172.21.15.86:0/1594265561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f561c15faa0 0x7f561c13cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:54.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.964+0000 7f5620c00640 1 -- 172.21.15.86:0/1594265561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f561c0aeea0 msgr2=0x7f561c1583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:54.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.964+0000 7f5620c00640 1 --2- 172.21.15.86:0/1594265561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f561c0aeea0 0x7f561c1583b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:54.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.964+0000 7f5620c00640 1 -- 172.21.15.86:0/1594265561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561c0b53c0 con 0x7f561c0b59e0 2024-04-25T14:04:54.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.964+0000 7f5622000640 1 --2- 172.21.15.86:0/1594265561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f561c15faa0 0x7f561c13cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:54.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.964+0000 7f5621600640 1 --2- 172.21.15.86:0/1594265561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f561c0aeea0 0x7f561c1583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:54.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.964+0000 7f5620c00640 1 --2- 172.21.15.86:0/1594265561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f561c0b59e0 0x7f561c15ea00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f560800c450 tx=0x7f560800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:54.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.964+0000 7f5619600640 1 -- 172.21.15.86:0/1594265561 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5608017070 con 0x7f561c0b59e0 2024-04-25T14:04:54.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.964+0000 7f5619600640 1 -- 172.21.15.86:0/1594265561 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5608005480 con 0x7f561c0b59e0 2024-04-25T14:04:54.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.964+0000 7f5624000640 1 -- 172.21.15.86:0/1594265561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f561c1588f0 con 0x7f561c0b59e0 2024-04-25T14:04:54.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.964+0000 7f5619600640 1 -- 172.21.15.86:0/1594265561 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5608003840 con 0x7f561c0b59e0 2024-04-25T14:04:54.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.964+0000 7f5624000640 1 -- 172.21.15.86:0/1594265561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f561c1590e0 con 0x7f561c0b59e0 2024-04-25T14:04:54.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.964+0000 7f5624000640 1 -- 172.21.15.86:0/1594265561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f561c074cf0 con 0x7f561c0b59e0 2024-04-25T14:04:54.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.968+0000 7f5619600640 1 -- 172.21.15.86:0/1594265561 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f560801a070 con 0x7f561c0b59e0 2024-04-25T14:04:54.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.968+0000 7f5619600640 1 --2- 172.21.15.86:0/1594265561 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f55fc06f430 0x7f55fc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:54.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.968+0000 7f5619600640 1 -- 172.21.15.86:0/1594265561 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f560808ea70 con 0x7f561c0b59e0 2024-04-25T14:04:54.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.968+0000 7f5621600640 1 --2- 172.21.15.86:0/1594265561 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f55fc06f430 0x7f55fc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:54.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.968+0000 7f5621600640 1 --2- 172.21.15.86:0/1594265561 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f55fc06f430 0x7f55fc071940 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5610005890 tx=0x7f5610009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:54.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:54.972+0000 7f5619600640 1 -- 172.21.15.86:0/1594265561 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f560805b4c0 con 0x7f561c0b59e0 2024-04-25T14:04:55.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.164+0000 7f5624000640 1 -- 172.21.15.86:0/1594265561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f561c162510 con 0x7f561c0b59e0 2024-04-25T14:04:55.167 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-25T14:04:55.164+0000 7fea29c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:04:55.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.168+0000 7f5619600640 1 -- 172.21.15.86:0/1594265561 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7f560805adc0 con 0x7f561c0b59e0 2024-04-25T14:04:55.170 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T14:04:55.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.168+0000 7f5624000640 1 -- 172.21.15.86:0/1594265561 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f55fc06f430 msgr2=0x7f55fc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:55.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.168+0000 7f5624000640 1 --2- 172.21.15.86:0/1594265561 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f55fc06f430 0x7f55fc071940 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5610005890 tx=0x7f5610009040 comp rx=0 tx=0).stop 2024-04-25T14:04:55.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.168+0000 7f5624000640 1 -- 172.21.15.86:0/1594265561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f561c0b59e0 msgr2=0x7f561c15ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:55.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.168+0000 7f5624000640 1 --2- 172.21.15.86:0/1594265561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f561c0b59e0 0x7f561c15ea00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f560800c450 tx=0x7f560800c910 comp rx=0 tx=0).stop 2024-04-25T14:04:55.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.168+0000 7f5624000640 1 -- 172.21.15.86:0/1594265561 shutdown_connections 2024-04-25T14:04:55.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.168+0000 7f5624000640 1 --2- 172.21.15.86:0/1594265561 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f55fc06f430 0x7f55fc071940 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:55.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.168+0000 7f5624000640 1 --2- 172.21.15.86:0/1594265561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f561c15faa0 0x7f561c13cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:55.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.168+0000 7f5624000640 1 --2- 172.21.15.86:0/1594265561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f561c0b59e0 0x7f561c15ea00 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:55.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.168+0000 7f5624000640 1 --2- 172.21.15.86:0/1594265561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f561c0aeea0 0x7f561c1583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:55.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.168+0000 7f5624000640 1 -- 172.21.15.86:0/1594265561 >> 172.21.15.86:0/1594265561 conn(0x7f561c0d2230 msgr2=0x7f561c0afc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:55.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.172+0000 7f5624000640 1 -- 172.21.15.86:0/1594265561 shutdown_connections 2024-04-25T14:04:55.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.172+0000 7f5624000640 1 -- 172.21.15.86:0/1594265561 wait complete. 2024-04-25T14:04:55.184 INFO:tasks.ceph.mds.d.smithi155.stderr: -5648> 2024-04-25T14:03:55.420+0000 7f6e4d800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T14:04:55.195 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T14:04:55.305 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2024-04-25T14:04:55.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.344+0000 7f4409000640 1 Processor -- start 2024-04-25T14:04:55.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.344+0000 7f4409000640 1 -- start start 2024-04-25T14:04:55.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.344+0000 7f4409000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f44040d16e0 0x7f44040d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:55.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.344+0000 7f4409000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44040d2030 con 0x7f44040cfca0 2024-04-25T14:04:55.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.344+0000 7f4409000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4404072bd0 con 0x7f44040730b0 2024-04-25T14:04:55.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.344+0000 7f4409000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f4404070470 con 0x7f44040d16e0 2024-04-25T14:04:55.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.344+0000 7f4401e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f44040cfca0 0x7f440406f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54488/0 (socket says 172.21.15.86:54488) 2024-04-25T14:04:55.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.344+0000 7f4401e00640 1 -- 172.21.15.86:0/2433232910 learned_addr learned my addr 172.21.15.86:0/2433232910 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:55.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.344+0000 7f43f9400640 1 --2- 172.21.15.86:0/2433232910 >> v2:172.21.15.155:3300/0 conn(0x7f44040d16e0 0x7f44040d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:55.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.344+0000 7f4401400640 1 -- 172.21.15.86:0/2433232910 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3357366193 0 0) 0x7f43f4001120 con 0x7f44040cfca0 2024-04-25T14:04:55.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.344+0000 7f4401400640 1 -- 172.21.15.86:0/2433232910 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43e8003530 con 0x7f44040cfca0 2024-04-25T14:04:55.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.344+0000 7f4401400640 1 -- 172.21.15.86:0/2433232910 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3940419933 0 0) 0x7f43e8003530 con 0x7f44040cfca0 2024-04-25T14:04:55.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.344+0000 7f43f9400640 1 -- 172.21.15.86:0/2433232910 >> v1:172.21.15.155:6789/0 conn(0x7f44040730b0 legacy=0x7f44040734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:55.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.344+0000 7f43f9400640 1 -- 172.21.15.86:0/2433232910 >> v1:172.21.15.86:6789/0 conn(0x7f44040cfca0 legacy=0x7f440406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:55.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.344+0000 7f43f9400640 1 -- 172.21.15.86:0/2433232910 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f440406fbd0 con 0x7f44040d16e0 2024-04-25T14:04:55.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.348+0000 7f43f9400640 1 --2- 172.21.15.86:0/2433232910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44040d16e0 0x7f44040d1af0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f43ec009740 tx=0x7f43ec01c1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2a5be7e5467450a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:55.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.348+0000 7f4401400640 1 -- 172.21.15.86:0/2433232910 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec0b0070 con 0x7f44040d16e0 2024-04-25T14:04:55.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.348+0000 7f4401400640 1 -- 172.21.15.86:0/2433232910 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43ec0ab400 con 0x7f44040d16e0 2024-04-25T14:04:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.348+0000 7f4401400640 1 -- 172.21.15.86:0/2433232910 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44040743e0 con 0x7f44040d16e0 2024-04-25T14:04:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.348+0000 7f4409000640 1 -- 172.21.15.86:0/2433232910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44040d16e0 msgr2=0x7f44040d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.348+0000 7f4409000640 1 --2- 172.21.15.86:0/2433232910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44040d16e0 0x7f44040d1af0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f43ec009740 tx=0x7f43ec01c1f0 comp rx=0 tx=0).stop 2024-04-25T14:04:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.348+0000 7f4409000640 1 -- 172.21.15.86:0/2433232910 shutdown_connections 2024-04-25T14:04:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.348+0000 7f4409000640 1 --2- 172.21.15.86:0/2433232910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44040d16e0 0x7f44040d1af0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.348+0000 7f4409000640 1 -- 172.21.15.86:0/2433232910 >> 172.21.15.86:0/2433232910 conn(0x7f44040b4650 msgr2=0x7f44040b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:55.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.348+0000 7f4409000640 1 -- 172.21.15.86:0/2433232910 shutdown_connections 2024-04-25T14:04:55.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.348+0000 7f4409000640 1 -- 172.21.15.86:0/2433232910 wait complete. 2024-04-25T14:04:55.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.352+0000 7f4409000640 1 Processor -- start 2024-04-25T14:04:55.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.352+0000 7f4409000640 1 -- start start 2024-04-25T14:04:55.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.352+0000 7f4409000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44040730b0 0x7f440409a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:55.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.352+0000 7f4409000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44040cfca0 0x7f4404159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:55.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.352+0000 7f4409000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44040d16e0 0x7f4404159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:55.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.352+0000 7f4409000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f440416aa80 con 0x7f44040d16e0 2024-04-25T14:04:55.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.352+0000 7f4409000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f440416a900 con 0x7f44040cfca0 2024-04-25T14:04:55.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.352+0000 7f4409000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f440416ac00 con 0x7f44040730b0 2024-04-25T14:04:55.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.352+0000 7f4402800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44040d16e0 0x7f4404159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:55.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.352+0000 7f4401e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44040730b0 0x7f440409a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:55.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.356+0000 7f4402800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44040d16e0 0x7f4404159780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52366/0 (socket says 172.21.15.86:52366) 2024-04-25T14:04:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.356+0000 7f43f9400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44040cfca0 0x7f4404159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.356+0000 7f4402800640 1 -- 172.21.15.86:0/1809838385 learned_addr learned my addr 172.21.15.86:0/1809838385 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.356+0000 7f4402800640 1 -- 172.21.15.86:0/1809838385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44040730b0 msgr2=0x7f440409a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.356+0000 7f4402800640 1 --2- 172.21.15.86:0/1809838385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44040730b0 0x7f440409a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.356+0000 7f4402800640 1 -- 172.21.15.86:0/1809838385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44040cfca0 msgr2=0x7f4404159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.356+0000 7f4402800640 1 --2- 172.21.15.86:0/1809838385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44040cfca0 0x7f4404159240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.356+0000 7f4402800640 1 -- 172.21.15.86:0/1809838385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4404155810 con 0x7f44040d16e0 2024-04-25T14:04:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.356+0000 7f4402800640 1 --2- 172.21.15.86:0/1809838385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44040d16e0 0x7f4404159780 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f43fc00c8c0 tx=0x7f43fc00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:55.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.356+0000 7f43faa00640 1 -- 172.21.15.86:0/1809838385 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc017070 con 0x7f44040d16e0 2024-04-25T14:04:55.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.356+0000 7f43faa00640 1 -- 172.21.15.86:0/1809838385 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43fc003db0 con 0x7f44040d16e0 2024-04-25T14:04:55.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.356+0000 7f4409000640 1 -- 172.21.15.86:0/1809838385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f440409aaf0 con 0x7f44040d16e0 2024-04-25T14:04:55.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.356+0000 7f43faa00640 1 -- 172.21.15.86:0/1809838385 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc005290 con 0x7f44040d16e0 2024-04-25T14:04:55.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.356+0000 7f4409000640 1 -- 172.21.15.86:0/1809838385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f440415da20 con 0x7f44040d16e0 2024-04-25T14:04:55.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.356+0000 7f4409000640 1 -- 172.21.15.86:0/1809838385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f440409f4b0 con 0x7f44040d16e0 2024-04-25T14:04:55.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.356+0000 7f43faa00640 1 -- 172.21.15.86:0/1809838385 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f43fc01a070 con 0x7f44040d16e0 2024-04-25T14:04:55.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.360+0000 7f43faa00640 1 --2- 172.21.15.86:0/1809838385 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f43e006f400 0x7f43e0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:55.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.360+0000 7f43faa00640 1 -- 172.21.15.86:0/1809838385 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7f43fc08eeb0 con 0x7f44040d16e0 2024-04-25T14:04:55.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.360+0000 7f4401e00640 1 --2- 172.21.15.86:0/1809838385 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f43e006f400 0x7f43e0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:55.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.360+0000 7f4401e00640 1 --2- 172.21.15.86:0/1809838385 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f43e006f400 0x7f43e0071910 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f43f4000fc0 tx=0x7f43f4001350 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:55.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.364+0000 7f43faa00640 1 -- 172.21.15.86:0/1809838385 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4404155810 con 0x7f44040d16e0 2024-04-25T14:04:55.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:55.560+0000 7f4409000640 1 -- 172.21.15.86:0/1809838385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4404069d30 con 0x7f44040d16e0 2024-04-25T14:04:56.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.168+0000 7f43faa00640 1 -- 172.21.15.86:0/1809838385 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f43fc05b250 con 0x7f44040d16e0 2024-04-25T14:04:56.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.172+0000 7f4409000640 1 -- 172.21.15.86:0/1809838385 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f43e006f400 msgr2=0x7f43e0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:56.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.172+0000 7f4409000640 1 --2- 172.21.15.86:0/1809838385 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f43e006f400 0x7f43e0071910 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f43f4000fc0 tx=0x7f43f4001350 comp rx=0 tx=0).stop 2024-04-25T14:04:56.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.172+0000 7f4409000640 1 -- 172.21.15.86:0/1809838385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44040d16e0 msgr2=0x7f4404159780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:56.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.172+0000 7f4409000640 1 --2- 172.21.15.86:0/1809838385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44040d16e0 0x7f4404159780 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f43fc00c8c0 tx=0x7f43fc00cd80 comp rx=0 tx=0).stop 2024-04-25T14:04:56.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.172+0000 7f4409000640 1 -- 172.21.15.86:0/1809838385 shutdown_connections 2024-04-25T14:04:56.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.172+0000 7f4409000640 1 --2- 172.21.15.86:0/1809838385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44040d16e0 0x7f4404159780 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:56.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.172+0000 7f4409000640 1 --2- 172.21.15.86:0/1809838385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44040cfca0 0x7f4404159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:56.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.172+0000 7f4409000640 1 --2- 172.21.15.86:0/1809838385 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f43e006f400 0x7f43e0071910 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:56.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.172+0000 7f4409000640 1 --2- 172.21.15.86:0/1809838385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44040730b0 0x7f440409a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:56.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.172+0000 7f4409000640 1 -- 172.21.15.86:0/1809838385 >> 172.21.15.86:0/1809838385 conn(0x7f44040b4650 msgr2=0x7f44040b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.172+0000 7f4409000640 1 -- 172.21.15.86:0/1809838385 shutdown_connections 2024-04-25T14:04:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.172+0000 7f4409000640 1 -- 172.21.15.86:0/1809838385 wait complete. 2024-04-25T14:04:56.195 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T14:04:56.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.344+0000 7f6c39e00640 1 Processor -- start 2024-04-25T14:04:56.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.344+0000 7f6c39e00640 1 -- start start 2024-04-25T14:04:56.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.348+0000 7f6c39e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c3415ca50 0x7f6c34073950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:56.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.348+0000 7f6c39e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6c34160d00 0x7f6c3406e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:56.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.348+0000 7f6c39e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c3415ce60 con 0x7f6c340743c0 2024-04-25T14:04:56.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.348+0000 7f6c39e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f6c34072b90 con 0x7f6c34160d00 2024-04-25T14:04:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.348+0000 7f6c39e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6c34072e70 con 0x7f6c3415ca50 2024-04-25T14:04:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.348+0000 7f6c32000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6c340743c0 0x7f6c341605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54504/0 (socket says 172.21.15.86:54504) 2024-04-25T14:04:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.348+0000 7f6c32000640 1 -- 172.21.15.86:0/2139400669 learned_addr learned my addr 172.21.15.86:0/2139400669 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.348+0000 7f6c32a00640 1 --2- 172.21.15.86:0/2139400669 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c3415ca50 0x7f6c34073950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.348+0000 7f6c33400640 1 --2- 172.21.15.86:0/2139400669 >> v2:172.21.15.155:3300/0 conn(0x7f6c34160d00 0x7f6c3406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:56.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.348+0000 7f6c31600640 1 -- 172.21.15.86:0/2139400669 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1330478769 0 0) 0x7f6c3415ce60 con 0x7f6c340743c0 2024-04-25T14:04:56.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.348+0000 7f6c31600640 1 -- 172.21.15.86:0/2139400669 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c18003530 con 0x7f6c340743c0 2024-04-25T14:04:56.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.348+0000 7f6c32a00640 1 -- 172.21.15.86:0/2139400669 >> v2:172.21.15.155:3300/0 conn(0x7f6c34160d00 msgr2=0x7f6c3406e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:56.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.348+0000 7f6c32a00640 1 --2- 172.21.15.86:0/2139400669 >> v2:172.21.15.155:3300/0 conn(0x7f6c34160d00 0x7f6c3406e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:56.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.348+0000 7f6c32a00640 1 -- 172.21.15.86:0/2139400669 >> v1:172.21.15.86:6789/0 conn(0x7f6c340743c0 legacy=0x7f6c341605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:56.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.348+0000 7f6c32a00640 1 -- 172.21.15.86:0/2139400669 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c34073150 con 0x7f6c3415ca50 2024-04-25T14:04:56.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.348+0000 7f6c33400640 1 --2- 172.21.15.86:0/2139400669 >> v2:172.21.15.155:3300/0 conn(0x7f6c34160d00 0x7f6c3406e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:56.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.352+0000 7f6c32a00640 1 --2- 172.21.15.86:0/2139400669 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c3415ca50 0x7f6c34073950 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6c240097c0 tx=0x7f6c2401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd1739cd8b4dad0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:56.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.352+0000 7f6c31600640 1 -- 172.21.15.86:0/2139400669 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c240b1070 con 0x7f6c3415ca50 2024-04-25T14:04:56.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.352+0000 7f6c31600640 1 -- 172.21.15.86:0/2139400669 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c240ac410 con 0x7f6c3415ca50 2024-04-25T14:04:56.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.352+0000 7f6c31600640 1 -- 172.21.15.86:0/2139400669 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c240ac7e0 con 0x7f6c3415ca50 2024-04-25T14:04:56.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.352+0000 7f6c39e00640 1 -- 172.21.15.86:0/2139400669 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c3415ca50 msgr2=0x7f6c34073950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:56.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.352+0000 7f6c39e00640 1 --2- 172.21.15.86:0/2139400669 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c3415ca50 0x7f6c34073950 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6c240097c0 tx=0x7f6c2401c4a0 comp rx=0 tx=0).stop 2024-04-25T14:04:56.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.352+0000 7f6c39e00640 1 -- 172.21.15.86:0/2139400669 shutdown_connections 2024-04-25T14:04:56.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.352+0000 7f6c39e00640 1 --2- 172.21.15.86:0/2139400669 >> v2:172.21.15.155:3300/0 conn(0x7f6c34160d00 0x7f6c3406e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:56.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.352+0000 7f6c39e00640 1 --2- 172.21.15.86:0/2139400669 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c3415ca50 0x7f6c34073950 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:56.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.352+0000 7f6c39e00640 1 -- 172.21.15.86:0/2139400669 >> 172.21.15.86:0/2139400669 conn(0x7f6c340b4a40 msgr2=0x7f6c340b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:56.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.352+0000 7f6c39e00640 1 -- 172.21.15.86:0/2139400669 shutdown_connections 2024-04-25T14:04:56.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.352+0000 7f6c39e00640 1 -- 172.21.15.86:0/2139400669 wait complete. 2024-04-25T14:04:56.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.352+0000 7f6c39e00640 1 Processor -- start 2024-04-25T14:04:56.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.352+0000 7f6c39e00640 1 -- start start 2024-04-25T14:04:56.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.356+0000 7f6c39e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c340743c0 0x7f6c3414e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:56.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.356+0000 7f6c39e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c3415ca50 0x7f6c3415e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:56.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.356+0000 7f6c39e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c3415efe0 0x7f6c3415f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:56.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.356+0000 7f6c39e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3406ca40 con 0x7f6c3415efe0 2024-04-25T14:04:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.356+0000 7f6c39e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3406c8c0 con 0x7f6c3415ca50 2024-04-25T14:04:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.356+0000 7f6c39e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6c3406cbc0 con 0x7f6c340743c0 2024-04-25T14:04:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.356+0000 7f6c33400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c3415efe0 0x7f6c3415f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.356+0000 7f6c33400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c3415efe0 0x7f6c3415f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52378/0 (socket says 172.21.15.86:52378) 2024-04-25T14:04:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.356+0000 7f6c33400640 1 -- 172.21.15.86:0/484343742 learned_addr learned my addr 172.21.15.86:0/484343742 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.356+0000 7f6c32000640 1 --2- 172.21.15.86:0/484343742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c3415ca50 0x7f6c3415e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.356+0000 7f6c32a00640 1 --2- 172.21.15.86:0/484343742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c340743c0 0x7f6c3414e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.356+0000 7f6c33400640 1 -- 172.21.15.86:0/484343742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c340743c0 msgr2=0x7f6c3414e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.356+0000 7f6c33400640 1 --2- 172.21.15.86:0/484343742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c340743c0 0x7f6c3414e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.356+0000 7f6c33400640 1 -- 172.21.15.86:0/484343742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c3415ca50 msgr2=0x7f6c3415e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.356+0000 7f6c33400640 1 --2- 172.21.15.86:0/484343742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c3415ca50 0x7f6c3415e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.356+0000 7f6c33400640 1 -- 172.21.15.86:0/484343742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c3415f930 con 0x7f6c3415efe0 2024-04-25T14:04:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.356+0000 7f6c32000640 1 --2- 172.21.15.86:0/484343742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c3415ca50 0x7f6c3415e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.356+0000 7f6c33400640 1 --2- 172.21.15.86:0/484343742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c3415efe0 0x7f6c3415f3f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009e60 tx=0x7f6c2c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:56.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.356+0000 7f6c2b400640 1 -- 172.21.15.86:0/484343742 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c018030 con 0x7f6c3415efe0 2024-04-25T14:04:56.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.356+0000 7f6c2b400640 1 -- 172.21.15.86:0/484343742 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c2c00cdf0 con 0x7f6c3415efe0 2024-04-25T14:04:56.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.356+0000 7f6c2b400640 1 -- 172.21.15.86:0/484343742 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c003bc0 con 0x7f6c3415efe0 2024-04-25T14:04:56.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.356+0000 7f6c39e00640 1 -- 172.21.15.86:0/484343742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c34157150 con 0x7f6c3415efe0 2024-04-25T14:04:56.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.356+0000 7f6c39e00640 1 -- 172.21.15.86:0/484343742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c34157710 con 0x7f6c3415efe0 2024-04-25T14:04:56.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.356+0000 7f6c39e00640 1 -- 172.21.15.86:0/484343742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c34050750 con 0x7f6c3415efe0 2024-04-25T14:04:56.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.360+0000 7f6c2b400640 1 -- 172.21.15.86:0/484343742 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6c2c01b070 con 0x7f6c3415efe0 2024-04-25T14:04:56.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.360+0000 7f6c2b400640 1 --2- 172.21.15.86:0/484343742 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6c1406f430 0x7f6c14071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:56.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.360+0000 7f6c2b400640 1 -- 172.21.15.86:0/484343742 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7f6c2c08ff80 con 0x7f6c3415efe0 2024-04-25T14:04:56.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.360+0000 7f6c32a00640 1 --2- 172.21.15.86:0/484343742 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6c1406f430 0x7f6c14071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:56.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.360+0000 7f6c32a00640 1 --2- 172.21.15.86:0/484343742 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6c1406f430 0x7f6c14071940 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6c240ae000 tx=0x7f6c24008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:56.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.364+0000 7f6c2b400640 1 -- 172.21.15.86:0/484343742 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6c2c022070 con 0x7f6c3415efe0 2024-04-25T14:04:56.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:56.524+0000 7f6c39e00640 1 -- 172.21.15.86:0/484343742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6c34156ea0 con 0x7f6c3415efe0 2024-04-25T14:04:57.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.176+0000 7f6c2b400640 1 -- 172.21.15.86:0/484343742 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) ==== 181+0+0 (secure 0 0 0) 0x7f6c3415f930 con 0x7f6c3415efe0 2024-04-25T14:04:57.180 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-04-25T14:04:57.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.180+0000 7f6c39e00640 1 -- 172.21.15.86:0/484343742 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6c1406f430 msgr2=0x7f6c14071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:57.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.180+0000 7f6c39e00640 1 --2- 172.21.15.86:0/484343742 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6c1406f430 0x7f6c14071940 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6c240ae000 tx=0x7f6c24008040 comp rx=0 tx=0).stop 2024-04-25T14:04:57.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.180+0000 7f6c39e00640 1 -- 172.21.15.86:0/484343742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c3415efe0 msgr2=0x7f6c3415f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:57.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.180+0000 7f6c39e00640 1 --2- 172.21.15.86:0/484343742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c3415efe0 0x7f6c3415f3f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009e60 tx=0x7f6c2c00c490 comp rx=0 tx=0).stop 2024-04-25T14:04:57.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.180+0000 7f6c39e00640 1 -- 172.21.15.86:0/484343742 shutdown_connections 2024-04-25T14:04:57.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.180+0000 7f6c39e00640 1 --2- 172.21.15.86:0/484343742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c3415efe0 0x7f6c3415f3f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.180+0000 7f6c39e00640 1 --2- 172.21.15.86:0/484343742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c3415ca50 0x7f6c3415e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.180+0000 7f6c39e00640 1 --2- 172.21.15.86:0/484343742 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f6c1406f430 0x7f6c14071940 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.180+0000 7f6c39e00640 1 --2- 172.21.15.86:0/484343742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c340743c0 0x7f6c3414e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.180+0000 7f6c39e00640 1 -- 172.21.15.86:0/484343742 >> 172.21.15.86:0/484343742 conn(0x7f6c340b4a40 msgr2=0x7f6c3415bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:57.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.180+0000 7f6c39e00640 1 -- 172.21.15.86:0/484343742 shutdown_connections 2024-04-25T14:04:57.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.180+0000 7f6c39e00640 1 -- 172.21.15.86:0/484343742 wait complete. 2024-04-25T14:04:57.207 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T14:04:57.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f3004c00640 1 Processor -- start 2024-04-25T14:04:57.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f3004c00640 1 -- start start 2024-04-25T14:04:57.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f3004c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f300015ca50 0x7f3000073950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:57.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f3004c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f300015ce60 con 0x7f3000160d00 2024-04-25T14:04:57.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f3004c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3000072b90 con 0x7f30000743c0 2024-04-25T14:04:57.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f3004c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3000072e70 con 0x7f300015ca50 2024-04-25T14:04:57.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f2ffe800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3000160d00 0x7f300006e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54518/0 (socket says 172.21.15.86:54518) 2024-04-25T14:04:57.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f2ffe800640 1 -- 172.21.15.86:0/891336284 learned_addr learned my addr 172.21.15.86:0/891336284 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:57.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f2ffde00640 1 --2- 172.21.15.86:0/891336284 >> v2:172.21.15.155:3300/0 conn(0x7f300015ca50 0x7f3000073950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:57.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f2ffca00640 1 -- 172.21.15.86:0/891336284 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2815655209 0 0) 0x7f300015ce60 con 0x7f3000160d00 2024-04-25T14:04:57.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f2ffca00640 1 -- 172.21.15.86:0/891336284 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fe4005740 con 0x7f3000160d00 2024-04-25T14:04:57.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f2ffca00640 1 -- 172.21.15.86:0/891336284 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3851375142 0 0) 0x7f2fe4005740 con 0x7f3000160d00 2024-04-25T14:04:57.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f2ffca00640 1 -- 172.21.15.86:0/891336284 >> v2:172.21.15.155:3300/0 conn(0x7f300015ca50 msgr2=0x7f3000073950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:57.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f2ffca00640 1 --2- 172.21.15.86:0/891336284 >> v2:172.21.15.155:3300/0 conn(0x7f300015ca50 0x7f3000073950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f2ffca00640 1 -- 172.21.15.86:0/891336284 >> v1:172.21.15.155:6789/0 conn(0x7f30000743c0 legacy=0x7f30001605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:57.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f2ffca00640 1 -- 172.21.15.86:0/891336284 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3000073150 con 0x7f3000160d00 2024-04-25T14:04:57.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f2ffca00640 1 -- 172.21.15.86:0/891336284 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f2ff8001a00 con 0x7f3000160d00 2024-04-25T14:04:57.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f2ffca00640 1 -- 172.21.15.86:0/891336284 >> v1:172.21.15.86:6789/0 conn(0x7f3000160d00 legacy=0x7f300006e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:57.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f2ffca00640 1 --2- 172.21.15.86:0/891336284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fe4005bd0 0x7f2fe4006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:57.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f2ffca00640 1 --2- 172.21.15.86:0/891336284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fe4006580 0x7f2fe4006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:57.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f2ffca00640 1 --2- 172.21.15.86:0/891336284 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fe4007200 0x7f2fe400bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:57.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f2ffca00640 1 -- 172.21.15.86:0/891336284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3000072e70 con 0x7f2fe4006580 2024-04-25T14:04:57.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f2ffca00640 1 -- 172.21.15.86:0/891336284 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe400c170 con 0x7f2fe4005bd0 2024-04-25T14:04:57.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f2ffca00640 1 -- 172.21.15.86:0/891336284 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2fe400c370 con 0x7f2fe4007200 2024-04-25T14:04:57.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f2ffd400640 1 --2- 172.21.15.86:0/891336284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fe4006580 0x7f2fe4006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:57.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f2ffde00640 1 --2- 172.21.15.86:0/891336284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fe4005bd0 0x7f2fe4006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:57.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f2ffe800640 1 --2- 172.21.15.86:0/891336284 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fe4007200 0x7f2fe400bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:57.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f2ffd400640 1 -- 172.21.15.86:0/891336284 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fe4007200 msgr2=0x7f2fe400bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:57.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.356+0000 7f2ffd400640 1 --2- 172.21.15.86:0/891336284 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fe4007200 0x7f2fe400bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.360+0000 7f2ffd400640 1 -- 172.21.15.86:0/891336284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fe4005bd0 msgr2=0x7f2fe4006040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:57.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.360+0000 7f2ffd400640 1 --2- 172.21.15.86:0/891336284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fe4005bd0 0x7f2fe4006040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.360+0000 7f2ffd400640 1 -- 172.21.15.86:0/891336284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe4005740 con 0x7f2fe4006580 2024-04-25T14:04:57.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.360+0000 7f2ffe800640 1 -- 172.21.15.86:0/891336284 reap_dead start 2024-04-25T14:04:57.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.360+0000 7f2ffd400640 1 --2- 172.21.15.86:0/891336284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fe4006580 0x7f2fe4006a20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2fec001780 tx=0x7f2fec01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f646c47ee7059877 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:57.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.360+0000 7f2ffca00640 1 -- 172.21.15.86:0/891336284 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec0013a0 con 0x7f2fe4006580 2024-04-25T14:04:57.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.360+0000 7f2ffca00640 1 -- 172.21.15.86:0/891336284 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fec0052f0 con 0x7f2fe4006580 2024-04-25T14:04:57.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.360+0000 7f2ffca00640 1 -- 172.21.15.86:0/891336284 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec003840 con 0x7f2fe4006580 2024-04-25T14:04:57.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.360+0000 7f3004c00640 1 -- 172.21.15.86:0/891336284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fe4006580 msgr2=0x7f2fe4006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:57.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.360+0000 7f3004c00640 1 --2- 172.21.15.86:0/891336284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fe4006580 0x7f2fe4006a20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2fec001780 tx=0x7f2fec01c400 comp rx=0 tx=0).stop 2024-04-25T14:04:57.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.360+0000 7f3004c00640 1 -- 172.21.15.86:0/891336284 shutdown_connections 2024-04-25T14:04:57.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.360+0000 7f3004c00640 1 --2- 172.21.15.86:0/891336284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fe4006580 0x7f2fe4006a20 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.360+0000 7f3004c00640 1 -- 172.21.15.86:0/891336284 >> 172.21.15.86:0/891336284 conn(0x7f30000b4a40 msgr2=0x7f30000b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:57.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.360+0000 7f3004c00640 1 -- 172.21.15.86:0/891336284 shutdown_connections 2024-04-25T14:04:57.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.360+0000 7f3004c00640 1 -- 172.21.15.86:0/891336284 wait complete. 2024-04-25T14:04:57.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.364+0000 7f3004c00640 1 Processor -- start 2024-04-25T14:04:57.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.364+0000 7f3004c00640 1 -- start start 2024-04-25T14:04:57.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.364+0000 7f3004c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fe4006580 0x7f3000152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:57.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.364+0000 7f3004c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f300009a6b0 0x7f3000150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:57.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.364+0000 7f3004c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f300009aac0 0x7f300015d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:57.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.364+0000 7f3004c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3000073950 con 0x7f2fe4006580 2024-04-25T14:04:57.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.364+0000 7f3004c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f30000749e0 con 0x7f300009a6b0 2024-04-25T14:04:57.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.364+0000 7f3004c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3000073ad0 con 0x7f300009aac0 2024-04-25T14:04:57.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.364+0000 7f2ffde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fe4006580 0x7f3000152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:57.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.364+0000 7f2ffde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fe4006580 0x7f3000152bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52398/0 (socket says 172.21.15.86:52398) 2024-04-25T14:04:57.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.364+0000 7f2ffde00640 1 -- 172.21.15.86:0/2800826165 learned_addr learned my addr 172.21.15.86:0/2800826165 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:57.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.368+0000 7f2ffd400640 1 --2- 172.21.15.86:0/2800826165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f300009a6b0 0x7f3000150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:57.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.368+0000 7f2ffe800640 1 --2- 172.21.15.86:0/2800826165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f300009aac0 0x7f300015d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:57.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.368+0000 7f2ffde00640 1 -- 172.21.15.86:0/2800826165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f300009aac0 msgr2=0x7f300015d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:57.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.368+0000 7f2ffde00640 1 --2- 172.21.15.86:0/2800826165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f300009aac0 0x7f300015d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.368+0000 7f2ffde00640 1 -- 172.21.15.86:0/2800826165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f300009a6b0 msgr2=0x7f3000150a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:57.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.368+0000 7f2ffde00640 1 --2- 172.21.15.86:0/2800826165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f300009a6b0 0x7f3000150a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.368+0000 7f2ffde00640 1 -- 172.21.15.86:0/2800826165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30001517c0 con 0x7f2fe4006580 2024-04-25T14:04:57.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.368+0000 7f2ffde00640 1 --2- 172.21.15.86:0/2800826165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fe4006580 0x7f3000152bc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2ff00038e0 tx=0x7f2ff0003da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:57.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.368+0000 7f2ff6000640 1 -- 172.21.15.86:0/2800826165 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0009c70 con 0x7f2fe4006580 2024-04-25T14:04:57.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.368+0000 7f2ff6000640 1 -- 172.21.15.86:0/2800826165 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff0020070 con 0x7f2fe4006580 2024-04-25T14:04:57.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.368+0000 7f3004c00640 1 -- 172.21.15.86:0/2800826165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30001520e0 con 0x7f2fe4006580 2024-04-25T14:04:57.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.368+0000 7f2ff6000640 1 -- 172.21.15.86:0/2800826165 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff001b680 con 0x7f2fe4006580 2024-04-25T14:04:57.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.368+0000 7f3004c00640 1 -- 172.21.15.86:0/2800826165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f300015dbb0 con 0x7f2fe4006580 2024-04-25T14:04:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.368+0000 7f3004c00640 1 -- 172.21.15.86:0/2800826165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3000050750 con 0x7f2fe4006580 2024-04-25T14:04:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.368+0000 7f2ff6000640 1 -- 172.21.15.86:0/2800826165 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2ff001b820 con 0x7f2fe4006580 2024-04-25T14:04:57.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.368+0000 7f2ff6000640 1 --2- 172.21.15.86:0/2800826165 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2fe006f400 0x7f2fe0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:57.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.372+0000 7f2ff6000640 1 -- 172.21.15.86:0/2800826165 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6383+0+0 (secure 0 0 0) 0x7f2ff0093f30 con 0x7f2fe4006580 2024-04-25T14:04:57.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.372+0000 7f2ffd400640 1 --2- 172.21.15.86:0/2800826165 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2fe006f400 0x7f2fe0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:57.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.372+0000 7f2ffd400640 1 --2- 172.21.15.86:0/2800826165 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2fe006f400 0x7f2fe0071910 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2fec001750 tx=0x7f2fec0011a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:57.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.376+0000 7f2ff6000640 1 -- 172.21.15.86:0/2800826165 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2ff0016050 con 0x7f2fe4006580 2024-04-25T14:04:57.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:57.532+0000 7f3004c00640 1 -- 172.21.15.86:0/2800826165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f30000b3d30 con 0x7f2fe4006580 2024-04-25T14:04:58.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.180+0000 7f2ff6000640 1 -- 172.21.15.86:0/2800826165 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) ==== 169+0+0 (secure 0 0 0) 0x7f2ff0060c30 con 0x7f2fe4006580 2024-04-25T14:04:58.182 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-04-25T14:04:58.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.184+0000 7f3004c00640 1 -- 172.21.15.86:0/2800826165 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2fe006f400 msgr2=0x7f2fe0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:58.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.184+0000 7f3004c00640 1 --2- 172.21.15.86:0/2800826165 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2fe006f400 0x7f2fe0071910 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2fec001750 tx=0x7f2fec0011a0 comp rx=0 tx=0).stop 2024-04-25T14:04:58.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.184+0000 7f3004c00640 1 -- 172.21.15.86:0/2800826165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fe4006580 msgr2=0x7f3000152bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:58.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.184+0000 7f3004c00640 1 --2- 172.21.15.86:0/2800826165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fe4006580 0x7f3000152bc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2ff00038e0 tx=0x7f2ff0003da0 comp rx=0 tx=0).stop 2024-04-25T14:04:58.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.188+0000 7f3004c00640 1 -- 172.21.15.86:0/2800826165 shutdown_connections 2024-04-25T14:04:58.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.188+0000 7f3004c00640 1 --2- 172.21.15.86:0/2800826165 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f2fe006f400 0x7f2fe0071910 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:58.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.188+0000 7f3004c00640 1 --2- 172.21.15.86:0/2800826165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f300009aac0 0x7f300015d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:58.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.188+0000 7f3004c00640 1 --2- 172.21.15.86:0/2800826165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f300009a6b0 0x7f3000150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:58.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.188+0000 7f3004c00640 1 --2- 172.21.15.86:0/2800826165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fe4006580 0x7f3000152bc0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:58.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.188+0000 7f3004c00640 1 -- 172.21.15.86:0/2800826165 >> 172.21.15.86:0/2800826165 conn(0x7f30000b4a40 msgr2=0x7f3000159ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:58.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.188+0000 7f3004c00640 1 -- 172.21.15.86:0/2800826165 shutdown_connections 2024-04-25T14:04:58.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.188+0000 7f3004c00640 1 -- 172.21.15.86:0/2800826165 wait complete. 2024-04-25T14:04:58.220 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-25T14:04:58.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.368+0000 7f3ed1800640 1 Processor -- start 2024-04-25T14:04:58.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.368+0000 7f3ed1800640 1 -- start start 2024-04-25T14:04:58.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.372+0000 7f3ed1800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3ecc0701f0 0x7f3ecc0729f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:58.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.372+0000 7f3ed1800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ecc070600 0x7f3ecc072f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:58.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.372+0000 7f3ed1800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ecc06f430 con 0x7f3ecc06c670 2024-04-25T14:04:58.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.372+0000 7f3ed1800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3ecc06f1e0 con 0x7f3ecc0701f0 2024-04-25T14:04:58.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.372+0000 7f3ed1800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3ecc06f710 con 0x7f3ecc070600 2024-04-25T14:04:58.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.372+0000 7f3ecaa00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3ecc0701f0 0x7f3ecc0729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:58.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.372+0000 7f3eca000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ecc070600 0x7f3ecc072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:58.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.372+0000 7f3ecaa00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3ecc0701f0 0x7f3ecc0729f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:54246/0 (socket says 172.21.15.86:54246) 2024-04-25T14:04:58.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.372+0000 7f3ecaa00640 1 -- 172.21.15.86:0/3235032271 learned_addr learned my addr 172.21.15.86:0/3235032271 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:58.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.372+0000 7f3ec9600640 1 -- 172.21.15.86:0/3235032271 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 514150895 0 0) 0x7f3ecc06f430 con 0x7f3ecc06c670 2024-04-25T14:04:58.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.372+0000 7f3ec9600640 1 -- 172.21.15.86:0/3235032271 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3eb0003530 con 0x7f3ecc06c670 2024-04-25T14:04:58.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.372+0000 7f3ecaa00640 1 -- 172.21.15.86:0/3235032271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ecc070600 msgr2=0x7f3ecc072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:58.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.372+0000 7f3ecaa00640 1 --2- 172.21.15.86:0/3235032271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ecc070600 0x7f3ecc072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:58.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.372+0000 7f3ecaa00640 1 -- 172.21.15.86:0/3235032271 >> v1:172.21.15.155:6789/0 conn(0x7f3ecc06c670 legacy=0x7f3ecc06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:58.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.372+0000 7f3ecaa00640 1 -- 172.21.15.86:0/3235032271 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ecc06f9f0 con 0x7f3ecc0701f0 2024-04-25T14:04:58.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.372+0000 7f3eca000640 1 --2- 172.21.15.86:0/3235032271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ecc070600 0x7f3ecc072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:58.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.376+0000 7f3ecaa00640 1 --2- 172.21.15.86:0/3235032271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ecc0701f0 0x7f3ecc0729f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009870 tx=0x7f3ebc01c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=898766508a811d62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:58.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.376+0000 7f3ec9600640 1 -- 172.21.15.86:0/3235032271 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc0b0070 con 0x7f3ecc0701f0 2024-04-25T14:04:58.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.376+0000 7f3ec9600640 1 -- 172.21.15.86:0/3235032271 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ebc0ab600 con 0x7f3ecc0701f0 2024-04-25T14:04:58.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.376+0000 7f3ec9600640 1 -- 172.21.15.86:0/3235032271 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc0ab9d0 con 0x7f3ecc0701f0 2024-04-25T14:04:58.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.376+0000 7f3ed1800640 1 -- 172.21.15.86:0/3235032271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ecc0701f0 msgr2=0x7f3ecc0729f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:58.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.376+0000 7f3ed1800640 1 --2- 172.21.15.86:0/3235032271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ecc0701f0 0x7f3ecc0729f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009870 tx=0x7f3ebc01c4d0 comp rx=0 tx=0).stop 2024-04-25T14:04:58.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.376+0000 7f3ed1800640 1 -- 172.21.15.86:0/3235032271 shutdown_connections 2024-04-25T14:04:58.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.376+0000 7f3ed1800640 1 --2- 172.21.15.86:0/3235032271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ecc070600 0x7f3ecc072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:58.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.376+0000 7f3ed1800640 1 --2- 172.21.15.86:0/3235032271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ecc0701f0 0x7f3ecc0729f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:58.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.376+0000 7f3ed1800640 1 -- 172.21.15.86:0/3235032271 >> 172.21.15.86:0/3235032271 conn(0x7f3ecc0b4a40 msgr2=0x7f3ecc0b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:58.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.376+0000 7f3ed1800640 1 -- 172.21.15.86:0/3235032271 shutdown_connections 2024-04-25T14:04:58.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.376+0000 7f3ed1800640 1 -- 172.21.15.86:0/3235032271 wait complete. 2024-04-25T14:04:58.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.376+0000 7f3ed1800640 1 Processor -- start 2024-04-25T14:04:58.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.376+0000 7f3ed1800640 1 -- start start 2024-04-25T14:04:58.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.380+0000 7f3ed1800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ecc06c670 0x7f3ecc14a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:58.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.380+0000 7f3ed1800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ecc0701f0 0x7f3ecc09ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:58.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.380+0000 7f3ed1800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ecc070600 0x7f3ecc15fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:58.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.380+0000 7f3ed1800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc168f30 con 0x7f3ecc06c670 2024-04-25T14:04:58.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.380+0000 7f3ed1800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc15d580 con 0x7f3ecc070600 2024-04-25T14:04:58.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.380+0000 7f3ed1800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3ecc1690b0 con 0x7f3ecc0701f0 2024-04-25T14:04:58.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.380+0000 7f3ecaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ecc06c670 0x7f3ecc14a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:58.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.380+0000 7f3ecaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ecc06c670 0x7f3ecc14a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52406/0 (socket says 172.21.15.86:52406) 2024-04-25T14:04:58.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.380+0000 7f3ecaa00640 1 -- 172.21.15.86:0/776496254 learned_addr learned my addr 172.21.15.86:0/776496254 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:58.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.380+0000 7f3eca000640 1 --2- 172.21.15.86:0/776496254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ecc0701f0 0x7f3ecc09ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:58.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.380+0000 7f3ecb400640 1 --2- 172.21.15.86:0/776496254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ecc070600 0x7f3ecc15fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:58.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.380+0000 7f3ecaa00640 1 -- 172.21.15.86:0/776496254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ecc0701f0 msgr2=0x7f3ecc09ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:58.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.380+0000 7f3ecaa00640 1 --2- 172.21.15.86:0/776496254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ecc0701f0 0x7f3ecc09ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:58.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.380+0000 7f3ecaa00640 1 -- 172.21.15.86:0/776496254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ecc070600 msgr2=0x7f3ecc15fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:58.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.380+0000 7f3ecaa00640 1 --2- 172.21.15.86:0/776496254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ecc070600 0x7f3ecc15fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:58.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.380+0000 7f3ecaa00640 1 -- 172.21.15.86:0/776496254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ecc14ca90 con 0x7f3ecc06c670 2024-04-25T14:04:58.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.380+0000 7f3ecb400640 1 --2- 172.21.15.86:0/776496254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ecc070600 0x7f3ecc15fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:58.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.380+0000 7f3ecaa00640 1 --2- 172.21.15.86:0/776496254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ecc06c670 0x7f3ecc14a610 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0ae040 tx=0x7f3ebc019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:58.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.380+0000 7f3ec2a00640 1 -- 172.21.15.86:0/776496254 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc0b0040 con 0x7f3ecc06c670 2024-04-25T14:04:58.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.380+0000 7f3ec2a00640 1 -- 172.21.15.86:0/776496254 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ebc0b9070 con 0x7f3ecc06c670 2024-04-25T14:04:58.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.380+0000 7f3ed1800640 1 -- 172.21.15.86:0/776496254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ecc14cf00 con 0x7f3ecc06c670 2024-04-25T14:04:58.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.380+0000 7f3ec2a00640 1 -- 172.21.15.86:0/776496254 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc0aa970 con 0x7f3ecc06c670 2024-04-25T14:04:58.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.380+0000 7f3ed1800640 1 -- 172.21.15.86:0/776496254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ecc14d4c0 con 0x7f3ecc06c670 2024-04-25T14:04:58.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.380+0000 7f3ed1800640 1 -- 172.21.15.86:0/776496254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ecc050750 con 0x7f3ecc06c670 2024-04-25T14:04:58.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.384+0000 7f3ec2a00640 1 -- 172.21.15.86:0/776496254 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3ebc0aab10 con 0x7f3ecc06c670 2024-04-25T14:04:58.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.384+0000 7f3ec2a00640 1 --2- 172.21.15.86:0/776496254 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3eac06f400 0x7f3eac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:58.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.384+0000 7f3ec2a00640 1 -- 172.21.15.86:0/776496254 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5853+0+0 (secure 0 0 0) 0x7f3ebc129270 con 0x7f3ecc06c670 2024-04-25T14:04:58.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.384+0000 7f3eca000640 1 --2- 172.21.15.86:0/776496254 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3eac06f400 0x7f3eac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:58.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.384+0000 7f3eca000640 1 --2- 172.21.15.86:0/776496254 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3eac06f400 0x7f3eac071910 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc158bc0 tx=0x7f3eb800a2a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.388+0000 7f3ec2a00640 1 -- 172.21.15.86:0/776496254 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3ebc016150 con 0x7f3ecc06c670 2024-04-25T14:04:58.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:58.544+0000 7f3ed1800640 1 -- 172.21.15.86:0/776496254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3ecc069870 con 0x7f3ecc06c670 2024-04-25T14:04:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.196+0000 7f3ec2a00640 1 -- 172.21.15.86:0/776496254 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) ==== 178+0+0 (secure 0 0 0) 0x7f3ebc0f6150 con 0x7f3ecc06c670 2024-04-25T14:04:59.198 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data_ec' removed 2024-04-25T14:04:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.196+0000 7f3ed1800640 1 -- 172.21.15.86:0/776496254 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3eac06f400 msgr2=0x7f3eac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.196+0000 7f3ed1800640 1 --2- 172.21.15.86:0/776496254 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3eac06f400 0x7f3eac071910 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc158bc0 tx=0x7f3eb800a2a0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.196+0000 7f3ed1800640 1 -- 172.21.15.86:0/776496254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ecc06c670 msgr2=0x7f3ecc14a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.196+0000 7f3ed1800640 1 --2- 172.21.15.86:0/776496254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ecc06c670 0x7f3ecc14a610 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0ae040 tx=0x7f3ebc019270 comp rx=0 tx=0).stop 2024-04-25T14:04:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.196+0000 7f3ed1800640 1 -- 172.21.15.86:0/776496254 shutdown_connections 2024-04-25T14:04:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.196+0000 7f3ed1800640 1 --2- 172.21.15.86:0/776496254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ecc070600 0x7f3ecc15fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.196+0000 7f3ed1800640 1 --2- 172.21.15.86:0/776496254 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f3eac06f400 0x7f3eac071910 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.196+0000 7f3ed1800640 1 --2- 172.21.15.86:0/776496254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ecc0701f0 0x7f3ecc09ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.196+0000 7f3ed1800640 1 --2- 172.21.15.86:0/776496254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ecc06c670 0x7f3ecc14a610 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.196+0000 7f3ed1800640 1 -- 172.21.15.86:0/776496254 >> 172.21.15.86:0/776496254 conn(0x7f3ecc0b4a40 msgr2=0x7f3ecc0739f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:59.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.196+0000 7f3ed1800640 1 -- 172.21.15.86:0/776496254 shutdown_connections 2024-04-25T14:04:59.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.196+0000 7f3ed1800640 1 -- 172.21.15.86:0/776496254 wait complete. 2024-04-25T14:04:59.231 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T14:04:59.231 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T14:04:59.231 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T14:04:59.231 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:04:59.232 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T14:04:59.232 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T14:04:59.232 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T14:04:59.232 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:04:59.232 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T14:04:59.232 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T14:04:59.232 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T14:04:59.233 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:04:59.233 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T14:04:59.233 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T14:04:59.233 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T14:04:59.233 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:04:59.245 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T14:04:59.245 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T14:04:59.247 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T14:04:59.247 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T14:04:59.248 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T14:04:59.248 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T14:04:59.255 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T14:04:59.256 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T14:04:59.290 INFO:tasks.ceph.mds.a:Started 2024-04-25T14:04:59.291 DEBUG:teuthology.parallel:result is None 2024-04-25T14:04:59.293 INFO:tasks.ceph.mds.c:Started 2024-04-25T14:04:59.293 DEBUG:teuthology.parallel:result is None 2024-04-25T14:04:59.298 INFO:tasks.ceph.mds.b:Started 2024-04-25T14:04:59.298 DEBUG:teuthology.parallel:result is None 2024-04-25T14:04:59.300 INFO:tasks.ceph.mds.d:Started 2024-04-25T14:04:59.300 DEBUG:teuthology.parallel:result is None 2024-04-25T14:04:59.301 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T14:04:59.395 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-25T14:04:59.396 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-04-25T14:04:59.416 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2024-04-25T14:04:59.416 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2024-04-25T14:04:59.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.460+0000 7fc5d5000640 1 Processor -- start 2024-04-25T14:04:59.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.460+0000 7fc5d5000640 1 -- start start 2024-04-25T14:04:59.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.460+0000 7fc5d5000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc5d01651d0 0x7fc5d01675f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:59.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.460+0000 7fc5d5000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5d00b0a60 con 0x7fc5d006fd50 2024-04-25T14:04:59.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.460+0000 7fc5d5000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5d00ae840 con 0x7fc5d015c2a0 2024-04-25T14:04:59.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.460+0000 7fc5d5000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc5d006ef90 con 0x7fc5d01651d0 2024-04-25T14:04:59.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.460+0000 7fc5c5400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc5d006fd50 0x7fc5d0070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52086/0 (socket says 172.21.15.86:52086) 2024-04-25T14:04:59.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.460+0000 7fc5c5400640 1 -- 172.21.15.86:0/68153033 learned_addr learned my addr 172.21.15.86:0/68153033 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:04:59.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.460+0000 7fc5ce800640 1 --2- 172.21.15.86:0/68153033 >> v2:172.21.15.155:3300/0 conn(0x7fc5d01651d0 0x7fc5d01675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:59.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.460+0000 7fc5cd400640 1 -- 172.21.15.86:0/68153033 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1191459344 0 0) 0x7fc5d00b0a60 con 0x7fc5d006fd50 2024-04-25T14:04:59.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.460+0000 7fc5cd400640 1 -- 172.21.15.86:0/68153033 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5b4003530 con 0x7fc5d006fd50 2024-04-25T14:04:59.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.460+0000 7fc5ce800640 1 -- 172.21.15.86:0/68153033 >> v1:172.21.15.155:6789/0 conn(0x7fc5d015c2a0 legacy=0x7fc5d00ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:59.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.460+0000 7fc5ce800640 1 -- 172.21.15.86:0/68153033 >> v1:172.21.15.86:6789/0 conn(0x7fc5d006fd50 legacy=0x7fc5d0070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:59.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.460+0000 7fc5cd400640 1 -- 172.21.15.86:0/68153033 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3751438157 0 0) 0x7fc5b4003530 con 0x7fc5d006fd50 2024-04-25T14:04:59.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.464+0000 7fc5ce800640 1 -- 172.21.15.86:0/68153033 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5d006f270 con 0x7fc5d01651d0 2024-04-25T14:04:59.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.464+0000 7fc5ce800640 1 --2- 172.21.15.86:0/68153033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5d01651d0 0x7fc5d01675f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8009d90 tx=0x7fc5c801c7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae7798f021d7c586 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:59.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.464+0000 7fc5cd400640 1 -- 172.21.15.86:0/68153033 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c80b2070 con 0x7fc5d01651d0 2024-04-25T14:04:59.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.464+0000 7fc5cd400640 1 -- 172.21.15.86:0/68153033 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5c80a5610 con 0x7fc5d01651d0 2024-04-25T14:04:59.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.464+0000 7fc5cd400640 1 -- 172.21.15.86:0/68153033 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8001680 con 0x7fc5d01651d0 2024-04-25T14:04:59.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.468+0000 7fc5d5000640 1 -- 172.21.15.86:0/68153033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5d01651d0 msgr2=0x7fc5d01675f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:59.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.468+0000 7fc5d5000640 1 --2- 172.21.15.86:0/68153033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5d01651d0 0x7fc5d01675f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8009d90 tx=0x7fc5c801c7d0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.468+0000 7fc5d5000640 1 -- 172.21.15.86:0/68153033 shutdown_connections 2024-04-25T14:04:59.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.468+0000 7fc5d5000640 1 --2- 172.21.15.86:0/68153033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5d01651d0 0x7fc5d01675f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.468+0000 7fc5d5000640 1 -- 172.21.15.86:0/68153033 >> 172.21.15.86:0/68153033 conn(0x7fc5d00b4650 msgr2=0x7fc5d00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:59.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.468+0000 7fc5d5000640 1 -- 172.21.15.86:0/68153033 shutdown_connections 2024-04-25T14:04:59.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.468+0000 7fc5d5000640 1 -- 172.21.15.86:0/68153033 wait complete. 2024-04-25T14:04:59.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.468+0000 7fc5d5000640 1 Processor -- start 2024-04-25T14:04:59.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.468+0000 7fc5d5000640 1 -- start start 2024-04-25T14:04:59.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.472+0000 7fc5d5000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5d006fd50 0x7fc5d0154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:59.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.472+0000 7fc5d5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5d015c2a0 0x7fc5d009a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:59.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.472+0000 7fc5d5000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc5d01651d0 0x7fc5d015c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:59.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.472+0000 7fc5d5000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d015c830 con 0x7fc5d015c2a0 2024-04-25T14:04:59.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.472+0000 7fc5d5000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d015c6b0 con 0x7fc5d006fd50 2024-04-25T14:04:59.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.472+0000 7fc5d5000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc5d00d16e0 con 0x7fc5d01651d0 2024-04-25T14:04:59.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.472+0000 7fc5c5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5d015c2a0 0x7fc5d009a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.472+0000 7fc5c5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5d015c2a0 0x7fc5d009a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35450/0 (socket says 172.21.15.86:35450) 2024-04-25T14:04:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.472+0000 7fc5c5400640 1 -- 172.21.15.86:0/4133695151 learned_addr learned my addr 172.21.15.86:0/4133695151 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:04:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.472+0000 7fc5cde00640 1 --2- 172.21.15.86:0/4133695151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5d006fd50 0x7fc5d0154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.472+0000 7fc5ce800640 1 --2- 172.21.15.86:0/4133695151 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc5d01651d0 0x7fc5d015c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.472+0000 7fc5c5400640 1 -- 172.21.15.86:0/4133695151 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc5d01651d0 msgr2=0x7fc5d015c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.472+0000 7fc5c5400640 1 --2- 172.21.15.86:0/4133695151 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc5d01651d0 0x7fc5d015c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.472+0000 7fc5c5400640 1 -- 172.21.15.86:0/4133695151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5d006fd50 msgr2=0x7fc5d0154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.472+0000 7fc5c5400640 1 --2- 172.21.15.86:0/4133695151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5d006fd50 0x7fc5d0154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.472+0000 7fc5c5400640 1 -- 172.21.15.86:0/4133695151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5d0155710 con 0x7fc5d015c2a0 2024-04-25T14:04:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.472+0000 7fc5c5400640 1 --2- 172.21.15.86:0/4133695151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5d015c2a0 0x7fc5d009a720 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc5b800c560 tx=0x7fc5b800ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.472+0000 7fc5c6a00640 1 -- 172.21.15.86:0/4133695151 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8017070 con 0x7fc5d015c2a0 2024-04-25T14:04:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.472+0000 7fc5c6a00640 1 -- 172.21.15.86:0/4133695151 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5b8005cf0 con 0x7fc5d015c2a0 2024-04-25T14:04:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.472+0000 7fc5c6a00640 1 -- 172.21.15.86:0/4133695151 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b80072b0 con 0x7fc5d015c2a0 2024-04-25T14:04:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.472+0000 7fc5d5000640 1 -- 172.21.15.86:0/4133695151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5d01559f0 con 0x7fc5d015c2a0 2024-04-25T14:04:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.472+0000 7fc5d5000640 1 -- 172.21.15.86:0/4133695151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5d0155fb0 con 0x7fc5d015c2a0 2024-04-25T14:04:59.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.472+0000 7fc5d5000640 1 -- 172.21.15.86:0/4133695151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5d0155480 con 0x7fc5d015c2a0 2024-04-25T14:04:59.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.472+0000 7fc5c6a00640 1 -- 172.21.15.86:0/4133695151 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc5b801a070 con 0x7fc5d015c2a0 2024-04-25T14:04:59.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.476+0000 7fc5c6a00640 1 --2- 172.21.15.86:0/4133695151 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fc5ac06f430 0x7fc5ac071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:59.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.476+0000 7fc5c6a00640 1 -- 172.21.15.86:0/4133695151 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5335+0+0 (secure 0 0 0) 0x7fc5b808e640 con 0x7fc5d015c2a0 2024-04-25T14:04:59.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.476+0000 7fc5cde00640 1 --2- 172.21.15.86:0/4133695151 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fc5ac06f430 0x7fc5ac071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:59.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.476+0000 7fc5cde00640 1 --2- 172.21.15.86:0/4133695151 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fc5ac06f430 0x7fc5ac071940 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0007fd0 tx=0x7fc5c0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:59.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.480+0000 7fc5c6a00640 1 -- 172.21.15.86:0/4133695151 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc5b805af30 con 0x7fc5d015c2a0 2024-04-25T14:04:59.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:04:59.652+0000 7fc5d5000640 1 -- 172.21.15.86:0/4133695151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fc5d0069d00 con 0x7fc5d015c2a0 2024-04-25T14:05:00.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.212+0000 7fc5c6a00640 1 -- 172.21.15.86:0/4133695151 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) ==== 97+0+0 (secure 0 0 0) 0x7fc5b805f230 con 0x7fc5d015c2a0 2024-04-25T14:05:00.216 INFO:teuthology.orchestra.run.smithi086.stderr: removed all blocklist entries 2024-04-25T14:05:00.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.216+0000 7fc5d5000640 1 -- 172.21.15.86:0/4133695151 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fc5ac06f430 msgr2=0x7fc5ac071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:00.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.216+0000 7fc5d5000640 1 --2- 172.21.15.86:0/4133695151 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fc5ac06f430 0x7fc5ac071940 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0007fd0 tx=0x7fc5c0009040 comp rx=0 tx=0).stop 2024-04-25T14:05:00.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.216+0000 7fc5d5000640 1 -- 172.21.15.86:0/4133695151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5d015c2a0 msgr2=0x7fc5d009a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:00.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.216+0000 7fc5d5000640 1 --2- 172.21.15.86:0/4133695151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5d015c2a0 0x7fc5d009a720 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc5b800c560 tx=0x7fc5b800ca20 comp rx=0 tx=0).stop 2024-04-25T14:05:00.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.216+0000 7fc5d5000640 1 -- 172.21.15.86:0/4133695151 shutdown_connections 2024-04-25T14:05:00.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.216+0000 7fc5d5000640 1 --2- 172.21.15.86:0/4133695151 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fc5ac06f430 0x7fc5ac071940 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:00.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.216+0000 7fc5d5000640 1 --2- 172.21.15.86:0/4133695151 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc5d01651d0 0x7fc5d015c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:00.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.216+0000 7fc5d5000640 1 --2- 172.21.15.86:0/4133695151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5d015c2a0 0x7fc5d009a720 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:00.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.216+0000 7fc5d5000640 1 --2- 172.21.15.86:0/4133695151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5d006fd50 0x7fc5d0154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:00.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.216+0000 7fc5d5000640 1 -- 172.21.15.86:0/4133695151 >> 172.21.15.86:0/4133695151 conn(0x7fc5d00b4650 msgr2=0x7fc5d0073bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:00.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.216+0000 7fc5d5000640 1 -- 172.21.15.86:0/4133695151 shutdown_connections 2024-04-25T14:05:00.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.216+0000 7fc5d5000640 1 -- 172.21.15.86:0/4133695151 wait complete. 2024-04-25T14:05:00.246 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T14:05:00.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.388+0000 7f1fb1400640 1 Processor -- start 2024-04-25T14:05:00.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.392+0000 7f1fb1400640 1 -- start start 2024-04-25T14:05:00.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.392+0000 7f1fb1400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1fac1642a0 0x7f1fac0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:00.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.392+0000 7f1fb1400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fac0d1c20 0x7f1fac0b0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:00.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.392+0000 7f1fb1400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fac0d2030 con 0x7f1fac0b0690 2024-04-25T14:05:00.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.392+0000 7f1fb1400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1fac0cea30 con 0x7f1fac1642a0 2024-04-25T14:05:00.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.392+0000 7f1fb1400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1fac0cde20 con 0x7f1fac0d1c20 2024-04-25T14:05:00.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.392+0000 7f1faa800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1fac1642a0 0x7f1fac0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:00.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.392+0000 7f1faa800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1fac1642a0 0x7f1fac0d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35466/0 (socket says 172.21.15.86:35466) 2024-04-25T14:05:00.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.392+0000 7f1faa800640 1 -- 172.21.15.86:0/1071730045 learned_addr learned my addr 172.21.15.86:0/1071730045 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:00.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.392+0000 7f1fa1e00640 1 --2- 172.21.15.86:0/1071730045 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fac0d1c20 0x7f1fac0b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:00.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.392+0000 7f1faa800640 1 -- 172.21.15.86:0/1071730045 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fac0d1c20 msgr2=0x7f1fac0b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:00.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.392+0000 7f1faa800640 1 --2- 172.21.15.86:0/1071730045 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fac0d1c20 0x7f1fac0b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:00.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.392+0000 7f1faa800640 1 -- 172.21.15.86:0/1071730045 >> v1:172.21.15.155:6789/0 conn(0x7f1fac0b0690 legacy=0x7f1fac0d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:00.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.392+0000 7f1faa800640 1 -- 172.21.15.86:0/1071730045 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fac0ce100 con 0x7f1fac1642a0 2024-04-25T14:05:00.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.396+0000 7f1faa800640 1 --2- 172.21.15.86:0/1071730045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fac1642a0 0x7f1fac0d16e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c009870 tx=0x7f1f9c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1806ad0c5cb3acd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:00.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.396+0000 7f1fa9e00640 1 -- 172.21.15.86:0/1071730045 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c0b0070 con 0x7f1fac1642a0 2024-04-25T14:05:00.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.396+0000 7f1fa9e00640 1 -- 172.21.15.86:0/1071730045 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f9c0ab600 con 0x7f1fac1642a0 2024-04-25T14:05:00.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.396+0000 7f1fa9e00640 1 -- 172.21.15.86:0/1071730045 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c0ab9d0 con 0x7f1fac1642a0 2024-04-25T14:05:00.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.396+0000 7f1fb1400640 1 -- 172.21.15.86:0/1071730045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fac1642a0 msgr2=0x7f1fac0d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:00.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.396+0000 7f1fb1400640 1 --2- 172.21.15.86:0/1071730045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fac1642a0 0x7f1fac0d16e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c009870 tx=0x7f1f9c01c4d0 comp rx=0 tx=0).stop 2024-04-25T14:05:00.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.396+0000 7f1fb1400640 1 -- 172.21.15.86:0/1071730045 shutdown_connections 2024-04-25T14:05:00.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.396+0000 7f1fb1400640 1 --2- 172.21.15.86:0/1071730045 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fac0d1c20 0x7f1fac0b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:00.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.396+0000 7f1fb1400640 1 --2- 172.21.15.86:0/1071730045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fac1642a0 0x7f1fac0d16e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:00.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.396+0000 7f1fb1400640 1 -- 172.21.15.86:0/1071730045 >> 172.21.15.86:0/1071730045 conn(0x7f1fac0b4650 msgr2=0x7f1fac0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:00.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.396+0000 7f1fb1400640 1 -- 172.21.15.86:0/1071730045 shutdown_connections 2024-04-25T14:05:00.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.396+0000 7f1fb1400640 1 -- 172.21.15.86:0/1071730045 wait complete. 2024-04-25T14:05:00.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.400+0000 7f1fb1400640 1 Processor -- start 2024-04-25T14:05:00.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.400+0000 7f1fb1400640 1 -- start start 2024-04-25T14:05:00.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.400+0000 7f1fb1400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fac0b0690 0x7f1fac099250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:00.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.400+0000 7f1fb1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fac0d1c20 0x7f1fac074390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:00.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.400+0000 7f1fb1400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fac1642a0 0x7f1fac0748d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:00.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.400+0000 7f1fb1400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1fac1646b0 con 0x7f1fac0d1c20 2024-04-25T14:05:00.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.400+0000 7f1fb1400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1fac0d1160 con 0x7f1fac1642a0 2024-04-25T14:05:00.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.400+0000 7f1fb1400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1fac164830 con 0x7f1fac0b0690 2024-04-25T14:05:00.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.400+0000 7f1fa1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fac0d1c20 0x7f1fac074390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:00.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.400+0000 7f1fa1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fac0d1c20 0x7f1fac074390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35478/0 (socket says 172.21.15.86:35478) 2024-04-25T14:05:00.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.400+0000 7f1fa1e00640 1 -- 172.21.15.86:0/1436849548 learned_addr learned my addr 172.21.15.86:0/1436849548 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:00.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.400+0000 7f1faa800640 1 --2- 172.21.15.86:0/1436849548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fac0b0690 0x7f1fac099250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:00.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.400+0000 7f1fab200640 1 --2- 172.21.15.86:0/1436849548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fac1642a0 0x7f1fac0748d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:00.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.400+0000 7f1fa1e00640 1 -- 172.21.15.86:0/1436849548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fac0b0690 msgr2=0x7f1fac099250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:00.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.400+0000 7f1fa1e00640 1 --2- 172.21.15.86:0/1436849548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fac0b0690 0x7f1fac099250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:00.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.400+0000 7f1fa1e00640 1 -- 172.21.15.86:0/1436849548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fac1642a0 msgr2=0x7f1fac0748d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:00.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.400+0000 7f1fa1e00640 1 --2- 172.21.15.86:0/1436849548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fac1642a0 0x7f1fac0748d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:00.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.400+0000 7f1fa1e00640 1 -- 172.21.15.86:0/1436849548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fac099790 con 0x7f1fac0d1c20 2024-04-25T14:05:00.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.400+0000 7f1fa1e00640 1 --2- 172.21.15.86:0/1436849548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fac0d1c20 0x7f1fac074390 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400e8a0 tx=0x7f1f9400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:00.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.400+0000 7f1fa3400640 1 -- 172.21.15.86:0/1436849548 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94017070 con 0x7f1fac0d1c20 2024-04-25T14:05:00.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.400+0000 7f1fa3400640 1 -- 172.21.15.86:0/1436849548 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f9400c940 con 0x7f1fac0d1c20 2024-04-25T14:05:00.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.400+0000 7f1fa3400640 1 -- 172.21.15.86:0/1436849548 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9400ccf0 con 0x7f1fac0d1c20 2024-04-25T14:05:00.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.400+0000 7f1fb1400640 1 -- 172.21.15.86:0/1436849548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fac151f30 con 0x7f1fac0d1c20 2024-04-25T14:05:00.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.400+0000 7f1fb1400640 1 -- 172.21.15.86:0/1436849548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fac1524f0 con 0x7f1fac0d1c20 2024-04-25T14:05:00.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.404+0000 7f1fb1400640 1 -- 172.21.15.86:0/1436849548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fac153920 con 0x7f1fac0d1c20 2024-04-25T14:05:00.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.404+0000 7f1fa3400640 1 -- 172.21.15.86:0/1436849548 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1f9401a070 con 0x7f1fac0d1c20 2024-04-25T14:05:00.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.404+0000 7f1fa3400640 1 --2- 172.21.15.86:0/1436849548 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1f8806f430 0x7f1f88071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:00.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.404+0000 7f1fa3400640 1 -- 172.21.15.86:0/1436849548 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f1f9408ef60 con 0x7f1fac0d1c20 2024-04-25T14:05:00.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.404+0000 7f1faa800640 1 --2- 172.21.15.86:0/1436849548 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1f8806f430 0x7f1f88071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:00.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.404+0000 7f1faa800640 1 --2- 172.21.15.86:0/1436849548 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1f8806f430 0x7f1f88071940 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c0ad000 tx=0x7f1f9c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:00.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.408+0000 7f1fa3400640 1 -- 172.21.15.86:0/1436849548 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1f9405b9a0 con 0x7f1fac0d1c20 2024-04-25T14:05:00.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.620+0000 7f1fb1400640 1 -- 172.21.15.86:0/1436849548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1fac069d00 con 0x7f1fac0d1c20 2024-04-25T14:05:00.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.620+0000 7f1fa3400640 1 -- 172.21.15.86:0/1436849548 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) ==== 80+0+6645 (secure 0 0 0) 0x7f1f9405fca0 con 0x7f1fac0d1c20 2024-04-25T14:05:00.622 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:00.622 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:05:00.622 INFO:teuthology.orchestra.run.smithi086.stdout: "auth_dump": [ 2024-04-25T14:05:00.622 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:00.622 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.a", 2024-04-25T14:05:00.623 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmWk5fAxAAKEBKhWewpl23MUcvMi8qDg==", 2024-04-25T14:05:00.623 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:05:00.623 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:05:00.623 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:05:00.623 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:05:00.623 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:05:00.623 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:00.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:05:00.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:00.623 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.b", 2024-04-25T14:05:00.623 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmDhzOEBAABNys/NAEhucQ7A+HzQlzQw==", 2024-04-25T14:05:00.623 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:05:00.623 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:05:00.623 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:05:00.624 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:05:00.624 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:05:00.624 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:00.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:05:00.624 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:00.624 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.c", 2024-04-25T14:05:00.624 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmA/hNCxAAclDGbIjLfjqyXn0MwX/IkA==", 2024-04-25T14:05:00.624 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:05:00.624 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:05:00.624 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:05:00.624 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:05:00.624 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:05:00.624 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:00.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:05:00.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:00.625 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.d", 2024-04-25T14:05:00.625 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmemCcGBAAGrCx6C850qRJSWYjebxOBA==", 2024-04-25T14:05:00.625 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:05:00.625 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:05:00.625 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:05:00.625 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:05:00.625 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:05:00.625 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:00.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:05:00.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:00.625 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.0", 2024-04-25T14:05:00.625 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCOYipm9etgGxAAemg9CqXN0ShduDtDuKF2Zw==", 2024-04-25T14:05:00.625 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:05:00.626 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:05:00.626 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:05:00.626 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:05:00.626 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:00.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:05:00.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:00.626 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.1", 2024-04-25T14:05:00.626 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCRYipmBCiNJhAABAWHk8NEGVvqieL2klGhmA==", 2024-04-25T14:05:00.626 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:05:00.626 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:05:00.626 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:05:00.626 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:05:00.626 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:00.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:05:00.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:00.627 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.2", 2024-04-25T14:05:00.627 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCUYipm/XGYLxAAPXsw1svKseFzFiuDUjAsJQ==", 2024-04-25T14:05:00.627 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:05:00.627 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:05:00.627 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:05:00.627 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:05:00.627 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:00.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:05:00.627 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:00.627 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.3", 2024-04-25T14:05:00.627 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCXYipmlSRaORAAwpKyzZDdVYHWl2pAsQjmpw==", 2024-04-25T14:05:00.627 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:05:00.627 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:05:00.628 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:05:00.628 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:05:00.628 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:00.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:05:00.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:00.628 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.4", 2024-04-25T14:05:00.628 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCdYipmBI3FMBAAV2NdggfRSFnQvrcOC2YqiA==", 2024-04-25T14:05:00.628 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:05:00.628 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:05:00.628 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:05:00.628 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:05:00.628 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:00.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:05:00.628 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:00.628 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.5", 2024-04-25T14:05:00.629 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQChYipmctYxABAABwmOMgyYuOfRWD9KdHILng==", 2024-04-25T14:05:00.629 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:05:00.629 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:05:00.629 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:05:00.629 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:05:00.629 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:00.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:05:00.629 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:00.629 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.6", 2024-04-25T14:05:00.629 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCkYipmSL7TCRAAhFMWg1AQm54bB+x1GDcusA==", 2024-04-25T14:05:00.629 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:05:00.629 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:05:00.629 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:05:00.629 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:05:00.630 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:00.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:05:00.630 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:00.630 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.7", 2024-04-25T14:05:00.630 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCnYipmZ4F+FRAAEGaZN+byv/qJUFZb7WxTFg==", 2024-04-25T14:05:00.630 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:05:00.630 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:05:00.630 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:05:00.630 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:05:00.630 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:00.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:05:00.630 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:00.630 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.0", 2024-04-25T14:05:00.631 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipm5u4xHRAACDFe3Eb4KLrsdLj4eFe7mA==", 2024-04-25T14:05:00.631 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:05:00.631 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:05:00.631 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-04-25T14:05:00.631 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-04-25T14:05:00.631 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-04-25T14:05:00.631 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:00.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:05:00.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:00.631 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.1", 2024-04-25T14:05:00.631 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmDp1hJRAAQkBc5PdQoMTT7W4HahkTyg==", 2024-04-25T14:05:00.631 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:05:00.631 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:05:00.631 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-04-25T14:05:00.631 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-04-25T14:05:00.632 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-04-25T14:05:00.632 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:00.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:05:00.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:00.632 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.2", 2024-04-25T14:05:00.632 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmhbhGIRAAxao/J0WY53IITLETFgwIVw==", 2024-04-25T14:05:00.632 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:05:00.632 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:05:00.632 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-04-25T14:05:00.632 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-04-25T14:05:00.632 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-04-25T14:05:00.632 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:00.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:05:00.632 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:00.633 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.3", 2024-04-25T14:05:00.633 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmJsCBKRAAzXn2qDtblIpqt7U/TUUlyA==", 2024-04-25T14:05:00.633 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:05:00.633 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:05:00.633 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-04-25T14:05:00.633 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-04-25T14:05:00.633 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-04-25T14:05:00.633 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:00.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:05:00.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:00.633 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.admin", 2024-04-25T14:05:00.633 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCKYipmTqbVIBAA1VpOtj24f2ibVEIT4EJwmw==", 2024-04-25T14:05:00.633 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:05:00.633 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-25T14:05:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:05:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:05:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:05:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:05:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.bootstrap-mds", 2024-04-25T14:05:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCtYipmW6RHKRAAcQ4umDhniSokfWpCXSM3qg==", 2024-04-25T14:05:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:05:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T14:05:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:05:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:00.634 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T14:05:00.635 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCtYipmobRHKRAAiDBNfOPlcoGxSy4wQHTULg==", 2024-04-25T14:05:00.635 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:05:00.635 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T14:05:00.635 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:00.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:05:00.635 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:00.635 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.bootstrap-osd", 2024-04-25T14:05:00.635 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCtYipmisNHKRAAYEd3kyVK66URE8jpGVW1Xw==", 2024-04-25T14:05:00.635 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:05:00.635 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T14:05:00.635 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:00.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:05:00.635 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:00.635 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T14:05:00.636 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCtYipm2NFHKRAAkLF8pORwuyFrS2EXrr8f7Q==", 2024-04-25T14:05:00.636 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:05:00.636 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T14:05:00.636 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:00.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:05:00.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:00.636 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T14:05:00.636 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCtYipm799HKRAA3/7eoeW5KQMznQpUaNMXgQ==", 2024-04-25T14:05:00.636 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:05:00.636 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T14:05:00.636 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:00.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:05:00.636 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:00.636 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T14:05:00.636 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCtYipmsutHKRAAqUY3dESCQO8jKGB3CXBJ+A==", 2024-04-25T14:05:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:05:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T14:05:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:05:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.x", 2024-04-25T14:05:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCKYipmKMzAORAAtRyTNJ5HWi2XwuamoqwB0Q==", 2024-04-25T14:05:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:05:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-25T14:05:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-25T14:05:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:05:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:05:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.y", 2024-04-25T14:05:00.638 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCKYipmC7A7NRAAEwVNA0yxt3fwdHXrsV0dvg==", 2024-04-25T14:05:00.638 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:05:00.638 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-25T14:05:00.638 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-25T14:05:00.638 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:05:00.638 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:00.638 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:00.638 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-25T14:05:00.638 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:05:00.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.620+0000 7f1fb1400640 1 -- 172.21.15.86:0/1436849548 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1f8806f430 msgr2=0x7f1f88071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:00.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.620+0000 7f1fb1400640 1 --2- 172.21.15.86:0/1436849548 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1f8806f430 0x7f1f88071940 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c0ad000 tx=0x7f1f9c008040 comp rx=0 tx=0).stop 2024-04-25T14:05:00.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.620+0000 7f1fb1400640 1 -- 172.21.15.86:0/1436849548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fac0d1c20 msgr2=0x7f1fac074390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:00.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.620+0000 7f1fb1400640 1 --2- 172.21.15.86:0/1436849548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fac0d1c20 0x7f1fac074390 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400e8a0 tx=0x7f1f9400eca0 comp rx=0 tx=0).stop 2024-04-25T14:05:00.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.620+0000 7f1fb1400640 1 -- 172.21.15.86:0/1436849548 shutdown_connections 2024-04-25T14:05:00.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.620+0000 7f1fb1400640 1 --2- 172.21.15.86:0/1436849548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fac1642a0 0x7f1fac0748d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:00.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.620+0000 7f1fb1400640 1 --2- 172.21.15.86:0/1436849548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fac0d1c20 0x7f1fac074390 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:00.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.620+0000 7f1fb1400640 1 --2- 172.21.15.86:0/1436849548 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1f8806f430 0x7f1f88071940 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:00.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.620+0000 7f1fb1400640 1 --2- 172.21.15.86:0/1436849548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fac0b0690 0x7f1fac099250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:00.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.620+0000 7f1fb1400640 1 -- 172.21.15.86:0/1436849548 >> 172.21.15.86:0/1436849548 conn(0x7f1fac0b4650 msgr2=0x7f1fac068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:00.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.620+0000 7f1fb1400640 1 -- 172.21.15.86:0/1436849548 shutdown_connections 2024-04-25T14:05:00.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.620+0000 7f1fb1400640 1 -- 172.21.15.86:0/1436849548 wait complete. 2024-04-25T14:05:00.643 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T14:05:00.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.788+0000 7f041f400640 1 Processor -- start 2024-04-25T14:05:00.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.788+0000 7f041f400640 1 -- start start 2024-04-25T14:05:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.792+0000 7f041f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f04180cfca0 0x7f041806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.792+0000 7f041f400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f04180730b0 0x7f04180734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.792+0000 7f041f400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0418072bd0 con 0x7f04180d16e0 2024-04-25T14:05:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.792+0000 7f041f400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0418070470 con 0x7f04180cfca0 2024-04-25T14:05:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.792+0000 7f041f400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f041806f930 con 0x7f04180730b0 2024-04-25T14:05:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.792+0000 7f041ca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f04180cfca0 0x7f041806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.792+0000 7f041ca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f04180cfca0 0x7f041806f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35482/0 (socket says 172.21.15.86:35482) 2024-04-25T14:05:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.792+0000 7f041ca00640 1 -- 172.21.15.86:0/2117947662 learned_addr learned my addr 172.21.15.86:0/2117947662 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.792+0000 7f041d400640 1 --2- 172.21.15.86:0/2117947662 >> v2:172.21.15.155:3300/0 conn(0x7f04180730b0 0x7f04180734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.792+0000 7f041ca00640 1 -- 172.21.15.86:0/2117947662 >> v2:172.21.15.155:3300/0 conn(0x7f04180730b0 msgr2=0x7f04180734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.792+0000 7f041ca00640 1 --2- 172.21.15.86:0/2117947662 >> v2:172.21.15.155:3300/0 conn(0x7f04180730b0 0x7f04180734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.792+0000 7f041ca00640 1 -- 172.21.15.86:0/2117947662 >> v1:172.21.15.155:6789/0 conn(0x7f04180d16e0 legacy=0x7f04180d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:00.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.792+0000 7f041ca00640 1 -- 172.21.15.86:0/2117947662 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041806fbd0 con 0x7f04180cfca0 2024-04-25T14:05:00.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.796+0000 7f041ca00640 1 --2- 172.21.15.86:0/2117947662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04180cfca0 0x7f041806f3f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f04180cb8c0 tx=0x7f040c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973d4ddc9692bdea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.796+0000 7f0417400640 1 -- 172.21.15.86:0/2117947662 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c0ab430 con 0x7f04180cfca0 2024-04-25T14:05:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.796+0000 7f0417400640 1 -- 172.21.15.86:0/2117947662 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f040c0b0070 con 0x7f04180cfca0 2024-04-25T14:05:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.796+0000 7f0417400640 1 -- 172.21.15.86:0/2117947662 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c0ab7f0 con 0x7f04180cfca0 2024-04-25T14:05:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.796+0000 7f041f400640 1 -- 172.21.15.86:0/2117947662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04180cfca0 msgr2=0x7f041806f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.796+0000 7f041f400640 1 --2- 172.21.15.86:0/2117947662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04180cfca0 0x7f041806f3f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f04180cb8c0 tx=0x7f040c01c520 comp rx=0 tx=0).stop 2024-04-25T14:05:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.796+0000 7f041f400640 1 -- 172.21.15.86:0/2117947662 shutdown_connections 2024-04-25T14:05:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.796+0000 7f041f400640 1 --2- 172.21.15.86:0/2117947662 >> v2:172.21.15.155:3300/0 conn(0x7f04180730b0 0x7f04180734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.796+0000 7f041f400640 1 --2- 172.21.15.86:0/2117947662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04180cfca0 0x7f041806f3f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.796+0000 7f041f400640 1 -- 172.21.15.86:0/2117947662 >> 172.21.15.86:0/2117947662 conn(0x7f04180b4650 msgr2=0x7f04180b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.796+0000 7f041f400640 1 -- 172.21.15.86:0/2117947662 shutdown_connections 2024-04-25T14:05:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.796+0000 7f041f400640 1 -- 172.21.15.86:0/2117947662 wait complete. 2024-04-25T14:05:00.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.796+0000 7f041f400640 1 Processor -- start 2024-04-25T14:05:00.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.796+0000 7f041f400640 1 -- start start 2024-04-25T14:05:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.800+0000 7f041f400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04180730b0 0x7f04180ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.800+0000 7f041f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04180cfca0 0x7f041815af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.800+0000 7f041f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04180d16e0 0x7f041815d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.800+0000 7f041f400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0418169a70 con 0x7f04180cfca0 2024-04-25T14:05:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.800+0000 7f041f400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f04181698f0 con 0x7f04180d16e0 2024-04-25T14:05:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.800+0000 7f041f400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0418169bf0 con 0x7f04180730b0 2024-04-25T14:05:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.800+0000 7f0417e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04180cfca0 0x7f041815af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.800+0000 7f0417e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04180cfca0 0x7f041815af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35484/0 (socket says 172.21.15.86:35484) 2024-04-25T14:05:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.800+0000 7f0417e00640 1 -- 172.21.15.86:0/485479026 learned_addr learned my addr 172.21.15.86:0/485479026 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.800+0000 7f041ca00640 1 --2- 172.21.15.86:0/485479026 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04180730b0 0x7f04180ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.800+0000 7f041d400640 1 --2- 172.21.15.86:0/485479026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04180d16e0 0x7f041815d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.800+0000 7f0417e00640 1 -- 172.21.15.86:0/485479026 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04180730b0 msgr2=0x7f04180ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.800+0000 7f0417e00640 1 --2- 172.21.15.86:0/485479026 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04180730b0 0x7f04180ccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.800+0000 7f0417e00640 1 -- 172.21.15.86:0/485479026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04180d16e0 msgr2=0x7f041815d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.800+0000 7f0417e00640 1 --2- 172.21.15.86:0/485479026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04180d16e0 0x7f041815d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.800+0000 7f0417e00640 1 -- 172.21.15.86:0/485479026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04180cd7c0 con 0x7f04180cfca0 2024-04-25T14:05:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.800+0000 7f0417e00640 1 --2- 172.21.15.86:0/485479026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04180cfca0 0x7f041815af00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f040400c340 tx=0x7f040400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.800+0000 7f0414c00640 1 -- 172.21.15.86:0/485479026 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04040052d0 con 0x7f04180cfca0 2024-04-25T14:05:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.800+0000 7f0414c00640 1 -- 172.21.15.86:0/485479026 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0404005470 con 0x7f04180cfca0 2024-04-25T14:05:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.800+0000 7f041f400640 1 -- 172.21.15.86:0/485479026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04180cdaa0 con 0x7f04180cfca0 2024-04-25T14:05:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.800+0000 7f0414c00640 1 -- 172.21.15.86:0/485479026 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404007260 con 0x7f04180cfca0 2024-04-25T14:05:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.800+0000 7f041f400640 1 -- 172.21.15.86:0/485479026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f041815a700 con 0x7f04180cfca0 2024-04-25T14:05:00.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.800+0000 7f041f400640 1 -- 172.21.15.86:0/485479026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04180cd530 con 0x7f04180cfca0 2024-04-25T14:05:00.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.804+0000 7f0414c00640 1 -- 172.21.15.86:0/485479026 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0404007400 con 0x7f04180cfca0 2024-04-25T14:05:00.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.804+0000 7f0414c00640 1 --2- 172.21.15.86:0/485479026 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f03f806f400 0x7f03f8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:00.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.804+0000 7f0414c00640 1 -- 172.21.15.86:0/485479026 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f040408e540 con 0x7f04180cfca0 2024-04-25T14:05:00.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.804+0000 7f041ca00640 1 --2- 172.21.15.86:0/485479026 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f03f806f400 0x7f03f8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:00.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.804+0000 7f041ca00640 1 --2- 172.21.15.86:0/485479026 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f03f806f400 0x7f03f8071910 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f040c0af000 tx=0x7f040c0a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:00.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:00.808+0000 7f0414c00640 1 -- 172.21.15.86:0/485479026 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f040405af90 con 0x7f04180cfca0 2024-04-25T14:05:01.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.024+0000 7f041f400640 1 -- 172.21.15.86:0/485479026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f041809f4b0 con 0x7f04180cfca0 2024-04-25T14:05:01.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.028+0000 7f0414c00640 1 -- 172.21.15.86:0/485479026 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7f040405f290 con 0x7f04180cfca0 2024-04-25T14:05:01.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.028+0000 7f041f400640 1 -- 172.21.15.86:0/485479026 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f03f806f400 msgr2=0x7f03f8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:01.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.028+0000 7f041f400640 1 --2- 172.21.15.86:0/485479026 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f03f806f400 0x7f03f8071910 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f040c0af000 tx=0x7f040c0a7000 comp rx=0 tx=0).stop 2024-04-25T14:05:01.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.028+0000 7f041f400640 1 -- 172.21.15.86:0/485479026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04180cfca0 msgr2=0x7f041815af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:01.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.028+0000 7f041f400640 1 --2- 172.21.15.86:0/485479026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04180cfca0 0x7f041815af00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f040400c340 tx=0x7f040400c800 comp rx=0 tx=0).stop 2024-04-25T14:05:01.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.028+0000 7f041f400640 1 -- 172.21.15.86:0/485479026 shutdown_connections 2024-04-25T14:05:01.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.028+0000 7f041f400640 1 --2- 172.21.15.86:0/485479026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04180d16e0 0x7f041815d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:01.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.028+0000 7f041f400640 1 --2- 172.21.15.86:0/485479026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04180cfca0 0x7f041815af00 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:01.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.028+0000 7f041f400640 1 --2- 172.21.15.86:0/485479026 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f03f806f400 0x7f03f8071910 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:01.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.028+0000 7f041f400640 1 --2- 172.21.15.86:0/485479026 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04180730b0 0x7f04180ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:01.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.028+0000 7f041f400640 1 -- 172.21.15.86:0/485479026 >> 172.21.15.86:0/485479026 conn(0x7f04180b4650 msgr2=0x7f04180b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:01.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.028+0000 7f041f400640 1 -- 172.21.15.86:0/485479026 shutdown_connections 2024-04-25T14:05:01.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.028+0000 7f041f400640 1 -- 172.21.15.86:0/485479026 wait complete. 2024-04-25T14:05:01.050 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T14:05:01.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.196+0000 7fd4cb800640 1 Processor -- start 2024-04-25T14:05:01.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.196+0000 7fd4cb800640 1 -- start start 2024-04-25T14:05:01.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.196+0000 7fd4cb800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd4c40cfca0 0x7fd4c406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:01.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.196+0000 7fd4cb800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4c4072bd0 con 0x7fd4c40d16e0 2024-04-25T14:05:01.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.196+0000 7fd4cb800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4c406f930 con 0x7fd4c40730b0 2024-04-25T14:05:01.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.196+0000 7fd4cb800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd4c4070470 con 0x7fd4c40cfca0 2024-04-25T14:05:01.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.196+0000 7fd4c8e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd4c40cfca0 0x7fd4c406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:01.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.196+0000 7fd4c3e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd4c40d16e0 0x7fd4c40d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52100/0 (socket says 172.21.15.86:52100) 2024-04-25T14:05:01.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.196+0000 7fd4c3e00640 1 -- 172.21.15.86:0/2094788971 learned_addr learned my addr 172.21.15.86:0/2094788971 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:01.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.196+0000 7fd4c8e00640 1 -- 172.21.15.86:0/2094788971 >> v1:172.21.15.155:6789/0 conn(0x7fd4c40730b0 legacy=0x7fd4c40734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:01.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.196+0000 7fd4c8e00640 1 -- 172.21.15.86:0/2094788971 >> v1:172.21.15.86:6789/0 conn(0x7fd4c40d16e0 legacy=0x7fd4c40d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:01.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.196+0000 7fd4c8e00640 1 -- 172.21.15.86:0/2094788971 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c406fbd0 con 0x7fd4c40cfca0 2024-04-25T14:05:01.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.200+0000 7fd4c8e00640 1 --2- 172.21.15.86:0/2094788971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c40cfca0 0x7fd4c406f3f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40d22a0 tx=0x7fd4b801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba2f908a28f6bdb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:01.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.204+0000 7fd4c3400640 1 -- 172.21.15.86:0/2094788971 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b80b0070 con 0x7fd4c40cfca0 2024-04-25T14:05:01.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.204+0000 7fd4c3400640 1 -- 172.21.15.86:0/2094788971 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4b80ab600 con 0x7fd4c40cfca0 2024-04-25T14:05:01.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.204+0000 7fd4c3400640 1 -- 172.21.15.86:0/2094788971 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c40743e0 con 0x7fd4c40cfca0 2024-04-25T14:05:01.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.204+0000 7fd4cb800640 1 -- 172.21.15.86:0/2094788971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c40cfca0 msgr2=0x7fd4c406f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:01.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.204+0000 7fd4cb800640 1 --2- 172.21.15.86:0/2094788971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c40cfca0 0x7fd4c406f3f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40d22a0 tx=0x7fd4b801c460 comp rx=0 tx=0).stop 2024-04-25T14:05:01.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.204+0000 7fd4cb800640 1 -- 172.21.15.86:0/2094788971 shutdown_connections 2024-04-25T14:05:01.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.204+0000 7fd4cb800640 1 --2- 172.21.15.86:0/2094788971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c40cfca0 0x7fd4c406f3f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:01.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.204+0000 7fd4cb800640 1 -- 172.21.15.86:0/2094788971 >> 172.21.15.86:0/2094788971 conn(0x7fd4c40b4650 msgr2=0x7fd4c40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:01.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.204+0000 7fd4cb800640 1 -- 172.21.15.86:0/2094788971 shutdown_connections 2024-04-25T14:05:01.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.204+0000 7fd4cb800640 1 -- 172.21.15.86:0/2094788971 wait complete. 2024-04-25T14:05:01.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.208+0000 7fd4cb800640 1 Processor -- start 2024-04-25T14:05:01.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.208+0000 7fd4cb800640 1 -- start start 2024-04-25T14:05:01.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.208+0000 7fd4cb800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c40730b0 0x7fd4c4099320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:01.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.208+0000 7fd4cb800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c40cfca0 0x7fd4c4099860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:01.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.208+0000 7fd4cb800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c40d16e0 0x7fd4c4099da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:01.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.208+0000 7fd4cb800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c416aa80 con 0x7fd4c40730b0 2024-04-25T14:05:01.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.208+0000 7fd4cb800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c416a900 con 0x7fd4c40d16e0 2024-04-25T14:05:01.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.208+0000 7fd4cb800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd4c416ac00 con 0x7fd4c40cfca0 2024-04-25T14:05:01.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.208+0000 7fd4c8e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c40730b0 0x7fd4c4099320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:01.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.208+0000 7fd4c8e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c40730b0 0x7fd4c4099320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35502/0 (socket says 172.21.15.86:35502) 2024-04-25T14:05:01.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.208+0000 7fd4c8e00640 1 -- 172.21.15.86:0/1733027674 learned_addr learned my addr 172.21.15.86:0/1733027674 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:01.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.208+0000 7fd4c3e00640 1 --2- 172.21.15.86:0/1733027674 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c40cfca0 0x7fd4c4099860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:01.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.208+0000 7fd4c9800640 1 --2- 172.21.15.86:0/1733027674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c40d16e0 0x7fd4c4099da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:01.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.208+0000 7fd4c8e00640 1 -- 172.21.15.86:0/1733027674 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c40cfca0 msgr2=0x7fd4c4099860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:01.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.208+0000 7fd4c8e00640 1 --2- 172.21.15.86:0/1733027674 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c40cfca0 0x7fd4c4099860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:01.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.208+0000 7fd4c8e00640 1 -- 172.21.15.86:0/1733027674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c40d16e0 msgr2=0x7fd4c4099da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:01.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.208+0000 7fd4c8e00640 1 --2- 172.21.15.86:0/1733027674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c40d16e0 0x7fd4c4099da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:01.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.208+0000 7fd4c8e00640 1 -- 172.21.15.86:0/1733027674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c4154230 con 0x7fd4c40730b0 2024-04-25T14:05:01.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.208+0000 7fd4c8e00640 1 --2- 172.21.15.86:0/1733027674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c40730b0 0x7fd4c4099320 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80ae040 tx=0x7fd4b801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:01.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.208+0000 7fd4c0c00640 1 -- 172.21.15.86:0/1733027674 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b80b0050 con 0x7fd4c40730b0 2024-04-25T14:05:01.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.208+0000 7fd4c0c00640 1 -- 172.21.15.86:0/1733027674 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4b80bc070 con 0x7fd4c40730b0 2024-04-25T14:05:01.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.208+0000 7fd4cb800640 1 -- 172.21.15.86:0/1733027674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4c4154510 con 0x7fd4c40730b0 2024-04-25T14:05:01.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.208+0000 7fd4c0c00640 1 -- 172.21.15.86:0/1733027674 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b80b5a50 con 0x7fd4c40730b0 2024-04-25T14:05:01.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.208+0000 7fd4cb800640 1 -- 172.21.15.86:0/1733027674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4c40c92c0 con 0x7fd4c40730b0 2024-04-25T14:05:01.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.212+0000 7fd4cb800640 1 -- 172.21.15.86:0/1733027674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4c409f4b0 con 0x7fd4c40730b0 2024-04-25T14:05:01.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.212+0000 7fd4c0c00640 1 -- 172.21.15.86:0/1733027674 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd4b80ab4c0 con 0x7fd4c40730b0 2024-04-25T14:05:01.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.212+0000 7fd4c0c00640 1 --2- 172.21.15.86:0/1733027674 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd4a406f430 0x7fd4a4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:01.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.216+0000 7fd4c0c00640 1 -- 172.21.15.86:0/1733027674 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fd4b8129760 con 0x7fd4c40730b0 2024-04-25T14:05:01.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.216+0000 7fd4c3e00640 1 --2- 172.21.15.86:0/1733027674 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd4a406f430 0x7fd4a4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:01.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.216+0000 7fd4c3e00640 1 --2- 172.21.15.86:0/1733027674 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd4a406f430 0x7fd4a4071940 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4070a60 tx=0x7fd4b0007e90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:01.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.220+0000 7fd4c0c00640 1 -- 172.21.15.86:0/1733027674 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd4b8016050 con 0x7fd4c40730b0 2024-04-25T14:05:01.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.432+0000 7fd4cb800640 1 -- 172.21.15.86:0/1733027674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fd4c4069d00 con 0x7fd4c40730b0 2024-04-25T14:05:01.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.436+0000 7fd4c0c00640 1 -- 172.21.15.86:0/1733027674 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7fd4b80f6250 con 0x7fd4c40730b0 2024-04-25T14:05:01.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.436+0000 7fd4cb800640 1 -- 172.21.15.86:0/1733027674 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd4a406f430 msgr2=0x7fd4a4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:01.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.436+0000 7fd4cb800640 1 --2- 172.21.15.86:0/1733027674 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd4a406f430 0x7fd4a4071940 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4070a60 tx=0x7fd4b0007e90 comp rx=0 tx=0).stop 2024-04-25T14:05:01.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.436+0000 7fd4cb800640 1 -- 172.21.15.86:0/1733027674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c40730b0 msgr2=0x7fd4c4099320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:01.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.436+0000 7fd4cb800640 1 --2- 172.21.15.86:0/1733027674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c40730b0 0x7fd4c4099320 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80ae040 tx=0x7fd4b801aca0 comp rx=0 tx=0).stop 2024-04-25T14:05:01.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.436+0000 7fd4cb800640 1 -- 172.21.15.86:0/1733027674 shutdown_connections 2024-04-25T14:05:01.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.436+0000 7fd4cb800640 1 --2- 172.21.15.86:0/1733027674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c40d16e0 0x7fd4c4099da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:01.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.436+0000 7fd4cb800640 1 --2- 172.21.15.86:0/1733027674 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd4a406f430 0x7fd4a4071940 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:01.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.436+0000 7fd4cb800640 1 --2- 172.21.15.86:0/1733027674 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c40cfca0 0x7fd4c4099860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:01.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.436+0000 7fd4cb800640 1 --2- 172.21.15.86:0/1733027674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c40730b0 0x7fd4c4099320 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:01.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.436+0000 7fd4cb800640 1 -- 172.21.15.86:0/1733027674 >> 172.21.15.86:0/1733027674 conn(0x7fd4c40b4650 msgr2=0x7fd4c4158340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:01.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.436+0000 7fd4cb800640 1 -- 172.21.15.86:0/1733027674 shutdown_connections 2024-04-25T14:05:01.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.436+0000 7fd4cb800640 1 -- 172.21.15.86:0/1733027674 wait complete. 2024-04-25T14:05:01.460 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T14:05:01.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.604+0000 7fc7a0400640 1 Processor -- start 2024-04-25T14:05:01.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.604+0000 7fc7a0400640 1 -- start start 2024-04-25T14:05:01.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.604+0000 7fc7a0400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc7980cfca0 0x7fc79806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.604+0000 7fc7a0400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc798072bd0 con 0x7fc7980d16e0 2024-04-25T14:05:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.604+0000 7fc7a0400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc79806f930 con 0x7fc7980730b0 2024-04-25T14:05:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.604+0000 7fc7a0400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc798070470 con 0x7fc7980cfca0 2024-04-25T14:05:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.604+0000 7fc79da00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc7980cfca0 0x7fc79806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.604+0000 7fc79da00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc7980cfca0 0x7fc79806f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35518/0 (socket says 172.21.15.86:35518) 2024-04-25T14:05:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.604+0000 7fc79da00640 1 -- 172.21.15.86:0/3413130750 learned_addr learned my addr 172.21.15.86:0/3413130750 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.604+0000 7fc79da00640 1 -- 172.21.15.86:0/3413130750 >> v1:172.21.15.155:6789/0 conn(0x7fc7980730b0 legacy=0x7fc7980734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.604+0000 7fc79da00640 1 -- 172.21.15.86:0/3413130750 >> v1:172.21.15.86:6789/0 conn(0x7fc7980d16e0 legacy=0x7fc7980d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.604+0000 7fc79da00640 1 -- 172.21.15.86:0/3413130750 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79806fbd0 con 0x7fc7980cfca0 2024-04-25T14:05:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.608+0000 7fc79da00640 1 --2- 172.21.15.86:0/3413130750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7980cfca0 0x7fc79806f3f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc7980d22a0 tx=0x7fc78c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655d6ec61cb30774 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.608+0000 7fc797e00640 1 -- 172.21.15.86:0/3413130750 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7980743e0 con 0x7fc7980cfca0 2024-04-25T14:05:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.608+0000 7fc797e00640 1 -- 172.21.15.86:0/3413130750 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc78c0b0070 con 0x7fc7980cfca0 2024-04-25T14:05:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.608+0000 7fc797e00640 1 -- 172.21.15.86:0/3413130750 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0ab830 con 0x7fc7980cfca0 2024-04-25T14:05:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.608+0000 7fc7a0400640 1 -- 172.21.15.86:0/3413130750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7980cfca0 msgr2=0x7fc79806f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.608+0000 7fc7a0400640 1 --2- 172.21.15.86:0/3413130750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7980cfca0 0x7fc79806f3f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc7980d22a0 tx=0x7fc78c01c500 comp rx=0 tx=0).stop 2024-04-25T14:05:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.608+0000 7fc7a0400640 1 -- 172.21.15.86:0/3413130750 shutdown_connections 2024-04-25T14:05:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.608+0000 7fc7a0400640 1 --2- 172.21.15.86:0/3413130750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7980cfca0 0x7fc79806f3f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.608+0000 7fc7a0400640 1 -- 172.21.15.86:0/3413130750 >> 172.21.15.86:0/3413130750 conn(0x7fc7980b4650 msgr2=0x7fc7980b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.608+0000 7fc7a0400640 1 -- 172.21.15.86:0/3413130750 shutdown_connections 2024-04-25T14:05:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.608+0000 7fc7a0400640 1 -- 172.21.15.86:0/3413130750 wait complete. 2024-04-25T14:05:01.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.612+0000 7fc7a0400640 1 Processor -- start 2024-04-25T14:05:01.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.612+0000 7fc7a0400640 1 -- start start 2024-04-25T14:05:01.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.612+0000 7fc7a0400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7980730b0 0x7fc7980ae130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:01.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.612+0000 7fc7a0400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7980cfca0 0x7fc7981552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:01.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.612+0000 7fc7a0400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7980d16e0 0x7fc7981fcca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:01.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.612+0000 7fc7a0400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc79816aa80 con 0x7fc7980730b0 2024-04-25T14:05:01.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.612+0000 7fc7a0400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc79816a900 con 0x7fc7980cfca0 2024-04-25T14:05:01.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.612+0000 7fc7a0400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc79816ac00 con 0x7fc7980d16e0 2024-04-25T14:05:01.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.612+0000 7fc79da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7980730b0 0x7fc7980ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:01.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.612+0000 7fc79da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7980730b0 0x7fc7980ae130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35520/0 (socket says 172.21.15.86:35520) 2024-04-25T14:05:01.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.612+0000 7fc79da00640 1 -- 172.21.15.86:0/1315369452 learned_addr learned my addr 172.21.15.86:0/1315369452 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:01.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.612+0000 7fc79d000640 1 --2- 172.21.15.86:0/1315369452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7980cfca0 0x7fc7981552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:01.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.612+0000 7fc79e400640 1 --2- 172.21.15.86:0/1315369452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7980d16e0 0x7fc7981fcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:01.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.612+0000 7fc79da00640 1 -- 172.21.15.86:0/1315369452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7980d16e0 msgr2=0x7fc7981fcca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:01.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.612+0000 7fc79da00640 1 --2- 172.21.15.86:0/1315369452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7980d16e0 0x7fc7981fcca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:01.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.612+0000 7fc79da00640 1 -- 172.21.15.86:0/1315369452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7980cfca0 msgr2=0x7fc7981552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:01.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.612+0000 7fc79da00640 1 --2- 172.21.15.86:0/1315369452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7980cfca0 0x7fc7981552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:01.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.612+0000 7fc79da00640 1 -- 172.21.15.86:0/1315369452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7980c9350 con 0x7fc7980730b0 2024-04-25T14:05:01.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.616+0000 7fc79da00640 1 --2- 172.21.15.86:0/1315369452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7980730b0 0x7fc7980ae130 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc78c019110 tx=0x7fc78c01ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:01.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.616+0000 7fc795600640 1 -- 172.21.15.86:0/1315369452 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0b4600 con 0x7fc7980730b0 2024-04-25T14:05:01.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.616+0000 7fc795600640 1 -- 172.21.15.86:0/1315369452 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc78c0b47a0 con 0x7fc7980730b0 2024-04-25T14:05:01.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.616+0000 7fc795600640 1 -- 172.21.15.86:0/1315369452 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0b4b90 con 0x7fc7980730b0 2024-04-25T14:05:01.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.616+0000 7fc7a0400640 1 -- 172.21.15.86:0/1315369452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7980c9630 con 0x7fc7980730b0 2024-04-25T14:05:01.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.616+0000 7fc7a0400640 1 -- 172.21.15.86:0/1315369452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7980c9bf0 con 0x7fc7980730b0 2024-04-25T14:05:01.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.616+0000 7fc7a0400640 1 -- 172.21.15.86:0/1315369452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc79809f4b0 con 0x7fc7980730b0 2024-04-25T14:05:01.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.616+0000 7fc795600640 1 -- 172.21.15.86:0/1315369452 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc78c0b3380 con 0x7fc7980730b0 2024-04-25T14:05:01.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.616+0000 7fc795600640 1 --2- 172.21.15.86:0/1315369452 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fc77806f430 0x7fc778071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:01.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.616+0000 7fc795600640 1 -- 172.21.15.86:0/1315369452 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fc78c129260 con 0x7fc7980730b0 2024-04-25T14:05:01.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.616+0000 7fc79d000640 1 --2- 172.21.15.86:0/1315369452 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fc77806f430 0x7fc778071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:01.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.620+0000 7fc79d000640 1 --2- 172.21.15.86:0/1315369452 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fc77806f430 0x7fc778071940 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc7980b2210 tx=0x7fc784009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:01.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.620+0000 7fc795600640 1 -- 172.21.15.86:0/1315369452 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc78c0a8050 con 0x7fc7980730b0 2024-04-25T14:05:01.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.836+0000 7fc7a0400640 1 -- 172.21.15.86:0/1315369452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fc798069d00 con 0x7fc7980730b0 2024-04-25T14:05:01.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.840+0000 7fc795600640 1 -- 172.21.15.86:0/1315369452 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) ==== 90+0+0 (secure 0 0 0) 0x7fc78c0160f0 con 0x7fc7980730b0 2024-04-25T14:05:01.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.840+0000 7fc7a0400640 1 -- 172.21.15.86:0/1315369452 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fc77806f430 msgr2=0x7fc778071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:01.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.840+0000 7fc7a0400640 1 --2- 172.21.15.86:0/1315369452 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fc77806f430 0x7fc778071940 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc7980b2210 tx=0x7fc784009040 comp rx=0 tx=0).stop 2024-04-25T14:05:01.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.840+0000 7fc7a0400640 1 -- 172.21.15.86:0/1315369452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7980730b0 msgr2=0x7fc7980ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:01.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.840+0000 7fc7a0400640 1 --2- 172.21.15.86:0/1315369452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7980730b0 0x7fc7980ae130 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc78c019110 tx=0x7fc78c01ad40 comp rx=0 tx=0).stop 2024-04-25T14:05:01.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.840+0000 7fc7a0400640 1 -- 172.21.15.86:0/1315369452 shutdown_connections 2024-04-25T14:05:01.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.840+0000 7fc7a0400640 1 --2- 172.21.15.86:0/1315369452 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fc77806f430 0x7fc778071940 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:01.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.840+0000 7fc7a0400640 1 --2- 172.21.15.86:0/1315369452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7980d16e0 0x7fc7981fcca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:01.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.840+0000 7fc7a0400640 1 --2- 172.21.15.86:0/1315369452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7980cfca0 0x7fc7981552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:01.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.840+0000 7fc7a0400640 1 --2- 172.21.15.86:0/1315369452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7980730b0 0x7fc7980ae130 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:01.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.840+0000 7fc7a0400640 1 -- 172.21.15.86:0/1315369452 >> 172.21.15.86:0/1315369452 conn(0x7fc7980b4650 msgr2=0x7fc79809a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:01.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.840+0000 7fc7a0400640 1 -- 172.21.15.86:0/1315369452 shutdown_connections 2024-04-25T14:05:01.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:01.840+0000 7fc7a0400640 1 -- 172.21.15.86:0/1315369452 wait complete. 2024-04-25T14:05:01.863 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T14:05:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.008+0000 7fb090a00640 1 Processor -- start 2024-04-25T14:05:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.008+0000 7fb090a00640 1 -- start start 2024-04-25T14:05:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.012+0000 7fb090a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb08c15c2a0 0x7fb08c0ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.012+0000 7fb090a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb08c06fd50 0x7fb08c070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.012+0000 7fb090a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb08c0b0a60 con 0x7fb08c06c670 2024-04-25T14:05:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.012+0000 7fb090a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb08c06ef90 con 0x7fb08c15c2a0 2024-04-25T14:05:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.012+0000 7fb090a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb08c06f270 con 0x7fb08c06fd50 2024-04-25T14:05:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.012+0000 7fb089e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb08c15c2a0 0x7fb08c0ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.012+0000 7fb089e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb08c15c2a0 0x7fb08c0ae130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35528/0 (socket says 172.21.15.86:35528) 2024-04-25T14:05:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.012+0000 7fb089e00640 1 -- 172.21.15.86:0/2832156900 learned_addr learned my addr 172.21.15.86:0/2832156900 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.012+0000 7fb081400640 1 --2- 172.21.15.86:0/2832156900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb08c06fd50 0x7fb08c070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.012+0000 7fb089e00640 1 -- 172.21.15.86:0/2832156900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb08c06fd50 msgr2=0x7fb08c070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.012+0000 7fb089e00640 1 --2- 172.21.15.86:0/2832156900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb08c06fd50 0x7fb08c070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.012+0000 7fb089e00640 1 -- 172.21.15.86:0/2832156900 >> v1:172.21.15.155:6789/0 conn(0x7fb08c06c670 legacy=0x7fb08c06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.012+0000 7fb089e00640 1 -- 172.21.15.86:0/2832156900 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08c06f550 con 0x7fb08c15c2a0 2024-04-25T14:05:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.012+0000 7fb089e00640 1 --2- 172.21.15.86:0/2832156900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08c15c2a0 0x7fb08c0ae130 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb07c009870 tx=0x7fb07c01d5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf3f3d08a48dc27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:02.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.016+0000 7fb089400640 1 -- 172.21.15.86:0/2832156900 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c0ab660 con 0x7fb08c15c2a0 2024-04-25T14:05:02.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.016+0000 7fb089400640 1 -- 172.21.15.86:0/2832156900 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb07c0b0070 con 0x7fb08c15c2a0 2024-04-25T14:05:02.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.016+0000 7fb089400640 1 -- 172.21.15.86:0/2832156900 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c0aba20 con 0x7fb08c15c2a0 2024-04-25T14:05:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.016+0000 7fb090a00640 1 -- 172.21.15.86:0/2832156900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08c15c2a0 msgr2=0x7fb08c0ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.016+0000 7fb090a00640 1 --2- 172.21.15.86:0/2832156900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08c15c2a0 0x7fb08c0ae130 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb07c009870 tx=0x7fb07c01d5b0 comp rx=0 tx=0).stop 2024-04-25T14:05:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.016+0000 7fb090a00640 1 -- 172.21.15.86:0/2832156900 shutdown_connections 2024-04-25T14:05:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.016+0000 7fb090a00640 1 --2- 172.21.15.86:0/2832156900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb08c06fd50 0x7fb08c070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.016+0000 7fb090a00640 1 --2- 172.21.15.86:0/2832156900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08c15c2a0 0x7fb08c0ae130 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.016+0000 7fb090a00640 1 -- 172.21.15.86:0/2832156900 >> 172.21.15.86:0/2832156900 conn(0x7fb08c0b4650 msgr2=0x7fb08c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.016+0000 7fb090a00640 1 -- 172.21.15.86:0/2832156900 shutdown_connections 2024-04-25T14:05:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.016+0000 7fb090a00640 1 -- 172.21.15.86:0/2832156900 wait complete. 2024-04-25T14:05:02.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.016+0000 7fb090a00640 1 Processor -- start 2024-04-25T14:05:02.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.016+0000 7fb090a00640 1 -- start start 2024-04-25T14:05:02.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.020+0000 7fb090a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08c06c670 0x7fb08c164a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:02.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.020+0000 7fb090a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb08c06fd50 0x7fb08c0c1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:02.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.020+0000 7fb090a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb08c15c2a0 0x7fb08c0b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:02.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.020+0000 7fb090a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c15c830 con 0x7fb08c06c670 2024-04-25T14:05:02.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.020+0000 7fb090a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c15c6b0 con 0x7fb08c15c2a0 2024-04-25T14:05:02.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.020+0000 7fb090a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb08c0ae670 con 0x7fb08c06fd50 2024-04-25T14:05:02.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.020+0000 7fb089e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08c06c670 0x7fb08c164a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:02.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.020+0000 7fb089e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08c06c670 0x7fb08c164a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35530/0 (socket says 172.21.15.86:35530) 2024-04-25T14:05:02.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.020+0000 7fb089e00640 1 -- 172.21.15.86:0/1807539889 learned_addr learned my addr 172.21.15.86:0/1807539889 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:02.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.020+0000 7fb08a800640 1 --2- 172.21.15.86:0/1807539889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb08c15c2a0 0x7fb08c0b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:02.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.020+0000 7fb081400640 1 --2- 172.21.15.86:0/1807539889 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb08c06fd50 0x7fb08c0c1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:02.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.020+0000 7fb089e00640 1 -- 172.21.15.86:0/1807539889 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb08c06fd50 msgr2=0x7fb08c0c1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:02.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.020+0000 7fb089e00640 1 --2- 172.21.15.86:0/1807539889 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb08c06fd50 0x7fb08c0c1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:02.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.020+0000 7fb089e00640 1 -- 172.21.15.86:0/1807539889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb08c15c2a0 msgr2=0x7fb08c0b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:02.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.020+0000 7fb089e00640 1 --2- 172.21.15.86:0/1807539889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb08c15c2a0 0x7fb08c0b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:02.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.020+0000 7fb089e00640 1 -- 172.21.15.86:0/1807539889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08c0c2a20 con 0x7fb08c06c670 2024-04-25T14:05:02.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.020+0000 7fb089e00640 1 --2- 172.21.15.86:0/1807539889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08c06c670 0x7fb08c164a50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb07c01b1e0 tx=0x7fb07c01a350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:02.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.020+0000 7fb082a00640 1 -- 172.21.15.86:0/1807539889 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c0b4620 con 0x7fb08c06c670 2024-04-25T14:05:02.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.020+0000 7fb082a00640 1 -- 172.21.15.86:0/1807539889 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb07c0bb070 con 0x7fb08c06c670 2024-04-25T14:05:02.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.020+0000 7fb090a00640 1 -- 172.21.15.86:0/1807539889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb08c0c2d00 con 0x7fb08c06c670 2024-04-25T14:05:02.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.020+0000 7fb082a00640 1 -- 172.21.15.86:0/1807539889 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c0b49e0 con 0x7fb08c06c670 2024-04-25T14:05:02.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.020+0000 7fb090a00640 1 -- 172.21.15.86:0/1807539889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb08c0c24e0 con 0x7fb08c06c670 2024-04-25T14:05:02.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.020+0000 7fb090a00640 1 -- 172.21.15.86:0/1807539889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb08c069d00 con 0x7fb08c06c670 2024-04-25T14:05:02.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.024+0000 7fb082a00640 1 -- 172.21.15.86:0/1807539889 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb07c0b4b80 con 0x7fb08c06c670 2024-04-25T14:05:02.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.024+0000 7fb082a00640 1 --2- 172.21.15.86:0/1807539889 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb06806f400 0x7fb068071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:02.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.024+0000 7fb082a00640 1 -- 172.21.15.86:0/1807539889 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fb07c129fe0 con 0x7fb08c06c670 2024-04-25T14:05:02.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.024+0000 7fb081400640 1 --2- 172.21.15.86:0/1807539889 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb06806f400 0x7fb068071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:02.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.024+0000 7fb081400640 1 --2- 172.21.15.86:0/1807539889 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb06806f400 0x7fb068071910 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb08c1553f0 tx=0x7fb074008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:02.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.028+0000 7fb082a00640 1 -- 172.21.15.86:0/1807539889 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb07c0c5020 con 0x7fb08c06c670 2024-04-25T14:05:02.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.244+0000 7fb090a00640 1 -- 172.21.15.86:0/1807539889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fb08c076bf0 con 0x7fb08c06c670 2024-04-25T14:05:02.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.244+0000 7fb082a00640 1 -- 172.21.15.86:0/1807539889 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7fb07c017260 con 0x7fb08c06c670 2024-04-25T14:05:02.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.248+0000 7fb090a00640 1 -- 172.21.15.86:0/1807539889 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb06806f400 msgr2=0x7fb068071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:02.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.248+0000 7fb090a00640 1 --2- 172.21.15.86:0/1807539889 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb06806f400 0x7fb068071910 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb08c1553f0 tx=0x7fb074008040 comp rx=0 tx=0).stop 2024-04-25T14:05:02.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.248+0000 7fb090a00640 1 -- 172.21.15.86:0/1807539889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08c06c670 msgr2=0x7fb08c164a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:02.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.248+0000 7fb090a00640 1 --2- 172.21.15.86:0/1807539889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08c06c670 0x7fb08c164a50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb07c01b1e0 tx=0x7fb07c01a350 comp rx=0 tx=0).stop 2024-04-25T14:05:02.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.248+0000 7fb090a00640 1 -- 172.21.15.86:0/1807539889 shutdown_connections 2024-04-25T14:05:02.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.248+0000 7fb090a00640 1 --2- 172.21.15.86:0/1807539889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb08c15c2a0 0x7fb08c0b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:02.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.248+0000 7fb090a00640 1 --2- 172.21.15.86:0/1807539889 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb06806f400 0x7fb068071910 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:02.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.248+0000 7fb090a00640 1 --2- 172.21.15.86:0/1807539889 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb08c06fd50 0x7fb08c0c1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:02.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.248+0000 7fb090a00640 1 --2- 172.21.15.86:0/1807539889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08c06c670 0x7fb08c164a50 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:02.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.248+0000 7fb090a00640 1 -- 172.21.15.86:0/1807539889 >> 172.21.15.86:0/1807539889 conn(0x7fb08c0b4650 msgr2=0x7fb08c154930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:02.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.248+0000 7fb090a00640 1 -- 172.21.15.86:0/1807539889 shutdown_connections 2024-04-25T14:05:02.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.248+0000 7fb090a00640 1 -- 172.21.15.86:0/1807539889 wait complete. 2024-04-25T14:05:02.271 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T14:05:02.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.416+0000 7f0f9ae00640 1 Processor -- start 2024-04-25T14:05:02.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.416+0000 7f0f9ae00640 1 -- start start 2024-04-25T14:05:02.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.416+0000 7f0f9ae00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f9415e860 0x7f0f940ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:02.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.416+0000 7f0f9ae00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0f940cf3e0 0x7f0f940d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:02.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.416+0000 7f0f9ae00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0f9415ec70 0x7f0f940d1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:02.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.416+0000 7f0f9ae00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0f9406cfa0 con 0x7f0f9415ec70 2024-04-25T14:05:02.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.416+0000 7f0f9ae00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0f940d19d0 con 0x7f0f940cf3e0 2024-04-25T14:05:02.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.416+0000 7f0f9ae00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0f940cf8f0 con 0x7f0f9415e860 2024-04-25T14:05:02.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.416+0000 7f0f98e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0f9415ec70 0x7f0f940d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:02.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.416+0000 7f0f98e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0f9415ec70 0x7f0f940d1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35534/0 (socket says 172.21.15.86:35534) 2024-04-25T14:05:02.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.416+0000 7f0f98e00640 1 -- 172.21.15.86:0/2004866209 learned_addr learned my addr 172.21.15.86:0/2004866209 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:02.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.416+0000 7f0f8b400640 1 --2- 172.21.15.86:0/2004866209 >> v2:172.21.15.155:3300/0 conn(0x7f0f940cf3e0 0x7f0f940d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:02.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.416+0000 7f0f93e00640 1 --2- 172.21.15.86:0/2004866209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f9415e860 0x7f0f940ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:02.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.420+0000 7f0f98e00640 1 -- 172.21.15.86:0/2004866209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f9415e860 msgr2=0x7f0f940ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:02.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.420+0000 7f0f98e00640 1 --2- 172.21.15.86:0/2004866209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f9415e860 0x7f0f940ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:02.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.420+0000 7f0f98e00640 1 -- 172.21.15.86:0/2004866209 >> v2:172.21.15.155:3300/0 conn(0x7f0f940cf3e0 msgr2=0x7f0f940d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:02.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.420+0000 7f0f98e00640 1 --2- 172.21.15.86:0/2004866209 >> v2:172.21.15.155:3300/0 conn(0x7f0f940cf3e0 0x7f0f940d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:02.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.420+0000 7f0f98e00640 1 -- 172.21.15.86:0/2004866209 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9406c670 con 0x7f0f9415ec70 2024-04-25T14:05:02.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.420+0000 7f0f98e00640 1 --2- 172.21.15.86:0/2004866209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f9415ec70 0x7f0f940d1490 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0f94072b80 tx=0x7f0f8c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2d178f241bf69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:02.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.420+0000 7f0f93400640 1 -- 172.21.15.86:0/2004866209 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c0a57b0 con 0x7f0f9415ec70 2024-04-25T14:05:02.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.420+0000 7f0f93400640 1 -- 172.21.15.86:0/2004866209 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f8c0a5950 con 0x7f0f9415ec70 2024-04-25T14:05:02.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.420+0000 7f0f93400640 1 -- 172.21.15.86:0/2004866209 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c0a5d20 con 0x7f0f9415ec70 2024-04-25T14:05:02.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.424+0000 7f0f9ae00640 1 -- 172.21.15.86:0/2004866209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f9415ec70 msgr2=0x7f0f940d1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:02.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.424+0000 7f0f9ae00640 1 --2- 172.21.15.86:0/2004866209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f9415ec70 0x7f0f940d1490 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0f94072b80 tx=0x7f0f8c01c930 comp rx=0 tx=0).stop 2024-04-25T14:05:02.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.424+0000 7f0f9ae00640 1 -- 172.21.15.86:0/2004866209 shutdown_connections 2024-04-25T14:05:02.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.424+0000 7f0f9ae00640 1 --2- 172.21.15.86:0/2004866209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f9415ec70 0x7f0f940d1490 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:02.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.424+0000 7f0f9ae00640 1 --2- 172.21.15.86:0/2004866209 >> v2:172.21.15.155:3300/0 conn(0x7f0f940cf3e0 0x7f0f940d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:02.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.424+0000 7f0f9ae00640 1 --2- 172.21.15.86:0/2004866209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f9415e860 0x7f0f940ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:02.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.424+0000 7f0f9ae00640 1 -- 172.21.15.86:0/2004866209 >> 172.21.15.86:0/2004866209 conn(0x7f0f940d2230 msgr2=0x7f0f940d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:02.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.424+0000 7f0f9ae00640 1 -- 172.21.15.86:0/2004866209 shutdown_connections 2024-04-25T14:05:02.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.424+0000 7f0f9ae00640 1 -- 172.21.15.86:0/2004866209 wait complete. 2024-04-25T14:05:02.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.424+0000 7f0f9ae00640 1 Processor -- start 2024-04-25T14:05:02.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.424+0000 7f0f9ae00640 1 -- start start 2024-04-25T14:05:02.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.428+0000 7f0f9ae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f940cf3e0 0x7f0f941528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:02.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.428+0000 7f0f9ae00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f9415e860 0x7f0f941fa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:02.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.428+0000 7f0f9ae00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f9415ec70 0x7f0f941f9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:02.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.428+0000 7f0f9ae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9406ddc0 con 0x7f0f940cf3e0 2024-04-25T14:05:02.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.428+0000 7f0f9ae00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9406dc40 con 0x7f0f9415ec70 2024-04-25T14:05:02.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.428+0000 7f0f9ae00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0f9406df40 con 0x7f0f9415e860 2024-04-25T14:05:02.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.428+0000 7f0f93e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f940cf3e0 0x7f0f941528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:02.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.428+0000 7f0f93e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f940cf3e0 0x7f0f941528e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35544/0 (socket says 172.21.15.86:35544) 2024-04-25T14:05:02.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.428+0000 7f0f93e00640 1 -- 172.21.15.86:0/3312630675 learned_addr learned my addr 172.21.15.86:0/3312630675 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:02.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.428+0000 7f0f8b400640 1 --2- 172.21.15.86:0/3312630675 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f9415e860 0x7f0f941fa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:02.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.428+0000 7f0f98e00640 1 --2- 172.21.15.86:0/3312630675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f9415ec70 0x7f0f941f9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:02.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.428+0000 7f0f93e00640 1 -- 172.21.15.86:0/3312630675 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f9415e860 msgr2=0x7f0f941fa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:02.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.428+0000 7f0f93e00640 1 --2- 172.21.15.86:0/3312630675 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f9415e860 0x7f0f941fa2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:02.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.428+0000 7f0f93e00640 1 -- 172.21.15.86:0/3312630675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f9415ec70 msgr2=0x7f0f941f9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:02.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.428+0000 7f0f93e00640 1 --2- 172.21.15.86:0/3312630675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f9415ec70 0x7f0f941f9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:02.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.428+0000 7f0f93e00640 1 -- 172.21.15.86:0/3312630675 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f94152e20 con 0x7f0f940cf3e0 2024-04-25T14:05:02.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.428+0000 7f0f93e00640 1 --2- 172.21.15.86:0/3312630675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f940cf3e0 0x7f0f941528e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400e870 tx=0x7f0f8400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:02.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.428+0000 7f0f90c00640 1 -- 172.21.15.86:0/3312630675 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f84017070 con 0x7f0f940cf3e0 2024-04-25T14:05:02.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.428+0000 7f0f90c00640 1 -- 172.21.15.86:0/3312630675 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f8400c940 con 0x7f0f940cf3e0 2024-04-25T14:05:02.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.428+0000 7f0f90c00640 1 -- 172.21.15.86:0/3312630675 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8400ccf0 con 0x7f0f940cf3e0 2024-04-25T14:05:02.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.428+0000 7f0f9ae00640 1 -- 172.21.15.86:0/3312630675 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f9406f8d0 con 0x7f0f940cf3e0 2024-04-25T14:05:02.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.428+0000 7f0f9ae00640 1 -- 172.21.15.86:0/3312630675 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f9406fe90 con 0x7f0f940cf3e0 2024-04-25T14:05:02.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.428+0000 7f0f9ae00640 1 -- 172.21.15.86:0/3312630675 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f94074cf0 con 0x7f0f940cf3e0 2024-04-25T14:05:02.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.428+0000 7f0f90c00640 1 -- 172.21.15.86:0/3312630675 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0f8401a070 con 0x7f0f940cf3e0 2024-04-25T14:05:02.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.432+0000 7f0f90c00640 1 --2- 172.21.15.86:0/3312630675 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0f7006f400 0x7f0f70071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:02.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.432+0000 7f0f90c00640 1 -- 172.21.15.86:0/3312630675 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f0f8408f3f0 con 0x7f0f940cf3e0 2024-04-25T14:05:02.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.432+0000 7f0f8b400640 1 --2- 172.21.15.86:0/3312630675 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0f7006f400 0x7f0f70071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:02.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.432+0000 7f0f8b400640 1 --2- 172.21.15.86:0/3312630675 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0f7006f400 0x7f0f70071910 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0f9406f630 tx=0x7f0f7c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:02.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.436+0000 7f0f90c00640 1 -- 172.21.15.86:0/3312630675 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0f8405c400 con 0x7f0f940cf3e0 2024-04-25T14:05:02.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.648+0000 7f0f9ae00640 1 -- 172.21.15.86:0/3312630675 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f0f94160470 con 0x7f0f940cf3e0 2024-04-25T14:05:02.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.652+0000 7f0f90c00640 1 -- 172.21.15.86:0/3312630675 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) ==== 97+0+0 (secure 0 0 0) 0x7f0f8405bd00 con 0x7f0f940cf3e0 2024-04-25T14:05:02.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.656+0000 7f0f9ae00640 1 -- 172.21.15.86:0/3312630675 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0f7006f400 msgr2=0x7f0f70071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:02.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.656+0000 7f0f9ae00640 1 --2- 172.21.15.86:0/3312630675 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0f7006f400 0x7f0f70071910 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0f9406f630 tx=0x7f0f7c008040 comp rx=0 tx=0).stop 2024-04-25T14:05:02.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.656+0000 7f0f9ae00640 1 -- 172.21.15.86:0/3312630675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f940cf3e0 msgr2=0x7f0f941528e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:02.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.656+0000 7f0f9ae00640 1 --2- 172.21.15.86:0/3312630675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f940cf3e0 0x7f0f941528e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400e870 tx=0x7f0f8400ec70 comp rx=0 tx=0).stop 2024-04-25T14:05:02.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.656+0000 7f0f9ae00640 1 -- 172.21.15.86:0/3312630675 shutdown_connections 2024-04-25T14:05:02.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.656+0000 7f0f9ae00640 1 --2- 172.21.15.86:0/3312630675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f9415ec70 0x7f0f941f9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:02.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.656+0000 7f0f9ae00640 1 --2- 172.21.15.86:0/3312630675 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0f7006f400 0x7f0f70071910 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:02.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.656+0000 7f0f9ae00640 1 --2- 172.21.15.86:0/3312630675 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f9415e860 0x7f0f941fa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:02.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.656+0000 7f0f9ae00640 1 --2- 172.21.15.86:0/3312630675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f940cf3e0 0x7f0f941528e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:02.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.656+0000 7f0f9ae00640 1 -- 172.21.15.86:0/3312630675 >> 172.21.15.86:0/3312630675 conn(0x7f0f940d2230 msgr2=0x7f0f9415f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:02.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.656+0000 7f0f9ae00640 1 -- 172.21.15.86:0/3312630675 shutdown_connections 2024-04-25T14:05:02.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.656+0000 7f0f9ae00640 1 -- 172.21.15.86:0/3312630675 wait complete. 2024-04-25T14:05:02.677 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T14:05:02.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.824+0000 7fdb52400640 1 Processor -- start 2024-04-25T14:05:02.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.824+0000 7fdb52400640 1 -- start start 2024-04-25T14:05:02.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.824+0000 7fdb52400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdb4c070330 0x7fdb4c0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:02.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.824+0000 7fdb52400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdb4c0ce790 0x7fdb4c06f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:02.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.824+0000 7fdb52400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb4c0d2030 con 0x7fdb4c0d1c20 2024-04-25T14:05:02.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.824+0000 7fdb52400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdb4c15e5c0 con 0x7fdb4c0ce790 2024-04-25T14:05:02.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.824+0000 7fdb52400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fdb4c15e8a0 con 0x7fdb4c070330 2024-04-25T14:05:02.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.824+0000 7fdb4be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdb4c0ce790 0x7fdb4c06f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:02.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.824+0000 7fdb4b400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdb4c070330 0x7fdb4c0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:02.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.824+0000 7fdb4be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdb4c0ce790 0x7fdb4c06f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35556/0 (socket says 172.21.15.86:35556) 2024-04-25T14:05:02.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.824+0000 7fdb4be00640 1 -- 172.21.15.86:0/1470009844 learned_addr learned my addr 172.21.15.86:0/1470009844 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:02.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.824+0000 7fdb4be00640 1 -- 172.21.15.86:0/1470009844 >> v2:172.21.15.155:3300/0 conn(0x7fdb4c070330 msgr2=0x7fdb4c0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:02.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.824+0000 7fdb4be00640 1 --2- 172.21.15.86:0/1470009844 >> v2:172.21.15.155:3300/0 conn(0x7fdb4c070330 0x7fdb4c0d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:02.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.824+0000 7fdb4be00640 1 -- 172.21.15.86:0/1470009844 >> v1:172.21.15.155:6789/0 conn(0x7fdb4c0d1c20 legacy=0x7fdb4c06ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:02.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.824+0000 7fdb4be00640 1 -- 172.21.15.86:0/1470009844 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4c15eb80 con 0x7fdb4c0ce790 2024-04-25T14:05:02.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.828+0000 7fdb4be00640 1 --2- 172.21.15.86:0/1470009844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb4c0ce790 0x7fdb4c06f440 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdb44009e50 tx=0x7fdb4401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d898c49ccb7cfe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:02.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.828+0000 7fdb4a000640 1 -- 172.21.15.86:0/1470009844 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb440a5620 con 0x7fdb4c0ce790 2024-04-25T14:05:02.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.828+0000 7fdb4a000640 1 -- 172.21.15.86:0/1470009844 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb440a57c0 con 0x7fdb4c0ce790 2024-04-25T14:05:02.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.828+0000 7fdb4a000640 1 -- 172.21.15.86:0/1470009844 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb440a5b90 con 0x7fdb4c0ce790 2024-04-25T14:05:02.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.828+0000 7fdb52400640 1 -- 172.21.15.86:0/1470009844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb4c0ce790 msgr2=0x7fdb4c06f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:02.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.828+0000 7fdb52400640 1 --2- 172.21.15.86:0/1470009844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb4c0ce790 0x7fdb4c06f440 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdb44009e50 tx=0x7fdb4401c960 comp rx=0 tx=0).stop 2024-04-25T14:05:02.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.828+0000 7fdb52400640 1 -- 172.21.15.86:0/1470009844 shutdown_connections 2024-04-25T14:05:02.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.828+0000 7fdb52400640 1 --2- 172.21.15.86:0/1470009844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb4c0ce790 0x7fdb4c06f440 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:02.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.828+0000 7fdb52400640 1 --2- 172.21.15.86:0/1470009844 >> v2:172.21.15.155:3300/0 conn(0x7fdb4c070330 0x7fdb4c0d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:02.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.828+0000 7fdb52400640 1 -- 172.21.15.86:0/1470009844 >> 172.21.15.86:0/1470009844 conn(0x7fdb4c0b4650 msgr2=0x7fdb4c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:02.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.828+0000 7fdb52400640 1 -- 172.21.15.86:0/1470009844 shutdown_connections 2024-04-25T14:05:02.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.828+0000 7fdb52400640 1 -- 172.21.15.86:0/1470009844 wait complete. 2024-04-25T14:05:02.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.832+0000 7fdb52400640 1 Processor -- start 2024-04-25T14:05:02.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.832+0000 7fdb52400640 1 -- start start 2024-04-25T14:05:02.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.832+0000 7fdb52400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb4c070330 0x7fdb4c15b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:02.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.832+0000 7fdb52400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb4c0ce790 0x7fdb4c0ae660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:02.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.832+0000 7fdb52400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb4c0d1c20 0x7fdb4c073500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:02.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.832+0000 7fdb52400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c0708c0 con 0x7fdb4c0d1c20 2024-04-25T14:05:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.832+0000 7fdb52400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c070740 con 0x7fdb4c0ce790 2024-04-25T14:05:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.832+0000 7fdb52400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdb4c16afc0 con 0x7fdb4c070330 2024-04-25T14:05:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.832+0000 7fdb4be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb4c0d1c20 0x7fdb4c073500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.832+0000 7fdb4be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb4c0d1c20 0x7fdb4c073500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35570/0 (socket says 172.21.15.86:35570) 2024-04-25T14:05:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.832+0000 7fdb4be00640 1 -- 172.21.15.86:0/1012346769 learned_addr learned my addr 172.21.15.86:0/1012346769 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.832+0000 7fdb4aa00640 1 --2- 172.21.15.86:0/1012346769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb4c0ce790 0x7fdb4c0ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.832+0000 7fdb4b400640 1 --2- 172.21.15.86:0/1012346769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb4c070330 0x7fdb4c15b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.832+0000 7fdb4be00640 1 -- 172.21.15.86:0/1012346769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb4c070330 msgr2=0x7fdb4c15b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.832+0000 7fdb4be00640 1 --2- 172.21.15.86:0/1012346769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb4c070330 0x7fdb4c15b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.832+0000 7fdb4be00640 1 -- 172.21.15.86:0/1012346769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb4c0ce790 msgr2=0x7fdb4c0ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.832+0000 7fdb4be00640 1 --2- 172.21.15.86:0/1012346769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb4c0ce790 0x7fdb4c0ae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.832+0000 7fdb4be00640 1 -- 172.21.15.86:0/1012346769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4c0af190 con 0x7fdb4c0d1c20 2024-04-25T14:05:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.832+0000 7fdb4b400640 1 --2- 172.21.15.86:0/1012346769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb4c070330 0x7fdb4c15b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:02.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.832+0000 7fdb4aa00640 1 --2- 172.21.15.86:0/1012346769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb4c0ce790 0x7fdb4c0ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:02.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.832+0000 7fdb4be00640 1 --2- 172.21.15.86:0/1012346769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb4c0d1c20 0x7fdb4c073500 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdb440af040 tx=0x7fdb44019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:02.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.832+0000 7fdb43400640 1 -- 172.21.15.86:0/1012346769 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb440bc020 con 0x7fdb4c0d1c20 2024-04-25T14:05:02.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.832+0000 7fdb43400640 1 -- 172.21.15.86:0/1012346769 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb440b8670 con 0x7fdb4c0d1c20 2024-04-25T14:05:02.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.832+0000 7fdb43400640 1 -- 172.21.15.86:0/1012346769 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb44009510 con 0x7fdb4c0d1c20 2024-04-25T14:05:02.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.836+0000 7fdb52400640 1 -- 172.21.15.86:0/1012346769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb4c0af470 con 0x7fdb4c0d1c20 2024-04-25T14:05:02.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.836+0000 7fdb52400640 1 -- 172.21.15.86:0/1012346769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb4c0d0900 con 0x7fdb4c0d1c20 2024-04-25T14:05:02.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.836+0000 7fdb52400640 1 -- 172.21.15.86:0/1012346769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb4c0af990 con 0x7fdb4c0d1c20 2024-04-25T14:05:02.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.836+0000 7fdb43400640 1 -- 172.21.15.86:0/1012346769 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdb440bf020 con 0x7fdb4c0d1c20 2024-04-25T14:05:02.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.836+0000 7fdb43400640 1 --2- 172.21.15.86:0/1012346769 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdb2c06f400 0x7fdb2c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:02.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.836+0000 7fdb43400640 1 -- 172.21.15.86:0/1012346769 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fdb44129fe0 con 0x7fdb4c0d1c20 2024-04-25T14:05:02.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.836+0000 7fdb4b400640 1 --2- 172.21.15.86:0/1012346769 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdb2c06f400 0x7fdb2c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:02.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.840+0000 7fdb4b400640 1 --2- 172.21.15.86:0/1012346769 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdb2c06f400 0x7fdb2c071910 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c072200 tx=0x7fdb3c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:02.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:02.840+0000 7fdb43400640 1 -- 172.21.15.86:0/1012346769 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdb440c6030 con 0x7fdb4c0d1c20 2024-04-25T14:05:03.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.056+0000 7fdb52400640 1 -- 172.21.15.86:0/1012346769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fdb4c069d00 con 0x7fdb4c0d1c20 2024-04-25T14:05:03.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.056+0000 7fdb43400640 1 -- 172.21.15.86:0/1012346769 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) ==== 90+0+0 (secure 0 0 0) 0x7fdb440165e0 con 0x7fdb4c0d1c20 2024-04-25T14:05:03.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.060+0000 7fdb52400640 1 -- 172.21.15.86:0/1012346769 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdb2c06f400 msgr2=0x7fdb2c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:03.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.060+0000 7fdb52400640 1 --2- 172.21.15.86:0/1012346769 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdb2c06f400 0x7fdb2c071910 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c072200 tx=0x7fdb3c008040 comp rx=0 tx=0).stop 2024-04-25T14:05:03.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.060+0000 7fdb52400640 1 -- 172.21.15.86:0/1012346769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb4c0d1c20 msgr2=0x7fdb4c073500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:03.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.060+0000 7fdb52400640 1 --2- 172.21.15.86:0/1012346769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb4c0d1c20 0x7fdb4c073500 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdb440af040 tx=0x7fdb44019700 comp rx=0 tx=0).stop 2024-04-25T14:05:03.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.060+0000 7fdb52400640 1 -- 172.21.15.86:0/1012346769 shutdown_connections 2024-04-25T14:05:03.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.060+0000 7fdb52400640 1 --2- 172.21.15.86:0/1012346769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb4c0d1c20 0x7fdb4c073500 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:03.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.060+0000 7fdb52400640 1 --2- 172.21.15.86:0/1012346769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb4c0ce790 0x7fdb4c0ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:03.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.060+0000 7fdb52400640 1 --2- 172.21.15.86:0/1012346769 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdb2c06f400 0x7fdb2c071910 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:03.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.060+0000 7fdb52400640 1 --2- 172.21.15.86:0/1012346769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb4c070330 0x7fdb4c15b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:03.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.060+0000 7fdb52400640 1 -- 172.21.15.86:0/1012346769 >> 172.21.15.86:0/1012346769 conn(0x7fdb4c0b4650 msgr2=0x7fdb4c167ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:03.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.060+0000 7fdb52400640 1 -- 172.21.15.86:0/1012346769 shutdown_connections 2024-04-25T14:05:03.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.060+0000 7fdb52400640 1 -- 172.21.15.86:0/1012346769 wait complete. 2024-04-25T14:05:03.082 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:05:03.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.224+0000 7f636ac00640 1 Processor -- start 2024-04-25T14:05:03.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.224+0000 7f636ac00640 1 -- start start 2024-04-25T14:05:03.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.228+0000 7f636ac00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6364163980 0x7f6364163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:03.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.228+0000 7f636ac00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f636406e340 0x7f636406e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:03.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.228+0000 7f636ac00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63641648e0 con 0x7f6364072630 2024-04-25T14:05:03.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.228+0000 7f636ac00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f63640d1520 con 0x7f6364163980 2024-04-25T14:05:03.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.228+0000 7f636ac00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f63640d23e0 con 0x7f636406e340 2024-04-25T14:05:03.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.228+0000 7f6363400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6364163980 0x7f6364163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:03.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.228+0000 7f6363400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6364163980 0x7f6364163d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35582/0 (socket says 172.21.15.86:35582) 2024-04-25T14:05:03.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.228+0000 7f6363400640 1 -- 172.21.15.86:0/2459608340 learned_addr learned my addr 172.21.15.86:0/2459608340 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:03.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.228+0000 7f6368c00640 1 --2- 172.21.15.86:0/2459608340 >> v2:172.21.15.155:3300/0 conn(0x7f636406e340 0x7f636406e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:03.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.228+0000 7f6363400640 1 -- 172.21.15.86:0/2459608340 >> v2:172.21.15.155:3300/0 conn(0x7f636406e340 msgr2=0x7f636406e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:03.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.228+0000 7f6363400640 1 --2- 172.21.15.86:0/2459608340 >> v2:172.21.15.155:3300/0 conn(0x7f636406e340 0x7f636406e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:03.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.228+0000 7f6363400640 1 -- 172.21.15.86:0/2459608340 >> v1:172.21.15.86:6789/0 conn(0x7f6364072630 legacy=0x7f6364072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:03.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.228+0000 7f6363400640 1 -- 172.21.15.86:0/2459608340 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6364164600 con 0x7f6364163980 2024-04-25T14:05:03.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.232+0000 7f6363400640 1 --2- 172.21.15.86:0/2459608340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6364163980 0x7f6364163d90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f63640c6b40 tx=0x7f635001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e90c4f64aa469f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:03.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.232+0000 7f6362a00640 1 -- 172.21.15.86:0/2459608340 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63500b0070 con 0x7f6364163980 2024-04-25T14:05:03.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.232+0000 7f6362a00640 1 -- 172.21.15.86:0/2459608340 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63500ab600 con 0x7f6364163980 2024-04-25T14:05:03.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.232+0000 7f6362a00640 1 -- 172.21.15.86:0/2459608340 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63500ab9d0 con 0x7f6364163980 2024-04-25T14:05:03.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.232+0000 7f636ac00640 1 -- 172.21.15.86:0/2459608340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6364163980 msgr2=0x7f6364163d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:03.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.232+0000 7f636ac00640 1 --2- 172.21.15.86:0/2459608340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6364163980 0x7f6364163d90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f63640c6b40 tx=0x7f635001c400 comp rx=0 tx=0).stop 2024-04-25T14:05:03.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.232+0000 7f636ac00640 1 -- 172.21.15.86:0/2459608340 shutdown_connections 2024-04-25T14:05:03.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.232+0000 7f636ac00640 1 --2- 172.21.15.86:0/2459608340 >> v2:172.21.15.155:3300/0 conn(0x7f636406e340 0x7f636406e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:03.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.232+0000 7f636ac00640 1 --2- 172.21.15.86:0/2459608340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6364163980 0x7f6364163d90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:03.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.232+0000 7f636ac00640 1 -- 172.21.15.86:0/2459608340 >> 172.21.15.86:0/2459608340 conn(0x7f63640d3580 msgr2=0x7f63640d3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:03.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.232+0000 7f636ac00640 1 -- 172.21.15.86:0/2459608340 shutdown_connections 2024-04-25T14:05:03.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.232+0000 7f636ac00640 1 -- 172.21.15.86:0/2459608340 wait complete. 2024-04-25T14:05:03.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.236+0000 7f636ac00640 1 Processor -- start 2024-04-25T14:05:03.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.236+0000 7f636ac00640 1 -- start start 2024-04-25T14:05:03.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.236+0000 7f636ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f636406e340 0x7f63640c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:03.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.236+0000 7f636ac00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6364072630 0x7f63641589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.236+0000 7f636ac00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6364163980 0x7f63640b3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.236+0000 7f636ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f636416d530 con 0x7f636406e340 2024-04-25T14:05:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.236+0000 7f636ac00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f636406f0a0 con 0x7f6364072630 2024-04-25T14:05:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.236+0000 7f636ac00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f636416d6b0 con 0x7f6364163980 2024-04-25T14:05:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.236+0000 7f6363e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f636406e340 0x7f63640c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.236+0000 7f6363e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f636406e340 0x7f63640c8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35584/0 (socket says 172.21.15.86:35584) 2024-04-25T14:05:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.236+0000 7f6363e00640 1 -- 172.21.15.86:0/1918681378 learned_addr learned my addr 172.21.15.86:0/1918681378 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.236+0000 7f6363400640 1 --2- 172.21.15.86:0/1918681378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6364072630 0x7f63641589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.236+0000 7f6368c00640 1 --2- 172.21.15.86:0/1918681378 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6364163980 0x7f63640b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.236+0000 7f6363e00640 1 -- 172.21.15.86:0/1918681378 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6364163980 msgr2=0x7f63640b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.236+0000 7f6363e00640 1 --2- 172.21.15.86:0/1918681378 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6364163980 0x7f63640b3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.236+0000 7f6363e00640 1 -- 172.21.15.86:0/1918681378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6364072630 msgr2=0x7f63641589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.236+0000 7f6363e00640 1 --2- 172.21.15.86:0/1918681378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6364072630 0x7f63641589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.236+0000 7f6363e00640 1 -- 172.21.15.86:0/1918681378 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636415e140 con 0x7f636406e340 2024-04-25T14:05:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.236+0000 7f6363e00640 1 --2- 172.21.15.86:0/1918681378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f636406e340 0x7f63640c8ed0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f635400c3a0 tx=0x7f635400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.236+0000 7f635be00640 1 -- 172.21.15.86:0/1918681378 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63540052d0 con 0x7f636406e340 2024-04-25T14:05:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.236+0000 7f635be00640 1 -- 172.21.15.86:0/1918681378 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6354005470 con 0x7f636406e340 2024-04-25T14:05:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.236+0000 7f635be00640 1 -- 172.21.15.86:0/1918681378 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354007260 con 0x7f636406e340 2024-04-25T14:05:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.236+0000 7f636ac00640 1 -- 172.21.15.86:0/1918681378 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f636415e420 con 0x7f636406e340 2024-04-25T14:05:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.236+0000 7f636ac00640 1 -- 172.21.15.86:0/1918681378 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63640cc750 con 0x7f636406e340 2024-04-25T14:05:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.236+0000 7f636ac00640 1 -- 172.21.15.86:0/1918681378 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6364072170 con 0x7f636406e340 2024-04-25T14:05:03.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.240+0000 7f635be00640 1 -- 172.21.15.86:0/1918681378 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6354007400 con 0x7f636406e340 2024-04-25T14:05:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.240+0000 7f635be00640 1 --2- 172.21.15.86:0/1918681378 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f634406f400 0x7f6344071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.240+0000 7f635be00640 1 -- 172.21.15.86:0/1918681378 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f635408e490 con 0x7f636406e340 2024-04-25T14:05:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.240+0000 7f6363400640 1 --2- 172.21.15.86:0/1918681378 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f634406f400 0x7f6344071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.240+0000 7f6363400640 1 --2- 172.21.15.86:0/1918681378 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f634406f400 0x7f6344071910 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f63500ad000 tx=0x7f6350008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:03.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.244+0000 7f635be00640 1 -- 172.21.15.86:0/1918681378 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f635405af80 con 0x7f636406e340 2024-04-25T14:05:03.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.404+0000 7f636ac00640 1 -- 172.21.15.86:0/1918681378 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6364076bf0 con 0x7f636406e340 2024-04-25T14:05:03.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.404+0000 7f635be00640 1 -- 172.21.15.86:0/1918681378 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+13164 (secure 0 0 0) 0x7f635405f280 con 0x7f636406e340 2024-04-25T14:05:03.408 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:03.408 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":34,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:05:00.233045+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:05:03.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.408+0000 7f636ac00640 1 -- 172.21.15.86:0/1918681378 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f634406f400 msgr2=0x7f6344071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:03.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.408+0000 7f636ac00640 1 --2- 172.21.15.86:0/1918681378 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f634406f400 0x7f6344071910 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f63500ad000 tx=0x7f6350008040 comp rx=0 tx=0).stop 2024-04-25T14:05:03.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.408+0000 7f636ac00640 1 -- 172.21.15.86:0/1918681378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f636406e340 msgr2=0x7f63640c8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:03.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.408+0000 7f636ac00640 1 --2- 172.21.15.86:0/1918681378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f636406e340 0x7f63640c8ed0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f635400c3a0 tx=0x7f635400c860 comp rx=0 tx=0).stop 2024-04-25T14:05:03.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.408+0000 7f636ac00640 1 -- 172.21.15.86:0/1918681378 shutdown_connections 2024-04-25T14:05:03.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.408+0000 7f636ac00640 1 --2- 172.21.15.86:0/1918681378 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f634406f400 0x7f6344071910 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:03.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.408+0000 7f636ac00640 1 --2- 172.21.15.86:0/1918681378 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6364163980 0x7f63640b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:03.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.408+0000 7f636ac00640 1 --2- 172.21.15.86:0/1918681378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6364072630 0x7f63641589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:03.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.408+0000 7f636ac00640 1 --2- 172.21.15.86:0/1918681378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f636406e340 0x7f63640c8ed0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:03.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.408+0000 7f636ac00640 1 -- 172.21.15.86:0/1918681378 >> 172.21.15.86:0/1918681378 conn(0x7f63640d3580 msgr2=0x7f6364159c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:03.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.408+0000 7f636ac00640 1 -- 172.21.15.86:0/1918681378 shutdown_connections 2024-04-25T14:05:03.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.408+0000 7f636ac00640 1 -- 172.21.15.86:0/1918681378 wait complete. 2024-04-25T14:05:03.430 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:05:03.430 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:05:03.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.576+0000 7f8401400640 1 Processor -- start 2024-04-25T14:05:03.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.576+0000 7f8401400640 1 -- start start 2024-04-25T14:05:03.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.580+0000 7f8401400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f83fc070330 0x7f83fc0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:03.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.580+0000 7f8401400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f83fc0d1c20 0x7f83fc06ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:03.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.580+0000 7f8401400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83fc0d2030 con 0x7f83fc06f270 2024-04-25T14:05:03.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.580+0000 7f8401400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f83fc15e5c0 con 0x7f83fc0d1c20 2024-04-25T14:05:03.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.580+0000 7f8401400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f83fc15e8a0 con 0x7f83fc070330 2024-04-25T14:05:03.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.580+0000 7f83f1e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f83fc0d1c20 0x7f83fc06ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:03.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.580+0000 7f83fb200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f83fc06f270 0x7f83fc06f680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52130/0 (socket says 172.21.15.86:52130) 2024-04-25T14:05:03.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.580+0000 7f83fb200640 1 -- 172.21.15.86:0/1735933867 learned_addr learned my addr 172.21.15.86:0/1735933867 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:03.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.580+0000 7f83fa800640 1 --2- 172.21.15.86:0/1735933867 >> v2:172.21.15.155:3300/0 conn(0x7f83fc070330 0x7f83fc0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:03.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.580+0000 7f83f1e00640 1 -- 172.21.15.86:0/1735933867 >> v2:172.21.15.155:3300/0 conn(0x7f83fc070330 msgr2=0x7f83fc0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:03.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.580+0000 7f83f1e00640 1 --2- 172.21.15.86:0/1735933867 >> v2:172.21.15.155:3300/0 conn(0x7f83fc070330 0x7f83fc0d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:03.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.580+0000 7f83f1e00640 1 -- 172.21.15.86:0/1735933867 >> v1:172.21.15.86:6789/0 conn(0x7f83fc06f270 legacy=0x7f83fc06f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:03.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.580+0000 7f83f9e00640 1 -- 172.21.15.86:0/1735933867 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4121404780 0 0) 0x7f83f40014a0 con 0x7f83fc06f270 2024-04-25T14:05:03.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.580+0000 7f83f1e00640 1 -- 172.21.15.86:0/1735933867 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83fc15eb80 con 0x7f83fc0d1c20 2024-04-25T14:05:03.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.584+0000 7f83f1e00640 1 --2- 172.21.15.86:0/1735933867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83fc0d1c20 0x7f83fc06ed30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f83e40097c0 tx=0x7f83e401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a03a1d06105e91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:03.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.584+0000 7f83f9e00640 1 -- 172.21.15.86:0/1735933867 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e40b0070 con 0x7f83fc0d1c20 2024-04-25T14:05:03.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.584+0000 7f83f9e00640 1 -- 172.21.15.86:0/1735933867 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83e40ab600 con 0x7f83fc0d1c20 2024-04-25T14:05:03.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.584+0000 7f83f9e00640 1 -- 172.21.15.86:0/1735933867 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e40a4e60 con 0x7f83fc0d1c20 2024-04-25T14:05:03.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.584+0000 7f8401400640 1 -- 172.21.15.86:0/1735933867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83fc0d1c20 msgr2=0x7f83fc06ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:03.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.584+0000 7f8401400640 1 --2- 172.21.15.86:0/1735933867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83fc0d1c20 0x7f83fc06ed30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f83e40097c0 tx=0x7f83e401c3d0 comp rx=0 tx=0).stop 2024-04-25T14:05:03.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.584+0000 7f8401400640 1 -- 172.21.15.86:0/1735933867 shutdown_connections 2024-04-25T14:05:03.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.584+0000 7f8401400640 1 --2- 172.21.15.86:0/1735933867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83fc0d1c20 0x7f83fc06ed30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:03.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.584+0000 7f8401400640 1 --2- 172.21.15.86:0/1735933867 >> v2:172.21.15.155:3300/0 conn(0x7f83fc070330 0x7f83fc0d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:03.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.584+0000 7f8401400640 1 -- 172.21.15.86:0/1735933867 >> 172.21.15.86:0/1735933867 conn(0x7f83fc0b4650 msgr2=0x7f83fc0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:03.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.584+0000 7f8401400640 1 -- 172.21.15.86:0/1735933867 shutdown_connections 2024-04-25T14:05:03.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.584+0000 7f8401400640 1 -- 172.21.15.86:0/1735933867 wait complete. 2024-04-25T14:05:03.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.588+0000 7f8401400640 1 Processor -- start 2024-04-25T14:05:03.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.588+0000 7f8401400640 1 -- start start 2024-04-25T14:05:03.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.588+0000 7f8401400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83fc06f270 0x7f83fc0c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:03.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.588+0000 7f8401400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83fc070330 0x7f83fc15a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:03.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.588+0000 7f8401400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83fc0d1c20 0x7f83fc0ae130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:03.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.588+0000 7f8401400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc0708c0 con 0x7f83fc06f270 2024-04-25T14:05:03.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.588+0000 7f8401400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc070740 con 0x7f83fc070330 2024-04-25T14:05:03.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.588+0000 7f8401400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f83fc16afc0 con 0x7f83fc0d1c20 2024-04-25T14:05:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.588+0000 7f83fa800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83fc06f270 0x7f83fc0c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.588+0000 7f83fa800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83fc06f270 0x7f83fc0c9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35590/0 (socket says 172.21.15.86:35590) 2024-04-25T14:05:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.588+0000 7f83fa800640 1 -- 172.21.15.86:0/3406509322 learned_addr learned my addr 172.21.15.86:0/3406509322 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.588+0000 7f83f1e00640 1 --2- 172.21.15.86:0/3406509322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83fc070330 0x7f83fc15a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.588+0000 7f83fb200640 1 --2- 172.21.15.86:0/3406509322 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83fc0d1c20 0x7f83fc0ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.588+0000 7f83fa800640 1 -- 172.21.15.86:0/3406509322 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83fc0d1c20 msgr2=0x7f83fc0ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.588+0000 7f83fa800640 1 --2- 172.21.15.86:0/3406509322 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83fc0d1c20 0x7f83fc0ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.588+0000 7f83fa800640 1 -- 172.21.15.86:0/3406509322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83fc070330 msgr2=0x7f83fc15a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.588+0000 7f83fa800640 1 --2- 172.21.15.86:0/3406509322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83fc070330 0x7f83fc15a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.588+0000 7f83fa800640 1 -- 172.21.15.86:0/3406509322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83fc0c55b0 con 0x7f83fc06f270 2024-04-25T14:05:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.588+0000 7f83f1e00640 1 --2- 172.21.15.86:0/3406509322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83fc070330 0x7f83fc15a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.588+0000 7f83fa800640 1 --2- 172.21.15.86:0/3406509322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83fc06f270 0x7f83fc0c9a10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00e870 tx=0x7f83ec00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:03.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.588+0000 7f83f3400640 1 -- 172.21.15.86:0/3406509322 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec017070 con 0x7f83fc06f270 2024-04-25T14:05:03.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.588+0000 7f83f3400640 1 -- 172.21.15.86:0/3406509322 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83ec00c940 con 0x7f83fc06f270 2024-04-25T14:05:03.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.588+0000 7f8401400640 1 -- 172.21.15.86:0/3406509322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83fc0c5890 con 0x7f83fc06f270 2024-04-25T14:05:03.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.588+0000 7f83f3400640 1 -- 172.21.15.86:0/3406509322 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec00cd10 con 0x7f83fc06f270 2024-04-25T14:05:03.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.588+0000 7f8401400640 1 -- 172.21.15.86:0/3406509322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83fc0ae7e0 con 0x7f83fc06f270 2024-04-25T14:05:03.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.592+0000 7f8401400640 1 -- 172.21.15.86:0/3406509322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83fc069d00 con 0x7f83fc06f270 2024-04-25T14:05:03.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.592+0000 7f83f3400640 1 -- 172.21.15.86:0/3406509322 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f83ec01a070 con 0x7f83fc06f270 2024-04-25T14:05:03.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.592+0000 7f83f3400640 1 --2- 172.21.15.86:0/3406509322 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f83d806f430 0x7f83d8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:03.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.592+0000 7f83f3400640 1 -- 172.21.15.86:0/3406509322 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f83ec08f690 con 0x7f83fc06f270 2024-04-25T14:05:03.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.592+0000 7f83f1e00640 1 --2- 172.21.15.86:0/3406509322 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f83d806f430 0x7f83d8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:03.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.592+0000 7f83f1e00640 1 --2- 172.21.15.86:0/3406509322 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f83d806f430 0x7f83d8071940 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f83e40ad000 tx=0x7f83e40a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:03.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.596+0000 7f83f3400640 1 -- 172.21.15.86:0/3406509322 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f83ec05c840 con 0x7f83fc06f270 2024-04-25T14:05:03.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.756+0000 7f8401400640 1 -- 172.21.15.86:0/3406509322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f83fc076bf0 con 0x7f83fc06f270 2024-04-25T14:05:03.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.756+0000 7f83f3400640 1 -- 172.21.15.86:0/3406509322 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7f83ec05c140 con 0x7f83fc06f270 2024-04-25T14:05:03.759 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:05:03.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.760+0000 7f8401400640 1 -- 172.21.15.86:0/3406509322 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f83d806f430 msgr2=0x7f83d8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:03.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.760+0000 7f8401400640 1 --2- 172.21.15.86:0/3406509322 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f83d806f430 0x7f83d8071940 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f83e40ad000 tx=0x7f83e40a6000 comp rx=0 tx=0).stop 2024-04-25T14:05:03.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.760+0000 7f8401400640 1 -- 172.21.15.86:0/3406509322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83fc06f270 msgr2=0x7f83fc0c9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:03.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.760+0000 7f8401400640 1 --2- 172.21.15.86:0/3406509322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83fc06f270 0x7f83fc0c9a10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00e870 tx=0x7f83ec00ec70 comp rx=0 tx=0).stop 2024-04-25T14:05:03.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.760+0000 7f8401400640 1 -- 172.21.15.86:0/3406509322 shutdown_connections 2024-04-25T14:05:03.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.760+0000 7f8401400640 1 --2- 172.21.15.86:0/3406509322 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f83d806f430 0x7f83d8071940 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:03.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.760+0000 7f8401400640 1 --2- 172.21.15.86:0/3406509322 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83fc0d1c20 0x7f83fc0ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:03.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.760+0000 7f8401400640 1 --2- 172.21.15.86:0/3406509322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83fc070330 0x7f83fc15a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:03.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.760+0000 7f8401400640 1 --2- 172.21.15.86:0/3406509322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83fc06f270 0x7f83fc0c9a10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:03.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.760+0000 7f8401400640 1 -- 172.21.15.86:0/3406509322 >> 172.21.15.86:0/3406509322 conn(0x7f83fc0b4650 msgr2=0x7f83fc0c6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:03.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.760+0000 7f8401400640 1 -- 172.21.15.86:0/3406509322 shutdown_connections 2024-04-25T14:05:03.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.760+0000 7f8401400640 1 -- 172.21.15.86:0/3406509322 wait complete. 2024-04-25T14:05:03.781 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T14:05:03.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.924+0000 7f74cb600640 1 Processor -- start 2024-04-25T14:05:03.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.924+0000 7f74cb600640 1 -- start start 2024-04-25T14:05:03.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74cb600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74c40740c0 0x7f74c41676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:03.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74cb600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74c4072bd0 con 0x7f74c40d16e0 2024-04-25T14:05:03.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74cb600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74c406fbd0 con 0x7f74c40cfca0 2024-04-25T14:05:03.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74cb600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f74c4070470 con 0x7f74c40740c0 2024-04-25T14:05:03.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74bbe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f74c40d16e0 0x7f74c40d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52142/0 (socket says 172.21.15.86:52142) 2024-04-25T14:05:03.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74bbe00640 1 -- 172.21.15.86:0/1258758939 learned_addr learned my addr 172.21.15.86:0/1258758939 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:03.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74c9600640 1 --2- 172.21.15.86:0/1258758939 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74c40740c0 0x7f74c41676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:03.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74c3e00640 1 -- 172.21.15.86:0/1258758939 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1495157045 0 0) 0x7f74c4072bd0 con 0x7f74c40d16e0 2024-04-25T14:05:03.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74c3e00640 1 -- 172.21.15.86:0/1258758939 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74a8003530 con 0x7f74c40d16e0 2024-04-25T14:05:03.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74c3e00640 1 -- 172.21.15.86:0/1258758939 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4189073502 0 0) 0x7f74c406fbd0 con 0x7f74c40cfca0 2024-04-25T14:05:03.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74c3e00640 1 -- 172.21.15.86:0/1258758939 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74c4072bd0 con 0x7f74c40cfca0 2024-04-25T14:05:03.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74c3e00640 1 -- 172.21.15.86:0/1258758939 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1315092414 0 0) 0x7f74a8003530 con 0x7f74c40d16e0 2024-04-25T14:05:03.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74c3e00640 1 -- 172.21.15.86:0/1258758939 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74c40740c0 msgr2=0x7f74c41676d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:03.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74c3e00640 1 --2- 172.21.15.86:0/1258758939 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74c40740c0 0x7f74c41676d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:03.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74c3e00640 1 -- 172.21.15.86:0/1258758939 >> v1:172.21.15.155:6789/0 conn(0x7f74c40cfca0 legacy=0x7f74c406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:03.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74c3e00640 1 -- 172.21.15.86:0/1258758939 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c406feb0 con 0x7f74c40d16e0 2024-04-25T14:05:03.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74c3e00640 1 -- 172.21.15.86:0/1258758939 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f74ac001680 con 0x7f74c40d16e0 2024-04-25T14:05:03.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74c3e00640 1 -- 172.21.15.86:0/1258758939 >> v1:172.21.15.86:6789/0 conn(0x7f74c40d16e0 legacy=0x7f74c40d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:03.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74c3e00640 1 --2- 172.21.15.86:0/1258758939 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74c40740c0 0x7f74a8003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:03.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74c3e00640 1 --2- 172.21.15.86:0/1258758939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74a8004150 0x7f74a8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:03.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74c3e00640 1 --2- 172.21.15.86:0/1258758939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74a8004d50 0x7f74a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:03.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74c3e00640 1 -- 172.21.15.86:0/1258758939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74c4070470 con 0x7f74a8004d50 2024-04-25T14:05:03.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74c3e00640 1 -- 172.21.15.86:0/1258758939 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f74a800bbc0 con 0x7f74a8004150 2024-04-25T14:05:03.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74c3e00640 1 -- 172.21.15.86:0/1258758939 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f74a800bdc0 con 0x7f74c40740c0 2024-04-25T14:05:03.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74c9600640 1 --2- 172.21.15.86:0/1258758939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74a8004d50 0x7f74a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:03.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74c8c00640 1 --2- 172.21.15.86:0/1258758939 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74c40740c0 0x7f74a8003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:03.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74bbe00640 1 --2- 172.21.15.86:0/1258758939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74a8004150 0x7f74a8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:03.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74c9600640 1 -- 172.21.15.86:0/1258758939 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74c40740c0 msgr2=0x7f74a8003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:03.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74c9600640 1 --2- 172.21.15.86:0/1258758939 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74c40740c0 0x7f74a8003bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:03.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74c9600640 1 -- 172.21.15.86:0/1258758939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74a8004150 msgr2=0x7f74a8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:03.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74c9600640 1 --2- 172.21.15.86:0/1258758939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74a8004150 0x7f74a8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:03.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.928+0000 7f74c9600640 1 -- 172.21.15.86:0/1258758939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a8003530 con 0x7f74a8004d50 2024-04-25T14:05:03.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.932+0000 7f74c9600640 1 --2- 172.21.15.86:0/1258758939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74a8004d50 0x7f74a80051c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f74c40729e0 tx=0x7f74bc01cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc8c4a52cad60d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:03.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.932+0000 7f74c3e00640 1 -- 172.21.15.86:0/1258758939 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74bc0b2070 con 0x7f74a8004d50 2024-04-25T14:05:03.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.932+0000 7f74c3e00640 1 -- 172.21.15.86:0/1258758939 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74bc0a5c10 con 0x7f74a8004d50 2024-04-25T14:05:03.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.932+0000 7f74c3e00640 1 -- 172.21.15.86:0/1258758939 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74bc009c10 con 0x7f74a8004d50 2024-04-25T14:05:03.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.932+0000 7f74cb600640 1 -- 172.21.15.86:0/1258758939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74a8004d50 msgr2=0x7f74a80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:03.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.932+0000 7f74cb600640 1 --2- 172.21.15.86:0/1258758939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74a8004d50 0x7f74a80051c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f74c40729e0 tx=0x7f74bc01cc00 comp rx=0 tx=0).stop 2024-04-25T14:05:03.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.932+0000 7f74c9600640 1 -- 172.21.15.86:0/1258758939 reap_dead start 2024-04-25T14:05:03.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.932+0000 7f74cb600640 1 -- 172.21.15.86:0/1258758939 shutdown_connections 2024-04-25T14:05:03.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.932+0000 7f74cb600640 1 -- 172.21.15.86:0/1258758939 >> 172.21.15.86:0/1258758939 conn(0x7f74c40b4650 msgr2=0x7f74c40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:03.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.932+0000 7f74cb600640 1 -- 172.21.15.86:0/1258758939 shutdown_connections 2024-04-25T14:05:03.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.932+0000 7f74cb600640 1 -- 172.21.15.86:0/1258758939 wait complete. 2024-04-25T14:05:03.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.936+0000 7f74cb600640 1 Processor -- start 2024-04-25T14:05:03.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.936+0000 7f74cb600640 1 -- start start 2024-04-25T14:05:03.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.936+0000 7f74cb600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74a8004d50 0x7f74c415b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:03.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.936+0000 7f74cb600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74c4157030 0x7f74c4157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:03.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.936+0000 7f74cb600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74c4159230 0x7f74c4159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:03.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.936+0000 7f74cb600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74c4165840 con 0x7f74c4159230 2024-04-25T14:05:03.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.936+0000 7f74cb600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f74c41656c0 con 0x7f74c4157030 2024-04-25T14:05:03.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.936+0000 7f74cb600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f74c41659c0 con 0x7f74a8004d50 2024-04-25T14:05:03.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.936+0000 7f74c9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74c4159230 0x7f74c4159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:03.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.936+0000 7f74c9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74c4159230 0x7f74c4159640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35602/0 (socket says 172.21.15.86:35602) 2024-04-25T14:05:03.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.936+0000 7f74c9600640 1 -- 172.21.15.86:0/2438126757 learned_addr learned my addr 172.21.15.86:0/2438126757 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:03.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.936+0000 7f74bbe00640 1 --2- 172.21.15.86:0/2438126757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74c4157030 0x7f74c4157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:03.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.936+0000 7f74c8c00640 1 --2- 172.21.15.86:0/2438126757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74a8004d50 0x7f74c415b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:03.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.936+0000 7f74c9600640 1 -- 172.21.15.86:0/2438126757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74a8004d50 msgr2=0x7f74c415b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:03.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.936+0000 7f74c9600640 1 --2- 172.21.15.86:0/2438126757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74a8004d50 0x7f74c415b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:03.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.936+0000 7f74c9600640 1 -- 172.21.15.86:0/2438126757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74c4157030 msgr2=0x7f74c4157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:03.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.936+0000 7f74c9600640 1 --2- 172.21.15.86:0/2438126757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74c4157030 0x7f74c4157440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:03.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.936+0000 7f74c9600640 1 -- 172.21.15.86:0/2438126757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c4157cc0 con 0x7f74c4159230 2024-04-25T14:05:03.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.936+0000 7f74c9600640 1 --2- 172.21.15.86:0/2438126757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74c4159230 0x7f74c4159640 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f74bc002240 tx=0x7f74bc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:03.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.936+0000 7f74c1600640 1 -- 172.21.15.86:0/2438126757 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74bc0b2040 con 0x7f74c4159230 2024-04-25T14:05:03.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.936+0000 7f74c1600640 1 -- 172.21.15.86:0/2438126757 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74bc0bc070 con 0x7f74c4159230 2024-04-25T14:05:03.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.936+0000 7f74cb600640 1 -- 172.21.15.86:0/2438126757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74c4157fa0 con 0x7f74c4159230 2024-04-25T14:05:03.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.936+0000 7f74c1600640 1 -- 172.21.15.86:0/2438126757 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74bc009c10 con 0x7f74c4159230 2024-04-25T14:05:03.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.936+0000 7f74cb600640 1 -- 172.21.15.86:0/2438126757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74c4158560 con 0x7f74c4159230 2024-04-25T14:05:03.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.940+0000 7f74cb600640 1 -- 172.21.15.86:0/2438126757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74c409f4b0 con 0x7f74c4159230 2024-04-25T14:05:03.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.940+0000 7f74c1600640 1 -- 172.21.15.86:0/2438126757 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f74bc0ae950 con 0x7f74c4159230 2024-04-25T14:05:03.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.940+0000 7f74c1600640 1 --2- 172.21.15.86:0/2438126757 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f74a006f430 0x7f74a0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:03.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.940+0000 7f74c1600640 1 -- 172.21.15.86:0/2438126757 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f74bc129a50 con 0x7f74c4159230 2024-04-25T14:05:03.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.940+0000 7f74c8c00640 1 --2- 172.21.15.86:0/2438126757 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f74a006f430 0x7f74a0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:03.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.940+0000 7f74c8c00640 1 --2- 172.21.15.86:0/2438126757 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f74a006f430 0x7f74a0071940 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f74b4000c10 tx=0x7f74b4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:03.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:03.944+0000 7f74c1600640 1 -- 172.21.15.86:0/2438126757 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f74bc016910 con 0x7f74c4159230 2024-04-25T14:05:04.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.104+0000 7f74cb600640 1 -- 172.21.15.86:0/2438126757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f74c4069d00 con 0x7f74c4159230 2024-04-25T14:05:04.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.104+0000 7f74c1600640 1 -- 172.21.15.86:0/2438126757 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) ==== 84+0+65 (secure 0 0 0) 0x7f74bc0f6a60 con 0x7f74c4159230 2024-04-25T14:05:04.108 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:04.108 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-04-25T14:05:04.108 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:04.108 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-04-25T14:05:04.108 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-04-25T14:05:04.108 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:04.108 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-04-25T14:05:04.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.108+0000 7f74cb600640 1 -- 172.21.15.86:0/2438126757 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f74a006f430 msgr2=0x7f74a0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:04.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.108+0000 7f74cb600640 1 --2- 172.21.15.86:0/2438126757 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f74a006f430 0x7f74a0071940 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f74b4000c10 tx=0x7f74b4009040 comp rx=0 tx=0).stop 2024-04-25T14:05:04.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.108+0000 7f74cb600640 1 -- 172.21.15.86:0/2438126757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74c4159230 msgr2=0x7f74c4159640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:04.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.108+0000 7f74cb600640 1 --2- 172.21.15.86:0/2438126757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74c4159230 0x7f74c4159640 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f74bc002240 tx=0x7f74bc009750 comp rx=0 tx=0).stop 2024-04-25T14:05:04.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.108+0000 7f74cb600640 1 -- 172.21.15.86:0/2438126757 shutdown_connections 2024-04-25T14:05:04.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.108+0000 7f74cb600640 1 --2- 172.21.15.86:0/2438126757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74c4159230 0x7f74c4159640 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:04.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.108+0000 7f74cb600640 1 --2- 172.21.15.86:0/2438126757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74c4157030 0x7f74c4157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:04.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.108+0000 7f74cb600640 1 --2- 172.21.15.86:0/2438126757 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f74a006f430 0x7f74a0071940 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:04.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.108+0000 7f74cb600640 1 --2- 172.21.15.86:0/2438126757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74a8004d50 0x7f74c415b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:04.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.108+0000 7f74cb600640 1 -- 172.21.15.86:0/2438126757 >> 172.21.15.86:0/2438126757 conn(0x7f74c40b4650 msgr2=0x7f74c40b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:04.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.108+0000 7f74cb600640 1 -- 172.21.15.86:0/2438126757 shutdown_connections 2024-04-25T14:05:04.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.108+0000 7f74cb600640 1 -- 172.21.15.86:0/2438126757 wait complete. 2024-04-25T14:05:04.129 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T14:05:04.129 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T14:05:04.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.280+0000 7f929b800640 1 Processor -- start 2024-04-25T14:05:04.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.280+0000 7f929b800640 1 -- start start 2024-04-25T14:05:04.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.280+0000 7f929b800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9294166660 0x7f9294168a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:04.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.280+0000 7f929b800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92940b0b20 con 0x7f929415e270 2024-04-25T14:05:04.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.280+0000 7f929b800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f929406d220 con 0x7f92940b0540 2024-04-25T14:05:04.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.280+0000 7f929b800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f929406d870 con 0x7f9294166660 2024-04-25T14:05:04.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.280+0000 7f9293e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f929415e270 0x7f929415e680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52148/0 (socket says 172.21.15.86:52148) 2024-04-25T14:05:04.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.280+0000 7f9293e00640 1 -- 172.21.15.86:0/3230240734 learned_addr learned my addr 172.21.15.86:0/3230240734 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:04.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.280+0000 7f9299800640 1 --2- 172.21.15.86:0/3230240734 >> v2:172.21.15.155:3300/0 conn(0x7f9294166660 0x7f9294168a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:04.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.280+0000 7f9293400640 1 -- 172.21.15.86:0/3230240734 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2159084316 0 0) 0x7f92940b0b20 con 0x7f929415e270 2024-04-25T14:05:04.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.280+0000 7f9293400640 1 -- 172.21.15.86:0/3230240734 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9278003530 con 0x7f929415e270 2024-04-25T14:05:04.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.284+0000 7f9293400640 1 -- 172.21.15.86:0/3230240734 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3692650830 0 0) 0x7f929406d220 con 0x7f92940b0540 2024-04-25T14:05:04.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.284+0000 7f9293400640 1 -- 172.21.15.86:0/3230240734 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92940b0b20 con 0x7f92940b0540 2024-04-25T14:05:04.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.284+0000 7f9293400640 1 -- 172.21.15.86:0/3230240734 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3907314682 0 0) 0x7f9278003530 con 0x7f929415e270 2024-04-25T14:05:04.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.284+0000 7f9293400640 1 -- 172.21.15.86:0/3230240734 >> v2:172.21.15.155:3300/0 conn(0x7f9294166660 msgr2=0x7f9294168a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:04.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.284+0000 7f9293400640 1 --2- 172.21.15.86:0/3230240734 >> v2:172.21.15.155:3300/0 conn(0x7f9294166660 0x7f9294168a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:04.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.284+0000 7f9293400640 1 -- 172.21.15.86:0/3230240734 >> v1:172.21.15.155:6789/0 conn(0x7f92940b0540 legacy=0x7f92940ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:04.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.284+0000 7f9293400640 1 -- 172.21.15.86:0/3230240734 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f929406d500 con 0x7f929415e270 2024-04-25T14:05:04.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.284+0000 7f9293400640 1 -- 172.21.15.86:0/3230240734 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f9280001680 con 0x7f929415e270 2024-04-25T14:05:04.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.284+0000 7f9293400640 1 -- 172.21.15.86:0/3230240734 >> v1:172.21.15.86:6789/0 conn(0x7f929415e270 legacy=0x7f929415e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:04.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.284+0000 7f9293400640 1 --2- 172.21.15.86:0/3230240734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9278003b50 0x7f9278003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:04.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.284+0000 7f9293400640 1 --2- 172.21.15.86:0/3230240734 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9278004810 0x7f9278004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:04.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.284+0000 7f9293400640 1 --2- 172.21.15.86:0/3230240734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9278005270 0x7f927800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:04.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.284+0000 7f9293400640 1 -- 172.21.15.86:0/3230240734 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f929406d870 con 0x7f9278005270 2024-04-25T14:05:04.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.284+0000 7f9293400640 1 -- 172.21.15.86:0/3230240734 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f927800c1d0 con 0x7f9278003b50 2024-04-25T14:05:04.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.284+0000 7f9293400640 1 -- 172.21.15.86:0/3230240734 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f927800c3d0 con 0x7f9278004810 2024-04-25T14:05:04.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.284+0000 7f9299800640 1 --2- 172.21.15.86:0/3230240734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9278005270 0x7f927800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:04.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.284+0000 7f9298e00640 1 --2- 172.21.15.86:0/3230240734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9278003b50 0x7f9278003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:04.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.284+0000 7f9293e00640 1 --2- 172.21.15.86:0/3230240734 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9278004810 0x7f9278004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:04.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.284+0000 7f9299800640 1 -- 172.21.15.86:0/3230240734 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9278004810 msgr2=0x7f9278004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:04.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.284+0000 7f9299800640 1 --2- 172.21.15.86:0/3230240734 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9278004810 0x7f9278004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:04.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.284+0000 7f9299800640 1 -- 172.21.15.86:0/3230240734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9278003b50 msgr2=0x7f9278003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:04.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.284+0000 7f9299800640 1 --2- 172.21.15.86:0/3230240734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9278003b50 0x7f9278003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:04.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.284+0000 7f9299800640 1 -- 172.21.15.86:0/3230240734 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9278003530 con 0x7f9278005270 2024-04-25T14:05:04.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.284+0000 7f9293e00640 1 --2- 172.21.15.86:0/3230240734 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9278004810 0x7f9278004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:04.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.284+0000 7f9299800640 1 -- 172.21.15.86:0/3230240734 reap_dead start 2024-04-25T14:05:04.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.284+0000 7f9299800640 1 --2- 172.21.15.86:0/3230240734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9278005270 0x7f927800b9d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f928c009f90 tx=0x7f928c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36942791974dd61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:04.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.288+0000 7f9293400640 1 -- 172.21.15.86:0/3230240734 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c009c10 con 0x7f9278005270 2024-04-25T14:05:04.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.288+0000 7f9293400640 1 -- 172.21.15.86:0/3230240734 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f928c0a5c90 con 0x7f9278005270 2024-04-25T14:05:04.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.288+0000 7f9293400640 1 -- 172.21.15.86:0/3230240734 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c003be0 con 0x7f9278005270 2024-04-25T14:05:04.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.288+0000 7f929b800640 1 -- 172.21.15.86:0/3230240734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9278005270 msgr2=0x7f927800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:04.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.288+0000 7f929b800640 1 --2- 172.21.15.86:0/3230240734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9278005270 0x7f927800b9d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f928c009f90 tx=0x7f928c01cc80 comp rx=0 tx=0).stop 2024-04-25T14:05:04.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.288+0000 7f929b800640 1 -- 172.21.15.86:0/3230240734 shutdown_connections 2024-04-25T14:05:04.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.288+0000 7f929b800640 1 --2- 172.21.15.86:0/3230240734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9278005270 0x7f927800b9d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:04.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.288+0000 7f929b800640 1 -- 172.21.15.86:0/3230240734 >> 172.21.15.86:0/3230240734 conn(0x7f92940b4a40 msgr2=0x7f92940b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:04.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.288+0000 7f929b800640 1 -- 172.21.15.86:0/3230240734 shutdown_connections 2024-04-25T14:05:04.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.288+0000 7f929b800640 1 -- 172.21.15.86:0/3230240734 wait complete. 2024-04-25T14:05:04.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.288+0000 7f929b800640 1 Processor -- start 2024-04-25T14:05:04.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.288+0000 7f929b800640 1 -- start start 2024-04-25T14:05:04.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.292+0000 7f929b800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9278005270 0x7f92940b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:04.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.292+0000 7f929b800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9294150340 0x7f92940b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:04.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.292+0000 7f929b800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92940cd910 0x7f92940cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:04.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.292+0000 7f929b800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92940ae630 con 0x7f9294150340 2024-04-25T14:05:04.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.292+0000 7f929b800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f92940ae4b0 con 0x7f92940cd910 2024-04-25T14:05:04.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.292+0000 7f929b800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f92940ae7b0 con 0x7f9278005270 2024-04-25T14:05:04.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.292+0000 7f9293e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9294150340 0x7f92940b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:04.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.292+0000 7f9298e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9278005270 0x7f92940b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:04.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.292+0000 7f9298e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9278005270 0x7f92940b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:56826/0 (socket says 172.21.15.86:56826) 2024-04-25T14:05:04.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.292+0000 7f9299800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92940cd910 0x7f92940cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:04.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.292+0000 7f9298e00640 1 -- 172.21.15.86:0/675973433 learned_addr learned my addr 172.21.15.86:0/675973433 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:04.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.292+0000 7f9293e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9294150340 0x7f92940b28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35630/0 (socket says 172.21.15.86:35630) 2024-04-25T14:05:04.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.292+0000 7f9293e00640 1 -- 172.21.15.86:0/675973433 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9278005270 msgr2=0x7f92940b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:04.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.292+0000 7f9293e00640 1 --2- 172.21.15.86:0/675973433 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9278005270 0x7f92940b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:04.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.292+0000 7f9293e00640 1 -- 172.21.15.86:0/675973433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92940cd910 msgr2=0x7f92940cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:04.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.292+0000 7f9293e00640 1 --2- 172.21.15.86:0/675973433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92940cd910 0x7f92940cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:04.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.292+0000 7f9293e00640 1 -- 172.21.15.86:0/675973433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92940b0fa0 con 0x7f9294150340 2024-04-25T14:05:04.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.292+0000 7f9293e00640 1 --2- 172.21.15.86:0/675973433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9294150340 0x7f92940b28d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f928000e7b0 tx=0x7f928000ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:04.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.292+0000 7f9290c00640 1 -- 172.21.15.86:0/675973433 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928000c6c0 con 0x7f9294150340 2024-04-25T14:05:04.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.292+0000 7f9290c00640 1 -- 172.21.15.86:0/675973433 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9280018070 con 0x7f9294150340 2024-04-25T14:05:04.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.292+0000 7f929b800640 1 -- 172.21.15.86:0/675973433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92940b1280 con 0x7f9294150340 2024-04-25T14:05:04.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.292+0000 7f9290c00640 1 -- 172.21.15.86:0/675973433 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928000ca60 con 0x7f9294150340 2024-04-25T14:05:04.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.292+0000 7f929b800640 1 -- 172.21.15.86:0/675973433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92940b1840 con 0x7f9294150340 2024-04-25T14:05:04.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.292+0000 7f929b800640 1 -- 172.21.15.86:0/675973433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9294050750 con 0x7f9294150340 2024-04-25T14:05:04.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.296+0000 7f9290c00640 1 -- 172.21.15.86:0/675973433 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f928000cc00 con 0x7f9294150340 2024-04-25T14:05:04.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.296+0000 7f9290c00640 1 --2- 172.21.15.86:0/675973433 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f927406f430 0x7f9274071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:04.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.296+0000 7f9290c00640 1 -- 172.21.15.86:0/675973433 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f928008ece0 con 0x7f9294150340 2024-04-25T14:05:04.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.296+0000 7f9298e00640 1 --2- 172.21.15.86:0/675973433 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f927406f430 0x7f9274071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:04.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.296+0000 7f9298e00640 1 --2- 172.21.15.86:0/675973433 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f927406f430 0x7f9274071940 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f92840014f0 tx=0x7f92840013e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:04.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.300+0000 7f9290c00640 1 -- 172.21.15.86:0/675973433 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9280095080 con 0x7f9294150340 2024-04-25T14:05:04.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:04.460+0000 7f929b800640 1 -- 172.21.15.86:0/675973433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f929406f220 con 0x7f9294150340 2024-04-25T14:05:05.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.224+0000 7f9290c00640 1 -- 172.21.15.86:0/675973433 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) ==== 138+0+0 (secure 0 0 0) 0x7f928005bdb0 con 0x7f9294150340 2024-04-25T14:05:05.229 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-04-25T14:05:05.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.228+0000 7f929b800640 1 -- 172.21.15.86:0/675973433 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f927406f430 msgr2=0x7f9274071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:05.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.228+0000 7f929b800640 1 --2- 172.21.15.86:0/675973433 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f927406f430 0x7f9274071940 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f92840014f0 tx=0x7f92840013e0 comp rx=0 tx=0).stop 2024-04-25T14:05:05.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.228+0000 7f929b800640 1 -- 172.21.15.86:0/675973433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9294150340 msgr2=0x7f92940b28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:05.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.228+0000 7f929b800640 1 --2- 172.21.15.86:0/675973433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9294150340 0x7f92940b28d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f928000e7b0 tx=0x7f928000ebb0 comp rx=0 tx=0).stop 2024-04-25T14:05:05.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.228+0000 7f929b800640 1 -- 172.21.15.86:0/675973433 shutdown_connections 2024-04-25T14:05:05.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.228+0000 7f929b800640 1 --2- 172.21.15.86:0/675973433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92940cd910 0x7f92940cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:05.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.228+0000 7f929b800640 1 --2- 172.21.15.86:0/675973433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9294150340 0x7f92940b28d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:05.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.228+0000 7f929b800640 1 --2- 172.21.15.86:0/675973433 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f927406f430 0x7f9274071940 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:05.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.228+0000 7f929b800640 1 --2- 172.21.15.86:0/675973433 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9278005270 0x7f92940b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:05.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.228+0000 7f929b800640 1 -- 172.21.15.86:0/675973433 >> 172.21.15.86:0/675973433 conn(0x7f92940b4a40 msgr2=0x7f929406e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:05.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.232+0000 7f929b800640 1 -- 172.21.15.86:0/675973433 shutdown_connections 2024-04-25T14:05:05.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.232+0000 7f929b800640 1 -- 172.21.15.86:0/675973433 wait complete. 2024-04-25T14:05:05.260 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T14:05:05.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.412+0000 7f0878a00640 1 Processor -- start 2024-04-25T14:05:05.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.412+0000 7f0878a00640 1 -- start start 2024-04-25T14:05:05.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.416+0000 7f0878a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f087415c2a0 0x7f08740ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:05.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.416+0000 7f0878a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08740b0a60 con 0x7f087406fd50 2024-04-25T14:05:05.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.416+0000 7f0878a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f087406ef90 con 0x7f087415feb0 2024-04-25T14:05:05.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.416+0000 7f0878a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f087406f270 con 0x7f087415c2a0 2024-04-25T14:05:05.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.416+0000 7f0871e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f087415c2a0 0x7f08740ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:05.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.416+0000 7f0871400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f087406fd50 0x7f0874070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52160/0 (socket says 172.21.15.86:52160) 2024-04-25T14:05:05.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.416+0000 7f0871400640 1 -- 172.21.15.86:0/934502615 learned_addr learned my addr 172.21.15.86:0/934502615 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:05.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.416+0000 7f0871e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f087415c2a0 0x7f08740ae130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35632/0 (socket says 172.21.15.86:35632) 2024-04-25T14:05:05.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.416+0000 7f0871e00640 1 -- 172.21.15.86:0/934502615 >> v1:172.21.15.155:6789/0 conn(0x7f087415feb0 legacy=0x7f08741602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:05.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.416+0000 7f0871e00640 1 -- 172.21.15.86:0/934502615 >> v1:172.21.15.86:6789/0 conn(0x7f087406fd50 legacy=0x7f0874070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:05.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.416+0000 7f0871e00640 1 -- 172.21.15.86:0/934502615 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f087406f550 con 0x7f087415c2a0 2024-04-25T14:05:05.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.416+0000 7f0870a00640 1 -- 172.21.15.86:0/934502615 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4147290787 0 0) 0x7f08740b0a60 con 0x7f087406fd50 2024-04-25T14:05:05.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.416+0000 7f0871e00640 1 --2- 172.21.15.86:0/934502615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f087415c2a0 0x7f08740ae130 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0864009d90 tx=0x7f086401c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731f766660b06b76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:05.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.416+0000 7f0870a00640 1 -- 172.21.15.86:0/934502615 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08640b2070 con 0x7f087415c2a0 2024-04-25T14:05:05.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.420+0000 7f0870a00640 1 -- 172.21.15.86:0/934502615 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08640a5840 con 0x7f087415c2a0 2024-04-25T14:05:05.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.420+0000 7f0870a00640 1 -- 172.21.15.86:0/934502615 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08640a5c10 con 0x7f087415c2a0 2024-04-25T14:05:05.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.420+0000 7f0878a00640 1 -- 172.21.15.86:0/934502615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f087415c2a0 msgr2=0x7f08740ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:05.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.420+0000 7f0878a00640 1 --2- 172.21.15.86:0/934502615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f087415c2a0 0x7f08740ae130 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0864009d90 tx=0x7f086401c870 comp rx=0 tx=0).stop 2024-04-25T14:05:05.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.420+0000 7f0878a00640 1 -- 172.21.15.86:0/934502615 shutdown_connections 2024-04-25T14:05:05.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.420+0000 7f0878a00640 1 --2- 172.21.15.86:0/934502615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f087415c2a0 0x7f08740ae130 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:05.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.420+0000 7f0878a00640 1 -- 172.21.15.86:0/934502615 >> 172.21.15.86:0/934502615 conn(0x7f08740b4650 msgr2=0x7f08740b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:05.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.420+0000 7f0878a00640 1 -- 172.21.15.86:0/934502615 shutdown_connections 2024-04-25T14:05:05.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.420+0000 7f0878a00640 1 -- 172.21.15.86:0/934502615 wait complete. 2024-04-25T14:05:05.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.420+0000 7f0878a00640 1 Processor -- start 2024-04-25T14:05:05.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.420+0000 7f0878a00640 1 -- start start 2024-04-25T14:05:05.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.424+0000 7f0878a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f087406fd50 0x7f087409a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:05.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.424+0000 7f0878a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f087415c2a0 0x7f08740b3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:05.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.424+0000 7f0878a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f087415feb0 0x7f08740b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:05.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.424+0000 7f0878a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08740ae7f0 con 0x7f087406fd50 2024-04-25T14:05:05.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.424+0000 7f0878a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f08740ae670 con 0x7f087415c2a0 2024-04-25T14:05:05.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.424+0000 7f0878a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f087416aa10 con 0x7f087415feb0 2024-04-25T14:05:05.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.424+0000 7f0871e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f087406fd50 0x7f087409a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:05.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.424+0000 7f0871e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f087406fd50 0x7f087409a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35634/0 (socket says 172.21.15.86:35634) 2024-04-25T14:05:05.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.424+0000 7f0871e00640 1 -- 172.21.15.86:0/3744289727 learned_addr learned my addr 172.21.15.86:0/3744289727 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:05.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.424+0000 7f0872800640 1 --2- 172.21.15.86:0/3744289727 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f087415feb0 0x7f08740b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:05.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.424+0000 7f0871400640 1 --2- 172.21.15.86:0/3744289727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f087415c2a0 0x7f08740b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:05.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.424+0000 7f0871e00640 1 -- 172.21.15.86:0/3744289727 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f087415feb0 msgr2=0x7f08740b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:05.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.424+0000 7f0871e00640 1 --2- 172.21.15.86:0/3744289727 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f087415feb0 0x7f08740b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:05.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.424+0000 7f0871e00640 1 -- 172.21.15.86:0/3744289727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f087415c2a0 msgr2=0x7f08740b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:05.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.424+0000 7f0871e00640 1 --2- 172.21.15.86:0/3744289727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f087415c2a0 0x7f08740b3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:05.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.424+0000 7f0871e00640 1 -- 172.21.15.86:0/3744289727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08741488c0 con 0x7f087406fd50 2024-04-25T14:05:05.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.424+0000 7f0871400640 1 --2- 172.21.15.86:0/3744289727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f087415c2a0 0x7f08740b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:05.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.424+0000 7f0872800640 1 --2- 172.21.15.86:0/3744289727 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f087415feb0 0x7f08740b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:05.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.424+0000 7f0871e00640 1 --2- 172.21.15.86:0/3744289727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f087406fd50 0x7f087409a7b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f08640b0040 tx=0x7f0864019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:05.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.424+0000 7f086a000640 1 -- 172.21.15.86:0/3744289727 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08640bd020 con 0x7f087406fd50 2024-04-25T14:05:05.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.424+0000 7f086a000640 1 -- 172.21.15.86:0/3744289727 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08640b8670 con 0x7f087406fd50 2024-04-25T14:05:05.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.424+0000 7f086a000640 1 -- 172.21.15.86:0/3744289727 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08640b2040 con 0x7f087406fd50 2024-04-25T14:05:05.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.424+0000 7f0878a00640 1 -- 172.21.15.86:0/3744289727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0874148ba0 con 0x7f087406fd50 2024-04-25T14:05:05.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.424+0000 7f0878a00640 1 -- 172.21.15.86:0/3744289727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f087415c900 con 0x7f087406fd50 2024-04-25T14:05:05.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.424+0000 7f0878a00640 1 -- 172.21.15.86:0/3744289727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0874069d00 con 0x7f087406fd50 2024-04-25T14:05:05.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.428+0000 7f086a000640 1 -- 172.21.15.86:0/3744289727 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f08640c0020 con 0x7f087406fd50 2024-04-25T14:05:05.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.428+0000 7f086a000640 1 --2- 172.21.15.86:0/3744289727 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f085406f400 0x7f0854071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:05.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.428+0000 7f086a000640 1 -- 172.21.15.86:0/3744289727 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5619+0+0 (secure 0 0 0) 0x7f086412a2c0 con 0x7f087406fd50 2024-04-25T14:05:05.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.428+0000 7f0871400640 1 --2- 172.21.15.86:0/3744289727 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f085406f400 0x7f0854071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:05.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.428+0000 7f0871400640 1 --2- 172.21.15.86:0/3744289727 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f085406f400 0x7f0854071910 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f087414a910 tx=0x7f0860009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:05.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.432+0000 7f086a000640 1 -- 172.21.15.86:0/3744289727 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f08640c7030 con 0x7f087406fd50 2024-04-25T14:05:05.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:05.592+0000 7f0878a00640 1 -- 172.21.15.86:0/3744289727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f0874167fd0 con 0x7f087406fd50 2024-04-25T14:05:06.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.228+0000 7f086a000640 1 -- 172.21.15.86:0/3744289727 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) ==== 170+0+0 (secure 0 0 0) 0x7f0864016520 con 0x7f087406fd50 2024-04-25T14:05:06.232 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-04-25T14:05:06.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.232+0000 7f0878a00640 1 -- 172.21.15.86:0/3744289727 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f085406f400 msgr2=0x7f0854071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:06.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.232+0000 7f0878a00640 1 --2- 172.21.15.86:0/3744289727 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f085406f400 0x7f0854071910 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f087414a910 tx=0x7f0860009040 comp rx=0 tx=0).stop 2024-04-25T14:05:06.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.232+0000 7f0878a00640 1 -- 172.21.15.86:0/3744289727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f087406fd50 msgr2=0x7f087409a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:06.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.232+0000 7f0878a00640 1 --2- 172.21.15.86:0/3744289727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f087406fd50 0x7f087409a7b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f08640b0040 tx=0x7f0864019610 comp rx=0 tx=0).stop 2024-04-25T14:05:06.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.232+0000 7f0878a00640 1 -- 172.21.15.86:0/3744289727 shutdown_connections 2024-04-25T14:05:06.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.232+0000 7f0878a00640 1 --2- 172.21.15.86:0/3744289727 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f085406f400 0x7f0854071910 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:06.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.232+0000 7f0878a00640 1 --2- 172.21.15.86:0/3744289727 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f087415feb0 0x7f08740b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:06.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.232+0000 7f0878a00640 1 --2- 172.21.15.86:0/3744289727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f087415c2a0 0x7f08740b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:06.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.232+0000 7f0878a00640 1 --2- 172.21.15.86:0/3744289727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f087406fd50 0x7f087409a7b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:06.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.232+0000 7f0878a00640 1 -- 172.21.15.86:0/3744289727 >> 172.21.15.86:0/3744289727 conn(0x7f08740b4650 msgr2=0x7f087415b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:06.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.236+0000 7f0878a00640 1 -- 172.21.15.86:0/3744289727 shutdown_connections 2024-04-25T14:05:06.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.236+0000 7f0878a00640 1 -- 172.21.15.86:0/3744289727 wait complete. 2024-04-25T14:05:06.276 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T14:05:06.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.424+0000 7fd9f1c00640 1 Processor -- start 2024-04-25T14:05:06.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.424+0000 7fd9f1c00640 1 -- start start 2024-04-25T14:05:06.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.424+0000 7fd9f1c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd9ec071530 0x7fd9ec06e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:06.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.424+0000 7fd9f1c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9ec06e7a0 0x7fd9ec0cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:06.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.424+0000 7fd9f1c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9ec06ef90 con 0x7fd9ec0cf970 2024-04-25T14:05:06.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.424+0000 7fd9f1c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd9ec163830 con 0x7fd9ec071530 2024-04-25T14:05:06.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.424+0000 7fd9f1c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd9ec163a80 con 0x7fd9ec06e7a0 2024-04-25T14:05:06.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.424+0000 7fd9ea000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd9ec071530 0x7fd9ec06e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:06.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.424+0000 7fd9eaa00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd9ec0cf970 0x7fd9ec070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52166/0 (socket says 172.21.15.86:52166) 2024-04-25T14:05:06.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.424+0000 7fd9eaa00640 1 -- 172.21.15.86:0/1554736864 learned_addr learned my addr 172.21.15.86:0/1554736864 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:06.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.424+0000 7fd9eb400640 1 --2- 172.21.15.86:0/1554736864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9ec06e7a0 0x7fd9ec0cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:06.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.428+0000 7fd9ea000640 1 -- 172.21.15.86:0/1554736864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9ec06e7a0 msgr2=0x7fd9ec0cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:06.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.428+0000 7fd9ea000640 1 --2- 172.21.15.86:0/1554736864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9ec06e7a0 0x7fd9ec0cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:06.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.428+0000 7fd9ea000640 1 -- 172.21.15.86:0/1554736864 >> v1:172.21.15.86:6789/0 conn(0x7fd9ec0cf970 legacy=0x7fd9ec070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:06.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.428+0000 7fd9ea000640 1 -- 172.21.15.86:0/1554736864 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9ec163d60 con 0x7fd9ec071530 2024-04-25T14:05:06.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.428+0000 7fd9e9600640 1 -- 172.21.15.86:0/1554736864 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 916439869 0 0) 0x7fd9ec06ef90 con 0x7fd9ec0cf970 2024-04-25T14:05:06.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.428+0000 7fd9ea000640 1 --2- 172.21.15.86:0/1554736864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9ec071530 0x7fd9ec06e260 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80097a0 tx=0x7fd9d801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8980a4c19380fd05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:06.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.428+0000 7fd9e9600640 1 -- 172.21.15.86:0/1554736864 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d80a7070 con 0x7fd9ec071530 2024-04-25T14:05:06.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.428+0000 7fd9e9600640 1 -- 172.21.15.86:0/1554736864 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9d80b0070 con 0x7fd9ec071530 2024-04-25T14:05:06.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.428+0000 7fd9e9600640 1 -- 172.21.15.86:0/1554736864 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d80ab890 con 0x7fd9ec071530 2024-04-25T14:05:06.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.432+0000 7fd9f1c00640 1 -- 172.21.15.86:0/1554736864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9ec071530 msgr2=0x7fd9ec06e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:06.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.432+0000 7fd9f1c00640 1 --2- 172.21.15.86:0/1554736864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9ec071530 0x7fd9ec06e260 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80097a0 tx=0x7fd9d801c450 comp rx=0 tx=0).stop 2024-04-25T14:05:06.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.432+0000 7fd9f1c00640 1 -- 172.21.15.86:0/1554736864 shutdown_connections 2024-04-25T14:05:06.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.432+0000 7fd9f1c00640 1 --2- 172.21.15.86:0/1554736864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9ec06e7a0 0x7fd9ec0cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:06.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.432+0000 7fd9f1c00640 1 --2- 172.21.15.86:0/1554736864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9ec071530 0x7fd9ec06e260 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:06.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.432+0000 7fd9f1c00640 1 -- 172.21.15.86:0/1554736864 >> 172.21.15.86:0/1554736864 conn(0x7fd9ec0cffc0 msgr2=0x7fd9ec0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:06.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.432+0000 7fd9f1c00640 1 -- 172.21.15.86:0/1554736864 shutdown_connections 2024-04-25T14:05:06.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.432+0000 7fd9f1c00640 1 -- 172.21.15.86:0/1554736864 wait complete. 2024-04-25T14:05:06.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.432+0000 7fd9f1c00640 1 Processor -- start 2024-04-25T14:05:06.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.432+0000 7fd9f1c00640 1 -- start start 2024-04-25T14:05:06.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.432+0000 7fd9f1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9ec06e7a0 0x7fd9ec15fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:06.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.432+0000 7fd9f1c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9ec0cf970 0x7fd9ec157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:06.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.432+0000 7fd9f1c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9ec0caf00 0x7fd9ec161840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:06.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.432+0000 7fd9f1c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec16cbd0 con 0x7fd9ec06e7a0 2024-04-25T14:05:06.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.432+0000 7fd9f1c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec0ced80 con 0x7fd9ec0cf970 2024-04-25T14:05:06.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.432+0000 7fd9f1c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd9ec16cd50 con 0x7fd9ec0caf00 2024-04-25T14:05:06.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.436+0000 7fd9eaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9ec06e7a0 0x7fd9ec15fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:06.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.436+0000 7fd9eaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9ec06e7a0 0x7fd9ec15fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35648/0 (socket says 172.21.15.86:35648) 2024-04-25T14:05:06.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.436+0000 7fd9eaa00640 1 -- 172.21.15.86:0/3162687319 learned_addr learned my addr 172.21.15.86:0/3162687319 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:06.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.436+0000 7fd9ea000640 1 --2- 172.21.15.86:0/3162687319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9ec0cf970 0x7fd9ec157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:06.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.436+0000 7fd9eb400640 1 --2- 172.21.15.86:0/3162687319 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9ec0caf00 0x7fd9ec161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:06.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.436+0000 7fd9eaa00640 1 -- 172.21.15.86:0/3162687319 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9ec0caf00 msgr2=0x7fd9ec161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:06.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.436+0000 7fd9eaa00640 1 --2- 172.21.15.86:0/3162687319 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9ec0caf00 0x7fd9ec161840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:06.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.436+0000 7fd9eaa00640 1 -- 172.21.15.86:0/3162687319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9ec0cf970 msgr2=0x7fd9ec157a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:06.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.436+0000 7fd9eaa00640 1 --2- 172.21.15.86:0/3162687319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9ec0cf970 0x7fd9ec157a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:06.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.436+0000 7fd9eaa00640 1 -- 172.21.15.86:0/3162687319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9ec157fa0 con 0x7fd9ec06e7a0 2024-04-25T14:05:06.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.436+0000 7fd9eb400640 1 --2- 172.21.15.86:0/3162687319 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9ec0caf00 0x7fd9ec161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:06.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.436+0000 7fd9eaa00640 1 --2- 172.21.15.86:0/3162687319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9ec06e7a0 0x7fd9ec15fd50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc00c7c0 tx=0x7fd9dc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:06.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.436+0000 7fd9e2a00640 1 -- 172.21.15.86:0/3162687319 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc018070 con 0x7fd9ec06e7a0 2024-04-25T14:05:06.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.436+0000 7fd9e2a00640 1 -- 172.21.15.86:0/3162687319 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9dc00a5e0 con 0x7fd9ec06e7a0 2024-04-25T14:05:06.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.436+0000 7fd9f1c00640 1 -- 172.21.15.86:0/3162687319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9ec15ecb0 con 0x7fd9ec06e7a0 2024-04-25T14:05:06.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.436+0000 7fd9e2a00640 1 -- 172.21.15.86:0/3162687319 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc00a9d0 con 0x7fd9ec06e7a0 2024-04-25T14:05:06.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.436+0000 7fd9f1c00640 1 -- 172.21.15.86:0/3162687319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9ec15f270 con 0x7fd9ec06e7a0 2024-04-25T14:05:06.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.436+0000 7fd9f1c00640 1 -- 172.21.15.86:0/3162687319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9ec0720e0 con 0x7fd9ec06e7a0 2024-04-25T14:05:06.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.436+0000 7fd9e2a00640 1 -- 172.21.15.86:0/3162687319 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd9dc01b070 con 0x7fd9ec06e7a0 2024-04-25T14:05:06.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.440+0000 7fd9e2a00640 1 --2- 172.21.15.86:0/3162687319 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd9cc06f430 0x7fd9cc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:06.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.440+0000 7fd9e2a00640 1 -- 172.21.15.86:0/3162687319 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6001+0+0 (secure 0 0 0) 0x7fd9dc08ec50 con 0x7fd9ec06e7a0 2024-04-25T14:05:06.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.440+0000 7fd9ea000640 1 --2- 172.21.15.86:0/3162687319 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd9cc06f430 0x7fd9cc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:06.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.440+0000 7fd9ea000640 1 --2- 172.21.15.86:0/3162687319 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd9cc06f430 0x7fd9cc071940 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80af000 tx=0x7fd9d80a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:06.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.444+0000 7fd9e2a00640 1 -- 172.21.15.86:0/3162687319 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd9dc05b4a0 con 0x7fd9ec06e7a0 2024-04-25T14:05:06.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:06.632+0000 7fd9f1c00640 1 -- 172.21.15.86:0/3162687319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd9ec0b94c0 con 0x7fd9ec06e7a0 2024-04-25T14:05:07.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.260+0000 7fd9e2a00640 1 -- 172.21.15.86:0/3162687319 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7fd9dc05f7a0 con 0x7fd9ec06e7a0 2024-04-25T14:05:07.266 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 5 and data pool 6 2024-04-25T14:05:07.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.280+0000 7fd9f1c00640 1 -- 172.21.15.86:0/3162687319 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd9cc06f430 msgr2=0x7fd9cc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:07.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.280+0000 7fd9f1c00640 1 --2- 172.21.15.86:0/3162687319 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd9cc06f430 0x7fd9cc071940 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80af000 tx=0x7fd9d80a7000 comp rx=0 tx=0).stop 2024-04-25T14:05:07.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.280+0000 7fd9f1c00640 1 -- 172.21.15.86:0/3162687319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9ec06e7a0 msgr2=0x7fd9ec15fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:07.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.280+0000 7fd9f1c00640 1 --2- 172.21.15.86:0/3162687319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9ec06e7a0 0x7fd9ec15fd50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc00c7c0 tx=0x7fd9dc00cc80 comp rx=0 tx=0).stop 2024-04-25T14:05:07.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.280+0000 7fd9f1c00640 1 -- 172.21.15.86:0/3162687319 shutdown_connections 2024-04-25T14:05:07.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.280+0000 7fd9f1c00640 1 --2- 172.21.15.86:0/3162687319 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd9cc06f430 0x7fd9cc071940 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:07.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.280+0000 7fd9f1c00640 1 --2- 172.21.15.86:0/3162687319 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9ec0caf00 0x7fd9ec161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:07.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.280+0000 7fd9f1c00640 1 --2- 172.21.15.86:0/3162687319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9ec0cf970 0x7fd9ec157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:07.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.280+0000 7fd9f1c00640 1 --2- 172.21.15.86:0/3162687319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9ec06e7a0 0x7fd9ec15fd50 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:07.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.280+0000 7fd9f1c00640 1 -- 172.21.15.86:0/3162687319 >> 172.21.15.86:0/3162687319 conn(0x7fd9ec0cffc0 msgr2=0x7fd9ec0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:07.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.280+0000 7fd9f1c00640 1 -- 172.21.15.86:0/3162687319 shutdown_connections 2024-04-25T14:05:07.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.280+0000 7fd9f1c00640 1 -- 172.21.15.86:0/3162687319 wait complete. 2024-04-25T14:05:07.310 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:05:07.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.456+0000 7f8cc9400640 1 Processor -- start 2024-04-25T14:05:07.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.456+0000 7f8cc9400640 1 -- start start 2024-04-25T14:05:07.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.456+0000 7f8cc9400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8cc40cdda0 0x7f8cc40cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:07.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.456+0000 7f8cc9400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cc40d18e0 0x7f8cc40d1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:07.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.456+0000 7f8cc9400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cc4163940 con 0x7f8cc40cec60 2024-04-25T14:05:07.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.456+0000 7f8cc9400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8cc40cfcc0 con 0x7f8cc40cdda0 2024-04-25T14:05:07.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.456+0000 7f8cc9400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8cc40d2f40 con 0x7f8cc40d18e0 2024-04-25T14:05:07.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.456+0000 7f8cc1e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8cc40cdda0 0x7f8cc40cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:07.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.456+0000 7f8cc2800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8cc40cec60 0x7f8cc40cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52180/0 (socket says 172.21.15.86:52180) 2024-04-25T14:05:07.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.456+0000 7f8cc1e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8cc40cdda0 0x7f8cc40cf780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35662/0 (socket says 172.21.15.86:35662) 2024-04-25T14:05:07.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.456+0000 7f8cc2800640 1 -- 172.21.15.86:0/2841628524 learned_addr learned my addr 172.21.15.86:0/2841628524 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:07.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.456+0000 7f8cc3200640 1 --2- 172.21.15.86:0/2841628524 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cc40d18e0 0x7f8cc40d1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:07.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.456+0000 7f8cc1e00640 1 -- 172.21.15.86:0/2841628524 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cc40d18e0 msgr2=0x7f8cc40d1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:07.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.456+0000 7f8cc1e00640 1 --2- 172.21.15.86:0/2841628524 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cc40d18e0 0x7f8cc40d1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:07.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.456+0000 7f8cc1e00640 1 -- 172.21.15.86:0/2841628524 >> v1:172.21.15.86:6789/0 conn(0x7f8cc40cec60 legacy=0x7f8cc40cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:07.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.456+0000 7f8cc1e00640 1 -- 172.21.15.86:0/2841628524 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc40b3720 con 0x7f8cc40cdda0 2024-04-25T14:05:07.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.456+0000 7f8cc1400640 1 -- 172.21.15.86:0/2841628524 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 92984404 0 0) 0x7f8cc4163940 con 0x7f8cc40cec60 2024-04-25T14:05:07.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.460+0000 7f8cc1e00640 1 --2- 172.21.15.86:0/2841628524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cc40cdda0 0x7f8cc40cf780 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00097c0 tx=0x7f8cb001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15978efbcbfaa0a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:07.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.460+0000 7f8cc1400640 1 -- 172.21.15.86:0/2841628524 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb00b0070 con 0x7f8cc40cdda0 2024-04-25T14:05:07.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.460+0000 7f8cc1400640 1 -- 172.21.15.86:0/2841628524 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cb00ab600 con 0x7f8cc40cdda0 2024-04-25T14:05:07.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.460+0000 7f8cc1400640 1 -- 172.21.15.86:0/2841628524 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb00ab9d0 con 0x7f8cc40cdda0 2024-04-25T14:05:07.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.460+0000 7f8cc9400640 1 -- 172.21.15.86:0/2841628524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cc40cdda0 msgr2=0x7f8cc40cf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:07.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.460+0000 7f8cc9400640 1 --2- 172.21.15.86:0/2841628524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cc40cdda0 0x7f8cc40cf780 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00097c0 tx=0x7f8cb001c4a0 comp rx=0 tx=0).stop 2024-04-25T14:05:07.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.460+0000 7f8cc9400640 1 -- 172.21.15.86:0/2841628524 shutdown_connections 2024-04-25T14:05:07.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.460+0000 7f8cc9400640 1 --2- 172.21.15.86:0/2841628524 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cc40d18e0 0x7f8cc40d1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:07.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.460+0000 7f8cc9400640 1 --2- 172.21.15.86:0/2841628524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cc40cdda0 0x7f8cc40cf780 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:07.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.460+0000 7f8cc9400640 1 -- 172.21.15.86:0/2841628524 >> 172.21.15.86:0/2841628524 conn(0x7f8cc409ba30 msgr2=0x7f8cc40b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:07.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.460+0000 7f8cc9400640 1 -- 172.21.15.86:0/2841628524 shutdown_connections 2024-04-25T14:05:07.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.460+0000 7f8cc9400640 1 -- 172.21.15.86:0/2841628524 wait complete. 2024-04-25T14:05:07.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.464+0000 7f8cc9400640 1 Processor -- start 2024-04-25T14:05:07.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.464+0000 7f8cc9400640 1 -- start start 2024-04-25T14:05:07.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.464+0000 7f8cc9400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cc40cdda0 0x7f8cc41fc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:07.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.464+0000 7f8cc9400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cc40cec60 0x7f8cc415b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:07.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.464+0000 7f8cc9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cc40d18e0 0x7f8cc415b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:07.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.464+0000 7f8cc9400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc416cbf0 con 0x7f8cc40d18e0 2024-04-25T14:05:07.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.464+0000 7f8cc9400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc40d2c50 con 0x7f8cc40cdda0 2024-04-25T14:05:07.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.464+0000 7f8cc9400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8cc416cd70 con 0x7f8cc40cec60 2024-04-25T14:05:07.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.464+0000 7f8cc3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cc40d18e0 0x7f8cc415b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:07.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.464+0000 7f8cc3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cc40d18e0 0x7f8cc415b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35666/0 (socket says 172.21.15.86:35666) 2024-04-25T14:05:07.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.464+0000 7f8cc3200640 1 -- 172.21.15.86:0/1840698869 learned_addr learned my addr 172.21.15.86:0/1840698869 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:07.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.464+0000 7f8cc2800640 1 --2- 172.21.15.86:0/1840698869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cc40cdda0 0x7f8cc41fc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:07.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.464+0000 7f8cc1e00640 1 --2- 172.21.15.86:0/1840698869 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cc40cec60 0x7f8cc415b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:07.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.464+0000 7f8cc3200640 1 -- 172.21.15.86:0/1840698869 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cc40cec60 msgr2=0x7f8cc415b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:07.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.464+0000 7f8cc3200640 1 --2- 172.21.15.86:0/1840698869 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cc40cec60 0x7f8cc415b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:07.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.464+0000 7f8cc3200640 1 -- 172.21.15.86:0/1840698869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cc40cdda0 msgr2=0x7f8cc41fc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:07.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.464+0000 7f8cc3200640 1 --2- 172.21.15.86:0/1840698869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cc40cdda0 0x7f8cc41fc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:07.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.464+0000 7f8cc3200640 1 -- 172.21.15.86:0/1840698869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc41fa370 con 0x7f8cc40d18e0 2024-04-25T14:05:07.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.468+0000 7f8cc3200640 1 --2- 172.21.15.86:0/1840698869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cc40d18e0 0x7f8cc415b6b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009cb0 tx=0x7f8cbc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:07.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.468+0000 7f8cbaa00640 1 -- 172.21.15.86:0/1840698869 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc018020 con 0x7f8cc40d18e0 2024-04-25T14:05:07.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.468+0000 7f8cbaa00640 1 -- 172.21.15.86:0/1840698869 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cbc014480 con 0x7f8cc40d18e0 2024-04-25T14:05:07.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.468+0000 7f8cc9400640 1 -- 172.21.15.86:0/1840698869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cc40c7a10 con 0x7f8cc40d18e0 2024-04-25T14:05:07.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.468+0000 7f8cbaa00640 1 -- 172.21.15.86:0/1840698869 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc014870 con 0x7f8cc40d18e0 2024-04-25T14:05:07.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.468+0000 7f8cc9400640 1 -- 172.21.15.86:0/1840698869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cc4159180 con 0x7f8cc40d18e0 2024-04-25T14:05:07.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.468+0000 7f8cc9400640 1 -- 172.21.15.86:0/1840698869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cc40c6170 con 0x7f8cc40d18e0 2024-04-25T14:05:07.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.468+0000 7f8cbaa00640 1 -- 172.21.15.86:0/1840698869 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8cbc01b020 con 0x7f8cc40d18e0 2024-04-25T14:05:07.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.468+0000 7f8cbaa00640 1 --2- 172.21.15.86:0/1840698869 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8ca406f400 0x7f8ca4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:07.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.468+0000 7f8cbaa00640 1 -- 172.21.15.86:0/1840698869 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7f8cbc08ffe0 con 0x7f8cc40d18e0 2024-04-25T14:05:07.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.468+0000 7f8cc2800640 1 --2- 172.21.15.86:0/1840698869 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8ca406f400 0x7f8ca4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:07.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.472+0000 7f8cc2800640 1 --2- 172.21.15.86:0/1840698869 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8ca406f400 0x7f8ca4071910 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4007fd0 tx=0x7f8cb4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:07.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.472+0000 7f8cbaa00640 1 -- 172.21.15.86:0/1840698869 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8cbc022030 con 0x7f8cc40d18e0 2024-04-25T14:05:07.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.632+0000 7f8cc9400640 1 -- 172.21.15.86:0/1840698869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8cc40a04c0 con 0x7f8cc40d18e0 2024-04-25T14:05:07.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.632+0000 7f8cbaa00640 1 -- 172.21.15.86:0/1840698869 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+16955 (secure 0 0 0) 0x7f8cbc010040 con 0x7f8cc40d18e0 2024-04-25T14:05:07.636 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:07.636 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":37,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:05:06.638260+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:05:04.472914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T14:05:05.604280+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:05:07.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.636+0000 7f8cc9400640 1 -- 172.21.15.86:0/1840698869 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8ca406f400 msgr2=0x7f8ca4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:07.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.636+0000 7f8cc9400640 1 --2- 172.21.15.86:0/1840698869 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8ca406f400 0x7f8ca4071910 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4007fd0 tx=0x7f8cb4009040 comp rx=0 tx=0).stop 2024-04-25T14:05:07.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.636+0000 7f8cc9400640 1 -- 172.21.15.86:0/1840698869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cc40d18e0 msgr2=0x7f8cc415b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:07.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.636+0000 7f8cc9400640 1 --2- 172.21.15.86:0/1840698869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cc40d18e0 0x7f8cc415b6b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009cb0 tx=0x7f8cbc00c490 comp rx=0 tx=0).stop 2024-04-25T14:05:07.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.636+0000 7f8cc9400640 1 -- 172.21.15.86:0/1840698869 shutdown_connections 2024-04-25T14:05:07.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.636+0000 7f8cc9400640 1 --2- 172.21.15.86:0/1840698869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cc40d18e0 0x7f8cc415b6b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:07.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.636+0000 7f8cc9400640 1 --2- 172.21.15.86:0/1840698869 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f8ca406f400 0x7f8ca4071910 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:07.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.636+0000 7f8cc9400640 1 --2- 172.21.15.86:0/1840698869 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cc40cec60 0x7f8cc415b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:07.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.636+0000 7f8cc9400640 1 --2- 172.21.15.86:0/1840698869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cc40cdda0 0x7f8cc41fc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:07.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.636+0000 7f8cc9400640 1 -- 172.21.15.86:0/1840698869 >> 172.21.15.86:0/1840698869 conn(0x7f8cc409ba30 msgr2=0x7f8cc4068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:07.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.636+0000 7f8cc9400640 1 -- 172.21.15.86:0/1840698869 shutdown_connections 2024-04-25T14:05:07.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.636+0000 7f8cc9400640 1 -- 172.21.15.86:0/1840698869 wait complete. 2024-04-25T14:05:07.658 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:05:07.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.800+0000 7fa4f1c00640 1 Processor -- start 2024-04-25T14:05:07.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.800+0000 7fa4f1c00640 1 -- start start 2024-04-25T14:05:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.804+0000 7fa4f1c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4ec0cfca0 0x7fa4ec06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.804+0000 7fa4f1c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa4ec0d16e0 0x7fa4ec0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.804+0000 7fa4f1c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4ec0d2030 con 0x7fa4ec06c670 2024-04-25T14:05:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.804+0000 7fa4f1c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa4ec070470 con 0x7fa4ec0d16e0 2024-04-25T14:05:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.804+0000 7fa4f1c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa4ec072bd0 con 0x7fa4ec0cfca0 2024-04-25T14:05:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.804+0000 7fa4e2000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa4ec0d16e0 0x7fa4ec0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.804+0000 7fa4eb400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa4ec06c670 0x7fa4ec06ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52192/0 (socket says 172.21.15.86:52192) 2024-04-25T14:05:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.804+0000 7fa4eb400640 1 -- 172.21.15.86:0/1154778123 learned_addr learned my addr 172.21.15.86:0/1154778123 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.804+0000 7fa4eaa00640 1 --2- 172.21.15.86:0/1154778123 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4ec0cfca0 0x7fa4ec06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.804+0000 7fa4e2000640 1 -- 172.21.15.86:0/1154778123 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4ec0cfca0 msgr2=0x7fa4ec06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.804+0000 7fa4e2000640 1 --2- 172.21.15.86:0/1154778123 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4ec0cfca0 0x7fa4ec06f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.804+0000 7fa4e2000640 1 -- 172.21.15.86:0/1154778123 >> v1:172.21.15.86:6789/0 conn(0x7fa4ec06c670 legacy=0x7fa4ec06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.804+0000 7fa4e2000640 1 -- 172.21.15.86:0/1154778123 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ec06f930 con 0x7fa4ec0d16e0 2024-04-25T14:05:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.804+0000 7fa4ea000640 1 -- 172.21.15.86:0/1154778123 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1572028745 0 0) 0x7fa4e40014a0 con 0x7fa4ec06c670 2024-04-25T14:05:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.808+0000 7fa4e2000640 1 --2- 172.21.15.86:0/1154778123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4ec0d16e0 0x7fa4ec0d1af0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40097c0 tx=0x7fa4d401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2f777ed9e3957 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.808+0000 7fa4ea000640 1 -- 172.21.15.86:0/1154778123 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d40b1070 con 0x7fa4ec0d16e0 2024-04-25T14:05:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.808+0000 7fa4ea000640 1 -- 172.21.15.86:0/1154778123 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4d40ac410 con 0x7fa4ec0d16e0 2024-04-25T14:05:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.808+0000 7fa4ea000640 1 -- 172.21.15.86:0/1154778123 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d40ac7e0 con 0x7fa4ec0d16e0 2024-04-25T14:05:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.808+0000 7fa4f1c00640 1 -- 172.21.15.86:0/1154778123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4ec0d16e0 msgr2=0x7fa4ec0d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.808+0000 7fa4f1c00640 1 --2- 172.21.15.86:0/1154778123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4ec0d16e0 0x7fa4ec0d1af0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40097c0 tx=0x7fa4d401c4a0 comp rx=0 tx=0).stop 2024-04-25T14:05:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.808+0000 7fa4f1c00640 1 -- 172.21.15.86:0/1154778123 shutdown_connections 2024-04-25T14:05:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.808+0000 7fa4f1c00640 1 --2- 172.21.15.86:0/1154778123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4ec0d16e0 0x7fa4ec0d1af0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.808+0000 7fa4f1c00640 1 --2- 172.21.15.86:0/1154778123 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4ec0cfca0 0x7fa4ec06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.808+0000 7fa4f1c00640 1 -- 172.21.15.86:0/1154778123 >> 172.21.15.86:0/1154778123 conn(0x7fa4ec0b4650 msgr2=0x7fa4ec0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.808+0000 7fa4f1c00640 1 -- 172.21.15.86:0/1154778123 shutdown_connections 2024-04-25T14:05:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.808+0000 7fa4f1c00640 1 -- 172.21.15.86:0/1154778123 wait complete. 2024-04-25T14:05:07.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.808+0000 7fa4f1c00640 1 Processor -- start 2024-04-25T14:05:07.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.808+0000 7fa4f1c00640 1 -- start start 2024-04-25T14:05:07.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.812+0000 7fa4f1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4ec06c670 0x7fa4ec15b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:07.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.812+0000 7fa4f1c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4ec0cfca0 0x7fa4ec09a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:07.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.812+0000 7fa4f1c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4ec0d16e0 0x7fa4ec068590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:07.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.812+0000 7fa4f1c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec16a070 con 0x7fa4ec06c670 2024-04-25T14:05:07.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.812+0000 7fa4f1c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec169ef0 con 0x7fa4ec0cfca0 2024-04-25T14:05:07.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.812+0000 7fa4f1c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa4ec16a1f0 con 0x7fa4ec0d16e0 2024-04-25T14:05:07.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.812+0000 7fa4eaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4ec06c670 0x7fa4ec15b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:07.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.812+0000 7fa4eaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4ec06c670 0x7fa4ec15b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35682/0 (socket says 172.21.15.86:35682) 2024-04-25T14:05:07.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.812+0000 7fa4eaa00640 1 -- 172.21.15.86:0/2490133526 learned_addr learned my addr 172.21.15.86:0/2490133526 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:07.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.812+0000 7fa4e2000640 1 --2- 172.21.15.86:0/2490133526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4ec0cfca0 0x7fa4ec09a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:07.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.812+0000 7fa4eaa00640 1 -- 172.21.15.86:0/2490133526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4ec0d16e0 msgr2=0x7fa4ec068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:07.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.812+0000 7fa4eb400640 1 --2- 172.21.15.86:0/2490133526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4ec0d16e0 0x7fa4ec068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:07.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.812+0000 7fa4eaa00640 1 --2- 172.21.15.86:0/2490133526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4ec0d16e0 0x7fa4ec068590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:07.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.812+0000 7fa4eaa00640 1 -- 172.21.15.86:0/2490133526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4ec0cfca0 msgr2=0x7fa4ec09a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:07.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.812+0000 7fa4eaa00640 1 --2- 172.21.15.86:0/2490133526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4ec0cfca0 0x7fa4ec09a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:07.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.812+0000 7fa4eaa00640 1 -- 172.21.15.86:0/2490133526 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ec0ae1d0 con 0x7fa4ec06c670 2024-04-25T14:05:07.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.812+0000 7fa4e2000640 1 --2- 172.21.15.86:0/2490133526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4ec0cfca0 0x7fa4ec09a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:07.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.812+0000 7fa4eb400640 1 --2- 172.21.15.86:0/2490133526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4ec0d16e0 0x7fa4ec068590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:07.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.812+0000 7fa4eaa00640 1 --2- 172.21.15.86:0/2490133526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4ec06c670 0x7fa4ec15b8b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc00e840 tx=0x7fa4dc00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:07.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.812+0000 7fa4e3400640 1 -- 172.21.15.86:0/2490133526 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc017070 con 0x7fa4ec06c670 2024-04-25T14:05:07.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.812+0000 7fa4e3400640 1 -- 172.21.15.86:0/2490133526 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4dc00ee90 con 0x7fa4ec06c670 2024-04-25T14:05:07.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.812+0000 7fa4f1c00640 1 -- 172.21.15.86:0/2490133526 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4ec09ae70 con 0x7fa4ec06c670 2024-04-25T14:05:07.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.812+0000 7fa4e3400640 1 -- 172.21.15.86:0/2490133526 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc00cb60 con 0x7fa4ec06c670 2024-04-25T14:05:07.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.812+0000 7fa4f1c00640 1 -- 172.21.15.86:0/2490133526 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4ec0ae720 con 0x7fa4ec06c670 2024-04-25T14:05:07.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.812+0000 7fa4f1c00640 1 -- 172.21.15.86:0/2490133526 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4ec09f4b0 con 0x7fa4ec06c670 2024-04-25T14:05:07.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.816+0000 7fa4e3400640 1 -- 172.21.15.86:0/2490133526 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa4dc01a070 con 0x7fa4ec06c670 2024-04-25T14:05:07.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.816+0000 7fa4e3400640 1 --2- 172.21.15.86:0/2490133526 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa4c806f430 0x7fa4c8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:07.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.816+0000 7fa4e3400640 1 -- 172.21.15.86:0/2490133526 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7fa4dc08f260 con 0x7fa4ec06c670 2024-04-25T14:05:07.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.816+0000 7fa4e2000640 1 --2- 172.21.15.86:0/2490133526 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa4c806f430 0x7fa4c8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:07.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.816+0000 7fa4e2000640 1 --2- 172.21.15.86:0/2490133526 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa4c806f430 0x7fa4c8071940 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40ae000 tx=0x7fa4d40a6040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:07.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.820+0000 7fa4e3400640 1 -- 172.21.15.86:0/2490133526 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa4dc05c050 con 0x7fa4ec06c670 2024-04-25T14:05:07.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.984+0000 7fa4f1c00640 1 -- 172.21.15.86:0/2490133526 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4ec069d00 con 0x7fa4ec06c670 2024-04-25T14:05:07.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.984+0000 7fa4e3400640 1 -- 172.21.15.86:0/2490133526 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+16955 (secure 0 0 0) 0x7fa4dc05b950 con 0x7fa4ec06c670 2024-04-25T14:05:07.987 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:07.988 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":37,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:05:06.638260+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:05:04.472914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T14:05:05.604280+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:05:07.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.988+0000 7fa4f1c00640 1 -- 172.21.15.86:0/2490133526 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa4c806f430 msgr2=0x7fa4c8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:07.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.988+0000 7fa4f1c00640 1 --2- 172.21.15.86:0/2490133526 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa4c806f430 0x7fa4c8071940 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40ae000 tx=0x7fa4d40a6040 comp rx=0 tx=0).stop 2024-04-25T14:05:07.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.988+0000 7fa4f1c00640 1 -- 172.21.15.86:0/2490133526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4ec06c670 msgr2=0x7fa4ec15b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:07.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.988+0000 7fa4f1c00640 1 --2- 172.21.15.86:0/2490133526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4ec06c670 0x7fa4ec15b8b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc00e840 tx=0x7fa4dc00ec40 comp rx=0 tx=0).stop 2024-04-25T14:05:07.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.988+0000 7fa4f1c00640 1 -- 172.21.15.86:0/2490133526 shutdown_connections 2024-04-25T14:05:07.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.988+0000 7fa4f1c00640 1 --2- 172.21.15.86:0/2490133526 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fa4c806f430 0x7fa4c8071940 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:07.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.988+0000 7fa4f1c00640 1 --2- 172.21.15.86:0/2490133526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4ec0d16e0 0x7fa4ec068590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:07.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.988+0000 7fa4f1c00640 1 --2- 172.21.15.86:0/2490133526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4ec0cfca0 0x7fa4ec09a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:07.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.988+0000 7fa4f1c00640 1 --2- 172.21.15.86:0/2490133526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4ec06c670 0x7fa4ec15b8b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:07.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.988+0000 7fa4f1c00640 1 -- 172.21.15.86:0/2490133526 >> 172.21.15.86:0/2490133526 conn(0x7fa4ec0b4650 msgr2=0x7fa4ec0b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:07.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.988+0000 7fa4f1c00640 1 -- 172.21.15.86:0/2490133526 shutdown_connections 2024-04-25T14:05:07.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:07.988+0000 7fa4f1c00640 1 -- 172.21.15.86:0/2490133526 wait complete. 2024-04-25T14:05:08.010 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T14:05:08.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.156+0000 7fcd97e00640 1 Processor -- start 2024-04-25T14:05:08.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.156+0000 7fcd97e00640 1 -- start start 2024-04-25T14:05:08.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.160+0000 7fcd97e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcd98071470 0x7fcd980d1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:08.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.160+0000 7fcd97e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd9806e0a0 con 0x7fcd9815fc30 2024-04-25T14:05:08.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.160+0000 7fcd97e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd9806e380 con 0x7fcd980c8550 2024-04-25T14:05:08.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.160+0000 7fcd97e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcd980d1380 con 0x7fcd98071470 2024-04-25T14:05:08.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.160+0000 7fcd97400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcd98071470 0x7fcd980d1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:08.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.160+0000 7fcd96a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fcd9815fc30 0x7fcd980d2a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52202/0 (socket says 172.21.15.86:52202) 2024-04-25T14:05:08.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.160+0000 7fcd96a00640 1 -- 172.21.15.86:0/513580477 learned_addr learned my addr 172.21.15.86:0/513580477 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:08.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.160+0000 7fcd97400640 1 -- 172.21.15.86:0/513580477 >> v1:172.21.15.155:6789/0 conn(0x7fcd980c8550 legacy=0x7fcd98070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:08.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.160+0000 7fcd97400640 1 -- 172.21.15.86:0/513580477 >> v1:172.21.15.86:6789/0 conn(0x7fcd9815fc30 legacy=0x7fcd980d2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:08.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.160+0000 7fcd97400640 1 -- 172.21.15.86:0/513580477 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd980d1660 con 0x7fcd98071470 2024-04-25T14:05:08.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.160+0000 7fcd96000640 1 -- 172.21.15.86:0/513580477 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 101158174 0 0) 0x7fcd9806e0a0 con 0x7fcd9815fc30 2024-04-25T14:05:08.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.164+0000 7fcd97400640 1 --2- 172.21.15.86:0/513580477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd98071470 0x7fcd980d1b80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fcd90009d90 tx=0x7fcd9001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b6f22b2ed1afcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:08.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.164+0000 7fcd96000640 1 -- 172.21.15.86:0/513580477 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd900b2070 con 0x7fcd98071470 2024-04-25T14:05:08.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.164+0000 7fcd96000640 1 -- 172.21.15.86:0/513580477 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd900a5a50 con 0x7fcd98071470 2024-04-25T14:05:08.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.164+0000 7fcd96000640 1 -- 172.21.15.86:0/513580477 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd900a5e20 con 0x7fcd98071470 2024-04-25T14:05:08.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.164+0000 7fcd97e00640 1 -- 172.21.15.86:0/513580477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd98071470 msgr2=0x7fcd980d1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:08.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.164+0000 7fcd97e00640 1 --2- 172.21.15.86:0/513580477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd98071470 0x7fcd980d1b80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fcd90009d90 tx=0x7fcd9001c970 comp rx=0 tx=0).stop 2024-04-25T14:05:08.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.164+0000 7fcd97e00640 1 -- 172.21.15.86:0/513580477 shutdown_connections 2024-04-25T14:05:08.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.164+0000 7fcd97e00640 1 --2- 172.21.15.86:0/513580477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd98071470 0x7fcd980d1b80 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:08.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.164+0000 7fcd97e00640 1 -- 172.21.15.86:0/513580477 >> 172.21.15.86:0/513580477 conn(0x7fcd980d3330 msgr2=0x7fcd980d3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:08.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.164+0000 7fcd97e00640 1 -- 172.21.15.86:0/513580477 shutdown_connections 2024-04-25T14:05:08.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.164+0000 7fcd97e00640 1 -- 172.21.15.86:0/513580477 wait complete. 2024-04-25T14:05:08.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.168+0000 7fcd97e00640 1 Processor -- start 2024-04-25T14:05:08.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.168+0000 7fcd97e00640 1 -- start start 2024-04-25T14:05:08.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.168+0000 7fcd97e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd980c8550 0x7fcd98159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:08.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.168+0000 7fcd97e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd9815fc30 0x7fcd9814afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:08.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.168+0000 7fcd97e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd9815f1f0 0x7fcd9814c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:08.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.168+0000 7fcd97e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9816e570 con 0x7fcd980c8550 2024-04-25T14:05:08.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.168+0000 7fcd97e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9816e3f0 con 0x7fcd9815fc30 2024-04-25T14:05:08.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.168+0000 7fcd97e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcd9816e6f0 con 0x7fcd9815f1f0 2024-04-25T14:05:08.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.168+0000 7fcd96a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd980c8550 0x7fcd98159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:08.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.168+0000 7fcd96a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd980c8550 0x7fcd98159d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35704/0 (socket says 172.21.15.86:35704) 2024-04-25T14:05:08.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.168+0000 7fcd96a00640 1 -- 172.21.15.86:0/4286841116 learned_addr learned my addr 172.21.15.86:0/4286841116 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:08.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.168+0000 7fcd8e000640 1 --2- 172.21.15.86:0/4286841116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd9815fc30 0x7fcd9814afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:08.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.168+0000 7fcd97400640 1 --2- 172.21.15.86:0/4286841116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd9815f1f0 0x7fcd9814c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:08.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.168+0000 7fcd96a00640 1 -- 172.21.15.86:0/4286841116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd9815f1f0 msgr2=0x7fcd9814c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:08.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.168+0000 7fcd96a00640 1 --2- 172.21.15.86:0/4286841116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd9815f1f0 0x7fcd9814c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:08.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.168+0000 7fcd96a00640 1 -- 172.21.15.86:0/4286841116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd9815fc30 msgr2=0x7fcd9814afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:08.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.168+0000 7fcd96a00640 1 --2- 172.21.15.86:0/4286841116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd9815fc30 0x7fcd9814afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:08.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.168+0000 7fcd96a00640 1 -- 172.21.15.86:0/4286841116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9814b500 con 0x7fcd980c8550 2024-04-25T14:05:08.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.168+0000 7fcd96a00640 1 --2- 172.21.15.86:0/4286841116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd980c8550 0x7fcd98159d40 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800c410 tx=0x7fcd8800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:08.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.168+0000 7fcd8f400640 1 -- 172.21.15.86:0/4286841116 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd880074e0 con 0x7fcd980c8550 2024-04-25T14:05:08.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.168+0000 7fcd8f400640 1 -- 172.21.15.86:0/4286841116 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd880047f0 con 0x7fcd980c8550 2024-04-25T14:05:08.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.168+0000 7fcd97e00640 1 -- 172.21.15.86:0/4286841116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd9815a590 con 0x7fcd980c8550 2024-04-25T14:05:08.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.168+0000 7fcd8f400640 1 -- 172.21.15.86:0/4286841116 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd88007680 con 0x7fcd980c8550 2024-04-25T14:05:08.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.168+0000 7fcd97e00640 1 -- 172.21.15.86:0/4286841116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd9815ab50 con 0x7fcd980c8550 2024-04-25T14:05:08.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.172+0000 7fcd97e00640 1 -- 172.21.15.86:0/4286841116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd98076b80 con 0x7fcd980c8550 2024-04-25T14:05:08.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.172+0000 7fcd8f400640 1 -- 172.21.15.86:0/4286841116 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcd88011070 con 0x7fcd980c8550 2024-04-25T14:05:08.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.172+0000 7fcd8f400640 1 --2- 172.21.15.86:0/4286841116 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fcd7406f400 0x7fcd74071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:08.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.172+0000 7fcd8f400640 1 -- 172.21.15.86:0/4286841116 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7fcd8808f030 con 0x7fcd980c8550 2024-04-25T14:05:08.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.172+0000 7fcd8e000640 1 --2- 172.21.15.86:0/4286841116 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fcd7406f400 0x7fcd74071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:08.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.172+0000 7fcd8e000640 1 --2- 172.21.15.86:0/4286841116 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fcd7406f400 0x7fcd74071910 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcd980cd980 tx=0x7fcd80009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:08.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.176+0000 7fcd8f400640 1 -- 172.21.15.86:0/4286841116 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcd8805bcf0 con 0x7fcd980c8550 2024-04-25T14:05:08.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:08.376+0000 7fcd97e00640 1 -- 172.21.15.86:0/4286841116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fcd980c8330 con 0x7fcd980c8550 2024-04-25T14:05:09.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.268+0000 7fcd8f400640 1 -- 172.21.15.86:0/4286841116 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fcd8805b5f0 con 0x7fcd980c8550 2024-04-25T14:05:09.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.272+0000 7fcd97e00640 1 -- 172.21.15.86:0/4286841116 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fcd7406f400 msgr2=0x7fcd74071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:09.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.272+0000 7fcd97e00640 1 --2- 172.21.15.86:0/4286841116 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fcd7406f400 0x7fcd74071910 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcd980cd980 tx=0x7fcd80009040 comp rx=0 tx=0).stop 2024-04-25T14:05:09.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.272+0000 7fcd97e00640 1 -- 172.21.15.86:0/4286841116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd980c8550 msgr2=0x7fcd98159d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:09.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.272+0000 7fcd97e00640 1 --2- 172.21.15.86:0/4286841116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd980c8550 0x7fcd98159d40 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800c410 tx=0x7fcd8800c8d0 comp rx=0 tx=0).stop 2024-04-25T14:05:09.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.272+0000 7fcd97e00640 1 -- 172.21.15.86:0/4286841116 shutdown_connections 2024-04-25T14:05:09.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.272+0000 7fcd97e00640 1 --2- 172.21.15.86:0/4286841116 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fcd7406f400 0x7fcd74071910 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:09.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.272+0000 7fcd97e00640 1 --2- 172.21.15.86:0/4286841116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd9815f1f0 0x7fcd9814c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:09.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.272+0000 7fcd97e00640 1 --2- 172.21.15.86:0/4286841116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd9815fc30 0x7fcd9814afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:09.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.272+0000 7fcd97e00640 1 --2- 172.21.15.86:0/4286841116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd980c8550 0x7fcd98159d40 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:09.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.272+0000 7fcd97e00640 1 -- 172.21.15.86:0/4286841116 >> 172.21.15.86:0/4286841116 conn(0x7fcd980d3330 msgr2=0x7fcd980cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:09.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.272+0000 7fcd97e00640 1 -- 172.21.15.86:0/4286841116 shutdown_connections 2024-04-25T14:05:09.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.272+0000 7fcd97e00640 1 -- 172.21.15.86:0/4286841116 wait complete. 2024-04-25T14:05:09.298 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T14:05:09.299 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T14:05:09.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.448+0000 7f04e7a00640 1 Processor -- start 2024-04-25T14:05:09.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.448+0000 7f04e7a00640 1 -- start start 2024-04-25T14:05:09.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04e7a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04e0070330 0x7f04e00d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:09.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04e7a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f04e00ce790 0x7f04e006f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:09.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04e7a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04e00d2030 con 0x7f04e00d1c20 2024-04-25T14:05:09.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04e7a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f04e015e5c0 con 0x7f04e00ce790 2024-04-25T14:05:09.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04e7a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f04e015e8a0 con 0x7f04e0070330 2024-04-25T14:05:09.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04d7e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f04e00d1c20 0x7f04e006ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37186/0 (socket says 172.21.15.86:37186) 2024-04-25T14:05:09.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04d7e00640 1 -- 172.21.15.86:0/1936649048 learned_addr learned my addr 172.21.15.86:0/1936649048 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:09.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04e5000640 1 --2- 172.21.15.86:0/1936649048 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04e0070330 0x7f04e00d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:09.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04dfe00640 1 -- 172.21.15.86:0/1936649048 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3349426 0 0) 0x7f04c8001120 con 0x7f04e00d1c20 2024-04-25T14:05:09.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04e5a00640 1 --2- 172.21.15.86:0/1936649048 >> v2:172.21.15.155:3300/0 conn(0x7f04e00ce790 0x7f04e006f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:09.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04dfe00640 1 -- 172.21.15.86:0/1936649048 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04c4003530 con 0x7f04e00d1c20 2024-04-25T14:05:09.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04dfe00640 1 -- 172.21.15.86:0/1936649048 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 719875242 0 0) 0x7f04c4003530 con 0x7f04e00d1c20 2024-04-25T14:05:09.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04dfe00640 1 -- 172.21.15.86:0/1936649048 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04e0070330 msgr2=0x7f04e00d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:09.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04dfe00640 1 --2- 172.21.15.86:0/1936649048 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04e0070330 0x7f04e00d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:09.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04dfe00640 1 -- 172.21.15.86:0/1936649048 >> v2:172.21.15.155:3300/0 conn(0x7f04e00ce790 msgr2=0x7f04e006f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:09.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04dfe00640 1 --2- 172.21.15.86:0/1936649048 >> v2:172.21.15.155:3300/0 conn(0x7f04e00ce790 0x7f04e006f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:09.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04dfe00640 1 -- 172.21.15.86:0/1936649048 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e015eb80 con 0x7f04e00d1c20 2024-04-25T14:05:09.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04dfe00640 1 -- 172.21.15.86:0/1936649048 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f04c8001850 con 0x7f04e00d1c20 2024-04-25T14:05:09.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04dfe00640 1 -- 172.21.15.86:0/1936649048 >> v1:172.21.15.86:6789/0 conn(0x7f04e00d1c20 legacy=0x7f04e006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:09.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04dfe00640 1 --2- 172.21.15.86:0/1936649048 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04e0070330 0x7f04c4003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:09.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04dfe00640 1 --2- 172.21.15.86:0/1936649048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04c4003fb0 0x7f04c4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:09.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04dfe00640 1 --2- 172.21.15.86:0/1936649048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04c4004d50 0x7f04c4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:09.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04dfe00640 1 -- 172.21.15.86:0/1936649048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f04e015e5c0 con 0x7f04c4004d50 2024-04-25T14:05:09.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04dfe00640 1 -- 172.21.15.86:0/1936649048 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f04e015e8a0 con 0x7f04c4003fb0 2024-04-25T14:05:09.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04dfe00640 1 -- 172.21.15.86:0/1936649048 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f04c400bb30 con 0x7f04e0070330 2024-04-25T14:05:09.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04e5a00640 1 --2- 172.21.15.86:0/1936649048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04c4004d50 0x7f04c4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:09.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04e5000640 1 --2- 172.21.15.86:0/1936649048 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04e0070330 0x7f04c4003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:09.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04d7e00640 1 --2- 172.21.15.86:0/1936649048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04c4003fb0 0x7f04c4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:09.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04e5a00640 1 -- 172.21.15.86:0/1936649048 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04e0070330 msgr2=0x7f04c4003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:09.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04e5a00640 1 --2- 172.21.15.86:0/1936649048 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04e0070330 0x7f04c4003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:09.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04e5a00640 1 -- 172.21.15.86:0/1936649048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04c4003fb0 msgr2=0x7f04c4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:09.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04e5a00640 1 --2- 172.21.15.86:0/1936649048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04c4003fb0 0x7f04c4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:09.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04e5a00640 1 -- 172.21.15.86:0/1936649048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c4003530 con 0x7f04c4004d50 2024-04-25T14:05:09.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.452+0000 7f04d7e00640 1 --2- 172.21.15.86:0/1936649048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04c4003fb0 0x7f04c4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:09.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.456+0000 7f04e5a00640 1 --2- 172.21.15.86:0/1936649048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04c4004d50 0x7f04c4005160 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009f90 tx=0x7f04d801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1044c2e9e3d37d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:09.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.456+0000 7f04dfe00640 1 -- 172.21.15.86:0/1936649048 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d80b2070 con 0x7f04c4004d50 2024-04-25T14:05:09.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.456+0000 7f04dfe00640 1 -- 172.21.15.86:0/1936649048 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04d80a5c90 con 0x7f04c4004d50 2024-04-25T14:05:09.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.456+0000 7f04dfe00640 1 -- 172.21.15.86:0/1936649048 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d8009c10 con 0x7f04c4004d50 2024-04-25T14:05:09.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.456+0000 7f04e7a00640 1 -- 172.21.15.86:0/1936649048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04c4004d50 msgr2=0x7f04c4005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:09.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.456+0000 7f04e7a00640 1 --2- 172.21.15.86:0/1936649048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04c4004d50 0x7f04c4005160 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009f90 tx=0x7f04d801cc80 comp rx=0 tx=0).stop 2024-04-25T14:05:09.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.456+0000 7f04e5a00640 1 -- 172.21.15.86:0/1936649048 reap_dead start 2024-04-25T14:05:09.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.456+0000 7f04e7a00640 1 -- 172.21.15.86:0/1936649048 shutdown_connections 2024-04-25T14:05:09.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.456+0000 7f04e7a00640 1 -- 172.21.15.86:0/1936649048 >> 172.21.15.86:0/1936649048 conn(0x7f04e00b4650 msgr2=0x7f04e00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:09.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.456+0000 7f04e7a00640 1 -- 172.21.15.86:0/1936649048 shutdown_connections 2024-04-25T14:05:09.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.456+0000 7f04e7a00640 1 -- 172.21.15.86:0/1936649048 wait complete. 2024-04-25T14:05:09.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.460+0000 7f04e7a00640 1 Processor -- start 2024-04-25T14:05:09.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.460+0000 7f04e7a00640 1 -- start start 2024-04-25T14:05:09.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.460+0000 7f04e7a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04c4004d50 0x7f04e00ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:09.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.460+0000 7f04e7a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04e01f5d60 0x7f04e00b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:09.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.460+0000 7f04e7a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04e00b3370 0x7f04e01f9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:09.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.460+0000 7f04e7a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f04e015f330 con 0x7f04c4004d50 2024-04-25T14:05:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.460+0000 7f04e7a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f04e006d170 con 0x7f04e01f5d60 2024-04-25T14:05:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.460+0000 7f04e7a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f04e015f4b0 con 0x7f04e00b3370 2024-04-25T14:05:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.460+0000 7f04e5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04c4004d50 0x7f04e00ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.460+0000 7f04e5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04c4004d50 0x7f04e00ca600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57474/0 (socket says 172.21.15.86:57474) 2024-04-25T14:05:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.460+0000 7f04e5000640 1 -- 172.21.15.86:0/3347206607 learned_addr learned my addr 172.21.15.86:0/3347206607 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.460+0000 7f04e5000640 1 -- 172.21.15.86:0/3347206607 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04e00b3370 msgr2=0x7f04e01f9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.460+0000 7f04d7e00640 1 --2- 172.21.15.86:0/3347206607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04e01f5d60 0x7f04e00b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.460+0000 7f04e5a00640 1 --2- 172.21.15.86:0/3347206607 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04e00b3370 0x7f04e01f9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.460+0000 7f04e5000640 1 --2- 172.21.15.86:0/3347206607 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04e00b3370 0x7f04e01f9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.460+0000 7f04e5000640 1 -- 172.21.15.86:0/3347206607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04e01f5d60 msgr2=0x7f04e00b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.460+0000 7f04e5000640 1 --2- 172.21.15.86:0/3347206607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04e01f5d60 0x7f04e00b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.460+0000 7f04e5000640 1 -- 172.21.15.86:0/3347206607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e00b3780 con 0x7f04c4004d50 2024-04-25T14:05:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.460+0000 7f04e5a00640 1 --2- 172.21.15.86:0/3347206607 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04e00b3370 0x7f04e01f9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:09.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.460+0000 7f04d7e00640 1 --2- 172.21.15.86:0/3347206607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04e01f5d60 0x7f04e00b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:09.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.460+0000 7f04e5000640 1 --2- 172.21.15.86:0/3347206607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04c4004d50 0x7f04e00ca600 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f04d0011960 tx=0x7f04d0011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:09.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.460+0000 7f04dd600640 1 -- 172.21.15.86:0/3347206607 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d000faf0 con 0x7f04c4004d50 2024-04-25T14:05:09.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.460+0000 7f04dd600640 1 -- 172.21.15.86:0/3347206607 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04d000fc90 con 0x7f04c4004d50 2024-04-25T14:05:09.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.460+0000 7f04dd600640 1 -- 172.21.15.86:0/3347206607 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0014440 con 0x7f04c4004d50 2024-04-25T14:05:09.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.460+0000 7f04e7a00640 1 -- 172.21.15.86:0/3347206607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e00c93c0 con 0x7f04c4004d50 2024-04-25T14:05:09.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.464+0000 7f04e7a00640 1 -- 172.21.15.86:0/3347206607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e00c9980 con 0x7f04c4004d50 2024-04-25T14:05:09.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.464+0000 7f04e7a00640 1 -- 172.21.15.86:0/3347206607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e0069d00 con 0x7f04c4004d50 2024-04-25T14:05:09.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.464+0000 7f04dd600640 1 -- 172.21.15.86:0/3347206607 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f04d0012040 con 0x7f04c4004d50 2024-04-25T14:05:09.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.464+0000 7f04dd600640 1 --2- 172.21.15.86:0/3347206607 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f04bc06f400 0x7f04bc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:09.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.464+0000 7f04dd600640 1 -- 172.21.15.86:0/3347206607 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f04d008fb50 con 0x7f04c4004d50 2024-04-25T14:05:09.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.464+0000 7f04d7e00640 1 --2- 172.21.15.86:0/3347206607 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f04bc06f400 0x7f04bc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:09.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.464+0000 7f04d7e00640 1 --2- 172.21.15.86:0/3347206607 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f04bc06f400 0x7f04bc071910 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f04c80012c0 tx=0x7f04c8001250 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:09.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.468+0000 7f04dd600640 1 -- 172.21.15.86:0/3347206607 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f04e00b3780 con 0x7f04c4004d50 2024-04-25T14:05:09.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:09.660+0000 7f04e7a00640 1 -- 172.21.15.86:0/3347206607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f04e006db60 con 0x7f04c4004d50 2024-04-25T14:05:10.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.276+0000 7f04dd600640 1 -- 172.21.15.86:0/3347206607 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f04d0019030 con 0x7f04c4004d50 2024-04-25T14:05:10.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.280+0000 7f04e7a00640 1 -- 172.21.15.86:0/3347206607 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f04bc06f400 msgr2=0x7f04bc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:10.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.280+0000 7f04e7a00640 1 --2- 172.21.15.86:0/3347206607 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f04bc06f400 0x7f04bc071910 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f04c80012c0 tx=0x7f04c8001250 comp rx=0 tx=0).stop 2024-04-25T14:05:10.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.280+0000 7f04e7a00640 1 -- 172.21.15.86:0/3347206607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04c4004d50 msgr2=0x7f04e00ca600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:10.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.280+0000 7f04e7a00640 1 --2- 172.21.15.86:0/3347206607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04c4004d50 0x7f04e00ca600 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f04d0011960 tx=0x7f04d0011e20 comp rx=0 tx=0).stop 2024-04-25T14:05:10.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.280+0000 7f04e7a00640 1 -- 172.21.15.86:0/3347206607 shutdown_connections 2024-04-25T14:05:10.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.280+0000 7f04e7a00640 1 --2- 172.21.15.86:0/3347206607 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f04bc06f400 0x7f04bc071910 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:10.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.280+0000 7f04e7a00640 1 --2- 172.21.15.86:0/3347206607 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04e00b3370 0x7f04e01f9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:10.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.280+0000 7f04e7a00640 1 --2- 172.21.15.86:0/3347206607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04e01f5d60 0x7f04e00b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:10.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.280+0000 7f04e7a00640 1 --2- 172.21.15.86:0/3347206607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04c4004d50 0x7f04e00ca600 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:10.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.280+0000 7f04e7a00640 1 -- 172.21.15.86:0/3347206607 >> 172.21.15.86:0/3347206607 conn(0x7f04e00b4650 msgr2=0x7f04e00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:10.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.280+0000 7f04e7a00640 1 -- 172.21.15.86:0/3347206607 shutdown_connections 2024-04-25T14:05:10.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.280+0000 7f04e7a00640 1 -- 172.21.15.86:0/3347206607 wait complete. 2024-04-25T14:05:10.302 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T14:05:10.302 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:05:10.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.444+0000 7f1c34a00640 1 Processor -- start 2024-04-25T14:05:10.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.444+0000 7f1c34a00640 1 -- start start 2024-04-25T14:05:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c34a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1c300d1df0 0x7f1c300cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c34a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c300cdcc0 0x7f1c300ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c34a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c3015e5c0 con 0x7f1c30070330 2024-04-25T14:05:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c34a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1c3006fd40 con 0x7f1c300d1df0 2024-04-25T14:05:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c34a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1c3015e8a0 con 0x7f1c300cdcc0 2024-04-25T14:05:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c2de00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1c30070330 0x7f1c300d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37198/0 (socket says 172.21.15.86:37198) 2024-04-25T14:05:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c2de00640 1 -- 172.21.15.86:0/1803600719 learned_addr learned my addr 172.21.15.86:0/1803600719 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c2e800640 1 --2- 172.21.15.86:0/1803600719 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c300cdcc0 0x7f1c300ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c25400640 1 --2- 172.21.15.86:0/1803600719 >> v2:172.21.15.155:3300/0 conn(0x7f1c300d1df0 0x7f1c300cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c2d400640 1 -- 172.21.15.86:0/1803600719 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1977153658 0 0) 0x7f1c3015e5c0 con 0x7f1c30070330 2024-04-25T14:05:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c2d400640 1 -- 172.21.15.86:0/1803600719 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c14003530 con 0x7f1c30070330 2024-04-25T14:05:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c2d400640 1 -- 172.21.15.86:0/1803600719 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3370471770 0 0) 0x7f1c14003530 con 0x7f1c30070330 2024-04-25T14:05:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c2d400640 1 -- 172.21.15.86:0/1803600719 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c300cdcc0 msgr2=0x7f1c300ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c2d400640 1 --2- 172.21.15.86:0/1803600719 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c300cdcc0 0x7f1c300ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c2d400640 1 -- 172.21.15.86:0/1803600719 >> v2:172.21.15.155:3300/0 conn(0x7f1c300d1df0 msgr2=0x7f1c300cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c2d400640 1 --2- 172.21.15.86:0/1803600719 >> v2:172.21.15.155:3300/0 conn(0x7f1c300d1df0 0x7f1c300cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c2d400640 1 -- 172.21.15.86:0/1803600719 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c3015eb80 con 0x7f1c30070330 2024-04-25T14:05:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c2d400640 1 -- 172.21.15.86:0/1803600719 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f1c20001a00 con 0x7f1c30070330 2024-04-25T14:05:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c2d400640 1 -- 172.21.15.86:0/1803600719 >> v1:172.21.15.86:6789/0 conn(0x7f1c30070330 legacy=0x7f1c300d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c2d400640 1 --2- 172.21.15.86:0/1803600719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c14003a30 0x7f1c14003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c2d400640 1 --2- 172.21.15.86:0/1803600719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c14004810 0x7f1c14004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c2d400640 1 --2- 172.21.15.86:0/1803600719 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c300cdcc0 0x7f1c14005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c2d400640 1 -- 172.21.15.86:0/1803600719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3015e8a0 con 0x7f1c14004810 2024-04-25T14:05:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c2d400640 1 -- 172.21.15.86:0/1803600719 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1400ba80 con 0x7f1c14003a30 2024-04-25T14:05:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c2d400640 1 -- 172.21.15.86:0/1803600719 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1c1400bc50 con 0x7f1c300cdcc0 2024-04-25T14:05:10.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c25400640 1 --2- 172.21.15.86:0/1803600719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c14004810 0x7f1c14004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:10.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c2de00640 1 --2- 172.21.15.86:0/1803600719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c14003a30 0x7f1c14003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:10.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c2e800640 1 --2- 172.21.15.86:0/1803600719 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c300cdcc0 0x7f1c14005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:10.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c25400640 1 -- 172.21.15.86:0/1803600719 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c300cdcc0 msgr2=0x7f1c14005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:10.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c25400640 1 --2- 172.21.15.86:0/1803600719 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c300cdcc0 0x7f1c14005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:10.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c25400640 1 -- 172.21.15.86:0/1803600719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c14003a30 msgr2=0x7f1c14003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:10.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c25400640 1 --2- 172.21.15.86:0/1803600719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c14003a30 0x7f1c14003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:10.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c25400640 1 -- 172.21.15.86:0/1803600719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c14003530 con 0x7f1c14004810 2024-04-25T14:05:10.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.448+0000 7f1c2de00640 1 --2- 172.21.15.86:0/1803600719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c14003a30 0x7f1c14003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:10.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.452+0000 7f1c25400640 1 --2- 172.21.15.86:0/1803600719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c14004810 0x7f1c14004c20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1c30074ed0 tx=0x7f1c1801ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1231aac02cbbefd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:10.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.452+0000 7f1c2d400640 1 -- 172.21.15.86:0/1803600719 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c180b2070 con 0x7f1c14004810 2024-04-25T14:05:10.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.452+0000 7f1c2d400640 1 -- 172.21.15.86:0/1803600719 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c180a55d0 con 0x7f1c14004810 2024-04-25T14:05:10.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.452+0000 7f1c2d400640 1 -- 172.21.15.86:0/1803600719 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c180096b0 con 0x7f1c14004810 2024-04-25T14:05:10.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.452+0000 7f1c34a00640 1 -- 172.21.15.86:0/1803600719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c14004810 msgr2=0x7f1c14004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:10.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.452+0000 7f1c34a00640 1 --2- 172.21.15.86:0/1803600719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c14004810 0x7f1c14004c20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1c30074ed0 tx=0x7f1c1801ca20 comp rx=0 tx=0).stop 2024-04-25T14:05:10.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.452+0000 7f1c2e800640 1 -- 172.21.15.86:0/1803600719 reap_dead start 2024-04-25T14:05:10.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.452+0000 7f1c34a00640 1 -- 172.21.15.86:0/1803600719 shutdown_connections 2024-04-25T14:05:10.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.452+0000 7f1c34a00640 1 -- 172.21.15.86:0/1803600719 >> 172.21.15.86:0/1803600719 conn(0x7f1c300b4650 msgr2=0x7f1c300b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:10.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.452+0000 7f1c34a00640 1 -- 172.21.15.86:0/1803600719 shutdown_connections 2024-04-25T14:05:10.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.452+0000 7f1c34a00640 1 -- 172.21.15.86:0/1803600719 wait complete. 2024-04-25T14:05:10.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.456+0000 7f1c34a00640 1 Processor -- start 2024-04-25T14:05:10.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.456+0000 7f1c34a00640 1 -- start start 2024-04-25T14:05:10.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.456+0000 7f1c34a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c14004810 0x7f1c3014cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:10.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.456+0000 7f1c34a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3015c530 0x7f1c3015c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:10.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.456+0000 7f1c34a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c3006f210 0x7f1c3006f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:10.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.456+0000 7f1c34a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c300708c0 con 0x7f1c3015c530 2024-04-25T14:05:10.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.456+0000 7f1c34a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1c30070740 con 0x7f1c3006f210 2024-04-25T14:05:10.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.456+0000 7f1c34a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1c300ce8e0 con 0x7f1c14004810 2024-04-25T14:05:10.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.456+0000 7f1c25400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3015c530 0x7f1c3015c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:10.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.456+0000 7f1c25400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3015c530 0x7f1c3015c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57488/0 (socket says 172.21.15.86:57488) 2024-04-25T14:05:10.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.456+0000 7f1c25400640 1 -- 172.21.15.86:0/1889738243 learned_addr learned my addr 172.21.15.86:0/1889738243 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:10.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.456+0000 7f1c2de00640 1 --2- 172.21.15.86:0/1889738243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c14004810 0x7f1c3014cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:10.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.456+0000 7f1c2e800640 1 --2- 172.21.15.86:0/1889738243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c3006f210 0x7f1c3006f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:10.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.456+0000 7f1c25400640 1 -- 172.21.15.86:0/1889738243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c14004810 msgr2=0x7f1c3014cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:10.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.456+0000 7f1c25400640 1 --2- 172.21.15.86:0/1889738243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c14004810 0x7f1c3014cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:10.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.456+0000 7f1c25400640 1 -- 172.21.15.86:0/1889738243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c3006f210 msgr2=0x7f1c3006f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:10.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.456+0000 7f1c25400640 1 --2- 172.21.15.86:0/1889738243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c3006f210 0x7f1c3006f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:10.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.456+0000 7f1c25400640 1 -- 172.21.15.86:0/1889738243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c3006fb60 con 0x7f1c3015c530 2024-04-25T14:05:10.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.456+0000 7f1c25400640 1 --2- 172.21.15.86:0/1889738243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3015c530 0x7f1c3015c940 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1c1801c850 tx=0x7f1c14005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:10.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.460+0000 7f1c26a00640 1 -- 172.21.15.86:0/1889738243 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c180b2040 con 0x7f1c3015c530 2024-04-25T14:05:10.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.460+0000 7f1c26a00640 1 -- 172.21.15.86:0/1889738243 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c1800f000 con 0x7f1c3015c530 2024-04-25T14:05:10.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.460+0000 7f1c34a00640 1 -- 172.21.15.86:0/1889738243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c300d0d20 con 0x7f1c3015c530 2024-04-25T14:05:10.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.460+0000 7f1c26a00640 1 -- 172.21.15.86:0/1889738243 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c180ad680 con 0x7f1c3015c530 2024-04-25T14:05:10.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.460+0000 7f1c34a00640 1 -- 172.21.15.86:0/1889738243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c3014d4d0 con 0x7f1c3015c530 2024-04-25T14:05:10.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.460+0000 7f1c34a00640 1 -- 172.21.15.86:0/1889738243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c30069d00 con 0x7f1c3015c530 2024-04-25T14:05:10.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.460+0000 7f1c26a00640 1 -- 172.21.15.86:0/1889738243 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1c180be030 con 0x7f1c3015c530 2024-04-25T14:05:10.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.460+0000 7f1c26a00640 1 --2- 172.21.15.86:0/1889738243 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1c0c06f400 0x7f1c0c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:10.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.460+0000 7f1c26a00640 1 -- 172.21.15.86:0/1889738243 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f1c18128a10 con 0x7f1c3015c530 2024-04-25T14:05:10.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.460+0000 7f1c2de00640 1 --2- 172.21.15.86:0/1889738243 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1c0c06f400 0x7f1c0c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:10.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.464+0000 7f1c2de00640 1 --2- 172.21.15.86:0/1889738243 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1c0c06f400 0x7f1c0c071910 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1c30072200 tx=0x7f1c20009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:10.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.464+0000 7f1c26a00640 1 -- 172.21.15.86:0/1889738243 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1c3006fb60 con 0x7f1c3015c530 2024-04-25T14:05:10.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.656+0000 7f1c34a00640 1 -- 172.21.15.86:0/1889738243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c3015cf80 con 0x7f1c3015c530 2024-04-25T14:05:10.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.656+0000 7f1c26a00640 1 -- 172.21.15.86:0/1889738243 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5184 (secure 0 0 0) 0x7f1c18016760 con 0x7f1c3015c530 2024-04-25T14:05:10.658 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:10.658 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3546153119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3546153119},{"type":"v1","addr":"172.21.15.155:6834","nonce":3546153119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5283,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3499340224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3499340224},{"type":"v1","addr":"172.21.15.86:6836","nonce":3499340224}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5284,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1466978742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1466978742},{"type":"v1","addr":"172.21.15.155:6835","nonce":1466978742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:06.638216+0000","modified":"2024-04-25T14:05:09.664776+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5286":{"gid":5286,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/716923566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":716923566},{"type":"v1","addr":"172.21.15.86:6837","nonce":716923566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5286,"qdb_cluster":[5286]},"id":2}]} 2024-04-25T14:05:10.658 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 18 2024-04-25T14:05:10.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.656+0000 7f1c34a00640 1 -- 172.21.15.86:0/1889738243 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1c0c06f400 msgr2=0x7f1c0c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:10.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.656+0000 7f1c34a00640 1 --2- 172.21.15.86:0/1889738243 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1c0c06f400 0x7f1c0c071910 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1c30072200 tx=0x7f1c20009040 comp rx=0 tx=0).stop 2024-04-25T14:05:10.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.656+0000 7f1c34a00640 1 -- 172.21.15.86:0/1889738243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3015c530 msgr2=0x7f1c3015c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:10.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.656+0000 7f1c34a00640 1 --2- 172.21.15.86:0/1889738243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3015c530 0x7f1c3015c940 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1c1801c850 tx=0x7f1c14005840 comp rx=0 tx=0).stop 2024-04-25T14:05:10.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.656+0000 7f1c34a00640 1 -- 172.21.15.86:0/1889738243 shutdown_connections 2024-04-25T14:05:10.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.656+0000 7f1c34a00640 1 --2- 172.21.15.86:0/1889738243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c3006f210 0x7f1c3006f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:10.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.656+0000 7f1c34a00640 1 --2- 172.21.15.86:0/1889738243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3015c530 0x7f1c3015c940 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:10.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.656+0000 7f1c34a00640 1 --2- 172.21.15.86:0/1889738243 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1c0c06f400 0x7f1c0c071910 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:10.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.656+0000 7f1c34a00640 1 --2- 172.21.15.86:0/1889738243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c14004810 0x7f1c3014cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:10.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.656+0000 7f1c34a00640 1 -- 172.21.15.86:0/1889738243 >> 172.21.15.86:0/1889738243 conn(0x7f1c300b4650 msgr2=0x7f1c3015b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:10.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.656+0000 7f1c34a00640 1 -- 172.21.15.86:0/1889738243 shutdown_connections 2024-04-25T14:05:10.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.656+0000 7f1c34a00640 1 -- 172.21.15.86:0/1889738243 wait complete. 2024-04-25T14:05:10.680 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:05:10.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.824+0000 7f689f200640 1 Processor -- start 2024-04-25T14:05:10.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.824+0000 7f689f200640 1 -- start start 2024-04-25T14:05:10.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.828+0000 7f689f200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f68a01635a0 0x7f68a006eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:10.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.828+0000 7f689f200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f68a006f060 0x7f68a00d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:10.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.828+0000 7f689f200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68a00d0b00 con 0x7f68a00d06f0 2024-04-25T14:05:10.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.828+0000 7f689f200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f68a006f470 con 0x7f68a01635a0 2024-04-25T14:05:10.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.828+0000 7f689f200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f68a0163d80 con 0x7f68a006f060 2024-04-25T14:05:10.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.828+0000 7f689de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f68a01635a0 0x7f68a006eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:10.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.828+0000 7f689e800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f68a00d06f0 0x7f68a00722a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37212/0 (socket says 172.21.15.86:37212) 2024-04-25T14:05:10.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.828+0000 7f689e800640 1 -- 172.21.15.86:0/2263287581 learned_addr learned my addr 172.21.15.86:0/2263287581 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:10.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.828+0000 7f6895400640 1 --2- 172.21.15.86:0/2263287581 >> v2:172.21.15.155:3300/0 conn(0x7f68a006f060 0x7f68a00d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:10.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.828+0000 7f689de00640 1 -- 172.21.15.86:0/2263287581 >> v2:172.21.15.155:3300/0 conn(0x7f68a006f060 msgr2=0x7f68a00d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:10.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.828+0000 7f689de00640 1 --2- 172.21.15.86:0/2263287581 >> v2:172.21.15.155:3300/0 conn(0x7f68a006f060 0x7f68a00d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:10.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.828+0000 7f689de00640 1 -- 172.21.15.86:0/2263287581 >> v1:172.21.15.86:6789/0 conn(0x7f68a00d06f0 legacy=0x7f68a00722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:10.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.828+0000 7f689de00640 1 -- 172.21.15.86:0/2263287581 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a0164060 con 0x7f68a01635a0 2024-04-25T14:05:10.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.828+0000 7f689d400640 1 -- 172.21.15.86:0/2263287581 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3135960313 0 0) 0x7f68a00d0b00 con 0x7f68a00d06f0 2024-04-25T14:05:10.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.828+0000 7f6895400640 1 --2- 172.21.15.86:0/2263287581 >> v2:172.21.15.155:3300/0 conn(0x7f68a006f060 0x7f68a00d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:10.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.832+0000 7f689de00640 1 --2- 172.21.15.86:0/2263287581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68a01635a0 0x7f68a006eb20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f68a00747f0 tx=0x7f689001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f7b1d3cd994dee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:10.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.832+0000 7f689d400640 1 -- 172.21.15.86:0/2263287581 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68900b0070 con 0x7f68a01635a0 2024-04-25T14:05:10.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.832+0000 7f689d400640 1 -- 172.21.15.86:0/2263287581 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68900ab600 con 0x7f68a01635a0 2024-04-25T14:05:10.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.832+0000 7f689d400640 1 -- 172.21.15.86:0/2263287581 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68900ab9d0 con 0x7f68a01635a0 2024-04-25T14:05:10.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.832+0000 7f689f200640 1 -- 172.21.15.86:0/2263287581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68a01635a0 msgr2=0x7f68a006eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:10.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.832+0000 7f689f200640 1 --2- 172.21.15.86:0/2263287581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68a01635a0 0x7f68a006eb20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f68a00747f0 tx=0x7f689001c480 comp rx=0 tx=0).stop 2024-04-25T14:05:10.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.832+0000 7f689f200640 1 -- 172.21.15.86:0/2263287581 shutdown_connections 2024-04-25T14:05:10.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.832+0000 7f689f200640 1 --2- 172.21.15.86:0/2263287581 >> v2:172.21.15.155:3300/0 conn(0x7f68a006f060 0x7f68a00d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:10.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.832+0000 7f689f200640 1 --2- 172.21.15.86:0/2263287581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68a01635a0 0x7f68a006eb20 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:10.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.832+0000 7f689f200640 1 -- 172.21.15.86:0/2263287581 >> 172.21.15.86:0/2263287581 conn(0x7f68a00b4f80 msgr2=0x7f68a00b5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:10.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.832+0000 7f689f200640 1 -- 172.21.15.86:0/2263287581 shutdown_connections 2024-04-25T14:05:10.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.832+0000 7f689f200640 1 -- 172.21.15.86:0/2263287581 wait complete. 2024-04-25T14:05:10.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.836+0000 7f689f200640 1 Processor -- start 2024-04-25T14:05:10.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.836+0000 7f689f200640 1 -- start start 2024-04-25T14:05:10.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.836+0000 7f689f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68a006f060 0x7f68a014eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:10.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.836+0000 7f689f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68a00d06f0 0x7f68a014f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:10.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.836+0000 7f689f200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68a01635a0 0x7f68a0148dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:10.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.836+0000 7f689f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f68a0163b30 con 0x7f68a006f060 2024-04-25T14:05:10.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.836+0000 7f689f200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f68a01639b0 con 0x7f68a00d06f0 2024-04-25T14:05:10.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.836+0000 7f689f200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f68a00729b0 con 0x7f68a01635a0 2024-04-25T14:05:10.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.836+0000 7f689de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68a006f060 0x7f68a014eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:10.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.836+0000 7f689de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68a006f060 0x7f68a014eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57510/0 (socket says 172.21.15.86:57510) 2024-04-25T14:05:10.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.836+0000 7f689de00640 1 -- 172.21.15.86:0/3328307213 learned_addr learned my addr 172.21.15.86:0/3328307213 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:10.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.836+0000 7f6895400640 1 --2- 172.21.15.86:0/3328307213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68a00d06f0 0x7f68a014f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:10.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.836+0000 7f689e800640 1 --2- 172.21.15.86:0/3328307213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68a01635a0 0x7f68a0148dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:10.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.836+0000 7f689de00640 1 -- 172.21.15.86:0/3328307213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68a01635a0 msgr2=0x7f68a0148dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:10.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.836+0000 7f689de00640 1 --2- 172.21.15.86:0/3328307213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68a01635a0 0x7f68a0148dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:10.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.836+0000 7f689de00640 1 -- 172.21.15.86:0/3328307213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68a00d06f0 msgr2=0x7f68a014f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:10.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.836+0000 7f689de00640 1 --2- 172.21.15.86:0/3328307213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68a00d06f0 0x7f68a014f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:10.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.836+0000 7f689de00640 1 -- 172.21.15.86:0/3328307213 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a014e0c0 con 0x7f68a006f060 2024-04-25T14:05:10.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.836+0000 7f6895400640 1 --2- 172.21.15.86:0/3328307213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68a00d06f0 0x7f68a014f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:10.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.836+0000 7f689de00640 1 --2- 172.21.15.86:0/3328307213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68a006f060 0x7f68a014eb80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f68900ae040 tx=0x7f689001acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:10.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.836+0000 7f6896a00640 1 -- 172.21.15.86:0/3328307213 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68900bd070 con 0x7f68a006f060 2024-04-25T14:05:10.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.836+0000 7f6896a00640 1 -- 172.21.15.86:0/3328307213 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6890004030 con 0x7f68a006f060 2024-04-25T14:05:10.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.836+0000 7f6896a00640 1 -- 172.21.15.86:0/3328307213 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68900b0040 con 0x7f68a006f060 2024-04-25T14:05:10.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.836+0000 7f689f200640 1 -- 172.21.15.86:0/3328307213 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a014ff90 con 0x7f68a006f060 2024-04-25T14:05:10.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.836+0000 7f689f200640 1 -- 172.21.15.86:0/3328307213 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a014e660 con 0x7f68a006f060 2024-04-25T14:05:10.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.836+0000 7f689f200640 1 -- 172.21.15.86:0/3328307213 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68a0076b80 con 0x7f68a006f060 2024-04-25T14:05:10.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.840+0000 7f6896a00640 1 -- 172.21.15.86:0/3328307213 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f68900c0070 con 0x7f68a006f060 2024-04-25T14:05:10.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.840+0000 7f6896a00640 1 --2- 172.21.15.86:0/3328307213 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f687c06f430 0x7f687c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:10.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.840+0000 7f6896a00640 1 -- 172.21.15.86:0/3328307213 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f68901290e0 con 0x7f68a006f060 2024-04-25T14:05:10.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.840+0000 7f6895400640 1 --2- 172.21.15.86:0/3328307213 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f687c06f430 0x7f687c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:10.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.840+0000 7f6895400640 1 --2- 172.21.15.86:0/3328307213 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f687c06f430 0x7f687c071940 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f68a00cd6f0 tx=0x7f6888008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:10.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:10.844+0000 7f6896a00640 1 -- 172.21.15.86:0/3328307213 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6890016070 con 0x7f68a006f060 2024-04-25T14:05:11.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.004+0000 7f689f200640 1 -- 172.21.15.86:0/3328307213 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f68a00d30f0 con 0x7f68a006f060 2024-04-25T14:05:11.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.004+0000 7f6896a00640 1 -- 172.21.15.86:0/3328307213 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16942 (secure 0 0 0) 0x7f68900f5750 con 0x7f68a006f060 2024-04-25T14:05:11.007 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:11.007 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":38,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:05:08.245617+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:05:04.472914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T14:05:05.604280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:05:11.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.008+0000 7f689f200640 1 -- 172.21.15.86:0/3328307213 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f687c06f430 msgr2=0x7f687c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.008+0000 7f689f200640 1 --2- 172.21.15.86:0/3328307213 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f687c06f430 0x7f687c071940 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f68a00cd6f0 tx=0x7f6888008040 comp rx=0 tx=0).stop 2024-04-25T14:05:11.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.008+0000 7f689f200640 1 -- 172.21.15.86:0/3328307213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68a006f060 msgr2=0x7f68a014eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.008+0000 7f689f200640 1 --2- 172.21.15.86:0/3328307213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68a006f060 0x7f68a014eb80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f68900ae040 tx=0x7f689001acc0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.008+0000 7f689f200640 1 -- 172.21.15.86:0/3328307213 shutdown_connections 2024-04-25T14:05:11.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.008+0000 7f689f200640 1 --2- 172.21.15.86:0/3328307213 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f687c06f430 0x7f687c071940 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.008+0000 7f689f200640 1 --2- 172.21.15.86:0/3328307213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68a01635a0 0x7f68a0148dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.008+0000 7f689f200640 1 --2- 172.21.15.86:0/3328307213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68a00d06f0 0x7f68a014f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.008+0000 7f689f200640 1 --2- 172.21.15.86:0/3328307213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68a006f060 0x7f68a014eb80 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.008+0000 7f689f200640 1 -- 172.21.15.86:0/3328307213 >> 172.21.15.86:0/3328307213 conn(0x7f68a00b4f80 msgr2=0x7f68a00d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:11.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.008+0000 7f689f200640 1 -- 172.21.15.86:0/3328307213 shutdown_connections 2024-04-25T14:05:11.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.008+0000 7f689f200640 1 -- 172.21.15.86:0/3328307213 wait complete. 2024-04-25T14:05:11.029 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T14:05:11.029 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:05:11.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.172+0000 7f63f5e00640 1 Processor -- start 2024-04-25T14:05:11.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.172+0000 7f63f5e00640 1 -- start start 2024-04-25T14:05:11.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.176+0000 7f63f5e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63f00cfca0 0x7f63f006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.176+0000 7f63f5e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f63f00d16e0 0x7f63f00d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.176+0000 7f63f5e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63f00d2030 con 0x7f63f006c670 2024-04-25T14:05:11.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.176+0000 7f63f5e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f63f0070470 con 0x7f63f00d16e0 2024-04-25T14:05:11.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.176+0000 7f63f5e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f63f0072bd0 con 0x7f63f00cfca0 2024-04-25T14:05:11.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.176+0000 7f63e7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f63f00d16e0 0x7f63f00d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.176+0000 7f63e7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f63f00d16e0 0x7f63f00d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57518/0 (socket says 172.21.15.86:57518) 2024-04-25T14:05:11.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.176+0000 7f63e7e00640 1 -- 172.21.15.86:0/1399293550 learned_addr learned my addr 172.21.15.86:0/1399293550 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:11.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.176+0000 7f63eea00640 1 --2- 172.21.15.86:0/1399293550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63f00cfca0 0x7f63f006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.176+0000 7f63e7e00640 1 -- 172.21.15.86:0/1399293550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63f00cfca0 msgr2=0x7f63f006f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:11.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.176+0000 7f63e7e00640 1 --2- 172.21.15.86:0/1399293550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63f00cfca0 0x7f63f006f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.176+0000 7f63e7e00640 1 -- 172.21.15.86:0/1399293550 >> v1:172.21.15.155:6789/0 conn(0x7f63f006c670 legacy=0x7f63f006ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.176+0000 7f63e7e00640 1 -- 172.21.15.86:0/1399293550 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f006f930 con 0x7f63f00d16e0 2024-04-25T14:05:11.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.180+0000 7f63e7e00640 1 --2- 172.21.15.86:0/1399293550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63f00d16e0 0x7f63f00d1af0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f63d8009870 tx=0x7f63d801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a793c69b0f5b72e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:11.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.180+0000 7f63ee000640 1 -- 172.21.15.86:0/1399293550 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d80a7070 con 0x7f63f00d16e0 2024-04-25T14:05:11.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.180+0000 7f63ee000640 1 -- 172.21.15.86:0/1399293550 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63d80b0070 con 0x7f63f00d16e0 2024-04-25T14:05:11.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.180+0000 7f63ee000640 1 -- 172.21.15.86:0/1399293550 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d80ab890 con 0x7f63f00d16e0 2024-04-25T14:05:11.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.180+0000 7f63f5e00640 1 -- 172.21.15.86:0/1399293550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63f00d16e0 msgr2=0x7f63f00d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.180+0000 7f63f5e00640 1 --2- 172.21.15.86:0/1399293550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63f00d16e0 0x7f63f00d1af0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f63d8009870 tx=0x7f63d801c560 comp rx=0 tx=0).stop 2024-04-25T14:05:11.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.180+0000 7f63f5e00640 1 -- 172.21.15.86:0/1399293550 shutdown_connections 2024-04-25T14:05:11.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.180+0000 7f63f5e00640 1 --2- 172.21.15.86:0/1399293550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63f00d16e0 0x7f63f00d1af0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.180+0000 7f63f5e00640 1 --2- 172.21.15.86:0/1399293550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63f00cfca0 0x7f63f006f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.180+0000 7f63f5e00640 1 -- 172.21.15.86:0/1399293550 >> 172.21.15.86:0/1399293550 conn(0x7f63f00b4650 msgr2=0x7f63f00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:11.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.180+0000 7f63f5e00640 1 -- 172.21.15.86:0/1399293550 shutdown_connections 2024-04-25T14:05:11.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.180+0000 7f63f5e00640 1 -- 172.21.15.86:0/1399293550 wait complete. 2024-04-25T14:05:11.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.184+0000 7f63f5e00640 1 Processor -- start 2024-04-25T14:05:11.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.184+0000 7f63f5e00640 1 -- start start 2024-04-25T14:05:11.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.184+0000 7f63f5e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63f006c670 0x7f63f00cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.184+0000 7f63f5e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63f00cfca0 0x7f63f01f85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.184+0000 7f63f5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63f00d16e0 0x7f63f00cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.184+0000 7f63f5e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f63f016a070 con 0x7f63f00d16e0 2024-04-25T14:05:11.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.184+0000 7f63f5e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f63f0169ef0 con 0x7f63f006c670 2024-04-25T14:05:11.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.184+0000 7f63f5e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f63f016a1f0 con 0x7f63f00cfca0 2024-04-25T14:05:11.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.184+0000 7f63ef400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63f00d16e0 0x7f63f00cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.184+0000 7f63ef400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63f00d16e0 0x7f63f00cde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57532/0 (socket says 172.21.15.86:57532) 2024-04-25T14:05:11.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.184+0000 7f63ef400640 1 -- 172.21.15.86:0/390780169 learned_addr learned my addr 172.21.15.86:0/390780169 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:11.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.184+0000 7f63e7e00640 1 --2- 172.21.15.86:0/390780169 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63f00cfca0 0x7f63f01f85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.184+0000 7f63eea00640 1 --2- 172.21.15.86:0/390780169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63f006c670 0x7f63f00cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.184+0000 7f63ef400640 1 -- 172.21.15.86:0/390780169 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63f00cfca0 msgr2=0x7f63f01f85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.184+0000 7f63ef400640 1 --2- 172.21.15.86:0/390780169 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63f00cfca0 0x7f63f01f85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.184+0000 7f63ef400640 1 -- 172.21.15.86:0/390780169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63f006c670 msgr2=0x7f63f00cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.184+0000 7f63ef400640 1 --2- 172.21.15.86:0/390780169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63f006c670 0x7f63f00cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.184+0000 7f63ef400640 1 -- 172.21.15.86:0/390780169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f015f990 con 0x7f63f00d16e0 2024-04-25T14:05:11.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.184+0000 7f63eea00640 1 --2- 172.21.15.86:0/390780169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63f006c670 0x7f63f00cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:11.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.184+0000 7f63ef400640 1 --2- 172.21.15.86:0/390780169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63f00d16e0 0x7f63f00cde10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f63e800c760 tx=0x7f63e800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:11.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.184+0000 7f63e7400640 1 -- 172.21.15.86:0/390780169 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e8003b80 con 0x7f63f00d16e0 2024-04-25T14:05:11.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.184+0000 7f63e7400640 1 -- 172.21.15.86:0/390780169 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63e8003d20 con 0x7f63f00d16e0 2024-04-25T14:05:11.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.184+0000 7f63f5e00640 1 -- 172.21.15.86:0/390780169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63f0099dd0 con 0x7f63f00d16e0 2024-04-25T14:05:11.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.184+0000 7f63e7400640 1 -- 172.21.15.86:0/390780169 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e800aa90 con 0x7f63f00d16e0 2024-04-25T14:05:11.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.184+0000 7f63f5e00640 1 -- 172.21.15.86:0/390780169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63f015fc30 con 0x7f63f00d16e0 2024-04-25T14:05:11.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.188+0000 7f63f5e00640 1 -- 172.21.15.86:0/390780169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63f009f4b0 con 0x7f63f00d16e0 2024-04-25T14:05:11.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.188+0000 7f63e7400640 1 -- 172.21.15.86:0/390780169 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f63e800ac30 con 0x7f63f00d16e0 2024-04-25T14:05:11.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.188+0000 7f63e7400640 1 --2- 172.21.15.86:0/390780169 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f63cc06f400 0x7f63cc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.188+0000 7f63e7400640 1 -- 172.21.15.86:0/390780169 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f63e808f230 con 0x7f63f00d16e0 2024-04-25T14:05:11.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.188+0000 7f63eea00640 1 --2- 172.21.15.86:0/390780169 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f63cc06f400 0x7f63cc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.188+0000 7f63eea00640 1 --2- 172.21.15.86:0/390780169 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f63cc06f400 0x7f63cc071910 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f63f0070710 tx=0x7f63e0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:11.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.192+0000 7f63e7400640 1 -- 172.21.15.86:0/390780169 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f63f00d2030 con 0x7f63f00d16e0 2024-04-25T14:05:11.222 INFO:tasks.ceph.mds.c.smithi086.stderr:2024-04-25T14:05:11.220+0000 7f92b6c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T14:05:11.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.352+0000 7f63f5e00640 1 -- 172.21.15.86:0/390780169 --> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f63f0069d00 con 0x7f63cc06f400 2024-04-25T14:05:11.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.360+0000 7f63e7400640 1 -- 172.21.15.86:0/390780169 <== mgr.4103 v2:172.21.15.86:6832/2968869755 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251946 (secure 0 0 0) 0x7f63f0069d00 con 0x7f63cc06f400 2024-04-25T14:05:11.364 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:11.366 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:05:11.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.364+0000 7f63f5e00640 1 -- 172.21.15.86:0/390780169 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f63cc06f400 msgr2=0x7f63cc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.364+0000 7f63f5e00640 1 --2- 172.21.15.86:0/390780169 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f63cc06f400 0x7f63cc071910 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f63f0070710 tx=0x7f63e0008040 comp rx=0 tx=0).stop 2024-04-25T14:05:11.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.364+0000 7f63f5e00640 1 -- 172.21.15.86:0/390780169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63f00d16e0 msgr2=0x7f63f00cde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.364+0000 7f63f5e00640 1 --2- 172.21.15.86:0/390780169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63f00d16e0 0x7f63f00cde10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f63e800c760 tx=0x7f63e800cc20 comp rx=0 tx=0).stop 2024-04-25T14:05:11.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.364+0000 7f63f5e00640 1 -- 172.21.15.86:0/390780169 shutdown_connections 2024-04-25T14:05:11.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.364+0000 7f63f5e00640 1 --2- 172.21.15.86:0/390780169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63f00d16e0 0x7f63f00cde10 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.364+0000 7f63f5e00640 1 --2- 172.21.15.86:0/390780169 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f63cc06f400 0x7f63cc071910 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.364+0000 7f63f5e00640 1 --2- 172.21.15.86:0/390780169 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63f00cfca0 0x7f63f01f85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.364+0000 7f63f5e00640 1 --2- 172.21.15.86:0/390780169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63f006c670 0x7f63f00cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.364+0000 7f63f5e00640 1 -- 172.21.15.86:0/390780169 >> 172.21.15.86:0/390780169 conn(0x7f63f00b4650 msgr2=0x7f63f00ccbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:11.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.364+0000 7f63f5e00640 1 -- 172.21.15.86:0/390780169 shutdown_connections 2024-04-25T14:05:11.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.364+0000 7f63f5e00640 1 -- 172.21.15.86:0/390780169 wait complete. 2024-04-25T14:05:11.385 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-04-25T14:05:10.729240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":138,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220768,"kb_used_data":4588,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753952,"statfs":{"total":773094113280,"available":772868046848,"internally_reserved":0,"allocated":4698112,"data_stored":2672320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.498378"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252558+0000","last_change":"2024-04-25T14:05:07.256777+0000","last_active":"2024-04-25T14:05:08.252558+0000","last_peered":"2024-04-25T14:05:08.252558+0000","last_clean":"2024-04-25T14:05:08.252558+0000","last_became_active":"2024-04-25T14:05:07.256450+0000","last_became_peered":"2024-04-25T14:05:07.256450+0000","last_unstale":"2024-04-25T14:05:08.252558+0000","last_undegraded":"2024-04-25T14:05:08.252558+0000","last_fullsized":"2024-04-25T14:05:08.252558+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:13.992505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252183+0000","last_change":"2024-04-25T14:05:07.274428+0000","last_active":"2024-04-25T14:05:08.252183+0000","last_peered":"2024-04-25T14:05:08.252183+0000","last_clean":"2024-04-25T14:05:08.252183+0000","last_became_active":"2024-04-25T14:05:07.274121+0000","last_became_peered":"2024-04-25T14:05:07.274121+0000","last_unstale":"2024-04-25T14:05:08.252183+0000","last_undegraded":"2024-04-25T14:05:08.252183+0000","last_fullsized":"2024-04-25T14:05:08.252183+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:43:48.479853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283661+0000","last_change":"2024-04-25T14:05:07.261520+0000","last_active":"2024-04-25T14:05:08.283661+0000","last_peered":"2024-04-25T14:05:08.283661+0000","last_clean":"2024-04-25T14:05:08.283661+0000","last_became_active":"2024-04-25T14:05:07.261158+0000","last_became_peered":"2024-04-25T14:05:07.261158+0000","last_unstale":"2024-04-25T14:05:08.283661+0000","last_undegraded":"2024-04-25T14:05:08.283661+0000","last_fullsized":"2024-04-25T14:05:08.283661+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:48:18.920873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.279423+0000","last_change":"2024-04-25T14:05:07.278871+0000","last_active":"2024-04-25T14:05:08.279423+0000","last_peered":"2024-04-25T14:05:08.279423+0000","last_clean":"2024-04-25T14:05:08.279423+0000","last_became_active":"2024-04-25T14:05:07.278018+0000","last_became_peered":"2024-04-25T14:05:07.278018+0000","last_unstale":"2024-04-25T14:05:08.279423+0000","last_undegraded":"2024-04-25T14:05:08.279423+0000","last_fullsized":"2024-04-25T14:05:08.279423+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:20.044385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.279352+0000","last_change":"2024-04-25T14:05:07.286426+0000","last_active":"2024-04-25T14:05:08.279352+0000","last_peered":"2024-04-25T14:05:08.279352+0000","last_clean":"2024-04-25T14:05:08.279352+0000","last_became_active":"2024-04-25T14:05:07.286114+0000","last_became_peered":"2024-04-25T14:05:07.286114+0000","last_unstale":"2024-04-25T14:05:08.279352+0000","last_undegraded":"2024-04-25T14:05:08.279352+0000","last_fullsized":"2024-04-25T14:05:08.279352+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:09.272639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.284244+0000","last_change":"2024-04-25T14:05:07.285651+0000","last_active":"2024-04-25T14:05:08.284244+0000","last_peered":"2024-04-25T14:05:08.284244+0000","last_clean":"2024-04-25T14:05:08.284244+0000","last_became_active":"2024-04-25T14:05:07.285337+0000","last_became_peered":"2024-04-25T14:05:07.285337+0000","last_unstale":"2024-04-25T14:05:08.284244+0000","last_undegraded":"2024-04-25T14:05:08.284244+0000","last_fullsized":"2024-04-25T14:05:08.284244+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:35.998555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.282722+0000","last_change":"2024-04-25T14:05:07.256962+0000","last_active":"2024-04-25T14:05:08.282722+0000","last_peered":"2024-04-25T14:05:08.282722+0000","last_clean":"2024-04-25T14:05:08.282722+0000","last_became_active":"2024-04-25T14:05:07.256595+0000","last_became_peered":"2024-04-25T14:05:07.256595+0000","last_unstale":"2024-04-25T14:05:08.282722+0000","last_undegraded":"2024-04-25T14:05:08.282722+0000","last_fullsized":"2024-04-25T14:05:08.282722+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:48:18.920873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.251979+0000","last_change":"2024-04-25T14:05:07.263138+0000","last_active":"2024-04-25T14:05:08.251979+0000","last_peered":"2024-04-25T14:05:08.251979+0000","last_clean":"2024-04-25T14:05:08.251979+0000","last_became_active":"2024-04-25T14:05:07.262819+0000","last_became_peered":"2024-04-25T14:05:07.262819+0000","last_unstale":"2024-04-25T14:05:08.251979+0000","last_undegraded":"2024-04-25T14:05:08.251979+0000","last_fullsized":"2024-04-25T14:05:08.251979+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:41.508799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.284059+0000","last_change":"2024-04-25T14:05:07.264088+0000","last_active":"2024-04-25T14:05:08.284059+0000","last_peered":"2024-04-25T14:05:08.284059+0000","last_clean":"2024-04-25T14:05:08.284059+0000","last_became_active":"2024-04-25T14:05:07.261204+0000","last_became_peered":"2024-04-25T14:05:07.261204+0000","last_unstale":"2024-04-25T14:05:08.284059+0000","last_undegraded":"2024-04-25T14:05:08.284059+0000","last_fullsized":"2024-04-25T14:05:08.284059+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:43:48.479853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283119+0000","last_change":"2024-04-25T14:05:07.266609+0000","last_active":"2024-04-25T14:05:08.283119+0000","last_peered":"2024-04-25T14:05:08.283119+0000","last_clean":"2024-04-25T14:05:08.283119+0000","last_became_active":"2024-04-25T14:05:07.266297+0000","last_became_peered":"2024-04-25T14:05:07.266297+0000","last_unstale":"2024-04-25T14:05:08.283119+0000","last_undegraded":"2024-04-25T14:05:08.283119+0000","last_fullsized":"2024-04-25T14:05:08.283119+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:56.697086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283958+0000","last_change":"2024-04-25T14:05:07.273257+0000","last_active":"2024-04-25T14:05:08.283958+0000","last_peered":"2024-04-25T14:05:08.283958+0000","last_clean":"2024-04-25T14:05:08.283958+0000","last_became_active":"2024-04-25T14:05:07.272896+0000","last_became_peered":"2024-04-25T14:05:07.272896+0000","last_unstale":"2024-04-25T14:05:08.283958+0000","last_undegraded":"2024-04-25T14:05:08.283958+0000","last_fullsized":"2024-04-25T14:05:08.283958+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:54.988190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283589+0000","last_change":"2024-04-25T14:05:07.271933+0000","last_active":"2024-04-25T14:05:08.283589+0000","last_peered":"2024-04-25T14:05:08.283589+0000","last_clean":"2024-04-25T14:05:08.283589+0000","last_became_active":"2024-04-25T14:05:07.271620+0000","last_became_peered":"2024-04-25T14:05:07.271620+0000","last_unstale":"2024-04-25T14:05:08.283589+0000","last_undegraded":"2024-04-25T14:05:08.283589+0000","last_fullsized":"2024-04-25T14:05:08.283589+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:09.919025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283820+0000","last_change":"2024-04-25T14:05:07.276415+0000","last_active":"2024-04-25T14:05:08.283820+0000","last_peered":"2024-04-25T14:05:08.283820+0000","last_clean":"2024-04-25T14:05:08.283820+0000","last_became_active":"2024-04-25T14:05:07.275613+0000","last_became_peered":"2024-04-25T14:05:07.275613+0000","last_unstale":"2024-04-25T14:05:08.283820+0000","last_undegraded":"2024-04-25T14:05:08.283820+0000","last_fullsized":"2024-04-25T14:05:08.283820+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:48.719845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283695+0000","last_change":"2024-04-25T14:05:07.269463+0000","last_active":"2024-04-25T14:05:08.283695+0000","last_peered":"2024-04-25T14:05:08.283695+0000","last_clean":"2024-04-25T14:05:08.283695+0000","last_became_active":"2024-04-25T14:05:07.269188+0000","last_became_peered":"2024-04-25T14:05:07.269188+0000","last_unstale":"2024-04-25T14:05:08.283695+0000","last_undegraded":"2024-04-25T14:05:08.283695+0000","last_fullsized":"2024-04-25T14:05:08.283695+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:13.217271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.282717+0000","last_change":"2024-04-25T14:05:07.273251+0000","last_active":"2024-04-25T14:05:08.282717+0000","last_peered":"2024-04-25T14:05:08.282717+0000","last_clean":"2024-04-25T14:05:08.282717+0000","last_became_active":"2024-04-25T14:05:07.272878+0000","last_became_peered":"2024-04-25T14:05:07.272878+0000","last_unstale":"2024-04-25T14:05:08.282717+0000","last_undegraded":"2024-04-25T14:05:08.282717+0000","last_fullsized":"2024-04-25T14:05:08.282717+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:43:48.479853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252191+0000","last_change":"2024-04-25T14:05:07.284939+0000","last_active":"2024-04-25T14:05:08.252191+0000","last_peered":"2024-04-25T14:05:08.252191+0000","last_clean":"2024-04-25T14:05:08.252191+0000","last_became_active":"2024-04-25T14:05:07.284619+0000","last_became_peered":"2024-04-25T14:05:07.284619+0000","last_unstale":"2024-04-25T14:05:08.252191+0000","last_undegraded":"2024-04-25T14:05:08.252191+0000","last_fullsized":"2024-04-25T14:05:08.252191+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:35.998555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252837+0000","last_change":"2024-04-25T14:05:07.291086+0000","last_active":"2024-04-25T14:05:08.252837+0000","last_peered":"2024-04-25T14:05:08.252837+0000","last_clean":"2024-04-25T14:05:08.252837+0000","last_became_active":"2024-04-25T14:05:07.290811+0000","last_became_peered":"2024-04-25T14:05:07.290811+0000","last_unstale":"2024-04-25T14:05:08.252837+0000","last_undegraded":"2024-04-25T14:05:08.252837+0000","last_fullsized":"2024-04-25T14:05:08.252837+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:03.144757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.251877+0000","last_change":"2024-04-25T14:05:07.275169+0000","last_active":"2024-04-25T14:05:08.251877+0000","last_peered":"2024-04-25T14:05:08.251877+0000","last_clean":"2024-04-25T14:05:08.251877+0000","last_became_active":"2024-04-25T14:05:07.274587+0000","last_became_peered":"2024-04-25T14:05:07.274587+0000","last_unstale":"2024-04-25T14:05:08.251877+0000","last_undegraded":"2024-04-25T14:05:08.251877+0000","last_fullsized":"2024-04-25T14:05:08.251877+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:55.634401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.279574+0000","last_change":"2024-04-25T14:05:07.280608+0000","last_active":"2024-04-25T14:05:08.279574+0000","last_peered":"2024-04-25T14:05:08.279574+0000","last_clean":"2024-04-25T14:05:08.279574+0000","last_became_active":"2024-04-25T14:05:07.280256+0000","last_became_peered":"2024-04-25T14:05:07.280256+0000","last_unstale":"2024-04-25T14:05:08.279574+0000","last_undegraded":"2024-04-25T14:05:08.279574+0000","last_fullsized":"2024-04-25T14:05:08.279574+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:21.535968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.251707+0000","last_change":"2024-04-25T14:05:07.274877+0000","last_active":"2024-04-25T14:05:08.251707+0000","last_peered":"2024-04-25T14:05:08.251707+0000","last_clean":"2024-04-25T14:05:08.251707+0000","last_became_active":"2024-04-25T14:05:07.274223+0000","last_became_peered":"2024-04-25T14:05:07.274223+0000","last_unstale":"2024-04-25T14:05:08.251707+0000","last_undegraded":"2024-04-25T14:05:08.251707+0000","last_fullsized":"2024-04-25T14:05:08.251707+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:21.535968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283121+0000","last_change":"2024-04-25T14:05:07.276724+0000","last_active":"2024-04-25T14:05:08.283121+0000","last_peered":"2024-04-25T14:05:08.283121+0000","last_clean":"2024-04-25T14:05:08.283121+0000","last_became_active":"2024-04-25T14:05:07.276036+0000","last_became_peered":"2024-04-25T14:05:07.276036+0000","last_unstale":"2024-04-25T14:05:08.283121+0000","last_undegraded":"2024-04-25T14:05:08.283121+0000","last_fullsized":"2024-04-25T14:05:08.283121+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:51.159581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283884+0000","last_change":"2024-04-25T14:05:07.274388+0000","last_active":"2024-04-25T14:05:08.283884+0000","last_peered":"2024-04-25T14:05:08.283884+0000","last_clean":"2024-04-25T14:05:08.283884+0000","last_became_active":"2024-04-25T14:05:07.273384+0000","last_became_peered":"2024-04-25T14:05:07.273384+0000","last_unstale":"2024-04-25T14:05:08.283884+0000","last_undegraded":"2024-04-25T14:05:08.283884+0000","last_fullsized":"2024-04-25T14:05:08.283884+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:01:19.113121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252284+0000","last_change":"2024-04-25T14:05:07.273281+0000","last_active":"2024-04-25T14:05:08.252284+0000","last_peered":"2024-04-25T14:05:08.252284+0000","last_clean":"2024-04-25T14:05:08.252284+0000","last_became_active":"2024-04-25T14:05:07.272962+0000","last_became_peered":"2024-04-25T14:05:07.272962+0000","last_unstale":"2024-04-25T14:05:08.252284+0000","last_undegraded":"2024-04-25T14:05:08.252284+0000","last_fullsized":"2024-04-25T14:05:08.252284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:39.756595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.279709+0000","last_change":"2024-04-25T14:05:07.268205+0000","last_active":"2024-04-25T14:05:08.279709+0000","last_peered":"2024-04-25T14:05:08.279709+0000","last_clean":"2024-04-25T14:05:08.279709+0000","last_became_active":"2024-04-25T14:05:07.267900+0000","last_became_peered":"2024-04-25T14:05:07.267900+0000","last_unstale":"2024-04-25T14:05:08.279709+0000","last_undegraded":"2024-04-25T14:05:08.279709+0000","last_fullsized":"2024-04-25T14:05:08.279709+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:55.634401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.251924+0000","last_change":"2024-04-25T14:05:07.258076+0000","last_active":"2024-04-25T14:05:08.251924+0000","last_peered":"2024-04-25T14:05:08.251924+0000","last_clean":"2024-04-25T14:05:08.251924+0000","last_became_active":"2024-04-25T14:05:07.257653+0000","last_became_peered":"2024-04-25T14:05:07.257653+0000","last_unstale":"2024-04-25T14:05:08.251924+0000","last_undegraded":"2024-04-25T14:05:08.251924+0000","last_fullsized":"2024-04-25T14:05:08.251924+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:55:55.223962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283926+0000","last_change":"2024-04-25T14:05:07.285796+0000","last_active":"2024-04-25T14:05:08.283926+0000","last_peered":"2024-04-25T14:05:08.283926+0000","last_clean":"2024-04-25T14:05:08.283926+0000","last_became_active":"2024-04-25T14:05:07.284802+0000","last_became_peered":"2024-04-25T14:05:07.284802+0000","last_unstale":"2024-04-25T14:05:08.283926+0000","last_undegraded":"2024-04-25T14:05:08.283926+0000","last_fullsized":"2024-04-25T14:05:08.283926+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:20.169051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.251941+0000","last_change":"2024-04-25T14:05:07.268639+0000","last_active":"2024-04-25T14:05:08.251941+0000","last_peered":"2024-04-25T14:05:08.251941+0000","last_clean":"2024-04-25T14:05:08.251941+0000","last_became_active":"2024-04-25T14:05:07.268256+0000","last_became_peered":"2024-04-25T14:05:07.268256+0000","last_unstale":"2024-04-25T14:05:08.251941+0000","last_undegraded":"2024-04-25T14:05:08.251941+0000","last_fullsized":"2024-04-25T14:05:08.251941+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:13.217271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283077+0000","last_change":"2024-04-25T14:05:07.283026+0000","last_active":"2024-04-25T14:05:08.283077+0000","last_peered":"2024-04-25T14:05:08.283077+0000","last_clean":"2024-04-25T14:05:08.283077+0000","last_became_active":"2024-04-25T14:05:07.282288+0000","last_became_peered":"2024-04-25T14:05:07.282288+0000","last_unstale":"2024-04-25T14:05:08.283077+0000","last_undegraded":"2024-04-25T14:05:08.283077+0000","last_fullsized":"2024-04-25T14:05:08.283077+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:54.320451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.251680+0000","last_change":"2024-04-25T14:05:07.275412+0000","last_active":"2024-04-25T14:05:08.251680+0000","last_peered":"2024-04-25T14:05:08.251680+0000","last_clean":"2024-04-25T14:05:08.251680+0000","last_became_active":"2024-04-25T14:05:07.275072+0000","last_became_peered":"2024-04-25T14:05:07.275072+0000","last_unstale":"2024-04-25T14:05:08.251680+0000","last_undegraded":"2024-04-25T14:05:08.251680+0000","last_fullsized":"2024-04-25T14:05:08.251680+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:20.044385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.251724+0000","last_change":"2024-04-25T14:05:07.258642+0000","last_active":"2024-04-25T14:05:08.251724+0000","last_peered":"2024-04-25T14:05:08.251724+0000","last_clean":"2024-04-25T14:05:08.251724+0000","last_became_active":"2024-04-25T14:05:07.258273+0000","last_became_peered":"2024-04-25T14:05:07.258273+0000","last_unstale":"2024-04-25T14:05:08.251724+0000","last_undegraded":"2024-04-25T14:05:08.251724+0000","last_fullsized":"2024-04-25T14:05:08.251724+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:08.852015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252300+0000","last_change":"2024-04-25T14:05:07.275998+0000","last_active":"2024-04-25T14:05:08.252300+0000","last_peered":"2024-04-25T14:05:08.252300+0000","last_clean":"2024-04-25T14:05:08.252300+0000","last_became_active":"2024-04-25T14:05:07.275663+0000","last_became_peered":"2024-04-25T14:05:07.275663+0000","last_unstale":"2024-04-25T14:05:08.252300+0000","last_undegraded":"2024-04-25T14:05:08.252300+0000","last_fullsized":"2024-04-25T14:05:08.252300+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:36.823492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.282977+0000","last_change":"2024-04-25T14:05:07.273743+0000","last_active":"2024-04-25T14:05:08.282977+0000","last_peered":"2024-04-25T14:05:08.282977+0000","last_clean":"2024-04-25T14:05:08.282977+0000","last_became_active":"2024-04-25T14:05:07.273435+0000","last_became_peered":"2024-04-25T14:05:07.273435+0000","last_unstale":"2024-04-25T14:05:08.282977+0000","last_undegraded":"2024-04-25T14:05:08.282977+0000","last_fullsized":"2024-04-25T14:05:08.282977+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:43.913161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252687+0000","last_change":"2024-04-25T14:05:07.286228+0000","last_active":"2024-04-25T14:05:08.252687+0000","last_peered":"2024-04-25T14:05:08.252687+0000","last_clean":"2024-04-25T14:05:08.252687+0000","last_became_active":"2024-04-25T14:05:07.285840+0000","last_became_peered":"2024-04-25T14:05:07.285840+0000","last_unstale":"2024-04-25T14:05:08.252687+0000","last_undegraded":"2024-04-25T14:05:08.252687+0000","last_fullsized":"2024-04-25T14:05:08.252687+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:48:18.920873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.251921+0000","last_change":"2024-04-25T14:05:06.263372+0000","last_active":"2024-04-25T14:05:08.251921+0000","last_peered":"2024-04-25T14:05:08.251921+0000","last_clean":"2024-04-25T14:05:08.251921+0000","last_became_active":"2024-04-25T14:05:06.262221+0000","last_became_peered":"2024-04-25T14:05:06.262221+0000","last_unstale":"2024-04-25T14:05:08.251921+0000","last_undegraded":"2024-04-25T14:05:08.251921+0000","last_fullsized":"2024-04-25T14:05:08.251921+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:20.185798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.284278+0000","last_change":"2024-04-25T14:05:07.286687+0000","last_active":"2024-04-25T14:05:08.284278+0000","last_peered":"2024-04-25T14:05:08.284278+0000","last_clean":"2024-04-25T14:05:08.284278+0000","last_became_active":"2024-04-25T14:05:07.286339+0000","last_became_peered":"2024-04-25T14:05:07.286339+0000","last_unstale":"2024-04-25T14:05:08.284278+0000","last_undegraded":"2024-04-25T14:05:08.284278+0000","last_fullsized":"2024-04-25T14:05:08.284278+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:20.873445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283628+0000","last_change":"2024-04-25T14:05:06.260377+0000","last_active":"2024-04-25T14:05:08.283628+0000","last_peered":"2024-04-25T14:05:08.283628+0000","last_clean":"2024-04-25T14:05:08.283628+0000","last_became_active":"2024-04-25T14:05:06.259976+0000","last_became_peered":"2024-04-25T14:05:06.259976+0000","last_unstale":"2024-04-25T14:05:08.283628+0000","last_undegraded":"2024-04-25T14:05:08.283628+0000","last_fullsized":"2024-04-25T14:05:08.283628+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:20.530061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.284307+0000","last_change":"2024-04-25T14:05:07.271748+0000","last_active":"2024-04-25T14:05:08.284307+0000","last_peered":"2024-04-25T14:05:08.284307+0000","last_clean":"2024-04-25T14:05:08.284307+0000","last_became_active":"2024-04-25T14:05:07.271244+0000","last_became_peered":"2024-04-25T14:05:07.271244+0000","last_unstale":"2024-04-25T14:05:08.284307+0000","last_undegraded":"2024-04-25T14:05:08.284307+0000","last_fullsized":"2024-04-25T14:05:08.284307+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:35.998555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.284023+0000","last_change":"2024-04-25T14:05:06.258406+0000","last_active":"2024-04-25T14:05:08.284023+0000","last_peered":"2024-04-25T14:05:08.284023+0000","last_clean":"2024-04-25T14:05:08.284023+0000","last_became_active":"2024-04-25T14:05:06.258080+0000","last_became_peered":"2024-04-25T14:05:06.258080+0000","last_unstale":"2024-04-25T14:05:08.284023+0000","last_undegraded":"2024-04-25T14:05:08.284023+0000","last_fullsized":"2024-04-25T14:05:08.284023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:29.416289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252572+0000","last_change":"2024-04-25T14:05:07.258537+0000","last_active":"2024-04-25T14:05:08.252572+0000","last_peered":"2024-04-25T14:05:08.252572+0000","last_clean":"2024-04-25T14:05:08.252572+0000","last_became_active":"2024-04-25T14:05:07.258220+0000","last_became_peered":"2024-04-25T14:05:07.258220+0000","last_unstale":"2024-04-25T14:05:08.252572+0000","last_undegraded":"2024-04-25T14:05:08.252572+0000","last_fullsized":"2024-04-25T14:05:08.252572+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:10.356490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.284109+0000","last_change":"2024-04-25T14:05:06.258099+0000","last_active":"2024-04-25T14:05:08.284109+0000","last_peered":"2024-04-25T14:05:08.284109+0000","last_clean":"2024-04-25T14:05:08.284109+0000","last_became_active":"2024-04-25T14:05:06.255967+0000","last_became_peered":"2024-04-25T14:05:06.255967+0000","last_unstale":"2024-04-25T14:05:08.284109+0000","last_undegraded":"2024-04-25T14:05:08.284109+0000","last_fullsized":"2024-04-25T14:05:08.284109+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:49.855989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.282956+0000","last_change":"2024-04-25T14:05:07.269898+0000","last_active":"2024-04-25T14:05:08.282956+0000","last_peered":"2024-04-25T14:05:08.282956+0000","last_clean":"2024-04-25T14:05:08.282956+0000","last_became_active":"2024-04-25T14:05:07.268973+0000","last_became_peered":"2024-04-25T14:05:07.268973+0000","last_unstale":"2024-04-25T14:05:08.282956+0000","last_undegraded":"2024-04-25T14:05:08.282956+0000","last_fullsized":"2024-04-25T14:05:08.282956+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:35.998555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283282+0000","last_change":"2024-04-25T14:05:06.263356+0000","last_active":"2024-04-25T14:05:08.283282+0000","last_peered":"2024-04-25T14:05:08.283282+0000","last_clean":"2024-04-25T14:05:08.283282+0000","last_became_active":"2024-04-25T14:05:06.263007+0000","last_became_peered":"2024-04-25T14:05:06.263007+0000","last_unstale":"2024-04-25T14:05:08.283282+0000","last_undegraded":"2024-04-25T14:05:08.283282+0000","last_fullsized":"2024-04-25T14:05:08.283282+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:29.416289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283311+0000","last_change":"2024-04-25T14:05:07.270469+0000","last_active":"2024-04-25T14:05:08.283311+0000","last_peered":"2024-04-25T14:05:08.283311+0000","last_clean":"2024-04-25T14:05:08.283311+0000","last_became_active":"2024-04-25T14:05:07.270115+0000","last_became_peered":"2024-04-25T14:05:07.270115+0000","last_unstale":"2024-04-25T14:05:08.283311+0000","last_undegraded":"2024-04-25T14:05:08.283311+0000","last_fullsized":"2024-04-25T14:05:08.283311+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:10.356490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252421+0000","last_change":"2024-04-25T14:05:06.261873+0000","last_active":"2024-04-25T14:05:08.252421+0000","last_peered":"2024-04-25T14:05:08.252421+0000","last_clean":"2024-04-25T14:05:08.252421+0000","last_became_active":"2024-04-25T14:05:06.261493+0000","last_became_peered":"2024-04-25T14:05:06.261493+0000","last_unstale":"2024-04-25T14:05:08.252421+0000","last_undegraded":"2024-04-25T14:05:08.252421+0000","last_fullsized":"2024-04-25T14:05:08.252421+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:17.077511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.284102+0000","last_change":"2024-04-25T14:05:07.272920+0000","last_active":"2024-04-25T14:05:08.284102+0000","last_peered":"2024-04-25T14:05:08.284102+0000","last_clean":"2024-04-25T14:05:08.284102+0000","last_became_active":"2024-04-25T14:05:07.269884+0000","last_became_peered":"2024-04-25T14:05:07.269884+0000","last_unstale":"2024-04-25T14:05:08.284102+0000","last_undegraded":"2024-04-25T14:05:08.284102+0000","last_fullsized":"2024-04-25T14:05:08.284102+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:05:13.234083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283326+0000","last_change":"2024-04-25T14:05:06.265735+0000","last_active":"2024-04-25T14:05:08.283326+0000","last_peered":"2024-04-25T14:05:08.283326+0000","last_clean":"2024-04-25T14:05:08.283326+0000","last_became_active":"2024-04-25T14:05:06.265476+0000","last_became_peered":"2024-04-25T14:05:06.265476+0000","last_unstale":"2024-04-25T14:05:08.283326+0000","last_undegraded":"2024-04-25T14:05:08.283326+0000","last_fullsized":"2024-04-25T14:05:08.283326+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:08.266732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.251980+0000","last_change":"2024-04-25T14:05:07.270163+0000","last_active":"2024-04-25T14:05:08.251980+0000","last_peered":"2024-04-25T14:05:08.251980+0000","last_clean":"2024-04-25T14:05:08.251980+0000","last_became_active":"2024-04-25T14:05:07.268272+0000","last_became_peered":"2024-04-25T14:05:07.268272+0000","last_unstale":"2024-04-25T14:05:08.251980+0000","last_undegraded":"2024-04-25T14:05:08.251980+0000","last_fullsized":"2024-04-25T14:05:08.251980+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:48:18.920873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283500+0000","last_change":"2024-04-25T14:05:06.260729+0000","last_active":"2024-04-25T14:05:08.283500+0000","last_peered":"2024-04-25T14:05:08.283500+0000","last_clean":"2024-04-25T14:05:08.283500+0000","last_became_active":"2024-04-25T14:05:06.260022+0000","last_became_peered":"2024-04-25T14:05:06.260022+0000","last_unstale":"2024-04-25T14:05:08.283500+0000","last_undegraded":"2024-04-25T14:05:08.283500+0000","last_fullsized":"2024-04-25T14:05:08.283500+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:20.185798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.279590+0000","last_change":"2024-04-25T14:05:07.279074+0000","last_active":"2024-04-25T14:05:08.279590+0000","last_peered":"2024-04-25T14:05:08.279590+0000","last_clean":"2024-04-25T14:05:08.279590+0000","last_became_active":"2024-04-25T14:05:07.278499+0000","last_became_peered":"2024-04-25T14:05:07.278499+0000","last_unstale":"2024-04-25T14:05:08.279590+0000","last_undegraded":"2024-04-25T14:05:08.279590+0000","last_fullsized":"2024-04-25T14:05:08.279590+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:08.852015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252618+0000","last_change":"2024-04-25T14:05:06.251848+0000","last_active":"2024-04-25T14:05:08.252618+0000","last_peered":"2024-04-25T14:05:08.252618+0000","last_clean":"2024-04-25T14:05:08.252618+0000","last_became_active":"2024-04-25T14:05:06.251541+0000","last_became_peered":"2024-04-25T14:05:06.251541+0000","last_unstale":"2024-04-25T14:05:08.252618+0000","last_undegraded":"2024-04-25T14:05:08.252618+0000","last_fullsized":"2024-04-25T14:05:08.252618+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:55:54.218054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252655+0000","last_change":"2024-04-25T14:05:07.278976+0000","last_active":"2024-04-25T14:05:08.252655+0000","last_peered":"2024-04-25T14:05:08.252655+0000","last_clean":"2024-04-25T14:05:08.252655+0000","last_became_active":"2024-04-25T14:05:07.278721+0000","last_became_peered":"2024-04-25T14:05:07.278721+0000","last_unstale":"2024-04-25T14:05:08.252655+0000","last_undegraded":"2024-04-25T14:05:08.252655+0000","last_fullsized":"2024-04-25T14:05:08.252655+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:41.508799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283244+0000","last_change":"2024-04-25T14:05:06.259232+0000","last_active":"2024-04-25T14:05:08.283244+0000","last_peered":"2024-04-25T14:05:08.283244+0000","last_clean":"2024-04-25T14:05:08.283244+0000","last_became_active":"2024-04-25T14:05:06.258861+0000","last_became_peered":"2024-04-25T14:05:06.258861+0000","last_unstale":"2024-04-25T14:05:08.283244+0000","last_undegraded":"2024-04-25T14:05:08.283244+0000","last_fullsized":"2024-04-25T14:05:08.283244+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:07.846107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283638+0000","last_change":"2024-04-25T14:05:07.273047+0000","last_active":"2024-04-25T14:05:08.283638+0000","last_peered":"2024-04-25T14:05:08.283638+0000","last_clean":"2024-04-25T14:05:08.283638+0000","last_became_active":"2024-04-25T14:05:07.272760+0000","last_became_peered":"2024-04-25T14:05:07.272760+0000","last_unstale":"2024-04-25T14:05:08.283638+0000","last_undegraded":"2024-04-25T14:05:08.283638+0000","last_fullsized":"2024-04-25T14:05:08.283638+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:41.508799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283628+0000","last_change":"2024-04-25T14:05:06.261078+0000","last_active":"2024-04-25T14:05:08.283628+0000","last_peered":"2024-04-25T14:05:08.283628+0000","last_clean":"2024-04-25T14:05:08.283628+0000","last_became_active":"2024-04-25T14:05:06.260706+0000","last_became_peered":"2024-04-25T14:05:06.260706+0000","last_unstale":"2024-04-25T14:05:08.283628+0000","last_undegraded":"2024-04-25T14:05:08.283628+0000","last_fullsized":"2024-04-25T14:05:08.283628+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:55:54.218054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.284336+0000","last_change":"2024-04-25T14:05:07.289198+0000","last_active":"2024-04-25T14:05:08.284336+0000","last_peered":"2024-04-25T14:05:08.284336+0000","last_clean":"2024-04-25T14:05:08.284336+0000","last_became_active":"2024-04-25T14:05:07.288732+0000","last_became_peered":"2024-04-25T14:05:07.288732+0000","last_unstale":"2024-04-25T14:05:08.284336+0000","last_undegraded":"2024-04-25T14:05:08.284336+0000","last_fullsized":"2024-04-25T14:05:08.284336+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:20.873445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252350+0000","last_change":"2024-04-25T14:05:06.247598+0000","last_active":"2024-04-25T14:05:08.252350+0000","last_peered":"2024-04-25T14:05:08.252350+0000","last_clean":"2024-04-25T14:05:08.252350+0000","last_became_active":"2024-04-25T14:05:06.246997+0000","last_became_peered":"2024-04-25T14:05:06.246997+0000","last_unstale":"2024-04-25T14:05:08.252350+0000","last_undegraded":"2024-04-25T14:05:08.252350+0000","last_fullsized":"2024-04-25T14:05:08.252350+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:07.846107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283051+0000","last_change":"2024-04-25T14:05:07.259497+0000","last_active":"2024-04-25T14:05:08.283051+0000","last_peered":"2024-04-25T14:05:08.283051+0000","last_clean":"2024-04-25T14:05:08.283051+0000","last_became_active":"2024-04-25T14:05:07.259222+0000","last_became_peered":"2024-04-25T14:05:07.259222+0000","last_unstale":"2024-04-25T14:05:08.283051+0000","last_undegraded":"2024-04-25T14:05:08.283051+0000","last_fullsized":"2024-04-25T14:05:08.283051+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:20.873445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252369+0000","last_change":"2024-04-25T14:05:06.255952+0000","last_active":"2024-04-25T14:05:08.252369+0000","last_peered":"2024-04-25T14:05:08.252369+0000","last_clean":"2024-04-25T14:05:08.252369+0000","last_became_active":"2024-04-25T14:05:06.255606+0000","last_became_peered":"2024-04-25T14:05:06.255606+0000","last_unstale":"2024-04-25T14:05:08.252369+0000","last_undegraded":"2024-04-25T14:05:08.252369+0000","last_fullsized":"2024-04-25T14:05:08.252369+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:54.628494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252252+0000","last_change":"2024-04-25T14:05:07.263037+0000","last_active":"2024-04-25T14:05:08.252252+0000","last_peered":"2024-04-25T14:05:08.252252+0000","last_clean":"2024-04-25T14:05:08.252252+0000","last_became_active":"2024-04-25T14:05:07.261865+0000","last_became_peered":"2024-04-25T14:05:07.261865+0000","last_unstale":"2024-04-25T14:05:08.252252+0000","last_undegraded":"2024-04-25T14:05:08.252252+0000","last_fullsized":"2024-04-25T14:05:08.252252+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:54.320451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283667+0000","last_change":"2024-04-25T14:05:06.261678+0000","last_active":"2024-04-25T14:05:08.283667+0000","last_peered":"2024-04-25T14:05:08.283667+0000","last_clean":"2024-04-25T14:05:08.283667+0000","last_became_active":"2024-04-25T14:05:06.261336+0000","last_became_peered":"2024-04-25T14:05:06.261336+0000","last_unstale":"2024-04-25T14:05:08.283667+0000","last_undegraded":"2024-04-25T14:05:08.283667+0000","last_fullsized":"2024-04-25T14:05:08.283667+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:17.077511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252270+0000","last_change":"2024-04-25T14:05:07.277302+0000","last_active":"2024-04-25T14:05:08.252270+0000","last_peered":"2024-04-25T14:05:08.252270+0000","last_clean":"2024-04-25T14:05:08.252270+0000","last_became_active":"2024-04-25T14:05:07.276991+0000","last_became_peered":"2024-04-25T14:05:07.276991+0000","last_unstale":"2024-04-25T14:05:08.252270+0000","last_undegraded":"2024-04-25T14:05:08.252270+0000","last_fullsized":"2024-04-25T14:05:08.252270+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:20.873445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.251448+0000","last_change":"2024-04-25T14:05:06.264082+0000","last_active":"2024-04-25T14:05:08.251448+0000","last_peered":"2024-04-25T14:05:08.251448+0000","last_clean":"2024-04-25T14:05:08.251448+0000","last_became_active":"2024-04-25T14:05:06.263590+0000","last_became_peered":"2024-04-25T14:05:06.263590+0000","last_unstale":"2024-04-25T14:05:08.251448+0000","last_undegraded":"2024-04-25T14:05:08.251448+0000","last_fullsized":"2024-04-25T14:05:08.251448+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:18.883712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252251+0000","last_change":"2024-04-25T14:05:07.255646+0000","last_active":"2024-04-25T14:05:08.252251+0000","last_peered":"2024-04-25T14:05:08.252251+0000","last_clean":"2024-04-25T14:05:08.252251+0000","last_became_active":"2024-04-25T14:05:07.255170+0000","last_became_peered":"2024-04-25T14:05:07.255170+0000","last_unstale":"2024-04-25T14:05:08.252251+0000","last_undegraded":"2024-04-25T14:05:08.252251+0000","last_fullsized":"2024-04-25T14:05:08.252251+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:56.697086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252341+0000","last_change":"2024-04-25T14:05:06.256481+0000","last_active":"2024-04-25T14:05:08.252341+0000","last_peered":"2024-04-25T14:05:08.252341+0000","last_clean":"2024-04-25T14:05:08.252341+0000","last_became_active":"2024-04-25T14:05:06.256100+0000","last_became_peered":"2024-04-25T14:05:06.256100+0000","last_unstale":"2024-04-25T14:05:08.252341+0000","last_undegraded":"2024-04-25T14:05:08.252341+0000","last_fullsized":"2024-04-25T14:05:08.252341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:23.283424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252235+0000","last_change":"2024-04-25T14:05:07.257837+0000","last_active":"2024-04-25T14:05:08.252235+0000","last_peered":"2024-04-25T14:05:08.252235+0000","last_clean":"2024-04-25T14:05:08.252235+0000","last_became_active":"2024-04-25T14:05:07.257527+0000","last_became_peered":"2024-04-25T14:05:07.257527+0000","last_unstale":"2024-04-25T14:05:08.252235+0000","last_undegraded":"2024-04-25T14:05:08.252235+0000","last_fullsized":"2024-04-25T14:05:08.252235+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:03.144757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283323+0000","last_change":"2024-04-25T14:05:06.264865+0000","last_active":"2024-04-25T14:05:08.283323+0000","last_peered":"2024-04-25T14:05:08.283323+0000","last_clean":"2024-04-25T14:05:08.283323+0000","last_became_active":"2024-04-25T14:05:06.264435+0000","last_became_peered":"2024-04-25T14:05:06.264435+0000","last_unstale":"2024-04-25T14:05:08.283323+0000","last_undegraded":"2024-04-25T14:05:08.283323+0000","last_fullsized":"2024-04-25T14:05:08.283323+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:19.038477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283509+0000","last_change":"2024-04-25T14:05:07.284240+0000","last_active":"2024-04-25T14:05:08.283509+0000","last_peered":"2024-04-25T14:05:08.283509+0000","last_clean":"2024-04-25T14:05:08.283509+0000","last_became_active":"2024-04-25T14:05:07.283884+0000","last_became_peered":"2024-04-25T14:05:07.283884+0000","last_unstale":"2024-04-25T14:05:08.283509+0000","last_undegraded":"2024-04-25T14:05:08.283509+0000","last_fullsized":"2024-04-25T14:05:08.283509+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:06.121071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.284251+0000","last_change":"2024-04-25T14:05:06.265405+0000","last_active":"2024-04-25T14:05:08.284251+0000","last_peered":"2024-04-25T14:05:08.284251+0000","last_clean":"2024-04-25T14:05:08.284251+0000","last_became_active":"2024-04-25T14:05:06.265091+0000","last_became_peered":"2024-04-25T14:05:06.265091+0000","last_unstale":"2024-04-25T14:05:08.284251+0000","last_undegraded":"2024-04-25T14:05:08.284251+0000","last_fullsized":"2024-04-25T14:05:08.284251+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:17.077511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.284167+0000","last_change":"2024-04-25T14:05:07.276399+0000","last_active":"2024-04-25T14:05:08.284167+0000","last_peered":"2024-04-25T14:05:08.284167+0000","last_clean":"2024-04-25T14:05:08.284167+0000","last_became_active":"2024-04-25T14:05:07.275998+0000","last_became_peered":"2024-04-25T14:05:07.275998+0000","last_unstale":"2024-04-25T14:05:08.284167+0000","last_undegraded":"2024-04-25T14:05:08.284167+0000","last_fullsized":"2024-04-25T14:05:08.284167+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:20.169051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.279508+0000","last_change":"2024-04-25T14:05:06.256923+0000","last_active":"2024-04-25T14:05:08.279508+0000","last_peered":"2024-04-25T14:05:08.279508+0000","last_clean":"2024-04-25T14:05:08.279508+0000","last_became_active":"2024-04-25T14:05:06.256596+0000","last_became_peered":"2024-04-25T14:05:06.256596+0000","last_unstale":"2024-04-25T14:05:08.279508+0000","last_undegraded":"2024-04-25T14:05:08.279508+0000","last_fullsized":"2024-04-25T14:05:08.279508+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:13.441560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.279400+0000","last_change":"2024-04-25T14:05:07.271107+0000","last_active":"2024-04-25T14:05:08.279400+0000","last_peered":"2024-04-25T14:05:08.279400+0000","last_clean":"2024-04-25T14:05:08.279400+0000","last_became_active":"2024-04-25T14:05:07.270646+0000","last_became_peered":"2024-04-25T14:05:07.270646+0000","last_unstale":"2024-04-25T14:05:08.279400+0000","last_undegraded":"2024-04-25T14:05:08.279400+0000","last_fullsized":"2024-04-25T14:05:08.279400+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:30.422197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283772+0000","last_change":"2024-04-25T14:05:06.258644+0000","last_active":"2024-04-25T14:05:08.283772+0000","last_peered":"2024-04-25T14:05:08.283772+0000","last_clean":"2024-04-25T14:05:08.283772+0000","last_became_active":"2024-04-25T14:05:06.258332+0000","last_became_peered":"2024-04-25T14:05:06.258332+0000","last_unstale":"2024-04-25T14:05:08.283772+0000","last_undegraded":"2024-04-25T14:05:08.283772+0000","last_fullsized":"2024-04-25T14:05:08.283772+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:23.283424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252607+0000","last_change":"2024-04-25T14:05:07.276322+0000","last_active":"2024-04-25T14:05:08.252607+0000","last_peered":"2024-04-25T14:05:08.252607+0000","last_clean":"2024-04-25T14:05:08.252607+0000","last_became_active":"2024-04-25T14:05:07.275949+0000","last_became_peered":"2024-04-25T14:05:07.275949+0000","last_unstale":"2024-04-25T14:05:08.252607+0000","last_undegraded":"2024-04-25T14:05:08.252607+0000","last_fullsized":"2024-04-25T14:05:08.252607+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:09.919025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.279480+0000","last_change":"2024-04-25T14:05:06.250207+0000","last_active":"2024-04-25T14:05:08.279480+0000","last_peered":"2024-04-25T14:05:08.279480+0000","last_clean":"2024-04-25T14:05:08.279480+0000","last_became_active":"2024-04-25T14:05:06.249868+0000","last_became_peered":"2024-04-25T14:05:06.249868+0000","last_unstale":"2024-04-25T14:05:08.279480+0000","last_undegraded":"2024-04-25T14:05:08.279480+0000","last_fullsized":"2024-04-25T14:05:08.279480+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:53.239152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283144+0000","last_change":"2024-04-25T14:05:07.267521+0000","last_active":"2024-04-25T14:05:08.283144+0000","last_peered":"2024-04-25T14:05:08.283144+0000","last_clean":"2024-04-25T14:05:08.283144+0000","last_became_active":"2024-04-25T14:05:07.267168+0000","last_became_peered":"2024-04-25T14:05:07.267168+0000","last_unstale":"2024-04-25T14:05:08.283144+0000","last_undegraded":"2024-04-25T14:05:08.283144+0000","last_fullsized":"2024-04-25T14:05:08.283144+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:36.823492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.279347+0000","last_change":"2024-04-25T14:05:06.259360+0000","last_active":"2024-04-25T14:05:08.279347+0000","last_peered":"2024-04-25T14:05:08.279347+0000","last_clean":"2024-04-25T14:05:08.279347+0000","last_became_active":"2024-04-25T14:05:06.259018+0000","last_became_peered":"2024-04-25T14:05:06.259018+0000","last_unstale":"2024-04-25T14:05:08.279347+0000","last_undegraded":"2024-04-25T14:05:08.279347+0000","last_fullsized":"2024-04-25T14:05:08.279347+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:40.623886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.282871+0000","last_change":"2024-04-25T14:05:07.264442+0000","last_active":"2024-04-25T14:05:08.282871+0000","last_peered":"2024-04-25T14:05:08.282871+0000","last_clean":"2024-04-25T14:05:08.282871+0000","last_became_active":"2024-04-25T14:05:07.264104+0000","last_became_peered":"2024-04-25T14:05:07.264104+0000","last_unstale":"2024-04-25T14:05:08.282871+0000","last_undegraded":"2024-04-25T14:05:08.282871+0000","last_fullsized":"2024-04-25T14:05:08.282871+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:03.144757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252447+0000","last_change":"2024-04-25T14:05:06.262040+0000","last_active":"2024-04-25T14:05:08.252447+0000","last_peered":"2024-04-25T14:05:08.252447+0000","last_clean":"2024-04-25T14:05:08.252447+0000","last_became_active":"2024-04-25T14:05:06.261582+0000","last_became_peered":"2024-04-25T14:05:06.261582+0000","last_unstale":"2024-04-25T14:05:08.252447+0000","last_undegraded":"2024-04-25T14:05:08.252447+0000","last_fullsized":"2024-04-25T14:05:08.252447+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:08.266732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252375+0000","last_change":"2024-04-25T14:05:07.265063+0000","last_active":"2024-04-25T14:05:08.252375+0000","last_peered":"2024-04-25T14:05:08.252375+0000","last_clean":"2024-04-25T14:05:08.252375+0000","last_became_active":"2024-04-25T14:05:07.264733+0000","last_became_peered":"2024-04-25T14:05:07.264733+0000","last_unstale":"2024-04-25T14:05:08.252375+0000","last_undegraded":"2024-04-25T14:05:08.252375+0000","last_fullsized":"2024-04-25T14:05:08.252375+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:51.159581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.279222+0000","last_change":"2024-04-25T14:05:06.256004+0000","last_active":"2024-04-25T14:05:08.279222+0000","last_peered":"2024-04-25T14:05:08.279222+0000","last_clean":"2024-04-25T14:05:08.279222+0000","last_became_active":"2024-04-25T14:05:06.255694+0000","last_became_peered":"2024-04-25T14:05:06.255694+0000","last_unstale":"2024-04-25T14:05:08.279222+0000","last_undegraded":"2024-04-25T14:05:08.279222+0000","last_fullsized":"2024-04-25T14:05:08.279222+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:26.860266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283602+0000","last_change":"2024-04-25T14:05:07.284497+0000","last_active":"2024-04-25T14:05:08.283602+0000","last_peered":"2024-04-25T14:05:08.283602+0000","last_clean":"2024-04-25T14:05:08.283602+0000","last_became_active":"2024-04-25T14:05:07.284279+0000","last_became_peered":"2024-04-25T14:05:07.284279+0000","last_unstale":"2024-04-25T14:05:08.283602+0000","last_undegraded":"2024-04-25T14:05:08.283602+0000","last_fullsized":"2024-04-25T14:05:08.283602+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:05:13.234083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283433+0000","last_change":"2024-04-25T14:05:06.259664+0000","last_active":"2024-04-25T14:05:08.283433+0000","last_peered":"2024-04-25T14:05:08.283433+0000","last_clean":"2024-04-25T14:05:08.283433+0000","last_became_active":"2024-04-25T14:05:06.259319+0000","last_became_peered":"2024-04-25T14:05:06.259319+0000","last_unstale":"2024-04-25T14:05:08.283433+0000","last_undegraded":"2024-04-25T14:05:08.283433+0000","last_fullsized":"2024-04-25T14:05:08.283433+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:20.530061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252723+0000","last_change":"2024-04-25T14:05:07.275152+0000","last_active":"2024-04-25T14:05:08.252723+0000","last_peered":"2024-04-25T14:05:08.252723+0000","last_clean":"2024-04-25T14:05:08.252723+0000","last_became_active":"2024-04-25T14:05:07.274827+0000","last_became_peered":"2024-04-25T14:05:07.274827+0000","last_unstale":"2024-04-25T14:05:08.252723+0000","last_undegraded":"2024-04-25T14:05:08.252723+0000","last_fullsized":"2024-04-25T14:05:08.252723+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:43:48.479853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.281871+0000","last_change":"2024-04-25T14:05:06.260706+0000","last_active":"2024-04-25T14:05:08.281871+0000","last_peered":"2024-04-25T14:05:08.281871+0000","last_clean":"2024-04-25T14:05:08.281871+0000","last_became_active":"2024-04-25T14:05:06.260410+0000","last_became_peered":"2024-04-25T14:05:06.260410+0000","last_unstale":"2024-04-25T14:05:08.281871+0000","last_undegraded":"2024-04-25T14:05:08.281871+0000","last_fullsized":"2024-04-25T14:05:08.281871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:24.896382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252757+0000","last_change":"2024-04-25T14:05:07.272991+0000","last_active":"2024-04-25T14:05:08.252757+0000","last_peered":"2024-04-25T14:05:08.252757+0000","last_clean":"2024-04-25T14:05:08.252757+0000","last_became_active":"2024-04-25T14:05:07.272691+0000","last_became_peered":"2024-04-25T14:05:07.272691+0000","last_unstale":"2024-04-25T14:05:08.252757+0000","last_undegraded":"2024-04-25T14:05:08.252757+0000","last_fullsized":"2024-04-25T14:05:08.252757+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:06.121071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.284161+0000","last_change":"2024-04-25T14:05:06.253126+0000","last_active":"2024-04-25T14:05:08.284161+0000","last_peered":"2024-04-25T14:05:08.284161+0000","last_clean":"2024-04-25T14:05:08.284161+0000","last_became_active":"2024-04-25T14:05:06.252761+0000","last_became_peered":"2024-04-25T14:05:06.252761+0000","last_unstale":"2024-04-25T14:05:08.284161+0000","last_undegraded":"2024-04-25T14:05:08.284161+0000","last_fullsized":"2024-04-25T14:05:08.284161+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:29.416289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283141+0000","last_change":"2024-04-25T14:05:07.271339+0000","last_active":"2024-04-25T14:05:08.283141+0000","last_peered":"2024-04-25T14:05:08.283141+0000","last_clean":"2024-04-25T14:05:08.283141+0000","last_became_active":"2024-04-25T14:05:07.271051+0000","last_became_peered":"2024-04-25T14:05:07.271051+0000","last_unstale":"2024-04-25T14:05:08.283141+0000","last_undegraded":"2024-04-25T14:05:08.283141+0000","last_fullsized":"2024-04-25T14:05:08.283141+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:39.756595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283493+0000","last_change":"2024-04-25T14:05:06.259878+0000","last_active":"2024-04-25T14:05:08.283493+0000","last_peered":"2024-04-25T14:05:08.283493+0000","last_clean":"2024-04-25T14:05:08.283493+0000","last_became_active":"2024-04-25T14:05:06.259534+0000","last_became_peered":"2024-04-25T14:05:06.259534+0000","last_unstale":"2024-04-25T14:05:08.283493+0000","last_undegraded":"2024-04-25T14:05:08.283493+0000","last_fullsized":"2024-04-25T14:05:08.283493+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:19.038477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283174+0000","last_change":"2024-04-25T14:05:07.272470+0000","last_active":"2024-04-25T14:05:08.283174+0000","last_peered":"2024-04-25T14:05:08.283174+0000","last_clean":"2024-04-25T14:05:08.283174+0000","last_became_active":"2024-04-25T14:05:07.272133+0000","last_became_peered":"2024-04-25T14:05:07.272133+0000","last_unstale":"2024-04-25T14:05:08.283174+0000","last_undegraded":"2024-04-25T14:05:08.283174+0000","last_fullsized":"2024-04-25T14:05:08.283174+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:01:19.113121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.251789+0000","last_change":"2024-04-25T14:05:06.255046+0000","last_active":"2024-04-25T14:05:08.251789+0000","last_peered":"2024-04-25T14:05:08.251789+0000","last_clean":"2024-04-25T14:05:08.251789+0000","last_became_active":"2024-04-25T14:05:06.254726+0000","last_became_peered":"2024-04-25T14:05:06.254726+0000","last_unstale":"2024-04-25T14:05:08.251789+0000","last_undegraded":"2024-04-25T14:05:08.251789+0000","last_fullsized":"2024-04-25T14:05:08.251789+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:24.896382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283861+0000","last_change":"2024-04-25T14:05:07.290722+0000","last_active":"2024-04-25T14:05:08.283861+0000","last_peered":"2024-04-25T14:05:08.283861+0000","last_clean":"2024-04-25T14:05:08.283861+0000","last_became_active":"2024-04-25T14:05:07.287769+0000","last_became_peered":"2024-04-25T14:05:07.287769+0000","last_unstale":"2024-04-25T14:05:08.283861+0000","last_undegraded":"2024-04-25T14:05:08.283861+0000","last_fullsized":"2024-04-25T14:05:08.283861+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:43.913161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283608+0000","last_change":"2024-04-25T14:05:06.256512+0000","last_active":"2024-04-25T14:05:08.283608+0000","last_peered":"2024-04-25T14:05:08.283608+0000","last_clean":"2024-04-25T14:05:08.283608+0000","last_became_active":"2024-04-25T14:05:06.255995+0000","last_became_peered":"2024-04-25T14:05:06.255995+0000","last_unstale":"2024-04-25T14:05:08.283608+0000","last_undegraded":"2024-04-25T14:05:08.283608+0000","last_fullsized":"2024-04-25T14:05:08.283608+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:39.951627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252896+0000","last_change":"2024-04-25T14:05:07.267897+0000","last_active":"2024-04-25T14:05:08.252896+0000","last_peered":"2024-04-25T14:05:08.252896+0000","last_clean":"2024-04-25T14:05:08.252896+0000","last_became_active":"2024-04-25T14:05:07.267609+0000","last_became_peered":"2024-04-25T14:05:07.267609+0000","last_unstale":"2024-04-25T14:05:08.252896+0000","last_undegraded":"2024-04-25T14:05:08.252896+0000","last_fullsized":"2024-04-25T14:05:08.252896+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:54.320451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252100+0000","last_change":"2024-04-25T14:05:06.252301+0000","last_active":"2024-04-25T14:05:08.252100+0000","last_peered":"2024-04-25T14:05:08.252100+0000","last_clean":"2024-04-25T14:05:08.252100+0000","last_became_active":"2024-04-25T14:05:06.251997+0000","last_became_peered":"2024-04-25T14:05:06.251997+0000","last_unstale":"2024-04-25T14:05:08.252100+0000","last_undegraded":"2024-04-25T14:05:08.252100+0000","last_fullsized":"2024-04-25T14:05:08.252100+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:20.530061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283217+0000","last_change":"2024-04-25T14:05:07.280588+0000","last_active":"2024-04-25T14:05:08.283217+0000","last_peered":"2024-04-25T14:05:08.283217+0000","last_clean":"2024-04-25T14:05:08.283217+0000","last_became_active":"2024-04-25T14:05:07.280269+0000","last_became_peered":"2024-04-25T14:05:07.280269+0000","last_unstale":"2024-04-25T14:05:08.283217+0000","last_undegraded":"2024-04-25T14:05:08.283217+0000","last_fullsized":"2024-04-25T14:05:08.283217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:13.992505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252805+0000","last_change":"2024-04-25T14:05:06.257515+0000","last_active":"2024-04-25T14:05:08.252805+0000","last_peered":"2024-04-25T14:05:08.252805+0000","last_clean":"2024-04-25T14:05:08.252805+0000","last_became_active":"2024-04-25T14:05:06.257198+0000","last_became_peered":"2024-04-25T14:05:06.257198+0000","last_unstale":"2024-04-25T14:05:08.252805+0000","last_undegraded":"2024-04-25T14:05:08.252805+0000","last_fullsized":"2024-04-25T14:05:08.252805+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:29.416289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":142,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252552+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:05:08.252552+0000","last_peered":"2024-04-25T14:05:08.252552+0000","last_clean":"2024-04-25T14:05:08.252552+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:05:08.252552+0000","last_undegraded":"2024-04-25T14:05:08.252552+0000","last_fullsized":"2024-04-25T14:05:08.252552+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1048576,"data_stored":782078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27444,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":430080,"data_stored":183918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27444,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":434176,"data_stored":183448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":190632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":184982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":183406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":183426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1036288,"data_stored":780430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:05:11.389 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:05:11.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.576+0000 7f7967600640 1 Processor -- start 2024-04-25T14:05:11.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.576+0000 7f7967600640 1 -- start start 2024-04-25T14:05:11.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.580+0000 7f7967600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7960071530 0x7f796006e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.580+0000 7f7967600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f796006e7a0 0x7f79600cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.580+0000 7f7967600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f796006ef90 con 0x7f79600cf970 2024-04-25T14:05:11.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.580+0000 7f7967600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7960163830 con 0x7f7960071530 2024-04-25T14:05:11.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.580+0000 7f7967600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f7960163a80 con 0x7f796006e7a0 2024-04-25T14:05:11.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.580+0000 7f7957e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7960071530 0x7f796006e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.580+0000 7f7957e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7960071530 0x7f796006e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57544/0 (socket says 172.21.15.86:57544) 2024-04-25T14:05:11.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.580+0000 7f7957e00640 1 -- 172.21.15.86:0/1907194833 learned_addr learned my addr 172.21.15.86:0/1907194833 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:11.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.580+0000 7f7965600640 1 --2- 172.21.15.86:0/1907194833 >> v2:172.21.15.155:3300/0 conn(0x7f796006e7a0 0x7f79600cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.580+0000 7f7957e00640 1 -- 172.21.15.86:0/1907194833 >> v2:172.21.15.155:3300/0 conn(0x7f796006e7a0 msgr2=0x7f79600cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:11.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.580+0000 7f7957e00640 1 --2- 172.21.15.86:0/1907194833 >> v2:172.21.15.155:3300/0 conn(0x7f796006e7a0 0x7f79600cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.580+0000 7f7957e00640 1 -- 172.21.15.86:0/1907194833 >> v1:172.21.15.155:6789/0 conn(0x7f79600cf970 legacy=0x7f7960070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.580+0000 7f7957e00640 1 -- 172.21.15.86:0/1907194833 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7960163d60 con 0x7f7960071530 2024-04-25T14:05:11.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.580+0000 7f7965600640 1 --2- 172.21.15.86:0/1907194833 >> v2:172.21.15.155:3300/0 conn(0x7f796006e7a0 0x7f79600cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:11.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.584+0000 7f7957e00640 1 --2- 172.21.15.86:0/1907194833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7960071530 0x7f796006e260 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f79601639b0 tx=0x7f794801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae41c7b588385ae6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:11.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.584+0000 7f795fe00640 1 -- 172.21.15.86:0/1907194833 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79480aa650 con 0x7f7960071530 2024-04-25T14:05:11.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.584+0000 7f795fe00640 1 -- 172.21.15.86:0/1907194833 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79480af070 con 0x7f7960071530 2024-04-25T14:05:11.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.584+0000 7f795fe00640 1 -- 172.21.15.86:0/1907194833 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79480aaa10 con 0x7f7960071530 2024-04-25T14:05:11.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.584+0000 7f7967600640 1 -- 172.21.15.86:0/1907194833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7960071530 msgr2=0x7f796006e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.584+0000 7f7967600640 1 --2- 172.21.15.86:0/1907194833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7960071530 0x7f796006e260 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f79601639b0 tx=0x7f794801c4d0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.584+0000 7f7967600640 1 -- 172.21.15.86:0/1907194833 shutdown_connections 2024-04-25T14:05:11.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.584+0000 7f7967600640 1 --2- 172.21.15.86:0/1907194833 >> v2:172.21.15.155:3300/0 conn(0x7f796006e7a0 0x7f79600cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.584+0000 7f7967600640 1 --2- 172.21.15.86:0/1907194833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7960071530 0x7f796006e260 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.584+0000 7f7967600640 1 -- 172.21.15.86:0/1907194833 >> 172.21.15.86:0/1907194833 conn(0x7f79600cffc0 msgr2=0x7f79600d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:11.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.584+0000 7f7967600640 1 -- 172.21.15.86:0/1907194833 shutdown_connections 2024-04-25T14:05:11.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.584+0000 7f7967600640 1 -- 172.21.15.86:0/1907194833 wait complete. 2024-04-25T14:05:11.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.588+0000 7f7967600640 1 Processor -- start 2024-04-25T14:05:11.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.588+0000 7f7967600640 1 -- start start 2024-04-25T14:05:11.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.588+0000 7f7967600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f796006e7a0 0x7f796015cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.588+0000 7f7967600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79600cf970 0x7f79600c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.588+0000 7f7967600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79600b6160 0x7f79600c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.588+0000 7f7967600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f796016cbd0 con 0x7f79600cf970 2024-04-25T14:05:11.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.588+0000 7f7967600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f79600ced80 con 0x7f796006e7a0 2024-04-25T14:05:11.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.588+0000 7f7967600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f796016cd50 con 0x7f79600b6160 2024-04-25T14:05:11.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.588+0000 7f7957e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79600cf970 0x7f79600c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.588+0000 7f7957e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79600cf970 0x7f79600c89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57550/0 (socket says 172.21.15.86:57550) 2024-04-25T14:05:11.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.588+0000 7f7957e00640 1 -- 172.21.15.86:0/2268818745 learned_addr learned my addr 172.21.15.86:0/2268818745 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:11.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.588+0000 7f7964c00640 1 --2- 172.21.15.86:0/2268818745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f796006e7a0 0x7f796015cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.588+0000 7f7965600640 1 --2- 172.21.15.86:0/2268818745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79600b6160 0x7f79600c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.588+0000 7f7957e00640 1 -- 172.21.15.86:0/2268818745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79600b6160 msgr2=0x7f79600c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.588+0000 7f7957e00640 1 --2- 172.21.15.86:0/2268818745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79600b6160 0x7f79600c8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.588+0000 7f7957e00640 1 -- 172.21.15.86:0/2268818745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f796006e7a0 msgr2=0x7f796015cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.588+0000 7f7957e00640 1 --2- 172.21.15.86:0/2268818745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f796006e7a0 0x7f796015cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.588+0000 7f7957e00640 1 -- 172.21.15.86:0/2268818745 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79600b8a30 con 0x7f79600cf970 2024-04-25T14:05:11.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.588+0000 7f7957e00640 1 --2- 172.21.15.86:0/2268818745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79600cf970 0x7f79600c89e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f794801a100 tx=0x7f7948019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:11.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.588+0000 7f795d600640 1 -- 172.21.15.86:0/2268818745 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7948003c70 con 0x7f79600cf970 2024-04-25T14:05:11.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.588+0000 7f795d600640 1 -- 172.21.15.86:0/2268818745 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79480bc070 con 0x7f79600cf970 2024-04-25T14:05:11.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.588+0000 7f795d600640 1 -- 172.21.15.86:0/2268818745 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79480aa670 con 0x7f79600cf970 2024-04-25T14:05:11.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.588+0000 7f7967600640 1 -- 172.21.15.86:0/2268818745 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79600b8d10 con 0x7f79600cf970 2024-04-25T14:05:11.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.588+0000 7f7967600640 1 -- 172.21.15.86:0/2268818745 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f796015f360 con 0x7f79600cf970 2024-04-25T14:05:11.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.592+0000 7f7967600640 1 -- 172.21.15.86:0/2268818745 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79600720e0 con 0x7f79600cf970 2024-04-25T14:05:11.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.592+0000 7f795d600640 1 -- 172.21.15.86:0/2268818745 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f79480aa810 con 0x7f79600cf970 2024-04-25T14:05:11.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.592+0000 7f795d600640 1 --2- 172.21.15.86:0/2268818745 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f793c06f400 0x7f793c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.592+0000 7f795d600640 1 -- 172.21.15.86:0/2268818745 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f7948129920 con 0x7f79600cf970 2024-04-25T14:05:11.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.592+0000 7f7964c00640 1 --2- 172.21.15.86:0/2268818745 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f793c06f400 0x7f793c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.592+0000 7f7964c00640 1 --2- 172.21.15.86:0/2268818745 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f793c06f400 0x7f793c071910 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f79500015b0 tx=0x7f7950009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:11.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.596+0000 7f795d600640 1 -- 172.21.15.86:0/2268818745 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f79480160f0 con 0x7f79600cf970 2024-04-25T14:05:11.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.752+0000 7f7967600640 1 -- 172.21.15.86:0/2268818745 --> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7960069d60 con 0x7f793c06f400 2024-04-25T14:05:11.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.760+0000 7f795d600640 1 -- 172.21.15.86:0/2268818745 <== mgr.4103 v2:172.21.15.86:6832/2968869755 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251946 (secure 0 0 0) 0x7f79500050d0 con 0x7f793c06f400 2024-04-25T14:05:11.763 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:11.765 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:05:11.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.760+0000 7f7967600640 1 -- 172.21.15.86:0/2268818745 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f793c06f400 msgr2=0x7f793c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.760+0000 7f7967600640 1 --2- 172.21.15.86:0/2268818745 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f793c06f400 0x7f793c071910 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f79500015b0 tx=0x7f7950009040 comp rx=0 tx=0).stop 2024-04-25T14:05:11.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.760+0000 7f7967600640 1 -- 172.21.15.86:0/2268818745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79600cf970 msgr2=0x7f79600c89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.760+0000 7f7967600640 1 --2- 172.21.15.86:0/2268818745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79600cf970 0x7f79600c89e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f794801a100 tx=0x7f7948019270 comp rx=0 tx=0).stop 2024-04-25T14:05:11.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.760+0000 7f7967600640 1 -- 172.21.15.86:0/2268818745 shutdown_connections 2024-04-25T14:05:11.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.760+0000 7f7967600640 1 --2- 172.21.15.86:0/2268818745 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f793c06f400 0x7f793c071910 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.760+0000 7f7967600640 1 --2- 172.21.15.86:0/2268818745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79600b6160 0x7f79600c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.760+0000 7f7967600640 1 --2- 172.21.15.86:0/2268818745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79600cf970 0x7f79600c89e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.760+0000 7f7967600640 1 --2- 172.21.15.86:0/2268818745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f796006e7a0 0x7f796015cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.760+0000 7f7967600640 1 -- 172.21.15.86:0/2268818745 >> 172.21.15.86:0/2268818745 conn(0x7f79600cffc0 msgr2=0x7f7960162dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:11.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.764+0000 7f7967600640 1 -- 172.21.15.86:0/2268818745 shutdown_connections 2024-04-25T14:05:11.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.764+0000 7f7967600640 1 -- 172.21.15.86:0/2268818745 wait complete. 2024-04-25T14:05:11.784 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-04-25T14:05:10.729240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":138,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220768,"kb_used_data":4588,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753952,"statfs":{"total":773094113280,"available":772868046848,"internally_reserved":0,"allocated":4698112,"data_stored":2672320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.498378"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252558+0000","last_change":"2024-04-25T14:05:07.256777+0000","last_active":"2024-04-25T14:05:08.252558+0000","last_peered":"2024-04-25T14:05:08.252558+0000","last_clean":"2024-04-25T14:05:08.252558+0000","last_became_active":"2024-04-25T14:05:07.256450+0000","last_became_peered":"2024-04-25T14:05:07.256450+0000","last_unstale":"2024-04-25T14:05:08.252558+0000","last_undegraded":"2024-04-25T14:05:08.252558+0000","last_fullsized":"2024-04-25T14:05:08.252558+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:13.992505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252183+0000","last_change":"2024-04-25T14:05:07.274428+0000","last_active":"2024-04-25T14:05:08.252183+0000","last_peered":"2024-04-25T14:05:08.252183+0000","last_clean":"2024-04-25T14:05:08.252183+0000","last_became_active":"2024-04-25T14:05:07.274121+0000","last_became_peered":"2024-04-25T14:05:07.274121+0000","last_unstale":"2024-04-25T14:05:08.252183+0000","last_undegraded":"2024-04-25T14:05:08.252183+0000","last_fullsized":"2024-04-25T14:05:08.252183+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:43:48.479853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283661+0000","last_change":"2024-04-25T14:05:07.261520+0000","last_active":"2024-04-25T14:05:08.283661+0000","last_peered":"2024-04-25T14:05:08.283661+0000","last_clean":"2024-04-25T14:05:08.283661+0000","last_became_active":"2024-04-25T14:05:07.261158+0000","last_became_peered":"2024-04-25T14:05:07.261158+0000","last_unstale":"2024-04-25T14:05:08.283661+0000","last_undegraded":"2024-04-25T14:05:08.283661+0000","last_fullsized":"2024-04-25T14:05:08.283661+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:48:18.920873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.279423+0000","last_change":"2024-04-25T14:05:07.278871+0000","last_active":"2024-04-25T14:05:08.279423+0000","last_peered":"2024-04-25T14:05:08.279423+0000","last_clean":"2024-04-25T14:05:08.279423+0000","last_became_active":"2024-04-25T14:05:07.278018+0000","last_became_peered":"2024-04-25T14:05:07.278018+0000","last_unstale":"2024-04-25T14:05:08.279423+0000","last_undegraded":"2024-04-25T14:05:08.279423+0000","last_fullsized":"2024-04-25T14:05:08.279423+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:20.044385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.279352+0000","last_change":"2024-04-25T14:05:07.286426+0000","last_active":"2024-04-25T14:05:08.279352+0000","last_peered":"2024-04-25T14:05:08.279352+0000","last_clean":"2024-04-25T14:05:08.279352+0000","last_became_active":"2024-04-25T14:05:07.286114+0000","last_became_peered":"2024-04-25T14:05:07.286114+0000","last_unstale":"2024-04-25T14:05:08.279352+0000","last_undegraded":"2024-04-25T14:05:08.279352+0000","last_fullsized":"2024-04-25T14:05:08.279352+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:09.272639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.284244+0000","last_change":"2024-04-25T14:05:07.285651+0000","last_active":"2024-04-25T14:05:08.284244+0000","last_peered":"2024-04-25T14:05:08.284244+0000","last_clean":"2024-04-25T14:05:08.284244+0000","last_became_active":"2024-04-25T14:05:07.285337+0000","last_became_peered":"2024-04-25T14:05:07.285337+0000","last_unstale":"2024-04-25T14:05:08.284244+0000","last_undegraded":"2024-04-25T14:05:08.284244+0000","last_fullsized":"2024-04-25T14:05:08.284244+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:35.998555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.282722+0000","last_change":"2024-04-25T14:05:07.256962+0000","last_active":"2024-04-25T14:05:08.282722+0000","last_peered":"2024-04-25T14:05:08.282722+0000","last_clean":"2024-04-25T14:05:08.282722+0000","last_became_active":"2024-04-25T14:05:07.256595+0000","last_became_peered":"2024-04-25T14:05:07.256595+0000","last_unstale":"2024-04-25T14:05:08.282722+0000","last_undegraded":"2024-04-25T14:05:08.282722+0000","last_fullsized":"2024-04-25T14:05:08.282722+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:48:18.920873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.251979+0000","last_change":"2024-04-25T14:05:07.263138+0000","last_active":"2024-04-25T14:05:08.251979+0000","last_peered":"2024-04-25T14:05:08.251979+0000","last_clean":"2024-04-25T14:05:08.251979+0000","last_became_active":"2024-04-25T14:05:07.262819+0000","last_became_peered":"2024-04-25T14:05:07.262819+0000","last_unstale":"2024-04-25T14:05:08.251979+0000","last_undegraded":"2024-04-25T14:05:08.251979+0000","last_fullsized":"2024-04-25T14:05:08.251979+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:41.508799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.284059+0000","last_change":"2024-04-25T14:05:07.264088+0000","last_active":"2024-04-25T14:05:08.284059+0000","last_peered":"2024-04-25T14:05:08.284059+0000","last_clean":"2024-04-25T14:05:08.284059+0000","last_became_active":"2024-04-25T14:05:07.261204+0000","last_became_peered":"2024-04-25T14:05:07.261204+0000","last_unstale":"2024-04-25T14:05:08.284059+0000","last_undegraded":"2024-04-25T14:05:08.284059+0000","last_fullsized":"2024-04-25T14:05:08.284059+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:43:48.479853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283119+0000","last_change":"2024-04-25T14:05:07.266609+0000","last_active":"2024-04-25T14:05:08.283119+0000","last_peered":"2024-04-25T14:05:08.283119+0000","last_clean":"2024-04-25T14:05:08.283119+0000","last_became_active":"2024-04-25T14:05:07.266297+0000","last_became_peered":"2024-04-25T14:05:07.266297+0000","last_unstale":"2024-04-25T14:05:08.283119+0000","last_undegraded":"2024-04-25T14:05:08.283119+0000","last_fullsized":"2024-04-25T14:05:08.283119+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:56.697086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283958+0000","last_change":"2024-04-25T14:05:07.273257+0000","last_active":"2024-04-25T14:05:08.283958+0000","last_peered":"2024-04-25T14:05:08.283958+0000","last_clean":"2024-04-25T14:05:08.283958+0000","last_became_active":"2024-04-25T14:05:07.272896+0000","last_became_peered":"2024-04-25T14:05:07.272896+0000","last_unstale":"2024-04-25T14:05:08.283958+0000","last_undegraded":"2024-04-25T14:05:08.283958+0000","last_fullsized":"2024-04-25T14:05:08.283958+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:54.988190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283589+0000","last_change":"2024-04-25T14:05:07.271933+0000","last_active":"2024-04-25T14:05:08.283589+0000","last_peered":"2024-04-25T14:05:08.283589+0000","last_clean":"2024-04-25T14:05:08.283589+0000","last_became_active":"2024-04-25T14:05:07.271620+0000","last_became_peered":"2024-04-25T14:05:07.271620+0000","last_unstale":"2024-04-25T14:05:08.283589+0000","last_undegraded":"2024-04-25T14:05:08.283589+0000","last_fullsized":"2024-04-25T14:05:08.283589+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:09.919025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283820+0000","last_change":"2024-04-25T14:05:07.276415+0000","last_active":"2024-04-25T14:05:08.283820+0000","last_peered":"2024-04-25T14:05:08.283820+0000","last_clean":"2024-04-25T14:05:08.283820+0000","last_became_active":"2024-04-25T14:05:07.275613+0000","last_became_peered":"2024-04-25T14:05:07.275613+0000","last_unstale":"2024-04-25T14:05:08.283820+0000","last_undegraded":"2024-04-25T14:05:08.283820+0000","last_fullsized":"2024-04-25T14:05:08.283820+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:48.719845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283695+0000","last_change":"2024-04-25T14:05:07.269463+0000","last_active":"2024-04-25T14:05:08.283695+0000","last_peered":"2024-04-25T14:05:08.283695+0000","last_clean":"2024-04-25T14:05:08.283695+0000","last_became_active":"2024-04-25T14:05:07.269188+0000","last_became_peered":"2024-04-25T14:05:07.269188+0000","last_unstale":"2024-04-25T14:05:08.283695+0000","last_undegraded":"2024-04-25T14:05:08.283695+0000","last_fullsized":"2024-04-25T14:05:08.283695+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:13.217271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.282717+0000","last_change":"2024-04-25T14:05:07.273251+0000","last_active":"2024-04-25T14:05:08.282717+0000","last_peered":"2024-04-25T14:05:08.282717+0000","last_clean":"2024-04-25T14:05:08.282717+0000","last_became_active":"2024-04-25T14:05:07.272878+0000","last_became_peered":"2024-04-25T14:05:07.272878+0000","last_unstale":"2024-04-25T14:05:08.282717+0000","last_undegraded":"2024-04-25T14:05:08.282717+0000","last_fullsized":"2024-04-25T14:05:08.282717+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:43:48.479853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252191+0000","last_change":"2024-04-25T14:05:07.284939+0000","last_active":"2024-04-25T14:05:08.252191+0000","last_peered":"2024-04-25T14:05:08.252191+0000","last_clean":"2024-04-25T14:05:08.252191+0000","last_became_active":"2024-04-25T14:05:07.284619+0000","last_became_peered":"2024-04-25T14:05:07.284619+0000","last_unstale":"2024-04-25T14:05:08.252191+0000","last_undegraded":"2024-04-25T14:05:08.252191+0000","last_fullsized":"2024-04-25T14:05:08.252191+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:35.998555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252837+0000","last_change":"2024-04-25T14:05:07.291086+0000","last_active":"2024-04-25T14:05:08.252837+0000","last_peered":"2024-04-25T14:05:08.252837+0000","last_clean":"2024-04-25T14:05:08.252837+0000","last_became_active":"2024-04-25T14:05:07.290811+0000","last_became_peered":"2024-04-25T14:05:07.290811+0000","last_unstale":"2024-04-25T14:05:08.252837+0000","last_undegraded":"2024-04-25T14:05:08.252837+0000","last_fullsized":"2024-04-25T14:05:08.252837+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:03.144757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.251877+0000","last_change":"2024-04-25T14:05:07.275169+0000","last_active":"2024-04-25T14:05:08.251877+0000","last_peered":"2024-04-25T14:05:08.251877+0000","last_clean":"2024-04-25T14:05:08.251877+0000","last_became_active":"2024-04-25T14:05:07.274587+0000","last_became_peered":"2024-04-25T14:05:07.274587+0000","last_unstale":"2024-04-25T14:05:08.251877+0000","last_undegraded":"2024-04-25T14:05:08.251877+0000","last_fullsized":"2024-04-25T14:05:08.251877+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:55.634401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.279574+0000","last_change":"2024-04-25T14:05:07.280608+0000","last_active":"2024-04-25T14:05:08.279574+0000","last_peered":"2024-04-25T14:05:08.279574+0000","last_clean":"2024-04-25T14:05:08.279574+0000","last_became_active":"2024-04-25T14:05:07.280256+0000","last_became_peered":"2024-04-25T14:05:07.280256+0000","last_unstale":"2024-04-25T14:05:08.279574+0000","last_undegraded":"2024-04-25T14:05:08.279574+0000","last_fullsized":"2024-04-25T14:05:08.279574+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:21.535968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.251707+0000","last_change":"2024-04-25T14:05:07.274877+0000","last_active":"2024-04-25T14:05:08.251707+0000","last_peered":"2024-04-25T14:05:08.251707+0000","last_clean":"2024-04-25T14:05:08.251707+0000","last_became_active":"2024-04-25T14:05:07.274223+0000","last_became_peered":"2024-04-25T14:05:07.274223+0000","last_unstale":"2024-04-25T14:05:08.251707+0000","last_undegraded":"2024-04-25T14:05:08.251707+0000","last_fullsized":"2024-04-25T14:05:08.251707+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:21.535968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283121+0000","last_change":"2024-04-25T14:05:07.276724+0000","last_active":"2024-04-25T14:05:08.283121+0000","last_peered":"2024-04-25T14:05:08.283121+0000","last_clean":"2024-04-25T14:05:08.283121+0000","last_became_active":"2024-04-25T14:05:07.276036+0000","last_became_peered":"2024-04-25T14:05:07.276036+0000","last_unstale":"2024-04-25T14:05:08.283121+0000","last_undegraded":"2024-04-25T14:05:08.283121+0000","last_fullsized":"2024-04-25T14:05:08.283121+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:51.159581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283884+0000","last_change":"2024-04-25T14:05:07.274388+0000","last_active":"2024-04-25T14:05:08.283884+0000","last_peered":"2024-04-25T14:05:08.283884+0000","last_clean":"2024-04-25T14:05:08.283884+0000","last_became_active":"2024-04-25T14:05:07.273384+0000","last_became_peered":"2024-04-25T14:05:07.273384+0000","last_unstale":"2024-04-25T14:05:08.283884+0000","last_undegraded":"2024-04-25T14:05:08.283884+0000","last_fullsized":"2024-04-25T14:05:08.283884+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:01:19.113121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252284+0000","last_change":"2024-04-25T14:05:07.273281+0000","last_active":"2024-04-25T14:05:08.252284+0000","last_peered":"2024-04-25T14:05:08.252284+0000","last_clean":"2024-04-25T14:05:08.252284+0000","last_became_active":"2024-04-25T14:05:07.272962+0000","last_became_peered":"2024-04-25T14:05:07.272962+0000","last_unstale":"2024-04-25T14:05:08.252284+0000","last_undegraded":"2024-04-25T14:05:08.252284+0000","last_fullsized":"2024-04-25T14:05:08.252284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:39.756595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.279709+0000","last_change":"2024-04-25T14:05:07.268205+0000","last_active":"2024-04-25T14:05:08.279709+0000","last_peered":"2024-04-25T14:05:08.279709+0000","last_clean":"2024-04-25T14:05:08.279709+0000","last_became_active":"2024-04-25T14:05:07.267900+0000","last_became_peered":"2024-04-25T14:05:07.267900+0000","last_unstale":"2024-04-25T14:05:08.279709+0000","last_undegraded":"2024-04-25T14:05:08.279709+0000","last_fullsized":"2024-04-25T14:05:08.279709+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:55.634401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.251924+0000","last_change":"2024-04-25T14:05:07.258076+0000","last_active":"2024-04-25T14:05:08.251924+0000","last_peered":"2024-04-25T14:05:08.251924+0000","last_clean":"2024-04-25T14:05:08.251924+0000","last_became_active":"2024-04-25T14:05:07.257653+0000","last_became_peered":"2024-04-25T14:05:07.257653+0000","last_unstale":"2024-04-25T14:05:08.251924+0000","last_undegraded":"2024-04-25T14:05:08.251924+0000","last_fullsized":"2024-04-25T14:05:08.251924+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:55:55.223962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283926+0000","last_change":"2024-04-25T14:05:07.285796+0000","last_active":"2024-04-25T14:05:08.283926+0000","last_peered":"2024-04-25T14:05:08.283926+0000","last_clean":"2024-04-25T14:05:08.283926+0000","last_became_active":"2024-04-25T14:05:07.284802+0000","last_became_peered":"2024-04-25T14:05:07.284802+0000","last_unstale":"2024-04-25T14:05:08.283926+0000","last_undegraded":"2024-04-25T14:05:08.283926+0000","last_fullsized":"2024-04-25T14:05:08.283926+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:20.169051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.251941+0000","last_change":"2024-04-25T14:05:07.268639+0000","last_active":"2024-04-25T14:05:08.251941+0000","last_peered":"2024-04-25T14:05:08.251941+0000","last_clean":"2024-04-25T14:05:08.251941+0000","last_became_active":"2024-04-25T14:05:07.268256+0000","last_became_peered":"2024-04-25T14:05:07.268256+0000","last_unstale":"2024-04-25T14:05:08.251941+0000","last_undegraded":"2024-04-25T14:05:08.251941+0000","last_fullsized":"2024-04-25T14:05:08.251941+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:13.217271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283077+0000","last_change":"2024-04-25T14:05:07.283026+0000","last_active":"2024-04-25T14:05:08.283077+0000","last_peered":"2024-04-25T14:05:08.283077+0000","last_clean":"2024-04-25T14:05:08.283077+0000","last_became_active":"2024-04-25T14:05:07.282288+0000","last_became_peered":"2024-04-25T14:05:07.282288+0000","last_unstale":"2024-04-25T14:05:08.283077+0000","last_undegraded":"2024-04-25T14:05:08.283077+0000","last_fullsized":"2024-04-25T14:05:08.283077+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:54.320451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.251680+0000","last_change":"2024-04-25T14:05:07.275412+0000","last_active":"2024-04-25T14:05:08.251680+0000","last_peered":"2024-04-25T14:05:08.251680+0000","last_clean":"2024-04-25T14:05:08.251680+0000","last_became_active":"2024-04-25T14:05:07.275072+0000","last_became_peered":"2024-04-25T14:05:07.275072+0000","last_unstale":"2024-04-25T14:05:08.251680+0000","last_undegraded":"2024-04-25T14:05:08.251680+0000","last_fullsized":"2024-04-25T14:05:08.251680+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:20.044385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.251724+0000","last_change":"2024-04-25T14:05:07.258642+0000","last_active":"2024-04-25T14:05:08.251724+0000","last_peered":"2024-04-25T14:05:08.251724+0000","last_clean":"2024-04-25T14:05:08.251724+0000","last_became_active":"2024-04-25T14:05:07.258273+0000","last_became_peered":"2024-04-25T14:05:07.258273+0000","last_unstale":"2024-04-25T14:05:08.251724+0000","last_undegraded":"2024-04-25T14:05:08.251724+0000","last_fullsized":"2024-04-25T14:05:08.251724+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:08.852015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252300+0000","last_change":"2024-04-25T14:05:07.275998+0000","last_active":"2024-04-25T14:05:08.252300+0000","last_peered":"2024-04-25T14:05:08.252300+0000","last_clean":"2024-04-25T14:05:08.252300+0000","last_became_active":"2024-04-25T14:05:07.275663+0000","last_became_peered":"2024-04-25T14:05:07.275663+0000","last_unstale":"2024-04-25T14:05:08.252300+0000","last_undegraded":"2024-04-25T14:05:08.252300+0000","last_fullsized":"2024-04-25T14:05:08.252300+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:36.823492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.282977+0000","last_change":"2024-04-25T14:05:07.273743+0000","last_active":"2024-04-25T14:05:08.282977+0000","last_peered":"2024-04-25T14:05:08.282977+0000","last_clean":"2024-04-25T14:05:08.282977+0000","last_became_active":"2024-04-25T14:05:07.273435+0000","last_became_peered":"2024-04-25T14:05:07.273435+0000","last_unstale":"2024-04-25T14:05:08.282977+0000","last_undegraded":"2024-04-25T14:05:08.282977+0000","last_fullsized":"2024-04-25T14:05:08.282977+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:43.913161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252687+0000","last_change":"2024-04-25T14:05:07.286228+0000","last_active":"2024-04-25T14:05:08.252687+0000","last_peered":"2024-04-25T14:05:08.252687+0000","last_clean":"2024-04-25T14:05:08.252687+0000","last_became_active":"2024-04-25T14:05:07.285840+0000","last_became_peered":"2024-04-25T14:05:07.285840+0000","last_unstale":"2024-04-25T14:05:08.252687+0000","last_undegraded":"2024-04-25T14:05:08.252687+0000","last_fullsized":"2024-04-25T14:05:08.252687+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:48:18.920873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.251921+0000","last_change":"2024-04-25T14:05:06.263372+0000","last_active":"2024-04-25T14:05:08.251921+0000","last_peered":"2024-04-25T14:05:08.251921+0000","last_clean":"2024-04-25T14:05:08.251921+0000","last_became_active":"2024-04-25T14:05:06.262221+0000","last_became_peered":"2024-04-25T14:05:06.262221+0000","last_unstale":"2024-04-25T14:05:08.251921+0000","last_undegraded":"2024-04-25T14:05:08.251921+0000","last_fullsized":"2024-04-25T14:05:08.251921+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:20.185798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.284278+0000","last_change":"2024-04-25T14:05:07.286687+0000","last_active":"2024-04-25T14:05:08.284278+0000","last_peered":"2024-04-25T14:05:08.284278+0000","last_clean":"2024-04-25T14:05:08.284278+0000","last_became_active":"2024-04-25T14:05:07.286339+0000","last_became_peered":"2024-04-25T14:05:07.286339+0000","last_unstale":"2024-04-25T14:05:08.284278+0000","last_undegraded":"2024-04-25T14:05:08.284278+0000","last_fullsized":"2024-04-25T14:05:08.284278+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:20.873445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283628+0000","last_change":"2024-04-25T14:05:06.260377+0000","last_active":"2024-04-25T14:05:08.283628+0000","last_peered":"2024-04-25T14:05:08.283628+0000","last_clean":"2024-04-25T14:05:08.283628+0000","last_became_active":"2024-04-25T14:05:06.259976+0000","last_became_peered":"2024-04-25T14:05:06.259976+0000","last_unstale":"2024-04-25T14:05:08.283628+0000","last_undegraded":"2024-04-25T14:05:08.283628+0000","last_fullsized":"2024-04-25T14:05:08.283628+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:20.530061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.284307+0000","last_change":"2024-04-25T14:05:07.271748+0000","last_active":"2024-04-25T14:05:08.284307+0000","last_peered":"2024-04-25T14:05:08.284307+0000","last_clean":"2024-04-25T14:05:08.284307+0000","last_became_active":"2024-04-25T14:05:07.271244+0000","last_became_peered":"2024-04-25T14:05:07.271244+0000","last_unstale":"2024-04-25T14:05:08.284307+0000","last_undegraded":"2024-04-25T14:05:08.284307+0000","last_fullsized":"2024-04-25T14:05:08.284307+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:35.998555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.284023+0000","last_change":"2024-04-25T14:05:06.258406+0000","last_active":"2024-04-25T14:05:08.284023+0000","last_peered":"2024-04-25T14:05:08.284023+0000","last_clean":"2024-04-25T14:05:08.284023+0000","last_became_active":"2024-04-25T14:05:06.258080+0000","last_became_peered":"2024-04-25T14:05:06.258080+0000","last_unstale":"2024-04-25T14:05:08.284023+0000","last_undegraded":"2024-04-25T14:05:08.284023+0000","last_fullsized":"2024-04-25T14:05:08.284023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:29.416289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252572+0000","last_change":"2024-04-25T14:05:07.258537+0000","last_active":"2024-04-25T14:05:08.252572+0000","last_peered":"2024-04-25T14:05:08.252572+0000","last_clean":"2024-04-25T14:05:08.252572+0000","last_became_active":"2024-04-25T14:05:07.258220+0000","last_became_peered":"2024-04-25T14:05:07.258220+0000","last_unstale":"2024-04-25T14:05:08.252572+0000","last_undegraded":"2024-04-25T14:05:08.252572+0000","last_fullsized":"2024-04-25T14:05:08.252572+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:10.356490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.284109+0000","last_change":"2024-04-25T14:05:06.258099+0000","last_active":"2024-04-25T14:05:08.284109+0000","last_peered":"2024-04-25T14:05:08.284109+0000","last_clean":"2024-04-25T14:05:08.284109+0000","last_became_active":"2024-04-25T14:05:06.255967+0000","last_became_peered":"2024-04-25T14:05:06.255967+0000","last_unstale":"2024-04-25T14:05:08.284109+0000","last_undegraded":"2024-04-25T14:05:08.284109+0000","last_fullsized":"2024-04-25T14:05:08.284109+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:49.855989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.282956+0000","last_change":"2024-04-25T14:05:07.269898+0000","last_active":"2024-04-25T14:05:08.282956+0000","last_peered":"2024-04-25T14:05:08.282956+0000","last_clean":"2024-04-25T14:05:08.282956+0000","last_became_active":"2024-04-25T14:05:07.268973+0000","last_became_peered":"2024-04-25T14:05:07.268973+0000","last_unstale":"2024-04-25T14:05:08.282956+0000","last_undegraded":"2024-04-25T14:05:08.282956+0000","last_fullsized":"2024-04-25T14:05:08.282956+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:35.998555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283282+0000","last_change":"2024-04-25T14:05:06.263356+0000","last_active":"2024-04-25T14:05:08.283282+0000","last_peered":"2024-04-25T14:05:08.283282+0000","last_clean":"2024-04-25T14:05:08.283282+0000","last_became_active":"2024-04-25T14:05:06.263007+0000","last_became_peered":"2024-04-25T14:05:06.263007+0000","last_unstale":"2024-04-25T14:05:08.283282+0000","last_undegraded":"2024-04-25T14:05:08.283282+0000","last_fullsized":"2024-04-25T14:05:08.283282+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:29.416289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283311+0000","last_change":"2024-04-25T14:05:07.270469+0000","last_active":"2024-04-25T14:05:08.283311+0000","last_peered":"2024-04-25T14:05:08.283311+0000","last_clean":"2024-04-25T14:05:08.283311+0000","last_became_active":"2024-04-25T14:05:07.270115+0000","last_became_peered":"2024-04-25T14:05:07.270115+0000","last_unstale":"2024-04-25T14:05:08.283311+0000","last_undegraded":"2024-04-25T14:05:08.283311+0000","last_fullsized":"2024-04-25T14:05:08.283311+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:10.356490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252421+0000","last_change":"2024-04-25T14:05:06.261873+0000","last_active":"2024-04-25T14:05:08.252421+0000","last_peered":"2024-04-25T14:05:08.252421+0000","last_clean":"2024-04-25T14:05:08.252421+0000","last_became_active":"2024-04-25T14:05:06.261493+0000","last_became_peered":"2024-04-25T14:05:06.261493+0000","last_unstale":"2024-04-25T14:05:08.252421+0000","last_undegraded":"2024-04-25T14:05:08.252421+0000","last_fullsized":"2024-04-25T14:05:08.252421+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:17.077511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.284102+0000","last_change":"2024-04-25T14:05:07.272920+0000","last_active":"2024-04-25T14:05:08.284102+0000","last_peered":"2024-04-25T14:05:08.284102+0000","last_clean":"2024-04-25T14:05:08.284102+0000","last_became_active":"2024-04-25T14:05:07.269884+0000","last_became_peered":"2024-04-25T14:05:07.269884+0000","last_unstale":"2024-04-25T14:05:08.284102+0000","last_undegraded":"2024-04-25T14:05:08.284102+0000","last_fullsized":"2024-04-25T14:05:08.284102+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:05:13.234083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283326+0000","last_change":"2024-04-25T14:05:06.265735+0000","last_active":"2024-04-25T14:05:08.283326+0000","last_peered":"2024-04-25T14:05:08.283326+0000","last_clean":"2024-04-25T14:05:08.283326+0000","last_became_active":"2024-04-25T14:05:06.265476+0000","last_became_peered":"2024-04-25T14:05:06.265476+0000","last_unstale":"2024-04-25T14:05:08.283326+0000","last_undegraded":"2024-04-25T14:05:08.283326+0000","last_fullsized":"2024-04-25T14:05:08.283326+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:08.266732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.251980+0000","last_change":"2024-04-25T14:05:07.270163+0000","last_active":"2024-04-25T14:05:08.251980+0000","last_peered":"2024-04-25T14:05:08.251980+0000","last_clean":"2024-04-25T14:05:08.251980+0000","last_became_active":"2024-04-25T14:05:07.268272+0000","last_became_peered":"2024-04-25T14:05:07.268272+0000","last_unstale":"2024-04-25T14:05:08.251980+0000","last_undegraded":"2024-04-25T14:05:08.251980+0000","last_fullsized":"2024-04-25T14:05:08.251980+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:48:18.920873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283500+0000","last_change":"2024-04-25T14:05:06.260729+0000","last_active":"2024-04-25T14:05:08.283500+0000","last_peered":"2024-04-25T14:05:08.283500+0000","last_clean":"2024-04-25T14:05:08.283500+0000","last_became_active":"2024-04-25T14:05:06.260022+0000","last_became_peered":"2024-04-25T14:05:06.260022+0000","last_unstale":"2024-04-25T14:05:08.283500+0000","last_undegraded":"2024-04-25T14:05:08.283500+0000","last_fullsized":"2024-04-25T14:05:08.283500+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:20.185798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.279590+0000","last_change":"2024-04-25T14:05:07.279074+0000","last_active":"2024-04-25T14:05:08.279590+0000","last_peered":"2024-04-25T14:05:08.279590+0000","last_clean":"2024-04-25T14:05:08.279590+0000","last_became_active":"2024-04-25T14:05:07.278499+0000","last_became_peered":"2024-04-25T14:05:07.278499+0000","last_unstale":"2024-04-25T14:05:08.279590+0000","last_undegraded":"2024-04-25T14:05:08.279590+0000","last_fullsized":"2024-04-25T14:05:08.279590+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:08.852015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252618+0000","last_change":"2024-04-25T14:05:06.251848+0000","last_active":"2024-04-25T14:05:08.252618+0000","last_peered":"2024-04-25T14:05:08.252618+0000","last_clean":"2024-04-25T14:05:08.252618+0000","last_became_active":"2024-04-25T14:05:06.251541+0000","last_became_peered":"2024-04-25T14:05:06.251541+0000","last_unstale":"2024-04-25T14:05:08.252618+0000","last_undegraded":"2024-04-25T14:05:08.252618+0000","last_fullsized":"2024-04-25T14:05:08.252618+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:55:54.218054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252655+0000","last_change":"2024-04-25T14:05:07.278976+0000","last_active":"2024-04-25T14:05:08.252655+0000","last_peered":"2024-04-25T14:05:08.252655+0000","last_clean":"2024-04-25T14:05:08.252655+0000","last_became_active":"2024-04-25T14:05:07.278721+0000","last_became_peered":"2024-04-25T14:05:07.278721+0000","last_unstale":"2024-04-25T14:05:08.252655+0000","last_undegraded":"2024-04-25T14:05:08.252655+0000","last_fullsized":"2024-04-25T14:05:08.252655+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:41.508799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283244+0000","last_change":"2024-04-25T14:05:06.259232+0000","last_active":"2024-04-25T14:05:08.283244+0000","last_peered":"2024-04-25T14:05:08.283244+0000","last_clean":"2024-04-25T14:05:08.283244+0000","last_became_active":"2024-04-25T14:05:06.258861+0000","last_became_peered":"2024-04-25T14:05:06.258861+0000","last_unstale":"2024-04-25T14:05:08.283244+0000","last_undegraded":"2024-04-25T14:05:08.283244+0000","last_fullsized":"2024-04-25T14:05:08.283244+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:07.846107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283638+0000","last_change":"2024-04-25T14:05:07.273047+0000","last_active":"2024-04-25T14:05:08.283638+0000","last_peered":"2024-04-25T14:05:08.283638+0000","last_clean":"2024-04-25T14:05:08.283638+0000","last_became_active":"2024-04-25T14:05:07.272760+0000","last_became_peered":"2024-04-25T14:05:07.272760+0000","last_unstale":"2024-04-25T14:05:08.283638+0000","last_undegraded":"2024-04-25T14:05:08.283638+0000","last_fullsized":"2024-04-25T14:05:08.283638+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:41.508799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283628+0000","last_change":"2024-04-25T14:05:06.261078+0000","last_active":"2024-04-25T14:05:08.283628+0000","last_peered":"2024-04-25T14:05:08.283628+0000","last_clean":"2024-04-25T14:05:08.283628+0000","last_became_active":"2024-04-25T14:05:06.260706+0000","last_became_peered":"2024-04-25T14:05:06.260706+0000","last_unstale":"2024-04-25T14:05:08.283628+0000","last_undegraded":"2024-04-25T14:05:08.283628+0000","last_fullsized":"2024-04-25T14:05:08.283628+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:55:54.218054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.284336+0000","last_change":"2024-04-25T14:05:07.289198+0000","last_active":"2024-04-25T14:05:08.284336+0000","last_peered":"2024-04-25T14:05:08.284336+0000","last_clean":"2024-04-25T14:05:08.284336+0000","last_became_active":"2024-04-25T14:05:07.288732+0000","last_became_peered":"2024-04-25T14:05:07.288732+0000","last_unstale":"2024-04-25T14:05:08.284336+0000","last_undegraded":"2024-04-25T14:05:08.284336+0000","last_fullsized":"2024-04-25T14:05:08.284336+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:20.873445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252350+0000","last_change":"2024-04-25T14:05:06.247598+0000","last_active":"2024-04-25T14:05:08.252350+0000","last_peered":"2024-04-25T14:05:08.252350+0000","last_clean":"2024-04-25T14:05:08.252350+0000","last_became_active":"2024-04-25T14:05:06.246997+0000","last_became_peered":"2024-04-25T14:05:06.246997+0000","last_unstale":"2024-04-25T14:05:08.252350+0000","last_undegraded":"2024-04-25T14:05:08.252350+0000","last_fullsized":"2024-04-25T14:05:08.252350+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:07.846107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283051+0000","last_change":"2024-04-25T14:05:07.259497+0000","last_active":"2024-04-25T14:05:08.283051+0000","last_peered":"2024-04-25T14:05:08.283051+0000","last_clean":"2024-04-25T14:05:08.283051+0000","last_became_active":"2024-04-25T14:05:07.259222+0000","last_became_peered":"2024-04-25T14:05:07.259222+0000","last_unstale":"2024-04-25T14:05:08.283051+0000","last_undegraded":"2024-04-25T14:05:08.283051+0000","last_fullsized":"2024-04-25T14:05:08.283051+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:20.873445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252369+0000","last_change":"2024-04-25T14:05:06.255952+0000","last_active":"2024-04-25T14:05:08.252369+0000","last_peered":"2024-04-25T14:05:08.252369+0000","last_clean":"2024-04-25T14:05:08.252369+0000","last_became_active":"2024-04-25T14:05:06.255606+0000","last_became_peered":"2024-04-25T14:05:06.255606+0000","last_unstale":"2024-04-25T14:05:08.252369+0000","last_undegraded":"2024-04-25T14:05:08.252369+0000","last_fullsized":"2024-04-25T14:05:08.252369+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:54.628494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252252+0000","last_change":"2024-04-25T14:05:07.263037+0000","last_active":"2024-04-25T14:05:08.252252+0000","last_peered":"2024-04-25T14:05:08.252252+0000","last_clean":"2024-04-25T14:05:08.252252+0000","last_became_active":"2024-04-25T14:05:07.261865+0000","last_became_peered":"2024-04-25T14:05:07.261865+0000","last_unstale":"2024-04-25T14:05:08.252252+0000","last_undegraded":"2024-04-25T14:05:08.252252+0000","last_fullsized":"2024-04-25T14:05:08.252252+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:54.320451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283667+0000","last_change":"2024-04-25T14:05:06.261678+0000","last_active":"2024-04-25T14:05:08.283667+0000","last_peered":"2024-04-25T14:05:08.283667+0000","last_clean":"2024-04-25T14:05:08.283667+0000","last_became_active":"2024-04-25T14:05:06.261336+0000","last_became_peered":"2024-04-25T14:05:06.261336+0000","last_unstale":"2024-04-25T14:05:08.283667+0000","last_undegraded":"2024-04-25T14:05:08.283667+0000","last_fullsized":"2024-04-25T14:05:08.283667+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:17.077511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252270+0000","last_change":"2024-04-25T14:05:07.277302+0000","last_active":"2024-04-25T14:05:08.252270+0000","last_peered":"2024-04-25T14:05:08.252270+0000","last_clean":"2024-04-25T14:05:08.252270+0000","last_became_active":"2024-04-25T14:05:07.276991+0000","last_became_peered":"2024-04-25T14:05:07.276991+0000","last_unstale":"2024-04-25T14:05:08.252270+0000","last_undegraded":"2024-04-25T14:05:08.252270+0000","last_fullsized":"2024-04-25T14:05:08.252270+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:20.873445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.251448+0000","last_change":"2024-04-25T14:05:06.264082+0000","last_active":"2024-04-25T14:05:08.251448+0000","last_peered":"2024-04-25T14:05:08.251448+0000","last_clean":"2024-04-25T14:05:08.251448+0000","last_became_active":"2024-04-25T14:05:06.263590+0000","last_became_peered":"2024-04-25T14:05:06.263590+0000","last_unstale":"2024-04-25T14:05:08.251448+0000","last_undegraded":"2024-04-25T14:05:08.251448+0000","last_fullsized":"2024-04-25T14:05:08.251448+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:18.883712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252251+0000","last_change":"2024-04-25T14:05:07.255646+0000","last_active":"2024-04-25T14:05:08.252251+0000","last_peered":"2024-04-25T14:05:08.252251+0000","last_clean":"2024-04-25T14:05:08.252251+0000","last_became_active":"2024-04-25T14:05:07.255170+0000","last_became_peered":"2024-04-25T14:05:07.255170+0000","last_unstale":"2024-04-25T14:05:08.252251+0000","last_undegraded":"2024-04-25T14:05:08.252251+0000","last_fullsized":"2024-04-25T14:05:08.252251+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:56.697086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252341+0000","last_change":"2024-04-25T14:05:06.256481+0000","last_active":"2024-04-25T14:05:08.252341+0000","last_peered":"2024-04-25T14:05:08.252341+0000","last_clean":"2024-04-25T14:05:08.252341+0000","last_became_active":"2024-04-25T14:05:06.256100+0000","last_became_peered":"2024-04-25T14:05:06.256100+0000","last_unstale":"2024-04-25T14:05:08.252341+0000","last_undegraded":"2024-04-25T14:05:08.252341+0000","last_fullsized":"2024-04-25T14:05:08.252341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:23.283424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252235+0000","last_change":"2024-04-25T14:05:07.257837+0000","last_active":"2024-04-25T14:05:08.252235+0000","last_peered":"2024-04-25T14:05:08.252235+0000","last_clean":"2024-04-25T14:05:08.252235+0000","last_became_active":"2024-04-25T14:05:07.257527+0000","last_became_peered":"2024-04-25T14:05:07.257527+0000","last_unstale":"2024-04-25T14:05:08.252235+0000","last_undegraded":"2024-04-25T14:05:08.252235+0000","last_fullsized":"2024-04-25T14:05:08.252235+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:03.144757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283323+0000","last_change":"2024-04-25T14:05:06.264865+0000","last_active":"2024-04-25T14:05:08.283323+0000","last_peered":"2024-04-25T14:05:08.283323+0000","last_clean":"2024-04-25T14:05:08.283323+0000","last_became_active":"2024-04-25T14:05:06.264435+0000","last_became_peered":"2024-04-25T14:05:06.264435+0000","last_unstale":"2024-04-25T14:05:08.283323+0000","last_undegraded":"2024-04-25T14:05:08.283323+0000","last_fullsized":"2024-04-25T14:05:08.283323+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:19.038477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283509+0000","last_change":"2024-04-25T14:05:07.284240+0000","last_active":"2024-04-25T14:05:08.283509+0000","last_peered":"2024-04-25T14:05:08.283509+0000","last_clean":"2024-04-25T14:05:08.283509+0000","last_became_active":"2024-04-25T14:05:07.283884+0000","last_became_peered":"2024-04-25T14:05:07.283884+0000","last_unstale":"2024-04-25T14:05:08.283509+0000","last_undegraded":"2024-04-25T14:05:08.283509+0000","last_fullsized":"2024-04-25T14:05:08.283509+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:06.121071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.284251+0000","last_change":"2024-04-25T14:05:06.265405+0000","last_active":"2024-04-25T14:05:08.284251+0000","last_peered":"2024-04-25T14:05:08.284251+0000","last_clean":"2024-04-25T14:05:08.284251+0000","last_became_active":"2024-04-25T14:05:06.265091+0000","last_became_peered":"2024-04-25T14:05:06.265091+0000","last_unstale":"2024-04-25T14:05:08.284251+0000","last_undegraded":"2024-04-25T14:05:08.284251+0000","last_fullsized":"2024-04-25T14:05:08.284251+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:17.077511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.284167+0000","last_change":"2024-04-25T14:05:07.276399+0000","last_active":"2024-04-25T14:05:08.284167+0000","last_peered":"2024-04-25T14:05:08.284167+0000","last_clean":"2024-04-25T14:05:08.284167+0000","last_became_active":"2024-04-25T14:05:07.275998+0000","last_became_peered":"2024-04-25T14:05:07.275998+0000","last_unstale":"2024-04-25T14:05:08.284167+0000","last_undegraded":"2024-04-25T14:05:08.284167+0000","last_fullsized":"2024-04-25T14:05:08.284167+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:20.169051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.279508+0000","last_change":"2024-04-25T14:05:06.256923+0000","last_active":"2024-04-25T14:05:08.279508+0000","last_peered":"2024-04-25T14:05:08.279508+0000","last_clean":"2024-04-25T14:05:08.279508+0000","last_became_active":"2024-04-25T14:05:06.256596+0000","last_became_peered":"2024-04-25T14:05:06.256596+0000","last_unstale":"2024-04-25T14:05:08.279508+0000","last_undegraded":"2024-04-25T14:05:08.279508+0000","last_fullsized":"2024-04-25T14:05:08.279508+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:13.441560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.279400+0000","last_change":"2024-04-25T14:05:07.271107+0000","last_active":"2024-04-25T14:05:08.279400+0000","last_peered":"2024-04-25T14:05:08.279400+0000","last_clean":"2024-04-25T14:05:08.279400+0000","last_became_active":"2024-04-25T14:05:07.270646+0000","last_became_peered":"2024-04-25T14:05:07.270646+0000","last_unstale":"2024-04-25T14:05:08.279400+0000","last_undegraded":"2024-04-25T14:05:08.279400+0000","last_fullsized":"2024-04-25T14:05:08.279400+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:30.422197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283772+0000","last_change":"2024-04-25T14:05:06.258644+0000","last_active":"2024-04-25T14:05:08.283772+0000","last_peered":"2024-04-25T14:05:08.283772+0000","last_clean":"2024-04-25T14:05:08.283772+0000","last_became_active":"2024-04-25T14:05:06.258332+0000","last_became_peered":"2024-04-25T14:05:06.258332+0000","last_unstale":"2024-04-25T14:05:08.283772+0000","last_undegraded":"2024-04-25T14:05:08.283772+0000","last_fullsized":"2024-04-25T14:05:08.283772+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:23.283424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252607+0000","last_change":"2024-04-25T14:05:07.276322+0000","last_active":"2024-04-25T14:05:08.252607+0000","last_peered":"2024-04-25T14:05:08.252607+0000","last_clean":"2024-04-25T14:05:08.252607+0000","last_became_active":"2024-04-25T14:05:07.275949+0000","last_became_peered":"2024-04-25T14:05:07.275949+0000","last_unstale":"2024-04-25T14:05:08.252607+0000","last_undegraded":"2024-04-25T14:05:08.252607+0000","last_fullsized":"2024-04-25T14:05:08.252607+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:09.919025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.279480+0000","last_change":"2024-04-25T14:05:06.250207+0000","last_active":"2024-04-25T14:05:08.279480+0000","last_peered":"2024-04-25T14:05:08.279480+0000","last_clean":"2024-04-25T14:05:08.279480+0000","last_became_active":"2024-04-25T14:05:06.249868+0000","last_became_peered":"2024-04-25T14:05:06.249868+0000","last_unstale":"2024-04-25T14:05:08.279480+0000","last_undegraded":"2024-04-25T14:05:08.279480+0000","last_fullsized":"2024-04-25T14:05:08.279480+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:53.239152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283144+0000","last_change":"2024-04-25T14:05:07.267521+0000","last_active":"2024-04-25T14:05:08.283144+0000","last_peered":"2024-04-25T14:05:08.283144+0000","last_clean":"2024-04-25T14:05:08.283144+0000","last_became_active":"2024-04-25T14:05:07.267168+0000","last_became_peered":"2024-04-25T14:05:07.267168+0000","last_unstale":"2024-04-25T14:05:08.283144+0000","last_undegraded":"2024-04-25T14:05:08.283144+0000","last_fullsized":"2024-04-25T14:05:08.283144+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:36.823492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.279347+0000","last_change":"2024-04-25T14:05:06.259360+0000","last_active":"2024-04-25T14:05:08.279347+0000","last_peered":"2024-04-25T14:05:08.279347+0000","last_clean":"2024-04-25T14:05:08.279347+0000","last_became_active":"2024-04-25T14:05:06.259018+0000","last_became_peered":"2024-04-25T14:05:06.259018+0000","last_unstale":"2024-04-25T14:05:08.279347+0000","last_undegraded":"2024-04-25T14:05:08.279347+0000","last_fullsized":"2024-04-25T14:05:08.279347+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:40.623886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.282871+0000","last_change":"2024-04-25T14:05:07.264442+0000","last_active":"2024-04-25T14:05:08.282871+0000","last_peered":"2024-04-25T14:05:08.282871+0000","last_clean":"2024-04-25T14:05:08.282871+0000","last_became_active":"2024-04-25T14:05:07.264104+0000","last_became_peered":"2024-04-25T14:05:07.264104+0000","last_unstale":"2024-04-25T14:05:08.282871+0000","last_undegraded":"2024-04-25T14:05:08.282871+0000","last_fullsized":"2024-04-25T14:05:08.282871+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:03.144757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252447+0000","last_change":"2024-04-25T14:05:06.262040+0000","last_active":"2024-04-25T14:05:08.252447+0000","last_peered":"2024-04-25T14:05:08.252447+0000","last_clean":"2024-04-25T14:05:08.252447+0000","last_became_active":"2024-04-25T14:05:06.261582+0000","last_became_peered":"2024-04-25T14:05:06.261582+0000","last_unstale":"2024-04-25T14:05:08.252447+0000","last_undegraded":"2024-04-25T14:05:08.252447+0000","last_fullsized":"2024-04-25T14:05:08.252447+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:08.266732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252375+0000","last_change":"2024-04-25T14:05:07.265063+0000","last_active":"2024-04-25T14:05:08.252375+0000","last_peered":"2024-04-25T14:05:08.252375+0000","last_clean":"2024-04-25T14:05:08.252375+0000","last_became_active":"2024-04-25T14:05:07.264733+0000","last_became_peered":"2024-04-25T14:05:07.264733+0000","last_unstale":"2024-04-25T14:05:08.252375+0000","last_undegraded":"2024-04-25T14:05:08.252375+0000","last_fullsized":"2024-04-25T14:05:08.252375+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:51.159581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.279222+0000","last_change":"2024-04-25T14:05:06.256004+0000","last_active":"2024-04-25T14:05:08.279222+0000","last_peered":"2024-04-25T14:05:08.279222+0000","last_clean":"2024-04-25T14:05:08.279222+0000","last_became_active":"2024-04-25T14:05:06.255694+0000","last_became_peered":"2024-04-25T14:05:06.255694+0000","last_unstale":"2024-04-25T14:05:08.279222+0000","last_undegraded":"2024-04-25T14:05:08.279222+0000","last_fullsized":"2024-04-25T14:05:08.279222+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:26.860266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283602+0000","last_change":"2024-04-25T14:05:07.284497+0000","last_active":"2024-04-25T14:05:08.283602+0000","last_peered":"2024-04-25T14:05:08.283602+0000","last_clean":"2024-04-25T14:05:08.283602+0000","last_became_active":"2024-04-25T14:05:07.284279+0000","last_became_peered":"2024-04-25T14:05:07.284279+0000","last_unstale":"2024-04-25T14:05:08.283602+0000","last_undegraded":"2024-04-25T14:05:08.283602+0000","last_fullsized":"2024-04-25T14:05:08.283602+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:05:13.234083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283433+0000","last_change":"2024-04-25T14:05:06.259664+0000","last_active":"2024-04-25T14:05:08.283433+0000","last_peered":"2024-04-25T14:05:08.283433+0000","last_clean":"2024-04-25T14:05:08.283433+0000","last_became_active":"2024-04-25T14:05:06.259319+0000","last_became_peered":"2024-04-25T14:05:06.259319+0000","last_unstale":"2024-04-25T14:05:08.283433+0000","last_undegraded":"2024-04-25T14:05:08.283433+0000","last_fullsized":"2024-04-25T14:05:08.283433+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:20.530061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252723+0000","last_change":"2024-04-25T14:05:07.275152+0000","last_active":"2024-04-25T14:05:08.252723+0000","last_peered":"2024-04-25T14:05:08.252723+0000","last_clean":"2024-04-25T14:05:08.252723+0000","last_became_active":"2024-04-25T14:05:07.274827+0000","last_became_peered":"2024-04-25T14:05:07.274827+0000","last_unstale":"2024-04-25T14:05:08.252723+0000","last_undegraded":"2024-04-25T14:05:08.252723+0000","last_fullsized":"2024-04-25T14:05:08.252723+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:43:48.479853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.281871+0000","last_change":"2024-04-25T14:05:06.260706+0000","last_active":"2024-04-25T14:05:08.281871+0000","last_peered":"2024-04-25T14:05:08.281871+0000","last_clean":"2024-04-25T14:05:08.281871+0000","last_became_active":"2024-04-25T14:05:06.260410+0000","last_became_peered":"2024-04-25T14:05:06.260410+0000","last_unstale":"2024-04-25T14:05:08.281871+0000","last_undegraded":"2024-04-25T14:05:08.281871+0000","last_fullsized":"2024-04-25T14:05:08.281871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:24.896382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252757+0000","last_change":"2024-04-25T14:05:07.272991+0000","last_active":"2024-04-25T14:05:08.252757+0000","last_peered":"2024-04-25T14:05:08.252757+0000","last_clean":"2024-04-25T14:05:08.252757+0000","last_became_active":"2024-04-25T14:05:07.272691+0000","last_became_peered":"2024-04-25T14:05:07.272691+0000","last_unstale":"2024-04-25T14:05:08.252757+0000","last_undegraded":"2024-04-25T14:05:08.252757+0000","last_fullsized":"2024-04-25T14:05:08.252757+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:06.121071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.284161+0000","last_change":"2024-04-25T14:05:06.253126+0000","last_active":"2024-04-25T14:05:08.284161+0000","last_peered":"2024-04-25T14:05:08.284161+0000","last_clean":"2024-04-25T14:05:08.284161+0000","last_became_active":"2024-04-25T14:05:06.252761+0000","last_became_peered":"2024-04-25T14:05:06.252761+0000","last_unstale":"2024-04-25T14:05:08.284161+0000","last_undegraded":"2024-04-25T14:05:08.284161+0000","last_fullsized":"2024-04-25T14:05:08.284161+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:29.416289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283141+0000","last_change":"2024-04-25T14:05:07.271339+0000","last_active":"2024-04-25T14:05:08.283141+0000","last_peered":"2024-04-25T14:05:08.283141+0000","last_clean":"2024-04-25T14:05:08.283141+0000","last_became_active":"2024-04-25T14:05:07.271051+0000","last_became_peered":"2024-04-25T14:05:07.271051+0000","last_unstale":"2024-04-25T14:05:08.283141+0000","last_undegraded":"2024-04-25T14:05:08.283141+0000","last_fullsized":"2024-04-25T14:05:08.283141+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:39.756595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283493+0000","last_change":"2024-04-25T14:05:06.259878+0000","last_active":"2024-04-25T14:05:08.283493+0000","last_peered":"2024-04-25T14:05:08.283493+0000","last_clean":"2024-04-25T14:05:08.283493+0000","last_became_active":"2024-04-25T14:05:06.259534+0000","last_became_peered":"2024-04-25T14:05:06.259534+0000","last_unstale":"2024-04-25T14:05:08.283493+0000","last_undegraded":"2024-04-25T14:05:08.283493+0000","last_fullsized":"2024-04-25T14:05:08.283493+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:19.038477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283174+0000","last_change":"2024-04-25T14:05:07.272470+0000","last_active":"2024-04-25T14:05:08.283174+0000","last_peered":"2024-04-25T14:05:08.283174+0000","last_clean":"2024-04-25T14:05:08.283174+0000","last_became_active":"2024-04-25T14:05:07.272133+0000","last_became_peered":"2024-04-25T14:05:07.272133+0000","last_unstale":"2024-04-25T14:05:08.283174+0000","last_undegraded":"2024-04-25T14:05:08.283174+0000","last_fullsized":"2024-04-25T14:05:08.283174+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:01:19.113121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.251789+0000","last_change":"2024-04-25T14:05:06.255046+0000","last_active":"2024-04-25T14:05:08.251789+0000","last_peered":"2024-04-25T14:05:08.251789+0000","last_clean":"2024-04-25T14:05:08.251789+0000","last_became_active":"2024-04-25T14:05:06.254726+0000","last_became_peered":"2024-04-25T14:05:06.254726+0000","last_unstale":"2024-04-25T14:05:08.251789+0000","last_undegraded":"2024-04-25T14:05:08.251789+0000","last_fullsized":"2024-04-25T14:05:08.251789+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:24.896382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283861+0000","last_change":"2024-04-25T14:05:07.290722+0000","last_active":"2024-04-25T14:05:08.283861+0000","last_peered":"2024-04-25T14:05:08.283861+0000","last_clean":"2024-04-25T14:05:08.283861+0000","last_became_active":"2024-04-25T14:05:07.287769+0000","last_became_peered":"2024-04-25T14:05:07.287769+0000","last_unstale":"2024-04-25T14:05:08.283861+0000","last_undegraded":"2024-04-25T14:05:08.283861+0000","last_fullsized":"2024-04-25T14:05:08.283861+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:43.913161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283608+0000","last_change":"2024-04-25T14:05:06.256512+0000","last_active":"2024-04-25T14:05:08.283608+0000","last_peered":"2024-04-25T14:05:08.283608+0000","last_clean":"2024-04-25T14:05:08.283608+0000","last_became_active":"2024-04-25T14:05:06.255995+0000","last_became_peered":"2024-04-25T14:05:06.255995+0000","last_unstale":"2024-04-25T14:05:08.283608+0000","last_undegraded":"2024-04-25T14:05:08.283608+0000","last_fullsized":"2024-04-25T14:05:08.283608+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:39.951627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252896+0000","last_change":"2024-04-25T14:05:07.267897+0000","last_active":"2024-04-25T14:05:08.252896+0000","last_peered":"2024-04-25T14:05:08.252896+0000","last_clean":"2024-04-25T14:05:08.252896+0000","last_became_active":"2024-04-25T14:05:07.267609+0000","last_became_peered":"2024-04-25T14:05:07.267609+0000","last_unstale":"2024-04-25T14:05:08.252896+0000","last_undegraded":"2024-04-25T14:05:08.252896+0000","last_fullsized":"2024-04-25T14:05:08.252896+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:54.320451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252100+0000","last_change":"2024-04-25T14:05:06.252301+0000","last_active":"2024-04-25T14:05:08.252100+0000","last_peered":"2024-04-25T14:05:08.252100+0000","last_clean":"2024-04-25T14:05:08.252100+0000","last_became_active":"2024-04-25T14:05:06.251997+0000","last_became_peered":"2024-04-25T14:05:06.251997+0000","last_unstale":"2024-04-25T14:05:08.252100+0000","last_undegraded":"2024-04-25T14:05:08.252100+0000","last_fullsized":"2024-04-25T14:05:08.252100+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:20.530061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.283217+0000","last_change":"2024-04-25T14:05:07.280588+0000","last_active":"2024-04-25T14:05:08.283217+0000","last_peered":"2024-04-25T14:05:08.283217+0000","last_clean":"2024-04-25T14:05:08.283217+0000","last_became_active":"2024-04-25T14:05:07.280269+0000","last_became_peered":"2024-04-25T14:05:07.280269+0000","last_unstale":"2024-04-25T14:05:08.283217+0000","last_undegraded":"2024-04-25T14:05:08.283217+0000","last_fullsized":"2024-04-25T14:05:08.283217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:06.230215+0000","last_clean_scrub_stamp":"2024-04-25T14:05:06.230215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:13.992505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252805+0000","last_change":"2024-04-25T14:05:06.257515+0000","last_active":"2024-04-25T14:05:08.252805+0000","last_peered":"2024-04-25T14:05:08.252805+0000","last_clean":"2024-04-25T14:05:08.252805+0000","last_became_active":"2024-04-25T14:05:06.257198+0000","last_became_peered":"2024-04-25T14:05:06.257198+0000","last_unstale":"2024-04-25T14:05:08.252805+0000","last_undegraded":"2024-04-25T14:05:08.252805+0000","last_fullsized":"2024-04-25T14:05:08.252805+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:05.224308+0000","last_clean_scrub_stamp":"2024-04-25T14:05:05.224308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:29.416289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":142,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T14:05:08.252552+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:05:08.252552+0000","last_peered":"2024-04-25T14:05:08.252552+0000","last_clean":"2024-04-25T14:05:08.252552+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:05:08.252552+0000","last_undegraded":"2024-04-25T14:05:08.252552+0000","last_fullsized":"2024-04-25T14:05:08.252552+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1048576,"data_stored":782078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27444,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":430080,"data_stored":183918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27444,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":434176,"data_stored":183448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":190632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":184982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":183406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":183426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1036288,"data_stored":780430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:05:11.788 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T14:05:11.788 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T14:05:11.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.980+0000 7f188cc00640 1 Processor -- start 2024-04-25T14:05:11.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.980+0000 7f188cc00640 1 -- start start 2024-04-25T14:05:11.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f188cc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1888070330 0x7f18880d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f188cc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18880d2030 con 0x7f18880d1c20 2024-04-25T14:05:11.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f188cc00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f188815e5c0 con 0x7f18880ce790 2024-04-25T14:05:11.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f188cc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f188815e8a0 con 0x7f1888070330 2024-04-25T14:05:11.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f187d400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f18880d1c20 0x7f188806ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37216/0 (socket says 172.21.15.86:37216) 2024-04-25T14:05:11.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f187d400640 1 -- 172.21.15.86:0/599417416 learned_addr learned my addr 172.21.15.86:0/599417416 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:11.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f1885e00640 1 --2- 172.21.15.86:0/599417416 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1888070330 0x7f18880d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f1885400640 1 -- 172.21.15.86:0/599417416 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1532596928 0 0) 0x7f1870001120 con 0x7f18880d1c20 2024-04-25T14:05:11.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f1885400640 1 -- 172.21.15.86:0/599417416 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f186c003530 con 0x7f18880d1c20 2024-04-25T14:05:11.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f1885400640 1 -- 172.21.15.86:0/599417416 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 854928122 0 0) 0x7f186c003530 con 0x7f18880d1c20 2024-04-25T14:05:11.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f1885400640 1 -- 172.21.15.86:0/599417416 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1888070330 msgr2=0x7f18880d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:11.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f1885400640 1 --2- 172.21.15.86:0/599417416 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1888070330 0x7f18880d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f1885400640 1 -- 172.21.15.86:0/599417416 >> v1:172.21.15.155:6789/0 conn(0x7f18880ce790 legacy=0x7f188806f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f1885400640 1 -- 172.21.15.86:0/599417416 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188815eb80 con 0x7f18880d1c20 2024-04-25T14:05:11.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f1885400640 1 -- 172.21.15.86:0/599417416 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f1870001850 con 0x7f18880d1c20 2024-04-25T14:05:11.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f1885400640 1 -- 172.21.15.86:0/599417416 >> v1:172.21.15.86:6789/0 conn(0x7f18880d1c20 legacy=0x7f188806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f1885400640 1 --2- 172.21.15.86:0/599417416 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1888070330 0x7f186c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f1885400640 1 --2- 172.21.15.86:0/599417416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f186c003fe0 0x7f186c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f1885400640 1 --2- 172.21.15.86:0/599417416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f186c004d50 0x7f186c0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f1885400640 1 -- 172.21.15.86:0/599417416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f188815e8a0 con 0x7f186c003fe0 2024-04-25T14:05:11.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f1885400640 1 -- 172.21.15.86:0/599417416 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f186c00bc80 con 0x7f186c004d50 2024-04-25T14:05:11.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f1885400640 1 -- 172.21.15.86:0/599417416 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f186c00be80 con 0x7f1888070330 2024-04-25T14:05:11.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f187d400640 1 --2- 172.21.15.86:0/599417416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f186c003fe0 0x7f186c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f1885e00640 1 --2- 172.21.15.86:0/599417416 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1888070330 0x7f186c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f1886800640 1 --2- 172.21.15.86:0/599417416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f186c004d50 0x7f186c0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f187d400640 1 -- 172.21.15.86:0/599417416 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1888070330 msgr2=0x7f186c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:11.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f187d400640 1 --2- 172.21.15.86:0/599417416 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1888070330 0x7f186c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f187d400640 1 -- 172.21.15.86:0/599417416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f186c004d50 msgr2=0x7f186c0051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:11.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f187d400640 1 --2- 172.21.15.86:0/599417416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f186c004d50 0x7f186c0051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.984+0000 7f187d400640 1 -- 172.21.15.86:0/599417416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186c003530 con 0x7f186c003fe0 2024-04-25T14:05:11.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.988+0000 7f187d400640 1 --2- 172.21.15.86:0/599417416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f186c003fe0 0x7f186c004810 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f187000c4c0 tx=0x7f187001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d77d6a98f7d088c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:11.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.988+0000 7f1885400640 1 -- 172.21.15.86:0/599417416 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187000c0e0 con 0x7f186c003fe0 2024-04-25T14:05:11.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.988+0000 7f1885400640 1 -- 172.21.15.86:0/599417416 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18700a9070 con 0x7f186c003fe0 2024-04-25T14:05:11.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.988+0000 7f1885400640 1 -- 172.21.15.86:0/599417416 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1870005d80 con 0x7f186c003fe0 2024-04-25T14:05:11.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.988+0000 7f188cc00640 1 -- 172.21.15.86:0/599417416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f186c003fe0 msgr2=0x7f186c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.988+0000 7f188cc00640 1 --2- 172.21.15.86:0/599417416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f186c003fe0 0x7f186c004810 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f187000c4c0 tx=0x7f187001cc80 comp rx=0 tx=0).stop 2024-04-25T14:05:11.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.988+0000 7f1886800640 1 -- 172.21.15.86:0/599417416 reap_dead start 2024-04-25T14:05:11.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.988+0000 7f188cc00640 1 -- 172.21.15.86:0/599417416 shutdown_connections 2024-04-25T14:05:11.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.988+0000 7f188cc00640 1 -- 172.21.15.86:0/599417416 >> 172.21.15.86:0/599417416 conn(0x7f18880b4650 msgr2=0x7f18880b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:11.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.988+0000 7f188cc00640 1 -- 172.21.15.86:0/599417416 shutdown_connections 2024-04-25T14:05:11.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.988+0000 7f188cc00640 1 -- 172.21.15.86:0/599417416 wait complete. 2024-04-25T14:05:11.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.992+0000 7f188cc00640 1 Processor -- start 2024-04-25T14:05:11.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.992+0000 7f188cc00640 1 -- start start 2024-04-25T14:05:11.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.992+0000 7f188cc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f186c003fe0 0x7f18880d0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.992+0000 7f188cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f188809a340 0x7f1888099ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.992+0000 7f188cc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1888151560 0x7f188815bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.992+0000 7f188cc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18880708c0 con 0x7f188809a340 2024-04-25T14:05:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.992+0000 7f188cc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1888070740 con 0x7f186c003fe0 2024-04-25T14:05:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.992+0000 7f188cc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f188806fb50 con 0x7f1888151560 2024-04-25T14:05:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.992+0000 7f187d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f188809a340 0x7f1888099ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.992+0000 7f187d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f188809a340 0x7f1888099ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57560/0 (socket says 172.21.15.86:57560) 2024-04-25T14:05:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.992+0000 7f187d400640 1 -- 172.21.15.86:0/4129879657 learned_addr learned my addr 172.21.15.86:0/4129879657 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.992+0000 7f1886800640 1 --2- 172.21.15.86:0/4129879657 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1888151560 0x7f188815bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.992+0000 7f1885e00640 1 --2- 172.21.15.86:0/4129879657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f186c003fe0 0x7f18880d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.992+0000 7f187d400640 1 -- 172.21.15.86:0/4129879657 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1888151560 msgr2=0x7f188815bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.992+0000 7f187d400640 1 --2- 172.21.15.86:0/4129879657 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1888151560 0x7f188815bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.992+0000 7f187d400640 1 -- 172.21.15.86:0/4129879657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f186c003fe0 msgr2=0x7f18880d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.992+0000 7f187d400640 1 --2- 172.21.15.86:0/4129879657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f186c003fe0 0x7f18880d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.992+0000 7f187d400640 1 -- 172.21.15.86:0/4129879657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188815dd00 con 0x7f188809a340 2024-04-25T14:05:11.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.992+0000 7f1885e00640 1 --2- 172.21.15.86:0/4129879657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f186c003fe0 0x7f18880d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:11.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.992+0000 7f187d400640 1 --2- 172.21.15.86:0/4129879657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f188809a340 0x7f1888099ad0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f187000c4c0 tx=0x7f186c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:11.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.992+0000 7f187ea00640 1 -- 172.21.15.86:0/4129879657 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1870000ee0 con 0x7f188809a340 2024-04-25T14:05:11.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.996+0000 7f187ea00640 1 -- 172.21.15.86:0/4129879657 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1870005d60 con 0x7f188809a340 2024-04-25T14:05:11.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.996+0000 7f188cc00640 1 -- 172.21.15.86:0/4129879657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f188815c960 con 0x7f188809a340 2024-04-25T14:05:11.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.996+0000 7f187ea00640 1 -- 172.21.15.86:0/4129879657 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1870001610 con 0x7f188809a340 2024-04-25T14:05:11.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.996+0000 7f188cc00640 1 -- 172.21.15.86:0/4129879657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f188815cee0 con 0x7f188809a340 2024-04-25T14:05:11.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.996+0000 7f188cc00640 1 -- 172.21.15.86:0/4129879657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1888069d00 con 0x7f188809a340 2024-04-25T14:05:11.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.996+0000 7f187ea00640 1 -- 172.21.15.86:0/4129879657 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f18700a5760 con 0x7f188809a340 2024-04-25T14:05:12.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.996+0000 7f187ea00640 1 --2- 172.21.15.86:0/4129879657 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f186406f5a0 0x7f1864071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.996+0000 7f187ea00640 1 -- 172.21.15.86:0/4129879657 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f1870128680 con 0x7f188809a340 2024-04-25T14:05:12.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:11.996+0000 7f1885e00640 1 --2- 172.21.15.86:0/4129879657 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f186406f5a0 0x7f1864071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.000+0000 7f1885e00640 1 --2- 172.21.15.86:0/4129879657 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f186406f5a0 0x7f1864071ab0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1888072200 tx=0x7f1878008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:12.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.000+0000 7f187ea00640 1 -- 172.21.15.86:0/4129879657 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f188815c960 con 0x7f188809a340 2024-04-25T14:05:12.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.212+0000 7f188cc00640 1 -- 172.21.15.86:0/4129879657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f18880b4b20 con 0x7f188809a340 2024-04-25T14:05:12.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.216+0000 7f187ea00640 1 -- 172.21.15.86:0/4129879657 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7f18700bf060 con 0x7f188809a340 2024-04-25T14:05:12.219 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-04-25T14:05:12.219 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCLYipm5u4xHRAACDFe3Eb4KLrsdLj4eFe7mA== 2024-04-25T14:05:12.219 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-04-25T14:05:12.219 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-04-25T14:05:12.219 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T14:05:12.220 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.0 2024-04-25T14:05:12.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.220+0000 7f188cc00640 1 -- 172.21.15.86:0/4129879657 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f186406f5a0 msgr2=0x7f1864071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.220+0000 7f188cc00640 1 --2- 172.21.15.86:0/4129879657 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f186406f5a0 0x7f1864071ab0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1888072200 tx=0x7f1878008040 comp rx=0 tx=0).stop 2024-04-25T14:05:12.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.220+0000 7f188cc00640 1 -- 172.21.15.86:0/4129879657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f188809a340 msgr2=0x7f1888099ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.220+0000 7f188cc00640 1 --2- 172.21.15.86:0/4129879657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f188809a340 0x7f1888099ad0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f187000c4c0 tx=0x7f186c005840 comp rx=0 tx=0).stop 2024-04-25T14:05:12.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.220+0000 7f188cc00640 1 -- 172.21.15.86:0/4129879657 shutdown_connections 2024-04-25T14:05:12.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.220+0000 7f188cc00640 1 --2- 172.21.15.86:0/4129879657 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f186406f5a0 0x7f1864071ab0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.220+0000 7f188cc00640 1 --2- 172.21.15.86:0/4129879657 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1888151560 0x7f188815bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.220+0000 7f188cc00640 1 --2- 172.21.15.86:0/4129879657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f188809a340 0x7f1888099ad0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.220+0000 7f188cc00640 1 --2- 172.21.15.86:0/4129879657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f186c003fe0 0x7f18880d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.220+0000 7f188cc00640 1 -- 172.21.15.86:0/4129879657 >> 172.21.15.86:0/4129879657 conn(0x7f18880b4650 msgr2=0x7f1888155030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:12.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.220+0000 7f188cc00640 1 -- 172.21.15.86:0/4129879657 shutdown_connections 2024-04-25T14:05:12.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.220+0000 7f188cc00640 1 -- 172.21.15.86:0/4129879657 wait complete. 2024-04-25T14:05:12.241 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:05:12.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.384+0000 7fdbad600640 1 Processor -- start 2024-04-25T14:05:12.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.384+0000 7fdbad600640 1 -- start start 2024-04-25T14:05:12.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.388+0000 7fdbad600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdba8072630 0x7fdba8072a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.388+0000 7fdbad600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdba806e340 0x7fdba806e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.388+0000 7fdbad600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdba8072f80 con 0x7fdba8163980 2024-04-25T14:05:12.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.388+0000 7fdbad600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fdba81648e0 con 0x7fdba806e340 2024-04-25T14:05:12.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.388+0000 7fdbad600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdba80d1520 con 0x7fdba8072630 2024-04-25T14:05:12.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.388+0000 7fdba7400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdba806e340 0x7fdba806e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.388+0000 7fdba6a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdba8072630 0x7fdba8072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.388+0000 7fdba7400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdba806e340 0x7fdba806e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:49418/0 (socket says 172.21.15.86:49418) 2024-04-25T14:05:12.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.388+0000 7fdba7400640 1 -- 172.21.15.86:0/3624778723 learned_addr learned my addr 172.21.15.86:0/3624778723 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:12.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.388+0000 7fdb9e000640 1 --1- 172.21.15.86:0/3624778723 >> v1:172.21.15.155:6789/0 conn(0x7fdba8163980 0x7fdba8163d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:47730/0 (socket says 172.21.15.86:47730) 2024-04-25T14:05:12.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.388+0000 7fdba6a00640 1 -- 172.21.15.86:0/3624778723 >> v2:172.21.15.155:3300/0 conn(0x7fdba806e340 msgr2=0x7fdba806e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.388+0000 7fdba6a00640 1 --2- 172.21.15.86:0/3624778723 >> v2:172.21.15.155:3300/0 conn(0x7fdba806e340 0x7fdba806e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.388+0000 7fdba6a00640 1 -- 172.21.15.86:0/3624778723 >> v1:172.21.15.155:6789/0 conn(0x7fdba8163980 legacy=0x7fdba8163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.388+0000 7fdba6a00640 1 -- 172.21.15.86:0/3624778723 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba80d23e0 con 0x7fdba8072630 2024-04-25T14:05:12.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.388+0000 7fdba7400640 1 --2- 172.21.15.86:0/3624778723 >> v2:172.21.15.155:3300/0 conn(0x7fdba806e340 0x7fdba806e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:12.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.388+0000 7fdba6000640 1 -- 172.21.15.86:0/3624778723 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 206997443 0 0) 0x7fdb90001120 con 0x7fdba8163980 2024-04-25T14:05:12.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.392+0000 7fdba6a00640 1 --2- 172.21.15.86:0/3624778723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdba8072630 0x7fdba8072a40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdb980097c0 tx=0x7fdb9801c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73161e7615fcee67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:12.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.392+0000 7fdba6000640 1 -- 172.21.15.86:0/3624778723 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb980b0070 con 0x7fdba8072630 2024-04-25T14:05:12.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.392+0000 7fdba6000640 1 -- 172.21.15.86:0/3624778723 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb980ab600 con 0x7fdba8072630 2024-04-25T14:05:12.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.392+0000 7fdba6000640 1 -- 172.21.15.86:0/3624778723 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb980ab9d0 con 0x7fdba8072630 2024-04-25T14:05:12.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.392+0000 7fdbad600640 1 -- 172.21.15.86:0/3624778723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdba8072630 msgr2=0x7fdba8072a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.392+0000 7fdbad600640 1 --2- 172.21.15.86:0/3624778723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdba8072630 0x7fdba8072a40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdb980097c0 tx=0x7fdb9801c380 comp rx=0 tx=0).stop 2024-04-25T14:05:12.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.392+0000 7fdbad600640 1 -- 172.21.15.86:0/3624778723 shutdown_connections 2024-04-25T14:05:12.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.392+0000 7fdbad600640 1 --2- 172.21.15.86:0/3624778723 >> v2:172.21.15.155:3300/0 conn(0x7fdba806e340 0x7fdba806e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.392+0000 7fdbad600640 1 --2- 172.21.15.86:0/3624778723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdba8072630 0x7fdba8072a40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.392+0000 7fdbad600640 1 -- 172.21.15.86:0/3624778723 >> 172.21.15.86:0/3624778723 conn(0x7fdba80d3580 msgr2=0x7fdba80d3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:12.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.392+0000 7fdbad600640 1 -- 172.21.15.86:0/3624778723 shutdown_connections 2024-04-25T14:05:12.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.392+0000 7fdbad600640 1 -- 172.21.15.86:0/3624778723 wait complete. 2024-04-25T14:05:12.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.396+0000 7fdbad600640 1 Processor -- start 2024-04-25T14:05:12.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.396+0000 7fdbad600640 1 -- start start 2024-04-25T14:05:12.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.396+0000 7fdbad600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdba806e340 0x7fdba80c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.396+0000 7fdbad600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdba8072630 0x7fdba81fcd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.396+0000 7fdbad600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdba8163980 0x7fdba80b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.396+0000 7fdbad600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdba816d530 con 0x7fdba806e340 2024-04-25T14:05:12.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.396+0000 7fdbad600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdba806f0a0 con 0x7fdba8163980 2024-04-25T14:05:12.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.396+0000 7fdbad600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdba816d6b0 con 0x7fdba8072630 2024-04-25T14:05:12.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.396+0000 7fdba6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdba806e340 0x7fdba80c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.396+0000 7fdba6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdba806e340 0x7fdba80c91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57562/0 (socket says 172.21.15.86:57562) 2024-04-25T14:05:12.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.396+0000 7fdba6a00640 1 -- 172.21.15.86:0/1264607696 learned_addr learned my addr 172.21.15.86:0/1264607696 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:12.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.396+0000 7fdba7400640 1 --2- 172.21.15.86:0/1264607696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdba8163980 0x7fdba80b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.396+0000 7fdb9e000640 1 --2- 172.21.15.86:0/1264607696 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdba8072630 0x7fdba81fcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.396+0000 7fdba6a00640 1 -- 172.21.15.86:0/1264607696 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdba8072630 msgr2=0x7fdba81fcd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.396+0000 7fdba6a00640 1 --2- 172.21.15.86:0/1264607696 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdba8072630 0x7fdba81fcd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.396+0000 7fdba6a00640 1 -- 172.21.15.86:0/1264607696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdba8163980 msgr2=0x7fdba80b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.396+0000 7fdba6a00640 1 --2- 172.21.15.86:0/1264607696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdba8163980 0x7fdba80b3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.396+0000 7fdba6a00640 1 -- 172.21.15.86:0/1264607696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba80c9720 con 0x7fdba806e340 2024-04-25T14:05:12.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.396+0000 7fdba6a00640 1 --2- 172.21.15.86:0/1264607696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdba806e340 0x7fdba80c91e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdb98019fb0 tx=0x7fdb9801abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:12.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.396+0000 7fdb9f400640 1 -- 172.21.15.86:0/1264607696 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb980b0050 con 0x7fdba806e340 2024-04-25T14:05:12.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.396+0000 7fdb9f400640 1 -- 172.21.15.86:0/1264607696 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb980bc070 con 0x7fdba806e340 2024-04-25T14:05:12.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.400+0000 7fdb9f400640 1 -- 172.21.15.86:0/1264607696 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb980b5830 con 0x7fdba806e340 2024-04-25T14:05:12.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.400+0000 7fdbad600640 1 -- 172.21.15.86:0/1264607696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdba81fa4a0 con 0x7fdba806e340 2024-04-25T14:05:12.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.400+0000 7fdbad600640 1 -- 172.21.15.86:0/1264607696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdba81faa60 con 0x7fdba806e340 2024-04-25T14:05:12.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.400+0000 7fdbad600640 1 -- 172.21.15.86:0/1264607696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdba8072170 con 0x7fdba806e340 2024-04-25T14:05:12.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.400+0000 7fdb9f400640 1 -- 172.21.15.86:0/1264607696 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdb980b59d0 con 0x7fdba806e340 2024-04-25T14:05:12.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.400+0000 7fdb9f400640 1 --2- 172.21.15.86:0/1264607696 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdb8406f400 0x7fdb84071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.400+0000 7fdb9f400640 1 -- 172.21.15.86:0/1264607696 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fdb98129b40 con 0x7fdba806e340 2024-04-25T14:05:12.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.400+0000 7fdb9e000640 1 --2- 172.21.15.86:0/1264607696 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdb8406f400 0x7fdb84071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.404+0000 7fdb9e000640 1 --2- 172.21.15.86:0/1264607696 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdb8406f400 0x7fdb84071910 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdb90000f70 tx=0x7fdb90007c30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:12.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.408+0000 7fdb9f400640 1 -- 172.21.15.86:0/1264607696 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdb980c6020 con 0x7fdba806e340 2024-04-25T14:05:12.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.596+0000 7fdbad600640 1 -- 172.21.15.86:0/1264607696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdba815da40 con 0x7fdba806e340 2024-04-25T14:05:12.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.596+0000 7fdb9f400640 1 -- 172.21.15.86:0/1264607696 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5184 (secure 0 0 0) 0x7fdb98016090 con 0x7fdba806e340 2024-04-25T14:05:12.598 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:12.598 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3546153119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3546153119},{"type":"v1","addr":"172.21.15.155:6834","nonce":3546153119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5283,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3499340224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3499340224},{"type":"v1","addr":"172.21.15.86:6836","nonce":3499340224}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5284,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1466978742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1466978742},{"type":"v1","addr":"172.21.15.155:6835","nonce":1466978742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:06.638216+0000","modified":"2024-04-25T14:05:09.664776+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5286":{"gid":5286,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/716923566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":716923566},{"type":"v1","addr":"172.21.15.86:6837","nonce":716923566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5286,"qdb_cluster":[5286]},"id":2}]} 2024-04-25T14:05:12.598 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 18 2024-04-25T14:05:12.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.596+0000 7fdbad600640 1 -- 172.21.15.86:0/1264607696 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdb8406f400 msgr2=0x7fdb84071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.596+0000 7fdbad600640 1 --2- 172.21.15.86:0/1264607696 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdb8406f400 0x7fdb84071910 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdb90000f70 tx=0x7fdb90007c30 comp rx=0 tx=0).stop 2024-04-25T14:05:12.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.596+0000 7fdbad600640 1 -- 172.21.15.86:0/1264607696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdba806e340 msgr2=0x7fdba80c91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.596+0000 7fdbad600640 1 --2- 172.21.15.86:0/1264607696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdba806e340 0x7fdba80c91e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdb98019fb0 tx=0x7fdb9801abc0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.596+0000 7fdbad600640 1 -- 172.21.15.86:0/1264607696 shutdown_connections 2024-04-25T14:05:12.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.596+0000 7fdbad600640 1 --2- 172.21.15.86:0/1264607696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdba8163980 0x7fdba80b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.596+0000 7fdbad600640 1 --2- 172.21.15.86:0/1264607696 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fdb8406f400 0x7fdb84071910 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.596+0000 7fdbad600640 1 --2- 172.21.15.86:0/1264607696 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdba8072630 0x7fdba81fcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.596+0000 7fdbad600640 1 --2- 172.21.15.86:0/1264607696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdba806e340 0x7fdba80c91e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.596+0000 7fdbad600640 1 -- 172.21.15.86:0/1264607696 >> 172.21.15.86:0/1264607696 conn(0x7fdba80d3580 msgr2=0x7fdba8161630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:12.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.596+0000 7fdbad600640 1 -- 172.21.15.86:0/1264607696 shutdown_connections 2024-04-25T14:05:12.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:12.596+0000 7fdbad600640 1 -- 172.21.15.86:0/1264607696 wait complete. 2024-04-25T14:05:13.023 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:05:06.638216+0000', 'modified': '2024-04-25T14:05:09.664776+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5286}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5286': {'gid': 5286, 'name': 'c', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/716923566', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6835', 'nonce': 716923566}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 716923566}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5286, 'qdb_cluster': [5286]} 2024-04-25T14:05:13.023 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T14:05:13.023 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T14:05:13.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.168+0000 7f4b57a00640 1 Processor -- start 2024-04-25T14:05:13.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.168+0000 7f4b57a00640 1 -- start start 2024-04-25T14:05:13.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.172+0000 7f4b57a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b500cec60 0x7f4b500cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.172+0000 7f4b57a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4b500cf5b0 0x7f4b500ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.172+0000 7f4b57a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4b500cf9c0 0x7f4b500cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.172+0000 7f4b57a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4b50163940 con 0x7f4b500cf5b0 2024-04-25T14:05:13.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.172+0000 7f4b57a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f4b500d2f40 con 0x7f4b500cf9c0 2024-04-25T14:05:13.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.172+0000 7f4b57a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4b500b3720 con 0x7f4b500cec60 2024-04-25T14:05:13.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.172+0000 7f4b4fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4b500cf5b0 0x7f4b500ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.172+0000 7f4b4fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4b500cf5b0 0x7f4b500ccd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57574/0 (socket says 172.21.15.86:57574) 2024-04-25T14:05:13.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.172+0000 7f4b4fe00640 1 -- 172.21.15.86:0/3253305897 learned_addr learned my addr 172.21.15.86:0/3253305897 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:13.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.172+0000 7f4b55a00640 1 --2- 172.21.15.86:0/3253305897 >> v2:172.21.15.155:3300/0 conn(0x7f4b500cf9c0 0x7f4b500cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.172+0000 7f4b55000640 1 --2- 172.21.15.86:0/3253305897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b500cec60 0x7f4b500cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.172+0000 7f4b4fe00640 1 -- 172.21.15.86:0/3253305897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b500cec60 msgr2=0x7f4b500cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.172+0000 7f4b4fe00640 1 --2- 172.21.15.86:0/3253305897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b500cec60 0x7f4b500cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.172+0000 7f4b4fe00640 1 -- 172.21.15.86:0/3253305897 >> v2:172.21.15.155:3300/0 conn(0x7f4b500cf9c0 msgr2=0x7f4b500cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.172+0000 7f4b4fe00640 1 --2- 172.21.15.86:0/3253305897 >> v2:172.21.15.155:3300/0 conn(0x7f4b500cf9c0 0x7f4b500cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.172+0000 7f4b4fe00640 1 -- 172.21.15.86:0/3253305897 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b500b3a00 con 0x7f4b500cf5b0 2024-04-25T14:05:13.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.172+0000 7f4b55a00640 1 --2- 172.21.15.86:0/3253305897 >> v2:172.21.15.155:3300/0 conn(0x7f4b500cf9c0 0x7f4b500cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:13.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.176+0000 7f4b4fe00640 1 --2- 172.21.15.86:0/3253305897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b500cf5b0 0x7f4b500ccd90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4b5006e740 tx=0x7f4b3c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98e401b4def13ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:13.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.176+0000 7f4b4f400640 1 -- 172.21.15.86:0/3253305897 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c0ab5f0 con 0x7f4b500cf5b0 2024-04-25T14:05:13.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.176+0000 7f4b4f400640 1 -- 172.21.15.86:0/3253305897 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b3c0b0070 con 0x7f4b500cf5b0 2024-04-25T14:05:13.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.176+0000 7f4b4f400640 1 -- 172.21.15.86:0/3253305897 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c0ab9b0 con 0x7f4b500cf5b0 2024-04-25T14:05:13.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.176+0000 7f4b57a00640 1 -- 172.21.15.86:0/3253305897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b500cf5b0 msgr2=0x7f4b500ccd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.176+0000 7f4b57a00640 1 --2- 172.21.15.86:0/3253305897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b500cf5b0 0x7f4b500ccd90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4b5006e740 tx=0x7f4b3c01c520 comp rx=0 tx=0).stop 2024-04-25T14:05:13.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.176+0000 7f4b57a00640 1 -- 172.21.15.86:0/3253305897 shutdown_connections 2024-04-25T14:05:13.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.176+0000 7f4b57a00640 1 --2- 172.21.15.86:0/3253305897 >> v2:172.21.15.155:3300/0 conn(0x7f4b500cf9c0 0x7f4b500cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.176+0000 7f4b57a00640 1 --2- 172.21.15.86:0/3253305897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b500cf5b0 0x7f4b500ccd90 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.176+0000 7f4b57a00640 1 --2- 172.21.15.86:0/3253305897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b500cec60 0x7f4b500cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.176+0000 7f4b57a00640 1 -- 172.21.15.86:0/3253305897 >> 172.21.15.86:0/3253305897 conn(0x7f4b5009ba30 msgr2=0x7f4b500b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:13.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.176+0000 7f4b57a00640 1 -- 172.21.15.86:0/3253305897 shutdown_connections 2024-04-25T14:05:13.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.176+0000 7f4b57a00640 1 -- 172.21.15.86:0/3253305897 wait complete. 2024-04-25T14:05:13.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.176+0000 7f4b57a00640 1 Processor -- start 2024-04-25T14:05:13.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.176+0000 7f4b57a00640 1 -- start start 2024-04-25T14:05:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.180+0000 7f4b57a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b500cec60 0x7f4b500cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.180+0000 7f4b57a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b500cf5b0 0x7f4b50149c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.180+0000 7f4b57a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b500cf9c0 0x7f4b501617f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.180+0000 7f4b57a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b500d24c0 con 0x7f4b500cf9c0 2024-04-25T14:05:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.180+0000 7f4b57a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4b500d2340 con 0x7f4b500cec60 2024-04-25T14:05:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.180+0000 7f4b57a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4b500d2640 con 0x7f4b500cf5b0 2024-04-25T14:05:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.180+0000 7f4b55a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b500cf9c0 0x7f4b501617f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.180+0000 7f4b55a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b500cf9c0 0x7f4b501617f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57586/0 (socket says 172.21.15.86:57586) 2024-04-25T14:05:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.180+0000 7f4b55a00640 1 -- 172.21.15.86:0/2651864460 learned_addr learned my addr 172.21.15.86:0/2651864460 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.180+0000 7f4b55000640 1 --2- 172.21.15.86:0/2651864460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b500cec60 0x7f4b500cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.180+0000 7f4b55a00640 1 -- 172.21.15.86:0/2651864460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b500cf5b0 msgr2=0x7f4b50149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.180+0000 7f4b4fe00640 1 --2- 172.21.15.86:0/2651864460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b500cf5b0 0x7f4b50149c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.180+0000 7f4b55a00640 1 --2- 172.21.15.86:0/2651864460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b500cf5b0 0x7f4b50149c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.180+0000 7f4b55a00640 1 -- 172.21.15.86:0/2651864460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b500cec60 msgr2=0x7f4b500cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.180+0000 7f4b55a00640 1 --2- 172.21.15.86:0/2651864460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b500cec60 0x7f4b500cb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.180+0000 7f4b55a00640 1 -- 172.21.15.86:0/2651864460 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b500cc450 con 0x7f4b500cf9c0 2024-04-25T14:05:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.180+0000 7f4b55000640 1 --2- 172.21.15.86:0/2651864460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b500cec60 0x7f4b500cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.180+0000 7f4b4fe00640 1 --2- 172.21.15.86:0/2651864460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b500cf5b0 0x7f4b50149c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.180+0000 7f4b55a00640 1 --2- 172.21.15.86:0/2651864460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b500cf9c0 0x7f4b501617f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800cd80 tx=0x7f4b4800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.180+0000 7f4b4cc00640 1 -- 172.21.15.86:0/2651864460 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48003ba0 con 0x7f4b500cf9c0 2024-04-25T14:05:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.180+0000 7f4b4cc00640 1 -- 172.21.15.86:0/2651864460 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b48003d60 con 0x7f4b500cf9c0 2024-04-25T14:05:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.180+0000 7f4b4cc00640 1 -- 172.21.15.86:0/2651864460 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48004260 con 0x7f4b500cf9c0 2024-04-25T14:05:13.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.180+0000 7f4b57a00640 1 -- 172.21.15.86:0/2651864460 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5014a4f0 con 0x7f4b500cf9c0 2024-04-25T14:05:13.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.180+0000 7f4b57a00640 1 -- 172.21.15.86:0/2651864460 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b500cc170 con 0x7f4b500cf9c0 2024-04-25T14:05:13.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.184+0000 7f4b4cc00640 1 -- 172.21.15.86:0/2651864460 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4b48004400 con 0x7f4b500cf9c0 2024-04-25T14:05:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.184+0000 7f4b4cc00640 1 --2- 172.21.15.86:0/2651864460 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f4b3006f400 0x7f4b30071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.184+0000 7f4b4cc00640 1 -- 172.21.15.86:0/2651864460 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f4b4808ff50 con 0x7f4b500cf9c0 2024-04-25T14:05:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.184+0000 7f4b55000640 1 --2- 172.21.15.86:0/2651864460 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f4b3006f400 0x7f4b30071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.184+0000 7f4b5f030000 1 Processor -- start 2024-04-25T14:05:13.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.184+0000 7f4b5f030000 1 -- start start 2024-04-25T14:05:13.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.184+0000 7f4b55000640 1 --2- 172.21.15.86:0/2651864460 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f4b3006f400 0x7f4b30071910 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009710 tx=0x7f4b40008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.184+0000 7f4b5f030000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e4c9448c20 0x55e4c95a6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.184+0000 7f4b5f030000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4c9434ea0 0x55e4c95a7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.184+0000 7f4b5f030000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4c902d280 0x55e4c95a7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.184+0000 7f4b5f030000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e4c9416160 con 0x55e4c9434ea0 2024-04-25T14:05:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.184+0000 7f4b5f030000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e4c93abe20 con 0x55e4c902d280 2024-04-25T14:05:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.184+0000 7f4b5f030000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55e4c9364a20 con 0x55e4c9448c20 2024-04-25T14:05:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.184+0000 7f4b4fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4c9434ea0 0x55e4c95a7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.188+0000 7f4b55000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e4c9448c20 0x55e4c95a6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.188+0000 7f4b4fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4c9434ea0 0x55e4c95a7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57598/0 (socket says 172.21.15.86:57598) 2024-04-25T14:05:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.188+0000 7f4b4fe00640 1 -- 172.21.15.86:0/3086064979 learned_addr learned my addr 172.21.15.86:0/3086064979 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.188+0000 7f4b55a00640 1 --2- 172.21.15.86:0/3086064979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4c902d280 0x55e4c95a7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.188+0000 7f4b4fe00640 1 -- 172.21.15.86:0/3086064979 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e4c9448c20 msgr2=0x55e4c95a6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.188+0000 7f4b4fe00640 1 --2- 172.21.15.86:0/3086064979 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e4c9448c20 0x55e4c95a6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.188+0000 7f4b4fe00640 1 -- 172.21.15.86:0/3086064979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4c902d280 msgr2=0x55e4c95a7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.188+0000 7f4b4fe00640 1 --2- 172.21.15.86:0/3086064979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4c902d280 0x55e4c95a7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.188+0000 7f4b4fe00640 1 -- 172.21.15.86:0/3086064979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4c9369100 con 0x55e4c9434ea0 2024-04-25T14:05:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.188+0000 7f4b4fe00640 1 --2- 172.21.15.86:0/3086064979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4c9434ea0 0x55e4c95a7420 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0ad040 tx=0x7f4b3c0a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48587912af88c7a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.188+0000 7f4b45600640 1 -- 172.21.15.86:0/3086064979 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c0ab8c0 con 0x55e4c9434ea0 2024-04-25T14:05:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.188+0000 7f4b45600640 1 -- 172.21.15.86:0/3086064979 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b3c0b0040 con 0x55e4c9434ea0 2024-04-25T14:05:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.188+0000 7f4b45600640 1 -- 172.21.15.86:0/3086064979 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c0abc80 con 0x55e4c9434ea0 2024-04-25T14:05:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.188+0000 7f4b5f030000 1 -- 172.21.15.86:0/3086064979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4c9434ea0 msgr2=0x55e4c95a7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.188+0000 7f4b5f030000 1 --2- 172.21.15.86:0/3086064979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4c9434ea0 0x55e4c95a7420 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0ad040 tx=0x7f4b3c0a7000 comp rx=0 tx=0).stop 2024-04-25T14:05:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.188+0000 7f4b5f030000 1 -- 172.21.15.86:0/3086064979 shutdown_connections 2024-04-25T14:05:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.188+0000 7f4b5f030000 1 --2- 172.21.15.86:0/3086064979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4c902d280 0x55e4c95a7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.188+0000 7f4b5f030000 1 --2- 172.21.15.86:0/3086064979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4c9434ea0 0x55e4c95a7420 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.188+0000 7f4b5f030000 1 --2- 172.21.15.86:0/3086064979 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e4c9448c20 0x55e4c95a6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.188+0000 7f4b5f030000 1 -- 172.21.15.86:0/3086064979 >> 172.21.15.86:0/3086064979 conn(0x55e4c906cff0 msgr2=0x55e4c955c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.188+0000 7f4b5f030000 1 -- 172.21.15.86:0/3086064979 shutdown_connections 2024-04-25T14:05:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.188+0000 7f4b5f030000 1 -- 172.21.15.86:0/3086064979 wait complete. 2024-04-25T14:05:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.188+0000 7f4b5f030000 1 -- start start 2024-04-25T14:05:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.188+0000 7f4b5f030000 1 Processor -- start 2024-04-25T14:05:13.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b5f030000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4c902d280 0x55e4c95ade60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b5f030000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e4c9434ea0 0x55e4c95ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b5f030000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4c9448c20 0x55e4c95690c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b5f030000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e4c93b9220 con 0x55e4c902d280 2024-04-25T14:05:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b5f030000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e4c934fea0 con 0x55e4c9448c20 2024-04-25T14:05:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b5f030000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55e4c93a6b00 con 0x55e4c9434ea0 2024-04-25T14:05:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b5f030000 10 client.0 _ll_register_callbacks cb 0x55e4c93b8dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b55000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4c902d280 0x55e4c95ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b55000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4c902d280 0x55e4c95ade60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57608/0 (socket says 172.21.15.86:57608) 2024-04-25T14:05:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b55000640 1 -- 172.21.15.86:0/581408747 learned_addr learned my addr 172.21.15.86:0/581408747 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b4fe00640 1 --2- 172.21.15.86:0/581408747 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e4c9434ea0 0x55e4c95ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b55a00640 1 --2- 172.21.15.86:0/581408747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4c9448c20 0x55e4c95690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b55000640 1 -- 172.21.15.86:0/581408747 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e4c9434ea0 msgr2=0x55e4c95ae3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b55000640 1 --2- 172.21.15.86:0/581408747 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e4c9434ea0 0x55e4c95ae3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b55000640 1 -- 172.21.15.86:0/581408747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4c9448c20 msgr2=0x55e4c95690c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b55000640 1 --2- 172.21.15.86:0/581408747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4c9448c20 0x55e4c95690c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b55000640 1 -- 172.21.15.86:0/581408747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4c934b500 con 0x55e4c902d280 2024-04-25T14:05:13.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b55000640 1 --2- 172.21.15.86:0/581408747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4c902d280 0x55e4c95ade60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000ef20 tx=0x7f4b400046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=253f56a22da72ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:13.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b3a000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:05:13.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b3a000640 1 -- 172.21.15.86:0/581408747 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4000f040 con 0x55e4c902d280 2024-04-25T14:05:13.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b3a000640 1 -- 172.21.15.86:0/581408747 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f4b40015d20 con 0x55e4c902d280 2024-04-25T14:05:13.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b5f030000 1 -- 172.21.15.86:0/581408747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b3c0b0040 con 0x55e4c902d280 2024-04-25T14:05:13.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b3a000640 1 -- 172.21.15.86:0/581408747 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f4b4002e050 con 0x55e4c902d280 2024-04-25T14:05:13.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b5f030000 10 client.5487 fetch_fsmap learned FSMap version 18 2024-04-25T14:05:13.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b5f030000 1 -- 172.21.15.86:0/581408747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55e4c9331ea0 con 0x55e4c902d280 2024-04-25T14:05:13.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b3a000640 1 -- 172.21.15.86:0/581408747 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b3c0b0040 con 0x55e4c902d280 2024-04-25T14:05:13.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b3a000640 1 -- 172.21.15.86:0/581408747 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f4b40031020 con 0x55e4c902d280 2024-04-25T14:05:13.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b5f030000 10 client.5487 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T14:05:13.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b5f030000 10 client.5487 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T14:05:13.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b5f030000 20 client.5487 populate_metadata read hostname 'smithi086' 2024-04-25T14:05:13.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b5f030000 1 --2- 172.21.15.86:0/581408747 >> [v2:172.21.15.86:6835/716923566,v1:172.21.15.86:6837/716923566] conn(0x55e4c90517d0 0x55e4c9569e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b5f030000 4 client.5487 mds_command: new command op to 5286 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:13.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b5f030000 1 -- 172.21.15.86:0/581408747 --> [v2:172.21.15.86:6835/716923566,v1:172.21.15.86:6837/716923566] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4c9344be0 con 0x55e4c90517d0 2024-04-25T14:05:13.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b4fe00640 1 --2- 172.21.15.86:0/581408747 >> [v2:172.21.15.86:6835/716923566,v1:172.21.15.86:6837/716923566] conn(0x55e4c90517d0 0x55e4c9569e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b4fe00640 1 --2- 172.21.15.86:0/581408747 >> [v2:172.21.15.86:6835/716923566,v1:172.21.15.86:6837/716923566] conn(0x55e4c90517d0 0x55e4c9569e00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c474df109871d server_cookie=88167535d47a9919 in_seq=0 out_seq=0 2024-04-25T14:05:13.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.192+0000 7f4b3a000640 10 client.5487 ms_handle_connect on v2:172.21.15.86:6835/716923566 2024-04-25T14:05:13.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.196+0000 7f4b3a000640 1 -- 172.21.15.86:0/581408747 <== mds.0 v2:172.21.15.86:6835/716923566 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e4c9344be0 con 0x55e4c90517d0 2024-04-25T14:05:13.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.196+0000 7f4b3a000640 10 client.5487 handle_command_reply: tid=0 2024-04-25T14:05:13.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.196+0000 7f4b5f030000 1 client.5487 shutdown 2024-04-25T14:05:13.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.196+0000 7f4b5f030000 1 -- 172.21.15.86:0/581408747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4c902d280 msgr2=0x55e4c95ade60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.196+0000 7f4b5f030000 1 --2- 172.21.15.86:0/581408747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4c902d280 0x55e4c95ade60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000ef20 tx=0x7f4b400046d0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.196+0000 7f4b5f030000 1 -- 172.21.15.86:0/581408747 shutdown_connections 2024-04-25T14:05:13.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.196+0000 7f4b5f030000 1 --2- 172.21.15.86:0/581408747 >> [v2:172.21.15.86:6835/716923566,v1:172.21.15.86:6837/716923566] conn(0x55e4c90517d0 0x55e4c9569e00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.196+0000 7f4b5f030000 1 --2- 172.21.15.86:0/581408747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4c9448c20 0x55e4c95690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.196+0000 7f4b5f030000 1 --2- 172.21.15.86:0/581408747 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e4c9434ea0 0x55e4c95ae3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.196+0000 7f4b5f030000 1 --2- 172.21.15.86:0/581408747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4c902d280 0x55e4c95ade60 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.196+0000 7f4b3a000640 0 client.5487 ms_handle_reset on v2:172.21.15.86:6835/716923566 2024-04-25T14:05:13.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.196+0000 7f4b5f030000 1 -- 172.21.15.86:0/581408747 >> 172.21.15.86:0/581408747 conn(0x55e4c906cff0 msgr2=0x55e4c955cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:13.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.196+0000 7f4b5f030000 1 -- 172.21.15.86:0/581408747 shutdown_connections 2024-04-25T14:05:13.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.196+0000 7f4b5f030000 1 -- 172.21.15.86:0/581408747 wait complete. 2024-04-25T14:05:13.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.196+0000 7f4b5f030000 20 client.5487 trim_cache size 0 max 16384 2024-04-25T14:05:13.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.212+0000 7f4b5f030000 1 Processor -- start 2024-04-25T14:05:13.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.212+0000 7f4b5f030000 1 -- start start 2024-04-25T14:05:13.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b5f030000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4c902d280 0x55e4c95b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b5f030000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4c9434ea0 0x55e4c95b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b5f030000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e4c9448c20 0x55e4c95bcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b5f030000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b50205930 con 0x55e4c9434ea0 2024-04-25T14:05:13.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b5f030000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5014bbb0 con 0x55e4c902d280 2024-04-25T14:05:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b5f030000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4b50205300 con 0x55e4c9448c20 2024-04-25T14:05:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b4fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4c9434ea0 0x55e4c95b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b4fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4c9434ea0 0x55e4c95b33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57624/0 (socket says 172.21.15.86:57624) 2024-04-25T14:05:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b4fe00640 1 -- 172.21.15.86:0/3969280439 learned_addr learned my addr 172.21.15.86:0/3969280439 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b55a00640 1 --2- 172.21.15.86:0/3969280439 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e4c9448c20 0x55e4c95bcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b4fe00640 1 -- 172.21.15.86:0/3969280439 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e4c9448c20 msgr2=0x55e4c95bcff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b55000640 1 --2- 172.21.15.86:0/3969280439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4c902d280 0x55e4c95b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b4fe00640 1 --2- 172.21.15.86:0/3969280439 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e4c9448c20 0x55e4c95bcff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b4fe00640 1 -- 172.21.15.86:0/3969280439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4c902d280 msgr2=0x55e4c95b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b4fe00640 1 --2- 172.21.15.86:0/3969280439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4c902d280 0x55e4c95b2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b4fe00640 1 -- 172.21.15.86:0/3969280439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4c9569430 con 0x55e4c9434ea0 2024-04-25T14:05:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b55a00640 1 --2- 172.21.15.86:0/3969280439 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e4c9448c20 0x55e4c95bcff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b55000640 1 --2- 172.21.15.86:0/3969280439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4c902d280 0x55e4c95b2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:13.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b4fe00640 1 --2- 172.21.15.86:0/3969280439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4c9434ea0 0x55e4c95b33e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c008010 tx=0x7f4b3c0a9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d642d805a3e0ae5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:13.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b44c00640 1 -- 172.21.15.86:0/3969280439 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c0044c0 con 0x55e4c9434ea0 2024-04-25T14:05:13.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b44c00640 1 -- 172.21.15.86:0/3969280439 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b3c0b0040 con 0x55e4c9434ea0 2024-04-25T14:05:13.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b44c00640 1 -- 172.21.15.86:0/3969280439 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c0ab8c0 con 0x55e4c9434ea0 2024-04-25T14:05:13.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b5f030000 1 -- 172.21.15.86:0/3969280439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4c9434ea0 msgr2=0x55e4c95b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b5f030000 1 --2- 172.21.15.86:0/3969280439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4c9434ea0 0x55e4c95b33e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c008010 tx=0x7f4b3c0a9000 comp rx=0 tx=0).stop 2024-04-25T14:05:13.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b5f030000 1 -- 172.21.15.86:0/3969280439 shutdown_connections 2024-04-25T14:05:13.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b5f030000 1 --2- 172.21.15.86:0/3969280439 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e4c9448c20 0x55e4c95bcff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b5f030000 1 --2- 172.21.15.86:0/3969280439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4c9434ea0 0x55e4c95b33e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b5f030000 1 --2- 172.21.15.86:0/3969280439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4c902d280 0x55e4c95b2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b5f030000 1 -- 172.21.15.86:0/3969280439 >> 172.21.15.86:0/3969280439 conn(0x55e4c906cff0 msgr2=0x55e4c9564040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:13.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b5f030000 1 -- 172.21.15.86:0/3969280439 shutdown_connections 2024-04-25T14:05:13.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b5f030000 1 -- 172.21.15.86:0/3969280439 wait complete. 2024-04-25T14:05:13.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b5f030000 1 -- start start 2024-04-25T14:05:13.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.216+0000 7f4b5f030000 1 Processor -- start 2024-04-25T14:05:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b5f030000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4c902d280 0x55e4c9619170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b5f030000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4c9434ea0 0x55e4c96196b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b5f030000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e4c9448c20 0x55e4c95b1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b5f030000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e4c93ab130 con 0x55e4c9434ea0 2024-04-25T14:05:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b5f030000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e4c93417d0 con 0x55e4c902d280 2024-04-25T14:05:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b5f030000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55e4c93b0450 con 0x55e4c9448c20 2024-04-25T14:05:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b5f030000 10 client.0 _ll_register_callbacks cb 0x7f4b4000e6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b4fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4c9434ea0 0x55e4c96196b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b4fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4c9434ea0 0x55e4c96196b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57636/0 (socket says 172.21.15.86:57636) 2024-04-25T14:05:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b4fe00640 1 -- 172.21.15.86:0/4183717680 learned_addr learned my addr 172.21.15.86:0/4183717680 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b55000640 1 --2- 172.21.15.86:0/4183717680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4c902d280 0x55e4c9619170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b55a00640 1 --2- 172.21.15.86:0/4183717680 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e4c9448c20 0x55e4c95b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b4fe00640 1 -- 172.21.15.86:0/4183717680 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e4c9448c20 msgr2=0x55e4c95b1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b4fe00640 1 --2- 172.21.15.86:0/4183717680 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e4c9448c20 0x55e4c95b1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b4fe00640 1 -- 172.21.15.86:0/4183717680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4c902d280 msgr2=0x55e4c9619170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b4fe00640 1 --2- 172.21.15.86:0/4183717680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4c902d280 0x55e4c9619170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b4fe00640 1 -- 172.21.15.86:0/4183717680 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4c9364c70 con 0x55e4c9434ea0 2024-04-25T14:05:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b55a00640 1 --2- 172.21.15.86:0/4183717680 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e4c9448c20 0x55e4c95b1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b4fe00640 1 --2- 172.21.15.86:0/4183717680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4c9434ea0 0x55e4c96196b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c01abd0 tx=0x7f4b3c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c3865ee049cd51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b3a000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:05:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b3a000640 1 -- 172.21.15.86:0/4183717680 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c004480 con 0x55e4c9434ea0 2024-04-25T14:05:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b3a000640 1 -- 172.21.15.86:0/4183717680 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f4b3c0c7410 con 0x55e4c9434ea0 2024-04-25T14:05:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b5f030000 1 -- 172.21.15.86:0/4183717680 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b3c0b0040 con 0x55e4c9434ea0 2024-04-25T14:05:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b3a000640 1 -- 172.21.15.86:0/4183717680 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f4b3c0d8050 con 0x55e4c9434ea0 2024-04-25T14:05:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b5f030000 10 client.5493 fetch_fsmap learned FSMap version 18 2024-04-25T14:05:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b5f030000 1 -- 172.21.15.86:0/4183717680 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55e4c942e180 con 0x55e4c9434ea0 2024-04-25T14:05:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b3a000640 1 -- 172.21.15.86:0/4183717680 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b3c0b0040 con 0x55e4c9434ea0 2024-04-25T14:05:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b3a000640 1 -- 172.21.15.86:0/4183717680 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f4b3c0db020 con 0x55e4c9434ea0 2024-04-25T14:05:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b5f030000 10 client.5493 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T14:05:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b5f030000 10 client.5493 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T14:05:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b5f030000 20 client.5493 populate_metadata read hostname 'smithi086' 2024-04-25T14:05:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b5f030000 1 --2- 172.21.15.86:0/4183717680 >> [v2:172.21.15.86:6835/716923566,v1:172.21.15.86:6837/716923566] conn(0x55e4c903f830 0x55e4c95b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b5f030000 4 client.5493 mds_command: new command op to 5286 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:05:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b5f030000 1 -- 172.21.15.86:0/4183717680 --> [v2:172.21.15.86:6835/716923566,v1:172.21.15.86:6837/716923566] -- command(tid 0: {"prefix": "status"}) -- 0x55e4c93b9a30 con 0x55e4c903f830 2024-04-25T14:05:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b55000640 1 --2- 172.21.15.86:0/4183717680 >> [v2:172.21.15.86:6835/716923566,v1:172.21.15.86:6837/716923566] conn(0x55e4c903f830 0x55e4c95b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b3a000640 10 client.5493 ms_handle_connect on v2:172.21.15.86:6835/716923566 2024-04-25T14:05:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.220+0000 7f4b55000640 1 --2- 172.21.15.86:0/4183717680 >> [v2:172.21.15.86:6835/716923566,v1:172.21.15.86:6837/716923566] conn(0x55e4c903f830 0x55e4c95b22e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53c7516f364af544 server_cookie=445ec91ac44ffde3 in_seq=0 out_seq=0 2024-04-25T14:05:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.224+0000 7f4b3a000640 1 -- 172.21.15.86:0/4183717680 <== mds.0 v2:172.21.15.86:6835/716923566 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55e4c93b9a30 con 0x55e4c903f830 2024-04-25T14:05:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.224+0000 7f4b3a000640 10 client.5493 handle_command_reply: tid=0 2024-04-25T14:05:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.224+0000 7f4b5f030000 1 client.5493 shutdown 2024-04-25T14:05:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.224+0000 7f4b5f030000 1 -- 172.21.15.86:0/4183717680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4c9434ea0 msgr2=0x55e4c96196b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.224+0000 7f4b5f030000 1 --2- 172.21.15.86:0/4183717680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4c9434ea0 0x55e4c96196b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c01abd0 tx=0x7f4b3c01ad60 comp rx=0 tx=0).stop 2024-04-25T14:05:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.224+0000 7f4b5f030000 1 -- 172.21.15.86:0/4183717680 shutdown_connections 2024-04-25T14:05:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.224+0000 7f4b5f030000 1 --2- 172.21.15.86:0/4183717680 >> [v2:172.21.15.86:6835/716923566,v1:172.21.15.86:6837/716923566] conn(0x55e4c903f830 0x55e4c95b22e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.224+0000 7f4b5f030000 1 --2- 172.21.15.86:0/4183717680 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e4c9448c20 0x55e4c95b1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.224+0000 7f4b5f030000 1 --2- 172.21.15.86:0/4183717680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4c9434ea0 0x55e4c96196b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.224+0000 7f4b3a000640 0 client.5493 ms_handle_reset on v2:172.21.15.86:6835/716923566 2024-04-25T14:05:13.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.224+0000 7f4b5f030000 1 --2- 172.21.15.86:0/4183717680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e4c902d280 0x55e4c9619170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.224+0000 7f4b5f030000 1 -- 172.21.15.86:0/4183717680 >> 172.21.15.86:0/4183717680 conn(0x55e4c906cff0 msgr2=0x55e4c95647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:13.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.224+0000 7f4b5f030000 1 -- 172.21.15.86:0/4183717680 shutdown_connections 2024-04-25T14:05:13.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.224+0000 7f4b5f030000 1 -- 172.21.15.86:0/4183717680 wait complete. 2024-04-25T14:05:13.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.224+0000 7f4b5f030000 20 client.5493 trim_cache size 0 max 16384 2024-04-25T14:05:13.229 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:05:13.229 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb", 2024-04-25T14:05:13.229 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-25T14:05:13.229 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 5286, 2024-04-25T14:05:13.230 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T14:05:13.230 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T14:05:13.230 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T14:05:13.230 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 5.9626370550000001, 2024-04-25T14:05:13.230 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 18, 2024-04-25T14:05:13.230 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 38, 2024-04-25T14:05:13.230 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 38, 2024-04-25T14:05:13.230 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 13.824675072 2024-04-25T14:05:13.230 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:05:13.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.224+0000 7f4b57a00640 1 -- 172.21.15.86:0/2651864460 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f4b3006f400 msgr2=0x7f4b30071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.224+0000 7f4b57a00640 1 --2- 172.21.15.86:0/2651864460 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f4b3006f400 0x7f4b30071910 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009710 tx=0x7f4b40008040 comp rx=0 tx=0).stop 2024-04-25T14:05:13.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.224+0000 7f4b57a00640 1 -- 172.21.15.86:0/2651864460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b500cf9c0 msgr2=0x7f4b501617f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.224+0000 7f4b57a00640 1 --2- 172.21.15.86:0/2651864460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b500cf9c0 0x7f4b501617f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800cd80 tx=0x7f4b4800a650 comp rx=0 tx=0).stop 2024-04-25T14:05:13.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.224+0000 7f4b57a00640 1 -- 172.21.15.86:0/2651864460 shutdown_connections 2024-04-25T14:05:13.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.224+0000 7f4b57a00640 1 --2- 172.21.15.86:0/2651864460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b500cf9c0 0x7f4b501617f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.224+0000 7f4b57a00640 1 --2- 172.21.15.86:0/2651864460 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f4b3006f400 0x7f4b30071910 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.224+0000 7f4b57a00640 1 --2- 172.21.15.86:0/2651864460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b500cf5b0 0x7f4b50149c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.224+0000 7f4b57a00640 1 --2- 172.21.15.86:0/2651864460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b500cec60 0x7f4b500cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.224+0000 7f4b57a00640 1 -- 172.21.15.86:0/2651864460 >> 172.21.15.86:0/2651864460 conn(0x7f4b5009ba30 msgr2=0x7f4b5006c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:13.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.224+0000 7f4b57a00640 1 -- 172.21.15.86:0/2651864460 shutdown_connections 2024-04-25T14:05:13.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.224+0000 7f4b57a00640 1 -- 172.21.15.86:0/2651864460 wait complete. 2024-04-25T14:05:13.245 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T14:05:13.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d62000640 1 Processor -- start 2024-04-25T14:05:13.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d62000640 1 -- start start 2024-04-25T14:05:13.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d62000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d5c0cdd70 0x7f1d5c0ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d62000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1d5c070e20 0x7f1d5c071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d62000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d5c071770 con 0x7f1d5c06e700 2024-04-25T14:05:13.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d62000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1d5c06d940 con 0x7f1d5c070e20 2024-04-25T14:05:13.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d62000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1d5c06dc20 con 0x7f1d5c0cdd70 2024-04-25T14:05:13.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d5be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1d5c06e700 0x7f1d5c06eb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37220/0 (socket says 172.21.15.86:37220) 2024-04-25T14:05:13.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d5be00640 1 -- 172.21.15.86:0/1013080364 learned_addr learned my addr 172.21.15.86:0/1013080364 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:13.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d5aa00640 1 --2- 172.21.15.86:0/1013080364 >> v2:172.21.15.155:3300/0 conn(0x7f1d5c070e20 0x7f1d5c071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d5b400640 1 --2- 172.21.15.86:0/1013080364 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d5c0cdd70 0x7f1d5c0ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d5a000640 1 -- 172.21.15.86:0/1013080364 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1488857038 0 0) 0x7f1d540014a0 con 0x7f1d5c06e700 2024-04-25T14:05:13.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d5a000640 1 -- 172.21.15.86:0/1013080364 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d40003530 con 0x7f1d5c06e700 2024-04-25T14:05:13.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d5a000640 1 -- 172.21.15.86:0/1013080364 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2391494867 0 0) 0x7f1d40003530 con 0x7f1d5c06e700 2024-04-25T14:05:13.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d5a000640 1 -- 172.21.15.86:0/1013080364 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d5c0cdd70 msgr2=0x7f1d5c0ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d5a000640 1 --2- 172.21.15.86:0/1013080364 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d5c0cdd70 0x7f1d5c0ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d5a000640 1 -- 172.21.15.86:0/1013080364 >> v2:172.21.15.155:3300/0 conn(0x7f1d5c070e20 msgr2=0x7f1d5c071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d5a000640 1 --2- 172.21.15.86:0/1013080364 >> v2:172.21.15.155:3300/0 conn(0x7f1d5c070e20 0x7f1d5c071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d5a000640 1 -- 172.21.15.86:0/1013080364 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d5c06df00 con 0x7f1d5c06e700 2024-04-25T14:05:13.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d5a000640 1 -- 172.21.15.86:0/1013080364 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f1d54001bf0 con 0x7f1d5c06e700 2024-04-25T14:05:13.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d5a000640 1 -- 172.21.15.86:0/1013080364 >> v1:172.21.15.86:6789/0 conn(0x7f1d5c06e700 legacy=0x7f1d5c06eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d5a000640 1 --2- 172.21.15.86:0/1013080364 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d5c0cdd70 0x7f1d40003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d5a000640 1 --2- 172.21.15.86:0/1013080364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d40003fb0 0x7f1d40004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d5a000640 1 --2- 172.21.15.86:0/1013080364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d40004d50 0x7f1d40005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d5a000640 1 -- 172.21.15.86:0/1013080364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c06d940 con 0x7f1d40003fb0 2024-04-25T14:05:13.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d5a000640 1 -- 172.21.15.86:0/1013080364 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c06dc20 con 0x7f1d40004d50 2024-04-25T14:05:13.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d5a000640 1 -- 172.21.15.86:0/1013080364 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1d4000bb30 con 0x7f1d5c0cdd70 2024-04-25T14:05:13.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d5aa00640 1 --2- 172.21.15.86:0/1013080364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d40003fb0 0x7f1d40004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d5b400640 1 --2- 172.21.15.86:0/1013080364 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d5c0cdd70 0x7f1d40003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.392+0000 7f1d5be00640 1 --2- 172.21.15.86:0/1013080364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d40004d50 0x7f1d40005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.396+0000 7f1d5aa00640 1 -- 172.21.15.86:0/1013080364 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d5c0cdd70 msgr2=0x7f1d40003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.396+0000 7f1d5aa00640 1 --2- 172.21.15.86:0/1013080364 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d5c0cdd70 0x7f1d40003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.396+0000 7f1d5aa00640 1 -- 172.21.15.86:0/1013080364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d40004d50 msgr2=0x7f1d40005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.396+0000 7f1d5aa00640 1 --2- 172.21.15.86:0/1013080364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d40004d50 0x7f1d40005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.396+0000 7f1d5aa00640 1 -- 172.21.15.86:0/1013080364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d40003530 con 0x7f1d40003fb0 2024-04-25T14:05:13.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.396+0000 7f1d5aa00640 1 --2- 172.21.15.86:0/1013080364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d40003fb0 0x7f1d40004810 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1d48009ac0 tx=0x7f1d4801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ea97f04f920b27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:13.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.396+0000 7f1d5a000640 1 -- 172.21.15.86:0/1013080364 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d480ac070 con 0x7f1d40003fb0 2024-04-25T14:05:13.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.396+0000 7f1d5a000640 1 -- 172.21.15.86:0/1013080364 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d480059a0 con 0x7f1d40003fb0 2024-04-25T14:05:13.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.396+0000 7f1d5a000640 1 -- 172.21.15.86:0/1013080364 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48009690 con 0x7f1d40003fb0 2024-04-25T14:05:13.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.396+0000 7f1d62000640 1 -- 172.21.15.86:0/1013080364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d40003fb0 msgr2=0x7f1d40004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.396+0000 7f1d62000640 1 --2- 172.21.15.86:0/1013080364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d40003fb0 0x7f1d40004810 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1d48009ac0 tx=0x7f1d4801ca90 comp rx=0 tx=0).stop 2024-04-25T14:05:13.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.396+0000 7f1d5be00640 1 -- 172.21.15.86:0/1013080364 reap_dead start 2024-04-25T14:05:13.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.400+0000 7f1d62000640 1 -- 172.21.15.86:0/1013080364 shutdown_connections 2024-04-25T14:05:13.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.400+0000 7f1d62000640 1 -- 172.21.15.86:0/1013080364 >> 172.21.15.86:0/1013080364 conn(0x7f1d5c0cffc0 msgr2=0x7f1d5c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:13.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.400+0000 7f1d62000640 1 -- 172.21.15.86:0/1013080364 shutdown_connections 2024-04-25T14:05:13.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.400+0000 7f1d62000640 1 -- 172.21.15.86:0/1013080364 wait complete. 2024-04-25T14:05:13.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.400+0000 7f1d62000640 1 Processor -- start 2024-04-25T14:05:13.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.400+0000 7f1d62000640 1 -- start start 2024-04-25T14:05:13.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.400+0000 7f1d62000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d40003fb0 0x7f1d5c15cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.400+0000 7f1d62000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d5c1699c0 0x7f1d5c0cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.400+0000 7f1d62000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d5c0cbd90 0x7f1d5c0ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.404+0000 7f1d62000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c16abf0 con 0x7f1d5c1699c0 2024-04-25T14:05:13.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.404+0000 7f1d62000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c164380 con 0x7f1d40003fb0 2024-04-25T14:05:13.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.404+0000 7f1d62000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1d5c16ad70 con 0x7f1d5c0cbd90 2024-04-25T14:05:13.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.404+0000 7f1d5aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d5c1699c0 0x7f1d5c0cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.404+0000 7f1d5aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d5c1699c0 0x7f1d5c0cb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57648/0 (socket says 172.21.15.86:57648) 2024-04-25T14:05:13.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.404+0000 7f1d5aa00640 1 -- 172.21.15.86:0/3211369306 learned_addr learned my addr 172.21.15.86:0/3211369306 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:13.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.404+0000 7f1d5b400640 1 --2- 172.21.15.86:0/3211369306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d40003fb0 0x7f1d5c15cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.404+0000 7f1d5be00640 1 --2- 172.21.15.86:0/3211369306 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d5c0cbd90 0x7f1d5c0ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.404+0000 7f1d5aa00640 1 -- 172.21.15.86:0/3211369306 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d5c0cbd90 msgr2=0x7f1d5c0ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.404+0000 7f1d5aa00640 1 --2- 172.21.15.86:0/3211369306 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d5c0cbd90 0x7f1d5c0ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.404+0000 7f1d5aa00640 1 -- 172.21.15.86:0/3211369306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d40003fb0 msgr2=0x7f1d5c15cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.404+0000 7f1d5aa00640 1 --2- 172.21.15.86:0/3211369306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d40003fb0 0x7f1d5c15cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.404+0000 7f1d5aa00640 1 -- 172.21.15.86:0/3211369306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d5c0cc1a0 con 0x7f1d5c1699c0 2024-04-25T14:05:13.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.404+0000 7f1d5aa00640 1 --2- 172.21.15.86:0/3211369306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d5c1699c0 0x7f1d5c0cb850 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1d4801cad0 tx=0x7f1d40005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:13.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.404+0000 7f1d53400640 1 -- 172.21.15.86:0/3211369306 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d480ac040 con 0x7f1d5c1699c0 2024-04-25T14:05:13.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.404+0000 7f1d53400640 1 -- 172.21.15.86:0/3211369306 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d48004020 con 0x7f1d5c1699c0 2024-04-25T14:05:13.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.404+0000 7f1d53400640 1 -- 172.21.15.86:0/3211369306 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48004410 con 0x7f1d5c1699c0 2024-04-25T14:05:13.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.404+0000 7f1d62000640 1 -- 172.21.15.86:0/3211369306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d5c167680 con 0x7f1d5c1699c0 2024-04-25T14:05:13.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.404+0000 7f1d62000640 1 -- 172.21.15.86:0/3211369306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d5c167c40 con 0x7f1d5c1699c0 2024-04-25T14:05:13.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.404+0000 7f1d62000640 1 -- 172.21.15.86:0/3211369306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d5c0720e0 con 0x7f1d5c1699c0 2024-04-25T14:05:13.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.404+0000 7f1d53400640 1 -- 172.21.15.86:0/3211369306 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1d480ba020 con 0x7f1d5c1699c0 2024-04-25T14:05:13.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.408+0000 7f1d53400640 1 --2- 172.21.15.86:0/3211369306 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1d3c06f400 0x7f1d3c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.408+0000 7f1d53400640 1 -- 172.21.15.86:0/3211369306 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f1d48128980 con 0x7f1d5c1699c0 2024-04-25T14:05:13.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.408+0000 7f1d5b400640 1 --2- 172.21.15.86:0/3211369306 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1d3c06f400 0x7f1d3c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.408+0000 7f1d5b400640 1 --2- 172.21.15.86:0/3211369306 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1d3c06f400 0x7f1d3c071910 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c009b40 tx=0x7f1d4c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:13.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.412+0000 7f1d53400640 1 -- 172.21.15.86:0/3211369306 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1d5c0cc1a0 con 0x7f1d5c1699c0 2024-04-25T14:05:13.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.604+0000 7f1d62000640 1 -- 172.21.15.86:0/3211369306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1d5c076bf0 con 0x7f1d5c1699c0 2024-04-25T14:05:13.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.604+0000 7f1d53400640 1 -- 172.21.15.86:0/3211369306 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7f1d480167a0 con 0x7f1d5c1699c0 2024-04-25T14:05:13.605 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T14:05:13.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.604+0000 7f1d62000640 1 -- 172.21.15.86:0/3211369306 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1d3c06f400 msgr2=0x7f1d3c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.604+0000 7f1d62000640 1 --2- 172.21.15.86:0/3211369306 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1d3c06f400 0x7f1d3c071910 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c009b40 tx=0x7f1d4c009190 comp rx=0 tx=0).stop 2024-04-25T14:05:13.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.604+0000 7f1d62000640 1 -- 172.21.15.86:0/3211369306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d5c1699c0 msgr2=0x7f1d5c0cb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.604+0000 7f1d62000640 1 --2- 172.21.15.86:0/3211369306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d5c1699c0 0x7f1d5c0cb850 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1d4801cad0 tx=0x7f1d40005840 comp rx=0 tx=0).stop 2024-04-25T14:05:13.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.604+0000 7f1d62000640 1 -- 172.21.15.86:0/3211369306 shutdown_connections 2024-04-25T14:05:13.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.604+0000 7f1d62000640 1 --2- 172.21.15.86:0/3211369306 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1d3c06f400 0x7f1d3c071910 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.604+0000 7f1d62000640 1 --2- 172.21.15.86:0/3211369306 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d5c0cbd90 0x7f1d5c0ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.604+0000 7f1d62000640 1 --2- 172.21.15.86:0/3211369306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d5c1699c0 0x7f1d5c0cb850 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.604+0000 7f1d62000640 1 --2- 172.21.15.86:0/3211369306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d40003fb0 0x7f1d5c15cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.604+0000 7f1d62000640 1 -- 172.21.15.86:0/3211369306 >> 172.21.15.86:0/3211369306 conn(0x7f1d5c0cffc0 msgr2=0x7f1d5c16a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:13.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.604+0000 7f1d62000640 1 -- 172.21.15.86:0/3211369306 shutdown_connections 2024-04-25T14:05:13.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13.604+0000 7f1d62000640 1 -- 172.21.15.86:0/3211369306 wait complete. 2024-04-25T14:05:13.626 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T14:05:13.626 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T14:05:13.626 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-25T14:05:13.626 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T14:05:13.626 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T14:05:13.627 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T14:05:13.627 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T14:05:13.627 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T14:05:13.627 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:05:13.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:13.627 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-25T14:05:13.636 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:05:13.636 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:05:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:05:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:05:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-25T14:05:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:05:13.637 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:05:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:05:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-25T14:05:13.637 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:05:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:05:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-25T14:05:13.637 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:05:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:05:13.637 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T14:05:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:05:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T14:05:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42351sec preferred_lft 42351sec 2024-04-25T14:05:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-25T14:05:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:05:13.638 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T14:05:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:dc:87:9c:96 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:05:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T14:05:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:05:13.638 INFO:teuthology.orchestra.run.smithi086.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T14:05:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T14:05:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T14:05:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:05:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T14:05:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:05:13.639 INFO:teuthology.orchestra.run.smithi086.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:05:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T14:05:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T14:05:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:05:13.639 INFO:teuthology.orchestra.run.smithi086.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:05:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T14:05:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T14:05:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:05:13.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:13.640 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T14:05:13.697 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-25T14:05:13.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:13.699 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-25T14:05:13.751 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-25T14:05:13.751 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T14:05:13.751 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T14:05:13.752 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T14:05:13.752 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T14:05:13.752 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T14:05:13.752 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T14:05:13.752 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:05:13.752 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:05:13.752 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:05:13.752 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T14:05:13.752 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:05:13.752 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T14:05:13.752 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T14:05:13.753 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T14:05:13.753 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T14:05:13.753 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T14:05:13.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:05:13.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:05:13.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:13.886 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T14:05:13.937 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T14:05:13.937 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T14:05:13.937 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:05:13.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:13.937 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-25T14:05:14.004 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:05:14.004 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T14:05:14.052 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:05:14.052 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:05:14.052 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T14:05:14.107 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T14:05:14.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:14.108 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:05:14.211 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:05:14.211 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T14:05:14.211 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:05:14.228 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:05:14.229 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T14:05:14.229 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:05:14.229 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:05:14.229 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:05:14.239 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:05:14.239 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T14:05:14.239 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:05:14.239 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:05:14.239 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:05:14.239 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790" 2024-04-25T14:05:14.239 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790:/ 2024-04-25T14:05:14.239 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb" will pass to kernel 2024-04-25T14:05:14.243 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T14:05:14.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:14.243 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T14:05:14.265 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:05:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:05:14.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:05:14.374 INFO:teuthology.orchestra.run.smithi086.stdout:client5054 2024-04-25T14:05:14.374 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:05:14.374 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:05:14.383 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T14:05:14.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.568+0000 7ff966600640 1 Processor -- start 2024-04-25T14:05:14.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.568+0000 7ff966600640 1 -- start start 2024-04-25T14:05:14.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.568+0000 7ff966600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff9600d2c80 0x7ff96015dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:14.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.568+0000 7ff966600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff960070e20 0x7ff960071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:14.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.568+0000 7ff966600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff960071770 con 0x7ff9600722f0 2024-04-25T14:05:14.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.568+0000 7ff966600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff9600ce720 con 0x7ff9600d2c80 2024-04-25T14:05:14.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.568+0000 7ff966600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff9600cd8d0 con 0x7ff960070e20 2024-04-25T14:05:14.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.568+0000 7ff95f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff9600d2c80 0x7ff96015dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:14.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.568+0000 7ff95fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff9600722f0 0x7ff960072700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37226/0 (socket says 172.21.15.86:37226) 2024-04-25T14:05:14.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.568+0000 7ff95fe00640 1 -- 172.21.15.86:0/862971063 learned_addr learned my addr 172.21.15.86:0/862971063 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:14.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.568+0000 7ff95ea00640 1 --2- 172.21.15.86:0/862971063 >> v2:172.21.15.155:3300/0 conn(0x7ff960070e20 0x7ff960071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:14.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.568+0000 7ff95f400640 1 -- 172.21.15.86:0/862971063 >> v2:172.21.15.155:3300/0 conn(0x7ff960070e20 msgr2=0x7ff960071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:14.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.568+0000 7ff95f400640 1 --2- 172.21.15.86:0/862971063 >> v2:172.21.15.155:3300/0 conn(0x7ff960070e20 0x7ff960071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:14.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.568+0000 7ff95f400640 1 -- 172.21.15.86:0/862971063 >> v1:172.21.15.86:6789/0 conn(0x7ff9600722f0 legacy=0x7ff960072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:14.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.568+0000 7ff95f400640 1 -- 172.21.15.86:0/862971063 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9600ce970 con 0x7ff9600d2c80 2024-04-25T14:05:14.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.568+0000 7ff95e000640 1 -- 172.21.15.86:0/862971063 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2008804749 0 0) 0x7ff9580014a0 con 0x7ff9600722f0 2024-04-25T14:05:14.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.572+0000 7ff95f400640 1 --2- 172.21.15.86:0/862971063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9600d2c80 0x7ff96015dc90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff9600b6070 tx=0x7ff95001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f500b8b6b0bed759 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:14.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.572+0000 7ff95e000640 1 -- 172.21.15.86:0/862971063 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9500b5070 con 0x7ff9600d2c80 2024-04-25T14:05:14.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.572+0000 7ff95e000640 1 -- 172.21.15.86:0/862971063 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9500a8970 con 0x7ff9600d2c80 2024-04-25T14:05:14.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.572+0000 7ff95e000640 1 -- 172.21.15.86:0/862971063 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9500a8d40 con 0x7ff9600d2c80 2024-04-25T14:05:14.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.572+0000 7ff966600640 1 -- 172.21.15.86:0/862971063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9600d2c80 msgr2=0x7ff96015dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:14.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.572+0000 7ff966600640 1 --2- 172.21.15.86:0/862971063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9600d2c80 0x7ff96015dc90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff9600b6070 tx=0x7ff95001ec70 comp rx=0 tx=0).stop 2024-04-25T14:05:14.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.572+0000 7ff966600640 1 -- 172.21.15.86:0/862971063 shutdown_connections 2024-04-25T14:05:14.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.572+0000 7ff966600640 1 --2- 172.21.15.86:0/862971063 >> v2:172.21.15.155:3300/0 conn(0x7ff960070e20 0x7ff960071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:14.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.572+0000 7ff966600640 1 --2- 172.21.15.86:0/862971063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9600d2c80 0x7ff96015dc90 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:14.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.572+0000 7ff966600640 1 -- 172.21.15.86:0/862971063 >> 172.21.15.86:0/862971063 conn(0x7ff9600cffc0 msgr2=0x7ff9600d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:14.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.572+0000 7ff966600640 1 -- 172.21.15.86:0/862971063 shutdown_connections 2024-04-25T14:05:14.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.572+0000 7ff966600640 1 -- 172.21.15.86:0/862971063 wait complete. 2024-04-25T14:05:14.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.576+0000 7ff966600640 1 Processor -- start 2024-04-25T14:05:14.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.576+0000 7ff966600640 1 -- start start 2024-04-25T14:05:14.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.576+0000 7ff966600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff960070e20 0x7ff96006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:14.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.576+0000 7ff966600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9600722f0 0x7ff9600aea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:14.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.576+0000 7ff966600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9600d2c80 0x7ff9600b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:14.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.576+0000 7ff966600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff9600d21a0 con 0x7ff960070e20 2024-04-25T14:05:14.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.576+0000 7ff966600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff9600d1ec0 con 0x7ff9600d2c80 2024-04-25T14:05:14.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.576+0000 7ff966600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff9600d2480 con 0x7ff9600722f0 2024-04-25T14:05:14.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.580+0000 7ff95f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff960070e20 0x7ff96006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:14.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.580+0000 7ff95f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff960070e20 0x7ff96006d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57666/0 (socket says 172.21.15.86:57666) 2024-04-25T14:05:14.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.580+0000 7ff95f400640 1 -- 172.21.15.86:0/1651469663 learned_addr learned my addr 172.21.15.86:0/1651469663 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:14.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.580+0000 7ff95ea00640 1 --2- 172.21.15.86:0/1651469663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9600722f0 0x7ff9600aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:14.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.580+0000 7ff95fe00640 1 --2- 172.21.15.86:0/1651469663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9600d2c80 0x7ff9600b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:14.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.580+0000 7ff95f400640 1 -- 172.21.15.86:0/1651469663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9600722f0 msgr2=0x7ff9600aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:14.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.580+0000 7ff95f400640 1 --2- 172.21.15.86:0/1651469663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9600722f0 0x7ff9600aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:14.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.580+0000 7ff95f400640 1 -- 172.21.15.86:0/1651469663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9600d2c80 msgr2=0x7ff9600b1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:14.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.580+0000 7ff95f400640 1 --2- 172.21.15.86:0/1651469663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9600d2c80 0x7ff9600b1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:14.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.580+0000 7ff95f400640 1 -- 172.21.15.86:0/1651469663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96006dcc0 con 0x7ff960070e20 2024-04-25T14:05:14.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.580+0000 7ff95fe00640 1 --2- 172.21.15.86:0/1651469663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9600d2c80 0x7ff9600b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:14.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.580+0000 7ff95f400640 1 --2- 172.21.15.86:0/1651469663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff960070e20 0x7ff96006d780 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff95001b880 tx=0x7ff95001ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:14.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.580+0000 7ff957400640 1 -- 172.21.15.86:0/1651469663 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9500c0070 con 0x7ff960070e20 2024-04-25T14:05:14.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.580+0000 7ff957400640 1 -- 172.21.15.86:0/1651469663 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff950004230 con 0x7ff960070e20 2024-04-25T14:05:14.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.580+0000 7ff957400640 1 -- 172.21.15.86:0/1651469663 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9500b5040 con 0x7ff960070e20 2024-04-25T14:05:14.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.580+0000 7ff966600640 1 -- 172.21.15.86:0/1651469663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96006e1d0 con 0x7ff960070e20 2024-04-25T14:05:14.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.580+0000 7ff966600640 1 -- 172.21.15.86:0/1651469663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff96006e790 con 0x7ff960070e20 2024-04-25T14:05:14.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.580+0000 7ff966600640 1 -- 172.21.15.86:0/1651469663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9600720e0 con 0x7ff960070e20 2024-04-25T14:05:14.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.580+0000 7ff957400640 1 -- 172.21.15.86:0/1651469663 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff950003620 con 0x7ff960070e20 2024-04-25T14:05:14.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.584+0000 7ff957400640 1 --2- 172.21.15.86:0/1651469663 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff94006f430 0x7ff940071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:14.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.584+0000 7ff957400640 1 -- 172.21.15.86:0/1651469663 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7ff95012bc90 con 0x7ff960070e20 2024-04-25T14:05:14.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.584+0000 7ff95ea00640 1 --2- 172.21.15.86:0/1651469663 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff94006f430 0x7ff940071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:14.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.584+0000 7ff95ea00640 1 --2- 172.21.15.86:0/1651469663 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff94006f430 0x7ff940071940 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff9600c9fd0 tx=0x7ff94c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:14.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.588+0000 7ff957400640 1 -- 172.21.15.86:0/1651469663 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff9500187d0 con 0x7ff960070e20 2024-04-25T14:05:14.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.780+0000 7ff966600640 1 -- 172.21.15.86:0/1651469663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff960076bf0 con 0x7ff960070e20 2024-04-25T14:05:14.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.780+0000 7ff957400640 1 -- 172.21.15.86:0/1651469663 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7ff9500f8420 con 0x7ff960070e20 2024-04-25T14:05:14.782 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T14:05:14.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.780+0000 7ff966600640 1 -- 172.21.15.86:0/1651469663 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff94006f430 msgr2=0x7ff940071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:14.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.780+0000 7ff966600640 1 --2- 172.21.15.86:0/1651469663 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff94006f430 0x7ff940071940 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff9600c9fd0 tx=0x7ff94c008040 comp rx=0 tx=0).stop 2024-04-25T14:05:14.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.780+0000 7ff966600640 1 -- 172.21.15.86:0/1651469663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff960070e20 msgr2=0x7ff96006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:14.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.780+0000 7ff966600640 1 --2- 172.21.15.86:0/1651469663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff960070e20 0x7ff96006d780 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff95001b880 tx=0x7ff95001ba10 comp rx=0 tx=0).stop 2024-04-25T14:05:14.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.780+0000 7ff966600640 1 -- 172.21.15.86:0/1651469663 shutdown_connections 2024-04-25T14:05:14.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.780+0000 7ff966600640 1 --2- 172.21.15.86:0/1651469663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9600d2c80 0x7ff9600b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:14.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.780+0000 7ff966600640 1 --2- 172.21.15.86:0/1651469663 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7ff94006f430 0x7ff940071940 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:14.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.780+0000 7ff966600640 1 --2- 172.21.15.86:0/1651469663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9600722f0 0x7ff9600aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:14.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.780+0000 7ff966600640 1 --2- 172.21.15.86:0/1651469663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff960070e20 0x7ff96006d780 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:14.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.780+0000 7ff966600640 1 -- 172.21.15.86:0/1651469663 >> 172.21.15.86:0/1651469663 conn(0x7ff9600cffc0 msgr2=0x7ff9600cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:14.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.780+0000 7ff966600640 1 -- 172.21.15.86:0/1651469663 shutdown_connections 2024-04-25T14:05:14.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:14.780+0000 7ff966600640 1 -- 172.21.15.86:0/1651469663 wait complete. 2024-04-25T14:05:14.803 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T14:05:14.803 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T14:05:14.803 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2024-04-25T14:05:14.803 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T14:05:14.803 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T14:05:14.803 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T14:05:14.803 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T14:05:14.803 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T14:05:14.803 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:05:14.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:14.804 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2024-04-25T14:05:14.814 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:05:14.814 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:05:14.814 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:05:14.814 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:05:14.815 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2024-04-25T14:05:14.815 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:05:14.815 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:05:14.815 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2024-04-25T14:05:14.815 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f0 2024-04-25T14:05:14.815 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:05:14.815 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2024-04-25T14:05:14.815 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f1 2024-04-25T14:05:14.815 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:05:14.815 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2024-04-25T14:05:14.815 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T14:05:14.815 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2024-04-25T14:05:14.815 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T14:05:14.816 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 42343sec preferred_lft 42343sec 2024-04-25T14:05:14.816 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::ec4:7aff:fe88:6ded/64 scope link 2024-04-25T14:05:14.816 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:05:14.816 INFO:teuthology.orchestra.run.smithi155.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T14:05:14.816 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 02:42:b1:ff:60:50 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:05:14.816 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T14:05:14.816 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:05:14.816 INFO:teuthology.orchestra.run.smithi155.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T14:05:14.816 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T14:05:14.816 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T14:05:14.816 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:05:14.816 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T14:05:14.816 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:05:14.816 INFO:teuthology.orchestra.run.smithi155.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:05:14.817 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T14:05:14.817 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T14:05:14.817 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:05:14.817 INFO:teuthology.orchestra.run.smithi155.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:05:14.817 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T14:05:14.817 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T14:05:14.817 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:05:14.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:14.818 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T14:05:14.884 INFO:teuthology.orchestra.run.smithi155.stdout:1 2024-04-25T14:05:14.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:14.886 DEBUG:teuthology.orchestra.run.smithi155:> route 2024-04-25T14:05:14.944 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2024-04-25T14:05:14.945 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T14:05:14.945 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T14:05:14.945 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T14:05:14.945 INFO:teuthology.orchestra.run.smithi155.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T14:05:14.945 INFO:teuthology.orchestra.run.smithi155.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T14:05:14.945 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T14:05:14.945 INFO:teuthology.orchestra.run.smithi155.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:05:14.945 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:05:14.945 INFO:teuthology.orchestra.run.smithi155.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:05:14.945 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T14:05:14.945 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:05:14.946 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-04-25T14:05:14.946 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T14:05:14.946 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T14:05:14.946 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T14:05:14.946 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-04-25T14:05:15.044 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:05:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:05:15.087 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:05:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:05:15.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:15.094 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2024-04-25T14:05:15.102 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T14:05:15.103 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T14:05:15.103 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:05:15.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:15.104 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.0 up 2024-04-25T14:05:15.168 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:05:15.168 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T14:05:15.219 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:05:15.220 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:05:15.220 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T14:05:15.272 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T14:05:15.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:15.272 DEBUG:teuthology.orchestra.run.smithi155:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:05:15.377 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:05:15.378 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T14:05:15.378 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:05:15.396 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:05:15.396 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T14:05:15.396 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:05:15.396 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:05:15.396 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:05:15.406 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:05:15.406 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T14:05:15.406 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:05:15.406 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:05:15.406 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:05:15.406 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790" 2024-04-25T14:05:15.407 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790:/ 2024-04-25T14:05:15.407 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb" will pass to kernel 2024-04-25T14:05:15.410 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T14:05:15.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:15.410 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T14:05:15.430 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:05:15.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:05:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:05:15.525 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:05:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:05:15.531 INFO:teuthology.orchestra.run.smithi155.stdout:client5060 2024-04-25T14:05:15.531 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:05:15.531 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:05:15.580 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:05:15.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.732+0000 7f3223800640 1 Processor -- start 2024-04-25T14:05:15.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.732+0000 7f3223800640 1 -- start start 2024-04-25T14:05:15.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.732+0000 7f3223800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f321c0d1610 0x7f321c0d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:15.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.732+0000 7f3223800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f321c0cd8d0 0x7f321c0cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:15.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.732+0000 7f3223800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f321c0ce220 con 0x7f321c0cf000 2024-04-25T14:05:15.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.732+0000 7f3223800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f321c0ce640 con 0x7f321c0cd8d0 2024-04-25T14:05:15.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.732+0000 7f3223800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f321c0ceb70 con 0x7f321c0d1610 2024-04-25T14:05:15.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.732+0000 7f3220e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f321c0d1610 0x7f321c0d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:15.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.732+0000 7f321be00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f321c0cd8d0 0x7f321c0cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:15.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.732+0000 7f3221800640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f321c0cf000 0x7f321c0cf410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:47736/0 (socket says 172.21.15.86:47736) 2024-04-25T14:05:15.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.732+0000 7f3221800640 1 -- 172.21.15.86:0/658835053 learned_addr learned my addr 172.21.15.86:0/658835053 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:15.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.732+0000 7f321b400640 1 -- 172.21.15.86:0/658835053 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2426129586 0 0) 0x7f321c0ce220 con 0x7f321c0cf000 2024-04-25T14:05:15.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.732+0000 7f321b400640 1 -- 172.21.15.86:0/658835053 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3200003530 con 0x7f321c0cf000 2024-04-25T14:05:15.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.736+0000 7f3220e00640 1 -- 172.21.15.86:0/658835053 >> v2:172.21.15.155:3300/0 conn(0x7f321c0cd8d0 msgr2=0x7f321c0cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:15.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.736+0000 7f3220e00640 1 --2- 172.21.15.86:0/658835053 >> v2:172.21.15.155:3300/0 conn(0x7f321c0cd8d0 0x7f321c0cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:15.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.736+0000 7f3220e00640 1 -- 172.21.15.86:0/658835053 >> v1:172.21.15.155:6789/0 conn(0x7f321c0cf000 legacy=0x7f321c0cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:15.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.736+0000 7f3220e00640 1 -- 172.21.15.86:0/658835053 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321c0ce890 con 0x7f321c0d1610 2024-04-25T14:05:15.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.736+0000 7f321be00640 1 --2- 172.21.15.86:0/658835053 >> v2:172.21.15.155:3300/0 conn(0x7f321c0cd8d0 0x7f321c0cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:15.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.736+0000 7f3220e00640 1 --2- 172.21.15.86:0/658835053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f321c0d1610 0x7f321c0d1a20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f320c0097c0 tx=0x7f320c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a82e2e5bbf14f2a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:15.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.736+0000 7f321b400640 1 -- 172.21.15.86:0/658835053 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c0b1070 con 0x7f321c0d1610 2024-04-25T14:05:15.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.736+0000 7f321b400640 1 -- 172.21.15.86:0/658835053 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f320c0ac410 con 0x7f321c0d1610 2024-04-25T14:05:15.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.736+0000 7f321b400640 1 -- 172.21.15.86:0/658835053 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c0ac7e0 con 0x7f321c0d1610 2024-04-25T14:05:15.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.736+0000 7f3223800640 1 -- 172.21.15.86:0/658835053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f321c0d1610 msgr2=0x7f321c0d1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:15.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.736+0000 7f3223800640 1 --2- 172.21.15.86:0/658835053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f321c0d1610 0x7f321c0d1a20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f320c0097c0 tx=0x7f320c01c4a0 comp rx=0 tx=0).stop 2024-04-25T14:05:15.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.740+0000 7f3223800640 1 -- 172.21.15.86:0/658835053 shutdown_connections 2024-04-25T14:05:15.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.740+0000 7f3223800640 1 --2- 172.21.15.86:0/658835053 >> v2:172.21.15.155:3300/0 conn(0x7f321c0cd8d0 0x7f321c0cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:15.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.740+0000 7f3223800640 1 --2- 172.21.15.86:0/658835053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f321c0d1610 0x7f321c0d1a20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:15.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.740+0000 7f3223800640 1 -- 172.21.15.86:0/658835053 >> 172.21.15.86:0/658835053 conn(0x7f321c09ba30 msgr2=0x7f321c0b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:15.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.740+0000 7f3223800640 1 -- 172.21.15.86:0/658835053 shutdown_connections 2024-04-25T14:05:15.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.740+0000 7f3223800640 1 -- 172.21.15.86:0/658835053 wait complete. 2024-04-25T14:05:15.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.740+0000 7f3223800640 1 Processor -- start 2024-04-25T14:05:15.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.740+0000 7f3223800640 1 -- start start 2024-04-25T14:05:15.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.740+0000 7f3223800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f321c0cd8d0 0x7f321c0c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:15.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.740+0000 7f3223800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f321c0cf000 0x7f321c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:15.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.740+0000 7f3223800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f321c0d1610 0x7f321c06d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:15.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.740+0000 7f3223800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f321c0cfca0 con 0x7f321c0d1610 2024-04-25T14:05:15.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.740+0000 7f3223800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f321c0cfb20 con 0x7f321c0cd8d0 2024-04-25T14:05:15.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.740+0000 7f3223800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f321c16d350 con 0x7f321c0cf000 2024-04-25T14:05:15.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.744+0000 7f3221800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f321c0d1610 0x7f321c06d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:15.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.744+0000 7f3221800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f321c0d1610 0x7f321c06d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57678/0 (socket says 172.21.15.86:57678) 2024-04-25T14:05:15.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.744+0000 7f3221800640 1 -- 172.21.15.86:0/1933823351 learned_addr learned my addr 172.21.15.86:0/1933823351 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:15.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.744+0000 7f321be00640 1 --2- 172.21.15.86:0/1933823351 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f321c0cf000 0x7f321c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:15.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.744+0000 7f3220e00640 1 --2- 172.21.15.86:0/1933823351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f321c0cd8d0 0x7f321c0c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:15.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.744+0000 7f3221800640 1 -- 172.21.15.86:0/1933823351 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f321c0cf000 msgr2=0x7f321c06d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:15.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.744+0000 7f3221800640 1 --2- 172.21.15.86:0/1933823351 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f321c0cf000 0x7f321c06d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:15.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.744+0000 7f3221800640 1 -- 172.21.15.86:0/1933823351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f321c0cd8d0 msgr2=0x7f321c0c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:15.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.744+0000 7f3221800640 1 --2- 172.21.15.86:0/1933823351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f321c0cd8d0 0x7f321c0c87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:15.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.744+0000 7f3221800640 1 -- 172.21.15.86:0/1933823351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321c0d09c0 con 0x7f321c0d1610 2024-04-25T14:05:15.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.744+0000 7f3221800640 1 --2- 172.21.15.86:0/1933823351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f321c0d1610 0x7f321c06d6c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f321400cb40 tx=0x7f321400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:15.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.744+0000 7f3218c00640 1 -- 172.21.15.86:0/1933823351 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214017070 con 0x7f321c0d1610 2024-04-25T14:05:15.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.744+0000 7f3218c00640 1 -- 172.21.15.86:0/1933823351 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f321400aca0 con 0x7f321c0d1610 2024-04-25T14:05:15.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.744+0000 7f3223800640 1 -- 172.21.15.86:0/1933823351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f321c06df50 con 0x7f321c0d1610 2024-04-25T14:05:15.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.744+0000 7f3218c00640 1 -- 172.21.15.86:0/1933823351 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214012600 con 0x7f321c0d1610 2024-04-25T14:05:15.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.744+0000 7f3223800640 1 -- 172.21.15.86:0/1933823351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f321c06e510 con 0x7f321c0d1610 2024-04-25T14:05:15.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.744+0000 7f3223800640 1 -- 172.21.15.86:0/1933823351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f321c076bf0 con 0x7f321c0d1610 2024-04-25T14:05:15.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.744+0000 7f3218c00640 1 -- 172.21.15.86:0/1933823351 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f321401a070 con 0x7f321c0d1610 2024-04-25T14:05:15.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.748+0000 7f3218c00640 1 --2- 172.21.15.86:0/1933823351 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f31f806f430 0x7f31f8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:15.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.748+0000 7f3218c00640 1 -- 172.21.15.86:0/1933823351 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f321408f160 con 0x7f321c0d1610 2024-04-25T14:05:15.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.748+0000 7f3220e00640 1 --2- 172.21.15.86:0/1933823351 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f31f806f430 0x7f31f8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:15.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.748+0000 7f3220e00640 1 --2- 172.21.15.86:0/1933823351 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f31f806f430 0x7f31f8071940 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f320c0ae000 tx=0x7f320c0a6040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:15.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.752+0000 7f3218c00640 1 -- 172.21.15.86:0/1933823351 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f321405b7d0 con 0x7f321c0d1610 2024-04-25T14:05:15.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.944+0000 7f3223800640 1 -- 172.21.15.86:0/1933823351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f321c15d300 con 0x7f321c0d1610 2024-04-25T14:05:15.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.944+0000 7f3218c00640 1 -- 172.21.15.86:0/1933823351 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5184 (secure 0 0 0) 0x7f321405fad0 con 0x7f321c0d1610 2024-04-25T14:05:15.946 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:15.946 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3546153119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3546153119},{"type":"v1","addr":"172.21.15.155:6834","nonce":3546153119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5283,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3499340224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3499340224},{"type":"v1","addr":"172.21.15.86:6836","nonce":3499340224}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5284,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1466978742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1466978742},{"type":"v1","addr":"172.21.15.155:6835","nonce":1466978742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:06.638216+0000","modified":"2024-04-25T14:05:09.664776+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5286":{"gid":5286,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/716923566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":716923566},{"type":"v1","addr":"172.21.15.86:6837","nonce":716923566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5286,"qdb_cluster":[5286]},"id":2}]} 2024-04-25T14:05:15.946 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 18 2024-04-25T14:05:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.944+0000 7f3223800640 1 -- 172.21.15.86:0/1933823351 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f31f806f430 msgr2=0x7f31f8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.944+0000 7f3223800640 1 --2- 172.21.15.86:0/1933823351 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f31f806f430 0x7f31f8071940 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f320c0ae000 tx=0x7f320c0a6040 comp rx=0 tx=0).stop 2024-04-25T14:05:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.944+0000 7f3223800640 1 -- 172.21.15.86:0/1933823351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f321c0d1610 msgr2=0x7f321c06d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.944+0000 7f3223800640 1 --2- 172.21.15.86:0/1933823351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f321c0d1610 0x7f321c06d6c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f321400cb40 tx=0x7f321400a060 comp rx=0 tx=0).stop 2024-04-25T14:05:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.944+0000 7f3223800640 1 -- 172.21.15.86:0/1933823351 shutdown_connections 2024-04-25T14:05:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.944+0000 7f3223800640 1 --2- 172.21.15.86:0/1933823351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f321c0d1610 0x7f321c06d6c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.944+0000 7f3223800640 1 --2- 172.21.15.86:0/1933823351 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f31f806f430 0x7f31f8071940 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.944+0000 7f3223800640 1 --2- 172.21.15.86:0/1933823351 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f321c0cf000 0x7f321c06d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.944+0000 7f3223800640 1 --2- 172.21.15.86:0/1933823351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f321c0cd8d0 0x7f321c0c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:15.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.944+0000 7f3223800640 1 -- 172.21.15.86:0/1933823351 >> 172.21.15.86:0/1933823351 conn(0x7f321c09ba30 msgr2=0x7f321c0b0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:15.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.944+0000 7f3223800640 1 -- 172.21.15.86:0/1933823351 shutdown_connections 2024-04-25T14:05:15.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:15.944+0000 7f3223800640 1 -- 172.21.15.86:0/1933823351 wait complete. 2024-04-25T14:05:15.970 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable stats 2024-04-25T14:05:16.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.116+0000 7f2702200640 1 Processor -- start 2024-04-25T14:05:16.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.116+0000 7f2702200640 1 -- start start 2024-04-25T14:05:16.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.120+0000 7f2702200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f26fc0d1df0 0x7f26fc0cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:16.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.120+0000 7f2702200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26fc0cdcc0 0x7f26fc0ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:16.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.120+0000 7f2702200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26fc15e5c0 con 0x7f26fc070330 2024-04-25T14:05:16.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.120+0000 7f2702200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f26fc06fd40 con 0x7f26fc0d1df0 2024-04-25T14:05:16.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.120+0000 7f2702200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f26fc15e8a0 con 0x7f26fc0cdcc0 2024-04-25T14:05:16.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.120+0000 7f26fb400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f26fc070330 0x7f26fc0d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37242/0 (socket says 172.21.15.86:37242) 2024-04-25T14:05:16.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.120+0000 7f26fb400640 1 -- 172.21.15.86:0/874706995 learned_addr learned my addr 172.21.15.86:0/874706995 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:16.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.120+0000 7f26fbe00640 1 --2- 172.21.15.86:0/874706995 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26fc0cdcc0 0x7f26fc0ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:16.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.120+0000 7f26f2a00640 1 --2- 172.21.15.86:0/874706995 >> v2:172.21.15.155:3300/0 conn(0x7f26fc0d1df0 0x7f26fc0cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:16.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.120+0000 7f26faa00640 1 -- 172.21.15.86:0/874706995 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1411427861 0 0) 0x7f26fc15e5c0 con 0x7f26fc070330 2024-04-25T14:05:16.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.120+0000 7f26faa00640 1 -- 172.21.15.86:0/874706995 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26e0003530 con 0x7f26fc070330 2024-04-25T14:05:16.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.120+0000 7f26fbe00640 1 -- 172.21.15.86:0/874706995 >> v2:172.21.15.155:3300/0 conn(0x7f26fc0d1df0 msgr2=0x7f26fc0cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:16.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.120+0000 7f26fbe00640 1 --2- 172.21.15.86:0/874706995 >> v2:172.21.15.155:3300/0 conn(0x7f26fc0d1df0 0x7f26fc0cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:16.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.120+0000 7f26fbe00640 1 -- 172.21.15.86:0/874706995 >> v1:172.21.15.86:6789/0 conn(0x7f26fc070330 legacy=0x7f26fc0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:16.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.120+0000 7f26fbe00640 1 -- 172.21.15.86:0/874706995 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26fc15eb80 con 0x7f26fc0cdcc0 2024-04-25T14:05:16.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.120+0000 7f26fbe00640 1 --2- 172.21.15.86:0/874706995 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26fc0cdcc0 0x7f26fc0ce0d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009d90 tx=0x7f26f401c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d020f9d1097928 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:16.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.124+0000 7f26faa00640 1 -- 172.21.15.86:0/874706995 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f40b2070 con 0x7f26fc0cdcc0 2024-04-25T14:05:16.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.124+0000 7f26faa00640 1 -- 172.21.15.86:0/874706995 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26f40a5950 con 0x7f26fc0cdcc0 2024-04-25T14:05:16.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.124+0000 7f26faa00640 1 -- 172.21.15.86:0/874706995 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f40a5d20 con 0x7f26fc0cdcc0 2024-04-25T14:05:16.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.124+0000 7f2702200640 1 -- 172.21.15.86:0/874706995 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26fc0cdcc0 msgr2=0x7f26fc0ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:16.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.124+0000 7f2702200640 1 --2- 172.21.15.86:0/874706995 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26fc0cdcc0 0x7f26fc0ce0d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009d90 tx=0x7f26f401c8b0 comp rx=0 tx=0).stop 2024-04-25T14:05:16.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.124+0000 7f2702200640 1 -- 172.21.15.86:0/874706995 shutdown_connections 2024-04-25T14:05:16.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.124+0000 7f2702200640 1 --2- 172.21.15.86:0/874706995 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26fc0cdcc0 0x7f26fc0ce0d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:16.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.124+0000 7f2702200640 1 --2- 172.21.15.86:0/874706995 >> v2:172.21.15.155:3300/0 conn(0x7f26fc0d1df0 0x7f26fc0cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:16.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.124+0000 7f2702200640 1 -- 172.21.15.86:0/874706995 >> 172.21.15.86:0/874706995 conn(0x7f26fc0b4650 msgr2=0x7f26fc0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:16.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.124+0000 7f2702200640 1 -- 172.21.15.86:0/874706995 shutdown_connections 2024-04-25T14:05:16.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.124+0000 7f2702200640 1 -- 172.21.15.86:0/874706995 wait complete. 2024-04-25T14:05:16.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.124+0000 7f2702200640 1 Processor -- start 2024-04-25T14:05:16.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.124+0000 7f2702200640 1 -- start start 2024-04-25T14:05:16.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.128+0000 7f2702200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26fc070330 0x7f26fc0cb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:16.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.128+0000 7f2702200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26fc0cdcc0 0x7f26fc1589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:16.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.128+0000 7f2702200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26fc0b1b90 0x7f26fc0b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:16.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.128+0000 7f2702200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc15f4b0 con 0x7f26fc0cdcc0 2024-04-25T14:05:16.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.128+0000 7f2702200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc15f330 con 0x7f26fc070330 2024-04-25T14:05:16.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.128+0000 7f2702200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f26fc070740 con 0x7f26fc0b1b90 2024-04-25T14:05:16.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.128+0000 7f26f2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26fc0cdcc0 0x7f26fc1589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:16.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.128+0000 7f26f2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26fc0cdcc0 0x7f26fc1589b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57680/0 (socket says 172.21.15.86:57680) 2024-04-25T14:05:16.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.128+0000 7f26f2a00640 1 -- 172.21.15.86:0/2206794992 learned_addr learned my addr 172.21.15.86:0/2206794992 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:16.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.128+0000 7f26fbe00640 1 --2- 172.21.15.86:0/2206794992 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26fc0b1b90 0x7f26fc0b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:16.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.128+0000 7f26fb400640 1 --2- 172.21.15.86:0/2206794992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26fc070330 0x7f26fc0cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:16.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.128+0000 7f26f2a00640 1 -- 172.21.15.86:0/2206794992 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26fc0b1b90 msgr2=0x7f26fc0b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:16.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.128+0000 7f26f2a00640 1 --2- 172.21.15.86:0/2206794992 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26fc0b1b90 0x7f26fc0b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:16.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.128+0000 7f26f2a00640 1 -- 172.21.15.86:0/2206794992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26fc070330 msgr2=0x7f26fc0cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:16.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.128+0000 7f26f2a00640 1 --2- 172.21.15.86:0/2206794992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26fc070330 0x7f26fc0cb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:16.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.128+0000 7f26f2a00640 1 -- 172.21.15.86:0/2206794992 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26fc0c3ba0 con 0x7f26fc0cdcc0 2024-04-25T14:05:16.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.128+0000 7f26f2a00640 1 --2- 172.21.15.86:0/2206794992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26fc0cdcc0 0x7f26fc1589b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f26e400e870 tx=0x7f26e400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:16.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.128+0000 7f26f3e00640 1 -- 172.21.15.86:0/2206794992 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e4017070 con 0x7f26fc0cdcc0 2024-04-25T14:05:16.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.128+0000 7f26f3e00640 1 -- 172.21.15.86:0/2206794992 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e400ee60 con 0x7f26fc0cdcc0 2024-04-25T14:05:16.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.128+0000 7f2702200640 1 -- 172.21.15.86:0/2206794992 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26fc0c3e80 con 0x7f26fc0cdcc0 2024-04-25T14:05:16.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.128+0000 7f26f3e00640 1 -- 172.21.15.86:0/2206794992 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e400cb40 con 0x7f26fc0cdcc0 2024-04-25T14:05:16.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.128+0000 7f2702200640 1 -- 172.21.15.86:0/2206794992 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26fc0c4160 con 0x7f26fc0cdcc0 2024-04-25T14:05:16.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.128+0000 7f2702200640 1 -- 172.21.15.86:0/2206794992 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26fc069d00 con 0x7f26fc0cdcc0 2024-04-25T14:05:16.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.132+0000 7f26f3e00640 1 -- 172.21.15.86:0/2206794992 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f26e401a070 con 0x7f26fc0cdcc0 2024-04-25T14:05:16.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.132+0000 7f26f3e00640 1 --2- 172.21.15.86:0/2206794992 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f26d806f430 0x7f26d8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:16.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.132+0000 7f26f3e00640 1 -- 172.21.15.86:0/2206794992 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f26e408f5c0 con 0x7f26fc0cdcc0 2024-04-25T14:05:16.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.132+0000 7f26fb400640 1 --2- 172.21.15.86:0/2206794992 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f26d806f430 0x7f26d8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:16.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.132+0000 7f26fb400640 1 --2- 172.21.15.86:0/2206794992 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f26d806f430 0x7f26d8071940 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f26ec007fd0 tx=0x7f26ec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:16.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.136+0000 7f26f3e00640 1 -- 172.21.15.86:0/2206794992 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f26e405bcc0 con 0x7f26fc0cdcc0 2024-04-25T14:05:16.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:16.332+0000 7f2702200640 1 -- 172.21.15.86:0/2206794992 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "stats"} v 0) -- 0x7f26fc076bf0 con 0x7f26fc0cdcc0 2024-04-25T14:05:17.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.308+0000 7f26f3e00640 1 -- 172.21.15.86:0/2206794992 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "stats"}]=0 v6) ==== 84+0+0 (secure 0 0 0) 0x7f26e405ffc0 con 0x7f26fc0cdcc0 2024-04-25T14:05:17.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.312+0000 7f2702200640 1 -- 172.21.15.86:0/2206794992 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f26d806f430 msgr2=0x7f26d8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:17.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.312+0000 7f2702200640 1 --2- 172.21.15.86:0/2206794992 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f26d806f430 0x7f26d8071940 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f26ec007fd0 tx=0x7f26ec009040 comp rx=0 tx=0).stop 2024-04-25T14:05:17.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.312+0000 7f2702200640 1 -- 172.21.15.86:0/2206794992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26fc0cdcc0 msgr2=0x7f26fc1589b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:17.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.312+0000 7f2702200640 1 --2- 172.21.15.86:0/2206794992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26fc0cdcc0 0x7f26fc1589b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f26e400e870 tx=0x7f26e400ed30 comp rx=0 tx=0).stop 2024-04-25T14:05:17.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.312+0000 7f2702200640 1 -- 172.21.15.86:0/2206794992 shutdown_connections 2024-04-25T14:05:17.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.312+0000 7f2702200640 1 --2- 172.21.15.86:0/2206794992 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f26d806f430 0x7f26d8071940 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:17.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.312+0000 7f2702200640 1 --2- 172.21.15.86:0/2206794992 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26fc0b1b90 0x7f26fc0b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:17.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.312+0000 7f2702200640 1 --2- 172.21.15.86:0/2206794992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26fc0cdcc0 0x7f26fc1589b0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:17.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.312+0000 7f2702200640 1 --2- 172.21.15.86:0/2206794992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26fc070330 0x7f26fc0cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:17.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.312+0000 7f2702200640 1 -- 172.21.15.86:0/2206794992 >> 172.21.15.86:0/2206794992 conn(0x7f26fc0b4650 msgr2=0x7f26fc0ae3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:17.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.312+0000 7f2702200640 1 -- 172.21.15.86:0/2206794992 shutdown_connections 2024-04-25T14:05:17.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.312+0000 7f2702200640 1 -- 172.21.15.86:0/2206794992 wait complete. 2024-04-25T14:05:17.342 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:05:17.342 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:05:17.351 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:05:17.352 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:05:17.406 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:05:17.559 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:05:17.554+0000 7f28af07d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:05:17.570 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:17.568+0000 7f7fa3e2c180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:05:17.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.600+0000 7fb693c00640 1 Processor -- start 2024-04-25T14:05:17.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.600+0000 7fb693c00640 1 -- start start 2024-04-25T14:05:17.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.600+0000 7fb693c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb68c0d2c80 0x7fb68c15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:17.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.600+0000 7fb693c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb68c070e20 0x7fb68c071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:17.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.600+0000 7fb693c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb68c0722f0 0x7fb68c072700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:17.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.600+0000 7fb693c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb68c0ce720 con 0x7fb68c070e20 2024-04-25T14:05:17.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.600+0000 7fb693c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb68c071770 con 0x7fb68c0722f0 2024-04-25T14:05:17.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.600+0000 7fb693c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb68c0cd8d0 con 0x7fb68c0d2c80 2024-04-25T14:05:17.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.600+0000 7fb68be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb68c070e20 0x7fb68c071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:17.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.600+0000 7fb68be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb68c070e20 0x7fb68c071230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57696/0 (socket says 172.21.15.86:57696) 2024-04-25T14:05:17.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.600+0000 7fb68be00640 1 -- 172.21.15.86:0/3349297921 learned_addr learned my addr 172.21.15.86:0/3349297921 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:17.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.600+0000 7fb691c00640 1 --2- 172.21.15.86:0/3349297921 >> v2:172.21.15.155:3300/0 conn(0x7fb68c0722f0 0x7fb68c072700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:17.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.600+0000 7fb691200640 1 --2- 172.21.15.86:0/3349297921 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb68c0d2c80 0x7fb68c15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:17.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.600+0000 7fb68be00640 1 -- 172.21.15.86:0/3349297921 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb68c0d2c80 msgr2=0x7fb68c15dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:17.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.600+0000 7fb68be00640 1 --2- 172.21.15.86:0/3349297921 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb68c0d2c80 0x7fb68c15dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:17.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.600+0000 7fb68be00640 1 -- 172.21.15.86:0/3349297921 >> v2:172.21.15.155:3300/0 conn(0x7fb68c0722f0 msgr2=0x7fb68c072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:17.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.600+0000 7fb68be00640 1 --2- 172.21.15.86:0/3349297921 >> v2:172.21.15.155:3300/0 conn(0x7fb68c0722f0 0x7fb68c072700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:17.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.600+0000 7fb68be00640 1 -- 172.21.15.86:0/3349297921 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb68c0ce970 con 0x7fb68c070e20 2024-04-25T14:05:17.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.600+0000 7fb691c00640 1 --2- 172.21.15.86:0/3349297921 >> v2:172.21.15.155:3300/0 conn(0x7fb68c0722f0 0x7fb68c072700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:17.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.600+0000 7fb691200640 1 --2- 172.21.15.86:0/3349297921 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb68c0d2c80 0x7fb68c15dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:17.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.604+0000 7fb68be00640 1 --2- 172.21.15.86:0/3349297921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb68c070e20 0x7fb68c071230 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0736f0 tx=0x7fb67801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d09da43af97780 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:17.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.604+0000 7fb68b400640 1 -- 172.21.15.86:0/3349297921 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6780b5070 con 0x7fb68c070e20 2024-04-25T14:05:17.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.604+0000 7fb68b400640 1 -- 172.21.15.86:0/3349297921 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6780a89d0 con 0x7fb68c070e20 2024-04-25T14:05:17.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.604+0000 7fb68b400640 1 -- 172.21.15.86:0/3349297921 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6780a8da0 con 0x7fb68c070e20 2024-04-25T14:05:17.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.604+0000 7fb693c00640 1 -- 172.21.15.86:0/3349297921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb68c070e20 msgr2=0x7fb68c071230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:17.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.604+0000 7fb693c00640 1 --2- 172.21.15.86:0/3349297921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb68c070e20 0x7fb68c071230 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0736f0 tx=0x7fb67801ed10 comp rx=0 tx=0).stop 2024-04-25T14:05:17.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.604+0000 7fb693c00640 1 -- 172.21.15.86:0/3349297921 shutdown_connections 2024-04-25T14:05:17.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.604+0000 7fb693c00640 1 --2- 172.21.15.86:0/3349297921 >> v2:172.21.15.155:3300/0 conn(0x7fb68c0722f0 0x7fb68c072700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:17.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.604+0000 7fb693c00640 1 --2- 172.21.15.86:0/3349297921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb68c070e20 0x7fb68c071230 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:17.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.604+0000 7fb693c00640 1 --2- 172.21.15.86:0/3349297921 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb68c0d2c80 0x7fb68c15dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:17.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.604+0000 7fb693c00640 1 -- 172.21.15.86:0/3349297921 >> 172.21.15.86:0/3349297921 conn(0x7fb68c0cffc0 msgr2=0x7fb68c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:17.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.604+0000 7fb693c00640 1 -- 172.21.15.86:0/3349297921 shutdown_connections 2024-04-25T14:05:17.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.604+0000 7fb693c00640 1 -- 172.21.15.86:0/3349297921 wait complete. 2024-04-25T14:05:17.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.608+0000 7fb693c00640 1 Processor -- start 2024-04-25T14:05:17.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.608+0000 7fb693c00640 1 -- start start 2024-04-25T14:05:17.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.608+0000 7fb693c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb68c070e20 0x7fb68c15e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:17.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.608+0000 7fb693c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb68c0722f0 0x7fb68c0cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:17.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.608+0000 7fb693c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb68c0d2c80 0x7fb68c1544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:17.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.608+0000 7fb693c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb68c0d21a0 con 0x7fb68c070e20 2024-04-25T14:05:17.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.608+0000 7fb693c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb68c0d1ec0 con 0x7fb68c0d2c80 2024-04-25T14:05:17.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.608+0000 7fb693c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb68c0d2480 con 0x7fb68c0722f0 2024-04-25T14:05:17.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.608+0000 7fb691200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb68c070e20 0x7fb68c15e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:17.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.608+0000 7fb691200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb68c070e20 0x7fb68c15e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57708/0 (socket says 172.21.15.86:57708) 2024-04-25T14:05:17.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.608+0000 7fb691200640 1 -- 172.21.15.86:0/3130051859 learned_addr learned my addr 172.21.15.86:0/3130051859 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:17.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.608+0000 7fb68be00640 1 --2- 172.21.15.86:0/3130051859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb68c0722f0 0x7fb68c0cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:17.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.612+0000 7fb691c00640 1 --2- 172.21.15.86:0/3130051859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb68c0d2c80 0x7fb68c1544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:17.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.612+0000 7fb691200640 1 -- 172.21.15.86:0/3130051859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb68c0722f0 msgr2=0x7fb68c0cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:17.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.612+0000 7fb691200640 1 --2- 172.21.15.86:0/3130051859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb68c0722f0 0x7fb68c0cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:17.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.612+0000 7fb691200640 1 -- 172.21.15.86:0/3130051859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb68c0d2c80 msgr2=0x7fb68c1544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:17.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.612+0000 7fb691200640 1 --2- 172.21.15.86:0/3130051859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb68c0d2c80 0x7fb68c1544c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:17.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.612+0000 7fb691200640 1 -- 172.21.15.86:0/3130051859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb68c15ebe0 con 0x7fb68c070e20 2024-04-25T14:05:17.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.612+0000 7fb691c00640 1 --2- 172.21.15.86:0/3130051859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb68c0d2c80 0x7fb68c1544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:17.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.612+0000 7fb691200640 1 --2- 172.21.15.86:0/3130051859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb68c070e20 0x7fb68c15e6a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb68000d800 tx=0x7fb68000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:17.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.612+0000 7fb688c00640 1 -- 172.21.15.86:0/3130051859 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb680018020 con 0x7fb68c070e20 2024-04-25T14:05:17.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.612+0000 7fb688c00640 1 -- 172.21.15.86:0/3130051859 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6800096c0 con 0x7fb68c070e20 2024-04-25T14:05:17.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.612+0000 7fb693c00640 1 -- 172.21.15.86:0/3130051859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb68c15f0f0 con 0x7fb68c070e20 2024-04-25T14:05:17.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.612+0000 7fb688c00640 1 -- 172.21.15.86:0/3130051859 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb680009ab0 con 0x7fb68c070e20 2024-04-25T14:05:17.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.612+0000 7fb693c00640 1 -- 172.21.15.86:0/3130051859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb68c15f6b0 con 0x7fb68c070e20 2024-04-25T14:05:17.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.612+0000 7fb693c00640 1 -- 172.21.15.86:0/3130051859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb68c0720e0 con 0x7fb68c070e20 2024-04-25T14:05:17.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.612+0000 7fb688c00640 1 -- 172.21.15.86:0/3130051859 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 6) ==== 93216+0+0 (secure 0 0 0) 0x7fb68001b020 con 0x7fb68c070e20 2024-04-25T14:05:17.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.616+0000 7fb688c00640 1 --2- 172.21.15.86:0/3130051859 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb66c06f450 0x7fb66c071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:17.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.616+0000 7fb688c00640 1 -- 172.21.15.86:0/3130051859 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fb68008f8b0 con 0x7fb68c070e20 2024-04-25T14:05:17.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.616+0000 7fb68be00640 1 -- 172.21.15.86:0/3130051859 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb66c06f450 msgr2=0x7fb66c071960 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2968869755 2024-04-25T14:05:17.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.616+0000 7fb68be00640 1 --2- 172.21.15.86:0/3130051859 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb66c06f450 0x7fb66c071960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:05:17.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.624+0000 7fb688c00640 1 -- 172.21.15.86:0/3130051859 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb680022020 con 0x7fb68c070e20 2024-04-25T14:05:17.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.784+0000 7fb693c00640 1 -- 172.21.15.86:0/3130051859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb68c076bf0 con 0x7fb68c070e20 2024-04-25T14:05:17.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.784+0000 7fb688c00640 1 -- 172.21.15.86:0/3130051859 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16942 (secure 0 0 0) 0x7fb68005c6a0 con 0x7fb68c070e20 2024-04-25T14:05:17.788 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:17.788 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":38,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:05:08.245617+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:05:04.472914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T14:05:05.604280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:05:17.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.788+0000 7fb693c00640 1 -- 172.21.15.86:0/3130051859 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb66c06f450 msgr2=0x7fb66c071960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:05:17.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.788+0000 7fb693c00640 1 --2- 172.21.15.86:0/3130051859 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb66c06f450 0x7fb66c071960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:17.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.788+0000 7fb693c00640 1 -- 172.21.15.86:0/3130051859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb68c070e20 msgr2=0x7fb68c15e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:17.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.788+0000 7fb693c00640 1 --2- 172.21.15.86:0/3130051859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb68c070e20 0x7fb68c15e6a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb68000d800 tx=0x7fb68000dcc0 comp rx=0 tx=0).stop 2024-04-25T14:05:17.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.788+0000 7fb693c00640 1 -- 172.21.15.86:0/3130051859 shutdown_connections 2024-04-25T14:05:17.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.788+0000 7fb693c00640 1 --2- 172.21.15.86:0/3130051859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb68c0d2c80 0x7fb68c1544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:17.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.788+0000 7fb693c00640 1 --2- 172.21.15.86:0/3130051859 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fb66c06f450 0x7fb66c071960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:17.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.788+0000 7fb693c00640 1 --2- 172.21.15.86:0/3130051859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb68c0722f0 0x7fb68c0cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:17.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.788+0000 7fb693c00640 1 --2- 172.21.15.86:0/3130051859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb68c070e20 0x7fb68c15e6a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:17.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.788+0000 7fb693c00640 1 -- 172.21.15.86:0/3130051859 >> 172.21.15.86:0/3130051859 conn(0x7fb68c0cffc0 msgr2=0x7fb68c06d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:17.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.788+0000 7fb693c00640 1 -- 172.21.15.86:0/3130051859 shutdown_connections 2024-04-25T14:05:17.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.788+0000 7fb693c00640 1 -- 172.21.15.86:0/3130051859 wait complete. 2024-04-25T14:05:17.810 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:05:04.472914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:05:05.604280+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:05:17.811 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:05:17.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.964+0000 7fcaa6000640 1 Processor -- start 2024-04-25T14:05:17.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.964+0000 7fcaa6000640 1 -- start start 2024-04-25T14:05:17.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.964+0000 7fcaa6000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcaa00ccab0 0x7fcaa00d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:17.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.964+0000 7fcaa6000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaa0162610 con 0x7fcaa00b79c0 2024-04-25T14:05:17.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.964+0000 7fcaa6000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaa00b3bd0 con 0x7fcaa00cf9d0 2024-04-25T14:05:17.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.964+0000 7fcaa6000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fcaa00cf190 con 0x7fcaa00ccab0 2024-04-25T14:05:17.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.964+0000 7fca9ea00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fcaa00b79c0 0x7fcaa01617f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37262/0 (socket says 172.21.15.86:37262) 2024-04-25T14:05:17.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.964+0000 7fca9ea00640 1 -- 172.21.15.86:0/2202634281 learned_addr learned my addr 172.21.15.86:0/2202634281 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:17.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.964+0000 7fca9f400640 1 --2- 172.21.15.86:0/2202634281 >> v2:172.21.15.155:3300/0 conn(0x7fcaa00ccab0 0x7fcaa00d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:17.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.964+0000 7fca9e000640 1 -- 172.21.15.86:0/2202634281 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 139665536 0 0) 0x7fcaa0162610 con 0x7fcaa00b79c0 2024-04-25T14:05:17.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.964+0000 7fca9e000640 1 -- 172.21.15.86:0/2202634281 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca84003530 con 0x7fcaa00b79c0 2024-04-25T14:05:17.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.964+0000 7fca9f400640 1 -- 172.21.15.86:0/2202634281 >> v1:172.21.15.155:6789/0 conn(0x7fcaa00cf9d0 legacy=0x7fcaa0161f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:17.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.964+0000 7fca9f400640 1 -- 172.21.15.86:0/2202634281 >> v1:172.21.15.86:6789/0 conn(0x7fcaa00b79c0 legacy=0x7fcaa01617f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:17.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.964+0000 7fca9f400640 1 -- 172.21.15.86:0/2202634281 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa00b1cf0 con 0x7fcaa00ccab0 2024-04-25T14:05:17.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.968+0000 7fca9f400640 1 --2- 172.21.15.86:0/2202634281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcaa00ccab0 0x7fcaa00d23d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fca900097a0 tx=0x7fca9001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=877099554505280a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:17.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.968+0000 7fca9e000640 1 -- 172.21.15.86:0/2202634281 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca900b1070 con 0x7fcaa00ccab0 2024-04-25T14:05:17.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.968+0000 7fca9e000640 1 -- 172.21.15.86:0/2202634281 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca900ac5f0 con 0x7fcaa00ccab0 2024-04-25T14:05:17.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.968+0000 7fca9e000640 1 -- 172.21.15.86:0/2202634281 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca900ac9a0 con 0x7fcaa00ccab0 2024-04-25T14:05:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.968+0000 7fcaa6000640 1 -- 172.21.15.86:0/2202634281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcaa00ccab0 msgr2=0x7fcaa00d23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.968+0000 7fcaa6000640 1 --2- 172.21.15.86:0/2202634281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcaa00ccab0 0x7fcaa00d23d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fca900097a0 tx=0x7fca9001c480 comp rx=0 tx=0).stop 2024-04-25T14:05:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.968+0000 7fcaa6000640 1 -- 172.21.15.86:0/2202634281 shutdown_connections 2024-04-25T14:05:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.968+0000 7fcaa6000640 1 --2- 172.21.15.86:0/2202634281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcaa00ccab0 0x7fcaa00d23d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.968+0000 7fcaa6000640 1 -- 172.21.15.86:0/2202634281 >> 172.21.15.86:0/2202634281 conn(0x7fcaa009ba30 msgr2=0x7fcaa00b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.968+0000 7fcaa6000640 1 -- 172.21.15.86:0/2202634281 shutdown_connections 2024-04-25T14:05:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.968+0000 7fcaa6000640 1 -- 172.21.15.86:0/2202634281 wait complete. 2024-04-25T14:05:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.972+0000 7fcaa6000640 1 Processor -- start 2024-04-25T14:05:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.972+0000 7fcaa6000640 1 -- start start 2024-04-25T14:05:17.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.972+0000 7fcaa6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa00b79c0 0x7fcaa0154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:17.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.972+0000 7fcaa6000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcaa00ccab0 0x7fcaa0155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:17.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.972+0000 7fcaa6000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcaa00cf9d0 0x7fcaa00c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:17.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.972+0000 7fcaa6000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa00d2d20 con 0x7fcaa00b79c0 2024-04-25T14:05:17.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.972+0000 7fcaa6000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa00b2620 con 0x7fcaa00cf9d0 2024-04-25T14:05:17.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.972+0000 7fcaa6000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcaa016c5b0 con 0x7fcaa00ccab0 2024-04-25T14:05:17.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.972+0000 7fca9f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa00b79c0 0x7fcaa0154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:17.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.972+0000 7fca9f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa00b79c0 0x7fcaa0154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57714/0 (socket says 172.21.15.86:57714) 2024-04-25T14:05:17.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.972+0000 7fca9f400640 1 -- 172.21.15.86:0/2660196503 learned_addr learned my addr 172.21.15.86:0/2660196503 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:17.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.972+0000 7fca9ea00640 1 --2- 172.21.15.86:0/2660196503 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcaa00ccab0 0x7fcaa0155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:17.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.972+0000 7fca9fe00640 1 --2- 172.21.15.86:0/2660196503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcaa00cf9d0 0x7fcaa00c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:17.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.976+0000 7fca9f400640 1 -- 172.21.15.86:0/2660196503 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcaa00ccab0 msgr2=0x7fcaa0155130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:17.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.976+0000 7fca9f400640 1 --2- 172.21.15.86:0/2660196503 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcaa00ccab0 0x7fcaa0155130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:17.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.976+0000 7fca9f400640 1 -- 172.21.15.86:0/2660196503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcaa00cf9d0 msgr2=0x7fcaa00c7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:17.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.976+0000 7fca9f400640 1 --2- 172.21.15.86:0/2660196503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcaa00cf9d0 0x7fcaa00c7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:17.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.976+0000 7fca9f400640 1 -- 172.21.15.86:0/2660196503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa0157fe0 con 0x7fcaa00b79c0 2024-04-25T14:05:17.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.976+0000 7fca9f400640 1 --2- 172.21.15.86:0/2660196503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa00b79c0 0x7fcaa0154bf0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fca9001ab30 tx=0x7fca9001acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:17.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.976+0000 7fca97400640 1 -- 172.21.15.86:0/2660196503 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca900b1050 con 0x7fcaa00b79c0 2024-04-25T14:05:17.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.976+0000 7fca97400640 1 -- 172.21.15.86:0/2660196503 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca900b2040 con 0x7fcaa00b79c0 2024-04-25T14:05:17.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.976+0000 7fcaa6000640 1 -- 172.21.15.86:0/2660196503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa01582c0 con 0x7fcaa00b79c0 2024-04-25T14:05:17.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.976+0000 7fca97400640 1 -- 172.21.15.86:0/2660196503 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90006dc0 con 0x7fcaa00b79c0 2024-04-25T14:05:17.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.976+0000 7fcaa6000640 1 -- 172.21.15.86:0/2660196503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa0158880 con 0x7fcaa00b79c0 2024-04-25T14:05:17.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.980+0000 7fcaa6000640 1 -- 172.21.15.86:0/2660196503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaa00d2f90 con 0x7fcaa00b79c0 2024-04-25T14:05:17.981 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:05:17.978+0000 7f28af07d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:05:17.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.980+0000 7fca97400640 1 -- 172.21.15.86:0/2660196503 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 6) ==== 93216+0+0 (secure 0 0 0) 0x7fca90004230 con 0x7fcaa00b79c0 2024-04-25T14:05:17.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.980+0000 7fca97400640 1 --2- 172.21.15.86:0/2660196503 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fca8006f450 0x7fca80071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:17.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.980+0000 7fca9ea00640 1 -- 172.21.15.86:0/2660196503 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fca8006f450 msgr2=0x7fca80071960 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2968869755 2024-04-25T14:05:17.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.980+0000 7fca97400640 1 -- 172.21.15.86:0/2660196503 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fca90128ce0 con 0x7fcaa00b79c0 2024-04-25T14:05:17.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.980+0000 7fca9ea00640 1 --2- 172.21.15.86:0/2660196503 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fca8006f450 0x7fca80071960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:05:17.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:17.988+0000 7fca97400640 1 -- 172.21.15.86:0/2660196503 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fca900be060 con 0x7fcaa00b79c0 2024-04-25T14:05:18.015 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:18.012+0000 7f7fa3e2c180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:05:18.129 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:05:18.126+0000 7f28af07d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:05:18.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.148+0000 7fcaa6000640 1 -- 172.21.15.86:0/2660196503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcaa0076bf0 con 0x7fcaa00b79c0 2024-04-25T14:05:18.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.148+0000 7fca97400640 1 -- 172.21.15.86:0/2660196503 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) ==== 93+0+10 (secure 0 0 0) 0x7fca900a9070 con 0x7fcaa00b79c0 2024-04-25T14:05:18.151 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:05:18.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.152+0000 7fcaa6000640 1 -- 172.21.15.86:0/2660196503 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fca8006f450 msgr2=0x7fca80071960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:05:18.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.152+0000 7fcaa6000640 1 --2- 172.21.15.86:0/2660196503 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fca8006f450 0x7fca80071960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:18.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.152+0000 7fcaa6000640 1 -- 172.21.15.86:0/2660196503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa00b79c0 msgr2=0x7fcaa0154bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:18.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.152+0000 7fcaa6000640 1 --2- 172.21.15.86:0/2660196503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa00b79c0 0x7fcaa0154bf0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fca9001ab30 tx=0x7fca9001acc0 comp rx=0 tx=0).stop 2024-04-25T14:05:18.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.152+0000 7fcaa6000640 1 -- 172.21.15.86:0/2660196503 shutdown_connections 2024-04-25T14:05:18.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.152+0000 7fcaa6000640 1 --2- 172.21.15.86:0/2660196503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcaa00cf9d0 0x7fcaa00c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:18.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.152+0000 7fcaa6000640 1 --2- 172.21.15.86:0/2660196503 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fca8006f450 0x7fca80071960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:18.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.152+0000 7fcaa6000640 1 --2- 172.21.15.86:0/2660196503 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcaa00ccab0 0x7fcaa0155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:18.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.152+0000 7fcaa6000640 1 --2- 172.21.15.86:0/2660196503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa00b79c0 0x7fcaa0154bf0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:18.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.152+0000 7fcaa6000640 1 -- 172.21.15.86:0/2660196503 >> 172.21.15.86:0/2660196503 conn(0x7fcaa009ba30 msgr2=0x7fcaa00b0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:18.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.152+0000 7fcaa6000640 1 -- 172.21.15.86:0/2660196503 shutdown_connections 2024-04-25T14:05:18.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.152+0000 7fcaa6000640 1 -- 172.21.15.86:0/2660196503 wait complete. 2024-04-25T14:05:18.166 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:18.164+0000 7f7fa3e2c180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:05:18.173 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:05:18.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.324+0000 7fc7c6800640 1 Processor -- start 2024-04-25T14:05:18.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.324+0000 7fc7c6800640 1 -- start start 2024-04-25T14:05:18.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.324+0000 7fc7c6800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc7c00cdd70 0x7fc7c00ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:18.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.324+0000 7fc7c6800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7c0070e20 0x7fc7c0071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:18.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.324+0000 7fc7c6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc7c006e700 0x7fc7c006eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:18.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.324+0000 7fc7c6800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc7c0071770 con 0x7fc7c006e700 2024-04-25T14:05:18.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.324+0000 7fc7c6800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc7c006f050 con 0x7fc7c00cdd70 2024-04-25T14:05:18.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.324+0000 7fc7c6800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc7c006d940 con 0x7fc7c0070e20 2024-04-25T14:05:18.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.324+0000 7fc7bfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc7c006e700 0x7fc7c006eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:18.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.324+0000 7fc7bf400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc7c00cdd70 0x7fc7c00ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:18.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.324+0000 7fc7bfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc7c006e700 0x7fc7c006eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57730/0 (socket says 172.21.15.86:57730) 2024-04-25T14:05:18.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.324+0000 7fc7bfe00640 1 -- 172.21.15.86:0/1277476839 learned_addr learned my addr 172.21.15.86:0/1277476839 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:18.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.324+0000 7fc7bea00640 1 --2- 172.21.15.86:0/1277476839 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7c0070e20 0x7fc7c0071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:18.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.328+0000 7fc7bfe00640 1 -- 172.21.15.86:0/1277476839 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7c0070e20 msgr2=0x7fc7c0071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:18.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.328+0000 7fc7bfe00640 1 --2- 172.21.15.86:0/1277476839 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7c0070e20 0x7fc7c0071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:18.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.328+0000 7fc7bfe00640 1 -- 172.21.15.86:0/1277476839 >> v2:172.21.15.155:3300/0 conn(0x7fc7c00cdd70 msgr2=0x7fc7c00ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:18.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.328+0000 7fc7bfe00640 1 --2- 172.21.15.86:0/1277476839 >> v2:172.21.15.155:3300/0 conn(0x7fc7c00cdd70 0x7fc7c00ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:18.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.328+0000 7fc7bfe00640 1 -- 172.21.15.86:0/1277476839 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c006dc20 con 0x7fc7c006e700 2024-04-25T14:05:18.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.328+0000 7fc7bf400640 1 --2- 172.21.15.86:0/1277476839 >> v2:172.21.15.155:3300/0 conn(0x7fc7c00cdd70 0x7fc7c00ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:18.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.328+0000 7fc7bfe00640 1 --2- 172.21.15.86:0/1277476839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7c006e700 0x7fc7c006eb10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0074610 tx=0x7fc7b801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91d1b5ac76cf8e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:18.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.328+0000 7fc7be000640 1 -- 172.21.15.86:0/1277476839 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b80a56d0 con 0x7fc7c006e700 2024-04-25T14:05:18.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.328+0000 7fc7be000640 1 -- 172.21.15.86:0/1277476839 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b80a5870 con 0x7fc7c006e700 2024-04-25T14:05:18.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.328+0000 7fc7be000640 1 -- 172.21.15.86:0/1277476839 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b80a5c40 con 0x7fc7c006e700 2024-04-25T14:05:18.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.332+0000 7fc7c6800640 1 -- 172.21.15.86:0/1277476839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7c006e700 msgr2=0x7fc7c006eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:18.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.332+0000 7fc7c6800640 1 --2- 172.21.15.86:0/1277476839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7c006e700 0x7fc7c006eb10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0074610 tx=0x7fc7b801c960 comp rx=0 tx=0).stop 2024-04-25T14:05:18.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.332+0000 7fc7c6800640 1 -- 172.21.15.86:0/1277476839 shutdown_connections 2024-04-25T14:05:18.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.332+0000 7fc7c6800640 1 --2- 172.21.15.86:0/1277476839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7c006e700 0x7fc7c006eb10 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:18.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.332+0000 7fc7c6800640 1 --2- 172.21.15.86:0/1277476839 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7c0070e20 0x7fc7c0071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:18.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.332+0000 7fc7c6800640 1 --2- 172.21.15.86:0/1277476839 >> v2:172.21.15.155:3300/0 conn(0x7fc7c00cdd70 0x7fc7c00ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:18.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.332+0000 7fc7c6800640 1 -- 172.21.15.86:0/1277476839 >> 172.21.15.86:0/1277476839 conn(0x7fc7c00cffc0 msgr2=0x7fc7c00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:18.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.332+0000 7fc7c6800640 1 -- 172.21.15.86:0/1277476839 shutdown_connections 2024-04-25T14:05:18.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.332+0000 7fc7c6800640 1 -- 172.21.15.86:0/1277476839 wait complete. 2024-04-25T14:05:18.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.332+0000 7fc7c6800640 1 Processor -- start 2024-04-25T14:05:18.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.332+0000 7fc7c6800640 1 -- start start 2024-04-25T14:05:18.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.336+0000 7fc7c6800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7c006e700 0x7fc7c014df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:18.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.336+0000 7fc7c6800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7c0070e20 0x7fc7c014c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:18.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.336+0000 7fc7c6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7c00cdd70 0x7fc7c014c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:18.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.336+0000 7fc7c6800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c01642f0 con 0x7fc7c00cdd70 2024-04-25T14:05:18.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.336+0000 7fc7c6800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c0164170 con 0x7fc7c006e700 2024-04-25T14:05:18.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.336+0000 7fc7c6800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc7c0164470 con 0x7fc7c0070e20 2024-04-25T14:05:18.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.336+0000 7fc7bfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7c00cdd70 0x7fc7c014c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:18.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.336+0000 7fc7bf400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7c006e700 0x7fc7c014df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:18.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.336+0000 7fc7bfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7c00cdd70 0x7fc7c014c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57742/0 (socket says 172.21.15.86:57742) 2024-04-25T14:05:18.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.336+0000 7fc7bfe00640 1 -- 172.21.15.86:0/4285637900 learned_addr learned my addr 172.21.15.86:0/4285637900 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:18.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.336+0000 7fc7bea00640 1 --2- 172.21.15.86:0/4285637900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7c0070e20 0x7fc7c014c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:18.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.336+0000 7fc7bfe00640 1 -- 172.21.15.86:0/4285637900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7c0070e20 msgr2=0x7fc7c014c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:18.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.336+0000 7fc7bfe00640 1 --2- 172.21.15.86:0/4285637900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7c0070e20 0x7fc7c014c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:18.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.336+0000 7fc7bfe00640 1 -- 172.21.15.86:0/4285637900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7c006e700 msgr2=0x7fc7c014df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:18.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.336+0000 7fc7bfe00640 1 --2- 172.21.15.86:0/4285637900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7c006e700 0x7fc7c014df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:18.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.336+0000 7fc7bfe00640 1 -- 172.21.15.86:0/4285637900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c01547d0 con 0x7fc7c00cdd70 2024-04-25T14:05:18.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.336+0000 7fc7bf400640 1 --2- 172.21.15.86:0/4285637900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7c006e700 0x7fc7c014df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:18.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.336+0000 7fc7bfe00640 1 --2- 172.21.15.86:0/4285637900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7c00cdd70 0x7fc7c014c6b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc7b80af040 tx=0x7fc7b8019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:18.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.336+0000 7fc7b7400640 1 -- 172.21.15.86:0/4285637900 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b80bc020 con 0x7fc7c00cdd70 2024-04-25T14:05:18.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.336+0000 7fc7b7400640 1 -- 172.21.15.86:0/4285637900 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b80b8690 con 0x7fc7c00cdd70 2024-04-25T14:05:18.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.336+0000 7fc7c6800640 1 -- 172.21.15.86:0/4285637900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7c0154ab0 con 0x7fc7c00cdd70 2024-04-25T14:05:18.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.336+0000 7fc7b7400640 1 -- 172.21.15.86:0/4285637900 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8009510 con 0x7fc7c00cdd70 2024-04-25T14:05:18.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.336+0000 7fc7c6800640 1 -- 172.21.15.86:0/4285637900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7c0155070 con 0x7fc7c00cdd70 2024-04-25T14:05:18.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.336+0000 7fc7c6800640 1 -- 172.21.15.86:0/4285637900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7c00720e0 con 0x7fc7c00cdd70 2024-04-25T14:05:18.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.340+0000 7fc7b7400640 1 -- 172.21.15.86:0/4285637900 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 6) ==== 93216+0+0 (secure 0 0 0) 0x7fc7b80bf020 con 0x7fc7c00cdd70 2024-04-25T14:05:18.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.340+0000 7fc7b7400640 1 --2- 172.21.15.86:0/4285637900 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fc7a006f450 0x7fc7a0071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:18.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.340+0000 7fc7bf400640 1 -- 172.21.15.86:0/4285637900 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fc7a006f450 msgr2=0x7fc7a0071960 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2968869755 2024-04-25T14:05:18.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.340+0000 7fc7b7400640 1 -- 172.21.15.86:0/4285637900 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fc7b812a1e0 con 0x7fc7c00cdd70 2024-04-25T14:05:18.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.340+0000 7fc7bf400640 1 --2- 172.21.15.86:0/4285637900 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fc7a006f450 0x7fc7a0071960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:05:18.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.344+0000 7fc7b7400640 1 -- 172.21.15.86:0/4285637900 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc7b80c6030 con 0x7fc7c00cdd70 2024-04-25T14:05:18.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.504+0000 7fc7c6800640 1 -- 172.21.15.86:0/4285637900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc7c0076bf0 con 0x7fc7c00cdd70 2024-04-25T14:05:18.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.504+0000 7fc7b7400640 1 -- 172.21.15.86:0/4285637900 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) ==== 104+0+11 (secure 0 0 0) 0x7fc7b80165e0 con 0x7fc7c00cdd70 2024-04-25T14:05:18.508 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:05:18.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.508+0000 7fc7c6800640 1 -- 172.21.15.86:0/4285637900 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fc7a006f450 msgr2=0x7fc7a0071960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:05:18.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.508+0000 7fc7c6800640 1 --2- 172.21.15.86:0/4285637900 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fc7a006f450 0x7fc7a0071960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:18.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.508+0000 7fc7c6800640 1 -- 172.21.15.86:0/4285637900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7c00cdd70 msgr2=0x7fc7c014c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:18.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.508+0000 7fc7c6800640 1 --2- 172.21.15.86:0/4285637900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7c00cdd70 0x7fc7c014c6b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc7b80af040 tx=0x7fc7b8019700 comp rx=0 tx=0).stop 2024-04-25T14:05:18.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.508+0000 7fc7c6800640 1 -- 172.21.15.86:0/4285637900 shutdown_connections 2024-04-25T14:05:18.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.508+0000 7fc7c6800640 1 --2- 172.21.15.86:0/4285637900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7c00cdd70 0x7fc7c014c6b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:18.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.508+0000 7fc7c6800640 1 --2- 172.21.15.86:0/4285637900 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fc7a006f450 0x7fc7a0071960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:18.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.508+0000 7fc7c6800640 1 --2- 172.21.15.86:0/4285637900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7c0070e20 0x7fc7c014c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:18.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.508+0000 7fc7c6800640 1 --2- 172.21.15.86:0/4285637900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7c006e700 0x7fc7c014df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:18.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.508+0000 7fc7c6800640 1 -- 172.21.15.86:0/4285637900 >> 172.21.15.86:0/4285637900 conn(0x7fc7c00cffc0 msgr2=0x7fc7c00cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:18.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.508+0000 7fc7c6800640 1 -- 172.21.15.86:0/4285637900 shutdown_connections 2024-04-25T14:05:18.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.508+0000 7fc7c6800640 1 -- 172.21.15.86:0/4285637900 wait complete. 2024-04-25T14:05:18.530 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:05:18.602 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:05:18.598+0000 7f28af07d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:05:18.659 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:18.656+0000 7f7fa3e2c180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:05:18.660 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:05:18.658+0000 7f28af07d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:05:18.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.680+0000 7fd2e3800640 1 Processor -- start 2024-04-25T14:05:18.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.680+0000 7fd2e3800640 1 -- start start 2024-04-25T14:05:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.680+0000 7fd2e3800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd2dc0cf970 0x7fd2dc070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.680+0000 7fd2e3800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd2dc06e970 0x7fd2dc0cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.680+0000 7fd2e3800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2dc071770 con 0x7fd2dc071360 2024-04-25T14:05:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.680+0000 7fd2e3800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd2dc163830 con 0x7fd2dc0cf970 2024-04-25T14:05:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.680+0000 7fd2e3800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd2dc06ef90 con 0x7fd2dc06e970 2024-04-25T14:05:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.680+0000 7fd2e0e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd2dc0cf970 0x7fd2dc070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.680+0000 7fd2dbe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd2dc071360 0x7fd2dc06e260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37278/0 (socket says 172.21.15.86:37278) 2024-04-25T14:05:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.680+0000 7fd2e0e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd2dc0cf970 0x7fd2dc070e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57752/0 (socket says 172.21.15.86:57752) 2024-04-25T14:05:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.680+0000 7fd2dbe00640 1 -- 172.21.15.86:0/3404928222 learned_addr learned my addr 172.21.15.86:0/3404928222 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.680+0000 7fd2e1800640 1 --2- 172.21.15.86:0/3404928222 >> v2:172.21.15.155:3300/0 conn(0x7fd2dc06e970 0x7fd2dc0cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.680+0000 7fd2e0e00640 1 -- 172.21.15.86:0/3404928222 >> v2:172.21.15.155:3300/0 conn(0x7fd2dc06e970 msgr2=0x7fd2dc0cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.680+0000 7fd2e0e00640 1 --2- 172.21.15.86:0/3404928222 >> v2:172.21.15.155:3300/0 conn(0x7fd2dc06e970 0x7fd2dc0cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.680+0000 7fd2e0e00640 1 -- 172.21.15.86:0/3404928222 >> v1:172.21.15.86:6789/0 conn(0x7fd2dc071360 legacy=0x7fd2dc06e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.680+0000 7fd2e0e00640 1 -- 172.21.15.86:0/3404928222 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2dc163a80 con 0x7fd2dc0cf970 2024-04-25T14:05:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.680+0000 7fd2e1800640 1 --2- 172.21.15.86:0/3404928222 >> v2:172.21.15.155:3300/0 conn(0x7fd2dc06e970 0x7fd2dc0cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:18.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.684+0000 7fd2e0e00640 1 --2- 172.21.15.86:0/3404928222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2dc0cf970 0x7fd2dc070e20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc1639b0 tx=0x7fd2d001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f9f23dc935ad6f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.684+0000 7fd2db400640 1 -- 172.21.15.86:0/3404928222 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d00b0070 con 0x7fd2dc0cf970 2024-04-25T14:05:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.684+0000 7fd2db400640 1 -- 172.21.15.86:0/3404928222 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2d0004230 con 0x7fd2dc0cf970 2024-04-25T14:05:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.684+0000 7fd2db400640 1 -- 172.21.15.86:0/3404928222 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0003620 con 0x7fd2dc0cf970 2024-04-25T14:05:18.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.684+0000 7fd2e3800640 1 -- 172.21.15.86:0/3404928222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2dc0cf970 msgr2=0x7fd2dc070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:18.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.684+0000 7fd2e3800640 1 --2- 172.21.15.86:0/3404928222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2dc0cf970 0x7fd2dc070e20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc1639b0 tx=0x7fd2d001c460 comp rx=0 tx=0).stop 2024-04-25T14:05:18.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.684+0000 7fd2e3800640 1 -- 172.21.15.86:0/3404928222 shutdown_connections 2024-04-25T14:05:18.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.684+0000 7fd2e3800640 1 --2- 172.21.15.86:0/3404928222 >> v2:172.21.15.155:3300/0 conn(0x7fd2dc06e970 0x7fd2dc0cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:18.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.684+0000 7fd2e3800640 1 --2- 172.21.15.86:0/3404928222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2dc0cf970 0x7fd2dc070e20 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:18.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.684+0000 7fd2e3800640 1 -- 172.21.15.86:0/3404928222 >> 172.21.15.86:0/3404928222 conn(0x7fd2dc0cffc0 msgr2=0x7fd2dc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:18.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.684+0000 7fd2e3800640 1 -- 172.21.15.86:0/3404928222 shutdown_connections 2024-04-25T14:05:18.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.684+0000 7fd2e3800640 1 -- 172.21.15.86:0/3404928222 wait complete. 2024-04-25T14:05:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.688+0000 7fd2e3800640 1 Processor -- start 2024-04-25T14:05:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.688+0000 7fd2e3800640 1 -- start start 2024-04-25T14:05:18.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.688+0000 7fd2e3800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2dc06e970 0x7fd2dc150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:18.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.688+0000 7fd2e3800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2dc071360 0x7fd2dc0cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:18.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.688+0000 7fd2e3800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2dc0cf970 0x7fd2dc0cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:18.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.688+0000 7fd2e3800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd2dc16cbd0 con 0x7fd2dc06e970 2024-04-25T14:05:18.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.688+0000 7fd2e3800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd2dc0ced80 con 0x7fd2dc0cf970 2024-04-25T14:05:18.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.688+0000 7fd2e3800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd2dc16cd50 con 0x7fd2dc071360 2024-04-25T14:05:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.688+0000 7fd2e0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2dc06e970 0x7fd2dc150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.688+0000 7fd2e0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2dc06e970 0x7fd2dc150960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57760/0 (socket says 172.21.15.86:57760) 2024-04-25T14:05:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.688+0000 7fd2e0e00640 1 -- 172.21.15.86:0/3915651924 learned_addr learned my addr 172.21.15.86:0/3915651924 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.688+0000 7fd2dbe00640 1 --2- 172.21.15.86:0/3915651924 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2dc071360 0x7fd2dc0cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.688+0000 7fd2e1800640 1 --2- 172.21.15.86:0/3915651924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2dc0cf970 0x7fd2dc0cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.692+0000 7fd2e0e00640 1 -- 172.21.15.86:0/3915651924 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2dc071360 msgr2=0x7fd2dc0cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.692+0000 7fd2e0e00640 1 --2- 172.21.15.86:0/3915651924 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2dc071360 0x7fd2dc0cc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.692+0000 7fd2e0e00640 1 -- 172.21.15.86:0/3915651924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2dc0cf970 msgr2=0x7fd2dc0cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.692+0000 7fd2e0e00640 1 --2- 172.21.15.86:0/3915651924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2dc0cf970 0x7fd2dc0cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.692+0000 7fd2e0e00640 1 -- 172.21.15.86:0/3915651924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2dc150ea0 con 0x7fd2dc06e970 2024-04-25T14:05:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.692+0000 7fd2e1800640 1 --2- 172.21.15.86:0/3915651924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2dc0cf970 0x7fd2dc0cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.692+0000 7fd2e0e00640 1 --2- 172.21.15.86:0/3915651924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2dc06e970 0x7fd2dc150960 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00ae040 tx=0x7fd2d0019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.692+0000 7fd2d8c00640 1 -- 172.21.15.86:0/3915651924 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d00b0040 con 0x7fd2dc06e970 2024-04-25T14:05:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.692+0000 7fd2e3800640 1 -- 172.21.15.86:0/3915651924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2dc0ccf10 con 0x7fd2dc06e970 2024-04-25T14:05:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.692+0000 7fd2d8c00640 1 -- 172.21.15.86:0/3915651924 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2d00bb070 con 0x7fd2dc06e970 2024-04-25T14:05:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.692+0000 7fd2d8c00640 1 -- 172.21.15.86:0/3915651924 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0006e20 con 0x7fd2dc06e970 2024-04-25T14:05:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.692+0000 7fd2e3800640 1 -- 172.21.15.86:0/3915651924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2dc0ccc10 con 0x7fd2dc06e970 2024-04-25T14:05:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.692+0000 7fd2e3800640 1 -- 172.21.15.86:0/3915651924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2dc0720e0 con 0x7fd2dc06e970 2024-04-25T14:05:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.692+0000 7fd2d8c00640 1 -- 172.21.15.86:0/3915651924 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 6) ==== 93216+0+0 (secure 0 0 0) 0x7fd2d00b3070 con 0x7fd2dc06e970 2024-04-25T14:05:18.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.696+0000 7fd2d8c00640 1 --2- 172.21.15.86:0/3915651924 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd2bc06f480 0x7fd2bc071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:18.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.696+0000 7fd2dbe00640 1 -- 172.21.15.86:0/3915651924 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd2bc06f480 msgr2=0x7fd2bc071990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2968869755 2024-04-25T14:05:18.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.696+0000 7fd2d8c00640 1 -- 172.21.15.86:0/3915651924 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fd2d0129c40 con 0x7fd2dc06e970 2024-04-25T14:05:18.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.696+0000 7fd2dbe00640 1 --2- 172.21.15.86:0/3915651924 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd2bc06f480 0x7fd2bc071990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:05:18.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.700+0000 7fd2d8c00640 1 -- 172.21.15.86:0/3915651924 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd2d0016050 con 0x7fd2dc06e970 2024-04-25T14:05:18.718 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:05:18.714+0000 7f28af07d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:05:18.726 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:18.724+0000 7f7fa3e2c180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:05:18.783 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:18.780+0000 7f7fa3e2c180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:05:18.807 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:05:18.802+0000 7f28af07d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:05:18.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.860+0000 7fd2e3800640 1 -- 172.21.15.86:0/3915651924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd2dc076bf0 con 0x7fd2dc06e970 2024-04-25T14:05:18.865 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:05:18.862+0000 7f28af07d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:05:18.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.864+0000 7fd2d8c00640 1 -- 172.21.15.86:0/3915651924 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) ==== 100+0+11 (secure 0 0 0) 0x7fd2d00f69a0 con 0x7fd2dc06e970 2024-04-25T14:05:18.865 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:05:18.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.864+0000 7fd2e3800640 1 -- 172.21.15.86:0/3915651924 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd2bc06f480 msgr2=0x7fd2bc071990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:05:18.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.864+0000 7fd2e3800640 1 --2- 172.21.15.86:0/3915651924 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd2bc06f480 0x7fd2bc071990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:18.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.864+0000 7fd2e3800640 1 -- 172.21.15.86:0/3915651924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2dc06e970 msgr2=0x7fd2dc150960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:18.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.864+0000 7fd2e3800640 1 --2- 172.21.15.86:0/3915651924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2dc06e970 0x7fd2dc150960 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00ae040 tx=0x7fd2d0019200 comp rx=0 tx=0).stop 2024-04-25T14:05:18.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.864+0000 7fd2e3800640 1 -- 172.21.15.86:0/3915651924 shutdown_connections 2024-04-25T14:05:18.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.864+0000 7fd2e3800640 1 --2- 172.21.15.86:0/3915651924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2dc0cf970 0x7fd2dc0cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:18.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.864+0000 7fd2e3800640 1 --2- 172.21.15.86:0/3915651924 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fd2bc06f480 0x7fd2bc071990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:18.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.864+0000 7fd2e3800640 1 --2- 172.21.15.86:0/3915651924 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2dc071360 0x7fd2dc0cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:18.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.864+0000 7fd2e3800640 1 --2- 172.21.15.86:0/3915651924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2dc06e970 0x7fd2dc150960 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:18.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.864+0000 7fd2e3800640 1 -- 172.21.15.86:0/3915651924 >> 172.21.15.86:0/3915651924 conn(0x7fd2dc0cffc0 msgr2=0x7fd2dc15e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:18.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.864+0000 7fd2e3800640 1 -- 172.21.15.86:0/3915651924 shutdown_connections 2024-04-25T14:05:18.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:18.864+0000 7fd2e3800640 1 -- 172.21.15.86:0/3915651924 wait complete. 2024-04-25T14:05:18.872 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:18.868+0000 7f7fa3e2c180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:05:18.887 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T14:05:18.931 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:18.928+0000 7f7fa3e2c180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:05:18.948 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:05:18.946+0000 7f28af07d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:05:18.963 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:05:18.971 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:05:19.010 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:05:19.006+0000 7f28af07d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:05:19.021 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:19.020+0000 7f7fa3e2c180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:05:19.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.036+0000 7f0ea4800640 1 Processor -- start 2024-04-25T14:05:19.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.036+0000 7f0ea4800640 1 -- start start 2024-04-25T14:05:19.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.040+0000 7f0ea4800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0e9c071530 0x7f0e9c0ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.040+0000 7f0ea4800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0e9c06e700 0x7f0e9c06eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.040+0000 7f0ea4800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e9c06f050 con 0x7f0e9c0cdd70 2024-04-25T14:05:19.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.040+0000 7f0ea4800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0e9c06d940 con 0x7f0e9c06e700 2024-04-25T14:05:19.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.040+0000 7f0ea4800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0e9c06dc20 con 0x7f0e9c071530 2024-04-25T14:05:19.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.040+0000 7f0ea2800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0e9c06e700 0x7f0e9c06eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.040+0000 7f0ea1e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0e9c0cdd70 0x7f0e9c070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47294/0 (socket says 172.21.15.86:47294) 2024-04-25T14:05:19.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.040+0000 7f0ea2800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0e9c06e700 0x7f0e9c06eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43136/0 (socket says 172.21.15.86:43136) 2024-04-25T14:05:19.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.040+0000 7f0ea1e00640 1 -- 172.21.15.86:0/1117024945 learned_addr learned my addr 172.21.15.86:0/1117024945 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:19.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.040+0000 7f0ea1400640 1 --2- 172.21.15.86:0/1117024945 >> v2:172.21.15.155:3300/0 conn(0x7f0e9c071530 0x7f0e9c0ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.040+0000 7f0ea2800640 1 -- 172.21.15.86:0/1117024945 >> v2:172.21.15.155:3300/0 conn(0x7f0e9c071530 msgr2=0x7f0e9c0ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:19.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.040+0000 7f0ea2800640 1 --2- 172.21.15.86:0/1117024945 >> v2:172.21.15.155:3300/0 conn(0x7f0e9c071530 0x7f0e9c0ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.040+0000 7f0ea2800640 1 -- 172.21.15.86:0/1117024945 >> v1:172.21.15.86:6789/0 conn(0x7f0e9c0cdd70 legacy=0x7f0e9c070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.040+0000 7f0ea0a00640 1 -- 172.21.15.86:0/1117024945 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4030779902 0 0) 0x7f0e9c06f050 con 0x7f0e9c0cdd70 2024-04-25T14:05:19.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.040+0000 7f0ea2800640 1 -- 172.21.15.86:0/1117024945 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9c06df00 con 0x7f0e9c06e700 2024-04-25T14:05:19.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.044+0000 7f0ea2800640 1 --2- 172.21.15.86:0/1117024945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e9c06e700 0x7f0e9c06eb10 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c073230 tx=0x7f0e9801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f16402ec34e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:19.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.044+0000 7f0ea0a00640 1 -- 172.21.15.86:0/1117024945 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e980b2070 con 0x7f0e9c06e700 2024-04-25T14:05:19.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.044+0000 7f0ea0a00640 1 -- 172.21.15.86:0/1117024945 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e980a57b0 con 0x7f0e9c06e700 2024-04-25T14:05:19.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.044+0000 7f0ea0a00640 1 -- 172.21.15.86:0/1117024945 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e980a5b80 con 0x7f0e9c06e700 2024-04-25T14:05:19.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.048+0000 7f0ea4800640 1 -- 172.21.15.86:0/1117024945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e9c06e700 msgr2=0x7f0e9c06eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.048+0000 7f0ea4800640 1 --2- 172.21.15.86:0/1117024945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e9c06e700 0x7f0e9c06eb10 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c073230 tx=0x7f0e9801c8a0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.048+0000 7f0ea4800640 1 -- 172.21.15.86:0/1117024945 shutdown_connections 2024-04-25T14:05:19.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.048+0000 7f0ea4800640 1 --2- 172.21.15.86:0/1117024945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e9c06e700 0x7f0e9c06eb10 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.048+0000 7f0ea4800640 1 --2- 172.21.15.86:0/1117024945 >> v2:172.21.15.155:3300/0 conn(0x7f0e9c071530 0x7f0e9c0ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.048+0000 7f0ea4800640 1 -- 172.21.15.86:0/1117024945 >> 172.21.15.86:0/1117024945 conn(0x7f0e9c0cffc0 msgr2=0x7f0e9c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:19.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.048+0000 7f0ea4800640 1 -- 172.21.15.86:0/1117024945 shutdown_connections 2024-04-25T14:05:19.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.048+0000 7f0ea4800640 1 -- 172.21.15.86:0/1117024945 wait complete. 2024-04-25T14:05:19.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.048+0000 7f0ea4800640 1 Processor -- start 2024-04-25T14:05:19.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.048+0000 7f0ea4800640 1 -- start start 2024-04-25T14:05:19.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.052+0000 7f0ea4800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e9c06e700 0x7f0e9c14c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.052+0000 7f0ea4800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e9c0cdd70 0x7f0e9c0b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.052+0000 7f0ea4800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e9c0b6670 0x7f0e9c161da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.052+0000 7f0ea4800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c16cc00 con 0x7f0e9c0b6670 2024-04-25T14:05:19.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.052+0000 7f0ea4800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c164380 con 0x7f0e9c0cdd70 2024-04-25T14:05:19.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.052+0000 7f0ea4800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0e9c16cd80 con 0x7f0e9c06e700 2024-04-25T14:05:19.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.052+0000 7f0ea2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e9c0b6670 0x7f0e9c161da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.052+0000 7f0ea2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e9c0b6670 0x7f0e9c161da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43146/0 (socket says 172.21.15.86:43146) 2024-04-25T14:05:19.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.052+0000 7f0ea2800640 1 -- 172.21.15.86:0/4102898618 learned_addr learned my addr 172.21.15.86:0/4102898618 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:19.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.052+0000 7f0ea1400640 1 --2- 172.21.15.86:0/4102898618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e9c0cdd70 0x7f0e9c0b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.052+0000 7f0ea1e00640 1 --2- 172.21.15.86:0/4102898618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e9c06e700 0x7f0e9c14c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.052+0000 7f0ea2800640 1 -- 172.21.15.86:0/4102898618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e9c06e700 msgr2=0x7f0e9c14c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.052+0000 7f0ea2800640 1 --2- 172.21.15.86:0/4102898618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e9c06e700 0x7f0e9c14c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.052+0000 7f0ea2800640 1 -- 172.21.15.86:0/4102898618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e9c0cdd70 msgr2=0x7f0e9c0b6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.052+0000 7f0ea2800640 1 --2- 172.21.15.86:0/4102898618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e9c0cdd70 0x7f0e9c0b6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.052+0000 7f0ea2800640 1 -- 172.21.15.86:0/4102898618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9c0b6a80 con 0x7f0e9c0b6670 2024-04-25T14:05:19.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.052+0000 7f0ea2800640 1 --2- 172.21.15.86:0/4102898618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e9c0b6670 0x7f0e9c161da0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0e9801a4d0 tx=0x7f0e98019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:19.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.052+0000 7f0e96000640 1 -- 172.21.15.86:0/4102898618 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e980b2050 con 0x7f0e9c0b6670 2024-04-25T14:05:19.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.052+0000 7f0e96000640 1 -- 172.21.15.86:0/4102898618 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e980be070 con 0x7f0e9c0b6670 2024-04-25T14:05:19.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.052+0000 7f0e96000640 1 -- 172.21.15.86:0/4102898618 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e980ad890 con 0x7f0e9c0b6670 2024-04-25T14:05:19.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.052+0000 7f0ea4800640 1 -- 172.21.15.86:0/4102898618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e9c160d00 con 0x7f0e9c0b6670 2024-04-25T14:05:19.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.052+0000 7f0ea4800640 1 -- 172.21.15.86:0/4102898618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e9c160fe0 con 0x7f0e9c0b6670 2024-04-25T14:05:19.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.052+0000 7f0ea4800640 1 -- 172.21.15.86:0/4102898618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e9c0720e0 con 0x7f0e9c0b6670 2024-04-25T14:05:19.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.056+0000 7f0e96000640 1 -- 172.21.15.86:0/4102898618 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 6) ==== 93216+0+0 (secure 0 0 0) 0x7f0e980ada30 con 0x7f0e9c0b6670 2024-04-25T14:05:19.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.056+0000 7f0e96000640 1 --2- 172.21.15.86:0/4102898618 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0e8006f450 0x7f0e80071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.056+0000 7f0e96000640 1 -- 172.21.15.86:0/4102898618 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f0e9812a420 con 0x7f0e9c0b6670 2024-04-25T14:05:19.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.056+0000 7f0ea1e00640 1 -- 172.21.15.86:0/4102898618 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0e8006f450 msgr2=0x7f0e80071960 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2968869755 2024-04-25T14:05:19.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.056+0000 7f0ea1e00640 1 --2- 172.21.15.86:0/4102898618 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0e8006f450 0x7f0e80071960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:05:19.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.060+0000 7f0e96000640 1 -- 172.21.15.86:0/4102898618 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0e9c0b6a80 con 0x7f0e9c0b6670 2024-04-25T14:05:19.087 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:19.084+0000 7f7fa3e2c180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:05:19.164 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:05:19.162+0000 7f28af07d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:05:19.223 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:05:19.218+0000 7f28af07d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:05:19.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.228+0000 7f0ea4800640 1 -- 172.21.15.86:0/4102898618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0e9c076bf0 con 0x7f0e9c0b6670 2024-04-25T14:05:19.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.228+0000 7f0e96000640 1 -- 172.21.15.86:0/4102898618 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v38) ==== 80+0+0 (secure 0 0 0) 0x7f0e98016520 con 0x7f0e9c0b6670 2024-04-25T14:05:19.232 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-25T14:05:19.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.232+0000 7f0ea4800640 1 -- 172.21.15.86:0/4102898618 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0e8006f450 msgr2=0x7f0e80071960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:05:19.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.232+0000 7f0ea4800640 1 --2- 172.21.15.86:0/4102898618 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0e8006f450 0x7f0e80071960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.232+0000 7f0ea4800640 1 -- 172.21.15.86:0/4102898618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e9c0b6670 msgr2=0x7f0e9c161da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.232+0000 7f0ea4800640 1 --2- 172.21.15.86:0/4102898618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e9c0b6670 0x7f0e9c161da0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0e9801a4d0 tx=0x7f0e98019640 comp rx=0 tx=0).stop 2024-04-25T14:05:19.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.232+0000 7f0ea4800640 1 -- 172.21.15.86:0/4102898618 shutdown_connections 2024-04-25T14:05:19.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.232+0000 7f0ea4800640 1 --2- 172.21.15.86:0/4102898618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e9c0b6670 0x7f0e9c161da0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.232+0000 7f0ea4800640 1 --2- 172.21.15.86:0/4102898618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e9c0cdd70 0x7f0e9c0b6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.232+0000 7f0ea4800640 1 --2- 172.21.15.86:0/4102898618 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0e8006f450 0x7f0e80071960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.232+0000 7f0ea4800640 1 --2- 172.21.15.86:0/4102898618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e9c06e700 0x7f0e9c14c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.232+0000 7f0ea4800640 1 -- 172.21.15.86:0/4102898618 >> 172.21.15.86:0/4102898618 conn(0x7f0e9c0cffc0 msgr2=0x7f0e9c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:19.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.232+0000 7f0ea4800640 1 -- 172.21.15.86:0/4102898618 shutdown_connections 2024-04-25T14:05:19.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.232+0000 7f0ea4800640 1 -- 172.21.15.86:0/4102898618 wait complete. 2024-04-25T14:05:19.243 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:19.240+0000 7f7fa3e2c180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:05:19.254 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T14:05:19.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:19.254 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T14:05:19.303 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:19.300+0000 7f7fa3e2c180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:05:19.325 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T14:05:19.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:19.325 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T14:05:19.377 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:05:19.377 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:05:19.404 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:05:19.404 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:05:19.457 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:05:19.466 INFO:tasks.ceph.mgr.x.smithi155.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:05:19.466 INFO:tasks.ceph.mgr.x.smithi155.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:05:19.466 INFO:tasks.ceph.mgr.x.smithi155.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:05:19.475 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:05:19.470+0000 7f28af07d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:05:19.549 INFO:tasks.ceph.mgr.y.smithi086.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:05:19.549 INFO:tasks.ceph.mgr.y.smithi086.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:05:19.549 INFO:tasks.ceph.mgr.y.smithi086.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:05:19.558 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:19.556+0000 7f7fa3e2c180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:05:19.624 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:05:19.622+0000 7f28af07d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:05:19.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.648+0000 7f0d91a00640 1 Processor -- start 2024-04-25T14:05:19.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.648+0000 7f0d91a00640 1 -- start start 2024-04-25T14:05:19.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.652+0000 7f0d91a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d8c1635a0 0x7f0d8c06eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.652+0000 7f0d91a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0d8c06f060 0x7f0d8c0d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.652+0000 7f0d91a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0d8c0d06f0 0x7f0d8c0722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.652+0000 7f0d91a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0d8c06f470 con 0x7f0d8c0d06f0 2024-04-25T14:05:19.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.652+0000 7f0d91a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0d8c0d0b00 con 0x7f0d8c06f060 2024-04-25T14:05:19.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.652+0000 7f0d91a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0d8c163d80 con 0x7f0d8c1635a0 2024-04-25T14:05:19.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.652+0000 7f0d91000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0d8c0d06f0 0x7f0d8c0722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.652+0000 7f0d91000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0d8c0d06f0 0x7f0d8c0722a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43162/0 (socket says 172.21.15.86:43162) 2024-04-25T14:05:19.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.652+0000 7f0d83e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0d8c06f060 0x7f0d8c0d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.652+0000 7f0d91000640 1 -- 172.21.15.86:0/1217085784 learned_addr learned my addr 172.21.15.86:0/1217085784 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:19.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.652+0000 7f0d8be00640 1 --2- 172.21.15.86:0/1217085784 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d8c1635a0 0x7f0d8c06eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.652+0000 7f0d91000640 1 -- 172.21.15.86:0/1217085784 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d8c1635a0 msgr2=0x7f0d8c06eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:19.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.652+0000 7f0d91000640 1 --2- 172.21.15.86:0/1217085784 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d8c1635a0 0x7f0d8c06eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.652+0000 7f0d91000640 1 -- 172.21.15.86:0/1217085784 >> v2:172.21.15.155:3300/0 conn(0x7f0d8c06f060 msgr2=0x7f0d8c0d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:19.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.652+0000 7f0d91000640 1 --2- 172.21.15.86:0/1217085784 >> v2:172.21.15.155:3300/0 conn(0x7f0d8c06f060 0x7f0d8c0d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.652+0000 7f0d91000640 1 -- 172.21.15.86:0/1217085784 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8c164060 con 0x7f0d8c0d06f0 2024-04-25T14:05:19.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.652+0000 7f0d91000640 1 --2- 172.21.15.86:0/1217085784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d8c0d06f0 0x7f0d8c0722a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c0b5fa0 tx=0x7f0d8401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48213805d977d0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:19.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.656+0000 7f0d8b400640 1 -- 172.21.15.86:0/1217085784 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d840a5670 con 0x7f0d8c0d06f0 2024-04-25T14:05:19.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.656+0000 7f0d8b400640 1 -- 172.21.15.86:0/1217085784 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d840a5810 con 0x7f0d8c0d06f0 2024-04-25T14:05:19.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.656+0000 7f0d8b400640 1 -- 172.21.15.86:0/1217085784 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d840a5be0 con 0x7f0d8c0d06f0 2024-04-25T14:05:19.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.656+0000 7f0d91a00640 1 -- 172.21.15.86:0/1217085784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d8c0d06f0 msgr2=0x7f0d8c0722a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.656+0000 7f0d91a00640 1 --2- 172.21.15.86:0/1217085784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d8c0d06f0 0x7f0d8c0722a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c0b5fa0 tx=0x7f0d8401c9f0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.656+0000 7f0d91a00640 1 -- 172.21.15.86:0/1217085784 shutdown_connections 2024-04-25T14:05:19.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.656+0000 7f0d91a00640 1 --2- 172.21.15.86:0/1217085784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d8c0d06f0 0x7f0d8c0722a0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.656+0000 7f0d91a00640 1 --2- 172.21.15.86:0/1217085784 >> v2:172.21.15.155:3300/0 conn(0x7f0d8c06f060 0x7f0d8c0d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.656+0000 7f0d91a00640 1 --2- 172.21.15.86:0/1217085784 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d8c1635a0 0x7f0d8c06eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.656+0000 7f0d91a00640 1 -- 172.21.15.86:0/1217085784 >> 172.21.15.86:0/1217085784 conn(0x7f0d8c0b4f80 msgr2=0x7f0d8c0b5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:19.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.656+0000 7f0d91a00640 1 -- 172.21.15.86:0/1217085784 shutdown_connections 2024-04-25T14:05:19.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.656+0000 7f0d91a00640 1 -- 172.21.15.86:0/1217085784 wait complete. 2024-04-25T14:05:19.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.656+0000 7f0d91a00640 1 Processor -- start 2024-04-25T14:05:19.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.656+0000 7f0d91a00640 1 -- start start 2024-04-25T14:05:19.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.660+0000 7f0d91a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d8c06f060 0x7f0d8c157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.660+0000 7f0d91a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d8c0d06f0 0x7f0d8c152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.660+0000 7f0d91a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d8c1635a0 0x7f0d8c15ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.660+0000 7f0d91a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c1639b0 con 0x7f0d8c06f060 2024-04-25T14:05:19.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.660+0000 7f0d91a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c072bf0 con 0x7f0d8c0d06f0 2024-04-25T14:05:19.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.660+0000 7f0d91a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0d8c163b30 con 0x7f0d8c1635a0 2024-04-25T14:05:19.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.660+0000 7f0d8be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d8c06f060 0x7f0d8c157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.660+0000 7f0d8be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d8c06f060 0x7f0d8c157df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43164/0 (socket says 172.21.15.86:43164) 2024-04-25T14:05:19.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.660+0000 7f0d8be00640 1 -- 172.21.15.86:0/1812037151 learned_addr learned my addr 172.21.15.86:0/1812037151 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:19.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.660+0000 7f0d83e00640 1 --2- 172.21.15.86:0/1812037151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d8c0d06f0 0x7f0d8c152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.660+0000 7f0d91000640 1 --2- 172.21.15.86:0/1812037151 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d8c1635a0 0x7f0d8c15ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.660+0000 7f0d8be00640 1 -- 172.21.15.86:0/1812037151 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d8c1635a0 msgr2=0x7f0d8c15ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.660+0000 7f0d8be00640 1 --2- 172.21.15.86:0/1812037151 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d8c1635a0 0x7f0d8c15ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.660+0000 7f0d8be00640 1 -- 172.21.15.86:0/1812037151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d8c0d06f0 msgr2=0x7f0d8c152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.660+0000 7f0d8be00640 1 --2- 172.21.15.86:0/1812037151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d8c0d06f0 0x7f0d8c152d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.660+0000 7f0d8be00640 1 -- 172.21.15.86:0/1812037151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8c15fff0 con 0x7f0d8c06f060 2024-04-25T14:05:19.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.660+0000 7f0d8be00640 1 --2- 172.21.15.86:0/1812037151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d8c06f060 0x7f0d8c157df0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00e870 tx=0x7f0d7c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:19.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.660+0000 7f0d88c00640 1 -- 172.21.15.86:0/1812037151 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c017070 con 0x7f0d8c06f060 2024-04-25T14:05:19.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.660+0000 7f0d88c00640 1 -- 172.21.15.86:0/1812037151 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d7c00c8e0 con 0x7f0d8c06f060 2024-04-25T14:05:19.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.660+0000 7f0d88c00640 1 -- 172.21.15.86:0/1812037151 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c00ccb0 con 0x7f0d8c06f060 2024-04-25T14:05:19.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.660+0000 7f0d91a00640 1 -- 172.21.15.86:0/1812037151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d8c1604f0 con 0x7f0d8c06f060 2024-04-25T14:05:19.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.660+0000 7f0d91a00640 1 -- 172.21.15.86:0/1812037151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d8c0d1800 con 0x7f0d8c06f060 2024-04-25T14:05:19.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.660+0000 7f0d91a00640 1 -- 172.21.15.86:0/1812037151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d8c076b80 con 0x7f0d8c06f060 2024-04-25T14:05:19.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.664+0000 7f0d88c00640 1 -- 172.21.15.86:0/1812037151 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 6) ==== 93216+0+0 (secure 0 0 0) 0x7f0d7c01a070 con 0x7f0d8c06f060 2024-04-25T14:05:19.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.664+0000 7f0d88c00640 1 --2- 172.21.15.86:0/1812037151 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0d6806f480 0x7f0d68071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.664+0000 7f0d83e00640 1 -- 172.21.15.86:0/1812037151 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0d6806f480 msgr2=0x7f0d68071990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2968869755 2024-04-25T14:05:19.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.664+0000 7f0d88c00640 1 -- 172.21.15.86:0/1812037151 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f0d7c08f160 con 0x7f0d8c06f060 2024-04-25T14:05:19.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.664+0000 7f0d83e00640 1 --2- 172.21.15.86:0/1812037151 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0d6806f480 0x7f0d68071990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:05:19.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.668+0000 7f0d88c00640 1 -- 172.21.15.86:0/1812037151 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0d7c05b870 con 0x7f0d8c06f060 2024-04-25T14:05:19.716 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:19.712+0000 7f7fa3e2c180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:05:19.752 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:05:19.750+0000 7f28af07d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:05:19.824 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:05:19.818+0000 7f28af07d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:05:19.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.832+0000 7f0d91a00640 1 -- 172.21.15.86:0/1812037151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d8c15d580 con 0x7f0d8c06f060 2024-04-25T14:05:19.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.832+0000 7f0d88c00640 1 -- 172.21.15.86:0/1812037151 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16942 (secure 0 0 0) 0x7f0d7c05fb70 con 0x7f0d8c06f060 2024-04-25T14:05:19.834 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:19.834 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":38,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:05:08.245617+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:05:04.472914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T14:05:05.604280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:05:19.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.832+0000 7f0d91a00640 1 -- 172.21.15.86:0/1812037151 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0d6806f480 msgr2=0x7f0d68071990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:05:19.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.832+0000 7f0d91a00640 1 --2- 172.21.15.86:0/1812037151 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0d6806f480 0x7f0d68071990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.832+0000 7f0d91a00640 1 -- 172.21.15.86:0/1812037151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d8c06f060 msgr2=0x7f0d8c157df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.832+0000 7f0d91a00640 1 --2- 172.21.15.86:0/1812037151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d8c06f060 0x7f0d8c157df0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00e870 tx=0x7f0d7c00ec70 comp rx=0 tx=0).stop 2024-04-25T14:05:19.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.832+0000 7f0d91a00640 1 -- 172.21.15.86:0/1812037151 shutdown_connections 2024-04-25T14:05:19.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.832+0000 7f0d91a00640 1 --2- 172.21.15.86:0/1812037151 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f0d6806f480 0x7f0d68071990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.832+0000 7f0d91a00640 1 --2- 172.21.15.86:0/1812037151 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d8c1635a0 0x7f0d8c15ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.832+0000 7f0d91a00640 1 --2- 172.21.15.86:0/1812037151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d8c0d06f0 0x7f0d8c152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.832+0000 7f0d91a00640 1 --2- 172.21.15.86:0/1812037151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d8c06f060 0x7f0d8c157df0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.832+0000 7f0d91a00640 1 -- 172.21.15.86:0/1812037151 >> 172.21.15.86:0/1812037151 conn(0x7f0d8c0b4f80 msgr2=0x7f0d8c0d1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:19.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.832+0000 7f0d91a00640 1 -- 172.21.15.86:0/1812037151 shutdown_connections 2024-04-25T14:05:19.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:19.832+0000 7f0d91a00640 1 -- 172.21.15.86:0/1812037151 wait complete. 2024-04-25T14:05:19.845 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:19.844+0000 7f7fa3e2c180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:05:19.858 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:05:04.472914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:05:05.604280+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:05:19.859 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:05:19.920 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:19.916+0000 7f7fa3e2c180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:05:19.975 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:05:19.970+0000 7f28af07d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:05:20.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.008+0000 7f33d4600640 1 Processor -- start 2024-04-25T14:05:20.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.008+0000 7f33d4600640 1 -- start start 2024-04-25T14:05:20.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.012+0000 7f33d4600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f33cc0d2c80 0x7f33cc15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.012+0000 7f33d4600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33cc070e20 0x7f33cc071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.012+0000 7f33d4600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33cc071770 con 0x7f33cc0722f0 2024-04-25T14:05:20.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.012+0000 7f33d4600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f33cc0ce720 con 0x7f33cc0d2c80 2024-04-25T14:05:20.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.012+0000 7f33d4600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f33cc0cd8d0 con 0x7f33cc070e20 2024-04-25T14:05:20.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.012+0000 7f33d2600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f33cc0722f0 0x7f33cc072700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47304/0 (socket says 172.21.15.86:47304) 2024-04-25T14:05:20.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.012+0000 7f33d2600640 1 -- 172.21.15.86:0/1393772305 learned_addr learned my addr 172.21.15.86:0/1393772305 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:20.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.012+0000 7f33d1200640 1 --2- 172.21.15.86:0/1393772305 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33cc070e20 0x7f33cc071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.012+0000 7f33d1c00640 1 --2- 172.21.15.86:0/1393772305 >> v2:172.21.15.155:3300/0 conn(0x7f33cc0d2c80 0x7f33cc15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.012+0000 7f33cbe00640 1 -- 172.21.15.86:0/1393772305 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1343676497 0 0) 0x7f33c40014a0 con 0x7f33cc0722f0 2024-04-25T14:05:20.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.012+0000 7f33cbe00640 1 -- 172.21.15.86:0/1393772305 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33b4003530 con 0x7f33cc0722f0 2024-04-25T14:05:20.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.012+0000 7f33d1200640 1 -- 172.21.15.86:0/1393772305 >> v2:172.21.15.155:3300/0 conn(0x7f33cc0d2c80 msgr2=0x7f33cc15dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:20.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.012+0000 7f33d1200640 1 --2- 172.21.15.86:0/1393772305 >> v2:172.21.15.155:3300/0 conn(0x7f33cc0d2c80 0x7f33cc15dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.012+0000 7f33d1200640 1 -- 172.21.15.86:0/1393772305 >> v1:172.21.15.86:6789/0 conn(0x7f33cc0722f0 legacy=0x7f33cc072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.012+0000 7f33d1200640 1 -- 172.21.15.86:0/1393772305 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33cc0ce970 con 0x7f33cc070e20 2024-04-25T14:05:20.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.012+0000 7f33d1c00640 1 --2- 172.21.15.86:0/1393772305 >> v2:172.21.15.155:3300/0 conn(0x7f33cc0d2c80 0x7f33cc15dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:20.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.016+0000 7f33d1200640 1 --2- 172.21.15.86:0/1393772305 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33cc070e20 0x7f33cc071230 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f33b80097a0 tx=0x7f33b801ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d81772d1768bd334 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:20.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.016+0000 7f33cbe00640 1 -- 172.21.15.86:0/1393772305 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b80b5070 con 0x7f33cc070e20 2024-04-25T14:05:20.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.016+0000 7f33cbe00640 1 -- 172.21.15.86:0/1393772305 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b80a8970 con 0x7f33cc070e20 2024-04-25T14:05:20.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.016+0000 7f33cbe00640 1 -- 172.21.15.86:0/1393772305 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b80a8d20 con 0x7f33cc070e20 2024-04-25T14:05:20.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.016+0000 7f33d4600640 1 -- 172.21.15.86:0/1393772305 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33cc070e20 msgr2=0x7f33cc071230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.016+0000 7f33d4600640 1 --2- 172.21.15.86:0/1393772305 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33cc070e20 0x7f33cc071230 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f33b80097a0 tx=0x7f33b801ec70 comp rx=0 tx=0).stop 2024-04-25T14:05:20.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.016+0000 7f33d4600640 1 -- 172.21.15.86:0/1393772305 shutdown_connections 2024-04-25T14:05:20.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.016+0000 7f33d4600640 1 --2- 172.21.15.86:0/1393772305 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33cc070e20 0x7f33cc071230 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.016+0000 7f33d4600640 1 --2- 172.21.15.86:0/1393772305 >> v2:172.21.15.155:3300/0 conn(0x7f33cc0d2c80 0x7f33cc15dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.016+0000 7f33d4600640 1 -- 172.21.15.86:0/1393772305 >> 172.21.15.86:0/1393772305 conn(0x7f33cc0cffc0 msgr2=0x7f33cc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:20.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.016+0000 7f33d4600640 1 -- 172.21.15.86:0/1393772305 shutdown_connections 2024-04-25T14:05:20.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.016+0000 7f33d4600640 1 -- 172.21.15.86:0/1393772305 wait complete. 2024-04-25T14:05:20.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.016+0000 7f33d4600640 1 Processor -- start 2024-04-25T14:05:20.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.016+0000 7f33d4600640 1 -- start start 2024-04-25T14:05:20.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.020+0000 7f33d4600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33cc070e20 0x7f33cc152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.020+0000 7f33d4600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33cc0722f0 0x7f33cc15c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.020+0000 7f33d4600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33cc0d2c80 0x7f33cc15c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.020+0000 7f33d4600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc0d21a0 con 0x7f33cc070e20 2024-04-25T14:05:20.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.020+0000 7f33d4600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc0d1ec0 con 0x7f33cc0722f0 2024-04-25T14:05:20.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.020+0000 7f33d4600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f33cc0d2480 con 0x7f33cc0d2c80 2024-04-25T14:05:20.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.020+0000 7f33d1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33cc070e20 0x7f33cc152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.020+0000 7f33d1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33cc070e20 0x7f33cc152880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43170/0 (socket says 172.21.15.86:43170) 2024-04-25T14:05:20.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.020+0000 7f33d1c00640 1 -- 172.21.15.86:0/4259907677 learned_addr learned my addr 172.21.15.86:0/4259907677 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:20.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.020+0000 7f33d1200640 1 --2- 172.21.15.86:0/4259907677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33cc0722f0 0x7f33cc15c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.020+0000 7f33d2600640 1 --2- 172.21.15.86:0/4259907677 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33cc0d2c80 0x7f33cc15c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.020+0000 7f33d1c00640 1 -- 172.21.15.86:0/4259907677 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33cc0d2c80 msgr2=0x7f33cc15c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.020+0000 7f33d1c00640 1 --2- 172.21.15.86:0/4259907677 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33cc0d2c80 0x7f33cc15c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.020+0000 7f33d1c00640 1 -- 172.21.15.86:0/4259907677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33cc0722f0 msgr2=0x7f33cc15c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.020+0000 7f33d1c00640 1 --2- 172.21.15.86:0/4259907677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33cc0722f0 0x7f33cc15c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.020+0000 7f33d1c00640 1 -- 172.21.15.86:0/4259907677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33cc1f6820 con 0x7f33cc070e20 2024-04-25T14:05:20.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.020+0000 7f33d1200640 1 --2- 172.21.15.86:0/4259907677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33cc0722f0 0x7f33cc15c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:20.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.020+0000 7f33d1c00640 1 --2- 172.21.15.86:0/4259907677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33cc070e20 0x7f33cc152880 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f33c000e9b0 tx=0x7f33c000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:20.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.020+0000 7f33c9600640 1 -- 172.21.15.86:0/4259907677 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c0017070 con 0x7f33cc070e20 2024-04-25T14:05:20.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.020+0000 7f33c9600640 1 -- 172.21.15.86:0/4259907677 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33c000ca00 con 0x7f33cc070e20 2024-04-25T14:05:20.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.020+0000 7f33c9600640 1 -- 172.21.15.86:0/4259907677 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c000cdd0 con 0x7f33cc070e20 2024-04-25T14:05:20.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.020+0000 7f33d4600640 1 -- 172.21.15.86:0/4259907677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33cc159340 con 0x7f33cc070e20 2024-04-25T14:05:20.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.020+0000 7f33d4600640 1 -- 172.21.15.86:0/4259907677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33cc15ef60 con 0x7f33cc070e20 2024-04-25T14:05:20.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.024+0000 7f33d4600640 1 -- 172.21.15.86:0/4259907677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33cc076bf0 con 0x7f33cc070e20 2024-04-25T14:05:20.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.024+0000 7f33c9600640 1 -- 172.21.15.86:0/4259907677 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 6) ==== 93216+0+0 (secure 0 0 0) 0x7f33c001a070 con 0x7f33cc070e20 2024-04-25T14:05:20.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.024+0000 7f33c9600640 1 --2- 172.21.15.86:0/4259907677 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f33ac06f480 0x7f33ac071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.024+0000 7f33d1200640 1 -- 172.21.15.86:0/4259907677 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f33ac06f480 msgr2=0x7f33ac071990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2968869755 2024-04-25T14:05:20.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.024+0000 7f33c9600640 1 -- 172.21.15.86:0/4259907677 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f33c008f310 con 0x7f33cc070e20 2024-04-25T14:05:20.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.024+0000 7f33d1200640 1 --2- 172.21.15.86:0/4259907677 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f33ac06f480 0x7f33ac071990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:05:20.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.028+0000 7f33c9600640 1 -- 172.21.15.86:0/4259907677 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f33c005b980 con 0x7f33cc070e20 2024-04-25T14:05:20.083 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:20.080+0000 7f7fa3e2c180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:05:20.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.196+0000 7f33d4600640 1 -- 172.21.15.86:0/4259907677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f33cc0d1470 con 0x7f33cc070e20 2024-04-25T14:05:20.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.196+0000 7f33c9600640 1 -- 172.21.15.86:0/4259907677 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) ==== 93+0+10 (secure 0 0 0) 0x7f33c005fc80 con 0x7f33cc070e20 2024-04-25T14:05:20.199 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:05:20.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.200+0000 7f33d4600640 1 -- 172.21.15.86:0/4259907677 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f33ac06f480 msgr2=0x7f33ac071990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:05:20.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.200+0000 7f33d4600640 1 --2- 172.21.15.86:0/4259907677 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f33ac06f480 0x7f33ac071990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.200+0000 7f33d4600640 1 -- 172.21.15.86:0/4259907677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33cc070e20 msgr2=0x7f33cc152880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.200+0000 7f33d4600640 1 --2- 172.21.15.86:0/4259907677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33cc070e20 0x7f33cc152880 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f33c000e9b0 tx=0x7f33c000ee70 comp rx=0 tx=0).stop 2024-04-25T14:05:20.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.200+0000 7f33d4600640 1 -- 172.21.15.86:0/4259907677 shutdown_connections 2024-04-25T14:05:20.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.200+0000 7f33d4600640 1 --2- 172.21.15.86:0/4259907677 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f33ac06f480 0x7f33ac071990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.200+0000 7f33d4600640 1 --2- 172.21.15.86:0/4259907677 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33cc0d2c80 0x7f33cc15c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.200+0000 7f33d4600640 1 --2- 172.21.15.86:0/4259907677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33cc0722f0 0x7f33cc15c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.200+0000 7f33d4600640 1 --2- 172.21.15.86:0/4259907677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33cc070e20 0x7f33cc152880 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.200+0000 7f33d4600640 1 -- 172.21.15.86:0/4259907677 >> 172.21.15.86:0/4259907677 conn(0x7f33cc0cffc0 msgr2=0x7f33cc0cdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:20.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.200+0000 7f33d4600640 1 -- 172.21.15.86:0/4259907677 shutdown_connections 2024-04-25T14:05:20.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.200+0000 7f33d4600640 1 -- 172.21.15.86:0/4259907677 wait complete. 2024-04-25T14:05:20.221 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:05:20.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.372+0000 7f1e9b400640 1 Processor -- start 2024-04-25T14:05:20.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.372+0000 7f1e9b400640 1 -- start start 2024-04-25T14:05:20.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.372+0000 7f1e9b400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1e94071530 0x7f1e9415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.372+0000 7f1e9b400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e940cd8d0 con 0x7f1e940d2c80 2024-04-25T14:05:20.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.372+0000 7f1e9b400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e940ce970 con 0x7f1e941616e0 2024-04-25T14:05:20.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.372+0000 7f1e9b400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1e940ce720 con 0x7f1e94071530 2024-04-25T14:05:20.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.372+0000 7f1e98a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1e940d2c80 0x7f1e94070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47320/0 (socket says 172.21.15.86:47320) 2024-04-25T14:05:20.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.372+0000 7f1e98a00640 1 -- 172.21.15.86:0/2655817230 learned_addr learned my addr 172.21.15.86:0/2655817230 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:20.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.372+0000 7f1e93e00640 1 --2- 172.21.15.86:0/2655817230 >> v2:172.21.15.155:3300/0 conn(0x7f1e94071530 0x7f1e9415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.372+0000 7f1e93400640 1 -- 172.21.15.86:0/2655817230 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4215012373 0 0) 0x7f1e940cd8d0 con 0x7f1e940d2c80 2024-04-25T14:05:20.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.372+0000 7f1e93400640 1 -- 172.21.15.86:0/2655817230 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e7c003530 con 0x7f1e940d2c80 2024-04-25T14:05:20.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.376+0000 7f1e93400640 1 -- 172.21.15.86:0/2655817230 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3945684174 0 0) 0x7f1e940ce970 con 0x7f1e941616e0 2024-04-25T14:05:20.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.376+0000 7f1e93400640 1 -- 172.21.15.86:0/2655817230 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e940cd8d0 con 0x7f1e941616e0 2024-04-25T14:05:20.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.376+0000 7f1e93400640 1 -- 172.21.15.86:0/2655817230 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1859487427 0 0) 0x7f1e7c003530 con 0x7f1e940d2c80 2024-04-25T14:05:20.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.376+0000 7f1e93e00640 1 -- 172.21.15.86:0/2655817230 >> v1:172.21.15.155:6789/0 conn(0x7f1e941616e0 legacy=0x7f1e94161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.376+0000 7f1e93e00640 1 -- 172.21.15.86:0/2655817230 >> v1:172.21.15.86:6789/0 conn(0x7f1e940d2c80 legacy=0x7f1e94070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.376+0000 7f1e93e00640 1 -- 172.21.15.86:0/2655817230 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e940d1be0 con 0x7f1e94071530 2024-04-25T14:05:20.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.376+0000 7f1e93e00640 1 --2- 172.21.15.86:0/2655817230 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e94071530 0x7f1e9415dc90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1e800097a0 tx=0x7f1e8001eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86f5ea75da123c71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:20.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.376+0000 7f1e93400640 1 -- 172.21.15.86:0/2655817230 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e800b5070 con 0x7f1e94071530 2024-04-25T14:05:20.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.376+0000 7f1e93400640 1 -- 172.21.15.86:0/2655817230 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e800a8610 con 0x7f1e94071530 2024-04-25T14:05:20.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.376+0000 7f1e93400640 1 -- 172.21.15.86:0/2655817230 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88001680 con 0x7f1e94071530 2024-04-25T14:05:20.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.380+0000 7f1e9b400640 1 -- 172.21.15.86:0/2655817230 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e94071530 msgr2=0x7f1e9415dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.380+0000 7f1e9b400640 1 --2- 172.21.15.86:0/2655817230 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e94071530 0x7f1e9415dc90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1e800097a0 tx=0x7f1e8001eb90 comp rx=0 tx=0).stop 2024-04-25T14:05:20.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.380+0000 7f1e9b400640 1 -- 172.21.15.86:0/2655817230 shutdown_connections 2024-04-25T14:05:20.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.380+0000 7f1e9b400640 1 --2- 172.21.15.86:0/2655817230 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e94071530 0x7f1e9415dc90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.380+0000 7f1e9b400640 1 -- 172.21.15.86:0/2655817230 >> 172.21.15.86:0/2655817230 conn(0x7f1e940cffc0 msgr2=0x7f1e940d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:20.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.380+0000 7f1e9b400640 1 -- 172.21.15.86:0/2655817230 shutdown_connections 2024-04-25T14:05:20.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.380+0000 7f1e9b400640 1 -- 172.21.15.86:0/2655817230 wait complete. 2024-04-25T14:05:20.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.380+0000 7f1e9b400640 1 Processor -- start 2024-04-25T14:05:20.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.380+0000 7f1e9b400640 1 -- start start 2024-04-25T14:05:20.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.384+0000 7f1e9b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e940d2c80 0x7f1e941f1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.384+0000 7f1e9b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e941616e0 0x7f1e941f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.384+0000 7f1e9b400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e94156190 0x7f1e94154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.384+0000 7f1e9b400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e940d2480 con 0x7f1e940d2c80 2024-04-25T14:05:20.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.384+0000 7f1e9b400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1e940d21a0 con 0x7f1e941616e0 2024-04-25T14:05:20.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.384+0000 7f1e9b400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1e940d29a0 con 0x7f1e94156190 2024-04-25T14:05:20.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.384+0000 7f1e98a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e940d2c80 0x7f1e941f1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.384+0000 7f1e98a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e940d2c80 0x7f1e941f1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43184/0 (socket says 172.21.15.86:43184) 2024-04-25T14:05:20.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.384+0000 7f1e98a00640 1 -- 172.21.15.86:0/1547107551 learned_addr learned my addr 172.21.15.86:0/1547107551 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:20.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.384+0000 7f1e93e00640 1 --2- 172.21.15.86:0/1547107551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e941616e0 0x7f1e941f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.384+0000 7f1e99400640 1 --2- 172.21.15.86:0/1547107551 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e94156190 0x7f1e94154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.384+0000 7f1e98a00640 1 -- 172.21.15.86:0/1547107551 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e94156190 msgr2=0x7f1e94154630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.384+0000 7f1e98a00640 1 --2- 172.21.15.86:0/1547107551 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e94156190 0x7f1e94154630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.384+0000 7f1e98a00640 1 -- 172.21.15.86:0/1547107551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e941616e0 msgr2=0x7f1e941f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.384+0000 7f1e98a00640 1 --2- 172.21.15.86:0/1547107551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e941616e0 0x7f1e941f3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.384+0000 7f1e98a00640 1 -- 172.21.15.86:0/1547107551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e940cb630 con 0x7f1e940d2c80 2024-04-25T14:05:20.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.384+0000 7f1e99400640 1 --2- 172.21.15.86:0/1547107551 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e94156190 0x7f1e94154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:20.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.384+0000 7f1e98a00640 1 --2- 172.21.15.86:0/1547107551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e940d2c80 0x7f1e941f1710 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1e8800c730 tx=0x7f1e8800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:20.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.384+0000 7f1e90c00640 1 -- 172.21.15.86:0/1547107551 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88017070 con 0x7f1e940d2c80 2024-04-25T14:05:20.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.384+0000 7f1e90c00640 1 -- 172.21.15.86:0/1547107551 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e88005b00 con 0x7f1e940d2c80 2024-04-25T14:05:20.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.384+0000 7f1e9b400640 1 -- 172.21.15.86:0/1547107551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e941f48e0 con 0x7f1e940d2c80 2024-04-25T14:05:20.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.384+0000 7f1e90c00640 1 -- 172.21.15.86:0/1547107551 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88007070 con 0x7f1e940d2c80 2024-04-25T14:05:20.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.384+0000 7f1e9b400640 1 -- 172.21.15.86:0/1547107551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e941f4ea0 con 0x7f1e940d2c80 2024-04-25T14:05:20.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.384+0000 7f1e9b400640 1 -- 172.21.15.86:0/1547107551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e940720e0 con 0x7f1e940d2c80 2024-04-25T14:05:20.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.388+0000 7f1e90c00640 1 -- 172.21.15.86:0/1547107551 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 6) ==== 93216+0+0 (secure 0 0 0) 0x7f1e8801a070 con 0x7f1e940d2c80 2024-04-25T14:05:20.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.388+0000 7f1e90c00640 1 --2- 172.21.15.86:0/1547107551 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1e7406f450 0x7f1e74071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.388+0000 7f1e93e00640 1 -- 172.21.15.86:0/1547107551 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1e7406f450 msgr2=0x7f1e74071960 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2968869755 2024-04-25T14:05:20.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.388+0000 7f1e90c00640 1 -- 172.21.15.86:0/1547107551 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f1e8808e940 con 0x7f1e940d2c80 2024-04-25T14:05:20.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.388+0000 7f1e93e00640 1 --2- 172.21.15.86:0/1547107551 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1e7406f450 0x7f1e74071960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:05:20.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.392+0000 7f1e90c00640 1 -- 172.21.15.86:0/1547107551 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1e8805b050 con 0x7f1e940d2c80 2024-04-25T14:05:20.522 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:05:20.518+0000 7f28af07d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:05:20.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.552+0000 7f1e9b400640 1 -- 172.21.15.86:0/1547107551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1e94076bf0 con 0x7f1e940d2c80 2024-04-25T14:05:20.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.552+0000 7f1e90c00640 1 -- 172.21.15.86:0/1547107551 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) ==== 104+0+11 (secure 0 0 0) 0x7f1e8805f350 con 0x7f1e940d2c80 2024-04-25T14:05:20.553 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:05:20.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.552+0000 7f1e9b400640 1 -- 172.21.15.86:0/1547107551 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1e7406f450 msgr2=0x7f1e74071960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:05:20.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.552+0000 7f1e9b400640 1 --2- 172.21.15.86:0/1547107551 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1e7406f450 0x7f1e74071960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.552+0000 7f1e9b400640 1 -- 172.21.15.86:0/1547107551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e940d2c80 msgr2=0x7f1e941f1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.552+0000 7f1e9b400640 1 --2- 172.21.15.86:0/1547107551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e940d2c80 0x7f1e941f1710 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1e8800c730 tx=0x7f1e8800cbf0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.552+0000 7f1e9b400640 1 -- 172.21.15.86:0/1547107551 shutdown_connections 2024-04-25T14:05:20.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.552+0000 7f1e9b400640 1 --2- 172.21.15.86:0/1547107551 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f1e7406f450 0x7f1e74071960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.552+0000 7f1e9b400640 1 --2- 172.21.15.86:0/1547107551 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e94156190 0x7f1e94154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.552+0000 7f1e9b400640 1 --2- 172.21.15.86:0/1547107551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e941616e0 0x7f1e941f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.552+0000 7f1e9b400640 1 --2- 172.21.15.86:0/1547107551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e940d2c80 0x7f1e941f1710 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.552+0000 7f1e9b400640 1 -- 172.21.15.86:0/1547107551 >> 172.21.15.86:0/1547107551 conn(0x7f1e940cffc0 msgr2=0x7f1e941599c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:20.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.552+0000 7f1e9b400640 1 -- 172.21.15.86:0/1547107551 shutdown_connections 2024-04-25T14:05:20.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.552+0000 7f1e9b400640 1 -- 172.21.15.86:0/1547107551 wait complete. 2024-04-25T14:05:20.575 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:05:20.607 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:05:20.602+0000 7f28af07d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:05:20.650 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:20.648+0000 7f7fa3e2c180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:05:20.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.724+0000 7fe70b000640 1 Processor -- start 2024-04-25T14:05:20.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.724+0000 7fe70b000640 1 -- start start 2024-04-25T14:05:20.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.728+0000 7fe70b000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe704070330 0x7fe7040d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.728+0000 7fe70b000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7040ce790 0x7fe70406f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.728+0000 7fe70b000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7040d2030 con 0x7fe7040d1c20 2024-04-25T14:05:20.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.728+0000 7fe70b000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe70415e5c0 con 0x7fe704070330 2024-04-25T14:05:20.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.728+0000 7fe70b000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe70415e8a0 con 0x7fe7040ce790 2024-04-25T14:05:20.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.728+0000 7fe703e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe704070330 0x7fe7040d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.728+0000 7fe709000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7040ce790 0x7fe70406f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.728+0000 7fe6fbe00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fe7040d1c20 0x7fe70406ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:39966/0 (socket says 172.21.15.86:39966) 2024-04-25T14:05:20.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.728+0000 7fe6fbe00640 1 -- 172.21.15.86:0/4214380759 learned_addr learned my addr 172.21.15.86:0/4214380759 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:20.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.728+0000 7fe703e00640 1 -- 172.21.15.86:0/4214380759 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7040ce790 msgr2=0x7fe70406f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:20.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.728+0000 7fe703e00640 1 --2- 172.21.15.86:0/4214380759 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7040ce790 0x7fe70406f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.728+0000 7fe703e00640 1 -- 172.21.15.86:0/4214380759 >> v1:172.21.15.155:6789/0 conn(0x7fe7040d1c20 legacy=0x7fe70406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.728+0000 7fe703e00640 1 -- 172.21.15.86:0/4214380759 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70415eb80 con 0x7fe704070330 2024-04-25T14:05:20.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.728+0000 7fe703400640 1 -- 172.21.15.86:0/4214380759 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1071725090 0 0) 0x7fe6ec001120 con 0x7fe7040d1c20 2024-04-25T14:05:20.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.728+0000 7fe709000640 1 --2- 172.21.15.86:0/4214380759 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7040ce790 0x7fe70406f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:20.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.732+0000 7fe703e00640 1 --2- 172.21.15.86:0/4214380759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe704070330 0x7fe7040d16e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40097c0 tx=0x7fe6f401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77931e3fd9034166 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:20.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.732+0000 7fe703400640 1 -- 172.21.15.86:0/4214380759 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f40b0070 con 0x7fe704070330 2024-04-25T14:05:20.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.732+0000 7fe703400640 1 -- 172.21.15.86:0/4214380759 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6f40ab600 con 0x7fe704070330 2024-04-25T14:05:20.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.732+0000 7fe703400640 1 -- 172.21.15.86:0/4214380759 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f40ab9d0 con 0x7fe704070330 2024-04-25T14:05:20.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.732+0000 7fe70b000640 1 -- 172.21.15.86:0/4214380759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe704070330 msgr2=0x7fe7040d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.732+0000 7fe70b000640 1 --2- 172.21.15.86:0/4214380759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe704070330 0x7fe7040d16e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40097c0 tx=0x7fe6f401c410 comp rx=0 tx=0).stop 2024-04-25T14:05:20.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.732+0000 7fe70b000640 1 -- 172.21.15.86:0/4214380759 shutdown_connections 2024-04-25T14:05:20.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.732+0000 7fe70b000640 1 --2- 172.21.15.86:0/4214380759 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7040ce790 0x7fe70406f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.732+0000 7fe70b000640 1 --2- 172.21.15.86:0/4214380759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe704070330 0x7fe7040d16e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.732+0000 7fe70b000640 1 -- 172.21.15.86:0/4214380759 >> 172.21.15.86:0/4214380759 conn(0x7fe7040b4650 msgr2=0x7fe7040b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:20.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.732+0000 7fe70b000640 1 -- 172.21.15.86:0/4214380759 shutdown_connections 2024-04-25T14:05:20.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.732+0000 7fe70b000640 1 -- 172.21.15.86:0/4214380759 wait complete. 2024-04-25T14:05:20.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.732+0000 7fe70b000640 1 Processor -- start 2024-04-25T14:05:20.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.736+0000 7fe70b000640 1 -- start start 2024-04-25T14:05:20.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.736+0000 7fe70b000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe704070330 0x7fe7040ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.736+0000 7fe70b000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7040ce790 0x7fe7040b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.736+0000 7fe70b000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7040d1c20 0x7fe7040b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.736+0000 7fe70b000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7040708c0 con 0x7fe704070330 2024-04-25T14:05:20.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.736+0000 7fe70b000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe704070740 con 0x7fe7040ce790 2024-04-25T14:05:20.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.736+0000 7fe70b000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe70416afc0 con 0x7fe7040d1c20 2024-04-25T14:05:20.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.736+0000 7fe703e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe704070330 0x7fe7040ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.736+0000 7fe703e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe704070330 0x7fe7040ca2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43186/0 (socket says 172.21.15.86:43186) 2024-04-25T14:05:20.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.736+0000 7fe703e00640 1 -- 172.21.15.86:0/455054158 learned_addr learned my addr 172.21.15.86:0/455054158 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:20.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.736+0000 7fe6fbe00640 1 --2- 172.21.15.86:0/455054158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7040ce790 0x7fe7040b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.736+0000 7fe709000640 1 --2- 172.21.15.86:0/455054158 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7040d1c20 0x7fe7040b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.736+0000 7fe709000640 1 -- 172.21.15.86:0/455054158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7040ce790 msgr2=0x7fe7040b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.736+0000 7fe709000640 1 --2- 172.21.15.86:0/455054158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7040ce790 0x7fe7040b21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.736+0000 7fe709000640 1 -- 172.21.15.86:0/455054158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe704070330 msgr2=0x7fe7040ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.736+0000 7fe709000640 1 --2- 172.21.15.86:0/455054158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe704070330 0x7fe7040ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.736+0000 7fe709000640 1 -- 172.21.15.86:0/455054158 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7040ca810 con 0x7fe7040d1c20 2024-04-25T14:05:20.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.736+0000 7fe6fbe00640 1 --2- 172.21.15.86:0/455054158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7040ce790 0x7fe7040b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:20.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.736+0000 7fe703e00640 1 --2- 172.21.15.86:0/455054158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe704070330 0x7fe7040ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:20.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.736+0000 7fe709000640 1 --2- 172.21.15.86:0/455054158 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7040d1c20 0x7fe7040b1350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009db0 tx=0x7fe6fc00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:20.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.736+0000 7fe700c00640 1 -- 172.21.15.86:0/455054158 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc018030 con 0x7fe7040d1c20 2024-04-25T14:05:20.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.736+0000 7fe700c00640 1 -- 172.21.15.86:0/455054158 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6fc00ccd0 con 0x7fe7040d1c20 2024-04-25T14:05:20.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.736+0000 7fe700c00640 1 -- 172.21.15.86:0/455054158 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc003bc0 con 0x7fe7040d1c20 2024-04-25T14:05:20.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.736+0000 7fe70b000640 1 -- 172.21.15.86:0/455054158 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe70409b4c0 con 0x7fe7040d1c20 2024-04-25T14:05:20.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.736+0000 7fe70b000640 1 -- 172.21.15.86:0/455054158 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe70409ba80 con 0x7fe7040d1c20 2024-04-25T14:05:20.741 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:20.740+0000 7f7fa3e2c180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:05:20.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.740+0000 7fe70b000640 1 -- 172.21.15.86:0/455054158 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe704069d00 con 0x7fe7040d1c20 2024-04-25T14:05:20.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.740+0000 7fe700c00640 1 -- 172.21.15.86:0/455054158 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 6) ==== 93216+0+0 (secure 0 0 0) 0x7fe6fc01b070 con 0x7fe7040d1c20 2024-04-25T14:05:20.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.740+0000 7fe700c00640 1 --2- 172.21.15.86:0/455054158 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe6e006f480 0x7fe6e0071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.740+0000 7fe703e00640 1 -- 172.21.15.86:0/455054158 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe6e006f480 msgr2=0x7fe6e0071990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2968869755 2024-04-25T14:05:20.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.740+0000 7fe700c00640 1 -- 172.21.15.86:0/455054158 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fe6fc08fb70 con 0x7fe7040d1c20 2024-04-25T14:05:20.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.740+0000 7fe703e00640 1 --2- 172.21.15.86:0/455054158 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe6e006f480 0x7fe6e0071990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:05:20.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.744+0000 7fe700c00640 1 -- 172.21.15.86:0/455054158 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe6fc010040 con 0x7fe7040d1c20 2024-04-25T14:05:20.770 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:05:20.766+0000 7f28af07d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:05:20.854 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:05:20.850+0000 7f28af07d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:05:20.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.944+0000 7fe703e00640 1 -- 172.21.15.86:0/455054158 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe6e006f480 msgr2=0x7fe6e0071990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2968869755 2024-04-25T14:05:20.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.944+0000 7fe703e00640 1 --2- 172.21.15.86:0/455054158 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe6e006f480 0x7fe6e0071990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:05:20.971 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:20.968+0000 7f7fa3e2c180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:05:20.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.984+0000 7fe70b000640 1 -- 172.21.15.86:0/455054158 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe704076bf0 con 0x7fe7040d1c20 2024-04-25T14:05:20.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.984+0000 7fe700c00640 1 -- 172.21.15.86:0/455054158 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) ==== 100+0+11 (secure 0 0 0) 0x7fe6fc05c270 con 0x7fe7040d1c20 2024-04-25T14:05:20.985 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:05:20.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.984+0000 7fe70b000640 1 -- 172.21.15.86:0/455054158 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe6e006f480 msgr2=0x7fe6e0071990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:05:20.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.984+0000 7fe70b000640 1 --2- 172.21.15.86:0/455054158 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe6e006f480 0x7fe6e0071990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.984+0000 7fe70b000640 1 -- 172.21.15.86:0/455054158 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7040d1c20 msgr2=0x7fe7040b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.984+0000 7fe70b000640 1 --2- 172.21.15.86:0/455054158 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7040d1c20 0x7fe7040b1350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009db0 tx=0x7fe6fc00c490 comp rx=0 tx=0).stop 2024-04-25T14:05:20.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.984+0000 7fe70b000640 1 -- 172.21.15.86:0/455054158 shutdown_connections 2024-04-25T14:05:20.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.984+0000 7fe70b000640 1 --2- 172.21.15.86:0/455054158 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7fe6e006f480 0x7fe6e0071990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.984+0000 7fe70b000640 1 --2- 172.21.15.86:0/455054158 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7040d1c20 0x7fe7040b1350 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.984+0000 7fe70b000640 1 --2- 172.21.15.86:0/455054158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7040ce790 0x7fe7040b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.984+0000 7fe70b000640 1 --2- 172.21.15.86:0/455054158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe704070330 0x7fe7040ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.984+0000 7fe70b000640 1 -- 172.21.15.86:0/455054158 >> 172.21.15.86:0/455054158 conn(0x7fe7040b4650 msgr2=0x7fe7040ae730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:20.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.984+0000 7fe70b000640 1 -- 172.21.15.86:0/455054158 shutdown_connections 2024-04-25T14:05:20.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:20.984+0000 7fe70b000640 1 -- 172.21.15.86:0/455054158 wait complete. 2024-04-25T14:05:21.009 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T14:05:21.060 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:21.060+0000 7f7fa3e2c180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:05:21.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.156+0000 7f5807600640 1 Processor -- start 2024-04-25T14:05:21.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.156+0000 7f5807600640 1 -- start start 2024-04-25T14:05:21.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.160+0000 7f5807600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f58000cdd70 0x7f58000ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.160+0000 7f5807600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5800070e20 0x7f5800071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.160+0000 7f5807600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f580006e700 0x7f580006eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.160+0000 7f5807600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5800071770 con 0x7f5800070e20 2024-04-25T14:05:21.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.160+0000 7f5807600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f580006f050 con 0x7f58000cdd70 2024-04-25T14:05:21.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.160+0000 7f5807600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f580006d940 con 0x7f580006e700 2024-04-25T14:05:21.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.160+0000 7f57ffe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5800070e20 0x7f5800071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.160+0000 7f57ffe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5800070e20 0x7f5800071230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43192/0 (socket says 172.21.15.86:43192) 2024-04-25T14:05:21.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.160+0000 7f57ffe00640 1 -- 172.21.15.86:0/1017469648 learned_addr learned my addr 172.21.15.86:0/1017469648 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:21.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.160+0000 7f5804c00640 1 --2- 172.21.15.86:0/1017469648 >> v2:172.21.15.155:3300/0 conn(0x7f58000cdd70 0x7f58000ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.160+0000 7f5805600640 1 --2- 172.21.15.86:0/1017469648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f580006e700 0x7f580006eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.160+0000 7f57ffe00640 1 -- 172.21.15.86:0/1017469648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f580006e700 msgr2=0x7f580006eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:21.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.160+0000 7f57ffe00640 1 --2- 172.21.15.86:0/1017469648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f580006e700 0x7f580006eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.160+0000 7f57ffe00640 1 -- 172.21.15.86:0/1017469648 >> v2:172.21.15.155:3300/0 conn(0x7f58000cdd70 msgr2=0x7f58000ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:21.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.160+0000 7f57ffe00640 1 --2- 172.21.15.86:0/1017469648 >> v2:172.21.15.155:3300/0 conn(0x7f58000cdd70 0x7f58000ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.160+0000 7f57ffe00640 1 -- 172.21.15.86:0/1017469648 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580006dc20 con 0x7f5800070e20 2024-04-25T14:05:21.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.160+0000 7f5805600640 1 --2- 172.21.15.86:0/1017469648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f580006e700 0x7f580006eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:21.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.160+0000 7f57ffe00640 1 --2- 172.21.15.86:0/1017469648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5800070e20 0x7f5800071230 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5800074610 tx=0x7f57ec01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd8cafa86899d9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:21.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.164+0000 7f57ff400640 1 -- 172.21.15.86:0/1017469648 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0b0070 con 0x7f5800070e20 2024-04-25T14:05:21.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.164+0000 7f57ff400640 1 -- 172.21.15.86:0/1017469648 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57ec0ab600 con 0x7f5800070e20 2024-04-25T14:05:21.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.164+0000 7f57ff400640 1 -- 172.21.15.86:0/1017469648 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0ab9d0 con 0x7f5800070e20 2024-04-25T14:05:21.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.164+0000 7f5807600640 1 -- 172.21.15.86:0/1017469648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5800070e20 msgr2=0x7f5800071230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.164+0000 7f5807600640 1 --2- 172.21.15.86:0/1017469648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5800070e20 0x7f5800071230 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5800074610 tx=0x7f57ec01c490 comp rx=0 tx=0).stop 2024-04-25T14:05:21.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.164+0000 7f5807600640 1 -- 172.21.15.86:0/1017469648 shutdown_connections 2024-04-25T14:05:21.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.164+0000 7f5807600640 1 --2- 172.21.15.86:0/1017469648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f580006e700 0x7f580006eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.164+0000 7f5807600640 1 --2- 172.21.15.86:0/1017469648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5800070e20 0x7f5800071230 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.164+0000 7f5807600640 1 --2- 172.21.15.86:0/1017469648 >> v2:172.21.15.155:3300/0 conn(0x7f58000cdd70 0x7f58000ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.164+0000 7f5807600640 1 -- 172.21.15.86:0/1017469648 >> 172.21.15.86:0/1017469648 conn(0x7f58000cffc0 msgr2=0x7f58000d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:21.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.164+0000 7f5807600640 1 -- 172.21.15.86:0/1017469648 shutdown_connections 2024-04-25T14:05:21.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.164+0000 7f5807600640 1 -- 172.21.15.86:0/1017469648 wait complete. 2024-04-25T14:05:21.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.164+0000 7f5807600640 1 Processor -- start 2024-04-25T14:05:21.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.164+0000 7f5807600640 1 -- start start 2024-04-25T14:05:21.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.168+0000 7f5807600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f580006e700 0x7f58001519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.168+0000 7f5807600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5800070e20 0x7f5800154520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.168+0000 7f5807600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58000cdd70 0x7f5800154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.168+0000 7f5807600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f58001642f0 con 0x7f5800070e20 2024-04-25T14:05:21.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.168+0000 7f5807600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5800164170 con 0x7f58000cdd70 2024-04-25T14:05:21.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.168+0000 7f5807600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5800164470 con 0x7f580006e700 2024-04-25T14:05:21.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.168+0000 7f57ffe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5800070e20 0x7f5800154520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.168+0000 7f57ffe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5800070e20 0x7f5800154520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43202/0 (socket says 172.21.15.86:43202) 2024-04-25T14:05:21.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.168+0000 7f57ffe00640 1 -- 172.21.15.86:0/1861067768 learned_addr learned my addr 172.21.15.86:0/1861067768 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:21.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.168+0000 7f5804c00640 1 --2- 172.21.15.86:0/1861067768 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f580006e700 0x7f58001519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.168+0000 7f5805600640 1 --2- 172.21.15.86:0/1861067768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58000cdd70 0x7f5800154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.168+0000 7f57ffe00640 1 -- 172.21.15.86:0/1861067768 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f580006e700 msgr2=0x7f58001519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.168+0000 7f57ffe00640 1 --2- 172.21.15.86:0/1861067768 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f580006e700 0x7f58001519b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.168+0000 7f57ffe00640 1 -- 172.21.15.86:0/1861067768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58000cdd70 msgr2=0x7f5800154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.168+0000 7f57ffe00640 1 --2- 172.21.15.86:0/1861067768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58000cdd70 0x7f5800154a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.168+0000 7f57ffe00640 1 -- 172.21.15.86:0/1861067768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58000d1d30 con 0x7f5800070e20 2024-04-25T14:05:21.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.168+0000 7f57ffe00640 1 --2- 172.21.15.86:0/1861067768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5800070e20 0x7f5800154520 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f57ec01cd30 tx=0x7f57ec019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:21.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.168+0000 7f57fcc00640 1 -- 172.21.15.86:0/1861067768 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0b0040 con 0x7f5800070e20 2024-04-25T14:05:21.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.168+0000 7f57fcc00640 1 -- 172.21.15.86:0/1861067768 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57ec004020 con 0x7f5800070e20 2024-04-25T14:05:21.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.168+0000 7f57fcc00640 1 -- 172.21.15.86:0/1861067768 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0043d0 con 0x7f5800070e20 2024-04-25T14:05:21.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.168+0000 7f5807600640 1 -- 172.21.15.86:0/1861067768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58000d2010 con 0x7f5800070e20 2024-04-25T14:05:21.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.168+0000 7f5807600640 1 -- 172.21.15.86:0/1861067768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5800160510 con 0x7f5800070e20 2024-04-25T14:05:21.173 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:05:21.170+0000 7f28af07d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:05:21.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.168+0000 7f5807600640 1 -- 172.21.15.86:0/1861067768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58000720e0 con 0x7f5800070e20 2024-04-25T14:05:21.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.172+0000 7f57fcc00640 1 -- 172.21.15.86:0/1861067768 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 6) ==== 93216+0+0 (secure 0 0 0) 0x7f57ec0bb060 con 0x7f5800070e20 2024-04-25T14:05:21.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.172+0000 7f57fcc00640 1 --2- 172.21.15.86:0/1861067768 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f57e006f450 0x7f57e0071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.172+0000 7f5804c00640 1 -- 172.21.15.86:0/1861067768 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f57e006f450 msgr2=0x7f57e0071960 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2968869755 2024-04-25T14:05:21.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.172+0000 7f57fcc00640 1 -- 172.21.15.86:0/1861067768 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f57ec128300 con 0x7f5800070e20 2024-04-25T14:05:21.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.172+0000 7f5804c00640 1 --2- 172.21.15.86:0/1861067768 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f57e006f450 0x7f57e0071960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:05:21.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.176+0000 7f57fcc00640 1 -- 172.21.15.86:0/1861067768 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f57ec016110 con 0x7f5800070e20 2024-04-25T14:05:21.235 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:05:21.230+0000 7f28af07d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:05:21.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.324+0000 7f57fcc00640 1 -- 172.21.15.86:0/1861067768 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 7) ==== 93216+0+0 (secure 0 0 0) 0x7f57ec0f0c90 con 0x7f5800070e20 2024-04-25T14:05:21.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.352+0000 7f5807600640 1 -- 172.21.15.86:0/1861067768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5800076bf0 con 0x7f5800070e20 2024-04-25T14:05:21.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.352+0000 7f57fcc00640 1 -- 172.21.15.86:0/1861067768 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v38) ==== 80+0+0 (secure 0 0 0) 0x7f57ec1125b0 con 0x7f5800070e20 2024-04-25T14:05:21.355 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-25T14:05:21.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.356+0000 7f5807600640 1 -- 172.21.15.86:0/1861067768 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f57e006f450 msgr2=0x7f57e0071960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:05:21.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.356+0000 7f5807600640 1 --2- 172.21.15.86:0/1861067768 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f57e006f450 0x7f57e0071960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.356+0000 7f5807600640 1 -- 172.21.15.86:0/1861067768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5800070e20 msgr2=0x7f5800154520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.356+0000 7f5807600640 1 --2- 172.21.15.86:0/1861067768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5800070e20 0x7f5800154520 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f57ec01cd30 tx=0x7f57ec019230 comp rx=0 tx=0).stop 2024-04-25T14:05:21.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.356+0000 7f5807600640 1 -- 172.21.15.86:0/1861067768 shutdown_connections 2024-04-25T14:05:21.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.356+0000 7f5807600640 1 --2- 172.21.15.86:0/1861067768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58000cdd70 0x7f5800154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.356+0000 7f5807600640 1 --2- 172.21.15.86:0/1861067768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5800070e20 0x7f5800154520 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.356+0000 7f5807600640 1 --2- 172.21.15.86:0/1861067768 >> [v2:172.21.15.86:6832/2968869755,v1:172.21.15.86:6833/2968869755] conn(0x7f57e006f450 0x7f57e0071960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.356+0000 7f5807600640 1 --2- 172.21.15.86:0/1861067768 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f580006e700 0x7f58001519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.356+0000 7f5807600640 1 -- 172.21.15.86:0/1861067768 >> 172.21.15.86:0/1861067768 conn(0x7f58000cffc0 msgr2=0x7f58000b44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:21.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.356+0000 7f5807600640 1 -- 172.21.15.86:0/1861067768 shutdown_connections 2024-04-25T14:05:21.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.356+0000 7f5807600640 1 -- 172.21.15.86:0/1861067768 wait complete. 2024-04-25T14:05:21.377 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T14:05:21.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:21.378 DEBUG:teuthology.orchestra.run.smithi155:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T14:05:21.394 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:21.392+0000 7f7fa3e2c180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:05:21.405 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T14:05:21.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:21.405 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T14:05:21.457 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:21.456+0000 7f7fa3e2c180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:05:21.460 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:05:21.534 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:21.532+0000 7f7f1e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:05:21.534 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:21.532+0000 7f7f1e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:05:21.534 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:21.532+0000 7f7f1e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:05:21.534 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:21.532+0000 7f7f1e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:05:21.534 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:21.532+0000 7f7f1e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:05:21.535 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:21.532+0000 7f7f22200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:05:21.535 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:21.532+0000 7f7f22200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:05:21.535 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:21.532+0000 7f7f22200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:05:21.535 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:21.532+0000 7f7f22200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:05:21.536 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:05:21.532+0000 7f7f22200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:05:21.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.620+0000 7efe6fc00640 1 Processor -- start 2024-04-25T14:05:21.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.620+0000 7efe6fc00640 1 -- start start 2024-04-25T14:05:21.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.620+0000 7efe6fc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efe6806c670 0x7efe6815de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.620+0000 7efe6fc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe680cef20 0x7efe680cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.620+0000 7efe6fc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe68161b40 con 0x7efe680cdf50 2024-04-25T14:05:21.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.620+0000 7efe6fc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7efe6815e380 con 0x7efe6806c670 2024-04-25T14:05:21.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.620+0000 7efe6fc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efe68162750 con 0x7efe680cef20 2024-04-25T14:05:21.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.620+0000 7efe6d200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efe6806c670 0x7efe6815de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.620+0000 7efe6d200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efe6806c670 0x7efe6815de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43326/0 (socket says 172.21.15.86:43326) 2024-04-25T14:05:21.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.620+0000 7efe5fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7efe680cdf50 0x7efe680ce360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47336/0 (socket says 172.21.15.86:47336) 2024-04-25T14:05:21.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.620+0000 7efe6d200640 1 -- 172.21.15.86:0/1154194011 learned_addr learned my addr 172.21.15.86:0/1154194011 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:21.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.620+0000 7efe6dc00640 1 --2- 172.21.15.86:0/1154194011 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe680cef20 0x7efe680cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.620+0000 7efe6d200640 1 -- 172.21.15.86:0/1154194011 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe680cef20 msgr2=0x7efe680cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:21.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.620+0000 7efe6d200640 1 --2- 172.21.15.86:0/1154194011 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe680cef20 0x7efe680cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.620+0000 7efe6d200640 1 -- 172.21.15.86:0/1154194011 >> v1:172.21.15.86:6789/0 conn(0x7efe680cdf50 legacy=0x7efe680ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.620+0000 7efe6d200640 1 -- 172.21.15.86:0/1154194011 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe68161e20 con 0x7efe6806c670 2024-04-25T14:05:21.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.620+0000 7efe6dc00640 1 --2- 172.21.15.86:0/1154194011 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe680cef20 0x7efe680cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:21.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.620+0000 7efe67e00640 1 -- 172.21.15.86:0/1154194011 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3158275822 0 0) 0x7efe68161b40 con 0x7efe680cdf50 2024-04-25T14:05:21.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.624+0000 7efe6d200640 1 --2- 172.21.15.86:0/1154194011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe6806c670 0x7efe6815de40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7efe58009870 tx=0x7efe5801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878474e5f41dd309 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:21.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.624+0000 7efe67e00640 1 -- 172.21.15.86:0/1154194011 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe580b1070 con 0x7efe6806c670 2024-04-25T14:05:21.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.624+0000 7efe67e00640 1 -- 172.21.15.86:0/1154194011 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe580ac620 con 0x7efe6806c670 2024-04-25T14:05:21.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.624+0000 7efe67e00640 1 -- 172.21.15.86:0/1154194011 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe580ac9f0 con 0x7efe6806c670 2024-04-25T14:05:21.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.624+0000 7efe6fc00640 1 -- 172.21.15.86:0/1154194011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe6806c670 msgr2=0x7efe6815de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.624+0000 7efe6fc00640 1 --2- 172.21.15.86:0/1154194011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe6806c670 0x7efe6815de40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7efe58009870 tx=0x7efe5801c4d0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.624+0000 7efe6fc00640 1 -- 172.21.15.86:0/1154194011 shutdown_connections 2024-04-25T14:05:21.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.624+0000 7efe6fc00640 1 --2- 172.21.15.86:0/1154194011 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe680cef20 0x7efe680cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.624+0000 7efe6fc00640 1 --2- 172.21.15.86:0/1154194011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe6806c670 0x7efe6815de40 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.624+0000 7efe6fc00640 1 -- 172.21.15.86:0/1154194011 >> 172.21.15.86:0/1154194011 conn(0x7efe6809ba30 msgr2=0x7efe680b73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:21.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.624+0000 7efe6fc00640 1 -- 172.21.15.86:0/1154194011 shutdown_connections 2024-04-25T14:05:21.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.624+0000 7efe6fc00640 1 -- 172.21.15.86:0/1154194011 wait complete. 2024-04-25T14:05:21.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.628+0000 7efe6fc00640 1 Processor -- start 2024-04-25T14:05:21.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.628+0000 7efe6fc00640 1 -- start start 2024-04-25T14:05:21.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.628+0000 7efe6fc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe6806c670 0x7efe68153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.628+0000 7efe6fc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe680cdf50 0x7efe680cb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.628+0000 7efe6fc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe680cef20 0x7efe680cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.628+0000 7efe6fc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe680b8060 con 0x7efe6806c670 2024-04-25T14:05:21.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.628+0000 7efe6fc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efe680b7ee0 con 0x7efe680cdf50 2024-04-25T14:05:21.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.628+0000 7efe6fc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efe6806ee70 con 0x7efe680cef20 2024-04-25T14:05:21.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.628+0000 7efe6d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe6806c670 0x7efe68153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.628+0000 7efe6d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe6806c670 0x7efe68153810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43332/0 (socket says 172.21.15.86:43332) 2024-04-25T14:05:21.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.628+0000 7efe6d200640 1 -- 172.21.15.86:0/575515903 learned_addr learned my addr 172.21.15.86:0/575515903 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:21.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.628+0000 7efe5fe00640 1 --2- 172.21.15.86:0/575515903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe680cdf50 0x7efe680cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.632+0000 7efe6dc00640 1 --2- 172.21.15.86:0/575515903 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe680cef20 0x7efe680cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.632+0000 7efe6d200640 1 -- 172.21.15.86:0/575515903 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe680cef20 msgr2=0x7efe680cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.632+0000 7efe6d200640 1 --2- 172.21.15.86:0/575515903 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe680cef20 0x7efe680cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.632+0000 7efe6d200640 1 -- 172.21.15.86:0/575515903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe680cdf50 msgr2=0x7efe680cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.632+0000 7efe6d200640 1 --2- 172.21.15.86:0/575515903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe680cdf50 0x7efe680cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.632+0000 7efe6d200640 1 -- 172.21.15.86:0/575515903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe680c7df0 con 0x7efe6806c670 2024-04-25T14:05:21.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.632+0000 7efe6d200640 1 --2- 172.21.15.86:0/575515903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe6806c670 0x7efe68153810 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7efe5801a100 tx=0x7efe5801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:21.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.632+0000 7efe65600640 1 -- 172.21.15.86:0/575515903 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe580bb020 con 0x7efe6806c670 2024-04-25T14:05:21.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.632+0000 7efe65600640 1 -- 172.21.15.86:0/575515903 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe580b7640 con 0x7efe6806c670 2024-04-25T14:05:21.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.632+0000 7efe6fc00640 1 -- 172.21.15.86:0/575515903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe680c80d0 con 0x7efe6806c670 2024-04-25T14:05:21.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.632+0000 7efe65600640 1 -- 172.21.15.86:0/575515903 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe580b1040 con 0x7efe6806c670 2024-04-25T14:05:21.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.632+0000 7efe6fc00640 1 -- 172.21.15.86:0/575515903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe680c8690 con 0x7efe6806c670 2024-04-25T14:05:21.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.632+0000 7efe6fc00640 1 -- 172.21.15.86:0/575515903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe680a0630 con 0x7efe6806c670 2024-04-25T14:05:21.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.632+0000 7efe65600640 1 -- 172.21.15.86:0/575515903 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 92927+0+0 (secure 0 0 0) 0x7efe580be020 con 0x7efe6806c670 2024-04-25T14:05:21.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.636+0000 7efe65600640 1 -- 172.21.15.86:0/575515903 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6359+0+0 (secure 0 0 0) 0x7efe58129340 con 0x7efe6806c670 2024-04-25T14:05:21.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.640+0000 7efe65600640 1 -- 172.21.15.86:0/575515903 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efe580c5030 con 0x7efe6806c670 2024-04-25T14:05:21.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.828+0000 7efe6fc00640 1 -- 172.21.15.86:0/575515903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe68076bf0 con 0x7efe6806c670 2024-04-25T14:05:21.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.828+0000 7efe65600640 1 -- 172.21.15.86:0/575515903 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5184 (secure 0 0 0) 0x7efe58016150 con 0x7efe6806c670 2024-04-25T14:05:21.831 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:21.832 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3546153119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3546153119},{"type":"v1","addr":"172.21.15.155:6834","nonce":3546153119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5283,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3499340224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3499340224},{"type":"v1","addr":"172.21.15.86:6836","nonce":3499340224}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5284,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1466978742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1466978742},{"type":"v1","addr":"172.21.15.155:6835","nonce":1466978742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:06.638216+0000","modified":"2024-04-25T14:05:09.664776+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5286":{"gid":5286,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/716923566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":716923566},{"type":"v1","addr":"172.21.15.86:6837","nonce":716923566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5286,"qdb_cluster":[5286]},"id":2}]} 2024-04-25T14:05:21.832 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 18 2024-04-25T14:05:21.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.832+0000 7efe6fc00640 1 -- 172.21.15.86:0/575515903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe6806c670 msgr2=0x7efe68153810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.832+0000 7efe6fc00640 1 --2- 172.21.15.86:0/575515903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe6806c670 0x7efe68153810 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7efe5801a100 tx=0x7efe5801ad10 comp rx=0 tx=0).stop 2024-04-25T14:05:21.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.832+0000 7efe6fc00640 1 -- 172.21.15.86:0/575515903 shutdown_connections 2024-04-25T14:05:21.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.832+0000 7efe6fc00640 1 --2- 172.21.15.86:0/575515903 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe680cef20 0x7efe680cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.832+0000 7efe6fc00640 1 --2- 172.21.15.86:0/575515903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe680cdf50 0x7efe680cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.832+0000 7efe6fc00640 1 --2- 172.21.15.86:0/575515903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe6806c670 0x7efe68153810 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.832+0000 7efe6fc00640 1 -- 172.21.15.86:0/575515903 >> 172.21.15.86:0/575515903 conn(0x7efe6809ba30 msgr2=0x7efe68156bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:21.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.832+0000 7efe6fc00640 1 -- 172.21.15.86:0/575515903 shutdown_connections 2024-04-25T14:05:21.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:21.832+0000 7efe6fc00640 1 -- 172.21.15.86:0/575515903 wait complete. 2024-04-25T14:05:21.852 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:05:22.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.000+0000 7f0163e00640 1 Processor -- start 2024-04-25T14:05:22.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.000+0000 7f0163e00640 1 -- start start 2024-04-25T14:05:22.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.000+0000 7f0163e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01640b36f0 0x7f01640b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:22.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.000+0000 7f0163e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f016406d780 0x7f016406db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:22.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.000+0000 7f0163e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0164160430 0x7f0164160840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:22.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.000+0000 7f0163e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f01640d1c40 con 0x7f0164160430 2024-04-25T14:05:22.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.000+0000 7f0163e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f01640b3b00 con 0x7f016406d780 2024-04-25T14:05:22.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.000+0000 7f0163e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f016406e0d0 con 0x7f01640b36f0 2024-04-25T14:05:22.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.000+0000 7f0163400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0164160430 0x7f0164160840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:22.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.000+0000 7f0163400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0164160430 0x7f0164160840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43340/0 (socket says 172.21.15.86:43340) 2024-04-25T14:05:22.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.000+0000 7f0163400640 1 -- 172.21.15.86:0/711307718 learned_addr learned my addr 172.21.15.86:0/711307718 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:22.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.000+0000 7f0162000640 1 --2- 172.21.15.86:0/711307718 >> v2:172.21.15.155:3300/0 conn(0x7f016406d780 0x7f016406db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:22.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.000+0000 7f0162a00640 1 --2- 172.21.15.86:0/711307718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01640b36f0 0x7f01640b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:22.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.000+0000 7f0163400640 1 -- 172.21.15.86:0/711307718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01640b36f0 msgr2=0x7f01640b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:22.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.000+0000 7f0163400640 1 --2- 172.21.15.86:0/711307718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01640b36f0 0x7f01640b77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.000+0000 7f0163400640 1 -- 172.21.15.86:0/711307718 >> v2:172.21.15.155:3300/0 conn(0x7f016406d780 msgr2=0x7f016406db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:22.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.000+0000 7f0163400640 1 --2- 172.21.15.86:0/711307718 >> v2:172.21.15.155:3300/0 conn(0x7f016406d780 0x7f016406db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.000+0000 7f0163400640 1 -- 172.21.15.86:0/711307718 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01640b1a90 con 0x7f0164160430 2024-04-25T14:05:22.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.000+0000 7f0162a00640 1 --2- 172.21.15.86:0/711307718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01640b36f0 0x7f01640b77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:22.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.004+0000 7f0163400640 1 --2- 172.21.15.86:0/711307718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0164160430 0x7f0164160840 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f016406e280 tx=0x7f015c01eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4602b89e21438615 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:22.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.004+0000 7f0161600640 1 -- 172.21.15.86:0/711307718 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c00b2d0 con 0x7f0164160430 2024-04-25T14:05:22.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.004+0000 7f0161600640 1 -- 172.21.15.86:0/711307718 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f015c00b470 con 0x7f0164160430 2024-04-25T14:05:22.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.004+0000 7f0161600640 1 -- 172.21.15.86:0/711307718 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c0a7a80 con 0x7f0164160430 2024-04-25T14:05:22.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.004+0000 7f0163e00640 1 -- 172.21.15.86:0/711307718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0164160430 msgr2=0x7f0164160840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:22.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.004+0000 7f0163e00640 1 --2- 172.21.15.86:0/711307718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0164160430 0x7f0164160840 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f016406e280 tx=0x7f015c01eb90 comp rx=0 tx=0).stop 2024-04-25T14:05:22.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.004+0000 7f0163e00640 1 -- 172.21.15.86:0/711307718 shutdown_connections 2024-04-25T14:05:22.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.004+0000 7f0163e00640 1 --2- 172.21.15.86:0/711307718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0164160430 0x7f0164160840 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.004+0000 7f0163e00640 1 --2- 172.21.15.86:0/711307718 >> v2:172.21.15.155:3300/0 conn(0x7f016406d780 0x7f016406db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.004+0000 7f0163e00640 1 --2- 172.21.15.86:0/711307718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01640b36f0 0x7f01640b77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.004+0000 7f0163e00640 1 -- 172.21.15.86:0/711307718 >> 172.21.15.86:0/711307718 conn(0x7f016409ba30 msgr2=0x7f01640b70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:22.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.004+0000 7f0163e00640 1 -- 172.21.15.86:0/711307718 shutdown_connections 2024-04-25T14:05:22.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.004+0000 7f0163e00640 1 -- 172.21.15.86:0/711307718 wait complete. 2024-04-25T14:05:22.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.008+0000 7f0163e00640 1 Processor -- start 2024-04-25T14:05:22.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.008+0000 7f0163e00640 1 -- start start 2024-04-25T14:05:22.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.008+0000 7f0163e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f016406d780 0x7f0164158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:22.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.008+0000 7f0163e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01640b36f0 0x7f01641553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:22.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.008+0000 7f0163e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0164160430 0x7f0164155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:22.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.008+0000 7f0163e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01640d3140 con 0x7f0164160430 2024-04-25T14:05:22.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.008+0000 7f0163e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f01640b1d30 con 0x7f01640b36f0 2024-04-25T14:05:22.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.008+0000 7f0163e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0164161190 con 0x7f016406d780 2024-04-25T14:05:22.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.008+0000 7f0163400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0164160430 0x7f0164155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:22.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.008+0000 7f0162a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f016406d780 0x7f0164158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:22.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.008+0000 7f0163400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0164160430 0x7f0164155900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43342/0 (socket says 172.21.15.86:43342) 2024-04-25T14:05:22.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.008+0000 7f0163400640 1 -- 172.21.15.86:0/440187190 learned_addr learned my addr 172.21.15.86:0/440187190 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:22.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.008+0000 7f0162000640 1 --2- 172.21.15.86:0/440187190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01640b36f0 0x7f01641553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:22.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.008+0000 7f0163400640 1 -- 172.21.15.86:0/440187190 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f016406d780 msgr2=0x7f0164158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:22.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.008+0000 7f0163400640 1 --2- 172.21.15.86:0/440187190 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f016406d780 0x7f0164158300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.008+0000 7f0163400640 1 -- 172.21.15.86:0/440187190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01640b36f0 msgr2=0x7f01641553c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:22.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.008+0000 7f0163400640 1 --2- 172.21.15.86:0/440187190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01640b36f0 0x7f01641553c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.008+0000 7f0163400640 1 -- 172.21.15.86:0/440187190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0164165a60 con 0x7f0164160430 2024-04-25T14:05:22.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.012+0000 7f0163400640 1 --2- 172.21.15.86:0/440187190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0164160430 0x7f0164155900 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f015c0b1040 tx=0x7f015c01b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:22.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.012+0000 7f015aa00640 1 -- 172.21.15.86:0/440187190 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c00b720 con 0x7f0164160430 2024-04-25T14:05:22.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.012+0000 7f015aa00640 1 -- 172.21.15.86:0/440187190 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f015c0be070 con 0x7f0164160430 2024-04-25T14:05:22.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.012+0000 7f015aa00640 1 -- 172.21.15.86:0/440187190 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c009c30 con 0x7f0164160430 2024-04-25T14:05:22.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.012+0000 7f0163e00640 1 -- 172.21.15.86:0/440187190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0164165d40 con 0x7f0164160430 2024-04-25T14:05:22.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.012+0000 7f0163e00640 1 -- 172.21.15.86:0/440187190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0164152690 con 0x7f0164160430 2024-04-25T14:05:22.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.012+0000 7f0163e00640 1 -- 172.21.15.86:0/440187190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0164076b80 con 0x7f0164160430 2024-04-25T14:05:22.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.012+0000 7f015aa00640 1 -- 172.21.15.86:0/440187190 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 92927+0+0 (secure 0 0 0) 0x7f015c0b6070 con 0x7f0164160430 2024-04-25T14:05:22.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.016+0000 7f015aa00640 1 -- 172.21.15.86:0/440187190 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6359+0+0 (secure 0 0 0) 0x7f015c1299b0 con 0x7f0164160430 2024-04-25T14:05:22.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.020+0000 7f015aa00640 1 -- 172.21.15.86:0/440187190 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f015c0c7020 con 0x7f0164160430 2024-04-25T14:05:22.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.176+0000 7f0163e00640 1 -- 172.21.15.86:0/440187190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f01640b67e0 con 0x7f0164160430 2024-04-25T14:05:22.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.180+0000 7f015aa00640 1 -- 172.21.15.86:0/440187190 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+17319 (secure 0 0 0) 0x7f015c018780 con 0x7f0164160430 2024-04-25T14:05:22.181 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:22.181 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":39,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:05:21.460905+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:05:04.472914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T14:05:05.604280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2073988773":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6833/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6832/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/3558294881":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/1843278167":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/2602032945":"2024-04-26T14:05:21.460855+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:05:22.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.180+0000 7f0163e00640 1 -- 172.21.15.86:0/440187190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0164160430 msgr2=0x7f0164155900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:22.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.180+0000 7f0163e00640 1 --2- 172.21.15.86:0/440187190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0164160430 0x7f0164155900 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f015c0b1040 tx=0x7f015c01b930 comp rx=0 tx=0).stop 2024-04-25T14:05:22.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.180+0000 7f0163e00640 1 -- 172.21.15.86:0/440187190 shutdown_connections 2024-04-25T14:05:22.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.180+0000 7f0163e00640 1 --2- 172.21.15.86:0/440187190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0164160430 0x7f0164155900 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.180+0000 7f0163e00640 1 --2- 172.21.15.86:0/440187190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01640b36f0 0x7f01641553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.180+0000 7f0163e00640 1 --2- 172.21.15.86:0/440187190 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f016406d780 0x7f0164158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.180+0000 7f0163e00640 1 -- 172.21.15.86:0/440187190 >> 172.21.15.86:0/440187190 conn(0x7f016409ba30 msgr2=0x7f016406f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:22.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.180+0000 7f0163e00640 1 -- 172.21.15.86:0/440187190 shutdown_connections 2024-04-25T14:05:22.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.180+0000 7f0163e00640 1 -- 172.21.15.86:0/440187190 wait complete. 2024-04-25T14:05:22.202 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:05:04.472914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:05:05.604280+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:05:22.203 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:05:22.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.356+0000 7f85b5000640 1 Processor -- start 2024-04-25T14:05:22.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.356+0000 7f85b5000640 1 -- start start 2024-04-25T14:05:22.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.356+0000 7f85b5000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85b00ccab0 0x7f85b00d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:22.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.356+0000 7f85b5000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85b0162610 con 0x7f85b00b79c0 2024-04-25T14:05:22.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.356+0000 7f85b5000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85b00b3bd0 con 0x7f85b00cf9d0 2024-04-25T14:05:22.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.356+0000 7f85b5000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f85b00cf190 con 0x7f85b00ccab0 2024-04-25T14:05:22.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.356+0000 7f85ad400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f85b00b79c0 0x7f85b01617f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47344/0 (socket says 172.21.15.86:47344) 2024-04-25T14:05:22.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.356+0000 7f85ad400640 1 -- 172.21.15.86:0/1555730302 learned_addr learned my addr 172.21.15.86:0/1555730302 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:22.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.356+0000 7f85ade00640 1 --2- 172.21.15.86:0/1555730302 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85b00ccab0 0x7f85b00d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:22.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.356+0000 7f85aca00640 1 -- 172.21.15.86:0/1555730302 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 482216689 0 0) 0x7f85b0162610 con 0x7f85b00b79c0 2024-04-25T14:05:22.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.356+0000 7f85aca00640 1 -- 172.21.15.86:0/1555730302 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8594003530 con 0x7f85b00b79c0 2024-04-25T14:05:22.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.356+0000 7f85aca00640 1 -- 172.21.15.86:0/1555730302 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 778266610 0 0) 0x7f85b00b3bd0 con 0x7f85b00cf9d0 2024-04-25T14:05:22.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.356+0000 7f85aca00640 1 -- 172.21.15.86:0/1555730302 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85b0162610 con 0x7f85b00cf9d0 2024-04-25T14:05:22.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.356+0000 7f85ade00640 1 -- 172.21.15.86:0/1555730302 >> v1:172.21.15.155:6789/0 conn(0x7f85b00cf9d0 legacy=0x7f85b0161f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:22.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.356+0000 7f85ade00640 1 -- 172.21.15.86:0/1555730302 >> v1:172.21.15.86:6789/0 conn(0x7f85b00b79c0 legacy=0x7f85b01617f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:22.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.356+0000 7f85ade00640 1 -- 172.21.15.86:0/1555730302 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b00b1cf0 con 0x7f85b00ccab0 2024-04-25T14:05:22.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.360+0000 7f85ade00640 1 --2- 172.21.15.86:0/1555730302 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85b00ccab0 0x7f85b00d23d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f85a00097d0 tx=0x7f85a001c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5647845d89a6415 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:22.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.360+0000 7f85aca00640 1 -- 172.21.15.86:0/1555730302 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a00b1070 con 0x7f85b00ccab0 2024-04-25T14:05:22.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.360+0000 7f85aca00640 1 -- 172.21.15.86:0/1555730302 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85a00ac630 con 0x7f85b00ccab0 2024-04-25T14:05:22.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.360+0000 7f85aca00640 1 -- 172.21.15.86:0/1555730302 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a00a8050 con 0x7f85b00ccab0 2024-04-25T14:05:22.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.360+0000 7f85b5000640 1 -- 172.21.15.86:0/1555730302 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85b00ccab0 msgr2=0x7f85b00d23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:22.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.360+0000 7f85b5000640 1 --2- 172.21.15.86:0/1555730302 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85b00ccab0 0x7f85b00d23d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f85a00097d0 tx=0x7f85a001c4c0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.360+0000 7f85b5000640 1 -- 172.21.15.86:0/1555730302 shutdown_connections 2024-04-25T14:05:22.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.360+0000 7f85b5000640 1 --2- 172.21.15.86:0/1555730302 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85b00ccab0 0x7f85b00d23d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.360+0000 7f85b5000640 1 -- 172.21.15.86:0/1555730302 >> 172.21.15.86:0/1555730302 conn(0x7f85b009ba30 msgr2=0x7f85b00b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:22.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.360+0000 7f85b5000640 1 -- 172.21.15.86:0/1555730302 shutdown_connections 2024-04-25T14:05:22.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.360+0000 7f85b5000640 1 -- 172.21.15.86:0/1555730302 wait complete. 2024-04-25T14:05:22.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.364+0000 7f85b5000640 1 Processor -- start 2024-04-25T14:05:22.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.364+0000 7f85b5000640 1 -- start start 2024-04-25T14:05:22.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.364+0000 7f85b5000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85b00b79c0 0x7f85b01674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:22.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.364+0000 7f85b5000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85b00ccab0 0x7f85b0070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:22.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.364+0000 7f85b5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85b00cf9d0 0x7f85b00711c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:22.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.364+0000 7f85b5000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85b00d2d20 con 0x7f85b00cf9d0 2024-04-25T14:05:22.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.364+0000 7f85b5000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f85b00b2620 con 0x7f85b00b79c0 2024-04-25T14:05:22.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.364+0000 7f85b5000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f85b016c5b0 con 0x7f85b00ccab0 2024-04-25T14:05:22.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.364+0000 7f85ae800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85b00cf9d0 0x7f85b00711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:22.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.364+0000 7f85ae800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85b00cf9d0 0x7f85b00711c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43358/0 (socket says 172.21.15.86:43358) 2024-04-25T14:05:22.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.364+0000 7f85ae800640 1 -- 172.21.15.86:0/3393186743 learned_addr learned my addr 172.21.15.86:0/3393186743 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:22.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.364+0000 7f85ade00640 1 --2- 172.21.15.86:0/3393186743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85b00b79c0 0x7f85b01674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:22.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.364+0000 7f85ad400640 1 --2- 172.21.15.86:0/3393186743 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85b00ccab0 0x7f85b0070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:22.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.364+0000 7f85ae800640 1 -- 172.21.15.86:0/3393186743 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85b00ccab0 msgr2=0x7f85b0070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:22.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.364+0000 7f85ae800640 1 --2- 172.21.15.86:0/3393186743 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85b00ccab0 0x7f85b0070c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.364+0000 7f85ae800640 1 -- 172.21.15.86:0/3393186743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85b00b79c0 msgr2=0x7f85b01674b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:22.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.364+0000 7f85ae800640 1 --2- 172.21.15.86:0/3393186743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85b00b79c0 0x7f85b01674b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.364+0000 7f85ae800640 1 -- 172.21.15.86:0/3393186743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b0167de0 con 0x7f85b00cf9d0 2024-04-25T14:05:22.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.368+0000 7f85ae800640 1 --2- 172.21.15.86:0/3393186743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85b00cf9d0 0x7f85b00711c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f85a800c6f0 tx=0x7f85a800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:22.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.368+0000 7f85a6000640 1 -- 172.21.15.86:0/3393186743 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8003980 con 0x7f85b00cf9d0 2024-04-25T14:05:22.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.368+0000 7f85a6000640 1 -- 172.21.15.86:0/3393186743 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85a8003b20 con 0x7f85b00cf9d0 2024-04-25T14:05:22.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.368+0000 7f85b5000640 1 -- 172.21.15.86:0/3393186743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b01680c0 con 0x7f85b00cf9d0 2024-04-25T14:05:22.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.368+0000 7f85a6000640 1 -- 172.21.15.86:0/3393186743 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a800a850 con 0x7f85b00cf9d0 2024-04-25T14:05:22.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.368+0000 7f85b5000640 1 -- 172.21.15.86:0/3393186743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b0168680 con 0x7f85b00cf9d0 2024-04-25T14:05:22.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.368+0000 7f85b5000640 1 -- 172.21.15.86:0/3393186743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b0068fb0 con 0x7f85b00cf9d0 2024-04-25T14:05:22.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.368+0000 7f85a6000640 1 -- 172.21.15.86:0/3393186743 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 92927+0+0 (secure 0 0 0) 0x7f85a800a9f0 con 0x7f85b00cf9d0 2024-04-25T14:05:22.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.368+0000 7f85a6000640 1 -- 172.21.15.86:0/3393186743 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6359+0+0 (secure 0 0 0) 0x7f85a808ee60 con 0x7f85b00cf9d0 2024-04-25T14:05:22.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.372+0000 7f85a6000640 1 -- 172.21.15.86:0/3393186743 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f85a805b860 con 0x7f85b00cf9d0 2024-04-25T14:05:22.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.468+0000 7f85a6000640 1 -- 172.21.15.86:0/3393186743 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 9) ==== 93054+0+0 (secure 0 0 0) 0x7f85a800ace0 con 0x7f85b00cf9d0 2024-04-25T14:05:22.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.468+0000 7f85a6000640 1 --2- 172.21.15.86:0/3393186743 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f8590074e90 0x7f85900772b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:22.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.472+0000 7f85ade00640 1 --2- 172.21.15.86:0/3393186743 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f8590074e90 0x7f85900772b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:22.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.476+0000 7f85ade00640 1 --2- 172.21.15.86:0/3393186743 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f8590074e90 0x7f85900772b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f85a00ae000 tx=0x7f85a0008040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:22.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.536+0000 7f85b5000640 1 -- 172.21.15.86:0/3393186743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f85b0076bf0 con 0x7f85b00cf9d0 2024-04-25T14:05:22.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.536+0000 7f85a6000640 1 -- 172.21.15.86:0/3393186743 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) ==== 93+0+10 (secure 0 0 0) 0x7f85a807bf40 con 0x7f85b00cf9d0 2024-04-25T14:05:22.540 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:05:22.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.540+0000 7f85b5000640 1 -- 172.21.15.86:0/3393186743 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f8590074e90 msgr2=0x7f85900772b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:22.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.540+0000 7f85b5000640 1 --2- 172.21.15.86:0/3393186743 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f8590074e90 0x7f85900772b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f85a00ae000 tx=0x7f85a0008040 comp rx=0 tx=0).stop 2024-04-25T14:05:22.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.540+0000 7f85b5000640 1 -- 172.21.15.86:0/3393186743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85b00cf9d0 msgr2=0x7f85b00711c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:22.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.540+0000 7f85b5000640 1 --2- 172.21.15.86:0/3393186743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85b00cf9d0 0x7f85b00711c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f85a800c6f0 tx=0x7f85a800cbb0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.540+0000 7f85b5000640 1 -- 172.21.15.86:0/3393186743 shutdown_connections 2024-04-25T14:05:22.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.540+0000 7f85b5000640 1 --2- 172.21.15.86:0/3393186743 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f8590074e90 0x7f85900772b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.540+0000 7f85b5000640 1 --2- 172.21.15.86:0/3393186743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85b00cf9d0 0x7f85b00711c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.540+0000 7f85b5000640 1 --2- 172.21.15.86:0/3393186743 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85b00ccab0 0x7f85b0070c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.540+0000 7f85b5000640 1 --2- 172.21.15.86:0/3393186743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85b00b79c0 0x7f85b01674b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.540+0000 7f85b5000640 1 -- 172.21.15.86:0/3393186743 >> 172.21.15.86:0/3393186743 conn(0x7f85b009ba30 msgr2=0x7f85b0159b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:22.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.540+0000 7f85b5000640 1 -- 172.21.15.86:0/3393186743 shutdown_connections 2024-04-25T14:05:22.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.540+0000 7f85b5000640 1 -- 172.21.15.86:0/3393186743 wait complete. 2024-04-25T14:05:22.561 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:05:22.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.708+0000 7fe4e9c00640 1 Processor -- start 2024-04-25T14:05:22.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.708+0000 7fe4e9c00640 1 -- start start 2024-04-25T14:05:22.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.708+0000 7fe4e9c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe4e40d16e0 0x7fe4e40d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:22.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.708+0000 7fe4e9c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4e40730b0 0x7fe4e40734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:22.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.708+0000 7fe4e9c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4e40d2030 con 0x7fe4e40cfca0 2024-04-25T14:05:22.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.708+0000 7fe4e9c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe4e4070470 con 0x7fe4e40d16e0 2024-04-25T14:05:22.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.708+0000 7fe4e9c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe4e4072bd0 con 0x7fe4e40730b0 2024-04-25T14:05:22.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.712+0000 7fe4e2000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe4e40d16e0 0x7fe4e40d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:22.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.712+0000 7fe4e2a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe4e40cfca0 0x7fe4e406f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47348/0 (socket says 172.21.15.86:47348) 2024-04-25T14:05:22.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.712+0000 7fe4e2a00640 1 -- 172.21.15.86:0/1197369121 learned_addr learned my addr 172.21.15.86:0/1197369121 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:22.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.712+0000 7fe4e3400640 1 --2- 172.21.15.86:0/1197369121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4e40730b0 0x7fe4e40734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:22.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.712+0000 7fe4e2000640 1 -- 172.21.15.86:0/1197369121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4e40730b0 msgr2=0x7fe4e40734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:22.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.712+0000 7fe4e2000640 1 --2- 172.21.15.86:0/1197369121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4e40730b0 0x7fe4e40734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.712+0000 7fe4e2000640 1 -- 172.21.15.86:0/1197369121 >> v1:172.21.15.86:6789/0 conn(0x7fe4e40cfca0 legacy=0x7fe4e406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:22.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.712+0000 7fe4e1600640 1 -- 172.21.15.86:0/1197369121 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 921097763 0 0) 0x7fe4d4001120 con 0x7fe4e40cfca0 2024-04-25T14:05:22.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.712+0000 7fe4e2000640 1 -- 172.21.15.86:0/1197369121 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e406fbd0 con 0x7fe4e40d16e0 2024-04-25T14:05:22.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.712+0000 7fe4e2000640 1 --2- 172.21.15.86:0/1197369121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4e40d16e0 0x7fe4e40d1af0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe4d00097c0 tx=0x7fe4d001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87dfb1dabed690c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:22.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.712+0000 7fe4e1600640 1 -- 172.21.15.86:0/1197369121 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d00b1070 con 0x7fe4e40d16e0 2024-04-25T14:05:22.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.712+0000 7fe4e1600640 1 -- 172.21.15.86:0/1197369121 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4d00ac410 con 0x7fe4e40d16e0 2024-04-25T14:05:22.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.712+0000 7fe4e1600640 1 -- 172.21.15.86:0/1197369121 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d00ac7e0 con 0x7fe4e40d16e0 2024-04-25T14:05:22.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.716+0000 7fe4e9c00640 1 -- 172.21.15.86:0/1197369121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4e40d16e0 msgr2=0x7fe4e40d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:22.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.716+0000 7fe4e9c00640 1 --2- 172.21.15.86:0/1197369121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4e40d16e0 0x7fe4e40d1af0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe4d00097c0 tx=0x7fe4d001c4a0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.716+0000 7fe4e9c00640 1 -- 172.21.15.86:0/1197369121 shutdown_connections 2024-04-25T14:05:22.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.716+0000 7fe4e9c00640 1 --2- 172.21.15.86:0/1197369121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4e40730b0 0x7fe4e40734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.716+0000 7fe4e9c00640 1 --2- 172.21.15.86:0/1197369121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4e40d16e0 0x7fe4e40d1af0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.716+0000 7fe4e9c00640 1 -- 172.21.15.86:0/1197369121 >> 172.21.15.86:0/1197369121 conn(0x7fe4e40b4650 msgr2=0x7fe4e40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:22.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.716+0000 7fe4e9c00640 1 -- 172.21.15.86:0/1197369121 shutdown_connections 2024-04-25T14:05:22.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.716+0000 7fe4e9c00640 1 -- 172.21.15.86:0/1197369121 wait complete. 2024-04-25T14:05:22.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.716+0000 7fe4e9c00640 1 Processor -- start 2024-04-25T14:05:22.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.716+0000 7fe4e9c00640 1 -- start start 2024-04-25T14:05:22.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.720+0000 7fe4e9c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4e40730b0 0x7fe4e409a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:22.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.720+0000 7fe4e9c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4e40cfca0 0x7fe4e415a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:22.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.720+0000 7fe4e9c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4e40d16e0 0x7fe4e415a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:22.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.720+0000 7fe4e9c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e4169a70 con 0x7fe4e40d16e0 2024-04-25T14:05:22.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.720+0000 7fe4e9c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e41698f0 con 0x7fe4e40cfca0 2024-04-25T14:05:22.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.720+0000 7fe4e9c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe4e4169bf0 con 0x7fe4e40730b0 2024-04-25T14:05:22.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.720+0000 7fe4e3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4e40d16e0 0x7fe4e415a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:22.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.720+0000 7fe4e3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4e40d16e0 0x7fe4e415a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43368/0 (socket says 172.21.15.86:43368) 2024-04-25T14:05:22.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.720+0000 7fe4e3400640 1 -- 172.21.15.86:0/1415494160 learned_addr learned my addr 172.21.15.86:0/1415494160 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:22.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.720+0000 7fe4e2a00640 1 --2- 172.21.15.86:0/1415494160 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4e40730b0 0x7fe4e409a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:22.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.720+0000 7fe4e2000640 1 --2- 172.21.15.86:0/1415494160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4e40cfca0 0x7fe4e415a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:22.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.720+0000 7fe4e3400640 1 -- 172.21.15.86:0/1415494160 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4e40730b0 msgr2=0x7fe4e409a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:22.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.720+0000 7fe4e3400640 1 --2- 172.21.15.86:0/1415494160 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4e40730b0 0x7fe4e409a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.720+0000 7fe4e3400640 1 -- 172.21.15.86:0/1415494160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4e40cfca0 msgr2=0x7fe4e415a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:22.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.720+0000 7fe4e3400640 1 --2- 172.21.15.86:0/1415494160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4e40cfca0 0x7fe4e415a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.720+0000 7fe4e3400640 1 -- 172.21.15.86:0/1415494160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e40cdd40 con 0x7fe4e40d16e0 2024-04-25T14:05:22.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.720+0000 7fe4e3400640 1 --2- 172.21.15.86:0/1415494160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4e40d16e0 0x7fe4e415a9f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00efc0 tx=0x7fe4dc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:22.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.720+0000 7fe4daa00640 1 -- 172.21.15.86:0/1415494160 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc018020 con 0x7fe4e40d16e0 2024-04-25T14:05:22.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.720+0000 7fe4daa00640 1 -- 172.21.15.86:0/1415494160 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4dc014480 con 0x7fe4e40d16e0 2024-04-25T14:05:22.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.720+0000 7fe4daa00640 1 -- 172.21.15.86:0/1415494160 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc009e60 con 0x7fe4e40d16e0 2024-04-25T14:05:22.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.720+0000 7fe4e9c00640 1 -- 172.21.15.86:0/1415494160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e40ce020 con 0x7fe4e40d16e0 2024-04-25T14:05:22.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.720+0000 7fe4e9c00640 1 -- 172.21.15.86:0/1415494160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e40ce5e0 con 0x7fe4e40d16e0 2024-04-25T14:05:22.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.720+0000 7fe4e9c00640 1 -- 172.21.15.86:0/1415494160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4e40cdab0 con 0x7fe4e40d16e0 2024-04-25T14:05:22.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.720+0000 7fe4daa00640 1 -- 172.21.15.86:0/1415494160 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 9) ==== 93054+0+0 (secure 0 0 0) 0x7fe4dc01b020 con 0x7fe4e40d16e0 2024-04-25T14:05:22.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.724+0000 7fe4daa00640 1 --2- 172.21.15.86:0/1415494160 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fe4c406f2d0 0x7fe4c40717e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:22.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.724+0000 7fe4daa00640 1 -- 172.21.15.86:0/1415494160 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6359+0+0 (secure 0 0 0) 0x7fe4dc08ffe0 con 0x7fe4e40d16e0 2024-04-25T14:05:22.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.724+0000 7fe4e2a00640 1 --2- 172.21.15.86:0/1415494160 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fe4c406f2d0 0x7fe4c40717e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:22.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.724+0000 7fe4e2a00640 1 --2- 172.21.15.86:0/1415494160 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fe4c406f2d0 0x7fe4c40717e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4000f70 tx=0x7fe4d4007c10 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:22.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.728+0000 7fe4daa00640 1 -- 172.21.15.86:0/1415494160 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe4dc022020 con 0x7fe4e40d16e0 2024-04-25T14:05:22.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.884+0000 7fe4e9c00640 1 -- 172.21.15.86:0/1415494160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe4e409f4b0 con 0x7fe4e40d16e0 2024-04-25T14:05:22.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.884+0000 7fe4daa00640 1 -- 172.21.15.86:0/1415494160 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) ==== 104+0+11 (secure 0 0 0) 0x7fe4dc010040 con 0x7fe4e40d16e0 2024-04-25T14:05:22.886 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:05:22.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.884+0000 7fe4e9c00640 1 -- 172.21.15.86:0/1415494160 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fe4c406f2d0 msgr2=0x7fe4c40717e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:22.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.884+0000 7fe4e9c00640 1 --2- 172.21.15.86:0/1415494160 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fe4c406f2d0 0x7fe4c40717e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4000f70 tx=0x7fe4d4007c10 comp rx=0 tx=0).stop 2024-04-25T14:05:22.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.884+0000 7fe4e9c00640 1 -- 172.21.15.86:0/1415494160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4e40d16e0 msgr2=0x7fe4e415a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:22.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.884+0000 7fe4e9c00640 1 --2- 172.21.15.86:0/1415494160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4e40d16e0 0x7fe4e415a9f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00efc0 tx=0x7fe4dc00c490 comp rx=0 tx=0).stop 2024-04-25T14:05:22.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.884+0000 7fe4e9c00640 1 -- 172.21.15.86:0/1415494160 shutdown_connections 2024-04-25T14:05:22.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.884+0000 7fe4e9c00640 1 --2- 172.21.15.86:0/1415494160 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fe4c406f2d0 0x7fe4c40717e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.884+0000 7fe4e9c00640 1 --2- 172.21.15.86:0/1415494160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4e40d16e0 0x7fe4e415a9f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.884+0000 7fe4e9c00640 1 --2- 172.21.15.86:0/1415494160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4e40cfca0 0x7fe4e415a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.884+0000 7fe4e9c00640 1 --2- 172.21.15.86:0/1415494160 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4e40730b0 0x7fe4e409a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:22.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.884+0000 7fe4e9c00640 1 -- 172.21.15.86:0/1415494160 >> 172.21.15.86:0/1415494160 conn(0x7fe4e40b4650 msgr2=0x7fe4e40b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:22.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.884+0000 7fe4e9c00640 1 -- 172.21.15.86:0/1415494160 shutdown_connections 2024-04-25T14:05:22.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:22.884+0000 7fe4e9c00640 1 -- 172.21.15.86:0/1415494160 wait complete. 2024-04-25T14:05:22.907 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:05:23.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.052+0000 7f253b200640 1 Processor -- start 2024-04-25T14:05:23.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.056+0000 7f253b200640 1 -- start start 2024-04-25T14:05:23.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.056+0000 7f253b200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f25340d1df0 0x7f25340cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:23.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.056+0000 7f253b200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f25340cdcc0 0x7f25340ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:23.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.056+0000 7f253b200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f253415e5c0 con 0x7f2534070330 2024-04-25T14:05:23.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.056+0000 7f253b200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f253406fd40 con 0x7f25340d1df0 2024-04-25T14:05:23.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.056+0000 7f253b200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f253415e8a0 con 0x7f25340cdcc0 2024-04-25T14:05:23.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.056+0000 7f252b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f25340d1df0 0x7f25340cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:23.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.056+0000 7f252b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f25340d1df0 0x7f25340cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43376/0 (socket says 172.21.15.86:43376) 2024-04-25T14:05:23.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.056+0000 7f252b400640 1 -- 172.21.15.86:0/3778762416 learned_addr learned my addr 172.21.15.86:0/3778762416 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:23.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.056+0000 7f2539200640 1 --2- 172.21.15.86:0/3778762416 >> v2:172.21.15.155:3300/0 conn(0x7f25340cdcc0 0x7f25340ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:23.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.056+0000 7f252b400640 1 -- 172.21.15.86:0/3778762416 >> v2:172.21.15.155:3300/0 conn(0x7f25340cdcc0 msgr2=0x7f25340ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:23.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.056+0000 7f252b400640 1 --2- 172.21.15.86:0/3778762416 >> v2:172.21.15.155:3300/0 conn(0x7f25340cdcc0 0x7f25340ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:23.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.056+0000 7f252b400640 1 -- 172.21.15.86:0/3778762416 >> v1:172.21.15.155:6789/0 conn(0x7f2534070330 legacy=0x7f25340d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:23.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.056+0000 7f252b400640 1 -- 172.21.15.86:0/3778762416 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f253415eb80 con 0x7f25340d1df0 2024-04-25T14:05:23.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.060+0000 7f252b400640 1 --2- 172.21.15.86:0/3778762416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25340d1df0 0x7f25340cd780 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f251c009870 tx=0x7f251c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a53292e8a294e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:23.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.060+0000 7f2533400640 1 -- 172.21.15.86:0/3778762416 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c0ab640 con 0x7f25340d1df0 2024-04-25T14:05:23.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.060+0000 7f2533400640 1 -- 172.21.15.86:0/3778762416 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f251c0b0070 con 0x7f25340d1df0 2024-04-25T14:05:23.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.060+0000 7f2533400640 1 -- 172.21.15.86:0/3778762416 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c0aba00 con 0x7f25340d1df0 2024-04-25T14:05:23.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.060+0000 7f253b200640 1 -- 172.21.15.86:0/3778762416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25340d1df0 msgr2=0x7f25340cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:23.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.060+0000 7f253b200640 1 --2- 172.21.15.86:0/3778762416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25340d1df0 0x7f25340cd780 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f251c009870 tx=0x7f251c01c460 comp rx=0 tx=0).stop 2024-04-25T14:05:23.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.060+0000 7f253b200640 1 -- 172.21.15.86:0/3778762416 shutdown_connections 2024-04-25T14:05:23.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.060+0000 7f253b200640 1 --2- 172.21.15.86:0/3778762416 >> v2:172.21.15.155:3300/0 conn(0x7f25340cdcc0 0x7f25340ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:23.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.060+0000 7f253b200640 1 --2- 172.21.15.86:0/3778762416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25340d1df0 0x7f25340cd780 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:23.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.060+0000 7f253b200640 1 -- 172.21.15.86:0/3778762416 >> 172.21.15.86:0/3778762416 conn(0x7f25340b4650 msgr2=0x7f25340b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:23.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.060+0000 7f253b200640 1 -- 172.21.15.86:0/3778762416 shutdown_connections 2024-04-25T14:05:23.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.060+0000 7f253b200640 1 -- 172.21.15.86:0/3778762416 wait complete. 2024-04-25T14:05:23.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.064+0000 7f253b200640 1 Processor -- start 2024-04-25T14:05:23.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.064+0000 7f253b200640 1 -- start start 2024-04-25T14:05:23.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.064+0000 7f253b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2534070330 0x7f25340ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:23.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.064+0000 7f253b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25340cdcc0 0x7f25340af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:23.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.064+0000 7f253b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25340caaa0 0x7f25340af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:23.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.064+0000 7f253b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f253415f4b0 con 0x7f25340caaa0 2024-04-25T14:05:23.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.064+0000 7f253b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f253415f330 con 0x7f2534070330 2024-04-25T14:05:23.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.064+0000 7f253b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2534070740 con 0x7f25340cdcc0 2024-04-25T14:05:23.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.064+0000 7f2539200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25340caaa0 0x7f25340af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:23.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.064+0000 7f2539200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25340caaa0 0x7f25340af7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43390/0 (socket says 172.21.15.86:43390) 2024-04-25T14:05:23.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.064+0000 7f2539200640 1 -- 172.21.15.86:0/2805752357 learned_addr learned my addr 172.21.15.86:0/2805752357 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:23.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.064+0000 7f2533e00640 1 --2- 172.21.15.86:0/2805752357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2534070330 0x7f25340ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:23.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.064+0000 7f252b400640 1 --2- 172.21.15.86:0/2805752357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25340cdcc0 0x7f25340af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:23.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.064+0000 7f2539200640 1 -- 172.21.15.86:0/2805752357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25340cdcc0 msgr2=0x7f25340af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:23.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.064+0000 7f2539200640 1 --2- 172.21.15.86:0/2805752357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25340cdcc0 0x7f25340af2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:23.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.064+0000 7f2539200640 1 -- 172.21.15.86:0/2805752357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2534070330 msgr2=0x7f25340ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:23.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.064+0000 7f2539200640 1 --2- 172.21.15.86:0/2805752357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2534070330 0x7f25340ca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:23.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.064+0000 7f2539200640 1 -- 172.21.15.86:0/2805752357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2534165e00 con 0x7f25340caaa0 2024-04-25T14:05:23.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.064+0000 7f2533e00640 1 --2- 172.21.15.86:0/2805752357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2534070330 0x7f25340ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:23.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.064+0000 7f2539200640 1 --2- 172.21.15.86:0/2805752357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25340caaa0 0x7f25340af7f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f252c009cb0 tx=0x7f252c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:23.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.064+0000 7f2530c00640 1 -- 172.21.15.86:0/2805752357 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c018020 con 0x7f25340caaa0 2024-04-25T14:05:23.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.064+0000 7f2530c00640 1 -- 172.21.15.86:0/2805752357 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f252c014480 con 0x7f25340caaa0 2024-04-25T14:05:23.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.064+0000 7f253b200640 1 -- 172.21.15.86:0/2805752357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2534166070 con 0x7f25340caaa0 2024-04-25T14:05:23.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.064+0000 7f2530c00640 1 -- 172.21.15.86:0/2805752357 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c014870 con 0x7f25340caaa0 2024-04-25T14:05:23.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.064+0000 7f253b200640 1 -- 172.21.15.86:0/2805752357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2534166630 con 0x7f25340caaa0 2024-04-25T14:05:23.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.068+0000 7f253b200640 1 -- 172.21.15.86:0/2805752357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2534069d00 con 0x7f25340caaa0 2024-04-25T14:05:23.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.068+0000 7f2530c00640 1 -- 172.21.15.86:0/2805752357 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 9) ==== 93054+0+0 (secure 0 0 0) 0x7f252c01b020 con 0x7f25340caaa0 2024-04-25T14:05:23.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.068+0000 7f2530c00640 1 --2- 172.21.15.86:0/2805752357 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f251006f2d0 0x7f25100717e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:23.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.068+0000 7f2530c00640 1 -- 172.21.15.86:0/2805752357 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6359+0+0 (secure 0 0 0) 0x7f252c08fe60 con 0x7f25340caaa0 2024-04-25T14:05:23.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.068+0000 7f2533e00640 1 --2- 172.21.15.86:0/2805752357 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f251006f2d0 0x7f25100717e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:23.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.068+0000 7f2533e00640 1 --2- 172.21.15.86:0/2805752357 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f251006f2d0 0x7f25100717e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2534072200 tx=0x7f2524009040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:23.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.072+0000 7f2530c00640 1 -- 172.21.15.86:0/2805752357 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f252c022030 con 0x7f25340caaa0 2024-04-25T14:05:23.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.232+0000 7f253b200640 1 -- 172.21.15.86:0/2805752357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f253409f430 con 0x7f25340caaa0 2024-04-25T14:05:23.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.232+0000 7f2530c00640 1 -- 172.21.15.86:0/2805752357 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) ==== 100+0+11 (secure 0 0 0) 0x7f252c010040 con 0x7f25340caaa0 2024-04-25T14:05:23.234 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:05:23.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.232+0000 7f253b200640 1 -- 172.21.15.86:0/2805752357 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f251006f2d0 msgr2=0x7f25100717e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:23.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.232+0000 7f253b200640 1 --2- 172.21.15.86:0/2805752357 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f251006f2d0 0x7f25100717e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2534072200 tx=0x7f2524009040 comp rx=0 tx=0).stop 2024-04-25T14:05:23.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.232+0000 7f253b200640 1 -- 172.21.15.86:0/2805752357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25340caaa0 msgr2=0x7f25340af7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:23.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.232+0000 7f253b200640 1 --2- 172.21.15.86:0/2805752357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25340caaa0 0x7f25340af7f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f252c009cb0 tx=0x7f252c00c490 comp rx=0 tx=0).stop 2024-04-25T14:05:23.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.232+0000 7f253b200640 1 -- 172.21.15.86:0/2805752357 shutdown_connections 2024-04-25T14:05:23.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.232+0000 7f253b200640 1 --2- 172.21.15.86:0/2805752357 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f251006f2d0 0x7f25100717e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:23.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.232+0000 7f253b200640 1 --2- 172.21.15.86:0/2805752357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25340caaa0 0x7f25340af7f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:23.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.232+0000 7f253b200640 1 --2- 172.21.15.86:0/2805752357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25340cdcc0 0x7f25340af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:23.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.232+0000 7f253b200640 1 --2- 172.21.15.86:0/2805752357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2534070330 0x7f25340ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:23.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.232+0000 7f253b200640 1 -- 172.21.15.86:0/2805752357 >> 172.21.15.86:0/2805752357 conn(0x7f25340b4650 msgr2=0x7f25340b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:23.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.232+0000 7f253b200640 1 -- 172.21.15.86:0/2805752357 shutdown_connections 2024-04-25T14:05:23.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.232+0000 7f253b200640 1 -- 172.21.15.86:0/2805752357 wait complete. 2024-04-25T14:05:23.256 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:05:23.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.400+0000 7f1ee7800640 1 Processor -- start 2024-04-25T14:05:23.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.400+0000 7f1ee7800640 1 -- start start 2024-04-25T14:05:23.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.404+0000 7f1ee7800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1ee00d15e0 0x7f1ee00d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:23.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.404+0000 7f1ee7800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ee00b3750 con 0x7f1ee01610d0 2024-04-25T14:05:23.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.404+0000 7f1ee7800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ee00ce7b0 con 0x7f1ee00b7a30 2024-04-25T14:05:23.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.404+0000 7f1ee7800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1ee0161620 con 0x7f1ee00d15e0 2024-04-25T14:05:23.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.404+0000 7f1ee4e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1ee01610d0 0x7f1ee00b2250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47362/0 (socket says 172.21.15.86:47362) 2024-04-25T14:05:23.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.404+0000 7f1ee4e00640 1 -- 172.21.15.86:0/1198247573 learned_addr learned my addr 172.21.15.86:0/1198247573 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:23.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.404+0000 7f1ee5800640 1 --2- 172.21.15.86:0/1198247573 >> v2:172.21.15.155:3300/0 conn(0x7f1ee00d15e0 0x7f1ee00d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:23.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.404+0000 7f1edf400640 1 -- 172.21.15.86:0/1198247573 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2474432019 0 0) 0x7f1ee00b3750 con 0x7f1ee01610d0 2024-04-25T14:05:23.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.404+0000 7f1edf400640 1 -- 172.21.15.86:0/1198247573 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ec4003530 con 0x7f1ee01610d0 2024-04-25T14:05:23.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.404+0000 7f1ee5800640 1 -- 172.21.15.86:0/1198247573 >> v1:172.21.15.155:6789/0 conn(0x7f1ee00b7a30 legacy=0x7f1ee00b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:23.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.404+0000 7f1edf400640 1 -- 172.21.15.86:0/1198247573 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4027182142 0 0) 0x7f1ec4003530 con 0x7f1ee01610d0 2024-04-25T14:05:23.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.404+0000 7f1ee5800640 1 -- 172.21.15.86:0/1198247573 >> v1:172.21.15.86:6789/0 conn(0x7f1ee01610d0 legacy=0x7f1ee00b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:23.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.404+0000 7f1ee5800640 1 -- 172.21.15.86:0/1198247573 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee00b39f0 con 0x7f1ee00d15e0 2024-04-25T14:05:23.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.408+0000 7f1ee5800640 1 --2- 172.21.15.86:0/1198247573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ee00d15e0 0x7f1ee00d19f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1ed80097c0 tx=0x7f1ed801c270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38431c129ac451db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:23.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.408+0000 7f1edf400640 1 -- 172.21.15.86:0/1198247573 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed80b0070 con 0x7f1ee00d15e0 2024-04-25T14:05:23.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.408+0000 7f1edf400640 1 -- 172.21.15.86:0/1198247573 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ed80ab400 con 0x7f1ee00d15e0 2024-04-25T14:05:23.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.408+0000 7f1edf400640 1 -- 172.21.15.86:0/1198247573 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0001a20 con 0x7f1ee00d15e0 2024-04-25T14:05:23.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.408+0000 7f1ee7800640 1 -- 172.21.15.86:0/1198247573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ee00d15e0 msgr2=0x7f1ee00d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:23.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.408+0000 7f1ee7800640 1 --2- 172.21.15.86:0/1198247573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ee00d15e0 0x7f1ee00d19f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1ed80097c0 tx=0x7f1ed801c270 comp rx=0 tx=0).stop 2024-04-25T14:05:23.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.408+0000 7f1ee7800640 1 -- 172.21.15.86:0/1198247573 shutdown_connections 2024-04-25T14:05:23.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.408+0000 7f1ee7800640 1 --2- 172.21.15.86:0/1198247573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ee00d15e0 0x7f1ee00d19f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:23.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.408+0000 7f1ee7800640 1 -- 172.21.15.86:0/1198247573 >> 172.21.15.86:0/1198247573 conn(0x7f1ee009ba30 msgr2=0x7f1ee00b7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:23.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.408+0000 7f1ee7800640 1 -- 172.21.15.86:0/1198247573 shutdown_connections 2024-04-25T14:05:23.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.408+0000 7f1ee7800640 1 -- 172.21.15.86:0/1198247573 wait complete. 2024-04-25T14:05:23.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.412+0000 7f1ee7800640 1 Processor -- start 2024-04-25T14:05:23.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.412+0000 7f1ee7800640 1 -- start start 2024-04-25T14:05:23.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.412+0000 7f1ee7800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ee00b7a30 0x7f1ee0156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:23.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.412+0000 7f1ee7800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ee00d15e0 0x7f1ee00b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:23.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.412+0000 7f1ee7800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ee01610d0 0x7f1ee00b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:23.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.412+0000 7f1ee7800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee016e8a0 con 0x7f1ee00d15e0 2024-04-25T14:05:23.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.412+0000 7f1ee7800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee016e720 con 0x7f1ee00b7a30 2024-04-25T14:05:23.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.412+0000 7f1ee7800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1ee016ea20 con 0x7f1ee01610d0 2024-04-25T14:05:23.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.412+0000 7f1edfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ee00d15e0 0x7f1ee00b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:23.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.412+0000 7f1edfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ee00d15e0 0x7f1ee00b17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43392/0 (socket says 172.21.15.86:43392) 2024-04-25T14:05:23.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.412+0000 7f1edfe00640 1 -- 172.21.15.86:0/735742844 learned_addr learned my addr 172.21.15.86:0/735742844 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:23.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.412+0000 7f1ee5800640 1 --2- 172.21.15.86:0/735742844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ee01610d0 0x7f1ee00b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:23.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.412+0000 7f1ee4e00640 1 --2- 172.21.15.86:0/735742844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ee00b7a30 0x7f1ee0156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:23.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.412+0000 7f1edfe00640 1 -- 172.21.15.86:0/735742844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ee01610d0 msgr2=0x7f1ee00b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:23.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.412+0000 7f1edfe00640 1 --2- 172.21.15.86:0/735742844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ee01610d0 0x7f1ee00b1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:23.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.412+0000 7f1edfe00640 1 -- 172.21.15.86:0/735742844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ee00b7a30 msgr2=0x7f1ee0156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:23.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.412+0000 7f1edfe00640 1 --2- 172.21.15.86:0/735742844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ee00b7a30 0x7f1ee0156830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:23.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.412+0000 7f1edfe00640 1 -- 172.21.15.86:0/735742844 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee00c6e00 con 0x7f1ee00d15e0 2024-04-25T14:05:23.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.412+0000 7f1edfe00640 1 --2- 172.21.15.86:0/735742844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ee00d15e0 0x7f1ee00b17f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc00c500 tx=0x7f1ecc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:23.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.416+0000 7f1edcc00640 1 -- 172.21.15.86:0/735742844 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc017070 con 0x7f1ee00d15e0 2024-04-25T14:05:23.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.416+0000 7f1edcc00640 1 -- 172.21.15.86:0/735742844 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ecc003600 con 0x7f1ee00d15e0 2024-04-25T14:05:23.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.416+0000 7f1edcc00640 1 -- 172.21.15.86:0/735742844 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc0039f0 con 0x7f1ee00d15e0 2024-04-25T14:05:23.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.416+0000 7f1ee7800640 1 -- 172.21.15.86:0/735742844 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ee00b0030 con 0x7f1ee00d15e0 2024-04-25T14:05:23.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.416+0000 7f1ee7800640 1 -- 172.21.15.86:0/735742844 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ee00c7370 con 0x7f1ee00d15e0 2024-04-25T14:05:23.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.416+0000 7f1ee7800640 1 -- 172.21.15.86:0/735742844 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ee00a05a0 con 0x7f1ee00d15e0 2024-04-25T14:05:23.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.416+0000 7f1edcc00640 1 -- 172.21.15.86:0/735742844 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 9) ==== 93054+0+0 (secure 0 0 0) 0x7f1ecc003b90 con 0x7f1ee00d15e0 2024-04-25T14:05:23.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.416+0000 7f1edcc00640 1 --2- 172.21.15.86:0/735742844 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f1ec006f2a0 0x7f1ec00717b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:23.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.416+0000 7f1edcc00640 1 -- 172.21.15.86:0/735742844 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6359+0+0 (secure 0 0 0) 0x7f1ecc08e410 con 0x7f1ee00d15e0 2024-04-25T14:05:23.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.416+0000 7f1ee4e00640 1 --2- 172.21.15.86:0/735742844 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f1ec006f2a0 0x7f1ec00717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:23.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.420+0000 7f1ee4e00640 1 --2- 172.21.15.86:0/735742844 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f1ec006f2a0 0x7f1ec00717b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0002240 tx=0x7f1ed0001cd0 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:23.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.420+0000 7f1edcc00640 1 -- 172.21.15.86:0/735742844 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1ecc05ad20 con 0x7f1ee00d15e0 2024-04-25T14:05:23.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.480+0000 7f1edcc00640 1 -- 172.21.15.86:0/735742844 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 10) ==== 93262+0+0 (secure 0 0 0) 0x7f1ecc056fb0 con 0x7f1ee00d15e0 2024-04-25T14:05:23.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.624+0000 7f1ee7800640 1 -- 172.21.15.86:0/735742844 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ee0076bf0 con 0x7f1ee00d15e0 2024-04-25T14:05:23.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.624+0000 7f1edcc00640 1 -- 172.21.15.86:0/735742844 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5184 (secure 0 0 0) 0x7f1ecc07e900 con 0x7f1ee00d15e0 2024-04-25T14:05:23.626 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:23.626 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3546153119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3546153119},{"type":"v1","addr":"172.21.15.155:6834","nonce":3546153119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5283,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3499340224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3499340224},{"type":"v1","addr":"172.21.15.86:6836","nonce":3499340224}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5284,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1466978742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1466978742},{"type":"v1","addr":"172.21.15.155:6835","nonce":1466978742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:06.638216+0000","modified":"2024-04-25T14:05:09.664776+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5286":{"gid":5286,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/716923566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":716923566},{"type":"v1","addr":"172.21.15.86:6837","nonce":716923566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5286,"qdb_cluster":[5286]},"id":2}]} 2024-04-25T14:05:23.626 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 18 2024-04-25T14:05:23.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.624+0000 7f1ee7800640 1 -- 172.21.15.86:0/735742844 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f1ec006f2a0 msgr2=0x7f1ec00717b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:23.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.624+0000 7f1ee7800640 1 --2- 172.21.15.86:0/735742844 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f1ec006f2a0 0x7f1ec00717b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0002240 tx=0x7f1ed0001cd0 comp rx=0 tx=0).stop 2024-04-25T14:05:23.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.624+0000 7f1ee7800640 1 -- 172.21.15.86:0/735742844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ee00d15e0 msgr2=0x7f1ee00b17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:23.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.624+0000 7f1ee7800640 1 --2- 172.21.15.86:0/735742844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ee00d15e0 0x7f1ee00b17f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc00c500 tx=0x7f1ecc00c9c0 comp rx=0 tx=0).stop 2024-04-25T14:05:23.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.624+0000 7f1ee7800640 1 -- 172.21.15.86:0/735742844 shutdown_connections 2024-04-25T14:05:23.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.624+0000 7f1ee7800640 1 --2- 172.21.15.86:0/735742844 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f1ec006f2a0 0x7f1ec00717b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:23.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.624+0000 7f1ee7800640 1 --2- 172.21.15.86:0/735742844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ee01610d0 0x7f1ee00b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:23.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.624+0000 7f1ee7800640 1 --2- 172.21.15.86:0/735742844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ee00d15e0 0x7f1ee00b17f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:23.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.624+0000 7f1ee7800640 1 --2- 172.21.15.86:0/735742844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ee00b7a30 0x7f1ee0156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:23.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.624+0000 7f1ee7800640 1 -- 172.21.15.86:0/735742844 >> 172.21.15.86:0/735742844 conn(0x7f1ee009ba30 msgr2=0x7f1ee006c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:23.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.624+0000 7f1ee7800640 1 -- 172.21.15.86:0/735742844 shutdown_connections 2024-04-25T14:05:23.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.624+0000 7f1ee7800640 1 -- 172.21.15.86:0/735742844 wait complete. 2024-04-25T14:05:23.647 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:05:23.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.792+0000 7fcd7be00640 1 Processor -- start 2024-04-25T14:05:23.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.792+0000 7fcd7be00640 1 -- start start 2024-04-25T14:05:23.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.792+0000 7fcd7be00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcd7c0d17c0 0x7fcd7c0d2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:23.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.792+0000 7fcd7be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcd7c06d780 0x7fcd7c160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:23.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.792+0000 7fcd7be00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd7c09be50 con 0x7fcd7c06db90 2024-04-25T14:05:23.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.792+0000 7fcd7be00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcd7c0b1c40 con 0x7fcd7c06d780 2024-04-25T14:05:23.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.792+0000 7fcd7be00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fcd7c0b1680 con 0x7fcd7c0d17c0 2024-04-25T14:05:23.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.792+0000 7fcd7a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcd7c06d780 0x7fcd7c160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:23.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.792+0000 7fcd7a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcd7c06d780 0x7fcd7c160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43428/0 (socket says 172.21.15.86:43428) 2024-04-25T14:05:23.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.792+0000 7fcd7a000640 1 -- 172.21.15.86:0/3739575492 learned_addr learned my addr 172.21.15.86:0/3739575492 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:23.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.792+0000 7fcd7aa00640 1 --2- 172.21.15.86:0/3739575492 >> v2:172.21.15.155:3300/0 conn(0x7fcd7c0d17c0 0x7fcd7c0d2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:23.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.792+0000 7fcd7a000640 1 -- 172.21.15.86:0/3739575492 >> v2:172.21.15.155:3300/0 conn(0x7fcd7c0d17c0 msgr2=0x7fcd7c0d2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:23.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.792+0000 7fcd7a000640 1 --2- 172.21.15.86:0/3739575492 >> v2:172.21.15.155:3300/0 conn(0x7fcd7c0d17c0 0x7fcd7c0d2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:23.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.792+0000 7fcd7a000640 1 -- 172.21.15.86:0/3739575492 >> v1:172.21.15.86:6789/0 conn(0x7fcd7c06db90 legacy=0x7fcd7c15d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:23.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.792+0000 7fcd7a000640 1 -- 172.21.15.86:0/3739575492 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7c0b1960 con 0x7fcd7c06d780 2024-04-25T14:05:23.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.796+0000 7fcd7a000640 1 --2- 172.21.15.86:0/3739575492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd7c06d780 0x7fcd7c160690 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c06f290 tx=0x7fcd6801eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f97a12eb98d50a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:23.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.796+0000 7fcd79600640 1 -- 172.21.15.86:0/3739575492 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd680b4070 con 0x7fcd7c06d780 2024-04-25T14:05:23.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.796+0000 7fcd79600640 1 -- 172.21.15.86:0/3739575492 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd680a7920 con 0x7fcd7c06d780 2024-04-25T14:05:23.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.796+0000 7fcd79600640 1 -- 172.21.15.86:0/3739575492 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd680a7cf0 con 0x7fcd7c06d780 2024-04-25T14:05:23.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.796+0000 7fcd7be00640 1 -- 172.21.15.86:0/3739575492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd7c06d780 msgr2=0x7fcd7c160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:23.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.796+0000 7fcd7be00640 1 --2- 172.21.15.86:0/3739575492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd7c06d780 0x7fcd7c160690 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c06f290 tx=0x7fcd6801eb60 comp rx=0 tx=0).stop 2024-04-25T14:05:23.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.796+0000 7fcd7be00640 1 -- 172.21.15.86:0/3739575492 shutdown_connections 2024-04-25T14:05:23.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.796+0000 7fcd7be00640 1 --2- 172.21.15.86:0/3739575492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd7c06d780 0x7fcd7c160690 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:23.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.796+0000 7fcd7be00640 1 --2- 172.21.15.86:0/3739575492 >> v2:172.21.15.155:3300/0 conn(0x7fcd7c0d17c0 0x7fcd7c0d2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:23.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.796+0000 7fcd7be00640 1 -- 172.21.15.86:0/3739575492 >> 172.21.15.86:0/3739575492 conn(0x7fcd7c073c00 msgr2=0x7fcd7c0b77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:23.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.796+0000 7fcd7be00640 1 -- 172.21.15.86:0/3739575492 shutdown_connections 2024-04-25T14:05:23.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.796+0000 7fcd7be00640 1 -- 172.21.15.86:0/3739575492 wait complete. 2024-04-25T14:05:23.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.800+0000 7fcd7be00640 1 Processor -- start 2024-04-25T14:05:23.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.800+0000 7fcd7be00640 1 -- start start 2024-04-25T14:05:23.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.800+0000 7fcd7be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd7c06d780 0x7fcd7c150250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:23.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.800+0000 7fcd7be00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd7c06db90 0x7fcd7c14cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:23.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.800+0000 7fcd7be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd7c0d17c0 0x7fcd7c148310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:23.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.800+0000 7fcd7be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7c06ebf0 con 0x7fcd7c06d780 2024-04-25T14:05:23.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.800+0000 7fcd7be00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7c160fe0 con 0x7fcd7c0d17c0 2024-04-25T14:05:23.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.800+0000 7fcd7be00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcd7c06ed70 con 0x7fcd7c06db90 2024-04-25T14:05:23.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.800+0000 7fcd7aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd7c06d780 0x7fcd7c150250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:23.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.800+0000 7fcd7aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd7c06d780 0x7fcd7c150250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43442/0 (socket says 172.21.15.86:43442) 2024-04-25T14:05:23.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.800+0000 7fcd7aa00640 1 -- 172.21.15.86:0/892547349 learned_addr learned my addr 172.21.15.86:0/892547349 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:23.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.800+0000 7fcd7a000640 1 --2- 172.21.15.86:0/892547349 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd7c06db90 0x7fcd7c14cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:23.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.800+0000 7fcd7b400640 1 --2- 172.21.15.86:0/892547349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd7c0d17c0 0x7fcd7c148310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:23.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.800+0000 7fcd7aa00640 1 -- 172.21.15.86:0/892547349 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd7c06db90 msgr2=0x7fcd7c14cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:23.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.800+0000 7fcd7aa00640 1 --2- 172.21.15.86:0/892547349 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd7c06db90 0x7fcd7c14cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:23.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.800+0000 7fcd7aa00640 1 -- 172.21.15.86:0/892547349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd7c0d17c0 msgr2=0x7fcd7c148310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:23.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.800+0000 7fcd7aa00640 1 --2- 172.21.15.86:0/892547349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd7c0d17c0 0x7fcd7c148310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:23.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.800+0000 7fcd7aa00640 1 -- 172.21.15.86:0/892547349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7c14f200 con 0x7fcd7c06d780 2024-04-25T14:05:23.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.800+0000 7fcd7a000640 1 --2- 172.21.15.86:0/892547349 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd7c06db90 0x7fcd7c14cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:23.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.804+0000 7fcd7aa00640 1 --2- 172.21.15.86:0/892547349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd7c06d780 0x7fcd7c150250 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00e8a0 tx=0x7fcd6c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:23.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.804+0000 7fcd73400640 1 -- 172.21.15.86:0/892547349 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c017070 con 0x7fcd7c06d780 2024-04-25T14:05:23.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.804+0000 7fcd73400640 1 -- 172.21.15.86:0/892547349 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd6c00c940 con 0x7fcd7c06d780 2024-04-25T14:05:23.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.804+0000 7fcd7be00640 1 -- 172.21.15.86:0/892547349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7c14f670 con 0x7fcd7c06d780 2024-04-25T14:05:23.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.804+0000 7fcd73400640 1 -- 172.21.15.86:0/892547349 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c00ccf0 con 0x7fcd7c06d780 2024-04-25T14:05:23.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.804+0000 7fcd7be00640 1 -- 172.21.15.86:0/892547349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd7c153ec0 con 0x7fcd7c06d780 2024-04-25T14:05:23.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.804+0000 7fcd7be00640 1 -- 172.21.15.86:0/892547349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd7c1539d0 con 0x7fcd7c06d780 2024-04-25T14:05:23.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.804+0000 7fcd73400640 1 -- 172.21.15.86:0/892547349 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 10) ==== 93262+0+0 (secure 0 0 0) 0x7fcd6c01a070 con 0x7fcd7c06d780 2024-04-25T14:05:23.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.804+0000 7fcd73400640 1 --2- 172.21.15.86:0/892547349 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcd5c06f4e0 0x7fcd5c0719f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:23.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.804+0000 7fcd73400640 1 -- 172.21.15.86:0/892547349 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6359+0+0 (secure 0 0 0) 0x7fcd6c08f600 con 0x7fcd7c06d780 2024-04-25T14:05:23.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.808+0000 7fcd7a000640 1 --2- 172.21.15.86:0/892547349 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcd5c06f4e0 0x7fcd5c0719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:23.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.808+0000 7fcd7a000640 1 --2- 172.21.15.86:0/892547349 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcd5c06f4e0 0x7fcd5c0719f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcd680b1000 tx=0x7fcd680a9070 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:23.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.812+0000 7fcd73400640 1 -- 172.21.15.86:0/892547349 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcd6c05c170 con 0x7fcd7c06d780 2024-04-25T14:05:23.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.968+0000 7fcd7be00640 1 -- 172.21.15.86:0/892547349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcd7c076b80 con 0x7fcd7c06d780 2024-04-25T14:05:23.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.972+0000 7fcd73400640 1 -- 172.21.15.86:0/892547349 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+17319 (secure 0 0 0) 0x7fcd6c05ba70 con 0x7fcd7c06d780 2024-04-25T14:05:23.972 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:23.973 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":39,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:05:21.460905+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:05:04.472914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T14:05:05.604280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2073988773":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6833/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6832/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/3558294881":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/1843278167":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/2602032945":"2024-04-26T14:05:21.460855+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:05:23.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.972+0000 7fcd7be00640 1 -- 172.21.15.86:0/892547349 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcd5c06f4e0 msgr2=0x7fcd5c0719f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:23.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.972+0000 7fcd7be00640 1 --2- 172.21.15.86:0/892547349 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcd5c06f4e0 0x7fcd5c0719f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcd680b1000 tx=0x7fcd680a9070 comp rx=0 tx=0).stop 2024-04-25T14:05:23.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.972+0000 7fcd7be00640 1 -- 172.21.15.86:0/892547349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd7c06d780 msgr2=0x7fcd7c150250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:23.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.972+0000 7fcd7be00640 1 --2- 172.21.15.86:0/892547349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd7c06d780 0x7fcd7c150250 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00e8a0 tx=0x7fcd6c00eca0 comp rx=0 tx=0).stop 2024-04-25T14:05:23.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.972+0000 7fcd7be00640 1 -- 172.21.15.86:0/892547349 shutdown_connections 2024-04-25T14:05:23.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.972+0000 7fcd7be00640 1 --2- 172.21.15.86:0/892547349 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcd5c06f4e0 0x7fcd5c0719f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:23.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.972+0000 7fcd7be00640 1 --2- 172.21.15.86:0/892547349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd7c0d17c0 0x7fcd7c148310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:23.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.972+0000 7fcd7be00640 1 --2- 172.21.15.86:0/892547349 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd7c06db90 0x7fcd7c14cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:23.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.972+0000 7fcd7be00640 1 --2- 172.21.15.86:0/892547349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd7c06d780 0x7fcd7c150250 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:23.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.972+0000 7fcd7be00640 1 -- 172.21.15.86:0/892547349 >> 172.21.15.86:0/892547349 conn(0x7fcd7c073c00 msgr2=0x7fcd7c0ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:23.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.972+0000 7fcd7be00640 1 -- 172.21.15.86:0/892547349 shutdown_connections 2024-04-25T14:05:23.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:23.972+0000 7fcd7be00640 1 -- 172.21.15.86:0/892547349 wait complete. 2024-04-25T14:05:23.994 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T14:05:23.994 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:05:24.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.136+0000 7fceef400640 1 Processor -- start 2024-04-25T14:05:24.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.136+0000 7fceef400640 1 -- start start 2024-04-25T14:05:24.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.140+0000 7fceef400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcee80d16e0 0x7fcee80d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:24.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.140+0000 7fceef400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcee80d2030 con 0x7fcee80730b0 2024-04-25T14:05:24.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.140+0000 7fceef400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcee8072bd0 con 0x7fcee80cfca0 2024-04-25T14:05:24.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.140+0000 7fceef400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcee8070470 con 0x7fcee80d16e0 2024-04-25T14:05:24.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.140+0000 7fcedfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcee80d16e0 0x7fcee80d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:24.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.140+0000 7fcedfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcee80d16e0 0x7fcee80d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43448/0 (socket says 172.21.15.86:43448) 2024-04-25T14:05:24.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.140+0000 7fcedfe00640 1 -- 172.21.15.86:0/2804578741 learned_addr learned my addr 172.21.15.86:0/2804578741 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:24.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.140+0000 7fcedfe00640 1 -- 172.21.15.86:0/2804578741 >> v1:172.21.15.155:6789/0 conn(0x7fcee80cfca0 legacy=0x7fcee806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:24.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.140+0000 7fcedfe00640 1 -- 172.21.15.86:0/2804578741 >> v1:172.21.15.86:6789/0 conn(0x7fcee80730b0 legacy=0x7fcee80734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:24.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.140+0000 7fcedfe00640 1 -- 172.21.15.86:0/2804578741 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee806fbd0 con 0x7fcee80d16e0 2024-04-25T14:05:24.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.144+0000 7fcedfe00640 1 --2- 172.21.15.86:0/2804578741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcee80d16e0 0x7fcee80d1af0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fced0009870 tx=0x7fced001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1761b32740eb598e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:24.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.144+0000 7fcee7e00640 1 -- 172.21.15.86:0/2804578741 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced00a7070 con 0x7fcee80d16e0 2024-04-25T14:05:24.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.144+0000 7fcee7e00640 1 -- 172.21.15.86:0/2804578741 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fced00b0070 con 0x7fcee80d16e0 2024-04-25T14:05:24.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.144+0000 7fcee7e00640 1 -- 172.21.15.86:0/2804578741 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee80743e0 con 0x7fcee80d16e0 2024-04-25T14:05:24.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.144+0000 7fceef400640 1 -- 172.21.15.86:0/2804578741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcee80d16e0 msgr2=0x7fcee80d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.144+0000 7fceef400640 1 --2- 172.21.15.86:0/2804578741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcee80d16e0 0x7fcee80d1af0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fced0009870 tx=0x7fced001c560 comp rx=0 tx=0).stop 2024-04-25T14:05:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.144+0000 7fceef400640 1 -- 172.21.15.86:0/2804578741 shutdown_connections 2024-04-25T14:05:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.144+0000 7fceef400640 1 --2- 172.21.15.86:0/2804578741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcee80d16e0 0x7fcee80d1af0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.144+0000 7fceef400640 1 -- 172.21.15.86:0/2804578741 >> 172.21.15.86:0/2804578741 conn(0x7fcee80b4650 msgr2=0x7fcee80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.144+0000 7fceef400640 1 -- 172.21.15.86:0/2804578741 shutdown_connections 2024-04-25T14:05:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.144+0000 7fceef400640 1 -- 172.21.15.86:0/2804578741 wait complete. 2024-04-25T14:05:24.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.144+0000 7fceef400640 1 Processor -- start 2024-04-25T14:05:24.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.144+0000 7fceef400640 1 -- start start 2024-04-25T14:05:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.148+0000 7fceef400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcee80730b0 0x7fcee809aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.148+0000 7fceef400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcee80cfca0 0x7fcee8156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.148+0000 7fceef400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcee80d16e0 0x7fcee8154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.148+0000 7fceef400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcee816aa80 con 0x7fcee80730b0 2024-04-25T14:05:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.148+0000 7fceef400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcee816a900 con 0x7fcee80d16e0 2024-04-25T14:05:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.148+0000 7fceef400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcee816ac00 con 0x7fcee80cfca0 2024-04-25T14:05:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.148+0000 7fceeca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcee80730b0 0x7fcee809aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.148+0000 7fceeca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcee80730b0 0x7fcee809aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43460/0 (socket says 172.21.15.86:43460) 2024-04-25T14:05:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.148+0000 7fceeca00640 1 -- 172.21.15.86:0/1939150046 learned_addr learned my addr 172.21.15.86:0/1939150046 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.148+0000 7fcedfe00640 1 --2- 172.21.15.86:0/1939150046 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcee80cfca0 0x7fcee8156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.148+0000 7fceed400640 1 --2- 172.21.15.86:0/1939150046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcee80d16e0 0x7fcee8154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.148+0000 7fceeca00640 1 -- 172.21.15.86:0/1939150046 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcee80cfca0 msgr2=0x7fcee8156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.148+0000 7fceeca00640 1 --2- 172.21.15.86:0/1939150046 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcee80cfca0 0x7fcee8156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.148+0000 7fceeca00640 1 -- 172.21.15.86:0/1939150046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcee80d16e0 msgr2=0x7fcee8154f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:24.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.148+0000 7fceeca00640 1 --2- 172.21.15.86:0/1939150046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcee80d16e0 0x7fcee8154f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.148+0000 7fceeca00640 1 -- 172.21.15.86:0/1939150046 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee8156550 con 0x7fcee80730b0 2024-04-25T14:05:24.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.148+0000 7fceed400640 1 --2- 172.21.15.86:0/1939150046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcee80d16e0 0x7fcee8154f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:24.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.148+0000 7fceeca00640 1 --2- 172.21.15.86:0/1939150046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcee80730b0 0x7fcee809aa50 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fced800c3c0 tx=0x7fced800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:24.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.148+0000 7fcee5600640 1 -- 172.21.15.86:0/1939150046 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8017070 con 0x7fcee80730b0 2024-04-25T14:05:24.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.148+0000 7fcee5600640 1 -- 172.21.15.86:0/1939150046 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fced80052d0 con 0x7fcee80730b0 2024-04-25T14:05:24.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.148+0000 7fcee5600640 1 -- 172.21.15.86:0/1939150046 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8003820 con 0x7fcee80730b0 2024-04-25T14:05:24.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.148+0000 7fceef400640 1 -- 172.21.15.86:0/1939150046 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee8156ee0 con 0x7fcee80730b0 2024-04-25T14:05:24.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.148+0000 7fceef400640 1 -- 172.21.15.86:0/1939150046 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee81574a0 con 0x7fcee80730b0 2024-04-25T14:05:24.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.148+0000 7fceef400640 1 -- 172.21.15.86:0/1939150046 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcee808cfc0 con 0x7fcee80730b0 2024-04-25T14:05:24.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.152+0000 7fcee5600640 1 -- 172.21.15.86:0/1939150046 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 10) ==== 93262+0+0 (secure 0 0 0) 0x7fced801a070 con 0x7fcee80730b0 2024-04-25T14:05:24.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.152+0000 7fcee5600640 1 --2- 172.21.15.86:0/1939150046 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcec406f4e0 0x7fcec40719f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:24.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.152+0000 7fcee5600640 1 -- 172.21.15.86:0/1939150046 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6359+0+0 (secure 0 0 0) 0x7fced808e920 con 0x7fcee80730b0 2024-04-25T14:05:24.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.152+0000 7fcedfe00640 1 --2- 172.21.15.86:0/1939150046 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcec406f4e0 0x7fcec40719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.152+0000 7fcedfe00640 1 --2- 172.21.15.86:0/1939150046 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcec406f4e0 0x7fcec40719f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fced00af000 tx=0x7fced00a7000 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:24.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.156+0000 7fcee5600640 1 -- 172.21.15.86:0/1939150046 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fced8005480 con 0x7fcee80730b0 2024-04-25T14:05:24.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.344+0000 7fceef400640 1 -- 172.21.15.86:0/1939150046 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcee809f4b0 con 0x7fcee80730b0 2024-04-25T14:05:24.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.344+0000 7fcee5600640 1 -- 172.21.15.86:0/1939150046 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5184 (secure 0 0 0) 0x7fced805ae40 con 0x7fcee80730b0 2024-04-25T14:05:24.348 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:24.348 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3546153119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3546153119},{"type":"v1","addr":"172.21.15.155:6834","nonce":3546153119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5283,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3499340224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3499340224},{"type":"v1","addr":"172.21.15.86:6836","nonce":3499340224}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5284,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1466978742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1466978742},{"type":"v1","addr":"172.21.15.155:6835","nonce":1466978742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:06.638216+0000","modified":"2024-04-25T14:05:09.664776+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5286":{"gid":5286,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/716923566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":716923566},{"type":"v1","addr":"172.21.15.86:6837","nonce":716923566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5286,"qdb_cluster":[5286]},"id":2}]} 2024-04-25T14:05:24.348 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 18 2024-04-25T14:05:24.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.348+0000 7fceef400640 1 -- 172.21.15.86:0/1939150046 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcec406f4e0 msgr2=0x7fcec40719f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:24.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.348+0000 7fceef400640 1 --2- 172.21.15.86:0/1939150046 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcec406f4e0 0x7fcec40719f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fced00af000 tx=0x7fced00a7000 comp rx=0 tx=0).stop 2024-04-25T14:05:24.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.348+0000 7fceef400640 1 -- 172.21.15.86:0/1939150046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcee80730b0 msgr2=0x7fcee809aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:24.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.348+0000 7fceef400640 1 --2- 172.21.15.86:0/1939150046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcee80730b0 0x7fcee809aa50 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fced800c3c0 tx=0x7fced800c880 comp rx=0 tx=0).stop 2024-04-25T14:05:24.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.348+0000 7fceef400640 1 -- 172.21.15.86:0/1939150046 shutdown_connections 2024-04-25T14:05:24.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.348+0000 7fceef400640 1 --2- 172.21.15.86:0/1939150046 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcec406f4e0 0x7fcec40719f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.348+0000 7fceef400640 1 --2- 172.21.15.86:0/1939150046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcee80d16e0 0x7fcee8154f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.348+0000 7fceef400640 1 --2- 172.21.15.86:0/1939150046 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcee80cfca0 0x7fcee8156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.348+0000 7fceef400640 1 --2- 172.21.15.86:0/1939150046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcee80730b0 0x7fcee809aa50 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.348+0000 7fceef400640 1 -- 172.21.15.86:0/1939150046 >> 172.21.15.86:0/1939150046 conn(0x7fcee80b4650 msgr2=0x7fcee80b2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:24.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.348+0000 7fceef400640 1 -- 172.21.15.86:0/1939150046 shutdown_connections 2024-04-25T14:05:24.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.348+0000 7fceef400640 1 -- 172.21.15.86:0/1939150046 wait complete. 2024-04-25T14:05:24.368 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:05:24.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.512+0000 7f91dba00640 1 Processor -- start 2024-04-25T14:05:24.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.512+0000 7f91dba00640 1 -- start start 2024-04-25T14:05:24.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.516+0000 7f91dba00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f91d415faa0 0x7f91d406d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:24.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.516+0000 7f91dba00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91d40aeea0 0x7f91d40af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:24.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.516+0000 7f91dba00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91d406c670 con 0x7f91d40b59e0 2024-04-25T14:05:24.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.516+0000 7f91dba00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f91d406d560 con 0x7f91d415faa0 2024-04-25T14:05:24.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.516+0000 7f91dba00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f91d406c950 con 0x7f91d40aeea0 2024-04-25T14:05:24.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.516+0000 7f91cbe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f91d40b59e0 0x7f91d40b5df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47378/0 (socket says 172.21.15.86:47378) 2024-04-25T14:05:24.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.516+0000 7f91cbe00640 1 -- 172.21.15.86:0/2706430282 learned_addr learned my addr 172.21.15.86:0/2706430282 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:24.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.516+0000 7f91d9000640 1 --2- 172.21.15.86:0/2706430282 >> v2:172.21.15.155:3300/0 conn(0x7f91d415faa0 0x7f91d406d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:24.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.516+0000 7f91d9a00640 1 --2- 172.21.15.86:0/2706430282 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91d40aeea0 0x7f91d40af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:24.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.516+0000 7f91d3e00640 1 -- 172.21.15.86:0/2706430282 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2507944991 0 0) 0x7f91d406c670 con 0x7f91d40b59e0 2024-04-25T14:05:24.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.516+0000 7f91d3e00640 1 -- 172.21.15.86:0/2706430282 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91b8003530 con 0x7f91d40b59e0 2024-04-25T14:05:24.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.516+0000 7f91d9000640 1 -- 172.21.15.86:0/2706430282 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91d40aeea0 msgr2=0x7f91d40af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:24.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.516+0000 7f91d9000640 1 --2- 172.21.15.86:0/2706430282 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91d40aeea0 0x7f91d40af2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.516+0000 7f91d9000640 1 -- 172.21.15.86:0/2706430282 >> v1:172.21.15.86:6789/0 conn(0x7f91d40b59e0 legacy=0x7f91d40b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:24.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.516+0000 7f91d9000640 1 -- 172.21.15.86:0/2706430282 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d406cc30 con 0x7f91d415faa0 2024-04-25T14:05:24.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.516+0000 7f91d9a00640 1 --2- 172.21.15.86:0/2706430282 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91d40aeea0 0x7f91d40af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:24.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.520+0000 7f91d9000640 1 --2- 172.21.15.86:0/2706430282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91d415faa0 0x7f91d406d9f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f91c40097c0 tx=0x7f91c401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2975713794e2ce5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:24.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.520+0000 7f91d3e00640 1 -- 172.21.15.86:0/2706430282 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c40aa600 con 0x7f91d415faa0 2024-04-25T14:05:24.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.520+0000 7f91d3e00640 1 -- 172.21.15.86:0/2706430282 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91c40af070 con 0x7f91d415faa0 2024-04-25T14:05:24.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.520+0000 7f91d3e00640 1 -- 172.21.15.86:0/2706430282 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c40aa9c0 con 0x7f91d415faa0 2024-04-25T14:05:24.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.520+0000 7f91dba00640 1 -- 172.21.15.86:0/2706430282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91d415faa0 msgr2=0x7f91d406d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:24.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.520+0000 7f91dba00640 1 --2- 172.21.15.86:0/2706430282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91d415faa0 0x7f91d406d9f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f91c40097c0 tx=0x7f91c401c4a0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.520+0000 7f91dba00640 1 -- 172.21.15.86:0/2706430282 shutdown_connections 2024-04-25T14:05:24.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.520+0000 7f91dba00640 1 --2- 172.21.15.86:0/2706430282 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91d40aeea0 0x7f91d40af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.520+0000 7f91dba00640 1 --2- 172.21.15.86:0/2706430282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91d415faa0 0x7f91d406d9f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.520+0000 7f91dba00640 1 -- 172.21.15.86:0/2706430282 >> 172.21.15.86:0/2706430282 conn(0x7f91d40d2230 msgr2=0x7f91d40d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:24.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.520+0000 7f91dba00640 1 -- 172.21.15.86:0/2706430282 shutdown_connections 2024-04-25T14:05:24.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.520+0000 7f91dba00640 1 -- 172.21.15.86:0/2706430282 wait complete. 2024-04-25T14:05:24.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.524+0000 7f91dba00640 1 Processor -- start 2024-04-25T14:05:24.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.524+0000 7f91dba00640 1 -- start start 2024-04-25T14:05:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.524+0000 7f91dba00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91d40aeea0 0x7f91d4159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.524+0000 7f91dba00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91d40b59e0 0x7f91d41520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.524+0000 7f91dba00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91d415faa0 0x7f91d4068590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.524+0000 7f91dba00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f91d4160030 con 0x7f91d40aeea0 2024-04-25T14:05:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.524+0000 7f91dba00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f91d415feb0 con 0x7f91d40b59e0 2024-04-25T14:05:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.524+0000 7f91dba00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f91d4070f30 con 0x7f91d415faa0 2024-04-25T14:05:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.524+0000 7f91d9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91d40aeea0 0x7f91d4159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.524+0000 7f91d9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91d40aeea0 0x7f91d4159860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43474/0 (socket says 172.21.15.86:43474) 2024-04-25T14:05:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.524+0000 7f91d9000640 1 -- 172.21.15.86:0/2018135089 learned_addr learned my addr 172.21.15.86:0/2018135089 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.524+0000 7f91cbe00640 1 --2- 172.21.15.86:0/2018135089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91d40b59e0 0x7f91d41520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.524+0000 7f91d9a00640 1 --2- 172.21.15.86:0/2018135089 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91d415faa0 0x7f91d4068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.524+0000 7f91d9000640 1 -- 172.21.15.86:0/2018135089 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91d415faa0 msgr2=0x7f91d4068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.524+0000 7f91d9000640 1 --2- 172.21.15.86:0/2018135089 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91d415faa0 0x7f91d4068590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.524+0000 7f91d9000640 1 -- 172.21.15.86:0/2018135089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91d40b59e0 msgr2=0x7f91d41520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:24.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.524+0000 7f91d9000640 1 --2- 172.21.15.86:0/2018135089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91d40b59e0 0x7f91d41520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.524+0000 7f91d9000640 1 -- 172.21.15.86:0/2018135089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d415a2e0 con 0x7f91d40aeea0 2024-04-25T14:05:24.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.524+0000 7f91d9000640 1 --2- 172.21.15.86:0/2018135089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91d40aeea0 0x7f91d4159860 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f91c401a0d0 tx=0x7f91c401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:24.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.524+0000 7f91d1600640 1 -- 172.21.15.86:0/2018135089 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c40be410 con 0x7f91d40aeea0 2024-04-25T14:05:24.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.524+0000 7f91d1600640 1 -- 172.21.15.86:0/2018135089 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91c40be5b0 con 0x7f91d40aeea0 2024-04-25T14:05:24.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.524+0000 7f91dba00640 1 -- 172.21.15.86:0/2018135089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91d415a5c0 con 0x7f91d40aeea0 2024-04-25T14:05:24.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.524+0000 7f91d1600640 1 -- 172.21.15.86:0/2018135089 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c40be9a0 con 0x7f91d40aeea0 2024-04-25T14:05:24.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.524+0000 7f91dba00640 1 -- 172.21.15.86:0/2018135089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91d415ab80 con 0x7f91d40aeea0 2024-04-25T14:05:24.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.528+0000 7f91dba00640 1 -- 172.21.15.86:0/2018135089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91d4074cf0 con 0x7f91d40aeea0 2024-04-25T14:05:24.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.528+0000 7f91d1600640 1 -- 172.21.15.86:0/2018135089 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 10) ==== 93262+0+0 (secure 0 0 0) 0x7f91c40bc020 con 0x7f91d40aeea0 2024-04-25T14:05:24.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.528+0000 7f91d1600640 1 --2- 172.21.15.86:0/2018135089 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f91b006f510 0x7f91b0071a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:24.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.528+0000 7f91d1600640 1 -- 172.21.15.86:0/2018135089 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6359+0+0 (secure 0 0 0) 0x7f91c4129b80 con 0x7f91d40aeea0 2024-04-25T14:05:24.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.528+0000 7f91cbe00640 1 --2- 172.21.15.86:0/2018135089 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f91b006f510 0x7f91b0071a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:24.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.528+0000 7f91cbe00640 1 --2- 172.21.15.86:0/2018135089 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f91b006f510 0x7f91b0071a20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f91d4164840 tx=0x7f91bc009040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:24.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.532+0000 7f91d1600640 1 -- 172.21.15.86:0/2018135089 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f91c4016090 con 0x7f91d40aeea0 2024-04-25T14:05:24.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.720+0000 7f91dba00640 1 -- 172.21.15.86:0/2018135089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91d40b55b0 con 0x7f91d40aeea0 2024-04-25T14:05:24.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.720+0000 7f91d1600640 1 -- 172.21.15.86:0/2018135089 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5184 (secure 0 0 0) 0x7f91c40f65e0 con 0x7f91d40aeea0 2024-04-25T14:05:24.724 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:24.725 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3546153119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3546153119},{"type":"v1","addr":"172.21.15.155:6834","nonce":3546153119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5283,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3499340224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3499340224},{"type":"v1","addr":"172.21.15.86:6836","nonce":3499340224}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5284,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1466978742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1466978742},{"type":"v1","addr":"172.21.15.155:6835","nonce":1466978742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:06.638216+0000","modified":"2024-04-25T14:05:09.664776+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5286},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5286":{"gid":5286,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/716923566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":716923566},{"type":"v1","addr":"172.21.15.86:6837","nonce":716923566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5286,"qdb_cluster":[5286]},"id":2}]} 2024-04-25T14:05:24.725 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 18 2024-04-25T14:05:24.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.724+0000 7f91dba00640 1 -- 172.21.15.86:0/2018135089 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f91b006f510 msgr2=0x7f91b0071a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:24.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.724+0000 7f91dba00640 1 --2- 172.21.15.86:0/2018135089 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f91b006f510 0x7f91b0071a20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f91d4164840 tx=0x7f91bc009040 comp rx=0 tx=0).stop 2024-04-25T14:05:24.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.724+0000 7f91dba00640 1 -- 172.21.15.86:0/2018135089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91d40aeea0 msgr2=0x7f91d4159860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:24.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.724+0000 7f91dba00640 1 --2- 172.21.15.86:0/2018135089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91d40aeea0 0x7f91d4159860 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f91c401a0d0 tx=0x7f91c401ace0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.724+0000 7f91dba00640 1 -- 172.21.15.86:0/2018135089 shutdown_connections 2024-04-25T14:05:24.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.724+0000 7f91dba00640 1 --2- 172.21.15.86:0/2018135089 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f91b006f510 0x7f91b0071a20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.724+0000 7f91dba00640 1 --2- 172.21.15.86:0/2018135089 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91d415faa0 0x7f91d4068590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.724+0000 7f91dba00640 1 --2- 172.21.15.86:0/2018135089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91d40b59e0 0x7f91d41520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.724+0000 7f91dba00640 1 --2- 172.21.15.86:0/2018135089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91d40aeea0 0x7f91d4159860 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.724+0000 7f91dba00640 1 -- 172.21.15.86:0/2018135089 >> 172.21.15.86:0/2018135089 conn(0x7f91d40d2230 msgr2=0x7f91d40b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:24.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.724+0000 7f91dba00640 1 -- 172.21.15.86:0/2018135089 shutdown_connections 2024-04-25T14:05:24.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.724+0000 7f91dba00640 1 -- 172.21.15.86:0/2018135089 wait complete. 2024-04-25T14:05:24.745 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:05:24.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.888+0000 7f926be00640 1 Processor -- start 2024-04-25T14:05:24.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.888+0000 7f926be00640 1 -- start start 2024-04-25T14:05:24.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f926be00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f926415faa0 0x7f926406d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:24.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f926be00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92640aeea0 0x7f92640af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:24.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f926be00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f926406c670 con 0x7f92640b59e0 2024-04-25T14:05:24.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f926be00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f926406d560 con 0x7f926415faa0 2024-04-25T14:05:24.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f926be00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f926406c950 con 0x7f92640aeea0 2024-04-25T14:05:24.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9268a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f92640b59e0 0x7f92640b5df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47384/0 (socket says 172.21.15.86:47384) 2024-04-25T14:05:24.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9268a00640 1 -- 172.21.15.86:0/1416214174 learned_addr learned my addr 172.21.15.86:0/1416214174 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:24.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9269400640 1 --2- 172.21.15.86:0/1416214174 >> v2:172.21.15.155:3300/0 conn(0x7f926415faa0 0x7f926406d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:24.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9269e00640 1 --2- 172.21.15.86:0/1416214174 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92640aeea0 0x7f92640af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:24.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9263e00640 1 -- 172.21.15.86:0/1416214174 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2657417439 0 0) 0x7f926406c670 con 0x7f92640b59e0 2024-04-25T14:05:24.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9263e00640 1 -- 172.21.15.86:0/1416214174 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f924c003530 con 0x7f92640b59e0 2024-04-25T14:05:24.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9263e00640 1 -- 172.21.15.86:0/1416214174 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 307090694 0 0) 0x7f924c003530 con 0x7f92640b59e0 2024-04-25T14:05:24.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9263e00640 1 -- 172.21.15.86:0/1416214174 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92640aeea0 msgr2=0x7f92640af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:24.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9263e00640 1 --2- 172.21.15.86:0/1416214174 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92640aeea0 0x7f92640af2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9263e00640 1 -- 172.21.15.86:0/1416214174 >> v2:172.21.15.155:3300/0 conn(0x7f926415faa0 msgr2=0x7f926406d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:24.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9263e00640 1 --2- 172.21.15.86:0/1416214174 >> v2:172.21.15.155:3300/0 conn(0x7f926415faa0 0x7f926406d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9263e00640 1 -- 172.21.15.86:0/1416214174 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f926406cc30 con 0x7f92640b59e0 2024-04-25T14:05:24.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9263e00640 1 -- 172.21.15.86:0/1416214174 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f9250001680 con 0x7f92640b59e0 2024-04-25T14:05:24.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9263e00640 1 -- 172.21.15.86:0/1416214174 >> v1:172.21.15.86:6789/0 conn(0x7f92640b59e0 legacy=0x7f92640b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:24.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9263e00640 1 --2- 172.21.15.86:0/1416214174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f924c003a30 0x7f924c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:24.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9263e00640 1 --2- 172.21.15.86:0/1416214174 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92640aeea0 0x7f924c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:24.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9263e00640 1 --2- 172.21.15.86:0/1416214174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f924c004d50 0x7f924c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:24.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9263e00640 1 -- 172.21.15.86:0/1416214174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f926406d560 con 0x7f924c003a30 2024-04-25T14:05:24.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9263e00640 1 -- 172.21.15.86:0/1416214174 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f926406c950 con 0x7f924c004d50 2024-04-25T14:05:24.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9263e00640 1 -- 172.21.15.86:0/1416214174 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f924c00ba80 con 0x7f92640aeea0 2024-04-25T14:05:24.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9269400640 1 --2- 172.21.15.86:0/1416214174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f924c003a30 0x7f924c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:24.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9269e00640 1 --2- 172.21.15.86:0/1416214174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f924c004d50 0x7f924c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:24.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9268a00640 1 --2- 172.21.15.86:0/1416214174 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92640aeea0 0x7f924c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:24.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9269400640 1 -- 172.21.15.86:0/1416214174 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92640aeea0 msgr2=0x7f924c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:24.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9269400640 1 --2- 172.21.15.86:0/1416214174 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92640aeea0 0x7f924c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9269400640 1 -- 172.21.15.86:0/1416214174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f924c004d50 msgr2=0x7f924c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:24.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9269400640 1 --2- 172.21.15.86:0/1416214174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f924c004d50 0x7f924c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9269400640 1 -- 172.21.15.86:0/1416214174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924c003530 con 0x7f924c003a30 2024-04-25T14:05:24.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.892+0000 7f9269e00640 1 --2- 172.21.15.86:0/1416214174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f924c004d50 0x7f924c005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:24.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.896+0000 7f9269400640 1 --2- 172.21.15.86:0/1416214174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f924c003a30 0x7f924c003ea0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9258009ae0 tx=0x7f925801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d1baa36b250a9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:24.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.896+0000 7f9263e00640 1 -- 172.21.15.86:0/1416214174 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92580b2070 con 0x7f924c003a30 2024-04-25T14:05:24.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.896+0000 7f9263e00640 1 -- 172.21.15.86:0/1416214174 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92580a5680 con 0x7f924c003a30 2024-04-25T14:05:24.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.896+0000 7f9263e00640 1 -- 172.21.15.86:0/1416214174 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92580096b0 con 0x7f924c003a30 2024-04-25T14:05:24.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.896+0000 7f926be00640 1 -- 172.21.15.86:0/1416214174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f924c003a30 msgr2=0x7f924c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:24.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.896+0000 7f926be00640 1 --2- 172.21.15.86:0/1416214174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f924c003a30 0x7f924c003ea0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9258009ae0 tx=0x7f925801cab0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.896+0000 7f9269e00640 1 -- 172.21.15.86:0/1416214174 reap_dead start 2024-04-25T14:05:24.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.896+0000 7f926be00640 1 -- 172.21.15.86:0/1416214174 shutdown_connections 2024-04-25T14:05:24.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.896+0000 7f926be00640 1 -- 172.21.15.86:0/1416214174 >> 172.21.15.86:0/1416214174 conn(0x7f92640d2230 msgr2=0x7f92640d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:24.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.896+0000 7f926be00640 1 -- 172.21.15.86:0/1416214174 shutdown_connections 2024-04-25T14:05:24.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.896+0000 7f926be00640 1 -- 172.21.15.86:0/1416214174 wait complete. 2024-04-25T14:05:24.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.900+0000 7f926be00640 1 Processor -- start 2024-04-25T14:05:24.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.900+0000 7f926be00640 1 -- start start 2024-04-25T14:05:24.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.900+0000 7f926be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f924c003a30 0x7f92641fd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:24.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.900+0000 7f926be00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9264149350 0x7f9264149760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:24.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.900+0000 7f926be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9264159690 0x7f9264159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:24.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.900+0000 7f926be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9264160030 con 0x7f9264159690 2024-04-25T14:05:24.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.900+0000 7f926be00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f926415feb0 con 0x7f924c003a30 2024-04-25T14:05:24.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.900+0000 7f926be00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9264070f30 con 0x7f9264149350 2024-04-25T14:05:24.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.900+0000 7f9269e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9264159690 0x7f9264159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:24.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.900+0000 7f9269e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9264159690 0x7f9264159aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43500/0 (socket says 172.21.15.86:43500) 2024-04-25T14:05:24.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.900+0000 7f9269e00640 1 -- 172.21.15.86:0/1950694729 learned_addr learned my addr 172.21.15.86:0/1950694729 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:24.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.900+0000 7f9269400640 1 --2- 172.21.15.86:0/1950694729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f924c003a30 0x7f92641fd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:24.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.900+0000 7f9268a00640 1 --2- 172.21.15.86:0/1950694729 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9264149350 0x7f9264149760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:24.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.900+0000 7f9269e00640 1 -- 172.21.15.86:0/1950694729 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9264149350 msgr2=0x7f9264149760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:24.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.900+0000 7f9269e00640 1 --2- 172.21.15.86:0/1950694729 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9264149350 0x7f9264149760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.900+0000 7f9269e00640 1 -- 172.21.15.86:0/1950694729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f924c003a30 msgr2=0x7f92641fd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:24.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.900+0000 7f9269e00640 1 --2- 172.21.15.86:0/1950694729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f924c003a30 0x7f92641fd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:24.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.900+0000 7f9269e00640 1 -- 172.21.15.86:0/1950694729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92641fdb90 con 0x7f9264159690 2024-04-25T14:05:24.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.904+0000 7f9269e00640 1 --2- 172.21.15.86:0/1950694729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9264159690 0x7f9264159aa0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9264072cf0 tx=0x7f925c010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:24.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.904+0000 7f9261600640 1 -- 172.21.15.86:0/1950694729 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c010d80 con 0x7f9264159690 2024-04-25T14:05:24.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.904+0000 7f9261600640 1 -- 172.21.15.86:0/1950694729 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f925c0039a0 con 0x7f9264159690 2024-04-25T14:05:24.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.904+0000 7f9261600640 1 -- 172.21.15.86:0/1950694729 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c003dd0 con 0x7f9264159690 2024-04-25T14:05:24.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.904+0000 7f926be00640 1 -- 172.21.15.86:0/1950694729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9264143e40 con 0x7f9264159690 2024-04-25T14:05:24.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.904+0000 7f926be00640 1 -- 172.21.15.86:0/1950694729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9264164e10 con 0x7f9264159690 2024-04-25T14:05:24.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.904+0000 7f926be00640 1 -- 172.21.15.86:0/1950694729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9264164b80 con 0x7f9264159690 2024-04-25T14:05:24.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.904+0000 7f9261600640 1 -- 172.21.15.86:0/1950694729 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 10) ==== 93262+0+0 (secure 0 0 0) 0x7f925c01e030 con 0x7f9264159690 2024-04-25T14:05:24.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.908+0000 7f9261600640 1 --2- 172.21.15.86:0/1950694729 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f924406f4e0 0x7f92440719f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:24.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.908+0000 7f9261600640 1 -- 172.21.15.86:0/1950694729 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6359+0+0 (secure 0 0 0) 0x7f925c08fde0 con 0x7f9264159690 2024-04-25T14:05:24.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.908+0000 7f9269400640 1 --2- 172.21.15.86:0/1950694729 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f924406f4e0 0x7f92440719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:24.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.908+0000 7f9269400640 1 --2- 172.21.15.86:0/1950694729 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f924406f4e0 0x7f92440719f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9258009ae0 tx=0x7f92580046d0 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:24.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:24.912+0000 7f9261600640 1 -- 172.21.15.86:0/1950694729 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f925c025020 con 0x7f9264159690 2024-04-25T14:05:25.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.068+0000 7f926be00640 1 -- 172.21.15.86:0/1950694729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9264074cf0 con 0x7f9264159690 2024-04-25T14:05:25.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.072+0000 7f9261600640 1 -- 172.21.15.86:0/1950694729 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+17319 (secure 0 0 0) 0x7f92641fdb90 con 0x7f9264159690 2024-04-25T14:05:25.073 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:25.073 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":39,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:05:21.460905+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:05:04.472914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T14:05:05.604280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2073988773":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6833/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6832/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/3558294881":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/1843278167":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/2602032945":"2024-04-26T14:05:21.460855+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:05:25.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.072+0000 7f926be00640 1 -- 172.21.15.86:0/1950694729 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f924406f4e0 msgr2=0x7f92440719f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:25.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.072+0000 7f926be00640 1 --2- 172.21.15.86:0/1950694729 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f924406f4e0 0x7f92440719f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9258009ae0 tx=0x7f92580046d0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.072+0000 7f926be00640 1 -- 172.21.15.86:0/1950694729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9264159690 msgr2=0x7f9264159aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:25.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.072+0000 7f926be00640 1 --2- 172.21.15.86:0/1950694729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9264159690 0x7f9264159aa0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9264072cf0 tx=0x7f925c010590 comp rx=0 tx=0).stop 2024-04-25T14:05:25.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.072+0000 7f926be00640 1 -- 172.21.15.86:0/1950694729 shutdown_connections 2024-04-25T14:05:25.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.072+0000 7f926be00640 1 --2- 172.21.15.86:0/1950694729 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f924406f4e0 0x7f92440719f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.072+0000 7f926be00640 1 --2- 172.21.15.86:0/1950694729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9264159690 0x7f9264159aa0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.072+0000 7f926be00640 1 --2- 172.21.15.86:0/1950694729 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9264149350 0x7f9264149760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.072+0000 7f926be00640 1 --2- 172.21.15.86:0/1950694729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f924c003a30 0x7f92641fd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.072+0000 7f926be00640 1 -- 172.21.15.86:0/1950694729 >> 172.21.15.86:0/1950694729 conn(0x7f92640d2230 msgr2=0x7f9264157030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:25.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.072+0000 7f926be00640 1 -- 172.21.15.86:0/1950694729 shutdown_connections 2024-04-25T14:05:25.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.072+0000 7f926be00640 1 -- 172.21.15.86:0/1950694729 wait complete. 2024-04-25T14:05:25.095 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T14:05:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.240+0000 7f07c6c00640 1 Processor -- start 2024-04-25T14:05:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.240+0000 7f07c6c00640 1 -- start start 2024-04-25T14:05:25.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.240+0000 7f07c6c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f07c00d1df0 0x7f07c00cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.240+0000 7f07c6c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07c00cdcc0 0x7f07c00ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.240+0000 7f07c6c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07c015e5c0 con 0x7f07c0070330 2024-04-25T14:05:25.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.240+0000 7f07c6c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f07c006fd40 con 0x7f07c00d1df0 2024-04-25T14:05:25.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.240+0000 7f07c6c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f07c015e8a0 con 0x7f07c00cdcc0 2024-04-25T14:05:25.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.240+0000 7f07bf400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f07c00d1df0 0x7f07c00cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.240+0000 7f07bf400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f07c00d1df0 0x7f07c00cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43504/0 (socket says 172.21.15.86:43504) 2024-04-25T14:05:25.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.240+0000 7f07bf400640 1 -- 172.21.15.86:0/3579884132 learned_addr learned my addr 172.21.15.86:0/3579884132 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:25.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.240+0000 7f07c4c00640 1 --2- 172.21.15.86:0/3579884132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07c00cdcc0 0x7f07c00ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.240+0000 7f07bf400640 1 -- 172.21.15.86:0/3579884132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07c00cdcc0 msgr2=0x7f07c00ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:25.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.240+0000 7f07bf400640 1 --2- 172.21.15.86:0/3579884132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07c00cdcc0 0x7f07c00ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.240+0000 7f07bf400640 1 -- 172.21.15.86:0/3579884132 >> v1:172.21.15.155:6789/0 conn(0x7f07c0070330 legacy=0x7f07c00d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:25.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.240+0000 7f07bf400640 1 -- 172.21.15.86:0/3579884132 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c015eb80 con 0x7f07c00d1df0 2024-04-25T14:05:25.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.240+0000 7f07c4c00640 1 --2- 172.21.15.86:0/3579884132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07c00cdcc0 0x7f07c00ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:25.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.244+0000 7f07bf400640 1 --2- 172.21.15.86:0/3579884132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c00d1df0 0x7f07c00cd780 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009870 tx=0x7f07ac01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d72b1b1f4173c60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:25.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.244+0000 7f07bea00640 1 -- 172.21.15.86:0/3579884132 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac0ab600 con 0x7f07c00d1df0 2024-04-25T14:05:25.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.244+0000 7f07bea00640 1 -- 172.21.15.86:0/3579884132 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07ac0b0070 con 0x7f07c00d1df0 2024-04-25T14:05:25.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.244+0000 7f07bea00640 1 -- 172.21.15.86:0/3579884132 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac0ab9c0 con 0x7f07c00d1df0 2024-04-25T14:05:25.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.244+0000 7f07c6c00640 1 -- 172.21.15.86:0/3579884132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c00d1df0 msgr2=0x7f07c00cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.244+0000 7f07c6c00640 1 --2- 172.21.15.86:0/3579884132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c00d1df0 0x7f07c00cd780 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009870 tx=0x7f07ac01c460 comp rx=0 tx=0).stop 2024-04-25T14:05:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.244+0000 7f07c6c00640 1 -- 172.21.15.86:0/3579884132 shutdown_connections 2024-04-25T14:05:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.244+0000 7f07c6c00640 1 --2- 172.21.15.86:0/3579884132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07c00cdcc0 0x7f07c00ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.244+0000 7f07c6c00640 1 --2- 172.21.15.86:0/3579884132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c00d1df0 0x7f07c00cd780 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.244+0000 7f07c6c00640 1 -- 172.21.15.86:0/3579884132 >> 172.21.15.86:0/3579884132 conn(0x7f07c00b4650 msgr2=0x7f07c00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.244+0000 7f07c6c00640 1 -- 172.21.15.86:0/3579884132 shutdown_connections 2024-04-25T14:05:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.244+0000 7f07c6c00640 1 -- 172.21.15.86:0/3579884132 wait complete. 2024-04-25T14:05:25.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.248+0000 7f07c6c00640 1 Processor -- start 2024-04-25T14:05:25.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.248+0000 7f07c6c00640 1 -- start start 2024-04-25T14:05:25.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.248+0000 7f07c6c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c0070330 0x7f07c00c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.248+0000 7f07c6c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07c00cdcc0 0x7f07c0159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.248+0000 7f07c6c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07c00c27e0 0x7f07c00c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.248+0000 7f07c6c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07c015f4b0 con 0x7f07c0070330 2024-04-25T14:05:25.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.248+0000 7f07c6c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f07c015f330 con 0x7f07c00cdcc0 2024-04-25T14:05:25.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.248+0000 7f07c6c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f07c0070740 con 0x7f07c00c27e0 2024-04-25T14:05:25.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.248+0000 7f07bfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c0070330 0x7f07c00c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.248+0000 7f07bfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c0070330 0x7f07c00c85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43516/0 (socket says 172.21.15.86:43516) 2024-04-25T14:05:25.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.248+0000 7f07bfe00640 1 -- 172.21.15.86:0/1913540453 learned_addr learned my addr 172.21.15.86:0/1913540453 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:25.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.248+0000 7f07bf400640 1 --2- 172.21.15.86:0/1913540453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07c00cdcc0 0x7f07c0159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.248+0000 7f07c4c00640 1 --2- 172.21.15.86:0/1913540453 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07c00c27e0 0x7f07c00c2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.248+0000 7f07bfe00640 1 -- 172.21.15.86:0/1913540453 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07c00c27e0 msgr2=0x7f07c00c2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:25.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.248+0000 7f07bfe00640 1 --2- 172.21.15.86:0/1913540453 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07c00c27e0 0x7f07c00c2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.248+0000 7f07bfe00640 1 -- 172.21.15.86:0/1913540453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07c00cdcc0 msgr2=0x7f07c0159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:25.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.248+0000 7f07bfe00640 1 --2- 172.21.15.86:0/1913540453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07c00cdcc0 0x7f07c0159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.248+0000 7f07bfe00640 1 -- 172.21.15.86:0/1913540453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c015b640 con 0x7f07c0070330 2024-04-25T14:05:25.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.252+0000 7f07bfe00640 1 --2- 172.21.15.86:0/1913540453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c0070330 0x7f07c00c85d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f07b000b3e0 tx=0x7f07b000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:25.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.252+0000 7f07b7e00640 1 -- 172.21.15.86:0/1913540453 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b0015020 con 0x7f07c0070330 2024-04-25T14:05:25.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.252+0000 7f07b7e00640 1 -- 172.21.15.86:0/1913540453 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07b0011830 con 0x7f07c0070330 2024-04-25T14:05:25.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.252+0000 7f07c6c00640 1 -- 172.21.15.86:0/1913540453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c00c7b10 con 0x7f07c0070330 2024-04-25T14:05:25.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.252+0000 7f07b7e00640 1 -- 172.21.15.86:0/1913540453 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b0011c20 con 0x7f07c0070330 2024-04-25T14:05:25.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.252+0000 7f07c6c00640 1 -- 172.21.15.86:0/1913540453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c00c7dd0 con 0x7f07c0070330 2024-04-25T14:05:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.252+0000 7f07c6c00640 1 -- 172.21.15.86:0/1913540453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07c0069d00 con 0x7f07c0070330 2024-04-25T14:05:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.252+0000 7f07b7e00640 1 -- 172.21.15.86:0/1913540453 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 10) ==== 93262+0+0 (secure 0 0 0) 0x7f07b0018020 con 0x7f07c0070330 2024-04-25T14:05:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.252+0000 7f07b7e00640 1 --2- 172.21.15.86:0/1913540453 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f07a006f4e0 0x7f07a00719f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.252+0000 7f07b7e00640 1 -- 172.21.15.86:0/1913540453 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6359+0+0 (secure 0 0 0) 0x7f07b008f4d0 con 0x7f07c0070330 2024-04-25T14:05:25.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.252+0000 7f07bf400640 1 --2- 172.21.15.86:0/1913540453 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f07a006f4e0 0x7f07a00719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.256+0000 7f07bf400640 1 --2- 172.21.15.86:0/1913540453 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f07a006f4e0 0x7f07a00719f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0af000 tx=0x7f07ac0a7000 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:25.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.256+0000 7f07b7e00640 1 -- 172.21.15.86:0/1913540453 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f07b001f020 con 0x7f07c0070330 2024-04-25T14:05:25.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.448+0000 7f07c6c00640 1 -- 172.21.15.86:0/1913540453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f07c0076bf0 con 0x7f07c0070330 2024-04-25T14:05:25.451 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-25T14:05:25.448+0000 7fea29c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:05:25.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.452+0000 7f07b7e00640 1 -- 172.21.15.86:0/1913540453 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7f07c015b640 con 0x7f07c0070330 2024-04-25T14:05:25.453 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T14:05:25.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.452+0000 7f07c6c00640 1 -- 172.21.15.86:0/1913540453 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f07a006f4e0 msgr2=0x7f07a00719f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:25.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.452+0000 7f07c6c00640 1 --2- 172.21.15.86:0/1913540453 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f07a006f4e0 0x7f07a00719f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0af000 tx=0x7f07ac0a7000 comp rx=0 tx=0).stop 2024-04-25T14:05:25.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.452+0000 7f07c6c00640 1 -- 172.21.15.86:0/1913540453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c0070330 msgr2=0x7f07c00c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:25.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.452+0000 7f07c6c00640 1 --2- 172.21.15.86:0/1913540453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c0070330 0x7f07c00c85d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f07b000b3e0 tx=0x7f07b000b8a0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.452+0000 7f07c6c00640 1 -- 172.21.15.86:0/1913540453 shutdown_connections 2024-04-25T14:05:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.452+0000 7f07c6c00640 1 --2- 172.21.15.86:0/1913540453 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f07a006f4e0 0x7f07a00719f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.452+0000 7f07c6c00640 1 --2- 172.21.15.86:0/1913540453 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07c00c27e0 0x7f07c00c2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.452+0000 7f07c6c00640 1 --2- 172.21.15.86:0/1913540453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07c00cdcc0 0x7f07c0159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.452+0000 7f07c6c00640 1 --2- 172.21.15.86:0/1913540453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c0070330 0x7f07c00c85d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.452+0000 7f07c6c00640 1 -- 172.21.15.86:0/1913540453 >> 172.21.15.86:0/1913540453 conn(0x7f07c00b4650 msgr2=0x7f07c00c4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.456+0000 7f07c6c00640 1 -- 172.21.15.86:0/1913540453 shutdown_connections 2024-04-25T14:05:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.456+0000 7f07c6c00640 1 -- 172.21.15.86:0/1913540453 wait complete. 2024-04-25T14:05:25.463 INFO:tasks.ceph.mds.c.smithi086.stderr: -2658> 2024-04-25T14:05:11.220+0000 7f92b6c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T14:05:25.483 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T14:05:25.584 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-04-25T14:05:25.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.632+0000 7f028d800640 1 Processor -- start 2024-04-25T14:05:25.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.632+0000 7f028d800640 1 -- start start 2024-04-25T14:05:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.636+0000 7f028d800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f028815d790 0x7f0288074130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.636+0000 7f028d800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0288074670 0x7f0288160300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.636+0000 7f028d800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0288074a80 con 0x7f0288160840 2024-04-25T14:05:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.636+0000 7f028d800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0288072bf0 con 0x7f0288074670 2024-04-25T14:05:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.636+0000 7f028d800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0288072ed0 con 0x7f028815d790 2024-04-25T14:05:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.636+0000 7f0287400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0288074670 0x7f0288160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.636+0000 7f0287400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0288074670 0x7f0288160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43536/0 (socket says 172.21.15.86:43536) 2024-04-25T14:05:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.636+0000 7f0287400640 1 -- 172.21.15.86:0/3320312455 learned_addr learned my addr 172.21.15.86:0/3320312455 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.636+0000 7f0287e00640 1 --2- 172.21.15.86:0/3320312455 >> v2:172.21.15.155:3300/0 conn(0x7f028815d790 0x7f0288074130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.636+0000 7f0287400640 1 -- 172.21.15.86:0/3320312455 >> v2:172.21.15.155:3300/0 conn(0x7f028815d790 msgr2=0x7f0288074130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.636+0000 7f0287400640 1 --2- 172.21.15.86:0/3320312455 >> v2:172.21.15.155:3300/0 conn(0x7f028815d790 0x7f0288074130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.636+0000 7f0287400640 1 -- 172.21.15.86:0/3320312455 >> v1:172.21.15.155:6789/0 conn(0x7f0288160840 legacy=0x7f02880af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.636+0000 7f0287400640 1 -- 172.21.15.86:0/3320312455 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02880731b0 con 0x7f0288074670 2024-04-25T14:05:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.640+0000 7f0287400640 1 --2- 172.21.15.86:0/3320312455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0288074670 0x7f0288160300 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f02880d2290 tx=0x7f027401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14609986d27ccc94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.640+0000 7f0286a00640 1 -- 172.21.15.86:0/3320312455 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02740ab660 con 0x7f0288074670 2024-04-25T14:05:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.640+0000 7f0286a00640 1 -- 172.21.15.86:0/3320312455 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02740b0070 con 0x7f0288074670 2024-04-25T14:05:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.640+0000 7f0286a00640 1 -- 172.21.15.86:0/3320312455 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02740aba20 con 0x7f0288074670 2024-04-25T14:05:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.640+0000 7f028d800640 1 -- 172.21.15.86:0/3320312455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0288074670 msgr2=0x7f0288160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.640+0000 7f028d800640 1 --2- 172.21.15.86:0/3320312455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0288074670 0x7f0288160300 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f02880d2290 tx=0x7f027401c460 comp rx=0 tx=0).stop 2024-04-25T14:05:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.640+0000 7f028d800640 1 -- 172.21.15.86:0/3320312455 shutdown_connections 2024-04-25T14:05:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.640+0000 7f028d800640 1 --2- 172.21.15.86:0/3320312455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0288074670 0x7f0288160300 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.640+0000 7f028d800640 1 --2- 172.21.15.86:0/3320312455 >> v2:172.21.15.155:3300/0 conn(0x7f028815d790 0x7f0288074130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.640+0000 7f028d800640 1 -- 172.21.15.86:0/3320312455 >> 172.21.15.86:0/3320312455 conn(0x7f02880b4740 msgr2=0x7f02880b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.640+0000 7f028d800640 1 -- 172.21.15.86:0/3320312455 shutdown_connections 2024-04-25T14:05:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.640+0000 7f028d800640 1 -- 172.21.15.86:0/3320312455 wait complete. 2024-04-25T14:05:25.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.640+0000 7f028d800640 1 Processor -- start 2024-04-25T14:05:25.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.640+0000 7f028d800640 1 -- start start 2024-04-25T14:05:25.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.644+0000 7f028d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0288074670 0x7f0288151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.644+0000 7f028d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f028815d790 0x7f02880d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.644+0000 7f028d800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0288160840 0x7f02880d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.644+0000 7f028d800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f028806dbb0 con 0x7f0288074670 2024-04-25T14:05:25.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.644+0000 7f028d800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0288072880 con 0x7f028815d790 2024-04-25T14:05:25.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.644+0000 7f028d800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f028815dba0 con 0x7f0288160840 2024-04-25T14:05:25.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.644+0000 7f0287e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0288074670 0x7f0288151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.644+0000 7f0287e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0288074670 0x7f0288151ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43548/0 (socket says 172.21.15.86:43548) 2024-04-25T14:05:25.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.644+0000 7f0287e00640 1 -- 172.21.15.86:0/2178357885 learned_addr learned my addr 172.21.15.86:0/2178357885 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:25.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.644+0000 7f0287e00640 1 -- 172.21.15.86:0/2178357885 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0288160840 msgr2=0x7f02880d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:25.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.644+0000 7f028ce00640 1 --2- 172.21.15.86:0/2178357885 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0288160840 0x7f02880d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.644+0000 7f0287400640 1 --2- 172.21.15.86:0/2178357885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f028815d790 0x7f02880d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.644+0000 7f0287e00640 1 --2- 172.21.15.86:0/2178357885 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0288160840 0x7f02880d13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.644+0000 7f0287e00640 1 -- 172.21.15.86:0/2178357885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f028815d790 msgr2=0x7f02880d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:25.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.644+0000 7f0287e00640 1 --2- 172.21.15.86:0/2178357885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f028815d790 0x7f02880d0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.644+0000 7f0287e00640 1 -- 172.21.15.86:0/2178357885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02880cf110 con 0x7f0288074670 2024-04-25T14:05:25.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.644+0000 7f028ce00640 1 --2- 172.21.15.86:0/2178357885 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0288160840 0x7f02880d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:25.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.644+0000 7f0287400640 1 --2- 172.21.15.86:0/2178357885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f028815d790 0x7f02880d0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:25.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.644+0000 7f0287e00640 1 --2- 172.21.15.86:0/2178357885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0288074670 0x7f0288151ce0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f027800efc0 tx=0x7f027800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:25.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.644+0000 7f027fe00640 1 -- 172.21.15.86:0/2178357885 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0278018020 con 0x7f0288074670 2024-04-25T14:05:25.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.644+0000 7f027fe00640 1 -- 172.21.15.86:0/2178357885 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0278009e40 con 0x7f0288074670 2024-04-25T14:05:25.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.644+0000 7f028d800640 1 -- 172.21.15.86:0/2178357885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02880cf3f0 con 0x7f0288074670 2024-04-25T14:05:25.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.644+0000 7f027fe00640 1 -- 172.21.15.86:0/2178357885 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02780146a0 con 0x7f0288074670 2024-04-25T14:05:25.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.644+0000 7f028d800640 1 -- 172.21.15.86:0/2178357885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02880cf9b0 con 0x7f0288074670 2024-04-25T14:05:25.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.644+0000 7f028d800640 1 -- 172.21.15.86:0/2178357885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0288076b80 con 0x7f0288074670 2024-04-25T14:05:25.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.648+0000 7f027fe00640 1 -- 172.21.15.86:0/2178357885 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 10) ==== 93262+0+0 (secure 0 0 0) 0x7f027801b020 con 0x7f0288074670 2024-04-25T14:05:25.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.648+0000 7f027fe00640 1 --2- 172.21.15.86:0/2178357885 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f026806f4e0 0x7f02680719f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.648+0000 7f027fe00640 1 -- 172.21.15.86:0/2178357885 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6445+0+0 (secure 0 0 0) 0x7f02780903e0 con 0x7f0288074670 2024-04-25T14:05:25.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.648+0000 7f0287400640 1 --2- 172.21.15.86:0/2178357885 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f026806f4e0 0x7f02680719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.648+0000 7f0287400640 1 --2- 172.21.15.86:0/2178357885 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f026806f4e0 0x7f02680719f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f02740af000 tx=0x7f0274008040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:25.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.652+0000 7f027fe00640 1 -- 172.21.15.86:0/2178357885 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0278022030 con 0x7f0288074670 2024-04-25T14:05:25.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:25.844+0000 7f028d800640 1 -- 172.21.15.86:0/2178357885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f02880b3b80 con 0x7f0288074670 2024-04-25T14:05:26.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.468+0000 7f027fe00640 1 -- 172.21.15.86:0/2178357885 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f0278012050 con 0x7f0288074670 2024-04-25T14:05:26.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.484+0000 7f027fe00640 1 -- 172.21.15.86:0/2178357885 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7f0278010040 con 0x7f0288074670 2024-04-25T14:05:26.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.488+0000 7f028d800640 1 -- 172.21.15.86:0/2178357885 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f026806f4e0 msgr2=0x7f02680719f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:26.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.488+0000 7f028d800640 1 --2- 172.21.15.86:0/2178357885 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f026806f4e0 0x7f02680719f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f02740af000 tx=0x7f0274008040 comp rx=0 tx=0).stop 2024-04-25T14:05:26.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.488+0000 7f028d800640 1 -- 172.21.15.86:0/2178357885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0288074670 msgr2=0x7f0288151ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:26.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.488+0000 7f028d800640 1 --2- 172.21.15.86:0/2178357885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0288074670 0x7f0288151ce0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f027800efc0 tx=0x7f027800c490 comp rx=0 tx=0).stop 2024-04-25T14:05:26.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.488+0000 7f028d800640 1 -- 172.21.15.86:0/2178357885 shutdown_connections 2024-04-25T14:05:26.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.488+0000 7f028d800640 1 --2- 172.21.15.86:0/2178357885 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f026806f4e0 0x7f02680719f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:26.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.488+0000 7f028d800640 1 --2- 172.21.15.86:0/2178357885 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0288160840 0x7f02880d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:26.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.488+0000 7f028d800640 1 --2- 172.21.15.86:0/2178357885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f028815d790 0x7f02880d0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:26.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.488+0000 7f028d800640 1 --2- 172.21.15.86:0/2178357885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0288074670 0x7f0288151ce0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:26.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.488+0000 7f028d800640 1 -- 172.21.15.86:0/2178357885 >> 172.21.15.86:0/2178357885 conn(0x7f02880b4740 msgr2=0x7f02880b2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:26.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.488+0000 7f028d800640 1 -- 172.21.15.86:0/2178357885 shutdown_connections 2024-04-25T14:05:26.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.488+0000 7f028d800640 1 -- 172.21.15.86:0/2178357885 wait complete. 2024-04-25T14:05:26.512 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T14:05:26.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.660+0000 7fe3c7c00640 1 Processor -- start 2024-04-25T14:05:26.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.660+0000 7fe3c7c00640 1 -- start start 2024-04-25T14:05:26.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.664+0000 7fe3c7c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3c00b0540 0x7fe3c00ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:26.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.664+0000 7fe3c7c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe3c0165650 0x7fe3c0167a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:26.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.664+0000 7fe3c7c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3c00b0b20 con 0x7fe3c015e270 2024-04-25T14:05:26.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.664+0000 7fe3c7c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe3c006d870 con 0x7fe3c0165650 2024-04-25T14:05:26.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.664+0000 7fe3c7c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe3c006d220 con 0x7fe3c00b0540 2024-04-25T14:05:26.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.664+0000 7fe3c5c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe3c0165650 0x7fe3c0167a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:26.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.664+0000 7fe3c5200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3c00b0540 0x7fe3c00ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:26.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.664+0000 7fe3c5c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe3c0165650 0x7fe3c0167a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:51894/0 (socket says 172.21.15.86:51894) 2024-04-25T14:05:26.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.664+0000 7fe3c5c00640 1 -- 172.21.15.86:0/3439332385 learned_addr learned my addr 172.21.15.86:0/3439332385 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:26.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.664+0000 7fe3c5c00640 1 -- 172.21.15.86:0/3439332385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3c00b0540 msgr2=0x7fe3c00ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:26.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.664+0000 7fe3c5c00640 1 --2- 172.21.15.86:0/3439332385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3c00b0540 0x7fe3c00ae2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:26.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.664+0000 7fe3c5c00640 1 -- 172.21.15.86:0/3439332385 >> v1:172.21.15.155:6789/0 conn(0x7fe3c015e270 legacy=0x7fe3c015e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:26.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.664+0000 7fe3c5c00640 1 -- 172.21.15.86:0/3439332385 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c006d500 con 0x7fe3c0165650 2024-04-25T14:05:26.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.664+0000 7fe3c5200640 1 --2- 172.21.15.86:0/3439332385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3c00b0540 0x7fe3c00ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:26.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.668+0000 7fe3c5c00640 1 --2- 172.21.15.86:0/3439332385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3c0165650 0x7fe3c0167a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8009e50 tx=0x7fe3b801c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c79575672c4b01fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:26.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.668+0000 7fe3bfe00640 1 -- 172.21.15.86:0/3439332385 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b80a5660 con 0x7fe3c0165650 2024-04-25T14:05:26.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.668+0000 7fe3bfe00640 1 -- 172.21.15.86:0/3439332385 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3b80a5800 con 0x7fe3c0165650 2024-04-25T14:05:26.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.668+0000 7fe3bfe00640 1 -- 172.21.15.86:0/3439332385 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b80a5bd0 con 0x7fe3c0165650 2024-04-25T14:05:26.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.668+0000 7fe3c7c00640 1 -- 172.21.15.86:0/3439332385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3c0165650 msgr2=0x7fe3c0167a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:26.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.668+0000 7fe3c7c00640 1 --2- 172.21.15.86:0/3439332385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3c0165650 0x7fe3c0167a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8009e50 tx=0x7fe3b801c9a0 comp rx=0 tx=0).stop 2024-04-25T14:05:26.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.668+0000 7fe3c7c00640 1 -- 172.21.15.86:0/3439332385 shutdown_connections 2024-04-25T14:05:26.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.668+0000 7fe3c7c00640 1 --2- 172.21.15.86:0/3439332385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3c0165650 0x7fe3c0167a70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:26.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.668+0000 7fe3c7c00640 1 --2- 172.21.15.86:0/3439332385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3c00b0540 0x7fe3c00ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:26.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.668+0000 7fe3c7c00640 1 -- 172.21.15.86:0/3439332385 >> 172.21.15.86:0/3439332385 conn(0x7fe3c00b4a40 msgr2=0x7fe3c00b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:26.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.668+0000 7fe3c7c00640 1 -- 172.21.15.86:0/3439332385 shutdown_connections 2024-04-25T14:05:26.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.668+0000 7fe3c7c00640 1 -- 172.21.15.86:0/3439332385 wait complete. 2024-04-25T14:05:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.672+0000 7fe3c7c00640 1 Processor -- start 2024-04-25T14:05:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.672+0000 7fe3c7c00640 1 -- start start 2024-04-25T14:05:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.672+0000 7fe3c7c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c00b0540 0x7fe3c0164a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.672+0000 7fe3c7c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3c015e270 0x7fe3c01445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.672+0000 7fe3c7c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3c0165650 0x7fe3c0156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.672+0000 7fe3c7c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c016ab60 con 0x7fe3c00b0540 2024-04-25T14:05:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.672+0000 7fe3c7c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c016a9e0 con 0x7fe3c015e270 2024-04-25T14:05:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.672+0000 7fe3c7c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe3c016ace0 con 0x7fe3c0165650 2024-04-25T14:05:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.672+0000 7fe3c5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c00b0540 0x7fe3c0164a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.672+0000 7fe3c5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c00b0540 0x7fe3c0164a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43562/0 (socket says 172.21.15.86:43562) 2024-04-25T14:05:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.672+0000 7fe3c5200640 1 -- 172.21.15.86:0/3951189257 learned_addr learned my addr 172.21.15.86:0/3951189257 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.672+0000 7fe3c5c00640 1 --2- 172.21.15.86:0/3951189257 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3c0165650 0x7fe3c0156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.672+0000 7fe3b7e00640 1 --2- 172.21.15.86:0/3951189257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3c015e270 0x7fe3c01445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.672+0000 7fe3c5200640 1 -- 172.21.15.86:0/3951189257 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3c0165650 msgr2=0x7fe3c0156900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.672+0000 7fe3c5200640 1 --2- 172.21.15.86:0/3951189257 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3c0165650 0x7fe3c0156900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.672+0000 7fe3c5200640 1 -- 172.21.15.86:0/3951189257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3c015e270 msgr2=0x7fe3c01445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.672+0000 7fe3c5200640 1 --2- 172.21.15.86:0/3951189257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3c015e270 0x7fe3c01445b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.672+0000 7fe3c5200640 1 -- 172.21.15.86:0/3951189257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c00b3170 con 0x7fe3c00b0540 2024-04-25T14:05:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.672+0000 7fe3c5200640 1 --2- 172.21.15.86:0/3951189257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c00b0540 0x7fe3c0164a20 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000e960 tx=0x7fe3b000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.672+0000 7fe3bd600640 1 -- 172.21.15.86:0/3951189257 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0017070 con 0x7fe3c00b0540 2024-04-25T14:05:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.672+0000 7fe3bd600640 1 -- 172.21.15.86:0/3951189257 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3b000ca00 con 0x7fe3c00b0540 2024-04-25T14:05:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.672+0000 7fe3bd600640 1 -- 172.21.15.86:0/3951189257 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b000cdd0 con 0x7fe3c00b0540 2024-04-25T14:05:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.672+0000 7fe3c7c00640 1 -- 172.21.15.86:0/3951189257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c00b3450 con 0x7fe3c00b0540 2024-04-25T14:05:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.672+0000 7fe3c7c00640 1 -- 172.21.15.86:0/3951189257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c006ea50 con 0x7fe3c00b0540 2024-04-25T14:05:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.676+0000 7fe3c7c00640 1 -- 172.21.15.86:0/3951189257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3c0050750 con 0x7fe3c00b0540 2024-04-25T14:05:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.676+0000 7fe3bd600640 1 -- 172.21.15.86:0/3951189257 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7fe3b001a070 con 0x7fe3c00b0540 2024-04-25T14:05:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.676+0000 7fe3bd600640 1 --2- 172.21.15.86:0/3951189257 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fe39c06f5a0 0x7fe39c071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.676+0000 7fe3bd600640 1 -- 172.21.15.86:0/3951189257 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6445+0+0 (secure 0 0 0) 0x7fe3b008f900 con 0x7fe3c00b0540 2024-04-25T14:05:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.676+0000 7fe3b7e00640 1 --2- 172.21.15.86:0/3951189257 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fe39c06f5a0 0x7fe39c071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.676+0000 7fe3b7e00640 1 --2- 172.21.15.86:0/3951189257 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fe39c06f5a0 0x7fe39c071ab0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe3c00b3d20 tx=0x7fe3a8009040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:26.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.680+0000 7fe3bd600640 1 -- 172.21.15.86:0/3951189257 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe3b005bdb0 con 0x7fe3c00b0540 2024-04-25T14:05:26.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:26.840+0000 7fe3c7c00640 1 -- 172.21.15.86:0/3951189257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe3c0167b50 con 0x7fe3c00b0540 2024-04-25T14:05:27.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.500+0000 7fe3bd600640 1 -- 172.21.15.86:0/3951189257 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v41) ==== 181+0+0 (secure 0 0 0) 0x7fe3b00600b0 con 0x7fe3c00b0540 2024-04-25T14:05:27.501 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-04-25T14:05:27.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.500+0000 7fe3c7c00640 1 -- 172.21.15.86:0/3951189257 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fe39c06f5a0 msgr2=0x7fe39c071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:27.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.500+0000 7fe3c7c00640 1 --2- 172.21.15.86:0/3951189257 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fe39c06f5a0 0x7fe39c071ab0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe3c00b3d20 tx=0x7fe3a8009040 comp rx=0 tx=0).stop 2024-04-25T14:05:27.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.500+0000 7fe3c7c00640 1 -- 172.21.15.86:0/3951189257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c00b0540 msgr2=0x7fe3c0164a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:27.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.500+0000 7fe3c7c00640 1 --2- 172.21.15.86:0/3951189257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c00b0540 0x7fe3c0164a20 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000e960 tx=0x7fe3b000ee20 comp rx=0 tx=0).stop 2024-04-25T14:05:27.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.500+0000 7fe3c7c00640 1 -- 172.21.15.86:0/3951189257 shutdown_connections 2024-04-25T14:05:27.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.500+0000 7fe3c7c00640 1 --2- 172.21.15.86:0/3951189257 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fe39c06f5a0 0x7fe39c071ab0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:27.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.500+0000 7fe3c7c00640 1 --2- 172.21.15.86:0/3951189257 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3c0165650 0x7fe3c0156900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:27.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.500+0000 7fe3c7c00640 1 --2- 172.21.15.86:0/3951189257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3c015e270 0x7fe3c01445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:27.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.500+0000 7fe3c7c00640 1 --2- 172.21.15.86:0/3951189257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c00b0540 0x7fe3c0164a20 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:27.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.500+0000 7fe3c7c00640 1 -- 172.21.15.86:0/3951189257 >> 172.21.15.86:0/3951189257 conn(0x7fe3c00b4a40 msgr2=0x7fe3c015e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:27.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.500+0000 7fe3c7c00640 1 -- 172.21.15.86:0/3951189257 shutdown_connections 2024-04-25T14:05:27.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.500+0000 7fe3c7c00640 1 -- 172.21.15.86:0/3951189257 wait complete. 2024-04-25T14:05:27.528 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T14:05:27.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.688+0000 7f868a800640 1 Processor -- start 2024-04-25T14:05:27.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.688+0000 7f868a800640 1 -- start start 2024-04-25T14:05:27.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.688+0000 7f868a800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f86840d25c0 0x7f86840b6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.688+0000 7f868a800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86840700a0 0x7f86841617b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.688+0000 7f868a800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f868415dfe0 con 0x7f868415dbd0 2024-04-25T14:05:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.688+0000 7f868a800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f86840ce4e0 con 0x7f86840d25c0 2024-04-25T14:05:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.688+0000 7f868a800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f86840ce730 con 0x7f86840700a0 2024-04-25T14:05:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.688+0000 7f8682a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f868415dbd0 0x7f868406f990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47392/0 (socket says 172.21.15.86:47392) 2024-04-25T14:05:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.688+0000 7f8682a00640 1 -- 172.21.15.86:0/3364251969 learned_addr learned my addr 172.21.15.86:0/3364251969 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.688+0000 7f8683400640 1 --2- 172.21.15.86:0/3364251969 >> v2:172.21.15.155:3300/0 conn(0x7f86840d25c0 0x7f86840b6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.688+0000 7f8683e00640 1 --2- 172.21.15.86:0/3364251969 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86840700a0 0x7f86841617b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.688+0000 7f8682000640 1 -- 172.21.15.86:0/3364251969 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4033312561 0 0) 0x7f868415dfe0 con 0x7f868415dbd0 2024-04-25T14:05:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.688+0000 7f8682000640 1 -- 172.21.15.86:0/3364251969 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8668003530 con 0x7f868415dbd0 2024-04-25T14:05:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.688+0000 7f8683400640 1 -- 172.21.15.86:0/3364251969 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86840700a0 msgr2=0x7f86841617b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.688+0000 7f8683400640 1 --2- 172.21.15.86:0/3364251969 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86840700a0 0x7f86841617b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.688+0000 7f8683400640 1 -- 172.21.15.86:0/3364251969 >> v1:172.21.15.86:6789/0 conn(0x7f868415dbd0 legacy=0x7f868406f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.688+0000 7f8683400640 1 -- 172.21.15.86:0/3364251969 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86840cdd80 con 0x7f86840d25c0 2024-04-25T14:05:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.688+0000 7f8683e00640 1 --2- 172.21.15.86:0/3364251969 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86840700a0 0x7f86841617b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:27.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.692+0000 7f8683400640 1 --2- 172.21.15.86:0/3364251969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86840d25c0 0x7f86840b6510 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8674009d90 tx=0x7f867401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ad402db3ec089b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:27.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.692+0000 7f8682000640 1 -- 172.21.15.86:0/3364251969 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86740b2070 con 0x7f86840d25c0 2024-04-25T14:05:27.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.692+0000 7f8682000640 1 -- 172.21.15.86:0/3364251969 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86740a58b0 con 0x7f86840d25c0 2024-04-25T14:05:27.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.692+0000 7f8682000640 1 -- 172.21.15.86:0/3364251969 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86740a5c60 con 0x7f86840d25c0 2024-04-25T14:05:27.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.692+0000 7f868a800640 1 -- 172.21.15.86:0/3364251969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86840d25c0 msgr2=0x7f86840b6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:27.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.692+0000 7f868a800640 1 --2- 172.21.15.86:0/3364251969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86840d25c0 0x7f86840b6510 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8674009d90 tx=0x7f867401c970 comp rx=0 tx=0).stop 2024-04-25T14:05:27.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.692+0000 7f868a800640 1 -- 172.21.15.86:0/3364251969 shutdown_connections 2024-04-25T14:05:27.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.692+0000 7f868a800640 1 --2- 172.21.15.86:0/3364251969 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86840700a0 0x7f86841617b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:27.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.692+0000 7f868a800640 1 --2- 172.21.15.86:0/3364251969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86840d25c0 0x7f86840b6510 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:27.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.692+0000 7f868a800640 1 -- 172.21.15.86:0/3364251969 >> 172.21.15.86:0/3364251969 conn(0x7f86840b5060 msgr2=0x7f86840b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:27.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.692+0000 7f868a800640 1 -- 172.21.15.86:0/3364251969 shutdown_connections 2024-04-25T14:05:27.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.692+0000 7f868a800640 1 -- 172.21.15.86:0/3364251969 wait complete. 2024-04-25T14:05:27.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.696+0000 7f868a800640 1 Processor -- start 2024-04-25T14:05:27.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.696+0000 7f868a800640 1 -- start start 2024-04-25T14:05:27.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.696+0000 7f868a800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86840d25c0 0x7f86841491f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:27.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.696+0000 7f868a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f868415dbd0 0x7f8684160850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:27.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.696+0000 7f868a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f868414ceb0 0x7f868414d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:27.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.696+0000 7f868a800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f868416abb0 con 0x7f868415dbd0 2024-04-25T14:05:27.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.696+0000 7f868a800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f868416aa30 con 0x7f868414ceb0 2024-04-25T14:05:27.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.696+0000 7f868a800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f868416ad30 con 0x7f86840d25c0 2024-04-25T14:05:27.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.696+0000 7f8682a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f868415dbd0 0x7f8684160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:27.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.696+0000 7f8682a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f868415dbd0 0x7f8684160850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43572/0 (socket says 172.21.15.86:43572) 2024-04-25T14:05:27.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.696+0000 7f8682a00640 1 -- 172.21.15.86:0/961832177 learned_addr learned my addr 172.21.15.86:0/961832177 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:27.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.700+0000 7f8683400640 1 --2- 172.21.15.86:0/961832177 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86840d25c0 0x7f86841491f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:27.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.700+0000 7f8683e00640 1 --2- 172.21.15.86:0/961832177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f868414ceb0 0x7f868414d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:27.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.700+0000 7f8682a00640 1 -- 172.21.15.86:0/961832177 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86840d25c0 msgr2=0x7f86841491f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:27.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.700+0000 7f8682a00640 1 --2- 172.21.15.86:0/961832177 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86840d25c0 0x7f86841491f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:27.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.700+0000 7f8682a00640 1 -- 172.21.15.86:0/961832177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f868414ceb0 msgr2=0x7f868414d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:27.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.700+0000 7f8682a00640 1 --2- 172.21.15.86:0/961832177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f868414ceb0 0x7f868414d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:27.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.700+0000 7f8682a00640 1 -- 172.21.15.86:0/961832177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8684149730 con 0x7f868415dbd0 2024-04-25T14:05:27.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.700+0000 7f8683e00640 1 --2- 172.21.15.86:0/961832177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f868414ceb0 0x7f868414d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:27.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.700+0000 7f8682a00640 1 --2- 172.21.15.86:0/961832177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f868415dbd0 0x7f8684160850 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f867000c3f0 tx=0x7f867000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:27.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.700+0000 7f867b400640 1 -- 172.21.15.86:0/961832177 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8670017070 con 0x7f868415dbd0 2024-04-25T14:05:27.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.700+0000 7f867b400640 1 -- 172.21.15.86:0/961832177 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8670005cf0 con 0x7f868415dbd0 2024-04-25T14:05:27.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.700+0000 7f867b400640 1 -- 172.21.15.86:0/961832177 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86700072f0 con 0x7f868415dbd0 2024-04-25T14:05:27.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.700+0000 7f868a800640 1 -- 172.21.15.86:0/961832177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8684151ab0 con 0x7f868415dbd0 2024-04-25T14:05:27.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.700+0000 7f868a800640 1 -- 172.21.15.86:0/961832177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f868414df10 con 0x7f868415dbd0 2024-04-25T14:05:27.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.700+0000 7f868a800640 1 -- 172.21.15.86:0/961832177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8684159dc0 con 0x7f868415dbd0 2024-04-25T14:05:27.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.700+0000 7f867b400640 1 -- 172.21.15.86:0/961832177 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f867001a070 con 0x7f868415dbd0 2024-04-25T14:05:27.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.704+0000 7f867b400640 1 --2- 172.21.15.86:0/961832177 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f866406f570 0x7f8664071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:27.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.704+0000 7f867b400640 1 -- 172.21.15.86:0/961832177 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6007+0+0 (secure 0 0 0) 0x7f867008ea60 con 0x7f868415dbd0 2024-04-25T14:05:27.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.704+0000 7f8683400640 1 --2- 172.21.15.86:0/961832177 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f866406f570 0x7f8664071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:27.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.704+0000 7f8683400640 1 --2- 172.21.15.86:0/961832177 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f866406f570 0x7f8664071a80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f86740af000 tx=0x7f86740a7070 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:27.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.708+0000 7f867b400640 1 -- 172.21.15.86:0/961832177 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8684149730 con 0x7f868415dbd0 2024-04-25T14:05:27.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:27.864+0000 7f868a800640 1 -- 172.21.15.86:0/961832177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8684050750 con 0x7f868415dbd0 2024-04-25T14:05:28.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.500+0000 7f867b400640 1 -- 172.21.15.86:0/961832177 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v42) ==== 169+0+0 (secure 0 0 0) 0x7f867005b700 con 0x7f868415dbd0 2024-04-25T14:05:28.503 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-04-25T14:05:28.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.504+0000 7f868a800640 1 -- 172.21.15.86:0/961832177 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f866406f570 msgr2=0x7f8664071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:28.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.504+0000 7f868a800640 1 --2- 172.21.15.86:0/961832177 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f866406f570 0x7f8664071a80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f86740af000 tx=0x7f86740a7070 comp rx=0 tx=0).stop 2024-04-25T14:05:28.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.504+0000 7f868a800640 1 -- 172.21.15.86:0/961832177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f868415dbd0 msgr2=0x7f8684160850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:28.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.504+0000 7f868a800640 1 --2- 172.21.15.86:0/961832177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f868415dbd0 0x7f8684160850 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f867000c3f0 tx=0x7f867000c8b0 comp rx=0 tx=0).stop 2024-04-25T14:05:28.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.504+0000 7f868a800640 1 -- 172.21.15.86:0/961832177 shutdown_connections 2024-04-25T14:05:28.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.504+0000 7f868a800640 1 --2- 172.21.15.86:0/961832177 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f866406f570 0x7f8664071a80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:28.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.504+0000 7f868a800640 1 --2- 172.21.15.86:0/961832177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f868414ceb0 0x7f868414d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:28.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.504+0000 7f868a800640 1 --2- 172.21.15.86:0/961832177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f868415dbd0 0x7f8684160850 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:28.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.504+0000 7f868a800640 1 --2- 172.21.15.86:0/961832177 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86840d25c0 0x7f86841491f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:28.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.504+0000 7f868a800640 1 -- 172.21.15.86:0/961832177 >> 172.21.15.86:0/961832177 conn(0x7f86840b5060 msgr2=0x7f868415b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:28.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.504+0000 7f868a800640 1 -- 172.21.15.86:0/961832177 shutdown_connections 2024-04-25T14:05:28.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.504+0000 7f868a800640 1 -- 172.21.15.86:0/961832177 wait complete. 2024-04-25T14:05:28.535 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-25T14:05:28.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.688+0000 7f5243e00640 1 Processor -- start 2024-04-25T14:05:28.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.688+0000 7f5243e00640 1 -- start start 2024-04-25T14:05:28.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.688+0000 7f5243e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f523c15ca50 0x7f523c073950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:28.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.688+0000 7f5243e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f523c0743c0 0x7f523c1605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:28.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.688+0000 7f5243e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f523c15ce60 con 0x7f523c0747d0 2024-04-25T14:05:28.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.688+0000 7f5243e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f523c072b90 con 0x7f523c0743c0 2024-04-25T14:05:28.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.688+0000 7f5243e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f523c072e70 con 0x7f523c15ca50 2024-04-25T14:05:28.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.688+0000 7f5240a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f523c0743c0 0x7f523c1605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:28.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.688+0000 7f5240a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f523c0743c0 0x7f523c1605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43586/0 (socket says 172.21.15.86:43586) 2024-04-25T14:05:28.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.688+0000 7f5240a00640 1 -- 172.21.15.86:0/3258599564 learned_addr learned my addr 172.21.15.86:0/3258599564 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:28.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.688+0000 7f5241400640 1 --2- 172.21.15.86:0/3258599564 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f523c15ca50 0x7f523c073950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:28.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.688+0000 7f5240a00640 1 -- 172.21.15.86:0/3258599564 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f523c15ca50 msgr2=0x7f523c073950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:28.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.688+0000 7f5240a00640 1 --2- 172.21.15.86:0/3258599564 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f523c15ca50 0x7f523c073950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:28.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.688+0000 7f5240a00640 1 -- 172.21.15.86:0/3258599564 >> v1:172.21.15.155:6789/0 conn(0x7f523c0747d0 legacy=0x7f523c06e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:28.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.688+0000 7f5240a00640 1 -- 172.21.15.86:0/3258599564 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523c073150 con 0x7f523c0743c0 2024-04-25T14:05:28.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.688+0000 7f5241400640 1 --2- 172.21.15.86:0/3258599564 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f523c15ca50 0x7f523c073950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:28.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.692+0000 7f5240a00640 1 --2- 172.21.15.86:0/3258599564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f523c0743c0 0x7f523c1605f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5228009870 tx=0x7f522801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d4f8d29ee460ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:28.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.692+0000 7f523be00640 1 -- 172.21.15.86:0/3258599564 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52280b0070 con 0x7f523c0743c0 2024-04-25T14:05:28.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.692+0000 7f523be00640 1 -- 172.21.15.86:0/3258599564 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52280ab600 con 0x7f523c0743c0 2024-04-25T14:05:28.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.692+0000 7f523be00640 1 -- 172.21.15.86:0/3258599564 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52280ab9d0 con 0x7f523c0743c0 2024-04-25T14:05:28.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.692+0000 7f5243e00640 1 -- 172.21.15.86:0/3258599564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f523c0743c0 msgr2=0x7f523c1605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:28.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.692+0000 7f5243e00640 1 --2- 172.21.15.86:0/3258599564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f523c0743c0 0x7f523c1605f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5228009870 tx=0x7f522801c520 comp rx=0 tx=0).stop 2024-04-25T14:05:28.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.692+0000 7f5243e00640 1 -- 172.21.15.86:0/3258599564 shutdown_connections 2024-04-25T14:05:28.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.692+0000 7f5243e00640 1 --2- 172.21.15.86:0/3258599564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f523c0743c0 0x7f523c1605f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:28.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.692+0000 7f5243e00640 1 --2- 172.21.15.86:0/3258599564 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f523c15ca50 0x7f523c073950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:28.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.692+0000 7f5243e00640 1 -- 172.21.15.86:0/3258599564 >> 172.21.15.86:0/3258599564 conn(0x7f523c0b4a40 msgr2=0x7f523c0b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:28.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.692+0000 7f5243e00640 1 -- 172.21.15.86:0/3258599564 shutdown_connections 2024-04-25T14:05:28.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.692+0000 7f5243e00640 1 -- 172.21.15.86:0/3258599564 wait complete. 2024-04-25T14:05:28.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.696+0000 7f5243e00640 1 Processor -- start 2024-04-25T14:05:28.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.696+0000 7f5243e00640 1 -- start start 2024-04-25T14:05:28.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.696+0000 7f5243e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f523c0743c0 0x7f523c14fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:28.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.696+0000 7f5243e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f523c0747d0 0x7f523c152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:28.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.696+0000 7f5243e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f523c15ca50 0x7f523c152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:28.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.696+0000 7f5243e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f523c16a530 con 0x7f523c15ca50 2024-04-25T14:05:28.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.696+0000 7f5243e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f523c16a3b0 con 0x7f523c0743c0 2024-04-25T14:05:28.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.696+0000 7f5243e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f523c16a6b0 con 0x7f523c0747d0 2024-04-25T14:05:28.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.696+0000 7f5241e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f523c15ca50 0x7f523c152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:28.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.696+0000 7f5241e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f523c15ca50 0x7f523c152b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43592/0 (socket says 172.21.15.86:43592) 2024-04-25T14:05:28.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.696+0000 7f5241e00640 1 -- 172.21.15.86:0/4042294989 learned_addr learned my addr 172.21.15.86:0/4042294989 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:28.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.696+0000 7f5241400640 1 --2- 172.21.15.86:0/4042294989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f523c0743c0 0x7f523c14fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:28.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.696+0000 7f5240a00640 1 --2- 172.21.15.86:0/4042294989 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f523c0747d0 0x7f523c152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:28.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.700+0000 7f5241e00640 1 -- 172.21.15.86:0/4042294989 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f523c0747d0 msgr2=0x7f523c152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:28.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.700+0000 7f5241e00640 1 --2- 172.21.15.86:0/4042294989 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f523c0747d0 0x7f523c152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:28.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.700+0000 7f5241e00640 1 -- 172.21.15.86:0/4042294989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f523c0743c0 msgr2=0x7f523c14fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:28.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.700+0000 7f5241e00640 1 --2- 172.21.15.86:0/4042294989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f523c0743c0 0x7f523c14fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:28.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.700+0000 7f5241e00640 1 -- 172.21.15.86:0/4042294989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523c158ec0 con 0x7f523c15ca50 2024-04-25T14:05:28.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.700+0000 7f5241e00640 1 --2- 172.21.15.86:0/4042294989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f523c15ca50 0x7f523c152b50 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f523400b490 tx=0x7f523400b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:28.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.700+0000 7f5239600640 1 -- 172.21.15.86:0/4042294989 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234015020 con 0x7f523c15ca50 2024-04-25T14:05:28.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.700+0000 7f5239600640 1 -- 172.21.15.86:0/4042294989 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5234011830 con 0x7f523c15ca50 2024-04-25T14:05:28.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.700+0000 7f5239600640 1 -- 172.21.15.86:0/4042294989 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234011c60 con 0x7f523c15ca50 2024-04-25T14:05:28.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.700+0000 7f5243e00640 1 -- 172.21.15.86:0/4042294989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f523c153310 con 0x7f523c15ca50 2024-04-25T14:05:28.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.700+0000 7f5243e00640 1 -- 172.21.15.86:0/4042294989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f523c1583e0 con 0x7f523c15ca50 2024-04-25T14:05:28.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.700+0000 7f5243e00640 1 -- 172.21.15.86:0/4042294989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f523c050750 con 0x7f523c15ca50 2024-04-25T14:05:28.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.700+0000 7f5239600640 1 -- 172.21.15.86:0/4042294989 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f5234018020 con 0x7f523c15ca50 2024-04-25T14:05:28.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.700+0000 7f5239600640 1 --2- 172.21.15.86:0/4042294989 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f521c06f570 0x7f521c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:28.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.700+0000 7f5239600640 1 -- 172.21.15.86:0/4042294989 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5593+0+0 (secure 0 0 0) 0x7f523408f170 con 0x7f523c15ca50 2024-04-25T14:05:28.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.700+0000 7f5241400640 1 --2- 172.21.15.86:0/4042294989 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f521c06f570 0x7f521c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:28.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.704+0000 7f5241400640 1 --2- 172.21.15.86:0/4042294989 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f521c06f570 0x7f521c071a80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f523c0aec40 tx=0x7f52300077c0 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:28.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.708+0000 7f5239600640 1 -- 172.21.15.86:0/4042294989 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f523401f020 con 0x7f523c15ca50 2024-04-25T14:05:28.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:28.896+0000 7f5243e00640 1 -- 172.21.15.86:0/4042294989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f523c0b3d30 con 0x7f523c15ca50 2024-04-25T14:05:29.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.516+0000 7f5239600640 1 -- 172.21.15.86:0/4042294989 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v21) ==== 136+0+0 (secure 0 0 0) 0x7f523405b960 con 0x7f523c15ca50 2024-04-25T14:05:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.520+0000 7f5243e00640 1 -- 172.21.15.86:0/4042294989 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f521c06f570 msgr2=0x7f521c071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.520+0000 7f5243e00640 1 --2- 172.21.15.86:0/4042294989 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f521c06f570 0x7f521c071a80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f523c0aec40 tx=0x7f52300077c0 comp rx=0 tx=0).stop 2024-04-25T14:05:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.520+0000 7f5243e00640 1 -- 172.21.15.86:0/4042294989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f523c15ca50 msgr2=0x7f523c152b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.520+0000 7f5243e00640 1 --2- 172.21.15.86:0/4042294989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f523c15ca50 0x7f523c152b50 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f523400b490 tx=0x7f523400b950 comp rx=0 tx=0).stop 2024-04-25T14:05:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.520+0000 7f5243e00640 1 -- 172.21.15.86:0/4042294989 shutdown_connections 2024-04-25T14:05:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.520+0000 7f5243e00640 1 --2- 172.21.15.86:0/4042294989 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f521c06f570 0x7f521c071a80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.520+0000 7f5243e00640 1 --2- 172.21.15.86:0/4042294989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f523c15ca50 0x7f523c152b50 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.520+0000 7f5243e00640 1 --2- 172.21.15.86:0/4042294989 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f523c0747d0 0x7f523c152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:29.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.520+0000 7f5243e00640 1 --2- 172.21.15.86:0/4042294989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f523c0743c0 0x7f523c14fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:29.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.520+0000 7f5243e00640 1 -- 172.21.15.86:0/4042294989 >> 172.21.15.86:0/4042294989 conn(0x7f523c0b4a40 msgr2=0x7f523c06d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:29.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.520+0000 7f5243e00640 1 -- 172.21.15.86:0/4042294989 shutdown_connections 2024-04-25T14:05:29.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.520+0000 7f5243e00640 1 -- 172.21.15.86:0/4042294989 wait complete. 2024-04-25T14:05:29.542 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:05:29.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.688+0000 7fb885e00640 1 Processor -- start 2024-04-25T14:05:29.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.688+0000 7fb885e00640 1 -- start start 2024-04-25T14:05:29.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.688+0000 7fb885e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8800ce570 0x7fb8800c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:29.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.688+0000 7fb885e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb8800c47f0 0x7fb880167e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:29.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.688+0000 7fb885e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8800c4c00 con 0x7fb880168360 2024-04-25T14:05:29.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.688+0000 7fb885e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb88006dcb0 con 0x7fb8800c47f0 2024-04-25T14:05:29.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.688+0000 7fb885e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb88006d0a0 con 0x7fb8800ce570 2024-04-25T14:05:29.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.692+0000 7fb876000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb8800c47f0 0x7fb880167e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:29.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.692+0000 7fb87ea00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8800ce570 0x7fb8800c42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:29.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.692+0000 7fb876000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb8800c47f0 0x7fb880167e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:50774/0 (socket says 172.21.15.86:50774) 2024-04-25T14:05:29.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.692+0000 7fb876000640 1 -- 172.21.15.86:0/1419566420 learned_addr learned my addr 172.21.15.86:0/1419566420 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:29.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.692+0000 7fb87ea00640 1 -- 172.21.15.86:0/1419566420 >> v2:172.21.15.155:3300/0 conn(0x7fb8800c47f0 msgr2=0x7fb880167e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:29.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.692+0000 7fb87ea00640 1 --2- 172.21.15.86:0/1419566420 >> v2:172.21.15.155:3300/0 conn(0x7fb8800c47f0 0x7fb880167e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:29.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.692+0000 7fb87ea00640 1 -- 172.21.15.86:0/1419566420 >> v1:172.21.15.155:6789/0 conn(0x7fb880168360 legacy=0x7fb88006e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:29.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.692+0000 7fb87ea00640 1 -- 172.21.15.86:0/1419566420 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88006d380 con 0x7fb8800ce570 2024-04-25T14:05:29.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.692+0000 7fb876000640 1 --2- 172.21.15.86:0/1419566420 >> v2:172.21.15.155:3300/0 conn(0x7fb8800c47f0 0x7fb880167e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:29.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.692+0000 7fb87ea00640 1 --2- 172.21.15.86:0/1419566420 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8800ce570 0x7fb8800c42b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb8700097c0 tx=0x7fb87001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7df13c0d59780dd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:29.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.696+0000 7fb87e000640 1 -- 172.21.15.86:0/1419566420 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8700b2070 con 0x7fb8800ce570 2024-04-25T14:05:29.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.696+0000 7fb87e000640 1 -- 172.21.15.86:0/1419566420 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8700ad640 con 0x7fb8800ce570 2024-04-25T14:05:29.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.696+0000 7fb87e000640 1 -- 172.21.15.86:0/1419566420 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8700ada10 con 0x7fb8800ce570 2024-04-25T14:05:29.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.696+0000 7fb885e00640 1 -- 172.21.15.86:0/1419566420 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8800ce570 msgr2=0x7fb8800c42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:29.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.696+0000 7fb885e00640 1 --2- 172.21.15.86:0/1419566420 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8800ce570 0x7fb8800c42b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb8700097c0 tx=0x7fb87001c4a0 comp rx=0 tx=0).stop 2024-04-25T14:05:29.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.696+0000 7fb885e00640 1 -- 172.21.15.86:0/1419566420 shutdown_connections 2024-04-25T14:05:29.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.696+0000 7fb885e00640 1 --2- 172.21.15.86:0/1419566420 >> v2:172.21.15.155:3300/0 conn(0x7fb8800c47f0 0x7fb880167e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:29.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.696+0000 7fb885e00640 1 --2- 172.21.15.86:0/1419566420 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8800ce570 0x7fb8800c42b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:29.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.696+0000 7fb885e00640 1 -- 172.21.15.86:0/1419566420 >> 172.21.15.86:0/1419566420 conn(0x7fb8800b4650 msgr2=0x7fb8800b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:29.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.696+0000 7fb885e00640 1 -- 172.21.15.86:0/1419566420 shutdown_connections 2024-04-25T14:05:29.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.696+0000 7fb885e00640 1 -- 172.21.15.86:0/1419566420 wait complete. 2024-04-25T14:05:29.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.696+0000 7fb885e00640 1 Processor -- start 2024-04-25T14:05:29.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.700+0000 7fb885e00640 1 -- start start 2024-04-25T14:05:29.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.700+0000 7fb885e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8800c47f0 0x7fb880073e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:29.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.700+0000 7fb885e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8800ce570 0x7fb880154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:29.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.700+0000 7fb885e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb880168360 0x7fb880164c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:29.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.700+0000 7fb885e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8800b07e0 con 0x7fb8800c47f0 2024-04-25T14:05:29.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.700+0000 7fb885e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb8800b0660 con 0x7fb8800ce570 2024-04-25T14:05:29.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.700+0000 7fb885e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb880167a00 con 0x7fb880168360 2024-04-25T14:05:29.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.700+0000 7fb87ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8800c47f0 0x7fb880073e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:29.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.700+0000 7fb87ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8800c47f0 0x7fb880073e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51936/0 (socket says 172.21.15.86:51936) 2024-04-25T14:05:29.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.700+0000 7fb87ea00640 1 -- 172.21.15.86:0/3543646985 learned_addr learned my addr 172.21.15.86:0/3543646985 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:29.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.700+0000 7fb876000640 1 --2- 172.21.15.86:0/3543646985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8800ce570 0x7fb880154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:29.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.700+0000 7fb87f400640 1 --2- 172.21.15.86:0/3543646985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb880168360 0x7fb880164c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:29.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.700+0000 7fb87ea00640 1 -- 172.21.15.86:0/3543646985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb880168360 msgr2=0x7fb880164c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:29.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.700+0000 7fb87ea00640 1 --2- 172.21.15.86:0/3543646985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb880168360 0x7fb880164c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:29.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.700+0000 7fb87ea00640 1 -- 172.21.15.86:0/3543646985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8800ce570 msgr2=0x7fb880154d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:29.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.700+0000 7fb87ea00640 1 --2- 172.21.15.86:0/3543646985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8800ce570 0x7fb880154d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:29.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.700+0000 7fb87ea00640 1 -- 172.21.15.86:0/3543646985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8801651b0 con 0x7fb8800c47f0 2024-04-25T14:05:29.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.700+0000 7fb87ea00640 1 --2- 172.21.15.86:0/3543646985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8800c47f0 0x7fb880073e60 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb87001a0d0 tx=0x7fb870019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:29.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.700+0000 7fb877e00640 1 -- 172.21.15.86:0/3543646985 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8700b2040 con 0x7fb8800c47f0 2024-04-25T14:05:29.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.700+0000 7fb877e00640 1 -- 172.21.15.86:0/3543646985 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8700bd070 con 0x7fb8800c47f0 2024-04-25T14:05:29.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.700+0000 7fb885e00640 1 -- 172.21.15.86:0/3543646985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb880161f30 con 0x7fb8800c47f0 2024-04-25T14:05:29.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.700+0000 7fb877e00640 1 -- 172.21.15.86:0/3543646985 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8700b78c0 con 0x7fb8800c47f0 2024-04-25T14:05:29.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.700+0000 7fb885e00640 1 -- 172.21.15.86:0/3543646985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8801621d0 con 0x7fb8800c47f0 2024-04-25T14:05:29.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.704+0000 7fb885e00640 1 -- 172.21.15.86:0/3543646985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb880069d00 con 0x7fb8800c47f0 2024-04-25T14:05:29.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.704+0000 7fb877e00640 1 -- 172.21.15.86:0/3543646985 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7fb8700b5070 con 0x7fb8800c47f0 2024-04-25T14:05:29.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.704+0000 7fb877e00640 1 --2- 172.21.15.86:0/3543646985 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb85c06f5a0 0x7fb85c071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:29.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.704+0000 7fb877e00640 1 -- 172.21.15.86:0/3543646985 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5593+0+0 (secure 0 0 0) 0x7fb870129120 con 0x7fb8800c47f0 2024-04-25T14:05:29.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.704+0000 7fb876000640 1 --2- 172.21.15.86:0/3543646985 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb85c06f5a0 0x7fb85c071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:29.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.704+0000 7fb876000640 1 --2- 172.21.15.86:0/3543646985 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb85c06f5a0 0x7fb85c071ab0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb8680096d0 tx=0x7fb868009240 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:29.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.708+0000 7fb877e00640 1 -- 172.21.15.86:0/3543646985 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb870016090 con 0x7fb8800c47f0 2024-04-25T14:05:29.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.876+0000 7fb885e00640 1 -- 172.21.15.86:0/3543646985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb880076bf0 con 0x7fb8800c47f0 2024-04-25T14:05:29.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.876+0000 7fb877e00640 1 -- 172.21.15.86:0/3543646985 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+13669 (secure 0 0 0) 0x7fb8700f5800 con 0x7fb8800c47f0 2024-04-25T14:05:29.878 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:29.878 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":42,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:05:28.502556+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6835/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:0/2073988773":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6833/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6832/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/3558294881":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/1843278167":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6837/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:0/2602032945":"2024-04-26T14:05:21.460855+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:05:29.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.876+0000 7fb885e00640 1 -- 172.21.15.86:0/3543646985 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb85c06f5a0 msgr2=0x7fb85c071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:29.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.876+0000 7fb885e00640 1 --2- 172.21.15.86:0/3543646985 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb85c06f5a0 0x7fb85c071ab0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb8680096d0 tx=0x7fb868009240 comp rx=0 tx=0).stop 2024-04-25T14:05:29.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.876+0000 7fb885e00640 1 -- 172.21.15.86:0/3543646985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8800c47f0 msgr2=0x7fb880073e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:29.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.876+0000 7fb885e00640 1 --2- 172.21.15.86:0/3543646985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8800c47f0 0x7fb880073e60 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb87001a0d0 tx=0x7fb870019240 comp rx=0 tx=0).stop 2024-04-25T14:05:29.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.876+0000 7fb885e00640 1 -- 172.21.15.86:0/3543646985 shutdown_connections 2024-04-25T14:05:29.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.876+0000 7fb885e00640 1 --2- 172.21.15.86:0/3543646985 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb85c06f5a0 0x7fb85c071ab0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:29.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.876+0000 7fb885e00640 1 --2- 172.21.15.86:0/3543646985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb880168360 0x7fb880164c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:29.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.876+0000 7fb885e00640 1 --2- 172.21.15.86:0/3543646985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8800ce570 0x7fb880154d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:29.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.876+0000 7fb885e00640 1 --2- 172.21.15.86:0/3543646985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8800c47f0 0x7fb880073e60 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:29.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.876+0000 7fb885e00640 1 -- 172.21.15.86:0/3543646985 >> 172.21.15.86:0/3543646985 conn(0x7fb8800b4650 msgr2=0x7fb8800b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:29.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.876+0000 7fb885e00640 1 -- 172.21.15.86:0/3543646985 shutdown_connections 2024-04-25T14:05:29.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:29.876+0000 7fb885e00640 1 -- 172.21.15.86:0/3543646985 wait complete. 2024-04-25T14:05:29.900 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:05:29.900 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:05:30.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.048+0000 7f75ec000640 1 Processor -- start 2024-04-25T14:05:30.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.048+0000 7f75ec000640 1 -- start start 2024-04-25T14:05:30.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.048+0000 7f75ec000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f75e41651d0 0x7f75e41675f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:30.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.048+0000 7f75ec000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75e40b0a60 con 0x7f75e406fd50 2024-04-25T14:05:30.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.048+0000 7f75ec000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75e40ae840 con 0x7f75e415c2a0 2024-04-25T14:05:30.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.048+0000 7f75ec000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f75e406ef90 con 0x7f75e41651d0 2024-04-25T14:05:30.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.048+0000 7f75ea000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f75e41651d0 0x7f75e41675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:30.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.048+0000 7f75ea000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f75e41651d0 0x7f75e41675f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51938/0 (socket says 172.21.15.86:51938) 2024-04-25T14:05:30.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.048+0000 7f75ea000640 1 -- 172.21.15.86:0/3677803495 learned_addr learned my addr 172.21.15.86:0/3677803495 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:30.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.048+0000 7f75e8c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f75e406fd50 0x7f75e4070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36646/0 (socket says 172.21.15.86:36646) 2024-04-25T14:05:30.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.052+0000 7f75ea000640 1 -- 172.21.15.86:0/3677803495 >> v1:172.21.15.155:6789/0 conn(0x7f75e415c2a0 legacy=0x7f75e40ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:30.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.052+0000 7f75ea000640 1 -- 172.21.15.86:0/3677803495 >> v1:172.21.15.86:6789/0 conn(0x7f75e406fd50 legacy=0x7f75e4070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:30.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.052+0000 7f75ea000640 1 -- 172.21.15.86:0/3677803495 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e406f270 con 0x7f75e41651d0 2024-04-25T14:05:30.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.052+0000 7f75ea000640 1 --2- 172.21.15.86:0/3677803495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75e41651d0 0x7f75e41675f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f75dc009e50 tx=0x7f75dc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a23bd993875526 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:30.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.052+0000 7f75e3e00640 1 -- 172.21.15.86:0/3677803495 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc0a57b0 con 0x7f75e41651d0 2024-04-25T14:05:30.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.052+0000 7f75e3e00640 1 -- 172.21.15.86:0/3677803495 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75dc0a5950 con 0x7f75e41651d0 2024-04-25T14:05:30.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.052+0000 7f75e3e00640 1 -- 172.21.15.86:0/3677803495 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc0a5d20 con 0x7f75e41651d0 2024-04-25T14:05:30.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.056+0000 7f75ec000640 1 -- 172.21.15.86:0/3677803495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75e41651d0 msgr2=0x7f75e41675f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:30.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.056+0000 7f75ec000640 1 --2- 172.21.15.86:0/3677803495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75e41651d0 0x7f75e41675f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f75dc009e50 tx=0x7f75dc01c970 comp rx=0 tx=0).stop 2024-04-25T14:05:30.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.056+0000 7f75ec000640 1 -- 172.21.15.86:0/3677803495 shutdown_connections 2024-04-25T14:05:30.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.056+0000 7f75ec000640 1 --2- 172.21.15.86:0/3677803495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75e41651d0 0x7f75e41675f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:30.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.056+0000 7f75ec000640 1 -- 172.21.15.86:0/3677803495 >> 172.21.15.86:0/3677803495 conn(0x7f75e40b4650 msgr2=0x7f75e40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:30.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.056+0000 7f75ec000640 1 -- 172.21.15.86:0/3677803495 shutdown_connections 2024-04-25T14:05:30.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.056+0000 7f75ec000640 1 -- 172.21.15.86:0/3677803495 wait complete. 2024-04-25T14:05:30.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.056+0000 7f75ec000640 1 Processor -- start 2024-04-25T14:05:30.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.056+0000 7f75ec000640 1 -- start start 2024-04-25T14:05:30.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.060+0000 7f75ec000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75e406fd50 0x7f75e40742a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:30.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.060+0000 7f75ec000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75e415c2a0 0x7f75e40747e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:30.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.060+0000 7f75ec000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75e41651d0 0x7f75e4166640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:30.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.060+0000 7f75ec000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f75e415c830 con 0x7f75e41651d0 2024-04-25T14:05:30.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.060+0000 7f75ec000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f75e415c6b0 con 0x7f75e415c2a0 2024-04-25T14:05:30.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.060+0000 7f75ec000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f75e416ac60 con 0x7f75e406fd50 2024-04-25T14:05:30.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.060+0000 7f75ea000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75e41651d0 0x7f75e4166640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:30.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.060+0000 7f75ea000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75e41651d0 0x7f75e4166640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51950/0 (socket says 172.21.15.86:51950) 2024-04-25T14:05:30.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.060+0000 7f75ea000640 1 -- 172.21.15.86:0/592409874 learned_addr learned my addr 172.21.15.86:0/592409874 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:30.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.060+0000 7f75e8c00640 1 --2- 172.21.15.86:0/592409874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75e415c2a0 0x7f75e40747e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:30.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.060+0000 7f75e9600640 1 --2- 172.21.15.86:0/592409874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75e406fd50 0x7f75e40742a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:30.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.060+0000 7f75ea000640 1 -- 172.21.15.86:0/592409874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75e406fd50 msgr2=0x7f75e40742a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:30.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.060+0000 7f75ea000640 1 --2- 172.21.15.86:0/592409874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75e406fd50 0x7f75e40742a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:30.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.060+0000 7f75ea000640 1 -- 172.21.15.86:0/592409874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75e415c2a0 msgr2=0x7f75e40747e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:30.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.060+0000 7f75ea000640 1 --2- 172.21.15.86:0/592409874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75e415c2a0 0x7f75e40747e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:30.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.060+0000 7f75ea000640 1 -- 172.21.15.86:0/592409874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e41655e0 con 0x7f75e41651d0 2024-04-25T14:05:30.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.060+0000 7f75e9600640 1 --2- 172.21.15.86:0/592409874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75e406fd50 0x7f75e40742a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:30.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.060+0000 7f75e8c00640 1 --2- 172.21.15.86:0/592409874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75e415c2a0 0x7f75e40747e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:30.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.060+0000 7f75ea000640 1 --2- 172.21.15.86:0/592409874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75e41651d0 0x7f75e4166640 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f75dc009e50 tx=0x7f75dc019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:30.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.060+0000 7f75e1600640 1 -- 172.21.15.86:0/592409874 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc0b8470 con 0x7f75e41651d0 2024-04-25T14:05:30.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.060+0000 7f75e1600640 1 -- 172.21.15.86:0/592409874 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75dc0b8630 con 0x7f75e41651d0 2024-04-25T14:05:30.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.060+0000 7f75e1600640 1 -- 172.21.15.86:0/592409874 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc0b8a20 con 0x7f75e41651d0 2024-04-25T14:05:30.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.060+0000 7f75ec000640 1 -- 172.21.15.86:0/592409874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75e4165880 con 0x7f75e41651d0 2024-04-25T14:05:30.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.060+0000 7f75ec000640 1 -- 172.21.15.86:0/592409874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75e4165e40 con 0x7f75e41651d0 2024-04-25T14:05:30.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.060+0000 7f75ec000640 1 -- 172.21.15.86:0/592409874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75e4069d00 con 0x7f75e41651d0 2024-04-25T14:05:30.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.060+0000 7f75e1600640 1 -- 172.21.15.86:0/592409874 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f75dc0c0020 con 0x7f75e41651d0 2024-04-25T14:05:30.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.064+0000 7f75e1600640 1 --2- 172.21.15.86:0/592409874 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f75c406f5a0 0x7f75c4071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:30.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.064+0000 7f75e1600640 1 -- 172.21.15.86:0/592409874 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5593+0+0 (secure 0 0 0) 0x7f75dc12a310 con 0x7f75e41651d0 2024-04-25T14:05:30.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.064+0000 7f75e9600640 1 --2- 172.21.15.86:0/592409874 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f75c406f5a0 0x7f75c4071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:30.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.064+0000 7f75e9600640 1 --2- 172.21.15.86:0/592409874 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f75c406f5a0 0x7f75c4071ab0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f75e40b4200 tx=0x7f75d8009040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:30.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.068+0000 7f75e1600640 1 -- 172.21.15.86:0/592409874 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f75dc016620 con 0x7f75e41651d0 2024-04-25T14:05:30.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.224+0000 7f75ec000640 1 -- 172.21.15.86:0/592409874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f75e416aa40 con 0x7f75e41651d0 2024-04-25T14:05:30.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.228+0000 7f75e1600640 1 -- 172.21.15.86:0/592409874 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) ==== 93+0+10 (secure 0 0 0) 0x7f75dc0f70c0 con 0x7f75e41651d0 2024-04-25T14:05:30.229 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:05:30.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.228+0000 7f75ec000640 1 -- 172.21.15.86:0/592409874 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f75c406f5a0 msgr2=0x7f75c4071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:30.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.228+0000 7f75ec000640 1 --2- 172.21.15.86:0/592409874 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f75c406f5a0 0x7f75c4071ab0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f75e40b4200 tx=0x7f75d8009040 comp rx=0 tx=0).stop 2024-04-25T14:05:30.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.228+0000 7f75ec000640 1 -- 172.21.15.86:0/592409874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75e41651d0 msgr2=0x7f75e4166640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:30.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.228+0000 7f75ec000640 1 --2- 172.21.15.86:0/592409874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75e41651d0 0x7f75e4166640 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f75dc009e50 tx=0x7f75dc019710 comp rx=0 tx=0).stop 2024-04-25T14:05:30.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.228+0000 7f75ec000640 1 -- 172.21.15.86:0/592409874 shutdown_connections 2024-04-25T14:05:30.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.228+0000 7f75ec000640 1 --2- 172.21.15.86:0/592409874 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f75c406f5a0 0x7f75c4071ab0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:30.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.228+0000 7f75ec000640 1 --2- 172.21.15.86:0/592409874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75e41651d0 0x7f75e4166640 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:30.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.228+0000 7f75ec000640 1 --2- 172.21.15.86:0/592409874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75e415c2a0 0x7f75e40747e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:30.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.228+0000 7f75ec000640 1 --2- 172.21.15.86:0/592409874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75e406fd50 0x7f75e40742a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:30.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.228+0000 7f75ec000640 1 -- 172.21.15.86:0/592409874 >> 172.21.15.86:0/592409874 conn(0x7f75e40b4650 msgr2=0x7f75e40af450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:30.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.228+0000 7f75ec000640 1 -- 172.21.15.86:0/592409874 shutdown_connections 2024-04-25T14:05:30.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.228+0000 7f75ec000640 1 -- 172.21.15.86:0/592409874 wait complete. 2024-04-25T14:05:30.250 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T14:05:30.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.396+0000 7f6256e00640 1 Processor -- start 2024-04-25T14:05:30.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.396+0000 7f6256e00640 1 -- start start 2024-04-25T14:05:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.396+0000 7f6256e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f62501610d0 0x7f62500b7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.396+0000 7f6256e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f62500b2250 0x7f62500b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.396+0000 7f6256e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62500b3750 con 0x7f62500b2ba0 2024-04-25T14:05:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.396+0000 7f6256e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f62500b7f70 con 0x7f62500b2250 2024-04-25T14:05:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.396+0000 7f6256e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f6250161620 con 0x7f62501610d0 2024-04-25T14:05:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.396+0000 7f6247400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f62500b2250 0x7f62500b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.396+0000 7f6247400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f62500b2250 0x7f62500b2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51954/0 (socket says 172.21.15.86:51954) 2024-04-25T14:05:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.396+0000 7f6247400640 1 -- 172.21.15.86:0/648259433 learned_addr learned my addr 172.21.15.86:0/648259433 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.396+0000 7f6247400640 1 -- 172.21.15.86:0/648259433 >> v2:172.21.15.155:3300/0 conn(0x7f62501610d0 msgr2=0x7f62500b7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.396+0000 7f6247400640 1 --2- 172.21.15.86:0/648259433 >> v2:172.21.15.155:3300/0 conn(0x7f62501610d0 0x7f62500b7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:30.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.396+0000 7f6247400640 1 -- 172.21.15.86:0/648259433 >> v1:172.21.15.155:6789/0 conn(0x7f62500b2ba0 legacy=0x7f62500d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:30.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.396+0000 7f6247400640 1 -- 172.21.15.86:0/648259433 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62500ce7b0 con 0x7f62500b2250 2024-04-25T14:05:30.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.400+0000 7f6247400640 1 --2- 172.21.15.86:0/648259433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62500b2250 0x7f62500b2660 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f62500b52f0 tx=0x7f623801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c5613279cc59f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:30.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.400+0000 7f624f400640 1 -- 172.21.15.86:0/648259433 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62380b0070 con 0x7f62500b2250 2024-04-25T14:05:30.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.400+0000 7f624f400640 1 -- 172.21.15.86:0/648259433 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62380ab600 con 0x7f62500b2250 2024-04-25T14:05:30.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.400+0000 7f624f400640 1 -- 172.21.15.86:0/648259433 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62380ab9d0 con 0x7f62500b2250 2024-04-25T14:05:30.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.400+0000 7f6256e00640 1 -- 172.21.15.86:0/648259433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62500b2250 msgr2=0x7f62500b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:30.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.400+0000 7f6256e00640 1 --2- 172.21.15.86:0/648259433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62500b2250 0x7f62500b2660 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f62500b52f0 tx=0x7f623801c420 comp rx=0 tx=0).stop 2024-04-25T14:05:30.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.400+0000 7f6256e00640 1 -- 172.21.15.86:0/648259433 shutdown_connections 2024-04-25T14:05:30.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.400+0000 7f6256e00640 1 --2- 172.21.15.86:0/648259433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62500b2250 0x7f62500b2660 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:30.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.400+0000 7f6256e00640 1 --2- 172.21.15.86:0/648259433 >> v2:172.21.15.155:3300/0 conn(0x7f62501610d0 0x7f62500b7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:30.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.400+0000 7f6256e00640 1 -- 172.21.15.86:0/648259433 >> 172.21.15.86:0/648259433 conn(0x7f625009ba30 msgr2=0x7f62500b7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:30.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.400+0000 7f6256e00640 1 -- 172.21.15.86:0/648259433 shutdown_connections 2024-04-25T14:05:30.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.400+0000 7f6256e00640 1 -- 172.21.15.86:0/648259433 wait complete. 2024-04-25T14:05:30.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.404+0000 7f6256e00640 1 Processor -- start 2024-04-25T14:05:30.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.404+0000 7f6256e00640 1 -- start start 2024-04-25T14:05:30.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.404+0000 7f6256e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62500b2250 0x7f62501563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:30.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.404+0000 7f6256e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62500b2ba0 0x7f62501fad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:30.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.404+0000 7f6256e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62501610d0 0x7f62500b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:30.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.404+0000 7f6256e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62500d2280 con 0x7f62500b2ba0 2024-04-25T14:05:30.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.404+0000 7f6256e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f62500d2100 con 0x7f62501610d0 2024-04-25T14:05:30.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.404+0000 7f6256e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f62500cf610 con 0x7f62500b2250 2024-04-25T14:05:30.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.404+0000 7f6247400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62500b2ba0 0x7f62501fad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:30.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.404+0000 7f6247400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62500b2ba0 0x7f62501fad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51968/0 (socket says 172.21.15.86:51968) 2024-04-25T14:05:30.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.404+0000 7f6247400640 1 -- 172.21.15.86:0/2253109312 learned_addr learned my addr 172.21.15.86:0/2253109312 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:30.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.408+0000 7f624fe00640 1 --2- 172.21.15.86:0/2253109312 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62500b2250 0x7f62501563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:30.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.408+0000 7f6254e00640 1 --2- 172.21.15.86:0/2253109312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62501610d0 0x7f62500b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:30.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.408+0000 7f6247400640 1 -- 172.21.15.86:0/2253109312 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62500b2250 msgr2=0x7f62501563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:30.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.408+0000 7f6247400640 1 --2- 172.21.15.86:0/2253109312 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62500b2250 0x7f62501563e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:30.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.408+0000 7f6247400640 1 -- 172.21.15.86:0/2253109312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62501610d0 msgr2=0x7f62500b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:30.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.408+0000 7f6247400640 1 --2- 172.21.15.86:0/2253109312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62501610d0 0x7f62500b0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:30.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.408+0000 7f6247400640 1 -- 172.21.15.86:0/2253109312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62501fb2b0 con 0x7f62500b2ba0 2024-04-25T14:05:30.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.408+0000 7f6247400640 1 --2- 172.21.15.86:0/2253109312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62500b2ba0 0x7f62501fad70 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f623801aad0 tx=0x7f623801ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:30.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.408+0000 7f624cc00640 1 -- 172.21.15.86:0/2253109312 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62380b0040 con 0x7f62500b2ba0 2024-04-25T14:05:30.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.408+0000 7f624cc00640 1 -- 172.21.15.86:0/2253109312 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62380bc070 con 0x7f62500b2ba0 2024-04-25T14:05:30.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.408+0000 7f624cc00640 1 -- 172.21.15.86:0/2253109312 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62380b6880 con 0x7f62500b2ba0 2024-04-25T14:05:30.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.408+0000 7f6256e00640 1 -- 172.21.15.86:0/2253109312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62500af750 con 0x7f62500b2ba0 2024-04-25T14:05:30.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.408+0000 7f6256e00640 1 -- 172.21.15.86:0/2253109312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62500afa30 con 0x7f62500b2ba0 2024-04-25T14:05:30.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.408+0000 7f6256e00640 1 -- 172.21.15.86:0/2253109312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62500a05a0 con 0x7f62500b2ba0 2024-04-25T14:05:30.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.408+0000 7f624cc00640 1 -- 172.21.15.86:0/2253109312 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f62380b6a20 con 0x7f62500b2ba0 2024-04-25T14:05:30.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.412+0000 7f624cc00640 1 --2- 172.21.15.86:0/2253109312 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f622c06f5a0 0x7f622c071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:30.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.412+0000 7f624cc00640 1 -- 172.21.15.86:0/2253109312 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5593+0+0 (secure 0 0 0) 0x7f6238129b10 con 0x7f62500b2ba0 2024-04-25T14:05:30.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.412+0000 7f624fe00640 1 --2- 172.21.15.86:0/2253109312 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f622c06f5a0 0x7f622c071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:30.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.412+0000 7f624fe00640 1 --2- 172.21.15.86:0/2253109312 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f622c06f5a0 0x7f622c071ab0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6240009f20 tx=0x7f62400094f0 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:30.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.416+0000 7f624cc00640 1 -- 172.21.15.86:0/2253109312 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f62380160d0 con 0x7f62500b2ba0 2024-04-25T14:05:30.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.576+0000 7f6256e00640 1 -- 172.21.15.86:0/2253109312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f62500c91d0 con 0x7f62500b2ba0 2024-04-25T14:05:30.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.576+0000 7f624cc00640 1 -- 172.21.15.86:0/2253109312 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v42) ==== 84+0+65 (secure 0 0 0) 0x7f62380f6840 con 0x7f62500b2ba0 2024-04-25T14:05:30.578 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:30.578 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-04-25T14:05:30.578 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:30.578 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-04-25T14:05:30.578 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-04-25T14:05:30.578 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:30.578 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-04-25T14:05:30.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.576+0000 7f6256e00640 1 -- 172.21.15.86:0/2253109312 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f622c06f5a0 msgr2=0x7f622c071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:30.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.576+0000 7f6256e00640 1 --2- 172.21.15.86:0/2253109312 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f622c06f5a0 0x7f622c071ab0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6240009f20 tx=0x7f62400094f0 comp rx=0 tx=0).stop 2024-04-25T14:05:30.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.576+0000 7f6256e00640 1 -- 172.21.15.86:0/2253109312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62500b2ba0 msgr2=0x7f62501fad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:30.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.576+0000 7f6256e00640 1 --2- 172.21.15.86:0/2253109312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62500b2ba0 0x7f62501fad70 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f623801aad0 tx=0x7f623801ac60 comp rx=0 tx=0).stop 2024-04-25T14:05:30.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.576+0000 7f6256e00640 1 -- 172.21.15.86:0/2253109312 shutdown_connections 2024-04-25T14:05:30.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.576+0000 7f6256e00640 1 --2- 172.21.15.86:0/2253109312 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f622c06f5a0 0x7f622c071ab0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:30.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.576+0000 7f6256e00640 1 --2- 172.21.15.86:0/2253109312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62501610d0 0x7f62500b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:30.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.576+0000 7f6256e00640 1 --2- 172.21.15.86:0/2253109312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62500b2ba0 0x7f62501fad70 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:30.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.576+0000 7f6256e00640 1 --2- 172.21.15.86:0/2253109312 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62500b2250 0x7f62501563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:30.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.576+0000 7f6256e00640 1 -- 172.21.15.86:0/2253109312 >> 172.21.15.86:0/2253109312 conn(0x7f625009ba30 msgr2=0x7f62500b7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:30.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.576+0000 7f6256e00640 1 -- 172.21.15.86:0/2253109312 shutdown_connections 2024-04-25T14:05:30.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.576+0000 7f6256e00640 1 -- 172.21.15.86:0/2253109312 wait complete. 2024-04-25T14:05:30.599 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'fs1' 2024-04-25T14:05:30.599 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create fs1_metadata --pg_num_min 64 2024-04-25T14:05:30.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.748+0000 7feed9c00640 1 Processor -- start 2024-04-25T14:05:30.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.748+0000 7feed9c00640 1 -- start start 2024-04-25T14:05:30.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.748+0000 7feed9c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feed40743c0 0x7feed41605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:30.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.748+0000 7feed9c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed415ce60 con 0x7feed415ca50 2024-04-25T14:05:30.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.748+0000 7feed9c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed4072b90 con 0x7feed40747d0 2024-04-25T14:05:30.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.748+0000 7feed9c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7feed4072e70 con 0x7feed40743c0 2024-04-25T14:05:30.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.748+0000 7feed2a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7feed415ca50 0x7feed4073950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36652/0 (socket says 172.21.15.86:36652) 2024-04-25T14:05:30.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.748+0000 7feed2a00640 1 -- 172.21.15.86:0/1835221101 learned_addr learned my addr 172.21.15.86:0/1835221101 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:30.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.748+0000 7feeca000640 1 --2- 172.21.15.86:0/1835221101 >> v2:172.21.15.86:3300/0 conn(0x7feed40743c0 0x7feed41605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:30.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.748+0000 7feed2000640 1 -- 172.21.15.86:0/1835221101 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4063377067 0 0) 0x7feed415ce60 con 0x7feed415ca50 2024-04-25T14:05:30.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.748+0000 7feed2000640 1 -- 172.21.15.86:0/1835221101 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feeb8003530 con 0x7feed415ca50 2024-04-25T14:05:30.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.748+0000 7feeca000640 1 -- 172.21.15.86:0/1835221101 >> v1:172.21.15.155:6789/0 conn(0x7feed40747d0 legacy=0x7feed406e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:30.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.748+0000 7feeca000640 1 -- 172.21.15.86:0/1835221101 >> v1:172.21.15.86:6789/0 conn(0x7feed415ca50 legacy=0x7feed4073950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:30.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.752+0000 7feeca000640 1 -- 172.21.15.86:0/1835221101 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed4073150 con 0x7feed40743c0 2024-04-25T14:05:30.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.752+0000 7feeca000640 1 --2- 172.21.15.86:0/1835221101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feed40743c0 0x7feed41605f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7feebc0097c0 tx=0x7feebc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34e3d0b5ddd4cf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:30.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.752+0000 7feed2000640 1 -- 172.21.15.86:0/1835221101 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc0b0070 con 0x7feed40743c0 2024-04-25T14:05:30.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.752+0000 7feed2000640 1 -- 172.21.15.86:0/1835221101 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feebc0ab600 con 0x7feed40743c0 2024-04-25T14:05:30.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.752+0000 7feed2000640 1 -- 172.21.15.86:0/1835221101 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc0ab9d0 con 0x7feed40743c0 2024-04-25T14:05:30.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.752+0000 7feed9c00640 1 -- 172.21.15.86:0/1835221101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feed40743c0 msgr2=0x7feed41605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:30.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.752+0000 7feed9c00640 1 --2- 172.21.15.86:0/1835221101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feed40743c0 0x7feed41605f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7feebc0097c0 tx=0x7feebc01c460 comp rx=0 tx=0).stop 2024-04-25T14:05:30.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.752+0000 7feed9c00640 1 -- 172.21.15.86:0/1835221101 shutdown_connections 2024-04-25T14:05:30.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.752+0000 7feed9c00640 1 --2- 172.21.15.86:0/1835221101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feed40743c0 0x7feed41605f0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:30.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.752+0000 7feed9c00640 1 -- 172.21.15.86:0/1835221101 >> 172.21.15.86:0/1835221101 conn(0x7feed40b4a40 msgr2=0x7feed40b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:30.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.756+0000 7feed9c00640 1 -- 172.21.15.86:0/1835221101 shutdown_connections 2024-04-25T14:05:30.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.756+0000 7feed9c00640 1 -- 172.21.15.86:0/1835221101 wait complete. 2024-04-25T14:05:30.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.756+0000 7feed9c00640 1 Processor -- start 2024-04-25T14:05:30.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.756+0000 7feed9c00640 1 -- start start 2024-04-25T14:05:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.756+0000 7feed9c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feed40743c0 0x7feed4156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.756+0000 7feed9c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feed40747d0 0x7feed41464c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.756+0000 7feed9c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feed415ca50 0x7feed41540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.756+0000 7feed9c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feed416b540 con 0x7feed415ca50 2024-04-25T14:05:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.756+0000 7feed9c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7feed416b3c0 con 0x7feed40747d0 2024-04-25T14:05:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.756+0000 7feed9c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7feed416b6c0 con 0x7feed40743c0 2024-04-25T14:05:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.760+0000 7feed3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feed415ca50 0x7feed41540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.760+0000 7feed3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feed415ca50 0x7feed41540f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51980/0 (socket says 172.21.15.86:51980) 2024-04-25T14:05:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.760+0000 7feed3400640 1 -- 172.21.15.86:0/2262186440 learned_addr learned my addr 172.21.15.86:0/2262186440 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.760+0000 7feeca000640 1 --2- 172.21.15.86:0/2262186440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feed40747d0 0x7feed41464c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.760+0000 7feed2a00640 1 --2- 172.21.15.86:0/2262186440 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feed40743c0 0x7feed4156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.760+0000 7feed3400640 1 -- 172.21.15.86:0/2262186440 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feed40743c0 msgr2=0x7feed4156cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.760+0000 7feed3400640 1 --2- 172.21.15.86:0/2262186440 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feed40743c0 0x7feed4156cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.760+0000 7feed3400640 1 -- 172.21.15.86:0/2262186440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feed40747d0 msgr2=0x7feed41464c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.760+0000 7feed3400640 1 --2- 172.21.15.86:0/2262186440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feed40747d0 0x7feed41464c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.760+0000 7feed3400640 1 -- 172.21.15.86:0/2262186440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed4165d80 con 0x7feed415ca50 2024-04-25T14:05:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.760+0000 7feed3400640 1 --2- 172.21.15.86:0/2262186440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feed415ca50 0x7feed41540f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7feecc00ba10 tx=0x7feecc00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:30.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.760+0000 7feecb400640 1 -- 172.21.15.86:0/2262186440 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc019070 con 0x7feed415ca50 2024-04-25T14:05:30.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.760+0000 7feecb400640 1 -- 172.21.15.86:0/2262186440 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feecc00cb40 con 0x7feed415ca50 2024-04-25T14:05:30.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.760+0000 7feed9c00640 1 -- 172.21.15.86:0/2262186440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed4157210 con 0x7feed415ca50 2024-04-25T14:05:30.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.760+0000 7feecb400640 1 -- 172.21.15.86:0/2262186440 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc014440 con 0x7feed415ca50 2024-04-25T14:05:30.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.760+0000 7feed9c00640 1 -- 172.21.15.86:0/2262186440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed4167b50 con 0x7feed415ca50 2024-04-25T14:05:30.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.760+0000 7feed9c00640 1 -- 172.21.15.86:0/2262186440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feed4050750 con 0x7feed415ca50 2024-04-25T14:05:30.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.760+0000 7feecb400640 1 -- 172.21.15.86:0/2262186440 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7feecc01c070 con 0x7feed415ca50 2024-04-25T14:05:30.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.764+0000 7feecb400640 1 --2- 172.21.15.86:0/2262186440 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7feeb006f5a0 0x7feeb0071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:30.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.764+0000 7feecb400640 1 -- 172.21.15.86:0/2262186440 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5593+0+0 (secure 0 0 0) 0x7feecc08ef40 con 0x7feed415ca50 2024-04-25T14:05:30.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.764+0000 7feed2a00640 1 --2- 172.21.15.86:0/2262186440 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7feeb006f5a0 0x7feeb0071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:30.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.764+0000 7feed2a00640 1 --2- 172.21.15.86:0/2262186440 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7feeb006f5a0 0x7feeb0071ab0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feec4005890 tx=0x7feec4009040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:30.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.768+0000 7feecb400640 1 -- 172.21.15.86:0/2262186440 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7feed4165d80 con 0x7feed415ca50 2024-04-25T14:05:30.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:30.924+0000 7feed9c00640 1 -- 172.21.15.86:0/2262186440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "fs1_metadata", "pg_num_min": 64} v 0) -- 0x7feed4167600 con 0x7feed415ca50 2024-04-25T14:05:31.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.524+0000 7feecb400640 1 -- 172.21.15.86:0/2262186440 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "fs1_metadata", "pg_num_min": 64}]=0 pool 'fs1_metadata' created v43) ==== 132+0+0 (secure 0 0 0) 0x7feecc05be20 con 0x7feed415ca50 2024-04-25T14:05:31.528 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'fs1_metadata' created 2024-04-25T14:05:31.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.528+0000 7feed9c00640 1 -- 172.21.15.86:0/2262186440 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7feeb006f5a0 msgr2=0x7feeb0071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:31.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.528+0000 7feed9c00640 1 --2- 172.21.15.86:0/2262186440 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7feeb006f5a0 0x7feeb0071ab0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feec4005890 tx=0x7feec4009040 comp rx=0 tx=0).stop 2024-04-25T14:05:31.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.528+0000 7feed9c00640 1 -- 172.21.15.86:0/2262186440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feed415ca50 msgr2=0x7feed41540f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:31.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.528+0000 7feed9c00640 1 --2- 172.21.15.86:0/2262186440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feed415ca50 0x7feed41540f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7feecc00ba10 tx=0x7feecc00bed0 comp rx=0 tx=0).stop 2024-04-25T14:05:31.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.528+0000 7feed9c00640 1 -- 172.21.15.86:0/2262186440 shutdown_connections 2024-04-25T14:05:31.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.528+0000 7feed9c00640 1 --2- 172.21.15.86:0/2262186440 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7feeb006f5a0 0x7feeb0071ab0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:31.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.528+0000 7feed9c00640 1 --2- 172.21.15.86:0/2262186440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feed415ca50 0x7feed41540f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:31.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.528+0000 7feed9c00640 1 --2- 172.21.15.86:0/2262186440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feed40747d0 0x7feed41464c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:31.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.528+0000 7feed9c00640 1 --2- 172.21.15.86:0/2262186440 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feed40743c0 0x7feed4156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:31.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.528+0000 7feed9c00640 1 -- 172.21.15.86:0/2262186440 >> 172.21.15.86:0/2262186440 conn(0x7feed40b4a40 msgr2=0x7feed4073850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:31.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.528+0000 7feed9c00640 1 -- 172.21.15.86:0/2262186440 shutdown_connections 2024-04-25T14:05:31.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.528+0000 7feed9c00640 1 -- 172.21.15.86:0/2262186440 wait complete. 2024-04-25T14:05:31.557 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create fs1_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T14:05:31.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.708+0000 7f5e9f200640 1 Processor -- start 2024-04-25T14:05:31.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.708+0000 7f5e9f200640 1 -- start start 2024-04-25T14:05:31.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.712+0000 7f5e9f200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e98070330 0x7f5e980d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:31.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.712+0000 7f5e9f200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5e980ce790 0x7f5e9806f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:31.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.712+0000 7f5e9f200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e980d2030 con 0x7f5e980d1c20 2024-04-25T14:05:31.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.712+0000 7f5e9f200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5e9815e5c0 con 0x7f5e980ce790 2024-04-25T14:05:31.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.712+0000 7f5e9f200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5e9815e8a0 con 0x7f5e98070330 2024-04-25T14:05:31.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.712+0000 7f5e9d200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5e980ce790 0x7f5e9806f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:31.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.712+0000 7f5e9d200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5e980ce790 0x7f5e9806f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51996/0 (socket says 172.21.15.86:51996) 2024-04-25T14:05:31.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.712+0000 7f5e9d200640 1 -- 172.21.15.86:0/3111351226 learned_addr learned my addr 172.21.15.86:0/3111351226 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:31.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.712+0000 7f5e8f400640 1 --1- 172.21.15.86:0/3111351226 >> v1:172.21.15.86:6789/0 conn(0x7f5e980d1c20 0x7f5e9806ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36660/0 (socket says 172.21.15.86:36660) 2024-04-25T14:05:31.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.712+0000 7f5e97e00640 1 --2- 172.21.15.86:0/3111351226 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e98070330 0x7f5e980d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:31.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.712+0000 7f5e9d200640 1 -- 172.21.15.86:0/3111351226 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e98070330 msgr2=0x7f5e980d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:31.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.712+0000 7f5e9d200640 1 --2- 172.21.15.86:0/3111351226 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e98070330 0x7f5e980d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:31.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.712+0000 7f5e9d200640 1 -- 172.21.15.86:0/3111351226 >> v1:172.21.15.86:6789/0 conn(0x7f5e980d1c20 legacy=0x7f5e9806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:31.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.712+0000 7f5e9d200640 1 -- 172.21.15.86:0/3111351226 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e9815eb80 con 0x7f5e980ce790 2024-04-25T14:05:31.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.712+0000 7f5e97400640 1 -- 172.21.15.86:0/3111351226 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3207692302 0 0) 0x7f5e80001120 con 0x7f5e980d1c20 2024-04-25T14:05:31.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.716+0000 7f5e9d200640 1 --2- 172.21.15.86:0/3111351226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e980ce790 0x7f5e9806f440 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5e90009e80 tx=0x7f5e9001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99d5044878efc79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:31.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.716+0000 7f5e97400640 1 -- 172.21.15.86:0/3111351226 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e900a5640 con 0x7f5e980ce790 2024-04-25T14:05:31.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.716+0000 7f5e97400640 1 -- 172.21.15.86:0/3111351226 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e900a57e0 con 0x7f5e980ce790 2024-04-25T14:05:31.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.716+0000 7f5e97400640 1 -- 172.21.15.86:0/3111351226 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e900a5bb0 con 0x7f5e980ce790 2024-04-25T14:05:31.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.716+0000 7f5e9f200640 1 -- 172.21.15.86:0/3111351226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e980ce790 msgr2=0x7f5e9806f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:31.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.716+0000 7f5e9f200640 1 --2- 172.21.15.86:0/3111351226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e980ce790 0x7f5e9806f440 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5e90009e80 tx=0x7f5e9001c980 comp rx=0 tx=0).stop 2024-04-25T14:05:31.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.716+0000 7f5e9f200640 1 -- 172.21.15.86:0/3111351226 shutdown_connections 2024-04-25T14:05:31.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.716+0000 7f5e9f200640 1 --2- 172.21.15.86:0/3111351226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e980ce790 0x7f5e9806f440 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:31.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.716+0000 7f5e9f200640 1 --2- 172.21.15.86:0/3111351226 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e98070330 0x7f5e980d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:31.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.716+0000 7f5e9f200640 1 -- 172.21.15.86:0/3111351226 >> 172.21.15.86:0/3111351226 conn(0x7f5e980b4650 msgr2=0x7f5e980b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:31.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.716+0000 7f5e9f200640 1 -- 172.21.15.86:0/3111351226 shutdown_connections 2024-04-25T14:05:31.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.716+0000 7f5e9f200640 1 -- 172.21.15.86:0/3111351226 wait complete. 2024-04-25T14:05:31.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.716+0000 7f5e9f200640 1 Processor -- start 2024-04-25T14:05:31.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.720+0000 7f5e9f200640 1 -- start start 2024-04-25T14:05:31.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.720+0000 7f5e9f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e98070330 0x7f5e98167e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:31.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.720+0000 7f5e9f200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e980ce790 0x7f5e9815bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:31.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.720+0000 7f5e9f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e980d1c20 0x7f5e9815c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:31.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.720+0000 7f5e9f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e980708c0 con 0x7f5e98070330 2024-04-25T14:05:31.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.720+0000 7f5e9f200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5e98070740 con 0x7f5e980d1c20 2024-04-25T14:05:31.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.720+0000 7f5e9f200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5e9816afc0 con 0x7f5e980ce790 2024-04-25T14:05:31.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.720+0000 7f5e97e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e98070330 0x7f5e98167e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:31.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.720+0000 7f5e97e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e98070330 0x7f5e98167e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52006/0 (socket says 172.21.15.86:52006) 2024-04-25T14:05:31.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.720+0000 7f5e97e00640 1 -- 172.21.15.86:0/2827861615 learned_addr learned my addr 172.21.15.86:0/2827861615 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:31.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.720+0000 7f5e8f400640 1 --2- 172.21.15.86:0/2827861615 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e980ce790 0x7f5e9815bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:31.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.720+0000 7f5e9d200640 1 --2- 172.21.15.86:0/2827861615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e980d1c20 0x7f5e9815c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:31.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.720+0000 7f5e97e00640 1 -- 172.21.15.86:0/2827861615 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e980ce790 msgr2=0x7f5e9815bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:31.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.720+0000 7f5e97e00640 1 --2- 172.21.15.86:0/2827861615 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e980ce790 0x7f5e9815bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:31.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.720+0000 7f5e97e00640 1 -- 172.21.15.86:0/2827861615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e980d1c20 msgr2=0x7f5e9815c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:31.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.720+0000 7f5e97e00640 1 --2- 172.21.15.86:0/2827861615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e980d1c20 0x7f5e9815c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:31.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.720+0000 7f5e97e00640 1 -- 172.21.15.86:0/2827861615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e9815da20 con 0x7f5e98070330 2024-04-25T14:05:31.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.720+0000 7f5e9d200640 1 --2- 172.21.15.86:0/2827861615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e980d1c20 0x7f5e9815c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:31.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.720+0000 7f5e97e00640 1 --2- 172.21.15.86:0/2827861615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e98070330 0x7f5e98167e20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800e870 tx=0x7f5e8800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:31.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.720+0000 7f5e94c00640 1 -- 172.21.15.86:0/2827861615 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e88017070 con 0x7f5e98070330 2024-04-25T14:05:31.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.720+0000 7f5e94c00640 1 -- 172.21.15.86:0/2827861615 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e88004020 con 0x7f5e98070330 2024-04-25T14:05:31.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.720+0000 7f5e9f200640 1 -- 172.21.15.86:0/2827861615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e98073930 con 0x7f5e98070330 2024-04-25T14:05:31.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.720+0000 7f5e94c00640 1 -- 172.21.15.86:0/2827861615 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e880043f0 con 0x7f5e98070330 2024-04-25T14:05:31.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.720+0000 7f5e9f200640 1 -- 172.21.15.86:0/2827861615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e98073ef0 con 0x7f5e98070330 2024-04-25T14:05:31.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.720+0000 7f5e9f200640 1 -- 172.21.15.86:0/2827861615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e98069d00 con 0x7f5e98070330 2024-04-25T14:05:31.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.724+0000 7f5e94c00640 1 -- 172.21.15.86:0/2827861615 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f5e8800c940 con 0x7f5e98070330 2024-04-25T14:05:31.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.724+0000 7f5e94c00640 1 --2- 172.21.15.86:0/2827861615 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f5e7406f570 0x7f5e74071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:31.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.724+0000 7f5e94c00640 1 -- 172.21.15.86:0/2827861615 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5960+0+0 (secure 0 0 0) 0x7f5e8808fbf0 con 0x7f5e98070330 2024-04-25T14:05:31.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.724+0000 7f5e8f400640 1 --2- 172.21.15.86:0/2827861615 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f5e7406f570 0x7f5e74071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:31.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.724+0000 7f5e8f400640 1 --2- 172.21.15.86:0/2827861615 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f5e7406f570 0x7f5e74071a80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5e980b0d50 tx=0x7f5e80007df0 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:31.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.728+0000 7f5e94c00640 1 -- 172.21.15.86:0/2827861615 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5e8805c260 con 0x7f5e98070330 2024-04-25T14:05:31.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:31.884+0000 7f5e9f200640 1 -- 172.21.15.86:0/2827861615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "fs1_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5e980ae9f0 con 0x7f5e98070330 2024-04-25T14:05:32.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.528+0000 7f5e94c00640 1 -- 172.21.15.86:0/2827861615 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "fs1_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'fs1_data' created v44) ==== 164+0+0 (secure 0 0 0) 0x7f5e88060560 con 0x7f5e98070330 2024-04-25T14:05:32.532 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'fs1_data' created 2024-04-25T14:05:32.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.532+0000 7f5e9f200640 1 -- 172.21.15.86:0/2827861615 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f5e7406f570 msgr2=0x7f5e74071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:32.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.532+0000 7f5e9f200640 1 --2- 172.21.15.86:0/2827861615 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f5e7406f570 0x7f5e74071a80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5e980b0d50 tx=0x7f5e80007df0 comp rx=0 tx=0).stop 2024-04-25T14:05:32.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.532+0000 7f5e9f200640 1 -- 172.21.15.86:0/2827861615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e98070330 msgr2=0x7f5e98167e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:32.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.532+0000 7f5e9f200640 1 --2- 172.21.15.86:0/2827861615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e98070330 0x7f5e98167e20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800e870 tx=0x7f5e8800ed30 comp rx=0 tx=0).stop 2024-04-25T14:05:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.532+0000 7f5e9f200640 1 -- 172.21.15.86:0/2827861615 shutdown_connections 2024-04-25T14:05:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.532+0000 7f5e9f200640 1 --2- 172.21.15.86:0/2827861615 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f5e7406f570 0x7f5e74071a80 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5e980b0d50 tx=0x7f5e80007df0 comp rx=0 tx=0).stop 2024-04-25T14:05:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.532+0000 7f5e9f200640 1 --2- 172.21.15.86:0/2827861615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e980d1c20 0x7f5e9815c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.532+0000 7f5e9f200640 1 --2- 172.21.15.86:0/2827861615 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e980ce790 0x7f5e9815bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.532+0000 7f5e9f200640 1 --2- 172.21.15.86:0/2827861615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e98070330 0x7f5e98167e20 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.532+0000 7f5e9f200640 1 -- 172.21.15.86:0/2827861615 >> 172.21.15.86:0/2827861615 conn(0x7f5e980b4650 msgr2=0x7f5e980b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.532+0000 7f5e9f200640 1 -- 172.21.15.86:0/2827861615 shutdown_connections 2024-04-25T14:05:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.532+0000 7f5e9f200640 1 -- 172.21.15.86:0/2827861615 wait complete. 2024-04-25T14:05:32.582 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new fs1 fs1_metadata fs1_data 2024-04-25T14:05:32.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.732+0000 7ff1d1e00640 1 Processor -- start 2024-04-25T14:05:32.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.732+0000 7ff1d1e00640 1 -- start start 2024-04-25T14:05:32.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.732+0000 7ff1d1e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff1cc0cfca0 0x7ff1cc06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:32.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.732+0000 7ff1d1e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1cc072bd0 con 0x7ff1cc0730b0 2024-04-25T14:05:32.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.732+0000 7ff1d1e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1cc06f930 con 0x7ff1cc0d16e0 2024-04-25T14:05:32.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.732+0000 7ff1d1e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff1cc070470 con 0x7ff1cc0cfca0 2024-04-25T14:05:32.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.732+0000 7ff1caa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff1cc0cfca0 0x7ff1cc06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:32.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.732+0000 7ff1caa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff1cc0cfca0 0x7ff1cc06f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52022/0 (socket says 172.21.15.86:52022) 2024-04-25T14:05:32.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.732+0000 7ff1caa00640 1 -- 172.21.15.86:0/3903923991 learned_addr learned my addr 172.21.15.86:0/3903923991 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:32.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.732+0000 7ff1caa00640 1 -- 172.21.15.86:0/3903923991 >> v1:172.21.15.155:6789/0 conn(0x7ff1cc0d16e0 legacy=0x7ff1cc0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:32.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.732+0000 7ff1caa00640 1 -- 172.21.15.86:0/3903923991 >> v1:172.21.15.86:6789/0 conn(0x7ff1cc0730b0 legacy=0x7ff1cc0734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:32.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.732+0000 7ff1caa00640 1 -- 172.21.15.86:0/3903923991 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1cc06fbd0 con 0x7ff1cc0cfca0 2024-04-25T14:05:32.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.736+0000 7ff1caa00640 1 --2- 172.21.15.86:0/3903923991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1cc0cfca0 0x7ff1cc06f3f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009870 tx=0x7ff1bc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa20f362aa15d653 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:32.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.736+0000 7ff1ca000640 1 -- 172.21.15.86:0/3903923991 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc0a7070 con 0x7ff1cc0cfca0 2024-04-25T14:05:32.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.736+0000 7ff1ca000640 1 -- 172.21.15.86:0/3903923991 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1bc0b0070 con 0x7ff1cc0cfca0 2024-04-25T14:05:32.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.736+0000 7ff1ca000640 1 -- 172.21.15.86:0/3903923991 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0743e0 con 0x7ff1cc0cfca0 2024-04-25T14:05:32.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.736+0000 7ff1d1e00640 1 -- 172.21.15.86:0/3903923991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1cc0cfca0 msgr2=0x7ff1cc06f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:32.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.736+0000 7ff1d1e00640 1 --2- 172.21.15.86:0/3903923991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1cc0cfca0 0x7ff1cc06f3f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009870 tx=0x7ff1bc01c560 comp rx=0 tx=0).stop 2024-04-25T14:05:32.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.736+0000 7ff1d1e00640 1 -- 172.21.15.86:0/3903923991 shutdown_connections 2024-04-25T14:05:32.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.736+0000 7ff1d1e00640 1 --2- 172.21.15.86:0/3903923991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1cc0cfca0 0x7ff1cc06f3f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:32.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.736+0000 7ff1d1e00640 1 -- 172.21.15.86:0/3903923991 >> 172.21.15.86:0/3903923991 conn(0x7ff1cc0b4650 msgr2=0x7ff1cc0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:32.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.736+0000 7ff1d1e00640 1 -- 172.21.15.86:0/3903923991 shutdown_connections 2024-04-25T14:05:32.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.736+0000 7ff1d1e00640 1 -- 172.21.15.86:0/3903923991 wait complete. 2024-04-25T14:05:32.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.740+0000 7ff1d1e00640 1 Processor -- start 2024-04-25T14:05:32.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.740+0000 7ff1d1e00640 1 -- start start 2024-04-25T14:05:32.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.740+0000 7ff1d1e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1cc0730b0 0x7ff1cc15ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:32.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.740+0000 7ff1d1e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1cc0cfca0 0x7ff1cc1fd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:32.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.740+0000 7ff1d1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1cc0d16e0 0x7ff1cc15a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:32.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.740+0000 7ff1d1e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc16aa80 con 0x7ff1cc0d16e0 2024-04-25T14:05:32.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.740+0000 7ff1d1e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc16a900 con 0x7ff1cc0cfca0 2024-04-25T14:05:32.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.740+0000 7ff1d1e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff1cc16ac00 con 0x7ff1cc0730b0 2024-04-25T14:05:32.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.740+0000 7ff1cb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1cc0d16e0 0x7ff1cc15a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:32.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.740+0000 7ff1cb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1cc0d16e0 0x7ff1cc15a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52024/0 (socket says 172.21.15.86:52024) 2024-04-25T14:05:32.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.740+0000 7ff1cb400640 1 -- 172.21.15.86:0/1670010065 learned_addr learned my addr 172.21.15.86:0/1670010065 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:32.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.740+0000 7ff1caa00640 1 --2- 172.21.15.86:0/1670010065 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1cc0730b0 0x7ff1cc15ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:32.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.740+0000 7ff1c3e00640 1 --2- 172.21.15.86:0/1670010065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1cc0cfca0 0x7ff1cc1fd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:32.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.740+0000 7ff1cb400640 1 -- 172.21.15.86:0/1670010065 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1cc0730b0 msgr2=0x7ff1cc15ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:32.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.740+0000 7ff1cb400640 1 --2- 172.21.15.86:0/1670010065 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1cc0730b0 0x7ff1cc15ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:32.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.740+0000 7ff1cb400640 1 -- 172.21.15.86:0/1670010065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1cc0cfca0 msgr2=0x7ff1cc1fd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:32.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.740+0000 7ff1cb400640 1 --2- 172.21.15.86:0/1670010065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1cc0cfca0 0x7ff1cc1fd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:32.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.740+0000 7ff1cb400640 1 -- 172.21.15.86:0/1670010065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1cc153ce0 con 0x7ff1cc0d16e0 2024-04-25T14:05:32.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.740+0000 7ff1cb400640 1 --2- 172.21.15.86:0/1670010065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1cc0d16e0 0x7ff1cc15a5f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400c730 tx=0x7ff1c400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:32.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.740+0000 7ff1c3400640 1 -- 172.21.15.86:0/1670010065 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4003b80 con 0x7ff1cc0d16e0 2024-04-25T14:05:32.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.740+0000 7ff1c3400640 1 -- 172.21.15.86:0/1670010065 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c4003d20 con 0x7ff1cc0d16e0 2024-04-25T14:05:32.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.740+0000 7ff1c3400640 1 -- 172.21.15.86:0/1670010065 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4005310 con 0x7ff1cc0d16e0 2024-04-25T14:05:32.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.740+0000 7ff1d1e00640 1 -- 172.21.15.86:0/1670010065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1cc153fc0 con 0x7ff1cc0d16e0 2024-04-25T14:05:32.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.740+0000 7ff1d1e00640 1 -- 172.21.15.86:0/1670010065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1cc154580 con 0x7ff1cc0d16e0 2024-04-25T14:05:32.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.744+0000 7ff1d1e00640 1 -- 172.21.15.86:0/1670010065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1cc09f4b0 con 0x7ff1cc0d16e0 2024-04-25T14:05:32.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.744+0000 7ff1c3400640 1 -- 172.21.15.86:0/1670010065 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7ff1c40056c0 con 0x7ff1cc0d16e0 2024-04-25T14:05:32.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.744+0000 7ff1c3400640 1 --2- 172.21.15.86:0/1670010065 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7ff1a806f570 0x7ff1a8071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:32.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.744+0000 7ff1c3400640 1 -- 172.21.15.86:0/1670010065 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6339+0+0 (secure 0 0 0) 0x7ff1c4090000 con 0x7ff1cc0d16e0 2024-04-25T14:05:32.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.744+0000 7ff1caa00640 1 --2- 172.21.15.86:0/1670010065 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7ff1a806f570 0x7ff1a8071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:32.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.748+0000 7ff1caa00640 1 --2- 172.21.15.86:0/1670010065 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7ff1a806f570 0x7ff1a8071a80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0af000 tx=0x7ff1bc0a7000 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:32.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.748+0000 7ff1c3400640 1 -- 172.21.15.86:0/1670010065 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff1c405c680 con 0x7ff1cc0d16e0 2024-04-25T14:05:32.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:32.940+0000 7ff1d1e00640 1 -- 172.21.15.86:0/1670010065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "fs1", "metadata": "fs1_metadata", "data": "fs1_data"} v 0) -- 0x7ff1cc069d00 con 0x7ff1cc0d16e0 2024-04-25T14:05:33.555 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 7 and data pool 8 2024-04-25T14:05:33.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.552+0000 7ff1c3400640 1 -- 172.21.15.86:0/1670010065 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "fs1", "metadata": "fs1_metadata", "data": "fs1_data"}]=0 new fs with metadata pool 7 and data pool 8 v22) ==== 163+0+0 (secure 0 0 0) 0x7ff1c4060980 con 0x7ff1cc0d16e0 2024-04-25T14:05:33.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.556+0000 7ff1d1e00640 1 -- 172.21.15.86:0/1670010065 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7ff1a806f570 msgr2=0x7ff1a8071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:33.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.556+0000 7ff1d1e00640 1 --2- 172.21.15.86:0/1670010065 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7ff1a806f570 0x7ff1a8071a80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0af000 tx=0x7ff1bc0a7000 comp rx=0 tx=0).stop 2024-04-25T14:05:33.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.556+0000 7ff1d1e00640 1 -- 172.21.15.86:0/1670010065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1cc0d16e0 msgr2=0x7ff1cc15a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:33.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.556+0000 7ff1d1e00640 1 --2- 172.21.15.86:0/1670010065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1cc0d16e0 0x7ff1cc15a5f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400c730 tx=0x7ff1c400cbf0 comp rx=0 tx=0).stop 2024-04-25T14:05:33.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.556+0000 7ff1d1e00640 1 -- 172.21.15.86:0/1670010065 shutdown_connections 2024-04-25T14:05:33.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.556+0000 7ff1d1e00640 1 --2- 172.21.15.86:0/1670010065 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7ff1a806f570 0x7ff1a8071a80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:33.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.556+0000 7ff1d1e00640 1 --2- 172.21.15.86:0/1670010065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1cc0d16e0 0x7ff1cc15a5f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:33.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.556+0000 7ff1d1e00640 1 --2- 172.21.15.86:0/1670010065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1cc0cfca0 0x7ff1cc1fd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:33.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.556+0000 7ff1d1e00640 1 --2- 172.21.15.86:0/1670010065 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1cc0730b0 0x7ff1cc15ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:33.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.556+0000 7ff1d1e00640 1 -- 172.21.15.86:0/1670010065 >> 172.21.15.86:0/1670010065 conn(0x7ff1cc0b4650 msgr2=0x7ff1cc0aee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.560+0000 7ff1d1e00640 1 -- 172.21.15.86:0/1670010065 shutdown_connections 2024-04-25T14:05:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.560+0000 7ff1d1e00640 1 -- 172.21.15.86:0/1670010065 wait complete. 2024-04-25T14:05:33.607 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:05:33.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.752+0000 7f8793a00640 1 Processor -- start 2024-04-25T14:05:33.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.752+0000 7f8793a00640 1 -- start start 2024-04-25T14:05:33.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.752+0000 7f8793a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f878c0cf970 0x7f878c070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:33.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.752+0000 7f8793a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f878c071360 0x7f878c06e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:33.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.752+0000 7f8793a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f878c071770 con 0x7f878c06e7a0 2024-04-25T14:05:33.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.752+0000 7f8793a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f878c0cedf0 con 0x7f878c0cf970 2024-04-25T14:05:33.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.752+0000 7f8793a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f878c163830 con 0x7f878c071360 2024-04-25T14:05:33.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.752+0000 7f8791000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f878c0cf970 0x7f878c070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:33.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.752+0000 7f8791000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f878c0cf970 0x7f878c070e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52036/0 (socket says 172.21.15.86:52036) 2024-04-25T14:05:33.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.752+0000 7f8791000640 1 -- 172.21.15.86:0/3549478561 learned_addr learned my addr 172.21.15.86:0/3549478561 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:33.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.756+0000 7f878be00640 1 --2- 172.21.15.86:0/3549478561 >> v2:172.21.15.155:3300/0 conn(0x7f878c071360 0x7f878c06e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:33.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.756+0000 7f8791000640 1 -- 172.21.15.86:0/3549478561 >> v2:172.21.15.155:3300/0 conn(0x7f878c071360 msgr2=0x7f878c06e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:33.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.756+0000 7f8791000640 1 --2- 172.21.15.86:0/3549478561 >> v2:172.21.15.155:3300/0 conn(0x7f878c071360 0x7f878c06e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:33.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.756+0000 7f8791000640 1 -- 172.21.15.86:0/3549478561 >> v1:172.21.15.155:6789/0 conn(0x7f878c06e7a0 legacy=0x7f878c0722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:33.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.756+0000 7f8791000640 1 -- 172.21.15.86:0/3549478561 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f878c06ef90 con 0x7f878c0cf970 2024-04-25T14:05:33.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.756+0000 7f8791000640 1 --2- 172.21.15.86:0/3549478561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f878c0cf970 0x7f878c070e20 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f878c06d8a0 tx=0x7f878001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380868077faaa571 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:33.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.756+0000 7f878b400640 1 -- 172.21.15.86:0/3549478561 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87800ab610 con 0x7f878c0cf970 2024-04-25T14:05:33.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.756+0000 7f878b400640 1 -- 172.21.15.86:0/3549478561 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87800b0070 con 0x7f878c0cf970 2024-04-25T14:05:33.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.756+0000 7f878b400640 1 -- 172.21.15.86:0/3549478561 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87800ab9d0 con 0x7f878c0cf970 2024-04-25T14:05:33.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.760+0000 7f8793a00640 1 -- 172.21.15.86:0/3549478561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f878c0cf970 msgr2=0x7f878c070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:33.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.760+0000 7f8793a00640 1 --2- 172.21.15.86:0/3549478561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f878c0cf970 0x7f878c070e20 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f878c06d8a0 tx=0x7f878001c4f0 comp rx=0 tx=0).stop 2024-04-25T14:05:33.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.760+0000 7f8793a00640 1 -- 172.21.15.86:0/3549478561 shutdown_connections 2024-04-25T14:05:33.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.760+0000 7f8793a00640 1 --2- 172.21.15.86:0/3549478561 >> v2:172.21.15.155:3300/0 conn(0x7f878c071360 0x7f878c06e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:33.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.760+0000 7f8793a00640 1 --2- 172.21.15.86:0/3549478561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f878c0cf970 0x7f878c070e20 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:33.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.760+0000 7f8793a00640 1 -- 172.21.15.86:0/3549478561 >> 172.21.15.86:0/3549478561 conn(0x7f878c0cffc0 msgr2=0x7f878c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:33.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.760+0000 7f8793a00640 1 -- 172.21.15.86:0/3549478561 shutdown_connections 2024-04-25T14:05:33.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.760+0000 7f8793a00640 1 -- 172.21.15.86:0/3549478561 wait complete. 2024-04-25T14:05:33.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.760+0000 7f8793a00640 1 Processor -- start 2024-04-25T14:05:33.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.760+0000 7f8793a00640 1 -- start start 2024-04-25T14:05:33.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.764+0000 7f8793a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f878c06e7a0 0x7f878c15d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:33.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.764+0000 7f8793a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f878c071360 0x7f878c164980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:33.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.764+0000 7f8793a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f878c0cf970 0x7f878c0af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:33.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.764+0000 7f8793a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f878c06ed30 con 0x7f878c06e7a0 2024-04-25T14:05:33.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.764+0000 7f8793a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f878c06ebb0 con 0x7f878c071360 2024-04-25T14:05:33.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.764+0000 7f8793a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f878c16cbd0 con 0x7f878c0cf970 2024-04-25T14:05:33.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.764+0000 7f8791000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f878c06e7a0 0x7f878c15d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:33.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.764+0000 7f8791000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f878c06e7a0 0x7f878c15d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52042/0 (socket says 172.21.15.86:52042) 2024-04-25T14:05:33.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.764+0000 7f8791000640 1 -- 172.21.15.86:0/900935206 learned_addr learned my addr 172.21.15.86:0/900935206 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:33.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.764+0000 7f878be00640 1 --2- 172.21.15.86:0/900935206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f878c071360 0x7f878c164980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:33.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.764+0000 7f8791a00640 1 --2- 172.21.15.86:0/900935206 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f878c0cf970 0x7f878c0af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:33.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.764+0000 7f8791000640 1 -- 172.21.15.86:0/900935206 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f878c0cf970 msgr2=0x7f878c0af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:33.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.764+0000 7f8791000640 1 --2- 172.21.15.86:0/900935206 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f878c0cf970 0x7f878c0af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:33.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.764+0000 7f8791000640 1 -- 172.21.15.86:0/900935206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f878c071360 msgr2=0x7f878c164980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:33.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.764+0000 7f8791000640 1 --2- 172.21.15.86:0/900935206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f878c071360 0x7f878c164980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:33.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.764+0000 7f8791000640 1 -- 172.21.15.86:0/900935206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f878c164ec0 con 0x7f878c06e7a0 2024-04-25T14:05:33.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.764+0000 7f8791000640 1 --2- 172.21.15.86:0/900935206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f878c06e7a0 0x7f878c15d470 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f87800ad040 tx=0x7f878001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:33.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.764+0000 7f8788c00640 1 -- 172.21.15.86:0/900935206 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87800be5f0 con 0x7f878c06e7a0 2024-04-25T14:05:33.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.764+0000 7f8788c00640 1 -- 172.21.15.86:0/900935206 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87800be7b0 con 0x7f878c06e7a0 2024-04-25T14:05:33.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.764+0000 7f8788c00640 1 -- 172.21.15.86:0/900935206 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87800beba0 con 0x7f878c06e7a0 2024-04-25T14:05:33.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.764+0000 7f8793a00640 1 -- 172.21.15.86:0/900935206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f878c0b91d0 con 0x7f878c06e7a0 2024-04-25T14:05:33.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.764+0000 7f8793a00640 1 -- 172.21.15.86:0/900935206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f878c0b7730 con 0x7f878c06e7a0 2024-04-25T14:05:33.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.764+0000 7f8793a00640 1 -- 172.21.15.86:0/900935206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f878c0720e0 con 0x7f878c06e7a0 2024-04-25T14:05:33.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.764+0000 7f8788c00640 1 -- 172.21.15.86:0/900935206 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f87800b43f0 con 0x7f878c06e7a0 2024-04-25T14:05:33.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.768+0000 7f8788c00640 1 --2- 172.21.15.86:0/900935206 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f876c06f5a0 0x7f876c071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:33.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.768+0000 7f8788c00640 1 -- 172.21.15.86:0/900935206 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6433+0+0 (secure 0 0 0) 0x7f8780129dc0 con 0x7f878c06e7a0 2024-04-25T14:05:33.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.768+0000 7f878be00640 1 --2- 172.21.15.86:0/900935206 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f876c06f5a0 0x7f876c071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:33.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.768+0000 7f878be00640 1 --2- 172.21.15.86:0/900935206 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f876c06f5a0 0x7f876c071ab0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f878c0b2570 tx=0x7f8778008040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:33.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.768+0000 7f8788c00640 1 -- 172.21.15.86:0/900935206 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f87800a8050 con 0x7f878c06e7a0 2024-04-25T14:05:33.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.928+0000 7f8793a00640 1 -- 172.21.15.86:0/900935206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f878c163120 con 0x7f878c06e7a0 2024-04-25T14:05:33.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.928+0000 7f8788c00640 1 -- 172.21.15.86:0/900935206 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+17448 (secure 0 0 0) 0x7f8780016110 con 0x7f878c06e7a0 2024-04-25T14:05:33.932 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:33.932 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":45,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:05:32.946371+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"fs1_metadata","create_time":"2024-04-25T14:05:30.939601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"fs1"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"fs1_data","create_time":"2024-04-25T14:05:31.898262+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"fs1"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6835/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:0/2073988773":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6833/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6832/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/3558294881":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/1843278167":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6837/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:0/2602032945":"2024-04-26T14:05:21.460855+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:05:33.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.932+0000 7f8793a00640 1 -- 172.21.15.86:0/900935206 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f876c06f5a0 msgr2=0x7f876c071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:33.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.932+0000 7f8793a00640 1 --2- 172.21.15.86:0/900935206 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f876c06f5a0 0x7f876c071ab0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f878c0b2570 tx=0x7f8778008040 comp rx=0 tx=0).stop 2024-04-25T14:05:33.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.932+0000 7f8793a00640 1 -- 172.21.15.86:0/900935206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f878c06e7a0 msgr2=0x7f878c15d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:33.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.932+0000 7f8793a00640 1 --2- 172.21.15.86:0/900935206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f878c06e7a0 0x7f878c15d470 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f87800ad040 tx=0x7f878001ad30 comp rx=0 tx=0).stop 2024-04-25T14:05:33.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.932+0000 7f8793a00640 1 -- 172.21.15.86:0/900935206 shutdown_connections 2024-04-25T14:05:33.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.932+0000 7f8793a00640 1 --2- 172.21.15.86:0/900935206 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f876c06f5a0 0x7f876c071ab0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:33.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.932+0000 7f8793a00640 1 --2- 172.21.15.86:0/900935206 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f878c0cf970 0x7f878c0af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:33.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.932+0000 7f8793a00640 1 --2- 172.21.15.86:0/900935206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f878c071360 0x7f878c164980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:33.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.932+0000 7f8793a00640 1 --2- 172.21.15.86:0/900935206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f878c06e7a0 0x7f878c15d470 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:33.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.932+0000 7f8793a00640 1 -- 172.21.15.86:0/900935206 >> 172.21.15.86:0/900935206 conn(0x7f878c0cffc0 msgr2=0x7f878c15fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:33.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.932+0000 7f8793a00640 1 -- 172.21.15.86:0/900935206 shutdown_connections 2024-04-25T14:05:33.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:33.932+0000 7f8793a00640 1 -- 172.21.15.86:0/900935206 wait complete. 2024-04-25T14:05:33.954 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:05:34.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.100+0000 7f4378c00640 1 Processor -- start 2024-04-25T14:05:34.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.100+0000 7f4378c00640 1 -- start start 2024-04-25T14:05:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.100+0000 7f4378c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f437415e8e0 0x7f437415f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.100+0000 7f4378c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f437415ecf0 0x7f437415f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.100+0000 7f4378c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f43740cf0e0 0x7f43740cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.100+0000 7f4378c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f437415e450 con 0x7f43740cf0e0 2024-04-25T14:05:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.100+0000 7f4378c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f437406d4a0 con 0x7f437415e8e0 2024-04-25T14:05:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.100+0000 7f4378c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f43740d0510 con 0x7f437415ecf0 2024-04-25T14:05:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.100+0000 7f4373e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f43740cf0e0 0x7f43740cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.100+0000 7f4373e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f43740cf0e0 0x7f43740cf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52048/0 (socket says 172.21.15.86:52048) 2024-04-25T14:05:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.100+0000 7f4373e00640 1 -- 172.21.15.86:0/2953371684 learned_addr learned my addr 172.21.15.86:0/2953371684 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.100+0000 7f436aa00640 1 --2- 172.21.15.86:0/2953371684 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f437415ecf0 0x7f437415f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.100+0000 7f4373e00640 1 -- 172.21.15.86:0/2953371684 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f437415ecf0 msgr2=0x7f437415f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.100+0000 7f4373400640 1 --2- 172.21.15.86:0/2953371684 >> v2:172.21.15.155:3300/0 conn(0x7f437415e8e0 0x7f437415f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.100+0000 7f4373e00640 1 --2- 172.21.15.86:0/2953371684 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f437415ecf0 0x7f437415f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.100+0000 7f4373e00640 1 -- 172.21.15.86:0/2953371684 >> v2:172.21.15.155:3300/0 conn(0x7f437415e8e0 msgr2=0x7f437415f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.100+0000 7f4373e00640 1 --2- 172.21.15.86:0/2953371684 >> v2:172.21.15.155:3300/0 conn(0x7f437415e8e0 0x7f437415f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.100+0000 7f4373e00640 1 -- 172.21.15.86:0/2953371684 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43740d07f0 con 0x7f43740cf0e0 2024-04-25T14:05:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.100+0000 7f436aa00640 1 --2- 172.21.15.86:0/2953371684 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f437415ecf0 0x7f437415f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.100+0000 7f4373400640 1 --2- 172.21.15.86:0/2953371684 >> v2:172.21.15.155:3300/0 conn(0x7f437415e8e0 0x7f437415f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.104+0000 7f4373e00640 1 --2- 172.21.15.86:0/2953371684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43740cf0e0 0x7f43740cf4f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4374071a90 tx=0x7f436c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e586b5fc482f398 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.104+0000 7f4372a00640 1 -- 172.21.15.86:0/2953371684 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c0a5780 con 0x7f43740cf0e0 2024-04-25T14:05:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.104+0000 7f4372a00640 1 -- 172.21.15.86:0/2953371684 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f436c0a5920 con 0x7f43740cf0e0 2024-04-25T14:05:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.104+0000 7f4372a00640 1 -- 172.21.15.86:0/2953371684 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c0a5cf0 con 0x7f43740cf0e0 2024-04-25T14:05:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.104+0000 7f4378c00640 1 -- 172.21.15.86:0/2953371684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43740cf0e0 msgr2=0x7f43740cf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.104+0000 7f4378c00640 1 --2- 172.21.15.86:0/2953371684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43740cf0e0 0x7f43740cf4f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4374071a90 tx=0x7f436c01c9f0 comp rx=0 tx=0).stop 2024-04-25T14:05:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.104+0000 7f4378c00640 1 -- 172.21.15.86:0/2953371684 shutdown_connections 2024-04-25T14:05:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.104+0000 7f4378c00640 1 --2- 172.21.15.86:0/2953371684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43740cf0e0 0x7f43740cf4f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.104+0000 7f4378c00640 1 --2- 172.21.15.86:0/2953371684 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f437415ecf0 0x7f437415f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.104+0000 7f4378c00640 1 --2- 172.21.15.86:0/2953371684 >> v2:172.21.15.155:3300/0 conn(0x7f437415e8e0 0x7f437415f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:34.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.104+0000 7f4378c00640 1 -- 172.21.15.86:0/2953371684 >> 172.21.15.86:0/2953371684 conn(0x7f43740b4850 msgr2=0x7f43740b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:34.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.104+0000 7f4378c00640 1 -- 172.21.15.86:0/2953371684 shutdown_connections 2024-04-25T14:05:34.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.104+0000 7f4378c00640 1 -- 172.21.15.86:0/2953371684 wait complete. 2024-04-25T14:05:34.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.108+0000 7f4378c00640 1 Processor -- start 2024-04-25T14:05:34.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.108+0000 7f4378c00640 1 -- start start 2024-04-25T14:05:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.108+0000 7f4378c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43740cf0e0 0x7f437414c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.108+0000 7f4378c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f437415e8e0 0x7f437414f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.108+0000 7f4378c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f437415ecf0 0x7f437409c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.108+0000 7f4378c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43740d0c50 con 0x7f437415ecf0 2024-04-25T14:05:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.108+0000 7f4378c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f43740d0ad0 con 0x7f437415e8e0 2024-04-25T14:05:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.108+0000 7f4378c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f437406ffe0 con 0x7f43740cf0e0 2024-04-25T14:05:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.108+0000 7f4373e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f437415ecf0 0x7f437409c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.108+0000 7f4373e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f437415ecf0 0x7f437409c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52056/0 (socket says 172.21.15.86:52056) 2024-04-25T14:05:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.108+0000 7f4373e00640 1 -- 172.21.15.86:0/1109020749 learned_addr learned my addr 172.21.15.86:0/1109020749 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.108+0000 7f4373400640 1 --2- 172.21.15.86:0/1109020749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43740cf0e0 0x7f437414c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.108+0000 7f436aa00640 1 --2- 172.21.15.86:0/1109020749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f437415e8e0 0x7f437414f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.108+0000 7f4373e00640 1 -- 172.21.15.86:0/1109020749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43740cf0e0 msgr2=0x7f437414c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.108+0000 7f4373e00640 1 --2- 172.21.15.86:0/1109020749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43740cf0e0 0x7f437414c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.108+0000 7f4373e00640 1 -- 172.21.15.86:0/1109020749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f437415e8e0 msgr2=0x7f437414f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.108+0000 7f4373e00640 1 --2- 172.21.15.86:0/1109020749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f437415e8e0 0x7f437414f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.108+0000 7f4373e00640 1 -- 172.21.15.86:0/1109020749 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437414e070 con 0x7f437415ecf0 2024-04-25T14:05:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.112+0000 7f4373e00640 1 --2- 172.21.15.86:0/1109020749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f437415ecf0 0x7f437409c2c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f436c01d290 tx=0x7f436c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.112+0000 7f436be00640 1 -- 172.21.15.86:0/1109020749 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c0b8670 con 0x7f437415ecf0 2024-04-25T14:05:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.112+0000 7f436be00640 1 -- 172.21.15.86:0/1109020749 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f436c0be070 con 0x7f437415ecf0 2024-04-25T14:05:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.112+0000 7f436be00640 1 -- 172.21.15.86:0/1109020749 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c0b8a90 con 0x7f437415ecf0 2024-04-25T14:05:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.112+0000 7f4378c00640 1 -- 172.21.15.86:0/1109020749 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f437414f610 con 0x7f437415ecf0 2024-04-25T14:05:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.112+0000 7f4378c00640 1 -- 172.21.15.86:0/1109020749 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f437414e5f0 con 0x7f437415ecf0 2024-04-25T14:05:34.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.112+0000 7f4378c00640 1 -- 172.21.15.86:0/1109020749 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4374076b80 con 0x7f437415ecf0 2024-04-25T14:05:34.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.112+0000 7f436be00640 1 -- 172.21.15.86:0/1109020749 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f436c0b8c30 con 0x7f437415ecf0 2024-04-25T14:05:34.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.112+0000 7f436be00640 1 --2- 172.21.15.86:0/1109020749 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f435006f5a0 0x7f4350071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:34.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.112+0000 7f436be00640 1 -- 172.21.15.86:0/1109020749 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6433+0+0 (secure 0 0 0) 0x7f436c12aa60 con 0x7f437415ecf0 2024-04-25T14:05:34.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.112+0000 7f4373400640 1 --2- 172.21.15.86:0/1109020749 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f435006f5a0 0x7f4350071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:34.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.116+0000 7f4373400640 1 --2- 172.21.15.86:0/1109020749 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f435006f5a0 0x7f4350071ab0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f436400aaf0 tx=0x7f4364008040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.116+0000 7f436be00640 1 -- 172.21.15.86:0/1109020749 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f436c0c7020 con 0x7f437415ecf0 2024-04-25T14:05:34.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.272+0000 7f4378c00640 1 -- 172.21.15.86:0/1109020749 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f43740725a0 con 0x7f437415ecf0 2024-04-25T14:05:34.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.276+0000 7f436be00640 1 -- 172.21.15.86:0/1109020749 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+17448 (secure 0 0 0) 0x7f436c0165e0 con 0x7f437415ecf0 2024-04-25T14:05:34.277 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:34.278 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":45,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:05:32.946371+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"fs1_metadata","create_time":"2024-04-25T14:05:30.939601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"fs1"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"fs1_data","create_time":"2024-04-25T14:05:31.898262+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"fs1"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6835/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:0/2073988773":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6833/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6832/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/3558294881":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/1843278167":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6837/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:0/2602032945":"2024-04-26T14:05:21.460855+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:05:34.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.276+0000 7f4378c00640 1 -- 172.21.15.86:0/1109020749 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f435006f5a0 msgr2=0x7f4350071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:34.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.276+0000 7f4378c00640 1 --2- 172.21.15.86:0/1109020749 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f435006f5a0 0x7f4350071ab0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f436400aaf0 tx=0x7f4364008040 comp rx=0 tx=0).stop 2024-04-25T14:05:34.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.276+0000 7f4378c00640 1 -- 172.21.15.86:0/1109020749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f437415ecf0 msgr2=0x7f437409c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:34.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.276+0000 7f4378c00640 1 --2- 172.21.15.86:0/1109020749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f437415ecf0 0x7f437409c2c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f436c01d290 tx=0x7f436c019790 comp rx=0 tx=0).stop 2024-04-25T14:05:34.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.276+0000 7f4378c00640 1 -- 172.21.15.86:0/1109020749 shutdown_connections 2024-04-25T14:05:34.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.276+0000 7f4378c00640 1 --2- 172.21.15.86:0/1109020749 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f435006f5a0 0x7f4350071ab0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:34.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.276+0000 7f4378c00640 1 --2- 172.21.15.86:0/1109020749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f437415ecf0 0x7f437409c2c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:34.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.276+0000 7f4378c00640 1 --2- 172.21.15.86:0/1109020749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f437415e8e0 0x7f437414f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:34.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.276+0000 7f4378c00640 1 --2- 172.21.15.86:0/1109020749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43740cf0e0 0x7f437414c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:34.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.276+0000 7f4378c00640 1 -- 172.21.15.86:0/1109020749 >> 172.21.15.86:0/1109020749 conn(0x7f43740b4850 msgr2=0x7f4374152610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:34.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.276+0000 7f4378c00640 1 -- 172.21.15.86:0/1109020749 shutdown_connections 2024-04-25T14:05:34.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.276+0000 7f4378c00640 1 -- 172.21.15.86:0/1109020749 wait complete. 2024-04-25T14:05:34.300 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set fs1 standby_count_wanted 0 2024-04-25T14:05:34.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.448+0000 7fdcd5800640 1 Processor -- start 2024-04-25T14:05:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.448+0000 7fdcd5800640 1 -- start start 2024-04-25T14:05:34.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.448+0000 7fdcd5800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdcd00cfca0 0x7fdcd006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:34.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.448+0000 7fdcd5800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdcd00d16e0 0x7fdcd00d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:34.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.448+0000 7fdcd5800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcd006c670 0x7fdcd006ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:34.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.448+0000 7fdcd5800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdcd0070470 con 0x7fdcd00d16e0 2024-04-25T14:05:34.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.448+0000 7fdcd5800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fdcd00d2030 con 0x7fdcd00cfca0 2024-04-25T14:05:34.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.448+0000 7fdcd5800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdcd0072bd0 con 0x7fdcd006c670 2024-04-25T14:05:34.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.448+0000 7fdcc6000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdcd00d16e0 0x7fdcd00d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:34.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.448+0000 7fdcc6000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdcd00d16e0 0x7fdcd00d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52064/0 (socket says 172.21.15.86:52064) 2024-04-25T14:05:34.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.448+0000 7fdcc6000640 1 -- 172.21.15.86:0/1574621419 learned_addr learned my addr 172.21.15.86:0/1574621419 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:34.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.448+0000 7fdccf400640 1 --2- 172.21.15.86:0/1574621419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcd006c670 0x7fdcd006ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:34.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.448+0000 7fdcc6000640 1 -- 172.21.15.86:0/1574621419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcd006c670 msgr2=0x7fdcd006ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:34.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.448+0000 7fdccea00640 1 --2- 172.21.15.86:0/1574621419 >> v2:172.21.15.155:3300/0 conn(0x7fdcd00cfca0 0x7fdcd006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:34.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.448+0000 7fdcc6000640 1 --2- 172.21.15.86:0/1574621419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcd006c670 0x7fdcd006ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:34.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.448+0000 7fdcc6000640 1 -- 172.21.15.86:0/1574621419 >> v2:172.21.15.155:3300/0 conn(0x7fdcd00cfca0 msgr2=0x7fdcd006f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:34.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.448+0000 7fdcc6000640 1 --2- 172.21.15.86:0/1574621419 >> v2:172.21.15.155:3300/0 conn(0x7fdcd00cfca0 0x7fdcd006f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:34.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.448+0000 7fdcc6000640 1 -- 172.21.15.86:0/1574621419 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd006f930 con 0x7fdcd00d16e0 2024-04-25T14:05:34.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.448+0000 7fdccf400640 1 --2- 172.21.15.86:0/1574621419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcd006c670 0x7fdcd006ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:34.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.448+0000 7fdccea00640 1 --2- 172.21.15.86:0/1574621419 >> v2:172.21.15.155:3300/0 conn(0x7fdcd00cfca0 0x7fdcd006f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:34.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.452+0000 7fdcc6000640 1 --2- 172.21.15.86:0/1574621419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd00d16e0 0x7fdcd00d1af0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fdcd00d22a0 tx=0x7fdcb801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480bebe5ed2a32fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:34.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.452+0000 7fdcce000640 1 -- 172.21.15.86:0/1574621419 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb80ab5f0 con 0x7fdcd00d16e0 2024-04-25T14:05:34.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.452+0000 7fdcce000640 1 -- 172.21.15.86:0/1574621419 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcb80b0070 con 0x7fdcd00d16e0 2024-04-25T14:05:34.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.452+0000 7fdcce000640 1 -- 172.21.15.86:0/1574621419 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb80ab9b0 con 0x7fdcd00d16e0 2024-04-25T14:05:34.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.452+0000 7fdcd5800640 1 -- 172.21.15.86:0/1574621419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd00d16e0 msgr2=0x7fdcd00d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:34.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.452+0000 7fdcd5800640 1 --2- 172.21.15.86:0/1574621419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd00d16e0 0x7fdcd00d1af0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fdcd00d22a0 tx=0x7fdcb801c520 comp rx=0 tx=0).stop 2024-04-25T14:05:34.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.452+0000 7fdcd5800640 1 -- 172.21.15.86:0/1574621419 shutdown_connections 2024-04-25T14:05:34.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.452+0000 7fdcd5800640 1 --2- 172.21.15.86:0/1574621419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcd006c670 0x7fdcd006ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:34.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.452+0000 7fdcd5800640 1 --2- 172.21.15.86:0/1574621419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd00d16e0 0x7fdcd00d1af0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:34.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.452+0000 7fdcd5800640 1 --2- 172.21.15.86:0/1574621419 >> v2:172.21.15.155:3300/0 conn(0x7fdcd00cfca0 0x7fdcd006f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:34.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.452+0000 7fdcd5800640 1 -- 172.21.15.86:0/1574621419 >> 172.21.15.86:0/1574621419 conn(0x7fdcd00b4650 msgr2=0x7fdcd00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:34.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.452+0000 7fdcd5800640 1 -- 172.21.15.86:0/1574621419 shutdown_connections 2024-04-25T14:05:34.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.452+0000 7fdcd5800640 1 -- 172.21.15.86:0/1574621419 wait complete. 2024-04-25T14:05:34.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.456+0000 7fdcd5800640 1 Processor -- start 2024-04-25T14:05:34.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.456+0000 7fdcd5800640 1 -- start start 2024-04-25T14:05:34.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.456+0000 7fdcd5800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcd006c670 0x7fdcd0156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:34.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.456+0000 7fdcd5800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd00cfca0 0x7fdcd009a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:34.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.456+0000 7fdcd5800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcd00d16e0 0x7fdcd015eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:34.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.456+0000 7fdcd5800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd00738c0 con 0x7fdcd00cfca0 2024-04-25T14:05:34.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.456+0000 7fdcd5800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd006d3d0 con 0x7fdcd00d16e0 2024-04-25T14:05:34.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.456+0000 7fdcd5800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdcd0073a40 con 0x7fdcd006c670 2024-04-25T14:05:34.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.456+0000 7fdcc6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd00cfca0 0x7fdcd009a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:34.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.456+0000 7fdcc6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd00cfca0 0x7fdcd009a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52074/0 (socket says 172.21.15.86:52074) 2024-04-25T14:05:34.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.456+0000 7fdcc6000640 1 -- 172.21.15.86:0/92314765 learned_addr learned my addr 172.21.15.86:0/92314765 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:34.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.456+0000 7fdccea00640 1 --2- 172.21.15.86:0/92314765 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcd006c670 0x7fdcd0156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:34.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.456+0000 7fdccf400640 1 --2- 172.21.15.86:0/92314765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcd00d16e0 0x7fdcd015eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:34.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.456+0000 7fdcc6000640 1 -- 172.21.15.86:0/92314765 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcd006c670 msgr2=0x7fdcd0156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:34.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.456+0000 7fdcc6000640 1 --2- 172.21.15.86:0/92314765 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcd006c670 0x7fdcd0156660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:34.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.456+0000 7fdcc6000640 1 -- 172.21.15.86:0/92314765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcd00d16e0 msgr2=0x7fdcd015eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:34.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.456+0000 7fdcc6000640 1 --2- 172.21.15.86:0/92314765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcd00d16e0 0x7fdcd015eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:34.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.456+0000 7fdcc6000640 1 -- 172.21.15.86:0/92314765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd01556f0 con 0x7fdcd00cfca0 2024-04-25T14:05:34.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.456+0000 7fdcc6000640 1 --2- 172.21.15.86:0/92314765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd00cfca0 0x7fdcd009a7b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fdcb801ca00 tx=0x7fdcb801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:34.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.456+0000 7fdcc7400640 1 -- 172.21.15.86:0/92314765 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb80be5f0 con 0x7fdcd00cfca0 2024-04-25T14:05:34.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.456+0000 7fdcc7400640 1 -- 172.21.15.86:0/92314765 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcb80be7b0 con 0x7fdcd00cfca0 2024-04-25T14:05:34.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.456+0000 7fdcd5800640 1 -- 172.21.15.86:0/92314765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcd01559d0 con 0x7fdcd00cfca0 2024-04-25T14:05:34.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.456+0000 7fdcc7400640 1 -- 172.21.15.86:0/92314765 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb80beba0 con 0x7fdcd00cfca0 2024-04-25T14:05:34.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.456+0000 7fdcd5800640 1 -- 172.21.15.86:0/92314765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcd0155f90 con 0x7fdcd00cfca0 2024-04-25T14:05:34.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.460+0000 7fdcd5800640 1 -- 172.21.15.86:0/92314765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcd009f4b0 con 0x7fdcd00cfca0 2024-04-25T14:05:34.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.460+0000 7fdcc7400640 1 -- 172.21.15.86:0/92314765 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7fdcb80b43f0 con 0x7fdcd00cfca0 2024-04-25T14:05:34.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.460+0000 7fdcc7400640 1 --2- 172.21.15.86:0/92314765 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fdcac06f5c0 0x7fdcac071ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:34.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.460+0000 7fdcc7400640 1 -- 172.21.15.86:0/92314765 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6433+0+0 (secure 0 0 0) 0x7fdcb812a000 con 0x7fdcd00cfca0 2024-04-25T14:05:34.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.460+0000 7fdccea00640 1 --2- 172.21.15.86:0/92314765 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fdcac06f5c0 0x7fdcac071ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:34.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.464+0000 7fdccea00640 1 --2- 172.21.15.86:0/92314765 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fdcac06f5c0 0x7fdcac071ad0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0070710 tx=0x7fdcc00077c0 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:34.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.464+0000 7fdcc7400640 1 -- 172.21.15.86:0/92314765 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdcb80c4060 con 0x7fdcd00cfca0 2024-04-25T14:05:34.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:34.660+0000 7fdcd5800640 1 -- 172.21.15.86:0/92314765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "fs1", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fdcd0155480 con 0x7fdcd00cfca0 2024-04-25T14:05:35.567 INFO:tasks.ceph.mgr.y.smithi086.stderr:Exception in thread Thread-5: 2024-04-25T14:05:35.568 INFO:tasks.ceph.mgr.y.smithi086.stderr:Traceback (most recent call last): 2024-04-25T14:05:35.568 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner 2024-04-25T14:05:35.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.568+0000 7fdcc7400640 1 -- 172.21.15.86:0/92314765 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "fs1", "var": "standby_count_wanted", "val": "0"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7fdcb8016110 con 0x7fdcd00cfca0 2024-04-25T14:05:35.574 INFO:tasks.ceph.mgr.y.smithi086.stderr: self.run() 2024-04-25T14:05:35.574 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/lib/python3.10/threading.py", line 1378, in run 2024-04-25T14:05:35.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.572+0000 7fdcd5800640 1 -- 172.21.15.86:0/92314765 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fdcac06f5c0 msgr2=0x7fdcac071ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:35.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.572+0000 7fdcd5800640 1 --2- 172.21.15.86:0/92314765 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fdcac06f5c0 0x7fdcac071ad0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0070710 tx=0x7fdcc00077c0 comp rx=0 tx=0).stop 2024-04-25T14:05:35.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.572+0000 7fdcd5800640 1 -- 172.21.15.86:0/92314765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd00cfca0 msgr2=0x7fdcd009a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:35.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.572+0000 7fdcd5800640 1 --2- 172.21.15.86:0/92314765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd00cfca0 0x7fdcd009a7b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fdcb801ca00 tx=0x7fdcb801ad60 comp rx=0 tx=0).stop 2024-04-25T14:05:35.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.572+0000 7fdcd5800640 1 -- 172.21.15.86:0/92314765 shutdown_connections 2024-04-25T14:05:35.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.572+0000 7fdcd5800640 1 --2- 172.21.15.86:0/92314765 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fdcac06f5c0 0x7fdcac071ad0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:35.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.572+0000 7fdcd5800640 1 --2- 172.21.15.86:0/92314765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcd00d16e0 0x7fdcd015eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:35.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.572+0000 7fdcd5800640 1 --2- 172.21.15.86:0/92314765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd00cfca0 0x7fdcd009a7b0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:35.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.572+0000 7fdcd5800640 1 --2- 172.21.15.86:0/92314765 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcd006c670 0x7fdcd0156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:35.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.572+0000 7fdcd5800640 1 -- 172.21.15.86:0/92314765 >> 172.21.15.86:0/92314765 conn(0x7fdcd00b4650 msgr2=0x7fdcd015bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:35.576 INFO:tasks.ceph.mgr.y.smithi086.stderr: self.function(*self.args, **self.kwargs) 2024-04-25T14:05:35.576 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/share/ceph/mgr/stats/fs/perf_stats.py", line 222, in re_register_queries 2024-04-25T14:05:35.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.572+0000 7fdcd5800640 1 -- 172.21.15.86:0/92314765 shutdown_connections 2024-04-25T14:05:35.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.572+0000 7fdcd5800640 1 -- 172.21.15.86:0/92314765 wait complete. 2024-04-25T14:05:35.577 INFO:tasks.ceph.mgr.y.smithi086.stderr: if self.mx_last_updated >= ua_last_updated: 2024-04-25T14:05:35.577 INFO:tasks.ceph.mgr.y.smithi086.stderr:AttributeError: 'FSPerfStats' object has no attribute 'mx_last_updated' 2024-04-25T14:05:35.598 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T14:05:35.598 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set fs1 allow_standby_replay false 2024-04-25T14:05:35.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.748+0000 7f1dcd200640 1 Processor -- start 2024-04-25T14:05:35.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.748+0000 7f1dcd200640 1 -- start start 2024-04-25T14:05:35.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.748+0000 7f1dcd200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1dc80d1df0 0x7f1dc80cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:35.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.748+0000 7f1dcd200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dc815e5c0 con 0x7f1dc80cdcc0 2024-04-25T14:05:35.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.748+0000 7f1dcd200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dc815e8a0 con 0x7f1dc8070330 2024-04-25T14:05:35.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.748+0000 7f1dcd200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1dc806fd40 con 0x7f1dc80d1df0 2024-04-25T14:05:35.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.748+0000 7f1dc5400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1dc80d1df0 0x7f1dc80cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:35.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.748+0000 7f1dc6800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1dc80cdcc0 0x7f1dc80ce0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36668/0 (socket says 172.21.15.86:36668) 2024-04-25T14:05:35.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.748+0000 7f1dc6800640 1 -- 172.21.15.86:0/2146075264 learned_addr learned my addr 172.21.15.86:0/2146075264 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:35.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.752+0000 7f1dc5400640 1 -- 172.21.15.86:0/2146075264 >> v1:172.21.15.155:6789/0 conn(0x7f1dc8070330 legacy=0x7f1dc80d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:35.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.752+0000 7f1dc5400640 1 -- 172.21.15.86:0/2146075264 >> v1:172.21.15.86:6789/0 conn(0x7f1dc80cdcc0 legacy=0x7f1dc80ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:35.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.752+0000 7f1dc5400640 1 -- 172.21.15.86:0/2146075264 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc815eb80 con 0x7f1dc80d1df0 2024-04-25T14:05:35.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.752+0000 7f1dc5400640 1 --2- 172.21.15.86:0/2146075264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dc80d1df0 0x7f1dc80cd780 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1db40097f0 tx=0x7f1db401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac20f0031c59a79e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:35.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.752+0000 7f1dc4a00640 1 -- 172.21.15.86:0/2146075264 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db40b0070 con 0x7f1dc80d1df0 2024-04-25T14:05:35.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.752+0000 7f1dc4a00640 1 -- 172.21.15.86:0/2146075264 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db40ab600 con 0x7f1dc80d1df0 2024-04-25T14:05:35.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.752+0000 7f1dc4a00640 1 -- 172.21.15.86:0/2146075264 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db40ab9d0 con 0x7f1dc80d1df0 2024-04-25T14:05:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.752+0000 7f1dcd200640 1 -- 172.21.15.86:0/2146075264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dc80d1df0 msgr2=0x7f1dc80cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.752+0000 7f1dcd200640 1 --2- 172.21.15.86:0/2146075264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dc80d1df0 0x7f1dc80cd780 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1db40097f0 tx=0x7f1db401c3e0 comp rx=0 tx=0).stop 2024-04-25T14:05:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.756+0000 7f1dcd200640 1 -- 172.21.15.86:0/2146075264 shutdown_connections 2024-04-25T14:05:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.756+0000 7f1dcd200640 1 --2- 172.21.15.86:0/2146075264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dc80d1df0 0x7f1dc80cd780 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.756+0000 7f1dcd200640 1 -- 172.21.15.86:0/2146075264 >> 172.21.15.86:0/2146075264 conn(0x7f1dc80b4650 msgr2=0x7f1dc80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.756+0000 7f1dcd200640 1 -- 172.21.15.86:0/2146075264 shutdown_connections 2024-04-25T14:05:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.756+0000 7f1dcd200640 1 -- 172.21.15.86:0/2146075264 wait complete. 2024-04-25T14:05:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.756+0000 7f1dcd200640 1 Processor -- start 2024-04-25T14:05:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.756+0000 7f1dcd200640 1 -- start start 2024-04-25T14:05:35.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.756+0000 7f1dcd200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1dc8070330 0x7f1dc8156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:35.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.756+0000 7f1dcd200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dc80cdcc0 0x7f1dc80cff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:35.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.756+0000 7f1dcd200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1dc809aa30 0x7f1dc80d0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:35.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.756+0000 7f1dcd200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc80708c0 con 0x7f1dc80cdcc0 2024-04-25T14:05:35.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.756+0000 7f1dcd200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc8070740 con 0x7f1dc809aa30 2024-04-25T14:05:35.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.756+0000 7f1dcd200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc816bfd0 con 0x7f1dc8070330 2024-04-25T14:05:35.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.760+0000 7f1dc5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dc80cdcc0 0x7f1dc80cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:35.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.760+0000 7f1dc5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dc80cdcc0 0x7f1dc80cff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52096/0 (socket says 172.21.15.86:52096) 2024-04-25T14:05:35.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.760+0000 7f1dc5400640 1 -- 172.21.15.86:0/43537533 learned_addr learned my addr 172.21.15.86:0/43537533 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:35.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.760+0000 7f1dc5e00640 1 --2- 172.21.15.86:0/43537533 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1dc8070330 0x7f1dc8156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:35.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.760+0000 7f1dc6800640 1 --2- 172.21.15.86:0/43537533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1dc809aa30 0x7f1dc80d0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:35.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.760+0000 7f1dc5400640 1 -- 172.21.15.86:0/43537533 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1dc8070330 msgr2=0x7f1dc8156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:35.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.760+0000 7f1dc5400640 1 --2- 172.21.15.86:0/43537533 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1dc8070330 0x7f1dc8156d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:35.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.760+0000 7f1dc5400640 1 -- 172.21.15.86:0/43537533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1dc809aa30 msgr2=0x7f1dc80d0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:35.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.760+0000 7f1dc5400640 1 --2- 172.21.15.86:0/43537533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1dc809aa30 0x7f1dc80d0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:35.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.760+0000 7f1dc5400640 1 -- 172.21.15.86:0/43537533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc80d09d0 con 0x7f1dc80cdcc0 2024-04-25T14:05:35.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.760+0000 7f1dc6800640 1 --2- 172.21.15.86:0/43537533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1dc809aa30 0x7f1dc80d0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:35.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.760+0000 7f1dc5400640 1 --2- 172.21.15.86:0/43537533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dc80cdcc0 0x7f1dc80cff50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1db40097f0 tx=0x7f1db4019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:35.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.760+0000 7f1dbe000640 1 -- 172.21.15.86:0/43537533 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db40bb070 con 0x7f1dc80cdcc0 2024-04-25T14:05:35.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.760+0000 7f1dbe000640 1 -- 172.21.15.86:0/43537533 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db40b7680 con 0x7f1dc80cdcc0 2024-04-25T14:05:35.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.760+0000 7f1dcd200640 1 -- 172.21.15.86:0/43537533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc815a270 con 0x7f1dc80cdcc0 2024-04-25T14:05:35.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.760+0000 7f1dbe000640 1 -- 172.21.15.86:0/43537533 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db40b0040 con 0x7f1dc80cdcc0 2024-04-25T14:05:35.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.760+0000 7f1dcd200640 1 -- 172.21.15.86:0/43537533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc8159250 con 0x7f1dc80cdcc0 2024-04-25T14:05:35.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.760+0000 7f1dcd200640 1 -- 172.21.15.86:0/43537533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dc8069d00 con 0x7f1dc80cdcc0 2024-04-25T14:05:35.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.760+0000 7f1dbe000640 1 -- 172.21.15.86:0/43537533 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f1db40be070 con 0x7f1dc80cdcc0 2024-04-25T14:05:35.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.764+0000 7f1dbe000640 1 --2- 172.21.15.86:0/43537533 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f1da806f570 0x7f1da8071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:35.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.764+0000 7f1dbe000640 1 -- 172.21.15.86:0/43537533 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6433+0+0 (secure 0 0 0) 0x7f1db4129d90 con 0x7f1dc80cdcc0 2024-04-25T14:05:35.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.764+0000 7f1dc5e00640 1 --2- 172.21.15.86:0/43537533 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f1da806f570 0x7f1da8071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:35.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.764+0000 7f1dc5e00640 1 --2- 172.21.15.86:0/43537533 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f1da806f570 0x7f1da8071a80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8072200 tx=0x7f1db8009040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:35.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.768+0000 7f1dbe000640 1 -- 172.21.15.86:0/43537533 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1dc80d09d0 con 0x7f1dc80cdcc0 2024-04-25T14:05:35.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:35.956+0000 7f1dcd200640 1 -- 172.21.15.86:0/43537533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "fs1", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1dc8076bf0 con 0x7f1dc80cdcc0 2024-04-25T14:05:36.496 INFO:tasks.ceph.mds.c.smithi086.stderr:2024-04-25T14:05:36.492+0000 7ff8bc400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T14:05:36.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.572+0000 7f1dbe000640 1 -- 172.21.15.86:0/43537533 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "fs1", "var": "allow_standby_replay", "val": "false"}]=0 v26) ==== 119+0+0 (secure 0 0 0) 0x7f1db4016090 con 0x7f1dc80cdcc0 2024-04-25T14:05:36.574 INFO:tasks.ceph.mgr.y.smithi086.stderr:Exception in thread Thread-6: 2024-04-25T14:05:36.574 INFO:tasks.ceph.mgr.y.smithi086.stderr:Traceback (most recent call last): 2024-04-25T14:05:36.574 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner 2024-04-25T14:05:36.575 INFO:tasks.ceph.mgr.y.smithi086.stderr: self.run() 2024-04-25T14:05:36.575 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/lib/python3.10/threading.py", line 1378, in run 2024-04-25T14:05:36.575 INFO:tasks.ceph.mgr.y.smithi086.stderr: self.function(*self.args, **self.kwargs) 2024-04-25T14:05:36.575 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/share/ceph/mgr/stats/fs/perf_stats.py", line 222, in re_register_queries 2024-04-25T14:05:36.575 INFO:tasks.ceph.mgr.y.smithi086.stderr: if self.mx_last_updated >= ua_last_updated: 2024-04-25T14:05:36.576 INFO:tasks.ceph.mgr.y.smithi086.stderr:AttributeError: 'FSPerfStats' object has no attribute 'mx_last_updated' 2024-04-25T14:05:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.576+0000 7f1dcd200640 1 -- 172.21.15.86:0/43537533 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f1da806f570 msgr2=0x7f1da8071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.576+0000 7f1dcd200640 1 --2- 172.21.15.86:0/43537533 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f1da806f570 0x7f1da8071a80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8072200 tx=0x7f1db8009040 comp rx=0 tx=0).stop 2024-04-25T14:05:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.576+0000 7f1dcd200640 1 -- 172.21.15.86:0/43537533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dc80cdcc0 msgr2=0x7f1dc80cff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.576+0000 7f1dcd200640 1 --2- 172.21.15.86:0/43537533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dc80cdcc0 0x7f1dc80cff50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1db40097f0 tx=0x7f1db4019180 comp rx=0 tx=0).stop 2024-04-25T14:05:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.576+0000 7f1dcd200640 1 -- 172.21.15.86:0/43537533 shutdown_connections 2024-04-25T14:05:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.576+0000 7f1dcd200640 1 --2- 172.21.15.86:0/43537533 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f1da806f570 0x7f1da8071a80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.576+0000 7f1dcd200640 1 --2- 172.21.15.86:0/43537533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1dc809aa30 0x7f1dc80d0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.576+0000 7f1dcd200640 1 --2- 172.21.15.86:0/43537533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dc80cdcc0 0x7f1dc80cff50 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.576+0000 7f1dcd200640 1 --2- 172.21.15.86:0/43537533 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1dc8070330 0x7f1dc8156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:36.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.576+0000 7f1dcd200640 1 -- 172.21.15.86:0/43537533 >> 172.21.15.86:0/43537533 conn(0x7f1dc80b4650 msgr2=0x7f1dc815c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:36.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.576+0000 7f1dcd200640 1 -- 172.21.15.86:0/43537533 shutdown_connections 2024-04-25T14:05:36.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.576+0000 7f1dcd200640 1 -- 172.21.15.86:0/43537533 wait complete. 2024-04-25T14:05:36.600 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'fs1' 2024-04-25T14:05:36.600 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:05:36.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.748+0000 7f7d6fe00640 1 Processor -- start 2024-04-25T14:05:36.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.748+0000 7f7d6fe00640 1 -- start start 2024-04-25T14:05:36.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.748+0000 7f7d6fe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7d680cdd70 0x7f7d680ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:36.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.748+0000 7f7d6fe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d68070e20 0x7f7d68071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:36.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.748+0000 7f7d6fe00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d68071770 con 0x7f7d6806e700 2024-04-25T14:05:36.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.748+0000 7f7d6fe00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f7d6806d940 con 0x7f7d680cdd70 2024-04-25T14:05:36.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.748+0000 7f7d6fe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7d6806dc20 con 0x7f7d68070e20 2024-04-25T14:05:36.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.748+0000 7f7d6d400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7d680cdd70 0x7f7d680ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:36.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.748+0000 7f7d6ca00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d68070e20 0x7f7d68071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:36.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.748+0000 7f7d6de00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f7d6806e700 0x7f7d6806eb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:55642/0 (socket says 172.21.15.86:55642) 2024-04-25T14:05:36.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.748+0000 7f7d6d400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7d680cdd70 0x7f7d680ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:50914/0 (socket says 172.21.15.86:50914) 2024-04-25T14:05:36.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.748+0000 7f7d6de00640 1 -- 172.21.15.86:0/1312584333 learned_addr learned my addr 172.21.15.86:0/1312584333 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:36.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.748+0000 7f7d6ca00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d68070e20 0x7f7d68071230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:35674/0 (socket says 172.21.15.86:35674) 2024-04-25T14:05:36.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.752+0000 7f7d6ca00640 1 -- 172.21.15.86:0/1312584333 >> v2:172.21.15.155:3300/0 conn(0x7f7d680cdd70 msgr2=0x7f7d680ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:36.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.752+0000 7f7d6ca00640 1 --2- 172.21.15.86:0/1312584333 >> v2:172.21.15.155:3300/0 conn(0x7f7d680cdd70 0x7f7d680ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:36.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.752+0000 7f7d6ca00640 1 -- 172.21.15.86:0/1312584333 >> v1:172.21.15.155:6789/0 conn(0x7f7d6806e700 legacy=0x7f7d6806eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:36.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.752+0000 7f7d6ca00640 1 -- 172.21.15.86:0/1312584333 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6806df00 con 0x7f7d68070e20 2024-04-25T14:05:36.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.752+0000 7f7d6d400640 1 --2- 172.21.15.86:0/1312584333 >> v2:172.21.15.155:3300/0 conn(0x7f7d680cdd70 0x7f7d680ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:36.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.752+0000 7f7d67e00640 1 -- 172.21.15.86:0/1312584333 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2079379525 0 0) 0x7f7d600014a0 con 0x7f7d6806e700 2024-04-25T14:05:36.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.752+0000 7f7d6ca00640 1 --2- 172.21.15.86:0/1312584333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d68070e20 0x7f7d68071230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7d540097a0 tx=0x7f7d5401c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab6ff6ed59bba632 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:36.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.752+0000 7f7d67e00640 1 -- 172.21.15.86:0/1312584333 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d540b0070 con 0x7f7d68070e20 2024-04-25T14:05:36.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.752+0000 7f7d67e00640 1 -- 172.21.15.86:0/1312584333 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d540ab600 con 0x7f7d68070e20 2024-04-25T14:05:36.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.752+0000 7f7d67e00640 1 -- 172.21.15.86:0/1312584333 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d540ab9d0 con 0x7f7d68070e20 2024-04-25T14:05:36.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.756+0000 7f7d6fe00640 1 -- 172.21.15.86:0/1312584333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d68070e20 msgr2=0x7f7d68071230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:36.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.756+0000 7f7d6fe00640 1 --2- 172.21.15.86:0/1312584333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d68070e20 0x7f7d68071230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7d540097a0 tx=0x7f7d5401c3f0 comp rx=0 tx=0).stop 2024-04-25T14:05:36.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.756+0000 7f7d6fe00640 1 -- 172.21.15.86:0/1312584333 shutdown_connections 2024-04-25T14:05:36.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.756+0000 7f7d6fe00640 1 --2- 172.21.15.86:0/1312584333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d68070e20 0x7f7d68071230 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:36.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.756+0000 7f7d6fe00640 1 --2- 172.21.15.86:0/1312584333 >> v2:172.21.15.155:3300/0 conn(0x7f7d680cdd70 0x7f7d680ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:36.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.756+0000 7f7d6fe00640 1 -- 172.21.15.86:0/1312584333 >> 172.21.15.86:0/1312584333 conn(0x7f7d680cffc0 msgr2=0x7f7d680d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:36.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.756+0000 7f7d6fe00640 1 -- 172.21.15.86:0/1312584333 shutdown_connections 2024-04-25T14:05:36.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.756+0000 7f7d6fe00640 1 -- 172.21.15.86:0/1312584333 wait complete. 2024-04-25T14:05:36.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.756+0000 7f7d6fe00640 1 Processor -- start 2024-04-25T14:05:36.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.756+0000 7f7d6fe00640 1 -- start start 2024-04-25T14:05:36.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.760+0000 7f7d6fe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d6806e700 0x7f7d680c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:36.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.760+0000 7f7d6fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d68070e20 0x7f7d680c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:36.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.760+0000 7f7d6fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d680cdd70 0x7f7d68160a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:36.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.760+0000 7f7d6fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6816cc00 con 0x7f7d680cdd70 2024-04-25T14:05:36.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.760+0000 7f7d6fe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7d68164400 con 0x7f7d68070e20 2024-04-25T14:05:36.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.760+0000 7f7d6fe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7d6816cd80 con 0x7f7d6806e700 2024-04-25T14:05:36.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.760+0000 7f7d6de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d680cdd70 0x7f7d68160a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:36.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.760+0000 7f7d6de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d680cdd70 0x7f7d68160a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52098/0 (socket says 172.21.15.86:52098) 2024-04-25T14:05:36.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.760+0000 7f7d6de00640 1 -- 172.21.15.86:0/1096371904 learned_addr learned my addr 172.21.15.86:0/1096371904 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:36.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.760+0000 7f7d6ca00640 1 --2- 172.21.15.86:0/1096371904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d68070e20 0x7f7d680c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:36.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.760+0000 7f7d6d400640 1 --2- 172.21.15.86:0/1096371904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d6806e700 0x7f7d680c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:36.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.760+0000 7f7d6de00640 1 -- 172.21.15.86:0/1096371904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d6806e700 msgr2=0x7f7d680c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:36.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.760+0000 7f7d6de00640 1 --2- 172.21.15.86:0/1096371904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d6806e700 0x7f7d680c89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:36.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.760+0000 7f7d6de00640 1 -- 172.21.15.86:0/1096371904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d68070e20 msgr2=0x7f7d680c8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:36.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.760+0000 7f7d6de00640 1 --2- 172.21.15.86:0/1096371904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d68070e20 0x7f7d680c8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:36.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.760+0000 7f7d6de00640 1 -- 172.21.15.86:0/1096371904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d68160380 con 0x7f7d680cdd70 2024-04-25T14:05:36.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.760+0000 7f7d6de00640 1 --2- 172.21.15.86:0/1096371904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d680cdd70 0x7f7d68160a70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000c700 tx=0x7f7d6000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:36.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.760+0000 7f7d65600640 1 -- 172.21.15.86:0/1096371904 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60018070 con 0x7f7d680cdd70 2024-04-25T14:05:36.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.760+0000 7f7d65600640 1 -- 172.21.15.86:0/1096371904 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d6000ce10 con 0x7f7d680cdd70 2024-04-25T14:05:36.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.760+0000 7f7d65600640 1 -- 172.21.15.86:0/1096371904 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6000a860 con 0x7f7d680cdd70 2024-04-25T14:05:36.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.760+0000 7f7d6fe00640 1 -- 172.21.15.86:0/1096371904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d680c9df0 con 0x7f7d680cdd70 2024-04-25T14:05:36.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.760+0000 7f7d6fe00640 1 -- 172.21.15.86:0/1096371904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d680c95f0 con 0x7f7d680cdd70 2024-04-25T14:05:36.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.760+0000 7f7d6fe00640 1 -- 172.21.15.86:0/1096371904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d680720e0 con 0x7f7d680cdd70 2024-04-25T14:05:36.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.764+0000 7f7d65600640 1 -- 172.21.15.86:0/1096371904 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f7d6000aa00 con 0x7f7d680cdd70 2024-04-25T14:05:36.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.764+0000 7f7d65600640 1 --2- 172.21.15.86:0/1096371904 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f7d4806f5a0 0x7f7d48071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:36.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.764+0000 7f7d65600640 1 -- 172.21.15.86:0/1096371904 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6433+0+0 (secure 0 0 0) 0x7f7d6008eb90 con 0x7f7d680cdd70 2024-04-25T14:05:36.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.764+0000 7f7d6d400640 1 --2- 172.21.15.86:0/1096371904 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f7d4806f5a0 0x7f7d48071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:36.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.764+0000 7f7d6d400640 1 --2- 172.21.15.86:0/1096371904 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f7d4806f5a0 0x7f7d48071ab0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009580 tx=0x7f7d5c009190 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:36.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.768+0000 7f7d65600640 1 -- 172.21.15.86:0/1096371904 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7d68071770 con 0x7f7d680cdd70 2024-04-25T14:05:36.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.956+0000 7f7d6fe00640 1 -- 172.21.15.86:0/1096371904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d68152200 con 0x7f7d680cdd70 2024-04-25T14:05:36.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.956+0000 7f7d65600640 1 -- 172.21.15.86:0/1096371904 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5181 (secure 0 0 0) 0x7f7d68152200 con 0x7f7d680cdd70 2024-04-25T14:05:36.960 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:36.960 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3546153119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3546153119},{"type":"v1","addr":"172.21.15.155:6834","nonce":3546153119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5283,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3499340224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3499340224},{"type":"v1","addr":"172.21.15.86:6836","nonce":3499340224}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5284,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1466978742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1466978742},{"type":"v1","addr":"172.21.15.155:6835","nonce":1466978742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:32.946320+0000","modified":"2024-04-25T14:05:35.961141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6837/615772001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":615772001},{"type":"v1","addr":"172.21.15.86:6837","nonce":615772001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"fs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5712,"qdb_cluster":[5712]},"id":3}]} 2024-04-25T14:05:36.960 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 26 2024-04-25T14:05:36.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.960+0000 7f7d6fe00640 1 -- 172.21.15.86:0/1096371904 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f7d4806f5a0 msgr2=0x7f7d48071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:36.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.960+0000 7f7d6fe00640 1 --2- 172.21.15.86:0/1096371904 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f7d4806f5a0 0x7f7d48071ab0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009580 tx=0x7f7d5c009190 comp rx=0 tx=0).stop 2024-04-25T14:05:36.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.960+0000 7f7d6fe00640 1 -- 172.21.15.86:0/1096371904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d680cdd70 msgr2=0x7f7d68160a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:36.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.960+0000 7f7d6fe00640 1 --2- 172.21.15.86:0/1096371904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d680cdd70 0x7f7d68160a70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000c700 tx=0x7f7d6000cbc0 comp rx=0 tx=0).stop 2024-04-25T14:05:36.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.960+0000 7f7d6fe00640 1 -- 172.21.15.86:0/1096371904 shutdown_connections 2024-04-25T14:05:36.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.960+0000 7f7d6fe00640 1 --2- 172.21.15.86:0/1096371904 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f7d4806f5a0 0x7f7d48071ab0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:36.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.960+0000 7f7d6fe00640 1 --2- 172.21.15.86:0/1096371904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d680cdd70 0x7f7d68160a70 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:36.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.960+0000 7f7d6fe00640 1 --2- 172.21.15.86:0/1096371904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d68070e20 0x7f7d680c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:36.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.960+0000 7f7d6fe00640 1 --2- 172.21.15.86:0/1096371904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d6806e700 0x7f7d680c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:36.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.960+0000 7f7d6fe00640 1 -- 172.21.15.86:0/1096371904 >> 172.21.15.86:0/1096371904 conn(0x7f7d680cffc0 msgr2=0x7f7d680cf210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:36.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.960+0000 7f7d6fe00640 1 -- 172.21.15.86:0/1096371904 shutdown_connections 2024-04-25T14:05:36.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:36.960+0000 7f7d6fe00640 1 -- 172.21.15.86:0/1096371904 wait complete. 2024-04-25T14:05:36.982 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:05:37.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.124+0000 7ff147a00640 1 Processor -- start 2024-04-25T14:05:37.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.124+0000 7ff147a00640 1 -- start start 2024-04-25T14:05:37.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.128+0000 7ff147a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff140159aa0 0x7ff140159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:37.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.128+0000 7ff147a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff14015a3f0 0x7ff14006c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:37.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.128+0000 7ff147a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1400b4c40 con 0x7ff140070750 2024-04-25T14:05:37.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.128+0000 7ff147a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff14015a800 con 0x7ff140159aa0 2024-04-25T14:05:37.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.128+0000 7ff147a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff1400b4ee0 con 0x7ff14015a3f0 2024-04-25T14:05:37.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.128+0000 7ff137e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff140159aa0 0x7ff140159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:37.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.128+0000 7ff145000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff140070750 0x7ff1400b03b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36672/0 (socket says 172.21.15.86:36672) 2024-04-25T14:05:37.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.128+0000 7ff145000640 1 -- 172.21.15.86:0/1473782874 learned_addr learned my addr 172.21.15.86:0/1473782874 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:37.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.128+0000 7ff145a00640 1 --2- 172.21.15.86:0/1473782874 >> v2:172.21.15.155:3300/0 conn(0x7ff14015a3f0 0x7ff14006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:37.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.128+0000 7ff137e00640 1 -- 172.21.15.86:0/1473782874 >> v2:172.21.15.155:3300/0 conn(0x7ff14015a3f0 msgr2=0x7ff14006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:37.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.128+0000 7ff137e00640 1 --2- 172.21.15.86:0/1473782874 >> v2:172.21.15.155:3300/0 conn(0x7ff14015a3f0 0x7ff14006c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:37.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.128+0000 7ff137e00640 1 -- 172.21.15.86:0/1473782874 >> v1:172.21.15.86:6789/0 conn(0x7ff140070750 legacy=0x7ff1400b03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:37.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.128+0000 7ff137e00640 1 -- 172.21.15.86:0/1473782874 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1400b51c0 con 0x7ff140159aa0 2024-04-25T14:05:37.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.128+0000 7ff13fe00640 1 -- 172.21.15.86:0/1473782874 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1727435122 0 0) 0x7ff1400b4c40 con 0x7ff140070750 2024-04-25T14:05:37.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.132+0000 7ff137e00640 1 --2- 172.21.15.86:0/1473782874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff140159aa0 0x7ff140159eb0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff1400b6760 tx=0x7ff12801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407d2a72b0e9f0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:37.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.132+0000 7ff13fe00640 1 -- 172.21.15.86:0/1473782874 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1280b0070 con 0x7ff140159aa0 2024-04-25T14:05:37.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.132+0000 7ff13fe00640 1 -- 172.21.15.86:0/1473782874 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1280ab600 con 0x7ff140159aa0 2024-04-25T14:05:37.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.132+0000 7ff13fe00640 1 -- 172.21.15.86:0/1473782874 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1280ab9d0 con 0x7ff140159aa0 2024-04-25T14:05:37.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.132+0000 7ff147a00640 1 -- 172.21.15.86:0/1473782874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff140159aa0 msgr2=0x7ff140159eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:37.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.132+0000 7ff147a00640 1 --2- 172.21.15.86:0/1473782874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff140159aa0 0x7ff140159eb0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff1400b6760 tx=0x7ff12801c3f0 comp rx=0 tx=0).stop 2024-04-25T14:05:37.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.132+0000 7ff147a00640 1 -- 172.21.15.86:0/1473782874 shutdown_connections 2024-04-25T14:05:37.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.132+0000 7ff147a00640 1 --2- 172.21.15.86:0/1473782874 >> v2:172.21.15.155:3300/0 conn(0x7ff14015a3f0 0x7ff14006c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:37.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.132+0000 7ff147a00640 1 --2- 172.21.15.86:0/1473782874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff140159aa0 0x7ff140159eb0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:37.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.132+0000 7ff147a00640 1 -- 172.21.15.86:0/1473782874 >> 172.21.15.86:0/1473782874 conn(0x7ff1400d19b0 msgr2=0x7ff1400d1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:37.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.132+0000 7ff147a00640 1 -- 172.21.15.86:0/1473782874 shutdown_connections 2024-04-25T14:05:37.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.132+0000 7ff147a00640 1 -- 172.21.15.86:0/1473782874 wait complete. 2024-04-25T14:05:37.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.136+0000 7ff147a00640 1 Processor -- start 2024-04-25T14:05:37.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.136+0000 7ff147a00640 1 -- start start 2024-04-25T14:05:37.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.136+0000 7ff147a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff140070750 0x7ff140158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:37.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.136+0000 7ff147a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff140159aa0 0x7ff1400aee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:37.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.136+0000 7ff147a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff14015a3f0 0x7ff1400af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:37.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.136+0000 7ff147a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff1401604f0 con 0x7ff140070750 2024-04-25T14:05:37.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.136+0000 7ff147a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff140160370 con 0x7ff140159aa0 2024-04-25T14:05:37.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.136+0000 7ff147a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff140160670 con 0x7ff14015a3f0 2024-04-25T14:05:37.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.136+0000 7ff145000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff140070750 0x7ff140158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:37.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.136+0000 7ff145000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff140070750 0x7ff140158dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52124/0 (socket says 172.21.15.86:52124) 2024-04-25T14:05:37.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.136+0000 7ff145000640 1 -- 172.21.15.86:0/1863169173 learned_addr learned my addr 172.21.15.86:0/1863169173 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:37.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.136+0000 7ff137e00640 1 --2- 172.21.15.86:0/1863169173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff140159aa0 0x7ff1400aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:37.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.136+0000 7ff145a00640 1 --2- 172.21.15.86:0/1863169173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff14015a3f0 0x7ff1400af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:37.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.136+0000 7ff145000640 1 -- 172.21.15.86:0/1863169173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff14015a3f0 msgr2=0x7ff1400af3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:37.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.136+0000 7ff145000640 1 --2- 172.21.15.86:0/1863169173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff14015a3f0 0x7ff1400af3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:37.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.136+0000 7ff145000640 1 -- 172.21.15.86:0/1863169173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff140159aa0 msgr2=0x7ff1400aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:37.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.136+0000 7ff145000640 1 --2- 172.21.15.86:0/1863169173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff140159aa0 0x7ff1400aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:37.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.136+0000 7ff145000640 1 -- 172.21.15.86:0/1863169173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff140151700 con 0x7ff140070750 2024-04-25T14:05:37.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.136+0000 7ff145000640 1 --2- 172.21.15.86:0/1863169173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff140070750 0x7ff140158dc0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff13000c490 tx=0x7ff13000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:37.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.136+0000 7ff13d600640 1 -- 172.21.15.86:0/1863169173 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff130017070 con 0x7ff140070750 2024-04-25T14:05:37.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.136+0000 7ff13d600640 1 -- 172.21.15.86:0/1863169173 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff130005cf0 con 0x7ff140070750 2024-04-25T14:05:37.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.136+0000 7ff147a00640 1 -- 172.21.15.86:0/1863169173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff140158620 con 0x7ff140070750 2024-04-25T14:05:37.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.136+0000 7ff13d600640 1 -- 172.21.15.86:0/1863169173 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1300072f0 con 0x7ff140070750 2024-04-25T14:05:37.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.136+0000 7ff147a00640 1 -- 172.21.15.86:0/1863169173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff140151c50 con 0x7ff140070750 2024-04-25T14:05:37.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.140+0000 7ff147a00640 1 -- 172.21.15.86:0/1863169173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1400748a0 con 0x7ff140070750 2024-04-25T14:05:37.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.140+0000 7ff13d600640 1 -- 172.21.15.86:0/1863169173 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7ff13001a070 con 0x7ff140070750 2024-04-25T14:05:37.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.140+0000 7ff13d600640 1 --2- 172.21.15.86:0/1863169173 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7ff11c06f5a0 0x7ff11c071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:37.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.140+0000 7ff13d600640 1 -- 172.21.15.86:0/1863169173 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6433+0+0 (secure 0 0 0) 0x7ff13008ea90 con 0x7ff140070750 2024-04-25T14:05:37.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.140+0000 7ff137e00640 1 --2- 172.21.15.86:0/1863169173 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7ff11c06f5a0 0x7ff11c071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:37.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.140+0000 7ff137e00640 1 --2- 172.21.15.86:0/1863169173 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7ff11c06f5a0 0x7ff11c071ab0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff1280ad000 tx=0x7ff1280a6000 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:37.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.144+0000 7ff13d600640 1 -- 172.21.15.86:0/1863169173 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff140158620 con 0x7ff140070750 2024-04-25T14:05:37.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.304+0000 7ff147a00640 1 -- 172.21.15.86:0/1863169173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff140076bf0 con 0x7ff140070750 2024-04-25T14:05:37.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.304+0000 7ff13d600640 1 -- 172.21.15.86:0/1863169173 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+17435 (secure 0 0 0) 0x7ff13005af30 con 0x7ff140070750 2024-04-25T14:05:37.306 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:37.306 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":46,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:05:34.556510+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"fs1_metadata","create_time":"2024-04-25T14:05:30.939601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"fs1"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"fs1_data","create_time":"2024-04-25T14:05:31.898262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"fs1"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6835/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:0/2073988773":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6833/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6832/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/3558294881":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/1843278167":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6837/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:0/2602032945":"2024-04-26T14:05:21.460855+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:05:37.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.304+0000 7ff147a00640 1 -- 172.21.15.86:0/1863169173 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7ff11c06f5a0 msgr2=0x7ff11c071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:37.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.304+0000 7ff147a00640 1 --2- 172.21.15.86:0/1863169173 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7ff11c06f5a0 0x7ff11c071ab0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff1280ad000 tx=0x7ff1280a6000 comp rx=0 tx=0).stop 2024-04-25T14:05:37.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.304+0000 7ff147a00640 1 -- 172.21.15.86:0/1863169173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff140070750 msgr2=0x7ff140158dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:37.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.304+0000 7ff147a00640 1 --2- 172.21.15.86:0/1863169173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff140070750 0x7ff140158dc0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff13000c490 tx=0x7ff13000c950 comp rx=0 tx=0).stop 2024-04-25T14:05:37.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.304+0000 7ff147a00640 1 -- 172.21.15.86:0/1863169173 shutdown_connections 2024-04-25T14:05:37.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.304+0000 7ff147a00640 1 --2- 172.21.15.86:0/1863169173 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7ff11c06f5a0 0x7ff11c071ab0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:37.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.304+0000 7ff147a00640 1 --2- 172.21.15.86:0/1863169173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff14015a3f0 0x7ff1400af3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:37.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.304+0000 7ff147a00640 1 --2- 172.21.15.86:0/1863169173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff140159aa0 0x7ff1400aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:37.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.304+0000 7ff147a00640 1 --2- 172.21.15.86:0/1863169173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff140070750 0x7ff140158dc0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:37.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.304+0000 7ff147a00640 1 -- 172.21.15.86:0/1863169173 >> 172.21.15.86:0/1863169173 conn(0x7ff1400d19b0 msgr2=0x7ff140155df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:37.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.304+0000 7ff147a00640 1 -- 172.21.15.86:0/1863169173 shutdown_connections 2024-04-25T14:05:37.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.308+0000 7ff147a00640 1 -- 172.21.15.86:0/1863169173 wait complete. 2024-04-25T14:05:37.328 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T14:05:37.329 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:05:37.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.472+0000 7f82eba00640 1 Processor -- start 2024-04-25T14:05:37.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.472+0000 7f82eba00640 1 -- start start 2024-04-25T14:05:37.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82eba00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82e406e9b0 0x7f82e415ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:37.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82eba00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82e406c710 con 0x7f82e40d2360 2024-04-25T14:05:37.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82eba00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82e406c9f0 con 0x7f82e406d7b0 2024-04-25T14:05:37.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82eba00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f82e406ccd0 con 0x7f82e406e9b0 2024-04-25T14:05:37.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82dbe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f82e40d2360 0x7f82e40d2770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36684/0 (socket says 172.21.15.86:36684) 2024-04-25T14:05:37.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82dbe00640 1 -- 172.21.15.86:0/315816741 learned_addr learned my addr 172.21.15.86:0/315816741 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:37.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82e9000640 1 --2- 172.21.15.86:0/315816741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82e406e9b0 0x7f82e415ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:37.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82e3e00640 1 -- 172.21.15.86:0/315816741 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3810179811 0 0) 0x7f82e406c710 con 0x7f82e40d2360 2024-04-25T14:05:37.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82e3e00640 1 -- 172.21.15.86:0/315816741 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82c8003530 con 0x7f82e40d2360 2024-04-25T14:05:37.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82e3e00640 1 -- 172.21.15.86:0/315816741 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 169761018 0 0) 0x7f82e406c9f0 con 0x7f82e406d7b0 2024-04-25T14:05:37.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82e3e00640 1 -- 172.21.15.86:0/315816741 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82e406c710 con 0x7f82e406d7b0 2024-04-25T14:05:37.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82e3e00640 1 -- 172.21.15.86:0/315816741 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3834128694 0 0) 0x7f82c8003530 con 0x7f82e40d2360 2024-04-25T14:05:37.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82e3e00640 1 -- 172.21.15.86:0/315816741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82e406e9b0 msgr2=0x7f82e415ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:37.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82e3e00640 1 --2- 172.21.15.86:0/315816741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82e406e9b0 0x7f82e415ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:37.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82e3e00640 1 -- 172.21.15.86:0/315816741 >> v1:172.21.15.155:6789/0 conn(0x7f82e406d7b0 legacy=0x7f82e406dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:37.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82e3e00640 1 -- 172.21.15.86:0/315816741 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e406cfb0 con 0x7f82e40d2360 2024-04-25T14:05:37.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82e3e00640 1 -- 172.21.15.86:0/315816741 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f82cc001680 con 0x7f82e40d2360 2024-04-25T14:05:37.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82e3e00640 1 -- 172.21.15.86:0/315816741 >> v1:172.21.15.86:6789/0 conn(0x7f82e40d2360 legacy=0x7f82e40d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:37.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82e3e00640 1 --2- 172.21.15.86:0/315816741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82c8003bf0 0x7f82c8004040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:37.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82e3e00640 1 --2- 172.21.15.86:0/315816741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82c8004810 0x7f82c8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:37.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82e3e00640 1 --2- 172.21.15.86:0/315816741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82e406e9b0 0x7f82c8005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:37.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82e3e00640 1 -- 172.21.15.86:0/315816741 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82e406ccd0 con 0x7f82c8004810 2024-04-25T14:05:37.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82e3e00640 1 -- 172.21.15.86:0/315816741 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f82c800bb60 con 0x7f82c8003bf0 2024-04-25T14:05:37.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82e3e00640 1 -- 172.21.15.86:0/315816741 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f82c800bd60 con 0x7f82e406e9b0 2024-04-25T14:05:37.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82dbe00640 1 --2- 172.21.15.86:0/315816741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82c8004810 0x7f82c8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:37.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82e9000640 1 --2- 172.21.15.86:0/315816741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82c8003bf0 0x7f82c8004040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:37.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82e9a00640 1 --2- 172.21.15.86:0/315816741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82e406e9b0 0x7f82c8005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:37.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82dbe00640 1 -- 172.21.15.86:0/315816741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82e406e9b0 msgr2=0x7f82c8005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:37.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82dbe00640 1 --2- 172.21.15.86:0/315816741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82e406e9b0 0x7f82c8005280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:37.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82dbe00640 1 -- 172.21.15.86:0/315816741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82c8003bf0 msgr2=0x7f82c8004040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:37.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82dbe00640 1 --2- 172.21.15.86:0/315816741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82c8003bf0 0x7f82c8004040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:37.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.476+0000 7f82dbe00640 1 -- 172.21.15.86:0/315816741 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c8003530 con 0x7f82c8004810 2024-04-25T14:05:37.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.480+0000 7f82dbe00640 1 --2- 172.21.15.86:0/315816741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82c8004810 0x7f82c8004c20 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f82e40a0570 tx=0x7f82cc01cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d80a5cefc098ccf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:37.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.480+0000 7f82e3e00640 1 -- 172.21.15.86:0/315816741 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc007cd0 con 0x7f82c8004810 2024-04-25T14:05:37.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.480+0000 7f82e3e00640 1 -- 172.21.15.86:0/315816741 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82cc0a9020 con 0x7f82c8004810 2024-04-25T14:05:37.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.480+0000 7f82e3e00640 1 -- 172.21.15.86:0/315816741 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc005b20 con 0x7f82c8004810 2024-04-25T14:05:37.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.480+0000 7f82eba00640 1 -- 172.21.15.86:0/315816741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82c8004810 msgr2=0x7f82c8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:37.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.480+0000 7f82eba00640 1 --2- 172.21.15.86:0/315816741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82c8004810 0x7f82c8004c20 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f82e40a0570 tx=0x7f82cc01cba0 comp rx=0 tx=0).stop 2024-04-25T14:05:37.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.480+0000 7f82e9a00640 1 -- 172.21.15.86:0/315816741 reap_dead start 2024-04-25T14:05:37.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.480+0000 7f82eba00640 1 -- 172.21.15.86:0/315816741 shutdown_connections 2024-04-25T14:05:37.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.480+0000 7f82eba00640 1 -- 172.21.15.86:0/315816741 >> 172.21.15.86:0/315816741 conn(0x7f82e409ba30 msgr2=0x7f82e40b7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:37.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.480+0000 7f82eba00640 1 -- 172.21.15.86:0/315816741 shutdown_connections 2024-04-25T14:05:37.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.480+0000 7f82eba00640 1 -- 172.21.15.86:0/315816741 wait complete. 2024-04-25T14:05:37.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.484+0000 7f82eba00640 1 Processor -- start 2024-04-25T14:05:37.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.484+0000 7f82eba00640 1 -- start start 2024-04-25T14:05:37.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.484+0000 7f82eba00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82c8004810 0x7f82e40d1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:37.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.484+0000 7f82eba00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82e41e7720 0x7f82e41e7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:37.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.484+0000 7f82eba00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82e40cbcf0 0x7f82e40cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:37.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.484+0000 7f82eba00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82e406f110 con 0x7f82e41e7720 2024-04-25T14:05:37.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.484+0000 7f82eba00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f82e406ef90 con 0x7f82e40cbcf0 2024-04-25T14:05:37.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.484+0000 7f82eba00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f82e406e2d0 con 0x7f82c8004810 2024-04-25T14:05:37.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.484+0000 7f82dbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82e41e7720 0x7f82e41e7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:37.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.484+0000 7f82dbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82e41e7720 0x7f82e41e7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52156/0 (socket says 172.21.15.86:52156) 2024-04-25T14:05:37.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.484+0000 7f82dbe00640 1 -- 172.21.15.86:0/4083899498 learned_addr learned my addr 172.21.15.86:0/4083899498 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:37.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.488+0000 7f82e9000640 1 --2- 172.21.15.86:0/4083899498 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82c8004810 0x7f82e40d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:37.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.488+0000 7f82e9a00640 1 --2- 172.21.15.86:0/4083899498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82e40cbcf0 0x7f82e40cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:37.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.488+0000 7f82dbe00640 1 -- 172.21.15.86:0/4083899498 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82c8004810 msgr2=0x7f82e40d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:37.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.488+0000 7f82dbe00640 1 --2- 172.21.15.86:0/4083899498 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82c8004810 0x7f82e40d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:37.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.488+0000 7f82dbe00640 1 -- 172.21.15.86:0/4083899498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82e40cbcf0 msgr2=0x7f82e40cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:37.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.488+0000 7f82dbe00640 1 --2- 172.21.15.86:0/4083899498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82e40cbcf0 0x7f82e40cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:37.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.488+0000 7f82dbe00640 1 -- 172.21.15.86:0/4083899498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e40cc640 con 0x7f82e41e7720 2024-04-25T14:05:37.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.488+0000 7f82dbe00640 1 --2- 172.21.15.86:0/4083899498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82e41e7720 0x7f82e41e7b30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f82cc01c610 tx=0x7f82c8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:37.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.488+0000 7f82e1600640 1 -- 172.21.15.86:0/4083899498 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc0bb070 con 0x7f82e41e7720 2024-04-25T14:05:37.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.488+0000 7f82e1600640 1 -- 172.21.15.86:0/4083899498 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82cc0047f0 con 0x7f82e41e7720 2024-04-25T14:05:37.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.488+0000 7f82eba00640 1 -- 172.21.15.86:0/4083899498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82e41e6060 con 0x7f82e41e7720 2024-04-25T14:05:37.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.488+0000 7f82e1600640 1 -- 172.21.15.86:0/4083899498 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc007cd0 con 0x7f82e41e7720 2024-04-25T14:05:37.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.488+0000 7f82eba00640 1 -- 172.21.15.86:0/4083899498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82e41e6620 con 0x7f82e41e7720 2024-04-25T14:05:37.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.488+0000 7f82eba00640 1 -- 172.21.15.86:0/4083899498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82e40a05a0 con 0x7f82e41e7720 2024-04-25T14:05:37.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.488+0000 7f82e1600640 1 -- 172.21.15.86:0/4083899498 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f82cc0be070 con 0x7f82e41e7720 2024-04-25T14:05:37.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.492+0000 7f82e1600640 1 --2- 172.21.15.86:0/4083899498 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f82c006f570 0x7f82c0071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:37.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.492+0000 7f82e1600640 1 -- 172.21.15.86:0/4083899498 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6433+0+0 (secure 0 0 0) 0x7f82cc128d30 con 0x7f82e41e7720 2024-04-25T14:05:37.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.492+0000 7f82e9000640 1 --2- 172.21.15.86:0/4083899498 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f82c006f570 0x7f82c0071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:37.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.492+0000 7f82e9000640 1 --2- 172.21.15.86:0/4083899498 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f82c006f570 0x7f82c0071a80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f82e40cc970 tx=0x7f82d4009190 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:37.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.496+0000 7f82e1600640 1 -- 172.21.15.86:0/4083899498 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f82e40cc640 con 0x7f82e41e7720 2024-04-25T14:05:37.575 INFO:tasks.ceph.mgr.y.smithi086.stderr:Exception in thread Thread-7: 2024-04-25T14:05:37.575 INFO:tasks.ceph.mgr.y.smithi086.stderr:Traceback (most recent call last): 2024-04-25T14:05:37.575 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner 2024-04-25T14:05:37.575 INFO:tasks.ceph.mgr.y.smithi086.stderr: self.run() 2024-04-25T14:05:37.575 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/lib/python3.10/threading.py", line 1378, in run 2024-04-25T14:05:37.576 INFO:tasks.ceph.mgr.y.smithi086.stderr: self.function(*self.args, **self.kwargs) 2024-04-25T14:05:37.576 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/share/ceph/mgr/stats/fs/perf_stats.py", line 222, in re_register_queries 2024-04-25T14:05:37.576 INFO:tasks.ceph.mgr.y.smithi086.stderr: if self.mx_last_updated >= ua_last_updated: 2024-04-25T14:05:37.576 INFO:tasks.ceph.mgr.y.smithi086.stderr:AttributeError: 'FSPerfStats' object has no attribute 'mx_last_updated' 2024-04-25T14:05:37.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.652+0000 7f82eba00640 1 -- 172.21.15.86:0/4083899498 --> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f82e4076bf0 con 0x7f82c006f570 2024-04-25T14:05:37.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.660+0000 7f82e1600640 1 -- 172.21.15.86:0/4083899498 <== mgr.5532 v2:172.21.15.86:6832/2427179681 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242125 (secure 0 0 0) 0x7f82e4076bf0 con 0x7f82c006f570 2024-04-25T14:05:37.663 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:37.665 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:05:37.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.664+0000 7f82eba00640 1 -- 172.21.15.86:0/4083899498 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f82c006f570 msgr2=0x7f82c0071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:37.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.664+0000 7f82eba00640 1 --2- 172.21.15.86:0/4083899498 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f82c006f570 0x7f82c0071a80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f82e40cc970 tx=0x7f82d4009190 comp rx=0 tx=0).stop 2024-04-25T14:05:37.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.664+0000 7f82eba00640 1 -- 172.21.15.86:0/4083899498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82e41e7720 msgr2=0x7f82e41e7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:37.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.664+0000 7f82eba00640 1 --2- 172.21.15.86:0/4083899498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82e41e7720 0x7f82e41e7b30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f82cc01c610 tx=0x7f82c8005840 comp rx=0 tx=0).stop 2024-04-25T14:05:37.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.664+0000 7f82eba00640 1 -- 172.21.15.86:0/4083899498 shutdown_connections 2024-04-25T14:05:37.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.664+0000 7f82eba00640 1 --2- 172.21.15.86:0/4083899498 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f82c006f570 0x7f82c0071a80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:37.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.664+0000 7f82eba00640 1 --2- 172.21.15.86:0/4083899498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82e40cbcf0 0x7f82e40cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:37.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.664+0000 7f82eba00640 1 --2- 172.21.15.86:0/4083899498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82e41e7720 0x7f82e41e7b30 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:37.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.664+0000 7f82eba00640 1 --2- 172.21.15.86:0/4083899498 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82c8004810 0x7f82e40d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:37.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.664+0000 7f82eba00640 1 -- 172.21.15.86:0/4083899498 >> 172.21.15.86:0/4083899498 conn(0x7f82e409ba30 msgr2=0x7f82e415bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:37.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.664+0000 7f82eba00640 1 -- 172.21.15.86:0/4083899498 shutdown_connections 2024-04-25T14:05:37.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.664+0000 7f82eba00640 1 -- 172.21.15.86:0/4083899498 wait complete. 2024-04-25T14:05:37.684 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-04-25T14:05:37.482712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591020,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":124,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88,"ondisk_log_size":88,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221384,"kb_used_data":5208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753336,"statfs":{"total":773094113280,"available":772867416064,"internally_reserved":0,"allocated":5332992,"data_stored":3003413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":652,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.952497"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.040189+0000","last_change":"2024-04-25T14:05:33.573200+0000","last_active":"2024-04-25T14:05:35.040189+0000","last_peered":"2024-04-25T14:05:35.040189+0000","last_clean":"2024-04-25T14:05:35.040189+0000","last_became_active":"2024-04-25T14:05:33.572858+0000","last_became_peered":"2024-04-25T14:05:33.572858+0000","last_unstale":"2024-04-25T14:05:35.040189+0000","last_undegraded":"2024-04-25T14:05:35.040189+0000","last_fullsized":"2024-04-25T14:05:35.040189+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:22.996486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039514+0000","last_change":"2024-04-25T14:05:33.571072+0000","last_active":"2024-04-25T14:05:35.039514+0000","last_peered":"2024-04-25T14:05:35.039514+0000","last_clean":"2024-04-25T14:05:35.039514+0000","last_became_active":"2024-04-25T14:05:33.570748+0000","last_became_peered":"2024-04-25T14:05:33.570748+0000","last_unstale":"2024-04-25T14:05:35.039514+0000","last_undegraded":"2024-04-25T14:05:35.039514+0000","last_fullsized":"2024-04-25T14:05:35.039514+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:18.074490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039301+0000","last_change":"2024-04-25T14:05:33.571248+0000","last_active":"2024-04-25T14:05:35.039301+0000","last_peered":"2024-04-25T14:05:35.039301+0000","last_clean":"2024-04-25T14:05:35.039301+0000","last_became_active":"2024-04-25T14:05:33.570916+0000","last_became_peered":"2024-04-25T14:05:33.570916+0000","last_unstale":"2024-04-25T14:05:35.039301+0000","last_undegraded":"2024-04-25T14:05:35.039301+0000","last_fullsized":"2024-04-25T14:05:35.039301+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:36.296676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039600+0000","last_change":"2024-04-25T14:05:33.576117+0000","last_active":"2024-04-25T14:05:35.039600+0000","last_peered":"2024-04-25T14:05:35.039600+0000","last_clean":"2024-04-25T14:05:35.039600+0000","last_became_active":"2024-04-25T14:05:33.575838+0000","last_became_peered":"2024-04-25T14:05:33.575838+0000","last_unstale":"2024-04-25T14:05:35.039600+0000","last_undegraded":"2024-04-25T14:05:35.039600+0000","last_fullsized":"2024-04-25T14:05:35.039600+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:20.619851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039689+0000","last_change":"2024-04-25T14:05:33.596454+0000","last_active":"2024-04-25T14:05:35.039689+0000","last_peered":"2024-04-25T14:05:35.039689+0000","last_clean":"2024-04-25T14:05:35.039689+0000","last_became_active":"2024-04-25T14:05:33.596225+0000","last_became_peered":"2024-04-25T14:05:33.596225+0000","last_unstale":"2024-04-25T14:05:35.039689+0000","last_undegraded":"2024-04-25T14:05:35.039689+0000","last_fullsized":"2024-04-25T14:05:35.039689+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:41.512359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"45'4","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039294+0000","last_change":"2024-04-25T14:05:32.556304+0000","last_active":"2024-04-25T14:05:35.039294+0000","last_peered":"2024-04-25T14:05:35.039294+0000","last_clean":"2024-04-25T14:05:35.039294+0000","last_became_active":"2024-04-25T14:05:32.556052+0000","last_became_peered":"2024-04-25T14:05:32.556052+0000","last_unstale":"2024-04-25T14:05:35.039294+0000","last_undegraded":"2024-04-25T14:05:35.039294+0000","last_fullsized":"2024-04-25T14:05:35.039294+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:16.452558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.532639+0000","last_change":"2024-04-25T14:05:31.532639+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.532639+0000","last_undegraded":"2024-04-25T14:05:31.532639+0000","last_fullsized":"2024-04-25T14:05:31.532639+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039348+0000","last_change":"2024-04-25T14:05:33.574504+0000","last_active":"2024-04-25T14:05:35.039348+0000","last_peered":"2024-04-25T14:05:35.039348+0000","last_clean":"2024-04-25T14:05:35.039348+0000","last_became_active":"2024-04-25T14:05:33.574191+0000","last_became_peered":"2024-04-25T14:05:33.574191+0000","last_unstale":"2024-04-25T14:05:35.039348+0000","last_undegraded":"2024-04-25T14:05:35.039348+0000","last_fullsized":"2024-04-25T14:05:35.039348+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:28.629474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.533022+0000","last_change":"2024-04-25T14:05:31.533022+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.533022+0000","last_undegraded":"2024-04-25T14:05:31.533022+0000","last_fullsized":"2024-04-25T14:05:31.533022+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.533643+0000","last_change":"2024-04-25T14:05:31.533643+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.533643+0000","last_undegraded":"2024-04-25T14:05:31.533643+0000","last_fullsized":"2024-04-25T14:05:31.533643+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.540931+0000","last_change":"2024-04-25T14:05:31.540931+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.540931+0000","last_undegraded":"2024-04-25T14:05:31.540931+0000","last_fullsized":"2024-04-25T14:05:31.540931+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.541762+0000","last_change":"2024-04-25T14:05:31.541762+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.541762+0000","last_undegraded":"2024-04-25T14:05:31.541762+0000","last_fullsized":"2024-04-25T14:05:31.541762+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039072+0000","last_change":"2024-04-25T14:05:32.551806+0000","last_active":"2024-04-25T14:05:35.039072+0000","last_peered":"2024-04-25T14:05:35.039072+0000","last_clean":"2024-04-25T14:05:35.039072+0000","last_became_active":"2024-04-25T14:05:32.551441+0000","last_became_peered":"2024-04-25T14:05:32.551441+0000","last_unstale":"2024-04-25T14:05:35.039072+0000","last_undegraded":"2024-04-25T14:05:35.039072+0000","last_fullsized":"2024-04-25T14:05:35.039072+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:31.414048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.538948+0000","last_change":"2024-04-25T14:05:31.538948+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.538948+0000","last_undegraded":"2024-04-25T14:05:31.538948+0000","last_fullsized":"2024-04-25T14:05:31.538948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.539519+0000","last_change":"2024-04-25T14:05:31.539519+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.539519+0000","last_undegraded":"2024-04-25T14:05:31.539519+0000","last_fullsized":"2024-04-25T14:05:31.539519+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.542126+0000","last_change":"2024-04-25T14:05:31.542126+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.542126+0000","last_undegraded":"2024-04-25T14:05:31.542126+0000","last_fullsized":"2024-04-25T14:05:31.542126+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.540063+0000","last_change":"2024-04-25T14:05:31.540063+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.540063+0000","last_undegraded":"2024-04-25T14:05:31.540063+0000","last_fullsized":"2024-04-25T14:05:31.540063+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.535803+0000","last_change":"2024-04-25T14:05:31.535803+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.535803+0000","last_undegraded":"2024-04-25T14:05:31.535803+0000","last_fullsized":"2024-04-25T14:05:31.535803+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.540372+0000","last_change":"2024-04-25T14:05:31.540372+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.540372+0000","last_undegraded":"2024-04-25T14:05:31.540372+0000","last_fullsized":"2024-04-25T14:05:31.540372+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039235+0000","last_change":"2024-04-25T14:05:32.562133+0000","last_active":"2024-04-25T14:05:35.039235+0000","last_peered":"2024-04-25T14:05:35.039235+0000","last_clean":"2024-04-25T14:05:35.039235+0000","last_became_active":"2024-04-25T14:05:32.561845+0000","last_became_peered":"2024-04-25T14:05:32.561845+0000","last_unstale":"2024-04-25T14:05:35.039235+0000","last_undegraded":"2024-04-25T14:05:35.039235+0000","last_fullsized":"2024-04-25T14:05:35.039235+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:14.012822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.539420+0000","last_change":"2024-04-25T14:05:31.539420+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.539420+0000","last_undegraded":"2024-04-25T14:05:31.539420+0000","last_fullsized":"2024-04-25T14:05:31.539420+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.535717+0000","last_change":"2024-04-25T14:05:31.535717+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.535717+0000","last_undegraded":"2024-04-25T14:05:31.535717+0000","last_fullsized":"2024-04-25T14:05:31.535717+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.539606+0000","last_change":"2024-04-25T14:05:31.539606+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.539606+0000","last_undegraded":"2024-04-25T14:05:31.539606+0000","last_fullsized":"2024-04-25T14:05:31.539606+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.040239+0000","last_change":"2024-04-25T14:05:32.559217+0000","last_active":"2024-04-25T14:05:35.040239+0000","last_peered":"2024-04-25T14:05:35.040239+0000","last_clean":"2024-04-25T14:05:35.040239+0000","last_became_active":"2024-04-25T14:05:32.558909+0000","last_became_peered":"2024-04-25T14:05:32.558909+0000","last_unstale":"2024-04-25T14:05:35.040239+0000","last_undegraded":"2024-04-25T14:05:35.040239+0000","last_fullsized":"2024-04-25T14:05:35.040239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:48:44.213850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039597+0000","last_change":"2024-04-25T14:05:32.566219+0000","last_active":"2024-04-25T14:05:35.039597+0000","last_peered":"2024-04-25T14:05:35.039597+0000","last_clean":"2024-04-25T14:05:35.039597+0000","last_became_active":"2024-04-25T14:05:32.565859+0000","last_became_peered":"2024-04-25T14:05:32.565859+0000","last_unstale":"2024-04-25T14:05:35.039597+0000","last_undegraded":"2024-04-25T14:05:35.039597+0000","last_fullsized":"2024-04-25T14:05:35.039597+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:35.649467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039730+0000","last_change":"2024-04-25T14:05:32.567474+0000","last_active":"2024-04-25T14:05:35.039730+0000","last_peered":"2024-04-25T14:05:35.039730+0000","last_clean":"2024-04-25T14:05:35.039730+0000","last_became_active":"2024-04-25T14:05:32.567061+0000","last_became_peered":"2024-04-25T14:05:32.567061+0000","last_unstale":"2024-04-25T14:05:35.039730+0000","last_undegraded":"2024-04-25T14:05:35.039730+0000","last_fullsized":"2024-04-25T14:05:35.039730+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:20.281168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.532537+0000","last_change":"2024-04-25T14:05:31.532537+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.532537+0000","last_undegraded":"2024-04-25T14:05:31.532537+0000","last_fullsized":"2024-04-25T14:05:31.532537+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039542+0000","last_change":"2024-04-25T14:05:33.570587+0000","last_active":"2024-04-25T14:05:35.039542+0000","last_peered":"2024-04-25T14:05:35.039542+0000","last_clean":"2024-04-25T14:05:35.039542+0000","last_became_active":"2024-04-25T14:05:33.570209+0000","last_became_peered":"2024-04-25T14:05:33.570209+0000","last_unstale":"2024-04-25T14:05:35.039542+0000","last_undegraded":"2024-04-25T14:05:35.039542+0000","last_fullsized":"2024-04-25T14:05:35.039542+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:13.979622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.535614+0000","last_change":"2024-04-25T14:05:31.535614+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.535614+0000","last_undegraded":"2024-04-25T14:05:31.535614+0000","last_fullsized":"2024-04-25T14:05:31.535614+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.541074+0000","last_change":"2024-04-25T14:05:31.541074+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.541074+0000","last_undegraded":"2024-04-25T14:05:31.541074+0000","last_fullsized":"2024-04-25T14:05:31.541074+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039205+0000","last_change":"2024-04-25T14:05:32.553201+0000","last_active":"2024-04-25T14:05:35.039205+0000","last_peered":"2024-04-25T14:05:35.039205+0000","last_clean":"2024-04-25T14:05:35.039205+0000","last_became_active":"2024-04-25T14:05:32.552846+0000","last_became_peered":"2024-04-25T14:05:32.552846+0000","last_unstale":"2024-04-25T14:05:35.039205+0000","last_undegraded":"2024-04-25T14:05:35.039205+0000","last_fullsized":"2024-04-25T14:05:35.039205+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:13.772830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039401+0000","last_change":"2024-04-25T14:05:32.561754+0000","last_active":"2024-04-25T14:05:35.039401+0000","last_peered":"2024-04-25T14:05:35.039401+0000","last_clean":"2024-04-25T14:05:35.039401+0000","last_became_active":"2024-04-25T14:05:32.561297+0000","last_became_peered":"2024-04-25T14:05:32.561297+0000","last_unstale":"2024-04-25T14:05:35.039401+0000","last_undegraded":"2024-04-25T14:05:35.039401+0000","last_fullsized":"2024-04-25T14:05:35.039401+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:39.285483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.539462+0000","last_change":"2024-04-25T14:05:31.539462+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.539462+0000","last_undegraded":"2024-04-25T14:05:31.539462+0000","last_fullsized":"2024-04-25T14:05:31.539462+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.539025+0000","last_change":"2024-04-25T14:05:31.539025+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.539025+0000","last_undegraded":"2024-04-25T14:05:31.539025+0000","last_fullsized":"2024-04-25T14:05:31.539025+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.535050+0000","last_change":"2024-04-25T14:05:31.535050+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.535050+0000","last_undegraded":"2024-04-25T14:05:31.535050+0000","last_fullsized":"2024-04-25T14:05:31.535050+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.535108+0000","last_change":"2024-04-25T14:05:31.535108+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.535108+0000","last_undegraded":"2024-04-25T14:05:31.535108+0000","last_fullsized":"2024-04-25T14:05:31.535108+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.541206+0000","last_change":"2024-04-25T14:05:31.541206+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.541206+0000","last_undegraded":"2024-04-25T14:05:31.541206+0000","last_fullsized":"2024-04-25T14:05:31.541206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.533456+0000","last_change":"2024-04-25T14:05:31.533456+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.533456+0000","last_undegraded":"2024-04-25T14:05:31.533456+0000","last_fullsized":"2024-04-25T14:05:31.533456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039142+0000","last_change":"2024-04-25T14:05:33.563768+0000","last_active":"2024-04-25T14:05:35.039142+0000","last_peered":"2024-04-25T14:05:35.039142+0000","last_clean":"2024-04-25T14:05:35.039142+0000","last_became_active":"2024-04-25T14:05:33.563379+0000","last_became_peered":"2024-04-25T14:05:33.563379+0000","last_unstale":"2024-04-25T14:05:35.039142+0000","last_undegraded":"2024-04-25T14:05:35.039142+0000","last_fullsized":"2024-04-25T14:05:35.039142+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:11.087435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.538960+0000","last_change":"2024-04-25T14:05:31.538960+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.538960+0000","last_undegraded":"2024-04-25T14:05:31.538960+0000","last_fullsized":"2024-04-25T14:05:31.538960+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"39'79","reported_seq":177,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T14:05:31.540738+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:05:31.540738+0000","last_peered":"2024-04-25T14:05:31.540738+0000","last_clean":"2024-04-25T14:05:31.540738+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:05:31.540738+0000","last_undegraded":"2024-04-25T14:05:31.540738+0000","last_fullsized":"2024-04-25T14:05:31.540738+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":2},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":652,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1089536,"data_stored":806812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":552960,"data_stored":245159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":222991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":222991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":222991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":246119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":222991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343740,"statfs":{"total":96636764160,"available":96607989760,"internally_reserved":0,"allocated":1101824,"data_stored":813359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:05:37.688 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:05:37.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.872+0000 7f98dd000640 1 Processor -- start 2024-04-25T14:05:37.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.872+0000 7f98dd000640 1 -- start start 2024-04-25T14:05:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.872+0000 7f98dd000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f98d80d2c80 0x7f98d815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.872+0000 7f98dd000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98d81616e0 0x7f98d8161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.872+0000 7f98dd000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98d80cd8d0 con 0x7f98d8070e20 2024-04-25T14:05:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.872+0000 7f98dd000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f98d80ce720 con 0x7f98d80d2c80 2024-04-25T14:05:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.872+0000 7f98dd000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f98d80ce970 con 0x7f98d81616e0 2024-04-25T14:05:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.872+0000 7f98cd400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f98d8070e20 0x7f98d8071230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36698/0 (socket says 172.21.15.86:36698) 2024-04-25T14:05:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.872+0000 7f98cd400640 1 -- 172.21.15.86:0/2247567927 learned_addr learned my addr 172.21.15.86:0/2247567927 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.872+0000 7f98d5e00640 1 --2- 172.21.15.86:0/2247567927 >> v2:172.21.15.155:3300/0 conn(0x7f98d80d2c80 0x7f98d815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.872+0000 7f98d6800640 1 --2- 172.21.15.86:0/2247567927 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98d81616e0 0x7f98d8161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.872+0000 7f98d5400640 1 -- 172.21.15.86:0/2247567927 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2657896065 0 0) 0x7f98d80cd8d0 con 0x7f98d8070e20 2024-04-25T14:05:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.872+0000 7f98d5400640 1 -- 172.21.15.86:0/2247567927 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98bc003530 con 0x7f98d8070e20 2024-04-25T14:05:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.872+0000 7f98d5400640 1 -- 172.21.15.86:0/2247567927 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 734381982 0 0) 0x7f98bc003530 con 0x7f98d8070e20 2024-04-25T14:05:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.872+0000 7f98d5400640 1 -- 172.21.15.86:0/2247567927 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98d81616e0 msgr2=0x7f98d8161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.872+0000 7f98d5400640 1 --2- 172.21.15.86:0/2247567927 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98d81616e0 0x7f98d8161af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.872+0000 7f98d5400640 1 -- 172.21.15.86:0/2247567927 >> v2:172.21.15.155:3300/0 conn(0x7f98d80d2c80 msgr2=0x7f98d815dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.872+0000 7f98d5400640 1 --2- 172.21.15.86:0/2247567927 >> v2:172.21.15.155:3300/0 conn(0x7f98d80d2c80 0x7f98d815dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.872+0000 7f98d5400640 1 -- 172.21.15.86:0/2247567927 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d80d1be0 con 0x7f98d8070e20 2024-04-25T14:05:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.872+0000 7f98d5400640 1 -- 172.21.15.86:0/2247567927 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f98c0001800 con 0x7f98d8070e20 2024-04-25T14:05:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.872+0000 7f98d5400640 1 -- 172.21.15.86:0/2247567927 >> v1:172.21.15.86:6789/0 conn(0x7f98d8070e20 legacy=0x7f98d8071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.872+0000 7f98d5400640 1 --2- 172.21.15.86:0/2247567927 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98d81616e0 0x7f98bc003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.872+0000 7f98d5400640 1 --2- 172.21.15.86:0/2247567927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98bc003fb0 0x7f98bc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.872+0000 7f98d5400640 1 --2- 172.21.15.86:0/2247567927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98bc004d50 0x7f98bc005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.872+0000 7f98d5400640 1 -- 172.21.15.86:0/2247567927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98d80ce720 con 0x7f98bc003fb0 2024-04-25T14:05:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.872+0000 7f98d5400640 1 -- 172.21.15.86:0/2247567927 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f98d80ce970 con 0x7f98bc004d50 2024-04-25T14:05:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.872+0000 7f98d5400640 1 -- 172.21.15.86:0/2247567927 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f98bc00bb30 con 0x7f98d81616e0 2024-04-25T14:05:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.876+0000 7f98cd400640 1 --2- 172.21.15.86:0/2247567927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98bc003fb0 0x7f98bc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.876+0000 7f98d5e00640 1 --2- 172.21.15.86:0/2247567927 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98d81616e0 0x7f98bc003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.876+0000 7f98d6800640 1 --2- 172.21.15.86:0/2247567927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98bc004d50 0x7f98bc005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.876+0000 7f98cd400640 1 -- 172.21.15.86:0/2247567927 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98d81616e0 msgr2=0x7f98bc003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.876+0000 7f98cd400640 1 --2- 172.21.15.86:0/2247567927 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98d81616e0 0x7f98bc003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.876+0000 7f98cd400640 1 -- 172.21.15.86:0/2247567927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98bc004d50 msgr2=0x7f98bc005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.876+0000 7f98cd400640 1 --2- 172.21.15.86:0/2247567927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98bc004d50 0x7f98bc005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.876+0000 7f98cd400640 1 -- 172.21.15.86:0/2247567927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98bc003530 con 0x7f98bc003fb0 2024-04-25T14:05:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.876+0000 7f98d5e00640 1 --2- 172.21.15.86:0/2247567927 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98d81616e0 0x7f98bc003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:37.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.876+0000 7f98cd400640 1 --2- 172.21.15.86:0/2247567927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98bc003fb0 0x7f98bc004810 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f98c000c210 tx=0x7f98c001cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d783b62fb250a76e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:37.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.876+0000 7f98d5400640 1 -- 172.21.15.86:0/2247567927 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c0007d60 con 0x7f98bc003fb0 2024-04-25T14:05:37.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.876+0000 7f98d5400640 1 -- 172.21.15.86:0/2247567927 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98c00a9020 con 0x7f98bc003fb0 2024-04-25T14:05:37.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.876+0000 7f98d5400640 1 -- 172.21.15.86:0/2247567927 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c00a5c70 con 0x7f98bc003fb0 2024-04-25T14:05:37.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.880+0000 7f98dd000640 1 -- 172.21.15.86:0/2247567927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98bc003fb0 msgr2=0x7f98bc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:37.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.880+0000 7f98dd000640 1 --2- 172.21.15.86:0/2247567927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98bc003fb0 0x7f98bc004810 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f98c000c210 tx=0x7f98c001cd30 comp rx=0 tx=0).stop 2024-04-25T14:05:37.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.880+0000 7f98d6800640 1 -- 172.21.15.86:0/2247567927 reap_dead start 2024-04-25T14:05:37.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.880+0000 7f98dd000640 1 -- 172.21.15.86:0/2247567927 shutdown_connections 2024-04-25T14:05:37.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.880+0000 7f98dd000640 1 -- 172.21.15.86:0/2247567927 >> 172.21.15.86:0/2247567927 conn(0x7f98d80cffc0 msgr2=0x7f98d80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:37.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.880+0000 7f98dd000640 1 -- 172.21.15.86:0/2247567927 shutdown_connections 2024-04-25T14:05:37.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.880+0000 7f98dd000640 1 -- 172.21.15.86:0/2247567927 wait complete. 2024-04-25T14:05:37.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.880+0000 7f98dd000640 1 Processor -- start 2024-04-25T14:05:37.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.880+0000 7f98dd000640 1 -- start start 2024-04-25T14:05:37.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.884+0000 7f98dd000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98bc003fb0 0x7f98d80c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:37.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.884+0000 7f98dd000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98d81eac70 0x7f98d81e8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:37.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.884+0000 7f98dd000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98d81e6800 0x7f98d81f9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:37.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.884+0000 7f98dd000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98d80d2480 con 0x7f98bc003fb0 2024-04-25T14:05:37.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.884+0000 7f98dd000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f98d80d21a0 con 0x7f98d81eac70 2024-04-25T14:05:37.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.884+0000 7f98dd000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f98d80d29a0 con 0x7f98d81e6800 2024-04-25T14:05:37.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.884+0000 7f98d5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98bc003fb0 0x7f98d80c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:37.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.884+0000 7f98d5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98bc003fb0 0x7f98d80c9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52168/0 (socket says 172.21.15.86:52168) 2024-04-25T14:05:37.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.884+0000 7f98d5e00640 1 -- 172.21.15.86:0/648493014 learned_addr learned my addr 172.21.15.86:0/648493014 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:37.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.884+0000 7f98d6800640 1 --2- 172.21.15.86:0/648493014 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98d81e6800 0x7f98d81f9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:37.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.884+0000 7f98cd400640 1 --2- 172.21.15.86:0/648493014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98d81eac70 0x7f98d81e8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:37.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.884+0000 7f98d5e00640 1 -- 172.21.15.86:0/648493014 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98d81e6800 msgr2=0x7f98d81f9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:37.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.884+0000 7f98d5e00640 1 --2- 172.21.15.86:0/648493014 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98d81e6800 0x7f98d81f9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:37.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.884+0000 7f98d5e00640 1 -- 172.21.15.86:0/648493014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98d81eac70 msgr2=0x7f98d81e8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:37.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.884+0000 7f98d5e00640 1 --2- 172.21.15.86:0/648493014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98d81eac70 0x7f98d81e8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:37.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.884+0000 7f98d5e00640 1 -- 172.21.15.86:0/648493014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d815d360 con 0x7f98bc003fb0 2024-04-25T14:05:37.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.884+0000 7f98cd400640 1 --2- 172.21.15.86:0/648493014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98d81eac70 0x7f98d81e8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:37.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.884+0000 7f98d5e00640 1 --2- 172.21.15.86:0/648493014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98bc003fb0 0x7f98d80c9150 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f98c8011620 tx=0x7f98c8011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:37.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.884+0000 7f98cea00640 1 -- 172.21.15.86:0/648493014 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c8004220 con 0x7f98bc003fb0 2024-04-25T14:05:37.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.884+0000 7f98cea00640 1 -- 172.21.15.86:0/648493014 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98c80043c0 con 0x7f98bc003fb0 2024-04-25T14:05:37.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.884+0000 7f98dd000640 1 -- 172.21.15.86:0/648493014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d81e9bd0 con 0x7f98bc003fb0 2024-04-25T14:05:37.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.884+0000 7f98cea00640 1 -- 172.21.15.86:0/648493014 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c800f920 con 0x7f98bc003fb0 2024-04-25T14:05:37.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.884+0000 7f98dd000640 1 -- 172.21.15.86:0/648493014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d81ea190 con 0x7f98bc003fb0 2024-04-25T14:05:37.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.884+0000 7f98dd000640 1 -- 172.21.15.86:0/648493014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98d8076bf0 con 0x7f98bc003fb0 2024-04-25T14:05:37.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.888+0000 7f98cea00640 1 -- 172.21.15.86:0/648493014 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f98c800fac0 con 0x7f98bc003fb0 2024-04-25T14:05:37.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.888+0000 7f98cea00640 1 --2- 172.21.15.86:0/648493014 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f98b406f5a0 0x7f98b4071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:37.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.888+0000 7f98cea00640 1 -- 172.21.15.86:0/648493014 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6433+0+0 (secure 0 0 0) 0x7f98c808fa50 con 0x7f98bc003fb0 2024-04-25T14:05:37.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.888+0000 7f98cd400640 1 --2- 172.21.15.86:0/648493014 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f98b406f5a0 0x7f98b4071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:37.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.888+0000 7f98cd400640 1 --2- 172.21.15.86:0/648493014 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f98b406f5a0 0x7f98b4071ab0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f98c00aa000 tx=0x7f98c00b0040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:37.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:37.892+0000 7f98cea00640 1 -- 172.21.15.86:0/648493014 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f98c8019030 con 0x7f98bc003fb0 2024-04-25T14:05:38.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.044+0000 7f98dd000640 1 -- 172.21.15.86:0/648493014 --> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f98d80d1470 con 0x7f98b406f5a0 2024-04-25T14:05:38.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.052+0000 7f98cea00640 1 -- 172.21.15.86:0/648493014 <== mgr.5532 v2:172.21.15.86:6832/2427179681 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242125 (secure 0 0 0) 0x7f98d80d1470 con 0x7f98b406f5a0 2024-04-25T14:05:38.056 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:38.057 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:05:38.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.056+0000 7f98dd000640 1 -- 172.21.15.86:0/648493014 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f98b406f5a0 msgr2=0x7f98b4071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:38.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.056+0000 7f98dd000640 1 --2- 172.21.15.86:0/648493014 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f98b406f5a0 0x7f98b4071ab0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f98c00aa000 tx=0x7f98c00b0040 comp rx=0 tx=0).stop 2024-04-25T14:05:38.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.056+0000 7f98dd000640 1 -- 172.21.15.86:0/648493014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98bc003fb0 msgr2=0x7f98d80c9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:38.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.056+0000 7f98dd000640 1 --2- 172.21.15.86:0/648493014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98bc003fb0 0x7f98d80c9150 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f98c8011620 tx=0x7f98c8011ae0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.056+0000 7f98dd000640 1 -- 172.21.15.86:0/648493014 shutdown_connections 2024-04-25T14:05:38.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.056+0000 7f98dd000640 1 --2- 172.21.15.86:0/648493014 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f98b406f5a0 0x7f98b4071ab0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.056+0000 7f98dd000640 1 --2- 172.21.15.86:0/648493014 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98d81e6800 0x7f98d81f9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.056+0000 7f98dd000640 1 --2- 172.21.15.86:0/648493014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98d81eac70 0x7f98d81e8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.056+0000 7f98dd000640 1 --2- 172.21.15.86:0/648493014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98bc003fb0 0x7f98d80c9150 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.056+0000 7f98dd000640 1 -- 172.21.15.86:0/648493014 >> 172.21.15.86:0/648493014 conn(0x7f98d80cffc0 msgr2=0x7f98d806e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:38.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.056+0000 7f98dd000640 1 -- 172.21.15.86:0/648493014 shutdown_connections 2024-04-25T14:05:38.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.056+0000 7f98dd000640 1 -- 172.21.15.86:0/648493014 wait complete. 2024-04-25T14:05:38.077 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-04-25T14:05:37.482712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591020,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":124,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88,"ondisk_log_size":88,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221384,"kb_used_data":5208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753336,"statfs":{"total":773094113280,"available":772867416064,"internally_reserved":0,"allocated":5332992,"data_stored":3003413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":652,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.952497"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.040189+0000","last_change":"2024-04-25T14:05:33.573200+0000","last_active":"2024-04-25T14:05:35.040189+0000","last_peered":"2024-04-25T14:05:35.040189+0000","last_clean":"2024-04-25T14:05:35.040189+0000","last_became_active":"2024-04-25T14:05:33.572858+0000","last_became_peered":"2024-04-25T14:05:33.572858+0000","last_unstale":"2024-04-25T14:05:35.040189+0000","last_undegraded":"2024-04-25T14:05:35.040189+0000","last_fullsized":"2024-04-25T14:05:35.040189+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:22.996486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039514+0000","last_change":"2024-04-25T14:05:33.571072+0000","last_active":"2024-04-25T14:05:35.039514+0000","last_peered":"2024-04-25T14:05:35.039514+0000","last_clean":"2024-04-25T14:05:35.039514+0000","last_became_active":"2024-04-25T14:05:33.570748+0000","last_became_peered":"2024-04-25T14:05:33.570748+0000","last_unstale":"2024-04-25T14:05:35.039514+0000","last_undegraded":"2024-04-25T14:05:35.039514+0000","last_fullsized":"2024-04-25T14:05:35.039514+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:18.074490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039301+0000","last_change":"2024-04-25T14:05:33.571248+0000","last_active":"2024-04-25T14:05:35.039301+0000","last_peered":"2024-04-25T14:05:35.039301+0000","last_clean":"2024-04-25T14:05:35.039301+0000","last_became_active":"2024-04-25T14:05:33.570916+0000","last_became_peered":"2024-04-25T14:05:33.570916+0000","last_unstale":"2024-04-25T14:05:35.039301+0000","last_undegraded":"2024-04-25T14:05:35.039301+0000","last_fullsized":"2024-04-25T14:05:35.039301+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:36.296676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039600+0000","last_change":"2024-04-25T14:05:33.576117+0000","last_active":"2024-04-25T14:05:35.039600+0000","last_peered":"2024-04-25T14:05:35.039600+0000","last_clean":"2024-04-25T14:05:35.039600+0000","last_became_active":"2024-04-25T14:05:33.575838+0000","last_became_peered":"2024-04-25T14:05:33.575838+0000","last_unstale":"2024-04-25T14:05:35.039600+0000","last_undegraded":"2024-04-25T14:05:35.039600+0000","last_fullsized":"2024-04-25T14:05:35.039600+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:20.619851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039689+0000","last_change":"2024-04-25T14:05:33.596454+0000","last_active":"2024-04-25T14:05:35.039689+0000","last_peered":"2024-04-25T14:05:35.039689+0000","last_clean":"2024-04-25T14:05:35.039689+0000","last_became_active":"2024-04-25T14:05:33.596225+0000","last_became_peered":"2024-04-25T14:05:33.596225+0000","last_unstale":"2024-04-25T14:05:35.039689+0000","last_undegraded":"2024-04-25T14:05:35.039689+0000","last_fullsized":"2024-04-25T14:05:35.039689+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:41.512359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"45'4","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039294+0000","last_change":"2024-04-25T14:05:32.556304+0000","last_active":"2024-04-25T14:05:35.039294+0000","last_peered":"2024-04-25T14:05:35.039294+0000","last_clean":"2024-04-25T14:05:35.039294+0000","last_became_active":"2024-04-25T14:05:32.556052+0000","last_became_peered":"2024-04-25T14:05:32.556052+0000","last_unstale":"2024-04-25T14:05:35.039294+0000","last_undegraded":"2024-04-25T14:05:35.039294+0000","last_fullsized":"2024-04-25T14:05:35.039294+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:16.452558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.532639+0000","last_change":"2024-04-25T14:05:31.532639+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.532639+0000","last_undegraded":"2024-04-25T14:05:31.532639+0000","last_fullsized":"2024-04-25T14:05:31.532639+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039348+0000","last_change":"2024-04-25T14:05:33.574504+0000","last_active":"2024-04-25T14:05:35.039348+0000","last_peered":"2024-04-25T14:05:35.039348+0000","last_clean":"2024-04-25T14:05:35.039348+0000","last_became_active":"2024-04-25T14:05:33.574191+0000","last_became_peered":"2024-04-25T14:05:33.574191+0000","last_unstale":"2024-04-25T14:05:35.039348+0000","last_undegraded":"2024-04-25T14:05:35.039348+0000","last_fullsized":"2024-04-25T14:05:35.039348+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:28.629474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.533022+0000","last_change":"2024-04-25T14:05:31.533022+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.533022+0000","last_undegraded":"2024-04-25T14:05:31.533022+0000","last_fullsized":"2024-04-25T14:05:31.533022+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.533643+0000","last_change":"2024-04-25T14:05:31.533643+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.533643+0000","last_undegraded":"2024-04-25T14:05:31.533643+0000","last_fullsized":"2024-04-25T14:05:31.533643+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.540931+0000","last_change":"2024-04-25T14:05:31.540931+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.540931+0000","last_undegraded":"2024-04-25T14:05:31.540931+0000","last_fullsized":"2024-04-25T14:05:31.540931+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.541762+0000","last_change":"2024-04-25T14:05:31.541762+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.541762+0000","last_undegraded":"2024-04-25T14:05:31.541762+0000","last_fullsized":"2024-04-25T14:05:31.541762+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039072+0000","last_change":"2024-04-25T14:05:32.551806+0000","last_active":"2024-04-25T14:05:35.039072+0000","last_peered":"2024-04-25T14:05:35.039072+0000","last_clean":"2024-04-25T14:05:35.039072+0000","last_became_active":"2024-04-25T14:05:32.551441+0000","last_became_peered":"2024-04-25T14:05:32.551441+0000","last_unstale":"2024-04-25T14:05:35.039072+0000","last_undegraded":"2024-04-25T14:05:35.039072+0000","last_fullsized":"2024-04-25T14:05:35.039072+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:31.414048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.538948+0000","last_change":"2024-04-25T14:05:31.538948+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.538948+0000","last_undegraded":"2024-04-25T14:05:31.538948+0000","last_fullsized":"2024-04-25T14:05:31.538948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.539519+0000","last_change":"2024-04-25T14:05:31.539519+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.539519+0000","last_undegraded":"2024-04-25T14:05:31.539519+0000","last_fullsized":"2024-04-25T14:05:31.539519+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.542126+0000","last_change":"2024-04-25T14:05:31.542126+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.542126+0000","last_undegraded":"2024-04-25T14:05:31.542126+0000","last_fullsized":"2024-04-25T14:05:31.542126+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.540063+0000","last_change":"2024-04-25T14:05:31.540063+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.540063+0000","last_undegraded":"2024-04-25T14:05:31.540063+0000","last_fullsized":"2024-04-25T14:05:31.540063+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.535803+0000","last_change":"2024-04-25T14:05:31.535803+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.535803+0000","last_undegraded":"2024-04-25T14:05:31.535803+0000","last_fullsized":"2024-04-25T14:05:31.535803+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.540372+0000","last_change":"2024-04-25T14:05:31.540372+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.540372+0000","last_undegraded":"2024-04-25T14:05:31.540372+0000","last_fullsized":"2024-04-25T14:05:31.540372+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039235+0000","last_change":"2024-04-25T14:05:32.562133+0000","last_active":"2024-04-25T14:05:35.039235+0000","last_peered":"2024-04-25T14:05:35.039235+0000","last_clean":"2024-04-25T14:05:35.039235+0000","last_became_active":"2024-04-25T14:05:32.561845+0000","last_became_peered":"2024-04-25T14:05:32.561845+0000","last_unstale":"2024-04-25T14:05:35.039235+0000","last_undegraded":"2024-04-25T14:05:35.039235+0000","last_fullsized":"2024-04-25T14:05:35.039235+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:14.012822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.539420+0000","last_change":"2024-04-25T14:05:31.539420+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.539420+0000","last_undegraded":"2024-04-25T14:05:31.539420+0000","last_fullsized":"2024-04-25T14:05:31.539420+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.535717+0000","last_change":"2024-04-25T14:05:31.535717+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.535717+0000","last_undegraded":"2024-04-25T14:05:31.535717+0000","last_fullsized":"2024-04-25T14:05:31.535717+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.539606+0000","last_change":"2024-04-25T14:05:31.539606+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.539606+0000","last_undegraded":"2024-04-25T14:05:31.539606+0000","last_fullsized":"2024-04-25T14:05:31.539606+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.040239+0000","last_change":"2024-04-25T14:05:32.559217+0000","last_active":"2024-04-25T14:05:35.040239+0000","last_peered":"2024-04-25T14:05:35.040239+0000","last_clean":"2024-04-25T14:05:35.040239+0000","last_became_active":"2024-04-25T14:05:32.558909+0000","last_became_peered":"2024-04-25T14:05:32.558909+0000","last_unstale":"2024-04-25T14:05:35.040239+0000","last_undegraded":"2024-04-25T14:05:35.040239+0000","last_fullsized":"2024-04-25T14:05:35.040239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:48:44.213850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039597+0000","last_change":"2024-04-25T14:05:32.566219+0000","last_active":"2024-04-25T14:05:35.039597+0000","last_peered":"2024-04-25T14:05:35.039597+0000","last_clean":"2024-04-25T14:05:35.039597+0000","last_became_active":"2024-04-25T14:05:32.565859+0000","last_became_peered":"2024-04-25T14:05:32.565859+0000","last_unstale":"2024-04-25T14:05:35.039597+0000","last_undegraded":"2024-04-25T14:05:35.039597+0000","last_fullsized":"2024-04-25T14:05:35.039597+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:35.649467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039730+0000","last_change":"2024-04-25T14:05:32.567474+0000","last_active":"2024-04-25T14:05:35.039730+0000","last_peered":"2024-04-25T14:05:35.039730+0000","last_clean":"2024-04-25T14:05:35.039730+0000","last_became_active":"2024-04-25T14:05:32.567061+0000","last_became_peered":"2024-04-25T14:05:32.567061+0000","last_unstale":"2024-04-25T14:05:35.039730+0000","last_undegraded":"2024-04-25T14:05:35.039730+0000","last_fullsized":"2024-04-25T14:05:35.039730+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:20.281168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.532537+0000","last_change":"2024-04-25T14:05:31.532537+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.532537+0000","last_undegraded":"2024-04-25T14:05:31.532537+0000","last_fullsized":"2024-04-25T14:05:31.532537+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039542+0000","last_change":"2024-04-25T14:05:33.570587+0000","last_active":"2024-04-25T14:05:35.039542+0000","last_peered":"2024-04-25T14:05:35.039542+0000","last_clean":"2024-04-25T14:05:35.039542+0000","last_became_active":"2024-04-25T14:05:33.570209+0000","last_became_peered":"2024-04-25T14:05:33.570209+0000","last_unstale":"2024-04-25T14:05:35.039542+0000","last_undegraded":"2024-04-25T14:05:35.039542+0000","last_fullsized":"2024-04-25T14:05:35.039542+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:13.979622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.535614+0000","last_change":"2024-04-25T14:05:31.535614+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.535614+0000","last_undegraded":"2024-04-25T14:05:31.535614+0000","last_fullsized":"2024-04-25T14:05:31.535614+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.541074+0000","last_change":"2024-04-25T14:05:31.541074+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.541074+0000","last_undegraded":"2024-04-25T14:05:31.541074+0000","last_fullsized":"2024-04-25T14:05:31.541074+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039205+0000","last_change":"2024-04-25T14:05:32.553201+0000","last_active":"2024-04-25T14:05:35.039205+0000","last_peered":"2024-04-25T14:05:35.039205+0000","last_clean":"2024-04-25T14:05:35.039205+0000","last_became_active":"2024-04-25T14:05:32.552846+0000","last_became_peered":"2024-04-25T14:05:32.552846+0000","last_unstale":"2024-04-25T14:05:35.039205+0000","last_undegraded":"2024-04-25T14:05:35.039205+0000","last_fullsized":"2024-04-25T14:05:35.039205+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:13.772830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039401+0000","last_change":"2024-04-25T14:05:32.561754+0000","last_active":"2024-04-25T14:05:35.039401+0000","last_peered":"2024-04-25T14:05:35.039401+0000","last_clean":"2024-04-25T14:05:35.039401+0000","last_became_active":"2024-04-25T14:05:32.561297+0000","last_became_peered":"2024-04-25T14:05:32.561297+0000","last_unstale":"2024-04-25T14:05:35.039401+0000","last_undegraded":"2024-04-25T14:05:35.039401+0000","last_fullsized":"2024-04-25T14:05:35.039401+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:39.285483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.539462+0000","last_change":"2024-04-25T14:05:31.539462+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.539462+0000","last_undegraded":"2024-04-25T14:05:31.539462+0000","last_fullsized":"2024-04-25T14:05:31.539462+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.539025+0000","last_change":"2024-04-25T14:05:31.539025+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.539025+0000","last_undegraded":"2024-04-25T14:05:31.539025+0000","last_fullsized":"2024-04-25T14:05:31.539025+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.535050+0000","last_change":"2024-04-25T14:05:31.535050+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.535050+0000","last_undegraded":"2024-04-25T14:05:31.535050+0000","last_fullsized":"2024-04-25T14:05:31.535050+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.535108+0000","last_change":"2024-04-25T14:05:31.535108+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.535108+0000","last_undegraded":"2024-04-25T14:05:31.535108+0000","last_fullsized":"2024-04-25T14:05:31.535108+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.541206+0000","last_change":"2024-04-25T14:05:31.541206+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.541206+0000","last_undegraded":"2024-04-25T14:05:31.541206+0000","last_fullsized":"2024-04-25T14:05:31.541206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.533456+0000","last_change":"2024-04-25T14:05:31.533456+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.533456+0000","last_undegraded":"2024-04-25T14:05:31.533456+0000","last_fullsized":"2024-04-25T14:05:31.533456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039142+0000","last_change":"2024-04-25T14:05:33.563768+0000","last_active":"2024-04-25T14:05:35.039142+0000","last_peered":"2024-04-25T14:05:35.039142+0000","last_clean":"2024-04-25T14:05:35.039142+0000","last_became_active":"2024-04-25T14:05:33.563379+0000","last_became_peered":"2024-04-25T14:05:33.563379+0000","last_unstale":"2024-04-25T14:05:35.039142+0000","last_undegraded":"2024-04-25T14:05:35.039142+0000","last_fullsized":"2024-04-25T14:05:35.039142+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:11.087435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.538960+0000","last_change":"2024-04-25T14:05:31.538960+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.538960+0000","last_undegraded":"2024-04-25T14:05:31.538960+0000","last_fullsized":"2024-04-25T14:05:31.538960+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"39'79","reported_seq":177,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T14:05:31.540738+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:05:31.540738+0000","last_peered":"2024-04-25T14:05:31.540738+0000","last_clean":"2024-04-25T14:05:31.540738+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:05:31.540738+0000","last_undegraded":"2024-04-25T14:05:31.540738+0000","last_fullsized":"2024-04-25T14:05:31.540738+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":2},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":652,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1089536,"data_stored":806812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":552960,"data_stored":245159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":222991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":222991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":222991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":246119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":222991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343740,"statfs":{"total":96636764160,"available":96607989760,"internally_reserved":0,"allocated":1101824,"data_stored":813359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:05:38.080 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:05:38.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.268+0000 7f7e3b200640 1 Processor -- start 2024-04-25T14:05:38.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.268+0000 7f7e3b200640 1 -- start start 2024-04-25T14:05:38.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.268+0000 7f7e3b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e3c0b6ee0 0x7f7e3c0d06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.268+0000 7f7e3b200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7e3c0729b0 0x7f7e3c1614c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.268+0000 7f7e3b200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e3c15df00 con 0x7f7e3c0d0c00 2024-04-25T14:05:38.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.268+0000 7f7e3b200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f7e3c15e1e0 con 0x7f7e3c0729b0 2024-04-25T14:05:38.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.268+0000 7f7e3b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7e3c15e4c0 con 0x7f7e3c0b6ee0 2024-04-25T14:05:38.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.268+0000 7f7e31400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7e3c0d0c00 0x7f7e3c0722a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36710/0 (socket says 172.21.15.86:36710) 2024-04-25T14:05:38.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.268+0000 7f7e31400640 1 -- 172.21.15.86:0/4205062790 learned_addr learned my addr 172.21.15.86:0/4205062790 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:38.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.268+0000 7f7e39e00640 1 --2- 172.21.15.86:0/4205062790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e3c0b6ee0 0x7f7e3c0d06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.268+0000 7f7e3a800640 1 --2- 172.21.15.86:0/4205062790 >> v2:172.21.15.155:3300/0 conn(0x7f7e3c0729b0 0x7f7e3c1614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.268+0000 7f7e39400640 1 -- 172.21.15.86:0/4205062790 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 835832591 0 0) 0x7f7e3c15df00 con 0x7f7e3c0d0c00 2024-04-25T14:05:38.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.268+0000 7f7e39400640 1 -- 172.21.15.86:0/4205062790 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e20003530 con 0x7f7e3c0d0c00 2024-04-25T14:05:38.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.268+0000 7f7e39e00640 1 -- 172.21.15.86:0/4205062790 >> v2:172.21.15.155:3300/0 conn(0x7f7e3c0729b0 msgr2=0x7f7e3c1614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:38.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.268+0000 7f7e39e00640 1 --2- 172.21.15.86:0/4205062790 >> v2:172.21.15.155:3300/0 conn(0x7f7e3c0729b0 0x7f7e3c1614c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.268+0000 7f7e39e00640 1 -- 172.21.15.86:0/4205062790 >> v1:172.21.15.86:6789/0 conn(0x7f7e3c0d0c00 legacy=0x7f7e3c0722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:38.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.268+0000 7f7e39e00640 1 -- 172.21.15.86:0/4205062790 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3c15e7a0 con 0x7f7e3c0b6ee0 2024-04-25T14:05:38.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.272+0000 7f7e39e00640 1 --2- 172.21.15.86:0/4205062790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e3c0b6ee0 0x7f7e3c0d06c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c0097c0 tx=0x7f7e2c01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c589779271f64c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:38.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.272+0000 7f7e39400640 1 -- 172.21.15.86:0/4205062790 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c0b0070 con 0x7f7e3c0b6ee0 2024-04-25T14:05:38.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.272+0000 7f7e39400640 1 -- 172.21.15.86:0/4205062790 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e2c0ab600 con 0x7f7e3c0b6ee0 2024-04-25T14:05:38.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.272+0000 7f7e39400640 1 -- 172.21.15.86:0/4205062790 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c0ab9d0 con 0x7f7e3c0b6ee0 2024-04-25T14:05:38.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.272+0000 7f7e3b200640 1 -- 172.21.15.86:0/4205062790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e3c0b6ee0 msgr2=0x7f7e3c0d06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:38.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.272+0000 7f7e3b200640 1 --2- 172.21.15.86:0/4205062790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e3c0b6ee0 0x7f7e3c0d06c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c0097c0 tx=0x7f7e2c01c410 comp rx=0 tx=0).stop 2024-04-25T14:05:38.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.272+0000 7f7e3b200640 1 -- 172.21.15.86:0/4205062790 shutdown_connections 2024-04-25T14:05:38.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.272+0000 7f7e3b200640 1 --2- 172.21.15.86:0/4205062790 >> v2:172.21.15.155:3300/0 conn(0x7f7e3c0729b0 0x7f7e3c1614c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.272+0000 7f7e3b200640 1 --2- 172.21.15.86:0/4205062790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e3c0b6ee0 0x7f7e3c0d06c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.272+0000 7f7e3b200640 1 -- 172.21.15.86:0/4205062790 >> 172.21.15.86:0/4205062790 conn(0x7f7e3c0b4f80 msgr2=0x7f7e3c0b5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:38.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.272+0000 7f7e3b200640 1 -- 172.21.15.86:0/4205062790 shutdown_connections 2024-04-25T14:05:38.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.272+0000 7f7e3b200640 1 -- 172.21.15.86:0/4205062790 wait complete. 2024-04-25T14:05:38.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.276+0000 7f7e3b200640 1 Processor -- start 2024-04-25T14:05:38.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.276+0000 7f7e3b200640 1 -- start start 2024-04-25T14:05:38.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.276+0000 7f7e3b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e3c0b6ee0 0x7f7e3c1f9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.276+0000 7f7e3b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e3c0d0c00 0x7f7e3c1591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.276+0000 7f7e3b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e3c140670 0x7f7e3c143170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.276+0000 7f7e3b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c16a970 con 0x7f7e3c140670 2024-04-25T14:05:38.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.276+0000 7f7e3b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c0cff50 con 0x7f7e3c0d0c00 2024-04-25T14:05:38.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.276+0000 7f7e3b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7e3c16aaf0 con 0x7f7e3c0b6ee0 2024-04-25T14:05:38.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.276+0000 7f7e3a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e3c140670 0x7f7e3c143170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.276+0000 7f7e39e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e3c0b6ee0 0x7f7e3c1f9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.276+0000 7f7e31400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e3c0d0c00 0x7f7e3c1591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.276+0000 7f7e3a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e3c140670 0x7f7e3c143170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52182/0 (socket says 172.21.15.86:52182) 2024-04-25T14:05:38.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.276+0000 7f7e39e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e3c0b6ee0 0x7f7e3c1f9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:35770/0 (socket says 172.21.15.86:35770) 2024-04-25T14:05:38.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.276+0000 7f7e3a800640 1 -- 172.21.15.86:0/2049162217 learned_addr learned my addr 172.21.15.86:0/2049162217 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:38.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.276+0000 7f7e31400640 1 --2- 172.21.15.86:0/2049162217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e3c0d0c00 0x7f7e3c1591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:51008/0 (socket says 172.21.15.86:51008) 2024-04-25T14:05:38.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.280+0000 7f7e3a800640 1 -- 172.21.15.86:0/2049162217 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e3c0b6ee0 msgr2=0x7f7e3c1f9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:38.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.280+0000 7f7e3a800640 1 --2- 172.21.15.86:0/2049162217 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e3c0b6ee0 0x7f7e3c1f9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.280+0000 7f7e3a800640 1 -- 172.21.15.86:0/2049162217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e3c0d0c00 msgr2=0x7f7e3c1591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:38.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.280+0000 7f7e3a800640 1 --2- 172.21.15.86:0/2049162217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e3c0d0c00 0x7f7e3c1591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.280+0000 7f7e3a800640 1 -- 172.21.15.86:0/2049162217 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3c1420d0 con 0x7f7e3c140670 2024-04-25T14:05:38.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.280+0000 7f7e3a800640 1 --2- 172.21.15.86:0/2049162217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e3c140670 0x7f7e3c143170 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400efc0 tx=0x7f7e3400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:38.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.280+0000 7f7e32a00640 1 -- 172.21.15.86:0/2049162217 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e34018020 con 0x7f7e3c140670 2024-04-25T14:05:38.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.280+0000 7f7e32a00640 1 -- 172.21.15.86:0/2049162217 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e34014480 con 0x7f7e3c140670 2024-04-25T14:05:38.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.280+0000 7f7e3b200640 1 -- 172.21.15.86:0/2049162217 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e3c1423b0 con 0x7f7e3c140670 2024-04-25T14:05:38.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.280+0000 7f7e32a00640 1 -- 172.21.15.86:0/2049162217 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e34009e60 con 0x7f7e3c140670 2024-04-25T14:05:38.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.280+0000 7f7e3b200640 1 -- 172.21.15.86:0/2049162217 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e3c142970 con 0x7f7e3c140670 2024-04-25T14:05:38.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.280+0000 7f7e3b200640 1 -- 172.21.15.86:0/2049162217 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e3c076b80 con 0x7f7e3c140670 2024-04-25T14:05:38.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.280+0000 7f7e32a00640 1 -- 172.21.15.86:0/2049162217 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f7e3401b020 con 0x7f7e3c140670 2024-04-25T14:05:38.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.284+0000 7f7e32a00640 1 --2- 172.21.15.86:0/2049162217 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f7e1806f570 0x7f7e18071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.284+0000 7f7e32a00640 1 -- 172.21.15.86:0/2049162217 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6433+0+0 (secure 0 0 0) 0x7f7e340904d0 con 0x7f7e3c140670 2024-04-25T14:05:38.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.284+0000 7f7e39e00640 1 --2- 172.21.15.86:0/2049162217 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f7e1806f570 0x7f7e18071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.284+0000 7f7e39e00640 1 --2- 172.21.15.86:0/2049162217 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f7e1806f570 0x7f7e18071a80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c0ad000 tx=0x7f7e2c0a6000 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:38.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.288+0000 7f7e32a00640 1 -- 172.21.15.86:0/2049162217 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7e34022020 con 0x7f7e3c140670 2024-04-25T14:05:38.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.440+0000 7f7e3b200640 1 -- 172.21.15.86:0/2049162217 --> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7e3c15c830 con 0x7f7e1806f570 2024-04-25T14:05:38.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.444+0000 7f7e32a00640 1 -- 172.21.15.86:0/2049162217 <== mgr.5532 v2:172.21.15.86:6832/2427179681 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242125 (secure 0 0 0) 0x7f7e3c15c830 con 0x7f7e1806f570 2024-04-25T14:05:38.450 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:38.451 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:05:38.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.448+0000 7f7e3b200640 1 -- 172.21.15.86:0/2049162217 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f7e1806f570 msgr2=0x7f7e18071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:38.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.448+0000 7f7e3b200640 1 --2- 172.21.15.86:0/2049162217 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f7e1806f570 0x7f7e18071a80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c0ad000 tx=0x7f7e2c0a6000 comp rx=0 tx=0).stop 2024-04-25T14:05:38.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.448+0000 7f7e3b200640 1 -- 172.21.15.86:0/2049162217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e3c140670 msgr2=0x7f7e3c143170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:38.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.448+0000 7f7e3b200640 1 --2- 172.21.15.86:0/2049162217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e3c140670 0x7f7e3c143170 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400efc0 tx=0x7f7e3400c490 comp rx=0 tx=0).stop 2024-04-25T14:05:38.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.448+0000 7f7e3b200640 1 -- 172.21.15.86:0/2049162217 shutdown_connections 2024-04-25T14:05:38.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.448+0000 7f7e3b200640 1 --2- 172.21.15.86:0/2049162217 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f7e1806f570 0x7f7e18071a80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.448+0000 7f7e3b200640 1 --2- 172.21.15.86:0/2049162217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e3c140670 0x7f7e3c143170 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.448+0000 7f7e3b200640 1 --2- 172.21.15.86:0/2049162217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e3c0d0c00 0x7f7e3c1591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.448+0000 7f7e3b200640 1 --2- 172.21.15.86:0/2049162217 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e3c0b6ee0 0x7f7e3c1f9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.448+0000 7f7e3b200640 1 -- 172.21.15.86:0/2049162217 >> 172.21.15.86:0/2049162217 conn(0x7f7e3c0b4f80 msgr2=0x7f7e3c0722a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:38.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.448+0000 7f7e3b200640 1 -- 172.21.15.86:0/2049162217 shutdown_connections 2024-04-25T14:05:38.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.448+0000 7f7e3b200640 1 -- 172.21.15.86:0/2049162217 wait complete. 2024-04-25T14:05:38.471 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-04-25T14:05:37.482712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591020,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":124,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88,"ondisk_log_size":88,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221384,"kb_used_data":5208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753336,"statfs":{"total":773094113280,"available":772867416064,"internally_reserved":0,"allocated":5332992,"data_stored":3003413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":652,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.952497"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.040189+0000","last_change":"2024-04-25T14:05:33.573200+0000","last_active":"2024-04-25T14:05:35.040189+0000","last_peered":"2024-04-25T14:05:35.040189+0000","last_clean":"2024-04-25T14:05:35.040189+0000","last_became_active":"2024-04-25T14:05:33.572858+0000","last_became_peered":"2024-04-25T14:05:33.572858+0000","last_unstale":"2024-04-25T14:05:35.040189+0000","last_undegraded":"2024-04-25T14:05:35.040189+0000","last_fullsized":"2024-04-25T14:05:35.040189+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:22.996486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039514+0000","last_change":"2024-04-25T14:05:33.571072+0000","last_active":"2024-04-25T14:05:35.039514+0000","last_peered":"2024-04-25T14:05:35.039514+0000","last_clean":"2024-04-25T14:05:35.039514+0000","last_became_active":"2024-04-25T14:05:33.570748+0000","last_became_peered":"2024-04-25T14:05:33.570748+0000","last_unstale":"2024-04-25T14:05:35.039514+0000","last_undegraded":"2024-04-25T14:05:35.039514+0000","last_fullsized":"2024-04-25T14:05:35.039514+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:18.074490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039301+0000","last_change":"2024-04-25T14:05:33.571248+0000","last_active":"2024-04-25T14:05:35.039301+0000","last_peered":"2024-04-25T14:05:35.039301+0000","last_clean":"2024-04-25T14:05:35.039301+0000","last_became_active":"2024-04-25T14:05:33.570916+0000","last_became_peered":"2024-04-25T14:05:33.570916+0000","last_unstale":"2024-04-25T14:05:35.039301+0000","last_undegraded":"2024-04-25T14:05:35.039301+0000","last_fullsized":"2024-04-25T14:05:35.039301+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:36.296676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039600+0000","last_change":"2024-04-25T14:05:33.576117+0000","last_active":"2024-04-25T14:05:35.039600+0000","last_peered":"2024-04-25T14:05:35.039600+0000","last_clean":"2024-04-25T14:05:35.039600+0000","last_became_active":"2024-04-25T14:05:33.575838+0000","last_became_peered":"2024-04-25T14:05:33.575838+0000","last_unstale":"2024-04-25T14:05:35.039600+0000","last_undegraded":"2024-04-25T14:05:35.039600+0000","last_fullsized":"2024-04-25T14:05:35.039600+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:20.619851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039689+0000","last_change":"2024-04-25T14:05:33.596454+0000","last_active":"2024-04-25T14:05:35.039689+0000","last_peered":"2024-04-25T14:05:35.039689+0000","last_clean":"2024-04-25T14:05:35.039689+0000","last_became_active":"2024-04-25T14:05:33.596225+0000","last_became_peered":"2024-04-25T14:05:33.596225+0000","last_unstale":"2024-04-25T14:05:35.039689+0000","last_undegraded":"2024-04-25T14:05:35.039689+0000","last_fullsized":"2024-04-25T14:05:35.039689+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:41.512359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"45'4","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039294+0000","last_change":"2024-04-25T14:05:32.556304+0000","last_active":"2024-04-25T14:05:35.039294+0000","last_peered":"2024-04-25T14:05:35.039294+0000","last_clean":"2024-04-25T14:05:35.039294+0000","last_became_active":"2024-04-25T14:05:32.556052+0000","last_became_peered":"2024-04-25T14:05:32.556052+0000","last_unstale":"2024-04-25T14:05:35.039294+0000","last_undegraded":"2024-04-25T14:05:35.039294+0000","last_fullsized":"2024-04-25T14:05:35.039294+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:16.452558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.532639+0000","last_change":"2024-04-25T14:05:31.532639+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.532639+0000","last_undegraded":"2024-04-25T14:05:31.532639+0000","last_fullsized":"2024-04-25T14:05:31.532639+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039348+0000","last_change":"2024-04-25T14:05:33.574504+0000","last_active":"2024-04-25T14:05:35.039348+0000","last_peered":"2024-04-25T14:05:35.039348+0000","last_clean":"2024-04-25T14:05:35.039348+0000","last_became_active":"2024-04-25T14:05:33.574191+0000","last_became_peered":"2024-04-25T14:05:33.574191+0000","last_unstale":"2024-04-25T14:05:35.039348+0000","last_undegraded":"2024-04-25T14:05:35.039348+0000","last_fullsized":"2024-04-25T14:05:35.039348+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:28.629474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.533022+0000","last_change":"2024-04-25T14:05:31.533022+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.533022+0000","last_undegraded":"2024-04-25T14:05:31.533022+0000","last_fullsized":"2024-04-25T14:05:31.533022+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.533643+0000","last_change":"2024-04-25T14:05:31.533643+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.533643+0000","last_undegraded":"2024-04-25T14:05:31.533643+0000","last_fullsized":"2024-04-25T14:05:31.533643+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.540931+0000","last_change":"2024-04-25T14:05:31.540931+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.540931+0000","last_undegraded":"2024-04-25T14:05:31.540931+0000","last_fullsized":"2024-04-25T14:05:31.540931+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.541762+0000","last_change":"2024-04-25T14:05:31.541762+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.541762+0000","last_undegraded":"2024-04-25T14:05:31.541762+0000","last_fullsized":"2024-04-25T14:05:31.541762+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039072+0000","last_change":"2024-04-25T14:05:32.551806+0000","last_active":"2024-04-25T14:05:35.039072+0000","last_peered":"2024-04-25T14:05:35.039072+0000","last_clean":"2024-04-25T14:05:35.039072+0000","last_became_active":"2024-04-25T14:05:32.551441+0000","last_became_peered":"2024-04-25T14:05:32.551441+0000","last_unstale":"2024-04-25T14:05:35.039072+0000","last_undegraded":"2024-04-25T14:05:35.039072+0000","last_fullsized":"2024-04-25T14:05:35.039072+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:31.414048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.538948+0000","last_change":"2024-04-25T14:05:31.538948+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.538948+0000","last_undegraded":"2024-04-25T14:05:31.538948+0000","last_fullsized":"2024-04-25T14:05:31.538948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.539519+0000","last_change":"2024-04-25T14:05:31.539519+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.539519+0000","last_undegraded":"2024-04-25T14:05:31.539519+0000","last_fullsized":"2024-04-25T14:05:31.539519+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.542126+0000","last_change":"2024-04-25T14:05:31.542126+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.542126+0000","last_undegraded":"2024-04-25T14:05:31.542126+0000","last_fullsized":"2024-04-25T14:05:31.542126+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.540063+0000","last_change":"2024-04-25T14:05:31.540063+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.540063+0000","last_undegraded":"2024-04-25T14:05:31.540063+0000","last_fullsized":"2024-04-25T14:05:31.540063+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.535803+0000","last_change":"2024-04-25T14:05:31.535803+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.535803+0000","last_undegraded":"2024-04-25T14:05:31.535803+0000","last_fullsized":"2024-04-25T14:05:31.535803+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.540372+0000","last_change":"2024-04-25T14:05:31.540372+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.540372+0000","last_undegraded":"2024-04-25T14:05:31.540372+0000","last_fullsized":"2024-04-25T14:05:31.540372+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039235+0000","last_change":"2024-04-25T14:05:32.562133+0000","last_active":"2024-04-25T14:05:35.039235+0000","last_peered":"2024-04-25T14:05:35.039235+0000","last_clean":"2024-04-25T14:05:35.039235+0000","last_became_active":"2024-04-25T14:05:32.561845+0000","last_became_peered":"2024-04-25T14:05:32.561845+0000","last_unstale":"2024-04-25T14:05:35.039235+0000","last_undegraded":"2024-04-25T14:05:35.039235+0000","last_fullsized":"2024-04-25T14:05:35.039235+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:14.012822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.539420+0000","last_change":"2024-04-25T14:05:31.539420+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.539420+0000","last_undegraded":"2024-04-25T14:05:31.539420+0000","last_fullsized":"2024-04-25T14:05:31.539420+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.535717+0000","last_change":"2024-04-25T14:05:31.535717+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.535717+0000","last_undegraded":"2024-04-25T14:05:31.535717+0000","last_fullsized":"2024-04-25T14:05:31.535717+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.539606+0000","last_change":"2024-04-25T14:05:31.539606+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.539606+0000","last_undegraded":"2024-04-25T14:05:31.539606+0000","last_fullsized":"2024-04-25T14:05:31.539606+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.040239+0000","last_change":"2024-04-25T14:05:32.559217+0000","last_active":"2024-04-25T14:05:35.040239+0000","last_peered":"2024-04-25T14:05:35.040239+0000","last_clean":"2024-04-25T14:05:35.040239+0000","last_became_active":"2024-04-25T14:05:32.558909+0000","last_became_peered":"2024-04-25T14:05:32.558909+0000","last_unstale":"2024-04-25T14:05:35.040239+0000","last_undegraded":"2024-04-25T14:05:35.040239+0000","last_fullsized":"2024-04-25T14:05:35.040239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:48:44.213850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039597+0000","last_change":"2024-04-25T14:05:32.566219+0000","last_active":"2024-04-25T14:05:35.039597+0000","last_peered":"2024-04-25T14:05:35.039597+0000","last_clean":"2024-04-25T14:05:35.039597+0000","last_became_active":"2024-04-25T14:05:32.565859+0000","last_became_peered":"2024-04-25T14:05:32.565859+0000","last_unstale":"2024-04-25T14:05:35.039597+0000","last_undegraded":"2024-04-25T14:05:35.039597+0000","last_fullsized":"2024-04-25T14:05:35.039597+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:35.649467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039730+0000","last_change":"2024-04-25T14:05:32.567474+0000","last_active":"2024-04-25T14:05:35.039730+0000","last_peered":"2024-04-25T14:05:35.039730+0000","last_clean":"2024-04-25T14:05:35.039730+0000","last_became_active":"2024-04-25T14:05:32.567061+0000","last_became_peered":"2024-04-25T14:05:32.567061+0000","last_unstale":"2024-04-25T14:05:35.039730+0000","last_undegraded":"2024-04-25T14:05:35.039730+0000","last_fullsized":"2024-04-25T14:05:35.039730+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:20.281168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.532537+0000","last_change":"2024-04-25T14:05:31.532537+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.532537+0000","last_undegraded":"2024-04-25T14:05:31.532537+0000","last_fullsized":"2024-04-25T14:05:31.532537+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039542+0000","last_change":"2024-04-25T14:05:33.570587+0000","last_active":"2024-04-25T14:05:35.039542+0000","last_peered":"2024-04-25T14:05:35.039542+0000","last_clean":"2024-04-25T14:05:35.039542+0000","last_became_active":"2024-04-25T14:05:33.570209+0000","last_became_peered":"2024-04-25T14:05:33.570209+0000","last_unstale":"2024-04-25T14:05:35.039542+0000","last_undegraded":"2024-04-25T14:05:35.039542+0000","last_fullsized":"2024-04-25T14:05:35.039542+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:13.979622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.535614+0000","last_change":"2024-04-25T14:05:31.535614+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.535614+0000","last_undegraded":"2024-04-25T14:05:31.535614+0000","last_fullsized":"2024-04-25T14:05:31.535614+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.541074+0000","last_change":"2024-04-25T14:05:31.541074+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.541074+0000","last_undegraded":"2024-04-25T14:05:31.541074+0000","last_fullsized":"2024-04-25T14:05:31.541074+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039205+0000","last_change":"2024-04-25T14:05:32.553201+0000","last_active":"2024-04-25T14:05:35.039205+0000","last_peered":"2024-04-25T14:05:35.039205+0000","last_clean":"2024-04-25T14:05:35.039205+0000","last_became_active":"2024-04-25T14:05:32.552846+0000","last_became_peered":"2024-04-25T14:05:32.552846+0000","last_unstale":"2024-04-25T14:05:35.039205+0000","last_undegraded":"2024-04-25T14:05:35.039205+0000","last_fullsized":"2024-04-25T14:05:35.039205+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:13.772830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039401+0000","last_change":"2024-04-25T14:05:32.561754+0000","last_active":"2024-04-25T14:05:35.039401+0000","last_peered":"2024-04-25T14:05:35.039401+0000","last_clean":"2024-04-25T14:05:35.039401+0000","last_became_active":"2024-04-25T14:05:32.561297+0000","last_became_peered":"2024-04-25T14:05:32.561297+0000","last_unstale":"2024-04-25T14:05:35.039401+0000","last_undegraded":"2024-04-25T14:05:35.039401+0000","last_fullsized":"2024-04-25T14:05:35.039401+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:39.285483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.539462+0000","last_change":"2024-04-25T14:05:31.539462+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.539462+0000","last_undegraded":"2024-04-25T14:05:31.539462+0000","last_fullsized":"2024-04-25T14:05:31.539462+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.539025+0000","last_change":"2024-04-25T14:05:31.539025+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.539025+0000","last_undegraded":"2024-04-25T14:05:31.539025+0000","last_fullsized":"2024-04-25T14:05:31.539025+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.535050+0000","last_change":"2024-04-25T14:05:31.535050+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.535050+0000","last_undegraded":"2024-04-25T14:05:31.535050+0000","last_fullsized":"2024-04-25T14:05:31.535050+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.535108+0000","last_change":"2024-04-25T14:05:31.535108+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.535108+0000","last_undegraded":"2024-04-25T14:05:31.535108+0000","last_fullsized":"2024-04-25T14:05:31.535108+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.541206+0000","last_change":"2024-04-25T14:05:31.541206+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.541206+0000","last_undegraded":"2024-04-25T14:05:31.541206+0000","last_fullsized":"2024-04-25T14:05:31.541206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.533456+0000","last_change":"2024-04-25T14:05:31.533456+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.533456+0000","last_undegraded":"2024-04-25T14:05:31.533456+0000","last_fullsized":"2024-04-25T14:05:31.533456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039142+0000","last_change":"2024-04-25T14:05:33.563768+0000","last_active":"2024-04-25T14:05:35.039142+0000","last_peered":"2024-04-25T14:05:35.039142+0000","last_clean":"2024-04-25T14:05:35.039142+0000","last_became_active":"2024-04-25T14:05:33.563379+0000","last_became_peered":"2024-04-25T14:05:33.563379+0000","last_unstale":"2024-04-25T14:05:35.039142+0000","last_undegraded":"2024-04-25T14:05:35.039142+0000","last_fullsized":"2024-04-25T14:05:35.039142+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:11.087435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.538960+0000","last_change":"2024-04-25T14:05:31.538960+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.538960+0000","last_undegraded":"2024-04-25T14:05:31.538960+0000","last_fullsized":"2024-04-25T14:05:31.538960+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"39'79","reported_seq":177,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T14:05:31.540738+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:05:31.540738+0000","last_peered":"2024-04-25T14:05:31.540738+0000","last_clean":"2024-04-25T14:05:31.540738+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:05:31.540738+0000","last_undegraded":"2024-04-25T14:05:31.540738+0000","last_fullsized":"2024-04-25T14:05:31.540738+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":2},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":652,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1089536,"data_stored":806812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":552960,"data_stored":245159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":222991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":222991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":222991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":246119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":222991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343740,"statfs":{"total":96636764160,"available":96607989760,"internally_reserved":0,"allocated":1101824,"data_stored":813359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:05:38.475 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3f is not active+clean 2024-04-25T14:05:38.475 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.475 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3e is not active+clean 2024-04-25T14:05:38.475 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.475 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3d is not active+clean 2024-04-25T14:05:38.475 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3c is not active+clean 2024-04-25T14:05:38.476 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3b is not active+clean 2024-04-25T14:05:38.476 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3a is not active+clean 2024-04-25T14:05:38.476 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.39 is not active+clean 2024-04-25T14:05:38.476 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.38 is not active+clean 2024-04-25T14:05:38.476 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.37 is not active+clean 2024-04-25T14:05:38.477 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.477 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.36 is not active+clean 2024-04-25T14:05:38.477 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.477 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.34 is not active+clean 2024-04-25T14:05:38.477 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.477 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.33 is not active+clean 2024-04-25T14:05:38.477 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.477 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.32 is not active+clean 2024-04-25T14:05:38.477 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.477 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.31 is not active+clean 2024-04-25T14:05:38.477 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.30 is not active+clean 2024-04-25T14:05:38.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2f is not active+clean 2024-04-25T14:05:38.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2e is not active+clean 2024-04-25T14:05:38.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2d is not active+clean 2024-04-25T14:05:38.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2a is not active+clean 2024-04-25T14:05:38.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.28 is not active+clean 2024-04-25T14:05:38.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.27 is not active+clean 2024-04-25T14:05:38.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.26 is not active+clean 2024-04-25T14:05:38.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.25 is not active+clean 2024-04-25T14:05:38.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.23 is not active+clean 2024-04-25T14:05:38.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.22 is not active+clean 2024-04-25T14:05:38.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.21 is not active+clean 2024-04-25T14:05:38.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.20 is not active+clean 2024-04-25T14:05:38.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.10 is not active+clean 2024-04-25T14:05:38.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.11 is not active+clean 2024-04-25T14:05:38.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2024-04-25T14:05:38.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:05:31.532639+0000', 'last_change': '2024-04-25T14:05:31.532639+0000', 'last_active': '2024-04-25T14:05:31.523192+0000', 'last_peered': '2024-04-25T14:05:31.523192+0000', 'last_clean': '2024-04-25T14:05:31.523192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:31.532639+0000', 'last_undegraded': '2024-04-25T14:05:31.532639+0000', 'last_fullsized': '2024-04-25T14:05:31.532639+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:05:38.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2024-04-25T14:05:38.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:05:31.533022+0000', 'last_change': '2024-04-25T14:05:31.533022+0000', 'last_active': '2024-04-25T14:05:31.523192+0000', 'last_peered': '2024-04-25T14:05:31.523192+0000', 'last_clean': '2024-04-25T14:05:31.523192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:31.533022+0000', 'last_undegraded': '2024-04-25T14:05:31.533022+0000', 'last_fullsized': '2024-04-25T14:05:31.533022+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:05:38.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2024-04-25T14:05:38.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2024-04-25T14:05:38.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:05:31.533643+0000', 'last_change': '2024-04-25T14:05:31.533643+0000', 'last_active': '2024-04-25T14:05:31.523192+0000', 'last_peered': '2024-04-25T14:05:31.523192+0000', 'last_clean': '2024-04-25T14:05:31.523192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:31.533643+0000', 'last_undegraded': '2024-04-25T14:05:31.533643+0000', 'last_fullsized': '2024-04-25T14:05:31.533643+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:05:38.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.14 is not active+clean 2024-04-25T14:05:38.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1b is not active+clean 2024-04-25T14:05:38.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:05:31.540931+0000', 'last_change': '2024-04-25T14:05:31.540931+0000', 'last_active': '2024-04-25T14:05:31.523192+0000', 'last_peered': '2024-04-25T14:05:31.523192+0000', 'last_clean': '2024-04-25T14:05:31.523192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:31.540931+0000', 'last_undegraded': '2024-04-25T14:05:31.540931+0000', 'last_fullsized': '2024-04-25T14:05:31.540931+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:05:38.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.15 is not active+clean 2024-04-25T14:05:38.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2024-04-25T14:05:38.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:05:31.541762+0000', 'last_change': '2024-04-25T14:05:31.541762+0000', 'last_active': '2024-04-25T14:05:31.523192+0000', 'last_peered': '2024-04-25T14:05:31.523192+0000', 'last_clean': '2024-04-25T14:05:31.523192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:31.541762+0000', 'last_undegraded': '2024-04-25T14:05:31.541762+0000', 'last_fullsized': '2024-04-25T14:05:31.541762+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:05:38.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2024-04-25T14:05:38.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.17 is not active+clean 2024-04-25T14:05:38.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.18 is not active+clean 2024-04-25T14:05:38.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:05:31.538948+0000', 'last_change': '2024-04-25T14:05:31.538948+0000', 'last_active': '2024-04-25T14:05:31.523192+0000', 'last_peered': '2024-04-25T14:05:31.523192+0000', 'last_clean': '2024-04-25T14:05:31.523192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:31.538948+0000', 'last_undegraded': '2024-04-25T14:05:31.538948+0000', 'last_fullsized': '2024-04-25T14:05:31.538948+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:05:38.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.18 is not active+clean 2024-04-25T14:05:38.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2024-04-25T14:05:38.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:05:31.539519+0000', 'last_change': '2024-04-25T14:05:31.539519+0000', 'last_active': '2024-04-25T14:05:31.523192+0000', 'last_peered': '2024-04-25T14:05:31.523192+0000', 'last_clean': '2024-04-25T14:05:31.523192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:31.539519+0000', 'last_undegraded': '2024-04-25T14:05:31.539519+0000', 'last_fullsized': '2024-04-25T14:05:31.539519+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:05:38.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.19 is not active+clean 2024-04-25T14:05:38.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2024-04-25T14:05:38.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:05:31.542126+0000', 'last_change': '2024-04-25T14:05:31.542126+0000', 'last_active': '2024-04-25T14:05:31.523192+0000', 'last_peered': '2024-04-25T14:05:31.523192+0000', 'last_clean': '2024-04-25T14:05:31.523192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:31.542126+0000', 'last_undegraded': '2024-04-25T14:05:31.542126+0000', 'last_fullsized': '2024-04-25T14:05:31.542126+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:05:38.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1a is not active+clean 2024-04-25T14:05:38.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2024-04-25T14:05:38.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:05:31.540063+0000', 'last_change': '2024-04-25T14:05:31.540063+0000', 'last_active': '2024-04-25T14:05:31.523192+0000', 'last_peered': '2024-04-25T14:05:31.523192+0000', 'last_clean': '2024-04-25T14:05:31.523192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:31.540063+0000', 'last_undegraded': '2024-04-25T14:05:31.540063+0000', 'last_fullsized': '2024-04-25T14:05:31.540063+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:05:38.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1b is not active+clean 2024-04-25T14:05:38.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2024-04-25T14:05:38.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:05:31.535803+0000', 'last_change': '2024-04-25T14:05:31.535803+0000', 'last_active': '2024-04-25T14:05:31.523192+0000', 'last_peered': '2024-04-25T14:05:31.523192+0000', 'last_clean': '2024-04-25T14:05:31.523192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:31.535803+0000', 'last_undegraded': '2024-04-25T14:05:31.535803+0000', 'last_fullsized': '2024-04-25T14:05:31.535803+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:05:38.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1c is not active+clean 2024-04-25T14:05:38.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.13 is not active+clean 2024-04-25T14:05:38.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:05:31.540372+0000', 'last_change': '2024-04-25T14:05:31.540372+0000', 'last_active': '2024-04-25T14:05:31.523192+0000', 'last_peered': '2024-04-25T14:05:31.523192+0000', 'last_clean': '2024-04-25T14:05:31.523192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:31.540372+0000', 'last_undegraded': '2024-04-25T14:05:31.540372+0000', 'last_fullsized': '2024-04-25T14:05:31.540372+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:05:38.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1d is not active+clean 2024-04-25T14:05:38.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1e is not active+clean 2024-04-25T14:05:38.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2024-04-25T14:05:38.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:05:31.539420+0000', 'last_change': '2024-04-25T14:05:31.539420+0000', 'last_active': '2024-04-25T14:05:31.523192+0000', 'last_peered': '2024-04-25T14:05:31.523192+0000', 'last_clean': '2024-04-25T14:05:31.523192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:31.539420+0000', 'last_undegraded': '2024-04-25T14:05:31.539420+0000', 'last_fullsized': '2024-04-25T14:05:31.539420+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:05:38.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1f is not active+clean 2024-04-25T14:05:38.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2024-04-25T14:05:38.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:05:31.535717+0000', 'last_change': '2024-04-25T14:05:31.535717+0000', 'last_active': '2024-04-25T14:05:31.523192+0000', 'last_peered': '2024-04-25T14:05:31.523192+0000', 'last_clean': '2024-04-25T14:05:31.523192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:31.535717+0000', 'last_undegraded': '2024-04-25T14:05:31.535717+0000', 'last_fullsized': '2024-04-25T14:05:31.535717+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:05:38.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2024-04-25T14:05:38.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.f is not active+clean 2024-04-25T14:05:38.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:05:31.539606+0000', 'last_change': '2024-04-25T14:05:31.539606+0000', 'last_active': '2024-04-25T14:05:31.523192+0000', 'last_peered': '2024-04-25T14:05:31.523192+0000', 'last_clean': '2024-04-25T14:05:31.523192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:31.539606+0000', 'last_undegraded': '2024-04-25T14:05:31.539606+0000', 'last_fullsized': '2024-04-25T14:05:31.539606+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:05:38.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1 is not active+clean 2024-04-25T14:05:38.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2 is not active+clean 2024-04-25T14:05:38.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3 is not active+clean 2024-04-25T14:05:38.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.4 is not active+clean 2024-04-25T14:05:38.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2024-04-25T14:05:38.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:05:31.532537+0000', 'last_change': '2024-04-25T14:05:31.532537+0000', 'last_active': '2024-04-25T14:05:31.523192+0000', 'last_peered': '2024-04-25T14:05:31.523192+0000', 'last_clean': '2024-04-25T14:05:31.523192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:31.532537+0000', 'last_undegraded': '2024-04-25T14:05:31.532537+0000', 'last_fullsized': '2024-04-25T14:05:31.532537+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:05:38.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2024-04-25T14:05:38.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:05:31.535614+0000', 'last_change': '2024-04-25T14:05:31.535614+0000', 'last_active': '2024-04-25T14:05:31.523192+0000', 'last_peered': '2024-04-25T14:05:31.523192+0000', 'last_clean': '2024-04-25T14:05:31.523192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:31.535614+0000', 'last_undegraded': '2024-04-25T14:05:31.535614+0000', 'last_fullsized': '2024-04-25T14:05:31.535614+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:05:38.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.6 is not active+clean 2024-04-25T14:05:38.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2024-04-25T14:05:38.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:05:31.541074+0000', 'last_change': '2024-04-25T14:05:31.541074+0000', 'last_active': '2024-04-25T14:05:31.523192+0000', 'last_peered': '2024-04-25T14:05:31.523192+0000', 'last_clean': '2024-04-25T14:05:31.523192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:31.541074+0000', 'last_undegraded': '2024-04-25T14:05:31.541074+0000', 'last_fullsized': '2024-04-25T14:05:31.541074+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:05:38.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.7 is not active+clean 2024-04-25T14:05:38.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.8 is not active+clean 2024-04-25T14:05:38.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.a is not active+clean 2024-04-25T14:05:38.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2024-04-25T14:05:38.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:05:31.539462+0000', 'last_change': '2024-04-25T14:05:31.539462+0000', 'last_active': '2024-04-25T14:05:31.523192+0000', 'last_peered': '2024-04-25T14:05:31.523192+0000', 'last_clean': '2024-04-25T14:05:31.523192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:31.539462+0000', 'last_undegraded': '2024-04-25T14:05:31.539462+0000', 'last_fullsized': '2024-04-25T14:05:31.539462+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:05:38.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2024-04-25T14:05:38.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.4 is not active+clean 2024-04-25T14:05:38.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:05:31.539025+0000', 'last_change': '2024-04-25T14:05:31.539025+0000', 'last_active': '2024-04-25T14:05:31.523192+0000', 'last_peered': '2024-04-25T14:05:31.523192+0000', 'last_clean': '2024-04-25T14:05:31.523192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:31.539025+0000', 'last_undegraded': '2024-04-25T14:05:31.539025+0000', 'last_fullsized': '2024-04-25T14:05:31.539025+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:05:38.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.c is not active+clean 2024-04-25T14:05:38.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2024-04-25T14:05:38.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:05:31.535050+0000', 'last_change': '2024-04-25T14:05:31.535050+0000', 'last_active': '2024-04-25T14:05:31.523192+0000', 'last_peered': '2024-04-25T14:05:31.523192+0000', 'last_clean': '2024-04-25T14:05:31.523192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:31.535050+0000', 'last_undegraded': '2024-04-25T14:05:31.535050+0000', 'last_fullsized': '2024-04-25T14:05:31.535050+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:05:38.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.d is not active+clean 2024-04-25T14:05:38.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2024-04-25T14:05:38.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:05:31.535108+0000', 'last_change': '2024-04-25T14:05:31.535108+0000', 'last_active': '2024-04-25T14:05:31.523192+0000', 'last_peered': '2024-04-25T14:05:31.523192+0000', 'last_clean': '2024-04-25T14:05:31.523192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:31.535108+0000', 'last_undegraded': '2024-04-25T14:05:31.535108+0000', 'last_fullsized': '2024-04-25T14:05:31.535108+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:05:38.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.e is not active+clean 2024-04-25T14:05:38.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2024-04-25T14:05:38.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:05:31.541206+0000', 'last_change': '2024-04-25T14:05:31.541206+0000', 'last_active': '2024-04-25T14:05:31.523192+0000', 'last_peered': '2024-04-25T14:05:31.523192+0000', 'last_clean': '2024-04-25T14:05:31.523192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:31.541206+0000', 'last_undegraded': '2024-04-25T14:05:31.541206+0000', 'last_fullsized': '2024-04-25T14:05:31.541206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:05:38.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.f is not active+clean 2024-04-25T14:05:38.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T14:05:32.529615+0000', 'last_change': '2024-04-25T14:05:32.529615+0000', 'last_active': '2024-04-25T14:05:32.529615+0000', 'last_peered': '2024-04-25T14:05:32.529615+0000', 'last_clean': '2024-04-25T14:05:32.529615+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:32.529615+0000', 'last_undegraded': '2024-04-25T14:05:32.529615+0000', 'last_fullsized': '2024-04-25T14:05:32.529615+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:32.529615+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T14:05:38.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2024-04-25T14:05:38.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:05:31.533456+0000', 'last_change': '2024-04-25T14:05:31.533456+0000', 'last_active': '2024-04-25T14:05:31.523192+0000', 'last_peered': '2024-04-25T14:05:31.523192+0000', 'last_clean': '2024-04-25T14:05:31.523192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:31.533456+0000', 'last_undegraded': '2024-04-25T14:05:31.533456+0000', 'last_fullsized': '2024-04-25T14:05:31.533456+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:05:38.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2024-04-25T14:05:38.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:05:31.538960+0000', 'last_change': '2024-04-25T14:05:31.538960+0000', 'last_active': '2024-04-25T14:05:31.523192+0000', 'last_peered': '2024-04-25T14:05:31.523192+0000', 'last_clean': '2024-04-25T14:05:31.523192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:05:31.538960+0000', 'last_undegraded': '2024-04-25T14:05:31.538960+0000', 'last_fullsized': '2024-04-25T14:05:31.538960+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'last_clean_scrub_stamp': '2024-04-25T14:05:31.523192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:05:38.491 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T14:05:38.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.656+0000 7f40fda00640 1 Processor -- start 2024-04-25T14:05:38.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.656+0000 7f40fda00640 1 -- start start 2024-04-25T14:05:38.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.656+0000 7f40fda00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f40f8071530 0x7f40f806e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.656+0000 7f40fda00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40f806e7a0 0x7f40f80cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.656+0000 7f40fda00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40f806ef90 con 0x7f40f80cf970 2024-04-25T14:05:38.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.656+0000 7f40fda00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f40f8163830 con 0x7f40f8071530 2024-04-25T14:05:38.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.656+0000 7f40fda00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f40f8163a80 con 0x7f40f806e7a0 2024-04-25T14:05:38.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.656+0000 7f40f6000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f40f8071530 0x7f40f806e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.656+0000 7f40f7400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40f806e7a0 0x7f40f80cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.660+0000 7f40f6a00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f40f80cf970 0x7f40f8070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:55662/0 (socket says 172.21.15.86:55662) 2024-04-25T14:05:38.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.660+0000 7f40f6a00640 1 -- 172.21.15.86:0/3661103479 learned_addr learned my addr 172.21.15.86:0/3661103479 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:38.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.660+0000 7f40f6000640 1 -- 172.21.15.86:0/3661103479 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40f806e7a0 msgr2=0x7f40f80cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:38.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.660+0000 7f40f6000640 1 --2- 172.21.15.86:0/3661103479 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40f806e7a0 0x7f40f80cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.660+0000 7f40f6000640 1 -- 172.21.15.86:0/3661103479 >> v1:172.21.15.155:6789/0 conn(0x7f40f80cf970 legacy=0x7f40f8070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:38.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.660+0000 7f40f6000640 1 -- 172.21.15.86:0/3661103479 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f8163d60 con 0x7f40f8071530 2024-04-25T14:05:38.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.660+0000 7f40f5600640 1 -- 172.21.15.86:0/3661103479 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3968759777 0 0) 0x7f40f806ef90 con 0x7f40f80cf970 2024-04-25T14:05:38.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.660+0000 7f40f7400640 1 --2- 172.21.15.86:0/3661103479 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40f806e7a0 0x7f40f80cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:38.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.660+0000 7f40f6000640 1 --2- 172.21.15.86:0/3661103479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40f8071530 0x7f40f806e260 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f40e40097a0 tx=0x7f40e401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edf6045c2d5c1cd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:38.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.660+0000 7f40f5600640 1 -- 172.21.15.86:0/3661103479 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e40a7070 con 0x7f40f8071530 2024-04-25T14:05:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.660+0000 7f40f5600640 1 -- 172.21.15.86:0/3661103479 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e40b0070 con 0x7f40f8071530 2024-04-25T14:05:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.664+0000 7f40f5600640 1 -- 172.21.15.86:0/3661103479 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e40ab890 con 0x7f40f8071530 2024-04-25T14:05:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.664+0000 7f40fda00640 1 -- 172.21.15.86:0/3661103479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40f8071530 msgr2=0x7f40f806e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.664+0000 7f40fda00640 1 --2- 172.21.15.86:0/3661103479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40f8071530 0x7f40f806e260 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f40e40097a0 tx=0x7f40e401c450 comp rx=0 tx=0).stop 2024-04-25T14:05:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.664+0000 7f40fda00640 1 -- 172.21.15.86:0/3661103479 shutdown_connections 2024-04-25T14:05:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.664+0000 7f40fda00640 1 --2- 172.21.15.86:0/3661103479 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40f806e7a0 0x7f40f80cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.664+0000 7f40fda00640 1 --2- 172.21.15.86:0/3661103479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40f8071530 0x7f40f806e260 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.664+0000 7f40fda00640 1 -- 172.21.15.86:0/3661103479 >> 172.21.15.86:0/3661103479 conn(0x7f40f80cffc0 msgr2=0x7f40f80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.664+0000 7f40fda00640 1 -- 172.21.15.86:0/3661103479 shutdown_connections 2024-04-25T14:05:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.664+0000 7f40fda00640 1 -- 172.21.15.86:0/3661103479 wait complete. 2024-04-25T14:05:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.664+0000 7f40fda00640 1 Processor -- start 2024-04-25T14:05:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.664+0000 7f40fda00640 1 -- start start 2024-04-25T14:05:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.668+0000 7f40fda00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40f806e7a0 0x7f40f81580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.668+0000 7f40fda00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40f80cf970 0x7f40f8158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.668+0000 7f40fda00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40f80b8480 0x7f40f80b8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.668+0000 7f40fda00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f40f816cbd0 con 0x7f40f806e7a0 2024-04-25T14:05:38.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.668+0000 7f40fda00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f40f80ced80 con 0x7f40f80cf970 2024-04-25T14:05:38.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.668+0000 7f40fda00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f40f816cd50 con 0x7f40f80b8480 2024-04-25T14:05:38.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.668+0000 7f40f6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40f806e7a0 0x7f40f81580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.668+0000 7f40f6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40f806e7a0 0x7f40f81580f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52190/0 (socket says 172.21.15.86:52190) 2024-04-25T14:05:38.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.668+0000 7f40f6a00640 1 -- 172.21.15.86:0/1132783718 learned_addr learned my addr 172.21.15.86:0/1132783718 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:38.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.668+0000 7f40f6000640 1 --2- 172.21.15.86:0/1132783718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40f80cf970 0x7f40f8158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.668+0000 7f40f7400640 1 --2- 172.21.15.86:0/1132783718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40f80b8480 0x7f40f80b8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.668+0000 7f40f6a00640 1 -- 172.21.15.86:0/1132783718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40f80b8480 msgr2=0x7f40f80b8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:38.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.668+0000 7f40f6a00640 1 --2- 172.21.15.86:0/1132783718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40f80b8480 0x7f40f80b8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.668+0000 7f40f6a00640 1 -- 172.21.15.86:0/1132783718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40f80cf970 msgr2=0x7f40f8158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.668+0000 7f40f6a00640 1 --2- 172.21.15.86:0/1132783718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40f80cf970 0x7f40f8158770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.668+0000 7f40f6a00640 1 -- 172.21.15.86:0/1132783718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f80b8dd0 con 0x7f40f806e7a0 2024-04-25T14:05:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.668+0000 7f40f6000640 1 --2- 172.21.15.86:0/1132783718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40f80cf970 0x7f40f8158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.668+0000 7f40f6a00640 1 --2- 172.21.15.86:0/1132783718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40f806e7a0 0x7f40f81580f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f40e800c4b0 tx=0x7f40e800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.668+0000 7f40eea00640 1 -- 172.21.15.86:0/1132783718 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e8018070 con 0x7f40f806e7a0 2024-04-25T14:05:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.668+0000 7f40eea00640 1 -- 172.21.15.86:0/1132783718 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e8007500 con 0x7f40f806e7a0 2024-04-25T14:05:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.668+0000 7f40fda00640 1 -- 172.21.15.86:0/1132783718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f8161530 con 0x7f40f806e7a0 2024-04-25T14:05:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.668+0000 7f40eea00640 1 -- 172.21.15.86:0/1132783718 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e8004a30 con 0x7f40f806e7a0 2024-04-25T14:05:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.668+0000 7f40fda00640 1 -- 172.21.15.86:0/1132783718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f8161af0 con 0x7f40f806e7a0 2024-04-25T14:05:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.668+0000 7f40fda00640 1 -- 172.21.15.86:0/1132783718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40f80720e0 con 0x7f40f806e7a0 2024-04-25T14:05:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.672+0000 7f40eea00640 1 -- 172.21.15.86:0/1132783718 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f40e801b070 con 0x7f40f806e7a0 2024-04-25T14:05:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.672+0000 7f40eea00640 1 --2- 172.21.15.86:0/1132783718 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f40d806f570 0x7f40d8071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.672+0000 7f40eea00640 1 -- 172.21.15.86:0/1132783718 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6433+0+0 (secure 0 0 0) 0x7f40e808eb00 con 0x7f40f806e7a0 2024-04-25T14:05:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.672+0000 7f40f6000640 1 --2- 172.21.15.86:0/1132783718 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f40d806f570 0x7f40d8071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.672+0000 7f40f6000640 1 --2- 172.21.15.86:0/1132783718 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f40d806f570 0x7f40d8071a80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f40e40af000 tx=0x7f40e40a7000 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:38.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.676+0000 7f40eea00640 1 -- 172.21.15.86:0/1132783718 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f40f80b8dd0 con 0x7f40f806e7a0 2024-04-25T14:05:38.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.888+0000 7f40fda00640 1 -- 172.21.15.86:0/1132783718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f40f8163500 con 0x7f40f806e7a0 2024-04-25T14:05:38.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.888+0000 7f40eea00640 1 -- 172.21.15.86:0/1132783718 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+3376 (secure 0 0 0) 0x7f40e805b560 con 0x7f40f806e7a0 2024-04-25T14:05:38.889 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:38.889 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":133,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":8,"num_up_osds":8,"osd_up_since":1714053818,"num_in_osds":8,"osd_in_since":1714053808,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":56},{"state_name":"creating+peering","count":24},{"state_name":"active+clean","count":17}],"num_pgs":97,"num_pools":3,"num_objects":7,"data_bytes":591020,"bytes_used":226697216,"bytes_avail":772867416064,"bytes_total":773094113280,"unknown_pgs_ratio":0.57731956243515015,"inactive_pgs_ratio":0.24742268025875092,"write_bytes_sec":344,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":26,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5712}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful","stats"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-04-25T14:05:25.480013+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T14:05:38.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.888+0000 7f40fda00640 1 -- 172.21.15.86:0/1132783718 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f40d806f570 msgr2=0x7f40d8071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:38.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.888+0000 7f40fda00640 1 --2- 172.21.15.86:0/1132783718 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f40d806f570 0x7f40d8071a80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f40e40af000 tx=0x7f40e40a7000 comp rx=0 tx=0).stop 2024-04-25T14:05:38.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.888+0000 7f40fda00640 1 -- 172.21.15.86:0/1132783718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40f806e7a0 msgr2=0x7f40f81580f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:38.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.888+0000 7f40fda00640 1 --2- 172.21.15.86:0/1132783718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40f806e7a0 0x7f40f81580f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f40e800c4b0 tx=0x7f40e800c970 comp rx=0 tx=0).stop 2024-04-25T14:05:38.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.888+0000 7f40fda00640 1 -- 172.21.15.86:0/1132783718 shutdown_connections 2024-04-25T14:05:38.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.888+0000 7f40fda00640 1 --2- 172.21.15.86:0/1132783718 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f40d806f570 0x7f40d8071a80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.888+0000 7f40fda00640 1 --2- 172.21.15.86:0/1132783718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40f80b8480 0x7f40f80b8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.888+0000 7f40fda00640 1 --2- 172.21.15.86:0/1132783718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40f80cf970 0x7f40f8158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.888+0000 7f40fda00640 1 --2- 172.21.15.86:0/1132783718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40f806e7a0 0x7f40f81580f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.888+0000 7f40fda00640 1 -- 172.21.15.86:0/1132783718 >> 172.21.15.86:0/1132783718 conn(0x7f40f80cffc0 msgr2=0x7f40f81593f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:38.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.888+0000 7f40fda00640 1 -- 172.21.15.86:0/1132783718 shutdown_connections 2024-04-25T14:05:38.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:38.888+0000 7f40fda00640 1 -- 172.21.15.86:0/1132783718 wait complete. 2024-04-25T14:05:38.911 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T14:05:38.911 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:05:39.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.056+0000 7fe4d6800640 1 Processor -- start 2024-04-25T14:05:39.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.056+0000 7fe4d6800640 1 -- start start 2024-04-25T14:05:39.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.056+0000 7fe4d6800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4d00b39d0 0x7fe4d00d2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:39.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.056+0000 7fe4d6800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe4d00d2ba0 0x7fe4d006de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:39.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.056+0000 7fe4d6800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4d00b2560 con 0x7fe4d00d2790 2024-04-25T14:05:39.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.056+0000 7fe4d6800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe4d00d33d0 con 0x7fe4d00d2ba0 2024-04-25T14:05:39.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.056+0000 7fe4d6800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d00b3de0 con 0x7fe4d00b39d0 2024-04-25T14:05:39.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.056+0000 7fe4cf400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4d00b39d0 0x7fe4d00d2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:39.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.056+0000 7fe4cfe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe4d00d2ba0 0x7fe4d006de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:39.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.056+0000 7fe4cfe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe4d00d2ba0 0x7fe4d006de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:55234/0 (socket says 172.21.15.86:55234) 2024-04-25T14:05:39.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.056+0000 7fe4cfe00640 1 -- 172.21.15.86:0/1175087298 learned_addr learned my addr 172.21.15.86:0/1175087298 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:39.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.056+0000 7fe4cf400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4d00b39d0 0x7fe4d00d2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:35158/0 (socket says 172.21.15.86:35158) 2024-04-25T14:05:39.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.056+0000 7fe4cea00640 1 --1- 172.21.15.86:0/1175087298 >> v1:172.21.15.155:6789/0 conn(0x7fe4d00d2790 0x7fe4d006d780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:33582/0 (socket says 172.21.15.86:33582) 2024-04-25T14:05:39.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.056+0000 7fe4cfe00640 1 -- 172.21.15.86:0/1175087298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4d00b39d0 msgr2=0x7fe4d00d2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:39.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.056+0000 7fe4cfe00640 1 --2- 172.21.15.86:0/1175087298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4d00b39d0 0x7fe4d00d2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.056+0000 7fe4cfe00640 1 -- 172.21.15.86:0/1175087298 >> v1:172.21.15.155:6789/0 conn(0x7fe4d00d2790 legacy=0x7fe4d006d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:39.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.056+0000 7fe4cfe00640 1 -- 172.21.15.86:0/1175087298 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d00b1fe0 con 0x7fe4d00d2ba0 2024-04-25T14:05:39.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.056+0000 7fe4ce000640 1 -- 172.21.15.86:0/1175087298 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1953241055 0 0) 0x7fe4d00b2560 con 0x7fe4d00d2790 2024-04-25T14:05:39.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.060+0000 7fe4cfe00640 1 --2- 172.21.15.86:0/1175087298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4d00d2ba0 0x7fe4d006de90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009e50 tx=0x7fe4c801f210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66a10b1d133ebee6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:39.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.060+0000 7fe4ce000640 1 -- 172.21.15.86:0/1175087298 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80b4070 con 0x7fe4d00d2ba0 2024-04-25T14:05:39.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.060+0000 7fe4ce000640 1 -- 172.21.15.86:0/1175087298 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c80af400 con 0x7fe4d00d2ba0 2024-04-25T14:05:39.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.060+0000 7fe4ce000640 1 -- 172.21.15.86:0/1175087298 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80af7d0 con 0x7fe4d00d2ba0 2024-04-25T14:05:39.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.060+0000 7fe4d6800640 1 -- 172.21.15.86:0/1175087298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4d00d2ba0 msgr2=0x7fe4d006de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:39.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.060+0000 7fe4d6800640 1 --2- 172.21.15.86:0/1175087298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4d00d2ba0 0x7fe4d006de90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009e50 tx=0x7fe4c801f210 comp rx=0 tx=0).stop 2024-04-25T14:05:39.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.060+0000 7fe4d6800640 1 -- 172.21.15.86:0/1175087298 shutdown_connections 2024-04-25T14:05:39.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.060+0000 7fe4d6800640 1 --2- 172.21.15.86:0/1175087298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4d00d2ba0 0x7fe4d006de90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.060+0000 7fe4d6800640 1 --2- 172.21.15.86:0/1175087298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4d00b39d0 0x7fe4d00d2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.060+0000 7fe4d6800640 1 -- 172.21.15.86:0/1175087298 >> 172.21.15.86:0/1175087298 conn(0x7fe4d009ba30 msgr2=0x7fe4d00b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:39.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.064+0000 7fe4d6800640 1 -- 172.21.15.86:0/1175087298 shutdown_connections 2024-04-25T14:05:39.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.064+0000 7fe4d6800640 1 -- 172.21.15.86:0/1175087298 wait complete. 2024-04-25T14:05:39.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.064+0000 7fe4d6800640 1 Processor -- start 2024-04-25T14:05:39.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.064+0000 7fe4d6800640 1 -- start start 2024-04-25T14:05:39.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.064+0000 7fe4d6800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4d00b39d0 0x7fe4d0156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:39.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.064+0000 7fe4d6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4d00d2790 0x7fe4d00c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:39.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.064+0000 7fe4d6800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4d00d2ba0 0x7fe4d01586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:39.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.064+0000 7fe4d6800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d015cea0 con 0x7fe4d00d2790 2024-04-25T14:05:39.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.064+0000 7fe4d6800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d00b2280 con 0x7fe4d00d2ba0 2024-04-25T14:05:39.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.064+0000 7fe4d6800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d015d020 con 0x7fe4d00b39d0 2024-04-25T14:05:39.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.068+0000 7fe4cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4d00d2790 0x7fe4d00c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:39.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.068+0000 7fe4cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4d00d2790 0x7fe4d00c9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47686/0 (socket says 172.21.15.86:47686) 2024-04-25T14:05:39.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.068+0000 7fe4cea00640 1 -- 172.21.15.86:0/2460511735 learned_addr learned my addr 172.21.15.86:0/2460511735 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:39.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.068+0000 7fe4cf400640 1 --2- 172.21.15.86:0/2460511735 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4d00b39d0 0x7fe4d0156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:39.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.068+0000 7fe4cfe00640 1 --2- 172.21.15.86:0/2460511735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4d00d2ba0 0x7fe4d01586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:39.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.068+0000 7fe4cea00640 1 -- 172.21.15.86:0/2460511735 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4d00b39d0 msgr2=0x7fe4d0156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:39.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.068+0000 7fe4cea00640 1 --2- 172.21.15.86:0/2460511735 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4d00b39d0 0x7fe4d0156c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.068+0000 7fe4cea00640 1 -- 172.21.15.86:0/2460511735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4d00d2ba0 msgr2=0x7fe4d01586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:39.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.068+0000 7fe4cea00640 1 --2- 172.21.15.86:0/2460511735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4d00d2ba0 0x7fe4d01586c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.068+0000 7fe4cea00640 1 -- 172.21.15.86:0/2460511735 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d015d4b0 con 0x7fe4d00d2790 2024-04-25T14:05:39.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.068+0000 7fe4cea00640 1 --2- 172.21.15.86:0/2460511735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4d00d2790 0x7fe4d00c9ee0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00c4b0 tx=0x7fe4bc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:39.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.068+0000 7fe4c7400640 1 -- 172.21.15.86:0/2460511735 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc018070 con 0x7fe4d00d2790 2024-04-25T14:05:39.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.068+0000 7fe4c7400640 1 -- 172.21.15.86:0/2460511735 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4bc007500 con 0x7fe4d00d2790 2024-04-25T14:05:39.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.068+0000 7fe4d6800640 1 -- 172.21.15.86:0/2460511735 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d015d790 con 0x7fe4d00d2790 2024-04-25T14:05:39.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.068+0000 7fe4c7400640 1 -- 172.21.15.86:0/2460511735 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc004a30 con 0x7fe4d00d2790 2024-04-25T14:05:39.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.068+0000 7fe4d6800640 1 -- 172.21.15.86:0/2460511735 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d015dd50 con 0x7fe4d00d2790 2024-04-25T14:05:39.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.068+0000 7fe4d6800640 1 -- 172.21.15.86:0/2460511735 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4d00a0630 con 0x7fe4d00d2790 2024-04-25T14:05:39.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.068+0000 7fe4c7400640 1 -- 172.21.15.86:0/2460511735 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7fe4bc004bd0 con 0x7fe4d00d2790 2024-04-25T14:05:39.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.072+0000 7fe4c7400640 1 --2- 172.21.15.86:0/2460511735 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fe4b006f570 0x7fe4b0071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:39.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.072+0000 7fe4c7400640 1 -- 172.21.15.86:0/2460511735 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6433+0+0 (secure 0 0 0) 0x7fe4bc08e870 con 0x7fe4d00d2790 2024-04-25T14:05:39.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.072+0000 7fe4cf400640 1 --2- 172.21.15.86:0/2460511735 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fe4b006f570 0x7fe4b0071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:39.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.072+0000 7fe4cf400640 1 --2- 172.21.15.86:0/2460511735 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fe4b006f570 0x7fe4b0071a80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009ac0 tx=0x7fe4c0009190 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:39.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.076+0000 7fe4c7400640 1 -- 172.21.15.86:0/2460511735 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe4bc05adc0 con 0x7fe4d00d2790 2024-04-25T14:05:39.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.228+0000 7fe4d6800640 1 -- 172.21.15.86:0/2460511735 --> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe4d015d240 con 0x7fe4b006f570 2024-04-25T14:05:39.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.236+0000 7fe4c7400640 1 -- 172.21.15.86:0/2460511735 <== mgr.5532 v2:172.21.15.86:6832/2427179681 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242125 (secure 0 0 0) 0x7fe4d015d240 con 0x7fe4b006f570 2024-04-25T14:05:39.239 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:39.240 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:05:39.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.240+0000 7fe4d6800640 1 -- 172.21.15.86:0/2460511735 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fe4b006f570 msgr2=0x7fe4b0071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:39.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.240+0000 7fe4d6800640 1 --2- 172.21.15.86:0/2460511735 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fe4b006f570 0x7fe4b0071a80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009ac0 tx=0x7fe4c0009190 comp rx=0 tx=0).stop 2024-04-25T14:05:39.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.240+0000 7fe4d6800640 1 -- 172.21.15.86:0/2460511735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4d00d2790 msgr2=0x7fe4d00c9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:39.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.240+0000 7fe4d6800640 1 --2- 172.21.15.86:0/2460511735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4d00d2790 0x7fe4d00c9ee0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00c4b0 tx=0x7fe4bc00c970 comp rx=0 tx=0).stop 2024-04-25T14:05:39.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.240+0000 7fe4d6800640 1 -- 172.21.15.86:0/2460511735 shutdown_connections 2024-04-25T14:05:39.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.240+0000 7fe4d6800640 1 --2- 172.21.15.86:0/2460511735 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fe4b006f570 0x7fe4b0071a80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.240+0000 7fe4d6800640 1 --2- 172.21.15.86:0/2460511735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4d00d2ba0 0x7fe4d01586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.240+0000 7fe4d6800640 1 --2- 172.21.15.86:0/2460511735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4d00d2790 0x7fe4d00c9ee0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.240+0000 7fe4d6800640 1 --2- 172.21.15.86:0/2460511735 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4d00b39d0 0x7fe4d0156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.240+0000 7fe4d6800640 1 -- 172.21.15.86:0/2460511735 >> 172.21.15.86:0/2460511735 conn(0x7fe4d009ba30 msgr2=0x7fe4d006dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:39.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.240+0000 7fe4d6800640 1 -- 172.21.15.86:0/2460511735 shutdown_connections 2024-04-25T14:05:39.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:39.240+0000 7fe4d6800640 1 -- 172.21.15.86:0/2460511735 wait complete. 2024-04-25T14:05:39.260 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-04-25T14:05:37.482712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591020,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":124,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88,"ondisk_log_size":88,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221384,"kb_used_data":5208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753336,"statfs":{"total":773094113280,"available":772867416064,"internally_reserved":0,"allocated":5332992,"data_stored":3003413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":652,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.952497"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.040189+0000","last_change":"2024-04-25T14:05:33.573200+0000","last_active":"2024-04-25T14:05:35.040189+0000","last_peered":"2024-04-25T14:05:35.040189+0000","last_clean":"2024-04-25T14:05:35.040189+0000","last_became_active":"2024-04-25T14:05:33.572858+0000","last_became_peered":"2024-04-25T14:05:33.572858+0000","last_unstale":"2024-04-25T14:05:35.040189+0000","last_undegraded":"2024-04-25T14:05:35.040189+0000","last_fullsized":"2024-04-25T14:05:35.040189+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:22.996486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039514+0000","last_change":"2024-04-25T14:05:33.571072+0000","last_active":"2024-04-25T14:05:35.039514+0000","last_peered":"2024-04-25T14:05:35.039514+0000","last_clean":"2024-04-25T14:05:35.039514+0000","last_became_active":"2024-04-25T14:05:33.570748+0000","last_became_peered":"2024-04-25T14:05:33.570748+0000","last_unstale":"2024-04-25T14:05:35.039514+0000","last_undegraded":"2024-04-25T14:05:35.039514+0000","last_fullsized":"2024-04-25T14:05:35.039514+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:18.074490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039301+0000","last_change":"2024-04-25T14:05:33.571248+0000","last_active":"2024-04-25T14:05:35.039301+0000","last_peered":"2024-04-25T14:05:35.039301+0000","last_clean":"2024-04-25T14:05:35.039301+0000","last_became_active":"2024-04-25T14:05:33.570916+0000","last_became_peered":"2024-04-25T14:05:33.570916+0000","last_unstale":"2024-04-25T14:05:35.039301+0000","last_undegraded":"2024-04-25T14:05:35.039301+0000","last_fullsized":"2024-04-25T14:05:35.039301+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:36.296676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039600+0000","last_change":"2024-04-25T14:05:33.576117+0000","last_active":"2024-04-25T14:05:35.039600+0000","last_peered":"2024-04-25T14:05:35.039600+0000","last_clean":"2024-04-25T14:05:35.039600+0000","last_became_active":"2024-04-25T14:05:33.575838+0000","last_became_peered":"2024-04-25T14:05:33.575838+0000","last_unstale":"2024-04-25T14:05:35.039600+0000","last_undegraded":"2024-04-25T14:05:35.039600+0000","last_fullsized":"2024-04-25T14:05:35.039600+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:20.619851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039689+0000","last_change":"2024-04-25T14:05:33.596454+0000","last_active":"2024-04-25T14:05:35.039689+0000","last_peered":"2024-04-25T14:05:35.039689+0000","last_clean":"2024-04-25T14:05:35.039689+0000","last_became_active":"2024-04-25T14:05:33.596225+0000","last_became_peered":"2024-04-25T14:05:33.596225+0000","last_unstale":"2024-04-25T14:05:35.039689+0000","last_undegraded":"2024-04-25T14:05:35.039689+0000","last_fullsized":"2024-04-25T14:05:35.039689+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:41.512359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"45'4","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039294+0000","last_change":"2024-04-25T14:05:32.556304+0000","last_active":"2024-04-25T14:05:35.039294+0000","last_peered":"2024-04-25T14:05:35.039294+0000","last_clean":"2024-04-25T14:05:35.039294+0000","last_became_active":"2024-04-25T14:05:32.556052+0000","last_became_peered":"2024-04-25T14:05:32.556052+0000","last_unstale":"2024-04-25T14:05:35.039294+0000","last_undegraded":"2024-04-25T14:05:35.039294+0000","last_fullsized":"2024-04-25T14:05:35.039294+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:16.452558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.532639+0000","last_change":"2024-04-25T14:05:31.532639+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.532639+0000","last_undegraded":"2024-04-25T14:05:31.532639+0000","last_fullsized":"2024-04-25T14:05:31.532639+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039348+0000","last_change":"2024-04-25T14:05:33.574504+0000","last_active":"2024-04-25T14:05:35.039348+0000","last_peered":"2024-04-25T14:05:35.039348+0000","last_clean":"2024-04-25T14:05:35.039348+0000","last_became_active":"2024-04-25T14:05:33.574191+0000","last_became_peered":"2024-04-25T14:05:33.574191+0000","last_unstale":"2024-04-25T14:05:35.039348+0000","last_undegraded":"2024-04-25T14:05:35.039348+0000","last_fullsized":"2024-04-25T14:05:35.039348+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:28.629474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.533022+0000","last_change":"2024-04-25T14:05:31.533022+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.533022+0000","last_undegraded":"2024-04-25T14:05:31.533022+0000","last_fullsized":"2024-04-25T14:05:31.533022+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.533643+0000","last_change":"2024-04-25T14:05:31.533643+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.533643+0000","last_undegraded":"2024-04-25T14:05:31.533643+0000","last_fullsized":"2024-04-25T14:05:31.533643+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.540931+0000","last_change":"2024-04-25T14:05:31.540931+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.540931+0000","last_undegraded":"2024-04-25T14:05:31.540931+0000","last_fullsized":"2024-04-25T14:05:31.540931+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.541762+0000","last_change":"2024-04-25T14:05:31.541762+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.541762+0000","last_undegraded":"2024-04-25T14:05:31.541762+0000","last_fullsized":"2024-04-25T14:05:31.541762+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039072+0000","last_change":"2024-04-25T14:05:32.551806+0000","last_active":"2024-04-25T14:05:35.039072+0000","last_peered":"2024-04-25T14:05:35.039072+0000","last_clean":"2024-04-25T14:05:35.039072+0000","last_became_active":"2024-04-25T14:05:32.551441+0000","last_became_peered":"2024-04-25T14:05:32.551441+0000","last_unstale":"2024-04-25T14:05:35.039072+0000","last_undegraded":"2024-04-25T14:05:35.039072+0000","last_fullsized":"2024-04-25T14:05:35.039072+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:31.414048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.538948+0000","last_change":"2024-04-25T14:05:31.538948+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.538948+0000","last_undegraded":"2024-04-25T14:05:31.538948+0000","last_fullsized":"2024-04-25T14:05:31.538948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.539519+0000","last_change":"2024-04-25T14:05:31.539519+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.539519+0000","last_undegraded":"2024-04-25T14:05:31.539519+0000","last_fullsized":"2024-04-25T14:05:31.539519+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.542126+0000","last_change":"2024-04-25T14:05:31.542126+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.542126+0000","last_undegraded":"2024-04-25T14:05:31.542126+0000","last_fullsized":"2024-04-25T14:05:31.542126+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.540063+0000","last_change":"2024-04-25T14:05:31.540063+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.540063+0000","last_undegraded":"2024-04-25T14:05:31.540063+0000","last_fullsized":"2024-04-25T14:05:31.540063+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.535803+0000","last_change":"2024-04-25T14:05:31.535803+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.535803+0000","last_undegraded":"2024-04-25T14:05:31.535803+0000","last_fullsized":"2024-04-25T14:05:31.535803+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.540372+0000","last_change":"2024-04-25T14:05:31.540372+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.540372+0000","last_undegraded":"2024-04-25T14:05:31.540372+0000","last_fullsized":"2024-04-25T14:05:31.540372+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039235+0000","last_change":"2024-04-25T14:05:32.562133+0000","last_active":"2024-04-25T14:05:35.039235+0000","last_peered":"2024-04-25T14:05:35.039235+0000","last_clean":"2024-04-25T14:05:35.039235+0000","last_became_active":"2024-04-25T14:05:32.561845+0000","last_became_peered":"2024-04-25T14:05:32.561845+0000","last_unstale":"2024-04-25T14:05:35.039235+0000","last_undegraded":"2024-04-25T14:05:35.039235+0000","last_fullsized":"2024-04-25T14:05:35.039235+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:14.012822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.539420+0000","last_change":"2024-04-25T14:05:31.539420+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.539420+0000","last_undegraded":"2024-04-25T14:05:31.539420+0000","last_fullsized":"2024-04-25T14:05:31.539420+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.535717+0000","last_change":"2024-04-25T14:05:31.535717+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.535717+0000","last_undegraded":"2024-04-25T14:05:31.535717+0000","last_fullsized":"2024-04-25T14:05:31.535717+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.539606+0000","last_change":"2024-04-25T14:05:31.539606+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.539606+0000","last_undegraded":"2024-04-25T14:05:31.539606+0000","last_fullsized":"2024-04-25T14:05:31.539606+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.040239+0000","last_change":"2024-04-25T14:05:32.559217+0000","last_active":"2024-04-25T14:05:35.040239+0000","last_peered":"2024-04-25T14:05:35.040239+0000","last_clean":"2024-04-25T14:05:35.040239+0000","last_became_active":"2024-04-25T14:05:32.558909+0000","last_became_peered":"2024-04-25T14:05:32.558909+0000","last_unstale":"2024-04-25T14:05:35.040239+0000","last_undegraded":"2024-04-25T14:05:35.040239+0000","last_fullsized":"2024-04-25T14:05:35.040239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:48:44.213850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039597+0000","last_change":"2024-04-25T14:05:32.566219+0000","last_active":"2024-04-25T14:05:35.039597+0000","last_peered":"2024-04-25T14:05:35.039597+0000","last_clean":"2024-04-25T14:05:35.039597+0000","last_became_active":"2024-04-25T14:05:32.565859+0000","last_became_peered":"2024-04-25T14:05:32.565859+0000","last_unstale":"2024-04-25T14:05:35.039597+0000","last_undegraded":"2024-04-25T14:05:35.039597+0000","last_fullsized":"2024-04-25T14:05:35.039597+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:35.649467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039730+0000","last_change":"2024-04-25T14:05:32.567474+0000","last_active":"2024-04-25T14:05:35.039730+0000","last_peered":"2024-04-25T14:05:35.039730+0000","last_clean":"2024-04-25T14:05:35.039730+0000","last_became_active":"2024-04-25T14:05:32.567061+0000","last_became_peered":"2024-04-25T14:05:32.567061+0000","last_unstale":"2024-04-25T14:05:35.039730+0000","last_undegraded":"2024-04-25T14:05:35.039730+0000","last_fullsized":"2024-04-25T14:05:35.039730+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:20.281168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.532537+0000","last_change":"2024-04-25T14:05:31.532537+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.532537+0000","last_undegraded":"2024-04-25T14:05:31.532537+0000","last_fullsized":"2024-04-25T14:05:31.532537+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039542+0000","last_change":"2024-04-25T14:05:33.570587+0000","last_active":"2024-04-25T14:05:35.039542+0000","last_peered":"2024-04-25T14:05:35.039542+0000","last_clean":"2024-04-25T14:05:35.039542+0000","last_became_active":"2024-04-25T14:05:33.570209+0000","last_became_peered":"2024-04-25T14:05:33.570209+0000","last_unstale":"2024-04-25T14:05:35.039542+0000","last_undegraded":"2024-04-25T14:05:35.039542+0000","last_fullsized":"2024-04-25T14:05:35.039542+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:13.979622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.535614+0000","last_change":"2024-04-25T14:05:31.535614+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.535614+0000","last_undegraded":"2024-04-25T14:05:31.535614+0000","last_fullsized":"2024-04-25T14:05:31.535614+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.541074+0000","last_change":"2024-04-25T14:05:31.541074+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.541074+0000","last_undegraded":"2024-04-25T14:05:31.541074+0000","last_fullsized":"2024-04-25T14:05:31.541074+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039205+0000","last_change":"2024-04-25T14:05:32.553201+0000","last_active":"2024-04-25T14:05:35.039205+0000","last_peered":"2024-04-25T14:05:35.039205+0000","last_clean":"2024-04-25T14:05:35.039205+0000","last_became_active":"2024-04-25T14:05:32.552846+0000","last_became_peered":"2024-04-25T14:05:32.552846+0000","last_unstale":"2024-04-25T14:05:35.039205+0000","last_undegraded":"2024-04-25T14:05:35.039205+0000","last_fullsized":"2024-04-25T14:05:35.039205+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:13.772830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039401+0000","last_change":"2024-04-25T14:05:32.561754+0000","last_active":"2024-04-25T14:05:35.039401+0000","last_peered":"2024-04-25T14:05:35.039401+0000","last_clean":"2024-04-25T14:05:35.039401+0000","last_became_active":"2024-04-25T14:05:32.561297+0000","last_became_peered":"2024-04-25T14:05:32.561297+0000","last_unstale":"2024-04-25T14:05:35.039401+0000","last_undegraded":"2024-04-25T14:05:35.039401+0000","last_fullsized":"2024-04-25T14:05:35.039401+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:39.285483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.539462+0000","last_change":"2024-04-25T14:05:31.539462+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.539462+0000","last_undegraded":"2024-04-25T14:05:31.539462+0000","last_fullsized":"2024-04-25T14:05:31.539462+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.539025+0000","last_change":"2024-04-25T14:05:31.539025+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.539025+0000","last_undegraded":"2024-04-25T14:05:31.539025+0000","last_fullsized":"2024-04-25T14:05:31.539025+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.535050+0000","last_change":"2024-04-25T14:05:31.535050+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.535050+0000","last_undegraded":"2024-04-25T14:05:31.535050+0000","last_fullsized":"2024-04-25T14:05:31.535050+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.535108+0000","last_change":"2024-04-25T14:05:31.535108+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.535108+0000","last_undegraded":"2024-04-25T14:05:31.535108+0000","last_fullsized":"2024-04-25T14:05:31.535108+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.541206+0000","last_change":"2024-04-25T14:05:31.541206+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.541206+0000","last_undegraded":"2024-04-25T14:05:31.541206+0000","last_fullsized":"2024-04-25T14:05:31.541206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:32.529615+0000","last_change":"2024-04-25T14:05:32.529615+0000","last_active":"2024-04-25T14:05:32.529615+0000","last_peered":"2024-04-25T14:05:32.529615+0000","last_clean":"2024-04-25T14:05:32.529615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:32.529615+0000","last_undegraded":"2024-04-25T14:05:32.529615+0000","last_fullsized":"2024-04-25T14:05:32.529615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.533456+0000","last_change":"2024-04-25T14:05:31.533456+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.533456+0000","last_undegraded":"2024-04-25T14:05:31.533456+0000","last_fullsized":"2024-04-25T14:05:31.533456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039142+0000","last_change":"2024-04-25T14:05:33.563768+0000","last_active":"2024-04-25T14:05:35.039142+0000","last_peered":"2024-04-25T14:05:35.039142+0000","last_clean":"2024-04-25T14:05:35.039142+0000","last_became_active":"2024-04-25T14:05:33.563379+0000","last_became_peered":"2024-04-25T14:05:33.563379+0000","last_unstale":"2024-04-25T14:05:35.039142+0000","last_undegraded":"2024-04-25T14:05:35.039142+0000","last_fullsized":"2024-04-25T14:05:35.039142+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:11.087435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-25T14:05:31.538960+0000","last_change":"2024-04-25T14:05:31.538960+0000","last_active":"2024-04-25T14:05:31.523192+0000","last_peered":"2024-04-25T14:05:31.523192+0000","last_clean":"2024-04-25T14:05:31.523192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:31.538960+0000","last_undegraded":"2024-04-25T14:05:31.538960+0000","last_fullsized":"2024-04-25T14:05:31.538960+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"39'79","reported_seq":177,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T14:05:31.540738+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:05:31.540738+0000","last_peered":"2024-04-25T14:05:31.540738+0000","last_clean":"2024-04-25T14:05:31.540738+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:05:31.540738+0000","last_undegraded":"2024-04-25T14:05:31.540738+0000","last_fullsized":"2024-04-25T14:05:31.540738+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":2},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":652,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1089536,"data_stored":806812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":552960,"data_stored":245159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":222991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":222991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":222991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":246119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":222991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343740,"statfs":{"total":96636764160,"available":96607989760,"internally_reserved":0,"allocated":1101824,"data_stored":813359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:05:42.265 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:05:42.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.424+0000 7f11bec00640 1 Processor -- start 2024-04-25T14:05:42.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.424+0000 7f11bec00640 1 -- start start 2024-04-25T14:05:42.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.424+0000 7f11bec00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f11b8070330 0x7f11b80d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:42.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.424+0000 7f11bec00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11b80d1c20 0x7f11b806ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:42.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.424+0000 7f11bec00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11b80d2030 con 0x7f11b806f270 2024-04-25T14:05:42.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.424+0000 7f11bec00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f11b815e5c0 con 0x7f11b8070330 2024-04-25T14:05:42.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.424+0000 7f11bec00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f11b815e8a0 con 0x7f11b80d1c20 2024-04-25T14:05:42.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.424+0000 7f11bcc00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f11b806f270 0x7f11b806f680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41244/0 (socket says 172.21.15.86:41244) 2024-04-25T14:05:42.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.424+0000 7f11bcc00640 1 -- 172.21.15.86:0/497674419 learned_addr learned my addr 172.21.15.86:0/497674419 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:42.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.424+0000 7f11b7e00640 1 --2- 172.21.15.86:0/497674419 >> v2:172.21.15.155:3300/0 conn(0x7f11b8070330 0x7f11b80d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:42.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.424+0000 7f11afe00640 1 --2- 172.21.15.86:0/497674419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11b80d1c20 0x7f11b806ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:42.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.424+0000 7f11b7400640 1 -- 172.21.15.86:0/497674419 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 620943854 0 0) 0x7f11b00014a0 con 0x7f11b806f270 2024-04-25T14:05:42.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.424+0000 7f11b7400640 1 -- 172.21.15.86:0/497674419 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f119c003530 con 0x7f11b806f270 2024-04-25T14:05:42.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.424+0000 7f11b7e00640 1 -- 172.21.15.86:0/497674419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11b80d1c20 msgr2=0x7f11b806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:42.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.424+0000 7f11b7e00640 1 --2- 172.21.15.86:0/497674419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11b80d1c20 0x7f11b806ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:42.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.424+0000 7f11b7e00640 1 -- 172.21.15.86:0/497674419 >> v1:172.21.15.86:6789/0 conn(0x7f11b806f270 legacy=0x7f11b806f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:42.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.424+0000 7f11b7400640 1 -- 172.21.15.86:0/497674419 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2249072729 0 0) 0x7f119c003530 con 0x7f11b806f270 2024-04-25T14:05:42.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.424+0000 7f11b7e00640 1 -- 172.21.15.86:0/497674419 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b815eb80 con 0x7f11b8070330 2024-04-25T14:05:42.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.424+0000 7f11afe00640 1 --2- 172.21.15.86:0/497674419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11b80d1c20 0x7f11b806ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:42.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.428+0000 7f11b7e00640 1 --2- 172.21.15.86:0/497674419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11b8070330 0x7f11b80d16e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f11a80097c0 tx=0x7f11a801c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98d95fe36ae98a97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:42.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.428+0000 7f11b7400640 1 -- 172.21.15.86:0/497674419 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a80b0070 con 0x7f11b8070330 2024-04-25T14:05:42.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.428+0000 7f11b7400640 1 -- 172.21.15.86:0/497674419 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11a80ab400 con 0x7f11b8070330 2024-04-25T14:05:42.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.428+0000 7f11b7400640 1 -- 172.21.15.86:0/497674419 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b0001bd0 con 0x7f11b8070330 2024-04-25T14:05:42.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.428+0000 7f11bec00640 1 -- 172.21.15.86:0/497674419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11b8070330 msgr2=0x7f11b80d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:42.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.428+0000 7f11bec00640 1 --2- 172.21.15.86:0/497674419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11b8070330 0x7f11b80d16e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f11a80097c0 tx=0x7f11a801c330 comp rx=0 tx=0).stop 2024-04-25T14:05:42.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.428+0000 7f11bec00640 1 -- 172.21.15.86:0/497674419 shutdown_connections 2024-04-25T14:05:42.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.428+0000 7f11bec00640 1 --2- 172.21.15.86:0/497674419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11b80d1c20 0x7f11b806ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:42.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.428+0000 7f11bec00640 1 --2- 172.21.15.86:0/497674419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11b8070330 0x7f11b80d16e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:42.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.428+0000 7f11bec00640 1 -- 172.21.15.86:0/497674419 >> 172.21.15.86:0/497674419 conn(0x7f11b80b4650 msgr2=0x7f11b80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:42.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.428+0000 7f11bec00640 1 -- 172.21.15.86:0/497674419 shutdown_connections 2024-04-25T14:05:42.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.428+0000 7f11bec00640 1 -- 172.21.15.86:0/497674419 wait complete. 2024-04-25T14:05:42.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.432+0000 7f11bec00640 1 Processor -- start 2024-04-25T14:05:42.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.432+0000 7f11bec00640 1 -- start start 2024-04-25T14:05:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.432+0000 7f11bec00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11b806f270 0x7f11b80cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.432+0000 7f11bec00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11b8070330 0x7f11b80b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.432+0000 7f11bec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11b80d1c20 0x7f11b8156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.432+0000 7f11bec00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11b80708c0 con 0x7f11b80d1c20 2024-04-25T14:05:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.432+0000 7f11bec00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f11b8070740 con 0x7f11b8070330 2024-04-25T14:05:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.432+0000 7f11bec00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f11b816afc0 con 0x7f11b806f270 2024-04-25T14:05:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.432+0000 7f11bcc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11b80d1c20 0x7f11b8156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.432+0000 7f11bcc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11b80d1c20 0x7f11b8156fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47702/0 (socket says 172.21.15.86:47702) 2024-04-25T14:05:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.432+0000 7f11bcc00640 1 -- 172.21.15.86:0/2492515015 learned_addr learned my addr 172.21.15.86:0/2492515015 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.432+0000 7f11afe00640 1 --2- 172.21.15.86:0/2492515015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11b8070330 0x7f11b80b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.432+0000 7f11b7e00640 1 --2- 172.21.15.86:0/2492515015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11b806f270 0x7f11b80cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.432+0000 7f11bcc00640 1 -- 172.21.15.86:0/2492515015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11b806f270 msgr2=0x7f11b80cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.432+0000 7f11bcc00640 1 --2- 172.21.15.86:0/2492515015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11b806f270 0x7f11b80cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.432+0000 7f11bcc00640 1 -- 172.21.15.86:0/2492515015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11b8070330 msgr2=0x7f11b80b1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.432+0000 7f11bcc00640 1 --2- 172.21.15.86:0/2492515015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11b8070330 0x7f11b80b1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.432+0000 7f11bcc00640 1 -- 172.21.15.86:0/2492515015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b8073730 con 0x7f11b80d1c20 2024-04-25T14:05:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.432+0000 7f11afe00640 1 --2- 172.21.15.86:0/2492515015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11b8070330 0x7f11b80b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.432+0000 7f11b7e00640 1 --2- 172.21.15.86:0/2492515015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11b806f270 0x7f11b80cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.436+0000 7f11bcc00640 1 --2- 172.21.15.86:0/2492515015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11b80d1c20 0x7f11b8156fd0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f11b000cc40 tx=0x7f11b000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:42.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.436+0000 7f11b4c00640 1 -- 172.21.15.86:0/2492515015 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b000ce50 con 0x7f11b80d1c20 2024-04-25T14:05:42.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.436+0000 7f11b4c00640 1 -- 172.21.15.86:0/2492515015 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11b0012400 con 0x7f11b80d1c20 2024-04-25T14:05:42.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.436+0000 7f11b4c00640 1 -- 172.21.15.86:0/2492515015 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b00127f0 con 0x7f11b80d1c20 2024-04-25T14:05:42.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.436+0000 7f11bec00640 1 -- 172.21.15.86:0/2492515015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11b8073c30 con 0x7f11b80d1c20 2024-04-25T14:05:42.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.436+0000 7f11bec00640 1 -- 172.21.15.86:0/2492515015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11b80741f0 con 0x7f11b80d1c20 2024-04-25T14:05:42.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.436+0000 7f11bec00640 1 -- 172.21.15.86:0/2492515015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11b80739a0 con 0x7f11b80d1c20 2024-04-25T14:05:42.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.436+0000 7f11b4c00640 1 -- 172.21.15.86:0/2492515015 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f11b0012990 con 0x7f11b80d1c20 2024-04-25T14:05:42.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.436+0000 7f11b4c00640 1 --2- 172.21.15.86:0/2492515015 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f119406f740 0x7f1194071c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:42.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.436+0000 7f11b4c00640 1 -- 172.21.15.86:0/2492515015 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6433+0+0 (secure 0 0 0) 0x7f11b008ff90 con 0x7f11b80d1c20 2024-04-25T14:05:42.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.440+0000 7f11b7e00640 1 --2- 172.21.15.86:0/2492515015 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f119406f740 0x7f1194071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:42.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.440+0000 7f11b7e00640 1 --2- 172.21.15.86:0/2492515015 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f119406f740 0x7f1194071c50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f11a80ad000 tx=0x7f11a8008040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:42.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.444+0000 7f11b4c00640 1 -- 172.21.15.86:0/2492515015 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f11b80d2030 con 0x7f11b80d1c20 2024-04-25T14:05:42.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.600+0000 7f11bec00640 1 -- 172.21.15.86:0/2492515015 --> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f11b8069d00 con 0x7f119406f740 2024-04-25T14:05:42.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.604+0000 7f11b4c00640 1 -- 172.21.15.86:0/2492515015 <== mgr.5532 v2:172.21.15.86:6832/2427179681 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251951 (secure 0 0 0) 0x7f11b8069d00 con 0x7f119406f740 2024-04-25T14:05:42.610 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:42.614 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:05:42.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.608+0000 7f11bec00640 1 -- 172.21.15.86:0/2492515015 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f119406f740 msgr2=0x7f1194071c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:42.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.608+0000 7f11bec00640 1 --2- 172.21.15.86:0/2492515015 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f119406f740 0x7f1194071c50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f11a80ad000 tx=0x7f11a8008040 comp rx=0 tx=0).stop 2024-04-25T14:05:42.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.608+0000 7f11bec00640 1 -- 172.21.15.86:0/2492515015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11b80d1c20 msgr2=0x7f11b8156fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:42.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.608+0000 7f11bec00640 1 --2- 172.21.15.86:0/2492515015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11b80d1c20 0x7f11b8156fd0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f11b000cc40 tx=0x7f11b000a430 comp rx=0 tx=0).stop 2024-04-25T14:05:42.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.608+0000 7f11bec00640 1 -- 172.21.15.86:0/2492515015 shutdown_connections 2024-04-25T14:05:42.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.608+0000 7f11bec00640 1 --2- 172.21.15.86:0/2492515015 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f119406f740 0x7f1194071c50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:42.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.608+0000 7f11bec00640 1 --2- 172.21.15.86:0/2492515015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11b80d1c20 0x7f11b8156fd0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:42.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.608+0000 7f11bec00640 1 --2- 172.21.15.86:0/2492515015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11b8070330 0x7f11b80b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:42.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.608+0000 7f11bec00640 1 --2- 172.21.15.86:0/2492515015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11b806f270 0x7f11b80cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:42.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.608+0000 7f11bec00640 1 -- 172.21.15.86:0/2492515015 >> 172.21.15.86:0/2492515015 conn(0x7f11b80b4650 msgr2=0x7f11b80b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:42.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.608+0000 7f11bec00640 1 -- 172.21.15.86:0/2492515015 shutdown_connections 2024-04-25T14:05:42.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.608+0000 7f11bec00640 1 -- 172.21.15.86:0/2492515015 wait complete. 2024-04-25T14:05:42.631 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-04-25T14:05:41.483705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221676,"kb_used_data":5480,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753044,"statfs":{"total":773094113280,"available":772867117056,"internally_reserved":0,"allocated":5611520,"data_stored":3141612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001778"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039801+0000","last_change":"2024-04-25T14:05:33.584174+0000","last_active":"2024-04-25T14:05:35.039801+0000","last_peered":"2024-04-25T14:05:35.039801+0000","last_clean":"2024-04-25T14:05:35.039801+0000","last_became_active":"2024-04-25T14:05:33.583867+0000","last_became_peered":"2024-04-25T14:05:33.583867+0000","last_unstale":"2024-04-25T14:05:35.039801+0000","last_undegraded":"2024-04-25T14:05:35.039801+0000","last_fullsized":"2024-04-25T14:05:35.039801+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:36.723420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.574204+0000","last_change":"2024-04-25T14:05:33.579643+0000","last_active":"2024-04-25T14:05:34.574204+0000","last_peered":"2024-04-25T14:05:34.574204+0000","last_clean":"2024-04-25T14:05:34.574204+0000","last_became_active":"2024-04-25T14:05:33.579334+0000","last_became_peered":"2024-04-25T14:05:33.579334+0000","last_unstale":"2024-04-25T14:05:34.574204+0000","last_undegraded":"2024-04-25T14:05:34.574204+0000","last_fullsized":"2024-04-25T14:05:34.574204+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:50.417159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.570528+0000","last_change":"2024-04-25T14:05:33.584460+0000","last_active":"2024-04-25T14:05:34.570528+0000","last_peered":"2024-04-25T14:05:34.570528+0000","last_clean":"2024-04-25T14:05:34.570528+0000","last_became_active":"2024-04-25T14:05:33.584178+0000","last_became_peered":"2024-04-25T14:05:33.584178+0000","last_unstale":"2024-04-25T14:05:34.570528+0000","last_undegraded":"2024-04-25T14:05:34.570528+0000","last_fullsized":"2024-04-25T14:05:34.570528+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:04.017485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.574053+0000","last_change":"2024-04-25T14:05:33.576013+0000","last_active":"2024-04-25T14:05:34.574053+0000","last_peered":"2024-04-25T14:05:34.574053+0000","last_clean":"2024-04-25T14:05:34.574053+0000","last_became_active":"2024-04-25T14:05:33.575693+0000","last_became_peered":"2024-04-25T14:05:33.575693+0000","last_unstale":"2024-04-25T14:05:34.574053+0000","last_undegraded":"2024-04-25T14:05:34.574053+0000","last_fullsized":"2024-04-25T14:05:34.574053+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:16.126866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.562073+0000","last_change":"2024-04-25T14:05:33.575718+0000","last_active":"2024-04-25T14:05:34.562073+0000","last_peered":"2024-04-25T14:05:34.562073+0000","last_clean":"2024-04-25T14:05:34.562073+0000","last_became_active":"2024-04-25T14:05:33.571491+0000","last_became_peered":"2024-04-25T14:05:33.571491+0000","last_unstale":"2024-04-25T14:05:34.562073+0000","last_undegraded":"2024-04-25T14:05:34.562073+0000","last_fullsized":"2024-04-25T14:05:34.562073+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:36.296676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039411+0000","last_change":"2024-04-25T14:05:33.573091+0000","last_active":"2024-04-25T14:05:35.039411+0000","last_peered":"2024-04-25T14:05:35.039411+0000","last_clean":"2024-04-25T14:05:35.039411+0000","last_became_active":"2024-04-25T14:05:33.572800+0000","last_became_peered":"2024-04-25T14:05:33.572800+0000","last_unstale":"2024-04-25T14:05:35.039411+0000","last_undegraded":"2024-04-25T14:05:35.039411+0000","last_fullsized":"2024-04-25T14:05:35.039411+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:39.721568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.569899+0000","last_change":"2024-04-25T14:05:33.588123+0000","last_active":"2024-04-25T14:05:34.569899+0000","last_peered":"2024-04-25T14:05:34.569899+0000","last_clean":"2024-04-25T14:05:34.569899+0000","last_became_active":"2024-04-25T14:05:33.587848+0000","last_became_peered":"2024-04-25T14:05:33.587848+0000","last_unstale":"2024-04-25T14:05:34.569899+0000","last_undegraded":"2024-04-25T14:05:34.569899+0000","last_fullsized":"2024-04-25T14:05:34.569899+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:00.811494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.560921+0000","last_change":"2024-04-25T14:05:33.567123+0000","last_active":"2024-04-25T14:05:34.560921+0000","last_peered":"2024-04-25T14:05:34.560921+0000","last_clean":"2024-04-25T14:05:34.560921+0000","last_became_active":"2024-04-25T14:05:33.563225+0000","last_became_peered":"2024-04-25T14:05:33.563225+0000","last_unstale":"2024-04-25T14:05:34.560921+0000","last_undegraded":"2024-04-25T14:05:34.560921+0000","last_fullsized":"2024-04-25T14:05:34.560921+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:20.253140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.561578+0000","last_change":"2024-04-25T14:05:33.560454+0000","last_active":"2024-04-25T14:05:34.561578+0000","last_peered":"2024-04-25T14:05:34.561578+0000","last_clean":"2024-04-25T14:05:34.561578+0000","last_became_active":"2024-04-25T14:05:33.560174+0000","last_became_peered":"2024-04-25T14:05:33.560174+0000","last_unstale":"2024-04-25T14:05:34.561578+0000","last_undegraded":"2024-04-25T14:05:34.561578+0000","last_fullsized":"2024-04-25T14:05:34.561578+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:07.917626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039521+0000","last_change":"2024-04-25T14:05:33.579278+0000","last_active":"2024-04-25T14:05:35.039521+0000","last_peered":"2024-04-25T14:05:35.039521+0000","last_clean":"2024-04-25T14:05:35.039521+0000","last_became_active":"2024-04-25T14:05:33.578940+0000","last_became_peered":"2024-04-25T14:05:33.578940+0000","last_unstale":"2024-04-25T14:05:35.039521+0000","last_undegraded":"2024-04-25T14:05:35.039521+0000","last_fullsized":"2024-04-25T14:05:35.039521+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:01.444836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.040189+0000","last_change":"2024-04-25T14:05:33.573200+0000","last_active":"2024-04-25T14:05:35.040189+0000","last_peered":"2024-04-25T14:05:35.040189+0000","last_clean":"2024-04-25T14:05:35.040189+0000","last_became_active":"2024-04-25T14:05:33.572858+0000","last_became_peered":"2024-04-25T14:05:33.572858+0000","last_unstale":"2024-04-25T14:05:35.040189+0000","last_undegraded":"2024-04-25T14:05:35.040189+0000","last_fullsized":"2024-04-25T14:05:35.040189+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:22.996486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.561334+0000","last_change":"2024-04-25T14:05:33.565074+0000","last_active":"2024-04-25T14:05:34.561334+0000","last_peered":"2024-04-25T14:05:34.561334+0000","last_clean":"2024-04-25T14:05:34.561334+0000","last_became_active":"2024-04-25T14:05:33.564800+0000","last_became_peered":"2024-04-25T14:05:33.564800+0000","last_unstale":"2024-04-25T14:05:34.561334+0000","last_undegraded":"2024-04-25T14:05:34.561334+0000","last_fullsized":"2024-04-25T14:05:34.561334+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:00.811494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039713+0000","last_change":"2024-04-25T14:05:33.572748+0000","last_active":"2024-04-25T14:05:35.039713+0000","last_peered":"2024-04-25T14:05:35.039713+0000","last_clean":"2024-04-25T14:05:35.039713+0000","last_became_active":"2024-04-25T14:05:33.572256+0000","last_became_peered":"2024-04-25T14:05:33.572256+0000","last_unstale":"2024-04-25T14:05:35.039713+0000","last_undegraded":"2024-04-25T14:05:35.039713+0000","last_fullsized":"2024-04-25T14:05:35.039713+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:43.410378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.570178+0000","last_change":"2024-04-25T14:05:33.581470+0000","last_active":"2024-04-25T14:05:34.570178+0000","last_peered":"2024-04-25T14:05:34.570178+0000","last_clean":"2024-04-25T14:05:34.570178+0000","last_became_active":"2024-04-25T14:05:33.581160+0000","last_became_peered":"2024-04-25T14:05:33.581160+0000","last_unstale":"2024-04-25T14:05:34.570178+0000","last_undegraded":"2024-04-25T14:05:34.570178+0000","last_fullsized":"2024-04-25T14:05:34.570178+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:47.818409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.573678+0000","last_change":"2024-04-25T14:05:33.575396+0000","last_active":"2024-04-25T14:05:34.573678+0000","last_peered":"2024-04-25T14:05:34.573678+0000","last_clean":"2024-04-25T14:05:34.573678+0000","last_became_active":"2024-04-25T14:05:33.574772+0000","last_became_peered":"2024-04-25T14:05:33.574772+0000","last_unstale":"2024-04-25T14:05:34.573678+0000","last_undegraded":"2024-04-25T14:05:34.573678+0000","last_fullsized":"2024-04-25T14:05:34.573678+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:25.659996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039304+0000","last_change":"2024-04-25T14:05:33.573271+0000","last_active":"2024-04-25T14:05:35.039304+0000","last_peered":"2024-04-25T14:05:35.039304+0000","last_clean":"2024-04-25T14:05:35.039304+0000","last_became_active":"2024-04-25T14:05:33.572858+0000","last_became_peered":"2024-04-25T14:05:33.572858+0000","last_unstale":"2024-04-25T14:05:35.039304+0000","last_undegraded":"2024-04-25T14:05:35.039304+0000","last_fullsized":"2024-04-25T14:05:35.039304+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:27.918457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.570915+0000","last_change":"2024-04-25T14:05:33.572330+0000","last_active":"2024-04-25T14:05:34.570915+0000","last_peered":"2024-04-25T14:05:34.570915+0000","last_clean":"2024-04-25T14:05:34.570915+0000","last_became_active":"2024-04-25T14:05:33.571842+0000","last_became_peered":"2024-04-25T14:05:33.571842+0000","last_unstale":"2024-04-25T14:05:34.570915+0000","last_undegraded":"2024-04-25T14:05:34.570915+0000","last_fullsized":"2024-04-25T14:05:34.570915+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:28.424318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.574230+0000","last_change":"2024-04-25T14:05:33.580135+0000","last_active":"2024-04-25T14:05:34.574230+0000","last_peered":"2024-04-25T14:05:34.574230+0000","last_clean":"2024-04-25T14:05:34.574230+0000","last_became_active":"2024-04-25T14:05:33.579677+0000","last_became_peered":"2024-04-25T14:05:33.579677+0000","last_unstale":"2024-04-25T14:05:34.574230+0000","last_undegraded":"2024-04-25T14:05:34.574230+0000","last_fullsized":"2024-04-25T14:05:34.574230+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:20.253140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.570558+0000","last_change":"2024-04-25T14:05:33.574245+0000","last_active":"2024-04-25T14:05:34.570558+0000","last_peered":"2024-04-25T14:05:34.570558+0000","last_clean":"2024-04-25T14:05:34.570558+0000","last_became_active":"2024-04-25T14:05:33.573980+0000","last_became_peered":"2024-04-25T14:05:33.573980+0000","last_unstale":"2024-04-25T14:05:34.570558+0000","last_undegraded":"2024-04-25T14:05:34.570558+0000","last_fullsized":"2024-04-25T14:05:34.570558+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:30.240956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039514+0000","last_change":"2024-04-25T14:05:33.571072+0000","last_active":"2024-04-25T14:05:35.039514+0000","last_peered":"2024-04-25T14:05:35.039514+0000","last_clean":"2024-04-25T14:05:35.039514+0000","last_became_active":"2024-04-25T14:05:33.570748+0000","last_became_peered":"2024-04-25T14:05:33.570748+0000","last_unstale":"2024-04-25T14:05:35.039514+0000","last_undegraded":"2024-04-25T14:05:35.039514+0000","last_fullsized":"2024-04-25T14:05:35.039514+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:18.074490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039301+0000","last_change":"2024-04-25T14:05:33.571248+0000","last_active":"2024-04-25T14:05:35.039301+0000","last_peered":"2024-04-25T14:05:35.039301+0000","last_clean":"2024-04-25T14:05:35.039301+0000","last_became_active":"2024-04-25T14:05:33.570916+0000","last_became_peered":"2024-04-25T14:05:33.570916+0000","last_unstale":"2024-04-25T14:05:35.039301+0000","last_undegraded":"2024-04-25T14:05:35.039301+0000","last_fullsized":"2024-04-25T14:05:35.039301+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:36.296676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.561023+0000","last_change":"2024-04-25T14:05:33.560440+0000","last_active":"2024-04-25T14:05:34.561023+0000","last_peered":"2024-04-25T14:05:34.561023+0000","last_clean":"2024-04-25T14:05:34.561023+0000","last_became_active":"2024-04-25T14:05:33.560066+0000","last_became_peered":"2024-04-25T14:05:33.560066+0000","last_unstale":"2024-04-25T14:05:34.561023+0000","last_undegraded":"2024-04-25T14:05:34.561023+0000","last_fullsized":"2024-04-25T14:05:34.561023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:32.906788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039600+0000","last_change":"2024-04-25T14:05:33.576117+0000","last_active":"2024-04-25T14:05:35.039600+0000","last_peered":"2024-04-25T14:05:35.039600+0000","last_clean":"2024-04-25T14:05:35.039600+0000","last_became_active":"2024-04-25T14:05:33.575838+0000","last_became_peered":"2024-04-25T14:05:33.575838+0000","last_unstale":"2024-04-25T14:05:35.039600+0000","last_undegraded":"2024-04-25T14:05:35.039600+0000","last_fullsized":"2024-04-25T14:05:35.039600+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:20.619851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.569739+0000","last_change":"2024-04-25T14:05:33.581503+0000","last_active":"2024-04-25T14:05:34.569739+0000","last_peered":"2024-04-25T14:05:34.569739+0000","last_clean":"2024-04-25T14:05:34.569739+0000","last_became_active":"2024-04-25T14:05:33.581124+0000","last_became_peered":"2024-04-25T14:05:33.581124+0000","last_unstale":"2024-04-25T14:05:34.569739+0000","last_undegraded":"2024-04-25T14:05:34.569739+0000","last_fullsized":"2024-04-25T14:05:34.569739+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:33.455514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.571072+0000","last_change":"2024-04-25T14:05:33.562329+0000","last_active":"2024-04-25T14:05:34.571072+0000","last_peered":"2024-04-25T14:05:34.571072+0000","last_clean":"2024-04-25T14:05:34.571072+0000","last_became_active":"2024-04-25T14:05:33.562039+0000","last_became_peered":"2024-04-25T14:05:33.562039+0000","last_unstale":"2024-04-25T14:05:34.571072+0000","last_undegraded":"2024-04-25T14:05:34.571072+0000","last_fullsized":"2024-04-25T14:05:34.571072+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:29.981652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.561318+0000","last_change":"2024-04-25T14:05:33.576318+0000","last_active":"2024-04-25T14:05:34.561318+0000","last_peered":"2024-04-25T14:05:34.561318+0000","last_clean":"2024-04-25T14:05:34.561318+0000","last_became_active":"2024-04-25T14:05:33.575764+0000","last_became_peered":"2024-04-25T14:05:33.575764+0000","last_unstale":"2024-04-25T14:05:34.561318+0000","last_undegraded":"2024-04-25T14:05:34.561318+0000","last_fullsized":"2024-04-25T14:05:34.561318+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:33.455514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039933+0000","last_change":"2024-04-25T14:05:33.591448+0000","last_active":"2024-04-25T14:05:35.039933+0000","last_peered":"2024-04-25T14:05:35.039933+0000","last_clean":"2024-04-25T14:05:35.039933+0000","last_became_active":"2024-04-25T14:05:33.590795+0000","last_became_peered":"2024-04-25T14:05:33.590795+0000","last_unstale":"2024-04-25T14:05:35.039933+0000","last_undegraded":"2024-04-25T14:05:35.039933+0000","last_fullsized":"2024-04-25T14:05:35.039933+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:33.637941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039689+0000","last_change":"2024-04-25T14:05:33.596454+0000","last_active":"2024-04-25T14:05:35.039689+0000","last_peered":"2024-04-25T14:05:35.039689+0000","last_clean":"2024-04-25T14:05:35.039689+0000","last_became_active":"2024-04-25T14:05:33.596225+0000","last_became_peered":"2024-04-25T14:05:33.596225+0000","last_unstale":"2024-04-25T14:05:35.039689+0000","last_undegraded":"2024-04-25T14:05:35.039689+0000","last_fullsized":"2024-04-25T14:05:35.039689+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:41.512359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.570322+0000","last_change":"2024-04-25T14:05:33.572734+0000","last_active":"2024-04-25T14:05:34.570322+0000","last_peered":"2024-04-25T14:05:34.570322+0000","last_clean":"2024-04-25T14:05:34.570322+0000","last_became_active":"2024-04-25T14:05:33.572231+0000","last_became_peered":"2024-04-25T14:05:33.572231+0000","last_unstale":"2024-04-25T14:05:34.570322+0000","last_undegraded":"2024-04-25T14:05:34.570322+0000","last_fullsized":"2024-04-25T14:05:34.570322+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:51.585470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.573782+0000","last_change":"2024-04-25T14:05:33.574825+0000","last_active":"2024-04-25T14:05:34.573782+0000","last_peered":"2024-04-25T14:05:34.573782+0000","last_clean":"2024-04-25T14:05:34.573782+0000","last_became_active":"2024-04-25T14:05:33.574492+0000","last_became_peered":"2024-04-25T14:05:33.574492+0000","last_unstale":"2024-04-25T14:05:34.573782+0000","last_undegraded":"2024-04-25T14:05:34.573782+0000","last_fullsized":"2024-04-25T14:05:34.573782+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:33.637941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.570046+0000","last_change":"2024-04-25T14:05:33.583788+0000","last_active":"2024-04-25T14:05:34.570046+0000","last_peered":"2024-04-25T14:05:34.570046+0000","last_clean":"2024-04-25T14:05:34.570046+0000","last_became_active":"2024-04-25T14:05:33.583507+0000","last_became_peered":"2024-04-25T14:05:33.583507+0000","last_unstale":"2024-04-25T14:05:34.570046+0000","last_undegraded":"2024-04-25T14:05:34.570046+0000","last_fullsized":"2024-04-25T14:05:34.570046+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:06.138261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.563194+0000","last_change":"2024-04-25T14:05:33.561025+0000","last_active":"2024-04-25T14:05:34.563194+0000","last_peered":"2024-04-25T14:05:34.563194+0000","last_clean":"2024-04-25T14:05:34.563194+0000","last_became_active":"2024-04-25T14:05:33.560525+0000","last_became_peered":"2024-04-25T14:05:33.560525+0000","last_unstale":"2024-04-25T14:05:34.563194+0000","last_undegraded":"2024-04-25T14:05:34.563194+0000","last_fullsized":"2024-04-25T14:05:34.563194+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:54.564632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.573564+0000","last_change":"2024-04-25T14:05:33.576943+0000","last_active":"2024-04-25T14:05:34.573564+0000","last_peered":"2024-04-25T14:05:34.573564+0000","last_clean":"2024-04-25T14:05:34.573564+0000","last_became_active":"2024-04-25T14:05:33.576387+0000","last_became_peered":"2024-04-25T14:05:33.576387+0000","last_unstale":"2024-04-25T14:05:34.573564+0000","last_undegraded":"2024-04-25T14:05:34.573564+0000","last_fullsized":"2024-04-25T14:05:34.573564+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:54.564632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"45'4","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039294+0000","last_change":"2024-04-25T14:05:32.556304+0000","last_active":"2024-04-25T14:05:35.039294+0000","last_peered":"2024-04-25T14:05:35.039294+0000","last_clean":"2024-04-25T14:05:35.039294+0000","last_became_active":"2024-04-25T14:05:32.556052+0000","last_became_peered":"2024-04-25T14:05:32.556052+0000","last_unstale":"2024-04-25T14:05:35.039294+0000","last_undegraded":"2024-04-25T14:05:35.039294+0000","last_fullsized":"2024-04-25T14:05:35.039294+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:16.452558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.563228+0000","last_change":"2024-04-25T14:05:33.573320+0000","last_active":"2024-04-25T14:05:34.563228+0000","last_peered":"2024-04-25T14:05:34.563228+0000","last_clean":"2024-04-25T14:05:34.563228+0000","last_became_active":"2024-04-25T14:05:33.573015+0000","last_became_peered":"2024-04-25T14:05:33.573015+0000","last_unstale":"2024-04-25T14:05:34.563228+0000","last_undegraded":"2024-04-25T14:05:34.563228+0000","last_fullsized":"2024-04-25T14:05:34.563228+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:25.659996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039390+0000","last_change":"2024-04-25T14:05:32.569704+0000","last_active":"2024-04-25T14:05:35.039390+0000","last_peered":"2024-04-25T14:05:35.039390+0000","last_clean":"2024-04-25T14:05:35.039390+0000","last_became_active":"2024-04-25T14:05:32.569349+0000","last_became_peered":"2024-04-25T14:05:32.569349+0000","last_unstale":"2024-04-25T14:05:35.039390+0000","last_undegraded":"2024-04-25T14:05:35.039390+0000","last_fullsized":"2024-04-25T14:05:35.039390+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:23.323591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039348+0000","last_change":"2024-04-25T14:05:33.574504+0000","last_active":"2024-04-25T14:05:35.039348+0000","last_peered":"2024-04-25T14:05:35.039348+0000","last_clean":"2024-04-25T14:05:35.039348+0000","last_became_active":"2024-04-25T14:05:33.574191+0000","last_became_peered":"2024-04-25T14:05:33.574191+0000","last_unstale":"2024-04-25T14:05:35.039348+0000","last_undegraded":"2024-04-25T14:05:35.039348+0000","last_fullsized":"2024-04-25T14:05:35.039348+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:28.629474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039936+0000","last_change":"2024-04-25T14:05:32.561304+0000","last_active":"2024-04-25T14:05:35.039936+0000","last_peered":"2024-04-25T14:05:35.039936+0000","last_clean":"2024-04-25T14:05:35.039936+0000","last_became_active":"2024-04-25T14:05:32.560988+0000","last_became_peered":"2024-04-25T14:05:32.560988+0000","last_unstale":"2024-04-25T14:05:35.039936+0000","last_undegraded":"2024-04-25T14:05:35.039936+0000","last_fullsized":"2024-04-25T14:05:35.039936+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:32.071918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.560965+0000","last_change":"2024-04-25T14:05:33.593213+0000","last_active":"2024-04-25T14:05:34.560965+0000","last_peered":"2024-04-25T14:05:34.560965+0000","last_clean":"2024-04-25T14:05:34.560965+0000","last_became_active":"2024-04-25T14:05:33.592923+0000","last_became_peered":"2024-04-25T14:05:33.592923+0000","last_unstale":"2024-04-25T14:05:34.560965+0000","last_undegraded":"2024-04-25T14:05:34.560965+0000","last_fullsized":"2024-04-25T14:05:34.560965+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:18.458163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.563732+0000","last_change":"2024-04-25T14:05:32.567307+0000","last_active":"2024-04-25T14:05:34.563732+0000","last_peered":"2024-04-25T14:05:34.563732+0000","last_clean":"2024-04-25T14:05:34.563732+0000","last_became_active":"2024-04-25T14:05:32.566940+0000","last_became_peered":"2024-04-25T14:05:32.566940+0000","last_unstale":"2024-04-25T14:05:34.563732+0000","last_undegraded":"2024-04-25T14:05:34.563732+0000","last_fullsized":"2024-04-25T14:05:34.563732+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:12.973199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.562395+0000","last_change":"2024-04-25T14:05:33.582814+0000","last_active":"2024-04-25T14:05:34.562395+0000","last_peered":"2024-04-25T14:05:34.562395+0000","last_clean":"2024-04-25T14:05:34.562395+0000","last_became_active":"2024-04-25T14:05:33.582127+0000","last_became_peered":"2024-04-25T14:05:33.582127+0000","last_unstale":"2024-04-25T14:05:34.562395+0000","last_undegraded":"2024-04-25T14:05:34.562395+0000","last_fullsized":"2024-04-25T14:05:34.562395+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:18.074490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.570095+0000","last_change":"2024-04-25T14:05:32.562043+0000","last_active":"2024-04-25T14:05:34.570095+0000","last_peered":"2024-04-25T14:05:34.570095+0000","last_clean":"2024-04-25T14:05:34.570095+0000","last_became_active":"2024-04-25T14:05:32.561736+0000","last_became_peered":"2024-04-25T14:05:32.561736+0000","last_unstale":"2024-04-25T14:05:34.570095+0000","last_undegraded":"2024-04-25T14:05:34.570095+0000","last_fullsized":"2024-04-25T14:05:34.570095+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:40.505936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.573942+0000","last_change":"2024-04-25T14:05:33.584433+0000","last_active":"2024-04-25T14:05:34.573942+0000","last_peered":"2024-04-25T14:05:34.573942+0000","last_clean":"2024-04-25T14:05:34.573942+0000","last_became_active":"2024-04-25T14:05:33.584130+0000","last_became_peered":"2024-04-25T14:05:33.584130+0000","last_unstale":"2024-04-25T14:05:34.573942+0000","last_undegraded":"2024-04-25T14:05:34.573942+0000","last_fullsized":"2024-04-25T14:05:34.573942+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:42.654848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.562063+0000","last_change":"2024-04-25T14:05:32.557698+0000","last_active":"2024-04-25T14:05:34.562063+0000","last_peered":"2024-04-25T14:05:34.562063+0000","last_clean":"2024-04-25T14:05:34.562063+0000","last_became_active":"2024-04-25T14:05:32.557358+0000","last_became_peered":"2024-04-25T14:05:32.557358+0000","last_unstale":"2024-04-25T14:05:34.562063+0000","last_undegraded":"2024-04-25T14:05:34.562063+0000","last_fullsized":"2024-04-25T14:05:34.562063+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:20.281168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.561439+0000","last_change":"2024-04-25T14:05:33.576549+0000","last_active":"2024-04-25T14:05:34.561439+0000","last_peered":"2024-04-25T14:05:34.561439+0000","last_clean":"2024-04-25T14:05:34.561439+0000","last_became_active":"2024-04-25T14:05:33.576037+0000","last_became_peered":"2024-04-25T14:05:33.576037+0000","last_unstale":"2024-04-25T14:05:34.561439+0000","last_undegraded":"2024-04-25T14:05:34.561439+0000","last_fullsized":"2024-04-25T14:05:34.561439+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:47.818409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039072+0000","last_change":"2024-04-25T14:05:32.551806+0000","last_active":"2024-04-25T14:05:35.039072+0000","last_peered":"2024-04-25T14:05:35.039072+0000","last_clean":"2024-04-25T14:05:35.039072+0000","last_became_active":"2024-04-25T14:05:32.551441+0000","last_became_peered":"2024-04-25T14:05:32.551441+0000","last_unstale":"2024-04-25T14:05:35.039072+0000","last_undegraded":"2024-04-25T14:05:35.039072+0000","last_fullsized":"2024-04-25T14:05:35.039072+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:31.414048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.574150+0000","last_change":"2024-04-25T14:05:33.583073+0000","last_active":"2024-04-25T14:05:34.574150+0000","last_peered":"2024-04-25T14:05:34.574150+0000","last_clean":"2024-04-25T14:05:34.574150+0000","last_became_active":"2024-04-25T14:05:33.582763+0000","last_became_peered":"2024-04-25T14:05:33.582763+0000","last_unstale":"2024-04-25T14:05:34.574150+0000","last_undegraded":"2024-04-25T14:05:34.574150+0000","last_fullsized":"2024-04-25T14:05:34.574150+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:18.567327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.569671+0000","last_change":"2024-04-25T14:05:32.563104+0000","last_active":"2024-04-25T14:05:34.569671+0000","last_peered":"2024-04-25T14:05:34.569671+0000","last_clean":"2024-04-25T14:05:34.569671+0000","last_became_active":"2024-04-25T14:05:32.560440+0000","last_became_peered":"2024-04-25T14:05:32.560440+0000","last_unstale":"2024-04-25T14:05:34.569671+0000","last_undegraded":"2024-04-25T14:05:34.569671+0000","last_fullsized":"2024-04-25T14:05:34.569671+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:12.973199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.561874+0000","last_change":"2024-04-25T14:05:33.583166+0000","last_active":"2024-04-25T14:05:34.561874+0000","last_peered":"2024-04-25T14:05:34.561874+0000","last_clean":"2024-04-25T14:05:34.561874+0000","last_became_active":"2024-04-25T14:05:33.582833+0000","last_became_peered":"2024-04-25T14:05:33.582833+0000","last_unstale":"2024-04-25T14:05:34.561874+0000","last_undegraded":"2024-04-25T14:05:34.561874+0000","last_fullsized":"2024-04-25T14:05:34.561874+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:11.087435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.574048+0000","last_change":"2024-04-25T14:05:32.550533+0000","last_active":"2024-04-25T14:05:34.574048+0000","last_peered":"2024-04-25T14:05:34.574048+0000","last_clean":"2024-04-25T14:05:34.574048+0000","last_became_active":"2024-04-25T14:05:32.550116+0000","last_became_peered":"2024-04-25T14:05:32.550116+0000","last_unstale":"2024-04-25T14:05:34.574048+0000","last_undegraded":"2024-04-25T14:05:34.574048+0000","last_fullsized":"2024-04-25T14:05:34.574048+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:27.623051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.563192+0000","last_change":"2024-04-25T14:05:33.573758+0000","last_active":"2024-04-25T14:05:34.563192+0000","last_peered":"2024-04-25T14:05:34.563192+0000","last_clean":"2024-04-25T14:05:34.563192+0000","last_became_active":"2024-04-25T14:05:33.572760+0000","last_became_peered":"2024-04-25T14:05:33.572760+0000","last_unstale":"2024-04-25T14:05:34.563192+0000","last_undegraded":"2024-04-25T14:05:34.563192+0000","last_fullsized":"2024-04-25T14:05:34.563192+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:36.723420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.562354+0000","last_change":"2024-04-25T14:05:32.563130+0000","last_active":"2024-04-25T14:05:34.562354+0000","last_peered":"2024-04-25T14:05:34.562354+0000","last_clean":"2024-04-25T14:05:34.562354+0000","last_became_active":"2024-04-25T14:05:32.562793+0000","last_became_peered":"2024-04-25T14:05:32.562793+0000","last_unstale":"2024-04-25T14:05:34.562354+0000","last_undegraded":"2024-04-25T14:05:34.562354+0000","last_fullsized":"2024-04-25T14:05:34.562354+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:43.267251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.570181+0000","last_change":"2024-04-25T14:05:33.577142+0000","last_active":"2024-04-25T14:05:34.570181+0000","last_peered":"2024-04-25T14:05:34.570181+0000","last_clean":"2024-04-25T14:05:34.570181+0000","last_became_active":"2024-04-25T14:05:33.576820+0000","last_became_peered":"2024-04-25T14:05:33.576820+0000","last_unstale":"2024-04-25T14:05:34.570181+0000","last_undegraded":"2024-04-25T14:05:34.570181+0000","last_fullsized":"2024-04-25T14:05:34.570181+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:07.917626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"45'3","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.573943+0000","last_change":"2024-04-25T14:05:32.566039+0000","last_active":"2024-04-25T14:05:34.573943+0000","last_peered":"2024-04-25T14:05:34.573943+0000","last_clean":"2024-04-25T14:05:34.573943+0000","last_became_active":"2024-04-25T14:05:32.565670+0000","last_became_peered":"2024-04-25T14:05:32.565670+0000","last_unstale":"2024-04-25T14:05:34.573943+0000","last_undegraded":"2024-04-25T14:05:34.573943+0000","last_fullsized":"2024-04-25T14:05:34.573943+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:12.973199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.562162+0000","last_change":"2024-04-25T14:05:33.593376+0000","last_active":"2024-04-25T14:05:34.562162+0000","last_peered":"2024-04-25T14:05:34.562162+0000","last_clean":"2024-04-25T14:05:34.562162+0000","last_became_active":"2024-04-25T14:05:33.593105+0000","last_became_peered":"2024-04-25T14:05:33.593105+0000","last_unstale":"2024-04-25T14:05:34.562162+0000","last_undegraded":"2024-04-25T14:05:34.562162+0000","last_fullsized":"2024-04-25T14:05:34.562162+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:28.629474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"45'6","reported_seq":20,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.561236+0000","last_change":"2024-04-25T14:05:32.564199+0000","last_active":"2024-04-25T14:05:34.561236+0000","last_peered":"2024-04-25T14:05:34.561236+0000","last_clean":"2024-04-25T14:05:34.561236+0000","last_became_active":"2024-04-25T14:05:32.563935+0000","last_became_peered":"2024-04-25T14:05:32.563935+0000","last_unstale":"2024-04-25T14:05:34.561236+0000","last_undegraded":"2024-04-25T14:05:34.561236+0000","last_fullsized":"2024-04-25T14:05:34.561236+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:38.715145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.563253+0000","last_change":"2024-04-25T14:05:33.569490+0000","last_active":"2024-04-25T14:05:34.563253+0000","last_peered":"2024-04-25T14:05:34.563253+0000","last_clean":"2024-04-25T14:05:34.563253+0000","last_became_active":"2024-04-25T14:05:33.569125+0000","last_became_peered":"2024-04-25T14:05:33.569125+0000","last_unstale":"2024-04-25T14:05:34.563253+0000","last_undegraded":"2024-04-25T14:05:34.563253+0000","last_fullsized":"2024-04-25T14:05:34.563253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:33.637941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.570213+0000","last_change":"2024-04-25T14:05:32.557589+0000","last_active":"2024-04-25T14:05:34.570213+0000","last_peered":"2024-04-25T14:05:34.570213+0000","last_clean":"2024-04-25T14:05:34.570213+0000","last_became_active":"2024-04-25T14:05:32.557260+0000","last_became_peered":"2024-04-25T14:05:32.557260+0000","last_unstale":"2024-04-25T14:05:34.570213+0000","last_undegraded":"2024-04-25T14:05:34.570213+0000","last_fullsized":"2024-04-25T14:05:34.570213+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:26.912034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.570684+0000","last_change":"2024-04-25T14:05:33.586628+0000","last_active":"2024-04-25T14:05:34.570684+0000","last_peered":"2024-04-25T14:05:34.570684+0000","last_clean":"2024-04-25T14:05:34.570684+0000","last_became_active":"2024-04-25T14:05:33.586317+0000","last_became_peered":"2024-04-25T14:05:33.586317+0000","last_unstale":"2024-04-25T14:05:34.570684+0000","last_undegraded":"2024-04-25T14:05:34.570684+0000","last_fullsized":"2024-04-25T14:05:34.570684+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:22.789679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039235+0000","last_change":"2024-04-25T14:05:32.562133+0000","last_active":"2024-04-25T14:05:35.039235+0000","last_peered":"2024-04-25T14:05:35.039235+0000","last_clean":"2024-04-25T14:05:35.039235+0000","last_became_active":"2024-04-25T14:05:32.561845+0000","last_became_peered":"2024-04-25T14:05:32.561845+0000","last_unstale":"2024-04-25T14:05:35.039235+0000","last_undegraded":"2024-04-25T14:05:35.039235+0000","last_fullsized":"2024-04-25T14:05:35.039235+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:14.012822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.561358+0000","last_change":"2024-04-25T14:05:33.571641+0000","last_active":"2024-04-25T14:05:34.561358+0000","last_peered":"2024-04-25T14:05:34.561358+0000","last_clean":"2024-04-25T14:05:34.561358+0000","last_became_active":"2024-04-25T14:05:33.571347+0000","last_became_peered":"2024-04-25T14:05:33.571347+0000","last_unstale":"2024-04-25T14:05:34.561358+0000","last_undegraded":"2024-04-25T14:05:34.561358+0000","last_fullsized":"2024-04-25T14:05:34.561358+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:06.138261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.573596+0000","last_change":"2024-04-25T14:05:32.553901+0000","last_active":"2024-04-25T14:05:34.573596+0000","last_peered":"2024-04-25T14:05:34.573596+0000","last_clean":"2024-04-25T14:05:34.573596+0000","last_became_active":"2024-04-25T14:05:32.552885+0000","last_became_peered":"2024-04-25T14:05:32.552885+0000","last_unstale":"2024-04-25T14:05:34.573596+0000","last_undegraded":"2024-04-25T14:05:34.573596+0000","last_fullsized":"2024-04-25T14:05:34.573596+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:17.068068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.570923+0000","last_change":"2024-04-25T14:05:33.566385+0000","last_active":"2024-04-25T14:05:34.570923+0000","last_peered":"2024-04-25T14:05:34.570923+0000","last_clean":"2024-04-25T14:05:34.570923+0000","last_became_active":"2024-04-25T14:05:33.566106+0000","last_became_peered":"2024-04-25T14:05:33.566106+0000","last_unstale":"2024-04-25T14:05:34.570923+0000","last_undegraded":"2024-04-25T14:05:34.570923+0000","last_fullsized":"2024-04-25T14:05:34.570923+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:51.751312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"45'8","reported_seq":22,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.560982+0000","last_change":"2024-04-25T14:05:32.554990+0000","last_active":"2024-04-25T14:05:34.560982+0000","last_peered":"2024-04-25T14:05:34.560982+0000","last_clean":"2024-04-25T14:05:34.560982+0000","last_became_active":"2024-04-25T14:05:32.554580+0000","last_became_peered":"2024-04-25T14:05:32.554580+0000","last_unstale":"2024-04-25T14:05:34.560982+0000","last_undegraded":"2024-04-25T14:05:34.560982+0000","last_fullsized":"2024-04-25T14:05:34.560982+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:26.912034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.561170+0000","last_change":"2024-04-25T14:05:33.568454+0000","last_active":"2024-04-25T14:05:34.561170+0000","last_peered":"2024-04-25T14:05:34.561170+0000","last_clean":"2024-04-25T14:05:34.561170+0000","last_became_active":"2024-04-25T14:05:33.568140+0000","last_became_peered":"2024-04-25T14:05:33.568140+0000","last_unstale":"2024-04-25T14:05:34.561170+0000","last_undegraded":"2024-04-25T14:05:34.561170+0000","last_fullsized":"2024-04-25T14:05:34.561170+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:09.011115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.570995+0000","last_change":"2024-04-25T14:05:32.569447+0000","last_active":"2024-04-25T14:05:34.570995+0000","last_peered":"2024-04-25T14:05:34.570995+0000","last_clean":"2024-04-25T14:05:34.570995+0000","last_became_active":"2024-04-25T14:05:32.568861+0000","last_became_peered":"2024-04-25T14:05:32.568861+0000","last_unstale":"2024-04-25T14:05:34.570995+0000","last_undegraded":"2024-04-25T14:05:34.570995+0000","last_fullsized":"2024-04-25T14:05:34.570995+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:00.438414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039382+0000","last_change":"2024-04-25T14:05:33.590347+0000","last_active":"2024-04-25T14:05:35.039382+0000","last_peered":"2024-04-25T14:05:35.039382+0000","last_clean":"2024-04-25T14:05:35.039382+0000","last_became_active":"2024-04-25T14:05:33.590073+0000","last_became_peered":"2024-04-25T14:05:33.590073+0000","last_unstale":"2024-04-25T14:05:35.039382+0000","last_undegraded":"2024-04-25T14:05:35.039382+0000","last_fullsized":"2024-04-25T14:05:35.039382+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:11.280699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.040239+0000","last_change":"2024-04-25T14:05:32.559217+0000","last_active":"2024-04-25T14:05:35.040239+0000","last_peered":"2024-04-25T14:05:35.040239+0000","last_clean":"2024-04-25T14:05:35.040239+0000","last_became_active":"2024-04-25T14:05:32.558909+0000","last_became_peered":"2024-04-25T14:05:32.558909+0000","last_unstale":"2024-04-25T14:05:35.040239+0000","last_undegraded":"2024-04-25T14:05:35.040239+0000","last_fullsized":"2024-04-25T14:05:35.040239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:48:44.213850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.562649+0000","last_change":"2024-04-25T14:05:33.582498+0000","last_active":"2024-04-25T14:05:34.562649+0000","last_peered":"2024-04-25T14:05:34.562649+0000","last_clean":"2024-04-25T14:05:34.562649+0000","last_became_active":"2024-04-25T14:05:33.582156+0000","last_became_peered":"2024-04-25T14:05:33.582156+0000","last_unstale":"2024-04-25T14:05:34.562649+0000","last_undegraded":"2024-04-25T14:05:34.562649+0000","last_fullsized":"2024-04-25T14:05:34.562649+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:13.979622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039597+0000","last_change":"2024-04-25T14:05:32.566219+0000","last_active":"2024-04-25T14:05:35.039597+0000","last_peered":"2024-04-25T14:05:35.039597+0000","last_clean":"2024-04-25T14:05:35.039597+0000","last_became_active":"2024-04-25T14:05:32.565859+0000","last_became_peered":"2024-04-25T14:05:32.565859+0000","last_unstale":"2024-04-25T14:05:35.039597+0000","last_undegraded":"2024-04-25T14:05:35.039597+0000","last_fullsized":"2024-04-25T14:05:35.039597+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:35.649467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039932+0000","last_change":"2024-04-25T14:05:33.572207+0000","last_active":"2024-04-25T14:05:35.039932+0000","last_peered":"2024-04-25T14:05:35.039932+0000","last_clean":"2024-04-25T14:05:35.039932+0000","last_became_active":"2024-04-25T14:05:33.570800+0000","last_became_peered":"2024-04-25T14:05:33.570800+0000","last_unstale":"2024-04-25T14:05:35.039932+0000","last_undegraded":"2024-04-25T14:05:35.039932+0000","last_fullsized":"2024-04-25T14:05:35.039932+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:25.659996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039730+0000","last_change":"2024-04-25T14:05:32.567474+0000","last_active":"2024-04-25T14:05:35.039730+0000","last_peered":"2024-04-25T14:05:35.039730+0000","last_clean":"2024-04-25T14:05:35.039730+0000","last_became_active":"2024-04-25T14:05:32.567061+0000","last_became_peered":"2024-04-25T14:05:32.567061+0000","last_unstale":"2024-04-25T14:05:35.039730+0000","last_undegraded":"2024-04-25T14:05:35.039730+0000","last_fullsized":"2024-04-25T14:05:35.039730+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:20.281168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.563908+0000","last_change":"2024-04-25T14:05:33.571493+0000","last_active":"2024-04-25T14:05:34.563908+0000","last_peered":"2024-04-25T14:05:34.563908+0000","last_clean":"2024-04-25T14:05:34.563908+0000","last_became_active":"2024-04-25T14:05:33.568719+0000","last_became_peered":"2024-04-25T14:05:33.568719+0000","last_unstale":"2024-04-25T14:05:34.563908+0000","last_undegraded":"2024-04-25T14:05:34.563908+0000","last_fullsized":"2024-04-25T14:05:34.563908+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:18.567327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039811+0000","last_change":"2024-04-25T14:05:32.555817+0000","last_active":"2024-04-25T14:05:35.039811+0000","last_peered":"2024-04-25T14:05:35.039811+0000","last_clean":"2024-04-25T14:05:35.039811+0000","last_became_active":"2024-04-25T14:05:32.553958+0000","last_became_peered":"2024-04-25T14:05:32.553958+0000","last_unstale":"2024-04-25T14:05:35.039811+0000","last_undegraded":"2024-04-25T14:05:35.039811+0000","last_fullsized":"2024-04-25T14:05:35.039811+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:43.267251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039542+0000","last_change":"2024-04-25T14:05:33.570587+0000","last_active":"2024-04-25T14:05:35.039542+0000","last_peered":"2024-04-25T14:05:35.039542+0000","last_clean":"2024-04-25T14:05:35.039542+0000","last_became_active":"2024-04-25T14:05:33.570209+0000","last_became_peered":"2024-04-25T14:05:33.570209+0000","last_unstale":"2024-04-25T14:05:35.039542+0000","last_undegraded":"2024-04-25T14:05:35.039542+0000","last_fullsized":"2024-04-25T14:05:35.039542+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:13.979622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.560823+0000","last_change":"2024-04-25T14:05:32.555416+0000","last_active":"2024-04-25T14:05:34.560823+0000","last_peered":"2024-04-25T14:05:34.560823+0000","last_clean":"2024-04-25T14:05:34.560823+0000","last_became_active":"2024-04-25T14:05:32.554652+0000","last_became_peered":"2024-04-25T14:05:32.554652+0000","last_unstale":"2024-04-25T14:05:34.560823+0000","last_undegraded":"2024-04-25T14:05:34.560823+0000","last_fullsized":"2024-04-25T14:05:34.560823+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:40.505936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.570677+0000","last_change":"2024-04-25T14:05:33.579664+0000","last_active":"2024-04-25T14:05:34.570677+0000","last_peered":"2024-04-25T14:05:34.570677+0000","last_clean":"2024-04-25T14:05:34.570677+0000","last_became_active":"2024-04-25T14:05:33.578924+0000","last_became_peered":"2024-04-25T14:05:33.578924+0000","last_unstale":"2024-04-25T14:05:34.570677+0000","last_undegraded":"2024-04-25T14:05:34.570677+0000","last_fullsized":"2024-04-25T14:05:34.570677+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:13.000228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.569956+0000","last_change":"2024-04-25T14:05:32.563774+0000","last_active":"2024-04-25T14:05:34.569956+0000","last_peered":"2024-04-25T14:05:34.569956+0000","last_clean":"2024-04-25T14:05:34.569956+0000","last_became_active":"2024-04-25T14:05:32.562877+0000","last_became_peered":"2024-04-25T14:05:32.562877+0000","last_unstale":"2024-04-25T14:05:34.569956+0000","last_undegraded":"2024-04-25T14:05:34.569956+0000","last_fullsized":"2024-04-25T14:05:34.569956+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:24.071915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039856+0000","last_change":"2024-04-25T14:05:33.579278+0000","last_active":"2024-04-25T14:05:35.039856+0000","last_peered":"2024-04-25T14:05:35.039856+0000","last_clean":"2024-04-25T14:05:35.039856+0000","last_became_active":"2024-04-25T14:05:33.578876+0000","last_became_peered":"2024-04-25T14:05:33.578876+0000","last_unstale":"2024-04-25T14:05:35.039856+0000","last_undegraded":"2024-04-25T14:05:35.039856+0000","last_fullsized":"2024-04-25T14:05:35.039856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:54.564632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039205+0000","last_change":"2024-04-25T14:05:32.553201+0000","last_active":"2024-04-25T14:05:35.039205+0000","last_peered":"2024-04-25T14:05:35.039205+0000","last_clean":"2024-04-25T14:05:35.039205+0000","last_became_active":"2024-04-25T14:05:32.552846+0000","last_became_peered":"2024-04-25T14:05:32.552846+0000","last_unstale":"2024-04-25T14:05:35.039205+0000","last_undegraded":"2024-04-25T14:05:35.039205+0000","last_fullsized":"2024-04-25T14:05:35.039205+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:13.772830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.563678+0000","last_change":"2024-04-25T14:05:33.561759+0000","last_active":"2024-04-25T14:05:34.563678+0000","last_peered":"2024-04-25T14:05:34.563678+0000","last_clean":"2024-04-25T14:05:34.563678+0000","last_became_active":"2024-04-25T14:05:33.561430+0000","last_became_peered":"2024-04-25T14:05:33.561430+0000","last_unstale":"2024-04-25T14:05:34.563678+0000","last_undegraded":"2024-04-25T14:05:34.563678+0000","last_fullsized":"2024-04-25T14:05:34.563678+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:16.126866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039401+0000","last_change":"2024-04-25T14:05:32.561754+0000","last_active":"2024-04-25T14:05:35.039401+0000","last_peered":"2024-04-25T14:05:35.039401+0000","last_clean":"2024-04-25T14:05:35.039401+0000","last_became_active":"2024-04-25T14:05:32.561297+0000","last_became_peered":"2024-04-25T14:05:32.561297+0000","last_unstale":"2024-04-25T14:05:35.039401+0000","last_undegraded":"2024-04-25T14:05:35.039401+0000","last_fullsized":"2024-04-25T14:05:35.039401+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:39.285483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039567+0000","last_change":"2024-04-25T14:05:33.574199+0000","last_active":"2024-04-25T14:05:35.039567+0000","last_peered":"2024-04-25T14:05:35.039567+0000","last_clean":"2024-04-25T14:05:35.039567+0000","last_became_active":"2024-04-25T14:05:33.572436+0000","last_became_peered":"2024-04-25T14:05:33.572436+0000","last_unstale":"2024-04-25T14:05:35.039567+0000","last_undegraded":"2024-04-25T14:05:35.039567+0000","last_fullsized":"2024-04-25T14:05:35.039567+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:59:41.873995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.574029+0000","last_change":"2024-04-25T14:05:32.562230+0000","last_active":"2024-04-25T14:05:34.574029+0000","last_peered":"2024-04-25T14:05:34.574029+0000","last_clean":"2024-04-25T14:05:34.574029+0000","last_became_active":"2024-04-25T14:05:32.561768+0000","last_became_peered":"2024-04-25T14:05:32.561768+0000","last_unstale":"2024-04-25T14:05:34.574029+0000","last_undegraded":"2024-04-25T14:05:34.574029+0000","last_fullsized":"2024-04-25T14:05:34.574029+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:40.505936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.561187+0000","last_change":"2024-04-25T14:05:33.573700+0000","last_active":"2024-04-25T14:05:34.561187+0000","last_peered":"2024-04-25T14:05:34.561187+0000","last_clean":"2024-04-25T14:05:34.561187+0000","last_became_active":"2024-04-25T14:05:33.573414+0000","last_became_peered":"2024-04-25T14:05:33.573414+0000","last_unstale":"2024-04-25T14:05:34.561187+0000","last_undegraded":"2024-04-25T14:05:34.561187+0000","last_fullsized":"2024-04-25T14:05:34.561187+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:30.202222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.570372+0000","last_change":"2024-04-25T14:05:32.558770+0000","last_active":"2024-04-25T14:05:34.570372+0000","last_peered":"2024-04-25T14:05:34.570372+0000","last_clean":"2024-04-25T14:05:34.570372+0000","last_became_active":"2024-04-25T14:05:32.557820+0000","last_became_peered":"2024-04-25T14:05:32.557820+0000","last_unstale":"2024-04-25T14:05:34.570372+0000","last_undegraded":"2024-04-25T14:05:34.570372+0000","last_fullsized":"2024-04-25T14:05:34.570372+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:10.274276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.570265+0000","last_change":"2024-04-25T14:05:33.561273+0000","last_active":"2024-04-25T14:05:34.570265+0000","last_peered":"2024-04-25T14:05:34.570265+0000","last_clean":"2024-04-25T14:05:34.570265+0000","last_became_active":"2024-04-25T14:05:33.560969+0000","last_became_peered":"2024-04-25T14:05:33.560969+0000","last_unstale":"2024-04-25T14:05:34.570265+0000","last_undegraded":"2024-04-25T14:05:34.570265+0000","last_fullsized":"2024-04-25T14:05:34.570265+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:44.696081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.561030+0000","last_change":"2024-04-25T14:05:32.555975+0000","last_active":"2024-04-25T14:05:34.561030+0000","last_peered":"2024-04-25T14:05:34.561030+0000","last_clean":"2024-04-25T14:05:34.561030+0000","last_became_active":"2024-04-25T14:05:32.555188+0000","last_became_peered":"2024-04-25T14:05:32.555188+0000","last_unstale":"2024-04-25T14:05:34.561030+0000","last_undegraded":"2024-04-25T14:05:34.561030+0000","last_fullsized":"2024-04-25T14:05:34.561030+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:10.274276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.563421+0000","last_change":"2024-04-25T14:05:33.570070+0000","last_active":"2024-04-25T14:05:34.563421+0000","last_peered":"2024-04-25T14:05:34.563421+0000","last_clean":"2024-04-25T14:05:34.563421+0000","last_became_active":"2024-04-25T14:05:33.569716+0000","last_became_peered":"2024-04-25T14:05:33.569716+0000","last_unstale":"2024-04-25T14:05:34.563421+0000","last_undegraded":"2024-04-25T14:05:34.563421+0000","last_fullsized":"2024-04-25T14:05:34.563421+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:42.654848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.560952+0000","last_change":"2024-04-25T14:05:32.555662+0000","last_active":"2024-04-25T14:05:34.560952+0000","last_peered":"2024-04-25T14:05:34.560952+0000","last_clean":"2024-04-25T14:05:34.560952+0000","last_became_active":"2024-04-25T14:05:32.555123+0000","last_became_peered":"2024-04-25T14:05:32.555123+0000","last_unstale":"2024-04-25T14:05:34.560952+0000","last_undegraded":"2024-04-25T14:05:34.560952+0000","last_fullsized":"2024-04-25T14:05:34.560952+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:24.071915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.570607+0000","last_change":"2024-04-25T14:05:33.573521+0000","last_active":"2024-04-25T14:05:34.570607+0000","last_peered":"2024-04-25T14:05:34.570607+0000","last_clean":"2024-04-25T14:05:34.570607+0000","last_became_active":"2024-04-25T14:05:33.573181+0000","last_became_peered":"2024-04-25T14:05:33.573181+0000","last_unstale":"2024-04-25T14:05:34.570607+0000","last_undegraded":"2024-04-25T14:05:34.570607+0000","last_fullsized":"2024-04-25T14:05:34.570607+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:33.035476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.561856+0000","last_change":"2024-04-25T14:05:32.543546+0000","last_active":"2024-04-25T14:05:34.561856+0000","last_peered":"2024-04-25T14:05:34.561856+0000","last_clean":"2024-04-25T14:05:34.561856+0000","last_became_active":"2024-04-25T14:05:32.543162+0000","last_became_peered":"2024-04-25T14:05:32.543162+0000","last_unstale":"2024-04-25T14:05:34.561856+0000","last_undegraded":"2024-04-25T14:05:34.561856+0000","last_fullsized":"2024-04-25T14:05:34.561856+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:14.012822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.561615+0000","last_change":"2024-04-25T14:05:33.575403+0000","last_active":"2024-04-25T14:05:34.561615+0000","last_peered":"2024-04-25T14:05:34.561615+0000","last_clean":"2024-04-25T14:05:34.561615+0000","last_became_active":"2024-04-25T14:05:33.575194+0000","last_became_peered":"2024-04-25T14:05:33.575194+0000","last_unstale":"2024-04-25T14:05:34.561615+0000","last_undegraded":"2024-04-25T14:05:34.561615+0000","last_fullsized":"2024-04-25T14:05:34.561615+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:44.696081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.563274+0000","last_change":"2024-04-25T14:05:32.564178+0000","last_active":"2024-04-25T14:05:34.563274+0000","last_peered":"2024-04-25T14:05:34.563274+0000","last_clean":"2024-04-25T14:05:34.563274+0000","last_became_active":"2024-04-25T14:05:32.563861+0000","last_became_peered":"2024-04-25T14:05:32.563861+0000","last_unstale":"2024-04-25T14:05:34.563274+0000","last_undegraded":"2024-04-25T14:05:34.563274+0000","last_fullsized":"2024-04-25T14:05:34.563274+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:40.505936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:35.039142+0000","last_change":"2024-04-25T14:05:33.563768+0000","last_active":"2024-04-25T14:05:35.039142+0000","last_peered":"2024-04-25T14:05:35.039142+0000","last_clean":"2024-04-25T14:05:35.039142+0000","last_became_active":"2024-04-25T14:05:33.563379+0000","last_became_peered":"2024-04-25T14:05:33.563379+0000","last_unstale":"2024-04-25T14:05:35.039142+0000","last_undegraded":"2024-04-25T14:05:35.039142+0000","last_fullsized":"2024-04-25T14:05:35.039142+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:11.087435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"46'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.570733+0000","last_change":"2024-04-25T14:05:32.559617+0000","last_active":"2024-04-25T14:05:34.570733+0000","last_peered":"2024-04-25T14:05:34.570733+0000","last_clean":"2024-04-25T14:05:34.570733+0000","last_became_active":"2024-04-25T14:05:32.558277+0000","last_became_peered":"2024-04-25T14:05:32.558277+0000","last_unstale":"2024-04-25T14:05:34.570733+0000","last_undegraded":"2024-04-25T14:05:34.570733+0000","last_fullsized":"2024-04-25T14:05:34.570733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:38.715145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"39'79","reported_seq":183,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T14:05:34.573461+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:05:34.573461+0000","last_peered":"2024-04-25T14:05:34.573461+0000","last_clean":"2024-04-25T14:05:34.573461+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:05:34.573461+0000","last_undegraded":"2024-04-25T14:05:34.573461+0000","last_fullsized":"2024-04-25T14:05:34.573461+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343696,"statfs":{"total":96636764160,"available":96607944704,"internally_reserved":0,"allocated":1146880,"data_stored":834999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":552960,"data_stored":245159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":544768,"data_stored":244507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":548864,"data_stored":245089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":561152,"data_stored":246141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":246119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":552960,"data_stored":244667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28148,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343692,"statfs":{"total":96636764160,"available":96607940608,"internally_reserved":0,"allocated":1146880,"data_stored":834931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:05:42.636 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T14:05:42.636 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T14:05:42.636 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T14:05:42.636 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T14:05:42.636 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:05:42.637 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T14:05:42.637 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T14:05:42.637 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T14:05:42.637 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:05:42.637 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T14:05:42.637 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T14:05:42.637 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T14:05:42.637 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:05:42.637 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T14:05:42.638 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T14:05:42.638 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T14:05:42.638 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:05:42.653 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T14:05:42.653 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T14:05:42.654 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T14:05:42.654 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T14:05:42.689 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T14:05:42.689 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T14:05:42.698 INFO:tasks.ceph.mds.b:Started 2024-04-25T14:05:42.698 DEBUG:teuthology.parallel:result is None 2024-04-25T14:05:42.700 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T14:05:42.700 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T14:05:42.700 INFO:tasks.ceph.mds.d:Started 2024-04-25T14:05:42.700 DEBUG:teuthology.parallel:result is None 2024-04-25T14:05:42.746 INFO:tasks.ceph.mds.a:Started 2024-04-25T14:05:42.746 DEBUG:teuthology.parallel:result is None 2024-04-25T14:05:42.748 INFO:tasks.ceph.mds.c:Started 2024-04-25T14:05:42.748 DEBUG:teuthology.parallel:result is None 2024-04-25T14:05:42.748 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:05:42.804 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2024-04-25T14:05:42.804 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2024-04-25T14:05:42.850 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-25T14:05:42.850 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-04-25T14:05:42.881 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-25T14:05:42.880+0000 7fea26a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:05:42.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.916+0000 7feca9800640 1 Processor -- start 2024-04-25T14:05:42.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.916+0000 7feca9800640 1 -- start start 2024-04-25T14:05:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.916+0000 7feca9800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feca4072630 0x7feca4072a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.916+0000 7feca9800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7feca406e340 0x7feca406e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.916+0000 7feca9800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feca4072f80 con 0x7feca4163980 2024-04-25T14:05:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.916+0000 7feca9800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7feca41648e0 con 0x7feca406e340 2024-04-25T14:05:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.916+0000 7feca9800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7feca40d1520 con 0x7feca4072630 2024-04-25T14:05:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.916+0000 7feca2a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feca4072630 0x7feca4072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.916+0000 7feca3400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7feca406e340 0x7feca406e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.916+0000 7feca2a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feca4072630 0x7feca4072a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:35212/0 (socket says 172.21.15.86:35212) 2024-04-25T14:05:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.916+0000 7feca2a00640 1 -- 172.21.15.86:0/1214644276 learned_addr learned my addr 172.21.15.86:0/1214644276 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.916+0000 7feca2000640 1 --1- 172.21.15.86:0/1214644276 >> v1:172.21.15.155:6789/0 conn(0x7feca4163980 0x7feca4163d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:33612/0 (socket says 172.21.15.86:33612) 2024-04-25T14:05:42.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.920+0000 7feca2a00640 1 -- 172.21.15.86:0/1214644276 >> v2:172.21.15.155:3300/0 conn(0x7feca406e340 msgr2=0x7feca406e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:42.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.920+0000 7feca2a00640 1 --2- 172.21.15.86:0/1214644276 >> v2:172.21.15.155:3300/0 conn(0x7feca406e340 0x7feca406e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:42.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.920+0000 7feca2a00640 1 -- 172.21.15.86:0/1214644276 >> v1:172.21.15.155:6789/0 conn(0x7feca4163980 legacy=0x7feca4163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:42.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.920+0000 7feca2a00640 1 -- 172.21.15.86:0/1214644276 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca40d23e0 con 0x7feca4072630 2024-04-25T14:05:42.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.920+0000 7feca3400640 1 --2- 172.21.15.86:0/1214644276 >> v2:172.21.15.155:3300/0 conn(0x7feca406e340 0x7feca406e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:42.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.920+0000 7feca2a00640 1 --2- 172.21.15.86:0/1214644276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feca4072630 0x7feca4072a40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fec94009870 tx=0x7fec9401c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23cc6cfef4dea6bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:42.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.920+0000 7feca1600640 1 -- 172.21.15.86:0/1214644276 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec940b0070 con 0x7feca4072630 2024-04-25T14:05:42.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.920+0000 7feca1600640 1 -- 172.21.15.86:0/1214644276 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec940ab600 con 0x7feca4072630 2024-04-25T14:05:42.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.920+0000 7feca1600640 1 -- 172.21.15.86:0/1214644276 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec940ab9d0 con 0x7feca4072630 2024-04-25T14:05:42.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.924+0000 7feca9800640 1 -- 172.21.15.86:0/1214644276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feca4072630 msgr2=0x7feca4072a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:42.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.924+0000 7feca9800640 1 --2- 172.21.15.86:0/1214644276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feca4072630 0x7feca4072a40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fec94009870 tx=0x7fec9401c440 comp rx=0 tx=0).stop 2024-04-25T14:05:42.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.924+0000 7feca9800640 1 -- 172.21.15.86:0/1214644276 shutdown_connections 2024-04-25T14:05:42.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.924+0000 7feca9800640 1 --2- 172.21.15.86:0/1214644276 >> v2:172.21.15.155:3300/0 conn(0x7feca406e340 0x7feca406e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:42.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.924+0000 7feca9800640 1 --2- 172.21.15.86:0/1214644276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feca4072630 0x7feca4072a40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:42.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.924+0000 7feca9800640 1 -- 172.21.15.86:0/1214644276 >> 172.21.15.86:0/1214644276 conn(0x7feca40d3580 msgr2=0x7feca40d3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:42.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.924+0000 7feca9800640 1 -- 172.21.15.86:0/1214644276 shutdown_connections 2024-04-25T14:05:42.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.924+0000 7feca9800640 1 -- 172.21.15.86:0/1214644276 wait complete. 2024-04-25T14:05:42.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.924+0000 7feca9800640 1 Processor -- start 2024-04-25T14:05:42.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.924+0000 7feca9800640 1 -- start start 2024-04-25T14:05:42.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.924+0000 7feca9800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feca406e340 0x7feca41569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:42.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.924+0000 7feca9800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feca4072630 0x7feca4158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:42.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.924+0000 7feca9800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feca4163980 0x7feca4158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:42.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.924+0000 7feca9800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feca416d530 con 0x7feca4072630 2024-04-25T14:05:42.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.924+0000 7feca9800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7feca406f0a0 con 0x7feca4163980 2024-04-25T14:05:42.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.924+0000 7feca9800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7feca416d6b0 con 0x7feca406e340 2024-04-25T14:05:42.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.928+0000 7feca2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feca4072630 0x7feca4158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:42.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.928+0000 7feca2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feca4072630 0x7feca4158560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47746/0 (socket says 172.21.15.86:47746) 2024-04-25T14:05:42.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.928+0000 7feca2000640 1 -- 172.21.15.86:0/4167961592 learned_addr learned my addr 172.21.15.86:0/4167961592 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:42.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.928+0000 7feca3400640 1 --2- 172.21.15.86:0/4167961592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feca4163980 0x7feca4158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:42.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.928+0000 7feca2a00640 1 --2- 172.21.15.86:0/4167961592 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feca406e340 0x7feca41569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:42.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.928+0000 7feca2000640 1 -- 172.21.15.86:0/4167961592 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feca406e340 msgr2=0x7feca41569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:42.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.928+0000 7feca2000640 1 --2- 172.21.15.86:0/4167961592 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feca406e340 0x7feca41569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:42.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.928+0000 7feca2000640 1 -- 172.21.15.86:0/4167961592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feca4163980 msgr2=0x7feca4158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:42.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.928+0000 7feca2000640 1 --2- 172.21.15.86:0/4167961592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feca4163980 0x7feca4158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:42.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.928+0000 7feca2000640 1 -- 172.21.15.86:0/4167961592 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca40ceef0 con 0x7feca4072630 2024-04-25T14:05:42.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.928+0000 7feca2000640 1 --2- 172.21.15.86:0/4167961592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feca4072630 0x7feca4158560 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fec9000c3f0 tx=0x7fec9000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:42.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.928+0000 7fec9aa00640 1 -- 172.21.15.86:0/4167961592 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec900074e0 con 0x7feca4072630 2024-04-25T14:05:42.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.928+0000 7fec9aa00640 1 -- 172.21.15.86:0/4167961592 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec900076a0 con 0x7feca4072630 2024-04-25T14:05:42.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.928+0000 7fec9aa00640 1 -- 172.21.15.86:0/4167961592 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90004a30 con 0x7feca4072630 2024-04-25T14:05:42.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.928+0000 7feca9800640 1 -- 172.21.15.86:0/4167961592 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca40c7370 con 0x7feca4072630 2024-04-25T14:05:42.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.928+0000 7feca9800640 1 -- 172.21.15.86:0/4167961592 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca415e4f0 con 0x7feca4072630 2024-04-25T14:05:42.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.928+0000 7feca9800640 1 -- 172.21.15.86:0/4167961592 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feca4072170 con 0x7feca4072630 2024-04-25T14:05:42.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.928+0000 7fec9aa00640 1 -- 172.21.15.86:0/4167961592 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7fec9001a020 con 0x7feca4072630 2024-04-25T14:05:42.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.932+0000 7fec9aa00640 1 --2- 172.21.15.86:0/4167961592 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fec8406f5a0 0x7fec84071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:42.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.932+0000 7fec9aa00640 1 -- 172.21.15.86:0/4167961592 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6519+0+0 (secure 0 0 0) 0x7fec9008ed20 con 0x7feca4072630 2024-04-25T14:05:42.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.932+0000 7feca2a00640 1 --2- 172.21.15.86:0/4167961592 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fec8406f5a0 0x7fec84071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:42.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.932+0000 7feca2a00640 1 --2- 172.21.15.86:0/4167961592 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fec8406f5a0 0x7fec84071ab0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fec940ad000 tx=0x7fec94008040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:42.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:42.936+0000 7fec9aa00640 1 -- 172.21.15.86:0/4167961592 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7feca40ceef0 con 0x7feca4072630 2024-04-25T14:05:43.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:43.124+0000 7feca9800640 1 -- 172.21.15.86:0/4167961592 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feca416aac0 con 0x7feca4072630 2024-04-25T14:05:43.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:43.124+0000 7fec9aa00640 1 -- 172.21.15.86:0/4167961592 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5175 (secure 0 0 0) 0x7feca4072f80 con 0x7feca4072630 2024-04-25T14:05:43.127 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:43.127 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5315,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3312926518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":3312926518},{"type":"v1","addr":"172.21.15.155:6835","nonce":3312926518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5776,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/898730495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":898730495},{"type":"v1","addr":"172.21.15.155:6834","nonce":898730495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5794,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2418778265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2418778265},{"type":"v1","addr":"172.21.15.86:6837","nonce":2418778265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:32.946320+0000","modified":"2024-04-25T14:05:42.892383+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5874},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5874":{"gid":5874,"name":"a","rank":0,"incarnation":29,"state":"up:replay","state_seq":1,"addr":"172.21.15.86:6836/1732615622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1732615622},{"type":"v1","addr":"172.21.15.86:6836","nonce":1732615622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"fs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-04-25T14:05:43.127 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-25T14:05:43.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:43.128+0000 7feca9800640 1 -- 172.21.15.86:0/4167961592 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fec8406f5a0 msgr2=0x7fec84071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:43.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:43.128+0000 7feca9800640 1 --2- 172.21.15.86:0/4167961592 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fec8406f5a0 0x7fec84071ab0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fec940ad000 tx=0x7fec94008040 comp rx=0 tx=0).stop 2024-04-25T14:05:43.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:43.128+0000 7feca9800640 1 -- 172.21.15.86:0/4167961592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feca4072630 msgr2=0x7feca4158560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:43.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:43.128+0000 7feca9800640 1 --2- 172.21.15.86:0/4167961592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feca4072630 0x7feca4158560 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fec9000c3f0 tx=0x7fec9000c7f0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:43.128+0000 7feca9800640 1 -- 172.21.15.86:0/4167961592 shutdown_connections 2024-04-25T14:05:43.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:43.128+0000 7feca9800640 1 --2- 172.21.15.86:0/4167961592 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fec8406f5a0 0x7fec84071ab0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:43.128+0000 7feca9800640 1 --2- 172.21.15.86:0/4167961592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feca4163980 0x7feca4158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:43.128+0000 7feca9800640 1 --2- 172.21.15.86:0/4167961592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feca4072630 0x7feca4158560 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:43.128+0000 7feca9800640 1 --2- 172.21.15.86:0/4167961592 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feca406e340 0x7feca41569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:43.128+0000 7feca9800640 1 -- 172.21.15.86:0/4167961592 >> 172.21.15.86:0/4167961592 conn(0x7feca40d3580 msgr2=0x7feca4163d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:43.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:43.128+0000 7feca9800640 1 -- 172.21.15.86:0/4167961592 shutdown_connections 2024-04-25T14:05:43.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:43.128+0000 7feca9800640 1 -- 172.21.15.86:0/4167961592 wait complete. 2024-04-25T14:05:43.551 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:05:32.946320+0000', 'modified': '2024-04-25T14:05:42.892383+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 50, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5874}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5874': {'gid': 5874, 'name': 'a', 'rank': 0, 'incarnation': 29, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.86:6836/1732615622', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6834', 'nonce': 1732615622}, {'type': 'v1', 'addr': '172.21.15.86:6836', 'nonce': 1732615622}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'fs1', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T14:05:43.551 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5874:up:replay 2024-04-25T14:05:43.865 INFO:tasks.ceph.mgr.y.smithi086.stderr:Exception in thread Thread-8: 2024-04-25T14:05:43.865 INFO:tasks.ceph.mgr.y.smithi086.stderr:Traceback (most recent call last): 2024-04-25T14:05:43.865 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner 2024-04-25T14:05:43.865 INFO:tasks.ceph.mgr.y.smithi086.stderr: self.run() 2024-04-25T14:05:43.865 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/lib/python3.10/threading.py", line 1378, in run 2024-04-25T14:05:43.865 INFO:tasks.ceph.mgr.y.smithi086.stderr: self.function(*self.args, **self.kwargs) 2024-04-25T14:05:43.865 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/share/ceph/mgr/stats/fs/perf_stats.py", line 222, in re_register_queries 2024-04-25T14:05:43.865 INFO:tasks.ceph.mgr.y.smithi086.stderr: if self.mx_last_updated >= ua_last_updated: 2024-04-25T14:05:43.865 INFO:tasks.ceph.mgr.y.smithi086.stderr:AttributeError: 'FSPerfStats' object has no attribute 'mx_last_updated' 2024-04-25T14:05:44.552 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:05:44.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.708+0000 7fed9ce00640 1 Processor -- start 2024-04-25T14:05:44.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.708+0000 7fed9ce00640 1 -- start start 2024-04-25T14:05:44.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.708+0000 7fed9ce00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fed980ccab0 0x7fed980d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.708+0000 7fed9ce00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed980b79c0 0x7fed980d2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.708+0000 7fed9ce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fed980cf9d0 0x7fed981617f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.708+0000 7fed9ce00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fed980b2620 con 0x7fed980cf9d0 2024-04-25T14:05:44.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.708+0000 7fed9ce00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fed980b3bd0 con 0x7fed980ccab0 2024-04-25T14:05:44.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.708+0000 7fed9ce00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fed980b1cf0 con 0x7fed980b79c0 2024-04-25T14:05:44.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.708+0000 7fed96800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fed980cf9d0 0x7fed981617f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.708+0000 7fed8fe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed980b79c0 0x7fed980d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.708+0000 7fed95e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fed980ccab0 0x7fed980d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.708+0000 7fed8fe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed980b79c0 0x7fed980d2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:35238/0 (socket says 172.21.15.86:35238) 2024-04-25T14:05:44.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.708+0000 7fed96800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fed980cf9d0 0x7fed981617f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47760/0 (socket says 172.21.15.86:47760) 2024-04-25T14:05:44.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.708+0000 7fed8fe00640 1 -- 172.21.15.86:0/2406460657 learned_addr learned my addr 172.21.15.86:0/2406460657 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:44.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.712+0000 7fed8fe00640 1 -- 172.21.15.86:0/2406460657 >> v2:172.21.15.155:3300/0 conn(0x7fed980ccab0 msgr2=0x7fed980d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:44.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.712+0000 7fed8fe00640 1 --2- 172.21.15.86:0/2406460657 >> v2:172.21.15.155:3300/0 conn(0x7fed980ccab0 0x7fed980d23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.712+0000 7fed8fe00640 1 -- 172.21.15.86:0/2406460657 >> v2:172.21.15.86:3300/0 conn(0x7fed980cf9d0 msgr2=0x7fed981617f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:44.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.712+0000 7fed8fe00640 1 --2- 172.21.15.86:0/2406460657 >> v2:172.21.15.86:3300/0 conn(0x7fed980cf9d0 0x7fed981617f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.712+0000 7fed8fe00640 1 -- 172.21.15.86:0/2406460657 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed980b1fd0 con 0x7fed980b79c0 2024-04-25T14:05:44.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.712+0000 7fed95e00640 1 --2- 172.21.15.86:0/2406460657 >> v2:172.21.15.155:3300/0 conn(0x7fed980ccab0 0x7fed980d23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:44.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.712+0000 7fed96800640 1 --2- 172.21.15.86:0/2406460657 >> v2:172.21.15.86:3300/0 conn(0x7fed980cf9d0 0x7fed981617f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:44.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.712+0000 7fed8fe00640 1 --2- 172.21.15.86:0/2406460657 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed980b79c0 0x7fed980d2910 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fed98073230 tx=0x7fed8001c260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2a074ba3e20f86c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:44.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.712+0000 7fed95400640 1 -- 172.21.15.86:0/2406460657 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed800b0070 con 0x7fed980b79c0 2024-04-25T14:05:44.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.712+0000 7fed95400640 1 -- 172.21.15.86:0/2406460657 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed800ab600 con 0x7fed980b79c0 2024-04-25T14:05:44.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.712+0000 7fed95400640 1 -- 172.21.15.86:0/2406460657 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed800ab9d0 con 0x7fed980b79c0 2024-04-25T14:05:44.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.716+0000 7fed9ce00640 1 -- 172.21.15.86:0/2406460657 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed980b79c0 msgr2=0x7fed980d2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.716+0000 7fed9ce00640 1 --2- 172.21.15.86:0/2406460657 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed980b79c0 0x7fed980d2910 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fed98073230 tx=0x7fed8001c260 comp rx=0 tx=0).stop 2024-04-25T14:05:44.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.716+0000 7fed9ce00640 1 -- 172.21.15.86:0/2406460657 shutdown_connections 2024-04-25T14:05:44.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.716+0000 7fed9ce00640 1 --2- 172.21.15.86:0/2406460657 >> v2:172.21.15.86:3300/0 conn(0x7fed980cf9d0 0x7fed981617f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.716+0000 7fed9ce00640 1 --2- 172.21.15.86:0/2406460657 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed980b79c0 0x7fed980d2910 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.716+0000 7fed9ce00640 1 --2- 172.21.15.86:0/2406460657 >> v2:172.21.15.155:3300/0 conn(0x7fed980ccab0 0x7fed980d23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.716+0000 7fed9ce00640 1 -- 172.21.15.86:0/2406460657 >> 172.21.15.86:0/2406460657 conn(0x7fed9809ba30 msgr2=0x7fed980b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:44.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.716+0000 7fed9ce00640 1 -- 172.21.15.86:0/2406460657 shutdown_connections 2024-04-25T14:05:44.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.716+0000 7fed9ce00640 1 -- 172.21.15.86:0/2406460657 wait complete. 2024-04-25T14:05:44.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.716+0000 7fed9ce00640 1 Processor -- start 2024-04-25T14:05:44.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.716+0000 7fed9ce00640 1 -- start start 2024-04-25T14:05:44.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.720+0000 7fed9ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed980b79c0 0x7fed9815cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.720+0000 7fed9ce00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed980ccab0 0x7fed9809c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.720+0000 7fed9ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed980cf9d0 0x7fed9809c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.720+0000 7fed9ce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed980ce550 con 0x7fed980b79c0 2024-04-25T14:05:44.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.720+0000 7fed9ce00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fed980ce3d0 con 0x7fed980cf9d0 2024-04-25T14:05:44.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.720+0000 7fed9ce00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fed980ce6d0 con 0x7fed980ccab0 2024-04-25T14:05:44.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.720+0000 7fed95e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed980b79c0 0x7fed9815cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.720+0000 7fed95e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed980b79c0 0x7fed9815cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47774/0 (socket says 172.21.15.86:47774) 2024-04-25T14:05:44.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.720+0000 7fed95e00640 1 -- 172.21.15.86:0/1229782317 learned_addr learned my addr 172.21.15.86:0/1229782317 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:44.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.720+0000 7fed8fe00640 1 --2- 172.21.15.86:0/1229782317 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed980ccab0 0x7fed9809c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.720+0000 7fed96800640 1 --2- 172.21.15.86:0/1229782317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed980cf9d0 0x7fed9809c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.720+0000 7fed95e00640 1 -- 172.21.15.86:0/1229782317 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed980ccab0 msgr2=0x7fed9809c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.720+0000 7fed95e00640 1 --2- 172.21.15.86:0/1229782317 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed980ccab0 0x7fed9809c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.720+0000 7fed95e00640 1 -- 172.21.15.86:0/1229782317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed980cf9d0 msgr2=0x7fed9809c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.720+0000 7fed95e00640 1 --2- 172.21.15.86:0/1229782317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed980cf9d0 0x7fed9809c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.720+0000 7fed95e00640 1 -- 172.21.15.86:0/1229782317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed98161070 con 0x7fed980b79c0 2024-04-25T14:05:44.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.720+0000 7fed95e00640 1 --2- 172.21.15.86:0/1229782317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed980b79c0 0x7fed9815cfe0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fed8800e910 tx=0x7fed8800edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:44.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.720+0000 7fed8e000640 1 -- 172.21.15.86:0/1229782317 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8800ca00 con 0x7fed980b79c0 2024-04-25T14:05:44.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.720+0000 7fed8e000640 1 -- 172.21.15.86:0/1229782317 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed8800cba0 con 0x7fed980b79c0 2024-04-25T14:05:44.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.720+0000 7fed9ce00640 1 -- 172.21.15.86:0/1229782317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed9815f2e0 con 0x7fed980b79c0 2024-04-25T14:05:44.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.720+0000 7fed8e000640 1 -- 172.21.15.86:0/1229782317 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed880059a0 con 0x7fed980b79c0 2024-04-25T14:05:44.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.720+0000 7fed9ce00640 1 -- 172.21.15.86:0/1229782317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed9815f8a0 con 0x7fed980b79c0 2024-04-25T14:05:44.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.720+0000 7fed9ce00640 1 -- 172.21.15.86:0/1229782317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed980a04c0 con 0x7fed980b79c0 2024-04-25T14:05:44.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.720+0000 7fed8e000640 1 -- 172.21.15.86:0/1229782317 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7fed88005b40 con 0x7fed980b79c0 2024-04-25T14:05:44.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.724+0000 7fed8e000640 1 --2- 172.21.15.86:0/1229782317 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fed7406f5a0 0x7fed74071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.724+0000 7fed8e000640 1 -- 172.21.15.86:0/1229782317 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6519+0+0 (secure 0 0 0) 0x7fed8808fa10 con 0x7fed980b79c0 2024-04-25T14:05:44.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.724+0000 7fed8fe00640 1 --2- 172.21.15.86:0/1229782317 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fed7406f5a0 0x7fed74071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.724+0000 7fed8fe00640 1 --2- 172.21.15.86:0/1229782317 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fed7406f5a0 0x7fed74071ab0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fed800ad000 tx=0x7fed80008040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:44.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.728+0000 7fed8e000640 1 -- 172.21.15.86:0/1229782317 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fed88017020 con 0x7fed980b79c0 2024-04-25T14:05:44.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.920+0000 7fed9ce00640 1 -- 172.21.15.86:0/1229782317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed980d2f90 con 0x7fed980b79c0 2024-04-25T14:05:44.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.920+0000 7fed8e000640 1 -- 172.21.15.86:0/1229782317 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+5175 (secure 0 0 0) 0x7fed8805be60 con 0x7fed980b79c0 2024-04-25T14:05:44.923 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:44.923 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":31,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5315,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3312926518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":3312926518},{"type":"v1","addr":"172.21.15.155:6835","nonce":3312926518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5776,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/898730495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":898730495},{"type":"v1","addr":"172.21.15.155:6834","nonce":898730495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5794,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2418778265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2418778265},{"type":"v1","addr":"172.21.15.86:6837","nonce":2418778265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:32.946320+0000","modified":"2024-04-25T14:05:43.902545+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5874},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5874":{"gid":5874,"name":"a","rank":0,"incarnation":29,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.86:6836/1732615622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1732615622},{"type":"v1","addr":"172.21.15.86:6836","nonce":1732615622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"fs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-04-25T14:05:44.923 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 31 2024-04-25T14:05:44.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.924+0000 7fed9ce00640 1 -- 172.21.15.86:0/1229782317 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fed7406f5a0 msgr2=0x7fed74071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.924+0000 7fed9ce00640 1 --2- 172.21.15.86:0/1229782317 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fed7406f5a0 0x7fed74071ab0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fed800ad000 tx=0x7fed80008040 comp rx=0 tx=0).stop 2024-04-25T14:05:44.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.924+0000 7fed9ce00640 1 -- 172.21.15.86:0/1229782317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed980b79c0 msgr2=0x7fed9815cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.924+0000 7fed9ce00640 1 --2- 172.21.15.86:0/1229782317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed980b79c0 0x7fed9815cfe0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fed8800e910 tx=0x7fed8800edd0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.924+0000 7fed9ce00640 1 -- 172.21.15.86:0/1229782317 shutdown_connections 2024-04-25T14:05:44.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.924+0000 7fed9ce00640 1 --2- 172.21.15.86:0/1229782317 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fed7406f5a0 0x7fed74071ab0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.924+0000 7fed9ce00640 1 --2- 172.21.15.86:0/1229782317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed980cf9d0 0x7fed9809c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.924+0000 7fed9ce00640 1 --2- 172.21.15.86:0/1229782317 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed980ccab0 0x7fed9809c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.924+0000 7fed9ce00640 1 --2- 172.21.15.86:0/1229782317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed980b79c0 0x7fed9815cfe0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.924+0000 7fed9ce00640 1 -- 172.21.15.86:0/1229782317 >> 172.21.15.86:0/1229782317 conn(0x7fed9809ba30 msgr2=0x7fed980d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:44.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.924+0000 7fed9ce00640 1 -- 172.21.15.86:0/1229782317 shutdown_connections 2024-04-25T14:05:44.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:44.924+0000 7fed9ce00640 1 -- 172.21.15.86:0/1229782317 wait complete. 2024-04-25T14:05:45.348 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 31, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:05:32.946320+0000', 'modified': '2024-04-25T14:05:43.902545+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 50, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5874}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5874': {'gid': 5874, 'name': 'a', 'rank': 0, 'incarnation': 29, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.86:6836/1732615622', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6834', 'nonce': 1732615622}, {'type': 'v1', 'addr': '172.21.15.86:6836', 'nonce': 1732615622}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'fs1', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T14:05:45.348 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5874:up:rejoin 2024-04-25T14:05:46.349 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:05:46.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.504+0000 7f2513000640 1 Processor -- start 2024-04-25T14:05:46.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.504+0000 7f2513000640 1 -- start start 2024-04-25T14:05:46.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.508+0000 7f2513000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f250c160f40 0x7f250c0d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.508+0000 7f2513000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f250c0ce380 con 0x7f250c0d2f40 2024-04-25T14:05:46.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.508+0000 7f2513000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f250c070ff0 con 0x7f250c0700a0 2024-04-25T14:05:46.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.508+0000 7f2513000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f250c0712d0 con 0x7f250c160f40 2024-04-25T14:05:46.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.508+0000 7f2511200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f250c0d2f40 0x7f250c06f990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41256/0 (socket says 172.21.15.86:41256) 2024-04-25T14:05:46.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.508+0000 7f2511200640 1 -- 172.21.15.86:0/3898408140 learned_addr learned my addr 172.21.15.86:0/3898408140 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:46.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.508+0000 7f2511c00640 1 --2- 172.21.15.86:0/3898408140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f250c160f40 0x7f250c0d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.508+0000 7f250be00640 1 -- 172.21.15.86:0/3898408140 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1131770783 0 0) 0x7f250c0ce380 con 0x7f250c0d2f40 2024-04-25T14:05:46.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.508+0000 7f250be00640 1 -- 172.21.15.86:0/3898408140 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24f4003530 con 0x7f250c0d2f40 2024-04-25T14:05:46.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.508+0000 7f250be00640 1 -- 172.21.15.86:0/3898408140 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1595547284 0 0) 0x7f250c070ff0 con 0x7f250c0700a0 2024-04-25T14:05:46.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.508+0000 7f250be00640 1 -- 172.21.15.86:0/3898408140 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f250c0ce380 con 0x7f250c0700a0 2024-04-25T14:05:46.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.508+0000 7f2511c00640 1 -- 172.21.15.86:0/3898408140 >> v1:172.21.15.155:6789/0 conn(0x7f250c0700a0 legacy=0x7f250c15e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.508+0000 7f2511c00640 1 -- 172.21.15.86:0/3898408140 >> v1:172.21.15.86:6789/0 conn(0x7f250c0d2f40 legacy=0x7f250c06f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.508+0000 7f250be00640 1 -- 172.21.15.86:0/3898408140 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3832806402 0 0) 0x7f24f4003530 con 0x7f250c0d2f40 2024-04-25T14:05:46.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.508+0000 7f2511c00640 1 -- 172.21.15.86:0/3898408140 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250c0715b0 con 0x7f250c160f40 2024-04-25T14:05:46.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.512+0000 7f2511c00640 1 --2- 172.21.15.86:0/3898408140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f250c160f40 0x7f250c0d2a00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f25000097f0 tx=0x7f250001c340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a7993c8fdc541f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:46.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.512+0000 7f250be00640 1 -- 172.21.15.86:0/3898408140 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25000b0070 con 0x7f250c160f40 2024-04-25T14:05:46.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.512+0000 7f250be00640 1 -- 172.21.15.86:0/3898408140 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25000ab400 con 0x7f250c160f40 2024-04-25T14:05:46.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.512+0000 7f250be00640 1 -- 172.21.15.86:0/3898408140 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8001680 con 0x7f250c160f40 2024-04-25T14:05:46.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.512+0000 7f2513000640 1 -- 172.21.15.86:0/3898408140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f250c160f40 msgr2=0x7f250c0d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.512+0000 7f2513000640 1 --2- 172.21.15.86:0/3898408140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f250c160f40 0x7f250c0d2a00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f25000097f0 tx=0x7f250001c340 comp rx=0 tx=0).stop 2024-04-25T14:05:46.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.512+0000 7f2513000640 1 -- 172.21.15.86:0/3898408140 shutdown_connections 2024-04-25T14:05:46.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.512+0000 7f2513000640 1 --2- 172.21.15.86:0/3898408140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f250c160f40 0x7f250c0d2a00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.512+0000 7f2513000640 1 -- 172.21.15.86:0/3898408140 >> 172.21.15.86:0/3898408140 conn(0x7f250c0cfe10 msgr2=0x7f250c0d0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:46.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.512+0000 7f2513000640 1 -- 172.21.15.86:0/3898408140 shutdown_connections 2024-04-25T14:05:46.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.512+0000 7f2513000640 1 -- 172.21.15.86:0/3898408140 wait complete. 2024-04-25T14:05:46.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.516+0000 7f2513000640 1 Processor -- start 2024-04-25T14:05:46.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.516+0000 7f2513000640 1 -- start start 2024-04-25T14:05:46.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.516+0000 7f2513000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f250c0d2f40 0x7f250c13dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.516+0000 7f2513000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f250c160f40 0x7f250c150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.516+0000 7f2513000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f250c156b90 0x7f250c156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.516+0000 7f2513000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f250c16ba20 con 0x7f250c156b90 2024-04-25T14:05:46.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.516+0000 7f2513000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f250c16b8a0 con 0x7f250c0d2f40 2024-04-25T14:05:46.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.516+0000 7f2513000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f250c16bba0 con 0x7f250c160f40 2024-04-25T14:05:46.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.516+0000 7f2512600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f250c156b90 0x7f250c156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.516+0000 7f2512600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f250c156b90 0x7f250c156fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47790/0 (socket says 172.21.15.86:47790) 2024-04-25T14:05:46.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.516+0000 7f2512600640 1 -- 172.21.15.86:0/822922060 learned_addr learned my addr 172.21.15.86:0/822922060 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:46.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.516+0000 7f2511c00640 1 --2- 172.21.15.86:0/822922060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f250c0d2f40 0x7f250c13dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.516+0000 7f2511200640 1 --2- 172.21.15.86:0/822922060 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f250c160f40 0x7f250c150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.516+0000 7f2512600640 1 -- 172.21.15.86:0/822922060 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f250c160f40 msgr2=0x7f250c150e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.516+0000 7f2512600640 1 --2- 172.21.15.86:0/822922060 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f250c160f40 0x7f250c150e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.516+0000 7f2512600640 1 -- 172.21.15.86:0/822922060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f250c0d2f40 msgr2=0x7f250c13dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.516+0000 7f2512600640 1 --2- 172.21.15.86:0/822922060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f250c0d2f40 0x7f250c13dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.516+0000 7f2512600640 1 -- 172.21.15.86:0/822922060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250c1574e0 con 0x7f250c156b90 2024-04-25T14:05:46.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.516+0000 7f2512600640 1 --2- 172.21.15.86:0/822922060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f250c156b90 0x7f250c156fa0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f250400c6f0 tx=0x7f250400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:46.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.516+0000 7f2509600640 1 -- 172.21.15.86:0/822922060 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2504003980 con 0x7f250c156b90 2024-04-25T14:05:46.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.516+0000 7f2509600640 1 -- 172.21.15.86:0/822922060 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2504003b20 con 0x7f250c156b90 2024-04-25T14:05:46.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.516+0000 7f2509600640 1 -- 172.21.15.86:0/822922060 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250400a850 con 0x7f250c156b90 2024-04-25T14:05:46.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.516+0000 7f2513000640 1 -- 172.21.15.86:0/822922060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f250c151a50 con 0x7f250c156b90 2024-04-25T14:05:46.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.516+0000 7f2513000640 1 -- 172.21.15.86:0/822922060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f250c06db00 con 0x7f250c156b90 2024-04-25T14:05:46.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.520+0000 7f2513000640 1 -- 172.21.15.86:0/822922060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f250c076b80 con 0x7f250c156b90 2024-04-25T14:05:46.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.520+0000 7f2509600640 1 -- 172.21.15.86:0/822922060 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f250400a9f0 con 0x7f250c156b90 2024-04-25T14:05:46.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.520+0000 7f2509600640 1 --2- 172.21.15.86:0/822922060 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f24ec06f5a0 0x7f24ec071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.520+0000 7f2509600640 1 -- 172.21.15.86:0/822922060 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6519+0+0 (secure 0 0 0) 0x7f2504090140 con 0x7f250c156b90 2024-04-25T14:05:46.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.520+0000 7f2511c00640 1 --2- 172.21.15.86:0/822922060 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f24ec06f5a0 0x7f24ec071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.520+0000 7f2511c00640 1 --2- 172.21.15.86:0/822922060 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f24ec06f5a0 0x7f24ec071ab0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f25000ad000 tx=0x7f25000a6000 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:46.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.524+0000 7f2509600640 1 -- 172.21.15.86:0/822922060 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f250c1574e0 con 0x7f250c156b90 2024-04-25T14:05:46.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.716+0000 7f2513000640 1 -- 172.21.15.86:0/822922060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f250c1517e0 con 0x7f250c156b90 2024-04-25T14:05:46.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.716+0000 7f2509600640 1 -- 172.21.15.86:0/822922060 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5182 (secure 0 0 0) 0x7f250405cc60 con 0x7f250c156b90 2024-04-25T14:05:46.717 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:46.717 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":32,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5315,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3312926518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":3312926518},{"type":"v1","addr":"172.21.15.155:6835","nonce":3312926518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5776,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/898730495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":898730495},{"type":"v1","addr":"172.21.15.155:6834","nonce":898730495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5794,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2418778265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2418778265},{"type":"v1","addr":"172.21.15.86:6837","nonce":2418778265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:32.946320+0000","modified":"2024-04-25T14:05:45.906925+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5874},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5874":{"gid":5874,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6836/1732615622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1732615622},{"type":"v1","addr":"172.21.15.86:6836","nonce":1732615622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"fs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5874,"qdb_cluster":[5874]},"id":3}]} 2024-04-25T14:05:46.717 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 32 2024-04-25T14:05:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.716+0000 7f2513000640 1 -- 172.21.15.86:0/822922060 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f24ec06f5a0 msgr2=0x7f24ec071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.716+0000 7f2513000640 1 --2- 172.21.15.86:0/822922060 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f24ec06f5a0 0x7f24ec071ab0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f25000ad000 tx=0x7f25000a6000 comp rx=0 tx=0).stop 2024-04-25T14:05:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.716+0000 7f2513000640 1 -- 172.21.15.86:0/822922060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f250c156b90 msgr2=0x7f250c156fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.716+0000 7f2513000640 1 --2- 172.21.15.86:0/822922060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f250c156b90 0x7f250c156fa0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f250400c6f0 tx=0x7f250400cbb0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.716+0000 7f2513000640 1 -- 172.21.15.86:0/822922060 shutdown_connections 2024-04-25T14:05:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.716+0000 7f2513000640 1 --2- 172.21.15.86:0/822922060 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f24ec06f5a0 0x7f24ec071ab0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.716+0000 7f2513000640 1 --2- 172.21.15.86:0/822922060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f250c156b90 0x7f250c156fa0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.716+0000 7f2513000640 1 --2- 172.21.15.86:0/822922060 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f250c160f40 0x7f250c150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.716+0000 7f2513000640 1 --2- 172.21.15.86:0/822922060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f250c0d2f40 0x7f250c13dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.716+0000 7f2513000640 1 -- 172.21.15.86:0/822922060 >> 172.21.15.86:0/822922060 conn(0x7f250c0cfe10 msgr2=0x7f250c0b6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:46.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.716+0000 7f2513000640 1 -- 172.21.15.86:0/822922060 shutdown_connections 2024-04-25T14:05:46.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:46.716+0000 7f2513000640 1 -- 172.21.15.86:0/822922060 wait complete. 2024-04-25T14:05:46.913 INFO:tasks.ceph.mgr.y.smithi086.stderr:Exception in thread Thread-9: 2024-04-25T14:05:46.913 INFO:tasks.ceph.mgr.y.smithi086.stderr:Traceback (most recent call last): 2024-04-25T14:05:46.914 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner 2024-04-25T14:05:46.914 INFO:tasks.ceph.mgr.y.smithi086.stderr: self.run() 2024-04-25T14:05:46.914 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/lib/python3.10/threading.py", line 1378, in run 2024-04-25T14:05:46.914 INFO:tasks.ceph.mgr.y.smithi086.stderr: self.function(*self.args, **self.kwargs) 2024-04-25T14:05:46.914 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/share/ceph/mgr/stats/fs/perf_stats.py", line 222, in re_register_queries 2024-04-25T14:05:46.914 INFO:tasks.ceph.mgr.y.smithi086.stderr: if self.mx_last_updated >= ua_last_updated: 2024-04-25T14:05:46.914 INFO:tasks.ceph.mgr.y.smithi086.stderr:AttributeError: 'FSPerfStats' object has no attribute 'mx_last_updated' 2024-04-25T14:05:47.141 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:05:32.946320+0000', 'modified': '2024-04-25T14:05:45.906925+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 50, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5874}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5874': {'gid': 5874, 'name': 'a', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.86:6836/1732615622', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6834', 'nonce': 1732615622}, {'type': 'v1', 'addr': '172.21.15.86:6836', 'nonce': 1732615622}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'fs1', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5874, 'qdb_cluster': [5874]} 2024-04-25T14:05:47.141 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T14:05:47.141 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T14:05:47.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.288+0000 7f210bc00640 1 Processor -- start 2024-04-25T14:05:47.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.288+0000 7f210bc00640 1 -- start start 2024-04-25T14:05:47.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.288+0000 7f210bc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f21040cfca0 0x7f210406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.288+0000 7f210bc00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f21040d16e0 0x7f21040d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.292+0000 7f210bc00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21040d2030 con 0x7f210406c670 2024-04-25T14:05:47.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.292+0000 7f210bc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2104070470 con 0x7f21040cfca0 2024-04-25T14:05:47.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.292+0000 7f210bc00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f2104072bd0 con 0x7f21040d16e0 2024-04-25T14:05:47.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.292+0000 7f2109200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f21040cfca0 0x7f210406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.292+0000 7f2109200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f21040cfca0 0x7f210406f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47792/0 (socket says 172.21.15.86:47792) 2024-04-25T14:05:47.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.292+0000 7f2109200640 1 -- 172.21.15.86:0/806823676 learned_addr learned my addr 172.21.15.86:0/806823676 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:47.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.292+0000 7f20fbe00640 1 --2- 172.21.15.86:0/806823676 >> v2:172.21.15.155:3300/0 conn(0x7f21040d16e0 0x7f21040d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.292+0000 7f2109200640 1 -- 172.21.15.86:0/806823676 >> v2:172.21.15.155:3300/0 conn(0x7f21040d16e0 msgr2=0x7f21040d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:47.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.292+0000 7f2109200640 1 --2- 172.21.15.86:0/806823676 >> v2:172.21.15.155:3300/0 conn(0x7f21040d16e0 0x7f21040d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.292+0000 7f2109200640 1 -- 172.21.15.86:0/806823676 >> v1:172.21.15.155:6789/0 conn(0x7f210406c670 legacy=0x7f210406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.292+0000 7f2109200640 1 -- 172.21.15.86:0/806823676 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f210406f930 con 0x7f21040cfca0 2024-04-25T14:05:47.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.292+0000 7f20fbe00640 1 --2- 172.21.15.86:0/806823676 >> v2:172.21.15.155:3300/0 conn(0x7f21040d16e0 0x7f21040d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:47.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.292+0000 7f2109200640 1 --2- 172.21.15.86:0/806823676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21040cfca0 0x7f210406f3f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f21040729c0 tx=0x7f20f401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf2f20cfa1a12e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:47.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.296+0000 7f2103e00640 1 -- 172.21.15.86:0/806823676 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f40ab430 con 0x7f21040cfca0 2024-04-25T14:05:47.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.296+0000 7f2103e00640 1 -- 172.21.15.86:0/806823676 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20f40b0070 con 0x7f21040cfca0 2024-04-25T14:05:47.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.296+0000 7f2103e00640 1 -- 172.21.15.86:0/806823676 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f40ab7f0 con 0x7f21040cfca0 2024-04-25T14:05:47.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.296+0000 7f210bc00640 1 -- 172.21.15.86:0/806823676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21040cfca0 msgr2=0x7f210406f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.296+0000 7f210bc00640 1 --2- 172.21.15.86:0/806823676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21040cfca0 0x7f210406f3f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f21040729c0 tx=0x7f20f401c520 comp rx=0 tx=0).stop 2024-04-25T14:05:47.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.296+0000 7f210bc00640 1 -- 172.21.15.86:0/806823676 shutdown_connections 2024-04-25T14:05:47.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.296+0000 7f210bc00640 1 --2- 172.21.15.86:0/806823676 >> v2:172.21.15.155:3300/0 conn(0x7f21040d16e0 0x7f21040d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.296+0000 7f210bc00640 1 --2- 172.21.15.86:0/806823676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21040cfca0 0x7f210406f3f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.296+0000 7f210bc00640 1 -- 172.21.15.86:0/806823676 >> 172.21.15.86:0/806823676 conn(0x7f21040b4650 msgr2=0x7f21040b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:47.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.296+0000 7f210bc00640 1 -- 172.21.15.86:0/806823676 shutdown_connections 2024-04-25T14:05:47.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.296+0000 7f210bc00640 1 -- 172.21.15.86:0/806823676 wait complete. 2024-04-25T14:05:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.296+0000 7f210bc00640 1 Processor -- start 2024-04-25T14:05:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.296+0000 7f210bc00640 1 -- start start 2024-04-25T14:05:47.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.300+0000 7f210bc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f210406c670 0x7f21041f9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.300+0000 7f210bc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21040cfca0 0x7f2104099a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.300+0000 7f210bc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21040d16e0 0x7f21040b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.300+0000 7f210bc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f210416a070 con 0x7f210406c670 2024-04-25T14:05:47.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.300+0000 7f210bc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2104169ef0 con 0x7f21040cfca0 2024-04-25T14:05:47.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.300+0000 7f210bc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f210416a1f0 con 0x7f21040d16e0 2024-04-25T14:05:47.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.300+0000 7f2109200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f210406c670 0x7f21041f9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.300+0000 7f2109200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f210406c670 0x7f21041f9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47794/0 (socket says 172.21.15.86:47794) 2024-04-25T14:05:47.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.300+0000 7f2109200640 1 -- 172.21.15.86:0/2463916341 learned_addr learned my addr 172.21.15.86:0/2463916341 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:47.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.300+0000 7f20fbe00640 1 --2- 172.21.15.86:0/2463916341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21040cfca0 0x7f2104099a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.300+0000 7f2109c00640 1 --2- 172.21.15.86:0/2463916341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21040d16e0 0x7f21040b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.300+0000 7f2109200640 1 -- 172.21.15.86:0/2463916341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21040d16e0 msgr2=0x7f21040b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.300+0000 7f2109200640 1 --2- 172.21.15.86:0/2463916341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21040d16e0 0x7f21040b1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.300+0000 7f2109200640 1 -- 172.21.15.86:0/2463916341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21040cfca0 msgr2=0x7f2104099a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.300+0000 7f2109200640 1 --2- 172.21.15.86:0/2463916341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21040cfca0 0x7f2104099a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.300+0000 7f2109200640 1 -- 172.21.15.86:0/2463916341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f210409a4c0 con 0x7f210406c670 2024-04-25T14:05:47.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.300+0000 7f2109200640 1 --2- 172.21.15.86:0/2463916341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f210406c670 0x7f21041f9840 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f20f4019130 tx=0x7f20f401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:47.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.300+0000 7f2101600640 1 -- 172.21.15.86:0/2463916341 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f40be5f0 con 0x7f210406c670 2024-04-25T14:05:47.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.300+0000 7f2101600640 1 -- 172.21.15.86:0/2463916341 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20f40be790 con 0x7f210406c670 2024-04-25T14:05:47.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.300+0000 7f210bc00640 1 -- 172.21.15.86:0/2463916341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f210409a7a0 con 0x7f210406c670 2024-04-25T14:05:47.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.300+0000 7f2101600640 1 -- 172.21.15.86:0/2463916341 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f40beb80 con 0x7f210406c670 2024-04-25T14:05:47.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.300+0000 7f210bc00640 1 -- 172.21.15.86:0/2463916341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f210409ad60 con 0x7f210406c670 2024-04-25T14:05:47.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.304+0000 7f2101600640 1 -- 172.21.15.86:0/2463916341 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f20f40b43f0 con 0x7f210406c670 2024-04-25T14:05:47.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.304+0000 7f2101600640 1 --2- 172.21.15.86:0/2463916341 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f20e006f5a0 0x7f20e0071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.304+0000 7f2101600640 1 -- 172.21.15.86:0/2463916341 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6519+0+0 (secure 0 0 0) 0x7f20f40a8070 con 0x7f210406c670 2024-04-25T14:05:47.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.304+0000 7f20fbe00640 1 --2- 172.21.15.86:0/2463916341 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f20e006f5a0 0x7f20e0071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.304+0000 7f211331f000 1 Processor -- start 2024-04-25T14:05:47.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.304+0000 7f211331f000 1 -- start start 2024-04-25T14:05:47.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.304+0000 7f20fbe00640 1 --2- 172.21.15.86:0/2463916341 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f20e006f5a0 0x7f20e0071ab0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f21040ce730 tx=0x7f20ec008040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.304+0000 7f211331f000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560882c0ac20 0x560882d68ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.304+0000 7f211331f000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560882bf6ea0 0x560882d69420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.304+0000 7f211331f000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608827ef280 0x560882d69960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.304+0000 7f211331f000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560882bd8160 con 0x5608827ef280 2024-04-25T14:05:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.304+0000 7f211331f000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560882b6de20 con 0x560882bf6ea0 2024-04-25T14:05:47.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.304+0000 7f211331f000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x560882b26a20 con 0x560882c0ac20 2024-04-25T14:05:47.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.304+0000 7f2109c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608827ef280 0x560882d69960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.304+0000 7f2109c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608827ef280 0x560882d69960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47796/0 (socket says 172.21.15.86:47796) 2024-04-25T14:05:47.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.304+0000 7f2109c00640 1 -- 172.21.15.86:0/2638560269 learned_addr learned my addr 172.21.15.86:0/2638560269 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:47.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.304+0000 7f2109200640 1 --2- 172.21.15.86:0/2638560269 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560882c0ac20 0x560882d68ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.304+0000 7f20fbe00640 1 --2- 172.21.15.86:0/2638560269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560882bf6ea0 0x560882d69420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.308+0000 7f2109c00640 1 -- 172.21.15.86:0/2638560269 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560882c0ac20 msgr2=0x560882d68ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:47.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.308+0000 7f2109c00640 1 --2- 172.21.15.86:0/2638560269 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560882c0ac20 0x560882d68ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.308+0000 7f2109c00640 1 -- 172.21.15.86:0/2638560269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560882bf6ea0 msgr2=0x560882d69420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:47.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.308+0000 7f2109c00640 1 --2- 172.21.15.86:0/2638560269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560882bf6ea0 0x560882d69420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.308+0000 7f2109c00640 1 -- 172.21.15.86:0/2638560269 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560882b2b100 con 0x5608827ef280 2024-04-25T14:05:47.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.308+0000 7f2109c00640 1 --2- 172.21.15.86:0/2638560269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608827ef280 0x560882d69960 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f20fc001b90 tx=0x7f20fc002050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a3eef6aeac33bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.308+0000 7f20f9600640 1 -- 172.21.15.86:0/2638560269 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc019070 con 0x5608827ef280 2024-04-25T14:05:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.308+0000 7f20f9600640 1 -- 172.21.15.86:0/2638560269 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20fc003b90 con 0x5608827ef280 2024-04-25T14:05:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.308+0000 7f20f9600640 1 -- 172.21.15.86:0/2638560269 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc00e800 con 0x5608827ef280 2024-04-25T14:05:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.308+0000 7f211331f000 1 -- 172.21.15.86:0/2638560269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608827ef280 msgr2=0x560882d69960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.308+0000 7f211331f000 1 --2- 172.21.15.86:0/2638560269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608827ef280 0x560882d69960 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f20fc001b90 tx=0x7f20fc002050 comp rx=0 tx=0).stop 2024-04-25T14:05:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.308+0000 7f211331f000 1 -- 172.21.15.86:0/2638560269 shutdown_connections 2024-04-25T14:05:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.308+0000 7f211331f000 1 --2- 172.21.15.86:0/2638560269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608827ef280 0x560882d69960 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.308+0000 7f211331f000 1 --2- 172.21.15.86:0/2638560269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560882bf6ea0 0x560882d69420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.308+0000 7f211331f000 1 --2- 172.21.15.86:0/2638560269 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560882c0ac20 0x560882d68ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.308+0000 7f211331f000 1 -- 172.21.15.86:0/2638560269 >> 172.21.15.86:0/2638560269 conn(0x56088282eff0 msgr2=0x560882d1e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:47.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.308+0000 7f211331f000 1 -- 172.21.15.86:0/2638560269 shutdown_connections 2024-04-25T14:05:47.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.308+0000 7f211331f000 1 -- 172.21.15.86:0/2638560269 wait complete. 2024-04-25T14:05:47.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.308+0000 7f211331f000 1 -- start start 2024-04-25T14:05:47.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.308+0000 7f211331f000 1 Processor -- start 2024-04-25T14:05:47.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f211331f000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608827ef280 0x560882d6fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f211331f000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560882bf6ea0 0x560882d703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f211331f000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560882c0ac20 0x560882d2b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f211331f000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560882b7b220 con 0x5608827ef280 2024-04-25T14:05:47.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f211331f000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560882b11ea0 con 0x560882bf6ea0 2024-04-25T14:05:47.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f211331f000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x560882b68b00 con 0x560882c0ac20 2024-04-25T14:05:47.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f211331f000 10 client.0 _ll_register_callbacks cb 0x560882b7adc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:47.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f2109200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608827ef280 0x560882d6fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f2109200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608827ef280 0x560882d6fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47798/0 (socket says 172.21.15.86:47798) 2024-04-25T14:05:47.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f2109200640 1 -- 172.21.15.86:0/3554357342 learned_addr learned my addr 172.21.15.86:0/3554357342 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:47.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f20fbe00640 1 --2- 172.21.15.86:0/3554357342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560882bf6ea0 0x560882d703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f2109c00640 1 --2- 172.21.15.86:0/3554357342 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560882c0ac20 0x560882d2b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f2109200640 1 -- 172.21.15.86:0/3554357342 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560882c0ac20 msgr2=0x560882d2b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:47.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f2109200640 1 --2- 172.21.15.86:0/3554357342 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560882c0ac20 0x560882d2b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f2109200640 1 -- 172.21.15.86:0/3554357342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560882bf6ea0 msgr2=0x560882d703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:47.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f2109200640 1 --2- 172.21.15.86:0/3554357342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560882bf6ea0 0x560882d703a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f2109200640 1 -- 172.21.15.86:0/3554357342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560882b0d500 con 0x5608827ef280 2024-04-25T14:05:47.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f2109200640 1 --2- 172.21.15.86:0/3554357342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608827ef280 0x560882d6fe60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f20f401ca90 tx=0x7f20f40f5d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9077b04d03b59d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:47.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f20f2000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:05:47.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f20f2000640 1 -- 172.21.15.86:0/3554357342 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f4124020 con 0x5608827ef280 2024-04-25T14:05:47.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f20f2000640 1 -- 172.21.15.86:0/3554357342 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(50..50 src has 1..50) ==== 6519+0+0 (secure 0 0 0) 0x7f20f4120bc0 con 0x5608827ef280 2024-04-25T14:05:47.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f211331f000 1 -- 172.21.15.86:0/3554357342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20fc003b90 con 0x5608827ef280 2024-04-25T14:05:47.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f20f2000640 1 -- 172.21.15.86:0/3554357342 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f20f4103050 con 0x5608827ef280 2024-04-25T14:05:47.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f211331f000 10 client.5904 fetch_fsmap learned FSMap version 32 2024-04-25T14:05:47.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f211331f000 1 -- 172.21.15.86:0/3554357342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x560882af3ea0 con 0x5608827ef280 2024-04-25T14:05:47.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f20f2000640 1 -- 172.21.15.86:0/3554357342 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20fc003b90 con 0x5608827ef280 2024-04-25T14:05:47.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f20f2000640 1 -- 172.21.15.86:0/3554357342 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 3491+0+0 (secure 0 0 0) 0x7f20f40aa070 con 0x5608827ef280 2024-04-25T14:05:47.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f211331f000 10 client.5904 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T14:05:47.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f211331f000 10 client.5904 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T14:05:47.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f211331f000 20 client.5904 populate_metadata read hostname 'smithi086' 2024-04-25T14:05:47.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f211331f000 1 --2- 172.21.15.86:0/3554357342 >> [v2:172.21.15.86:6834/1732615622,v1:172.21.15.86:6836/1732615622] conn(0x5608828137d0 0x560882d2bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f211331f000 4 client.5904 mds_command: new command op to 5874 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:47.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f211331f000 1 -- 172.21.15.86:0/3554357342 --> [v2:172.21.15.86:6834/1732615622,v1:172.21.15.86:6836/1732615622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560882b06be0 con 0x5608828137d0 2024-04-25T14:05:47.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f20fbe00640 1 --2- 172.21.15.86:0/3554357342 >> [v2:172.21.15.86:6834/1732615622,v1:172.21.15.86:6836/1732615622] conn(0x5608828137d0 0x560882d2bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f20f2000640 10 client.5904 ms_handle_connect on v2:172.21.15.86:6834/1732615622 2024-04-25T14:05:47.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f20fbe00640 1 --2- 172.21.15.86:0/3554357342 >> [v2:172.21.15.86:6834/1732615622,v1:172.21.15.86:6836/1732615622] conn(0x5608828137d0 0x560882d2bda0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c12b8c0b1f4893d server_cookie=9c4b54dc3a35c91a in_seq=0 out_seq=0 2024-04-25T14:05:47.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f20f2000640 1 -- 172.21.15.86:0/3554357342 <== mds.0 v2:172.21.15.86:6834/1732615622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560882b06be0 con 0x5608828137d0 2024-04-25T14:05:47.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.312+0000 7f20f2000640 10 client.5904 handle_command_reply: tid=0 2024-04-25T14:05:47.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.316+0000 7f211331f000 1 client.5904 shutdown 2024-04-25T14:05:47.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.316+0000 7f211331f000 1 -- 172.21.15.86:0/3554357342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608827ef280 msgr2=0x560882d6fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.316+0000 7f211331f000 1 --2- 172.21.15.86:0/3554357342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608827ef280 0x560882d6fe60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f20f401ca90 tx=0x7f20f40f5d90 comp rx=0 tx=0).stop 2024-04-25T14:05:47.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.316+0000 7f211331f000 1 -- 172.21.15.86:0/3554357342 shutdown_connections 2024-04-25T14:05:47.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.316+0000 7f211331f000 1 --2- 172.21.15.86:0/3554357342 >> [v2:172.21.15.86:6834/1732615622,v1:172.21.15.86:6836/1732615622] conn(0x5608828137d0 0x560882d2bda0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.316+0000 7f211331f000 1 --2- 172.21.15.86:0/3554357342 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560882c0ac20 0x560882d2b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.316+0000 7f211331f000 1 --2- 172.21.15.86:0/3554357342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560882bf6ea0 0x560882d703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.316+0000 7f211331f000 1 --2- 172.21.15.86:0/3554357342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608827ef280 0x560882d6fe60 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.316+0000 7f20f2000640 0 client.5904 ms_handle_reset on v2:172.21.15.86:6834/1732615622 2024-04-25T14:05:47.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.316+0000 7f211331f000 1 -- 172.21.15.86:0/3554357342 >> 172.21.15.86:0/3554357342 conn(0x56088282eff0 msgr2=0x560882d1ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:47.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.316+0000 7f211331f000 1 -- 172.21.15.86:0/3554357342 shutdown_connections 2024-04-25T14:05:47.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.316+0000 7f211331f000 1 -- 172.21.15.86:0/3554357342 wait complete. 2024-04-25T14:05:47.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.316+0000 7f211331f000 20 client.5904 trim_cache size 0 max 16384 2024-04-25T14:05:47.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.332+0000 7f211331f000 1 Processor -- start 2024-04-25T14:05:47.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.332+0000 7f211331f000 1 -- start start 2024-04-25T14:05:47.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.332+0000 7f211331f000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608827ef280 0x560882d74ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.332+0000 7f211331f000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560882bf6ea0 0x560882d753e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.332+0000 7f211331f000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560882c0ac20 0x560882d7eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.332+0000 7f211331f000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2104205ca0 con 0x5608827ef280 2024-04-25T14:05:47.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.332+0000 7f211331f000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2104158150 con 0x560882c0ac20 2024-04-25T14:05:47.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.332+0000 7f211331f000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2104205680 con 0x560882bf6ea0 2024-04-25T14:05:47.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.336+0000 7f2109200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608827ef280 0x560882d74ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.336+0000 7f2109200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608827ef280 0x560882d74ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47806/0 (socket says 172.21.15.86:47806) 2024-04-25T14:05:47.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.336+0000 7f2109200640 1 -- 172.21.15.86:0/2714660019 learned_addr learned my addr 172.21.15.86:0/2714660019 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:47.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.336+0000 7f2109200640 1 -- 172.21.15.86:0/2714660019 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560882bf6ea0 msgr2=0x560882d753e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:47.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.336+0000 7f2109200640 1 --2- 172.21.15.86:0/2714660019 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560882bf6ea0 0x560882d753e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.336+0000 7f2109200640 1 -- 172.21.15.86:0/2714660019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560882c0ac20 msgr2=0x560882d7eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:47.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.336+0000 7f2109200640 1 --2- 172.21.15.86:0/2714660019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560882c0ac20 0x560882d7eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.336+0000 7f2109200640 1 -- 172.21.15.86:0/2714660019 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560882d2b430 con 0x5608827ef280 2024-04-25T14:05:47.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.336+0000 7f2109200640 1 --2- 172.21.15.86:0/2714660019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608827ef280 0x560882d74ea0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f20f40f5bb0 tx=0x7f20f40f8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cec60d4f1e1adf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:47.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.336+0000 7f20f8c00640 1 -- 172.21.15.86:0/2714660019 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f4124070 con 0x5608827ef280 2024-04-25T14:05:47.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.336+0000 7f20f8c00640 1 -- 172.21.15.86:0/2714660019 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20f40b0060 con 0x5608827ef280 2024-04-25T14:05:47.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.336+0000 7f20f8c00640 1 -- 172.21.15.86:0/2714660019 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f4120dc0 con 0x5608827ef280 2024-04-25T14:05:47.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.336+0000 7f211331f000 1 -- 172.21.15.86:0/2714660019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608827ef280 msgr2=0x560882d74ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.336+0000 7f211331f000 1 --2- 172.21.15.86:0/2714660019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608827ef280 0x560882d74ea0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f20f40f5bb0 tx=0x7f20f40f8000 comp rx=0 tx=0).stop 2024-04-25T14:05:47.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.336+0000 7f211331f000 1 -- 172.21.15.86:0/2714660019 shutdown_connections 2024-04-25T14:05:47.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.336+0000 7f211331f000 1 --2- 172.21.15.86:0/2714660019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560882c0ac20 0x560882d7eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.336+0000 7f211331f000 1 --2- 172.21.15.86:0/2714660019 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560882bf6ea0 0x560882d753e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.336+0000 7f211331f000 1 --2- 172.21.15.86:0/2714660019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608827ef280 0x560882d74ea0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.336+0000 7f211331f000 1 -- 172.21.15.86:0/2714660019 >> 172.21.15.86:0/2714660019 conn(0x56088282eff0 msgr2=0x560882d26000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:47.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.336+0000 7f211331f000 1 -- 172.21.15.86:0/2714660019 shutdown_connections 2024-04-25T14:05:47.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.336+0000 7f211331f000 1 -- 172.21.15.86:0/2714660019 wait complete. 2024-04-25T14:05:47.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.336+0000 7f211331f000 1 -- start start 2024-04-25T14:05:47.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.336+0000 7f211331f000 1 Processor -- start 2024-04-25T14:05:47.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f211331f000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608827ef280 0x560882ddb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f211331f000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560882bf6ea0 0x560882ddb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f211331f000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560882c0ac20 0x560882d734f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f211331f000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560882b6d130 con 0x5608827ef280 2024-04-25T14:05:47.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f211331f000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560882b037d0 con 0x560882bf6ea0 2024-04-25T14:05:47.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f211331f000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x560882b72450 con 0x560882c0ac20 2024-04-25T14:05:47.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f211331f000 10 client.0 _ll_register_callbacks cb 0x7f20f40f1df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:47.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f2109200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608827ef280 0x560882ddb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f2109200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608827ef280 0x560882ddb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47822/0 (socket says 172.21.15.86:47822) 2024-04-25T14:05:47.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f2109200640 1 -- 172.21.15.86:0/2661174961 learned_addr learned my addr 172.21.15.86:0/2661174961 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:47.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f2109200640 1 -- 172.21.15.86:0/2661174961 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560882c0ac20 msgr2=0x560882d734f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:47.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f2109200640 1 --2- 172.21.15.86:0/2661174961 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560882c0ac20 0x560882d734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f2109200640 1 -- 172.21.15.86:0/2661174961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560882bf6ea0 msgr2=0x560882ddb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:47.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f2109200640 1 --2- 172.21.15.86:0/2661174961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560882bf6ea0 0x560882ddb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f2109200640 1 -- 172.21.15.86:0/2661174961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560882b26c70 con 0x5608827ef280 2024-04-25T14:05:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f2109200640 1 --2- 172.21.15.86:0/2661174961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608827ef280 0x560882ddb170 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f20f40f24a0 tx=0x7f20f401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19fc003b76f6e001 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f20f2000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:05:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f20f2000640 1 -- 172.21.15.86:0/2661174961 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f4124030 con 0x5608827ef280 2024-04-25T14:05:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f20f2000640 1 -- 172.21.15.86:0/2661174961 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(50..50 src has 1..50) ==== 6519+0+0 (secure 0 0 0) 0x7f20f40f1a20 con 0x5608827ef280 2024-04-25T14:05:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f211331f000 1 -- 172.21.15.86:0/2661174961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20f40b0060 con 0x5608827ef280 2024-04-25T14:05:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f20f2000640 1 -- 172.21.15.86:0/2661174961 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f20f40fec60 con 0x5608827ef280 2024-04-25T14:05:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f211331f000 10 client.5910 fetch_fsmap learned FSMap version 32 2024-04-25T14:05:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f211331f000 1 -- 172.21.15.86:0/2661174961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x560882bf0180 con 0x5608827ef280 2024-04-25T14:05:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f20f2000640 1 -- 172.21.15.86:0/2661174961 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20f40b0060 con 0x5608827ef280 2024-04-25T14:05:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f20f2000640 1 -- 172.21.15.86:0/2661174961 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 3491+0+0 (secure 0 0 0) 0x7f20f40aa040 con 0x5608827ef280 2024-04-25T14:05:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f211331f000 10 client.5910 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T14:05:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f211331f000 10 client.5910 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T14:05:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f211331f000 20 client.5910 populate_metadata read hostname 'smithi086' 2024-04-25T14:05:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f211331f000 1 --2- 172.21.15.86:0/2661174961 >> [v2:172.21.15.86:6834/1732615622,v1:172.21.15.86:6836/1732615622] conn(0x560882801830 0x560882d73eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f211331f000 4 client.5910 mds_command: new command op to 5874 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:05:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f211331f000 1 -- 172.21.15.86:0/2661174961 --> [v2:172.21.15.86:6834/1732615622,v1:172.21.15.86:6836/1732615622] -- command(tid 0: {"prefix": "status"}) -- 0x560882b7ba30 con 0x560882801830 2024-04-25T14:05:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f20fbe00640 1 --2- 172.21.15.86:0/2661174961 >> [v2:172.21.15.86:6834/1732615622,v1:172.21.15.86:6836/1732615622] conn(0x560882801830 0x560882d73eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f20fbe00640 1 --2- 172.21.15.86:0/2661174961 >> [v2:172.21.15.86:6834/1732615622,v1:172.21.15.86:6836/1732615622] conn(0x560882801830 0x560882d73eb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35927e5272fd7599 server_cookie=985e2e523cf0032f in_seq=0 out_seq=0 2024-04-25T14:05:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f20f2000640 10 client.5910 ms_handle_connect on v2:172.21.15.86:6834/1732615622 2024-04-25T14:05:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f20f2000640 1 -- 172.21.15.86:0/2661174961 <== mds.0 v2:172.21.15.86:6834/1732615622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x560882b7ba30 con 0x560882801830 2024-04-25T14:05:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f20f2000640 10 client.5910 handle_command_reply: tid=0 2024-04-25T14:05:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f211331f000 1 client.5910 shutdown 2024-04-25T14:05:47.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f211331f000 1 -- 172.21.15.86:0/2661174961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608827ef280 msgr2=0x560882ddb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f211331f000 1 --2- 172.21.15.86:0/2661174961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608827ef280 0x560882ddb170 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f20f40f24a0 tx=0x7f20f401c520 comp rx=0 tx=0).stop 2024-04-25T14:05:47.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f211331f000 1 -- 172.21.15.86:0/2661174961 shutdown_connections 2024-04-25T14:05:47.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f211331f000 1 --2- 172.21.15.86:0/2661174961 >> [v2:172.21.15.86:6834/1732615622,v1:172.21.15.86:6836/1732615622] conn(0x560882801830 0x560882d73eb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f211331f000 1 --2- 172.21.15.86:0/2661174961 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560882c0ac20 0x560882d734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f211331f000 1 --2- 172.21.15.86:0/2661174961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560882bf6ea0 0x560882ddb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f20f2000640 0 client.5910 ms_handle_reset on v2:172.21.15.86:6834/1732615622 2024-04-25T14:05:47.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f211331f000 1 --2- 172.21.15.86:0/2661174961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608827ef280 0x560882ddb170 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f211331f000 1 -- 172.21.15.86:0/2661174961 >> 172.21.15.86:0/2661174961 conn(0x56088282eff0 msgr2=0x560882d6dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:47.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f211331f000 1 -- 172.21.15.86:0/2661174961 shutdown_connections 2024-04-25T14:05:47.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f211331f000 1 -- 172.21.15.86:0/2661174961 wait complete. 2024-04-25T14:05:47.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.340+0000 7f211331f000 20 client.5910 trim_cache size 0 max 16384 2024-04-25T14:05:47.349 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:05:47.349 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb", 2024-04-25T14:05:47.349 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-25T14:05:47.349 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 5874, 2024-04-25T14:05:47.349 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T14:05:47.349 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T14:05:47.349 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "fs1", 2024-04-25T14:05:47.349 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 4.4456288470000001, 2024-04-25T14:05:47.350 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 32, 2024-04-25T14:05:47.350 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 50, 2024-04-25T14:05:47.350 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 50, 2024-04-25T14:05:47.350 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 4.490083898 2024-04-25T14:05:47.350 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:05:47.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.344+0000 7f210bc00640 1 -- 172.21.15.86:0/2463916341 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f20e006f5a0 msgr2=0x7f20e0071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.344+0000 7f210bc00640 1 --2- 172.21.15.86:0/2463916341 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f20e006f5a0 0x7f20e0071ab0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f21040ce730 tx=0x7f20ec008040 comp rx=0 tx=0).stop 2024-04-25T14:05:47.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.344+0000 7f210bc00640 1 -- 172.21.15.86:0/2463916341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f210406c670 msgr2=0x7f21041f9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.344+0000 7f210bc00640 1 --2- 172.21.15.86:0/2463916341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f210406c670 0x7f21041f9840 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f20f4019130 tx=0x7f20f401ad60 comp rx=0 tx=0).stop 2024-04-25T14:05:47.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.344+0000 7f210bc00640 1 -- 172.21.15.86:0/2463916341 shutdown_connections 2024-04-25T14:05:47.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.344+0000 7f210bc00640 1 --2- 172.21.15.86:0/2463916341 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f20e006f5a0 0x7f20e0071ab0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.344+0000 7f210bc00640 1 --2- 172.21.15.86:0/2463916341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21040d16e0 0x7f21040b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.344+0000 7f210bc00640 1 --2- 172.21.15.86:0/2463916341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21040cfca0 0x7f2104099a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.344+0000 7f210bc00640 1 --2- 172.21.15.86:0/2463916341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f210406c670 0x7f21041f9840 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.344+0000 7f210bc00640 1 -- 172.21.15.86:0/2463916341 >> 172.21.15.86:0/2463916341 conn(0x7f21040b4650 msgr2=0x7f21040b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:47.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.344+0000 7f210bc00640 1 -- 172.21.15.86:0/2463916341 shutdown_connections 2024-04-25T14:05:47.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.344+0000 7f210bc00640 1 -- 172.21.15.86:0/2463916341 wait complete. 2024-04-25T14:05:47.364 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=fs1_data' 2024-04-25T14:05:47.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.516+0000 7f5c19000640 1 Processor -- start 2024-04-25T14:05:47.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.516+0000 7f5c19000640 1 -- start start 2024-04-25T14:05:47.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.520+0000 7f5c19000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c14071530 0x7f5c140ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.520+0000 7f5c19000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c1406d940 con 0x7f5c140cdd70 2024-04-25T14:05:47.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.520+0000 7f5c19000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c1406dc20 con 0x7f5c1406e700 2024-04-25T14:05:47.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.520+0000 7f5c19000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5c1406df00 con 0x7f5c14071530 2024-04-25T14:05:47.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.520+0000 7f5c11e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5c140cdd70 0x7f5c14070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41266/0 (socket says 172.21.15.86:41266) 2024-04-25T14:05:47.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.520+0000 7f5c11e00640 1 -- 172.21.15.86:0/2836502094 learned_addr learned my addr 172.21.15.86:0/2836502094 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:47.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.520+0000 7f5c11400640 1 --2- 172.21.15.86:0/2836502094 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c14071530 0x7f5c140ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.520+0000 7f5c10a00640 1 -- 172.21.15.86:0/2836502094 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2058401904 0 0) 0x7f5c1406d940 con 0x7f5c140cdd70 2024-04-25T14:05:47.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.520+0000 7f5c10a00640 1 -- 172.21.15.86:0/2836502094 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bf8003530 con 0x7f5c140cdd70 2024-04-25T14:05:47.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.520+0000 7f5c10a00640 1 -- 172.21.15.86:0/2836502094 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1518423078 0 0) 0x7f5c1406dc20 con 0x7f5c1406e700 2024-04-25T14:05:47.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.520+0000 7f5c10a00640 1 -- 172.21.15.86:0/2836502094 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c1406d940 con 0x7f5c1406e700 2024-04-25T14:05:47.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.520+0000 7f5c10a00640 1 -- 172.21.15.86:0/2836502094 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3772373791 0 0) 0x7f5bf8003530 con 0x7f5c140cdd70 2024-04-25T14:05:47.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.520+0000 7f5c11400640 1 -- 172.21.15.86:0/2836502094 >> v1:172.21.15.155:6789/0 conn(0x7f5c1406e700 legacy=0x7f5c1406eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.520+0000 7f5c11400640 1 -- 172.21.15.86:0/2836502094 >> v1:172.21.15.86:6789/0 conn(0x7f5c140cdd70 legacy=0x7f5c14070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.520+0000 7f5c11400640 1 -- 172.21.15.86:0/2836502094 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c1406e1e0 con 0x7f5c14071530 2024-04-25T14:05:47.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.524+0000 7f5c11400640 1 --2- 172.21.15.86:0/2836502094 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c14071530 0x7f5c140ce180 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5c000097d0 tx=0x7f5c0001c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=947b63a80fac2a69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:47.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.524+0000 7f5c10a00640 1 -- 172.21.15.86:0/2836502094 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c000b0070 con 0x7f5c14071530 2024-04-25T14:05:47.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.524+0000 7f5c10a00640 1 -- 172.21.15.86:0/2836502094 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c000ab400 con 0x7f5c14071530 2024-04-25T14:05:47.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.524+0000 7f5c10a00640 1 -- 172.21.15.86:0/2836502094 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c04001680 con 0x7f5c14071530 2024-04-25T14:05:47.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.524+0000 7f5c19000640 1 -- 172.21.15.86:0/2836502094 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c14071530 msgr2=0x7f5c140ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.524+0000 7f5c19000640 1 --2- 172.21.15.86:0/2836502094 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c14071530 0x7f5c140ce180 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5c000097d0 tx=0x7f5c0001c350 comp rx=0 tx=0).stop 2024-04-25T14:05:47.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.524+0000 7f5c19000640 1 -- 172.21.15.86:0/2836502094 shutdown_connections 2024-04-25T14:05:47.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.524+0000 7f5c19000640 1 --2- 172.21.15.86:0/2836502094 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c14071530 0x7f5c140ce180 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.524+0000 7f5c19000640 1 -- 172.21.15.86:0/2836502094 >> 172.21.15.86:0/2836502094 conn(0x7f5c140cffc0 msgr2=0x7f5c140d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:47.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.524+0000 7f5c19000640 1 -- 172.21.15.86:0/2836502094 shutdown_connections 2024-04-25T14:05:47.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.524+0000 7f5c19000640 1 -- 172.21.15.86:0/2836502094 wait complete. 2024-04-25T14:05:47.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.528+0000 7f5c19000640 1 Processor -- start 2024-04-25T14:05:47.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.528+0000 7f5c19000640 1 -- start start 2024-04-25T14:05:47.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.528+0000 7f5c19000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c1406e700 0x7f5c14152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.528+0000 7f5c19000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c140cdd70 0x7f5c1414da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.528+0000 7f5c19000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c141602e0 0x7f5c141606f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.528+0000 7f5c19000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1416dc10 con 0x7f5c140cdd70 2024-04-25T14:05:47.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.528+0000 7f5c19000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5c141643c0 con 0x7f5c141602e0 2024-04-25T14:05:47.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.528+0000 7f5c19000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5c1416dd90 con 0x7f5c1406e700 2024-04-25T14:05:47.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.528+0000 7f5c11400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c140cdd70 0x7f5c1414da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.528+0000 7f5c11400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c140cdd70 0x7f5c1414da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47838/0 (socket says 172.21.15.86:47838) 2024-04-25T14:05:47.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.528+0000 7f5c11400640 1 -- 172.21.15.86:0/761324130 learned_addr learned my addr 172.21.15.86:0/761324130 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:47.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.528+0000 7f5c11e00640 1 --2- 172.21.15.86:0/761324130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c1406e700 0x7f5c14152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.528+0000 7f5c12800640 1 --2- 172.21.15.86:0/761324130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c141602e0 0x7f5c141606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.528+0000 7f5c11400640 1 -- 172.21.15.86:0/761324130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c1406e700 msgr2=0x7f5c14152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.528+0000 7f5c11400640 1 --2- 172.21.15.86:0/761324130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c1406e700 0x7f5c14152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.528+0000 7f5c11400640 1 -- 172.21.15.86:0/761324130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c141602e0 msgr2=0x7f5c141606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.528+0000 7f5c11400640 1 --2- 172.21.15.86:0/761324130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c141602e0 0x7f5c141606f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.528+0000 7f5c11400640 1 -- 172.21.15.86:0/761324130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c14151c20 con 0x7f5c140cdd70 2024-04-25T14:05:47.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.528+0000 7f5c12800640 1 --2- 172.21.15.86:0/761324130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c141602e0 0x7f5c141606f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:47.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.528+0000 7f5c11400640 1 --2- 172.21.15.86:0/761324130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c140cdd70 0x7f5c1414da70 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5c000097d0 tx=0x7f5c000190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:47.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.528+0000 7f5c0a000640 1 -- 172.21.15.86:0/761324130 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c000b0040 con 0x7f5c140cdd70 2024-04-25T14:05:47.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.528+0000 7f5c0a000640 1 -- 172.21.15.86:0/761324130 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c00005bc0 con 0x7f5c140cdd70 2024-04-25T14:05:47.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.528+0000 7f5c19000640 1 -- 172.21.15.86:0/761324130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c14151f00 con 0x7f5c140cdd70 2024-04-25T14:05:47.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.528+0000 7f5c0a000640 1 -- 172.21.15.86:0/761324130 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c000abe60 con 0x7f5c140cdd70 2024-04-25T14:05:47.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.528+0000 7f5c19000640 1 -- 172.21.15.86:0/761324130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c141524c0 con 0x7f5c140cdd70 2024-04-25T14:05:47.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.532+0000 7f5c19000640 1 -- 172.21.15.86:0/761324130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c14151990 con 0x7f5c140cdd70 2024-04-25T14:05:47.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.532+0000 7f5c0a000640 1 -- 172.21.15.86:0/761324130 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f5c00004030 con 0x7f5c140cdd70 2024-04-25T14:05:47.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.532+0000 7f5c0a000640 1 --2- 172.21.15.86:0/761324130 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f5bf406f570 0x7f5bf4071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.532+0000 7f5c0a000640 1 -- 172.21.15.86:0/761324130 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6519+0+0 (secure 0 0 0) 0x7f5c00129c40 con 0x7f5c140cdd70 2024-04-25T14:05:47.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.532+0000 7f5c11e00640 1 --2- 172.21.15.86:0/761324130 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f5bf406f570 0x7f5bf4071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.532+0000 7f5c11e00640 1 --2- 172.21.15.86:0/761324130 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f5bf406f570 0x7f5bf4071a80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5c04000fe0 tx=0x7f5c04001160 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:47.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.536+0000 7f5c0a000640 1 -- 172.21.15.86:0/761324130 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5c000c1070 con 0x7f5c140cdd70 2024-04-25T14:05:47.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.748+0000 7f5c19000640 1 -- 172.21.15.86:0/761324130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=fs1_data"]} v 0) -- 0x7f5c140720e0 con 0x7f5c140cdd70 2024-04-25T14:05:47.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.752+0000 7f5c0a000640 1 -- 172.21.15.86:0/761324130 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=fs1_data"]}]=0 updated caps for client.0 v16) ==== 181+0+138 (secure 0 0 0) 0x7f5c000b8030 con 0x7f5c140cdd70 2024-04-25T14:05:47.755 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-04-25T14:05:47.755 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCLYipm5u4xHRAACDFe3Eb4KLrsdLj4eFe7mA== 2024-04-25T14:05:47.755 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-04-25T14:05:47.755 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-04-25T14:05:47.755 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw pool=fs1_data" 2024-04-25T14:05:47.756 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.0 2024-04-25T14:05:47.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.756+0000 7f5c19000640 1 -- 172.21.15.86:0/761324130 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f5bf406f570 msgr2=0x7f5bf4071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.756+0000 7f5c19000640 1 --2- 172.21.15.86:0/761324130 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f5bf406f570 0x7f5bf4071a80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5c04000fe0 tx=0x7f5c04001160 comp rx=0 tx=0).stop 2024-04-25T14:05:47.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.756+0000 7f5c19000640 1 -- 172.21.15.86:0/761324130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c140cdd70 msgr2=0x7f5c1414da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.756+0000 7f5c19000640 1 --2- 172.21.15.86:0/761324130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c140cdd70 0x7f5c1414da70 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5c000097d0 tx=0x7f5c000190f0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.756+0000 7f5c19000640 1 -- 172.21.15.86:0/761324130 shutdown_connections 2024-04-25T14:05:47.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.756+0000 7f5c19000640 1 --2- 172.21.15.86:0/761324130 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f5bf406f570 0x7f5bf4071a80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.756+0000 7f5c19000640 1 --2- 172.21.15.86:0/761324130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c141602e0 0x7f5c141606f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.756+0000 7f5c19000640 1 --2- 172.21.15.86:0/761324130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c140cdd70 0x7f5c1414da70 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.756+0000 7f5c19000640 1 --2- 172.21.15.86:0/761324130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c1406e700 0x7f5c14152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.756+0000 7f5c19000640 1 -- 172.21.15.86:0/761324130 >> 172.21.15.86:0/761324130 conn(0x7f5c140cffc0 msgr2=0x7f5c1406eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:47.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.756+0000 7f5c19000640 1 -- 172.21.15.86:0/761324130 shutdown_connections 2024-04-25T14:05:47.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.756+0000 7f5c19000640 1 -- 172.21.15.86:0/761324130 wait complete. 2024-04-25T14:05:47.777 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=fs1_data' 2024-04-25T14:05:47.900 INFO:tasks.ceph.mds.a.smithi086.stderr:2024-04-25T14:05:47.896+0000 7f5f09400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T14:05:47.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.932+0000 7f6686400640 1 Processor -- start 2024-04-25T14:05:47.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.932+0000 7f6686400640 1 -- start start 2024-04-25T14:05:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.932+0000 7f6686400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66800cfca0 0x7f668006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.932+0000 7f6686400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f66800d16e0 0x7f66800d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.932+0000 7f6686400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66800d2030 con 0x7f668006c670 2024-04-25T14:05:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.932+0000 7f6686400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f6680070470 con 0x7f66800d16e0 2024-04-25T14:05:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.932+0000 7f6686400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6680072bd0 con 0x7f66800cfca0 2024-04-25T14:05:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.932+0000 7f667f400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66800cfca0 0x7f668006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.932+0000 7f6676a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f66800d16e0 0x7f66800d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.932+0000 7f667f400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66800cfca0 0x7f668006f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:35306/0 (socket says 172.21.15.86:35306) 2024-04-25T14:05:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.932+0000 7f667f400640 1 -- 172.21.15.86:0/3836265261 learned_addr learned my addr 172.21.15.86:0/3836265261 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.932+0000 7f667f400640 1 -- 172.21.15.86:0/3836265261 >> v2:172.21.15.155:3300/0 conn(0x7f66800d16e0 msgr2=0x7f66800d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.932+0000 7f667f400640 1 --2- 172.21.15.86:0/3836265261 >> v2:172.21.15.155:3300/0 conn(0x7f66800d16e0 0x7f66800d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.932+0000 7f667f400640 1 -- 172.21.15.86:0/3836265261 >> v1:172.21.15.155:6789/0 conn(0x7f668006c670 legacy=0x7f668006ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.932+0000 7f667f400640 1 -- 172.21.15.86:0/3836265261 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f668006f930 con 0x7f66800cfca0 2024-04-25T14:05:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.932+0000 7f6676a00640 1 --2- 172.21.15.86:0/3836265261 >> v2:172.21.15.155:3300/0 conn(0x7f66800d16e0 0x7f66800d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:47.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.936+0000 7f667f400640 1 --2- 172.21.15.86:0/3836265261 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66800cfca0 0x7f668006f3f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6670009e50 tx=0x7f667001ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8643dcf789b2db4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:47.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.936+0000 7f667ea00640 1 -- 172.21.15.86:0/3836265261 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66700a5810 con 0x7f66800cfca0 2024-04-25T14:05:47.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.936+0000 7f667ea00640 1 -- 172.21.15.86:0/3836265261 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66700a59b0 con 0x7f66800cfca0 2024-04-25T14:05:47.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.936+0000 7f667ea00640 1 -- 172.21.15.86:0/3836265261 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66700a5d80 con 0x7f66800cfca0 2024-04-25T14:05:47.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.936+0000 7f6686400640 1 -- 172.21.15.86:0/3836265261 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66800cfca0 msgr2=0x7f668006f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.936+0000 7f6686400640 1 --2- 172.21.15.86:0/3836265261 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66800cfca0 0x7f668006f3f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6670009e50 tx=0x7f667001ca30 comp rx=0 tx=0).stop 2024-04-25T14:05:47.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.936+0000 7f6686400640 1 -- 172.21.15.86:0/3836265261 shutdown_connections 2024-04-25T14:05:47.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.936+0000 7f6686400640 1 --2- 172.21.15.86:0/3836265261 >> v2:172.21.15.155:3300/0 conn(0x7f66800d16e0 0x7f66800d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.936+0000 7f6686400640 1 --2- 172.21.15.86:0/3836265261 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66800cfca0 0x7f668006f3f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.936+0000 7f6686400640 1 -- 172.21.15.86:0/3836265261 >> 172.21.15.86:0/3836265261 conn(0x7f66800b4650 msgr2=0x7f66800b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:47.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.936+0000 7f6686400640 1 -- 172.21.15.86:0/3836265261 shutdown_connections 2024-04-25T14:05:47.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.936+0000 7f6686400640 1 -- 172.21.15.86:0/3836265261 wait complete. 2024-04-25T14:05:47.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.940+0000 7f6686400640 1 Processor -- start 2024-04-25T14:05:47.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.940+0000 7f6686400640 1 -- start start 2024-04-25T14:05:47.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.940+0000 7f6686400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f668006c670 0x7f668009ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.940+0000 7f6686400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66800cfca0 0x7f6680155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.940+0000 7f6686400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66800d16e0 0x7f66800af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.940+0000 7f6686400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6680168060 con 0x7f668006c670 2024-04-25T14:05:47.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.940+0000 7f6686400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6680167ee0 con 0x7f66800cfca0 2024-04-25T14:05:47.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.940+0000 7f6686400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f66801681e0 con 0x7f66800d16e0 2024-04-25T14:05:47.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.940+0000 7f667f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f668006c670 0x7f668009ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.940+0000 7f667f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f668006c670 0x7f668009ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47848/0 (socket says 172.21.15.86:47848) 2024-04-25T14:05:47.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.940+0000 7f667f400640 1 -- 172.21.15.86:0/2688348219 learned_addr learned my addr 172.21.15.86:0/2688348219 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:47.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.940+0000 7f6676a00640 1 --2- 172.21.15.86:0/2688348219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66800cfca0 0x7f6680155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.940+0000 7f667fe00640 1 --2- 172.21.15.86:0/2688348219 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66800d16e0 0x7f66800af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.940+0000 7f667f400640 1 -- 172.21.15.86:0/2688348219 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66800d16e0 msgr2=0x7f66800af7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.940+0000 7f667f400640 1 --2- 172.21.15.86:0/2688348219 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66800d16e0 0x7f66800af7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.940+0000 7f667f400640 1 -- 172.21.15.86:0/2688348219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66800cfca0 msgr2=0x7f6680155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.940+0000 7f667f400640 1 --2- 172.21.15.86:0/2688348219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66800cfca0 0x7f6680155110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.940+0000 7f667f400640 1 -- 172.21.15.86:0/2688348219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66800d0610 con 0x7f668006c670 2024-04-25T14:05:47.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.940+0000 7f667f400640 1 --2- 172.21.15.86:0/2688348219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f668006c670 0x7f668009ab20 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f667001b0e0 tx=0x7f66700197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:47.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.940+0000 7f6677e00640 1 -- 172.21.15.86:0/2688348219 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66700b8670 con 0x7f668006c670 2024-04-25T14:05:47.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.940+0000 7f6677e00640 1 -- 172.21.15.86:0/2688348219 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66700be070 con 0x7f668006c670 2024-04-25T14:05:47.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.940+0000 7f6677e00640 1 -- 172.21.15.86:0/2688348219 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66700b8a90 con 0x7f668006c670 2024-04-25T14:05:47.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.940+0000 7f6686400640 1 -- 172.21.15.86:0/2688348219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f668015fce0 con 0x7f668006c670 2024-04-25T14:05:47.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.944+0000 7f6686400640 1 -- 172.21.15.86:0/2688348219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66800cbc70 con 0x7f668006c670 2024-04-25T14:05:47.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.944+0000 7f6686400640 1 -- 172.21.15.86:0/2688348219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f668009f4b0 con 0x7f668006c670 2024-04-25T14:05:47.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.944+0000 7f6677e00640 1 -- 172.21.15.86:0/2688348219 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f66700b8c30 con 0x7f668006c670 2024-04-25T14:05:47.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.944+0000 7f6677e00640 1 --2- 172.21.15.86:0/2688348219 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f665c06f570 0x7f665c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.944+0000 7f6677e00640 1 -- 172.21.15.86:0/2688348219 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6519+0+0 (secure 0 0 0) 0x7f667012a6d0 con 0x7f668006c670 2024-04-25T14:05:47.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.944+0000 7f6676a00640 1 --2- 172.21.15.86:0/2688348219 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f665c06f570 0x7f665c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.944+0000 7f6676a00640 1 --2- 172.21.15.86:0/2688348219 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f665c06f570 0x7f665c071a80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f66800b3770 tx=0x7f6668009190 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:47.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:47.948+0000 7f6677e00640 1 -- 172.21.15.86:0/2688348219 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6670016620 con 0x7f668006c670 2024-04-25T14:05:48.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.160+0000 7f6686400640 1 -- 172.21.15.86:0/2688348219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=fs1_data"]} v 0) -- 0x7f6680069d00 con 0x7f668006c670 2024-04-25T14:05:48.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.164+0000 7f6677e00640 1 -- 172.21.15.86:0/2688348219 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=fs1_data"]}]=0 updated caps for client.1 v17) ==== 181+0+138 (secure 0 0 0) 0x7f66700ff030 con 0x7f668006c670 2024-04-25T14:05:48.165 INFO:teuthology.orchestra.run.smithi086.stdout:[client.1] 2024-04-25T14:05:48.165 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCLYipmDp1hJRAAQkBc5PdQoMTT7W4HahkTyg== 2024-04-25T14:05:48.165 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-04-25T14:05:48.166 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-04-25T14:05:48.166 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw pool=fs1_data" 2024-04-25T14:05:48.166 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.1 2024-04-25T14:05:48.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.164+0000 7f6686400640 1 -- 172.21.15.86:0/2688348219 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f665c06f570 msgr2=0x7f665c071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:48.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.164+0000 7f6686400640 1 --2- 172.21.15.86:0/2688348219 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f665c06f570 0x7f665c071a80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f66800b3770 tx=0x7f6668009190 comp rx=0 tx=0).stop 2024-04-25T14:05:48.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.164+0000 7f6686400640 1 -- 172.21.15.86:0/2688348219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f668006c670 msgr2=0x7f668009ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:48.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.164+0000 7f6686400640 1 --2- 172.21.15.86:0/2688348219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f668006c670 0x7f668009ab20 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f667001b0e0 tx=0x7f66700197d0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.164+0000 7f6686400640 1 -- 172.21.15.86:0/2688348219 shutdown_connections 2024-04-25T14:05:48.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.164+0000 7f6686400640 1 --2- 172.21.15.86:0/2688348219 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f665c06f570 0x7f665c071a80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.164+0000 7f6686400640 1 --2- 172.21.15.86:0/2688348219 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66800d16e0 0x7f66800af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.164+0000 7f6686400640 1 --2- 172.21.15.86:0/2688348219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66800cfca0 0x7f6680155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.164+0000 7f6686400640 1 --2- 172.21.15.86:0/2688348219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f668006c670 0x7f668009ab20 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.164+0000 7f6686400640 1 -- 172.21.15.86:0/2688348219 >> 172.21.15.86:0/2688348219 conn(0x7f66800b4650 msgr2=0x7f6680157c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:48.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.164+0000 7f6686400640 1 -- 172.21.15.86:0/2688348219 shutdown_connections 2024-04-25T14:05:48.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.164+0000 7f6686400640 1 -- 172.21.15.86:0/2688348219 wait complete. 2024-04-25T14:05:48.187 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=fs1_data' 2024-04-25T14:05:48.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.340+0000 7f3db9e00640 1 Processor -- start 2024-04-25T14:05:48.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.340+0000 7f3db9e00640 1 -- start start 2024-04-25T14:05:48.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.344+0000 7f3db9e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3db415c2a0 0x7f3db40ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.344+0000 7f3db9e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3db415feb0 0x7f3db41602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.344+0000 7f3db9e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3db40b0a60 con 0x7f3db406fd50 2024-04-25T14:05:48.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.344+0000 7f3db9e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3db406ef90 con 0x7f3db415feb0 2024-04-25T14:05:48.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.344+0000 7f3db9e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3db406f270 con 0x7f3db415c2a0 2024-04-25T14:05:48.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.344+0000 7f3db2a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3db415c2a0 0x7f3db40ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.344+0000 7f3db3400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3db415feb0 0x7f3db41602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.344+0000 7f3db3400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3db415feb0 0x7f3db41602c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:55404/0 (socket says 172.21.15.86:55404) 2024-04-25T14:05:48.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.344+0000 7f3db3400640 1 -- 172.21.15.86:0/1399100270 learned_addr learned my addr 172.21.15.86:0/1399100270 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:48.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.344+0000 7f3db3400640 1 -- 172.21.15.86:0/1399100270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3db415c2a0 msgr2=0x7f3db40ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:48.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.344+0000 7f3db3400640 1 --2- 172.21.15.86:0/1399100270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3db415c2a0 0x7f3db40ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.344+0000 7f3db3400640 1 -- 172.21.15.86:0/1399100270 >> v1:172.21.15.155:6789/0 conn(0x7f3db406fd50 legacy=0x7f3db4070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:48.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.344+0000 7f3db3400640 1 -- 172.21.15.86:0/1399100270 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db406f550 con 0x7f3db415feb0 2024-04-25T14:05:48.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.344+0000 7f3db1600640 1 -- 172.21.15.86:0/1399100270 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2799829843 0 0) 0x7f3db40b0a60 con 0x7f3db406fd50 2024-04-25T14:05:48.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.348+0000 7f3db3400640 1 --2- 172.21.15.86:0/1399100270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3db415feb0 0x7f3db41602c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3dac00d150 tx=0x7f3dac01f5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dea166c6f266e63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:48.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.348+0000 7f3db1600640 1 -- 172.21.15.86:0/1399100270 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dac0ad070 con 0x7f3db415feb0 2024-04-25T14:05:48.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.348+0000 7f3db1600640 1 -- 172.21.15.86:0/1399100270 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dac0a7be0 con 0x7f3db415feb0 2024-04-25T14:05:48.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.348+0000 7f3db1600640 1 -- 172.21.15.86:0/1399100270 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dac0b1400 con 0x7f3db415feb0 2024-04-25T14:05:48.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.348+0000 7f3db9e00640 1 -- 172.21.15.86:0/1399100270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3db415feb0 msgr2=0x7f3db41602c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:48.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.348+0000 7f3db9e00640 1 --2- 172.21.15.86:0/1399100270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3db415feb0 0x7f3db41602c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3dac00d150 tx=0x7f3dac01f5d0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.348+0000 7f3db9e00640 1 -- 172.21.15.86:0/1399100270 shutdown_connections 2024-04-25T14:05:48.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.348+0000 7f3db9e00640 1 --2- 172.21.15.86:0/1399100270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3db415feb0 0x7f3db41602c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.348+0000 7f3db9e00640 1 --2- 172.21.15.86:0/1399100270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3db415c2a0 0x7f3db40ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.348+0000 7f3db9e00640 1 -- 172.21.15.86:0/1399100270 >> 172.21.15.86:0/1399100270 conn(0x7f3db40b4650 msgr2=0x7f3db40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:48.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.348+0000 7f3db9e00640 1 -- 172.21.15.86:0/1399100270 shutdown_connections 2024-04-25T14:05:48.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.348+0000 7f3db9e00640 1 -- 172.21.15.86:0/1399100270 wait complete. 2024-04-25T14:05:48.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.348+0000 7f3db9e00640 1 Processor -- start 2024-04-25T14:05:48.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.348+0000 7f3db9e00640 1 -- start start 2024-04-25T14:05:48.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.352+0000 7f3db9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3db406fd50 0x7f3db4166990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.352+0000 7f3db9e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3db415c2a0 0x7f3db4152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.352+0000 7f3db9e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3db415feb0 0x7f3db4152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.352+0000 7f3db9e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3db4160d90 con 0x7f3db406fd50 2024-04-25T14:05:48.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.352+0000 7f3db9e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3db4160c10 con 0x7f3db415feb0 2024-04-25T14:05:48.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.352+0000 7f3db9e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3db40ae670 con 0x7f3db415c2a0 2024-04-25T14:05:48.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.352+0000 7f3db2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3db406fd50 0x7f3db4166990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.352+0000 7f3db2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3db406fd50 0x7f3db4166990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47864/0 (socket says 172.21.15.86:47864) 2024-04-25T14:05:48.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.352+0000 7f3db2a00640 1 -- 172.21.15.86:0/3737598145 learned_addr learned my addr 172.21.15.86:0/3737598145 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:48.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.352+0000 7f3db2000640 1 --2- 172.21.15.86:0/3737598145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3db415c2a0 0x7f3db4152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.352+0000 7f3db3400640 1 --2- 172.21.15.86:0/3737598145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3db415feb0 0x7f3db4152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.352+0000 7f3db2a00640 1 -- 172.21.15.86:0/3737598145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3db415c2a0 msgr2=0x7f3db4152a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:48.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.352+0000 7f3db2a00640 1 --2- 172.21.15.86:0/3737598145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3db415c2a0 0x7f3db4152a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.352+0000 7f3db2a00640 1 -- 172.21.15.86:0/3737598145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3db415feb0 msgr2=0x7f3db4152f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:48.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.352+0000 7f3db2a00640 1 --2- 172.21.15.86:0/3737598145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3db415feb0 0x7f3db4152f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.352+0000 7f3db2a00640 1 -- 172.21.15.86:0/3737598145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db414e230 con 0x7f3db406fd50 2024-04-25T14:05:48.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.352+0000 7f3db2a00640 1 --2- 172.21.15.86:0/3737598145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3db406fd50 0x7f3db4166990 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f3da400e870 tx=0x7f3da400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:48.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.352+0000 7f3dab400640 1 -- 172.21.15.86:0/3737598145 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da4017070 con 0x7f3db406fd50 2024-04-25T14:05:48.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.352+0000 7f3dab400640 1 -- 172.21.15.86:0/3737598145 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3da400ee60 con 0x7f3db406fd50 2024-04-25T14:05:48.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.352+0000 7f3dab400640 1 -- 172.21.15.86:0/3737598145 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da400cb40 con 0x7f3db406fd50 2024-04-25T14:05:48.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.352+0000 7f3db9e00640 1 -- 172.21.15.86:0/3737598145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3db414e510 con 0x7f3db406fd50 2024-04-25T14:05:48.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.352+0000 7f3db9e00640 1 -- 172.21.15.86:0/3737598145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3db414ead0 con 0x7f3db406fd50 2024-04-25T14:05:48.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.352+0000 7f3db9e00640 1 -- 172.21.15.86:0/3737598145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3db4069d00 con 0x7f3db406fd50 2024-04-25T14:05:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.356+0000 7f3dab400640 1 -- 172.21.15.86:0/3737598145 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f3da401a070 con 0x7f3db406fd50 2024-04-25T14:05:48.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.356+0000 7f3dab400640 1 --2- 172.21.15.86:0/3737598145 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f3d9406f570 0x7f3d94071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.356+0000 7f3dab400640 1 -- 172.21.15.86:0/3737598145 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6519+0+0 (secure 0 0 0) 0x7f3da408fa00 con 0x7f3db406fd50 2024-04-25T14:05:48.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.356+0000 7f3db2000640 1 --2- 172.21.15.86:0/3737598145 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f3d9406f570 0x7f3d94071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.356+0000 7f3db2000640 1 --2- 172.21.15.86:0/3737598145 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f3d9406f570 0x7f3d94071a80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3db40b4de0 tx=0x7f3da0009040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:48.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.360+0000 7f3dab400640 1 -- 172.21.15.86:0/3737598145 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3da405bdc0 con 0x7f3db406fd50 2024-04-25T14:05:48.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.576+0000 7f3db9e00640 1 -- 172.21.15.86:0/3737598145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=fs1_data"]} v 0) -- 0x7f3db406d560 con 0x7f3db406fd50 2024-04-25T14:05:48.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.580+0000 7f3dab400640 1 -- 172.21.15.86:0/3737598145 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=fs1_data"]}]=0 updated caps for client.2 v18) ==== 181+0+138 (secure 0 0 0) 0x7f3da40600c0 con 0x7f3db406fd50 2024-04-25T14:05:48.581 INFO:teuthology.orchestra.run.smithi086.stdout:[client.2] 2024-04-25T14:05:48.581 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCLYipmhbhGIRAAxao/J0WY53IITLETFgwIVw== 2024-04-25T14:05:48.581 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-04-25T14:05:48.581 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-04-25T14:05:48.581 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw pool=fs1_data" 2024-04-25T14:05:48.581 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.2 2024-04-25T14:05:48.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.580+0000 7f3db9e00640 1 -- 172.21.15.86:0/3737598145 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f3d9406f570 msgr2=0x7f3d94071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:48.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.580+0000 7f3db9e00640 1 --2- 172.21.15.86:0/3737598145 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f3d9406f570 0x7f3d94071a80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3db40b4de0 tx=0x7f3da0009040 comp rx=0 tx=0).stop 2024-04-25T14:05:48.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.580+0000 7f3db9e00640 1 -- 172.21.15.86:0/3737598145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3db406fd50 msgr2=0x7f3db4166990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:48.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.580+0000 7f3db9e00640 1 --2- 172.21.15.86:0/3737598145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3db406fd50 0x7f3db4166990 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f3da400e870 tx=0x7f3da400ed30 comp rx=0 tx=0).stop 2024-04-25T14:05:48.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.580+0000 7f3db9e00640 1 -- 172.21.15.86:0/3737598145 shutdown_connections 2024-04-25T14:05:48.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.580+0000 7f3db9e00640 1 --2- 172.21.15.86:0/3737598145 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f3d9406f570 0x7f3d94071a80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.580+0000 7f3db9e00640 1 --2- 172.21.15.86:0/3737598145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3db415feb0 0x7f3db4152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.580+0000 7f3db9e00640 1 --2- 172.21.15.86:0/3737598145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3db415c2a0 0x7f3db4152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.580+0000 7f3db9e00640 1 --2- 172.21.15.86:0/3737598145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3db406fd50 0x7f3db4166990 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.580+0000 7f3db9e00640 1 -- 172.21.15.86:0/3737598145 >> 172.21.15.86:0/3737598145 conn(0x7f3db40b4650 msgr2=0x7f3db40ae9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:48.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.580+0000 7f3db9e00640 1 -- 172.21.15.86:0/3737598145 shutdown_connections 2024-04-25T14:05:48.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.580+0000 7f3db9e00640 1 -- 172.21.15.86:0/3737598145 wait complete. 2024-04-25T14:05:48.603 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.3 mds allow mon 'allow r' osd 'allow rw pool=fs1_data' 2024-04-25T14:05:48.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.756+0000 7f98cc000640 1 Processor -- start 2024-04-25T14:05:48.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.756+0000 7f98cc000640 1 -- start start 2024-04-25T14:05:48.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.756+0000 7f98cc000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f98c40cfca0 0x7f98c406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.756+0000 7f98cc000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f98c40d16e0 0x7f98c40d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.756+0000 7f98cc000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98c406c670 0x7f98c406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.756+0000 7f98cc000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f98c4070470 con 0x7f98c40d16e0 2024-04-25T14:05:48.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.756+0000 7f98cc000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f98c40d2030 con 0x7f98c40cfca0 2024-04-25T14:05:48.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.756+0000 7f98cc000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f98c4072bd0 con 0x7f98c406c670 2024-04-25T14:05:48.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.756+0000 7f98c8c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f98c40d16e0 0x7f98c40d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.756+0000 7f98c8c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f98c40d16e0 0x7f98c40d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47868/0 (socket says 172.21.15.86:47868) 2024-04-25T14:05:48.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.756+0000 7f98c8c00640 1 -- 172.21.15.86:0/600143761 learned_addr learned my addr 172.21.15.86:0/600143761 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:48.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.756+0000 7f98c9600640 1 --2- 172.21.15.86:0/600143761 >> v2:172.21.15.155:3300/0 conn(0x7f98c40cfca0 0x7f98c406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.756+0000 7f98ca000640 1 --2- 172.21.15.86:0/600143761 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98c406c670 0x7f98c406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.756+0000 7f98c8c00640 1 -- 172.21.15.86:0/600143761 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98c406c670 msgr2=0x7f98c406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:48.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.756+0000 7f98c8c00640 1 --2- 172.21.15.86:0/600143761 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98c406c670 0x7f98c406ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.756+0000 7f98c8c00640 1 -- 172.21.15.86:0/600143761 >> v2:172.21.15.155:3300/0 conn(0x7f98c40cfca0 msgr2=0x7f98c406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:48.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.756+0000 7f98c8c00640 1 --2- 172.21.15.86:0/600143761 >> v2:172.21.15.155:3300/0 conn(0x7f98c40cfca0 0x7f98c406f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.756+0000 7f98c8c00640 1 -- 172.21.15.86:0/600143761 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c406f930 con 0x7f98c40d16e0 2024-04-25T14:05:48.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.760+0000 7f98c8c00640 1 --2- 172.21.15.86:0/600143761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c40d16e0 0x7f98c40d1af0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f98c40d22a0 tx=0x7f98b001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9319f2994ca70f4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:48.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.760+0000 7f98c3e00640 1 -- 172.21.15.86:0/600143761 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b00ab5f0 con 0x7f98c40d16e0 2024-04-25T14:05:48.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.760+0000 7f98c3e00640 1 -- 172.21.15.86:0/600143761 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98b00b0070 con 0x7f98c40d16e0 2024-04-25T14:05:48.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.760+0000 7f98c3e00640 1 -- 172.21.15.86:0/600143761 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b00ab9b0 con 0x7f98c40d16e0 2024-04-25T14:05:48.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.760+0000 7f98cc000640 1 -- 172.21.15.86:0/600143761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c40d16e0 msgr2=0x7f98c40d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:48.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.760+0000 7f98cc000640 1 --2- 172.21.15.86:0/600143761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c40d16e0 0x7f98c40d1af0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f98c40d22a0 tx=0x7f98b001c520 comp rx=0 tx=0).stop 2024-04-25T14:05:48.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.760+0000 7f98cc000640 1 -- 172.21.15.86:0/600143761 shutdown_connections 2024-04-25T14:05:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.760+0000 7f98cc000640 1 --2- 172.21.15.86:0/600143761 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98c406c670 0x7f98c406ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.760+0000 7f98cc000640 1 --2- 172.21.15.86:0/600143761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c40d16e0 0x7f98c40d1af0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.760+0000 7f98cc000640 1 --2- 172.21.15.86:0/600143761 >> v2:172.21.15.155:3300/0 conn(0x7f98c40cfca0 0x7f98c406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.760+0000 7f98cc000640 1 -- 172.21.15.86:0/600143761 >> 172.21.15.86:0/600143761 conn(0x7f98c40b4650 msgr2=0x7f98c40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.760+0000 7f98cc000640 1 -- 172.21.15.86:0/600143761 shutdown_connections 2024-04-25T14:05:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.760+0000 7f98cc000640 1 -- 172.21.15.86:0/600143761 wait complete. 2024-04-25T14:05:48.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.764+0000 7f98cc000640 1 Processor -- start 2024-04-25T14:05:48.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.764+0000 7f98cc000640 1 -- start start 2024-04-25T14:05:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.764+0000 7f98cc000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98c406c670 0x7f98c40ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.764+0000 7f98cc000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98c40cfca0 0x7f98c409ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.764+0000 7f98cc000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c40d16e0 0x7f98c415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.764+0000 7f98cc000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98c40738c0 con 0x7f98c40d16e0 2024-04-25T14:05:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.764+0000 7f98cc000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f98c406d3d0 con 0x7f98c40cfca0 2024-04-25T14:05:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.764+0000 7f98cc000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f98c4073a40 con 0x7f98c406c670 2024-04-25T14:05:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.764+0000 7f98ca000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c40d16e0 0x7f98c415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.764+0000 7f98ca000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c40d16e0 0x7f98c415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47880/0 (socket says 172.21.15.86:47880) 2024-04-25T14:05:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.764+0000 7f98ca000640 1 -- 172.21.15.86:0/2872669110 learned_addr learned my addr 172.21.15.86:0/2872669110 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.764+0000 7f98c9600640 1 --2- 172.21.15.86:0/2872669110 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98c406c670 0x7f98c40ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.764+0000 7f98c8c00640 1 --2- 172.21.15.86:0/2872669110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98c40cfca0 0x7f98c409ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.764+0000 7f98ca000640 1 -- 172.21.15.86:0/2872669110 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98c406c670 msgr2=0x7f98c40ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.764+0000 7f98ca000640 1 --2- 172.21.15.86:0/2872669110 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98c406c670 0x7f98c40ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.768+0000 7f98ca000640 1 -- 172.21.15.86:0/2872669110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98c40cfca0 msgr2=0x7f98c409ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:48.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.768+0000 7f98ca000640 1 --2- 172.21.15.86:0/2872669110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98c40cfca0 0x7f98c409ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.768+0000 7f98ca000640 1 -- 172.21.15.86:0/2872669110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c40caed0 con 0x7f98c40d16e0 2024-04-25T14:05:48.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.768+0000 7f98ca000640 1 --2- 172.21.15.86:0/2872669110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c40d16e0 0x7f98c415abd0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f98bc007d80 tx=0x7f98bc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:48.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.768+0000 7f98c1600640 1 -- 172.21.15.86:0/2872669110 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc014420 con 0x7f98c40d16e0 2024-04-25T14:05:48.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.768+0000 7f98c1600640 1 -- 172.21.15.86:0/2872669110 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98bc019070 con 0x7f98c40d16e0 2024-04-25T14:05:48.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.768+0000 7f98cc000640 1 -- 172.21.15.86:0/2872669110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c41f84e0 con 0x7f98c40d16e0 2024-04-25T14:05:48.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.768+0000 7f98c1600640 1 -- 172.21.15.86:0/2872669110 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc014800 con 0x7f98c40d16e0 2024-04-25T14:05:48.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.768+0000 7f98cc000640 1 -- 172.21.15.86:0/2872669110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c41f8aa0 con 0x7f98c40d16e0 2024-04-25T14:05:48.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.768+0000 7f98cc000640 1 -- 172.21.15.86:0/2872669110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98c409f4b0 con 0x7f98c40d16e0 2024-04-25T14:05:48.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.768+0000 7f98c1600640 1 -- 172.21.15.86:0/2872669110 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f98bc0149a0 con 0x7f98c40d16e0 2024-04-25T14:05:48.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.772+0000 7f98c1600640 1 --2- 172.21.15.86:0/2872669110 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f98a406f570 0x7f98a4071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.772+0000 7f98c1600640 1 -- 172.21.15.86:0/2872669110 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6519+0+0 (secure 0 0 0) 0x7f98bc08fe70 con 0x7f98c40d16e0 2024-04-25T14:05:48.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.772+0000 7f98c9600640 1 --2- 172.21.15.86:0/2872669110 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f98a406f570 0x7f98a4071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.772+0000 7f98c9600640 1 --2- 172.21.15.86:0/2872669110 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f98a406f570 0x7f98a4071a80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f98c4070710 tx=0x7f98b8008040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:48.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.776+0000 7f98c1600640 1 -- 172.21.15.86:0/2872669110 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f98bc05c350 con 0x7f98c40d16e0 2024-04-25T14:05:48.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.988+0000 7f98cc000640 1 -- 172.21.15.86:0/2872669110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=fs1_data"]} v 0) -- 0x7f98c4076bf0 con 0x7f98c40d16e0 2024-04-25T14:05:48.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.992+0000 7f98c1600640 1 -- 172.21.15.86:0/2872669110 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=fs1_data"]}]=0 updated caps for client.3 v19) ==== 181+0+138 (secure 0 0 0) 0x7f98bc060650 con 0x7f98c40d16e0 2024-04-25T14:05:48.993 INFO:teuthology.orchestra.run.smithi086.stdout:[client.3] 2024-04-25T14:05:48.993 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCLYipmJsCBKRAAzXn2qDtblIpqt7U/TUUlyA== 2024-04-25T14:05:48.993 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-04-25T14:05:48.993 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-04-25T14:05:48.993 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw pool=fs1_data" 2024-04-25T14:05:48.993 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.3 2024-04-25T14:05:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.992+0000 7f98cc000640 1 -- 172.21.15.86:0/2872669110 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f98a406f570 msgr2=0x7f98a4071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.992+0000 7f98cc000640 1 --2- 172.21.15.86:0/2872669110 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f98a406f570 0x7f98a4071a80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f98c4070710 tx=0x7f98b8008040 comp rx=0 tx=0).stop 2024-04-25T14:05:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.992+0000 7f98cc000640 1 -- 172.21.15.86:0/2872669110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c40d16e0 msgr2=0x7f98c415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.992+0000 7f98cc000640 1 --2- 172.21.15.86:0/2872669110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c40d16e0 0x7f98c415abd0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f98bc007d80 tx=0x7f98bc00e4b0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.992+0000 7f98cc000640 1 -- 172.21.15.86:0/2872669110 shutdown_connections 2024-04-25T14:05:48.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.992+0000 7f98cc000640 1 --2- 172.21.15.86:0/2872669110 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f98a406f570 0x7f98a4071a80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.992+0000 7f98cc000640 1 --2- 172.21.15.86:0/2872669110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c40d16e0 0x7f98c415abd0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.992+0000 7f98cc000640 1 --2- 172.21.15.86:0/2872669110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98c40cfca0 0x7f98c409ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.992+0000 7f98cc000640 1 --2- 172.21.15.86:0/2872669110 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98c406c670 0x7f98c40ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.992+0000 7f98cc000640 1 -- 172.21.15.86:0/2872669110 >> 172.21.15.86:0/2872669110 conn(0x7f98c40b4650 msgr2=0x7f98c40b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:48.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.992+0000 7f98cc000640 1 -- 172.21.15.86:0/2872669110 shutdown_connections 2024-04-25T14:05:48.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:48.992+0000 7f98cc000640 1 -- 172.21.15.86:0/2872669110 wait complete. 2024-04-25T14:05:49.015 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T14:05:49.026 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:05:49.062 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:05:49.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.160+0000 7f5ca6000640 1 Processor -- start 2024-04-25T14:05:49.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.164+0000 7f5ca6000640 1 -- start start 2024-04-25T14:05:49.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.164+0000 7f5ca6000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca00cfca0 0x7f5ca006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:49.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.164+0000 7f5ca6000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5ca00730b0 0x7f5ca00734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:49.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.164+0000 7f5ca6000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ca0072bd0 con 0x7f5ca00d16e0 2024-04-25T14:05:49.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.164+0000 7f5ca6000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f5ca0070470 con 0x7f5ca00730b0 2024-04-25T14:05:49.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.164+0000 7f5ca6000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5ca006f930 con 0x7f5ca00cfca0 2024-04-25T14:05:49.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.164+0000 7f5c96a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5ca00d16e0 0x7f5ca00d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53310/0 (socket says 172.21.15.86:53310) 2024-04-25T14:05:49.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.164+0000 7f5c96a00640 1 -- 172.21.15.86:0/2789898448 learned_addr learned my addr 172.21.15.86:0/2789898448 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:49.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.164+0000 7f5c9f400640 1 --2- 172.21.15.86:0/2789898448 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca00cfca0 0x7f5ca006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:49.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.164+0000 7f5c9fe00640 1 --2- 172.21.15.86:0/2789898448 >> v2:172.21.15.155:3300/0 conn(0x7f5ca00730b0 0x7f5ca00734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:49.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.164+0000 7f5c9ea00640 1 -- 172.21.15.86:0/2789898448 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3884365614 0 0) 0x7f5ca0072bd0 con 0x7f5ca00d16e0 2024-04-25T14:05:49.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.164+0000 7f5c9ea00640 1 -- 172.21.15.86:0/2789898448 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c84003530 con 0x7f5ca00d16e0 2024-04-25T14:05:49.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.164+0000 7f5c9f400640 1 -- 172.21.15.86:0/2789898448 >> v2:172.21.15.155:3300/0 conn(0x7f5ca00730b0 msgr2=0x7f5ca00734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:49.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.164+0000 7f5c9f400640 1 --2- 172.21.15.86:0/2789898448 >> v2:172.21.15.155:3300/0 conn(0x7f5ca00730b0 0x7f5ca00734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:49.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.164+0000 7f5c9ea00640 1 -- 172.21.15.86:0/2789898448 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 481457774 0 0) 0x7f5c84003530 con 0x7f5ca00d16e0 2024-04-25T14:05:49.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.164+0000 7f5c9f400640 1 -- 172.21.15.86:0/2789898448 >> v1:172.21.15.86:6789/0 conn(0x7f5ca00d16e0 legacy=0x7f5ca00d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:49.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.164+0000 7f5c9f400640 1 -- 172.21.15.86:0/2789898448 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca006fbd0 con 0x7f5ca00cfca0 2024-04-25T14:05:49.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.168+0000 7f5c9f400640 1 --2- 172.21.15.86:0/2789898448 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca00cfca0 0x7f5ca006f3f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5c880097c0 tx=0x7f5c8801c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4ccd81895cecdd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:49.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.168+0000 7f5c9ea00640 1 -- 172.21.15.86:0/2789898448 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c880b0070 con 0x7f5ca00cfca0 2024-04-25T14:05:49.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.168+0000 7f5c9ea00640 1 -- 172.21.15.86:0/2789898448 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c880ab400 con 0x7f5ca00cfca0 2024-04-25T14:05:49.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.168+0000 7f5c9ea00640 1 -- 172.21.15.86:0/2789898448 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90001680 con 0x7f5ca00cfca0 2024-04-25T14:05:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.168+0000 7f5ca6000640 1 -- 172.21.15.86:0/2789898448 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca00cfca0 msgr2=0x7f5ca006f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.168+0000 7f5ca6000640 1 --2- 172.21.15.86:0/2789898448 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca00cfca0 0x7f5ca006f3f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5c880097c0 tx=0x7f5c8801c3c0 comp rx=0 tx=0).stop 2024-04-25T14:05:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.168+0000 7f5ca6000640 1 -- 172.21.15.86:0/2789898448 shutdown_connections 2024-04-25T14:05:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.168+0000 7f5ca6000640 1 --2- 172.21.15.86:0/2789898448 >> v2:172.21.15.155:3300/0 conn(0x7f5ca00730b0 0x7f5ca00734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.168+0000 7f5ca6000640 1 --2- 172.21.15.86:0/2789898448 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca00cfca0 0x7f5ca006f3f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.168+0000 7f5ca6000640 1 -- 172.21.15.86:0/2789898448 >> 172.21.15.86:0/2789898448 conn(0x7f5ca00b4650 msgr2=0x7f5ca00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.168+0000 7f5ca6000640 1 -- 172.21.15.86:0/2789898448 shutdown_connections 2024-04-25T14:05:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.168+0000 7f5ca6000640 1 -- 172.21.15.86:0/2789898448 wait complete. 2024-04-25T14:05:49.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.172+0000 7f5ca6000640 1 Processor -- start 2024-04-25T14:05:49.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.172+0000 7f5ca6000640 1 -- start start 2024-04-25T14:05:49.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.172+0000 7f5ca6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ca00730b0 0x7f5ca0158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:49.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.172+0000 7f5ca6000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca00cfca0 0x7f5ca00c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.172+0000 7f5ca6000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca00d16e0 0x7f5ca009bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.172+0000 7f5ca6000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca00cdee0 con 0x7f5ca00730b0 2024-04-25T14:05:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.172+0000 7f5ca6000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca00cdd60 con 0x7f5ca00d16e0 2024-04-25T14:05:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.172+0000 7f5ca6000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5ca00ce060 con 0x7f5ca00cfca0 2024-04-25T14:05:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.172+0000 7f5c9f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ca00730b0 0x7f5ca0158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.172+0000 7f5c9f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ca00730b0 0x7f5ca0158e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55032/0 (socket says 172.21.15.86:55032) 2024-04-25T14:05:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.172+0000 7f5c9f400640 1 -- 172.21.15.86:0/3490088896 learned_addr learned my addr 172.21.15.86:0/3490088896 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.172+0000 7f5c96a00640 1 --2- 172.21.15.86:0/3490088896 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca00cfca0 0x7f5ca00c4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.172+0000 7f5c9fe00640 1 --2- 172.21.15.86:0/3490088896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca00d16e0 0x7f5ca009bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.172+0000 7f5c9f400640 1 -- 172.21.15.86:0/3490088896 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca00cfca0 msgr2=0x7f5ca00c4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.172+0000 7f5c9f400640 1 --2- 172.21.15.86:0/3490088896 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca00cfca0 0x7f5ca00c4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.172+0000 7f5c9f400640 1 -- 172.21.15.86:0/3490088896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca00d16e0 msgr2=0x7f5ca009bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.172+0000 7f5c9f400640 1 --2- 172.21.15.86:0/3490088896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca00d16e0 0x7f5ca009bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.172+0000 7f5c9f400640 1 -- 172.21.15.86:0/3490088896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca015fd60 con 0x7f5ca00730b0 2024-04-25T14:05:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.172+0000 7f5c9f400640 1 --2- 172.21.15.86:0/3490088896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ca00730b0 0x7f5ca0158e20 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5c8801cc60 tx=0x7f5c88019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.172+0000 7f5c97e00640 1 -- 172.21.15.86:0/3490088896 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c880ba070 con 0x7f5ca00730b0 2024-04-25T14:05:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.172+0000 7f5c97e00640 1 -- 172.21.15.86:0/3490088896 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c880038e0 con 0x7f5ca00730b0 2024-04-25T14:05:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.172+0000 7f5c97e00640 1 -- 172.21.15.86:0/3490088896 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c880b0040 con 0x7f5ca00730b0 2024-04-25T14:05:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.172+0000 7f5ca6000640 1 -- 172.21.15.86:0/3490088896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca009b450 con 0x7f5ca00730b0 2024-04-25T14:05:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.176+0000 7f5ca6000640 1 -- 172.21.15.86:0/3490088896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca009ba10 con 0x7f5ca00730b0 2024-04-25T14:05:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.176+0000 7f5ca6000640 1 -- 172.21.15.86:0/3490088896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ca009f4b0 con 0x7f5ca00730b0 2024-04-25T14:05:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.176+0000 7f5c97e00640 1 -- 172.21.15.86:0/3490088896 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f5c88003c90 con 0x7f5ca00730b0 2024-04-25T14:05:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.176+0000 7f5c97e00640 1 --2- 172.21.15.86:0/3490088896 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f5c7c06f570 0x7f5c7c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.176+0000 7f5c97e00640 1 -- 172.21.15.86:0/3490088896 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6519+0+0 (secure 0 0 0) 0x7f5c88128990 con 0x7f5ca00730b0 2024-04-25T14:05:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.176+0000 7f5c96a00640 1 --2- 172.21.15.86:0/3490088896 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f5c7c06f570 0x7f5c7c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.180+0000 7f5c96a00640 1 --2- 172.21.15.86:0/3490088896 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f5c7c06f570 0x7f5c7c071a80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5ca00cdaf0 tx=0x7f5c90009040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:49.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.180+0000 7f5c97e00640 1 -- 172.21.15.86:0/3490088896 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5c880efe70 con 0x7f5ca00730b0 2024-04-25T14:05:49.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.372+0000 7f5ca6000640 1 -- 172.21.15.86:0/3490088896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5ca0069d00 con 0x7f5ca00730b0 2024-04-25T14:05:49.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.372+0000 7f5c97e00640 1 -- 172.21.15.86:0/3490088896 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) ==== 53+0+64 (secure 0 0 0) 0x7f5c88015fb0 con 0x7f5ca00730b0 2024-04-25T14:05:49.376 INFO:teuthology.orchestra.run.smithi086.stdout:name: fs1, metadata pool: fs1_metadata, data pools: [fs1_data ] 2024-04-25T14:05:49.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.376+0000 7f5ca6000640 1 -- 172.21.15.86:0/3490088896 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f5c7c06f570 msgr2=0x7f5c7c071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:49.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.376+0000 7f5ca6000640 1 --2- 172.21.15.86:0/3490088896 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f5c7c06f570 0x7f5c7c071a80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5ca00cdaf0 tx=0x7f5c90009040 comp rx=0 tx=0).stop 2024-04-25T14:05:49.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.376+0000 7f5ca6000640 1 -- 172.21.15.86:0/3490088896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ca00730b0 msgr2=0x7f5ca0158e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:49.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.376+0000 7f5ca6000640 1 --2- 172.21.15.86:0/3490088896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ca00730b0 0x7f5ca0158e20 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5c8801cc60 tx=0x7f5c88019160 comp rx=0 tx=0).stop 2024-04-25T14:05:49.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.376+0000 7f5ca6000640 1 -- 172.21.15.86:0/3490088896 shutdown_connections 2024-04-25T14:05:49.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.376+0000 7f5ca6000640 1 --2- 172.21.15.86:0/3490088896 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f5c7c06f570 0x7f5c7c071a80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:49.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.376+0000 7f5ca6000640 1 --2- 172.21.15.86:0/3490088896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca00d16e0 0x7f5ca009bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:49.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.376+0000 7f5ca6000640 1 --2- 172.21.15.86:0/3490088896 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca00cfca0 0x7f5ca00c4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:49.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.376+0000 7f5ca6000640 1 --2- 172.21.15.86:0/3490088896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ca00730b0 0x7f5ca0158e20 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:49.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.376+0000 7f5ca6000640 1 -- 172.21.15.86:0/3490088896 >> 172.21.15.86:0/3490088896 conn(0x7f5ca00b4650 msgr2=0x7f5ca00b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:49.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.376+0000 7f5ca6000640 1 -- 172.21.15.86:0/3490088896 shutdown_connections 2024-04-25T14:05:49.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49.376+0000 7f5ca6000640 1 -- 172.21.15.86:0/3490088896 wait complete. 2024-04-25T14:05:49.397 INFO:tasks.cephfs.mount:Mounting Ceph FS fs1; just confirmed its presence on cluster 2024-04-25T14:05:49.397 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T14:05:49.397 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-25T14:05:49.397 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T14:05:49.397 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T14:05:49.397 INFO:tasks.cephfs.mount:self.cephfs_name = fs1 2024-04-25T14:05:49.397 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T14:05:49.397 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T14:05:49.397 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:05:49.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:49.398 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-25T14:05:49.407 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:05:49.407 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:05:49.407 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:05:49.407 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:05:49.407 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-25T14:05:49.407 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:05:49.407 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:05:49.408 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:05:49.408 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-25T14:05:49.408 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:05:49.408 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:05:49.408 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-25T14:05:49.408 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:05:49.408 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:05:49.408 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T14:05:49.408 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:05:49.408 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T14:05:49.409 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42315sec preferred_lft 42315sec 2024-04-25T14:05:49.409 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-25T14:05:49.409 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:05:49.409 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T14:05:49.409 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:dc:87:9c:96 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:05:49.409 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T14:05:49.409 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:05:49.409 INFO:teuthology.orchestra.run.smithi086.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T14:05:49.410 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T14:05:49.410 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T14:05:49.410 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:05:49.410 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T14:05:49.410 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:05:49.410 INFO:teuthology.orchestra.run.smithi086.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:05:49.410 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T14:05:49.410 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T14:05:49.410 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:05:49.410 INFO:teuthology.orchestra.run.smithi086.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:05:49.410 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T14:05:49.410 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T14:05:49.410 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:05:49.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:49.411 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T14:05:49.469 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-25T14:05:49.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:49.471 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-25T14:05:49.528 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-25T14:05:49.528 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T14:05:49.528 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T14:05:49.528 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T14:05:49.528 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T14:05:49.529 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T14:05:49.529 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T14:05:49.529 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:05:49.529 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:05:49.529 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:05:49.529 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T14:05:49.529 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:05:49.529 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T14:05:49.529 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T14:05:49.529 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T14:05:49.529 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T14:05:49.529 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T14:05:49.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:05:49.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:05:49.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:49.670 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T14:05:49.679 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T14:05:49.679 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T14:05:49.679 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:05:49.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:49.679 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-25T14:05:49.739 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:05:49.739 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T14:05:49.787 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:05:49.787 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:05:49.788 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T14:05:49.839 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T14:05:49.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:49.840 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=fs1 2024-04-25T14:05:49.948 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=fs1 2024-04-25T14:05:49.948 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=fs1". 2024-04-25T14:05:49.948 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:05:49.965 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=fs1 2024-04-25T14:05:49.966 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=fs1". 2024-04-25T14:05:49.966 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:05:49.966 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:05:49.966 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:05:49.976 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=fs1 2024-04-25T14:05:49.976 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=fs1". 2024-04-25T14:05:49.976 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:05:49.976 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:05:49.976 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:05:49.976 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790" 2024-04-25T14:05:49.976 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790:/ 2024-04-25T14:05:49.976 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=fs1,key=0,fsid=8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb" will pass to kernel 2024-04-25T14:05:49.980 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T14:05:49.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:49.980 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T14:05:50.002 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:05:50.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:05:50.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:05:50.107 INFO:teuthology.orchestra.run.smithi086.stdout:client5384 2024-04-25T14:05:50.108 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:05:50.108 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:05:50.117 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:05:50.117 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:05:50.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:50.172 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=pad.bin bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:05:50.255 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:05:50.255 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:05:50.255 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257322 s, 40.7 MB/s 2024-04-25T14:05:50.256 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:05:50.257 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:05:50.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:50.308 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test.bin bs=1M conv=fdatasync count=2 seek=0) 2024-04-25T14:05:50.393 INFO:teuthology.orchestra.run.smithi086.stderr:2+0 records in 2024-04-25T14:05:50.393 INFO:teuthology.orchestra.run.smithi086.stderr:2+0 records out 2024-04-25T14:05:50.393 INFO:teuthology.orchestra.run.smithi086.stderr:2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0315413 s, 66.5 MB/s 2024-04-25T14:05:50.395 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-25T14:05:50.395 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-25T14:05:50.395 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-25T14:05:50.395 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-25T14:05:50.395 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/test.bin").st_ino) 2024-04-25T14:05:50.395 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-25T14:05:50.500 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627777 2024-04-25T14:05:50.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:05:50.509 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:05:50.509 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:05:50.560 INFO:tasks.cephfs.mount:Creating file a 2024-04-25T14:05:50.561 DEBUG:teuthology.orchestra.run.smithi086:> touch /home/ubuntu/cephtest/mnt.0/a 2024-04-25T14:05:50.611 INFO:tasks.cephfs.mount:Creating file b 2024-04-25T14:05:50.612 DEBUG:teuthology.orchestra.run.smithi086:> touch /home/ubuntu/cephtest/mnt.0/b 2024-04-25T14:05:50.664 INFO:tasks.cephfs.mount:Creating file c 2024-04-25T14:05:50.664 DEBUG:teuthology.orchestra.run.smithi086:> touch /home/ubuntu/cephtest/mnt.0/c 2024-04-25T14:05:50.717 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:05:50.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.904+0000 7fde12e00640 1 Processor -- start 2024-04-25T14:05:50.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.904+0000 7fde12e00640 1 -- start start 2024-04-25T14:05:50.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.908+0000 7fde12e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fde0c0d16e0 0x7fde0c0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:50.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.908+0000 7fde12e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde0c0730b0 0x7fde0c0734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:50.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.908+0000 7fde12e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde0c0d2030 con 0x7fde0c0cfca0 2024-04-25T14:05:50.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.908+0000 7fde12e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fde0c070470 con 0x7fde0c0d16e0 2024-04-25T14:05:50.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.908+0000 7fde12e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fde0c072bd0 con 0x7fde0c0730b0 2024-04-25T14:05:50.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.908+0000 7fde0b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fde0c0d16e0 0x7fde0c0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:50.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.908+0000 7fde0b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fde0c0d16e0 0x7fde0c0d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55036/0 (socket says 172.21.15.86:55036) 2024-04-25T14:05:50.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.908+0000 7fde0b400640 1 -- 172.21.15.86:0/4252447145 learned_addr learned my addr 172.21.15.86:0/4252447145 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:50.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.908+0000 7fde10e00640 1 --2- 172.21.15.86:0/4252447145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde0c0730b0 0x7fde0c0734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:50.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.908+0000 7fde0b400640 1 -- 172.21.15.86:0/4252447145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde0c0730b0 msgr2=0x7fde0c0734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:50.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.908+0000 7fde0b400640 1 --2- 172.21.15.86:0/4252447145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde0c0730b0 0x7fde0c0734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:50.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.908+0000 7fde0b400640 1 -- 172.21.15.86:0/4252447145 >> v1:172.21.15.155:6789/0 conn(0x7fde0c0cfca0 legacy=0x7fde0c06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:50.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.908+0000 7fde0b400640 1 -- 172.21.15.86:0/4252447145 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0c06fbd0 con 0x7fde0c0d16e0 2024-04-25T14:05:50.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.912+0000 7fde0b400640 1 --2- 172.21.15.86:0/4252447145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde0c0d16e0 0x7fde0c0d1af0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009870 tx=0x7fddf801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e756fd2d62b73716 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:50.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.912+0000 7fde0aa00640 1 -- 172.21.15.86:0/4252447145 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf80a7070 con 0x7fde0c0d16e0 2024-04-25T14:05:50.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.912+0000 7fde0aa00640 1 -- 172.21.15.86:0/4252447145 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddf80b0070 con 0x7fde0c0d16e0 2024-04-25T14:05:50.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.912+0000 7fde0aa00640 1 -- 172.21.15.86:0/4252447145 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf80ab890 con 0x7fde0c0d16e0 2024-04-25T14:05:50.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.912+0000 7fde12e00640 1 -- 172.21.15.86:0/4252447145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde0c0d16e0 msgr2=0x7fde0c0d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:50.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.912+0000 7fde12e00640 1 --2- 172.21.15.86:0/4252447145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde0c0d16e0 0x7fde0c0d1af0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009870 tx=0x7fddf801c560 comp rx=0 tx=0).stop 2024-04-25T14:05:50.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.912+0000 7fde12e00640 1 -- 172.21.15.86:0/4252447145 shutdown_connections 2024-04-25T14:05:50.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.912+0000 7fde12e00640 1 --2- 172.21.15.86:0/4252447145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde0c0730b0 0x7fde0c0734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:50.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.912+0000 7fde12e00640 1 --2- 172.21.15.86:0/4252447145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde0c0d16e0 0x7fde0c0d1af0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:50.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.912+0000 7fde12e00640 1 -- 172.21.15.86:0/4252447145 >> 172.21.15.86:0/4252447145 conn(0x7fde0c0b4650 msgr2=0x7fde0c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:50.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.912+0000 7fde12e00640 1 -- 172.21.15.86:0/4252447145 shutdown_connections 2024-04-25T14:05:50.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.912+0000 7fde12e00640 1 -- 172.21.15.86:0/4252447145 wait complete. 2024-04-25T14:05:50.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.912+0000 7fde12e00640 1 Processor -- start 2024-04-25T14:05:50.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.912+0000 7fde12e00640 1 -- start start 2024-04-25T14:05:50.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.916+0000 7fde12e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde0c0730b0 0x7fde0c157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:50.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.916+0000 7fde12e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde0c0cfca0 0x7fde0c0cc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:50.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.916+0000 7fde12e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde0c0d16e0 0x7fde0c0ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:50.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.916+0000 7fde12e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c169a70 con 0x7fde0c0730b0 2024-04-25T14:05:50.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.916+0000 7fde12e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c1698f0 con 0x7fde0c0cfca0 2024-04-25T14:05:50.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.916+0000 7fde12e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fde0c169bf0 con 0x7fde0c0d16e0 2024-04-25T14:05:50.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.916+0000 7fde0be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde0c0730b0 0x7fde0c157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:50.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.916+0000 7fde0be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde0c0730b0 0x7fde0c157ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55050/0 (socket says 172.21.15.86:55050) 2024-04-25T14:05:50.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.916+0000 7fde0be00640 1 -- 172.21.15.86:0/903614501 learned_addr learned my addr 172.21.15.86:0/903614501 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:50.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.916+0000 7fde10e00640 1 --2- 172.21.15.86:0/903614501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde0c0d16e0 0x7fde0c0ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:50.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.916+0000 7fde0b400640 1 --2- 172.21.15.86:0/903614501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde0c0cfca0 0x7fde0c0cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:50.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.916+0000 7fde0be00640 1 -- 172.21.15.86:0/903614501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde0c0d16e0 msgr2=0x7fde0c0ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:50.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.916+0000 7fde0be00640 1 --2- 172.21.15.86:0/903614501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde0c0d16e0 0x7fde0c0ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:50.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.916+0000 7fde0be00640 1 -- 172.21.15.86:0/903614501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde0c0cfca0 msgr2=0x7fde0c0cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:50.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.916+0000 7fde0be00640 1 --2- 172.21.15.86:0/903614501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde0c0cfca0 0x7fde0c0cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:50.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.916+0000 7fde0be00640 1 -- 172.21.15.86:0/903614501 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0c159bb0 con 0x7fde0c0730b0 2024-04-25T14:05:50.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.916+0000 7fde10e00640 1 --2- 172.21.15.86:0/903614501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde0c0d16e0 0x7fde0c0ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:50.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.916+0000 7fde0be00640 1 --2- 172.21.15.86:0/903614501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde0c0730b0 0x7fde0c157ad0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00c340 tx=0x7fddfc00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:50.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.916+0000 7fde03e00640 1 -- 172.21.15.86:0/903614501 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc0052d0 con 0x7fde0c0730b0 2024-04-25T14:05:50.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.916+0000 7fde03e00640 1 -- 172.21.15.86:0/903614501 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddfc005470 con 0x7fde0c0730b0 2024-04-25T14:05:50.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.916+0000 7fde12e00640 1 -- 172.21.15.86:0/903614501 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde0c1584f0 con 0x7fde0c0730b0 2024-04-25T14:05:50.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.916+0000 7fde03e00640 1 -- 172.21.15.86:0/903614501 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc007260 con 0x7fde0c0730b0 2024-04-25T14:05:50.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.916+0000 7fde12e00640 1 -- 172.21.15.86:0/903614501 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde0c0cea30 con 0x7fde0c0730b0 2024-04-25T14:05:50.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.916+0000 7fde12e00640 1 -- 172.21.15.86:0/903614501 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde0c09f4b0 con 0x7fde0c0730b0 2024-04-25T14:05:50.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.920+0000 7fde03e00640 1 -- 172.21.15.86:0/903614501 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7fddfc007400 con 0x7fde0c0730b0 2024-04-25T14:05:50.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.920+0000 7fde03e00640 1 --2- 172.21.15.86:0/903614501 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fddec06f570 0x7fddec071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:50.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.920+0000 7fde03e00640 1 -- 172.21.15.86:0/903614501 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6519+0+0 (secure 0 0 0) 0x7fddfc08eb60 con 0x7fde0c0730b0 2024-04-25T14:05:50.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.920+0000 7fde0b400640 1 --2- 172.21.15.86:0/903614501 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fddec06f570 0x7fddec071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:50.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.920+0000 7fde0b400640 1 --2- 172.21.15.86:0/903614501 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fddec06f570 0x7fddec071a80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fddf80af000 tx=0x7fddf80a7000 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:50.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:50.924+0000 7fde03e00640 1 -- 172.21.15.86:0/903614501 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fde0c0d2030 con 0x7fde0c0730b0 2024-04-25T14:05:51.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.080+0000 7fde12e00640 1 -- 172.21.15.86:0/903614501 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde0c069d00 con 0x7fde0c0730b0 2024-04-25T14:05:51.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.080+0000 7fde03e00640 1 -- 172.21.15.86:0/903614501 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+17563 (secure 0 0 0) 0x7fddfc05b050 con 0x7fde0c0730b0 2024-04-25T14:05:51.083 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:51.083 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":50,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:05:42.882894+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"fs1_metadata","create_time":"2024-04-25T14:05:30.939601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"fs1"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"fs1_data","create_time":"2024-04-25T14:05:31.898262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"fs1"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/615772001":"2024-04-26T14:05:42.882867+0000","172.21.15.86:6835/615772001":"2024-04-26T14:05:42.882867+0000","172.21.15.86:6835/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:0/2073988773":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6833/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6832/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/3558294881":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/1843278167":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6837/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:0/2602032945":"2024-04-26T14:05:21.460855+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:05:51.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.084+0000 7fde12e00640 1 -- 172.21.15.86:0/903614501 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fddec06f570 msgr2=0x7fddec071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:51.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.084+0000 7fde12e00640 1 --2- 172.21.15.86:0/903614501 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fddec06f570 0x7fddec071a80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fddf80af000 tx=0x7fddf80a7000 comp rx=0 tx=0).stop 2024-04-25T14:05:51.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.084+0000 7fde12e00640 1 -- 172.21.15.86:0/903614501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde0c0730b0 msgr2=0x7fde0c157ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:51.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.084+0000 7fde12e00640 1 --2- 172.21.15.86:0/903614501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde0c0730b0 0x7fde0c157ad0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00c340 tx=0x7fddfc00c800 comp rx=0 tx=0).stop 2024-04-25T14:05:51.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.084+0000 7fde12e00640 1 -- 172.21.15.86:0/903614501 shutdown_connections 2024-04-25T14:05:51.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.084+0000 7fde12e00640 1 --2- 172.21.15.86:0/903614501 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fddec06f570 0x7fddec071a80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.084+0000 7fde12e00640 1 --2- 172.21.15.86:0/903614501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde0c0d16e0 0x7fde0c0ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.084+0000 7fde12e00640 1 --2- 172.21.15.86:0/903614501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde0c0cfca0 0x7fde0c0cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.084+0000 7fde12e00640 1 --2- 172.21.15.86:0/903614501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde0c0730b0 0x7fde0c157ad0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.084+0000 7fde12e00640 1 -- 172.21.15.86:0/903614501 >> 172.21.15.86:0/903614501 conn(0x7fde0c0b4650 msgr2=0x7fde0c156590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:51.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.084+0000 7fde12e00640 1 -- 172.21.15.86:0/903614501 shutdown_connections 2024-04-25T14:05:51.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.084+0000 7fde12e00640 1 -- 172.21.15.86:0/903614501 wait complete. 2024-04-25T14:05:51.105 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'fs1_metadata', 'create_time': '2024-04-25T14:05:30.939601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'fs1'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'fs1_data', 'create_time': '2024-04-25T14:05:31.898262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'fs1'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:05:51.106 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:05:51.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.252+0000 7feceec00640 1 Processor -- start 2024-04-25T14:05:51.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.252+0000 7feceec00640 1 -- start start 2024-04-25T14:05:51.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.256+0000 7feceec00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fece815c2a0 0x7fece80ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:51.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.256+0000 7feceec00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fece815feb0 0x7fece81602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:51.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.256+0000 7feceec00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fece80b0a60 con 0x7fece806fd50 2024-04-25T14:05:51.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.256+0000 7feceec00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fece806ef90 con 0x7fece815c2a0 2024-04-25T14:05:51.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.256+0000 7feceec00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fece806f270 con 0x7fece815feb0 2024-04-25T14:05:51.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.256+0000 7fece7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fece815c2a0 0x7fece80ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:51.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.256+0000 7fece7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fece815c2a0 0x7fece80ae130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55056/0 (socket says 172.21.15.86:55056) 2024-04-25T14:05:51.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.256+0000 7fece7e00640 1 -- 172.21.15.86:0/4090476245 learned_addr learned my addr 172.21.15.86:0/4090476245 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:51.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.256+0000 7fececc00640 1 --2- 172.21.15.86:0/4090476245 >> v2:172.21.15.155:3300/0 conn(0x7fece815feb0 0x7fece81602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:51.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.256+0000 7fece7e00640 1 -- 172.21.15.86:0/4090476245 >> v2:172.21.15.155:3300/0 conn(0x7fece815feb0 msgr2=0x7fece81602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:51.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.256+0000 7fece7e00640 1 --2- 172.21.15.86:0/4090476245 >> v2:172.21.15.155:3300/0 conn(0x7fece815feb0 0x7fece81602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.256+0000 7fece7e00640 1 -- 172.21.15.86:0/4090476245 >> v1:172.21.15.155:6789/0 conn(0x7fece806fd50 legacy=0x7fece8070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:51.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.256+0000 7fece7e00640 1 -- 172.21.15.86:0/4090476245 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece806f550 con 0x7fece815c2a0 2024-04-25T14:05:51.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.256+0000 7fececc00640 1 --2- 172.21.15.86:0/4090476245 >> v2:172.21.15.155:3300/0 conn(0x7fece815feb0 0x7fece81602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:51.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.260+0000 7fece7e00640 1 --2- 172.21.15.86:0/4090476245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece815c2a0 0x7fece80ae130 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fece80708a0 tx=0x7fecd801ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad3af33f3dba192 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:51.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.260+0000 7fece7400640 1 -- 172.21.15.86:0/4090476245 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd80b0070 con 0x7fece815c2a0 2024-04-25T14:05:51.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.260+0000 7fece7400640 1 -- 172.21.15.86:0/4090476245 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecd80ab600 con 0x7fece815c2a0 2024-04-25T14:05:51.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.260+0000 7fece7400640 1 -- 172.21.15.86:0/4090476245 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd80ab9d0 con 0x7fece815c2a0 2024-04-25T14:05:51.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.260+0000 7feceec00640 1 -- 172.21.15.86:0/4090476245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece815c2a0 msgr2=0x7fece80ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:51.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.260+0000 7feceec00640 1 --2- 172.21.15.86:0/4090476245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece815c2a0 0x7fece80ae130 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fece80708a0 tx=0x7fecd801ce80 comp rx=0 tx=0).stop 2024-04-25T14:05:51.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.260+0000 7feceec00640 1 -- 172.21.15.86:0/4090476245 shutdown_connections 2024-04-25T14:05:51.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.260+0000 7feceec00640 1 --2- 172.21.15.86:0/4090476245 >> v2:172.21.15.155:3300/0 conn(0x7fece815feb0 0x7fece81602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.260+0000 7feceec00640 1 --2- 172.21.15.86:0/4090476245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece815c2a0 0x7fece80ae130 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.260+0000 7feceec00640 1 -- 172.21.15.86:0/4090476245 >> 172.21.15.86:0/4090476245 conn(0x7fece80b4650 msgr2=0x7fece80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:51.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.260+0000 7feceec00640 1 -- 172.21.15.86:0/4090476245 shutdown_connections 2024-04-25T14:05:51.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.260+0000 7feceec00640 1 -- 172.21.15.86:0/4090476245 wait complete. 2024-04-25T14:05:51.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.264+0000 7feceec00640 1 Processor -- start 2024-04-25T14:05:51.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.264+0000 7feceec00640 1 -- start start 2024-04-25T14:05:51.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.264+0000 7feceec00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fece806fd50 0x7fece8159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:51.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.264+0000 7feceec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece815c2a0 0x7fece815a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:51.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.264+0000 7feceec00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fece815feb0 0x7fece8096ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:51.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.264+0000 7feceec00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fece8160d90 con 0x7fece815c2a0 2024-04-25T14:05:51.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.264+0000 7feceec00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fece8160c10 con 0x7fece815feb0 2024-04-25T14:05:51.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.264+0000 7feceec00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fece80ae670 con 0x7fece806fd50 2024-04-25T14:05:51.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.264+0000 7fecdf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece815c2a0 0x7fece815a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:51.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.264+0000 7fecdf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece815c2a0 0x7fece815a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55058/0 (socket says 172.21.15.86:55058) 2024-04-25T14:05:51.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.264+0000 7fecdf400640 1 -- 172.21.15.86:0/2076811695 learned_addr learned my addr 172.21.15.86:0/2076811695 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:51.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.264+0000 7fece7e00640 1 --2- 172.21.15.86:0/2076811695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fece806fd50 0x7fece8159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:51.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.264+0000 7fececc00640 1 --2- 172.21.15.86:0/2076811695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fece815feb0 0x7fece8096ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:51.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.264+0000 7fecdf400640 1 -- 172.21.15.86:0/2076811695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fece806fd50 msgr2=0x7fece8159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:51.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.264+0000 7fecdf400640 1 --2- 172.21.15.86:0/2076811695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fece806fd50 0x7fece8159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.264+0000 7fecdf400640 1 -- 172.21.15.86:0/2076811695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fece815feb0 msgr2=0x7fece8096ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:51.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.264+0000 7fecdf400640 1 --2- 172.21.15.86:0/2076811695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fece815feb0 0x7fece8096ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.264+0000 7fecdf400640 1 -- 172.21.15.86:0/2076811695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece809aa80 con 0x7fece815c2a0 2024-04-25T14:05:51.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.264+0000 7fecdf400640 1 --2- 172.21.15.86:0/2076811695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece815c2a0 0x7fece815a040 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fecd000b410 tx=0x7fecd000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:51.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.264+0000 7fece4c00640 1 -- 172.21.15.86:0/2076811695 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd0015030 con 0x7fece815c2a0 2024-04-25T14:05:51.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.264+0000 7fece4c00640 1 -- 172.21.15.86:0/2076811695 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecd0004020 con 0x7fece815c2a0 2024-04-25T14:05:51.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.264+0000 7feceec00640 1 -- 172.21.15.86:0/2076811695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fece809ad60 con 0x7fece815c2a0 2024-04-25T14:05:51.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.264+0000 7fece4c00640 1 -- 172.21.15.86:0/2076811695 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd0004410 con 0x7fece815c2a0 2024-04-25T14:05:51.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.264+0000 7feceec00640 1 -- 172.21.15.86:0/2076811695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fece815a580 con 0x7fece815c2a0 2024-04-25T14:05:51.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.268+0000 7feceec00640 1 -- 172.21.15.86:0/2076811695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fece80c3750 con 0x7fece815c2a0 2024-04-25T14:05:51.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.268+0000 7fece4c00640 1 -- 172.21.15.86:0/2076811695 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7fecd0018070 con 0x7fece815c2a0 2024-04-25T14:05:51.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.268+0000 7fece4c00640 1 --2- 172.21.15.86:0/2076811695 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fecc406f570 0x7fecc4071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:51.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.268+0000 7fece4c00640 1 -- 172.21.15.86:0/2076811695 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6519+0+0 (secure 0 0 0) 0x7fecd008f2e0 con 0x7fece815c2a0 2024-04-25T14:05:51.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.268+0000 7fece7e00640 1 --2- 172.21.15.86:0/2076811695 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fecc406f570 0x7fecc4071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:51.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.268+0000 7fece7e00640 1 --2- 172.21.15.86:0/2076811695 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fecc406f570 0x7fecc4071a80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fecd80ad000 tx=0x7fecd8004740 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:51.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.272+0000 7fece4c00640 1 -- 172.21.15.86:0/2076811695 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fecd005b6a0 con 0x7fece815c2a0 2024-04-25T14:05:51.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.428+0000 7feceec00640 1 -- 172.21.15.86:0/2076811695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fece8069d00 con 0x7fece815c2a0 2024-04-25T14:05:51.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.428+0000 7fece4c00640 1 -- 172.21.15.86:0/2076811695 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) ==== 93+0+10 (secure 0 0 0) 0x7fecd005f9a0 con 0x7fece815c2a0 2024-04-25T14:05:51.432 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:05:51.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.432+0000 7feceec00640 1 -- 172.21.15.86:0/2076811695 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fecc406f570 msgr2=0x7fecc4071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:51.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.432+0000 7feceec00640 1 --2- 172.21.15.86:0/2076811695 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fecc406f570 0x7fecc4071a80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fecd80ad000 tx=0x7fecd8004740 comp rx=0 tx=0).stop 2024-04-25T14:05:51.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.432+0000 7feceec00640 1 -- 172.21.15.86:0/2076811695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece815c2a0 msgr2=0x7fece815a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:51.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.432+0000 7feceec00640 1 --2- 172.21.15.86:0/2076811695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece815c2a0 0x7fece815a040 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fecd000b410 tx=0x7fecd000b8d0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.432+0000 7feceec00640 1 -- 172.21.15.86:0/2076811695 shutdown_connections 2024-04-25T14:05:51.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.432+0000 7feceec00640 1 --2- 172.21.15.86:0/2076811695 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fecc406f570 0x7fecc4071a80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.432+0000 7feceec00640 1 --2- 172.21.15.86:0/2076811695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fece815feb0 0x7fece8096ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.432+0000 7feceec00640 1 --2- 172.21.15.86:0/2076811695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece815c2a0 0x7fece815a040 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.432+0000 7feceec00640 1 --2- 172.21.15.86:0/2076811695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fece806fd50 0x7fece8159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.432+0000 7feceec00640 1 -- 172.21.15.86:0/2076811695 >> 172.21.15.86:0/2076811695 conn(0x7fece80b4650 msgr2=0x7fece8155ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:51.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.432+0000 7feceec00640 1 -- 172.21.15.86:0/2076811695 shutdown_connections 2024-04-25T14:05:51.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.432+0000 7feceec00640 1 -- 172.21.15.86:0/2076811695 wait complete. 2024-04-25T14:05:51.453 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get fs1_metadata pg_num 2024-04-25T14:05:51.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.604+0000 7f0407a00640 1 Processor -- start 2024-04-25T14:05:51.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.604+0000 7f0407a00640 1 -- start start 2024-04-25T14:05:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.604+0000 7f0407a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04000b59e0 0x7f04000b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.604+0000 7f0407a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f04000aeea0 0x7f04000af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.604+0000 7f0407a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04000b6330 con 0x7f040015faa0 2024-04-25T14:05:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.604+0000 7f0407a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f040006d560 con 0x7f04000aeea0 2024-04-25T14:05:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.604+0000 7f0407a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f040006c670 con 0x7f04000b59e0 2024-04-25T14:05:51.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.604+0000 7f03ffe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04000b59e0 0x7f04000b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:51.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.604+0000 7f0405a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f04000aeea0 0x7f04000af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:51.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.604+0000 7f0405a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f04000aeea0 0x7f04000af2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:39660/0 (socket says 172.21.15.86:39660) 2024-04-25T14:05:51.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.604+0000 7f0405a00640 1 -- 172.21.15.86:0/637489430 learned_addr learned my addr 172.21.15.86:0/637489430 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:51.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.604+0000 7f03ffe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04000b59e0 0x7f04000b5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:34612/0 (socket says 172.21.15.86:34612) 2024-04-25T14:05:51.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.604+0000 7f03ffe00640 1 -- 172.21.15.86:0/637489430 >> v2:172.21.15.155:3300/0 conn(0x7f04000aeea0 msgr2=0x7f04000af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:51.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.604+0000 7f03ffe00640 1 --2- 172.21.15.86:0/637489430 >> v2:172.21.15.155:3300/0 conn(0x7f04000aeea0 0x7f04000af2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.604+0000 7f03ffe00640 1 -- 172.21.15.86:0/637489430 >> v1:172.21.15.155:6789/0 conn(0x7f040015faa0 legacy=0x7f040006d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:51.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.604+0000 7f03ffe00640 1 -- 172.21.15.86:0/637489430 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040006c950 con 0x7f04000b59e0 2024-04-25T14:05:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.608+0000 7f03ffe00640 1 --2- 172.21.15.86:0/637489430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04000b59e0 0x7f04000b5df0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0097c0 tx=0x7f03ec01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e14f244aa0819ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.608+0000 7f03ff400640 1 -- 172.21.15.86:0/637489430 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec0aa620 con 0x7f04000b59e0 2024-04-25T14:05:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.608+0000 7f03ff400640 1 -- 172.21.15.86:0/637489430 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03ec0af070 con 0x7f04000b59e0 2024-04-25T14:05:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.608+0000 7f03ff400640 1 -- 172.21.15.86:0/637489430 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec0aa9e0 con 0x7f04000b59e0 2024-04-25T14:05:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.608+0000 7f0407a00640 1 -- 172.21.15.86:0/637489430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04000b59e0 msgr2=0x7f04000b5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.608+0000 7f0407a00640 1 --2- 172.21.15.86:0/637489430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04000b59e0 0x7f04000b5df0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0097c0 tx=0x7f03ec01c4a0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.608+0000 7f0407a00640 1 -- 172.21.15.86:0/637489430 shutdown_connections 2024-04-25T14:05:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.608+0000 7f0407a00640 1 --2- 172.21.15.86:0/637489430 >> v2:172.21.15.155:3300/0 conn(0x7f04000aeea0 0x7f04000af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.608+0000 7f0407a00640 1 --2- 172.21.15.86:0/637489430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04000b59e0 0x7f04000b5df0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.608+0000 7f0407a00640 1 -- 172.21.15.86:0/637489430 >> 172.21.15.86:0/637489430 conn(0x7f04000d2230 msgr2=0x7f04000d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.608+0000 7f0407a00640 1 -- 172.21.15.86:0/637489430 shutdown_connections 2024-04-25T14:05:51.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.608+0000 7f0407a00640 1 -- 172.21.15.86:0/637489430 wait complete. 2024-04-25T14:05:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.612+0000 7f0407a00640 1 Processor -- start 2024-04-25T14:05:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.612+0000 7f0407a00640 1 -- start start 2024-04-25T14:05:51.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.612+0000 7f0407a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04000aeea0 0x7f040015bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:51.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.612+0000 7f0407a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04000b59e0 0x7f04001514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:51.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.612+0000 7f0407a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f040015faa0 0x7f0400151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.612+0000 7f0407a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0400160030 con 0x7f040015faa0 2024-04-25T14:05:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.612+0000 7f0407a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f040015feb0 con 0x7f04000aeea0 2024-04-25T14:05:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.612+0000 7f0407a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0400070f30 con 0x7f04000b59e0 2024-04-25T14:05:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.612+0000 7f0405a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f040015faa0 0x7f0400151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.612+0000 7f0405a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f040015faa0 0x7f0400151a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55062/0 (socket says 172.21.15.86:55062) 2024-04-25T14:05:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.612+0000 7f0405a00640 1 -- 172.21.15.86:0/801121934 learned_addr learned my addr 172.21.15.86:0/801121934 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.612+0000 7f0405a00640 1 -- 172.21.15.86:0/801121934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04000b59e0 msgr2=0x7f04001514e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.612+0000 7f03ffe00640 1 --2- 172.21.15.86:0/801121934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04000b59e0 0x7f04001514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.612+0000 7f0405a00640 1 --2- 172.21.15.86:0/801121934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04000b59e0 0x7f04001514e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.612+0000 7f0405000640 1 --2- 172.21.15.86:0/801121934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04000aeea0 0x7f040015bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.612+0000 7f0405a00640 1 -- 172.21.15.86:0/801121934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04000aeea0 msgr2=0x7f040015bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.612+0000 7f0405a00640 1 --2- 172.21.15.86:0/801121934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04000aeea0 0x7f040015bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.612+0000 7f0405a00640 1 -- 172.21.15.86:0/801121934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040015e790 con 0x7f040015faa0 2024-04-25T14:05:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.612+0000 7f03ffe00640 1 --2- 172.21.15.86:0/801121934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04000b59e0 0x7f04001514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:51.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.612+0000 7f0405000640 1 --2- 172.21.15.86:0/801121934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04000aeea0 0x7f040015bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:51.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.616+0000 7f0405a00640 1 --2- 172.21.15.86:0/801121934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f040015faa0 0x7f0400151a20 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f03f800efc0 tx=0x7f03f800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:51.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.616+0000 7f03fcc00640 1 -- 172.21.15.86:0/801121934 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f800cd20 con 0x7f040015faa0 2024-04-25T14:05:51.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.616+0000 7f03fcc00640 1 -- 172.21.15.86:0/801121934 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f80039a0 con 0x7f040015faa0 2024-04-25T14:05:51.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.616+0000 7f03fcc00640 1 -- 172.21.15.86:0/801121934 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8003d90 con 0x7f040015faa0 2024-04-25T14:05:51.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.616+0000 7f0407a00640 1 -- 172.21.15.86:0/801121934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0400162a40 con 0x7f040015faa0 2024-04-25T14:05:51.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.616+0000 7f0407a00640 1 -- 172.21.15.86:0/801121934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f040015adf0 con 0x7f040015faa0 2024-04-25T14:05:51.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.616+0000 7f0407a00640 1 -- 172.21.15.86:0/801121934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0400074cf0 con 0x7f040015faa0 2024-04-25T14:05:51.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.616+0000 7f03fcc00640 1 -- 172.21.15.86:0/801121934 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f03f8012050 con 0x7f040015faa0 2024-04-25T14:05:51.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.616+0000 7f03fcc00640 1 --2- 172.21.15.86:0/801121934 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f03e006f5a0 0x7f03e0071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:51.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.616+0000 7f03fcc00640 1 -- 172.21.15.86:0/801121934 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6519+0+0 (secure 0 0 0) 0x7f03f808fd00 con 0x7f040015faa0 2024-04-25T14:05:51.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.616+0000 7f0405000640 1 --2- 172.21.15.86:0/801121934 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f03e006f5a0 0x7f03e0071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:51.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.620+0000 7f0405000640 1 --2- 172.21.15.86:0/801121934 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f03e006f5a0 0x7f03e0071ab0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f03f4001650 tx=0x7f03f4009040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:51.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.624+0000 7f03fcc00640 1 -- 172.21.15.86:0/801121934 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f03f8022020 con 0x7f040015faa0 2024-04-25T14:05:51.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.780+0000 7f0407a00640 1 -- 172.21.15.86:0/801121934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "fs1_metadata", "var": "pg_num"} v 0) -- 0x7f04000b55b0 con 0x7f040015faa0 2024-04-25T14:05:51.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.780+0000 7f03fcc00640 1 -- 172.21.15.86:0/801121934 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "fs1_metadata", "var": "pg_num"}]=0 v50) ==== 101+0+11 (secure 0 0 0) 0x7f03f8010040 con 0x7f040015faa0 2024-04-25T14:05:51.784 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:05:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.784+0000 7f0407a00640 1 -- 172.21.15.86:0/801121934 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f03e006f5a0 msgr2=0x7f03e0071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.784+0000 7f0407a00640 1 --2- 172.21.15.86:0/801121934 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f03e006f5a0 0x7f03e0071ab0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f03f4001650 tx=0x7f03f4009040 comp rx=0 tx=0).stop 2024-04-25T14:05:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.784+0000 7f0407a00640 1 -- 172.21.15.86:0/801121934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f040015faa0 msgr2=0x7f0400151a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.784+0000 7f0407a00640 1 --2- 172.21.15.86:0/801121934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f040015faa0 0x7f0400151a20 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f03f800efc0 tx=0x7f03f800c490 comp rx=0 tx=0).stop 2024-04-25T14:05:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.784+0000 7f0407a00640 1 -- 172.21.15.86:0/801121934 shutdown_connections 2024-04-25T14:05:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.784+0000 7f0407a00640 1 --2- 172.21.15.86:0/801121934 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f03e006f5a0 0x7f03e0071ab0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.784+0000 7f0407a00640 1 --2- 172.21.15.86:0/801121934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f040015faa0 0x7f0400151a20 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.784+0000 7f0407a00640 1 --2- 172.21.15.86:0/801121934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04000b59e0 0x7f04001514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.784+0000 7f0407a00640 1 --2- 172.21.15.86:0/801121934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04000aeea0 0x7f040015bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.784+0000 7f0407a00640 1 -- 172.21.15.86:0/801121934 >> 172.21.15.86:0/801121934 conn(0x7f04000d2230 msgr2=0x7f040015ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:51.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.784+0000 7f0407a00640 1 -- 172.21.15.86:0/801121934 shutdown_connections 2024-04-25T14:05:51.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.784+0000 7f0407a00640 1 -- 172.21.15.86:0/801121934 wait complete. 2024-04-25T14:05:51.805 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get fs1_data pg_num 2024-04-25T14:05:51.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.952+0000 7f3905200640 1 Processor -- start 2024-04-25T14:05:51.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.952+0000 7f3905200640 1 -- start start 2024-04-25T14:05:51.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.956+0000 7f3905200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f39000b7a30 0x7f39000b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:51.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.956+0000 7f3905200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f39000d15e0 0x7f39000d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:51.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.956+0000 7f3905200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39000b3750 con 0x7f39001610d0 2024-04-25T14:05:51.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.956+0000 7f3905200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3900161620 con 0x7f39000d15e0 2024-04-25T14:05:51.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.956+0000 7f3905200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f39000ce7b0 con 0x7f39000b7a30 2024-04-25T14:05:51.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.956+0000 7f38fe800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f39000d15e0 0x7f39000d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:51.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.956+0000 7f38fde00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f39001610d0 0x7f39000b2250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53316/0 (socket says 172.21.15.86:53316) 2024-04-25T14:05:51.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.956+0000 7f38fde00640 1 -- 172.21.15.86:0/3664476288 learned_addr learned my addr 172.21.15.86:0/3664476288 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:51.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.956+0000 7f38fd400640 1 --2- 172.21.15.86:0/3664476288 >> v2:172.21.15.155:3300/0 conn(0x7f39000b7a30 0x7f39000b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:51.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.956+0000 7f38fe800640 1 -- 172.21.15.86:0/3664476288 >> v2:172.21.15.155:3300/0 conn(0x7f39000b7a30 msgr2=0x7f39000b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:51.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.956+0000 7f38fe800640 1 --2- 172.21.15.86:0/3664476288 >> v2:172.21.15.155:3300/0 conn(0x7f39000b7a30 0x7f39000b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.956+0000 7f38fe800640 1 -- 172.21.15.86:0/3664476288 >> v1:172.21.15.86:6789/0 conn(0x7f39001610d0 legacy=0x7f39000b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:51.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.956+0000 7f38fe800640 1 -- 172.21.15.86:0/3664476288 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39000b39f0 con 0x7f39000d15e0 2024-04-25T14:05:51.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.956+0000 7f38fd400640 1 --2- 172.21.15.86:0/3664476288 >> v2:172.21.15.155:3300/0 conn(0x7f39000b7a30 0x7f39000b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:51.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.956+0000 7f38fca00640 1 -- 172.21.15.86:0/3664476288 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3964832046 0 0) 0x7f39000b3750 con 0x7f39001610d0 2024-04-25T14:05:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.960+0000 7f38fe800640 1 --2- 172.21.15.86:0/3664476288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39000d15e0 0x7f39000d19f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f39000b52f0 tx=0x7f38f801c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45dea70321d41e22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.960+0000 7f38fca00640 1 -- 172.21.15.86:0/3664476288 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f80b2070 con 0x7f39000d15e0 2024-04-25T14:05:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.960+0000 7f38fca00640 1 -- 172.21.15.86:0/3664476288 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38f80a5730 con 0x7f39000d15e0 2024-04-25T14:05:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.960+0000 7f38fca00640 1 -- 172.21.15.86:0/3664476288 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f80a5b00 con 0x7f39000d15e0 2024-04-25T14:05:51.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.960+0000 7f3905200640 1 -- 172.21.15.86:0/3664476288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39000d15e0 msgr2=0x7f39000d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:51.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.960+0000 7f3905200640 1 --2- 172.21.15.86:0/3664476288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39000d15e0 0x7f39000d19f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f39000b52f0 tx=0x7f38f801c870 comp rx=0 tx=0).stop 2024-04-25T14:05:51.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.960+0000 7f3905200640 1 -- 172.21.15.86:0/3664476288 shutdown_connections 2024-04-25T14:05:51.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.960+0000 7f3905200640 1 --2- 172.21.15.86:0/3664476288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39000d15e0 0x7f39000d19f0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.960+0000 7f3905200640 1 --2- 172.21.15.86:0/3664476288 >> v2:172.21.15.155:3300/0 conn(0x7f39000b7a30 0x7f39000b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.960+0000 7f3905200640 1 -- 172.21.15.86:0/3664476288 >> 172.21.15.86:0/3664476288 conn(0x7f390009ba30 msgr2=0x7f39000b7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:51.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.960+0000 7f3905200640 1 -- 172.21.15.86:0/3664476288 shutdown_connections 2024-04-25T14:05:51.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.960+0000 7f3905200640 1 -- 172.21.15.86:0/3664476288 wait complete. 2024-04-25T14:05:51.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.960+0000 7f3905200640 1 Processor -- start 2024-04-25T14:05:51.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.960+0000 7f3905200640 1 -- start start 2024-04-25T14:05:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.964+0000 7f3905200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39000b7a30 0x7f3900156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.964+0000 7f3905200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39000d15e0 0x7f3900154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.964+0000 7f3905200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39001610d0 0x7f390015e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.964+0000 7f3905200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39000cf610 con 0x7f39001610d0 2024-04-25T14:05:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.964+0000 7f3905200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f39000b7e40 con 0x7f39000d15e0 2024-04-25T14:05:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.964+0000 7f3905200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f39000cf790 con 0x7f39000b7a30 2024-04-25T14:05:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.964+0000 7f38fe800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39001610d0 0x7f390015e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.964+0000 7f38fe800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39001610d0 0x7f390015e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55074/0 (socket says 172.21.15.86:55074) 2024-04-25T14:05:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.964+0000 7f38fe800640 1 -- 172.21.15.86:0/2010516162 learned_addr learned my addr 172.21.15.86:0/2010516162 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.964+0000 7f38fd400640 1 --2- 172.21.15.86:0/2010516162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39000d15e0 0x7f3900154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.964+0000 7f38fde00640 1 --2- 172.21.15.86:0/2010516162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39000b7a30 0x7f3900156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.964+0000 7f38fe800640 1 -- 172.21.15.86:0/2010516162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39000b7a30 msgr2=0x7f3900156750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.964+0000 7f38fe800640 1 --2- 172.21.15.86:0/2010516162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39000b7a30 0x7f3900156750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.964+0000 7f38fe800640 1 -- 172.21.15.86:0/2010516162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39000d15e0 msgr2=0x7f3900154710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.964+0000 7f38fe800640 1 --2- 172.21.15.86:0/2010516162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39000d15e0 0x7f3900154710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.964+0000 7f38fe800640 1 -- 172.21.15.86:0/2010516162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39001556b0 con 0x7f39001610d0 2024-04-25T14:05:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.964+0000 7f38fe800640 1 --2- 172.21.15.86:0/2010516162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39001610d0 0x7f390015e320 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f38f80b0040 tx=0x7f38f8019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.964+0000 7f38f6000640 1 -- 172.21.15.86:0/2010516162 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f80b2050 con 0x7f39001610d0 2024-04-25T14:05:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.964+0000 7f38f6000640 1 -- 172.21.15.86:0/2010516162 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38f80b3040 con 0x7f39001610d0 2024-04-25T14:05:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.964+0000 7f3905200640 1 -- 172.21.15.86:0/2010516162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3900155990 con 0x7f39001610d0 2024-04-25T14:05:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.964+0000 7f38f6000640 1 -- 172.21.15.86:0/2010516162 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f8003ba0 con 0x7f39001610d0 2024-04-25T14:05:51.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.964+0000 7f3905200640 1 -- 172.21.15.86:0/2010516162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3900155f50 con 0x7f39001610d0 2024-04-25T14:05:51.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.964+0000 7f3905200640 1 -- 172.21.15.86:0/2010516162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39000d1d10 con 0x7f39001610d0 2024-04-25T14:05:51.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.968+0000 7f38f6000640 1 -- 172.21.15.86:0/2010516162 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f38f80c1060 con 0x7f39001610d0 2024-04-25T14:05:51.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.968+0000 7f38f6000640 1 --2- 172.21.15.86:0/2010516162 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f38e006f570 0x7f38e0071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:51.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.968+0000 7f38f6000640 1 -- 172.21.15.86:0/2010516162 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6519+0+0 (secure 0 0 0) 0x7f38f8129a50 con 0x7f39001610d0 2024-04-25T14:05:51.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.968+0000 7f38fde00640 1 --2- 172.21.15.86:0/2010516162 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f38e006f570 0x7f38e0071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:51.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.968+0000 7f38fde00640 1 --2- 172.21.15.86:0/2010516162 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f38e006f570 0x7f38e0071a80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f38f0007fd0 tx=0x7f38f0009040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:51.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:51.972+0000 7f38f6000640 1 -- 172.21.15.86:0/2010516162 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f38f8016520 con 0x7f39001610d0 2024-04-25T14:05:52.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.128+0000 7f3905200640 1 -- 172.21.15.86:0/2010516162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "fs1_data", "var": "pg_num"} v 0) -- 0x7f3900155440 con 0x7f39001610d0 2024-04-25T14:05:52.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.128+0000 7f38f6000640 1 -- 172.21.15.86:0/2010516162 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "fs1_data", "var": "pg_num"}]=0 v50) ==== 97+0+11 (secure 0 0 0) 0x7f38f80f6460 con 0x7f39001610d0 2024-04-25T14:05:52.130 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:05:52.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.128+0000 7f3905200640 1 -- 172.21.15.86:0/2010516162 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f38e006f570 msgr2=0x7f38e0071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.128+0000 7f3905200640 1 --2- 172.21.15.86:0/2010516162 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f38e006f570 0x7f38e0071a80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f38f0007fd0 tx=0x7f38f0009040 comp rx=0 tx=0).stop 2024-04-25T14:05:52.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.132+0000 7f3905200640 1 -- 172.21.15.86:0/2010516162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39001610d0 msgr2=0x7f390015e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.132+0000 7f3905200640 1 --2- 172.21.15.86:0/2010516162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39001610d0 0x7f390015e320 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f38f80b0040 tx=0x7f38f8019610 comp rx=0 tx=0).stop 2024-04-25T14:05:52.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.132+0000 7f3905200640 1 -- 172.21.15.86:0/2010516162 shutdown_connections 2024-04-25T14:05:52.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.132+0000 7f3905200640 1 --2- 172.21.15.86:0/2010516162 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f38e006f570 0x7f38e0071a80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.132+0000 7f3905200640 1 --2- 172.21.15.86:0/2010516162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39001610d0 0x7f390015e320 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.132+0000 7f3905200640 1 --2- 172.21.15.86:0/2010516162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39000d15e0 0x7f3900154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.132+0000 7f3905200640 1 --2- 172.21.15.86:0/2010516162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39000b7a30 0x7f3900156750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.132+0000 7f3905200640 1 -- 172.21.15.86:0/2010516162 >> 172.21.15.86:0/2010516162 conn(0x7f390009ba30 msgr2=0x7f39000b7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.132+0000 7f3905200640 1 -- 172.21.15.86:0/2010516162 shutdown_connections 2024-04-25T14:05:52.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.132+0000 7f3905200640 1 -- 172.21.15.86:0/2010516162 wait complete. 2024-04-25T14:05:52.154 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T14:05:52.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.296+0000 7ff5aa800640 1 Processor -- start 2024-04-25T14:05:52.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.296+0000 7ff5aa800640 1 -- start start 2024-04-25T14:05:52.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.300+0000 7ff5aa800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5a40cfca0 0x7ff5a406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.300+0000 7ff5aa800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff5a40730b0 0x7ff5a40734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.300+0000 7ff5aa800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5a4072bd0 con 0x7ff5a40d16e0 2024-04-25T14:05:52.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.300+0000 7ff5aa800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff5a4070470 con 0x7ff5a40730b0 2024-04-25T14:05:52.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.300+0000 7ff5aa800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff5a406f930 con 0x7ff5a40cfca0 2024-04-25T14:05:52.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.300+0000 7ff5a3400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5a40cfca0 0x7ff5a406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.300+0000 7ff5a3e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff5a40730b0 0x7ff5a40734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.300+0000 7ff5a3e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff5a40730b0 0x7ff5a40734c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:39694/0 (socket says 172.21.15.86:39694) 2024-04-25T14:05:52.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.300+0000 7ff5a3e00640 1 -- 172.21.15.86:0/2956007296 learned_addr learned my addr 172.21.15.86:0/2956007296 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:52.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.300+0000 7ff5a3e00640 1 -- 172.21.15.86:0/2956007296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5a40cfca0 msgr2=0x7ff5a406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.300+0000 7ff5a3e00640 1 --2- 172.21.15.86:0/2956007296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5a40cfca0 0x7ff5a406f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.300+0000 7ff5a3e00640 1 -- 172.21.15.86:0/2956007296 >> v1:172.21.15.155:6789/0 conn(0x7ff5a40d16e0 legacy=0x7ff5a40d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.300+0000 7ff5a3e00640 1 -- 172.21.15.86:0/2956007296 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a406fbd0 con 0x7ff5a40730b0 2024-04-25T14:05:52.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.300+0000 7ff5a3400640 1 --2- 172.21.15.86:0/2956007296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5a40cfca0 0x7ff5a406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:52.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.304+0000 7ff5a3e00640 1 --2- 172.21.15.86:0/2956007296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5a40730b0 0x7ff5a40734c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff59c009e50 tx=0x7ff59c01ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=942f1063eb004c73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.304+0000 7ff5a2a00640 1 -- 172.21.15.86:0/2956007296 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0a5810 con 0x7ff5a40730b0 2024-04-25T14:05:52.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.304+0000 7ff5a2a00640 1 -- 172.21.15.86:0/2956007296 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff59c0a59b0 con 0x7ff5a40730b0 2024-04-25T14:05:52.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.304+0000 7ff5a2a00640 1 -- 172.21.15.86:0/2956007296 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0a5d80 con 0x7ff5a40730b0 2024-04-25T14:05:52.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.304+0000 7ff5aa800640 1 -- 172.21.15.86:0/2956007296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5a40730b0 msgr2=0x7ff5a40734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.304+0000 7ff5aa800640 1 --2- 172.21.15.86:0/2956007296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5a40730b0 0x7ff5a40734c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff59c009e50 tx=0x7ff59c01ca30 comp rx=0 tx=0).stop 2024-04-25T14:05:52.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.304+0000 7ff5aa800640 1 -- 172.21.15.86:0/2956007296 shutdown_connections 2024-04-25T14:05:52.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.304+0000 7ff5aa800640 1 --2- 172.21.15.86:0/2956007296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5a40730b0 0x7ff5a40734c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.304+0000 7ff5aa800640 1 --2- 172.21.15.86:0/2956007296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5a40cfca0 0x7ff5a406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.304+0000 7ff5aa800640 1 -- 172.21.15.86:0/2956007296 >> 172.21.15.86:0/2956007296 conn(0x7ff5a40b4650 msgr2=0x7ff5a40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.304+0000 7ff5aa800640 1 -- 172.21.15.86:0/2956007296 shutdown_connections 2024-04-25T14:05:52.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.304+0000 7ff5aa800640 1 -- 172.21.15.86:0/2956007296 wait complete. 2024-04-25T14:05:52.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.308+0000 7ff5aa800640 1 Processor -- start 2024-04-25T14:05:52.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.308+0000 7ff5aa800640 1 -- start start 2024-04-25T14:05:52.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.308+0000 7ff5aa800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5a40730b0 0x7ff5a4154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.308+0000 7ff5aa800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5a40cfca0 0x7ff5a41597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.308+0000 7ff5aa800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5a40d16e0 0x7ff5a4159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.308+0000 7ff5aa800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a4169a70 con 0x7ff5a40730b0 2024-04-25T14:05:52.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.308+0000 7ff5aa800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a41698f0 con 0x7ff5a40cfca0 2024-04-25T14:05:52.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.308+0000 7ff5aa800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff5a4169bf0 con 0x7ff5a40d16e0 2024-04-25T14:05:52.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.308+0000 7ff5a3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5a40730b0 0x7ff5a4154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.308+0000 7ff5a3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5a40730b0 0x7ff5a4154020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55080/0 (socket says 172.21.15.86:55080) 2024-04-25T14:05:52.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.308+0000 7ff5a3400640 1 -- 172.21.15.86:0/2243106232 learned_addr learned my addr 172.21.15.86:0/2243106232 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:52.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.308+0000 7ff59aa00640 1 --2- 172.21.15.86:0/2243106232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5a40cfca0 0x7ff5a41597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.308+0000 7ff5a3e00640 1 --2- 172.21.15.86:0/2243106232 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5a40d16e0 0x7ff5a4159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.308+0000 7ff5a3400640 1 -- 172.21.15.86:0/2243106232 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5a40d16e0 msgr2=0x7ff5a4159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.308+0000 7ff5a3400640 1 --2- 172.21.15.86:0/2243106232 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5a40d16e0 0x7ff5a4159cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.308+0000 7ff5a3400640 1 -- 172.21.15.86:0/2243106232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5a40cfca0 msgr2=0x7ff5a41597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.308+0000 7ff5a3400640 1 --2- 172.21.15.86:0/2243106232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5a40cfca0 0x7ff5a41597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.308+0000 7ff5a3400640 1 -- 172.21.15.86:0/2243106232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a40b2730 con 0x7ff5a40730b0 2024-04-25T14:05:52.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.308+0000 7ff5a3400640 1 --2- 172.21.15.86:0/2243106232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5a40730b0 0x7ff5a4154020 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff59400e960 tx=0x7ff59400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.308+0000 7ff59be00640 1 -- 172.21.15.86:0/2243106232 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff594017070 con 0x7ff5a40730b0 2024-04-25T14:05:52.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.308+0000 7ff5aa800640 1 -- 172.21.15.86:0/2243106232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a40b29f0 con 0x7ff5a40730b0 2024-04-25T14:05:52.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.308+0000 7ff59be00640 1 -- 172.21.15.86:0/2243106232 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff59400ca00 con 0x7ff5a40730b0 2024-04-25T14:05:52.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.308+0000 7ff59be00640 1 -- 172.21.15.86:0/2243106232 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59400cdd0 con 0x7ff5a40730b0 2024-04-25T14:05:52.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.308+0000 7ff5aa800640 1 -- 172.21.15.86:0/2243106232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a40b2fb0 con 0x7ff5a40730b0 2024-04-25T14:05:52.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.312+0000 7ff5aa800640 1 -- 172.21.15.86:0/2243106232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5a409f4b0 con 0x7ff5a40730b0 2024-04-25T14:05:52.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.312+0000 7ff59be00640 1 -- 172.21.15.86:0/2243106232 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7ff59401a070 con 0x7ff5a40730b0 2024-04-25T14:05:52.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.312+0000 7ff59be00640 1 --2- 172.21.15.86:0/2243106232 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7ff58006f5a0 0x7ff580071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.312+0000 7ff59be00640 1 -- 172.21.15.86:0/2243106232 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6519+0+0 (secure 0 0 0) 0x7ff59408fa70 con 0x7ff5a40730b0 2024-04-25T14:05:52.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.312+0000 7ff59aa00640 1 --2- 172.21.15.86:0/2243106232 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7ff58006f5a0 0x7ff580071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.312+0000 7ff59aa00640 1 --2- 172.21.15.86:0/2243106232 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7ff58006f5a0 0x7ff580071ab0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4157870 tx=0x7ff58c009040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.316+0000 7ff59be00640 1 -- 172.21.15.86:0/2243106232 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff59405bdb0 con 0x7ff5a40730b0 2024-04-25T14:05:52.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.472+0000 7ff5aa800640 1 -- 172.21.15.86:0/2243106232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff5a4069d00 con 0x7ff5a40730b0 2024-04-25T14:05:52.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.476+0000 7ff59be00640 1 -- 172.21.15.86:0/2243106232 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v50) ==== 84+0+201 (secure 0 0 0) 0x7ff5940600b0 con 0x7ff5a40730b0 2024-04-25T14:05:52.476 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:52.477 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-04-25T14:05:52.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:52.477 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-04-25T14:05:52.477 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-04-25T14:05:52.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:05:52.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:52.477 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 7, 2024-04-25T14:05:52.477 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": "fs1_metadata" 2024-04-25T14:05:52.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:05:52.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:05:52.477 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 8, 2024-04-25T14:05:52.477 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": "fs1_data" 2024-04-25T14:05:52.477 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:05:52.477 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-04-25T14:05:52.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.476+0000 7ff5aa800640 1 -- 172.21.15.86:0/2243106232 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7ff58006f5a0 msgr2=0x7ff580071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.476+0000 7ff5aa800640 1 --2- 172.21.15.86:0/2243106232 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7ff58006f5a0 0x7ff580071ab0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4157870 tx=0x7ff58c009040 comp rx=0 tx=0).stop 2024-04-25T14:05:52.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.476+0000 7ff5aa800640 1 -- 172.21.15.86:0/2243106232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5a40730b0 msgr2=0x7ff5a4154020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.476+0000 7ff5aa800640 1 --2- 172.21.15.86:0/2243106232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5a40730b0 0x7ff5a4154020 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff59400e960 tx=0x7ff59400ee20 comp rx=0 tx=0).stop 2024-04-25T14:05:52.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.476+0000 7ff5aa800640 1 -- 172.21.15.86:0/2243106232 shutdown_connections 2024-04-25T14:05:52.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.476+0000 7ff5aa800640 1 --2- 172.21.15.86:0/2243106232 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7ff58006f5a0 0x7ff580071ab0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.476+0000 7ff5aa800640 1 --2- 172.21.15.86:0/2243106232 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5a40d16e0 0x7ff5a4159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.476+0000 7ff5aa800640 1 --2- 172.21.15.86:0/2243106232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5a40cfca0 0x7ff5a41597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.476+0000 7ff5aa800640 1 --2- 172.21.15.86:0/2243106232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5a40730b0 0x7ff5a4154020 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.476+0000 7ff5aa800640 1 -- 172.21.15.86:0/2243106232 >> 172.21.15.86:0/2243106232 conn(0x7ff5a40b4650 msgr2=0x7ff5a40cc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.476+0000 7ff5aa800640 1 -- 172.21.15.86:0/2243106232 shutdown_connections 2024-04-25T14:05:52.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.476+0000 7ff5aa800640 1 -- 172.21.15.86:0/2243106232 wait complete. 2024-04-25T14:05:52.499 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'fs2' 2024-04-25T14:05:52.499 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create fs2_metadata --pg_num_min 64 2024-04-25T14:05:52.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.648+0000 7fbf4c800640 1 Processor -- start 2024-04-25T14:05:52.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.648+0000 7fbf4c800640 1 -- start start 2024-04-25T14:05:52.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.652+0000 7fbf4c800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbf4406ec00 0x7fbf4406f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.652+0000 7fbf4c800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf4406fe30 0x7fbf440ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.652+0000 7fbf4c800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf4406d120 con 0x7fbf4406fa20 2024-04-25T14:05:52.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.652+0000 7fbf4c800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fbf441600c0 con 0x7fbf4406ec00 2024-04-25T14:05:52.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.652+0000 7fbf4c800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbf4406d3c0 con 0x7fbf4406fe30 2024-04-25T14:05:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.652+0000 7fbf4a800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf4406fe30 0x7fbf440ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.652+0000 7fbf49e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbf4406ec00 0x7fbf4406f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.652+0000 7fbf49e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbf4406ec00 0x7fbf4406f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:39718/0 (socket says 172.21.15.86:39718) 2024-04-25T14:05:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.652+0000 7fbf49e00640 1 -- 172.21.15.86:0/3104981466 learned_addr learned my addr 172.21.15.86:0/3104981466 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.652+0000 7fbf4a800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf4406fe30 0x7fbf440ae9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:34668/0 (socket says 172.21.15.86:34668) 2024-04-25T14:05:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.652+0000 7fbf49400640 1 --1- 172.21.15.86:0/3104981466 >> v1:172.21.15.155:6789/0 conn(0x7fbf4406fa20 0x7fbf440ae2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:60184/0 (socket says 172.21.15.86:60184) 2024-04-25T14:05:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.652+0000 7fbf4a800640 1 -- 172.21.15.86:0/3104981466 >> v2:172.21.15.155:3300/0 conn(0x7fbf4406ec00 msgr2=0x7fbf4406f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.652+0000 7fbf4a800640 1 --2- 172.21.15.86:0/3104981466 >> v2:172.21.15.155:3300/0 conn(0x7fbf4406ec00 0x7fbf4406f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.652+0000 7fbf4a800640 1 -- 172.21.15.86:0/3104981466 >> v1:172.21.15.155:6789/0 conn(0x7fbf4406fa20 legacy=0x7fbf440ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.652+0000 7fbf4a800640 1 -- 172.21.15.86:0/3104981466 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf4406d6a0 con 0x7fbf4406fe30 2024-04-25T14:05:52.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.656+0000 7fbf4a800640 1 --2- 172.21.15.86:0/3104981466 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf4406fe30 0x7fbf440ae9b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbf40009d90 tx=0x7fbf4001f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7783a8a6eb88482 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.656+0000 7fbf48a00640 1 -- 172.21.15.86:0/3104981466 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf400b4070 con 0x7fbf4406fe30 2024-04-25T14:05:52.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.656+0000 7fbf48a00640 1 -- 172.21.15.86:0/3104981466 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf400af600 con 0x7fbf4406fe30 2024-04-25T14:05:52.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.656+0000 7fbf48a00640 1 -- 172.21.15.86:0/3104981466 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf400af9d0 con 0x7fbf4406fe30 2024-04-25T14:05:52.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.656+0000 7fbf4c800640 1 -- 172.21.15.86:0/3104981466 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf4406fe30 msgr2=0x7fbf440ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.656+0000 7fbf4c800640 1 --2- 172.21.15.86:0/3104981466 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf4406fe30 0x7fbf440ae9b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbf40009d90 tx=0x7fbf4001f150 comp rx=0 tx=0).stop 2024-04-25T14:05:52.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.656+0000 7fbf4c800640 1 -- 172.21.15.86:0/3104981466 shutdown_connections 2024-04-25T14:05:52.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.656+0000 7fbf4c800640 1 --2- 172.21.15.86:0/3104981466 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf4406fe30 0x7fbf440ae9b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.656+0000 7fbf4c800640 1 --2- 172.21.15.86:0/3104981466 >> v2:172.21.15.155:3300/0 conn(0x7fbf4406ec00 0x7fbf4406f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.656+0000 7fbf4c800640 1 -- 172.21.15.86:0/3104981466 >> 172.21.15.86:0/3104981466 conn(0x7fbf440b4a40 msgr2=0x7fbf440b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.656+0000 7fbf4c800640 1 -- 172.21.15.86:0/3104981466 shutdown_connections 2024-04-25T14:05:52.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.656+0000 7fbf4c800640 1 -- 172.21.15.86:0/3104981466 wait complete. 2024-04-25T14:05:52.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.656+0000 7fbf4c800640 1 Processor -- start 2024-04-25T14:05:52.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.656+0000 7fbf4c800640 1 -- start start 2024-04-25T14:05:52.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.660+0000 7fbf4c800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf4406ec00 0x7fbf441f7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.660+0000 7fbf4c800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf4406fa20 0x7fbf440cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.660+0000 7fbf4c800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf4406fe30 0x7fbf440cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.660+0000 7fbf4c800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4406f010 con 0x7fbf4406fa20 2024-04-25T14:05:52.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.660+0000 7fbf4c800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbf44073740 con 0x7fbf4406ec00 2024-04-25T14:05:52.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.660+0000 7fbf4c800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbf4406dea0 con 0x7fbf4406fe30 2024-04-25T14:05:52.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.660+0000 7fbf49400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf4406fa20 0x7fbf440cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.660+0000 7fbf49400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf4406fa20 0x7fbf440cc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55082/0 (socket says 172.21.15.86:55082) 2024-04-25T14:05:52.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.660+0000 7fbf49400640 1 -- 172.21.15.86:0/3730821960 learned_addr learned my addr 172.21.15.86:0/3730821960 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:52.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.660+0000 7fbf49e00640 1 --2- 172.21.15.86:0/3730821960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf4406ec00 0x7fbf441f7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.660+0000 7fbf4a800640 1 --2- 172.21.15.86:0/3730821960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf4406fe30 0x7fbf440cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.660+0000 7fbf49400640 1 -- 172.21.15.86:0/3730821960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf4406fe30 msgr2=0x7fbf440cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.660+0000 7fbf49400640 1 --2- 172.21.15.86:0/3730821960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf4406fe30 0x7fbf440cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.660+0000 7fbf49400640 1 -- 172.21.15.86:0/3730821960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf4406ec00 msgr2=0x7fbf441f7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.660+0000 7fbf49400640 1 --2- 172.21.15.86:0/3730821960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf4406ec00 0x7fbf441f7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.660+0000 7fbf49400640 1 -- 172.21.15.86:0/3730821960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf441f8880 con 0x7fbf4406fa20 2024-04-25T14:05:52.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.660+0000 7fbf4a800640 1 --2- 172.21.15.86:0/3730821960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf4406fe30 0x7fbf440cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:52.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.660+0000 7fbf49e00640 1 --2- 172.21.15.86:0/3730821960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf4406ec00 0x7fbf441f7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:52.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.660+0000 7fbf49400640 1 --2- 172.21.15.86:0/3730821960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf4406fa20 0x7fbf440cc970 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400c370 tx=0x7fbf3400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.660+0000 7fbf3e000640 1 -- 172.21.15.86:0/3730821960 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf340074e0 con 0x7fbf4406fa20 2024-04-25T14:05:52.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.660+0000 7fbf3e000640 1 -- 172.21.15.86:0/3730821960 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf340076a0 con 0x7fbf4406fa20 2024-04-25T14:05:52.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.660+0000 7fbf4c800640 1 -- 172.21.15.86:0/3730821960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf441f8b60 con 0x7fbf4406fa20 2024-04-25T14:05:52.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.660+0000 7fbf3e000640 1 -- 172.21.15.86:0/3730821960 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf34004a10 con 0x7fbf4406fa20 2024-04-25T14:05:52.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.660+0000 7fbf4c800640 1 -- 172.21.15.86:0/3730821960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf441f9120 con 0x7fbf4406fa20 2024-04-25T14:05:52.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.660+0000 7fbf4c800640 1 -- 172.21.15.86:0/3730821960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf441f85f0 con 0x7fbf4406fa20 2024-04-25T14:05:52.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.664+0000 7fbf3e000640 1 -- 172.21.15.86:0/3730821960 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7fbf3401a020 con 0x7fbf4406fa20 2024-04-25T14:05:52.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.664+0000 7fbf3e000640 1 --2- 172.21.15.86:0/3730821960 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fbf2806f5a0 0x7fbf28071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.664+0000 7fbf3e000640 1 -- 172.21.15.86:0/3730821960 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6519+0+0 (secure 0 0 0) 0x7fbf3408efd0 con 0x7fbf4406fa20 2024-04-25T14:05:52.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.664+0000 7fbf49e00640 1 --2- 172.21.15.86:0/3730821960 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fbf2806f5a0 0x7fbf28071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.664+0000 7fbf49e00640 1 --2- 172.21.15.86:0/3730821960 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fbf2806f5a0 0x7fbf28071ab0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009b00 tx=0x7fbf38008040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.668+0000 7fbf3e000640 1 -- 172.21.15.86:0/3730821960 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbf3405b9e0 con 0x7fbf4406fa20 2024-04-25T14:05:52.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:52.828+0000 7fbf4c800640 1 -- 172.21.15.86:0/3730821960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "fs2_metadata", "pg_num_min": 64} v 0) -- 0x7fbf44050750 con 0x7fbf4406fa20 2024-04-25T14:05:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.148+0000 7fbf3e000640 1 -- 172.21.15.86:0/3730821960 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "fs2_metadata", "pg_num_min": 64}]=0 pool 'fs2_metadata' created v51) ==== 132+0+0 (secure 0 0 0) 0x7fbf3405b2e0 con 0x7fbf4406fa20 2024-04-25T14:05:53.150 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'fs2_metadata' created 2024-04-25T14:05:53.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.152+0000 7fbf4c800640 1 -- 172.21.15.86:0/3730821960 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fbf2806f5a0 msgr2=0x7fbf28071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.152+0000 7fbf4c800640 1 --2- 172.21.15.86:0/3730821960 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fbf2806f5a0 0x7fbf28071ab0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009b00 tx=0x7fbf38008040 comp rx=0 tx=0).stop 2024-04-25T14:05:53.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.152+0000 7fbf4c800640 1 -- 172.21.15.86:0/3730821960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf4406fa20 msgr2=0x7fbf440cc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.152+0000 7fbf4c800640 1 --2- 172.21.15.86:0/3730821960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf4406fa20 0x7fbf440cc970 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400c370 tx=0x7fbf3400c830 comp rx=0 tx=0).stop 2024-04-25T14:05:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.152+0000 7fbf4c800640 1 -- 172.21.15.86:0/3730821960 shutdown_connections 2024-04-25T14:05:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.152+0000 7fbf4c800640 1 --2- 172.21.15.86:0/3730821960 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fbf2806f5a0 0x7fbf28071ab0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.152+0000 7fbf4c800640 1 --2- 172.21.15.86:0/3730821960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf4406fe30 0x7fbf440cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.152+0000 7fbf4c800640 1 --2- 172.21.15.86:0/3730821960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf4406fa20 0x7fbf440cc970 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.152+0000 7fbf4c800640 1 --2- 172.21.15.86:0/3730821960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf4406ec00 0x7fbf441f7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.152+0000 7fbf4c800640 1 -- 172.21.15.86:0/3730821960 >> 172.21.15.86:0/3730821960 conn(0x7fbf440b4a40 msgr2=0x7fbf4415c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.152+0000 7fbf4c800640 1 -- 172.21.15.86:0/3730821960 shutdown_connections 2024-04-25T14:05:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.152+0000 7fbf4c800640 1 -- 172.21.15.86:0/3730821960 wait complete. 2024-04-25T14:05:53.179 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create fs2_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T14:05:53.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.332+0000 7f9c40600640 1 Processor -- start 2024-04-25T14:05:53.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.332+0000 7f9c40600640 1 -- start start 2024-04-25T14:05:53.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.332+0000 7f9c40600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9c38072630 0x7f9c38072a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.332+0000 7f9c40600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9c38163980 0x7f9c38163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.332+0000 7f9c40600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c3806e340 0x7f9c3806e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.332+0000 7f9c40600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9c38072f80 con 0x7f9c38163980 2024-04-25T14:05:53.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.332+0000 7f9c40600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9c381648e0 con 0x7f9c38072630 2024-04-25T14:05:53.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.332+0000 7f9c40600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9c380d1520 con 0x7f9c3806e340 2024-04-25T14:05:53.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.332+0000 7f9c3d200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9c38163980 0x7f9c38163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.332+0000 7f9c3d200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9c38163980 0x7f9c38163d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55096/0 (socket says 172.21.15.86:55096) 2024-04-25T14:05:53.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.332+0000 7f9c3d200640 1 -- 172.21.15.86:0/3115514923 learned_addr learned my addr 172.21.15.86:0/3115514923 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:53.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.332+0000 7f9c3dc00640 1 --2- 172.21.15.86:0/3115514923 >> v2:172.21.15.155:3300/0 conn(0x7f9c38072630 0x7f9c38072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.332+0000 7f9c3e600640 1 --2- 172.21.15.86:0/3115514923 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c3806e340 0x7f9c3806e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.332+0000 7f9c3d200640 1 -- 172.21.15.86:0/3115514923 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c3806e340 msgr2=0x7f9c3806e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:53.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.332+0000 7f9c3d200640 1 --2- 172.21.15.86:0/3115514923 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c3806e340 0x7f9c3806e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.332+0000 7f9c3d200640 1 -- 172.21.15.86:0/3115514923 >> v2:172.21.15.155:3300/0 conn(0x7f9c38072630 msgr2=0x7f9c38072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:53.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.332+0000 7f9c3d200640 1 --2- 172.21.15.86:0/3115514923 >> v2:172.21.15.155:3300/0 conn(0x7f9c38072630 0x7f9c38072a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.332+0000 7f9c3d200640 1 -- 172.21.15.86:0/3115514923 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c380d23e0 con 0x7f9c38163980 2024-04-25T14:05:53.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.332+0000 7f9c3dc00640 1 --2- 172.21.15.86:0/3115514923 >> v2:172.21.15.155:3300/0 conn(0x7f9c38072630 0x7f9c38072a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:53.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.332+0000 7f9c3e600640 1 --2- 172.21.15.86:0/3115514923 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c3806e340 0x7f9c3806e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:53.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.336+0000 7f9c3d200640 1 --2- 172.21.15.86:0/3115514923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c38163980 0x7f9c38163d90 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9c3806d8e0 tx=0x7f9c2401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b137b07ab96a75e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:53.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.336+0000 7f9c37e00640 1 -- 172.21.15.86:0/3115514923 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c240b0070 con 0x7f9c38163980 2024-04-25T14:05:53.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.336+0000 7f9c37e00640 1 -- 172.21.15.86:0/3115514923 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c240a4e50 con 0x7f9c38163980 2024-04-25T14:05:53.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.336+0000 7f9c37e00640 1 -- 172.21.15.86:0/3115514923 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c240ab820 con 0x7f9c38163980 2024-04-25T14:05:53.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.336+0000 7f9c40600640 1 -- 172.21.15.86:0/3115514923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c38163980 msgr2=0x7f9c38163d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.336+0000 7f9c40600640 1 --2- 172.21.15.86:0/3115514923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c38163980 0x7f9c38163d90 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9c3806d8e0 tx=0x7f9c2401c400 comp rx=0 tx=0).stop 2024-04-25T14:05:53.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.336+0000 7f9c40600640 1 -- 172.21.15.86:0/3115514923 shutdown_connections 2024-04-25T14:05:53.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.336+0000 7f9c40600640 1 --2- 172.21.15.86:0/3115514923 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c3806e340 0x7f9c3806e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.336+0000 7f9c40600640 1 --2- 172.21.15.86:0/3115514923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c38163980 0x7f9c38163d90 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.336+0000 7f9c40600640 1 --2- 172.21.15.86:0/3115514923 >> v2:172.21.15.155:3300/0 conn(0x7f9c38072630 0x7f9c38072a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.336+0000 7f9c40600640 1 -- 172.21.15.86:0/3115514923 >> 172.21.15.86:0/3115514923 conn(0x7f9c380d3580 msgr2=0x7f9c380d3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:53.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.336+0000 7f9c40600640 1 -- 172.21.15.86:0/3115514923 shutdown_connections 2024-04-25T14:05:53.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.336+0000 7f9c40600640 1 -- 172.21.15.86:0/3115514923 wait complete. 2024-04-25T14:05:53.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.340+0000 7f9c40600640 1 Processor -- start 2024-04-25T14:05:53.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.340+0000 7f9c40600640 1 -- start start 2024-04-25T14:05:53.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.340+0000 7f9c40600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c3806e340 0x7f9c380ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.340+0000 7f9c40600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c38072630 0x7f9c380cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.340+0000 7f9c40600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c38163980 0x7f9c3814e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.340+0000 7f9c40600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c380cfcb0 con 0x7f9c38072630 2024-04-25T14:05:53.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.340+0000 7f9c40600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3806f0a0 con 0x7f9c3806e340 2024-04-25T14:05:53.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.340+0000 7f9c40600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9c380cfe30 con 0x7f9c38163980 2024-04-25T14:05:53.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.340+0000 7f9c3d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c38072630 0x7f9c380cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.340+0000 7f9c3d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c38072630 0x7f9c380cbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55112/0 (socket says 172.21.15.86:55112) 2024-04-25T14:05:53.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.340+0000 7f9c3d200640 1 -- 172.21.15.86:0/2425765900 learned_addr learned my addr 172.21.15.86:0/2425765900 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:53.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.340+0000 7f9c3dc00640 1 --2- 172.21.15.86:0/2425765900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c3806e340 0x7f9c380ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.340+0000 7f9c3e600640 1 --2- 172.21.15.86:0/2425765900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c38163980 0x7f9c3814e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.340+0000 7f9c3d200640 1 -- 172.21.15.86:0/2425765900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c38163980 msgr2=0x7f9c3814e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.340+0000 7f9c3d200640 1 --2- 172.21.15.86:0/2425765900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c38163980 0x7f9c3814e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.340+0000 7f9c3d200640 1 -- 172.21.15.86:0/2425765900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c3806e340 msgr2=0x7f9c380ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.340+0000 7f9c3d200640 1 --2- 172.21.15.86:0/2425765900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c3806e340 0x7f9c380ca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.340+0000 7f9c3d200640 1 -- 172.21.15.86:0/2425765900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c380cc440 con 0x7f9c38072630 2024-04-25T14:05:53.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.344+0000 7f9c3d200640 1 --2- 172.21.15.86:0/2425765900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c38072630 0x7f9c380cbf00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9c2401aab0 tx=0x7f9c240191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:53.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.344+0000 7f9c35600640 1 -- 172.21.15.86:0/2425765900 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c240b0040 con 0x7f9c38072630 2024-04-25T14:05:53.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.344+0000 7f9c35600640 1 -- 172.21.15.86:0/2425765900 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c240aa660 con 0x7f9c38072630 2024-04-25T14:05:53.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.344+0000 7f9c35600640 1 -- 172.21.15.86:0/2425765900 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c240aaa50 con 0x7f9c38072630 2024-04-25T14:05:53.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.344+0000 7f9c40600640 1 -- 172.21.15.86:0/2425765900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c380cae60 con 0x7f9c38072630 2024-04-25T14:05:53.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.344+0000 7f9c40600640 1 -- 172.21.15.86:0/2425765900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c380cb420 con 0x7f9c38072630 2024-04-25T14:05:53.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.344+0000 7f9c40600640 1 -- 172.21.15.86:0/2425765900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c380caba0 con 0x7f9c38072630 2024-04-25T14:05:53.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.344+0000 7f9c35600640 1 -- 172.21.15.86:0/2425765900 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f9c240aabf0 con 0x7f9c38072630 2024-04-25T14:05:53.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.348+0000 7f9c35600640 1 --2- 172.21.15.86:0/2425765900 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f9c1806f570 0x7f9c18071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.348+0000 7f9c35600640 1 -- 172.21.15.86:0/2425765900 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6886+0+0 (secure 0 0 0) 0x7f9c24129890 con 0x7f9c38072630 2024-04-25T14:05:53.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.348+0000 7f9c3dc00640 1 --2- 172.21.15.86:0/2425765900 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f9c1806f570 0x7f9c18071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.348+0000 7f9c3dc00640 1 --2- 172.21.15.86:0/2425765900 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f9c1806f570 0x7f9c18071a80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c006fd0 tx=0x7f9c2c008040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:53.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.352+0000 7f9c35600640 1 -- 172.21.15.86:0/2425765900 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9c24016110 con 0x7f9c38072630 2024-04-25T14:05:53.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:53.512+0000 7f9c40600640 1 -- 172.21.15.86:0/2425765900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "fs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9c38072170 con 0x7f9c38072630 2024-04-25T14:05:54.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.156+0000 7f9c35600640 1 -- 172.21.15.86:0/2425765900 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "fs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'fs2_data' created v52) ==== 164+0+0 (secure 0 0 0) 0x7f9c240f59d0 con 0x7f9c38072630 2024-04-25T14:05:54.158 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'fs2_data' created 2024-04-25T14:05:54.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.160+0000 7f9c40600640 1 -- 172.21.15.86:0/2425765900 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f9c1806f570 msgr2=0x7f9c18071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:54.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.160+0000 7f9c40600640 1 --2- 172.21.15.86:0/2425765900 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f9c1806f570 0x7f9c18071a80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c006fd0 tx=0x7f9c2c008040 comp rx=0 tx=0).stop 2024-04-25T14:05:54.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.160+0000 7f9c40600640 1 -- 172.21.15.86:0/2425765900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c38072630 msgr2=0x7f9c380cbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:54.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.160+0000 7f9c40600640 1 --2- 172.21.15.86:0/2425765900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c38072630 0x7f9c380cbf00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9c2401aab0 tx=0x7f9c240191a0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.160+0000 7f9c40600640 1 -- 172.21.15.86:0/2425765900 shutdown_connections 2024-04-25T14:05:54.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.160+0000 7f9c40600640 1 --2- 172.21.15.86:0/2425765900 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f9c1806f570 0x7f9c18071a80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.160+0000 7f9c40600640 1 --2- 172.21.15.86:0/2425765900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c38163980 0x7f9c3814e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.160+0000 7f9c40600640 1 --2- 172.21.15.86:0/2425765900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c38072630 0x7f9c380cbf00 secure :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9c2401aab0 tx=0x7f9c240191a0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.160+0000 7f9c40600640 1 --2- 172.21.15.86:0/2425765900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c3806e340 0x7f9c380ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.160+0000 7f9c40600640 1 -- 172.21.15.86:0/2425765900 >> 172.21.15.86:0/2425765900 conn(0x7f9c380d3580 msgr2=0x7f9c38161c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:54.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.164+0000 7f9c40600640 1 -- 172.21.15.86:0/2425765900 shutdown_connections 2024-04-25T14:05:54.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.164+0000 7f9c40600640 1 -- 172.21.15.86:0/2425765900 wait complete. 2024-04-25T14:05:54.204 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new fs2 fs2_metadata fs2_data 2024-04-25T14:05:54.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.352+0000 7f52eac00640 1 Processor -- start 2024-04-25T14:05:54.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.352+0000 7f52eac00640 1 -- start start 2024-04-25T14:05:54.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.352+0000 7f52eac00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f52e4070330 0x7f52e40d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.352+0000 7f52eac00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f52e40d1c20 0x7f52e406ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.352+0000 7f52eac00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52e406f270 0x7f52e406f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.352+0000 7f52eac00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f52e40d2030 con 0x7f52e40d1c20 2024-04-25T14:05:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.352+0000 7f52eac00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f52e415e5c0 con 0x7f52e4070330 2024-04-25T14:05:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.352+0000 7f52eac00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f52e415e8a0 con 0x7f52e406f270 2024-04-25T14:05:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.356+0000 7f52db400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f52e40d1c20 0x7f52e406ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.356+0000 7f52db400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f52e40d1c20 0x7f52e406ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55128/0 (socket says 172.21.15.86:55128) 2024-04-25T14:05:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.356+0000 7f52db400640 1 -- 172.21.15.86:0/2605671131 learned_addr learned my addr 172.21.15.86:0/2605671131 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.356+0000 7f52e8c00640 1 --2- 172.21.15.86:0/2605671131 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52e406f270 0x7f52e406f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.356+0000 7f52e3e00640 1 --2- 172.21.15.86:0/2605671131 >> v2:172.21.15.155:3300/0 conn(0x7f52e4070330 0x7f52e40d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.356+0000 7f52db400640 1 -- 172.21.15.86:0/2605671131 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52e406f270 msgr2=0x7f52e406f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.356+0000 7f52db400640 1 --2- 172.21.15.86:0/2605671131 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52e406f270 0x7f52e406f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.356+0000 7f52db400640 1 -- 172.21.15.86:0/2605671131 >> v2:172.21.15.155:3300/0 conn(0x7f52e4070330 msgr2=0x7f52e40d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.356+0000 7f52db400640 1 --2- 172.21.15.86:0/2605671131 >> v2:172.21.15.155:3300/0 conn(0x7f52e4070330 0x7f52e40d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.356+0000 7f52db400640 1 -- 172.21.15.86:0/2605671131 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e415eb80 con 0x7f52e40d1c20 2024-04-25T14:05:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.356+0000 7f52e8c00640 1 --2- 172.21.15.86:0/2605671131 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52e406f270 0x7f52e406f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.356+0000 7f52e3e00640 1 --2- 172.21.15.86:0/2605671131 >> v2:172.21.15.155:3300/0 conn(0x7f52e4070330 0x7f52e40d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:54.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.356+0000 7f52db400640 1 --2- 172.21.15.86:0/2605671131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52e40d1c20 0x7f52e406ed30 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f52e40d23c0 tx=0x7f52cc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9a469936bc67a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:54.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.356+0000 7f52e3400640 1 -- 172.21.15.86:0/2605671131 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc0ab600 con 0x7f52e40d1c20 2024-04-25T14:05:54.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.356+0000 7f52e3400640 1 -- 172.21.15.86:0/2605671131 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52cc0b0070 con 0x7f52e40d1c20 2024-04-25T14:05:54.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.356+0000 7f52e3400640 1 -- 172.21.15.86:0/2605671131 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc0ab9c0 con 0x7f52e40d1c20 2024-04-25T14:05:54.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.360+0000 7f52eac00640 1 -- 172.21.15.86:0/2605671131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52e40d1c20 msgr2=0x7f52e406ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:54.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.360+0000 7f52eac00640 1 --2- 172.21.15.86:0/2605671131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52e40d1c20 0x7f52e406ed30 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f52e40d23c0 tx=0x7f52cc01c460 comp rx=0 tx=0).stop 2024-04-25T14:05:54.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.360+0000 7f52eac00640 1 -- 172.21.15.86:0/2605671131 shutdown_connections 2024-04-25T14:05:54.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.360+0000 7f52eac00640 1 --2- 172.21.15.86:0/2605671131 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52e406f270 0x7f52e406f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.360+0000 7f52eac00640 1 --2- 172.21.15.86:0/2605671131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52e40d1c20 0x7f52e406ed30 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.360+0000 7f52eac00640 1 --2- 172.21.15.86:0/2605671131 >> v2:172.21.15.155:3300/0 conn(0x7f52e4070330 0x7f52e40d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.360+0000 7f52eac00640 1 -- 172.21.15.86:0/2605671131 >> 172.21.15.86:0/2605671131 conn(0x7f52e40b4650 msgr2=0x7f52e40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:54.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.360+0000 7f52eac00640 1 -- 172.21.15.86:0/2605671131 shutdown_connections 2024-04-25T14:05:54.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.360+0000 7f52eac00640 1 -- 172.21.15.86:0/2605671131 wait complete. 2024-04-25T14:05:54.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.360+0000 7f52eac00640 1 Processor -- start 2024-04-25T14:05:54.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.360+0000 7f52eac00640 1 -- start start 2024-04-25T14:05:54.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.364+0000 7f52eac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52e406f270 0x7f52e409a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.364+0000 7f52eac00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52e4070330 0x7f52e4099b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.364+0000 7f52eac00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52e40d1c20 0x7f52e4158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.364+0000 7f52eac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f52e415f4b0 con 0x7f52e406f270 2024-04-25T14:05:54.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.364+0000 7f52eac00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f52e415f330 con 0x7f52e4070330 2024-04-25T14:05:54.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.364+0000 7f52eac00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f52e4070740 con 0x7f52e40d1c20 2024-04-25T14:05:54.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.364+0000 7f52e3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52e406f270 0x7f52e409a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.364+0000 7f52e3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52e406f270 0x7f52e409a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55138/0 (socket says 172.21.15.86:55138) 2024-04-25T14:05:54.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.364+0000 7f52e3e00640 1 -- 172.21.15.86:0/4247748 learned_addr learned my addr 172.21.15.86:0/4247748 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:54.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.364+0000 7f52db400640 1 --2- 172.21.15.86:0/4247748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52e4070330 0x7f52e4099b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.364+0000 7f52e8c00640 1 --2- 172.21.15.86:0/4247748 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52e40d1c20 0x7f52e4158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.364+0000 7f52e3e00640 1 -- 172.21.15.86:0/4247748 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52e40d1c20 msgr2=0x7f52e4158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:54.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.364+0000 7f52e3e00640 1 --2- 172.21.15.86:0/4247748 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52e40d1c20 0x7f52e4158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.364+0000 7f52e3e00640 1 -- 172.21.15.86:0/4247748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52e4070330 msgr2=0x7f52e4099b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:54.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.364+0000 7f52e3e00640 1 --2- 172.21.15.86:0/4247748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52e4070330 0x7f52e4099b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.364+0000 7f52e3e00640 1 -- 172.21.15.86:0/4247748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e40cccb0 con 0x7f52e406f270 2024-04-25T14:05:54.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.364+0000 7f52e3e00640 1 --2- 172.21.15.86:0/4247748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52e406f270 0x7f52e409a3c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f52d400c7f0 tx=0x7f52d400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:54.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.364+0000 7f52e0c00640 1 -- 172.21.15.86:0/4247748 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d4015070 con 0x7f52e406f270 2024-04-25T14:05:54.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.364+0000 7f52e0c00640 1 -- 172.21.15.86:0/4247748 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d4007820 con 0x7f52e406f270 2024-04-25T14:05:54.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.364+0000 7f52e0c00640 1 -- 172.21.15.86:0/4247748 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d4007bd0 con 0x7f52e406f270 2024-04-25T14:05:54.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.364+0000 7f52eac00640 1 -- 172.21.15.86:0/4247748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e406f9b0 con 0x7f52e406f270 2024-04-25T14:05:54.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.364+0000 7f52eac00640 1 -- 172.21.15.86:0/4247748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e41558e0 con 0x7f52e406f270 2024-04-25T14:05:54.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.364+0000 7f52eac00640 1 -- 172.21.15.86:0/4247748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52e4069d00 con 0x7f52e406f270 2024-04-25T14:05:54.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.368+0000 7f52e0c00640 1 -- 172.21.15.86:0/4247748 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f52d4018070 con 0x7f52e406f270 2024-04-25T14:05:54.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.368+0000 7f52e0c00640 1 --2- 172.21.15.86:0/4247748 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f52c006f5a0 0x7f52c0071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.368+0000 7f52e0c00640 1 -- 172.21.15.86:0/4247748 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 7265+0+0 (secure 0 0 0) 0x7f52d40900b0 con 0x7f52e406f270 2024-04-25T14:05:54.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.368+0000 7f52db400640 1 --2- 172.21.15.86:0/4247748 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f52c006f5a0 0x7f52c0071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.368+0000 7f52db400640 1 --2- 172.21.15.86:0/4247748 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f52c006f5a0 0x7f52c0071ab0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f52cc0af000 tx=0x7f52cc0a7000 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:54.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.372+0000 7f52e0c00640 1 -- 172.21.15.86:0/4247748 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f52d405c180 con 0x7f52e406f270 2024-04-25T14:05:54.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:54.560+0000 7f52eac00640 1 -- 172.21.15.86:0/4247748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "fs2", "metadata": "fs2_metadata", "data": "fs2_data"} v 0) -- 0x7f52e4076bf0 con 0x7f52e406f270 2024-04-25T14:05:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.156+0000 7f52e0c00640 1 -- 172.21.15.86:0/4247748 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "fs2", "metadata": "fs2_metadata", "data": "fs2_data"}]=0 new fs with metadata pool 9 and data pool 10 v33) ==== 164+0+0 (secure 0 0 0) 0x7f52d4060480 con 0x7f52e406f270 2024-04-25T14:05:55.160 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 9 and data pool 10 2024-04-25T14:05:55.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.160+0000 7f52eac00640 1 -- 172.21.15.86:0/4247748 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f52c006f5a0 msgr2=0x7f52c0071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:55.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.160+0000 7f52eac00640 1 --2- 172.21.15.86:0/4247748 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f52c006f5a0 0x7f52c0071ab0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f52cc0af000 tx=0x7f52cc0a7000 comp rx=0 tx=0).stop 2024-04-25T14:05:55.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.160+0000 7f52eac00640 1 -- 172.21.15.86:0/4247748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52e406f270 msgr2=0x7f52e409a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:55.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.160+0000 7f52eac00640 1 --2- 172.21.15.86:0/4247748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52e406f270 0x7f52e409a3c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f52d400c7f0 tx=0x7f52d400ccb0 comp rx=0 tx=0).stop 2024-04-25T14:05:55.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.160+0000 7f52eac00640 1 -- 172.21.15.86:0/4247748 shutdown_connections 2024-04-25T14:05:55.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.160+0000 7f52eac00640 1 --2- 172.21.15.86:0/4247748 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f52c006f5a0 0x7f52c0071ab0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:55.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.160+0000 7f52eac00640 1 --2- 172.21.15.86:0/4247748 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52e40d1c20 0x7f52e4158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:55.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.160+0000 7f52eac00640 1 --2- 172.21.15.86:0/4247748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52e4070330 0x7f52e4099b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:55.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.160+0000 7f52eac00640 1 --2- 172.21.15.86:0/4247748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52e406f270 0x7f52e409a3c0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:55.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.160+0000 7f52eac00640 1 -- 172.21.15.86:0/4247748 >> 172.21.15.86:0/4247748 conn(0x7f52e40b4650 msgr2=0x7f52e40cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:55.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.160+0000 7f52eac00640 1 -- 172.21.15.86:0/4247748 shutdown_connections 2024-04-25T14:05:55.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.160+0000 7f52eac00640 1 -- 172.21.15.86:0/4247748 wait complete. 2024-04-25T14:05:55.205 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:05:55.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.356+0000 7f654bc00640 1 Processor -- start 2024-04-25T14:05:55.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.356+0000 7f654bc00640 1 -- start start 2024-04-25T14:05:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.356+0000 7f654bc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6544070330 0x7f65440d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.356+0000 7f654bc00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f65440d1c20 0x7f654406ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.356+0000 7f654bc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65440d2030 con 0x7f654406f270 2024-04-25T14:05:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.356+0000 7f654bc00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f654415e5c0 con 0x7f65440d1c20 2024-04-25T14:05:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.356+0000 7f654bc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f654415e8a0 con 0x7f6544070330 2024-04-25T14:05:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.356+0000 7f6549c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f654406f270 0x7f654406f680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53332/0 (socket says 172.21.15.86:53332) 2024-04-25T14:05:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.356+0000 7f6549c00640 1 -- 172.21.15.86:0/3769991033 learned_addr learned my addr 172.21.15.86:0/3769991033 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:55.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.356+0000 7f6549200640 1 --2- 172.21.15.86:0/3769991033 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6544070330 0x7f65440d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:55.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.356+0000 7f6543e00640 1 --2- 172.21.15.86:0/3769991033 >> v2:172.21.15.155:3300/0 conn(0x7f65440d1c20 0x7f654406ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:55.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.356+0000 7f6543400640 1 -- 172.21.15.86:0/3769991033 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3844794396 0 0) 0x7f653c0014a0 con 0x7f654406f270 2024-04-25T14:05:55.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.356+0000 7f6543400640 1 -- 172.21.15.86:0/3769991033 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f652c003530 con 0x7f654406f270 2024-04-25T14:05:55.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.356+0000 7f6549200640 1 -- 172.21.15.86:0/3769991033 >> v2:172.21.15.155:3300/0 conn(0x7f65440d1c20 msgr2=0x7f654406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:55.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.356+0000 7f6549200640 1 --2- 172.21.15.86:0/3769991033 >> v2:172.21.15.155:3300/0 conn(0x7f65440d1c20 0x7f654406ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:55.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.356+0000 7f6543400640 1 -- 172.21.15.86:0/3769991033 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 963605489 0 0) 0x7f652c003530 con 0x7f654406f270 2024-04-25T14:05:55.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.356+0000 7f6549200640 1 -- 172.21.15.86:0/3769991033 >> v1:172.21.15.86:6789/0 conn(0x7f654406f270 legacy=0x7f654406f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:55.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.356+0000 7f6549200640 1 -- 172.21.15.86:0/3769991033 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f654415eb80 con 0x7f6544070330 2024-04-25T14:05:55.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.356+0000 7f6543e00640 1 --2- 172.21.15.86:0/3769991033 >> v2:172.21.15.155:3300/0 conn(0x7f65440d1c20 0x7f654406ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:55.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.360+0000 7f6549200640 1 --2- 172.21.15.86:0/3769991033 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6544070330 0x7f65440d16e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f65380097f0 tx=0x7f653801c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d38bc9282e5b070b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:55.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.360+0000 7f6543400640 1 -- 172.21.15.86:0/3769991033 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65380b0070 con 0x7f6544070330 2024-04-25T14:05:55.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.360+0000 7f6543400640 1 -- 172.21.15.86:0/3769991033 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65380ab400 con 0x7f6544070330 2024-04-25T14:05:55.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.360+0000 7f6543400640 1 -- 172.21.15.86:0/3769991033 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c001bd0 con 0x7f6544070330 2024-04-25T14:05:55.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.360+0000 7f654bc00640 1 -- 172.21.15.86:0/3769991033 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6544070330 msgr2=0x7f65440d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:55.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.360+0000 7f654bc00640 1 --2- 172.21.15.86:0/3769991033 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6544070330 0x7f65440d16e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f65380097f0 tx=0x7f653801c330 comp rx=0 tx=0).stop 2024-04-25T14:05:55.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.360+0000 7f654bc00640 1 -- 172.21.15.86:0/3769991033 shutdown_connections 2024-04-25T14:05:55.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.360+0000 7f654bc00640 1 --2- 172.21.15.86:0/3769991033 >> v2:172.21.15.155:3300/0 conn(0x7f65440d1c20 0x7f654406ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:55.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.360+0000 7f654bc00640 1 --2- 172.21.15.86:0/3769991033 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6544070330 0x7f65440d16e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:55.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.360+0000 7f654bc00640 1 -- 172.21.15.86:0/3769991033 >> 172.21.15.86:0/3769991033 conn(0x7f65440b4650 msgr2=0x7f65440b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:55.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.360+0000 7f654bc00640 1 -- 172.21.15.86:0/3769991033 shutdown_connections 2024-04-25T14:05:55.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.360+0000 7f654bc00640 1 -- 172.21.15.86:0/3769991033 wait complete. 2024-04-25T14:05:55.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.364+0000 7f654bc00640 1 Processor -- start 2024-04-25T14:05:55.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.364+0000 7f654bc00640 1 -- start start 2024-04-25T14:05:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.364+0000 7f654bc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f654406f270 0x7f6544072880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.364+0000 7f654bc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6544070330 0x7f654415bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.364+0000 7f654bc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65440d1c20 0x7f654415a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.364+0000 7f654bc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f65440708c0 con 0x7f6544070330 2024-04-25T14:05:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.364+0000 7f654bc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6544070740 con 0x7f654406f270 2024-04-25T14:05:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.364+0000 7f654bc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f65440ceeb0 con 0x7f65440d1c20 2024-04-25T14:05:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.364+0000 7f6543e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6544070330 0x7f654415bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.364+0000 7f6543e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6544070330 0x7f654415bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55140/0 (socket says 172.21.15.86:55140) 2024-04-25T14:05:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.364+0000 7f6543e00640 1 -- 172.21.15.86:0/2399831377 learned_addr learned my addr 172.21.15.86:0/2399831377 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.364+0000 7f6549200640 1 --2- 172.21.15.86:0/2399831377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f654406f270 0x7f6544072880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.368+0000 7f6549c00640 1 --2- 172.21.15.86:0/2399831377 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65440d1c20 0x7f654415a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.368+0000 7f6543e00640 1 -- 172.21.15.86:0/2399831377 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65440d1c20 msgr2=0x7f654415a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.368+0000 7f6543e00640 1 --2- 172.21.15.86:0/2399831377 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65440d1c20 0x7f654415a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.368+0000 7f6543e00640 1 -- 172.21.15.86:0/2399831377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f654406f270 msgr2=0x7f6544072880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.368+0000 7f6543e00640 1 --2- 172.21.15.86:0/2399831377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f654406f270 0x7f6544072880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.368+0000 7f6543e00640 1 -- 172.21.15.86:0/2399831377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65440745d0 con 0x7f6544070330 2024-04-25T14:05:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.368+0000 7f6549c00640 1 --2- 172.21.15.86:0/2399831377 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65440d1c20 0x7f654415a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.368+0000 7f6543e00640 1 --2- 172.21.15.86:0/2399831377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6544070330 0x7f654415bf60 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f653000e9b0 tx=0x7f653000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.368+0000 7f6540c00640 1 -- 172.21.15.86:0/2399831377 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530017070 con 0x7f6544070330 2024-04-25T14:05:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.368+0000 7f654bc00640 1 -- 172.21.15.86:0/2399831377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65441579e0 con 0x7f6544070330 2024-04-25T14:05:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.368+0000 7f6540c00640 1 -- 172.21.15.86:0/2399831377 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f653000ca00 con 0x7f6544070330 2024-04-25T14:05:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.368+0000 7f6540c00640 1 -- 172.21.15.86:0/2399831377 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653000cdd0 con 0x7f6544070330 2024-04-25T14:05:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.368+0000 7f654bc00640 1 -- 172.21.15.86:0/2399831377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65440c4070 con 0x7f6544070330 2024-04-25T14:05:55.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.368+0000 7f654bc00640 1 -- 172.21.15.86:0/2399831377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6544069d00 con 0x7f6544070330 2024-04-25T14:05:55.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.368+0000 7f6540c00640 1 -- 172.21.15.86:0/2399831377 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f653001a070 con 0x7f6544070330 2024-04-25T14:05:55.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.372+0000 7f6540c00640 1 --2- 172.21.15.86:0/2399831377 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f652406f5a0 0x7f6524071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:55.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.372+0000 7f6540c00640 1 -- 172.21.15.86:0/2399831377 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 7359+0+0 (secure 0 0 0) 0x7f653008fcb0 con 0x7f6544070330 2024-04-25T14:05:55.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.372+0000 7f6549200640 1 --2- 172.21.15.86:0/2399831377 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f652406f5a0 0x7f6524071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:55.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.372+0000 7f6549200640 1 --2- 172.21.15.86:0/2399831377 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f652406f5a0 0x7f6524071ab0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f65380ad000 tx=0x7f65380a6000 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:55.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.376+0000 7f6540c00640 1 -- 172.21.15.86:0/2399831377 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f653005bdc0 con 0x7f6544070330 2024-04-25T14:05:55.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.536+0000 7f654bc00640 1 -- 172.21.15.86:0/2399831377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f65440b3bb0 con 0x7f6544070330 2024-04-25T14:05:55.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.536+0000 7f6540c00640 1 -- 172.21.15.86:0/2399831377 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+21284 (secure 0 0 0) 0x7f653007c020 con 0x7f6544070330 2024-04-25T14:05:55.539 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:55.539 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":53,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:05:54.565574+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"fs1_metadata","create_time":"2024-04-25T14:05:30.939601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"fs1"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"fs1_data","create_time":"2024-04-25T14:05:31.898262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"fs1"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"fs2_metadata","create_time":"2024-04-25T14:05:52.841764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"fs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"fs2_data","create_time":"2024-04-25T14:05:53.524454+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"fs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/615772001":"2024-04-26T14:05:42.882867+0000","172.21.15.86:6835/615772001":"2024-04-26T14:05:42.882867+0000","172.21.15.86:6835/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:0/2073988773":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6833/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6832/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/3558294881":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/1843278167":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6837/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:0/2602032945":"2024-04-26T14:05:21.460855+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:05:55.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.540+0000 7f654bc00640 1 -- 172.21.15.86:0/2399831377 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f652406f5a0 msgr2=0x7f6524071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:55.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.540+0000 7f654bc00640 1 --2- 172.21.15.86:0/2399831377 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f652406f5a0 0x7f6524071ab0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f65380ad000 tx=0x7f65380a6000 comp rx=0 tx=0).stop 2024-04-25T14:05:55.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.540+0000 7f654bc00640 1 -- 172.21.15.86:0/2399831377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6544070330 msgr2=0x7f654415bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:55.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.540+0000 7f654bc00640 1 --2- 172.21.15.86:0/2399831377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6544070330 0x7f654415bf60 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f653000e9b0 tx=0x7f653000ee70 comp rx=0 tx=0).stop 2024-04-25T14:05:55.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.540+0000 7f654bc00640 1 -- 172.21.15.86:0/2399831377 shutdown_connections 2024-04-25T14:05:55.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.540+0000 7f654bc00640 1 --2- 172.21.15.86:0/2399831377 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f652406f5a0 0x7f6524071ab0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:55.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.540+0000 7f654bc00640 1 --2- 172.21.15.86:0/2399831377 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65440d1c20 0x7f654415a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:55.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.540+0000 7f654bc00640 1 --2- 172.21.15.86:0/2399831377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6544070330 0x7f654415bf60 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:55.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.540+0000 7f654bc00640 1 --2- 172.21.15.86:0/2399831377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f654406f270 0x7f6544072880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:55.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.540+0000 7f654bc00640 1 -- 172.21.15.86:0/2399831377 >> 172.21.15.86:0/2399831377 conn(0x7f65440b4650 msgr2=0x7f65440b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:55.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.540+0000 7f654bc00640 1 -- 172.21.15.86:0/2399831377 shutdown_connections 2024-04-25T14:05:55.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.540+0000 7f654bc00640 1 -- 172.21.15.86:0/2399831377 wait complete. 2024-04-25T14:05:55.561 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:05:55.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.704+0000 7faccc200640 1 Processor -- start 2024-04-25T14:05:55.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.704+0000 7faccc200640 1 -- start start 2024-04-25T14:05:55.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.708+0000 7faccc200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7facc40cfca0 0x7facc406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:55.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.708+0000 7faccc200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facc40730b0 0x7facc40734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:55.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.708+0000 7faccc200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facc4072bd0 con 0x7facc40d16e0 2024-04-25T14:05:55.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.708+0000 7faccc200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7facc4070470 con 0x7facc40cfca0 2024-04-25T14:05:55.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.708+0000 7faccc200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7facc406f930 con 0x7facc40730b0 2024-04-25T14:05:55.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.708+0000 7facca200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facc40730b0 0x7facc40734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:55.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.708+0000 7facc9800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7facc40cfca0 0x7facc406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:55.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.708+0000 7facca200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facc40730b0 0x7facc40734c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:34744/0 (socket says 172.21.15.86:34744) 2024-04-25T14:05:55.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.708+0000 7facca200640 1 -- 172.21.15.86:0/3159714187 learned_addr learned my addr 172.21.15.86:0/3159714187 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:55.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.708+0000 7facc9800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7facc40cfca0 0x7facc406f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:39786/0 (socket says 172.21.15.86:39786) 2024-04-25T14:05:55.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.708+0000 7facc8e00640 1 --1- 172.21.15.86:0/3159714187 >> v1:172.21.15.155:6789/0 conn(0x7facc40d16e0 0x7facc40d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:60194/0 (socket says 172.21.15.86:60194) 2024-04-25T14:05:55.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.708+0000 7facc3e00640 1 -- 172.21.15.86:0/3159714187 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1094725516 0 0) 0x7facc4072bd0 con 0x7facc40d16e0 2024-04-25T14:05:55.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.708+0000 7facc3e00640 1 -- 172.21.15.86:0/3159714187 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7facac005740 con 0x7facc40d16e0 2024-04-25T14:05:55.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.708+0000 7facca200640 1 -- 172.21.15.86:0/3159714187 >> v2:172.21.15.155:3300/0 conn(0x7facc40cfca0 msgr2=0x7facc406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:55.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.708+0000 7facca200640 1 --2- 172.21.15.86:0/3159714187 >> v2:172.21.15.155:3300/0 conn(0x7facc40cfca0 0x7facc406f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:55.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.708+0000 7facca200640 1 -- 172.21.15.86:0/3159714187 >> v1:172.21.15.155:6789/0 conn(0x7facc40d16e0 legacy=0x7facc40d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:55.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.708+0000 7facca200640 1 -- 172.21.15.86:0/3159714187 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc406fbd0 con 0x7facc40730b0 2024-04-25T14:05:55.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.708+0000 7facc9800640 1 --2- 172.21.15.86:0/3159714187 >> v2:172.21.15.155:3300/0 conn(0x7facc40cfca0 0x7facc406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:55.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.712+0000 7facca200640 1 --2- 172.21.15.86:0/3159714187 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facc40730b0 0x7facc40734c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7facbc009e50 tx=0x7facbc01ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d3c243279e91035 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:55.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.712+0000 7facc3e00640 1 -- 172.21.15.86:0/3159714187 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc0a5810 con 0x7facc40730b0 2024-04-25T14:05:55.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.712+0000 7facc3e00640 1 -- 172.21.15.86:0/3159714187 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facbc0a59b0 con 0x7facc40730b0 2024-04-25T14:05:55.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.712+0000 7facc3e00640 1 -- 172.21.15.86:0/3159714187 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc0a5d80 con 0x7facc40730b0 2024-04-25T14:05:55.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.712+0000 7faccc200640 1 -- 172.21.15.86:0/3159714187 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facc40730b0 msgr2=0x7facc40734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:55.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.712+0000 7faccc200640 1 --2- 172.21.15.86:0/3159714187 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facc40730b0 0x7facc40734c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7facbc009e50 tx=0x7facbc01ca30 comp rx=0 tx=0).stop 2024-04-25T14:05:55.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.712+0000 7faccc200640 1 -- 172.21.15.86:0/3159714187 shutdown_connections 2024-04-25T14:05:55.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.712+0000 7faccc200640 1 --2- 172.21.15.86:0/3159714187 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facc40730b0 0x7facc40734c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:55.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.712+0000 7faccc200640 1 --2- 172.21.15.86:0/3159714187 >> v2:172.21.15.155:3300/0 conn(0x7facc40cfca0 0x7facc406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:55.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.712+0000 7faccc200640 1 -- 172.21.15.86:0/3159714187 >> 172.21.15.86:0/3159714187 conn(0x7facc40b4650 msgr2=0x7facc40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:55.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.712+0000 7faccc200640 1 -- 172.21.15.86:0/3159714187 shutdown_connections 2024-04-25T14:05:55.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.712+0000 7faccc200640 1 -- 172.21.15.86:0/3159714187 wait complete. 2024-04-25T14:05:55.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.712+0000 7faccc200640 1 Processor -- start 2024-04-25T14:05:55.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.716+0000 7faccc200640 1 -- start start 2024-04-25T14:05:55.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.716+0000 7faccc200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facc40730b0 0x7facc409a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:55.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.716+0000 7faccc200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facc40cfca0 0x7facc41f9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:55.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.716+0000 7faccc200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc40d16e0 0x7facc40cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:55.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.716+0000 7faccc200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7facc40ce0f0 con 0x7facc40d16e0 2024-04-25T14:05:55.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.716+0000 7faccc200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7facc40cdf70 con 0x7facc40cfca0 2024-04-25T14:05:55.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.716+0000 7faccc200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7facc40ce270 con 0x7facc40730b0 2024-04-25T14:05:55.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.716+0000 7facca200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc40d16e0 0x7facc40cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:55.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.716+0000 7facca200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc40d16e0 0x7facc40cb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55148/0 (socket says 172.21.15.86:55148) 2024-04-25T14:05:55.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.716+0000 7facca200640 1 -- 172.21.15.86:0/2148973841 learned_addr learned my addr 172.21.15.86:0/2148973841 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:55.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.716+0000 7facc9800640 1 --2- 172.21.15.86:0/2148973841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facc40730b0 0x7facc409a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:55.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.716+0000 7facc8e00640 1 --2- 172.21.15.86:0/2148973841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facc40cfca0 0x7facc41f9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:55.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.716+0000 7facca200640 1 -- 172.21.15.86:0/2148973841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facc40730b0 msgr2=0x7facc409a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:55.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.716+0000 7facca200640 1 --2- 172.21.15.86:0/2148973841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facc40730b0 0x7facc409a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:55.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.716+0000 7facca200640 1 -- 172.21.15.86:0/2148973841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facc40cfca0 msgr2=0x7facc41f9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:55.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.716+0000 7facca200640 1 --2- 172.21.15.86:0/2148973841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facc40cfca0 0x7facc41f9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:55.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.716+0000 7facca200640 1 -- 172.21.15.86:0/2148973841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc40caba0 con 0x7facc40d16e0 2024-04-25T14:05:55.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.716+0000 7facca200640 1 --2- 172.21.15.86:0/2148973841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc40d16e0 0x7facc40cb080 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7facbc01a660 tx=0x7facbc0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:55.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.716+0000 7facc1600640 1 -- 172.21.15.86:0/2148973841 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc009510 con 0x7facc40d16e0 2024-04-25T14:05:55.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.716+0000 7facc1600640 1 -- 172.21.15.86:0/2148973841 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facbc0be070 con 0x7facc40d16e0 2024-04-25T14:05:55.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.716+0000 7facc1600640 1 -- 172.21.15.86:0/2148973841 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc0b88b0 con 0x7facc40d16e0 2024-04-25T14:05:55.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.716+0000 7faccc200640 1 -- 172.21.15.86:0/2148973841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc4159b20 con 0x7facc40d16e0 2024-04-25T14:05:55.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.716+0000 7faccc200640 1 -- 172.21.15.86:0/2148973841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc409a610 con 0x7facc40d16e0 2024-04-25T14:05:55.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.720+0000 7faccc200640 1 -- 172.21.15.86:0/2148973841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facc4160080 con 0x7facc40d16e0 2024-04-25T14:05:55.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.720+0000 7facc1600640 1 -- 172.21.15.86:0/2148973841 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7facbc0b8a50 con 0x7facc40d16e0 2024-04-25T14:05:55.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.720+0000 7facc1600640 1 --2- 172.21.15.86:0/2148973841 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7faca406f5a0 0x7faca4071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:55.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.720+0000 7facc1600640 1 -- 172.21.15.86:0/2148973841 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 7359+0+0 (secure 0 0 0) 0x7facbc12b3e0 con 0x7facc40d16e0 2024-04-25T14:05:55.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.720+0000 7facc9800640 1 --2- 172.21.15.86:0/2148973841 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7faca406f5a0 0x7faca4071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:55.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.720+0000 7facc9800640 1 --2- 172.21.15.86:0/2148973841 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7faca406f5a0 0x7faca4071ab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7facb8009580 tx=0x7facb8009190 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:55.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.728+0000 7facc1600640 1 -- 172.21.15.86:0/2148973841 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7facbc016620 con 0x7facc40d16e0 2024-04-25T14:05:55.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.888+0000 7faccc200640 1 -- 172.21.15.86:0/2148973841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7facc409f4b0 con 0x7facc40d16e0 2024-04-25T14:05:55.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.888+0000 7facc1600640 1 -- 172.21.15.86:0/2148973841 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+21284 (secure 0 0 0) 0x7facbc0f74e0 con 0x7facc40d16e0 2024-04-25T14:05:55.891 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:55.891 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":53,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:05:54.565574+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"fs1_metadata","create_time":"2024-04-25T14:05:30.939601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"fs1"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"fs1_data","create_time":"2024-04-25T14:05:31.898262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"fs1"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"fs2_metadata","create_time":"2024-04-25T14:05:52.841764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"fs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"fs2_data","create_time":"2024-04-25T14:05:53.524454+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"fs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/615772001":"2024-04-26T14:05:42.882867+0000","172.21.15.86:6835/615772001":"2024-04-26T14:05:42.882867+0000","172.21.15.86:6835/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:0/2073988773":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6833/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6832/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/3558294881":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/1843278167":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6837/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:0/2602032945":"2024-04-26T14:05:21.460855+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:05:55.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.892+0000 7faccc200640 1 -- 172.21.15.86:0/2148973841 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7faca406f5a0 msgr2=0x7faca4071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:55.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.892+0000 7faccc200640 1 --2- 172.21.15.86:0/2148973841 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7faca406f5a0 0x7faca4071ab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7facb8009580 tx=0x7facb8009190 comp rx=0 tx=0).stop 2024-04-25T14:05:55.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.892+0000 7faccc200640 1 -- 172.21.15.86:0/2148973841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc40d16e0 msgr2=0x7facc40cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:55.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.892+0000 7faccc200640 1 --2- 172.21.15.86:0/2148973841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc40d16e0 0x7facc40cb080 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7facbc01a660 tx=0x7facbc0197d0 comp rx=0 tx=0).stop 2024-04-25T14:05:55.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.892+0000 7faccc200640 1 -- 172.21.15.86:0/2148973841 shutdown_connections 2024-04-25T14:05:55.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.892+0000 7faccc200640 1 --2- 172.21.15.86:0/2148973841 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7faca406f5a0 0x7faca4071ab0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:55.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.892+0000 7faccc200640 1 --2- 172.21.15.86:0/2148973841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc40d16e0 0x7facc40cb080 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:55.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.892+0000 7faccc200640 1 --2- 172.21.15.86:0/2148973841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facc40cfca0 0x7facc41f9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:55.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.892+0000 7faccc200640 1 --2- 172.21.15.86:0/2148973841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facc40730b0 0x7facc409a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:55.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.892+0000 7faccc200640 1 -- 172.21.15.86:0/2148973841 >> 172.21.15.86:0/2148973841 conn(0x7facc40b4650 msgr2=0x7facc415be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:55.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.892+0000 7faccc200640 1 -- 172.21.15.86:0/2148973841 shutdown_connections 2024-04-25T14:05:55.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:55.892+0000 7faccc200640 1 -- 172.21.15.86:0/2148973841 wait complete. 2024-04-25T14:05:55.913 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set fs2 standby_count_wanted 0 2024-04-25T14:05:56.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.060+0000 7fe3abe00640 1 Processor -- start 2024-04-25T14:05:56.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.060+0000 7fe3abe00640 1 -- start start 2024-04-25T14:05:56.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.064+0000 7fe3abe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe3a40cfca0 0x7fe3a406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:56.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.064+0000 7fe3abe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3a40d16e0 0x7fe3a40d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:56.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.064+0000 7fe3abe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3a40d2030 con 0x7fe3a406c670 2024-04-25T14:05:56.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.064+0000 7fe3abe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe3a4070470 con 0x7fe3a40cfca0 2024-04-25T14:05:56.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.064+0000 7fe3abe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe3a4072bd0 con 0x7fe3a40d16e0 2024-04-25T14:05:56.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.064+0000 7fe3a9400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe3a40cfca0 0x7fe3a406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:56.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.064+0000 7fe3a9e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe3a406c670 0x7fe3a406ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53340/0 (socket says 172.21.15.86:53340) 2024-04-25T14:05:56.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.064+0000 7fe3a9e00640 1 -- 172.21.15.86:0/3306154266 learned_addr learned my addr 172.21.15.86:0/3306154266 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:56.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.064+0000 7fe3a8a00640 1 --2- 172.21.15.86:0/3306154266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3a40d16e0 0x7fe3a40d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:56.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.064+0000 7fe3a9400640 1 -- 172.21.15.86:0/3306154266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3a40d16e0 msgr2=0x7fe3a40d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:56.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.064+0000 7fe3a9400640 1 --2- 172.21.15.86:0/3306154266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3a40d16e0 0x7fe3a40d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:56.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.064+0000 7fe3a9400640 1 -- 172.21.15.86:0/3306154266 >> v1:172.21.15.86:6789/0 conn(0x7fe3a406c670 legacy=0x7fe3a406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:56.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.064+0000 7fe3a9400640 1 -- 172.21.15.86:0/3306154266 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a406f930 con 0x7fe3a40cfca0 2024-04-25T14:05:56.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.064+0000 7fe3a3e00640 1 -- 172.21.15.86:0/3306154266 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2116820608 0 0) 0x7fe39c0014a0 con 0x7fe3a406c670 2024-04-25T14:05:56.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.068+0000 7fe3a9400640 1 --2- 172.21.15.86:0/3306154266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3a40cfca0 0x7fe3a406f3f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe3980097c0 tx=0x7fe39801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3e68468d699bec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:56.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.068+0000 7fe3a3e00640 1 -- 172.21.15.86:0/3306154266 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3980b1070 con 0x7fe3a40cfca0 2024-04-25T14:05:56.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.068+0000 7fe3a3e00640 1 -- 172.21.15.86:0/3306154266 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3980ac410 con 0x7fe3a40cfca0 2024-04-25T14:05:56.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.068+0000 7fe3a3e00640 1 -- 172.21.15.86:0/3306154266 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3980ac7e0 con 0x7fe3a40cfca0 2024-04-25T14:05:56.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.068+0000 7fe3abe00640 1 -- 172.21.15.86:0/3306154266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3a40cfca0 msgr2=0x7fe3a406f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:56.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.068+0000 7fe3abe00640 1 --2- 172.21.15.86:0/3306154266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3a40cfca0 0x7fe3a406f3f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe3980097c0 tx=0x7fe39801c4a0 comp rx=0 tx=0).stop 2024-04-25T14:05:56.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.068+0000 7fe3abe00640 1 -- 172.21.15.86:0/3306154266 shutdown_connections 2024-04-25T14:05:56.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.068+0000 7fe3abe00640 1 --2- 172.21.15.86:0/3306154266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3a40d16e0 0x7fe3a40d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:56.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.068+0000 7fe3abe00640 1 --2- 172.21.15.86:0/3306154266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3a40cfca0 0x7fe3a406f3f0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:56.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.068+0000 7fe3abe00640 1 -- 172.21.15.86:0/3306154266 >> 172.21.15.86:0/3306154266 conn(0x7fe3a40b4650 msgr2=0x7fe3a40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:56.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.068+0000 7fe3abe00640 1 -- 172.21.15.86:0/3306154266 shutdown_connections 2024-04-25T14:05:56.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.068+0000 7fe3abe00640 1 -- 172.21.15.86:0/3306154266 wait complete. 2024-04-25T14:05:56.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.068+0000 7fe3abe00640 1 Processor -- start 2024-04-25T14:05:56.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.068+0000 7fe3abe00640 1 -- start start 2024-04-25T14:05:56.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.072+0000 7fe3abe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3a406c670 0x7fe3a409a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:56.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.072+0000 7fe3abe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3a40cfca0 0x7fe3a4154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:56.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.072+0000 7fe3abe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3a40d16e0 0x7fe3a41550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:56.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.072+0000 7fe3abe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a416a070 con 0x7fe3a40cfca0 2024-04-25T14:05:56.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.072+0000 7fe3abe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a4169ef0 con 0x7fe3a406c670 2024-04-25T14:05:56.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.072+0000 7fe3abe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe3a416a1f0 con 0x7fe3a40d16e0 2024-04-25T14:05:56.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.072+0000 7fe3a8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3a40cfca0 0x7fe3a4154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:56.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.072+0000 7fe3a8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3a40cfca0 0x7fe3a4154bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55168/0 (socket says 172.21.15.86:55168) 2024-04-25T14:05:56.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.072+0000 7fe3a8a00640 1 -- 172.21.15.86:0/1785332806 learned_addr learned my addr 172.21.15.86:0/1785332806 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:56.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.072+0000 7fe3a9400640 1 --2- 172.21.15.86:0/1785332806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3a406c670 0x7fe3a409a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:56.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.072+0000 7fe3a9e00640 1 --2- 172.21.15.86:0/1785332806 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3a40d16e0 0x7fe3a41550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:56.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.072+0000 7fe3a8a00640 1 -- 172.21.15.86:0/1785332806 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3a40d16e0 msgr2=0x7fe3a41550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:56.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.072+0000 7fe3a8a00640 1 --2- 172.21.15.86:0/1785332806 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3a40d16e0 0x7fe3a41550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:56.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.072+0000 7fe3a8a00640 1 -- 172.21.15.86:0/1785332806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3a406c670 msgr2=0x7fe3a409a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:56.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.072+0000 7fe3a8a00640 1 --2- 172.21.15.86:0/1785332806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3a406c670 0x7fe3a409a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:56.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.072+0000 7fe3a8a00640 1 -- 172.21.15.86:0/1785332806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a415f990 con 0x7fe3a40cfca0 2024-04-25T14:05:56.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.072+0000 7fe3a9e00640 1 --2- 172.21.15.86:0/1785332806 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3a40d16e0 0x7fe3a41550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:56.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.072+0000 7fe3a8a00640 1 --2- 172.21.15.86:0/1785332806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3a40cfca0 0x7fe3a4154bb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe39000d650 tx=0x7fe39000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:56.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.072+0000 7fe3a1600640 1 -- 172.21.15.86:0/1785332806 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39000b6c0 con 0x7fe3a40cfca0 2024-04-25T14:05:56.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.072+0000 7fe3a1600640 1 -- 172.21.15.86:0/1785332806 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe390019070 con 0x7fe3a40cfca0 2024-04-25T14:05:56.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.072+0000 7fe3a1600640 1 -- 172.21.15.86:0/1785332806 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39000ba80 con 0x7fe3a40cfca0 2024-04-25T14:05:56.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.072+0000 7fe3abe00640 1 -- 172.21.15.86:0/1785332806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a415fc70 con 0x7fe3a40cfca0 2024-04-25T14:05:56.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.072+0000 7fe3abe00640 1 -- 172.21.15.86:0/1785332806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a40b2060 con 0x7fe3a40cfca0 2024-04-25T14:05:56.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.072+0000 7fe3abe00640 1 -- 172.21.15.86:0/1785332806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3a415d5c0 con 0x7fe3a40cfca0 2024-04-25T14:05:56.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.076+0000 7fe3a1600640 1 -- 172.21.15.86:0/1785332806 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7fe39000bc20 con 0x7fe3a40cfca0 2024-04-25T14:05:56.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.076+0000 7fe3a1600640 1 --2- 172.21.15.86:0/1785332806 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fe38406f5a0 0x7fe384071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:56.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.076+0000 7fe3a1600640 1 -- 172.21.15.86:0/1785332806 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 7359+0+0 (secure 0 0 0) 0x7fe39008fe60 con 0x7fe3a40cfca0 2024-04-25T14:05:56.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.076+0000 7fe3a9400640 1 --2- 172.21.15.86:0/1785332806 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fe38406f5a0 0x7fe384071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:56.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.076+0000 7fe3a9400640 1 --2- 172.21.15.86:0/1785332806 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fe38406f5a0 0x7fe384071ab0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe3980ae000 tx=0x7fe398008040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:56.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.080+0000 7fe3a1600640 1 -- 172.21.15.86:0/1785332806 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe39005c5c0 con 0x7fe3a40cfca0 2024-04-25T14:05:56.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:56.276+0000 7fe3abe00640 1 -- 172.21.15.86:0/1785332806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "fs2", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe3a409f4b0 con 0x7fe3a40cfca0 2024-04-25T14:05:56.494 INFO:tasks.ceph.mgr.y.smithi086.stderr:Exception in thread Thread-12: 2024-04-25T14:05:56.494 INFO:tasks.ceph.mgr.y.smithi086.stderr:Traceback (most recent call last): 2024-04-25T14:05:56.494 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner 2024-04-25T14:05:56.495 INFO:tasks.ceph.mgr.y.smithi086.stderr: self.run() 2024-04-25T14:05:56.495 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/lib/python3.10/threading.py", line 1378, in run 2024-04-25T14:05:56.495 INFO:tasks.ceph.mgr.y.smithi086.stderr: self.function(*self.args, **self.kwargs) 2024-04-25T14:05:56.495 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/share/ceph/mgr/stats/fs/perf_stats.py", line 222, in re_register_queries 2024-04-25T14:05:56.495 INFO:tasks.ceph.mgr.y.smithi086.stderr: if self.mx_last_updated >= ua_last_updated: 2024-04-25T14:05:56.495 INFO:tasks.ceph.mgr.y.smithi086.stderr:AttributeError: 'FSPerfStats' object has no attribute 'mx_last_updated' 2024-04-25T14:05:57.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.188+0000 7fe3a1600640 1 -- 172.21.15.86:0/1785332806 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "fs2", "var": "standby_count_wanted", "val": "0"}]=0 v35) ==== 115+0+0 (secure 0 0 0) 0x7fe39005bec0 con 0x7fe3a40cfca0 2024-04-25T14:05:57.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.188+0000 7fe3abe00640 1 -- 172.21.15.86:0/1785332806 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fe38406f5a0 msgr2=0x7fe384071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:57.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.188+0000 7fe3abe00640 1 --2- 172.21.15.86:0/1785332806 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fe38406f5a0 0x7fe384071ab0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe3980ae000 tx=0x7fe398008040 comp rx=0 tx=0).stop 2024-04-25T14:05:57.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.188+0000 7fe3abe00640 1 -- 172.21.15.86:0/1785332806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3a40cfca0 msgr2=0x7fe3a4154bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:57.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.188+0000 7fe3abe00640 1 --2- 172.21.15.86:0/1785332806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3a40cfca0 0x7fe3a4154bb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe39000d650 tx=0x7fe39000db10 comp rx=0 tx=0).stop 2024-04-25T14:05:57.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.188+0000 7fe3abe00640 1 -- 172.21.15.86:0/1785332806 shutdown_connections 2024-04-25T14:05:57.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.188+0000 7fe3abe00640 1 --2- 172.21.15.86:0/1785332806 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fe38406f5a0 0x7fe384071ab0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:57.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.188+0000 7fe3abe00640 1 --2- 172.21.15.86:0/1785332806 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3a40d16e0 0x7fe3a41550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:57.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.188+0000 7fe3abe00640 1 --2- 172.21.15.86:0/1785332806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3a40cfca0 0x7fe3a4154bb0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:57.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.188+0000 7fe3abe00640 1 --2- 172.21.15.86:0/1785332806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3a406c670 0x7fe3a409a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:57.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.188+0000 7fe3abe00640 1 -- 172.21.15.86:0/1785332806 >> 172.21.15.86:0/1785332806 conn(0x7fe3a40b4650 msgr2=0x7fe3a41560d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:57.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.188+0000 7fe3abe00640 1 -- 172.21.15.86:0/1785332806 shutdown_connections 2024-04-25T14:05:57.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.188+0000 7fe3abe00640 1 -- 172.21.15.86:0/1785332806 wait complete. 2024-04-25T14:05:57.213 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T14:05:57.213 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set fs2 allow_standby_replay false 2024-04-25T14:05:57.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72d9400640 1 Processor -- start 2024-04-25T14:05:57.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72d9400640 1 -- start start 2024-04-25T14:05:57.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72d9400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d415c2a0 0x7f72d40ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:57.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72d9400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f72d406fd50 0x7f72d4070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:57.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72d9400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72d40b0a60 con 0x7f72d406c670 2024-04-25T14:05:57.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72d9400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f72d406ef90 con 0x7f72d406fd50 2024-04-25T14:05:57.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72d9400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f72d406f270 con 0x7f72d415c2a0 2024-04-25T14:05:57.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72d3200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f72d406c670 0x7f72d406ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53356/0 (socket says 172.21.15.86:53356) 2024-04-25T14:05:57.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72d3200640 1 -- 172.21.15.86:0/1243815980 learned_addr learned my addr 172.21.15.86:0/1243815980 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:57.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72d2800640 1 --2- 172.21.15.86:0/1243815980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d415c2a0 0x7f72d40ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:57.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72c9e00640 1 --2- 172.21.15.86:0/1243815980 >> v2:172.21.15.155:3300/0 conn(0x7f72d406fd50 0x7f72d4070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:57.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72d1e00640 1 -- 172.21.15.86:0/1243815980 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 369522042 0 0) 0x7f72d40b0a60 con 0x7f72d406c670 2024-04-25T14:05:57.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72d1e00640 1 -- 172.21.15.86:0/1243815980 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72b8005740 con 0x7f72d406c670 2024-04-25T14:05:57.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72d1e00640 1 -- 172.21.15.86:0/1243815980 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3634015498 0 0) 0x7f72b8005740 con 0x7f72d406c670 2024-04-25T14:05:57.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72d1e00640 1 -- 172.21.15.86:0/1243815980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d415c2a0 msgr2=0x7f72d40ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:57.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72d1e00640 1 --2- 172.21.15.86:0/1243815980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d415c2a0 0x7f72d40ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:57.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72d1e00640 1 -- 172.21.15.86:0/1243815980 >> v2:172.21.15.155:3300/0 conn(0x7f72d406fd50 msgr2=0x7f72d4070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:57.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72d1e00640 1 --2- 172.21.15.86:0/1243815980 >> v2:172.21.15.155:3300/0 conn(0x7f72d406fd50 0x7f72d4070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:57.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72d1e00640 1 -- 172.21.15.86:0/1243815980 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d406f550 con 0x7f72d406c670 2024-04-25T14:05:57.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72d1e00640 1 -- 172.21.15.86:0/1243815980 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f72cc001a00 con 0x7f72d406c670 2024-04-25T14:05:57.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72d1e00640 1 -- 172.21.15.86:0/1243815980 >> v1:172.21.15.86:6789/0 conn(0x7f72d406c670 legacy=0x7f72d406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:57.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72d1e00640 1 --2- 172.21.15.86:0/1243815980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d415c2a0 0x7f72b8005c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:57.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72d1e00640 1 --2- 172.21.15.86:0/1243815980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72b80061c0 0x7f72b8006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:57.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72d1e00640 1 --2- 172.21.15.86:0/1243815980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b8006f60 0x7f72b8007370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:57.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72d1e00640 1 -- 172.21.15.86:0/1243815980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72d406ef90 con 0x7f72b8006f60 2024-04-25T14:05:57.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72d1e00640 1 -- 172.21.15.86:0/1243815980 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f72d406f270 con 0x7f72b80061c0 2024-04-25T14:05:57.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72d1e00640 1 -- 172.21.15.86:0/1243815980 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f72b800dd40 con 0x7f72d415c2a0 2024-04-25T14:05:57.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72d3200640 1 --2- 172.21.15.86:0/1243815980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b8006f60 0x7f72b8007370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:57.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72d2800640 1 --2- 172.21.15.86:0/1243815980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d415c2a0 0x7f72b8005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:57.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.364+0000 7f72c9e00640 1 --2- 172.21.15.86:0/1243815980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72b80061c0 0x7f72b8006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:57.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.368+0000 7f72d2800640 1 -- 172.21.15.86:0/1243815980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72b80061c0 msgr2=0x7f72b8006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:57.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.368+0000 7f72d2800640 1 --2- 172.21.15.86:0/1243815980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72b80061c0 0x7f72b8006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:57.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.368+0000 7f72d2800640 1 -- 172.21.15.86:0/1243815980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b8006f60 msgr2=0x7f72b8007370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:57.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.368+0000 7f72d2800640 1 --2- 172.21.15.86:0/1243815980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b8006f60 0x7f72b8007370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:57.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.368+0000 7f72d2800640 1 -- 172.21.15.86:0/1243815980 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b8005740 con 0x7f72d415c2a0 2024-04-25T14:05:57.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.368+0000 7f72d3200640 1 --2- 172.21.15.86:0/1243815980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b8006f60 0x7f72b8007370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:57.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.368+0000 7f72d2800640 1 --2- 172.21.15.86:0/1243815980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d415c2a0 0x7f72b8005c40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f72d4070bb0 tx=0x7f72bc01ca50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2086b18453c680d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:57.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.368+0000 7f72d1e00640 1 -- 172.21.15.86:0/1243815980 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc0b2070 con 0x7f72d415c2a0 2024-04-25T14:05:57.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.368+0000 7f72d1e00640 1 -- 172.21.15.86:0/1243815980 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72bc0a5640 con 0x7f72d415c2a0 2024-04-25T14:05:57.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.368+0000 7f72d1e00640 1 -- 172.21.15.86:0/1243815980 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc0096b0 con 0x7f72d415c2a0 2024-04-25T14:05:57.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.368+0000 7f72d9400640 1 -- 172.21.15.86:0/1243815980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d415c2a0 msgr2=0x7f72b8005c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:57.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.368+0000 7f72d9400640 1 --2- 172.21.15.86:0/1243815980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d415c2a0 0x7f72b8005c40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f72d4070bb0 tx=0x7f72bc01ca50 comp rx=0 tx=0).stop 2024-04-25T14:05:57.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.372+0000 7f72d3200640 1 -- 172.21.15.86:0/1243815980 reap_dead start 2024-04-25T14:05:57.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.372+0000 7f72d9400640 1 -- 172.21.15.86:0/1243815980 shutdown_connections 2024-04-25T14:05:57.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.372+0000 7f72d9400640 1 -- 172.21.15.86:0/1243815980 >> 172.21.15.86:0/1243815980 conn(0x7f72d40b4650 msgr2=0x7f72d40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:57.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.372+0000 7f72d9400640 1 -- 172.21.15.86:0/1243815980 shutdown_connections 2024-04-25T14:05:57.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.372+0000 7f72d9400640 1 -- 172.21.15.86:0/1243815980 wait complete. 2024-04-25T14:05:57.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.372+0000 7f72d9400640 1 Processor -- start 2024-04-25T14:05:57.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.372+0000 7f72d9400640 1 -- start start 2024-04-25T14:05:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.376+0000 7f72d9400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d415c2a0 0x7f72d409ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.376+0000 7f72d9400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d4154a20 0x7f72d4158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.376+0000 7f72d9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72d4152620 0x7f72d41f2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.376+0000 7f72d9400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72d40d0fc0 con 0x7f72d4152620 2024-04-25T14:05:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.376+0000 7f72d9400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f72d40706a0 con 0x7f72d4154a20 2024-04-25T14:05:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.376+0000 7f72d9400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f72d415c6b0 con 0x7f72d415c2a0 2024-04-25T14:05:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.376+0000 7f72d3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72d4152620 0x7f72d41f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.376+0000 7f72d3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72d4152620 0x7f72d41f2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55198/0 (socket says 172.21.15.86:55198) 2024-04-25T14:05:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.376+0000 7f72d3200640 1 -- 172.21.15.86:0/2546996255 learned_addr learned my addr 172.21.15.86:0/2546996255 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.376+0000 7f72d2800640 1 --2- 172.21.15.86:0/2546996255 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d415c2a0 0x7f72d409ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.376+0000 7f72c9e00640 1 --2- 172.21.15.86:0/2546996255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d4154a20 0x7f72d4158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.376+0000 7f72d3200640 1 -- 172.21.15.86:0/2546996255 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d415c2a0 msgr2=0x7f72d409ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:57.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.376+0000 7f72d3200640 1 --2- 172.21.15.86:0/2546996255 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d415c2a0 0x7f72d409ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:57.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.376+0000 7f72d3200640 1 -- 172.21.15.86:0/2546996255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d4154a20 msgr2=0x7f72d4158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:57.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.376+0000 7f72d3200640 1 --2- 172.21.15.86:0/2546996255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d4154a20 0x7f72d4158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:57.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.376+0000 7f72d3200640 1 -- 172.21.15.86:0/2546996255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d4155200 con 0x7f72d4152620 2024-04-25T14:05:57.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.376+0000 7f72c9e00640 1 --2- 172.21.15.86:0/2546996255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d4154a20 0x7f72d4158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:57.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.376+0000 7f72d3200640 1 --2- 172.21.15.86:0/2546996255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72d4152620 0x7f72d41f2800 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f72d40cfea0 tx=0x7f72cc012630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:57.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.376+0000 7f72cb400640 1 -- 172.21.15.86:0/2546996255 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72cc00a4c0 con 0x7f72d4152620 2024-04-25T14:05:57.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.376+0000 7f72cb400640 1 -- 172.21.15.86:0/2546996255 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72cc007110 con 0x7f72d4152620 2024-04-25T14:05:57.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.376+0000 7f72cb400640 1 -- 172.21.15.86:0/2546996255 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72cc0072c0 con 0x7f72d4152620 2024-04-25T14:05:57.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.376+0000 7f72d9400640 1 -- 172.21.15.86:0/2546996255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72d4157840 con 0x7f72d4152620 2024-04-25T14:05:57.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.376+0000 7f72d9400640 1 -- 172.21.15.86:0/2546996255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72d41f30e0 con 0x7f72d4152620 2024-04-25T14:05:57.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.376+0000 7f72d9400640 1 -- 172.21.15.86:0/2546996255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72d41f3340 con 0x7f72d4152620 2024-04-25T14:05:57.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.376+0000 7f72cb400640 1 -- 172.21.15.86:0/2546996255 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f72cc01e020 con 0x7f72d4152620 2024-04-25T14:05:57.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.380+0000 7f72cb400640 1 --2- 172.21.15.86:0/2546996255 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f72b006f5a0 0x7f72b0071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:57.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.380+0000 7f72cb400640 1 -- 172.21.15.86:0/2546996255 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 7359+0+0 (secure 0 0 0) 0x7f72cc093210 con 0x7f72d4152620 2024-04-25T14:05:57.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.380+0000 7f72d2800640 1 --2- 172.21.15.86:0/2546996255 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f72b006f5a0 0x7f72b0071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:57.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.380+0000 7f72d2800640 1 --2- 172.21.15.86:0/2546996255 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f72b006f5a0 0x7f72b0071ab0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f72bc0af000 tx=0x7f72bc0046d0 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:57.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.384+0000 7f72cb400640 1 -- 172.21.15.86:0/2546996255 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f72cc05faa0 con 0x7f72d4152620 2024-04-25T14:05:57.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:57.588+0000 7f72d9400640 1 -- 172.21.15.86:0/2546996255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "fs2", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f72d4069d00 con 0x7f72d4152620 2024-04-25T14:05:57.894 INFO:tasks.ceph.mds.c.smithi086.stderr:2024-04-25T14:05:57.892+0000 7f018d000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T14:05:58.189 INFO:tasks.ceph.mgr.y.smithi086.stderr:Exception in thread Thread-14: 2024-04-25T14:05:58.190 INFO:tasks.ceph.mgr.y.smithi086.stderr:Traceback (most recent call last): 2024-04-25T14:05:58.190 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner 2024-04-25T14:05:58.190 INFO:tasks.ceph.mgr.y.smithi086.stderr: self.run() 2024-04-25T14:05:58.190 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/lib/python3.10/threading.py", line 1378, in run 2024-04-25T14:05:58.190 INFO:tasks.ceph.mgr.y.smithi086.stderr: self.function(*self.args, **self.kwargs) 2024-04-25T14:05:58.190 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/share/ceph/mgr/stats/fs/perf_stats.py", line 222, in re_register_queries 2024-04-25T14:05:58.191 INFO:tasks.ceph.mgr.y.smithi086.stderr: if self.mx_last_updated >= ua_last_updated: 2024-04-25T14:05:58.191 INFO:tasks.ceph.mgr.y.smithi086.stderr:AttributeError: 'FSPerfStats' object has no attribute 'mx_last_updated' 2024-04-25T14:05:58.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.196+0000 7f72cb400640 1 -- 172.21.15.86:0/2546996255 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "fs2", "var": "allow_standby_replay", "val": "false"}]=0 v36) ==== 119+0+0 (secure 0 0 0) 0x7f72cc05f3a0 con 0x7f72d4152620 2024-04-25T14:05:58.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.200+0000 7f72d9400640 1 -- 172.21.15.86:0/2546996255 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f72b006f5a0 msgr2=0x7f72b0071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:58.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.200+0000 7f72d9400640 1 --2- 172.21.15.86:0/2546996255 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f72b006f5a0 0x7f72b0071ab0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f72bc0af000 tx=0x7f72bc0046d0 comp rx=0 tx=0).stop 2024-04-25T14:05:58.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.200+0000 7f72d9400640 1 -- 172.21.15.86:0/2546996255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72d4152620 msgr2=0x7f72d41f2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:58.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.200+0000 7f72d9400640 1 --2- 172.21.15.86:0/2546996255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72d4152620 0x7f72d41f2800 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f72d40cfea0 tx=0x7f72cc012630 comp rx=0 tx=0).stop 2024-04-25T14:05:58.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.200+0000 7f72d9400640 1 -- 172.21.15.86:0/2546996255 shutdown_connections 2024-04-25T14:05:58.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.200+0000 7f72d9400640 1 --2- 172.21.15.86:0/2546996255 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f72b006f5a0 0x7f72b0071ab0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:58.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.200+0000 7f72d9400640 1 --2- 172.21.15.86:0/2546996255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72d4152620 0x7f72d41f2800 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:58.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.200+0000 7f72d9400640 1 --2- 172.21.15.86:0/2546996255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d4154a20 0x7f72d4158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:58.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.200+0000 7f72d9400640 1 --2- 172.21.15.86:0/2546996255 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d415c2a0 0x7f72d409ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:58.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.200+0000 7f72d9400640 1 -- 172.21.15.86:0/2546996255 >> 172.21.15.86:0/2546996255 conn(0x7f72d40b4650 msgr2=0x7f72d4073d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:58.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.200+0000 7f72d9400640 1 -- 172.21.15.86:0/2546996255 shutdown_connections 2024-04-25T14:05:58.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.200+0000 7f72d9400640 1 -- 172.21.15.86:0/2546996255 wait complete. 2024-04-25T14:05:58.227 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'fs2' 2024-04-25T14:05:58.227 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:05:58.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.376+0000 7fcc39000640 1 Processor -- start 2024-04-25T14:05:58.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.376+0000 7fcc39000640 1 -- start start 2024-04-25T14:05:58.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.376+0000 7fcc39000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc34163f20 0x7fcc3416ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:58.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.376+0000 7fcc39000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc3406f0a0 con 0x7fcc3415fc70 2024-04-25T14:05:58.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.376+0000 7fcc39000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc341646d0 con 0x7fcc34071530 2024-04-25T14:05:58.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.376+0000 7fcc39000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcc3406e7f0 con 0x7fcc34163f20 2024-04-25T14:05:58.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.376+0000 7fcc31e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fcc3415fc70 0x7fcc34070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53362/0 (socket says 172.21.15.86:53362) 2024-04-25T14:05:58.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.376+0000 7fcc31e00640 1 -- 172.21.15.86:0/3710964336 learned_addr learned my addr 172.21.15.86:0/3710964336 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:58.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.376+0000 7fcc32800640 1 --2- 172.21.15.86:0/3710964336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc34163f20 0x7fcc3416ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:58.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.380+0000 7fcc31400640 1 -- 172.21.15.86:0/3710964336 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 919674203 0 0) 0x7fcc24001120 con 0x7fcc3415fc70 2024-04-25T14:05:58.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.380+0000 7fcc31400640 1 -- 172.21.15.86:0/3710964336 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc18003530 con 0x7fcc3415fc70 2024-04-25T14:05:58.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.380+0000 7fcc31400640 1 -- 172.21.15.86:0/3710964336 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1881353545 0 0) 0x7fcc341646d0 con 0x7fcc34071530 2024-04-25T14:05:58.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.380+0000 7fcc31400640 1 -- 172.21.15.86:0/3710964336 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc24001120 con 0x7fcc34071530 2024-04-25T14:05:58.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.380+0000 7fcc32800640 1 -- 172.21.15.86:0/3710964336 >> v1:172.21.15.155:6789/0 conn(0x7fcc34071530 legacy=0x7fcc34163810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:58.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.380+0000 7fcc31400640 1 -- 172.21.15.86:0/3710964336 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 626595179 0 0) 0x7fcc18003530 con 0x7fcc3415fc70 2024-04-25T14:05:58.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.380+0000 7fcc32800640 1 -- 172.21.15.86:0/3710964336 >> v1:172.21.15.86:6789/0 conn(0x7fcc3415fc70 legacy=0x7fcc34070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:58.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.380+0000 7fcc32800640 1 -- 172.21.15.86:0/3710964336 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc341643f0 con 0x7fcc34163f20 2024-04-25T14:05:58.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.380+0000 7fcc32800640 1 --2- 172.21.15.86:0/3710964336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc34163f20 0x7fcc3416ab60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcc3406e970 tx=0x7fcc2c01c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d45d7bb13976982 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:58.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.380+0000 7fcc31400640 1 -- 172.21.15.86:0/3710964336 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c0b2070 con 0x7fcc34163f20 2024-04-25T14:05:58.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.380+0000 7fcc31400640 1 -- 172.21.15.86:0/3710964336 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc2c0a56f0 con 0x7fcc34163f20 2024-04-25T14:05:58.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.380+0000 7fcc31400640 1 -- 172.21.15.86:0/3710964336 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc24001850 con 0x7fcc34163f20 2024-04-25T14:05:58.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.384+0000 7fcc39000640 1 -- 172.21.15.86:0/3710964336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc34163f20 msgr2=0x7fcc3416ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:58.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.384+0000 7fcc39000640 1 --2- 172.21.15.86:0/3710964336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc34163f20 0x7fcc3416ab60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcc3406e970 tx=0x7fcc2c01c8b0 comp rx=0 tx=0).stop 2024-04-25T14:05:58.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.384+0000 7fcc39000640 1 -- 172.21.15.86:0/3710964336 shutdown_connections 2024-04-25T14:05:58.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.384+0000 7fcc39000640 1 --2- 172.21.15.86:0/3710964336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc34163f20 0x7fcc3416ab60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:58.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.384+0000 7fcc39000640 1 -- 172.21.15.86:0/3710964336 >> 172.21.15.86:0/3710964336 conn(0x7fcc340cffc0 msgr2=0x7fcc340d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:58.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.384+0000 7fcc39000640 1 -- 172.21.15.86:0/3710964336 shutdown_connections 2024-04-25T14:05:58.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.384+0000 7fcc39000640 1 -- 172.21.15.86:0/3710964336 wait complete. 2024-04-25T14:05:58.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.384+0000 7fcc39000640 1 Processor -- start 2024-04-25T14:05:58.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.384+0000 7fcc39000640 1 -- start start 2024-04-25T14:05:58.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.388+0000 7fcc39000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc3415fc70 0x7fcc340cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:58.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.388+0000 7fcc39000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc34163f20 0x7fcc341570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:58.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.388+0000 7fcc39000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc340c6fa0 0x7fcc34157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:58.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.388+0000 7fcc39000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3416da40 con 0x7fcc340c6fa0 2024-04-25T14:05:58.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.388+0000 7fcc39000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3416d8c0 con 0x7fcc34163f20 2024-04-25T14:05:58.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.388+0000 7fcc39000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcc3416dbc0 con 0x7fcc3415fc70 2024-04-25T14:05:58.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.388+0000 7fcc32800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc340c6fa0 0x7fcc34157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:58.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.388+0000 7fcc32800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc340c6fa0 0x7fcc34157630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55204/0 (socket says 172.21.15.86:55204) 2024-04-25T14:05:58.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.388+0000 7fcc32800640 1 -- 172.21.15.86:0/3105044469 learned_addr learned my addr 172.21.15.86:0/3105044469 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:58.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.388+0000 7fcc31e00640 1 --2- 172.21.15.86:0/3105044469 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc3415fc70 0x7fcc340cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:58.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.388+0000 7fcc29400640 1 --2- 172.21.15.86:0/3105044469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc34163f20 0x7fcc341570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:58.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.388+0000 7fcc32800640 1 -- 172.21.15.86:0/3105044469 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc3415fc70 msgr2=0x7fcc340cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.388+0000 7fcc32800640 1 --2- 172.21.15.86:0/3105044469 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc3415fc70 0x7fcc340cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.388+0000 7fcc32800640 1 -- 172.21.15.86:0/3105044469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc34163f20 msgr2=0x7fcc341570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.388+0000 7fcc32800640 1 --2- 172.21.15.86:0/3105044469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc34163f20 0x7fcc341570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.388+0000 7fcc32800640 1 -- 172.21.15.86:0/3105044469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc34157b70 con 0x7fcc340c6fa0 2024-04-25T14:05:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.388+0000 7fcc31e00640 1 --2- 172.21.15.86:0/3105044469 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc3415fc70 0x7fcc340cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.388+0000 7fcc32800640 1 --2- 172.21.15.86:0/3105044469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc340c6fa0 0x7fcc34157630 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c0b0040 tx=0x7fcc2c019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.388+0000 7fcc2aa00640 1 -- 172.21.15.86:0/3105044469 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c0b2040 con 0x7fcc340c6fa0 2024-04-25T14:05:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.388+0000 7fcc2aa00640 1 -- 172.21.15.86:0/3105044469 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc2c0bf070 con 0x7fcc340c6fa0 2024-04-25T14:05:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.388+0000 7fcc2aa00640 1 -- 172.21.15.86:0/3105044469 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c009450 con 0x7fcc340c6fa0 2024-04-25T14:05:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.388+0000 7fcc39000640 1 -- 172.21.15.86:0/3105044469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc34154e30 con 0x7fcc340c6fa0 2024-04-25T14:05:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.388+0000 7fcc39000640 1 -- 172.21.15.86:0/3105044469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc34156610 con 0x7fcc340c6fa0 2024-04-25T14:05:58.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.388+0000 7fcc39000640 1 -- 172.21.15.86:0/3105044469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc340720e0 con 0x7fcc340c6fa0 2024-04-25T14:05:58.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.388+0000 7fcc2aa00640 1 -- 172.21.15.86:0/3105044469 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7fcc2c0b8890 con 0x7fcc340c6fa0 2024-04-25T14:05:58.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.392+0000 7fcc2aa00640 1 --2- 172.21.15.86:0/3105044469 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcc1006f570 0x7fcc10071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:58.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.392+0000 7fcc2aa00640 1 -- 172.21.15.86:0/3105044469 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 7359+0+0 (secure 0 0 0) 0x7fcc2c12aa40 con 0x7fcc340c6fa0 2024-04-25T14:05:58.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.392+0000 7fcc31e00640 1 --2- 172.21.15.86:0/3105044469 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcc1006f570 0x7fcc10071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:58.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.392+0000 7fcc31e00640 1 --2- 172.21.15.86:0/3105044469 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcc1006f570 0x7fcc10071a80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcc24001ec0 tx=0x7fcc24009040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:58.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.396+0000 7fcc2aa00640 1 -- 172.21.15.86:0/3105044469 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcc2c0164a0 con 0x7fcc340c6fa0 2024-04-25T14:05:58.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.584+0000 7fcc39000640 1 -- 172.21.15.86:0/3105044469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc340cce00 con 0x7fcc340c6fa0 2024-04-25T14:05:58.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.584+0000 7fcc2aa00640 1 -- 172.21.15.86:0/3105044469 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+6494 (secure 0 0 0) 0x7fcc2c0f7110 con 0x7fcc340c6fa0 2024-04-25T14:05:58.587 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:58.588 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5315,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3312926518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":3312926518},{"type":"v1","addr":"172.21.15.155:6835","nonce":3312926518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5776,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/898730495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":898730495},{"type":"v1","addr":"172.21.15.155:6834","nonce":898730495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:32.946320+0000","modified":"2024-04-25T14:05:45.906925+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5874},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5874":{"gid":5874,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6836/1732615622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1732615622},{"type":"v1","addr":"172.21.15.86:6836","nonce":1732615622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"fs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5874,"qdb_cluster":[5874]},"id":3},{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:54.565528+0000","modified":"2024-04-25T14:05:57.592887+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5794":{"gid":5794,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":5,"addr":"172.21.15.86:6837/2418778265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2418778265},{"type":"v1","addr":"172.21.15.86:6837","nonce":2418778265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"fs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5794,"qdb_cluster":[5794]},"id":4}]} 2024-04-25T14:05:58.588 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 36 2024-04-25T14:05:58.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.588+0000 7fcc39000640 1 -- 172.21.15.86:0/3105044469 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcc1006f570 msgr2=0x7fcc10071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:58.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.588+0000 7fcc39000640 1 --2- 172.21.15.86:0/3105044469 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcc1006f570 0x7fcc10071a80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcc24001ec0 tx=0x7fcc24009040 comp rx=0 tx=0).stop 2024-04-25T14:05:58.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.588+0000 7fcc39000640 1 -- 172.21.15.86:0/3105044469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc340c6fa0 msgr2=0x7fcc34157630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:58.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.588+0000 7fcc39000640 1 --2- 172.21.15.86:0/3105044469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc340c6fa0 0x7fcc34157630 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c0b0040 tx=0x7fcc2c019650 comp rx=0 tx=0).stop 2024-04-25T14:05:58.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.588+0000 7fcc39000640 1 -- 172.21.15.86:0/3105044469 shutdown_connections 2024-04-25T14:05:58.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.588+0000 7fcc39000640 1 --2- 172.21.15.86:0/3105044469 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcc1006f570 0x7fcc10071a80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:58.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.588+0000 7fcc39000640 1 --2- 172.21.15.86:0/3105044469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc340c6fa0 0x7fcc34157630 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:58.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.588+0000 7fcc39000640 1 --2- 172.21.15.86:0/3105044469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc34163f20 0x7fcc341570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:58.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.588+0000 7fcc39000640 1 --2- 172.21.15.86:0/3105044469 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc3415fc70 0x7fcc340cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:58.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.588+0000 7fcc39000640 1 -- 172.21.15.86:0/3105044469 >> 172.21.15.86:0/3105044469 conn(0x7fcc340cffc0 msgr2=0x7fcc34161e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:58.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.588+0000 7fcc39000640 1 -- 172.21.15.86:0/3105044469 shutdown_connections 2024-04-25T14:05:58.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.588+0000 7fcc39000640 1 -- 172.21.15.86:0/3105044469 wait complete. 2024-04-25T14:05:58.610 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:05:58.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.756+0000 7fa59ea00640 1 Processor -- start 2024-04-25T14:05:58.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.756+0000 7fa59ea00640 1 -- start start 2024-04-25T14:05:58.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.756+0000 7fa59ea00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa598071530 0x7fa5981638a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:58.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.756+0000 7fa59ea00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa598163de0 0x7fa5980cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:58.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.756+0000 7fa59ea00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa59806dc30 con 0x7fa59815fe00 2024-04-25T14:05:58.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.756+0000 7fa59ea00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa598160210 con 0x7fa598163de0 2024-04-25T14:05:58.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.756+0000 7fa59ea00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa5981641f0 con 0x7fa598071530 2024-04-25T14:05:58.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.756+0000 7fa59ca00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa598163de0 0x7fa5980cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:58.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.756+0000 7fa597400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa598071530 0x7fa5981638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:58.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.756+0000 7fa59ca00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa598163de0 0x7fa5980cdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:39850/0 (socket says 172.21.15.86:39850) 2024-04-25T14:05:58.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.756+0000 7fa59ca00640 1 -- 172.21.15.86:0/1417122117 learned_addr learned my addr 172.21.15.86:0/1417122117 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:58.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.756+0000 7fa597e00640 1 --1- 172.21.15.86:0/1417122117 >> v1:172.21.15.155:6789/0 conn(0x7fa59815fe00 0x7fa598070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:60214/0 (socket says 172.21.15.86:60214) 2024-04-25T14:05:58.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.760+0000 7fa59ca00640 1 -- 172.21.15.86:0/1417122117 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa598071530 msgr2=0x7fa5981638a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:58.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.760+0000 7fa59ca00640 1 --2- 172.21.15.86:0/1417122117 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa598071530 0x7fa5981638a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:58.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.760+0000 7fa59ca00640 1 -- 172.21.15.86:0/1417122117 >> v1:172.21.15.155:6789/0 conn(0x7fa59815fe00 legacy=0x7fa598070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:58.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.760+0000 7fa59ca00640 1 -- 172.21.15.86:0/1417122117 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa598164480 con 0x7fa598163de0 2024-04-25T14:05:58.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.760+0000 7fa596a00640 1 -- 172.21.15.86:0/1417122117 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 376586793 0 0) 0x7fa59806dc30 con 0x7fa59815fe00 2024-04-25T14:05:58.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.760+0000 7fa59ca00640 1 --2- 172.21.15.86:0/1417122117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa598163de0 0x7fa5980cdde0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa590009e50 tx=0x7fa59001ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=694e0375dc54165f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:58.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.760+0000 7fa596a00640 1 -- 172.21.15.86:0/1417122117 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5900ac020 con 0x7fa598163de0 2024-04-25T14:05:58.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.760+0000 7fa596a00640 1 -- 172.21.15.86:0/1417122117 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5900a6430 con 0x7fa598163de0 2024-04-25T14:05:58.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.760+0000 7fa596a00640 1 -- 172.21.15.86:0/1417122117 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5900a6800 con 0x7fa598163de0 2024-04-25T14:05:58.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.764+0000 7fa59ea00640 1 -- 172.21.15.86:0/1417122117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa598163de0 msgr2=0x7fa5980cdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:58.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.764+0000 7fa59ea00640 1 --2- 172.21.15.86:0/1417122117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa598163de0 0x7fa5980cdde0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa590009e50 tx=0x7fa59001ca30 comp rx=0 tx=0).stop 2024-04-25T14:05:58.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.764+0000 7fa59ea00640 1 -- 172.21.15.86:0/1417122117 shutdown_connections 2024-04-25T14:05:58.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.764+0000 7fa59ea00640 1 --2- 172.21.15.86:0/1417122117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa598163de0 0x7fa5980cdde0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:58.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.764+0000 7fa59ea00640 1 --2- 172.21.15.86:0/1417122117 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa598071530 0x7fa5981638a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:58.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.764+0000 7fa59ea00640 1 -- 172.21.15.86:0/1417122117 >> 172.21.15.86:0/1417122117 conn(0x7fa5980cffc0 msgr2=0x7fa5980d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:58.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.764+0000 7fa59ea00640 1 -- 172.21.15.86:0/1417122117 shutdown_connections 2024-04-25T14:05:58.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.764+0000 7fa59ea00640 1 -- 172.21.15.86:0/1417122117 wait complete. 2024-04-25T14:05:58.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.764+0000 7fa59ea00640 1 Processor -- start 2024-04-25T14:05:58.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.764+0000 7fa59ea00640 1 -- start start 2024-04-25T14:05:58.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.764+0000 7fa59ea00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa59815fe00 0x7fa59814e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:58.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.764+0000 7fa59ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa598163de0 0x7fa59814acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:58.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.764+0000 7fa59ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5980b3560 0x7fa598152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:58.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.768+0000 7fa59ea00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa59816cda0 con 0x7fa5980b3560 2024-04-25T14:05:58.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.768+0000 7fa59ea00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa59816cc20 con 0x7fa598163de0 2024-04-25T14:05:58.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.768+0000 7fa59ea00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa59816cf20 con 0x7fa59815fe00 2024-04-25T14:05:58.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.768+0000 7fa59ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5980b3560 0x7fa598152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:58.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.768+0000 7fa59ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5980b3560 0x7fa598152060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55212/0 (socket says 172.21.15.86:55212) 2024-04-25T14:05:58.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.768+0000 7fa59ca00640 1 -- 172.21.15.86:0/2476416780 learned_addr learned my addr 172.21.15.86:0/2476416780 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:58.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.768+0000 7fa597e00640 1 --2- 172.21.15.86:0/2476416780 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa59815fe00 0x7fa59814e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:58.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.768+0000 7fa597400640 1 --2- 172.21.15.86:0/2476416780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa598163de0 0x7fa59814acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:58.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.768+0000 7fa59ca00640 1 -- 172.21.15.86:0/2476416780 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa59815fe00 msgr2=0x7fa59814e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:58.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.768+0000 7fa59ca00640 1 --2- 172.21.15.86:0/2476416780 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa59815fe00 0x7fa59814e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:58.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.768+0000 7fa59ca00640 1 -- 172.21.15.86:0/2476416780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa598163de0 msgr2=0x7fa59814acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:58.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.768+0000 7fa59ca00640 1 --2- 172.21.15.86:0/2476416780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa598163de0 0x7fa59814acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:58.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.768+0000 7fa59ca00640 1 -- 172.21.15.86:0/2476416780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59814b1f0 con 0x7fa5980b3560 2024-04-25T14:05:58.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.768+0000 7fa59ca00640 1 --2- 172.21.15.86:0/2476416780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5980b3560 0x7fa598152060 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa5900ab040 tx=0x7fa5900197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:58.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.768+0000 7fa58fe00640 1 -- 172.21.15.86:0/2476416780 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5900b7070 con 0x7fa5980b3560 2024-04-25T14:05:58.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.768+0000 7fa58fe00640 1 -- 172.21.15.86:0/2476416780 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5900b1420 con 0x7fa5980b3560 2024-04-25T14:05:58.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.768+0000 7fa58fe00640 1 -- 172.21.15.86:0/2476416780 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590009510 con 0x7fa5980b3560 2024-04-25T14:05:58.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.768+0000 7fa59ea00640 1 -- 172.21.15.86:0/2476416780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa59814f540 con 0x7fa5980b3560 2024-04-25T14:05:58.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.768+0000 7fa59ea00640 1 -- 172.21.15.86:0/2476416780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa598161aa0 con 0x7fa5980b3560 2024-04-25T14:05:58.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.768+0000 7fa59ea00640 1 -- 172.21.15.86:0/2476416780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa59814efb0 con 0x7fa5980b3560 2024-04-25T14:05:58.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.768+0000 7fa58fe00640 1 -- 172.21.15.86:0/2476416780 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7fa5900af020 con 0x7fa5980b3560 2024-04-25T14:05:58.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.772+0000 7fa58fe00640 1 --2- 172.21.15.86:0/2476416780 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fa57806f570 0x7fa578071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:58.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.772+0000 7fa58fe00640 1 -- 172.21.15.86:0/2476416780 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 7359+0+0 (secure 0 0 0) 0x7fa590129ae0 con 0x7fa5980b3560 2024-04-25T14:05:58.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.772+0000 7fa597e00640 1 --2- 172.21.15.86:0/2476416780 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fa57806f570 0x7fa578071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:58.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.772+0000 7fa597e00640 1 --2- 172.21.15.86:0/2476416780 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fa57806f570 0x7fa578071a80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa588000f70 tx=0x7fa588007c10 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:58.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.776+0000 7fa58fe00640 1 -- 172.21.15.86:0/2476416780 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa590016620 con 0x7fa5980b3560 2024-04-25T14:05:58.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.936+0000 7fa59ea00640 1 -- 172.21.15.86:0/2476416780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5980720e0 con 0x7fa5980b3560 2024-04-25T14:05:58.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.936+0000 7fa58fe00640 1 -- 172.21.15.86:0/2476416780 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+21271 (secure 0 0 0) 0x7fa5900f61b0 con 0x7fa5980b3560 2024-04-25T14:05:58.940 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:58.940 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":54,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:05:56.176129+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"fs1_metadata","create_time":"2024-04-25T14:05:30.939601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"fs1"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"fs1_data","create_time":"2024-04-25T14:05:31.898262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"fs1"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"fs2_metadata","create_time":"2024-04-25T14:05:52.841764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"fs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"fs2_data","create_time":"2024-04-25T14:05:53.524454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"fs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/615772001":"2024-04-26T14:05:42.882867+0000","172.21.15.86:6835/615772001":"2024-04-26T14:05:42.882867+0000","172.21.15.86:6835/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:0/2073988773":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6833/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6832/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/3558294881":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/1843278167":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6837/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:0/2602032945":"2024-04-26T14:05:21.460855+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:05:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.940+0000 7fa59ea00640 1 -- 172.21.15.86:0/2476416780 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fa57806f570 msgr2=0x7fa578071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.940+0000 7fa59ea00640 1 --2- 172.21.15.86:0/2476416780 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fa57806f570 0x7fa578071a80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa588000f70 tx=0x7fa588007c10 comp rx=0 tx=0).stop 2024-04-25T14:05:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.940+0000 7fa59ea00640 1 -- 172.21.15.86:0/2476416780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5980b3560 msgr2=0x7fa598152060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.940+0000 7fa59ea00640 1 --2- 172.21.15.86:0/2476416780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5980b3560 0x7fa598152060 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa5900ab040 tx=0x7fa5900197d0 comp rx=0 tx=0).stop 2024-04-25T14:05:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.940+0000 7fa59ea00640 1 -- 172.21.15.86:0/2476416780 shutdown_connections 2024-04-25T14:05:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.940+0000 7fa59ea00640 1 --2- 172.21.15.86:0/2476416780 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fa57806f570 0x7fa578071a80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.940+0000 7fa59ea00640 1 --2- 172.21.15.86:0/2476416780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5980b3560 0x7fa598152060 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.940+0000 7fa59ea00640 1 --2- 172.21.15.86:0/2476416780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa598163de0 0x7fa59814acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.940+0000 7fa59ea00640 1 --2- 172.21.15.86:0/2476416780 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa59815fe00 0x7fa59814e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:58.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.940+0000 7fa59ea00640 1 -- 172.21.15.86:0/2476416780 >> 172.21.15.86:0/2476416780 conn(0x7fa5980cffc0 msgr2=0x7fa598152c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:58.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.940+0000 7fa59ea00640 1 -- 172.21.15.86:0/2476416780 shutdown_connections 2024-04-25T14:05:58.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:58.940+0000 7fa59ea00640 1 -- 172.21.15.86:0/2476416780 wait complete. 2024-04-25T14:05:58.962 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T14:05:58.962 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:05:59.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.104+0000 7fb4f0000640 1 Processor -- start 2024-04-25T14:05:59.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.104+0000 7fb4f0000640 1 -- start start 2024-04-25T14:05:59.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.108+0000 7fb4f0000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb4e81642a0 0x7fb4e80d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:59.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.108+0000 7fb4f0000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb4e80b0860 0x7fb4e80d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:59.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.108+0000 7fb4f0000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4e80d2030 con 0x7fb4e80d1c20 2024-04-25T14:05:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.108+0000 7fb4f0000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb4e80cea30 con 0x7fb4e81642a0 2024-04-25T14:05:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.108+0000 7fb4f0000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb4e80cde20 con 0x7fb4e80b0860 2024-04-25T14:05:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.108+0000 7fb4ed600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb4e81642a0 0x7fb4e80d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.108+0000 7fb4ed600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb4e81642a0 0x7fb4e80d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41400/0 (socket says 172.21.15.86:41400) 2024-04-25T14:05:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.108+0000 7fb4ed600640 1 -- 172.21.15.86:0/2803728591 learned_addr learned my addr 172.21.15.86:0/2803728591 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.108+0000 7fb4ee000640 1 --2- 172.21.15.86:0/2803728591 >> v2:172.21.15.155:3300/0 conn(0x7fb4e80b0860 0x7fb4e80d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.108+0000 7fb4ed600640 1 -- 172.21.15.86:0/2803728591 >> v2:172.21.15.155:3300/0 conn(0x7fb4e80b0860 msgr2=0x7fb4e80d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:59.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.108+0000 7fb4ed600640 1 --2- 172.21.15.86:0/2803728591 >> v2:172.21.15.155:3300/0 conn(0x7fb4e80b0860 0x7fb4e80d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:59.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.108+0000 7fb4ed600640 1 -- 172.21.15.86:0/2803728591 >> v1:172.21.15.155:6789/0 conn(0x7fb4e80d1c20 legacy=0x7fb4e80b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:59.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.108+0000 7fb4ed600640 1 -- 172.21.15.86:0/2803728591 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e80ce100 con 0x7fb4e81642a0 2024-04-25T14:05:59.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.108+0000 7fb4ee000640 1 --2- 172.21.15.86:0/2803728591 >> v2:172.21.15.155:3300/0 conn(0x7fb4e80b0860 0x7fb4e80d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:59.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.112+0000 7fb4ed600640 1 --2- 172.21.15.86:0/2803728591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e81642a0 0x7fb4e80d16e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80ced40 tx=0x7fb4dc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d77f42a869f05b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.112+0000 7fb4e7e00640 1 -- 172.21.15.86:0/2803728591 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc0b0070 con 0x7fb4e81642a0 2024-04-25T14:05:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.112+0000 7fb4e7e00640 1 -- 172.21.15.86:0/2803728591 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4dc0ab600 con 0x7fb4e81642a0 2024-04-25T14:05:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.112+0000 7fb4e7e00640 1 -- 172.21.15.86:0/2803728591 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc0ab9d0 con 0x7fb4e81642a0 2024-04-25T14:05:59.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.112+0000 7fb4f0000640 1 -- 172.21.15.86:0/2803728591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e81642a0 msgr2=0x7fb4e80d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:59.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.112+0000 7fb4f0000640 1 --2- 172.21.15.86:0/2803728591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e81642a0 0x7fb4e80d16e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80ced40 tx=0x7fb4dc01c490 comp rx=0 tx=0).stop 2024-04-25T14:05:59.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.112+0000 7fb4f0000640 1 -- 172.21.15.86:0/2803728591 shutdown_connections 2024-04-25T14:05:59.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.112+0000 7fb4f0000640 1 --2- 172.21.15.86:0/2803728591 >> v2:172.21.15.155:3300/0 conn(0x7fb4e80b0860 0x7fb4e80d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:59.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.112+0000 7fb4f0000640 1 --2- 172.21.15.86:0/2803728591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e81642a0 0x7fb4e80d16e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:59.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.112+0000 7fb4f0000640 1 -- 172.21.15.86:0/2803728591 >> 172.21.15.86:0/2803728591 conn(0x7fb4e80b4650 msgr2=0x7fb4e80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:59.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.112+0000 7fb4f0000640 1 -- 172.21.15.86:0/2803728591 shutdown_connections 2024-04-25T14:05:59.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.112+0000 7fb4f0000640 1 -- 172.21.15.86:0/2803728591 wait complete. 2024-04-25T14:05:59.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.116+0000 7fb4f0000640 1 Processor -- start 2024-04-25T14:05:59.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.116+0000 7fb4f0000640 1 -- start start 2024-04-25T14:05:59.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.116+0000 7fb4f0000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4e80b0860 0x7fb4e8099510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:59.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.116+0000 7fb4f0000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e80d1c20 0x7fb4e8157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:59.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.116+0000 7fb4f0000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4e81642a0 0x7fb4e8157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:59.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.116+0000 7fb4f0000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e8164830 con 0x7fb4e80d1c20 2024-04-25T14:05:59.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.116+0000 7fb4f0000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e81646b0 con 0x7fb4e81642a0 2024-04-25T14:05:59.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.116+0000 7fb4f0000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb4e8160da0 con 0x7fb4e80b0860 2024-04-25T14:05:59.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.116+0000 7fb4ecc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e80d1c20 0x7fb4e8157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:59.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.116+0000 7fb4ecc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e80d1c20 0x7fb4e8157700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41414/0 (socket says 172.21.15.86:41414) 2024-04-25T14:05:59.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.116+0000 7fb4ecc00640 1 -- 172.21.15.86:0/1909274321 learned_addr learned my addr 172.21.15.86:0/1909274321 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:59.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.116+0000 7fb4ed600640 1 --2- 172.21.15.86:0/1909274321 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4e80b0860 0x7fb4e8099510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:59.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.116+0000 7fb4ee000640 1 --2- 172.21.15.86:0/1909274321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4e81642a0 0x7fb4e8157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:59.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.116+0000 7fb4ecc00640 1 -- 172.21.15.86:0/1909274321 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4e80b0860 msgr2=0x7fb4e8099510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:59.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.116+0000 7fb4ecc00640 1 --2- 172.21.15.86:0/1909274321 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4e80b0860 0x7fb4e8099510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:59.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.116+0000 7fb4ecc00640 1 -- 172.21.15.86:0/1909274321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4e81642a0 msgr2=0x7fb4e8157c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:59.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.116+0000 7fb4ecc00640 1 --2- 172.21.15.86:0/1909274321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4e81642a0 0x7fb4e8157c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:59.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.116+0000 7fb4ecc00640 1 -- 172.21.15.86:0/1909274321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e8154e40 con 0x7fb4e80d1c20 2024-04-25T14:05:59.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.116+0000 7fb4ecc00640 1 --2- 172.21.15.86:0/1909274321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e80d1c20 0x7fb4e8157700 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400c370 tx=0x7fb4d400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:59.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.116+0000 7fb4e5600640 1 -- 172.21.15.86:0/1909274321 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d40052d0 con 0x7fb4e80d1c20 2024-04-25T14:05:59.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.116+0000 7fb4e5600640 1 -- 172.21.15.86:0/1909274321 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4d4005470 con 0x7fb4e80d1c20 2024-04-25T14:05:59.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.116+0000 7fb4e5600640 1 -- 172.21.15.86:0/1909274321 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4007260 con 0x7fb4e80d1c20 2024-04-25T14:05:59.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.116+0000 7fb4f0000640 1 -- 172.21.15.86:0/1909274321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4e8155120 con 0x7fb4e80d1c20 2024-04-25T14:05:59.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.116+0000 7fb4f0000640 1 -- 172.21.15.86:0/1909274321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4e8155400 con 0x7fb4e80d1c20 2024-04-25T14:05:59.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.120+0000 7fb4f0000640 1 -- 172.21.15.86:0/1909274321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4e809f4b0 con 0x7fb4e80d1c20 2024-04-25T14:05:59.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.120+0000 7fb4e5600640 1 -- 172.21.15.86:0/1909274321 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7fb4d4007400 con 0x7fb4e80d1c20 2024-04-25T14:05:59.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.120+0000 7fb4e5600640 1 --2- 172.21.15.86:0/1909274321 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb4c806f570 0x7fb4c8071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:59.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.120+0000 7fb4e5600640 1 -- 172.21.15.86:0/1909274321 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 7359+0+0 (secure 0 0 0) 0x7fb4d408ede0 con 0x7fb4e80d1c20 2024-04-25T14:05:59.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.120+0000 7fb4ed600640 1 --2- 172.21.15.86:0/1909274321 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb4c806f570 0x7fb4c8071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:59.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.120+0000 7fb4ed600640 1 --2- 172.21.15.86:0/1909274321 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb4c806f570 0x7fb4c8071a80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc0ad000 tx=0x7fb4dc008040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:59.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.124+0000 7fb4e5600640 1 -- 172.21.15.86:0/1909274321 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb4e80d2030 con 0x7fb4e80d1c20 2024-04-25T14:05:59.201 INFO:tasks.ceph.mgr.y.smithi086.stderr:Exception in thread Thread-16: 2024-04-25T14:05:59.201 INFO:tasks.ceph.mgr.y.smithi086.stderr:Traceback (most recent call last): 2024-04-25T14:05:59.201 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner 2024-04-25T14:05:59.201 INFO:tasks.ceph.mgr.y.smithi086.stderr: self.run() 2024-04-25T14:05:59.201 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/lib/python3.10/threading.py", line 1378, in run 2024-04-25T14:05:59.202 INFO:tasks.ceph.mgr.y.smithi086.stderr: self.function(*self.args, **self.kwargs) 2024-04-25T14:05:59.202 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/share/ceph/mgr/stats/fs/perf_stats.py", line 222, in re_register_queries 2024-04-25T14:05:59.202 INFO:tasks.ceph.mgr.y.smithi086.stderr: if self.mx_last_updated >= ua_last_updated: 2024-04-25T14:05:59.202 INFO:tasks.ceph.mgr.y.smithi086.stderr:AttributeError: 'FSPerfStats' object has no attribute 'mx_last_updated' 2024-04-25T14:05:59.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.280+0000 7fb4f0000640 1 -- 172.21.15.86:0/1909274321 --> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb4e8156f80 con 0x7fb4c806f570 2024-04-25T14:05:59.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.292+0000 7fb4e5600640 1 -- 172.21.15.86:0/1909274321 <== mgr.5532 v2:172.21.15.86:6832/2427179681 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+482359 (secure 0 0 0) 0x7fb4e8156f80 con 0x7fb4c806f570 2024-04-25T14:05:59.301 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:59.305 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:05:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.300+0000 7fb4f0000640 1 -- 172.21.15.86:0/1909274321 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb4c806f570 msgr2=0x7fb4c8071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.300+0000 7fb4f0000640 1 --2- 172.21.15.86:0/1909274321 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb4c806f570 0x7fb4c8071a80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc0ad000 tx=0x7fb4dc008040 comp rx=0 tx=0).stop 2024-04-25T14:05:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.300+0000 7fb4f0000640 1 -- 172.21.15.86:0/1909274321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e80d1c20 msgr2=0x7fb4e8157700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.300+0000 7fb4f0000640 1 --2- 172.21.15.86:0/1909274321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e80d1c20 0x7fb4e8157700 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400c370 tx=0x7fb4d400c830 comp rx=0 tx=0).stop 2024-04-25T14:05:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.300+0000 7fb4f0000640 1 -- 172.21.15.86:0/1909274321 shutdown_connections 2024-04-25T14:05:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.300+0000 7fb4f0000640 1 --2- 172.21.15.86:0/1909274321 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb4c806f570 0x7fb4c8071a80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.300+0000 7fb4f0000640 1 --2- 172.21.15.86:0/1909274321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4e81642a0 0x7fb4e8157c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.300+0000 7fb4f0000640 1 --2- 172.21.15.86:0/1909274321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4e80d1c20 0x7fb4e8157700 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:59.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.300+0000 7fb4f0000640 1 --2- 172.21.15.86:0/1909274321 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4e80b0860 0x7fb4e8099510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:59.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.300+0000 7fb4f0000640 1 -- 172.21.15.86:0/1909274321 >> 172.21.15.86:0/1909274321 conn(0x7fb4e80b4650 msgr2=0x7fb4e8162bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:59.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.300+0000 7fb4f0000640 1 -- 172.21.15.86:0/1909274321 shutdown_connections 2024-04-25T14:05:59.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.300+0000 7fb4f0000640 1 -- 172.21.15.86:0/1909274321 wait complete. 2024-04-25T14:05:59.321 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-04-25T14:05:57.486872+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3749962,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":205,"num_read_kb":313,"num_write":177,"num_write_kb":4730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":276,"acting":276,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":264,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":228388,"kb_used_data":12216,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754746332,"statfs":{"total":773094113280,"available":772860243968,"internally_reserved":0,"allocated":12509184,"data_stored":9788735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5089,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.335032"},"pg_stats":[{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.186876+0000","last_change":"2024-04-25T14:05:55.210641+0000","last_active":"2024-04-25T14:05:56.186876+0000","last_peered":"2024-04-25T14:05:56.186876+0000","last_clean":"2024-04-25T14:05:56.186876+0000","last_became_active":"2024-04-25T14:05:55.209998+0000","last_became_peered":"2024-04-25T14:05:55.209998+0000","last_unstale":"2024-04-25T14:05:56.186876+0000","last_undegraded":"2024-04-25T14:05:56.186876+0000","last_fullsized":"2024-04-25T14:05:56.186876+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:44.412223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"39'79","reported_seq":190,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.906197+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:05:42.906197+0000","last_peered":"2024-04-25T14:05:42.906197+0000","last_clean":"2024-04-25T14:05:42.906197+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:05:42.906197+0000","last_undegraded":"2024-04-25T14:05:42.906197+0000","last_fullsized":"2024-04-25T14:05:42.906197+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"50'4","reported_seq":33,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.832585+0000","last_change":"2024-04-25T14:05:32.559617+0000","last_active":"2024-04-25T14:05:56.832585+0000","last_peered":"2024-04-25T14:05:56.832585+0000","last_clean":"2024-04-25T14:05:56.832585+0000","last_became_active":"2024-04-25T14:05:32.558277+0000","last_became_peered":"2024-04-25T14:05:32.558277+0000","last_unstale":"2024-04-25T14:05:56.832585+0000","last_undegraded":"2024-04-25T14:05:56.832585+0000","last_fullsized":"2024-04-25T14:05:56.832585+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:38.715145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":25,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.185922+0000","last_change":"2024-04-25T14:05:33.563768+0000","last_active":"2024-04-25T14:05:56.185922+0000","last_peered":"2024-04-25T14:05:56.185922+0000","last_clean":"2024-04-25T14:05:56.185922+0000","last_became_active":"2024-04-25T14:05:33.563379+0000","last_became_peered":"2024-04-25T14:05:33.563379+0000","last_unstale":"2024-04-25T14:05:56.185922+0000","last_undegraded":"2024-04-25T14:05:56.185922+0000","last_fullsized":"2024-04-25T14:05:56.185922+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:11.087435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180424+0000","last_change":"2024-04-25T14:05:54.189656+0000","last_active":"2024-04-25T14:05:56.180424+0000","last_peered":"2024-04-25T14:05:56.180424+0000","last_clean":"2024-04-25T14:05:56.180424+0000","last_became_active":"2024-04-25T14:05:54.188737+0000","last_became_peered":"2024-04-25T14:05:54.188737+0000","last_unstale":"2024-04-25T14:05:56.180424+0000","last_undegraded":"2024-04-25T14:05:56.180424+0000","last_fullsized":"2024-04-25T14:05:56.180424+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:54.607236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.186225+0000","last_change":"2024-04-25T14:05:55.205021+0000","last_active":"2024-04-25T14:05:56.186225+0000","last_peered":"2024-04-25T14:05:56.186225+0000","last_clean":"2024-04-25T14:05:56.186225+0000","last_became_active":"2024-04-25T14:05:55.204257+0000","last_became_peered":"2024-04-25T14:05:55.204257+0000","last_unstale":"2024-04-25T14:05:56.186225+0000","last_undegraded":"2024-04-25T14:05:56.186225+0000","last_fullsized":"2024-04-25T14:05:56.186225+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:29.540170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":21,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.888539+0000","last_change":"2024-04-25T14:05:32.564178+0000","last_active":"2024-04-25T14:05:42.888539+0000","last_peered":"2024-04-25T14:05:42.888539+0000","last_clean":"2024-04-25T14:05:42.888539+0000","last_became_active":"2024-04-25T14:05:32.563861+0000","last_became_peered":"2024-04-25T14:05:32.563861+0000","last_unstale":"2024-04-25T14:05:42.888539+0000","last_undegraded":"2024-04-25T14:05:42.888539+0000","last_fullsized":"2024-04-25T14:05:42.888539+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:40.505936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.181631+0000","last_change":"2024-04-25T14:05:33.575403+0000","last_active":"2024-04-25T14:05:56.181631+0000","last_peered":"2024-04-25T14:05:56.181631+0000","last_clean":"2024-04-25T14:05:56.181631+0000","last_became_active":"2024-04-25T14:05:33.575194+0000","last_became_peered":"2024-04-25T14:05:33.575194+0000","last_unstale":"2024-04-25T14:05:56.181631+0000","last_undegraded":"2024-04-25T14:05:56.181631+0000","last_fullsized":"2024-04-25T14:05:56.181631+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:44.696081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:53.144076+0000","last_change":"2024-04-25T14:05:53.144076+0000","last_active":"2024-04-25T14:05:53.144076+0000","last_peered":"2024-04-25T14:05:53.144076+0000","last_clean":"2024-04-25T14:05:53.144076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:53.144076+0000","last_undegraded":"2024-04-25T14:05:53.144076+0000","last_fullsized":"2024-04-25T14:05:53.144076+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":19,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.911512+0000","last_change":"2024-04-25T14:05:32.543546+0000","last_active":"2024-04-25T14:05:42.911512+0000","last_peered":"2024-04-25T14:05:42.911512+0000","last_clean":"2024-04-25T14:05:42.911512+0000","last_became_active":"2024-04-25T14:05:32.543162+0000","last_became_peered":"2024-04-25T14:05:32.543162+0000","last_unstale":"2024-04-25T14:05:42.911512+0000","last_undegraded":"2024-04-25T14:05:42.911512+0000","last_fullsized":"2024-04-25T14:05:42.911512+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:14.012822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833857+0000","last_change":"2024-04-25T14:05:33.573521+0000","last_active":"2024-04-25T14:05:56.833857+0000","last_peered":"2024-04-25T14:05:56.833857+0000","last_clean":"2024-04-25T14:05:56.833857+0000","last_became_active":"2024-04-25T14:05:33.573181+0000","last_became_peered":"2024-04-25T14:05:33.573181+0000","last_unstale":"2024-04-25T14:05:56.833857+0000","last_undegraded":"2024-04-25T14:05:56.833857+0000","last_fullsized":"2024-04-25T14:05:56.833857+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:33.035476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:53.144076+0000","last_change":"2024-04-25T14:05:53.144076+0000","last_active":"2024-04-25T14:05:53.144076+0000","last_peered":"2024-04-25T14:05:53.144076+0000","last_clean":"2024-04-25T14:05:53.144076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:53.144076+0000","last_undegraded":"2024-04-25T14:05:53.144076+0000","last_fullsized":"2024-04-25T14:05:53.144076+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.187009+0000","last_change":"2024-04-25T14:05:55.211098+0000","last_active":"2024-04-25T14:05:56.187009+0000","last_peered":"2024-04-25T14:05:56.187009+0000","last_clean":"2024-04-25T14:05:56.187009+0000","last_became_active":"2024-04-25T14:05:55.210732+0000","last_became_peered":"2024-04-25T14:05:55.210732+0000","last_unstale":"2024-04-25T14:05:56.187009+0000","last_undegraded":"2024-04-25T14:05:56.187009+0000","last_fullsized":"2024-04-25T14:05:56.187009+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:50.046862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":29,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180900+0000","last_change":"2024-04-25T14:05:32.555662+0000","last_active":"2024-04-25T14:05:56.180900+0000","last_peered":"2024-04-25T14:05:56.180900+0000","last_clean":"2024-04-25T14:05:56.180900+0000","last_became_active":"2024-04-25T14:05:32.555123+0000","last_became_peered":"2024-04-25T14:05:32.555123+0000","last_unstale":"2024-04-25T14:05:56.180900+0000","last_undegraded":"2024-04-25T14:05:56.180900+0000","last_fullsized":"2024-04-25T14:05:56.180900+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:24.071915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":19,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.888631+0000","last_change":"2024-04-25T14:05:33.570070+0000","last_active":"2024-04-25T14:05:42.888631+0000","last_peered":"2024-04-25T14:05:42.888631+0000","last_clean":"2024-04-25T14:05:42.888631+0000","last_became_active":"2024-04-25T14:05:33.569716+0000","last_became_peered":"2024-04-25T14:05:33.569716+0000","last_unstale":"2024-04-25T14:05:42.888631+0000","last_undegraded":"2024-04-25T14:05:42.888631+0000","last_fullsized":"2024-04-25T14:05:42.888631+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:42.654848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.181210+0000","last_change":"2024-04-25T14:05:54.181106+0000","last_active":"2024-04-25T14:05:56.181210+0000","last_peered":"2024-04-25T14:05:56.181210+0000","last_clean":"2024-04-25T14:05:56.181210+0000","last_became_active":"2024-04-25T14:05:54.180829+0000","last_became_peered":"2024-04-25T14:05:54.180829+0000","last_unstale":"2024-04-25T14:05:56.181210+0000","last_undegraded":"2024-04-25T14:05:56.181210+0000","last_fullsized":"2024-04-25T14:05:56.181210+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:25.927992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":29,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.181185+0000","last_change":"2024-04-25T14:05:32.555975+0000","last_active":"2024-04-25T14:05:56.181185+0000","last_peered":"2024-04-25T14:05:56.181185+0000","last_clean":"2024-04-25T14:05:56.181185+0000","last_became_active":"2024-04-25T14:05:32.555188+0000","last_became_peered":"2024-04-25T14:05:32.555188+0000","last_unstale":"2024-04-25T14:05:56.181185+0000","last_undegraded":"2024-04-25T14:05:56.181185+0000","last_fullsized":"2024-04-25T14:05:56.181185+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:10.274276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833504+0000","last_change":"2024-04-25T14:05:33.561273+0000","last_active":"2024-04-25T14:05:56.833504+0000","last_peered":"2024-04-25T14:05:56.833504+0000","last_clean":"2024-04-25T14:05:56.833504+0000","last_became_active":"2024-04-25T14:05:33.560969+0000","last_became_peered":"2024-04-25T14:05:33.560969+0000","last_unstale":"2024-04-25T14:05:56.833504+0000","last_undegraded":"2024-04-25T14:05:56.833504+0000","last_fullsized":"2024-04-25T14:05:56.833504+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:44.696081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:53.144076+0000","last_change":"2024-04-25T14:05:53.144076+0000","last_active":"2024-04-25T14:05:53.144076+0000","last_peered":"2024-04-25T14:05:53.144076+0000","last_clean":"2024-04-25T14:05:53.144076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:53.144076+0000","last_undegraded":"2024-04-25T14:05:53.144076+0000","last_fullsized":"2024-04-25T14:05:53.144076+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":28,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833507+0000","last_change":"2024-04-25T14:05:32.558770+0000","last_active":"2024-04-25T14:05:56.833507+0000","last_peered":"2024-04-25T14:05:56.833507+0000","last_clean":"2024-04-25T14:05:56.833507+0000","last_became_active":"2024-04-25T14:05:32.557820+0000","last_became_peered":"2024-04-25T14:05:32.557820+0000","last_unstale":"2024-04-25T14:05:56.833507+0000","last_undegraded":"2024-04-25T14:05:56.833507+0000","last_fullsized":"2024-04-25T14:05:56.833507+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:10.274276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.181395+0000","last_change":"2024-04-25T14:05:33.573700+0000","last_active":"2024-04-25T14:05:56.181395+0000","last_peered":"2024-04-25T14:05:56.181395+0000","last_clean":"2024-04-25T14:05:56.181395+0000","last_became_active":"2024-04-25T14:05:33.573414+0000","last_became_peered":"2024-04-25T14:05:33.573414+0000","last_unstale":"2024-04-25T14:05:56.181395+0000","last_undegraded":"2024-04-25T14:05:56.181395+0000","last_fullsized":"2024-04-25T14:05:56.181395+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:30.202222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:53.144076+0000","last_change":"2024-04-25T14:05:53.144076+0000","last_active":"2024-04-25T14:05:53.144076+0000","last_peered":"2024-04-25T14:05:53.144076+0000","last_clean":"2024-04-25T14:05:53.144076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:53.144076+0000","last_undegraded":"2024-04-25T14:05:53.144076+0000","last_fullsized":"2024-04-25T14:05:53.144076+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833001+0000","last_change":"2024-04-25T14:05:55.208062+0000","last_active":"2024-04-25T14:05:56.833001+0000","last_peered":"2024-04-25T14:05:56.833001+0000","last_clean":"2024-04-25T14:05:56.833001+0000","last_became_active":"2024-04-25T14:05:55.206431+0000","last_became_peered":"2024-04-25T14:05:55.206431+0000","last_unstale":"2024-04-25T14:05:56.833001+0000","last_undegraded":"2024-04-25T14:05:56.833001+0000","last_fullsized":"2024-04-25T14:05:56.833001+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:38.575335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":20,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.906721+0000","last_change":"2024-04-25T14:05:32.562230+0000","last_active":"2024-04-25T14:05:42.906721+0000","last_peered":"2024-04-25T14:05:42.906721+0000","last_clean":"2024-04-25T14:05:42.906721+0000","last_became_active":"2024-04-25T14:05:32.561768+0000","last_became_peered":"2024-04-25T14:05:32.561768+0000","last_unstale":"2024-04-25T14:05:42.906721+0000","last_undegraded":"2024-04-25T14:05:42.906721+0000","last_fullsized":"2024-04-25T14:05:42.906721+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:40.505936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":19,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.905021+0000","last_change":"2024-04-25T14:05:33.574199+0000","last_active":"2024-04-25T14:05:42.905021+0000","last_peered":"2024-04-25T14:05:42.905021+0000","last_clean":"2024-04-25T14:05:42.905021+0000","last_became_active":"2024-04-25T14:05:33.572436+0000","last_became_peered":"2024-04-25T14:05:33.572436+0000","last_unstale":"2024-04-25T14:05:42.905021+0000","last_undegraded":"2024-04-25T14:05:42.905021+0000","last_fullsized":"2024-04-25T14:05:42.905021+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:59:41.873995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833230+0000","last_change":"2024-04-25T14:05:54.190851+0000","last_active":"2024-04-25T14:05:56.833230+0000","last_peered":"2024-04-25T14:05:56.833230+0000","last_clean":"2024-04-25T14:05:56.833230+0000","last_became_active":"2024-04-25T14:05:54.190478+0000","last_became_peered":"2024-04-25T14:05:54.190478+0000","last_unstale":"2024-04-25T14:05:56.833230+0000","last_undegraded":"2024-04-25T14:05:56.833230+0000","last_fullsized":"2024-04-25T14:05:56.833230+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:52.895016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.186745+0000","last_change":"2024-04-25T14:05:55.211644+0000","last_active":"2024-04-25T14:05:56.186745+0000","last_peered":"2024-04-25T14:05:56.186745+0000","last_clean":"2024-04-25T14:05:56.186745+0000","last_became_active":"2024-04-25T14:05:55.210906+0000","last_became_peered":"2024-04-25T14:05:55.210906+0000","last_unstale":"2024-04-25T14:05:56.186745+0000","last_undegraded":"2024-04-25T14:05:56.186745+0000","last_fullsized":"2024-04-25T14:05:56.186745+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:54.658020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.7","version":"45'2","reported_seq":31,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.187436+0000","last_change":"2024-04-25T14:05:32.561754+0000","last_active":"2024-04-25T14:05:56.187436+0000","last_peered":"2024-04-25T14:05:56.187436+0000","last_clean":"2024-04-25T14:05:56.187436+0000","last_became_active":"2024-04-25T14:05:32.561297+0000","last_became_peered":"2024-04-25T14:05:32.561297+0000","last_unstale":"2024-04-25T14:05:56.187436+0000","last_undegraded":"2024-04-25T14:05:56.187436+0000","last_fullsized":"2024-04-25T14:05:56.187436+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:39.285483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":19,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.888436+0000","last_change":"2024-04-25T14:05:33.561759+0000","last_active":"2024-04-25T14:05:42.888436+0000","last_peered":"2024-04-25T14:05:42.888436+0000","last_clean":"2024-04-25T14:05:42.888436+0000","last_became_active":"2024-04-25T14:05:33.561430+0000","last_became_peered":"2024-04-25T14:05:33.561430+0000","last_unstale":"2024-04-25T14:05:42.888436+0000","last_undegraded":"2024-04-25T14:05:42.888436+0000","last_fullsized":"2024-04-25T14:05:42.888436+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:16.126866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.832766+0000","last_change":"2024-04-25T14:05:54.184027+0000","last_active":"2024-04-25T14:05:56.832766+0000","last_peered":"2024-04-25T14:05:56.832766+0000","last_clean":"2024-04-25T14:05:56.832766+0000","last_became_active":"2024-04-25T14:05:54.183683+0000","last_became_peered":"2024-04-25T14:05:54.183683+0000","last_unstale":"2024-04-25T14:05:56.832766+0000","last_undegraded":"2024-04-25T14:05:56.832766+0000","last_fullsized":"2024-04-25T14:05:56.832766+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:25.927992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:57.055590+0000","last_change":"2024-04-25T14:05:55.199570+0000","last_active":"2024-04-25T14:05:57.055590+0000","last_peered":"2024-04-25T14:05:57.055590+0000","last_clean":"2024-04-25T14:05:57.055590+0000","last_became_active":"2024-04-25T14:05:55.199153+0000","last_became_peered":"2024-04-25T14:05:55.199153+0000","last_unstale":"2024-04-25T14:05:57.055590+0000","last_undegraded":"2024-04-25T14:05:57.055590+0000","last_fullsized":"2024-04-25T14:05:57.055590+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:49.541001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":28,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:57.054422+0000","last_change":"2024-04-25T14:05:32.553201+0000","last_active":"2024-04-25T14:05:57.054422+0000","last_peered":"2024-04-25T14:05:57.054422+0000","last_clean":"2024-04-25T14:05:57.054422+0000","last_became_active":"2024-04-25T14:05:32.552846+0000","last_became_peered":"2024-04-25T14:05:32.552846+0000","last_unstale":"2024-04-25T14:05:57.054422+0000","last_undegraded":"2024-04-25T14:05:57.054422+0000","last_fullsized":"2024-04-25T14:05:57.054422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:13.772830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"50'1","reported_seq":20,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.905470+0000","last_change":"2024-04-25T14:05:33.579278+0000","last_active":"2024-04-25T14:05:42.905470+0000","last_peered":"2024-04-25T14:05:42.905470+0000","last_clean":"2024-04-25T14:05:42.905470+0000","last_became_active":"2024-04-25T14:05:33.578876+0000","last_became_peered":"2024-04-25T14:05:33.578876+0000","last_unstale":"2024-04-25T14:05:42.905470+0000","last_undegraded":"2024-04-25T14:05:42.905470+0000","last_fullsized":"2024-04-25T14:05:42.905470+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:54.564632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:53.144076+0000","last_change":"2024-04-25T14:05:53.144076+0000","last_active":"2024-04-25T14:05:53.144076+0000","last_peered":"2024-04-25T14:05:53.144076+0000","last_clean":"2024-04-25T14:05:53.144076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:53.144076+0000","last_undegraded":"2024-04-25T14:05:53.144076+0000","last_fullsized":"2024-04-25T14:05:53.144076+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833222+0000","last_change":"2024-04-25T14:05:55.205672+0000","last_active":"2024-04-25T14:05:56.833222+0000","last_peered":"2024-04-25T14:05:56.833222+0000","last_clean":"2024-04-25T14:05:56.833222+0000","last_became_active":"2024-04-25T14:05:55.205254+0000","last_became_peered":"2024-04-25T14:05:55.205254+0000","last_unstale":"2024-04-25T14:05:56.833222+0000","last_undegraded":"2024-04-25T14:05:56.833222+0000","last_fullsized":"2024-04-25T14:05:56.833222+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:37.389511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":28,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.832556+0000","last_change":"2024-04-25T14:05:32.563774+0000","last_active":"2024-04-25T14:05:56.832556+0000","last_peered":"2024-04-25T14:05:56.832556+0000","last_clean":"2024-04-25T14:05:56.832556+0000","last_became_active":"2024-04-25T14:05:32.562877+0000","last_became_peered":"2024-04-25T14:05:32.562877+0000","last_unstale":"2024-04-25T14:05:56.832556+0000","last_undegraded":"2024-04-25T14:05:56.832556+0000","last_fullsized":"2024-04-25T14:05:56.832556+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:24.071915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833925+0000","last_change":"2024-04-25T14:05:33.579664+0000","last_active":"2024-04-25T14:05:56.833925+0000","last_peered":"2024-04-25T14:05:56.833925+0000","last_clean":"2024-04-25T14:05:56.833925+0000","last_became_active":"2024-04-25T14:05:33.578924+0000","last_became_peered":"2024-04-25T14:05:33.578924+0000","last_unstale":"2024-04-25T14:05:56.833925+0000","last_undegraded":"2024-04-25T14:05:56.833925+0000","last_fullsized":"2024-04-25T14:05:56.833925+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:13.000228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:53.144076+0000","last_change":"2024-04-25T14:05:53.144076+0000","last_active":"2024-04-25T14:05:53.144076+0000","last_peered":"2024-04-25T14:05:53.144076+0000","last_clean":"2024-04-25T14:05:53.144076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:53.144076+0000","last_undegraded":"2024-04-25T14:05:53.144076+0000","last_fullsized":"2024-04-25T14:05:53.144076+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.187348+0000","last_change":"2024-04-25T14:05:55.209087+0000","last_active":"2024-04-25T14:05:56.187348+0000","last_peered":"2024-04-25T14:05:56.187348+0000","last_clean":"2024-04-25T14:05:56.187348+0000","last_became_active":"2024-04-25T14:05:55.208771+0000","last_became_peered":"2024-04-25T14:05:55.208771+0000","last_unstale":"2024-04-25T14:05:56.187348+0000","last_undegraded":"2024-04-25T14:05:56.187348+0000","last_fullsized":"2024-04-25T14:05:56.187348+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:13.373855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":29,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180786+0000","last_change":"2024-04-25T14:05:32.555416+0000","last_active":"2024-04-25T14:05:56.180786+0000","last_peered":"2024-04-25T14:05:56.180786+0000","last_clean":"2024-04-25T14:05:56.180786+0000","last_became_active":"2024-04-25T14:05:32.554652+0000","last_became_peered":"2024-04-25T14:05:32.554652+0000","last_unstale":"2024-04-25T14:05:56.180786+0000","last_undegraded":"2024-04-25T14:05:56.180786+0000","last_fullsized":"2024-04-25T14:05:56.180786+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:40.505936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":25,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.187288+0000","last_change":"2024-04-25T14:05:33.570587+0000","last_active":"2024-04-25T14:05:56.187288+0000","last_peered":"2024-04-25T14:05:56.187288+0000","last_clean":"2024-04-25T14:05:56.187288+0000","last_became_active":"2024-04-25T14:05:33.570209+0000","last_became_peered":"2024-04-25T14:05:33.570209+0000","last_unstale":"2024-04-25T14:05:56.187288+0000","last_undegraded":"2024-04-25T14:05:56.187288+0000","last_fullsized":"2024-04-25T14:05:56.187288+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:13.979622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:53.144076+0000","last_change":"2024-04-25T14:05:53.144076+0000","last_active":"2024-04-25T14:05:53.144076+0000","last_peered":"2024-04-25T14:05:53.144076+0000","last_clean":"2024-04-25T14:05:53.144076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:53.144076+0000","last_undegraded":"2024-04-25T14:05:53.144076+0000","last_fullsized":"2024-04-25T14:05:53.144076+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"50'3","reported_seq":25,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.905566+0000","last_change":"2024-04-25T14:05:32.555817+0000","last_active":"2024-04-25T14:05:42.905566+0000","last_peered":"2024-04-25T14:05:42.905566+0000","last_clean":"2024-04-25T14:05:42.905566+0000","last_became_active":"2024-04-25T14:05:32.553958+0000","last_became_peered":"2024-04-25T14:05:32.553958+0000","last_unstale":"2024-04-25T14:05:42.905566+0000","last_undegraded":"2024-04-25T14:05:42.905566+0000","last_fullsized":"2024-04-25T14:05:42.905566+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:43.267251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":19,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.888781+0000","last_change":"2024-04-25T14:05:33.571493+0000","last_active":"2024-04-25T14:05:42.888781+0000","last_peered":"2024-04-25T14:05:42.888781+0000","last_clean":"2024-04-25T14:05:42.888781+0000","last_became_active":"2024-04-25T14:05:33.568719+0000","last_became_peered":"2024-04-25T14:05:33.568719+0000","last_unstale":"2024-04-25T14:05:42.888781+0000","last_undegraded":"2024-04-25T14:05:42.888781+0000","last_fullsized":"2024-04-25T14:05:42.888781+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:18.567327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833569+0000","last_change":"2024-04-25T14:05:54.189553+0000","last_active":"2024-04-25T14:05:56.833569+0000","last_peered":"2024-04-25T14:05:56.833569+0000","last_clean":"2024-04-25T14:05:56.833569+0000","last_became_active":"2024-04-25T14:05:54.182191+0000","last_became_peered":"2024-04-25T14:05:54.182191+0000","last_unstale":"2024-04-25T14:05:56.833569+0000","last_undegraded":"2024-04-25T14:05:56.833569+0000","last_fullsized":"2024-04-25T14:05:56.833569+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:12.135098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.187031+0000","last_change":"2024-04-25T14:05:32.567474+0000","last_active":"2024-04-25T14:05:56.187031+0000","last_peered":"2024-04-25T14:05:56.187031+0000","last_clean":"2024-04-25T14:05:56.187031+0000","last_became_active":"2024-04-25T14:05:32.567061+0000","last_became_peered":"2024-04-25T14:05:32.567061+0000","last_unstale":"2024-04-25T14:05:56.187031+0000","last_undegraded":"2024-04-25T14:05:56.187031+0000","last_fullsized":"2024-04-25T14:05:56.187031+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:20.281168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":19,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.905101+0000","last_change":"2024-04-25T14:05:33.572207+0000","last_active":"2024-04-25T14:05:42.905101+0000","last_peered":"2024-04-25T14:05:42.905101+0000","last_clean":"2024-04-25T14:05:42.905101+0000","last_became_active":"2024-04-25T14:05:33.570800+0000","last_became_peered":"2024-04-25T14:05:33.570800+0000","last_unstale":"2024-04-25T14:05:42.905101+0000","last_undegraded":"2024-04-25T14:05:42.905101+0000","last_fullsized":"2024-04-25T14:05:42.905101+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:25.659996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180722+0000","last_change":"2024-04-25T14:05:54.173170+0000","last_active":"2024-04-25T14:05:56.180722+0000","last_peered":"2024-04-25T14:05:56.180722+0000","last_clean":"2024-04-25T14:05:56.180722+0000","last_became_active":"2024-04-25T14:05:54.172825+0000","last_became_peered":"2024-04-25T14:05:54.172825+0000","last_unstale":"2024-04-25T14:05:56.180722+0000","last_undegraded":"2024-04-25T14:05:56.180722+0000","last_fullsized":"2024-04-25T14:05:56.180722+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:11.757925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.185842+0000","last_change":"2024-04-25T14:05:55.210220+0000","last_active":"2024-04-25T14:05:56.185842+0000","last_peered":"2024-04-25T14:05:56.185842+0000","last_clean":"2024-04-25T14:05:56.185842+0000","last_became_active":"2024-04-25T14:05:55.209839+0000","last_became_peered":"2024-04-25T14:05:55.209839+0000","last_unstale":"2024-04-25T14:05:56.185842+0000","last_undegraded":"2024-04-25T14:05:56.185842+0000","last_fullsized":"2024-04-25T14:05:56.185842+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:09.440953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.d","version":"45'1","reported_seq":29,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.187454+0000","last_change":"2024-04-25T14:05:32.566219+0000","last_active":"2024-04-25T14:05:56.187454+0000","last_peered":"2024-04-25T14:05:56.187454+0000","last_clean":"2024-04-25T14:05:56.187454+0000","last_became_active":"2024-04-25T14:05:32.565859+0000","last_became_peered":"2024-04-25T14:05:32.565859+0000","last_unstale":"2024-04-25T14:05:56.187454+0000","last_undegraded":"2024-04-25T14:05:56.187454+0000","last_fullsized":"2024-04-25T14:05:56.187454+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:35.649467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.911459+0000","last_change":"2024-04-25T14:05:33.582498+0000","last_active":"2024-04-25T14:05:42.911459+0000","last_peered":"2024-04-25T14:05:42.911459+0000","last_clean":"2024-04-25T14:05:42.911459+0000","last_became_active":"2024-04-25T14:05:33.582156+0000","last_became_peered":"2024-04-25T14:05:33.582156+0000","last_unstale":"2024-04-25T14:05:42.911459+0000","last_undegraded":"2024-04-25T14:05:42.911459+0000","last_fullsized":"2024-04-25T14:05:42.911459+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:13.979622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:53.144076+0000","last_change":"2024-04-25T14:05:53.144076+0000","last_active":"2024-04-25T14:05:53.144076+0000","last_peered":"2024-04-25T14:05:53.144076+0000","last_clean":"2024-04-25T14:05:53.144076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:53.144076+0000","last_undegraded":"2024-04-25T14:05:53.144076+0000","last_fullsized":"2024-04-25T14:05:53.144076+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:57.055206+0000","last_change":"2024-04-25T14:05:55.192954+0000","last_active":"2024-04-25T14:05:57.055206+0000","last_peered":"2024-04-25T14:05:57.055206+0000","last_clean":"2024-04-25T14:05:57.055206+0000","last_became_active":"2024-04-25T14:05:55.192613+0000","last_became_peered":"2024-04-25T14:05:55.192613+0000","last_unstale":"2024-04-25T14:05:57.055206+0000","last_undegraded":"2024-04-25T14:05:57.055206+0000","last_fullsized":"2024-04-25T14:05:57.055206+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:46.700882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"45'2","reported_seq":32,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:57.054616+0000","last_change":"2024-04-25T14:05:32.559217+0000","last_active":"2024-04-25T14:05:57.054616+0000","last_peered":"2024-04-25T14:05:57.054616+0000","last_clean":"2024-04-25T14:05:57.054616+0000","last_became_active":"2024-04-25T14:05:32.558909+0000","last_became_peered":"2024-04-25T14:05:32.558909+0000","last_unstale":"2024-04-25T14:05:57.054616+0000","last_undegraded":"2024-04-25T14:05:57.054616+0000","last_fullsized":"2024-04-25T14:05:57.054616+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:48:44.213850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":19,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.904947+0000","last_change":"2024-04-25T14:05:33.590347+0000","last_active":"2024-04-25T14:05:42.904947+0000","last_peered":"2024-04-25T14:05:42.904947+0000","last_clean":"2024-04-25T14:05:42.904947+0000","last_became_active":"2024-04-25T14:05:33.590073+0000","last_became_peered":"2024-04-25T14:05:33.590073+0000","last_unstale":"2024-04-25T14:05:42.904947+0000","last_undegraded":"2024-04-25T14:05:42.904947+0000","last_fullsized":"2024-04-25T14:05:42.904947+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:11.280699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:53.144076+0000","last_change":"2024-04-25T14:05:53.144076+0000","last_active":"2024-04-25T14:05:53.144076+0000","last_peered":"2024-04-25T14:05:53.144076+0000","last_clean":"2024-04-25T14:05:53.144076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:53.144076+0000","last_undegraded":"2024-04-25T14:05:53.144076+0000","last_fullsized":"2024-04-25T14:05:53.144076+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"45'1","reported_seq":30,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.832610+0000","last_change":"2024-04-25T14:05:32.569447+0000","last_active":"2024-04-25T14:05:56.832610+0000","last_peered":"2024-04-25T14:05:56.832610+0000","last_clean":"2024-04-25T14:05:56.832610+0000","last_became_active":"2024-04-25T14:05:32.568861+0000","last_became_peered":"2024-04-25T14:05:32.568861+0000","last_unstale":"2024-04-25T14:05:56.832610+0000","last_undegraded":"2024-04-25T14:05:56.832610+0000","last_fullsized":"2024-04-25T14:05:56.832610+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:00.438414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180486+0000","last_change":"2024-04-25T14:05:33.568454+0000","last_active":"2024-04-25T14:05:56.180486+0000","last_peered":"2024-04-25T14:05:56.180486+0000","last_clean":"2024-04-25T14:05:56.180486+0000","last_became_active":"2024-04-25T14:05:33.568140+0000","last_became_peered":"2024-04-25T14:05:33.568140+0000","last_unstale":"2024-04-25T14:05:56.180486+0000","last_undegraded":"2024-04-25T14:05:56.180486+0000","last_fullsized":"2024-04-25T14:05:56.180486+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:09.011115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180568+0000","last_change":"2024-04-25T14:05:54.190640+0000","last_active":"2024-04-25T14:05:56.180568+0000","last_peered":"2024-04-25T14:05:56.180568+0000","last_clean":"2024-04-25T14:05:56.180568+0000","last_became_active":"2024-04-25T14:05:54.189911+0000","last_became_peered":"2024-04-25T14:05:54.189911+0000","last_unstale":"2024-04-25T14:05:56.180568+0000","last_undegraded":"2024-04-25T14:05:56.180568+0000","last_fullsized":"2024-04-25T14:05:56.180568+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:43:53.870402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833842+0000","last_change":"2024-04-25T14:05:54.184175+0000","last_active":"2024-04-25T14:05:56.833842+0000","last_peered":"2024-04-25T14:05:56.833842+0000","last_clean":"2024-04-25T14:05:56.833842+0000","last_became_active":"2024-04-25T14:05:54.183867+0000","last_became_peered":"2024-04-25T14:05:54.183867+0000","last_unstale":"2024-04-25T14:05:56.833842+0000","last_undegraded":"2024-04-25T14:05:56.833842+0000","last_fullsized":"2024-04-25T14:05:56.833842+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:13.060977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"45'8","reported_seq":45,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180589+0000","last_change":"2024-04-25T14:05:32.554990+0000","last_active":"2024-04-25T14:05:56.180589+0000","last_peered":"2024-04-25T14:05:56.180589+0000","last_clean":"2024-04-25T14:05:56.180589+0000","last_became_active":"2024-04-25T14:05:32.554580+0000","last_became_peered":"2024-04-25T14:05:32.554580+0000","last_unstale":"2024-04-25T14:05:56.180589+0000","last_undegraded":"2024-04-25T14:05:56.180589+0000","last_fullsized":"2024-04-25T14:05:56.180589+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:26.912034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833753+0000","last_change":"2024-04-25T14:05:33.566385+0000","last_active":"2024-04-25T14:05:56.833753+0000","last_peered":"2024-04-25T14:05:56.833753+0000","last_clean":"2024-04-25T14:05:56.833753+0000","last_became_active":"2024-04-25T14:05:33.566106+0000","last_became_peered":"2024-04-25T14:05:33.566106+0000","last_unstale":"2024-04-25T14:05:56.833753+0000","last_undegraded":"2024-04-25T14:05:56.833753+0000","last_fullsized":"2024-04-25T14:05:56.833753+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:51.751312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.186808+0000","last_change":"2024-04-25T14:05:55.210034+0000","last_active":"2024-04-25T14:05:56.186808+0000","last_peered":"2024-04-25T14:05:56.186808+0000","last_clean":"2024-04-25T14:05:56.186808+0000","last_became_active":"2024-04-25T14:05:55.209669+0000","last_became_peered":"2024-04-25T14:05:55.209669+0000","last_unstale":"2024-04-25T14:05:56.186808+0000","last_undegraded":"2024-04-25T14:05:56.186808+0000","last_fullsized":"2024-04-25T14:05:56.186808+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:34.622772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:53.144076+0000","last_change":"2024-04-25T14:05:53.144076+0000","last_active":"2024-04-25T14:05:53.144076+0000","last_peered":"2024-04-25T14:05:53.144076+0000","last_clean":"2024-04-25T14:05:53.144076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:53.144076+0000","last_undegraded":"2024-04-25T14:05:53.144076+0000","last_fullsized":"2024-04-25T14:05:53.144076+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":20,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.906573+0000","last_change":"2024-04-25T14:05:32.553901+0000","last_active":"2024-04-25T14:05:42.906573+0000","last_peered":"2024-04-25T14:05:42.906573+0000","last_clean":"2024-04-25T14:05:42.906573+0000","last_became_active":"2024-04-25T14:05:32.552885+0000","last_became_peered":"2024-04-25T14:05:32.552885+0000","last_unstale":"2024-04-25T14:05:42.906573+0000","last_undegraded":"2024-04-25T14:05:42.906573+0000","last_fullsized":"2024-04-25T14:05:42.906573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:17.068068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.181395+0000","last_change":"2024-04-25T14:05:33.571641+0000","last_active":"2024-04-25T14:05:56.181395+0000","last_peered":"2024-04-25T14:05:56.181395+0000","last_clean":"2024-04-25T14:05:56.181395+0000","last_became_active":"2024-04-25T14:05:33.571347+0000","last_became_peered":"2024-04-25T14:05:33.571347+0000","last_unstale":"2024-04-25T14:05:56.181395+0000","last_undegraded":"2024-04-25T14:05:56.181395+0000","last_fullsized":"2024-04-25T14:05:56.181395+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:06.138261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.187718+0000","last_change":"2024-04-25T14:05:32.562133+0000","last_active":"2024-04-25T14:05:56.187718+0000","last_peered":"2024-04-25T14:05:56.187718+0000","last_clean":"2024-04-25T14:05:56.187718+0000","last_became_active":"2024-04-25T14:05:32.561845+0000","last_became_peered":"2024-04-25T14:05:32.561845+0000","last_unstale":"2024-04-25T14:05:56.187718+0000","last_undegraded":"2024-04-25T14:05:56.187718+0000","last_fullsized":"2024-04-25T14:05:56.187718+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:14.012822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833776+0000","last_change":"2024-04-25T14:05:33.586628+0000","last_active":"2024-04-25T14:05:56.833776+0000","last_peered":"2024-04-25T14:05:56.833776+0000","last_clean":"2024-04-25T14:05:56.833776+0000","last_became_active":"2024-04-25T14:05:33.586317+0000","last_became_peered":"2024-04-25T14:05:33.586317+0000","last_unstale":"2024-04-25T14:05:56.833776+0000","last_undegraded":"2024-04-25T14:05:56.833776+0000","last_fullsized":"2024-04-25T14:05:56.833776+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:22.789679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1c","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833199+0000","last_change":"2024-04-25T14:05:54.193210+0000","last_active":"2024-04-25T14:05:56.833199+0000","last_peered":"2024-04-25T14:05:56.833199+0000","last_clean":"2024-04-25T14:05:56.833199+0000","last_became_active":"2024-04-25T14:05:54.192855+0000","last_became_peered":"2024-04-25T14:05:54.192855+0000","last_unstale":"2024-04-25T14:05:56.833199+0000","last_undegraded":"2024-04-25T14:05:56.833199+0000","last_fullsized":"2024-04-25T14:05:56.833199+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:57:02.409492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833688+0000","last_change":"2024-04-25T14:05:55.197405+0000","last_active":"2024-04-25T14:05:56.833688+0000","last_peered":"2024-04-25T14:05:56.833688+0000","last_clean":"2024-04-25T14:05:56.833688+0000","last_became_active":"2024-04-25T14:05:55.197111+0000","last_became_peered":"2024-04-25T14:05:55.197111+0000","last_unstale":"2024-04-25T14:05:56.833688+0000","last_undegraded":"2024-04-25T14:05:56.833688+0000","last_fullsized":"2024-04-25T14:05:56.833688+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.045928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.13","version":"45'2","reported_seq":32,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.832543+0000","last_change":"2024-04-25T14:05:32.557589+0000","last_active":"2024-04-25T14:05:56.832543+0000","last_peered":"2024-04-25T14:05:56.832543+0000","last_clean":"2024-04-25T14:05:56.832543+0000","last_became_active":"2024-04-25T14:05:32.557260+0000","last_became_peered":"2024-04-25T14:05:32.557260+0000","last_unstale":"2024-04-25T14:05:56.832543+0000","last_undegraded":"2024-04-25T14:05:56.832543+0000","last_fullsized":"2024-04-25T14:05:56.832543+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:26.912034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":19,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.888490+0000","last_change":"2024-04-25T14:05:33.569490+0000","last_active":"2024-04-25T14:05:42.888490+0000","last_peered":"2024-04-25T14:05:42.888490+0000","last_clean":"2024-04-25T14:05:42.888490+0000","last_became_active":"2024-04-25T14:05:33.569125+0000","last_became_peered":"2024-04-25T14:05:33.569125+0000","last_unstale":"2024-04-25T14:05:42.888490+0000","last_undegraded":"2024-04-25T14:05:42.888490+0000","last_fullsized":"2024-04-25T14:05:42.888490+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:33.637941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833464+0000","last_change":"2024-04-25T14:05:54.190315+0000","last_active":"2024-04-25T14:05:56.833464+0000","last_peered":"2024-04-25T14:05:56.833464+0000","last_clean":"2024-04-25T14:05:56.833464+0000","last_became_active":"2024-04-25T14:05:54.189897+0000","last_became_peered":"2024-04-25T14:05:54.189897+0000","last_unstale":"2024-04-25T14:05:56.833464+0000","last_undegraded":"2024-04-25T14:05:56.833464+0000","last_fullsized":"2024-04-25T14:05:56.833464+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:55.222389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"50'12","reported_seq":47,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.181507+0000","last_change":"2024-04-25T14:05:32.564199+0000","last_active":"2024-04-25T14:05:56.181507+0000","last_peered":"2024-04-25T14:05:56.181507+0000","last_clean":"2024-04-25T14:05:56.181507+0000","last_became_active":"2024-04-25T14:05:32.563935+0000","last_became_peered":"2024-04-25T14:05:32.563935+0000","last_unstale":"2024-04-25T14:05:56.181507+0000","last_undegraded":"2024-04-25T14:05:56.181507+0000","last_fullsized":"2024-04-25T14:05:56.181507+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:38.715145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13068,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":11,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.912015+0000","last_change":"2024-04-25T14:05:33.593376+0000","last_active":"2024-04-25T14:05:42.912015+0000","last_peered":"2024-04-25T14:05:42.912015+0000","last_clean":"2024-04-25T14:05:42.912015+0000","last_became_active":"2024-04-25T14:05:33.593105+0000","last_became_peered":"2024-04-25T14:05:33.593105+0000","last_unstale":"2024-04-25T14:05:42.912015+0000","last_undegraded":"2024-04-25T14:05:42.912015+0000","last_fullsized":"2024-04-25T14:05:42.912015+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:28.629474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1a","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180726+0000","last_change":"2024-04-25T14:05:54.192638+0000","last_active":"2024-04-25T14:05:56.180726+0000","last_peered":"2024-04-25T14:05:56.180726+0000","last_clean":"2024-04-25T14:05:56.180726+0000","last_became_active":"2024-04-25T14:05:54.192278+0000","last_became_peered":"2024-04-25T14:05:54.192278+0000","last_unstale":"2024-04-25T14:05:56.180726+0000","last_undegraded":"2024-04-25T14:05:56.180726+0000","last_fullsized":"2024-04-25T14:05:56.180726+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:00:09.293862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:57.054780+0000","last_change":"2024-04-25T14:05:55.200575+0000","last_active":"2024-04-25T14:05:57.054780+0000","last_peered":"2024-04-25T14:05:57.054780+0000","last_clean":"2024-04-25T14:05:57.054780+0000","last_became_active":"2024-04-25T14:05:55.199674+0000","last_became_peered":"2024-04-25T14:05:55.199674+0000","last_unstale":"2024-04-25T14:05:57.054780+0000","last_undegraded":"2024-04-25T14:05:57.054780+0000","last_fullsized":"2024-04-25T14:05:57.054780+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:39.697034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"45'3","reported_seq":26,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:45.920247+0000","last_change":"2024-04-25T14:05:32.566039+0000","last_active":"2024-04-25T14:05:45.920247+0000","last_peered":"2024-04-25T14:05:45.920247+0000","last_clean":"2024-04-25T14:05:45.920247+0000","last_became_active":"2024-04-25T14:05:32.565670+0000","last_became_peered":"2024-04-25T14:05:32.565670+0000","last_unstale":"2024-04-25T14:05:45.920247+0000","last_undegraded":"2024-04-25T14:05:45.920247+0000","last_fullsized":"2024-04-25T14:05:45.920247+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:12.973199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833469+0000","last_change":"2024-04-25T14:05:33.577142+0000","last_active":"2024-04-25T14:05:56.833469+0000","last_peered":"2024-04-25T14:05:56.833469+0000","last_clean":"2024-04-25T14:05:56.833469+0000","last_became_active":"2024-04-25T14:05:33.576820+0000","last_became_peered":"2024-04-25T14:05:33.576820+0000","last_unstale":"2024-04-25T14:05:56.833469+0000","last_undegraded":"2024-04-25T14:05:56.833469+0000","last_fullsized":"2024-04-25T14:05:56.833469+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:07.917626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1b","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180733+0000","last_change":"2024-04-25T14:05:54.180076+0000","last_active":"2024-04-25T14:05:56.180733+0000","last_peered":"2024-04-25T14:05:56.180733+0000","last_clean":"2024-04-25T14:05:56.180733+0000","last_became_active":"2024-04-25T14:05:54.179725+0000","last_became_peered":"2024-04-25T14:05:54.179725+0000","last_unstale":"2024-04-25T14:05:56.180733+0000","last_undegraded":"2024-04-25T14:05:56.180733+0000","last_fullsized":"2024-04-25T14:05:56.180733+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:38.204467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.181130+0000","last_change":"2024-04-25T14:05:55.176089+0000","last_active":"2024-04-25T14:05:56.181130+0000","last_peered":"2024-04-25T14:05:56.181130+0000","last_clean":"2024-04-25T14:05:56.181130+0000","last_became_active":"2024-04-25T14:05:55.175181+0000","last_became_peered":"2024-04-25T14:05:55.175181+0000","last_unstale":"2024-04-25T14:05:56.181130+0000","last_undegraded":"2024-04-25T14:05:56.181130+0000","last_fullsized":"2024-04-25T14:05:56.181130+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:39.678190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":19,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.912027+0000","last_change":"2024-04-25T14:05:32.563130+0000","last_active":"2024-04-25T14:05:42.912027+0000","last_peered":"2024-04-25T14:05:42.912027+0000","last_clean":"2024-04-25T14:05:42.912027+0000","last_became_active":"2024-04-25T14:05:32.562793+0000","last_became_peered":"2024-04-25T14:05:32.562793+0000","last_unstale":"2024-04-25T14:05:42.912027+0000","last_undegraded":"2024-04-25T14:05:42.912027+0000","last_fullsized":"2024-04-25T14:05:42.912027+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:43.267251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":19,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.888431+0000","last_change":"2024-04-25T14:05:33.573758+0000","last_active":"2024-04-25T14:05:42.888431+0000","last_peered":"2024-04-25T14:05:42.888431+0000","last_clean":"2024-04-25T14:05:42.888431+0000","last_became_active":"2024-04-25T14:05:33.572760+0000","last_became_peered":"2024-04-25T14:05:33.572760+0000","last_unstale":"2024-04-25T14:05:42.888431+0000","last_undegraded":"2024-04-25T14:05:42.888431+0000","last_fullsized":"2024-04-25T14:05:42.888431+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:36.723420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.186337+0000","last_change":"2024-04-25T14:05:54.193681+0000","last_active":"2024-04-25T14:05:56.186337+0000","last_peered":"2024-04-25T14:05:56.186337+0000","last_clean":"2024-04-25T14:05:56.186337+0000","last_became_active":"2024-04-25T14:05:54.190583+0000","last_became_peered":"2024-04-25T14:05:54.190583+0000","last_unstale":"2024-04-25T14:05:56.186337+0000","last_undegraded":"2024-04-25T14:05:56.186337+0000","last_fullsized":"2024-04-25T14:05:56.186337+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:46.274456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.187626+0000","last_change":"2024-04-25T14:05:55.211925+0000","last_active":"2024-04-25T14:05:56.187626+0000","last_peered":"2024-04-25T14:05:56.187626+0000","last_clean":"2024-04-25T14:05:56.187626+0000","last_became_active":"2024-04-25T14:05:55.211644+0000","last_became_peered":"2024-04-25T14:05:55.211644+0000","last_unstale":"2024-04-25T14:05:56.187626+0000","last_undegraded":"2024-04-25T14:05:56.187626+0000","last_fullsized":"2024-04-25T14:05:56.187626+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:51.604196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":20,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.907471+0000","last_change":"2024-04-25T14:05:32.550533+0000","last_active":"2024-04-25T14:05:42.907471+0000","last_peered":"2024-04-25T14:05:42.907471+0000","last_clean":"2024-04-25T14:05:42.907471+0000","last_became_active":"2024-04-25T14:05:32.550116+0000","last_became_peered":"2024-04-25T14:05:32.550116+0000","last_unstale":"2024-04-25T14:05:42.907471+0000","last_undegraded":"2024-04-25T14:05:42.907471+0000","last_fullsized":"2024-04-25T14:05:42.907471+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:27.623051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.911413+0000","last_change":"2024-04-25T14:05:33.583166+0000","last_active":"2024-04-25T14:05:42.911413+0000","last_peered":"2024-04-25T14:05:42.911413+0000","last_clean":"2024-04-25T14:05:42.911413+0000","last_became_active":"2024-04-25T14:05:33.582833+0000","last_became_peered":"2024-04-25T14:05:33.582833+0000","last_unstale":"2024-04-25T14:05:42.911413+0000","last_undegraded":"2024-04-25T14:05:42.911413+0000","last_fullsized":"2024-04-25T14:05:42.911413+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:11.087435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180458+0000","last_change":"2024-04-25T14:05:54.190905+0000","last_active":"2024-04-25T14:05:56.180458+0000","last_peered":"2024-04-25T14:05:56.180458+0000","last_clean":"2024-04-25T14:05:56.180458+0000","last_became_active":"2024-04-25T14:05:54.190297+0000","last_became_peered":"2024-04-25T14:05:54.190297+0000","last_unstale":"2024-04-25T14:05:56.180458+0000","last_undegraded":"2024-04-25T14:05:56.180458+0000","last_fullsized":"2024-04-25T14:05:56.180458+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:23.256150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.181689+0000","last_change":"2024-04-25T14:05:55.175335+0000","last_active":"2024-04-25T14:05:56.181689+0000","last_peered":"2024-04-25T14:05:56.181689+0000","last_clean":"2024-04-25T14:05:56.181689+0000","last_became_active":"2024-04-25T14:05:55.174104+0000","last_became_peered":"2024-04-25T14:05:55.174104+0000","last_unstale":"2024-04-25T14:05:56.181689+0000","last_undegraded":"2024-04-25T14:05:56.181689+0000","last_fullsized":"2024-04-25T14:05:56.181689+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:42:48.413708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":28,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.832556+0000","last_change":"2024-04-25T14:05:32.563104+0000","last_active":"2024-04-25T14:05:56.832556+0000","last_peered":"2024-04-25T14:05:56.832556+0000","last_clean":"2024-04-25T14:05:56.832556+0000","last_became_active":"2024-04-25T14:05:32.560440+0000","last_became_peered":"2024-04-25T14:05:32.560440+0000","last_unstale":"2024-04-25T14:05:56.832556+0000","last_undegraded":"2024-04-25T14:05:56.832556+0000","last_fullsized":"2024-04-25T14:05:56.832556+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:12.973199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.907269+0000","last_change":"2024-04-25T14:05:33.583073+0000","last_active":"2024-04-25T14:05:42.907269+0000","last_peered":"2024-04-25T14:05:42.907269+0000","last_clean":"2024-04-25T14:05:42.907269+0000","last_became_active":"2024-04-25T14:05:33.582763+0000","last_became_peered":"2024-04-25T14:05:33.582763+0000","last_unstale":"2024-04-25T14:05:42.907269+0000","last_undegraded":"2024-04-25T14:05:42.907269+0000","last_fullsized":"2024-04-25T14:05:42.907269+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:18.567327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:53.144076+0000","last_change":"2024-04-25T14:05:53.144076+0000","last_active":"2024-04-25T14:05:53.144076+0000","last_peered":"2024-04-25T14:05:53.144076+0000","last_clean":"2024-04-25T14:05:53.144076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:53.144076+0000","last_undegraded":"2024-04-25T14:05:53.144076+0000","last_fullsized":"2024-04-25T14:05:53.144076+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.186131+0000","last_change":"2024-04-25T14:05:32.551806+0000","last_active":"2024-04-25T14:05:56.186131+0000","last_peered":"2024-04-25T14:05:56.186131+0000","last_clean":"2024-04-25T14:05:56.186131+0000","last_became_active":"2024-04-25T14:05:32.551441+0000","last_became_peered":"2024-04-25T14:05:32.551441+0000","last_unstale":"2024-04-25T14:05:56.186131+0000","last_undegraded":"2024-04-25T14:05:56.186131+0000","last_fullsized":"2024-04-25T14:05:56.186131+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:31.414048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.181552+0000","last_change":"2024-04-25T14:05:33.576549+0000","last_active":"2024-04-25T14:05:56.181552+0000","last_peered":"2024-04-25T14:05:56.181552+0000","last_clean":"2024-04-25T14:05:56.181552+0000","last_became_active":"2024-04-25T14:05:33.576037+0000","last_became_peered":"2024-04-25T14:05:33.576037+0000","last_unstale":"2024-04-25T14:05:56.181552+0000","last_undegraded":"2024-04-25T14:05:56.181552+0000","last_fullsized":"2024-04-25T14:05:56.181552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:47.818409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:53.144076+0000","last_change":"2024-04-25T14:05:53.144076+0000","last_active":"2024-04-25T14:05:53.144076+0000","last_peered":"2024-04-25T14:05:53.144076+0000","last_clean":"2024-04-25T14:05:53.144076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:53.144076+0000","last_undegraded":"2024-04-25T14:05:53.144076+0000","last_fullsized":"2024-04-25T14:05:53.144076+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"50'2","reported_seq":21,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.911919+0000","last_change":"2024-04-25T14:05:32.557698+0000","last_active":"2024-04-25T14:05:42.911919+0000","last_peered":"2024-04-25T14:05:42.911919+0000","last_clean":"2024-04-25T14:05:42.911919+0000","last_became_active":"2024-04-25T14:05:32.557358+0000","last_became_peered":"2024-04-25T14:05:32.557358+0000","last_unstale":"2024-04-25T14:05:42.911919+0000","last_undegraded":"2024-04-25T14:05:42.911919+0000","last_fullsized":"2024-04-25T14:05:42.911919+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:20.281168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.906326+0000","last_change":"2024-04-25T14:05:33.584433+0000","last_active":"2024-04-25T14:05:42.906326+0000","last_peered":"2024-04-25T14:05:42.906326+0000","last_clean":"2024-04-25T14:05:42.906326+0000","last_became_active":"2024-04-25T14:05:33.584130+0000","last_became_peered":"2024-04-25T14:05:33.584130+0000","last_unstale":"2024-04-25T14:05:42.906326+0000","last_undegraded":"2024-04-25T14:05:42.906326+0000","last_fullsized":"2024-04-25T14:05:42.906326+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:42.654848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:53.144076+0000","last_change":"2024-04-25T14:05:53.144076+0000","last_active":"2024-04-25T14:05:53.144076+0000","last_peered":"2024-04-25T14:05:53.144076+0000","last_clean":"2024-04-25T14:05:53.144076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:53.144076+0000","last_undegraded":"2024-04-25T14:05:53.144076+0000","last_fullsized":"2024-04-25T14:05:53.144076+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.186682+0000","last_change":"2024-04-25T14:05:55.202255+0000","last_active":"2024-04-25T14:05:56.186682+0000","last_peered":"2024-04-25T14:05:56.186682+0000","last_clean":"2024-04-25T14:05:56.186682+0000","last_became_active":"2024-04-25T14:05:55.201856+0000","last_became_peered":"2024-04-25T14:05:55.201856+0000","last_unstale":"2024-04-25T14:05:56.186682+0000","last_undegraded":"2024-04-25T14:05:56.186682+0000","last_fullsized":"2024-04-25T14:05:56.186682+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:51.863500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"50'2","reported_seq":30,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833327+0000","last_change":"2024-04-25T14:05:32.562043+0000","last_active":"2024-04-25T14:05:56.833327+0000","last_peered":"2024-04-25T14:05:56.833327+0000","last_clean":"2024-04-25T14:05:56.833327+0000","last_became_active":"2024-04-25T14:05:32.561736+0000","last_became_peered":"2024-04-25T14:05:32.561736+0000","last_unstale":"2024-04-25T14:05:56.833327+0000","last_undegraded":"2024-04-25T14:05:56.833327+0000","last_fullsized":"2024-04-25T14:05:56.833327+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:40.505936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.912247+0000","last_change":"2024-04-25T14:05:33.582814+0000","last_active":"2024-04-25T14:05:42.912247+0000","last_peered":"2024-04-25T14:05:42.912247+0000","last_clean":"2024-04-25T14:05:42.912247+0000","last_became_active":"2024-04-25T14:05:33.582127+0000","last_became_peered":"2024-04-25T14:05:33.582127+0000","last_unstale":"2024-04-25T14:05:42.912247+0000","last_undegraded":"2024-04-25T14:05:42.912247+0000","last_fullsized":"2024-04-25T14:05:42.912247+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:18.074490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:53.144076+0000","last_change":"2024-04-25T14:05:53.144076+0000","last_active":"2024-04-25T14:05:53.144076+0000","last_peered":"2024-04-25T14:05:53.144076+0000","last_clean":"2024-04-25T14:05:53.144076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:53.144076+0000","last_undegraded":"2024-04-25T14:05:53.144076+0000","last_fullsized":"2024-04-25T14:05:53.144076+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"45'2","reported_seq":25,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:45.923387+0000","last_change":"2024-04-25T14:05:32.567307+0000","last_active":"2024-04-25T14:05:45.923387+0000","last_peered":"2024-04-25T14:05:45.923387+0000","last_clean":"2024-04-25T14:05:45.923387+0000","last_became_active":"2024-04-25T14:05:32.566940+0000","last_became_peered":"2024-04-25T14:05:32.566940+0000","last_unstale":"2024-04-25T14:05:45.923387+0000","last_undegraded":"2024-04-25T14:05:45.923387+0000","last_fullsized":"2024-04-25T14:05:45.923387+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:12.973199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180912+0000","last_change":"2024-04-25T14:05:33.593213+0000","last_active":"2024-04-25T14:05:56.180912+0000","last_peered":"2024-04-25T14:05:56.180912+0000","last_clean":"2024-04-25T14:05:56.180912+0000","last_became_active":"2024-04-25T14:05:33.592923+0000","last_became_peered":"2024-04-25T14:05:33.592923+0000","last_unstale":"2024-04-25T14:05:56.180912+0000","last_undegraded":"2024-04-25T14:05:56.180912+0000","last_fullsized":"2024-04-25T14:05:56.180912+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:18.458163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:53.144076+0000","last_change":"2024-04-25T14:05:53.144076+0000","last_active":"2024-04-25T14:05:53.144076+0000","last_peered":"2024-04-25T14:05:53.144076+0000","last_clean":"2024-04-25T14:05:53.144076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:53.144076+0000","last_undegraded":"2024-04-25T14:05:53.144076+0000","last_fullsized":"2024-04-25T14:05:53.144076+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"45'2","reported_seq":25,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:45.920852+0000","last_change":"2024-04-25T14:05:32.561304+0000","last_active":"2024-04-25T14:05:45.920852+0000","last_peered":"2024-04-25T14:05:45.920852+0000","last_clean":"2024-04-25T14:05:45.920852+0000","last_became_active":"2024-04-25T14:05:32.560988+0000","last_became_peered":"2024-04-25T14:05:32.560988+0000","last_unstale":"2024-04-25T14:05:45.920852+0000","last_undegraded":"2024-04-25T14:05:45.920852+0000","last_fullsized":"2024-04-25T14:05:45.920852+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:32.071918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":25,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.186810+0000","last_change":"2024-04-25T14:05:33.574504+0000","last_active":"2024-04-25T14:05:56.186810+0000","last_peered":"2024-04-25T14:05:56.186810+0000","last_clean":"2024-04-25T14:05:56.186810+0000","last_became_active":"2024-04-25T14:05:33.574191+0000","last_became_peered":"2024-04-25T14:05:33.574191+0000","last_unstale":"2024-04-25T14:05:56.186810+0000","last_undegraded":"2024-04-25T14:05:56.186810+0000","last_fullsized":"2024-04-25T14:05:56.186810+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:28.629474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:53.144076+0000","last_change":"2024-04-25T14:05:53.144076+0000","last_active":"2024-04-25T14:05:53.144076+0000","last_peered":"2024-04-25T14:05:53.144076+0000","last_clean":"2024-04-25T14:05:53.144076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:53.144076+0000","last_undegraded":"2024-04-25T14:05:53.144076+0000","last_fullsized":"2024-04-25T14:05:53.144076+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"50'2","reported_seq":23,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.905524+0000","last_change":"2024-04-25T14:05:32.569704+0000","last_active":"2024-04-25T14:05:42.905524+0000","last_peered":"2024-04-25T14:05:42.905524+0000","last_clean":"2024-04-25T14:05:42.905524+0000","last_became_active":"2024-04-25T14:05:32.569349+0000","last_became_peered":"2024-04-25T14:05:32.569349+0000","last_unstale":"2024-04-25T14:05:42.905524+0000","last_undegraded":"2024-04-25T14:05:42.905524+0000","last_fullsized":"2024-04-25T14:05:42.905524+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:23.323591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":19,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.888465+0000","last_change":"2024-04-25T14:05:33.573320+0000","last_active":"2024-04-25T14:05:42.888465+0000","last_peered":"2024-04-25T14:05:42.888465+0000","last_clean":"2024-04-25T14:05:42.888465+0000","last_became_active":"2024-04-25T14:05:33.573015+0000","last_became_peered":"2024-04-25T14:05:33.573015+0000","last_unstale":"2024-04-25T14:05:42.888465+0000","last_undegraded":"2024-04-25T14:05:42.888465+0000","last_fullsized":"2024-04-25T14:05:42.888465+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:25.659996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:53.144076+0000","last_change":"2024-04-25T14:05:53.144076+0000","last_active":"2024-04-25T14:05:53.144076+0000","last_peered":"2024-04-25T14:05:53.144076+0000","last_clean":"2024-04-25T14:05:53.144076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:53.144076+0000","last_undegraded":"2024-04-25T14:05:53.144076+0000","last_fullsized":"2024-04-25T14:05:53.144076+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.186271+0000","last_change":"2024-04-25T14:05:55.208703+0000","last_active":"2024-04-25T14:05:56.186271+0000","last_peered":"2024-04-25T14:05:56.186271+0000","last_clean":"2024-04-25T14:05:56.186271+0000","last_became_active":"2024-04-25T14:05:55.208391+0000","last_became_peered":"2024-04-25T14:05:55.208391+0000","last_unstale":"2024-04-25T14:05:56.186271+0000","last_undegraded":"2024-04-25T14:05:56.186271+0000","last_fullsized":"2024-04-25T14:05:56.186271+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:06.318625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"50'5","reported_seq":34,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.187081+0000","last_change":"2024-04-25T14:05:32.556304+0000","last_active":"2024-04-25T14:05:56.187081+0000","last_peered":"2024-04-25T14:05:56.187081+0000","last_clean":"2024-04-25T14:05:56.187081+0000","last_became_active":"2024-04-25T14:05:32.556052+0000","last_became_peered":"2024-04-25T14:05:32.556052+0000","last_unstale":"2024-04-25T14:05:56.187081+0000","last_undegraded":"2024-04-25T14:05:56.187081+0000","last_fullsized":"2024-04-25T14:05:56.187081+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:16.452558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.905934+0000","last_change":"2024-04-25T14:05:33.576943+0000","last_active":"2024-04-25T14:05:42.905934+0000","last_peered":"2024-04-25T14:05:42.905934+0000","last_clean":"2024-04-25T14:05:42.905934+0000","last_became_active":"2024-04-25T14:05:33.576387+0000","last_became_peered":"2024-04-25T14:05:33.576387+0000","last_unstale":"2024-04-25T14:05:42.905934+0000","last_undegraded":"2024-04-25T14:05:42.905934+0000","last_fullsized":"2024-04-25T14:05:42.905934+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:54.564632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:53.144076+0000","last_change":"2024-04-25T14:05:53.144076+0000","last_active":"2024-04-25T14:05:53.144076+0000","last_peered":"2024-04-25T14:05:53.144076+0000","last_clean":"2024-04-25T14:05:53.144076+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:53.144076+0000","last_undegraded":"2024-04-25T14:05:53.144076+0000","last_fullsized":"2024-04-25T14:05:53.144076+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":19,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.888427+0000","last_change":"2024-04-25T14:05:33.561025+0000","last_active":"2024-04-25T14:05:42.888427+0000","last_peered":"2024-04-25T14:05:42.888427+0000","last_clean":"2024-04-25T14:05:42.888427+0000","last_became_active":"2024-04-25T14:05:33.560525+0000","last_became_peered":"2024-04-25T14:05:33.560525+0000","last_unstale":"2024-04-25T14:05:42.888427+0000","last_undegraded":"2024-04-25T14:05:42.888427+0000","last_fullsized":"2024-04-25T14:05:42.888427+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:54.564632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.832945+0000","last_change":"2024-04-25T14:05:33.583788+0000","last_active":"2024-04-25T14:05:56.832945+0000","last_peered":"2024-04-25T14:05:56.832945+0000","last_clean":"2024-04-25T14:05:56.832945+0000","last_became_active":"2024-04-25T14:05:33.583507+0000","last_became_peered":"2024-04-25T14:05:33.583507+0000","last_unstale":"2024-04-25T14:05:56.832945+0000","last_undegraded":"2024-04-25T14:05:56.832945+0000","last_fullsized":"2024-04-25T14:05:56.832945+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:06.138261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.832968+0000","last_change":"2024-04-25T14:05:55.207790+0000","last_active":"2024-04-25T14:05:56.832968+0000","last_peered":"2024-04-25T14:05:56.832968+0000","last_clean":"2024-04-25T14:05:56.832968+0000","last_became_active":"2024-04-25T14:05:55.206817+0000","last_became_peered":"2024-04-25T14:05:55.206817+0000","last_unstale":"2024-04-25T14:05:56.832968+0000","last_undegraded":"2024-04-25T14:05:56.832968+0000","last_fullsized":"2024-04-25T14:05:56.832968+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:42:48.413708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.906505+0000","last_change":"2024-04-25T14:05:33.574825+0000","last_active":"2024-04-25T14:05:42.906505+0000","last_peered":"2024-04-25T14:05:42.906505+0000","last_clean":"2024-04-25T14:05:42.906505+0000","last_became_active":"2024-04-25T14:05:33.574492+0000","last_became_peered":"2024-04-25T14:05:33.574492+0000","last_unstale":"2024-04-25T14:05:42.906505+0000","last_undegraded":"2024-04-25T14:05:42.906505+0000","last_fullsized":"2024-04-25T14:05:42.906505+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:33.637941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833177+0000","last_change":"2024-04-25T14:05:33.572734+0000","last_active":"2024-04-25T14:05:56.833177+0000","last_peered":"2024-04-25T14:05:56.833177+0000","last_clean":"2024-04-25T14:05:56.833177+0000","last_became_active":"2024-04-25T14:05:33.572231+0000","last_became_peered":"2024-04-25T14:05:33.572231+0000","last_unstale":"2024-04-25T14:05:56.833177+0000","last_undegraded":"2024-04-25T14:05:56.833177+0000","last_fullsized":"2024-04-25T14:05:56.833177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:51.585470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":25,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.186901+0000","last_change":"2024-04-25T14:05:33.596454+0000","last_active":"2024-04-25T14:05:56.186901+0000","last_peered":"2024-04-25T14:05:56.186901+0000","last_clean":"2024-04-25T14:05:56.186901+0000","last_became_active":"2024-04-25T14:05:33.596225+0000","last_became_peered":"2024-04-25T14:05:33.596225+0000","last_unstale":"2024-04-25T14:05:56.186901+0000","last_undegraded":"2024-04-25T14:05:56.186901+0000","last_fullsized":"2024-04-25T14:05:56.186901+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:41.512359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":19,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.905290+0000","last_change":"2024-04-25T14:05:33.591448+0000","last_active":"2024-04-25T14:05:42.905290+0000","last_peered":"2024-04-25T14:05:42.905290+0000","last_clean":"2024-04-25T14:05:42.905290+0000","last_became_active":"2024-04-25T14:05:33.590795+0000","last_became_peered":"2024-04-25T14:05:33.590795+0000","last_unstale":"2024-04-25T14:05:42.905290+0000","last_undegraded":"2024-04-25T14:05:42.905290+0000","last_fullsized":"2024-04-25T14:05:42.905290+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:33.637941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.181438+0000","last_change":"2024-04-25T14:05:33.576318+0000","last_active":"2024-04-25T14:05:56.181438+0000","last_peered":"2024-04-25T14:05:56.181438+0000","last_clean":"2024-04-25T14:05:56.181438+0000","last_became_active":"2024-04-25T14:05:33.575764+0000","last_became_peered":"2024-04-25T14:05:33.575764+0000","last_unstale":"2024-04-25T14:05:56.181438+0000","last_undegraded":"2024-04-25T14:05:56.181438+0000","last_fullsized":"2024-04-25T14:05:56.181438+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:33.455514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833857+0000","last_change":"2024-04-25T14:05:33.562329+0000","last_active":"2024-04-25T14:05:56.833857+0000","last_peered":"2024-04-25T14:05:56.833857+0000","last_clean":"2024-04-25T14:05:56.833857+0000","last_became_active":"2024-04-25T14:05:33.562039+0000","last_became_peered":"2024-04-25T14:05:33.562039+0000","last_unstale":"2024-04-25T14:05:56.833857+0000","last_undegraded":"2024-04-25T14:05:56.833857+0000","last_fullsized":"2024-04-25T14:05:56.833857+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:29.981652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180509+0000","last_change":"2024-04-25T14:05:55.186637+0000","last_active":"2024-04-25T14:05:56.180509+0000","last_peered":"2024-04-25T14:05:56.180509+0000","last_clean":"2024-04-25T14:05:56.180509+0000","last_became_active":"2024-04-25T14:05:55.185799+0000","last_became_peered":"2024-04-25T14:05:55.185799+0000","last_unstale":"2024-04-25T14:05:56.180509+0000","last_undegraded":"2024-04-25T14:05:56.180509+0000","last_fullsized":"2024-04-25T14:05:56.180509+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:49:29.357854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833087+0000","last_change":"2024-04-25T14:05:33.581503+0000","last_active":"2024-04-25T14:05:56.833087+0000","last_peered":"2024-04-25T14:05:56.833087+0000","last_clean":"2024-04-25T14:05:56.833087+0000","last_became_active":"2024-04-25T14:05:33.581124+0000","last_became_peered":"2024-04-25T14:05:33.581124+0000","last_unstale":"2024-04-25T14:05:56.833087+0000","last_undegraded":"2024-04-25T14:05:56.833087+0000","last_fullsized":"2024-04-25T14:05:56.833087+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:33.455514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:57.055125+0000","last_change":"2024-04-25T14:05:33.576117+0000","last_active":"2024-04-25T14:05:57.055125+0000","last_peered":"2024-04-25T14:05:57.055125+0000","last_clean":"2024-04-25T14:05:57.055125+0000","last_became_active":"2024-04-25T14:05:33.575838+0000","last_became_peered":"2024-04-25T14:05:33.575838+0000","last_unstale":"2024-04-25T14:05:57.055125+0000","last_undegraded":"2024-04-25T14:05:57.055125+0000","last_fullsized":"2024-04-25T14:05:57.055125+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:20.619851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180782+0000","last_change":"2024-04-25T14:05:33.560440+0000","last_active":"2024-04-25T14:05:56.180782+0000","last_peered":"2024-04-25T14:05:56.180782+0000","last_clean":"2024-04-25T14:05:56.180782+0000","last_became_active":"2024-04-25T14:05:33.560066+0000","last_became_peered":"2024-04-25T14:05:33.560066+0000","last_unstale":"2024-04-25T14:05:56.180782+0000","last_undegraded":"2024-04-25T14:05:56.180782+0000","last_fullsized":"2024-04-25T14:05:56.180782+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:32.906788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180496+0000","last_change":"2024-04-25T14:05:55.186492+0000","last_active":"2024-04-25T14:05:56.180496+0000","last_peered":"2024-04-25T14:05:56.180496+0000","last_clean":"2024-04-25T14:05:56.180496+0000","last_became_active":"2024-04-25T14:05:55.186183+0000","last_became_peered":"2024-04-25T14:05:55.186183+0000","last_unstale":"2024-04-25T14:05:56.180496+0000","last_undegraded":"2024-04-25T14:05:56.180496+0000","last_fullsized":"2024-04-25T14:05:56.180496+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:27.511725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":25,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.187035+0000","last_change":"2024-04-25T14:05:33.571248+0000","last_active":"2024-04-25T14:05:56.187035+0000","last_peered":"2024-04-25T14:05:56.187035+0000","last_clean":"2024-04-25T14:05:56.187035+0000","last_became_active":"2024-04-25T14:05:33.570916+0000","last_became_peered":"2024-04-25T14:05:33.570916+0000","last_unstale":"2024-04-25T14:05:56.187035+0000","last_undegraded":"2024-04-25T14:05:56.187035+0000","last_fullsized":"2024-04-25T14:05:56.187035+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:36.296676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":25,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.186754+0000","last_change":"2024-04-25T14:05:33.571072+0000","last_active":"2024-04-25T14:05:56.186754+0000","last_peered":"2024-04-25T14:05:56.186754+0000","last_clean":"2024-04-25T14:05:56.186754+0000","last_became_active":"2024-04-25T14:05:33.570748+0000","last_became_peered":"2024-04-25T14:05:33.570748+0000","last_unstale":"2024-04-25T14:05:56.186754+0000","last_undegraded":"2024-04-25T14:05:56.186754+0000","last_fullsized":"2024-04-25T14:05:56.186754+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:18.074490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833644+0000","last_change":"2024-04-25T14:05:33.574245+0000","last_active":"2024-04-25T14:05:56.833644+0000","last_peered":"2024-04-25T14:05:56.833644+0000","last_clean":"2024-04-25T14:05:56.833644+0000","last_became_active":"2024-04-25T14:05:33.573980+0000","last_became_peered":"2024-04-25T14:05:33.573980+0000","last_unstale":"2024-04-25T14:05:56.833644+0000","last_undegraded":"2024-04-25T14:05:56.833644+0000","last_fullsized":"2024-04-25T14:05:56.833644+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:30.240956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.181101+0000","last_change":"2024-04-25T14:05:55.212520+0000","last_active":"2024-04-25T14:05:56.181101+0000","last_peered":"2024-04-25T14:05:56.181101+0000","last_clean":"2024-04-25T14:05:56.181101+0000","last_became_active":"2024-04-25T14:05:55.212236+0000","last_became_peered":"2024-04-25T14:05:55.212236+0000","last_unstale":"2024-04-25T14:05:56.181101+0000","last_undegraded":"2024-04-25T14:05:56.181101+0000","last_fullsized":"2024-04-25T14:05:56.181101+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:07.087465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.907290+0000","last_change":"2024-04-25T14:05:33.580135+0000","last_active":"2024-04-25T14:05:42.907290+0000","last_peered":"2024-04-25T14:05:42.907290+0000","last_clean":"2024-04-25T14:05:42.907290+0000","last_became_active":"2024-04-25T14:05:33.579677+0000","last_became_peered":"2024-04-25T14:05:33.579677+0000","last_unstale":"2024-04-25T14:05:42.907290+0000","last_undegraded":"2024-04-25T14:05:42.907290+0000","last_fullsized":"2024-04-25T14:05:42.907290+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:20.253140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.187368+0000","last_change":"2024-04-25T14:05:55.207851+0000","last_active":"2024-04-25T14:05:56.187368+0000","last_peered":"2024-04-25T14:05:56.187368+0000","last_clean":"2024-04-25T14:05:56.187368+0000","last_became_active":"2024-04-25T14:05:55.204703+0000","last_became_peered":"2024-04-25T14:05:55.204703+0000","last_unstale":"2024-04-25T14:05:56.187368+0000","last_undegraded":"2024-04-25T14:05:56.187368+0000","last_fullsized":"2024-04-25T14:05:56.187368+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:13.208013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833751+0000","last_change":"2024-04-25T14:05:33.572330+0000","last_active":"2024-04-25T14:05:56.833751+0000","last_peered":"2024-04-25T14:05:56.833751+0000","last_clean":"2024-04-25T14:05:56.833751+0000","last_became_active":"2024-04-25T14:05:33.571842+0000","last_became_peered":"2024-04-25T14:05:33.571842+0000","last_unstale":"2024-04-25T14:05:56.833751+0000","last_undegraded":"2024-04-25T14:05:56.833751+0000","last_fullsized":"2024-04-25T14:05:56.833751+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:28.424318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"50'2","reported_seq":21,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:50.241122+0000","last_change":"2024-04-25T14:05:33.573271+0000","last_active":"2024-04-25T14:05:50.241122+0000","last_peered":"2024-04-25T14:05:50.241122+0000","last_clean":"2024-04-25T14:05:50.241122+0000","last_became_active":"2024-04-25T14:05:33.572858+0000","last_became_peered":"2024-04-25T14:05:33.572858+0000","last_unstale":"2024-04-25T14:05:50.241122+0000","last_undegraded":"2024-04-25T14:05:50.241122+0000","last_fullsized":"2024-04-25T14:05:50.241122+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:27.918457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.906547+0000","last_change":"2024-04-25T14:05:33.575396+0000","last_active":"2024-04-25T14:05:42.906547+0000","last_peered":"2024-04-25T14:05:42.906547+0000","last_clean":"2024-04-25T14:05:42.906547+0000","last_became_active":"2024-04-25T14:05:33.574772+0000","last_became_peered":"2024-04-25T14:05:33.574772+0000","last_unstale":"2024-04-25T14:05:42.906547+0000","last_undegraded":"2024-04-25T14:05:42.906547+0000","last_fullsized":"2024-04-25T14:05:42.906547+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:25.659996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:57.054786+0000","last_change":"2024-04-25T14:05:55.192624+0000","last_active":"2024-04-25T14:05:57.054786+0000","last_peered":"2024-04-25T14:05:57.054786+0000","last_clean":"2024-04-25T14:05:57.054786+0000","last_became_active":"2024-04-25T14:05:55.192246+0000","last_became_peered":"2024-04-25T14:05:55.192246+0000","last_unstale":"2024-04-25T14:05:57.054786+0000","last_undegraded":"2024-04-25T14:05:57.054786+0000","last_fullsized":"2024-04-25T14:05:57.054786+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:35.602166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833391+0000","last_change":"2024-04-25T14:05:33.581470+0000","last_active":"2024-04-25T14:05:56.833391+0000","last_peered":"2024-04-25T14:05:56.833391+0000","last_clean":"2024-04-25T14:05:56.833391+0000","last_became_active":"2024-04-25T14:05:33.581160+0000","last_became_peered":"2024-04-25T14:05:33.581160+0000","last_unstale":"2024-04-25T14:05:56.833391+0000","last_undegraded":"2024-04-25T14:05:56.833391+0000","last_fullsized":"2024-04-25T14:05:56.833391+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:47.818409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":19,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.905650+0000","last_change":"2024-04-25T14:05:33.572748+0000","last_active":"2024-04-25T14:05:42.905650+0000","last_peered":"2024-04-25T14:05:42.905650+0000","last_clean":"2024-04-25T14:05:42.905650+0000","last_became_active":"2024-04-25T14:05:33.572256+0000","last_became_peered":"2024-04-25T14:05:33.572256+0000","last_unstale":"2024-04-25T14:05:42.905650+0000","last_undegraded":"2024-04-25T14:05:42.905650+0000","last_fullsized":"2024-04-25T14:05:42.905650+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:43.410378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.181463+0000","last_change":"2024-04-25T14:05:33.565074+0000","last_active":"2024-04-25T14:05:56.181463+0000","last_peered":"2024-04-25T14:05:56.181463+0000","last_clean":"2024-04-25T14:05:56.181463+0000","last_became_active":"2024-04-25T14:05:33.564800+0000","last_became_peered":"2024-04-25T14:05:33.564800+0000","last_unstale":"2024-04-25T14:05:56.181463+0000","last_undegraded":"2024-04-25T14:05:56.181463+0000","last_fullsized":"2024-04-25T14:05:56.181463+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:00.811494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:57.055456+0000","last_change":"2024-04-25T14:05:33.573200+0000","last_active":"2024-04-25T14:05:57.055456+0000","last_peered":"2024-04-25T14:05:57.055456+0000","last_clean":"2024-04-25T14:05:57.055456+0000","last_became_active":"2024-04-25T14:05:33.572858+0000","last_became_peered":"2024-04-25T14:05:33.572858+0000","last_unstale":"2024-04-25T14:05:57.055456+0000","last_undegraded":"2024-04-25T14:05:57.055456+0000","last_fullsized":"2024-04-25T14:05:57.055456+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:22.996486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":19,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.905336+0000","last_change":"2024-04-25T14:05:33.579278+0000","last_active":"2024-04-25T14:05:42.905336+0000","last_peered":"2024-04-25T14:05:42.905336+0000","last_clean":"2024-04-25T14:05:42.905336+0000","last_became_active":"2024-04-25T14:05:33.578940+0000","last_became_peered":"2024-04-25T14:05:33.578940+0000","last_unstale":"2024-04-25T14:05:42.905336+0000","last_undegraded":"2024-04-25T14:05:42.905336+0000","last_fullsized":"2024-04-25T14:05:42.905336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:01.444836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.181532+0000","last_change":"2024-04-25T14:05:33.560454+0000","last_active":"2024-04-25T14:05:56.181532+0000","last_peered":"2024-04-25T14:05:56.181532+0000","last_clean":"2024-04-25T14:05:56.181532+0000","last_became_active":"2024-04-25T14:05:33.560174+0000","last_became_peered":"2024-04-25T14:05:33.560174+0000","last_unstale":"2024-04-25T14:05:56.181532+0000","last_undegraded":"2024-04-25T14:05:56.181532+0000","last_fullsized":"2024-04-25T14:05:56.181532+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:07.917626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833399+0000","last_change":"2024-04-25T14:05:55.207186+0000","last_active":"2024-04-25T14:05:56.833399+0000","last_peered":"2024-04-25T14:05:56.833399+0000","last_clean":"2024-04-25T14:05:56.833399+0000","last_became_active":"2024-04-25T14:05:55.205535+0000","last_became_peered":"2024-04-25T14:05:55.205535+0000","last_unstale":"2024-04-25T14:05:56.833399+0000","last_undegraded":"2024-04-25T14:05:56.833399+0000","last_fullsized":"2024-04-25T14:05:56.833399+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:48.919549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180448+0000","last_change":"2024-04-25T14:05:33.567123+0000","last_active":"2024-04-25T14:05:56.180448+0000","last_peered":"2024-04-25T14:05:56.180448+0000","last_clean":"2024-04-25T14:05:56.180448+0000","last_became_active":"2024-04-25T14:05:33.563225+0000","last_became_peered":"2024-04-25T14:05:33.563225+0000","last_unstale":"2024-04-25T14:05:56.180448+0000","last_undegraded":"2024-04-25T14:05:56.180448+0000","last_fullsized":"2024-04-25T14:05:56.180448+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:20.253140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.186648+0000","last_change":"2024-04-25T14:05:55.209615+0000","last_active":"2024-04-25T14:05:56.186648+0000","last_peered":"2024-04-25T14:05:56.186648+0000","last_clean":"2024-04-25T14:05:56.186648+0000","last_became_active":"2024-04-25T14:05:55.204910+0000","last_became_peered":"2024-04-25T14:05:55.204910+0000","last_unstale":"2024-04-25T14:05:56.186648+0000","last_undegraded":"2024-04-25T14:05:56.186648+0000","last_fullsized":"2024-04-25T14:05:56.186648+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:25.640028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.832870+0000","last_change":"2024-04-25T14:05:33.588123+0000","last_active":"2024-04-25T14:05:56.832870+0000","last_peered":"2024-04-25T14:05:56.832870+0000","last_clean":"2024-04-25T14:05:56.832870+0000","last_became_active":"2024-04-25T14:05:33.587848+0000","last_became_peered":"2024-04-25T14:05:33.587848+0000","last_unstale":"2024-04-25T14:05:56.832870+0000","last_undegraded":"2024-04-25T14:05:56.832870+0000","last_fullsized":"2024-04-25T14:05:56.832870+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:00.811494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":19,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.904896+0000","last_change":"2024-04-25T14:05:33.573091+0000","last_active":"2024-04-25T14:05:42.904896+0000","last_peered":"2024-04-25T14:05:42.904896+0000","last_clean":"2024-04-25T14:05:42.904896+0000","last_became_active":"2024-04-25T14:05:33.572800+0000","last_became_peered":"2024-04-25T14:05:33.572800+0000","last_unstale":"2024-04-25T14:05:42.904896+0000","last_undegraded":"2024-04-25T14:05:42.904896+0000","last_fullsized":"2024-04-25T14:05:42.904896+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:39.721568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:57.054944+0000","last_change":"2024-04-25T14:05:55.192600+0000","last_active":"2024-04-25T14:05:57.054944+0000","last_peered":"2024-04-25T14:05:57.054944+0000","last_clean":"2024-04-25T14:05:57.054944+0000","last_became_active":"2024-04-25T14:05:55.192221+0000","last_became_peered":"2024-04-25T14:05:55.192221+0000","last_unstale":"2024-04-25T14:05:57.054944+0000","last_undegraded":"2024-04-25T14:05:57.054944+0000","last_fullsized":"2024-04-25T14:05:57.054944+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:03.134903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.911975+0000","last_change":"2024-04-25T14:05:33.575718+0000","last_active":"2024-04-25T14:05:42.911975+0000","last_peered":"2024-04-25T14:05:42.911975+0000","last_clean":"2024-04-25T14:05:42.911975+0000","last_became_active":"2024-04-25T14:05:33.571491+0000","last_became_peered":"2024-04-25T14:05:33.571491+0000","last_unstale":"2024-04-25T14:05:42.911975+0000","last_undegraded":"2024-04-25T14:05:42.911975+0000","last_fullsized":"2024-04-25T14:05:42.911975+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:36.296676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.907077+0000","last_change":"2024-04-25T14:05:33.576013+0000","last_active":"2024-04-25T14:05:42.907077+0000","last_peered":"2024-04-25T14:05:42.907077+0000","last_clean":"2024-04-25T14:05:42.907077+0000","last_became_active":"2024-04-25T14:05:33.575693+0000","last_became_peered":"2024-04-25T14:05:33.575693+0000","last_unstale":"2024-04-25T14:05:42.907077+0000","last_undegraded":"2024-04-25T14:05:42.907077+0000","last_fullsized":"2024-04-25T14:05:42.907077+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:16.126866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833639+0000","last_change":"2024-04-25T14:05:33.584460+0000","last_active":"2024-04-25T14:05:56.833639+0000","last_peered":"2024-04-25T14:05:56.833639+0000","last_clean":"2024-04-25T14:05:56.833639+0000","last_became_active":"2024-04-25T14:05:33.584178+0000","last_became_peered":"2024-04-25T14:05:33.584178+0000","last_unstale":"2024-04-25T14:05:56.833639+0000","last_undegraded":"2024-04-25T14:05:56.833639+0000","last_fullsized":"2024-04-25T14:05:56.833639+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:04.017485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.907671+0000","last_change":"2024-04-25T14:05:33.579643+0000","last_active":"2024-04-25T14:05:42.907671+0000","last_peered":"2024-04-25T14:05:42.907671+0000","last_clean":"2024-04-25T14:05:42.907671+0000","last_became_active":"2024-04-25T14:05:33.579334+0000","last_became_peered":"2024-04-25T14:05:33.579334+0000","last_unstale":"2024-04-25T14:05:42.907671+0000","last_undegraded":"2024-04-25T14:05:42.907671+0000","last_fullsized":"2024-04-25T14:05:42.907671+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:50.417159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T14:05:54.152159+0000","last_change":"2024-04-25T14:05:54.152159+0000","last_active":"2024-04-25T14:05:54.152159+0000","last_peered":"2024-04-25T14:05:54.152159+0000","last_clean":"2024-04-25T14:05:54.152159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:05:54.152159+0000","last_undegraded":"2024-04-25T14:05:54.152159+0000","last_fullsized":"2024-04-25T14:05:54.152159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3f","version":"0'0","reported_seq":19,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T14:05:42.905286+0000","last_change":"2024-04-25T14:05:33.584174+0000","last_active":"2024-04-25T14:05:42.905286+0000","last_peered":"2024-04-25T14:05:42.905286+0000","last_clean":"2024-04-25T14:05:42.905286+0000","last_became_active":"2024-04-25T14:05:33.583867+0000","last_became_peered":"2024-04-25T14:05:33.583867+0000","last_unstale":"2024-04-25T14:05:42.905286+0000","last_undegraded":"2024-04-25T14:05:42.905286+0000","last_fullsized":"2024-04-25T14:05:42.905286+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:36.723420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":54,"acting":54,"num_store_stats":4},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":28,"acting":28,"num_store_stats":4},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":3145728,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":13866,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":82,"num_write":55,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":27732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53,"ondisk_log_size":53,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28184,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343656,"statfs":{"total":96636764160,"available":96607903744,"internally_reserved":0,"allocated":1196032,"data_stored":861703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":655360,"data_stored":303656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":593920,"data_stored":271211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28624,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343216,"statfs":{"total":96636764160,"available":96607453184,"internally_reserved":0,"allocated":1646592,"data_stored":1320369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29728,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342112,"statfs":{"total":96636764160,"available":96606322688,"internally_reserved":0,"allocated":2768896,"data_stored":2413212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":667648,"data_stored":316038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30676,"kb_used_data":3660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341164,"statfs":{"total":96636764160,"available":96605351936,"internally_reserved":0,"allocated":3747840,"data_stored":3417099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28228,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343612,"statfs":{"total":96636764160,"available":96607858688,"internally_reserved":0,"allocated":1232896,"data_stored":885447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":13068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":13090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3145728,"data_stored":3145728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:05:59.330 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:05:59.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.508+0000 7f8be3a00640 1 Processor -- start 2024-04-25T14:05:59.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.508+0000 7f8be3a00640 1 -- start start 2024-04-25T14:05:59.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.508+0000 7f8be3a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8bdc0cfca0 0x7f8bdc06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:59.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.508+0000 7f8be3a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bdc072bd0 con 0x7f8bdc0d16e0 2024-04-25T14:05:59.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.512+0000 7f8be3a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bdc06f930 con 0x7f8bdc0730b0 2024-04-25T14:05:59.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.512+0000 7f8be3a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8bdc070470 con 0x7f8bdc0cfca0 2024-04-25T14:05:59.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.512+0000 7f8be1000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8bdc0cfca0 0x7f8bdc06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:59.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.512+0000 7f8bd3e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8bdc0d16e0 0x7f8bdc0d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50936/0 (socket says 172.21.15.86:50936) 2024-04-25T14:05:59.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.512+0000 7f8bd3e00640 1 -- 172.21.15.86:0/3846213800 learned_addr learned my addr 172.21.15.86:0/3846213800 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:59.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.512+0000 7f8be1000640 1 -- 172.21.15.86:0/3846213800 >> v1:172.21.15.155:6789/0 conn(0x7f8bdc0730b0 legacy=0x7f8bdc0734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:59.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.512+0000 7f8be1000640 1 -- 172.21.15.86:0/3846213800 >> v1:172.21.15.86:6789/0 conn(0x7f8bdc0d16e0 legacy=0x7f8bdc0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:59.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.512+0000 7f8be1000640 1 -- 172.21.15.86:0/3846213800 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bdc06fbd0 con 0x7f8bdc0cfca0 2024-04-25T14:05:59.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.512+0000 7f8bdbe00640 1 -- 172.21.15.86:0/3846213800 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3301037658 0 0) 0x7f8bdc072bd0 con 0x7f8bdc0d16e0 2024-04-25T14:05:59.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.512+0000 7f8be1000640 1 --2- 172.21.15.86:0/3846213800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bdc0cfca0 0x7f8bdc06f3f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc0d22a0 tx=0x7f8bcc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1d2d552ea4574e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:59.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.512+0000 7f8bdbe00640 1 -- 172.21.15.86:0/3846213800 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc0b0070 con 0x7f8bdc0cfca0 2024-04-25T14:05:59.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.512+0000 7f8bdbe00640 1 -- 172.21.15.86:0/3846213800 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bcc0ab600 con 0x7f8bdc0cfca0 2024-04-25T14:05:59.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.512+0000 7f8bdbe00640 1 -- 172.21.15.86:0/3846213800 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc0743e0 con 0x7f8bdc0cfca0 2024-04-25T14:05:59.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.516+0000 7f8be3a00640 1 -- 172.21.15.86:0/3846213800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bdc0cfca0 msgr2=0x7f8bdc06f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:59.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.516+0000 7f8be3a00640 1 --2- 172.21.15.86:0/3846213800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bdc0cfca0 0x7f8bdc06f3f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc0d22a0 tx=0x7f8bcc01c460 comp rx=0 tx=0).stop 2024-04-25T14:05:59.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.516+0000 7f8be3a00640 1 -- 172.21.15.86:0/3846213800 shutdown_connections 2024-04-25T14:05:59.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.516+0000 7f8be3a00640 1 --2- 172.21.15.86:0/3846213800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bdc0cfca0 0x7f8bdc06f3f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:59.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.516+0000 7f8be3a00640 1 -- 172.21.15.86:0/3846213800 >> 172.21.15.86:0/3846213800 conn(0x7f8bdc0b4650 msgr2=0x7f8bdc0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:59.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.516+0000 7f8be3a00640 1 -- 172.21.15.86:0/3846213800 shutdown_connections 2024-04-25T14:05:59.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.516+0000 7f8be3a00640 1 -- 172.21.15.86:0/3846213800 wait complete. 2024-04-25T14:05:59.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.516+0000 7f8be3a00640 1 Processor -- start 2024-04-25T14:05:59.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.516+0000 7f8be3a00640 1 -- start start 2024-04-25T14:05:59.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.520+0000 7f8be3a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bdc0730b0 0x7f8bdc1f9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:59.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.520+0000 7f8be3a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bdc0cfca0 0x7f8bdc0c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:59.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.520+0000 7f8be3a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bdc0d16e0 0x7f8bdc0b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:59.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.520+0000 7f8be3a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc16aa80 con 0x7f8bdc0730b0 2024-04-25T14:05:59.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.520+0000 7f8be3a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc16a900 con 0x7f8bdc0d16e0 2024-04-25T14:05:59.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.520+0000 7f8be3a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8bdc16ac00 con 0x7f8bdc0cfca0 2024-04-25T14:05:59.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.520+0000 7f8be1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bdc0730b0 0x7f8bdc1f9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:59.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.520+0000 7f8be1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bdc0730b0 0x7f8bdc1f9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41436/0 (socket says 172.21.15.86:41436) 2024-04-25T14:05:59.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.520+0000 7f8be1000640 1 -- 172.21.15.86:0/1654739210 learned_addr learned my addr 172.21.15.86:0/1654739210 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:59.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.520+0000 7f8bd3e00640 1 --2- 172.21.15.86:0/1654739210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bdc0cfca0 0x7f8bdc0c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:59.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.520+0000 7f8be1000640 1 -- 172.21.15.86:0/1654739210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bdc0cfca0 msgr2=0x7f8bdc0c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:59.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.520+0000 7f8be1a00640 1 --2- 172.21.15.86:0/1654739210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bdc0d16e0 0x7f8bdc0b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:59.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.520+0000 7f8be1000640 1 --2- 172.21.15.86:0/1654739210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bdc0cfca0 0x7f8bdc0c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:59.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.520+0000 7f8be1000640 1 -- 172.21.15.86:0/1654739210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bdc0d16e0 msgr2=0x7f8bdc0b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:59.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.520+0000 7f8be1000640 1 --2- 172.21.15.86:0/1654739210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bdc0d16e0 0x7f8bdc0b1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:59.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.520+0000 7f8be1000640 1 -- 172.21.15.86:0/1654739210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bdc15b200 con 0x7f8bdc0730b0 2024-04-25T14:05:59.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.520+0000 7f8bd3e00640 1 --2- 172.21.15.86:0/1654739210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bdc0cfca0 0x7f8bdc0c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:59.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.520+0000 7f8be1a00640 1 --2- 172.21.15.86:0/1654739210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bdc0d16e0 0x7f8bdc0b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:59.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.520+0000 7f8be1000640 1 --2- 172.21.15.86:0/1654739210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bdc0730b0 0x7f8bdc1f9500 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc019070 tx=0x7f8bcc01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:59.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.520+0000 7f8bd9600640 1 -- 172.21.15.86:0/1654739210 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc0b0050 con 0x7f8bdc0730b0 2024-04-25T14:05:59.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.520+0000 7f8bd9600640 1 -- 172.21.15.86:0/1654739210 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bcc0bc070 con 0x7f8bdc0730b0 2024-04-25T14:05:59.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.520+0000 7f8bd9600640 1 -- 172.21.15.86:0/1654739210 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc0b5a30 con 0x7f8bdc0730b0 2024-04-25T14:05:59.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.520+0000 7f8be3a00640 1 -- 172.21.15.86:0/1654739210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bdc15b670 con 0x7f8bdc0730b0 2024-04-25T14:05:59.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.520+0000 7f8be3a00640 1 -- 172.21.15.86:0/1654739210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bdc15bc30 con 0x7f8bdc0730b0 2024-04-25T14:05:59.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.520+0000 7f8be3a00640 1 -- 172.21.15.86:0/1654739210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bdc09f4b0 con 0x7f8bdc0730b0 2024-04-25T14:05:59.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.524+0000 7f8bd9600640 1 -- 172.21.15.86:0/1654739210 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f8bcc0ab4c0 con 0x7f8bdc0730b0 2024-04-25T14:05:59.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.524+0000 7f8bd9600640 1 --2- 172.21.15.86:0/1654739210 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f8bb806f570 0x7f8bb8071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:59.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.524+0000 7f8bd9600640 1 -- 172.21.15.86:0/1654739210 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 7359+0+0 (secure 0 0 0) 0x7f8bcc12afe0 con 0x7f8bdc0730b0 2024-04-25T14:05:59.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.524+0000 7f8bd3e00640 1 --2- 172.21.15.86:0/1654739210 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f8bb806f570 0x7f8bb8071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:59.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.524+0000 7f8bd3e00640 1 --2- 172.21.15.86:0/1654739210 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f8bb806f570 0x7f8bb8071a80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc15f620 tx=0x7f8bc4009040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:59.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.528+0000 7f8bd9600640 1 -- 172.21.15.86:0/1654739210 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8bcc0c6020 con 0x7f8bdc0730b0 2024-04-25T14:05:59.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.680+0000 7f8be3a00640 1 -- 172.21.15.86:0/1654739210 --> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8bdc069d00 con 0x7f8bb806f570 2024-04-25T14:05:59.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.692+0000 7f8bd9600640 1 -- 172.21.15.86:0/1654739210 <== mgr.5532 v2:172.21.15.86:6832/2427179681 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+490584 (secure 0 0 0) 0x7f8bdc069d00 con 0x7f8bb806f570 2024-04-25T14:05:59.698 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:05:59.702 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:05:59.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.696+0000 7f8be3a00640 1 -- 172.21.15.86:0/1654739210 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f8bb806f570 msgr2=0x7f8bb8071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:59.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.696+0000 7f8be3a00640 1 --2- 172.21.15.86:0/1654739210 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f8bb806f570 0x7f8bb8071a80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc15f620 tx=0x7f8bc4009040 comp rx=0 tx=0).stop 2024-04-25T14:05:59.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.696+0000 7f8be3a00640 1 -- 172.21.15.86:0/1654739210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bdc0730b0 msgr2=0x7f8bdc1f9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:59.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.696+0000 7f8be3a00640 1 --2- 172.21.15.86:0/1654739210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bdc0730b0 0x7f8bdc1f9500 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc019070 tx=0x7f8bcc01aca0 comp rx=0 tx=0).stop 2024-04-25T14:05:59.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.696+0000 7f8be3a00640 1 -- 172.21.15.86:0/1654739210 shutdown_connections 2024-04-25T14:05:59.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.696+0000 7f8be3a00640 1 --2- 172.21.15.86:0/1654739210 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f8bb806f570 0x7f8bb8071a80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:59.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.696+0000 7f8be3a00640 1 --2- 172.21.15.86:0/1654739210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bdc0d16e0 0x7f8bdc0b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:59.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.696+0000 7f8be3a00640 1 --2- 172.21.15.86:0/1654739210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bdc0cfca0 0x7f8bdc0c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:59.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.696+0000 7f8be3a00640 1 --2- 172.21.15.86:0/1654739210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bdc0730b0 0x7f8bdc1f9500 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:59.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.696+0000 7f8be3a00640 1 -- 172.21.15.86:0/1654739210 >> 172.21.15.86:0/1654739210 conn(0x7f8bdc0b4650 msgr2=0x7f8bdc154f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:59.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.696+0000 7f8be3a00640 1 -- 172.21.15.86:0/1654739210 shutdown_connections 2024-04-25T14:05:59.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.696+0000 7f8be3a00640 1 -- 172.21.15.86:0/1654739210 wait complete. 2024-04-25T14:05:59.719 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-04-25T14:05:59.487477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3752372,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":205,"num_read_kb":313,"num_write":215,"num_write_kb":4744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":174,"ondisk_log_size":174,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":375,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":232804,"kb_used_data":12456,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754741916,"statfs":{"total":773094113280,"available":772855721984,"internally_reserved":0,"allocated":12754944,"data_stored":9912610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7499,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001417"},"pg_stats":[{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.186876+0000","last_change":"2024-04-25T14:05:55.210641+0000","last_active":"2024-04-25T14:05:56.186876+0000","last_peered":"2024-04-25T14:05:56.186876+0000","last_clean":"2024-04-25T14:05:56.186876+0000","last_became_active":"2024-04-25T14:05:55.209998+0000","last_became_peered":"2024-04-25T14:05:55.209998+0000","last_unstale":"2024-04-25T14:05:56.186876+0000","last_undegraded":"2024-04-25T14:05:56.186876+0000","last_fullsized":"2024-04-25T14:05:56.186876+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:44.412223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"39'79","reported_seq":198,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.241405+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:05:56.241405+0000","last_peered":"2024-04-25T14:05:56.241405+0000","last_clean":"2024-04-25T14:05:56.241405+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:05:56.241405+0000","last_undegraded":"2024-04-25T14:05:56.241405+0000","last_fullsized":"2024-04-25T14:05:56.241405+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"50'4","reported_seq":33,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.832585+0000","last_change":"2024-04-25T14:05:32.559617+0000","last_active":"2024-04-25T14:05:56.832585+0000","last_peered":"2024-04-25T14:05:56.832585+0000","last_clean":"2024-04-25T14:05:56.832585+0000","last_became_active":"2024-04-25T14:05:32.558277+0000","last_became_peered":"2024-04-25T14:05:32.558277+0000","last_unstale":"2024-04-25T14:05:56.832585+0000","last_undegraded":"2024-04-25T14:05:56.832585+0000","last_fullsized":"2024-04-25T14:05:56.832585+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:38.715145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":25,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.185922+0000","last_change":"2024-04-25T14:05:33.563768+0000","last_active":"2024-04-25T14:05:56.185922+0000","last_peered":"2024-04-25T14:05:56.185922+0000","last_clean":"2024-04-25T14:05:56.185922+0000","last_became_active":"2024-04-25T14:05:33.563379+0000","last_became_peered":"2024-04-25T14:05:33.563379+0000","last_unstale":"2024-04-25T14:05:56.185922+0000","last_undegraded":"2024-04-25T14:05:56.185922+0000","last_fullsized":"2024-04-25T14:05:56.185922+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:11.087435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180424+0000","last_change":"2024-04-25T14:05:54.189656+0000","last_active":"2024-04-25T14:05:56.180424+0000","last_peered":"2024-04-25T14:05:56.180424+0000","last_clean":"2024-04-25T14:05:56.180424+0000","last_became_active":"2024-04-25T14:05:54.188737+0000","last_became_peered":"2024-04-25T14:05:54.188737+0000","last_unstale":"2024-04-25T14:05:56.180424+0000","last_undegraded":"2024-04-25T14:05:56.180424+0000","last_fullsized":"2024-04-25T14:05:56.180424+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:54.607236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.186225+0000","last_change":"2024-04-25T14:05:55.205021+0000","last_active":"2024-04-25T14:05:56.186225+0000","last_peered":"2024-04-25T14:05:56.186225+0000","last_clean":"2024-04-25T14:05:56.186225+0000","last_became_active":"2024-04-25T14:05:55.204257+0000","last_became_peered":"2024-04-25T14:05:55.204257+0000","last_unstale":"2024-04-25T14:05:56.186225+0000","last_undegraded":"2024-04-25T14:05:56.186225+0000","last_fullsized":"2024-04-25T14:05:56.186225+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:29.540170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":29,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.186015+0000","last_change":"2024-04-25T14:05:32.564178+0000","last_active":"2024-04-25T14:05:56.186015+0000","last_peered":"2024-04-25T14:05:56.186015+0000","last_clean":"2024-04-25T14:05:56.186015+0000","last_became_active":"2024-04-25T14:05:32.563861+0000","last_became_peered":"2024-04-25T14:05:32.563861+0000","last_unstale":"2024-04-25T14:05:56.186015+0000","last_undegraded":"2024-04-25T14:05:56.186015+0000","last_fullsized":"2024-04-25T14:05:56.186015+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:40.505936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.181631+0000","last_change":"2024-04-25T14:05:33.575403+0000","last_active":"2024-04-25T14:05:56.181631+0000","last_peered":"2024-04-25T14:05:56.181631+0000","last_clean":"2024-04-25T14:05:56.181631+0000","last_became_active":"2024-04-25T14:05:33.575194+0000","last_became_peered":"2024-04-25T14:05:33.575194+0000","last_unstale":"2024-04-25T14:05:56.181631+0000","last_undegraded":"2024-04-25T14:05:56.181631+0000","last_fullsized":"2024-04-25T14:05:56.181631+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:44.696081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.242205+0000","last_change":"2024-04-25T14:05:54.191603+0000","last_active":"2024-04-25T14:05:56.242205+0000","last_peered":"2024-04-25T14:05:56.242205+0000","last_clean":"2024-04-25T14:05:56.242205+0000","last_became_active":"2024-04-25T14:05:54.191217+0000","last_became_peered":"2024-04-25T14:05:54.191217+0000","last_unstale":"2024-04-25T14:05:56.242205+0000","last_undegraded":"2024-04-25T14:05:56.242205+0000","last_fullsized":"2024-04-25T14:05:56.242205+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:43.404140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.241900+0000","last_change":"2024-04-25T14:05:55.190611+0000","last_active":"2024-04-25T14:05:56.241900+0000","last_peered":"2024-04-25T14:05:56.241900+0000","last_clean":"2024-04-25T14:05:56.241900+0000","last_became_active":"2024-04-25T14:05:55.189282+0000","last_became_peered":"2024-04-25T14:05:55.189282+0000","last_unstale":"2024-04-25T14:05:56.241900+0000","last_undegraded":"2024-04-25T14:05:56.241900+0000","last_fullsized":"2024-04-25T14:05:56.241900+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:57:03.417575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.880418+0000","last_change":"2024-04-25T14:05:32.543546+0000","last_active":"2024-04-25T14:05:56.880418+0000","last_peered":"2024-04-25T14:05:56.880418+0000","last_clean":"2024-04-25T14:05:56.880418+0000","last_became_active":"2024-04-25T14:05:32.543162+0000","last_became_peered":"2024-04-25T14:05:32.543162+0000","last_unstale":"2024-04-25T14:05:56.880418+0000","last_undegraded":"2024-04-25T14:05:56.880418+0000","last_fullsized":"2024-04-25T14:05:56.880418+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:14.012822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833857+0000","last_change":"2024-04-25T14:05:33.573521+0000","last_active":"2024-04-25T14:05:56.833857+0000","last_peered":"2024-04-25T14:05:56.833857+0000","last_clean":"2024-04-25T14:05:56.833857+0000","last_became_active":"2024-04-25T14:05:33.573181+0000","last_became_peered":"2024-04-25T14:05:33.573181+0000","last_unstale":"2024-04-25T14:05:56.833857+0000","last_undegraded":"2024-04-25T14:05:56.833857+0000","last_fullsized":"2024-04-25T14:05:56.833857+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:33.035476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.188370+0000","last_change":"2024-04-25T14:05:54.183530+0000","last_active":"2024-04-25T14:05:56.188370+0000","last_peered":"2024-04-25T14:05:56.188370+0000","last_clean":"2024-04-25T14:05:56.188370+0000","last_became_active":"2024-04-25T14:05:54.183138+0000","last_became_peered":"2024-04-25T14:05:54.183138+0000","last_unstale":"2024-04-25T14:05:56.188370+0000","last_undegraded":"2024-04-25T14:05:56.188370+0000","last_fullsized":"2024-04-25T14:05:56.188370+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:50.855416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.187009+0000","last_change":"2024-04-25T14:05:55.211098+0000","last_active":"2024-04-25T14:05:56.187009+0000","last_peered":"2024-04-25T14:05:56.187009+0000","last_clean":"2024-04-25T14:05:56.187009+0000","last_became_active":"2024-04-25T14:05:55.210732+0000","last_became_peered":"2024-04-25T14:05:55.210732+0000","last_unstale":"2024-04-25T14:05:56.187009+0000","last_undegraded":"2024-04-25T14:05:56.187009+0000","last_fullsized":"2024-04-25T14:05:56.187009+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:50.046862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":29,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180900+0000","last_change":"2024-04-25T14:05:32.555662+0000","last_active":"2024-04-25T14:05:56.180900+0000","last_peered":"2024-04-25T14:05:56.180900+0000","last_clean":"2024-04-25T14:05:56.180900+0000","last_became_active":"2024-04-25T14:05:32.555123+0000","last_became_peered":"2024-04-25T14:05:32.555123+0000","last_unstale":"2024-04-25T14:05:56.180900+0000","last_undegraded":"2024-04-25T14:05:56.180900+0000","last_fullsized":"2024-04-25T14:05:56.180900+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:24.071915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.188294+0000","last_change":"2024-04-25T14:05:33.570070+0000","last_active":"2024-04-25T14:05:56.188294+0000","last_peered":"2024-04-25T14:05:56.188294+0000","last_clean":"2024-04-25T14:05:56.188294+0000","last_became_active":"2024-04-25T14:05:33.569716+0000","last_became_peered":"2024-04-25T14:05:33.569716+0000","last_unstale":"2024-04-25T14:05:56.188294+0000","last_undegraded":"2024-04-25T14:05:56.188294+0000","last_fullsized":"2024-04-25T14:05:56.188294+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:42.654848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.181210+0000","last_change":"2024-04-25T14:05:54.181106+0000","last_active":"2024-04-25T14:05:56.181210+0000","last_peered":"2024-04-25T14:05:56.181210+0000","last_clean":"2024-04-25T14:05:56.181210+0000","last_became_active":"2024-04-25T14:05:54.180829+0000","last_became_peered":"2024-04-25T14:05:54.180829+0000","last_unstale":"2024-04-25T14:05:56.181210+0000","last_undegraded":"2024-04-25T14:05:56.181210+0000","last_fullsized":"2024-04-25T14:05:56.181210+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:25.927992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.881885+0000","last_change":"2024-04-25T14:05:55.207031+0000","last_active":"2024-04-25T14:05:56.881885+0000","last_peered":"2024-04-25T14:05:56.881885+0000","last_clean":"2024-04-25T14:05:56.881885+0000","last_became_active":"2024-04-25T14:05:55.204726+0000","last_became_peered":"2024-04-25T14:05:55.204726+0000","last_unstale":"2024-04-25T14:05:56.881885+0000","last_undegraded":"2024-04-25T14:05:56.881885+0000","last_fullsized":"2024-04-25T14:05:56.881885+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:34.622772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":29,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.181185+0000","last_change":"2024-04-25T14:05:32.555975+0000","last_active":"2024-04-25T14:05:56.181185+0000","last_peered":"2024-04-25T14:05:56.181185+0000","last_clean":"2024-04-25T14:05:56.181185+0000","last_became_active":"2024-04-25T14:05:32.555188+0000","last_became_peered":"2024-04-25T14:05:32.555188+0000","last_unstale":"2024-04-25T14:05:56.181185+0000","last_undegraded":"2024-04-25T14:05:56.181185+0000","last_fullsized":"2024-04-25T14:05:56.181185+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:10.274276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833504+0000","last_change":"2024-04-25T14:05:33.561273+0000","last_active":"2024-04-25T14:05:56.833504+0000","last_peered":"2024-04-25T14:05:56.833504+0000","last_clean":"2024-04-25T14:05:56.833504+0000","last_became_active":"2024-04-25T14:05:33.560969+0000","last_became_peered":"2024-04-25T14:05:33.560969+0000","last_unstale":"2024-04-25T14:05:56.833504+0000","last_undegraded":"2024-04-25T14:05:56.833504+0000","last_fullsized":"2024-04-25T14:05:56.833504+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:44.696081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.241901+0000","last_change":"2024-04-25T14:05:54.194358+0000","last_active":"2024-04-25T14:05:56.241901+0000","last_peered":"2024-04-25T14:05:56.241901+0000","last_clean":"2024-04-25T14:05:56.241901+0000","last_became_active":"2024-04-25T14:05:54.193510+0000","last_became_peered":"2024-04-25T14:05:54.193510+0000","last_unstale":"2024-04-25T14:05:56.241901+0000","last_undegraded":"2024-04-25T14:05:56.241901+0000","last_fullsized":"2024-04-25T14:05:56.241901+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:53.649937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.185751+0000","last_change":"2024-04-25T14:05:55.201745+0000","last_active":"2024-04-25T14:05:56.185751+0000","last_peered":"2024-04-25T14:05:56.185751+0000","last_clean":"2024-04-25T14:05:56.185751+0000","last_became_active":"2024-04-25T14:05:55.197144+0000","last_became_peered":"2024-04-25T14:05:55.197144+0000","last_unstale":"2024-04-25T14:05:56.185751+0000","last_undegraded":"2024-04-25T14:05:56.185751+0000","last_fullsized":"2024-04-25T14:05:56.185751+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:24.264233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":28,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833507+0000","last_change":"2024-04-25T14:05:32.558770+0000","last_active":"2024-04-25T14:05:56.833507+0000","last_peered":"2024-04-25T14:05:56.833507+0000","last_clean":"2024-04-25T14:05:56.833507+0000","last_became_active":"2024-04-25T14:05:32.557820+0000","last_became_peered":"2024-04-25T14:05:32.557820+0000","last_unstale":"2024-04-25T14:05:56.833507+0000","last_undegraded":"2024-04-25T14:05:56.833507+0000","last_fullsized":"2024-04-25T14:05:56.833507+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:10.274276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.181395+0000","last_change":"2024-04-25T14:05:33.573700+0000","last_active":"2024-04-25T14:05:56.181395+0000","last_peered":"2024-04-25T14:05:56.181395+0000","last_clean":"2024-04-25T14:05:56.181395+0000","last_became_active":"2024-04-25T14:05:33.573414+0000","last_became_peered":"2024-04-25T14:05:33.573414+0000","last_unstale":"2024-04-25T14:05:56.181395+0000","last_undegraded":"2024-04-25T14:05:56.181395+0000","last_fullsized":"2024-04-25T14:05:56.181395+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:30.202222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.187294+0000","last_change":"2024-04-25T14:05:54.189249+0000","last_active":"2024-04-25T14:05:56.187294+0000","last_peered":"2024-04-25T14:05:56.187294+0000","last_clean":"2024-04-25T14:05:56.187294+0000","last_became_active":"2024-04-25T14:05:54.184099+0000","last_became_peered":"2024-04-25T14:05:54.184099+0000","last_unstale":"2024-04-25T14:05:56.187294+0000","last_undegraded":"2024-04-25T14:05:56.187294+0000","last_fullsized":"2024-04-25T14:05:56.187294+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:12.199930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833001+0000","last_change":"2024-04-25T14:05:55.208062+0000","last_active":"2024-04-25T14:05:56.833001+0000","last_peered":"2024-04-25T14:05:56.833001+0000","last_clean":"2024-04-25T14:05:56.833001+0000","last_became_active":"2024-04-25T14:05:55.206431+0000","last_became_peered":"2024-04-25T14:05:55.206431+0000","last_unstale":"2024-04-25T14:05:56.833001+0000","last_undegraded":"2024-04-25T14:05:56.833001+0000","last_fullsized":"2024-04-25T14:05:56.833001+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:38.575335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":28,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.242389+0000","last_change":"2024-04-25T14:05:32.562230+0000","last_active":"2024-04-25T14:05:56.242389+0000","last_peered":"2024-04-25T14:05:56.242389+0000","last_clean":"2024-04-25T14:05:56.242389+0000","last_became_active":"2024-04-25T14:05:32.561768+0000","last_became_peered":"2024-04-25T14:05:32.561768+0000","last_unstale":"2024-04-25T14:05:56.242389+0000","last_undegraded":"2024-04-25T14:05:56.242389+0000","last_fullsized":"2024-04-25T14:05:56.242389+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:40.505936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.312917+0000","last_change":"2024-04-25T14:05:33.574199+0000","last_active":"2024-04-25T14:05:56.312917+0000","last_peered":"2024-04-25T14:05:56.312917+0000","last_clean":"2024-04-25T14:05:56.312917+0000","last_became_active":"2024-04-25T14:05:33.572436+0000","last_became_peered":"2024-04-25T14:05:33.572436+0000","last_unstale":"2024-04-25T14:05:56.312917+0000","last_undegraded":"2024-04-25T14:05:56.312917+0000","last_fullsized":"2024-04-25T14:05:56.312917+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:59:41.873995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833230+0000","last_change":"2024-04-25T14:05:54.190851+0000","last_active":"2024-04-25T14:05:56.833230+0000","last_peered":"2024-04-25T14:05:56.833230+0000","last_clean":"2024-04-25T14:05:56.833230+0000","last_became_active":"2024-04-25T14:05:54.190478+0000","last_became_peered":"2024-04-25T14:05:54.190478+0000","last_unstale":"2024-04-25T14:05:56.833230+0000","last_undegraded":"2024-04-25T14:05:56.833230+0000","last_fullsized":"2024-04-25T14:05:56.833230+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:52.895016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.186745+0000","last_change":"2024-04-25T14:05:55.211644+0000","last_active":"2024-04-25T14:05:56.186745+0000","last_peered":"2024-04-25T14:05:56.186745+0000","last_clean":"2024-04-25T14:05:56.186745+0000","last_became_active":"2024-04-25T14:05:55.210906+0000","last_became_peered":"2024-04-25T14:05:55.210906+0000","last_unstale":"2024-04-25T14:05:56.186745+0000","last_undegraded":"2024-04-25T14:05:56.186745+0000","last_fullsized":"2024-04-25T14:05:56.186745+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:54.658020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.7","version":"45'2","reported_seq":31,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.187436+0000","last_change":"2024-04-25T14:05:32.561754+0000","last_active":"2024-04-25T14:05:56.187436+0000","last_peered":"2024-04-25T14:05:56.187436+0000","last_clean":"2024-04-25T14:05:56.187436+0000","last_became_active":"2024-04-25T14:05:32.561297+0000","last_became_peered":"2024-04-25T14:05:32.561297+0000","last_unstale":"2024-04-25T14:05:56.187436+0000","last_undegraded":"2024-04-25T14:05:56.187436+0000","last_fullsized":"2024-04-25T14:05:56.187436+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:39.285483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.185900+0000","last_change":"2024-04-25T14:05:33.561759+0000","last_active":"2024-04-25T14:05:56.185900+0000","last_peered":"2024-04-25T14:05:56.185900+0000","last_clean":"2024-04-25T14:05:56.185900+0000","last_became_active":"2024-04-25T14:05:33.561430+0000","last_became_peered":"2024-04-25T14:05:33.561430+0000","last_unstale":"2024-04-25T14:05:56.185900+0000","last_undegraded":"2024-04-25T14:05:56.185900+0000","last_fullsized":"2024-04-25T14:05:56.185900+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:16.126866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.832766+0000","last_change":"2024-04-25T14:05:54.184027+0000","last_active":"2024-04-25T14:05:56.832766+0000","last_peered":"2024-04-25T14:05:56.832766+0000","last_clean":"2024-04-25T14:05:56.832766+0000","last_became_active":"2024-04-25T14:05:54.183683+0000","last_became_peered":"2024-04-25T14:05:54.183683+0000","last_unstale":"2024-04-25T14:05:56.832766+0000","last_undegraded":"2024-04-25T14:05:56.832766+0000","last_fullsized":"2024-04-25T14:05:56.832766+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:25.927992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:57.055590+0000","last_change":"2024-04-25T14:05:55.199570+0000","last_active":"2024-04-25T14:05:57.055590+0000","last_peered":"2024-04-25T14:05:57.055590+0000","last_clean":"2024-04-25T14:05:57.055590+0000","last_became_active":"2024-04-25T14:05:55.199153+0000","last_became_peered":"2024-04-25T14:05:55.199153+0000","last_unstale":"2024-04-25T14:05:57.055590+0000","last_undegraded":"2024-04-25T14:05:57.055590+0000","last_fullsized":"2024-04-25T14:05:57.055590+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:49.541001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":28,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:57.054422+0000","last_change":"2024-04-25T14:05:32.553201+0000","last_active":"2024-04-25T14:05:57.054422+0000","last_peered":"2024-04-25T14:05:57.054422+0000","last_clean":"2024-04-25T14:05:57.054422+0000","last_became_active":"2024-04-25T14:05:32.552846+0000","last_became_peered":"2024-04-25T14:05:32.552846+0000","last_unstale":"2024-04-25T14:05:57.054422+0000","last_undegraded":"2024-04-25T14:05:57.054422+0000","last_fullsized":"2024-04-25T14:05:57.054422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:13.772830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"50'1","reported_seq":28,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.313152+0000","last_change":"2024-04-25T14:05:33.579278+0000","last_active":"2024-04-25T14:05:56.313152+0000","last_peered":"2024-04-25T14:05:56.313152+0000","last_clean":"2024-04-25T14:05:56.313152+0000","last_became_active":"2024-04-25T14:05:33.578876+0000","last_became_peered":"2024-04-25T14:05:33.578876+0000","last_unstale":"2024-04-25T14:05:56.313152+0000","last_undegraded":"2024-04-25T14:05:56.313152+0000","last_fullsized":"2024-04-25T14:05:56.313152+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:54.564632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.188556+0000","last_change":"2024-04-25T14:05:54.191606+0000","last_active":"2024-04-25T14:05:56.188556+0000","last_peered":"2024-04-25T14:05:56.188556+0000","last_clean":"2024-04-25T14:05:56.188556+0000","last_became_active":"2024-04-25T14:05:54.191303+0000","last_became_peered":"2024-04-25T14:05:54.191303+0000","last_unstale":"2024-04-25T14:05:56.188556+0000","last_undegraded":"2024-04-25T14:05:56.188556+0000","last_fullsized":"2024-04-25T14:05:56.188556+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:24.631945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833222+0000","last_change":"2024-04-25T14:05:55.205672+0000","last_active":"2024-04-25T14:05:56.833222+0000","last_peered":"2024-04-25T14:05:56.833222+0000","last_clean":"2024-04-25T14:05:56.833222+0000","last_became_active":"2024-04-25T14:05:55.205254+0000","last_became_peered":"2024-04-25T14:05:55.205254+0000","last_unstale":"2024-04-25T14:05:56.833222+0000","last_undegraded":"2024-04-25T14:05:56.833222+0000","last_fullsized":"2024-04-25T14:05:56.833222+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:37.389511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":28,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.832556+0000","last_change":"2024-04-25T14:05:32.563774+0000","last_active":"2024-04-25T14:05:56.832556+0000","last_peered":"2024-04-25T14:05:56.832556+0000","last_clean":"2024-04-25T14:05:56.832556+0000","last_became_active":"2024-04-25T14:05:32.562877+0000","last_became_peered":"2024-04-25T14:05:32.562877+0000","last_unstale":"2024-04-25T14:05:56.832556+0000","last_undegraded":"2024-04-25T14:05:56.832556+0000","last_fullsized":"2024-04-25T14:05:56.832556+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:24.071915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833925+0000","last_change":"2024-04-25T14:05:33.579664+0000","last_active":"2024-04-25T14:05:56.833925+0000","last_peered":"2024-04-25T14:05:56.833925+0000","last_clean":"2024-04-25T14:05:56.833925+0000","last_became_active":"2024-04-25T14:05:33.578924+0000","last_became_peered":"2024-04-25T14:05:33.578924+0000","last_unstale":"2024-04-25T14:05:56.833925+0000","last_undegraded":"2024-04-25T14:05:56.833925+0000","last_fullsized":"2024-04-25T14:05:56.833925+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:13.000228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.188516+0000","last_change":"2024-04-25T14:05:54.188062+0000","last_active":"2024-04-25T14:05:56.188516+0000","last_peered":"2024-04-25T14:05:56.188516+0000","last_clean":"2024-04-25T14:05:56.188516+0000","last_became_active":"2024-04-25T14:05:54.187724+0000","last_became_peered":"2024-04-25T14:05:54.187724+0000","last_unstale":"2024-04-25T14:05:56.188516+0000","last_undegraded":"2024-04-25T14:05:56.188516+0000","last_fullsized":"2024-04-25T14:05:56.188516+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:28.532087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.187348+0000","last_change":"2024-04-25T14:05:55.209087+0000","last_active":"2024-04-25T14:05:56.187348+0000","last_peered":"2024-04-25T14:05:56.187348+0000","last_clean":"2024-04-25T14:05:56.187348+0000","last_became_active":"2024-04-25T14:05:55.208771+0000","last_became_peered":"2024-04-25T14:05:55.208771+0000","last_unstale":"2024-04-25T14:05:56.187348+0000","last_undegraded":"2024-04-25T14:05:56.187348+0000","last_fullsized":"2024-04-25T14:05:56.187348+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:13.373855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":29,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180786+0000","last_change":"2024-04-25T14:05:32.555416+0000","last_active":"2024-04-25T14:05:56.180786+0000","last_peered":"2024-04-25T14:05:56.180786+0000","last_clean":"2024-04-25T14:05:56.180786+0000","last_became_active":"2024-04-25T14:05:32.554652+0000","last_became_peered":"2024-04-25T14:05:32.554652+0000","last_unstale":"2024-04-25T14:05:56.180786+0000","last_undegraded":"2024-04-25T14:05:56.180786+0000","last_fullsized":"2024-04-25T14:05:56.180786+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:40.505936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":25,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.187288+0000","last_change":"2024-04-25T14:05:33.570587+0000","last_active":"2024-04-25T14:05:56.187288+0000","last_peered":"2024-04-25T14:05:56.187288+0000","last_clean":"2024-04-25T14:05:56.187288+0000","last_became_active":"2024-04-25T14:05:33.570209+0000","last_became_peered":"2024-04-25T14:05:33.570209+0000","last_unstale":"2024-04-25T14:05:56.187288+0000","last_undegraded":"2024-04-25T14:05:56.187288+0000","last_fullsized":"2024-04-25T14:05:56.187288+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:13.979622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.312966+0000","last_change":"2024-04-25T14:05:54.187130+0000","last_active":"2024-04-25T14:05:56.312966+0000","last_peered":"2024-04-25T14:05:56.312966+0000","last_clean":"2024-04-25T14:05:56.312966+0000","last_became_active":"2024-04-25T14:05:54.186628+0000","last_became_peered":"2024-04-25T14:05:54.186628+0000","last_unstale":"2024-04-25T14:05:56.312966+0000","last_undegraded":"2024-04-25T14:05:56.312966+0000","last_fullsized":"2024-04-25T14:05:56.312966+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:26.752722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.188315+0000","last_change":"2024-04-25T14:05:55.203312+0000","last_active":"2024-04-25T14:05:56.188315+0000","last_peered":"2024-04-25T14:05:56.188315+0000","last_clean":"2024-04-25T14:05:56.188315+0000","last_became_active":"2024-04-25T14:05:55.203018+0000","last_became_peered":"2024-04-25T14:05:55.203018+0000","last_unstale":"2024-04-25T14:05:56.188315+0000","last_undegraded":"2024-04-25T14:05:56.188315+0000","last_fullsized":"2024-04-25T14:05:56.188315+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:53.903099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"50'3","reported_seq":33,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.312748+0000","last_change":"2024-04-25T14:05:32.555817+0000","last_active":"2024-04-25T14:05:56.312748+0000","last_peered":"2024-04-25T14:05:56.312748+0000","last_clean":"2024-04-25T14:05:56.312748+0000","last_became_active":"2024-04-25T14:05:32.553958+0000","last_became_peered":"2024-04-25T14:05:32.553958+0000","last_unstale":"2024-04-25T14:05:56.312748+0000","last_undegraded":"2024-04-25T14:05:56.312748+0000","last_fullsized":"2024-04-25T14:05:56.312748+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:43.267251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.188327+0000","last_change":"2024-04-25T14:05:33.571493+0000","last_active":"2024-04-25T14:05:56.188327+0000","last_peered":"2024-04-25T14:05:56.188327+0000","last_clean":"2024-04-25T14:05:56.188327+0000","last_became_active":"2024-04-25T14:05:33.568719+0000","last_became_peered":"2024-04-25T14:05:33.568719+0000","last_unstale":"2024-04-25T14:05:56.188327+0000","last_undegraded":"2024-04-25T14:05:56.188327+0000","last_fullsized":"2024-04-25T14:05:56.188327+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:18.567327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833569+0000","last_change":"2024-04-25T14:05:54.189553+0000","last_active":"2024-04-25T14:05:56.833569+0000","last_peered":"2024-04-25T14:05:56.833569+0000","last_clean":"2024-04-25T14:05:56.833569+0000","last_became_active":"2024-04-25T14:05:54.182191+0000","last_became_peered":"2024-04-25T14:05:54.182191+0000","last_unstale":"2024-04-25T14:05:56.833569+0000","last_undegraded":"2024-04-25T14:05:56.833569+0000","last_fullsized":"2024-04-25T14:05:56.833569+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:12.135098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.880809+0000","last_change":"2024-04-25T14:05:55.185045+0000","last_active":"2024-04-25T14:05:56.880809+0000","last_peered":"2024-04-25T14:05:56.880809+0000","last_clean":"2024-04-25T14:05:56.880809+0000","last_became_active":"2024-04-25T14:05:55.184003+0000","last_became_peered":"2024-04-25T14:05:55.184003+0000","last_unstale":"2024-04-25T14:05:56.880809+0000","last_undegraded":"2024-04-25T14:05:56.880809+0000","last_fullsized":"2024-04-25T14:05:56.880809+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:09.440953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.187031+0000","last_change":"2024-04-25T14:05:32.567474+0000","last_active":"2024-04-25T14:05:56.187031+0000","last_peered":"2024-04-25T14:05:56.187031+0000","last_clean":"2024-04-25T14:05:56.187031+0000","last_became_active":"2024-04-25T14:05:32.567061+0000","last_became_peered":"2024-04-25T14:05:32.567061+0000","last_unstale":"2024-04-25T14:05:56.187031+0000","last_undegraded":"2024-04-25T14:05:56.187031+0000","last_fullsized":"2024-04-25T14:05:56.187031+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:20.281168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.312590+0000","last_change":"2024-04-25T14:05:33.572207+0000","last_active":"2024-04-25T14:05:56.312590+0000","last_peered":"2024-04-25T14:05:56.312590+0000","last_clean":"2024-04-25T14:05:56.312590+0000","last_became_active":"2024-04-25T14:05:33.570800+0000","last_became_peered":"2024-04-25T14:05:33.570800+0000","last_unstale":"2024-04-25T14:05:56.312590+0000","last_undegraded":"2024-04-25T14:05:56.312590+0000","last_fullsized":"2024-04-25T14:05:56.312590+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:25.659996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180722+0000","last_change":"2024-04-25T14:05:54.173170+0000","last_active":"2024-04-25T14:05:56.180722+0000","last_peered":"2024-04-25T14:05:56.180722+0000","last_clean":"2024-04-25T14:05:56.180722+0000","last_became_active":"2024-04-25T14:05:54.172825+0000","last_became_peered":"2024-04-25T14:05:54.172825+0000","last_unstale":"2024-04-25T14:05:56.180722+0000","last_undegraded":"2024-04-25T14:05:56.180722+0000","last_fullsized":"2024-04-25T14:05:56.180722+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:11.757925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.185842+0000","last_change":"2024-04-25T14:05:55.210220+0000","last_active":"2024-04-25T14:05:56.185842+0000","last_peered":"2024-04-25T14:05:56.185842+0000","last_clean":"2024-04-25T14:05:56.185842+0000","last_became_active":"2024-04-25T14:05:55.209839+0000","last_became_peered":"2024-04-25T14:05:55.209839+0000","last_unstale":"2024-04-25T14:05:56.185842+0000","last_undegraded":"2024-04-25T14:05:56.185842+0000","last_fullsized":"2024-04-25T14:05:56.185842+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:09.440953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.d","version":"45'1","reported_seq":29,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.187454+0000","last_change":"2024-04-25T14:05:32.566219+0000","last_active":"2024-04-25T14:05:56.187454+0000","last_peered":"2024-04-25T14:05:56.187454+0000","last_clean":"2024-04-25T14:05:56.187454+0000","last_became_active":"2024-04-25T14:05:32.565859+0000","last_became_peered":"2024-04-25T14:05:32.565859+0000","last_unstale":"2024-04-25T14:05:56.187454+0000","last_undegraded":"2024-04-25T14:05:56.187454+0000","last_fullsized":"2024-04-25T14:05:56.187454+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:35.649467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":25,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.880769+0000","last_change":"2024-04-25T14:05:33.582498+0000","last_active":"2024-04-25T14:05:56.880769+0000","last_peered":"2024-04-25T14:05:56.880769+0000","last_clean":"2024-04-25T14:05:56.880769+0000","last_became_active":"2024-04-25T14:05:33.582156+0000","last_became_peered":"2024-04-25T14:05:33.582156+0000","last_unstale":"2024-04-25T14:05:56.880769+0000","last_undegraded":"2024-04-25T14:05:56.880769+0000","last_fullsized":"2024-04-25T14:05:56.880769+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:13.979622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.241813+0000","last_change":"2024-04-25T14:05:54.191868+0000","last_active":"2024-04-25T14:05:56.241813+0000","last_peered":"2024-04-25T14:05:56.241813+0000","last_clean":"2024-04-25T14:05:56.241813+0000","last_became_active":"2024-04-25T14:05:54.190618+0000","last_became_peered":"2024-04-25T14:05:54.190618+0000","last_unstale":"2024-04-25T14:05:56.241813+0000","last_undegraded":"2024-04-25T14:05:56.241813+0000","last_fullsized":"2024-04-25T14:05:56.241813+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:49.038778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:57.055206+0000","last_change":"2024-04-25T14:05:55.192954+0000","last_active":"2024-04-25T14:05:57.055206+0000","last_peered":"2024-04-25T14:05:57.055206+0000","last_clean":"2024-04-25T14:05:57.055206+0000","last_became_active":"2024-04-25T14:05:55.192613+0000","last_became_peered":"2024-04-25T14:05:55.192613+0000","last_unstale":"2024-04-25T14:05:57.055206+0000","last_undegraded":"2024-04-25T14:05:57.055206+0000","last_fullsized":"2024-04-25T14:05:57.055206+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:46.700882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"45'2","reported_seq":32,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:57.054616+0000","last_change":"2024-04-25T14:05:32.559217+0000","last_active":"2024-04-25T14:05:57.054616+0000","last_peered":"2024-04-25T14:05:57.054616+0000","last_clean":"2024-04-25T14:05:57.054616+0000","last_became_active":"2024-04-25T14:05:32.558909+0000","last_became_peered":"2024-04-25T14:05:32.558909+0000","last_unstale":"2024-04-25T14:05:57.054616+0000","last_undegraded":"2024-04-25T14:05:57.054616+0000","last_fullsized":"2024-04-25T14:05:57.054616+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:48:44.213850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.312280+0000","last_change":"2024-04-25T14:05:33.590347+0000","last_active":"2024-04-25T14:05:56.312280+0000","last_peered":"2024-04-25T14:05:56.312280+0000","last_clean":"2024-04-25T14:05:56.312280+0000","last_became_active":"2024-04-25T14:05:33.590073+0000","last_became_peered":"2024-04-25T14:05:33.590073+0000","last_unstale":"2024-04-25T14:05:56.312280+0000","last_undegraded":"2024-04-25T14:05:56.312280+0000","last_fullsized":"2024-04-25T14:05:56.312280+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:11.280699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.312238+0000","last_change":"2024-04-25T14:05:54.188951+0000","last_active":"2024-04-25T14:05:56.312238+0000","last_peered":"2024-04-25T14:05:56.312238+0000","last_clean":"2024-04-25T14:05:56.312238+0000","last_became_active":"2024-04-25T14:05:54.185442+0000","last_became_peered":"2024-04-25T14:05:54.185442+0000","last_unstale":"2024-04-25T14:05:56.312238+0000","last_undegraded":"2024-04-25T14:05:56.312238+0000","last_fullsized":"2024-04-25T14:05:56.312238+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:50.816682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.312329+0000","last_change":"2024-04-25T14:05:55.193872+0000","last_active":"2024-04-25T14:05:56.312329+0000","last_peered":"2024-04-25T14:05:56.312329+0000","last_clean":"2024-04-25T14:05:56.312329+0000","last_became_active":"2024-04-25T14:05:55.193648+0000","last_became_peered":"2024-04-25T14:05:55.193648+0000","last_unstale":"2024-04-25T14:05:56.312329+0000","last_undegraded":"2024-04-25T14:05:56.312329+0000","last_fullsized":"2024-04-25T14:05:56.312329+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:13.373855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"45'1","reported_seq":30,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.832610+0000","last_change":"2024-04-25T14:05:32.569447+0000","last_active":"2024-04-25T14:05:56.832610+0000","last_peered":"2024-04-25T14:05:56.832610+0000","last_clean":"2024-04-25T14:05:56.832610+0000","last_became_active":"2024-04-25T14:05:32.568861+0000","last_became_peered":"2024-04-25T14:05:32.568861+0000","last_unstale":"2024-04-25T14:05:56.832610+0000","last_undegraded":"2024-04-25T14:05:56.832610+0000","last_fullsized":"2024-04-25T14:05:56.832610+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:00.438414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180486+0000","last_change":"2024-04-25T14:05:33.568454+0000","last_active":"2024-04-25T14:05:56.180486+0000","last_peered":"2024-04-25T14:05:56.180486+0000","last_clean":"2024-04-25T14:05:56.180486+0000","last_became_active":"2024-04-25T14:05:33.568140+0000","last_became_peered":"2024-04-25T14:05:33.568140+0000","last_unstale":"2024-04-25T14:05:56.180486+0000","last_undegraded":"2024-04-25T14:05:56.180486+0000","last_fullsized":"2024-04-25T14:05:56.180486+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:09.011115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180568+0000","last_change":"2024-04-25T14:05:54.190640+0000","last_active":"2024-04-25T14:05:56.180568+0000","last_peered":"2024-04-25T14:05:56.180568+0000","last_clean":"2024-04-25T14:05:56.180568+0000","last_became_active":"2024-04-25T14:05:54.189911+0000","last_became_peered":"2024-04-25T14:05:54.189911+0000","last_unstale":"2024-04-25T14:05:56.180568+0000","last_undegraded":"2024-04-25T14:05:56.180568+0000","last_fullsized":"2024-04-25T14:05:56.180568+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:43:53.870402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.881650+0000","last_change":"2024-04-25T14:05:55.205355+0000","last_active":"2024-04-25T14:05:56.881650+0000","last_peered":"2024-04-25T14:05:56.881650+0000","last_clean":"2024-04-25T14:05:56.881650+0000","last_became_active":"2024-04-25T14:05:55.204838+0000","last_became_peered":"2024-04-25T14:05:55.204838+0000","last_unstale":"2024-04-25T14:05:56.881650+0000","last_undegraded":"2024-04-25T14:05:56.881650+0000","last_fullsized":"2024-04-25T14:05:56.881650+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:06.318625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833842+0000","last_change":"2024-04-25T14:05:54.184175+0000","last_active":"2024-04-25T14:05:56.833842+0000","last_peered":"2024-04-25T14:05:56.833842+0000","last_clean":"2024-04-25T14:05:56.833842+0000","last_became_active":"2024-04-25T14:05:54.183867+0000","last_became_peered":"2024-04-25T14:05:54.183867+0000","last_unstale":"2024-04-25T14:05:56.833842+0000","last_undegraded":"2024-04-25T14:05:56.833842+0000","last_fullsized":"2024-04-25T14:05:56.833842+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:13.060977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"45'8","reported_seq":45,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180589+0000","last_change":"2024-04-25T14:05:32.554990+0000","last_active":"2024-04-25T14:05:56.180589+0000","last_peered":"2024-04-25T14:05:56.180589+0000","last_clean":"2024-04-25T14:05:56.180589+0000","last_became_active":"2024-04-25T14:05:32.554580+0000","last_became_peered":"2024-04-25T14:05:32.554580+0000","last_unstale":"2024-04-25T14:05:56.180589+0000","last_undegraded":"2024-04-25T14:05:56.180589+0000","last_fullsized":"2024-04-25T14:05:56.180589+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:26.912034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833753+0000","last_change":"2024-04-25T14:05:33.566385+0000","last_active":"2024-04-25T14:05:56.833753+0000","last_peered":"2024-04-25T14:05:56.833753+0000","last_clean":"2024-04-25T14:05:56.833753+0000","last_became_active":"2024-04-25T14:05:33.566106+0000","last_became_peered":"2024-04-25T14:05:33.566106+0000","last_unstale":"2024-04-25T14:05:56.833753+0000","last_undegraded":"2024-04-25T14:05:56.833753+0000","last_fullsized":"2024-04-25T14:05:56.833753+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:51.751312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.186808+0000","last_change":"2024-04-25T14:05:55.210034+0000","last_active":"2024-04-25T14:05:56.186808+0000","last_peered":"2024-04-25T14:05:56.186808+0000","last_clean":"2024-04-25T14:05:56.186808+0000","last_became_active":"2024-04-25T14:05:55.209669+0000","last_became_peered":"2024-04-25T14:05:55.209669+0000","last_unstale":"2024-04-25T14:05:56.186808+0000","last_undegraded":"2024-04-25T14:05:56.186808+0000","last_fullsized":"2024-04-25T14:05:56.186808+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:34.622772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"53'4","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.313104+0000","last_change":"2024-04-25T14:05:54.188059+0000","last_active":"2024-04-25T14:05:56.313104+0000","last_peered":"2024-04-25T14:05:56.313104+0000","last_clean":"2024-04-25T14:05:56.313104+0000","last_became_active":"2024-04-25T14:05:54.187771+0000","last_became_peered":"2024-04-25T14:05:54.187771+0000","last_unstale":"2024-04-25T14:05:56.313104+0000","last_undegraded":"2024-04-25T14:05:56.313104+0000","last_fullsized":"2024-04-25T14:05:56.313104+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:21.425955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":28,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.241775+0000","last_change":"2024-04-25T14:05:32.553901+0000","last_active":"2024-04-25T14:05:56.241775+0000","last_peered":"2024-04-25T14:05:56.241775+0000","last_clean":"2024-04-25T14:05:56.241775+0000","last_became_active":"2024-04-25T14:05:32.552885+0000","last_became_peered":"2024-04-25T14:05:32.552885+0000","last_unstale":"2024-04-25T14:05:56.241775+0000","last_undegraded":"2024-04-25T14:05:56.241775+0000","last_fullsized":"2024-04-25T14:05:56.241775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:17.068068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.181395+0000","last_change":"2024-04-25T14:05:33.571641+0000","last_active":"2024-04-25T14:05:56.181395+0000","last_peered":"2024-04-25T14:05:56.181395+0000","last_clean":"2024-04-25T14:05:56.181395+0000","last_became_active":"2024-04-25T14:05:33.571347+0000","last_became_peered":"2024-04-25T14:05:33.571347+0000","last_unstale":"2024-04-25T14:05:56.181395+0000","last_undegraded":"2024-04-25T14:05:56.181395+0000","last_fullsized":"2024-04-25T14:05:56.181395+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:06.138261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.242425+0000","last_change":"2024-04-25T14:05:55.200097+0000","last_active":"2024-04-25T14:05:56.242425+0000","last_peered":"2024-04-25T14:05:56.242425+0000","last_clean":"2024-04-25T14:05:56.242425+0000","last_became_active":"2024-04-25T14:05:55.199757+0000","last_became_peered":"2024-04-25T14:05:55.199757+0000","last_unstale":"2024-04-25T14:05:56.242425+0000","last_undegraded":"2024-04-25T14:05:56.242425+0000","last_fullsized":"2024-04-25T14:05:56.242425+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:57:31.699454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.187718+0000","last_change":"2024-04-25T14:05:32.562133+0000","last_active":"2024-04-25T14:05:56.187718+0000","last_peered":"2024-04-25T14:05:56.187718+0000","last_clean":"2024-04-25T14:05:56.187718+0000","last_became_active":"2024-04-25T14:05:32.561845+0000","last_became_peered":"2024-04-25T14:05:32.561845+0000","last_unstale":"2024-04-25T14:05:56.187718+0000","last_undegraded":"2024-04-25T14:05:56.187718+0000","last_fullsized":"2024-04-25T14:05:56.187718+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:14.012822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833776+0000","last_change":"2024-04-25T14:05:33.586628+0000","last_active":"2024-04-25T14:05:56.833776+0000","last_peered":"2024-04-25T14:05:56.833776+0000","last_clean":"2024-04-25T14:05:56.833776+0000","last_became_active":"2024-04-25T14:05:33.586317+0000","last_became_peered":"2024-04-25T14:05:33.586317+0000","last_unstale":"2024-04-25T14:05:56.833776+0000","last_undegraded":"2024-04-25T14:05:56.833776+0000","last_fullsized":"2024-04-25T14:05:56.833776+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:22.789679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1c","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833199+0000","last_change":"2024-04-25T14:05:54.193210+0000","last_active":"2024-04-25T14:05:56.833199+0000","last_peered":"2024-04-25T14:05:56.833199+0000","last_clean":"2024-04-25T14:05:56.833199+0000","last_became_active":"2024-04-25T14:05:54.192855+0000","last_became_peered":"2024-04-25T14:05:54.192855+0000","last_unstale":"2024-04-25T14:05:56.833199+0000","last_undegraded":"2024-04-25T14:05:56.833199+0000","last_fullsized":"2024-04-25T14:05:56.833199+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:57:02.409492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833688+0000","last_change":"2024-04-25T14:05:55.197405+0000","last_active":"2024-04-25T14:05:56.833688+0000","last_peered":"2024-04-25T14:05:56.833688+0000","last_clean":"2024-04-25T14:05:56.833688+0000","last_became_active":"2024-04-25T14:05:55.197111+0000","last_became_peered":"2024-04-25T14:05:55.197111+0000","last_unstale":"2024-04-25T14:05:56.833688+0000","last_undegraded":"2024-04-25T14:05:56.833688+0000","last_fullsized":"2024-04-25T14:05:56.833688+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.045928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.13","version":"45'2","reported_seq":32,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.832543+0000","last_change":"2024-04-25T14:05:32.557589+0000","last_active":"2024-04-25T14:05:56.832543+0000","last_peered":"2024-04-25T14:05:56.832543+0000","last_clean":"2024-04-25T14:05:56.832543+0000","last_became_active":"2024-04-25T14:05:32.557260+0000","last_became_peered":"2024-04-25T14:05:32.557260+0000","last_unstale":"2024-04-25T14:05:56.832543+0000","last_undegraded":"2024-04-25T14:05:56.832543+0000","last_fullsized":"2024-04-25T14:05:56.832543+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:26.912034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.188073+0000","last_change":"2024-04-25T14:05:33.569490+0000","last_active":"2024-04-25T14:05:56.188073+0000","last_peered":"2024-04-25T14:05:56.188073+0000","last_clean":"2024-04-25T14:05:56.188073+0000","last_became_active":"2024-04-25T14:05:33.569125+0000","last_became_peered":"2024-04-25T14:05:33.569125+0000","last_unstale":"2024-04-25T14:05:56.188073+0000","last_undegraded":"2024-04-25T14:05:56.188073+0000","last_fullsized":"2024-04-25T14:05:56.188073+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:33.637941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833464+0000","last_change":"2024-04-25T14:05:54.190315+0000","last_active":"2024-04-25T14:05:56.833464+0000","last_peered":"2024-04-25T14:05:56.833464+0000","last_clean":"2024-04-25T14:05:56.833464+0000","last_became_active":"2024-04-25T14:05:54.189897+0000","last_became_peered":"2024-04-25T14:05:54.189897+0000","last_unstale":"2024-04-25T14:05:56.833464+0000","last_undegraded":"2024-04-25T14:05:56.833464+0000","last_fullsized":"2024-04-25T14:05:56.833464+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:55.222389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.312268+0000","last_change":"2024-04-25T14:05:55.202808+0000","last_active":"2024-04-25T14:05:56.312268+0000","last_peered":"2024-04-25T14:05:56.312268+0000","last_clean":"2024-04-25T14:05:56.312268+0000","last_became_active":"2024-04-25T14:05:55.202231+0000","last_became_peered":"2024-04-25T14:05:55.202231+0000","last_unstale":"2024-04-25T14:05:56.312268+0000","last_undegraded":"2024-04-25T14:05:56.312268+0000","last_fullsized":"2024-04-25T14:05:56.312268+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:50.046862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.14","version":"50'12","reported_seq":47,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.181507+0000","last_change":"2024-04-25T14:05:32.564199+0000","last_active":"2024-04-25T14:05:56.181507+0000","last_peered":"2024-04-25T14:05:56.181507+0000","last_clean":"2024-04-25T14:05:56.181507+0000","last_became_active":"2024-04-25T14:05:32.563935+0000","last_became_peered":"2024-04-25T14:05:32.563935+0000","last_unstale":"2024-04-25T14:05:56.181507+0000","last_undegraded":"2024-04-25T14:05:56.181507+0000","last_fullsized":"2024-04-25T14:05:56.181507+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:38.715145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13068,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":11,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":25,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.881275+0000","last_change":"2024-04-25T14:05:33.593376+0000","last_active":"2024-04-25T14:05:56.881275+0000","last_peered":"2024-04-25T14:05:56.881275+0000","last_clean":"2024-04-25T14:05:56.881275+0000","last_became_active":"2024-04-25T14:05:33.593105+0000","last_became_peered":"2024-04-25T14:05:33.593105+0000","last_unstale":"2024-04-25T14:05:56.881275+0000","last_undegraded":"2024-04-25T14:05:56.881275+0000","last_fullsized":"2024-04-25T14:05:56.881275+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:28.629474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1a","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180726+0000","last_change":"2024-04-25T14:05:54.192638+0000","last_active":"2024-04-25T14:05:56.180726+0000","last_peered":"2024-04-25T14:05:56.180726+0000","last_clean":"2024-04-25T14:05:56.180726+0000","last_became_active":"2024-04-25T14:05:54.192278+0000","last_became_peered":"2024-04-25T14:05:54.192278+0000","last_unstale":"2024-04-25T14:05:56.180726+0000","last_undegraded":"2024-04-25T14:05:56.180726+0000","last_fullsized":"2024-04-25T14:05:56.180726+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:00:09.293862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:57.054780+0000","last_change":"2024-04-25T14:05:55.200575+0000","last_active":"2024-04-25T14:05:57.054780+0000","last_peered":"2024-04-25T14:05:57.054780+0000","last_clean":"2024-04-25T14:05:57.054780+0000","last_became_active":"2024-04-25T14:05:55.199674+0000","last_became_peered":"2024-04-25T14:05:55.199674+0000","last_unstale":"2024-04-25T14:05:57.054780+0000","last_undegraded":"2024-04-25T14:05:57.054780+0000","last_fullsized":"2024-04-25T14:05:57.054780+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:39.697034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"45'3","reported_seq":34,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.242251+0000","last_change":"2024-04-25T14:05:32.566039+0000","last_active":"2024-04-25T14:05:56.242251+0000","last_peered":"2024-04-25T14:05:56.242251+0000","last_clean":"2024-04-25T14:05:56.242251+0000","last_became_active":"2024-04-25T14:05:32.565670+0000","last_became_peered":"2024-04-25T14:05:32.565670+0000","last_unstale":"2024-04-25T14:05:56.242251+0000","last_undegraded":"2024-04-25T14:05:56.242251+0000","last_fullsized":"2024-04-25T14:05:56.242251+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:12.973199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833469+0000","last_change":"2024-04-25T14:05:33.577142+0000","last_active":"2024-04-25T14:05:56.833469+0000","last_peered":"2024-04-25T14:05:56.833469+0000","last_clean":"2024-04-25T14:05:56.833469+0000","last_became_active":"2024-04-25T14:05:33.576820+0000","last_became_peered":"2024-04-25T14:05:33.576820+0000","last_unstale":"2024-04-25T14:05:56.833469+0000","last_undegraded":"2024-04-25T14:05:56.833469+0000","last_fullsized":"2024-04-25T14:05:56.833469+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:07.917626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1b","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180733+0000","last_change":"2024-04-25T14:05:54.180076+0000","last_active":"2024-04-25T14:05:56.180733+0000","last_peered":"2024-04-25T14:05:56.180733+0000","last_clean":"2024-04-25T14:05:56.180733+0000","last_became_active":"2024-04-25T14:05:54.179725+0000","last_became_peered":"2024-04-25T14:05:54.179725+0000","last_unstale":"2024-04-25T14:05:56.180733+0000","last_undegraded":"2024-04-25T14:05:56.180733+0000","last_fullsized":"2024-04-25T14:05:56.180733+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:38.204467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.181130+0000","last_change":"2024-04-25T14:05:55.176089+0000","last_active":"2024-04-25T14:05:56.181130+0000","last_peered":"2024-04-25T14:05:56.181130+0000","last_clean":"2024-04-25T14:05:56.181130+0000","last_became_active":"2024-04-25T14:05:55.175181+0000","last_became_peered":"2024-04-25T14:05:55.175181+0000","last_unstale":"2024-04-25T14:05:56.181130+0000","last_undegraded":"2024-04-25T14:05:56.181130+0000","last_fullsized":"2024-04-25T14:05:56.181130+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:39.678190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.881812+0000","last_change":"2024-04-25T14:05:32.563130+0000","last_active":"2024-04-25T14:05:56.881812+0000","last_peered":"2024-04-25T14:05:56.881812+0000","last_clean":"2024-04-25T14:05:56.881812+0000","last_became_active":"2024-04-25T14:05:32.562793+0000","last_became_peered":"2024-04-25T14:05:32.562793+0000","last_unstale":"2024-04-25T14:05:56.881812+0000","last_undegraded":"2024-04-25T14:05:56.881812+0000","last_fullsized":"2024-04-25T14:05:56.881812+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:43.267251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.186805+0000","last_change":"2024-04-25T14:05:33.573758+0000","last_active":"2024-04-25T14:05:56.186805+0000","last_peered":"2024-04-25T14:05:56.186805+0000","last_clean":"2024-04-25T14:05:56.186805+0000","last_became_active":"2024-04-25T14:05:33.572760+0000","last_became_peered":"2024-04-25T14:05:33.572760+0000","last_unstale":"2024-04-25T14:05:56.186805+0000","last_undegraded":"2024-04-25T14:05:56.186805+0000","last_fullsized":"2024-04-25T14:05:56.186805+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:36.723420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.186337+0000","last_change":"2024-04-25T14:05:54.193681+0000","last_active":"2024-04-25T14:05:56.186337+0000","last_peered":"2024-04-25T14:05:56.186337+0000","last_clean":"2024-04-25T14:05:56.186337+0000","last_became_active":"2024-04-25T14:05:54.190583+0000","last_became_peered":"2024-04-25T14:05:54.190583+0000","last_unstale":"2024-04-25T14:05:56.186337+0000","last_undegraded":"2024-04-25T14:05:56.186337+0000","last_fullsized":"2024-04-25T14:05:56.186337+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:46.274456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.187626+0000","last_change":"2024-04-25T14:05:55.211925+0000","last_active":"2024-04-25T14:05:56.187626+0000","last_peered":"2024-04-25T14:05:56.187626+0000","last_clean":"2024-04-25T14:05:56.187626+0000","last_became_active":"2024-04-25T14:05:55.211644+0000","last_became_peered":"2024-04-25T14:05:55.211644+0000","last_unstale":"2024-04-25T14:05:56.187626+0000","last_undegraded":"2024-04-25T14:05:56.187626+0000","last_fullsized":"2024-04-25T14:05:56.187626+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:51.604196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":28,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.242610+0000","last_change":"2024-04-25T14:05:32.550533+0000","last_active":"2024-04-25T14:05:56.242610+0000","last_peered":"2024-04-25T14:05:56.242610+0000","last_clean":"2024-04-25T14:05:56.242610+0000","last_became_active":"2024-04-25T14:05:32.550116+0000","last_became_peered":"2024-04-25T14:05:32.550116+0000","last_unstale":"2024-04-25T14:05:56.242610+0000","last_undegraded":"2024-04-25T14:05:56.242610+0000","last_fullsized":"2024-04-25T14:05:56.242610+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:27.623051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":25,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.880918+0000","last_change":"2024-04-25T14:05:33.583166+0000","last_active":"2024-04-25T14:05:56.880918+0000","last_peered":"2024-04-25T14:05:56.880918+0000","last_clean":"2024-04-25T14:05:56.880918+0000","last_became_active":"2024-04-25T14:05:33.582833+0000","last_became_peered":"2024-04-25T14:05:33.582833+0000","last_unstale":"2024-04-25T14:05:56.880918+0000","last_undegraded":"2024-04-25T14:05:56.880918+0000","last_fullsized":"2024-04-25T14:05:56.880918+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:11.087435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180458+0000","last_change":"2024-04-25T14:05:54.190905+0000","last_active":"2024-04-25T14:05:56.180458+0000","last_peered":"2024-04-25T14:05:56.180458+0000","last_clean":"2024-04-25T14:05:56.180458+0000","last_became_active":"2024-04-25T14:05:54.190297+0000","last_became_peered":"2024-04-25T14:05:54.190297+0000","last_unstale":"2024-04-25T14:05:56.180458+0000","last_undegraded":"2024-04-25T14:05:56.180458+0000","last_fullsized":"2024-04-25T14:05:56.180458+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:23.256150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.181689+0000","last_change":"2024-04-25T14:05:55.175335+0000","last_active":"2024-04-25T14:05:56.181689+0000","last_peered":"2024-04-25T14:05:56.181689+0000","last_clean":"2024-04-25T14:05:56.181689+0000","last_became_active":"2024-04-25T14:05:55.174104+0000","last_became_peered":"2024-04-25T14:05:55.174104+0000","last_unstale":"2024-04-25T14:05:56.181689+0000","last_undegraded":"2024-04-25T14:05:56.181689+0000","last_fullsized":"2024-04-25T14:05:56.181689+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:42:48.413708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":28,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.832556+0000","last_change":"2024-04-25T14:05:32.563104+0000","last_active":"2024-04-25T14:05:56.832556+0000","last_peered":"2024-04-25T14:05:56.832556+0000","last_clean":"2024-04-25T14:05:56.832556+0000","last_became_active":"2024-04-25T14:05:32.560440+0000","last_became_peered":"2024-04-25T14:05:32.560440+0000","last_unstale":"2024-04-25T14:05:56.832556+0000","last_undegraded":"2024-04-25T14:05:56.832556+0000","last_fullsized":"2024-04-25T14:05:56.832556+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:12.973199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.242580+0000","last_change":"2024-04-25T14:05:33.583073+0000","last_active":"2024-04-25T14:05:56.242580+0000","last_peered":"2024-04-25T14:05:56.242580+0000","last_clean":"2024-04-25T14:05:56.242580+0000","last_became_active":"2024-04-25T14:05:33.582763+0000","last_became_peered":"2024-04-25T14:05:33.582763+0000","last_unstale":"2024-04-25T14:05:56.242580+0000","last_undegraded":"2024-04-25T14:05:56.242580+0000","last_fullsized":"2024-04-25T14:05:56.242580+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:18.567327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.881608+0000","last_change":"2024-04-25T14:05:54.189972+0000","last_active":"2024-04-25T14:05:56.881608+0000","last_peered":"2024-04-25T14:05:56.881608+0000","last_clean":"2024-04-25T14:05:56.881608+0000","last_became_active":"2024-04-25T14:05:54.189495+0000","last_became_peered":"2024-04-25T14:05:54.189495+0000","last_unstale":"2024-04-25T14:05:56.881608+0000","last_undegraded":"2024-04-25T14:05:56.881608+0000","last_fullsized":"2024-04-25T14:05:56.881608+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:46.274456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.312600+0000","last_change":"2024-04-25T14:05:55.204348+0000","last_active":"2024-04-25T14:05:56.312600+0000","last_peered":"2024-04-25T14:05:56.312600+0000","last_clean":"2024-04-25T14:05:56.312600+0000","last_became_active":"2024-04-25T14:05:55.204016+0000","last_became_peered":"2024-04-25T14:05:55.204016+0000","last_unstale":"2024-04-25T14:05:56.312600+0000","last_undegraded":"2024-04-25T14:05:56.312600+0000","last_fullsized":"2024-04-25T14:05:56.312600+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:51.604196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.186131+0000","last_change":"2024-04-25T14:05:32.551806+0000","last_active":"2024-04-25T14:05:56.186131+0000","last_peered":"2024-04-25T14:05:56.186131+0000","last_clean":"2024-04-25T14:05:56.186131+0000","last_became_active":"2024-04-25T14:05:32.551441+0000","last_became_peered":"2024-04-25T14:05:32.551441+0000","last_unstale":"2024-04-25T14:05:56.186131+0000","last_undegraded":"2024-04-25T14:05:56.186131+0000","last_fullsized":"2024-04-25T14:05:56.186131+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:31.414048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.181552+0000","last_change":"2024-04-25T14:05:33.576549+0000","last_active":"2024-04-25T14:05:56.181552+0000","last_peered":"2024-04-25T14:05:56.181552+0000","last_clean":"2024-04-25T14:05:56.181552+0000","last_became_active":"2024-04-25T14:05:33.576037+0000","last_became_peered":"2024-04-25T14:05:33.576037+0000","last_unstale":"2024-04-25T14:05:56.181552+0000","last_undegraded":"2024-04-25T14:05:56.181552+0000","last_fullsized":"2024-04-25T14:05:56.181552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:47.818409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.242575+0000","last_change":"2024-04-25T14:05:54.193885+0000","last_active":"2024-04-25T14:05:56.242575+0000","last_peered":"2024-04-25T14:05:56.242575+0000","last_clean":"2024-04-25T14:05:56.242575+0000","last_became_active":"2024-04-25T14:05:54.193530+0000","last_became_peered":"2024-04-25T14:05:54.193530+0000","last_unstale":"2024-04-25T14:05:56.242575+0000","last_undegraded":"2024-04-25T14:05:56.242575+0000","last_fullsized":"2024-04-25T14:05:56.242575+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:50.596113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.881592+0000","last_change":"2024-04-25T14:05:55.177312+0000","last_active":"2024-04-25T14:05:56.881592+0000","last_peered":"2024-04-25T14:05:56.881592+0000","last_clean":"2024-04-25T14:05:56.881592+0000","last_became_active":"2024-04-25T14:05:55.176744+0000","last_became_peered":"2024-04-25T14:05:55.176744+0000","last_unstale":"2024-04-25T14:05:56.881592+0000","last_undegraded":"2024-04-25T14:05:56.881592+0000","last_fullsized":"2024-04-25T14:05:56.881592+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:25.640028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1a","version":"50'2","reported_seq":29,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.880401+0000","last_change":"2024-04-25T14:05:32.557698+0000","last_active":"2024-04-25T14:05:56.880401+0000","last_peered":"2024-04-25T14:05:56.880401+0000","last_clean":"2024-04-25T14:05:56.880401+0000","last_became_active":"2024-04-25T14:05:32.557358+0000","last_became_peered":"2024-04-25T14:05:32.557358+0000","last_unstale":"2024-04-25T14:05:56.880401+0000","last_undegraded":"2024-04-25T14:05:56.880401+0000","last_fullsized":"2024-04-25T14:05:56.880401+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:20.281168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.241991+0000","last_change":"2024-04-25T14:05:33.584433+0000","last_active":"2024-04-25T14:05:56.241991+0000","last_peered":"2024-04-25T14:05:56.241991+0000","last_clean":"2024-04-25T14:05:56.241991+0000","last_became_active":"2024-04-25T14:05:33.584130+0000","last_became_peered":"2024-04-25T14:05:33.584130+0000","last_unstale":"2024-04-25T14:05:56.241991+0000","last_undegraded":"2024-04-25T14:05:56.241991+0000","last_fullsized":"2024-04-25T14:05:56.241991+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:42.654848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.14","version":"53'6","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.312756+0000","last_change":"2024-04-25T14:05:54.184181+0000","last_active":"2024-04-25T14:05:56.312756+0000","last_peered":"2024-04-25T14:05:56.312756+0000","last_clean":"2024-04-25T14:05:56.312756+0000","last_became_active":"2024-04-25T14:05:54.183878+0000","last_became_peered":"2024-04-25T14:05:54.183878+0000","last_unstale":"2024-04-25T14:05:56.312756+0000","last_undegraded":"2024-04-25T14:05:56.312756+0000","last_fullsized":"2024-04-25T14:05:56.312756+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:54.069975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.186682+0000","last_change":"2024-04-25T14:05:55.202255+0000","last_active":"2024-04-25T14:05:56.186682+0000","last_peered":"2024-04-25T14:05:56.186682+0000","last_clean":"2024-04-25T14:05:56.186682+0000","last_became_active":"2024-04-25T14:05:55.201856+0000","last_became_peered":"2024-04-25T14:05:55.201856+0000","last_unstale":"2024-04-25T14:05:56.186682+0000","last_undegraded":"2024-04-25T14:05:56.186682+0000","last_fullsized":"2024-04-25T14:05:56.186682+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:51.863500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"50'2","reported_seq":30,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833327+0000","last_change":"2024-04-25T14:05:32.562043+0000","last_active":"2024-04-25T14:05:56.833327+0000","last_peered":"2024-04-25T14:05:56.833327+0000","last_clean":"2024-04-25T14:05:56.833327+0000","last_became_active":"2024-04-25T14:05:32.561736+0000","last_became_peered":"2024-04-25T14:05:32.561736+0000","last_unstale":"2024-04-25T14:05:56.833327+0000","last_undegraded":"2024-04-25T14:05:56.833327+0000","last_fullsized":"2024-04-25T14:05:56.833327+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:40.505936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":25,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.881511+0000","last_change":"2024-04-25T14:05:33.582814+0000","last_active":"2024-04-25T14:05:56.881511+0000","last_peered":"2024-04-25T14:05:56.881511+0000","last_clean":"2024-04-25T14:05:56.881511+0000","last_became_active":"2024-04-25T14:05:33.582127+0000","last_became_peered":"2024-04-25T14:05:33.582127+0000","last_unstale":"2024-04-25T14:05:56.881511+0000","last_undegraded":"2024-04-25T14:05:56.881511+0000","last_fullsized":"2024-04-25T14:05:56.881511+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:18.074490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.15","version":"53'3","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.242352+0000","last_change":"2024-04-25T14:05:54.194460+0000","last_active":"2024-04-25T14:05:56.242352+0000","last_peered":"2024-04-25T14:05:56.242352+0000","last_clean":"2024-04-25T14:05:56.242352+0000","last_became_active":"2024-04-25T14:05:54.194040+0000","last_became_peered":"2024-04-25T14:05:54.194040+0000","last_unstale":"2024-04-25T14:05:56.242352+0000","last_undegraded":"2024-04-25T14:05:56.242352+0000","last_fullsized":"2024-04-25T14:05:56.242352+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:12.365772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.185762+0000","last_change":"2024-04-25T14:05:55.192270+0000","last_active":"2024-04-25T14:05:56.185762+0000","last_peered":"2024-04-25T14:05:56.185762+0000","last_clean":"2024-04-25T14:05:56.185762+0000","last_became_active":"2024-04-25T14:05:55.191968+0000","last_became_peered":"2024-04-25T14:05:55.191968+0000","last_unstale":"2024-04-25T14:05:56.185762+0000","last_undegraded":"2024-04-25T14:05:56.185762+0000","last_fullsized":"2024-04-25T14:05:56.185762+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:43:54.878485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"45'2","reported_seq":33,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.188292+0000","last_change":"2024-04-25T14:05:32.567307+0000","last_active":"2024-04-25T14:05:56.188292+0000","last_peered":"2024-04-25T14:05:56.188292+0000","last_clean":"2024-04-25T14:05:56.188292+0000","last_became_active":"2024-04-25T14:05:32.566940+0000","last_became_peered":"2024-04-25T14:05:32.566940+0000","last_unstale":"2024-04-25T14:05:56.188292+0000","last_undegraded":"2024-04-25T14:05:56.188292+0000","last_fullsized":"2024-04-25T14:05:56.188292+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:12.973199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180912+0000","last_change":"2024-04-25T14:05:33.593213+0000","last_active":"2024-04-25T14:05:56.180912+0000","last_peered":"2024-04-25T14:05:56.180912+0000","last_clean":"2024-04-25T14:05:56.180912+0000","last_became_active":"2024-04-25T14:05:33.592923+0000","last_became_peered":"2024-04-25T14:05:33.592923+0000","last_unstale":"2024-04-25T14:05:56.180912+0000","last_undegraded":"2024-04-25T14:05:56.180912+0000","last_fullsized":"2024-04-25T14:05:56.180912+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:18.458163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.187145+0000","last_change":"2024-04-25T14:05:54.188883+0000","last_active":"2024-04-25T14:05:56.187145+0000","last_peered":"2024-04-25T14:05:56.187145+0000","last_clean":"2024-04-25T14:05:56.187145+0000","last_became_active":"2024-04-25T14:05:54.185400+0000","last_became_peered":"2024-04-25T14:05:54.185400+0000","last_unstale":"2024-04-25T14:05:56.187145+0000","last_undegraded":"2024-04-25T14:05:56.187145+0000","last_fullsized":"2024-04-25T14:05:56.187145+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:05.310541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.880733+0000","last_change":"2024-04-25T14:05:55.190580+0000","last_active":"2024-04-25T14:05:56.880733+0000","last_peered":"2024-04-25T14:05:56.880733+0000","last_clean":"2024-04-25T14:05:56.880733+0000","last_became_active":"2024-04-25T14:05:55.189691+0000","last_became_peered":"2024-04-25T14:05:55.189691+0000","last_unstale":"2024-04-25T14:05:56.880733+0000","last_undegraded":"2024-04-25T14:05:56.880733+0000","last_fullsized":"2024-04-25T14:05:56.880733+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:22.434038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1d","version":"45'2","reported_seq":33,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.313086+0000","last_change":"2024-04-25T14:05:32.561304+0000","last_active":"2024-04-25T14:05:56.313086+0000","last_peered":"2024-04-25T14:05:56.313086+0000","last_clean":"2024-04-25T14:05:56.313086+0000","last_became_active":"2024-04-25T14:05:32.560988+0000","last_became_peered":"2024-04-25T14:05:32.560988+0000","last_unstale":"2024-04-25T14:05:56.313086+0000","last_undegraded":"2024-04-25T14:05:56.313086+0000","last_fullsized":"2024-04-25T14:05:56.313086+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:32.071918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":25,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.186810+0000","last_change":"2024-04-25T14:05:33.574504+0000","last_active":"2024-04-25T14:05:56.186810+0000","last_peered":"2024-04-25T14:05:56.186810+0000","last_clean":"2024-04-25T14:05:56.186810+0000","last_became_active":"2024-04-25T14:05:33.574191+0000","last_became_peered":"2024-04-25T14:05:33.574191+0000","last_unstale":"2024-04-25T14:05:56.186810+0000","last_undegraded":"2024-04-25T14:05:56.186810+0000","last_fullsized":"2024-04-25T14:05:56.186810+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:28.629474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.13","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.241716+0000","last_change":"2024-04-25T14:05:54.195064+0000","last_active":"2024-04-25T14:05:56.241716+0000","last_peered":"2024-04-25T14:05:56.241716+0000","last_clean":"2024-04-25T14:05:56.241716+0000","last_became_active":"2024-04-25T14:05:54.194656+0000","last_became_peered":"2024-04-25T14:05:54.194656+0000","last_unstale":"2024-04-25T14:05:56.241716+0000","last_undegraded":"2024-04-25T14:05:56.241716+0000","last_fullsized":"2024-04-25T14:05:56.241716+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:33.614689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.312671+0000","last_change":"2024-04-25T14:05:55.205818+0000","last_active":"2024-04-25T14:05:56.312671+0000","last_peered":"2024-04-25T14:05:56.312671+0000","last_clean":"2024-04-25T14:05:56.312671+0000","last_became_active":"2024-04-25T14:05:55.205498+0000","last_became_peered":"2024-04-25T14:05:55.205498+0000","last_unstale":"2024-04-25T14:05:56.312671+0000","last_undegraded":"2024-04-25T14:05:56.312671+0000","last_fullsized":"2024-04-25T14:05:56.312671+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:22.718215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1e","version":"50'2","reported_seq":31,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.312909+0000","last_change":"2024-04-25T14:05:32.569704+0000","last_active":"2024-04-25T14:05:56.312909+0000","last_peered":"2024-04-25T14:05:56.312909+0000","last_clean":"2024-04-25T14:05:56.312909+0000","last_became_active":"2024-04-25T14:05:32.569349+0000","last_became_peered":"2024-04-25T14:05:32.569349+0000","last_unstale":"2024-04-25T14:05:56.312909+0000","last_undegraded":"2024-04-25T14:05:56.312909+0000","last_fullsized":"2024-04-25T14:05:56.312909+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:23.323591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.186929+0000","last_change":"2024-04-25T14:05:33.573320+0000","last_active":"2024-04-25T14:05:56.186929+0000","last_peered":"2024-04-25T14:05:56.186929+0000","last_clean":"2024-04-25T14:05:56.186929+0000","last_became_active":"2024-04-25T14:05:33.573015+0000","last_became_peered":"2024-04-25T14:05:33.573015+0000","last_unstale":"2024-04-25T14:05:56.186929+0000","last_undegraded":"2024-04-25T14:05:56.186929+0000","last_fullsized":"2024-04-25T14:05:56.186929+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:25.659996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.10","version":"53'8","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.880693+0000","last_change":"2024-04-25T14:05:54.188045+0000","last_active":"2024-04-25T14:05:56.880693+0000","last_peered":"2024-04-25T14:05:56.880693+0000","last_clean":"2024-04-25T14:05:56.880693+0000","last_became_active":"2024-04-25T14:05:54.187386+0000","last_became_peered":"2024-04-25T14:05:54.187386+0000","last_unstale":"2024-04-25T14:05:56.880693+0000","last_undegraded":"2024-04-25T14:05:56.880693+0000","last_fullsized":"2024-04-25T14:05:56.880693+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:43:15.179093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.186271+0000","last_change":"2024-04-25T14:05:55.208703+0000","last_active":"2024-04-25T14:05:56.186271+0000","last_peered":"2024-04-25T14:05:56.186271+0000","last_clean":"2024-04-25T14:05:56.186271+0000","last_became_active":"2024-04-25T14:05:55.208391+0000","last_became_peered":"2024-04-25T14:05:55.208391+0000","last_unstale":"2024-04-25T14:05:56.186271+0000","last_undegraded":"2024-04-25T14:05:56.186271+0000","last_fullsized":"2024-04-25T14:05:56.186271+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:06.318625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"50'5","reported_seq":34,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.187081+0000","last_change":"2024-04-25T14:05:32.556304+0000","last_active":"2024-04-25T14:05:56.187081+0000","last_peered":"2024-04-25T14:05:56.187081+0000","last_clean":"2024-04-25T14:05:56.187081+0000","last_became_active":"2024-04-25T14:05:32.556052+0000","last_became_peered":"2024-04-25T14:05:32.556052+0000","last_unstale":"2024-04-25T14:05:56.187081+0000","last_undegraded":"2024-04-25T14:05:56.187081+0000","last_fullsized":"2024-04-25T14:05:56.187081+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:31.523192+0000","last_clean_scrub_stamp":"2024-04-25T14:05:31.523192+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:16.452558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.241376+0000","last_change":"2024-04-25T14:05:33.576943+0000","last_active":"2024-04-25T14:05:56.241376+0000","last_peered":"2024-04-25T14:05:56.241376+0000","last_clean":"2024-04-25T14:05:56.241376+0000","last_became_active":"2024-04-25T14:05:33.576387+0000","last_became_peered":"2024-04-25T14:05:33.576387+0000","last_unstale":"2024-04-25T14:05:56.241376+0000","last_undegraded":"2024-04-25T14:05:56.241376+0000","last_fullsized":"2024-04-25T14:05:56.241376+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:54.564632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.241301+0000","last_change":"2024-04-25T14:05:54.186157+0000","last_active":"2024-04-25T14:05:56.241301+0000","last_peered":"2024-04-25T14:05:56.241301+0000","last_clean":"2024-04-25T14:05:56.241301+0000","last_became_active":"2024-04-25T14:05:54.185797+0000","last_became_peered":"2024-04-25T14:05:54.185797+0000","last_unstale":"2024-04-25T14:05:56.241301+0000","last_undegraded":"2024-04-25T14:05:56.241301+0000","last_fullsized":"2024-04-25T14:05:56.241301+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:53.144076+0000","last_clean_scrub_stamp":"2024-04-25T14:05:53.144076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:50.855416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.187010+0000","last_change":"2024-04-25T14:05:55.187889+0000","last_active":"2024-04-25T14:05:56.187010+0000","last_peered":"2024-04-25T14:05:56.187010+0000","last_clean":"2024-04-25T14:05:56.187010+0000","last_became_active":"2024-04-25T14:05:55.187589+0000","last_became_peered":"2024-04-25T14:05:55.187589+0000","last_unstale":"2024-04-25T14:05:56.187010+0000","last_undegraded":"2024-04-25T14:05:56.187010+0000","last_fullsized":"2024-04-25T14:05:56.187010+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:00:10.301945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.185783+0000","last_change":"2024-04-25T14:05:33.561025+0000","last_active":"2024-04-25T14:05:56.185783+0000","last_peered":"2024-04-25T14:05:56.185783+0000","last_clean":"2024-04-25T14:05:56.185783+0000","last_became_active":"2024-04-25T14:05:33.560525+0000","last_became_peered":"2024-04-25T14:05:33.560525+0000","last_unstale":"2024-04-25T14:05:56.185783+0000","last_undegraded":"2024-04-25T14:05:56.185783+0000","last_fullsized":"2024-04-25T14:05:56.185783+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:54.564632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.241635+0000","last_change":"2024-04-25T14:05:55.195221+0000","last_active":"2024-04-25T14:05:56.241635+0000","last_peered":"2024-04-25T14:05:56.241635+0000","last_clean":"2024-04-25T14:05:56.241635+0000","last_became_active":"2024-04-25T14:05:55.194826+0000","last_became_peered":"2024-04-25T14:05:55.194826+0000","last_unstale":"2024-04-25T14:05:56.241635+0000","last_undegraded":"2024-04-25T14:05:56.241635+0000","last_fullsized":"2024-04-25T14:05:56.241635+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:56.230473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.832945+0000","last_change":"2024-04-25T14:05:33.583788+0000","last_active":"2024-04-25T14:05:56.832945+0000","last_peered":"2024-04-25T14:05:56.832945+0000","last_clean":"2024-04-25T14:05:56.832945+0000","last_became_active":"2024-04-25T14:05:33.583507+0000","last_became_peered":"2024-04-25T14:05:33.583507+0000","last_unstale":"2024-04-25T14:05:56.832945+0000","last_undegraded":"2024-04-25T14:05:56.832945+0000","last_fullsized":"2024-04-25T14:05:56.832945+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:06.138261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.832968+0000","last_change":"2024-04-25T14:05:55.207790+0000","last_active":"2024-04-25T14:05:56.832968+0000","last_peered":"2024-04-25T14:05:56.832968+0000","last_clean":"2024-04-25T14:05:56.832968+0000","last_became_active":"2024-04-25T14:05:55.206817+0000","last_became_peered":"2024-04-25T14:05:55.206817+0000","last_unstale":"2024-04-25T14:05:56.832968+0000","last_undegraded":"2024-04-25T14:05:56.832968+0000","last_fullsized":"2024-04-25T14:05:56.832968+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:42:48.413708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.241673+0000","last_change":"2024-04-25T14:05:33.574825+0000","last_active":"2024-04-25T14:05:56.241673+0000","last_peered":"2024-04-25T14:05:56.241673+0000","last_clean":"2024-04-25T14:05:56.241673+0000","last_became_active":"2024-04-25T14:05:33.574492+0000","last_became_peered":"2024-04-25T14:05:33.574492+0000","last_unstale":"2024-04-25T14:05:56.241673+0000","last_undegraded":"2024-04-25T14:05:56.241673+0000","last_fullsized":"2024-04-25T14:05:56.241673+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:33.637941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.880740+0000","last_change":"2024-04-25T14:05:55.201123+0000","last_active":"2024-04-25T14:05:56.880740+0000","last_peered":"2024-04-25T14:05:56.880740+0000","last_clean":"2024-04-25T14:05:56.880740+0000","last_became_active":"2024-04-25T14:05:55.197333+0000","last_became_peered":"2024-04-25T14:05:55.197333+0000","last_unstale":"2024-04-25T14:05:56.880740+0000","last_undegraded":"2024-04-25T14:05:56.880740+0000","last_fullsized":"2024-04-25T14:05:56.880740+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:27.760805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833177+0000","last_change":"2024-04-25T14:05:33.572734+0000","last_active":"2024-04-25T14:05:56.833177+0000","last_peered":"2024-04-25T14:05:56.833177+0000","last_clean":"2024-04-25T14:05:56.833177+0000","last_became_active":"2024-04-25T14:05:33.572231+0000","last_became_peered":"2024-04-25T14:05:33.572231+0000","last_unstale":"2024-04-25T14:05:56.833177+0000","last_undegraded":"2024-04-25T14:05:56.833177+0000","last_fullsized":"2024-04-25T14:05:56.833177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:51.585470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.188198+0000","last_change":"2024-04-25T14:05:55.196754+0000","last_active":"2024-04-25T14:05:56.188198+0000","last_peered":"2024-04-25T14:05:56.188198+0000","last_clean":"2024-04-25T14:05:56.188198+0000","last_became_active":"2024-04-25T14:05:55.196439+0000","last_became_peered":"2024-04-25T14:05:55.196439+0000","last_unstale":"2024-04-25T14:05:56.188198+0000","last_undegraded":"2024-04-25T14:05:56.188198+0000","last_fullsized":"2024-04-25T14:05:56.188198+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:57:03.417575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":25,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.186901+0000","last_change":"2024-04-25T14:05:33.596454+0000","last_active":"2024-04-25T14:05:56.186901+0000","last_peered":"2024-04-25T14:05:56.186901+0000","last_clean":"2024-04-25T14:05:56.186901+0000","last_became_active":"2024-04-25T14:05:33.596225+0000","last_became_peered":"2024-04-25T14:05:33.596225+0000","last_unstale":"2024-04-25T14:05:56.186901+0000","last_undegraded":"2024-04-25T14:05:56.186901+0000","last_fullsized":"2024-04-25T14:05:56.186901+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:41.512359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.188161+0000","last_change":"2024-04-25T14:05:55.201329+0000","last_active":"2024-04-25T14:05:56.188161+0000","last_peered":"2024-04-25T14:05:56.188161+0000","last_clean":"2024-04-25T14:05:56.188161+0000","last_became_active":"2024-04-25T14:05:55.196667+0000","last_became_peered":"2024-04-25T14:05:55.196667+0000","last_unstale":"2024-04-25T14:05:56.188161+0000","last_undegraded":"2024-04-25T14:05:56.188161+0000","last_fullsized":"2024-04-25T14:05:56.188161+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:13.143182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.313039+0000","last_change":"2024-04-25T14:05:33.591448+0000","last_active":"2024-04-25T14:05:56.313039+0000","last_peered":"2024-04-25T14:05:56.313039+0000","last_clean":"2024-04-25T14:05:56.313039+0000","last_became_active":"2024-04-25T14:05:33.590795+0000","last_became_peered":"2024-04-25T14:05:33.590795+0000","last_unstale":"2024-04-25T14:05:56.313039+0000","last_undegraded":"2024-04-25T14:05:56.313039+0000","last_fullsized":"2024-04-25T14:05:56.313039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:33.637941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.312850+0000","last_change":"2024-04-25T14:05:55.211726+0000","last_active":"2024-04-25T14:05:56.312850+0000","last_peered":"2024-04-25T14:05:56.312850+0000","last_clean":"2024-04-25T14:05:56.312850+0000","last_became_active":"2024-04-25T14:05:55.211333+0000","last_became_peered":"2024-04-25T14:05:55.211333+0000","last_unstale":"2024-04-25T14:05:56.312850+0000","last_undegraded":"2024-04-25T14:05:56.312850+0000","last_fullsized":"2024-04-25T14:05:56.312850+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:18.962103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.181438+0000","last_change":"2024-04-25T14:05:33.576318+0000","last_active":"2024-04-25T14:05:56.181438+0000","last_peered":"2024-04-25T14:05:56.181438+0000","last_clean":"2024-04-25T14:05:56.181438+0000","last_became_active":"2024-04-25T14:05:33.575764+0000","last_became_peered":"2024-04-25T14:05:33.575764+0000","last_unstale":"2024-04-25T14:05:56.181438+0000","last_undegraded":"2024-04-25T14:05:56.181438+0000","last_fullsized":"2024-04-25T14:05:56.181438+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:33.455514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.881320+0000","last_change":"2024-04-25T14:05:55.196765+0000","last_active":"2024-04-25T14:05:56.881320+0000","last_peered":"2024-04-25T14:05:56.881320+0000","last_clean":"2024-04-25T14:05:56.881320+0000","last_became_active":"2024-04-25T14:05:55.196092+0000","last_became_peered":"2024-04-25T14:05:55.196092+0000","last_unstale":"2024-04-25T14:05:56.881320+0000","last_undegraded":"2024-04-25T14:05:56.881320+0000","last_fullsized":"2024-04-25T14:05:56.881320+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:29.540170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833857+0000","last_change":"2024-04-25T14:05:33.562329+0000","last_active":"2024-04-25T14:05:56.833857+0000","last_peered":"2024-04-25T14:05:56.833857+0000","last_clean":"2024-04-25T14:05:56.833857+0000","last_became_active":"2024-04-25T14:05:33.562039+0000","last_became_peered":"2024-04-25T14:05:33.562039+0000","last_unstale":"2024-04-25T14:05:56.833857+0000","last_undegraded":"2024-04-25T14:05:56.833857+0000","last_fullsized":"2024-04-25T14:05:56.833857+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:29.981652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180509+0000","last_change":"2024-04-25T14:05:55.186637+0000","last_active":"2024-04-25T14:05:56.180509+0000","last_peered":"2024-04-25T14:05:56.180509+0000","last_clean":"2024-04-25T14:05:56.180509+0000","last_became_active":"2024-04-25T14:05:55.185799+0000","last_became_peered":"2024-04-25T14:05:55.185799+0000","last_unstale":"2024-04-25T14:05:56.180509+0000","last_undegraded":"2024-04-25T14:05:56.180509+0000","last_fullsized":"2024-04-25T14:05:56.180509+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:49:29.357854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833087+0000","last_change":"2024-04-25T14:05:33.581503+0000","last_active":"2024-04-25T14:05:56.833087+0000","last_peered":"2024-04-25T14:05:56.833087+0000","last_clean":"2024-04-25T14:05:56.833087+0000","last_became_active":"2024-04-25T14:05:33.581124+0000","last_became_peered":"2024-04-25T14:05:33.581124+0000","last_unstale":"2024-04-25T14:05:56.833087+0000","last_undegraded":"2024-04-25T14:05:56.833087+0000","last_fullsized":"2024-04-25T14:05:56.833087+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:33.455514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.187873+0000","last_change":"2024-04-25T14:05:55.202826+0000","last_active":"2024-04-25T14:05:56.187873+0000","last_peered":"2024-04-25T14:05:56.187873+0000","last_clean":"2024-04-25T14:05:56.187873+0000","last_became_active":"2024-04-25T14:05:55.202526+0000","last_became_peered":"2024-04-25T14:05:55.202526+0000","last_unstale":"2024-04-25T14:05:56.187873+0000","last_undegraded":"2024-04-25T14:05:56.187873+0000","last_fullsized":"2024-04-25T14:05:56.187873+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:26.936075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:57.055125+0000","last_change":"2024-04-25T14:05:33.576117+0000","last_active":"2024-04-25T14:05:57.055125+0000","last_peered":"2024-04-25T14:05:57.055125+0000","last_clean":"2024-04-25T14:05:57.055125+0000","last_became_active":"2024-04-25T14:05:33.575838+0000","last_became_peered":"2024-04-25T14:05:33.575838+0000","last_unstale":"2024-04-25T14:05:57.055125+0000","last_undegraded":"2024-04-25T14:05:57.055125+0000","last_fullsized":"2024-04-25T14:05:57.055125+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:20.619851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.241295+0000","last_change":"2024-04-25T14:05:55.200346+0000","last_active":"2024-04-25T14:05:56.241295+0000","last_peered":"2024-04-25T14:05:56.241295+0000","last_clean":"2024-04-25T14:05:56.241295+0000","last_became_active":"2024-04-25T14:05:55.199215+0000","last_became_peered":"2024-04-25T14:05:55.199215+0000","last_unstale":"2024-04-25T14:05:56.241295+0000","last_undegraded":"2024-04-25T14:05:56.241295+0000","last_fullsized":"2024-04-25T14:05:56.241295+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:12.766009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180782+0000","last_change":"2024-04-25T14:05:33.560440+0000","last_active":"2024-04-25T14:05:56.180782+0000","last_peered":"2024-04-25T14:05:56.180782+0000","last_clean":"2024-04-25T14:05:56.180782+0000","last_became_active":"2024-04-25T14:05:33.560066+0000","last_became_peered":"2024-04-25T14:05:33.560066+0000","last_unstale":"2024-04-25T14:05:56.180782+0000","last_undegraded":"2024-04-25T14:05:56.180782+0000","last_fullsized":"2024-04-25T14:05:56.180782+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:32.906788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180496+0000","last_change":"2024-04-25T14:05:55.186492+0000","last_active":"2024-04-25T14:05:56.180496+0000","last_peered":"2024-04-25T14:05:56.180496+0000","last_clean":"2024-04-25T14:05:56.180496+0000","last_became_active":"2024-04-25T14:05:55.186183+0000","last_became_peered":"2024-04-25T14:05:55.186183+0000","last_unstale":"2024-04-25T14:05:56.180496+0000","last_undegraded":"2024-04-25T14:05:56.180496+0000","last_fullsized":"2024-04-25T14:05:56.180496+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:27.511725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":25,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.187035+0000","last_change":"2024-04-25T14:05:33.571248+0000","last_active":"2024-04-25T14:05:56.187035+0000","last_peered":"2024-04-25T14:05:56.187035+0000","last_clean":"2024-04-25T14:05:56.187035+0000","last_became_active":"2024-04-25T14:05:33.570916+0000","last_became_peered":"2024-04-25T14:05:33.570916+0000","last_unstale":"2024-04-25T14:05:56.187035+0000","last_undegraded":"2024-04-25T14:05:56.187035+0000","last_fullsized":"2024-04-25T14:05:56.187035+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:36.296676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.881719+0000","last_change":"2024-04-25T14:05:55.197821+0000","last_active":"2024-04-25T14:05:56.881719+0000","last_peered":"2024-04-25T14:05:56.881719+0000","last_clean":"2024-04-25T14:05:56.881719+0000","last_became_active":"2024-04-25T14:05:55.197337+0000","last_became_peered":"2024-04-25T14:05:55.197337+0000","last_unstale":"2024-04-25T14:05:56.881719+0000","last_undegraded":"2024-04-25T14:05:56.881719+0000","last_fullsized":"2024-04-25T14:05:56.881719+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:51.863500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":25,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.186754+0000","last_change":"2024-04-25T14:05:33.571072+0000","last_active":"2024-04-25T14:05:56.186754+0000","last_peered":"2024-04-25T14:05:56.186754+0000","last_clean":"2024-04-25T14:05:56.186754+0000","last_became_active":"2024-04-25T14:05:33.570748+0000","last_became_peered":"2024-04-25T14:05:33.570748+0000","last_unstale":"2024-04-25T14:05:56.186754+0000","last_undegraded":"2024-04-25T14:05:56.186754+0000","last_fullsized":"2024-04-25T14:05:56.186754+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:18.074490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.312942+0000","last_change":"2024-04-25T14:05:55.202255+0000","last_active":"2024-04-25T14:05:56.312942+0000","last_peered":"2024-04-25T14:05:56.312942+0000","last_clean":"2024-04-25T14:05:56.312942+0000","last_became_active":"2024-04-25T14:05:55.201879+0000","last_became_peered":"2024-04-25T14:05:55.201879+0000","last_unstale":"2024-04-25T14:05:56.312942+0000","last_undegraded":"2024-04-25T14:05:56.312942+0000","last_fullsized":"2024-04-25T14:05:56.312942+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:58:44.753232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833644+0000","last_change":"2024-04-25T14:05:33.574245+0000","last_active":"2024-04-25T14:05:56.833644+0000","last_peered":"2024-04-25T14:05:56.833644+0000","last_clean":"2024-04-25T14:05:56.833644+0000","last_became_active":"2024-04-25T14:05:33.573980+0000","last_became_peered":"2024-04-25T14:05:33.573980+0000","last_unstale":"2024-04-25T14:05:56.833644+0000","last_undegraded":"2024-04-25T14:05:56.833644+0000","last_fullsized":"2024-04-25T14:05:56.833644+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:30.240956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.181101+0000","last_change":"2024-04-25T14:05:55.212520+0000","last_active":"2024-04-25T14:05:56.181101+0000","last_peered":"2024-04-25T14:05:56.181101+0000","last_clean":"2024-04-25T14:05:56.181101+0000","last_became_active":"2024-04-25T14:05:55.212236+0000","last_became_peered":"2024-04-25T14:05:55.212236+0000","last_unstale":"2024-04-25T14:05:56.181101+0000","last_undegraded":"2024-04-25T14:05:56.181101+0000","last_fullsized":"2024-04-25T14:05:56.181101+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:07.087465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.242422+0000","last_change":"2024-04-25T14:05:33.580135+0000","last_active":"2024-04-25T14:05:56.242422+0000","last_peered":"2024-04-25T14:05:56.242422+0000","last_clean":"2024-04-25T14:05:56.242422+0000","last_became_active":"2024-04-25T14:05:33.579677+0000","last_became_peered":"2024-04-25T14:05:33.579677+0000","last_unstale":"2024-04-25T14:05:56.242422+0000","last_undegraded":"2024-04-25T14:05:56.242422+0000","last_fullsized":"2024-04-25T14:05:56.242422+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:20.253140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.187368+0000","last_change":"2024-04-25T14:05:55.207851+0000","last_active":"2024-04-25T14:05:56.187368+0000","last_peered":"2024-04-25T14:05:56.187368+0000","last_clean":"2024-04-25T14:05:56.187368+0000","last_became_active":"2024-04-25T14:05:55.204703+0000","last_became_peered":"2024-04-25T14:05:55.204703+0000","last_unstale":"2024-04-25T14:05:56.187368+0000","last_undegraded":"2024-04-25T14:05:56.187368+0000","last_fullsized":"2024-04-25T14:05:56.187368+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:13.208013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833751+0000","last_change":"2024-04-25T14:05:33.572330+0000","last_active":"2024-04-25T14:05:56.833751+0000","last_peered":"2024-04-25T14:05:56.833751+0000","last_clean":"2024-04-25T14:05:56.833751+0000","last_became_active":"2024-04-25T14:05:33.571842+0000","last_became_peered":"2024-04-25T14:05:33.571842+0000","last_unstale":"2024-04-25T14:05:56.833751+0000","last_undegraded":"2024-04-25T14:05:56.833751+0000","last_fullsized":"2024-04-25T14:05:56.833751+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:28.424318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.241660+0000","last_change":"2024-04-25T14:05:55.199125+0000","last_active":"2024-04-25T14:05:56.241660+0000","last_peered":"2024-04-25T14:05:56.241660+0000","last_clean":"2024-04-25T14:05:56.241660+0000","last_became_active":"2024-04-25T14:05:55.198751+0000","last_became_peered":"2024-04-25T14:05:55.198751+0000","last_unstale":"2024-04-25T14:05:56.241660+0000","last_undegraded":"2024-04-25T14:05:56.241660+0000","last_fullsized":"2024-04-25T14:05:56.241660+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:13.143182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"50'2","reported_seq":29,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.312239+0000","last_change":"2024-04-25T14:05:33.573271+0000","last_active":"2024-04-25T14:05:56.312239+0000","last_peered":"2024-04-25T14:05:56.312239+0000","last_clean":"2024-04-25T14:05:56.312239+0000","last_became_active":"2024-04-25T14:05:33.572858+0000","last_became_peered":"2024-04-25T14:05:33.572858+0000","last_unstale":"2024-04-25T14:05:56.312239+0000","last_undegraded":"2024-04-25T14:05:56.312239+0000","last_fullsized":"2024-04-25T14:05:56.312239+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:27.918457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.187732+0000","last_change":"2024-04-25T14:05:55.176165+0000","last_active":"2024-04-25T14:05:56.187732+0000","last_peered":"2024-04-25T14:05:56.187732+0000","last_clean":"2024-04-25T14:05:56.187732+0000","last_became_active":"2024-04-25T14:05:55.175277+0000","last_became_peered":"2024-04-25T14:05:55.175277+0000","last_unstale":"2024-04-25T14:05:56.187732+0000","last_undegraded":"2024-04-25T14:05:56.187732+0000","last_fullsized":"2024-04-25T14:05:56.187732+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:12.766009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.241644+0000","last_change":"2024-04-25T14:05:33.575396+0000","last_active":"2024-04-25T14:05:56.241644+0000","last_peered":"2024-04-25T14:05:56.241644+0000","last_clean":"2024-04-25T14:05:56.241644+0000","last_became_active":"2024-04-25T14:05:33.574772+0000","last_became_peered":"2024-04-25T14:05:33.574772+0000","last_unstale":"2024-04-25T14:05:56.241644+0000","last_undegraded":"2024-04-25T14:05:56.241644+0000","last_fullsized":"2024-04-25T14:05:56.241644+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:25.659996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:57.054786+0000","last_change":"2024-04-25T14:05:55.192624+0000","last_active":"2024-04-25T14:05:57.054786+0000","last_peered":"2024-04-25T14:05:57.054786+0000","last_clean":"2024-04-25T14:05:57.054786+0000","last_became_active":"2024-04-25T14:05:55.192246+0000","last_became_peered":"2024-04-25T14:05:55.192246+0000","last_unstale":"2024-04-25T14:05:57.054786+0000","last_undegraded":"2024-04-25T14:05:57.054786+0000","last_fullsized":"2024-04-25T14:05:57.054786+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:35.602166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833391+0000","last_change":"2024-04-25T14:05:33.581470+0000","last_active":"2024-04-25T14:05:56.833391+0000","last_peered":"2024-04-25T14:05:56.833391+0000","last_clean":"2024-04-25T14:05:56.833391+0000","last_became_active":"2024-04-25T14:05:33.581160+0000","last_became_peered":"2024-04-25T14:05:33.581160+0000","last_unstale":"2024-04-25T14:05:56.833391+0000","last_undegraded":"2024-04-25T14:05:56.833391+0000","last_fullsized":"2024-04-25T14:05:56.833391+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:47.818409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.241435+0000","last_change":"2024-04-25T14:05:55.198939+0000","last_active":"2024-04-25T14:05:56.241435+0000","last_peered":"2024-04-25T14:05:56.241435+0000","last_clean":"2024-04-25T14:05:56.241435+0000","last_became_active":"2024-04-25T14:05:55.198614+0000","last_became_peered":"2024-04-25T14:05:55.198614+0000","last_unstale":"2024-04-25T14:05:56.241435+0000","last_undegraded":"2024-04-25T14:05:56.241435+0000","last_fullsized":"2024-04-25T14:05:56.241435+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:26.936075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.312403+0000","last_change":"2024-04-25T14:05:33.572748+0000","last_active":"2024-04-25T14:05:56.312403+0000","last_peered":"2024-04-25T14:05:56.312403+0000","last_clean":"2024-04-25T14:05:56.312403+0000","last_became_active":"2024-04-25T14:05:33.572256+0000","last_became_peered":"2024-04-25T14:05:33.572256+0000","last_unstale":"2024-04-25T14:05:56.312403+0000","last_undegraded":"2024-04-25T14:05:56.312403+0000","last_fullsized":"2024-04-25T14:05:56.312403+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:43.410378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.242214+0000","last_change":"2024-04-25T14:05:55.208136+0000","last_active":"2024-04-25T14:05:56.242214+0000","last_peered":"2024-04-25T14:05:56.242214+0000","last_clean":"2024-04-25T14:05:56.242214+0000","last_became_active":"2024-04-25T14:05:55.204236+0000","last_became_peered":"2024-04-25T14:05:55.204236+0000","last_unstale":"2024-04-25T14:05:56.242214+0000","last_undegraded":"2024-04-25T14:05:56.242214+0000","last_fullsized":"2024-04-25T14:05:56.242214+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:14.069060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.181463+0000","last_change":"2024-04-25T14:05:33.565074+0000","last_active":"2024-04-25T14:05:56.181463+0000","last_peered":"2024-04-25T14:05:56.181463+0000","last_clean":"2024-04-25T14:05:56.181463+0000","last_became_active":"2024-04-25T14:05:33.564800+0000","last_became_peered":"2024-04-25T14:05:33.564800+0000","last_unstale":"2024-04-25T14:05:56.181463+0000","last_undegraded":"2024-04-25T14:05:56.181463+0000","last_fullsized":"2024-04-25T14:05:56.181463+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:00.811494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.242427+0000","last_change":"2024-04-25T14:05:55.205146+0000","last_active":"2024-04-25T14:05:56.242427+0000","last_peered":"2024-04-25T14:05:56.242427+0000","last_clean":"2024-04-25T14:05:56.242427+0000","last_became_active":"2024-04-25T14:05:55.203344+0000","last_became_peered":"2024-04-25T14:05:55.203344+0000","last_unstale":"2024-04-25T14:05:56.242427+0000","last_undegraded":"2024-04-25T14:05:56.242427+0000","last_fullsized":"2024-04-25T14:05:56.242427+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:27.511725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:57.055456+0000","last_change":"2024-04-25T14:05:33.573200+0000","last_active":"2024-04-25T14:05:57.055456+0000","last_peered":"2024-04-25T14:05:57.055456+0000","last_clean":"2024-04-25T14:05:57.055456+0000","last_became_active":"2024-04-25T14:05:33.572858+0000","last_became_peered":"2024-04-25T14:05:33.572858+0000","last_unstale":"2024-04-25T14:05:57.055456+0000","last_undegraded":"2024-04-25T14:05:57.055456+0000","last_fullsized":"2024-04-25T14:05:57.055456+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:22.996486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.187984+0000","last_change":"2024-04-25T14:05:55.186857+0000","last_active":"2024-04-25T14:05:56.187984+0000","last_peered":"2024-04-25T14:05:56.187984+0000","last_clean":"2024-04-25T14:05:56.187984+0000","last_became_active":"2024-04-25T14:05:55.186358+0000","last_became_peered":"2024-04-25T14:05:55.186358+0000","last_unstale":"2024-04-25T14:05:56.187984+0000","last_undegraded":"2024-04-25T14:05:56.187984+0000","last_fullsized":"2024-04-25T14:05:56.187984+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:56.230473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.312628+0000","last_change":"2024-04-25T14:05:33.579278+0000","last_active":"2024-04-25T14:05:56.312628+0000","last_peered":"2024-04-25T14:05:56.312628+0000","last_clean":"2024-04-25T14:05:56.312628+0000","last_became_active":"2024-04-25T14:05:33.578940+0000","last_became_peered":"2024-04-25T14:05:33.578940+0000","last_unstale":"2024-04-25T14:05:56.312628+0000","last_undegraded":"2024-04-25T14:05:56.312628+0000","last_fullsized":"2024-04-25T14:05:56.312628+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:01.444836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.881556+0000","last_change":"2024-04-25T14:05:55.196997+0000","last_active":"2024-04-25T14:05:56.881556+0000","last_peered":"2024-04-25T14:05:56.881556+0000","last_clean":"2024-04-25T14:05:56.881556+0000","last_became_active":"2024-04-25T14:05:55.196470+0000","last_became_peered":"2024-04-25T14:05:55.196470+0000","last_unstale":"2024-04-25T14:05:56.881556+0000","last_undegraded":"2024-04-25T14:05:56.881556+0000","last_fullsized":"2024-04-25T14:05:56.881556+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:13.208013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.181532+0000","last_change":"2024-04-25T14:05:33.560454+0000","last_active":"2024-04-25T14:05:56.181532+0000","last_peered":"2024-04-25T14:05:56.181532+0000","last_clean":"2024-04-25T14:05:56.181532+0000","last_became_active":"2024-04-25T14:05:33.560174+0000","last_became_peered":"2024-04-25T14:05:33.560174+0000","last_unstale":"2024-04-25T14:05:56.181532+0000","last_undegraded":"2024-04-25T14:05:56.181532+0000","last_fullsized":"2024-04-25T14:05:56.181532+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:07.917626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833399+0000","last_change":"2024-04-25T14:05:55.207186+0000","last_active":"2024-04-25T14:05:56.833399+0000","last_peered":"2024-04-25T14:05:56.833399+0000","last_clean":"2024-04-25T14:05:56.833399+0000","last_became_active":"2024-04-25T14:05:55.205535+0000","last_became_peered":"2024-04-25T14:05:55.205535+0000","last_unstale":"2024-04-25T14:05:56.833399+0000","last_undegraded":"2024-04-25T14:05:56.833399+0000","last_fullsized":"2024-04-25T14:05:56.833399+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:48.919549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.180448+0000","last_change":"2024-04-25T14:05:33.567123+0000","last_active":"2024-04-25T14:05:56.180448+0000","last_peered":"2024-04-25T14:05:56.180448+0000","last_clean":"2024-04-25T14:05:56.180448+0000","last_became_active":"2024-04-25T14:05:33.563225+0000","last_became_peered":"2024-04-25T14:05:33.563225+0000","last_unstale":"2024-04-25T14:05:56.180448+0000","last_undegraded":"2024-04-25T14:05:56.180448+0000","last_fullsized":"2024-04-25T14:05:56.180448+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:20.253140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.186648+0000","last_change":"2024-04-25T14:05:55.209615+0000","last_active":"2024-04-25T14:05:56.186648+0000","last_peered":"2024-04-25T14:05:56.186648+0000","last_clean":"2024-04-25T14:05:56.186648+0000","last_became_active":"2024-04-25T14:05:55.204910+0000","last_became_peered":"2024-04-25T14:05:55.204910+0000","last_unstale":"2024-04-25T14:05:56.186648+0000","last_undegraded":"2024-04-25T14:05:56.186648+0000","last_fullsized":"2024-04-25T14:05:56.186648+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:25.640028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.832870+0000","last_change":"2024-04-25T14:05:33.588123+0000","last_active":"2024-04-25T14:05:56.832870+0000","last_peered":"2024-04-25T14:05:56.832870+0000","last_clean":"2024-04-25T14:05:56.832870+0000","last_became_active":"2024-04-25T14:05:33.587848+0000","last_became_peered":"2024-04-25T14:05:33.587848+0000","last_unstale":"2024-04-25T14:05:56.832870+0000","last_undegraded":"2024-04-25T14:05:56.832870+0000","last_fullsized":"2024-04-25T14:05:56.832870+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:00.811494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.186210+0000","last_change":"2024-04-25T14:05:55.191912+0000","last_active":"2024-04-25T14:05:56.186210+0000","last_peered":"2024-04-25T14:05:56.186210+0000","last_clean":"2024-04-25T14:05:56.186210+0000","last_became_active":"2024-04-25T14:05:55.191605+0000","last_became_peered":"2024-04-25T14:05:55.191605+0000","last_unstale":"2024-04-25T14:05:56.186210+0000","last_undegraded":"2024-04-25T14:05:56.186210+0000","last_fullsized":"2024-04-25T14:05:56.186210+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:39.212550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.312868+0000","last_change":"2024-04-25T14:05:33.573091+0000","last_active":"2024-04-25T14:05:56.312868+0000","last_peered":"2024-04-25T14:05:56.312868+0000","last_clean":"2024-04-25T14:05:56.312868+0000","last_became_active":"2024-04-25T14:05:33.572800+0000","last_became_peered":"2024-04-25T14:05:33.572800+0000","last_unstale":"2024-04-25T14:05:56.312868+0000","last_undegraded":"2024-04-25T14:05:56.312868+0000","last_fullsized":"2024-04-25T14:05:56.312868+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:39.721568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:57.054944+0000","last_change":"2024-04-25T14:05:55.192600+0000","last_active":"2024-04-25T14:05:57.054944+0000","last_peered":"2024-04-25T14:05:57.054944+0000","last_clean":"2024-04-25T14:05:57.054944+0000","last_became_active":"2024-04-25T14:05:55.192221+0000","last_became_peered":"2024-04-25T14:05:55.192221+0000","last_unstale":"2024-04-25T14:05:57.054944+0000","last_undegraded":"2024-04-25T14:05:57.054944+0000","last_fullsized":"2024-04-25T14:05:57.054944+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:03.134903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":25,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.881156+0000","last_change":"2024-04-25T14:05:33.575718+0000","last_active":"2024-04-25T14:05:56.881156+0000","last_peered":"2024-04-25T14:05:56.881156+0000","last_clean":"2024-04-25T14:05:56.881156+0000","last_became_active":"2024-04-25T14:05:33.571491+0000","last_became_peered":"2024-04-25T14:05:33.571491+0000","last_unstale":"2024-04-25T14:05:56.881156+0000","last_undegraded":"2024-04-25T14:05:56.881156+0000","last_fullsized":"2024-04-25T14:05:56.881156+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:36.296676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.312816+0000","last_change":"2024-04-25T14:05:55.209013+0000","last_active":"2024-04-25T14:05:56.312816+0000","last_peered":"2024-04-25T14:05:56.312816+0000","last_clean":"2024-04-25T14:05:56.312816+0000","last_became_active":"2024-04-25T14:05:55.208467+0000","last_became_peered":"2024-04-25T14:05:55.208467+0000","last_unstale":"2024-04-25T14:05:56.312816+0000","last_undegraded":"2024-04-25T14:05:56.312816+0000","last_fullsized":"2024-04-25T14:05:56.312816+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:02.484939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.242080+0000","last_change":"2024-04-25T14:05:33.576013+0000","last_active":"2024-04-25T14:05:56.242080+0000","last_peered":"2024-04-25T14:05:56.242080+0000","last_clean":"2024-04-25T14:05:56.242080+0000","last_became_active":"2024-04-25T14:05:33.575693+0000","last_became_peered":"2024-04-25T14:05:33.575693+0000","last_unstale":"2024-04-25T14:05:56.242080+0000","last_undegraded":"2024-04-25T14:05:56.242080+0000","last_fullsized":"2024-04-25T14:05:56.242080+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:16.126866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.312878+0000","last_change":"2024-04-25T14:05:55.198113+0000","last_active":"2024-04-25T14:05:56.312878+0000","last_peered":"2024-04-25T14:05:56.312878+0000","last_clean":"2024-04-25T14:05:56.312878+0000","last_became_active":"2024-04-25T14:05:55.195434+0000","last_became_peered":"2024-04-25T14:05:55.195434+0000","last_unstale":"2024-04-25T14:05:56.312878+0000","last_undegraded":"2024-04-25T14:05:56.312878+0000","last_fullsized":"2024-04-25T14:05:56.312878+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:23.155887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.833639+0000","last_change":"2024-04-25T14:05:33.584460+0000","last_active":"2024-04-25T14:05:56.833639+0000","last_peered":"2024-04-25T14:05:56.833639+0000","last_clean":"2024-04-25T14:05:56.833639+0000","last_became_active":"2024-04-25T14:05:33.584178+0000","last_became_peered":"2024-04-25T14:05:33.584178+0000","last_unstale":"2024-04-25T14:05:56.833639+0000","last_undegraded":"2024-04-25T14:05:56.833639+0000","last_fullsized":"2024-04-25T14:05:56.833639+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:04.017485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.881755+0000","last_change":"2024-04-25T14:05:55.201466+0000","last_active":"2024-04-25T14:05:56.881755+0000","last_peered":"2024-04-25T14:05:56.881755+0000","last_clean":"2024-04-25T14:05:56.881755+0000","last_became_active":"2024-04-25T14:05:55.200903+0000","last_became_peered":"2024-04-25T14:05:55.200903+0000","last_unstale":"2024-04-25T14:05:56.881755+0000","last_undegraded":"2024-04-25T14:05:56.881755+0000","last_fullsized":"2024-04-25T14:05:56.881755+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:54.658020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.242446+0000","last_change":"2024-04-25T14:05:33.579643+0000","last_active":"2024-04-25T14:05:56.242446+0000","last_peered":"2024-04-25T14:05:56.242446+0000","last_clean":"2024-04-25T14:05:56.242446+0000","last_became_active":"2024-04-25T14:05:33.579334+0000","last_became_peered":"2024-04-25T14:05:33.579334+0000","last_unstale":"2024-04-25T14:05:56.242446+0000","last_undegraded":"2024-04-25T14:05:56.242446+0000","last_fullsized":"2024-04-25T14:05:56.242446+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:50.417159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.242150+0000","last_change":"2024-04-25T14:05:55.199672+0000","last_active":"2024-04-25T14:05:56.242150+0000","last_peered":"2024-04-25T14:05:56.242150+0000","last_clean":"2024-04-25T14:05:56.242150+0000","last_became_active":"2024-04-25T14:05:55.198888+0000","last_became_peered":"2024-04-25T14:05:55.198888+0000","last_unstale":"2024-04-25T14:05:56.242150+0000","last_undegraded":"2024-04-25T14:05:56.242150+0000","last_fullsized":"2024-04-25T14:05:56.242150+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:54.152159+0000","last_clean_scrub_stamp":"2024-04-25T14:05:54.152159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:53.903099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3f","version":"0'0","reported_seq":27,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T14:05:56.313043+0000","last_change":"2024-04-25T14:05:33.584174+0000","last_active":"2024-04-25T14:05:56.313043+0000","last_peered":"2024-04-25T14:05:56.313043+0000","last_clean":"2024-04-25T14:05:56.313043+0000","last_became_active":"2024-04-25T14:05:33.583867+0000","last_became_peered":"2024-04-25T14:05:33.583867+0000","last_unstale":"2024-04-25T14:05:56.313043+0000","last_undegraded":"2024-04-25T14:05:56.313043+0000","last_fullsized":"2024-04-25T14:05:56.313043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:05:32.529615+0000","last_clean_scrub_stamp":"2024-04-25T14:05:32.529615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:36.723420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":3145728,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":13866,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":82,"num_write":55,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":27732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53,"ondisk_log_size":53,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28256,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343584,"statfs":{"total":96636764160,"available":96607830016,"internally_reserved":0,"allocated":1261568,"data_stored":894238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":655360,"data_stored":303656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574879,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":647168,"data_stored":302992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28676,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343164,"statfs":{"total":96636764160,"available":96607399936,"internally_reserved":0,"allocated":1695744,"data_stored":1345793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29728,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342112,"statfs":{"total":96636764160,"available":96606322688,"internally_reserved":0,"allocated":2768896,"data_stored":2413212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":667648,"data_stored":316038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34920,"kb_used_data":3736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336920,"statfs":{"total":96636764160,"available":96601006080,"internally_reserved":0,"allocated":3825664,"data_stored":3451234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28228,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343612,"statfs":{"total":96636764160,"available":96607858688,"internally_reserved":0,"allocated":1232896,"data_stored":885447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":13068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":13090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3145728,"data_stored":3145728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:05:59.726 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T14:05:59.726 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T14:05:59.726 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T14:05:59.726 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T14:05:59.727 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:05:59.727 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T14:05:59.727 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T14:05:59.727 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T14:05:59.727 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:05:59.727 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T14:05:59.727 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T14:05:59.728 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T14:05:59.728 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:05:59.728 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T14:05:59.728 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T14:05:59.728 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T14:05:59.728 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:05:59.748 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T14:05:59.748 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T14:05:59.749 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T14:05:59.749 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T14:05:59.774 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T14:05:59.775 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T14:05:59.777 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T14:05:59.777 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T14:05:59.798 INFO:tasks.ceph.mds.b:Started 2024-04-25T14:05:59.798 DEBUG:teuthology.parallel:result is None 2024-04-25T14:05:59.800 INFO:tasks.ceph.mds.d:Started 2024-04-25T14:05:59.800 DEBUG:teuthology.parallel:result is None 2024-04-25T14:05:59.821 INFO:tasks.ceph.mds.c:Started 2024-04-25T14:05:59.822 DEBUG:teuthology.parallel:result is None 2024-04-25T14:05:59.823 INFO:tasks.ceph.mds.a:Started 2024-04-25T14:05:59.824 DEBUG:teuthology.parallel:result is None 2024-04-25T14:05:59.824 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:05:59.906 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2024-04-25T14:05:59.907 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2024-04-25T14:05:59.922 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-04-25T14:05:59.925 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-25T14:05:59.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.976+0000 7f002ac00640 1 Processor -- start 2024-04-25T14:05:59.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.976+0000 7f002ac00640 1 -- start start 2024-04-25T14:05:59.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.980+0000 7f002ac00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0024070330 0x7f00240d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:59.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.980+0000 7f002ac00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f00240d1c20 0x7f002406ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:59.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.980+0000 7f002ac00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00240d2030 con 0x7f002406f270 2024-04-25T14:05:59.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.980+0000 7f002ac00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f002415e5c0 con 0x7f00240d1c20 2024-04-25T14:05:59.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.980+0000 7f002ac00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f002415e8a0 con 0x7f0024070330 2024-04-25T14:05:59.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.980+0000 7f0028c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f002406f270 0x7f002406f680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50958/0 (socket says 172.21.15.86:50958) 2024-04-25T14:05:59.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.980+0000 7f0028c00640 1 -- 172.21.15.86:0/4210160007 learned_addr learned my addr 172.21.15.86:0/4210160007 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:05:59.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.980+0000 7f0023e00640 1 --2- 172.21.15.86:0/4210160007 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0024070330 0x7f00240d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:59.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.980+0000 7f001be00640 1 --2- 172.21.15.86:0/4210160007 >> v2:172.21.15.155:3300/0 conn(0x7f00240d1c20 0x7f002406ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:59.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.980+0000 7f0023400640 1 -- 172.21.15.86:0/4210160007 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1839466862 0 0) 0x7f001c0014a0 con 0x7f002406f270 2024-04-25T14:05:59.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.980+0000 7f0023400640 1 -- 172.21.15.86:0/4210160007 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0008003530 con 0x7f002406f270 2024-04-25T14:05:59.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.980+0000 7f0023e00640 1 -- 172.21.15.86:0/4210160007 >> v2:172.21.15.155:3300/0 conn(0x7f00240d1c20 msgr2=0x7f002406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:59.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.980+0000 7f0023e00640 1 --2- 172.21.15.86:0/4210160007 >> v2:172.21.15.155:3300/0 conn(0x7f00240d1c20 0x7f002406ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:59.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.980+0000 7f0023e00640 1 -- 172.21.15.86:0/4210160007 >> v1:172.21.15.86:6789/0 conn(0x7f002406f270 legacy=0x7f002406f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:59.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.980+0000 7f0023e00640 1 -- 172.21.15.86:0/4210160007 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f002415eb80 con 0x7f0024070330 2024-04-25T14:05:59.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.984+0000 7f0023e00640 1 --2- 172.21.15.86:0/4210160007 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0024070330 0x7f00240d16e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f00140097f0 tx=0x7f001401c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb8c0a9bffec465f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:59.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.984+0000 7f0023400640 1 -- 172.21.15.86:0/4210160007 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00140b0070 con 0x7f0024070330 2024-04-25T14:05:59.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.984+0000 7f0023400640 1 -- 172.21.15.86:0/4210160007 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00140ab600 con 0x7f0024070330 2024-04-25T14:05:59.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.984+0000 7f0023400640 1 -- 172.21.15.86:0/4210160007 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00140ab9d0 con 0x7f0024070330 2024-04-25T14:05:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.984+0000 7f002ac00640 1 -- 172.21.15.86:0/4210160007 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0024070330 msgr2=0x7f00240d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.984+0000 7f002ac00640 1 --2- 172.21.15.86:0/4210160007 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0024070330 0x7f00240d16e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f00140097f0 tx=0x7f001401c410 comp rx=0 tx=0).stop 2024-04-25T14:05:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.984+0000 7f002ac00640 1 -- 172.21.15.86:0/4210160007 shutdown_connections 2024-04-25T14:05:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.984+0000 7f002ac00640 1 --2- 172.21.15.86:0/4210160007 >> v2:172.21.15.155:3300/0 conn(0x7f00240d1c20 0x7f002406ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.984+0000 7f002ac00640 1 --2- 172.21.15.86:0/4210160007 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0024070330 0x7f00240d16e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.984+0000 7f002ac00640 1 -- 172.21.15.86:0/4210160007 >> 172.21.15.86:0/4210160007 conn(0x7f00240b4650 msgr2=0x7f00240b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.984+0000 7f002ac00640 1 -- 172.21.15.86:0/4210160007 shutdown_connections 2024-04-25T14:05:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.984+0000 7f002ac00640 1 -- 172.21.15.86:0/4210160007 wait complete. 2024-04-25T14:05:59.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.988+0000 7f002ac00640 1 Processor -- start 2024-04-25T14:05:59.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.988+0000 7f002ac00640 1 -- start start 2024-04-25T14:05:59.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.988+0000 7f002ac00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f002406f270 0x7f0024072880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:59.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.988+0000 7f002ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0024070330 0x7f0024154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:59.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.988+0000 7f002ac00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00240d1c20 0x7f0024159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:59.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.988+0000 7f002ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f00240708c0 con 0x7f0024070330 2024-04-25T14:05:59.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.988+0000 7f002ac00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0024070740 con 0x7f00240d1c20 2024-04-25T14:05:59.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.988+0000 7f002ac00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f002416afc0 con 0x7f002406f270 2024-04-25T14:05:59.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.988+0000 7f0023e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f002406f270 0x7f0024072880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:59.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.988+0000 7f001be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0024070330 0x7f0024154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:59.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.988+0000 7f0023e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f002406f270 0x7f0024072880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:59332/0 (socket says 172.21.15.86:59332) 2024-04-25T14:05:59.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.988+0000 7f0023e00640 1 -- 172.21.15.86:0/676591312 learned_addr learned my addr 172.21.15.86:0/676591312 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:05:59.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.988+0000 7f0028c00640 1 --2- 172.21.15.86:0/676591312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00240d1c20 0x7f0024159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:59.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.988+0000 7f0023e00640 1 -- 172.21.15.86:0/676591312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00240d1c20 msgr2=0x7f0024159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:59.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.988+0000 7f0023e00640 1 --2- 172.21.15.86:0/676591312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00240d1c20 0x7f0024159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:59.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.988+0000 7f0023e00640 1 -- 172.21.15.86:0/676591312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0024070330 msgr2=0x7f0024154bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:59.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.988+0000 7f0023e00640 1 --2- 172.21.15.86:0/676591312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0024070330 0x7f0024154bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:59.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.988+0000 7f0023e00640 1 -- 172.21.15.86:0/676591312 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00240c8ff0 con 0x7f002406f270 2024-04-25T14:05:59.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.988+0000 7f001be00640 1 --2- 172.21.15.86:0/676591312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0024070330 0x7f0024154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:59.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.988+0000 7f0023e00640 1 --2- 172.21.15.86:0/676591312 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f002406f270 0x7f0024072880 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f001401a040 tx=0x7f001401ac50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.988+0000 7f0020c00640 1 -- 172.21.15.86:0/676591312 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00140b0050 con 0x7f002406f270 2024-04-25T14:05:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.988+0000 7f0020c00640 1 -- 172.21.15.86:0/676591312 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00140bc070 con 0x7f002406f270 2024-04-25T14:05:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.988+0000 7f0020c00640 1 -- 172.21.15.86:0/676591312 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0014006e20 con 0x7f002406f270 2024-04-25T14:05:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.988+0000 7f002ac00640 1 -- 172.21.15.86:0/676591312 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0024072dc0 con 0x7f002406f270 2024-04-25T14:05:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.988+0000 7f002ac00640 1 -- 172.21.15.86:0/676591312 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f00240cf6f0 con 0x7f002406f270 2024-04-25T14:05:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.992+0000 7f002ac00640 1 -- 172.21.15.86:0/676591312 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0024069d00 con 0x7f002406f270 2024-04-25T14:05:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.992+0000 7f0020c00640 1 -- 172.21.15.86:0/676591312 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f00140b5830 con 0x7f002406f270 2024-04-25T14:05:59.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.992+0000 7f0020c00640 1 --2- 172.21.15.86:0/676591312 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f000006f5a0 0x7f0000071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:59.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.992+0000 7f0020c00640 1 -- 172.21.15.86:0/676591312 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(54..54 src has 1..54) ==== 7359+0+0 (secure 0 0 0) 0x7f001412a3d0 con 0x7f002406f270 2024-04-25T14:05:59.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.992+0000 7f001be00640 1 --2- 172.21.15.86:0/676591312 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f000006f5a0 0x7f0000071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:59.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.992+0000 7f001be00640 1 --2- 172.21.15.86:0/676591312 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f000006f5a0 0x7f0000071ab0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f002406da70 tx=0x7f000c009190 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:59.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:05:59.996+0000 7f0020c00640 1 -- 172.21.15.86:0/676591312 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0014016090 con 0x7f002406f270 2024-04-25T14:06:00.057 INFO:tasks.ceph.mds.a.smithi086.stderr:2024-04-25T14:06:00.056+0000 7f33d2a00640 -1 --2- [v2:172.21.15.86:6836/766530004,v1:172.21.15.86:6837/766530004] >> conn(0x561b23f4e480 0x561b23265080 unknown :-1 s=BANNER_ACCEPTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer is using msgr V1 protocol 2024-04-25T14:06:00.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.188+0000 7f002ac00640 1 -- 172.21.15.86:0/676591312 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0024076bf0 con 0x7f002406f270 2024-04-25T14:06:00.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.188+0000 7f0020c00640 1 -- 172.21.15.86:0/676591312 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+6494 (secure 0 0 0) 0x7f00140f6450 con 0x7f002406f270 2024-04-25T14:06:00.192 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:00.192 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5315,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3312926518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":3312926518},{"type":"v1","addr":"172.21.15.155:6835","nonce":3312926518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5776,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/898730495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":898730495},{"type":"v1","addr":"172.21.15.155:6834","nonce":898730495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:32.946320+0000","modified":"2024-04-25T14:05:45.906925+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5874},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5874":{"gid":5874,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6836/1732615622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1732615622},{"type":"v1","addr":"172.21.15.86:6836","nonce":1732615622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"fs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5874,"qdb_cluster":[5874]},"id":3},{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:54.565528+0000","modified":"2024-04-25T14:05:57.592887+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5794":{"gid":5794,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":5,"addr":"172.21.15.86:6837/2418778265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2418778265},{"type":"v1","addr":"172.21.15.86:6837","nonce":2418778265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"fs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5794,"qdb_cluster":[5794]},"id":4}]} 2024-04-25T14:06:00.192 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 36 2024-04-25T14:06:00.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.192+0000 7f002ac00640 1 -- 172.21.15.86:0/676591312 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f000006f5a0 msgr2=0x7f0000071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:00.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.192+0000 7f002ac00640 1 --2- 172.21.15.86:0/676591312 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f000006f5a0 0x7f0000071ab0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f002406da70 tx=0x7f000c009190 comp rx=0 tx=0).stop 2024-04-25T14:06:00.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.192+0000 7f002ac00640 1 -- 172.21.15.86:0/676591312 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f002406f270 msgr2=0x7f0024072880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:00.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.192+0000 7f002ac00640 1 --2- 172.21.15.86:0/676591312 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f002406f270 0x7f0024072880 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f001401a040 tx=0x7f001401ac50 comp rx=0 tx=0).stop 2024-04-25T14:06:00.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.192+0000 7f002ac00640 1 -- 172.21.15.86:0/676591312 shutdown_connections 2024-04-25T14:06:00.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.192+0000 7f002ac00640 1 --2- 172.21.15.86:0/676591312 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f000006f5a0 0x7f0000071ab0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.192+0000 7f002ac00640 1 --2- 172.21.15.86:0/676591312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00240d1c20 0x7f0024159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.192+0000 7f002ac00640 1 --2- 172.21.15.86:0/676591312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0024070330 0x7f0024154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.192+0000 7f002ac00640 1 --2- 172.21.15.86:0/676591312 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f002406f270 0x7f0024072880 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.192+0000 7f002ac00640 1 -- 172.21.15.86:0/676591312 >> 172.21.15.86:0/676591312 conn(0x7f00240b4650 msgr2=0x7f002415bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:00.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.192+0000 7f002ac00640 1 -- 172.21.15.86:0/676591312 shutdown_connections 2024-04-25T14:06:00.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.192+0000 7f002ac00640 1 -- 172.21.15.86:0/676591312 wait complete. 2024-04-25T14:06:00.221 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-25T14:06:00.220+0000 7fea26a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:06:00.621 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:05:54.565528+0000', 'modified': '2024-04-25T14:05:57.592887+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5794}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5794': {'gid': 5794, 'name': 'c', 'rank': 0, 'incarnation': 33, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.86:6837/2418778265', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6835', 'nonce': 2418778265}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 2418778265}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'fs2', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5794, 'qdb_cluster': [5794]} 2024-04-25T14:06:00.622 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T14:06:00.622 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T14:06:00.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.768+0000 7fd687c00640 1 Processor -- start 2024-04-25T14:06:00.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.768+0000 7fd687c00640 1 -- start start 2024-04-25T14:06:00.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.772+0000 7fd687c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd68015faa0 0x7fd68006d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:00.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.772+0000 7fd687c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd6800aeea0 0x7fd6800af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:00.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.772+0000 7fd687c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd68006c670 con 0x7fd6800b59e0 2024-04-25T14:06:00.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.772+0000 7fd687c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd68006d560 con 0x7fd68015faa0 2024-04-25T14:06:00.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.772+0000 7fd687c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd68006c950 con 0x7fd6800aeea0 2024-04-25T14:06:00.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.772+0000 7fd685200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd68015faa0 0x7fd68006d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:00.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.772+0000 7fd67fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd6800b59e0 0x7fd6800b5df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50960/0 (socket says 172.21.15.86:50960) 2024-04-25T14:06:00.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.772+0000 7fd67fe00640 1 -- 172.21.15.86:0/2350695364 learned_addr learned my addr 172.21.15.86:0/2350695364 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:00.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.772+0000 7fd685c00640 1 --2- 172.21.15.86:0/2350695364 >> v2:172.21.15.155:3300/0 conn(0x7fd6800aeea0 0x7fd6800af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:00.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.772+0000 7fd685200640 1 -- 172.21.15.86:0/2350695364 >> v2:172.21.15.155:3300/0 conn(0x7fd6800aeea0 msgr2=0x7fd6800af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:00.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.772+0000 7fd685200640 1 --2- 172.21.15.86:0/2350695364 >> v2:172.21.15.155:3300/0 conn(0x7fd6800aeea0 0x7fd6800af2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.772+0000 7fd685200640 1 -- 172.21.15.86:0/2350695364 >> v1:172.21.15.86:6789/0 conn(0x7fd6800b59e0 legacy=0x7fd6800b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:00.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.772+0000 7fd685200640 1 -- 172.21.15.86:0/2350695364 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68006cc30 con 0x7fd68015faa0 2024-04-25T14:06:00.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.776+0000 7fd685200640 1 --2- 172.21.15.86:0/2350695364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd68015faa0 0x7fd68006d9f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fd6800765a0 tx=0x7fd67001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a0e4936b79f45c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:00.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.776+0000 7fd67f400640 1 -- 172.21.15.86:0/2350695364 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6700b2070 con 0x7fd68015faa0 2024-04-25T14:06:00.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.776+0000 7fd67f400640 1 -- 172.21.15.86:0/2350695364 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6700a59f0 con 0x7fd68015faa0 2024-04-25T14:06:00.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.776+0000 7fd67f400640 1 -- 172.21.15.86:0/2350695364 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6700a5dc0 con 0x7fd68015faa0 2024-04-25T14:06:00.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.776+0000 7fd687c00640 1 -- 172.21.15.86:0/2350695364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd68015faa0 msgr2=0x7fd68006d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:00.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.776+0000 7fd687c00640 1 --2- 172.21.15.86:0/2350695364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd68015faa0 0x7fd68006d9f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fd6800765a0 tx=0x7fd67001c930 comp rx=0 tx=0).stop 2024-04-25T14:06:00.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.776+0000 7fd687c00640 1 -- 172.21.15.86:0/2350695364 shutdown_connections 2024-04-25T14:06:00.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.776+0000 7fd687c00640 1 --2- 172.21.15.86:0/2350695364 >> v2:172.21.15.155:3300/0 conn(0x7fd6800aeea0 0x7fd6800af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.776+0000 7fd687c00640 1 --2- 172.21.15.86:0/2350695364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd68015faa0 0x7fd68006d9f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.776+0000 7fd687c00640 1 -- 172.21.15.86:0/2350695364 >> 172.21.15.86:0/2350695364 conn(0x7fd6800d2230 msgr2=0x7fd6800d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:00.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.776+0000 7fd687c00640 1 -- 172.21.15.86:0/2350695364 shutdown_connections 2024-04-25T14:06:00.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.776+0000 7fd687c00640 1 -- 172.21.15.86:0/2350695364 wait complete. 2024-04-25T14:06:00.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.780+0000 7fd687c00640 1 Processor -- start 2024-04-25T14:06:00.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.780+0000 7fd687c00640 1 -- start start 2024-04-25T14:06:00.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.780+0000 7fd687c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6800aeea0 0x7fd680162be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:00.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.780+0000 7fd687c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6800b59e0 0x7fd68014ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:00.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.780+0000 7fd687c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd68015faa0 0x7fd6801574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:00.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.780+0000 7fd687c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd680160030 con 0x7fd68015faa0 2024-04-25T14:06:00.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.780+0000 7fd687c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd68015feb0 con 0x7fd6800aeea0 2024-04-25T14:06:00.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.780+0000 7fd687c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd680070f30 con 0x7fd6800b59e0 2024-04-25T14:06:00.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.780+0000 7fd685c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd68015faa0 0x7fd6801574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:00.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.780+0000 7fd685c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd68015faa0 0x7fd6801574f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41490/0 (socket says 172.21.15.86:41490) 2024-04-25T14:06:00.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.780+0000 7fd685c00640 1 -- 172.21.15.86:0/3502062531 learned_addr learned my addr 172.21.15.86:0/3502062531 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:00.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.780+0000 7fd67fe00640 1 --2- 172.21.15.86:0/3502062531 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6800b59e0 0x7fd68014ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:00.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.780+0000 7fd685200640 1 --2- 172.21.15.86:0/3502062531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6800aeea0 0x7fd680162be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:00.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.780+0000 7fd685c00640 1 -- 172.21.15.86:0/3502062531 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6800b59e0 msgr2=0x7fd68014ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:00.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.780+0000 7fd685c00640 1 --2- 172.21.15.86:0/3502062531 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6800b59e0 0x7fd68014ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.780+0000 7fd685c00640 1 -- 172.21.15.86:0/3502062531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6800aeea0 msgr2=0x7fd680162be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:00.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.780+0000 7fd685c00640 1 --2- 172.21.15.86:0/3502062531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6800aeea0 0x7fd680162be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.780+0000 7fd685c00640 1 -- 172.21.15.86:0/3502062531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68014dee0 con 0x7fd68015faa0 2024-04-25T14:06:00.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.780+0000 7fd685c00640 1 --2- 172.21.15.86:0/3502062531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd68015faa0 0x7fd6801574f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd67800d590 tx=0x7fd67800da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:00.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.780+0000 7fd67cc00640 1 -- 172.21.15.86:0/3502062531 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6780125f0 con 0x7fd68015faa0 2024-04-25T14:06:00.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.780+0000 7fd67cc00640 1 -- 172.21.15.86:0/3502062531 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd678019070 con 0x7fd68015faa0 2024-04-25T14:06:00.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.780+0000 7fd687c00640 1 -- 172.21.15.86:0/3502062531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd68014e180 con 0x7fd68015faa0 2024-04-25T14:06:00.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.780+0000 7fd67cc00640 1 -- 172.21.15.86:0/3502062531 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6780129f0 con 0x7fd68015faa0 2024-04-25T14:06:00.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.780+0000 7fd687c00640 1 -- 172.21.15.86:0/3502062531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd68014f5b0 con 0x7fd68015faa0 2024-04-25T14:06:00.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.784+0000 7fd67cc00640 1 -- 172.21.15.86:0/3502062531 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7fd678012b90 con 0x7fd68015faa0 2024-04-25T14:06:00.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.784+0000 7fd67cc00640 1 --2- 172.21.15.86:0/3502062531 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fd65c06f570 0x7fd65c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:00.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.784+0000 7fd67cc00640 1 -- 172.21.15.86:0/3502062531 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7fd67808fea0 con 0x7fd68015faa0 2024-04-25T14:06:00.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.784+0000 7fd685200640 1 --2- 172.21.15.86:0/3502062531 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fd65c06f570 0x7fd65c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:00.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.784+0000 7fd68f1b0000 1 Processor -- start 2024-04-25T14:06:00.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.784+0000 7fd68f1b0000 1 -- start start 2024-04-25T14:06:00.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd685200640 1 --2- 172.21.15.86:0/3502062531 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fd65c06f570 0x7fd65c071a80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd6700af000 tx=0x7fd6700a7070 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:00.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd68f1b0000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399d32ac20 0x56399d488ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:00.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd68f1b0000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56399d316ea0 0x56399d489420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:00.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd68f1b0000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56399cf0f280 0x56399d489960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:00.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd68f1b0000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56399d2f8160 con 0x56399d316ea0 2024-04-25T14:06:00.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd68f1b0000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56399d28de20 con 0x56399d32ac20 2024-04-25T14:06:00.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd68f1b0000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x56399d246a20 con 0x56399cf0f280 2024-04-25T14:06:00.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd67fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56399d316ea0 0x56399d489420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:00.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd67fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56399d316ea0 0x56399d489420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41492/0 (socket says 172.21.15.86:41492) 2024-04-25T14:06:00.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd67fe00640 1 -- 172.21.15.86:0/4172475714 learned_addr learned my addr 172.21.15.86:0/4172475714 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:00.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd67fe00640 1 -- 172.21.15.86:0/4172475714 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56399cf0f280 msgr2=0x56399d489960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:00.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd685200640 1 --2- 172.21.15.86:0/4172475714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399d32ac20 0x56399d488ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:00.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd685c00640 1 --2- 172.21.15.86:0/4172475714 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56399cf0f280 0x56399d489960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:00.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd67fe00640 1 --2- 172.21.15.86:0/4172475714 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56399cf0f280 0x56399d489960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd67fe00640 1 -- 172.21.15.86:0/4172475714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399d32ac20 msgr2=0x56399d488ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:00.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd67fe00640 1 --2- 172.21.15.86:0/4172475714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399d32ac20 0x56399d488ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd67fe00640 1 -- 172.21.15.86:0/4172475714 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56399d24b100 con 0x56399d316ea0 2024-04-25T14:06:00.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd685200640 1 --2- 172.21.15.86:0/4172475714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399d32ac20 0x56399d488ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:00.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd685c00640 1 --2- 172.21.15.86:0/4172475714 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56399cf0f280 0x56399d489960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:00.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd67fe00640 1 --2- 172.21.15.86:0/4172475714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56399d316ea0 0x56399d489420 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd668004e90 tx=0x7fd668007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4086ea4db1c377 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:00.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd675600640 1 -- 172.21.15.86:0/4172475714 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668001730 con 0x56399d316ea0 2024-04-25T14:06:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd675600640 1 -- 172.21.15.86:0/4172475714 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd668018070 con 0x56399d316ea0 2024-04-25T14:06:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd675600640 1 -- 172.21.15.86:0/4172475714 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668001ad0 con 0x56399d316ea0 2024-04-25T14:06:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd68f1b0000 1 -- 172.21.15.86:0/4172475714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56399d316ea0 msgr2=0x56399d489420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd68f1b0000 1 --2- 172.21.15.86:0/4172475714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56399d316ea0 0x56399d489420 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd668004e90 tx=0x7fd668007430 comp rx=0 tx=0).stop 2024-04-25T14:06:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd68f1b0000 1 -- 172.21.15.86:0/4172475714 shutdown_connections 2024-04-25T14:06:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd68f1b0000 1 --2- 172.21.15.86:0/4172475714 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56399cf0f280 0x56399d489960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd68f1b0000 1 --2- 172.21.15.86:0/4172475714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56399d316ea0 0x56399d489420 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd68f1b0000 1 --2- 172.21.15.86:0/4172475714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399d32ac20 0x56399d488ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd68f1b0000 1 -- 172.21.15.86:0/4172475714 >> 172.21.15.86:0/4172475714 conn(0x56399cf4eff0 msgr2=0x56399d43e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd68f1b0000 1 -- 172.21.15.86:0/4172475714 shutdown_connections 2024-04-25T14:06:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.788+0000 7fd68f1b0000 1 -- 172.21.15.86:0/4172475714 wait complete. 2024-04-25T14:06:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.792+0000 7fd68f1b0000 1 -- start start 2024-04-25T14:06:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.792+0000 7fd68f1b0000 1 Processor -- start 2024-04-25T14:06:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.792+0000 7fd68f1b0000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56399cf0f280 0x56399d490070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.792+0000 7fd68f1b0000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399d316ea0 0x56399d4905b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.792+0000 7fd68f1b0000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56399d32ac20 0x56399d44b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.792+0000 7fd68f1b0000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56399d29b220 con 0x56399d32ac20 2024-04-25T14:06:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.792+0000 7fd68f1b0000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56399d231ea0 con 0x56399d316ea0 2024-04-25T14:06:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.792+0000 7fd68f1b0000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x56399d288b00 con 0x56399cf0f280 2024-04-25T14:06:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.792+0000 7fd68f1b0000 10 client.0 _ll_register_callbacks cb 0x56399d29adc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.792+0000 7fd685c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56399d32ac20 0x56399d44b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.792+0000 7fd685c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56399d32ac20 0x56399d44b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41494/0 (socket says 172.21.15.86:41494) 2024-04-25T14:06:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.792+0000 7fd685c00640 1 -- 172.21.15.86:0/345009238 learned_addr learned my addr 172.21.15.86:0/345009238 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.792+0000 7fd67fe00640 1 --2- 172.21.15.86:0/345009238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399d316ea0 0x56399d4905b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.792+0000 7fd685200640 1 --2- 172.21.15.86:0/345009238 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56399cf0f280 0x56399d490070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.792+0000 7fd685c00640 1 -- 172.21.15.86:0/345009238 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56399cf0f280 msgr2=0x56399d490070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.792+0000 7fd685c00640 1 --2- 172.21.15.86:0/345009238 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56399cf0f280 0x56399d490070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.792+0000 7fd685c00640 1 -- 172.21.15.86:0/345009238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399d316ea0 msgr2=0x56399d4905b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.792+0000 7fd685c00640 1 --2- 172.21.15.86:0/345009238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399d316ea0 0x56399d4905b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.792+0000 7fd685c00640 1 -- 172.21.15.86:0/345009238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56399d22d500 con 0x56399d32ac20 2024-04-25T14:06:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.792+0000 7fd685c00640 1 --2- 172.21.15.86:0/345009238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56399d32ac20 0x56399d44b0c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd678059170 tx=0x7fd67805b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc94f6e5bd6ae5b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.792+0000 7fd66e000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:06:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.792+0000 7fd66e000640 1 -- 172.21.15.86:0/345009238 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67808b060 con 0x56399d32ac20 2024-04-25T14:06:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.792+0000 7fd66e000640 1 -- 172.21.15.86:0/345009238 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7fd678062410 con 0x56399d32ac20 2024-04-25T14:06:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.792+0000 7fd68f1b0000 1 -- 172.21.15.86:0/345009238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd668018070 con 0x56399d32ac20 2024-04-25T14:06:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.792+0000 7fd66e000640 1 -- 172.21.15.86:0/345009238 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd67807a050 con 0x56399d32ac20 2024-04-25T14:06:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.792+0000 7fd68f1b0000 10 client.6072 fetch_fsmap learned FSMap version 40 2024-04-25T14:06:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.792+0000 7fd68f1b0000 1 -- 172.21.15.86:0/345009238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x56399d213ea0 con 0x56399d32ac20 2024-04-25T14:06:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.796+0000 7fd66e000640 1 -- 172.21.15.86:0/345009238 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd668018070 con 0x56399d32ac20 2024-04-25T14:06:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.796+0000 7fd66e000640 1 -- 172.21.15.86:0/345009238 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 40) ==== 4116+0+0 (secure 0 0 0) 0x7fd678013070 con 0x56399d32ac20 2024-04-25T14:06:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.796+0000 7fd68f1b0000 10 client.6072 fetch_fsmap finished waiting for FSMap version 40 2024-04-25T14:06:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.796+0000 7fd68f1b0000 10 client.6072 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T14:06:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.796+0000 7fd68f1b0000 20 client.6072 populate_metadata read hostname 'smithi086' 2024-04-25T14:06:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.796+0000 7fd68f1b0000 1 --2- 172.21.15.86:0/345009238 >> [v2:172.21.15.86:6834/2169924270,v1:172.21.15.86:6835/2169924270] conn(0x56399cf337d0 0x56399d44be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.796+0000 7fd68f1b0000 4 client.6072 mds_command: new command op to 6048 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:00.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.796+0000 7fd68f1b0000 1 -- 172.21.15.86:0/345009238 --> [v2:172.21.15.86:6834/2169924270,v1:172.21.15.86:6835/2169924270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56399d226be0 con 0x56399cf337d0 2024-04-25T14:06:00.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.796+0000 7fd685200640 1 --2- 172.21.15.86:0/345009238 >> [v2:172.21.15.86:6834/2169924270,v1:172.21.15.86:6835/2169924270] conn(0x56399cf337d0 0x56399d44be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:00.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.796+0000 7fd685200640 1 --2- 172.21.15.86:0/345009238 >> [v2:172.21.15.86:6834/2169924270,v1:172.21.15.86:6835/2169924270] conn(0x56399cf337d0 0x56399d44be00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2970af46a4446baf server_cookie=6a54ee58b285e465 in_seq=0 out_seq=0 2024-04-25T14:06:00.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.796+0000 7fd66e000640 10 client.6072 ms_handle_connect on v2:172.21.15.86:6834/2169924270 2024-04-25T14:06:00.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.796+0000 7fd66e000640 1 -- 172.21.15.86:0/345009238 <== mds.0 v2:172.21.15.86:6834/2169924270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56399d226be0 con 0x56399cf337d0 2024-04-25T14:06:00.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.796+0000 7fd66e000640 10 client.6072 handle_command_reply: tid=0 2024-04-25T14:06:00.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.796+0000 7fd68f1b0000 1 client.6072 shutdown 2024-04-25T14:06:00.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.796+0000 7fd68f1b0000 1 -- 172.21.15.86:0/345009238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56399d32ac20 msgr2=0x56399d44b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:00.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.796+0000 7fd68f1b0000 1 --2- 172.21.15.86:0/345009238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56399d32ac20 0x56399d44b0c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd678059170 tx=0x7fd67805b770 comp rx=0 tx=0).stop 2024-04-25T14:06:00.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.796+0000 7fd68f1b0000 1 -- 172.21.15.86:0/345009238 shutdown_connections 2024-04-25T14:06:00.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.796+0000 7fd68f1b0000 1 --2- 172.21.15.86:0/345009238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56399d32ac20 0x56399d44b0c0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.796+0000 7fd68f1b0000 1 --2- 172.21.15.86:0/345009238 >> [v2:172.21.15.86:6834/2169924270,v1:172.21.15.86:6835/2169924270] conn(0x56399cf337d0 0x56399d44be00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.796+0000 7fd68f1b0000 1 --2- 172.21.15.86:0/345009238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399d316ea0 0x56399d4905b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.796+0000 7fd68f1b0000 1 --2- 172.21.15.86:0/345009238 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56399cf0f280 0x56399d490070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.796+0000 7fd66e000640 0 client.6072 ms_handle_reset on v2:172.21.15.86:6834/2169924270 2024-04-25T14:06:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.796+0000 7fd68f1b0000 1 -- 172.21.15.86:0/345009238 >> 172.21.15.86:0/345009238 conn(0x56399cf4eff0 msgr2=0x56399d43ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.796+0000 7fd68f1b0000 1 -- 172.21.15.86:0/345009238 shutdown_connections 2024-04-25T14:06:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.796+0000 7fd68f1b0000 1 -- 172.21.15.86:0/345009238 wait complete. 2024-04-25T14:06:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.796+0000 7fd68f1b0000 20 client.6072 trim_cache size 0 max 16384 2024-04-25T14:06:00.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.816+0000 7fd68f1b0000 1 Processor -- start 2024-04-25T14:06:00.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.816+0000 7fd68f1b0000 1 -- start start 2024-04-25T14:06:00.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.816+0000 7fd68f1b0000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399cf0f280 0x56399d4a0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:00.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.816+0000 7fd68f1b0000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56399d316ea0 0x56399d4a0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:00.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.816+0000 7fd68f1b0000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56399d32ac20 0x56399d48e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:00.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.816+0000 7fd68f1b0000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6802097d0 con 0x56399d32ac20 2024-04-25T14:06:00.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.816+0000 7fd68f1b0000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd6802072f0 con 0x56399cf0f280 2024-04-25T14:06:00.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.816+0000 7fd68f1b0000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd6802078f0 con 0x56399d316ea0 2024-04-25T14:06:00.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.816+0000 7fd685c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56399d32ac20 0x56399d48e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:00.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.816+0000 7fd685c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56399d32ac20 0x56399d48e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41498/0 (socket says 172.21.15.86:41498) 2024-04-25T14:06:00.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.816+0000 7fd685c00640 1 -- 172.21.15.86:0/3615605650 learned_addr learned my addr 172.21.15.86:0/3615605650 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:00.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.816+0000 7fd685c00640 1 -- 172.21.15.86:0/3615605650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56399d316ea0 msgr2=0x56399d4a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:00.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.816+0000 7fd685c00640 1 --2- 172.21.15.86:0/3615605650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56399d316ea0 0x56399d4a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.816+0000 7fd685c00640 1 -- 172.21.15.86:0/3615605650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399cf0f280 msgr2=0x56399d4a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:00.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.816+0000 7fd685c00640 1 --2- 172.21.15.86:0/3615605650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399cf0f280 0x56399d4a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.816+0000 7fd685c00640 1 -- 172.21.15.86:0/3615605650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56399d442840 con 0x56399d32ac20 2024-04-25T14:06:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.816+0000 7fd685c00640 1 --2- 172.21.15.86:0/3615605650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56399d32ac20 0x56399d48e010 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd67800df50 tx=0x7fd67805e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71788fb07e629728 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.816+0000 7fd674c00640 1 -- 172.21.15.86:0/3615605650 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678080070 con 0x56399d32ac20 2024-04-25T14:06:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.816+0000 7fd674c00640 1 -- 172.21.15.86:0/3615605650 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd67805e070 con 0x56399d32ac20 2024-04-25T14:06:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.816+0000 7fd674c00640 1 -- 172.21.15.86:0/3615605650 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678057dc0 con 0x56399d32ac20 2024-04-25T14:06:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.820+0000 7fd68f1b0000 1 -- 172.21.15.86:0/3615605650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56399d32ac20 msgr2=0x56399d48e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.820+0000 7fd68f1b0000 1 --2- 172.21.15.86:0/3615605650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56399d32ac20 0x56399d48e010 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd67800df50 tx=0x7fd67805e000 comp rx=0 tx=0).stop 2024-04-25T14:06:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.820+0000 7fd68f1b0000 1 -- 172.21.15.86:0/3615605650 shutdown_connections 2024-04-25T14:06:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.820+0000 7fd68f1b0000 1 --2- 172.21.15.86:0/3615605650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56399d32ac20 0x56399d48e010 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.820+0000 7fd68f1b0000 1 --2- 172.21.15.86:0/3615605650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56399d316ea0 0x56399d4a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.820+0000 7fd68f1b0000 1 --2- 172.21.15.86:0/3615605650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399cf0f280 0x56399d4a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.820+0000 7fd68f1b0000 1 -- 172.21.15.86:0/3615605650 >> 172.21.15.86:0/3615605650 conn(0x56399cf4eff0 msgr2=0x56399d446040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:00.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.820+0000 7fd68f1b0000 1 -- 172.21.15.86:0/3615605650 shutdown_connections 2024-04-25T14:06:00.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.820+0000 7fd68f1b0000 1 -- 172.21.15.86:0/3615605650 wait complete. 2024-04-25T14:06:00.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.820+0000 7fd68f1b0000 1 -- start start 2024-04-25T14:06:00.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.820+0000 7fd68f1b0000 1 Processor -- start 2024-04-25T14:06:00.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.820+0000 7fd68f1b0000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56399cf0f280 0x56399d493f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:00.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.820+0000 7fd68f1b0000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56399d316ea0 0x56399d4944b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:00.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.820+0000 7fd68f1b0000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399d32ac20 0x56399d4949f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:00.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.820+0000 7fd68f1b0000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56399d28d130 con 0x56399d316ea0 2024-04-25T14:06:00.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.820+0000 7fd68f1b0000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56399d2237d0 con 0x56399d32ac20 2024-04-25T14:06:00.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.820+0000 7fd68f1b0000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x56399d29a0c0 con 0x56399cf0f280 2024-04-25T14:06:00.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.820+0000 7fd68f1b0000 10 client.0 _ll_register_callbacks cb 0x7fd6780585d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:00.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.820+0000 7fd67fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56399d316ea0 0x56399d4944b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:00.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.820+0000 7fd67fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56399d316ea0 0x56399d4944b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41504/0 (socket says 172.21.15.86:41504) 2024-04-25T14:06:00.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.820+0000 7fd67fe00640 1 -- 172.21.15.86:0/2132364837 learned_addr learned my addr 172.21.15.86:0/2132364837 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:00.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.820+0000 7fd685200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56399cf0f280 0x56399d493f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:00.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.820+0000 7fd685c00640 1 --2- 172.21.15.86:0/2132364837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399d32ac20 0x56399d4949f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:00.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.820+0000 7fd67fe00640 1 -- 172.21.15.86:0/2132364837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56399cf0f280 msgr2=0x56399d493f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:00.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.820+0000 7fd67fe00640 1 --2- 172.21.15.86:0/2132364837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56399cf0f280 0x56399d493f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.820+0000 7fd67fe00640 1 -- 172.21.15.86:0/2132364837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399d32ac20 msgr2=0x56399d4949f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:00.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.820+0000 7fd67fe00640 1 --2- 172.21.15.86:0/2132364837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399d32ac20 0x56399d4949f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.820+0000 7fd67fe00640 1 -- 172.21.15.86:0/2132364837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56399d44b430 con 0x56399d316ea0 2024-04-25T14:06:00.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd67fe00640 1 --2- 172.21.15.86:0/2132364837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56399d316ea0 0x56399d4944b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd668007470 tx=0x7fd668000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26acdf21a461856 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:00.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd66e000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:06:00.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd66e000640 1 -- 172.21.15.86:0/2132364837 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66801e850 con 0x56399d316ea0 2024-04-25T14:06:00.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd66e000640 1 -- 172.21.15.86:0/2132364837 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7fd66801e9f0 con 0x56399d316ea0 2024-04-25T14:06:00.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd68f1b0000 1 -- 172.21.15.86:0/2132364837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd67805e070 con 0x56399d316ea0 2024-04-25T14:06:00.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd66e000640 1 -- 172.21.15.86:0/2132364837 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd66801ec10 con 0x56399d316ea0 2024-04-25T14:06:00.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd68f1b0000 10 client.6078 fetch_fsmap learned FSMap version 40 2024-04-25T14:06:00.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd68f1b0000 1 -- 172.21.15.86:0/2132364837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x56399d246c70 con 0x56399d316ea0 2024-04-25T14:06:00.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd66e000640 1 -- 172.21.15.86:0/2132364837 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd668024030 con 0x56399d316ea0 2024-04-25T14:06:00.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd66e000640 1 -- 172.21.15.86:0/2132364837 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 40) ==== 4116+0+0 (secure 0 0 0) 0x7fd66802a440 con 0x56399d316ea0 2024-04-25T14:06:00.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd68f1b0000 10 client.6078 fetch_fsmap finished waiting for FSMap version 40 2024-04-25T14:06:00.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd68f1b0000 10 client.6078 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T14:06:00.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd68f1b0000 20 client.6078 populate_metadata read hostname 'smithi086' 2024-04-25T14:06:00.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd68f1b0000 1 --2- 172.21.15.86:0/2132364837 >> [v2:172.21.15.86:6834/2169924270,v1:172.21.15.86:6835/2169924270] conn(0x56399cf21830 0x56399d502fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:00.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd68f1b0000 4 client.6078 mds_command: new command op to 6048 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:06:00.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd68f1b0000 1 -- 172.21.15.86:0/2132364837 --> [v2:172.21.15.86:6834/2169924270,v1:172.21.15.86:6835/2169924270] -- command(tid 0: {"prefix": "status"}) -- 0x56399d29ba30 con 0x56399cf21830 2024-04-25T14:06:00.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd685200640 1 --2- 172.21.15.86:0/2132364837 >> [v2:172.21.15.86:6834/2169924270,v1:172.21.15.86:6835/2169924270] conn(0x56399cf21830 0x56399d502fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:00.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd685200640 1 --2- 172.21.15.86:0/2132364837 >> [v2:172.21.15.86:6834/2169924270,v1:172.21.15.86:6835/2169924270] conn(0x56399cf21830 0x56399d502fd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=400ec9089526be43 server_cookie=458876e86e4ca6a1 in_seq=0 out_seq=0 2024-04-25T14:06:00.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd66e000640 10 client.6078 ms_handle_connect on v2:172.21.15.86:6834/2169924270 2024-04-25T14:06:00.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd66e000640 1 -- 172.21.15.86:0/2132364837 <== mds.0 v2:172.21.15.86:6834/2169924270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+528 (crc 0 0 0) 0x56399d29ba30 con 0x56399cf21830 2024-04-25T14:06:00.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd66e000640 10 client.6078 handle_command_reply: tid=0 2024-04-25T14:06:00.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd68f1b0000 1 client.6078 shutdown 2024-04-25T14:06:00.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd68f1b0000 1 -- 172.21.15.86:0/2132364837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56399d316ea0 msgr2=0x56399d4944b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:00.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd68f1b0000 1 --2- 172.21.15.86:0/2132364837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56399d316ea0 0x56399d4944b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd668007470 tx=0x7fd668000ee0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd68f1b0000 1 -- 172.21.15.86:0/2132364837 shutdown_connections 2024-04-25T14:06:00.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd68f1b0000 1 --2- 172.21.15.86:0/2132364837 >> [v2:172.21.15.86:6834/2169924270,v1:172.21.15.86:6835/2169924270] conn(0x56399cf21830 0x56399d502fd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd68f1b0000 1 --2- 172.21.15.86:0/2132364837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56399d32ac20 0x56399d4949f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd68f1b0000 1 --2- 172.21.15.86:0/2132364837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56399d316ea0 0x56399d4944b0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd66e000640 0 client.6078 ms_handle_reset on v2:172.21.15.86:6834/2169924270 2024-04-25T14:06:00.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd68f1b0000 1 --2- 172.21.15.86:0/2132364837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56399cf0f280 0x56399d493f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd68f1b0000 1 -- 172.21.15.86:0/2132364837 >> 172.21.15.86:0/2132364837 conn(0x56399cf4eff0 msgr2=0x56399d49f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:00.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd68f1b0000 1 -- 172.21.15.86:0/2132364837 shutdown_connections 2024-04-25T14:06:00.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd68f1b0000 1 -- 172.21.15.86:0/2132364837 wait complete. 2024-04-25T14:06:00.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd68f1b0000 20 client.6078 trim_cache size 0 max 16384 2024-04-25T14:06:00.833 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:06:00.833 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb", 2024-04-25T14:06:00.833 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-25T14:06:00.833 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 6048, 2024-04-25T14:06:00.833 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:reconnect", 2024-04-25T14:06:00.833 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:replay", 2024-04-25T14:06:00.833 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "fs1", 2024-04-25T14:06:00.833 INFO:teuthology.orchestra.run.smithi086.stdout: "replay_status": { 2024-04-25T14:06:00.834 INFO:teuthology.orchestra.run.smithi086.stdout: "journal_read_pos": 4206700, 2024-04-25T14:06:00.834 INFO:teuthology.orchestra.run.smithi086.stdout: "journal_write_pos": 4206700, 2024-04-25T14:06:00.834 INFO:teuthology.orchestra.run.smithi086.stdout: "journal_expire_pos": 4194304, 2024-04-25T14:06:00.834 INFO:teuthology.orchestra.run.smithi086.stdout: "num_events": 11, 2024-04-25T14:06:00.834 INFO:teuthology.orchestra.run.smithi086.stdout: "num_segments": 2 2024-04-25T14:06:00.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:06:00.834 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 0.58595014599999995, 2024-04-25T14:06:00.834 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 40, 2024-04-25T14:06:00.834 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 58, 2024-04-25T14:06:00.834 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 0, 2024-04-25T14:06:00.834 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 0.89920820400000001 2024-04-25T14:06:00.834 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:06:00.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd687c00640 1 -- 172.21.15.86:0/3502062531 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fd65c06f570 msgr2=0x7fd65c071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:00.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd687c00640 1 --2- 172.21.15.86:0/3502062531 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fd65c06f570 0x7fd65c071a80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd6700af000 tx=0x7fd6700a7070 comp rx=0 tx=0).stop 2024-04-25T14:06:00.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd687c00640 1 -- 172.21.15.86:0/3502062531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd68015faa0 msgr2=0x7fd6801574f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:00.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd687c00640 1 --2- 172.21.15.86:0/3502062531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd68015faa0 0x7fd6801574f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd67800d590 tx=0x7fd67800da50 comp rx=0 tx=0).stop 2024-04-25T14:06:00.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd687c00640 1 -- 172.21.15.86:0/3502062531 shutdown_connections 2024-04-25T14:06:00.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd687c00640 1 --2- 172.21.15.86:0/3502062531 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fd65c06f570 0x7fd65c071a80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd687c00640 1 --2- 172.21.15.86:0/3502062531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd68015faa0 0x7fd6801574f0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd687c00640 1 --2- 172.21.15.86:0/3502062531 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6800b59e0 0x7fd68014ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd687c00640 1 --2- 172.21.15.86:0/3502062531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6800aeea0 0x7fd680162be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:00.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd687c00640 1 -- 172.21.15.86:0/3502062531 >> 172.21.15.86:0/3502062531 conn(0x7fd6800d2230 msgr2=0x7fd68015f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:00.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd687c00640 1 -- 172.21.15.86:0/3502062531 shutdown_connections 2024-04-25T14:06:00.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:00.824+0000 7fd687c00640 1 -- 172.21.15.86:0/3502062531 wait complete. 2024-04-25T14:06:01.226 INFO:tasks.ceph.mgr.y.smithi086.stderr:Exception in thread Thread-19: 2024-04-25T14:06:01.226 INFO:tasks.ceph.mgr.y.smithi086.stderr:Traceback (most recent call last): 2024-04-25T14:06:01.226 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner 2024-04-25T14:06:01.226 INFO:tasks.ceph.mgr.y.smithi086.stderr: self.run() 2024-04-25T14:06:01.226 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/lib/python3.10/threading.py", line 1378, in run 2024-04-25T14:06:01.226 INFO:tasks.ceph.mgr.y.smithi086.stderr: self.function(*self.args, **self.kwargs) 2024-04-25T14:06:01.226 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/share/ceph/mgr/stats/fs/perf_stats.py", line 222, in re_register_queries 2024-04-25T14:06:01.226 INFO:tasks.ceph.mgr.y.smithi086.stderr: if self.mx_last_updated >= ua_last_updated: 2024-04-25T14:06:01.226 INFO:tasks.ceph.mgr.y.smithi086.stderr:AttributeError: 'FSPerfStats' object has no attribute 'mx_last_updated' 2024-04-25T14:06:01.847 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:06:02.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.004+0000 7fb94a600640 1 Processor -- start 2024-04-25T14:06:02.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.004+0000 7fb94a600640 1 -- start start 2024-04-25T14:06:02.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.004+0000 7fb94a600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb9440cfca0 0x7fb94406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:02.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.004+0000 7fb94a600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb944072bd0 con 0x7fb9440d16e0 2024-04-25T14:06:02.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.004+0000 7fb94a600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb94406f930 con 0x7fb9440730b0 2024-04-25T14:06:02.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.004+0000 7fb94a600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb944070470 con 0x7fb9440cfca0 2024-04-25T14:06:02.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.004+0000 7fb943400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb9440cfca0 0x7fb94406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:02.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.004+0000 7fb943400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb9440cfca0 0x7fb94406f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41506/0 (socket says 172.21.15.86:41506) 2024-04-25T14:06:02.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.004+0000 7fb943400640 1 -- 172.21.15.86:0/1592072467 learned_addr learned my addr 172.21.15.86:0/1592072467 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:02.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.004+0000 7fb942a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb9440d16e0 0x7fb9440d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50976/0 (socket says 172.21.15.86:50976) 2024-04-25T14:06:02.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.004+0000 7fb943400640 1 -- 172.21.15.86:0/1592072467 >> v1:172.21.15.155:6789/0 conn(0x7fb9440730b0 legacy=0x7fb9440734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:02.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.004+0000 7fb943400640 1 -- 172.21.15.86:0/1592072467 >> v1:172.21.15.86:6789/0 conn(0x7fb9440d16e0 legacy=0x7fb9440d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:02.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.004+0000 7fb943400640 1 -- 172.21.15.86:0/1592072467 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb94406fbd0 con 0x7fb9440cfca0 2024-04-25T14:06:02.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.008+0000 7fb943400640 1 --2- 172.21.15.86:0/1592072467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9440cfca0 0x7fb94406f3f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb9440d22a0 tx=0x7fb93401c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7866d4216ee3ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:02.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.008+0000 7fb942000640 1 -- 172.21.15.86:0/1592072467 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9440743e0 con 0x7fb9440cfca0 2024-04-25T14:06:02.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.008+0000 7fb942000640 1 -- 172.21.15.86:0/1592072467 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9340a57a0 con 0x7fb9440cfca0 2024-04-25T14:06:02.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.008+0000 7fb942000640 1 -- 172.21.15.86:0/1592072467 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9340a5b70 con 0x7fb9440cfca0 2024-04-25T14:06:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.008+0000 7fb94a600640 1 -- 172.21.15.86:0/1592072467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9440cfca0 msgr2=0x7fb94406f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.008+0000 7fb94a600640 1 --2- 172.21.15.86:0/1592072467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9440cfca0 0x7fb94406f3f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb9440d22a0 tx=0x7fb93401c9d0 comp rx=0 tx=0).stop 2024-04-25T14:06:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.008+0000 7fb94a600640 1 -- 172.21.15.86:0/1592072467 shutdown_connections 2024-04-25T14:06:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.008+0000 7fb94a600640 1 --2- 172.21.15.86:0/1592072467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9440cfca0 0x7fb94406f3f0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.008+0000 7fb94a600640 1 -- 172.21.15.86:0/1592072467 >> 172.21.15.86:0/1592072467 conn(0x7fb9440b4650 msgr2=0x7fb9440b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.008+0000 7fb94a600640 1 -- 172.21.15.86:0/1592072467 shutdown_connections 2024-04-25T14:06:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.008+0000 7fb94a600640 1 -- 172.21.15.86:0/1592072467 wait complete. 2024-04-25T14:06:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.012+0000 7fb94a600640 1 Processor -- start 2024-04-25T14:06:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.012+0000 7fb94a600640 1 -- start start 2024-04-25T14:06:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.012+0000 7fb94a600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9440730b0 0x7fb9441fd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.012+0000 7fb94a600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9440cfca0 0x7fb944156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.012+0000 7fb94a600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9440d16e0 0x7fb944156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.012+0000 7fb94a600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb94416aa80 con 0x7fb9440730b0 2024-04-25T14:06:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.012+0000 7fb94a600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb94416a900 con 0x7fb9440cfca0 2024-04-25T14:06:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.012+0000 7fb94a600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb94416ac00 con 0x7fb9440d16e0 2024-04-25T14:06:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.012+0000 7fb943400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9440730b0 0x7fb9441fd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.012+0000 7fb943400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9440730b0 0x7fb9441fd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41508/0 (socket says 172.21.15.86:41508) 2024-04-25T14:06:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.012+0000 7fb943400640 1 -- 172.21.15.86:0/870300474 learned_addr learned my addr 172.21.15.86:0/870300474 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.012+0000 7fb942a00640 1 --2- 172.21.15.86:0/870300474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9440cfca0 0x7fb944156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.012+0000 7fb943e00640 1 --2- 172.21.15.86:0/870300474 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9440d16e0 0x7fb944156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.012+0000 7fb943400640 1 -- 172.21.15.86:0/870300474 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9440d16e0 msgr2=0x7fb944156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.012+0000 7fb943400640 1 --2- 172.21.15.86:0/870300474 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9440d16e0 0x7fb944156bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.012+0000 7fb943400640 1 -- 172.21.15.86:0/870300474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9440cfca0 msgr2=0x7fb944156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.012+0000 7fb943400640 1 --2- 172.21.15.86:0/870300474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9440cfca0 0x7fb944156690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:02.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.012+0000 7fb943400640 1 -- 172.21.15.86:0/870300474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9440afea0 con 0x7fb9440730b0 2024-04-25T14:06:02.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.016+0000 7fb943400640 1 --2- 172.21.15.86:0/870300474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9440730b0 0x7fb9441fd970 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb9340af040 tx=0x7fb934019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.016+0000 7fb93b400640 1 -- 172.21.15.86:0/870300474 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9340b8670 con 0x7fb9440730b0 2024-04-25T14:06:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.016+0000 7fb93b400640 1 -- 172.21.15.86:0/870300474 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9340be070 con 0x7fb9440730b0 2024-04-25T14:06:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.016+0000 7fb93b400640 1 -- 172.21.15.86:0/870300474 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9340b8a50 con 0x7fb9440730b0 2024-04-25T14:06:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.016+0000 7fb94a600640 1 -- 172.21.15.86:0/870300474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb944158510 con 0x7fb9440730b0 2024-04-25T14:06:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.016+0000 7fb94a600640 1 -- 172.21.15.86:0/870300474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb94415fd20 con 0x7fb9440730b0 2024-04-25T14:06:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.016+0000 7fb94a600640 1 -- 172.21.15.86:0/870300474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb944158300 con 0x7fb9440730b0 2024-04-25T14:06:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.016+0000 7fb93b400640 1 -- 172.21.15.86:0/870300474 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7fb9340b8bf0 con 0x7fb9440730b0 2024-04-25T14:06:02.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.016+0000 7fb93b400640 1 --2- 172.21.15.86:0/870300474 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb92406f570 0x7fb924071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:02.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.016+0000 7fb93b400640 1 -- 172.21.15.86:0/870300474 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7fb93412ac50 con 0x7fb9440730b0 2024-04-25T14:06:02.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.016+0000 7fb942a00640 1 --2- 172.21.15.86:0/870300474 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb92406f570 0x7fb924071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:02.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.020+0000 7fb942a00640 1 --2- 172.21.15.86:0/870300474 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb92406f570 0x7fb924071a80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb9440b08c0 tx=0x7fb930007cf0 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:02.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.024+0000 7fb93b400640 1 -- 172.21.15.86:0/870300474 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb934130020 con 0x7fb9440730b0 2024-04-25T14:06:02.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.216+0000 7fb94a600640 1 -- 172.21.15.86:0/870300474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb94409f4b0 con 0x7fb9440730b0 2024-04-25T14:06:02.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.216+0000 7fb93b400640 1 -- 172.21.15.86:0/870300474 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+6487 (secure 0 0 0) 0x7fb9340165c0 con 0x7fb9440730b0 2024-04-25T14:06:02.217 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:02.218 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":41,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3545772170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3545772170},{"type":"v1","addr":"172.21.15.155:6834","nonce":3545772170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5980,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/766530004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":766530004},{"type":"v1","addr":"172.21.15.86:6837","nonce":766530004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:32.946320+0000","modified":"2024-04-25T14:06:00.257227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6048},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6048":{"gid":6048,"name":"c","rank":0,"incarnation":40,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.86:6835/2169924270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2169924270},{"type":"v1","addr":"172.21.15.86:6835","nonce":2169924270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"fs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3},{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:54.565528+0000","modified":"2024-04-25T14:06:00.260267+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6036":{"gid":6036,"name":"d","rank":0,"incarnation":40,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.155:6835/2914930843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2914930843},{"type":"v1","addr":"172.21.15.155:6835","nonce":2914930843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"fs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-04-25T14:06:02.218 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 41 2024-04-25T14:06:02.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.216+0000 7fb94a600640 1 -- 172.21.15.86:0/870300474 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb92406f570 msgr2=0x7fb924071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:02.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.216+0000 7fb94a600640 1 --2- 172.21.15.86:0/870300474 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb92406f570 0x7fb924071a80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb9440b08c0 tx=0x7fb930007cf0 comp rx=0 tx=0).stop 2024-04-25T14:06:02.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.216+0000 7fb94a600640 1 -- 172.21.15.86:0/870300474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9440730b0 msgr2=0x7fb9441fd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:02.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.216+0000 7fb94a600640 1 --2- 172.21.15.86:0/870300474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9440730b0 0x7fb9441fd970 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb9340af040 tx=0x7fb934019770 comp rx=0 tx=0).stop 2024-04-25T14:06:02.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.216+0000 7fb94a600640 1 -- 172.21.15.86:0/870300474 shutdown_connections 2024-04-25T14:06:02.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.216+0000 7fb94a600640 1 --2- 172.21.15.86:0/870300474 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb92406f570 0x7fb924071a80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:02.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.216+0000 7fb94a600640 1 --2- 172.21.15.86:0/870300474 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9440d16e0 0x7fb944156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:02.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.216+0000 7fb94a600640 1 --2- 172.21.15.86:0/870300474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9440cfca0 0x7fb944156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:02.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.216+0000 7fb94a600640 1 --2- 172.21.15.86:0/870300474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9440730b0 0x7fb9441fd970 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:02.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.216+0000 7fb94a600640 1 -- 172.21.15.86:0/870300474 >> 172.21.15.86:0/870300474 conn(0x7fb9440b4650 msgr2=0x7fb9440b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:02.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.216+0000 7fb94a600640 1 -- 172.21.15.86:0/870300474 shutdown_connections 2024-04-25T14:06:02.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:02.216+0000 7fb94a600640 1 -- 172.21.15.86:0/870300474 wait complete. 2024-04-25T14:06:02.642 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 41, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:05:54.565528+0000', 'modified': '2024-04-25T14:06:00.260267+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 57, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6036}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6036': {'gid': 6036, 'name': 'd', 'rank': 0, 'incarnation': 40, 'state': 'up:reconnect', 'state_seq': 2, 'addr': '172.21.15.155:6835/2914930843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6833', 'nonce': 2914930843}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 2914930843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'fs2', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T14:06:02.643 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6036:up:reconnect 2024-04-25T14:06:03.643 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:06:03.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.800+0000 7f5b6de00640 1 Processor -- start 2024-04-25T14:06:03.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.800+0000 7f5b6de00640 1 -- start start 2024-04-25T14:06:03.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.804+0000 7f5b6de00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5b68070330 0x7f5b680d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:03.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.804+0000 7f5b6de00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b680d1c20 0x7f5b6806ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:03.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.804+0000 7f5b6de00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b680d2030 con 0x7f5b6806f270 2024-04-25T14:06:03.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.804+0000 7f5b6de00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f5b6815e5c0 con 0x7f5b68070330 2024-04-25T14:06:03.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.804+0000 7f5b6de00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5b6815e8a0 con 0x7f5b680d1c20 2024-04-25T14:06:03.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.804+0000 7f5b67400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5b6806f270 0x7f5b6806f680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50988/0 (socket says 172.21.15.86:50988) 2024-04-25T14:06:03.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.804+0000 7f5b67400640 1 -- 172.21.15.86:0/645610173 learned_addr learned my addr 172.21.15.86:0/645610173 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:03.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.804+0000 7f5b66000640 1 --2- 172.21.15.86:0/645610173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b680d1c20 0x7f5b6806ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:03.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.804+0000 7f5b66a00640 1 --2- 172.21.15.86:0/645610173 >> v2:172.21.15.155:3300/0 conn(0x7f5b68070330 0x7f5b680d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:03.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.804+0000 7f5b65600640 1 -- 172.21.15.86:0/645610173 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 466162122 0 0) 0x7f5b60001120 con 0x7f5b6806f270 2024-04-25T14:06:03.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.804+0000 7f5b65600640 1 -- 172.21.15.86:0/645610173 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b4c003530 con 0x7f5b6806f270 2024-04-25T14:06:03.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.804+0000 7f5b66000640 1 -- 172.21.15.86:0/645610173 >> v2:172.21.15.155:3300/0 conn(0x7f5b68070330 msgr2=0x7f5b680d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:03.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.804+0000 7f5b66000640 1 --2- 172.21.15.86:0/645610173 >> v2:172.21.15.155:3300/0 conn(0x7f5b68070330 0x7f5b680d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:03.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.804+0000 7f5b66000640 1 -- 172.21.15.86:0/645610173 >> v1:172.21.15.86:6789/0 conn(0x7f5b6806f270 legacy=0x7f5b6806f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:03.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.804+0000 7f5b66000640 1 -- 172.21.15.86:0/645610173 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6815eb80 con 0x7f5b680d1c20 2024-04-25T14:06:03.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.804+0000 7f5b66a00640 1 --2- 172.21.15.86:0/645610173 >> v2:172.21.15.155:3300/0 conn(0x7f5b68070330 0x7f5b680d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:03.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.808+0000 7f5b66000640 1 --2- 172.21.15.86:0/645610173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b680d1c20 0x7f5b6806ed30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5b540097c0 tx=0x7f5b5401c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6846c32e5a8dcaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:03.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.808+0000 7f5b65600640 1 -- 172.21.15.86:0/645610173 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b540b0070 con 0x7f5b680d1c20 2024-04-25T14:06:03.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.808+0000 7f5b65600640 1 -- 172.21.15.86:0/645610173 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b540ab600 con 0x7f5b680d1c20 2024-04-25T14:06:03.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.808+0000 7f5b65600640 1 -- 172.21.15.86:0/645610173 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b540ab9b0 con 0x7f5b680d1c20 2024-04-25T14:06:03.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.808+0000 7f5b6de00640 1 -- 172.21.15.86:0/645610173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b680d1c20 msgr2=0x7f5b6806ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:03.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.808+0000 7f5b6de00640 1 --2- 172.21.15.86:0/645610173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b680d1c20 0x7f5b6806ed30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5b540097c0 tx=0x7f5b5401c410 comp rx=0 tx=0).stop 2024-04-25T14:06:03.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.808+0000 7f5b6de00640 1 -- 172.21.15.86:0/645610173 shutdown_connections 2024-04-25T14:06:03.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.808+0000 7f5b6de00640 1 --2- 172.21.15.86:0/645610173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b680d1c20 0x7f5b6806ed30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:03.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.808+0000 7f5b6de00640 1 --2- 172.21.15.86:0/645610173 >> v2:172.21.15.155:3300/0 conn(0x7f5b68070330 0x7f5b680d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:03.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.808+0000 7f5b6de00640 1 -- 172.21.15.86:0/645610173 >> 172.21.15.86:0/645610173 conn(0x7f5b680b4650 msgr2=0x7f5b680b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:03.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.808+0000 7f5b6de00640 1 -- 172.21.15.86:0/645610173 shutdown_connections 2024-04-25T14:06:03.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.808+0000 7f5b6de00640 1 -- 172.21.15.86:0/645610173 wait complete. 2024-04-25T14:06:03.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.808+0000 7f5b6de00640 1 Processor -- start 2024-04-25T14:06:03.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.808+0000 7f5b6de00640 1 -- start start 2024-04-25T14:06:03.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.812+0000 7f5b6de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b6806f270 0x7f5b681f8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:03.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.812+0000 7f5b6de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b68070330 0x7f5b680cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:03.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.812+0000 7f5b6de00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b680d1c20 0x7f5b681faeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:03.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.812+0000 7f5b6de00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b680708c0 con 0x7f5b6806f270 2024-04-25T14:06:03.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.812+0000 7f5b6de00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5b68070740 con 0x7f5b68070330 2024-04-25T14:06:03.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.812+0000 7f5b6de00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5b6816afc0 con 0x7f5b680d1c20 2024-04-25T14:06:03.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.812+0000 7f5b66a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b6806f270 0x7f5b681f8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:03.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.812+0000 7f5b66a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b6806f270 0x7f5b681f8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41522/0 (socket says 172.21.15.86:41522) 2024-04-25T14:06:03.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.812+0000 7f5b66a00640 1 -- 172.21.15.86:0/2584671877 learned_addr learned my addr 172.21.15.86:0/2584671877 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:03.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.812+0000 7f5b67400640 1 --2- 172.21.15.86:0/2584671877 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b680d1c20 0x7f5b681faeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:03.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.812+0000 7f5b66000640 1 --2- 172.21.15.86:0/2584671877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b68070330 0x7f5b680cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:03.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.812+0000 7f5b66a00640 1 -- 172.21.15.86:0/2584671877 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b680d1c20 msgr2=0x7f5b681faeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:03.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.812+0000 7f5b66a00640 1 --2- 172.21.15.86:0/2584671877 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b680d1c20 0x7f5b681faeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:03.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.812+0000 7f5b66a00640 1 -- 172.21.15.86:0/2584671877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b68070330 msgr2=0x7f5b680cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:03.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.812+0000 7f5b66a00640 1 --2- 172.21.15.86:0/2584671877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b68070330 0x7f5b680cb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:03.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.812+0000 7f5b66a00640 1 -- 172.21.15.86:0/2584671877 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b680c8e40 con 0x7f5b6806f270 2024-04-25T14:06:03.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.812+0000 7f5b66000640 1 --2- 172.21.15.86:0/2584671877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b68070330 0x7f5b680cb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:03.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.812+0000 7f5b66a00640 1 --2- 172.21.15.86:0/2584671877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b6806f270 0x7f5b681f8380 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5b58009e60 tx=0x7f5b5800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:03.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.812+0000 7f5b5f400640 1 -- 172.21.15.86:0/2584671877 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b58018020 con 0x7f5b6806f270 2024-04-25T14:06:03.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.812+0000 7f5b5f400640 1 -- 172.21.15.86:0/2584671877 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b58014680 con 0x7f5b6806f270 2024-04-25T14:06:03.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.812+0000 7f5b6de00640 1 -- 172.21.15.86:0/2584671877 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b681fb930 con 0x7f5b6806f270 2024-04-25T14:06:03.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.812+0000 7f5b5f400640 1 -- 172.21.15.86:0/2584671877 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b58014a50 con 0x7f5b6806f270 2024-04-25T14:06:03.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.812+0000 7f5b6de00640 1 -- 172.21.15.86:0/2584671877 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b681f7b80 con 0x7f5b6806f270 2024-04-25T14:06:03.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.812+0000 7f5b6de00640 1 -- 172.21.15.86:0/2584671877 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b68069d00 con 0x7f5b6806f270 2024-04-25T14:06:03.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.816+0000 7f5b5f400640 1 -- 172.21.15.86:0/2584671877 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f5b5801b020 con 0x7f5b6806f270 2024-04-25T14:06:03.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.816+0000 7f5b5f400640 1 --2- 172.21.15.86:0/2584671877 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f5b4806f570 0x7f5b48071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:03.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.816+0000 7f5b5f400640 1 -- 172.21.15.86:0/2584671877 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7f5b580909f0 con 0x7f5b6806f270 2024-04-25T14:06:03.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.816+0000 7f5b66000640 1 --2- 172.21.15.86:0/2584671877 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f5b4806f570 0x7f5b48071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:03.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.816+0000 7f5b66000640 1 --2- 172.21.15.86:0/2584671877 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f5b4806f570 0x7f5b48071a80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5b540ad000 tx=0x7f5b540a6000 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:03.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:03.820+0000 7f5b5f400640 1 -- 172.21.15.86:0/2584671877 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5b58022030 con 0x7f5b6806f270 2024-04-25T14:06:04.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.008+0000 7f5b6de00640 1 -- 172.21.15.86:0/2584671877 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b68076bf0 con 0x7f5b6806f270 2024-04-25T14:06:04.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.012+0000 7f5b5f400640 1 -- 172.21.15.86:0/2584671877 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+6495 (secure 0 0 0) 0x7f5b58010040 con 0x7f5b6806f270 2024-04-25T14:06:04.012 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:04.012 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3545772170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3545772170},{"type":"v1","addr":"172.21.15.155:6834","nonce":3545772170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5980,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/766530004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":766530004},{"type":"v1","addr":"172.21.15.86:6837","nonce":766530004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:32.946320+0000","modified":"2024-04-25T14:06:03.253171+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6048},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6048":{"gid":6048,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6835/2169924270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2169924270},{"type":"v1","addr":"172.21.15.86:6835","nonce":2169924270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"fs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6048,"qdb_cluster":[6048]},"id":3},{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:54.565528+0000","modified":"2024-04-25T14:06:03.253173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6036":{"gid":6036,"name":"d","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6835/2914930843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2914930843},{"type":"v1","addr":"172.21.15.155:6835","nonce":2914930843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"fs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6036,"qdb_cluster":[6036]},"id":4}]} 2024-04-25T14:06:04.012 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 43 2024-04-25T14:06:04.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.012+0000 7f5b6de00640 1 -- 172.21.15.86:0/2584671877 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f5b4806f570 msgr2=0x7f5b48071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:04.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.012+0000 7f5b6de00640 1 --2- 172.21.15.86:0/2584671877 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f5b4806f570 0x7f5b48071a80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5b540ad000 tx=0x7f5b540a6000 comp rx=0 tx=0).stop 2024-04-25T14:06:04.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.012+0000 7f5b6de00640 1 -- 172.21.15.86:0/2584671877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b6806f270 msgr2=0x7f5b681f8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:04.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.012+0000 7f5b6de00640 1 --2- 172.21.15.86:0/2584671877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b6806f270 0x7f5b681f8380 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5b58009e60 tx=0x7f5b5800c490 comp rx=0 tx=0).stop 2024-04-25T14:06:04.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.012+0000 7f5b6de00640 1 -- 172.21.15.86:0/2584671877 shutdown_connections 2024-04-25T14:06:04.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.012+0000 7f5b6de00640 1 --2- 172.21.15.86:0/2584671877 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f5b4806f570 0x7f5b48071a80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.012+0000 7f5b6de00640 1 --2- 172.21.15.86:0/2584671877 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b680d1c20 0x7f5b681faeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.012+0000 7f5b6de00640 1 --2- 172.21.15.86:0/2584671877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b68070330 0x7f5b680cb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.012+0000 7f5b6de00640 1 --2- 172.21.15.86:0/2584671877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b6806f270 0x7f5b681f8380 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.012+0000 7f5b6de00640 1 -- 172.21.15.86:0/2584671877 >> 172.21.15.86:0/2584671877 conn(0x7f5b680b4650 msgr2=0x7f5b6815d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:04.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.012+0000 7f5b6de00640 1 -- 172.21.15.86:0/2584671877 shutdown_connections 2024-04-25T14:06:04.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.012+0000 7f5b6de00640 1 -- 172.21.15.86:0/2584671877 wait complete. 2024-04-25T14:06:04.260 INFO:tasks.ceph.mgr.y.smithi086.stderr:Exception in thread Thread-21: 2024-04-25T14:06:04.260 INFO:tasks.ceph.mgr.y.smithi086.stderr:Traceback (most recent call last): 2024-04-25T14:06:04.260 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner 2024-04-25T14:06:04.261 INFO:tasks.ceph.mgr.y.smithi086.stderr: self.run() 2024-04-25T14:06:04.261 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/lib/python3.10/threading.py", line 1378, in run 2024-04-25T14:06:04.261 INFO:tasks.ceph.mgr.y.smithi086.stderr: self.function(*self.args, **self.kwargs) 2024-04-25T14:06:04.261 INFO:tasks.ceph.mgr.y.smithi086.stderr: File "/usr/share/ceph/mgr/stats/fs/perf_stats.py", line 222, in re_register_queries 2024-04-25T14:06:04.261 INFO:tasks.ceph.mgr.y.smithi086.stderr: if self.mx_last_updated >= ua_last_updated: 2024-04-25T14:06:04.261 INFO:tasks.ceph.mgr.y.smithi086.stderr:AttributeError: 'FSPerfStats' object has no attribute 'mx_last_updated' 2024-04-25T14:06:04.436 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 43, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:05:54.565528+0000', 'modified': '2024-04-25T14:06:03.253173+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 57, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6036}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6036': {'gid': 6036, 'name': 'd', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.155:6835/2914930843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6833', 'nonce': 2914930843}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 2914930843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'fs2', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6036, 'qdb_cluster': [6036]} 2024-04-25T14:06:04.436 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T14:06:04.437 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T14:06:04.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.584+0000 7fd6c5e00640 1 Processor -- start 2024-04-25T14:06:04.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.584+0000 7fd6c5e00640 1 -- start start 2024-04-25T14:06:04.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6c5e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd6c0070330 0x7fd6c00d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:04.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6c5e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6c00d2030 con 0x7fd6c00ce790 2024-04-25T14:06:04.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6c5e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6c015e5c0 con 0x7fd6c00d1c20 2024-04-25T14:06:04.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6c5e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd6c015e8a0 con 0x7fd6c0070330 2024-04-25T14:06:04.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6bf400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd6c00ce790 0x7fd6c006f440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50998/0 (socket says 172.21.15.86:50998) 2024-04-25T14:06:04.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6bf400640 1 -- 172.21.15.86:0/3178719931 learned_addr learned my addr 172.21.15.86:0/3178719931 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:04.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6bea00640 1 --2- 172.21.15.86:0/3178719931 >> v2:172.21.15.155:3300/0 conn(0x7fd6c0070330 0x7fd6c00d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:04.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6bd600640 1 -- 172.21.15.86:0/3178719931 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1988620221 0 0) 0x7fd6b80014a0 con 0x7fd6c00ce790 2024-04-25T14:06:04.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6bd600640 1 -- 172.21.15.86:0/3178719931 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6a4003530 con 0x7fd6c00ce790 2024-04-25T14:06:04.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6bd600640 1 -- 172.21.15.86:0/3178719931 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3310110901 0 0) 0x7fd6a4003530 con 0x7fd6c00ce790 2024-04-25T14:06:04.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6bd600640 1 -- 172.21.15.86:0/3178719931 >> v2:172.21.15.155:3300/0 conn(0x7fd6c0070330 msgr2=0x7fd6c00d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:04.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6bd600640 1 --2- 172.21.15.86:0/3178719931 >> v2:172.21.15.155:3300/0 conn(0x7fd6c0070330 0x7fd6c00d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6bd600640 1 -- 172.21.15.86:0/3178719931 >> v1:172.21.15.155:6789/0 conn(0x7fd6c00d1c20 legacy=0x7fd6c006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:04.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6bd600640 1 -- 172.21.15.86:0/3178719931 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c015eb80 con 0x7fd6c00ce790 2024-04-25T14:06:04.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6bd600640 1 -- 172.21.15.86:0/3178719931 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fd6b8001bd0 con 0x7fd6c00ce790 2024-04-25T14:06:04.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6bd600640 1 -- 172.21.15.86:0/3178719931 >> v1:172.21.15.86:6789/0 conn(0x7fd6c00ce790 legacy=0x7fd6c006f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:04.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6bd600640 1 --2- 172.21.15.86:0/3178719931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6a4003a30 0x7fd6a4003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:04.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6bd600640 1 --2- 172.21.15.86:0/3178719931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6a4004810 0x7fd6a4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:04.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6bd600640 1 --2- 172.21.15.86:0/3178719931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6a4005360 0x7fd6a400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:04.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6bd600640 1 -- 172.21.15.86:0/3178719931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c015e8a0 con 0x7fd6a4005360 2024-04-25T14:06:04.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6bd600640 1 -- 172.21.15.86:0/3178719931 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a400c260 con 0x7fd6a4004810 2024-04-25T14:06:04.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6bd600640 1 -- 172.21.15.86:0/3178719931 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd6a400c460 con 0x7fd6a4003a30 2024-04-25T14:06:04.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6bea00640 1 --2- 172.21.15.86:0/3178719931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6a4003a30 0x7fd6a4003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:04.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6be000640 1 --2- 172.21.15.86:0/3178719931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6a4004810 0x7fd6a4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:04.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6bf400640 1 --2- 172.21.15.86:0/3178719931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6a4005360 0x7fd6a400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:04.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6bea00640 1 -- 172.21.15.86:0/3178719931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6a4004810 msgr2=0x7fd6a4004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:04.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6bea00640 1 --2- 172.21.15.86:0/3178719931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6a4004810 0x7fd6a4004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6bea00640 1 -- 172.21.15.86:0/3178719931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6a4005360 msgr2=0x7fd6a400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:04.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6bea00640 1 --2- 172.21.15.86:0/3178719931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6a4005360 0x7fd6a400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6bea00640 1 -- 172.21.15.86:0/3178719931 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a4003530 con 0x7fd6a4003a30 2024-04-25T14:06:04.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6bf400640 1 --2- 172.21.15.86:0/3178719931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6a4005360 0x7fd6a400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:04.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6be000640 1 --2- 172.21.15.86:0/3178719931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6a4004810 0x7fd6a4004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:04.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.588+0000 7fd6bf400640 1 -- 172.21.15.86:0/3178719931 reap_dead start 2024-04-25T14:06:04.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.592+0000 7fd6bea00640 1 --2- 172.21.15.86:0/3178719931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6a4003a30 0x7fd6a4003ea0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0009ab0 tx=0x7fd6b001ca50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=582b0ca603ebfb05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:04.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.592+0000 7fd6bd600640 1 -- 172.21.15.86:0/3178719931 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b00b2070 con 0x7fd6a4003a30 2024-04-25T14:06:04.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.592+0000 7fd6bd600640 1 -- 172.21.15.86:0/3178719931 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b00a5530 con 0x7fd6a4003a30 2024-04-25T14:06:04.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.592+0000 7fd6bd600640 1 -- 172.21.15.86:0/3178719931 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b00096b0 con 0x7fd6a4003a30 2024-04-25T14:06:04.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.592+0000 7fd6c5e00640 1 -- 172.21.15.86:0/3178719931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6a4003a30 msgr2=0x7fd6a4003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:04.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.592+0000 7fd6c5e00640 1 --2- 172.21.15.86:0/3178719931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6a4003a30 0x7fd6a4003ea0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0009ab0 tx=0x7fd6b001ca50 comp rx=0 tx=0).stop 2024-04-25T14:06:04.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.592+0000 7fd6c5e00640 1 -- 172.21.15.86:0/3178719931 shutdown_connections 2024-04-25T14:06:04.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.592+0000 7fd6c5e00640 1 --2- 172.21.15.86:0/3178719931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6a4003a30 0x7fd6a4003ea0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.592+0000 7fd6c5e00640 1 -- 172.21.15.86:0/3178719931 >> 172.21.15.86:0/3178719931 conn(0x7fd6c00b4650 msgr2=0x7fd6c00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:04.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.592+0000 7fd6c5e00640 1 -- 172.21.15.86:0/3178719931 shutdown_connections 2024-04-25T14:06:04.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.592+0000 7fd6c5e00640 1 -- 172.21.15.86:0/3178719931 wait complete. 2024-04-25T14:06:04.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.596+0000 7fd6c5e00640 1 Processor -- start 2024-04-25T14:06:04.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.596+0000 7fd6c5e00640 1 -- start start 2024-04-25T14:06:04.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.596+0000 7fd6c5e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6a4003a30 0x7fd6c00c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:04.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.596+0000 7fd6c5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c009d110 0x7fd6c0150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:04.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.596+0000 7fd6c5e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6c009a4e0 0x7fd6c009e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:04.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.596+0000 7fd6c5e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c015f330 con 0x7fd6c009d110 2024-04-25T14:06:04.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.596+0000 7fd6c5e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c006d170 con 0x7fd6a4003a30 2024-04-25T14:06:04.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.596+0000 7fd6c5e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd6c015f4b0 con 0x7fd6c009a4e0 2024-04-25T14:06:04.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.596+0000 7fd6be000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c009d110 0x7fd6c0150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:04.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.596+0000 7fd6be000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c009d110 0x7fd6c0150a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41528/0 (socket says 172.21.15.86:41528) 2024-04-25T14:06:04.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.596+0000 7fd6be000640 1 -- 172.21.15.86:0/3358159606 learned_addr learned my addr 172.21.15.86:0/3358159606 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:04.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.596+0000 7fd6bea00640 1 --2- 172.21.15.86:0/3358159606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6a4003a30 0x7fd6c00c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:04.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.596+0000 7fd6bf400640 1 --2- 172.21.15.86:0/3358159606 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6c009a4e0 0x7fd6c009e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:04.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.596+0000 7fd6be000640 1 -- 172.21.15.86:0/3358159606 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6c009a4e0 msgr2=0x7fd6c009e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:04.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.596+0000 7fd6be000640 1 --2- 172.21.15.86:0/3358159606 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6c009a4e0 0x7fd6c009e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.596+0000 7fd6be000640 1 -- 172.21.15.86:0/3358159606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6a4003a30 msgr2=0x7fd6c00c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:04.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.596+0000 7fd6be000640 1 --2- 172.21.15.86:0/3358159606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6a4003a30 0x7fd6c00c4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.596+0000 7fd6be000640 1 -- 172.21.15.86:0/3358159606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c009d8e0 con 0x7fd6c009d110 2024-04-25T14:06:04.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.600+0000 7fd6bea00640 1 --2- 172.21.15.86:0/3358159606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6a4003a30 0x7fd6c00c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:04.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.600+0000 7fd6be000640 1 --2- 172.21.15.86:0/3358159606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c009d110 0x7fd6c0150a20 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0074ed0 tx=0x7fd6ac00fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:04.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.600+0000 7fd6b7400640 1 -- 172.21.15.86:0/3358159606 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac01a070 con 0x7fd6c009d110 2024-04-25T14:06:04.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.600+0000 7fd6b7400640 1 -- 172.21.15.86:0/3358159606 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6ac005320 con 0x7fd6c009d110 2024-04-25T14:06:04.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.600+0000 7fd6c5e00640 1 -- 172.21.15.86:0/3358159606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6c009dbc0 con 0x7fd6c009d110 2024-04-25T14:06:04.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.600+0000 7fd6b7400640 1 -- 172.21.15.86:0/3358159606 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac0013a0 con 0x7fd6c009d110 2024-04-25T14:06:04.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.600+0000 7fd6c5e00640 1 -- 172.21.15.86:0/3358159606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6c009e180 con 0x7fd6c009d110 2024-04-25T14:06:04.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.600+0000 7fd6b7400640 1 -- 172.21.15.86:0/3358159606 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7fd6ac01d070 con 0x7fd6c009d110 2024-04-25T14:06:04.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.600+0000 7fd6b7400640 1 --2- 172.21.15.86:0/3358159606 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fd6a006f570 0x7fd6a0071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:04.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.600+0000 7fd6b7400640 1 -- 172.21.15.86:0/3358159606 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7fd6ac08f2a0 con 0x7fd6c009d110 2024-04-25T14:06:04.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.600+0000 7fd6bea00640 1 --2- 172.21.15.86:0/3358159606 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fd6a006f570 0x7fd6a0071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:04.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6cd53c000 1 Processor -- start 2024-04-25T14:06:04.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6cd53c000 1 -- start start 2024-04-25T14:06:04.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6bea00640 1 --2- 172.21.15.86:0/3358159606 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fd6a006f570 0x7fd6a0071a80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00af000 tx=0x7fd6b0008040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:04.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6cd53c000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5577bae31c20 0x5577baf94340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:04.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6cd53c000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577bae1dea0 0x5577baf94880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:04.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6cd53c000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577baa16280 0x5577baf94dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:04.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6cd53c000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5577badff160 con 0x5577baa16280 2024-04-25T14:06:04.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6cd53c000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5577bad94e20 con 0x5577bae1dea0 2024-04-25T14:06:04.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6cd53c000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5577bad4da20 con 0x5577bae31c20 2024-04-25T14:06:04.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577baa16280 0x5577baf94dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:04.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577baa16280 0x5577baf94dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41538/0 (socket says 172.21.15.86:41538) 2024-04-25T14:06:04.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6bf400640 1 -- 172.21.15.86:0/1553116859 learned_addr learned my addr 172.21.15.86:0/1553116859 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:04.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6bea00640 1 --2- 172.21.15.86:0/1553116859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5577bae31c20 0x5577baf94340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:04.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6be000640 1 --2- 172.21.15.86:0/1553116859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577bae1dea0 0x5577baf94880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:04.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6bf400640 1 -- 172.21.15.86:0/1553116859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5577bae31c20 msgr2=0x5577baf94340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:04.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6bf400640 1 --2- 172.21.15.86:0/1553116859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5577bae31c20 0x5577baf94340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6bf400640 1 -- 172.21.15.86:0/1553116859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577bae1dea0 msgr2=0x5577baf94880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:04.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6bf400640 1 --2- 172.21.15.86:0/1553116859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577bae1dea0 0x5577baf94880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6bf400640 1 -- 172.21.15.86:0/1553116859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577bad52100 con 0x5577baa16280 2024-04-25T14:06:04.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6be000640 1 --2- 172.21.15.86:0/1553116859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577bae1dea0 0x5577baf94880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:04.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6bf400640 1 --2- 172.21.15.86:0/1553116859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577baa16280 0x5577baf94dc0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8000f90 tx=0x7fd6b8007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41cef8f56f3a9feb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:04.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6ab400640 1 -- 172.21.15.86:0/1553116859 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b800a650 con 0x5577baa16280 2024-04-25T14:06:04.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6ab400640 1 -- 172.21.15.86:0/1553116859 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b8007110 con 0x5577baa16280 2024-04-25T14:06:04.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6ab400640 1 -- 172.21.15.86:0/1553116859 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b80072c0 con 0x5577baa16280 2024-04-25T14:06:04.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6cd53c000 1 -- 172.21.15.86:0/1553116859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577baa16280 msgr2=0x5577baf94dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:04.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6cd53c000 1 --2- 172.21.15.86:0/1553116859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577baa16280 0x5577baf94dc0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8000f90 tx=0x7fd6b8007830 comp rx=0 tx=0).stop 2024-04-25T14:06:04.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6cd53c000 1 -- 172.21.15.86:0/1553116859 shutdown_connections 2024-04-25T14:06:04.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6cd53c000 1 --2- 172.21.15.86:0/1553116859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577baa16280 0x5577baf94dc0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.604+0000 7fd6cd53c000 1 --2- 172.21.15.86:0/1553116859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577bae1dea0 0x5577baf94880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.608+0000 7fd6cd53c000 1 --2- 172.21.15.86:0/1553116859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5577bae31c20 0x5577baf94340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.608+0000 7fd6cd53c000 1 -- 172.21.15.86:0/1553116859 >> 172.21.15.86:0/1553116859 conn(0x5577baa55ff0 msgr2=0x5577baf454c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:04.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.608+0000 7fd6cd53c000 1 -- 172.21.15.86:0/1553116859 shutdown_connections 2024-04-25T14:06:04.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.608+0000 7fd6cd53c000 1 -- 172.21.15.86:0/1553116859 wait complete. 2024-04-25T14:06:04.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.608+0000 7fd6cd53c000 1 -- start start 2024-04-25T14:06:04.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.608+0000 7fd6cd53c000 1 Processor -- start 2024-04-25T14:06:04.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.608+0000 7fd6cd53c000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5577baa16280 0x5577baf99810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:04.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.608+0000 7fd6cd53c000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577bae1dea0 0x5577baf99d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:04.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.608+0000 7fd6cd53c000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577bae31c20 0x5577baf54440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:04.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.608+0000 7fd6cd53c000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5577bada2220 con 0x5577bae31c20 2024-04-25T14:06:04.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.608+0000 7fd6cd53c000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5577bad38ea0 con 0x5577bae1dea0 2024-04-25T14:06:04.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.608+0000 7fd6cd53c000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5577bad8fb00 con 0x5577baa16280 2024-04-25T14:06:04.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.608+0000 7fd6cd53c000 10 client.0 _ll_register_callbacks cb 0x5577bada1dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:04.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.608+0000 7fd6bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577bae31c20 0x5577baf54440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:04.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.608+0000 7fd6bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577bae31c20 0x5577baf54440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41552/0 (socket says 172.21.15.86:41552) 2024-04-25T14:06:04.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.608+0000 7fd6bf400640 1 -- 172.21.15.86:0/2924863576 learned_addr learned my addr 172.21.15.86:0/2924863576 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:04.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.608+0000 7fd6be000640 1 --2- 172.21.15.86:0/2924863576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577bae1dea0 0x5577baf99d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:04.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.608+0000 7fd6bea00640 1 --2- 172.21.15.86:0/2924863576 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5577baa16280 0x5577baf99810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:04.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.608+0000 7fd6bf400640 1 -- 172.21.15.86:0/2924863576 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5577baa16280 msgr2=0x5577baf99810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:04.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.608+0000 7fd6bf400640 1 --2- 172.21.15.86:0/2924863576 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5577baa16280 0x5577baf99810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.608+0000 7fd6bf400640 1 -- 172.21.15.86:0/2924863576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577bae1dea0 msgr2=0x5577baf99d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:04.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.608+0000 7fd6bf400640 1 --2- 172.21.15.86:0/2924863576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577bae1dea0 0x5577baf99d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.608+0000 7fd6bf400640 1 -- 172.21.15.86:0/2924863576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577bad34500 con 0x5577bae31c20 2024-04-25T14:06:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.608+0000 7fd6bf400640 1 --2- 172.21.15.86:0/2924863576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577bae31c20 0x5577baf54440 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd6c015bd10 tx=0x7fd6c015e790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e7a39cb5aeb4c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.608+0000 7fd69fe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:06:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd69fe00640 1 -- 172.21.15.86:0/2924863576 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b800a650 con 0x5577bae31c20 2024-04-25T14:06:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd69fe00640 1 -- 172.21.15.86:0/2924863576 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7fd6b8014c20 con 0x5577bae31c20 2024-04-25T14:06:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd6cd53c000 1 -- 172.21.15.86:0/2924863576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6b8007110 con 0x5577bae31c20 2024-04-25T14:06:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd69fe00640 1 -- 172.21.15.86:0/2924863576 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd6b8034050 con 0x5577bae31c20 2024-04-25T14:06:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd6cd53c000 10 client.6105 fetch_fsmap learned FSMap version 43 2024-04-25T14:06:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd6cd53c000 1 -- 172.21.15.86:0/2924863576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x5577bad1aea0 con 0x5577bae31c20 2024-04-25T14:06:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd69fe00640 1 -- 172.21.15.86:0/2924863576 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b8007110 con 0x5577bae31c20 2024-04-25T14:06:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd69fe00640 1 -- 172.21.15.86:0/2924863576 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 4132+0+0 (secure 0 0 0) 0x7fd6b8037020 con 0x5577bae31c20 2024-04-25T14:06:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd6cd53c000 10 client.6105 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T14:06:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd6cd53c000 10 client.6105 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T14:06:04.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd6cd53c000 20 client.6105 populate_metadata read hostname 'smithi086' 2024-04-25T14:06:04.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd6cd53c000 1 --2- 172.21.15.86:0/2924863576 >> [v2:172.21.15.155:6833/2914930843,v1:172.21.15.155:6835/2914930843] conn(0x5577baa3a7d0 0x5577baf551e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:04.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd6cd53c000 4 client.6105 mds_command: new command op to 6036 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:04.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd6cd53c000 1 -- 172.21.15.86:0/2924863576 --> [v2:172.21.15.155:6833/2914930843,v1:172.21.15.155:6835/2914930843] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577bad2dbe0 con 0x5577baa3a7d0 2024-04-25T14:06:04.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd6bea00640 1 --2- 172.21.15.86:0/2924863576 >> [v2:172.21.15.155:6833/2914930843,v1:172.21.15.155:6835/2914930843] conn(0x5577baa3a7d0 0x5577baf551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:04.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd69fe00640 10 client.6105 ms_handle_connect on v2:172.21.15.155:6833/2914930843 2024-04-25T14:06:04.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd6bea00640 1 --2- 172.21.15.86:0/2924863576 >> [v2:172.21.15.155:6833/2914930843,v1:172.21.15.155:6835/2914930843] conn(0x5577baa3a7d0 0x5577baf551e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a716bc4aef11e5 server_cookie=24d0f6ca12b15d74 in_seq=0 out_seq=0 2024-04-25T14:06:04.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd69fe00640 1 -- 172.21.15.86:0/2924863576 <== mds.0 v2:172.21.15.155:6833/2914930843 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5577bad2dbe0 con 0x5577baa3a7d0 2024-04-25T14:06:04.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd69fe00640 10 client.6105 handle_command_reply: tid=0 2024-04-25T14:06:04.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd6cd53c000 1 client.6105 shutdown 2024-04-25T14:06:04.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd6cd53c000 1 -- 172.21.15.86:0/2924863576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577bae31c20 msgr2=0x5577baf54440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:04.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd6cd53c000 1 --2- 172.21.15.86:0/2924863576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577bae31c20 0x5577baf54440 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd6c015bd10 tx=0x7fd6c015e790 comp rx=0 tx=0).stop 2024-04-25T14:06:04.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd6cd53c000 1 -- 172.21.15.86:0/2924863576 shutdown_connections 2024-04-25T14:06:04.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd6cd53c000 1 --2- 172.21.15.86:0/2924863576 >> [v2:172.21.15.155:6833/2914930843,v1:172.21.15.155:6835/2914930843] conn(0x5577baa3a7d0 0x5577baf551e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd6cd53c000 1 --2- 172.21.15.86:0/2924863576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577bae31c20 0x5577baf54440 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd69fe00640 0 client.6105 ms_handle_reset on v2:172.21.15.155:6833/2914930843 2024-04-25T14:06:04.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd6cd53c000 1 --2- 172.21.15.86:0/2924863576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577bae1dea0 0x5577baf99d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd6cd53c000 1 --2- 172.21.15.86:0/2924863576 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5577baa16280 0x5577baf99810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd6cd53c000 1 -- 172.21.15.86:0/2924863576 >> 172.21.15.86:0/2924863576 conn(0x5577baa55ff0 msgr2=0x5577baf4c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:04.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd6cd53c000 1 -- 172.21.15.86:0/2924863576 shutdown_connections 2024-04-25T14:06:04.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd6cd53c000 1 -- 172.21.15.86:0/2924863576 wait complete. 2024-04-25T14:06:04.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.612+0000 7fd6cd53c000 20 client.6105 trim_cache size 0 max 16384 2024-04-25T14:06:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.632+0000 7fd6cd53c000 1 Processor -- start 2024-04-25T14:06:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.632+0000 7fd6cd53c000 1 -- start start 2024-04-25T14:06:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.632+0000 7fd6cd53c000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577baa16280 0x5577baf9fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.632+0000 7fd6cd53c000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577bae1dea0 0x5577baf9ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.632+0000 7fd6cd53c000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5577bae31c20 0x5577baf9b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.632+0000 7fd6cd53c000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c0202e00 con 0x5577bae1dea0 2024-04-25T14:06:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.632+0000 7fd6cd53c000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c00d0e10 con 0x5577baa16280 2024-04-25T14:06:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.632+0000 7fd6cd53c000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd6c0200ed0 con 0x5577bae31c20 2024-04-25T14:06:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.632+0000 7fd6be000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577bae1dea0 0x5577baf9ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.632+0000 7fd6be000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577bae1dea0 0x5577baf9ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41558/0 (socket says 172.21.15.86:41558) 2024-04-25T14:06:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.632+0000 7fd6be000640 1 -- 172.21.15.86:0/2547699708 learned_addr learned my addr 172.21.15.86:0/2547699708 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.632+0000 7fd6bf400640 1 --2- 172.21.15.86:0/2547699708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5577bae31c20 0x5577baf9b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.632+0000 7fd6bea00640 1 --2- 172.21.15.86:0/2547699708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577baa16280 0x5577baf9fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:04.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.632+0000 7fd6be000640 1 -- 172.21.15.86:0/2547699708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5577bae31c20 msgr2=0x5577baf9b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:04.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.632+0000 7fd6be000640 1 --2- 172.21.15.86:0/2547699708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5577bae31c20 0x5577baf9b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.632+0000 7fd6be000640 1 -- 172.21.15.86:0/2547699708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577baa16280 msgr2=0x5577baf9fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:04.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.632+0000 7fd6be000640 1 --2- 172.21.15.86:0/2547699708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577baa16280 0x5577baf9fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.636+0000 7fd6be000640 1 -- 172.21.15.86:0/2547699708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577bad4dc70 con 0x5577bae1dea0 2024-04-25T14:06:04.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.636+0000 7fd6be000640 1 --2- 172.21.15.86:0/2547699708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577bae1dea0 0x5577baf9ffb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac0584c0 tx=0x7fd6ac057240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffaac1c8c889b77d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:04.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.636+0000 7fd6aaa00640 1 -- 172.21.15.86:0/2547699708 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac05ad60 con 0x5577bae1dea0 2024-04-25T14:06:04.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.636+0000 7fd6aaa00640 1 -- 172.21.15.86:0/2547699708 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6ac05b950 con 0x5577bae1dea0 2024-04-25T14:06:04.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.636+0000 7fd6aaa00640 1 -- 172.21.15.86:0/2547699708 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac05a6b0 con 0x5577bae1dea0 2024-04-25T14:06:04.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.636+0000 7fd6cd53c000 1 -- 172.21.15.86:0/2547699708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577bae1dea0 msgr2=0x5577baf9ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:04.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.636+0000 7fd6cd53c000 1 --2- 172.21.15.86:0/2547699708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577bae1dea0 0x5577baf9ffb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac0584c0 tx=0x7fd6ac057240 comp rx=0 tx=0).stop 2024-04-25T14:06:04.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.636+0000 7fd6cd53c000 1 -- 172.21.15.86:0/2547699708 shutdown_connections 2024-04-25T14:06:04.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.636+0000 7fd6cd53c000 1 --2- 172.21.15.86:0/2547699708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5577bae31c20 0x5577baf9b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.636+0000 7fd6cd53c000 1 --2- 172.21.15.86:0/2547699708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577bae1dea0 0x5577baf9ffb0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.636+0000 7fd6cd53c000 1 --2- 172.21.15.86:0/2547699708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577baa16280 0x5577baf9fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.636+0000 7fd6cd53c000 1 -- 172.21.15.86:0/2547699708 >> 172.21.15.86:0/2547699708 conn(0x5577baa55ff0 msgr2=0x5577baf45d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:04.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.636+0000 7fd6cd53c000 1 -- 172.21.15.86:0/2547699708 shutdown_connections 2024-04-25T14:06:04.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.636+0000 7fd6cd53c000 1 -- 172.21.15.86:0/2547699708 wait complete. 2024-04-25T14:06:04.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.636+0000 7fd6cd53c000 1 -- start start 2024-04-25T14:06:04.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.636+0000 7fd6cd53c000 1 Processor -- start 2024-04-25T14:06:04.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.636+0000 7fd6cd53c000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5577baa16280 0x5577baf9f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:04.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.636+0000 7fd6cd53c000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577bae1dea0 0x5577bafc2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:04.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.636+0000 7fd6cd53c000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577bae31c20 0x5577bafc2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:04.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.636+0000 7fd6cd53c000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5577bad94130 con 0x5577bae31c20 2024-04-25T14:06:04.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.636+0000 7fd6cd53c000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5577bad2a7d0 con 0x5577bae1dea0 2024-04-25T14:06:04.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.636+0000 7fd6cd53c000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5577bada10c0 con 0x5577baa16280 2024-04-25T14:06:04.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.636+0000 7fd6cd53c000 10 client.0 _ll_register_callbacks cb 0x7fd6b00b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:04.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577bae31c20 0x5577bafc2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:04.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577bae31c20 0x5577bafc2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41570/0 (socket says 172.21.15.86:41570) 2024-04-25T14:06:04.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6bf400640 1 -- 172.21.15.86:0/3023385393 learned_addr learned my addr 172.21.15.86:0/3023385393 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:04.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6be000640 1 --2- 172.21.15.86:0/3023385393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577bae1dea0 0x5577bafc2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:04.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6bea00640 1 --2- 172.21.15.86:0/3023385393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5577baa16280 0x5577baf9f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:04.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6bf400640 1 -- 172.21.15.86:0/3023385393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5577baa16280 msgr2=0x5577baf9f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:04.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6bf400640 1 --2- 172.21.15.86:0/3023385393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5577baa16280 0x5577baf9f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6bf400640 1 -- 172.21.15.86:0/3023385393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577bae1dea0 msgr2=0x5577bafc2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:04.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6bf400640 1 --2- 172.21.15.86:0/3023385393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577bae1dea0 0x5577bafc2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6bf400640 1 -- 172.21.15.86:0/3023385393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577bae17180 con 0x5577bae31c20 2024-04-25T14:06:04.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6bf400640 1 --2- 172.21.15.86:0/3023385393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577bae31c20 0x5577bafc2850 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd6b80072f0 tx=0x7fd6b800a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8af5b5a5e88f29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:04.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd69fe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:06:04.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd69fe00640 1 -- 172.21.15.86:0/3023385393 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b803e060 con 0x5577bae31c20 2024-04-25T14:06:04.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd69fe00640 1 -- 172.21.15.86:0/3023385393 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7fd6b8017990 con 0x5577bae31c20 2024-04-25T14:06:04.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6cd53c000 1 -- 172.21.15.86:0/3023385393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6ac05b950 con 0x5577bae31c20 2024-04-25T14:06:04.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd69fe00640 1 -- 172.21.15.86:0/3023385393 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd6b8044050 con 0x5577bae31c20 2024-04-25T14:06:04.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6cd53c000 10 client.6111 fetch_fsmap learned FSMap version 43 2024-04-25T14:06:04.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6cd53c000 1 -- 172.21.15.86:0/3023385393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x5577baa4e0c0 con 0x5577bae31c20 2024-04-25T14:06:04.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd69fe00640 1 -- 172.21.15.86:0/3023385393 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6ac05b950 con 0x5577bae31c20 2024-04-25T14:06:04.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd69fe00640 1 -- 172.21.15.86:0/3023385393 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 43) ==== 4132+0+0 (secure 0 0 0) 0x7fd6b8047020 con 0x5577bae31c20 2024-04-25T14:06:04.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6cd53c000 10 client.6111 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T14:06:04.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6cd53c000 10 client.6111 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T14:06:04.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6cd53c000 20 client.6111 populate_metadata read hostname 'smithi086' 2024-04-25T14:06:04.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6cd53c000 1 --2- 172.21.15.86:0/3023385393 >> [v2:172.21.15.155:6833/2914930843,v1:172.21.15.155:6835/2914930843] conn(0x5577baa28830 0x5577bb005950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:04.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6cd53c000 4 client.6111 mds_command: new command op to 6036 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:06:04.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6cd53c000 1 -- 172.21.15.86:0/3023385393 --> [v2:172.21.15.155:6833/2914930843,v1:172.21.15.155:6835/2914930843] -- command(tid 0: {"prefix": "status"}) -- 0x5577bada2a30 con 0x5577baa28830 2024-04-25T14:06:04.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6bea00640 1 --2- 172.21.15.86:0/3023385393 >> [v2:172.21.15.155:6833/2914930843,v1:172.21.15.155:6835/2914930843] conn(0x5577baa28830 0x5577bb005950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:04.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd69fe00640 10 client.6111 ms_handle_connect on v2:172.21.15.155:6833/2914930843 2024-04-25T14:06:04.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6bea00640 1 --2- 172.21.15.86:0/3023385393 >> [v2:172.21.15.155:6833/2914930843,v1:172.21.15.155:6835/2914930843] conn(0x5577baa28830 0x5577bb005950 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfd0c22945b62e71 server_cookie=1e023a33d06f9b1 in_seq=0 out_seq=0 2024-04-25T14:06:04.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd69fe00640 1 -- 172.21.15.86:0/3023385393 <== mds.0 v2:172.21.15.155:6833/2914930843 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x5577bada2a30 con 0x5577baa28830 2024-04-25T14:06:04.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd69fe00640 10 client.6111 handle_command_reply: tid=0 2024-04-25T14:06:04.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6cd53c000 1 client.6111 shutdown 2024-04-25T14:06:04.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6cd53c000 1 -- 172.21.15.86:0/3023385393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577bae31c20 msgr2=0x5577bafc2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:04.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6cd53c000 1 --2- 172.21.15.86:0/3023385393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577bae31c20 0x5577bafc2850 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd6b80072f0 tx=0x7fd6b800a210 comp rx=0 tx=0).stop 2024-04-25T14:06:04.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6cd53c000 1 -- 172.21.15.86:0/3023385393 shutdown_connections 2024-04-25T14:06:04.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6cd53c000 1 --2- 172.21.15.86:0/3023385393 >> [v2:172.21.15.155:6833/2914930843,v1:172.21.15.155:6835/2914930843] conn(0x5577baa28830 0x5577bb005950 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6cd53c000 1 --2- 172.21.15.86:0/3023385393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577bae31c20 0x5577bafc2850 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6cd53c000 1 --2- 172.21.15.86:0/3023385393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577bae1dea0 0x5577bafc2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6cd53c000 1 --2- 172.21.15.86:0/3023385393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5577baa16280 0x5577baf9f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd69fe00640 0 client.6111 ms_handle_reset on v2:172.21.15.155:6833/2914930843 2024-04-25T14:06:04.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6cd53c000 1 -- 172.21.15.86:0/3023385393 >> 172.21.15.86:0/3023385393 conn(0x5577baa55ff0 msgr2=0x5577bafa89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:04.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6cd53c000 1 -- 172.21.15.86:0/3023385393 shutdown_connections 2024-04-25T14:06:04.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6cd53c000 1 -- 172.21.15.86:0/3023385393 wait complete. 2024-04-25T14:06:04.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.640+0000 7fd6cd53c000 20 client.6111 trim_cache size 0 max 16384 2024-04-25T14:06:04.647 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:06:04.647 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb", 2024-04-25T14:06:04.647 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-25T14:06:04.647 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 6036, 2024-04-25T14:06:04.648 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T14:06:04.648 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T14:06:04.648 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "fs2", 2024-04-25T14:06:04.648 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 4.4028668289999997, 2024-04-25T14:06:04.648 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 43, 2024-04-25T14:06:04.648 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 58, 2024-04-25T14:06:04.648 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 58, 2024-04-25T14:06:04.648 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 4.7322557510000003 2024-04-25T14:06:04.648 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:06:04.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.644+0000 7fd6c5e00640 1 -- 172.21.15.86:0/3358159606 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fd6a006f570 msgr2=0x7fd6a0071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:04.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.644+0000 7fd6c5e00640 1 --2- 172.21.15.86:0/3358159606 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fd6a006f570 0x7fd6a0071a80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00af000 tx=0x7fd6b0008040 comp rx=0 tx=0).stop 2024-04-25T14:06:04.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.644+0000 7fd6c5e00640 1 -- 172.21.15.86:0/3358159606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c009d110 msgr2=0x7fd6c0150a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:04.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.644+0000 7fd6c5e00640 1 --2- 172.21.15.86:0/3358159606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c009d110 0x7fd6c0150a20 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0074ed0 tx=0x7fd6ac00fc00 comp rx=0 tx=0).stop 2024-04-25T14:06:04.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.644+0000 7fd6c5e00640 1 -- 172.21.15.86:0/3358159606 shutdown_connections 2024-04-25T14:06:04.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.644+0000 7fd6c5e00640 1 --2- 172.21.15.86:0/3358159606 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fd6a006f570 0x7fd6a0071a80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.644+0000 7fd6c5e00640 1 --2- 172.21.15.86:0/3358159606 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6c009a4e0 0x7fd6c009e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.644+0000 7fd6c5e00640 1 --2- 172.21.15.86:0/3358159606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c009d110 0x7fd6c0150a20 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.644+0000 7fd6c5e00640 1 --2- 172.21.15.86:0/3358159606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6a4003a30 0x7fd6c00c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.644+0000 7fd6c5e00640 1 -- 172.21.15.86:0/3358159606 >> 172.21.15.86:0/3358159606 conn(0x7fd6c00b4650 msgr2=0x7fd6c00b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:04.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.644+0000 7fd6c5e00640 1 -- 172.21.15.86:0/3358159606 shutdown_connections 2024-04-25T14:06:04.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.644+0000 7fd6c5e00640 1 -- 172.21.15.86:0/3358159606 wait complete. 2024-04-25T14:06:04.663 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=fs2_data' 2024-04-25T14:06:04.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.816+0000 7f2c38000640 1 Processor -- start 2024-04-25T14:06:04.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.816+0000 7f2c38000640 1 -- start start 2024-04-25T14:06:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.820+0000 7f2c38000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2c300cfca0 0x7f2c3006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.820+0000 7f2c38000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c300730b0 0x7f2c300734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.820+0000 7f2c38000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c30072bd0 con 0x7f2c300d16e0 2024-04-25T14:06:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.820+0000 7f2c38000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2c30070470 con 0x7f2c300cfca0 2024-04-25T14:06:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.820+0000 7f2c38000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2c3006f930 con 0x7f2c300730b0 2024-04-25T14:06:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.820+0000 7f2c35600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2c300cfca0 0x7f2c3006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.820+0000 7f2c35600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2c300cfca0 0x7f2c3006f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41586/0 (socket says 172.21.15.86:41586) 2024-04-25T14:06:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.820+0000 7f2c35600640 1 -- 172.21.15.86:0/2282288329 learned_addr learned my addr 172.21.15.86:0/2282288329 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.820+0000 7f2c36000640 1 --2- 172.21.15.86:0/2282288329 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c300730b0 0x7f2c300734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.820+0000 7f2c35600640 1 -- 172.21.15.86:0/2282288329 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c300730b0 msgr2=0x7f2c300734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.820+0000 7f2c35600640 1 --2- 172.21.15.86:0/2282288329 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c300730b0 0x7f2c300734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.820+0000 7f2c35600640 1 -- 172.21.15.86:0/2282288329 >> v1:172.21.15.155:6789/0 conn(0x7f2c300d16e0 legacy=0x7f2c300d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.820+0000 7f2c35600640 1 -- 172.21.15.86:0/2282288329 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3006fbd0 con 0x7f2c300cfca0 2024-04-25T14:06:04.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.820+0000 7f2c36000640 1 --2- 172.21.15.86:0/2282288329 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c300730b0 0x7f2c300734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:04.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.820+0000 7f2c35600640 1 --2- 172.21.15.86:0/2282288329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c300cfca0 0x7f2c3006f3f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f2c24009870 tx=0x7f2c2401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc2cfcba0a5e489 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:04.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.820+0000 7f2c2fe00640 1 -- 172.21.15.86:0/2282288329 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c240a7070 con 0x7f2c300cfca0 2024-04-25T14:06:04.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.824+0000 7f2c2fe00640 1 -- 172.21.15.86:0/2282288329 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c240b0070 con 0x7f2c300cfca0 2024-04-25T14:06:04.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.824+0000 7f2c2fe00640 1 -- 172.21.15.86:0/2282288329 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c240ab890 con 0x7f2c300cfca0 2024-04-25T14:06:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.824+0000 7f2c38000640 1 -- 172.21.15.86:0/2282288329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c300cfca0 msgr2=0x7f2c3006f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.824+0000 7f2c38000640 1 --2- 172.21.15.86:0/2282288329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c300cfca0 0x7f2c3006f3f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f2c24009870 tx=0x7f2c2401c560 comp rx=0 tx=0).stop 2024-04-25T14:06:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.824+0000 7f2c38000640 1 -- 172.21.15.86:0/2282288329 shutdown_connections 2024-04-25T14:06:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.824+0000 7f2c38000640 1 --2- 172.21.15.86:0/2282288329 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c300730b0 0x7f2c300734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.824+0000 7f2c38000640 1 --2- 172.21.15.86:0/2282288329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c300cfca0 0x7f2c3006f3f0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.824+0000 7f2c38000640 1 -- 172.21.15.86:0/2282288329 >> 172.21.15.86:0/2282288329 conn(0x7f2c300b4650 msgr2=0x7f2c300b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.824+0000 7f2c38000640 1 -- 172.21.15.86:0/2282288329 shutdown_connections 2024-04-25T14:06:04.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.824+0000 7f2c38000640 1 -- 172.21.15.86:0/2282288329 wait complete. 2024-04-25T14:06:04.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.824+0000 7f2c38000640 1 Processor -- start 2024-04-25T14:06:04.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.824+0000 7f2c38000640 1 -- start start 2024-04-25T14:06:04.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.828+0000 7f2c38000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c300730b0 0x7f2c3015d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:04.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.828+0000 7f2c38000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c300cfca0 0x7f2c301fd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:04.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.828+0000 7f2c38000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c300d16e0 0x7f2c3015ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:04.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.828+0000 7f2c38000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c30169a70 con 0x7f2c300d16e0 2024-04-25T14:06:04.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.828+0000 7f2c38000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2c301698f0 con 0x7f2c300cfca0 2024-04-25T14:06:04.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.828+0000 7f2c38000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2c30169bf0 con 0x7f2c300730b0 2024-04-25T14:06:04.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.828+0000 7f2c36000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c300d16e0 0x7f2c3015ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:04.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.828+0000 7f2c36000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c300d16e0 0x7f2c3015ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41588/0 (socket says 172.21.15.86:41588) 2024-04-25T14:06:04.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.828+0000 7f2c36000640 1 -- 172.21.15.86:0/3526977895 learned_addr learned my addr 172.21.15.86:0/3526977895 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:04.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.828+0000 7f2c35600640 1 --2- 172.21.15.86:0/3526977895 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c300730b0 0x7f2c3015d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:04.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.828+0000 7f2c34c00640 1 --2- 172.21.15.86:0/3526977895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c300cfca0 0x7f2c301fd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:04.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.828+0000 7f2c36000640 1 -- 172.21.15.86:0/3526977895 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c300730b0 msgr2=0x7f2c3015d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:04.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.828+0000 7f2c36000640 1 --2- 172.21.15.86:0/3526977895 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c300730b0 0x7f2c3015d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.828+0000 7f2c36000640 1 -- 172.21.15.86:0/3526977895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c300cfca0 msgr2=0x7f2c301fd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:04.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.828+0000 7f2c36000640 1 --2- 172.21.15.86:0/3526977895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c300cfca0 0x7f2c301fd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:04.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.828+0000 7f2c36000640 1 -- 172.21.15.86:0/3526977895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c300cae90 con 0x7f2c300d16e0 2024-04-25T14:06:04.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.828+0000 7f2c36000640 1 --2- 172.21.15.86:0/3526977895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c300d16e0 0x7f2c3015ed20 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f2c28007e40 tx=0x7f2c2800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:04.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.828+0000 7f2c2d600640 1 -- 172.21.15.86:0/3526977895 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2801a070 con 0x7f2c300d16e0 2024-04-25T14:06:04.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.828+0000 7f2c2d600640 1 -- 172.21.15.86:0/3526977895 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c28003bc0 con 0x7f2c300d16e0 2024-04-25T14:06:04.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.828+0000 7f2c38000640 1 -- 172.21.15.86:0/3526977895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c3009a4e0 con 0x7f2c300d16e0 2024-04-25T14:06:04.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.828+0000 7f2c2d600640 1 -- 172.21.15.86:0/3526977895 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2800de20 con 0x7f2c300d16e0 2024-04-25T14:06:04.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.828+0000 7f2c38000640 1 -- 172.21.15.86:0/3526977895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c3009a900 con 0x7f2c300d16e0 2024-04-25T14:06:04.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.828+0000 7f2c38000640 1 -- 172.21.15.86:0/3526977895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c3009f4b0 con 0x7f2c300d16e0 2024-04-25T14:06:04.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.832+0000 7f2c2d600640 1 -- 172.21.15.86:0/3526977895 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f2c2801d070 con 0x7f2c300d16e0 2024-04-25T14:06:04.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.832+0000 7f2c2d600640 1 --2- 172.21.15.86:0/3526977895 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f2c1006f570 0x7f2c10071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:04.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.832+0000 7f2c2d600640 1 -- 172.21.15.86:0/3526977895 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7f2c2808ff30 con 0x7f2c300d16e0 2024-04-25T14:06:04.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.832+0000 7f2c35600640 1 --2- 172.21.15.86:0/3526977895 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f2c1006f570 0x7f2c10071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:04.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.832+0000 7f2c35600640 1 --2- 172.21.15.86:0/3526977895 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f2c1006f570 0x7f2c10071a80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2c240af000 tx=0x7f2c240a7000 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:04.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:04.836+0000 7f2c2d600640 1 -- 172.21.15.86:0/3526977895 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2c28010070 con 0x7f2c300d16e0 2024-04-25T14:06:05.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.052+0000 7f2c38000640 1 -- 172.21.15.86:0/3526977895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=fs2_data"]} v 0) -- 0x7f2c30069d00 con 0x7f2c300d16e0 2024-04-25T14:06:05.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.056+0000 7f2c2d600640 1 -- 172.21.15.86:0/3526977895 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=fs2_data"]}]=0 updated caps for client.0 v20) ==== 181+0+138 (secure 0 0 0) 0x7f2c2805bdf0 con 0x7f2c300d16e0 2024-04-25T14:06:05.057 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-04-25T14:06:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCLYipm5u4xHRAACDFe3Eb4KLrsdLj4eFe7mA== 2024-04-25T14:06:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-04-25T14:06:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-04-25T14:06:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw pool=fs2_data" 2024-04-25T14:06:05.057 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.0 2024-04-25T14:06:05.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.056+0000 7f2c38000640 1 -- 172.21.15.86:0/3526977895 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f2c1006f570 msgr2=0x7f2c10071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:05.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.056+0000 7f2c38000640 1 --2- 172.21.15.86:0/3526977895 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f2c1006f570 0x7f2c10071a80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2c240af000 tx=0x7f2c240a7000 comp rx=0 tx=0).stop 2024-04-25T14:06:05.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.056+0000 7f2c38000640 1 -- 172.21.15.86:0/3526977895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c300d16e0 msgr2=0x7f2c3015ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:05.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.056+0000 7f2c38000640 1 --2- 172.21.15.86:0/3526977895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c300d16e0 0x7f2c3015ed20 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f2c28007e40 tx=0x7f2c2800d4a0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.056+0000 7f2c38000640 1 -- 172.21.15.86:0/3526977895 shutdown_connections 2024-04-25T14:06:05.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.056+0000 7f2c38000640 1 --2- 172.21.15.86:0/3526977895 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f2c1006f570 0x7f2c10071a80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.056+0000 7f2c38000640 1 --2- 172.21.15.86:0/3526977895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c300d16e0 0x7f2c3015ed20 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.056+0000 7f2c38000640 1 --2- 172.21.15.86:0/3526977895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c300cfca0 0x7f2c301fd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.056+0000 7f2c38000640 1 --2- 172.21.15.86:0/3526977895 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c300730b0 0x7f2c3015d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.056+0000 7f2c38000640 1 -- 172.21.15.86:0/3526977895 >> 172.21.15.86:0/3526977895 conn(0x7f2c300b4650 msgr2=0x7f2c30159ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:05.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.056+0000 7f2c38000640 1 -- 172.21.15.86:0/3526977895 shutdown_connections 2024-04-25T14:06:05.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.056+0000 7f2c38000640 1 -- 172.21.15.86:0/3526977895 wait complete. 2024-04-25T14:06:05.080 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=fs2_data' 2024-04-25T14:06:05.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.232+0000 7f9c5fe00640 1 Processor -- start 2024-04-25T14:06:05.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.232+0000 7f9c5fe00640 1 -- start start 2024-04-25T14:06:05.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.232+0000 7f9c5fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9c580d1df0 0x7f9c580b0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.232+0000 7f9c5fe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c580cde20 con 0x7f9c580b0690 2024-04-25T14:06:05.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.232+0000 7f9c5fe00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c580ce100 con 0x7f9c581642a0 2024-04-25T14:06:05.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.232+0000 7f9c5fe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9c580cea30 con 0x7f9c580d1df0 2024-04-25T14:06:05.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.232+0000 7f9c5ca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9c580d1df0 0x7f9c580b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.232+0000 7f9c5de00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9c580b0690 0x7f9c580d0640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51014/0 (socket says 172.21.15.86:51014) 2024-04-25T14:06:05.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.232+0000 7f9c5de00640 1 -- 172.21.15.86:0/2871686235 learned_addr learned my addr 172.21.15.86:0/2871686235 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:05.236 INFO:tasks.ceph.mds.d.smithi155.stderr:2024-04-25T14:06:05.237+0000 7f74a4200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T14:06:05.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.232+0000 7f9c5ca00640 1 -- 172.21.15.86:0/2871686235 >> v1:172.21.15.155:6789/0 conn(0x7f9c581642a0 legacy=0x7f9c580d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:05.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.232+0000 7f9c5ca00640 1 -- 172.21.15.86:0/2871686235 >> v1:172.21.15.86:6789/0 conn(0x7f9c580b0690 legacy=0x7f9c580d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:05.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.232+0000 7f9c5ca00640 1 -- 172.21.15.86:0/2871686235 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c580ce3e0 con 0x7f9c580d1df0 2024-04-25T14:06:05.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.232+0000 7f9c57e00640 1 -- 172.21.15.86:0/2871686235 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2721649589 0 0) 0x7f9c580cde20 con 0x7f9c580b0690 2024-04-25T14:06:05.237 INFO:tasks.ceph.mds.c.smithi086.stderr:2024-04-25T14:06:05.232+0000 7f63a1a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T14:06:05.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.236+0000 7f9c5ca00640 1 --2- 172.21.15.86:0/2871686235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c580d1df0 0x7f9c580b0150 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9c440097c0 tx=0x7f9c4401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f114f4fb696f4ca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:05.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.236+0000 7f9c57e00640 1 -- 172.21.15.86:0/2871686235 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c440b0070 con 0x7f9c580d1df0 2024-04-25T14:06:05.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.236+0000 7f9c57e00640 1 -- 172.21.15.86:0/2871686235 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c440ab600 con 0x7f9c580d1df0 2024-04-25T14:06:05.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.236+0000 7f9c57e00640 1 -- 172.21.15.86:0/2871686235 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c440ab9d0 con 0x7f9c580d1df0 2024-04-25T14:06:05.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.236+0000 7f9c5fe00640 1 -- 172.21.15.86:0/2871686235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c580d1df0 msgr2=0x7f9c580b0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:05.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.236+0000 7f9c5fe00640 1 --2- 172.21.15.86:0/2871686235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c580d1df0 0x7f9c580b0150 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9c440097c0 tx=0x7f9c4401c410 comp rx=0 tx=0).stop 2024-04-25T14:06:05.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.236+0000 7f9c5fe00640 1 -- 172.21.15.86:0/2871686235 shutdown_connections 2024-04-25T14:06:05.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.236+0000 7f9c5fe00640 1 --2- 172.21.15.86:0/2871686235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c580d1df0 0x7f9c580b0150 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.236+0000 7f9c5fe00640 1 -- 172.21.15.86:0/2871686235 >> 172.21.15.86:0/2871686235 conn(0x7f9c580b4650 msgr2=0x7f9c580b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:05.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.236+0000 7f9c5fe00640 1 -- 172.21.15.86:0/2871686235 shutdown_connections 2024-04-25T14:06:05.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.236+0000 7f9c5fe00640 1 -- 172.21.15.86:0/2871686235 wait complete. 2024-04-25T14:06:05.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.240+0000 7f9c5fe00640 1 Processor -- start 2024-04-25T14:06:05.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.240+0000 7f9c5fe00640 1 -- start start 2024-04-25T14:06:05.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.240+0000 7f9c5fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c580b0690 0x7f9c5815cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.240+0000 7f9c5fe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c581642a0 0x7f9c58074270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.240+0000 7f9c5fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5815d260 0x7f9c580747b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.240+0000 7f9c5fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c581648c0 con 0x7f9c5815d260 2024-04-25T14:06:05.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.240+0000 7f9c5fe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9c580d1160 con 0x7f9c580b0690 2024-04-25T14:06:05.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.240+0000 7f9c5fe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9c58160da0 con 0x7f9c581642a0 2024-04-25T14:06:05.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.240+0000 7f9c5de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5815d260 0x7f9c580747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.240+0000 7f9c5de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5815d260 0x7f9c580747b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41614/0 (socket says 172.21.15.86:41614) 2024-04-25T14:06:05.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.240+0000 7f9c5de00640 1 -- 172.21.15.86:0/1659904019 learned_addr learned my addr 172.21.15.86:0/1659904019 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:05.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.240+0000 7f9c5ca00640 1 --2- 172.21.15.86:0/1659904019 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c581642a0 0x7f9c58074270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.240+0000 7f9c5d400640 1 --2- 172.21.15.86:0/1659904019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c580b0690 0x7f9c5815cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.244+0000 7f9c5de00640 1 -- 172.21.15.86:0/1659904019 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c581642a0 msgr2=0x7f9c58074270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:05.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.244+0000 7f9c5de00640 1 --2- 172.21.15.86:0/1659904019 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c581642a0 0x7f9c58074270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.244+0000 7f9c5de00640 1 -- 172.21.15.86:0/1659904019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c580b0690 msgr2=0x7f9c5815cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:05.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.244+0000 7f9c5de00640 1 --2- 172.21.15.86:0/1659904019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c580b0690 0x7f9c5815cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.244+0000 7f9c5de00640 1 -- 172.21.15.86:0/1659904019 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5815bf20 con 0x7f9c5815d260 2024-04-25T14:06:05.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.244+0000 7f9c5de00640 1 --2- 172.21.15.86:0/1659904019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5815d260 0x7f9c580747b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9c5000c440 tx=0x7f9c5000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:05.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.244+0000 7f9c55600640 1 -- 172.21.15.86:0/1659904019 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c500074e0 con 0x7f9c5815d260 2024-04-25T14:06:05.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.244+0000 7f9c55600640 1 -- 172.21.15.86:0/1659904019 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c500047f0 con 0x7f9c5815d260 2024-04-25T14:06:05.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.244+0000 7f9c5fe00640 1 -- 172.21.15.86:0/1659904019 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c5815ae80 con 0x7f9c5815d260 2024-04-25T14:06:05.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.244+0000 7f9c55600640 1 -- 172.21.15.86:0/1659904019 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c50004be0 con 0x7f9c5815d260 2024-04-25T14:06:05.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.244+0000 7f9c5fe00640 1 -- 172.21.15.86:0/1659904019 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c5815b440 con 0x7f9c5815d260 2024-04-25T14:06:05.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.244+0000 7f9c5fe00640 1 -- 172.21.15.86:0/1659904019 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c58069d00 con 0x7f9c5815d260 2024-04-25T14:06:05.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.244+0000 7f9c55600640 1 -- 172.21.15.86:0/1659904019 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f9c50011070 con 0x7f9c5815d260 2024-04-25T14:06:05.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.248+0000 7f9c55600640 1 --2- 172.21.15.86:0/1659904019 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f9c3806f570 0x7f9c38071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.248+0000 7f9c55600640 1 -- 172.21.15.86:0/1659904019 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7f9c5008f670 con 0x7f9c5815d260 2024-04-25T14:06:05.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.248+0000 7f9c5d400640 1 --2- 172.21.15.86:0/1659904019 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f9c3806f570 0x7f9c38071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.248+0000 7f9c5d400640 1 --2- 172.21.15.86:0/1659904019 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f9c3806f570 0x7f9c38071a80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0012f0 tx=0x7f9c4c009040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:05.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.252+0000 7f9c55600640 1 -- 172.21.15.86:0/1659904019 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9c5005b640 con 0x7f9c5815d260 2024-04-25T14:06:05.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.464+0000 7f9c5fe00640 1 -- 172.21.15.86:0/1659904019 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=fs2_data"]} v 0) -- 0x7f9c58076bf0 con 0x7f9c5815d260 2024-04-25T14:06:05.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.468+0000 7f9c55600640 1 -- 172.21.15.86:0/1659904019 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=fs2_data"]}]=0 updated caps for client.1 v21) ==== 181+0+138 (secure 0 0 0) 0x7f9c5005f940 con 0x7f9c5815d260 2024-04-25T14:06:05.471 INFO:teuthology.orchestra.run.smithi086.stdout:[client.1] 2024-04-25T14:06:05.471 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCLYipmDp1hJRAAQkBc5PdQoMTT7W4HahkTyg== 2024-04-25T14:06:05.471 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-04-25T14:06:05.471 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-04-25T14:06:05.472 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw pool=fs2_data" 2024-04-25T14:06:05.472 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.1 2024-04-25T14:06:05.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.472+0000 7f9c5fe00640 1 -- 172.21.15.86:0/1659904019 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f9c3806f570 msgr2=0x7f9c38071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:05.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.472+0000 7f9c5fe00640 1 --2- 172.21.15.86:0/1659904019 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f9c3806f570 0x7f9c38071a80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0012f0 tx=0x7f9c4c009040 comp rx=0 tx=0).stop 2024-04-25T14:06:05.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.472+0000 7f9c5fe00640 1 -- 172.21.15.86:0/1659904019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5815d260 msgr2=0x7f9c580747b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:05.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.472+0000 7f9c5fe00640 1 --2- 172.21.15.86:0/1659904019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5815d260 0x7f9c580747b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9c5000c440 tx=0x7f9c5000c840 comp rx=0 tx=0).stop 2024-04-25T14:06:05.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.472+0000 7f9c5fe00640 1 -- 172.21.15.86:0/1659904019 shutdown_connections 2024-04-25T14:06:05.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.472+0000 7f9c5fe00640 1 --2- 172.21.15.86:0/1659904019 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f9c3806f570 0x7f9c38071a80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.472+0000 7f9c5fe00640 1 --2- 172.21.15.86:0/1659904019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5815d260 0x7f9c580747b0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.472+0000 7f9c5fe00640 1 --2- 172.21.15.86:0/1659904019 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c581642a0 0x7f9c58074270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.472+0000 7f9c5fe00640 1 --2- 172.21.15.86:0/1659904019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c580b0690 0x7f9c5815cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.472+0000 7f9c5fe00640 1 -- 172.21.15.86:0/1659904019 >> 172.21.15.86:0/1659904019 conn(0x7f9c580b4650 msgr2=0x7f9c58162a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:05.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.472+0000 7f9c5fe00640 1 -- 172.21.15.86:0/1659904019 shutdown_connections 2024-04-25T14:06:05.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.472+0000 7f9c5fe00640 1 -- 172.21.15.86:0/1659904019 wait complete. 2024-04-25T14:06:05.494 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=fs2_data' 2024-04-25T14:06:05.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.652+0000 7f0bb4a00640 1 Processor -- start 2024-04-25T14:06:05.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.652+0000 7f0bb4a00640 1 -- start start 2024-04-25T14:06:05.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.652+0000 7f0bb4a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0bb00d1df0 0x7f0bb00cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.652+0000 7f0bb4a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0bb00cdcc0 0x7f0bb00ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.652+0000 7f0bb4a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bb015e5c0 con 0x7f0bb0070330 2024-04-25T14:06:05.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.652+0000 7f0bb4a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0bb006fd40 con 0x7f0bb00d1df0 2024-04-25T14:06:05.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.652+0000 7f0bb4a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0bb015e8a0 con 0x7f0bb00cdcc0 2024-04-25T14:06:05.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.652+0000 7f0ba7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0bb00d1df0 0x7f0bb00cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.652+0000 7f0ba7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0bb00d1df0 0x7f0bb00cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41616/0 (socket says 172.21.15.86:41616) 2024-04-25T14:06:05.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.652+0000 7f0ba7e00640 1 -- 172.21.15.86:0/107450012 learned_addr learned my addr 172.21.15.86:0/107450012 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:05.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.652+0000 7f0bae800640 1 --2- 172.21.15.86:0/107450012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0bb00cdcc0 0x7f0bb00ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.652+0000 7f0ba7e00640 1 -- 172.21.15.86:0/107450012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0bb00cdcc0 msgr2=0x7f0bb00ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:05.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.652+0000 7f0ba7e00640 1 --2- 172.21.15.86:0/107450012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0bb00cdcc0 0x7f0bb00ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.652+0000 7f0ba7e00640 1 -- 172.21.15.86:0/107450012 >> v1:172.21.15.155:6789/0 conn(0x7f0bb0070330 legacy=0x7f0bb00d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:05.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.652+0000 7f0ba7e00640 1 -- 172.21.15.86:0/107450012 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb015eb80 con 0x7f0bb00d1df0 2024-04-25T14:06:05.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.656+0000 7f0ba7e00640 1 --2- 172.21.15.86:0/107450012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bb00d1df0 0x7f0bb00cd780 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009870 tx=0x7f0b9801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2763313da4df5ffa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:05.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.656+0000 7f0bad400640 1 -- 172.21.15.86:0/107450012 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b980ab680 con 0x7f0bb00d1df0 2024-04-25T14:06:05.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.656+0000 7f0bad400640 1 -- 172.21.15.86:0/107450012 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b980b0070 con 0x7f0bb00d1df0 2024-04-25T14:06:05.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.656+0000 7f0bad400640 1 -- 172.21.15.86:0/107450012 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b980aba40 con 0x7f0bb00d1df0 2024-04-25T14:06:05.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.656+0000 7f0bb4a00640 1 -- 172.21.15.86:0/107450012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bb00d1df0 msgr2=0x7f0bb00cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:05.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.656+0000 7f0bb4a00640 1 --2- 172.21.15.86:0/107450012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bb00d1df0 0x7f0bb00cd780 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009870 tx=0x7f0b9801c4a0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.656+0000 7f0bb4a00640 1 -- 172.21.15.86:0/107450012 shutdown_connections 2024-04-25T14:06:05.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.656+0000 7f0bb4a00640 1 --2- 172.21.15.86:0/107450012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0bb00cdcc0 0x7f0bb00ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.656+0000 7f0bb4a00640 1 --2- 172.21.15.86:0/107450012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bb00d1df0 0x7f0bb00cd780 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.656+0000 7f0bb4a00640 1 -- 172.21.15.86:0/107450012 >> 172.21.15.86:0/107450012 conn(0x7f0bb00b4650 msgr2=0x7f0bb00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:05.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.656+0000 7f0bb4a00640 1 -- 172.21.15.86:0/107450012 shutdown_connections 2024-04-25T14:06:05.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.656+0000 7f0bb4a00640 1 -- 172.21.15.86:0/107450012 wait complete. 2024-04-25T14:06:05.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.660+0000 7f0bb4a00640 1 Processor -- start 2024-04-25T14:06:05.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.660+0000 7f0bb4a00640 1 -- start start 2024-04-25T14:06:05.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.660+0000 7f0bb4a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0bb0070330 0x7f0bb00c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.660+0000 7f0bb4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bb00cdcc0 0x7f0bb00af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.660+0000 7f0bb4a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bb00cff00 0x7f0bb00d0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.660+0000 7f0bb4a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb015f4b0 con 0x7f0bb00cdcc0 2024-04-25T14:06:05.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.660+0000 7f0bb4a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb015f330 con 0x7f0bb00cff00 2024-04-25T14:06:05.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.660+0000 7f0bb4a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0bb0070740 con 0x7f0bb0070330 2024-04-25T14:06:05.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.660+0000 7f0ba7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bb00cdcc0 0x7f0bb00af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.660+0000 7f0ba7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bb00cdcc0 0x7f0bb00af7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41624/0 (socket says 172.21.15.86:41624) 2024-04-25T14:06:05.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.660+0000 7f0ba7e00640 1 -- 172.21.15.86:0/245352159 learned_addr learned my addr 172.21.15.86:0/245352159 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:05.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.660+0000 7f0bae800640 1 --2- 172.21.15.86:0/245352159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bb00cff00 0x7f0bb00d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.660+0000 7f0bade00640 1 --2- 172.21.15.86:0/245352159 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0bb0070330 0x7f0bb00c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.660+0000 7f0ba7e00640 1 -- 172.21.15.86:0/245352159 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0bb0070330 msgr2=0x7f0bb00c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:05.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.660+0000 7f0ba7e00640 1 --2- 172.21.15.86:0/245352159 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0bb0070330 0x7f0bb00c5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.660+0000 7f0ba7e00640 1 -- 172.21.15.86:0/245352159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bb00cff00 msgr2=0x7f0bb00d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:05.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.660+0000 7f0ba7e00640 1 --2- 172.21.15.86:0/245352159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bb00cff00 0x7f0bb00d0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.660+0000 7f0ba7e00640 1 -- 172.21.15.86:0/245352159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb00d0850 con 0x7f0bb00cdcc0 2024-04-25T14:06:05.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.660+0000 7f0ba7e00640 1 --2- 172.21.15.86:0/245352159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bb00cdcc0 0x7f0bb00af7f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0b9801ab50 tx=0x7f0b98019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:05.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.660+0000 7f0ba6000640 1 -- 172.21.15.86:0/245352159 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b980bf610 con 0x7f0bb00cdcc0 2024-04-25T14:06:05.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.660+0000 7f0ba6000640 1 -- 172.21.15.86:0/245352159 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b980a7070 con 0x7f0bb00cdcc0 2024-04-25T14:06:05.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.660+0000 7f0ba6000640 1 -- 172.21.15.86:0/245352159 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b980bfa10 con 0x7f0bb00cdcc0 2024-04-25T14:06:05.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.660+0000 7f0bb4a00640 1 -- 172.21.15.86:0/245352159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bb00ae410 con 0x7f0bb00cdcc0 2024-04-25T14:06:05.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.664+0000 7f0bb4a00640 1 -- 172.21.15.86:0/245352159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bb00ae6f0 con 0x7f0bb00cdcc0 2024-04-25T14:06:05.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.664+0000 7f0bb4a00640 1 -- 172.21.15.86:0/245352159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bb0069d00 con 0x7f0bb00cdcc0 2024-04-25T14:06:05.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.664+0000 7f0ba6000640 1 -- 172.21.15.86:0/245352159 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f0b980bd020 con 0x7f0bb00cdcc0 2024-04-25T14:06:05.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.664+0000 7f0ba6000640 1 --2- 172.21.15.86:0/245352159 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f0b8c06f570 0x7f0b8c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.664+0000 7f0ba6000640 1 -- 172.21.15.86:0/245352159 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7f0b98129e90 con 0x7f0bb00cdcc0 2024-04-25T14:06:05.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.664+0000 7f0bade00640 1 --2- 172.21.15.86:0/245352159 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f0b8c06f570 0x7f0b8c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.664+0000 7f0bade00640 1 --2- 172.21.15.86:0/245352159 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f0b8c06f570 0x7f0b8c071a80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0072200 tx=0x7f0ba0009040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:05.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.668+0000 7f0ba6000640 1 -- 172.21.15.86:0/245352159 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0b980c6070 con 0x7f0bb00cdcc0 2024-04-25T14:06:05.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.880+0000 7f0bb4a00640 1 -- 172.21.15.86:0/245352159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=fs2_data"]} v 0) -- 0x7f0bb00ae180 con 0x7f0bb00cdcc0 2024-04-25T14:06:05.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.888+0000 7f0ba6000640 1 -- 172.21.15.86:0/245352159 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=fs2_data"]}]=0 updated caps for client.2 v22) ==== 181+0+138 (secure 0 0 0) 0x7f0bb00d0850 con 0x7f0bb00cdcc0 2024-04-25T14:06:05.888 INFO:teuthology.orchestra.run.smithi086.stdout:[client.2] 2024-04-25T14:06:05.888 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCLYipmhbhGIRAAxao/J0WY53IITLETFgwIVw== 2024-04-25T14:06:05.888 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-04-25T14:06:05.888 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-04-25T14:06:05.888 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw pool=fs2_data" 2024-04-25T14:06:05.889 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.2 2024-04-25T14:06:05.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.888+0000 7f0bb4a00640 1 -- 172.21.15.86:0/245352159 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f0b8c06f570 msgr2=0x7f0b8c071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:05.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.888+0000 7f0bb4a00640 1 --2- 172.21.15.86:0/245352159 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f0b8c06f570 0x7f0b8c071a80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0072200 tx=0x7f0ba0009040 comp rx=0 tx=0).stop 2024-04-25T14:06:05.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.888+0000 7f0bb4a00640 1 -- 172.21.15.86:0/245352159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bb00cdcc0 msgr2=0x7f0bb00af7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:05.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.888+0000 7f0bb4a00640 1 --2- 172.21.15.86:0/245352159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bb00cdcc0 0x7f0bb00af7f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0b9801ab50 tx=0x7f0b98019240 comp rx=0 tx=0).stop 2024-04-25T14:06:05.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.888+0000 7f0bb4a00640 1 -- 172.21.15.86:0/245352159 shutdown_connections 2024-04-25T14:06:05.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.888+0000 7f0bb4a00640 1 --2- 172.21.15.86:0/245352159 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f0b8c06f570 0x7f0b8c071a80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.888+0000 7f0bb4a00640 1 --2- 172.21.15.86:0/245352159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bb00cff00 0x7f0bb00d0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.888+0000 7f0bb4a00640 1 --2- 172.21.15.86:0/245352159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bb00cdcc0 0x7f0bb00af7f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.888+0000 7f0bb4a00640 1 --2- 172.21.15.86:0/245352159 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0bb0070330 0x7f0bb00c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.888+0000 7f0bb4a00640 1 -- 172.21.15.86:0/245352159 >> 172.21.15.86:0/245352159 conn(0x7f0bb00b4650 msgr2=0x7f0bb00d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:05.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.888+0000 7f0bb4a00640 1 -- 172.21.15.86:0/245352159 shutdown_connections 2024-04-25T14:06:05.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:05.888+0000 7f0bb4a00640 1 -- 172.21.15.86:0/245352159 wait complete. 2024-04-25T14:06:05.910 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.3 mds allow mon 'allow r' osd 'allow rw pool=fs2_data' 2024-04-25T14:06:06.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.060+0000 7fcf65000640 1 Processor -- start 2024-04-25T14:06:06.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.060+0000 7fcf65000640 1 -- start start 2024-04-25T14:06:06.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.064+0000 7fcf65000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcf6015fe90 0x7fcf601643b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:06.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.064+0000 7fcf65000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf601648f0 con 0x7fcf600d10c0 2024-04-25T14:06:06.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.064+0000 7fcf65000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf600d19c0 con 0x7fcf600d14d0 2024-04-25T14:06:06.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.064+0000 7fcf65000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcf600d1ca0 con 0x7fcf6015fe90 2024-04-25T14:06:06.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.064+0000 7fcf5de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcf6015fe90 0x7fcf601643b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:06.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.064+0000 7fcf55400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fcf600d10c0 0x7fcf600cfdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51018/0 (socket says 172.21.15.86:51018) 2024-04-25T14:06:06.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.064+0000 7fcf55400640 1 -- 172.21.15.86:0/2011625299 learned_addr learned my addr 172.21.15.86:0/2011625299 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:06.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.064+0000 7fcf5de00640 1 -- 172.21.15.86:0/2011625299 >> v1:172.21.15.155:6789/0 conn(0x7fcf600d14d0 legacy=0x7fcf600b7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:06.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.064+0000 7fcf5de00640 1 -- 172.21.15.86:0/2011625299 >> v1:172.21.15.86:6789/0 conn(0x7fcf600d10c0 legacy=0x7fcf600cfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:06.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.064+0000 7fcf5d400640 1 -- 172.21.15.86:0/2011625299 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 367388337 0 0) 0x7fcf601648f0 con 0x7fcf600d10c0 2024-04-25T14:06:06.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.064+0000 7fcf5de00640 1 -- 172.21.15.86:0/2011625299 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf60163bb0 con 0x7fcf6015fe90 2024-04-25T14:06:06.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.068+0000 7fcf5de00640 1 --2- 172.21.15.86:0/2011625299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf6015fe90 0x7fcf601643b0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fcf500097c0 tx=0x7fcf5001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90667f1030a84a49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:06.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.068+0000 7fcf5d400640 1 -- 172.21.15.86:0/2011625299 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf500b1070 con 0x7fcf6015fe90 2024-04-25T14:06:06.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.068+0000 7fcf5d400640 1 -- 172.21.15.86:0/2011625299 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf500ac600 con 0x7fcf6015fe90 2024-04-25T14:06:06.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.068+0000 7fcf5d400640 1 -- 172.21.15.86:0/2011625299 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf500ac9d0 con 0x7fcf6015fe90 2024-04-25T14:06:06.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.068+0000 7fcf65000640 1 -- 172.21.15.86:0/2011625299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf6015fe90 msgr2=0x7fcf601643b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:06.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.068+0000 7fcf65000640 1 --2- 172.21.15.86:0/2011625299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf6015fe90 0x7fcf601643b0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fcf500097c0 tx=0x7fcf5001c3e0 comp rx=0 tx=0).stop 2024-04-25T14:06:06.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.068+0000 7fcf65000640 1 -- 172.21.15.86:0/2011625299 shutdown_connections 2024-04-25T14:06:06.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.068+0000 7fcf65000640 1 --2- 172.21.15.86:0/2011625299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf6015fe90 0x7fcf601643b0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:06.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.068+0000 7fcf65000640 1 -- 172.21.15.86:0/2011625299 >> 172.21.15.86:0/2011625299 conn(0x7fcf600b5020 msgr2=0x7fcf600b5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:06.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.068+0000 7fcf65000640 1 -- 172.21.15.86:0/2011625299 shutdown_connections 2024-04-25T14:06:06.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.068+0000 7fcf65000640 1 -- 172.21.15.86:0/2011625299 wait complete. 2024-04-25T14:06:06.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.068+0000 7fcf65000640 1 Processor -- start 2024-04-25T14:06:06.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.072+0000 7fcf65000640 1 -- start start 2024-04-25T14:06:06.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.072+0000 7fcf65000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf600d10c0 0x7fcf600b83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:06.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.072+0000 7fcf65000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf600d14d0 0x7fcf600b8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:06.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.072+0000 7fcf65000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf6015fe90 0x7fcf6015c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:06.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.072+0000 7fcf65000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6016e6f0 con 0x7fcf600d14d0 2024-04-25T14:06:06.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.072+0000 7fcf65000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6016e570 con 0x7fcf600d10c0 2024-04-25T14:06:06.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.072+0000 7fcf65000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcf6016e870 con 0x7fcf6015fe90 2024-04-25T14:06:06.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.072+0000 7fcf55400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf600d14d0 0x7fcf600b8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.072+0000 7fcf55400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf600d14d0 0x7fcf600b8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41632/0 (socket says 172.21.15.86:41632) 2024-04-25T14:06:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.072+0000 7fcf55400640 1 -- 172.21.15.86:0/3549668328 learned_addr learned my addr 172.21.15.86:0/3549668328 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.072+0000 7fcf5de00640 1 --2- 172.21.15.86:0/3549668328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf600d10c0 0x7fcf600b83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.072+0000 7fcf5e800640 1 --2- 172.21.15.86:0/3549668328 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf6015fe90 0x7fcf6015c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.072+0000 7fcf55400640 1 -- 172.21.15.86:0/3549668328 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf6015fe90 msgr2=0x7fcf6015c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.072+0000 7fcf55400640 1 --2- 172.21.15.86:0/3549668328 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf6015fe90 0x7fcf6015c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.072+0000 7fcf55400640 1 -- 172.21.15.86:0/3549668328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf600d10c0 msgr2=0x7fcf600b83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.072+0000 7fcf55400640 1 --2- 172.21.15.86:0/3549668328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf600d10c0 0x7fcf600b83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.072+0000 7fcf55400640 1 -- 172.21.15.86:0/3549668328 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf60146b10 con 0x7fcf600d14d0 2024-04-25T14:06:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.072+0000 7fcf55400640 1 --2- 172.21.15.86:0/3549668328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf600d14d0 0x7fcf600b8930 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800c440 tx=0x7fcf4800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.072+0000 7fcf56a00640 1 -- 172.21.15.86:0/3549668328 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48018070 con 0x7fcf600d14d0 2024-04-25T14:06:06.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.072+0000 7fcf56a00640 1 -- 172.21.15.86:0/3549668328 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf480074e0 con 0x7fcf600d14d0 2024-04-25T14:06:06.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.072+0000 7fcf56a00640 1 -- 172.21.15.86:0/3549668328 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48004a10 con 0x7fcf600d14d0 2024-04-25T14:06:06.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.072+0000 7fcf65000640 1 -- 172.21.15.86:0/3549668328 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf60146df0 con 0x7fcf600d14d0 2024-04-25T14:06:06.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.072+0000 7fcf65000640 1 -- 172.21.15.86:0/3549668328 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf601473b0 con 0x7fcf600d14d0 2024-04-25T14:06:06.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.076+0000 7fcf65000640 1 -- 172.21.15.86:0/3549668328 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf60072170 con 0x7fcf600d14d0 2024-04-25T14:06:06.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.076+0000 7fcf56a00640 1 -- 172.21.15.86:0/3549668328 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7fcf4801b070 con 0x7fcf600d14d0 2024-04-25T14:06:06.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.076+0000 7fcf56a00640 1 --2- 172.21.15.86:0/3549668328 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcf3c06f570 0x7fcf3c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:06.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.076+0000 7fcf56a00640 1 -- 172.21.15.86:0/3549668328 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7fcf4808edd0 con 0x7fcf600d14d0 2024-04-25T14:06:06.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.076+0000 7fcf5de00640 1 --2- 172.21.15.86:0/3549668328 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcf3c06f570 0x7fcf3c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:06.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.076+0000 7fcf5de00640 1 --2- 172.21.15.86:0/3549668328 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcf3c06f570 0x7fcf3c071a80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcf500ae000 tx=0x7fcf500a7000 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:06.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.080+0000 7fcf56a00640 1 -- 172.21.15.86:0/3549668328 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcf4805ae30 con 0x7fcf600d14d0 2024-04-25T14:06:06.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.292+0000 7fcf65000640 1 -- 172.21.15.86:0/3549668328 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=fs2_data"]} v 0) -- 0x7fcf60146870 con 0x7fcf600d14d0 2024-04-25T14:06:06.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.296+0000 7fcf56a00640 1 -- 172.21.15.86:0/3549668328 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=fs2_data"]}]=0 updated caps for client.3 v23) ==== 181+0+138 (secure 0 0 0) 0x7fcf4805f130 con 0x7fcf600d14d0 2024-04-25T14:06:06.298 INFO:teuthology.orchestra.run.smithi086.stdout:[client.3] 2024-04-25T14:06:06.298 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCLYipmJsCBKRAAzXn2qDtblIpqt7U/TUUlyA== 2024-04-25T14:06:06.298 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-04-25T14:06:06.298 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-04-25T14:06:06.298 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw pool=fs2_data" 2024-04-25T14:06:06.299 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.3 2024-04-25T14:06:06.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.300+0000 7fcf65000640 1 -- 172.21.15.86:0/3549668328 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcf3c06f570 msgr2=0x7fcf3c071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:06.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.300+0000 7fcf65000640 1 --2- 172.21.15.86:0/3549668328 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcf3c06f570 0x7fcf3c071a80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcf500ae000 tx=0x7fcf500a7000 comp rx=0 tx=0).stop 2024-04-25T14:06:06.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.300+0000 7fcf65000640 1 -- 172.21.15.86:0/3549668328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf600d14d0 msgr2=0x7fcf600b8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:06.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.300+0000 7fcf65000640 1 --2- 172.21.15.86:0/3549668328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf600d14d0 0x7fcf600b8930 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800c440 tx=0x7fcf4800c900 comp rx=0 tx=0).stop 2024-04-25T14:06:06.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.300+0000 7fcf65000640 1 -- 172.21.15.86:0/3549668328 shutdown_connections 2024-04-25T14:06:06.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.300+0000 7fcf65000640 1 --2- 172.21.15.86:0/3549668328 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcf3c06f570 0x7fcf3c071a80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:06.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.300+0000 7fcf65000640 1 --2- 172.21.15.86:0/3549668328 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf6015fe90 0x7fcf6015c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:06.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.300+0000 7fcf65000640 1 --2- 172.21.15.86:0/3549668328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf600d14d0 0x7fcf600b8930 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:06.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.300+0000 7fcf65000640 1 --2- 172.21.15.86:0/3549668328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf600d10c0 0x7fcf600b83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:06.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.300+0000 7fcf65000640 1 -- 172.21.15.86:0/3549668328 >> 172.21.15.86:0/3549668328 conn(0x7fcf600b5020 msgr2=0x7fcf60162520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:06.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.300+0000 7fcf65000640 1 -- 172.21.15.86:0/3549668328 shutdown_connections 2024-04-25T14:06:06.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.300+0000 7fcf65000640 1 -- 172.21.15.86:0/3549668328 wait complete. 2024-04-25T14:06:06.320 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T14:06:06.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.464+0000 7f558aa00640 1 Processor -- start 2024-04-25T14:06:06.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.464+0000 7f558aa00640 1 -- start start 2024-04-25T14:06:06.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.464+0000 7f558aa00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5584070330 0x7f55840d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:06.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.464+0000 7f558aa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55840ce790 0x7f558406f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:06.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.464+0000 7f558aa00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55840d2030 con 0x7f55840d1c20 2024-04-25T14:06:06.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.464+0000 7f558aa00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f558415e5c0 con 0x7f5584070330 2024-04-25T14:06:06.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.464+0000 7f558aa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f558415e8a0 con 0x7f55840ce790 2024-04-25T14:06:06.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.468+0000 7f5583e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5584070330 0x7f55840d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:06.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.468+0000 7f5588a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55840ce790 0x7f558406f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:06.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.468+0000 7f5583e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5584070330 0x7f55840d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:37406/0 (socket says 172.21.15.86:37406) 2024-04-25T14:06:06.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.468+0000 7f5583e00640 1 -- 172.21.15.86:0/558184790 learned_addr learned my addr 172.21.15.86:0/558184790 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:06.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.468+0000 7f5588a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55840ce790 0x7f558406f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:59556/0 (socket says 172.21.15.86:59556) 2024-04-25T14:06:06.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.468+0000 7f5583400640 1 --1- 172.21.15.86:0/558184790 >> v1:172.21.15.155:6789/0 conn(0x7f55840d1c20 0x7f558406ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:37734/0 (socket says 172.21.15.86:37734) 2024-04-25T14:06:06.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.468+0000 7f5588a00640 1 -- 172.21.15.86:0/558184790 >> v2:172.21.15.155:3300/0 conn(0x7f5584070330 msgr2=0x7f55840d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:06.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.468+0000 7f5588a00640 1 --2- 172.21.15.86:0/558184790 >> v2:172.21.15.155:3300/0 conn(0x7f5584070330 0x7f55840d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:06.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.468+0000 7f5588a00640 1 -- 172.21.15.86:0/558184790 >> v1:172.21.15.155:6789/0 conn(0x7f55840d1c20 legacy=0x7f558406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:06.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.468+0000 7f5582a00640 1 -- 172.21.15.86:0/558184790 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2140527922 0 0) 0x7f5570001120 con 0x7f55840d1c20 2024-04-25T14:06:06.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.468+0000 7f5588a00640 1 -- 172.21.15.86:0/558184790 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558415eb80 con 0x7f55840ce790 2024-04-25T14:06:06.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.468+0000 7f5583e00640 1 --2- 172.21.15.86:0/558184790 >> v2:172.21.15.155:3300/0 conn(0x7f5584070330 0x7f55840d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:06.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.472+0000 7f5588a00640 1 --2- 172.21.15.86:0/558184790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55840ce790 0x7f558406f440 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f557c009d90 tx=0x7f557c01c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57d4c98346a1eb34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:06.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.472+0000 7f5582a00640 1 -- 172.21.15.86:0/558184790 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c0b2070 con 0x7f55840ce790 2024-04-25T14:06:06.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.472+0000 7f5582a00640 1 -- 172.21.15.86:0/558184790 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f557c0a57a0 con 0x7f55840ce790 2024-04-25T14:06:06.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.472+0000 7f5582a00640 1 -- 172.21.15.86:0/558184790 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c0a5b70 con 0x7f55840ce790 2024-04-25T14:06:06.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.472+0000 7f558aa00640 1 -- 172.21.15.86:0/558184790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55840ce790 msgr2=0x7f558406f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:06.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.472+0000 7f558aa00640 1 --2- 172.21.15.86:0/558184790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55840ce790 0x7f558406f440 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f557c009d90 tx=0x7f557c01c8e0 comp rx=0 tx=0).stop 2024-04-25T14:06:06.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.472+0000 7f558aa00640 1 -- 172.21.15.86:0/558184790 shutdown_connections 2024-04-25T14:06:06.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.472+0000 7f558aa00640 1 --2- 172.21.15.86:0/558184790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55840ce790 0x7f558406f440 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:06.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.472+0000 7f558aa00640 1 --2- 172.21.15.86:0/558184790 >> v2:172.21.15.155:3300/0 conn(0x7f5584070330 0x7f55840d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:06.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.472+0000 7f558aa00640 1 -- 172.21.15.86:0/558184790 >> 172.21.15.86:0/558184790 conn(0x7f55840b4650 msgr2=0x7f55840b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:06.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.472+0000 7f558aa00640 1 -- 172.21.15.86:0/558184790 shutdown_connections 2024-04-25T14:06:06.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.472+0000 7f558aa00640 1 -- 172.21.15.86:0/558184790 wait complete. 2024-04-25T14:06:06.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.472+0000 7f558aa00640 1 Processor -- start 2024-04-25T14:06:06.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.472+0000 7f558aa00640 1 -- start start 2024-04-25T14:06:06.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.476+0000 7f558aa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5584070330 0x7f558415a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:06.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.476+0000 7f558aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55840ce790 0x7f55841fe0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:06.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.476+0000 7f558aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55840d1c20 0x7f55840ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:06.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.476+0000 7f558aa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55840708c0 con 0x7f55840d1c20 2024-04-25T14:06:06.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.476+0000 7f558aa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5584070740 con 0x7f55840ce790 2024-04-25T14:06:06.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.476+0000 7f558aa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f558416afc0 con 0x7f5584070330 2024-04-25T14:06:06.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.476+0000 7f5588a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55840d1c20 0x7f55840ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:06.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.476+0000 7f5588a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55840d1c20 0x7f55840ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41636/0 (socket says 172.21.15.86:41636) 2024-04-25T14:06:06.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.476+0000 7f5588a00640 1 -- 172.21.15.86:0/3347250610 learned_addr learned my addr 172.21.15.86:0/3347250610 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:06.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.476+0000 7f5583400640 1 --2- 172.21.15.86:0/3347250610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55840ce790 0x7f55841fe0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:06.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.476+0000 7f5583e00640 1 --2- 172.21.15.86:0/3347250610 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5584070330 0x7f558415a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:06.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.476+0000 7f5588a00640 1 -- 172.21.15.86:0/3347250610 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5584070330 msgr2=0x7f558415a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:06.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.476+0000 7f5588a00640 1 --2- 172.21.15.86:0/3347250610 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5584070330 0x7f558415a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:06.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.476+0000 7f5588a00640 1 -- 172.21.15.86:0/3347250610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55840ce790 msgr2=0x7f55841fe0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:06.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.476+0000 7f5588a00640 1 --2- 172.21.15.86:0/3347250610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55840ce790 0x7f55841fe0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:06.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.476+0000 7f5588a00640 1 -- 172.21.15.86:0/3347250610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55840c4960 con 0x7f55840d1c20 2024-04-25T14:06:06.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.476+0000 7f5583400640 1 --2- 172.21.15.86:0/3347250610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55840ce790 0x7f55841fe0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:06.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.476+0000 7f5588a00640 1 --2- 172.21.15.86:0/3347250610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55840d1c20 0x7f55840ceeb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f557c01af90 tx=0x7f557c019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:06.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.476+0000 7f557be00640 1 -- 172.21.15.86:0/3347250610 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c0b2050 con 0x7f55840d1c20 2024-04-25T14:06:06.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.476+0000 7f557be00640 1 -- 172.21.15.86:0/3347250610 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f557c0be070 con 0x7f55840d1c20 2024-04-25T14:06:06.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.476+0000 7f558aa00640 1 -- 172.21.15.86:0/3347250610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55840c4c40 con 0x7f55840d1c20 2024-04-25T14:06:06.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.476+0000 7f557be00640 1 -- 172.21.15.86:0/3347250610 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c0ad850 con 0x7f55840d1c20 2024-04-25T14:06:06.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.476+0000 7f558aa00640 1 -- 172.21.15.86:0/3347250610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55841fd8f0 con 0x7f55840d1c20 2024-04-25T14:06:06.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.476+0000 7f558aa00640 1 -- 172.21.15.86:0/3347250610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5584069d00 con 0x7f55840d1c20 2024-04-25T14:06:06.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.480+0000 7f557be00640 1 -- 172.21.15.86:0/3347250610 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f557c0ad9f0 con 0x7f55840d1c20 2024-04-25T14:06:06.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.480+0000 7f557be00640 1 --2- 172.21.15.86:0/3347250610 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f556406f570 0x7f5564071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:06.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.480+0000 7f557be00640 1 -- 172.21.15.86:0/3347250610 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7f557c12ab70 con 0x7f55840d1c20 2024-04-25T14:06:06.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.480+0000 7f5583e00640 1 --2- 172.21.15.86:0/3347250610 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f556406f570 0x7f5564071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:06.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.480+0000 7f5583e00640 1 --2- 172.21.15.86:0/3347250610 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f556406f570 0x7f5564071a80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5574006cd0 tx=0x7f5574009240 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:06.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.484+0000 7f557be00640 1 -- 172.21.15.86:0/3347250610 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f557c016560 con 0x7f55840d1c20 2024-04-25T14:06:06.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.676+0000 7f558aa00640 1 -- 172.21.15.86:0/3347250610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f55840aec40 con 0x7f55840d1c20 2024-04-25T14:06:06.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.676+0000 7f557be00640 1 -- 172.21.15.86:0/3347250610 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) ==== 53+0+128 (secure 0 0 0) 0x7f557c0f7190 con 0x7f55840d1c20 2024-04-25T14:06:06.678 INFO:teuthology.orchestra.run.smithi086.stdout:name: fs1, metadata pool: fs1_metadata, data pools: [fs1_data ] 2024-04-25T14:06:06.678 INFO:teuthology.orchestra.run.smithi086.stdout:name: fs2, metadata pool: fs2_metadata, data pools: [fs2_data ] 2024-04-25T14:06:06.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.676+0000 7f558aa00640 1 -- 172.21.15.86:0/3347250610 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f556406f570 msgr2=0x7f5564071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:06.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.676+0000 7f558aa00640 1 --2- 172.21.15.86:0/3347250610 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f556406f570 0x7f5564071a80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5574006cd0 tx=0x7f5574009240 comp rx=0 tx=0).stop 2024-04-25T14:06:06.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.676+0000 7f558aa00640 1 -- 172.21.15.86:0/3347250610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55840d1c20 msgr2=0x7f55840ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:06.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.676+0000 7f558aa00640 1 --2- 172.21.15.86:0/3347250610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55840d1c20 0x7f55840ceeb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f557c01af90 tx=0x7f557c019680 comp rx=0 tx=0).stop 2024-04-25T14:06:06.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.676+0000 7f558aa00640 1 -- 172.21.15.86:0/3347250610 shutdown_connections 2024-04-25T14:06:06.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.676+0000 7f558aa00640 1 --2- 172.21.15.86:0/3347250610 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f556406f570 0x7f5564071a80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:06.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.676+0000 7f558aa00640 1 --2- 172.21.15.86:0/3347250610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55840d1c20 0x7f55840ceeb0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:06.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.676+0000 7f558aa00640 1 --2- 172.21.15.86:0/3347250610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55840ce790 0x7f55841fe0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:06.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.676+0000 7f558aa00640 1 --2- 172.21.15.86:0/3347250610 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5584070330 0x7f558415a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:06.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.680+0000 7f558aa00640 1 -- 172.21.15.86:0/3347250610 >> 172.21.15.86:0/3347250610 conn(0x7f55840b4650 msgr2=0x7f5584167b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:06.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.680+0000 7f558aa00640 1 -- 172.21.15.86:0/3347250610 shutdown_connections 2024-04-25T14:06:06.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:06.680+0000 7f558aa00640 1 -- 172.21.15.86:0/3347250610 wait complete. 2024-04-25T14:06:06.700 INFO:tasks.cephfs.mount:Mounting Ceph FS fs2; just confirmed its presence on cluster 2024-04-25T14:06:06.700 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T14:06:06.700 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2024-04-25T14:06:06.700 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T14:06:06.700 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T14:06:06.700 INFO:tasks.cephfs.mount:self.cephfs_name = fs2 2024-04-25T14:06:06.700 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T14:06:06.700 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T14:06:06.700 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:06:06.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:06.701 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2024-04-25T14:06:06.711 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:06:06.711 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:06:06.711 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:06:06.711 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:06:06.712 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2024-04-25T14:06:06.712 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:06:06.712 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:06:06.712 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2024-04-25T14:06:06.712 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f0 2024-04-25T14:06:06.712 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:06:06.712 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2024-04-25T14:06:06.712 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f1 2024-04-25T14:06:06.712 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:06:06.712 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2024-04-25T14:06:06.712 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T14:06:06.712 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2024-04-25T14:06:06.712 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T14:06:06.712 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 42291sec preferred_lft 42291sec 2024-04-25T14:06:06.713 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::ec4:7aff:fe88:6ded/64 scope link 2024-04-25T14:06:06.713 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:06:06.713 INFO:teuthology.orchestra.run.smithi155.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T14:06:06.713 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 02:42:b1:ff:60:50 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:06:06.713 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T14:06:06.713 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:06:06.713 INFO:teuthology.orchestra.run.smithi155.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T14:06:06.713 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T14:06:06.713 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T14:06:06.713 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:06:06.713 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T14:06:06.713 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:06:06.713 INFO:teuthology.orchestra.run.smithi155.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:06:06.713 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T14:06:06.714 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T14:06:06.714 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:06:06.714 INFO:teuthology.orchestra.run.smithi155.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:06:06.714 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T14:06:06.714 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T14:06:06.714 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:06:06.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:06.715 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T14:06:06.778 INFO:teuthology.orchestra.run.smithi155.stdout:1 2024-04-25T14:06:06.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:06.781 DEBUG:teuthology.orchestra.run.smithi155:> route 2024-04-25T14:06:06.836 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2024-04-25T14:06:06.836 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T14:06:06.836 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T14:06:06.836 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T14:06:06.836 INFO:teuthology.orchestra.run.smithi155.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T14:06:06.837 INFO:teuthology.orchestra.run.smithi155.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T14:06:06.837 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T14:06:06.837 INFO:teuthology.orchestra.run.smithi155.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:06:06.837 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:06:06.837 INFO:teuthology.orchestra.run.smithi155.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:06:06.837 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T14:06:06.837 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:06:06.837 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-04-25T14:06:06.837 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T14:06:06.837 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T14:06:06.837 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T14:06:06.838 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-04-25T14:06:06.931 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:06:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:06:06.974 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:06:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:06:06.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:06.981 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2024-04-25T14:06:06.991 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T14:06:06.991 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T14:06:06.992 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:06:06.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:06.992 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.0 up 2024-04-25T14:06:07.059 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:06:07.059 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T14:06:07.111 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:06:07.111 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:06:07.111 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T14:06:07.166 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T14:06:07.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:07.167 DEBUG:teuthology.orchestra.run.smithi155:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=fs2 2024-04-25T14:06:07.271 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=fs2 2024-04-25T14:06:07.271 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=fs2". 2024-04-25T14:06:07.271 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:06:07.290 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=fs2 2024-04-25T14:06:07.290 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=fs2". 2024-04-25T14:06:07.290 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:06:07.290 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:06:07.290 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:06:07.302 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=fs2 2024-04-25T14:06:07.302 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=fs2". 2024-04-25T14:06:07.302 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:06:07.302 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:06:07.302 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:06:07.302 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790" 2024-04-25T14:06:07.302 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790:/ 2024-04-25T14:06:07.302 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=fs2,key=1,fsid=8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb" will pass to kernel 2024-04-25T14:06:07.307 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T14:06:07.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:07.307 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T14:06:07.327 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:06:07.423 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:06:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:06:07.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:06:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:06:07.439 INFO:teuthology.orchestra.run.smithi155.stdout:client6061 2024-04-25T14:06:07.440 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:06:07.440 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:06:07.491 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:06:07.492 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:06:07.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:07.543 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest/mnt.1 && exec dd if=/dev/urandom of=test.bin bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:06:07.625 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-04-25T14:06:07.625 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-04-25T14:06:07.625 INFO:teuthology.orchestra.run.smithi155.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233089 s, 45.0 MB/s 2024-04-25T14:06:07.626 DEBUG:teuthology.orchestra.run.smithi155:> stdin-killer -- python3 -c ' 2024-04-25T14:06:07.627 DEBUG:teuthology.orchestra.run.smithi155:> import os 2024-04-25T14:06:07.627 DEBUG:teuthology.orchestra.run.smithi155:> import stat 2024-04-25T14:06:07.627 DEBUG:teuthology.orchestra.run.smithi155:> 2024-04-25T14:06:07.627 DEBUG:teuthology.orchestra.run.smithi155:> print(os.stat("/home/ubuntu/cephtest/mnt.1/test.bin").st_ino) 2024-04-25T14:06:07.627 DEBUG:teuthology.orchestra.run.smithi155:> ' 2024-04-25T14:06:07.730 INFO:teuthology.orchestra.run.smithi155.stdout:1099511627776 2024-04-25T14:06:07.733 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:06:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:06:07.739 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:06:07.739 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:06:07.795 INFO:tasks.cephfs.mount:Creating file a 2024-04-25T14:06:07.795 DEBUG:teuthology.orchestra.run.smithi155:> touch /home/ubuntu/cephtest/mnt.1/a 2024-04-25T14:06:07.852 INFO:tasks.cephfs.mount:Creating file b 2024-04-25T14:06:07.852 DEBUG:teuthology.orchestra.run.smithi155:> touch /home/ubuntu/cephtest/mnt.1/b 2024-04-25T14:06:07.903 INFO:tasks.cephfs.mount:Creating file c 2024-04-25T14:06:07.903 DEBUG:teuthology.orchestra.run.smithi155:> touch /home/ubuntu/cephtest/mnt.1/c 2024-04-25T14:06:07.956 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:06:08.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.112+0000 7fabed400640 1 Processor -- start 2024-04-25T14:06:08.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.112+0000 7fabed400640 1 -- start start 2024-04-25T14:06:08.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.116+0000 7fabed400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fabe80cfca0 0x7fabe806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:08.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.116+0000 7fabed400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fabe80d16e0 0x7fabe80d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:08.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.116+0000 7fabed400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fabe806c670 0x7fabe806ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:08.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.116+0000 7fabed400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fabe8070470 con 0x7fabe80cfca0 2024-04-25T14:06:08.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.116+0000 7fabed400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fabe80d2030 con 0x7fabe80d16e0 2024-04-25T14:06:08.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.116+0000 7fabed400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fabe8072bd0 con 0x7fabe806c670 2024-04-25T14:06:08.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.116+0000 7fabe6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fabe80cfca0 0x7fabe806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:08.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.116+0000 7fabe6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fabe80cfca0 0x7fabe806f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41638/0 (socket says 172.21.15.86:41638) 2024-04-25T14:06:08.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.116+0000 7fabe6800640 1 -- 172.21.15.86:0/339298206 learned_addr learned my addr 172.21.15.86:0/339298206 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:08.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.116+0000 7fabdde00640 1 --2- 172.21.15.86:0/339298206 >> v2:172.21.15.155:3300/0 conn(0x7fabe80d16e0 0x7fabe80d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:08.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.116+0000 7fabe7200640 1 --2- 172.21.15.86:0/339298206 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fabe806c670 0x7fabe806ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:08.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.116+0000 7fabe6800640 1 -- 172.21.15.86:0/339298206 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fabe806c670 msgr2=0x7fabe806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:08.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.116+0000 7fabe6800640 1 --2- 172.21.15.86:0/339298206 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fabe806c670 0x7fabe806ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:08.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.116+0000 7fabe6800640 1 -- 172.21.15.86:0/339298206 >> v2:172.21.15.155:3300/0 conn(0x7fabe80d16e0 msgr2=0x7fabe80d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:08.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.116+0000 7fabe6800640 1 --2- 172.21.15.86:0/339298206 >> v2:172.21.15.155:3300/0 conn(0x7fabe80d16e0 0x7fabe80d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:08.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.116+0000 7fabe6800640 1 -- 172.21.15.86:0/339298206 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe806f930 con 0x7fabe80cfca0 2024-04-25T14:06:08.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.116+0000 7fabdde00640 1 --2- 172.21.15.86:0/339298206 >> v2:172.21.15.155:3300/0 conn(0x7fabe80d16e0 0x7fabe80d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:08.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.120+0000 7fabe6800640 1 --2- 172.21.15.86:0/339298206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabe80cfca0 0x7fabe806f3f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fabe80729c0 tx=0x7fabd801ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e86d26fc8a843b09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:08.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.120+0000 7fabe5e00640 1 -- 172.21.15.86:0/339298206 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd800b2d0 con 0x7fabe80cfca0 2024-04-25T14:06:08.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.120+0000 7fabe5e00640 1 -- 172.21.15.86:0/339298206 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabd800b470 con 0x7fabe80cfca0 2024-04-25T14:06:08.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.120+0000 7fabe5e00640 1 -- 172.21.15.86:0/339298206 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd80a7ab0 con 0x7fabe80cfca0 2024-04-25T14:06:08.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.120+0000 7fabed400640 1 -- 172.21.15.86:0/339298206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabe80cfca0 msgr2=0x7fabe806f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:08.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.120+0000 7fabed400640 1 --2- 172.21.15.86:0/339298206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabe80cfca0 0x7fabe806f3f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fabe80729c0 tx=0x7fabd801ebf0 comp rx=0 tx=0).stop 2024-04-25T14:06:08.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.120+0000 7fabed400640 1 -- 172.21.15.86:0/339298206 shutdown_connections 2024-04-25T14:06:08.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.120+0000 7fabed400640 1 --2- 172.21.15.86:0/339298206 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fabe806c670 0x7fabe806ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:08.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.120+0000 7fabed400640 1 --2- 172.21.15.86:0/339298206 >> v2:172.21.15.155:3300/0 conn(0x7fabe80d16e0 0x7fabe80d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:08.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.120+0000 7fabed400640 1 --2- 172.21.15.86:0/339298206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabe80cfca0 0x7fabe806f3f0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:08.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.120+0000 7fabed400640 1 -- 172.21.15.86:0/339298206 >> 172.21.15.86:0/339298206 conn(0x7fabe80b4650 msgr2=0x7fabe80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:08.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.120+0000 7fabed400640 1 -- 172.21.15.86:0/339298206 shutdown_connections 2024-04-25T14:06:08.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.120+0000 7fabed400640 1 -- 172.21.15.86:0/339298206 wait complete. 2024-04-25T14:06:08.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.124+0000 7fabed400640 1 Processor -- start 2024-04-25T14:06:08.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.124+0000 7fabed400640 1 -- start start 2024-04-25T14:06:08.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.124+0000 7fabed400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabe806c670 0x7fabe809a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:08.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.124+0000 7fabed400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabe80cfca0 0x7fabe80ca690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:08.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.124+0000 7fabed400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fabe80d16e0 0x7fabe815de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:08.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.124+0000 7fabed400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fabe80ce570 con 0x7fabe80cfca0 2024-04-25T14:06:08.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.124+0000 7fabed400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fabe806d3d0 con 0x7fabe806c670 2024-04-25T14:06:08.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.124+0000 7fabed400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fabe80ce6f0 con 0x7fabe80d16e0 2024-04-25T14:06:08.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.124+0000 7fabdde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabe80cfca0 0x7fabe80ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:08.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.124+0000 7fabdde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabe80cfca0 0x7fabe80ca690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41646/0 (socket says 172.21.15.86:41646) 2024-04-25T14:06:08.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.124+0000 7fabdde00640 1 -- 172.21.15.86:0/1522079576 learned_addr learned my addr 172.21.15.86:0/1522079576 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:08.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.124+0000 7fabe6800640 1 --2- 172.21.15.86:0/1522079576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabe806c670 0x7fabe809a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:08.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.124+0000 7fabe7200640 1 --2- 172.21.15.86:0/1522079576 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fabe80d16e0 0x7fabe815de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:08.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.124+0000 7fabdde00640 1 -- 172.21.15.86:0/1522079576 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fabe80d16e0 msgr2=0x7fabe815de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:08.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.124+0000 7fabdde00640 1 --2- 172.21.15.86:0/1522079576 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fabe80d16e0 0x7fabe815de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:08.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.124+0000 7fabdde00640 1 -- 172.21.15.86:0/1522079576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabe806c670 msgr2=0x7fabe809a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:08.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.124+0000 7fabdde00640 1 --2- 172.21.15.86:0/1522079576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabe806c670 0x7fabe809a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:08.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.124+0000 7fabdde00640 1 -- 172.21.15.86:0/1522079576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe80cafe0 con 0x7fabe80cfca0 2024-04-25T14:06:08.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.124+0000 7fabdde00640 1 --2- 172.21.15.86:0/1522079576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabe80cfca0 0x7fabe80ca690 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fabd000d830 tx=0x7fabd000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:08.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.124+0000 7fabdf400640 1 -- 172.21.15.86:0/1522079576 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd000b6c0 con 0x7fabe80cfca0 2024-04-25T14:06:08.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.124+0000 7fabdf400640 1 -- 172.21.15.86:0/1522079576 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabd0019070 con 0x7fabe80cfca0 2024-04-25T14:06:08.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.124+0000 7fabdf400640 1 -- 172.21.15.86:0/1522079576 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd000ba80 con 0x7fabe80cfca0 2024-04-25T14:06:08.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.124+0000 7fabed400640 1 -- 172.21.15.86:0/1522079576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe815d340 con 0x7fabe80cfca0 2024-04-25T14:06:08.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.124+0000 7fabed400640 1 -- 172.21.15.86:0/1522079576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe815d900 con 0x7fabe80cfca0 2024-04-25T14:06:08.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.128+0000 7fabed400640 1 -- 172.21.15.86:0/1522079576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabe809f4b0 con 0x7fabe80cfca0 2024-04-25T14:06:08.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.128+0000 7fabdf400640 1 -- 172.21.15.86:0/1522079576 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7fabd001e460 con 0x7fabe80cfca0 2024-04-25T14:06:08.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.128+0000 7fabdf400640 1 --2- 172.21.15.86:0/1522079576 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fabc406f5a0 0x7fabc4071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:08.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.128+0000 7fabdf400640 1 -- 172.21.15.86:0/1522079576 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7fabd008ff00 con 0x7fabe80cfca0 2024-04-25T14:06:08.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.128+0000 7fabe6800640 1 --2- 172.21.15.86:0/1522079576 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fabc406f5a0 0x7fabc4071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.128+0000 7fabe6800640 1 --2- 172.21.15.86:0/1522079576 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fabc406f5a0 0x7fabc4071ab0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fabd80b0000 tx=0x7fabd80a9070 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:08.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.132+0000 7fabdf400640 1 -- 172.21.15.86:0/1522079576 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fabd005c090 con 0x7fabe80cfca0 2024-04-25T14:06:08.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.312+0000 7fabed400640 1 -- 172.21.15.86:0/1522079576 --> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7fabe8069d00 con 0x7fabc406f5a0 2024-04-25T14:06:08.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.316+0000 7fabdf400640 1 -- 172.21.15.86:0/1522079576 <== mgr.5532 v2:172.21.15.86:6832/2427179681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fabe8069d00 con 0x7fabc406f5a0 2024-04-25T14:06:08.317 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {}, "global_metrics": {}, "metrics": {"delayed_ranks": []}} 2024-04-25T14:06:08.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.316+0000 7fabed400640 1 -- 172.21.15.86:0/1522079576 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fabc406f5a0 msgr2=0x7fabc4071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:08.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.316+0000 7fabed400640 1 --2- 172.21.15.86:0/1522079576 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fabc406f5a0 0x7fabc4071ab0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fabd80b0000 tx=0x7fabd80a9070 comp rx=0 tx=0).stop 2024-04-25T14:06:08.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.316+0000 7fabed400640 1 -- 172.21.15.86:0/1522079576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabe80cfca0 msgr2=0x7fabe80ca690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:08.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.316+0000 7fabed400640 1 --2- 172.21.15.86:0/1522079576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabe80cfca0 0x7fabe80ca690 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fabd000d830 tx=0x7fabd000dcf0 comp rx=0 tx=0).stop 2024-04-25T14:06:08.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.316+0000 7fabed400640 1 -- 172.21.15.86:0/1522079576 shutdown_connections 2024-04-25T14:06:08.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.316+0000 7fabed400640 1 --2- 172.21.15.86:0/1522079576 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fabc406f5a0 0x7fabc4071ab0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:08.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.316+0000 7fabed400640 1 --2- 172.21.15.86:0/1522079576 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fabe80d16e0 0x7fabe815de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:08.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.316+0000 7fabed400640 1 --2- 172.21.15.86:0/1522079576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabe80cfca0 0x7fabe80ca690 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:08.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.316+0000 7fabed400640 1 --2- 172.21.15.86:0/1522079576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabe806c670 0x7fabe809a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:08.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.316+0000 7fabed400640 1 -- 172.21.15.86:0/1522079576 >> 172.21.15.86:0/1522079576 conn(0x7fabe80b4650 msgr2=0x7fabe815a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:08.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.316+0000 7fabed400640 1 -- 172.21.15.86:0/1522079576 shutdown_connections 2024-04-25T14:06:08.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:08.316+0000 7fabed400640 1 -- 172.21.15.86:0/1522079576 wait complete. 2024-04-25T14:06:09.339 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:06:09.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.496+0000 7f3cd6200640 1 Processor -- start 2024-04-25T14:06:09.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.496+0000 7f3cd6200640 1 -- start start 2024-04-25T14:06:09.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.500+0000 7f3cd6200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd015c2a0 0x7f3cd00ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:09.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.500+0000 7f3cd6200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3cd006fd50 0x7f3cd0070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:09.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.500+0000 7f3cd6200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cd00b0a60 con 0x7f3cd006c670 2024-04-25T14:06:09.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.500+0000 7f3cd6200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3cd006ef90 con 0x7f3cd006fd50 2024-04-25T14:06:09.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.500+0000 7f3cd6200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3cd006f270 con 0x7f3cd015c2a0 2024-04-25T14:06:09.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.500+0000 7f3ccf400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd015c2a0 0x7f3cd00ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:09.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.500+0000 7f3cc6a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3cd006fd50 0x7f3cd0070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:09.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.500+0000 7f3ccf400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd015c2a0 0x7f3cd00ae130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:46034/0 (socket says 172.21.15.86:46034) 2024-04-25T14:06:09.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.500+0000 7f3cc6a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3cd006fd50 0x7f3cd0070160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:52358/0 (socket says 172.21.15.86:52358) 2024-04-25T14:06:09.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.500+0000 7f3ccf400640 1 -- 172.21.15.86:0/1909401448 learned_addr learned my addr 172.21.15.86:0/1909401448 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:09.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.500+0000 7f3ccfe00640 1 --1- 172.21.15.86:0/1909401448 >> v1:172.21.15.155:6789/0 conn(0x7f3cd006c670 0x7f3cd006ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:43276/0 (socket says 172.21.15.86:43276) 2024-04-25T14:06:09.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.500+0000 7f3cc6a00640 1 -- 172.21.15.86:0/1909401448 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd015c2a0 msgr2=0x7f3cd00ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:09.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.500+0000 7f3cc6a00640 1 --2- 172.21.15.86:0/1909401448 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd015c2a0 0x7f3cd00ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:09.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.500+0000 7f3cc6a00640 1 -- 172.21.15.86:0/1909401448 >> v1:172.21.15.155:6789/0 conn(0x7f3cd006c670 legacy=0x7f3cd006ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:09.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.500+0000 7f3cc6a00640 1 -- 172.21.15.86:0/1909401448 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd006f550 con 0x7f3cd006fd50 2024-04-25T14:06:09.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.500+0000 7f3ccea00640 1 -- 172.21.15.86:0/1909401448 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2810003478 0 0) 0x7f3cd00b0a60 con 0x7f3cd006c670 2024-04-25T14:06:09.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.500+0000 7f3ccf400640 1 --2- 172.21.15.86:0/1909401448 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd015c2a0 0x7f3cd00ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:09.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.504+0000 7f3cc6a00640 1 --2- 172.21.15.86:0/1909401448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cd006fd50 0x7f3cd0070160 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80097c0 tx=0x7f3cb801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2415ed5826cb4bc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:09.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.504+0000 7f3ccea00640 1 -- 172.21.15.86:0/1909401448 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb80b0070 con 0x7f3cd006fd50 2024-04-25T14:06:09.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.504+0000 7f3ccea00640 1 -- 172.21.15.86:0/1909401448 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cb80ab600 con 0x7f3cd006fd50 2024-04-25T14:06:09.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.504+0000 7f3ccea00640 1 -- 172.21.15.86:0/1909401448 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb80ab9d0 con 0x7f3cd006fd50 2024-04-25T14:06:09.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.504+0000 7f3cd6200640 1 -- 172.21.15.86:0/1909401448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cd006fd50 msgr2=0x7f3cd0070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:09.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.504+0000 7f3cd6200640 1 --2- 172.21.15.86:0/1909401448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cd006fd50 0x7f3cd0070160 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80097c0 tx=0x7f3cb801c3e0 comp rx=0 tx=0).stop 2024-04-25T14:06:09.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.504+0000 7f3cd6200640 1 -- 172.21.15.86:0/1909401448 shutdown_connections 2024-04-25T14:06:09.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.504+0000 7f3cd6200640 1 --2- 172.21.15.86:0/1909401448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cd006fd50 0x7f3cd0070160 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:09.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.504+0000 7f3cd6200640 1 --2- 172.21.15.86:0/1909401448 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd015c2a0 0x7f3cd00ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:09.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.504+0000 7f3cd6200640 1 -- 172.21.15.86:0/1909401448 >> 172.21.15.86:0/1909401448 conn(0x7f3cd00b4650 msgr2=0x7f3cd00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:09.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.504+0000 7f3cd6200640 1 -- 172.21.15.86:0/1909401448 shutdown_connections 2024-04-25T14:06:09.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.504+0000 7f3cd6200640 1 -- 172.21.15.86:0/1909401448 wait complete. 2024-04-25T14:06:09.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.508+0000 7f3cd6200640 1 Processor -- start 2024-04-25T14:06:09.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.508+0000 7f3cd6200640 1 -- start start 2024-04-25T14:06:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.508+0000 7f3cd6200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd006c670 0x7f3cd01fda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.508+0000 7f3cd6200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cd006fd50 0x7f3cd00c0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.508+0000 7f3cd6200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cd015c2a0 0x7f3cd0073fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.508+0000 7f3cd6200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd015c830 con 0x7f3cd006fd50 2024-04-25T14:06:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.508+0000 7f3cd6200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd015c6b0 con 0x7f3cd015c2a0 2024-04-25T14:06:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.508+0000 7f3cd6200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3cd00ae670 con 0x7f3cd006c670 2024-04-25T14:06:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.508+0000 7f3cc6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cd006fd50 0x7f3cd00c0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.508+0000 7f3cc6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cd006fd50 0x7f3cd00c0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47324/0 (socket says 172.21.15.86:47324) 2024-04-25T14:06:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.508+0000 7f3cc6a00640 1 -- 172.21.15.86:0/3350284228 learned_addr learned my addr 172.21.15.86:0/3350284228 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.508+0000 7f3ccfe00640 1 --2- 172.21.15.86:0/3350284228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cd015c2a0 0x7f3cd0073fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.508+0000 7f3ccf400640 1 --2- 172.21.15.86:0/3350284228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd006c670 0x7f3cd01fda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.508+0000 7f3cc6a00640 1 -- 172.21.15.86:0/3350284228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd006c670 msgr2=0x7f3cd01fda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:09.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.508+0000 7f3cc6a00640 1 --2- 172.21.15.86:0/3350284228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd006c670 0x7f3cd01fda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:09.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.508+0000 7f3cc6a00640 1 -- 172.21.15.86:0/3350284228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cd015c2a0 msgr2=0x7f3cd0073fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:09.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.508+0000 7f3cc6a00640 1 --2- 172.21.15.86:0/3350284228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cd015c2a0 0x7f3cd0073fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:09.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.508+0000 7f3cc6a00640 1 -- 172.21.15.86:0/3350284228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd0164130 con 0x7f3cd006fd50 2024-04-25T14:06:09.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.508+0000 7f3ccf400640 1 --2- 172.21.15.86:0/3350284228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd006c670 0x7f3cd01fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:09.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.508+0000 7f3ccfe00640 1 --2- 172.21.15.86:0/3350284228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cd015c2a0 0x7f3cd0073fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:09.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.508+0000 7f3cc6a00640 1 --2- 172.21.15.86:0/3350284228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cd006fd50 0x7f3cd00c0910 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80ae040 tx=0x7f3cb801ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:09.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.508+0000 7f3cc7e00640 1 -- 172.21.15.86:0/3350284228 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb80bb070 con 0x7f3cd006fd50 2024-04-25T14:06:09.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.508+0000 7f3cc7e00640 1 -- 172.21.15.86:0/3350284228 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cb8004020 con 0x7f3cd006fd50 2024-04-25T14:06:09.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.512+0000 7f3cd6200640 1 -- 172.21.15.86:0/3350284228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd0164410 con 0x7f3cd006fd50 2024-04-25T14:06:09.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.512+0000 7f3cc7e00640 1 -- 172.21.15.86:0/3350284228 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb80b0040 con 0x7f3cd006fd50 2024-04-25T14:06:09.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.512+0000 7f3cd6200640 1 -- 172.21.15.86:0/3350284228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd0166130 con 0x7f3cd006fd50 2024-04-25T14:06:09.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.512+0000 7f3cd6200640 1 -- 172.21.15.86:0/3350284228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cd0069d00 con 0x7f3cd006fd50 2024-04-25T14:06:09.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.512+0000 7f3cc7e00640 1 -- 172.21.15.86:0/3350284228 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f3cb8003620 con 0x7f3cd006fd50 2024-04-25T14:06:09.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.512+0000 7f3cc7e00640 1 --2- 172.21.15.86:0/3350284228 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f3cac06f570 0x7f3cac071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:09.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.512+0000 7f3cc7e00640 1 -- 172.21.15.86:0/3350284228 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7f3cb8129ff0 con 0x7f3cd006fd50 2024-04-25T14:06:09.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.512+0000 7f3ccf400640 1 --2- 172.21.15.86:0/3350284228 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f3cac06f570 0x7f3cac071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:09.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.512+0000 7f3ccf400640 1 --2- 172.21.15.86:0/3350284228 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f3cac06f570 0x7f3cac071a80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00b4200 tx=0x7f3cc000b450 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:09.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.516+0000 7f3cc7e00640 1 -- 172.21.15.86:0/3350284228 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3cb80c5020 con 0x7f3cd006fd50 2024-04-25T14:06:09.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.696+0000 7f3cd6200640 1 -- 172.21.15.86:0/3350284228 --> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f3cd0076bf0 con 0x7f3cac06f570 2024-04-25T14:06:09.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.696+0000 7f3cc7e00640 1 -- 172.21.15.86:0/3350284228 <== mgr.5532 v2:172.21.15.86:6832/2427179681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f3cd0076bf0 con 0x7f3cac06f570 2024-04-25T14:06:09.697 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {}, "global_metrics": {}, "metrics": {"delayed_ranks": []}} 2024-04-25T14:06:09.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.696+0000 7f3cd6200640 1 -- 172.21.15.86:0/3350284228 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f3cac06f570 msgr2=0x7f3cac071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:09.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.696+0000 7f3cd6200640 1 --2- 172.21.15.86:0/3350284228 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f3cac06f570 0x7f3cac071a80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00b4200 tx=0x7f3cc000b450 comp rx=0 tx=0).stop 2024-04-25T14:06:09.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.696+0000 7f3cd6200640 1 -- 172.21.15.86:0/3350284228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cd006fd50 msgr2=0x7f3cd00c0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:09.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.696+0000 7f3cd6200640 1 --2- 172.21.15.86:0/3350284228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cd006fd50 0x7f3cd00c0910 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80ae040 tx=0x7f3cb801ac20 comp rx=0 tx=0).stop 2024-04-25T14:06:09.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.696+0000 7f3cd6200640 1 -- 172.21.15.86:0/3350284228 shutdown_connections 2024-04-25T14:06:09.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.696+0000 7f3cd6200640 1 --2- 172.21.15.86:0/3350284228 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f3cac06f570 0x7f3cac071a80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:09.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.696+0000 7f3cd6200640 1 --2- 172.21.15.86:0/3350284228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cd015c2a0 0x7f3cd0073fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:09.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.696+0000 7f3cd6200640 1 --2- 172.21.15.86:0/3350284228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cd006fd50 0x7f3cd00c0910 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:09.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.696+0000 7f3cd6200640 1 --2- 172.21.15.86:0/3350284228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd006c670 0x7f3cd01fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:09.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.696+0000 7f3cd6200640 1 -- 172.21.15.86:0/3350284228 >> 172.21.15.86:0/3350284228 conn(0x7f3cd00b4650 msgr2=0x7f3cd015e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:09.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.696+0000 7f3cd6200640 1 -- 172.21.15.86:0/3350284228 shutdown_connections 2024-04-25T14:06:09.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:09.696+0000 7f3cd6200640 1 -- 172.21.15.86:0/3350284228 wait complete. 2024-04-25T14:06:10.721 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:06:10.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.880+0000 7fc112600640 1 Processor -- start 2024-04-25T14:06:10.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.880+0000 7fc112600640 1 -- start start 2024-04-25T14:06:10.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.880+0000 7fc112600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc10c0d16e0 0x7fc10c0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:10.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.880+0000 7fc112600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc10c0d2030 con 0x7fc10c0730b0 2024-04-25T14:06:10.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.880+0000 7fc112600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc10c072bd0 con 0x7fc10c0cfca0 2024-04-25T14:06:10.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.880+0000 7fc112600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc10c070470 con 0x7fc10c0d16e0 2024-04-25T14:06:10.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.880+0000 7fc10be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc10c0730b0 0x7fc10c0734c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59784/0 (socket says 172.21.15.86:59784) 2024-04-25T14:06:10.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.880+0000 7fc10be00640 1 -- 172.21.15.86:0/1397483289 learned_addr learned my addr 172.21.15.86:0/1397483289 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:10.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.880+0000 7fc103e00640 1 --2- 172.21.15.86:0/1397483289 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc10c0d16e0 0x7fc10c0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:10.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.880+0000 7fc10aa00640 1 -- 172.21.15.86:0/1397483289 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3636197697 0 0) 0x7fc1040014a0 con 0x7fc10c0730b0 2024-04-25T14:06:10.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.880+0000 7fc10aa00640 1 -- 172.21.15.86:0/1397483289 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0f0003530 con 0x7fc10c0730b0 2024-04-25T14:06:10.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.880+0000 7fc10aa00640 1 -- 172.21.15.86:0/1397483289 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 730485553 0 0) 0x7fc10c072bd0 con 0x7fc10c0cfca0 2024-04-25T14:06:10.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.880+0000 7fc10aa00640 1 -- 172.21.15.86:0/1397483289 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1040014a0 con 0x7fc10c0cfca0 2024-04-25T14:06:10.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.880+0000 7fc103e00640 1 -- 172.21.15.86:0/1397483289 >> v1:172.21.15.155:6789/0 conn(0x7fc10c0cfca0 legacy=0x7fc10c06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:10.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.880+0000 7fc103e00640 1 -- 172.21.15.86:0/1397483289 >> v1:172.21.15.86:6789/0 conn(0x7fc10c0730b0 legacy=0x7fc10c0734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:10.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.880+0000 7fc10aa00640 1 -- 172.21.15.86:0/1397483289 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3139580498 0 0) 0x7fc0f0003530 con 0x7fc10c0730b0 2024-04-25T14:06:10.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.880+0000 7fc103e00640 1 -- 172.21.15.86:0/1397483289 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10c06fbd0 con 0x7fc10c0d16e0 2024-04-25T14:06:10.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.884+0000 7fc103e00640 1 --2- 172.21.15.86:0/1397483289 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc10c0d16e0 0x7fc10c0d1af0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc10c072da0 tx=0x7fc0f401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4c137689f384ae6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:10.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.884+0000 7fc10aa00640 1 -- 172.21.15.86:0/1397483289 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f40b0070 con 0x7fc10c0d16e0 2024-04-25T14:06:10.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.884+0000 7fc10aa00640 1 -- 172.21.15.86:0/1397483289 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0f40ab400 con 0x7fc10c0d16e0 2024-04-25T14:06:10.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.884+0000 7fc10aa00640 1 -- 172.21.15.86:0/1397483289 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c0743e0 con 0x7fc10c0d16e0 2024-04-25T14:06:10.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.884+0000 7fc112600640 1 -- 172.21.15.86:0/1397483289 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc10c0d16e0 msgr2=0x7fc10c0d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:10.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.884+0000 7fc112600640 1 --2- 172.21.15.86:0/1397483289 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc10c0d16e0 0x7fc10c0d1af0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc10c072da0 tx=0x7fc0f401c3e0 comp rx=0 tx=0).stop 2024-04-25T14:06:10.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.884+0000 7fc112600640 1 -- 172.21.15.86:0/1397483289 shutdown_connections 2024-04-25T14:06:10.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.884+0000 7fc112600640 1 --2- 172.21.15.86:0/1397483289 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc10c0d16e0 0x7fc10c0d1af0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:10.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.884+0000 7fc112600640 1 -- 172.21.15.86:0/1397483289 >> 172.21.15.86:0/1397483289 conn(0x7fc10c0b4650 msgr2=0x7fc10c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:10.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.884+0000 7fc112600640 1 -- 172.21.15.86:0/1397483289 shutdown_connections 2024-04-25T14:06:10.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.884+0000 7fc112600640 1 -- 172.21.15.86:0/1397483289 wait complete. 2024-04-25T14:06:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.888+0000 7fc112600640 1 Processor -- start 2024-04-25T14:06:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.888+0000 7fc112600640 1 -- start start 2024-04-25T14:06:10.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.888+0000 7fc112600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc10c0730b0 0x7fc10c15f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:10.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.888+0000 7fc112600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc10c0cfca0 0x7fc10c1572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:10.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.888+0000 7fc112600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc10c0d16e0 0x7fc10c157810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:10.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.888+0000 7fc112600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c16aa80 con 0x7fc10c0730b0 2024-04-25T14:06:10.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.888+0000 7fc112600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c16a900 con 0x7fc10c0cfca0 2024-04-25T14:06:10.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.888+0000 7fc112600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc10c16ac00 con 0x7fc10c0d16e0 2024-04-25T14:06:10.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.888+0000 7fc10b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc10c0730b0 0x7fc10c15f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:10.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.888+0000 7fc10b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc10c0730b0 0x7fc10c15f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47340/0 (socket says 172.21.15.86:47340) 2024-04-25T14:06:10.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.888+0000 7fc10b400640 1 -- 172.21.15.86:0/624845831 learned_addr learned my addr 172.21.15.86:0/624845831 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:10.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.892+0000 7fc103e00640 1 --2- 172.21.15.86:0/624845831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc10c0cfca0 0x7fc10c1572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:10.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.892+0000 7fc10be00640 1 --2- 172.21.15.86:0/624845831 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc10c0d16e0 0x7fc10c157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:10.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.892+0000 7fc10b400640 1 -- 172.21.15.86:0/624845831 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc10c0d16e0 msgr2=0x7fc10c157810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:10.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.892+0000 7fc10b400640 1 --2- 172.21.15.86:0/624845831 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc10c0d16e0 0x7fc10c157810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:10.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.892+0000 7fc10b400640 1 -- 172.21.15.86:0/624845831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc10c0cfca0 msgr2=0x7fc10c1572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:10.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.892+0000 7fc10b400640 1 --2- 172.21.15.86:0/624845831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc10c0cfca0 0x7fc10c1572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:10.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.892+0000 7fc10b400640 1 -- 172.21.15.86:0/624845831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10c1592a0 con 0x7fc10c0730b0 2024-04-25T14:06:10.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.892+0000 7fc10b400640 1 --2- 172.21.15.86:0/624845831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc10c0730b0 0x7fc10c15f870 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00c460 tx=0x7fc0fc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:10.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.892+0000 7fc103400640 1 -- 172.21.15.86:0/624845831 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc017070 con 0x7fc10c0730b0 2024-04-25T14:06:10.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.892+0000 7fc103400640 1 -- 172.21.15.86:0/624845831 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0fc0052f0 con 0x7fc10c0730b0 2024-04-25T14:06:10.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.892+0000 7fc103400640 1 -- 172.21.15.86:0/624845831 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc003840 con 0x7fc10c0730b0 2024-04-25T14:06:10.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.892+0000 7fc112600640 1 -- 172.21.15.86:0/624845831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc10c15c570 con 0x7fc10c0730b0 2024-04-25T14:06:10.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.892+0000 7fc112600640 1 -- 172.21.15.86:0/624845831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc10c15c850 con 0x7fc10c0730b0 2024-04-25T14:06:10.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.892+0000 7fc112600640 1 -- 172.21.15.86:0/624845831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc10c15c090 con 0x7fc10c0730b0 2024-04-25T14:06:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.892+0000 7fc103400640 1 -- 172.21.15.86:0/624845831 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7fc0fc01a070 con 0x7fc10c0730b0 2024-04-25T14:06:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.896+0000 7fc103400640 1 --2- 172.21.15.86:0/624845831 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fc0e806f570 0x7fc0e8071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.896+0000 7fc103400640 1 -- 172.21.15.86:0/624845831 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7fc0fc08ee30 con 0x7fc10c0730b0 2024-04-25T14:06:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.896+0000 7fc103e00640 1 --2- 172.21.15.86:0/624845831 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fc0e806f570 0x7fc0e8071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.896+0000 7fc103e00640 1 --2- 172.21.15.86:0/624845831 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fc0e806f570 0x7fc0e8071a80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40ad000 tx=0x7fc0f40a6000 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:10.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:10.900+0000 7fc103400640 1 -- 172.21.15.86:0/624845831 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc0fc05af30 con 0x7fc10c0730b0 2024-04-25T14:06:11.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:11.080+0000 7fc112600640 1 -- 172.21.15.86:0/624845831 --> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7fc10c09f4b0 con 0x7fc0e806f570 2024-04-25T14:06:11.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:11.080+0000 7fc103400640 1 -- 172.21.15.86:0/624845831 <== mgr.5532 v2:172.21.15.86:6832/2427179681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fc10c09f4b0 con 0x7fc0e806f570 2024-04-25T14:06:11.083 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {}, "global_metrics": {}, "metrics": {"delayed_ranks": []}} 2024-04-25T14:06:11.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:11.084+0000 7fc112600640 1 -- 172.21.15.86:0/624845831 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fc0e806f570 msgr2=0x7fc0e8071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:11.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:11.084+0000 7fc112600640 1 --2- 172.21.15.86:0/624845831 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fc0e806f570 0x7fc0e8071a80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40ad000 tx=0x7fc0f40a6000 comp rx=0 tx=0).stop 2024-04-25T14:06:11.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:11.084+0000 7fc112600640 1 -- 172.21.15.86:0/624845831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc10c0730b0 msgr2=0x7fc10c15f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:11.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:11.084+0000 7fc112600640 1 --2- 172.21.15.86:0/624845831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc10c0730b0 0x7fc10c15f870 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00c460 tx=0x7fc0fc00c920 comp rx=0 tx=0).stop 2024-04-25T14:06:11.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:11.084+0000 7fc112600640 1 -- 172.21.15.86:0/624845831 shutdown_connections 2024-04-25T14:06:11.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:11.084+0000 7fc112600640 1 --2- 172.21.15.86:0/624845831 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fc0e806f570 0x7fc0e8071a80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:11.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:11.084+0000 7fc112600640 1 --2- 172.21.15.86:0/624845831 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc10c0d16e0 0x7fc10c157810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:11.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:11.084+0000 7fc112600640 1 --2- 172.21.15.86:0/624845831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc10c0cfca0 0x7fc10c1572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:11.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:11.084+0000 7fc112600640 1 --2- 172.21.15.86:0/624845831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc10c0730b0 0x7fc10c15f870 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:11.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:11.084+0000 7fc112600640 1 -- 172.21.15.86:0/624845831 >> 172.21.15.86:0/624845831 conn(0x7fc10c0b4650 msgr2=0x7fc10c0b1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:11.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:11.084+0000 7fc112600640 1 -- 172.21.15.86:0/624845831 shutdown_connections 2024-04-25T14:06:11.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:11.084+0000 7fc112600640 1 -- 172.21.15.86:0/624845831 wait complete. 2024-04-25T14:06:12.106 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:06:12.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.264+0000 7f3974a00640 1 Processor -- start 2024-04-25T14:06:12.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.264+0000 7f3974a00640 1 -- start start 2024-04-25T14:06:12.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.264+0000 7f3974a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39700d1df0 0x7f39700b0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:12.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.264+0000 7f3974a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f39700b0690 0x7f39700d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:12.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.264+0000 7f3974a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39700cde20 con 0x7f39701642a0 2024-04-25T14:06:12.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.264+0000 7f3974a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f39700cea30 con 0x7f39700b0690 2024-04-25T14:06:12.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.264+0000 7f3974a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f39700ce100 con 0x7f39700d1df0 2024-04-25T14:06:12.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.264+0000 7f396de00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f39701642a0 0x7f39700d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59788/0 (socket says 172.21.15.86:59788) 2024-04-25T14:06:12.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.264+0000 7f396de00640 1 -- 172.21.15.86:0/2656710154 learned_addr learned my addr 172.21.15.86:0/2656710154 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:12.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.264+0000 7f396d400640 1 --2- 172.21.15.86:0/2656710154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39700d1df0 0x7f39700b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:12.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.264+0000 7f396e800640 1 --2- 172.21.15.86:0/2656710154 >> v2:172.21.15.155:3300/0 conn(0x7f39700b0690 0x7f39700d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:12.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.264+0000 7f396ca00640 1 -- 172.21.15.86:0/2656710154 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3085487076 0 0) 0x7f39700cde20 con 0x7f39701642a0 2024-04-25T14:06:12.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.264+0000 7f396ca00640 1 -- 172.21.15.86:0/2656710154 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3954003530 con 0x7f39701642a0 2024-04-25T14:06:12.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.264+0000 7f396ca00640 1 -- 172.21.15.86:0/2656710154 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1362856902 0 0) 0x7f3954003530 con 0x7f39701642a0 2024-04-25T14:06:12.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.264+0000 7f396ca00640 1 -- 172.21.15.86:0/2656710154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39700d1df0 msgr2=0x7f39700b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:12.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.264+0000 7f396ca00640 1 --2- 172.21.15.86:0/2656710154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39700d1df0 0x7f39700b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:12.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.264+0000 7f396ca00640 1 -- 172.21.15.86:0/2656710154 >> v2:172.21.15.155:3300/0 conn(0x7f39700b0690 msgr2=0x7f39700d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:12.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.264+0000 7f396ca00640 1 --2- 172.21.15.86:0/2656710154 >> v2:172.21.15.155:3300/0 conn(0x7f39700b0690 0x7f39700d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:12.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.264+0000 7f396ca00640 1 -- 172.21.15.86:0/2656710154 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39700ce3e0 con 0x7f39701642a0 2024-04-25T14:06:12.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.264+0000 7f396ca00640 1 -- 172.21.15.86:0/2656710154 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f3960001a00 con 0x7f39701642a0 2024-04-25T14:06:12.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.264+0000 7f396e800640 1 --2- 172.21.15.86:0/2656710154 >> v2:172.21.15.155:3300/0 conn(0x7f39700b0690 0x7f39700d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:12.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.264+0000 7f396ca00640 1 -- 172.21.15.86:0/2656710154 >> v1:172.21.15.86:6789/0 conn(0x7f39701642a0 legacy=0x7f39700d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:12.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.264+0000 7f396ca00640 1 --2- 172.21.15.86:0/2656710154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3954003a30 0x7f3954003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:12.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.268+0000 7f396ca00640 1 --2- 172.21.15.86:0/2656710154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3954004810 0x7f3954004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:12.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.268+0000 7f396ca00640 1 --2- 172.21.15.86:0/2656710154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3954005160 0x7f395400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:12.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.268+0000 7f396ca00640 1 -- 172.21.15.86:0/2656710154 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39700cea30 con 0x7f3954004810 2024-04-25T14:06:12.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.268+0000 7f396ca00640 1 -- 172.21.15.86:0/2656710154 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f39700ce100 con 0x7f3954005160 2024-04-25T14:06:12.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.268+0000 7f396ca00640 1 -- 172.21.15.86:0/2656710154 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f395400bf40 con 0x7f3954003a30 2024-04-25T14:06:12.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.268+0000 7f396d400640 1 --2- 172.21.15.86:0/2656710154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3954004810 0x7f3954004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:12.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.268+0000 7f396de00640 1 --2- 172.21.15.86:0/2656710154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3954003a30 0x7f3954003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:12.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.268+0000 7f396e800640 1 --2- 172.21.15.86:0/2656710154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3954005160 0x7f395400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:12.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.268+0000 7f396d400640 1 -- 172.21.15.86:0/2656710154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3954003a30 msgr2=0x7f3954003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:12.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.268+0000 7f396d400640 1 --2- 172.21.15.86:0/2656710154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3954003a30 0x7f3954003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:12.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.268+0000 7f396d400640 1 -- 172.21.15.86:0/2656710154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3954005160 msgr2=0x7f395400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:12.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.268+0000 7f396d400640 1 --2- 172.21.15.86:0/2656710154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3954005160 0x7f395400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:12.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.268+0000 7f396d400640 1 -- 172.21.15.86:0/2656710154 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3954003530 con 0x7f3954004810 2024-04-25T14:06:12.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.268+0000 7f396d400640 1 --2- 172.21.15.86:0/2656710154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3954004810 0x7f3954004c20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f395c009ac0 tx=0x7f395c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66fe915b09f94732 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:12.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.268+0000 7f396ca00640 1 -- 172.21.15.86:0/2656710154 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c009690 con 0x7f3954004810 2024-04-25T14:06:12.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.268+0000 7f396ca00640 1 -- 172.21.15.86:0/2656710154 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f395c0a6420 con 0x7f3954004810 2024-04-25T14:06:12.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.268+0000 7f396ca00640 1 -- 172.21.15.86:0/2656710154 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c0a67f0 con 0x7f3954004810 2024-04-25T14:06:12.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.272+0000 7f3974a00640 1 -- 172.21.15.86:0/2656710154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3954004810 msgr2=0x7f3954004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:12.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.272+0000 7f3974a00640 1 --2- 172.21.15.86:0/2656710154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3954004810 0x7f3954004c20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f395c009ac0 tx=0x7f395c01ca70 comp rx=0 tx=0).stop 2024-04-25T14:06:12.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.272+0000 7f396e800640 1 -- 172.21.15.86:0/2656710154 reap_dead start 2024-04-25T14:06:12.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.272+0000 7f3974a00640 1 -- 172.21.15.86:0/2656710154 shutdown_connections 2024-04-25T14:06:12.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.272+0000 7f3974a00640 1 -- 172.21.15.86:0/2656710154 >> 172.21.15.86:0/2656710154 conn(0x7f39700b4650 msgr2=0x7f39700b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:12.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.272+0000 7f3974a00640 1 -- 172.21.15.86:0/2656710154 shutdown_connections 2024-04-25T14:06:12.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.272+0000 7f3974a00640 1 -- 172.21.15.86:0/2656710154 wait complete. 2024-04-25T14:06:12.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.272+0000 7f3974a00640 1 Processor -- start 2024-04-25T14:06:12.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.272+0000 7f3974a00640 1 -- start start 2024-04-25T14:06:12.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.276+0000 7f3974a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3954004810 0x7f397015e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:12.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.276+0000 7f3974a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f397015d0a0 0x7f39700ca560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:12.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.276+0000 7f3974a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3970072e10 0x7f3970073220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:12.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.276+0000 7f3974a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39700d1df0 con 0x7f3970072e10 2024-04-25T14:06:12.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.276+0000 7f3974a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f39701648c0 con 0x7f3954004810 2024-04-25T14:06:12.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.276+0000 7f3974a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f39700d1f70 con 0x7f397015d0a0 2024-04-25T14:06:12.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.276+0000 7f396e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3970072e10 0x7f3970073220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:12.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.276+0000 7f396e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3970072e10 0x7f3970073220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47372/0 (socket says 172.21.15.86:47372) 2024-04-25T14:06:12.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.276+0000 7f396e800640 1 -- 172.21.15.86:0/2299026319 learned_addr learned my addr 172.21.15.86:0/2299026319 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:12.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.276+0000 7f396d400640 1 --2- 172.21.15.86:0/2299026319 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f397015d0a0 0x7f39700ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:12.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.276+0000 7f396de00640 1 --2- 172.21.15.86:0/2299026319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3954004810 0x7f397015e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:12.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.276+0000 7f396e800640 1 -- 172.21.15.86:0/2299026319 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f397015d0a0 msgr2=0x7f39700ca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:12.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.276+0000 7f396e800640 1 --2- 172.21.15.86:0/2299026319 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f397015d0a0 0x7f39700ca560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:12.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.276+0000 7f396e800640 1 -- 172.21.15.86:0/2299026319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3954004810 msgr2=0x7f397015e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:12.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.276+0000 7f396e800640 1 --2- 172.21.15.86:0/2299026319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3954004810 0x7f397015e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:12.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.276+0000 7f396e800640 1 -- 172.21.15.86:0/2299026319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3970073760 con 0x7f3970072e10 2024-04-25T14:06:12.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.276+0000 7f396de00640 1 --2- 172.21.15.86:0/2299026319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3954004810 0x7f397015e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:12.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.276+0000 7f396e800640 1 --2- 172.21.15.86:0/2299026319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3970072e10 0x7f3970073220 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f39700cf5e0 tx=0x7f3968011e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:12.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.276+0000 7f3966000640 1 -- 172.21.15.86:0/2299026319 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396800faa0 con 0x7f3970072e10 2024-04-25T14:06:12.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.276+0000 7f3974a00640 1 -- 172.21.15.86:0/2299026319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f397015f5a0 con 0x7f3970072e10 2024-04-25T14:06:12.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.276+0000 7f3974a00640 1 -- 172.21.15.86:0/2299026319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f397015f880 con 0x7f3970072e10 2024-04-25T14:06:12.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.276+0000 7f3974a00640 1 -- 172.21.15.86:0/2299026319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3970069d00 con 0x7f3970072e10 2024-04-25T14:06:12.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.280+0000 7f3966000640 1 -- 172.21.15.86:0/2299026319 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f396800fc40 con 0x7f3970072e10 2024-04-25T14:06:12.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.280+0000 7f3966000640 1 -- 172.21.15.86:0/2299026319 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396800fdf0 con 0x7f3970072e10 2024-04-25T14:06:12.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.284+0000 7f3966000640 1 -- 172.21.15.86:0/2299026319 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f396801e020 con 0x7f3970072e10 2024-04-25T14:06:12.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.284+0000 7f3966000640 1 --2- 172.21.15.86:0/2299026319 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f395006f570 0x7f3950071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:12.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.284+0000 7f3966000640 1 -- 172.21.15.86:0/2299026319 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7f3968090360 con 0x7f3970072e10 2024-04-25T14:06:12.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.284+0000 7f396de00640 1 --2- 172.21.15.86:0/2299026319 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f395006f570 0x7f3950071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:12.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.284+0000 7f396de00640 1 --2- 172.21.15.86:0/2299026319 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f395006f570 0x7f3950071a80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3960000f90 tx=0x7f3960006040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:12.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.288+0000 7f3966000640 1 -- 172.21.15.86:0/2299026319 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3970073760 con 0x7f3970072e10 2024-04-25T14:06:12.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.468+0000 7f3974a00640 1 -- 172.21.15.86:0/2299026319 --> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f3970076bf0 con 0x7f395006f570 2024-04-25T14:06:12.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.468+0000 7f3966000640 1 -- 172.21.15.86:0/2299026319 <== mgr.5532 v2:172.21.15.86:6832/2427179681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f3970076bf0 con 0x7f395006f570 2024-04-25T14:06:12.470 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {}, "global_metrics": {}, "metrics": {"delayed_ranks": []}} 2024-04-25T14:06:12.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.472+0000 7f3974a00640 1 -- 172.21.15.86:0/2299026319 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f395006f570 msgr2=0x7f3950071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:12.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.472+0000 7f3974a00640 1 --2- 172.21.15.86:0/2299026319 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f395006f570 0x7f3950071a80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3960000f90 tx=0x7f3960006040 comp rx=0 tx=0).stop 2024-04-25T14:06:12.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.472+0000 7f3974a00640 1 -- 172.21.15.86:0/2299026319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3970072e10 msgr2=0x7f3970073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:12.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.472+0000 7f3974a00640 1 --2- 172.21.15.86:0/2299026319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3970072e10 0x7f3970073220 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f39700cf5e0 tx=0x7f3968011e30 comp rx=0 tx=0).stop 2024-04-25T14:06:12.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.472+0000 7f3974a00640 1 -- 172.21.15.86:0/2299026319 shutdown_connections 2024-04-25T14:06:12.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.472+0000 7f3974a00640 1 --2- 172.21.15.86:0/2299026319 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f395006f570 0x7f3950071a80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:12.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.472+0000 7f3974a00640 1 --2- 172.21.15.86:0/2299026319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3970072e10 0x7f3970073220 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:12.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.472+0000 7f3974a00640 1 --2- 172.21.15.86:0/2299026319 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f397015d0a0 0x7f39700ca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:12.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.472+0000 7f3974a00640 1 --2- 172.21.15.86:0/2299026319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3954004810 0x7f397015e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:12.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.472+0000 7f3974a00640 1 -- 172.21.15.86:0/2299026319 >> 172.21.15.86:0/2299026319 conn(0x7f39700b4650 msgr2=0x7f39700d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:12.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.472+0000 7f3974a00640 1 -- 172.21.15.86:0/2299026319 shutdown_connections 2024-04-25T14:06:12.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:12.472+0000 7f3974a00640 1 -- 172.21.15.86:0/2299026319 wait complete. 2024-04-25T14:06:13.494 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:06:13.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.656+0000 7f0f31000640 1 Processor -- start 2024-04-25T14:06:13.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.656+0000 7f0f31000640 1 -- start start 2024-04-25T14:06:13.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.656+0000 7f0f31000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0f2c0cfca0 0x7f0f2c06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:13.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.656+0000 7f0f31000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f2c0d16e0 0x7f0f2c0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:13.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.656+0000 7f0f31000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0f2c06c670 0x7f0f2c06ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:13.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.656+0000 7f0f31000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0f2c070470 con 0x7f0f2c06c670 2024-04-25T14:06:13.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.656+0000 7f0f31000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0f2c0d2030 con 0x7f0f2c0cfca0 2024-04-25T14:06:13.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.656+0000 7f0f31000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0f2c072bd0 con 0x7f0f2c0d16e0 2024-04-25T14:06:13.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.656+0000 7f0f2a800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0f2c06c670 0x7f0f2c06ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:13.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.656+0000 7f0f2a800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0f2c06c670 0x7f0f2c06ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47378/0 (socket says 172.21.15.86:47378) 2024-04-25T14:06:13.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.656+0000 7f0f2a800640 1 -- 172.21.15.86:0/392029354 learned_addr learned my addr 172.21.15.86:0/392029354 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:13.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.656+0000 7f0f21400640 1 --2- 172.21.15.86:0/392029354 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f2c0d16e0 0x7f0f2c0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:13.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.656+0000 7f0f29e00640 1 --2- 172.21.15.86:0/392029354 >> v2:172.21.15.155:3300/0 conn(0x7f0f2c0cfca0 0x7f0f2c06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:13.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.656+0000 7f0f2a800640 1 -- 172.21.15.86:0/392029354 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f2c0d16e0 msgr2=0x7f0f2c0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:13.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.656+0000 7f0f2a800640 1 --2- 172.21.15.86:0/392029354 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f2c0d16e0 0x7f0f2c0d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:13.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.656+0000 7f0f2a800640 1 -- 172.21.15.86:0/392029354 >> v2:172.21.15.155:3300/0 conn(0x7f0f2c0cfca0 msgr2=0x7f0f2c06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:13.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.656+0000 7f0f2a800640 1 --2- 172.21.15.86:0/392029354 >> v2:172.21.15.155:3300/0 conn(0x7f0f2c0cfca0 0x7f0f2c06f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:13.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.656+0000 7f0f2a800640 1 -- 172.21.15.86:0/392029354 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2c06f930 con 0x7f0f2c06c670 2024-04-25T14:06:13.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.656+0000 7f0f21400640 1 --2- 172.21.15.86:0/392029354 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f2c0d16e0 0x7f0f2c0d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:13.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.656+0000 7f0f29e00640 1 --2- 172.21.15.86:0/392029354 >> v2:172.21.15.155:3300/0 conn(0x7f0f2c0cfca0 0x7f0f2c06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:13.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.660+0000 7f0f2a800640 1 --2- 172.21.15.86:0/392029354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f2c06c670 0x7f0f2c06ca80 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c0d22a0 tx=0x7f0f2401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d06ac4ace1994b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:13.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.660+0000 7f0f29400640 1 -- 172.21.15.86:0/392029354 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f240a5780 con 0x7f0f2c06c670 2024-04-25T14:06:13.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.660+0000 7f0f29400640 1 -- 172.21.15.86:0/392029354 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f240a5920 con 0x7f0f2c06c670 2024-04-25T14:06:13.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.660+0000 7f0f29400640 1 -- 172.21.15.86:0/392029354 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f240a5cf0 con 0x7f0f2c06c670 2024-04-25T14:06:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.660+0000 7f0f31000640 1 -- 172.21.15.86:0/392029354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f2c06c670 msgr2=0x7f0f2c06ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.660+0000 7f0f31000640 1 --2- 172.21.15.86:0/392029354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f2c06c670 0x7f0f2c06ca80 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c0d22a0 tx=0x7f0f2401c9f0 comp rx=0 tx=0).stop 2024-04-25T14:06:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.660+0000 7f0f31000640 1 -- 172.21.15.86:0/392029354 shutdown_connections 2024-04-25T14:06:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.660+0000 7f0f31000640 1 --2- 172.21.15.86:0/392029354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f2c06c670 0x7f0f2c06ca80 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.660+0000 7f0f31000640 1 --2- 172.21.15.86:0/392029354 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f2c0d16e0 0x7f0f2c0d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.660+0000 7f0f31000640 1 --2- 172.21.15.86:0/392029354 >> v2:172.21.15.155:3300/0 conn(0x7f0f2c0cfca0 0x7f0f2c06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.660+0000 7f0f31000640 1 -- 172.21.15.86:0/392029354 >> 172.21.15.86:0/392029354 conn(0x7f0f2c0b4650 msgr2=0x7f0f2c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.660+0000 7f0f31000640 1 -- 172.21.15.86:0/392029354 shutdown_connections 2024-04-25T14:06:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.660+0000 7f0f31000640 1 -- 172.21.15.86:0/392029354 wait complete. 2024-04-25T14:06:13.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.664+0000 7f0f31000640 1 Processor -- start 2024-04-25T14:06:13.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.664+0000 7f0f31000640 1 -- start start 2024-04-25T14:06:13.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.664+0000 7f0f31000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f2c06c670 0x7f0f2c0ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:13.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.664+0000 7f0f31000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f2c0cfca0 0x7f0f2c15aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:13.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.664+0000 7f0f31000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f2c0d16e0 0x7f0f2c15af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:13.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.664+0000 7f0f31000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2c0738c0 con 0x7f0f2c0cfca0 2024-04-25T14:06:13.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.664+0000 7f0f31000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2c06d3d0 con 0x7f0f2c0d16e0 2024-04-25T14:06:13.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.664+0000 7f0f31000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0f2c073a40 con 0x7f0f2c06c670 2024-04-25T14:06:13.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.664+0000 7f0f21400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f2c0cfca0 0x7f0f2c15aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:13.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.664+0000 7f0f21400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f2c0cfca0 0x7f0f2c15aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47380/0 (socket says 172.21.15.86:47380) 2024-04-25T14:06:13.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.664+0000 7f0f21400640 1 -- 172.21.15.86:0/1156705225 learned_addr learned my addr 172.21.15.86:0/1156705225 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:13.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.664+0000 7f0f2a800640 1 --2- 172.21.15.86:0/1156705225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f2c0d16e0 0x7f0f2c15af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:13.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.664+0000 7f0f29e00640 1 --2- 172.21.15.86:0/1156705225 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f2c06c670 0x7f0f2c0ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:13.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.664+0000 7f0f21400640 1 -- 172.21.15.86:0/1156705225 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f2c06c670 msgr2=0x7f0f2c0ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:13.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.664+0000 7f0f21400640 1 --2- 172.21.15.86:0/1156705225 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f2c06c670 0x7f0f2c0ce5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:13.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.664+0000 7f0f21400640 1 -- 172.21.15.86:0/1156705225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f2c0d16e0 msgr2=0x7f0f2c15af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:13.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.664+0000 7f0f21400640 1 --2- 172.21.15.86:0/1156705225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f2c0d16e0 0x7f0f2c15af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:13.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.664+0000 7f0f21400640 1 -- 172.21.15.86:0/1156705225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2c0cf210 con 0x7f0f2c0cfca0 2024-04-25T14:06:13.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.664+0000 7f0f21400640 1 --2- 172.21.15.86:0/1156705225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f2c0cfca0 0x7f0f2c15aa40 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400c820 tx=0x7f0f1400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:13.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.664+0000 7f0f22a00640 1 -- 172.21.15.86:0/1156705225 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f14015070 con 0x7f0f2c0cfca0 2024-04-25T14:06:13.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.664+0000 7f0f22a00640 1 -- 172.21.15.86:0/1156705225 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f14007a30 con 0x7f0f2c0cfca0 2024-04-25T14:06:13.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.664+0000 7f0f22a00640 1 -- 172.21.15.86:0/1156705225 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f14007e40 con 0x7f0f2c0cfca0 2024-04-25T14:06:13.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.664+0000 7f0f31000640 1 -- 172.21.15.86:0/1156705225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f2c15fab0 con 0x7f0f2c0cfca0 2024-04-25T14:06:13.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.664+0000 7f0f31000640 1 -- 172.21.15.86:0/1156705225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f2c0cbd50 con 0x7f0f2c0cfca0 2024-04-25T14:06:13.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.668+0000 7f0f31000640 1 -- 172.21.15.86:0/1156705225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f2c09f4b0 con 0x7f0f2c0cfca0 2024-04-25T14:06:13.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.668+0000 7f0f22a00640 1 -- 172.21.15.86:0/1156705225 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f0f14018070 con 0x7f0f2c0cfca0 2024-04-25T14:06:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.668+0000 7f0f22a00640 1 --2- 172.21.15.86:0/1156705225 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f0f0806f570 0x7f0f08071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.668+0000 7f0f22a00640 1 -- 172.21.15.86:0/1156705225 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7f0f1408fb30 con 0x7f0f2c0cfca0 2024-04-25T14:06:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.668+0000 7f0f29e00640 1 --2- 172.21.15.86:0/1156705225 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f0f0806f570 0x7f0f08071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:13.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.668+0000 7f0f29e00640 1 --2- 172.21.15.86:0/1156705225 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f0f0806f570 0x7f0f08071a80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c006fd0 tx=0x7f0f1c008040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:13.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.672+0000 7f0f22a00640 1 -- 172.21.15.86:0/1156705225 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0f1405c170 con 0x7f0f2c0cfca0 2024-04-25T14:06:13.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.852+0000 7f0f31000640 1 -- 172.21.15.86:0/1156705225 --> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f0f2c15ffd0 con 0x7f0f0806f570 2024-04-25T14:06:13.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.852+0000 7f0f22a00640 1 -- 172.21.15.86:0/1156705225 <== mgr.5532 v2:172.21.15.86:6832/2427179681 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+1841 (secure 0 0 0) 0x7f0f1c00a1d0 con 0x7f0f0806f570 2024-04-25T14:06:13.854 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {"fs1": {"client.5384": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:172.21.15.86"}}, "fs2": {"client.6061": {"hostname": "smithi155", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:192.168.0.1"}}}, "global_metrics": {"fs1": {"client.5384": [[39, 26], [0, 0], [0, 33106355], [0, 22287970], [1, 0], [0, 1], [1, 1], [0, 1], [0, 0], [2, 3145728], [0, 0], [0, 0], [0, 16553177], [9689284024995, 2], [0, 619110], [18803815034015, 36]]}, "fs2": {"client.6061": [[27, 25], [0, 0], [0, 12482082], [0, 26837967], [1, 0], [0, 5], [5, 5], [0, 5], [0, 0], [1, 1048576], [0, 0], [0, 0], [0, 12482082], [0, 1], [0, 789355], [28375131535503, 34]]}}, "metrics": {"delayed_ranks": [], "mds.0": {"client.5384": [], "client.6061": []}}} 2024-04-25T14:06:13.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.852+0000 7f0f31000640 1 -- 172.21.15.86:0/1156705225 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f0f0806f570 msgr2=0x7f0f08071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:13.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.852+0000 7f0f31000640 1 --2- 172.21.15.86:0/1156705225 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f0f0806f570 0x7f0f08071a80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c006fd0 tx=0x7f0f1c008040 comp rx=0 tx=0).stop 2024-04-25T14:06:13.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.852+0000 7f0f31000640 1 -- 172.21.15.86:0/1156705225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f2c0cfca0 msgr2=0x7f0f2c15aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:13.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.852+0000 7f0f31000640 1 --2- 172.21.15.86:0/1156705225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f2c0cfca0 0x7f0f2c15aa40 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400c820 tx=0x7f0f1400cce0 comp rx=0 tx=0).stop 2024-04-25T14:06:13.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.856+0000 7f0f31000640 1 -- 172.21.15.86:0/1156705225 shutdown_connections 2024-04-25T14:06:13.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.856+0000 7f0f31000640 1 --2- 172.21.15.86:0/1156705225 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f0f0806f570 0x7f0f08071a80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:13.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.856+0000 7f0f31000640 1 --2- 172.21.15.86:0/1156705225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f2c0d16e0 0x7f0f2c15af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:13.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.856+0000 7f0f31000640 1 --2- 172.21.15.86:0/1156705225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f2c0cfca0 0x7f0f2c15aa40 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:13.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.856+0000 7f0f31000640 1 --2- 172.21.15.86:0/1156705225 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f2c06c670 0x7f0f2c0ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:13.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.856+0000 7f0f31000640 1 -- 172.21.15.86:0/1156705225 >> 172.21.15.86:0/1156705225 conn(0x7f0f2c0b4650 msgr2=0x7f0f2c06eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:13.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.856+0000 7f0f31000640 1 -- 172.21.15.86:0/1156705225 shutdown_connections 2024-04-25T14:06:13.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:13.856+0000 7f0f31000640 1 -- 172.21.15.86:0/1156705225 wait complete. 2024-04-25T14:06:13.876 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:06:14.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.020+0000 7f4a9b200640 1 Processor -- start 2024-04-25T14:06:14.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.024+0000 7f4a9b200640 1 -- start start 2024-04-25T14:06:14.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.024+0000 7f4a9b200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4a940cfca0 0x7f4a9406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:14.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.024+0000 7f4a9b200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4a940730b0 0x7f4a940734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:14.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.024+0000 7f4a9b200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a94072bd0 con 0x7f4a940d16e0 2024-04-25T14:06:14.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.024+0000 7f4a9b200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4a94070470 con 0x7f4a940730b0 2024-04-25T14:06:14.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.024+0000 7f4a9b200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f4a9406f930 con 0x7f4a940cfca0 2024-04-25T14:06:14.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.024+0000 7f4a99200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4a940730b0 0x7f4a940734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:14.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.024+0000 7f4a99200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4a940730b0 0x7f4a940734c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47392/0 (socket says 172.21.15.86:47392) 2024-04-25T14:06:14.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.024+0000 7f4a99200640 1 -- 172.21.15.86:0/369255614 learned_addr learned my addr 172.21.15.86:0/369255614 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:14.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.024+0000 7f4a93e00640 1 --2- 172.21.15.86:0/369255614 >> v2:172.21.15.155:3300/0 conn(0x7f4a940cfca0 0x7f4a9406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:14.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.024+0000 7f4a99200640 1 -- 172.21.15.86:0/369255614 >> v2:172.21.15.155:3300/0 conn(0x7f4a940cfca0 msgr2=0x7f4a9406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:14.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.024+0000 7f4a99200640 1 --2- 172.21.15.86:0/369255614 >> v2:172.21.15.155:3300/0 conn(0x7f4a940cfca0 0x7f4a9406f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:14.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.024+0000 7f4a99200640 1 -- 172.21.15.86:0/369255614 >> v1:172.21.15.155:6789/0 conn(0x7f4a940d16e0 legacy=0x7f4a940d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:14.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.024+0000 7f4a99200640 1 -- 172.21.15.86:0/369255614 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9406fbd0 con 0x7f4a940730b0 2024-04-25T14:06:14.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.024+0000 7f4a93e00640 1 --2- 172.21.15.86:0/369255614 >> v2:172.21.15.155:3300/0 conn(0x7f4a940cfca0 0x7f4a9406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:14.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.028+0000 7f4a99200640 1 --2- 172.21.15.86:0/369255614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a940730b0 0x7f4a940734c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4a940d22a0 tx=0x7f4a8c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b972763b2dba7fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:14.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.028+0000 7f4a93400640 1 -- 172.21.15.86:0/369255614 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0a57d0 con 0x7f4a940730b0 2024-04-25T14:06:14.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.028+0000 7f4a93400640 1 -- 172.21.15.86:0/369255614 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a8c0a5970 con 0x7f4a940730b0 2024-04-25T14:06:14.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.028+0000 7f4a93400640 1 -- 172.21.15.86:0/369255614 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0a5d40 con 0x7f4a940730b0 2024-04-25T14:06:14.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.028+0000 7f4a9b200640 1 -- 172.21.15.86:0/369255614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a940730b0 msgr2=0x7f4a940734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:14.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.028+0000 7f4a9b200640 1 --2- 172.21.15.86:0/369255614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a940730b0 0x7f4a940734c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4a940d22a0 tx=0x7f4a8c01c9f0 comp rx=0 tx=0).stop 2024-04-25T14:06:14.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.028+0000 7f4a9b200640 1 -- 172.21.15.86:0/369255614 shutdown_connections 2024-04-25T14:06:14.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.028+0000 7f4a9b200640 1 --2- 172.21.15.86:0/369255614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a940730b0 0x7f4a940734c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:14.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.028+0000 7f4a9b200640 1 --2- 172.21.15.86:0/369255614 >> v2:172.21.15.155:3300/0 conn(0x7f4a940cfca0 0x7f4a9406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:14.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.028+0000 7f4a9b200640 1 -- 172.21.15.86:0/369255614 >> 172.21.15.86:0/369255614 conn(0x7f4a940b4650 msgr2=0x7f4a940b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:14.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.028+0000 7f4a9b200640 1 -- 172.21.15.86:0/369255614 shutdown_connections 2024-04-25T14:06:14.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.028+0000 7f4a9b200640 1 -- 172.21.15.86:0/369255614 wait complete. 2024-04-25T14:06:14.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.032+0000 7f4a9b200640 1 Processor -- start 2024-04-25T14:06:14.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.032+0000 7f4a9b200640 1 -- start start 2024-04-25T14:06:14.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.032+0000 7f4a9b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a940730b0 0x7f4a9415fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:14.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.032+0000 7f4a9b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a940cfca0 0x7f4a940c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:14.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.032+0000 7f4a9b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a940d16e0 0x7f4a940cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:14.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.032+0000 7f4a9b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a94169a70 con 0x7f4a940d16e0 2024-04-25T14:06:14.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.032+0000 7f4a9b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4a941698f0 con 0x7f4a940cfca0 2024-04-25T14:06:14.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.032+0000 7f4a9b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4a94169bf0 con 0x7f4a940730b0 2024-04-25T14:06:14.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.032+0000 7f4a99200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a940d16e0 0x7f4a940cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:14.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.032+0000 7f4a99200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a940d16e0 0x7f4a940cd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47396/0 (socket says 172.21.15.86:47396) 2024-04-25T14:06:14.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.032+0000 7f4a99200640 1 -- 172.21.15.86:0/3029601670 learned_addr learned my addr 172.21.15.86:0/3029601670 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:14.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.032+0000 7f4a93e00640 1 --2- 172.21.15.86:0/3029601670 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a940730b0 0x7f4a9415fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:14.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.032+0000 7f4a8b400640 1 --2- 172.21.15.86:0/3029601670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a940cfca0 0x7f4a940c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:14.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.032+0000 7f4a99200640 1 -- 172.21.15.86:0/3029601670 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a940730b0 msgr2=0x7f4a9415fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:14.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.032+0000 7f4a99200640 1 --2- 172.21.15.86:0/3029601670 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a940730b0 0x7f4a9415fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:14.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.032+0000 7f4a99200640 1 -- 172.21.15.86:0/3029601670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a940cfca0 msgr2=0x7f4a940c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:14.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.032+0000 7f4a99200640 1 --2- 172.21.15.86:0/3029601670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a940cfca0 0x7f4a940c97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:14.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.032+0000 7f4a99200640 1 -- 172.21.15.86:0/3029601670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a940ca470 con 0x7f4a940d16e0 2024-04-25T14:06:14.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.032+0000 7f4a99200640 1 --2- 172.21.15.86:0/3029601670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a940d16e0 0x7f4a940cd880 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0af040 tx=0x7f4a8c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:14.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.032+0000 7f4a90c00640 1 -- 172.21.15.86:0/3029601670 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0bc020 con 0x7f4a940d16e0 2024-04-25T14:06:14.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.032+0000 7f4a90c00640 1 -- 172.21.15.86:0/3029601670 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a8c0b8670 con 0x7f4a940d16e0 2024-04-25T14:06:14.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.032+0000 7f4a90c00640 1 -- 172.21.15.86:0/3029601670 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0b8a60 con 0x7f4a940d16e0 2024-04-25T14:06:14.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.032+0000 7f4a9b200640 1 -- 172.21.15.86:0/3029601670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a940ca750 con 0x7f4a940d16e0 2024-04-25T14:06:14.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.032+0000 7f4a9b200640 1 -- 172.21.15.86:0/3029601670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a9415ef00 con 0x7f4a940d16e0 2024-04-25T14:06:14.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.036+0000 7f4a9b200640 1 -- 172.21.15.86:0/3029601670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a940cac00 con 0x7f4a940d16e0 2024-04-25T14:06:14.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.036+0000 7f4a90c00640 1 -- 172.21.15.86:0/3029601670 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7f4a8c0bf020 con 0x7f4a940d16e0 2024-04-25T14:06:14.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.036+0000 7f4a90c00640 1 --2- 172.21.15.86:0/3029601670 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f4a7006f5a0 0x7f4a70071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:14.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.036+0000 7f4a90c00640 1 -- 172.21.15.86:0/3029601670 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7f4a8c12ad60 con 0x7f4a940d16e0 2024-04-25T14:06:14.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.036+0000 7f4a93e00640 1 --2- 172.21.15.86:0/3029601670 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f4a7006f5a0 0x7f4a70071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:14.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.036+0000 7f4a93e00640 1 --2- 172.21.15.86:0/3029601670 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f4a7006f5a0 0x7f4a70071ab0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4a94070710 tx=0x7f4a84008040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:14.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.040+0000 7f4a90c00640 1 -- 172.21.15.86:0/3029601670 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4a8c0c6030 con 0x7f4a940d16e0 2024-04-25T14:06:14.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.232+0000 7f4a9b200640 1 -- 172.21.15.86:0/3029601670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a940ca200 con 0x7f4a940d16e0 2024-04-25T14:06:14.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.232+0000 7f4a90c00640 1 -- 172.21.15.86:0/3029601670 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+6495 (secure 0 0 0) 0x7f4a8c0165e0 con 0x7f4a940d16e0 2024-04-25T14:06:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:14.234 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3545772170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3545772170},{"type":"v1","addr":"172.21.15.155:6834","nonce":3545772170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5980,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/766530004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":766530004},{"type":"v1","addr":"172.21.15.86:6837","nonce":766530004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:32.946320+0000","modified":"2024-04-25T14:06:03.253171+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6048},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6048":{"gid":6048,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6835/2169924270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2169924270},{"type":"v1","addr":"172.21.15.86:6835","nonce":2169924270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"fs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6048,"qdb_cluster":[6048]},"id":3},{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:54.565528+0000","modified":"2024-04-25T14:06:03.253173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6036":{"gid":6036,"name":"d","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6835/2914930843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2914930843},{"type":"v1","addr":"172.21.15.155:6835","nonce":2914930843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"fs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6036,"qdb_cluster":[6036]},"id":4}]} 2024-04-25T14:06:14.234 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 43 2024-04-25T14:06:14.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.232+0000 7f4a9b200640 1 -- 172.21.15.86:0/3029601670 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f4a7006f5a0 msgr2=0x7f4a70071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:14.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.232+0000 7f4a9b200640 1 --2- 172.21.15.86:0/3029601670 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f4a7006f5a0 0x7f4a70071ab0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4a94070710 tx=0x7f4a84008040 comp rx=0 tx=0).stop 2024-04-25T14:06:14.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.232+0000 7f4a9b200640 1 -- 172.21.15.86:0/3029601670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a940d16e0 msgr2=0x7f4a940cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:14.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.232+0000 7f4a9b200640 1 --2- 172.21.15.86:0/3029601670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a940d16e0 0x7f4a940cd880 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0af040 tx=0x7f4a8c019790 comp rx=0 tx=0).stop 2024-04-25T14:06:14.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.232+0000 7f4a9b200640 1 -- 172.21.15.86:0/3029601670 shutdown_connections 2024-04-25T14:06:14.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.232+0000 7f4a9b200640 1 --2- 172.21.15.86:0/3029601670 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f4a7006f5a0 0x7f4a70071ab0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:14.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.232+0000 7f4a9b200640 1 --2- 172.21.15.86:0/3029601670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a940d16e0 0x7f4a940cd880 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:14.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.232+0000 7f4a9b200640 1 --2- 172.21.15.86:0/3029601670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a940cfca0 0x7f4a940c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:14.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.232+0000 7f4a9b200640 1 --2- 172.21.15.86:0/3029601670 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a940730b0 0x7f4a9415fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:14.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.232+0000 7f4a9b200640 1 -- 172.21.15.86:0/3029601670 >> 172.21.15.86:0/3029601670 conn(0x7f4a940b4650 msgr2=0x7f4a940b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:14.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.232+0000 7f4a9b200640 1 -- 172.21.15.86:0/3029601670 shutdown_connections 2024-04-25T14:06:14.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.232+0000 7f4a9b200640 1 -- 172.21.15.86:0/3029601670 wait complete. 2024-04-25T14:06:14.256 DEBUG:tasks.cephfs.test_mds_metrics:metrics={'version': 2, 'global_counters': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'counters': [], 'client_metadata': {'fs1': {'client.5384': {'hostname': 'smithi086', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'kernel_version': '6.9.0-rc3-g806ec36b2112', 'IP': 'v1:172.21.15.86'}}, 'fs2': {'client.6061': {'hostname': 'smithi155', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'kernel_version': '6.9.0-rc3-g806ec36b2112', 'IP': 'v1:192.168.0.1'}}}, 'global_metrics': {'fs1': {'client.5384': [[39, 26], [0, 0], [0, 33106355], [0, 22287970], [1, 0], [0, 1], [1, 1], [0, 1], [0, 0], [2, 3145728], [0, 0], [0, 0], [0, 16553177], [9689284024995, 2], [0, 619110], [18803815034015, 36]]}, 'fs2': {'client.6061': [[27, 25], [0, 0], [0, 12482082], [0, 26837967], [1, 0], [0, 5], [5, 5], [0, 5], [0, 0], [1, 1048576], [0, 0], [0, 0], [0, 12482082], [0, 1], [0, 789355], [28375131535503, 34]]}}, 'metrics': {'delayed_ranks': [], 'mds.0': {'client.5384': [], 'client.6061': []}}} 2024-04-25T14:06:14.257 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable stats 2024-04-25T14:06:14.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.404+0000 7fb0c9e00640 1 Processor -- start 2024-04-25T14:06:14.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.404+0000 7fb0c9e00640 1 -- start start 2024-04-25T14:06:14.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c9e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb0c40740c0 0x7fb0c41676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:14.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c9e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0c4072bd0 con 0x7fb0c40d16e0 2024-04-25T14:06:14.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c9e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0c406fbd0 con 0x7fb0c40cfca0 2024-04-25T14:06:14.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c9e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb0c4070470 con 0x7fb0c40740c0 2024-04-25T14:06:14.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c2000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb0c40d16e0 0x7fb0c40d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59790/0 (socket says 172.21.15.86:59790) 2024-04-25T14:06:14.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c2000640 1 -- 172.21.15.86:0/874069190 learned_addr learned my addr 172.21.15.86:0/874069190 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:14.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c3400640 1 --2- 172.21.15.86:0/874069190 >> v2:172.21.15.155:3300/0 conn(0x7fb0c40740c0 0x7fb0c41676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:14.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c1600640 1 -- 172.21.15.86:0/874069190 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1229860913 0 0) 0x7fb0c4072bd0 con 0x7fb0c40d16e0 2024-04-25T14:06:14.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c1600640 1 -- 172.21.15.86:0/874069190 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0a8003530 con 0x7fb0c40d16e0 2024-04-25T14:06:14.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c1600640 1 -- 172.21.15.86:0/874069190 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 290207840 0 0) 0x7fb0a8003530 con 0x7fb0c40d16e0 2024-04-25T14:06:14.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c1600640 1 -- 172.21.15.86:0/874069190 >> v2:172.21.15.155:3300/0 conn(0x7fb0c40740c0 msgr2=0x7fb0c41676d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:14.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c1600640 1 --2- 172.21.15.86:0/874069190 >> v2:172.21.15.155:3300/0 conn(0x7fb0c40740c0 0x7fb0c41676d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:14.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c1600640 1 -- 172.21.15.86:0/874069190 >> v1:172.21.15.155:6789/0 conn(0x7fb0c40cfca0 legacy=0x7fb0c406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:14.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c1600640 1 -- 172.21.15.86:0/874069190 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c406feb0 con 0x7fb0c40d16e0 2024-04-25T14:06:14.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c1600640 1 -- 172.21.15.86:0/874069190 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fb0ac0016a0 con 0x7fb0c40d16e0 2024-04-25T14:06:14.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c3400640 1 --2- 172.21.15.86:0/874069190 >> v2:172.21.15.155:3300/0 conn(0x7fb0c40740c0 0x7fb0c41676d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:14.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c1600640 1 -- 172.21.15.86:0/874069190 >> v1:172.21.15.86:6789/0 conn(0x7fb0c40d16e0 legacy=0x7fb0c40d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:14.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c1600640 1 --2- 172.21.15.86:0/874069190 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a8003a30 0x7fb0a8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:14.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c1600640 1 --2- 172.21.15.86:0/874069190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0a8004810 0x7fb0a8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:14.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c1600640 1 --2- 172.21.15.86:0/874069190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0a8005230 0x7fb0a800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:14.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c1600640 1 -- 172.21.15.86:0/874069190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c4070470 con 0x7fb0a8005230 2024-04-25T14:06:14.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c1600640 1 -- 172.21.15.86:0/874069190 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a800c1c0 con 0x7fb0a8004810 2024-04-25T14:06:14.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c1600640 1 -- 172.21.15.86:0/874069190 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb0a800e5d0 con 0x7fb0a8003a30 2024-04-25T14:06:14.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c3400640 1 --2- 172.21.15.86:0/874069190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0a8005230 0x7fb0a800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:14.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c2000640 1 --2- 172.21.15.86:0/874069190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0a8004810 0x7fb0a8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:14.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c2a00640 1 --2- 172.21.15.86:0/874069190 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a8003a30 0x7fb0a8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:14.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c3400640 1 -- 172.21.15.86:0/874069190 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a8003a30 msgr2=0x7fb0a8003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:14.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c3400640 1 --2- 172.21.15.86:0/874069190 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a8003a30 0x7fb0a8003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:14.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c3400640 1 -- 172.21.15.86:0/874069190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0a8004810 msgr2=0x7fb0a8004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:14.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c3400640 1 --2- 172.21.15.86:0/874069190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0a8004810 0x7fb0a8004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:14.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.408+0000 7fb0c3400640 1 -- 172.21.15.86:0/874069190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a8003530 con 0x7fb0a8005230 2024-04-25T14:06:14.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.412+0000 7fb0c3400640 1 -- 172.21.15.86:0/874069190 reap_dead start 2024-04-25T14:06:14.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.412+0000 7fb0c3400640 1 --2- 172.21.15.86:0/874069190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0a8005230 0x7fb0a800b9d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc00e320 tx=0x7fb0bc01ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26bbf2f4dfa80334 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:14.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.412+0000 7fb0c1600640 1 -- 172.21.15.86:0/874069190 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc009de0 con 0x7fb0a8005230 2024-04-25T14:06:14.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.412+0000 7fb0c1600640 1 -- 172.21.15.86:0/874069190 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0bc0a5e50 con 0x7fb0a8005230 2024-04-25T14:06:14.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.412+0000 7fb0c1600640 1 -- 172.21.15.86:0/874069190 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc003be0 con 0x7fb0a8005230 2024-04-25T14:06:14.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.412+0000 7fb0c9e00640 1 -- 172.21.15.86:0/874069190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0a8005230 msgr2=0x7fb0a800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:14.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.412+0000 7fb0c9e00640 1 --2- 172.21.15.86:0/874069190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0a8005230 0x7fb0a800b9d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc00e320 tx=0x7fb0bc01ce40 comp rx=0 tx=0).stop 2024-04-25T14:06:14.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.412+0000 7fb0c9e00640 1 -- 172.21.15.86:0/874069190 shutdown_connections 2024-04-25T14:06:14.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.412+0000 7fb0c9e00640 1 --2- 172.21.15.86:0/874069190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0a8005230 0x7fb0a800b9d0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:14.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.412+0000 7fb0c9e00640 1 -- 172.21.15.86:0/874069190 >> 172.21.15.86:0/874069190 conn(0x7fb0c40b4650 msgr2=0x7fb0c40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:14.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.412+0000 7fb0c9e00640 1 -- 172.21.15.86:0/874069190 shutdown_connections 2024-04-25T14:06:14.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.412+0000 7fb0c9e00640 1 -- 172.21.15.86:0/874069190 wait complete. 2024-04-25T14:06:14.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.416+0000 7fb0c9e00640 1 Processor -- start 2024-04-25T14:06:14.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.416+0000 7fb0c9e00640 1 -- start start 2024-04-25T14:06:14.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.416+0000 7fb0c9e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a8005230 0x7fb0c41f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:14.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.416+0000 7fb0c9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0c40cbbd0 0x7fb0c40cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:14.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.416+0000 7fb0c9e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0c40cdfe0 0x7fb0c40ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:14.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.416+0000 7fb0c9e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c406f570 con 0x7fb0c40cbbd0 2024-04-25T14:06:14.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.416+0000 7fb0c9e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c406f3f0 con 0x7fb0c40cdfe0 2024-04-25T14:06:14.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.416+0000 7fb0c9e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb0c406f6f0 con 0x7fb0a8005230 2024-04-25T14:06:14.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.416+0000 7fb0c2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0c40cbbd0 0x7fb0c40cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:14.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.416+0000 7fb0c2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0c40cbbd0 0x7fb0c40cbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47402/0 (socket says 172.21.15.86:47402) 2024-04-25T14:06:14.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.416+0000 7fb0c2000640 1 -- 172.21.15.86:0/1329385089 learned_addr learned my addr 172.21.15.86:0/1329385089 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:14.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.416+0000 7fb0c2a00640 1 --2- 172.21.15.86:0/1329385089 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a8005230 0x7fb0c41f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:14.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.416+0000 7fb0c3400640 1 --2- 172.21.15.86:0/1329385089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0c40cdfe0 0x7fb0c40ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:14.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.416+0000 7fb0c2000640 1 -- 172.21.15.86:0/1329385089 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a8005230 msgr2=0x7fb0c41f5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:14.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.416+0000 7fb0c2000640 1 --2- 172.21.15.86:0/1329385089 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a8005230 0x7fb0c41f5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:14.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.416+0000 7fb0c2000640 1 -- 172.21.15.86:0/1329385089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0c40cdfe0 msgr2=0x7fb0c40ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:14.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.416+0000 7fb0c2000640 1 --2- 172.21.15.86:0/1329385089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0c40cdfe0 0x7fb0c40ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:14.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.416+0000 7fb0c2000640 1 -- 172.21.15.86:0/1329385089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c40ce930 con 0x7fb0c40cbbd0 2024-04-25T14:06:14.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.416+0000 7fb0c2a00640 1 --2- 172.21.15.86:0/1329385089 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a8005230 0x7fb0c41f5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:14.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.416+0000 7fb0c3400640 1 --2- 172.21.15.86:0/1329385089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0c40cdfe0 0x7fb0c40ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:14.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.416+0000 7fb0c2000640 1 --2- 172.21.15.86:0/1329385089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0c40cbbd0 0x7fb0c40cbfe0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac007290 tx=0x7fb0ac007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:14.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.416+0000 7fb0bb400640 1 -- 172.21.15.86:0/1329385089 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac00f940 con 0x7fb0c40cbbd0 2024-04-25T14:06:14.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.420+0000 7fb0bb400640 1 -- 172.21.15.86:0/1329385089 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0ac018070 con 0x7fb0c40cbbd0 2024-04-25T14:06:14.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.420+0000 7fb0bb400640 1 -- 172.21.15.86:0/1329385089 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac00fd40 con 0x7fb0c40cbbd0 2024-04-25T14:06:14.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.420+0000 7fb0c9e00640 1 -- 172.21.15.86:0/1329385089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c41f63b0 con 0x7fb0c40cbbd0 2024-04-25T14:06:14.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.420+0000 7fb0c9e00640 1 -- 172.21.15.86:0/1329385089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c40ccb50 con 0x7fb0c40cbbd0 2024-04-25T14:06:14.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.420+0000 7fb0c9e00640 1 -- 172.21.15.86:0/1329385089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0c409f4b0 con 0x7fb0c40cbbd0 2024-04-25T14:06:14.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.420+0000 7fb0bb400640 1 -- 172.21.15.86:0/1329385089 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 93318+0+0 (secure 0 0 0) 0x7fb0ac01d460 con 0x7fb0c40cbbd0 2024-04-25T14:06:14.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.420+0000 7fb0bb400640 1 --2- 172.21.15.86:0/1329385089 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb0a006f570 0x7fb0a0071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:14.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.420+0000 7fb0bb400640 1 -- 172.21.15.86:0/1329385089 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7fb0ac08fd20 con 0x7fb0c40cbbd0 2024-04-25T14:06:14.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.420+0000 7fb0c2a00640 1 --2- 172.21.15.86:0/1329385089 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb0a006f570 0x7fb0a0071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:14.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.424+0000 7fb0c2a00640 1 --2- 172.21.15.86:0/1329385089 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb0a006f570 0x7fb0a0071a80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4007fd0 tx=0x7fb0b4009040 comp rx=0 tx=0).ready entity=mgr.5532 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:14.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.424+0000 7fb0bb400640 1 -- 172.21.15.86:0/1329385089 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb0c40ce930 con 0x7fb0c40cbbd0 2024-04-25T14:06:14.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:14.620+0000 7fb0c9e00640 1 -- 172.21.15.86:0/1329385089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "stats"} v 0) -- 0x7fb0c4069d00 con 0x7fb0c40cbbd0 2024-04-25T14:06:15.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.300+0000 7fb0bb400640 1 -- 172.21.15.86:0/1329385089 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "stats"}]=0 v12) ==== 85+0+0 (secure 0 0 0) 0x7fb0ac05c340 con 0x7fb0c40cbbd0 2024-04-25T14:06:15.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.304+0000 7fb0c9e00640 1 -- 172.21.15.86:0/1329385089 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb0a006f570 msgr2=0x7fb0a0071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:15.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.304+0000 7fb0c9e00640 1 --2- 172.21.15.86:0/1329385089 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb0a006f570 0x7fb0a0071a80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4007fd0 tx=0x7fb0b4009040 comp rx=0 tx=0).stop 2024-04-25T14:06:15.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.304+0000 7fb0c9e00640 1 -- 172.21.15.86:0/1329385089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0c40cbbd0 msgr2=0x7fb0c40cbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:15.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.304+0000 7fb0c9e00640 1 --2- 172.21.15.86:0/1329385089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0c40cbbd0 0x7fb0c40cbfe0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac007290 tx=0x7fb0ac007750 comp rx=0 tx=0).stop 2024-04-25T14:06:15.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.304+0000 7fb0c9e00640 1 -- 172.21.15.86:0/1329385089 shutdown_connections 2024-04-25T14:06:15.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.304+0000 7fb0c9e00640 1 --2- 172.21.15.86:0/1329385089 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb0a006f570 0x7fb0a0071a80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:15.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.304+0000 7fb0c9e00640 1 --2- 172.21.15.86:0/1329385089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0c40cdfe0 0x7fb0c40ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:15.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.304+0000 7fb0c9e00640 1 --2- 172.21.15.86:0/1329385089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0c40cbbd0 0x7fb0c40cbfe0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:15.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.304+0000 7fb0c9e00640 1 --2- 172.21.15.86:0/1329385089 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a8005230 0x7fb0c41f5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:15.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.304+0000 7fb0c9e00640 1 -- 172.21.15.86:0/1329385089 >> 172.21.15.86:0/1329385089 conn(0x7fb0c40b4650 msgr2=0x7fb0c40d1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:15.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.308+0000 7fb0c9e00640 1 -- 172.21.15.86:0/1329385089 shutdown_connections 2024-04-25T14:06:15.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.308+0000 7fb0c9e00640 1 -- 172.21.15.86:0/1329385089 wait complete. 2024-04-25T14:06:15.333 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T14:06:15.334 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:06:15.438 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:06:15.518 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T14:06:15.518 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:06:15.518 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:06:15.518 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:06:15.527 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:06:15.527 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:06:15.561 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:06:15.561+0000 7f95af866180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:06:15.565 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:15.564+0000 7fc57b0d6180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:06:15.574 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:06:15.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.760+0000 7f4434600640 1 Processor -- start 2024-04-25T14:06:15.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.764+0000 7f4434600640 1 -- start start 2024-04-25T14:06:15.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.764+0000 7f4434600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f442c0cd090 0x7f442c0cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:15.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.764+0000 7f4434600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f442c0cd9e0 0x7f442c0cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:15.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.764+0000 7f4434600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f442c0d2960 con 0x7f442c0b3740 2024-04-25T14:06:15.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.764+0000 7f4434600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f442c06c670 con 0x7f442c0cd090 2024-04-25T14:06:15.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.764+0000 7f4434600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f442c0d2730 con 0x7f442c0cd9e0 2024-04-25T14:06:15.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.764+0000 7f4432600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f442c0cd9e0 0x7f442c0cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:15.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.764+0000 7f4431200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f442c0cd090 0x7f442c0cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:15.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.764+0000 7f4432600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f442c0cd9e0 0x7f442c0cddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:46170/0 (socket says 172.21.15.86:46170) 2024-04-25T14:06:15.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.764+0000 7f4432600640 1 -- 172.21.15.86:0/1836522860 learned_addr learned my addr 172.21.15.86:0/1836522860 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:15.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.764+0000 7f4432600640 1 -- 172.21.15.86:0/1836522860 >> v2:172.21.15.155:3300/0 conn(0x7f442c0cd090 msgr2=0x7f442c0cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:15.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.764+0000 7f4432600640 1 --2- 172.21.15.86:0/1836522860 >> v2:172.21.15.155:3300/0 conn(0x7f442c0cd090 0x7f442c0cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:15.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.764+0000 7f4432600640 1 -- 172.21.15.86:0/1836522860 >> v1:172.21.15.155:6789/0 conn(0x7f442c0b3740 legacy=0x7f442c0ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:15.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.764+0000 7f4432600640 1 -- 172.21.15.86:0/1836522860 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f442c0d2c40 con 0x7f442c0cd9e0 2024-04-25T14:06:15.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.764+0000 7f4431200640 1 --2- 172.21.15.86:0/1836522860 >> v2:172.21.15.155:3300/0 conn(0x7f442c0cd090 0x7f442c0cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:15.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.768+0000 7f4432600640 1 --2- 172.21.15.86:0/1836522860 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f442c0cd9e0 0x7f442c0cddf0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4424009e50 tx=0x7f442401ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86f6c3854963e96d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:15.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.768+0000 7f442be00640 1 -- 172.21.15.86:0/1836522860 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44240a5700 con 0x7f442c0cd9e0 2024-04-25T14:06:15.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.768+0000 7f442be00640 1 -- 172.21.15.86:0/1836522860 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44240a58a0 con 0x7f442c0cd9e0 2024-04-25T14:06:15.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.768+0000 7f442be00640 1 -- 172.21.15.86:0/1836522860 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44240a5c70 con 0x7f442c0cd9e0 2024-04-25T14:06:15.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.768+0000 7f4434600640 1 -- 172.21.15.86:0/1836522860 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f442c0cd9e0 msgr2=0x7f442c0cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:15.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.768+0000 7f4434600640 1 --2- 172.21.15.86:0/1836522860 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f442c0cd9e0 0x7f442c0cddf0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4424009e50 tx=0x7f442401ca30 comp rx=0 tx=0).stop 2024-04-25T14:06:15.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.768+0000 7f4434600640 1 -- 172.21.15.86:0/1836522860 shutdown_connections 2024-04-25T14:06:15.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.768+0000 7f4434600640 1 --2- 172.21.15.86:0/1836522860 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f442c0cd9e0 0x7f442c0cddf0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:15.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.768+0000 7f4434600640 1 --2- 172.21.15.86:0/1836522860 >> v2:172.21.15.155:3300/0 conn(0x7f442c0cd090 0x7f442c0cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:15.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.768+0000 7f4434600640 1 -- 172.21.15.86:0/1836522860 >> 172.21.15.86:0/1836522860 conn(0x7f442c09ba30 msgr2=0x7f442c0b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:15.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.768+0000 7f4434600640 1 -- 172.21.15.86:0/1836522860 shutdown_connections 2024-04-25T14:06:15.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.768+0000 7f4434600640 1 -- 172.21.15.86:0/1836522860 wait complete. 2024-04-25T14:06:15.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.772+0000 7f4434600640 1 Processor -- start 2024-04-25T14:06:15.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.772+0000 7f4434600640 1 -- start start 2024-04-25T14:06:15.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.772+0000 7f4434600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f442c0b3740 0x7f442c14c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:15.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.772+0000 7f4434600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f442c0cd090 0x7f442c158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:15.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.772+0000 7f4434600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f442c0cd9e0 0x7f442c158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:15.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.772+0000 7f4434600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f442c0cfca0 con 0x7f442c0cd9e0 2024-04-25T14:06:15.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.772+0000 7f4434600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f442c0b3ce0 con 0x7f442c0cd090 2024-04-25T14:06:15.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.772+0000 7f4434600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f442c16d1f0 con 0x7f442c0b3740 2024-04-25T14:06:15.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.772+0000 7f4432600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f442c0cd9e0 0x7f442c158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:15.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.772+0000 7f4431c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f442c0b3740 0x7f442c14c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:15.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.772+0000 7f4432600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f442c0cd9e0 0x7f442c158860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47410/0 (socket says 172.21.15.86:47410) 2024-04-25T14:06:15.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.772+0000 7f4432600640 1 -- 172.21.15.86:0/3898138023 learned_addr learned my addr 172.21.15.86:0/3898138023 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:15.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.772+0000 7f4431200640 1 --2- 172.21.15.86:0/3898138023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f442c0cd090 0x7f442c158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:15.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.772+0000 7f4432600640 1 -- 172.21.15.86:0/3898138023 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f442c0b3740 msgr2=0x7f442c14c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:15.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.772+0000 7f4432600640 1 --2- 172.21.15.86:0/3898138023 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f442c0b3740 0x7f442c14c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:15.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.772+0000 7f4432600640 1 -- 172.21.15.86:0/3898138023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f442c0cd090 msgr2=0x7f442c158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:15.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.772+0000 7f4432600640 1 --2- 172.21.15.86:0/3898138023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f442c0cd090 0x7f442c158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:15.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.772+0000 7f4432600640 1 -- 172.21.15.86:0/3898138023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f442c153410 con 0x7f442c0cd9e0 2024-04-25T14:06:15.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.772+0000 7f4431c00640 1 --2- 172.21.15.86:0/3898138023 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f442c0b3740 0x7f442c14c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:15.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.776+0000 7f4432600640 1 --2- 172.21.15.86:0/3898138023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f442c0cd9e0 0x7f442c158860 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f44240af040 tx=0x7f44240197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:15.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.776+0000 7f4429600640 1 -- 172.21.15.86:0/3898138023 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4424009510 con 0x7f442c0cd9e0 2024-04-25T14:06:15.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.776+0000 7f4429600640 1 -- 172.21.15.86:0/3898138023 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44240be070 con 0x7f442c0cd9e0 2024-04-25T14:06:15.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.776+0000 7f4434600640 1 -- 172.21.15.86:0/3898138023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f442c160670 con 0x7f442c0cd9e0 2024-04-25T14:06:15.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.776+0000 7f4429600640 1 -- 172.21.15.86:0/3898138023 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44240b88d0 con 0x7f442c0cd9e0 2024-04-25T14:06:15.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.776+0000 7f4434600640 1 -- 172.21.15.86:0/3898138023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f442c153990 con 0x7f442c0cd9e0 2024-04-25T14:06:15.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.776+0000 7f4434600640 1 -- 172.21.15.86:0/3898138023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f442c0a0630 con 0x7f442c0cd9e0 2024-04-25T14:06:15.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.776+0000 7f4429600640 1 -- 172.21.15.86:0/3898138023 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) ==== 93309+0+0 (secure 0 0 0) 0x7f44240b8a70 con 0x7f442c0cd9e0 2024-04-25T14:06:15.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.776+0000 7f4429600640 1 --2- 172.21.15.86:0/3898138023 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f440c06f520 0x7f440c071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:15.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.780+0000 7f4431c00640 1 -- 172.21.15.86:0/3898138023 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f440c06f520 msgr2=0x7f440c071a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2427179681 2024-04-25T14:06:15.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.780+0000 7f4429600640 1 -- 172.21.15.86:0/3898138023 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7f442412a960 con 0x7f442c0cd9e0 2024-04-25T14:06:15.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.780+0000 7f4431c00640 1 --2- 172.21.15.86:0/3898138023 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f440c06f520 0x7f440c071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:06:15.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.784+0000 7f4429600640 1 -- 172.21.15.86:0/3898138023 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4424016620 con 0x7f442c0cd9e0 2024-04-25T14:06:15.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.944+0000 7f4434600640 1 -- 172.21.15.86:0/3898138023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f442c0cd530 con 0x7f442c0cd9e0 2024-04-25T14:06:15.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.944+0000 7f4429600640 1 -- 172.21.15.86:0/3898138023 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+21531 (secure 0 0 0) 0x7f44240f6a60 con 0x7f442c0cd9e0 2024-04-25T14:06:15.946 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:15.946 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":58,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:06:00.230399+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"fs1_metadata","create_time":"2024-04-25T14:05:30.939601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"fs1"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"fs1_data","create_time":"2024-04-25T14:05:31.898262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"fs1"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"fs2_metadata","create_time":"2024-04-25T14:05:52.841764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"fs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"fs2_data","create_time":"2024-04-25T14:05:53.524454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"fs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6836/1732615622":"2024-04-26T14:06:00.230367+0000","172.21.15.86:6837/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:6837/2418778265":"2024-04-26T14:06:00.222915+0000","172.21.15.86:0/1843278167":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6832/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/2073988773":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6835/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:0/2602032945":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/3558294881":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6835/615772001":"2024-04-26T14:05:42.882867+0000","172.21.15.86:6837/615772001":"2024-04-26T14:05:42.882867+0000","172.21.15.86:6833/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6835/2418778265":"2024-04-26T14:06:00.222915+0000","172.21.15.86:6834/1732615622":"2024-04-26T14:06:00.230367+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:06:15.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.944+0000 7f4434600640 1 -- 172.21.15.86:0/3898138023 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f440c06f520 msgr2=0x7f440c071a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:06:15.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.944+0000 7f4434600640 1 --2- 172.21.15.86:0/3898138023 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f440c06f520 0x7f440c071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:15.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.944+0000 7f4434600640 1 -- 172.21.15.86:0/3898138023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f442c0cd9e0 msgr2=0x7f442c158860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.944+0000 7f4434600640 1 --2- 172.21.15.86:0/3898138023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f442c0cd9e0 0x7f442c158860 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f44240af040 tx=0x7f44240197d0 comp rx=0 tx=0).stop 2024-04-25T14:06:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.944+0000 7f4434600640 1 -- 172.21.15.86:0/3898138023 shutdown_connections 2024-04-25T14:06:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.944+0000 7f4434600640 1 --2- 172.21.15.86:0/3898138023 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f440c06f520 0x7f440c071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.944+0000 7f4434600640 1 --2- 172.21.15.86:0/3898138023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f442c0cd9e0 0x7f442c158860 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.944+0000 7f4434600640 1 --2- 172.21.15.86:0/3898138023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f442c0cd090 0x7f442c158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.944+0000 7f4434600640 1 --2- 172.21.15.86:0/3898138023 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f442c0b3740 0x7f442c14c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.944+0000 7f4434600640 1 -- 172.21.15.86:0/3898138023 >> 172.21.15.86:0/3898138023 conn(0x7f442c09ba30 msgr2=0x7f442c06fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.944+0000 7f4434600640 1 -- 172.21.15.86:0/3898138023 shutdown_connections 2024-04-25T14:06:15.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:15.944+0000 7f4434600640 1 -- 172.21.15.86:0/3898138023 wait complete. 2024-04-25T14:06:15.968 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'fs1_metadata', 'create_time': '2024-04-25T14:05:30.939601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'fs1'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'fs1_data', 'create_time': '2024-04-25T14:05:31.898262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'fs1'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'fs2_metadata', 'create_time': '2024-04-25T14:05:52.841764+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'fs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'fs2_data', 'create_time': '2024-04-25T14:05:53.524454+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'fs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:06:15.969 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:06:15.976 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:06:15.977+0000 7f95af866180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:06:16.002 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:16.000+0000 7fc57b0d6180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:06:16.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.120+0000 7f9069600640 1 Processor -- start 2024-04-25T14:06:16.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.124+0000 7f9069600640 1 -- start start 2024-04-25T14:06:16.125 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:06:16.125+0000 7f95af866180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:06:16.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.124+0000 7f9069600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f90640d2c80 0x7f906415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:16.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.124+0000 7f9069600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9064070e20 0x7f9064071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:16.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.124+0000 7f9069600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9064071770 con 0x7f90640722f0 2024-04-25T14:06:16.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.124+0000 7f9069600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f90640ce720 con 0x7f90640d2c80 2024-04-25T14:06:16.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.124+0000 7f9069600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f90640cd8d0 con 0x7f9064070e20 2024-04-25T14:06:16.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.124+0000 7f9062a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f90640d2c80 0x7f906415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:16.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.124+0000 7f9062a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f90640d2c80 0x7f906415dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47422/0 (socket says 172.21.15.86:47422) 2024-04-25T14:06:16.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.124+0000 7f9062a00640 1 -- 172.21.15.86:0/3887431722 learned_addr learned my addr 172.21.15.86:0/3887431722 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:16.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.124+0000 7f905be00640 1 --2- 172.21.15.86:0/3887431722 >> v2:172.21.15.155:3300/0 conn(0x7f9064070e20 0x7f9064071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:16.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.124+0000 7f9062a00640 1 -- 172.21.15.86:0/3887431722 >> v2:172.21.15.155:3300/0 conn(0x7f9064070e20 msgr2=0x7f9064071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:16.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.124+0000 7f9062a00640 1 --2- 172.21.15.86:0/3887431722 >> v2:172.21.15.155:3300/0 conn(0x7f9064070e20 0x7f9064071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:16.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.124+0000 7f9062a00640 1 -- 172.21.15.86:0/3887431722 >> v1:172.21.15.155:6789/0 conn(0x7f90640722f0 legacy=0x7f9064072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:16.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.124+0000 7f9062a00640 1 -- 172.21.15.86:0/3887431722 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90640ce970 con 0x7f90640d2c80 2024-04-25T14:06:16.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.128+0000 7f9062a00640 1 --2- 172.21.15.86:0/3887431722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90640d2c80 0x7f906415dc90 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f90640b6070 tx=0x7f905401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf3af8b1145802d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:16.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.128+0000 7f9062000640 1 -- 172.21.15.86:0/3887431722 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90540b5070 con 0x7f90640d2c80 2024-04-25T14:06:16.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.128+0000 7f9062000640 1 -- 172.21.15.86:0/3887431722 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90540a8a10 con 0x7f90640d2c80 2024-04-25T14:06:16.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.128+0000 7f9062000640 1 -- 172.21.15.86:0/3887431722 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90540a8de0 con 0x7f90640d2c80 2024-04-25T14:06:16.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.128+0000 7f9069600640 1 -- 172.21.15.86:0/3887431722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90640d2c80 msgr2=0x7f906415dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:16.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.128+0000 7f9069600640 1 --2- 172.21.15.86:0/3887431722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90640d2c80 0x7f906415dc90 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f90640b6070 tx=0x7f905401ed10 comp rx=0 tx=0).stop 2024-04-25T14:06:16.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.128+0000 7f9069600640 1 -- 172.21.15.86:0/3887431722 shutdown_connections 2024-04-25T14:06:16.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.128+0000 7f9069600640 1 --2- 172.21.15.86:0/3887431722 >> v2:172.21.15.155:3300/0 conn(0x7f9064070e20 0x7f9064071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:16.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.128+0000 7f9069600640 1 --2- 172.21.15.86:0/3887431722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90640d2c80 0x7f906415dc90 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:16.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.128+0000 7f9069600640 1 -- 172.21.15.86:0/3887431722 >> 172.21.15.86:0/3887431722 conn(0x7f90640cffc0 msgr2=0x7f90640d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:16.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.128+0000 7f9069600640 1 -- 172.21.15.86:0/3887431722 shutdown_connections 2024-04-25T14:06:16.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.128+0000 7f9069600640 1 -- 172.21.15.86:0/3887431722 wait complete. 2024-04-25T14:06:16.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.132+0000 7f9069600640 1 Processor -- start 2024-04-25T14:06:16.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.132+0000 7f9069600640 1 -- start start 2024-04-25T14:06:16.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.132+0000 7f9069600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9064070e20 0x7f90641534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:16.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.132+0000 7f9069600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90640722f0 0x7f90641e8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:16.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.132+0000 7f9069600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90640d2c80 0x7f9064156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:16.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.132+0000 7f9069600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f90640d21a0 con 0x7f90640d2c80 2024-04-25T14:06:16.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.132+0000 7f9069600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f90640d1ec0 con 0x7f90640722f0 2024-04-25T14:06:16.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.132+0000 7f9069600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f90640d2480 con 0x7f9064070e20 2024-04-25T14:06:16.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.132+0000 7f9063400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90640d2c80 0x7f9064156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:16.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.132+0000 7f9062a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9064070e20 0x7f90641534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:16.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.132+0000 7f905be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90640722f0 0x7f90641e8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:16.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.132+0000 7f9063400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90640d2c80 0x7f9064156610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47426/0 (socket says 172.21.15.86:47426) 2024-04-25T14:06:16.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.132+0000 7f9063400640 1 -- 172.21.15.86:0/503645750 learned_addr learned my addr 172.21.15.86:0/503645750 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:16.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.132+0000 7f9063400640 1 -- 172.21.15.86:0/503645750 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9064070e20 msgr2=0x7f90641534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:16.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.132+0000 7f9063400640 1 --2- 172.21.15.86:0/503645750 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9064070e20 0x7f90641534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:16.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.132+0000 7f9063400640 1 -- 172.21.15.86:0/503645750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90640722f0 msgr2=0x7f90641e8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:16.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.132+0000 7f9063400640 1 --2- 172.21.15.86:0/503645750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90640722f0 0x7f90641e8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:16.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.132+0000 7f9063400640 1 -- 172.21.15.86:0/503645750 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90641e6f50 con 0x7f90640d2c80 2024-04-25T14:06:16.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.132+0000 7f905be00640 1 --2- 172.21.15.86:0/503645750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90640722f0 0x7f90641e8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:16.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.132+0000 7f9063400640 1 --2- 172.21.15.86:0/503645750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90640d2c80 0x7f9064156610 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f905c00c760 tx=0x7f905c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:16.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.132+0000 7f905aa00640 1 -- 172.21.15.86:0/503645750 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c003b80 con 0x7f90640d2c80 2024-04-25T14:06:16.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.132+0000 7f905aa00640 1 -- 172.21.15.86:0/503645750 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f905c003d20 con 0x7f90640d2c80 2024-04-25T14:06:16.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.132+0000 7f9069600640 1 -- 172.21.15.86:0/503645750 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90641e7230 con 0x7f90640d2c80 2024-04-25T14:06:16.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.132+0000 7f905aa00640 1 -- 172.21.15.86:0/503645750 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c00aa90 con 0x7f90640d2c80 2024-04-25T14:06:16.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.132+0000 7f9069600640 1 -- 172.21.15.86:0/503645750 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90641560f0 con 0x7f90640d2c80 2024-04-25T14:06:16.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.136+0000 7f9069600640 1 -- 172.21.15.86:0/503645750 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9064160a40 con 0x7f90640d2c80 2024-04-25T14:06:16.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.136+0000 7f905aa00640 1 -- 172.21.15.86:0/503645750 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) ==== 93309+0+0 (secure 0 0 0) 0x7f905c00ac30 con 0x7f90640d2c80 2024-04-25T14:06:16.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.136+0000 7f905aa00640 1 --2- 172.21.15.86:0/503645750 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f904006f550 0x7f9040071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:16.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.136+0000 7f905aa00640 1 -- 172.21.15.86:0/503645750 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7f905c08f800 con 0x7f90640d2c80 2024-04-25T14:06:16.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.136+0000 7f9062a00640 1 -- 172.21.15.86:0/503645750 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f904006f550 msgr2=0x7f9040071a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2427179681 2024-04-25T14:06:16.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.136+0000 7f9062a00640 1 --2- 172.21.15.86:0/503645750 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f904006f550 0x7f9040071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:06:16.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.140+0000 7f905aa00640 1 -- 172.21.15.86:0/503645750 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9064071770 con 0x7f90640d2c80 2024-04-25T14:06:16.158 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:16.156+0000 7fc57b0d6180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:06:16.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.300+0000 7f9069600640 1 -- 172.21.15.86:0/503645750 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f906415f6b0 con 0x7f90640d2c80 2024-04-25T14:06:16.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.300+0000 7f905aa00640 1 -- 172.21.15.86:0/503645750 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) ==== 93+0+10 (secure 0 0 0) 0x7f905c05b900 con 0x7f90640d2c80 2024-04-25T14:06:16.301 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:06:16.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.300+0000 7f9069600640 1 -- 172.21.15.86:0/503645750 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f904006f550 msgr2=0x7f9040071a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:06:16.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.300+0000 7f9069600640 1 --2- 172.21.15.86:0/503645750 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f904006f550 0x7f9040071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:16.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.300+0000 7f9069600640 1 -- 172.21.15.86:0/503645750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90640d2c80 msgr2=0x7f9064156610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:16.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.300+0000 7f9069600640 1 --2- 172.21.15.86:0/503645750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90640d2c80 0x7f9064156610 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f905c00c760 tx=0x7f905c00cc20 comp rx=0 tx=0).stop 2024-04-25T14:06:16.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.300+0000 7f9069600640 1 -- 172.21.15.86:0/503645750 shutdown_connections 2024-04-25T14:06:16.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.300+0000 7f9069600640 1 --2- 172.21.15.86:0/503645750 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f904006f550 0x7f9040071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:16.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.300+0000 7f9069600640 1 --2- 172.21.15.86:0/503645750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90640d2c80 0x7f9064156610 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:16.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.300+0000 7f9069600640 1 --2- 172.21.15.86:0/503645750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90640722f0 0x7f90641e8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:16.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.300+0000 7f9069600640 1 --2- 172.21.15.86:0/503645750 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9064070e20 0x7f90641534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:16.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.300+0000 7f9069600640 1 -- 172.21.15.86:0/503645750 >> 172.21.15.86:0/503645750 conn(0x7f90640cffc0 msgr2=0x7f90640b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:16.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.300+0000 7f9069600640 1 -- 172.21.15.86:0/503645750 shutdown_connections 2024-04-25T14:06:16.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.300+0000 7f9069600640 1 -- 172.21.15.86:0/503645750 wait complete. 2024-04-25T14:06:16.324 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get fs1_metadata pg_num 2024-04-25T14:06:16.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.520+0000 7ff7fcc00640 1 Processor -- start 2024-04-25T14:06:16.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.520+0000 7ff7fcc00640 1 -- start start 2024-04-25T14:06:16.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.520+0000 7ff7fcc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7f806e9b0 0x7ff7f815ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:16.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.520+0000 7ff7fcc00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff7f80d2360 0x7ff7f80d2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:16.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.524+0000 7ff7fcc00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7f806e7c0 con 0x7ff7f80cde60 2024-04-25T14:06:16.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.524+0000 7ff7fcc00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff7f80d2cb0 con 0x7ff7f80d2360 2024-04-25T14:06:16.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.524+0000 7ff7fcc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff7f806c710 con 0x7ff7f806e9b0 2024-04-25T14:06:16.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.524+0000 7ff7f5e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7f806e9b0 0x7ff7f815ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:16.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.524+0000 7ff7f5400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff7f80d2360 0x7ff7f80d2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:16.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.524+0000 7ff7f5e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7f806e9b0 0x7ff7f815ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:46186/0 (socket says 172.21.15.86:46186) 2024-04-25T14:06:16.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.524+0000 7ff7f5e00640 1 -- 172.21.15.86:0/3294097659 learned_addr learned my addr 172.21.15.86:0/3294097659 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:16.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.524+0000 7ff7f6800640 1 --1- 172.21.15.86:0/3294097659 >> v1:172.21.15.155:6789/0 conn(0x7ff7f80cde60 0x7ff7f80ce270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:43332/0 (socket says 172.21.15.86:43332) 2024-04-25T14:06:16.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.524+0000 7ff7f5e00640 1 -- 172.21.15.86:0/3294097659 >> v2:172.21.15.155:3300/0 conn(0x7ff7f80d2360 msgr2=0x7ff7f80d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:16.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.524+0000 7ff7f5e00640 1 --2- 172.21.15.86:0/3294097659 >> v2:172.21.15.155:3300/0 conn(0x7ff7f80d2360 0x7ff7f80d2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:16.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.524+0000 7ff7f5e00640 1 -- 172.21.15.86:0/3294097659 >> v1:172.21.15.155:6789/0 conn(0x7ff7f80cde60 legacy=0x7ff7f80ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:16.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.524+0000 7ff7f5e00640 1 -- 172.21.15.86:0/3294097659 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f806c9f0 con 0x7ff7f806e9b0 2024-04-25T14:06:16.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.524+0000 7ff7f5400640 1 --2- 172.21.15.86:0/3294097659 >> v2:172.21.15.155:3300/0 conn(0x7ff7f80d2360 0x7ff7f80d2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:16.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.528+0000 7ff7f5e00640 1 --2- 172.21.15.86:0/3294097659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7f806e9b0 0x7ff7f815ddf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009870 tx=0x7ff7e801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1b10b0a35d6b678 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:16.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.528+0000 7ff7f4a00640 1 -- 172.21.15.86:0/3294097659 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e80a7070 con 0x7ff7f806e9b0 2024-04-25T14:06:16.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.528+0000 7ff7f4a00640 1 -- 172.21.15.86:0/3294097659 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7e80b0070 con 0x7ff7f806e9b0 2024-04-25T14:06:16.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.528+0000 7ff7f4a00640 1 -- 172.21.15.86:0/3294097659 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e80ab890 con 0x7ff7f806e9b0 2024-04-25T14:06:16.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.528+0000 7ff7fcc00640 1 -- 172.21.15.86:0/3294097659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7f806e9b0 msgr2=0x7ff7f815ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:16.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.528+0000 7ff7fcc00640 1 --2- 172.21.15.86:0/3294097659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7f806e9b0 0x7ff7f815ddf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009870 tx=0x7ff7e801c560 comp rx=0 tx=0).stop 2024-04-25T14:06:16.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.528+0000 7ff7fcc00640 1 -- 172.21.15.86:0/3294097659 shutdown_connections 2024-04-25T14:06:16.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.528+0000 7ff7fcc00640 1 --2- 172.21.15.86:0/3294097659 >> v2:172.21.15.155:3300/0 conn(0x7ff7f80d2360 0x7ff7f80d2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:16.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.528+0000 7ff7fcc00640 1 --2- 172.21.15.86:0/3294097659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7f806e9b0 0x7ff7f815ddf0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:16.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.528+0000 7ff7fcc00640 1 -- 172.21.15.86:0/3294097659 >> 172.21.15.86:0/3294097659 conn(0x7ff7f809ba30 msgr2=0x7ff7f80b7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:16.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.528+0000 7ff7fcc00640 1 -- 172.21.15.86:0/3294097659 shutdown_connections 2024-04-25T14:06:16.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.528+0000 7ff7fcc00640 1 -- 172.21.15.86:0/3294097659 wait complete. 2024-04-25T14:06:16.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.528+0000 7ff7fcc00640 1 Processor -- start 2024-04-25T14:06:16.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.528+0000 7ff7fcc00640 1 -- start start 2024-04-25T14:06:16.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.532+0000 7ff7fcc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7f806e9b0 0x7ff7f81f9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:16.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.532+0000 7ff7fcc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7f80cde60 0x7ff7f815faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:16.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.532+0000 7ff7fcc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7f80d2360 0x7ff7f815ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:16.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.532+0000 7ff7fcc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f806f110 con 0x7ff7f806e9b0 2024-04-25T14:06:16.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.532+0000 7ff7fcc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f806ef90 con 0x7ff7f80d2360 2024-04-25T14:06:16.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.532+0000 7ff7fcc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff7f815e330 con 0x7ff7f80cde60 2024-04-25T14:06:16.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.532+0000 7ff7f5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7f806e9b0 0x7ff7f81f9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:16.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.532+0000 7ff7f5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7f806e9b0 0x7ff7f81f9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47432/0 (socket says 172.21.15.86:47432) 2024-04-25T14:06:16.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.532+0000 7ff7f5e00640 1 -- 172.21.15.86:0/3658847701 learned_addr learned my addr 172.21.15.86:0/3658847701 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:16.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.532+0000 7ff7f5400640 1 --2- 172.21.15.86:0/3658847701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7f80cde60 0x7ff7f815faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:16.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.532+0000 7ff7f6800640 1 --2- 172.21.15.86:0/3658847701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7f80d2360 0x7ff7f815ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:16.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.532+0000 7ff7f5e00640 1 -- 172.21.15.86:0/3658847701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7f80cde60 msgr2=0x7ff7f815faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:16.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.532+0000 7ff7f5e00640 1 --2- 172.21.15.86:0/3658847701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7f80cde60 0x7ff7f815faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:16.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.532+0000 7ff7f5e00640 1 -- 172.21.15.86:0/3658847701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7f80d2360 msgr2=0x7ff7f815ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:16.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.532+0000 7ff7f5e00640 1 --2- 172.21.15.86:0/3658847701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7f80d2360 0x7ff7f815ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:16.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.532+0000 7ff7f5e00640 1 -- 172.21.15.86:0/3658847701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f8152660 con 0x7ff7f806e9b0 2024-04-25T14:06:16.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.532+0000 7ff7f5e00640 1 --2- 172.21.15.86:0/3658847701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7f806e9b0 0x7ff7f81f9590 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff7e80ad040 tx=0x7ff7e801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:16.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.532+0000 7ff7ee000640 1 -- 172.21.15.86:0/3658847701 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e80be5f0 con 0x7ff7f806e9b0 2024-04-25T14:06:16.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.532+0000 7ff7ee000640 1 -- 172.21.15.86:0/3658847701 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7e80be790 con 0x7ff7f806e9b0 2024-04-25T14:06:16.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.532+0000 7ff7fcc00640 1 -- 172.21.15.86:0/3658847701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f8152940 con 0x7ff7f806e9b0 2024-04-25T14:06:16.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.532+0000 7ff7ee000640 1 -- 172.21.15.86:0/3658847701 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e80beb80 con 0x7ff7f806e9b0 2024-04-25T14:06:16.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.532+0000 7ff7fcc00640 1 -- 172.21.15.86:0/3658847701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f80c6200 con 0x7ff7f806e9b0 2024-04-25T14:06:16.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.532+0000 7ff7fcc00640 1 -- 172.21.15.86:0/3658847701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7f80a05a0 con 0x7ff7f806e9b0 2024-04-25T14:06:16.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.536+0000 7ff7ee000640 1 -- 172.21.15.86:0/3658847701 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) ==== 93309+0+0 (secure 0 0 0) 0x7ff7e80b43f0 con 0x7ff7f806e9b0 2024-04-25T14:06:16.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.536+0000 7ff7ee000640 1 --2- 172.21.15.86:0/3658847701 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7ff7d806f520 0x7ff7d8071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:16.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.536+0000 7ff7ee000640 1 -- 172.21.15.86:0/3658847701 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7ff7e812a4a0 con 0x7ff7f806e9b0 2024-04-25T14:06:16.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.536+0000 7ff7f5400640 1 -- 172.21.15.86:0/3658847701 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7ff7d806f520 msgr2=0x7ff7d8071a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2427179681 2024-04-25T14:06:16.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.536+0000 7ff7f5400640 1 --2- 172.21.15.86:0/3658847701 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7ff7d806f520 0x7ff7d8071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:06:16.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.540+0000 7ff7ee000640 1 -- 172.21.15.86:0/3658847701 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff7e8016150 con 0x7ff7f806e9b0 2024-04-25T14:06:16.596 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:06:16.597+0000 7f95af866180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:06:16.650 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:16.648+0000 7fc57b0d6180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:06:16.654 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:06:16.653+0000 7f95af866180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:06:16.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.700+0000 7ff7fcc00640 1 -- 172.21.15.86:0/3658847701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "fs1_metadata", "var": "pg_num"} v 0) -- 0x7ff7f8076bf0 con 0x7ff7f806e9b0 2024-04-25T14:06:16.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.700+0000 7ff7ee000640 1 -- 172.21.15.86:0/3658847701 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "fs1_metadata", "var": "pg_num"}]=0 v58) ==== 101+0+11 (secure 0 0 0) 0x7ff7e80f6400 con 0x7ff7f806e9b0 2024-04-25T14:06:16.703 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:06:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.704+0000 7ff7fcc00640 1 -- 172.21.15.86:0/3658847701 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7ff7d806f520 msgr2=0x7ff7d8071a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:06:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.704+0000 7ff7fcc00640 1 --2- 172.21.15.86:0/3658847701 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7ff7d806f520 0x7ff7d8071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.704+0000 7ff7fcc00640 1 -- 172.21.15.86:0/3658847701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7f806e9b0 msgr2=0x7ff7f81f9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.704+0000 7ff7fcc00640 1 --2- 172.21.15.86:0/3658847701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7f806e9b0 0x7ff7f81f9590 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff7e80ad040 tx=0x7ff7e801ada0 comp rx=0 tx=0).stop 2024-04-25T14:06:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.704+0000 7ff7fcc00640 1 -- 172.21.15.86:0/3658847701 shutdown_connections 2024-04-25T14:06:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.704+0000 7ff7fcc00640 1 --2- 172.21.15.86:0/3658847701 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7ff7d806f520 0x7ff7d8071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.704+0000 7ff7fcc00640 1 --2- 172.21.15.86:0/3658847701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7f80d2360 0x7ff7f815ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:16.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.704+0000 7ff7fcc00640 1 --2- 172.21.15.86:0/3658847701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7f80cde60 0x7ff7f815faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:16.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.704+0000 7ff7fcc00640 1 --2- 172.21.15.86:0/3658847701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7f806e9b0 0x7ff7f81f9590 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:16.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.704+0000 7ff7fcc00640 1 -- 172.21.15.86:0/3658847701 >> 172.21.15.86:0/3658847701 conn(0x7ff7f809ba30 msgr2=0x7ff7f8155a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:16.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.704+0000 7ff7fcc00640 1 -- 172.21.15.86:0/3658847701 shutdown_connections 2024-04-25T14:06:16.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.704+0000 7ff7fcc00640 1 -- 172.21.15.86:0/3658847701 wait complete. 2024-04-25T14:06:16.710 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:06:16.709+0000 7f95af866180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:06:16.711 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:16.708+0000 7fc57b0d6180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:06:16.726 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get fs1_data pg_num 2024-04-25T14:06:16.770 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:16.768+0000 7fc57b0d6180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:06:16.798 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:06:16.797+0000 7f95af866180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:06:16.855 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:06:16.853+0000 7f95af866180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:06:16.862 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:16.860+0000 7fc57b0d6180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:06:16.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.880+0000 7f71da600640 1 Processor -- start 2024-04-25T14:06:16.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.880+0000 7f71da600640 1 -- start start 2024-04-25T14:06:16.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.884+0000 7f71da600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71d40d2c80 0x7f71d415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:16.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.884+0000 7f71da600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f71d4070e20 0x7f71d4071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:16.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.884+0000 7f71da600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71d4071770 con 0x7f71d40722f0 2024-04-25T14:06:16.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.884+0000 7f71da600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f71d40ce720 con 0x7f71d4070e20 2024-04-25T14:06:16.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.884+0000 7f71da600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f71d40cd8d0 con 0x7f71d40d2c80 2024-04-25T14:06:16.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.884+0000 7f71d2a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f71d4070e20 0x7f71d4071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:16.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.884+0000 7f71d2a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f71d4070e20 0x7f71d4071230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47436/0 (socket says 172.21.15.86:47436) 2024-04-25T14:06:16.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.884+0000 7f71d2a00640 1 -- 172.21.15.86:0/3680597714 learned_addr learned my addr 172.21.15.86:0/3680597714 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:16.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.884+0000 7f71d3400640 1 --2- 172.21.15.86:0/3680597714 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71d40d2c80 0x7f71d415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:16.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.884+0000 7f71d2a00640 1 -- 172.21.15.86:0/3680597714 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71d40d2c80 msgr2=0x7f71d415dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:16.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.884+0000 7f71d2a00640 1 --2- 172.21.15.86:0/3680597714 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71d40d2c80 0x7f71d415dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:16.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.884+0000 7f71d2a00640 1 -- 172.21.15.86:0/3680597714 >> v1:172.21.15.155:6789/0 conn(0x7f71d40722f0 legacy=0x7f71d4072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:16.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.884+0000 7f71d2a00640 1 -- 172.21.15.86:0/3680597714 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d40ce970 con 0x7f71d4070e20 2024-04-25T14:06:16.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.888+0000 7f71d2a00640 1 --2- 172.21.15.86:0/3680597714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71d4070e20 0x7f71d4071230 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f71c0006fd0 tx=0x7f71c001ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18d8fe40390ab58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:16.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.888+0000 7f71d2000640 1 -- 172.21.15.86:0/3680597714 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c00b5070 con 0x7f71d4070e20 2024-04-25T14:06:16.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.888+0000 7f71d2000640 1 -- 172.21.15.86:0/3680597714 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71c00a8a30 con 0x7f71d4070e20 2024-04-25T14:06:16.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.888+0000 7f71d2000640 1 -- 172.21.15.86:0/3680597714 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c00a8e00 con 0x7f71d4070e20 2024-04-25T14:06:16.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.888+0000 7f71da600640 1 -- 172.21.15.86:0/3680597714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71d4070e20 msgr2=0x7f71d4071230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:16.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.888+0000 7f71da600640 1 --2- 172.21.15.86:0/3680597714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71d4070e20 0x7f71d4071230 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f71c0006fd0 tx=0x7f71c001ed30 comp rx=0 tx=0).stop 2024-04-25T14:06:16.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.888+0000 7f71da600640 1 -- 172.21.15.86:0/3680597714 shutdown_connections 2024-04-25T14:06:16.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.888+0000 7f71da600640 1 --2- 172.21.15.86:0/3680597714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71d4070e20 0x7f71d4071230 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:16.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.888+0000 7f71da600640 1 --2- 172.21.15.86:0/3680597714 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71d40d2c80 0x7f71d415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:16.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.888+0000 7f71da600640 1 -- 172.21.15.86:0/3680597714 >> 172.21.15.86:0/3680597714 conn(0x7f71d40cffc0 msgr2=0x7f71d40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:16.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.888+0000 7f71da600640 1 -- 172.21.15.86:0/3680597714 shutdown_connections 2024-04-25T14:06:16.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.888+0000 7f71da600640 1 -- 172.21.15.86:0/3680597714 wait complete. 2024-04-25T14:06:16.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.888+0000 7f71da600640 1 Processor -- start 2024-04-25T14:06:16.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.888+0000 7f71da600640 1 -- start start 2024-04-25T14:06:16.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.892+0000 7f71da600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71d4070e20 0x7f71d4158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:16.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.892+0000 7f71da600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71d40722f0 0x7f71d41f6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:16.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.892+0000 7f71da600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71d40d2c80 0x7f71d40b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:16.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.892+0000 7f71da600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71d40d21a0 con 0x7f71d40722f0 2024-04-25T14:06:16.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.892+0000 7f71da600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f71d40d1ec0 con 0x7f71d4070e20 2024-04-25T14:06:16.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.892+0000 7f71da600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f71d40d2480 con 0x7f71d40d2c80 2024-04-25T14:06:16.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.892+0000 7f71d2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71d40722f0 0x7f71d41f6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:16.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.892+0000 7f71d3400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71d4070e20 0x7f71d4158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:16.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.892+0000 7f71d3400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71d4070e20 0x7f71d4158ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:52564/0 (socket says 172.21.15.86:52564) 2024-04-25T14:06:16.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.892+0000 7f71d3400640 1 -- 172.21.15.86:0/1546442623 learned_addr learned my addr 172.21.15.86:0/1546442623 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:16.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.892+0000 7f71d3e00640 1 --2- 172.21.15.86:0/1546442623 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71d40d2c80 0x7f71d40b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:16.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.892+0000 7f71d3400640 1 -- 172.21.15.86:0/1546442623 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71d40d2c80 msgr2=0x7f71d40b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:16.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.892+0000 7f71d3400640 1 --2- 172.21.15.86:0/1546442623 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71d40d2c80 0x7f71d40b16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:16.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.892+0000 7f71d3400640 1 -- 172.21.15.86:0/1546442623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71d40722f0 msgr2=0x7f71d41f6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:16.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.892+0000 7f71d3400640 1 --2- 172.21.15.86:0/1546442623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71d40722f0 0x7f71d41f6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:16.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.892+0000 7f71d3400640 1 -- 172.21.15.86:0/1546442623 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d41f7430 con 0x7f71d4070e20 2024-04-25T14:06:16.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.892+0000 7f71d3400640 1 --2- 172.21.15.86:0/1546442623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71d4070e20 0x7f71d4158ee0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f71c400e8d0 tx=0x7f71c400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:16.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.892+0000 7f71cb400640 1 -- 172.21.15.86:0/1546442623 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4017070 con 0x7f71d4070e20 2024-04-25T14:06:16.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.892+0000 7f71cb400640 1 -- 172.21.15.86:0/1546442623 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71c400c940 con 0x7f71d4070e20 2024-04-25T14:06:16.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.892+0000 7f71cb400640 1 -- 172.21.15.86:0/1546442623 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c400ccf0 con 0x7f71d4070e20 2024-04-25T14:06:16.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.892+0000 7f71da600640 1 -- 172.21.15.86:0/1546442623 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d40afbf0 con 0x7f71d4070e20 2024-04-25T14:06:16.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.892+0000 7f71da600640 1 -- 172.21.15.86:0/1546442623 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d406e140 con 0x7f71d4070e20 2024-04-25T14:06:16.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.896+0000 7f71da600640 1 -- 172.21.15.86:0/1546442623 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71d40720e0 con 0x7f71d4070e20 2024-04-25T14:06:16.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.896+0000 7f71cb400640 1 -- 172.21.15.86:0/1546442623 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 12) ==== 93309+0+0 (secure 0 0 0) 0x7f71c401a070 con 0x7f71d4070e20 2024-04-25T14:06:16.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.896+0000 7f71cb400640 1 --2- 172.21.15.86:0/1546442623 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f71b406f520 0x7f71b4071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:16.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.896+0000 7f71cb400640 1 -- 172.21.15.86:0/1546442623 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7f71c408f8e0 con 0x7f71d4070e20 2024-04-25T14:06:16.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.896+0000 7f71d2a00640 1 -- 172.21.15.86:0/1546442623 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f71b406f520 msgr2=0x7f71b4071a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2427179681 2024-04-25T14:06:16.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.896+0000 7f71d2a00640 1 --2- 172.21.15.86:0/1546442623 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f71b406f520 0x7f71b4071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:06:16.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:16.900+0000 7f71cb400640 1 -- 172.21.15.86:0/1546442623 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f71d41f7430 con 0x7f71d4070e20 2024-04-25T14:06:16.937 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:16.936+0000 7fc57b0d6180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:06:16.941 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:06:16.941+0000 7f95af866180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:06:17.001 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:06:17.001+0000 7f95af866180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:06:17.023 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:17.020+0000 7fc57b0d6180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:06:17.083 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:17.080+0000 7fc57b0d6180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:06:17.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.088+0000 7f71da600640 1 -- 172.21.15.86:0/1546442623 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "fs1_data", "var": "pg_num"} v 0) -- 0x7f71d4076bf0 con 0x7f71d4070e20 2024-04-25T14:06:17.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.088+0000 7f71cb400640 1 -- 172.21.15.86:0/1546442623 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "fs1_data", "var": "pg_num"}]=0 v58) ==== 97+0+11 (secure 0 0 0) 0x7f71c405b9d0 con 0x7f71d4070e20 2024-04-25T14:06:17.089 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:06:17.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.088+0000 7f71da600640 1 -- 172.21.15.86:0/1546442623 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f71b406f520 msgr2=0x7f71b4071a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:06:17.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.088+0000 7f71da600640 1 --2- 172.21.15.86:0/1546442623 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f71b406f520 0x7f71b4071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:17.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.088+0000 7f71da600640 1 -- 172.21.15.86:0/1546442623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71d4070e20 msgr2=0x7f71d4158ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:17.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.088+0000 7f71da600640 1 --2- 172.21.15.86:0/1546442623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71d4070e20 0x7f71d4158ee0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f71c400e8d0 tx=0x7f71c400ecd0 comp rx=0 tx=0).stop 2024-04-25T14:06:17.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.088+0000 7f71da600640 1 -- 172.21.15.86:0/1546442623 shutdown_connections 2024-04-25T14:06:17.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.088+0000 7f71da600640 1 --2- 172.21.15.86:0/1546442623 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f71b406f520 0x7f71b4071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:17.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.088+0000 7f71da600640 1 --2- 172.21.15.86:0/1546442623 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71d40d2c80 0x7f71d40b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:17.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.088+0000 7f71da600640 1 --2- 172.21.15.86:0/1546442623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71d40722f0 0x7f71d41f6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:17.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.088+0000 7f71da600640 1 --2- 172.21.15.86:0/1546442623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71d4070e20 0x7f71d4158ee0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:17.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.088+0000 7f71da600640 1 -- 172.21.15.86:0/1546442623 >> 172.21.15.86:0/1546442623 conn(0x7f71d40cffc0 msgr2=0x7f71d4160ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:17.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.088+0000 7f71da600640 1 -- 172.21.15.86:0/1546442623 shutdown_connections 2024-04-25T14:06:17.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.088+0000 7f71da600640 1 -- 172.21.15.86:0/1546442623 wait complete. 2024-04-25T14:06:17.112 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get fs2_metadata pg_num 2024-04-25T14:06:17.152 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:06:17.153+0000 7f95af866180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:06:17.211 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:06:17.209+0000 7f95af866180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:06:17.242 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:17.240+0000 7fc57b0d6180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:06:17.303 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:17.300+0000 7fc57b0d6180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:06:17.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.304+0000 7fcee8600640 1 Processor -- start 2024-04-25T14:06:17.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.304+0000 7fcee8600640 1 -- start start 2024-04-25T14:06:17.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.308+0000 7fcee8600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcee00cfca0 0x7fcee006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:17.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.308+0000 7fcee8600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcee0072bd0 con 0x7fcee00d16e0 2024-04-25T14:06:17.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.308+0000 7fcee8600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcee006f930 con 0x7fcee00730b0 2024-04-25T14:06:17.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.308+0000 7fcee8600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcee0070470 con 0x7fcee00cfca0 2024-04-25T14:06:17.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.308+0000 7fcee5200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fcee00d16e0 0x7fcee00d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59804/0 (socket says 172.21.15.86:59804) 2024-04-25T14:06:17.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.308+0000 7fcee5200640 1 -- 172.21.15.86:0/2209120651 learned_addr learned my addr 172.21.15.86:0/2209120651 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:17.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.308+0000 7fcee5c00640 1 --2- 172.21.15.86:0/2209120651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcee00cfca0 0x7fcee006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:17.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.308+0000 7fcedfe00640 1 -- 172.21.15.86:0/2209120651 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3297654007 0 0) 0x7fcee0072bd0 con 0x7fcee00d16e0 2024-04-25T14:06:17.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.308+0000 7fcedfe00640 1 -- 172.21.15.86:0/2209120651 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcec8005740 con 0x7fcee00d16e0 2024-04-25T14:06:17.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.308+0000 7fcedfe00640 1 -- 172.21.15.86:0/2209120651 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2732929733 0 0) 0x7fcee006f930 con 0x7fcee00730b0 2024-04-25T14:06:17.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.308+0000 7fcee5c00640 1 -- 172.21.15.86:0/2209120651 >> v1:172.21.15.155:6789/0 conn(0x7fcee00730b0 legacy=0x7fcee00734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:17.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.308+0000 7fcee5c00640 1 -- 172.21.15.86:0/2209120651 >> v1:172.21.15.86:6789/0 conn(0x7fcee00d16e0 legacy=0x7fcee00d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:17.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.308+0000 7fcee5c00640 1 -- 172.21.15.86:0/2209120651 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee006fbd0 con 0x7fcee00cfca0 2024-04-25T14:06:17.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.312+0000 7fcee5c00640 1 --2- 172.21.15.86:0/2209120651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcee00cfca0 0x7fcee006f3f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fced40097f0 tx=0x7fced401c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55a121e034eb0ad4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:17.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.312+0000 7fcedfe00640 1 -- 172.21.15.86:0/2209120651 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee00743e0 con 0x7fcee00cfca0 2024-04-25T14:06:17.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.312+0000 7fcedfe00640 1 -- 172.21.15.86:0/2209120651 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fced40b0070 con 0x7fcee00cfca0 2024-04-25T14:06:17.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.312+0000 7fcedfe00640 1 -- 172.21.15.86:0/2209120651 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced40ab7f0 con 0x7fcee00cfca0 2024-04-25T14:06:17.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.312+0000 7fcee8600640 1 -- 172.21.15.86:0/2209120651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcee00cfca0 msgr2=0x7fcee006f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:17.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.312+0000 7fcee8600640 1 --2- 172.21.15.86:0/2209120651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcee00cfca0 0x7fcee006f3f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fced40097f0 tx=0x7fced401c4e0 comp rx=0 tx=0).stop 2024-04-25T14:06:17.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.312+0000 7fcee8600640 1 -- 172.21.15.86:0/2209120651 shutdown_connections 2024-04-25T14:06:17.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.312+0000 7fcee8600640 1 --2- 172.21.15.86:0/2209120651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcee00cfca0 0x7fcee006f3f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:17.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.312+0000 7fcee8600640 1 -- 172.21.15.86:0/2209120651 >> 172.21.15.86:0/2209120651 conn(0x7fcee00b4650 msgr2=0x7fcee00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:17.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.312+0000 7fcee8600640 1 -- 172.21.15.86:0/2209120651 shutdown_connections 2024-04-25T14:06:17.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.312+0000 7fcee8600640 1 -- 172.21.15.86:0/2209120651 wait complete. 2024-04-25T14:06:17.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.316+0000 7fcee8600640 1 Processor -- start 2024-04-25T14:06:17.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.316+0000 7fcee8600640 1 -- start start 2024-04-25T14:06:17.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.316+0000 7fcee8600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcee00730b0 0x7fcee009ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:17.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.316+0000 7fcee8600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcee00cfca0 0x7fcee015f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:17.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.316+0000 7fcee8600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcee00d16e0 0x7fcee009ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:17.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.316+0000 7fcee8600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcee00ce0f0 con 0x7fcee00cfca0 2024-04-25T14:06:17.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.316+0000 7fcee8600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcee00cdf70 con 0x7fcee00730b0 2024-04-25T14:06:17.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.316+0000 7fcee8600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcee00ce270 con 0x7fcee00d16e0 2024-04-25T14:06:17.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.316+0000 7fcee5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcee00cfca0 0x7fcee015f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:17.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.316+0000 7fcee5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcee00cfca0 0x7fcee015f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47456/0 (socket says 172.21.15.86:47456) 2024-04-25T14:06:17.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.316+0000 7fcee5200640 1 -- 172.21.15.86:0/1899882010 learned_addr learned my addr 172.21.15.86:0/1899882010 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:17.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.316+0000 7fcee5c00640 1 --2- 172.21.15.86:0/1899882010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcee00730b0 0x7fcee009ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:17.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.316+0000 7fcee6600640 1 --2- 172.21.15.86:0/1899882010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcee00d16e0 0x7fcee009ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:17.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.320+0000 7fcee5200640 1 -- 172.21.15.86:0/1899882010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcee00d16e0 msgr2=0x7fcee009ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:17.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.320+0000 7fcee5200640 1 --2- 172.21.15.86:0/1899882010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcee00d16e0 0x7fcee009ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:17.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.320+0000 7fcee5200640 1 -- 172.21.15.86:0/1899882010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcee00730b0 msgr2=0x7fcee009ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:17.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.320+0000 7fcee5200640 1 --2- 172.21.15.86:0/1899882010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcee00730b0 0x7fcee009ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:17.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.320+0000 7fcee5200640 1 -- 172.21.15.86:0/1899882010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee0069970 con 0x7fcee00cfca0 2024-04-25T14:06:17.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.320+0000 7fcee5200640 1 --2- 172.21.15.86:0/1899882010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcee00cfca0 0x7fcee015f880 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fcecc00c3f0 tx=0x7fcecc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:17.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.320+0000 7fcedd600640 1 -- 172.21.15.86:0/1899882010 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc017070 con 0x7fcee00cfca0 2024-04-25T14:06:17.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.320+0000 7fcedd600640 1 -- 172.21.15.86:0/1899882010 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcecc005cf0 con 0x7fcee00cfca0 2024-04-25T14:06:17.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.320+0000 7fcee8600640 1 -- 172.21.15.86:0/1899882010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee00aef90 con 0x7fcee00cfca0 2024-04-25T14:06:17.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.320+0000 7fcedd600640 1 -- 172.21.15.86:0/1899882010 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc0072f0 con 0x7fcee00cfca0 2024-04-25T14:06:17.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.320+0000 7fcee8600640 1 -- 172.21.15.86:0/1899882010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee00af550 con 0x7fcee00cfca0 2024-04-25T14:06:17.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.320+0000 7fcee8600640 1 -- 172.21.15.86:0/1899882010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcee009f4b0 con 0x7fcee00cfca0 2024-04-25T14:06:17.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.320+0000 7fcedd600640 1 -- 172.21.15.86:0/1899882010 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) ==== 93309+0+0 (secure 0 0 0) 0x7fcecc01a070 con 0x7fcee00cfca0 2024-04-25T14:06:17.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.324+0000 7fcedd600640 1 --2- 172.21.15.86:0/1899882010 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcec006f520 0x7fcec0071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:17.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.324+0000 7fcee5c00640 1 -- 172.21.15.86:0/1899882010 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcec006f520 msgr2=0x7fcec0071a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2427179681 2024-04-25T14:06:17.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.324+0000 7fcedd600640 1 -- 172.21.15.86:0/1899882010 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7fcecc08eee0 con 0x7fcee00cfca0 2024-04-25T14:06:17.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.324+0000 7fcee5c00640 1 --2- 172.21.15.86:0/1899882010 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcec006f520 0x7fcec0071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:06:17.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.328+0000 7fcedd600640 1 -- 172.21.15.86:0/1899882010 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcecc05af40 con 0x7fcee00cfca0 2024-04-25T14:06:17.452 INFO:tasks.ceph.mgr.x.smithi155.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:06:17.452 INFO:tasks.ceph.mgr.x.smithi155.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:06:17.452 INFO:tasks.ceph.mgr.x.smithi155.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:06:17.460 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:06:17.461+0000 7f95af866180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:06:17.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.528+0000 7fcee5c00640 1 -- 172.21.15.86:0/1899882010 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcec006f520 msgr2=0x7fcec0071a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2427179681 2024-04-25T14:06:17.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.528+0000 7fcee5c00640 1 --2- 172.21.15.86:0/1899882010 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcec006f520 0x7fcec0071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:06:17.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.528+0000 7fcee8600640 1 -- 172.21.15.86:0/1899882010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "fs2_metadata", "var": "pg_num"} v 0) -- 0x7fcee0069d00 con 0x7fcee00cfca0 2024-04-25T14:06:17.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.532+0000 7fcedd600640 1 -- 172.21.15.86:0/1899882010 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "fs2_metadata", "var": "pg_num"}]=0 v58) ==== 101+0+11 (secure 0 0 0) 0x7fcecc05f240 con 0x7fcee00cfca0 2024-04-25T14:06:17.532 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:06:17.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.532+0000 7fcee8600640 1 -- 172.21.15.86:0/1899882010 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcec006f520 msgr2=0x7fcec0071a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:06:17.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.532+0000 7fcee8600640 1 --2- 172.21.15.86:0/1899882010 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcec006f520 0x7fcec0071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:17.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.532+0000 7fcee8600640 1 -- 172.21.15.86:0/1899882010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcee00cfca0 msgr2=0x7fcee015f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:17.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.532+0000 7fcee8600640 1 --2- 172.21.15.86:0/1899882010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcee00cfca0 0x7fcee015f880 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fcecc00c3f0 tx=0x7fcecc00c8b0 comp rx=0 tx=0).stop 2024-04-25T14:06:17.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.532+0000 7fcee8600640 1 -- 172.21.15.86:0/1899882010 shutdown_connections 2024-04-25T14:06:17.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.532+0000 7fcee8600640 1 --2- 172.21.15.86:0/1899882010 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fcec006f520 0x7fcec0071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:17.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.532+0000 7fcee8600640 1 --2- 172.21.15.86:0/1899882010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcee00d16e0 0x7fcee009ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:17.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.532+0000 7fcee8600640 1 --2- 172.21.15.86:0/1899882010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcee00cfca0 0x7fcee015f880 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:17.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.532+0000 7fcee8600640 1 --2- 172.21.15.86:0/1899882010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcee00730b0 0x7fcee009ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:17.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.532+0000 7fcee8600640 1 -- 172.21.15.86:0/1899882010 >> 172.21.15.86:0/1899882010 conn(0x7fcee00b4650 msgr2=0x7fcee00b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:17.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.532+0000 7fcee8600640 1 -- 172.21.15.86:0/1899882010 shutdown_connections 2024-04-25T14:06:17.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.532+0000 7fcee8600640 1 -- 172.21.15.86:0/1899882010 wait complete. 2024-04-25T14:06:17.552 INFO:tasks.ceph.mgr.y.smithi086.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:06:17.552 INFO:tasks.ceph.mgr.y.smithi086.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:06:17.552 INFO:tasks.ceph.mgr.y.smithi086.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:06:17.554 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get fs2_data pg_num 2024-04-25T14:06:17.561 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:17.560+0000 7fc57b0d6180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:06:17.607 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:06:17.609+0000 7f95af866180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:06:17.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.704+0000 7f9f8b200640 1 Processor -- start 2024-04-25T14:06:17.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.704+0000 7f9f8b200640 1 -- start start 2024-04-25T14:06:17.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.708+0000 7f9f8b200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9f84071530 0x7f9f841638a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:17.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.708+0000 7f9f8b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f84163de0 0x7f9f840cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:17.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.708+0000 7f9f8b200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f8406dc30 con 0x7f9f8415fe00 2024-04-25T14:06:17.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.708+0000 7f9f8b200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9f84160210 con 0x7f9f84071530 2024-04-25T14:06:17.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.708+0000 7f9f8b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9f841641f0 con 0x7f9f84163de0 2024-04-25T14:06:17.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.708+0000 7f9f83400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9f84071530 0x7f9f841638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:17.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.708+0000 7f9f83400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9f84071530 0x7f9f841638a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47462/0 (socket says 172.21.15.86:47462) 2024-04-25T14:06:17.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.708+0000 7f9f83400640 1 -- 172.21.15.86:0/546369313 learned_addr learned my addr 172.21.15.86:0/546369313 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:17.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.708+0000 7f9f89200640 1 --2- 172.21.15.86:0/546369313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f84163de0 0x7f9f840cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:17.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.708+0000 7f9f83e00640 1 --1- 172.21.15.86:0/546369313 >> v1:172.21.15.155:6789/0 conn(0x7f9f8415fe00 0x7f9f84070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:43352/0 (socket says 172.21.15.86:43352) 2024-04-25T14:06:17.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.708+0000 7f9f82a00640 1 -- 172.21.15.86:0/546369313 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4123823845 0 0) 0x7f9f8406dc30 con 0x7f9f8415fe00 2024-04-25T14:06:17.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.708+0000 7f9f82a00640 1 -- 172.21.15.86:0/546369313 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f68003530 con 0x7f9f8415fe00 2024-04-25T14:06:17.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.708+0000 7f9f89200640 1 -- 172.21.15.86:0/546369313 >> v2:172.21.15.86:3300/0 conn(0x7f9f84071530 msgr2=0x7f9f841638a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:17.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.708+0000 7f9f89200640 1 --2- 172.21.15.86:0/546369313 >> v2:172.21.15.86:3300/0 conn(0x7f9f84071530 0x7f9f841638a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:17.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.708+0000 7f9f89200640 1 -- 172.21.15.86:0/546369313 >> v1:172.21.15.155:6789/0 conn(0x7f9f8415fe00 legacy=0x7f9f84070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:17.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.708+0000 7f9f89200640 1 -- 172.21.15.86:0/546369313 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f84164480 con 0x7f9f84163de0 2024-04-25T14:06:17.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.708+0000 7f9f83400640 1 --2- 172.21.15.86:0/546369313 >> v2:172.21.15.86:3300/0 conn(0x7f9f84071530 0x7f9f841638a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:17.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.712+0000 7f9f89200640 1 --2- 172.21.15.86:0/546369313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f84163de0 0x7f9f840cdde0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c009d90 tx=0x7f9f7c01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb902098985ad6e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:17.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.712+0000 7f9f82a00640 1 -- 172.21.15.86:0/546369313 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c0ad070 con 0x7f9f84163de0 2024-04-25T14:06:17.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.712+0000 7f9f82a00640 1 -- 172.21.15.86:0/546369313 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f7c0a9660 con 0x7f9f84163de0 2024-04-25T14:06:17.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.712+0000 7f9f82a00640 1 -- 172.21.15.86:0/546369313 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c0a9a10 con 0x7f9f84163de0 2024-04-25T14:06:17.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.712+0000 7f9f8b200640 1 -- 172.21.15.86:0/546369313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f84163de0 msgr2=0x7f9f840cdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:17.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.712+0000 7f9f8b200640 1 --2- 172.21.15.86:0/546369313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f84163de0 0x7f9f840cdde0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c009d90 tx=0x7f9f7c01c970 comp rx=0 tx=0).stop 2024-04-25T14:06:17.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.712+0000 7f9f8b200640 1 -- 172.21.15.86:0/546369313 shutdown_connections 2024-04-25T14:06:17.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.712+0000 7f9f8b200640 1 --2- 172.21.15.86:0/546369313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f84163de0 0x7f9f840cdde0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:17.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.712+0000 7f9f8b200640 1 --2- 172.21.15.86:0/546369313 >> v2:172.21.15.86:3300/0 conn(0x7f9f84071530 0x7f9f841638a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:17.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.712+0000 7f9f8b200640 1 -- 172.21.15.86:0/546369313 >> 172.21.15.86:0/546369313 conn(0x7f9f840cffc0 msgr2=0x7f9f840d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:17.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.712+0000 7f9f8b200640 1 -- 172.21.15.86:0/546369313 shutdown_connections 2024-04-25T14:06:17.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.712+0000 7f9f8b200640 1 -- 172.21.15.86:0/546369313 wait complete. 2024-04-25T14:06:17.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.712+0000 7f9f8b200640 1 Processor -- start 2024-04-25T14:06:17.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.716+0000 7f9f8b200640 1 -- start start 2024-04-25T14:06:17.716 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:17.716+0000 7fc57b0d6180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:06:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.716+0000 7f9f8b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f8415fe00 0x7f9f84151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.716+0000 7f9f8b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f84163de0 0x7f9f84153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.716+0000 7f9f8b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f84162de0 0x7f9f841535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.716+0000 7f9f8b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8416cda0 con 0x7f9f8415fe00 2024-04-25T14:06:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.716+0000 7f9f8b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8416cc20 con 0x7f9f84162de0 2024-04-25T14:06:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.716+0000 7f9f8b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9f8416cf20 con 0x7f9f84163de0 2024-04-25T14:06:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.716+0000 7f9f83e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f8415fe00 0x7f9f84151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.716+0000 7f9f83e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f8415fe00 0x7f9f84151fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47464/0 (socket says 172.21.15.86:47464) 2024-04-25T14:06:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.716+0000 7f9f83e00640 1 -- 172.21.15.86:0/2594953639 learned_addr learned my addr 172.21.15.86:0/2594953639 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.716+0000 7f9f83400640 1 --2- 172.21.15.86:0/2594953639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f84163de0 0x7f9f84153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.716+0000 7f9f89200640 1 --2- 172.21.15.86:0/2594953639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f84162de0 0x7f9f841535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.716+0000 7f9f83e00640 1 -- 172.21.15.86:0/2594953639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f84163de0 msgr2=0x7f9f84153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.716+0000 7f9f83e00640 1 --2- 172.21.15.86:0/2594953639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f84163de0 0x7f9f84153060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.716+0000 7f9f83e00640 1 -- 172.21.15.86:0/2594953639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f84162de0 msgr2=0x7f9f841535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.716+0000 7f9f83e00640 1 --2- 172.21.15.86:0/2594953639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f84162de0 0x7f9f841535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.716+0000 7f9f83e00640 1 -- 172.21.15.86:0/2594953639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f84158550 con 0x7f9f8415fe00 2024-04-25T14:06:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.716+0000 7f9f83e00640 1 --2- 172.21.15.86:0/2594953639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f8415fe00 0x7f9f84151fe0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9f740010f0 tx=0x7f9f7400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.716+0000 7f9f7be00640 1 -- 172.21.15.86:0/2594953639 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f740074e0 con 0x7f9f8415fe00 2024-04-25T14:06:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.716+0000 7f9f7be00640 1 -- 172.21.15.86:0/2594953639 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f74007680 con 0x7f9f8415fe00 2024-04-25T14:06:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.716+0000 7f9f8b200640 1 -- 172.21.15.86:0/2594953639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f84158830 con 0x7f9f8415fe00 2024-04-25T14:06:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.716+0000 7f9f7be00640 1 -- 172.21.15.86:0/2594953639 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f74004a30 con 0x7f9f8415fe00 2024-04-25T14:06:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.716+0000 7f9f8b200640 1 -- 172.21.15.86:0/2594953639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f84158b10 con 0x7f9f8415fe00 2024-04-25T14:06:17.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.720+0000 7f9f8b200640 1 -- 172.21.15.86:0/2594953639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f841631f0 con 0x7f9f8415fe00 2024-04-25T14:06:17.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.720+0000 7f9f7be00640 1 -- 172.21.15.86:0/2594953639 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) ==== 93309+0+0 (secure 0 0 0) 0x7f9f7401a020 con 0x7f9f8415fe00 2024-04-25T14:06:17.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.720+0000 7f9f7be00640 1 --2- 172.21.15.86:0/2594953639 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f9f6406f550 0x7f9f64071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:17.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.720+0000 7f9f7be00640 1 -- 172.21.15.86:0/2594953639 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7f9f7408f1e0 con 0x7f9f8415fe00 2024-04-25T14:06:17.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.720+0000 7f9f83400640 1 -- 172.21.15.86:0/2594953639 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f9f6406f550 msgr2=0x7f9f64071a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2427179681 2024-04-25T14:06:17.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.720+0000 7f9f83400640 1 --2- 172.21.15.86:0/2594953639 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f9f6406f550 0x7f9f64071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:06:17.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.724+0000 7f9f7be00640 1 -- 172.21.15.86:0/2594953639 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9f841631f0 con 0x7f9f8415fe00 2024-04-25T14:06:17.733 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:06:17.733+0000 7f95af866180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:06:17.803 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:06:17.805+0000 7f95af866180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:06:17.847 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:17.844+0000 7fc57b0d6180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:06:17.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.884+0000 7f9f8b200640 1 -- 172.21.15.86:0/2594953639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "fs2_data", "var": "pg_num"} v 0) -- 0x7f9f840720e0 con 0x7f9f8415fe00 2024-04-25T14:06:17.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.884+0000 7f9f7be00640 1 -- 172.21.15.86:0/2594953639 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "fs2_data", "var": "pg_num"}]=0 v58) ==== 97+0+11 (secure 0 0 0) 0x7f9f7405b250 con 0x7f9f8415fe00 2024-04-25T14:06:17.887 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:06:17.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.888+0000 7f9f8b200640 1 -- 172.21.15.86:0/2594953639 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f9f6406f550 msgr2=0x7f9f64071a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:06:17.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.888+0000 7f9f8b200640 1 --2- 172.21.15.86:0/2594953639 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f9f6406f550 0x7f9f64071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:17.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.888+0000 7f9f8b200640 1 -- 172.21.15.86:0/2594953639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f8415fe00 msgr2=0x7f9f84151fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:17.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.888+0000 7f9f8b200640 1 --2- 172.21.15.86:0/2594953639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f8415fe00 0x7f9f84151fe0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9f740010f0 tx=0x7f9f7400c820 comp rx=0 tx=0).stop 2024-04-25T14:06:17.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.888+0000 7f9f8b200640 1 -- 172.21.15.86:0/2594953639 shutdown_connections 2024-04-25T14:06:17.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.888+0000 7f9f8b200640 1 --2- 172.21.15.86:0/2594953639 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f9f6406f550 0x7f9f64071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:17.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.888+0000 7f9f8b200640 1 --2- 172.21.15.86:0/2594953639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f84162de0 0x7f9f841535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:17.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.888+0000 7f9f8b200640 1 --2- 172.21.15.86:0/2594953639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f84163de0 0x7f9f84153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:17.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.888+0000 7f9f8b200640 1 --2- 172.21.15.86:0/2594953639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f8415fe00 0x7f9f84151fe0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:17.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.888+0000 7f9f8b200640 1 -- 172.21.15.86:0/2594953639 >> 172.21.15.86:0/2594953639 conn(0x7f9f840cffc0 msgr2=0x7f9f841560e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:17.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.888+0000 7f9f8b200640 1 -- 172.21.15.86:0/2594953639 shutdown_connections 2024-04-25T14:06:17.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:17.888+0000 7f9f8b200640 1 -- 172.21.15.86:0/2594953639 wait complete. 2024-04-25T14:06:17.910 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T14:06:17.921 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:17.920+0000 7fc57b0d6180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:06:17.952 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:06:17.953+0000 7f95af866180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:06:18.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.068+0000 7fa21dc00640 1 Processor -- start 2024-04-25T14:06:18.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.068+0000 7fa21dc00640 1 -- start start 2024-04-25T14:06:18.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.072+0000 7fa21dc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa2180cf970 0x7fa218070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.072+0000 7fa21dc00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa218071360 0x7fa21806e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.072+0000 7fa21dc00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa218071770 con 0x7fa21806e7a0 2024-04-25T14:06:18.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.072+0000 7fa21dc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa2180cedf0 con 0x7fa2180cf970 2024-04-25T14:06:18.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.072+0000 7fa21dc00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa218163830 con 0x7fa218071360 2024-04-25T14:06:18.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.072+0000 7fa216a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa2180cf970 0x7fa218070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.072+0000 7fa216a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa2180cf970 0x7fa218070e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47476/0 (socket says 172.21.15.86:47476) 2024-04-25T14:06:18.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.072+0000 7fa216a00640 1 -- 172.21.15.86:0/1439928989 learned_addr learned my addr 172.21.15.86:0/1439928989 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:18.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.072+0000 7fa216000640 1 --2- 172.21.15.86:0/1439928989 >> v2:172.21.15.155:3300/0 conn(0x7fa218071360 0x7fa21806e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.072+0000 7fa216a00640 1 -- 172.21.15.86:0/1439928989 >> v2:172.21.15.155:3300/0 conn(0x7fa218071360 msgr2=0x7fa21806e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.072+0000 7fa216a00640 1 --2- 172.21.15.86:0/1439928989 >> v2:172.21.15.155:3300/0 conn(0x7fa218071360 0x7fa21806e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.072+0000 7fa216a00640 1 -- 172.21.15.86:0/1439928989 >> v1:172.21.15.155:6789/0 conn(0x7fa21806e7a0 legacy=0x7fa2180722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.072+0000 7fa216a00640 1 -- 172.21.15.86:0/1439928989 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa21806ef90 con 0x7fa2180cf970 2024-04-25T14:06:18.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.072+0000 7fa215600640 1 -- 172.21.15.86:0/1439928989 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 986683749 0 0) 0x7fa2100014a0 con 0x7fa21806e7a0 2024-04-25T14:06:18.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.076+0000 7fa216a00640 1 --2- 172.21.15.86:0/1439928989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2180cf970 0x7fa218070e20 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa21806d8a0 tx=0x7fa20801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e970e0803da841e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.076+0000 7fa215600640 1 -- 172.21.15.86:0/1439928989 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2080ab610 con 0x7fa2180cf970 2024-04-25T14:06:18.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.076+0000 7fa215600640 1 -- 172.21.15.86:0/1439928989 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2080b0070 con 0x7fa2180cf970 2024-04-25T14:06:18.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.076+0000 7fa215600640 1 -- 172.21.15.86:0/1439928989 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2080ab9d0 con 0x7fa2180cf970 2024-04-25T14:06:18.077 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:18.076+0000 7fc57b0d6180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:06:18.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.076+0000 7fa21dc00640 1 -- 172.21.15.86:0/1439928989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2180cf970 msgr2=0x7fa218070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.076+0000 7fa21dc00640 1 --2- 172.21.15.86:0/1439928989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2180cf970 0x7fa218070e20 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa21806d8a0 tx=0x7fa20801c4f0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.076+0000 7fa21dc00640 1 -- 172.21.15.86:0/1439928989 shutdown_connections 2024-04-25T14:06:18.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.076+0000 7fa21dc00640 1 --2- 172.21.15.86:0/1439928989 >> v2:172.21.15.155:3300/0 conn(0x7fa218071360 0x7fa21806e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.076+0000 7fa21dc00640 1 --2- 172.21.15.86:0/1439928989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2180cf970 0x7fa218070e20 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.076+0000 7fa21dc00640 1 -- 172.21.15.86:0/1439928989 >> 172.21.15.86:0/1439928989 conn(0x7fa2180cffc0 msgr2=0x7fa2180d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.076+0000 7fa21dc00640 1 -- 172.21.15.86:0/1439928989 shutdown_connections 2024-04-25T14:06:18.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.076+0000 7fa21dc00640 1 -- 172.21.15.86:0/1439928989 wait complete. 2024-04-25T14:06:18.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.080+0000 7fa21dc00640 1 Processor -- start 2024-04-25T14:06:18.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.080+0000 7fa21dc00640 1 -- start start 2024-04-25T14:06:18.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.080+0000 7fa21dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa21806e7a0 0x7fa2180b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.080+0000 7fa21dc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa218071360 0x7fa2180b3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.080+0000 7fa21dc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2180cf970 0x7fa2180b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.080+0000 7fa21dc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa21806ed30 con 0x7fa21806e7a0 2024-04-25T14:06:18.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.080+0000 7fa21dc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa21806ebb0 con 0x7fa2180cf970 2024-04-25T14:06:18.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.080+0000 7fa21dc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa21816cbd0 con 0x7fa218071360 2024-04-25T14:06:18.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.080+0000 7fa216a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa21806e7a0 0x7fa2180b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.080+0000 7fa216a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa21806e7a0 0x7fa2180b6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47492/0 (socket says 172.21.15.86:47492) 2024-04-25T14:06:18.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.080+0000 7fa216a00640 1 -- 172.21.15.86:0/4113951897 learned_addr learned my addr 172.21.15.86:0/4113951897 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:18.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.080+0000 7fa217400640 1 --2- 172.21.15.86:0/4113951897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2180cf970 0x7fa2180b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.080+0000 7fa216000640 1 --2- 172.21.15.86:0/4113951897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa218071360 0x7fa2180b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.080+0000 7fa216a00640 1 -- 172.21.15.86:0/4113951897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa218071360 msgr2=0x7fa2180b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.080+0000 7fa216a00640 1 --2- 172.21.15.86:0/4113951897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa218071360 0x7fa2180b3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.080+0000 7fa216a00640 1 -- 172.21.15.86:0/4113951897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2180cf970 msgr2=0x7fa2180b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.080+0000 7fa216a00640 1 --2- 172.21.15.86:0/4113951897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2180cf970 0x7fa2180b3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.080+0000 7fa216a00640 1 -- 172.21.15.86:0/4113951897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2180b6e00 con 0x7fa21806e7a0 2024-04-25T14:06:18.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.080+0000 7fa216000640 1 --2- 172.21.15.86:0/4113951897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa218071360 0x7fa2180b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:18.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.080+0000 7fa216a00640 1 --2- 172.21.15.86:0/4113951897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa21806e7a0 0x7fa2180b6160 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa2080ad040 tx=0x7fa20801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.080+0000 7fa20ea00640 1 -- 172.21.15.86:0/4113951897 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2080be5f0 con 0x7fa21806e7a0 2024-04-25T14:06:18.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.080+0000 7fa20ea00640 1 -- 172.21.15.86:0/4113951897 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2080be7b0 con 0x7fa21806e7a0 2024-04-25T14:06:18.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.080+0000 7fa21dc00640 1 -- 172.21.15.86:0/4113951897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2180b70e0 con 0x7fa21806e7a0 2024-04-25T14:06:18.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.080+0000 7fa20ea00640 1 -- 172.21.15.86:0/4113951897 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2080beba0 con 0x7fa21806e7a0 2024-04-25T14:06:18.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.080+0000 7fa21dc00640 1 -- 172.21.15.86:0/4113951897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2180cb2a0 con 0x7fa21806e7a0 2024-04-25T14:06:18.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.084+0000 7fa21dc00640 1 -- 172.21.15.86:0/4113951897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2180720e0 con 0x7fa21806e7a0 2024-04-25T14:06:18.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.084+0000 7fa20ea00640 1 -- 172.21.15.86:0/4113951897 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) ==== 93309+0+0 (secure 0 0 0) 0x7fa2080b43f0 con 0x7fa21806e7a0 2024-04-25T14:06:18.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.084+0000 7fa20ea00640 1 --2- 172.21.15.86:0/4113951897 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fa1f806f520 0x7fa1f8071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.084+0000 7fa20ea00640 1 -- 172.21.15.86:0/4113951897 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7fa208129bf0 con 0x7fa21806e7a0 2024-04-25T14:06:18.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.084+0000 7fa216000640 1 -- 172.21.15.86:0/4113951897 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fa1f806f520 msgr2=0x7fa1f8071a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2427179681 2024-04-25T14:06:18.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.088+0000 7fa216000640 1 --2- 172.21.15.86:0/4113951897 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fa1f806f520 0x7fa1f8071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:06:18.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.092+0000 7fa20ea00640 1 -- 172.21.15.86:0/4113951897 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa2080c4060 con 0x7fa21806e7a0 2024-04-25T14:06:18.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.264+0000 7fa21dc00640 1 -- 172.21.15.86:0/4113951897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa218076bf0 con 0x7fa21806e7a0 2024-04-25T14:06:18.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.264+0000 7fa20ea00640 1 -- 172.21.15.86:0/4113951897 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 14 entries v58) ==== 81+0+838 (secure 0 0 0) 0x7fa208127e20 con 0x7fa21806e7a0 2024-04-25T14:06:18.266 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6836/1732615622 2024-04-26T14:06:00.230367+0000 2024-04-25T14:06:18.266 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6837/716923566 2024-04-26T14:05:25.452701+0000 2024-04-25T14:06:18.266 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6837/2418778265 2024-04-26T14:06:00.222915+0000 2024-04-25T14:06:18.266 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1843278167 2024-04-26T14:05:21.460855+0000 2024-04-25T14:06:18.266 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6832/2968869755 2024-04-26T14:05:21.460855+0000 2024-04-25T14:06:18.266 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2073988773 2024-04-26T14:05:21.460855+0000 2024-04-25T14:06:18.266 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6835/716923566 2024-04-26T14:05:25.452701+0000 2024-04-25T14:06:18.266 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2602032945 2024-04-26T14:05:21.460855+0000 2024-04-25T14:06:18.266 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3558294881 2024-04-26T14:05:21.460855+0000 2024-04-25T14:06:18.266 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6835/615772001 2024-04-26T14:05:42.882867+0000 2024-04-25T14:06:18.266 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6837/615772001 2024-04-26T14:05:42.882867+0000 2024-04-25T14:06:18.267 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6833/2968869755 2024-04-26T14:05:21.460855+0000 2024-04-25T14:06:18.267 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6835/2418778265 2024-04-26T14:06:00.222915+0000 2024-04-25T14:06:18.267 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6834/1732615622 2024-04-26T14:06:00.230367+0000 2024-04-25T14:06:18.268 INFO:teuthology.orchestra.run.smithi086.stderr:listed 14 entries 2024-04-25T14:06:18.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.264+0000 7fa21dc00640 1 -- 172.21.15.86:0/4113951897 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fa1f806f520 msgr2=0x7fa1f8071a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:06:18.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.264+0000 7fa21dc00640 1 --2- 172.21.15.86:0/4113951897 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fa1f806f520 0x7fa1f8071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.264+0000 7fa21dc00640 1 -- 172.21.15.86:0/4113951897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa21806e7a0 msgr2=0x7fa2180b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.264+0000 7fa21dc00640 1 --2- 172.21.15.86:0/4113951897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa21806e7a0 0x7fa2180b6160 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa2080ad040 tx=0x7fa20801ad30 comp rx=0 tx=0).stop 2024-04-25T14:06:18.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.264+0000 7fa21dc00640 1 -- 172.21.15.86:0/4113951897 shutdown_connections 2024-04-25T14:06:18.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.264+0000 7fa21dc00640 1 --2- 172.21.15.86:0/4113951897 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fa1f806f520 0x7fa1f8071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.264+0000 7fa21dc00640 1 --2- 172.21.15.86:0/4113951897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2180cf970 0x7fa2180b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.264+0000 7fa21dc00640 1 --2- 172.21.15.86:0/4113951897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa218071360 0x7fa2180b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.264+0000 7fa21dc00640 1 --2- 172.21.15.86:0/4113951897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa21806e7a0 0x7fa2180b6160 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.264+0000 7fa21dc00640 1 -- 172.21.15.86:0/4113951897 >> 172.21.15.86:0/4113951897 conn(0x7fa2180cffc0 msgr2=0x7fa2180d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.268+0000 7fa21dc00640 1 -- 172.21.15.86:0/4113951897 shutdown_connections 2024-04-25T14:06:18.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.268+0000 7fa21dc00640 1 -- 172.21.15.86:0/4113951897 wait complete. 2024-04-25T14:06:18.287 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T14:06:18.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:18.288 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T14:06:18.314 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T14:06:18.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:18.314 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T14:06:18.364 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:06:18.364 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:06:18.364 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:06:18.373 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:06:18.373 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:06:18.423 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:06:18.500 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:06:18.501+0000 7f95af866180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:06:18.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.572+0000 7f8021800640 1 Processor -- start 2024-04-25T14:06:18.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.572+0000 7f8021800640 1 -- start start 2024-04-25T14:06:18.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.576+0000 7f8021800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f801c0d28f0 0x7f801c16abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.576+0000 7f8021800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f801c163940 con 0x7f801c0cdda0 2024-04-25T14:06:18.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.576+0000 7f8021800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f801c0d2f40 con 0x7f801c0cec60 2024-04-25T14:06:18.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.576+0000 7f8021800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f801c0d2d00 con 0x7f801c0d28f0 2024-04-25T14:06:18.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.576+0000 7f801b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f801c0d28f0 0x7f801c16abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.576+0000 7f801a000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f801c0cdda0 0x7f801c0cf780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59812/0 (socket says 172.21.15.86:59812) 2024-04-25T14:06:18.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.576+0000 7f801a000640 1 -- 172.21.15.86:0/1043813431 learned_addr learned my addr 172.21.15.86:0/1043813431 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:18.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.576+0000 7f801b400640 1 -- 172.21.15.86:0/1043813431 >> v1:172.21.15.155:6789/0 conn(0x7f801c0cec60 legacy=0x7f801c0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.576+0000 7f801b400640 1 -- 172.21.15.86:0/1043813431 >> v1:172.21.15.86:6789/0 conn(0x7f801c0cdda0 legacy=0x7f801c0cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.576+0000 7f801b400640 1 -- 172.21.15.86:0/1043813431 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801c0b3720 con 0x7f801c0d28f0 2024-04-25T14:06:18.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.576+0000 7f801b400640 1 --2- 172.21.15.86:0/1043813431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801c0d28f0 0x7f801c16abe0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f8014009d90 tx=0x7f801401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea75a105f79debc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.576+0000 7f8019600640 1 -- 172.21.15.86:0/1043813431 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80140b2070 con 0x7f801c0d28f0 2024-04-25T14:06:18.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.576+0000 7f8019600640 1 -- 172.21.15.86:0/1043813431 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80140a5830 con 0x7f801c0d28f0 2024-04-25T14:06:18.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.580+0000 7f8019600640 1 -- 172.21.15.86:0/1043813431 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80140a5c00 con 0x7f801c0d28f0 2024-04-25T14:06:18.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.580+0000 7f8021800640 1 -- 172.21.15.86:0/1043813431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801c0d28f0 msgr2=0x7f801c16abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.580+0000 7f8021800640 1 --2- 172.21.15.86:0/1043813431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801c0d28f0 0x7f801c16abe0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f8014009d90 tx=0x7f801401c970 comp rx=0 tx=0).stop 2024-04-25T14:06:18.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.580+0000 7f8021800640 1 -- 172.21.15.86:0/1043813431 shutdown_connections 2024-04-25T14:06:18.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.580+0000 7f8021800640 1 --2- 172.21.15.86:0/1043813431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801c0d28f0 0x7f801c16abe0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.580+0000 7f8021800640 1 -- 172.21.15.86:0/1043813431 >> 172.21.15.86:0/1043813431 conn(0x7f801c09ba30 msgr2=0x7f801c0b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.580+0000 7f8021800640 1 -- 172.21.15.86:0/1043813431 shutdown_connections 2024-04-25T14:06:18.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.580+0000 7f8021800640 1 -- 172.21.15.86:0/1043813431 wait complete. 2024-04-25T14:06:18.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.580+0000 7f8021800640 1 Processor -- start 2024-04-25T14:06:18.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.580+0000 7f8021800640 1 -- start start 2024-04-25T14:06:18.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.584+0000 7f8021800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f801c0cdda0 0x7f801c158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.584+0000 7f8021800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801c0cec60 0x7f801c0af140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.584+0000 7f8021800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f801c0d28f0 0x7f801c0af680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.584+0000 7f8021800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f801c16dcd0 con 0x7f801c0cec60 2024-04-25T14:06:18.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.584+0000 7f8021800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f801c16db50 con 0x7f801c0d28f0 2024-04-25T14:06:18.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.584+0000 7f8021800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f801c16de50 con 0x7f801c0cdda0 2024-04-25T14:06:18.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.584+0000 7f801a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801c0cec60 0x7f801c0af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.584+0000 7f801a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801c0cec60 0x7f801c0af140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47508/0 (socket says 172.21.15.86:47508) 2024-04-25T14:06:18.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.584+0000 7f801a000640 1 -- 172.21.15.86:0/3607591156 learned_addr learned my addr 172.21.15.86:0/3607591156 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:18.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.584+0000 7f801aa00640 1 --2- 172.21.15.86:0/3607591156 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f801c0cdda0 0x7f801c158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.584+0000 7f801b400640 1 --2- 172.21.15.86:0/3607591156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f801c0d28f0 0x7f801c0af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.584+0000 7f801a000640 1 -- 172.21.15.86:0/3607591156 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f801c0cdda0 msgr2=0x7f801c158a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.584+0000 7f801a000640 1 --2- 172.21.15.86:0/3607591156 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f801c0cdda0 0x7f801c158a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.584+0000 7f801a000640 1 -- 172.21.15.86:0/3607591156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f801c0d28f0 msgr2=0x7f801c0af680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.584+0000 7f801a000640 1 --2- 172.21.15.86:0/3607591156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f801c0d28f0 0x7f801c0af680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.584+0000 7f801a000640 1 -- 172.21.15.86:0/3607591156 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801c158fc0 con 0x7f801c0cec60 2024-04-25T14:06:18.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.584+0000 7f801a000640 1 --2- 172.21.15.86:0/3607591156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801c0cec60 0x7f801c0af140 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f800800c430 tx=0x7f800800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.587 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:06:18.585+0000 7f95af866180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:06:18.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.584+0000 7f8012a00640 1 -- 172.21.15.86:0/3607591156 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8008017070 con 0x7f801c0cec60 2024-04-25T14:06:18.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.584+0000 7f8012a00640 1 -- 172.21.15.86:0/3607591156 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8008005cf0 con 0x7f801c0cec60 2024-04-25T14:06:18.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.584+0000 7f8021800640 1 -- 172.21.15.86:0/3607591156 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f801c0c8ce0 con 0x7f801c0cec60 2024-04-25T14:06:18.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.584+0000 7f8012a00640 1 -- 172.21.15.86:0/3607591156 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80080072d0 con 0x7f801c0cec60 2024-04-25T14:06:18.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.584+0000 7f8021800640 1 -- 172.21.15.86:0/3607591156 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f801c15a0a0 con 0x7f801c0cec60 2024-04-25T14:06:18.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.584+0000 7f8021800640 1 -- 172.21.15.86:0/3607591156 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f801c0a04c0 con 0x7f801c0cec60 2024-04-25T14:06:18.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.588+0000 7f8012a00640 1 -- 172.21.15.86:0/3607591156 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) ==== 93309+0+0 (secure 0 0 0) 0x7f800801a070 con 0x7f801c0cec60 2024-04-25T14:06:18.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.588+0000 7f8012a00640 1 --2- 172.21.15.86:0/3607591156 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f7ffc06f520 0x7f7ffc071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.588+0000 7f801aa00640 1 -- 172.21.15.86:0/3607591156 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f7ffc06f520 msgr2=0x7f7ffc071a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2427179681 2024-04-25T14:06:18.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.588+0000 7f8012a00640 1 -- 172.21.15.86:0/3607591156 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7f800808ed70 con 0x7f801c0cec60 2024-04-25T14:06:18.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.588+0000 7f801aa00640 1 --2- 172.21.15.86:0/3607591156 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f7ffc06f520 0x7f7ffc071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:06:18.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.596+0000 7f8012a00640 1 -- 172.21.15.86:0/3607591156 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f800805ae70 con 0x7f801c0cec60 2024-04-25T14:06:18.652 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:18.648+0000 7fc57b0d6180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:06:18.738 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:18.736+0000 7fc57b0d6180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:06:18.746 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:06:18.745+0000 7f95af866180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:06:18.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.756+0000 7f8021800640 1 -- 172.21.15.86:0/3607591156 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f801c076bf0 con 0x7f801c0cec60 2024-04-25T14:06:18.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.760+0000 7f8012a00640 1 -- 172.21.15.86:0/3607591156 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+21531 (secure 0 0 0) 0x7f800805f170 con 0x7f801c0cec60 2024-04-25T14:06:18.761 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:18.761 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":58,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:06:00.230399+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"fs1_metadata","create_time":"2024-04-25T14:05:30.939601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"fs1"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"fs1_data","create_time":"2024-04-25T14:05:31.898262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"fs1"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"fs2_metadata","create_time":"2024-04-25T14:05:52.841764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"fs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"fs2_data","create_time":"2024-04-25T14:05:53.524454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"fs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6836/1732615622":"2024-04-26T14:06:00.230367+0000","172.21.15.86:6837/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:6837/2418778265":"2024-04-26T14:06:00.222915+0000","172.21.15.86:0/1843278167":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6832/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/2073988773":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6835/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:0/2602032945":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/3558294881":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6835/615772001":"2024-04-26T14:05:42.882867+0000","172.21.15.86:6837/615772001":"2024-04-26T14:05:42.882867+0000","172.21.15.86:6833/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6835/2418778265":"2024-04-26T14:06:00.222915+0000","172.21.15.86:6834/1732615622":"2024-04-26T14:06:00.230367+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:06:18.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.760+0000 7f8021800640 1 -- 172.21.15.86:0/3607591156 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f7ffc06f520 msgr2=0x7f7ffc071a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:06:18.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.760+0000 7f8021800640 1 --2- 172.21.15.86:0/3607591156 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f7ffc06f520 0x7f7ffc071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.760+0000 7f8021800640 1 -- 172.21.15.86:0/3607591156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801c0cec60 msgr2=0x7f801c0af140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.760+0000 7f8021800640 1 --2- 172.21.15.86:0/3607591156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801c0cec60 0x7f801c0af140 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f800800c430 tx=0x7f800800c8f0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.760+0000 7f8021800640 1 -- 172.21.15.86:0/3607591156 shutdown_connections 2024-04-25T14:06:18.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.760+0000 7f8021800640 1 --2- 172.21.15.86:0/3607591156 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f7ffc06f520 0x7f7ffc071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.760+0000 7f8021800640 1 --2- 172.21.15.86:0/3607591156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f801c0d28f0 0x7f801c0af680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.760+0000 7f8021800640 1 --2- 172.21.15.86:0/3607591156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801c0cec60 0x7f801c0af140 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.760+0000 7f8021800640 1 --2- 172.21.15.86:0/3607591156 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f801c0cdda0 0x7f801c158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.760+0000 7f8021800640 1 -- 172.21.15.86:0/3607591156 >> 172.21.15.86:0/3607591156 conn(0x7f801c09ba30 msgr2=0x7f801c15bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.760+0000 7f8021800640 1 -- 172.21.15.86:0/3607591156 shutdown_connections 2024-04-25T14:06:18.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.760+0000 7f8021800640 1 -- 172.21.15.86:0/3607591156 wait complete. 2024-04-25T14:06:18.783 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'fs1_metadata', 'create_time': '2024-04-25T14:05:30.939601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'fs1'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'fs1_data', 'create_time': '2024-04-25T14:05:31.898262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'fs1'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'fs2_metadata', 'create_time': '2024-04-25T14:05:52.841764+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'fs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'fs2_data', 'create_time': '2024-04-25T14:05:53.524454+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'fs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:06:18.783 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:06:18.829 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:06:18.829+0000 7f95af866180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:06:18.907 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:18.904+0000 7fc57b0d6180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:06:18.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.936+0000 7fb8ac800640 1 Processor -- start 2024-04-25T14:06:18.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.936+0000 7fb8ac800640 1 -- start start 2024-04-25T14:06:18.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.936+0000 7fb8ac800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8a40cdd70 0x7fb8a40ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.936+0000 7fb8ac800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb8a4070e20 0x7fb8a4071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.936+0000 7fb8ac800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb8a406e700 0x7fb8a406eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.936+0000 7fb8ac800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb8a4071770 con 0x7fb8a4070e20 2024-04-25T14:06:18.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.936+0000 7fb8ac800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb8a406f050 con 0x7fb8a406e700 2024-04-25T14:06:18.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.936+0000 7fb8ac800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb8a406d940 con 0x7fb8a40cdd70 2024-04-25T14:06:18.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.936+0000 7fb8a9400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb8a4070e20 0x7fb8a4071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.936+0000 7fb8a9400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb8a4070e20 0x7fb8a4071230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55052/0 (socket says 172.21.15.86:55052) 2024-04-25T14:06:18.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.936+0000 7fb8a9400640 1 -- 172.21.15.86:0/2465124444 learned_addr learned my addr 172.21.15.86:0/2465124444 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:18.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.936+0000 7fb8a9e00640 1 --2- 172.21.15.86:0/2465124444 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8a40cdd70 0x7fb8a40ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.936+0000 7fb8aa800640 1 --2- 172.21.15.86:0/2465124444 >> v2:172.21.15.155:3300/0 conn(0x7fb8a406e700 0x7fb8a406eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.936+0000 7fb8a9400640 1 -- 172.21.15.86:0/2465124444 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8a40cdd70 msgr2=0x7fb8a40ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.936+0000 7fb8a9400640 1 --2- 172.21.15.86:0/2465124444 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8a40cdd70 0x7fb8a40ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.936+0000 7fb8a9400640 1 -- 172.21.15.86:0/2465124444 >> v2:172.21.15.155:3300/0 conn(0x7fb8a406e700 msgr2=0x7fb8a406eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.936+0000 7fb8a9400640 1 --2- 172.21.15.86:0/2465124444 >> v2:172.21.15.155:3300/0 conn(0x7fb8a406e700 0x7fb8a406eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.936+0000 7fb8a9400640 1 -- 172.21.15.86:0/2465124444 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a406dc20 con 0x7fb8a4070e20 2024-04-25T14:06:18.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.940+0000 7fb8a9400640 1 --2- 172.21.15.86:0/2465124444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8a4070e20 0x7fb8a4071230 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4071b00 tx=0x7fb89401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7db9f3cc7fbac8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.940+0000 7fb8a8a00640 1 -- 172.21.15.86:0/2465124444 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8940b0070 con 0x7fb8a4070e20 2024-04-25T14:06:18.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.940+0000 7fb8a8a00640 1 -- 172.21.15.86:0/2465124444 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8940ab600 con 0x7fb8a4070e20 2024-04-25T14:06:18.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.940+0000 7fb8a8a00640 1 -- 172.21.15.86:0/2465124444 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8940ab9d0 con 0x7fb8a4070e20 2024-04-25T14:06:18.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.940+0000 7fb8ac800640 1 -- 172.21.15.86:0/2465124444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8a4070e20 msgr2=0x7fb8a4071230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.940+0000 7fb8ac800640 1 --2- 172.21.15.86:0/2465124444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8a4070e20 0x7fb8a4071230 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4071b00 tx=0x7fb89401c490 comp rx=0 tx=0).stop 2024-04-25T14:06:18.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.940+0000 7fb8ac800640 1 -- 172.21.15.86:0/2465124444 shutdown_connections 2024-04-25T14:06:18.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.940+0000 7fb8ac800640 1 --2- 172.21.15.86:0/2465124444 >> v2:172.21.15.155:3300/0 conn(0x7fb8a406e700 0x7fb8a406eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.940+0000 7fb8ac800640 1 --2- 172.21.15.86:0/2465124444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8a4070e20 0x7fb8a4071230 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.940+0000 7fb8ac800640 1 --2- 172.21.15.86:0/2465124444 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8a40cdd70 0x7fb8a40ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.940+0000 7fb8ac800640 1 -- 172.21.15.86:0/2465124444 >> 172.21.15.86:0/2465124444 conn(0x7fb8a40cffc0 msgr2=0x7fb8a40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.940+0000 7fb8ac800640 1 -- 172.21.15.86:0/2465124444 shutdown_connections 2024-04-25T14:06:18.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.940+0000 7fb8ac800640 1 -- 172.21.15.86:0/2465124444 wait complete. 2024-04-25T14:06:18.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.944+0000 7fb8ac800640 1 Processor -- start 2024-04-25T14:06:18.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.944+0000 7fb8ac800640 1 -- start start 2024-04-25T14:06:18.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.944+0000 7fb8ac800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8a406e700 0x7fb8a40cd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.944+0000 7fb8ac800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8a4070e20 0x7fb8a4164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.944+0000 7fb8ac800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8a40cdd70 0x7fb8a4164ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.944+0000 7fb8ac800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a41642f0 con 0x7fb8a4070e20 2024-04-25T14:06:18.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.944+0000 7fb8ac800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a4164170 con 0x7fb8a40cdd70 2024-04-25T14:06:18.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.944+0000 7fb8ac800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb8a4164470 con 0x7fb8a406e700 2024-04-25T14:06:18.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.944+0000 7fb8a9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8a4070e20 0x7fb8a4164990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.944+0000 7fb8a9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8a4070e20 0x7fb8a4164990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55060/0 (socket says 172.21.15.86:55060) 2024-04-25T14:06:18.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.944+0000 7fb8a9400640 1 -- 172.21.15.86:0/3157305387 learned_addr learned my addr 172.21.15.86:0/3157305387 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:18.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.944+0000 7fb8aa800640 1 --2- 172.21.15.86:0/3157305387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8a40cdd70 0x7fb8a4164ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.944+0000 7fb8a9e00640 1 --2- 172.21.15.86:0/3157305387 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8a406e700 0x7fb8a40cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.944+0000 7fb8a9400640 1 -- 172.21.15.86:0/3157305387 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8a406e700 msgr2=0x7fb8a40cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.944+0000 7fb8a9400640 1 --2- 172.21.15.86:0/3157305387 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8a406e700 0x7fb8a40cd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.944+0000 7fb8a9400640 1 -- 172.21.15.86:0/3157305387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8a40cdd70 msgr2=0x7fb8a4164ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.944+0000 7fb8a9400640 1 --2- 172.21.15.86:0/3157305387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8a40cdd70 0x7fb8a4164ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.944+0000 7fb8a9400640 1 -- 172.21.15.86:0/3157305387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a40cb110 con 0x7fb8a4070e20 2024-04-25T14:06:18.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.944+0000 7fb8a9400640 1 --2- 172.21.15.86:0/3157305387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8a4070e20 0x7fb8a4164990 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb89401cd30 tx=0x7fb894019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.948+0000 7fb89e000640 1 -- 172.21.15.86:0/3157305387 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8940b0040 con 0x7fb8a4070e20 2024-04-25T14:06:18.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.948+0000 7fb89e000640 1 -- 172.21.15.86:0/3157305387 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb894004020 con 0x7fb8a4070e20 2024-04-25T14:06:18.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.948+0000 7fb89e000640 1 -- 172.21.15.86:0/3157305387 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8940043d0 con 0x7fb8a4070e20 2024-04-25T14:06:18.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.948+0000 7fb8ac800640 1 -- 172.21.15.86:0/3157305387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8a40cb3f0 con 0x7fb8a4070e20 2024-04-25T14:06:18.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.948+0000 7fb8ac800640 1 -- 172.21.15.86:0/3157305387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8a40cb9b0 con 0x7fb8a4070e20 2024-04-25T14:06:18.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.948+0000 7fb8ac800640 1 -- 172.21.15.86:0/3157305387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8a40720e0 con 0x7fb8a4070e20 2024-04-25T14:06:18.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.948+0000 7fb89e000640 1 -- 172.21.15.86:0/3157305387 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) ==== 93309+0+0 (secure 0 0 0) 0x7fb8940bb060 con 0x7fb8a4070e20 2024-04-25T14:06:18.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.952+0000 7fb89e000640 1 --2- 172.21.15.86:0/3157305387 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb88806f550 0x7fb888071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.952+0000 7fb89e000640 1 -- 172.21.15.86:0/3157305387 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7fb894128930 con 0x7fb8a4070e20 2024-04-25T14:06:18.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.952+0000 7fb8a9e00640 1 -- 172.21.15.86:0/3157305387 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb88806f550 msgr2=0x7fb888071a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2427179681 2024-04-25T14:06:18.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.952+0000 7fb8a9e00640 1 --2- 172.21.15.86:0/3157305387 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb88806f550 0x7fb888071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:06:18.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:18.956+0000 7fb89e000640 1 -- 172.21.15.86:0/3157305387 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb894016110 con 0x7fb8a4070e20 2024-04-25T14:06:18.997 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:18.996+0000 7fc57b0d6180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:06:19.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.116+0000 7fb8ac800640 1 -- 172.21.15.86:0/3157305387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb8a4076bf0 con 0x7fb8a4070e20 2024-04-25T14:06:19.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.116+0000 7fb89e000640 1 -- 172.21.15.86:0/3157305387 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) ==== 93+0+10 (secure 0 0 0) 0x7fb8940f49b0 con 0x7fb8a4070e20 2024-04-25T14:06:19.118 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:06:19.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.116+0000 7fb8ac800640 1 -- 172.21.15.86:0/3157305387 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb88806f550 msgr2=0x7fb888071a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:06:19.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.116+0000 7fb8ac800640 1 --2- 172.21.15.86:0/3157305387 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb88806f550 0x7fb888071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.116+0000 7fb8ac800640 1 -- 172.21.15.86:0/3157305387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8a4070e20 msgr2=0x7fb8a4164990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:19.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.120+0000 7fb8ac800640 1 --2- 172.21.15.86:0/3157305387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8a4070e20 0x7fb8a4164990 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb89401cd30 tx=0x7fb894019230 comp rx=0 tx=0).stop 2024-04-25T14:06:19.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.120+0000 7fb8ac800640 1 -- 172.21.15.86:0/3157305387 shutdown_connections 2024-04-25T14:06:19.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.120+0000 7fb8ac800640 1 --2- 172.21.15.86:0/3157305387 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7fb88806f550 0x7fb888071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.120+0000 7fb8ac800640 1 --2- 172.21.15.86:0/3157305387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8a40cdd70 0x7fb8a4164ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.120+0000 7fb8ac800640 1 --2- 172.21.15.86:0/3157305387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8a4070e20 0x7fb8a4164990 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.120+0000 7fb8ac800640 1 --2- 172.21.15.86:0/3157305387 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8a406e700 0x7fb8a40cd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.120+0000 7fb8ac800640 1 -- 172.21.15.86:0/3157305387 >> 172.21.15.86:0/3157305387 conn(0x7fb8a40cffc0 msgr2=0x7fb8a415cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:19.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.120+0000 7fb8ac800640 1 -- 172.21.15.86:0/3157305387 shutdown_connections 2024-04-25T14:06:19.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.120+0000 7fb8ac800640 1 -- 172.21.15.86:0/3157305387 wait complete. 2024-04-25T14:06:19.133 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:06:19.136 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:06:19.140 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get fs1_metadata pg_num 2024-04-25T14:06:19.147 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:06:19.149+0000 7f95af866180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:06:19.215 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:06:19.213+0000 7f95af866180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:06:19.339 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:19.336+0000 7fc57b0d6180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:06:19.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.340+0000 7f79d7800640 1 Processor -- start 2024-04-25T14:06:19.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.340+0000 7f79d7800640 1 -- start start 2024-04-25T14:06:19.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.344+0000 7f79d7800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f79d00d2c80 0x7f79d015dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.344+0000 7f79d7800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f79d01616e0 0x7f79d0161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.344+0000 7f79d7800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79d00cd8d0 con 0x7f79d0070e20 2024-04-25T14:06:19.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.344+0000 7f79d7800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f79d00ce720 con 0x7f79d00d2c80 2024-04-25T14:06:19.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.344+0000 7f79d7800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f79d00ce970 con 0x7f79d01616e0 2024-04-25T14:06:19.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.344+0000 7f79d4e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f79d00d2c80 0x7f79d015dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.344+0000 7f79cfe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f79d0070e20 0x7f79d0071230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60968/0 (socket says 172.21.15.86:60968) 2024-04-25T14:06:19.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.344+0000 7f79cfe00640 1 -- 172.21.15.86:0/4133827290 learned_addr learned my addr 172.21.15.86:0/4133827290 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:19.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.344+0000 7f79d5800640 1 --2- 172.21.15.86:0/4133827290 >> v2:172.21.15.155:3300/0 conn(0x7f79d01616e0 0x7f79d0161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.344+0000 7f79cf400640 1 -- 172.21.15.86:0/4133827290 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1415755795 0 0) 0x7f79d00cd8d0 con 0x7f79d0070e20 2024-04-25T14:06:19.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.344+0000 7f79d4e00640 1 -- 172.21.15.86:0/4133827290 >> v2:172.21.15.155:3300/0 conn(0x7f79d01616e0 msgr2=0x7f79d0161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:19.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.344+0000 7f79d4e00640 1 --2- 172.21.15.86:0/4133827290 >> v2:172.21.15.155:3300/0 conn(0x7f79d01616e0 0x7f79d0161af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.344+0000 7f79d4e00640 1 -- 172.21.15.86:0/4133827290 >> v1:172.21.15.86:6789/0 conn(0x7f79d0070e20 legacy=0x7f79d0071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:19.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.344+0000 7f79d4e00640 1 -- 172.21.15.86:0/4133827290 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d00d1be0 con 0x7f79d00d2c80 2024-04-25T14:06:19.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.348+0000 7f79d4e00640 1 --2- 172.21.15.86:0/4133827290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79d00d2c80 0x7f79d015dc90 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f79d0074af0 tx=0x7f79c401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2205185a70f24ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:19.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.348+0000 7f79cf400640 1 -- 172.21.15.86:0/4133827290 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c40b5070 con 0x7f79d00d2c80 2024-04-25T14:06:19.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.348+0000 7f79cf400640 1 -- 172.21.15.86:0/4133827290 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79c40a8970 con 0x7f79d00d2c80 2024-04-25T14:06:19.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.348+0000 7f79cf400640 1 -- 172.21.15.86:0/4133827290 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c40a8d40 con 0x7f79d00d2c80 2024-04-25T14:06:19.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.348+0000 7f79d7800640 1 -- 172.21.15.86:0/4133827290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79d00d2c80 msgr2=0x7f79d015dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:19.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.348+0000 7f79d7800640 1 --2- 172.21.15.86:0/4133827290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79d00d2c80 0x7f79d015dc90 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f79d0074af0 tx=0x7f79c401ec70 comp rx=0 tx=0).stop 2024-04-25T14:06:19.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.348+0000 7f79d7800640 1 -- 172.21.15.86:0/4133827290 shutdown_connections 2024-04-25T14:06:19.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.348+0000 7f79d7800640 1 --2- 172.21.15.86:0/4133827290 >> v2:172.21.15.155:3300/0 conn(0x7f79d01616e0 0x7f79d0161af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.348+0000 7f79d7800640 1 --2- 172.21.15.86:0/4133827290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79d00d2c80 0x7f79d015dc90 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.348+0000 7f79d7800640 1 -- 172.21.15.86:0/4133827290 >> 172.21.15.86:0/4133827290 conn(0x7f79d00cffc0 msgr2=0x7f79d00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:19.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.348+0000 7f79d7800640 1 -- 172.21.15.86:0/4133827290 shutdown_connections 2024-04-25T14:06:19.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.348+0000 7f79d7800640 1 -- 172.21.15.86:0/4133827290 wait complete. 2024-04-25T14:06:19.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.348+0000 7f79d7800640 1 Processor -- start 2024-04-25T14:06:19.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.352+0000 7f79d7800640 1 -- start start 2024-04-25T14:06:19.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.352+0000 7f79d7800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79d0070e20 0x7f79d01f6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.352+0000 7f79d7800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79d00d2c80 0x7f79d006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.352+0000 7f79d7800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79d01616e0 0x7f79d01fa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.352+0000 7f79d7800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79d00d2480 con 0x7f79d00d2c80 2024-04-25T14:06:19.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.352+0000 7f79d7800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f79d00d21a0 con 0x7f79d0070e20 2024-04-25T14:06:19.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.352+0000 7f79d7800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f79d00d29a0 con 0x7f79d01616e0 2024-04-25T14:06:19.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.352+0000 7f79cfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79d00d2c80 0x7f79d006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.352+0000 7f79cfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79d00d2c80 0x7f79d006d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55086/0 (socket says 172.21.15.86:55086) 2024-04-25T14:06:19.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.352+0000 7f79cfe00640 1 -- 172.21.15.86:0/722949885 learned_addr learned my addr 172.21.15.86:0/722949885 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:19.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.352+0000 7f79d4e00640 1 --2- 172.21.15.86:0/722949885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79d0070e20 0x7f79d01f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.352+0000 7f79d5800640 1 --2- 172.21.15.86:0/722949885 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79d01616e0 0x7f79d01fa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.352+0000 7f79cfe00640 1 -- 172.21.15.86:0/722949885 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79d01616e0 msgr2=0x7f79d01fa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:19.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.352+0000 7f79cfe00640 1 --2- 172.21.15.86:0/722949885 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79d01616e0 0x7f79d01fa5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.352+0000 7f79cfe00640 1 -- 172.21.15.86:0/722949885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79d0070e20 msgr2=0x7f79d01f6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:19.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.352+0000 7f79cfe00640 1 --2- 172.21.15.86:0/722949885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79d0070e20 0x7f79d01f6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.352+0000 7f79cfe00640 1 -- 172.21.15.86:0/722949885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d006ec90 con 0x7f79d00d2c80 2024-04-25T14:06:19.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.352+0000 7f79d4e00640 1 --2- 172.21.15.86:0/722949885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79d0070e20 0x7f79d01f6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:19.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.352+0000 7f79cfe00640 1 --2- 172.21.15.86:0/722949885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79d00d2c80 0x7f79d006d780 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f79bc0010f0 tx=0x7f79bc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:19.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.352+0000 7f79ccc00640 1 -- 172.21.15.86:0/722949885 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc017070 con 0x7f79d00d2c80 2024-04-25T14:06:19.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.352+0000 7f79ccc00640 1 -- 172.21.15.86:0/722949885 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79bc0074e0 con 0x7f79d00d2c80 2024-04-25T14:06:19.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.352+0000 7f79d7800640 1 -- 172.21.15.86:0/722949885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79d006f250 con 0x7f79d00d2c80 2024-04-25T14:06:19.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.352+0000 7f79ccc00640 1 -- 172.21.15.86:0/722949885 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc0059f0 con 0x7f79d00d2c80 2024-04-25T14:06:19.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.352+0000 7f79d7800640 1 -- 172.21.15.86:0/722949885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79d01f9db0 con 0x7f79d00d2c80 2024-04-25T14:06:19.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.352+0000 7f79d7800640 1 -- 172.21.15.86:0/722949885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79d00720e0 con 0x7f79d00d2c80 2024-04-25T14:06:19.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.356+0000 7f79ccc00640 1 -- 172.21.15.86:0/722949885 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 93309+0+0 (secure 0 0 0) 0x7f79bc01a070 con 0x7f79d00d2c80 2024-04-25T14:06:19.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.356+0000 7f79ccc00640 1 --2- 172.21.15.86:0/722949885 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f79b006f520 0x7f79b0071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.356+0000 7f79d4e00640 1 -- 172.21.15.86:0/722949885 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f79b006f520 msgr2=0x7f79b0071a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2427179681 2024-04-25T14:06:19.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.356+0000 7f79ccc00640 1 -- 172.21.15.86:0/722949885 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 7531+0+0 (secure 0 0 0) 0x7f79bc08eec0 con 0x7f79d00d2c80 2024-04-25T14:06:19.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.356+0000 7f79d4e00640 1 --2- 172.21.15.86:0/722949885 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f79b006f520 0x7f79b0071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:06:19.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.360+0000 7f79ccc00640 1 -- 172.21.15.86:0/722949885 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f79bc05b610 con 0x7f79d00d2c80 2024-04-25T14:06:19.405 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:19.404+0000 7fc57b0d6180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:06:19.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.416+0000 7f79ccc00640 1 -- 172.21.15.86:0/722949885 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 14) ==== 92918+0+0 (secure 0 0 0) 0x7f79bc08ccc0 con 0x7f79d00d2c80 2024-04-25T14:06:19.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.416+0000 7f79ccc00640 1 -- 172.21.15.86:0/722949885 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f79b006f520 msgr2=0x7f79b0071a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:06:19.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.416+0000 7f79ccc00640 1 --2- 172.21.15.86:0/722949885 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f79b006f520 0x7f79b0071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.495 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:19.492+0000 7fc4fd600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:06:19.495 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:19.492+0000 7fc4fd600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:06:19.495 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:19.492+0000 7fc4fd600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:06:19.496 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:19.492+0000 7fc4fd600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:06:19.496 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:19.492+0000 7fc4fd600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:06:19.496 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:19.492+0000 7fc4f9a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:06:19.496 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:19.492+0000 7fc4f9a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:06:19.496 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:19.492+0000 7fc4f9a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:06:19.496 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:19.492+0000 7fc4f9a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:06:19.496 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:19.492+0000 7fc4f9a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:06:19.514 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:19.512+0000 7fc4f9a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:06:19.514 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:19.512+0000 7fc4f9a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:06:19.514 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:19.512+0000 7fc4f9a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:06:19.514 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:19.512+0000 7fc4f9a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:06:19.514 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:19.512+0000 7fc4f9a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:06:19.515 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:19.512+0000 7fc4fd600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:06:19.515 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:19.512+0000 7fc4fd600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:06:19.515 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:19.512+0000 7fc4fd600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:06:19.515 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:19.512+0000 7fc4fd600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:06:19.515 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:19.512+0000 7fc4fd600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:06:19.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.556+0000 7f79d7800640 1 -- 172.21.15.86:0/722949885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "fs1_metadata", "var": "pg_num"} v 0) -- 0x7f79d0076bf0 con 0x7f79d00d2c80 2024-04-25T14:06:19.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.556+0000 7f79ccc00640 1 -- 172.21.15.86:0/722949885 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "fs1_metadata", "var": "pg_num"}]=0 v59) ==== 101+0+11 (secure 0 0 0) 0x7f79bc07c860 con 0x7f79d00d2c80 2024-04-25T14:06:19.558 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:06:19.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.560+0000 7f79d7800640 1 -- 172.21.15.86:0/722949885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79d00d2c80 msgr2=0x7f79d006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:19.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.560+0000 7f79d7800640 1 --2- 172.21.15.86:0/722949885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79d00d2c80 0x7f79d006d780 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f79bc0010f0 tx=0x7f79bc00c8b0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.560+0000 7f79d7800640 1 -- 172.21.15.86:0/722949885 shutdown_connections 2024-04-25T14:06:19.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.560+0000 7f79d7800640 1 --2- 172.21.15.86:0/722949885 >> [v2:172.21.15.86:6832/2427179681,v1:172.21.15.86:6833/2427179681] conn(0x7f79b006f520 0x7f79b0071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.560+0000 7f79d7800640 1 --2- 172.21.15.86:0/722949885 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79d01616e0 0x7f79d01fa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.560+0000 7f79d7800640 1 --2- 172.21.15.86:0/722949885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79d00d2c80 0x7f79d006d780 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.560+0000 7f79d7800640 1 --2- 172.21.15.86:0/722949885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79d0070e20 0x7f79d01f6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.560+0000 7f79d7800640 1 -- 172.21.15.86:0/722949885 >> 172.21.15.86:0/722949885 conn(0x7f79d00cffc0 msgr2=0x7f79d0159cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:19.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.560+0000 7f79d7800640 1 -- 172.21.15.86:0/722949885 shutdown_connections 2024-04-25T14:06:19.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.560+0000 7f79d7800640 1 -- 172.21.15.86:0/722949885 wait complete. 2024-04-25T14:06:19.579 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get fs1_data pg_num 2024-04-25T14:06:19.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.728+0000 7f9d5ce00640 1 Processor -- start 2024-04-25T14:06:19.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.728+0000 7f9d5ce00640 1 -- start start 2024-04-25T14:06:19.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.728+0000 7f9d5ce00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d580d16e0 0x7f9d580d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.728+0000 7f9d5ce00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d580d2030 con 0x7f9d580cfca0 2024-04-25T14:06:19.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.728+0000 7f9d5ce00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d58072bd0 con 0x7f9d580730b0 2024-04-25T14:06:19.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.728+0000 7f9d5ce00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9d58070470 con 0x7f9d580d16e0 2024-04-25T14:06:19.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.728+0000 7f9d55e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9d580cfca0 0x7f9d5806f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60970/0 (socket says 172.21.15.86:60970) 2024-04-25T14:06:19.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.728+0000 7f9d55e00640 1 -- 172.21.15.86:0/2155159672 learned_addr learned my addr 172.21.15.86:0/2155159672 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:19.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.728+0000 7f9d55400640 1 --2- 172.21.15.86:0/2155159672 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d580d16e0 0x7f9d580d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.728+0000 7f9d54a00640 1 -- 172.21.15.86:0/2155159672 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 905667363 0 0) 0x7f9d48001120 con 0x7f9d580cfca0 2024-04-25T14:06:19.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.728+0000 7f9d54a00640 1 -- 172.21.15.86:0/2155159672 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d3c003530 con 0x7f9d580cfca0 2024-04-25T14:06:19.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.728+0000 7f9d54a00640 1 -- 172.21.15.86:0/2155159672 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1924614265 0 0) 0x7f9d3c003530 con 0x7f9d580cfca0 2024-04-25T14:06:19.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.728+0000 7f9d54a00640 1 -- 172.21.15.86:0/2155159672 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d580d16e0 msgr2=0x7f9d580d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:19.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.728+0000 7f9d54a00640 1 --2- 172.21.15.86:0/2155159672 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d580d16e0 0x7f9d580d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.728+0000 7f9d54a00640 1 -- 172.21.15.86:0/2155159672 >> v1:172.21.15.155:6789/0 conn(0x7f9d580730b0 legacy=0x7f9d580734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:19.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.728+0000 7f9d54a00640 1 -- 172.21.15.86:0/2155159672 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d5806fbd0 con 0x7f9d580cfca0 2024-04-25T14:06:19.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.728+0000 7f9d54a00640 1 -- 172.21.15.86:0/2155159672 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f9d48001850 con 0x7f9d580cfca0 2024-04-25T14:06:19.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.728+0000 7f9d54a00640 1 -- 172.21.15.86:0/2155159672 >> v1:172.21.15.86:6789/0 conn(0x7f9d580cfca0 legacy=0x7f9d5806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:19.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.728+0000 7f9d54a00640 1 --2- 172.21.15.86:0/2155159672 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d580d16e0 0x7f9d3c003a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.728+0000 7f9d54a00640 1 --2- 172.21.15.86:0/2155159672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d3c004000 0x7f9d3c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.728+0000 7f9d54a00640 1 --2- 172.21.15.86:0/2155159672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d3c004d50 0x7f9d3c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.728+0000 7f9d54a00640 1 -- 172.21.15.86:0/2155159672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d58070470 con 0x7f9d3c004000 2024-04-25T14:06:19.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.728+0000 7f9d54a00640 1 -- 172.21.15.86:0/2155159672 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c00bbe0 con 0x7f9d3c004d50 2024-04-25T14:06:19.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.728+0000 7f9d54a00640 1 -- 172.21.15.86:0/2155159672 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9d3c00bde0 con 0x7f9d580d16e0 2024-04-25T14:06:19.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.728+0000 7f9d55400640 1 --2- 172.21.15.86:0/2155159672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d3c004000 0x7f9d3c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.728+0000 7f9d55e00640 1 --2- 172.21.15.86:0/2155159672 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d580d16e0 0x7f9d3c003a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.732+0000 7f9d56800640 1 --2- 172.21.15.86:0/2155159672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d3c004d50 0x7f9d3c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.732+0000 7f9d55400640 1 -- 172.21.15.86:0/2155159672 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d580d16e0 msgr2=0x7f9d3c003a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:19.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.732+0000 7f9d55400640 1 --2- 172.21.15.86:0/2155159672 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d580d16e0 0x7f9d3c003a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.732+0000 7f9d55400640 1 -- 172.21.15.86:0/2155159672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d3c004d50 msgr2=0x7f9d3c005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:19.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.732+0000 7f9d55400640 1 --2- 172.21.15.86:0/2155159672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d3c004d50 0x7f9d3c005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.732+0000 7f9d55400640 1 -- 172.21.15.86:0/2155159672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d3c003530 con 0x7f9d3c004000 2024-04-25T14:06:19.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.732+0000 7f9d56800640 1 --2- 172.21.15.86:0/2155159672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d3c004d50 0x7f9d3c005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:19.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.732+0000 7f9d55400640 1 --2- 172.21.15.86:0/2155159672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d3c004000 0x7f9d3c004810 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9d44009ae0 tx=0x7f9d4401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef56595a5ed9e5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:19.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.732+0000 7f9d54a00640 1 -- 172.21.15.86:0/2155159672 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d440b2070 con 0x7f9d3c004000 2024-04-25T14:06:19.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.732+0000 7f9d54a00640 1 -- 172.21.15.86:0/2155159672 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d440a5570 con 0x7f9d3c004000 2024-04-25T14:06:19.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.732+0000 7f9d54a00640 1 -- 172.21.15.86:0/2155159672 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d440096b0 con 0x7f9d3c004000 2024-04-25T14:06:19.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.736+0000 7f9d5ce00640 1 -- 172.21.15.86:0/2155159672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d3c004000 msgr2=0x7f9d3c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:19.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.736+0000 7f9d5ce00640 1 --2- 172.21.15.86:0/2155159672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d3c004000 0x7f9d3c004810 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9d44009ae0 tx=0x7f9d4401ca90 comp rx=0 tx=0).stop 2024-04-25T14:06:19.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.736+0000 7f9d56800640 1 -- 172.21.15.86:0/2155159672 reap_dead start 2024-04-25T14:06:19.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.736+0000 7f9d5ce00640 1 -- 172.21.15.86:0/2155159672 shutdown_connections 2024-04-25T14:06:19.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.736+0000 7f9d5ce00640 1 -- 172.21.15.86:0/2155159672 >> 172.21.15.86:0/2155159672 conn(0x7f9d580b4650 msgr2=0x7f9d580b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:19.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.736+0000 7f9d5ce00640 1 -- 172.21.15.86:0/2155159672 shutdown_connections 2024-04-25T14:06:19.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.736+0000 7f9d5ce00640 1 -- 172.21.15.86:0/2155159672 wait complete. 2024-04-25T14:06:19.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.736+0000 7f9d5ce00640 1 Processor -- start 2024-04-25T14:06:19.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.736+0000 7f9d5ce00640 1 -- start start 2024-04-25T14:06:19.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.736+0000 7f9d5ce00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d3c004000 0x7f9d580b7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.740+0000 7f9d5ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d581f92c0 0x7f9d580cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.740+0000 7f9d5ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d580ce2a0 0x7f9d580b9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.740+0000 7f9d5ce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d580d1af0 con 0x7f9d580ce2a0 2024-04-25T14:06:19.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.740+0000 7f9d5ce00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9d58070190 con 0x7f9d581f92c0 2024-04-25T14:06:19.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.740+0000 7f9d5ce00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9d580d1c70 con 0x7f9d3c004000 2024-04-25T14:06:19.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.740+0000 7f9d56800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d580ce2a0 0x7f9d580b9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.740+0000 7f9d56800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d580ce2a0 0x7f9d580b9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55250/0 (socket says 172.21.15.86:55250) 2024-04-25T14:06:19.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.740+0000 7f9d56800640 1 -- 172.21.15.86:0/1690604163 learned_addr learned my addr 172.21.15.86:0/1690604163 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:19.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.740+0000 7f9d55e00640 1 --2- 172.21.15.86:0/1690604163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d3c004000 0x7f9d580b7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.740+0000 7f9d55400640 1 --2- 172.21.15.86:0/1690604163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d581f92c0 0x7f9d580cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.740+0000 7f9d56800640 1 -- 172.21.15.86:0/1690604163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d3c004000 msgr2=0x7f9d580b7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:19.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.740+0000 7f9d56800640 1 --2- 172.21.15.86:0/1690604163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d3c004000 0x7f9d580b7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.740+0000 7f9d56800640 1 -- 172.21.15.86:0/1690604163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d581f92c0 msgr2=0x7f9d580cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:19.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.740+0000 7f9d56800640 1 --2- 172.21.15.86:0/1690604163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d581f92c0 0x7f9d580cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.740+0000 7f9d56800640 1 -- 172.21.15.86:0/1690604163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d580ce6b0 con 0x7f9d580ce2a0 2024-04-25T14:06:19.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.740+0000 7f9d56800640 1 --2- 172.21.15.86:0/1690604163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d580ce2a0 0x7f9d580b9820 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f9d580b3f10 tx=0x7f9d5000ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:19.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.740+0000 7f9d4e000640 1 -- 172.21.15.86:0/1690604163 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d50003b90 con 0x7f9d580ce2a0 2024-04-25T14:06:19.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.740+0000 7f9d4e000640 1 -- 172.21.15.86:0/1690604163 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d50003d50 con 0x7f9d580ce2a0 2024-04-25T14:06:19.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.740+0000 7f9d4e000640 1 -- 172.21.15.86:0/1690604163 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d50006b60 con 0x7f9d580ce2a0 2024-04-25T14:06:19.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.740+0000 7f9d5ce00640 1 -- 172.21.15.86:0/1690604163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d580c5a20 con 0x7f9d580ce2a0 2024-04-25T14:06:19.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.740+0000 7f9d5ce00640 1 -- 172.21.15.86:0/1690604163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d580c5fe0 con 0x7f9d580ce2a0 2024-04-25T14:06:19.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.740+0000 7f9d5ce00640 1 -- 172.21.15.86:0/1690604163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d5809f4b0 con 0x7f9d580ce2a0 2024-04-25T14:06:19.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.740+0000 7f9d4e000640 1 -- 172.21.15.86:0/1690604163 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 14) ==== 92918+0+0 (secure 0 0 0) 0x7f9d5001a020 con 0x7f9d580ce2a0 2024-04-25T14:06:19.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.744+0000 7f9d4e000640 1 -- 172.21.15.86:0/1690604163 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7875+0+0 (secure 0 0 0) 0x7f9d5008ebc0 con 0x7f9d580ce2a0 2024-04-25T14:06:19.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.748+0000 7f9d4e000640 1 -- 172.21.15.86:0/1690604163 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9d580ce6b0 con 0x7f9d580ce2a0 2024-04-25T14:06:19.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.904+0000 7f9d5ce00640 1 -- 172.21.15.86:0/1690604163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "fs1_data", "var": "pg_num"} v 0) -- 0x7f9d58069d00 con 0x7f9d580ce2a0 2024-04-25T14:06:19.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.904+0000 7f9d4e000640 1 -- 172.21.15.86:0/1690604163 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "fs1_data", "var": "pg_num"}]=0 v59) ==== 97+0+11 (secure 0 0 0) 0x7f9d5005b620 con 0x7f9d580ce2a0 2024-04-25T14:06:19.905 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:06:19.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.904+0000 7f9d5ce00640 1 -- 172.21.15.86:0/1690604163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d580ce2a0 msgr2=0x7f9d580b9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:19.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.904+0000 7f9d5ce00640 1 --2- 172.21.15.86:0/1690604163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d580ce2a0 0x7f9d580b9820 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f9d580b3f10 tx=0x7f9d5000ead0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.904+0000 7f9d5ce00640 1 -- 172.21.15.86:0/1690604163 shutdown_connections 2024-04-25T14:06:19.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.904+0000 7f9d5ce00640 1 --2- 172.21.15.86:0/1690604163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d580ce2a0 0x7f9d580b9820 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.904+0000 7f9d5ce00640 1 --2- 172.21.15.86:0/1690604163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d581f92c0 0x7f9d580cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.904+0000 7f9d5ce00640 1 --2- 172.21.15.86:0/1690604163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d3c004000 0x7f9d580b7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.904+0000 7f9d5ce00640 1 -- 172.21.15.86:0/1690604163 >> 172.21.15.86:0/1690604163 conn(0x7f9d580b4650 msgr2=0x7f9d5815c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:19.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.904+0000 7f9d5ce00640 1 -- 172.21.15.86:0/1690604163 shutdown_connections 2024-04-25T14:06:19.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:19.904+0000 7f9d5ce00640 1 -- 172.21.15.86:0/1690604163 wait complete. 2024-04-25T14:06:19.926 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get fs2_metadata pg_num 2024-04-25T14:06:20.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.076+0000 7f15fb200640 1 Processor -- start 2024-04-25T14:06:20.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.076+0000 7f15fb200640 1 -- start start 2024-04-25T14:06:20.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.076+0000 7f15fb200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f15f40d16e0 0x7f15f40d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.076+0000 7f15fb200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15f40730b0 0x7f15f40734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.076+0000 7f15fb200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15f40d2030 con 0x7f15f40cfca0 2024-04-25T14:06:20.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.076+0000 7f15fb200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f15f4070470 con 0x7f15f40d16e0 2024-04-25T14:06:20.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.076+0000 7f15fb200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f15f4072bd0 con 0x7f15f40730b0 2024-04-25T14:06:20.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.076+0000 7f15f9200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15f40730b0 0x7f15f40734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.076+0000 7f15f3400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f15f40d16e0 0x7f15f40d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.076+0000 7f15f9200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15f40730b0 0x7f15f40734c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:40682/0 (socket says 172.21.15.86:40682) 2024-04-25T14:06:20.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.076+0000 7f15f3400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f15f40d16e0 0x7f15f40d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:36886/0 (socket says 172.21.15.86:36886) 2024-04-25T14:06:20.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.076+0000 7f15f9200640 1 -- 172.21.15.86:0/1101711378 learned_addr learned my addr 172.21.15.86:0/1101711378 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:20.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.076+0000 7f15f9200640 1 -- 172.21.15.86:0/1101711378 >> v2:172.21.15.155:3300/0 conn(0x7f15f40d16e0 msgr2=0x7f15f40d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:20.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.076+0000 7f15f9200640 1 --2- 172.21.15.86:0/1101711378 >> v2:172.21.15.155:3300/0 conn(0x7f15f40d16e0 0x7f15f40d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.076+0000 7f15f9200640 1 -- 172.21.15.86:0/1101711378 >> v1:172.21.15.155:6789/0 conn(0x7f15f40cfca0 legacy=0x7f15f406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.076+0000 7f15f9200640 1 -- 172.21.15.86:0/1101711378 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f406fbd0 con 0x7f15f40730b0 2024-04-25T14:06:20.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.080+0000 7f15f9200640 1 --2- 172.21.15.86:0/1101711378 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15f40730b0 0x7f15f40734c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f15ec009e50 tx=0x7f15ec01ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acad529a9b61ae79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:20.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.080+0000 7f15f2a00640 1 -- 172.21.15.86:0/1101711378 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec0a5810 con 0x7f15f40730b0 2024-04-25T14:06:20.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.080+0000 7f15f2a00640 1 -- 172.21.15.86:0/1101711378 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15ec0a59b0 con 0x7f15f40730b0 2024-04-25T14:06:20.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.080+0000 7f15f2a00640 1 -- 172.21.15.86:0/1101711378 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec0a5d80 con 0x7f15f40730b0 2024-04-25T14:06:20.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.080+0000 7f15fb200640 1 -- 172.21.15.86:0/1101711378 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15f40730b0 msgr2=0x7f15f40734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.080+0000 7f15fb200640 1 --2- 172.21.15.86:0/1101711378 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15f40730b0 0x7f15f40734c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f15ec009e50 tx=0x7f15ec01ca30 comp rx=0 tx=0).stop 2024-04-25T14:06:20.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.080+0000 7f15fb200640 1 -- 172.21.15.86:0/1101711378 shutdown_connections 2024-04-25T14:06:20.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.080+0000 7f15fb200640 1 --2- 172.21.15.86:0/1101711378 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15f40730b0 0x7f15f40734c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.080+0000 7f15fb200640 1 --2- 172.21.15.86:0/1101711378 >> v2:172.21.15.155:3300/0 conn(0x7f15f40d16e0 0x7f15f40d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.080+0000 7f15fb200640 1 -- 172.21.15.86:0/1101711378 >> 172.21.15.86:0/1101711378 conn(0x7f15f40b4650 msgr2=0x7f15f40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:20.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.080+0000 7f15fb200640 1 -- 172.21.15.86:0/1101711378 shutdown_connections 2024-04-25T14:06:20.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.080+0000 7f15fb200640 1 -- 172.21.15.86:0/1101711378 wait complete. 2024-04-25T14:06:20.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.084+0000 7f15fb200640 1 Processor -- start 2024-04-25T14:06:20.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.084+0000 7f15fb200640 1 -- start start 2024-04-25T14:06:20.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.084+0000 7f15fb200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15f40730b0 0x7f15f40ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.084+0000 7f15fb200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15f40cfca0 0x7f15f41577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.084+0000 7f15fb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f40d16e0 0x7f15f4157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.084+0000 7f15fb200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f15f4169a70 con 0x7f15f40d16e0 2024-04-25T14:06:20.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.084+0000 7f15fb200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f15f41698f0 con 0x7f15f40730b0 2024-04-25T14:06:20.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.084+0000 7f15fb200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f15f4169bf0 con 0x7f15f40cfca0 2024-04-25T14:06:20.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.084+0000 7f15f9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f40d16e0 0x7f15f4157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.084+0000 7f15f9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f40d16e0 0x7f15f4157ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55262/0 (socket says 172.21.15.86:55262) 2024-04-25T14:06:20.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.084+0000 7f15f9200640 1 -- 172.21.15.86:0/346065991 learned_addr learned my addr 172.21.15.86:0/346065991 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:20.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.084+0000 7f15f3400640 1 --2- 172.21.15.86:0/346065991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15f40cfca0 0x7f15f41577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.084+0000 7f15f3e00640 1 --2- 172.21.15.86:0/346065991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15f40730b0 0x7f15f40ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.084+0000 7f15f9200640 1 -- 172.21.15.86:0/346065991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15f40cfca0 msgr2=0x7f15f41577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.084+0000 7f15f9200640 1 --2- 172.21.15.86:0/346065991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15f40cfca0 0x7f15f41577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.084+0000 7f15f9200640 1 -- 172.21.15.86:0/346065991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15f40730b0 msgr2=0x7f15f40ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.084+0000 7f15f9200640 1 --2- 172.21.15.86:0/346065991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15f40730b0 0x7f15f40ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.084+0000 7f15f9200640 1 -- 172.21.15.86:0/346065991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f40b2370 con 0x7f15f40d16e0 2024-04-25T14:06:20.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.084+0000 7f15f9200640 1 --2- 172.21.15.86:0/346065991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f40d16e0 0x7f15f4157ce0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0af040 tx=0x7f15ec0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:20.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.088+0000 7f15ebe00640 1 -- 172.21.15.86:0/346065991 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec0b8670 con 0x7f15f40d16e0 2024-04-25T14:06:20.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.088+0000 7f15ebe00640 1 -- 172.21.15.86:0/346065991 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15ec0be070 con 0x7f15f40d16e0 2024-04-25T14:06:20.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.088+0000 7f15fb200640 1 -- 172.21.15.86:0/346065991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f40b28a0 con 0x7f15f40d16e0 2024-04-25T14:06:20.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.088+0000 7f15ebe00640 1 -- 172.21.15.86:0/346065991 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec0b8a70 con 0x7f15f40d16e0 2024-04-25T14:06:20.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.088+0000 7f15fb200640 1 -- 172.21.15.86:0/346065991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f40ca6b0 con 0x7f15f40d16e0 2024-04-25T14:06:20.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.088+0000 7f15fb200640 1 -- 172.21.15.86:0/346065991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15f409f4b0 con 0x7f15f40d16e0 2024-04-25T14:06:20.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.088+0000 7f15ebe00640 1 -- 172.21.15.86:0/346065991 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 14) ==== 92918+0+0 (secure 0 0 0) 0x7f15ec0b8c10 con 0x7f15f40d16e0 2024-04-25T14:06:20.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.088+0000 7f15ebe00640 1 -- 172.21.15.86:0/346065991 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7875+0+0 (secure 0 0 0) 0x7f15ec12b010 con 0x7f15f40d16e0 2024-04-25T14:06:20.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.092+0000 7f15ebe00640 1 -- 172.21.15.86:0/346065991 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f15ec016620 con 0x7f15f40d16e0 2024-04-25T14:06:20.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.248+0000 7f15fb200640 1 -- 172.21.15.86:0/346065991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "fs2_metadata", "var": "pg_num"} v 0) -- 0x7f15f415d970 con 0x7f15f40d16e0 2024-04-25T14:06:20.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.248+0000 7f15ebe00640 1 -- 172.21.15.86:0/346065991 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "fs2_metadata", "var": "pg_num"}]=0 v59) ==== 101+0+11 (secure 0 0 0) 0x7f15ec0f7c00 con 0x7f15f40d16e0 2024-04-25T14:06:20.251 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:06:20.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.252+0000 7f15fb200640 1 -- 172.21.15.86:0/346065991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f40d16e0 msgr2=0x7f15f4157ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.252+0000 7f15fb200640 1 --2- 172.21.15.86:0/346065991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f40d16e0 0x7f15f4157ce0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0af040 tx=0x7f15ec0197d0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.252+0000 7f15fb200640 1 -- 172.21.15.86:0/346065991 shutdown_connections 2024-04-25T14:06:20.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.252+0000 7f15fb200640 1 --2- 172.21.15.86:0/346065991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f40d16e0 0x7f15f4157ce0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.252+0000 7f15fb200640 1 --2- 172.21.15.86:0/346065991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15f40cfca0 0x7f15f41577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.252+0000 7f15fb200640 1 --2- 172.21.15.86:0/346065991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15f40730b0 0x7f15f40ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.252+0000 7f15fb200640 1 -- 172.21.15.86:0/346065991 >> 172.21.15.86:0/346065991 conn(0x7f15f40b4650 msgr2=0x7f15f415c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:20.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.252+0000 7f15fb200640 1 -- 172.21.15.86:0/346065991 shutdown_connections 2024-04-25T14:06:20.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.252+0000 7f15fb200640 1 -- 172.21.15.86:0/346065991 wait complete. 2024-04-25T14:06:20.272 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get fs2_data pg_num 2024-04-25T14:06:20.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.432+0000 7f5b4b600640 1 Processor -- start 2024-04-25T14:06:20.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.432+0000 7f5b4b600640 1 -- start start 2024-04-25T14:06:20.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.432+0000 7f5b4b600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5b44071530 0x7f5b44163e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.432+0000 7f5b4b600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b440d1c10 con 0x7f5b4406ebb0 2024-04-25T14:06:20.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.432+0000 7f5b4b600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b440d1ef0 con 0x7f5b4406e7a0 2024-04-25T14:06:20.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.432+0000 7f5b4b600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f5b441609b0 con 0x7f5b44071530 2024-04-25T14:06:20.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.436+0000 7f5b43e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5b4406ebb0 0x7f5b44070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60982/0 (socket says 172.21.15.86:60982) 2024-04-25T14:06:20.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.436+0000 7f5b43e00640 1 -- 172.21.15.86:0/3228048443 learned_addr learned my addr 172.21.15.86:0/3228048443 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:20.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.436+0000 7f5b49600640 1 --2- 172.21.15.86:0/3228048443 >> v2:172.21.15.155:3300/0 conn(0x7f5b44071530 0x7f5b44163e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.436+0000 7f5b43400640 1 -- 172.21.15.86:0/3228048443 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 40519560 0 0) 0x7f5b440d1c10 con 0x7f5b4406ebb0 2024-04-25T14:06:20.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.436+0000 7f5b43400640 1 -- 172.21.15.86:0/3228048443 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b2c003530 con 0x7f5b4406ebb0 2024-04-25T14:06:20.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.436+0000 7f5b43400640 1 -- 172.21.15.86:0/3228048443 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1745489718 0 0) 0x7f5b440d1ef0 con 0x7f5b4406e7a0 2024-04-25T14:06:20.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.436+0000 7f5b43400640 1 -- 172.21.15.86:0/3228048443 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b440d1c10 con 0x7f5b4406e7a0 2024-04-25T14:06:20.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.436+0000 7f5b49600640 1 -- 172.21.15.86:0/3228048443 >> v1:172.21.15.155:6789/0 conn(0x7f5b4406e7a0 legacy=0x7f5b441602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.436+0000 7f5b49600640 1 -- 172.21.15.86:0/3228048443 >> v1:172.21.15.86:6789/0 conn(0x7f5b4406ebb0 legacy=0x7f5b44070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.436+0000 7f5b49600640 1 -- 172.21.15.86:0/3228048443 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b440d17d0 con 0x7f5b44071530 2024-04-25T14:06:20.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.436+0000 7f5b49600640 1 --2- 172.21.15.86:0/3228048443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b44071530 0x7f5b44163e10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009d90 tx=0x7f5b3c01c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b807d13832552dcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:20.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.440+0000 7f5b43400640 1 -- 172.21.15.86:0/3228048443 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c0b2070 con 0x7f5b44071530 2024-04-25T14:06:20.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.440+0000 7f5b43400640 1 -- 172.21.15.86:0/3228048443 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b3c0a57a0 con 0x7f5b44071530 2024-04-25T14:06:20.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.440+0000 7f5b43400640 1 -- 172.21.15.86:0/3228048443 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c0a5b50 con 0x7f5b44071530 2024-04-25T14:06:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.440+0000 7f5b4b600640 1 -- 172.21.15.86:0/3228048443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b44071530 msgr2=0x7f5b44163e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.440+0000 7f5b4b600640 1 --2- 172.21.15.86:0/3228048443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b44071530 0x7f5b44163e10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009d90 tx=0x7f5b3c01c8e0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.440+0000 7f5b4b600640 1 -- 172.21.15.86:0/3228048443 shutdown_connections 2024-04-25T14:06:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.440+0000 7f5b4b600640 1 --2- 172.21.15.86:0/3228048443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b44071530 0x7f5b44163e10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.440+0000 7f5b4b600640 1 -- 172.21.15.86:0/3228048443 >> 172.21.15.86:0/3228048443 conn(0x7f5b440cffc0 msgr2=0x7f5b440d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.440+0000 7f5b4b600640 1 -- 172.21.15.86:0/3228048443 shutdown_connections 2024-04-25T14:06:20.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.440+0000 7f5b4b600640 1 -- 172.21.15.86:0/3228048443 wait complete. 2024-04-25T14:06:20.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.440+0000 7f5b4b600640 1 Processor -- start 2024-04-25T14:06:20.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.440+0000 7f5b4b600640 1 -- start start 2024-04-25T14:06:20.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.444+0000 7f5b4b600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b4406e7a0 0x7f5b4414b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.444+0000 7f5b4b600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b4406ebb0 0x7f5b4414a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.444+0000 7f5b4b600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b4414d5a0 0x7f5b44161f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.444+0000 7f5b4b600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4416e7e0 con 0x7f5b4406ebb0 2024-04-25T14:06:20.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.444+0000 7f5b4b600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4416e660 con 0x7f5b4414d5a0 2024-04-25T14:06:20.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.444+0000 7f5b4b600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5b4416e960 con 0x7f5b4406e7a0 2024-04-25T14:06:20.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.444+0000 7f5b43e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b4406ebb0 0x7f5b4414a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.444+0000 7f5b43e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b4406ebb0 0x7f5b4414a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55270/0 (socket says 172.21.15.86:55270) 2024-04-25T14:06:20.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.444+0000 7f5b43e00640 1 -- 172.21.15.86:0/340339285 learned_addr learned my addr 172.21.15.86:0/340339285 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:20.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.444+0000 7f5b49600640 1 --2- 172.21.15.86:0/340339285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b4414d5a0 0x7f5b44161f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.444+0000 7f5b48c00640 1 --2- 172.21.15.86:0/340339285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b4406e7a0 0x7f5b4414b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.444+0000 7f5b43e00640 1 -- 172.21.15.86:0/340339285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b4406e7a0 msgr2=0x7f5b4414b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.444+0000 7f5b43e00640 1 --2- 172.21.15.86:0/340339285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b4406e7a0 0x7f5b4414b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.444+0000 7f5b43e00640 1 -- 172.21.15.86:0/340339285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b4414d5a0 msgr2=0x7f5b44161f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.444+0000 7f5b43e00640 1 --2- 172.21.15.86:0/340339285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b4414d5a0 0x7f5b44161f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.444+0000 7f5b43e00640 1 -- 172.21.15.86:0/340339285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4414adf0 con 0x7f5b4406ebb0 2024-04-25T14:06:20.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.444+0000 7f5b43e00640 1 --2- 172.21.15.86:0/340339285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b4406ebb0 0x7f5b4414a8b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000c4d0 tx=0x7f5b3000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:20.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.444+0000 7f5b40c00640 1 -- 172.21.15.86:0/340339285 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30017070 con 0x7f5b4406ebb0 2024-04-25T14:06:20.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.444+0000 7f5b40c00640 1 -- 172.21.15.86:0/340339285 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b30007480 con 0x7f5b4406ebb0 2024-04-25T14:06:20.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.444+0000 7f5b40c00640 1 -- 172.21.15.86:0/340339285 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30007630 con 0x7f5b4406ebb0 2024-04-25T14:06:20.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.444+0000 7f5b4b600640 1 -- 172.21.15.86:0/340339285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b44160ef0 con 0x7f5b4406ebb0 2024-04-25T14:06:20.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.444+0000 7f5b4b600640 1 -- 172.21.15.86:0/340339285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b441614b0 con 0x7f5b4406ebb0 2024-04-25T14:06:20.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.444+0000 7f5b4b600640 1 -- 172.21.15.86:0/340339285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b440720e0 con 0x7f5b4406ebb0 2024-04-25T14:06:20.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.448+0000 7f5b40c00640 1 -- 172.21.15.86:0/340339285 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 15) ==== 93045+0+0 (secure 0 0 0) 0x7f5b3001a070 con 0x7f5b4406ebb0 2024-04-25T14:06:20.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.448+0000 7f5b40c00640 1 --2- 172.21.15.86:0/340339285 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f5b2406f280 0x7f5b24071790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.448+0000 7f5b40c00640 1 -- 172.21.15.86:0/340339285 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7875+0+0 (secure 0 0 0) 0x7f5b3008ea10 con 0x7f5b4406ebb0 2024-04-25T14:06:20.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.448+0000 7f5b48c00640 1 --2- 172.21.15.86:0/340339285 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f5b2406f280 0x7f5b24071790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.448+0000 7f5b48c00640 1 --2- 172.21.15.86:0/340339285 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f5b2406f280 0x7f5b24071790 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5b380011f0 tx=0x7f5b38009040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:20.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.452+0000 7f5b40c00640 1 -- 172.21.15.86:0/340339285 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5b3005ad30 con 0x7f5b4406ebb0 2024-04-25T14:06:20.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.612+0000 7f5b4b600640 1 -- 172.21.15.86:0/340339285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "fs2_data", "var": "pg_num"} v 0) -- 0x7f5b44076bf0 con 0x7f5b4406ebb0 2024-04-25T14:06:20.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.616+0000 7f5b40c00640 1 -- 172.21.15.86:0/340339285 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "fs2_data", "var": "pg_num"}]=0 v59) ==== 97+0+11 (secure 0 0 0) 0x7f5b30063370 con 0x7f5b4406ebb0 2024-04-25T14:06:20.616 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:06:20.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.616+0000 7f5b4b600640 1 -- 172.21.15.86:0/340339285 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f5b2406f280 msgr2=0x7f5b24071790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.616+0000 7f5b4b600640 1 --2- 172.21.15.86:0/340339285 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f5b2406f280 0x7f5b24071790 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5b380011f0 tx=0x7f5b38009040 comp rx=0 tx=0).stop 2024-04-25T14:06:20.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.616+0000 7f5b4b600640 1 -- 172.21.15.86:0/340339285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b4406ebb0 msgr2=0x7f5b4414a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.616+0000 7f5b4b600640 1 --2- 172.21.15.86:0/340339285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b4406ebb0 0x7f5b4414a8b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000c4d0 tx=0x7f5b3000c990 comp rx=0 tx=0).stop 2024-04-25T14:06:20.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.616+0000 7f5b4b600640 1 -- 172.21.15.86:0/340339285 shutdown_connections 2024-04-25T14:06:20.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.616+0000 7f5b4b600640 1 --2- 172.21.15.86:0/340339285 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f5b2406f280 0x7f5b24071790 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.616+0000 7f5b4b600640 1 --2- 172.21.15.86:0/340339285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b4414d5a0 0x7f5b44161f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.616+0000 7f5b4b600640 1 --2- 172.21.15.86:0/340339285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b4406ebb0 0x7f5b4414a8b0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.616+0000 7f5b4b600640 1 --2- 172.21.15.86:0/340339285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b4406e7a0 0x7f5b4414b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.616+0000 7f5b4b600640 1 -- 172.21.15.86:0/340339285 >> 172.21.15.86:0/340339285 conn(0x7f5b440cffc0 msgr2=0x7f5b440b5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:20.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.616+0000 7f5b4b600640 1 -- 172.21.15.86:0/340339285 shutdown_connections 2024-04-25T14:06:20.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.616+0000 7f5b4b600640 1 -- 172.21.15.86:0/340339285 wait complete. 2024-04-25T14:06:20.637 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T14:06:20.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.784+0000 7fe4bdc00640 1 Processor -- start 2024-04-25T14:06:20.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.784+0000 7fe4bdc00640 1 -- start start 2024-04-25T14:06:20.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.784+0000 7fe4bdc00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe4b80d16e0 0x7fe4b80d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.784+0000 7fe4bdc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4b80d2030 con 0x7fe4b80cfca0 2024-04-25T14:06:20.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.784+0000 7fe4bdc00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4b8072bd0 con 0x7fe4b80730b0 2024-04-25T14:06:20.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.784+0000 7fe4bdc00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe4b8070470 con 0x7fe4b80d16e0 2024-04-25T14:06:20.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.784+0000 7fe4b6a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe4b80cfca0 0x7fe4b806f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60996/0 (socket says 172.21.15.86:60996) 2024-04-25T14:06:20.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.784+0000 7fe4b6a00640 1 -- 172.21.15.86:0/3765799580 learned_addr learned my addr 172.21.15.86:0/3765799580 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:20.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.784+0000 7fe4ae000640 1 --2- 172.21.15.86:0/3765799580 >> v2:172.21.15.155:3300/0 conn(0x7fe4b80d16e0 0x7fe4b80d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.784+0000 7fe4b6000640 1 -- 172.21.15.86:0/3765799580 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2965992525 0 0) 0x7fe4a80014a0 con 0x7fe4b80cfca0 2024-04-25T14:06:20.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.784+0000 7fe4b6000640 1 -- 172.21.15.86:0/3765799580 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe49c003530 con 0x7fe4b80cfca0 2024-04-25T14:06:20.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.784+0000 7fe4ae000640 1 -- 172.21.15.86:0/3765799580 >> v1:172.21.15.155:6789/0 conn(0x7fe4b80730b0 legacy=0x7fe4b80734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.784+0000 7fe4ae000640 1 -- 172.21.15.86:0/3765799580 >> v1:172.21.15.86:6789/0 conn(0x7fe4b80cfca0 legacy=0x7fe4b806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.784+0000 7fe4ae000640 1 -- 172.21.15.86:0/3765799580 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b806fbd0 con 0x7fe4b80d16e0 2024-04-25T14:06:20.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.788+0000 7fe4ae000640 1 --2- 172.21.15.86:0/3765799580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4b80d16e0 0x7fe4b80d1af0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00097c0 tx=0x7fe4a001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfb72f4de82c0c83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:20.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.788+0000 7fe4b6000640 1 -- 172.21.15.86:0/3765799580 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a00b1070 con 0x7fe4b80d16e0 2024-04-25T14:06:20.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.788+0000 7fe4b6000640 1 -- 172.21.15.86:0/3765799580 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4a00ac410 con 0x7fe4b80d16e0 2024-04-25T14:06:20.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.788+0000 7fe4b6000640 1 -- 172.21.15.86:0/3765799580 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b80743e0 con 0x7fe4b80d16e0 2024-04-25T14:06:20.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.788+0000 7fe4bdc00640 1 -- 172.21.15.86:0/3765799580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4b80d16e0 msgr2=0x7fe4b80d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.788+0000 7fe4bdc00640 1 --2- 172.21.15.86:0/3765799580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4b80d16e0 0x7fe4b80d1af0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00097c0 tx=0x7fe4a001c4a0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.788+0000 7fe4bdc00640 1 -- 172.21.15.86:0/3765799580 shutdown_connections 2024-04-25T14:06:20.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.788+0000 7fe4bdc00640 1 --2- 172.21.15.86:0/3765799580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4b80d16e0 0x7fe4b80d1af0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.788+0000 7fe4bdc00640 1 -- 172.21.15.86:0/3765799580 >> 172.21.15.86:0/3765799580 conn(0x7fe4b80b4650 msgr2=0x7fe4b80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:20.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.788+0000 7fe4bdc00640 1 -- 172.21.15.86:0/3765799580 shutdown_connections 2024-04-25T14:06:20.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.788+0000 7fe4bdc00640 1 -- 172.21.15.86:0/3765799580 wait complete. 2024-04-25T14:06:20.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.792+0000 7fe4bdc00640 1 Processor -- start 2024-04-25T14:06:20.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.792+0000 7fe4bdc00640 1 -- start start 2024-04-25T14:06:20.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.792+0000 7fe4bdc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4b80730b0 0x7fe4b809a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.792+0000 7fe4bdc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4b80cfca0 0x7fe4b815f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.792+0000 7fe4bdc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4b80d16e0 0x7fe4b80b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.792+0000 7fe4bdc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b816aa80 con 0x7fe4b80cfca0 2024-04-25T14:06:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.792+0000 7fe4bdc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b816a900 con 0x7fe4b80d16e0 2024-04-25T14:06:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.792+0000 7fe4bdc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b816ac00 con 0x7fe4b80730b0 2024-04-25T14:06:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.792+0000 7fe4ae000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4b80cfca0 0x7fe4b815f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.792+0000 7fe4ae000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4b80cfca0 0x7fe4b815f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55286/0 (socket says 172.21.15.86:55286) 2024-04-25T14:06:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.792+0000 7fe4ae000640 1 -- 172.21.15.86:0/3652880089 learned_addr learned my addr 172.21.15.86:0/3652880089 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.792+0000 7fe4b6a00640 1 --2- 172.21.15.86:0/3652880089 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4b80730b0 0x7fe4b809a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.792+0000 7fe4b7400640 1 --2- 172.21.15.86:0/3652880089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4b80d16e0 0x7fe4b80b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.796+0000 7fe4ae000640 1 -- 172.21.15.86:0/3652880089 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4b80730b0 msgr2=0x7fe4b809a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.796+0000 7fe4ae000640 1 --2- 172.21.15.86:0/3652880089 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4b80730b0 0x7fe4b809a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.796+0000 7fe4ae000640 1 -- 172.21.15.86:0/3652880089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4b80d16e0 msgr2=0x7fe4b80b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.796+0000 7fe4ae000640 1 --2- 172.21.15.86:0/3652880089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4b80d16e0 0x7fe4b80b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.796+0000 7fe4ae000640 1 -- 172.21.15.86:0/3652880089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b815e9d0 con 0x7fe4b80cfca0 2024-04-25T14:06:20.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.796+0000 7fe4ae000640 1 --2- 172.21.15.86:0/3652880089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4b80cfca0 0x7fe4b815f6e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00097c0 tx=0x7fe4a0019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:20.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.796+0000 7fe4af400640 1 -- 172.21.15.86:0/3652880089 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a00bc070 con 0x7fe4b80cfca0 2024-04-25T14:06:20.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.796+0000 7fe4af400640 1 -- 172.21.15.86:0/3652880089 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4a00acd70 con 0x7fe4b80cfca0 2024-04-25T14:06:20.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.796+0000 7fe4af400640 1 -- 172.21.15.86:0/3652880089 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a00b1040 con 0x7fe4b80cfca0 2024-04-25T14:06:20.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.796+0000 7fe4bdc00640 1 -- 172.21.15.86:0/3652880089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b815ecb0 con 0x7fe4b80cfca0 2024-04-25T14:06:20.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.796+0000 7fe4bdc00640 1 -- 172.21.15.86:0/3652880089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b80c92c0 con 0x7fe4b80cfca0 2024-04-25T14:06:20.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.796+0000 7fe4bdc00640 1 -- 172.21.15.86:0/3652880089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4b809f4b0 con 0x7fe4b80cfca0 2024-04-25T14:06:20.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.796+0000 7fe4af400640 1 -- 172.21.15.86:0/3652880089 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 15) ==== 93045+0+0 (secure 0 0 0) 0x7fe4a0004230 con 0x7fe4b80cfca0 2024-04-25T14:06:20.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.800+0000 7fe4af400640 1 --2- 172.21.15.86:0/3652880089 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fe49406f250 0x7fe494071760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.800+0000 7fe4af400640 1 -- 172.21.15.86:0/3652880089 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7875+0+0 (secure 0 0 0) 0x7fe4a0128670 con 0x7fe4b80cfca0 2024-04-25T14:06:20.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.800+0000 7fe4b6a00640 1 --2- 172.21.15.86:0/3652880089 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fe49406f250 0x7fe494071760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.800+0000 7fe4b6a00640 1 --2- 172.21.15.86:0/3652880089 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fe49406f250 0x7fe494071760 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8001640 tx=0x7fe4a8009040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:20.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.804+0000 7fe4af400640 1 -- 172.21.15.86:0/3652880089 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe4a0016090 con 0x7fe4b80cfca0 2024-04-25T14:06:20.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.972+0000 7fe4bdc00640 1 -- 172.21.15.86:0/3652880089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe4b8069d00 con 0x7fe4b80cfca0 2024-04-25T14:06:20.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.972+0000 7fe4af400640 1 -- 172.21.15.86:0/3652880089 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 22 entries v59) ==== 81+0+1306 (secure 0 0 0) 0x7fe4a00f5070 con 0x7fe4b80cfca0 2024-04-25T14:06:20.974 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/145108428 2024-04-26T14:06:19.409429+0000 2024-04-25T14:06:20.975 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6833/2427179681 2024-04-26T14:06:19.409429+0000 2024-04-25T14:06:20.975 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6832/2427179681 2024-04-26T14:06:19.409429+0000 2024-04-25T14:06:20.975 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1321309855 2024-04-26T14:06:19.409429+0000 2024-04-25T14:06:20.975 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/525010030 2024-04-26T14:06:19.409429+0000 2024-04-25T14:06:20.975 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3470358079 2024-04-26T14:06:19.409429+0000 2024-04-25T14:06:20.975 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1177884430 2024-04-26T14:06:19.409429+0000 2024-04-25T14:06:20.975 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3644218120 2024-04-26T14:06:19.409429+0000 2024-04-25T14:06:20.975 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6836/1732615622 2024-04-26T14:06:00.230367+0000 2024-04-25T14:06:20.975 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6837/716923566 2024-04-26T14:05:25.452701+0000 2024-04-25T14:06:20.975 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6837/2418778265 2024-04-26T14:06:00.222915+0000 2024-04-25T14:06:20.975 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1843278167 2024-04-26T14:05:21.460855+0000 2024-04-25T14:06:20.975 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6832/2968869755 2024-04-26T14:05:21.460855+0000 2024-04-25T14:06:20.975 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2073988773 2024-04-26T14:05:21.460855+0000 2024-04-25T14:06:20.975 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6835/716923566 2024-04-26T14:05:25.452701+0000 2024-04-25T14:06:20.976 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2602032945 2024-04-26T14:05:21.460855+0000 2024-04-25T14:06:20.976 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3558294881 2024-04-26T14:05:21.460855+0000 2024-04-25T14:06:20.976 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6835/615772001 2024-04-26T14:05:42.882867+0000 2024-04-25T14:06:20.976 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6837/615772001 2024-04-26T14:05:42.882867+0000 2024-04-25T14:06:20.976 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6833/2968869755 2024-04-26T14:05:21.460855+0000 2024-04-25T14:06:20.976 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6835/2418778265 2024-04-26T14:06:00.222915+0000 2024-04-25T14:06:20.976 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6834/1732615622 2024-04-26T14:06:00.230367+0000 2024-04-25T14:06:20.977 INFO:teuthology.orchestra.run.smithi086.stderr:listed 22 entries 2024-04-25T14:06:20.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.976+0000 7fe4bdc00640 1 -- 172.21.15.86:0/3652880089 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fe49406f250 msgr2=0x7fe494071760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.976+0000 7fe4bdc00640 1 --2- 172.21.15.86:0/3652880089 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fe49406f250 0x7fe494071760 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8001640 tx=0x7fe4a8009040 comp rx=0 tx=0).stop 2024-04-25T14:06:20.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.976+0000 7fe4bdc00640 1 -- 172.21.15.86:0/3652880089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4b80cfca0 msgr2=0x7fe4b815f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.976+0000 7fe4bdc00640 1 --2- 172.21.15.86:0/3652880089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4b80cfca0 0x7fe4b815f6e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00097c0 tx=0x7fe4a0019240 comp rx=0 tx=0).stop 2024-04-25T14:06:20.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.976+0000 7fe4bdc00640 1 -- 172.21.15.86:0/3652880089 shutdown_connections 2024-04-25T14:06:20.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.976+0000 7fe4bdc00640 1 --2- 172.21.15.86:0/3652880089 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fe49406f250 0x7fe494071760 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.976+0000 7fe4bdc00640 1 --2- 172.21.15.86:0/3652880089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4b80d16e0 0x7fe4b80b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.976+0000 7fe4bdc00640 1 --2- 172.21.15.86:0/3652880089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4b80cfca0 0x7fe4b815f6e0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.976+0000 7fe4bdc00640 1 --2- 172.21.15.86:0/3652880089 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4b80730b0 0x7fe4b809a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.976+0000 7fe4bdc00640 1 -- 172.21.15.86:0/3652880089 >> 172.21.15.86:0/3652880089 conn(0x7fe4b80b4650 msgr2=0x7fe4b80b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:20.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.976+0000 7fe4bdc00640 1 -- 172.21.15.86:0/3652880089 shutdown_connections 2024-04-25T14:06:20.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:20.976+0000 7fe4bdc00640 1 -- 172.21.15.86:0/3652880089 wait complete. 2024-04-25T14:06:20.996 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T14:06:20.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:20.996 DEBUG:teuthology.orchestra.run.smithi155:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T14:06:21.068 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T14:06:21.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:21.069 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T14:06:21.088 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:06:21.089 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:06:21.089 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:06:21.099 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2893 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T14:06:21.099 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:06:21.099 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:06:21.099 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:06:21.143 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4517 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T14:06:21.143 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:06:21.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.288+0000 7f0d93c00640 1 Processor -- start 2024-04-25T14:06:21.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.288+0000 7f0d93c00640 1 -- start start 2024-04-25T14:06:21.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.292+0000 7f0d93c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d8c1610d0 0x7f0d8c0b7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.292+0000 7f0d93c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0d8c0b2250 0x7f0d8c0b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.292+0000 7f0d93c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d8c0b3750 con 0x7f0d8c0b2ba0 2024-04-25T14:06:21.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.292+0000 7f0d93c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0d8c0b7f70 con 0x7f0d8c0b2250 2024-04-25T14:06:21.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.292+0000 7f0d93c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0d8c161620 con 0x7f0d8c1610d0 2024-04-25T14:06:21.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.292+0000 7f0d91c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0d8c0b2ba0 0x7f0d8c0d15e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:32780/0 (socket says 172.21.15.86:32780) 2024-04-25T14:06:21.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.292+0000 7f0d91c00640 1 -- 172.21.15.86:0/1640417933 learned_addr learned my addr 172.21.15.86:0/1640417933 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:21.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.292+0000 7f0d83e00640 1 --2- 172.21.15.86:0/1640417933 >> v2:172.21.15.155:3300/0 conn(0x7f0d8c0b2250 0x7f0d8c0b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:21.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.292+0000 7f0d91200640 1 --2- 172.21.15.86:0/1640417933 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d8c1610d0 0x7f0d8c0b7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:21.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.292+0000 7f0d8be00640 1 -- 172.21.15.86:0/1640417933 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 968221766 0 0) 0x7f0d8c0b3750 con 0x7f0d8c0b2ba0 2024-04-25T14:06:21.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.292+0000 7f0d8be00640 1 -- 172.21.15.86:0/1640417933 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d70003530 con 0x7f0d8c0b2ba0 2024-04-25T14:06:21.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.292+0000 7f0d91200640 1 -- 172.21.15.86:0/1640417933 >> v2:172.21.15.155:3300/0 conn(0x7f0d8c0b2250 msgr2=0x7f0d8c0b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:21.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.292+0000 7f0d91200640 1 --2- 172.21.15.86:0/1640417933 >> v2:172.21.15.155:3300/0 conn(0x7f0d8c0b2250 0x7f0d8c0b2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.292+0000 7f0d91200640 1 -- 172.21.15.86:0/1640417933 >> v1:172.21.15.86:6789/0 conn(0x7f0d8c0b2ba0 legacy=0x7f0d8c0d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:21.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.292+0000 7f0d91200640 1 -- 172.21.15.86:0/1640417933 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8c0ce7b0 con 0x7f0d8c1610d0 2024-04-25T14:06:21.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.292+0000 7f0d83e00640 1 --2- 172.21.15.86:0/1640417933 >> v2:172.21.15.155:3300/0 conn(0x7f0d8c0b2250 0x7f0d8c0b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:21.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.296+0000 7f0d91200640 1 --2- 172.21.15.86:0/1640417933 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d8c1610d0 0x7f0d8c0b7a30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c0097c0 tx=0x7f0d7c01c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=688cd8de4bb1bc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:21.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.296+0000 7f0d8be00640 1 -- 172.21.15.86:0/1640417933 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c0b0070 con 0x7f0d8c1610d0 2024-04-25T14:06:21.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.296+0000 7f0d8be00640 1 -- 172.21.15.86:0/1640417933 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d7c0ab600 con 0x7f0d8c1610d0 2024-04-25T14:06:21.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.296+0000 7f0d8be00640 1 -- 172.21.15.86:0/1640417933 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c0ab9d0 con 0x7f0d8c1610d0 2024-04-25T14:06:21.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.296+0000 7f0d93c00640 1 -- 172.21.15.86:0/1640417933 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d8c1610d0 msgr2=0x7f0d8c0b7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:21.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.296+0000 7f0d93c00640 1 --2- 172.21.15.86:0/1640417933 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d8c1610d0 0x7f0d8c0b7a30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c0097c0 tx=0x7f0d7c01c3e0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.296+0000 7f0d93c00640 1 -- 172.21.15.86:0/1640417933 shutdown_connections 2024-04-25T14:06:21.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.296+0000 7f0d93c00640 1 --2- 172.21.15.86:0/1640417933 >> v2:172.21.15.155:3300/0 conn(0x7f0d8c0b2250 0x7f0d8c0b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.296+0000 7f0d93c00640 1 --2- 172.21.15.86:0/1640417933 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d8c1610d0 0x7f0d8c0b7a30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.296+0000 7f0d93c00640 1 -- 172.21.15.86:0/1640417933 >> 172.21.15.86:0/1640417933 conn(0x7f0d8c09ba30 msgr2=0x7f0d8c0b7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:21.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.296+0000 7f0d93c00640 1 -- 172.21.15.86:0/1640417933 shutdown_connections 2024-04-25T14:06:21.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.296+0000 7f0d93c00640 1 -- 172.21.15.86:0/1640417933 wait complete. 2024-04-25T14:06:21.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.300+0000 7f0d93c00640 1 Processor -- start 2024-04-25T14:06:21.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.300+0000 7f0d93c00640 1 -- start start 2024-04-25T14:06:21.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.300+0000 7f0d93c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d8c0b2250 0x7f0d8c15b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.300+0000 7f0d93c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d8c0b2ba0 0x7f0d8c146ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.300+0000 7f0d93c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d8c1610d0 0x7f0d8c0b7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.300+0000 7f0d93c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c0d2280 con 0x7f0d8c1610d0 2024-04-25T14:06:21.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.300+0000 7f0d93c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c0d2100 con 0x7f0d8c0b2ba0 2024-04-25T14:06:21.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.300+0000 7f0d93c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0d8c0cf610 con 0x7f0d8c0b2250 2024-04-25T14:06:21.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.300+0000 7f0d91c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d8c1610d0 0x7f0d8c0b7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:21.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.300+0000 7f0d91c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d8c1610d0 0x7f0d8c0b7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55294/0 (socket says 172.21.15.86:55294) 2024-04-25T14:06:21.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.300+0000 7f0d91c00640 1 -- 172.21.15.86:0/2765453824 learned_addr learned my addr 172.21.15.86:0/2765453824 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:21.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.300+0000 7f0d83e00640 1 --2- 172.21.15.86:0/2765453824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d8c0b2ba0 0x7f0d8c146ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:21.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.300+0000 7f0d91200640 1 --2- 172.21.15.86:0/2765453824 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d8c0b2250 0x7f0d8c15b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:21.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.300+0000 7f0d91c00640 1 -- 172.21.15.86:0/2765453824 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d8c0b2250 msgr2=0x7f0d8c15b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:21.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.300+0000 7f0d91c00640 1 --2- 172.21.15.86:0/2765453824 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d8c0b2250 0x7f0d8c15b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.300+0000 7f0d91c00640 1 -- 172.21.15.86:0/2765453824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d8c0b2ba0 msgr2=0x7f0d8c146ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:21.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.300+0000 7f0d91c00640 1 --2- 172.21.15.86:0/2765453824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d8c0b2ba0 0x7f0d8c146ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.300+0000 7f0d91c00640 1 -- 172.21.15.86:0/2765453824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8c15c0d0 con 0x7f0d8c1610d0 2024-04-25T14:06:21.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.300+0000 7f0d91c00640 1 --2- 172.21.15.86:0/2765453824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d8c1610d0 0x7f0d8c0b7220 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400ca70 tx=0x7f0d8400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:21.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.300+0000 7f0d89600640 1 -- 172.21.15.86:0/2765453824 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d84017070 con 0x7f0d8c1610d0 2024-04-25T14:06:21.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.300+0000 7f0d89600640 1 -- 172.21.15.86:0/2765453824 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d8400abd0 con 0x7f0d8c1610d0 2024-04-25T14:06:21.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.300+0000 7f0d89600640 1 -- 172.21.15.86:0/2765453824 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d84012400 con 0x7f0d8c1610d0 2024-04-25T14:06:21.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.300+0000 7f0d93c00640 1 -- 172.21.15.86:0/2765453824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d8c15c3b0 con 0x7f0d8c1610d0 2024-04-25T14:06:21.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.300+0000 7f0d93c00640 1 -- 172.21.15.86:0/2765453824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d8c15c970 con 0x7f0d8c1610d0 2024-04-25T14:06:21.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.304+0000 7f0d93c00640 1 -- 172.21.15.86:0/2765453824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d8c147790 con 0x7f0d8c1610d0 2024-04-25T14:06:21.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.304+0000 7f0d89600640 1 -- 172.21.15.86:0/2765453824 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 15) ==== 93045+0+0 (secure 0 0 0) 0x7f0d8401a070 con 0x7f0d8c1610d0 2024-04-25T14:06:21.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.304+0000 7f0d89600640 1 --2- 172.21.15.86:0/2765453824 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f0d6806f250 0x7f0d68071760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.304+0000 7f0d89600640 1 -- 172.21.15.86:0/2765453824 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7875+0+0 (secure 0 0 0) 0x7f0d8408f400 con 0x7f0d8c1610d0 2024-04-25T14:06:21.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.304+0000 7f0d91200640 1 --2- 172.21.15.86:0/2765453824 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f0d6806f250 0x7f0d68071760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:21.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.304+0000 7f0d91200640 1 --2- 172.21.15.86:0/2765453824 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f0d6806f250 0x7f0d68071760 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c0ad000 tx=0x7f0d7c008040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:21.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.308+0000 7f0d89600640 1 -- 172.21.15.86:0/2765453824 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0d8405b720 con 0x7f0d8c1610d0 2024-04-25T14:06:21.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.428+0000 7f0d89600640 1 -- 172.21.15.86:0/2765453824 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 16) ==== 93355+0+0 (secure 0 0 0) 0x7f0d840579b0 con 0x7f0d8c1610d0 2024-04-25T14:06:21.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.512+0000 7f0d93c00640 1 -- 172.21.15.86:0/2765453824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d8c074750 con 0x7f0d8c1610d0 2024-04-25T14:06:21.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.512+0000 7f0d89600640 1 -- 172.21.15.86:0/2765453824 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+6495 (secure 0 0 0) 0x7f0d8407d500 con 0x7f0d8c1610d0 2024-04-25T14:06:21.514 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:21.514 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3545772170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3545772170},{"type":"v1","addr":"172.21.15.155:6834","nonce":3545772170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5980,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/766530004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":766530004},{"type":"v1","addr":"172.21.15.86:6837","nonce":766530004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:32.946320+0000","modified":"2024-04-25T14:06:03.253171+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6048},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6048":{"gid":6048,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6835/2169924270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2169924270},{"type":"v1","addr":"172.21.15.86:6835","nonce":2169924270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"fs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6048,"qdb_cluster":[6048]},"id":3},{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:54.565528+0000","modified":"2024-04-25T14:06:03.253173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6036":{"gid":6036,"name":"d","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6835/2914930843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2914930843},{"type":"v1","addr":"172.21.15.155:6835","nonce":2914930843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"fs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6036,"qdb_cluster":[6036]},"id":4}]} 2024-04-25T14:06:21.514 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 43 2024-04-25T14:06:21.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.516+0000 7f0d93c00640 1 -- 172.21.15.86:0/2765453824 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f0d6806f250 msgr2=0x7f0d68071760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:21.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.516+0000 7f0d93c00640 1 --2- 172.21.15.86:0/2765453824 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f0d6806f250 0x7f0d68071760 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c0ad000 tx=0x7f0d7c008040 comp rx=0 tx=0).stop 2024-04-25T14:06:21.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.516+0000 7f0d93c00640 1 -- 172.21.15.86:0/2765453824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d8c1610d0 msgr2=0x7f0d8c0b7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:21.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.516+0000 7f0d93c00640 1 --2- 172.21.15.86:0/2765453824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d8c1610d0 0x7f0d8c0b7220 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400ca70 tx=0x7f0d8400cf30 comp rx=0 tx=0).stop 2024-04-25T14:06:21.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.516+0000 7f0d93c00640 1 -- 172.21.15.86:0/2765453824 shutdown_connections 2024-04-25T14:06:21.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.516+0000 7f0d93c00640 1 --2- 172.21.15.86:0/2765453824 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f0d6806f250 0x7f0d68071760 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.516+0000 7f0d93c00640 1 --2- 172.21.15.86:0/2765453824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d8c1610d0 0x7f0d8c0b7220 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.516+0000 7f0d93c00640 1 --2- 172.21.15.86:0/2765453824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d8c0b2ba0 0x7f0d8c146ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.516+0000 7f0d93c00640 1 --2- 172.21.15.86:0/2765453824 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d8c0b2250 0x7f0d8c15b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.516+0000 7f0d93c00640 1 -- 172.21.15.86:0/2765453824 >> 172.21.15.86:0/2765453824 conn(0x7f0d8c09ba30 msgr2=0x7f0d8c0712b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:21.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.516+0000 7f0d93c00640 1 -- 172.21.15.86:0/2765453824 shutdown_connections 2024-04-25T14:06:21.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.516+0000 7f0d93c00640 1 -- 172.21.15.86:0/2765453824 wait complete. 2024-04-25T14:06:21.536 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:06:21.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.684+0000 7f2af3c00640 1 Processor -- start 2024-04-25T14:06:21.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.684+0000 7f2af3c00640 1 -- start start 2024-04-25T14:06:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.688+0000 7f2af3c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2aec070330 0x7f2aec0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.688+0000 7f2af3c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aec0d2030 con 0x7f2aec0d1c20 2024-04-25T14:06:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.688+0000 7f2af3c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aec15e5c0 con 0x7f2aec0ce790 2024-04-25T14:06:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.688+0000 7f2af3c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f2aec15e8a0 con 0x7f2aec070330 2024-04-25T14:06:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.688+0000 7f2aebe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2aec0d1c20 0x7f2aec06ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:32788/0 (socket says 172.21.15.86:32788) 2024-04-25T14:06:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.688+0000 7f2aebe00640 1 -- 172.21.15.86:0/1725078653 learned_addr learned my addr 172.21.15.86:0/1725078653 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.688+0000 7f2af1200640 1 --2- 172.21.15.86:0/1725078653 >> v2:172.21.15.155:3300/0 conn(0x7f2aec070330 0x7f2aec0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.688+0000 7f2aeb400640 1 -- 172.21.15.86:0/1725078653 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 704374853 0 0) 0x7f2ad8001120 con 0x7f2aec0d1c20 2024-04-25T14:06:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.688+0000 7f2aeb400640 1 -- 172.21.15.86:0/1725078653 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ad4003530 con 0x7f2aec0d1c20 2024-04-25T14:06:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.688+0000 7f2aeb400640 1 -- 172.21.15.86:0/1725078653 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2613373718 0 0) 0x7f2aec15e5c0 con 0x7f2aec0ce790 2024-04-25T14:06:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.688+0000 7f2aeb400640 1 -- 172.21.15.86:0/1725078653 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ad8001120 con 0x7f2aec0ce790 2024-04-25T14:06:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.688+0000 7f2af1200640 1 -- 172.21.15.86:0/1725078653 >> v1:172.21.15.155:6789/0 conn(0x7f2aec0ce790 legacy=0x7f2aec06f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.688+0000 7f2af1200640 1 -- 172.21.15.86:0/1725078653 >> v1:172.21.15.86:6789/0 conn(0x7f2aec0d1c20 legacy=0x7f2aec06ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.688+0000 7f2af1200640 1 -- 172.21.15.86:0/1725078653 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aec15eb80 con 0x7f2aec070330 2024-04-25T14:06:21.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.692+0000 7f2af1200640 1 --2- 172.21.15.86:0/1725078653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2aec070330 0x7f2aec0d16e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2ae00097f0 tx=0x7f2ae001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3b4cd7b776d6451 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:21.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.692+0000 7f2aeb400640 1 -- 172.21.15.86:0/1725078653 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae00b0070 con 0x7f2aec070330 2024-04-25T14:06:21.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.692+0000 7f2aeb400640 1 -- 172.21.15.86:0/1725078653 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ae00ab600 con 0x7f2aec070330 2024-04-25T14:06:21.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.692+0000 7f2aeb400640 1 -- 172.21.15.86:0/1725078653 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae00ab9d0 con 0x7f2aec070330 2024-04-25T14:06:21.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.692+0000 7f2af3c00640 1 -- 172.21.15.86:0/1725078653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2aec070330 msgr2=0x7f2aec0d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:21.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.692+0000 7f2af3c00640 1 --2- 172.21.15.86:0/1725078653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2aec070330 0x7f2aec0d16e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2ae00097f0 tx=0x7f2ae001c410 comp rx=0 tx=0).stop 2024-04-25T14:06:21.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.692+0000 7f2af3c00640 1 -- 172.21.15.86:0/1725078653 shutdown_connections 2024-04-25T14:06:21.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.692+0000 7f2af3c00640 1 --2- 172.21.15.86:0/1725078653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2aec070330 0x7f2aec0d16e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.692+0000 7f2af3c00640 1 -- 172.21.15.86:0/1725078653 >> 172.21.15.86:0/1725078653 conn(0x7f2aec0b4650 msgr2=0x7f2aec0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:21.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.692+0000 7f2af3c00640 1 -- 172.21.15.86:0/1725078653 shutdown_connections 2024-04-25T14:06:21.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.692+0000 7f2af3c00640 1 -- 172.21.15.86:0/1725078653 wait complete. 2024-04-25T14:06:21.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.696+0000 7f2af3c00640 1 Processor -- start 2024-04-25T14:06:21.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.696+0000 7f2af3c00640 1 -- start start 2024-04-25T14:06:21.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.696+0000 7f2af3c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2aec070330 0x7f2aec0989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.696+0000 7f2af3c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aec0ce790 0x7f2aec152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.696+0000 7f2af3c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2aec0d1c20 0x7f2aec153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.696+0000 7f2af3c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec0708c0 con 0x7f2aec0ce790 2024-04-25T14:06:21.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.696+0000 7f2af3c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec070740 con 0x7f2aec070330 2024-04-25T14:06:21.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.696+0000 7f2af3c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2aec06fb50 con 0x7f2aec0d1c20 2024-04-25T14:06:21.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.696+0000 7f2aebe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aec0ce790 0x7f2aec152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:21.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.696+0000 7f2aebe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aec0ce790 0x7f2aec152e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55312/0 (socket says 172.21.15.86:55312) 2024-04-25T14:06:21.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.696+0000 7f2aebe00640 1 -- 172.21.15.86:0/4081568150 learned_addr learned my addr 172.21.15.86:0/4081568150 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:21.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.696+0000 7f2af1200640 1 --2- 172.21.15.86:0/4081568150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2aec070330 0x7f2aec0989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:21.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.696+0000 7f2af1c00640 1 --2- 172.21.15.86:0/4081568150 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2aec0d1c20 0x7f2aec153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:21.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.696+0000 7f2aebe00640 1 -- 172.21.15.86:0/4081568150 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2aec0d1c20 msgr2=0x7f2aec153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:21.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.696+0000 7f2aebe00640 1 --2- 172.21.15.86:0/4081568150 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2aec0d1c20 0x7f2aec153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.696+0000 7f2aebe00640 1 -- 172.21.15.86:0/4081568150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2aec070330 msgr2=0x7f2aec0989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:21.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.696+0000 7f2aebe00640 1 --2- 172.21.15.86:0/4081568150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2aec070330 0x7f2aec0989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.696+0000 7f2aebe00640 1 -- 172.21.15.86:0/4081568150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aec15b4f0 con 0x7f2aec0ce790 2024-04-25T14:06:21.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.696+0000 7f2af1c00640 1 --2- 172.21.15.86:0/4081568150 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2aec0d1c20 0x7f2aec153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:21.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.696+0000 7f2aebe00640 1 --2- 172.21.15.86:0/4081568150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aec0ce790 0x7f2aec152e10 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f2ad800c760 tx=0x7f2ad800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:21.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.696+0000 7f2ae8c00640 1 -- 172.21.15.86:0/4081568150 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8017070 con 0x7f2aec0ce790 2024-04-25T14:06:21.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.696+0000 7f2ae8c00640 1 -- 172.21.15.86:0/4081568150 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad8005aa0 con 0x7f2aec0ce790 2024-04-25T14:06:21.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.696+0000 7f2af3c00640 1 -- 172.21.15.86:0/4081568150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aec0c4540 con 0x7f2aec0ce790 2024-04-25T14:06:21.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.696+0000 7f2ae8c00640 1 -- 172.21.15.86:0/4081568150 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8007070 con 0x7f2aec0ce790 2024-04-25T14:06:21.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.696+0000 7f2af3c00640 1 -- 172.21.15.86:0/4081568150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aec0c4b00 con 0x7f2aec0ce790 2024-04-25T14:06:21.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.700+0000 7f2af3c00640 1 -- 172.21.15.86:0/4081568150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aec069d00 con 0x7f2aec0ce790 2024-04-25T14:06:21.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.700+0000 7f2ae8c00640 1 -- 172.21.15.86:0/4081568150 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 93355+0+0 (secure 0 0 0) 0x7f2ad801a070 con 0x7f2aec0ce790 2024-04-25T14:06:21.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.700+0000 7f2ae8c00640 1 --2- 172.21.15.86:0/4081568150 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f2acc06f5b0 0x7f2acc071ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.700+0000 7f2ae8c00640 1 -- 172.21.15.86:0/4081568150 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7875+0+0 (secure 0 0 0) 0x7f2ad808f390 con 0x7f2aec0ce790 2024-04-25T14:06:21.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.700+0000 7f2af1200640 1 --2- 172.21.15.86:0/4081568150 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f2acc06f5b0 0x7f2acc071ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:21.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.700+0000 7f2af1200640 1 --2- 172.21.15.86:0/4081568150 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f2acc06f5b0 0x7f2acc071ac0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2ae00ad000 tx=0x7f2ae00a6000 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:21.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.704+0000 7f2ae8c00640 1 -- 172.21.15.86:0/4081568150 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2aec0d2030 con 0x7f2aec0ce790 2024-04-25T14:06:21.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.864+0000 7f2af3c00640 1 -- 172.21.15.86:0/4081568150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2aec076bf0 con 0x7f2aec0ce790 2024-04-25T14:06:21.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.864+0000 7f2ae8c00640 1 -- 172.21.15.86:0/4081568150 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+22031 (secure 0 0 0) 0x7f2ad805b810 con 0x7f2aec0ce790 2024-04-25T14:06:21.867 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:21.867 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":59,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:06:19.409466+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"fs1_metadata","create_time":"2024-04-25T14:05:30.939601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"fs1"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"fs1_data","create_time":"2024-04-25T14:05:31.898262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"fs1"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"fs2_metadata","create_time":"2024-04-25T14:05:52.841764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"fs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"fs2_data","create_time":"2024-04-25T14:05:53.524454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"fs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/145108428":"2024-04-26T14:06:19.409429+0000","172.21.15.86:6833/2427179681":"2024-04-26T14:06:19.409429+0000","172.21.15.86:6832/2427179681":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/1321309855":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/525010030":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/3470358079":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/1177884430":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/3644218120":"2024-04-26T14:06:19.409429+0000","172.21.15.86:6836/1732615622":"2024-04-26T14:06:00.230367+0000","172.21.15.86:6837/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:6837/2418778265":"2024-04-26T14:06:00.222915+0000","172.21.15.86:0/1843278167":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6832/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/2073988773":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6835/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:0/2602032945":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/3558294881":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6835/615772001":"2024-04-26T14:05:42.882867+0000","172.21.15.86:6837/615772001":"2024-04-26T14:05:42.882867+0000","172.21.15.86:6833/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6835/2418778265":"2024-04-26T14:06:00.222915+0000","172.21.15.86:6834/1732615622":"2024-04-26T14:06:00.230367+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:06:21.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.868+0000 7f2af3c00640 1 -- 172.21.15.86:0/4081568150 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f2acc06f5b0 msgr2=0x7f2acc071ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:21.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.868+0000 7f2af3c00640 1 --2- 172.21.15.86:0/4081568150 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f2acc06f5b0 0x7f2acc071ac0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2ae00ad000 tx=0x7f2ae00a6000 comp rx=0 tx=0).stop 2024-04-25T14:06:21.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.868+0000 7f2af3c00640 1 -- 172.21.15.86:0/4081568150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aec0ce790 msgr2=0x7f2aec152e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:21.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.868+0000 7f2af3c00640 1 --2- 172.21.15.86:0/4081568150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aec0ce790 0x7f2aec152e10 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f2ad800c760 tx=0x7f2ad800cc20 comp rx=0 tx=0).stop 2024-04-25T14:06:21.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.868+0000 7f2af3c00640 1 -- 172.21.15.86:0/4081568150 shutdown_connections 2024-04-25T14:06:21.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.868+0000 7f2af3c00640 1 --2- 172.21.15.86:0/4081568150 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f2acc06f5b0 0x7f2acc071ac0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.868+0000 7f2af3c00640 1 --2- 172.21.15.86:0/4081568150 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2aec0d1c20 0x7f2aec153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.868+0000 7f2af3c00640 1 --2- 172.21.15.86:0/4081568150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aec0ce790 0x7f2aec152e10 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.868+0000 7f2af3c00640 1 --2- 172.21.15.86:0/4081568150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2aec070330 0x7f2aec0989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.868+0000 7f2af3c00640 1 -- 172.21.15.86:0/4081568150 >> 172.21.15.86:0/4081568150 conn(0x7f2aec0b4650 msgr2=0x7f2aec0b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:21.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.868+0000 7f2af3c00640 1 -- 172.21.15.86:0/4081568150 shutdown_connections 2024-04-25T14:06:21.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:21.868+0000 7f2af3c00640 1 -- 172.21.15.86:0/4081568150 wait complete. 2024-04-25T14:06:21.889 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'fs1_metadata', 'create_time': '2024-04-25T14:05:30.939601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'fs1'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'fs1_data', 'create_time': '2024-04-25T14:05:31.898262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'fs1'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'fs2_metadata', 'create_time': '2024-04-25T14:05:52.841764+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'fs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'fs2_data', 'create_time': '2024-04-25T14:05:53.524454+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'fs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:06:21.890 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:06:22.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.036+0000 7f7d58000640 1 Processor -- start 2024-04-25T14:06:22.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.036+0000 7f7d58000640 1 -- start start 2024-04-25T14:06:22.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.040+0000 7f7d58000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d50070330 0x7f7d500d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:22.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.040+0000 7f7d58000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7d500ce790 0x7f7d5006f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:22.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.040+0000 7f7d58000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d500d2030 con 0x7f7d500d1c20 2024-04-25T14:06:22.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.040+0000 7f7d58000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7d5015e5c0 con 0x7f7d500ce790 2024-04-25T14:06:22.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.040+0000 7f7d58000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7d5015e8a0 con 0x7f7d50070330 2024-04-25T14:06:22.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.040+0000 7f7d56000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7d500ce790 0x7f7d5006f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:22.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.040+0000 7f7d56000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7d500ce790 0x7f7d5006f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55326/0 (socket says 172.21.15.86:55326) 2024-04-25T14:06:22.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.040+0000 7f7d56000640 1 -- 172.21.15.86:0/3619083941 learned_addr learned my addr 172.21.15.86:0/3619083941 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:22.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.040+0000 7f7d55600640 1 --2- 172.21.15.86:0/3619083941 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d50070330 0x7f7d500d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:22.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.040+0000 7f7d56000640 1 -- 172.21.15.86:0/3619083941 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d50070330 msgr2=0x7f7d500d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:22.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.040+0000 7f7d56000640 1 --2- 172.21.15.86:0/3619083941 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d50070330 0x7f7d500d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.040+0000 7f7d56000640 1 -- 172.21.15.86:0/3619083941 >> v1:172.21.15.155:6789/0 conn(0x7f7d500d1c20 legacy=0x7f7d5006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:22.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.040+0000 7f7d56000640 1 -- 172.21.15.86:0/3619083941 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d5015eb80 con 0x7f7d500ce790 2024-04-25T14:06:22.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.040+0000 7f7d56000640 1 --2- 172.21.15.86:0/3619083941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d500ce790 0x7f7d5006f440 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009e80 tx=0x7f7d4801c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef496a53bca53e38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:22.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.044+0000 7f7d4fe00640 1 -- 172.21.15.86:0/3619083941 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d480a5640 con 0x7f7d500ce790 2024-04-25T14:06:22.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.044+0000 7f7d4fe00640 1 -- 172.21.15.86:0/3619083941 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d480a57e0 con 0x7f7d500ce790 2024-04-25T14:06:22.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.044+0000 7f7d4fe00640 1 -- 172.21.15.86:0/3619083941 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d480a5bb0 con 0x7f7d500ce790 2024-04-25T14:06:22.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.044+0000 7f7d58000640 1 -- 172.21.15.86:0/3619083941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d500ce790 msgr2=0x7f7d5006f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:22.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.044+0000 7f7d58000640 1 --2- 172.21.15.86:0/3619083941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d500ce790 0x7f7d5006f440 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009e80 tx=0x7f7d4801c980 comp rx=0 tx=0).stop 2024-04-25T14:06:22.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.044+0000 7f7d58000640 1 -- 172.21.15.86:0/3619083941 shutdown_connections 2024-04-25T14:06:22.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.044+0000 7f7d58000640 1 --2- 172.21.15.86:0/3619083941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d500ce790 0x7f7d5006f440 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.044+0000 7f7d58000640 1 --2- 172.21.15.86:0/3619083941 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d50070330 0x7f7d500d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.044+0000 7f7d58000640 1 -- 172.21.15.86:0/3619083941 >> 172.21.15.86:0/3619083941 conn(0x7f7d500b4650 msgr2=0x7f7d500b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:22.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.044+0000 7f7d58000640 1 -- 172.21.15.86:0/3619083941 shutdown_connections 2024-04-25T14:06:22.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.044+0000 7f7d58000640 1 -- 172.21.15.86:0/3619083941 wait complete. 2024-04-25T14:06:22.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.044+0000 7f7d58000640 1 Processor -- start 2024-04-25T14:06:22.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.044+0000 7f7d58000640 1 -- start start 2024-04-25T14:06:22.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.048+0000 7f7d58000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d50070330 0x7f7d501fa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:22.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.048+0000 7f7d58000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d500ce790 0x7f7d5015d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:22.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.048+0000 7f7d58000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d500d1c20 0x7f7d500ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:22.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.048+0000 7f7d58000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d500708c0 con 0x7f7d50070330 2024-04-25T14:06:22.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.048+0000 7f7d58000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7d50070740 con 0x7f7d500d1c20 2024-04-25T14:06:22.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.048+0000 7f7d58000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7d5016afc0 con 0x7f7d500ce790 2024-04-25T14:06:22.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.048+0000 7f7d55600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d50070330 0x7f7d501fa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:22.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.048+0000 7f7d55600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d50070330 0x7f7d501fa6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55332/0 (socket says 172.21.15.86:55332) 2024-04-25T14:06:22.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.048+0000 7f7d55600640 1 -- 172.21.15.86:0/2626183000 learned_addr learned my addr 172.21.15.86:0/2626183000 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:22.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.048+0000 7f7d56000640 1 --2- 172.21.15.86:0/2626183000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d500d1c20 0x7f7d500ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:22.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.048+0000 7f7d54c00640 1 --2- 172.21.15.86:0/2626183000 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d500ce790 0x7f7d5015d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:22.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.048+0000 7f7d55600640 1 -- 172.21.15.86:0/2626183000 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d500ce790 msgr2=0x7f7d5015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:22.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.048+0000 7f7d55600640 1 --2- 172.21.15.86:0/2626183000 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d500ce790 0x7f7d5015d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.048+0000 7f7d55600640 1 -- 172.21.15.86:0/2626183000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d500d1c20 msgr2=0x7f7d500ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:22.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.048+0000 7f7d55600640 1 --2- 172.21.15.86:0/2626183000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d500d1c20 0x7f7d500ca810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.048+0000 7f7d55600640 1 -- 172.21.15.86:0/2626183000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d500c44d0 con 0x7f7d50070330 2024-04-25T14:06:22.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.048+0000 7f7d55600640 1 --2- 172.21.15.86:0/2626183000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d50070330 0x7f7d501fa6e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000e870 tx=0x7f7d4000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:22.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.048+0000 7f7d4d600640 1 -- 172.21.15.86:0/2626183000 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d40017070 con 0x7f7d50070330 2024-04-25T14:06:22.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.048+0000 7f7d4d600640 1 -- 172.21.15.86:0/2626183000 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d4000c8e0 con 0x7f7d50070330 2024-04-25T14:06:22.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.048+0000 7f7d58000640 1 -- 172.21.15.86:0/2626183000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d500cb3f0 con 0x7f7d50070330 2024-04-25T14:06:22.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.048+0000 7f7d4d600640 1 -- 172.21.15.86:0/2626183000 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4000ccb0 con 0x7f7d50070330 2024-04-25T14:06:22.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.048+0000 7f7d58000640 1 -- 172.21.15.86:0/2626183000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d501f9ee0 con 0x7f7d50070330 2024-04-25T14:06:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.048+0000 7f7d58000640 1 -- 172.21.15.86:0/2626183000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d50069d00 con 0x7f7d50070330 2024-04-25T14:06:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.052+0000 7f7d4d600640 1 -- 172.21.15.86:0/2626183000 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 93355+0+0 (secure 0 0 0) 0x7f7d4001a070 con 0x7f7d50070330 2024-04-25T14:06:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.052+0000 7f7d4d600640 1 --2- 172.21.15.86:0/2626183000 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f7d3406f5b0 0x7f7d34071ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.052+0000 7f7d4d600640 1 -- 172.21.15.86:0/2626183000 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7875+0+0 (secure 0 0 0) 0x7f7d4008fef0 con 0x7f7d50070330 2024-04-25T14:06:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.052+0000 7f7d54c00640 1 --2- 172.21.15.86:0/2626183000 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f7d3406f5b0 0x7f7d34071ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:22.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.052+0000 7f7d54c00640 1 --2- 172.21.15.86:0/2626183000 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f7d3406f5b0 0x7f7d34071ac0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7d5015b990 tx=0x7f7d44009040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:22.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.056+0000 7f7d4d600640 1 -- 172.21.15.86:0/2626183000 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7d500c44d0 con 0x7f7d50070330 2024-04-25T14:06:22.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.212+0000 7f7d58000640 1 -- 172.21.15.86:0/2626183000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7d5015ad40 con 0x7f7d50070330 2024-04-25T14:06:22.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.212+0000 7f7d4d600640 1 -- 172.21.15.86:0/2626183000 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) ==== 93+0+10 (secure 0 0 0) 0x7f7d4005bd40 con 0x7f7d50070330 2024-04-25T14:06:22.214 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:06:22.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.212+0000 7f7d58000640 1 -- 172.21.15.86:0/2626183000 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f7d3406f5b0 msgr2=0x7f7d34071ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:22.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.212+0000 7f7d58000640 1 --2- 172.21.15.86:0/2626183000 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f7d3406f5b0 0x7f7d34071ac0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7d5015b990 tx=0x7f7d44009040 comp rx=0 tx=0).stop 2024-04-25T14:06:22.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.212+0000 7f7d58000640 1 -- 172.21.15.86:0/2626183000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d50070330 msgr2=0x7f7d501fa6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:22.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.212+0000 7f7d58000640 1 --2- 172.21.15.86:0/2626183000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d50070330 0x7f7d501fa6e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000e870 tx=0x7f7d4000ec70 comp rx=0 tx=0).stop 2024-04-25T14:06:22.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.216+0000 7f7d58000640 1 -- 172.21.15.86:0/2626183000 shutdown_connections 2024-04-25T14:06:22.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.216+0000 7f7d58000640 1 --2- 172.21.15.86:0/2626183000 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f7d3406f5b0 0x7f7d34071ac0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.216+0000 7f7d58000640 1 --2- 172.21.15.86:0/2626183000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d500d1c20 0x7f7d500ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.216+0000 7f7d58000640 1 --2- 172.21.15.86:0/2626183000 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d500ce790 0x7f7d5015d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.216+0000 7f7d58000640 1 --2- 172.21.15.86:0/2626183000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d50070330 0x7f7d501fa6e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.216+0000 7f7d58000640 1 -- 172.21.15.86:0/2626183000 >> 172.21.15.86:0/2626183000 conn(0x7f7d500b4650 msgr2=0x7f7d500aef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:22.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.216+0000 7f7d58000640 1 -- 172.21.15.86:0/2626183000 shutdown_connections 2024-04-25T14:06:22.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.216+0000 7f7d58000640 1 -- 172.21.15.86:0/2626183000 wait complete. 2024-04-25T14:06:22.235 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get fs1_metadata pg_num 2024-04-25T14:06:22.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.384+0000 7f5d4ba00640 1 Processor -- start 2024-04-25T14:06:22.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.384+0000 7f5d4ba00640 1 -- start start 2024-04-25T14:06:22.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.384+0000 7f5d4ba00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5d4415c2a0 0x7f5d440ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:22.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.384+0000 7f5d4ba00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d4406fd50 0x7f5d44070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:22.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.384+0000 7f5d4ba00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d440b0a60 con 0x7f5d4406c670 2024-04-25T14:06:22.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.384+0000 7f5d4ba00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5d4406ef90 con 0x7f5d4415c2a0 2024-04-25T14:06:22.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.384+0000 7f5d4ba00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5d4406f270 con 0x7f5d4406fd50 2024-04-25T14:06:22.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.384+0000 7f5d49000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5d4415c2a0 0x7f5d440ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:22.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.384+0000 7f5d49000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5d4415c2a0 0x7f5d440ae130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55336/0 (socket says 172.21.15.86:55336) 2024-04-25T14:06:22.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.384+0000 7f5d49000640 1 -- 172.21.15.86:0/3112377703 learned_addr learned my addr 172.21.15.86:0/3112377703 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:22.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.384+0000 7f5d3be00640 1 --2- 172.21.15.86:0/3112377703 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d4406fd50 0x7f5d44070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:22.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.384+0000 7f5d49000640 1 -- 172.21.15.86:0/3112377703 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d4406fd50 msgr2=0x7f5d44070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:22.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.384+0000 7f5d49000640 1 --2- 172.21.15.86:0/3112377703 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d4406fd50 0x7f5d44070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.384+0000 7f5d49000640 1 -- 172.21.15.86:0/3112377703 >> v1:172.21.15.155:6789/0 conn(0x7f5d4406c670 legacy=0x7f5d4406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:22.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.384+0000 7f5d49000640 1 -- 172.21.15.86:0/3112377703 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4406f550 con 0x7f5d4415c2a0 2024-04-25T14:06:22.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.384+0000 7f5d3be00640 1 --2- 172.21.15.86:0/3112377703 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d4406fd50 0x7f5d44070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:22.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.388+0000 7f5d49000640 1 --2- 172.21.15.86:0/3112377703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d4415c2a0 0x7f5d440ae130 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009870 tx=0x7f5d3401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3eaa9dfd6c6249d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:22.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.388+0000 7f5d43e00640 1 -- 172.21.15.86:0/3112377703 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d340b1070 con 0x7f5d4415c2a0 2024-04-25T14:06:22.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.388+0000 7f5d43e00640 1 -- 172.21.15.86:0/3112377703 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d340a8050 con 0x7f5d4415c2a0 2024-04-25T14:06:22.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.388+0000 7f5d43e00640 1 -- 172.21.15.86:0/3112377703 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d340ac860 con 0x7f5d4415c2a0 2024-04-25T14:06:22.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.388+0000 7f5d4ba00640 1 -- 172.21.15.86:0/3112377703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d4415c2a0 msgr2=0x7f5d440ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:22.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.388+0000 7f5d4ba00640 1 --2- 172.21.15.86:0/3112377703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d4415c2a0 0x7f5d440ae130 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009870 tx=0x7f5d3401c460 comp rx=0 tx=0).stop 2024-04-25T14:06:22.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.388+0000 7f5d4ba00640 1 -- 172.21.15.86:0/3112377703 shutdown_connections 2024-04-25T14:06:22.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.388+0000 7f5d4ba00640 1 --2- 172.21.15.86:0/3112377703 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d4406fd50 0x7f5d44070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.388+0000 7f5d4ba00640 1 --2- 172.21.15.86:0/3112377703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d4415c2a0 0x7f5d440ae130 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.388+0000 7f5d4ba00640 1 -- 172.21.15.86:0/3112377703 >> 172.21.15.86:0/3112377703 conn(0x7f5d440b4650 msgr2=0x7f5d440b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:22.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.388+0000 7f5d4ba00640 1 -- 172.21.15.86:0/3112377703 shutdown_connections 2024-04-25T14:06:22.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.388+0000 7f5d4ba00640 1 -- 172.21.15.86:0/3112377703 wait complete. 2024-04-25T14:06:22.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.392+0000 7f5d4ba00640 1 Processor -- start 2024-04-25T14:06:22.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.392+0000 7f5d4ba00640 1 -- start start 2024-04-25T14:06:22.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.392+0000 7f5d4ba00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d4406c670 0x7f5d44156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:22.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.392+0000 7f5d4ba00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d4406fd50 0x7f5d441fadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:22.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.392+0000 7f5d4ba00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d4415c2a0 0x7f5d441fb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:22.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.392+0000 7f5d4ba00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4415c830 con 0x7f5d4406c670 2024-04-25T14:06:22.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.392+0000 7f5d4ba00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4415c6b0 con 0x7f5d4415c2a0 2024-04-25T14:06:22.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.392+0000 7f5d4ba00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5d440ae670 con 0x7f5d4406fd50 2024-04-25T14:06:22.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.392+0000 7f5d49000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d4406c670 0x7f5d44156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:22.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.392+0000 7f5d49000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d4406c670 0x7f5d44156440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55344/0 (socket says 172.21.15.86:55344) 2024-04-25T14:06:22.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.392+0000 7f5d49000640 1 -- 172.21.15.86:0/3723463059 learned_addr learned my addr 172.21.15.86:0/3723463059 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:22.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.392+0000 7f5d49000640 1 -- 172.21.15.86:0/3723463059 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d4406fd50 msgr2=0x7f5d441fadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:22.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.392+0000 7f5d49000640 1 --2- 172.21.15.86:0/3723463059 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d4406fd50 0x7f5d441fadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.392+0000 7f5d49000640 1 -- 172.21.15.86:0/3723463059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d4415c2a0 msgr2=0x7f5d441fb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:22.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.392+0000 7f5d49000640 1 --2- 172.21.15.86:0/3723463059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d4415c2a0 0x7f5d441fb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.392+0000 7f5d49000640 1 -- 172.21.15.86:0/3723463059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d440b1e60 con 0x7f5d4406c670 2024-04-25T14:06:22.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.396+0000 7f5d49000640 1 --2- 172.21.15.86:0/3723463059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d4406c670 0x7f5d44156440 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f5d3401cd00 tx=0x7f5d3401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:22.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.396+0000 7f5d41600640 1 -- 172.21.15.86:0/3723463059 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d340b1040 con 0x7f5d4406c670 2024-04-25T14:06:22.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.396+0000 7f5d41600640 1 -- 172.21.15.86:0/3723463059 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d340bd070 con 0x7f5d4406c670 2024-04-25T14:06:22.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.396+0000 7f5d41600640 1 -- 172.21.15.86:0/3723463059 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d340b7660 con 0x7f5d4406c670 2024-04-25T14:06:22.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.396+0000 7f5d4ba00640 1 -- 172.21.15.86:0/3723463059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d440b2140 con 0x7f5d4406c670 2024-04-25T14:06:22.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.396+0000 7f5d4ba00640 1 -- 172.21.15.86:0/3723463059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d44148dc0 con 0x7f5d4406c670 2024-04-25T14:06:22.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.396+0000 7f5d4ba00640 1 -- 172.21.15.86:0/3723463059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d44069d00 con 0x7f5d4406c670 2024-04-25T14:06:22.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.396+0000 7f5d41600640 1 -- 172.21.15.86:0/3723463059 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 93355+0+0 (secure 0 0 0) 0x7f5d340b7800 con 0x7f5d4406c670 2024-04-25T14:06:22.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.396+0000 7f5d41600640 1 --2- 172.21.15.86:0/3723463059 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f5d2006f5b0 0x7f5d20071ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:22.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.396+0000 7f5d41600640 1 -- 172.21.15.86:0/3723463059 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7875+0+0 (secure 0 0 0) 0x7f5d3412a350 con 0x7f5d4406c670 2024-04-25T14:06:22.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.396+0000 7f5d3be00640 1 --2- 172.21.15.86:0/3723463059 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f5d2006f5b0 0x7f5d20071ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:22.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.400+0000 7f5d3be00640 1 --2- 172.21.15.86:0/3723463059 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f5d2006f5b0 0x7f5d20071ac0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5d44167160 tx=0x7f5d2c008040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:22.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.404+0000 7f5d41600640 1 -- 172.21.15.86:0/3723463059 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5d34016110 con 0x7f5d4406c670 2024-04-25T14:06:22.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.560+0000 7f5d4ba00640 1 -- 172.21.15.86:0/3723463059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "fs1_metadata", "var": "pg_num"} v 0) -- 0x7f5d44076bf0 con 0x7f5d4406c670 2024-04-25T14:06:22.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.560+0000 7f5d41600640 1 -- 172.21.15.86:0/3723463059 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "fs1_metadata", "var": "pg_num"}]=0 v59) ==== 101+0+11 (secure 0 0 0) 0x7f5d340f66a0 con 0x7f5d4406c670 2024-04-25T14:06:22.562 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:06:22.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.564+0000 7f5d4ba00640 1 -- 172.21.15.86:0/3723463059 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f5d2006f5b0 msgr2=0x7f5d20071ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:22.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.564+0000 7f5d4ba00640 1 --2- 172.21.15.86:0/3723463059 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f5d2006f5b0 0x7f5d20071ac0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5d44167160 tx=0x7f5d2c008040 comp rx=0 tx=0).stop 2024-04-25T14:06:22.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.564+0000 7f5d4ba00640 1 -- 172.21.15.86:0/3723463059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d4406c670 msgr2=0x7f5d44156440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:22.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.564+0000 7f5d4ba00640 1 --2- 172.21.15.86:0/3723463059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d4406c670 0x7f5d44156440 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f5d3401cd00 tx=0x7f5d3401aca0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.564+0000 7f5d4ba00640 1 -- 172.21.15.86:0/3723463059 shutdown_connections 2024-04-25T14:06:22.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.564+0000 7f5d4ba00640 1 --2- 172.21.15.86:0/3723463059 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f5d2006f5b0 0x7f5d20071ac0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.564+0000 7f5d4ba00640 1 --2- 172.21.15.86:0/3723463059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d4415c2a0 0x7f5d441fb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.564+0000 7f5d4ba00640 1 --2- 172.21.15.86:0/3723463059 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d4406fd50 0x7f5d441fadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.564+0000 7f5d4ba00640 1 --2- 172.21.15.86:0/3723463059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d4406c670 0x7f5d44156440 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.564+0000 7f5d4ba00640 1 -- 172.21.15.86:0/3723463059 >> 172.21.15.86:0/3723463059 conn(0x7f5d440b4650 msgr2=0x7f5d44158c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:22.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.564+0000 7f5d4ba00640 1 -- 172.21.15.86:0/3723463059 shutdown_connections 2024-04-25T14:06:22.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.564+0000 7f5d4ba00640 1 -- 172.21.15.86:0/3723463059 wait complete. 2024-04-25T14:06:22.584 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get fs1_data pg_num 2024-04-25T14:06:22.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.732+0000 7fad52c00640 1 Processor -- start 2024-04-25T14:06:22.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.732+0000 7fad52c00640 1 -- start start 2024-04-25T14:06:22.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.732+0000 7fad52c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad4c070330 0x7fad4c0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:22.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.732+0000 7fad52c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fad4c0d1c20 0x7fad4c06ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:22.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.732+0000 7fad52c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fad4c06f270 0x7fad4c06f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:22.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.732+0000 7fad52c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fad4c0d2030 con 0x7fad4c0d1c20 2024-04-25T14:06:22.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.732+0000 7fad52c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fad4c15e5c0 con 0x7fad4c06f270 2024-04-25T14:06:22.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.732+0000 7fad52c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fad4c15e8a0 con 0x7fad4c070330 2024-04-25T14:06:22.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.732+0000 7fad43400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fad4c0d1c20 0x7fad4c06ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:22.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.732+0000 7fad43400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fad4c0d1c20 0x7fad4c06ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55360/0 (socket says 172.21.15.86:55360) 2024-04-25T14:06:22.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.732+0000 7fad43400640 1 -- 172.21.15.86:0/3166029577 learned_addr learned my addr 172.21.15.86:0/3166029577 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:22.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.732+0000 7fad4be00640 1 --2- 172.21.15.86:0/3166029577 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad4c070330 0x7fad4c0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:22.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.732+0000 7fad50c00640 1 --2- 172.21.15.86:0/3166029577 >> v2:172.21.15.155:3300/0 conn(0x7fad4c06f270 0x7fad4c06f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:22.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.732+0000 7fad43400640 1 -- 172.21.15.86:0/3166029577 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad4c070330 msgr2=0x7fad4c0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:22.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.732+0000 7fad43400640 1 --2- 172.21.15.86:0/3166029577 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad4c070330 0x7fad4c0d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.732+0000 7fad43400640 1 -- 172.21.15.86:0/3166029577 >> v2:172.21.15.155:3300/0 conn(0x7fad4c06f270 msgr2=0x7fad4c06f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:22.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.732+0000 7fad43400640 1 --2- 172.21.15.86:0/3166029577 >> v2:172.21.15.155:3300/0 conn(0x7fad4c06f270 0x7fad4c06f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.732+0000 7fad43400640 1 -- 172.21.15.86:0/3166029577 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4c15eb80 con 0x7fad4c0d1c20 2024-04-25T14:06:22.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.732+0000 7fad4be00640 1 --2- 172.21.15.86:0/3166029577 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad4c070330 0x7fad4c0d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:22.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.736+0000 7fad43400640 1 --2- 172.21.15.86:0/3166029577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad4c0d1c20 0x7fad4c06ed30 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fad34009870 tx=0x7fad3401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c957bdeb198f85f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:22.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.736+0000 7fad4b400640 1 -- 172.21.15.86:0/3166029577 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad340ab600 con 0x7fad4c0d1c20 2024-04-25T14:06:22.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.736+0000 7fad4b400640 1 -- 172.21.15.86:0/3166029577 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad340b0070 con 0x7fad4c0d1c20 2024-04-25T14:06:22.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.736+0000 7fad4b400640 1 -- 172.21.15.86:0/3166029577 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad340ab9c0 con 0x7fad4c0d1c20 2024-04-25T14:06:22.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.736+0000 7fad52c00640 1 -- 172.21.15.86:0/3166029577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad4c0d1c20 msgr2=0x7fad4c06ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:22.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.736+0000 7fad52c00640 1 --2- 172.21.15.86:0/3166029577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad4c0d1c20 0x7fad4c06ed30 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fad34009870 tx=0x7fad3401c460 comp rx=0 tx=0).stop 2024-04-25T14:06:22.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.736+0000 7fad52c00640 1 -- 172.21.15.86:0/3166029577 shutdown_connections 2024-04-25T14:06:22.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.736+0000 7fad52c00640 1 --2- 172.21.15.86:0/3166029577 >> v2:172.21.15.155:3300/0 conn(0x7fad4c06f270 0x7fad4c06f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.736+0000 7fad52c00640 1 --2- 172.21.15.86:0/3166029577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad4c0d1c20 0x7fad4c06ed30 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.736+0000 7fad52c00640 1 --2- 172.21.15.86:0/3166029577 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad4c070330 0x7fad4c0d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.736+0000 7fad52c00640 1 -- 172.21.15.86:0/3166029577 >> 172.21.15.86:0/3166029577 conn(0x7fad4c0b4650 msgr2=0x7fad4c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:22.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.736+0000 7fad52c00640 1 -- 172.21.15.86:0/3166029577 shutdown_connections 2024-04-25T14:06:22.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.736+0000 7fad52c00640 1 -- 172.21.15.86:0/3166029577 wait complete. 2024-04-25T14:06:22.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.740+0000 7fad52c00640 1 Processor -- start 2024-04-25T14:06:22.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.740+0000 7fad52c00640 1 -- start start 2024-04-25T14:06:22.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.740+0000 7fad52c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad4c06f270 0x7fad4c0c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:22.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.740+0000 7fad52c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad4c070330 0x7fad4c0c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:22.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.740+0000 7fad52c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad4c0d1c20 0x7fad4c097570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:22.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.740+0000 7fad52c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c15f4b0 con 0x7fad4c06f270 2024-04-25T14:06:22.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.740+0000 7fad52c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c15f330 con 0x7fad4c0d1c20 2024-04-25T14:06:22.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.740+0000 7fad52c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fad4c070740 con 0x7fad4c070330 2024-04-25T14:06:22.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.740+0000 7fad4be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad4c06f270 0x7fad4c0c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:22.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.740+0000 7fad4be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad4c06f270 0x7fad4c0c8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55376/0 (socket says 172.21.15.86:55376) 2024-04-25T14:06:22.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.740+0000 7fad4be00640 1 -- 172.21.15.86:0/636581194 learned_addr learned my addr 172.21.15.86:0/636581194 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:22.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.740+0000 7fad43400640 1 --2- 172.21.15.86:0/636581194 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad4c070330 0x7fad4c0c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:22.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.740+0000 7fad50c00640 1 --2- 172.21.15.86:0/636581194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad4c0d1c20 0x7fad4c097570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:22.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.744+0000 7fad4be00640 1 -- 172.21.15.86:0/636581194 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad4c070330 msgr2=0x7fad4c0c8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:22.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.744+0000 7fad4be00640 1 --2- 172.21.15.86:0/636581194 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad4c070330 0x7fad4c0c8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.744+0000 7fad4be00640 1 -- 172.21.15.86:0/636581194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad4c0d1c20 msgr2=0x7fad4c097570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:22.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.744+0000 7fad4be00640 1 --2- 172.21.15.86:0/636581194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad4c0d1c20 0x7fad4c097570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.744+0000 7fad4be00640 1 -- 172.21.15.86:0/636581194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4c0b1d70 con 0x7fad4c06f270 2024-04-25T14:06:22.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.744+0000 7fad50c00640 1 --2- 172.21.15.86:0/636581194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad4c0d1c20 0x7fad4c097570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:22.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.744+0000 7fad4be00640 1 --2- 172.21.15.86:0/636581194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad4c06f270 0x7fad4c0c8230 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fad3c00d7d0 tx=0x7fad3c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:22.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.744+0000 7fad48c00640 1 -- 172.21.15.86:0/636581194 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3c018020 con 0x7fad4c06f270 2024-04-25T14:06:22.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.744+0000 7fad48c00640 1 -- 172.21.15.86:0/636581194 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad3c0096a0 con 0x7fad4c06f270 2024-04-25T14:06:22.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.744+0000 7fad48c00640 1 -- 172.21.15.86:0/636581194 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3c009a90 con 0x7fad4c06f270 2024-04-25T14:06:22.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.744+0000 7fad52c00640 1 -- 172.21.15.86:0/636581194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad4c072880 con 0x7fad4c06f270 2024-04-25T14:06:22.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.744+0000 7fad52c00640 1 -- 172.21.15.86:0/636581194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad4c072e40 con 0x7fad4c06f270 2024-04-25T14:06:22.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.744+0000 7fad52c00640 1 -- 172.21.15.86:0/636581194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad4c073080 con 0x7fad4c06f270 2024-04-25T14:06:22.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.744+0000 7fad48c00640 1 -- 172.21.15.86:0/636581194 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 93355+0+0 (secure 0 0 0) 0x7fad3c01b020 con 0x7fad4c06f270 2024-04-25T14:06:22.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.744+0000 7fad48c00640 1 --2- 172.21.15.86:0/636581194 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fad2806f5b0 0x7fad28071ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:22.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.744+0000 7fad48c00640 1 -- 172.21.15.86:0/636581194 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7875+0+0 (secure 0 0 0) 0x7fad3c090340 con 0x7fad4c06f270 2024-04-25T14:06:22.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.748+0000 7fad43400640 1 --2- 172.21.15.86:0/636581194 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fad2806f5b0 0x7fad28071ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:22.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.748+0000 7fad43400640 1 --2- 172.21.15.86:0/636581194 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fad2806f5b0 0x7fad28071ac0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fad340af000 tx=0x7fad340a7000 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:22.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.752+0000 7fad48c00640 1 -- 172.21.15.86:0/636581194 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fad3c022020 con 0x7fad4c06f270 2024-04-25T14:06:22.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.904+0000 7fad52c00640 1 -- 172.21.15.86:0/636581194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "fs1_data", "var": "pg_num"} v 0) -- 0x7fad4c069d00 con 0x7fad4c06f270 2024-04-25T14:06:22.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.904+0000 7fad48c00640 1 -- 172.21.15.86:0/636581194 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "fs1_data", "var": "pg_num"}]=0 v59) ==== 97+0+11 (secure 0 0 0) 0x7fad3c05c720 con 0x7fad4c06f270 2024-04-25T14:06:22.906 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:06:22.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.908+0000 7fad52c00640 1 -- 172.21.15.86:0/636581194 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fad2806f5b0 msgr2=0x7fad28071ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:22.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.908+0000 7fad52c00640 1 --2- 172.21.15.86:0/636581194 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fad2806f5b0 0x7fad28071ac0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fad340af000 tx=0x7fad340a7000 comp rx=0 tx=0).stop 2024-04-25T14:06:22.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.908+0000 7fad52c00640 1 -- 172.21.15.86:0/636581194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad4c06f270 msgr2=0x7fad4c0c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:22.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.908+0000 7fad52c00640 1 --2- 172.21.15.86:0/636581194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad4c06f270 0x7fad4c0c8230 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fad3c00d7d0 tx=0x7fad3c00dc90 comp rx=0 tx=0).stop 2024-04-25T14:06:22.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.908+0000 7fad52c00640 1 -- 172.21.15.86:0/636581194 shutdown_connections 2024-04-25T14:06:22.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.908+0000 7fad52c00640 1 --2- 172.21.15.86:0/636581194 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fad2806f5b0 0x7fad28071ac0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.908+0000 7fad52c00640 1 --2- 172.21.15.86:0/636581194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad4c0d1c20 0x7fad4c097570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.908+0000 7fad52c00640 1 --2- 172.21.15.86:0/636581194 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad4c070330 0x7fad4c0c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.908+0000 7fad52c00640 1 --2- 172.21.15.86:0/636581194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad4c06f270 0x7fad4c0c8230 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:22.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.908+0000 7fad52c00640 1 -- 172.21.15.86:0/636581194 >> 172.21.15.86:0/636581194 conn(0x7fad4c0b4650 msgr2=0x7fad4c0c4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:22.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.908+0000 7fad52c00640 1 -- 172.21.15.86:0/636581194 shutdown_connections 2024-04-25T14:06:22.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:22.908+0000 7fad52c00640 1 -- 172.21.15.86:0/636581194 wait complete. 2024-04-25T14:06:22.927 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get fs2_metadata pg_num 2024-04-25T14:06:23.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.072+0000 7f85de200640 1 Processor -- start 2024-04-25T14:06:23.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.072+0000 7f85de200640 1 -- start start 2024-04-25T14:06:23.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.076+0000 7f85de200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f85d806f990 0x7f85d806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:23.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.076+0000 7f85de200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f85d81615c0 0x7f85d81619d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:23.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.076+0000 7f85de200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85d80702e0 con 0x7f85d80b6530 2024-04-25T14:06:23.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.076+0000 7f85de200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f85d81622f0 con 0x7f85d81615c0 2024-04-25T14:06:23.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.076+0000 7f85de200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f85d80d1730 con 0x7f85d806f990 2024-04-25T14:06:23.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.076+0000 7f85cfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f85d81615c0 0x7f85d81619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:23.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.076+0000 7f85cfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f85d81615c0 0x7f85d81619d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55392/0 (socket says 172.21.15.86:55392) 2024-04-25T14:06:23.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.076+0000 7f85cfe00640 1 -- 172.21.15.86:0/2870587929 learned_addr learned my addr 172.21.15.86:0/2870587929 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.076+0000 7f85dce00640 1 --2- 172.21.15.86:0/2870587929 >> v2:172.21.15.155:3300/0 conn(0x7f85d806f990 0x7f85d806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.076+0000 7f85cfe00640 1 -- 172.21.15.86:0/2870587929 >> v2:172.21.15.155:3300/0 conn(0x7f85d806f990 msgr2=0x7f85d806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.076+0000 7f85cfe00640 1 --2- 172.21.15.86:0/2870587929 >> v2:172.21.15.155:3300/0 conn(0x7f85d806f990 0x7f85d806fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.076+0000 7f85cfe00640 1 -- 172.21.15.86:0/2870587929 >> v1:172.21.15.155:6789/0 conn(0x7f85d80b6530 legacy=0x7f85d80b6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:23.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.076+0000 7f85cfe00640 1 -- 172.21.15.86:0/2870587929 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d80d1a10 con 0x7f85d81615c0 2024-04-25T14:06:23.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.080+0000 7f85cfe00640 1 --2- 172.21.15.86:0/2870587929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85d81615c0 0x7f85d81619d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f85d80739b0 tx=0x7f85c001c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8732253dbd8f927f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:23.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.080+0000 7f85d7e00640 1 -- 172.21.15.86:0/2870587929 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c00b0070 con 0x7f85d81615c0 2024-04-25T14:06:23.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.080+0000 7f85d7e00640 1 -- 172.21.15.86:0/2870587929 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85c00ab600 con 0x7f85d81615c0 2024-04-25T14:06:23.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.080+0000 7f85d7e00640 1 -- 172.21.15.86:0/2870587929 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c00ab9d0 con 0x7f85d81615c0 2024-04-25T14:06:23.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.080+0000 7f85de200640 1 -- 172.21.15.86:0/2870587929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85d81615c0 msgr2=0x7f85d81619d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:23.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.080+0000 7f85de200640 1 --2- 172.21.15.86:0/2870587929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85d81615c0 0x7f85d81619d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f85d80739b0 tx=0x7f85c001c270 comp rx=0 tx=0).stop 2024-04-25T14:06:23.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.080+0000 7f85de200640 1 -- 172.21.15.86:0/2870587929 shutdown_connections 2024-04-25T14:06:23.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.080+0000 7f85de200640 1 --2- 172.21.15.86:0/2870587929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85d81615c0 0x7f85d81619d0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.080+0000 7f85de200640 1 --2- 172.21.15.86:0/2870587929 >> v2:172.21.15.155:3300/0 conn(0x7f85d806f990 0x7f85d806fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.080+0000 7f85de200640 1 -- 172.21.15.86:0/2870587929 >> 172.21.15.86:0/2870587929 conn(0x7f85d80cfea0 msgr2=0x7f85d80d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:23.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.080+0000 7f85de200640 1 -- 172.21.15.86:0/2870587929 shutdown_connections 2024-04-25T14:06:23.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.080+0000 7f85de200640 1 -- 172.21.15.86:0/2870587929 wait complete. 2024-04-25T14:06:23.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.084+0000 7f85de200640 1 Processor -- start 2024-04-25T14:06:23.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.084+0000 7f85de200640 1 -- start start 2024-04-25T14:06:23.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.084+0000 7f85de200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85d806f990 0x7f85d80cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:23.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.084+0000 7f85de200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85d80b6530 0x7f85d81559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:23.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.084+0000 7f85de200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85d81615c0 0x7f85d80b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:23.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.084+0000 7f85de200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85d815de90 con 0x7f85d806f990 2024-04-25T14:06:23.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.084+0000 7f85de200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f85d815dd10 con 0x7f85d81615c0 2024-04-25T14:06:23.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.084+0000 7f85de200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f85d80d2630 con 0x7f85d80b6530 2024-04-25T14:06:23.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.084+0000 7f85dce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85d806f990 0x7f85d80cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:23.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.084+0000 7f85dce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85d806f990 0x7f85d80cd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55396/0 (socket says 172.21.15.86:55396) 2024-04-25T14:06:23.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.084+0000 7f85dce00640 1 -- 172.21.15.86:0/1437868352 learned_addr learned my addr 172.21.15.86:0/1437868352 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:23.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.084+0000 7f85dd800640 1 --2- 172.21.15.86:0/1437868352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85d81615c0 0x7f85d80b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:23.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.084+0000 7f85cfe00640 1 --2- 172.21.15.86:0/1437868352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85d80b6530 0x7f85d81559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:23.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.084+0000 7f85dce00640 1 -- 172.21.15.86:0/1437868352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85d80b6530 msgr2=0x7f85d81559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:23.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.084+0000 7f85dce00640 1 --2- 172.21.15.86:0/1437868352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85d80b6530 0x7f85d81559d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.084+0000 7f85dce00640 1 -- 172.21.15.86:0/1437868352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85d81615c0 msgr2=0x7f85d80b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:23.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.084+0000 7f85dce00640 1 --2- 172.21.15.86:0/1437868352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85d81615c0 0x7f85d80b3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.084+0000 7f85dce00640 1 -- 172.21.15.86:0/1437868352 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d81f9700 con 0x7f85d806f990 2024-04-25T14:06:23.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.084+0000 7f85dd800640 1 --2- 172.21.15.86:0/1437868352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85d81615c0 0x7f85d80b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:23.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.084+0000 7f85dce00640 1 --2- 172.21.15.86:0/1437868352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85d806f990 0x7f85d80cd1f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f85c800d650 tx=0x7f85c800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:23.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.084+0000 7f85d5600640 1 -- 172.21.15.86:0/1437868352 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c800b6c0 con 0x7f85d806f990 2024-04-25T14:06:23.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.084+0000 7f85d5600640 1 -- 172.21.15.86:0/1437868352 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85c8019070 con 0x7f85d806f990 2024-04-25T14:06:23.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.084+0000 7f85de200640 1 -- 172.21.15.86:0/1437868352 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d81f99e0 con 0x7f85d806f990 2024-04-25T14:06:23.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.084+0000 7f85d5600640 1 -- 172.21.15.86:0/1437868352 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c800ba80 con 0x7f85d806f990 2024-04-25T14:06:23.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.084+0000 7f85de200640 1 -- 172.21.15.86:0/1437868352 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d8155f10 con 0x7f85d806f990 2024-04-25T14:06:23.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.088+0000 7f85de200640 1 -- 172.21.15.86:0/1437868352 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85d8076b80 con 0x7f85d806f990 2024-04-25T14:06:23.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.088+0000 7f85d5600640 1 -- 172.21.15.86:0/1437868352 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 93355+0+0 (secure 0 0 0) 0x7f85c800bc20 con 0x7f85d806f990 2024-04-25T14:06:23.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.088+0000 7f85d5600640 1 --2- 172.21.15.86:0/1437868352 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f85b406f5e0 0x7f85b4071af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:23.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.088+0000 7f85d5600640 1 -- 172.21.15.86:0/1437868352 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7875+0+0 (secure 0 0 0) 0x7f85c8090350 con 0x7f85d806f990 2024-04-25T14:06:23.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.088+0000 7f85cfe00640 1 --2- 172.21.15.86:0/1437868352 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f85b406f5e0 0x7f85b4071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:23.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.088+0000 7f85cfe00640 1 --2- 172.21.15.86:0/1437868352 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f85b406f5e0 0x7f85b4071af0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f85c00ad000 tx=0x7f85c0008040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:23.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.092+0000 7f85d5600640 1 -- 172.21.15.86:0/1437868352 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f85c805c6a0 con 0x7f85d806f990 2024-04-25T14:06:23.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.248+0000 7f85de200640 1 -- 172.21.15.86:0/1437868352 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "fs2_metadata", "var": "pg_num"} v 0) -- 0x7f85d8069d00 con 0x7f85d806f990 2024-04-25T14:06:23.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.252+0000 7f85d5600640 1 -- 172.21.15.86:0/1437868352 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "fs2_metadata", "var": "pg_num"}]=0 v59) ==== 101+0+11 (secure 0 0 0) 0x7f85c805bfa0 con 0x7f85d806f990 2024-04-25T14:06:23.252 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:06:23.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.252+0000 7f85de200640 1 -- 172.21.15.86:0/1437868352 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f85b406f5e0 msgr2=0x7f85b4071af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:23.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.252+0000 7f85de200640 1 --2- 172.21.15.86:0/1437868352 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f85b406f5e0 0x7f85b4071af0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f85c00ad000 tx=0x7f85c0008040 comp rx=0 tx=0).stop 2024-04-25T14:06:23.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.252+0000 7f85de200640 1 -- 172.21.15.86:0/1437868352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85d806f990 msgr2=0x7f85d80cd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:23.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.252+0000 7f85de200640 1 --2- 172.21.15.86:0/1437868352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85d806f990 0x7f85d80cd1f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f85c800d650 tx=0x7f85c800db10 comp rx=0 tx=0).stop 2024-04-25T14:06:23.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.252+0000 7f85de200640 1 -- 172.21.15.86:0/1437868352 shutdown_connections 2024-04-25T14:06:23.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.252+0000 7f85de200640 1 --2- 172.21.15.86:0/1437868352 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f85b406f5e0 0x7f85b4071af0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.252+0000 7f85de200640 1 --2- 172.21.15.86:0/1437868352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85d81615c0 0x7f85d80b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.252+0000 7f85de200640 1 --2- 172.21.15.86:0/1437868352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85d80b6530 0x7f85d81559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.252+0000 7f85de200640 1 --2- 172.21.15.86:0/1437868352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85d806f990 0x7f85d80cd1f0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.252+0000 7f85de200640 1 -- 172.21.15.86:0/1437868352 >> 172.21.15.86:0/1437868352 conn(0x7f85d80cfea0 msgr2=0x7f85d80d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:23.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.252+0000 7f85de200640 1 -- 172.21.15.86:0/1437868352 shutdown_connections 2024-04-25T14:06:23.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.252+0000 7f85de200640 1 -- 172.21.15.86:0/1437868352 wait complete. 2024-04-25T14:06:23.273 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get fs2_data pg_num 2024-04-25T14:06:23.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.420+0000 7f7ce9a00640 1 Processor -- start 2024-04-25T14:06:23.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.420+0000 7f7ce9a00640 1 -- start start 2024-04-25T14:06:23.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.420+0000 7f7ce9a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7ce4070330 0x7f7ce40d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:23.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.420+0000 7f7ce9a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7ce40d1c20 0x7f7ce406ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:23.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.420+0000 7f7ce9a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ce40d2030 con 0x7f7ce406f270 2024-04-25T14:06:23.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.424+0000 7f7ce9a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7ce415e5c0 con 0x7f7ce4070330 2024-04-25T14:06:23.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.424+0000 7f7ce9a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f7ce415e8a0 con 0x7f7ce40d1c20 2024-04-25T14:06:23.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.424+0000 7f7ce2a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7ce4070330 0x7f7ce40d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:23.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.424+0000 7f7ce3400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7ce406f270 0x7f7ce406f680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:32796/0 (socket says 172.21.15.86:32796) 2024-04-25T14:06:23.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.424+0000 7f7ce3400640 1 -- 172.21.15.86:0/3877808879 learned_addr learned my addr 172.21.15.86:0/3877808879 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:23.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.424+0000 7f7cda000640 1 --2- 172.21.15.86:0/3877808879 >> v2:172.21.15.155:3300/0 conn(0x7f7ce40d1c20 0x7f7ce406ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:23.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.424+0000 7f7ce2a00640 1 -- 172.21.15.86:0/3877808879 >> v2:172.21.15.155:3300/0 conn(0x7f7ce40d1c20 msgr2=0x7f7ce406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:23.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.424+0000 7f7ce2a00640 1 --2- 172.21.15.86:0/3877808879 >> v2:172.21.15.155:3300/0 conn(0x7f7ce40d1c20 0x7f7ce406ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.424+0000 7f7ce2a00640 1 -- 172.21.15.86:0/3877808879 >> v1:172.21.15.86:6789/0 conn(0x7f7ce406f270 legacy=0x7f7ce406f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:23.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.424+0000 7f7ce2000640 1 -- 172.21.15.86:0/3877808879 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 627598038 0 0) 0x7f7cdc0014a0 con 0x7f7ce406f270 2024-04-25T14:06:23.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.424+0000 7f7ce2a00640 1 -- 172.21.15.86:0/3877808879 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce415eb80 con 0x7f7ce4070330 2024-04-25T14:06:23.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.424+0000 7f7ce2a00640 1 --2- 172.21.15.86:0/3877808879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ce4070330 0x7f7ce40d16e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f7cd40097c0 tx=0x7f7cd401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5456a5ac37d48679 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:23.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.424+0000 7f7ce2000640 1 -- 172.21.15.86:0/3877808879 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd40b0070 con 0x7f7ce4070330 2024-04-25T14:06:23.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.424+0000 7f7ce2000640 1 -- 172.21.15.86:0/3877808879 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cd40ab600 con 0x7f7ce4070330 2024-04-25T14:06:23.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.424+0000 7f7ce2000640 1 -- 172.21.15.86:0/3877808879 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd40a4e60 con 0x7f7ce4070330 2024-04-25T14:06:23.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.428+0000 7f7ce9a00640 1 -- 172.21.15.86:0/3877808879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ce4070330 msgr2=0x7f7ce40d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:23.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.428+0000 7f7ce9a00640 1 --2- 172.21.15.86:0/3877808879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ce4070330 0x7f7ce40d16e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f7cd40097c0 tx=0x7f7cd401c3d0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.428+0000 7f7ce9a00640 1 -- 172.21.15.86:0/3877808879 shutdown_connections 2024-04-25T14:06:23.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.428+0000 7f7ce9a00640 1 --2- 172.21.15.86:0/3877808879 >> v2:172.21.15.155:3300/0 conn(0x7f7ce40d1c20 0x7f7ce406ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.428+0000 7f7ce9a00640 1 --2- 172.21.15.86:0/3877808879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ce4070330 0x7f7ce40d16e0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.428+0000 7f7ce9a00640 1 -- 172.21.15.86:0/3877808879 >> 172.21.15.86:0/3877808879 conn(0x7f7ce40b4650 msgr2=0x7f7ce40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:23.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.428+0000 7f7ce9a00640 1 -- 172.21.15.86:0/3877808879 shutdown_connections 2024-04-25T14:06:23.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.428+0000 7f7ce9a00640 1 -- 172.21.15.86:0/3877808879 wait complete. 2024-04-25T14:06:23.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.428+0000 7f7ce9a00640 1 Processor -- start 2024-04-25T14:06:23.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.428+0000 7f7ce9a00640 1 -- start start 2024-04-25T14:06:23.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.432+0000 7f7ce9a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ce406f270 0x7f7ce40c4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:23.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.432+0000 7f7ce9a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ce4070330 0x7f7ce40ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:23.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.432+0000 7f7ce9a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ce40d1c20 0x7f7ce40ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:23.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.432+0000 7f7ce9a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce40708c0 con 0x7f7ce40d1c20 2024-04-25T14:06:23.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.432+0000 7f7ce9a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce4070740 con 0x7f7ce4070330 2024-04-25T14:06:23.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.432+0000 7f7ce9a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7ce416afc0 con 0x7f7ce406f270 2024-04-25T14:06:23.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.432+0000 7f7ce3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ce40d1c20 0x7f7ce40ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:23.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.432+0000 7f7ce3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ce40d1c20 0x7f7ce40ca630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55406/0 (socket says 172.21.15.86:55406) 2024-04-25T14:06:23.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.432+0000 7f7ce3400640 1 -- 172.21.15.86:0/2969353650 learned_addr learned my addr 172.21.15.86:0/2969353650 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:23.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.432+0000 7f7cda000640 1 --2- 172.21.15.86:0/2969353650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ce4070330 0x7f7ce40ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:23.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.432+0000 7f7ce2a00640 1 --2- 172.21.15.86:0/2969353650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ce406f270 0x7f7ce40c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:23.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.432+0000 7f7ce3400640 1 -- 172.21.15.86:0/2969353650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ce406f270 msgr2=0x7f7ce40c4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:23.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.432+0000 7f7ce3400640 1 --2- 172.21.15.86:0/2969353650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ce406f270 0x7f7ce40c4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.432+0000 7f7ce3400640 1 -- 172.21.15.86:0/2969353650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ce4070330 msgr2=0x7f7ce40ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:23.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.432+0000 7f7ce3400640 1 --2- 172.21.15.86:0/2969353650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ce4070330 0x7f7ce40ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.432+0000 7f7ce3400640 1 -- 172.21.15.86:0/2969353650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce40c25a0 con 0x7f7ce40d1c20 2024-04-25T14:06:23.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.432+0000 7f7ce3400640 1 --2- 172.21.15.86:0/2969353650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ce40d1c20 0x7f7ce40ca630 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00c6d0 tx=0x7f7cdc00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:23.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.432+0000 7f7cdb400640 1 -- 172.21.15.86:0/2969353650 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc018070 con 0x7f7ce40d1c20 2024-04-25T14:06:23.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.432+0000 7f7cdb400640 1 -- 172.21.15.86:0/2969353650 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cdc004d70 con 0x7f7ce40d1c20 2024-04-25T14:06:23.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.432+0000 7f7cdb400640 1 -- 172.21.15.86:0/2969353650 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc00a880 con 0x7f7ce40d1c20 2024-04-25T14:06:23.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.432+0000 7f7ce9a00640 1 -- 172.21.15.86:0/2969353650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ce40d05b0 con 0x7f7ce40d1c20 2024-04-25T14:06:23.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.432+0000 7f7ce9a00640 1 -- 172.21.15.86:0/2969353650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ce40d0d90 con 0x7f7ce40d1c20 2024-04-25T14:06:23.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.432+0000 7f7ce9a00640 1 -- 172.21.15.86:0/2969353650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ce40c2770 con 0x7f7ce40d1c20 2024-04-25T14:06:23.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.436+0000 7f7cdb400640 1 -- 172.21.15.86:0/2969353650 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 93355+0+0 (secure 0 0 0) 0x7f7cdc00ac30 con 0x7f7ce40d1c20 2024-04-25T14:06:23.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.436+0000 7f7cdb400640 1 --2- 172.21.15.86:0/2969353650 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f7cc006f5b0 0x7f7cc0071ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:23.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.436+0000 7f7cdb400640 1 -- 172.21.15.86:0/2969353650 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7875+0+0 (secure 0 0 0) 0x7f7cdc08f390 con 0x7f7ce40d1c20 2024-04-25T14:06:23.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.436+0000 7f7ce2a00640 1 --2- 172.21.15.86:0/2969353650 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f7cc006f5b0 0x7f7cc0071ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:23.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.436+0000 7f7ce2a00640 1 --2- 172.21.15.86:0/2969353650 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f7cc006f5b0 0x7f7cc0071ac0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7cd40ad000 tx=0x7f7cd40a6000 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:23.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.440+0000 7f7cdb400640 1 -- 172.21.15.86:0/2969353650 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7ce40d2030 con 0x7f7ce40d1c20 2024-04-25T14:06:23.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.448+0000 7f7cdb400640 1 -- 172.21.15.86:0/2969353650 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f7cdc0574f0 con 0x7f7ce40d1c20 2024-04-25T14:06:23.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.604+0000 7f7ce9a00640 1 -- 172.21.15.86:0/2969353650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "fs2_data", "var": "pg_num"} v 0) -- 0x7f7ce40d0820 con 0x7f7ce40d1c20 2024-04-25T14:06:23.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.604+0000 7f7cdb400640 1 -- 172.21.15.86:0/2969353650 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "fs2_data", "var": "pg_num"}]=0 v59) ==== 97+0+11 (secure 0 0 0) 0x7f7cdc00a840 con 0x7f7ce40d1c20 2024-04-25T14:06:23.605 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:06:23.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.604+0000 7f7ce9a00640 1 -- 172.21.15.86:0/2969353650 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f7cc006f5b0 msgr2=0x7f7cc0071ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:23.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.604+0000 7f7ce9a00640 1 --2- 172.21.15.86:0/2969353650 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f7cc006f5b0 0x7f7cc0071ac0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7cd40ad000 tx=0x7f7cd40a6000 comp rx=0 tx=0).stop 2024-04-25T14:06:23.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.604+0000 7f7ce9a00640 1 -- 172.21.15.86:0/2969353650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ce40d1c20 msgr2=0x7f7ce40ca630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:23.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.604+0000 7f7ce9a00640 1 --2- 172.21.15.86:0/2969353650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ce40d1c20 0x7f7ce40ca630 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00c6d0 tx=0x7f7cdc00cb90 comp rx=0 tx=0).stop 2024-04-25T14:06:23.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.604+0000 7f7ce9a00640 1 -- 172.21.15.86:0/2969353650 shutdown_connections 2024-04-25T14:06:23.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.604+0000 7f7ce9a00640 1 --2- 172.21.15.86:0/2969353650 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f7cc006f5b0 0x7f7cc0071ac0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.604+0000 7f7ce9a00640 1 --2- 172.21.15.86:0/2969353650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ce40d1c20 0x7f7ce40ca630 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.604+0000 7f7ce9a00640 1 --2- 172.21.15.86:0/2969353650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ce4070330 0x7f7ce40ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.604+0000 7f7ce9a00640 1 --2- 172.21.15.86:0/2969353650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ce406f270 0x7f7ce40c4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.604+0000 7f7ce9a00640 1 -- 172.21.15.86:0/2969353650 >> 172.21.15.86:0/2969353650 conn(0x7f7ce40b4650 msgr2=0x7f7ce40c49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:23.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.604+0000 7f7ce9a00640 1 -- 172.21.15.86:0/2969353650 shutdown_connections 2024-04-25T14:06:23.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.604+0000 7f7ce9a00640 1 -- 172.21.15.86:0/2969353650 wait complete. 2024-04-25T14:06:23.627 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:06:23.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.772+0000 7ffabf400640 1 Processor -- start 2024-04-25T14:06:23.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.772+0000 7ffabf400640 1 -- start start 2024-04-25T14:06:23.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.772+0000 7ffabf400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffab80cde90 0x7ffab80ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:23.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.772+0000 7ffabf400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffab815e5c0 con 0x7ffab8070330 2024-04-25T14:06:23.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.772+0000 7ffabf400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffab815e8a0 con 0x7ffab80d1df0 2024-04-25T14:06:23.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.772+0000 7ffabf400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ffab806fd40 con 0x7ffab80cde90 2024-04-25T14:06:23.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.776+0000 7ffabd400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffab80cde90 0x7ffab80ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:23.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.776+0000 7ffabd400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffab80cde90 0x7ffab80ce2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55410/0 (socket says 172.21.15.86:55410) 2024-04-25T14:06:23.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.776+0000 7ffabd400640 1 -- 172.21.15.86:0/3404338820 learned_addr learned my addr 172.21.15.86:0/3404338820 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:23.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.776+0000 7ffabd400640 1 -- 172.21.15.86:0/3404338820 >> v1:172.21.15.155:6789/0 conn(0x7ffab80d1df0 legacy=0x7ffab80cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:23.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.776+0000 7ffabd400640 1 -- 172.21.15.86:0/3404338820 >> v1:172.21.15.86:6789/0 conn(0x7ffab8070330 legacy=0x7ffab80d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:23.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.776+0000 7ffabd400640 1 -- 172.21.15.86:0/3404338820 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab815eb80 con 0x7ffab80cde90 2024-04-25T14:06:23.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.776+0000 7ffabd400640 1 --2- 172.21.15.86:0/3404338820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffab80cde90 0x7ffab80ce2a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ffab0009e80 tx=0x7ffab001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974044a39bb5e873 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:23.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.776+0000 7ffab7400640 1 -- 172.21.15.86:0/3404338820 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab00a5720 con 0x7ffab80cde90 2024-04-25T14:06:23.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.776+0000 7ffab7400640 1 -- 172.21.15.86:0/3404338820 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffab00a58c0 con 0x7ffab80cde90 2024-04-25T14:06:23.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.776+0000 7ffab7400640 1 -- 172.21.15.86:0/3404338820 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab00a5c90 con 0x7ffab80cde90 2024-04-25T14:06:23.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.780+0000 7ffabf400640 1 -- 172.21.15.86:0/3404338820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffab80cde90 msgr2=0x7ffab80ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:23.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.780+0000 7ffabf400640 1 --2- 172.21.15.86:0/3404338820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffab80cde90 0x7ffab80ce2a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ffab0009e80 tx=0x7ffab001c950 comp rx=0 tx=0).stop 2024-04-25T14:06:23.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.780+0000 7ffabf400640 1 -- 172.21.15.86:0/3404338820 shutdown_connections 2024-04-25T14:06:23.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.780+0000 7ffabf400640 1 --2- 172.21.15.86:0/3404338820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffab80cde90 0x7ffab80ce2a0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.780+0000 7ffabf400640 1 -- 172.21.15.86:0/3404338820 >> 172.21.15.86:0/3404338820 conn(0x7ffab80b4650 msgr2=0x7ffab80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:23.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.780+0000 7ffabf400640 1 -- 172.21.15.86:0/3404338820 shutdown_connections 2024-04-25T14:06:23.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.780+0000 7ffabf400640 1 -- 172.21.15.86:0/3404338820 wait complete. 2024-04-25T14:06:23.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.780+0000 7ffabf400640 1 Processor -- start 2024-04-25T14:06:23.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.780+0000 7ffabf400640 1 -- start start 2024-04-25T14:06:23.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.784+0000 7ffabf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffab8070330 0x7ffab809a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:23.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.784+0000 7ffabf400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffab80cde90 0x7ffab815a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:23.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.784+0000 7ffabf400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffab815a9b0 0x7ffab80c2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:23.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.784+0000 7ffabf400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffab80708c0 con 0x7ffab8070330 2024-04-25T14:06:23.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.784+0000 7ffabf400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffab8070740 con 0x7ffab80cde90 2024-04-25T14:06:23.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.784+0000 7ffabf400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ffab816bfd0 con 0x7ffab815a9b0 2024-04-25T14:06:23.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.784+0000 7ffabca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffab8070330 0x7ffab809a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:23.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.784+0000 7ffabca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffab8070330 0x7ffab809a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55418/0 (socket says 172.21.15.86:55418) 2024-04-25T14:06:23.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.784+0000 7ffabca00640 1 -- 172.21.15.86:0/1765506917 learned_addr learned my addr 172.21.15.86:0/1765506917 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:23.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.784+0000 7ffabd400640 1 --2- 172.21.15.86:0/1765506917 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffab815a9b0 0x7ffab80c2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:23.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.784+0000 7ffab7e00640 1 --2- 172.21.15.86:0/1765506917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffab80cde90 0x7ffab815a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:23.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.784+0000 7ffabca00640 1 -- 172.21.15.86:0/1765506917 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffab815a9b0 msgr2=0x7ffab80c2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:23.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.784+0000 7ffabca00640 1 --2- 172.21.15.86:0/1765506917 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffab815a9b0 0x7ffab80c2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.784+0000 7ffabca00640 1 -- 172.21.15.86:0/1765506917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffab80cde90 msgr2=0x7ffab815a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:23.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.784+0000 7ffabca00640 1 --2- 172.21.15.86:0/1765506917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffab80cde90 0x7ffab815a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.784+0000 7ffabca00640 1 -- 172.21.15.86:0/1765506917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab815adc0 con 0x7ffab8070330 2024-04-25T14:06:23.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.784+0000 7ffab7e00640 1 --2- 172.21.15.86:0/1765506917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffab80cde90 0x7ffab815a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:23.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.784+0000 7ffabca00640 1 --2- 172.21.15.86:0/1765506917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffab8070330 0x7ffab809a9c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00c340 tx=0x7ffaac00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:23.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.784+0000 7ffab4c00640 1 -- 172.21.15.86:0/1765506917 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac0052d0 con 0x7ffab8070330 2024-04-25T14:06:23.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.784+0000 7ffab4c00640 1 -- 172.21.15.86:0/1765506917 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaac005470 con 0x7ffab8070330 2024-04-25T14:06:23.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.784+0000 7ffab4c00640 1 -- 172.21.15.86:0/1765506917 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac007260 con 0x7ffab8070330 2024-04-25T14:06:23.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.784+0000 7ffabf400640 1 -- 172.21.15.86:0/1765506917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffab80b0300 con 0x7ffab8070330 2024-04-25T14:06:23.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.784+0000 7ffabf400640 1 -- 172.21.15.86:0/1765506917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffab81589a0 con 0x7ffab8070330 2024-04-25T14:06:23.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.784+0000 7ffabf400640 1 -- 172.21.15.86:0/1765506917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffab8069d00 con 0x7ffab8070330 2024-04-25T14:06:23.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.788+0000 7ffab4c00640 1 -- 172.21.15.86:0/1765506917 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7ffaac007400 con 0x7ffab8070330 2024-04-25T14:06:23.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.788+0000 7ffab4c00640 1 --2- 172.21.15.86:0/1765506917 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7ffa9806f640 0x7ffa98071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:23.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.788+0000 7ffab4c00640 1 -- 172.21.15.86:0/1765506917 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7875+0+0 (secure 0 0 0) 0x7ffaac08f240 con 0x7ffab8070330 2024-04-25T14:06:23.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.788+0000 7ffab7e00640 1 --2- 172.21.15.86:0/1765506917 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7ffa9806f640 0x7ffa98071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:23.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.788+0000 7ffab7e00640 1 --2- 172.21.15.86:0/1765506917 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7ffa9806f640 0x7ffa98071b50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffab815bf10 tx=0x7ffaa4009040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:23.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.792+0000 7ffab4c00640 1 -- 172.21.15.86:0/1765506917 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffab815adc0 con 0x7ffab8070330 2024-04-25T14:06:23.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.980+0000 7ffabf400640 1 -- 172.21.15.86:0/1765506917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffab8076bf0 con 0x7ffab8070330 2024-04-25T14:06:23.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.980+0000 7ffab4c00640 1 -- 172.21.15.86:0/1765506917 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+6495 (secure 0 0 0) 0x7ffaac05b060 con 0x7ffab8070330 2024-04-25T14:06:23.982 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:23.982 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3545772170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3545772170},{"type":"v1","addr":"172.21.15.155:6834","nonce":3545772170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5980,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/766530004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":766530004},{"type":"v1","addr":"172.21.15.86:6837","nonce":766530004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:32.946320+0000","modified":"2024-04-25T14:06:03.253171+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6048},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6048":{"gid":6048,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6835/2169924270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2169924270},{"type":"v1","addr":"172.21.15.86:6835","nonce":2169924270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"fs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6048,"qdb_cluster":[6048]},"id":3},{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:54.565528+0000","modified":"2024-04-25T14:06:03.253173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6036":{"gid":6036,"name":"d","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6835/2914930843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2914930843},{"type":"v1","addr":"172.21.15.155:6835","nonce":2914930843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"fs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6036,"qdb_cluster":[6036]},"id":4}]} 2024-04-25T14:06:23.982 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 43 2024-04-25T14:06:23.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.980+0000 7ffabf400640 1 -- 172.21.15.86:0/1765506917 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7ffa9806f640 msgr2=0x7ffa98071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:23.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.984+0000 7ffabf400640 1 --2- 172.21.15.86:0/1765506917 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7ffa9806f640 0x7ffa98071b50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffab815bf10 tx=0x7ffaa4009040 comp rx=0 tx=0).stop 2024-04-25T14:06:23.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.984+0000 7ffabf400640 1 -- 172.21.15.86:0/1765506917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffab8070330 msgr2=0x7ffab809a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:23.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.984+0000 7ffabf400640 1 --2- 172.21.15.86:0/1765506917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffab8070330 0x7ffab809a9c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00c340 tx=0x7ffaac00c800 comp rx=0 tx=0).stop 2024-04-25T14:06:23.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.984+0000 7ffabf400640 1 -- 172.21.15.86:0/1765506917 shutdown_connections 2024-04-25T14:06:23.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.984+0000 7ffabf400640 1 --2- 172.21.15.86:0/1765506917 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7ffa9806f640 0x7ffa98071b50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.984+0000 7ffabf400640 1 --2- 172.21.15.86:0/1765506917 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffab815a9b0 0x7ffab80c2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.984+0000 7ffabf400640 1 --2- 172.21.15.86:0/1765506917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffab80cde90 0x7ffab815a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.984+0000 7ffabf400640 1 --2- 172.21.15.86:0/1765506917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffab8070330 0x7ffab809a9c0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:23.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.984+0000 7ffabf400640 1 -- 172.21.15.86:0/1765506917 >> 172.21.15.86:0/1765506917 conn(0x7ffab80b4650 msgr2=0x7ffab815d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:23.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.984+0000 7ffabf400640 1 -- 172.21.15.86:0/1765506917 shutdown_connections 2024-04-25T14:06:23.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:23.984+0000 7ffabf400640 1 -- 172.21.15.86:0/1765506917 wait complete. 2024-04-25T14:06:24.004 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:06:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.148+0000 7f473d000640 1 Processor -- start 2024-04-25T14:06:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.148+0000 7f473d000640 1 -- start start 2024-04-25T14:06:24.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.152+0000 7f473d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f47380cfca0 0x7f473806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.152+0000 7f473d000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f47380d16e0 0x7f47380d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.152+0000 7f473d000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47380d2030 con 0x7f473806c670 2024-04-25T14:06:24.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.152+0000 7f473d000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4738070470 con 0x7f47380cfca0 2024-04-25T14:06:24.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.152+0000 7f473d000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f4738072bd0 con 0x7f47380d16e0 2024-04-25T14:06:24.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.152+0000 7f4735e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f47380cfca0 0x7f473806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.152+0000 7f4735e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f47380cfca0 0x7f473806f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55420/0 (socket says 172.21.15.86:55420) 2024-04-25T14:06:24.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.152+0000 7f4736800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f473806c670 0x7f473806ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:32816/0 (socket says 172.21.15.86:32816) 2024-04-25T14:06:24.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.152+0000 7f4735e00640 1 -- 172.21.15.86:0/1676444963 learned_addr learned my addr 172.21.15.86:0/1676444963 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:24.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.152+0000 7f4735400640 1 --2- 172.21.15.86:0/1676444963 >> v2:172.21.15.155:3300/0 conn(0x7f47380d16e0 0x7f47380d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.152+0000 7f4735e00640 1 -- 172.21.15.86:0/1676444963 >> v2:172.21.15.155:3300/0 conn(0x7f47380d16e0 msgr2=0x7f47380d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:24.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.152+0000 7f4735e00640 1 --2- 172.21.15.86:0/1676444963 >> v2:172.21.15.155:3300/0 conn(0x7f47380d16e0 0x7f47380d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.152+0000 7f4735e00640 1 -- 172.21.15.86:0/1676444963 >> v1:172.21.15.86:6789/0 conn(0x7f473806c670 legacy=0x7f473806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:24.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.152+0000 7f4735e00640 1 -- 172.21.15.86:0/1676444963 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f473806f930 con 0x7f47380cfca0 2024-04-25T14:06:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.152+0000 7f4735400640 1 --2- 172.21.15.86:0/1676444963 >> v2:172.21.15.155:3300/0 conn(0x7f47380d16e0 0x7f47380d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:24.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.156+0000 7f4735e00640 1 --2- 172.21.15.86:0/1676444963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47380cfca0 0x7f473806f3f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f47380729c0 tx=0x7f472801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe32109b1917651 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:24.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.156+0000 7f4734a00640 1 -- 172.21.15.86:0/1676444963 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47280ab430 con 0x7f47380cfca0 2024-04-25T14:06:24.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.156+0000 7f4734a00640 1 -- 172.21.15.86:0/1676444963 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47280b0070 con 0x7f47380cfca0 2024-04-25T14:06:24.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.156+0000 7f4734a00640 1 -- 172.21.15.86:0/1676444963 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47280ab7f0 con 0x7f47380cfca0 2024-04-25T14:06:24.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.156+0000 7f473d000640 1 -- 172.21.15.86:0/1676444963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47380cfca0 msgr2=0x7f473806f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:24.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.156+0000 7f473d000640 1 --2- 172.21.15.86:0/1676444963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47380cfca0 0x7f473806f3f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f47380729c0 tx=0x7f472801c520 comp rx=0 tx=0).stop 2024-04-25T14:06:24.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.156+0000 7f473d000640 1 -- 172.21.15.86:0/1676444963 shutdown_connections 2024-04-25T14:06:24.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.156+0000 7f473d000640 1 --2- 172.21.15.86:0/1676444963 >> v2:172.21.15.155:3300/0 conn(0x7f47380d16e0 0x7f47380d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.156+0000 7f473d000640 1 --2- 172.21.15.86:0/1676444963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47380cfca0 0x7f473806f3f0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.156+0000 7f473d000640 1 -- 172.21.15.86:0/1676444963 >> 172.21.15.86:0/1676444963 conn(0x7f47380b4650 msgr2=0x7f47380b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:24.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.156+0000 7f473d000640 1 -- 172.21.15.86:0/1676444963 shutdown_connections 2024-04-25T14:06:24.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.156+0000 7f473d000640 1 -- 172.21.15.86:0/1676444963 wait complete. 2024-04-25T14:06:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.160+0000 7f473d000640 1 Processor -- start 2024-04-25T14:06:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.160+0000 7f473d000640 1 -- start start 2024-04-25T14:06:24.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.160+0000 7f473d000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f473806c670 0x7f47380cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.160+0000 7f473d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47380cfca0 0x7f4738154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.160+0000 7f473d000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47380d16e0 0x7f4738155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.160+0000 7f473d000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4738168060 con 0x7f47380cfca0 2024-04-25T14:06:24.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.160+0000 7f473d000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4738167ee0 con 0x7f47380d16e0 2024-04-25T14:06:24.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.160+0000 7f473d000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f47381681e0 con 0x7f473806c670 2024-04-25T14:06:24.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.160+0000 7f4735400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47380cfca0 0x7f4738154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.160+0000 7f4735400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47380cfca0 0x7f4738154ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55434/0 (socket says 172.21.15.86:55434) 2024-04-25T14:06:24.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.160+0000 7f4735400640 1 -- 172.21.15.86:0/3932860861 learned_addr learned my addr 172.21.15.86:0/3932860861 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:24.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.160+0000 7f4735e00640 1 --2- 172.21.15.86:0/3932860861 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f473806c670 0x7f47380cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.160+0000 7f4736800640 1 --2- 172.21.15.86:0/3932860861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47380d16e0 0x7f4738155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.160+0000 7f4735400640 1 -- 172.21.15.86:0/3932860861 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f473806c670 msgr2=0x7f47380cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:24.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.160+0000 7f4735400640 1 --2- 172.21.15.86:0/3932860861 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f473806c670 0x7f47380cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.160+0000 7f4735400640 1 -- 172.21.15.86:0/3932860861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47380d16e0 msgr2=0x7f4738155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:24.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.160+0000 7f4735400640 1 --2- 172.21.15.86:0/3932860861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47380d16e0 0x7f4738155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.160+0000 7f4735400640 1 -- 172.21.15.86:0/3932860861 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47380ccbe0 con 0x7f47380cfca0 2024-04-25T14:06:24.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.160+0000 7f4736800640 1 --2- 172.21.15.86:0/3932860861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47380d16e0 0x7f4738155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:24.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.160+0000 7f4735400640 1 --2- 172.21.15.86:0/3932860861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47380cfca0 0x7f4738154ce0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f472400c7f0 tx=0x7f472400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:24.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.160+0000 7f472e000640 1 -- 172.21.15.86:0/3932860861 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724015070 con 0x7f47380cfca0 2024-04-25T14:06:24.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.160+0000 7f472e000640 1 -- 172.21.15.86:0/3932860861 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4724004220 con 0x7f47380cfca0 2024-04-25T14:06:24.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.160+0000 7f473d000640 1 -- 172.21.15.86:0/3932860861 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47380cc340 con 0x7f47380cfca0 2024-04-25T14:06:24.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.160+0000 7f472e000640 1 -- 172.21.15.86:0/3932860861 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724003620 con 0x7f47380cfca0 2024-04-25T14:06:24.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.160+0000 7f473d000640 1 -- 172.21.15.86:0/3932860861 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47380cc900 con 0x7f47380cfca0 2024-04-25T14:06:24.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.164+0000 7f473d000640 1 -- 172.21.15.86:0/3932860861 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47380cc0b0 con 0x7f47380cfca0 2024-04-25T14:06:24.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.164+0000 7f472e000640 1 -- 172.21.15.86:0/3932860861 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f47240039d0 con 0x7f47380cfca0 2024-04-25T14:06:24.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.164+0000 7f472e000640 1 --2- 172.21.15.86:0/3932860861 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f471806f640 0x7f4718071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.164+0000 7f472e000640 1 -- 172.21.15.86:0/3932860861 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7875+0+0 (secure 0 0 0) 0x7f4724090080 con 0x7f47380cfca0 2024-04-25T14:06:24.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.164+0000 7f4735e00640 1 --2- 172.21.15.86:0/3932860861 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f471806f640 0x7f4718071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.164+0000 7f4735e00640 1 --2- 172.21.15.86:0/3932860861 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f471806f640 0x7f4718071b50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f47280af000 tx=0x7f47280a7000 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:24.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.168+0000 7f472e000640 1 -- 172.21.15.86:0/3932860861 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f472405c2f0 con 0x7f47380cfca0 2024-04-25T14:06:24.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.324+0000 7f473d000640 1 -- 172.21.15.86:0/3932860861 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f473809f4b0 con 0x7f47380cfca0 2024-04-25T14:06:24.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.328+0000 7f472e000640 1 -- 172.21.15.86:0/3932860861 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+22031 (secure 0 0 0) 0x7f472405bbf0 con 0x7f47380cfca0 2024-04-25T14:06:24.329 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:24.329 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":59,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:06:19.409466+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"fs1_metadata","create_time":"2024-04-25T14:05:30.939601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"fs1"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"fs1_data","create_time":"2024-04-25T14:05:31.898262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"fs1"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"fs2_metadata","create_time":"2024-04-25T14:05:52.841764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"fs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"fs2_data","create_time":"2024-04-25T14:05:53.524454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"fs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/145108428":"2024-04-26T14:06:19.409429+0000","172.21.15.86:6833/2427179681":"2024-04-26T14:06:19.409429+0000","172.21.15.86:6832/2427179681":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/1321309855":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/525010030":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/3470358079":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/1177884430":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/3644218120":"2024-04-26T14:06:19.409429+0000","172.21.15.86:6836/1732615622":"2024-04-26T14:06:00.230367+0000","172.21.15.86:6837/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:6837/2418778265":"2024-04-26T14:06:00.222915+0000","172.21.15.86:0/1843278167":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6832/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/2073988773":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6835/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:0/2602032945":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/3558294881":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6835/615772001":"2024-04-26T14:05:42.882867+0000","172.21.15.86:6837/615772001":"2024-04-26T14:05:42.882867+0000","172.21.15.86:6833/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6835/2418778265":"2024-04-26T14:06:00.222915+0000","172.21.15.86:6834/1732615622":"2024-04-26T14:06:00.230367+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:06:24.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.328+0000 7f473d000640 1 -- 172.21.15.86:0/3932860861 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f471806f640 msgr2=0x7f4718071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:24.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.328+0000 7f473d000640 1 --2- 172.21.15.86:0/3932860861 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f471806f640 0x7f4718071b50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f47280af000 tx=0x7f47280a7000 comp rx=0 tx=0).stop 2024-04-25T14:06:24.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.328+0000 7f473d000640 1 -- 172.21.15.86:0/3932860861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47380cfca0 msgr2=0x7f4738154ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:24.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.328+0000 7f473d000640 1 --2- 172.21.15.86:0/3932860861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47380cfca0 0x7f4738154ce0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f472400c7f0 tx=0x7f472400ccb0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.328+0000 7f473d000640 1 -- 172.21.15.86:0/3932860861 shutdown_connections 2024-04-25T14:06:24.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.328+0000 7f473d000640 1 --2- 172.21.15.86:0/3932860861 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f471806f640 0x7f4718071b50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.328+0000 7f473d000640 1 --2- 172.21.15.86:0/3932860861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47380d16e0 0x7f4738155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.328+0000 7f473d000640 1 --2- 172.21.15.86:0/3932860861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47380cfca0 0x7f4738154ce0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.328+0000 7f473d000640 1 --2- 172.21.15.86:0/3932860861 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f473806c670 0x7f47380cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.328+0000 7f473d000640 1 -- 172.21.15.86:0/3932860861 >> 172.21.15.86:0/3932860861 conn(0x7f47380b4650 msgr2=0x7f47380ceeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:24.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.328+0000 7f473d000640 1 -- 172.21.15.86:0/3932860861 shutdown_connections 2024-04-25T14:06:24.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.328+0000 7f473d000640 1 -- 172.21.15.86:0/3932860861 wait complete. 2024-04-25T14:06:24.350 INFO:tasks.cephfs.filesystem:Destroying file system fs1 and related pools 2024-04-25T14:06:24.350 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:06:24.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.496+0000 7f3485000640 1 Processor -- start 2024-04-25T14:06:24.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.496+0000 7f3485000640 1 -- start start 2024-04-25T14:06:24.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f3485000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f348006e340 0x7f348006e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f3485000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34801648e0 con 0x7f3480072630 2024-04-25T14:06:24.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f3485000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34800d1520 con 0x7f3480163980 2024-04-25T14:06:24.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f3485000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f34800d23e0 con 0x7f348006e340 2024-04-25T14:06:24.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f347de00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3480072630 0x7f3480072a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:32820/0 (socket says 172.21.15.86:32820) 2024-04-25T14:06:24.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f347de00640 1 -- 172.21.15.86:0/2969530239 learned_addr learned my addr 172.21.15.86:0/2969530239 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:24.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f347e800640 1 --2- 172.21.15.86:0/2969530239 >> v2:172.21.15.155:3300/0 conn(0x7f348006e340 0x7f348006e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f347d400640 1 -- 172.21.15.86:0/2969530239 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2264089821 0 0) 0x7f34801648e0 con 0x7f3480072630 2024-04-25T14:06:24.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f347d400640 1 -- 172.21.15.86:0/2969530239 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3464003530 con 0x7f3480072630 2024-04-25T14:06:24.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f347d400640 1 -- 172.21.15.86:0/2969530239 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 779787751 0 0) 0x7f3464003530 con 0x7f3480072630 2024-04-25T14:06:24.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f347d400640 1 -- 172.21.15.86:0/2969530239 >> v2:172.21.15.155:3300/0 conn(0x7f348006e340 msgr2=0x7f348006e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:24.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f347d400640 1 --2- 172.21.15.86:0/2969530239 >> v2:172.21.15.155:3300/0 conn(0x7f348006e340 0x7f348006e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f347d400640 1 -- 172.21.15.86:0/2969530239 >> v1:172.21.15.155:6789/0 conn(0x7f3480163980 legacy=0x7f3480163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:24.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f347d400640 1 -- 172.21.15.86:0/2969530239 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3480164600 con 0x7f3480072630 2024-04-25T14:06:24.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f347d400640 1 -- 172.21.15.86:0/2969530239 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f3470001680 con 0x7f3480072630 2024-04-25T14:06:24.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f347d400640 1 -- 172.21.15.86:0/2969530239 >> v1:172.21.15.86:6789/0 conn(0x7f3480072630 legacy=0x7f3480072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:24.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f347d400640 1 --2- 172.21.15.86:0/2969530239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3464003a30 0x7f3464003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f347d400640 1 --2- 172.21.15.86:0/2969530239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3464004810 0x7f3464004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f347d400640 1 --2- 172.21.15.86:0/2969530239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3464005360 0x7f346400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f347d400640 1 -- 172.21.15.86:0/2969530239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34800d23e0 con 0x7f3464005360 2024-04-25T14:06:24.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f347d400640 1 -- 172.21.15.86:0/2969530239 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f346400c290 con 0x7f3464003a30 2024-04-25T14:06:24.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f347d400640 1 -- 172.21.15.86:0/2969530239 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f346400c490 con 0x7f3464004810 2024-04-25T14:06:24.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f347e800640 1 --2- 172.21.15.86:0/2969530239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3464005360 0x7f346400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f347de00640 1 --2- 172.21.15.86:0/2969530239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3464003a30 0x7f3464003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f3475400640 1 --2- 172.21.15.86:0/2969530239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3464004810 0x7f3464004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f347e800640 1 -- 172.21.15.86:0/2969530239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3464004810 msgr2=0x7f3464004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:24.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f347e800640 1 --2- 172.21.15.86:0/2969530239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3464004810 0x7f3464004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f347e800640 1 -- 172.21.15.86:0/2969530239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3464003a30 msgr2=0x7f3464003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:24.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f347e800640 1 --2- 172.21.15.86:0/2969530239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3464003a30 0x7f3464003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.500+0000 7f347e800640 1 -- 172.21.15.86:0/2969530239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3464003530 con 0x7f3464005360 2024-04-25T14:06:24.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.504+0000 7f347e800640 1 -- 172.21.15.86:0/2969530239 reap_dead start 2024-04-25T14:06:24.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.504+0000 7f347e800640 1 --2- 172.21.15.86:0/2969530239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3464005360 0x7f346400b9d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3478009f90 tx=0x7f347801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3a67295faeded0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:24.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.504+0000 7f347d400640 1 -- 172.21.15.86:0/2969530239 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34780b2070 con 0x7f3464005360 2024-04-25T14:06:24.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.504+0000 7f347d400640 1 -- 172.21.15.86:0/2969530239 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34780a5c90 con 0x7f3464005360 2024-04-25T14:06:24.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.504+0000 7f347d400640 1 -- 172.21.15.86:0/2969530239 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3478009c10 con 0x7f3464005360 2024-04-25T14:06:24.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.504+0000 7f3485000640 1 -- 172.21.15.86:0/2969530239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3464005360 msgr2=0x7f346400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:24.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.504+0000 7f3485000640 1 --2- 172.21.15.86:0/2969530239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3464005360 0x7f346400b9d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3478009f90 tx=0x7f347801cc80 comp rx=0 tx=0).stop 2024-04-25T14:06:24.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.504+0000 7f3485000640 1 -- 172.21.15.86:0/2969530239 shutdown_connections 2024-04-25T14:06:24.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.504+0000 7f3485000640 1 --2- 172.21.15.86:0/2969530239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3464005360 0x7f346400b9d0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.504+0000 7f3485000640 1 -- 172.21.15.86:0/2969530239 >> 172.21.15.86:0/2969530239 conn(0x7f34800d3580 msgr2=0x7f34800d3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:24.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.504+0000 7f3485000640 1 -- 172.21.15.86:0/2969530239 shutdown_connections 2024-04-25T14:06:24.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.504+0000 7f3485000640 1 -- 172.21.15.86:0/2969530239 wait complete. 2024-04-25T14:06:24.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.508+0000 7f3485000640 1 Processor -- start 2024-04-25T14:06:24.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.508+0000 7f3485000640 1 -- start start 2024-04-25T14:06:24.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.508+0000 7f3485000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3464005360 0x7f34801ec1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.508+0000 7f3485000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34800afcf0 0x7f3480162940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.508+0000 7f3485000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3480162e80 0x7f3480152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.508+0000 7f3485000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34800cfe30 con 0x7f34800afcf0 2024-04-25T14:06:24.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.508+0000 7f3485000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f34800cfcb0 con 0x7f3464005360 2024-04-25T14:06:24.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.508+0000 7f3485000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3480072a40 con 0x7f3480162e80 2024-04-25T14:06:24.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.508+0000 7f3475400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34800afcf0 0x7f3480162940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.508+0000 7f3475400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34800afcf0 0x7f3480162940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55448/0 (socket says 172.21.15.86:55448) 2024-04-25T14:06:24.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.508+0000 7f3475400640 1 -- 172.21.15.86:0/1017600366 learned_addr learned my addr 172.21.15.86:0/1017600366 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:24.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.508+0000 7f347de00640 1 --2- 172.21.15.86:0/1017600366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3464005360 0x7f34801ec1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.508+0000 7f347e800640 1 --2- 172.21.15.86:0/1017600366 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3480162e80 0x7f3480152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.508+0000 7f3475400640 1 -- 172.21.15.86:0/1017600366 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3480162e80 msgr2=0x7f3480152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:24.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.508+0000 7f3475400640 1 --2- 172.21.15.86:0/1017600366 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3480162e80 0x7f3480152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.508+0000 7f3475400640 1 -- 172.21.15.86:0/1017600366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3464005360 msgr2=0x7f34801ec1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:24.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.508+0000 7f3475400640 1 --2- 172.21.15.86:0/1017600366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3464005360 0x7f34801ec1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.508+0000 7f3475400640 1 -- 172.21.15.86:0/1017600366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34800aec50 con 0x7f34800afcf0 2024-04-25T14:06:24.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.512+0000 7f3475400640 1 --2- 172.21.15.86:0/1017600366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34800afcf0 0x7f3480162940 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f3468012880 tx=0x7f3468012c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:24.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.512+0000 7f3476a00640 1 -- 172.21.15.86:0/1017600366 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3468010bd0 con 0x7f34800afcf0 2024-04-25T14:06:24.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.512+0000 7f3476a00640 1 -- 172.21.15.86:0/1017600366 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f346801b070 con 0x7f34800afcf0 2024-04-25T14:06:24.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.512+0000 7f3476a00640 1 -- 172.21.15.86:0/1017600366 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3468016480 con 0x7f34800afcf0 2024-04-25T14:06:24.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.512+0000 7f3485000640 1 -- 172.21.15.86:0/1017600366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34800aef30 con 0x7f34800afcf0 2024-04-25T14:06:24.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.512+0000 7f3485000640 1 -- 172.21.15.86:0/1017600366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34800af4f0 con 0x7f34800afcf0 2024-04-25T14:06:24.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.512+0000 7f3485000640 1 -- 172.21.15.86:0/1017600366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3480072170 con 0x7f34800afcf0 2024-04-25T14:06:24.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.512+0000 7f3476a00640 1 -- 172.21.15.86:0/1017600366 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f3468016620 con 0x7f34800afcf0 2024-04-25T14:06:24.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.512+0000 7f3476a00640 1 --2- 172.21.15.86:0/1017600366 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f345c06f640 0x7f345c071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.516+0000 7f3476a00640 1 -- 172.21.15.86:0/1017600366 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7875+0+0 (secure 0 0 0) 0x7f3468093650 con 0x7f34800afcf0 2024-04-25T14:06:24.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.516+0000 7f347de00640 1 --2- 172.21.15.86:0/1017600366 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f345c06f640 0x7f345c071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.516+0000 7f347de00640 1 --2- 172.21.15.86:0/1017600366 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f345c06f640 0x7f345c071b50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3470000c10 tx=0x7f3470007900 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:24.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.520+0000 7f3476a00640 1 -- 172.21.15.86:0/1017600366 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f346805f950 con 0x7f34800afcf0 2024-04-25T14:06:24.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.708+0000 7f3485000640 1 -- 172.21.15.86:0/1017600366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3480076bf0 con 0x7f34800afcf0 2024-04-25T14:06:24.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.708+0000 7f3476a00640 1 -- 172.21.15.86:0/1017600366 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+6495 (secure 0 0 0) 0x7f346805f250 con 0x7f34800afcf0 2024-04-25T14:06:24.709 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:24.709 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3545772170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3545772170},{"type":"v1","addr":"172.21.15.155:6834","nonce":3545772170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5980,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/766530004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":766530004},{"type":"v1","addr":"172.21.15.86:6837","nonce":766530004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:32.946320+0000","modified":"2024-04-25T14:06:03.253171+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6048},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6048":{"gid":6048,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6835/2169924270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2169924270},{"type":"v1","addr":"172.21.15.86:6835","nonce":2169924270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"fs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6048,"qdb_cluster":[6048]},"id":3},{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:54.565528+0000","modified":"2024-04-25T14:06:03.253173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6036":{"gid":6036,"name":"d","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6835/2914930843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2914930843},{"type":"v1","addr":"172.21.15.155:6835","nonce":2914930843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"fs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6036,"qdb_cluster":[6036]},"id":4}]} 2024-04-25T14:06:24.710 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 43 2024-04-25T14:06:24.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.708+0000 7f3485000640 1 -- 172.21.15.86:0/1017600366 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f345c06f640 msgr2=0x7f345c071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:24.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.708+0000 7f3485000640 1 --2- 172.21.15.86:0/1017600366 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f345c06f640 0x7f345c071b50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3470000c10 tx=0x7f3470007900 comp rx=0 tx=0).stop 2024-04-25T14:06:24.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.708+0000 7f3485000640 1 -- 172.21.15.86:0/1017600366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34800afcf0 msgr2=0x7f3480162940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:24.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.708+0000 7f3485000640 1 --2- 172.21.15.86:0/1017600366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34800afcf0 0x7f3480162940 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f3468012880 tx=0x7f3468012c80 comp rx=0 tx=0).stop 2024-04-25T14:06:24.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.708+0000 7f3485000640 1 -- 172.21.15.86:0/1017600366 shutdown_connections 2024-04-25T14:06:24.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.708+0000 7f3485000640 1 --2- 172.21.15.86:0/1017600366 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f345c06f640 0x7f345c071b50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.708+0000 7f3485000640 1 --2- 172.21.15.86:0/1017600366 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3480162e80 0x7f3480152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.708+0000 7f3485000640 1 --2- 172.21.15.86:0/1017600366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34800afcf0 0x7f3480162940 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.708+0000 7f3485000640 1 --2- 172.21.15.86:0/1017600366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3464005360 0x7f34801ec1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.708+0000 7f3485000640 1 -- 172.21.15.86:0/1017600366 >> 172.21.15.86:0/1017600366 conn(0x7f34800d3580 msgr2=0x7f3480161b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:24.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.708+0000 7f3485000640 1 -- 172.21.15.86:0/1017600366 shutdown_connections 2024-04-25T14:06:24.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.708+0000 7f3485000640 1 -- 172.21.15.86:0/1017600366 wait complete. 2024-04-25T14:06:24.730 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:06:24.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.876+0000 7f3c10c00640 1 Processor -- start 2024-04-25T14:06:24.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.876+0000 7f3c10c00640 1 -- start start 2024-04-25T14:06:24.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.876+0000 7f3c10c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c0c0cfca0 0x7f3c0c06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.876+0000 7f3c10c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3c0c0d16e0 0x7f3c0c0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.876+0000 7f3c10c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c0c0d2030 con 0x7f3c0c06c670 2024-04-25T14:06:24.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.876+0000 7f3c10c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3c0c070470 con 0x7f3c0c0d16e0 2024-04-25T14:06:24.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.876+0000 7f3c10c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3c0c072bd0 con 0x7f3c0c0cfca0 2024-04-25T14:06:24.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.880+0000 7f3c01400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3c0c0d16e0 0x7f3c0c0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.880+0000 7f3c09e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c0c0cfca0 0x7f3c0c06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.880+0000 7f3c0a800640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f3c0c06c670 0x7f3c0c06ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:60036/0 (socket says 172.21.15.86:60036) 2024-04-25T14:06:24.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.880+0000 7f3c0a800640 1 -- 172.21.15.86:0/650864581 learned_addr learned my addr 172.21.15.86:0/650864581 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:24.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.880+0000 7f3c09400640 1 -- 172.21.15.86:0/650864581 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1284802323 0 0) 0x7f3c04001120 con 0x7f3c0c06c670 2024-04-25T14:06:24.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.880+0000 7f3c09400640 1 -- 172.21.15.86:0/650864581 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3bf0003530 con 0x7f3c0c06c670 2024-04-25T14:06:24.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.880+0000 7f3c09e00640 1 -- 172.21.15.86:0/650864581 >> v2:172.21.15.155:3300/0 conn(0x7f3c0c0d16e0 msgr2=0x7f3c0c0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:24.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.880+0000 7f3c09e00640 1 --2- 172.21.15.86:0/650864581 >> v2:172.21.15.155:3300/0 conn(0x7f3c0c0d16e0 0x7f3c0c0d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.880+0000 7f3c09e00640 1 -- 172.21.15.86:0/650864581 >> v1:172.21.15.155:6789/0 conn(0x7f3c0c06c670 legacy=0x7f3c0c06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:24.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.880+0000 7f3c09e00640 1 -- 172.21.15.86:0/650864581 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c0c06f930 con 0x7f3c0c0cfca0 2024-04-25T14:06:24.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.880+0000 7f3c01400640 1 --2- 172.21.15.86:0/650864581 >> v2:172.21.15.155:3300/0 conn(0x7f3c0c0d16e0 0x7f3c0c0d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:24.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.880+0000 7f3c09e00640 1 --2- 172.21.15.86:0/650864581 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c0c0cfca0 0x7f3c0c06f3f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc009d90 tx=0x7f3bfc01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=979416679847d771 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:24.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.884+0000 7f3c09400640 1 -- 172.21.15.86:0/650864581 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc0b2070 con 0x7f3c0c0cfca0 2024-04-25T14:06:24.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.884+0000 7f3c09400640 1 -- 172.21.15.86:0/650864581 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bfc0a5940 con 0x7f3c0c0cfca0 2024-04-25T14:06:24.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.884+0000 7f3c09400640 1 -- 172.21.15.86:0/650864581 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc0a5d10 con 0x7f3c0c0cfca0 2024-04-25T14:06:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.884+0000 7f3c10c00640 1 -- 172.21.15.86:0/650864581 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c0c0cfca0 msgr2=0x7f3c0c06f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.884+0000 7f3c10c00640 1 --2- 172.21.15.86:0/650864581 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c0c0cfca0 0x7f3c0c06f3f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc009d90 tx=0x7f3bfc01c970 comp rx=0 tx=0).stop 2024-04-25T14:06:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.884+0000 7f3c10c00640 1 -- 172.21.15.86:0/650864581 shutdown_connections 2024-04-25T14:06:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.884+0000 7f3c10c00640 1 --2- 172.21.15.86:0/650864581 >> v2:172.21.15.155:3300/0 conn(0x7f3c0c0d16e0 0x7f3c0c0d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.884+0000 7f3c10c00640 1 --2- 172.21.15.86:0/650864581 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c0c0cfca0 0x7f3c0c06f3f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.884+0000 7f3c10c00640 1 -- 172.21.15.86:0/650864581 >> 172.21.15.86:0/650864581 conn(0x7f3c0c0b4650 msgr2=0x7f3c0c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.884+0000 7f3c10c00640 1 -- 172.21.15.86:0/650864581 shutdown_connections 2024-04-25T14:06:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.884+0000 7f3c10c00640 1 -- 172.21.15.86:0/650864581 wait complete. 2024-04-25T14:06:24.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.884+0000 7f3c10c00640 1 Processor -- start 2024-04-25T14:06:24.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.884+0000 7f3c10c00640 1 -- start start 2024-04-25T14:06:24.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.888+0000 7f3c10c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c0c06c670 0x7f3c0c09a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.888+0000 7f3c10c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c0c0cfca0 0x7f3c0c156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.888+0000 7f3c10c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c0c0d16e0 0x7f3c0c1566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.888+0000 7f3c10c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c0cdee0 con 0x7f3c0c0d16e0 2024-04-25T14:06:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.888+0000 7f3c10c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c0cdd60 con 0x7f3c0c06c670 2024-04-25T14:06:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.888+0000 7f3c10c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3c0c0ce060 con 0x7f3c0c0cfca0 2024-04-25T14:06:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.888+0000 7f3c0a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c0c0d16e0 0x7f3c0c1566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.888+0000 7f3c0a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c0c0d16e0 0x7f3c0c1566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55450/0 (socket says 172.21.15.86:55450) 2024-04-25T14:06:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.888+0000 7f3c0a800640 1 -- 172.21.15.86:0/1939728660 learned_addr learned my addr 172.21.15.86:0/1939728660 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.888+0000 7f3c09e00640 1 --2- 172.21.15.86:0/1939728660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c0c06c670 0x7f3c0c09a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.888+0000 7f3c01400640 1 --2- 172.21.15.86:0/1939728660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c0c0cfca0 0x7f3c0c156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.888+0000 7f3c0a800640 1 -- 172.21.15.86:0/1939728660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c0c0cfca0 msgr2=0x7f3c0c156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.888+0000 7f3c0a800640 1 --2- 172.21.15.86:0/1939728660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c0c0cfca0 0x7f3c0c156170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.888+0000 7f3c0a800640 1 -- 172.21.15.86:0/1939728660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c0c06c670 msgr2=0x7f3c0c09a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.888+0000 7f3c0a800640 1 --2- 172.21.15.86:0/1939728660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c0c06c670 0x7f3c0c09a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.888+0000 7f3c0a800640 1 -- 172.21.15.86:0/1939728660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c0c15a130 con 0x7f3c0c0d16e0 2024-04-25T14:06:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.888+0000 7f3c09e00640 1 --2- 172.21.15.86:0/1939728660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c0c06c670 0x7f3c0c09a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.888+0000 7f3c01400640 1 --2- 172.21.15.86:0/1939728660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c0c0cfca0 0x7f3c0c156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.888+0000 7f3c0a800640 1 --2- 172.21.15.86:0/1939728660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c0c0d16e0 0x7f3c0c1566b0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400c420 tx=0x7f3c0400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.888+0000 7f3c02a00640 1 -- 172.21.15.86:0/1939728660 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04017070 con 0x7f3c0c0d16e0 2024-04-25T14:06:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.888+0000 7f3c02a00640 1 -- 172.21.15.86:0/1939728660 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c04005cf0 con 0x7f3c0c0d16e0 2024-04-25T14:06:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.888+0000 7f3c02a00640 1 -- 172.21.15.86:0/1939728660 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c040072d0 con 0x7f3c0c0d16e0 2024-04-25T14:06:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.888+0000 7f3c10c00640 1 -- 172.21.15.86:0/1939728660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c0c158050 con 0x7f3c0c0d16e0 2024-04-25T14:06:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.888+0000 7f3c10c00640 1 -- 172.21.15.86:0/1939728660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c0c1582f0 con 0x7f3c0c0d16e0 2024-04-25T14:06:24.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.888+0000 7f3c10c00640 1 -- 172.21.15.86:0/1939728660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c0c09f4b0 con 0x7f3c0c0d16e0 2024-04-25T14:06:24.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.892+0000 7f3c02a00640 1 -- 172.21.15.86:0/1939728660 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f3c0401a070 con 0x7f3c0c0d16e0 2024-04-25T14:06:24.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.892+0000 7f3c02a00640 1 --2- 172.21.15.86:0/1939728660 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3be806f670 0x7f3be8071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.892+0000 7f3c02a00640 1 -- 172.21.15.86:0/1939728660 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7875+0+0 (secure 0 0 0) 0x7f3c0408f0e0 con 0x7f3c0c0d16e0 2024-04-25T14:06:24.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.892+0000 7f3c09e00640 1 --2- 172.21.15.86:0/1939728660 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3be806f670 0x7f3be8071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.892+0000 7f3c09e00640 1 --2- 172.21.15.86:0/1939728660 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3be806f670 0x7f3be8071b80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0af000 tx=0x7f3bfc0a7070 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:24.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:24.896+0000 7f3c02a00640 1 -- 172.21.15.86:0/1939728660 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3c0c0d2030 con 0x7f3c0c0d16e0 2024-04-25T14:06:25.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.088+0000 7f3c10c00640 1 -- 172.21.15.86:0/1939728660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c0c06cf70 con 0x7f3c0c0d16e0 2024-04-25T14:06:25.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.088+0000 7f3c02a00640 1 -- 172.21.15.86:0/1939728660 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+6495 (secure 0 0 0) 0x7f3c0405ae20 con 0x7f3c0c0d16e0 2024-04-25T14:06:25.090 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:25.090 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3545772170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3545772170},{"type":"v1","addr":"172.21.15.155:6834","nonce":3545772170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5980,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/766530004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":766530004},{"type":"v1","addr":"172.21.15.86:6837","nonce":766530004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:32.946320+0000","modified":"2024-04-25T14:06:03.253171+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6048},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6048":{"gid":6048,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6835/2169924270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2169924270},{"type":"v1","addr":"172.21.15.86:6835","nonce":2169924270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"fs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6048,"qdb_cluster":[6048]},"id":3},{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:54.565528+0000","modified":"2024-04-25T14:06:03.253173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6036":{"gid":6036,"name":"d","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6835/2914930843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2914930843},{"type":"v1","addr":"172.21.15.155:6835","nonce":2914930843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"fs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6036,"qdb_cluster":[6036]},"id":4}]} 2024-04-25T14:06:25.091 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 43 2024-04-25T14:06:25.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.092+0000 7f3c10c00640 1 -- 172.21.15.86:0/1939728660 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3be806f670 msgr2=0x7f3be8071b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.092+0000 7f3c10c00640 1 --2- 172.21.15.86:0/1939728660 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3be806f670 0x7f3be8071b80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0af000 tx=0x7f3bfc0a7070 comp rx=0 tx=0).stop 2024-04-25T14:06:25.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.092+0000 7f3c10c00640 1 -- 172.21.15.86:0/1939728660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c0c0d16e0 msgr2=0x7f3c0c1566b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.092+0000 7f3c10c00640 1 --2- 172.21.15.86:0/1939728660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c0c0d16e0 0x7f3c0c1566b0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400c420 tx=0x7f3c0400c8e0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.092+0000 7f3c10c00640 1 -- 172.21.15.86:0/1939728660 shutdown_connections 2024-04-25T14:06:25.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.092+0000 7f3c10c00640 1 --2- 172.21.15.86:0/1939728660 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3be806f670 0x7f3be8071b80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.092+0000 7f3c10c00640 1 --2- 172.21.15.86:0/1939728660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c0c0d16e0 0x7f3c0c1566b0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.092+0000 7f3c10c00640 1 --2- 172.21.15.86:0/1939728660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c0c0cfca0 0x7f3c0c156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.092+0000 7f3c10c00640 1 --2- 172.21.15.86:0/1939728660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c0c06c670 0x7f3c0c09a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.092+0000 7f3c10c00640 1 -- 172.21.15.86:0/1939728660 >> 172.21.15.86:0/1939728660 conn(0x7f3c0c0b4650 msgr2=0x7f3c0c0af620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:25.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.092+0000 7f3c10c00640 1 -- 172.21.15.86:0/1939728660 shutdown_connections 2024-04-25T14:06:25.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.092+0000 7f3c10c00640 1 -- 172.21.15.86:0/1939728660 wait complete. 2024-04-25T14:06:25.112 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:06:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.256+0000 7f774fa00640 1 Processor -- start 2024-04-25T14:06:25.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.256+0000 7f774fa00640 1 -- start start 2024-04-25T14:06:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.260+0000 7f774fa00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f77480d2420 0x7f77480d2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.260+0000 7f774fa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f77480b6490 0x7f77480b68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.260+0000 7f774fa00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7748162480 con 0x7f7748070e20 2024-04-25T14:06:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.260+0000 7f774fa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f77480d16f0 con 0x7f77480b6490 2024-04-25T14:06:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.260+0000 7f774fa00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f77480d1940 con 0x7f77480d2420 2024-04-25T14:06:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.260+0000 7f774da00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f77480b6490 0x7f77480b68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.260+0000 7f774da00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f77480b6490 0x7f77480b68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55458/0 (socket says 172.21.15.86:55458) 2024-04-25T14:06:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.260+0000 7f774da00640 1 -- 172.21.15.86:0/312024794 learned_addr learned my addr 172.21.15.86:0/312024794 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.260+0000 7f774d000640 1 --2- 172.21.15.86:0/312024794 >> v2:172.21.15.155:3300/0 conn(0x7f77480d2420 0x7f77480d2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.260+0000 7f774da00640 1 -- 172.21.15.86:0/312024794 >> v2:172.21.15.155:3300/0 conn(0x7f77480d2420 msgr2=0x7f77480d2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.260+0000 7f774da00640 1 --2- 172.21.15.86:0/312024794 >> v2:172.21.15.155:3300/0 conn(0x7f77480d2420 0x7f77480d2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.260+0000 7f774da00640 1 -- 172.21.15.86:0/312024794 >> v1:172.21.15.155:6789/0 conn(0x7f7748070e20 legacy=0x7f7748071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.260+0000 7f774da00640 1 -- 172.21.15.86:0/312024794 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77480d1c20 con 0x7f77480b6490 2024-04-25T14:06:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.264+0000 7f774da00640 1 --2- 172.21.15.86:0/312024794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77480b6490 0x7f77480b68a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f77480b6070 tx=0x7f774001c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18623dd2d20932a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.264+0000 7f7747400640 1 -- 172.21.15.86:0/312024794 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77400a5720 con 0x7f77480b6490 2024-04-25T14:06:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.264+0000 7f7747400640 1 -- 172.21.15.86:0/312024794 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77400a58c0 con 0x7f77480b6490 2024-04-25T14:06:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.264+0000 7f7747400640 1 -- 172.21.15.86:0/312024794 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77400a5c90 con 0x7f77480b6490 2024-04-25T14:06:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.264+0000 7f774fa00640 1 -- 172.21.15.86:0/312024794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77480b6490 msgr2=0x7f77480b68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.264+0000 7f774fa00640 1 --2- 172.21.15.86:0/312024794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77480b6490 0x7f77480b68a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f77480b6070 tx=0x7f774001c9b0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.264+0000 7f774fa00640 1 -- 172.21.15.86:0/312024794 shutdown_connections 2024-04-25T14:06:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.264+0000 7f774fa00640 1 --2- 172.21.15.86:0/312024794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77480b6490 0x7f77480b68a0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.264+0000 7f774fa00640 1 --2- 172.21.15.86:0/312024794 >> v2:172.21.15.155:3300/0 conn(0x7f77480d2420 0x7f77480d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.264+0000 7f774fa00640 1 -- 172.21.15.86:0/312024794 >> 172.21.15.86:0/312024794 conn(0x7f77480cffc0 msgr2=0x7f77480d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.264+0000 7f774fa00640 1 -- 172.21.15.86:0/312024794 shutdown_connections 2024-04-25T14:06:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.264+0000 7f774fa00640 1 -- 172.21.15.86:0/312024794 wait complete. 2024-04-25T14:06:25.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.264+0000 7f774fa00640 1 Processor -- start 2024-04-25T14:06:25.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.264+0000 7f774fa00640 1 -- start start 2024-04-25T14:06:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.268+0000 7f774fa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7748070e20 0x7f774815eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.268+0000 7f774fa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77480b6490 0x7f77481405a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.268+0000 7f774fa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77480d2420 0x7f7748140ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.268+0000 7f774fa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7748161310 con 0x7f77480d2420 2024-04-25T14:06:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.268+0000 7f774fa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f77480cdf30 con 0x7f77480b6490 2024-04-25T14:06:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.268+0000 7f774fa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7748141220 con 0x7f7748070e20 2024-04-25T14:06:25.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.268+0000 7f774da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77480d2420 0x7f7748140ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.268+0000 7f774da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77480d2420 0x7f7748140ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55466/0 (socket says 172.21.15.86:55466) 2024-04-25T14:06:25.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.268+0000 7f774da00640 1 -- 172.21.15.86:0/435425883 learned_addr learned my addr 172.21.15.86:0/435425883 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:25.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.268+0000 7f774d000640 1 --2- 172.21.15.86:0/435425883 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7748070e20 0x7f774815eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.268+0000 7f7747e00640 1 --2- 172.21.15.86:0/435425883 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77480b6490 0x7f77481405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.268+0000 7f774da00640 1 -- 172.21.15.86:0/435425883 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7748070e20 msgr2=0x7f774815eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.268+0000 7f774da00640 1 --2- 172.21.15.86:0/435425883 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7748070e20 0x7f774815eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.268+0000 7f774da00640 1 -- 172.21.15.86:0/435425883 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77480b6490 msgr2=0x7f77481405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.268+0000 7f774da00640 1 --2- 172.21.15.86:0/435425883 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77480b6490 0x7f77481405a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.268+0000 7f774da00640 1 -- 172.21.15.86:0/435425883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774813e7a0 con 0x7f77480d2420 2024-04-25T14:06:25.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.268+0000 7f774d000640 1 --2- 172.21.15.86:0/435425883 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7748070e20 0x7f774815eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:25.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.268+0000 7f7747e00640 1 --2- 172.21.15.86:0/435425883 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77480b6490 0x7f77481405a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:25.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.268+0000 7f774da00640 1 --2- 172.21.15.86:0/435425883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77480d2420 0x7f7748140ae0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f77400af040 tx=0x7f7740019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:25.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.268+0000 7f7744c00640 1 -- 172.21.15.86:0/435425883 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7740009510 con 0x7f77480d2420 2024-04-25T14:06:25.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.268+0000 7f7744c00640 1 -- 172.21.15.86:0/435425883 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77400be070 con 0x7f77480d2420 2024-04-25T14:06:25.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.268+0000 7f7744c00640 1 -- 172.21.15.86:0/435425883 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77400b88b0 con 0x7f77480d2420 2024-04-25T14:06:25.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.268+0000 7f774fa00640 1 -- 172.21.15.86:0/435425883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7748141470 con 0x7f77480d2420 2024-04-25T14:06:25.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.268+0000 7f774fa00640 1 -- 172.21.15.86:0/435425883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77481402e0 con 0x7f77480d2420 2024-04-25T14:06:25.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.268+0000 7f774fa00640 1 -- 172.21.15.86:0/435425883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7748076bf0 con 0x7f77480d2420 2024-04-25T14:06:25.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.272+0000 7f7744c00640 1 -- 172.21.15.86:0/435425883 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f77400b8a50 con 0x7f77480d2420 2024-04-25T14:06:25.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.272+0000 7f7744c00640 1 --2- 172.21.15.86:0/435425883 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f772806f670 0x7f7728071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.272+0000 7f7744c00640 1 -- 172.21.15.86:0/435425883 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7875+0+0 (secure 0 0 0) 0x7f774012ac00 con 0x7f77480d2420 2024-04-25T14:06:25.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.272+0000 7f774d000640 1 --2- 172.21.15.86:0/435425883 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f772806f670 0x7f7728071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.272+0000 7f774d000640 1 --2- 172.21.15.86:0/435425883 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f772806f670 0x7f7728071b80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f773c0097a0 tx=0x7f773c008040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:25.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.276+0000 7f7744c00640 1 -- 172.21.15.86:0/435425883 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f77400c7020 con 0x7f77480d2420 2024-04-25T14:06:25.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.432+0000 7f774fa00640 1 -- 172.21.15.86:0/435425883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77480ce2f0 con 0x7f77480d2420 2024-04-25T14:06:25.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.436+0000 7f7744c00640 1 -- 172.21.15.86:0/435425883 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+22031 (secure 0 0 0) 0x7f77400fde20 con 0x7f77480d2420 2024-04-25T14:06:25.437 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:25.437 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":59,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:06:19.409466+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"fs1_metadata","create_time":"2024-04-25T14:05:30.939601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"fs1"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"fs1_data","create_time":"2024-04-25T14:05:31.898262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"fs1"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"fs2_metadata","create_time":"2024-04-25T14:05:52.841764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"fs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"fs2_data","create_time":"2024-04-25T14:05:53.524454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"fs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/145108428":"2024-04-26T14:06:19.409429+0000","172.21.15.86:6833/2427179681":"2024-04-26T14:06:19.409429+0000","172.21.15.86:6832/2427179681":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/1321309855":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/525010030":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/3470358079":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/1177884430":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/3644218120":"2024-04-26T14:06:19.409429+0000","172.21.15.86:6836/1732615622":"2024-04-26T14:06:00.230367+0000","172.21.15.86:6837/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:6837/2418778265":"2024-04-26T14:06:00.222915+0000","172.21.15.86:0/1843278167":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6832/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/2073988773":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6835/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:0/2602032945":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/3558294881":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6835/615772001":"2024-04-26T14:05:42.882867+0000","172.21.15.86:6837/615772001":"2024-04-26T14:05:42.882867+0000","172.21.15.86:6833/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6835/2418778265":"2024-04-26T14:06:00.222915+0000","172.21.15.86:6834/1732615622":"2024-04-26T14:06:00.230367+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:06:25.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.436+0000 7f774fa00640 1 -- 172.21.15.86:0/435425883 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f772806f670 msgr2=0x7f7728071b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.436+0000 7f774fa00640 1 --2- 172.21.15.86:0/435425883 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f772806f670 0x7f7728071b80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f773c0097a0 tx=0x7f773c008040 comp rx=0 tx=0).stop 2024-04-25T14:06:25.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.436+0000 7f774fa00640 1 -- 172.21.15.86:0/435425883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77480d2420 msgr2=0x7f7748140ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.436+0000 7f774fa00640 1 --2- 172.21.15.86:0/435425883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77480d2420 0x7f7748140ae0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f77400af040 tx=0x7f7740019750 comp rx=0 tx=0).stop 2024-04-25T14:06:25.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.436+0000 7f774fa00640 1 -- 172.21.15.86:0/435425883 shutdown_connections 2024-04-25T14:06:25.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.436+0000 7f774fa00640 1 --2- 172.21.15.86:0/435425883 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f772806f670 0x7f7728071b80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.436+0000 7f774fa00640 1 --2- 172.21.15.86:0/435425883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77480d2420 0x7f7748140ae0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.436+0000 7f774fa00640 1 --2- 172.21.15.86:0/435425883 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77480b6490 0x7f77481405a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.436+0000 7f774fa00640 1 --2- 172.21.15.86:0/435425883 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7748070e20 0x7f774815eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.436+0000 7f774fa00640 1 -- 172.21.15.86:0/435425883 >> 172.21.15.86:0/435425883 conn(0x7f77480cffc0 msgr2=0x7f77480b7430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:25.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.436+0000 7f774fa00640 1 -- 172.21.15.86:0/435425883 shutdown_connections 2024-04-25T14:06:25.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.436+0000 7f774fa00640 1 -- 172.21.15.86:0/435425883 wait complete. 2024-04-25T14:06:25.461 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail fs1 2024-04-25T14:06:25.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.608+0000 7f05bc600640 1 Processor -- start 2024-04-25T14:06:25.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.608+0000 7f05bc600640 1 -- start start 2024-04-25T14:06:25.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.608+0000 7f05bc600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f05b40cfca0 0x7f05b406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.608+0000 7f05bc600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f05b40d16e0 0x7f05b40d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.608+0000 7f05bc600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05b406c670 0x7f05b406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.608+0000 7f05bc600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f05b4070470 con 0x7f05b40d16e0 2024-04-25T14:06:25.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.608+0000 7f05bc600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f05b40d2030 con 0x7f05b40cfca0 2024-04-25T14:06:25.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.608+0000 7f05bc600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f05b4072bd0 con 0x7f05b406c670 2024-04-25T14:06:25.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.608+0000 7f05b9200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f05b40d16e0 0x7f05b40d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.608+0000 7f05b9200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f05b40d16e0 0x7f05b40d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55472/0 (socket says 172.21.15.86:55472) 2024-04-25T14:06:25.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.608+0000 7f05b9200640 1 -- 172.21.15.86:0/1803574868 learned_addr learned my addr 172.21.15.86:0/1803574868 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:25.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.608+0000 7f05ba600640 1 --2- 172.21.15.86:0/1803574868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05b406c670 0x7f05b406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.608+0000 7f05b9c00640 1 --2- 172.21.15.86:0/1803574868 >> v2:172.21.15.155:3300/0 conn(0x7f05b40cfca0 0x7f05b406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.608+0000 7f05b9200640 1 -- 172.21.15.86:0/1803574868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05b406c670 msgr2=0x7f05b406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:25.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.608+0000 7f05b9200640 1 --2- 172.21.15.86:0/1803574868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05b406c670 0x7f05b406ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.608+0000 7f05b9200640 1 -- 172.21.15.86:0/1803574868 >> v2:172.21.15.155:3300/0 conn(0x7f05b40cfca0 msgr2=0x7f05b406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:25.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.608+0000 7f05b9200640 1 --2- 172.21.15.86:0/1803574868 >> v2:172.21.15.155:3300/0 conn(0x7f05b40cfca0 0x7f05b406f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.608+0000 7f05b9200640 1 -- 172.21.15.86:0/1803574868 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b406f930 con 0x7f05b40d16e0 2024-04-25T14:06:25.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.612+0000 7f05b9200640 1 --2- 172.21.15.86:0/1803574868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b40d16e0 0x7f05b40d1af0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f05b40d22a0 tx=0x7f05a001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d53d6fba162380 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:25.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.612+0000 7f05b3e00640 1 -- 172.21.15.86:0/1803574868 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a00b0070 con 0x7f05b40d16e0 2024-04-25T14:06:25.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.612+0000 7f05b3e00640 1 -- 172.21.15.86:0/1803574868 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05a00ab600 con 0x7f05b40d16e0 2024-04-25T14:06:25.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.612+0000 7f05b3e00640 1 -- 172.21.15.86:0/1803574868 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a00ab9d0 con 0x7f05b40d16e0 2024-04-25T14:06:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.612+0000 7f05bc600640 1 -- 172.21.15.86:0/1803574868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b40d16e0 msgr2=0x7f05b40d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.612+0000 7f05bc600640 1 --2- 172.21.15.86:0/1803574868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b40d16e0 0x7f05b40d1af0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f05b40d22a0 tx=0x7f05a001c520 comp rx=0 tx=0).stop 2024-04-25T14:06:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.612+0000 7f05bc600640 1 -- 172.21.15.86:0/1803574868 shutdown_connections 2024-04-25T14:06:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.612+0000 7f05bc600640 1 --2- 172.21.15.86:0/1803574868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05b406c670 0x7f05b406ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.612+0000 7f05bc600640 1 --2- 172.21.15.86:0/1803574868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b40d16e0 0x7f05b40d1af0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.612+0000 7f05bc600640 1 --2- 172.21.15.86:0/1803574868 >> v2:172.21.15.155:3300/0 conn(0x7f05b40cfca0 0x7f05b406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.612+0000 7f05bc600640 1 -- 172.21.15.86:0/1803574868 >> 172.21.15.86:0/1803574868 conn(0x7f05b40b4650 msgr2=0x7f05b40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.612+0000 7f05bc600640 1 -- 172.21.15.86:0/1803574868 shutdown_connections 2024-04-25T14:06:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.612+0000 7f05bc600640 1 -- 172.21.15.86:0/1803574868 wait complete. 2024-04-25T14:06:25.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.616+0000 7f05bc600640 1 Processor -- start 2024-04-25T14:06:25.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.616+0000 7f05bc600640 1 -- start start 2024-04-25T14:06:25.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.616+0000 7f05bc600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05b406c670 0x7f05b41578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.616+0000 7f05bc600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05b40cfca0 0x7f05b409b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.616+0000 7f05bc600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b40d16e0 0x7f05b409b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.616+0000 7f05bc600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05b40ce570 con 0x7f05b40d16e0 2024-04-25T14:06:25.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.616+0000 7f05bc600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f05b406d3d0 con 0x7f05b406c670 2024-04-25T14:06:25.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.616+0000 7f05bc600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f05b40ce6f0 con 0x7f05b40cfca0 2024-04-25T14:06:25.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.616+0000 7f05ba600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b40d16e0 0x7f05b409b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.616+0000 7f05ba600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b40d16e0 0x7f05b409b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55482/0 (socket says 172.21.15.86:55482) 2024-04-25T14:06:25.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.616+0000 7f05ba600640 1 -- 172.21.15.86:0/4077790568 learned_addr learned my addr 172.21.15.86:0/4077790568 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:25.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.616+0000 7f05b9200640 1 --2- 172.21.15.86:0/4077790568 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05b40cfca0 0x7f05b409b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.616+0000 7f05b9c00640 1 --2- 172.21.15.86:0/4077790568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05b406c670 0x7f05b41578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.620+0000 7f05ba600640 1 -- 172.21.15.86:0/4077790568 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05b40cfca0 msgr2=0x7f05b409b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.620+0000 7f05ba600640 1 --2- 172.21.15.86:0/4077790568 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05b40cfca0 0x7f05b409b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.620+0000 7f05ba600640 1 -- 172.21.15.86:0/4077790568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05b406c670 msgr2=0x7f05b41578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.620+0000 7f05ba600640 1 --2- 172.21.15.86:0/4077790568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05b406c670 0x7f05b41578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.620+0000 7f05ba600640 1 -- 172.21.15.86:0/4077790568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b409eb60 con 0x7f05b40d16e0 2024-04-25T14:06:25.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.620+0000 7f05b9200640 1 --2- 172.21.15.86:0/4077790568 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05b40cfca0 0x7f05b409b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:25.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.620+0000 7f05ba600640 1 --2- 172.21.15.86:0/4077790568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b40d16e0 0x7f05b409b7f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f05ac009cb0 tx=0x7f05ac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:25.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.620+0000 7f05b1600640 1 -- 172.21.15.86:0/4077790568 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac018020 con 0x7f05b40d16e0 2024-04-25T14:06:25.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.620+0000 7f05b1600640 1 -- 172.21.15.86:0/4077790568 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05ac014480 con 0x7f05b40d16e0 2024-04-25T14:06:25.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.620+0000 7f05b1600640 1 -- 172.21.15.86:0/4077790568 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac014870 con 0x7f05b40d16e0 2024-04-25T14:06:25.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.620+0000 7f05bc600640 1 -- 172.21.15.86:0/4077790568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b409f060 con 0x7f05b40d16e0 2024-04-25T14:06:25.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.620+0000 7f05bc600640 1 -- 172.21.15.86:0/4077790568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b415bdb0 con 0x7f05b40d16e0 2024-04-25T14:06:25.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.620+0000 7f05bc600640 1 -- 172.21.15.86:0/4077790568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05b409f4b0 con 0x7f05b40d16e0 2024-04-25T14:06:25.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.620+0000 7f05b1600640 1 -- 172.21.15.86:0/4077790568 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f05ac01b020 con 0x7f05b40d16e0 2024-04-25T14:06:25.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.620+0000 7f05b1600640 1 --2- 172.21.15.86:0/4077790568 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f059406f670 0x7f0594071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.620+0000 7f05b1600640 1 -- 172.21.15.86:0/4077790568 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 7875+0+0 (secure 0 0 0) 0x7f05ac090e10 con 0x7f05b40d16e0 2024-04-25T14:06:25.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.620+0000 7f05b9c00640 1 --2- 172.21.15.86:0/4077790568 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f059406f670 0x7f0594071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.624+0000 7f05b9c00640 1 --2- 172.21.15.86:0/4077790568 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f059406f670 0x7f0594071b80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f05b4070710 tx=0x7f05a8009190 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:25.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.628+0000 7f05b1600640 1 -- 172.21.15.86:0/4077790568 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f05ac094030 con 0x7f05b40d16e0 2024-04-25T14:06:25.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.816+0000 7f05bc600640 1 -- 172.21.15.86:0/4077790568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "fs1"} v 0) -- 0x7f05b40cb6a0 con 0x7f05b40d16e0 2024-04-25T14:06:25.819 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-25T14:06:25.816+0000 7fea29c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:06:25.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.820+0000 7f05b1600640 1 -- 172.21.15.86:0/4077790568 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "fs1"}]=0 fs1 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v44) ==== 155+0+0 (secure 0 0 0) 0x7f05ac022020 con 0x7f05b40d16e0 2024-04-25T14:06:25.821 INFO:teuthology.orchestra.run.smithi086.stderr:fs1 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T14:06:25.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.824+0000 7f05bc600640 1 -- 172.21.15.86:0/4077790568 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f059406f670 msgr2=0x7f0594071b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.824+0000 7f05bc600640 1 --2- 172.21.15.86:0/4077790568 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f059406f670 0x7f0594071b80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f05b4070710 tx=0x7f05a8009190 comp rx=0 tx=0).stop 2024-04-25T14:06:25.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.824+0000 7f05bc600640 1 -- 172.21.15.86:0/4077790568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b40d16e0 msgr2=0x7f05b409b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.824+0000 7f05bc600640 1 --2- 172.21.15.86:0/4077790568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b40d16e0 0x7f05b409b7f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f05ac009cb0 tx=0x7f05ac00c490 comp rx=0 tx=0).stop 2024-04-25T14:06:25.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.824+0000 7f05bc600640 1 -- 172.21.15.86:0/4077790568 shutdown_connections 2024-04-25T14:06:25.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.824+0000 7f05bc600640 1 --2- 172.21.15.86:0/4077790568 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f059406f670 0x7f0594071b80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.824+0000 7f05bc600640 1 --2- 172.21.15.86:0/4077790568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05b40d16e0 0x7f05b409b7f0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.824+0000 7f05bc600640 1 --2- 172.21.15.86:0/4077790568 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05b40cfca0 0x7f05b409b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.824+0000 7f05bc600640 1 --2- 172.21.15.86:0/4077790568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05b406c670 0x7f05b41578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.824+0000 7f05bc600640 1 -- 172.21.15.86:0/4077790568 >> 172.21.15.86:0/4077790568 conn(0x7f05b40b4650 msgr2=0x7f05b40b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:25.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.824+0000 7f05bc600640 1 -- 172.21.15.86:0/4077790568 shutdown_connections 2024-04-25T14:06:25.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:25.824+0000 7f05bc600640 1 -- 172.21.15.86:0/4077790568 wait complete. 2024-04-25T14:06:25.841 INFO:tasks.ceph.mds.c.smithi086.stderr: -1759> 2024-04-25T14:06:05.232+0000 7f63a1a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T14:06:25.854 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm fs1 --yes-i-really-mean-it 2024-04-25T14:06:25.962 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-04-25T14:06:26.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.008+0000 7f3f97e00640 1 Processor -- start 2024-04-25T14:06:26.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.008+0000 7f3f97e00640 1 -- start start 2024-04-25T14:06:26.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.012+0000 7f3f97e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3f9015faa0 0x7f3f9006d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.012+0000 7f3f97e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3f900b59e0 0x7f3f900b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.012+0000 7f3f97e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f900aeea0 0x7f3f900af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.012+0000 7f3f97e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3f9006d560 con 0x7f3f9015faa0 2024-04-25T14:06:26.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.012+0000 7f3f97e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3f900b6330 con 0x7f3f900b59e0 2024-04-25T14:06:26.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.012+0000 7f3f97e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3f9006c670 con 0x7f3f900aeea0 2024-04-25T14:06:26.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.012+0000 7f3f95400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3f9015faa0 0x7f3f9006d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.012+0000 7f3f95400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3f9015faa0 0x7f3f9006d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55520/0 (socket says 172.21.15.86:55520) 2024-04-25T14:06:26.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.012+0000 7f3f95400640 1 -- 172.21.15.86:0/1521666128 learned_addr learned my addr 172.21.15.86:0/1521666128 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:26.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.012+0000 7f3f95e00640 1 --2- 172.21.15.86:0/1521666128 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f900aeea0 0x7f3f900af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.012+0000 7f3f94a00640 1 --2- 172.21.15.86:0/1521666128 >> v2:172.21.15.155:3300/0 conn(0x7f3f900b59e0 0x7f3f900b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.012+0000 7f3f95400640 1 -- 172.21.15.86:0/1521666128 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f900aeea0 msgr2=0x7f3f900af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:26.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.012+0000 7f3f95400640 1 --2- 172.21.15.86:0/1521666128 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f900aeea0 0x7f3f900af2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.012+0000 7f3f95400640 1 -- 172.21.15.86:0/1521666128 >> v2:172.21.15.155:3300/0 conn(0x7f3f900b59e0 msgr2=0x7f3f900b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:26.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.012+0000 7f3f95400640 1 --2- 172.21.15.86:0/1521666128 >> v2:172.21.15.155:3300/0 conn(0x7f3f900b59e0 0x7f3f900b5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.012+0000 7f3f95400640 1 -- 172.21.15.86:0/1521666128 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f9006c950 con 0x7f3f9015faa0 2024-04-25T14:06:26.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.012+0000 7f3f95400640 1 --2- 172.21.15.86:0/1521666128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f9015faa0 0x7f3f9006d9f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3f900b7250 tx=0x7f3f8401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c276300ab4184b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:26.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.016+0000 7f3f8fe00640 1 -- 172.21.15.86:0/1521666128 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f840a6040 con 0x7f3f9015faa0 2024-04-25T14:06:26.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.016+0000 7f3f8fe00640 1 -- 172.21.15.86:0/1521666128 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f840af070 con 0x7f3f9015faa0 2024-04-25T14:06:26.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.016+0000 7f3f8fe00640 1 -- 172.21.15.86:0/1521666128 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f840aa860 con 0x7f3f9015faa0 2024-04-25T14:06:26.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.016+0000 7f3f97e00640 1 -- 172.21.15.86:0/1521666128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f9015faa0 msgr2=0x7f3f9006d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:26.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.016+0000 7f3f97e00640 1 --2- 172.21.15.86:0/1521666128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f9015faa0 0x7f3f9006d9f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3f900b7250 tx=0x7f3f8401c520 comp rx=0 tx=0).stop 2024-04-25T14:06:26.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.016+0000 7f3f97e00640 1 -- 172.21.15.86:0/1521666128 shutdown_connections 2024-04-25T14:06:26.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.016+0000 7f3f97e00640 1 --2- 172.21.15.86:0/1521666128 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f900aeea0 0x7f3f900af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.016+0000 7f3f97e00640 1 --2- 172.21.15.86:0/1521666128 >> v2:172.21.15.155:3300/0 conn(0x7f3f900b59e0 0x7f3f900b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.016+0000 7f3f97e00640 1 --2- 172.21.15.86:0/1521666128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f9015faa0 0x7f3f9006d9f0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.016+0000 7f3f97e00640 1 -- 172.21.15.86:0/1521666128 >> 172.21.15.86:0/1521666128 conn(0x7f3f900d2230 msgr2=0x7f3f900d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:26.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.016+0000 7f3f97e00640 1 -- 172.21.15.86:0/1521666128 shutdown_connections 2024-04-25T14:06:26.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.016+0000 7f3f97e00640 1 -- 172.21.15.86:0/1521666128 wait complete. 2024-04-25T14:06:26.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.016+0000 7f3f97e00640 1 Processor -- start 2024-04-25T14:06:26.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.016+0000 7f3f97e00640 1 -- start start 2024-04-25T14:06:26.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.020+0000 7f3f97e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f900aeea0 0x7f3f90161940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.020+0000 7f3f97e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f900b59e0 0x7f3f901507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.020+0000 7f3f97e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f9015faa0 0x7f3f901565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.020+0000 7f3f97e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9015feb0 con 0x7f3f900aeea0 2024-04-25T14:06:26.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.020+0000 7f3f97e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9006df30 con 0x7f3f900b59e0 2024-04-25T14:06:26.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.020+0000 7f3f97e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3f90160030 con 0x7f3f9015faa0 2024-04-25T14:06:26.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.020+0000 7f3f95400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f900aeea0 0x7f3f90161940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.020+0000 7f3f95400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f900aeea0 0x7f3f90161940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55524/0 (socket says 172.21.15.86:55524) 2024-04-25T14:06:26.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.020+0000 7f3f95400640 1 -- 172.21.15.86:0/1032622556 learned_addr learned my addr 172.21.15.86:0/1032622556 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:26.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.020+0000 7f3f94a00640 1 --2- 172.21.15.86:0/1032622556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f900b59e0 0x7f3f901507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.020+0000 7f3f95e00640 1 --2- 172.21.15.86:0/1032622556 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f9015faa0 0x7f3f901565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.020+0000 7f3f95400640 1 -- 172.21.15.86:0/1032622556 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f9015faa0 msgr2=0x7f3f901565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:26.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.020+0000 7f3f95400640 1 --2- 172.21.15.86:0/1032622556 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f9015faa0 0x7f3f901565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.020+0000 7f3f95400640 1 -- 172.21.15.86:0/1032622556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f900b59e0 msgr2=0x7f3f901507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:26.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.020+0000 7f3f95400640 1 --2- 172.21.15.86:0/1032622556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f900b59e0 0x7f3f901507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.020+0000 7f3f95400640 1 -- 172.21.15.86:0/1032622556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f90162190 con 0x7f3f900aeea0 2024-04-25T14:06:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.020+0000 7f3f95e00640 1 --2- 172.21.15.86:0/1032622556 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f9015faa0 0x7f3f901565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.020+0000 7f3f94a00640 1 --2- 172.21.15.86:0/1032622556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f900b59e0 0x7f3f901507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.020+0000 7f3f95400640 1 --2- 172.21.15.86:0/1032622556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f900aeea0 0x7f3f90161940 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f3f8401cdc0 tx=0x7f3f8401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.020+0000 7f3f8d600640 1 -- 172.21.15.86:0/1032622556 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f840bd3f0 con 0x7f3f900aeea0 2024-04-25T14:06:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.020+0000 7f3f8d600640 1 -- 172.21.15.86:0/1032622556 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f840bd590 con 0x7f3f900aeea0 2024-04-25T14:06:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.020+0000 7f3f8d600640 1 -- 172.21.15.86:0/1032622556 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f840bd940 con 0x7f3f900aeea0 2024-04-25T14:06:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.020+0000 7f3f97e00640 1 -- 172.21.15.86:0/1032622556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f90162470 con 0x7f3f900aeea0 2024-04-25T14:06:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.020+0000 7f3f97e00640 1 -- 172.21.15.86:0/1032622556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f90155190 con 0x7f3f900aeea0 2024-04-25T14:06:26.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.020+0000 7f3f97e00640 1 -- 172.21.15.86:0/1032622556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f90162670 con 0x7f3f900aeea0 2024-04-25T14:06:26.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.024+0000 7f3f8d600640 1 -- 172.21.15.86:0/1032622556 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f3f840b33f0 con 0x7f3f900aeea0 2024-04-25T14:06:26.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.024+0000 7f3f8d600640 1 --2- 172.21.15.86:0/1032622556 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3f7006f640 0x7f3f70071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.024+0000 7f3f8d600640 1 -- 172.21.15.86:0/1032622556 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 7961+0+0 (secure 0 0 0) 0x7f3f84129ad0 con 0x7f3f900aeea0 2024-04-25T14:06:26.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.024+0000 7f3f94a00640 1 --2- 172.21.15.86:0/1032622556 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3f7006f640 0x7f3f70071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.024+0000 7f3f94a00640 1 --2- 172.21.15.86:0/1032622556 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3f7006f640 0x7f3f70071b50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3f90068810 tx=0x7f3f7c008040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:26.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.028+0000 7f3f8d600640 1 -- 172.21.15.86:0/1032622556 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3f90162670 con 0x7f3f900aeea0 2024-04-25T14:06:26.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.220+0000 7f3f97e00640 1 -- 172.21.15.86:0/1032622556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "fs1", "yes_i_really_mean_it": true} v 0) -- 0x7f3f90074cf0 con 0x7f3f900aeea0 2024-04-25T14:06:26.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.824+0000 7f3f8d600640 1 -- 172.21.15.86:0/1032622556 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "fs1", "yes_i_really_mean_it": true}]=0 v45) ==== 101+0+0 (secure 0 0 0) 0x7f3f84016110 con 0x7f3f900aeea0 2024-04-25T14:06:26.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.824+0000 7f3f97e00640 1 -- 172.21.15.86:0/1032622556 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3f7006f640 msgr2=0x7f3f70071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:26.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.824+0000 7f3f97e00640 1 --2- 172.21.15.86:0/1032622556 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3f7006f640 0x7f3f70071b50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3f90068810 tx=0x7f3f7c008040 comp rx=0 tx=0).stop 2024-04-25T14:06:26.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.824+0000 7f3f97e00640 1 -- 172.21.15.86:0/1032622556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f900aeea0 msgr2=0x7f3f90161940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:26.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.824+0000 7f3f97e00640 1 --2- 172.21.15.86:0/1032622556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f900aeea0 0x7f3f90161940 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f3f8401cdc0 tx=0x7f3f8401ad60 comp rx=0 tx=0).stop 2024-04-25T14:06:26.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.824+0000 7f3f97e00640 1 -- 172.21.15.86:0/1032622556 shutdown_connections 2024-04-25T14:06:26.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.824+0000 7f3f97e00640 1 --2- 172.21.15.86:0/1032622556 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3f7006f640 0x7f3f70071b50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.824+0000 7f3f97e00640 1 --2- 172.21.15.86:0/1032622556 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f9015faa0 0x7f3f901565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.824+0000 7f3f97e00640 1 --2- 172.21.15.86:0/1032622556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f900b59e0 0x7f3f901507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.824+0000 7f3f97e00640 1 --2- 172.21.15.86:0/1032622556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f900aeea0 0x7f3f90161940 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.824+0000 7f3f97e00640 1 -- 172.21.15.86:0/1032622556 >> 172.21.15.86:0/1032622556 conn(0x7f3f900d2230 msgr2=0x7f3f900704c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:26.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.824+0000 7f3f97e00640 1 -- 172.21.15.86:0/1032622556 shutdown_connections 2024-04-25T14:06:26.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:26.824+0000 7f3f97e00640 1 -- 172.21.15.86:0/1032622556 wait complete. 2024-04-25T14:06:26.850 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm fs1_metadata fs1_metadata --yes-i-really-really-mean-it 2024-04-25T14:06:27.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.000+0000 7f1ba6a00640 1 Processor -- start 2024-04-25T14:06:27.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.000+0000 7f1ba6a00640 1 -- start start 2024-04-25T14:06:27.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.000+0000 7f1ba6a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1ba00b0540 0x7f1ba00ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.000+0000 7f1ba6a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1ba015e270 0x7f1ba015e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.000+0000 7f1ba6a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ba00b0b20 con 0x7f1ba015ebc0 2024-04-25T14:06:27.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.000+0000 7f1ba6a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1ba006d870 con 0x7f1ba00b0540 2024-04-25T14:06:27.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.000+0000 7f1ba6a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1ba006d220 con 0x7f1ba015e270 2024-04-25T14:06:27.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.000+0000 7f1b9fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1ba00b0540 0x7f1ba00ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.000+0000 7f1b9fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1ba00b0540 0x7f1ba00ae2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55530/0 (socket says 172.21.15.86:55530) 2024-04-25T14:06:27.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.000+0000 7f1b9fe00640 1 -- 172.21.15.86:0/692298399 learned_addr learned my addr 172.21.15.86:0/692298399 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:27.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.004+0000 7f1b97400640 1 --2- 172.21.15.86:0/692298399 >> v2:172.21.15.155:3300/0 conn(0x7f1ba015e270 0x7f1ba015e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.004+0000 7f1b9fe00640 1 -- 172.21.15.86:0/692298399 >> v2:172.21.15.155:3300/0 conn(0x7f1ba015e270 msgr2=0x7f1ba015e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:27.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.004+0000 7f1b9fe00640 1 --2- 172.21.15.86:0/692298399 >> v2:172.21.15.155:3300/0 conn(0x7f1ba015e270 0x7f1ba015e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.004+0000 7f1b9fe00640 1 -- 172.21.15.86:0/692298399 >> v1:172.21.15.155:6789/0 conn(0x7f1ba015ebc0 legacy=0x7f1ba00733c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:27.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.004+0000 7f1b9fe00640 1 -- 172.21.15.86:0/692298399 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba006d500 con 0x7f1ba00b0540 2024-04-25T14:06:27.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.004+0000 7f1b9fe00640 1 --2- 172.21.15.86:0/692298399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba00b0540 0x7f1ba00ae2a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00afcf0 tx=0x7f1b9001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b739aecb7ca571 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:27.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.004+0000 7f1b9f400640 1 -- 172.21.15.86:0/692298399 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b900a5520 con 0x7f1ba00b0540 2024-04-25T14:06:27.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.004+0000 7f1b9f400640 1 -- 172.21.15.86:0/692298399 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b900a56c0 con 0x7f1ba00b0540 2024-04-25T14:06:27.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.004+0000 7f1b9f400640 1 -- 172.21.15.86:0/692298399 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b900a5a90 con 0x7f1ba00b0540 2024-04-25T14:06:27.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.008+0000 7f1ba6a00640 1 -- 172.21.15.86:0/692298399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba00b0540 msgr2=0x7f1ba00ae2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:27.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.008+0000 7f1ba6a00640 1 --2- 172.21.15.86:0/692298399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba00b0540 0x7f1ba00ae2a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00afcf0 tx=0x7f1b9001c960 comp rx=0 tx=0).stop 2024-04-25T14:06:27.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.008+0000 7f1ba6a00640 1 -- 172.21.15.86:0/692298399 shutdown_connections 2024-04-25T14:06:27.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.008+0000 7f1ba6a00640 1 --2- 172.21.15.86:0/692298399 >> v2:172.21.15.155:3300/0 conn(0x7f1ba015e270 0x7f1ba015e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.008+0000 7f1ba6a00640 1 --2- 172.21.15.86:0/692298399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba00b0540 0x7f1ba00ae2a0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.008+0000 7f1ba6a00640 1 -- 172.21.15.86:0/692298399 >> 172.21.15.86:0/692298399 conn(0x7f1ba00b4a40 msgr2=0x7f1ba00b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:27.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.008+0000 7f1ba6a00640 1 -- 172.21.15.86:0/692298399 shutdown_connections 2024-04-25T14:06:27.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.008+0000 7f1ba6a00640 1 -- 172.21.15.86:0/692298399 wait complete. 2024-04-25T14:06:27.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.008+0000 7f1ba6a00640 1 Processor -- start 2024-04-25T14:06:27.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.008+0000 7f1ba6a00640 1 -- start start 2024-04-25T14:06:27.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.012+0000 7f1ba6a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ba00b0540 0x7f1ba01565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.012+0000 7f1ba6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba015e270 0x7f1ba014c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.012+0000 7f1ba6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ba015ebc0 0x7f1ba01f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.012+0000 7f1ba6a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba016b220 con 0x7f1ba015e270 2024-04-25T14:06:27.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.012+0000 7f1ba6a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba016b0a0 con 0x7f1ba015ebc0 2024-04-25T14:06:27.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.012+0000 7f1ba6a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba016b3a0 con 0x7f1ba00b0540 2024-04-25T14:06:27.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.012+0000 7f1b97400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba015e270 0x7f1ba014c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.012+0000 7f1b97400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba015e270 0x7f1ba014c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55532/0 (socket says 172.21.15.86:55532) 2024-04-25T14:06:27.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.012+0000 7f1b97400640 1 -- 172.21.15.86:0/2214168975 learned_addr learned my addr 172.21.15.86:0/2214168975 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:27.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.012+0000 7f1ba4a00640 1 --2- 172.21.15.86:0/2214168975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ba015ebc0 0x7f1ba01f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.012+0000 7f1b9fe00640 1 --2- 172.21.15.86:0/2214168975 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ba00b0540 0x7f1ba01565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.012+0000 7f1b97400640 1 -- 172.21.15.86:0/2214168975 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ba00b0540 msgr2=0x7f1ba01565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:27.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.012+0000 7f1b97400640 1 --2- 172.21.15.86:0/2214168975 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ba00b0540 0x7f1ba01565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.012+0000 7f1b97400640 1 -- 172.21.15.86:0/2214168975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ba015ebc0 msgr2=0x7f1ba01f9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:27.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.012+0000 7f1b97400640 1 --2- 172.21.15.86:0/2214168975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ba015ebc0 0x7f1ba01f9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.012+0000 7f1b97400640 1 -- 172.21.15.86:0/2214168975 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba0155810 con 0x7f1ba015e270 2024-04-25T14:06:27.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.012+0000 7f1ba4a00640 1 --2- 172.21.15.86:0/2214168975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ba015ebc0 0x7f1ba01f9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:27.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.012+0000 7f1b97400640 1 --2- 172.21.15.86:0/2214168975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba015e270 0x7f1ba014c300 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800d620 tx=0x7f1b8800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:27.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.012+0000 7f1b9cc00640 1 -- 172.21.15.86:0/2214168975 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8800b620 con 0x7f1ba015e270 2024-04-25T14:06:27.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.012+0000 7f1b9cc00640 1 -- 172.21.15.86:0/2214168975 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b88019070 con 0x7f1ba015e270 2024-04-25T14:06:27.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.012+0000 7f1ba6a00640 1 -- 172.21.15.86:0/2214168975 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba01fa8b0 con 0x7f1ba015e270 2024-04-25T14:06:27.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.012+0000 7f1b9cc00640 1 -- 172.21.15.86:0/2214168975 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8800b9e0 con 0x7f1ba015e270 2024-04-25T14:06:27.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.012+0000 7f1ba6a00640 1 -- 172.21.15.86:0/2214168975 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba0155db0 con 0x7f1ba015e270 2024-04-25T14:06:27.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.012+0000 7f1ba6a00640 1 -- 172.21.15.86:0/2214168975 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba0050750 con 0x7f1ba015e270 2024-04-25T14:06:27.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.012+0000 7f1b9cc00640 1 -- 172.21.15.86:0/2214168975 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f1b8800bb80 con 0x7f1ba015e270 2024-04-25T14:06:27.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.016+0000 7f1b9cc00640 1 --2- 172.21.15.86:0/2214168975 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f1b7c06f640 0x7f1b7c071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.016+0000 7f1b9cc00640 1 -- 172.21.15.86:0/2214168975 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 7961+0+0 (secure 0 0 0) 0x7f1b88090400 con 0x7f1ba015e270 2024-04-25T14:06:27.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.016+0000 7f1b9fe00640 1 --2- 172.21.15.86:0/2214168975 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f1b7c06f640 0x7f1b7c071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.016+0000 7f1b9fe00640 1 --2- 172.21.15.86:0/2214168975 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f1b7c06f640 0x7f1b7c071b50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1b900ae000 tx=0x7f1b900a7070 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:27.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.020+0000 7f1b9cc00640 1 -- 172.21.15.86:0/2214168975 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1b8805c730 con 0x7f1ba015e270 2024-04-25T14:06:27.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.176+0000 7f1ba6a00640 1 -- 172.21.15.86:0/2214168975 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "fs1_metadata", "pool2": "fs1_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1ba01fa640 con 0x7f1ba015e270 2024-04-25T14:06:27.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.828+0000 7f1b9cc00640 1 -- 172.21.15.86:0/2214168975 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "fs1_metadata", "pool2": "fs1_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'fs1_metadata' removed v61) ==== 172+0+0 (secure 0 0 0) 0x7f1b8805c030 con 0x7f1ba015e270 2024-04-25T14:06:27.830 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'fs1_metadata' removed 2024-04-25T14:06:27.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.832+0000 7f1ba6a00640 1 -- 172.21.15.86:0/2214168975 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f1b7c06f640 msgr2=0x7f1b7c071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:27.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.832+0000 7f1ba6a00640 1 --2- 172.21.15.86:0/2214168975 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f1b7c06f640 0x7f1b7c071b50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1b900ae000 tx=0x7f1b900a7070 comp rx=0 tx=0).stop 2024-04-25T14:06:27.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.832+0000 7f1ba6a00640 1 -- 172.21.15.86:0/2214168975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba015e270 msgr2=0x7f1ba014c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:27.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.832+0000 7f1ba6a00640 1 --2- 172.21.15.86:0/2214168975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba015e270 0x7f1ba014c300 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800d620 tx=0x7f1b8800dae0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.832+0000 7f1ba6a00640 1 -- 172.21.15.86:0/2214168975 shutdown_connections 2024-04-25T14:06:27.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.832+0000 7f1ba6a00640 1 --2- 172.21.15.86:0/2214168975 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f1b7c06f640 0x7f1b7c071b50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.832+0000 7f1ba6a00640 1 --2- 172.21.15.86:0/2214168975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ba015ebc0 0x7f1ba01f9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.832+0000 7f1ba6a00640 1 --2- 172.21.15.86:0/2214168975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba015e270 0x7f1ba014c300 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.832+0000 7f1ba6a00640 1 --2- 172.21.15.86:0/2214168975 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ba00b0540 0x7f1ba01565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.832+0000 7f1ba6a00640 1 -- 172.21.15.86:0/2214168975 >> 172.21.15.86:0/2214168975 conn(0x7f1ba00b4a40 msgr2=0x7f1ba01536a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:27.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.836+0000 7f1ba6a00640 1 -- 172.21.15.86:0/2214168975 shutdown_connections 2024-04-25T14:06:27.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:27.836+0000 7f1ba6a00640 1 -- 172.21.15.86:0/2214168975 wait complete. 2024-04-25T14:06:27.860 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm fs1_data fs1_data --yes-i-really-really-mean-it 2024-04-25T14:06:28.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.012+0000 7f45f0800640 1 Processor -- start 2024-04-25T14:06:28.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.012+0000 7f45f0800640 1 -- start start 2024-04-25T14:06:28.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.012+0000 7f45f0800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f45e815ca50 0x7f45e8073950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:28.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.012+0000 7f45f0800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45e80743c0 0x7f45e81605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:28.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.012+0000 7f45f0800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45e815ce60 con 0x7f45e80747d0 2024-04-25T14:06:28.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.012+0000 7f45f0800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f45e8072b90 con 0x7f45e815ca50 2024-04-25T14:06:28.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.012+0000 7f45f0800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f45e8072e70 con 0x7f45e80743c0 2024-04-25T14:06:28.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.012+0000 7f45ede00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f45e815ca50 0x7f45e8073950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:28.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.012+0000 7f45ee800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f45e80747d0 0x7f45e806e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:32832/0 (socket says 172.21.15.86:32832) 2024-04-25T14:06:28.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.012+0000 7f45ee800640 1 -- 172.21.15.86:0/3902846079 learned_addr learned my addr 172.21.15.86:0/3902846079 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:28.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.012+0000 7f45ed400640 1 --2- 172.21.15.86:0/3902846079 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45e80743c0 0x7f45e81605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:28.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.012+0000 7f45ede00640 1 -- 172.21.15.86:0/3902846079 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45e80743c0 msgr2=0x7f45e81605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:28.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.012+0000 7f45ede00640 1 --2- 172.21.15.86:0/3902846079 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45e80743c0 0x7f45e81605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:28.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.012+0000 7f45ede00640 1 -- 172.21.15.86:0/3902846079 >> v1:172.21.15.86:6789/0 conn(0x7f45e80747d0 legacy=0x7f45e806e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:28.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.012+0000 7f45ede00640 1 -- 172.21.15.86:0/3902846079 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e8073150 con 0x7f45e815ca50 2024-04-25T14:06:28.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.012+0000 7f45eca00640 1 -- 172.21.15.86:0/3902846079 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1178725022 0 0) 0x7f45e815ce60 con 0x7f45e80747d0 2024-04-25T14:06:28.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.016+0000 7f45ede00640 1 --2- 172.21.15.86:0/3902846079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45e815ca50 0x7f45e8073950 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0097c0 tx=0x7f45dc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f025ce84c4f8f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:28.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.016+0000 7f45eca00640 1 -- 172.21.15.86:0/3902846079 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc0b0070 con 0x7f45e815ca50 2024-04-25T14:06:28.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.016+0000 7f45eca00640 1 -- 172.21.15.86:0/3902846079 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45dc0ab600 con 0x7f45e815ca50 2024-04-25T14:06:28.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.016+0000 7f45eca00640 1 -- 172.21.15.86:0/3902846079 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc0ab9d0 con 0x7f45e815ca50 2024-04-25T14:06:28.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.016+0000 7f45f0800640 1 -- 172.21.15.86:0/3902846079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45e815ca50 msgr2=0x7f45e8073950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:28.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.016+0000 7f45f0800640 1 --2- 172.21.15.86:0/3902846079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45e815ca50 0x7f45e8073950 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0097c0 tx=0x7f45dc01c4a0 comp rx=0 tx=0).stop 2024-04-25T14:06:28.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.016+0000 7f45f0800640 1 -- 172.21.15.86:0/3902846079 shutdown_connections 2024-04-25T14:06:28.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.016+0000 7f45f0800640 1 --2- 172.21.15.86:0/3902846079 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45e80743c0 0x7f45e81605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:28.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.016+0000 7f45f0800640 1 --2- 172.21.15.86:0/3902846079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45e815ca50 0x7f45e8073950 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:28.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.016+0000 7f45f0800640 1 -- 172.21.15.86:0/3902846079 >> 172.21.15.86:0/3902846079 conn(0x7f45e80b4a40 msgr2=0x7f45e80b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:28.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.016+0000 7f45f0800640 1 -- 172.21.15.86:0/3902846079 shutdown_connections 2024-04-25T14:06:28.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.016+0000 7f45f0800640 1 -- 172.21.15.86:0/3902846079 wait complete. 2024-04-25T14:06:28.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.020+0000 7f45f0800640 1 Processor -- start 2024-04-25T14:06:28.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.020+0000 7f45f0800640 1 -- start start 2024-04-25T14:06:28.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.020+0000 7f45f0800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45e80743c0 0x7f45e815fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:28.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.020+0000 7f45f0800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45e80747d0 0x7f45e816a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:28.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.020+0000 7f45f0800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45e815ca50 0x7f45e81552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:28.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.020+0000 7f45f0800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f45e8168520 con 0x7f45e80743c0 2024-04-25T14:06:28.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.020+0000 7f45f0800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f45e81683a0 con 0x7f45e815ca50 2024-04-25T14:06:28.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.020+0000 7f45f0800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f45e81686a0 con 0x7f45e80747d0 2024-04-25T14:06:28.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.020+0000 7f45ede00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45e80743c0 0x7f45e815fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:28.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.020+0000 7f45ede00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45e80743c0 0x7f45e815fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55548/0 (socket says 172.21.15.86:55548) 2024-04-25T14:06:28.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.020+0000 7f45ede00640 1 -- 172.21.15.86:0/2052080684 learned_addr learned my addr 172.21.15.86:0/2052080684 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:28.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.020+0000 7f45ede00640 1 -- 172.21.15.86:0/2052080684 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45e80747d0 msgr2=0x7f45e816a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:28.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.020+0000 7f45ed400640 1 --2- 172.21.15.86:0/2052080684 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45e80747d0 0x7f45e816a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:28.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.020+0000 7f45ee800640 1 --2- 172.21.15.86:0/2052080684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45e815ca50 0x7f45e81552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:28.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.020+0000 7f45ede00640 1 --2- 172.21.15.86:0/2052080684 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45e80747d0 0x7f45e816a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:28.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.020+0000 7f45ede00640 1 -- 172.21.15.86:0/2052080684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45e815ca50 msgr2=0x7f45e81552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.020+0000 7f45ede00640 1 --2- 172.21.15.86:0/2052080684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45e815ca50 0x7f45e81552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.020+0000 7f45ede00640 1 -- 172.21.15.86:0/2052080684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e8155820 con 0x7f45e80743c0 2024-04-25T14:06:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.020+0000 7f45ed400640 1 --2- 172.21.15.86:0/2052080684 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45e80747d0 0x7f45e816a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.020+0000 7f45ee800640 1 --2- 172.21.15.86:0/2052080684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45e815ca50 0x7f45e81552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.020+0000 7f45ede00640 1 --2- 172.21.15.86:0/2052080684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45e80743c0 0x7f45e815fb60 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0097c0 tx=0x7f45dc01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.024+0000 7f45e2000640 1 -- 172.21.15.86:0/2052080684 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc0b0050 con 0x7f45e80743c0 2024-04-25T14:06:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.024+0000 7f45e2000640 1 -- 172.21.15.86:0/2052080684 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45dc0bc070 con 0x7f45e80743c0 2024-04-25T14:06:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.024+0000 7f45f0800640 1 -- 172.21.15.86:0/2052080684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45e815e190 con 0x7f45e80743c0 2024-04-25T14:06:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.024+0000 7f45e2000640 1 -- 172.21.15.86:0/2052080684 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc0aac00 con 0x7f45e80743c0 2024-04-25T14:06:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.024+0000 7f45f0800640 1 -- 172.21.15.86:0/2052080684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45e8169a40 con 0x7f45e80743c0 2024-04-25T14:06:28.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.024+0000 7f45f0800640 1 -- 172.21.15.86:0/2052080684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45e8050750 con 0x7f45e80743c0 2024-04-25T14:06:28.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.024+0000 7f45e2000640 1 -- 172.21.15.86:0/2052080684 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f45dc0c1460 con 0x7f45e80743c0 2024-04-25T14:06:28.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.024+0000 7f45e2000640 1 --2- 172.21.15.86:0/2052080684 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f45cc06f670 0x7f45cc071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.024+0000 7f45e2000640 1 -- 172.21.15.86:0/2052080684 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 7529+0+0 (secure 0 0 0) 0x7f45dc12b490 con 0x7f45e80743c0 2024-04-25T14:06:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.024+0000 7f45ed400640 1 --2- 172.21.15.86:0/2052080684 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f45cc06f670 0x7f45cc071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.028+0000 7f45ed400640 1 --2- 172.21.15.86:0/2052080684 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f45cc06f670 0x7f45cc071b80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f45e8160660 tx=0x7f45d8008040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:28.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.028+0000 7f45e2000640 1 -- 172.21.15.86:0/2052080684 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f45dc0c6020 con 0x7f45e80743c0 2024-04-25T14:06:28.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.196+0000 7f45f0800640 1 -- 172.21.15.86:0/2052080684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "fs1_data", "pool2": "fs1_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f45e80b3d30 con 0x7f45e80743c0 2024-04-25T14:06:28.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.832+0000 7f45e2000640 1 -- 172.21.15.86:0/2052080684 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "fs1_data", "pool2": "fs1_data", "yes_i_really_really_mean_it": true}]=0 pool 'fs1_data' removed v62) ==== 160+0+0 (secure 0 0 0) 0x7f45e8160f40 con 0x7f45e80743c0 2024-04-25T14:06:28.832 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'fs1_data' removed 2024-04-25T14:06:28.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.836+0000 7f45f0800640 1 -- 172.21.15.86:0/2052080684 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f45cc06f670 msgr2=0x7f45cc071b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:28.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.836+0000 7f45f0800640 1 --2- 172.21.15.86:0/2052080684 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f45cc06f670 0x7f45cc071b80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f45e8160660 tx=0x7f45d8008040 comp rx=0 tx=0).stop 2024-04-25T14:06:28.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.836+0000 7f45f0800640 1 -- 172.21.15.86:0/2052080684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45e80743c0 msgr2=0x7f45e815fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:28.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.836+0000 7f45f0800640 1 --2- 172.21.15.86:0/2052080684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45e80743c0 0x7f45e815fb60 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0097c0 tx=0x7f45dc01ace0 comp rx=0 tx=0).stop 2024-04-25T14:06:28.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.836+0000 7f45f0800640 1 -- 172.21.15.86:0/2052080684 shutdown_connections 2024-04-25T14:06:28.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.836+0000 7f45f0800640 1 --2- 172.21.15.86:0/2052080684 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f45cc06f670 0x7f45cc071b80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:28.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.836+0000 7f45f0800640 1 --2- 172.21.15.86:0/2052080684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45e815ca50 0x7f45e81552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:28.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.836+0000 7f45f0800640 1 --2- 172.21.15.86:0/2052080684 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45e80747d0 0x7f45e816a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:28.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.836+0000 7f45f0800640 1 --2- 172.21.15.86:0/2052080684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45e80743c0 0x7f45e815fb60 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:28.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.836+0000 7f45f0800640 1 -- 172.21.15.86:0/2052080684 >> 172.21.15.86:0/2052080684 conn(0x7f45e80b4a40 msgr2=0x7f45e80af490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:28.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.836+0000 7f45f0800640 1 -- 172.21.15.86:0/2052080684 shutdown_connections 2024-04-25T14:06:28.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:28.836+0000 7f45f0800640 1 -- 172.21.15.86:0/2052080684 wait complete. 2024-04-25T14:06:28.860 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:06:29.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.004+0000 7f6f08400640 1 Processor -- start 2024-04-25T14:06:29.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.004+0000 7f6f08400640 1 -- start start 2024-04-25T14:06:29.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.008+0000 7f6f08400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6f00072630 0x7f6f00072a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:29.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.008+0000 7f6f08400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f00163980 0x7f6f00163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:29.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.008+0000 7f6f08400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f00072f80 con 0x7f6f0006e340 2024-04-25T14:06:29.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.008+0000 7f6f08400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6f001648e0 con 0x7f6f00072630 2024-04-25T14:06:29.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.008+0000 7f6f08400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6f000d1520 con 0x7f6f00163980 2024-04-25T14:06:29.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.008+0000 7f6f05a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6f00072630 0x7f6f00072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:29.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.008+0000 7f6f05a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6f00072630 0x7f6f00072a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49142/0 (socket says 172.21.15.86:49142) 2024-04-25T14:06:29.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.008+0000 7f6f05a00640 1 -- 172.21.15.86:0/3908598132 learned_addr learned my addr 172.21.15.86:0/3908598132 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:29.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.008+0000 7f6f05a00640 1 -- 172.21.15.86:0/3908598132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f00163980 msgr2=0x7f6f00163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:29.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.008+0000 7f6f05000640 1 --2- 172.21.15.86:0/3908598132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f00163980 0x7f6f00163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:29.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.008+0000 7f6f05a00640 1 --2- 172.21.15.86:0/3908598132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f00163980 0x7f6f00163d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.008+0000 7f6f05a00640 1 -- 172.21.15.86:0/3908598132 >> v1:172.21.15.155:6789/0 conn(0x7f6f0006e340 legacy=0x7f6f0006e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:29.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.008+0000 7f6f05a00640 1 -- 172.21.15.86:0/3908598132 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f000d23e0 con 0x7f6f00072630 2024-04-25T14:06:29.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.008+0000 7f6f05000640 1 --2- 172.21.15.86:0/3908598132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f00163980 0x7f6f00163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:29.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.008+0000 7f6f05a00640 1 --2- 172.21.15.86:0/3908598132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f00072630 0x7f6f00072a40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f6f000d25b0 tx=0x7f6ef401c200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b765a9f2895d00e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:29.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.012+0000 7f6effe00640 1 -- 172.21.15.86:0/3908598132 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef40b0070 con 0x7f6f00072630 2024-04-25T14:06:29.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.012+0000 7f6effe00640 1 -- 172.21.15.86:0/3908598132 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ef40ab600 con 0x7f6f00072630 2024-04-25T14:06:29.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.012+0000 7f6effe00640 1 -- 172.21.15.86:0/3908598132 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef40ab9d0 con 0x7f6f00072630 2024-04-25T14:06:29.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.012+0000 7f6f08400640 1 -- 172.21.15.86:0/3908598132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f00072630 msgr2=0x7f6f00072a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:29.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.012+0000 7f6f08400640 1 --2- 172.21.15.86:0/3908598132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f00072630 0x7f6f00072a40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f6f000d25b0 tx=0x7f6ef401c200 comp rx=0 tx=0).stop 2024-04-25T14:06:29.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.012+0000 7f6f08400640 1 -- 172.21.15.86:0/3908598132 shutdown_connections 2024-04-25T14:06:29.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.012+0000 7f6f08400640 1 --2- 172.21.15.86:0/3908598132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f00163980 0x7f6f00163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.012+0000 7f6f08400640 1 --2- 172.21.15.86:0/3908598132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f00072630 0x7f6f00072a40 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.012+0000 7f6f08400640 1 -- 172.21.15.86:0/3908598132 >> 172.21.15.86:0/3908598132 conn(0x7f6f000d3580 msgr2=0x7f6f000d3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:29.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.012+0000 7f6f08400640 1 -- 172.21.15.86:0/3908598132 shutdown_connections 2024-04-25T14:06:29.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.012+0000 7f6f08400640 1 -- 172.21.15.86:0/3908598132 wait complete. 2024-04-25T14:06:29.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.012+0000 7f6f08400640 1 Processor -- start 2024-04-25T14:06:29.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.012+0000 7f6f08400640 1 -- start start 2024-04-25T14:06:29.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.016+0000 7f6f08400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f0006e340 0x7f6f0015a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:29.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.016+0000 7f6f08400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f00072630 0x7f6f000c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:29.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.016+0000 7f6f08400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f00163980 0x7f6f000cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:29.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.016+0000 7f6f08400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f000cfe30 con 0x7f6f00163980 2024-04-25T14:06:29.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.016+0000 7f6f08400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6f000cfcb0 con 0x7f6f0006e340 2024-04-25T14:06:29.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.016+0000 7f6f08400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6f0016d530 con 0x7f6f00072630 2024-04-25T14:06:29.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.016+0000 7f6f06400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f00163980 0x7f6f000cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:29.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.016+0000 7f6f06400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f00163980 0x7f6f000cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49156/0 (socket says 172.21.15.86:49156) 2024-04-25T14:06:29.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.016+0000 7f6f06400640 1 -- 172.21.15.86:0/3803365036 learned_addr learned my addr 172.21.15.86:0/3803365036 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:29.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.016+0000 7f6f05000640 1 --2- 172.21.15.86:0/3803365036 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f00072630 0x7f6f000c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:29.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.016+0000 7f6f05a00640 1 --2- 172.21.15.86:0/3803365036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f0006e340 0x7f6f0015a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:29.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.016+0000 7f6f06400640 1 -- 172.21.15.86:0/3803365036 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f00072630 msgr2=0x7f6f000c75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:29.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.016+0000 7f6f06400640 1 --2- 172.21.15.86:0/3803365036 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f00072630 0x7f6f000c75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.016+0000 7f6f06400640 1 -- 172.21.15.86:0/3803365036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f0006e340 msgr2=0x7f6f0015a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:29.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.016+0000 7f6f06400640 1 --2- 172.21.15.86:0/3803365036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f0006e340 0x7f6f0015a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.016+0000 7f6f06400640 1 -- 172.21.15.86:0/3803365036 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f000c7f60 con 0x7f6f00163980 2024-04-25T14:06:29.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.016+0000 7f6f05a00640 1 --2- 172.21.15.86:0/3803365036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f0006e340 0x7f6f0015a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:29.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.016+0000 7f6f06400640 1 --2- 172.21.15.86:0/3803365036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f00163980 0x7f6f000cce70 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800be90 tx=0x7f6ef800bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:29.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.016+0000 7f6efd600640 1 -- 172.21.15.86:0/3803365036 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8019070 con 0x7f6f00163980 2024-04-25T14:06:29.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.016+0000 7f6efd600640 1 -- 172.21.15.86:0/3803365036 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ef8014440 con 0x7f6f00163980 2024-04-25T14:06:29.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.016+0000 7f6efd600640 1 -- 172.21.15.86:0/3803365036 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8014870 con 0x7f6f00163980 2024-04-25T14:06:29.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.016+0000 7f6f08400640 1 -- 172.21.15.86:0/3803365036 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f000c7ae0 con 0x7f6f00163980 2024-04-25T14:06:29.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.016+0000 7f6f08400640 1 -- 172.21.15.86:0/3803365036 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f00159b20 con 0x7f6f00163980 2024-04-25T14:06:29.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.016+0000 7f6f08400640 1 -- 172.21.15.86:0/3803365036 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f000c7cb0 con 0x7f6f00163980 2024-04-25T14:06:29.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.020+0000 7f6efd600640 1 -- 172.21.15.86:0/3803365036 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f6ef801c070 con 0x7f6f00163980 2024-04-25T14:06:29.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.020+0000 7f6efd600640 1 --2- 172.21.15.86:0/3803365036 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f6ee006f640 0x7f6ee0071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:29.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.020+0000 7f6efd600640 1 -- 172.21.15.86:0/3803365036 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 7121+0+0 (secure 0 0 0) 0x7f6ef808fc80 con 0x7f6f00163980 2024-04-25T14:06:29.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.020+0000 7f6f05a00640 1 --2- 172.21.15.86:0/3803365036 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f6ee006f640 0x7f6ee0071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:29.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.020+0000 7f6f05a00640 1 --2- 172.21.15.86:0/3803365036 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f6ee006f640 0x7f6ee0071b50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6ef40ad000 tx=0x7f6ef40a6000 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:29.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.024+0000 7f6efd600640 1 -- 172.21.15.86:0/3803365036 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6f000c7f60 con 0x7f6f00163980 2024-04-25T14:06:29.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.180+0000 7f6f08400640 1 -- 172.21.15.86:0/3803365036 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6f00072170 con 0x7f6f00163980 2024-04-25T14:06:29.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.180+0000 7f6efd600640 1 -- 172.21.15.86:0/3803365036 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+18395 (secure 0 0 0) 0x7f6ef805c270 con 0x7f6f00163980 2024-04-25T14:06:29.183 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:29.183 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":62,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:06:28.832848+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"fs2_metadata","create_time":"2024-04-25T14:05:52.841764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"fs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"fs2_data","create_time":"2024-04-25T14:05:53.524454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"fs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6835/2169924270":"2024-04-26T14:06:25.821769+0000","172.21.15.86:6834/2169924270":"2024-04-26T14:06:25.821769+0000","172.21.15.86:0/145108428":"2024-04-26T14:06:19.409429+0000","172.21.15.86:6833/2427179681":"2024-04-26T14:06:19.409429+0000","172.21.15.86:6832/2427179681":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/1321309855":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/525010030":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/3470358079":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/1177884430":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/3644218120":"2024-04-26T14:06:19.409429+0000","172.21.15.86:6836/1732615622":"2024-04-26T14:06:00.230367+0000","172.21.15.86:6837/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:6837/2418778265":"2024-04-26T14:06:00.222915+0000","172.21.15.86:0/1843278167":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6832/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/2073988773":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6835/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:0/2602032945":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/3558294881":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6835/615772001":"2024-04-26T14:05:42.882867+0000","172.21.15.86:6837/615772001":"2024-04-26T14:05:42.882867+0000","172.21.15.86:6833/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6835/2418778265":"2024-04-26T14:06:00.222915+0000","172.21.15.86:6834/1732615622":"2024-04-26T14:06:00.230367+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:06:29.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.184+0000 7f6f08400640 1 -- 172.21.15.86:0/3803365036 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f6ee006f640 msgr2=0x7f6ee0071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:29.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.184+0000 7f6f08400640 1 --2- 172.21.15.86:0/3803365036 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f6ee006f640 0x7f6ee0071b50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6ef40ad000 tx=0x7f6ef40a6000 comp rx=0 tx=0).stop 2024-04-25T14:06:29.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.184+0000 7f6f08400640 1 -- 172.21.15.86:0/3803365036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f00163980 msgr2=0x7f6f000cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:29.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.184+0000 7f6f08400640 1 --2- 172.21.15.86:0/3803365036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f00163980 0x7f6f000cce70 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800be90 tx=0x7f6ef800bf80 comp rx=0 tx=0).stop 2024-04-25T14:06:29.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.184+0000 7f6f08400640 1 -- 172.21.15.86:0/3803365036 shutdown_connections 2024-04-25T14:06:29.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.184+0000 7f6f08400640 1 --2- 172.21.15.86:0/3803365036 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f6ee006f640 0x7f6ee0071b50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.184+0000 7f6f08400640 1 --2- 172.21.15.86:0/3803365036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f00163980 0x7f6f000cce70 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.184+0000 7f6f08400640 1 --2- 172.21.15.86:0/3803365036 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f00072630 0x7f6f000c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.184+0000 7f6f08400640 1 --2- 172.21.15.86:0/3803365036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f0006e340 0x7f6f0015a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.184+0000 7f6f08400640 1 -- 172.21.15.86:0/3803365036 >> 172.21.15.86:0/3803365036 conn(0x7f6f000d3580 msgr2=0x7f6f000d3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:29.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.184+0000 7f6f08400640 1 -- 172.21.15.86:0/3803365036 shutdown_connections 2024-04-25T14:06:29.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.184+0000 7f6f08400640 1 -- 172.21.15.86:0/3803365036 wait complete. 2024-04-25T14:06:29.205 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'fs2_metadata', 'create_time': '2024-04-25T14:05:52.841764+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'fs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'fs2_data', 'create_time': '2024-04-25T14:05:53.524454+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'fs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:06:29.205 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:06:29.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.352+0000 7f12b5200640 1 Processor -- start 2024-04-25T14:06:29.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.352+0000 7f12b5200640 1 -- start start 2024-04-25T14:06:29.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.356+0000 7f12b5200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f12b00740c0 0x7f12b01676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:29.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.356+0000 7f12b5200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12b0072bd0 con 0x7f12b00cfca0 2024-04-25T14:06:29.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.356+0000 7f12b5200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12b006fbd0 con 0x7f12b00d16e0 2024-04-25T14:06:29.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.356+0000 7f12b5200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f12b0070470 con 0x7f12b00740c0 2024-04-25T14:06:29.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.356+0000 7f12ade00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f12b00cfca0 0x7f12b006f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59958/0 (socket says 172.21.15.86:59958) 2024-04-25T14:06:29.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.356+0000 7f12ade00640 1 -- 172.21.15.86:0/2630005217 learned_addr learned my addr 172.21.15.86:0/2630005217 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:29.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.356+0000 7f12ae800640 1 --2- 172.21.15.86:0/2630005217 >> v2:172.21.15.86:3300/0 conn(0x7f12b00740c0 0x7f12b01676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:29.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.356+0000 7f12ad400640 1 -- 172.21.15.86:0/2630005217 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2092187425 0 0) 0x7f12b0072bd0 con 0x7f12b00cfca0 2024-04-25T14:06:29.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.356+0000 7f12ad400640 1 -- 172.21.15.86:0/2630005217 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1294005740 con 0x7f12b00cfca0 2024-04-25T14:06:29.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.356+0000 7f12ae800640 1 -- 172.21.15.86:0/2630005217 >> v1:172.21.15.155:6789/0 conn(0x7f12b00d16e0 legacy=0x7f12b00d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:29.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.356+0000 7f12ae800640 1 -- 172.21.15.86:0/2630005217 >> v1:172.21.15.86:6789/0 conn(0x7f12b00cfca0 legacy=0x7f12b006f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:29.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.356+0000 7f12ae800640 1 -- 172.21.15.86:0/2630005217 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b006feb0 con 0x7f12b00740c0 2024-04-25T14:06:29.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.360+0000 7f12ae800640 1 --2- 172.21.15.86:0/2630005217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b00740c0 0x7f12b01676d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009d90 tx=0x7f12a801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb06d62cef96b339 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:29.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.360+0000 7f12ad400640 1 -- 172.21.15.86:0/2630005217 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a80b2070 con 0x7f12b00740c0 2024-04-25T14:06:29.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.360+0000 7f12ad400640 1 -- 172.21.15.86:0/2630005217 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12a80a5840 con 0x7f12b00740c0 2024-04-25T14:06:29.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.360+0000 7f12ad400640 1 -- 172.21.15.86:0/2630005217 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a80a5c10 con 0x7f12b00740c0 2024-04-25T14:06:29.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.360+0000 7f12b5200640 1 -- 172.21.15.86:0/2630005217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b00740c0 msgr2=0x7f12b01676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:29.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.360+0000 7f12b5200640 1 --2- 172.21.15.86:0/2630005217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b00740c0 0x7f12b01676d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009d90 tx=0x7f12a801c970 comp rx=0 tx=0).stop 2024-04-25T14:06:29.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.360+0000 7f12b5200640 1 -- 172.21.15.86:0/2630005217 shutdown_connections 2024-04-25T14:06:29.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.360+0000 7f12b5200640 1 --2- 172.21.15.86:0/2630005217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b00740c0 0x7f12b01676d0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.360+0000 7f12b5200640 1 -- 172.21.15.86:0/2630005217 >> 172.21.15.86:0/2630005217 conn(0x7f12b00b4650 msgr2=0x7f12b00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:29.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.360+0000 7f12b5200640 1 -- 172.21.15.86:0/2630005217 shutdown_connections 2024-04-25T14:06:29.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.360+0000 7f12b5200640 1 -- 172.21.15.86:0/2630005217 wait complete. 2024-04-25T14:06:29.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.360+0000 7f12b5200640 1 Processor -- start 2024-04-25T14:06:29.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.360+0000 7f12b5200640 1 -- start start 2024-04-25T14:06:29.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.364+0000 7f12b5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b00740c0 0x7f12b0158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:29.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.364+0000 7f12b5200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12b00cfca0 0x7f12b015b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:29.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.364+0000 7f12b5200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12b00d16e0 0x7f12b015b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:29.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.364+0000 7f12b5200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12b00ce900 con 0x7f12b00740c0 2024-04-25T14:06:29.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.364+0000 7f12b5200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f12b00ce780 con 0x7f12b00cfca0 2024-04-25T14:06:29.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.364+0000 7f12b5200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f12b00cea80 con 0x7f12b00d16e0 2024-04-25T14:06:29.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.364+0000 7f12ade00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b00740c0 0x7f12b0158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:29.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.364+0000 7f12ade00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b00740c0 0x7f12b0158c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49178/0 (socket says 172.21.15.86:49178) 2024-04-25T14:06:29.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.364+0000 7f12ade00640 1 -- 172.21.15.86:0/1835438459 learned_addr learned my addr 172.21.15.86:0/1835438459 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:29.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.364+0000 7f12ae800640 1 --2- 172.21.15.86:0/1835438459 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12b00d16e0 0x7f12b015b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:29.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.364+0000 7f12a5400640 1 --2- 172.21.15.86:0/1835438459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12b00cfca0 0x7f12b015b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:29.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.364+0000 7f12ade00640 1 -- 172.21.15.86:0/1835438459 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12b00d16e0 msgr2=0x7f12b015b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:29.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.364+0000 7f12ade00640 1 --2- 172.21.15.86:0/1835438459 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12b00d16e0 0x7f12b015b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.364+0000 7f12ade00640 1 -- 172.21.15.86:0/1835438459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12b00cfca0 msgr2=0x7f12b015b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:29.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.364+0000 7f12ade00640 1 --2- 172.21.15.86:0/1835438459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12b00cfca0 0x7f12b015b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.364+0000 7f12ade00640 1 -- 172.21.15.86:0/1835438459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b015c450 con 0x7f12b00740c0 2024-04-25T14:06:29.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.364+0000 7f12a5400640 1 --2- 172.21.15.86:0/1835438459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12b00cfca0 0x7f12b015b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:29.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.364+0000 7f12ade00640 1 --2- 172.21.15.86:0/1835438459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b00740c0 0x7f12b0158c00 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f12a000c430 tx=0x7f12a000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:29.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.364+0000 7f12a6a00640 1 -- 172.21.15.86:0/1835438459 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a0017070 con 0x7f12b00740c0 2024-04-25T14:06:29.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.364+0000 7f12a6a00640 1 -- 172.21.15.86:0/1835438459 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12a0005cf0 con 0x7f12b00740c0 2024-04-25T14:06:29.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.364+0000 7f12b5200640 1 -- 172.21.15.86:0/1835438459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b015d290 con 0x7f12b00740c0 2024-04-25T14:06:29.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.364+0000 7f12a6a00640 1 -- 172.21.15.86:0/1835438459 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a00072f0 con 0x7f12b00740c0 2024-04-25T14:06:29.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.364+0000 7f12b5200640 1 -- 172.21.15.86:0/1835438459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b015eb10 con 0x7f12b00740c0 2024-04-25T14:06:29.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.364+0000 7f12b5200640 1 -- 172.21.15.86:0/1835438459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12b015c620 con 0x7f12b00740c0 2024-04-25T14:06:29.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.368+0000 7f12a6a00640 1 -- 172.21.15.86:0/1835438459 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f12a001a070 con 0x7f12b00740c0 2024-04-25T14:06:29.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.368+0000 7f12a6a00640 1 --2- 172.21.15.86:0/1835438459 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f128c06f640 0x7f128c071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:29.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.368+0000 7f12a6a00640 1 -- 172.21.15.86:0/1835438459 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 7121+0+0 (secure 0 0 0) 0x7f12a008ef70 con 0x7f12b00740c0 2024-04-25T14:06:29.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.368+0000 7f12a5400640 1 --2- 172.21.15.86:0/1835438459 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f128c06f640 0x7f128c071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:29.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.368+0000 7f12a5400640 1 --2- 172.21.15.86:0/1835438459 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f128c06f640 0x7f128c071b50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f12b00cbcd0 tx=0x7f1298009040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:29.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.372+0000 7f12a6a00640 1 -- 172.21.15.86:0/1835438459 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f12a005afb0 con 0x7f12b00740c0 2024-04-25T14:06:29.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.532+0000 7f12b5200640 1 -- 172.21.15.86:0/1835438459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f12b0069d00 con 0x7f12b00740c0 2024-04-25T14:06:29.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.532+0000 7f12a6a00640 1 -- 172.21.15.86:0/1835438459 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) ==== 93+0+10 (secure 0 0 0) 0x7f12a005f2b0 con 0x7f12b00740c0 2024-04-25T14:06:29.534 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:06:29.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.536+0000 7f12b5200640 1 -- 172.21.15.86:0/1835438459 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f128c06f640 msgr2=0x7f128c071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:29.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.536+0000 7f12b5200640 1 --2- 172.21.15.86:0/1835438459 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f128c06f640 0x7f128c071b50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f12b00cbcd0 tx=0x7f1298009040 comp rx=0 tx=0).stop 2024-04-25T14:06:29.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.536+0000 7f12b5200640 1 -- 172.21.15.86:0/1835438459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b00740c0 msgr2=0x7f12b0158c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:29.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.536+0000 7f12b5200640 1 --2- 172.21.15.86:0/1835438459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b00740c0 0x7f12b0158c00 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f12a000c430 tx=0x7f12a000c8f0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.536+0000 7f12b5200640 1 -- 172.21.15.86:0/1835438459 shutdown_connections 2024-04-25T14:06:29.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.536+0000 7f12b5200640 1 --2- 172.21.15.86:0/1835438459 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f128c06f640 0x7f128c071b50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.536+0000 7f12b5200640 1 --2- 172.21.15.86:0/1835438459 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12b00d16e0 0x7f12b015b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.536+0000 7f12b5200640 1 --2- 172.21.15.86:0/1835438459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12b00cfca0 0x7f12b015b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.536+0000 7f12b5200640 1 --2- 172.21.15.86:0/1835438459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b00740c0 0x7f12b0158c00 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.536+0000 7f12b5200640 1 -- 172.21.15.86:0/1835438459 >> 172.21.15.86:0/1835438459 conn(0x7f12b00b4650 msgr2=0x7f12b00b0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:29.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.536+0000 7f12b5200640 1 -- 172.21.15.86:0/1835438459 shutdown_connections 2024-04-25T14:06:29.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.536+0000 7f12b5200640 1 -- 172.21.15.86:0/1835438459 wait complete. 2024-04-25T14:06:29.556 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get fs2_metadata pg_num 2024-04-25T14:06:29.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.704+0000 7f7e32c00640 1 Processor -- start 2024-04-25T14:06:29.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e32c00640 1 -- start start 2024-04-25T14:06:29.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e32c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7e2c0aeea0 0x7f7e2c0af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:29.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e32c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e2c06c670 con 0x7f7e2c0b59e0 2024-04-25T14:06:29.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e32c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e2c06c950 con 0x7f7e2c15faa0 2024-04-25T14:06:29.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e32c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f7e2c06d560 con 0x7f7e2c0aeea0 2024-04-25T14:06:29.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e23400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7e2c0b59e0 0x7f7e2c0b5df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59964/0 (socket says 172.21.15.86:59964) 2024-04-25T14:06:29.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e23400640 1 -- 172.21.15.86:0/2991283339 learned_addr learned my addr 172.21.15.86:0/2991283339 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:29.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e30c00640 1 --2- 172.21.15.86:0/2991283339 >> v2:172.21.15.155:3300/0 conn(0x7f7e2c0aeea0 0x7f7e2c0af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:29.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e2b400640 1 -- 172.21.15.86:0/2991283339 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 108401673 0 0) 0x7f7e2c06c670 con 0x7f7e2c0b59e0 2024-04-25T14:06:29.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e2b400640 1 -- 172.21.15.86:0/2991283339 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e10003530 con 0x7f7e2c0b59e0 2024-04-25T14:06:29.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e2b400640 1 -- 172.21.15.86:0/2991283339 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1103325703 0 0) 0x7f7e10003530 con 0x7f7e2c0b59e0 2024-04-25T14:06:29.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e2b400640 1 -- 172.21.15.86:0/2991283339 >> v2:172.21.15.155:3300/0 conn(0x7f7e2c0aeea0 msgr2=0x7f7e2c0af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:29.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e2b400640 1 --2- 172.21.15.86:0/2991283339 >> v2:172.21.15.155:3300/0 conn(0x7f7e2c0aeea0 0x7f7e2c0af2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e2b400640 1 -- 172.21.15.86:0/2991283339 >> v1:172.21.15.155:6789/0 conn(0x7f7e2c15faa0 legacy=0x7f7e2c06d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:29.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e2b400640 1 -- 172.21.15.86:0/2991283339 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2c06cc30 con 0x7f7e2c0b59e0 2024-04-25T14:06:29.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e2b400640 1 -- 172.21.15.86:0/2991283339 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f7e14001680 con 0x7f7e2c0b59e0 2024-04-25T14:06:29.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e2b400640 1 -- 172.21.15.86:0/2991283339 >> v1:172.21.15.86:6789/0 conn(0x7f7e2c0b59e0 legacy=0x7f7e2c0b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e2b400640 1 --2- 172.21.15.86:0/2991283339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e10003a30 0x7f7e10003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e2b400640 1 --2- 172.21.15.86:0/2991283339 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e10004810 0x7f7e10004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e2b400640 1 --2- 172.21.15.86:0/2991283339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e10005230 0x7f7e1000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e2b400640 1 -- 172.21.15.86:0/2991283339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c06d560 con 0x7f7e10005230 2024-04-25T14:06:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e2b400640 1 -- 172.21.15.86:0/2991283339 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1000c1c0 con 0x7f7e10003a30 2024-04-25T14:06:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e2b400640 1 -- 172.21.15.86:0/2991283339 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7e1000c3c0 con 0x7f7e10004810 2024-04-25T14:06:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e2be00640 1 --2- 172.21.15.86:0/2991283339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e10003a30 0x7f7e10003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e23400640 1 --2- 172.21.15.86:0/2991283339 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e10004810 0x7f7e10004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e30c00640 1 --2- 172.21.15.86:0/2991283339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e10005230 0x7f7e1000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e30c00640 1 -- 172.21.15.86:0/2991283339 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e10004810 msgr2=0x7f7e10004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e30c00640 1 --2- 172.21.15.86:0/2991283339 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e10004810 0x7f7e10004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e30c00640 1 -- 172.21.15.86:0/2991283339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e10003a30 msgr2=0x7f7e10003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e30c00640 1 --2- 172.21.15.86:0/2991283339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e10003a30 0x7f7e10003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e30c00640 1 -- 172.21.15.86:0/2991283339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e10003530 con 0x7f7e10005230 2024-04-25T14:06:29.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.708+0000 7f7e2be00640 1 --2- 172.21.15.86:0/2991283339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e10003a30 0x7f7e10003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:29.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.712+0000 7f7e30c00640 1 -- 172.21.15.86:0/2991283339 reap_dead start 2024-04-25T14:06:29.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.712+0000 7f7e30c00640 1 --2- 172.21.15.86:0/2991283339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e10005230 0x7f7e1000b9d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009ae0 tx=0x7f7e2401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde63a0e3448178 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:29.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.712+0000 7f7e2b400640 1 -- 172.21.15.86:0/2991283339 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c001a90 con 0x7f7e10005230 2024-04-25T14:06:29.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.712+0000 7f7e2b400640 1 -- 172.21.15.86:0/2991283339 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e24004020 con 0x7f7e10005230 2024-04-25T14:06:29.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.712+0000 7f7e2b400640 1 -- 172.21.15.86:0/2991283339 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e240096b0 con 0x7f7e10005230 2024-04-25T14:06:29.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.712+0000 7f7e32c00640 1 -- 172.21.15.86:0/2991283339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e10005230 msgr2=0x7f7e1000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:29.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.712+0000 7f7e32c00640 1 --2- 172.21.15.86:0/2991283339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e10005230 0x7f7e1000b9d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009ae0 tx=0x7f7e2401cab0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.712+0000 7f7e32c00640 1 -- 172.21.15.86:0/2991283339 shutdown_connections 2024-04-25T14:06:29.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.712+0000 7f7e32c00640 1 --2- 172.21.15.86:0/2991283339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e10005230 0x7f7e1000b9d0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.712+0000 7f7e32c00640 1 -- 172.21.15.86:0/2991283339 >> 172.21.15.86:0/2991283339 conn(0x7f7e2c0d2230 msgr2=0x7f7e2c0d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:29.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.716+0000 7f7e32c00640 1 -- 172.21.15.86:0/2991283339 shutdown_connections 2024-04-25T14:06:29.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.716+0000 7f7e32c00640 1 -- 172.21.15.86:0/2991283339 wait complete. 2024-04-25T14:06:29.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.716+0000 7f7e32c00640 1 Processor -- start 2024-04-25T14:06:29.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.716+0000 7f7e32c00640 1 -- start start 2024-04-25T14:06:29.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.716+0000 7f7e32c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e10005230 0x7f7e2c146790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:29.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.716+0000 7f7e32c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e2c15cc20 0x7f7e2c0af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:29.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.716+0000 7f7e32c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e2c1f9c90 0x7f7e2c1fa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:29.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.716+0000 7f7e32c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c15feb0 con 0x7f7e2c15cc20 2024-04-25T14:06:29.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.716+0000 7f7e32c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c06d1f0 con 0x7f7e2c1f9c90 2024-04-25T14:06:29.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.716+0000 7f7e32c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7e2c160030 con 0x7f7e10005230 2024-04-25T14:06:29.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.720+0000 7f7e23400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e2c15cc20 0x7f7e2c0af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:29.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.720+0000 7f7e2be00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e10005230 0x7f7e2c146790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:29.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.720+0000 7f7e23400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e2c15cc20 0x7f7e2c0af2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49192/0 (socket says 172.21.15.86:49192) 2024-04-25T14:06:29.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.720+0000 7f7e2be00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e10005230 0x7f7e2c146790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:53126/0 (socket says 172.21.15.86:53126) 2024-04-25T14:06:29.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.720+0000 7f7e23400640 1 -- 172.21.15.86:0/208063734 learned_addr learned my addr 172.21.15.86:0/208063734 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:29.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.720+0000 7f7e30c00640 1 --2- 172.21.15.86:0/208063734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e2c1f9c90 0x7f7e2c1fa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:29.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.720+0000 7f7e23400640 1 -- 172.21.15.86:0/208063734 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e10005230 msgr2=0x7f7e2c146790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:29.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.720+0000 7f7e23400640 1 --2- 172.21.15.86:0/208063734 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e10005230 0x7f7e2c146790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.720+0000 7f7e23400640 1 -- 172.21.15.86:0/208063734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e2c1f9c90 msgr2=0x7f7e2c1fa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:29.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.720+0000 7f7e23400640 1 --2- 172.21.15.86:0/208063734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e2c1f9c90 0x7f7e2c1fa0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.720+0000 7f7e23400640 1 -- 172.21.15.86:0/208063734 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2c1fa5e0 con 0x7f7e2c15cc20 2024-04-25T14:06:29.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.720+0000 7f7e30c00640 1 --2- 172.21.15.86:0/208063734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e2c1f9c90 0x7f7e2c1fa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:29.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.720+0000 7f7e23400640 1 --2- 172.21.15.86:0/208063734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e2c15cc20 0x7f7e2c0af2b0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7e14007fb0 tx=0x7f7e140074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:29.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.720+0000 7f7e28c00640 1 -- 172.21.15.86:0/208063734 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e14007740 con 0x7f7e2c15cc20 2024-04-25T14:06:29.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.720+0000 7f7e28c00640 1 -- 172.21.15.86:0/208063734 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e14018070 con 0x7f7e2c15cc20 2024-04-25T14:06:29.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.720+0000 7f7e28c00640 1 -- 172.21.15.86:0/208063734 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1400fb80 con 0x7f7e2c15cc20 2024-04-25T14:06:29.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.720+0000 7f7e32c00640 1 -- 172.21.15.86:0/208063734 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e2c0b10e0 con 0x7f7e2c15cc20 2024-04-25T14:06:29.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.720+0000 7f7e32c00640 1 -- 172.21.15.86:0/208063734 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e2c0b16a0 con 0x7f7e2c15cc20 2024-04-25T14:06:29.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.720+0000 7f7e32c00640 1 -- 172.21.15.86:0/208063734 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e2c074cf0 con 0x7f7e2c15cc20 2024-04-25T14:06:29.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.720+0000 7f7e28c00640 1 -- 172.21.15.86:0/208063734 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f7e1400fd20 con 0x7f7e2c15cc20 2024-04-25T14:06:29.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.724+0000 7f7e28c00640 1 --2- 172.21.15.86:0/208063734 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f7e0806f670 0x7f7e08071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:29.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.724+0000 7f7e28c00640 1 -- 172.21.15.86:0/208063734 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 7121+0+0 (secure 0 0 0) 0x7f7e1408fb40 con 0x7f7e2c15cc20 2024-04-25T14:06:29.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.724+0000 7f7e2be00640 1 --2- 172.21.15.86:0/208063734 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f7e0806f670 0x7f7e08071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:29.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.724+0000 7f7e2be00640 1 --2- 172.21.15.86:0/208063734 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f7e0806f670 0x7f7e08071b80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c000f90 tx=0x7f7e1c009040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:29.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.728+0000 7f7e28c00640 1 -- 172.21.15.86:0/208063734 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7e2c1fa5e0 con 0x7f7e2c15cc20 2024-04-25T14:06:29.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.896+0000 7f7e32c00640 1 -- 172.21.15.86:0/208063734 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "fs2_metadata", "var": "pg_num"} v 0) -- 0x7f7e2c0b30d0 con 0x7f7e2c15cc20 2024-04-25T14:06:29.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.896+0000 7f7e28c00640 1 -- 172.21.15.86:0/208063734 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "fs2_metadata", "var": "pg_num"}]=0 v62) ==== 101+0+11 (secure 0 0 0) 0x7f7e1405c130 con 0x7f7e2c15cc20 2024-04-25T14:06:29.899 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:06:29.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.900+0000 7f7e32c00640 1 -- 172.21.15.86:0/208063734 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f7e0806f670 msgr2=0x7f7e08071b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:29.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.900+0000 7f7e32c00640 1 --2- 172.21.15.86:0/208063734 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f7e0806f670 0x7f7e08071b80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c000f90 tx=0x7f7e1c009040 comp rx=0 tx=0).stop 2024-04-25T14:06:29.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.900+0000 7f7e32c00640 1 -- 172.21.15.86:0/208063734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e2c15cc20 msgr2=0x7f7e2c0af2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:29.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.900+0000 7f7e32c00640 1 --2- 172.21.15.86:0/208063734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e2c15cc20 0x7f7e2c0af2b0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7e14007fb0 tx=0x7f7e140074f0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.900+0000 7f7e32c00640 1 -- 172.21.15.86:0/208063734 shutdown_connections 2024-04-25T14:06:29.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.900+0000 7f7e32c00640 1 --2- 172.21.15.86:0/208063734 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f7e0806f670 0x7f7e08071b80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.900+0000 7f7e32c00640 1 --2- 172.21.15.86:0/208063734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e2c1f9c90 0x7f7e2c1fa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.900+0000 7f7e32c00640 1 --2- 172.21.15.86:0/208063734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e2c15cc20 0x7f7e2c0af2b0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.900+0000 7f7e32c00640 1 --2- 172.21.15.86:0/208063734 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e10005230 0x7f7e2c146790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:29.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.900+0000 7f7e32c00640 1 -- 172.21.15.86:0/208063734 >> 172.21.15.86:0/208063734 conn(0x7f7e2c0d2230 msgr2=0x7f7e2c160210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:29.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.900+0000 7f7e32c00640 1 -- 172.21.15.86:0/208063734 shutdown_connections 2024-04-25T14:06:29.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:29.900+0000 7f7e32c00640 1 -- 172.21.15.86:0/208063734 wait complete. 2024-04-25T14:06:29.921 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get fs2_data pg_num 2024-04-25T14:06:30.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.068+0000 7f7a65c00640 1 Processor -- start 2024-04-25T14:06:30.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.068+0000 7f7a65c00640 1 -- start start 2024-04-25T14:06:30.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a65c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a600d1df0 0x7f7a600cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:30.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a65c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a6015e5c0 con 0x7f7a600cdcc0 2024-04-25T14:06:30.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a65c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a6015e8a0 con 0x7f7a60070330 2024-04-25T14:06:30.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a65c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7a6006fd40 con 0x7f7a600d1df0 2024-04-25T14:06:30.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a5f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7a600cdcc0 0x7f7a600ce0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59968/0 (socket says 172.21.15.86:59968) 2024-04-25T14:06:30.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a5f400640 1 -- 172.21.15.86:0/618220258 learned_addr learned my addr 172.21.15.86:0/618220258 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:30.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a56000640 1 --2- 172.21.15.86:0/618220258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a600d1df0 0x7f7a600cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:30.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a5e000640 1 -- 172.21.15.86:0/618220258 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1078706230 0 0) 0x7f7a6015e5c0 con 0x7f7a600cdcc0 2024-04-25T14:06:30.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a5e000640 1 -- 172.21.15.86:0/618220258 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a44003530 con 0x7f7a600cdcc0 2024-04-25T14:06:30.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a5e000640 1 -- 172.21.15.86:0/618220258 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1147514291 0 0) 0x7f7a6015e8a0 con 0x7f7a60070330 2024-04-25T14:06:30.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a5e000640 1 -- 172.21.15.86:0/618220258 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a6015e5c0 con 0x7f7a60070330 2024-04-25T14:06:30.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a5e000640 1 -- 172.21.15.86:0/618220258 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3686952902 0 0) 0x7f7a44003530 con 0x7f7a600cdcc0 2024-04-25T14:06:30.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a5e000640 1 -- 172.21.15.86:0/618220258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a600d1df0 msgr2=0x7f7a600cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:30.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a5e000640 1 --2- 172.21.15.86:0/618220258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a600d1df0 0x7f7a600cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:30.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a5e000640 1 -- 172.21.15.86:0/618220258 >> v1:172.21.15.155:6789/0 conn(0x7f7a60070330 legacy=0x7f7a600d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:30.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a5e000640 1 -- 172.21.15.86:0/618220258 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6015eb80 con 0x7f7a600cdcc0 2024-04-25T14:06:30.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a5e000640 1 -- 172.21.15.86:0/618220258 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f7a58001680 con 0x7f7a600cdcc0 2024-04-25T14:06:30.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a5e000640 1 -- 172.21.15.86:0/618220258 >> v1:172.21.15.86:6789/0 conn(0x7f7a600cdcc0 legacy=0x7f7a600ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:30.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a5e000640 1 --2- 172.21.15.86:0/618220258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a44003bf0 0x7f7a44004000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:30.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a5e000640 1 --2- 172.21.15.86:0/618220258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a44004810 0x7f7a44004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:30.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a5e000640 1 --2- 172.21.15.86:0/618220258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a44005160 0x7f7a4400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:30.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a5e000640 1 -- 172.21.15.86:0/618220258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4400c040 con 0x7f7a44004810 2024-04-25T14:06:30.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a5e000640 1 -- 172.21.15.86:0/618220258 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4400c1f0 con 0x7f7a44005160 2024-04-25T14:06:30.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a5e000640 1 -- 172.21.15.86:0/618220258 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7a4400c3f0 con 0x7f7a44003bf0 2024-04-25T14:06:30.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a56000640 1 --2- 172.21.15.86:0/618220258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a44004810 0x7f7a44004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:30.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a5f400640 1 --2- 172.21.15.86:0/618220258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a44005160 0x7f7a4400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:30.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a5ea00640 1 --2- 172.21.15.86:0/618220258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a44003bf0 0x7f7a44004000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:30.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a56000640 1 -- 172.21.15.86:0/618220258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a44003bf0 msgr2=0x7f7a44004000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:30.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a56000640 1 --2- 172.21.15.86:0/618220258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a44003bf0 0x7f7a44004000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:30.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a56000640 1 -- 172.21.15.86:0/618220258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a44005160 msgr2=0x7f7a4400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:30.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a56000640 1 --2- 172.21.15.86:0/618220258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a44005160 0x7f7a4400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:30.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.072+0000 7f7a56000640 1 -- 172.21.15.86:0/618220258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a44003530 con 0x7f7a44004810 2024-04-25T14:06:30.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.076+0000 7f7a56000640 1 --2- 172.21.15.86:0/618220258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a44004810 0x7f7a44004c20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009ae0 tx=0x7f7a4801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83afa5d0f2a23409 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:30.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.076+0000 7f7a5e000640 1 -- 172.21.15.86:0/618220258 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a480b2070 con 0x7f7a44004810 2024-04-25T14:06:30.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.076+0000 7f7a5e000640 1 -- 172.21.15.86:0/618220258 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a480a5550 con 0x7f7a44004810 2024-04-25T14:06:30.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.076+0000 7f7a5e000640 1 -- 172.21.15.86:0/618220258 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a480096b0 con 0x7f7a44004810 2024-04-25T14:06:30.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.076+0000 7f7a65c00640 1 -- 172.21.15.86:0/618220258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a44004810 msgr2=0x7f7a44004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:30.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.076+0000 7f7a65c00640 1 --2- 172.21.15.86:0/618220258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a44004810 0x7f7a44004c20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009ae0 tx=0x7f7a4801c9a0 comp rx=0 tx=0).stop 2024-04-25T14:06:30.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.076+0000 7f7a5f400640 1 -- 172.21.15.86:0/618220258 reap_dead start 2024-04-25T14:06:30.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.076+0000 7f7a65c00640 1 -- 172.21.15.86:0/618220258 shutdown_connections 2024-04-25T14:06:30.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.076+0000 7f7a65c00640 1 -- 172.21.15.86:0/618220258 >> 172.21.15.86:0/618220258 conn(0x7f7a600b4650 msgr2=0x7f7a600b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:30.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.076+0000 7f7a65c00640 1 -- 172.21.15.86:0/618220258 shutdown_connections 2024-04-25T14:06:30.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.076+0000 7f7a65c00640 1 -- 172.21.15.86:0/618220258 wait complete. 2024-04-25T14:06:30.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.080+0000 7f7a65c00640 1 Processor -- start 2024-04-25T14:06:30.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.080+0000 7f7a65c00640 1 -- start start 2024-04-25T14:06:30.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.080+0000 7f7a65c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a44004810 0x7f7a601f9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:30.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.080+0000 7f7a65c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a601583f0 0x7f7a6015d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:30.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.080+0000 7f7a65c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a6015da20 0x7f7a60168840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:30.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.080+0000 7f7a65c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7a600708c0 con 0x7f7a601583f0 2024-04-25T14:06:30.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.080+0000 7f7a65c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7a60070740 con 0x7f7a44004810 2024-04-25T14:06:30.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.080+0000 7f7a65c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7a600d1df0 con 0x7f7a6015da20 2024-04-25T14:06:30.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.080+0000 7f7a56000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a601583f0 0x7f7a6015d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:30.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.080+0000 7f7a56000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a601583f0 0x7f7a6015d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49214/0 (socket says 172.21.15.86:49214) 2024-04-25T14:06:30.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.080+0000 7f7a56000640 1 -- 172.21.15.86:0/978307561 learned_addr learned my addr 172.21.15.86:0/978307561 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:30.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.080+0000 7f7a5ea00640 1 --2- 172.21.15.86:0/978307561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a44004810 0x7f7a601f9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:30.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.080+0000 7f7a5f400640 1 --2- 172.21.15.86:0/978307561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a6015da20 0x7f7a60168840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:30.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.080+0000 7f7a56000640 1 -- 172.21.15.86:0/978307561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a6015da20 msgr2=0x7f7a60168840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:30.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.080+0000 7f7a56000640 1 --2- 172.21.15.86:0/978307561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a6015da20 0x7f7a60168840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:30.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.080+0000 7f7a56000640 1 -- 172.21.15.86:0/978307561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a44004810 msgr2=0x7f7a601f9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:30.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.080+0000 7f7a56000640 1 --2- 172.21.15.86:0/978307561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a44004810 0x7f7a601f9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:30.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.080+0000 7f7a56000640 1 -- 172.21.15.86:0/978307561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6009c0a0 con 0x7f7a601583f0 2024-04-25T14:06:30.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.080+0000 7f7a56000640 1 --2- 172.21.15.86:0/978307561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a601583f0 0x7f7a6015d4e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009ae0 tx=0x7f7a44005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:30.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.080+0000 7f7a57400640 1 -- 172.21.15.86:0/978307561 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a480bd070 con 0x7f7a601583f0 2024-04-25T14:06:30.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.080+0000 7f7a57400640 1 -- 172.21.15.86:0/978307561 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a480a5d00 con 0x7f7a601583f0 2024-04-25T14:06:30.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.084+0000 7f7a57400640 1 -- 172.21.15.86:0/978307561 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a480b2040 con 0x7f7a601583f0 2024-04-25T14:06:30.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.084+0000 7f7a65c00640 1 -- 172.21.15.86:0/978307561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a6009c380 con 0x7f7a601583f0 2024-04-25T14:06:30.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.084+0000 7f7a65c00640 1 -- 172.21.15.86:0/978307561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a6009c940 con 0x7f7a601583f0 2024-04-25T14:06:30.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.084+0000 7f7a65c00640 1 -- 172.21.15.86:0/978307561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a60069d00 con 0x7f7a601583f0 2024-04-25T14:06:30.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.084+0000 7f7a57400640 1 -- 172.21.15.86:0/978307561 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f7a480c0070 con 0x7f7a601583f0 2024-04-25T14:06:30.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.084+0000 7f7a57400640 1 --2- 172.21.15.86:0/978307561 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f7a3c06f640 0x7f7a3c071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:30.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.084+0000 7f7a57400640 1 -- 172.21.15.86:0/978307561 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 7121+0+0 (secure 0 0 0) 0x7f7a481297f0 con 0x7f7a601583f0 2024-04-25T14:06:30.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.084+0000 7f7a5ea00640 1 --2- 172.21.15.86:0/978307561 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f7a3c06f640 0x7f7a3c071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:30.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.088+0000 7f7a5ea00640 1 --2- 172.21.15.86:0/978307561 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f7a3c06f640 0x7f7a3c071b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7a50000f90 tx=0x7f7a50009040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:30.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.088+0000 7f7a57400640 1 -- 172.21.15.86:0/978307561 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7a480166e0 con 0x7f7a601583f0 2024-04-25T14:06:30.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.248+0000 7f7a65c00640 1 -- 172.21.15.86:0/978307561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "fs2_data", "var": "pg_num"} v 0) -- 0x7f7a6009be30 con 0x7f7a601583f0 2024-04-25T14:06:30.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.248+0000 7f7a57400640 1 -- 172.21.15.86:0/978307561 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "fs2_data", "var": "pg_num"}]=0 v62) ==== 97+0+11 (secure 0 0 0) 0x7f7a480f5d50 con 0x7f7a601583f0 2024-04-25T14:06:30.249 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:06:30.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.248+0000 7f7a65c00640 1 -- 172.21.15.86:0/978307561 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f7a3c06f640 msgr2=0x7f7a3c071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:30.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.248+0000 7f7a65c00640 1 --2- 172.21.15.86:0/978307561 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f7a3c06f640 0x7f7a3c071b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7a50000f90 tx=0x7f7a50009040 comp rx=0 tx=0).stop 2024-04-25T14:06:30.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.248+0000 7f7a65c00640 1 -- 172.21.15.86:0/978307561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a601583f0 msgr2=0x7f7a6015d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:30.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.248+0000 7f7a65c00640 1 --2- 172.21.15.86:0/978307561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a601583f0 0x7f7a6015d4e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009ae0 tx=0x7f7a44005840 comp rx=0 tx=0).stop 2024-04-25T14:06:30.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.248+0000 7f7a65c00640 1 -- 172.21.15.86:0/978307561 shutdown_connections 2024-04-25T14:06:30.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.248+0000 7f7a65c00640 1 --2- 172.21.15.86:0/978307561 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f7a3c06f640 0x7f7a3c071b50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:30.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.248+0000 7f7a65c00640 1 --2- 172.21.15.86:0/978307561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a6015da20 0x7f7a60168840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:30.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.248+0000 7f7a65c00640 1 --2- 172.21.15.86:0/978307561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a601583f0 0x7f7a6015d4e0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:30.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.248+0000 7f7a65c00640 1 --2- 172.21.15.86:0/978307561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a44004810 0x7f7a601f9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:30.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.248+0000 7f7a65c00640 1 -- 172.21.15.86:0/978307561 >> 172.21.15.86:0/978307561 conn(0x7f7a600b4650 msgr2=0x7f7a6015b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:30.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.248+0000 7f7a65c00640 1 -- 172.21.15.86:0/978307561 shutdown_connections 2024-04-25T14:06:30.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.248+0000 7f7a65c00640 1 -- 172.21.15.86:0/978307561 wait complete. 2024-04-25T14:06:30.270 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:06:30.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.416+0000 7fdde1800640 1 Processor -- start 2024-04-25T14:06:30.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.416+0000 7fdde1800640 1 -- start start 2024-04-25T14:06:30.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.416+0000 7fdde1800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdddc0cdda0 0x7fdddc0cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:30.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.416+0000 7fdde1800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdddc0d18e0 0x7fdddc0d1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:30.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.416+0000 7fdde1800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdddc163940 con 0x7fdddc0cec60 2024-04-25T14:06:30.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.416+0000 7fdde1800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdddc0cfcc0 con 0x7fdddc0d18e0 2024-04-25T14:06:30.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.416+0000 7fdde1800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdddc0d2f40 con 0x7fdddc0cdda0 2024-04-25T14:06:30.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.416+0000 7fdddaa00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fdddc0cec60 0x7fdddc0cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59978/0 (socket says 172.21.15.86:59978) 2024-04-25T14:06:30.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.416+0000 7fdddaa00640 1 -- 172.21.15.86:0/1324126557 learned_addr learned my addr 172.21.15.86:0/1324126557 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:30.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.416+0000 7fdddb400640 1 --2- 172.21.15.86:0/1324126557 >> v2:172.21.15.86:3300/0 conn(0x7fdddc0d18e0 0x7fdddc0d1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:30.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.416+0000 7fddd2000640 1 --2- 172.21.15.86:0/1324126557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdddc0cdda0 0x7fdddc0cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:30.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.416+0000 7fddda000640 1 -- 172.21.15.86:0/1324126557 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4089806855 0 0) 0x7fdddc163940 con 0x7fdddc0cec60 2024-04-25T14:06:30.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.416+0000 7fddda000640 1 -- 172.21.15.86:0/1324126557 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddc0003530 con 0x7fdddc0cec60 2024-04-25T14:06:30.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.416+0000 7fdddb400640 1 -- 172.21.15.86:0/1324126557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdddc0cdda0 msgr2=0x7fdddc0cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:30.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.416+0000 7fdddb400640 1 --2- 172.21.15.86:0/1324126557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdddc0cdda0 0x7fdddc0cf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:30.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.416+0000 7fdddb400640 1 -- 172.21.15.86:0/1324126557 >> v1:172.21.15.86:6789/0 conn(0x7fdddc0cec60 legacy=0x7fdddc0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:30.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.416+0000 7fdddb400640 1 -- 172.21.15.86:0/1324126557 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdddc0b3720 con 0x7fdddc0d18e0 2024-04-25T14:06:30.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.420+0000 7fdddb400640 1 --2- 172.21.15.86:0/1324126557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdddc0d18e0 0x7fdddc0d1cf0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009d90 tx=0x7fddd401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5d981817e4ec85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:30.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.420+0000 7fddda000640 1 -- 172.21.15.86:0/1324126557 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd40b2070 con 0x7fdddc0d18e0 2024-04-25T14:06:30.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.420+0000 7fddda000640 1 -- 172.21.15.86:0/1324126557 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddd40a5830 con 0x7fdddc0d18e0 2024-04-25T14:06:30.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.420+0000 7fddda000640 1 -- 172.21.15.86:0/1324126557 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd40a5c00 con 0x7fdddc0d18e0 2024-04-25T14:06:30.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.420+0000 7fdde1800640 1 -- 172.21.15.86:0/1324126557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdddc0d18e0 msgr2=0x7fdddc0d1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:30.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.420+0000 7fdde1800640 1 --2- 172.21.15.86:0/1324126557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdddc0d18e0 0x7fdddc0d1cf0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009d90 tx=0x7fddd401c970 comp rx=0 tx=0).stop 2024-04-25T14:06:30.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.420+0000 7fdde1800640 1 -- 172.21.15.86:0/1324126557 shutdown_connections 2024-04-25T14:06:30.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.420+0000 7fdde1800640 1 --2- 172.21.15.86:0/1324126557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdddc0d18e0 0x7fdddc0d1cf0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:30.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.420+0000 7fdde1800640 1 --2- 172.21.15.86:0/1324126557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdddc0cdda0 0x7fdddc0cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:30.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.420+0000 7fdde1800640 1 -- 172.21.15.86:0/1324126557 >> 172.21.15.86:0/1324126557 conn(0x7fdddc09ba30 msgr2=0x7fdddc0b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:30.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.420+0000 7fdde1800640 1 -- 172.21.15.86:0/1324126557 shutdown_connections 2024-04-25T14:06:30.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.420+0000 7fdde1800640 1 -- 172.21.15.86:0/1324126557 wait complete. 2024-04-25T14:06:30.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.424+0000 7fdde1800640 1 Processor -- start 2024-04-25T14:06:30.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.424+0000 7fdde1800640 1 -- start start 2024-04-25T14:06:30.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.424+0000 7fdde1800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdddc0cdda0 0x7fdddc1fd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:30.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.424+0000 7fdde1800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdddc0cec60 0x7fdddc158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:30.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.424+0000 7fdde1800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdddc0d18e0 0x7fdddc15a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:30.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.424+0000 7fdde1800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc16cbf0 con 0x7fdddc0cdda0 2024-04-25T14:06:30.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.424+0000 7fdde1800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc0d2c50 con 0x7fdddc0d18e0 2024-04-25T14:06:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.424+0000 7fdde1800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdddc16cd70 con 0x7fdddc0cec60 2024-04-25T14:06:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.424+0000 7fdddaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdddc0cdda0 0x7fdddc1fd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.424+0000 7fdddaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdddc0cdda0 0x7fdddc1fd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49240/0 (socket says 172.21.15.86:49240) 2024-04-25T14:06:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.424+0000 7fdddaa00640 1 -- 172.21.15.86:0/1945868962 learned_addr learned my addr 172.21.15.86:0/1945868962 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.428+0000 7fddd2000640 1 --2- 172.21.15.86:0/1945868962 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdddc0cec60 0x7fdddc158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.428+0000 7fdddb400640 1 --2- 172.21.15.86:0/1945868962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdddc0d18e0 0x7fdddc15a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.428+0000 7fdddaa00640 1 -- 172.21.15.86:0/1945868962 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdddc0cec60 msgr2=0x7fdddc158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.428+0000 7fdddaa00640 1 --2- 172.21.15.86:0/1945868962 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdddc0cec60 0x7fdddc158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.428+0000 7fdddaa00640 1 -- 172.21.15.86:0/1945868962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdddc0d18e0 msgr2=0x7fdddc15a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.428+0000 7fdddaa00640 1 --2- 172.21.15.86:0/1945868962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdddc0d18e0 0x7fdddc15a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.428+0000 7fdddaa00640 1 -- 172.21.15.86:0/1945868962 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdddc0cc320 con 0x7fdddc0cdda0 2024-04-25T14:06:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.428+0000 7fddd2000640 1 --2- 172.21.15.86:0/1945868962 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdddc0cec60 0x7fdddc158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.428+0000 7fdddb400640 1 --2- 172.21.15.86:0/1945868962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdddc0d18e0 0x7fdddc15a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.428+0000 7fdddaa00640 1 --2- 172.21.15.86:0/1945868962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdddc0cdda0 0x7fdddc1fd440 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fddcc00c430 tx=0x7fddcc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.428+0000 7fddd3400640 1 -- 172.21.15.86:0/1945868962 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc017070 con 0x7fdddc0cdda0 2024-04-25T14:06:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.428+0000 7fddd3400640 1 -- 172.21.15.86:0/1945868962 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddcc007500 con 0x7fdddc0cdda0 2024-04-25T14:06:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.428+0000 7fddd3400640 1 -- 172.21.15.86:0/1945868962 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc0059f0 con 0x7fdddc0cdda0 2024-04-25T14:06:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.428+0000 7fdde1800640 1 -- 172.21.15.86:0/1945868962 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdddc0af140 con 0x7fdddc0cdda0 2024-04-25T14:06:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.428+0000 7fdde1800640 1 -- 172.21.15.86:0/1945868962 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdddc15e9a0 con 0x7fdddc0cdda0 2024-04-25T14:06:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.428+0000 7fdde1800640 1 -- 172.21.15.86:0/1945868962 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdddc0af310 con 0x7fdddc0cdda0 2024-04-25T14:06:30.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.428+0000 7fddd3400640 1 -- 172.21.15.86:0/1945868962 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7fddcc01a070 con 0x7fdddc0cdda0 2024-04-25T14:06:30.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.432+0000 7fddd3400640 1 --2- 172.21.15.86:0/1945868962 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fddb806f640 0x7fddb8071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:30.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.432+0000 7fddd3400640 1 -- 172.21.15.86:0/1945868962 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 7121+0+0 (secure 0 0 0) 0x7fddcc08f0a0 con 0x7fdddc0cdda0 2024-04-25T14:06:30.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.432+0000 7fddd2000640 1 --2- 172.21.15.86:0/1945868962 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fddb806f640 0x7fddb8071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:30.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.432+0000 7fddd2000640 1 --2- 172.21.15.86:0/1945868962 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fddb806f640 0x7fddb8071b50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0b2240 tx=0x7fddc4008040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:30.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.436+0000 7fddd3400640 1 -- 172.21.15.86:0/1945868962 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdddc0cc320 con 0x7fdddc0cdda0 2024-04-25T14:06:30.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.624+0000 7fdde1800640 1 -- 172.21.15.86:0/1945868962 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdddc0a04c0 con 0x7fdddc0cdda0 2024-04-25T14:06:30.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.628+0000 7fddd3400640 1 -- 172.21.15.86:0/1945868962 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5184 (secure 0 0 0) 0x7fddcc05b6a0 con 0x7fdddc0cdda0 2024-04-25T14:06:30.628 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:30.628 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":45,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3545772170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3545772170},{"type":"v1","addr":"172.21.15.155:6834","nonce":3545772170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5980,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/766530004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":766530004},{"type":"v1","addr":"172.21.15.86:6837","nonce":766530004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6414,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2676840983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2676840983},{"type":"v1","addr":"172.21.15.86:6835","nonce":2676840983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:54.565528+0000","modified":"2024-04-25T14:06:03.253173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6036":{"gid":6036,"name":"d","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6835/2914930843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2914930843},{"type":"v1","addr":"172.21.15.155:6835","nonce":2914930843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"fs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6036,"qdb_cluster":[6036]},"id":4}]} 2024-04-25T14:06:30.629 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 45 2024-04-25T14:06:30.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.628+0000 7fdde1800640 1 -- 172.21.15.86:0/1945868962 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fddb806f640 msgr2=0x7fddb8071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:30.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.628+0000 7fdde1800640 1 --2- 172.21.15.86:0/1945868962 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fddb806f640 0x7fddb8071b50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0b2240 tx=0x7fddc4008040 comp rx=0 tx=0).stop 2024-04-25T14:06:30.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.628+0000 7fdde1800640 1 -- 172.21.15.86:0/1945868962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdddc0cdda0 msgr2=0x7fdddc1fd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:30.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.628+0000 7fdde1800640 1 --2- 172.21.15.86:0/1945868962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdddc0cdda0 0x7fdddc1fd440 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fddcc00c430 tx=0x7fddcc00c8f0 comp rx=0 tx=0).stop 2024-04-25T14:06:30.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.628+0000 7fdde1800640 1 -- 172.21.15.86:0/1945868962 shutdown_connections 2024-04-25T14:06:30.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.628+0000 7fdde1800640 1 --2- 172.21.15.86:0/1945868962 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fddb806f640 0x7fddb8071b50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:30.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.628+0000 7fdde1800640 1 --2- 172.21.15.86:0/1945868962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdddc0d18e0 0x7fdddc15a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:30.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.628+0000 7fdde1800640 1 --2- 172.21.15.86:0/1945868962 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdddc0cec60 0x7fdddc158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:30.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.628+0000 7fdde1800640 1 --2- 172.21.15.86:0/1945868962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdddc0cdda0 0x7fdddc1fd440 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:30.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.628+0000 7fdde1800640 1 -- 172.21.15.86:0/1945868962 >> 172.21.15.86:0/1945868962 conn(0x7fdddc09ba30 msgr2=0x7fdddc15b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:30.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.628+0000 7fdde1800640 1 -- 172.21.15.86:0/1945868962 shutdown_connections 2024-04-25T14:06:30.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.628+0000 7fdde1800640 1 -- 172.21.15.86:0/1945868962 wait complete. 2024-04-25T14:06:30.650 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:06:30.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.796+0000 7f9d9e200640 1 Processor -- start 2024-04-25T14:06:30.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.796+0000 7f9d9e200640 1 -- start start 2024-04-25T14:06:30.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.796+0000 7f9d9e200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9d98071530 0x7f9d9815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:30.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.796+0000 7f9d9e200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d980cd8d0 con 0x7f9d981616e0 2024-04-25T14:06:30.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.796+0000 7f9d9e200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d980ce970 con 0x7f9d980d2c80 2024-04-25T14:06:30.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.796+0000 7f9d9e200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9d980ce720 con 0x7f9d98071530 2024-04-25T14:06:30.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.796+0000 7f9d96a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9d98071530 0x7f9d9815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:30.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.796+0000 7f9d97e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9d981616e0 0x7f9d98161af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59980/0 (socket says 172.21.15.86:59980) 2024-04-25T14:06:30.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.796+0000 7f9d97e00640 1 -- 172.21.15.86:0/677682170 learned_addr learned my addr 172.21.15.86:0/677682170 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:30.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.796+0000 7f9d96a00640 1 -- 172.21.15.86:0/677682170 >> v1:172.21.15.155:6789/0 conn(0x7f9d980d2c80 legacy=0x7f9d98070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:30.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.796+0000 7f9d96a00640 1 -- 172.21.15.86:0/677682170 >> v1:172.21.15.86:6789/0 conn(0x7f9d981616e0 legacy=0x7f9d98161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:30.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.796+0000 7f9d96a00640 1 -- 172.21.15.86:0/677682170 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d980d1be0 con 0x7f9d98071530 2024-04-25T14:06:30.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.800+0000 7f9d96a00640 1 --2- 172.21.15.86:0/677682170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d98071530 0x7f9d9815dc90 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9d840097a0 tx=0x7f9d8401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6acea55a9e93bf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:30.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.800+0000 7f9d96000640 1 -- 172.21.15.86:0/677682170 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d840b5070 con 0x7f9d98071530 2024-04-25T14:06:30.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.800+0000 7f9d96000640 1 -- 172.21.15.86:0/677682170 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d840a8970 con 0x7f9d98071530 2024-04-25T14:06:30.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.800+0000 7f9d96000640 1 -- 172.21.15.86:0/677682170 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d840a8d40 con 0x7f9d98071530 2024-04-25T14:06:30.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.800+0000 7f9d9e200640 1 -- 172.21.15.86:0/677682170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d98071530 msgr2=0x7f9d9815dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:30.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.800+0000 7f9d9e200640 1 --2- 172.21.15.86:0/677682170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d98071530 0x7f9d9815dc90 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9d840097a0 tx=0x7f9d8401ec70 comp rx=0 tx=0).stop 2024-04-25T14:06:30.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.800+0000 7f9d9e200640 1 -- 172.21.15.86:0/677682170 shutdown_connections 2024-04-25T14:06:30.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.800+0000 7f9d9e200640 1 --2- 172.21.15.86:0/677682170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d98071530 0x7f9d9815dc90 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:30.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.800+0000 7f9d9e200640 1 -- 172.21.15.86:0/677682170 >> 172.21.15.86:0/677682170 conn(0x7f9d980cffc0 msgr2=0x7f9d980d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:30.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.800+0000 7f9d9e200640 1 -- 172.21.15.86:0/677682170 shutdown_connections 2024-04-25T14:06:30.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.800+0000 7f9d9e200640 1 -- 172.21.15.86:0/677682170 wait complete. 2024-04-25T14:06:30.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.804+0000 7f9d9e200640 1 Processor -- start 2024-04-25T14:06:30.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.804+0000 7f9d9e200640 1 -- start start 2024-04-25T14:06:30.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.804+0000 7f9d9e200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d980d2c80 0x7f9d9814e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:30.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.804+0000 7f9d9e200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d981616e0 0x7f9d9815e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:30.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.804+0000 7f9d9e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d981527e0 0x7f9d9815e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:30.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.804+0000 7f9d9e200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d980d2480 con 0x7f9d981527e0 2024-04-25T14:06:30.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.804+0000 7f9d9e200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9d980d21a0 con 0x7f9d981616e0 2024-04-25T14:06:30.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.804+0000 7f9d9e200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9d980d29a0 con 0x7f9d980d2c80 2024-04-25T14:06:30.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.804+0000 7f9d97e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d981527e0 0x7f9d9815e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:30.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.804+0000 7f9d97e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d981527e0 0x7f9d9815e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49252/0 (socket says 172.21.15.86:49252) 2024-04-25T14:06:30.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.804+0000 7f9d97e00640 1 -- 172.21.15.86:0/1831000520 learned_addr learned my addr 172.21.15.86:0/1831000520 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:30.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.804+0000 7f9d96a00640 1 --2- 172.21.15.86:0/1831000520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d981616e0 0x7f9d9815e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:30.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.804+0000 7f9d97e00640 1 -- 172.21.15.86:0/1831000520 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d980d2c80 msgr2=0x7f9d9814e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:30.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.804+0000 7f9d97400640 1 --2- 172.21.15.86:0/1831000520 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d980d2c80 0x7f9d9814e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:30.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.804+0000 7f9d97e00640 1 --2- 172.21.15.86:0/1831000520 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d980d2c80 0x7f9d9814e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:30.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.804+0000 7f9d97e00640 1 -- 172.21.15.86:0/1831000520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d981616e0 msgr2=0x7f9d9815e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:30.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.804+0000 7f9d97e00640 1 --2- 172.21.15.86:0/1831000520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d981616e0 0x7f9d9815e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:30.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.804+0000 7f9d97e00640 1 -- 172.21.15.86:0/1831000520 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d98160bb0 con 0x7f9d981527e0 2024-04-25T14:06:30.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.804+0000 7f9d97400640 1 --2- 172.21.15.86:0/1831000520 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d980d2c80 0x7f9d9814e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:30.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.804+0000 7f9d97e00640 1 --2- 172.21.15.86:0/1831000520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d981527e0 0x7f9d9815e850 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000cad0 tx=0x7f9d9000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:30.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.808+0000 7f9d8f400640 1 -- 172.21.15.86:0/1831000520 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90017070 con 0x7f9d981527e0 2024-04-25T14:06:30.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.808+0000 7f9d8f400640 1 -- 172.21.15.86:0/1831000520 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d9000ac30 con 0x7f9d981527e0 2024-04-25T14:06:30.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.808+0000 7f9d9e200640 1 -- 172.21.15.86:0/1831000520 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d981558f0 con 0x7f9d981527e0 2024-04-25T14:06:30.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.808+0000 7f9d8f400640 1 -- 172.21.15.86:0/1831000520 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90012400 con 0x7f9d981527e0 2024-04-25T14:06:30.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.808+0000 7f9d9e200640 1 -- 172.21.15.86:0/1831000520 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d98160840 con 0x7f9d981527e0 2024-04-25T14:06:30.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.808+0000 7f9d9e200640 1 -- 172.21.15.86:0/1831000520 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d980720e0 con 0x7f9d981527e0 2024-04-25T14:06:30.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.808+0000 7f9d8f400640 1 -- 172.21.15.86:0/1831000520 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f9d9001a070 con 0x7f9d981527e0 2024-04-25T14:06:30.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.808+0000 7f9d8f400640 1 --2- 172.21.15.86:0/1831000520 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f9d7806f640 0x7f9d78071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:30.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.808+0000 7f9d8f400640 1 -- 172.21.15.86:0/1831000520 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 7121+0+0 (secure 0 0 0) 0x7f9d9008f900 con 0x7f9d981527e0 2024-04-25T14:06:30.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.808+0000 7f9d97400640 1 --2- 172.21.15.86:0/1831000520 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f9d7806f640 0x7f9d78071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:30.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.812+0000 7f9d97400640 1 --2- 172.21.15.86:0/1831000520 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f9d7806f640 0x7f9d78071b50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9d88001650 tx=0x7f9d88009040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:30.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.816+0000 7f9d8f400640 1 -- 172.21.15.86:0/1831000520 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9d9005b810 con 0x7f9d981527e0 2024-04-25T14:06:30.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.972+0000 7f9d9e200640 1 -- 172.21.15.86:0/1831000520 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d98155680 con 0x7f9d981527e0 2024-04-25T14:06:30.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.972+0000 7f9d8f400640 1 -- 172.21.15.86:0/1831000520 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+18395 (secure 0 0 0) 0x7f9d9005fb10 con 0x7f9d981527e0 2024-04-25T14:06:30.975 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:30.975 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":62,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:06:28.832848+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"fs2_metadata","create_time":"2024-04-25T14:05:52.841764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"fs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"fs2_data","create_time":"2024-04-25T14:05:53.524454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"fs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6835/2169924270":"2024-04-26T14:06:25.821769+0000","172.21.15.86:6834/2169924270":"2024-04-26T14:06:25.821769+0000","172.21.15.86:0/145108428":"2024-04-26T14:06:19.409429+0000","172.21.15.86:6833/2427179681":"2024-04-26T14:06:19.409429+0000","172.21.15.86:6832/2427179681":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/1321309855":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/525010030":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/3470358079":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/1177884430":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/3644218120":"2024-04-26T14:06:19.409429+0000","172.21.15.86:6836/1732615622":"2024-04-26T14:06:00.230367+0000","172.21.15.86:6837/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:6837/2418778265":"2024-04-26T14:06:00.222915+0000","172.21.15.86:0/1843278167":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6832/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/2073988773":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6835/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:0/2602032945":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/3558294881":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6835/615772001":"2024-04-26T14:05:42.882867+0000","172.21.15.86:6837/615772001":"2024-04-26T14:05:42.882867+0000","172.21.15.86:6833/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6835/2418778265":"2024-04-26T14:06:00.222915+0000","172.21.15.86:6834/1732615622":"2024-04-26T14:06:00.230367+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:06:30.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.976+0000 7f9d9e200640 1 -- 172.21.15.86:0/1831000520 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f9d7806f640 msgr2=0x7f9d78071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:30.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.976+0000 7f9d9e200640 1 --2- 172.21.15.86:0/1831000520 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f9d7806f640 0x7f9d78071b50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9d88001650 tx=0x7f9d88009040 comp rx=0 tx=0).stop 2024-04-25T14:06:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.976+0000 7f9d9e200640 1 -- 172.21.15.86:0/1831000520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d981527e0 msgr2=0x7f9d9815e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.976+0000 7f9d9e200640 1 --2- 172.21.15.86:0/1831000520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d981527e0 0x7f9d9815e850 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000cad0 tx=0x7f9d9000cf90 comp rx=0 tx=0).stop 2024-04-25T14:06:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.976+0000 7f9d9e200640 1 -- 172.21.15.86:0/1831000520 shutdown_connections 2024-04-25T14:06:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.976+0000 7f9d9e200640 1 --2- 172.21.15.86:0/1831000520 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f9d7806f640 0x7f9d78071b50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.976+0000 7f9d9e200640 1 --2- 172.21.15.86:0/1831000520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d981527e0 0x7f9d9815e850 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.976+0000 7f9d9e200640 1 --2- 172.21.15.86:0/1831000520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d981616e0 0x7f9d9815e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.976+0000 7f9d9e200640 1 --2- 172.21.15.86:0/1831000520 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d980d2c80 0x7f9d9814e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.976+0000 7f9d9e200640 1 -- 172.21.15.86:0/1831000520 >> 172.21.15.86:0/1831000520 conn(0x7f9d980cffc0 msgr2=0x7f9d98159d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.976+0000 7f9d9e200640 1 -- 172.21.15.86:0/1831000520 shutdown_connections 2024-04-25T14:06:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:30.976+0000 7f9d9e200640 1 -- 172.21.15.86:0/1831000520 wait complete. 2024-04-25T14:06:30.998 INFO:tasks.cephfs.filesystem:Destroying file system fs2 and related pools 2024-04-25T14:06:30.998 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:06:31.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.140+0000 7f572fc00640 1 Processor -- start 2024-04-25T14:06:31.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.140+0000 7f572fc00640 1 -- start start 2024-04-25T14:06:31.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f572fc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5728070330 0x7f57280d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f572fc00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f57280ce790 0x7f572806f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f572fc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57280d2030 con 0x7f57280d1c20 2024-04-25T14:06:31.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f572fc00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f572815e5c0 con 0x7f57280ce790 2024-04-25T14:06:31.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f572fc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f572815e8a0 con 0x7f5728070330 2024-04-25T14:06:31.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f571fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f57280d1c20 0x7f572806ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59992/0 (socket says 172.21.15.86:59992) 2024-04-25T14:06:31.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f571fe00640 1 -- 172.21.15.86:0/2128522331 learned_addr learned my addr 172.21.15.86:0/2128522331 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:31.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f572d200640 1 --2- 172.21.15.86:0/2128522331 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5728070330 0x7f57280d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f572dc00640 1 --2- 172.21.15.86:0/2128522331 >> v2:172.21.15.155:3300/0 conn(0x7f57280ce790 0x7f572806f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f5727e00640 1 -- 172.21.15.86:0/2128522331 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2758041124 0 0) 0x7f5710001120 con 0x7f57280d1c20 2024-04-25T14:06:31.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f5727e00640 1 -- 172.21.15.86:0/2128522331 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f570c003530 con 0x7f57280d1c20 2024-04-25T14:06:31.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f5727e00640 1 -- 172.21.15.86:0/2128522331 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 195401148 0 0) 0x7f570c003530 con 0x7f57280d1c20 2024-04-25T14:06:31.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f5727e00640 1 -- 172.21.15.86:0/2128522331 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5728070330 msgr2=0x7f57280d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:31.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f5727e00640 1 --2- 172.21.15.86:0/2128522331 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5728070330 0x7f57280d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f5727e00640 1 -- 172.21.15.86:0/2128522331 >> v2:172.21.15.155:3300/0 conn(0x7f57280ce790 msgr2=0x7f572806f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:31.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f5727e00640 1 --2- 172.21.15.86:0/2128522331 >> v2:172.21.15.155:3300/0 conn(0x7f57280ce790 0x7f572806f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f5727e00640 1 -- 172.21.15.86:0/2128522331 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f572815eb80 con 0x7f57280d1c20 2024-04-25T14:06:31.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f5727e00640 1 -- 172.21.15.86:0/2128522331 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f5710001850 con 0x7f57280d1c20 2024-04-25T14:06:31.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f5727e00640 1 -- 172.21.15.86:0/2128522331 >> v1:172.21.15.86:6789/0 conn(0x7f57280d1c20 legacy=0x7f572806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:31.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f5727e00640 1 --2- 172.21.15.86:0/2128522331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f570c003a30 0x7f570c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f5727e00640 1 --2- 172.21.15.86:0/2128522331 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5728070330 0x7f570c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f5727e00640 1 --2- 172.21.15.86:0/2128522331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f570c004d50 0x7f570c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f5727e00640 1 -- 172.21.15.86:0/2128522331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f572815e5c0 con 0x7f570c003a30 2024-04-25T14:06:31.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f5727e00640 1 -- 172.21.15.86:0/2128522331 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f572815e8a0 con 0x7f570c004d50 2024-04-25T14:06:31.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f5727e00640 1 -- 172.21.15.86:0/2128522331 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f570c00bb00 con 0x7f5728070330 2024-04-25T14:06:31.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f572d200640 1 --2- 172.21.15.86:0/2128522331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f570c003a30 0x7f570c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f571fe00640 1 --2- 172.21.15.86:0/2128522331 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5728070330 0x7f570c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f572dc00640 1 --2- 172.21.15.86:0/2128522331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f570c004d50 0x7f570c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f572d200640 1 -- 172.21.15.86:0/2128522331 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5728070330 msgr2=0x7f570c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:31.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f572d200640 1 --2- 172.21.15.86:0/2128522331 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5728070330 0x7f570c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f572d200640 1 -- 172.21.15.86:0/2128522331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f570c004d50 msgr2=0x7f570c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:31.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f572d200640 1 --2- 172.21.15.86:0/2128522331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f570c004d50 0x7f570c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.144+0000 7f572d200640 1 -- 172.21.15.86:0/2128522331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570c003530 con 0x7f570c003a30 2024-04-25T14:06:31.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.148+0000 7f572d200640 1 --2- 172.21.15.86:0/2128522331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f570c003a30 0x7f570c003ea0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f5718009a90 tx=0x7f571801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac52ba9625821ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:31.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.148+0000 7f5727e00640 1 -- 172.21.15.86:0/2128522331 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5718009690 con 0x7f570c003a30 2024-04-25T14:06:31.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.148+0000 7f5727e00640 1 -- 172.21.15.86:0/2128522331 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57180a6430 con 0x7f570c003a30 2024-04-25T14:06:31.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.148+0000 7f5727e00640 1 -- 172.21.15.86:0/2128522331 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57180a6800 con 0x7f570c003a30 2024-04-25T14:06:31.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.148+0000 7f572fc00640 1 -- 172.21.15.86:0/2128522331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f570c003a30 msgr2=0x7f570c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:31.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.148+0000 7f572fc00640 1 --2- 172.21.15.86:0/2128522331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f570c003a30 0x7f570c003ea0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f5718009a90 tx=0x7f571801ca50 comp rx=0 tx=0).stop 2024-04-25T14:06:31.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.148+0000 7f572dc00640 1 -- 172.21.15.86:0/2128522331 reap_dead start 2024-04-25T14:06:31.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.148+0000 7f572fc00640 1 -- 172.21.15.86:0/2128522331 shutdown_connections 2024-04-25T14:06:31.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.148+0000 7f572fc00640 1 -- 172.21.15.86:0/2128522331 >> 172.21.15.86:0/2128522331 conn(0x7f57280b4650 msgr2=0x7f57280b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:31.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.148+0000 7f572fc00640 1 -- 172.21.15.86:0/2128522331 shutdown_connections 2024-04-25T14:06:31.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.148+0000 7f572fc00640 1 -- 172.21.15.86:0/2128522331 wait complete. 2024-04-25T14:06:31.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.152+0000 7f572fc00640 1 Processor -- start 2024-04-25T14:06:31.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.152+0000 7f572fc00640 1 -- start start 2024-04-25T14:06:31.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.152+0000 7f572fc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f570c003a30 0x7f5728158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.152+0000 7f572fc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57280ae6d0 0x7f572814de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.152+0000 7f572fc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57280ca180 0x7f57281601a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.152+0000 7f572fc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57280708c0 con 0x7f57280ae6d0 2024-04-25T14:06:31.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.152+0000 7f572fc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5728070740 con 0x7f570c003a30 2024-04-25T14:06:31.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.152+0000 7f572fc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5728168fb0 con 0x7f57280ca180 2024-04-25T14:06:31.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.152+0000 7f571fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57280ae6d0 0x7f572814de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.152+0000 7f571fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57280ae6d0 0x7f572814de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49260/0 (socket says 172.21.15.86:49260) 2024-04-25T14:06:31.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.152+0000 7f571fe00640 1 -- 172.21.15.86:0/2439764022 learned_addr learned my addr 172.21.15.86:0/2439764022 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:31.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.152+0000 7f572d200640 1 --2- 172.21.15.86:0/2439764022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f570c003a30 0x7f5728158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.152+0000 7f572dc00640 1 --2- 172.21.15.86:0/2439764022 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57280ca180 0x7f57281601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.152+0000 7f571fe00640 1 -- 172.21.15.86:0/2439764022 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57280ca180 msgr2=0x7f57281601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:31.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.152+0000 7f571fe00640 1 --2- 172.21.15.86:0/2439764022 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57280ca180 0x7f57281601a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.152+0000 7f571fe00640 1 -- 172.21.15.86:0/2439764022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f570c003a30 msgr2=0x7f5728158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:31.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.152+0000 7f571fe00640 1 --2- 172.21.15.86:0/2439764022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f570c003a30 0x7f5728158560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.152+0000 7f571fe00640 1 -- 172.21.15.86:0/2439764022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57280c93c0 con 0x7f57280ae6d0 2024-04-25T14:06:31.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.152+0000 7f571fe00640 1 --2- 172.21.15.86:0/2439764022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57280ae6d0 0x7f572814de80 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f5710007290 tx=0x7f5710007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:31.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.152+0000 7f5725600640 1 -- 172.21.15.86:0/2439764022 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710007e40 con 0x7f57280ae6d0 2024-04-25T14:06:31.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.152+0000 7f5725600640 1 -- 172.21.15.86:0/2439764022 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5710018070 con 0x7f57280ae6d0 2024-04-25T14:06:31.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.152+0000 7f572fc00640 1 -- 172.21.15.86:0/2439764022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57280c96a0 con 0x7f57280ae6d0 2024-04-25T14:06:31.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.152+0000 7f5725600640 1 -- 172.21.15.86:0/2439764022 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571000fae0 con 0x7f57280ae6d0 2024-04-25T14:06:31.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.156+0000 7f572fc00640 1 -- 172.21.15.86:0/2439764022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57281606e0 con 0x7f57280ae6d0 2024-04-25T14:06:31.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.156+0000 7f572fc00640 1 -- 172.21.15.86:0/2439764022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5728160970 con 0x7f57280ae6d0 2024-04-25T14:06:31.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.156+0000 7f5725600640 1 -- 172.21.15.86:0/2439764022 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f571000fc80 con 0x7f57280ae6d0 2024-04-25T14:06:31.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.156+0000 7f5725600640 1 --2- 172.21.15.86:0/2439764022 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f570406f670 0x7f5704071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.156+0000 7f5725600640 1 -- 172.21.15.86:0/2439764022 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 7121+0+0 (secure 0 0 0) 0x7f571008fbb0 con 0x7f57280ae6d0 2024-04-25T14:06:31.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.156+0000 7f572d200640 1 --2- 172.21.15.86:0/2439764022 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f570406f670 0x7f5704071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.160+0000 7f572d200640 1 --2- 172.21.15.86:0/2439764022 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f570406f670 0x7f5704071b80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f57180ad000 tx=0x7f57180b2040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:31.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.160+0000 7f5725600640 1 -- 172.21.15.86:0/2439764022 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f57280d2030 con 0x7f57280ae6d0 2024-04-25T14:06:31.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.352+0000 7f572fc00640 1 -- 172.21.15.86:0/2439764022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5728069d00 con 0x7f57280ae6d0 2024-04-25T14:06:31.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.352+0000 7f5725600640 1 -- 172.21.15.86:0/2439764022 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5184 (secure 0 0 0) 0x7f571005c240 con 0x7f57280ae6d0 2024-04-25T14:06:31.353 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:31.353 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":45,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3545772170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3545772170},{"type":"v1","addr":"172.21.15.155:6834","nonce":3545772170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5980,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/766530004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":766530004},{"type":"v1","addr":"172.21.15.86:6837","nonce":766530004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6414,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2676840983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2676840983},{"type":"v1","addr":"172.21.15.86:6835","nonce":2676840983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:54.565528+0000","modified":"2024-04-25T14:06:03.253173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6036":{"gid":6036,"name":"d","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6835/2914930843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2914930843},{"type":"v1","addr":"172.21.15.155:6835","nonce":2914930843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"fs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6036,"qdb_cluster":[6036]},"id":4}]} 2024-04-25T14:06:31.354 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 45 2024-04-25T14:06:31.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.352+0000 7f572fc00640 1 -- 172.21.15.86:0/2439764022 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f570406f670 msgr2=0x7f5704071b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:31.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.352+0000 7f572fc00640 1 --2- 172.21.15.86:0/2439764022 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f570406f670 0x7f5704071b80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f57180ad000 tx=0x7f57180b2040 comp rx=0 tx=0).stop 2024-04-25T14:06:31.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.352+0000 7f572fc00640 1 -- 172.21.15.86:0/2439764022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57280ae6d0 msgr2=0x7f572814de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:31.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.352+0000 7f572fc00640 1 --2- 172.21.15.86:0/2439764022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57280ae6d0 0x7f572814de80 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f5710007290 tx=0x7f5710007750 comp rx=0 tx=0).stop 2024-04-25T14:06:31.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.352+0000 7f572fc00640 1 -- 172.21.15.86:0/2439764022 shutdown_connections 2024-04-25T14:06:31.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.352+0000 7f572fc00640 1 --2- 172.21.15.86:0/2439764022 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f570406f670 0x7f5704071b80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.352+0000 7f572fc00640 1 --2- 172.21.15.86:0/2439764022 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57280ca180 0x7f57281601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.352+0000 7f572fc00640 1 --2- 172.21.15.86:0/2439764022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57280ae6d0 0x7f572814de80 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.352+0000 7f572fc00640 1 --2- 172.21.15.86:0/2439764022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f570c003a30 0x7f5728158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.352+0000 7f572fc00640 1 -- 172.21.15.86:0/2439764022 >> 172.21.15.86:0/2439764022 conn(0x7f57280b4650 msgr2=0x7f572815b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:31.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.352+0000 7f572fc00640 1 -- 172.21.15.86:0/2439764022 shutdown_connections 2024-04-25T14:06:31.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.352+0000 7f572fc00640 1 -- 172.21.15.86:0/2439764022 wait complete. 2024-04-25T14:06:31.375 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:06:31.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.520+0000 7f3e66600640 1 Processor -- start 2024-04-25T14:06:31.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.520+0000 7f3e66600640 1 -- start start 2024-04-25T14:06:31.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.524+0000 7f3e66600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e60070330 0x7f3e600d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.524+0000 7f3e66600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3e600ce790 0x7f3e6006f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.524+0000 7f3e66600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e600d2030 con 0x7f3e600d1c20 2024-04-25T14:06:31.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.524+0000 7f3e66600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3e6015e5c0 con 0x7f3e600ce790 2024-04-25T14:06:31.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.524+0000 7f3e66600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3e6015e8a0 con 0x7f3e60070330 2024-04-25T14:06:31.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.524+0000 7f3e5fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3e600ce790 0x7f3e6006f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.524+0000 7f3e5fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3e600ce790 0x7f3e6006f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49264/0 (socket says 172.21.15.86:49264) 2024-04-25T14:06:31.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.524+0000 7f3e5fe00640 1 -- 172.21.15.86:0/539513542 learned_addr learned my addr 172.21.15.86:0/539513542 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:31.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.524+0000 7f3e56a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3e600d1c20 0x7f3e6006ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60008/0 (socket says 172.21.15.86:60008) 2024-04-25T14:06:31.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.524+0000 7f3e5f400640 1 --2- 172.21.15.86:0/539513542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e60070330 0x7f3e600d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.524+0000 7f3e5fe00640 1 -- 172.21.15.86:0/539513542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e60070330 msgr2=0x7f3e600d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:31.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.524+0000 7f3e5fe00640 1 --2- 172.21.15.86:0/539513542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e60070330 0x7f3e600d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.524+0000 7f3e5fe00640 1 -- 172.21.15.86:0/539513542 >> v1:172.21.15.86:6789/0 conn(0x7f3e600d1c20 legacy=0x7f3e6006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:31.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.524+0000 7f3e5ea00640 1 -- 172.21.15.86:0/539513542 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 730510390 0 0) 0x7f3e48001120 con 0x7f3e600d1c20 2024-04-25T14:06:31.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.524+0000 7f3e5fe00640 1 -- 172.21.15.86:0/539513542 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e6015eb80 con 0x7f3e600ce790 2024-04-25T14:06:31.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.524+0000 7f3e5f400640 1 --2- 172.21.15.86:0/539513542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e60070330 0x7f3e600d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:31.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.524+0000 7f3e5fe00640 1 --2- 172.21.15.86:0/539513542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e600ce790 0x7f3e6006f440 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f3e580098a0 tx=0x7f3e5801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22458813fcbc383c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:31.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.524+0000 7f3e5ea00640 1 -- 172.21.15.86:0/539513542 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e580b0070 con 0x7f3e600ce790 2024-04-25T14:06:31.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.528+0000 7f3e5ea00640 1 -- 172.21.15.86:0/539513542 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e580ab600 con 0x7f3e600ce790 2024-04-25T14:06:31.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.528+0000 7f3e5ea00640 1 -- 172.21.15.86:0/539513542 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e580ab9d0 con 0x7f3e600ce790 2024-04-25T14:06:31.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.528+0000 7f3e66600640 1 -- 172.21.15.86:0/539513542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e600ce790 msgr2=0x7f3e6006f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:31.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.528+0000 7f3e66600640 1 --2- 172.21.15.86:0/539513542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e600ce790 0x7f3e6006f440 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f3e580098a0 tx=0x7f3e5801c4b0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.528+0000 7f3e66600640 1 -- 172.21.15.86:0/539513542 shutdown_connections 2024-04-25T14:06:31.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.528+0000 7f3e66600640 1 --2- 172.21.15.86:0/539513542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e600ce790 0x7f3e6006f440 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.528+0000 7f3e66600640 1 --2- 172.21.15.86:0/539513542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e60070330 0x7f3e600d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.528+0000 7f3e66600640 1 -- 172.21.15.86:0/539513542 >> 172.21.15.86:0/539513542 conn(0x7f3e600b4650 msgr2=0x7f3e600b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:31.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.528+0000 7f3e66600640 1 -- 172.21.15.86:0/539513542 shutdown_connections 2024-04-25T14:06:31.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.528+0000 7f3e66600640 1 -- 172.21.15.86:0/539513542 wait complete. 2024-04-25T14:06:31.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.528+0000 7f3e66600640 1 Processor -- start 2024-04-25T14:06:31.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.528+0000 7f3e66600640 1 -- start start 2024-04-25T14:06:31.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.532+0000 7f3e66600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e60070330 0x7f3e600c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.532+0000 7f3e66600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e600ce790 0x7f3e600c3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.532+0000 7f3e66600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e600d1c20 0x7f3e6015c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.532+0000 7f3e66600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e600708c0 con 0x7f3e600d1c20 2024-04-25T14:06:31.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.532+0000 7f3e66600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3e60070740 con 0x7f3e600ce790 2024-04-25T14:06:31.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.532+0000 7f3e66600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3e6016afc0 con 0x7f3e60070330 2024-04-25T14:06:31.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.532+0000 7f3e5fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e600d1c20 0x7f3e6015c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.532+0000 7f3e5fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e600d1c20 0x7f3e6015c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49278/0 (socket says 172.21.15.86:49278) 2024-04-25T14:06:31.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.532+0000 7f3e5fe00640 1 -- 172.21.15.86:0/4195038406 learned_addr learned my addr 172.21.15.86:0/4195038406 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:31.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.532+0000 7f3e56a00640 1 --2- 172.21.15.86:0/4195038406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e600ce790 0x7f3e600c3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.532+0000 7f3e5f400640 1 --2- 172.21.15.86:0/4195038406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e60070330 0x7f3e600c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.532+0000 7f3e5fe00640 1 -- 172.21.15.86:0/4195038406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e60070330 msgr2=0x7f3e600c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:31.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.532+0000 7f3e5fe00640 1 --2- 172.21.15.86:0/4195038406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e60070330 0x7f3e600c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.532+0000 7f3e5fe00640 1 -- 172.21.15.86:0/4195038406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e600ce790 msgr2=0x7f3e600c3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:31.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.532+0000 7f3e5fe00640 1 --2- 172.21.15.86:0/4195038406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e600ce790 0x7f3e600c3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.532+0000 7f3e5fe00640 1 -- 172.21.15.86:0/4195038406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e600c49f0 con 0x7f3e600d1c20 2024-04-25T14:06:31.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.532+0000 7f3e5f400640 1 --2- 172.21.15.86:0/4195038406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e60070330 0x7f3e600c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:31.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.532+0000 7f3e5fe00640 1 --2- 172.21.15.86:0/4195038406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e600d1c20 0x7f3e6015c9d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3e580098a0 tx=0x7f3e5801acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:31.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.532+0000 7f3e57e00640 1 -- 172.21.15.86:0/4195038406 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e580bc070 con 0x7f3e600d1c20 2024-04-25T14:06:31.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.532+0000 7f3e57e00640 1 -- 172.21.15.86:0/4195038406 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e58004030 con 0x7f3e600d1c20 2024-04-25T14:06:31.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.532+0000 7f3e66600640 1 -- 172.21.15.86:0/4195038406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e600c4cd0 con 0x7f3e600d1c20 2024-04-25T14:06:31.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.532+0000 7f3e57e00640 1 -- 172.21.15.86:0/4195038406 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e580b0040 con 0x7f3e600d1c20 2024-04-25T14:06:31.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.532+0000 7f3e66600640 1 -- 172.21.15.86:0/4195038406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e600c5290 con 0x7f3e600d1c20 2024-04-25T14:06:31.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.532+0000 7f3e66600640 1 -- 172.21.15.86:0/4195038406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e60069d00 con 0x7f3e600d1c20 2024-04-25T14:06:31.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.536+0000 7f3e57e00640 1 -- 172.21.15.86:0/4195038406 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f3e58003620 con 0x7f3e600d1c20 2024-04-25T14:06:31.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.536+0000 7f3e57e00640 1 --2- 172.21.15.86:0/4195038406 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3e3c06f670 0x7f3e3c071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.536+0000 7f3e57e00640 1 -- 172.21.15.86:0/4195038406 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 7121+0+0 (secure 0 0 0) 0x7f3e58129370 con 0x7f3e600d1c20 2024-04-25T14:06:31.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.536+0000 7f3e5f400640 1 --2- 172.21.15.86:0/4195038406 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3e3c06f670 0x7f3e3c071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.536+0000 7f3e5f400640 1 --2- 172.21.15.86:0/4195038406 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3e3c06f670 0x7f3e3c071b80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009540 tx=0x7f3e5000b040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:31.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.540+0000 7f3e57e00640 1 -- 172.21.15.86:0/4195038406 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3e58016130 con 0x7f3e600d1c20 2024-04-25T14:06:31.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.728+0000 7f3e66600640 1 -- 172.21.15.86:0/4195038406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e60068d70 con 0x7f3e600d1c20 2024-04-25T14:06:31.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.728+0000 7f3e57e00640 1 -- 172.21.15.86:0/4195038406 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5184 (secure 0 0 0) 0x7f3e580f5890 con 0x7f3e600d1c20 2024-04-25T14:06:31.731 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:31.731 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":45,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3545772170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3545772170},{"type":"v1","addr":"172.21.15.155:6834","nonce":3545772170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5980,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/766530004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":766530004},{"type":"v1","addr":"172.21.15.86:6837","nonce":766530004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6414,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2676840983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2676840983},{"type":"v1","addr":"172.21.15.86:6835","nonce":2676840983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:05:54.565528+0000","modified":"2024-04-25T14:06:03.253173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6036":{"gid":6036,"name":"d","rank":0,"incarnation":40,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6835/2914930843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2914930843},{"type":"v1","addr":"172.21.15.155:6835","nonce":2914930843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"fs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6036,"qdb_cluster":[6036]},"id":4}]} 2024-04-25T14:06:31.731 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 45 2024-04-25T14:06:31.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.732+0000 7f3e66600640 1 -- 172.21.15.86:0/4195038406 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3e3c06f670 msgr2=0x7f3e3c071b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:31.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.732+0000 7f3e66600640 1 --2- 172.21.15.86:0/4195038406 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3e3c06f670 0x7f3e3c071b80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009540 tx=0x7f3e5000b040 comp rx=0 tx=0).stop 2024-04-25T14:06:31.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.732+0000 7f3e66600640 1 -- 172.21.15.86:0/4195038406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e600d1c20 msgr2=0x7f3e6015c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:31.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.732+0000 7f3e66600640 1 --2- 172.21.15.86:0/4195038406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e600d1c20 0x7f3e6015c9d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3e580098a0 tx=0x7f3e5801acf0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.732+0000 7f3e66600640 1 -- 172.21.15.86:0/4195038406 shutdown_connections 2024-04-25T14:06:31.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.732+0000 7f3e66600640 1 --2- 172.21.15.86:0/4195038406 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3e3c06f670 0x7f3e3c071b80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.732+0000 7f3e66600640 1 --2- 172.21.15.86:0/4195038406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e600d1c20 0x7f3e6015c9d0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.732+0000 7f3e66600640 1 --2- 172.21.15.86:0/4195038406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e600ce790 0x7f3e600c3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.732+0000 7f3e66600640 1 --2- 172.21.15.86:0/4195038406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e60070330 0x7f3e600c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.732+0000 7f3e66600640 1 -- 172.21.15.86:0/4195038406 >> 172.21.15.86:0/4195038406 conn(0x7f3e600b4650 msgr2=0x7f3e600cabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:31.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.732+0000 7f3e66600640 1 -- 172.21.15.86:0/4195038406 shutdown_connections 2024-04-25T14:06:31.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.732+0000 7f3e66600640 1 -- 172.21.15.86:0/4195038406 wait complete. 2024-04-25T14:06:31.753 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:06:31.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.900+0000 7f198b200640 1 Processor -- start 2024-04-25T14:06:31.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.900+0000 7f198b200640 1 -- start start 2024-04-25T14:06:31.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.900+0000 7f198b200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f198c1472c0 0x7f198c06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.900+0000 7f198b200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f198c0706a0 con 0x7f198c0cde30 2024-04-25T14:06:31.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.900+0000 7f198b200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f198c070980 con 0x7f198c06c670 2024-04-25T14:06:31.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.900+0000 7f198b200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f198c070c60 con 0x7f198c1472c0 2024-04-25T14:06:31.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.900+0000 7f1989400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f198c0cde30 0x7f198c0ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60014/0 (socket says 172.21.15.86:60014) 2024-04-25T14:06:31.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.900+0000 7f1989400640 1 -- 172.21.15.86:0/1563177968 learned_addr learned my addr 172.21.15.86:0/1563177968 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:31.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.900+0000 7f1989e00640 1 --2- 172.21.15.86:0/1563177968 >> v2:172.21.15.155:3300/0 conn(0x7f198c1472c0 0x7f198c06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.900+0000 7f1988a00640 1 -- 172.21.15.86:0/1563177968 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2157657073 0 0) 0x7f198c0706a0 con 0x7f198c0cde30 2024-04-25T14:06:31.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.900+0000 7f1988a00640 1 -- 172.21.15.86:0/1563177968 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1970003530 con 0x7f198c0cde30 2024-04-25T14:06:31.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.900+0000 7f1988a00640 1 -- 172.21.15.86:0/1563177968 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1305741241 0 0) 0x7f1970003530 con 0x7f198c0cde30 2024-04-25T14:06:31.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.900+0000 7f1989e00640 1 -- 172.21.15.86:0/1563177968 >> v1:172.21.15.155:6789/0 conn(0x7f198c06c670 legacy=0x7f198c06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:31.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.900+0000 7f1989e00640 1 -- 172.21.15.86:0/1563177968 >> v1:172.21.15.86:6789/0 conn(0x7f198c0cde30 legacy=0x7f198c0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:31.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.900+0000 7f1989e00640 1 -- 172.21.15.86:0/1563177968 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198c070f40 con 0x7f198c1472c0 2024-04-25T14:06:31.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.904+0000 7f1989e00640 1 --2- 172.21.15.86:0/1563177968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f198c1472c0 0x7f198c06fda0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f197c0097c0 tx=0x7f197c01c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c41de22ffb6cfd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:31.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.904+0000 7f1988a00640 1 -- 172.21.15.86:0/1563177968 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c0b0070 con 0x7f198c1472c0 2024-04-25T14:06:31.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.904+0000 7f1988a00640 1 -- 172.21.15.86:0/1563177968 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f197c0ab400 con 0x7f198c1472c0 2024-04-25T14:06:31.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.904+0000 7f1988a00640 1 -- 172.21.15.86:0/1563177968 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1978001680 con 0x7f198c1472c0 2024-04-25T14:06:31.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.904+0000 7f198b200640 1 -- 172.21.15.86:0/1563177968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f198c1472c0 msgr2=0x7f198c06fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:31.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.904+0000 7f198b200640 1 --2- 172.21.15.86:0/1563177968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f198c1472c0 0x7f198c06fda0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f197c0097c0 tx=0x7f197c01c300 comp rx=0 tx=0).stop 2024-04-25T14:06:31.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.904+0000 7f198b200640 1 -- 172.21.15.86:0/1563177968 shutdown_connections 2024-04-25T14:06:31.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.904+0000 7f198b200640 1 --2- 172.21.15.86:0/1563177968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f198c1472c0 0x7f198c06fda0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.904+0000 7f198b200640 1 -- 172.21.15.86:0/1563177968 >> 172.21.15.86:0/1563177968 conn(0x7f198c0d1b50 msgr2=0x7f198c0d1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:31.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.904+0000 7f198b200640 1 -- 172.21.15.86:0/1563177968 shutdown_connections 2024-04-25T14:06:31.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.904+0000 7f198b200640 1 -- 172.21.15.86:0/1563177968 wait complete. 2024-04-25T14:06:31.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.908+0000 7f198b200640 1 Processor -- start 2024-04-25T14:06:31.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.908+0000 7f198b200640 1 -- start start 2024-04-25T14:06:31.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.908+0000 7f198b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f198c06c670 0x7f198c0aa490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.908+0000 7f198b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f198c0cde30 0x7f198c1fc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.908+0000 7f198b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f198c1472c0 0x7f198c143260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.908+0000 7f198b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f198c16b960 con 0x7f198c1472c0 2024-04-25T14:06:31.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.908+0000 7f198b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f198c16b7e0 con 0x7f198c06c670 2024-04-25T14:06:31.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.908+0000 7f198b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f198c16bae0 con 0x7f198c0cde30 2024-04-25T14:06:31.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.908+0000 7f198a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f198c1472c0 0x7f198c143260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.908+0000 7f198a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f198c1472c0 0x7f198c143260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49280/0 (socket says 172.21.15.86:49280) 2024-04-25T14:06:31.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.908+0000 7f198a800640 1 -- 172.21.15.86:0/3441909736 learned_addr learned my addr 172.21.15.86:0/3441909736 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:31.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.908+0000 7f1989e00640 1 --2- 172.21.15.86:0/3441909736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f198c06c670 0x7f198c0aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.908+0000 7f1989400640 1 --2- 172.21.15.86:0/3441909736 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f198c0cde30 0x7f198c1fc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.912+0000 7f198a800640 1 -- 172.21.15.86:0/3441909736 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f198c0cde30 msgr2=0x7f198c1fc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:31.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.912+0000 7f198a800640 1 --2- 172.21.15.86:0/3441909736 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f198c0cde30 0x7f198c1fc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.912+0000 7f198a800640 1 -- 172.21.15.86:0/3441909736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f198c06c670 msgr2=0x7f198c0aa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:31.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.912+0000 7f198a800640 1 --2- 172.21.15.86:0/3441909736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f198c06c670 0x7f198c0aa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.912+0000 7f198a800640 1 -- 172.21.15.86:0/3441909736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198c13b1f0 con 0x7f198c1472c0 2024-04-25T14:06:31.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.912+0000 7f198a800640 1 --2- 172.21.15.86:0/3441909736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f198c1472c0 0x7f198c143260 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f198400c760 tx=0x7f198400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:31.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.912+0000 7f1982000640 1 -- 172.21.15.86:0/3441909736 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1984003b80 con 0x7f198c1472c0 2024-04-25T14:06:31.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.912+0000 7f1982000640 1 -- 172.21.15.86:0/3441909736 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1984003d20 con 0x7f198c1472c0 2024-04-25T14:06:31.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.912+0000 7f1982000640 1 -- 172.21.15.86:0/3441909736 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198400aa90 con 0x7f198c1472c0 2024-04-25T14:06:31.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.912+0000 7f198b200640 1 -- 172.21.15.86:0/3441909736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f198c142780 con 0x7f198c1472c0 2024-04-25T14:06:31.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.912+0000 7f198b200640 1 -- 172.21.15.86:0/3441909736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f198c0ab0a0 con 0x7f198c1472c0 2024-04-25T14:06:31.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.912+0000 7f198b200640 1 -- 172.21.15.86:0/3441909736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f198c13af60 con 0x7f198c1472c0 2024-04-25T14:06:31.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.912+0000 7f1982000640 1 -- 172.21.15.86:0/3441909736 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f198400ac30 con 0x7f198c1472c0 2024-04-25T14:06:31.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.916+0000 7f1982000640 1 --2- 172.21.15.86:0/3441909736 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f196c06f670 0x7f196c071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.916+0000 7f1982000640 1 -- 172.21.15.86:0/3441909736 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 7121+0+0 (secure 0 0 0) 0x7f198408fb00 con 0x7f198c1472c0 2024-04-25T14:06:31.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.916+0000 7f1989e00640 1 --2- 172.21.15.86:0/3441909736 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f196c06f670 0x7f196c071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.916+0000 7f1989e00640 1 --2- 172.21.15.86:0/3441909736 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f196c06f670 0x7f196c071b80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f197c0ad000 tx=0x7f197c0a6000 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:31.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:31.920+0000 7f1982000640 1 -- 172.21.15.86:0/3441909736 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f198405c060 con 0x7f198c1472c0 2024-04-25T14:06:32.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.076+0000 7f198b200640 1 -- 172.21.15.86:0/3441909736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f198c142570 con 0x7f198c1472c0 2024-04-25T14:06:32.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.076+0000 7f1982000640 1 -- 172.21.15.86:0/3441909736 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+18395 (secure 0 0 0) 0x7f198408de20 con 0x7f198c1472c0 2024-04-25T14:06:32.078 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:32.078 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":62,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:06:28.832848+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"fs2_metadata","create_time":"2024-04-25T14:05:52.841764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"fs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"fs2_data","create_time":"2024-04-25T14:05:53.524454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"fs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6835/2169924270":"2024-04-26T14:06:25.821769+0000","172.21.15.86:6834/2169924270":"2024-04-26T14:06:25.821769+0000","172.21.15.86:0/145108428":"2024-04-26T14:06:19.409429+0000","172.21.15.86:6833/2427179681":"2024-04-26T14:06:19.409429+0000","172.21.15.86:6832/2427179681":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/1321309855":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/525010030":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/3470358079":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/1177884430":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/3644218120":"2024-04-26T14:06:19.409429+0000","172.21.15.86:6836/1732615622":"2024-04-26T14:06:00.230367+0000","172.21.15.86:6837/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:6837/2418778265":"2024-04-26T14:06:00.222915+0000","172.21.15.86:0/1843278167":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6832/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/2073988773":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6835/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:0/2602032945":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/3558294881":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6835/615772001":"2024-04-26T14:05:42.882867+0000","172.21.15.86:6837/615772001":"2024-04-26T14:05:42.882867+0000","172.21.15.86:6833/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6835/2418778265":"2024-04-26T14:06:00.222915+0000","172.21.15.86:6834/1732615622":"2024-04-26T14:06:00.230367+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:06:32.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.080+0000 7f198b200640 1 -- 172.21.15.86:0/3441909736 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f196c06f670 msgr2=0x7f196c071b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:32.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.080+0000 7f198b200640 1 --2- 172.21.15.86:0/3441909736 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f196c06f670 0x7f196c071b80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f197c0ad000 tx=0x7f197c0a6000 comp rx=0 tx=0).stop 2024-04-25T14:06:32.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.080+0000 7f198b200640 1 -- 172.21.15.86:0/3441909736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f198c1472c0 msgr2=0x7f198c143260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:32.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.080+0000 7f198b200640 1 --2- 172.21.15.86:0/3441909736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f198c1472c0 0x7f198c143260 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f198400c760 tx=0x7f198400cc20 comp rx=0 tx=0).stop 2024-04-25T14:06:32.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.080+0000 7f198b200640 1 -- 172.21.15.86:0/3441909736 shutdown_connections 2024-04-25T14:06:32.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.080+0000 7f198b200640 1 --2- 172.21.15.86:0/3441909736 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f196c06f670 0x7f196c071b80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:32.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.080+0000 7f198b200640 1 --2- 172.21.15.86:0/3441909736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f198c1472c0 0x7f198c143260 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:32.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.080+0000 7f198b200640 1 --2- 172.21.15.86:0/3441909736 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f198c0cde30 0x7f198c1fc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:32.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.080+0000 7f198b200640 1 --2- 172.21.15.86:0/3441909736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f198c06c670 0x7f198c0aa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:32.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.080+0000 7f198b200640 1 -- 172.21.15.86:0/3441909736 >> 172.21.15.86:0/3441909736 conn(0x7f198c0d1b50 msgr2=0x7f198c15ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:32.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.080+0000 7f198b200640 1 -- 172.21.15.86:0/3441909736 shutdown_connections 2024-04-25T14:06:32.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.080+0000 7f198b200640 1 -- 172.21.15.86:0/3441909736 wait complete. 2024-04-25T14:06:32.101 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail fs2 2024-04-25T14:06:32.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.248+0000 7fd5e4600640 1 Processor -- start 2024-04-25T14:06:32.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.248+0000 7fd5e4600640 1 -- start start 2024-04-25T14:06:32.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.248+0000 7fd5e4600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd5dc0cfca0 0x7fd5dc06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:32.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.248+0000 7fd5e4600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd5dc0d16e0 0x7fd5dc0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:32.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.248+0000 7fd5e4600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5dc06c670 0x7fd5dc06ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:32.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.248+0000 7fd5e4600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd5dc070470 con 0x7fd5dc0cfca0 2024-04-25T14:06:32.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.248+0000 7fd5e4600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd5dc0d2030 con 0x7fd5dc0d16e0 2024-04-25T14:06:32.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.248+0000 7fd5e4600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd5dc072bd0 con 0x7fd5dc06c670 2024-04-25T14:06:32.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.248+0000 7fd5e1c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd5dc0cfca0 0x7fd5dc06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:32.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.248+0000 7fd5e1c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd5dc0cfca0 0x7fd5dc06f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49282/0 (socket says 172.21.15.86:49282) 2024-04-25T14:06:32.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.248+0000 7fd5e1c00640 1 -- 172.21.15.86:0/659682469 learned_addr learned my addr 172.21.15.86:0/659682469 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:32.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.248+0000 7fd5e1200640 1 --2- 172.21.15.86:0/659682469 >> v2:172.21.15.155:3300/0 conn(0x7fd5dc0d16e0 0x7fd5dc0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:32.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.248+0000 7fd5e2600640 1 --2- 172.21.15.86:0/659682469 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5dc06c670 0x7fd5dc06ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:32.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.248+0000 7fd5e1c00640 1 -- 172.21.15.86:0/659682469 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5dc06c670 msgr2=0x7fd5dc06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:32.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.248+0000 7fd5e1c00640 1 --2- 172.21.15.86:0/659682469 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5dc06c670 0x7fd5dc06ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:32.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.248+0000 7fd5e1c00640 1 -- 172.21.15.86:0/659682469 >> v2:172.21.15.155:3300/0 conn(0x7fd5dc0d16e0 msgr2=0x7fd5dc0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:32.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.248+0000 7fd5e1c00640 1 --2- 172.21.15.86:0/659682469 >> v2:172.21.15.155:3300/0 conn(0x7fd5dc0d16e0 0x7fd5dc0d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:32.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.248+0000 7fd5e1c00640 1 -- 172.21.15.86:0/659682469 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5dc06f930 con 0x7fd5dc0cfca0 2024-04-25T14:06:32.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.248+0000 7fd5e1200640 1 --2- 172.21.15.86:0/659682469 >> v2:172.21.15.155:3300/0 conn(0x7fd5dc0d16e0 0x7fd5dc0d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:32.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.252+0000 7fd5e1c00640 1 --2- 172.21.15.86:0/659682469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5dc0cfca0 0x7fd5dc06f3f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0729c0 tx=0x7fd5d001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6fd8cd7245fc5b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:32.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.252+0000 7fd5dbe00640 1 -- 172.21.15.86:0/659682469 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d00ab5f0 con 0x7fd5dc0cfca0 2024-04-25T14:06:32.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.252+0000 7fd5dbe00640 1 -- 172.21.15.86:0/659682469 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5d00b0070 con 0x7fd5dc0cfca0 2024-04-25T14:06:32.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.252+0000 7fd5dbe00640 1 -- 172.21.15.86:0/659682469 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d00ab9b0 con 0x7fd5dc0cfca0 2024-04-25T14:06:32.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.252+0000 7fd5e4600640 1 -- 172.21.15.86:0/659682469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5dc0cfca0 msgr2=0x7fd5dc06f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:32.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.252+0000 7fd5e4600640 1 --2- 172.21.15.86:0/659682469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5dc0cfca0 0x7fd5dc06f3f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0729c0 tx=0x7fd5d001c520 comp rx=0 tx=0).stop 2024-04-25T14:06:32.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.252+0000 7fd5e4600640 1 -- 172.21.15.86:0/659682469 shutdown_connections 2024-04-25T14:06:32.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.252+0000 7fd5e4600640 1 --2- 172.21.15.86:0/659682469 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5dc06c670 0x7fd5dc06ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:32.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.252+0000 7fd5e4600640 1 --2- 172.21.15.86:0/659682469 >> v2:172.21.15.155:3300/0 conn(0x7fd5dc0d16e0 0x7fd5dc0d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:32.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.252+0000 7fd5e4600640 1 --2- 172.21.15.86:0/659682469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5dc0cfca0 0x7fd5dc06f3f0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:32.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.256+0000 7fd5e4600640 1 -- 172.21.15.86:0/659682469 >> 172.21.15.86:0/659682469 conn(0x7fd5dc0b4650 msgr2=0x7fd5dc0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:32.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.256+0000 7fd5e4600640 1 -- 172.21.15.86:0/659682469 shutdown_connections 2024-04-25T14:06:32.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.256+0000 7fd5e4600640 1 -- 172.21.15.86:0/659682469 wait complete. 2024-04-25T14:06:32.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.256+0000 7fd5e4600640 1 Processor -- start 2024-04-25T14:06:32.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.256+0000 7fd5e4600640 1 -- start start 2024-04-25T14:06:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.256+0000 7fd5e4600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5dc06c670 0x7fd5dc0ce380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.256+0000 7fd5e4600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5dc0cfca0 0x7fd5dc1fc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.256+0000 7fd5e4600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5dc0d16e0 0x7fd5dc15e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.256+0000 7fd5e4600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5dc0738c0 con 0x7fd5dc06c670 2024-04-25T14:06:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.256+0000 7fd5e4600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd5dc06d3d0 con 0x7fd5dc0cfca0 2024-04-25T14:06:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.256+0000 7fd5e4600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd5dc073a40 con 0x7fd5dc0d16e0 2024-04-25T14:06:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.256+0000 7fd5e1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5dc06c670 0x7fd5dc0ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.260+0000 7fd5e1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5dc06c670 0x7fd5dc0ce380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49298/0 (socket says 172.21.15.86:49298) 2024-04-25T14:06:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.260+0000 7fd5e1c00640 1 -- 172.21.15.86:0/2198914540 learned_addr learned my addr 172.21.15.86:0/2198914540 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.260+0000 7fd5e1200640 1 --2- 172.21.15.86:0/2198914540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5dc0cfca0 0x7fd5dc1fc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:32.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.260+0000 7fd5e2600640 1 --2- 172.21.15.86:0/2198914540 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5dc0d16e0 0x7fd5dc15e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:32.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.260+0000 7fd5e1c00640 1 -- 172.21.15.86:0/2198914540 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5dc0d16e0 msgr2=0x7fd5dc15e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:32.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.260+0000 7fd5e1c00640 1 --2- 172.21.15.86:0/2198914540 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5dc0d16e0 0x7fd5dc15e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:32.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.260+0000 7fd5e1c00640 1 -- 172.21.15.86:0/2198914540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5dc0cfca0 msgr2=0x7fd5dc1fc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:32.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.260+0000 7fd5e1c00640 1 --2- 172.21.15.86:0/2198914540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5dc0cfca0 0x7fd5dc1fc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:32.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.260+0000 7fd5e1c00640 1 -- 172.21.15.86:0/2198914540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5dc0cd300 con 0x7fd5dc06c670 2024-04-25T14:06:32.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.260+0000 7fd5e2600640 1 --2- 172.21.15.86:0/2198914540 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5dc0d16e0 0x7fd5dc15e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:32.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.260+0000 7fd5e1c00640 1 --2- 172.21.15.86:0/2198914540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5dc06c670 0x7fd5dc0ce380 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fd5d001ca00 tx=0x7fd5d001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:32.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.260+0000 7fd5d9600640 1 -- 172.21.15.86:0/2198914540 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d00be5f0 con 0x7fd5dc06c670 2024-04-25T14:06:32.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.260+0000 7fd5d9600640 1 -- 172.21.15.86:0/2198914540 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5d00be790 con 0x7fd5dc06c670 2024-04-25T14:06:32.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.260+0000 7fd5d9600640 1 -- 172.21.15.86:0/2198914540 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d00beb80 con 0x7fd5dc06c670 2024-04-25T14:06:32.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.260+0000 7fd5e4600640 1 -- 172.21.15.86:0/2198914540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5dc0cd5c0 con 0x7fd5dc06c670 2024-04-25T14:06:32.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.260+0000 7fd5e4600640 1 -- 172.21.15.86:0/2198914540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5dc0cdb80 con 0x7fd5dc06c670 2024-04-25T14:06:32.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.260+0000 7fd5e4600640 1 -- 172.21.15.86:0/2198914540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5dc09f4b0 con 0x7fd5dc06c670 2024-04-25T14:06:32.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.260+0000 7fd5d9600640 1 -- 172.21.15.86:0/2198914540 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7fd5d00b43f0 con 0x7fd5dc06c670 2024-04-25T14:06:32.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.264+0000 7fd5d9600640 1 --2- 172.21.15.86:0/2198914540 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fd5bc06f640 0x7fd5bc071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:32.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.264+0000 7fd5d9600640 1 -- 172.21.15.86:0/2198914540 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 7121+0+0 (secure 0 0 0) 0x7fd5d012a300 con 0x7fd5dc06c670 2024-04-25T14:06:32.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.264+0000 7fd5e1200640 1 --2- 172.21.15.86:0/2198914540 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fd5bc06f640 0x7fd5bc071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:32.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.264+0000 7fd5e1200640 1 --2- 172.21.15.86:0/2198914540 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fd5bc06f640 0x7fd5bc071b50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc15bf00 tx=0x7fd5c8008040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:32.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.268+0000 7fd5d9600640 1 -- 172.21.15.86:0/2198914540 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd5d0016110 con 0x7fd5dc06c670 2024-04-25T14:06:32.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.460+0000 7fd5e4600640 1 -- 172.21.15.86:0/2198914540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "fs2"} v 0) -- 0x7fd5dc069d00 con 0x7fd5dc06c670 2024-04-25T14:06:32.460 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-25T14:06:32.460+0000 7fea29c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:06:32.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.460+0000 7fd5d9600640 1 -- 172.21.15.86:0/2198914540 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "fs2"}]=0 fs2 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v46) ==== 155+0+0 (secure 0 0 0) 0x7fd5d00abe20 con 0x7fd5dc06c670 2024-04-25T14:06:32.462 INFO:teuthology.orchestra.run.smithi086.stderr:fs2 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T14:06:32.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.464+0000 7fd5e4600640 1 -- 172.21.15.86:0/2198914540 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fd5bc06f640 msgr2=0x7fd5bc071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:32.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.464+0000 7fd5e4600640 1 --2- 172.21.15.86:0/2198914540 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fd5bc06f640 0x7fd5bc071b50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc15bf00 tx=0x7fd5c8008040 comp rx=0 tx=0).stop 2024-04-25T14:06:32.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.464+0000 7fd5e4600640 1 -- 172.21.15.86:0/2198914540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5dc06c670 msgr2=0x7fd5dc0ce380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:32.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.464+0000 7fd5e4600640 1 --2- 172.21.15.86:0/2198914540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5dc06c670 0x7fd5dc0ce380 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fd5d001ca00 tx=0x7fd5d001ad60 comp rx=0 tx=0).stop 2024-04-25T14:06:32.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.464+0000 7fd5e4600640 1 -- 172.21.15.86:0/2198914540 shutdown_connections 2024-04-25T14:06:32.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.464+0000 7fd5e4600640 1 --2- 172.21.15.86:0/2198914540 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fd5bc06f640 0x7fd5bc071b50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:32.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.464+0000 7fd5e4600640 1 --2- 172.21.15.86:0/2198914540 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5dc0d16e0 0x7fd5dc15e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:32.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.464+0000 7fd5e4600640 1 --2- 172.21.15.86:0/2198914540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5dc0cfca0 0x7fd5dc1fc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:32.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.464+0000 7fd5e4600640 1 --2- 172.21.15.86:0/2198914540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5dc06c670 0x7fd5dc0ce380 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:32.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.464+0000 7fd5e4600640 1 -- 172.21.15.86:0/2198914540 >> 172.21.15.86:0/2198914540 conn(0x7fd5dc0b4650 msgr2=0x7fd5dc068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:32.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.464+0000 7fd5e4600640 1 -- 172.21.15.86:0/2198914540 shutdown_connections 2024-04-25T14:06:32.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.464+0000 7fd5e4600640 1 -- 172.21.15.86:0/2198914540 wait complete. 2024-04-25T14:06:32.479 INFO:tasks.ceph.mds.d.smithi155.stderr: -5975> 2024-04-25T14:06:05.237+0000 7f74a4200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T14:06:32.486 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm fs2 --yes-i-really-mean-it 2024-04-25T14:06:32.599 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2024-04-25T14:06:32.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.632+0000 7f83aae00640 1 Processor -- start 2024-04-25T14:06:32.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.636+0000 7f83aae00640 1 -- start start 2024-04-25T14:06:32.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.636+0000 7f83aae00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f83a40d2c80 0x7f83a415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:32.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.636+0000 7f83aae00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83a40cd8d0 con 0x7f83a4070e20 2024-04-25T14:06:32.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.636+0000 7f83aae00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83a40ce970 con 0x7f83a41616e0 2024-04-25T14:06:32.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.636+0000 7f83aae00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f83a40ce720 con 0x7f83a40d2c80 2024-04-25T14:06:32.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.636+0000 7f83a3e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f83a40d2c80 0x7f83a415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:32.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.636+0000 7f83a3400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f83a4070e20 0x7f83a4071230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60020/0 (socket says 172.21.15.86:60020) 2024-04-25T14:06:32.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.636+0000 7f83a3400640 1 -- 172.21.15.86:0/2563321857 learned_addr learned my addr 172.21.15.86:0/2563321857 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:32.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.636+0000 7f83a2a00640 1 -- 172.21.15.86:0/2563321857 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3908652311 0 0) 0x7f83a40cd8d0 con 0x7f83a4070e20 2024-04-25T14:06:32.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.636+0000 7f83a3e00640 1 -- 172.21.15.86:0/2563321857 >> v1:172.21.15.155:6789/0 conn(0x7f83a41616e0 legacy=0x7f83a4161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:32.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.636+0000 7f83a3e00640 1 -- 172.21.15.86:0/2563321857 >> v1:172.21.15.86:6789/0 conn(0x7f83a4070e20 legacy=0x7f83a4071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:32.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.636+0000 7f83a3e00640 1 -- 172.21.15.86:0/2563321857 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a40d1be0 con 0x7f83a40d2c80 2024-04-25T14:06:32.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.640+0000 7f83a3e00640 1 --2- 172.21.15.86:0/2563321857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83a40d2c80 0x7f83a415dc90 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f83940097c0 tx=0x7f839401ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b82b5bcf6f02bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:32.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.640+0000 7f83a2a00640 1 -- 172.21.15.86:0/2563321857 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83940b5070 con 0x7f83a40d2c80 2024-04-25T14:06:32.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.640+0000 7f83a2a00640 1 -- 172.21.15.86:0/2563321857 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83940a8990 con 0x7f83a40d2c80 2024-04-25T14:06:32.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.640+0000 7f83a2a00640 1 -- 172.21.15.86:0/2563321857 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83940a8d60 con 0x7f83a40d2c80 2024-04-25T14:06:32.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.640+0000 7f83aae00640 1 -- 172.21.15.86:0/2563321857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83a40d2c80 msgr2=0x7f83a415dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:32.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.640+0000 7f83aae00640 1 --2- 172.21.15.86:0/2563321857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83a40d2c80 0x7f83a415dc90 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f83940097c0 tx=0x7f839401ec90 comp rx=0 tx=0).stop 2024-04-25T14:06:32.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.640+0000 7f83aae00640 1 -- 172.21.15.86:0/2563321857 shutdown_connections 2024-04-25T14:06:32.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.640+0000 7f83aae00640 1 --2- 172.21.15.86:0/2563321857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83a40d2c80 0x7f83a415dc90 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:32.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.640+0000 7f83aae00640 1 -- 172.21.15.86:0/2563321857 >> 172.21.15.86:0/2563321857 conn(0x7f83a40cffc0 msgr2=0x7f83a40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:32.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.640+0000 7f83aae00640 1 -- 172.21.15.86:0/2563321857 shutdown_connections 2024-04-25T14:06:32.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.640+0000 7f83aae00640 1 -- 172.21.15.86:0/2563321857 wait complete. 2024-04-25T14:06:32.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.644+0000 7f83aae00640 1 Processor -- start 2024-04-25T14:06:32.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.644+0000 7f83aae00640 1 -- start start 2024-04-25T14:06:32.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.644+0000 7f83aae00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83a4070e20 0x7f83a414e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:32.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.644+0000 7f83aae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83a40d2c80 0x7f83a41f3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:32.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.644+0000 7f83aae00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83a41616e0 0x7f83a4153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:32.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.644+0000 7f83aae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83a40d2480 con 0x7f83a40d2c80 2024-04-25T14:06:32.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.644+0000 7f83aae00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f83a40d21a0 con 0x7f83a41616e0 2024-04-25T14:06:32.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.644+0000 7f83aae00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f83a40d29a0 con 0x7f83a4070e20 2024-04-25T14:06:32.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.644+0000 7f83a3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83a40d2c80 0x7f83a41f3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:32.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.644+0000 7f83a3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83a40d2c80 0x7f83a41f3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49322/0 (socket says 172.21.15.86:49322) 2024-04-25T14:06:32.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.644+0000 7f83a3400640 1 -- 172.21.15.86:0/818706511 learned_addr learned my addr 172.21.15.86:0/818706511 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:32.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.644+0000 7f83a3e00640 1 --2- 172.21.15.86:0/818706511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83a4070e20 0x7f83a414e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:32.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.644+0000 7f83a8e00640 1 --2- 172.21.15.86:0/818706511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83a41616e0 0x7f83a4153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:32.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.644+0000 7f83a3400640 1 -- 172.21.15.86:0/818706511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83a4070e20 msgr2=0x7f83a414e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:32.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.644+0000 7f83a3400640 1 --2- 172.21.15.86:0/818706511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83a4070e20 0x7f83a414e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:32.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.644+0000 7f83a3400640 1 -- 172.21.15.86:0/818706511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83a41616e0 msgr2=0x7f83a4153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:32.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.644+0000 7f83a3400640 1 --2- 172.21.15.86:0/818706511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83a41616e0 0x7f83a4153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:32.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.644+0000 7f83a3400640 1 -- 172.21.15.86:0/818706511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a41611d0 con 0x7f83a40d2c80 2024-04-25T14:06:32.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.644+0000 7f83a3e00640 1 --2- 172.21.15.86:0/818706511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83a4070e20 0x7f83a414e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:32.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.644+0000 7f83a8e00640 1 --2- 172.21.15.86:0/818706511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83a41616e0 0x7f83a4153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:32.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.644+0000 7f83a3400640 1 --2- 172.21.15.86:0/818706511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83a40d2c80 0x7f83a41f3f20 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f839000c3c0 tx=0x7f839000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:32.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.644+0000 7f839be00640 1 -- 172.21.15.86:0/818706511 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390017070 con 0x7f83a40d2c80 2024-04-25T14:06:32.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.644+0000 7f839be00640 1 -- 172.21.15.86:0/818706511 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8390007500 con 0x7f83a40d2c80 2024-04-25T14:06:32.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.644+0000 7f83aae00640 1 -- 172.21.15.86:0/818706511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a414ed50 con 0x7f83a40d2c80 2024-04-25T14:06:32.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.644+0000 7f839be00640 1 -- 172.21.15.86:0/818706511 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83900059f0 con 0x7f83a40d2c80 2024-04-25T14:06:32.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.644+0000 7f83aae00640 1 -- 172.21.15.86:0/818706511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a4156d40 con 0x7f83a40d2c80 2024-04-25T14:06:32.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.648+0000 7f83aae00640 1 -- 172.21.15.86:0/818706511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a40720e0 con 0x7f83a40d2c80 2024-04-25T14:06:32.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.648+0000 7f839be00640 1 -- 172.21.15.86:0/818706511 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f839001a070 con 0x7f83a40d2c80 2024-04-25T14:06:32.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.648+0000 7f839be00640 1 --2- 172.21.15.86:0/818706511 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f838406f640 0x7f8384071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:32.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.648+0000 7f839be00640 1 -- 172.21.15.86:0/818706511 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 7207+0+0 (secure 0 0 0) 0x7f839008efd0 con 0x7f83a40d2c80 2024-04-25T14:06:32.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.648+0000 7f83a3e00640 1 --2- 172.21.15.86:0/818706511 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f838406f640 0x7f8384071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:32.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.648+0000 7f83a3e00640 1 --2- 172.21.15.86:0/818706511 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f838406f640 0x7f8384071b50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f83940b2000 tx=0x7f83940aa070 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:32.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.652+0000 7f839be00640 1 -- 172.21.15.86:0/818706511 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f839005b600 con 0x7f83a40d2c80 2024-04-25T14:06:32.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:32.844+0000 7f83aae00640 1 -- 172.21.15.86:0/818706511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "fs2", "yes_i_really_mean_it": true} v 0) -- 0x7f83a4076bf0 con 0x7f83a40d2c80 2024-04-25T14:06:33.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.464+0000 7f839be00640 1 -- 172.21.15.86:0/818706511 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "fs2", "yes_i_really_mean_it": true}]=0 v47) ==== 101+0+0 (secure 0 0 0) 0x7f839005af00 con 0x7f83a40d2c80 2024-04-25T14:06:33.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.464+0000 7f83aae00640 1 -- 172.21.15.86:0/818706511 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f838406f640 msgr2=0x7f8384071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:33.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.464+0000 7f83aae00640 1 --2- 172.21.15.86:0/818706511 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f838406f640 0x7f8384071b50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f83940b2000 tx=0x7f83940aa070 comp rx=0 tx=0).stop 2024-04-25T14:06:33.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.464+0000 7f83aae00640 1 -- 172.21.15.86:0/818706511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83a40d2c80 msgr2=0x7f83a41f3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:33.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.464+0000 7f83aae00640 1 --2- 172.21.15.86:0/818706511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83a40d2c80 0x7f83a41f3f20 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f839000c3c0 tx=0x7f839000c880 comp rx=0 tx=0).stop 2024-04-25T14:06:33.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.464+0000 7f83aae00640 1 -- 172.21.15.86:0/818706511 shutdown_connections 2024-04-25T14:06:33.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.464+0000 7f83aae00640 1 --2- 172.21.15.86:0/818706511 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f838406f640 0x7f8384071b50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:33.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.464+0000 7f83aae00640 1 --2- 172.21.15.86:0/818706511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83a41616e0 0x7f83a4153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:33.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.464+0000 7f83aae00640 1 --2- 172.21.15.86:0/818706511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83a40d2c80 0x7f83a41f3f20 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:33.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.464+0000 7f83aae00640 1 --2- 172.21.15.86:0/818706511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83a4070e20 0x7f83a414e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:33.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.464+0000 7f83aae00640 1 -- 172.21.15.86:0/818706511 >> 172.21.15.86:0/818706511 conn(0x7f83a40cffc0 msgr2=0x7f83a4158e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:33.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.464+0000 7f83aae00640 1 -- 172.21.15.86:0/818706511 shutdown_connections 2024-04-25T14:06:33.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.464+0000 7f83aae00640 1 -- 172.21.15.86:0/818706511 wait complete. 2024-04-25T14:06:33.488 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm fs2_metadata fs2_metadata --yes-i-really-really-mean-it 2024-04-25T14:06:33.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.636+0000 7fd8b6800640 1 Processor -- start 2024-04-25T14:06:33.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.636+0000 7fd8b6800640 1 -- start start 2024-04-25T14:06:33.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8b6800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd8b015ca50 0x7fd8b0073950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:33.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8b6800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8b015ce60 con 0x7fd8b00743c0 2024-04-25T14:06:33.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8b6800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8b0072b90 con 0x7fd8b0160d00 2024-04-25T14:06:33.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8b6800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd8b0072e70 con 0x7fd8b015ca50 2024-04-25T14:06:33.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8a7e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd8b00743c0 0x7fd8b01605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60026/0 (socket says 172.21.15.86:60026) 2024-04-25T14:06:33.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8a7e00640 1 -- 172.21.15.86:0/836883096 learned_addr learned my addr 172.21.15.86:0/836883096 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:33.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8af400640 1 --2- 172.21.15.86:0/836883096 >> v2:172.21.15.155:3300/0 conn(0x7fd8b015ca50 0x7fd8b0073950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:33.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8aea00640 1 -- 172.21.15.86:0/836883096 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 26589926 0 0) 0x7fd8b015ce60 con 0x7fd8b00743c0 2024-04-25T14:06:33.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8aea00640 1 -- 172.21.15.86:0/836883096 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd894003530 con 0x7fd8b00743c0 2024-04-25T14:06:33.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8aea00640 1 -- 172.21.15.86:0/836883096 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4247315550 0 0) 0x7fd8b0072b90 con 0x7fd8b0160d00 2024-04-25T14:06:33.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8aea00640 1 -- 172.21.15.86:0/836883096 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8b015ce60 con 0x7fd8b0160d00 2024-04-25T14:06:33.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8aea00640 1 -- 172.21.15.86:0/836883096 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 766338317 0 0) 0x7fd894003530 con 0x7fd8b00743c0 2024-04-25T14:06:33.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8aea00640 1 -- 172.21.15.86:0/836883096 >> v2:172.21.15.155:3300/0 conn(0x7fd8b015ca50 msgr2=0x7fd8b0073950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:33.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8aea00640 1 --2- 172.21.15.86:0/836883096 >> v2:172.21.15.155:3300/0 conn(0x7fd8b015ca50 0x7fd8b0073950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:33.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8aea00640 1 -- 172.21.15.86:0/836883096 >> v1:172.21.15.155:6789/0 conn(0x7fd8b0160d00 legacy=0x7fd8b006e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:33.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8aea00640 1 -- 172.21.15.86:0/836883096 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b0073150 con 0x7fd8b00743c0 2024-04-25T14:06:33.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8aea00640 1 -- 172.21.15.86:0/836883096 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fd898001680 con 0x7fd8b00743c0 2024-04-25T14:06:33.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8af400640 1 --2- 172.21.15.86:0/836883096 >> v2:172.21.15.155:3300/0 conn(0x7fd8b015ca50 0x7fd8b0073950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:33.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8aea00640 1 -- 172.21.15.86:0/836883096 >> v1:172.21.15.86:6789/0 conn(0x7fd8b00743c0 legacy=0x7fd8b01605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:33.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8aea00640 1 --2- 172.21.15.86:0/836883096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd894003b50 0x7fd894003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:33.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8aea00640 1 --2- 172.21.15.86:0/836883096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd894004810 0x7fd894004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:33.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8aea00640 1 --2- 172.21.15.86:0/836883096 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8940051b0 0x7fd89400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:33.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8aea00640 1 -- 172.21.15.86:0/836883096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b0072e70 con 0x7fd894003b50 2024-04-25T14:06:33.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8aea00640 1 -- 172.21.15.86:0/836883096 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd89400c150 con 0x7fd894004810 2024-04-25T14:06:33.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8aea00640 1 -- 172.21.15.86:0/836883096 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd89400e560 con 0x7fd8940051b0 2024-04-25T14:06:33.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8af400640 1 --2- 172.21.15.86:0/836883096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd894003b50 0x7fd894003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:33.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8a7e00640 1 --2- 172.21.15.86:0/836883096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd894004810 0x7fd894004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:33.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8afe00640 1 --2- 172.21.15.86:0/836883096 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8940051b0 0x7fd89400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:33.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8af400640 1 -- 172.21.15.86:0/836883096 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8940051b0 msgr2=0x7fd89400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:33.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8af400640 1 --2- 172.21.15.86:0/836883096 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8940051b0 0x7fd89400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:33.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8af400640 1 -- 172.21.15.86:0/836883096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd894004810 msgr2=0x7fd894004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:33.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8af400640 1 --2- 172.21.15.86:0/836883096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd894004810 0x7fd894004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:33.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8af400640 1 -- 172.21.15.86:0/836883096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd894003530 con 0x7fd894003b50 2024-04-25T14:06:33.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8afe00640 1 --2- 172.21.15.86:0/836883096 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8940051b0 0x7fd89400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:33.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.640+0000 7fd8afe00640 1 -- 172.21.15.86:0/836883096 reap_dead start 2024-04-25T14:06:33.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.644+0000 7fd8af400640 1 --2- 172.21.15.86:0/836883096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd894003b50 0x7fd894003fa0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd8940057f0 tx=0x7fd8a001cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e7b7604fd535df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:33.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.644+0000 7fd8aea00640 1 -- 172.21.15.86:0/836883096 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0009880 con 0x7fd894003b50 2024-04-25T14:06:33.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.644+0000 7fd8aea00640 1 -- 172.21.15.86:0/836883096 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a0004020 con 0x7fd894003b50 2024-04-25T14:06:33.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.644+0000 7fd8aea00640 1 -- 172.21.15.86:0/836883096 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0004430 con 0x7fd894003b50 2024-04-25T14:06:33.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.644+0000 7fd8b6800640 1 -- 172.21.15.86:0/836883096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd894003b50 msgr2=0x7fd894003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:33.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.644+0000 7fd8b6800640 1 --2- 172.21.15.86:0/836883096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd894003b50 0x7fd894003fa0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd8940057f0 tx=0x7fd8a001cb10 comp rx=0 tx=0).stop 2024-04-25T14:06:33.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.644+0000 7fd8b6800640 1 -- 172.21.15.86:0/836883096 shutdown_connections 2024-04-25T14:06:33.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.644+0000 7fd8b6800640 1 --2- 172.21.15.86:0/836883096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd894003b50 0x7fd894003fa0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:33.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.644+0000 7fd8b6800640 1 -- 172.21.15.86:0/836883096 >> 172.21.15.86:0/836883096 conn(0x7fd8b00b4a40 msgr2=0x7fd8b00b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:33.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.644+0000 7fd8b6800640 1 -- 172.21.15.86:0/836883096 shutdown_connections 2024-04-25T14:06:33.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.644+0000 7fd8b6800640 1 -- 172.21.15.86:0/836883096 wait complete. 2024-04-25T14:06:33.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.648+0000 7fd8b6800640 1 Processor -- start 2024-04-25T14:06:33.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.648+0000 7fd8b6800640 1 -- start start 2024-04-25T14:06:33.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.648+0000 7fd8b6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd894003b50 0x7fd8b0157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:33.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.648+0000 7fd8b6800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8b0150770 0x7fd8b0154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:33.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.648+0000 7fd8b6800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8b0154e40 0x7fd8b0151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:33.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.648+0000 7fd8b6800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b0073ce0 con 0x7fd894003b50 2024-04-25T14:06:33.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.648+0000 7fd8b6800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b0073b60 con 0x7fd8b0150770 2024-04-25T14:06:33.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.648+0000 7fd8b6800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd8b0073e60 con 0x7fd8b0154e40 2024-04-25T14:06:33.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.648+0000 7fd8af400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd894003b50 0x7fd8b0157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:33.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.648+0000 7fd8af400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd894003b50 0x7fd8b0157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49338/0 (socket says 172.21.15.86:49338) 2024-04-25T14:06:33.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.648+0000 7fd8af400640 1 -- 172.21.15.86:0/1271759207 learned_addr learned my addr 172.21.15.86:0/1271759207 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:33.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.648+0000 7fd8afe00640 1 --2- 172.21.15.86:0/1271759207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8b0154e40 0x7fd8b0151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:33.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.648+0000 7fd8a7e00640 1 --2- 172.21.15.86:0/1271759207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8b0150770 0x7fd8b0154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:33.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.648+0000 7fd8af400640 1 -- 172.21.15.86:0/1271759207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8b0154e40 msgr2=0x7fd8b0151380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:33.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.648+0000 7fd8af400640 1 --2- 172.21.15.86:0/1271759207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8b0154e40 0x7fd8b0151380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:33.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.648+0000 7fd8af400640 1 -- 172.21.15.86:0/1271759207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8b0150770 msgr2=0x7fd8b0154900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:33.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.648+0000 7fd8af400640 1 --2- 172.21.15.86:0/1271759207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8b0150770 0x7fd8b0154900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:33.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.652+0000 7fd8af400640 1 -- 172.21.15.86:0/1271759207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b0155250 con 0x7fd894003b50 2024-04-25T14:06:33.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.652+0000 7fd8a7e00640 1 --2- 172.21.15.86:0/1271759207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8b0150770 0x7fd8b0154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:33.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.652+0000 7fd8afe00640 1 --2- 172.21.15.86:0/1271759207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8b0154e40 0x7fd8b0151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:33.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.652+0000 7fd8af400640 1 --2- 172.21.15.86:0/1271759207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd894003b50 0x7fd8b0157710 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd8a00af040 tx=0x7fd894005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:33.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.652+0000 7fd8a7400640 1 -- 172.21.15.86:0/1271759207 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0005e30 con 0x7fd894003b50 2024-04-25T14:06:33.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.652+0000 7fd8a7400640 1 -- 172.21.15.86:0/1271759207 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a00bc070 con 0x7fd894003b50 2024-04-25T14:06:33.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.652+0000 7fd8a7400640 1 -- 172.21.15.86:0/1271759207 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a00a5990 con 0x7fd894003b50 2024-04-25T14:06:33.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.652+0000 7fd8b6800640 1 -- 172.21.15.86:0/1271759207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b0157c50 con 0x7fd894003b50 2024-04-25T14:06:33.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.652+0000 7fd8b6800640 1 -- 172.21.15.86:0/1271759207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8b015e6a0 con 0x7fd894003b50 2024-04-25T14:06:33.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.652+0000 7fd8b6800640 1 -- 172.21.15.86:0/1271759207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8b0050750 con 0x7fd894003b50 2024-04-25T14:06:33.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.652+0000 7fd8a7400640 1 -- 172.21.15.86:0/1271759207 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7fd8a00a5b30 con 0x7fd894003b50 2024-04-25T14:06:33.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.652+0000 7fd8a7400640 1 --2- 172.21.15.86:0/1271759207 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fd88c06f640 0x7fd88c071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:33.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.652+0000 7fd8a7400640 1 -- 172.21.15.86:0/1271759207 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 7207+0+0 (secure 0 0 0) 0x7fd8a012a6e0 con 0x7fd894003b50 2024-04-25T14:06:33.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.652+0000 7fd8a7e00640 1 --2- 172.21.15.86:0/1271759207 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fd88c06f640 0x7fd88c071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:33.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.656+0000 7fd8a7e00640 1 --2- 172.21.15.86:0/1271759207 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fd88c06f640 0x7fd88c071b50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd898001210 tx=0x7fd898001120 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:33.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.660+0000 7fd8a7400640 1 -- 172.21.15.86:0/1271759207 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd8b0155250 con 0x7fd894003b50 2024-04-25T14:06:33.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:33.820+0000 7fd8b6800640 1 -- 172.21.15.86:0/1271759207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "fs2_metadata", "pool2": "fs2_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd8b00d1fb0 con 0x7fd894003b50 2024-04-25T14:06:34.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.464+0000 7fd8a7400640 1 -- 172.21.15.86:0/1271759207 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "fs2_metadata", "pool2": "fs2_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'fs2_metadata' removed v64) ==== 172+0+0 (secure 0 0 0) 0x7fd8a0016860 con 0x7fd894003b50 2024-04-25T14:06:34.467 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'fs2_metadata' removed 2024-04-25T14:06:34.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.468+0000 7fd8b6800640 1 -- 172.21.15.86:0/1271759207 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fd88c06f640 msgr2=0x7fd88c071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:34.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.468+0000 7fd8b6800640 1 --2- 172.21.15.86:0/1271759207 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fd88c06f640 0x7fd88c071b50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd898001210 tx=0x7fd898001120 comp rx=0 tx=0).stop 2024-04-25T14:06:34.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.468+0000 7fd8b6800640 1 -- 172.21.15.86:0/1271759207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd894003b50 msgr2=0x7fd8b0157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:34.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.468+0000 7fd8b6800640 1 --2- 172.21.15.86:0/1271759207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd894003b50 0x7fd8b0157710 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd8a00af040 tx=0x7fd894005840 comp rx=0 tx=0).stop 2024-04-25T14:06:34.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.468+0000 7fd8b6800640 1 -- 172.21.15.86:0/1271759207 shutdown_connections 2024-04-25T14:06:34.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.468+0000 7fd8b6800640 1 --2- 172.21.15.86:0/1271759207 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fd88c06f640 0x7fd88c071b50 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd898001210 tx=0x7fd898001120 comp rx=0 tx=0).stop 2024-04-25T14:06:34.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.468+0000 7fd8b6800640 1 --2- 172.21.15.86:0/1271759207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8b0154e40 0x7fd8b0151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:34.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.468+0000 7fd8b6800640 1 --2- 172.21.15.86:0/1271759207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8b0150770 0x7fd8b0154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:34.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.468+0000 7fd8b6800640 1 --2- 172.21.15.86:0/1271759207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd894003b50 0x7fd8b0157710 secure :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd8a00af040 tx=0x7fd894005840 comp rx=0 tx=0).stop 2024-04-25T14:06:34.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.468+0000 7fd8b6800640 1 -- 172.21.15.86:0/1271759207 >> 172.21.15.86:0/1271759207 conn(0x7fd8b00b4a40 msgr2=0x7fd8b015ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:34.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.468+0000 7fd8b6800640 1 -- 172.21.15.86:0/1271759207 shutdown_connections 2024-04-25T14:06:34.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.468+0000 7fd8b6800640 1 -- 172.21.15.86:0/1271759207 wait complete. 2024-04-25T14:06:34.495 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm fs2_data fs2_data --yes-i-really-really-mean-it 2024-04-25T14:06:34.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.652+0000 7f82ae000640 1 Processor -- start 2024-04-25T14:06:34.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.652+0000 7f82ae000640 1 -- start start 2024-04-25T14:06:34.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.652+0000 7f82ae000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f82a80cdca0 0x7f82a80cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:34.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.652+0000 7f82ae000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f82a8071450 0x7f82a8168550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:34.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.652+0000 7f82ae000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82a815dd70 con 0x7f82a80d0150 2024-04-25T14:06:34.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.652+0000 7f82ae000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f82a80d1480 con 0x7f82a80cdca0 2024-04-25T14:06:34.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.652+0000 7f82ae000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f82a815ed40 con 0x7f82a8071450 2024-04-25T14:06:34.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.652+0000 7f82acc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f82a80cdca0 0x7f82a80cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:34.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.652+0000 7f82acc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f82a80cdca0 0x7f82a80cfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49346/0 (socket says 172.21.15.86:49346) 2024-04-25T14:06:34.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.652+0000 7f829fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f82a80d0150 0x7f82a8070d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60040/0 (socket says 172.21.15.86:60040) 2024-04-25T14:06:34.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.652+0000 7f82acc00640 1 -- 172.21.15.86:0/1606259321 learned_addr learned my addr 172.21.15.86:0/1606259321 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:34.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.652+0000 7f82acc00640 1 -- 172.21.15.86:0/1606259321 >> v2:172.21.15.155:3300/0 conn(0x7f82a8071450 msgr2=0x7f82a8168550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:34.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.652+0000 7f82acc00640 1 --2- 172.21.15.86:0/1606259321 >> v2:172.21.15.155:3300/0 conn(0x7f82a8071450 0x7f82a8168550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:34.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.652+0000 7f82acc00640 1 -- 172.21.15.86:0/1606259321 >> v1:172.21.15.86:6789/0 conn(0x7f82a80d0150 legacy=0x7f82a8070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:34.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.652+0000 7f82acc00640 1 -- 172.21.15.86:0/1606259321 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a80d16d0 con 0x7f82a80cdca0 2024-04-25T14:06:34.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.656+0000 7f82acc00640 1 --2- 172.21.15.86:0/1606259321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a80cdca0 0x7f82a80cfc10 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f82a80cf1c0 tx=0x7f829801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d4145949b9c398 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:34.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.656+0000 7f82a7e00640 1 -- 172.21.15.86:0/1606259321 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82980ab610 con 0x7f82a80cdca0 2024-04-25T14:06:34.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.656+0000 7f82a7e00640 1 -- 172.21.15.86:0/1606259321 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82980b0070 con 0x7f82a80cdca0 2024-04-25T14:06:34.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.656+0000 7f82a7e00640 1 -- 172.21.15.86:0/1606259321 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82980ab9d0 con 0x7f82a80cdca0 2024-04-25T14:06:34.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.656+0000 7f82ae000640 1 -- 172.21.15.86:0/1606259321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a80cdca0 msgr2=0x7f82a80cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:34.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.656+0000 7f82ae000640 1 --2- 172.21.15.86:0/1606259321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a80cdca0 0x7f82a80cfc10 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f82a80cf1c0 tx=0x7f829801c520 comp rx=0 tx=0).stop 2024-04-25T14:06:34.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.656+0000 7f82ae000640 1 -- 172.21.15.86:0/1606259321 shutdown_connections 2024-04-25T14:06:34.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.656+0000 7f82ae000640 1 --2- 172.21.15.86:0/1606259321 >> v2:172.21.15.155:3300/0 conn(0x7f82a8071450 0x7f82a8168550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:34.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.656+0000 7f82ae000640 1 --2- 172.21.15.86:0/1606259321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a80cdca0 0x7f82a80cfc10 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:34.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.656+0000 7f82ae000640 1 -- 172.21.15.86:0/1606259321 >> 172.21.15.86:0/1606259321 conn(0x7f82a80701b0 msgr2=0x7f82a80b4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:34.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.656+0000 7f82ae000640 1 -- 172.21.15.86:0/1606259321 shutdown_connections 2024-04-25T14:06:34.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.656+0000 7f82ae000640 1 -- 172.21.15.86:0/1606259321 wait complete. 2024-04-25T14:06:34.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.660+0000 7f82ae000640 1 Processor -- start 2024-04-25T14:06:34.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.660+0000 7f82ae000640 1 -- start start 2024-04-25T14:06:34.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.660+0000 7f82ae000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a8071450 0x7f82a8152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:34.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.660+0000 7f82ae000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82a80cdca0 0x7f82a815f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:34.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.660+0000 7f82ae000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82a80d0150 0x7f82a815f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:34.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.660+0000 7f82ae000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82a80ce5c0 con 0x7f82a8071450 2024-04-25T14:06:34.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.660+0000 7f82ae000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f82a80ce0b0 con 0x7f82a80d0150 2024-04-25T14:06:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.660+0000 7f82ae000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f82a80ce740 con 0x7f82a80cdca0 2024-04-25T14:06:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.660+0000 7f82acc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a8071450 0x7f82a8152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.660+0000 7f82acc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a8071450 0x7f82a8152d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49356/0 (socket says 172.21.15.86:49356) 2024-04-25T14:06:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.660+0000 7f82acc00640 1 -- 172.21.15.86:0/3138164583 learned_addr learned my addr 172.21.15.86:0/3138164583 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.660+0000 7f82acc00640 1 -- 172.21.15.86:0/3138164583 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82a80cdca0 msgr2=0x7f82a815f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.660+0000 7f82acc00640 1 --2- 172.21.15.86:0/3138164583 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82a80cdca0 0x7f82a815f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.660+0000 7f82acc00640 1 -- 172.21.15.86:0/3138164583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82a80d0150 msgr2=0x7f82a815f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.660+0000 7f82acc00640 1 --2- 172.21.15.86:0/3138164583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82a80d0150 0x7f82a815f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.660+0000 7f82acc00640 1 -- 172.21.15.86:0/3138164583 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a80b8000 con 0x7f82a8071450 2024-04-25T14:06:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.664+0000 7f82acc00640 1 --2- 172.21.15.86:0/3138164583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a8071450 0x7f82a8152d60 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f829801a150 tx=0x7f829801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:34.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.664+0000 7f82a5600640 1 -- 172.21.15.86:0/3138164583 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82980be5f0 con 0x7f82a8071450 2024-04-25T14:06:34.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.664+0000 7f82a5600640 1 -- 172.21.15.86:0/3138164583 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82980be7b0 con 0x7f82a8071450 2024-04-25T14:06:34.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.664+0000 7f82a5600640 1 -- 172.21.15.86:0/3138164583 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82980beba0 con 0x7f82a8071450 2024-04-25T14:06:34.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.664+0000 7f82ae000640 1 -- 172.21.15.86:0/3138164583 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a806d920 con 0x7f82a8071450 2024-04-25T14:06:34.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.664+0000 7f82ae000640 1 -- 172.21.15.86:0/3138164583 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a81532a0 con 0x7f82a8071450 2024-04-25T14:06:34.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.664+0000 7f82ae000640 1 -- 172.21.15.86:0/3138164583 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82a8076b80 con 0x7f82a8071450 2024-04-25T14:06:34.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.664+0000 7f82a5600640 1 -- 172.21.15.86:0/3138164583 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f82980b4600 con 0x7f82a8071450 2024-04-25T14:06:34.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.664+0000 7f82a5600640 1 --2- 172.21.15.86:0/3138164583 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f828406f670 0x7f8284071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:34.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.664+0000 7f82a5600640 1 -- 172.21.15.86:0/3138164583 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6775+0+0 (secure 0 0 0) 0x7f829812a5e0 con 0x7f82a8071450 2024-04-25T14:06:34.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.668+0000 7f829fe00640 1 --2- 172.21.15.86:0/3138164583 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f828406f670 0x7f8284071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:34.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.668+0000 7f829fe00640 1 --2- 172.21.15.86:0/3138164583 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f828406f670 0x7f8284071b80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8290000f70 tx=0x7f8290007a10 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:34.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.668+0000 7f82a5600640 1 -- 172.21.15.86:0/3138164583 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f82980c4060 con 0x7f82a8071450 2024-04-25T14:06:34.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:34.824+0000 7f82ae000640 1 -- 172.21.15.86:0/3138164583 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "fs2_data", "pool2": "fs2_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f82a815d1d0 con 0x7f82a8071450 2024-04-25T14:06:35.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.471+0000 7f82a5600640 1 -- 172.21.15.86:0/3138164583 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "fs2_data", "pool2": "fs2_data", "yes_i_really_really_mean_it": true}]=0 pool 'fs2_data' removed v65) ==== 160+0+0 (secure 0 0 0) 0x7f82980a8050 con 0x7f82a8071450 2024-04-25T14:06:35.473 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'fs2_data' removed 2024-04-25T14:06:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.475+0000 7f82ae000640 1 -- 172.21.15.86:0/3138164583 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f828406f670 msgr2=0x7f8284071b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.475+0000 7f82ae000640 1 --2- 172.21.15.86:0/3138164583 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f828406f670 0x7f8284071b80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8290000f70 tx=0x7f8290007a10 comp rx=0 tx=0).stop 2024-04-25T14:06:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.475+0000 7f82ae000640 1 -- 172.21.15.86:0/3138164583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a8071450 msgr2=0x7f82a8152d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.475+0000 7f82ae000640 1 --2- 172.21.15.86:0/3138164583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a8071450 0x7f82a8152d60 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f829801a150 tx=0x7f829801ad60 comp rx=0 tx=0).stop 2024-04-25T14:06:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.475+0000 7f82ae000640 1 -- 172.21.15.86:0/3138164583 shutdown_connections 2024-04-25T14:06:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.475+0000 7f82ae000640 1 --2- 172.21.15.86:0/3138164583 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f828406f670 0x7f8284071b80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.475+0000 7f82ae000640 1 --2- 172.21.15.86:0/3138164583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82a80d0150 0x7f82a815f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.475+0000 7f82ae000640 1 --2- 172.21.15.86:0/3138164583 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82a80cdca0 0x7f82a815f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.475+0000 7f82ae000640 1 --2- 172.21.15.86:0/3138164583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a8071450 0x7f82a8152d60 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.475+0000 7f82ae000640 1 -- 172.21.15.86:0/3138164583 >> 172.21.15.86:0/3138164583 conn(0x7f82a80701b0 msgr2=0x7f82a80d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.475+0000 7f82ae000640 1 -- 172.21.15.86:0/3138164583 shutdown_connections 2024-04-25T14:06:35.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.475+0000 7f82ae000640 1 -- 172.21.15.86:0/3138164583 wait complete. 2024-04-25T14:06:35.503 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T14:06:35.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.651+0000 7f1b90600640 1 Processor -- start 2024-04-25T14:06:35.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.651+0000 7f1b90600640 1 -- start start 2024-04-25T14:06:35.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.651+0000 7f1b90600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b880d1df0 0x7f1b880cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:35.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.651+0000 7f1b90600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1b880cdcc0 0x7f1b880ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:35.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.651+0000 7f1b90600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b8815e5c0 con 0x7f1b88070330 2024-04-25T14:06:35.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.651+0000 7f1b90600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1b8806fd40 con 0x7f1b880cdcc0 2024-04-25T14:06:35.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.651+0000 7f1b90600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1b8815e8a0 con 0x7f1b880d1df0 2024-04-25T14:06:35.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.651+0000 7f1b8e600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1b880cdcc0 0x7f1b880ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:35.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.651+0000 7f1b8dc00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1b88070330 0x7f1b880d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60052/0 (socket says 172.21.15.86:60052) 2024-04-25T14:06:35.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.651+0000 7f1b8dc00640 1 -- 172.21.15.86:0/2188946069 learned_addr learned my addr 172.21.15.86:0/2188946069 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:35.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.651+0000 7f1b8e600640 1 -- 172.21.15.86:0/2188946069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b880d1df0 msgr2=0x7f1b880cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:35.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.651+0000 7f1b8d200640 1 --2- 172.21.15.86:0/2188946069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b880d1df0 0x7f1b880cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:35.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.651+0000 7f1b8e600640 1 --2- 172.21.15.86:0/2188946069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b880d1df0 0x7f1b880cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.651+0000 7f1b87e00640 1 -- 172.21.15.86:0/2188946069 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3766713913 0 0) 0x7f1b8815e5c0 con 0x7f1b88070330 2024-04-25T14:06:35.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.651+0000 7f1b8e600640 1 -- 172.21.15.86:0/2188946069 >> v1:172.21.15.86:6789/0 conn(0x7f1b88070330 legacy=0x7f1b880d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:35.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.651+0000 7f1b8e600640 1 -- 172.21.15.86:0/2188946069 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b8815eb80 con 0x7f1b880cdcc0 2024-04-25T14:06:35.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.651+0000 7f1b8d200640 1 --2- 172.21.15.86:0/2188946069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b880d1df0 0x7f1b880cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:35.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.655+0000 7f1b8e600640 1 --2- 172.21.15.86:0/2188946069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b880cdcc0 0x7f1b880ce0d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f1b80009d90 tx=0x7f1b8001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a659f78a1fb60b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:35.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.655+0000 7f1b87e00640 1 -- 172.21.15.86:0/2188946069 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b800b2070 con 0x7f1b880cdcc0 2024-04-25T14:06:35.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.655+0000 7f1b87e00640 1 -- 172.21.15.86:0/2188946069 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b800a5950 con 0x7f1b880cdcc0 2024-04-25T14:06:35.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.655+0000 7f1b87e00640 1 -- 172.21.15.86:0/2188946069 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b800a5d20 con 0x7f1b880cdcc0 2024-04-25T14:06:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.655+0000 7f1b90600640 1 -- 172.21.15.86:0/2188946069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b880cdcc0 msgr2=0x7f1b880ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.655+0000 7f1b90600640 1 --2- 172.21.15.86:0/2188946069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b880cdcc0 0x7f1b880ce0d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f1b80009d90 tx=0x7f1b8001c8b0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.655+0000 7f1b90600640 1 -- 172.21.15.86:0/2188946069 shutdown_connections 2024-04-25T14:06:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.655+0000 7f1b90600640 1 --2- 172.21.15.86:0/2188946069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b880cdcc0 0x7f1b880ce0d0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.655+0000 7f1b90600640 1 --2- 172.21.15.86:0/2188946069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b880d1df0 0x7f1b880cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.655+0000 7f1b90600640 1 -- 172.21.15.86:0/2188946069 >> 172.21.15.86:0/2188946069 conn(0x7f1b880b4650 msgr2=0x7f1b880b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:35.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.655+0000 7f1b90600640 1 -- 172.21.15.86:0/2188946069 shutdown_connections 2024-04-25T14:06:35.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.655+0000 7f1b90600640 1 -- 172.21.15.86:0/2188946069 wait complete. 2024-04-25T14:06:35.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.659+0000 7f1b90600640 1 Processor -- start 2024-04-25T14:06:35.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.659+0000 7f1b90600640 1 -- start start 2024-04-25T14:06:35.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.659+0000 7f1b90600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b88070330 0x7f1b880c3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:35.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.659+0000 7f1b90600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b880cdcc0 0x7f1b880c3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:35.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.659+0000 7f1b90600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b8815acb0 0x7f1b880c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:35.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.659+0000 7f1b90600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8815f4b0 con 0x7f1b88070330 2024-04-25T14:06:35.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.659+0000 7f1b90600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8815f330 con 0x7f1b880cdcc0 2024-04-25T14:06:35.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.659+0000 7f1b90600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1b88070740 con 0x7f1b8815acb0 2024-04-25T14:06:35.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.659+0000 7f1b8dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b88070330 0x7f1b880c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.659+0000 7f1b8dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b88070330 0x7f1b880c3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49382/0 (socket says 172.21.15.86:49382) 2024-04-25T14:06:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.659+0000 7f1b8dc00640 1 -- 172.21.15.86:0/859458608 learned_addr learned my addr 172.21.15.86:0/859458608 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.659+0000 7f1b8d200640 1 --2- 172.21.15.86:0/859458608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b880cdcc0 0x7f1b880c3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.659+0000 7f1b8e600640 1 --2- 172.21.15.86:0/859458608 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b8815acb0 0x7f1b880c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.659+0000 7f1b8dc00640 1 -- 172.21.15.86:0/859458608 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b8815acb0 msgr2=0x7f1b880c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.659+0000 7f1b8dc00640 1 --2- 172.21.15.86:0/859458608 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b8815acb0 0x7f1b880c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.659+0000 7f1b8dc00640 1 -- 172.21.15.86:0/859458608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b880cdcc0 msgr2=0x7f1b880c3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.659+0000 7f1b8dc00640 1 --2- 172.21.15.86:0/859458608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b880cdcc0 0x7f1b880c3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.659+0000 7f1b8dc00640 1 -- 172.21.15.86:0/859458608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b880b28c0 con 0x7f1b88070330 2024-04-25T14:06:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.659+0000 7f1b8dc00640 1 --2- 172.21.15.86:0/859458608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b88070330 0x7f1b880c3650 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c00c480 tx=0x7f1b7c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.659+0000 7f1b85600640 1 -- 172.21.15.86:0/859458608 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c018070 con 0x7f1b88070330 2024-04-25T14:06:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.659+0000 7f1b85600640 1 -- 172.21.15.86:0/859458608 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b7c0074e0 con 0x7f1b88070330 2024-04-25T14:06:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.659+0000 7f1b85600640 1 -- 172.21.15.86:0/859458608 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c0049f0 con 0x7f1b88070330 2024-04-25T14:06:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.659+0000 7f1b90600640 1 -- 172.21.15.86:0/859458608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b880b2ba0 con 0x7f1b88070330 2024-04-25T14:06:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.659+0000 7f1b90600640 1 -- 172.21.15.86:0/859458608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b880c2e50 con 0x7f1b88070330 2024-04-25T14:06:35.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.663+0000 7f1b90600640 1 -- 172.21.15.86:0/859458608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b88069d00 con 0x7f1b88070330 2024-04-25T14:06:35.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.663+0000 7f1b85600640 1 -- 172.21.15.86:0/859458608 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f1b7c004b90 con 0x7f1b88070330 2024-04-25T14:06:35.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.663+0000 7f1b85600640 1 --2- 172.21.15.86:0/859458608 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f1b6806f640 0x7f1b68071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:35.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.663+0000 7f1b85600640 1 -- 172.21.15.86:0/859458608 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6367+0+0 (secure 0 0 0) 0x7f1b7c08ef30 con 0x7f1b88070330 2024-04-25T14:06:35.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.663+0000 7f1b8d200640 1 --2- 172.21.15.86:0/859458608 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f1b6806f640 0x7f1b68071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:35.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.667+0000 7f1b8d200640 1 --2- 172.21.15.86:0/859458608 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f1b6806f640 0x7f1b68071b50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1b74006fd0 tx=0x7f1b74008040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:35.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.667+0000 7f1b85600640 1 -- 172.21.15.86:0/859458608 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1b7c0bbe70 con 0x7f1b88070330 2024-04-25T14:06:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.827+0000 7f1b90600640 1 -- 172.21.15.86:0/859458608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f1b88076bf0 con 0x7f1b88070330 2024-04-25T14:06:35.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.831+0000 7f1b85600640 1 -- 172.21.15.86:0/859458608 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f1b7c021080 con 0x7f1b88070330 2024-04-25T14:06:35.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.831+0000 7f1b90600640 1 -- 172.21.15.86:0/859458608 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f1b6806f640 msgr2=0x7f1b68071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:35.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.831+0000 7f1b90600640 1 --2- 172.21.15.86:0/859458608 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f1b6806f640 0x7f1b68071b50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1b74006fd0 tx=0x7f1b74008040 comp rx=0 tx=0).stop 2024-04-25T14:06:35.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.831+0000 7f1b90600640 1 -- 172.21.15.86:0/859458608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b88070330 msgr2=0x7f1b880c3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:35.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.831+0000 7f1b90600640 1 --2- 172.21.15.86:0/859458608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b88070330 0x7f1b880c3650 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c00c480 tx=0x7f1b7c00c940 comp rx=0 tx=0).stop 2024-04-25T14:06:35.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.831+0000 7f1b90600640 1 -- 172.21.15.86:0/859458608 shutdown_connections 2024-04-25T14:06:35.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.831+0000 7f1b90600640 1 --2- 172.21.15.86:0/859458608 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f1b6806f640 0x7f1b68071b50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.831+0000 7f1b90600640 1 --2- 172.21.15.86:0/859458608 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b8815acb0 0x7f1b880c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.831+0000 7f1b90600640 1 --2- 172.21.15.86:0/859458608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b880cdcc0 0x7f1b880c3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.831+0000 7f1b90600640 1 --2- 172.21.15.86:0/859458608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b88070330 0x7f1b880c3650 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.831+0000 7f1b90600640 1 -- 172.21.15.86:0/859458608 >> 172.21.15.86:0/859458608 conn(0x7f1b880b4650 msgr2=0x7f1b880aef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:35.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.831+0000 7f1b90600640 1 -- 172.21.15.86:0/859458608 shutdown_connections 2024-04-25T14:06:35.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.831+0000 7f1b90600640 1 -- 172.21.15.86:0/859458608 wait complete. 2024-04-25T14:06:35.855 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_client_metrics_and_metadata' 2024-04-25T14:06:36.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.999+0000 7f945e600640 1 Processor -- start 2024-04-25T14:06:36.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.999+0000 7f945e600640 1 -- start start 2024-04-25T14:06:36.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.999+0000 7f945e600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94580af390 0x7f945815fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:36.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.999+0000 7f945e600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f945806d560 0x7f945806d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:36.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.999+0000 7f945e600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94581602f0 con 0x7f94580d2200 2024-04-25T14:06:36.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.999+0000 7f945e600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f94580b7c00 con 0x7f945806d560 2024-04-25T14:06:36.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:35.999+0000 7f945e600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f945806c7a0 con 0x7f94580af390 2024-04-25T14:06:36.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.003+0000 7f9457e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f945806d560 0x7f945806d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:36.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.003+0000 7f944ea00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f94580d2200 0x7f94580d2610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60060/0 (socket says 172.21.15.86:60060) 2024-04-25T14:06:36.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.003+0000 7f944ea00640 1 -- 172.21.15.86:0/872487519 learned_addr learned my addr 172.21.15.86:0/872487519 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:36.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.003+0000 7f9457400640 1 --2- 172.21.15.86:0/872487519 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94580af390 0x7f945815fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:36.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.003+0000 7f9457e00640 1 -- 172.21.15.86:0/872487519 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94580af390 msgr2=0x7f945815fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:36.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.003+0000 7f9457e00640 1 --2- 172.21.15.86:0/872487519 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94580af390 0x7f945815fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:36.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.003+0000 7f9457e00640 1 -- 172.21.15.86:0/872487519 >> v1:172.21.15.86:6789/0 conn(0x7f94580d2200 legacy=0x7f94580d2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:36.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.003+0000 7f9457e00640 1 -- 172.21.15.86:0/872487519 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f945806ca80 con 0x7f945806d560 2024-04-25T14:06:36.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.003+0000 7f9456a00640 1 -- 172.21.15.86:0/872487519 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2721364514 0 0) 0x7f94581602f0 con 0x7f94580d2200 2024-04-25T14:06:36.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.003+0000 7f9457e00640 1 --2- 172.21.15.86:0/872487519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f945806d560 0x7f945806d970 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f9450009d90 tx=0x7f945001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a050c3e62b9a041e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:36.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.003+0000 7f9456a00640 1 -- 172.21.15.86:0/872487519 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94500b2070 con 0x7f945806d560 2024-04-25T14:06:36.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.003+0000 7f9456a00640 1 -- 172.21.15.86:0/872487519 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94500a5970 con 0x7f945806d560 2024-04-25T14:06:36.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.003+0000 7f9456a00640 1 -- 172.21.15.86:0/872487519 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94500a5d40 con 0x7f945806d560 2024-04-25T14:06:36.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.007+0000 7f945e600640 1 -- 172.21.15.86:0/872487519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f945806d560 msgr2=0x7f945806d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:36.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.007+0000 7f945e600640 1 --2- 172.21.15.86:0/872487519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f945806d560 0x7f945806d970 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f9450009d90 tx=0x7f945001c8b0 comp rx=0 tx=0).stop 2024-04-25T14:06:36.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.007+0000 7f945e600640 1 -- 172.21.15.86:0/872487519 shutdown_connections 2024-04-25T14:06:36.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.007+0000 7f945e600640 1 --2- 172.21.15.86:0/872487519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f945806d560 0x7f945806d970 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:36.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.007+0000 7f945e600640 1 --2- 172.21.15.86:0/872487519 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94580af390 0x7f945815fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:36.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.007+0000 7f945e600640 1 -- 172.21.15.86:0/872487519 >> 172.21.15.86:0/872487519 conn(0x7f9458073ce0 msgr2=0x7f945809ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:36.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.007+0000 7f945e600640 1 -- 172.21.15.86:0/872487519 shutdown_connections 2024-04-25T14:06:36.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.007+0000 7f945e600640 1 -- 172.21.15.86:0/872487519 wait complete. 2024-04-25T14:06:36.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.007+0000 7f945e600640 1 Processor -- start 2024-04-25T14:06:36.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.007+0000 7f945e600640 1 -- start start 2024-04-25T14:06:36.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.011+0000 7f945e600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f945806d560 0x7f945815e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:36.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.011+0000 7f945e600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94580af390 0x7f945815bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:36.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.011+0000 7f945e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94580d2200 0x7f945815f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:36.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.011+0000 7f945e600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f945816cde0 con 0x7f94580d2200 2024-04-25T14:06:36.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.011+0000 7f945e600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f94580b78d0 con 0x7f945806d560 2024-04-25T14:06:36.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.011+0000 7f945e600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f945816cf60 con 0x7f94580af390 2024-04-25T14:06:36.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.011+0000 7f9457e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94580d2200 0x7f945815f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:36.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.011+0000 7f9457e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94580d2200 0x7f945815f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49408/0 (socket says 172.21.15.86:49408) 2024-04-25T14:06:36.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.011+0000 7f9457e00640 1 -- 172.21.15.86:0/845957225 learned_addr learned my addr 172.21.15.86:0/845957225 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:36.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.011+0000 7f9457400640 1 --2- 172.21.15.86:0/845957225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f945806d560 0x7f945815e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:36.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.011+0000 7f944ea00640 1 --2- 172.21.15.86:0/845957225 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94580af390 0x7f945815bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:36.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.011+0000 7f9457e00640 1 -- 172.21.15.86:0/845957225 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94580af390 msgr2=0x7f945815bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:36.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.011+0000 7f9457e00640 1 --2- 172.21.15.86:0/845957225 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94580af390 0x7f945815bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:36.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.011+0000 7f9457e00640 1 -- 172.21.15.86:0/845957225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f945806d560 msgr2=0x7f945815e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:36.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.011+0000 7f9457e00640 1 --2- 172.21.15.86:0/845957225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f945806d560 0x7f945815e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:36.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.011+0000 7f9457e00640 1 -- 172.21.15.86:0/845957225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f945815c810 con 0x7f94580d2200 2024-04-25T14:06:36.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.011+0000 7f9457400640 1 --2- 172.21.15.86:0/845957225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f945806d560 0x7f945815e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:36.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.011+0000 7f9457e00640 1 --2- 172.21.15.86:0/845957225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94580d2200 0x7f945815f0d0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f9450009d90 tx=0x7f9450019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:36.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.011+0000 7f944fe00640 1 -- 172.21.15.86:0/845957225 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94500b2050 con 0x7f94580d2200 2024-04-25T14:06:36.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.011+0000 7f944fe00640 1 -- 172.21.15.86:0/845957225 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94500b3040 con 0x7f94580d2200 2024-04-25T14:06:36.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.011+0000 7f945e600640 1 -- 172.21.15.86:0/845957225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f945815caf0 con 0x7f94580d2200 2024-04-25T14:06:36.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.011+0000 7f944fe00640 1 -- 172.21.15.86:0/845957225 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94500ad850 con 0x7f94580d2200 2024-04-25T14:06:36.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.011+0000 7f945e600640 1 -- 172.21.15.86:0/845957225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f945809c6f0 con 0x7f94580d2200 2024-04-25T14:06:36.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.011+0000 7f945e600640 1 -- 172.21.15.86:0/845957225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9458076bf0 con 0x7f94580d2200 2024-04-25T14:06:36.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.015+0000 7f944fe00640 1 -- 172.21.15.86:0/845957225 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f94500c0020 con 0x7f94580d2200 2024-04-25T14:06:36.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.015+0000 7f944fe00640 1 --2- 172.21.15.86:0/845957225 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f943406f670 0x7f9434071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:36.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.015+0000 7f944fe00640 1 -- 172.21.15.86:0/845957225 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6367+0+0 (secure 0 0 0) 0x7f945012a330 con 0x7f94580d2200 2024-04-25T14:06:36.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.015+0000 7f9457400640 1 --2- 172.21.15.86:0/845957225 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f943406f670 0x7f9434071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:36.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.015+0000 7f9457400640 1 --2- 172.21.15.86:0/845957225 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f943406f670 0x7f9434071b80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f94580d3180 tx=0x7f9448008040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:36.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.019+0000 7f944fe00640 1 -- 172.21.15.86:0/845957225 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9450016560 con 0x7f94580d2200 2024-04-25T14:06:36.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.235+0000 7f945e600640 1 -- 172.21.15.86:0/845957225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_client_metrics_and_metadata"]} v 0) -- 0x7f945815c5a0 con 0x7f94580d2200 2024-04-25T14:06:36.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.471+0000 7f944fe00640 1 -- 172.21.15.86:0/845957225 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_client_metrics_and_metadata"]}]=0 v184) ==== 156+0+0 (secure 0 0 0) 0x7f94500f6ca0 con 0x7f94580d2200 2024-04-25T14:06:36.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.475+0000 7f945e600640 1 -- 172.21.15.86:0/845957225 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f943406f670 msgr2=0x7f9434071b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:36.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.475+0000 7f945e600640 1 --2- 172.21.15.86:0/845957225 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f943406f670 0x7f9434071b80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f94580d3180 tx=0x7f9448008040 comp rx=0 tx=0).stop 2024-04-25T14:06:36.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.475+0000 7f945e600640 1 -- 172.21.15.86:0/845957225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94580d2200 msgr2=0x7f945815f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:36.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.475+0000 7f945e600640 1 --2- 172.21.15.86:0/845957225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94580d2200 0x7f945815f0d0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f9450009d90 tx=0x7f9450019650 comp rx=0 tx=0).stop 2024-04-25T14:06:36.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.475+0000 7f945e600640 1 -- 172.21.15.86:0/845957225 shutdown_connections 2024-04-25T14:06:36.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.475+0000 7f945e600640 1 --2- 172.21.15.86:0/845957225 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f943406f670 0x7f9434071b80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:36.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.475+0000 7f945e600640 1 --2- 172.21.15.86:0/845957225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94580d2200 0x7f945815f0d0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:36.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.475+0000 7f945e600640 1 --2- 172.21.15.86:0/845957225 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94580af390 0x7f945815bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:36.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.475+0000 7f945e600640 1 --2- 172.21.15.86:0/845957225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f945806d560 0x7f945815e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:36.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.475+0000 7f945e600640 1 -- 172.21.15.86:0/845957225 >> 172.21.15.86:0/845957225 conn(0x7f9458073ce0 msgr2=0x7f94580b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:36.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.475+0000 7f945e600640 1 -- 172.21.15.86:0/845957225 shutdown_connections 2024-04-25T14:06:36.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.475+0000 7f945e600640 1 -- 172.21.15.86:0/845957225 wait complete. 2024-04-25T14:06:36.497 INFO:tasks.cephfs_test_runner:test_client_metrics_and_metadata (tasks.cephfs.test_mds_metrics.TestMDSMetrics) ... ok 2024-04-25T14:06:36.498 INFO:tasks.cephfs_test_runner:Starting test: test_delayed_metrics (tasks.cephfs.test_mds_metrics.TestMDSMetrics) 2024-04-25T14:06:36.498 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:06:36.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.643+0000 7f0429400640 1 Processor -- start 2024-04-25T14:06:36.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.643+0000 7f0429400640 1 -- start start 2024-04-25T14:06:36.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.643+0000 7f0429400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f04240cdd70 0x7f04240ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:36.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.643+0000 7f0429400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0424070e20 0x7f0424071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:36.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.643+0000 7f0429400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0424071770 con 0x7f042406e700 2024-04-25T14:06:36.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.643+0000 7f0429400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f042406d940 con 0x7f04240cdd70 2024-04-25T14:06:36.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.643+0000 7f0429400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f042406dc20 con 0x7f0424070e20 2024-04-25T14:06:36.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.643+0000 7f0422800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f04240cdd70 0x7f04240ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:36.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.643+0000 7f0422800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f04240cdd70 0x7f04240ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49420/0 (socket says 172.21.15.86:49420) 2024-04-25T14:06:36.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.643+0000 7f0422800640 1 -- 172.21.15.86:0/2280295710 learned_addr learned my addr 172.21.15.86:0/2280295710 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:36.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.647+0000 7f0419e00640 1 --2- 172.21.15.86:0/2280295710 >> v2:172.21.15.155:3300/0 conn(0x7f0424070e20 0x7f0424071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:36.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.647+0000 7f0422800640 1 -- 172.21.15.86:0/2280295710 >> v2:172.21.15.155:3300/0 conn(0x7f0424070e20 msgr2=0x7f0424071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:36.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.647+0000 7f0422800640 1 --2- 172.21.15.86:0/2280295710 >> v2:172.21.15.155:3300/0 conn(0x7f0424070e20 0x7f0424071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:36.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.647+0000 7f0422800640 1 -- 172.21.15.86:0/2280295710 >> v1:172.21.15.86:6789/0 conn(0x7f042406e700 legacy=0x7f042406eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:36.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.647+0000 7f0422800640 1 -- 172.21.15.86:0/2280295710 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042406df00 con 0x7f04240cdd70 2024-04-25T14:06:36.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.647+0000 7f0422800640 1 --2- 172.21.15.86:0/2280295710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04240cdd70 0x7f04240ce180 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0424073230 tx=0x7f040c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad48702e285ab63b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:36.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.647+0000 7f0421e00640 1 -- 172.21.15.86:0/2280295710 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c0b0070 con 0x7f04240cdd70 2024-04-25T14:06:36.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.647+0000 7f0421e00640 1 -- 172.21.15.86:0/2280295710 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f040c0ab600 con 0x7f04240cdd70 2024-04-25T14:06:36.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.647+0000 7f0421e00640 1 -- 172.21.15.86:0/2280295710 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c0ab9d0 con 0x7f04240cdd70 2024-04-25T14:06:36.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.651+0000 7f0429400640 1 -- 172.21.15.86:0/2280295710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04240cdd70 msgr2=0x7f04240ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:36.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.651+0000 7f0429400640 1 --2- 172.21.15.86:0/2280295710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04240cdd70 0x7f04240ce180 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0424073230 tx=0x7f040c01c490 comp rx=0 tx=0).stop 2024-04-25T14:06:36.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.651+0000 7f0429400640 1 -- 172.21.15.86:0/2280295710 shutdown_connections 2024-04-25T14:06:36.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.651+0000 7f0429400640 1 --2- 172.21.15.86:0/2280295710 >> v2:172.21.15.155:3300/0 conn(0x7f0424070e20 0x7f0424071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:36.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.651+0000 7f0429400640 1 --2- 172.21.15.86:0/2280295710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04240cdd70 0x7f04240ce180 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:36.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.651+0000 7f0429400640 1 -- 172.21.15.86:0/2280295710 >> 172.21.15.86:0/2280295710 conn(0x7f04240cffc0 msgr2=0x7f04240d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:36.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.651+0000 7f0429400640 1 -- 172.21.15.86:0/2280295710 shutdown_connections 2024-04-25T14:06:36.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.651+0000 7f0429400640 1 -- 172.21.15.86:0/2280295710 wait complete. 2024-04-25T14:06:36.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.651+0000 7f0429400640 1 Processor -- start 2024-04-25T14:06:36.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.651+0000 7f0429400640 1 -- start start 2024-04-25T14:06:36.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.651+0000 7f0429400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f042406e700 0x7f0424150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:36.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.655+0000 7f0429400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0424070e20 0x7f04240b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:36.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.655+0000 7f0429400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04240cdd70 0x7f04240b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:36.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.655+0000 7f0429400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f042416cc00 con 0x7f0424070e20 2024-04-25T14:06:36.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.655+0000 7f0429400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0424164380 con 0x7f042406e700 2024-04-25T14:06:36.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.655+0000 7f0429400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f042416cd80 con 0x7f04240cdd70 2024-04-25T14:06:36.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.655+0000 7f0419e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0424070e20 0x7f04240b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:36.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.655+0000 7f0419e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0424070e20 0x7f04240b3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49422/0 (socket says 172.21.15.86:49422) 2024-04-25T14:06:36.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.655+0000 7f0419e00640 1 -- 172.21.15.86:0/2883734488 learned_addr learned my addr 172.21.15.86:0/2883734488 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:36.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.655+0000 7f0422800640 1 --2- 172.21.15.86:0/2883734488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f042406e700 0x7f0424150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:36.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.655+0000 7f0423200640 1 --2- 172.21.15.86:0/2883734488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04240cdd70 0x7f04240b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:36.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.655+0000 7f0419e00640 1 -- 172.21.15.86:0/2883734488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04240cdd70 msgr2=0x7f04240b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:36.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.655+0000 7f0419e00640 1 --2- 172.21.15.86:0/2883734488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04240cdd70 0x7f04240b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:36.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.655+0000 7f0419e00640 1 -- 172.21.15.86:0/2883734488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f042406e700 msgr2=0x7f0424150a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:36.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.655+0000 7f0419e00640 1 --2- 172.21.15.86:0/2883734488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f042406e700 0x7f0424150a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:36.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.655+0000 7f0419e00640 1 -- 172.21.15.86:0/2883734488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0424162740 con 0x7f0424070e20 2024-04-25T14:06:36.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.655+0000 7f0419e00640 1 --2- 172.21.15.86:0/2883734488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0424070e20 0x7f04240b3ea0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f041400d5d0 tx=0x7f041400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:36.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.655+0000 7f041b400640 1 -- 172.21.15.86:0/2883734488 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04140059a0 con 0x7f0424070e20 2024-04-25T14:06:36.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.655+0000 7f041b400640 1 -- 172.21.15.86:0/2883734488 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0414005b40 con 0x7f0424070e20 2024-04-25T14:06:36.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.655+0000 7f0429400640 1 -- 172.21.15.86:0/2883734488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0424157ad0 con 0x7f0424070e20 2024-04-25T14:06:36.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.655+0000 7f041b400640 1 -- 172.21.15.86:0/2883734488 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414005d10 con 0x7f0424070e20 2024-04-25T14:06:36.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.655+0000 7f0429400640 1 -- 172.21.15.86:0/2883734488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0424162ce0 con 0x7f0424070e20 2024-04-25T14:06:36.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.655+0000 7f0429400640 1 -- 172.21.15.86:0/2883734488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04240720e0 con 0x7f0424070e20 2024-04-25T14:06:36.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.655+0000 7f041b400640 1 -- 172.21.15.86:0/2883734488 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f041401b020 con 0x7f0424070e20 2024-04-25T14:06:36.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.659+0000 7f041b400640 1 --2- 172.21.15.86:0/2883734488 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f040006f670 0x7f0400071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:36.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.659+0000 7f041b400640 1 -- 172.21.15.86:0/2883734488 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6367+0+0 (secure 0 0 0) 0x7f041408f5d0 con 0x7f0424070e20 2024-04-25T14:06:36.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.659+0000 7f0422800640 1 --2- 172.21.15.86:0/2883734488 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f040006f670 0x7f0400071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:36.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.659+0000 7f0422800640 1 --2- 172.21.15.86:0/2883734488 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f040006f670 0x7f0400071b80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f040c0ad000 tx=0x7f040c0a6000 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:36.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.663+0000 7f041b400640 1 -- 172.21.15.86:0/2883734488 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f041405bf60 con 0x7f0424070e20 2024-04-25T14:06:36.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.819+0000 7f0429400640 1 -- 172.21.15.86:0/2883734488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0424076bf0 con 0x7f0424070e20 2024-04-25T14:06:36.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.823+0000 7f041b400640 1 -- 172.21.15.86:0/2883734488 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+14820 (secure 0 0 0) 0x7f041405b860 con 0x7f0424070e20 2024-04-25T14:06:36.824 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:36.824 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":65,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:06:35.469493+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6835/2169924270":"2024-04-26T14:06:25.821769+0000","172.21.15.86:6834/2169924270":"2024-04-26T14:06:25.821769+0000","172.21.15.86:0/145108428":"2024-04-26T14:06:19.409429+0000","172.21.15.86:6833/2427179681":"2024-04-26T14:06:19.409429+0000","172.21.15.86:6832/2427179681":"2024-04-26T14:06:19.409429+0000","172.21.15.155:6833/2914930843":"2024-04-26T14:06:32.462884+0000","172.21.15.86:0/1321309855":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/525010030":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/3470358079":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/1177884430":"2024-04-26T14:06:19.409429+0000","172.21.15.86:0/3644218120":"2024-04-26T14:06:19.409429+0000","172.21.15.86:6836/1732615622":"2024-04-26T14:06:00.230367+0000","172.21.15.86:6837/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.86:6837/2418778265":"2024-04-26T14:06:00.222915+0000","172.21.15.86:0/1843278167":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6832/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/2073988773":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6835/716923566":"2024-04-26T14:05:25.452701+0000","172.21.15.155:6835/2914930843":"2024-04-26T14:06:32.462884+0000","172.21.15.86:0/2602032945":"2024-04-26T14:05:21.460855+0000","172.21.15.86:0/3558294881":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6835/615772001":"2024-04-26T14:05:42.882867+0000","172.21.15.86:6837/615772001":"2024-04-26T14:05:42.882867+0000","172.21.15.86:6833/2968869755":"2024-04-26T14:05:21.460855+0000","172.21.15.86:6835/2418778265":"2024-04-26T14:06:00.222915+0000","172.21.15.86:6834/1732615622":"2024-04-26T14:06:00.230367+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:06:36.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.823+0000 7f0429400640 1 -- 172.21.15.86:0/2883734488 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f040006f670 msgr2=0x7f0400071b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:36.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.823+0000 7f0429400640 1 --2- 172.21.15.86:0/2883734488 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f040006f670 0x7f0400071b80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f040c0ad000 tx=0x7f040c0a6000 comp rx=0 tx=0).stop 2024-04-25T14:06:36.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.823+0000 7f0429400640 1 -- 172.21.15.86:0/2883734488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0424070e20 msgr2=0x7f04240b3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:36.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.823+0000 7f0429400640 1 --2- 172.21.15.86:0/2883734488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0424070e20 0x7f04240b3ea0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f041400d5d0 tx=0x7f041400da90 comp rx=0 tx=0).stop 2024-04-25T14:06:36.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.823+0000 7f0429400640 1 -- 172.21.15.86:0/2883734488 shutdown_connections 2024-04-25T14:06:36.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.823+0000 7f0429400640 1 --2- 172.21.15.86:0/2883734488 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f040006f670 0x7f0400071b80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:36.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.823+0000 7f0429400640 1 --2- 172.21.15.86:0/2883734488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04240cdd70 0x7f04240b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:36.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.823+0000 7f0429400640 1 --2- 172.21.15.86:0/2883734488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0424070e20 0x7f04240b3ea0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:36.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.823+0000 7f0429400640 1 --2- 172.21.15.86:0/2883734488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f042406e700 0x7f0424150a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:36.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.823+0000 7f0429400640 1 -- 172.21.15.86:0/2883734488 >> 172.21.15.86:0/2883734488 conn(0x7f04240cffc0 msgr2=0x7f042415d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:36.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.823+0000 7f0429400640 1 -- 172.21.15.86:0/2883734488 shutdown_connections 2024-04-25T14:06:36.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.823+0000 7f0429400640 1 -- 172.21.15.86:0/2883734488 wait complete. 2024-04-25T14:06:36.846 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:06:36.846 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:06:36.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.991+0000 7f3f79600640 1 Processor -- start 2024-04-25T14:06:36.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.991+0000 7f3f79600640 1 -- start start 2024-04-25T14:06:36.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.995+0000 7f3f79600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3f7406f990 0x7f3f7406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:36.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.995+0000 7f3f79600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f740b6530 0x7f3f740b6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:36.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.995+0000 7f3f79600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f740702e0 con 0x7f3f741615c0 2024-04-25T14:06:36.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.995+0000 7f3f79600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3f741622f0 con 0x7f3f7406f990 2024-04-25T14:06:36.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.995+0000 7f3f79600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3f740d1730 con 0x7f3f740b6530 2024-04-25T14:06:36.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.995+0000 7f3f6b400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3f741615c0 0x7f3f741619d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60082/0 (socket says 172.21.15.86:60082) 2024-04-25T14:06:36.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.995+0000 7f3f6b400640 1 -- 172.21.15.86:0/2726451842 learned_addr learned my addr 172.21.15.86:0/2726451842 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:36.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.995+0000 7f3f78c00640 1 --2- 172.21.15.86:0/2726451842 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f740b6530 0x7f3f740b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:36.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.995+0000 7f3f73e00640 1 --2- 172.21.15.86:0/2726451842 >> v2:172.21.15.155:3300/0 conn(0x7f3f7406f990 0x7f3f7406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:36.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.995+0000 7f3f73400640 1 -- 172.21.15.86:0/2726451842 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3840198968 0 0) 0x7f3f5c001120 con 0x7f3f741615c0 2024-04-25T14:06:36.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.995+0000 7f3f73400640 1 -- 172.21.15.86:0/2726451842 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f58003530 con 0x7f3f741615c0 2024-04-25T14:06:36.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.995+0000 7f3f73e00640 1 -- 172.21.15.86:0/2726451842 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f740b6530 msgr2=0x7f3f740b6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:36.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.995+0000 7f3f73e00640 1 --2- 172.21.15.86:0/2726451842 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f740b6530 0x7f3f740b6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:36.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.995+0000 7f3f73e00640 1 -- 172.21.15.86:0/2726451842 >> v1:172.21.15.86:6789/0 conn(0x7f3f741615c0 legacy=0x7f3f741619d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:36.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.995+0000 7f3f73e00640 1 -- 172.21.15.86:0/2726451842 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f740d1a10 con 0x7f3f7406f990 2024-04-25T14:06:36.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.995+0000 7f3f78c00640 1 --2- 172.21.15.86:0/2726451842 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f740b6530 0x7f3f740b6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:37.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.999+0000 7f3f73e00640 1 --2- 172.21.15.86:0/2726451842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f7406f990 0x7f3f7406fda0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3f64009d90 tx=0x7f3f6401c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56db3807c90baa01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:37.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.999+0000 7f3f73400640 1 -- 172.21.15.86:0/2726451842 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f640b2070 con 0x7f3f7406f990 2024-04-25T14:06:37.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.999+0000 7f3f73400640 1 -- 172.21.15.86:0/2726451842 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f640a57a0 con 0x7f3f7406f990 2024-04-25T14:06:37.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.999+0000 7f3f73400640 1 -- 172.21.15.86:0/2726451842 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f640a5b70 con 0x7f3f7406f990 2024-04-25T14:06:37.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.999+0000 7f3f79600640 1 -- 172.21.15.86:0/2726451842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f7406f990 msgr2=0x7f3f7406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:37.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.999+0000 7f3f79600640 1 --2- 172.21.15.86:0/2726451842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f7406f990 0x7f3f7406fda0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3f64009d90 tx=0x7f3f6401c8e0 comp rx=0 tx=0).stop 2024-04-25T14:06:37.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.999+0000 7f3f79600640 1 -- 172.21.15.86:0/2726451842 shutdown_connections 2024-04-25T14:06:37.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.999+0000 7f3f79600640 1 --2- 172.21.15.86:0/2726451842 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f740b6530 0x7f3f740b6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:37.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.999+0000 7f3f79600640 1 --2- 172.21.15.86:0/2726451842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f7406f990 0x7f3f7406fda0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:37.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.999+0000 7f3f79600640 1 -- 172.21.15.86:0/2726451842 >> 172.21.15.86:0/2726451842 conn(0x7f3f740cfea0 msgr2=0x7f3f740d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:37.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.999+0000 7f3f79600640 1 -- 172.21.15.86:0/2726451842 shutdown_connections 2024-04-25T14:06:37.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:36.999+0000 7f3f79600640 1 -- 172.21.15.86:0/2726451842 wait complete. 2024-04-25T14:06:37.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.003+0000 7f3f79600640 1 Processor -- start 2024-04-25T14:06:37.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.003+0000 7f3f79600640 1 -- start start 2024-04-25T14:06:37.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.003+0000 7f3f79600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f7406f990 0x7f3f7414fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:37.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.003+0000 7f3f79600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f740b6530 0x7f3f7415e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:37.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.003+0000 7f3f79600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f741615c0 0x7f3f7406e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:37.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.003+0000 7f3f79600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3f740d2880 con 0x7f3f7406f990 2024-04-25T14:06:37.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.003+0000 7f3f79600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7415df20 con 0x7f3f741615c0 2024-04-25T14:06:37.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.003+0000 7f3f79600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3f7416b170 con 0x7f3f740b6530 2024-04-25T14:06:37.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.003+0000 7f3f73e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f7406f990 0x7f3f7414fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:37.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.003+0000 7f3f73e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f7406f990 0x7f3f7414fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49432/0 (socket says 172.21.15.86:49432) 2024-04-25T14:06:37.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.003+0000 7f3f73e00640 1 -- 172.21.15.86:0/931936333 learned_addr learned my addr 172.21.15.86:0/931936333 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:37.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.003+0000 7f3f78c00640 1 --2- 172.21.15.86:0/931936333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f741615c0 0x7f3f7406e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:37.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.003+0000 7f3f6b400640 1 --2- 172.21.15.86:0/931936333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f740b6530 0x7f3f7415e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:37.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.003+0000 7f3f73e00640 1 -- 172.21.15.86:0/931936333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f740b6530 msgr2=0x7f3f7415e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:37.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.003+0000 7f3f73e00640 1 --2- 172.21.15.86:0/931936333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f740b6530 0x7f3f7415e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:37.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.003+0000 7f3f73e00640 1 -- 172.21.15.86:0/931936333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f741615c0 msgr2=0x7f3f7406e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:37.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.003+0000 7f3f73e00640 1 --2- 172.21.15.86:0/931936333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f741615c0 0x7f3f7406e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:37.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.003+0000 7f3f73e00640 1 -- 172.21.15.86:0/931936333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f74168360 con 0x7f3f7406f990 2024-04-25T14:06:37.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.003+0000 7f3f73e00640 1 --2- 172.21.15.86:0/931936333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f7406f990 0x7f3f7414fa00 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f3f640b0040 tx=0x7f3f64019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:37.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.003+0000 7f3f70c00640 1 -- 172.21.15.86:0/931936333 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f640b2050 con 0x7f3f7406f990 2024-04-25T14:06:37.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.003+0000 7f3f70c00640 1 -- 172.21.15.86:0/931936333 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f640be070 con 0x7f3f7406f990 2024-04-25T14:06:37.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.003+0000 7f3f79600640 1 -- 172.21.15.86:0/931936333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f74168640 con 0x7f3f7406f990 2024-04-25T14:06:37.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.003+0000 7f3f70c00640 1 -- 172.21.15.86:0/931936333 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f64009450 con 0x7f3f7406f990 2024-04-25T14:06:37.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.003+0000 7f3f79600640 1 -- 172.21.15.86:0/931936333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f7406da60 con 0x7f3f7406f990 2024-04-25T14:06:37.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.007+0000 7f3f79600640 1 -- 172.21.15.86:0/931936333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f74076b80 con 0x7f3f7406f990 2024-04-25T14:06:37.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.007+0000 7f3f70c00640 1 -- 172.21.15.86:0/931936333 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f3f640ad850 con 0x7f3f7406f990 2024-04-25T14:06:37.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.007+0000 7f3f70c00640 1 --2- 172.21.15.86:0/931936333 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3f5006f670 0x7f3f50071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:37.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.007+0000 7f3f70c00640 1 -- 172.21.15.86:0/931936333 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6367+0+0 (secure 0 0 0) 0x7f3f6412a950 con 0x7f3f7406f990 2024-04-25T14:06:37.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.007+0000 7f3f6b400640 1 --2- 172.21.15.86:0/931936333 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3f5006f670 0x7f3f50071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:37.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.007+0000 7f3f6b400640 1 --2- 172.21.15.86:0/931936333 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3f5006f670 0x7f3f50071b80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c005890 tx=0x7f3f5c009040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:37.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.011+0000 7f3f70c00640 1 -- 172.21.15.86:0/931936333 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3f64016560 con 0x7f3f7406f990 2024-04-25T14:06:37.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.167+0000 7f3f79600640 1 -- 172.21.15.86:0/931936333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3f740b2c40 con 0x7f3f7406f990 2024-04-25T14:06:37.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.167+0000 7f3f70c00640 1 -- 172.21.15.86:0/931936333 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) ==== 93+0+10 (secure 0 0 0) 0x7f3f640f71c0 con 0x7f3f7406f990 2024-04-25T14:06:37.174 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:06:37.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.171+0000 7f3f79600640 1 -- 172.21.15.86:0/931936333 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3f5006f670 msgr2=0x7f3f50071b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:37.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.171+0000 7f3f79600640 1 --2- 172.21.15.86:0/931936333 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3f5006f670 0x7f3f50071b80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c005890 tx=0x7f3f5c009040 comp rx=0 tx=0).stop 2024-04-25T14:06:37.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.171+0000 7f3f79600640 1 -- 172.21.15.86:0/931936333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f7406f990 msgr2=0x7f3f7414fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:37.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.171+0000 7f3f79600640 1 --2- 172.21.15.86:0/931936333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f7406f990 0x7f3f7414fa00 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f3f640b0040 tx=0x7f3f64019680 comp rx=0 tx=0).stop 2024-04-25T14:06:37.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.171+0000 7f3f79600640 1 -- 172.21.15.86:0/931936333 shutdown_connections 2024-04-25T14:06:37.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.171+0000 7f3f79600640 1 --2- 172.21.15.86:0/931936333 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3f5006f670 0x7f3f50071b80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:37.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.171+0000 7f3f79600640 1 --2- 172.21.15.86:0/931936333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f741615c0 0x7f3f7406e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:37.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.171+0000 7f3f79600640 1 --2- 172.21.15.86:0/931936333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f740b6530 0x7f3f7415e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:37.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.171+0000 7f3f79600640 1 --2- 172.21.15.86:0/931936333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f7406f990 0x7f3f7414fa00 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:37.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.171+0000 7f3f79600640 1 -- 172.21.15.86:0/931936333 >> 172.21.15.86:0/931936333 conn(0x7f3f740cfea0 msgr2=0x7f3f7415cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:37.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.171+0000 7f3f79600640 1 -- 172.21.15.86:0/931936333 shutdown_connections 2024-04-25T14:06:37.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.171+0000 7f3f79600640 1 -- 172.21.15.86:0/931936333 wait complete. 2024-04-25T14:06:37.192 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_delayed_metrics' 2024-04-25T14:06:37.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.335+0000 7f4b4f000640 1 Processor -- start 2024-04-25T14:06:37.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.335+0000 7f4b4f000640 1 -- start start 2024-04-25T14:06:37.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.339+0000 7f4b4f000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4b480d1df0 0x7f4b480b0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:37.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.339+0000 7f4b4f000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4b480b0690 0x7f4b480d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:37.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.339+0000 7f4b4f000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b480cde20 con 0x7f4b481642a0 2024-04-25T14:06:37.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.339+0000 7f4b4f000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4b480cea30 con 0x7f4b480b0690 2024-04-25T14:06:37.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.339+0000 7f4b4f000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f4b480ce100 con 0x7f4b480d1df0 2024-04-25T14:06:37.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.339+0000 7f4b4d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4b480b0690 0x7f4b480d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:37.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.339+0000 7f4b47e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f4b481642a0 0x7f4b480d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60094/0 (socket says 172.21.15.86:60094) 2024-04-25T14:06:37.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.339+0000 7f4b47e00640 1 -- 172.21.15.86:0/74634305 learned_addr learned my addr 172.21.15.86:0/74634305 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:37.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.339+0000 7f4b3f400640 1 --2- 172.21.15.86:0/74634305 >> v2:172.21.15.155:3300/0 conn(0x7f4b480d1df0 0x7f4b480b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:37.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.339+0000 7f4b4d000640 1 -- 172.21.15.86:0/74634305 >> v2:172.21.15.155:3300/0 conn(0x7f4b480d1df0 msgr2=0x7f4b480b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:37.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.339+0000 7f4b4d000640 1 --2- 172.21.15.86:0/74634305 >> v2:172.21.15.155:3300/0 conn(0x7f4b480d1df0 0x7f4b480b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:37.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.339+0000 7f4b4d000640 1 -- 172.21.15.86:0/74634305 >> v1:172.21.15.86:6789/0 conn(0x7f4b481642a0 legacy=0x7f4b480d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:37.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.339+0000 7f4b47400640 1 -- 172.21.15.86:0/74634305 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 336068134 0 0) 0x7f4b480cde20 con 0x7f4b481642a0 2024-04-25T14:06:37.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.339+0000 7f4b4d000640 1 -- 172.21.15.86:0/74634305 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b480ce3e0 con 0x7f4b480b0690 2024-04-25T14:06:37.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.343+0000 7f4b4d000640 1 --2- 172.21.15.86:0/74634305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b480b0690 0x7f4b480d0640 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f4b4806df80 tx=0x7f4b4001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27100e3c81edbe5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:37.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.343+0000 7f4b47400640 1 -- 172.21.15.86:0/74634305 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b400b0070 con 0x7f4b480b0690 2024-04-25T14:06:37.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.343+0000 7f4b47400640 1 -- 172.21.15.86:0/74634305 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b400ab600 con 0x7f4b480b0690 2024-04-25T14:06:37.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.343+0000 7f4b47400640 1 -- 172.21.15.86:0/74634305 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b400ab9d0 con 0x7f4b480b0690 2024-04-25T14:06:37.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.343+0000 7f4b4f000640 1 -- 172.21.15.86:0/74634305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b480b0690 msgr2=0x7f4b480d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:37.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.343+0000 7f4b4f000640 1 --2- 172.21.15.86:0/74634305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b480b0690 0x7f4b480d0640 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f4b4806df80 tx=0x7f4b4001c3f0 comp rx=0 tx=0).stop 2024-04-25T14:06:37.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.343+0000 7f4b4f000640 1 -- 172.21.15.86:0/74634305 shutdown_connections 2024-04-25T14:06:37.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.343+0000 7f4b4f000640 1 --2- 172.21.15.86:0/74634305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b480b0690 0x7f4b480d0640 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:37.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.343+0000 7f4b4f000640 1 --2- 172.21.15.86:0/74634305 >> v2:172.21.15.155:3300/0 conn(0x7f4b480d1df0 0x7f4b480b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:37.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.343+0000 7f4b4f000640 1 -- 172.21.15.86:0/74634305 >> 172.21.15.86:0/74634305 conn(0x7f4b480b4650 msgr2=0x7f4b480b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:37.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.343+0000 7f4b4f000640 1 -- 172.21.15.86:0/74634305 shutdown_connections 2024-04-25T14:06:37.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.343+0000 7f4b4f000640 1 -- 172.21.15.86:0/74634305 wait complete. 2024-04-25T14:06:37.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.347+0000 7f4b4f000640 1 Processor -- start 2024-04-25T14:06:37.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.347+0000 7f4b4f000640 1 -- start start 2024-04-25T14:06:37.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.347+0000 7f4b4f000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b480b0690 0x7f4b4809aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:37.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.347+0000 7f4b4f000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b481642a0 0x7f4b4809b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:37.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.347+0000 7f4b4f000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b4815be10 0x7f4b48161d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:37.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.347+0000 7f4b4f000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b48164830 con 0x7f4b481642a0 2024-04-25T14:06:37.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.347+0000 7f4b4f000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4b481646b0 con 0x7f4b4815be10 2024-04-25T14:06:37.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.347+0000 7f4b4f000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4b48160da0 con 0x7f4b480b0690 2024-04-25T14:06:37.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.347+0000 7f4b3f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b481642a0 0x7f4b4809b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:37.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.347+0000 7f4b3f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b481642a0 0x7f4b4809b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49452/0 (socket says 172.21.15.86:49452) 2024-04-25T14:06:37.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.347+0000 7f4b3f400640 1 -- 172.21.15.86:0/2739930333 learned_addr learned my addr 172.21.15.86:0/2739930333 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:37.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.347+0000 7f4b4d000640 1 --2- 172.21.15.86:0/2739930333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b4815be10 0x7f4b48161d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:37.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.347+0000 7f4b3f400640 1 -- 172.21.15.86:0/2739930333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b480b0690 msgr2=0x7f4b4809aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:37.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.347+0000 7f4b3f400640 1 --2- 172.21.15.86:0/2739930333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b480b0690 0x7f4b4809aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:37.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.347+0000 7f4b3f400640 1 -- 172.21.15.86:0/2739930333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b4815be10 msgr2=0x7f4b48161d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:37.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.347+0000 7f4b3f400640 1 --2- 172.21.15.86:0/2739930333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b4815be10 0x7f4b48161d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:37.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.347+0000 7f4b3f400640 1 -- 172.21.15.86:0/2739930333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4815c5f0 con 0x7f4b481642a0 2024-04-25T14:06:37.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.347+0000 7f4b4d000640 1 --2- 172.21.15.86:0/2739930333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b4815be10 0x7f4b48161d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:37.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.347+0000 7f4b3f400640 1 --2- 172.21.15.86:0/2739930333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b481642a0 0x7f4b4809b6b0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f4b3000d7d0 tx=0x7f4b3000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:37.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.347+0000 7f4b44c00640 1 -- 172.21.15.86:0/2739930333 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3000b6c0 con 0x7f4b481642a0 2024-04-25T14:06:37.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.347+0000 7f4b44c00640 1 -- 172.21.15.86:0/2739930333 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b30019070 con 0x7f4b481642a0 2024-04-25T14:06:37.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.347+0000 7f4b44c00640 1 -- 172.21.15.86:0/2739930333 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3000ba80 con 0x7f4b481642a0 2024-04-25T14:06:37.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.347+0000 7f4b4f000640 1 -- 172.21.15.86:0/2739930333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b4815c8d0 con 0x7f4b481642a0 2024-04-25T14:06:37.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.347+0000 7f4b4f000640 1 -- 172.21.15.86:0/2739930333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b4809d450 con 0x7f4b481642a0 2024-04-25T14:06:37.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.351+0000 7f4b4f000640 1 -- 172.21.15.86:0/2739930333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b4815cdf0 con 0x7f4b481642a0 2024-04-25T14:06:37.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.351+0000 7f4b44c00640 1 -- 172.21.15.86:0/2739930333 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f4b3000bc20 con 0x7f4b481642a0 2024-04-25T14:06:37.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.351+0000 7f4b44c00640 1 --2- 172.21.15.86:0/2739930333 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f4b2406f670 0x7f4b24071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:37.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.351+0000 7f4b44c00640 1 -- 172.21.15.86:0/2739930333 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6367+0+0 (secure 0 0 0) 0x7f4b3008fd10 con 0x7f4b481642a0 2024-04-25T14:06:37.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.351+0000 7f4b47e00640 1 --2- 172.21.15.86:0/2739930333 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f4b2406f670 0x7f4b24071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:37.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.351+0000 7f4b47e00640 1 --2- 172.21.15.86:0/2739930333 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f4b2406f670 0x7f4b24071b80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4b48070ac0 tx=0x7f4b38007cf0 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:37.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.355+0000 7f4b44c00640 1 -- 172.21.15.86:0/2739930333 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4b3005c6a0 con 0x7f4b481642a0 2024-04-25T14:06:37.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:37.571+0000 7f4b4f000640 1 -- 172.21.15.86:0/2739930333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_delayed_metrics"]} v 0) -- 0x7f4b48069d00 con 0x7f4b481642a0 2024-04-25T14:06:38.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.479+0000 7f4b44c00640 1 -- 172.21.15.86:0/2739930333 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_delayed_metrics"]}]=0 v186) ==== 147+0+0 (secure 0 0 0) 0x7f4b3005bfa0 con 0x7f4b481642a0 2024-04-25T14:06:38.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.483+0000 7f4b4f000640 1 -- 172.21.15.86:0/2739930333 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f4b2406f670 msgr2=0x7f4b24071b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:38.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.483+0000 7f4b4f000640 1 --2- 172.21.15.86:0/2739930333 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f4b2406f670 0x7f4b24071b80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4b48070ac0 tx=0x7f4b38007cf0 comp rx=0 tx=0).stop 2024-04-25T14:06:38.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.483+0000 7f4b4f000640 1 -- 172.21.15.86:0/2739930333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b481642a0 msgr2=0x7f4b4809b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:38.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.483+0000 7f4b4f000640 1 --2- 172.21.15.86:0/2739930333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b481642a0 0x7f4b4809b6b0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f4b3000d7d0 tx=0x7f4b3000dc90 comp rx=0 tx=0).stop 2024-04-25T14:06:38.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.483+0000 7f4b4f000640 1 -- 172.21.15.86:0/2739930333 shutdown_connections 2024-04-25T14:06:38.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.483+0000 7f4b4f000640 1 --2- 172.21.15.86:0/2739930333 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f4b2406f670 0x7f4b24071b80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:38.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.483+0000 7f4b4f000640 1 --2- 172.21.15.86:0/2739930333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b4815be10 0x7f4b48161d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:38.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.483+0000 7f4b4f000640 1 --2- 172.21.15.86:0/2739930333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b481642a0 0x7f4b4809b6b0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:38.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.483+0000 7f4b4f000640 1 --2- 172.21.15.86:0/2739930333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b480b0690 0x7f4b4809aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:38.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.483+0000 7f4b4f000640 1 -- 172.21.15.86:0/2739930333 >> 172.21.15.86:0/2739930333 conn(0x7f4b480b4650 msgr2=0x7f4b480b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:38.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.483+0000 7f4b4f000640 1 -- 172.21.15.86:0/2739930333 shutdown_connections 2024-04-25T14:06:38.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.483+0000 7f4b4f000640 1 -- 172.21.15.86:0/2739930333 wait complete. 2024-04-25T14:06:38.508 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T14:06:38.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.659+0000 7f02a5600640 1 Processor -- start 2024-04-25T14:06:38.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.659+0000 7f02a5600640 1 -- start start 2024-04-25T14:06:38.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.659+0000 7f02a5600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f02a00cfca0 0x7f02a006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:38.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.659+0000 7f02a5600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02a00730b0 0x7f02a00734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:38.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.659+0000 7f02a5600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02a0072bd0 con 0x7f02a00d16e0 2024-04-25T14:06:38.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.659+0000 7f02a5600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f02a0070470 con 0x7f02a00cfca0 2024-04-25T14:06:38.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.659+0000 7f02a5600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f02a006f930 con 0x7f02a00730b0 2024-04-25T14:06:38.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.659+0000 7f029ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f02a00cfca0 0x7f02a006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:38.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.659+0000 7f029ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f02a00cfca0 0x7f02a006f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49460/0 (socket says 172.21.15.86:49460) 2024-04-25T14:06:38.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.659+0000 7f029ea00640 1 -- 172.21.15.86:0/307877451 learned_addr learned my addr 172.21.15.86:0/307877451 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:38.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.659+0000 7f0296000640 1 --1- 172.21.15.86:0/307877451 >> v1:172.21.15.86:6789/0 conn(0x7f02a00d16e0 0x7f02a00d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60096/0 (socket says 172.21.15.86:60096) 2024-04-25T14:06:38.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.659+0000 7f029ea00640 1 -- 172.21.15.86:0/307877451 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02a00730b0 msgr2=0x7f02a00734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:38.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.659+0000 7f029ea00640 1 --2- 172.21.15.86:0/307877451 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02a00730b0 0x7f02a00734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:38.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.659+0000 7f029ea00640 1 -- 172.21.15.86:0/307877451 >> v1:172.21.15.86:6789/0 conn(0x7f02a00d16e0 legacy=0x7f02a00d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:38.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.659+0000 7f029ea00640 1 -- 172.21.15.86:0/307877451 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a006fbd0 con 0x7f02a00cfca0 2024-04-25T14:06:38.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.663+0000 7f029ea00640 1 --2- 172.21.15.86:0/307877451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02a00cfca0 0x7f02a006f3f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f0290009870 tx=0x7f029001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a573d0df175f345d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:38.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.663+0000 7f029e000640 1 -- 172.21.15.86:0/307877451 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02900a7070 con 0x7f02a00cfca0 2024-04-25T14:06:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.663+0000 7f029e000640 1 -- 172.21.15.86:0/307877451 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02900b0070 con 0x7f02a00cfca0 2024-04-25T14:06:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.663+0000 7f029e000640 1 -- 172.21.15.86:0/307877451 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02900ab890 con 0x7f02a00cfca0 2024-04-25T14:06:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.663+0000 7f02a5600640 1 -- 172.21.15.86:0/307877451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02a00cfca0 msgr2=0x7f02a006f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.663+0000 7f02a5600640 1 --2- 172.21.15.86:0/307877451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02a00cfca0 0x7f02a006f3f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f0290009870 tx=0x7f029001c560 comp rx=0 tx=0).stop 2024-04-25T14:06:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.663+0000 7f02a5600640 1 -- 172.21.15.86:0/307877451 shutdown_connections 2024-04-25T14:06:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.663+0000 7f02a5600640 1 --2- 172.21.15.86:0/307877451 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02a00730b0 0x7f02a00734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.663+0000 7f02a5600640 1 --2- 172.21.15.86:0/307877451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02a00cfca0 0x7f02a006f3f0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.663+0000 7f02a5600640 1 -- 172.21.15.86:0/307877451 >> 172.21.15.86:0/307877451 conn(0x7f02a00b4650 msgr2=0x7f02a00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.663+0000 7f02a5600640 1 -- 172.21.15.86:0/307877451 shutdown_connections 2024-04-25T14:06:38.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.663+0000 7f02a5600640 1 -- 172.21.15.86:0/307877451 wait complete. 2024-04-25T14:06:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.667+0000 7f02a5600640 1 Processor -- start 2024-04-25T14:06:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.667+0000 7f02a5600640 1 -- start start 2024-04-25T14:06:38.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.667+0000 7f02a5600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02a00730b0 0x7f02a0156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.667+0000 7f02a5600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02a00cfca0 0x7f02a009e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.667+0000 7f02a5600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02a00d16e0 0x7f02a009e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.667+0000 7f02a5600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02a00ce0f0 con 0x7f02a00730b0 2024-04-25T14:06:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.667+0000 7f02a5600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f02a00cdf70 con 0x7f02a00d16e0 2024-04-25T14:06:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.667+0000 7f02a5600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f02a00ce270 con 0x7f02a00cfca0 2024-04-25T14:06:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.667+0000 7f029ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02a00730b0 0x7f02a0156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.667+0000 7f029ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02a00730b0 0x7f02a0156e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49472/0 (socket says 172.21.15.86:49472) 2024-04-25T14:06:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.667+0000 7f029ea00640 1 -- 172.21.15.86:0/2654000665 learned_addr learned my addr 172.21.15.86:0/2654000665 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.667+0000 7f0296000640 1 --2- 172.21.15.86:0/2654000665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02a00cfca0 0x7f02a009e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.667+0000 7f029f400640 1 --2- 172.21.15.86:0/2654000665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02a00d16e0 0x7f02a009e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:38.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.667+0000 7f029ea00640 1 -- 172.21.15.86:0/2654000665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02a00cfca0 msgr2=0x7f02a009e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.667+0000 7f029ea00640 1 --2- 172.21.15.86:0/2654000665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02a00cfca0 0x7f02a009e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.667+0000 7f029ea00640 1 -- 172.21.15.86:0/2654000665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02a00d16e0 msgr2=0x7f02a009e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.667+0000 7f029ea00640 1 --2- 172.21.15.86:0/2654000665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02a00d16e0 0x7f02a009e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.667+0000 7f029ea00640 1 -- 172.21.15.86:0/2654000665 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a009f1a0 con 0x7f02a00730b0 2024-04-25T14:06:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.667+0000 7f029ea00640 1 --2- 172.21.15.86:0/2654000665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02a00730b0 0x7f02a0156e70 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f029001a190 tx=0x7f029001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.667+0000 7f0297400640 1 -- 172.21.15.86:0/2654000665 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02900b4800 con 0x7f02a00730b0 2024-04-25T14:06:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.671+0000 7f0297400640 1 -- 172.21.15.86:0/2654000665 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02900b49c0 con 0x7f02a00730b0 2024-04-25T14:06:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.671+0000 7f02a5600640 1 -- 172.21.15.86:0/2654000665 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a015fa60 con 0x7f02a00730b0 2024-04-25T14:06:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.671+0000 7f0297400640 1 -- 172.21.15.86:0/2654000665 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02900b4db0 con 0x7f02a00730b0 2024-04-25T14:06:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.671+0000 7f02a5600640 1 -- 172.21.15.86:0/2654000665 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a009bd60 con 0x7f02a00730b0 2024-04-25T14:06:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.671+0000 7f02a5600640 1 -- 172.21.15.86:0/2654000665 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02a009f4b0 con 0x7f02a00730b0 2024-04-25T14:06:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.671+0000 7f0297400640 1 -- 172.21.15.86:0/2654000665 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f02900c7390 con 0x7f02a00730b0 2024-04-25T14:06:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.671+0000 7f0297400640 1 --2- 172.21.15.86:0/2654000665 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f027c06f670 0x7f027c071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.671+0000 7f0297400640 1 -- 172.21.15.86:0/2654000665 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6367+0+0 (secure 0 0 0) 0x7f029012a250 con 0x7f02a00730b0 2024-04-25T14:06:38.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.671+0000 7f0296000640 1 --2- 172.21.15.86:0/2654000665 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f027c06f670 0x7f027c071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:38.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.675+0000 7f0296000640 1 --2- 172.21.15.86:0/2654000665 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f027c06f670 0x7f027c071b80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f02a00b2d00 tx=0x7f0288007c50 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:38.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.679+0000 7f0297400640 1 -- 172.21.15.86:0/2654000665 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f02900c4060 con 0x7f02a00730b0 2024-04-25T14:06:38.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.835+0000 7f02a5600640 1 -- 172.21.15.86:0/2654000665 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f02a0069d00 con 0x7f02a00730b0 2024-04-25T14:06:38.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.839+0000 7f0297400640 1 -- 172.21.15.86:0/2654000665 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f0290016150 con 0x7f02a00730b0 2024-04-25T14:06:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.839+0000 7f02a5600640 1 -- 172.21.15.86:0/2654000665 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f027c06f670 msgr2=0x7f027c071b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.839+0000 7f02a5600640 1 --2- 172.21.15.86:0/2654000665 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f027c06f670 0x7f027c071b80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f02a00b2d00 tx=0x7f0288007c50 comp rx=0 tx=0).stop 2024-04-25T14:06:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.839+0000 7f02a5600640 1 -- 172.21.15.86:0/2654000665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02a00730b0 msgr2=0x7f02a0156e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.839+0000 7f02a5600640 1 --2- 172.21.15.86:0/2654000665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02a00730b0 0x7f02a0156e70 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f029001a190 tx=0x7f029001ada0 comp rx=0 tx=0).stop 2024-04-25T14:06:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.839+0000 7f02a5600640 1 -- 172.21.15.86:0/2654000665 shutdown_connections 2024-04-25T14:06:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.839+0000 7f02a5600640 1 --2- 172.21.15.86:0/2654000665 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f027c06f670 0x7f027c071b80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:38.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.839+0000 7f02a5600640 1 --2- 172.21.15.86:0/2654000665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02a00d16e0 0x7f02a009e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:38.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.839+0000 7f02a5600640 1 --2- 172.21.15.86:0/2654000665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02a00cfca0 0x7f02a009e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:38.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.839+0000 7f02a5600640 1 --2- 172.21.15.86:0/2654000665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02a00730b0 0x7f02a0156e70 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:38.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.839+0000 7f02a5600640 1 -- 172.21.15.86:0/2654000665 >> 172.21.15.86:0/2654000665 conn(0x7f02a00b4650 msgr2=0x7f02a006eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:38.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.839+0000 7f02a5600640 1 -- 172.21.15.86:0/2654000665 shutdown_connections 2024-04-25T14:06:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:38.843+0000 7f02a5600640 1 -- 172.21.15.86:0/2654000665 wait complete. 2024-04-25T14:06:38.864 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T14:06:38.865 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:06:38.942 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:06:39.038 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T14:06:39.038 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:06:39.038 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:06:39.047 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2893 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T14:06:39.047 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:06:39.047 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:06:39.057 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4517 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T14:06:39.058 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:06:39.058 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:06:39.099 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2893 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T14:06:39.099 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:06:39.099 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:06:39.111 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4517 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T14:06:39.112 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T14:06:39.112 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:06:39.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.287+0000 7f8cbfa00640 1 Processor -- start 2024-04-25T14:06:39.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.287+0000 7f8cbfa00640 1 -- start start 2024-04-25T14:06:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.287+0000 7f8cbfa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8cb80d1200 0x7f8cb80d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.287+0000 7f8cbfa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cb80b4440 0x7f8cb80d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.287+0000 7f8cbfa00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cb80b4a30 con 0x7f8cb80d3e70 2024-04-25T14:06:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.287+0000 7f8cbfa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8cb80d2850 con 0x7f8cb80d1200 2024-04-25T14:06:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.287+0000 7f8cbfa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8cb80b4d10 con 0x7f8cb80b4440 2024-04-25T14:06:39.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.287+0000 7f8cbd000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8cb80d1200 0x7f8cb80d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:39.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.287+0000 7f8cbd000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8cb80d1200 0x7f8cb80d1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37998/0 (socket says 172.21.15.86:37998) 2024-04-25T14:06:39.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.287+0000 7f8cbd000640 1 -- 172.21.15.86:0/3437740603 learned_addr learned my addr 172.21.15.86:0/3437740603 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:39.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.287+0000 7f8cb7e00640 1 --2- 172.21.15.86:0/3437740603 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cb80b4440 0x7f8cb80d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:39.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.287+0000 7f8cbd000640 1 -- 172.21.15.86:0/3437740603 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cb80b4440 msgr2=0x7f8cb80d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:39.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.287+0000 7f8cbd000640 1 --2- 172.21.15.86:0/3437740603 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cb80b4440 0x7f8cb80d2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:39.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.287+0000 7f8cbd000640 1 -- 172.21.15.86:0/3437740603 >> v1:172.21.15.155:6789/0 conn(0x7f8cb80d3e70 legacy=0x7f8cb80d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:39.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.287+0000 7f8cbd000640 1 -- 172.21.15.86:0/3437740603 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb806d9c0 con 0x7f8cb80d1200 2024-04-25T14:06:39.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.291+0000 7f8cbd000640 1 --2- 172.21.15.86:0/3437740603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cb80d1200 0x7f8cb80d1dd0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8009870 tx=0x7f8ca801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4d7f4b968e84e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:39.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.291+0000 7f8cb7400640 1 -- 172.21.15.86:0/3437740603 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca80b0070 con 0x7f8cb80d1200 2024-04-25T14:06:39.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.291+0000 7f8cb7400640 1 -- 172.21.15.86:0/3437740603 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ca80ab600 con 0x7f8cb80d1200 2024-04-25T14:06:39.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.291+0000 7f8cb7400640 1 -- 172.21.15.86:0/3437740603 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca80ab9d0 con 0x7f8cb80d1200 2024-04-25T14:06:39.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.291+0000 7f8cbfa00640 1 -- 172.21.15.86:0/3437740603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cb80d1200 msgr2=0x7f8cb80d1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:39.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.291+0000 7f8cbfa00640 1 --2- 172.21.15.86:0/3437740603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cb80d1200 0x7f8cb80d1dd0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8009870 tx=0x7f8ca801c490 comp rx=0 tx=0).stop 2024-04-25T14:06:39.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.291+0000 7f8cbfa00640 1 -- 172.21.15.86:0/3437740603 shutdown_connections 2024-04-25T14:06:39.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.291+0000 7f8cbfa00640 1 --2- 172.21.15.86:0/3437740603 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cb80b4440 0x7f8cb80d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:39.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.291+0000 7f8cbfa00640 1 --2- 172.21.15.86:0/3437740603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cb80d1200 0x7f8cb80d1dd0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:39.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.291+0000 7f8cbfa00640 1 -- 172.21.15.86:0/3437740603 >> 172.21.15.86:0/3437740603 conn(0x7f8cb80b72d0 msgr2=0x7f8cb80b76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:39.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.291+0000 7f8cbfa00640 1 -- 172.21.15.86:0/3437740603 shutdown_connections 2024-04-25T14:06:39.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.291+0000 7f8cbfa00640 1 -- 172.21.15.86:0/3437740603 wait complete. 2024-04-25T14:06:39.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.295+0000 7f8cbfa00640 1 Processor -- start 2024-04-25T14:06:39.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.295+0000 7f8cbfa00640 1 -- start start 2024-04-25T14:06:39.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.295+0000 7f8cbfa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cb80b4440 0x7f8cb81515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:39.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.295+0000 7f8cbfa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cb80d1200 0x7f8cb80cf010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:39.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.295+0000 7f8cbfa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cb80d3e70 0x7f8cb8151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:39.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.295+0000 7f8cbfa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb816d050 con 0x7f8cb80d3e70 2024-04-25T14:06:39.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.295+0000 7f8cbfa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb816ced0 con 0x7f8cb80b4440 2024-04-25T14:06:39.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.295+0000 7f8cbfa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8cb816d1d0 con 0x7f8cb80d1200 2024-04-25T14:06:39.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.295+0000 7f8cbda00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cb80d3e70 0x7f8cb8151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:39.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.295+0000 7f8cbda00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cb80d3e70 0x7f8cb8151c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38010/0 (socket says 172.21.15.86:38010) 2024-04-25T14:06:39.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.295+0000 7f8cbda00640 1 -- 172.21.15.86:0/3572393940 learned_addr learned my addr 172.21.15.86:0/3572393940 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:39.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.299+0000 7f8cbd000640 1 --2- 172.21.15.86:0/3572393940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cb80b4440 0x7f8cb81515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:39.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.299+0000 7f8cb7e00640 1 --2- 172.21.15.86:0/3572393940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cb80d1200 0x7f8cb80cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:39.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.299+0000 7f8cbda00640 1 -- 172.21.15.86:0/3572393940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cb80d1200 msgr2=0x7f8cb80cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:39.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.299+0000 7f8cbda00640 1 --2- 172.21.15.86:0/3572393940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cb80d1200 0x7f8cb80cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:39.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.299+0000 7f8cbda00640 1 -- 172.21.15.86:0/3572393940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cb80b4440 msgr2=0x7f8cb81515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:39.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.299+0000 7f8cbda00640 1 --2- 172.21.15.86:0/3572393940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cb80b4440 0x7f8cb81515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:39.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.299+0000 7f8cbda00640 1 -- 172.21.15.86:0/3572393940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb80ccbb0 con 0x7f8cb80d3e70 2024-04-25T14:06:39.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.299+0000 7f8cb7e00640 1 --2- 172.21.15.86:0/3572393940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cb80d1200 0x7f8cb80cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:39.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.299+0000 7f8cbda00640 1 --2- 172.21.15.86:0/3572393940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cb80d3e70 0x7f8cb8151c90 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000c780 tx=0x7f8cb000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:39.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.299+0000 7f8cb4c00640 1 -- 172.21.15.86:0/3572393940 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0016020 con 0x7f8cb80d3e70 2024-04-25T14:06:39.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.299+0000 7f8cb4c00640 1 -- 172.21.15.86:0/3572393940 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cb000aa70 con 0x7f8cb80d3e70 2024-04-25T14:06:39.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.299+0000 7f8cb4c00640 1 -- 172.21.15.86:0/3572393940 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb000ce30 con 0x7f8cb80d3e70 2024-04-25T14:06:39.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.299+0000 7f8cbfa00640 1 -- 172.21.15.86:0/3572393940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cb80cce90 con 0x7f8cb80d3e70 2024-04-25T14:06:39.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.299+0000 7f8cbfa00640 1 -- 172.21.15.86:0/3572393940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cb80cf550 con 0x7f8cb80d3e70 2024-04-25T14:06:39.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.299+0000 7f8cbfa00640 1 -- 172.21.15.86:0/3572393940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cb806ef80 con 0x7f8cb80d3e70 2024-04-25T14:06:39.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.299+0000 7f8cb4c00640 1 -- 172.21.15.86:0/3572393940 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f8cb0019020 con 0x7f8cb80d3e70 2024-04-25T14:06:39.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.303+0000 7f8cb4c00640 1 --2- 172.21.15.86:0/3572393940 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8c9406f670 0x7f8c94071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:39.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.303+0000 7f8cb4c00640 1 -- 172.21.15.86:0/3572393940 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6367+0+0 (secure 0 0 0) 0x7f8cb008fb00 con 0x7f8cb80d3e70 2024-04-25T14:06:39.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.303+0000 7f8cbd000640 1 --2- 172.21.15.86:0/3572393940 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8c9406f670 0x7f8c94071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:39.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.303+0000 7f8cbd000640 1 --2- 172.21.15.86:0/3572393940 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8c9406f670 0x7f8c94071b80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80ad000 tx=0x7f8ca80a6000 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:39.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.307+0000 7f8cb4c00640 1 -- 172.21.15.86:0/3572393940 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8cb0020020 con 0x7f8cb80d3e70 2024-04-25T14:06:39.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.499+0000 7f8cbfa00640 1 -- 172.21.15.86:0/3572393940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8cb8076bf0 con 0x7f8cb80d3e70 2024-04-25T14:06:39.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.499+0000 7f8cb4c00640 1 -- 172.21.15.86:0/3572393940 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+3872 (secure 0 0 0) 0x7f8cb008fe20 con 0x7f8cb80d3e70 2024-04-25T14:06:39.501 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:39.502 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":47,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3545772170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3545772170},{"type":"v1","addr":"172.21.15.155:6834","nonce":3545772170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5980,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/766530004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":766530004},{"type":"v1","addr":"172.21.15.86:6837","nonce":766530004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6414,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2676840983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2676840983},{"type":"v1","addr":"172.21.15.86:6835","nonce":2676840983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6513,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1604641346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1604641346},{"type":"v1","addr":"172.21.15.155:6835","nonce":1604641346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[]} 2024-04-25T14:06:39.502 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 47 2024-04-25T14:06:39.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.499+0000 7f8cbfa00640 1 -- 172.21.15.86:0/3572393940 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8c9406f670 msgr2=0x7f8c94071b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:39.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.499+0000 7f8cbfa00640 1 --2- 172.21.15.86:0/3572393940 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8c9406f670 0x7f8c94071b80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80ad000 tx=0x7f8ca80a6000 comp rx=0 tx=0).stop 2024-04-25T14:06:39.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.499+0000 7f8cbfa00640 1 -- 172.21.15.86:0/3572393940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cb80d3e70 msgr2=0x7f8cb8151c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:39.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.499+0000 7f8cbfa00640 1 --2- 172.21.15.86:0/3572393940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cb80d3e70 0x7f8cb8151c90 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000c780 tx=0x7f8cb000cc40 comp rx=0 tx=0).stop 2024-04-25T14:06:39.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.499+0000 7f8cbfa00640 1 -- 172.21.15.86:0/3572393940 shutdown_connections 2024-04-25T14:06:39.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.499+0000 7f8cbfa00640 1 --2- 172.21.15.86:0/3572393940 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8c9406f670 0x7f8c94071b80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:39.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.499+0000 7f8cbfa00640 1 --2- 172.21.15.86:0/3572393940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cb80d3e70 0x7f8cb8151c90 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:39.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.499+0000 7f8cbfa00640 1 --2- 172.21.15.86:0/3572393940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cb80d1200 0x7f8cb80cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:39.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.499+0000 7f8cbfa00640 1 --2- 172.21.15.86:0/3572393940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cb80b4440 0x7f8cb81515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:39.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.503+0000 7f8cbfa00640 1 -- 172.21.15.86:0/3572393940 >> 172.21.15.86:0/3572393940 conn(0x7f8cb80b72d0 msgr2=0x7f8cb8159340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:39.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.503+0000 7f8cbfa00640 1 -- 172.21.15.86:0/3572393940 shutdown_connections 2024-04-25T14:06:39.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.503+0000 7f8cbfa00640 1 -- 172.21.15.86:0/3572393940 wait complete. 2024-04-25T14:06:39.524 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T14:06:39.524 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T14:06:39.524 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T14:06:39.524 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:06:39.524 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T14:06:39.524 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T14:06:39.525 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T14:06:39.525 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:06:39.525 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T14:06:39.525 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T14:06:39.525 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T14:06:39.525 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:06:39.525 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T14:06:39.525 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T14:06:39.526 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T14:06:39.526 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:06:39.538 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T14:06:39.538 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T14:06:39.543 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T14:06:39.543 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T14:06:39.546 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T14:06:39.546 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T14:06:39.552 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T14:06:39.552 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T14:06:39.589 INFO:tasks.ceph.mds.a:Started 2024-04-25T14:06:39.589 DEBUG:teuthology.parallel:result is None 2024-04-25T14:06:39.591 INFO:tasks.ceph.mds.c:Started 2024-04-25T14:06:39.591 DEBUG:teuthology.parallel:result is None 2024-04-25T14:06:39.601 INFO:tasks.ceph.mds.b:Started 2024-04-25T14:06:39.601 DEBUG:teuthology.parallel:result is None 2024-04-25T14:06:39.603 INFO:tasks.ceph.mds.d:Started 2024-04-25T14:06:39.603 DEBUG:teuthology.parallel:result is None 2024-04-25T14:06:39.603 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T14:06:39.689 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-04-25T14:06:39.692 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-25T14:06:39.706 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2024-04-25T14:06:39.710 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2024-04-25T14:06:39.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.759+0000 7ff9e9800640 1 Processor -- start 2024-04-25T14:06:39.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.759+0000 7ff9e9800640 1 -- start start 2024-04-25T14:06:39.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.759+0000 7ff9e9800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff9e40cfca0 0x7ff9e406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:39.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.759+0000 7ff9e9800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9e40d16e0 0x7ff9e40d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:39.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.759+0000 7ff9e9800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9e40d2030 con 0x7ff9e406c670 2024-04-25T14:06:39.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.759+0000 7ff9e9800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff9e4070470 con 0x7ff9e40cfca0 2024-04-25T14:06:39.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.759+0000 7ff9e9800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff9e4072bd0 con 0x7ff9e40d16e0 2024-04-25T14:06:39.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.759+0000 7ff9e3400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff9e406c670 0x7ff9e406ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42070/0 (socket says 172.21.15.86:42070) 2024-04-25T14:06:39.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.759+0000 7ff9e3400640 1 -- 172.21.15.86:0/3009563439 learned_addr learned my addr 172.21.15.86:0/3009563439 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:39.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.759+0000 7ff9e2000640 1 --2- 172.21.15.86:0/3009563439 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9e40d16e0 0x7ff9e40d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:39.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.759+0000 7ff9e2a00640 1 --2- 172.21.15.86:0/3009563439 >> v2:172.21.15.155:3300/0 conn(0x7ff9e40cfca0 0x7ff9e406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:39.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.759+0000 7ff9e1600640 1 -- 172.21.15.86:0/3009563439 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1528439353 0 0) 0x7ff9dc0014a0 con 0x7ff9e406c670 2024-04-25T14:06:39.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.759+0000 7ff9e1600640 1 -- 172.21.15.86:0/3009563439 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9c8005740 con 0x7ff9e406c670 2024-04-25T14:06:39.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.759+0000 7ff9e2000640 1 -- 172.21.15.86:0/3009563439 >> v2:172.21.15.155:3300/0 conn(0x7ff9e40cfca0 msgr2=0x7ff9e406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:39.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.759+0000 7ff9e2000640 1 --2- 172.21.15.86:0/3009563439 >> v2:172.21.15.155:3300/0 conn(0x7ff9e40cfca0 0x7ff9e406f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:39.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.759+0000 7ff9e2000640 1 -- 172.21.15.86:0/3009563439 >> v1:172.21.15.86:6789/0 conn(0x7ff9e406c670 legacy=0x7ff9e406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:39.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.759+0000 7ff9e2000640 1 -- 172.21.15.86:0/3009563439 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e406f930 con 0x7ff9e40d16e0 2024-04-25T14:06:39.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.763+0000 7ff9e2000640 1 --2- 172.21.15.86:0/3009563439 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9e40d16e0 0x7ff9e40d1af0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff9d00097c0 tx=0x7ff9d001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5773a8a776e92dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:39.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.763+0000 7ff9e1600640 1 -- 172.21.15.86:0/3009563439 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d00b0070 con 0x7ff9e40d16e0 2024-04-25T14:06:39.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.763+0000 7ff9e1600640 1 -- 172.21.15.86:0/3009563439 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d00ab600 con 0x7ff9e40d16e0 2024-04-25T14:06:39.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.763+0000 7ff9e1600640 1 -- 172.21.15.86:0/3009563439 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d00ab9d0 con 0x7ff9e40d16e0 2024-04-25T14:06:39.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.763+0000 7ff9e9800640 1 -- 172.21.15.86:0/3009563439 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9e40d16e0 msgr2=0x7ff9e40d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:39.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.763+0000 7ff9e9800640 1 --2- 172.21.15.86:0/3009563439 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9e40d16e0 0x7ff9e40d1af0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff9d00097c0 tx=0x7ff9d001c4a0 comp rx=0 tx=0).stop 2024-04-25T14:06:39.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.763+0000 7ff9e9800640 1 -- 172.21.15.86:0/3009563439 shutdown_connections 2024-04-25T14:06:39.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.763+0000 7ff9e9800640 1 --2- 172.21.15.86:0/3009563439 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9e40d16e0 0x7ff9e40d1af0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:39.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.763+0000 7ff9e9800640 1 --2- 172.21.15.86:0/3009563439 >> v2:172.21.15.155:3300/0 conn(0x7ff9e40cfca0 0x7ff9e406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:39.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.763+0000 7ff9e9800640 1 -- 172.21.15.86:0/3009563439 >> 172.21.15.86:0/3009563439 conn(0x7ff9e40b4650 msgr2=0x7ff9e40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:39.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.763+0000 7ff9e9800640 1 -- 172.21.15.86:0/3009563439 shutdown_connections 2024-04-25T14:06:39.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.763+0000 7ff9e9800640 1 -- 172.21.15.86:0/3009563439 wait complete. 2024-04-25T14:06:39.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.767+0000 7ff9e9800640 1 Processor -- start 2024-04-25T14:06:39.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.767+0000 7ff9e9800640 1 -- start start 2024-04-25T14:06:39.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.767+0000 7ff9e9800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9e406c670 0x7ff9e409a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:39.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.767+0000 7ff9e9800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9e40cfca0 0x7ff9e41fd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:39.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.767+0000 7ff9e9800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9e40d16e0 0x7ff9e4159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:39.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.767+0000 7ff9e9800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e40ce6f0 con 0x7ff9e406c670 2024-04-25T14:06:39.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.767+0000 7ff9e9800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e40ce570 con 0x7ff9e40cfca0 2024-04-25T14:06:39.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.767+0000 7ff9e9800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff9e40ce870 con 0x7ff9e40d16e0 2024-04-25T14:06:39.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.767+0000 7ff9e2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9e406c670 0x7ff9e409a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:39.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.767+0000 7ff9e2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9e406c670 0x7ff9e409a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38034/0 (socket says 172.21.15.86:38034) 2024-04-25T14:06:39.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.767+0000 7ff9e2a00640 1 -- 172.21.15.86:0/34561691 learned_addr learned my addr 172.21.15.86:0/34561691 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:39.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.767+0000 7ff9e2a00640 1 -- 172.21.15.86:0/34561691 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9e40d16e0 msgr2=0x7ff9e4159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:39.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.767+0000 7ff9e2a00640 1 --2- 172.21.15.86:0/34561691 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9e40d16e0 0x7ff9e4159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:39.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.767+0000 7ff9e2a00640 1 -- 172.21.15.86:0/34561691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9e40cfca0 msgr2=0x7ff9e41fd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:39.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.767+0000 7ff9e2a00640 1 --2- 172.21.15.86:0/34561691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9e40cfca0 0x7ff9e41fd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:39.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.767+0000 7ff9e2a00640 1 -- 172.21.15.86:0/34561691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e41fe140 con 0x7ff9e406c670 2024-04-25T14:06:39.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.767+0000 7ff9e2a00640 1 --2- 172.21.15.86:0/34561691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9e406c670 0x7ff9e409a9e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400e840 tx=0x7ff9d400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:39.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.771+0000 7ff9daa00640 1 -- 172.21.15.86:0/34561691 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4017070 con 0x7ff9e406c670 2024-04-25T14:06:39.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.771+0000 7ff9daa00640 1 -- 172.21.15.86:0/34561691 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d400ee90 con 0x7ff9e406c670 2024-04-25T14:06:39.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.771+0000 7ff9daa00640 1 -- 172.21.15.86:0/34561691 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d400cb60 con 0x7ff9e406c670 2024-04-25T14:06:39.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.771+0000 7ff9e9800640 1 -- 172.21.15.86:0/34561691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9e40b0f40 con 0x7ff9e406c670 2024-04-25T14:06:39.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.771+0000 7ff9e9800640 1 -- 172.21.15.86:0/34561691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9e415e060 con 0x7ff9e406c670 2024-04-25T14:06:39.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.771+0000 7ff9e9800640 1 -- 172.21.15.86:0/34561691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9e409f4b0 con 0x7ff9e406c670 2024-04-25T14:06:39.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.771+0000 7ff9daa00640 1 -- 172.21.15.86:0/34561691 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7ff9d401a070 con 0x7ff9e406c670 2024-04-25T14:06:39.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.771+0000 7ff9daa00640 1 --2- 172.21.15.86:0/34561691 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7ff9c406f640 0x7ff9c4071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:39.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.771+0000 7ff9daa00640 1 -- 172.21.15.86:0/34561691 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6367+0+0 (secure 0 0 0) 0x7ff9d408fc00 con 0x7ff9e406c670 2024-04-25T14:06:39.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.771+0000 7ff9e2000640 1 --2- 172.21.15.86:0/34561691 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7ff9c406f640 0x7ff9c4071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:39.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.775+0000 7ff9e2000640 1 --2- 172.21.15.86:0/34561691 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7ff9c406f640 0x7ff9c4071b50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff9d00ad000 tx=0x7ff9d0004740 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:39.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.775+0000 7ff9daa00640 1 -- 172.21.15.86:0/34561691 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff9d405c4f0 con 0x7ff9e406c670 2024-04-25T14:06:39.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:39.955+0000 7ff9e9800640 1 -- 172.21.15.86:0/34561691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff9e4069d00 con 0x7ff9e406c670 2024-04-25T14:06:40.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:40.855+0000 7ff9daa00640 1 -- 172.21.15.86:0/34561691 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v70) ==== 97+0+0 (secure 0 0 0) 0x7ff9d405bdf0 con 0x7ff9e406c670 2024-04-25T14:06:40.856 INFO:teuthology.orchestra.run.smithi086.stderr: removed all blocklist entries 2024-04-25T14:06:40.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:40.855+0000 7ff9e9800640 1 -- 172.21.15.86:0/34561691 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7ff9c406f640 msgr2=0x7ff9c4071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:40.855+0000 7ff9e9800640 1 --2- 172.21.15.86:0/34561691 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7ff9c406f640 0x7ff9c4071b50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff9d00ad000 tx=0x7ff9d0004740 comp rx=0 tx=0).stop 2024-04-25T14:06:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:40.855+0000 7ff9e9800640 1 -- 172.21.15.86:0/34561691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9e406c670 msgr2=0x7ff9e409a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:40.855+0000 7ff9e9800640 1 --2- 172.21.15.86:0/34561691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9e406c670 0x7ff9e409a9e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400e840 tx=0x7ff9d400ec40 comp rx=0 tx=0).stop 2024-04-25T14:06:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:40.855+0000 7ff9e9800640 1 -- 172.21.15.86:0/34561691 shutdown_connections 2024-04-25T14:06:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:40.855+0000 7ff9e9800640 1 --2- 172.21.15.86:0/34561691 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7ff9c406f640 0x7ff9c4071b50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:40.855+0000 7ff9e9800640 1 --2- 172.21.15.86:0/34561691 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9e40d16e0 0x7ff9e4159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:40.855+0000 7ff9e9800640 1 --2- 172.21.15.86:0/34561691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9e40cfca0 0x7ff9e41fd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:40.855+0000 7ff9e9800640 1 --2- 172.21.15.86:0/34561691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9e406c670 0x7ff9e409a9e0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:40.855+0000 7ff9e9800640 1 -- 172.21.15.86:0/34561691 >> 172.21.15.86:0/34561691 conn(0x7ff9e40b4650 msgr2=0x7ff9e406ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:40.855+0000 7ff9e9800640 1 -- 172.21.15.86:0/34561691 shutdown_connections 2024-04-25T14:06:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:40.855+0000 7ff9e9800640 1 -- 172.21.15.86:0/34561691 wait complete. 2024-04-25T14:06:40.880 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T14:06:41.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.023+0000 7fa85d400640 1 Processor -- start 2024-04-25T14:06:41.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.023+0000 7fa85d400640 1 -- start start 2024-04-25T14:06:41.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa85d400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8580d1df0 0x7fa8580cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:41.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa85d400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa8580cdcc0 0x7fa8580ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:41.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa85d400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa85815e5c0 con 0x7fa858070330 2024-04-25T14:06:41.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa85d400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa85806fd40 con 0x7fa8580cdcc0 2024-04-25T14:06:41.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa85d400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa85815e8a0 con 0x7fa8580d1df0 2024-04-25T14:06:41.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa856800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa858070330 0x7fa8580d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42072/0 (socket says 172.21.15.86:42072) 2024-04-25T14:06:41.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa856800640 1 -- 172.21.15.86:0/1703219222 learned_addr learned my addr 172.21.15.86:0/1703219222 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:41.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa857200640 1 --2- 172.21.15.86:0/1703219222 >> v2:172.21.15.155:3300/0 conn(0x7fa8580cdcc0 0x7fa8580ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:41.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa855e00640 1 --2- 172.21.15.86:0/1703219222 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8580d1df0 0x7fa8580cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:41.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa855400640 1 -- 172.21.15.86:0/1703219222 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2399517532 0 0) 0x7fa85815e5c0 con 0x7fa858070330 2024-04-25T14:06:41.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa855400640 1 -- 172.21.15.86:0/1703219222 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa83c003530 con 0x7fa858070330 2024-04-25T14:06:41.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa855400640 1 -- 172.21.15.86:0/1703219222 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 260832690 0 0) 0x7fa83c003530 con 0x7fa858070330 2024-04-25T14:06:41.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa855400640 1 -- 172.21.15.86:0/1703219222 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8580d1df0 msgr2=0x7fa8580cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:41.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa855400640 1 --2- 172.21.15.86:0/1703219222 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8580d1df0 0x7fa8580cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa855400640 1 -- 172.21.15.86:0/1703219222 >> v2:172.21.15.155:3300/0 conn(0x7fa8580cdcc0 msgr2=0x7fa8580ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:41.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa855400640 1 --2- 172.21.15.86:0/1703219222 >> v2:172.21.15.155:3300/0 conn(0x7fa8580cdcc0 0x7fa8580ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa855400640 1 -- 172.21.15.86:0/1703219222 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85815eb80 con 0x7fa858070330 2024-04-25T14:06:41.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa855400640 1 -- 172.21.15.86:0/1703219222 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fa848001680 con 0x7fa858070330 2024-04-25T14:06:41.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa855400640 1 -- 172.21.15.86:0/1703219222 >> v1:172.21.15.86:6789/0 conn(0x7fa858070330 legacy=0x7fa8580d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:41.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa855400640 1 --2- 172.21.15.86:0/1703219222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa83c003a30 0x7fa83c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:41.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa855400640 1 --2- 172.21.15.86:0/1703219222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa83c004810 0x7fa83c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:41.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa855400640 1 --2- 172.21.15.86:0/1703219222 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa83c005160 0x7fa83c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:41.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa855400640 1 -- 172.21.15.86:0/1703219222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa85815e8a0 con 0x7fa83c004810 2024-04-25T14:06:41.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa855400640 1 -- 172.21.15.86:0/1703219222 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa83c00bf10 con 0x7fa83c003a30 2024-04-25T14:06:41.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa855400640 1 -- 172.21.15.86:0/1703219222 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa83c00c090 con 0x7fa83c005160 2024-04-25T14:06:41.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa855e00640 1 --2- 172.21.15.86:0/1703219222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa83c004810 0x7fa83c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:41.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa856800640 1 --2- 172.21.15.86:0/1703219222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa83c003a30 0x7fa83c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:41.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa857200640 1 --2- 172.21.15.86:0/1703219222 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa83c005160 0x7fa83c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:41.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa855e00640 1 -- 172.21.15.86:0/1703219222 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa83c005160 msgr2=0x7fa83c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:41.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa855e00640 1 --2- 172.21.15.86:0/1703219222 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa83c005160 0x7fa83c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa855e00640 1 -- 172.21.15.86:0/1703219222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa83c003a30 msgr2=0x7fa83c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:41.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa855e00640 1 --2- 172.21.15.86:0/1703219222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa83c003a30 0x7fa83c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa855e00640 1 -- 172.21.15.86:0/1703219222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83c003530 con 0x7fa83c004810 2024-04-25T14:06:41.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa856800640 1 --2- 172.21.15.86:0/1703219222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa83c003a30 0x7fa83c003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:41.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.027+0000 7fa857200640 1 --2- 172.21.15.86:0/1703219222 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa83c005160 0x7fa83c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:41.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.031+0000 7fa855e00640 1 --2- 172.21.15.86:0/1703219222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa83c004810 0x7fa83c004c20 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa844009ae0 tx=0x7fa84401ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61d9f0accf668c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:41.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.031+0000 7fa855400640 1 -- 172.21.15.86:0/1703219222 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8440b2070 con 0x7fa83c004810 2024-04-25T14:06:41.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.031+0000 7fa855400640 1 -- 172.21.15.86:0/1703219222 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8440a5610 con 0x7fa83c004810 2024-04-25T14:06:41.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.031+0000 7fa855400640 1 -- 172.21.15.86:0/1703219222 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8440096b0 con 0x7fa83c004810 2024-04-25T14:06:41.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.031+0000 7fa85d400640 1 -- 172.21.15.86:0/1703219222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa83c004810 msgr2=0x7fa83c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:41.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.031+0000 7fa85d400640 1 --2- 172.21.15.86:0/1703219222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa83c004810 0x7fa83c004c20 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa844009ae0 tx=0x7fa84401ca60 comp rx=0 tx=0).stop 2024-04-25T14:06:41.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.031+0000 7fa857200640 1 -- 172.21.15.86:0/1703219222 reap_dead start 2024-04-25T14:06:41.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.031+0000 7fa85d400640 1 -- 172.21.15.86:0/1703219222 shutdown_connections 2024-04-25T14:06:41.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.031+0000 7fa85d400640 1 -- 172.21.15.86:0/1703219222 >> 172.21.15.86:0/1703219222 conn(0x7fa8580b4650 msgr2=0x7fa8580b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:41.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.031+0000 7fa85d400640 1 -- 172.21.15.86:0/1703219222 shutdown_connections 2024-04-25T14:06:41.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.031+0000 7fa85d400640 1 -- 172.21.15.86:0/1703219222 wait complete. 2024-04-25T14:06:41.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.035+0000 7fa85d400640 1 Processor -- start 2024-04-25T14:06:41.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.035+0000 7fa85d400640 1 -- start start 2024-04-25T14:06:41.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.035+0000 7fa85d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa83c004810 0x7fa85814a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:41.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.035+0000 7fa85d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8580d00c0 0x7fa858072880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:41.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.035+0000 7fa85d400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8580d04d0 0x7fa858072dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:41.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.035+0000 7fa85d400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa8580708c0 con 0x7fa8580d00c0 2024-04-25T14:06:41.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.035+0000 7fa85d400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa858070740 con 0x7fa83c004810 2024-04-25T14:06:41.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.035+0000 7fa85d400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa8580d1df0 con 0x7fa8580d04d0 2024-04-25T14:06:41.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.035+0000 7fa855e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8580d00c0 0x7fa858072880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:41.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.035+0000 7fa855e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8580d00c0 0x7fa858072880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38046/0 (socket says 172.21.15.86:38046) 2024-04-25T14:06:41.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.035+0000 7fa855e00640 1 -- 172.21.15.86:0/170675392 learned_addr learned my addr 172.21.15.86:0/170675392 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:41.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.035+0000 7fa857200640 1 --2- 172.21.15.86:0/170675392 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8580d04d0 0x7fa858072dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:41.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.035+0000 7fa856800640 1 --2- 172.21.15.86:0/170675392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa83c004810 0x7fa85814a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:41.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.035+0000 7fa855e00640 1 -- 172.21.15.86:0/170675392 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8580d04d0 msgr2=0x7fa858072dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:41.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.035+0000 7fa855e00640 1 --2- 172.21.15.86:0/170675392 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8580d04d0 0x7fa858072dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.035+0000 7fa855e00640 1 -- 172.21.15.86:0/170675392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa83c004810 msgr2=0x7fa85814a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:41.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.035+0000 7fa855e00640 1 --2- 172.21.15.86:0/170675392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa83c004810 0x7fa85814a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.035+0000 7fa855e00640 1 -- 172.21.15.86:0/170675392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8580b2220 con 0x7fa8580d00c0 2024-04-25T14:06:41.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.035+0000 7fa855e00640 1 --2- 172.21.15.86:0/170675392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8580d00c0 0x7fa858072880 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa844009ae0 tx=0x7fa83c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:41.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.035+0000 7fa84ea00640 1 -- 172.21.15.86:0/170675392 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8440bd070 con 0x7fa8580d00c0 2024-04-25T14:06:41.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.035+0000 7fa84ea00640 1 -- 172.21.15.86:0/170675392 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8440a5dc0 con 0x7fa8580d00c0 2024-04-25T14:06:41.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.035+0000 7fa85d400640 1 -- 172.21.15.86:0/170675392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8580b1180 con 0x7fa8580d00c0 2024-04-25T14:06:41.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.035+0000 7fa84ea00640 1 -- 172.21.15.86:0/170675392 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8440b2040 con 0x7fa8580d00c0 2024-04-25T14:06:41.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.035+0000 7fa85d400640 1 -- 172.21.15.86:0/170675392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8580b1740 con 0x7fa8580d00c0 2024-04-25T14:06:41.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.039+0000 7fa85d400640 1 -- 172.21.15.86:0/170675392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa858069d00 con 0x7fa8580d00c0 2024-04-25T14:06:41.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.039+0000 7fa84ea00640 1 -- 172.21.15.86:0/170675392 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7fa8440c0070 con 0x7fa8580d00c0 2024-04-25T14:06:41.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.039+0000 7fa84ea00640 1 --2- 172.21.15.86:0/170675392 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fa83806f640 0x7fa838071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:41.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.039+0000 7fa84ea00640 1 -- 172.21.15.86:0/170675392 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5249+0+0 (secure 0 0 0) 0x7fa844128fe0 con 0x7fa8580d00c0 2024-04-25T14:06:41.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.039+0000 7fa856800640 1 --2- 172.21.15.86:0/170675392 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fa83806f640 0x7fa838071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:41.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.039+0000 7fa856800640 1 --2- 172.21.15.86:0/170675392 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fa83806f640 0x7fa838071b50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa858072200 tx=0x7fa848009040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:41.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.043+0000 7fa84ea00640 1 -- 172.21.15.86:0/170675392 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa8440167a0 con 0x7fa8580d00c0 2024-04-25T14:06:41.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.255+0000 7fa85d400640 1 -- 172.21.15.86:0/170675392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fa858076bf0 con 0x7fa8580d00c0 2024-04-25T14:06:41.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.255+0000 7fa84ea00640 1 -- 172.21.15.86:0/170675392 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) ==== 80+0+5229 (secure 0 0 0) 0x7fa8440f56e0 con 0x7fa8580d00c0 2024-04-25T14:06:41.257 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:41.258 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:06:41.258 INFO:teuthology.orchestra.run.smithi086.stdout: "auth_dump": [ 2024-04-25T14:06:41.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:06:41.258 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.a", 2024-04-25T14:06:41.258 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmWk5fAxAAKEBKhWewpl23MUcvMi8qDg==", 2024-04-25T14:06:41.258 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:06:41.258 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:06:41.258 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:06:41.258 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:06:41.258 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:06:41.258 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:06:41.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:06:41.258 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:06:41.259 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.b", 2024-04-25T14:06:41.259 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmDhzOEBAABNys/NAEhucQ7A+HzQlzQw==", 2024-04-25T14:06:41.259 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:06:41.259 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:06:41.259 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:06:41.259 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:06:41.259 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:06:41.261 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:06:41.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:06:41.261 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:06:41.261 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.c", 2024-04-25T14:06:41.262 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmA/hNCxAAclDGbIjLfjqyXn0MwX/IkA==", 2024-04-25T14:06:41.262 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:06:41.262 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:06:41.262 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:06:41.262 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:06:41.262 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:06:41.262 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:06:41.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:06:41.262 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:06:41.262 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.d", 2024-04-25T14:06:41.262 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmemCcGBAAGrCx6C850qRJSWYjebxOBA==", 2024-04-25T14:06:41.262 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:06:41.262 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:06:41.263 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:06:41.263 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:06:41.263 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:06:41.263 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:06:41.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:06:41.263 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:06:41.263 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.0", 2024-04-25T14:06:41.263 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCOYipm9etgGxAAemg9CqXN0ShduDtDuKF2Zw==", 2024-04-25T14:06:41.263 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:06:41.263 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:06:41.263 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:06:41.263 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:06:41.263 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:06:41.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:06:41.264 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:06:41.264 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.1", 2024-04-25T14:06:41.264 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCRYipmBCiNJhAABAWHk8NEGVvqieL2klGhmA==", 2024-04-25T14:06:41.264 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:06:41.264 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:06:41.264 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:06:41.264 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:06:41.264 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:06:41.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:06:41.264 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:06:41.264 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.2", 2024-04-25T14:06:41.264 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCUYipm/XGYLxAAPXsw1svKseFzFiuDUjAsJQ==", 2024-04-25T14:06:41.264 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:06:41.265 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:06:41.265 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:06:41.265 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:06:41.265 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:06:41.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:06:41.265 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:06:41.265 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.3", 2024-04-25T14:06:41.265 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCXYipmlSRaORAAwpKyzZDdVYHWl2pAsQjmpw==", 2024-04-25T14:06:41.265 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:06:41.265 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:06:41.265 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:06:41.265 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:06:41.265 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:06:41.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:06:41.266 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:06:41.266 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.4", 2024-04-25T14:06:41.266 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCdYipmBI3FMBAAV2NdggfRSFnQvrcOC2YqiA==", 2024-04-25T14:06:41.266 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:06:41.266 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:06:41.266 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:06:41.266 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:06:41.266 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:06:41.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:06:41.266 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:06:41.266 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.5", 2024-04-25T14:06:41.266 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQChYipmctYxABAABwmOMgyYuOfRWD9KdHILng==", 2024-04-25T14:06:41.266 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:06:41.267 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:06:41.267 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:06:41.267 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:06:41.267 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:06:41.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:06:41.267 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:06:41.267 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.6", 2024-04-25T14:06:41.267 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCkYipmSL7TCRAAhFMWg1AQm54bB+x1GDcusA==", 2024-04-25T14:06:41.267 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:06:41.267 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:06:41.267 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:06:41.267 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:06:41.267 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:06:41.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:06:41.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:06:41.268 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.7", 2024-04-25T14:06:41.268 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCnYipmZ4F+FRAAEGaZN+byv/qJUFZb7WxTFg==", 2024-04-25T14:06:41.268 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:06:41.268 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:06:41.268 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:06:41.268 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:06:41.268 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:06:41.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:06:41.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:06:41.268 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.0", 2024-04-25T14:06:41.268 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipm5u4xHRAACDFe3Eb4KLrsdLj4eFe7mA==", 2024-04-25T14:06:41.268 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:06:41.269 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:06:41.269 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-25T14:06:41.269 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw pool=fs2_data" 2024-04-25T14:06:41.269 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:06:41.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:06:41.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:06:41.269 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.1", 2024-04-25T14:06:41.269 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmDp1hJRAAQkBc5PdQoMTT7W4HahkTyg==", 2024-04-25T14:06:41.269 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:06:41.269 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:06:41.269 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-25T14:06:41.269 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw pool=fs2_data" 2024-04-25T14:06:41.269 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:06:41.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:06:41.270 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:06:41.270 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.2", 2024-04-25T14:06:41.270 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmhbhGIRAAxao/J0WY53IITLETFgwIVw==", 2024-04-25T14:06:41.270 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:06:41.270 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:06:41.270 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-25T14:06:41.270 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw pool=fs2_data" 2024-04-25T14:06:41.270 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:06:41.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:06:41.270 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:06:41.270 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.3", 2024-04-25T14:06:41.270 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmJsCBKRAAzXn2qDtblIpqt7U/TUUlyA==", 2024-04-25T14:06:41.270 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:06:41.271 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:06:41.271 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-25T14:06:41.271 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw pool=fs2_data" 2024-04-25T14:06:41.271 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:06:41.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:06:41.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:06:41.271 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.admin", 2024-04-25T14:06:41.271 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCKYipmTqbVIBAA1VpOtj24f2ibVEIT4EJwmw==", 2024-04-25T14:06:41.271 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:06:41.271 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-25T14:06:41.271 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:06:41.271 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:06:41.271 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:06:41.271 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:06:41.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:06:41.272 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:06:41.272 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.x", 2024-04-25T14:06:41.272 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCKYipmKMzAORAAtRyTNJ5HWi2XwuamoqwB0Q==", 2024-04-25T14:06:41.272 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:06:41.272 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-25T14:06:41.272 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-25T14:06:41.272 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:06:41.272 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:06:41.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:06:41.272 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:06:41.272 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.y", 2024-04-25T14:06:41.272 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCKYipmC7A7NRAAEwVNA0yxt3fwdHXrsV0dvg==", 2024-04-25T14:06:41.272 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:06:41.272 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-25T14:06:41.273 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-25T14:06:41.273 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:06:41.273 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:06:41.273 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:06:41.273 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-25T14:06:41.273 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:06:41.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.259+0000 7fa85d400640 1 -- 172.21.15.86:0/170675392 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fa83806f640 msgr2=0x7fa838071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:41.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.259+0000 7fa85d400640 1 --2- 172.21.15.86:0/170675392 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fa83806f640 0x7fa838071b50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa858072200 tx=0x7fa848009040 comp rx=0 tx=0).stop 2024-04-25T14:06:41.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.259+0000 7fa85d400640 1 -- 172.21.15.86:0/170675392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8580d00c0 msgr2=0x7fa858072880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:41.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.259+0000 7fa85d400640 1 --2- 172.21.15.86:0/170675392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8580d00c0 0x7fa858072880 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa844009ae0 tx=0x7fa83c005840 comp rx=0 tx=0).stop 2024-04-25T14:06:41.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.259+0000 7fa85d400640 1 -- 172.21.15.86:0/170675392 shutdown_connections 2024-04-25T14:06:41.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.259+0000 7fa85d400640 1 --2- 172.21.15.86:0/170675392 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fa83806f640 0x7fa838071b50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.259+0000 7fa85d400640 1 --2- 172.21.15.86:0/170675392 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8580d04d0 0x7fa858072dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.259+0000 7fa85d400640 1 --2- 172.21.15.86:0/170675392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8580d00c0 0x7fa858072880 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.259+0000 7fa85d400640 1 --2- 172.21.15.86:0/170675392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa83c004810 0x7fa85814a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.259+0000 7fa85d400640 1 -- 172.21.15.86:0/170675392 >> 172.21.15.86:0/170675392 conn(0x7fa8580b4650 msgr2=0x7fa8580b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:41.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.259+0000 7fa85d400640 1 -- 172.21.15.86:0/170675392 shutdown_connections 2024-04-25T14:06:41.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.259+0000 7fa85d400640 1 -- 172.21.15.86:0/170675392 wait complete. 2024-04-25T14:06:41.279 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:06:41.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.423+0000 7f2769e00640 1 Processor -- start 2024-04-25T14:06:41.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.423+0000 7f2769e00640 1 -- start start 2024-04-25T14:06:41.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.423+0000 7f2769e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f27640cfca0 0x7f276406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:41.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.423+0000 7f2769e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f27640d16e0 0x7f27640d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:41.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.423+0000 7f2769e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f276406c670 0x7f276406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:41.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.423+0000 7f2769e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2764070470 con 0x7f27640cfca0 2024-04-25T14:06:41.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.423+0000 7f2769e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f27640d2030 con 0x7f27640d16e0 2024-04-25T14:06:41.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.423+0000 7f2769e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2764072bd0 con 0x7f276406c670 2024-04-25T14:06:41.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.427+0000 7f2762a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f27640cfca0 0x7f276406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:41.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.427+0000 7f2762a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f27640cfca0 0x7f276406f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38054/0 (socket says 172.21.15.86:38054) 2024-04-25T14:06:41.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.427+0000 7f2762a00640 1 -- 172.21.15.86:0/2047693203 learned_addr learned my addr 172.21.15.86:0/2047693203 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:41.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.427+0000 7f2762a00640 1 -- 172.21.15.86:0/2047693203 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f276406c670 msgr2=0x7f276406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.427+0000 7f2762a00640 1 --2- 172.21.15.86:0/2047693203 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f276406c670 0x7f276406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.427+0000 7f2762a00640 1 -- 172.21.15.86:0/2047693203 >> v2:172.21.15.155:3300/0 conn(0x7f27640d16e0 msgr2=0x7f27640d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.427+0000 7f2762a00640 1 --2- 172.21.15.86:0/2047693203 >> v2:172.21.15.155:3300/0 conn(0x7f27640d16e0 0x7f27640d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.427+0000 7f2762a00640 1 -- 172.21.15.86:0/2047693203 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276406f930 con 0x7f27640cfca0 2024-04-25T14:06:41.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.427+0000 7f2762a00640 1 --2- 172.21.15.86:0/2047693203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27640cfca0 0x7f276406f3f0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f27640729c0 tx=0x7f275401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1178055777e18f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:41.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.427+0000 7f2762000640 1 -- 172.21.15.86:0/2047693203 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27540a5680 con 0x7f27640cfca0 2024-04-25T14:06:41.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.427+0000 7f2762000640 1 -- 172.21.15.86:0/2047693203 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27540a5820 con 0x7f27640cfca0 2024-04-25T14:06:41.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.427+0000 7f2762000640 1 -- 172.21.15.86:0/2047693203 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27540a5bf0 con 0x7f27640cfca0 2024-04-25T14:06:41.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.431+0000 7f2769e00640 1 -- 172.21.15.86:0/2047693203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27640cfca0 msgr2=0x7f276406f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:41.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.431+0000 7f2769e00640 1 --2- 172.21.15.86:0/2047693203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27640cfca0 0x7f276406f3f0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f27640729c0 tx=0x7f275401c9f0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.431+0000 7f2769e00640 1 -- 172.21.15.86:0/2047693203 shutdown_connections 2024-04-25T14:06:41.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.431+0000 7f2769e00640 1 --2- 172.21.15.86:0/2047693203 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f276406c670 0x7f276406ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.431+0000 7f2769e00640 1 --2- 172.21.15.86:0/2047693203 >> v2:172.21.15.155:3300/0 conn(0x7f27640d16e0 0x7f27640d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.431+0000 7f2769e00640 1 --2- 172.21.15.86:0/2047693203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27640cfca0 0x7f276406f3f0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.431+0000 7f2769e00640 1 -- 172.21.15.86:0/2047693203 >> 172.21.15.86:0/2047693203 conn(0x7f27640b4650 msgr2=0x7f27640b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:41.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.431+0000 7f2769e00640 1 -- 172.21.15.86:0/2047693203 shutdown_connections 2024-04-25T14:06:41.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.431+0000 7f2769e00640 1 -- 172.21.15.86:0/2047693203 wait complete. 2024-04-25T14:06:41.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.431+0000 7f2769e00640 1 Processor -- start 2024-04-25T14:06:41.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.431+0000 7f2769e00640 1 -- start start 2024-04-25T14:06:41.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.435+0000 7f2769e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f276406c670 0x7f276415a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:41.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.435+0000 7f2769e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27640cfca0 0x7f2764156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:41.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.435+0000 7f2769e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27640d16e0 0x7f27641f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:41.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.435+0000 7f2769e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f27640ce570 con 0x7f276406c670 2024-04-25T14:06:41.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.435+0000 7f2769e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f276406d3d0 con 0x7f27640cfca0 2024-04-25T14:06:41.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.435+0000 7f2769e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f27640ce6f0 con 0x7f27640d16e0 2024-04-25T14:06:41.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.435+0000 7f2762a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f276406c670 0x7f276415a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:41.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.435+0000 7f2762a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f276406c670 0x7f276415a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38060/0 (socket says 172.21.15.86:38060) 2024-04-25T14:06:41.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.435+0000 7f2762a00640 1 -- 172.21.15.86:0/306185242 learned_addr learned my addr 172.21.15.86:0/306185242 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:41.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.435+0000 7f2762a00640 1 -- 172.21.15.86:0/306185242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27640d16e0 msgr2=0x7f27641f9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:41.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.435+0000 7f2762a00640 1 --2- 172.21.15.86:0/306185242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27640d16e0 0x7f27641f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.435+0000 7f2762a00640 1 -- 172.21.15.86:0/306185242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27640cfca0 msgr2=0x7f2764156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:41.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.435+0000 7f2762a00640 1 --2- 172.21.15.86:0/306185242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27640cfca0 0x7f2764156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.435+0000 7f2762a00640 1 -- 172.21.15.86:0/306185242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27640cf210 con 0x7f276406c670 2024-04-25T14:06:41.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.435+0000 7f2762a00640 1 --2- 172.21.15.86:0/306185242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f276406c670 0x7f276415a380 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f27540af040 tx=0x7f2754019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:41.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.435+0000 7f275be00640 1 -- 172.21.15.86:0/306185242 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27540bc020 con 0x7f276406c670 2024-04-25T14:06:41.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.435+0000 7f275be00640 1 -- 172.21.15.86:0/306185242 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27540b8670 con 0x7f276406c670 2024-04-25T14:06:41.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.435+0000 7f275be00640 1 -- 172.21.15.86:0/306185242 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27540b8a60 con 0x7f276406c670 2024-04-25T14:06:41.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.435+0000 7f2769e00640 1 -- 172.21.15.86:0/306185242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f276415fd30 con 0x7f276406c670 2024-04-25T14:06:41.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.435+0000 7f2769e00640 1 -- 172.21.15.86:0/306185242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27641602b0 con 0x7f276406c670 2024-04-25T14:06:41.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.435+0000 7f2769e00640 1 -- 172.21.15.86:0/306185242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f276409f4b0 con 0x7f276406c670 2024-04-25T14:06:41.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.439+0000 7f275be00640 1 -- 172.21.15.86:0/306185242 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f27540bf020 con 0x7f276406c670 2024-04-25T14:06:41.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.439+0000 7f275be00640 1 --2- 172.21.15.86:0/306185242 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f274006f640 0x7f2740071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:41.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.439+0000 7f275be00640 1 -- 172.21.15.86:0/306185242 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5249+0+0 (secure 0 0 0) 0x7f275412a490 con 0x7f276406c670 2024-04-25T14:06:41.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.439+0000 7f275a000640 1 --2- 172.21.15.86:0/306185242 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f274006f640 0x7f2740071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:41.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.439+0000 7f275a000640 1 --2- 172.21.15.86:0/306185242 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f274006f640 0x7f2740071b50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f276406e110 tx=0x7f274c006c00 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:41.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.443+0000 7f275be00640 1 -- 172.21.15.86:0/306185242 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f27540c6030 con 0x7f276406c670 2024-04-25T14:06:41.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.603+0000 7f2769e00640 1 -- 172.21.15.86:0/306185242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2764069d00 con 0x7f276406c670 2024-04-25T14:06:41.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.603+0000 7f275be00640 1 -- 172.21.15.86:0/306185242 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+13165 (secure 0 0 0) 0x7f27540165e0 con 0x7f276406c670 2024-04-25T14:06:41.606 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:41.606 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":70,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:06:40.856521+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:06:41.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.607+0000 7f2769e00640 1 -- 172.21.15.86:0/306185242 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f274006f640 msgr2=0x7f2740071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:41.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.607+0000 7f2769e00640 1 --2- 172.21.15.86:0/306185242 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f274006f640 0x7f2740071b50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f276406e110 tx=0x7f274c006c00 comp rx=0 tx=0).stop 2024-04-25T14:06:41.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.607+0000 7f2769e00640 1 -- 172.21.15.86:0/306185242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f276406c670 msgr2=0x7f276415a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:41.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.607+0000 7f2769e00640 1 --2- 172.21.15.86:0/306185242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f276406c670 0x7f276415a380 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f27540af040 tx=0x7f2754019790 comp rx=0 tx=0).stop 2024-04-25T14:06:41.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.607+0000 7f2769e00640 1 -- 172.21.15.86:0/306185242 shutdown_connections 2024-04-25T14:06:41.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.607+0000 7f2769e00640 1 --2- 172.21.15.86:0/306185242 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f274006f640 0x7f2740071b50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.607+0000 7f2769e00640 1 --2- 172.21.15.86:0/306185242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27640d16e0 0x7f27641f9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.607+0000 7f2769e00640 1 --2- 172.21.15.86:0/306185242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27640cfca0 0x7f2764156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.607+0000 7f2769e00640 1 --2- 172.21.15.86:0/306185242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f276406c670 0x7f276415a380 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.607+0000 7f2769e00640 1 -- 172.21.15.86:0/306185242 >> 172.21.15.86:0/306185242 conn(0x7f27640b4650 msgr2=0x7f27641578f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:41.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.607+0000 7f2769e00640 1 -- 172.21.15.86:0/306185242 shutdown_connections 2024-04-25T14:06:41.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.607+0000 7f2769e00640 1 -- 172.21.15.86:0/306185242 wait complete. 2024-04-25T14:06:41.629 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:06:41.629 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:06:41.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.775+0000 7f8bc8000640 1 Processor -- start 2024-04-25T14:06:41.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.775+0000 7f8bc8000640 1 -- start start 2024-04-25T14:06:41.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.779+0000 7f8bc8000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bc00d16e0 0x7f8bc00d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:41.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.779+0000 7f8bc8000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8bc00730b0 0x7f8bc00734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:41.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.779+0000 7f8bc8000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bc00d2030 con 0x7f8bc00cfca0 2024-04-25T14:06:41.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.779+0000 7f8bc8000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8bc0070470 con 0x7f8bc00730b0 2024-04-25T14:06:41.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.779+0000 7f8bc8000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8bc0072bd0 con 0x7f8bc00d16e0 2024-04-25T14:06:41.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.779+0000 7f8bc6000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8bc00730b0 0x7f8bc00734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:41.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.779+0000 7f8bc6000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8bc00730b0 0x7f8bc00734c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38072/0 (socket says 172.21.15.86:38072) 2024-04-25T14:06:41.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.779+0000 7f8bc6000640 1 -- 172.21.15.86:0/4126544320 learned_addr learned my addr 172.21.15.86:0/4126544320 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:41.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.779+0000 7f8bc5600640 1 --1- 172.21.15.86:0/4126544320 >> v1:172.21.15.86:6789/0 conn(0x7f8bc00cfca0 0x7f8bc006f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42080/0 (socket says 172.21.15.86:42080) 2024-04-25T14:06:41.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.779+0000 7f8bc4c00640 1 --2- 172.21.15.86:0/4126544320 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bc00d16e0 0x7f8bc00d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:41.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.779+0000 7f8bc6000640 1 -- 172.21.15.86:0/4126544320 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bc00d16e0 msgr2=0x7f8bc00d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:41.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.779+0000 7f8bc6000640 1 --2- 172.21.15.86:0/4126544320 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bc00d16e0 0x7f8bc00d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.779+0000 7f8bc6000640 1 -- 172.21.15.86:0/4126544320 >> v1:172.21.15.86:6789/0 conn(0x7f8bc00cfca0 legacy=0x7f8bc006f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:41.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.779+0000 7f8bc6000640 1 -- 172.21.15.86:0/4126544320 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc006fbd0 con 0x7f8bc00730b0 2024-04-25T14:06:41.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.779+0000 7f8bbfe00640 1 -- 172.21.15.86:0/4126544320 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 602320421 0 0) 0x7f8bb4001120 con 0x7f8bc00cfca0 2024-04-25T14:06:41.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.779+0000 7f8bc4c00640 1 --2- 172.21.15.86:0/4126544320 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bc00d16e0 0x7f8bc00d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:41.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.783+0000 7f8bc6000640 1 --2- 172.21.15.86:0/4126544320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bc00730b0 0x7f8bc00734c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009e50 tx=0x7f8bb801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69caf5b97099d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:41.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.783+0000 7f8bbfe00640 1 -- 172.21.15.86:0/4126544320 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb80a5810 con 0x7f8bc00730b0 2024-04-25T14:06:41.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.783+0000 7f8bbfe00640 1 -- 172.21.15.86:0/4126544320 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bb80a59b0 con 0x7f8bc00730b0 2024-04-25T14:06:41.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.783+0000 7f8bbfe00640 1 -- 172.21.15.86:0/4126544320 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb80a5d80 con 0x7f8bc00730b0 2024-04-25T14:06:41.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.783+0000 7f8bc8000640 1 -- 172.21.15.86:0/4126544320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bc00730b0 msgr2=0x7f8bc00734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:41.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.783+0000 7f8bc8000640 1 --2- 172.21.15.86:0/4126544320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bc00730b0 0x7f8bc00734c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009e50 tx=0x7f8bb801ca30 comp rx=0 tx=0).stop 2024-04-25T14:06:41.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.783+0000 7f8bc8000640 1 -- 172.21.15.86:0/4126544320 shutdown_connections 2024-04-25T14:06:41.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.783+0000 7f8bc8000640 1 --2- 172.21.15.86:0/4126544320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bc00730b0 0x7f8bc00734c0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.783+0000 7f8bc8000640 1 --2- 172.21.15.86:0/4126544320 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bc00d16e0 0x7f8bc00d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.783+0000 7f8bc8000640 1 -- 172.21.15.86:0/4126544320 >> 172.21.15.86:0/4126544320 conn(0x7f8bc00b4650 msgr2=0x7f8bc00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:41.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.783+0000 7f8bc8000640 1 -- 172.21.15.86:0/4126544320 shutdown_connections 2024-04-25T14:06:41.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.783+0000 7f8bc8000640 1 -- 172.21.15.86:0/4126544320 wait complete. 2024-04-25T14:06:41.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.783+0000 7f8bc8000640 1 Processor -- start 2024-04-25T14:06:41.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.783+0000 7f8bc8000640 1 -- start start 2024-04-25T14:06:41.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.787+0000 7f8bc8000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bc00730b0 0x7f8bc015c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:41.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.787+0000 7f8bc8000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bc00cfca0 0x7f8bc0099a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:41.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.787+0000 7f8bc8000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bc00d16e0 0x7f8bc0099f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:41.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.787+0000 7f8bc8000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc00ce130 con 0x7f8bc00730b0 2024-04-25T14:06:41.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.787+0000 7f8bc8000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc00cdfb0 con 0x7f8bc00cfca0 2024-04-25T14:06:41.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.787+0000 7f8bc8000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8bc00ce2b0 con 0x7f8bc00d16e0 2024-04-25T14:06:41.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.787+0000 7f8bc5600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bc00730b0 0x7f8bc015c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:41.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.787+0000 7f8bc5600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bc00730b0 0x7f8bc015c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38076/0 (socket says 172.21.15.86:38076) 2024-04-25T14:06:41.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.787+0000 7f8bc5600640 1 -- 172.21.15.86:0/3099383721 learned_addr learned my addr 172.21.15.86:0/3099383721 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:41.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.787+0000 7f8bc5600640 1 -- 172.21.15.86:0/3099383721 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bc00d16e0 msgr2=0x7f8bc0099f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:41.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.787+0000 7f8bc5600640 1 --2- 172.21.15.86:0/3099383721 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bc00d16e0 0x7f8bc0099f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.787+0000 7f8bc5600640 1 -- 172.21.15.86:0/3099383721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bc00cfca0 msgr2=0x7f8bc0099a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:41.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.787+0000 7f8bc5600640 1 --2- 172.21.15.86:0/3099383721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bc00cfca0 0x7f8bc0099a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.787+0000 7f8bc5600640 1 -- 172.21.15.86:0/3099383721 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc009a620 con 0x7f8bc00730b0 2024-04-25T14:06:41.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.787+0000 7f8bc5600640 1 --2- 172.21.15.86:0/3099383721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bc00730b0 0x7f8bc015c000 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400c480 tx=0x7f8bb400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:41.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.787+0000 7f8bbd600640 1 -- 172.21.15.86:0/3099383721 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4018070 con 0x7f8bc00730b0 2024-04-25T14:06:41.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.787+0000 7f8bbd600640 1 -- 172.21.15.86:0/3099383721 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bb40074e0 con 0x7f8bc00730b0 2024-04-25T14:06:41.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.787+0000 7f8bc8000640 1 -- 172.21.15.86:0/3099383721 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc01fdaa0 con 0x7f8bc00730b0 2024-04-25T14:06:41.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.787+0000 7f8bbd600640 1 -- 172.21.15.86:0/3099383721 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb40049f0 con 0x7f8bc00730b0 2024-04-25T14:06:41.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.787+0000 7f8bc8000640 1 -- 172.21.15.86:0/3099383721 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc016b360 con 0x7f8bc00730b0 2024-04-25T14:06:41.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.787+0000 7f8bc8000640 1 -- 172.21.15.86:0/3099383721 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bc009f4b0 con 0x7f8bc00730b0 2024-04-25T14:06:41.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.791+0000 7f8bbd600640 1 -- 172.21.15.86:0/3099383721 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f8bb401b070 con 0x7f8bc00730b0 2024-04-25T14:06:41.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.791+0000 7f8bbd600640 1 --2- 172.21.15.86:0/3099383721 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8ba006f640 0x7f8ba0071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:41.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.791+0000 7f8bbd600640 1 -- 172.21.15.86:0/3099383721 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5249+0+0 (secure 0 0 0) 0x7f8bb408e8b0 con 0x7f8bc00730b0 2024-04-25T14:06:41.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.791+0000 7f8bc4c00640 1 --2- 172.21.15.86:0/3099383721 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8ba006f640 0x7f8ba0071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:41.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.791+0000 7f8bc4c00640 1 --2- 172.21.15.86:0/3099383721 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8ba006f640 0x7f8ba0071b50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8bac006e90 tx=0x7f8bac008040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:41.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.795+0000 7f8bbd600640 1 -- 172.21.15.86:0/3099383721 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8bc00d2030 con 0x7f8bc00730b0 2024-04-25T14:06:41.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.955+0000 7f8bc8000640 1 -- 172.21.15.86:0/3099383721 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8bc0069d00 con 0x7f8bc00730b0 2024-04-25T14:06:41.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.955+0000 7f8bbd600640 1 -- 172.21.15.86:0/3099383721 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7f8bb405b680 con 0x7f8bc00730b0 2024-04-25T14:06:41.956 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:06:41.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.955+0000 7f8bc8000640 1 -- 172.21.15.86:0/3099383721 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8ba006f640 msgr2=0x7f8ba0071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:41.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.955+0000 7f8bc8000640 1 --2- 172.21.15.86:0/3099383721 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8ba006f640 0x7f8ba0071b50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8bac006e90 tx=0x7f8bac008040 comp rx=0 tx=0).stop 2024-04-25T14:06:41.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.955+0000 7f8bc8000640 1 -- 172.21.15.86:0/3099383721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bc00730b0 msgr2=0x7f8bc015c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:41.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.955+0000 7f8bc8000640 1 --2- 172.21.15.86:0/3099383721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bc00730b0 0x7f8bc015c000 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400c480 tx=0x7f8bb400c940 comp rx=0 tx=0).stop 2024-04-25T14:06:41.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.955+0000 7f8bc8000640 1 -- 172.21.15.86:0/3099383721 shutdown_connections 2024-04-25T14:06:41.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.955+0000 7f8bc8000640 1 --2- 172.21.15.86:0/3099383721 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8ba006f640 0x7f8ba0071b50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.955+0000 7f8bc8000640 1 --2- 172.21.15.86:0/3099383721 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bc00d16e0 0x7f8bc0099f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.955+0000 7f8bc8000640 1 --2- 172.21.15.86:0/3099383721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bc00cfca0 0x7f8bc0099a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.955+0000 7f8bc8000640 1 --2- 172.21.15.86:0/3099383721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bc00730b0 0x7f8bc015c000 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:41.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.955+0000 7f8bc8000640 1 -- 172.21.15.86:0/3099383721 >> 172.21.15.86:0/3099383721 conn(0x7f8bc00b4650 msgr2=0x7f8bc00af520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:41.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.955+0000 7f8bc8000640 1 -- 172.21.15.86:0/3099383721 shutdown_connections 2024-04-25T14:06:41.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:41.955+0000 7f8bc8000640 1 -- 172.21.15.86:0/3099383721 wait complete. 2024-04-25T14:06:41.977 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T14:06:42.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.123+0000 7fe7c6800640 1 Processor -- start 2024-04-25T14:06:42.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.123+0000 7fe7c6800640 1 -- start start 2024-04-25T14:06:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.123+0000 7fe7c6800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7c0070330 0x7fe7c00d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.123+0000 7fe7c6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe7c00d1c20 0x7fe7c006ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.123+0000 7fe7c6800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7c00d2030 con 0x7fe7c006f270 2024-04-25T14:06:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.123+0000 7fe7c6800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe7c015e5c0 con 0x7fe7c00d1c20 2024-04-25T14:06:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.123+0000 7fe7c6800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe7c015e8a0 con 0x7fe7c0070330 2024-04-25T14:06:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.123+0000 7fe7b6a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe7c00d1c20 0x7fe7c006ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.123+0000 7fe7b6a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe7c00d1c20 0x7fe7c006ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38078/0 (socket says 172.21.15.86:38078) 2024-04-25T14:06:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.123+0000 7fe7b6a00640 1 -- 172.21.15.86:0/819877908 learned_addr learned my addr 172.21.15.86:0/819877908 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.123+0000 7fe7bf400640 1 --2- 172.21.15.86:0/819877908 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7c0070330 0x7fe7c00d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.123+0000 7fe7b6a00640 1 -- 172.21.15.86:0/819877908 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7c0070330 msgr2=0x7fe7c00d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.123+0000 7fe7b6a00640 1 --2- 172.21.15.86:0/819877908 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7c0070330 0x7fe7c00d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:42.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.123+0000 7fe7b6a00640 1 -- 172.21.15.86:0/819877908 >> v1:172.21.15.155:6789/0 conn(0x7fe7c006f270 legacy=0x7fe7c006f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:42.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.123+0000 7fe7b6a00640 1 -- 172.21.15.86:0/819877908 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c015eb80 con 0x7fe7c00d1c20 2024-04-25T14:06:42.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.127+0000 7fe7b6a00640 1 --2- 172.21.15.86:0/819877908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c00d1c20 0x7fe7c006ed30 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe7a80098a0 tx=0x7fe7a801c290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8780f57fa8e98be4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:42.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.127+0000 7fe7bea00640 1 -- 172.21.15.86:0/819877908 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a80b0070 con 0x7fe7c00d1c20 2024-04-25T14:06:42.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.127+0000 7fe7bea00640 1 -- 172.21.15.86:0/819877908 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7a80ab600 con 0x7fe7c00d1c20 2024-04-25T14:06:42.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.127+0000 7fe7bea00640 1 -- 172.21.15.86:0/819877908 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a80ab9d0 con 0x7fe7c00d1c20 2024-04-25T14:06:42.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.127+0000 7fe7c6800640 1 -- 172.21.15.86:0/819877908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c00d1c20 msgr2=0x7fe7c006ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:42.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.127+0000 7fe7c6800640 1 --2- 172.21.15.86:0/819877908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c00d1c20 0x7fe7c006ed30 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe7a80098a0 tx=0x7fe7a801c290 comp rx=0 tx=0).stop 2024-04-25T14:06:42.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.127+0000 7fe7c6800640 1 -- 172.21.15.86:0/819877908 shutdown_connections 2024-04-25T14:06:42.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.127+0000 7fe7c6800640 1 --2- 172.21.15.86:0/819877908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c00d1c20 0x7fe7c006ed30 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:42.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.127+0000 7fe7c6800640 1 --2- 172.21.15.86:0/819877908 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7c0070330 0x7fe7c00d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:42.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.127+0000 7fe7c6800640 1 -- 172.21.15.86:0/819877908 >> 172.21.15.86:0/819877908 conn(0x7fe7c00b4650 msgr2=0x7fe7c00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:42.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.127+0000 7fe7c6800640 1 -- 172.21.15.86:0/819877908 shutdown_connections 2024-04-25T14:06:42.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.127+0000 7fe7c6800640 1 -- 172.21.15.86:0/819877908 wait complete. 2024-04-25T14:06:42.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.131+0000 7fe7c6800640 1 Processor -- start 2024-04-25T14:06:42.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.131+0000 7fe7c6800640 1 -- start start 2024-04-25T14:06:42.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.131+0000 7fe7c6800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7c006f270 0x7fe7c009a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:42.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.131+0000 7fe7c6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c0070330 0x7fe7c00c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:42.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.131+0000 7fe7c6800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7c00d1c20 0x7fe7c00c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:42.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.131+0000 7fe7c6800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c00708c0 con 0x7fe7c0070330 2024-04-25T14:06:42.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.131+0000 7fe7c6800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c0070740 con 0x7fe7c00d1c20 2024-04-25T14:06:42.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.131+0000 7fe7c6800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe7c016afc0 con 0x7fe7c006f270 2024-04-25T14:06:42.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.131+0000 7fe7b6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c0070330 0x7fe7c00c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:42.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.131+0000 7fe7b6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c0070330 0x7fe7c00c5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38082/0 (socket says 172.21.15.86:38082) 2024-04-25T14:06:42.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.131+0000 7fe7b6a00640 1 -- 172.21.15.86:0/710941596 learned_addr learned my addr 172.21.15.86:0/710941596 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:42.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.131+0000 7fe7bf400640 1 --2- 172.21.15.86:0/710941596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7c006f270 0x7fe7c009a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:42.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.131+0000 7fe7bfe00640 1 --2- 172.21.15.86:0/710941596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7c00d1c20 0x7fe7c00c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:42.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.131+0000 7fe7b6a00640 1 -- 172.21.15.86:0/710941596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7c006f270 msgr2=0x7fe7c009a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:42.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.131+0000 7fe7b6a00640 1 --2- 172.21.15.86:0/710941596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7c006f270 0x7fe7c009a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:42.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.131+0000 7fe7b6a00640 1 -- 172.21.15.86:0/710941596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7c00d1c20 msgr2=0x7fe7c00c61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:42.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.131+0000 7fe7b6a00640 1 --2- 172.21.15.86:0/710941596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7c00d1c20 0x7fe7c00c61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:42.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.131+0000 7fe7b6a00640 1 -- 172.21.15.86:0/710941596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c00c5ac0 con 0x7fe7c0070330 2024-04-25T14:06:42.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.131+0000 7fe7bf400640 1 --2- 172.21.15.86:0/710941596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7c006f270 0x7fe7c009a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:42.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.131+0000 7fe7b6a00640 1 --2- 172.21.15.86:0/710941596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c0070330 0x7fe7c00c5cb0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe7a801cb30 tx=0x7fe7a8019030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:42.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.131+0000 7fe7b7e00640 1 -- 172.21.15.86:0/710941596 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a80b0040 con 0x7fe7c0070330 2024-04-25T14:06:42.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.131+0000 7fe7b7e00640 1 -- 172.21.15.86:0/710941596 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7a80b6420 con 0x7fe7c0070330 2024-04-25T14:06:42.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.131+0000 7fe7b7e00640 1 -- 172.21.15.86:0/710941596 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a80b6810 con 0x7fe7c0070330 2024-04-25T14:06:42.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.131+0000 7fe7c6800640 1 -- 172.21.15.86:0/710941596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c00745e0 con 0x7fe7c0070330 2024-04-25T14:06:42.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.131+0000 7fe7c6800640 1 -- 172.21.15.86:0/710941596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c00b1c70 con 0x7fe7c0070330 2024-04-25T14:06:42.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.135+0000 7fe7c6800640 1 -- 172.21.15.86:0/710941596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7c0074350 con 0x7fe7c0070330 2024-04-25T14:06:42.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.135+0000 7fe7b7e00640 1 -- 172.21.15.86:0/710941596 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7fe7a80be020 con 0x7fe7c0070330 2024-04-25T14:06:42.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.135+0000 7fe7b7e00640 1 --2- 172.21.15.86:0/710941596 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fe79c06f670 0x7fe79c071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:42.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.135+0000 7fe7b7e00640 1 -- 172.21.15.86:0/710941596 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5249+0+0 (secure 0 0 0) 0x7fe7a81292c0 con 0x7fe7c0070330 2024-04-25T14:06:42.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.135+0000 7fe7bf400640 1 --2- 172.21.15.86:0/710941596 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fe79c06f670 0x7fe79c071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:42.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.135+0000 7fe7bf400640 1 --2- 172.21.15.86:0/710941596 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fe79c06f670 0x7fe79c071b80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0072200 tx=0x7fe7b0008040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:42.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.139+0000 7fe7b7e00640 1 -- 172.21.15.86:0/710941596 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe7c00d2030 con 0x7fe7c0070330 2024-04-25T14:06:42.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.299+0000 7fe7c6800640 1 -- 172.21.15.86:0/710941596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe7c0069d00 con 0x7fe7c0070330 2024-04-25T14:06:42.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.299+0000 7fe7b7e00640 1 -- 172.21.15.86:0/710941596 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) ==== 84+0+65 (secure 0 0 0) 0x7fe7a80f5fd0 con 0x7fe7c0070330 2024-04-25T14:06:42.302 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:42.302 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-04-25T14:06:42.303 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:06:42.303 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-04-25T14:06:42.303 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-04-25T14:06:42.303 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:06:42.303 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-04-25T14:06:42.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.303+0000 7fe7c6800640 1 -- 172.21.15.86:0/710941596 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fe79c06f670 msgr2=0x7fe79c071b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:42.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.303+0000 7fe7c6800640 1 --2- 172.21.15.86:0/710941596 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fe79c06f670 0x7fe79c071b80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0072200 tx=0x7fe7b0008040 comp rx=0 tx=0).stop 2024-04-25T14:06:42.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.303+0000 7fe7c6800640 1 -- 172.21.15.86:0/710941596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c0070330 msgr2=0x7fe7c00c5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:42.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.303+0000 7fe7c6800640 1 --2- 172.21.15.86:0/710941596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c0070330 0x7fe7c00c5cb0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe7a801cb30 tx=0x7fe7a8019030 comp rx=0 tx=0).stop 2024-04-25T14:06:42.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.303+0000 7fe7c6800640 1 -- 172.21.15.86:0/710941596 shutdown_connections 2024-04-25T14:06:42.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.303+0000 7fe7c6800640 1 --2- 172.21.15.86:0/710941596 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fe79c06f670 0x7fe79c071b80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:42.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.303+0000 7fe7c6800640 1 --2- 172.21.15.86:0/710941596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7c00d1c20 0x7fe7c00c61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:42.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.303+0000 7fe7c6800640 1 --2- 172.21.15.86:0/710941596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7c0070330 0x7fe7c00c5cb0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:42.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.303+0000 7fe7c6800640 1 --2- 172.21.15.86:0/710941596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7c006f270 0x7fe7c009a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:42.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.303+0000 7fe7c6800640 1 -- 172.21.15.86:0/710941596 >> 172.21.15.86:0/710941596 conn(0x7fe7c00b4650 msgr2=0x7fe7c00b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:42.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.303+0000 7fe7c6800640 1 -- 172.21.15.86:0/710941596 shutdown_connections 2024-04-25T14:06:42.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.303+0000 7fe7c6800640 1 -- 172.21.15.86:0/710941596 wait complete. 2024-04-25T14:06:42.324 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T14:06:42.324 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T14:06:42.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.471+0000 7fb290e00640 1 Processor -- start 2024-04-25T14:06:42.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.471+0000 7fb290e00640 1 -- start start 2024-04-25T14:06:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.475+0000 7fb290e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb28c0ce4a0 0x7fb28c0b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.475+0000 7fb290e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb28c0cca40 0x7fb28c0cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.475+0000 7fb290e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb28c0b1f30 con 0x7fb28c06d780 2024-04-25T14:06:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.475+0000 7fb290e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb28c0d3380 con 0x7fb28c0ce4a0 2024-04-25T14:06:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.475+0000 7fb290e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb28c0b3dc0 con 0x7fb28c0cca40 2024-04-25T14:06:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.475+0000 7fb289e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb28c0ce4a0 0x7fb28c0b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.475+0000 7fb289e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb28c0ce4a0 0x7fb28c0b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38092/0 (socket says 172.21.15.86:38092) 2024-04-25T14:06:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.475+0000 7fb289e00640 1 -- 172.21.15.86:0/1450275343 learned_addr learned my addr 172.21.15.86:0/1450275343 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.475+0000 7fb28a800640 1 --2- 172.21.15.86:0/1450275343 >> v2:172.21.15.155:3300/0 conn(0x7fb28c0cca40 0x7fb28c0cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.475+0000 7fb289e00640 1 -- 172.21.15.86:0/1450275343 >> v2:172.21.15.155:3300/0 conn(0x7fb28c0cca40 msgr2=0x7fb28c0cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.475+0000 7fb289e00640 1 --2- 172.21.15.86:0/1450275343 >> v2:172.21.15.155:3300/0 conn(0x7fb28c0cca40 0x7fb28c0cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.475+0000 7fb289e00640 1 -- 172.21.15.86:0/1450275343 >> v1:172.21.15.155:6789/0 conn(0x7fb28c06d780 legacy=0x7fb28c06db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.475+0000 7fb289e00640 1 -- 172.21.15.86:0/1450275343 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb28c0b2210 con 0x7fb28c0ce4a0 2024-04-25T14:06:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.475+0000 7fb28a800640 1 --2- 172.21.15.86:0/1450275343 >> v2:172.21.15.155:3300/0 conn(0x7fb28c0cca40 0x7fb28c0cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.479+0000 7fb289e00640 1 --2- 172.21.15.86:0/1450275343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb28c0ce4a0 0x7fb28c0b3880 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb28c0cfe30 tx=0x7fb27c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77dc9b338931fa94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:42.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.479+0000 7fb288a00640 1 -- 172.21.15.86:0/1450275343 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c0b0070 con 0x7fb28c0ce4a0 2024-04-25T14:06:42.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.479+0000 7fb288a00640 1 -- 172.21.15.86:0/1450275343 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb27c0ab600 con 0x7fb28c0ce4a0 2024-04-25T14:06:42.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.479+0000 7fb288a00640 1 -- 172.21.15.86:0/1450275343 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c0ab9d0 con 0x7fb28c0ce4a0 2024-04-25T14:06:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.479+0000 7fb290e00640 1 -- 172.21.15.86:0/1450275343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb28c0ce4a0 msgr2=0x7fb28c0b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.479+0000 7fb290e00640 1 --2- 172.21.15.86:0/1450275343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb28c0ce4a0 0x7fb28c0b3880 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb28c0cfe30 tx=0x7fb27c01c4e0 comp rx=0 tx=0).stop 2024-04-25T14:06:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.479+0000 7fb290e00640 1 -- 172.21.15.86:0/1450275343 shutdown_connections 2024-04-25T14:06:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.479+0000 7fb290e00640 1 --2- 172.21.15.86:0/1450275343 >> v2:172.21.15.155:3300/0 conn(0x7fb28c0cca40 0x7fb28c0cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.479+0000 7fb290e00640 1 --2- 172.21.15.86:0/1450275343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb28c0ce4a0 0x7fb28c0b3880 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.479+0000 7fb290e00640 1 -- 172.21.15.86:0/1450275343 >> 172.21.15.86:0/1450275343 conn(0x7fb28c09ba30 msgr2=0x7fb28c0b7240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.479+0000 7fb290e00640 1 -- 172.21.15.86:0/1450275343 shutdown_connections 2024-04-25T14:06:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.479+0000 7fb290e00640 1 -- 172.21.15.86:0/1450275343 wait complete. 2024-04-25T14:06:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.479+0000 7fb290e00640 1 Processor -- start 2024-04-25T14:06:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.479+0000 7fb290e00640 1 -- start start 2024-04-25T14:06:42.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.483+0000 7fb290e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb28c06d780 0x7fb28c145ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:42.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.483+0000 7fb290e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb28c0cca40 0x7fb28c15fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:42.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.483+0000 7fb290e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb28c0ce4a0 0x7fb28c0cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:42.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.483+0000 7fb290e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb28c06ca80 con 0x7fb28c0cca40 2024-04-25T14:06:42.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.483+0000 7fb290e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb28c06c900 con 0x7fb28c06d780 2024-04-25T14:06:42.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.483+0000 7fb290e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb28c06cc00 con 0x7fb28c0ce4a0 2024-04-25T14:06:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.483+0000 7fb289400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb28c0cca40 0x7fb28c15fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.483+0000 7fb289400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb28c0cca40 0x7fb28c15fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38096/0 (socket says 172.21.15.86:38096) 2024-04-25T14:06:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.483+0000 7fb289400640 1 -- 172.21.15.86:0/2712216247 learned_addr learned my addr 172.21.15.86:0/2712216247 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.483+0000 7fb289e00640 1 --2- 172.21.15.86:0/2712216247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb28c06d780 0x7fb28c145ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.483+0000 7fb28a800640 1 --2- 172.21.15.86:0/2712216247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb28c0ce4a0 0x7fb28c0cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.483+0000 7fb289400640 1 -- 172.21.15.86:0/2712216247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb28c0ce4a0 msgr2=0x7fb28c0cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.483+0000 7fb289400640 1 --2- 172.21.15.86:0/2712216247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb28c0ce4a0 0x7fb28c0cb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.483+0000 7fb289400640 1 -- 172.21.15.86:0/2712216247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb28c06d780 msgr2=0x7fb28c145ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.483+0000 7fb289400640 1 --2- 172.21.15.86:0/2712216247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb28c06d780 0x7fb28c145ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.483+0000 7fb289400640 1 -- 172.21.15.86:0/2712216247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb28c15c690 con 0x7fb28c0cca40 2024-04-25T14:06:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.483+0000 7fb289400640 1 --2- 172.21.15.86:0/2712216247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb28c0cca40 0x7fb28c15fa80 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb27800b410 tx=0x7fb27800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.483+0000 7fb282000640 1 -- 172.21.15.86:0/2712216247 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb278015030 con 0x7fb28c0cca40 2024-04-25T14:06:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.483+0000 7fb282000640 1 -- 172.21.15.86:0/2712216247 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb278004020 con 0x7fb28c0cca40 2024-04-25T14:06:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.483+0000 7fb282000640 1 -- 172.21.15.86:0/2712216247 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb278004410 con 0x7fb28c0cca40 2024-04-25T14:06:42.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.483+0000 7fb290e00640 1 -- 172.21.15.86:0/2712216247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb28c15c970 con 0x7fb28c0cca40 2024-04-25T14:06:42.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.483+0000 7fb290e00640 1 -- 172.21.15.86:0/2712216247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb28c146110 con 0x7fb28c0cca40 2024-04-25T14:06:42.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.483+0000 7fb290e00640 1 -- 172.21.15.86:0/2712216247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb28c050750 con 0x7fb28c0cca40 2024-04-25T14:06:42.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.487+0000 7fb282000640 1 -- 172.21.15.86:0/2712216247 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7fb278018070 con 0x7fb28c0cca40 2024-04-25T14:06:42.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.487+0000 7fb282000640 1 --2- 172.21.15.86:0/2712216247 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fb26c06f640 0x7fb26c071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.487+0000 7fb282000640 1 -- 172.21.15.86:0/2712216247 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5249+0+0 (secure 0 0 0) 0x7fb27808efe0 con 0x7fb28c0cca40 2024-04-25T14:06:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.487+0000 7fb289e00640 1 --2- 172.21.15.86:0/2712216247 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fb26c06f640 0x7fb26c071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.487+0000 7fb289e00640 1 --2- 172.21.15.86:0/2712216247 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fb26c06f640 0x7fb26c071b50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb27c0ad000 tx=0x7fb27c008040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.491+0000 7fb282000640 1 -- 172.21.15.86:0/2712216247 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb27805bd80 con 0x7fb28c0cca40 2024-04-25T14:06:42.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:42.651+0000 7fb290e00640 1 -- 172.21.15.86:0/2712216247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb28c159d30 con 0x7fb28c0cca40 2024-04-25T14:06:43.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.503+0000 7fb282000640 1 -- 172.21.15.86:0/2712216247 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v71) ==== 138+0+0 (secure 0 0 0) 0x7fb27805b680 con 0x7fb28c0cca40 2024-04-25T14:06:43.505 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-04-25T14:06:43.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.507+0000 7fb290e00640 1 -- 172.21.15.86:0/2712216247 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fb26c06f640 msgr2=0x7fb26c071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:43.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.507+0000 7fb290e00640 1 --2- 172.21.15.86:0/2712216247 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fb26c06f640 0x7fb26c071b50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb27c0ad000 tx=0x7fb27c008040 comp rx=0 tx=0).stop 2024-04-25T14:06:43.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.507+0000 7fb290e00640 1 -- 172.21.15.86:0/2712216247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb28c0cca40 msgr2=0x7fb28c15fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:43.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.507+0000 7fb290e00640 1 --2- 172.21.15.86:0/2712216247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb28c0cca40 0x7fb28c15fa80 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb27800b410 tx=0x7fb27800b8d0 comp rx=0 tx=0).stop 2024-04-25T14:06:43.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.507+0000 7fb290e00640 1 -- 172.21.15.86:0/2712216247 shutdown_connections 2024-04-25T14:06:43.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.507+0000 7fb290e00640 1 --2- 172.21.15.86:0/2712216247 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fb26c06f640 0x7fb26c071b50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:43.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.507+0000 7fb290e00640 1 --2- 172.21.15.86:0/2712216247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb28c0ce4a0 0x7fb28c0cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:43.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.507+0000 7fb290e00640 1 --2- 172.21.15.86:0/2712216247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb28c0cca40 0x7fb28c15fa80 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:43.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.507+0000 7fb290e00640 1 --2- 172.21.15.86:0/2712216247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb28c06d780 0x7fb28c145ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:43.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.507+0000 7fb290e00640 1 -- 172.21.15.86:0/2712216247 >> 172.21.15.86:0/2712216247 conn(0x7fb28c09ba30 msgr2=0x7fb28c158e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:43.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.507+0000 7fb290e00640 1 -- 172.21.15.86:0/2712216247 shutdown_connections 2024-04-25T14:06:43.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.507+0000 7fb290e00640 1 -- 172.21.15.86:0/2712216247 wait complete. 2024-04-25T14:06:43.537 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T14:06:43.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.691+0000 7f804b000640 1 Processor -- start 2024-04-25T14:06:43.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.691+0000 7f804b000640 1 -- start start 2024-04-25T14:06:43.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.695+0000 7f804b000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8044071530 0x7f80441638a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:43.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.695+0000 7f804b000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8044163de0 0x7f80440cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:43.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.695+0000 7f804b000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f804406dc30 con 0x7f804415fe00 2024-04-25T14:06:43.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.695+0000 7f804b000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8044160210 con 0x7f8044071530 2024-04-25T14:06:43.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.695+0000 7f804b000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f80441641f0 con 0x7f8044163de0 2024-04-25T14:06:43.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.695+0000 7f803b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8044071530 0x7f80441638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:43.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.695+0000 7f803b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8044071530 0x7f80441638a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38102/0 (socket says 172.21.15.86:38102) 2024-04-25T14:06:43.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.695+0000 7f803b400640 1 -- 172.21.15.86:0/1506534076 learned_addr learned my addr 172.21.15.86:0/1506534076 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:43.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.695+0000 7f8049000640 1 --2- 172.21.15.86:0/1506534076 >> v2:172.21.15.155:3300/0 conn(0x7f8044163de0 0x7f80440cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:43.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.695+0000 7f803b400640 1 -- 172.21.15.86:0/1506534076 >> v2:172.21.15.155:3300/0 conn(0x7f8044163de0 msgr2=0x7f80440cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:43.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.695+0000 7f803b400640 1 --2- 172.21.15.86:0/1506534076 >> v2:172.21.15.155:3300/0 conn(0x7f8044163de0 0x7f80440cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:43.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.695+0000 7f803b400640 1 -- 172.21.15.86:0/1506534076 >> v1:172.21.15.155:6789/0 conn(0x7f804415fe00 legacy=0x7f8044070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:43.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.695+0000 7f803b400640 1 -- 172.21.15.86:0/1506534076 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8044164480 con 0x7f8044071530 2024-04-25T14:06:43.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.695+0000 7f803b400640 1 --2- 172.21.15.86:0/1506534076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8044071530 0x7f80441638a0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f8044071a70 tx=0x7f802c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6502b1a313af71f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:43.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.699+0000 7f8043400640 1 -- 172.21.15.86:0/1506534076 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c0b0070 con 0x7f8044071530 2024-04-25T14:06:43.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.699+0000 7f8043400640 1 -- 172.21.15.86:0/1506534076 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f802c0ab600 con 0x7f8044071530 2024-04-25T14:06:43.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.699+0000 7f8043400640 1 -- 172.21.15.86:0/1506534076 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c0ab9d0 con 0x7f8044071530 2024-04-25T14:06:43.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.699+0000 7f804b000640 1 -- 172.21.15.86:0/1506534076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8044071530 msgr2=0x7f80441638a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:43.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.699+0000 7f804b000640 1 --2- 172.21.15.86:0/1506534076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8044071530 0x7f80441638a0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f8044071a70 tx=0x7f802c01c520 comp rx=0 tx=0).stop 2024-04-25T14:06:43.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.699+0000 7f804b000640 1 -- 172.21.15.86:0/1506534076 shutdown_connections 2024-04-25T14:06:43.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.699+0000 7f804b000640 1 --2- 172.21.15.86:0/1506534076 >> v2:172.21.15.155:3300/0 conn(0x7f8044163de0 0x7f80440cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:43.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.699+0000 7f804b000640 1 --2- 172.21.15.86:0/1506534076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8044071530 0x7f80441638a0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:43.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.699+0000 7f804b000640 1 -- 172.21.15.86:0/1506534076 >> 172.21.15.86:0/1506534076 conn(0x7f80440cffc0 msgr2=0x7f80440d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:43.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.699+0000 7f804b000640 1 -- 172.21.15.86:0/1506534076 shutdown_connections 2024-04-25T14:06:43.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.699+0000 7f804b000640 1 -- 172.21.15.86:0/1506534076 wait complete. 2024-04-25T14:06:43.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.699+0000 7f804b000640 1 Processor -- start 2024-04-25T14:06:43.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.699+0000 7f804b000640 1 -- start start 2024-04-25T14:06:43.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.703+0000 7f804b000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f804415fe00 0x7f8044151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:43.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.703+0000 7f804b000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8044163de0 0x7f8044150b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:43.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.703+0000 7f804b000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80440c88d0 0x7f80440b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:43.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.703+0000 7f804b000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f804416cda0 con 0x7f80440c88d0 2024-04-25T14:06:43.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.703+0000 7f804b000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f804416cc20 con 0x7f804415fe00 2024-04-25T14:06:43.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.703+0000 7f804b000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f804416cf20 con 0x7f8044163de0 2024-04-25T14:06:43.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.703+0000 7f8049000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80440c88d0 0x7f80440b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:43.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.703+0000 7f8049000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80440c88d0 0x7f80440b57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38104/0 (socket says 172.21.15.86:38104) 2024-04-25T14:06:43.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.703+0000 7f8049000640 1 -- 172.21.15.86:0/2165617709 learned_addr learned my addr 172.21.15.86:0/2165617709 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:43.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.703+0000 7f8043e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f804415fe00 0x7f8044151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:43.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.703+0000 7f803b400640 1 --2- 172.21.15.86:0/2165617709 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8044163de0 0x7f8044150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:43.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.703+0000 7f8049000640 1 -- 172.21.15.86:0/2165617709 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8044163de0 msgr2=0x7f8044150b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:43.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.703+0000 7f8049000640 1 --2- 172.21.15.86:0/2165617709 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8044163de0 0x7f8044150b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:43.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.703+0000 7f8049000640 1 -- 172.21.15.86:0/2165617709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f804415fe00 msgr2=0x7f8044151fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:43.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.703+0000 7f8049000640 1 --2- 172.21.15.86:0/2165617709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f804415fe00 0x7f8044151fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:43.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.703+0000 7f8049000640 1 -- 172.21.15.86:0/2165617709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8044148320 con 0x7f80440c88d0 2024-04-25T14:06:43.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.703+0000 7f8043e00640 1 --2- 172.21.15.86:0/2165617709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f804415fe00 0x7f8044151fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:43.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.703+0000 7f8049000640 1 --2- 172.21.15.86:0/2165617709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80440c88d0 0x7f80440b57f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f803c007e60 tx=0x7f803c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:43.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.703+0000 7f8040c00640 1 -- 172.21.15.86:0/2165617709 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c014420 con 0x7f80440c88d0 2024-04-25T14:06:43.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.703+0000 7f8040c00640 1 -- 172.21.15.86:0/2165617709 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f803c019070 con 0x7f80440c88d0 2024-04-25T14:06:43.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.703+0000 7f804b000640 1 -- 172.21.15.86:0/2165617709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80440b31b0 con 0x7f80440c88d0 2024-04-25T14:06:43.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.703+0000 7f8040c00640 1 -- 172.21.15.86:0/2165617709 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c014820 con 0x7f80440c88d0 2024-04-25T14:06:43.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.703+0000 7f804b000640 1 -- 172.21.15.86:0/2165617709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80441510a0 con 0x7f80440c88d0 2024-04-25T14:06:43.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.703+0000 7f804b000640 1 -- 172.21.15.86:0/2165617709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8044152500 con 0x7f80440c88d0 2024-04-25T14:06:43.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.707+0000 7f8040c00640 1 -- 172.21.15.86:0/2165617709 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f803c0149c0 con 0x7f80440c88d0 2024-04-25T14:06:43.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.707+0000 7f8040c00640 1 --2- 172.21.15.86:0/2165617709 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f802006f670 0x7f8020071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:43.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.707+0000 7f8040c00640 1 -- 172.21.15.86:0/2165617709 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5619+0+0 (secure 0 0 0) 0x7f803c090700 con 0x7f80440c88d0 2024-04-25T14:06:43.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.707+0000 7f8043e00640 1 --2- 172.21.15.86:0/2165617709 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f802006f670 0x7f8020071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:43.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.707+0000 7f8043e00640 1 --2- 172.21.15.86:0/2165617709 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f802006f670 0x7f8020071b80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8034007fd0 tx=0x7f8034009040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:43.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.711+0000 7f8040c00640 1 -- 172.21.15.86:0/2165617709 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f803c05ce40 con 0x7f80440c88d0 2024-04-25T14:06:43.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:43.867+0000 7f804b000640 1 -- 172.21.15.86:0/2165617709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f80440b2fa0 con 0x7f80440c88d0 2024-04-25T14:06:44.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.507+0000 7f8040c00640 1 -- 172.21.15.86:0/2165617709 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v72) ==== 170+0+0 (secure 0 0 0) 0x7f803c061140 con 0x7f80440c88d0 2024-04-25T14:06:44.509 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-04-25T14:06:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.507+0000 7f804b000640 1 -- 172.21.15.86:0/2165617709 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f802006f670 msgr2=0x7f8020071b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.507+0000 7f804b000640 1 --2- 172.21.15.86:0/2165617709 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f802006f670 0x7f8020071b80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8034007fd0 tx=0x7f8034009040 comp rx=0 tx=0).stop 2024-04-25T14:06:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.507+0000 7f804b000640 1 -- 172.21.15.86:0/2165617709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80440c88d0 msgr2=0x7f80440b57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.507+0000 7f804b000640 1 --2- 172.21.15.86:0/2165617709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80440c88d0 0x7f80440b57f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f803c007e60 tx=0x7f803c00e4b0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.511+0000 7f804b000640 1 -- 172.21.15.86:0/2165617709 shutdown_connections 2024-04-25T14:06:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.511+0000 7f804b000640 1 --2- 172.21.15.86:0/2165617709 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f802006f670 0x7f8020071b80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.511+0000 7f804b000640 1 --2- 172.21.15.86:0/2165617709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80440c88d0 0x7f80440b57f0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.511+0000 7f804b000640 1 --2- 172.21.15.86:0/2165617709 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8044163de0 0x7f8044150b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.511+0000 7f804b000640 1 --2- 172.21.15.86:0/2165617709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f804415fe00 0x7f8044151fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.511+0000 7f804b000640 1 -- 172.21.15.86:0/2165617709 >> 172.21.15.86:0/2165617709 conn(0x7f80440cffc0 msgr2=0x7f804415d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.511+0000 7f804b000640 1 -- 172.21.15.86:0/2165617709 shutdown_connections 2024-04-25T14:06:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.511+0000 7f804b000640 1 -- 172.21.15.86:0/2165617709 wait complete. 2024-04-25T14:06:44.548 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T14:06:44.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.695+0000 7f2796000640 1 Processor -- start 2024-04-25T14:06:44.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.695+0000 7f2796000640 1 -- start start 2024-04-25T14:06:44.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.699+0000 7f2796000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f279015d9c0 0x7f279006eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:44.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.699+0000 7f2796000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f27900d0950 0x7f27900d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:44.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.699+0000 7f2796000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f279006f0b0 con 0x7f27900b4d40 2024-04-25T14:06:44.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.699+0000 7f2796000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f27900ce5e0 con 0x7f279015d9c0 2024-04-25T14:06:44.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.699+0000 7f2796000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f27900b6c10 con 0x7f27900d0950 2024-04-25T14:06:44.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.699+0000 7f278f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f279015d9c0 0x7f279006eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:44.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.699+0000 7f2786a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f27900b4d40 0x7f27900b5150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42086/0 (socket says 172.21.15.86:42086) 2024-04-25T14:06:44.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.699+0000 7f2786a00640 1 -- 172.21.15.86:0/3481377971 learned_addr learned my addr 172.21.15.86:0/3481377971 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:44.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.699+0000 7f278fe00640 1 --2- 172.21.15.86:0/3481377971 >> v2:172.21.15.155:3300/0 conn(0x7f27900d0950 0x7f27900d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:44.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.699+0000 7f278f400640 1 -- 172.21.15.86:0/3481377971 >> v2:172.21.15.155:3300/0 conn(0x7f27900d0950 msgr2=0x7f27900d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:44.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.699+0000 7f278f400640 1 --2- 172.21.15.86:0/3481377971 >> v2:172.21.15.155:3300/0 conn(0x7f27900d0950 0x7f27900d0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.699+0000 7f278f400640 1 -- 172.21.15.86:0/3481377971 >> v1:172.21.15.86:6789/0 conn(0x7f27900b4d40 legacy=0x7f27900b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:44.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.699+0000 7f278ea00640 1 -- 172.21.15.86:0/3481377971 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3872658965 0 0) 0x7f279006f0b0 con 0x7f27900b4d40 2024-04-25T14:06:44.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.699+0000 7f278f400640 1 -- 172.21.15.86:0/3481377971 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27900cfb90 con 0x7f279015d9c0 2024-04-25T14:06:44.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.703+0000 7f278f400640 1 --2- 172.21.15.86:0/3481377971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f279015d9c0 0x7f279006eb70 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f27900cf550 tx=0x7f278001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4156a442b91acc57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:44.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.703+0000 7f278ea00640 1 -- 172.21.15.86:0/3481377971 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27800b1070 con 0x7f279015d9c0 2024-04-25T14:06:44.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.703+0000 7f278ea00640 1 -- 172.21.15.86:0/3481377971 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27800ac620 con 0x7f279015d9c0 2024-04-25T14:06:44.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.703+0000 7f2796000640 1 -- 172.21.15.86:0/3481377971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f279015d9c0 msgr2=0x7f279006eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:44.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.703+0000 7f2796000640 1 --2- 172.21.15.86:0/3481377971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f279015d9c0 0x7f279006eb70 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f27900cf550 tx=0x7f278001c3c0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.703+0000 7f278ea00640 1 -- 172.21.15.86:0/3481377971 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27800ac9f0 con 0x7f279015d9c0 2024-04-25T14:06:44.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.703+0000 7f2796000640 1 -- 172.21.15.86:0/3481377971 shutdown_connections 2024-04-25T14:06:44.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.703+0000 7f2796000640 1 --2- 172.21.15.86:0/3481377971 >> v2:172.21.15.155:3300/0 conn(0x7f27900d0950 0x7f27900d0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.703+0000 7f2796000640 1 --2- 172.21.15.86:0/3481377971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f279015d9c0 0x7f279006eb70 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.703+0000 7f2796000640 1 -- 172.21.15.86:0/3481377971 >> 172.21.15.86:0/3481377971 conn(0x7f27900d19b0 msgr2=0x7f27900d1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:44.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.703+0000 7f2796000640 1 -- 172.21.15.86:0/3481377971 shutdown_connections 2024-04-25T14:06:44.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.703+0000 7f2796000640 1 -- 172.21.15.86:0/3481377971 wait complete. 2024-04-25T14:06:44.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.703+0000 7f2796000640 1 Processor -- start 2024-04-25T14:06:44.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.703+0000 7f2796000640 1 -- start start 2024-04-25T14:06:44.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.707+0000 7f2796000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27900b4d40 0x7f27900b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:44.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.707+0000 7f2796000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27900d0950 0x7f2790161a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:44.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.707+0000 7f2796000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f279015d9c0 0x7f279006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:44.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.707+0000 7f2796000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f279015ddd0 con 0x7f27900b4d40 2024-04-25T14:06:44.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.707+0000 7f2796000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f27900b5ae0 con 0x7f279015d9c0 2024-04-25T14:06:44.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.707+0000 7f2796000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f27900b6780 con 0x7f27900d0950 2024-04-25T14:06:44.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.707+0000 7f278f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27900b4d40 0x7f27900b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:44.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.707+0000 7f278f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27900b4d40 0x7f27900b2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38110/0 (socket says 172.21.15.86:38110) 2024-04-25T14:06:44.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.707+0000 7f278f400640 1 -- 172.21.15.86:0/2418208773 learned_addr learned my addr 172.21.15.86:0/2418208773 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:44.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.707+0000 7f2786a00640 1 --2- 172.21.15.86:0/2418208773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27900d0950 0x7f2790161a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:44.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.707+0000 7f278fe00640 1 --2- 172.21.15.86:0/2418208773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f279015d9c0 0x7f279006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:44.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.707+0000 7f278f400640 1 -- 172.21.15.86:0/2418208773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27900d0950 msgr2=0x7f2790161a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:44.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.707+0000 7f278f400640 1 --2- 172.21.15.86:0/2418208773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27900d0950 0x7f2790161a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.707+0000 7f278f400640 1 -- 172.21.15.86:0/2418208773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f279015d9c0 msgr2=0x7f279006d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:44.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.707+0000 7f278f400640 1 --2- 172.21.15.86:0/2418208773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f279015d9c0 0x7f279006d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.707+0000 7f278f400640 1 -- 172.21.15.86:0/2418208773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27900b31c0 con 0x7f27900b4d40 2024-04-25T14:06:44.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.707+0000 7f2786a00640 1 --2- 172.21.15.86:0/2418208773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27900d0950 0x7f2790161a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:44.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.707+0000 7f278fe00640 1 --2- 172.21.15.86:0/2418208773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f279015d9c0 0x7f279006d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:44.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.707+0000 7f278f400640 1 --2- 172.21.15.86:0/2418208773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27900b4d40 0x7f27900b2c80 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f2780018fd0 tx=0x7f278001ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:44.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.707+0000 7f2787e00640 1 -- 172.21.15.86:0/2418208773 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27800be070 con 0x7f27900b4d40 2024-04-25T14:06:44.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.707+0000 7f2787e00640 1 -- 172.21.15.86:0/2418208773 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2780004030 con 0x7f27900b4d40 2024-04-25T14:06:44.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.707+0000 7f2787e00640 1 -- 172.21.15.86:0/2418208773 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27800b1040 con 0x7f27900b4d40 2024-04-25T14:06:44.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.707+0000 7f2796000640 1 -- 172.21.15.86:0/2418208773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27901fcf50 con 0x7f27900b4d40 2024-04-25T14:06:44.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.707+0000 7f2796000640 1 -- 172.21.15.86:0/2418208773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27901fd510 con 0x7f27900b4d40 2024-04-25T14:06:44.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.707+0000 7f2796000640 1 -- 172.21.15.86:0/2418208773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27900748a0 con 0x7f27900b4d40 2024-04-25T14:06:44.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.711+0000 7f2787e00640 1 -- 172.21.15.86:0/2418208773 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f2780003620 con 0x7f27900b4d40 2024-04-25T14:06:44.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.711+0000 7f2787e00640 1 --2- 172.21.15.86:0/2418208773 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f276c06f640 0x7f276c071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:44.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.711+0000 7f2787e00640 1 -- 172.21.15.86:0/2418208773 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6001+0+0 (secure 0 0 0) 0x7f2780129610 con 0x7f27900b4d40 2024-04-25T14:06:44.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.711+0000 7f2786a00640 1 --2- 172.21.15.86:0/2418208773 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f276c06f640 0x7f276c071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:44.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.711+0000 7f2786a00640 1 --2- 172.21.15.86:0/2418208773 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f276c06f640 0x7f276c071b50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f27900b4790 tx=0x7f2778009040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:44.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.715+0000 7f2787e00640 1 -- 172.21.15.86:0/2418208773 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2780016070 con 0x7f27900b4d40 2024-04-25T14:06:44.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:44.903+0000 7f2796000640 1 -- 172.21.15.86:0/2418208773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f279006cce0 con 0x7f27900b4d40 2024-04-25T14:06:45.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.539+0000 7f2787e00640 1 -- 172.21.15.86:0/2418208773 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v49) ==== 174+0+0 (secure 0 0 0) 0x7f27800f6040 con 0x7f27900b4d40 2024-04-25T14:06:45.543 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 11 and data pool 12 2024-04-25T14:06:45.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.547+0000 7f2796000640 1 -- 172.21.15.86:0/2418208773 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f276c06f640 msgr2=0x7f276c071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:45.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.547+0000 7f2796000640 1 --2- 172.21.15.86:0/2418208773 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f276c06f640 0x7f276c071b50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f27900b4790 tx=0x7f2778009040 comp rx=0 tx=0).stop 2024-04-25T14:06:45.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.547+0000 7f2796000640 1 -- 172.21.15.86:0/2418208773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27900b4d40 msgr2=0x7f27900b2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:45.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.547+0000 7f2796000640 1 --2- 172.21.15.86:0/2418208773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27900b4d40 0x7f27900b2c80 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f2780018fd0 tx=0x7f278001ac00 comp rx=0 tx=0).stop 2024-04-25T14:06:45.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.551+0000 7f2796000640 1 -- 172.21.15.86:0/2418208773 shutdown_connections 2024-04-25T14:06:45.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.551+0000 7f2796000640 1 --2- 172.21.15.86:0/2418208773 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f276c06f640 0x7f276c071b50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:45.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.551+0000 7f2796000640 1 --2- 172.21.15.86:0/2418208773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f279015d9c0 0x7f279006d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:45.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.551+0000 7f2796000640 1 --2- 172.21.15.86:0/2418208773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27900d0950 0x7f2790161a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:45.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.551+0000 7f2796000640 1 --2- 172.21.15.86:0/2418208773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27900b4d40 0x7f27900b2c80 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:45.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.551+0000 7f2796000640 1 -- 172.21.15.86:0/2418208773 >> 172.21.15.86:0/2418208773 conn(0x7f27900d19b0 msgr2=0x7f279006ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:45.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.559+0000 7f2796000640 1 -- 172.21.15.86:0/2418208773 shutdown_connections 2024-04-25T14:06:45.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.559+0000 7f2796000640 1 -- 172.21.15.86:0/2418208773 wait complete. 2024-04-25T14:06:45.594 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:06:45.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.739+0000 7f666de00640 1 Processor -- start 2024-04-25T14:06:45.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.739+0000 7f666de00640 1 -- start start 2024-04-25T14:06:45.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.743+0000 7f666de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6668070330 0x7f66680d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:45.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.743+0000 7f666de00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66680ce790 0x7f666806f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:45.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.743+0000 7f666de00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66680d2030 con 0x7f66680d1c20 2024-04-25T14:06:45.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.743+0000 7f666de00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f666815e5c0 con 0x7f6668070330 2024-04-25T14:06:45.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.743+0000 7f666de00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f666815e8a0 con 0x7f66680ce790 2024-04-25T14:06:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.743+0000 7f6666a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6668070330 0x7f66680d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.743+0000 7f6666a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6668070330 0x7f66680d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38112/0 (socket says 172.21.15.86:38112) 2024-04-25T14:06:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.743+0000 7f6666a00640 1 -- 172.21.15.86:0/830590110 learned_addr learned my addr 172.21.15.86:0/830590110 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.743+0000 7f6667400640 1 --2- 172.21.15.86:0/830590110 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66680ce790 0x7f666806f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.743+0000 7f6666a00640 1 -- 172.21.15.86:0/830590110 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66680ce790 msgr2=0x7f666806f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.743+0000 7f6666a00640 1 --2- 172.21.15.86:0/830590110 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66680ce790 0x7f666806f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.743+0000 7f6666a00640 1 -- 172.21.15.86:0/830590110 >> v1:172.21.15.86:6789/0 conn(0x7f66680d1c20 legacy=0x7f666806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.743+0000 7f6666a00640 1 -- 172.21.15.86:0/830590110 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666815eb80 con 0x7f6668070330 2024-04-25T14:06:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.743+0000 7f6666000640 1 -- 172.21.15.86:0/830590110 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1201741033 0 0) 0x7f6650001120 con 0x7f66680d1c20 2024-04-25T14:06:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.743+0000 7f6667400640 1 --2- 172.21.15.86:0/830590110 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66680ce790 0x7f666806f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:45.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.743+0000 7f6666a00640 1 --2- 172.21.15.86:0/830590110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6668070330 0x7f66680d16e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f6658009870 tx=0x7f665801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345ec29220f5ad38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:45.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.747+0000 7f6666000640 1 -- 172.21.15.86:0/830590110 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66580b0070 con 0x7f6668070330 2024-04-25T14:06:45.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.747+0000 7f6666000640 1 -- 172.21.15.86:0/830590110 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66580ab600 con 0x7f6668070330 2024-04-25T14:06:45.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.747+0000 7f6666000640 1 -- 172.21.15.86:0/830590110 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66580ab9d0 con 0x7f6668070330 2024-04-25T14:06:45.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.747+0000 7f666de00640 1 -- 172.21.15.86:0/830590110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6668070330 msgr2=0x7f66680d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:45.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.747+0000 7f666de00640 1 --2- 172.21.15.86:0/830590110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6668070330 0x7f66680d16e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f6658009870 tx=0x7f665801c4d0 comp rx=0 tx=0).stop 2024-04-25T14:06:45.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.747+0000 7f666de00640 1 -- 172.21.15.86:0/830590110 shutdown_connections 2024-04-25T14:06:45.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.747+0000 7f666de00640 1 --2- 172.21.15.86:0/830590110 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66680ce790 0x7f666806f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:45.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.747+0000 7f666de00640 1 --2- 172.21.15.86:0/830590110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6668070330 0x7f66680d16e0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:45.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.747+0000 7f666de00640 1 -- 172.21.15.86:0/830590110 >> 172.21.15.86:0/830590110 conn(0x7f66680b4650 msgr2=0x7f66680b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:45.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.747+0000 7f666de00640 1 -- 172.21.15.86:0/830590110 shutdown_connections 2024-04-25T14:06:45.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.747+0000 7f666de00640 1 -- 172.21.15.86:0/830590110 wait complete. 2024-04-25T14:06:45.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.747+0000 7f666de00640 1 Processor -- start 2024-04-25T14:06:45.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.747+0000 7f666de00640 1 -- start start 2024-04-25T14:06:45.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.751+0000 7f666de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6668070330 0x7f66680c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:45.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.751+0000 7f666de00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66680ce790 0x7f66680c2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:45.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.751+0000 7f666de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66680d1c20 0x7f66680cf310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:45.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.751+0000 7f666de00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f66680708c0 con 0x7f66680d1c20 2024-04-25T14:06:45.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.751+0000 7f666de00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6668070740 con 0x7f6668070330 2024-04-25T14:06:45.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.751+0000 7f666de00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f666816afc0 con 0x7f66680ce790 2024-04-25T14:06:45.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.751+0000 7f6667400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66680d1c20 0x7f66680cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:45.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.751+0000 7f6667400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66680d1c20 0x7f66680cf310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38114/0 (socket says 172.21.15.86:38114) 2024-04-25T14:06:45.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.751+0000 7f6667400640 1 -- 172.21.15.86:0/359375915 learned_addr learned my addr 172.21.15.86:0/359375915 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:45.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.751+0000 7f6666a00640 1 --2- 172.21.15.86:0/359375915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6668070330 0x7f66680c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:45.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.751+0000 7f665fe00640 1 --2- 172.21.15.86:0/359375915 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66680ce790 0x7f66680c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:45.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.751+0000 7f6667400640 1 -- 172.21.15.86:0/359375915 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66680ce790 msgr2=0x7f66680c2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:45.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.751+0000 7f6667400640 1 --2- 172.21.15.86:0/359375915 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66680ce790 0x7f66680c2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:45.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.751+0000 7f6667400640 1 -- 172.21.15.86:0/359375915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6668070330 msgr2=0x7f66680c7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:45.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.751+0000 7f6667400640 1 --2- 172.21.15.86:0/359375915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6668070330 0x7f66680c7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:45.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.751+0000 7f6667400640 1 -- 172.21.15.86:0/359375915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66680c7cb0 con 0x7f66680d1c20 2024-04-25T14:06:45.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.751+0000 7f6667400640 1 --2- 172.21.15.86:0/359375915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66680d1c20 0x7f66680cf310 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f666000cd80 tx=0x7f666000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:45.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.751+0000 7f665f400640 1 -- 172.21.15.86:0/359375915 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666000f680 con 0x7f66680d1c20 2024-04-25T14:06:45.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.751+0000 7f665f400640 1 -- 172.21.15.86:0/359375915 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f666000f840 con 0x7f66680d1c20 2024-04-25T14:06:45.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.751+0000 7f665f400640 1 -- 172.21.15.86:0/359375915 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666000fc30 con 0x7f66680d1c20 2024-04-25T14:06:45.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.751+0000 7f666de00640 1 -- 172.21.15.86:0/359375915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f666815c2a0 con 0x7f66680d1c20 2024-04-25T14:06:45.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.751+0000 7f666de00640 1 -- 172.21.15.86:0/359375915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f666815c860 con 0x7f66680d1c20 2024-04-25T14:06:45.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.751+0000 7f666de00640 1 -- 172.21.15.86:0/359375915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6668069d00 con 0x7f66680d1c20 2024-04-25T14:06:45.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.755+0000 7f665f400640 1 -- 172.21.15.86:0/359375915 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f666000e3e0 con 0x7f66680d1c20 2024-04-25T14:06:45.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.755+0000 7f665f400640 1 --2- 172.21.15.86:0/359375915 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f664406f640 0x7f6644071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:45.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.755+0000 7f665f400640 1 -- 172.21.15.86:0/359375915 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6101+0+0 (secure 0 0 0) 0x7f6660090620 con 0x7f66680d1c20 2024-04-25T14:06:45.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.755+0000 7f6666a00640 1 --2- 172.21.15.86:0/359375915 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f664406f640 0x7f6644071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:45.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.755+0000 7f6666a00640 1 --2- 172.21.15.86:0/359375915 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f664406f640 0x7f6644071b50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f66580ad000 tx=0x7f66580a6000 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:45.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.759+0000 7f665f400640 1 -- 172.21.15.86:0/359375915 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6660014030 con 0x7f66680d1c20 2024-04-25T14:06:45.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.915+0000 7f666de00640 1 -- 172.21.15.86:0/359375915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6668073070 con 0x7f66680d1c20 2024-04-25T14:06:45.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.915+0000 7f665f400640 1 -- 172.21.15.86:0/359375915 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+16958 (secure 0 0 0) 0x7f666005c820 con 0x7f66680d1c20 2024-04-25T14:06:45.919 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:45.919 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":73,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:06:44.910759+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:06:42.666742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T14:06:43.884074+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:06:45.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.919+0000 7f666de00640 1 -- 172.21.15.86:0/359375915 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f664406f640 msgr2=0x7f6644071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:45.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.919+0000 7f666de00640 1 --2- 172.21.15.86:0/359375915 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f664406f640 0x7f6644071b50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f66580ad000 tx=0x7f66580a6000 comp rx=0 tx=0).stop 2024-04-25T14:06:45.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.919+0000 7f666de00640 1 -- 172.21.15.86:0/359375915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66680d1c20 msgr2=0x7f66680cf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:45.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.919+0000 7f666de00640 1 --2- 172.21.15.86:0/359375915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66680d1c20 0x7f66680cf310 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f666000cd80 tx=0x7f666000a650 comp rx=0 tx=0).stop 2024-04-25T14:06:45.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.919+0000 7f666de00640 1 -- 172.21.15.86:0/359375915 shutdown_connections 2024-04-25T14:06:45.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.919+0000 7f666de00640 1 --2- 172.21.15.86:0/359375915 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f664406f640 0x7f6644071b50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:45.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.919+0000 7f666de00640 1 --2- 172.21.15.86:0/359375915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66680d1c20 0x7f66680cf310 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:45.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.919+0000 7f666de00640 1 --2- 172.21.15.86:0/359375915 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66680ce790 0x7f66680c2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:45.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.919+0000 7f666de00640 1 --2- 172.21.15.86:0/359375915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6668070330 0x7f66680c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:45.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.919+0000 7f666de00640 1 -- 172.21.15.86:0/359375915 >> 172.21.15.86:0/359375915 conn(0x7f66680b4650 msgr2=0x7f6668072880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:45.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.919+0000 7f666de00640 1 -- 172.21.15.86:0/359375915 shutdown_connections 2024-04-25T14:06:45.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:45.919+0000 7f666de00640 1 -- 172.21.15.86:0/359375915 wait complete. 2024-04-25T14:06:45.941 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:06:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.087+0000 7ff4c6800640 1 Processor -- start 2024-04-25T14:06:46.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.087+0000 7ff4c6800640 1 -- start start 2024-04-25T14:06:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.087+0000 7ff4c6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff4c00d2c80 0x7ff4c015dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.087+0000 7ff4c6800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4c01616e0 0x7ff4c0161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.087+0000 7ff4c6800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4c00cd8d0 con 0x7ff4c0070e20 2024-04-25T14:06:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.087+0000 7ff4c6800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff4c00ce720 con 0x7ff4c00d2c80 2024-04-25T14:06:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.087+0000 7ff4c6800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff4c00ce970 con 0x7ff4c01616e0 2024-04-25T14:06:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.087+0000 7ff4bf400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff4c00d2c80 0x7ff4c015dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.087+0000 7ff4bf400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff4c00d2c80 0x7ff4c015dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38130/0 (socket says 172.21.15.86:38130) 2024-04-25T14:06:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.087+0000 7ff4bf400640 1 -- 172.21.15.86:0/3820415710 learned_addr learned my addr 172.21.15.86:0/3820415710 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.087+0000 7ff4bfe00640 1 --2- 172.21.15.86:0/3820415710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4c01616e0 0x7ff4c0161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.087+0000 7ff4bf400640 1 -- 172.21.15.86:0/3820415710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4c01616e0 msgr2=0x7ff4c0161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:46.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.087+0000 7ff4bf400640 1 --2- 172.21.15.86:0/3820415710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4c01616e0 0x7ff4c0161af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:46.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.087+0000 7ff4bf400640 1 -- 172.21.15.86:0/3820415710 >> v1:172.21.15.155:6789/0 conn(0x7ff4c0070e20 legacy=0x7ff4c0071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:46.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.087+0000 7ff4bf400640 1 -- 172.21.15.86:0/3820415710 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c00d1be0 con 0x7ff4c00d2c80 2024-04-25T14:06:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.091+0000 7ff4bf400640 1 --2- 172.21.15.86:0/3820415710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4c00d2c80 0x7ff4c015dc90 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0009870 tx=0x7ff4b001ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b4961014fdae11c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.091+0000 7ff4be000640 1 -- 172.21.15.86:0/3820415710 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b00b5070 con 0x7ff4c00d2c80 2024-04-25T14:06:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.091+0000 7ff4be000640 1 -- 172.21.15.86:0/3820415710 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b00a8a50 con 0x7ff4c00d2c80 2024-04-25T14:06:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.091+0000 7ff4be000640 1 -- 172.21.15.86:0/3820415710 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b00a8e20 con 0x7ff4c00d2c80 2024-04-25T14:06:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.091+0000 7ff4c6800640 1 -- 172.21.15.86:0/3820415710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4c00d2c80 msgr2=0x7ff4c015dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.091+0000 7ff4c6800640 1 --2- 172.21.15.86:0/3820415710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4c00d2c80 0x7ff4c015dc90 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0009870 tx=0x7ff4b001ed50 comp rx=0 tx=0).stop 2024-04-25T14:06:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.091+0000 7ff4c6800640 1 -- 172.21.15.86:0/3820415710 shutdown_connections 2024-04-25T14:06:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.091+0000 7ff4c6800640 1 --2- 172.21.15.86:0/3820415710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4c01616e0 0x7ff4c0161af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.091+0000 7ff4c6800640 1 --2- 172.21.15.86:0/3820415710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4c00d2c80 0x7ff4c015dc90 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.091+0000 7ff4c6800640 1 -- 172.21.15.86:0/3820415710 >> 172.21.15.86:0/3820415710 conn(0x7ff4c00cffc0 msgr2=0x7ff4c00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.091+0000 7ff4c6800640 1 -- 172.21.15.86:0/3820415710 shutdown_connections 2024-04-25T14:06:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.091+0000 7ff4c6800640 1 -- 172.21.15.86:0/3820415710 wait complete. 2024-04-25T14:06:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.095+0000 7ff4c6800640 1 Processor -- start 2024-04-25T14:06:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.095+0000 7ff4c6800640 1 -- start start 2024-04-25T14:06:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.095+0000 7ff4c6800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4c0070e20 0x7ff4c00c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.095+0000 7ff4c6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4c00d2c80 0x7ff4c0151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.095+0000 7ff4c6800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4c01616e0 0x7ff4c015cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.095+0000 7ff4c6800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c00d2480 con 0x7ff4c00d2c80 2024-04-25T14:06:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.095+0000 7ff4c6800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c00d21a0 con 0x7ff4c0070e20 2024-04-25T14:06:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.095+0000 7ff4c6800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff4c00d29a0 con 0x7ff4c01616e0 2024-04-25T14:06:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.095+0000 7ff4bea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4c00d2c80 0x7ff4c0151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.095+0000 7ff4bea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4c00d2c80 0x7ff4c0151730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38134/0 (socket says 172.21.15.86:38134) 2024-04-25T14:06:46.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.095+0000 7ff4bea00640 1 -- 172.21.15.86:0/2446921881 learned_addr learned my addr 172.21.15.86:0/2446921881 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:46.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.095+0000 7ff4bf400640 1 --2- 172.21.15.86:0/2446921881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4c0070e20 0x7ff4c00c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:46.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.095+0000 7ff4bfe00640 1 --2- 172.21.15.86:0/2446921881 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4c01616e0 0x7ff4c015cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:46.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.099+0000 7ff4bf400640 1 -- 172.21.15.86:0/2446921881 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4c01616e0 msgr2=0x7ff4c015cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:46.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.099+0000 7ff4bf400640 1 --2- 172.21.15.86:0/2446921881 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4c01616e0 0x7ff4c015cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:46.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.099+0000 7ff4bf400640 1 -- 172.21.15.86:0/2446921881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4c00d2c80 msgr2=0x7ff4c0151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:46.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.099+0000 7ff4bf400640 1 --2- 172.21.15.86:0/2446921881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4c00d2c80 0x7ff4c0151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:46.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.099+0000 7ff4bf400640 1 -- 172.21.15.86:0/2446921881 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c0151c70 con 0x7ff4c0070e20 2024-04-25T14:06:46.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.099+0000 7ff4bea00640 1 --2- 172.21.15.86:0/2446921881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4c00d2c80 0x7ff4c0151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:46.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.099+0000 7ff4bf400640 1 --2- 172.21.15.86:0/2446921881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4c0070e20 0x7ff4c00c92d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff4c00733a0 tx=0x7ff4b001baf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:46.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.099+0000 7ff4b7400640 1 -- 172.21.15.86:0/2446921881 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b00c0070 con 0x7ff4c0070e20 2024-04-25T14:06:46.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.099+0000 7ff4b7400640 1 -- 172.21.15.86:0/2446921881 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b00ae550 con 0x7ff4c0070e20 2024-04-25T14:06:46.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.099+0000 7ff4c6800640 1 -- 172.21.15.86:0/2446921881 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c0152c20 con 0x7ff4c0070e20 2024-04-25T14:06:46.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.099+0000 7ff4b7400640 1 -- 172.21.15.86:0/2446921881 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b00b5070 con 0x7ff4c0070e20 2024-04-25T14:06:46.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.099+0000 7ff4c6800640 1 -- 172.21.15.86:0/2446921881 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c0154180 con 0x7ff4c0070e20 2024-04-25T14:06:46.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.099+0000 7ff4c6800640 1 -- 172.21.15.86:0/2446921881 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4c00720e0 con 0x7ff4c0070e20 2024-04-25T14:06:46.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.099+0000 7ff4b7400640 1 -- 172.21.15.86:0/2446921881 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7ff4b00b9070 con 0x7ff4c0070e20 2024-04-25T14:06:46.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.103+0000 7ff4b7400640 1 --2- 172.21.15.86:0/2446921881 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7ff4a006f670 0x7ff4a0071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:46.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.103+0000 7ff4b7400640 1 -- 172.21.15.86:0/2446921881 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6101+0+0 (secure 0 0 0) 0x7ff4b012d1a0 con 0x7ff4c0070e20 2024-04-25T14:06:46.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.103+0000 7ff4bea00640 1 --2- 172.21.15.86:0/2446921881 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7ff4a006f670 0x7ff4a0071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:46.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.103+0000 7ff4bea00640 1 --2- 172.21.15.86:0/2446921881 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7ff4a006f670 0x7ff4a0071b80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff4c00cce60 tx=0x7ff4ac009040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:46.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.107+0000 7ff4b7400640 1 -- 172.21.15.86:0/2446921881 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff4b00c9020 con 0x7ff4c0070e20 2024-04-25T14:06:46.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.263+0000 7ff4c6800640 1 -- 172.21.15.86:0/2446921881 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4c0076bf0 con 0x7ff4c0070e20 2024-04-25T14:06:46.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.263+0000 7ff4b7400640 1 -- 172.21.15.86:0/2446921881 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+16958 (secure 0 0 0) 0x7ff4b00188b0 con 0x7ff4c0070e20 2024-04-25T14:06:46.267 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:46.267 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":73,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:06:44.910759+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:06:42.666742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T14:06:43.884074+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:06:46.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.267+0000 7ff4c6800640 1 -- 172.21.15.86:0/2446921881 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7ff4a006f670 msgr2=0x7ff4a0071b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:46.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.267+0000 7ff4c6800640 1 --2- 172.21.15.86:0/2446921881 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7ff4a006f670 0x7ff4a0071b80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff4c00cce60 tx=0x7ff4ac009040 comp rx=0 tx=0).stop 2024-04-25T14:06:46.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.267+0000 7ff4c6800640 1 -- 172.21.15.86:0/2446921881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4c0070e20 msgr2=0x7ff4c00c92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:46.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.267+0000 7ff4c6800640 1 --2- 172.21.15.86:0/2446921881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4c0070e20 0x7ff4c00c92d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff4c00733a0 tx=0x7ff4b001baf0 comp rx=0 tx=0).stop 2024-04-25T14:06:46.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.267+0000 7ff4c6800640 1 -- 172.21.15.86:0/2446921881 shutdown_connections 2024-04-25T14:06:46.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.267+0000 7ff4c6800640 1 --2- 172.21.15.86:0/2446921881 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7ff4a006f670 0x7ff4a0071b80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:46.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.267+0000 7ff4c6800640 1 --2- 172.21.15.86:0/2446921881 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4c01616e0 0x7ff4c015cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:46.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.267+0000 7ff4c6800640 1 --2- 172.21.15.86:0/2446921881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4c00d2c80 0x7ff4c0151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:46.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.267+0000 7ff4c6800640 1 --2- 172.21.15.86:0/2446921881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4c0070e20 0x7ff4c00c92d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:46.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.267+0000 7ff4c6800640 1 -- 172.21.15.86:0/2446921881 >> 172.21.15.86:0/2446921881 conn(0x7ff4c00cffc0 msgr2=0x7ff4c00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:46.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.267+0000 7ff4c6800640 1 -- 172.21.15.86:0/2446921881 shutdown_connections 2024-04-25T14:06:46.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.267+0000 7ff4c6800640 1 -- 172.21.15.86:0/2446921881 wait complete. 2024-04-25T14:06:46.290 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T14:06:46.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.439+0000 7efdc2600640 1 Processor -- start 2024-04-25T14:06:46.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.439+0000 7efdc2600640 1 -- start start 2024-04-25T14:06:46.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.439+0000 7efdc2600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7efdbc070330 0x7efdbc0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:46.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.439+0000 7efdc2600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdbc0ce790 0x7efdbc06f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:46.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.439+0000 7efdc2600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdbc0d2030 con 0x7efdbc0d1c20 2024-04-25T14:06:46.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.439+0000 7efdc2600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7efdbc15e5c0 con 0x7efdbc070330 2024-04-25T14:06:46.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.439+0000 7efdc2600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efdbc15e8a0 con 0x7efdbc0ce790 2024-04-25T14:06:46.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.439+0000 7efdbb400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7efdbc070330 0x7efdbc0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:46.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.439+0000 7efdbbe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdbc0ce790 0x7efdbc06f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:46.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.439+0000 7efdbb400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7efdbc070330 0x7efdbc0d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:59492/0 (socket says 172.21.15.86:59492) 2024-04-25T14:06:46.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.439+0000 7efdbb400640 1 -- 172.21.15.86:0/1682755296 learned_addr learned my addr 172.21.15.86:0/1682755296 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:46.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.439+0000 7efdb2a00640 1 --1- 172.21.15.86:0/1682755296 >> v1:172.21.15.155:6789/0 conn(0x7efdbc0d1c20 0x7efdbc06ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:42830/0 (socket says 172.21.15.86:42830) 2024-04-25T14:06:46.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.439+0000 7efdbb400640 1 -- 172.21.15.86:0/1682755296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdbc0ce790 msgr2=0x7efdbc06f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:46.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.439+0000 7efdbb400640 1 --2- 172.21.15.86:0/1682755296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdbc0ce790 0x7efdbc06f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:46.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.439+0000 7efdbb400640 1 -- 172.21.15.86:0/1682755296 >> v1:172.21.15.155:6789/0 conn(0x7efdbc0d1c20 legacy=0x7efdbc06ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:46.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.439+0000 7efdbb400640 1 -- 172.21.15.86:0/1682755296 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdbc15eb80 con 0x7efdbc070330 2024-04-25T14:06:46.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.443+0000 7efdbb400640 1 --2- 172.21.15.86:0/1682755296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdbc070330 0x7efdbc0d16e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7efdac009e50 tx=0x7efdac01c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a72a0746d3e1e689 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:46.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.443+0000 7efdbaa00640 1 -- 172.21.15.86:0/1682755296 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac0a5660 con 0x7efdbc070330 2024-04-25T14:06:46.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.443+0000 7efdbaa00640 1 -- 172.21.15.86:0/1682755296 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdac0a5800 con 0x7efdbc070330 2024-04-25T14:06:46.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.443+0000 7efdbaa00640 1 -- 172.21.15.86:0/1682755296 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac0a5bd0 con 0x7efdbc070330 2024-04-25T14:06:46.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.443+0000 7efdc2600640 1 -- 172.21.15.86:0/1682755296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdbc070330 msgr2=0x7efdbc0d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:46.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.443+0000 7efdc2600640 1 --2- 172.21.15.86:0/1682755296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdbc070330 0x7efdbc0d16e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7efdac009e50 tx=0x7efdac01c9a0 comp rx=0 tx=0).stop 2024-04-25T14:06:46.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.443+0000 7efdc2600640 1 -- 172.21.15.86:0/1682755296 shutdown_connections 2024-04-25T14:06:46.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.443+0000 7efdc2600640 1 --2- 172.21.15.86:0/1682755296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdbc0ce790 0x7efdbc06f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:46.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.443+0000 7efdc2600640 1 --2- 172.21.15.86:0/1682755296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdbc070330 0x7efdbc0d16e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:46.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.443+0000 7efdc2600640 1 -- 172.21.15.86:0/1682755296 >> 172.21.15.86:0/1682755296 conn(0x7efdbc0b4650 msgr2=0x7efdbc0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:46.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.443+0000 7efdc2600640 1 -- 172.21.15.86:0/1682755296 shutdown_connections 2024-04-25T14:06:46.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.443+0000 7efdc2600640 1 -- 172.21.15.86:0/1682755296 wait complete. 2024-04-25T14:06:46.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.447+0000 7efdc2600640 1 Processor -- start 2024-04-25T14:06:46.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.447+0000 7efdc2600640 1 -- start start 2024-04-25T14:06:46.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.447+0000 7efdc2600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdbc070330 0x7efdbc0ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:46.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.447+0000 7efdc2600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdbc0ce790 0x7efdbc1f9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:46.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.447+0000 7efdc2600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdbc0d1c20 0x7efdbc1fa230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:46.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.447+0000 7efdc2600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efdbc0708c0 con 0x7efdbc070330 2024-04-25T14:06:46.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.447+0000 7efdc2600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efdbc070740 con 0x7efdbc0ce790 2024-04-25T14:06:46.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.447+0000 7efdc2600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efdbc16afc0 con 0x7efdbc0d1c20 2024-04-25T14:06:46.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.447+0000 7efdbb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdbc070330 0x7efdbc0ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:46.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.447+0000 7efdbb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdbc070330 0x7efdbc0ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38140/0 (socket says 172.21.15.86:38140) 2024-04-25T14:06:46.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.447+0000 7efdbb400640 1 -- 172.21.15.86:0/2310362937 learned_addr learned my addr 172.21.15.86:0/2310362937 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:46.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.447+0000 7efdbbe00640 1 --2- 172.21.15.86:0/2310362937 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdbc0d1c20 0x7efdbc1fa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:46.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.447+0000 7efdb2a00640 1 --2- 172.21.15.86:0/2310362937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdbc0ce790 0x7efdbc1f9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:46.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.447+0000 7efdbb400640 1 -- 172.21.15.86:0/2310362937 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdbc0d1c20 msgr2=0x7efdbc1fa230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:46.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.447+0000 7efdbb400640 1 --2- 172.21.15.86:0/2310362937 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdbc0d1c20 0x7efdbc1fa230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:46.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.447+0000 7efdbb400640 1 -- 172.21.15.86:0/2310362937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdbc0ce790 msgr2=0x7efdbc1f9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:46.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.447+0000 7efdbb400640 1 --2- 172.21.15.86:0/2310362937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdbc0ce790 0x7efdbc1f9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:46.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.447+0000 7efdbb400640 1 -- 172.21.15.86:0/2310362937 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdbc0741e0 con 0x7efdbc070330 2024-04-25T14:06:46.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.447+0000 7efdb2a00640 1 --2- 172.21.15.86:0/2310362937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdbc0ce790 0x7efdbc1f9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:46.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.451+0000 7efdbb400640 1 --2- 172.21.15.86:0/2310362937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdbc070330 0x7efdbc0ceeb0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7efdac01a5d0 tx=0x7efdac019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:46.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.451+0000 7efdb3e00640 1 -- 172.21.15.86:0/2310362937 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac0bc020 con 0x7efdbc070330 2024-04-25T14:06:46.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.451+0000 7efdb3e00640 1 -- 172.21.15.86:0/2310362937 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdac0b8670 con 0x7efdbc070330 2024-04-25T14:06:46.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.451+0000 7efdb3e00640 1 -- 172.21.15.86:0/2310362937 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac009510 con 0x7efdbc070330 2024-04-25T14:06:46.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.451+0000 7efdc2600640 1 -- 172.21.15.86:0/2310362937 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdbc0ca380 con 0x7efdbc070330 2024-04-25T14:06:46.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.451+0000 7efdc2600640 1 -- 172.21.15.86:0/2310362937 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdbc073f00 con 0x7efdbc070330 2024-04-25T14:06:46.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.451+0000 7efdc2600640 1 -- 172.21.15.86:0/2310362937 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdbc069d00 con 0x7efdbc070330 2024-04-25T14:06:46.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.451+0000 7efdb3e00640 1 -- 172.21.15.86:0/2310362937 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7efdac0bf020 con 0x7efdbc070330 2024-04-25T14:06:46.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.451+0000 7efdb3e00640 1 --2- 172.21.15.86:0/2310362937 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7efd9806f640 0x7efd98071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:46.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.451+0000 7efdb3e00640 1 -- 172.21.15.86:0/2310362937 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6101+0+0 (secure 0 0 0) 0x7efdac12a690 con 0x7efdbc070330 2024-04-25T14:06:46.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.451+0000 7efdb2a00640 1 --2- 172.21.15.86:0/2310362937 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7efd9806f640 0x7efd98071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:46.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.455+0000 7efdb2a00640 1 --2- 172.21.15.86:0/2310362937 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7efd9806f640 0x7efd98071b50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efdbc06f830 tx=0x7efda4007d90 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:46.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.459+0000 7efdb3e00640 1 -- 172.21.15.86:0/2310362937 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efdac0c6030 con 0x7efdbc070330 2024-04-25T14:06:46.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:46.655+0000 7efdc2600640 1 -- 172.21.15.86:0/2310362937 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7efdbc076bf0 con 0x7efdbc070330 2024-04-25T14:06:47.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.555+0000 7efdb3e00640 1 -- 172.21.15.86:0/2310362937 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v52) ==== 118+0+0 (secure 0 0 0) 0x7efdac016620 con 0x7efdbc070330 2024-04-25T14:06:47.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.559+0000 7efdc2600640 1 -- 172.21.15.86:0/2310362937 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7efd9806f640 msgr2=0x7efd98071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:47.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.559+0000 7efdc2600640 1 --2- 172.21.15.86:0/2310362937 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7efd9806f640 0x7efd98071b50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efdbc06f830 tx=0x7efda4007d90 comp rx=0 tx=0).stop 2024-04-25T14:06:47.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.559+0000 7efdc2600640 1 -- 172.21.15.86:0/2310362937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdbc070330 msgr2=0x7efdbc0ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:47.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.559+0000 7efdc2600640 1 --2- 172.21.15.86:0/2310362937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdbc070330 0x7efdbc0ceeb0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7efdac01a5d0 tx=0x7efdac019740 comp rx=0 tx=0).stop 2024-04-25T14:06:47.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.559+0000 7efdc2600640 1 -- 172.21.15.86:0/2310362937 shutdown_connections 2024-04-25T14:06:47.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.559+0000 7efdc2600640 1 --2- 172.21.15.86:0/2310362937 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7efd9806f640 0x7efd98071b50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:47.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.559+0000 7efdc2600640 1 --2- 172.21.15.86:0/2310362937 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdbc0d1c20 0x7efdbc1fa230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:47.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.559+0000 7efdc2600640 1 --2- 172.21.15.86:0/2310362937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdbc0ce790 0x7efdbc1f9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:47.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.559+0000 7efdc2600640 1 --2- 172.21.15.86:0/2310362937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdbc070330 0x7efdbc0ceeb0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:47.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.559+0000 7efdc2600640 1 -- 172.21.15.86:0/2310362937 >> 172.21.15.86:0/2310362937 conn(0x7efdbc0b4650 msgr2=0x7efdbc0b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:47.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.559+0000 7efdc2600640 1 -- 172.21.15.86:0/2310362937 shutdown_connections 2024-04-25T14:06:47.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.559+0000 7efdc2600640 1 -- 172.21.15.86:0/2310362937 wait complete. 2024-04-25T14:06:47.584 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T14:06:47.585 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T14:06:47.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.739+0000 7fc0c6400640 1 Processor -- start 2024-04-25T14:06:47.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.739+0000 7fc0c6400640 1 -- start start 2024-04-25T14:06:47.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.739+0000 7fc0c6400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0c0070330 0x7fc0c00d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:47.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.739+0000 7fc0c6400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc0c00d1c20 0x7fc0c006ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:47.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.739+0000 7fc0c6400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0c00d2030 con 0x7fc0c006f270 2024-04-25T14:06:47.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.739+0000 7fc0c6400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc0c015e5c0 con 0x7fc0c00d1c20 2024-04-25T14:06:47.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.739+0000 7fc0c6400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c015e8a0 con 0x7fc0c0070330 2024-04-25T14:06:47.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.739+0000 7fc0bf400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0c0070330 0x7fc0c00d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:47.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.739+0000 7fc0bea00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc0c00d1c20 0x7fc0c006ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:47.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.739+0000 7fc0bea00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc0c00d1c20 0x7fc0c006ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:59522/0 (socket says 172.21.15.86:59522) 2024-04-25T14:06:47.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.739+0000 7fc0bea00640 1 -- 172.21.15.86:0/3099084917 learned_addr learned my addr 172.21.15.86:0/3099084917 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:47.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.739+0000 7fc0bf400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0c0070330 0x7fc0c00d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:39034/0 (socket says 172.21.15.86:39034) 2024-04-25T14:06:47.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.739+0000 7fc0bf400640 1 -- 172.21.15.86:0/3099084917 >> v2:172.21.15.155:3300/0 conn(0x7fc0c00d1c20 msgr2=0x7fc0c006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:47.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.739+0000 7fc0bf400640 1 --2- 172.21.15.86:0/3099084917 >> v2:172.21.15.155:3300/0 conn(0x7fc0c00d1c20 0x7fc0c006ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:47.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.739+0000 7fc0bf400640 1 -- 172.21.15.86:0/3099084917 >> v1:172.21.15.155:6789/0 conn(0x7fc0c006f270 legacy=0x7fc0c006f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:47.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.739+0000 7fc0bf400640 1 -- 172.21.15.86:0/3099084917 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c015eb80 con 0x7fc0c0070330 2024-04-25T14:06:47.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.739+0000 7fc0bea00640 1 --2- 172.21.15.86:0/3099084917 >> v2:172.21.15.155:3300/0 conn(0x7fc0c00d1c20 0x7fc0c006ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:47.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.743+0000 7fc0bf400640 1 --2- 172.21.15.86:0/3099084917 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0c0070330 0x7fc0c00d16e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc0b00097f0 tx=0x7fc0b001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fcf54e1aa5f4625 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:47.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.743+0000 7fc0be000640 1 -- 172.21.15.86:0/3099084917 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b00b0070 con 0x7fc0c0070330 2024-04-25T14:06:47.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.743+0000 7fc0be000640 1 -- 172.21.15.86:0/3099084917 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0b00ab600 con 0x7fc0c0070330 2024-04-25T14:06:47.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.743+0000 7fc0be000640 1 -- 172.21.15.86:0/3099084917 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b00ab9d0 con 0x7fc0c0070330 2024-04-25T14:06:47.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.743+0000 7fc0c6400640 1 -- 172.21.15.86:0/3099084917 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0c0070330 msgr2=0x7fc0c00d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:47.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.743+0000 7fc0c6400640 1 --2- 172.21.15.86:0/3099084917 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0c0070330 0x7fc0c00d16e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc0b00097f0 tx=0x7fc0b001c410 comp rx=0 tx=0).stop 2024-04-25T14:06:47.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.743+0000 7fc0c6400640 1 -- 172.21.15.86:0/3099084917 shutdown_connections 2024-04-25T14:06:47.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.743+0000 7fc0c6400640 1 --2- 172.21.15.86:0/3099084917 >> v2:172.21.15.155:3300/0 conn(0x7fc0c00d1c20 0x7fc0c006ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:47.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.743+0000 7fc0c6400640 1 --2- 172.21.15.86:0/3099084917 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0c0070330 0x7fc0c00d16e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:47.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.743+0000 7fc0c6400640 1 -- 172.21.15.86:0/3099084917 >> 172.21.15.86:0/3099084917 conn(0x7fc0c00b4650 msgr2=0x7fc0c00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:47.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.743+0000 7fc0c6400640 1 -- 172.21.15.86:0/3099084917 shutdown_connections 2024-04-25T14:06:47.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.743+0000 7fc0c6400640 1 -- 172.21.15.86:0/3099084917 wait complete. 2024-04-25T14:06:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.747+0000 7fc0c6400640 1 Processor -- start 2024-04-25T14:06:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.747+0000 7fc0c6400640 1 -- start start 2024-04-25T14:06:47.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.747+0000 7fc0c6400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0c006f270 0x7fc0c00c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:47.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.747+0000 7fc0c6400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0c0070330 0x7fc0c00c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:47.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.747+0000 7fc0c6400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0c00d1c20 0x7fc0c00cff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:47.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.747+0000 7fc0c6400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c00708c0 con 0x7fc0c006f270 2024-04-25T14:06:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.747+0000 7fc0c6400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c0070740 con 0x7fc0c00d1c20 2024-04-25T14:06:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.747+0000 7fc0c6400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c016afc0 con 0x7fc0c0070330 2024-04-25T14:06:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.747+0000 7fc0bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0c006f270 0x7fc0c00c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.747+0000 7fc0bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0c006f270 0x7fc0c00c6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38146/0 (socket says 172.21.15.86:38146) 2024-04-25T14:06:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.747+0000 7fc0bf400640 1 -- 172.21.15.86:0/1424854522 learned_addr learned my addr 172.21.15.86:0/1424854522 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.751+0000 7fc0bfe00640 1 --2- 172.21.15.86:0/1424854522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0c00d1c20 0x7fc0c00cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.751+0000 7fc0bea00640 1 --2- 172.21.15.86:0/1424854522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0c0070330 0x7fc0c00c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.751+0000 7fc0bf400640 1 -- 172.21.15.86:0/1424854522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0c0070330 msgr2=0x7fc0c00c3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.751+0000 7fc0bf400640 1 --2- 172.21.15.86:0/1424854522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0c0070330 0x7fc0c00c3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.751+0000 7fc0bf400640 1 -- 172.21.15.86:0/1424854522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0c00d1c20 msgr2=0x7fc0c00cff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.751+0000 7fc0bf400640 1 --2- 172.21.15.86:0/1424854522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0c00d1c20 0x7fc0c00cff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.751+0000 7fc0bf400640 1 -- 172.21.15.86:0/1424854522 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c00d16e0 con 0x7fc0c006f270 2024-04-25T14:06:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.751+0000 7fc0bfe00640 1 --2- 172.21.15.86:0/1424854522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0c00d1c20 0x7fc0c00cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.751+0000 7fc0bea00640 1 --2- 172.21.15.86:0/1424854522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0c0070330 0x7fc0c00c3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:47.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.751+0000 7fc0bf400640 1 --2- 172.21.15.86:0/1424854522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0c006f270 0x7fc0c00c6170 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fc0b00ae040 tx=0x7fc0b001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:47.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.751+0000 7fc0b7400640 1 -- 172.21.15.86:0/1424854522 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b00b0050 con 0x7fc0c006f270 2024-04-25T14:06:47.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.751+0000 7fc0b7400640 1 -- 172.21.15.86:0/1424854522 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0b00bc070 con 0x7fc0c006f270 2024-04-25T14:06:47.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.751+0000 7fc0b7400640 1 -- 172.21.15.86:0/1424854522 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b00b5830 con 0x7fc0c006f270 2024-04-25T14:06:47.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.751+0000 7fc0c6400640 1 -- 172.21.15.86:0/1424854522 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c00c2dc0 con 0x7fc0c006f270 2024-04-25T14:06:47.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.751+0000 7fc0c6400640 1 -- 172.21.15.86:0/1424854522 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c00c3380 con 0x7fc0c006f270 2024-04-25T14:06:47.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.751+0000 7fc0c6400640 1 -- 172.21.15.86:0/1424854522 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0c0069d00 con 0x7fc0c006f270 2024-04-25T14:06:47.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.751+0000 7fc0b7400640 1 -- 172.21.15.86:0/1424854522 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7fc0b00b59d0 con 0x7fc0c006f270 2024-04-25T14:06:47.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.755+0000 7fc0b7400640 1 --2- 172.21.15.86:0/1424854522 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fc0a006f640 0x7fc0a0071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:47.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.755+0000 7fc0b7400640 1 -- 172.21.15.86:0/1424854522 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7fc0b0129fb0 con 0x7fc0c006f270 2024-04-25T14:06:47.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.755+0000 7fc0bea00640 1 --2- 172.21.15.86:0/1424854522 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fc0a006f640 0x7fc0a0071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:47.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.755+0000 7fc0bea00640 1 --2- 172.21.15.86:0/1424854522 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fc0a006f640 0x7fc0a0071b50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00c8b30 tx=0x7fc0ac008040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:47.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.759+0000 7fc0b7400640 1 -- 172.21.15.86:0/1424854522 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc0b00c6020 con 0x7fc0c006f270 2024-04-25T14:06:47.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:47.947+0000 7fc0c6400640 1 -- 172.21.15.86:0/1424854522 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc0c0076bf0 con 0x7fc0c006f270 2024-04-25T14:06:48.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.559+0000 7fc0b7400640 1 -- 172.21.15.86:0/1424854522 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v53) ==== 122+0+0 (secure 0 0 0) 0x7fc0b0016090 con 0x7fc0c006f270 2024-04-25T14:06:48.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.559+0000 7fc0c6400640 1 -- 172.21.15.86:0/1424854522 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fc0a006f640 msgr2=0x7fc0a0071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:48.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.559+0000 7fc0c6400640 1 --2- 172.21.15.86:0/1424854522 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fc0a006f640 0x7fc0a0071b50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00c8b30 tx=0x7fc0ac008040 comp rx=0 tx=0).stop 2024-04-25T14:06:48.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.559+0000 7fc0c6400640 1 -- 172.21.15.86:0/1424854522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0c006f270 msgr2=0x7fc0c00c6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:48.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.559+0000 7fc0c6400640 1 --2- 172.21.15.86:0/1424854522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0c006f270 0x7fc0c00c6170 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fc0b00ae040 tx=0x7fc0b001ac50 comp rx=0 tx=0).stop 2024-04-25T14:06:48.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.559+0000 7fc0c6400640 1 -- 172.21.15.86:0/1424854522 shutdown_connections 2024-04-25T14:06:48.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.559+0000 7fc0c6400640 1 --2- 172.21.15.86:0/1424854522 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fc0a006f640 0x7fc0a0071b50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.559+0000 7fc0c6400640 1 --2- 172.21.15.86:0/1424854522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0c00d1c20 0x7fc0c00cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.559+0000 7fc0c6400640 1 --2- 172.21.15.86:0/1424854522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0c0070330 0x7fc0c00c3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.559+0000 7fc0c6400640 1 --2- 172.21.15.86:0/1424854522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0c006f270 0x7fc0c00c6170 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.559+0000 7fc0c6400640 1 -- 172.21.15.86:0/1424854522 >> 172.21.15.86:0/1424854522 conn(0x7fc0c00b4650 msgr2=0x7fc0c015bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:48.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.559+0000 7fc0c6400640 1 -- 172.21.15.86:0/1424854522 shutdown_connections 2024-04-25T14:06:48.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.559+0000 7fc0c6400640 1 -- 172.21.15.86:0/1424854522 wait complete. 2024-04-25T14:06:48.583 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T14:06:48.584 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:06:48.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.731+0000 7fea64200640 1 Processor -- start 2024-04-25T14:06:48.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.731+0000 7fea64200640 1 -- start start 2024-04-25T14:06:48.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.731+0000 7fea64200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fea5c0d1df0 0x7fea5c0cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:48.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.731+0000 7fea64200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fea5c0cdcc0 0x7fea5c0ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:48.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.731+0000 7fea64200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea5c15e5c0 con 0x7fea5c070330 2024-04-25T14:06:48.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.731+0000 7fea64200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fea5c06fd40 con 0x7fea5c0cdcc0 2024-04-25T14:06:48.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.731+0000 7fea64200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fea5c15e8a0 con 0x7fea5c0d1df0 2024-04-25T14:06:48.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.731+0000 7fea62200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fea5c0cdcc0 0x7fea5c0ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:48.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.731+0000 7fea62200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fea5c0cdcc0 0x7fea5c0ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38150/0 (socket says 172.21.15.86:38150) 2024-04-25T14:06:48.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.731+0000 7fea62200640 1 -- 172.21.15.86:0/2738973772 learned_addr learned my addr 172.21.15.86:0/2738973772 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:48.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.731+0000 7fea60e00640 1 --2- 172.21.15.86:0/2738973772 >> v2:172.21.15.155:3300/0 conn(0x7fea5c0d1df0 0x7fea5c0cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:48.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.731+0000 7fea62200640 1 -- 172.21.15.86:0/2738973772 >> v2:172.21.15.155:3300/0 conn(0x7fea5c0d1df0 msgr2=0x7fea5c0cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:48.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.731+0000 7fea62200640 1 --2- 172.21.15.86:0/2738973772 >> v2:172.21.15.155:3300/0 conn(0x7fea5c0d1df0 0x7fea5c0cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.731+0000 7fea62200640 1 -- 172.21.15.86:0/2738973772 >> v1:172.21.15.155:6789/0 conn(0x7fea5c070330 legacy=0x7fea5c0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:48.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.731+0000 7fea62200640 1 -- 172.21.15.86:0/2738973772 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5c15eb80 con 0x7fea5c0cdcc0 2024-04-25T14:06:48.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.735+0000 7fea62200640 1 --2- 172.21.15.86:0/2738973772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea5c0cdcc0 0x7fea5c0ce0d0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fea54009e50 tx=0x7fea5401c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4944f16b75791d54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:48.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.735+0000 7fea5be00640 1 -- 172.21.15.86:0/2738973772 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea540a5790 con 0x7fea5c0cdcc0 2024-04-25T14:06:48.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.735+0000 7fea5be00640 1 -- 172.21.15.86:0/2738973772 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea540a5930 con 0x7fea5c0cdcc0 2024-04-25T14:06:48.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.735+0000 7fea5be00640 1 -- 172.21.15.86:0/2738973772 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea540a5d00 con 0x7fea5c0cdcc0 2024-04-25T14:06:48.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.735+0000 7fea64200640 1 -- 172.21.15.86:0/2738973772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea5c0cdcc0 msgr2=0x7fea5c0ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:48.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.735+0000 7fea64200640 1 --2- 172.21.15.86:0/2738973772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea5c0cdcc0 0x7fea5c0ce0d0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fea54009e50 tx=0x7fea5401c8f0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.735+0000 7fea64200640 1 -- 172.21.15.86:0/2738973772 shutdown_connections 2024-04-25T14:06:48.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.735+0000 7fea64200640 1 --2- 172.21.15.86:0/2738973772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea5c0cdcc0 0x7fea5c0ce0d0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.735+0000 7fea64200640 1 --2- 172.21.15.86:0/2738973772 >> v2:172.21.15.155:3300/0 conn(0x7fea5c0d1df0 0x7fea5c0cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.735+0000 7fea64200640 1 -- 172.21.15.86:0/2738973772 >> 172.21.15.86:0/2738973772 conn(0x7fea5c0b4650 msgr2=0x7fea5c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:48.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.735+0000 7fea64200640 1 -- 172.21.15.86:0/2738973772 shutdown_connections 2024-04-25T14:06:48.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.735+0000 7fea64200640 1 -- 172.21.15.86:0/2738973772 wait complete. 2024-04-25T14:06:48.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.739+0000 7fea64200640 1 Processor -- start 2024-04-25T14:06:48.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.739+0000 7fea64200640 1 -- start start 2024-04-25T14:06:48.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.739+0000 7fea64200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea5c070330 0x7fea5c0aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:48.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.739+0000 7fea64200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea5c0cdcc0 0x7fea5c14dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:48.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.739+0000 7fea64200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea5c0af0b0 0x7fea5c14dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:48.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.739+0000 7fea64200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c15f4b0 con 0x7fea5c0af0b0 2024-04-25T14:06:48.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.739+0000 7fea64200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c15f330 con 0x7fea5c070330 2024-04-25T14:06:48.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.739+0000 7fea64200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fea5c070740 con 0x7fea5c0cdcc0 2024-04-25T14:06:48.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.739+0000 7fea62200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea5c0af0b0 0x7fea5c14dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:48.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.739+0000 7fea62200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea5c0af0b0 0x7fea5c14dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38156/0 (socket says 172.21.15.86:38156) 2024-04-25T14:06:48.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.739+0000 7fea62200640 1 -- 172.21.15.86:0/2499986506 learned_addr learned my addr 172.21.15.86:0/2499986506 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:48.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.739+0000 7fea61800640 1 --2- 172.21.15.86:0/2499986506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea5c070330 0x7fea5c0aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:48.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.739+0000 7fea60e00640 1 --2- 172.21.15.86:0/2499986506 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea5c0cdcc0 0x7fea5c14dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:48.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.743+0000 7fea62200640 1 -- 172.21.15.86:0/2499986506 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea5c0cdcc0 msgr2=0x7fea5c14dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:48.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.743+0000 7fea62200640 1 --2- 172.21.15.86:0/2499986506 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea5c0cdcc0 0x7fea5c14dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.743+0000 7fea62200640 1 -- 172.21.15.86:0/2499986506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea5c070330 msgr2=0x7fea5c0aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:48.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.743+0000 7fea62200640 1 --2- 172.21.15.86:0/2499986506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea5c070330 0x7fea5c0aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.743+0000 7fea62200640 1 -- 172.21.15.86:0/2499986506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5c073570 con 0x7fea5c0af0b0 2024-04-25T14:06:48.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.743+0000 7fea62200640 1 --2- 172.21.15.86:0/2499986506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea5c0af0b0 0x7fea5c14dff0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fea54009e50 tx=0x7fea54019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:48.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.743+0000 7fea59600640 1 -- 172.21.15.86:0/2499986506 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea540b8670 con 0x7fea5c0af0b0 2024-04-25T14:06:48.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.743+0000 7fea59600640 1 -- 172.21.15.86:0/2499986506 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea540be070 con 0x7fea5c0af0b0 2024-04-25T14:06:48.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.743+0000 7fea59600640 1 -- 172.21.15.86:0/2499986506 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea540b8a70 con 0x7fea5c0af0b0 2024-04-25T14:06:48.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.743+0000 7fea64200640 1 -- 172.21.15.86:0/2499986506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea5c073850 con 0x7fea5c0af0b0 2024-04-25T14:06:48.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.743+0000 7fea64200640 1 -- 172.21.15.86:0/2499986506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea5c073e10 con 0x7fea5c0af0b0 2024-04-25T14:06:48.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.743+0000 7fea64200640 1 -- 172.21.15.86:0/2499986506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea5c069d00 con 0x7fea5c0af0b0 2024-04-25T14:06:48.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.743+0000 7fea59600640 1 -- 172.21.15.86:0/2499986506 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7fea540b8c10 con 0x7fea5c0af0b0 2024-04-25T14:06:48.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.747+0000 7fea59600640 1 --2- 172.21.15.86:0/2499986506 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fea3c06f640 0x7fea3c071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:48.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.747+0000 7fea59600640 1 -- 172.21.15.86:0/2499986506 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7fea5412a710 con 0x7fea5c0af0b0 2024-04-25T14:06:48.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.747+0000 7fea61800640 1 --2- 172.21.15.86:0/2499986506 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fea3c06f640 0x7fea3c071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:48.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.747+0000 7fea61800640 1 --2- 172.21.15.86:0/2499986506 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fea3c06f640 0x7fea3c071b50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fea5c072200 tx=0x7fea50009060 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:48.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.751+0000 7fea59600640 1 -- 172.21.15.86:0/2499986506 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fea540c8020 con 0x7fea5c0af0b0 2024-04-25T14:06:48.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.939+0000 7fea64200640 1 -- 172.21.15.86:0/2499986506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea5c076bf0 con 0x7fea5c0af0b0 2024-04-25T14:06:48.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.939+0000 7fea59600640 1 -- 172.21.15.86:0/2499986506 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5189 (secure 0 0 0) 0x7fea540165a0 con 0x7fea5c0af0b0 2024-04-25T14:06:48.941 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:48.942 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1210051860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1210051860},{"type":"v1","addr":"172.21.15.155:6833","nonce":1210051860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2868300438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2868300438},{"type":"v1","addr":"172.21.15.155:6835","nonce":2868300438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6600,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2682115891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2682115891},{"type":"v1","addr":"172.21.15.86:6835","nonce":2682115891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:06:44.910717+0000","modified":"2024-04-25T14:06:47.953301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6603},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6603":{"gid":6603,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/1088108173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1088108173},{"type":"v1","addr":"172.21.15.86:6837","nonce":1088108173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6603,"qdb_cluster":[6603]},"id":5}]} 2024-04-25T14:06:48.942 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 53 2024-04-25T14:06:48.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.939+0000 7fea64200640 1 -- 172.21.15.86:0/2499986506 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fea3c06f640 msgr2=0x7fea3c071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:48.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.939+0000 7fea64200640 1 --2- 172.21.15.86:0/2499986506 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fea3c06f640 0x7fea3c071b50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fea5c072200 tx=0x7fea50009060 comp rx=0 tx=0).stop 2024-04-25T14:06:48.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.939+0000 7fea64200640 1 -- 172.21.15.86:0/2499986506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea5c0af0b0 msgr2=0x7fea5c14dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:48.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.939+0000 7fea64200640 1 --2- 172.21.15.86:0/2499986506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea5c0af0b0 0x7fea5c14dff0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fea54009e50 tx=0x7fea54019690 comp rx=0 tx=0).stop 2024-04-25T14:06:48.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.943+0000 7fea64200640 1 -- 172.21.15.86:0/2499986506 shutdown_connections 2024-04-25T14:06:48.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.943+0000 7fea64200640 1 --2- 172.21.15.86:0/2499986506 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fea3c06f640 0x7fea3c071b50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.943+0000 7fea64200640 1 --2- 172.21.15.86:0/2499986506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea5c0af0b0 0x7fea5c14dff0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.943+0000 7fea64200640 1 --2- 172.21.15.86:0/2499986506 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea5c0cdcc0 0x7fea5c14dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.943+0000 7fea64200640 1 --2- 172.21.15.86:0/2499986506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea5c070330 0x7fea5c0aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.943+0000 7fea64200640 1 -- 172.21.15.86:0/2499986506 >> 172.21.15.86:0/2499986506 conn(0x7fea5c0b4650 msgr2=0x7fea5c0b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:48.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.943+0000 7fea64200640 1 -- 172.21.15.86:0/2499986506 shutdown_connections 2024-04-25T14:06:48.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:48.943+0000 7fea64200640 1 -- 172.21.15.86:0/2499986506 wait complete. 2024-04-25T14:06:48.964 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:06:49.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.107+0000 7f3b4cc00640 1 Processor -- start 2024-04-25T14:06:49.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.107+0000 7f3b4cc00640 1 -- start start 2024-04-25T14:06:49.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.111+0000 7f3b4cc00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3b48072630 0x7f3b48072a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:49.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.111+0000 7f3b4cc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b4806e340 0x7f3b4806e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:49.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.111+0000 7f3b4cc00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b48072f80 con 0x7f3b48163980 2024-04-25T14:06:49.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.111+0000 7f3b4cc00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3b481648e0 con 0x7f3b48072630 2024-04-25T14:06:49.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.111+0000 7f3b4cc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3b480d1520 con 0x7f3b4806e340 2024-04-25T14:06:49.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.111+0000 7f3b45e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3b48072630 0x7f3b48072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:49.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.111+0000 7f3b46800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b4806e340 0x7f3b4806e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:49.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.111+0000 7f3b45e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3b48072630 0x7f3b48072a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:43174/0 (socket says 172.21.15.86:43174) 2024-04-25T14:06:49.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.111+0000 7f3b45e00640 1 -- 172.21.15.86:0/993514206 learned_addr learned my addr 172.21.15.86:0/993514206 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:49.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.111+0000 7f3b3d400640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f3b48163980 0x7f3b48163d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:44912/0 (socket says 172.21.15.86:44912) 2024-04-25T14:06:49.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.111+0000 7f3b45e00640 1 -- 172.21.15.86:0/993514206 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b4806e340 msgr2=0x7f3b4806e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:49.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.111+0000 7f3b45e00640 1 --2- 172.21.15.86:0/993514206 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b4806e340 0x7f3b4806e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.111+0000 7f3b45e00640 1 -- 172.21.15.86:0/993514206 >> v1:172.21.15.155:6789/0 conn(0x7f3b48163980 legacy=0x7f3b48163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:49.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.111+0000 7f3b45e00640 1 -- 172.21.15.86:0/993514206 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b480d23e0 con 0x7f3b48072630 2024-04-25T14:06:49.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.111+0000 7f3b45400640 1 -- 172.21.15.86:0/993514206 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3504784391 0 0) 0x7f3b30001120 con 0x7f3b48163980 2024-04-25T14:06:49.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.115+0000 7f3b45e00640 1 --2- 172.21.15.86:0/993514206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b48072630 0x7f3b48072a40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3b480d25b0 tx=0x7f3b3801c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c1b66ee259a381d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:49.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.115+0000 7f3b45400640 1 -- 172.21.15.86:0/993514206 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b380b0070 con 0x7f3b48072630 2024-04-25T14:06:49.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.115+0000 7f3b45400640 1 -- 172.21.15.86:0/993514206 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b380ab600 con 0x7f3b48072630 2024-04-25T14:06:49.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.115+0000 7f3b45400640 1 -- 172.21.15.86:0/993514206 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b380ab9d0 con 0x7f3b48072630 2024-04-25T14:06:49.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.115+0000 7f3b4cc00640 1 -- 172.21.15.86:0/993514206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b48072630 msgr2=0x7f3b48072a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:49.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.115+0000 7f3b4cc00640 1 --2- 172.21.15.86:0/993514206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b48072630 0x7f3b48072a40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3b480d25b0 tx=0x7f3b3801c420 comp rx=0 tx=0).stop 2024-04-25T14:06:49.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.115+0000 7f3b4cc00640 1 -- 172.21.15.86:0/993514206 shutdown_connections 2024-04-25T14:06:49.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.115+0000 7f3b4cc00640 1 --2- 172.21.15.86:0/993514206 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b4806e340 0x7f3b4806e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.115+0000 7f3b4cc00640 1 --2- 172.21.15.86:0/993514206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b48072630 0x7f3b48072a40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.115+0000 7f3b4cc00640 1 -- 172.21.15.86:0/993514206 >> 172.21.15.86:0/993514206 conn(0x7f3b480d3580 msgr2=0x7f3b480d3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:49.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.115+0000 7f3b4cc00640 1 -- 172.21.15.86:0/993514206 shutdown_connections 2024-04-25T14:06:49.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.115+0000 7f3b4cc00640 1 -- 172.21.15.86:0/993514206 wait complete. 2024-04-25T14:06:49.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.119+0000 7f3b4cc00640 1 Processor -- start 2024-04-25T14:06:49.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.119+0000 7f3b4cc00640 1 -- start start 2024-04-25T14:06:49.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.119+0000 7f3b4cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b4806e340 0x7f3b480b8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:49.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.119+0000 7f3b4cc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b48072630 0x7f3b480b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:49.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.119+0000 7f3b4cc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b48163980 0x7f3b480b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:49.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.119+0000 7f3b4cc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4816d530 con 0x7f3b4806e340 2024-04-25T14:06:49.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.119+0000 7f3b4cc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4806f0a0 con 0x7f3b48072630 2024-04-25T14:06:49.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.119+0000 7f3b4cc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3b4816d6b0 con 0x7f3b48163980 2024-04-25T14:06:49.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.119+0000 7f3b45e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b4806e340 0x7f3b480b8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:49.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.119+0000 7f3b45e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b4806e340 0x7f3b480b8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56416/0 (socket says 172.21.15.86:56416) 2024-04-25T14:06:49.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.119+0000 7f3b45e00640 1 -- 172.21.15.86:0/2569122059 learned_addr learned my addr 172.21.15.86:0/2569122059 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:49.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.119+0000 7f3b3d400640 1 --2- 172.21.15.86:0/2569122059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b48072630 0x7f3b480b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:49.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.119+0000 7f3b45e00640 1 -- 172.21.15.86:0/2569122059 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b48163980 msgr2=0x7f3b480b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:49.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.119+0000 7f3b45e00640 1 --2- 172.21.15.86:0/2569122059 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b48163980 0x7f3b480b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.119+0000 7f3b45e00640 1 -- 172.21.15.86:0/2569122059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b48072630 msgr2=0x7f3b480b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:49.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.119+0000 7f3b45e00640 1 --2- 172.21.15.86:0/2569122059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b48072630 0x7f3b480b6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.119+0000 7f3b45e00640 1 -- 172.21.15.86:0/2569122059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4815f3b0 con 0x7f3b4806e340 2024-04-25T14:06:49.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.119+0000 7f3b3d400640 1 --2- 172.21.15.86:0/2569122059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b48072630 0x7f3b480b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:49.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.119+0000 7f3b45e00640 1 --2- 172.21.15.86:0/2569122059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b4806e340 0x7f3b480b8e50 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f3b380ae040 tx=0x7f3b380191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:49.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.119+0000 7f3b3ea00640 1 -- 172.21.15.86:0/2569122059 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b380b0040 con 0x7f3b4806e340 2024-04-25T14:06:49.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.119+0000 7f3b3ea00640 1 -- 172.21.15.86:0/2569122059 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b380b9070 con 0x7f3b4806e340 2024-04-25T14:06:49.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.119+0000 7f3b4cc00640 1 -- 172.21.15.86:0/2569122059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4815f690 con 0x7f3b4806e340 2024-04-25T14:06:49.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.119+0000 7f3b3ea00640 1 -- 172.21.15.86:0/2569122059 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b380b46a0 con 0x7f3b4806e340 2024-04-25T14:06:49.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.119+0000 7f3b4cc00640 1 -- 172.21.15.86:0/2569122059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b4815fc50 con 0x7f3b4806e340 2024-04-25T14:06:49.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.123+0000 7f3b4cc00640 1 -- 172.21.15.86:0/2569122059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b48072170 con 0x7f3b4806e340 2024-04-25T14:06:49.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.123+0000 7f3b3ea00640 1 -- 172.21.15.86:0/2569122059 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f3b380ab4c0 con 0x7f3b4806e340 2024-04-25T14:06:49.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.123+0000 7f3b3ea00640 1 --2- 172.21.15.86:0/2569122059 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3b2406f640 0x7f3b24071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:49.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.123+0000 7f3b3ea00640 1 -- 172.21.15.86:0/2569122059 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f3b38129430 con 0x7f3b4806e340 2024-04-25T14:06:49.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.123+0000 7f3b3d400640 1 --2- 172.21.15.86:0/2569122059 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3b2406f640 0x7f3b24071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:49.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.123+0000 7f3b3d400640 1 --2- 172.21.15.86:0/2569122059 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3b2406f640 0x7f3b24071b50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3b4815da00 tx=0x7f3b30009040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:49.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.127+0000 7f3b3ea00640 1 -- 172.21.15.86:0/2569122059 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3b38016130 con 0x7f3b4806e340 2024-04-25T14:06:49.189 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:06:49.192 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:06:49.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.295+0000 7f3b4cc00640 1 -- 172.21.15.86:0/2569122059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3b480c7fd0 con 0x7f3b4806e340 2024-04-25T14:06:49.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.299+0000 7f3b3ea00640 1 -- 172.21.15.86:0/2569122059 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16945 (secure 0 0 0) 0x7f3b380fce20 con 0x7f3b4806e340 2024-04-25T14:06:49.300 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:49.300 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":74,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:06:46.540235+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:06:42.666742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T14:06:43.884074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:06:49.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.299+0000 7f3b4cc00640 1 -- 172.21.15.86:0/2569122059 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3b2406f640 msgr2=0x7f3b24071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:49.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.299+0000 7f3b4cc00640 1 --2- 172.21.15.86:0/2569122059 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3b2406f640 0x7f3b24071b50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3b4815da00 tx=0x7f3b30009040 comp rx=0 tx=0).stop 2024-04-25T14:06:49.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.299+0000 7f3b4cc00640 1 -- 172.21.15.86:0/2569122059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b4806e340 msgr2=0x7f3b480b8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:49.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.299+0000 7f3b4cc00640 1 --2- 172.21.15.86:0/2569122059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b4806e340 0x7f3b480b8e50 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f3b380ae040 tx=0x7f3b380191c0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.299+0000 7f3b4cc00640 1 -- 172.21.15.86:0/2569122059 shutdown_connections 2024-04-25T14:06:49.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.299+0000 7f3b4cc00640 1 --2- 172.21.15.86:0/2569122059 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f3b2406f640 0x7f3b24071b50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.299+0000 7f3b4cc00640 1 --2- 172.21.15.86:0/2569122059 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b48163980 0x7f3b480b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.299+0000 7f3b4cc00640 1 --2- 172.21.15.86:0/2569122059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b48072630 0x7f3b480b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.299+0000 7f3b4cc00640 1 --2- 172.21.15.86:0/2569122059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b4806e340 0x7f3b480b8e50 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.299+0000 7f3b4cc00640 1 -- 172.21.15.86:0/2569122059 >> 172.21.15.86:0/2569122059 conn(0x7f3b480d3580 msgr2=0x7f3b4806f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:49.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.299+0000 7f3b4cc00640 1 -- 172.21.15.86:0/2569122059 shutdown_connections 2024-04-25T14:06:49.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.299+0000 7f3b4cc00640 1 -- 172.21.15.86:0/2569122059 wait complete. 2024-04-25T14:06:49.323 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T14:06:49.323 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:06:49.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.467+0000 7f4391a00640 1 Processor -- start 2024-04-25T14:06:49.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.467+0000 7f4391a00640 1 -- start start 2024-04-25T14:06:49.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.471+0000 7f4391a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f438c0d08c0 0x7f438c15c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:49.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.471+0000 7f4391a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f438c0ae130 0x7f438c0ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:49.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.471+0000 7f4391a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f438c06d120 con 0x7f438c06e830 2024-04-25T14:06:49.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.471+0000 7f4391a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f438c06d400 con 0x7f438c0d08c0 2024-04-25T14:06:49.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.471+0000 7f4391a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f438c06d6e0 con 0x7f438c0ae130 2024-04-25T14:06:49.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.471+0000 7f438be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f438c0d08c0 0x7f438c15c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:49.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.471+0000 7f438be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f438c0d08c0 0x7f438c15c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56430/0 (socket says 172.21.15.86:56430) 2024-04-25T14:06:49.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.471+0000 7f438be00640 1 -- 172.21.15.86:0/2519564180 learned_addr learned my addr 172.21.15.86:0/2519564180 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:49.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.471+0000 7f4391000640 1 --2- 172.21.15.86:0/2519564180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f438c0ae130 0x7f438c0ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:49.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.471+0000 7f438be00640 1 -- 172.21.15.86:0/2519564180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f438c0ae130 msgr2=0x7f438c0ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:49.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.471+0000 7f438be00640 1 --2- 172.21.15.86:0/2519564180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f438c0ae130 0x7f438c0ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.471+0000 7f438be00640 1 -- 172.21.15.86:0/2519564180 >> v1:172.21.15.155:6789/0 conn(0x7f438c06e830 legacy=0x7f438c06ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:49.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.471+0000 7f438be00640 1 -- 172.21.15.86:0/2519564180 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f438c06d9c0 con 0x7f438c0d08c0 2024-04-25T14:06:49.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.475+0000 7f438be00640 1 --2- 172.21.15.86:0/2519564180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f438c0d08c0 0x7f438c15c990 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f437c00bb10 tx=0x7f437c01ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3d84eea4cf18b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:49.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.475+0000 7f438b400640 1 -- 172.21.15.86:0/2519564180 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437c0b4070 con 0x7f438c0d08c0 2024-04-25T14:06:49.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.475+0000 7f438b400640 1 -- 172.21.15.86:0/2519564180 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f437c0a7b00 con 0x7f438c0d08c0 2024-04-25T14:06:49.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.475+0000 7f438b400640 1 -- 172.21.15.86:0/2519564180 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437c0af440 con 0x7f438c0d08c0 2024-04-25T14:06:49.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.475+0000 7f4391a00640 1 -- 172.21.15.86:0/2519564180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f438c0d08c0 msgr2=0x7f438c15c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:49.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.475+0000 7f4391a00640 1 --2- 172.21.15.86:0/2519564180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f438c0d08c0 0x7f438c15c990 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f437c00bb10 tx=0x7f437c01ec30 comp rx=0 tx=0).stop 2024-04-25T14:06:49.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.475+0000 7f4391a00640 1 -- 172.21.15.86:0/2519564180 shutdown_connections 2024-04-25T14:06:49.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.475+0000 7f4391a00640 1 --2- 172.21.15.86:0/2519564180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f438c0ae130 0x7f438c0ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.475+0000 7f4391a00640 1 --2- 172.21.15.86:0/2519564180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f438c0d08c0 0x7f438c15c990 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.475+0000 7f4391a00640 1 -- 172.21.15.86:0/2519564180 >> 172.21.15.86:0/2519564180 conn(0x7f438c0b4850 msgr2=0x7f438c0b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:49.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.475+0000 7f4391a00640 1 -- 172.21.15.86:0/2519564180 shutdown_connections 2024-04-25T14:06:49.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.475+0000 7f4391a00640 1 -- 172.21.15.86:0/2519564180 wait complete. 2024-04-25T14:06:49.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.475+0000 7f4391a00640 1 Processor -- start 2024-04-25T14:06:49.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.475+0000 7f4391a00640 1 -- start start 2024-04-25T14:06:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.479+0000 7f4391a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f438c06e830 0x7f438c148380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.479+0000 7f4391a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f438c0ae130 0x7f438c147640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.479+0000 7f4391a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f438c0d08c0 0x7f438c14aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.479+0000 7f4391a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f438c0cdde0 con 0x7f438c0ae130 2024-04-25T14:06:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.479+0000 7f4391a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f438c0cdc60 con 0x7f438c06e830 2024-04-25T14:06:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.479+0000 7f4391a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f438c0cdf60 con 0x7f438c0d08c0 2024-04-25T14:06:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.479+0000 7f4383400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f438c0ae130 0x7f438c147640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.479+0000 7f4383400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f438c0ae130 0x7f438c147640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56446/0 (socket says 172.21.15.86:56446) 2024-04-25T14:06:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.479+0000 7f4383400640 1 -- 172.21.15.86:0/2045909448 learned_addr learned my addr 172.21.15.86:0/2045909448 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.479+0000 7f438be00640 1 --2- 172.21.15.86:0/2045909448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f438c06e830 0x7f438c148380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.479+0000 7f4391000640 1 --2- 172.21.15.86:0/2045909448 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f438c0d08c0 0x7f438c14aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.479+0000 7f4383400640 1 -- 172.21.15.86:0/2045909448 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f438c0d08c0 msgr2=0x7f438c14aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.479+0000 7f4383400640 1 --2- 172.21.15.86:0/2045909448 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f438c0d08c0 0x7f438c14aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.479+0000 7f4383400640 1 -- 172.21.15.86:0/2045909448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f438c06e830 msgr2=0x7f438c148380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.479+0000 7f4383400640 1 --2- 172.21.15.86:0/2045909448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f438c06e830 0x7f438c148380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.479+0000 7f4383400640 1 -- 172.21.15.86:0/2045909448 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f438c148cf0 con 0x7f438c0ae130 2024-04-25T14:06:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.479+0000 7f4383400640 1 --2- 172.21.15.86:0/2045909448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f438c0ae130 0x7f438c147640 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f437400c760 tx=0x7f437400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.479+0000 7f4388c00640 1 -- 172.21.15.86:0/2045909448 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4374016020 con 0x7f438c0ae130 2024-04-25T14:06:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.479+0000 7f4388c00640 1 -- 172.21.15.86:0/2045909448 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4374006d60 con 0x7f438c0ae130 2024-04-25T14:06:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.479+0000 7f4388c00640 1 -- 172.21.15.86:0/2045909448 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4374003bc0 con 0x7f438c0ae130 2024-04-25T14:06:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.479+0000 7f4391a00640 1 -- 172.21.15.86:0/2045909448 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f438c148fd0 con 0x7f438c0ae130 2024-04-25T14:06:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.479+0000 7f4391a00640 1 -- 172.21.15.86:0/2045909448 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f438c149590 con 0x7f438c0ae130 2024-04-25T14:06:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.479+0000 7f4391a00640 1 -- 172.21.15.86:0/2045909448 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f438c148a60 con 0x7f438c0ae130 2024-04-25T14:06:49.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.483+0000 7f4388c00640 1 -- 172.21.15.86:0/2045909448 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f437400a850 con 0x7f438c0ae130 2024-04-25T14:06:49.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.483+0000 7f4388c00640 1 --2- 172.21.15.86:0/2045909448 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f436806f640 0x7f4368071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:49.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.483+0000 7f4388c00640 1 -- 172.21.15.86:0/2045909448 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f437408f510 con 0x7f438c0ae130 2024-04-25T14:06:49.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.483+0000 7f438be00640 1 --2- 172.21.15.86:0/2045909448 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f436806f640 0x7f4368071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:49.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.483+0000 7f438be00640 1 --2- 172.21.15.86:0/2045909448 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f436806f640 0x7f4368071b50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f437c0b1000 tx=0x7f437c0a9070 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:49.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.487+0000 7f4388c00640 1 -- 172.21.15.86:0/2045909448 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4374020020 con 0x7f438c0ae130 2024-04-25T14:06:49.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.647+0000 7f4391a00640 1 -- 172.21.15.86:0/2045909448 --> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f438c076b80 con 0x7f436806f640 2024-04-25T14:06:49.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.655+0000 7f4388c00640 1 -- 172.21.15.86:0/2045909448 <== mgr.6198 v2:172.21.15.86:6832/1025829557 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246744 (secure 0 0 0) 0x7f438c076b80 con 0x7f436806f640 2024-04-25T14:06:49.659 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:49.661 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:06:49.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.659+0000 7f4391a00640 1 -- 172.21.15.86:0/2045909448 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f436806f640 msgr2=0x7f4368071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:49.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.659+0000 7f4391a00640 1 --2- 172.21.15.86:0/2045909448 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f436806f640 0x7f4368071b50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f437c0b1000 tx=0x7f437c0a9070 comp rx=0 tx=0).stop 2024-04-25T14:06:49.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.659+0000 7f4391a00640 1 -- 172.21.15.86:0/2045909448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f438c0ae130 msgr2=0x7f438c147640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:49.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.659+0000 7f4391a00640 1 --2- 172.21.15.86:0/2045909448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f438c0ae130 0x7f438c147640 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f437400c760 tx=0x7f437400cc20 comp rx=0 tx=0).stop 2024-04-25T14:06:49.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.659+0000 7f4391a00640 1 -- 172.21.15.86:0/2045909448 shutdown_connections 2024-04-25T14:06:49.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.659+0000 7f4391a00640 1 --2- 172.21.15.86:0/2045909448 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f436806f640 0x7f4368071b50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.659+0000 7f4391a00640 1 --2- 172.21.15.86:0/2045909448 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f438c0d08c0 0x7f438c14aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.659+0000 7f4391a00640 1 --2- 172.21.15.86:0/2045909448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f438c0ae130 0x7f438c147640 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.659+0000 7f4391a00640 1 --2- 172.21.15.86:0/2045909448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f438c06e830 0x7f438c148380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.659+0000 7f4391a00640 1 -- 172.21.15.86:0/2045909448 >> 172.21.15.86:0/2045909448 conn(0x7f438c0b4850 msgr2=0x7f438c152460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:49.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.659+0000 7f4391a00640 1 -- 172.21.15.86:0/2045909448 shutdown_connections 2024-04-25T14:06:49.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.659+0000 7f4391a00640 1 -- 172.21.15.86:0/2045909448 wait complete. 2024-04-25T14:06:49.681 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-04-25T14:06:49.438066+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253352,"kb_used_data":8040,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721368,"statfs":{"total":773094113280,"available":772834680832,"internally_reserved":0,"allocated":8232960,"data_stored":4649552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.927238"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.528048+0000","last_change":"2024-04-25T14:06:44.528048+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.528048+0000","last_undegraded":"2024-04-25T14:06:44.528048+0000","last_fullsized":"2024-04-25T14:06:44.528048+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.522967+0000","last_change":"2024-04-25T14:06:44.522967+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.522967+0000","last_undegraded":"2024-04-25T14:06:44.522967+0000","last_fullsized":"2024-04-25T14:06:44.522967+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.530589+0000","last_change":"2024-04-25T14:06:44.530589+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.530589+0000","last_undegraded":"2024-04-25T14:06:44.530589+0000","last_fullsized":"2024-04-25T14:06:44.530589+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.521588+0000","last_change":"2024-04-25T14:06:44.521588+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.521588+0000","last_undegraded":"2024-04-25T14:06:44.521588+0000","last_fullsized":"2024-04-25T14:06:44.521588+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.529696+0000","last_change":"2024-04-25T14:06:44.529696+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.529696+0000","last_undegraded":"2024-04-25T14:06:44.529696+0000","last_fullsized":"2024-04-25T14:06:44.529696+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.520096+0000","last_change":"2024-04-25T14:06:44.520096+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.520096+0000","last_undegraded":"2024-04-25T14:06:44.520096+0000","last_fullsized":"2024-04-25T14:06:44.520096+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.532154+0000","last_change":"2024-04-25T14:06:44.532154+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.532154+0000","last_undegraded":"2024-04-25T14:06:44.532154+0000","last_fullsized":"2024-04-25T14:06:44.532154+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.524922+0000","last_change":"2024-04-25T14:06:44.524922+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.524922+0000","last_undegraded":"2024-04-25T14:06:44.524922+0000","last_fullsized":"2024-04-25T14:06:44.524922+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525831+0000","last_change":"2024-04-25T14:06:44.525831+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525831+0000","last_undegraded":"2024-04-25T14:06:44.525831+0000","last_fullsized":"2024-04-25T14:06:44.525831+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.522110+0000","last_change":"2024-04-25T14:06:44.522110+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.522110+0000","last_undegraded":"2024-04-25T14:06:44.522110+0000","last_fullsized":"2024-04-25T14:06:44.522110+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.524249+0000","last_change":"2024-04-25T14:06:44.524249+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.524249+0000","last_undegraded":"2024-04-25T14:06:44.524249+0000","last_fullsized":"2024-04-25T14:06:44.524249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.520624+0000","last_change":"2024-04-25T14:06:44.520624+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.520624+0000","last_undegraded":"2024-04-25T14:06:44.520624+0000","last_fullsized":"2024-04-25T14:06:44.520624+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.524668+0000","last_change":"2024-04-25T14:06:44.524668+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.524668+0000","last_undegraded":"2024-04-25T14:06:44.524668+0000","last_fullsized":"2024-04-25T14:06:44.524668+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.519647+0000","last_change":"2024-04-25T14:06:44.519647+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.519647+0000","last_undegraded":"2024-04-25T14:06:44.519647+0000","last_fullsized":"2024-04-25T14:06:44.519647+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.521818+0000","last_change":"2024-04-25T14:06:44.521818+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.521818+0000","last_undegraded":"2024-04-25T14:06:44.521818+0000","last_fullsized":"2024-04-25T14:06:44.521818+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.524664+0000","last_change":"2024-04-25T14:06:44.524664+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.524664+0000","last_undegraded":"2024-04-25T14:06:44.524664+0000","last_fullsized":"2024-04-25T14:06:44.524664+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.526595+0000","last_change":"2024-04-25T14:06:44.526595+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.526595+0000","last_undegraded":"2024-04-25T14:06:44.526595+0000","last_fullsized":"2024-04-25T14:06:44.526595+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.535681+0000","last_change":"2024-04-25T14:06:44.535681+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.535681+0000","last_undegraded":"2024-04-25T14:06:44.535681+0000","last_fullsized":"2024-04-25T14:06:44.535681+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.519158+0000","last_change":"2024-04-25T14:06:44.519158+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.519158+0000","last_undegraded":"2024-04-25T14:06:44.519158+0000","last_fullsized":"2024-04-25T14:06:44.519158+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.524178+0000","last_change":"2024-04-25T14:06:44.524178+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.524178+0000","last_undegraded":"2024-04-25T14:06:44.524178+0000","last_fullsized":"2024-04-25T14:06:44.524178+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525944+0000","last_change":"2024-04-25T14:06:44.525944+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525944+0000","last_undegraded":"2024-04-25T14:06:44.525944+0000","last_fullsized":"2024-04-25T14:06:44.525944+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525179+0000","last_change":"2024-04-25T14:06:44.525179+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525179+0000","last_undegraded":"2024-04-25T14:06:44.525179+0000","last_fullsized":"2024-04-25T14:06:44.525179+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.523722+0000","last_change":"2024-04-25T14:06:44.523722+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.523722+0000","last_undegraded":"2024-04-25T14:06:44.523722+0000","last_fullsized":"2024-04-25T14:06:44.523722+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.521337+0000","last_change":"2024-04-25T14:06:44.521337+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.521337+0000","last_undegraded":"2024-04-25T14:06:44.521337+0000","last_fullsized":"2024-04-25T14:06:44.521337+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.520503+0000","last_change":"2024-04-25T14:06:44.520503+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.520503+0000","last_undegraded":"2024-04-25T14:06:44.520503+0000","last_fullsized":"2024-04-25T14:06:44.520503+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.520274+0000","last_change":"2024-04-25T14:06:44.520274+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.520274+0000","last_undegraded":"2024-04-25T14:06:44.520274+0000","last_fullsized":"2024-04-25T14:06:44.520274+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.526216+0000","last_change":"2024-04-25T14:06:44.526216+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.526216+0000","last_undegraded":"2024-04-25T14:06:44.526216+0000","last_fullsized":"2024-04-25T14:06:44.526216+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.523614+0000","last_change":"2024-04-25T14:06:44.523614+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.523614+0000","last_undegraded":"2024-04-25T14:06:44.523614+0000","last_fullsized":"2024-04-25T14:06:44.523614+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.521807+0000","last_change":"2024-04-25T14:06:44.521807+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.521807+0000","last_undegraded":"2024-04-25T14:06:44.521807+0000","last_fullsized":"2024-04-25T14:06:44.521807+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.527187+0000","last_change":"2024-04-25T14:06:44.527187+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.527187+0000","last_undegraded":"2024-04-25T14:06:44.527187+0000","last_fullsized":"2024-04-25T14:06:44.527187+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.528762+0000","last_change":"2024-04-25T14:06:44.528762+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.528762+0000","last_undegraded":"2024-04-25T14:06:44.528762+0000","last_fullsized":"2024-04-25T14:06:44.528762+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.524006+0000","last_change":"2024-04-25T14:06:44.524006+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.524006+0000","last_undegraded":"2024-04-25T14:06:44.524006+0000","last_fullsized":"2024-04-25T14:06:44.524006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.529058+0000","last_change":"2024-04-25T14:06:44.529058+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.529058+0000","last_undegraded":"2024-04-25T14:06:44.529058+0000","last_fullsized":"2024-04-25T14:06:44.529058+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.544431+0000","last_change":"2024-04-25T14:06:44.538430+0000","last_active":"2024-04-25T14:06:44.544431+0000","last_peered":"2024-04-25T14:06:44.544431+0000","last_clean":"2024-04-25T14:06:44.544431+0000","last_became_active":"2024-04-25T14:06:44.538081+0000","last_became_peered":"2024-04-25T14:06:44.538081+0000","last_unstale":"2024-04-25T14:06:44.544431+0000","last_undegraded":"2024-04-25T14:06:44.544431+0000","last_fullsized":"2024-04-25T14:06:44.544431+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:19.980060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.521384+0000","last_change":"2024-04-25T14:06:44.521384+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.521384+0000","last_undegraded":"2024-04-25T14:06:44.521384+0000","last_fullsized":"2024-04-25T14:06:44.521384+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.547856+0000","last_change":"2024-04-25T14:06:44.544725+0000","last_active":"2024-04-25T14:06:44.547856+0000","last_peered":"2024-04-25T14:06:44.547856+0000","last_clean":"2024-04-25T14:06:44.547856+0000","last_became_active":"2024-04-25T14:06:44.544154+0000","last_became_peered":"2024-04-25T14:06:44.544154+0000","last_unstale":"2024-04-25T14:06:44.547856+0000","last_undegraded":"2024-04-25T14:06:44.547856+0000","last_fullsized":"2024-04-25T14:06:44.547856+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:27.974323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525629+0000","last_change":"2024-04-25T14:06:44.525629+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525629+0000","last_undegraded":"2024-04-25T14:06:44.525629+0000","last_fullsized":"2024-04-25T14:06:44.525629+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.544093+0000","last_change":"2024-04-25T14:06:44.537708+0000","last_active":"2024-04-25T14:06:44.544093+0000","last_peered":"2024-04-25T14:06:44.544093+0000","last_clean":"2024-04-25T14:06:44.544093+0000","last_became_active":"2024-04-25T14:06:44.537349+0000","last_became_peered":"2024-04-25T14:06:44.537349+0000","last_unstale":"2024-04-25T14:06:44.544093+0000","last_undegraded":"2024-04-25T14:06:44.544093+0000","last_fullsized":"2024-04-25T14:06:44.544093+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:14.448012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525353+0000","last_change":"2024-04-25T14:06:44.525353+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525353+0000","last_undegraded":"2024-04-25T14:06:44.525353+0000","last_fullsized":"2024-04-25T14:06:44.525353+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.550329+0000","last_change":"2024-04-25T14:06:44.546446+0000","last_active":"2024-04-25T14:06:44.550329+0000","last_peered":"2024-04-25T14:06:44.550329+0000","last_clean":"2024-04-25T14:06:44.550329+0000","last_became_active":"2024-04-25T14:06:44.546076+0000","last_became_peered":"2024-04-25T14:06:44.546076+0000","last_unstale":"2024-04-25T14:06:44.550329+0000","last_undegraded":"2024-04-25T14:06:44.550329+0000","last_fullsized":"2024-04-25T14:06:44.550329+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:03.415461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.522329+0000","last_change":"2024-04-25T14:06:44.522329+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.522329+0000","last_undegraded":"2024-04-25T14:06:44.522329+0000","last_fullsized":"2024-04-25T14:06:44.522329+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.544176+0000","last_change":"2024-04-25T14:06:44.538629+0000","last_active":"2024-04-25T14:06:44.544176+0000","last_peered":"2024-04-25T14:06:44.544176+0000","last_clean":"2024-04-25T14:06:44.544176+0000","last_became_active":"2024-04-25T14:06:44.538329+0000","last_became_peered":"2024-04-25T14:06:44.538329+0000","last_unstale":"2024-04-25T14:06:44.544176+0000","last_undegraded":"2024-04-25T14:06:44.544176+0000","last_fullsized":"2024-04-25T14:06:44.544176+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:02.112409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.523772+0000","last_change":"2024-04-25T14:06:44.523772+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.523772+0000","last_undegraded":"2024-04-25T14:06:44.523772+0000","last_fullsized":"2024-04-25T14:06:44.523772+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.534286+0000","last_change":"2024-04-25T14:06:44.530649+0000","last_active":"2024-04-25T14:06:44.534286+0000","last_peered":"2024-04-25T14:06:44.534286+0000","last_clean":"2024-04-25T14:06:44.534286+0000","last_became_active":"2024-04-25T14:06:44.530025+0000","last_became_peered":"2024-04-25T14:06:44.530025+0000","last_unstale":"2024-04-25T14:06:44.534286+0000","last_undegraded":"2024-04-25T14:06:44.534286+0000","last_fullsized":"2024-04-25T14:06:44.534286+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:34.160672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525151+0000","last_change":"2024-04-25T14:06:44.525151+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525151+0000","last_undegraded":"2024-04-25T14:06:44.525151+0000","last_fullsized":"2024-04-25T14:06:44.525151+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.542594+0000","last_change":"2024-04-25T14:06:44.537817+0000","last_active":"2024-04-25T14:06:44.542594+0000","last_peered":"2024-04-25T14:06:44.542594+0000","last_clean":"2024-04-25T14:06:44.542594+0000","last_became_active":"2024-04-25T14:06:44.537198+0000","last_became_peered":"2024-04-25T14:06:44.537198+0000","last_unstale":"2024-04-25T14:06:44.542594+0000","last_undegraded":"2024-04-25T14:06:44.542594+0000","last_fullsized":"2024-04-25T14:06:44.542594+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:42.088259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.536745+0000","last_change":"2024-04-25T14:06:44.536745+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.536745+0000","last_undegraded":"2024-04-25T14:06:44.536745+0000","last_fullsized":"2024-04-25T14:06:44.536745+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.544663+0000","last_change":"2024-04-25T14:06:44.539464+0000","last_active":"2024-04-25T14:06:44.544663+0000","last_peered":"2024-04-25T14:06:44.544663+0000","last_clean":"2024-04-25T14:06:44.544663+0000","last_became_active":"2024-04-25T14:06:44.539110+0000","last_became_peered":"2024-04-25T14:06:44.539110+0000","last_unstale":"2024-04-25T14:06:44.544663+0000","last_undegraded":"2024-04-25T14:06:44.544663+0000","last_fullsized":"2024-04-25T14:06:44.544663+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:41.171166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.520415+0000","last_change":"2024-04-25T14:06:44.520415+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.520415+0000","last_undegraded":"2024-04-25T14:06:44.520415+0000","last_fullsized":"2024-04-25T14:06:44.520415+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.552725+0000","last_change":"2024-04-25T14:06:44.548815+0000","last_active":"2024-04-25T14:06:44.552725+0000","last_peered":"2024-04-25T14:06:44.552725+0000","last_clean":"2024-04-25T14:06:44.552725+0000","last_became_active":"2024-04-25T14:06:44.548469+0000","last_became_peered":"2024-04-25T14:06:44.548469+0000","last_unstale":"2024-04-25T14:06:44.552725+0000","last_undegraded":"2024-04-25T14:06:44.552725+0000","last_fullsized":"2024-04-25T14:06:44.552725+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:24.243932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.519850+0000","last_change":"2024-04-25T14:06:44.519850+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.519850+0000","last_undegraded":"2024-04-25T14:06:44.519850+0000","last_fullsized":"2024-04-25T14:06:44.519850+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.543141+0000","last_change":"2024-04-25T14:06:44.540795+0000","last_active":"2024-04-25T14:06:44.543141+0000","last_peered":"2024-04-25T14:06:44.543141+0000","last_clean":"2024-04-25T14:06:44.543141+0000","last_became_active":"2024-04-25T14:06:44.540467+0000","last_became_peered":"2024-04-25T14:06:44.540467+0000","last_unstale":"2024-04-25T14:06:44.543141+0000","last_undegraded":"2024-04-25T14:06:44.543141+0000","last_fullsized":"2024-04-25T14:06:44.543141+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:14.448012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.519140+0000","last_change":"2024-04-25T14:06:44.519140+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.519140+0000","last_undegraded":"2024-04-25T14:06:44.519140+0000","last_fullsized":"2024-04-25T14:06:44.519140+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.546703+0000","last_change":"2024-04-25T14:06:44.544689+0000","last_active":"2024-04-25T14:06:44.546703+0000","last_peered":"2024-04-25T14:06:44.546703+0000","last_clean":"2024-04-25T14:06:44.546703+0000","last_became_active":"2024-04-25T14:06:44.544095+0000","last_became_peered":"2024-04-25T14:06:44.544095+0000","last_unstale":"2024-04-25T14:06:44.546703+0000","last_undegraded":"2024-04-25T14:06:44.546703+0000","last_fullsized":"2024-04-25T14:06:44.546703+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:11.780439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.529532+0000","last_change":"2024-04-25T14:06:44.529532+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.529532+0000","last_undegraded":"2024-04-25T14:06:44.529532+0000","last_fullsized":"2024-04-25T14:06:44.529532+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.550353+0000","last_change":"2024-04-25T14:06:44.545874+0000","last_active":"2024-04-25T14:06:44.550353+0000","last_peered":"2024-04-25T14:06:44.550353+0000","last_clean":"2024-04-25T14:06:44.550353+0000","last_became_active":"2024-04-25T14:06:44.545363+0000","last_became_peered":"2024-04-25T14:06:44.545363+0000","last_unstale":"2024-04-25T14:06:44.550353+0000","last_undegraded":"2024-04-25T14:06:44.550353+0000","last_fullsized":"2024-04-25T14:06:44.550353+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:16.858125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.519818+0000","last_change":"2024-04-25T14:06:44.519818+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.519818+0000","last_undegraded":"2024-04-25T14:06:44.519818+0000","last_fullsized":"2024-04-25T14:06:44.519818+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.538447+0000","last_change":"2024-04-25T14:06:44.533506+0000","last_active":"2024-04-25T14:06:44.538447+0000","last_peered":"2024-04-25T14:06:44.538447+0000","last_clean":"2024-04-25T14:06:44.538447+0000","last_became_active":"2024-04-25T14:06:44.533189+0000","last_became_peered":"2024-04-25T14:06:44.533189+0000","last_unstale":"2024-04-25T14:06:44.538447+0000","last_undegraded":"2024-04-25T14:06:44.538447+0000","last_fullsized":"2024-04-25T14:06:44.538447+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:27.614025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.534844+0000","last_change":"2024-04-25T14:06:44.534844+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.534844+0000","last_undegraded":"2024-04-25T14:06:44.534844+0000","last_fullsized":"2024-04-25T14:06:44.534844+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.547834+0000","last_change":"2024-04-25T14:06:44.544582+0000","last_active":"2024-04-25T14:06:44.547834+0000","last_peered":"2024-04-25T14:06:44.547834+0000","last_clean":"2024-04-25T14:06:44.547834+0000","last_became_active":"2024-04-25T14:06:44.544049+0000","last_became_peered":"2024-04-25T14:06:44.544049+0000","last_unstale":"2024-04-25T14:06:44.547834+0000","last_undegraded":"2024-04-25T14:06:44.547834+0000","last_fullsized":"2024-04-25T14:06:44.547834+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:22.780792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.529499+0000","last_change":"2024-04-25T14:06:44.529499+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.529499+0000","last_undegraded":"2024-04-25T14:06:44.529499+0000","last_fullsized":"2024-04-25T14:06:44.529499+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.542053+0000","last_change":"2024-04-25T14:06:44.534032+0000","last_active":"2024-04-25T14:06:44.542053+0000","last_peered":"2024-04-25T14:06:44.542053+0000","last_clean":"2024-04-25T14:06:44.542053+0000","last_became_active":"2024-04-25T14:06:44.533673+0000","last_became_peered":"2024-04-25T14:06:44.533673+0000","last_unstale":"2024-04-25T14:06:44.542053+0000","last_undegraded":"2024-04-25T14:06:44.542053+0000","last_fullsized":"2024-04-25T14:06:44.542053+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:29.427107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.531944+0000","last_change":"2024-04-25T14:06:44.531944+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.531944+0000","last_undegraded":"2024-04-25T14:06:44.531944+0000","last_fullsized":"2024-04-25T14:06:44.531944+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.542699+0000","last_change":"2024-04-25T14:06:44.538705+0000","last_active":"2024-04-25T14:06:44.542699+0000","last_peered":"2024-04-25T14:06:44.542699+0000","last_clean":"2024-04-25T14:06:44.542699+0000","last_became_active":"2024-04-25T14:06:44.538404+0000","last_became_peered":"2024-04-25T14:06:44.538404+0000","last_unstale":"2024-04-25T14:06:44.542699+0000","last_undegraded":"2024-04-25T14:06:44.542699+0000","last_fullsized":"2024-04-25T14:06:44.542699+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:16.314477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.518861+0000","last_change":"2024-04-25T14:06:44.518861+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.518861+0000","last_undegraded":"2024-04-25T14:06:44.518861+0000","last_fullsized":"2024-04-25T14:06:44.518861+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.550286+0000","last_change":"2024-04-25T14:06:44.547295+0000","last_active":"2024-04-25T14:06:44.550286+0000","last_peered":"2024-04-25T14:06:44.550286+0000","last_clean":"2024-04-25T14:06:44.550286+0000","last_became_active":"2024-04-25T14:06:44.546930+0000","last_became_peered":"2024-04-25T14:06:44.546930+0000","last_unstale":"2024-04-25T14:06:44.550286+0000","last_undegraded":"2024-04-25T14:06:44.550286+0000","last_fullsized":"2024-04-25T14:06:44.550286+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:46.781154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.523760+0000","last_change":"2024-04-25T14:06:44.523760+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.523760+0000","last_undegraded":"2024-04-25T14:06:44.523760+0000","last_fullsized":"2024-04-25T14:06:44.523760+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.546025+0000","last_change":"2024-04-25T14:06:44.543006+0000","last_active":"2024-04-25T14:06:44.546025+0000","last_peered":"2024-04-25T14:06:44.546025+0000","last_clean":"2024-04-25T14:06:44.546025+0000","last_became_active":"2024-04-25T14:06:44.542678+0000","last_became_peered":"2024-04-25T14:06:44.542678+0000","last_unstale":"2024-04-25T14:06:44.546025+0000","last_undegraded":"2024-04-25T14:06:44.546025+0000","last_fullsized":"2024-04-25T14:06:44.546025+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:16.282476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525734+0000","last_change":"2024-04-25T14:06:44.525734+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525734+0000","last_undegraded":"2024-04-25T14:06:44.525734+0000","last_fullsized":"2024-04-25T14:06:44.525734+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.544904+0000","last_change":"2024-04-25T14:06:44.541885+0000","last_active":"2024-04-25T14:06:44.544904+0000","last_peered":"2024-04-25T14:06:44.544904+0000","last_clean":"2024-04-25T14:06:44.544904+0000","last_became_active":"2024-04-25T14:06:44.541501+0000","last_became_peered":"2024-04-25T14:06:44.541501+0000","last_unstale":"2024-04-25T14:06:44.544904+0000","last_undegraded":"2024-04-25T14:06:44.544904+0000","last_fullsized":"2024-04-25T14:06:44.544904+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:44.424458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.519094+0000","last_change":"2024-04-25T14:06:44.519094+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.519094+0000","last_undegraded":"2024-04-25T14:06:44.519094+0000","last_fullsized":"2024-04-25T14:06:44.519094+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.546591+0000","last_change":"2024-04-25T14:06:44.544634+0000","last_active":"2024-04-25T14:06:44.546591+0000","last_peered":"2024-04-25T14:06:44.546591+0000","last_clean":"2024-04-25T14:06:44.546591+0000","last_became_active":"2024-04-25T14:06:44.544365+0000","last_became_peered":"2024-04-25T14:06:44.544365+0000","last_unstale":"2024-04-25T14:06:44.546591+0000","last_undegraded":"2024-04-25T14:06:44.546591+0000","last_fullsized":"2024-04-25T14:06:44.546591+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:54:29.024591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.521205+0000","last_change":"2024-04-25T14:06:44.521205+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.521205+0000","last_undegraded":"2024-04-25T14:06:44.521205+0000","last_fullsized":"2024-04-25T14:06:44.521205+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.543827+0000","last_change":"2024-04-25T14:06:44.541617+0000","last_active":"2024-04-25T14:06:44.543827+0000","last_peered":"2024-04-25T14:06:44.543827+0000","last_clean":"2024-04-25T14:06:44.543827+0000","last_became_active":"2024-04-25T14:06:44.540854+0000","last_became_peered":"2024-04-25T14:06:44.540854+0000","last_unstale":"2024-04-25T14:06:44.543827+0000","last_undegraded":"2024-04-25T14:06:44.543827+0000","last_fullsized":"2024-04-25T14:06:44.543827+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:14.236144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.520875+0000","last_change":"2024-04-25T14:06:44.520875+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.520875+0000","last_undegraded":"2024-04-25T14:06:44.520875+0000","last_fullsized":"2024-04-25T14:06:44.520875+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.543972+0000","last_change":"2024-04-25T14:06:44.537769+0000","last_active":"2024-04-25T14:06:44.543972+0000","last_peered":"2024-04-25T14:06:44.543972+0000","last_clean":"2024-04-25T14:06:44.543972+0000","last_became_active":"2024-04-25T14:06:44.537407+0000","last_became_peered":"2024-04-25T14:06:44.537407+0000","last_unstale":"2024-04-25T14:06:44.543972+0000","last_undegraded":"2024-04-25T14:06:44.543972+0000","last_fullsized":"2024-04-25T14:06:44.543972+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:58.086378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.529722+0000","last_change":"2024-04-25T14:06:44.529722+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.529722+0000","last_undegraded":"2024-04-25T14:06:44.529722+0000","last_fullsized":"2024-04-25T14:06:44.529722+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.542491+0000","last_change":"2024-04-25T14:06:44.538743+0000","last_active":"2024-04-25T14:06:44.542491+0000","last_peered":"2024-04-25T14:06:44.542491+0000","last_clean":"2024-04-25T14:06:44.542491+0000","last_became_active":"2024-04-25T14:06:44.538365+0000","last_became_peered":"2024-04-25T14:06:44.538365+0000","last_unstale":"2024-04-25T14:06:44.542491+0000","last_undegraded":"2024-04-25T14:06:44.542491+0000","last_fullsized":"2024-04-25T14:06:44.542491+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:47.507460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.524176+0000","last_change":"2024-04-25T14:06:44.524176+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.524176+0000","last_undegraded":"2024-04-25T14:06:44.524176+0000","last_fullsized":"2024-04-25T14:06:44.524176+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.547650+0000","last_change":"2024-04-25T14:06:44.541609+0000","last_active":"2024-04-25T14:06:44.547650+0000","last_peered":"2024-04-25T14:06:44.547650+0000","last_clean":"2024-04-25T14:06:44.547650+0000","last_became_active":"2024-04-25T14:06:44.541009+0000","last_became_peered":"2024-04-25T14:06:44.541009+0000","last_unstale":"2024-04-25T14:06:44.547650+0000","last_undegraded":"2024-04-25T14:06:44.547650+0000","last_fullsized":"2024-04-25T14:06:44.547650+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:58.970595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.529423+0000","last_change":"2024-04-25T14:06:44.529423+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.529423+0000","last_undegraded":"2024-04-25T14:06:44.529423+0000","last_fullsized":"2024-04-25T14:06:44.529423+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.539705+0000","last_change":"2024-04-25T14:06:44.535705+0000","last_active":"2024-04-25T14:06:44.539705+0000","last_peered":"2024-04-25T14:06:44.539705+0000","last_clean":"2024-04-25T14:06:44.539705+0000","last_became_active":"2024-04-25T14:06:44.535457+0000","last_became_peered":"2024-04-25T14:06:44.535457+0000","last_unstale":"2024-04-25T14:06:44.539705+0000","last_undegraded":"2024-04-25T14:06:44.539705+0000","last_fullsized":"2024-04-25T14:06:44.539705+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:06.121310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.523979+0000","last_change":"2024-04-25T14:06:44.523979+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.523979+0000","last_undegraded":"2024-04-25T14:06:44.523979+0000","last_fullsized":"2024-04-25T14:06:44.523979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.544953+0000","last_change":"2024-04-25T14:06:44.542180+0000","last_active":"2024-04-25T14:06:44.544953+0000","last_peered":"2024-04-25T14:06:44.544953+0000","last_clean":"2024-04-25T14:06:44.544953+0000","last_became_active":"2024-04-25T14:06:44.540654+0000","last_became_peered":"2024-04-25T14:06:44.540654+0000","last_unstale":"2024-04-25T14:06:44.544953+0000","last_undegraded":"2024-04-25T14:06:44.544953+0000","last_fullsized":"2024-04-25T14:06:44.544953+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:58.970595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.520299+0000","last_change":"2024-04-25T14:06:44.520299+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.520299+0000","last_undegraded":"2024-04-25T14:06:44.520299+0000","last_fullsized":"2024-04-25T14:06:44.520299+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.547611+0000","last_change":"2024-04-25T14:06:44.543878+0000","last_active":"2024-04-25T14:06:44.547611+0000","last_peered":"2024-04-25T14:06:44.547611+0000","last_clean":"2024-04-25T14:06:44.547611+0000","last_became_active":"2024-04-25T14:06:44.543347+0000","last_became_peered":"2024-04-25T14:06:44.543347+0000","last_unstale":"2024-04-25T14:06:44.547611+0000","last_undegraded":"2024-04-25T14:06:44.547611+0000","last_fullsized":"2024-04-25T14:06:44.547611+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:56.180081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.532473+0000","last_change":"2024-04-25T14:06:44.532473+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.532473+0000","last_undegraded":"2024-04-25T14:06:44.532473+0000","last_fullsized":"2024-04-25T14:06:44.532473+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.550348+0000","last_change":"2024-04-25T14:06:44.546730+0000","last_active":"2024-04-25T14:06:44.550348+0000","last_peered":"2024-04-25T14:06:44.550348+0000","last_clean":"2024-04-25T14:06:44.550348+0000","last_became_active":"2024-04-25T14:06:44.546050+0000","last_became_peered":"2024-04-25T14:06:44.546050+0000","last_unstale":"2024-04-25T14:06:44.550348+0000","last_undegraded":"2024-04-25T14:06:44.550348+0000","last_fullsized":"2024-04-25T14:06:44.550348+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:58:21.045855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.531523+0000","last_change":"2024-04-25T14:06:44.531523+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.531523+0000","last_undegraded":"2024-04-25T14:06:44.531523+0000","last_fullsized":"2024-04-25T14:06:44.531523+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.540642+0000","last_change":"2024-04-25T14:06:44.534134+0000","last_active":"2024-04-25T14:06:44.540642+0000","last_peered":"2024-04-25T14:06:44.540642+0000","last_clean":"2024-04-25T14:06:44.540642+0000","last_became_active":"2024-04-25T14:06:44.533705+0000","last_became_peered":"2024-04-25T14:06:44.533705+0000","last_unstale":"2024-04-25T14:06:44.540642+0000","last_undegraded":"2024-04-25T14:06:44.540642+0000","last_fullsized":"2024-04-25T14:06:44.540642+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:43.875733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.535223+0000","last_change":"2024-04-25T14:06:44.535223+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.535223+0000","last_undegraded":"2024-04-25T14:06:44.535223+0000","last_fullsized":"2024-04-25T14:06:44.535223+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.545851+0000","last_change":"2024-04-25T14:06:44.538191+0000","last_active":"2024-04-25T14:06:44.545851+0000","last_peered":"2024-04-25T14:06:44.545851+0000","last_clean":"2024-04-25T14:06:44.545851+0000","last_became_active":"2024-04-25T14:06:44.537838+0000","last_became_peered":"2024-04-25T14:06:44.537838+0000","last_unstale":"2024-04-25T14:06:44.545851+0000","last_undegraded":"2024-04-25T14:06:44.545851+0000","last_fullsized":"2024-04-25T14:06:44.545851+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:43.249499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.532357+0000","last_change":"2024-04-25T14:06:44.532357+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.532357+0000","last_undegraded":"2024-04-25T14:06:44.532357+0000","last_fullsized":"2024-04-25T14:06:44.532357+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.543260+0000","last_change":"2024-04-25T14:06:44.539538+0000","last_active":"2024-04-25T14:06:44.543260+0000","last_peered":"2024-04-25T14:06:44.543260+0000","last_clean":"2024-04-25T14:06:44.543260+0000","last_became_active":"2024-04-25T14:06:44.538128+0000","last_became_peered":"2024-04-25T14:06:44.538128+0000","last_unstale":"2024-04-25T14:06:44.543260+0000","last_undegraded":"2024-04-25T14:06:44.543260+0000","last_fullsized":"2024-04-25T14:06:44.543260+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:56.433866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525676+0000","last_change":"2024-04-25T14:06:44.525676+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525676+0000","last_undegraded":"2024-04-25T14:06:44.525676+0000","last_fullsized":"2024-04-25T14:06:44.525676+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.542537+0000","last_change":"2024-04-25T14:06:44.539206+0000","last_active":"2024-04-25T14:06:44.542537+0000","last_peered":"2024-04-25T14:06:44.542537+0000","last_clean":"2024-04-25T14:06:44.542537+0000","last_became_active":"2024-04-25T14:06:44.538913+0000","last_became_peered":"2024-04-25T14:06:44.538913+0000","last_unstale":"2024-04-25T14:06:44.542537+0000","last_undegraded":"2024-04-25T14:06:44.542537+0000","last_fullsized":"2024-04-25T14:06:44.542537+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:09.304323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"59'86","reported_seq":255,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.525284+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:06:44.525284+0000","last_peered":"2024-04-25T14:06:44.525284+0000","last_clean":"2024-04-25T14:06:44.525284+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:06:44.525284+0000","last_undegraded":"2024-04-25T14:06:44.525284+0000","last_fullsized":"2024-04-25T14:06:44.525284+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1474560,"data_stored":1023970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":433602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":880640,"data_stored":433602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":433602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":433602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":433602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":433602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1474560,"data_stored":1023970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:06:49.685 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:06:49.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.867+0000 7f809f800640 1 Processor -- start 2024-04-25T14:06:49.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.867+0000 7f809f800640 1 -- start start 2024-04-25T14:06:49.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.871+0000 7f809f800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f809815ec70 0x7f80980d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:49.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.871+0000 7f809f800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80980d1490 0x7f809806d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:49.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.871+0000 7f809f800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80980cf8f0 con 0x7f809815e860 2024-04-25T14:06:49.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.871+0000 7f809f800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f809806cfa0 con 0x7f809815ec70 2024-04-25T14:06:49.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.871+0000 7f809f800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f809806c670 con 0x7f80980d1490 2024-04-25T14:06:49.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.871+0000 7f808fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f809815ec70 0x7f80980d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:49.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.871+0000 7f808fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f809815ec70 0x7f80980d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56454/0 (socket says 172.21.15.86:56454) 2024-04-25T14:06:49.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.871+0000 7f808fe00640 1 -- 172.21.15.86:0/1340565489 learned_addr learned my addr 172.21.15.86:0/1340565489 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:49.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.871+0000 7f809d800640 1 --2- 172.21.15.86:0/1340565489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80980d1490 0x7f809806d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:49.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.871+0000 7f808fe00640 1 -- 172.21.15.86:0/1340565489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80980d1490 msgr2=0x7f809806d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:49.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.871+0000 7f808fe00640 1 --2- 172.21.15.86:0/1340565489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80980d1490 0x7f809806d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.871+0000 7f808fe00640 1 -- 172.21.15.86:0/1340565489 >> v1:172.21.15.155:6789/0 conn(0x7f809815e860 legacy=0x7f80980ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:49.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.871+0000 7f808fe00640 1 -- 172.21.15.86:0/1340565489 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f809806c950 con 0x7f809815ec70 2024-04-25T14:06:49.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.871+0000 7f808fe00640 1 --2- 172.21.15.86:0/1340565489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f809815ec70 0x7f80980d0f50 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8080009870 tx=0x7f808001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8f69d5c2a2ff6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:49.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.875+0000 7f8097e00640 1 -- 172.21.15.86:0/1340565489 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80800ab430 con 0x7f809815ec70 2024-04-25T14:06:49.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.875+0000 7f8097e00640 1 -- 172.21.15.86:0/1340565489 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80800b0070 con 0x7f809815ec70 2024-04-25T14:06:49.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.875+0000 7f8097e00640 1 -- 172.21.15.86:0/1340565489 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80800ab7f0 con 0x7f809815ec70 2024-04-25T14:06:49.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.875+0000 7f809f800640 1 -- 172.21.15.86:0/1340565489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f809815ec70 msgr2=0x7f80980d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:49.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.875+0000 7f809f800640 1 --2- 172.21.15.86:0/1340565489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f809815ec70 0x7f80980d0f50 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8080009870 tx=0x7f808001c4a0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.875+0000 7f809f800640 1 -- 172.21.15.86:0/1340565489 shutdown_connections 2024-04-25T14:06:49.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.875+0000 7f809f800640 1 --2- 172.21.15.86:0/1340565489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80980d1490 0x7f809806d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.875+0000 7f809f800640 1 --2- 172.21.15.86:0/1340565489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f809815ec70 0x7f80980d0f50 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.875+0000 7f809f800640 1 -- 172.21.15.86:0/1340565489 >> 172.21.15.86:0/1340565489 conn(0x7f80980d2230 msgr2=0x7f80980d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:49.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.875+0000 7f809f800640 1 -- 172.21.15.86:0/1340565489 shutdown_connections 2024-04-25T14:06:49.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.875+0000 7f809f800640 1 -- 172.21.15.86:0/1340565489 wait complete. 2024-04-25T14:06:49.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.875+0000 7f809f800640 1 Processor -- start 2024-04-25T14:06:49.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.875+0000 7f809f800640 1 -- start start 2024-04-25T14:06:49.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.879+0000 7f809f800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80980d1490 0x7f8098151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:49.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.879+0000 7f809f800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f809815e860 0x7f80980b59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:49.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.879+0000 7f809f800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f809815ec70 0x7f80980b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:49.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.879+0000 7f809f800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f809816a720 con 0x7f809815e860 2024-04-25T14:06:49.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.879+0000 7f809f800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f80980d18a0 con 0x7f80980d1490 2024-04-25T14:06:49.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.879+0000 7f809f800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f809816a8a0 con 0x7f809815ec70 2024-04-25T14:06:49.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.879+0000 7f808fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f809815e860 0x7f80980b59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:49.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.879+0000 7f808fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f809815e860 0x7f80980b59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56468/0 (socket says 172.21.15.86:56468) 2024-04-25T14:06:49.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.879+0000 7f808fe00640 1 -- 172.21.15.86:0/2088580987 learned_addr learned my addr 172.21.15.86:0/2088580987 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:49.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.879+0000 7f809ce00640 1 --2- 172.21.15.86:0/2088580987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80980d1490 0x7f8098151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:49.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.879+0000 7f809d800640 1 --2- 172.21.15.86:0/2088580987 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f809815ec70 0x7f80980b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:49.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.879+0000 7f808fe00640 1 -- 172.21.15.86:0/2088580987 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f809815ec70 msgr2=0x7f80980b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:49.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.879+0000 7f808fe00640 1 --2- 172.21.15.86:0/2088580987 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f809815ec70 0x7f80980b5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.879+0000 7f808fe00640 1 -- 172.21.15.86:0/2088580987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80980d1490 msgr2=0x7f8098151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:49.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.879+0000 7f808fe00640 1 --2- 172.21.15.86:0/2088580987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80980d1490 0x7f8098151d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.879+0000 7f808fe00640 1 -- 172.21.15.86:0/2088580987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8098164210 con 0x7f809815e860 2024-04-25T14:06:49.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.879+0000 7f809d800640 1 --2- 172.21.15.86:0/2088580987 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f809815ec70 0x7f80980b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:49.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.879+0000 7f808fe00640 1 --2- 172.21.15.86:0/2088580987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f809815e860 0x7f80980b59f0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f80800ad040 tx=0x7f809806e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:49.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.879+0000 7f8095600640 1 -- 172.21.15.86:0/2088580987 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80800be410 con 0x7f809815e860 2024-04-25T14:06:49.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.879+0000 7f8095600640 1 -- 172.21.15.86:0/2088580987 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80800be5d0 con 0x7f809815e860 2024-04-25T14:06:49.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.879+0000 7f809f800640 1 -- 172.21.15.86:0/2088580987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80981644f0 con 0x7f809815e860 2024-04-25T14:06:49.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.879+0000 7f8095600640 1 -- 172.21.15.86:0/2088580987 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80800be9c0 con 0x7f809815e860 2024-04-25T14:06:49.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.879+0000 7f809f800640 1 -- 172.21.15.86:0/2088580987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8098164ab0 con 0x7f809815e860 2024-04-25T14:06:49.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.879+0000 7f809f800640 1 -- 172.21.15.86:0/2088580987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8098074cf0 con 0x7f809815e860 2024-04-25T14:06:49.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.883+0000 7f8095600640 1 -- 172.21.15.86:0/2088580987 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f80800bc020 con 0x7f809815e860 2024-04-25T14:06:49.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.883+0000 7f8095600640 1 --2- 172.21.15.86:0/2088580987 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f807406f640 0x7f8074071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:49.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.883+0000 7f8095600640 1 -- 172.21.15.86:0/2088580987 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f8080129b10 con 0x7f809815e860 2024-04-25T14:06:49.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.883+0000 7f809ce00640 1 --2- 172.21.15.86:0/2088580987 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f807406f640 0x7f8074071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:49.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.883+0000 7f809ce00640 1 --2- 172.21.15.86:0/2088580987 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f807406f640 0x7f8074071b50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8088001770 tx=0x7f8088009040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:49.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:49.887+0000 7f8095600640 1 -- 172.21.15.86:0/2088580987 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8080016150 con 0x7f809815e860 2024-04-25T14:06:50.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.039+0000 7f809f800640 1 -- 172.21.15.86:0/2088580987 --> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8098076bf0 con 0x7f807406f640 2024-04-25T14:06:50.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.047+0000 7f8095600640 1 -- 172.21.15.86:0/2088580987 <== mgr.6198 v2:172.21.15.86:6832/1025829557 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246744 (secure 0 0 0) 0x7f8098076bf0 con 0x7f807406f640 2024-04-25T14:06:50.051 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:50.054 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:06:50.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.051+0000 7f809f800640 1 -- 172.21.15.86:0/2088580987 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f807406f640 msgr2=0x7f8074071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.051+0000 7f809f800640 1 --2- 172.21.15.86:0/2088580987 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f807406f640 0x7f8074071b50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8088001770 tx=0x7f8088009040 comp rx=0 tx=0).stop 2024-04-25T14:06:50.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.051+0000 7f809f800640 1 -- 172.21.15.86:0/2088580987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f809815e860 msgr2=0x7f80980b59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.051+0000 7f809f800640 1 --2- 172.21.15.86:0/2088580987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f809815e860 0x7f80980b59f0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f80800ad040 tx=0x7f809806e180 comp rx=0 tx=0).stop 2024-04-25T14:06:50.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.051+0000 7f809f800640 1 -- 172.21.15.86:0/2088580987 shutdown_connections 2024-04-25T14:06:50.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.051+0000 7f809f800640 1 --2- 172.21.15.86:0/2088580987 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f807406f640 0x7f8074071b50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.051+0000 7f809f800640 1 --2- 172.21.15.86:0/2088580987 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f809815ec70 0x7f80980b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.051+0000 7f809f800640 1 --2- 172.21.15.86:0/2088580987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f809815e860 0x7f80980b59f0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.051+0000 7f809f800640 1 --2- 172.21.15.86:0/2088580987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80980d1490 0x7f8098151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.051+0000 7f809f800640 1 -- 172.21.15.86:0/2088580987 >> 172.21.15.86:0/2088580987 conn(0x7f80980d2230 msgr2=0x7f80980b6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:50.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.051+0000 7f809f800640 1 -- 172.21.15.86:0/2088580987 shutdown_connections 2024-04-25T14:06:50.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.051+0000 7f809f800640 1 -- 172.21.15.86:0/2088580987 wait complete. 2024-04-25T14:06:50.075 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-04-25T14:06:49.438066+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253352,"kb_used_data":8040,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721368,"statfs":{"total":773094113280,"available":772834680832,"internally_reserved":0,"allocated":8232960,"data_stored":4649552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.927238"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.528048+0000","last_change":"2024-04-25T14:06:44.528048+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.528048+0000","last_undegraded":"2024-04-25T14:06:44.528048+0000","last_fullsized":"2024-04-25T14:06:44.528048+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.522967+0000","last_change":"2024-04-25T14:06:44.522967+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.522967+0000","last_undegraded":"2024-04-25T14:06:44.522967+0000","last_fullsized":"2024-04-25T14:06:44.522967+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.530589+0000","last_change":"2024-04-25T14:06:44.530589+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.530589+0000","last_undegraded":"2024-04-25T14:06:44.530589+0000","last_fullsized":"2024-04-25T14:06:44.530589+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.521588+0000","last_change":"2024-04-25T14:06:44.521588+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.521588+0000","last_undegraded":"2024-04-25T14:06:44.521588+0000","last_fullsized":"2024-04-25T14:06:44.521588+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.529696+0000","last_change":"2024-04-25T14:06:44.529696+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.529696+0000","last_undegraded":"2024-04-25T14:06:44.529696+0000","last_fullsized":"2024-04-25T14:06:44.529696+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.520096+0000","last_change":"2024-04-25T14:06:44.520096+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.520096+0000","last_undegraded":"2024-04-25T14:06:44.520096+0000","last_fullsized":"2024-04-25T14:06:44.520096+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.532154+0000","last_change":"2024-04-25T14:06:44.532154+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.532154+0000","last_undegraded":"2024-04-25T14:06:44.532154+0000","last_fullsized":"2024-04-25T14:06:44.532154+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.524922+0000","last_change":"2024-04-25T14:06:44.524922+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.524922+0000","last_undegraded":"2024-04-25T14:06:44.524922+0000","last_fullsized":"2024-04-25T14:06:44.524922+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525831+0000","last_change":"2024-04-25T14:06:44.525831+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525831+0000","last_undegraded":"2024-04-25T14:06:44.525831+0000","last_fullsized":"2024-04-25T14:06:44.525831+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.522110+0000","last_change":"2024-04-25T14:06:44.522110+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.522110+0000","last_undegraded":"2024-04-25T14:06:44.522110+0000","last_fullsized":"2024-04-25T14:06:44.522110+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.524249+0000","last_change":"2024-04-25T14:06:44.524249+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.524249+0000","last_undegraded":"2024-04-25T14:06:44.524249+0000","last_fullsized":"2024-04-25T14:06:44.524249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.520624+0000","last_change":"2024-04-25T14:06:44.520624+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.520624+0000","last_undegraded":"2024-04-25T14:06:44.520624+0000","last_fullsized":"2024-04-25T14:06:44.520624+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.524668+0000","last_change":"2024-04-25T14:06:44.524668+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.524668+0000","last_undegraded":"2024-04-25T14:06:44.524668+0000","last_fullsized":"2024-04-25T14:06:44.524668+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.519647+0000","last_change":"2024-04-25T14:06:44.519647+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.519647+0000","last_undegraded":"2024-04-25T14:06:44.519647+0000","last_fullsized":"2024-04-25T14:06:44.519647+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.521818+0000","last_change":"2024-04-25T14:06:44.521818+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.521818+0000","last_undegraded":"2024-04-25T14:06:44.521818+0000","last_fullsized":"2024-04-25T14:06:44.521818+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.524664+0000","last_change":"2024-04-25T14:06:44.524664+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.524664+0000","last_undegraded":"2024-04-25T14:06:44.524664+0000","last_fullsized":"2024-04-25T14:06:44.524664+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.526595+0000","last_change":"2024-04-25T14:06:44.526595+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.526595+0000","last_undegraded":"2024-04-25T14:06:44.526595+0000","last_fullsized":"2024-04-25T14:06:44.526595+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.535681+0000","last_change":"2024-04-25T14:06:44.535681+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.535681+0000","last_undegraded":"2024-04-25T14:06:44.535681+0000","last_fullsized":"2024-04-25T14:06:44.535681+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.519158+0000","last_change":"2024-04-25T14:06:44.519158+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.519158+0000","last_undegraded":"2024-04-25T14:06:44.519158+0000","last_fullsized":"2024-04-25T14:06:44.519158+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.524178+0000","last_change":"2024-04-25T14:06:44.524178+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.524178+0000","last_undegraded":"2024-04-25T14:06:44.524178+0000","last_fullsized":"2024-04-25T14:06:44.524178+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525944+0000","last_change":"2024-04-25T14:06:44.525944+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525944+0000","last_undegraded":"2024-04-25T14:06:44.525944+0000","last_fullsized":"2024-04-25T14:06:44.525944+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525179+0000","last_change":"2024-04-25T14:06:44.525179+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525179+0000","last_undegraded":"2024-04-25T14:06:44.525179+0000","last_fullsized":"2024-04-25T14:06:44.525179+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.523722+0000","last_change":"2024-04-25T14:06:44.523722+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.523722+0000","last_undegraded":"2024-04-25T14:06:44.523722+0000","last_fullsized":"2024-04-25T14:06:44.523722+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.521337+0000","last_change":"2024-04-25T14:06:44.521337+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.521337+0000","last_undegraded":"2024-04-25T14:06:44.521337+0000","last_fullsized":"2024-04-25T14:06:44.521337+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.520503+0000","last_change":"2024-04-25T14:06:44.520503+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.520503+0000","last_undegraded":"2024-04-25T14:06:44.520503+0000","last_fullsized":"2024-04-25T14:06:44.520503+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.520274+0000","last_change":"2024-04-25T14:06:44.520274+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.520274+0000","last_undegraded":"2024-04-25T14:06:44.520274+0000","last_fullsized":"2024-04-25T14:06:44.520274+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.526216+0000","last_change":"2024-04-25T14:06:44.526216+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.526216+0000","last_undegraded":"2024-04-25T14:06:44.526216+0000","last_fullsized":"2024-04-25T14:06:44.526216+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.523614+0000","last_change":"2024-04-25T14:06:44.523614+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.523614+0000","last_undegraded":"2024-04-25T14:06:44.523614+0000","last_fullsized":"2024-04-25T14:06:44.523614+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.521807+0000","last_change":"2024-04-25T14:06:44.521807+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.521807+0000","last_undegraded":"2024-04-25T14:06:44.521807+0000","last_fullsized":"2024-04-25T14:06:44.521807+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.527187+0000","last_change":"2024-04-25T14:06:44.527187+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.527187+0000","last_undegraded":"2024-04-25T14:06:44.527187+0000","last_fullsized":"2024-04-25T14:06:44.527187+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.528762+0000","last_change":"2024-04-25T14:06:44.528762+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.528762+0000","last_undegraded":"2024-04-25T14:06:44.528762+0000","last_fullsized":"2024-04-25T14:06:44.528762+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.524006+0000","last_change":"2024-04-25T14:06:44.524006+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.524006+0000","last_undegraded":"2024-04-25T14:06:44.524006+0000","last_fullsized":"2024-04-25T14:06:44.524006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.529058+0000","last_change":"2024-04-25T14:06:44.529058+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.529058+0000","last_undegraded":"2024-04-25T14:06:44.529058+0000","last_fullsized":"2024-04-25T14:06:44.529058+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.544431+0000","last_change":"2024-04-25T14:06:44.538430+0000","last_active":"2024-04-25T14:06:44.544431+0000","last_peered":"2024-04-25T14:06:44.544431+0000","last_clean":"2024-04-25T14:06:44.544431+0000","last_became_active":"2024-04-25T14:06:44.538081+0000","last_became_peered":"2024-04-25T14:06:44.538081+0000","last_unstale":"2024-04-25T14:06:44.544431+0000","last_undegraded":"2024-04-25T14:06:44.544431+0000","last_fullsized":"2024-04-25T14:06:44.544431+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:19.980060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.521384+0000","last_change":"2024-04-25T14:06:44.521384+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.521384+0000","last_undegraded":"2024-04-25T14:06:44.521384+0000","last_fullsized":"2024-04-25T14:06:44.521384+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.547856+0000","last_change":"2024-04-25T14:06:44.544725+0000","last_active":"2024-04-25T14:06:44.547856+0000","last_peered":"2024-04-25T14:06:44.547856+0000","last_clean":"2024-04-25T14:06:44.547856+0000","last_became_active":"2024-04-25T14:06:44.544154+0000","last_became_peered":"2024-04-25T14:06:44.544154+0000","last_unstale":"2024-04-25T14:06:44.547856+0000","last_undegraded":"2024-04-25T14:06:44.547856+0000","last_fullsized":"2024-04-25T14:06:44.547856+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:27.974323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525629+0000","last_change":"2024-04-25T14:06:44.525629+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525629+0000","last_undegraded":"2024-04-25T14:06:44.525629+0000","last_fullsized":"2024-04-25T14:06:44.525629+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.544093+0000","last_change":"2024-04-25T14:06:44.537708+0000","last_active":"2024-04-25T14:06:44.544093+0000","last_peered":"2024-04-25T14:06:44.544093+0000","last_clean":"2024-04-25T14:06:44.544093+0000","last_became_active":"2024-04-25T14:06:44.537349+0000","last_became_peered":"2024-04-25T14:06:44.537349+0000","last_unstale":"2024-04-25T14:06:44.544093+0000","last_undegraded":"2024-04-25T14:06:44.544093+0000","last_fullsized":"2024-04-25T14:06:44.544093+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:14.448012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525353+0000","last_change":"2024-04-25T14:06:44.525353+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525353+0000","last_undegraded":"2024-04-25T14:06:44.525353+0000","last_fullsized":"2024-04-25T14:06:44.525353+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.550329+0000","last_change":"2024-04-25T14:06:44.546446+0000","last_active":"2024-04-25T14:06:44.550329+0000","last_peered":"2024-04-25T14:06:44.550329+0000","last_clean":"2024-04-25T14:06:44.550329+0000","last_became_active":"2024-04-25T14:06:44.546076+0000","last_became_peered":"2024-04-25T14:06:44.546076+0000","last_unstale":"2024-04-25T14:06:44.550329+0000","last_undegraded":"2024-04-25T14:06:44.550329+0000","last_fullsized":"2024-04-25T14:06:44.550329+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:03.415461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.522329+0000","last_change":"2024-04-25T14:06:44.522329+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.522329+0000","last_undegraded":"2024-04-25T14:06:44.522329+0000","last_fullsized":"2024-04-25T14:06:44.522329+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.544176+0000","last_change":"2024-04-25T14:06:44.538629+0000","last_active":"2024-04-25T14:06:44.544176+0000","last_peered":"2024-04-25T14:06:44.544176+0000","last_clean":"2024-04-25T14:06:44.544176+0000","last_became_active":"2024-04-25T14:06:44.538329+0000","last_became_peered":"2024-04-25T14:06:44.538329+0000","last_unstale":"2024-04-25T14:06:44.544176+0000","last_undegraded":"2024-04-25T14:06:44.544176+0000","last_fullsized":"2024-04-25T14:06:44.544176+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:02.112409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.523772+0000","last_change":"2024-04-25T14:06:44.523772+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.523772+0000","last_undegraded":"2024-04-25T14:06:44.523772+0000","last_fullsized":"2024-04-25T14:06:44.523772+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.534286+0000","last_change":"2024-04-25T14:06:44.530649+0000","last_active":"2024-04-25T14:06:44.534286+0000","last_peered":"2024-04-25T14:06:44.534286+0000","last_clean":"2024-04-25T14:06:44.534286+0000","last_became_active":"2024-04-25T14:06:44.530025+0000","last_became_peered":"2024-04-25T14:06:44.530025+0000","last_unstale":"2024-04-25T14:06:44.534286+0000","last_undegraded":"2024-04-25T14:06:44.534286+0000","last_fullsized":"2024-04-25T14:06:44.534286+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:34.160672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525151+0000","last_change":"2024-04-25T14:06:44.525151+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525151+0000","last_undegraded":"2024-04-25T14:06:44.525151+0000","last_fullsized":"2024-04-25T14:06:44.525151+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.542594+0000","last_change":"2024-04-25T14:06:44.537817+0000","last_active":"2024-04-25T14:06:44.542594+0000","last_peered":"2024-04-25T14:06:44.542594+0000","last_clean":"2024-04-25T14:06:44.542594+0000","last_became_active":"2024-04-25T14:06:44.537198+0000","last_became_peered":"2024-04-25T14:06:44.537198+0000","last_unstale":"2024-04-25T14:06:44.542594+0000","last_undegraded":"2024-04-25T14:06:44.542594+0000","last_fullsized":"2024-04-25T14:06:44.542594+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:42.088259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.536745+0000","last_change":"2024-04-25T14:06:44.536745+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.536745+0000","last_undegraded":"2024-04-25T14:06:44.536745+0000","last_fullsized":"2024-04-25T14:06:44.536745+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.544663+0000","last_change":"2024-04-25T14:06:44.539464+0000","last_active":"2024-04-25T14:06:44.544663+0000","last_peered":"2024-04-25T14:06:44.544663+0000","last_clean":"2024-04-25T14:06:44.544663+0000","last_became_active":"2024-04-25T14:06:44.539110+0000","last_became_peered":"2024-04-25T14:06:44.539110+0000","last_unstale":"2024-04-25T14:06:44.544663+0000","last_undegraded":"2024-04-25T14:06:44.544663+0000","last_fullsized":"2024-04-25T14:06:44.544663+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:41.171166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.520415+0000","last_change":"2024-04-25T14:06:44.520415+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.520415+0000","last_undegraded":"2024-04-25T14:06:44.520415+0000","last_fullsized":"2024-04-25T14:06:44.520415+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.552725+0000","last_change":"2024-04-25T14:06:44.548815+0000","last_active":"2024-04-25T14:06:44.552725+0000","last_peered":"2024-04-25T14:06:44.552725+0000","last_clean":"2024-04-25T14:06:44.552725+0000","last_became_active":"2024-04-25T14:06:44.548469+0000","last_became_peered":"2024-04-25T14:06:44.548469+0000","last_unstale":"2024-04-25T14:06:44.552725+0000","last_undegraded":"2024-04-25T14:06:44.552725+0000","last_fullsized":"2024-04-25T14:06:44.552725+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:24.243932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.519850+0000","last_change":"2024-04-25T14:06:44.519850+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.519850+0000","last_undegraded":"2024-04-25T14:06:44.519850+0000","last_fullsized":"2024-04-25T14:06:44.519850+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.543141+0000","last_change":"2024-04-25T14:06:44.540795+0000","last_active":"2024-04-25T14:06:44.543141+0000","last_peered":"2024-04-25T14:06:44.543141+0000","last_clean":"2024-04-25T14:06:44.543141+0000","last_became_active":"2024-04-25T14:06:44.540467+0000","last_became_peered":"2024-04-25T14:06:44.540467+0000","last_unstale":"2024-04-25T14:06:44.543141+0000","last_undegraded":"2024-04-25T14:06:44.543141+0000","last_fullsized":"2024-04-25T14:06:44.543141+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:14.448012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.519140+0000","last_change":"2024-04-25T14:06:44.519140+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.519140+0000","last_undegraded":"2024-04-25T14:06:44.519140+0000","last_fullsized":"2024-04-25T14:06:44.519140+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.546703+0000","last_change":"2024-04-25T14:06:44.544689+0000","last_active":"2024-04-25T14:06:44.546703+0000","last_peered":"2024-04-25T14:06:44.546703+0000","last_clean":"2024-04-25T14:06:44.546703+0000","last_became_active":"2024-04-25T14:06:44.544095+0000","last_became_peered":"2024-04-25T14:06:44.544095+0000","last_unstale":"2024-04-25T14:06:44.546703+0000","last_undegraded":"2024-04-25T14:06:44.546703+0000","last_fullsized":"2024-04-25T14:06:44.546703+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:11.780439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.529532+0000","last_change":"2024-04-25T14:06:44.529532+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.529532+0000","last_undegraded":"2024-04-25T14:06:44.529532+0000","last_fullsized":"2024-04-25T14:06:44.529532+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.550353+0000","last_change":"2024-04-25T14:06:44.545874+0000","last_active":"2024-04-25T14:06:44.550353+0000","last_peered":"2024-04-25T14:06:44.550353+0000","last_clean":"2024-04-25T14:06:44.550353+0000","last_became_active":"2024-04-25T14:06:44.545363+0000","last_became_peered":"2024-04-25T14:06:44.545363+0000","last_unstale":"2024-04-25T14:06:44.550353+0000","last_undegraded":"2024-04-25T14:06:44.550353+0000","last_fullsized":"2024-04-25T14:06:44.550353+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:16.858125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.519818+0000","last_change":"2024-04-25T14:06:44.519818+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.519818+0000","last_undegraded":"2024-04-25T14:06:44.519818+0000","last_fullsized":"2024-04-25T14:06:44.519818+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.538447+0000","last_change":"2024-04-25T14:06:44.533506+0000","last_active":"2024-04-25T14:06:44.538447+0000","last_peered":"2024-04-25T14:06:44.538447+0000","last_clean":"2024-04-25T14:06:44.538447+0000","last_became_active":"2024-04-25T14:06:44.533189+0000","last_became_peered":"2024-04-25T14:06:44.533189+0000","last_unstale":"2024-04-25T14:06:44.538447+0000","last_undegraded":"2024-04-25T14:06:44.538447+0000","last_fullsized":"2024-04-25T14:06:44.538447+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:27.614025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.534844+0000","last_change":"2024-04-25T14:06:44.534844+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.534844+0000","last_undegraded":"2024-04-25T14:06:44.534844+0000","last_fullsized":"2024-04-25T14:06:44.534844+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.547834+0000","last_change":"2024-04-25T14:06:44.544582+0000","last_active":"2024-04-25T14:06:44.547834+0000","last_peered":"2024-04-25T14:06:44.547834+0000","last_clean":"2024-04-25T14:06:44.547834+0000","last_became_active":"2024-04-25T14:06:44.544049+0000","last_became_peered":"2024-04-25T14:06:44.544049+0000","last_unstale":"2024-04-25T14:06:44.547834+0000","last_undegraded":"2024-04-25T14:06:44.547834+0000","last_fullsized":"2024-04-25T14:06:44.547834+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:22.780792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.529499+0000","last_change":"2024-04-25T14:06:44.529499+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.529499+0000","last_undegraded":"2024-04-25T14:06:44.529499+0000","last_fullsized":"2024-04-25T14:06:44.529499+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.542053+0000","last_change":"2024-04-25T14:06:44.534032+0000","last_active":"2024-04-25T14:06:44.542053+0000","last_peered":"2024-04-25T14:06:44.542053+0000","last_clean":"2024-04-25T14:06:44.542053+0000","last_became_active":"2024-04-25T14:06:44.533673+0000","last_became_peered":"2024-04-25T14:06:44.533673+0000","last_unstale":"2024-04-25T14:06:44.542053+0000","last_undegraded":"2024-04-25T14:06:44.542053+0000","last_fullsized":"2024-04-25T14:06:44.542053+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:29.427107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.531944+0000","last_change":"2024-04-25T14:06:44.531944+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.531944+0000","last_undegraded":"2024-04-25T14:06:44.531944+0000","last_fullsized":"2024-04-25T14:06:44.531944+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.542699+0000","last_change":"2024-04-25T14:06:44.538705+0000","last_active":"2024-04-25T14:06:44.542699+0000","last_peered":"2024-04-25T14:06:44.542699+0000","last_clean":"2024-04-25T14:06:44.542699+0000","last_became_active":"2024-04-25T14:06:44.538404+0000","last_became_peered":"2024-04-25T14:06:44.538404+0000","last_unstale":"2024-04-25T14:06:44.542699+0000","last_undegraded":"2024-04-25T14:06:44.542699+0000","last_fullsized":"2024-04-25T14:06:44.542699+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:16.314477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.518861+0000","last_change":"2024-04-25T14:06:44.518861+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.518861+0000","last_undegraded":"2024-04-25T14:06:44.518861+0000","last_fullsized":"2024-04-25T14:06:44.518861+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.550286+0000","last_change":"2024-04-25T14:06:44.547295+0000","last_active":"2024-04-25T14:06:44.550286+0000","last_peered":"2024-04-25T14:06:44.550286+0000","last_clean":"2024-04-25T14:06:44.550286+0000","last_became_active":"2024-04-25T14:06:44.546930+0000","last_became_peered":"2024-04-25T14:06:44.546930+0000","last_unstale":"2024-04-25T14:06:44.550286+0000","last_undegraded":"2024-04-25T14:06:44.550286+0000","last_fullsized":"2024-04-25T14:06:44.550286+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:46.781154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.523760+0000","last_change":"2024-04-25T14:06:44.523760+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.523760+0000","last_undegraded":"2024-04-25T14:06:44.523760+0000","last_fullsized":"2024-04-25T14:06:44.523760+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.546025+0000","last_change":"2024-04-25T14:06:44.543006+0000","last_active":"2024-04-25T14:06:44.546025+0000","last_peered":"2024-04-25T14:06:44.546025+0000","last_clean":"2024-04-25T14:06:44.546025+0000","last_became_active":"2024-04-25T14:06:44.542678+0000","last_became_peered":"2024-04-25T14:06:44.542678+0000","last_unstale":"2024-04-25T14:06:44.546025+0000","last_undegraded":"2024-04-25T14:06:44.546025+0000","last_fullsized":"2024-04-25T14:06:44.546025+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:16.282476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525734+0000","last_change":"2024-04-25T14:06:44.525734+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525734+0000","last_undegraded":"2024-04-25T14:06:44.525734+0000","last_fullsized":"2024-04-25T14:06:44.525734+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.544904+0000","last_change":"2024-04-25T14:06:44.541885+0000","last_active":"2024-04-25T14:06:44.544904+0000","last_peered":"2024-04-25T14:06:44.544904+0000","last_clean":"2024-04-25T14:06:44.544904+0000","last_became_active":"2024-04-25T14:06:44.541501+0000","last_became_peered":"2024-04-25T14:06:44.541501+0000","last_unstale":"2024-04-25T14:06:44.544904+0000","last_undegraded":"2024-04-25T14:06:44.544904+0000","last_fullsized":"2024-04-25T14:06:44.544904+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:44.424458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.519094+0000","last_change":"2024-04-25T14:06:44.519094+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.519094+0000","last_undegraded":"2024-04-25T14:06:44.519094+0000","last_fullsized":"2024-04-25T14:06:44.519094+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.546591+0000","last_change":"2024-04-25T14:06:44.544634+0000","last_active":"2024-04-25T14:06:44.546591+0000","last_peered":"2024-04-25T14:06:44.546591+0000","last_clean":"2024-04-25T14:06:44.546591+0000","last_became_active":"2024-04-25T14:06:44.544365+0000","last_became_peered":"2024-04-25T14:06:44.544365+0000","last_unstale":"2024-04-25T14:06:44.546591+0000","last_undegraded":"2024-04-25T14:06:44.546591+0000","last_fullsized":"2024-04-25T14:06:44.546591+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:54:29.024591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.521205+0000","last_change":"2024-04-25T14:06:44.521205+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.521205+0000","last_undegraded":"2024-04-25T14:06:44.521205+0000","last_fullsized":"2024-04-25T14:06:44.521205+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.543827+0000","last_change":"2024-04-25T14:06:44.541617+0000","last_active":"2024-04-25T14:06:44.543827+0000","last_peered":"2024-04-25T14:06:44.543827+0000","last_clean":"2024-04-25T14:06:44.543827+0000","last_became_active":"2024-04-25T14:06:44.540854+0000","last_became_peered":"2024-04-25T14:06:44.540854+0000","last_unstale":"2024-04-25T14:06:44.543827+0000","last_undegraded":"2024-04-25T14:06:44.543827+0000","last_fullsized":"2024-04-25T14:06:44.543827+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:14.236144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.520875+0000","last_change":"2024-04-25T14:06:44.520875+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.520875+0000","last_undegraded":"2024-04-25T14:06:44.520875+0000","last_fullsized":"2024-04-25T14:06:44.520875+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.543972+0000","last_change":"2024-04-25T14:06:44.537769+0000","last_active":"2024-04-25T14:06:44.543972+0000","last_peered":"2024-04-25T14:06:44.543972+0000","last_clean":"2024-04-25T14:06:44.543972+0000","last_became_active":"2024-04-25T14:06:44.537407+0000","last_became_peered":"2024-04-25T14:06:44.537407+0000","last_unstale":"2024-04-25T14:06:44.543972+0000","last_undegraded":"2024-04-25T14:06:44.543972+0000","last_fullsized":"2024-04-25T14:06:44.543972+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:58.086378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.529722+0000","last_change":"2024-04-25T14:06:44.529722+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.529722+0000","last_undegraded":"2024-04-25T14:06:44.529722+0000","last_fullsized":"2024-04-25T14:06:44.529722+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.542491+0000","last_change":"2024-04-25T14:06:44.538743+0000","last_active":"2024-04-25T14:06:44.542491+0000","last_peered":"2024-04-25T14:06:44.542491+0000","last_clean":"2024-04-25T14:06:44.542491+0000","last_became_active":"2024-04-25T14:06:44.538365+0000","last_became_peered":"2024-04-25T14:06:44.538365+0000","last_unstale":"2024-04-25T14:06:44.542491+0000","last_undegraded":"2024-04-25T14:06:44.542491+0000","last_fullsized":"2024-04-25T14:06:44.542491+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:47.507460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.524176+0000","last_change":"2024-04-25T14:06:44.524176+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.524176+0000","last_undegraded":"2024-04-25T14:06:44.524176+0000","last_fullsized":"2024-04-25T14:06:44.524176+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.547650+0000","last_change":"2024-04-25T14:06:44.541609+0000","last_active":"2024-04-25T14:06:44.547650+0000","last_peered":"2024-04-25T14:06:44.547650+0000","last_clean":"2024-04-25T14:06:44.547650+0000","last_became_active":"2024-04-25T14:06:44.541009+0000","last_became_peered":"2024-04-25T14:06:44.541009+0000","last_unstale":"2024-04-25T14:06:44.547650+0000","last_undegraded":"2024-04-25T14:06:44.547650+0000","last_fullsized":"2024-04-25T14:06:44.547650+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:58.970595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.529423+0000","last_change":"2024-04-25T14:06:44.529423+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.529423+0000","last_undegraded":"2024-04-25T14:06:44.529423+0000","last_fullsized":"2024-04-25T14:06:44.529423+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.539705+0000","last_change":"2024-04-25T14:06:44.535705+0000","last_active":"2024-04-25T14:06:44.539705+0000","last_peered":"2024-04-25T14:06:44.539705+0000","last_clean":"2024-04-25T14:06:44.539705+0000","last_became_active":"2024-04-25T14:06:44.535457+0000","last_became_peered":"2024-04-25T14:06:44.535457+0000","last_unstale":"2024-04-25T14:06:44.539705+0000","last_undegraded":"2024-04-25T14:06:44.539705+0000","last_fullsized":"2024-04-25T14:06:44.539705+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:06.121310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.523979+0000","last_change":"2024-04-25T14:06:44.523979+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.523979+0000","last_undegraded":"2024-04-25T14:06:44.523979+0000","last_fullsized":"2024-04-25T14:06:44.523979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.544953+0000","last_change":"2024-04-25T14:06:44.542180+0000","last_active":"2024-04-25T14:06:44.544953+0000","last_peered":"2024-04-25T14:06:44.544953+0000","last_clean":"2024-04-25T14:06:44.544953+0000","last_became_active":"2024-04-25T14:06:44.540654+0000","last_became_peered":"2024-04-25T14:06:44.540654+0000","last_unstale":"2024-04-25T14:06:44.544953+0000","last_undegraded":"2024-04-25T14:06:44.544953+0000","last_fullsized":"2024-04-25T14:06:44.544953+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:58.970595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.520299+0000","last_change":"2024-04-25T14:06:44.520299+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.520299+0000","last_undegraded":"2024-04-25T14:06:44.520299+0000","last_fullsized":"2024-04-25T14:06:44.520299+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.547611+0000","last_change":"2024-04-25T14:06:44.543878+0000","last_active":"2024-04-25T14:06:44.547611+0000","last_peered":"2024-04-25T14:06:44.547611+0000","last_clean":"2024-04-25T14:06:44.547611+0000","last_became_active":"2024-04-25T14:06:44.543347+0000","last_became_peered":"2024-04-25T14:06:44.543347+0000","last_unstale":"2024-04-25T14:06:44.547611+0000","last_undegraded":"2024-04-25T14:06:44.547611+0000","last_fullsized":"2024-04-25T14:06:44.547611+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:56.180081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.532473+0000","last_change":"2024-04-25T14:06:44.532473+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.532473+0000","last_undegraded":"2024-04-25T14:06:44.532473+0000","last_fullsized":"2024-04-25T14:06:44.532473+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.550348+0000","last_change":"2024-04-25T14:06:44.546730+0000","last_active":"2024-04-25T14:06:44.550348+0000","last_peered":"2024-04-25T14:06:44.550348+0000","last_clean":"2024-04-25T14:06:44.550348+0000","last_became_active":"2024-04-25T14:06:44.546050+0000","last_became_peered":"2024-04-25T14:06:44.546050+0000","last_unstale":"2024-04-25T14:06:44.550348+0000","last_undegraded":"2024-04-25T14:06:44.550348+0000","last_fullsized":"2024-04-25T14:06:44.550348+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:58:21.045855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.531523+0000","last_change":"2024-04-25T14:06:44.531523+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.531523+0000","last_undegraded":"2024-04-25T14:06:44.531523+0000","last_fullsized":"2024-04-25T14:06:44.531523+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.540642+0000","last_change":"2024-04-25T14:06:44.534134+0000","last_active":"2024-04-25T14:06:44.540642+0000","last_peered":"2024-04-25T14:06:44.540642+0000","last_clean":"2024-04-25T14:06:44.540642+0000","last_became_active":"2024-04-25T14:06:44.533705+0000","last_became_peered":"2024-04-25T14:06:44.533705+0000","last_unstale":"2024-04-25T14:06:44.540642+0000","last_undegraded":"2024-04-25T14:06:44.540642+0000","last_fullsized":"2024-04-25T14:06:44.540642+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:43.875733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.535223+0000","last_change":"2024-04-25T14:06:44.535223+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.535223+0000","last_undegraded":"2024-04-25T14:06:44.535223+0000","last_fullsized":"2024-04-25T14:06:44.535223+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.545851+0000","last_change":"2024-04-25T14:06:44.538191+0000","last_active":"2024-04-25T14:06:44.545851+0000","last_peered":"2024-04-25T14:06:44.545851+0000","last_clean":"2024-04-25T14:06:44.545851+0000","last_became_active":"2024-04-25T14:06:44.537838+0000","last_became_peered":"2024-04-25T14:06:44.537838+0000","last_unstale":"2024-04-25T14:06:44.545851+0000","last_undegraded":"2024-04-25T14:06:44.545851+0000","last_fullsized":"2024-04-25T14:06:44.545851+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:43.249499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.532357+0000","last_change":"2024-04-25T14:06:44.532357+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.532357+0000","last_undegraded":"2024-04-25T14:06:44.532357+0000","last_fullsized":"2024-04-25T14:06:44.532357+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.543260+0000","last_change":"2024-04-25T14:06:44.539538+0000","last_active":"2024-04-25T14:06:44.543260+0000","last_peered":"2024-04-25T14:06:44.543260+0000","last_clean":"2024-04-25T14:06:44.543260+0000","last_became_active":"2024-04-25T14:06:44.538128+0000","last_became_peered":"2024-04-25T14:06:44.538128+0000","last_unstale":"2024-04-25T14:06:44.543260+0000","last_undegraded":"2024-04-25T14:06:44.543260+0000","last_fullsized":"2024-04-25T14:06:44.543260+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:56.433866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525676+0000","last_change":"2024-04-25T14:06:44.525676+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525676+0000","last_undegraded":"2024-04-25T14:06:44.525676+0000","last_fullsized":"2024-04-25T14:06:44.525676+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.542537+0000","last_change":"2024-04-25T14:06:44.539206+0000","last_active":"2024-04-25T14:06:44.542537+0000","last_peered":"2024-04-25T14:06:44.542537+0000","last_clean":"2024-04-25T14:06:44.542537+0000","last_became_active":"2024-04-25T14:06:44.538913+0000","last_became_peered":"2024-04-25T14:06:44.538913+0000","last_unstale":"2024-04-25T14:06:44.542537+0000","last_undegraded":"2024-04-25T14:06:44.542537+0000","last_fullsized":"2024-04-25T14:06:44.542537+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:09.304323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"59'86","reported_seq":255,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.525284+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:06:44.525284+0000","last_peered":"2024-04-25T14:06:44.525284+0000","last_clean":"2024-04-25T14:06:44.525284+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:06:44.525284+0000","last_undegraded":"2024-04-25T14:06:44.525284+0000","last_fullsized":"2024-04-25T14:06:44.525284+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1474560,"data_stored":1023970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":433602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":880640,"data_stored":433602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":433602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":433602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":433602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":433602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1474560,"data_stored":1023970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:06:50.079 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:06:50.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.263+0000 7f2592400640 1 Processor -- start 2024-04-25T14:06:50.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.263+0000 7f2592400640 1 -- start start 2024-04-25T14:06:50.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.267+0000 7f2592400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f258c071530 0x7f258c15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.267+0000 7f2592400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f258c0cd8d0 con 0x7f258c0d2c80 2024-04-25T14:06:50.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.267+0000 7f2592400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f258c0ce970 con 0x7f258c1616e0 2024-04-25T14:06:50.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.267+0000 7f2592400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f258c0ce720 con 0x7f258c071530 2024-04-25T14:06:50.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.267+0000 7f258b400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f258c0d2c80 0x7f258c070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51718/0 (socket says 172.21.15.86:51718) 2024-04-25T14:06:50.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.267+0000 7f258b400640 1 -- 172.21.15.86:0/396716001 learned_addr learned my addr 172.21.15.86:0/396716001 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:50.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.267+0000 7f2583e00640 1 --2- 172.21.15.86:0/396716001 >> v2:172.21.15.155:3300/0 conn(0x7f258c071530 0x7f258c15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.267+0000 7f258aa00640 1 -- 172.21.15.86:0/396716001 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4233712362 0 0) 0x7f258c0cd8d0 con 0x7f258c0d2c80 2024-04-25T14:06:50.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.267+0000 7f258aa00640 1 -- 172.21.15.86:0/396716001 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2570003530 con 0x7f258c0d2c80 2024-04-25T14:06:50.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.267+0000 7f2583e00640 1 -- 172.21.15.86:0/396716001 >> v1:172.21.15.155:6789/0 conn(0x7f258c1616e0 legacy=0x7f258c161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.267+0000 7f2583e00640 1 -- 172.21.15.86:0/396716001 >> v1:172.21.15.86:6789/0 conn(0x7f258c0d2c80 legacy=0x7f258c070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.267+0000 7f2583e00640 1 -- 172.21.15.86:0/396716001 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258c0d1be0 con 0x7f258c071530 2024-04-25T14:06:50.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.271+0000 7f2583e00640 1 --2- 172.21.15.86:0/396716001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f258c071530 0x7f258c15dc90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f25740097a0 tx=0x7f257401ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48e64822e97c147e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:50.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.271+0000 7f258aa00640 1 -- 172.21.15.86:0/396716001 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25740b5070 con 0x7f258c071530 2024-04-25T14:06:50.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.271+0000 7f258aa00640 1 -- 172.21.15.86:0/396716001 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25740a8970 con 0x7f258c071530 2024-04-25T14:06:50.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.271+0000 7f258aa00640 1 -- 172.21.15.86:0/396716001 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25740a8d20 con 0x7f258c071530 2024-04-25T14:06:50.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.271+0000 7f2592400640 1 -- 172.21.15.86:0/396716001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f258c071530 msgr2=0x7f258c15dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.271+0000 7f2592400640 1 --2- 172.21.15.86:0/396716001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f258c071530 0x7f258c15dc90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f25740097a0 tx=0x7f257401ec70 comp rx=0 tx=0).stop 2024-04-25T14:06:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.271+0000 7f2592400640 1 -- 172.21.15.86:0/396716001 shutdown_connections 2024-04-25T14:06:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.271+0000 7f2592400640 1 --2- 172.21.15.86:0/396716001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f258c071530 0x7f258c15dc90 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.271+0000 7f2592400640 1 -- 172.21.15.86:0/396716001 >> 172.21.15.86:0/396716001 conn(0x7f258c0cffc0 msgr2=0x7f258c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.271+0000 7f2592400640 1 -- 172.21.15.86:0/396716001 shutdown_connections 2024-04-25T14:06:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.271+0000 7f2592400640 1 -- 172.21.15.86:0/396716001 wait complete. 2024-04-25T14:06:50.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.271+0000 7f2592400640 1 Processor -- start 2024-04-25T14:06:50.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.271+0000 7f2592400640 1 -- start start 2024-04-25T14:06:50.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.275+0000 7f2592400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258c0d2c80 0x7f258c15e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.275+0000 7f2592400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f258c1616e0 0x7f258c0b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.275+0000 7f2592400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f258c0b4cd0 0x7f258c0b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.275+0000 7f2592400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f258c0d2480 con 0x7f258c0b4cd0 2024-04-25T14:06:50.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.275+0000 7f2592400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f258c0d21a0 con 0x7f258c1616e0 2024-04-25T14:06:50.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.275+0000 7f2592400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f258c0d29a0 con 0x7f258c0d2c80 2024-04-25T14:06:50.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.275+0000 7f258b400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258c0d2c80 0x7f258c15e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.275+0000 7f258be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f258c0b4cd0 0x7f258c0b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.275+0000 7f258b400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258c0d2c80 0x7f258c15e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:45888/0 (socket says 172.21.15.86:45888) 2024-04-25T14:06:50.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.275+0000 7f258b400640 1 -- 172.21.15.86:0/4198507113 learned_addr learned my addr 172.21.15.86:0/4198507113 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:50.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.275+0000 7f258be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f258c0b4cd0 0x7f258c0b50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56474/0 (socket says 172.21.15.86:56474) 2024-04-25T14:06:50.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.275+0000 7f2583e00640 1 --2- 172.21.15.86:0/4198507113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f258c1616e0 0x7f258c0b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.275+0000 7f258b400640 1 -- 172.21.15.86:0/4198507113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f258c1616e0 msgr2=0x7f258c0b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.275+0000 7f258b400640 1 --2- 172.21.15.86:0/4198507113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f258c1616e0 0x7f258c0b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.275+0000 7f258b400640 1 -- 172.21.15.86:0/4198507113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f258c0b4cd0 msgr2=0x7f258c0b50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.275+0000 7f258b400640 1 --2- 172.21.15.86:0/4198507113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f258c0b4cd0 0x7f258c0b50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.275+0000 7f258b400640 1 -- 172.21.15.86:0/4198507113 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258c0b24e0 con 0x7f258c0d2c80 2024-04-25T14:06:50.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.275+0000 7f2583e00640 1 --2- 172.21.15.86:0/4198507113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f258c1616e0 0x7f258c0b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:50.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.275+0000 7f258b400640 1 --2- 172.21.15.86:0/4198507113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258c0d2c80 0x7f258c15e600 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f257c00c450 tx=0x7f257c00c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:50.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.275+0000 7f2583400640 1 -- 172.21.15.86:0/4198507113 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c017070 con 0x7f258c0d2c80 2024-04-25T14:06:50.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.275+0000 7f2583400640 1 -- 172.21.15.86:0/4198507113 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f257c005cf0 con 0x7f258c0d2c80 2024-04-25T14:06:50.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.275+0000 7f2592400640 1 -- 172.21.15.86:0/4198507113 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f258c06ef00 con 0x7f258c0d2c80 2024-04-25T14:06:50.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.275+0000 7f2583400640 1 -- 172.21.15.86:0/4198507113 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c007270 con 0x7f258c0d2c80 2024-04-25T14:06:50.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.275+0000 7f2592400640 1 -- 172.21.15.86:0/4198507113 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f258c15f1a0 con 0x7f258c0d2c80 2024-04-25T14:06:50.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.275+0000 7f2592400640 1 -- 172.21.15.86:0/4198507113 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f258c0720e0 con 0x7f258c0d2c80 2024-04-25T14:06:50.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.279+0000 7f2583400640 1 -- 172.21.15.86:0/4198507113 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f257c01a070 con 0x7f258c0d2c80 2024-04-25T14:06:50.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.279+0000 7f2583400640 1 --2- 172.21.15.86:0/4198507113 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f256806f640 0x7f2568071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.279+0000 7f2583400640 1 -- 172.21.15.86:0/4198507113 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f257c08eb50 con 0x7f258c0d2c80 2024-04-25T14:06:50.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.279+0000 7f2583e00640 1 --2- 172.21.15.86:0/4198507113 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f256806f640 0x7f2568071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.283+0000 7f2583e00640 1 --2- 172.21.15.86:0/4198507113 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f256806f640 0x7f2568071b50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f25740b2000 tx=0x7f25740aa070 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:50.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.287+0000 7f2583400640 1 -- 172.21.15.86:0/4198507113 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f257c05af00 con 0x7f258c0d2c80 2024-04-25T14:06:50.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.443+0000 7f2592400640 1 -- 172.21.15.86:0/4198507113 --> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f258c069d60 con 0x7f256806f640 2024-04-25T14:06:50.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.447+0000 7f2583400640 1 -- 172.21.15.86:0/4198507113 <== mgr.6198 v2:172.21.15.86:6832/1025829557 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246744 (secure 0 0 0) 0x7f25740a9960 con 0x7f256806f640 2024-04-25T14:06:50.454 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:50.455 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:06:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.451+0000 7f2592400640 1 -- 172.21.15.86:0/4198507113 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f256806f640 msgr2=0x7f2568071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.451+0000 7f2592400640 1 --2- 172.21.15.86:0/4198507113 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f256806f640 0x7f2568071b50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f25740b2000 tx=0x7f25740aa070 comp rx=0 tx=0).stop 2024-04-25T14:06:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.451+0000 7f2592400640 1 -- 172.21.15.86:0/4198507113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258c0d2c80 msgr2=0x7f258c15e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.451+0000 7f2592400640 1 --2- 172.21.15.86:0/4198507113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258c0d2c80 0x7f258c15e600 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f257c00c450 tx=0x7f257c00c910 comp rx=0 tx=0).stop 2024-04-25T14:06:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.451+0000 7f2592400640 1 -- 172.21.15.86:0/4198507113 shutdown_connections 2024-04-25T14:06:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.451+0000 7f2592400640 1 --2- 172.21.15.86:0/4198507113 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f256806f640 0x7f2568071b50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.451+0000 7f2592400640 1 --2- 172.21.15.86:0/4198507113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f258c0b4cd0 0x7f258c0b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.451+0000 7f2592400640 1 --2- 172.21.15.86:0/4198507113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f258c1616e0 0x7f258c0b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.451+0000 7f2592400640 1 --2- 172.21.15.86:0/4198507113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258c0d2c80 0x7f258c15e600 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.451+0000 7f2592400640 1 -- 172.21.15.86:0/4198507113 >> 172.21.15.86:0/4198507113 conn(0x7f258c0cffc0 msgr2=0x7f258c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:50.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.451+0000 7f2592400640 1 -- 172.21.15.86:0/4198507113 shutdown_connections 2024-04-25T14:06:50.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.451+0000 7f2592400640 1 -- 172.21.15.86:0/4198507113 wait complete. 2024-04-25T14:06:50.475 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-04-25T14:06:49.438066+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253352,"kb_used_data":8040,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721368,"statfs":{"total":773094113280,"available":772834680832,"internally_reserved":0,"allocated":8232960,"data_stored":4649552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.927238"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.528048+0000","last_change":"2024-04-25T14:06:44.528048+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.528048+0000","last_undegraded":"2024-04-25T14:06:44.528048+0000","last_fullsized":"2024-04-25T14:06:44.528048+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.522967+0000","last_change":"2024-04-25T14:06:44.522967+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.522967+0000","last_undegraded":"2024-04-25T14:06:44.522967+0000","last_fullsized":"2024-04-25T14:06:44.522967+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.530589+0000","last_change":"2024-04-25T14:06:44.530589+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.530589+0000","last_undegraded":"2024-04-25T14:06:44.530589+0000","last_fullsized":"2024-04-25T14:06:44.530589+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.521588+0000","last_change":"2024-04-25T14:06:44.521588+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.521588+0000","last_undegraded":"2024-04-25T14:06:44.521588+0000","last_fullsized":"2024-04-25T14:06:44.521588+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.529696+0000","last_change":"2024-04-25T14:06:44.529696+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.529696+0000","last_undegraded":"2024-04-25T14:06:44.529696+0000","last_fullsized":"2024-04-25T14:06:44.529696+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.520096+0000","last_change":"2024-04-25T14:06:44.520096+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.520096+0000","last_undegraded":"2024-04-25T14:06:44.520096+0000","last_fullsized":"2024-04-25T14:06:44.520096+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.532154+0000","last_change":"2024-04-25T14:06:44.532154+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.532154+0000","last_undegraded":"2024-04-25T14:06:44.532154+0000","last_fullsized":"2024-04-25T14:06:44.532154+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.524922+0000","last_change":"2024-04-25T14:06:44.524922+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.524922+0000","last_undegraded":"2024-04-25T14:06:44.524922+0000","last_fullsized":"2024-04-25T14:06:44.524922+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525831+0000","last_change":"2024-04-25T14:06:44.525831+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525831+0000","last_undegraded":"2024-04-25T14:06:44.525831+0000","last_fullsized":"2024-04-25T14:06:44.525831+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.522110+0000","last_change":"2024-04-25T14:06:44.522110+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.522110+0000","last_undegraded":"2024-04-25T14:06:44.522110+0000","last_fullsized":"2024-04-25T14:06:44.522110+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.524249+0000","last_change":"2024-04-25T14:06:44.524249+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.524249+0000","last_undegraded":"2024-04-25T14:06:44.524249+0000","last_fullsized":"2024-04-25T14:06:44.524249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.520624+0000","last_change":"2024-04-25T14:06:44.520624+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.520624+0000","last_undegraded":"2024-04-25T14:06:44.520624+0000","last_fullsized":"2024-04-25T14:06:44.520624+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.524668+0000","last_change":"2024-04-25T14:06:44.524668+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.524668+0000","last_undegraded":"2024-04-25T14:06:44.524668+0000","last_fullsized":"2024-04-25T14:06:44.524668+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.519647+0000","last_change":"2024-04-25T14:06:44.519647+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.519647+0000","last_undegraded":"2024-04-25T14:06:44.519647+0000","last_fullsized":"2024-04-25T14:06:44.519647+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.521818+0000","last_change":"2024-04-25T14:06:44.521818+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.521818+0000","last_undegraded":"2024-04-25T14:06:44.521818+0000","last_fullsized":"2024-04-25T14:06:44.521818+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.524664+0000","last_change":"2024-04-25T14:06:44.524664+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.524664+0000","last_undegraded":"2024-04-25T14:06:44.524664+0000","last_fullsized":"2024-04-25T14:06:44.524664+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.526595+0000","last_change":"2024-04-25T14:06:44.526595+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.526595+0000","last_undegraded":"2024-04-25T14:06:44.526595+0000","last_fullsized":"2024-04-25T14:06:44.526595+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.535681+0000","last_change":"2024-04-25T14:06:44.535681+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.535681+0000","last_undegraded":"2024-04-25T14:06:44.535681+0000","last_fullsized":"2024-04-25T14:06:44.535681+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.519158+0000","last_change":"2024-04-25T14:06:44.519158+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.519158+0000","last_undegraded":"2024-04-25T14:06:44.519158+0000","last_fullsized":"2024-04-25T14:06:44.519158+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.524178+0000","last_change":"2024-04-25T14:06:44.524178+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.524178+0000","last_undegraded":"2024-04-25T14:06:44.524178+0000","last_fullsized":"2024-04-25T14:06:44.524178+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525944+0000","last_change":"2024-04-25T14:06:44.525944+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525944+0000","last_undegraded":"2024-04-25T14:06:44.525944+0000","last_fullsized":"2024-04-25T14:06:44.525944+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525179+0000","last_change":"2024-04-25T14:06:44.525179+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525179+0000","last_undegraded":"2024-04-25T14:06:44.525179+0000","last_fullsized":"2024-04-25T14:06:44.525179+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.523722+0000","last_change":"2024-04-25T14:06:44.523722+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.523722+0000","last_undegraded":"2024-04-25T14:06:44.523722+0000","last_fullsized":"2024-04-25T14:06:44.523722+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.521337+0000","last_change":"2024-04-25T14:06:44.521337+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.521337+0000","last_undegraded":"2024-04-25T14:06:44.521337+0000","last_fullsized":"2024-04-25T14:06:44.521337+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.520503+0000","last_change":"2024-04-25T14:06:44.520503+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.520503+0000","last_undegraded":"2024-04-25T14:06:44.520503+0000","last_fullsized":"2024-04-25T14:06:44.520503+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.520274+0000","last_change":"2024-04-25T14:06:44.520274+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.520274+0000","last_undegraded":"2024-04-25T14:06:44.520274+0000","last_fullsized":"2024-04-25T14:06:44.520274+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.526216+0000","last_change":"2024-04-25T14:06:44.526216+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.526216+0000","last_undegraded":"2024-04-25T14:06:44.526216+0000","last_fullsized":"2024-04-25T14:06:44.526216+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.523614+0000","last_change":"2024-04-25T14:06:44.523614+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.523614+0000","last_undegraded":"2024-04-25T14:06:44.523614+0000","last_fullsized":"2024-04-25T14:06:44.523614+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.521807+0000","last_change":"2024-04-25T14:06:44.521807+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.521807+0000","last_undegraded":"2024-04-25T14:06:44.521807+0000","last_fullsized":"2024-04-25T14:06:44.521807+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.527187+0000","last_change":"2024-04-25T14:06:44.527187+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.527187+0000","last_undegraded":"2024-04-25T14:06:44.527187+0000","last_fullsized":"2024-04-25T14:06:44.527187+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.528762+0000","last_change":"2024-04-25T14:06:44.528762+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.528762+0000","last_undegraded":"2024-04-25T14:06:44.528762+0000","last_fullsized":"2024-04-25T14:06:44.528762+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.524006+0000","last_change":"2024-04-25T14:06:44.524006+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.524006+0000","last_undegraded":"2024-04-25T14:06:44.524006+0000","last_fullsized":"2024-04-25T14:06:44.524006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.529058+0000","last_change":"2024-04-25T14:06:44.529058+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.529058+0000","last_undegraded":"2024-04-25T14:06:44.529058+0000","last_fullsized":"2024-04-25T14:06:44.529058+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.544431+0000","last_change":"2024-04-25T14:06:44.538430+0000","last_active":"2024-04-25T14:06:44.544431+0000","last_peered":"2024-04-25T14:06:44.544431+0000","last_clean":"2024-04-25T14:06:44.544431+0000","last_became_active":"2024-04-25T14:06:44.538081+0000","last_became_peered":"2024-04-25T14:06:44.538081+0000","last_unstale":"2024-04-25T14:06:44.544431+0000","last_undegraded":"2024-04-25T14:06:44.544431+0000","last_fullsized":"2024-04-25T14:06:44.544431+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:19.980060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.521384+0000","last_change":"2024-04-25T14:06:44.521384+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.521384+0000","last_undegraded":"2024-04-25T14:06:44.521384+0000","last_fullsized":"2024-04-25T14:06:44.521384+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.547856+0000","last_change":"2024-04-25T14:06:44.544725+0000","last_active":"2024-04-25T14:06:44.547856+0000","last_peered":"2024-04-25T14:06:44.547856+0000","last_clean":"2024-04-25T14:06:44.547856+0000","last_became_active":"2024-04-25T14:06:44.544154+0000","last_became_peered":"2024-04-25T14:06:44.544154+0000","last_unstale":"2024-04-25T14:06:44.547856+0000","last_undegraded":"2024-04-25T14:06:44.547856+0000","last_fullsized":"2024-04-25T14:06:44.547856+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:27.974323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525629+0000","last_change":"2024-04-25T14:06:44.525629+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525629+0000","last_undegraded":"2024-04-25T14:06:44.525629+0000","last_fullsized":"2024-04-25T14:06:44.525629+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.544093+0000","last_change":"2024-04-25T14:06:44.537708+0000","last_active":"2024-04-25T14:06:44.544093+0000","last_peered":"2024-04-25T14:06:44.544093+0000","last_clean":"2024-04-25T14:06:44.544093+0000","last_became_active":"2024-04-25T14:06:44.537349+0000","last_became_peered":"2024-04-25T14:06:44.537349+0000","last_unstale":"2024-04-25T14:06:44.544093+0000","last_undegraded":"2024-04-25T14:06:44.544093+0000","last_fullsized":"2024-04-25T14:06:44.544093+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:14.448012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525353+0000","last_change":"2024-04-25T14:06:44.525353+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525353+0000","last_undegraded":"2024-04-25T14:06:44.525353+0000","last_fullsized":"2024-04-25T14:06:44.525353+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.550329+0000","last_change":"2024-04-25T14:06:44.546446+0000","last_active":"2024-04-25T14:06:44.550329+0000","last_peered":"2024-04-25T14:06:44.550329+0000","last_clean":"2024-04-25T14:06:44.550329+0000","last_became_active":"2024-04-25T14:06:44.546076+0000","last_became_peered":"2024-04-25T14:06:44.546076+0000","last_unstale":"2024-04-25T14:06:44.550329+0000","last_undegraded":"2024-04-25T14:06:44.550329+0000","last_fullsized":"2024-04-25T14:06:44.550329+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:03.415461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.522329+0000","last_change":"2024-04-25T14:06:44.522329+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.522329+0000","last_undegraded":"2024-04-25T14:06:44.522329+0000","last_fullsized":"2024-04-25T14:06:44.522329+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.544176+0000","last_change":"2024-04-25T14:06:44.538629+0000","last_active":"2024-04-25T14:06:44.544176+0000","last_peered":"2024-04-25T14:06:44.544176+0000","last_clean":"2024-04-25T14:06:44.544176+0000","last_became_active":"2024-04-25T14:06:44.538329+0000","last_became_peered":"2024-04-25T14:06:44.538329+0000","last_unstale":"2024-04-25T14:06:44.544176+0000","last_undegraded":"2024-04-25T14:06:44.544176+0000","last_fullsized":"2024-04-25T14:06:44.544176+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:02.112409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.523772+0000","last_change":"2024-04-25T14:06:44.523772+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.523772+0000","last_undegraded":"2024-04-25T14:06:44.523772+0000","last_fullsized":"2024-04-25T14:06:44.523772+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.534286+0000","last_change":"2024-04-25T14:06:44.530649+0000","last_active":"2024-04-25T14:06:44.534286+0000","last_peered":"2024-04-25T14:06:44.534286+0000","last_clean":"2024-04-25T14:06:44.534286+0000","last_became_active":"2024-04-25T14:06:44.530025+0000","last_became_peered":"2024-04-25T14:06:44.530025+0000","last_unstale":"2024-04-25T14:06:44.534286+0000","last_undegraded":"2024-04-25T14:06:44.534286+0000","last_fullsized":"2024-04-25T14:06:44.534286+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:34.160672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525151+0000","last_change":"2024-04-25T14:06:44.525151+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525151+0000","last_undegraded":"2024-04-25T14:06:44.525151+0000","last_fullsized":"2024-04-25T14:06:44.525151+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.542594+0000","last_change":"2024-04-25T14:06:44.537817+0000","last_active":"2024-04-25T14:06:44.542594+0000","last_peered":"2024-04-25T14:06:44.542594+0000","last_clean":"2024-04-25T14:06:44.542594+0000","last_became_active":"2024-04-25T14:06:44.537198+0000","last_became_peered":"2024-04-25T14:06:44.537198+0000","last_unstale":"2024-04-25T14:06:44.542594+0000","last_undegraded":"2024-04-25T14:06:44.542594+0000","last_fullsized":"2024-04-25T14:06:44.542594+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:42.088259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.536745+0000","last_change":"2024-04-25T14:06:44.536745+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.536745+0000","last_undegraded":"2024-04-25T14:06:44.536745+0000","last_fullsized":"2024-04-25T14:06:44.536745+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.544663+0000","last_change":"2024-04-25T14:06:44.539464+0000","last_active":"2024-04-25T14:06:44.544663+0000","last_peered":"2024-04-25T14:06:44.544663+0000","last_clean":"2024-04-25T14:06:44.544663+0000","last_became_active":"2024-04-25T14:06:44.539110+0000","last_became_peered":"2024-04-25T14:06:44.539110+0000","last_unstale":"2024-04-25T14:06:44.544663+0000","last_undegraded":"2024-04-25T14:06:44.544663+0000","last_fullsized":"2024-04-25T14:06:44.544663+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:41.171166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.520415+0000","last_change":"2024-04-25T14:06:44.520415+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.520415+0000","last_undegraded":"2024-04-25T14:06:44.520415+0000","last_fullsized":"2024-04-25T14:06:44.520415+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.552725+0000","last_change":"2024-04-25T14:06:44.548815+0000","last_active":"2024-04-25T14:06:44.552725+0000","last_peered":"2024-04-25T14:06:44.552725+0000","last_clean":"2024-04-25T14:06:44.552725+0000","last_became_active":"2024-04-25T14:06:44.548469+0000","last_became_peered":"2024-04-25T14:06:44.548469+0000","last_unstale":"2024-04-25T14:06:44.552725+0000","last_undegraded":"2024-04-25T14:06:44.552725+0000","last_fullsized":"2024-04-25T14:06:44.552725+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:24.243932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.519850+0000","last_change":"2024-04-25T14:06:44.519850+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.519850+0000","last_undegraded":"2024-04-25T14:06:44.519850+0000","last_fullsized":"2024-04-25T14:06:44.519850+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.543141+0000","last_change":"2024-04-25T14:06:44.540795+0000","last_active":"2024-04-25T14:06:44.543141+0000","last_peered":"2024-04-25T14:06:44.543141+0000","last_clean":"2024-04-25T14:06:44.543141+0000","last_became_active":"2024-04-25T14:06:44.540467+0000","last_became_peered":"2024-04-25T14:06:44.540467+0000","last_unstale":"2024-04-25T14:06:44.543141+0000","last_undegraded":"2024-04-25T14:06:44.543141+0000","last_fullsized":"2024-04-25T14:06:44.543141+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:14.448012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.519140+0000","last_change":"2024-04-25T14:06:44.519140+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.519140+0000","last_undegraded":"2024-04-25T14:06:44.519140+0000","last_fullsized":"2024-04-25T14:06:44.519140+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.546703+0000","last_change":"2024-04-25T14:06:44.544689+0000","last_active":"2024-04-25T14:06:44.546703+0000","last_peered":"2024-04-25T14:06:44.546703+0000","last_clean":"2024-04-25T14:06:44.546703+0000","last_became_active":"2024-04-25T14:06:44.544095+0000","last_became_peered":"2024-04-25T14:06:44.544095+0000","last_unstale":"2024-04-25T14:06:44.546703+0000","last_undegraded":"2024-04-25T14:06:44.546703+0000","last_fullsized":"2024-04-25T14:06:44.546703+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:11.780439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.529532+0000","last_change":"2024-04-25T14:06:44.529532+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.529532+0000","last_undegraded":"2024-04-25T14:06:44.529532+0000","last_fullsized":"2024-04-25T14:06:44.529532+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.550353+0000","last_change":"2024-04-25T14:06:44.545874+0000","last_active":"2024-04-25T14:06:44.550353+0000","last_peered":"2024-04-25T14:06:44.550353+0000","last_clean":"2024-04-25T14:06:44.550353+0000","last_became_active":"2024-04-25T14:06:44.545363+0000","last_became_peered":"2024-04-25T14:06:44.545363+0000","last_unstale":"2024-04-25T14:06:44.550353+0000","last_undegraded":"2024-04-25T14:06:44.550353+0000","last_fullsized":"2024-04-25T14:06:44.550353+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:16.858125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.519818+0000","last_change":"2024-04-25T14:06:44.519818+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.519818+0000","last_undegraded":"2024-04-25T14:06:44.519818+0000","last_fullsized":"2024-04-25T14:06:44.519818+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.538447+0000","last_change":"2024-04-25T14:06:44.533506+0000","last_active":"2024-04-25T14:06:44.538447+0000","last_peered":"2024-04-25T14:06:44.538447+0000","last_clean":"2024-04-25T14:06:44.538447+0000","last_became_active":"2024-04-25T14:06:44.533189+0000","last_became_peered":"2024-04-25T14:06:44.533189+0000","last_unstale":"2024-04-25T14:06:44.538447+0000","last_undegraded":"2024-04-25T14:06:44.538447+0000","last_fullsized":"2024-04-25T14:06:44.538447+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:27.614025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.534844+0000","last_change":"2024-04-25T14:06:44.534844+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.534844+0000","last_undegraded":"2024-04-25T14:06:44.534844+0000","last_fullsized":"2024-04-25T14:06:44.534844+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.547834+0000","last_change":"2024-04-25T14:06:44.544582+0000","last_active":"2024-04-25T14:06:44.547834+0000","last_peered":"2024-04-25T14:06:44.547834+0000","last_clean":"2024-04-25T14:06:44.547834+0000","last_became_active":"2024-04-25T14:06:44.544049+0000","last_became_peered":"2024-04-25T14:06:44.544049+0000","last_unstale":"2024-04-25T14:06:44.547834+0000","last_undegraded":"2024-04-25T14:06:44.547834+0000","last_fullsized":"2024-04-25T14:06:44.547834+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:22.780792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.529499+0000","last_change":"2024-04-25T14:06:44.529499+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.529499+0000","last_undegraded":"2024-04-25T14:06:44.529499+0000","last_fullsized":"2024-04-25T14:06:44.529499+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.542053+0000","last_change":"2024-04-25T14:06:44.534032+0000","last_active":"2024-04-25T14:06:44.542053+0000","last_peered":"2024-04-25T14:06:44.542053+0000","last_clean":"2024-04-25T14:06:44.542053+0000","last_became_active":"2024-04-25T14:06:44.533673+0000","last_became_peered":"2024-04-25T14:06:44.533673+0000","last_unstale":"2024-04-25T14:06:44.542053+0000","last_undegraded":"2024-04-25T14:06:44.542053+0000","last_fullsized":"2024-04-25T14:06:44.542053+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:29.427107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.531944+0000","last_change":"2024-04-25T14:06:44.531944+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.531944+0000","last_undegraded":"2024-04-25T14:06:44.531944+0000","last_fullsized":"2024-04-25T14:06:44.531944+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.542699+0000","last_change":"2024-04-25T14:06:44.538705+0000","last_active":"2024-04-25T14:06:44.542699+0000","last_peered":"2024-04-25T14:06:44.542699+0000","last_clean":"2024-04-25T14:06:44.542699+0000","last_became_active":"2024-04-25T14:06:44.538404+0000","last_became_peered":"2024-04-25T14:06:44.538404+0000","last_unstale":"2024-04-25T14:06:44.542699+0000","last_undegraded":"2024-04-25T14:06:44.542699+0000","last_fullsized":"2024-04-25T14:06:44.542699+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:16.314477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.518861+0000","last_change":"2024-04-25T14:06:44.518861+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.518861+0000","last_undegraded":"2024-04-25T14:06:44.518861+0000","last_fullsized":"2024-04-25T14:06:44.518861+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.550286+0000","last_change":"2024-04-25T14:06:44.547295+0000","last_active":"2024-04-25T14:06:44.550286+0000","last_peered":"2024-04-25T14:06:44.550286+0000","last_clean":"2024-04-25T14:06:44.550286+0000","last_became_active":"2024-04-25T14:06:44.546930+0000","last_became_peered":"2024-04-25T14:06:44.546930+0000","last_unstale":"2024-04-25T14:06:44.550286+0000","last_undegraded":"2024-04-25T14:06:44.550286+0000","last_fullsized":"2024-04-25T14:06:44.550286+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:46.781154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.523760+0000","last_change":"2024-04-25T14:06:44.523760+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.523760+0000","last_undegraded":"2024-04-25T14:06:44.523760+0000","last_fullsized":"2024-04-25T14:06:44.523760+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.546025+0000","last_change":"2024-04-25T14:06:44.543006+0000","last_active":"2024-04-25T14:06:44.546025+0000","last_peered":"2024-04-25T14:06:44.546025+0000","last_clean":"2024-04-25T14:06:44.546025+0000","last_became_active":"2024-04-25T14:06:44.542678+0000","last_became_peered":"2024-04-25T14:06:44.542678+0000","last_unstale":"2024-04-25T14:06:44.546025+0000","last_undegraded":"2024-04-25T14:06:44.546025+0000","last_fullsized":"2024-04-25T14:06:44.546025+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:16.282476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525734+0000","last_change":"2024-04-25T14:06:44.525734+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525734+0000","last_undegraded":"2024-04-25T14:06:44.525734+0000","last_fullsized":"2024-04-25T14:06:44.525734+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.544904+0000","last_change":"2024-04-25T14:06:44.541885+0000","last_active":"2024-04-25T14:06:44.544904+0000","last_peered":"2024-04-25T14:06:44.544904+0000","last_clean":"2024-04-25T14:06:44.544904+0000","last_became_active":"2024-04-25T14:06:44.541501+0000","last_became_peered":"2024-04-25T14:06:44.541501+0000","last_unstale":"2024-04-25T14:06:44.544904+0000","last_undegraded":"2024-04-25T14:06:44.544904+0000","last_fullsized":"2024-04-25T14:06:44.544904+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:44.424458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.519094+0000","last_change":"2024-04-25T14:06:44.519094+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.519094+0000","last_undegraded":"2024-04-25T14:06:44.519094+0000","last_fullsized":"2024-04-25T14:06:44.519094+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.546591+0000","last_change":"2024-04-25T14:06:44.544634+0000","last_active":"2024-04-25T14:06:44.546591+0000","last_peered":"2024-04-25T14:06:44.546591+0000","last_clean":"2024-04-25T14:06:44.546591+0000","last_became_active":"2024-04-25T14:06:44.544365+0000","last_became_peered":"2024-04-25T14:06:44.544365+0000","last_unstale":"2024-04-25T14:06:44.546591+0000","last_undegraded":"2024-04-25T14:06:44.546591+0000","last_fullsized":"2024-04-25T14:06:44.546591+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:54:29.024591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.521205+0000","last_change":"2024-04-25T14:06:44.521205+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.521205+0000","last_undegraded":"2024-04-25T14:06:44.521205+0000","last_fullsized":"2024-04-25T14:06:44.521205+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.543827+0000","last_change":"2024-04-25T14:06:44.541617+0000","last_active":"2024-04-25T14:06:44.543827+0000","last_peered":"2024-04-25T14:06:44.543827+0000","last_clean":"2024-04-25T14:06:44.543827+0000","last_became_active":"2024-04-25T14:06:44.540854+0000","last_became_peered":"2024-04-25T14:06:44.540854+0000","last_unstale":"2024-04-25T14:06:44.543827+0000","last_undegraded":"2024-04-25T14:06:44.543827+0000","last_fullsized":"2024-04-25T14:06:44.543827+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:14.236144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.520875+0000","last_change":"2024-04-25T14:06:44.520875+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.520875+0000","last_undegraded":"2024-04-25T14:06:44.520875+0000","last_fullsized":"2024-04-25T14:06:44.520875+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.543972+0000","last_change":"2024-04-25T14:06:44.537769+0000","last_active":"2024-04-25T14:06:44.543972+0000","last_peered":"2024-04-25T14:06:44.543972+0000","last_clean":"2024-04-25T14:06:44.543972+0000","last_became_active":"2024-04-25T14:06:44.537407+0000","last_became_peered":"2024-04-25T14:06:44.537407+0000","last_unstale":"2024-04-25T14:06:44.543972+0000","last_undegraded":"2024-04-25T14:06:44.543972+0000","last_fullsized":"2024-04-25T14:06:44.543972+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:58.086378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.529722+0000","last_change":"2024-04-25T14:06:44.529722+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.529722+0000","last_undegraded":"2024-04-25T14:06:44.529722+0000","last_fullsized":"2024-04-25T14:06:44.529722+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.542491+0000","last_change":"2024-04-25T14:06:44.538743+0000","last_active":"2024-04-25T14:06:44.542491+0000","last_peered":"2024-04-25T14:06:44.542491+0000","last_clean":"2024-04-25T14:06:44.542491+0000","last_became_active":"2024-04-25T14:06:44.538365+0000","last_became_peered":"2024-04-25T14:06:44.538365+0000","last_unstale":"2024-04-25T14:06:44.542491+0000","last_undegraded":"2024-04-25T14:06:44.542491+0000","last_fullsized":"2024-04-25T14:06:44.542491+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:47.507460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.524176+0000","last_change":"2024-04-25T14:06:44.524176+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.524176+0000","last_undegraded":"2024-04-25T14:06:44.524176+0000","last_fullsized":"2024-04-25T14:06:44.524176+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.547650+0000","last_change":"2024-04-25T14:06:44.541609+0000","last_active":"2024-04-25T14:06:44.547650+0000","last_peered":"2024-04-25T14:06:44.547650+0000","last_clean":"2024-04-25T14:06:44.547650+0000","last_became_active":"2024-04-25T14:06:44.541009+0000","last_became_peered":"2024-04-25T14:06:44.541009+0000","last_unstale":"2024-04-25T14:06:44.547650+0000","last_undegraded":"2024-04-25T14:06:44.547650+0000","last_fullsized":"2024-04-25T14:06:44.547650+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:58.970595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.529423+0000","last_change":"2024-04-25T14:06:44.529423+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.529423+0000","last_undegraded":"2024-04-25T14:06:44.529423+0000","last_fullsized":"2024-04-25T14:06:44.529423+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.539705+0000","last_change":"2024-04-25T14:06:44.535705+0000","last_active":"2024-04-25T14:06:44.539705+0000","last_peered":"2024-04-25T14:06:44.539705+0000","last_clean":"2024-04-25T14:06:44.539705+0000","last_became_active":"2024-04-25T14:06:44.535457+0000","last_became_peered":"2024-04-25T14:06:44.535457+0000","last_unstale":"2024-04-25T14:06:44.539705+0000","last_undegraded":"2024-04-25T14:06:44.539705+0000","last_fullsized":"2024-04-25T14:06:44.539705+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:06.121310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.523979+0000","last_change":"2024-04-25T14:06:44.523979+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.523979+0000","last_undegraded":"2024-04-25T14:06:44.523979+0000","last_fullsized":"2024-04-25T14:06:44.523979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.544953+0000","last_change":"2024-04-25T14:06:44.542180+0000","last_active":"2024-04-25T14:06:44.544953+0000","last_peered":"2024-04-25T14:06:44.544953+0000","last_clean":"2024-04-25T14:06:44.544953+0000","last_became_active":"2024-04-25T14:06:44.540654+0000","last_became_peered":"2024-04-25T14:06:44.540654+0000","last_unstale":"2024-04-25T14:06:44.544953+0000","last_undegraded":"2024-04-25T14:06:44.544953+0000","last_fullsized":"2024-04-25T14:06:44.544953+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:58.970595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.520299+0000","last_change":"2024-04-25T14:06:44.520299+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.520299+0000","last_undegraded":"2024-04-25T14:06:44.520299+0000","last_fullsized":"2024-04-25T14:06:44.520299+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.547611+0000","last_change":"2024-04-25T14:06:44.543878+0000","last_active":"2024-04-25T14:06:44.547611+0000","last_peered":"2024-04-25T14:06:44.547611+0000","last_clean":"2024-04-25T14:06:44.547611+0000","last_became_active":"2024-04-25T14:06:44.543347+0000","last_became_peered":"2024-04-25T14:06:44.543347+0000","last_unstale":"2024-04-25T14:06:44.547611+0000","last_undegraded":"2024-04-25T14:06:44.547611+0000","last_fullsized":"2024-04-25T14:06:44.547611+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:56.180081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.532473+0000","last_change":"2024-04-25T14:06:44.532473+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.532473+0000","last_undegraded":"2024-04-25T14:06:44.532473+0000","last_fullsized":"2024-04-25T14:06:44.532473+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.550348+0000","last_change":"2024-04-25T14:06:44.546730+0000","last_active":"2024-04-25T14:06:44.550348+0000","last_peered":"2024-04-25T14:06:44.550348+0000","last_clean":"2024-04-25T14:06:44.550348+0000","last_became_active":"2024-04-25T14:06:44.546050+0000","last_became_peered":"2024-04-25T14:06:44.546050+0000","last_unstale":"2024-04-25T14:06:44.550348+0000","last_undegraded":"2024-04-25T14:06:44.550348+0000","last_fullsized":"2024-04-25T14:06:44.550348+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:58:21.045855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.531523+0000","last_change":"2024-04-25T14:06:44.531523+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.531523+0000","last_undegraded":"2024-04-25T14:06:44.531523+0000","last_fullsized":"2024-04-25T14:06:44.531523+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.540642+0000","last_change":"2024-04-25T14:06:44.534134+0000","last_active":"2024-04-25T14:06:44.540642+0000","last_peered":"2024-04-25T14:06:44.540642+0000","last_clean":"2024-04-25T14:06:44.540642+0000","last_became_active":"2024-04-25T14:06:44.533705+0000","last_became_peered":"2024-04-25T14:06:44.533705+0000","last_unstale":"2024-04-25T14:06:44.540642+0000","last_undegraded":"2024-04-25T14:06:44.540642+0000","last_fullsized":"2024-04-25T14:06:44.540642+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:43.875733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.535223+0000","last_change":"2024-04-25T14:06:44.535223+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.535223+0000","last_undegraded":"2024-04-25T14:06:44.535223+0000","last_fullsized":"2024-04-25T14:06:44.535223+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.545851+0000","last_change":"2024-04-25T14:06:44.538191+0000","last_active":"2024-04-25T14:06:44.545851+0000","last_peered":"2024-04-25T14:06:44.545851+0000","last_clean":"2024-04-25T14:06:44.545851+0000","last_became_active":"2024-04-25T14:06:44.537838+0000","last_became_peered":"2024-04-25T14:06:44.537838+0000","last_unstale":"2024-04-25T14:06:44.545851+0000","last_undegraded":"2024-04-25T14:06:44.545851+0000","last_fullsized":"2024-04-25T14:06:44.545851+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:43.249499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.532357+0000","last_change":"2024-04-25T14:06:44.532357+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.532357+0000","last_undegraded":"2024-04-25T14:06:44.532357+0000","last_fullsized":"2024-04-25T14:06:44.532357+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.543260+0000","last_change":"2024-04-25T14:06:44.539538+0000","last_active":"2024-04-25T14:06:44.543260+0000","last_peered":"2024-04-25T14:06:44.543260+0000","last_clean":"2024-04-25T14:06:44.543260+0000","last_became_active":"2024-04-25T14:06:44.538128+0000","last_became_peered":"2024-04-25T14:06:44.538128+0000","last_unstale":"2024-04-25T14:06:44.543260+0000","last_undegraded":"2024-04-25T14:06:44.543260+0000","last_fullsized":"2024-04-25T14:06:44.543260+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:56.433866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525676+0000","last_change":"2024-04-25T14:06:44.525676+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525676+0000","last_undegraded":"2024-04-25T14:06:44.525676+0000","last_fullsized":"2024-04-25T14:06:44.525676+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.542537+0000","last_change":"2024-04-25T14:06:44.539206+0000","last_active":"2024-04-25T14:06:44.542537+0000","last_peered":"2024-04-25T14:06:44.542537+0000","last_clean":"2024-04-25T14:06:44.542537+0000","last_became_active":"2024-04-25T14:06:44.538913+0000","last_became_peered":"2024-04-25T14:06:44.538913+0000","last_unstale":"2024-04-25T14:06:44.542537+0000","last_undegraded":"2024-04-25T14:06:44.542537+0000","last_fullsized":"2024-04-25T14:06:44.542537+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:09.304323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"59'86","reported_seq":255,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.525284+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:06:44.525284+0000","last_peered":"2024-04-25T14:06:44.525284+0000","last_clean":"2024-04-25T14:06:44.525284+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:06:44.525284+0000","last_undegraded":"2024-04-25T14:06:44.525284+0000","last_fullsized":"2024-04-25T14:06:44.525284+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1474560,"data_stored":1023970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":433602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":880640,"data_stored":433602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":433602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":433602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":433602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":433602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1474560,"data_stored":1023970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:06:50.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3f is not active+clean 2024-04-25T14:06:50.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.528048+0000', 'last_change': '2024-04-25T14:06:44.528048+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.528048+0000', 'last_undegraded': '2024-04-25T14:06:44.528048+0000', 'last_fullsized': '2024-04-25T14:06:44.528048+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:06:50.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3e is not active+clean 2024-04-25T14:06:50.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.522967+0000', 'last_change': '2024-04-25T14:06:44.522967+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.522967+0000', 'last_undegraded': '2024-04-25T14:06:44.522967+0000', 'last_fullsized': '2024-04-25T14:06:44.522967+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:06:50.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3d is not active+clean 2024-04-25T14:06:50.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.530589+0000', 'last_change': '2024-04-25T14:06:44.530589+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.530589+0000', 'last_undegraded': '2024-04-25T14:06:44.530589+0000', 'last_fullsized': '2024-04-25T14:06:44.530589+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:06:50.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3c is not active+clean 2024-04-25T14:06:50.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.521588+0000', 'last_change': '2024-04-25T14:06:44.521588+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.521588+0000', 'last_undegraded': '2024-04-25T14:06:44.521588+0000', 'last_fullsized': '2024-04-25T14:06:44.521588+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:06:50.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3b is not active+clean 2024-04-25T14:06:50.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.529696+0000', 'last_change': '2024-04-25T14:06:44.529696+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.529696+0000', 'last_undegraded': '2024-04-25T14:06:44.529696+0000', 'last_fullsized': '2024-04-25T14:06:44.529696+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:06:50.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3a is not active+clean 2024-04-25T14:06:50.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.520096+0000', 'last_change': '2024-04-25T14:06:44.520096+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.520096+0000', 'last_undegraded': '2024-04-25T14:06:44.520096+0000', 'last_fullsized': '2024-04-25T14:06:44.520096+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:06:50.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.39 is not active+clean 2024-04-25T14:06:50.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.532154+0000', 'last_change': '2024-04-25T14:06:44.532154+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.532154+0000', 'last_undegraded': '2024-04-25T14:06:44.532154+0000', 'last_fullsized': '2024-04-25T14:06:44.532154+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:06:50.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.38 is not active+clean 2024-04-25T14:06:50.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.524922+0000', 'last_change': '2024-04-25T14:06:44.524922+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.524922+0000', 'last_undegraded': '2024-04-25T14:06:44.524922+0000', 'last_fullsized': '2024-04-25T14:06:44.524922+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:06:50.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.37 is not active+clean 2024-04-25T14:06:50.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.525831+0000', 'last_change': '2024-04-25T14:06:44.525831+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.525831+0000', 'last_undegraded': '2024-04-25T14:06:44.525831+0000', 'last_fullsized': '2024-04-25T14:06:44.525831+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:06:50.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.36 is not active+clean 2024-04-25T14:06:50.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.522110+0000', 'last_change': '2024-04-25T14:06:44.522110+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.522110+0000', 'last_undegraded': '2024-04-25T14:06:44.522110+0000', 'last_fullsized': '2024-04-25T14:06:44.522110+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:06:50.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.35 is not active+clean 2024-04-25T14:06:50.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.524249+0000', 'last_change': '2024-04-25T14:06:44.524249+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.524249+0000', 'last_undegraded': '2024-04-25T14:06:44.524249+0000', 'last_fullsized': '2024-04-25T14:06:44.524249+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:06:50.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.34 is not active+clean 2024-04-25T14:06:50.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.520624+0000', 'last_change': '2024-04-25T14:06:44.520624+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.520624+0000', 'last_undegraded': '2024-04-25T14:06:44.520624+0000', 'last_fullsized': '2024-04-25T14:06:44.520624+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:06:50.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.33 is not active+clean 2024-04-25T14:06:50.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.524668+0000', 'last_change': '2024-04-25T14:06:44.524668+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.524668+0000', 'last_undegraded': '2024-04-25T14:06:44.524668+0000', 'last_fullsized': '2024-04-25T14:06:44.524668+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:06:50.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.32 is not active+clean 2024-04-25T14:06:50.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.519647+0000', 'last_change': '2024-04-25T14:06:44.519647+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.519647+0000', 'last_undegraded': '2024-04-25T14:06:44.519647+0000', 'last_fullsized': '2024-04-25T14:06:44.519647+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:06:50.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.31 is not active+clean 2024-04-25T14:06:50.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.521818+0000', 'last_change': '2024-04-25T14:06:44.521818+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.521818+0000', 'last_undegraded': '2024-04-25T14:06:44.521818+0000', 'last_fullsized': '2024-04-25T14:06:44.521818+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:06:50.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.30 is not active+clean 2024-04-25T14:06:50.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.524664+0000', 'last_change': '2024-04-25T14:06:44.524664+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.524664+0000', 'last_undegraded': '2024-04-25T14:06:44.524664+0000', 'last_fullsized': '2024-04-25T14:06:44.524664+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:06:50.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2f is not active+clean 2024-04-25T14:06:50.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.526595+0000', 'last_change': '2024-04-25T14:06:44.526595+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.526595+0000', 'last_undegraded': '2024-04-25T14:06:44.526595+0000', 'last_fullsized': '2024-04-25T14:06:44.526595+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:06:50.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2e is not active+clean 2024-04-25T14:06:50.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.535681+0000', 'last_change': '2024-04-25T14:06:44.535681+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.535681+0000', 'last_undegraded': '2024-04-25T14:06:44.535681+0000', 'last_fullsized': '2024-04-25T14:06:44.535681+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:06:50.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2d is not active+clean 2024-04-25T14:06:50.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.519158+0000', 'last_change': '2024-04-25T14:06:44.519158+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.519158+0000', 'last_undegraded': '2024-04-25T14:06:44.519158+0000', 'last_fullsized': '2024-04-25T14:06:44.519158+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:06:50.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2c is not active+clean 2024-04-25T14:06:50.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.524178+0000', 'last_change': '2024-04-25T14:06:44.524178+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.524178+0000', 'last_undegraded': '2024-04-25T14:06:44.524178+0000', 'last_fullsized': '2024-04-25T14:06:44.524178+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:06:50.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2b is not active+clean 2024-04-25T14:06:50.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.525944+0000', 'last_change': '2024-04-25T14:06:44.525944+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.525944+0000', 'last_undegraded': '2024-04-25T14:06:44.525944+0000', 'last_fullsized': '2024-04-25T14:06:44.525944+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:06:50.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2a is not active+clean 2024-04-25T14:06:50.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.525179+0000', 'last_change': '2024-04-25T14:06:44.525179+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.525179+0000', 'last_undegraded': '2024-04-25T14:06:44.525179+0000', 'last_fullsized': '2024-04-25T14:06:44.525179+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:06:50.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.29 is not active+clean 2024-04-25T14:06:50.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.523722+0000', 'last_change': '2024-04-25T14:06:44.523722+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.523722+0000', 'last_undegraded': '2024-04-25T14:06:44.523722+0000', 'last_fullsized': '2024-04-25T14:06:44.523722+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:06:50.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.28 is not active+clean 2024-04-25T14:06:50.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.521337+0000', 'last_change': '2024-04-25T14:06:44.521337+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.521337+0000', 'last_undegraded': '2024-04-25T14:06:44.521337+0000', 'last_fullsized': '2024-04-25T14:06:44.521337+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:06:50.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.27 is not active+clean 2024-04-25T14:06:50.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.520503+0000', 'last_change': '2024-04-25T14:06:44.520503+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.520503+0000', 'last_undegraded': '2024-04-25T14:06:44.520503+0000', 'last_fullsized': '2024-04-25T14:06:44.520503+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:06:50.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.26 is not active+clean 2024-04-25T14:06:50.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.520274+0000', 'last_change': '2024-04-25T14:06:44.520274+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.520274+0000', 'last_undegraded': '2024-04-25T14:06:44.520274+0000', 'last_fullsized': '2024-04-25T14:06:44.520274+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:06:50.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.25 is not active+clean 2024-04-25T14:06:50.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.526216+0000', 'last_change': '2024-04-25T14:06:44.526216+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.526216+0000', 'last_undegraded': '2024-04-25T14:06:44.526216+0000', 'last_fullsized': '2024-04-25T14:06:44.526216+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:06:50.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.24 is not active+clean 2024-04-25T14:06:50.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.523614+0000', 'last_change': '2024-04-25T14:06:44.523614+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.523614+0000', 'last_undegraded': '2024-04-25T14:06:44.523614+0000', 'last_fullsized': '2024-04-25T14:06:44.523614+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:06:50.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.23 is not active+clean 2024-04-25T14:06:50.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.521807+0000', 'last_change': '2024-04-25T14:06:44.521807+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.521807+0000', 'last_undegraded': '2024-04-25T14:06:44.521807+0000', 'last_fullsized': '2024-04-25T14:06:44.521807+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:06:50.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.22 is not active+clean 2024-04-25T14:06:50.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.527187+0000', 'last_change': '2024-04-25T14:06:44.527187+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.527187+0000', 'last_undegraded': '2024-04-25T14:06:44.527187+0000', 'last_fullsized': '2024-04-25T14:06:44.527187+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:06:50.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.21 is not active+clean 2024-04-25T14:06:50.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.528762+0000', 'last_change': '2024-04-25T14:06:44.528762+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.528762+0000', 'last_undegraded': '2024-04-25T14:06:44.528762+0000', 'last_fullsized': '2024-04-25T14:06:44.528762+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:06:50.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.20 is not active+clean 2024-04-25T14:06:50.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.524006+0000', 'last_change': '2024-04-25T14:06:44.524006+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.524006+0000', 'last_undegraded': '2024-04-25T14:06:44.524006+0000', 'last_fullsized': '2024-04-25T14:06:44.524006+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:06:50.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.18 is not active+clean 2024-04-25T14:06:50.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.529058+0000', 'last_change': '2024-04-25T14:06:44.529058+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.529058+0000', 'last_undegraded': '2024-04-25T14:06:44.529058+0000', 'last_fullsized': '2024-04-25T14:06:44.529058+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:06:50.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.19 is not active+clean 2024-04-25T14:06:50.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.521384+0000', 'last_change': '2024-04-25T14:06:44.521384+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.521384+0000', 'last_undegraded': '2024-04-25T14:06:44.521384+0000', 'last_fullsized': '2024-04-25T14:06:44.521384+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:06:50.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1a is not active+clean 2024-04-25T14:06:50.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.525629+0000', 'last_change': '2024-04-25T14:06:44.525629+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.525629+0000', 'last_undegraded': '2024-04-25T14:06:44.525629+0000', 'last_fullsized': '2024-04-25T14:06:44.525629+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:06:50.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1b is not active+clean 2024-04-25T14:06:50.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.525353+0000', 'last_change': '2024-04-25T14:06:44.525353+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.525353+0000', 'last_undegraded': '2024-04-25T14:06:44.525353+0000', 'last_fullsized': '2024-04-25T14:06:44.525353+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:06:50.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1c is not active+clean 2024-04-25T14:06:50.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.522329+0000', 'last_change': '2024-04-25T14:06:44.522329+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.522329+0000', 'last_undegraded': '2024-04-25T14:06:44.522329+0000', 'last_fullsized': '2024-04-25T14:06:44.522329+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:06:50.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1d is not active+clean 2024-04-25T14:06:50.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.523772+0000', 'last_change': '2024-04-25T14:06:44.523772+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.523772+0000', 'last_undegraded': '2024-04-25T14:06:44.523772+0000', 'last_fullsized': '2024-04-25T14:06:44.523772+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:06:50.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1e is not active+clean 2024-04-25T14:06:50.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.525151+0000', 'last_change': '2024-04-25T14:06:44.525151+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.525151+0000', 'last_undegraded': '2024-04-25T14:06:44.525151+0000', 'last_fullsized': '2024-04-25T14:06:44.525151+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:06:50.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1f is not active+clean 2024-04-25T14:06:50.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.536745+0000', 'last_change': '2024-04-25T14:06:44.536745+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.536745+0000', 'last_undegraded': '2024-04-25T14:06:44.536745+0000', 'last_fullsized': '2024-04-25T14:06:44.536745+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:06:50.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.10 is not active+clean 2024-04-25T14:06:50.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.520415+0000', 'last_change': '2024-04-25T14:06:44.520415+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.520415+0000', 'last_undegraded': '2024-04-25T14:06:44.520415+0000', 'last_fullsized': '2024-04-25T14:06:44.520415+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:06:50.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.11 is not active+clean 2024-04-25T14:06:50.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.519850+0000', 'last_change': '2024-04-25T14:06:44.519850+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.519850+0000', 'last_undegraded': '2024-04-25T14:06:44.519850+0000', 'last_fullsized': '2024-04-25T14:06:44.519850+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:06:50.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.12 is not active+clean 2024-04-25T14:06:50.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.519140+0000', 'last_change': '2024-04-25T14:06:44.519140+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.519140+0000', 'last_undegraded': '2024-04-25T14:06:44.519140+0000', 'last_fullsized': '2024-04-25T14:06:44.519140+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:06:50.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.13 is not active+clean 2024-04-25T14:06:50.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.529532+0000', 'last_change': '2024-04-25T14:06:44.529532+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.529532+0000', 'last_undegraded': '2024-04-25T14:06:44.529532+0000', 'last_fullsized': '2024-04-25T14:06:44.529532+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:06:50.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.14 is not active+clean 2024-04-25T14:06:50.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.519818+0000', 'last_change': '2024-04-25T14:06:44.519818+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.519818+0000', 'last_undegraded': '2024-04-25T14:06:44.519818+0000', 'last_fullsized': '2024-04-25T14:06:44.519818+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:06:50.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.15 is not active+clean 2024-04-25T14:06:50.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.534844+0000', 'last_change': '2024-04-25T14:06:44.534844+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.534844+0000', 'last_undegraded': '2024-04-25T14:06:44.534844+0000', 'last_fullsized': '2024-04-25T14:06:44.534844+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:06:50.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.16 is not active+clean 2024-04-25T14:06:50.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.529499+0000', 'last_change': '2024-04-25T14:06:44.529499+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.529499+0000', 'last_undegraded': '2024-04-25T14:06:44.529499+0000', 'last_fullsized': '2024-04-25T14:06:44.529499+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:06:50.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.17 is not active+clean 2024-04-25T14:06:50.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.531944+0000', 'last_change': '2024-04-25T14:06:44.531944+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.531944+0000', 'last_undegraded': '2024-04-25T14:06:44.531944+0000', 'last_fullsized': '2024-04-25T14:06:44.531944+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:06:50.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.8 is not active+clean 2024-04-25T14:06:50.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.518861+0000', 'last_change': '2024-04-25T14:06:44.518861+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.518861+0000', 'last_undegraded': '2024-04-25T14:06:44.518861+0000', 'last_fullsized': '2024-04-25T14:06:44.518861+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:06:50.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.9 is not active+clean 2024-04-25T14:06:50.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.523760+0000', 'last_change': '2024-04-25T14:06:44.523760+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.523760+0000', 'last_undegraded': '2024-04-25T14:06:44.523760+0000', 'last_fullsized': '2024-04-25T14:06:44.523760+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:06:50.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.a is not active+clean 2024-04-25T14:06:50.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.525734+0000', 'last_change': '2024-04-25T14:06:44.525734+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.525734+0000', 'last_undegraded': '2024-04-25T14:06:44.525734+0000', 'last_fullsized': '2024-04-25T14:06:44.525734+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:06:50.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.b is not active+clean 2024-04-25T14:06:50.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.519094+0000', 'last_change': '2024-04-25T14:06:44.519094+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.519094+0000', 'last_undegraded': '2024-04-25T14:06:44.519094+0000', 'last_fullsized': '2024-04-25T14:06:44.519094+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:06:50.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.c is not active+clean 2024-04-25T14:06:50.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.521205+0000', 'last_change': '2024-04-25T14:06:44.521205+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.521205+0000', 'last_undegraded': '2024-04-25T14:06:44.521205+0000', 'last_fullsized': '2024-04-25T14:06:44.521205+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:06:50.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.e is not active+clean 2024-04-25T14:06:50.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.520875+0000', 'last_change': '2024-04-25T14:06:44.520875+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.520875+0000', 'last_undegraded': '2024-04-25T14:06:44.520875+0000', 'last_fullsized': '2024-04-25T14:06:44.520875+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:06:50.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.f is not active+clean 2024-04-25T14:06:50.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.529722+0000', 'last_change': '2024-04-25T14:06:44.529722+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.529722+0000', 'last_undegraded': '2024-04-25T14:06:44.529722+0000', 'last_fullsized': '2024-04-25T14:06:44.529722+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:06:50.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.0 is not active+clean 2024-04-25T14:06:50.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.524176+0000', 'last_change': '2024-04-25T14:06:44.524176+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.524176+0000', 'last_undegraded': '2024-04-25T14:06:44.524176+0000', 'last_fullsized': '2024-04-25T14:06:44.524176+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:06:50.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1 is not active+clean 2024-04-25T14:06:50.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.529423+0000', 'last_change': '2024-04-25T14:06:44.529423+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.529423+0000', 'last_undegraded': '2024-04-25T14:06:44.529423+0000', 'last_fullsized': '2024-04-25T14:06:44.529423+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:06:50.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2 is not active+clean 2024-04-25T14:06:50.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.523979+0000', 'last_change': '2024-04-25T14:06:44.523979+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.523979+0000', 'last_undegraded': '2024-04-25T14:06:44.523979+0000', 'last_fullsized': '2024-04-25T14:06:44.523979+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:06:50.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3 is not active+clean 2024-04-25T14:06:50.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.520299+0000', 'last_change': '2024-04-25T14:06:44.520299+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.520299+0000', 'last_undegraded': '2024-04-25T14:06:44.520299+0000', 'last_fullsized': '2024-04-25T14:06:44.520299+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:06:50.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.4 is not active+clean 2024-04-25T14:06:50.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.532473+0000', 'last_change': '2024-04-25T14:06:44.532473+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.532473+0000', 'last_undegraded': '2024-04-25T14:06:44.532473+0000', 'last_fullsized': '2024-04-25T14:06:44.532473+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:06:50.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.5 is not active+clean 2024-04-25T14:06:50.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.531523+0000', 'last_change': '2024-04-25T14:06:44.531523+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.531523+0000', 'last_undegraded': '2024-04-25T14:06:44.531523+0000', 'last_fullsized': '2024-04-25T14:06:44.531523+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:06:50.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.6 is not active+clean 2024-04-25T14:06:50.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.535223+0000', 'last_change': '2024-04-25T14:06:44.535223+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.535223+0000', 'last_undegraded': '2024-04-25T14:06:44.535223+0000', 'last_fullsized': '2024-04-25T14:06:44.535223+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:06:50.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.7 is not active+clean 2024-04-25T14:06:50.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.532357+0000', 'last_change': '2024-04-25T14:06:44.532357+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.532357+0000', 'last_undegraded': '2024-04-25T14:06:44.532357+0000', 'last_fullsized': '2024-04-25T14:06:44.532357+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:06:50.492 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.d is not active+clean 2024-04-25T14:06:50.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:06:44.525676+0000', 'last_change': '2024-04-25T14:06:44.525676+0000', 'last_active': '2024-04-25T14:06:44.508104+0000', 'last_peered': '2024-04-25T14:06:44.508104+0000', 'last_clean': '2024-04-25T14:06:44.508104+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:06:44.525676+0000', 'last_undegraded': '2024-04-25T14:06:44.525676+0000', 'last_fullsized': '2024-04-25T14:06:44.525676+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'last_clean_scrub_stamp': '2024-04-25T14:06:44.508104+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:06:50.492 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T14:06:50.496 INFO:tasks.ceph.mds.a.smithi086.stderr:2024-04-25T14:06:50.495+0000 7f4bf8000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T14:06:50.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.639+0000 7fe43f200640 1 Processor -- start 2024-04-25T14:06:50.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.639+0000 7fe43f200640 1 -- start start 2024-04-25T14:06:50.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.643+0000 7fe43f200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe4380ced30 0x7fe4380b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.643+0000 7fe43f200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe43815dcd0 0x7fe4380d1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.643+0000 7fe43f200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe438161b30 con 0x7fe4380d1970 2024-04-25T14:06:50.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.643+0000 7fe43f200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe4381618e0 con 0x7fe4380ced30 2024-04-25T14:06:50.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.643+0000 7fe43f200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe438161e10 con 0x7fe43815dcd0 2024-04-25T14:06:50.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.643+0000 7fe43d400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe43815dcd0 0x7fe4380d1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.643+0000 7fe43de00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe4380ced30 0x7fe4380b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.643+0000 7fe43de00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe4380ced30 0x7fe4380b1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:43226/0 (socket says 172.21.15.86:43226) 2024-04-25T14:06:50.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.643+0000 7fe43e800640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fe4380d1970 0x7fe43815e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:44970/0 (socket says 172.21.15.86:44970) 2024-04-25T14:06:50.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.643+0000 7fe43d400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe43815dcd0 0x7fe4380d1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:45902/0 (socket says 172.21.15.86:45902) 2024-04-25T14:06:50.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.643+0000 7fe43de00640 1 -- 172.21.15.86:0/2170691631 learned_addr learned my addr 172.21.15.86:0/2170691631 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:50.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.643+0000 7fe43ca00640 1 -- 172.21.15.86:0/2170691631 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2075107286 0 0) 0x7fe438161b30 con 0x7fe4380d1970 2024-04-25T14:06:50.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.643+0000 7fe43ca00640 1 -- 172.21.15.86:0/2170691631 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe420003530 con 0x7fe4380d1970 2024-04-25T14:06:50.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.643+0000 7fe43d400640 1 -- 172.21.15.86:0/2170691631 >> v2:172.21.15.155:3300/0 conn(0x7fe4380ced30 msgr2=0x7fe4380b1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:50.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.643+0000 7fe43d400640 1 --2- 172.21.15.86:0/2170691631 >> v2:172.21.15.155:3300/0 conn(0x7fe4380ced30 0x7fe4380b1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.643+0000 7fe43d400640 1 -- 172.21.15.86:0/2170691631 >> v1:172.21.15.155:6789/0 conn(0x7fe4380d1970 legacy=0x7fe43815e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.643+0000 7fe43d400640 1 -- 172.21.15.86:0/2170691631 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4381620f0 con 0x7fe43815dcd0 2024-04-25T14:06:50.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.643+0000 7fe43de00640 1 --2- 172.21.15.86:0/2170691631 >> v2:172.21.15.155:3300/0 conn(0x7fe4380ced30 0x7fe4380b1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:50.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.647+0000 7fe43d400640 1 --2- 172.21.15.86:0/2170691631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe43815dcd0 0x7fe4380d1430 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe4280097c0 tx=0x7fe42801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d94a7e19ee18d53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:50.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.647+0000 7fe43ca00640 1 -- 172.21.15.86:0/2170691631 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4280ab650 con 0x7fe43815dcd0 2024-04-25T14:06:50.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.647+0000 7fe43ca00640 1 -- 172.21.15.86:0/2170691631 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4280b0070 con 0x7fe43815dcd0 2024-04-25T14:06:50.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.647+0000 7fe43ca00640 1 -- 172.21.15.86:0/2170691631 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4280aba10 con 0x7fe43815dcd0 2024-04-25T14:06:50.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.647+0000 7fe43f200640 1 -- 172.21.15.86:0/2170691631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe43815dcd0 msgr2=0x7fe4380d1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.647+0000 7fe43f200640 1 --2- 172.21.15.86:0/2170691631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe43815dcd0 0x7fe4380d1430 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe4280097c0 tx=0x7fe42801c470 comp rx=0 tx=0).stop 2024-04-25T14:06:50.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.647+0000 7fe43f200640 1 -- 172.21.15.86:0/2170691631 shutdown_connections 2024-04-25T14:06:50.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.647+0000 7fe43f200640 1 --2- 172.21.15.86:0/2170691631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe43815dcd0 0x7fe4380d1430 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.647+0000 7fe43f200640 1 --2- 172.21.15.86:0/2170691631 >> v2:172.21.15.155:3300/0 conn(0x7fe4380ced30 0x7fe4380b1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.647+0000 7fe43f200640 1 -- 172.21.15.86:0/2170691631 >> 172.21.15.86:0/2170691631 conn(0x7fe43809ba30 msgr2=0x7fe4380b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:50.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.647+0000 7fe43f200640 1 -- 172.21.15.86:0/2170691631 shutdown_connections 2024-04-25T14:06:50.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.647+0000 7fe43f200640 1 -- 172.21.15.86:0/2170691631 wait complete. 2024-04-25T14:06:50.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.651+0000 7fe43f200640 1 Processor -- start 2024-04-25T14:06:50.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.651+0000 7fe43f200640 1 -- start start 2024-04-25T14:06:50.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.651+0000 7fe43f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4380ced30 0x7fe438148fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.651+0000 7fe43f200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4380d1970 0x7fe4380b0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.651+0000 7fe43f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe43815dcd0 0x7fe4381588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.651+0000 7fe43f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe43816aa70 con 0x7fe43815dcd0 2024-04-25T14:06:50.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.651+0000 7fe43f200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe4380d1d80 con 0x7fe4380ced30 2024-04-25T14:06:50.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.651+0000 7fe43f200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe43816abf0 con 0x7fe4380d1970 2024-04-25T14:06:50.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.651+0000 7fe43e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe43815dcd0 0x7fe4381588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.651+0000 7fe43e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe43815dcd0 0x7fe4381588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56476/0 (socket says 172.21.15.86:56476) 2024-04-25T14:06:50.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.651+0000 7fe43e800640 1 -- 172.21.15.86:0/1490751463 learned_addr learned my addr 172.21.15.86:0/1490751463 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:50.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.651+0000 7fe43d400640 1 --2- 172.21.15.86:0/1490751463 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4380d1970 0x7fe4380b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.651+0000 7fe43de00640 1 --2- 172.21.15.86:0/1490751463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4380ced30 0x7fe438148fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.651+0000 7fe43e800640 1 -- 172.21.15.86:0/1490751463 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4380d1970 msgr2=0x7fe4380b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.651+0000 7fe43e800640 1 --2- 172.21.15.86:0/1490751463 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4380d1970 0x7fe4380b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.651+0000 7fe43e800640 1 -- 172.21.15.86:0/1490751463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4380ced30 msgr2=0x7fe438148fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.651+0000 7fe43e800640 1 --2- 172.21.15.86:0/1490751463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4380ced30 0x7fe438148fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.651+0000 7fe43e800640 1 -- 172.21.15.86:0/1490751463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4380b09e0 con 0x7fe43815dcd0 2024-04-25T14:06:50.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.651+0000 7fe43e800640 1 --2- 172.21.15.86:0/1490751463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe43815dcd0 0x7fe4381588a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe43400c7a0 tx=0x7fe43400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:50.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.655+0000 7fe432000640 1 -- 172.21.15.86:0/1490751463 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe434017070 con 0x7fe43815dcd0 2024-04-25T14:06:50.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.655+0000 7fe432000640 1 -- 172.21.15.86:0/1490751463 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe43400a640 con 0x7fe43815dcd0 2024-04-25T14:06:50.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.655+0000 7fe432000640 1 -- 172.21.15.86:0/1490751463 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43400a9f0 con 0x7fe43815dcd0 2024-04-25T14:06:50.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.655+0000 7fe43f200640 1 -- 172.21.15.86:0/1490751463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4380b0cc0 con 0x7fe43815dcd0 2024-04-25T14:06:50.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.655+0000 7fe43f200640 1 -- 172.21.15.86:0/1490751463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4380b1280 con 0x7fe43815dcd0 2024-04-25T14:06:50.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.655+0000 7fe43f200640 1 -- 172.21.15.86:0/1490751463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4380b0730 con 0x7fe43815dcd0 2024-04-25T14:06:50.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.655+0000 7fe432000640 1 -- 172.21.15.86:0/1490751463 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7fe43401a070 con 0x7fe43815dcd0 2024-04-25T14:06:50.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.655+0000 7fe432000640 1 --2- 172.21.15.86:0/1490751463 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fe41c06f670 0x7fe41c071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.655+0000 7fe432000640 1 -- 172.21.15.86:0/1490751463 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7fe43408ed70 con 0x7fe43815dcd0 2024-04-25T14:06:50.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.655+0000 7fe43de00640 1 --2- 172.21.15.86:0/1490751463 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fe41c06f670 0x7fe41c071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.659+0000 7fe43de00640 1 --2- 172.21.15.86:0/1490751463 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fe41c06f670 0x7fe41c071b80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe42c006cd0 tx=0x7fe42c009240 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:50.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.663+0000 7fe432000640 1 -- 172.21.15.86:0/1490751463 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe43405b1a0 con 0x7fe43815dcd0 2024-04-25T14:06:50.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.875+0000 7fe43f200640 1 -- 172.21.15.86:0/1490751463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fe438076b80 con 0x7fe43815dcd0 2024-04-25T14:06:50.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.875+0000 7fe432000640 1 -- 172.21.15.86:0/1490751463 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+3229 (secure 0 0 0) 0x7fe43405f4a0 con 0x7fe43815dcd0 2024-04-25T14:06:50.877 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:50.878 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":205,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":74,"num_osds":8,"num_up_osds":8,"osd_up_since":1714053818,"num_in_osds":8,"osd_in_since":1714053808,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":259432448,"bytes_avail":772834680832,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":53,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"a","status":"up:active","gid":6603}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-04-25T14:06:33.434688+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T14:06:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.875+0000 7fe43f200640 1 -- 172.21.15.86:0/1490751463 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fe41c06f670 msgr2=0x7fe41c071b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.875+0000 7fe43f200640 1 --2- 172.21.15.86:0/1490751463 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fe41c06f670 0x7fe41c071b80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe42c006cd0 tx=0x7fe42c009240 comp rx=0 tx=0).stop 2024-04-25T14:06:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.875+0000 7fe43f200640 1 -- 172.21.15.86:0/1490751463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe43815dcd0 msgr2=0x7fe4381588a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.875+0000 7fe43f200640 1 --2- 172.21.15.86:0/1490751463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe43815dcd0 0x7fe4381588a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe43400c7a0 tx=0x7fe43400cc60 comp rx=0 tx=0).stop 2024-04-25T14:06:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.875+0000 7fe43f200640 1 -- 172.21.15.86:0/1490751463 shutdown_connections 2024-04-25T14:06:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.875+0000 7fe43f200640 1 --2- 172.21.15.86:0/1490751463 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fe41c06f670 0x7fe41c071b80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.875+0000 7fe43f200640 1 --2- 172.21.15.86:0/1490751463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe43815dcd0 0x7fe4381588a0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.875+0000 7fe43f200640 1 --2- 172.21.15.86:0/1490751463 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4380d1970 0x7fe4380b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.875+0000 7fe43f200640 1 --2- 172.21.15.86:0/1490751463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4380ced30 0x7fe438148fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.875+0000 7fe43f200640 1 -- 172.21.15.86:0/1490751463 >> 172.21.15.86:0/1490751463 conn(0x7fe43809ba30 msgr2=0x7fe43815add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.875+0000 7fe43f200640 1 -- 172.21.15.86:0/1490751463 shutdown_connections 2024-04-25T14:06:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:50.875+0000 7fe43f200640 1 -- 172.21.15.86:0/1490751463 wait complete. 2024-04-25T14:06:50.898 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T14:06:50.898 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:06:51.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.043+0000 7f8c05400640 1 Processor -- start 2024-04-25T14:06:51.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.043+0000 7f8c05400640 1 -- start start 2024-04-25T14:06:51.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.043+0000 7f8c05400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8c00159d40 0x7f8c0015d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.043+0000 7f8c05400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8c0006c670 0x7f8c0006ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.043+0000 7f8c05400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c0015d5f0 con 0x7f8c0015fc20 2024-04-25T14:06:51.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.043+0000 7f8c05400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8c0006cfc0 con 0x7f8c00159d40 2024-04-25T14:06:51.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.043+0000 7f8c05400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f8c0006ea40 con 0x7f8c0006c670 2024-04-25T14:06:51.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.047+0000 7f8bffe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8c00159d40 0x7f8c0015d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.047+0000 7f8c04a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8c0015fc20 0x7f8c00160030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51728/0 (socket says 172.21.15.86:51728) 2024-04-25T14:06:51.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.047+0000 7f8c04a00640 1 -- 172.21.15.86:0/2527548125 learned_addr learned my addr 172.21.15.86:0/2527548125 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:51.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.047+0000 7f8bffe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8c00159d40 0x7f8c0015d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56480/0 (socket says 172.21.15.86:56480) 2024-04-25T14:06:51.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.047+0000 7f8bf7400640 1 --2- 172.21.15.86:0/2527548125 >> v2:172.21.15.155:3300/0 conn(0x7f8c0006c670 0x7f8c0006ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.047+0000 7f8bffe00640 1 -- 172.21.15.86:0/2527548125 >> v2:172.21.15.155:3300/0 conn(0x7f8c0006c670 msgr2=0x7f8c0006ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:51.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.047+0000 7f8bffe00640 1 --2- 172.21.15.86:0/2527548125 >> v2:172.21.15.155:3300/0 conn(0x7f8c0006c670 0x7f8c0006ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.047+0000 7f8bffe00640 1 -- 172.21.15.86:0/2527548125 >> v1:172.21.15.86:6789/0 conn(0x7f8c0015fc20 legacy=0x7f8c00160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.047+0000 7f8bffe00640 1 -- 172.21.15.86:0/2527548125 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0006ed20 con 0x7f8c00159d40 2024-04-25T14:06:51.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.047+0000 7f8bff400640 1 -- 172.21.15.86:0/2527548125 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 31124219 0 0) 0x7f8bf80014a0 con 0x7f8c0015fc20 2024-04-25T14:06:51.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.047+0000 7f8bffe00640 1 --2- 172.21.15.86:0/2527548125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c00159d40 0x7f8c0015d0b0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8c00072810 tx=0x7f8bf001d7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3b8147505ba7e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:51.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.047+0000 7f8bff400640 1 -- 172.21.15.86:0/2527548125 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf00b3070 con 0x7f8c00159d40 2024-04-25T14:06:51.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.047+0000 7f8bff400640 1 -- 172.21.15.86:0/2527548125 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf00a6760 con 0x7f8c00159d40 2024-04-25T14:06:51.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.047+0000 7f8bff400640 1 -- 172.21.15.86:0/2527548125 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf000b040 con 0x7f8c00159d40 2024-04-25T14:06:51.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.051+0000 7f8c05400640 1 -- 172.21.15.86:0/2527548125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c00159d40 msgr2=0x7f8c0015d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.051+0000 7f8c05400640 1 --2- 172.21.15.86:0/2527548125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c00159d40 0x7f8c0015d0b0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8c00072810 tx=0x7f8bf001d7c0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.051+0000 7f8c05400640 1 -- 172.21.15.86:0/2527548125 shutdown_connections 2024-04-25T14:06:51.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.051+0000 7f8c05400640 1 --2- 172.21.15.86:0/2527548125 >> v2:172.21.15.155:3300/0 conn(0x7f8c0006c670 0x7f8c0006ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.051+0000 7f8c05400640 1 --2- 172.21.15.86:0/2527548125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c00159d40 0x7f8c0015d0b0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.051+0000 7f8c05400640 1 -- 172.21.15.86:0/2527548125 >> 172.21.15.86:0/2527548125 conn(0x7f8c000b4850 msgr2=0x7f8c000b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:51.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.051+0000 7f8c05400640 1 -- 172.21.15.86:0/2527548125 shutdown_connections 2024-04-25T14:06:51.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.051+0000 7f8c05400640 1 -- 172.21.15.86:0/2527548125 wait complete. 2024-04-25T14:06:51.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.051+0000 7f8c05400640 1 Processor -- start 2024-04-25T14:06:51.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.051+0000 7f8c05400640 1 -- start start 2024-04-25T14:06:51.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.055+0000 7f8c05400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c0006c670 0x7f8c0009bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.055+0000 7f8c05400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c00159d40 0x7f8c00146530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.055+0000 7f8c05400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c0015fc20 0x7f8c00156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.055+0000 7f8c05400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c00169140 con 0x7f8c0006c670 2024-04-25T14:06:51.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.055+0000 7f8c05400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8c00168fc0 con 0x7f8c00159d40 2024-04-25T14:06:51.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.055+0000 7f8c05400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8c001692c0 con 0x7f8c0015fc20 2024-04-25T14:06:51.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.055+0000 7f8bffe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c0006c670 0x7f8c0009bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.055+0000 7f8bffe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c0006c670 0x7f8c0009bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56488/0 (socket says 172.21.15.86:56488) 2024-04-25T14:06:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.055+0000 7f8bffe00640 1 -- 172.21.15.86:0/4116777042 learned_addr learned my addr 172.21.15.86:0/4116777042 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.055+0000 7f8bffe00640 1 -- 172.21.15.86:0/4116777042 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c0015fc20 msgr2=0x7f8c00156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.055+0000 7f8bffe00640 1 --2- 172.21.15.86:0/4116777042 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c0015fc20 0x7f8c00156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.055+0000 7f8bffe00640 1 -- 172.21.15.86:0/4116777042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c00159d40 msgr2=0x7f8c00146530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.055+0000 7f8bffe00640 1 --2- 172.21.15.86:0/4116777042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c00159d40 0x7f8c00146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.055+0000 7f8bffe00640 1 -- 172.21.15.86:0/4116777042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0009e520 con 0x7f8c0006c670 2024-04-25T14:06:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.055+0000 7f8bffe00640 1 --2- 172.21.15.86:0/4116777042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c0006c670 0x7f8c0009bc70 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00b1040 tx=0x7f8bf001a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.055+0000 7f8bfcc00640 1 -- 172.21.15.86:0/4116777042 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf00b3040 con 0x7f8c0006c670 2024-04-25T14:06:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.055+0000 7f8bfcc00640 1 -- 172.21.15.86:0/4116777042 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf00bc070 con 0x7f8c0006c670 2024-04-25T14:06:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.055+0000 7f8bfcc00640 1 -- 172.21.15.86:0/4116777042 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf00ae660 con 0x7f8c0006c670 2024-04-25T14:06:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.055+0000 7f8c05400640 1 -- 172.21.15.86:0/4116777042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c0009e800 con 0x7f8c0006c670 2024-04-25T14:06:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.055+0000 7f8c05400640 1 -- 172.21.15.86:0/4116777042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c0009edc0 con 0x7f8c0006c670 2024-04-25T14:06:51.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.055+0000 7f8c05400640 1 -- 172.21.15.86:0/4116777042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c00076b80 con 0x7f8c0006c670 2024-04-25T14:06:51.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.055+0000 7f8bfcc00640 1 -- 172.21.15.86:0/4116777042 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f8bf00ae800 con 0x7f8c0006c670 2024-04-25T14:06:51.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.059+0000 7f8bfcc00640 1 --2- 172.21.15.86:0/4116777042 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8bdc06f670 0x7f8bdc071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.059+0000 7f8bfcc00640 1 -- 172.21.15.86:0/4116777042 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f8bf01292f0 con 0x7f8c0006c670 2024-04-25T14:06:51.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.059+0000 7f8bf7400640 1 --2- 172.21.15.86:0/4116777042 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8bdc06f670 0x7f8bdc071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.059+0000 7f8bf7400640 1 --2- 172.21.15.86:0/4116777042 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8bdc06f670 0x7f8bdc071b80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8be8006aa0 tx=0x7f8be8006a30 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:51.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.063+0000 7f8bfcc00640 1 -- 172.21.15.86:0/4116777042 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8bf00c5060 con 0x7f8c0006c670 2024-04-25T14:06:51.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.215+0000 7f8c05400640 1 -- 172.21.15.86:0/4116777042 --> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8c0009c4a0 con 0x7f8bdc06f670 2024-04-25T14:06:51.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.219+0000 7f8bfcc00640 1 -- 172.21.15.86:0/4116777042 <== mgr.6198 v2:172.21.15.86:6832/1025829557 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246744 (secure 0 0 0) 0x7f8be800e7a0 con 0x7f8bdc06f670 2024-04-25T14:06:51.225 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:51.227 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:06:51.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.223+0000 7f8c05400640 1 -- 172.21.15.86:0/4116777042 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8bdc06f670 msgr2=0x7f8bdc071b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.223+0000 7f8c05400640 1 --2- 172.21.15.86:0/4116777042 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8bdc06f670 0x7f8bdc071b80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8be8006aa0 tx=0x7f8be8006a30 comp rx=0 tx=0).stop 2024-04-25T14:06:51.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.223+0000 7f8c05400640 1 -- 172.21.15.86:0/4116777042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c0006c670 msgr2=0x7f8c0009bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.223+0000 7f8c05400640 1 --2- 172.21.15.86:0/4116777042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c0006c670 0x7f8c0009bc70 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00b1040 tx=0x7f8bf001a560 comp rx=0 tx=0).stop 2024-04-25T14:06:51.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.223+0000 7f8c05400640 1 -- 172.21.15.86:0/4116777042 shutdown_connections 2024-04-25T14:06:51.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.223+0000 7f8c05400640 1 --2- 172.21.15.86:0/4116777042 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8bdc06f670 0x7f8bdc071b80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.223+0000 7f8c05400640 1 --2- 172.21.15.86:0/4116777042 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c0015fc20 0x7f8c00156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.223+0000 7f8c05400640 1 --2- 172.21.15.86:0/4116777042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c00159d40 0x7f8c00146530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.223+0000 7f8c05400640 1 --2- 172.21.15.86:0/4116777042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c0006c670 0x7f8c0009bc70 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.223+0000 7f8c05400640 1 -- 172.21.15.86:0/4116777042 >> 172.21.15.86:0/4116777042 conn(0x7f8c000b4850 msgr2=0x7f8c00160030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:51.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.223+0000 7f8c05400640 1 -- 172.21.15.86:0/4116777042 shutdown_connections 2024-04-25T14:06:51.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:51.223+0000 7f8c05400640 1 -- 172.21.15.86:0/4116777042 wait complete. 2024-04-25T14:06:51.246 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-04-25T14:06:49.438066+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253352,"kb_used_data":8040,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721368,"statfs":{"total":773094113280,"available":772834680832,"internally_reserved":0,"allocated":8232960,"data_stored":4649552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.927238"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.528048+0000","last_change":"2024-04-25T14:06:44.528048+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.528048+0000","last_undegraded":"2024-04-25T14:06:44.528048+0000","last_fullsized":"2024-04-25T14:06:44.528048+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.522967+0000","last_change":"2024-04-25T14:06:44.522967+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.522967+0000","last_undegraded":"2024-04-25T14:06:44.522967+0000","last_fullsized":"2024-04-25T14:06:44.522967+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.530589+0000","last_change":"2024-04-25T14:06:44.530589+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.530589+0000","last_undegraded":"2024-04-25T14:06:44.530589+0000","last_fullsized":"2024-04-25T14:06:44.530589+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.521588+0000","last_change":"2024-04-25T14:06:44.521588+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.521588+0000","last_undegraded":"2024-04-25T14:06:44.521588+0000","last_fullsized":"2024-04-25T14:06:44.521588+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.529696+0000","last_change":"2024-04-25T14:06:44.529696+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.529696+0000","last_undegraded":"2024-04-25T14:06:44.529696+0000","last_fullsized":"2024-04-25T14:06:44.529696+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.520096+0000","last_change":"2024-04-25T14:06:44.520096+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.520096+0000","last_undegraded":"2024-04-25T14:06:44.520096+0000","last_fullsized":"2024-04-25T14:06:44.520096+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.532154+0000","last_change":"2024-04-25T14:06:44.532154+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.532154+0000","last_undegraded":"2024-04-25T14:06:44.532154+0000","last_fullsized":"2024-04-25T14:06:44.532154+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.524922+0000","last_change":"2024-04-25T14:06:44.524922+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.524922+0000","last_undegraded":"2024-04-25T14:06:44.524922+0000","last_fullsized":"2024-04-25T14:06:44.524922+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525831+0000","last_change":"2024-04-25T14:06:44.525831+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525831+0000","last_undegraded":"2024-04-25T14:06:44.525831+0000","last_fullsized":"2024-04-25T14:06:44.525831+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.522110+0000","last_change":"2024-04-25T14:06:44.522110+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.522110+0000","last_undegraded":"2024-04-25T14:06:44.522110+0000","last_fullsized":"2024-04-25T14:06:44.522110+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.524249+0000","last_change":"2024-04-25T14:06:44.524249+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.524249+0000","last_undegraded":"2024-04-25T14:06:44.524249+0000","last_fullsized":"2024-04-25T14:06:44.524249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.520624+0000","last_change":"2024-04-25T14:06:44.520624+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.520624+0000","last_undegraded":"2024-04-25T14:06:44.520624+0000","last_fullsized":"2024-04-25T14:06:44.520624+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.524668+0000","last_change":"2024-04-25T14:06:44.524668+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.524668+0000","last_undegraded":"2024-04-25T14:06:44.524668+0000","last_fullsized":"2024-04-25T14:06:44.524668+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.519647+0000","last_change":"2024-04-25T14:06:44.519647+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.519647+0000","last_undegraded":"2024-04-25T14:06:44.519647+0000","last_fullsized":"2024-04-25T14:06:44.519647+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.521818+0000","last_change":"2024-04-25T14:06:44.521818+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.521818+0000","last_undegraded":"2024-04-25T14:06:44.521818+0000","last_fullsized":"2024-04-25T14:06:44.521818+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.524664+0000","last_change":"2024-04-25T14:06:44.524664+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.524664+0000","last_undegraded":"2024-04-25T14:06:44.524664+0000","last_fullsized":"2024-04-25T14:06:44.524664+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.526595+0000","last_change":"2024-04-25T14:06:44.526595+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.526595+0000","last_undegraded":"2024-04-25T14:06:44.526595+0000","last_fullsized":"2024-04-25T14:06:44.526595+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.535681+0000","last_change":"2024-04-25T14:06:44.535681+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.535681+0000","last_undegraded":"2024-04-25T14:06:44.535681+0000","last_fullsized":"2024-04-25T14:06:44.535681+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.519158+0000","last_change":"2024-04-25T14:06:44.519158+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.519158+0000","last_undegraded":"2024-04-25T14:06:44.519158+0000","last_fullsized":"2024-04-25T14:06:44.519158+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.524178+0000","last_change":"2024-04-25T14:06:44.524178+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.524178+0000","last_undegraded":"2024-04-25T14:06:44.524178+0000","last_fullsized":"2024-04-25T14:06:44.524178+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525944+0000","last_change":"2024-04-25T14:06:44.525944+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525944+0000","last_undegraded":"2024-04-25T14:06:44.525944+0000","last_fullsized":"2024-04-25T14:06:44.525944+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525179+0000","last_change":"2024-04-25T14:06:44.525179+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525179+0000","last_undegraded":"2024-04-25T14:06:44.525179+0000","last_fullsized":"2024-04-25T14:06:44.525179+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.523722+0000","last_change":"2024-04-25T14:06:44.523722+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.523722+0000","last_undegraded":"2024-04-25T14:06:44.523722+0000","last_fullsized":"2024-04-25T14:06:44.523722+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.521337+0000","last_change":"2024-04-25T14:06:44.521337+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.521337+0000","last_undegraded":"2024-04-25T14:06:44.521337+0000","last_fullsized":"2024-04-25T14:06:44.521337+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.520503+0000","last_change":"2024-04-25T14:06:44.520503+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.520503+0000","last_undegraded":"2024-04-25T14:06:44.520503+0000","last_fullsized":"2024-04-25T14:06:44.520503+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.520274+0000","last_change":"2024-04-25T14:06:44.520274+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.520274+0000","last_undegraded":"2024-04-25T14:06:44.520274+0000","last_fullsized":"2024-04-25T14:06:44.520274+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.526216+0000","last_change":"2024-04-25T14:06:44.526216+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.526216+0000","last_undegraded":"2024-04-25T14:06:44.526216+0000","last_fullsized":"2024-04-25T14:06:44.526216+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.523614+0000","last_change":"2024-04-25T14:06:44.523614+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.523614+0000","last_undegraded":"2024-04-25T14:06:44.523614+0000","last_fullsized":"2024-04-25T14:06:44.523614+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.521807+0000","last_change":"2024-04-25T14:06:44.521807+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.521807+0000","last_undegraded":"2024-04-25T14:06:44.521807+0000","last_fullsized":"2024-04-25T14:06:44.521807+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.527187+0000","last_change":"2024-04-25T14:06:44.527187+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.527187+0000","last_undegraded":"2024-04-25T14:06:44.527187+0000","last_fullsized":"2024-04-25T14:06:44.527187+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.528762+0000","last_change":"2024-04-25T14:06:44.528762+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.528762+0000","last_undegraded":"2024-04-25T14:06:44.528762+0000","last_fullsized":"2024-04-25T14:06:44.528762+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.524006+0000","last_change":"2024-04-25T14:06:44.524006+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.524006+0000","last_undegraded":"2024-04-25T14:06:44.524006+0000","last_fullsized":"2024-04-25T14:06:44.524006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.529058+0000","last_change":"2024-04-25T14:06:44.529058+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.529058+0000","last_undegraded":"2024-04-25T14:06:44.529058+0000","last_fullsized":"2024-04-25T14:06:44.529058+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.544431+0000","last_change":"2024-04-25T14:06:44.538430+0000","last_active":"2024-04-25T14:06:44.544431+0000","last_peered":"2024-04-25T14:06:44.544431+0000","last_clean":"2024-04-25T14:06:44.544431+0000","last_became_active":"2024-04-25T14:06:44.538081+0000","last_became_peered":"2024-04-25T14:06:44.538081+0000","last_unstale":"2024-04-25T14:06:44.544431+0000","last_undegraded":"2024-04-25T14:06:44.544431+0000","last_fullsized":"2024-04-25T14:06:44.544431+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:19.980060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.521384+0000","last_change":"2024-04-25T14:06:44.521384+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.521384+0000","last_undegraded":"2024-04-25T14:06:44.521384+0000","last_fullsized":"2024-04-25T14:06:44.521384+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.547856+0000","last_change":"2024-04-25T14:06:44.544725+0000","last_active":"2024-04-25T14:06:44.547856+0000","last_peered":"2024-04-25T14:06:44.547856+0000","last_clean":"2024-04-25T14:06:44.547856+0000","last_became_active":"2024-04-25T14:06:44.544154+0000","last_became_peered":"2024-04-25T14:06:44.544154+0000","last_unstale":"2024-04-25T14:06:44.547856+0000","last_undegraded":"2024-04-25T14:06:44.547856+0000","last_fullsized":"2024-04-25T14:06:44.547856+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:27.974323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525629+0000","last_change":"2024-04-25T14:06:44.525629+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525629+0000","last_undegraded":"2024-04-25T14:06:44.525629+0000","last_fullsized":"2024-04-25T14:06:44.525629+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.544093+0000","last_change":"2024-04-25T14:06:44.537708+0000","last_active":"2024-04-25T14:06:44.544093+0000","last_peered":"2024-04-25T14:06:44.544093+0000","last_clean":"2024-04-25T14:06:44.544093+0000","last_became_active":"2024-04-25T14:06:44.537349+0000","last_became_peered":"2024-04-25T14:06:44.537349+0000","last_unstale":"2024-04-25T14:06:44.544093+0000","last_undegraded":"2024-04-25T14:06:44.544093+0000","last_fullsized":"2024-04-25T14:06:44.544093+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:14.448012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525353+0000","last_change":"2024-04-25T14:06:44.525353+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525353+0000","last_undegraded":"2024-04-25T14:06:44.525353+0000","last_fullsized":"2024-04-25T14:06:44.525353+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.550329+0000","last_change":"2024-04-25T14:06:44.546446+0000","last_active":"2024-04-25T14:06:44.550329+0000","last_peered":"2024-04-25T14:06:44.550329+0000","last_clean":"2024-04-25T14:06:44.550329+0000","last_became_active":"2024-04-25T14:06:44.546076+0000","last_became_peered":"2024-04-25T14:06:44.546076+0000","last_unstale":"2024-04-25T14:06:44.550329+0000","last_undegraded":"2024-04-25T14:06:44.550329+0000","last_fullsized":"2024-04-25T14:06:44.550329+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:03.415461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.522329+0000","last_change":"2024-04-25T14:06:44.522329+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.522329+0000","last_undegraded":"2024-04-25T14:06:44.522329+0000","last_fullsized":"2024-04-25T14:06:44.522329+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.544176+0000","last_change":"2024-04-25T14:06:44.538629+0000","last_active":"2024-04-25T14:06:44.544176+0000","last_peered":"2024-04-25T14:06:44.544176+0000","last_clean":"2024-04-25T14:06:44.544176+0000","last_became_active":"2024-04-25T14:06:44.538329+0000","last_became_peered":"2024-04-25T14:06:44.538329+0000","last_unstale":"2024-04-25T14:06:44.544176+0000","last_undegraded":"2024-04-25T14:06:44.544176+0000","last_fullsized":"2024-04-25T14:06:44.544176+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:02.112409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.523772+0000","last_change":"2024-04-25T14:06:44.523772+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.523772+0000","last_undegraded":"2024-04-25T14:06:44.523772+0000","last_fullsized":"2024-04-25T14:06:44.523772+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.534286+0000","last_change":"2024-04-25T14:06:44.530649+0000","last_active":"2024-04-25T14:06:44.534286+0000","last_peered":"2024-04-25T14:06:44.534286+0000","last_clean":"2024-04-25T14:06:44.534286+0000","last_became_active":"2024-04-25T14:06:44.530025+0000","last_became_peered":"2024-04-25T14:06:44.530025+0000","last_unstale":"2024-04-25T14:06:44.534286+0000","last_undegraded":"2024-04-25T14:06:44.534286+0000","last_fullsized":"2024-04-25T14:06:44.534286+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:34.160672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525151+0000","last_change":"2024-04-25T14:06:44.525151+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525151+0000","last_undegraded":"2024-04-25T14:06:44.525151+0000","last_fullsized":"2024-04-25T14:06:44.525151+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.542594+0000","last_change":"2024-04-25T14:06:44.537817+0000","last_active":"2024-04-25T14:06:44.542594+0000","last_peered":"2024-04-25T14:06:44.542594+0000","last_clean":"2024-04-25T14:06:44.542594+0000","last_became_active":"2024-04-25T14:06:44.537198+0000","last_became_peered":"2024-04-25T14:06:44.537198+0000","last_unstale":"2024-04-25T14:06:44.542594+0000","last_undegraded":"2024-04-25T14:06:44.542594+0000","last_fullsized":"2024-04-25T14:06:44.542594+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:42.088259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.536745+0000","last_change":"2024-04-25T14:06:44.536745+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.536745+0000","last_undegraded":"2024-04-25T14:06:44.536745+0000","last_fullsized":"2024-04-25T14:06:44.536745+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.544663+0000","last_change":"2024-04-25T14:06:44.539464+0000","last_active":"2024-04-25T14:06:44.544663+0000","last_peered":"2024-04-25T14:06:44.544663+0000","last_clean":"2024-04-25T14:06:44.544663+0000","last_became_active":"2024-04-25T14:06:44.539110+0000","last_became_peered":"2024-04-25T14:06:44.539110+0000","last_unstale":"2024-04-25T14:06:44.544663+0000","last_undegraded":"2024-04-25T14:06:44.544663+0000","last_fullsized":"2024-04-25T14:06:44.544663+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:41.171166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.520415+0000","last_change":"2024-04-25T14:06:44.520415+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.520415+0000","last_undegraded":"2024-04-25T14:06:44.520415+0000","last_fullsized":"2024-04-25T14:06:44.520415+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.552725+0000","last_change":"2024-04-25T14:06:44.548815+0000","last_active":"2024-04-25T14:06:44.552725+0000","last_peered":"2024-04-25T14:06:44.552725+0000","last_clean":"2024-04-25T14:06:44.552725+0000","last_became_active":"2024-04-25T14:06:44.548469+0000","last_became_peered":"2024-04-25T14:06:44.548469+0000","last_unstale":"2024-04-25T14:06:44.552725+0000","last_undegraded":"2024-04-25T14:06:44.552725+0000","last_fullsized":"2024-04-25T14:06:44.552725+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:24.243932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.519850+0000","last_change":"2024-04-25T14:06:44.519850+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.519850+0000","last_undegraded":"2024-04-25T14:06:44.519850+0000","last_fullsized":"2024-04-25T14:06:44.519850+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.543141+0000","last_change":"2024-04-25T14:06:44.540795+0000","last_active":"2024-04-25T14:06:44.543141+0000","last_peered":"2024-04-25T14:06:44.543141+0000","last_clean":"2024-04-25T14:06:44.543141+0000","last_became_active":"2024-04-25T14:06:44.540467+0000","last_became_peered":"2024-04-25T14:06:44.540467+0000","last_unstale":"2024-04-25T14:06:44.543141+0000","last_undegraded":"2024-04-25T14:06:44.543141+0000","last_fullsized":"2024-04-25T14:06:44.543141+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:14.448012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.519140+0000","last_change":"2024-04-25T14:06:44.519140+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.519140+0000","last_undegraded":"2024-04-25T14:06:44.519140+0000","last_fullsized":"2024-04-25T14:06:44.519140+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.546703+0000","last_change":"2024-04-25T14:06:44.544689+0000","last_active":"2024-04-25T14:06:44.546703+0000","last_peered":"2024-04-25T14:06:44.546703+0000","last_clean":"2024-04-25T14:06:44.546703+0000","last_became_active":"2024-04-25T14:06:44.544095+0000","last_became_peered":"2024-04-25T14:06:44.544095+0000","last_unstale":"2024-04-25T14:06:44.546703+0000","last_undegraded":"2024-04-25T14:06:44.546703+0000","last_fullsized":"2024-04-25T14:06:44.546703+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:11.780439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.529532+0000","last_change":"2024-04-25T14:06:44.529532+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.529532+0000","last_undegraded":"2024-04-25T14:06:44.529532+0000","last_fullsized":"2024-04-25T14:06:44.529532+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.550353+0000","last_change":"2024-04-25T14:06:44.545874+0000","last_active":"2024-04-25T14:06:44.550353+0000","last_peered":"2024-04-25T14:06:44.550353+0000","last_clean":"2024-04-25T14:06:44.550353+0000","last_became_active":"2024-04-25T14:06:44.545363+0000","last_became_peered":"2024-04-25T14:06:44.545363+0000","last_unstale":"2024-04-25T14:06:44.550353+0000","last_undegraded":"2024-04-25T14:06:44.550353+0000","last_fullsized":"2024-04-25T14:06:44.550353+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:16.858125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.519818+0000","last_change":"2024-04-25T14:06:44.519818+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.519818+0000","last_undegraded":"2024-04-25T14:06:44.519818+0000","last_fullsized":"2024-04-25T14:06:44.519818+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.538447+0000","last_change":"2024-04-25T14:06:44.533506+0000","last_active":"2024-04-25T14:06:44.538447+0000","last_peered":"2024-04-25T14:06:44.538447+0000","last_clean":"2024-04-25T14:06:44.538447+0000","last_became_active":"2024-04-25T14:06:44.533189+0000","last_became_peered":"2024-04-25T14:06:44.533189+0000","last_unstale":"2024-04-25T14:06:44.538447+0000","last_undegraded":"2024-04-25T14:06:44.538447+0000","last_fullsized":"2024-04-25T14:06:44.538447+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:27.614025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.534844+0000","last_change":"2024-04-25T14:06:44.534844+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.534844+0000","last_undegraded":"2024-04-25T14:06:44.534844+0000","last_fullsized":"2024-04-25T14:06:44.534844+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.547834+0000","last_change":"2024-04-25T14:06:44.544582+0000","last_active":"2024-04-25T14:06:44.547834+0000","last_peered":"2024-04-25T14:06:44.547834+0000","last_clean":"2024-04-25T14:06:44.547834+0000","last_became_active":"2024-04-25T14:06:44.544049+0000","last_became_peered":"2024-04-25T14:06:44.544049+0000","last_unstale":"2024-04-25T14:06:44.547834+0000","last_undegraded":"2024-04-25T14:06:44.547834+0000","last_fullsized":"2024-04-25T14:06:44.547834+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:22.780792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.529499+0000","last_change":"2024-04-25T14:06:44.529499+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.529499+0000","last_undegraded":"2024-04-25T14:06:44.529499+0000","last_fullsized":"2024-04-25T14:06:44.529499+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.542053+0000","last_change":"2024-04-25T14:06:44.534032+0000","last_active":"2024-04-25T14:06:44.542053+0000","last_peered":"2024-04-25T14:06:44.542053+0000","last_clean":"2024-04-25T14:06:44.542053+0000","last_became_active":"2024-04-25T14:06:44.533673+0000","last_became_peered":"2024-04-25T14:06:44.533673+0000","last_unstale":"2024-04-25T14:06:44.542053+0000","last_undegraded":"2024-04-25T14:06:44.542053+0000","last_fullsized":"2024-04-25T14:06:44.542053+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:29.427107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.531944+0000","last_change":"2024-04-25T14:06:44.531944+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.531944+0000","last_undegraded":"2024-04-25T14:06:44.531944+0000","last_fullsized":"2024-04-25T14:06:44.531944+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.542699+0000","last_change":"2024-04-25T14:06:44.538705+0000","last_active":"2024-04-25T14:06:44.542699+0000","last_peered":"2024-04-25T14:06:44.542699+0000","last_clean":"2024-04-25T14:06:44.542699+0000","last_became_active":"2024-04-25T14:06:44.538404+0000","last_became_peered":"2024-04-25T14:06:44.538404+0000","last_unstale":"2024-04-25T14:06:44.542699+0000","last_undegraded":"2024-04-25T14:06:44.542699+0000","last_fullsized":"2024-04-25T14:06:44.542699+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:16.314477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.518861+0000","last_change":"2024-04-25T14:06:44.518861+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.518861+0000","last_undegraded":"2024-04-25T14:06:44.518861+0000","last_fullsized":"2024-04-25T14:06:44.518861+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.550286+0000","last_change":"2024-04-25T14:06:44.547295+0000","last_active":"2024-04-25T14:06:44.550286+0000","last_peered":"2024-04-25T14:06:44.550286+0000","last_clean":"2024-04-25T14:06:44.550286+0000","last_became_active":"2024-04-25T14:06:44.546930+0000","last_became_peered":"2024-04-25T14:06:44.546930+0000","last_unstale":"2024-04-25T14:06:44.550286+0000","last_undegraded":"2024-04-25T14:06:44.550286+0000","last_fullsized":"2024-04-25T14:06:44.550286+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:46.781154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.523760+0000","last_change":"2024-04-25T14:06:44.523760+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.523760+0000","last_undegraded":"2024-04-25T14:06:44.523760+0000","last_fullsized":"2024-04-25T14:06:44.523760+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.546025+0000","last_change":"2024-04-25T14:06:44.543006+0000","last_active":"2024-04-25T14:06:44.546025+0000","last_peered":"2024-04-25T14:06:44.546025+0000","last_clean":"2024-04-25T14:06:44.546025+0000","last_became_active":"2024-04-25T14:06:44.542678+0000","last_became_peered":"2024-04-25T14:06:44.542678+0000","last_unstale":"2024-04-25T14:06:44.546025+0000","last_undegraded":"2024-04-25T14:06:44.546025+0000","last_fullsized":"2024-04-25T14:06:44.546025+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:16.282476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525734+0000","last_change":"2024-04-25T14:06:44.525734+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525734+0000","last_undegraded":"2024-04-25T14:06:44.525734+0000","last_fullsized":"2024-04-25T14:06:44.525734+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.544904+0000","last_change":"2024-04-25T14:06:44.541885+0000","last_active":"2024-04-25T14:06:44.544904+0000","last_peered":"2024-04-25T14:06:44.544904+0000","last_clean":"2024-04-25T14:06:44.544904+0000","last_became_active":"2024-04-25T14:06:44.541501+0000","last_became_peered":"2024-04-25T14:06:44.541501+0000","last_unstale":"2024-04-25T14:06:44.544904+0000","last_undegraded":"2024-04-25T14:06:44.544904+0000","last_fullsized":"2024-04-25T14:06:44.544904+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:44.424458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.519094+0000","last_change":"2024-04-25T14:06:44.519094+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.519094+0000","last_undegraded":"2024-04-25T14:06:44.519094+0000","last_fullsized":"2024-04-25T14:06:44.519094+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.546591+0000","last_change":"2024-04-25T14:06:44.544634+0000","last_active":"2024-04-25T14:06:44.546591+0000","last_peered":"2024-04-25T14:06:44.546591+0000","last_clean":"2024-04-25T14:06:44.546591+0000","last_became_active":"2024-04-25T14:06:44.544365+0000","last_became_peered":"2024-04-25T14:06:44.544365+0000","last_unstale":"2024-04-25T14:06:44.546591+0000","last_undegraded":"2024-04-25T14:06:44.546591+0000","last_fullsized":"2024-04-25T14:06:44.546591+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:54:29.024591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.521205+0000","last_change":"2024-04-25T14:06:44.521205+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.521205+0000","last_undegraded":"2024-04-25T14:06:44.521205+0000","last_fullsized":"2024-04-25T14:06:44.521205+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.543827+0000","last_change":"2024-04-25T14:06:44.541617+0000","last_active":"2024-04-25T14:06:44.543827+0000","last_peered":"2024-04-25T14:06:44.543827+0000","last_clean":"2024-04-25T14:06:44.543827+0000","last_became_active":"2024-04-25T14:06:44.540854+0000","last_became_peered":"2024-04-25T14:06:44.540854+0000","last_unstale":"2024-04-25T14:06:44.543827+0000","last_undegraded":"2024-04-25T14:06:44.543827+0000","last_fullsized":"2024-04-25T14:06:44.543827+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:14.236144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.520875+0000","last_change":"2024-04-25T14:06:44.520875+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.520875+0000","last_undegraded":"2024-04-25T14:06:44.520875+0000","last_fullsized":"2024-04-25T14:06:44.520875+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.543972+0000","last_change":"2024-04-25T14:06:44.537769+0000","last_active":"2024-04-25T14:06:44.543972+0000","last_peered":"2024-04-25T14:06:44.543972+0000","last_clean":"2024-04-25T14:06:44.543972+0000","last_became_active":"2024-04-25T14:06:44.537407+0000","last_became_peered":"2024-04-25T14:06:44.537407+0000","last_unstale":"2024-04-25T14:06:44.543972+0000","last_undegraded":"2024-04-25T14:06:44.543972+0000","last_fullsized":"2024-04-25T14:06:44.543972+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:58.086378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.529722+0000","last_change":"2024-04-25T14:06:44.529722+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.529722+0000","last_undegraded":"2024-04-25T14:06:44.529722+0000","last_fullsized":"2024-04-25T14:06:44.529722+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.542491+0000","last_change":"2024-04-25T14:06:44.538743+0000","last_active":"2024-04-25T14:06:44.542491+0000","last_peered":"2024-04-25T14:06:44.542491+0000","last_clean":"2024-04-25T14:06:44.542491+0000","last_became_active":"2024-04-25T14:06:44.538365+0000","last_became_peered":"2024-04-25T14:06:44.538365+0000","last_unstale":"2024-04-25T14:06:44.542491+0000","last_undegraded":"2024-04-25T14:06:44.542491+0000","last_fullsized":"2024-04-25T14:06:44.542491+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:47.507460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.524176+0000","last_change":"2024-04-25T14:06:44.524176+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.524176+0000","last_undegraded":"2024-04-25T14:06:44.524176+0000","last_fullsized":"2024-04-25T14:06:44.524176+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.547650+0000","last_change":"2024-04-25T14:06:44.541609+0000","last_active":"2024-04-25T14:06:44.547650+0000","last_peered":"2024-04-25T14:06:44.547650+0000","last_clean":"2024-04-25T14:06:44.547650+0000","last_became_active":"2024-04-25T14:06:44.541009+0000","last_became_peered":"2024-04-25T14:06:44.541009+0000","last_unstale":"2024-04-25T14:06:44.547650+0000","last_undegraded":"2024-04-25T14:06:44.547650+0000","last_fullsized":"2024-04-25T14:06:44.547650+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:58.970595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.529423+0000","last_change":"2024-04-25T14:06:44.529423+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.529423+0000","last_undegraded":"2024-04-25T14:06:44.529423+0000","last_fullsized":"2024-04-25T14:06:44.529423+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.539705+0000","last_change":"2024-04-25T14:06:44.535705+0000","last_active":"2024-04-25T14:06:44.539705+0000","last_peered":"2024-04-25T14:06:44.539705+0000","last_clean":"2024-04-25T14:06:44.539705+0000","last_became_active":"2024-04-25T14:06:44.535457+0000","last_became_peered":"2024-04-25T14:06:44.535457+0000","last_unstale":"2024-04-25T14:06:44.539705+0000","last_undegraded":"2024-04-25T14:06:44.539705+0000","last_fullsized":"2024-04-25T14:06:44.539705+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:06.121310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.523979+0000","last_change":"2024-04-25T14:06:44.523979+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.523979+0000","last_undegraded":"2024-04-25T14:06:44.523979+0000","last_fullsized":"2024-04-25T14:06:44.523979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.544953+0000","last_change":"2024-04-25T14:06:44.542180+0000","last_active":"2024-04-25T14:06:44.544953+0000","last_peered":"2024-04-25T14:06:44.544953+0000","last_clean":"2024-04-25T14:06:44.544953+0000","last_became_active":"2024-04-25T14:06:44.540654+0000","last_became_peered":"2024-04-25T14:06:44.540654+0000","last_unstale":"2024-04-25T14:06:44.544953+0000","last_undegraded":"2024-04-25T14:06:44.544953+0000","last_fullsized":"2024-04-25T14:06:44.544953+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:58.970595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.520299+0000","last_change":"2024-04-25T14:06:44.520299+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.520299+0000","last_undegraded":"2024-04-25T14:06:44.520299+0000","last_fullsized":"2024-04-25T14:06:44.520299+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.547611+0000","last_change":"2024-04-25T14:06:44.543878+0000","last_active":"2024-04-25T14:06:44.547611+0000","last_peered":"2024-04-25T14:06:44.547611+0000","last_clean":"2024-04-25T14:06:44.547611+0000","last_became_active":"2024-04-25T14:06:44.543347+0000","last_became_peered":"2024-04-25T14:06:44.543347+0000","last_unstale":"2024-04-25T14:06:44.547611+0000","last_undegraded":"2024-04-25T14:06:44.547611+0000","last_fullsized":"2024-04-25T14:06:44.547611+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:56.180081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.532473+0000","last_change":"2024-04-25T14:06:44.532473+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.532473+0000","last_undegraded":"2024-04-25T14:06:44.532473+0000","last_fullsized":"2024-04-25T14:06:44.532473+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.550348+0000","last_change":"2024-04-25T14:06:44.546730+0000","last_active":"2024-04-25T14:06:44.550348+0000","last_peered":"2024-04-25T14:06:44.550348+0000","last_clean":"2024-04-25T14:06:44.550348+0000","last_became_active":"2024-04-25T14:06:44.546050+0000","last_became_peered":"2024-04-25T14:06:44.546050+0000","last_unstale":"2024-04-25T14:06:44.550348+0000","last_undegraded":"2024-04-25T14:06:44.550348+0000","last_fullsized":"2024-04-25T14:06:44.550348+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:58:21.045855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.531523+0000","last_change":"2024-04-25T14:06:44.531523+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.531523+0000","last_undegraded":"2024-04-25T14:06:44.531523+0000","last_fullsized":"2024-04-25T14:06:44.531523+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.540642+0000","last_change":"2024-04-25T14:06:44.534134+0000","last_active":"2024-04-25T14:06:44.540642+0000","last_peered":"2024-04-25T14:06:44.540642+0000","last_clean":"2024-04-25T14:06:44.540642+0000","last_became_active":"2024-04-25T14:06:44.533705+0000","last_became_peered":"2024-04-25T14:06:44.533705+0000","last_unstale":"2024-04-25T14:06:44.540642+0000","last_undegraded":"2024-04-25T14:06:44.540642+0000","last_fullsized":"2024-04-25T14:06:44.540642+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:43.875733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.535223+0000","last_change":"2024-04-25T14:06:44.535223+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.535223+0000","last_undegraded":"2024-04-25T14:06:44.535223+0000","last_fullsized":"2024-04-25T14:06:44.535223+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.545851+0000","last_change":"2024-04-25T14:06:44.538191+0000","last_active":"2024-04-25T14:06:44.545851+0000","last_peered":"2024-04-25T14:06:44.545851+0000","last_clean":"2024-04-25T14:06:44.545851+0000","last_became_active":"2024-04-25T14:06:44.537838+0000","last_became_peered":"2024-04-25T14:06:44.537838+0000","last_unstale":"2024-04-25T14:06:44.545851+0000","last_undegraded":"2024-04-25T14:06:44.545851+0000","last_fullsized":"2024-04-25T14:06:44.545851+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:43.249499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.532357+0000","last_change":"2024-04-25T14:06:44.532357+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.532357+0000","last_undegraded":"2024-04-25T14:06:44.532357+0000","last_fullsized":"2024-04-25T14:06:44.532357+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.543260+0000","last_change":"2024-04-25T14:06:44.539538+0000","last_active":"2024-04-25T14:06:44.543260+0000","last_peered":"2024-04-25T14:06:44.543260+0000","last_clean":"2024-04-25T14:06:44.543260+0000","last_became_active":"2024-04-25T14:06:44.538128+0000","last_became_peered":"2024-04-25T14:06:44.538128+0000","last_unstale":"2024-04-25T14:06:44.543260+0000","last_undegraded":"2024-04-25T14:06:44.543260+0000","last_fullsized":"2024-04-25T14:06:44.543260+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:56.433866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-25T14:06:44.525676+0000","last_change":"2024-04-25T14:06:44.525676+0000","last_active":"2024-04-25T14:06:44.508104+0000","last_peered":"2024-04-25T14:06:44.508104+0000","last_clean":"2024-04-25T14:06:44.508104+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:06:44.525676+0000","last_undegraded":"2024-04-25T14:06:44.525676+0000","last_fullsized":"2024-04-25T14:06:44.525676+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.542537+0000","last_change":"2024-04-25T14:06:44.539206+0000","last_active":"2024-04-25T14:06:44.542537+0000","last_peered":"2024-04-25T14:06:44.542537+0000","last_clean":"2024-04-25T14:06:44.542537+0000","last_became_active":"2024-04-25T14:06:44.538913+0000","last_became_peered":"2024-04-25T14:06:44.538913+0000","last_unstale":"2024-04-25T14:06:44.542537+0000","last_undegraded":"2024-04-25T14:06:44.542537+0000","last_fullsized":"2024-04-25T14:06:44.542537+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:09.304323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"59'86","reported_seq":255,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-25T14:06:44.525284+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:06:44.525284+0000","last_peered":"2024-04-25T14:06:44.525284+0000","last_clean":"2024-04-25T14:06:44.525284+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:06:44.525284+0000","last_undegraded":"2024-04-25T14:06:44.525284+0000","last_fullsized":"2024-04-25T14:06:44.525284+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1474560,"data_stored":1023970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":433602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":880640,"data_stored":433602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":433602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":433602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":433602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":433602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1474560,"data_stored":1023970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:06:54.252 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:06:54.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.407+0000 7fb0a1e00640 1 Processor -- start 2024-04-25T14:06:54.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.407+0000 7fb0a1e00640 1 -- start start 2024-04-25T14:06:54.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb0a1e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb09c0b59e0 0x7fb09c0b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:54.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb0a1e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb09c0b6330 con 0x7fb09c0aeea0 2024-04-25T14:06:54.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb0a1e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb09c06c670 con 0x7fb09c15faa0 2024-04-25T14:06:54.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb0a1e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb09c06d560 con 0x7fb09c0b59e0 2024-04-25T14:06:54.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb09b400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb09c0aeea0 0x7fb09c0af2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51730/0 (socket says 172.21.15.86:51730) 2024-04-25T14:06:54.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb09b400640 1 -- 172.21.15.86:0/1834904717 learned_addr learned my addr 172.21.15.86:0/1834904717 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:54.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb09a000640 1 --2- 172.21.15.86:0/1834904717 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb09c0b59e0 0x7fb09c0b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:54.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb099600640 1 -- 172.21.15.86:0/1834904717 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 673335120 0 0) 0x7fb0940014a0 con 0x7fb09c0aeea0 2024-04-25T14:06:54.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb099600640 1 -- 172.21.15.86:0/1834904717 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb080003530 con 0x7fb09c0aeea0 2024-04-25T14:06:54.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb099600640 1 -- 172.21.15.86:0/1834904717 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1132373981 0 0) 0x7fb09c06c670 con 0x7fb09c15faa0 2024-04-25T14:06:54.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb099600640 1 -- 172.21.15.86:0/1834904717 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0940014a0 con 0x7fb09c15faa0 2024-04-25T14:06:54.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb099600640 1 -- 172.21.15.86:0/1834904717 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3618585789 0 0) 0x7fb080003530 con 0x7fb09c0aeea0 2024-04-25T14:06:54.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb099600640 1 -- 172.21.15.86:0/1834904717 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb09c0b59e0 msgr2=0x7fb09c0b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:54.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb099600640 1 --2- 172.21.15.86:0/1834904717 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb09c0b59e0 0x7fb09c0b5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:54.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb099600640 1 -- 172.21.15.86:0/1834904717 >> v1:172.21.15.155:6789/0 conn(0x7fb09c15faa0 legacy=0x7fb09c06d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:54.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb099600640 1 -- 172.21.15.86:0/1834904717 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09c06c950 con 0x7fb09c0aeea0 2024-04-25T14:06:54.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb099600640 1 -- 172.21.15.86:0/1834904717 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fb094001bd0 con 0x7fb09c0aeea0 2024-04-25T14:06:54.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb099600640 1 -- 172.21.15.86:0/1834904717 >> v1:172.21.15.86:6789/0 conn(0x7fb09c0aeea0 legacy=0x7fb09c0af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:54.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb099600640 1 --2- 172.21.15.86:0/1834904717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb080003bf0 0x7fb080004040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:54.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb099600640 1 --2- 172.21.15.86:0/1834904717 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb09c0b59e0 0x7fb080004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:54.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb099600640 1 --2- 172.21.15.86:0/1834904717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb080004d50 0x7fb0800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:54.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb099600640 1 -- 172.21.15.86:0/1834904717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c06d560 con 0x7fb080003bf0 2024-04-25T14:06:54.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb099600640 1 -- 172.21.15.86:0/1834904717 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb08000bb60 con 0x7fb080004d50 2024-04-25T14:06:54.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb099600640 1 -- 172.21.15.86:0/1834904717 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb08000bd60 con 0x7fb09c0b59e0 2024-04-25T14:06:54.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb09aa00640 1 --2- 172.21.15.86:0/1834904717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb080003bf0 0x7fb080004040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:54.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb09b400640 1 --2- 172.21.15.86:0/1834904717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb080004d50 0x7fb0800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:54.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb09a000640 1 --2- 172.21.15.86:0/1834904717 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb09c0b59e0 0x7fb080004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:54.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb09aa00640 1 -- 172.21.15.86:0/1834904717 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb09c0b59e0 msgr2=0x7fb080004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:54.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb09aa00640 1 --2- 172.21.15.86:0/1834904717 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb09c0b59e0 0x7fb080004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:54.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb09aa00640 1 -- 172.21.15.86:0/1834904717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb080004d50 msgr2=0x7fb0800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:54.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb09aa00640 1 --2- 172.21.15.86:0/1834904717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb080004d50 0x7fb0800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:54.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.411+0000 7fb09aa00640 1 -- 172.21.15.86:0/1834904717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb080003530 con 0x7fb080003bf0 2024-04-25T14:06:54.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.415+0000 7fb09aa00640 1 --2- 172.21.15.86:0/1834904717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb080003bf0 0x7fb080004040 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fb0880018d0 tx=0x7fb08801c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee96bb7c1985a7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:54.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.415+0000 7fb099600640 1 -- 172.21.15.86:0/1834904717 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0880af070 con 0x7fb080003bf0 2024-04-25T14:06:54.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.415+0000 7fb099600640 1 -- 172.21.15.86:0/1834904717 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0880052f0 con 0x7fb080003bf0 2024-04-25T14:06:54.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.415+0000 7fb099600640 1 -- 172.21.15.86:0/1834904717 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb088001520 con 0x7fb080003bf0 2024-04-25T14:06:54.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.415+0000 7fb0a1e00640 1 -- 172.21.15.86:0/1834904717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb080003bf0 msgr2=0x7fb080004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:54.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.415+0000 7fb0a1e00640 1 --2- 172.21.15.86:0/1834904717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb080003bf0 0x7fb080004040 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fb0880018d0 tx=0x7fb08801c580 comp rx=0 tx=0).stop 2024-04-25T14:06:54.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.415+0000 7fb09b400640 1 -- 172.21.15.86:0/1834904717 reap_dead start 2024-04-25T14:06:54.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.415+0000 7fb0a1e00640 1 -- 172.21.15.86:0/1834904717 shutdown_connections 2024-04-25T14:06:54.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.415+0000 7fb0a1e00640 1 -- 172.21.15.86:0/1834904717 >> 172.21.15.86:0/1834904717 conn(0x7fb09c0d2230 msgr2=0x7fb09c0d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:54.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.415+0000 7fb0a1e00640 1 -- 172.21.15.86:0/1834904717 shutdown_connections 2024-04-25T14:06:54.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.415+0000 7fb0a1e00640 1 -- 172.21.15.86:0/1834904717 wait complete. 2024-04-25T14:06:54.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.419+0000 7fb0a1e00640 1 Processor -- start 2024-04-25T14:06:54.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.419+0000 7fb0a1e00640 1 -- start start 2024-04-25T14:06:54.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.419+0000 7fb0a1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb080003bf0 0x7fb09c1613b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:54.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.419+0000 7fb0a1e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb09c1618f0 0x7fb09c162700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:54.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.419+0000 7fb0a1e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb09c162c40 0x7fb09c160330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:54.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.419+0000 7fb0a1e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c15feb0 con 0x7fb080003bf0 2024-04-25T14:06:54.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.419+0000 7fb0a1e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c0af9c0 con 0x7fb09c1618f0 2024-04-25T14:06:54.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.419+0000 7fb0a1e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb09c160030 con 0x7fb09c162c40 2024-04-25T14:06:54.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.419+0000 7fb09aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb080003bf0 0x7fb09c1613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:54.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.419+0000 7fb09aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb080003bf0 0x7fb09c1613b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56512/0 (socket says 172.21.15.86:56512) 2024-04-25T14:06:54.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.419+0000 7fb09aa00640 1 -- 172.21.15.86:0/2952275960 learned_addr learned my addr 172.21.15.86:0/2952275960 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:54.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.419+0000 7fb09a000640 1 --2- 172.21.15.86:0/2952275960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb09c1618f0 0x7fb09c162700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:54.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.419+0000 7fb09b400640 1 --2- 172.21.15.86:0/2952275960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb09c162c40 0x7fb09c160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:54.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.419+0000 7fb09aa00640 1 -- 172.21.15.86:0/2952275960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb09c162c40 msgr2=0x7fb09c160330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:54.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.419+0000 7fb09aa00640 1 --2- 172.21.15.86:0/2952275960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb09c162c40 0x7fb09c160330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:54.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.423+0000 7fb09aa00640 1 -- 172.21.15.86:0/2952275960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb09c1618f0 msgr2=0x7fb09c162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:54.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.423+0000 7fb09aa00640 1 --2- 172.21.15.86:0/2952275960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb09c1618f0 0x7fb09c162700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:54.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.423+0000 7fb09aa00640 1 -- 172.21.15.86:0/2952275960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09c163050 con 0x7fb080003bf0 2024-04-25T14:06:54.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.423+0000 7fb09b400640 1 --2- 172.21.15.86:0/2952275960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb09c162c40 0x7fb09c160330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:54.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.423+0000 7fb09aa00640 1 --2- 172.21.15.86:0/2952275960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb080003bf0 0x7fb09c1613b0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fb0880ad040 tx=0x7fb080005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:54.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.423+0000 7fb093400640 1 -- 172.21.15.86:0/2952275960 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0880be070 con 0x7fb080003bf0 2024-04-25T14:06:54.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.423+0000 7fb093400640 1 -- 172.21.15.86:0/2952275960 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb088001500 con 0x7fb080003bf0 2024-04-25T14:06:54.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.423+0000 7fb093400640 1 -- 172.21.15.86:0/2952275960 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0880af040 con 0x7fb080003bf0 2024-04-25T14:06:54.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.423+0000 7fb0a1e00640 1 -- 172.21.15.86:0/2952275960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb09c165b80 con 0x7fb080003bf0 2024-04-25T14:06:54.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.423+0000 7fb0a1e00640 1 -- 172.21.15.86:0/2952275960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb09c166140 con 0x7fb080003bf0 2024-04-25T14:06:54.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.423+0000 7fb0a1e00640 1 -- 172.21.15.86:0/2952275960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb09c074cf0 con 0x7fb080003bf0 2024-04-25T14:06:54.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.423+0000 7fb093400640 1 -- 172.21.15.86:0/2952275960 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7fb088006a00 con 0x7fb080003bf0 2024-04-25T14:06:54.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.423+0000 7fb093400640 1 --2- 172.21.15.86:0/2952275960 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fb07c06f640 0x7fb07c071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:54.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.423+0000 7fb093400640 1 -- 172.21.15.86:0/2952275960 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7fb088128ab0 con 0x7fb080003bf0 2024-04-25T14:06:54.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.427+0000 7fb09a000640 1 --2- 172.21.15.86:0/2952275960 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fb07c06f640 0x7fb07c071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:54.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.427+0000 7fb09a000640 1 --2- 172.21.15.86:0/2952275960 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fb07c06f640 0x7fb07c071b50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0b1730 tx=0x7fb08c009190 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:54.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.431+0000 7fb093400640 1 -- 172.21.15.86:0/2952275960 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb09c163050 con 0x7fb080003bf0 2024-04-25T14:06:54.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.583+0000 7fb0a1e00640 1 -- 172.21.15.86:0/2952275960 --> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb09c0b55b0 con 0x7fb07c06f640 2024-04-25T14:06:54.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.591+0000 7fb093400640 1 -- 172.21.15.86:0/2952275960 <== mgr.6198 v2:172.21.15.86:6832/1025829557 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252067 (secure 0 0 0) 0x7fb09c0b55b0 con 0x7fb07c06f640 2024-04-25T14:06:54.594 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:54.596 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:06:54.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.595+0000 7fb0a1e00640 1 -- 172.21.15.86:0/2952275960 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fb07c06f640 msgr2=0x7fb07c071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:54.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.595+0000 7fb0a1e00640 1 --2- 172.21.15.86:0/2952275960 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fb07c06f640 0x7fb07c071b50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0b1730 tx=0x7fb08c009190 comp rx=0 tx=0).stop 2024-04-25T14:06:54.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.595+0000 7fb0a1e00640 1 -- 172.21.15.86:0/2952275960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb080003bf0 msgr2=0x7fb09c1613b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:54.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.595+0000 7fb0a1e00640 1 --2- 172.21.15.86:0/2952275960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb080003bf0 0x7fb09c1613b0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fb0880ad040 tx=0x7fb080005840 comp rx=0 tx=0).stop 2024-04-25T14:06:54.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.595+0000 7fb0a1e00640 1 -- 172.21.15.86:0/2952275960 shutdown_connections 2024-04-25T14:06:54.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.595+0000 7fb0a1e00640 1 --2- 172.21.15.86:0/2952275960 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fb07c06f640 0x7fb07c071b50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:54.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.595+0000 7fb0a1e00640 1 --2- 172.21.15.86:0/2952275960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb09c162c40 0x7fb09c160330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:54.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.595+0000 7fb0a1e00640 1 --2- 172.21.15.86:0/2952275960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb09c1618f0 0x7fb09c162700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:54.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.595+0000 7fb0a1e00640 1 --2- 172.21.15.86:0/2952275960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb080003bf0 0x7fb09c1613b0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:54.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.595+0000 7fb0a1e00640 1 -- 172.21.15.86:0/2952275960 >> 172.21.15.86:0/2952275960 conn(0x7fb09c0d2230 msgr2=0x7fb09c15e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:54.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.595+0000 7fb0a1e00640 1 -- 172.21.15.86:0/2952275960 shutdown_connections 2024-04-25T14:06:54.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.595+0000 7fb0a1e00640 1 -- 172.21.15.86:0/2952275960 wait complete. 2024-04-25T14:06:54.615 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-04-25T14:06:53.439242+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":170,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125,"ondisk_log_size":125,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253584,"kb_used_data":8296,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721136,"statfs":{"total":773094113280,"available":772834443264,"internally_reserved":0,"allocated":8495104,"data_stored":4765244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002358"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:47.113061+0000","last_change":"2024-04-25T14:06:45.555573+0000","last_active":"2024-04-25T14:06:47.113061+0000","last_peered":"2024-04-25T14:06:47.113061+0000","last_clean":"2024-04-25T14:06:47.113061+0000","last_became_active":"2024-04-25T14:06:45.555268+0000","last_became_peered":"2024-04-25T14:06:45.555268+0000","last_unstale":"2024-04-25T14:06:47.113061+0000","last_undegraded":"2024-04-25T14:06:47.113061+0000","last_fullsized":"2024-04-25T14:06:47.113061+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:19.197890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.545073+0000","last_change":"2024-04-25T14:06:45.559810+0000","last_active":"2024-04-25T14:06:46.545073+0000","last_peered":"2024-04-25T14:06:46.545073+0000","last_clean":"2024-04-25T14:06:46.545073+0000","last_became_active":"2024-04-25T14:06:45.559199+0000","last_became_peered":"2024-04-25T14:06:45.559199+0000","last_unstale":"2024-04-25T14:06:46.545073+0000","last_undegraded":"2024-04-25T14:06:46.545073+0000","last_fullsized":"2024-04-25T14:06:46.545073+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:46.208706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:47.111972+0000","last_change":"2024-04-25T14:06:45.556031+0000","last_active":"2024-04-25T14:06:47.111972+0000","last_peered":"2024-04-25T14:06:47.111972+0000","last_clean":"2024-04-25T14:06:47.111972+0000","last_became_active":"2024-04-25T14:06:45.555727+0000","last_became_peered":"2024-04-25T14:06:45.555727+0000","last_unstale":"2024-04-25T14:06:47.111972+0000","last_undegraded":"2024-04-25T14:06:47.111972+0000","last_fullsized":"2024-04-25T14:06:47.111972+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:48.517004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.546958+0000","last_change":"2024-04-25T14:06:45.556602+0000","last_active":"2024-04-25T14:06:46.546958+0000","last_peered":"2024-04-25T14:06:46.546958+0000","last_clean":"2024-04-25T14:06:46.546958+0000","last_became_active":"2024-04-25T14:06:45.555414+0000","last_became_peered":"2024-04-25T14:06:45.555414+0000","last_unstale":"2024-04-25T14:06:46.546958+0000","last_undegraded":"2024-04-25T14:06:46.546958+0000","last_fullsized":"2024-04-25T14:06:46.546958+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:26.121876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.555464+0000","last_change":"2024-04-25T14:06:45.557953+0000","last_active":"2024-04-25T14:06:46.555464+0000","last_peered":"2024-04-25T14:06:46.555464+0000","last_clean":"2024-04-25T14:06:46.555464+0000","last_became_active":"2024-04-25T14:06:45.555441+0000","last_became_peered":"2024-04-25T14:06:45.555441+0000","last_unstale":"2024-04-25T14:06:46.555464+0000","last_undegraded":"2024-04-25T14:06:46.555464+0000","last_fullsized":"2024-04-25T14:06:46.555464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:59:35.109177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.695416+0000","last_change":"2024-04-25T14:06:45.549137+0000","last_active":"2024-04-25T14:06:46.695416+0000","last_peered":"2024-04-25T14:06:46.695416+0000","last_clean":"2024-04-25T14:06:46.695416+0000","last_became_active":"2024-04-25T14:06:45.548828+0000","last_became_peered":"2024-04-25T14:06:45.548828+0000","last_unstale":"2024-04-25T14:06:46.695416+0000","last_undegraded":"2024-04-25T14:06:46.695416+0000","last_fullsized":"2024-04-25T14:06:46.695416+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:15.506001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.561437+0000","last_change":"2024-04-25T14:06:45.555239+0000","last_active":"2024-04-25T14:06:46.561437+0000","last_peered":"2024-04-25T14:06:46.561437+0000","last_clean":"2024-04-25T14:06:46.561437+0000","last_became_active":"2024-04-25T14:06:45.551371+0000","last_became_peered":"2024-04-25T14:06:45.551371+0000","last_unstale":"2024-04-25T14:06:46.561437+0000","last_undegraded":"2024-04-25T14:06:46.561437+0000","last_fullsized":"2024-04-25T14:06:46.561437+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:47.790698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.555229+0000","last_change":"2024-04-25T14:06:45.555140+0000","last_active":"2024-04-25T14:06:46.555229+0000","last_peered":"2024-04-25T14:06:46.555229+0000","last_clean":"2024-04-25T14:06:46.555229+0000","last_became_active":"2024-04-25T14:06:45.554825+0000","last_became_peered":"2024-04-25T14:06:45.554825+0000","last_unstale":"2024-04-25T14:06:46.555229+0000","last_undegraded":"2024-04-25T14:06:46.555229+0000","last_fullsized":"2024-04-25T14:06:46.555229+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:52.840883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.545978+0000","last_change":"2024-04-25T14:06:45.550566+0000","last_active":"2024-04-25T14:06:46.545978+0000","last_peered":"2024-04-25T14:06:46.545978+0000","last_clean":"2024-04-25T14:06:46.545978+0000","last_became_active":"2024-04-25T14:06:45.550157+0000","last_became_peered":"2024-04-25T14:06:45.550157+0000","last_unstale":"2024-04-25T14:06:46.545978+0000","last_undegraded":"2024-04-25T14:06:46.545978+0000","last_fullsized":"2024-04-25T14:06:46.545978+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:27.745456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.547396+0000","last_change":"2024-04-25T14:06:45.549648+0000","last_active":"2024-04-25T14:06:46.547396+0000","last_peered":"2024-04-25T14:06:46.547396+0000","last_clean":"2024-04-25T14:06:46.547396+0000","last_became_active":"2024-04-25T14:06:45.548013+0000","last_became_peered":"2024-04-25T14:06:45.548013+0000","last_unstale":"2024-04-25T14:06:46.547396+0000","last_undegraded":"2024-04-25T14:06:46.547396+0000","last_fullsized":"2024-04-25T14:06:46.547396+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:29.910494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.544938+0000","last_change":"2024-04-25T14:06:45.557670+0000","last_active":"2024-04-25T14:06:46.544938+0000","last_peered":"2024-04-25T14:06:46.544938+0000","last_clean":"2024-04-25T14:06:46.544938+0000","last_became_active":"2024-04-25T14:06:45.557339+0000","last_became_peered":"2024-04-25T14:06:45.557339+0000","last_unstale":"2024-04-25T14:06:46.544938+0000","last_undegraded":"2024-04-25T14:06:46.544938+0000","last_fullsized":"2024-04-25T14:06:46.544938+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:56.251206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.546999+0000","last_change":"2024-04-25T14:06:45.551950+0000","last_active":"2024-04-25T14:06:46.546999+0000","last_peered":"2024-04-25T14:06:46.546999+0000","last_clean":"2024-04-25T14:06:46.546999+0000","last_became_active":"2024-04-25T14:06:45.551655+0000","last_became_peered":"2024-04-25T14:06:45.551655+0000","last_unstale":"2024-04-25T14:06:46.546999+0000","last_undegraded":"2024-04-25T14:06:46.546999+0000","last_fullsized":"2024-04-25T14:06:46.546999+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:52.363199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.555599+0000","last_change":"2024-04-25T14:06:45.558257+0000","last_active":"2024-04-25T14:06:46.555599+0000","last_peered":"2024-04-25T14:06:46.555599+0000","last_clean":"2024-04-25T14:06:46.555599+0000","last_became_active":"2024-04-25T14:06:45.557605+0000","last_became_peered":"2024-04-25T14:06:45.557605+0000","last_unstale":"2024-04-25T14:06:46.555599+0000","last_undegraded":"2024-04-25T14:06:46.555599+0000","last_fullsized":"2024-04-25T14:06:46.555599+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:45.234430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.695464+0000","last_change":"2024-04-25T14:06:45.546008+0000","last_active":"2024-04-25T14:06:46.695464+0000","last_peered":"2024-04-25T14:06:46.695464+0000","last_clean":"2024-04-25T14:06:46.695464+0000","last_became_active":"2024-04-25T14:06:45.545669+0000","last_became_peered":"2024-04-25T14:06:45.545669+0000","last_unstale":"2024-04-25T14:06:46.695464+0000","last_undegraded":"2024-04-25T14:06:46.695464+0000","last_fullsized":"2024-04-25T14:06:46.695464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:19.197890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.544316+0000","last_change":"2024-04-25T14:06:45.552543+0000","last_active":"2024-04-25T14:06:46.544316+0000","last_peered":"2024-04-25T14:06:46.544316+0000","last_clean":"2024-04-25T14:06:46.544316+0000","last_became_active":"2024-04-25T14:06:45.552188+0000","last_became_peered":"2024-04-25T14:06:45.552188+0000","last_unstale":"2024-04-25T14:06:46.544316+0000","last_undegraded":"2024-04-25T14:06:46.544316+0000","last_fullsized":"2024-04-25T14:06:46.544316+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:48.959731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:47.112574+0000","last_change":"2024-04-25T14:06:45.551883+0000","last_active":"2024-04-25T14:06:47.112574+0000","last_peered":"2024-04-25T14:06:47.112574+0000","last_clean":"2024-04-25T14:06:47.112574+0000","last_became_active":"2024-04-25T14:06:45.551171+0000","last_became_peered":"2024-04-25T14:06:45.551171+0000","last_unstale":"2024-04-25T14:06:47.112574+0000","last_undegraded":"2024-04-25T14:06:47.112574+0000","last_fullsized":"2024-04-25T14:06:47.112574+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:35.960539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.556069+0000","last_change":"2024-04-25T14:06:45.551076+0000","last_active":"2024-04-25T14:06:46.556069+0000","last_peered":"2024-04-25T14:06:46.556069+0000","last_clean":"2024-04-25T14:06:46.556069+0000","last_became_active":"2024-04-25T14:06:45.550768+0000","last_became_peered":"2024-04-25T14:06:45.550768+0000","last_unstale":"2024-04-25T14:06:46.556069+0000","last_undegraded":"2024-04-25T14:06:46.556069+0000","last_fullsized":"2024-04-25T14:06:46.556069+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:03.121953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:47.111967+0000","last_change":"2024-04-25T14:06:45.560878+0000","last_active":"2024-04-25T14:06:47.111967+0000","last_peered":"2024-04-25T14:06:47.111967+0000","last_clean":"2024-04-25T14:06:47.111967+0000","last_became_active":"2024-04-25T14:06:45.560393+0000","last_became_peered":"2024-04-25T14:06:45.560393+0000","last_unstale":"2024-04-25T14:06:47.111967+0000","last_undegraded":"2024-04-25T14:06:47.111967+0000","last_fullsized":"2024-04-25T14:06:47.111967+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:25.253476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.695582+0000","last_change":"2024-04-25T14:06:45.563551+0000","last_active":"2024-04-25T14:06:46.695582+0000","last_peered":"2024-04-25T14:06:46.695582+0000","last_clean":"2024-04-25T14:06:46.695582+0000","last_became_active":"2024-04-25T14:06:45.563269+0000","last_became_peered":"2024-04-25T14:06:45.563269+0000","last_unstale":"2024-04-25T14:06:46.695582+0000","last_undegraded":"2024-04-25T14:06:46.695582+0000","last_fullsized":"2024-04-25T14:06:46.695582+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:10.273391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.555730+0000","last_change":"2024-04-25T14:06:45.556841+0000","last_active":"2024-04-25T14:06:46.555730+0000","last_peered":"2024-04-25T14:06:46.555730+0000","last_clean":"2024-04-25T14:06:46.555730+0000","last_became_active":"2024-04-25T14:06:45.556510+0000","last_became_peered":"2024-04-25T14:06:45.556510+0000","last_unstale":"2024-04-25T14:06:46.555730+0000","last_undegraded":"2024-04-25T14:06:46.555730+0000","last_fullsized":"2024-04-25T14:06:46.555730+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:29.568495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.546207+0000","last_change":"2024-04-25T14:06:45.554445+0000","last_active":"2024-04-25T14:06:46.546207+0000","last_peered":"2024-04-25T14:06:46.546207+0000","last_clean":"2024-04-25T14:06:46.546207+0000","last_became_active":"2024-04-25T14:06:45.554117+0000","last_became_peered":"2024-04-25T14:06:45.554117+0000","last_unstale":"2024-04-25T14:06:46.546207+0000","last_undegraded":"2024-04-25T14:06:46.546207+0000","last_fullsized":"2024-04-25T14:06:46.546207+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:18.005539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.555543+0000","last_change":"2024-04-25T14:06:45.555522+0000","last_active":"2024-04-25T14:06:46.555543+0000","last_peered":"2024-04-25T14:06:46.555543+0000","last_clean":"2024-04-25T14:06:46.555543+0000","last_became_active":"2024-04-25T14:06:45.555043+0000","last_became_peered":"2024-04-25T14:06:45.555043+0000","last_unstale":"2024-04-25T14:06:46.555543+0000","last_undegraded":"2024-04-25T14:06:46.555543+0000","last_fullsized":"2024-04-25T14:06:46.555543+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:14.620178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.545628+0000","last_change":"2024-04-25T14:06:45.553892+0000","last_active":"2024-04-25T14:06:46.545628+0000","last_peered":"2024-04-25T14:06:46.545628+0000","last_clean":"2024-04-25T14:06:46.545628+0000","last_became_active":"2024-04-25T14:06:45.553162+0000","last_became_peered":"2024-04-25T14:06:45.553162+0000","last_unstale":"2024-04-25T14:06:46.545628+0000","last_undegraded":"2024-04-25T14:06:46.545628+0000","last_fullsized":"2024-04-25T14:06:46.545628+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:28.931279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.544302+0000","last_change":"2024-04-25T14:06:45.550631+0000","last_active":"2024-04-25T14:06:46.544302+0000","last_peered":"2024-04-25T14:06:46.544302+0000","last_clean":"2024-04-25T14:06:46.544302+0000","last_became_active":"2024-04-25T14:06:45.550155+0000","last_became_peered":"2024-04-25T14:06:45.550155+0000","last_unstale":"2024-04-25T14:06:46.544302+0000","last_undegraded":"2024-04-25T14:06:46.544302+0000","last_fullsized":"2024-04-25T14:06:46.544302+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:23.644758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.545047+0000","last_change":"2024-04-25T14:06:45.546272+0000","last_active":"2024-04-25T14:06:46.545047+0000","last_peered":"2024-04-25T14:06:46.545047+0000","last_clean":"2024-04-25T14:06:46.545047+0000","last_became_active":"2024-04-25T14:06:45.545951+0000","last_became_peered":"2024-04-25T14:06:45.545951+0000","last_unstale":"2024-04-25T14:06:46.545047+0000","last_undegraded":"2024-04-25T14:06:46.545047+0000","last_fullsized":"2024-04-25T14:06:46.545047+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:56:09.478105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.547401+0000","last_change":"2024-04-25T14:06:45.547961+0000","last_active":"2024-04-25T14:06:46.547401+0000","last_peered":"2024-04-25T14:06:46.547401+0000","last_clean":"2024-04-25T14:06:46.547401+0000","last_became_active":"2024-04-25T14:06:45.547605+0000","last_became_peered":"2024-04-25T14:06:45.547605+0000","last_unstale":"2024-04-25T14:06:46.547401+0000","last_undegraded":"2024-04-25T14:06:46.547401+0000","last_fullsized":"2024-04-25T14:06:46.547401+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:46.038778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:47.112799+0000","last_change":"2024-04-25T14:06:45.553140+0000","last_active":"2024-04-25T14:06:47.112799+0000","last_peered":"2024-04-25T14:06:47.112799+0000","last_clean":"2024-04-25T14:06:47.112799+0000","last_became_active":"2024-04-25T14:06:45.552827+0000","last_became_peered":"2024-04-25T14:06:45.552827+0000","last_unstale":"2024-04-25T14:06:47.112799+0000","last_undegraded":"2024-04-25T14:06:47.112799+0000","last_fullsized":"2024-04-25T14:06:47.112799+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:15.506001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.555788+0000","last_change":"2024-04-25T14:06:45.557826+0000","last_active":"2024-04-25T14:06:46.555788+0000","last_peered":"2024-04-25T14:06:46.555788+0000","last_clean":"2024-04-25T14:06:46.555788+0000","last_became_active":"2024-04-25T14:06:45.556788+0000","last_became_peered":"2024-04-25T14:06:45.556788+0000","last_unstale":"2024-04-25T14:06:46.555788+0000","last_undegraded":"2024-04-25T14:06:46.555788+0000","last_fullsized":"2024-04-25T14:06:46.555788+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:00.657890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.544691+0000","last_change":"2024-04-25T14:06:45.552944+0000","last_active":"2024-04-25T14:06:46.544691+0000","last_peered":"2024-04-25T14:06:46.544691+0000","last_clean":"2024-04-25T14:06:46.544691+0000","last_became_active":"2024-04-25T14:06:45.551059+0000","last_became_peered":"2024-04-25T14:06:45.551059+0000","last_unstale":"2024-04-25T14:06:46.544691+0000","last_undegraded":"2024-04-25T14:06:46.544691+0000","last_fullsized":"2024-04-25T14:06:46.544691+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:00.883283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:47.112507+0000","last_change":"2024-04-25T14:06:45.557049+0000","last_active":"2024-04-25T14:06:47.112507+0000","last_peered":"2024-04-25T14:06:47.112507+0000","last_clean":"2024-04-25T14:06:47.112507+0000","last_became_active":"2024-04-25T14:06:45.556654+0000","last_became_peered":"2024-04-25T14:06:45.556654+0000","last_unstale":"2024-04-25T14:06:47.112507+0000","last_undegraded":"2024-04-25T14:06:47.112507+0000","last_fullsized":"2024-04-25T14:06:47.112507+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:21.244452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:47.111609+0000","last_change":"2024-04-25T14:06:45.564603+0000","last_active":"2024-04-25T14:06:47.111609+0000","last_peered":"2024-04-25T14:06:47.111609+0000","last_clean":"2024-04-25T14:06:47.111609+0000","last_became_active":"2024-04-25T14:06:45.560882+0000","last_became_peered":"2024-04-25T14:06:45.560882+0000","last_unstale":"2024-04-25T14:06:47.111609+0000","last_undegraded":"2024-04-25T14:06:47.111609+0000","last_fullsized":"2024-04-25T14:06:47.111609+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:57.189625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:47.112517+0000","last_change":"2024-04-25T14:06:45.552167+0000","last_active":"2024-04-25T14:06:47.112517+0000","last_peered":"2024-04-25T14:06:47.112517+0000","last_clean":"2024-04-25T14:06:47.112517+0000","last_became_active":"2024-04-25T14:06:45.551579+0000","last_became_peered":"2024-04-25T14:06:45.551579+0000","last_unstale":"2024-04-25T14:06:47.112517+0000","last_undegraded":"2024-04-25T14:06:47.112517+0000","last_fullsized":"2024-04-25T14:06:47.112517+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:42.845801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:47.111639+0000","last_change":"2024-04-25T14:06:45.554170+0000","last_active":"2024-04-25T14:06:47.111639+0000","last_peered":"2024-04-25T14:06:47.111639+0000","last_clean":"2024-04-25T14:06:47.111639+0000","last_became_active":"2024-04-25T14:06:45.553860+0000","last_became_peered":"2024-04-25T14:06:45.553860+0000","last_unstale":"2024-04-25T14:06:47.111639+0000","last_undegraded":"2024-04-25T14:06:47.111639+0000","last_fullsized":"2024-04-25T14:06:47.111639+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:23.790336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"73'4","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.556037+0000","last_change":"2024-04-25T14:06:44.538430+0000","last_active":"2024-04-25T14:06:46.556037+0000","last_peered":"2024-04-25T14:06:46.556037+0000","last_clean":"2024-04-25T14:06:46.556037+0000","last_became_active":"2024-04-25T14:06:44.538081+0000","last_became_peered":"2024-04-25T14:06:44.538081+0000","last_unstale":"2024-04-25T14:06:46.556037+0000","last_undegraded":"2024-04-25T14:06:46.556037+0000","last_fullsized":"2024-04-25T14:06:46.556037+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:02.554414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.544330+0000","last_change":"2024-04-25T14:06:45.546257+0000","last_active":"2024-04-25T14:06:46.544330+0000","last_peered":"2024-04-25T14:06:46.544330+0000","last_clean":"2024-04-25T14:06:46.544330+0000","last_became_active":"2024-04-25T14:06:45.545908+0000","last_became_peered":"2024-04-25T14:06:45.545908+0000","last_unstale":"2024-04-25T14:06:46.544330+0000","last_undegraded":"2024-04-25T14:06:46.544330+0000","last_fullsized":"2024-04-25T14:06:46.544330+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:02.926990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:47.112918+0000","last_change":"2024-04-25T14:06:44.544725+0000","last_active":"2024-04-25T14:06:47.112918+0000","last_peered":"2024-04-25T14:06:47.112918+0000","last_clean":"2024-04-25T14:06:47.112918+0000","last_became_active":"2024-04-25T14:06:44.544154+0000","last_became_peered":"2024-04-25T14:06:44.544154+0000","last_unstale":"2024-04-25T14:06:47.112918+0000","last_undegraded":"2024-04-25T14:06:47.112918+0000","last_fullsized":"2024-04-25T14:06:47.112918+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:06.121310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.546056+0000","last_change":"2024-04-25T14:06:45.549075+0000","last_active":"2024-04-25T14:06:46.546056+0000","last_peered":"2024-04-25T14:06:46.546056+0000","last_clean":"2024-04-25T14:06:46.546056+0000","last_became_active":"2024-04-25T14:06:45.548734+0000","last_became_peered":"2024-04-25T14:06:45.548734+0000","last_unstale":"2024-04-25T14:06:46.546056+0000","last_undegraded":"2024-04-25T14:06:46.546056+0000","last_fullsized":"2024-04-25T14:06:46.546056+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:09.401872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:47.112741+0000","last_change":"2024-04-25T14:06:44.537708+0000","last_active":"2024-04-25T14:06:47.112741+0000","last_peered":"2024-04-25T14:06:47.112741+0000","last_clean":"2024-04-25T14:06:47.112741+0000","last_became_active":"2024-04-25T14:06:44.537349+0000","last_became_peered":"2024-04-25T14:06:44.537349+0000","last_unstale":"2024-04-25T14:06:47.112741+0000","last_undegraded":"2024-04-25T14:06:47.112741+0000","last_fullsized":"2024-04-25T14:06:47.112741+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:42.088259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.546396+0000","last_change":"2024-04-25T14:06:45.553458+0000","last_active":"2024-04-25T14:06:46.546396+0000","last_peered":"2024-04-25T14:06:46.546396+0000","last_clean":"2024-04-25T14:06:46.546396+0000","last_became_active":"2024-04-25T14:06:45.553076+0000","last_became_peered":"2024-04-25T14:06:45.553076+0000","last_unstale":"2024-04-25T14:06:46.546396+0000","last_undegraded":"2024-04-25T14:06:46.546396+0000","last_fullsized":"2024-04-25T14:06:46.546396+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:28.623569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.561419+0000","last_change":"2024-04-25T14:06:44.546446+0000","last_active":"2024-04-25T14:06:46.561419+0000","last_peered":"2024-04-25T14:06:46.561419+0000","last_clean":"2024-04-25T14:06:46.561419+0000","last_became_active":"2024-04-25T14:06:44.546076+0000","last_became_peered":"2024-04-25T14:06:44.546076+0000","last_unstale":"2024-04-25T14:06:46.561419+0000","last_undegraded":"2024-04-25T14:06:46.561419+0000","last_fullsized":"2024-04-25T14:06:46.561419+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:18.704254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.544719+0000","last_change":"2024-04-25T14:06:45.563881+0000","last_active":"2024-04-25T14:06:46.544719+0000","last_peered":"2024-04-25T14:06:46.544719+0000","last_clean":"2024-04-25T14:06:46.544719+0000","last_became_active":"2024-04-25T14:06:45.563469+0000","last_became_peered":"2024-04-25T14:06:45.563469+0000","last_unstale":"2024-04-25T14:06:46.544719+0000","last_undegraded":"2024-04-25T14:06:46.544719+0000","last_fullsized":"2024-04-25T14:06:46.544719+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:52.242305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.546567+0000","last_change":"2024-04-25T14:06:44.538629+0000","last_active":"2024-04-25T14:06:46.546567+0000","last_peered":"2024-04-25T14:06:46.546567+0000","last_clean":"2024-04-25T14:06:46.546567+0000","last_became_active":"2024-04-25T14:06:44.538329+0000","last_became_peered":"2024-04-25T14:06:44.538329+0000","last_unstale":"2024-04-25T14:06:46.546567+0000","last_undegraded":"2024-04-25T14:06:46.546567+0000","last_fullsized":"2024-04-25T14:06:46.546567+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:45.576873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.545173+0000","last_change":"2024-04-25T14:06:45.565376+0000","last_active":"2024-04-25T14:06:46.545173+0000","last_peered":"2024-04-25T14:06:46.545173+0000","last_clean":"2024-04-25T14:06:46.545173+0000","last_became_active":"2024-04-25T14:06:45.565089+0000","last_became_peered":"2024-04-25T14:06:45.565089+0000","last_unstale":"2024-04-25T14:06:46.545173+0000","last_undegraded":"2024-04-25T14:06:46.545173+0000","last_fullsized":"2024-04-25T14:06:46.545173+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:08.067785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.546648+0000","last_change":"2024-04-25T14:06:44.530649+0000","last_active":"2024-04-25T14:06:46.546648+0000","last_peered":"2024-04-25T14:06:46.546648+0000","last_clean":"2024-04-25T14:06:46.546648+0000","last_became_active":"2024-04-25T14:06:44.530025+0000","last_became_peered":"2024-04-25T14:06:44.530025+0000","last_unstale":"2024-04-25T14:06:46.546648+0000","last_undegraded":"2024-04-25T14:06:46.546648+0000","last_fullsized":"2024-04-25T14:06:46.546648+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:22.780792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.546033+0000","last_change":"2024-04-25T14:06:45.549769+0000","last_active":"2024-04-25T14:06:46.546033+0000","last_peered":"2024-04-25T14:06:46.546033+0000","last_clean":"2024-04-25T14:06:46.546033+0000","last_became_active":"2024-04-25T14:06:45.549435+0000","last_became_peered":"2024-04-25T14:06:45.549435+0000","last_unstale":"2024-04-25T14:06:46.546033+0000","last_undegraded":"2024-04-25T14:06:46.546033+0000","last_fullsized":"2024-04-25T14:06:46.546033+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:39.275493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.544441+0000","last_change":"2024-04-25T14:06:44.537817+0000","last_active":"2024-04-25T14:06:46.544441+0000","last_peered":"2024-04-25T14:06:46.544441+0000","last_clean":"2024-04-25T14:06:46.544441+0000","last_became_active":"2024-04-25T14:06:44.537198+0000","last_became_peered":"2024-04-25T14:06:44.537198+0000","last_unstale":"2024-04-25T14:06:46.544441+0000","last_undegraded":"2024-04-25T14:06:46.544441+0000","last_fullsized":"2024-04-25T14:06:46.544441+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:51.353655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.561838+0000","last_change":"2024-04-25T14:06:45.561422+0000","last_active":"2024-04-25T14:06:46.561838+0000","last_peered":"2024-04-25T14:06:46.561838+0000","last_clean":"2024-04-25T14:06:46.561838+0000","last_became_active":"2024-04-25T14:06:45.560686+0000","last_became_peered":"2024-04-25T14:06:45.560686+0000","last_unstale":"2024-04-25T14:06:46.561838+0000","last_undegraded":"2024-04-25T14:06:46.561838+0000","last_fullsized":"2024-04-25T14:06:46.561838+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:10.313867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.555266+0000","last_change":"2024-04-25T14:06:44.539464+0000","last_active":"2024-04-25T14:06:46.555266+0000","last_peered":"2024-04-25T14:06:46.555266+0000","last_clean":"2024-04-25T14:06:46.555266+0000","last_became_active":"2024-04-25T14:06:44.539110+0000","last_became_peered":"2024-04-25T14:06:44.539110+0000","last_unstale":"2024-04-25T14:06:46.555266+0000","last_undegraded":"2024-04-25T14:06:46.555266+0000","last_fullsized":"2024-04-25T14:06:46.555266+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:17.107206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.544302+0000","last_change":"2024-04-25T14:06:45.551272+0000","last_active":"2024-04-25T14:06:46.544302+0000","last_peered":"2024-04-25T14:06:46.544302+0000","last_clean":"2024-04-25T14:06:46.544302+0000","last_became_active":"2024-04-25T14:06:45.550974+0000","last_became_peered":"2024-04-25T14:06:45.550974+0000","last_unstale":"2024-04-25T14:06:46.544302+0000","last_undegraded":"2024-04-25T14:06:46.544302+0000","last_fullsized":"2024-04-25T14:06:46.544302+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:16.407519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:47.113020+0000","last_change":"2024-04-25T14:06:44.548815+0000","last_active":"2024-04-25T14:06:47.113020+0000","last_peered":"2024-04-25T14:06:47.113020+0000","last_clean":"2024-04-25T14:06:47.113020+0000","last_became_active":"2024-04-25T14:06:44.548469+0000","last_became_peered":"2024-04-25T14:06:44.548469+0000","last_unstale":"2024-04-25T14:06:47.113020+0000","last_undegraded":"2024-04-25T14:06:47.113020+0000","last_fullsized":"2024-04-25T14:06:47.113020+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:47.507460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.546676+0000","last_change":"2024-04-25T14:06:45.544572+0000","last_active":"2024-04-25T14:06:46.546676+0000","last_peered":"2024-04-25T14:06:46.546676+0000","last_clean":"2024-04-25T14:06:46.546676+0000","last_became_active":"2024-04-25T14:06:45.544222+0000","last_became_peered":"2024-04-25T14:06:45.544222+0000","last_unstale":"2024-04-25T14:06:46.546676+0000","last_undegraded":"2024-04-25T14:06:46.546676+0000","last_fullsized":"2024-04-25T14:06:46.546676+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:17.324021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.547539+0000","last_change":"2024-04-25T14:06:44.540795+0000","last_active":"2024-04-25T14:06:46.547539+0000","last_peered":"2024-04-25T14:06:46.547539+0000","last_clean":"2024-04-25T14:06:46.547539+0000","last_became_active":"2024-04-25T14:06:44.540467+0000","last_became_peered":"2024-04-25T14:06:44.540467+0000","last_unstale":"2024-04-25T14:06:46.547539+0000","last_undegraded":"2024-04-25T14:06:46.547539+0000","last_fullsized":"2024-04-25T14:06:46.547539+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:46.781154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.695677+0000","last_change":"2024-04-25T14:06:45.565925+0000","last_active":"2024-04-25T14:06:46.695677+0000","last_peered":"2024-04-25T14:06:46.695677+0000","last_clean":"2024-04-25T14:06:46.695677+0000","last_became_active":"2024-04-25T14:06:45.565549+0000","last_became_peered":"2024-04-25T14:06:45.565549+0000","last_unstale":"2024-04-25T14:06:46.695677+0000","last_undegraded":"2024-04-25T14:06:46.695677+0000","last_fullsized":"2024-04-25T14:06:46.695677+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:52.162725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"73'3","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.555926+0000","last_change":"2024-04-25T14:06:44.544689+0000","last_active":"2024-04-25T14:06:46.555926+0000","last_peered":"2024-04-25T14:06:46.555926+0000","last_clean":"2024-04-25T14:06:46.555926+0000","last_became_active":"2024-04-25T14:06:44.544095+0000","last_became_peered":"2024-04-25T14:06:44.544095+0000","last_unstale":"2024-04-25T14:06:46.555926+0000","last_undegraded":"2024-04-25T14:06:46.555926+0000","last_fullsized":"2024-04-25T14:06:46.555926+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:55.665025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:47.111717+0000","last_change":"2024-04-25T14:06:45.554799+0000","last_active":"2024-04-25T14:06:47.111717+0000","last_peered":"2024-04-25T14:06:47.111717+0000","last_clean":"2024-04-25T14:06:47.111717+0000","last_became_active":"2024-04-25T14:06:45.554574+0000","last_became_peered":"2024-04-25T14:06:45.554574+0000","last_unstale":"2024-04-25T14:06:47.111717+0000","last_undegraded":"2024-04-25T14:06:47.111717+0000","last_fullsized":"2024-04-25T14:06:47.111717+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:28.983868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"73'6","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.561573+0000","last_change":"2024-04-25T14:06:44.545874+0000","last_active":"2024-04-25T14:06:46.561573+0000","last_peered":"2024-04-25T14:06:46.561573+0000","last_clean":"2024-04-25T14:06:46.561573+0000","last_became_active":"2024-04-25T14:06:44.545363+0000","last_became_peered":"2024-04-25T14:06:44.545363+0000","last_unstale":"2024-04-25T14:06:46.561573+0000","last_undegraded":"2024-04-25T14:06:46.561573+0000","last_fullsized":"2024-04-25T14:06:46.561573+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:56.433866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.547159+0000","last_change":"2024-04-25T14:06:45.546882+0000","last_active":"2024-04-25T14:06:46.547159+0000","last_peered":"2024-04-25T14:06:46.547159+0000","last_clean":"2024-04-25T14:06:46.547159+0000","last_became_active":"2024-04-25T14:06:45.545835+0000","last_became_peered":"2024-04-25T14:06:45.545835+0000","last_unstale":"2024-04-25T14:06:46.547159+0000","last_undegraded":"2024-04-25T14:06:46.547159+0000","last_fullsized":"2024-04-25T14:06:46.547159+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:17.074961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.546400+0000","last_change":"2024-04-25T14:06:44.533506+0000","last_active":"2024-04-25T14:06:46.546400+0000","last_peered":"2024-04-25T14:06:46.546400+0000","last_clean":"2024-04-25T14:06:46.546400+0000","last_became_active":"2024-04-25T14:06:44.533189+0000","last_became_peered":"2024-04-25T14:06:44.533189+0000","last_unstale":"2024-04-25T14:06:46.546400+0000","last_undegraded":"2024-04-25T14:06:46.546400+0000","last_fullsized":"2024-04-25T14:06:46.546400+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:56.180081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.561371+0000","last_change":"2024-04-25T14:06:45.558283+0000","last_active":"2024-04-25T14:06:46.561371+0000","last_peered":"2024-04-25T14:06:46.561371+0000","last_clean":"2024-04-25T14:06:46.561371+0000","last_became_active":"2024-04-25T14:06:45.557897+0000","last_became_peered":"2024-04-25T14:06:45.557897+0000","last_unstale":"2024-04-25T14:06:46.561371+0000","last_undegraded":"2024-04-25T14:06:46.561371+0000","last_fullsized":"2024-04-25T14:06:46.561371+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:23.790336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:47.112571+0000","last_change":"2024-04-25T14:06:44.544582+0000","last_active":"2024-04-25T14:06:47.112571+0000","last_peered":"2024-04-25T14:06:47.112571+0000","last_clean":"2024-04-25T14:06:47.112571+0000","last_became_active":"2024-04-25T14:06:44.544049+0000","last_became_peered":"2024-04-25T14:06:44.544049+0000","last_unstale":"2024-04-25T14:06:47.112571+0000","last_undegraded":"2024-04-25T14:06:47.112571+0000","last_fullsized":"2024-04-25T14:06:47.112571+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:58.086378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.561721+0000","last_change":"2024-04-25T14:06:45.546382+0000","last_active":"2024-04-25T14:06:46.561721+0000","last_peered":"2024-04-25T14:06:46.561721+0000","last_clean":"2024-04-25T14:06:46.561721+0000","last_became_active":"2024-04-25T14:06:45.546045+0000","last_became_peered":"2024-04-25T14:06:45.546045+0000","last_unstale":"2024-04-25T14:06:46.561721+0000","last_undegraded":"2024-04-25T14:06:46.561721+0000","last_fullsized":"2024-04-25T14:06:46.561721+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:48.517004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.555326+0000","last_change":"2024-04-25T14:06:44.534032+0000","last_active":"2024-04-25T14:06:46.555326+0000","last_peered":"2024-04-25T14:06:46.555326+0000","last_clean":"2024-04-25T14:06:46.555326+0000","last_became_active":"2024-04-25T14:06:44.533673+0000","last_became_peered":"2024-04-25T14:06:44.533673+0000","last_unstale":"2024-04-25T14:06:46.555326+0000","last_undegraded":"2024-04-25T14:06:46.555326+0000","last_fullsized":"2024-04-25T14:06:46.555326+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:58.787353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.561866+0000","last_change":"2024-04-25T14:06:45.546221+0000","last_active":"2024-04-25T14:06:46.561866+0000","last_peered":"2024-04-25T14:06:46.561866+0000","last_clean":"2024-04-25T14:06:46.561866+0000","last_became_active":"2024-04-25T14:06:45.545126+0000","last_became_peered":"2024-04-25T14:06:45.545126+0000","last_unstale":"2024-04-25T14:06:46.561866+0000","last_undegraded":"2024-04-25T14:06:46.561866+0000","last_fullsized":"2024-04-25T14:06:46.561866+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:15.245688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"73'8","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.544468+0000","last_change":"2024-04-25T14:06:44.538705+0000","last_active":"2024-04-25T14:06:46.544468+0000","last_peered":"2024-04-25T14:06:46.544468+0000","last_clean":"2024-04-25T14:06:46.544468+0000","last_became_active":"2024-04-25T14:06:44.538404+0000","last_became_peered":"2024-04-25T14:06:44.538404+0000","last_unstale":"2024-04-25T14:06:46.544468+0000","last_undegraded":"2024-04-25T14:06:46.544468+0000","last_fullsized":"2024-04-25T14:06:46.544468+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:25.112332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.695290+0000","last_change":"2024-04-25T14:06:45.562634+0000","last_active":"2024-04-25T14:06:46.695290+0000","last_peered":"2024-04-25T14:06:46.695290+0000","last_clean":"2024-04-25T14:06:46.695290+0000","last_became_active":"2024-04-25T14:06:45.562313+0000","last_became_peered":"2024-04-25T14:06:45.562313+0000","last_unstale":"2024-04-25T14:06:46.695290+0000","last_undegraded":"2024-04-25T14:06:46.695290+0000","last_fullsized":"2024-04-25T14:06:46.695290+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:35.960539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:47.113118+0000","last_change":"2024-04-25T14:06:44.547295+0000","last_active":"2024-04-25T14:06:47.113118+0000","last_peered":"2024-04-25T14:06:47.113118+0000","last_clean":"2024-04-25T14:06:47.113118+0000","last_became_active":"2024-04-25T14:06:44.546930+0000","last_became_peered":"2024-04-25T14:06:44.546930+0000","last_unstale":"2024-04-25T14:06:47.113118+0000","last_undegraded":"2024-04-25T14:06:47.113118+0000","last_fullsized":"2024-04-25T14:06:47.113118+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:09.304323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.555385+0000","last_change":"2024-04-25T14:06:45.549948+0000","last_active":"2024-04-25T14:06:46.555385+0000","last_peered":"2024-04-25T14:06:46.555385+0000","last_clean":"2024-04-25T14:06:46.555385+0000","last_became_active":"2024-04-25T14:06:45.549627+0000","last_became_peered":"2024-04-25T14:06:45.549627+0000","last_unstale":"2024-04-25T14:06:46.555385+0000","last_undegraded":"2024-04-25T14:06:46.555385+0000","last_fullsized":"2024-04-25T14:06:46.555385+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:13.511832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.546160+0000","last_change":"2024-04-25T14:06:44.543006+0000","last_active":"2024-04-25T14:06:46.546160+0000","last_peered":"2024-04-25T14:06:46.546160+0000","last_clean":"2024-04-25T14:06:46.546160+0000","last_became_active":"2024-04-25T14:06:44.542678+0000","last_became_peered":"2024-04-25T14:06:44.542678+0000","last_unstale":"2024-04-25T14:06:46.546160+0000","last_undegraded":"2024-04-25T14:06:46.546160+0000","last_fullsized":"2024-04-25T14:06:46.546160+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:02.489582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.555482+0000","last_change":"2024-04-25T14:06:45.557858+0000","last_active":"2024-04-25T14:06:46.555482+0000","last_peered":"2024-04-25T14:06:46.555482+0000","last_clean":"2024-04-25T14:06:46.555482+0000","last_became_active":"2024-04-25T14:06:45.556716+0000","last_became_peered":"2024-04-25T14:06:45.556716+0000","last_unstale":"2024-04-25T14:06:46.555482+0000","last_undegraded":"2024-04-25T14:06:46.555482+0000","last_fullsized":"2024-04-25T14:06:46.555482+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:45.971264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.555954+0000","last_change":"2024-04-25T14:06:44.541885+0000","last_active":"2024-04-25T14:06:46.555954+0000","last_peered":"2024-04-25T14:06:46.555954+0000","last_clean":"2024-04-25T14:06:46.555954+0000","last_became_active":"2024-04-25T14:06:44.541501+0000","last_became_peered":"2024-04-25T14:06:44.541501+0000","last_unstale":"2024-04-25T14:06:46.555954+0000","last_undegraded":"2024-04-25T14:06:46.555954+0000","last_fullsized":"2024-04-25T14:06:46.555954+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:14.986429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.695450+0000","last_change":"2024-04-25T14:06:45.560159+0000","last_active":"2024-04-25T14:06:46.695450+0000","last_peered":"2024-04-25T14:06:46.695450+0000","last_clean":"2024-04-25T14:06:46.695450+0000","last_became_active":"2024-04-25T14:06:45.559801+0000","last_became_peered":"2024-04-25T14:06:45.559801+0000","last_unstale":"2024-04-25T14:06:46.695450+0000","last_undegraded":"2024-04-25T14:06:46.695450+0000","last_fullsized":"2024-04-25T14:06:46.695450+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:27.268956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:47.111780+0000","last_change":"2024-04-25T14:06:44.544634+0000","last_active":"2024-04-25T14:06:47.111780+0000","last_peered":"2024-04-25T14:06:47.111780+0000","last_clean":"2024-04-25T14:06:47.111780+0000","last_became_active":"2024-04-25T14:06:44.544365+0000","last_became_peered":"2024-04-25T14:06:44.544365+0000","last_unstale":"2024-04-25T14:06:47.111780+0000","last_undegraded":"2024-04-25T14:06:47.111780+0000","last_fullsized":"2024-04-25T14:06:47.111780+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:27.921735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.544763+0000","last_change":"2024-04-25T14:06:45.552416+0000","last_active":"2024-04-25T14:06:46.544763+0000","last_peered":"2024-04-25T14:06:46.544763+0000","last_clean":"2024-04-25T14:06:46.544763+0000","last_became_active":"2024-04-25T14:06:45.552104+0000","last_became_peered":"2024-04-25T14:06:45.552104+0000","last_unstale":"2024-04-25T14:06:46.544763+0000","last_undegraded":"2024-04-25T14:06:46.544763+0000","last_fullsized":"2024-04-25T14:06:46.544763+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:43.672342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.695434+0000","last_change":"2024-04-25T14:06:44.541617+0000","last_active":"2024-04-25T14:06:46.695434+0000","last_peered":"2024-04-25T14:06:46.695434+0000","last_clean":"2024-04-25T14:06:46.695434+0000","last_became_active":"2024-04-25T14:06:44.540854+0000","last_became_peered":"2024-04-25T14:06:44.540854+0000","last_unstale":"2024-04-25T14:06:46.695434+0000","last_undegraded":"2024-04-25T14:06:46.695434+0000","last_fullsized":"2024-04-25T14:06:46.695434+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:16.314477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.545111+0000","last_change":"2024-04-25T14:06:45.559078+0000","last_active":"2024-04-25T14:06:46.545111+0000","last_peered":"2024-04-25T14:06:46.545111+0000","last_clean":"2024-04-25T14:06:46.545111+0000","last_became_active":"2024-04-25T14:06:45.558718+0000","last_became_peered":"2024-04-25T14:06:45.558718+0000","last_unstale":"2024-04-25T14:06:46.545111+0000","last_undegraded":"2024-04-25T14:06:46.545111+0000","last_fullsized":"2024-04-25T14:06:46.545111+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:18.873956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.695273+0000","last_change":"2024-04-25T14:06:44.537769+0000","last_active":"2024-04-25T14:06:46.695273+0000","last_peered":"2024-04-25T14:06:46.695273+0000","last_clean":"2024-04-25T14:06:46.695273+0000","last_became_active":"2024-04-25T14:06:44.537407+0000","last_became_peered":"2024-04-25T14:06:44.537407+0000","last_unstale":"2024-04-25T14:06:46.695273+0000","last_undegraded":"2024-04-25T14:06:46.695273+0000","last_fullsized":"2024-04-25T14:06:46.695273+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:42.088259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.561952+0000","last_change":"2024-04-25T14:06:45.551151+0000","last_active":"2024-04-25T14:06:46.561952+0000","last_peered":"2024-04-25T14:06:46.561952+0000","last_clean":"2024-04-25T14:06:46.561952+0000","last_became_active":"2024-04-25T14:06:45.550709+0000","last_became_peered":"2024-04-25T14:06:45.550709+0000","last_unstale":"2024-04-25T14:06:46.561952+0000","last_undegraded":"2024-04-25T14:06:46.561952+0000","last_fullsized":"2024-04-25T14:06:46.561952+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:07.130854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.695229+0000","last_change":"2024-04-25T14:06:44.538743+0000","last_active":"2024-04-25T14:06:46.695229+0000","last_peered":"2024-04-25T14:06:46.695229+0000","last_clean":"2024-04-25T14:06:46.695229+0000","last_became_active":"2024-04-25T14:06:44.538365+0000","last_became_peered":"2024-04-25T14:06:44.538365+0000","last_unstale":"2024-04-25T14:06:46.695229+0000","last_undegraded":"2024-04-25T14:06:46.695229+0000","last_fullsized":"2024-04-25T14:06:46.695229+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:06.121310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.545579+0000","last_change":"2024-04-25T14:06:45.554616+0000","last_active":"2024-04-25T14:06:46.545579+0000","last_peered":"2024-04-25T14:06:46.545579+0000","last_clean":"2024-04-25T14:06:46.545579+0000","last_became_active":"2024-04-25T14:06:45.554242+0000","last_became_peered":"2024-04-25T14:06:45.554242+0000","last_unstale":"2024-04-25T14:06:46.545579+0000","last_undegraded":"2024-04-25T14:06:46.545579+0000","last_fullsized":"2024-04-25T14:06:46.545579+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:43:38.769653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.547358+0000","last_change":"2024-04-25T14:06:44.541609+0000","last_active":"2024-04-25T14:06:46.547358+0000","last_peered":"2024-04-25T14:06:46.547358+0000","last_clean":"2024-04-25T14:06:46.547358+0000","last_became_active":"2024-04-25T14:06:44.541009+0000","last_became_peered":"2024-04-25T14:06:44.541009+0000","last_unstale":"2024-04-25T14:06:46.547358+0000","last_undegraded":"2024-04-25T14:06:46.547358+0000","last_fullsized":"2024-04-25T14:06:46.547358+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:27.974323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.561414+0000","last_change":"2024-04-25T14:06:45.547082+0000","last_active":"2024-04-25T14:06:46.561414+0000","last_peered":"2024-04-25T14:06:46.561414+0000","last_clean":"2024-04-25T14:06:46.561414+0000","last_became_active":"2024-04-25T14:06:45.546762+0000","last_became_peered":"2024-04-25T14:06:45.546762+0000","last_unstale":"2024-04-25T14:06:46.561414+0000","last_undegraded":"2024-04-25T14:06:46.561414+0000","last_fullsized":"2024-04-25T14:06:46.561414+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:28.983868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"74'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.545177+0000","last_change":"2024-04-25T14:06:44.535705+0000","last_active":"2024-04-25T14:06:46.545177+0000","last_peered":"2024-04-25T14:06:46.545177+0000","last_clean":"2024-04-25T14:06:46.545177+0000","last_became_active":"2024-04-25T14:06:44.535457+0000","last_became_peered":"2024-04-25T14:06:44.535457+0000","last_unstale":"2024-04-25T14:06:46.545177+0000","last_undegraded":"2024-04-25T14:06:46.545177+0000","last_fullsized":"2024-04-25T14:06:46.545177+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:16.065417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.546239+0000","last_change":"2024-04-25T14:06:45.548981+0000","last_active":"2024-04-25T14:06:46.546239+0000","last_peered":"2024-04-25T14:06:46.546239+0000","last_clean":"2024-04-25T14:06:46.546239+0000","last_became_active":"2024-04-25T14:06:45.548223+0000","last_became_peered":"2024-04-25T14:06:45.548223+0000","last_unstale":"2024-04-25T14:06:46.546239+0000","last_undegraded":"2024-04-25T14:06:46.546239+0000","last_fullsized":"2024-04-25T14:06:46.546239+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:35.170216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:47.112942+0000","last_change":"2024-04-25T14:06:44.542180+0000","last_active":"2024-04-25T14:06:47.112942+0000","last_peered":"2024-04-25T14:06:47.112942+0000","last_clean":"2024-04-25T14:06:47.112942+0000","last_became_active":"2024-04-25T14:06:44.540654+0000","last_became_peered":"2024-04-25T14:06:44.540654+0000","last_unstale":"2024-04-25T14:06:47.112942+0000","last_undegraded":"2024-04-25T14:06:47.112942+0000","last_fullsized":"2024-04-25T14:06:47.112942+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:16.314477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.544764+0000","last_change":"2024-04-25T14:06:45.555860+0000","last_active":"2024-04-25T14:06:46.544764+0000","last_peered":"2024-04-25T14:06:46.544764+0000","last_clean":"2024-04-25T14:06:46.544764+0000","last_became_active":"2024-04-25T14:06:45.555511+0000","last_became_peered":"2024-04-25T14:06:45.555511+0000","last_unstale":"2024-04-25T14:06:46.544764+0000","last_undegraded":"2024-04-25T14:06:46.544764+0000","last_fullsized":"2024-04-25T14:06:46.544764+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:15.471101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:47.112624+0000","last_change":"2024-04-25T14:06:44.543878+0000","last_active":"2024-04-25T14:06:47.112624+0000","last_peered":"2024-04-25T14:06:47.112624+0000","last_clean":"2024-04-25T14:06:47.112624+0000","last_became_active":"2024-04-25T14:06:44.543347+0000","last_became_peered":"2024-04-25T14:06:44.543347+0000","last_unstale":"2024-04-25T14:06:47.112624+0000","last_undegraded":"2024-04-25T14:06:47.112624+0000","last_fullsized":"2024-04-25T14:06:47.112624+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:14.236144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.561353+0000","last_change":"2024-04-25T14:06:45.545647+0000","last_active":"2024-04-25T14:06:46.561353+0000","last_peered":"2024-04-25T14:06:46.561353+0000","last_clean":"2024-04-25T14:06:46.561353+0000","last_became_active":"2024-04-25T14:06:45.545298+0000","last_became_peered":"2024-04-25T14:06:45.545298+0000","last_unstale":"2024-04-25T14:06:46.561353+0000","last_undegraded":"2024-04-25T14:06:46.561353+0000","last_fullsized":"2024-04-25T14:06:46.561353+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:59.095922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.561291+0000","last_change":"2024-04-25T14:06:44.546730+0000","last_active":"2024-04-25T14:06:46.561291+0000","last_peered":"2024-04-25T14:06:46.561291+0000","last_clean":"2024-04-25T14:06:46.561291+0000","last_became_active":"2024-04-25T14:06:44.546050+0000","last_became_peered":"2024-04-25T14:06:44.546050+0000","last_unstale":"2024-04-25T14:06:46.561291+0000","last_undegraded":"2024-04-25T14:06:46.561291+0000","last_fullsized":"2024-04-25T14:06:46.561291+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:54:29.024591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.561568+0000","last_change":"2024-04-25T14:06:45.560486+0000","last_active":"2024-04-25T14:06:46.561568+0000","last_peered":"2024-04-25T14:06:46.561568+0000","last_clean":"2024-04-25T14:06:46.561568+0000","last_became_active":"2024-04-25T14:06:45.560182+0000","last_became_peered":"2024-04-25T14:06:45.560182+0000","last_unstale":"2024-04-25T14:06:46.561568+0000","last_undegraded":"2024-04-25T14:06:46.561568+0000","last_fullsized":"2024-04-25T14:06:46.561568+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:25.253476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.555588+0000","last_change":"2024-04-25T14:06:44.534134+0000","last_active":"2024-04-25T14:06:46.555588+0000","last_peered":"2024-04-25T14:06:46.555588+0000","last_clean":"2024-04-25T14:06:46.555588+0000","last_became_active":"2024-04-25T14:06:44.533705+0000","last_became_peered":"2024-04-25T14:06:44.533705+0000","last_unstale":"2024-04-25T14:06:46.555588+0000","last_undegraded":"2024-04-25T14:06:46.555588+0000","last_fullsized":"2024-04-25T14:06:46.555588+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:18.886571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:47.111938+0000","last_change":"2024-04-25T14:06:45.562004+0000","last_active":"2024-04-25T14:06:47.111938+0000","last_peered":"2024-04-25T14:06:47.111938+0000","last_clean":"2024-04-25T14:06:47.111938+0000","last_became_active":"2024-04-25T14:06:45.561313+0000","last_became_peered":"2024-04-25T14:06:45.561313+0000","last_unstale":"2024-04-25T14:06:47.111938+0000","last_undegraded":"2024-04-25T14:06:47.111938+0000","last_fullsized":"2024-04-25T14:06:47.111938+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:47.790698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.561638+0000","last_change":"2024-04-25T14:06:44.538191+0000","last_active":"2024-04-25T14:06:46.561638+0000","last_peered":"2024-04-25T14:06:46.561638+0000","last_clean":"2024-04-25T14:06:46.561638+0000","last_became_active":"2024-04-25T14:06:44.537838+0000","last_became_peered":"2024-04-25T14:06:44.537838+0000","last_unstale":"2024-04-25T14:06:46.561638+0000","last_undegraded":"2024-04-25T14:06:46.561638+0000","last_fullsized":"2024-04-25T14:06:46.561638+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:43:37.760109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:47.111978+0000","last_change":"2024-04-25T14:06:45.564623+0000","last_active":"2024-04-25T14:06:47.111978+0000","last_peered":"2024-04-25T14:06:47.111978+0000","last_clean":"2024-04-25T14:06:47.111978+0000","last_became_active":"2024-04-25T14:06:45.561137+0000","last_became_peered":"2024-04-25T14:06:45.561137+0000","last_unstale":"2024-04-25T14:06:47.111978+0000","last_undegraded":"2024-04-25T14:06:47.111978+0000","last_fullsized":"2024-04-25T14:06:47.111978+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:59.095922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:47.111513+0000","last_change":"2024-04-25T14:06:44.539538+0000","last_active":"2024-04-25T14:06:47.111513+0000","last_peered":"2024-04-25T14:06:47.111513+0000","last_clean":"2024-04-25T14:06:47.111513+0000","last_became_active":"2024-04-25T14:06:44.538128+0000","last_became_peered":"2024-04-25T14:06:44.538128+0000","last_unstale":"2024-04-25T14:06:47.111513+0000","last_undegraded":"2024-04-25T14:06:47.111513+0000","last_fullsized":"2024-04-25T14:06:47.111513+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:43:37.760109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:47.112986+0000","last_change":"2024-04-25T14:06:45.556869+0000","last_active":"2024-04-25T14:06:47.112986+0000","last_peered":"2024-04-25T14:06:47.112986+0000","last_clean":"2024-04-25T14:06:47.112986+0000","last_became_active":"2024-04-25T14:06:45.556546+0000","last_became_peered":"2024-04-25T14:06:45.556546+0000","last_unstale":"2024-04-25T14:06:47.112986+0000","last_undegraded":"2024-04-25T14:06:47.112986+0000","last_fullsized":"2024-04-25T14:06:47.112986+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:44.508104+0000","last_clean_scrub_stamp":"2024-04-25T14:06:44.508104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:27.268956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:46.695662+0000","last_change":"2024-04-25T14:06:44.539206+0000","last_active":"2024-04-25T14:06:46.695662+0000","last_peered":"2024-04-25T14:06:46.695662+0000","last_clean":"2024-04-25T14:06:46.695662+0000","last_became_active":"2024-04-25T14:06:44.538913+0000","last_became_peered":"2024-04-25T14:06:44.538913+0000","last_unstale":"2024-04-25T14:06:46.695662+0000","last_undegraded":"2024-04-25T14:06:46.695662+0000","last_fullsized":"2024-04-25T14:06:46.695662+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:06:43.498560+0000","last_clean_scrub_stamp":"2024-04-25T14:06:43.498560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:25.112332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"59'86","reported_seq":259,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T14:06:47.112734+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:06:47.112734+0000","last_peered":"2024-04-25T14:06:47.112734+0000","last_clean":"2024-04-25T14:06:47.112734+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:06:47.112734+0000","last_undegraded":"2024-04-25T14:06:47.112734+0000","last_fullsized":"2024-04-25T14:06:47.112734+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1503232,"data_stored":1037851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":917504,"data_stored":449073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":909312,"data_stored":447495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":925696,"data_stored":449177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":909312,"data_stored":448043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":917504,"data_stored":448203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":913408,"data_stored":447573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1499136,"data_stored":1037829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:06:54.619 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T14:06:54.619 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T14:06:54.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.811+0000 7f0102200640 1 Processor -- start 2024-04-25T14:06:54.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.811+0000 7f0102200640 1 -- start start 2024-04-25T14:06:54.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.811+0000 7f0102200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f00fc070330 0x7f00fc0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:54.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.811+0000 7f0102200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00fc0ce790 0x7f00fc06f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:54.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.811+0000 7f0102200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00fc0d2030 con 0x7f00fc0d1c20 2024-04-25T14:06:54.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.811+0000 7f0102200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f00fc15e5c0 con 0x7f00fc070330 2024-04-25T14:06:54.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.811+0000 7f0102200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f00fc15e8a0 con 0x7f00fc0ce790 2024-04-25T14:06:54.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.811+0000 7f00fb400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f00fc070330 0x7f00fc0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:54.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.811+0000 7f00fbe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00fc0ce790 0x7f00fc06f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:54.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.811+0000 7f00fb400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f00fc070330 0x7f00fc0d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:43270/0 (socket says 172.21.15.86:43270) 2024-04-25T14:06:54.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.811+0000 7f00f2a00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f00fc0d1c20 0x7f00fc06ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:44988/0 (socket says 172.21.15.86:44988) 2024-04-25T14:06:54.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.811+0000 7f00fb400640 1 -- 172.21.15.86:0/3652850051 learned_addr learned my addr 172.21.15.86:0/3652850051 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:54.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.811+0000 7f00fbe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00fc0ce790 0x7f00fc06f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:45950/0 (socket says 172.21.15.86:45950) 2024-04-25T14:06:54.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.811+0000 7f00fb400640 1 -- 172.21.15.86:0/3652850051 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00fc0ce790 msgr2=0x7f00fc06f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:54.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.811+0000 7f00fb400640 1 --2- 172.21.15.86:0/3652850051 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00fc0ce790 0x7f00fc06f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:54.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.811+0000 7f00fb400640 1 -- 172.21.15.86:0/3652850051 >> v1:172.21.15.155:6789/0 conn(0x7f00fc0d1c20 legacy=0x7f00fc06ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:54.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.811+0000 7f00fb400640 1 -- 172.21.15.86:0/3652850051 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00fc15eb80 con 0x7f00fc070330 2024-04-25T14:06:54.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.811+0000 7f00faa00640 1 -- 172.21.15.86:0/3652850051 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4155491224 0 0) 0x7f00e4001120 con 0x7f00fc0d1c20 2024-04-25T14:06:54.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.815+0000 7f00fb400640 1 --2- 172.21.15.86:0/3652850051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00fc070330 0x7f00fc0d16e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f00ec009870 tx=0x7f00ec01c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea75b681826ec891 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:54.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.815+0000 7f00faa00640 1 -- 172.21.15.86:0/3652850051 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec0b0070 con 0x7f00fc070330 2024-04-25T14:06:54.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.815+0000 7f00faa00640 1 -- 172.21.15.86:0/3652850051 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00ec0ab600 con 0x7f00fc070330 2024-04-25T14:06:54.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.815+0000 7f00faa00640 1 -- 172.21.15.86:0/3652850051 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec0ab9d0 con 0x7f00fc070330 2024-04-25T14:06:54.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.815+0000 7f0102200640 1 -- 172.21.15.86:0/3652850051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00fc070330 msgr2=0x7f00fc0d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:54.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.815+0000 7f0102200640 1 --2- 172.21.15.86:0/3652850051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00fc070330 0x7f00fc0d16e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f00ec009870 tx=0x7f00ec01c4d0 comp rx=0 tx=0).stop 2024-04-25T14:06:54.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.815+0000 7f0102200640 1 -- 172.21.15.86:0/3652850051 shutdown_connections 2024-04-25T14:06:54.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.815+0000 7f0102200640 1 --2- 172.21.15.86:0/3652850051 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00fc0ce790 0x7f00fc06f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:54.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.815+0000 7f0102200640 1 --2- 172.21.15.86:0/3652850051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00fc070330 0x7f00fc0d16e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:54.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.815+0000 7f0102200640 1 -- 172.21.15.86:0/3652850051 >> 172.21.15.86:0/3652850051 conn(0x7f00fc0b4650 msgr2=0x7f00fc0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:54.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.815+0000 7f0102200640 1 -- 172.21.15.86:0/3652850051 shutdown_connections 2024-04-25T14:06:54.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.815+0000 7f0102200640 1 -- 172.21.15.86:0/3652850051 wait complete. 2024-04-25T14:06:54.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.819+0000 7f0102200640 1 Processor -- start 2024-04-25T14:06:54.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.819+0000 7f0102200640 1 -- start start 2024-04-25T14:06:54.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.819+0000 7f0102200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00fc070330 0x7f00fc15c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:54.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.819+0000 7f0102200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00fc0ce790 0x7f00fc15b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:54.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.819+0000 7f0102200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00fc0d1c20 0x7f00fc163bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:54.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.819+0000 7f0102200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc0708c0 con 0x7f00fc070330 2024-04-25T14:06:54.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.819+0000 7f0102200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc070740 con 0x7f00fc0ce790 2024-04-25T14:06:54.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.819+0000 7f0102200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f00fc16afc0 con 0x7f00fc0d1c20 2024-04-25T14:06:54.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.819+0000 7f00fb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00fc070330 0x7f00fc15c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:54.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.819+0000 7f00fb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00fc070330 0x7f00fc15c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56526/0 (socket says 172.21.15.86:56526) 2024-04-25T14:06:54.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.819+0000 7f00fb400640 1 -- 172.21.15.86:0/3203469730 learned_addr learned my addr 172.21.15.86:0/3203469730 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:54.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.819+0000 7f00fbe00640 1 --2- 172.21.15.86:0/3203469730 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00fc0d1c20 0x7f00fc163bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:54.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.819+0000 7f00f2a00640 1 --2- 172.21.15.86:0/3203469730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00fc0ce790 0x7f00fc15b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:54.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.819+0000 7f00fb400640 1 -- 172.21.15.86:0/3203469730 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00fc0d1c20 msgr2=0x7f00fc163bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:54.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.819+0000 7f00fb400640 1 --2- 172.21.15.86:0/3203469730 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00fc0d1c20 0x7f00fc163bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:54.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.819+0000 7f00fb400640 1 -- 172.21.15.86:0/3203469730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00fc0ce790 msgr2=0x7f00fc15b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:54.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.819+0000 7f00fb400640 1 --2- 172.21.15.86:0/3203469730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00fc0ce790 0x7f00fc15b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:54.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.819+0000 7f00fb400640 1 -- 172.21.15.86:0/3203469730 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00fc15ccc0 con 0x7f00fc070330 2024-04-25T14:06:54.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.819+0000 7f00f2a00640 1 --2- 172.21.15.86:0/3203469730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00fc0ce790 0x7f00fc15b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:54.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.823+0000 7f00fb400640 1 --2- 172.21.15.86:0/3203469730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00fc070330 0x7f00fc15c780 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f00ec01a100 tx=0x7f00ec01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:54.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.823+0000 7f00f3e00640 1 -- 172.21.15.86:0/3203469730 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec0b0040 con 0x7f00fc070330 2024-04-25T14:06:54.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.823+0000 7f00f3e00640 1 -- 172.21.15.86:0/3203469730 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00ec0bc070 con 0x7f00fc070330 2024-04-25T14:06:54.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.823+0000 7f0102200640 1 -- 172.21.15.86:0/3203469730 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00fc15d1d0 con 0x7f00fc070330 2024-04-25T14:06:54.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.823+0000 7f00f3e00640 1 -- 172.21.15.86:0/3203469730 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec0b6640 con 0x7f00fc070330 2024-04-25T14:06:54.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.823+0000 7f0102200640 1 -- 172.21.15.86:0/3203469730 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00fc166130 con 0x7f00fc070330 2024-04-25T14:06:54.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.823+0000 7f0102200640 1 -- 172.21.15.86:0/3203469730 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00fc069d00 con 0x7f00fc070330 2024-04-25T14:06:54.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.823+0000 7f00f3e00640 1 -- 172.21.15.86:0/3203469730 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f00ec0b67e0 con 0x7f00fc070330 2024-04-25T14:06:54.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.823+0000 7f00f3e00640 1 --2- 172.21.15.86:0/3203469730 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f00d806f640 0x7f00d8071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:54.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.823+0000 7f00f3e00640 1 -- 172.21.15.86:0/3203469730 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f00ec129d50 con 0x7f00fc070330 2024-04-25T14:06:54.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.823+0000 7f00f2a00640 1 --2- 172.21.15.86:0/3203469730 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f00d806f640 0x7f00d8071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:54.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.827+0000 7f00f2a00640 1 --2- 172.21.15.86:0/3203469730 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f00d806f640 0x7f00d8071b50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0b2660 tx=0x7f00e4009040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:54.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:54.831+0000 7f00f3e00640 1 -- 172.21.15.86:0/3203469730 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f00ec016150 con 0x7f00fc070330 2024-04-25T14:06:55.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.039+0000 7f0102200640 1 -- 172.21.15.86:0/3203469730 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f00fc076bf0 con 0x7f00fc070330 2024-04-25T14:06:55.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.043+0000 7f00f3e00640 1 -- 172.21.15.86:0/3203469730 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) ==== 190+0+147 (secure 0 0 0) 0x7f00ec0f65f0 con 0x7f00fc070330 2024-04-25T14:06:55.051 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-04-25T14:06:55.054 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCLYipm5u4xHRAACDFe3Eb4KLrsdLj4eFe7mA== 2024-04-25T14:06:55.054 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-04-25T14:06:55.054 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-04-25T14:06:55.054 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T14:06:55.055 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.0 2024-04-25T14:06:55.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.047+0000 7f0102200640 1 -- 172.21.15.86:0/3203469730 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f00d806f640 msgr2=0x7f00d8071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:55.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.047+0000 7f0102200640 1 --2- 172.21.15.86:0/3203469730 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f00d806f640 0x7f00d8071b50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0b2660 tx=0x7f00e4009040 comp rx=0 tx=0).stop 2024-04-25T14:06:55.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.047+0000 7f0102200640 1 -- 172.21.15.86:0/3203469730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00fc070330 msgr2=0x7f00fc15c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:55.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.047+0000 7f0102200640 1 --2- 172.21.15.86:0/3203469730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00fc070330 0x7f00fc15c780 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f00ec01a100 tx=0x7f00ec01ad10 comp rx=0 tx=0).stop 2024-04-25T14:06:55.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.047+0000 7f0102200640 1 -- 172.21.15.86:0/3203469730 shutdown_connections 2024-04-25T14:06:55.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.047+0000 7f0102200640 1 --2- 172.21.15.86:0/3203469730 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f00d806f640 0x7f00d8071b50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:55.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.047+0000 7f0102200640 1 --2- 172.21.15.86:0/3203469730 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00fc0d1c20 0x7f00fc163bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:55.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.047+0000 7f0102200640 1 --2- 172.21.15.86:0/3203469730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00fc0ce790 0x7f00fc15b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:55.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.047+0000 7f0102200640 1 --2- 172.21.15.86:0/3203469730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00fc070330 0x7f00fc15c780 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:55.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.047+0000 7f0102200640 1 -- 172.21.15.86:0/3203469730 >> 172.21.15.86:0/3203469730 conn(0x7f00fc0b4650 msgr2=0x7f00fc0aeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:55.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.047+0000 7f0102200640 1 -- 172.21.15.86:0/3203469730 shutdown_connections 2024-04-25T14:06:55.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.047+0000 7f0102200640 1 -- 172.21.15.86:0/3203469730 wait complete. 2024-04-25T14:06:55.071 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:06:55.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.215+0000 7f58fec00640 1 Processor -- start 2024-04-25T14:06:55.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.215+0000 7f58fec00640 1 -- start start 2024-04-25T14:06:55.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.219+0000 7f58fec00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f58f80cfca0 0x7f58f806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:55.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.219+0000 7f58fec00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f58f80730b0 0x7f58f80734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:55.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.219+0000 7f58fec00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58f8072bd0 con 0x7f58f80d16e0 2024-04-25T14:06:55.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.219+0000 7f58fec00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f58f8070470 con 0x7f58f80cfca0 2024-04-25T14:06:55.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.219+0000 7f58fec00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f58f806f930 con 0x7f58f80730b0 2024-04-25T14:06:55.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.219+0000 7f58f7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f58f80cfca0 0x7f58f806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:55.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.219+0000 7f58f7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f58f80cfca0 0x7f58f806f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56532/0 (socket says 172.21.15.86:56532) 2024-04-25T14:06:55.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.219+0000 7f58f7e00640 1 -- 172.21.15.86:0/3576041920 learned_addr learned my addr 172.21.15.86:0/3576041920 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:55.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.219+0000 7f58f7e00640 1 -- 172.21.15.86:0/3576041920 >> v2:172.21.15.155:3300/0 conn(0x7f58f80730b0 msgr2=0x7f58f80734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:55.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.219+0000 7f58fcc00640 1 --2- 172.21.15.86:0/3576041920 >> v2:172.21.15.155:3300/0 conn(0x7f58f80730b0 0x7f58f80734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:55.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.219+0000 7f58f7e00640 1 --2- 172.21.15.86:0/3576041920 >> v2:172.21.15.155:3300/0 conn(0x7f58f80730b0 0x7f58f80734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:55.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.219+0000 7f58f7e00640 1 -- 172.21.15.86:0/3576041920 >> v1:172.21.15.155:6789/0 conn(0x7f58f80d16e0 legacy=0x7f58f80d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:55.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.219+0000 7f58f7e00640 1 -- 172.21.15.86:0/3576041920 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f806fbd0 con 0x7f58f80cfca0 2024-04-25T14:06:55.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.219+0000 7f58fcc00640 1 --2- 172.21.15.86:0/3576041920 >> v2:172.21.15.155:3300/0 conn(0x7f58f80730b0 0x7f58f80734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:55.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.219+0000 7f58f7e00640 1 --2- 172.21.15.86:0/3576041920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58f80cfca0 0x7f58f806f3f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f58f80cb8c0 tx=0x7f58e801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d62fbb1a56a3dd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:55.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.223+0000 7f58f6a00640 1 -- 172.21.15.86:0/3576041920 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e80ab5f0 con 0x7f58f80cfca0 2024-04-25T14:06:55.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.223+0000 7f58f6a00640 1 -- 172.21.15.86:0/3576041920 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58e80b0070 con 0x7f58f80cfca0 2024-04-25T14:06:55.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.223+0000 7f58f6a00640 1 -- 172.21.15.86:0/3576041920 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e80ab9b0 con 0x7f58f80cfca0 2024-04-25T14:06:55.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.223+0000 7f58fec00640 1 -- 172.21.15.86:0/3576041920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58f80cfca0 msgr2=0x7f58f806f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:55.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.223+0000 7f58fec00640 1 --2- 172.21.15.86:0/3576041920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58f80cfca0 0x7f58f806f3f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f58f80cb8c0 tx=0x7f58e801c4e0 comp rx=0 tx=0).stop 2024-04-25T14:06:55.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.223+0000 7f58fec00640 1 -- 172.21.15.86:0/3576041920 shutdown_connections 2024-04-25T14:06:55.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.223+0000 7f58fec00640 1 --2- 172.21.15.86:0/3576041920 >> v2:172.21.15.155:3300/0 conn(0x7f58f80730b0 0x7f58f80734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:55.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.223+0000 7f58fec00640 1 --2- 172.21.15.86:0/3576041920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58f80cfca0 0x7f58f806f3f0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:55.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.223+0000 7f58fec00640 1 -- 172.21.15.86:0/3576041920 >> 172.21.15.86:0/3576041920 conn(0x7f58f80b4650 msgr2=0x7f58f80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:55.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.223+0000 7f58fec00640 1 -- 172.21.15.86:0/3576041920 shutdown_connections 2024-04-25T14:06:55.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.223+0000 7f58fec00640 1 -- 172.21.15.86:0/3576041920 wait complete. 2024-04-25T14:06:55.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.223+0000 7f58fec00640 1 Processor -- start 2024-04-25T14:06:55.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.223+0000 7f58fec00640 1 -- start start 2024-04-25T14:06:55.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.227+0000 7f58fec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58f80730b0 0x7f58f8157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:55.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.227+0000 7f58fec00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58f80cfca0 0x7f58f809d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:55.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.227+0000 7f58fec00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58f80d16e0 0x7f58f809da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:55.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.227+0000 7f58fec00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f58f8169a70 con 0x7f58f80730b0 2024-04-25T14:06:55.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.227+0000 7f58fec00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f58f81698f0 con 0x7f58f80cfca0 2024-04-25T14:06:55.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.227+0000 7f58fec00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f58f8169bf0 con 0x7f58f80d16e0 2024-04-25T14:06:55.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.227+0000 7f58f7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58f80730b0 0x7f58f8157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:55.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.227+0000 7f58f7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58f80730b0 0x7f58f8157b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56542/0 (socket says 172.21.15.86:56542) 2024-04-25T14:06:55.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.227+0000 7f58f7e00640 1 -- 172.21.15.86:0/3984243560 learned_addr learned my addr 172.21.15.86:0/3984243560 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:55.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.227+0000 7f58f7400640 1 --2- 172.21.15.86:0/3984243560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58f80cfca0 0x7f58f809d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:55.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.227+0000 7f58fcc00640 1 --2- 172.21.15.86:0/3984243560 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58f80d16e0 0x7f58f809da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:55.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.227+0000 7f58f7e00640 1 -- 172.21.15.86:0/3984243560 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58f80d16e0 msgr2=0x7f58f809da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:55.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.227+0000 7f58f7e00640 1 --2- 172.21.15.86:0/3984243560 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58f80d16e0 0x7f58f809da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:55.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.227+0000 7f58f7e00640 1 -- 172.21.15.86:0/3984243560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58f80cfca0 msgr2=0x7f58f809d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:55.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.227+0000 7f58f7e00640 1 --2- 172.21.15.86:0/3984243560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58f80cfca0 0x7f58f809d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:55.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.227+0000 7f58f7e00640 1 -- 172.21.15.86:0/3984243560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f815a500 con 0x7f58f80730b0 2024-04-25T14:06:55.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.227+0000 7f58f7e00640 1 --2- 172.21.15.86:0/3984243560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58f80730b0 0x7f58f8157b40 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f58e80ad040 tx=0x7f58e801ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:55.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.227+0000 7f58efe00640 1 -- 172.21.15.86:0/3984243560 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e80be5f0 con 0x7f58f80730b0 2024-04-25T14:06:55.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.227+0000 7f58efe00640 1 -- 172.21.15.86:0/3984243560 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58e80be7b0 con 0x7f58f80730b0 2024-04-25T14:06:55.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.227+0000 7f58efe00640 1 -- 172.21.15.86:0/3984243560 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e80beba0 con 0x7f58f80730b0 2024-04-25T14:06:55.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.227+0000 7f58fec00640 1 -- 172.21.15.86:0/3984243560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f815a7a0 con 0x7f58f80730b0 2024-04-25T14:06:55.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.227+0000 7f58fec00640 1 -- 172.21.15.86:0/3984243560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f815ad60 con 0x7f58f80730b0 2024-04-25T14:06:55.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.227+0000 7f58fec00640 1 -- 172.21.15.86:0/3984243560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58f809f4b0 con 0x7f58f80730b0 2024-04-25T14:06:55.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.231+0000 7f58efe00640 1 -- 172.21.15.86:0/3984243560 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f58e80c9430 con 0x7f58f80730b0 2024-04-25T14:06:55.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.231+0000 7f58efe00640 1 --2- 172.21.15.86:0/3984243560 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f58d806f640 0x7f58d8071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:55.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.231+0000 7f58efe00640 1 -- 172.21.15.86:0/3984243560 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f58e812a290 con 0x7f58f80730b0 2024-04-25T14:06:55.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.231+0000 7f58f7400640 1 --2- 172.21.15.86:0/3984243560 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f58d806f640 0x7f58d8071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:55.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.231+0000 7f58f7400640 1 --2- 172.21.15.86:0/3984243560 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f58d806f640 0x7f58d8071b50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f58f80b1110 tx=0x7f58e4008040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:55.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.235+0000 7f58efe00640 1 -- 172.21.15.86:0/3984243560 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f58e80160d0 con 0x7f58f80730b0 2024-04-25T14:06:55.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.427+0000 7f58fec00640 1 -- 172.21.15.86:0/3984243560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58f8069d00 con 0x7f58f80730b0 2024-04-25T14:06:55.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.427+0000 7f58efe00640 1 -- 172.21.15.86:0/3984243560 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5189 (secure 0 0 0) 0x7f58e80f66c0 con 0x7f58f80730b0 2024-04-25T14:06:55.429 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:55.429 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1210051860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1210051860},{"type":"v1","addr":"172.21.15.155:6833","nonce":1210051860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2868300438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2868300438},{"type":"v1","addr":"172.21.15.155:6835","nonce":2868300438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6600,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2682115891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2682115891},{"type":"v1","addr":"172.21.15.86:6835","nonce":2682115891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:06:44.910717+0000","modified":"2024-04-25T14:06:47.953301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6603},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6603":{"gid":6603,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/1088108173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1088108173},{"type":"v1","addr":"172.21.15.86:6837","nonce":1088108173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6603,"qdb_cluster":[6603]},"id":5}]} 2024-04-25T14:06:55.429 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 53 2024-04-25T14:06:55.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.427+0000 7f58fec00640 1 -- 172.21.15.86:0/3984243560 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f58d806f640 msgr2=0x7f58d8071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:55.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.427+0000 7f58fec00640 1 --2- 172.21.15.86:0/3984243560 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f58d806f640 0x7f58d8071b50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f58f80b1110 tx=0x7f58e4008040 comp rx=0 tx=0).stop 2024-04-25T14:06:55.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.427+0000 7f58fec00640 1 -- 172.21.15.86:0/3984243560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58f80730b0 msgr2=0x7f58f8157b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:55.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.427+0000 7f58fec00640 1 --2- 172.21.15.86:0/3984243560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58f80730b0 0x7f58f8157b40 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f58e80ad040 tx=0x7f58e801ad20 comp rx=0 tx=0).stop 2024-04-25T14:06:55.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.427+0000 7f58fec00640 1 -- 172.21.15.86:0/3984243560 shutdown_connections 2024-04-25T14:06:55.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.427+0000 7f58fec00640 1 --2- 172.21.15.86:0/3984243560 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f58d806f640 0x7f58d8071b50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:55.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.427+0000 7f58fec00640 1 --2- 172.21.15.86:0/3984243560 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58f80d16e0 0x7f58f809da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:55.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.427+0000 7f58fec00640 1 --2- 172.21.15.86:0/3984243560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58f80cfca0 0x7f58f809d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:55.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.427+0000 7f58fec00640 1 --2- 172.21.15.86:0/3984243560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58f80730b0 0x7f58f8157b40 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:55.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.427+0000 7f58fec00640 1 -- 172.21.15.86:0/3984243560 >> 172.21.15.86:0/3984243560 conn(0x7f58f80b4650 msgr2=0x7f58f815c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:55.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.427+0000 7f58fec00640 1 -- 172.21.15.86:0/3984243560 shutdown_connections 2024-04-25T14:06:55.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.427+0000 7f58fec00640 1 -- 172.21.15.86:0/3984243560 wait complete. 2024-04-25T14:06:55.854 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:06:44.910717+0000', 'modified': '2024-04-25T14:06:47.953301+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6603}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6603': {'gid': 6603, 'name': 'a', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/1088108173', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 1088108173}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 1088108173}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6603, 'qdb_cluster': [6603]} 2024-04-25T14:06:55.854 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T14:06:55.854 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T14:06:56.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:55.999+0000 7f5e69000640 1 Processor -- start 2024-04-25T14:06:56.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.003+0000 7f5e69000640 1 -- start start 2024-04-25T14:06:56.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.003+0000 7f5e69000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5e640cfca0 0x7f5e6406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:56.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.003+0000 7f5e69000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e640d16e0 0x7f5e640d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:56.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.003+0000 7f5e69000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e640d2030 con 0x7f5e6406c670 2024-04-25T14:06:56.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.003+0000 7f5e69000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5e64070470 con 0x7f5e640cfca0 2024-04-25T14:06:56.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.003+0000 7f5e69000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5e64072bd0 con 0x7f5e640d16e0 2024-04-25T14:06:56.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.003+0000 7f5e61e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5e640cfca0 0x7f5e6406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:56.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.003+0000 7f5e62800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5e6406c670 0x7f5e6406ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51734/0 (socket says 172.21.15.86:51734) 2024-04-25T14:06:56.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.003+0000 7f5e62800640 1 -- 172.21.15.86:0/3769873892 learned_addr learned my addr 172.21.15.86:0/3769873892 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:56.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.003+0000 7f5e59400640 1 --2- 172.21.15.86:0/3769873892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e640d16e0 0x7f5e640d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:56.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.003+0000 7f5e61e00640 1 -- 172.21.15.86:0/3769873892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e640d16e0 msgr2=0x7f5e640d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:56.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.003+0000 7f5e61e00640 1 --2- 172.21.15.86:0/3769873892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e640d16e0 0x7f5e640d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.003+0000 7f5e61e00640 1 -- 172.21.15.86:0/3769873892 >> v1:172.21.15.86:6789/0 conn(0x7f5e6406c670 legacy=0x7f5e6406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:56.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.003+0000 7f5e61e00640 1 -- 172.21.15.86:0/3769873892 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e6406f930 con 0x7f5e640cfca0 2024-04-25T14:06:56.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.003+0000 7f5e59400640 1 --2- 172.21.15.86:0/3769873892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e640d16e0 0x7f5e640d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:56.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.003+0000 7f5e61400640 1 -- 172.21.15.86:0/3769873892 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1446636254 0 0) 0x7f5e5c0014a0 con 0x7f5e6406c670 2024-04-25T14:06:56.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.007+0000 7f5e61e00640 1 --2- 172.21.15.86:0/3769873892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e640cfca0 0x7f5e6406f3f0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f5e540097c0 tx=0x7f5e5401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871fccf655157741 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:56.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.007+0000 7f5e61400640 1 -- 172.21.15.86:0/3769873892 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e540b1070 con 0x7f5e640cfca0 2024-04-25T14:06:56.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.007+0000 7f5e61400640 1 -- 172.21.15.86:0/3769873892 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e540ac410 con 0x7f5e640cfca0 2024-04-25T14:06:56.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.007+0000 7f5e61400640 1 -- 172.21.15.86:0/3769873892 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e540ac7e0 con 0x7f5e640cfca0 2024-04-25T14:06:56.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.007+0000 7f5e69000640 1 -- 172.21.15.86:0/3769873892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e640cfca0 msgr2=0x7f5e6406f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:56.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.007+0000 7f5e69000640 1 --2- 172.21.15.86:0/3769873892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e640cfca0 0x7f5e6406f3f0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f5e540097c0 tx=0x7f5e5401c4a0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.007+0000 7f5e69000640 1 -- 172.21.15.86:0/3769873892 shutdown_connections 2024-04-25T14:06:56.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.007+0000 7f5e69000640 1 --2- 172.21.15.86:0/3769873892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e640d16e0 0x7f5e640d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.007+0000 7f5e69000640 1 --2- 172.21.15.86:0/3769873892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e640cfca0 0x7f5e6406f3f0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.007+0000 7f5e69000640 1 -- 172.21.15.86:0/3769873892 >> 172.21.15.86:0/3769873892 conn(0x7f5e640b4650 msgr2=0x7f5e640b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:56.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.007+0000 7f5e69000640 1 -- 172.21.15.86:0/3769873892 shutdown_connections 2024-04-25T14:06:56.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.007+0000 7f5e69000640 1 -- 172.21.15.86:0/3769873892 wait complete. 2024-04-25T14:06:56.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.011+0000 7f5e69000640 1 Processor -- start 2024-04-25T14:06:56.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.011+0000 7f5e69000640 1 -- start start 2024-04-25T14:06:56.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.011+0000 7f5e69000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e6406c670 0x7f5e641fa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:56.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.011+0000 7f5e69000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e640cfca0 0x7f5e641582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:56.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.011+0000 7f5e69000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e640d16e0 0x7f5e640ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:56.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.011+0000 7f5e69000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e640cdee0 con 0x7f5e640cfca0 2024-04-25T14:06:56.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.011+0000 7f5e69000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5e640cdd60 con 0x7f5e640d16e0 2024-04-25T14:06:56.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.011+0000 7f5e69000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5e640ce060 con 0x7f5e6406c670 2024-04-25T14:06:56.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.011+0000 7f5e59400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e640cfca0 0x7f5e641582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:56.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.011+0000 7f5e59400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e640cfca0 0x7f5e641582f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56552/0 (socket says 172.21.15.86:56552) 2024-04-25T14:06:56.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.011+0000 7f5e59400640 1 -- 172.21.15.86:0/944668491 learned_addr learned my addr 172.21.15.86:0/944668491 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:56.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.011+0000 7f5e62800640 1 --2- 172.21.15.86:0/944668491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e640d16e0 0x7f5e640ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:56.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.011+0000 7f5e61e00640 1 --2- 172.21.15.86:0/944668491 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e6406c670 0x7f5e641fa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:56.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.011+0000 7f5e59400640 1 -- 172.21.15.86:0/944668491 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e6406c670 msgr2=0x7f5e641fa5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:56.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.011+0000 7f5e59400640 1 --2- 172.21.15.86:0/944668491 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e6406c670 0x7f5e641fa5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.011+0000 7f5e59400640 1 -- 172.21.15.86:0/944668491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e640d16e0 msgr2=0x7f5e640ca3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:56.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.011+0000 7f5e59400640 1 --2- 172.21.15.86:0/944668491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e640d16e0 0x7f5e640ca3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.011+0000 7f5e59400640 1 -- 172.21.15.86:0/944668491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e641fab30 con 0x7f5e640cfca0 2024-04-25T14:06:56.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.011+0000 7f5e62800640 1 --2- 172.21.15.86:0/944668491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e640d16e0 0x7f5e640ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:56.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.011+0000 7f5e61e00640 1 --2- 172.21.15.86:0/944668491 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e6406c670 0x7f5e641fa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:56.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.011+0000 7f5e59400640 1 --2- 172.21.15.86:0/944668491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e640cfca0 0x7f5e641582f0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00d800 tx=0x7f5e4c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:56.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.011+0000 7f5e5aa00640 1 -- 172.21.15.86:0/944668491 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c00b6c0 con 0x7f5e640cfca0 2024-04-25T14:06:56.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.011+0000 7f5e5aa00640 1 -- 172.21.15.86:0/944668491 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e4c019070 con 0x7f5e640cfca0 2024-04-25T14:06:56.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.011+0000 7f5e69000640 1 -- 172.21.15.86:0/944668491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e64158830 con 0x7f5e640cfca0 2024-04-25T14:06:56.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.011+0000 7f5e5aa00640 1 -- 172.21.15.86:0/944668491 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c00baa0 con 0x7f5e640cfca0 2024-04-25T14:06:56.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.011+0000 7f5e69000640 1 -- 172.21.15.86:0/944668491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e6415e120 con 0x7f5e640cfca0 2024-04-25T14:06:56.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.015+0000 7f5e5aa00640 1 -- 172.21.15.86:0/944668491 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f5e4c00bc40 con 0x7f5e640cfca0 2024-04-25T14:06:56.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.015+0000 7f5e5aa00640 1 --2- 172.21.15.86:0/944668491 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f5e4006f640 0x7f5e40071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:56.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.015+0000 7f5e5aa00640 1 -- 172.21.15.86:0/944668491 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f5e4c08fc80 con 0x7f5e640cfca0 2024-04-25T14:06:56.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.015+0000 7f5e61e00640 1 --2- 172.21.15.86:0/944668491 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f5e4006f640 0x7f5e40071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:56.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.015+0000 7f5e7077f000 1 Processor -- start 2024-04-25T14:06:56.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.015+0000 7f5e7077f000 1 -- start start 2024-04-25T14:06:56.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.015+0000 7f5e61e00640 1 --2- 172.21.15.86:0/944668491 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f5e4006f640 0x7f5e40071b50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5e540ae000 tx=0x7f5e540a6040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:56.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e7077f000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bde0b3bc20 0x55bde0c99ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:56.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e7077f000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde0b27ea0 0x55bde0c9a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:56.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e7077f000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bde0720280 0x55bde0c9a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:56.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e7077f000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bde0b09160 con 0x55bde0b3bc20 2024-04-25T14:06:56.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e7077f000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55bde0a9ee20 con 0x55bde0b27ea0 2024-04-25T14:06:56.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e7077f000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55bde0a57a20 con 0x55bde0720280 2024-04-25T14:06:56.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e61e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bde0b3bc20 0x55bde0c99ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:56.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e61e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bde0b3bc20 0x55bde0c99ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56560/0 (socket says 172.21.15.86:56560) 2024-04-25T14:06:56.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e61e00640 1 -- 172.21.15.86:0/3695838122 learned_addr learned my addr 172.21.15.86:0/3695838122 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:56.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e59400640 1 --2- 172.21.15.86:0/3695838122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde0b27ea0 0x55bde0c9a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:56.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e62800640 1 --2- 172.21.15.86:0/3695838122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bde0720280 0x55bde0c9a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:56.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e61e00640 1 -- 172.21.15.86:0/3695838122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bde0720280 msgr2=0x55bde0c9a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:56.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e61e00640 1 --2- 172.21.15.86:0/3695838122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bde0720280 0x55bde0c9a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e61e00640 1 -- 172.21.15.86:0/3695838122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde0b27ea0 msgr2=0x55bde0c9a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:56.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e61e00640 1 --2- 172.21.15.86:0/3695838122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde0b27ea0 0x55bde0c9a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e61e00640 1 -- 172.21.15.86:0/3695838122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bde0a5c100 con 0x55bde0b3bc20 2024-04-25T14:06:56.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e61e00640 1 --2- 172.21.15.86:0/3695838122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bde0b3bc20 0x55bde0c99ee0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f5e5401a0d0 tx=0x7f5e54004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad3cb30a2353986 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:56.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e52a00640 1 -- 172.21.15.86:0/3695838122 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e540b1040 con 0x55bde0b3bc20 2024-04-25T14:06:56.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e52a00640 1 -- 172.21.15.86:0/3695838122 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e540c6070 con 0x55bde0b3bc20 2024-04-25T14:06:56.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e52a00640 1 -- 172.21.15.86:0/3695838122 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e540b7920 con 0x55bde0b3bc20 2024-04-25T14:06:56.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e7077f000 1 -- 172.21.15.86:0/3695838122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bde0b3bc20 msgr2=0x55bde0c99ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e7077f000 1 --2- 172.21.15.86:0/3695838122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bde0b3bc20 0x55bde0c99ee0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f5e5401a0d0 tx=0x7f5e54004740 comp rx=0 tx=0).stop 2024-04-25T14:06:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e7077f000 1 -- 172.21.15.86:0/3695838122 shutdown_connections 2024-04-25T14:06:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e7077f000 1 --2- 172.21.15.86:0/3695838122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bde0720280 0x55bde0c9a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e7077f000 1 --2- 172.21.15.86:0/3695838122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde0b27ea0 0x55bde0c9a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e7077f000 1 --2- 172.21.15.86:0/3695838122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bde0b3bc20 0x55bde0c99ee0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e7077f000 1 -- 172.21.15.86:0/3695838122 >> 172.21.15.86:0/3695838122 conn(0x55bde075fff0 msgr2=0x55bde0c4f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e7077f000 1 -- 172.21.15.86:0/3695838122 shutdown_connections 2024-04-25T14:06:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.019+0000 7f5e7077f000 1 -- 172.21.15.86:0/3695838122 wait complete. 2024-04-25T14:06:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e7077f000 1 -- start start 2024-04-25T14:06:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e7077f000 1 Processor -- start 2024-04-25T14:06:56.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e7077f000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde0720280 0x55bde0ca0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:56.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e7077f000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bde0b27ea0 0x55bde0ca13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:56.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e7077f000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bde0b3bc20 0x55bde0c5c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:56.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e7077f000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bde0aac220 con 0x55bde0b27ea0 2024-04-25T14:06:56.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e7077f000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55bde0a42ea0 con 0x55bde0720280 2024-04-25T14:06:56.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e7077f000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55bde0a99b00 con 0x55bde0b3bc20 2024-04-25T14:06:56.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e7077f000 10 client.0 _ll_register_callbacks cb 0x55bde0aabdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:56.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e59400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bde0b27ea0 0x55bde0ca13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:56.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e59400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bde0b27ea0 0x55bde0ca13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56562/0 (socket says 172.21.15.86:56562) 2024-04-25T14:06:56.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e59400640 1 -- 172.21.15.86:0/1857168267 learned_addr learned my addr 172.21.15.86:0/1857168267 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:56.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e61e00640 1 --2- 172.21.15.86:0/1857168267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde0720280 0x55bde0ca0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:56.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e62800640 1 --2- 172.21.15.86:0/1857168267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bde0b3bc20 0x55bde0c5c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:56.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e59400640 1 -- 172.21.15.86:0/1857168267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bde0b3bc20 msgr2=0x55bde0c5c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:56.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e59400640 1 --2- 172.21.15.86:0/1857168267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bde0b3bc20 0x55bde0c5c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e59400640 1 -- 172.21.15.86:0/1857168267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde0720280 msgr2=0x55bde0ca0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:56.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e59400640 1 --2- 172.21.15.86:0/1857168267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde0720280 0x55bde0ca0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e59400640 1 -- 172.21.15.86:0/1857168267 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bde0a3e500 con 0x55bde0b27ea0 2024-04-25T14:06:56.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e59400640 1 --2- 172.21.15.86:0/1857168267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bde0b27ea0 0x55bde0ca13a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00d7d0 tx=0x7f5e4c05c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79539602d7eeaa90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:56.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e47400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:06:56.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e47400640 1 -- 172.21.15.86:0/1857168267 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c05c000 con 0x55bde0b27ea0 2024-04-25T14:06:56.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e47400640 1 -- 172.21.15.86:0/1857168267 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f5e4c05b260 con 0x55bde0b27ea0 2024-04-25T14:06:56.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e7077f000 1 -- 172.21.15.86:0/1857168267 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e540c6070 con 0x55bde0b27ea0 2024-04-25T14:06:56.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e47400640 1 -- 172.21.15.86:0/1857168267 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f5e4c07b050 con 0x55bde0b27ea0 2024-04-25T14:06:56.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e7077f000 10 client.6756 fetch_fsmap learned FSMap version 53 2024-04-25T14:06:56.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e7077f000 1 -- 172.21.15.86:0/1857168267 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55bde0a24ea0 con 0x55bde0b27ea0 2024-04-25T14:06:56.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e47400640 1 -- 172.21.15.86:0/1857168267 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e540c6070 con 0x55bde0b27ea0 2024-04-25T14:06:56.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e47400640 1 -- 172.21.15.86:0/1857168267 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f5e4c013070 con 0x55bde0b27ea0 2024-04-25T14:06:56.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e7077f000 10 client.6756 fetch_fsmap finished waiting for FSMap version 53 2024-04-25T14:06:56.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e7077f000 10 client.6756 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T14:06:56.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e7077f000 20 client.6756 populate_metadata read hostname 'smithi086' 2024-04-25T14:06:56.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e7077f000 1 --2- 172.21.15.86:0/1857168267 >> [v2:172.21.15.86:6836/1088108173,v1:172.21.15.86:6837/1088108173] conn(0x55bde07447d0 0x55bde0c5ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:56.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e7077f000 4 client.6756 mds_command: new command op to 6603 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:56.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e7077f000 1 -- 172.21.15.86:0/1857168267 --> [v2:172.21.15.86:6836/1088108173,v1:172.21.15.86:6837/1088108173] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bde0a37be0 con 0x55bde07447d0 2024-04-25T14:06:56.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.023+0000 7f5e61e00640 1 --2- 172.21.15.86:0/1857168267 >> [v2:172.21.15.86:6836/1088108173,v1:172.21.15.86:6837/1088108173] conn(0x55bde07447d0 0x55bde0c5ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:56.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.027+0000 7f5e47400640 10 client.6756 ms_handle_connect on v2:172.21.15.86:6836/1088108173 2024-04-25T14:06:56.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.027+0000 7f5e61e00640 1 --2- 172.21.15.86:0/1857168267 >> [v2:172.21.15.86:6836/1088108173,v1:172.21.15.86:6837/1088108173] conn(0x55bde07447d0 0x55bde0c5ce60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=625a4e1ba4145394 server_cookie=a5a436bfe97eca99 in_seq=0 out_seq=0 2024-04-25T14:06:56.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.027+0000 7f5e47400640 1 -- 172.21.15.86:0/1857168267 <== mds.0 v2:172.21.15.86:6836/1088108173 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bde0a37be0 con 0x55bde07447d0 2024-04-25T14:06:56.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.027+0000 7f5e47400640 10 client.6756 handle_command_reply: tid=0 2024-04-25T14:06:56.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.027+0000 7f5e7077f000 1 client.6756 shutdown 2024-04-25T14:06:56.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.027+0000 7f5e7077f000 1 -- 172.21.15.86:0/1857168267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bde0b27ea0 msgr2=0x55bde0ca13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:56.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.027+0000 7f5e7077f000 1 --2- 172.21.15.86:0/1857168267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bde0b27ea0 0x55bde0ca13a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00d7d0 tx=0x7f5e4c05c620 comp rx=0 tx=0).stop 2024-04-25T14:06:56.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.027+0000 7f5e7077f000 1 -- 172.21.15.86:0/1857168267 shutdown_connections 2024-04-25T14:06:56.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.027+0000 7f5e7077f000 1 --2- 172.21.15.86:0/1857168267 >> [v2:172.21.15.86:6836/1088108173,v1:172.21.15.86:6837/1088108173] conn(0x55bde07447d0 0x55bde0c5ce60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.027+0000 7f5e7077f000 1 --2- 172.21.15.86:0/1857168267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bde0b3bc20 0x55bde0c5c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.027+0000 7f5e7077f000 1 --2- 172.21.15.86:0/1857168267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bde0b27ea0 0x55bde0ca13a0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.027+0000 7f5e47400640 0 client.6756 ms_handle_reset on v2:172.21.15.86:6836/1088108173 2024-04-25T14:06:56.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.027+0000 7f5e7077f000 1 --2- 172.21.15.86:0/1857168267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde0720280 0x55bde0ca0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.027+0000 7f5e7077f000 1 -- 172.21.15.86:0/1857168267 >> 172.21.15.86:0/1857168267 conn(0x55bde075fff0 msgr2=0x55bde0c4fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:56.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.027+0000 7f5e7077f000 1 -- 172.21.15.86:0/1857168267 shutdown_connections 2024-04-25T14:06:56.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.027+0000 7f5e7077f000 1 -- 172.21.15.86:0/1857168267 wait complete. 2024-04-25T14:06:56.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.027+0000 7f5e7077f000 20 client.6756 trim_cache size 0 max 16384 2024-04-25T14:06:56.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.043+0000 7f5e7077f000 1 Processor -- start 2024-04-25T14:06:56.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.043+0000 7f5e7077f000 1 -- start start 2024-04-25T14:06:56.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e7077f000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bde0720280 0x55bde0ca64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:56.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e7077f000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde0b27ea0 0x55bde0cafff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:56.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e7077f000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bde0b3bc20 0x55bde0cb0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:56.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e7077f000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e64205a20 con 0x55bde0b3bc20 2024-04-25T14:06:56.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e7077f000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5e641fbc90 con 0x55bde0b27ea0 2024-04-25T14:06:56.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e7077f000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5e642054d0 con 0x55bde0720280 2024-04-25T14:06:56.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e62800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bde0b3bc20 0x55bde0cb0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:56.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e62800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bde0b3bc20 0x55bde0cb0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56566/0 (socket says 172.21.15.86:56566) 2024-04-25T14:06:56.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e62800640 1 -- 172.21.15.86:0/903734499 learned_addr learned my addr 172.21.15.86:0/903734499 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:56.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e62800640 1 -- 172.21.15.86:0/903734499 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bde0720280 msgr2=0x55bde0ca64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:56.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e59400640 1 --2- 172.21.15.86:0/903734499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde0b27ea0 0x55bde0cafff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:56.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e61e00640 1 --2- 172.21.15.86:0/903734499 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bde0720280 0x55bde0ca64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:56.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e62800640 1 --2- 172.21.15.86:0/903734499 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bde0720280 0x55bde0ca64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e62800640 1 -- 172.21.15.86:0/903734499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde0b27ea0 msgr2=0x55bde0cafff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:56.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e62800640 1 --2- 172.21.15.86:0/903734499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde0b27ea0 0x55bde0cafff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e62800640 1 -- 172.21.15.86:0/903734499 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bde0c5c430 con 0x55bde0b3bc20 2024-04-25T14:06:56.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e59400640 1 --2- 172.21.15.86:0/903734499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde0b27ea0 0x55bde0cafff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:56.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e61e00640 1 --2- 172.21.15.86:0/903734499 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bde0720280 0x55bde0ca64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:56.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e62800640 1 --2- 172.21.15.86:0/903734499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bde0b3bc20 0x55bde0cb0530 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c0075b0 tx=0x7f5e5c007a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f8c6fee5c16159 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:56.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e52000640 1 -- 172.21.15.86:0/903734499 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c019070 con 0x55bde0b3bc20 2024-04-25T14:06:56.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e52000640 1 -- 172.21.15.86:0/903734499 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e5c012c30 con 0x55bde0b3bc20 2024-04-25T14:06:56.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e52000640 1 -- 172.21.15.86:0/903734499 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c015440 con 0x55bde0b3bc20 2024-04-25T14:06:56.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e7077f000 1 -- 172.21.15.86:0/903734499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bde0b3bc20 msgr2=0x55bde0cb0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:56.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e7077f000 1 --2- 172.21.15.86:0/903734499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bde0b3bc20 0x55bde0cb0530 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c0075b0 tx=0x7f5e5c007a70 comp rx=0 tx=0).stop 2024-04-25T14:06:56.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e7077f000 1 -- 172.21.15.86:0/903734499 shutdown_connections 2024-04-25T14:06:56.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e7077f000 1 --2- 172.21.15.86:0/903734499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bde0b3bc20 0x55bde0cb0530 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e7077f000 1 --2- 172.21.15.86:0/903734499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde0b27ea0 0x55bde0cafff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e7077f000 1 --2- 172.21.15.86:0/903734499 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bde0720280 0x55bde0ca64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e7077f000 1 -- 172.21.15.86:0/903734499 >> 172.21.15.86:0/903734499 conn(0x55bde075fff0 msgr2=0x55bde0c57000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:56.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e7077f000 1 -- 172.21.15.86:0/903734499 shutdown_connections 2024-04-25T14:06:56.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.047+0000 7f5e7077f000 1 -- 172.21.15.86:0/903734499 wait complete. 2024-04-25T14:06:56.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e7077f000 1 -- start start 2024-04-25T14:06:56.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e7077f000 1 Processor -- start 2024-04-25T14:06:56.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e7077f000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bde0720280 0x55bde0d0c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:56.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e7077f000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde0b27ea0 0x55bde0d0cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:56.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e7077f000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bde0b3bc20 0x55bde0ca49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:56.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e7077f000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bde0a9e130 con 0x55bde0b3bc20 2024-04-25T14:06:56.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e7077f000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55bde0a347d0 con 0x55bde0b27ea0 2024-04-25T14:06:56.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e7077f000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55bde0aa3450 con 0x55bde0720280 2024-04-25T14:06:56.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e7077f000 10 client.0 _ll_register_callbacks cb 0x7f5e4c0584c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:56.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e62800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bde0b3bc20 0x55bde0ca49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:56.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e62800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bde0b3bc20 0x55bde0ca49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56578/0 (socket says 172.21.15.86:56578) 2024-04-25T14:06:56.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e62800640 1 -- 172.21.15.86:0/1540148265 learned_addr learned my addr 172.21.15.86:0/1540148265 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:56.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e59400640 1 --2- 172.21.15.86:0/1540148265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde0b27ea0 0x55bde0d0cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:56.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e61e00640 1 --2- 172.21.15.86:0/1540148265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bde0720280 0x55bde0d0c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:56.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e62800640 1 -- 172.21.15.86:0/1540148265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bde0720280 msgr2=0x55bde0d0c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:56.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e62800640 1 --2- 172.21.15.86:0/1540148265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bde0720280 0x55bde0d0c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e62800640 1 -- 172.21.15.86:0/1540148265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde0b27ea0 msgr2=0x55bde0d0cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:56.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e62800640 1 --2- 172.21.15.86:0/1540148265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde0b27ea0 0x55bde0d0cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e62800640 1 -- 172.21.15.86:0/1540148265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bde0a57c70 con 0x55bde0b3bc20 2024-04-25T14:06:56.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e62800640 1 --2- 172.21.15.86:0/1540148265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bde0b3bc20 0x55bde0ca49b0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c000f90 tx=0x7f5e5c007cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd75c716a9022ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:56.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e47400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:06:56.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e47400640 1 -- 172.21.15.86:0/1540148265 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c019070 con 0x55bde0b3bc20 2024-04-25T14:06:56.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e47400640 1 -- 172.21.15.86:0/1540148265 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f5e5c01fbe0 con 0x55bde0b3bc20 2024-04-25T14:06:56.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e7077f000 1 -- 172.21.15.86:0/1540148265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e5c012c30 con 0x55bde0b3bc20 2024-04-25T14:06:56.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e47400640 1 -- 172.21.15.86:0/1540148265 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f5e5c01fe20 con 0x55bde0b3bc20 2024-04-25T14:06:56.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e7077f000 10 client.6762 fetch_fsmap learned FSMap version 53 2024-04-25T14:06:56.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e7077f000 1 -- 172.21.15.86:0/1540148265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55bde0b21180 con 0x55bde0b3bc20 2024-04-25T14:06:56.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e47400640 1 -- 172.21.15.86:0/1540148265 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e5c012c30 con 0x55bde0b3bc20 2024-04-25T14:06:56.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e47400640 1 -- 172.21.15.86:0/1540148265 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f5e5c015440 con 0x55bde0b3bc20 2024-04-25T14:06:56.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e7077f000 10 client.6762 fetch_fsmap finished waiting for FSMap version 53 2024-04-25T14:06:56.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e7077f000 10 client.6762 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T14:06:56.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e7077f000 20 client.6762 populate_metadata read hostname 'smithi086' 2024-04-25T14:06:56.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e7077f000 1 --2- 172.21.15.86:0/1540148265 >> [v2:172.21.15.86:6836/1088108173,v1:172.21.15.86:6837/1088108173] conn(0x55bde0732830 0x55bde0ca5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:56.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e7077f000 4 client.6762 mds_command: new command op to 6603 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:06:56.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e7077f000 1 -- 172.21.15.86:0/1540148265 --> [v2:172.21.15.86:6836/1088108173,v1:172.21.15.86:6837/1088108173] -- command(tid 0: {"prefix": "status"}) -- 0x55bde0aaca30 con 0x55bde0732830 2024-04-25T14:06:56.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.051+0000 7f5e61e00640 1 --2- 172.21.15.86:0/1540148265 >> [v2:172.21.15.86:6836/1088108173,v1:172.21.15.86:6837/1088108173] conn(0x55bde0732830 0x55bde0ca5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:56.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e47400640 10 client.6762 ms_handle_connect on v2:172.21.15.86:6836/1088108173 2024-04-25T14:06:56.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e61e00640 1 --2- 172.21.15.86:0/1540148265 >> [v2:172.21.15.86:6836/1088108173,v1:172.21.15.86:6837/1088108173] conn(0x55bde0732830 0x55bde0ca5370 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af4f98d8683bb919 server_cookie=2042f5aa9711e72d in_seq=0 out_seq=0 2024-04-25T14:06:56.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e47400640 1 -- 172.21.15.86:0/1540148265 <== mds.0 v2:172.21.15.86:6836/1088108173 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55bde0aaca30 con 0x55bde0732830 2024-04-25T14:06:56.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e47400640 10 client.6762 handle_command_reply: tid=0 2024-04-25T14:06:56.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e7077f000 1 client.6762 shutdown 2024-04-25T14:06:56.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e7077f000 1 -- 172.21.15.86:0/1540148265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bde0b3bc20 msgr2=0x55bde0ca49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:56.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e7077f000 1 --2- 172.21.15.86:0/1540148265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bde0b3bc20 0x55bde0ca49b0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c000f90 tx=0x7f5e5c007cd0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e7077f000 1 -- 172.21.15.86:0/1540148265 shutdown_connections 2024-04-25T14:06:56.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e7077f000 1 --2- 172.21.15.86:0/1540148265 >> [v2:172.21.15.86:6836/1088108173,v1:172.21.15.86:6837/1088108173] conn(0x55bde0732830 0x55bde0ca5370 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e7077f000 1 --2- 172.21.15.86:0/1540148265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bde0b3bc20 0x55bde0ca49b0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e7077f000 1 --2- 172.21.15.86:0/1540148265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde0b27ea0 0x55bde0d0cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e7077f000 1 --2- 172.21.15.86:0/1540148265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bde0720280 0x55bde0d0c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e47400640 0 client.6762 ms_handle_reset on v2:172.21.15.86:6836/1088108173 2024-04-25T14:06:56.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e7077f000 1 -- 172.21.15.86:0/1540148265 >> 172.21.15.86:0/1540148265 conn(0x55bde075fff0 msgr2=0x55bde0c9f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:56.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e7077f000 1 -- 172.21.15.86:0/1540148265 shutdown_connections 2024-04-25T14:06:56.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e7077f000 1 -- 172.21.15.86:0/1540148265 wait complete. 2024-04-25T14:06:56.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e7077f000 20 client.6762 trim_cache size 0 max 16384 2024-04-25T14:06:56.060 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:06:56.060 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb", 2024-04-25T14:06:56.060 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-25T14:06:56.060 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 6603, 2024-04-25T14:06:56.060 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T14:06:56.060 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T14:06:56.060 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T14:06:56.060 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 10.514284099999999, 2024-04-25T14:06:56.060 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 53, 2024-04-25T14:06:56.060 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 74, 2024-04-25T14:06:56.060 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 74, 2024-04-25T14:06:56.061 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 16.359377196000001 2024-04-25T14:06:56.061 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:06:56.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e69000640 1 -- 172.21.15.86:0/944668491 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f5e4006f640 msgr2=0x7f5e40071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:56.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e69000640 1 --2- 172.21.15.86:0/944668491 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f5e4006f640 0x7f5e40071b50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5e540ae000 tx=0x7f5e540a6040 comp rx=0 tx=0).stop 2024-04-25T14:06:56.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e69000640 1 -- 172.21.15.86:0/944668491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e640cfca0 msgr2=0x7f5e641582f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:56.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e69000640 1 --2- 172.21.15.86:0/944668491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e640cfca0 0x7f5e641582f0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00d800 tx=0x7f5e4c00dcc0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e69000640 1 -- 172.21.15.86:0/944668491 shutdown_connections 2024-04-25T14:06:56.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e69000640 1 --2- 172.21.15.86:0/944668491 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f5e4006f640 0x7f5e40071b50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e69000640 1 --2- 172.21.15.86:0/944668491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e640d16e0 0x7f5e640ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e69000640 1 --2- 172.21.15.86:0/944668491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e640cfca0 0x7f5e641582f0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e69000640 1 --2- 172.21.15.86:0/944668491 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e6406c670 0x7f5e641fa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e69000640 1 -- 172.21.15.86:0/944668491 >> 172.21.15.86:0/944668491 conn(0x7f5e640b4650 msgr2=0x7f5e640ae130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:56.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e69000640 1 -- 172.21.15.86:0/944668491 shutdown_connections 2024-04-25T14:06:56.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.055+0000 7f5e69000640 1 -- 172.21.15.86:0/944668491 wait complete. 2024-04-25T14:06:56.077 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T14:06:56.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.223+0000 7f6a06800640 1 Processor -- start 2024-04-25T14:06:56.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.223+0000 7f6a06800640 1 -- start start 2024-04-25T14:06:56.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.223+0000 7f6a06800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6a000ce660 0x7f6a0015de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:56.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.223+0000 7f6a06800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a000cef20 0x7f6a000cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:56.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.223+0000 7f6a06800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a00161b40 con 0x7f6a0006c670 2024-04-25T14:06:56.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.223+0000 7f6a06800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f6a0015e380 con 0x7f6a000ce660 2024-04-25T14:06:56.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.223+0000 7f6a06800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6a00162750 con 0x7f6a000cef20 2024-04-25T14:06:56.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.223+0000 7f69f6a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6a000ce660 0x7f6a0015de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:56.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.223+0000 7f69ffe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a000cef20 0x7f6a000cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:56.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.223+0000 7f69ff400640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f6a0006c670 0x7f6a000cdf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:45004/0 (socket says 172.21.15.86:45004) 2024-04-25T14:06:56.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.223+0000 7f69f6a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6a000ce660 0x7f6a0015de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:43360/0 (socket says 172.21.15.86:43360) 2024-04-25T14:06:56.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.223+0000 7f69ff400640 1 -- 172.21.15.86:0/1407914073 learned_addr learned my addr 172.21.15.86:0/1407914073 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:56.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.223+0000 7f69f6a00640 1 -- 172.21.15.86:0/1407914073 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a000cef20 msgr2=0x7f6a000cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:56.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.223+0000 7f69f6a00640 1 --2- 172.21.15.86:0/1407914073 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a000cef20 0x7f6a000cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.223+0000 7f69f6a00640 1 -- 172.21.15.86:0/1407914073 >> v1:172.21.15.155:6789/0 conn(0x7f6a0006c670 legacy=0x7f6a000cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:56.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.223+0000 7f69f6a00640 1 -- 172.21.15.86:0/1407914073 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a00161e20 con 0x7f6a000ce660 2024-04-25T14:06:56.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.227+0000 7f69f6a00640 1 --2- 172.21.15.86:0/1407914073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a000ce660 0x7f6a0015de40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f69e80097c0 tx=0x7f69e801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f322f102a971bdc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:56.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.227+0000 7f69fea00640 1 -- 172.21.15.86:0/1407914073 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e80b0070 con 0x7f6a000ce660 2024-04-25T14:06:56.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.227+0000 7f69fea00640 1 -- 172.21.15.86:0/1407914073 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69e80ab600 con 0x7f6a000ce660 2024-04-25T14:06:56.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.227+0000 7f69fea00640 1 -- 172.21.15.86:0/1407914073 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e80ab9d0 con 0x7f6a000ce660 2024-04-25T14:06:56.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.227+0000 7f6a06800640 1 -- 172.21.15.86:0/1407914073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a000ce660 msgr2=0x7f6a0015de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:56.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.227+0000 7f6a06800640 1 --2- 172.21.15.86:0/1407914073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a000ce660 0x7f6a0015de40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f69e80097c0 tx=0x7f69e801c410 comp rx=0 tx=0).stop 2024-04-25T14:06:56.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.227+0000 7f6a06800640 1 -- 172.21.15.86:0/1407914073 shutdown_connections 2024-04-25T14:06:56.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.227+0000 7f6a06800640 1 --2- 172.21.15.86:0/1407914073 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a000cef20 0x7f6a000cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.227+0000 7f6a06800640 1 --2- 172.21.15.86:0/1407914073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a000ce660 0x7f6a0015de40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.227+0000 7f6a06800640 1 -- 172.21.15.86:0/1407914073 >> 172.21.15.86:0/1407914073 conn(0x7f6a0009ba30 msgr2=0x7f6a000b73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:56.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.231+0000 7f6a06800640 1 -- 172.21.15.86:0/1407914073 shutdown_connections 2024-04-25T14:06:56.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.231+0000 7f6a06800640 1 -- 172.21.15.86:0/1407914073 wait complete. 2024-04-25T14:06:56.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.231+0000 7f6a06800640 1 Processor -- start 2024-04-25T14:06:56.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.231+0000 7f6a06800640 1 -- start start 2024-04-25T14:06:56.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.231+0000 7f6a06800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a0006c670 0x7f6a00150a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:56.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.231+0000 7f6a06800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a000ce660 0x7f6a000cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:56.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.231+0000 7f6a06800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a000cef20 0x7f6a000cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:56.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.231+0000 7f6a06800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6a000b8060 con 0x7f6a0006c670 2024-04-25T14:06:56.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.231+0000 7f6a06800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6a000b7ee0 con 0x7f6a000cef20 2024-04-25T14:06:56.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.231+0000 7f6a06800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6a0006ee70 con 0x7f6a000ce660 2024-04-25T14:06:56.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.235+0000 7f69ff400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a0006c670 0x7f6a00150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:56.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.235+0000 7f69ff400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a0006c670 0x7f6a00150a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56584/0 (socket says 172.21.15.86:56584) 2024-04-25T14:06:56.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.235+0000 7f69ff400640 1 -- 172.21.15.86:0/1995479058 learned_addr learned my addr 172.21.15.86:0/1995479058 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:56.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.235+0000 7f69f6a00640 1 --2- 172.21.15.86:0/1995479058 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a000ce660 0x7f6a000cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:56.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.235+0000 7f69ff400640 1 -- 172.21.15.86:0/1995479058 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a000ce660 msgr2=0x7f6a000cb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:56.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.235+0000 7f69ff400640 1 --2- 172.21.15.86:0/1995479058 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a000ce660 0x7f6a000cb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.235+0000 7f69ff400640 1 -- 172.21.15.86:0/1995479058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a000cef20 msgr2=0x7f6a000cbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:56.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.235+0000 7f69ff400640 1 --2- 172.21.15.86:0/1995479058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a000cef20 0x7f6a000cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.235+0000 7f69ff400640 1 -- 172.21.15.86:0/1995479058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a000c8ee0 con 0x7f6a0006c670 2024-04-25T14:06:56.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.235+0000 7f69ff400640 1 --2- 172.21.15.86:0/1995479058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a0006c670 0x7f6a00150a50 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f69f000cad0 tx=0x7f69f000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:56.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.235+0000 7f69f7e00640 1 -- 172.21.15.86:0/1995479058 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f0017070 con 0x7f6a0006c670 2024-04-25T14:06:56.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.235+0000 7f69f7e00640 1 -- 172.21.15.86:0/1995479058 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69f000ab70 con 0x7f6a0006c670 2024-04-25T14:06:56.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.235+0000 7f6a06800640 1 -- 172.21.15.86:0/1995479058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a0015b450 con 0x7f6a0006c670 2024-04-25T14:06:56.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.235+0000 7f69f7e00640 1 -- 172.21.15.86:0/1995479058 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f0012400 con 0x7f6a0006c670 2024-04-25T14:06:56.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.235+0000 7f6a06800640 1 -- 172.21.15.86:0/1995479058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a0015ba10 con 0x7f6a0006c670 2024-04-25T14:06:56.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.235+0000 7f6a06800640 1 -- 172.21.15.86:0/1995479058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a000a0630 con 0x7f6a0006c670 2024-04-25T14:06:56.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.235+0000 7f69f7e00640 1 -- 172.21.15.86:0/1995479058 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f69f001a070 con 0x7f6a0006c670 2024-04-25T14:06:56.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.239+0000 7f69f7e00640 1 --2- 172.21.15.86:0/1995479058 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f69dc06f670 0x7f69dc071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:56.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.239+0000 7f69f7e00640 1 -- 172.21.15.86:0/1995479058 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f69f008f590 con 0x7f6a0006c670 2024-04-25T14:06:56.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.239+0000 7f69f6a00640 1 --2- 172.21.15.86:0/1995479058 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f69dc06f670 0x7f69dc071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:56.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.239+0000 7f69f6a00640 1 --2- 172.21.15.86:0/1995479058 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f69dc06f670 0x7f69dc071b80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f69e80ad000 tx=0x7f69e8008040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:56.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.243+0000 7f69f7e00640 1 -- 172.21.15.86:0/1995479058 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f69f005bf70 con 0x7f6a0006c670 2024-04-25T14:06:56.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.431+0000 7f6a06800640 1 -- 172.21.15.86:0/1995479058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6a0015d260 con 0x7f6a0006c670 2024-04-25T14:06:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.435+0000 7f69f7e00640 1 -- 172.21.15.86:0/1995479058 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) ==== 53+0+73 (secure 0 0 0) 0x7f69f005b870 con 0x7f6a0006c670 2024-04-25T14:06:56.436 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T14:06:56.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.435+0000 7f6a06800640 1 -- 172.21.15.86:0/1995479058 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f69dc06f670 msgr2=0x7f69dc071b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:56.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.435+0000 7f6a06800640 1 --2- 172.21.15.86:0/1995479058 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f69dc06f670 0x7f69dc071b80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f69e80ad000 tx=0x7f69e8008040 comp rx=0 tx=0).stop 2024-04-25T14:06:56.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.435+0000 7f6a06800640 1 -- 172.21.15.86:0/1995479058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a0006c670 msgr2=0x7f6a00150a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.435+0000 7f6a06800640 1 --2- 172.21.15.86:0/1995479058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a0006c670 0x7f6a00150a50 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f69f000cad0 tx=0x7f69f000cf90 comp rx=0 tx=0).stop 2024-04-25T14:06:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.435+0000 7f6a06800640 1 -- 172.21.15.86:0/1995479058 shutdown_connections 2024-04-25T14:06:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.435+0000 7f6a06800640 1 --2- 172.21.15.86:0/1995479058 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f69dc06f670 0x7f69dc071b80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.435+0000 7f6a06800640 1 --2- 172.21.15.86:0/1995479058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a000cef20 0x7f6a000cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.435+0000 7f6a06800640 1 --2- 172.21.15.86:0/1995479058 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a000ce660 0x7f6a000cb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.435+0000 7f6a06800640 1 --2- 172.21.15.86:0/1995479058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a0006c670 0x7f6a00150a50 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.435+0000 7f6a06800640 1 -- 172.21.15.86:0/1995479058 >> 172.21.15.86:0/1995479058 conn(0x7f6a0009ba30 msgr2=0x7f6a00068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.435+0000 7f6a06800640 1 -- 172.21.15.86:0/1995479058 shutdown_connections 2024-04-25T14:06:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56.435+0000 7f6a06800640 1 -- 172.21.15.86:0/1995479058 wait complete. 2024-04-25T14:06:56.458 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T14:06:56.458 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T14:06:56.458 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-25T14:06:56.458 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T14:06:56.458 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T14:06:56.458 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T14:06:56.459 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T14:06:56.459 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T14:06:56.459 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:06:56.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:56.460 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-25T14:06:56.470 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:06:56.470 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:06:56.470 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:06:56.470 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:06:56.470 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-25T14:06:56.471 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:06:56.471 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:06:56.471 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:06:56.471 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-25T14:06:56.471 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:06:56.471 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:06:56.472 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-25T14:06:56.472 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:06:56.472 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:06:56.472 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T14:06:56.472 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:06:56.472 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T14:06:56.472 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42248sec preferred_lft 42248sec 2024-04-25T14:06:56.473 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-25T14:06:56.473 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:06:56.473 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T14:06:56.473 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:dc:87:9c:96 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:06:56.473 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T14:06:56.473 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:06:56.473 INFO:teuthology.orchestra.run.smithi086.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T14:06:56.474 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T14:06:56.474 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T14:06:56.474 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:06:56.474 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T14:06:56.474 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:06:56.474 INFO:teuthology.orchestra.run.smithi086.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:06:56.475 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T14:06:56.475 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T14:06:56.475 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:06:56.475 INFO:teuthology.orchestra.run.smithi086.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:06:56.475 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T14:06:56.475 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T14:06:56.475 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:06:56.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:56.476 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T14:06:56.528 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-25T14:06:56.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:56.531 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-25T14:06:56.586 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-25T14:06:56.587 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T14:06:56.587 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T14:06:56.587 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T14:06:56.587 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T14:06:56.587 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T14:06:56.587 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T14:06:56.587 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:06:56.587 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:06:56.587 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:06:56.587 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T14:06:56.587 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:06:56.588 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T14:06:56.588 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T14:06:56.588 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T14:06:56.588 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T14:06:56.588 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T14:06:56.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:06:56.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:06:56.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:56.719 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T14:06:56.728 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T14:06:56.728 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T14:06:56.729 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:06:56.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:56.729 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-25T14:06:56.790 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:06:56.791 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T14:06:56.839 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:06:56.839 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:06:56.839 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T14:06:56.890 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T14:06:56.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:56.891 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:06:56.993 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:06:56.994 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T14:06:56.994 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:06:57.011 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:06:57.011 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T14:06:57.011 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:06:57.011 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:06:57.012 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:06:57.023 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:06:57.023 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T14:06:57.023 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:06:57.023 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:06:57.023 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:06:57.023 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790" 2024-04-25T14:06:57.023 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790:/ 2024-04-25T14:06:57.023 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb" will pass to kernel 2024-04-25T14:06:57.027 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T14:06:57.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:57.028 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T14:06:57.047 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:06:57.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:06:57.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:06:57.152 INFO:teuthology.orchestra.run.smithi086.stdout:client5945 2024-04-25T14:06:57.153 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:06:57.153 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:06:57.162 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T14:06:57.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.347+0000 7f0d6f800640 1 Processor -- start 2024-04-25T14:06:57.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.347+0000 7f0d6f800640 1 -- start start 2024-04-25T14:06:57.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.347+0000 7f0d6f800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0d680d16e0 0x7f0d680d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:57.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.347+0000 7f0d6f800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0d680730b0 0x7f0d680734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:57.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.347+0000 7f0d6f800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d680d2030 con 0x7f0d680cfca0 2024-04-25T14:06:57.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.347+0000 7f0d6f800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0d68070470 con 0x7f0d680d16e0 2024-04-25T14:06:57.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.347+0000 7f0d6f800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0d68072bd0 con 0x7f0d680730b0 2024-04-25T14:06:57.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.347+0000 7f0d67e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0d680d16e0 0x7f0d680d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:57.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.347+0000 7f0d67e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0d680d16e0 0x7f0d680d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56594/0 (socket says 172.21.15.86:56594) 2024-04-25T14:06:57.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.347+0000 7f0d67e00640 1 -- 172.21.15.86:0/2062019384 learned_addr learned my addr 172.21.15.86:0/2062019384 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:57.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.347+0000 7f0d6d800640 1 --2- 172.21.15.86:0/2062019384 >> v2:172.21.15.155:3300/0 conn(0x7f0d680730b0 0x7f0d680734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:57.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.347+0000 7f0d67e00640 1 -- 172.21.15.86:0/2062019384 >> v2:172.21.15.155:3300/0 conn(0x7f0d680730b0 msgr2=0x7f0d680734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:57.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.347+0000 7f0d67e00640 1 --2- 172.21.15.86:0/2062019384 >> v2:172.21.15.155:3300/0 conn(0x7f0d680730b0 0x7f0d680734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.347+0000 7f0d67e00640 1 -- 172.21.15.86:0/2062019384 >> v1:172.21.15.155:6789/0 conn(0x7f0d680cfca0 legacy=0x7f0d6806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:57.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.347+0000 7f0d67e00640 1 -- 172.21.15.86:0/2062019384 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d6806fbd0 con 0x7f0d680d16e0 2024-04-25T14:06:57.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.351+0000 7f0d67e00640 1 --2- 172.21.15.86:0/2062019384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d680d16e0 0x7f0d680d1af0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0d680729c0 tx=0x7f0d5401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7ded4e58d890ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:57.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.351+0000 7f0d67400640 1 -- 172.21.15.86:0/2062019384 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d540ab430 con 0x7f0d680d16e0 2024-04-25T14:06:57.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.351+0000 7f0d67400640 1 -- 172.21.15.86:0/2062019384 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d540b0070 con 0x7f0d680d16e0 2024-04-25T14:06:57.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.351+0000 7f0d67400640 1 -- 172.21.15.86:0/2062019384 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d540ab7f0 con 0x7f0d680d16e0 2024-04-25T14:06:57.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.351+0000 7f0d6f800640 1 -- 172.21.15.86:0/2062019384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d680d16e0 msgr2=0x7f0d680d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:57.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.351+0000 7f0d6f800640 1 --2- 172.21.15.86:0/2062019384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d680d16e0 0x7f0d680d1af0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0d680729c0 tx=0x7f0d5401c520 comp rx=0 tx=0).stop 2024-04-25T14:06:57.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.351+0000 7f0d6f800640 1 -- 172.21.15.86:0/2062019384 shutdown_connections 2024-04-25T14:06:57.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.351+0000 7f0d6f800640 1 --2- 172.21.15.86:0/2062019384 >> v2:172.21.15.155:3300/0 conn(0x7f0d680730b0 0x7f0d680734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.351+0000 7f0d6f800640 1 --2- 172.21.15.86:0/2062019384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d680d16e0 0x7f0d680d1af0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.351+0000 7f0d6f800640 1 -- 172.21.15.86:0/2062019384 >> 172.21.15.86:0/2062019384 conn(0x7f0d680b4650 msgr2=0x7f0d680b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:57.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.351+0000 7f0d6f800640 1 -- 172.21.15.86:0/2062019384 shutdown_connections 2024-04-25T14:06:57.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.351+0000 7f0d6f800640 1 -- 172.21.15.86:0/2062019384 wait complete. 2024-04-25T14:06:57.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.355+0000 7f0d6f800640 1 Processor -- start 2024-04-25T14:06:57.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.355+0000 7f0d6f800640 1 -- start start 2024-04-25T14:06:57.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.355+0000 7f0d6f800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d680730b0 0x7f0d680997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:57.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.355+0000 7f0d6f800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d680cfca0 0x7f0d680cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:57.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.355+0000 7f0d6f800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d680d16e0 0x7f0d680cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:57.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.355+0000 7f0d6f800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d68169a70 con 0x7f0d680730b0 2024-04-25T14:06:57.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.355+0000 7f0d6f800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0d681698f0 con 0x7f0d680d16e0 2024-04-25T14:06:57.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.355+0000 7f0d6f800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0d68169bf0 con 0x7f0d680cfca0 2024-04-25T14:06:57.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.355+0000 7f0d6ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d680730b0 0x7f0d680997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:57.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.355+0000 7f0d6ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d680730b0 0x7f0d680997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56600/0 (socket says 172.21.15.86:56600) 2024-04-25T14:06:57.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.355+0000 7f0d6ce00640 1 -- 172.21.15.86:0/1092024008 learned_addr learned my addr 172.21.15.86:0/1092024008 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:57.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.355+0000 7f0d6d800640 1 --2- 172.21.15.86:0/1092024008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d680d16e0 0x7f0d680cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:57.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.355+0000 7f0d67e00640 1 --2- 172.21.15.86:0/1092024008 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d680cfca0 0x7f0d680cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:57.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.359+0000 7f0d6ce00640 1 -- 172.21.15.86:0/1092024008 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d680cfca0 msgr2=0x7f0d680cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:57.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.359+0000 7f0d6ce00640 1 --2- 172.21.15.86:0/1092024008 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d680cfca0 0x7f0d680cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.359+0000 7f0d6ce00640 1 -- 172.21.15.86:0/1092024008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d680d16e0 msgr2=0x7f0d680cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:57.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.359+0000 7f0d6ce00640 1 --2- 172.21.15.86:0/1092024008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d680d16e0 0x7f0d680cc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.359+0000 7f0d6ce00640 1 -- 172.21.15.86:0/1092024008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d6815bd80 con 0x7f0d680730b0 2024-04-25T14:06:57.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.359+0000 7f0d6ce00640 1 --2- 172.21.15.86:0/1092024008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d680730b0 0x7f0d680997d0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0d5800c760 tx=0x7f0d5800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:57.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.359+0000 7f0d64c00640 1 -- 172.21.15.86:0/1092024008 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d58003b80 con 0x7f0d680730b0 2024-04-25T14:06:57.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.359+0000 7f0d64c00640 1 -- 172.21.15.86:0/1092024008 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d58003d60 con 0x7f0d680730b0 2024-04-25T14:06:57.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.359+0000 7f0d6f800640 1 -- 172.21.15.86:0/1092024008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d680cca80 con 0x7f0d680730b0 2024-04-25T14:06:57.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.359+0000 7f0d64c00640 1 -- 172.21.15.86:0/1092024008 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5800aa90 con 0x7f0d680730b0 2024-04-25T14:06:57.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.359+0000 7f0d6f800640 1 -- 172.21.15.86:0/1092024008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d680cd040 con 0x7f0d680730b0 2024-04-25T14:06:57.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.359+0000 7f0d6f800640 1 -- 172.21.15.86:0/1092024008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d6809f4b0 con 0x7f0d680730b0 2024-04-25T14:06:57.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.359+0000 7f0d64c00640 1 -- 172.21.15.86:0/1092024008 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7f0d5800ac30 con 0x7f0d680730b0 2024-04-25T14:06:57.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.363+0000 7f0d64c00640 1 --2- 172.21.15.86:0/1092024008 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f0d4806f670 0x7f0d48071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:57.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.363+0000 7f0d64c00640 1 -- 172.21.15.86:0/1092024008 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f0d5808ffe0 con 0x7f0d680730b0 2024-04-25T14:06:57.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.363+0000 7f0d67e00640 1 --2- 172.21.15.86:0/1092024008 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f0d4806f670 0x7f0d48071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:57.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.363+0000 7f0d67e00640 1 --2- 172.21.15.86:0/1092024008 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f0d4806f670 0x7f0d48071b80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0d540af000 tx=0x7f0d540a7000 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:57.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.367+0000 7f0d64c00640 1 -- 172.21.15.86:0/1092024008 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0d680d2030 con 0x7f0d680730b0 2024-04-25T14:06:57.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.559+0000 7f0d6f800640 1 -- 172.21.15.86:0/1092024008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0d68069d00 con 0x7f0d680730b0 2024-04-25T14:06:57.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.559+0000 7f0d64c00640 1 -- 172.21.15.86:0/1092024008 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) ==== 53+0+73 (secure 0 0 0) 0x7f0d5805c470 con 0x7f0d680730b0 2024-04-25T14:06:57.561 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T14:06:57.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.559+0000 7f0d6f800640 1 -- 172.21.15.86:0/1092024008 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f0d4806f670 msgr2=0x7f0d48071b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:57.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.559+0000 7f0d6f800640 1 --2- 172.21.15.86:0/1092024008 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f0d4806f670 0x7f0d48071b80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0d540af000 tx=0x7f0d540a7000 comp rx=0 tx=0).stop 2024-04-25T14:06:57.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.559+0000 7f0d6f800640 1 -- 172.21.15.86:0/1092024008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d680730b0 msgr2=0x7f0d680997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:57.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.559+0000 7f0d6f800640 1 --2- 172.21.15.86:0/1092024008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d680730b0 0x7f0d680997d0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0d5800c760 tx=0x7f0d5800cc20 comp rx=0 tx=0).stop 2024-04-25T14:06:57.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.559+0000 7f0d6f800640 1 -- 172.21.15.86:0/1092024008 shutdown_connections 2024-04-25T14:06:57.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.559+0000 7f0d6f800640 1 --2- 172.21.15.86:0/1092024008 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f0d4806f670 0x7f0d48071b80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.559+0000 7f0d6f800640 1 --2- 172.21.15.86:0/1092024008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d680d16e0 0x7f0d680cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.559+0000 7f0d6f800640 1 --2- 172.21.15.86:0/1092024008 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d680cfca0 0x7f0d680cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.559+0000 7f0d6f800640 1 --2- 172.21.15.86:0/1092024008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d680730b0 0x7f0d680997d0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.559+0000 7f0d6f800640 1 -- 172.21.15.86:0/1092024008 >> 172.21.15.86:0/1092024008 conn(0x7f0d680b4650 msgr2=0x7f0d680b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:57.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.563+0000 7f0d6f800640 1 -- 172.21.15.86:0/1092024008 shutdown_connections 2024-04-25T14:06:57.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:57.563+0000 7f0d6f800640 1 -- 172.21.15.86:0/1092024008 wait complete. 2024-04-25T14:06:57.583 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T14:06:57.583 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T14:06:57.583 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2024-04-25T14:06:57.583 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T14:06:57.583 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T14:06:57.583 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T14:06:57.583 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T14:06:57.583 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T14:06:57.584 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:06:57.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:57.584 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2024-04-25T14:06:57.595 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:06:57.595 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:06:57.595 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:06:57.595 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:06:57.595 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2024-04-25T14:06:57.595 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:06:57.596 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:06:57.596 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2024-04-25T14:06:57.596 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f0 2024-04-25T14:06:57.596 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:06:57.596 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2024-04-25T14:06:57.596 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f1 2024-04-25T14:06:57.596 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:06:57.596 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2024-04-25T14:06:57.596 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T14:06:57.596 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2024-04-25T14:06:57.596 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T14:06:57.596 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 42240sec preferred_lft 42240sec 2024-04-25T14:06:57.596 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::ec4:7aff:fe88:6ded/64 scope link 2024-04-25T14:06:57.596 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:06:57.597 INFO:teuthology.orchestra.run.smithi155.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T14:06:57.597 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 02:42:b1:ff:60:50 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:06:57.597 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T14:06:57.597 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:06:57.597 INFO:teuthology.orchestra.run.smithi155.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T14:06:57.597 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T14:06:57.597 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T14:06:57.597 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:06:57.597 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T14:06:57.597 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:06:57.597 INFO:teuthology.orchestra.run.smithi155.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:06:57.597 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T14:06:57.597 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T14:06:57.597 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:06:57.598 INFO:teuthology.orchestra.run.smithi155.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:06:57.598 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T14:06:57.598 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T14:06:57.598 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:06:57.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:57.599 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T14:06:57.662 INFO:teuthology.orchestra.run.smithi155.stdout:1 2024-04-25T14:06:57.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:57.664 DEBUG:teuthology.orchestra.run.smithi155:> route 2024-04-25T14:06:57.718 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2024-04-25T14:06:57.719 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T14:06:57.719 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T14:06:57.719 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T14:06:57.719 INFO:teuthology.orchestra.run.smithi155.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T14:06:57.719 INFO:teuthology.orchestra.run.smithi155.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T14:06:57.719 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T14:06:57.719 INFO:teuthology.orchestra.run.smithi155.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:06:57.719 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:06:57.719 INFO:teuthology.orchestra.run.smithi155.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:06:57.719 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T14:06:57.720 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:06:57.720 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-04-25T14:06:57.720 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T14:06:57.720 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T14:06:57.720 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T14:06:57.720 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-04-25T14:06:57.811 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:06:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:06:57.853 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:06:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:06:57.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:57.860 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2024-04-25T14:06:57.869 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T14:06:57.869 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T14:06:57.870 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:06:57.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:57.870 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.0 up 2024-04-25T14:06:57.930 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:06:57.930 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T14:06:57.978 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:06:57.979 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:06:57.979 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T14:06:58.034 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T14:06:58.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:58.034 DEBUG:teuthology.orchestra.run.smithi155:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:06:58.138 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:06:58.138 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T14:06:58.138 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:06:58.156 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:06:58.156 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T14:06:58.156 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:06:58.157 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:06:58.157 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:06:58.168 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:06:58.168 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T14:06:58.168 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:06:58.168 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:06:58.168 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:06:58.168 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790" 2024-04-25T14:06:58.168 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790:/ 2024-04-25T14:06:58.168 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb" will pass to kernel 2024-04-25T14:06:58.172 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T14:06:58.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:58.172 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T14:06:58.191 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:06:58.281 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:06:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:06:58.284 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:06:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:06:58.290 INFO:teuthology.orchestra.run.smithi155.stdout:client5951 2024-04-25T14:06:58.291 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:06:58.291 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:06:58.347 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:06:58.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.503+0000 7fa10b800640 1 Processor -- start 2024-04-25T14:06:58.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.503+0000 7fa10b800640 1 -- start start 2024-04-25T14:06:58.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.503+0000 7fa10b800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa1040d16e0 0x7fa1040d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.503+0000 7fa10b800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1040730b0 0x7fa1040734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.503+0000 7fa10b800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1040d2030 con 0x7fa1040cfca0 2024-04-25T14:06:58.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.503+0000 7fa10b800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa104070470 con 0x7fa1040d16e0 2024-04-25T14:06:58.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.503+0000 7fa10b800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa104072bd0 con 0x7fa1040730b0 2024-04-25T14:06:58.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.503+0000 7fa108e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa1040cfca0 0x7fa10406f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51746/0 (socket says 172.21.15.86:51746) 2024-04-25T14:06:58.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.503+0000 7fa108e00640 1 -- 172.21.15.86:0/4228254991 learned_addr learned my addr 172.21.15.86:0/4228254991 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:58.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.503+0000 7fa103e00640 1 -- 172.21.15.86:0/4228254991 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4020706928 0 0) 0x7fa0f4001120 con 0x7fa1040cfca0 2024-04-25T14:06:58.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.503+0000 7fa103e00640 1 -- 172.21.15.86:0/4228254991 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0e8003530 con 0x7fa1040cfca0 2024-04-25T14:06:58.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.503+0000 7fa109800640 1 --2- 172.21.15.86:0/4228254991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1040730b0 0x7fa1040734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.503+0000 7fa0fbe00640 1 --2- 172.21.15.86:0/4228254991 >> v2:172.21.15.155:3300/0 conn(0x7fa1040d16e0 0x7fa1040d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.507+0000 7fa103e00640 1 -- 172.21.15.86:0/4228254991 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2604257611 0 0) 0x7fa0e8003530 con 0x7fa1040cfca0 2024-04-25T14:06:58.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.507+0000 7fa103e00640 1 -- 172.21.15.86:0/4228254991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1040730b0 msgr2=0x7fa1040734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.507+0000 7fa103e00640 1 --2- 172.21.15.86:0/4228254991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1040730b0 0x7fa1040734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.507+0000 7fa103e00640 1 -- 172.21.15.86:0/4228254991 >> v2:172.21.15.155:3300/0 conn(0x7fa1040d16e0 msgr2=0x7fa1040d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.507+0000 7fa103e00640 1 --2- 172.21.15.86:0/4228254991 >> v2:172.21.15.155:3300/0 conn(0x7fa1040d16e0 0x7fa1040d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.507+0000 7fa103e00640 1 -- 172.21.15.86:0/4228254991 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10406fbd0 con 0x7fa1040cfca0 2024-04-25T14:06:58.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.507+0000 7fa103e00640 1 -- 172.21.15.86:0/4228254991 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fa0f4001850 con 0x7fa1040cfca0 2024-04-25T14:06:58.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.507+0000 7fa103e00640 1 -- 172.21.15.86:0/4228254991 >> v1:172.21.15.86:6789/0 conn(0x7fa1040cfca0 legacy=0x7fa10406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.507+0000 7fa103e00640 1 --2- 172.21.15.86:0/4228254991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0e8003a30 0x7fa0e8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.507+0000 7fa103e00640 1 --2- 172.21.15.86:0/4228254991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e8004810 0x7fa0e8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.507+0000 7fa103e00640 1 --2- 172.21.15.86:0/4228254991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1040730b0 0x7fa0e8005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.507+0000 7fa103e00640 1 -- 172.21.15.86:0/4228254991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa104070470 con 0x7fa0e8004810 2024-04-25T14:06:58.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.507+0000 7fa103e00640 1 -- 172.21.15.86:0/4228254991 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa104072bd0 con 0x7fa0e8003a30 2024-04-25T14:06:58.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.507+0000 7fa103e00640 1 -- 172.21.15.86:0/4228254991 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa0e800dc90 con 0x7fa1040730b0 2024-04-25T14:06:58.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.507+0000 7fa0fbe00640 1 --2- 172.21.15.86:0/4228254991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e8004810 0x7fa0e8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.507+0000 7fa108e00640 1 --2- 172.21.15.86:0/4228254991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0e8003a30 0x7fa0e8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.507+0000 7fa109800640 1 --2- 172.21.15.86:0/4228254991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1040730b0 0x7fa0e8005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.507+0000 7fa0fbe00640 1 -- 172.21.15.86:0/4228254991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1040730b0 msgr2=0x7fa0e8005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.507+0000 7fa0fbe00640 1 --2- 172.21.15.86:0/4228254991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1040730b0 0x7fa0e8005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.507+0000 7fa0fbe00640 1 -- 172.21.15.86:0/4228254991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0e8003a30 msgr2=0x7fa0e8003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.507+0000 7fa0fbe00640 1 --2- 172.21.15.86:0/4228254991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0e8003a30 0x7fa0e8003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.507+0000 7fa0fbe00640 1 -- 172.21.15.86:0/4228254991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e8003530 con 0x7fa0e8004810 2024-04-25T14:06:58.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.511+0000 7fa0fbe00640 1 --2- 172.21.15.86:0/4228254991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e8004810 0x7fa0e8004c20 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec009b10 tx=0x7fa0ec01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f7564920a6a342 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:58.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.511+0000 7fa103e00640 1 -- 172.21.15.86:0/4228254991 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec0b2070 con 0x7fa0e8004810 2024-04-25T14:06:58.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.511+0000 7fa103e00640 1 -- 172.21.15.86:0/4228254991 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0ec0a5590 con 0x7fa0e8004810 2024-04-25T14:06:58.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.511+0000 7fa103e00640 1 -- 172.21.15.86:0/4228254991 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec0096b0 con 0x7fa0e8004810 2024-04-25T14:06:58.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.511+0000 7fa10b800640 1 -- 172.21.15.86:0/4228254991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e8004810 msgr2=0x7fa0e8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.511+0000 7fa10b800640 1 --2- 172.21.15.86:0/4228254991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e8004810 0x7fa0e8004c20 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec009b10 tx=0x7fa0ec01cab0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.511+0000 7fa109800640 1 -- 172.21.15.86:0/4228254991 reap_dead start 2024-04-25T14:06:58.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.511+0000 7fa10b800640 1 -- 172.21.15.86:0/4228254991 shutdown_connections 2024-04-25T14:06:58.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.511+0000 7fa10b800640 1 -- 172.21.15.86:0/4228254991 >> 172.21.15.86:0/4228254991 conn(0x7fa1040b4650 msgr2=0x7fa1040b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:58.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.511+0000 7fa10b800640 1 -- 172.21.15.86:0/4228254991 shutdown_connections 2024-04-25T14:06:58.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.511+0000 7fa10b800640 1 -- 172.21.15.86:0/4228254991 wait complete. 2024-04-25T14:06:58.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.511+0000 7fa10b800640 1 Processor -- start 2024-04-25T14:06:58.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.511+0000 7fa10b800640 1 -- start start 2024-04-25T14:06:58.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.515+0000 7fa10b800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e8004810 0x7fa1040cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.515+0000 7fa10b800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa104154c60 0x7fa104155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.515+0000 7fa10b800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1040ca4b0 0x7fa1040ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.515+0000 7fa10b800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa1040734c0 con 0x7fa0e8004810 2024-04-25T14:06:58.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.515+0000 7fa10b800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa104070190 con 0x7fa104154c60 2024-04-25T14:06:58.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.515+0000 7fa10b800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa104073640 con 0x7fa1040ca4b0 2024-04-25T14:06:58.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.515+0000 7fa108e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e8004810 0x7fa1040cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.515+0000 7fa108e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e8004810 0x7fa1040cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56624/0 (socket says 172.21.15.86:56624) 2024-04-25T14:06:58.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.515+0000 7fa108e00640 1 -- 172.21.15.86:0/2372992275 learned_addr learned my addr 172.21.15.86:0/2372992275 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:58.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.515+0000 7fa0fbe00640 1 --2- 172.21.15.86:0/2372992275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa104154c60 0x7fa104155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.515+0000 7fa109800640 1 --2- 172.21.15.86:0/2372992275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1040ca4b0 0x7fa1040ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.515+0000 7fa108e00640 1 -- 172.21.15.86:0/2372992275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1040ca4b0 msgr2=0x7fa1040ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.515+0000 7fa108e00640 1 --2- 172.21.15.86:0/2372992275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1040ca4b0 0x7fa1040ca8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.515+0000 7fa108e00640 1 -- 172.21.15.86:0/2372992275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa104154c60 msgr2=0x7fa104155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.515+0000 7fa108e00640 1 --2- 172.21.15.86:0/2372992275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa104154c60 0x7fa104155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.515+0000 7fa108e00640 1 -- 172.21.15.86:0/2372992275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1040cae00 con 0x7fa0e8004810 2024-04-25T14:06:58.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.515+0000 7fa109800640 1 --2- 172.21.15.86:0/2372992275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1040ca4b0 0x7fa1040ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:58.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.515+0000 7fa108e00640 1 --2- 172.21.15.86:0/2372992275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e8004810 0x7fa1040cc900 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4001b00 tx=0x7fa0f4012630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:58.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.515+0000 7fa101600640 1 -- 172.21.15.86:0/2372992275 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f401c070 con 0x7fa0e8004810 2024-04-25T14:06:58.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.515+0000 7fa101600640 1 -- 172.21.15.86:0/2372992275 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f4015600 con 0x7fa0e8004810 2024-04-25T14:06:58.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.515+0000 7fa101600640 1 -- 172.21.15.86:0/2372992275 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f40159f0 con 0x7fa0e8004810 2024-04-25T14:06:58.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.515+0000 7fa10b800640 1 -- 172.21.15.86:0/2372992275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa104156150 con 0x7fa0e8004810 2024-04-25T14:06:58.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.515+0000 7fa10b800640 1 -- 172.21.15.86:0/2372992275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1040cbe20 con 0x7fa0e8004810 2024-04-25T14:06:58.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.515+0000 7fa10b800640 1 -- 172.21.15.86:0/2372992275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa10409f4b0 con 0x7fa0e8004810 2024-04-25T14:06:58.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.519+0000 7fa101600640 1 -- 172.21.15.86:0/2372992275 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7fa0f4015b90 con 0x7fa0e8004810 2024-04-25T14:06:58.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.519+0000 7fa101600640 1 --2- 172.21.15.86:0/2372992275 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fa0e006f670 0x7fa0e0071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.519+0000 7fa101600640 1 -- 172.21.15.86:0/2372992275 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7fa0f4091e80 con 0x7fa0e8004810 2024-04-25T14:06:58.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.519+0000 7fa0fbe00640 1 --2- 172.21.15.86:0/2372992275 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fa0e006f670 0x7fa0e0071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.519+0000 7fa0fbe00640 1 --2- 172.21.15.86:0/2372992275 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fa0e006f670 0x7fa0e0071b80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec0af000 tx=0x7fa0ec0046d0 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:58.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.523+0000 7fa101600640 1 -- 172.21.15.86:0/2372992275 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa1040cae00 con 0x7fa0e8004810 2024-04-25T14:06:58.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.711+0000 7fa10b800640 1 -- 172.21.15.86:0/2372992275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa104069d00 con 0x7fa0e8004810 2024-04-25T14:06:58.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.711+0000 7fa101600640 1 -- 172.21.15.86:0/2372992275 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5189 (secure 0 0 0) 0x7fa1040d2030 con 0x7fa0e8004810 2024-04-25T14:06:58.715 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:06:58.715 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1210051860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1210051860},{"type":"v1","addr":"172.21.15.155:6833","nonce":1210051860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2868300438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2868300438},{"type":"v1","addr":"172.21.15.155:6835","nonce":2868300438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6600,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2682115891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2682115891},{"type":"v1","addr":"172.21.15.86:6835","nonce":2682115891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:06:44.910717+0000","modified":"2024-04-25T14:06:47.953301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6603},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6603":{"gid":6603,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/1088108173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1088108173},{"type":"v1","addr":"172.21.15.86:6837","nonce":1088108173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6603,"qdb_cluster":[6603]},"id":5}]} 2024-04-25T14:06:58.715 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 53 2024-04-25T14:06:58.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.715+0000 7fa10b800640 1 -- 172.21.15.86:0/2372992275 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fa0e006f670 msgr2=0x7fa0e0071b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.715+0000 7fa10b800640 1 --2- 172.21.15.86:0/2372992275 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fa0e006f670 0x7fa0e0071b80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec0af000 tx=0x7fa0ec0046d0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.715+0000 7fa10b800640 1 -- 172.21.15.86:0/2372992275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e8004810 msgr2=0x7fa1040cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.715+0000 7fa10b800640 1 --2- 172.21.15.86:0/2372992275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e8004810 0x7fa1040cc900 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4001b00 tx=0x7fa0f4012630 comp rx=0 tx=0).stop 2024-04-25T14:06:58.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.715+0000 7fa10b800640 1 -- 172.21.15.86:0/2372992275 shutdown_connections 2024-04-25T14:06:58.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.715+0000 7fa10b800640 1 --2- 172.21.15.86:0/2372992275 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fa0e006f670 0x7fa0e0071b80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.715+0000 7fa10b800640 1 --2- 172.21.15.86:0/2372992275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1040ca4b0 0x7fa1040ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.715+0000 7fa10b800640 1 --2- 172.21.15.86:0/2372992275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa104154c60 0x7fa104155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.715+0000 7fa10b800640 1 --2- 172.21.15.86:0/2372992275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e8004810 0x7fa1040cc900 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.715+0000 7fa10b800640 1 -- 172.21.15.86:0/2372992275 >> 172.21.15.86:0/2372992275 conn(0x7fa1040b4650 msgr2=0x7fa10415c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:58.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.715+0000 7fa10b800640 1 -- 172.21.15.86:0/2372992275 shutdown_connections 2024-04-25T14:06:58.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.715+0000 7fa10b800640 1 -- 172.21.15.86:0/2372992275 wait complete. 2024-04-25T14:06:58.737 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable stats 2024-04-25T14:06:58.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.883+0000 7fc606e00640 1 Processor -- start 2024-04-25T14:06:58.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.883+0000 7fc606e00640 1 -- start start 2024-04-25T14:06:58.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.883+0000 7fc606e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc6000cfca0 0x7fc60006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.883+0000 7fc606e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6000730b0 0x7fc6000734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.883+0000 7fc606e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc600072bd0 con 0x7fc6000d16e0 2024-04-25T14:06:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.883+0000 7fc606e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc600070470 con 0x7fc6000cfca0 2024-04-25T14:06:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.883+0000 7fc606e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc60006f930 con 0x7fc6000730b0 2024-04-25T14:06:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.883+0000 7fc5f7400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc6000d16e0 0x7fc6000d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37752/0 (socket says 172.21.15.86:37752) 2024-04-25T14:06:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.883+0000 7fc5f7400640 1 -- 172.21.15.86:0/3890984651 learned_addr learned my addr 172.21.15.86:0/3890984651 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:58.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.883+0000 7fc604e00640 1 --2- 172.21.15.86:0/3890984651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6000730b0 0x7fc6000734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.883+0000 7fc5ff400640 1 -- 172.21.15.86:0/3890984651 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2224757617 0 0) 0x7fc600072bd0 con 0x7fc6000d16e0 2024-04-25T14:06:58.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.883+0000 7fc5ffe00640 1 --2- 172.21.15.86:0/3890984651 >> v2:172.21.15.155:3300/0 conn(0x7fc6000cfca0 0x7fc60006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.883+0000 7fc5ff400640 1 -- 172.21.15.86:0/3890984651 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5e4003530 con 0x7fc6000d16e0 2024-04-25T14:06:58.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.887+0000 7fc5ff400640 1 -- 172.21.15.86:0/3890984651 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2909218169 0 0) 0x7fc5e4003530 con 0x7fc6000d16e0 2024-04-25T14:06:58.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.887+0000 7fc5ff400640 1 -- 172.21.15.86:0/3890984651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6000730b0 msgr2=0x7fc6000734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.887+0000 7fc5ff400640 1 --2- 172.21.15.86:0/3890984651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6000730b0 0x7fc6000734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.887+0000 7fc5ff400640 1 -- 172.21.15.86:0/3890984651 >> v2:172.21.15.155:3300/0 conn(0x7fc6000cfca0 msgr2=0x7fc60006f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.887+0000 7fc5ff400640 1 --2- 172.21.15.86:0/3890984651 >> v2:172.21.15.155:3300/0 conn(0x7fc6000cfca0 0x7fc60006f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.887+0000 7fc5ff400640 1 -- 172.21.15.86:0/3890984651 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60006fbd0 con 0x7fc6000d16e0 2024-04-25T14:06:58.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.887+0000 7fc5ff400640 1 -- 172.21.15.86:0/3890984651 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fc5e80016a0 con 0x7fc6000d16e0 2024-04-25T14:06:58.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.887+0000 7fc5ff400640 1 -- 172.21.15.86:0/3890984651 >> v1:172.21.15.86:6789/0 conn(0x7fc6000d16e0 legacy=0x7fc6000d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.887+0000 7fc5ff400640 1 --2- 172.21.15.86:0/3890984651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5e4003a30 0x7fc5e4003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.887+0000 7fc5ff400640 1 --2- 172.21.15.86:0/3890984651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6000730b0 0x7fc5e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.887+0000 7fc5ff400640 1 --2- 172.21.15.86:0/3890984651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5e4004d50 0x7fc5e4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.887+0000 7fc5ff400640 1 -- 172.21.15.86:0/3890984651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc600070470 con 0x7fc5e4003a30 2024-04-25T14:06:58.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.887+0000 7fc5ff400640 1 -- 172.21.15.86:0/3890984651 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc60006f930 con 0x7fc5e4004d50 2024-04-25T14:06:58.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.887+0000 7fc5ff400640 1 -- 172.21.15.86:0/3890984651 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc5e400ba80 con 0x7fc6000730b0 2024-04-25T14:06:58.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.887+0000 7fc5ffe00640 1 --2- 172.21.15.86:0/3890984651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5e4003a30 0x7fc5e4003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.887+0000 7fc604e00640 1 --2- 172.21.15.86:0/3890984651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5e4004d50 0x7fc5e4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.887+0000 7fc5f7400640 1 --2- 172.21.15.86:0/3890984651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6000730b0 0x7fc5e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.887+0000 7fc5ffe00640 1 -- 172.21.15.86:0/3890984651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6000730b0 msgr2=0x7fc5e4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.887+0000 7fc5ffe00640 1 --2- 172.21.15.86:0/3890984651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6000730b0 0x7fc5e4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.887+0000 7fc5ffe00640 1 -- 172.21.15.86:0/3890984651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5e4004d50 msgr2=0x7fc5e4005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.887+0000 7fc5ffe00640 1 --2- 172.21.15.86:0/3890984651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5e4004d50 0x7fc5e4005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.887+0000 7fc5ffe00640 1 -- 172.21.15.86:0/3890984651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e4003530 con 0x7fc5e4003a30 2024-04-25T14:06:58.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.891+0000 7fc5ffe00640 1 --2- 172.21.15.86:0/3890984651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5e4003a30 0x7fc5e4003ea0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0009f90 tx=0x7fc5f001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a667854b9324e234 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.891+0000 7fc5ff400640 1 -- 172.21.15.86:0/3890984651 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f00b2070 con 0x7fc5e4003a30 2024-04-25T14:06:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.891+0000 7fc5ff400640 1 -- 172.21.15.86:0/3890984651 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5f00a5b80 con 0x7fc5e4003a30 2024-04-25T14:06:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.891+0000 7fc5ff400640 1 -- 172.21.15.86:0/3890984651 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0009c10 con 0x7fc5e4003a30 2024-04-25T14:06:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.891+0000 7fc606e00640 1 -- 172.21.15.86:0/3890984651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5e4003a30 msgr2=0x7fc5e4003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.891+0000 7fc606e00640 1 --2- 172.21.15.86:0/3890984651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5e4003a30 0x7fc5e4003ea0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0009f90 tx=0x7fc5f001cc80 comp rx=0 tx=0).stop 2024-04-25T14:06:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.891+0000 7fc604e00640 1 -- 172.21.15.86:0/3890984651 reap_dead start 2024-04-25T14:06:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.891+0000 7fc606e00640 1 -- 172.21.15.86:0/3890984651 shutdown_connections 2024-04-25T14:06:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.891+0000 7fc606e00640 1 -- 172.21.15.86:0/3890984651 >> 172.21.15.86:0/3890984651 conn(0x7fc6000b4650 msgr2=0x7fc6000b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.891+0000 7fc606e00640 1 -- 172.21.15.86:0/3890984651 shutdown_connections 2024-04-25T14:06:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.891+0000 7fc606e00640 1 -- 172.21.15.86:0/3890984651 wait complete. 2024-04-25T14:06:58.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.891+0000 7fc606e00640 1 Processor -- start 2024-04-25T14:06:58.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.891+0000 7fc606e00640 1 -- start start 2024-04-25T14:06:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.895+0000 7fc606e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5e4003a30 0x7fc6000c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.895+0000 7fc606e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6000c6190 0x7fc6000c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.895+0000 7fc606e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6000c5c40 0x7fc60015a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.895+0000 7fc606e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc600074060 con 0x7fc6000c5c40 2024-04-25T14:06:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.895+0000 7fc606e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc600073ee0 con 0x7fc5e4003a30 2024-04-25T14:06:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.895+0000 7fc606e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc6000741e0 con 0x7fc6000c6190 2024-04-25T14:06:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.895+0000 7fc604e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6000c5c40 0x7fc60015a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.895+0000 7fc604e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6000c5c40 0x7fc60015a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47606/0 (socket says 172.21.15.86:47606) 2024-04-25T14:06:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.895+0000 7fc604e00640 1 -- 172.21.15.86:0/3404935539 learned_addr learned my addr 172.21.15.86:0/3404935539 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.895+0000 7fc5ffe00640 1 --2- 172.21.15.86:0/3404935539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5e4003a30 0x7fc6000c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.895+0000 7fc5f7400640 1 --2- 172.21.15.86:0/3404935539 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6000c6190 0x7fc6000c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.895+0000 7fc604e00640 1 -- 172.21.15.86:0/3404935539 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6000c6190 msgr2=0x7fc6000c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.895+0000 7fc604e00640 1 --2- 172.21.15.86:0/3404935539 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6000c6190 0x7fc6000c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.895+0000 7fc604e00640 1 -- 172.21.15.86:0/3404935539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5e4003a30 msgr2=0x7fc6000c83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.895+0000 7fc604e00640 1 --2- 172.21.15.86:0/3404935539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5e4003a30 0x7fc6000c83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.895+0000 7fc604e00640 1 -- 172.21.15.86:0/3404935539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6000c8e30 con 0x7fc6000c5c40 2024-04-25T14:06:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.895+0000 7fc5ffe00640 1 --2- 172.21.15.86:0/3404935539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5e4003a30 0x7fc6000c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.895+0000 7fc604e00640 1 --2- 172.21.15.86:0/3404935539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6000c5c40 0x7fc60015a790 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fc6000cb8e0 tx=0x7fc5f8011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.895+0000 7fc5fcc00640 1 -- 172.21.15.86:0/3404935539 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8004030 con 0x7fc6000c5c40 2024-04-25T14:06:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.895+0000 7fc5fcc00640 1 -- 172.21.15.86:0/3404935539 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5f801b070 con 0x7fc6000c5c40 2024-04-25T14:06:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.895+0000 7fc5fcc00640 1 -- 172.21.15.86:0/3404935539 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8004450 con 0x7fc6000c5c40 2024-04-25T14:06:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.895+0000 7fc606e00640 1 -- 172.21.15.86:0/3404935539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6000c9330 con 0x7fc6000c5c40 2024-04-25T14:06:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.895+0000 7fc606e00640 1 -- 172.21.15.86:0/3404935539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6000c98f0 con 0x7fc6000c5c40 2024-04-25T14:06:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.895+0000 7fc606e00640 1 -- 172.21.15.86:0/3404935539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6000c9af0 con 0x7fc6000c5c40 2024-04-25T14:06:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.899+0000 7fc5fcc00640 1 -- 172.21.15.86:0/3404935539 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 93411+0+0 (secure 0 0 0) 0x7fc5f8012040 con 0x7fc6000c5c40 2024-04-25T14:06:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.899+0000 7fc5fcc00640 1 --2- 172.21.15.86:0/3404935539 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fc5dc06f640 0x7fc5dc071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.899+0000 7fc5fcc00640 1 -- 172.21.15.86:0/3404935539 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7fc5f808f940 con 0x7fc6000c5c40 2024-04-25T14:06:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.899+0000 7fc5ffe00640 1 --2- 172.21.15.86:0/3404935539 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fc5dc06f640 0x7fc5dc071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.899+0000 7fc5ffe00640 1 --2- 172.21.15.86:0/3404935539 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fc5dc06f640 0x7fc5dc071b50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0009f90 tx=0x7fc5f00a7040 comp rx=0 tx=0).ready entity=mgr.6198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:58.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:58.903+0000 7fc5fcc00640 1 -- 172.21.15.86:0/3404935539 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc5f805c330 con 0x7fc6000c5c40 2024-04-25T14:06:59.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.099+0000 7fc606e00640 1 -- 172.21.15.86:0/3404935539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "stats"} v 0) -- 0x7fc6000c90c0 con 0x7fc6000c5c40 2024-04-25T14:06:59.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.123+0000 7fc5fcc00640 1 -- 172.21.15.86:0/3404935539 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "stats"}]=0 v18) ==== 84+0+0 (secure 0 0 0) 0x7fc5f805bc30 con 0x7fc6000c5c40 2024-04-25T14:06:59.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.123+0000 7fc606e00640 1 -- 172.21.15.86:0/3404935539 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fc5dc06f640 msgr2=0x7fc5dc071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.123+0000 7fc606e00640 1 --2- 172.21.15.86:0/3404935539 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fc5dc06f640 0x7fc5dc071b50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0009f90 tx=0x7fc5f00a7040 comp rx=0 tx=0).stop 2024-04-25T14:06:59.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.123+0000 7fc606e00640 1 -- 172.21.15.86:0/3404935539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6000c5c40 msgr2=0x7fc60015a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.123+0000 7fc606e00640 1 --2- 172.21.15.86:0/3404935539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6000c5c40 0x7fc60015a790 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fc6000cb8e0 tx=0x7fc5f8011cf0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.123+0000 7fc606e00640 1 -- 172.21.15.86:0/3404935539 shutdown_connections 2024-04-25T14:06:59.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.123+0000 7fc606e00640 1 --2- 172.21.15.86:0/3404935539 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7fc5dc06f640 0x7fc5dc071b50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.123+0000 7fc606e00640 1 --2- 172.21.15.86:0/3404935539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6000c5c40 0x7fc60015a790 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.123+0000 7fc606e00640 1 --2- 172.21.15.86:0/3404935539 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6000c6190 0x7fc6000c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.123+0000 7fc606e00640 1 --2- 172.21.15.86:0/3404935539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5e4003a30 0x7fc6000c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.123+0000 7fc606e00640 1 -- 172.21.15.86:0/3404935539 >> 172.21.15.86:0/3404935539 conn(0x7fc6000b4650 msgr2=0x7fc6000cae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:59.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.123+0000 7fc606e00640 1 -- 172.21.15.86:0/3404935539 shutdown_connections 2024-04-25T14:06:59.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.127+0000 7fc606e00640 1 -- 172.21.15.86:0/3404935539 wait complete. 2024-04-25T14:06:59.151 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:06:59.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.303+0000 7f8cb2c00640 1 Processor -- start 2024-04-25T14:06:59.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.303+0000 7f8cb2c00640 1 -- start start 2024-04-25T14:06:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.303+0000 7f8cb2c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cac0d28f0 0x7f8cac16abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.303+0000 7f8cb2c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cac163940 con 0x7f8cac0cec60 2024-04-25T14:06:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.303+0000 7f8cb2c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cac0d2f40 con 0x7f8cac0cdda0 2024-04-25T14:06:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.303+0000 7f8cb2c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8cac0d2d00 con 0x7f8cac0d28f0 2024-04-25T14:06:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.307+0000 7f8cabe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8cac0cec60 0x7f8cac0cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37776/0 (socket says 172.21.15.86:37776) 2024-04-25T14:06:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.307+0000 7f8cabe00640 1 -- 172.21.15.86:0/1209623750 learned_addr learned my addr 172.21.15.86:0/1209623750 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:06:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.307+0000 7f8cb0c00640 1 --2- 172.21.15.86:0/1209623750 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cac0d28f0 0x7f8cac16abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.307+0000 7f8caaa00640 1 -- 172.21.15.86:0/1209623750 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 897309794 0 0) 0x7f8cac0d2f40 con 0x7f8cac0cdda0 2024-04-25T14:06:59.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.307+0000 7f8caaa00640 1 -- 172.21.15.86:0/1209623750 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c90003530 con 0x7f8cac0cdda0 2024-04-25T14:06:59.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.307+0000 7f8caaa00640 1 -- 172.21.15.86:0/1209623750 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1787997288 0 0) 0x7f8cac163940 con 0x7f8cac0cec60 2024-04-25T14:06:59.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.307+0000 7f8caaa00640 1 -- 172.21.15.86:0/1209623750 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cac0d2f40 con 0x7f8cac0cec60 2024-04-25T14:06:59.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.307+0000 7f8cb0c00640 1 -- 172.21.15.86:0/1209623750 >> v1:172.21.15.155:6789/0 conn(0x7f8cac0cdda0 legacy=0x7f8cac0cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.307+0000 7f8cb0c00640 1 -- 172.21.15.86:0/1209623750 >> v1:172.21.15.86:6789/0 conn(0x7f8cac0cec60 legacy=0x7f8cac0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.307+0000 7f8cb0c00640 1 -- 172.21.15.86:0/1209623750 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cac0b3720 con 0x7f8cac0d28f0 2024-04-25T14:06:59.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.307+0000 7f8cb0c00640 1 --2- 172.21.15.86:0/1209623750 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cac0d28f0 0x7f8cac16abe0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4009dc0 tx=0x7f8ca401c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e498f7365acf933 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:59.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.307+0000 7f8caaa00640 1 -- 172.21.15.86:0/1209623750 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca40b2070 con 0x7f8cac0d28f0 2024-04-25T14:06:59.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.307+0000 7f8caaa00640 1 -- 172.21.15.86:0/1209623750 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ca40a5850 con 0x7f8cac0d28f0 2024-04-25T14:06:59.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.307+0000 7f8caaa00640 1 -- 172.21.15.86:0/1209623750 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca40a5c20 con 0x7f8cac0d28f0 2024-04-25T14:06:59.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.311+0000 7f8cb2c00640 1 -- 172.21.15.86:0/1209623750 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cac0d28f0 msgr2=0x7f8cac16abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.311+0000 7f8cb2c00640 1 --2- 172.21.15.86:0/1209623750 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cac0d28f0 0x7f8cac16abe0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4009dc0 tx=0x7f8ca401c990 comp rx=0 tx=0).stop 2024-04-25T14:06:59.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.311+0000 7f8cb2c00640 1 -- 172.21.15.86:0/1209623750 shutdown_connections 2024-04-25T14:06:59.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.311+0000 7f8cb2c00640 1 --2- 172.21.15.86:0/1209623750 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cac0d28f0 0x7f8cac16abe0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.311+0000 7f8cb2c00640 1 -- 172.21.15.86:0/1209623750 >> 172.21.15.86:0/1209623750 conn(0x7f8cac09ba30 msgr2=0x7f8cac0b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:59.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.311+0000 7f8cb2c00640 1 -- 172.21.15.86:0/1209623750 shutdown_connections 2024-04-25T14:06:59.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.311+0000 7f8cb2c00640 1 -- 172.21.15.86:0/1209623750 wait complete. 2024-04-25T14:06:59.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.311+0000 7f8cb2c00640 1 Processor -- start 2024-04-25T14:06:59.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.311+0000 7f8cb2c00640 1 -- start start 2024-04-25T14:06:59.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.315+0000 7f8cb2c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cac0cdda0 0x7f8cac15ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.315+0000 7f8cb2c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cac0cec60 0x7f8cac15f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.315+0000 7f8cb2c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cac0d28f0 0x7f8cac0c6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.315+0000 7f8cb2c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac16dcd0 con 0x7f8cac0cdda0 2024-04-25T14:06:59.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.315+0000 7f8cb2c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac16db50 con 0x7f8cac0cec60 2024-04-25T14:06:59.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.315+0000 7f8cb2c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8cac16de50 con 0x7f8cac0d28f0 2024-04-25T14:06:59.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.315+0000 7f8cabe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cac0cdda0 0x7f8cac15ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.315+0000 7f8cabe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cac0cdda0 0x7f8cac15ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47634/0 (socket says 172.21.15.86:47634) 2024-04-25T14:06:59.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.315+0000 7f8cabe00640 1 -- 172.21.15.86:0/1860317347 learned_addr learned my addr 172.21.15.86:0/1860317347 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:06:59.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.315+0000 7f8cb0c00640 1 --2- 172.21.15.86:0/1860317347 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cac0d28f0 0x7f8cac0c6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.315+0000 7f8cab400640 1 --2- 172.21.15.86:0/1860317347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cac0cec60 0x7f8cac15f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.315+0000 7f8cabe00640 1 -- 172.21.15.86:0/1860317347 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cac0d28f0 msgr2=0x7f8cac0c6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.315+0000 7f8cabe00640 1 --2- 172.21.15.86:0/1860317347 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cac0d28f0 0x7f8cac0c6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.315+0000 7f8cabe00640 1 -- 172.21.15.86:0/1860317347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cac0cec60 msgr2=0x7f8cac15f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.315+0000 7f8cabe00640 1 --2- 172.21.15.86:0/1860317347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cac0cec60 0x7f8cac15f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.315+0000 7f8cabe00640 1 -- 172.21.15.86:0/1860317347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cac0c64c0 con 0x7f8cac0cdda0 2024-04-25T14:06:59.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.315+0000 7f8cabe00640 1 --2- 172.21.15.86:0/1860317347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cac0cdda0 0x7f8cac15ab00 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00c430 tx=0x7f8c9c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:59.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.315+0000 7f8ca3e00640 1 -- 172.21.15.86:0/1860317347 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c017070 con 0x7f8cac0cdda0 2024-04-25T14:06:59.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.315+0000 7f8ca3e00640 1 -- 172.21.15.86:0/1860317347 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c9c007500 con 0x7f8cac0cdda0 2024-04-25T14:06:59.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.315+0000 7f8cb2c00640 1 -- 172.21.15.86:0/1860317347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cac0afb50 con 0x7f8cac0cdda0 2024-04-25T14:06:59.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.315+0000 7f8ca3e00640 1 -- 172.21.15.86:0/1860317347 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0059f0 con 0x7f8cac0cdda0 2024-04-25T14:06:59.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.315+0000 7f8cb2c00640 1 -- 172.21.15.86:0/1860317347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cac0b0110 con 0x7f8cac0cdda0 2024-04-25T14:06:59.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.315+0000 7f8cb2c00640 1 -- 172.21.15.86:0/1860317347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cac0a04c0 con 0x7f8cac0cdda0 2024-04-25T14:06:59.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.319+0000 7f8ca3e00640 1 -- 172.21.15.86:0/1860317347 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 93420+0+0 (secure 0 0 0) 0x7f8c9c01a070 con 0x7f8cac0cdda0 2024-04-25T14:06:59.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.319+0000 7f8ca3e00640 1 --2- 172.21.15.86:0/1860317347 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8c8c06f6c0 0x7f8c8c071bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.319+0000 7f8cab400640 1 -- 172.21.15.86:0/1860317347 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8c8c06f6c0 msgr2=0x7f8c8c071bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/1025829557 2024-04-25T14:06:59.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.319+0000 7f8ca3e00640 1 -- 172.21.15.86:0/1860317347 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f8c9c08ecb0 con 0x7f8cac0cdda0 2024-04-25T14:06:59.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.319+0000 7f8cab400640 1 --2- 172.21.15.86:0/1860317347 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8c8c06f6c0 0x7f8c8c071bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:06:59.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.327+0000 7f8ca3e00640 1 -- 172.21.15.86:0/1860317347 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8c9c05afe0 con 0x7f8cac0cdda0 2024-04-25T14:06:59.378 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:06:59.377+0000 7fe07b26d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:06:59.392 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:59.391+0000 7f85c0c2c180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:06:59.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.511+0000 7f8cb2c00640 1 -- 172.21.15.86:0/1860317347 --> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f8cac076bf0 con 0x7f8c8c06f6c0 2024-04-25T14:06:59.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.523+0000 7f8cab400640 1 -- 172.21.15.86:0/1860317347 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8c8c06f6c0 msgr2=0x7f8c8c071bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/1025829557 2024-04-25T14:06:59.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.523+0000 7f8cab400640 1 --2- 172.21.15.86:0/1860317347 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8c8c06f6c0 0x7f8c8c071bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:06:59.797 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:06:59.797+0000 7fe07b26d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:06:59.812 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:59.811+0000 7f85c0c2c180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:06:59.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.927+0000 7f8cab400640 1 -- 172.21.15.86:0/1860317347 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8c8c06f6c0 msgr2=0x7f8c8c071bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/1025829557 2024-04-25T14:06:59.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:06:59.927+0000 7f8cab400640 1 --2- 172.21.15.86:0/1860317347 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8c8c06f6c0 0x7f8c8c071bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T14:06:59.946 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:06:59.945+0000 7fe07b26d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:06:59.961 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:06:59.959+0000 7f85c0c2c180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:07:00.417 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:07:00.417+0000 7fe07b26d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:07:00.433 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:00.431+0000 7f85c0c2c180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:07:00.475 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:07:00.477+0000 7fe07b26d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:07:00.491 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:00.491+0000 7f85c0c2c180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:07:00.531 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:07:00.533+0000 7fe07b26d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:07:00.550 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:00.547+0000 7f85c0c2c180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:07:00.621 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:07:00.621+0000 7fe07b26d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:07:00.638 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:00.635+0000 7f85c0c2c180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:07:00.678 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:07:00.677+0000 7fe07b26d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:07:00.695 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:00.691+0000 7f85c0c2c180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:07:00.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:00.731+0000 7f8cab400640 1 -- 172.21.15.86:0/1860317347 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8c8c06f6c0 msgr2=0x7f8c8c071bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/1025829557 2024-04-25T14:07:00.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:00.731+0000 7f8cab400640 1 --2- 172.21.15.86:0/1860317347 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8c8c06f6c0 0x7f8c8c071bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T14:07:00.762 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:07:00.761+0000 7fe07b26d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:07:00.779 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:00.779+0000 7f85c0c2c180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:07:00.821 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:07:00.821+0000 7fe07b26d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:07:00.838 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:00.835+0000 7f85c0c2c180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:07:00.973 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:07:00.973+0000 7fe07b26d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:07:00.990 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:00.987+0000 7f85c0c2c180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:07:01.032 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:07:01.033+0000 7fe07b26d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:07:01.049 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:01.047+0000 7f85c0c2c180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:07:01.271 INFO:tasks.ceph.mgr.x.smithi155.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:07:01.271 INFO:tasks.ceph.mgr.x.smithi155.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:07:01.271 INFO:tasks.ceph.mgr.x.smithi155.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:07:01.279 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:07:01.281+0000 7fe07b26d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:07:01.288 INFO:tasks.ceph.mgr.y.smithi086.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:07:01.288 INFO:tasks.ceph.mgr.y.smithi086.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:07:01.288 INFO:tasks.ceph.mgr.y.smithi086.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:07:01.297 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:01.295+0000 7f85c0c2c180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:07:01.428 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:07:01.429+0000 7fe07b26d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:07:01.445 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:01.443+0000 7f85c0c2c180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:07:01.556 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:07:01.557+0000 7fe07b26d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:07:01.574 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:01.571+0000 7f85c0c2c180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:07:01.627 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:07:01.629+0000 7fe07b26d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:07:01.646 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:01.643+0000 7f85c0c2c180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:07:01.776 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:07:01.777+0000 7fe07b26d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:07:01.796 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:01.795+0000 7f85c0c2c180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:07:02.320 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:07:02.321+0000 7fe07b26d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:07:02.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:02.335+0000 7f8cab400640 1 -- 172.21.15.86:0/1860317347 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8c8c06f6c0 msgr2=0x7f8c8c071bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/1025829557 2024-04-25T14:07:02.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:02.335+0000 7f8cab400640 1 --2- 172.21.15.86:0/1860317347 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8c8c06f6c0 0x7f8c8c071bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T14:07:02.340 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:02.339+0000 7f85c0c2c180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:07:02.403 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:07:02.405+0000 7fe07b26d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:07:02.425 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:02.423+0000 7f85c0c2c180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:07:02.562 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:07:02.561+0000 7fe07b26d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:07:02.585 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:02.583+0000 7f85c0c2c180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:07:02.645 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:07:02.645+0000 7fe07b26d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:07:02.669 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:02.667+0000 7f85c0c2c180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:07:02.965 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:07:02.965+0000 7fe07b26d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:07:02.988 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:02.987+0000 7f85c0c2c180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:07:03.027 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:07:03.025+0000 7fe07b26d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:07:03.051 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:03.047+0000 7f85c0c2c180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:07:03.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:03.063+0000 7f8ca3e00640 1 -- 172.21.15.86:0/1860317347 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 19) ==== 92927+0+0 (secure 0 0 0) 0x7f8c9c057270 con 0x7f8cac0cdda0 2024-04-25T14:07:03.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:03.063+0000 7f8ca3e00640 1 -- 172.21.15.86:0/1860317347 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8c8c06f6c0 msgr2=0x7f8c8c071bd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:07:03.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:03.063+0000 7f8ca3e00640 1 --2- 172.21.15.86:0/1860317347 >> [v2:172.21.15.86:6832/1025829557,v1:172.21.15.86:6833/1025829557] conn(0x7f8c8c06f6c0 0x7f8c8c071bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:03.132 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:03.131+0000 7f8540400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:07:03.132 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:03.131+0000 7f8540400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:07:03.132 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:03.131+0000 7f8540400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:07:03.132 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:03.131+0000 7f8540400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:07:03.132 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:03.131+0000 7f8540400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:07:03.133 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:03.131+0000 7f853c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:07:03.133 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:03.131+0000 7f853c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:07:03.133 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:03.131+0000 7f853c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:07:03.133 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:03.131+0000 7f853c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:07:03.133 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:07:03.131+0000 7f853c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:07:04.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:04.095+0000 7f8ca3e00640 1 -- 172.21.15.86:0/1860317347 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mgrmap(e 20) ==== 93054+0+0 (secure 0 0 0) 0x7f8c9c05b240 con 0x7f8cac0cdda0 2024-04-25T14:07:04.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:04.095+0000 7f8ca3e00640 1 --2- 172.21.15.86:0/1860317347 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f8c8c077c00 0x7f8c8c07a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:04.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:04.095+0000 7f8ca3e00640 1 -- 172.21.15.86:0/1860317347 --> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f8c9c045940 con 0x7f8c8c077c00 2024-04-25T14:07:04.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:04.099+0000 7f8cab400640 1 --2- 172.21.15.86:0/1860317347 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f8c8c077c00 0x7f8c8c07a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:04.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:04.103+0000 7f8cab400640 1 --2- 172.21.15.86:0/1860317347 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f8c8c077c00 0x7f8c8c07a020 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8c980011f0 tx=0x7f8c98009040 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:04.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:04.111+0000 7f8ca3e00640 1 -- 172.21.15.86:0/1860317347 <== mgr.6801 v2:172.21.15.86:6832/203029814 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f8c9c045940 con 0x7f8c8c077c00 2024-04-25T14:07:04.111 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {}, "global_metrics": {}, "metrics": {"delayed_ranks": []}} 2024-04-25T14:07:04.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:04.111+0000 7f8cb2c00640 1 -- 172.21.15.86:0/1860317347 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f8c8c077c00 msgr2=0x7f8c8c07a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:04.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:04.111+0000 7f8cb2c00640 1 --2- 172.21.15.86:0/1860317347 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f8c8c077c00 0x7f8c8c07a020 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8c980011f0 tx=0x7f8c98009040 comp rx=0 tx=0).stop 2024-04-25T14:07:04.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:04.111+0000 7f8cb2c00640 1 -- 172.21.15.86:0/1860317347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cac0cdda0 msgr2=0x7f8cac15ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:04.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:04.111+0000 7f8cb2c00640 1 --2- 172.21.15.86:0/1860317347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cac0cdda0 0x7f8cac15ab00 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00c430 tx=0x7f8c9c00c8f0 comp rx=0 tx=0).stop 2024-04-25T14:07:04.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:04.111+0000 7f8cb0c00640 1 -- 172.21.15.86:0/1860317347 reap_dead start 2024-04-25T14:07:04.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:04.111+0000 7f8cb2c00640 1 -- 172.21.15.86:0/1860317347 shutdown_connections 2024-04-25T14:07:04.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:04.111+0000 7f8cb2c00640 1 -- 172.21.15.86:0/1860317347 >> 172.21.15.86:0/1860317347 conn(0x7f8cac09ba30 msgr2=0x7f8cac15b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:04.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:04.111+0000 7f8cb2c00640 1 -- 172.21.15.86:0/1860317347 shutdown_connections 2024-04-25T14:07:04.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:04.111+0000 7f8cb2c00640 1 -- 172.21.15.86:0/1860317347 wait complete. 2024-04-25T14:07:05.138 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:07:05.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.287+0000 7f9cca800640 1 Processor -- start 2024-04-25T14:07:05.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.287+0000 7f9cca800640 1 -- start start 2024-04-25T14:07:05.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.291+0000 7f9cca800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9cc4070330 0x7f9cc40d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:05.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.291+0000 7f9cca800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cc40d1c20 0x7f9cc406ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:05.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.291+0000 7f9cca800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9cc406f270 0x7f9cc406f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:05.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.291+0000 7f9cca800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9cc40d2030 con 0x7f9cc4070330 2024-04-25T14:07:05.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.291+0000 7f9cca800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9cc415e5c0 con 0x7f9cc406f270 2024-04-25T14:07:05.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.291+0000 7f9cca800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9cc415e8a0 con 0x7f9cc40d1c20 2024-04-25T14:07:05.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.291+0000 7f9cc3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9cc4070330 0x7f9cc40d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:05.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.291+0000 7f9cc3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9cc4070330 0x7f9cc40d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47794/0 (socket says 172.21.15.86:47794) 2024-04-25T14:07:05.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.291+0000 7f9cc3400640 1 -- 172.21.15.86:0/4118652630 learned_addr learned my addr 172.21.15.86:0/4118652630 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:05.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.291+0000 7f9cbbe00640 1 --2- 172.21.15.86:0/4118652630 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cc40d1c20 0x7f9cc406ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:05.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.291+0000 7f9cc3e00640 1 --2- 172.21.15.86:0/4118652630 >> v2:172.21.15.155:3300/0 conn(0x7f9cc406f270 0x7f9cc406f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:05.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.291+0000 7f9cc3400640 1 -- 172.21.15.86:0/4118652630 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cc40d1c20 msgr2=0x7f9cc406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:05.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.291+0000 7f9cc3400640 1 --2- 172.21.15.86:0/4118652630 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cc40d1c20 0x7f9cc406ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:05.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.291+0000 7f9cc3400640 1 -- 172.21.15.86:0/4118652630 >> v2:172.21.15.155:3300/0 conn(0x7f9cc406f270 msgr2=0x7f9cc406f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:05.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.291+0000 7f9cc3400640 1 --2- 172.21.15.86:0/4118652630 >> v2:172.21.15.155:3300/0 conn(0x7f9cc406f270 0x7f9cc406f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:05.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.291+0000 7f9cc3400640 1 -- 172.21.15.86:0/4118652630 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc415eb80 con 0x7f9cc4070330 2024-04-25T14:07:05.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.291+0000 7f9cc3e00640 1 --2- 172.21.15.86:0/4118652630 >> v2:172.21.15.155:3300/0 conn(0x7f9cc406f270 0x7f9cc406f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:05.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.295+0000 7f9cc3400640 1 --2- 172.21.15.86:0/4118652630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cc4070330 0x7f9cc40d16e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f9cc406e9e0 tx=0x7f9cb401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3aeea37d26dd639 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:05.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.295+0000 7f9cc2a00640 1 -- 172.21.15.86:0/4118652630 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb40ab600 con 0x7f9cc4070330 2024-04-25T14:07:05.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.295+0000 7f9cc2a00640 1 -- 172.21.15.86:0/4118652630 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb40b0070 con 0x7f9cc4070330 2024-04-25T14:07:05.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.295+0000 7f9cc2a00640 1 -- 172.21.15.86:0/4118652630 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb40ab9c0 con 0x7f9cc4070330 2024-04-25T14:07:05.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.295+0000 7f9cca800640 1 -- 172.21.15.86:0/4118652630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cc4070330 msgr2=0x7f9cc40d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:05.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.295+0000 7f9cca800640 1 --2- 172.21.15.86:0/4118652630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cc4070330 0x7f9cc40d16e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f9cc406e9e0 tx=0x7f9cb401c460 comp rx=0 tx=0).stop 2024-04-25T14:07:05.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.295+0000 7f9cca800640 1 -- 172.21.15.86:0/4118652630 shutdown_connections 2024-04-25T14:07:05.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.295+0000 7f9cca800640 1 --2- 172.21.15.86:0/4118652630 >> v2:172.21.15.155:3300/0 conn(0x7f9cc406f270 0x7f9cc406f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:05.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.295+0000 7f9cca800640 1 --2- 172.21.15.86:0/4118652630 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cc40d1c20 0x7f9cc406ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:05.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.295+0000 7f9cca800640 1 --2- 172.21.15.86:0/4118652630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cc4070330 0x7f9cc40d16e0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:05.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.295+0000 7f9cca800640 1 -- 172.21.15.86:0/4118652630 >> 172.21.15.86:0/4118652630 conn(0x7f9cc40b4650 msgr2=0x7f9cc40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:05.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.295+0000 7f9cca800640 1 -- 172.21.15.86:0/4118652630 shutdown_connections 2024-04-25T14:07:05.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.295+0000 7f9cca800640 1 -- 172.21.15.86:0/4118652630 wait complete. 2024-04-25T14:07:05.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.299+0000 7f9cca800640 1 Processor -- start 2024-04-25T14:07:05.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.299+0000 7f9cca800640 1 -- start start 2024-04-25T14:07:05.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.299+0000 7f9cca800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cc406f270 0x7f9cc409aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:05.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.299+0000 7f9cca800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cc4070330 0x7f9cc4147780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:05.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.299+0000 7f9cca800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cc40d1c20 0x7f9cc4147cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:05.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.299+0000 7f9cca800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc415f4b0 con 0x7f9cc406f270 2024-04-25T14:07:05.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.299+0000 7f9cca800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc415f330 con 0x7f9cc40d1c20 2024-04-25T14:07:05.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.299+0000 7f9cca800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9cc4070740 con 0x7f9cc4070330 2024-04-25T14:07:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.299+0000 7f9cc3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cc406f270 0x7f9cc409aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.299+0000 7f9cc3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cc406f270 0x7f9cc409aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47810/0 (socket says 172.21.15.86:47810) 2024-04-25T14:07:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.299+0000 7f9cc3400640 1 -- 172.21.15.86:0/2649468709 learned_addr learned my addr 172.21.15.86:0/2649468709 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.299+0000 7f9cbbe00640 1 --2- 172.21.15.86:0/2649468709 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cc4070330 0x7f9cc4147780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.299+0000 7f9cc3e00640 1 --2- 172.21.15.86:0/2649468709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cc40d1c20 0x7f9cc4147cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.299+0000 7f9cc3400640 1 -- 172.21.15.86:0/2649468709 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cc4070330 msgr2=0x7f9cc4147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.299+0000 7f9cc3400640 1 --2- 172.21.15.86:0/2649468709 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cc4070330 0x7f9cc4147780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.299+0000 7f9cc3400640 1 -- 172.21.15.86:0/2649468709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cc40d1c20 msgr2=0x7f9cc4147cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.299+0000 7f9cc3400640 1 --2- 172.21.15.86:0/2649468709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cc40d1c20 0x7f9cc4147cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.299+0000 7f9cc3400640 1 -- 172.21.15.86:0/2649468709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc4148540 con 0x7f9cc406f270 2024-04-25T14:07:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.299+0000 7f9cc3400640 1 --2- 172.21.15.86:0/2649468709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cc406f270 0x7f9cc409aa70 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f9cb401ab10 tx=0x7f9cb401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.299+0000 7f9cbb400640 1 -- 172.21.15.86:0/2649468709 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb40be5f0 con 0x7f9cc406f270 2024-04-25T14:07:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.299+0000 7f9cbb400640 1 -- 172.21.15.86:0/2649468709 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb40abe50 con 0x7f9cc406f270 2024-04-25T14:07:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.299+0000 7f9cbb400640 1 -- 172.21.15.86:0/2649468709 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb40be9f0 con 0x7f9cc406f270 2024-04-25T14:07:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.299+0000 7f9cca800640 1 -- 172.21.15.86:0/2649468709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc4159780 con 0x7f9cc406f270 2024-04-25T14:07:05.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.299+0000 7f9cca800640 1 -- 172.21.15.86:0/2649468709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc40c8520 con 0x7f9cc406f270 2024-04-25T14:07:05.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.303+0000 7f9cca800640 1 -- 172.21.15.86:0/2649468709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cc40cb730 con 0x7f9cc406f270 2024-04-25T14:07:05.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.303+0000 7f9cbb400640 1 -- 172.21.15.86:0/2649468709 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 21) ==== 93262+0+0 (secure 0 0 0) 0x7f9cb40aa070 con 0x7f9cc406f270 2024-04-25T14:07:05.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.303+0000 7f9cbb400640 1 --2- 172.21.15.86:0/2649468709 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f9ca006f510 0x7f9ca0071a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:05.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.303+0000 7f9cbb400640 1 -- 172.21.15.86:0/2649468709 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f9cb4129460 con 0x7f9cc406f270 2024-04-25T14:07:05.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.303+0000 7f9cbbe00640 1 --2- 172.21.15.86:0/2649468709 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f9ca006f510 0x7f9ca0071a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:05.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.303+0000 7f9cbbe00640 1 --2- 172.21.15.86:0/2649468709 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f9ca006f510 0x7f9ca0071a20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40ae980 tx=0x7f9cac009190 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:05.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.307+0000 7f9cbb400640 1 -- 172.21.15.86:0/2649468709 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9cc40cb730 con 0x7f9cc406f270 2024-04-25T14:07:05.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.483+0000 7f9cca800640 1 -- 172.21.15.86:0/2649468709 --> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f9cc4069d00 con 0x7f9ca006f510 2024-04-25T14:07:05.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.483+0000 7f9cbb400640 1 -- 172.21.15.86:0/2649468709 <== mgr.6801 v2:172.21.15.86:6832/203029814 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f9cc4069d00 con 0x7f9ca006f510 2024-04-25T14:07:05.487 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {}, "global_metrics": {}, "metrics": {"delayed_ranks": []}} 2024-04-25T14:07:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.487+0000 7f9cca800640 1 -- 172.21.15.86:0/2649468709 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f9ca006f510 msgr2=0x7f9ca0071a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.487+0000 7f9cca800640 1 --2- 172.21.15.86:0/2649468709 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f9ca006f510 0x7f9ca0071a20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40ae980 tx=0x7f9cac009190 comp rx=0 tx=0).stop 2024-04-25T14:07:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.487+0000 7f9cca800640 1 -- 172.21.15.86:0/2649468709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cc406f270 msgr2=0x7f9cc409aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.487+0000 7f9cca800640 1 --2- 172.21.15.86:0/2649468709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cc406f270 0x7f9cc409aa70 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f9cb401ab10 tx=0x7f9cb401aca0 comp rx=0 tx=0).stop 2024-04-25T14:07:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.487+0000 7f9cca800640 1 -- 172.21.15.86:0/2649468709 shutdown_connections 2024-04-25T14:07:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.487+0000 7f9cca800640 1 --2- 172.21.15.86:0/2649468709 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f9ca006f510 0x7f9ca0071a20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.487+0000 7f9cca800640 1 --2- 172.21.15.86:0/2649468709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cc40d1c20 0x7f9cc4147cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.487+0000 7f9cca800640 1 --2- 172.21.15.86:0/2649468709 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cc4070330 0x7f9cc4147780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.487+0000 7f9cca800640 1 --2- 172.21.15.86:0/2649468709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cc406f270 0x7f9cc409aa70 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.487+0000 7f9cca800640 1 -- 172.21.15.86:0/2649468709 >> 172.21.15.86:0/2649468709 conn(0x7f9cc40b4650 msgr2=0x7f9cc415b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:05.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.487+0000 7f9cca800640 1 -- 172.21.15.86:0/2649468709 shutdown_connections 2024-04-25T14:07:05.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:05.487+0000 7f9cca800640 1 -- 172.21.15.86:0/2649468709 wait complete. 2024-04-25T14:07:06.510 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:07:06.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.663+0000 7f2cba200640 1 Processor -- start 2024-04-25T14:07:06.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.663+0000 7f2cba200640 1 -- start start 2024-04-25T14:07:06.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.663+0000 7f2cba200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2cb40cfca0 0x7f2cb406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:06.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.663+0000 7f2cba200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2cb40730b0 0x7f2cb40734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:06.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.663+0000 7f2cba200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cb4072bd0 con 0x7f2cb40d16e0 2024-04-25T14:07:06.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.663+0000 7f2cba200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2cb4070470 con 0x7f2cb40730b0 2024-04-25T14:07:06.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.663+0000 7f2cba200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f2cb406f930 con 0x7f2cb40cfca0 2024-04-25T14:07:06.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.663+0000 7f2cb3e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2cb40730b0 0x7f2cb40734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:06.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.663+0000 7f2cb3e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2cb40730b0 0x7f2cb40734c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47814/0 (socket says 172.21.15.86:47814) 2024-04-25T14:07:06.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.663+0000 7f2cb3e00640 1 -- 172.21.15.86:0/3794718531 learned_addr learned my addr 172.21.15.86:0/3794718531 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:06.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.663+0000 7f2cb3400640 1 --2- 172.21.15.86:0/3794718531 >> v2:172.21.15.155:3300/0 conn(0x7f2cb40cfca0 0x7f2cb406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:06.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.663+0000 7f2cb3e00640 1 -- 172.21.15.86:0/3794718531 >> v2:172.21.15.155:3300/0 conn(0x7f2cb40cfca0 msgr2=0x7f2cb406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:06.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.663+0000 7f2cb3e00640 1 --2- 172.21.15.86:0/3794718531 >> v2:172.21.15.155:3300/0 conn(0x7f2cb40cfca0 0x7f2cb406f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:06.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.663+0000 7f2cb3e00640 1 -- 172.21.15.86:0/3794718531 >> v1:172.21.15.155:6789/0 conn(0x7f2cb40d16e0 legacy=0x7f2cb40d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:06.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.663+0000 7f2cb3e00640 1 -- 172.21.15.86:0/3794718531 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb406fbd0 con 0x7f2cb40730b0 2024-04-25T14:07:06.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.663+0000 7f2cb3400640 1 --2- 172.21.15.86:0/3794718531 >> v2:172.21.15.155:3300/0 conn(0x7f2cb40cfca0 0x7f2cb406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:06.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.667+0000 7f2cb3e00640 1 --2- 172.21.15.86:0/3794718531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cb40730b0 0x7f2cb40734c0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f2cb40d22a0 tx=0x7f2cac01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c323bdabb266eeba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:06.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.667+0000 7f2cb2a00640 1 -- 172.21.15.86:0/3794718531 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac0a5780 con 0x7f2cb40730b0 2024-04-25T14:07:06.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.667+0000 7f2cb2a00640 1 -- 172.21.15.86:0/3794718531 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cac0a5920 con 0x7f2cb40730b0 2024-04-25T14:07:06.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.667+0000 7f2cb2a00640 1 -- 172.21.15.86:0/3794718531 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac0a5cf0 con 0x7f2cb40730b0 2024-04-25T14:07:06.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.667+0000 7f2cba200640 1 -- 172.21.15.86:0/3794718531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cb40730b0 msgr2=0x7f2cb40734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:06.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.667+0000 7f2cba200640 1 --2- 172.21.15.86:0/3794718531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cb40730b0 0x7f2cb40734c0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f2cb40d22a0 tx=0x7f2cac01c9b0 comp rx=0 tx=0).stop 2024-04-25T14:07:06.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.667+0000 7f2cba200640 1 -- 172.21.15.86:0/3794718531 shutdown_connections 2024-04-25T14:07:06.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.667+0000 7f2cba200640 1 --2- 172.21.15.86:0/3794718531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cb40730b0 0x7f2cb40734c0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:06.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.667+0000 7f2cba200640 1 --2- 172.21.15.86:0/3794718531 >> v2:172.21.15.155:3300/0 conn(0x7f2cb40cfca0 0x7f2cb406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:06.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.667+0000 7f2cba200640 1 -- 172.21.15.86:0/3794718531 >> 172.21.15.86:0/3794718531 conn(0x7f2cb40b4650 msgr2=0x7f2cb40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:06.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.667+0000 7f2cba200640 1 -- 172.21.15.86:0/3794718531 shutdown_connections 2024-04-25T14:07:06.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.667+0000 7f2cba200640 1 -- 172.21.15.86:0/3794718531 wait complete. 2024-04-25T14:07:06.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.671+0000 7f2cba200640 1 Processor -- start 2024-04-25T14:07:06.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.671+0000 7f2cba200640 1 -- start start 2024-04-25T14:07:06.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.671+0000 7f2cba200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb40730b0 0x7f2cb409a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:06.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.671+0000 7f2cba200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cb40cfca0 0x7f2cb40cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:06.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.671+0000 7f2cba200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2cb40d16e0 0x7f2cb4158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:06.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.671+0000 7f2cba200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb4169a70 con 0x7f2cb40cfca0 2024-04-25T14:07:06.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.671+0000 7f2cba200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb41698f0 con 0x7f2cb40730b0 2024-04-25T14:07:06.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.671+0000 7f2cba200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2cb4169bf0 con 0x7f2cb40d16e0 2024-04-25T14:07:06.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.671+0000 7f2caaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cb40cfca0 0x7f2cb40cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:06.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.671+0000 7f2caaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cb40cfca0 0x7f2cb40cacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47828/0 (socket says 172.21.15.86:47828) 2024-04-25T14:07:06.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.671+0000 7f2caaa00640 1 -- 172.21.15.86:0/2027352651 learned_addr learned my addr 172.21.15.86:0/2027352651 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:06.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.671+0000 7f2cb3400640 1 --2- 172.21.15.86:0/2027352651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb40730b0 0x7f2cb409a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:06.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.671+0000 7f2cb3e00640 1 --2- 172.21.15.86:0/2027352651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2cb40d16e0 0x7f2cb4158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:06.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.675+0000 7f2caaa00640 1 -- 172.21.15.86:0/2027352651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2cb40d16e0 msgr2=0x7f2cb4158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:06.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.675+0000 7f2caaa00640 1 --2- 172.21.15.86:0/2027352651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2cb40d16e0 0x7f2cb4158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:06.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.675+0000 7f2caaa00640 1 -- 172.21.15.86:0/2027352651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb40730b0 msgr2=0x7f2cb409a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:06.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.675+0000 7f2caaa00640 1 --2- 172.21.15.86:0/2027352651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb40730b0 0x7f2cb409a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:06.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.675+0000 7f2caaa00640 1 -- 172.21.15.86:0/2027352651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb40ae5c0 con 0x7f2cb40cfca0 2024-04-25T14:07:06.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.675+0000 7f2cb3e00640 1 --2- 172.21.15.86:0/2027352651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2cb40d16e0 0x7f2cb4158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:06.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.675+0000 7f2cb3400640 1 --2- 172.21.15.86:0/2027352651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb40730b0 0x7f2cb409a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:06.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.675+0000 7f2caaa00640 1 --2- 172.21.15.86:0/2027352651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cb40cfca0 0x7f2cb40cacd0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400b410 tx=0x7f2ca400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:06.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.675+0000 7f2cabe00640 1 -- 172.21.15.86:0/2027352651 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4015030 con 0x7f2cb40cfca0 2024-04-25T14:07:06.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.675+0000 7f2cabe00640 1 -- 172.21.15.86:0/2027352651 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ca4004020 con 0x7f2cb40cfca0 2024-04-25T14:07:06.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.675+0000 7f2cba200640 1 -- 172.21.15.86:0/2027352651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb40ae8a0 con 0x7f2cb40cfca0 2024-04-25T14:07:06.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.675+0000 7f2cabe00640 1 -- 172.21.15.86:0/2027352651 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4004410 con 0x7f2cb40cfca0 2024-04-25T14:07:06.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.675+0000 7f2cba200640 1 -- 172.21.15.86:0/2027352651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb40aee60 con 0x7f2cb40cfca0 2024-04-25T14:07:06.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.675+0000 7f2cba200640 1 -- 172.21.15.86:0/2027352651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cb40ae330 con 0x7f2cb40cfca0 2024-04-25T14:07:06.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.675+0000 7f2cabe00640 1 -- 172.21.15.86:0/2027352651 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 21) ==== 93262+0+0 (secure 0 0 0) 0x7f2ca4018070 con 0x7f2cb40cfca0 2024-04-25T14:07:06.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.679+0000 7f2cabe00640 1 --2- 172.21.15.86:0/2027352651 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f2c9006f4e0 0x7f2c900719f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:06.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.679+0000 7f2cabe00640 1 -- 172.21.15.86:0/2027352651 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f2ca408f320 con 0x7f2cb40cfca0 2024-04-25T14:07:06.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.679+0000 7f2cb3400640 1 --2- 172.21.15.86:0/2027352651 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f2c9006f4e0 0x7f2c900719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:06.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.679+0000 7f2cb3400640 1 --2- 172.21.15.86:0/2027352651 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f2c9006f4e0 0x7f2c900719f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c006fd0 tx=0x7f2c9c008040 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:06.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.683+0000 7f2cabe00640 1 -- 172.21.15.86:0/2027352651 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2ca405bde0 con 0x7f2cb40cfca0 2024-04-25T14:07:06.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.859+0000 7f2cba200640 1 -- 172.21.15.86:0/2027352651 --> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f2cb409f4b0 con 0x7f2c9006f4e0 2024-04-25T14:07:06.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.863+0000 7f2cabe00640 1 -- 172.21.15.86:0/2027352651 <== mgr.6801 v2:172.21.15.86:6832/203029814 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f2cb409f4b0 con 0x7f2c9006f4e0 2024-04-25T14:07:06.863 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {}, "global_metrics": {}, "metrics": {"delayed_ranks": []}} 2024-04-25T14:07:06.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.863+0000 7f2cba200640 1 -- 172.21.15.86:0/2027352651 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f2c9006f4e0 msgr2=0x7f2c900719f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:06.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.863+0000 7f2cba200640 1 --2- 172.21.15.86:0/2027352651 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f2c9006f4e0 0x7f2c900719f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c006fd0 tx=0x7f2c9c008040 comp rx=0 tx=0).stop 2024-04-25T14:07:06.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.863+0000 7f2cba200640 1 -- 172.21.15.86:0/2027352651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cb40cfca0 msgr2=0x7f2cb40cacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:06.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.863+0000 7f2cba200640 1 --2- 172.21.15.86:0/2027352651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cb40cfca0 0x7f2cb40cacd0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400b410 tx=0x7f2ca400b8d0 comp rx=0 tx=0).stop 2024-04-25T14:07:06.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.863+0000 7f2cba200640 1 -- 172.21.15.86:0/2027352651 shutdown_connections 2024-04-25T14:07:06.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.863+0000 7f2cba200640 1 --2- 172.21.15.86:0/2027352651 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f2c9006f4e0 0x7f2c900719f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:06.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.863+0000 7f2cba200640 1 --2- 172.21.15.86:0/2027352651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2cb40d16e0 0x7f2cb4158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:06.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.863+0000 7f2cba200640 1 --2- 172.21.15.86:0/2027352651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cb40cfca0 0x7f2cb40cacd0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:06.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.863+0000 7f2cba200640 1 --2- 172.21.15.86:0/2027352651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb40730b0 0x7f2cb409a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:06.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.863+0000 7f2cba200640 1 -- 172.21.15.86:0/2027352651 >> 172.21.15.86:0/2027352651 conn(0x7f2cb40b4650 msgr2=0x7f2cb40cde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:06.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.863+0000 7f2cba200640 1 -- 172.21.15.86:0/2027352651 shutdown_connections 2024-04-25T14:07:06.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:06.863+0000 7f2cba200640 1 -- 172.21.15.86:0/2027352651 wait complete. 2024-04-25T14:07:07.885 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:07:08.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.043+0000 7f648c200640 1 Processor -- start 2024-04-25T14:07:08.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.043+0000 7f648c200640 1 -- start start 2024-04-25T14:07:08.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.043+0000 7f648c200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f64840d2420 0x7f64840d2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:08.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.043+0000 7f648c200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6484070e20 0x7f6484071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:08.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.043+0000 7f648c200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6484071770 con 0x7f64840b6490 2024-04-25T14:07:08.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.043+0000 7f648c200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f64840d16f0 con 0x7f6484070e20 2024-04-25T14:07:08.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.043+0000 7f648c200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f6484162480 con 0x7f64840d2420 2024-04-25T14:07:08.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.043+0000 7f6488e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6484070e20 0x7f6484071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:08.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.043+0000 7f6488e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6484070e20 0x7f6484071230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47844/0 (socket says 172.21.15.86:47844) 2024-04-25T14:07:08.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.043+0000 7f6488e00640 1 -- 172.21.15.86:0/454220471 learned_addr learned my addr 172.21.15.86:0/454220471 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:08.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.043+0000 7f6489800640 1 --2- 172.21.15.86:0/454220471 >> v2:172.21.15.155:3300/0 conn(0x7f64840d2420 0x7f64840d2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:08.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.043+0000 7f6488e00640 1 -- 172.21.15.86:0/454220471 >> v2:172.21.15.155:3300/0 conn(0x7f64840d2420 msgr2=0x7f64840d2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:08.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.043+0000 7f6488e00640 1 --2- 172.21.15.86:0/454220471 >> v2:172.21.15.155:3300/0 conn(0x7f64840d2420 0x7f64840d2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:08.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.043+0000 7f6488e00640 1 -- 172.21.15.86:0/454220471 >> v1:172.21.15.155:6789/0 conn(0x7f64840b6490 legacy=0x7f64840b68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:08.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.043+0000 7f6488e00640 1 -- 172.21.15.86:0/454220471 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64840d1940 con 0x7f6484070e20 2024-04-25T14:07:08.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.047+0000 7f6488e00640 1 --2- 172.21.15.86:0/454220471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6484070e20 0x7f6484071230 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f64840b6070 tx=0x7f647001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35bbccf6a25d9b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:08.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.047+0000 7f6483e00640 1 -- 172.21.15.86:0/454220471 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64700ab610 con 0x7f6484070e20 2024-04-25T14:07:08.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.047+0000 7f6483e00640 1 -- 172.21.15.86:0/454220471 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64700b0070 con 0x7f6484070e20 2024-04-25T14:07:08.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.047+0000 7f6483e00640 1 -- 172.21.15.86:0/454220471 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64700ab9d0 con 0x7f6484070e20 2024-04-25T14:07:08.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.047+0000 7f648c200640 1 -- 172.21.15.86:0/454220471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6484070e20 msgr2=0x7f6484071230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:08.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.047+0000 7f648c200640 1 --2- 172.21.15.86:0/454220471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6484070e20 0x7f6484071230 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f64840b6070 tx=0x7f647001c520 comp rx=0 tx=0).stop 2024-04-25T14:07:08.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.047+0000 7f648c200640 1 -- 172.21.15.86:0/454220471 shutdown_connections 2024-04-25T14:07:08.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.047+0000 7f648c200640 1 --2- 172.21.15.86:0/454220471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6484070e20 0x7f6484071230 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:08.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.047+0000 7f648c200640 1 --2- 172.21.15.86:0/454220471 >> v2:172.21.15.155:3300/0 conn(0x7f64840d2420 0x7f64840d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:08.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.047+0000 7f648c200640 1 -- 172.21.15.86:0/454220471 >> 172.21.15.86:0/454220471 conn(0x7f64840cffc0 msgr2=0x7f64840d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:08.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.051+0000 7f648c200640 1 -- 172.21.15.86:0/454220471 shutdown_connections 2024-04-25T14:07:08.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.051+0000 7f648c200640 1 -- 172.21.15.86:0/454220471 wait complete. 2024-04-25T14:07:08.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.051+0000 7f648c200640 1 Processor -- start 2024-04-25T14:07:08.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.051+0000 7f648c200640 1 -- start start 2024-04-25T14:07:08.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.051+0000 7f648c200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6484070e20 0x7f64840c9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:08.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.051+0000 7f648c200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64840b6490 0x7f64840ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:08.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.051+0000 7f648c200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64840d2420 0x7f64840cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:08.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.051+0000 7f648c200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6484161310 con 0x7f6484070e20 2024-04-25T14:07:08.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.051+0000 7f648c200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f64840cdf30 con 0x7f64840b6490 2024-04-25T14:07:08.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.051+0000 7f648c200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f648415ffb0 con 0x7f64840d2420 2024-04-25T14:07:08.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.055+0000 7f6489800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6484070e20 0x7f64840c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:08.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.055+0000 7f6489800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6484070e20 0x7f64840c9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47854/0 (socket says 172.21.15.86:47854) 2024-04-25T14:07:08.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.055+0000 7f6489800640 1 -- 172.21.15.86:0/3397167571 learned_addr learned my addr 172.21.15.86:0/3397167571 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:08.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.055+0000 7f6488e00640 1 --2- 172.21.15.86:0/3397167571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64840b6490 0x7f64840ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:08.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.055+0000 7f648a200640 1 --2- 172.21.15.86:0/3397167571 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64840d2420 0x7f64840cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:08.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.055+0000 7f6489800640 1 -- 172.21.15.86:0/3397167571 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64840d2420 msgr2=0x7f64840cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:08.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.055+0000 7f6489800640 1 --2- 172.21.15.86:0/3397167571 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64840d2420 0x7f64840cb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:08.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.055+0000 7f6489800640 1 -- 172.21.15.86:0/3397167571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64840b6490 msgr2=0x7f64840ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:08.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.055+0000 7f6489800640 1 --2- 172.21.15.86:0/3397167571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64840b6490 0x7f64840ca6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:08.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.055+0000 7f6489800640 1 -- 172.21.15.86:0/3397167571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648415d790 con 0x7f6484070e20 2024-04-25T14:07:08.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.055+0000 7f6488e00640 1 --2- 172.21.15.86:0/3397167571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64840b6490 0x7f64840ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:08.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.055+0000 7f6489800640 1 --2- 172.21.15.86:0/3397167571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6484070e20 0x7f64840c9240 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f647800d5f0 tx=0x7f647800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:08.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.055+0000 7f6481600640 1 -- 172.21.15.86:0/3397167571 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64780125f0 con 0x7f6484070e20 2024-04-25T14:07:08.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.055+0000 7f6481600640 1 -- 172.21.15.86:0/3397167571 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6478019070 con 0x7f6484070e20 2024-04-25T14:07:08.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.055+0000 7f6481600640 1 -- 172.21.15.86:0/3397167571 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64780129d0 con 0x7f6484070e20 2024-04-25T14:07:08.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.055+0000 7f648c200640 1 -- 172.21.15.86:0/3397167571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64840c9780 con 0x7f6484070e20 2024-04-25T14:07:08.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.055+0000 7f648c200640 1 -- 172.21.15.86:0/3397167571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f648415ce60 con 0x7f6484070e20 2024-04-25T14:07:08.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.055+0000 7f648c200640 1 -- 172.21.15.86:0/3397167571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f648415e9d0 con 0x7f6484070e20 2024-04-25T14:07:08.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.055+0000 7f6481600640 1 -- 172.21.15.86:0/3397167571 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7f6478012b70 con 0x7f6484070e20 2024-04-25T14:07:08.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.059+0000 7f6481600640 1 --2- 172.21.15.86:0/3397167571 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f646406f570 0x7f6464071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:08.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.059+0000 7f6481600640 1 -- 172.21.15.86:0/3397167571 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f647808fc50 con 0x7f6484070e20 2024-04-25T14:07:08.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.059+0000 7f6488e00640 1 --2- 172.21.15.86:0/3397167571 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f646406f570 0x7f6464071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:08.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.059+0000 7f6488e00640 1 --2- 172.21.15.86:0/3397167571 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f646406f570 0x7f6464071a80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f64700af000 tx=0x7f64700a7000 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:08.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.063+0000 7f6481600640 1 -- 172.21.15.86:0/3397167571 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f647805c770 con 0x7f6484070e20 2024-04-25T14:07:08.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.239+0000 7f648c200640 1 -- 172.21.15.86:0/3397167571 --> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f64840b9000 con 0x7f646406f570 2024-04-25T14:07:08.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.243+0000 7f6481600640 1 -- 172.21.15.86:0/3397167571 <== mgr.6801 v2:172.21.15.86:6832/203029814 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f64840b9000 con 0x7f646406f570 2024-04-25T14:07:08.244 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {}, "global_metrics": {}, "metrics": {"delayed_ranks": []}} 2024-04-25T14:07:08.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.243+0000 7f648c200640 1 -- 172.21.15.86:0/3397167571 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f646406f570 msgr2=0x7f6464071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:08.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.243+0000 7f648c200640 1 --2- 172.21.15.86:0/3397167571 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f646406f570 0x7f6464071a80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f64700af000 tx=0x7f64700a7000 comp rx=0 tx=0).stop 2024-04-25T14:07:08.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.243+0000 7f648c200640 1 -- 172.21.15.86:0/3397167571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6484070e20 msgr2=0x7f64840c9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:08.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.243+0000 7f648c200640 1 --2- 172.21.15.86:0/3397167571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6484070e20 0x7f64840c9240 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f647800d5f0 tx=0x7f647800dab0 comp rx=0 tx=0).stop 2024-04-25T14:07:08.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.243+0000 7f648c200640 1 -- 172.21.15.86:0/3397167571 shutdown_connections 2024-04-25T14:07:08.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.243+0000 7f648c200640 1 --2- 172.21.15.86:0/3397167571 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f646406f570 0x7f6464071a80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:08.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.243+0000 7f648c200640 1 --2- 172.21.15.86:0/3397167571 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64840d2420 0x7f64840cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:08.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.243+0000 7f648c200640 1 --2- 172.21.15.86:0/3397167571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64840b6490 0x7f64840ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:08.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.243+0000 7f648c200640 1 --2- 172.21.15.86:0/3397167571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6484070e20 0x7f64840c9240 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:08.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.243+0000 7f648c200640 1 -- 172.21.15.86:0/3397167571 >> 172.21.15.86:0/3397167571 conn(0x7f64840cffc0 msgr2=0x7f648415daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:08.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.243+0000 7f648c200640 1 -- 172.21.15.86:0/3397167571 shutdown_connections 2024-04-25T14:07:08.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:08.243+0000 7f648c200640 1 -- 172.21.15.86:0/3397167571 wait complete. 2024-04-25T14:07:09.267 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:07:09.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.415+0000 7f5e13a00640 1 Processor -- start 2024-04-25T14:07:09.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.415+0000 7f5e13a00640 1 -- start start 2024-04-25T14:07:09.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.419+0000 7f5e13a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5e0c0d1df0 0x7f5e0c0cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:09.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.419+0000 7f5e13a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5e0c0cdcc0 0x7f5e0c0ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:09.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.419+0000 7f5e13a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e0c15e5c0 con 0x7f5e0c070330 2024-04-25T14:07:09.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.419+0000 7f5e13a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5e0c06fd40 con 0x7f5e0c0d1df0 2024-04-25T14:07:09.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.419+0000 7f5e13a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f5e0c15e8a0 con 0x7f5e0c0cdcc0 2024-04-25T14:07:09.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.419+0000 7f5e03e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5e0c0d1df0 0x7f5e0c0cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:09.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.419+0000 7f5e03e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5e0c0d1df0 0x7f5e0c0cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37764/0 (socket says 172.21.15.86:37764) 2024-04-25T14:07:09.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.419+0000 7f5e03e00640 1 -- 172.21.15.86:0/3140125472 learned_addr learned my addr 172.21.15.86:0/3140125472 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:09.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.419+0000 7f5e11a00640 1 --2- 172.21.15.86:0/3140125472 >> v2:172.21.15.155:3300/0 conn(0x7f5e0c0cdcc0 0x7f5e0c0ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:09.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.419+0000 7f5e03e00640 1 -- 172.21.15.86:0/3140125472 >> v2:172.21.15.155:3300/0 conn(0x7f5e0c0cdcc0 msgr2=0x7f5e0c0ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:09.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.419+0000 7f5e03e00640 1 --2- 172.21.15.86:0/3140125472 >> v2:172.21.15.155:3300/0 conn(0x7f5e0c0cdcc0 0x7f5e0c0ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:09.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.419+0000 7f5e03e00640 1 -- 172.21.15.86:0/3140125472 >> v1:172.21.15.155:6789/0 conn(0x7f5e0c070330 legacy=0x7f5e0c0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:09.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.419+0000 7f5e03e00640 1 -- 172.21.15.86:0/3140125472 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0c15eb80 con 0x7f5e0c0d1df0 2024-04-25T14:07:09.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.419+0000 7f5e11a00640 1 --2- 172.21.15.86:0/3140125472 >> v2:172.21.15.155:3300/0 conn(0x7f5e0c0cdcc0 0x7f5e0c0ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:09.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.423+0000 7f5e03e00640 1 --2- 172.21.15.86:0/3140125472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e0c0d1df0 0x7f5e0c0cd780 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f5df4009870 tx=0x7f5df401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab81c65afc22d9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:09.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.423+0000 7f5e0be00640 1 -- 172.21.15.86:0/3140125472 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df40ab600 con 0x7f5e0c0d1df0 2024-04-25T14:07:09.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.423+0000 7f5e0be00640 1 -- 172.21.15.86:0/3140125472 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5df40b0070 con 0x7f5e0c0d1df0 2024-04-25T14:07:09.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.423+0000 7f5e0be00640 1 -- 172.21.15.86:0/3140125472 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df40ab9c0 con 0x7f5e0c0d1df0 2024-04-25T14:07:09.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.423+0000 7f5e13a00640 1 -- 172.21.15.86:0/3140125472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e0c0d1df0 msgr2=0x7f5e0c0cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:09.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.423+0000 7f5e13a00640 1 --2- 172.21.15.86:0/3140125472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e0c0d1df0 0x7f5e0c0cd780 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f5df4009870 tx=0x7f5df401c420 comp rx=0 tx=0).stop 2024-04-25T14:07:09.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.423+0000 7f5e13a00640 1 -- 172.21.15.86:0/3140125472 shutdown_connections 2024-04-25T14:07:09.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.423+0000 7f5e13a00640 1 --2- 172.21.15.86:0/3140125472 >> v2:172.21.15.155:3300/0 conn(0x7f5e0c0cdcc0 0x7f5e0c0ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:09.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.423+0000 7f5e13a00640 1 --2- 172.21.15.86:0/3140125472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e0c0d1df0 0x7f5e0c0cd780 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:09.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.423+0000 7f5e13a00640 1 -- 172.21.15.86:0/3140125472 >> 172.21.15.86:0/3140125472 conn(0x7f5e0c0b4650 msgr2=0x7f5e0c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:09.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.423+0000 7f5e13a00640 1 -- 172.21.15.86:0/3140125472 shutdown_connections 2024-04-25T14:07:09.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.423+0000 7f5e13a00640 1 -- 172.21.15.86:0/3140125472 wait complete. 2024-04-25T14:07:09.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.427+0000 7f5e13a00640 1 Processor -- start 2024-04-25T14:07:09.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.427+0000 7f5e13a00640 1 -- start start 2024-04-25T14:07:09.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.427+0000 7f5e13a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e0c070330 0x7f5e0c0af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:09.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.427+0000 7f5e13a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e0c0cdcc0 0x7f5e0c15a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:09.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.427+0000 7f5e13a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e0c073290 0x7f5e0c0c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:09.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.427+0000 7f5e13a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c15f4b0 con 0x7f5e0c0cdcc0 2024-04-25T14:07:09.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.427+0000 7f5e13a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c15f330 con 0x7f5e0c070330 2024-04-25T14:07:09.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.427+0000 7f5e13a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5e0c070740 con 0x7f5e0c073290 2024-04-25T14:07:09.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.427+0000 7f5e03e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e0c0cdcc0 0x7f5e0c15a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:09.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.427+0000 7f5e03e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e0c0cdcc0 0x7f5e0c15a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37778/0 (socket says 172.21.15.86:37778) 2024-04-25T14:07:09.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.427+0000 7f5e03e00640 1 -- 172.21.15.86:0/3632967334 learned_addr learned my addr 172.21.15.86:0/3632967334 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:09.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.427+0000 7f5e11a00640 1 --2- 172.21.15.86:0/3632967334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e0c073290 0x7f5e0c0c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.427+0000 7f5e11000640 1 --2- 172.21.15.86:0/3632967334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e0c070330 0x7f5e0c0af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.427+0000 7f5e03e00640 1 -- 172.21.15.86:0/3632967334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e0c073290 msgr2=0x7f5e0c0c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.427+0000 7f5e03e00640 1 --2- 172.21.15.86:0/3632967334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e0c073290 0x7f5e0c0c9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.427+0000 7f5e03e00640 1 -- 172.21.15.86:0/3632967334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e0c070330 msgr2=0x7f5e0c0af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.427+0000 7f5e03e00640 1 --2- 172.21.15.86:0/3632967334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e0c070330 0x7f5e0c0af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.427+0000 7f5e03e00640 1 -- 172.21.15.86:0/3632967334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0c0bf730 con 0x7f5e0c0cdcc0 2024-04-25T14:07:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.427+0000 7f5e03e00640 1 --2- 172.21.15.86:0/3632967334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e0c0cdcc0 0x7f5e0c15a140 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f5df401aad0 tx=0x7f5df40191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:09.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.427+0000 7f5e09600640 1 -- 172.21.15.86:0/3632967334 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df40b5810 con 0x7f5e0c0cdcc0 2024-04-25T14:07:09.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.427+0000 7f5e09600640 1 -- 172.21.15.86:0/3632967334 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5df40b59b0 con 0x7f5e0c0cdcc0 2024-04-25T14:07:09.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.427+0000 7f5e13a00640 1 -- 172.21.15.86:0/3632967334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e0c073a30 con 0x7f5e0c0cdcc0 2024-04-25T14:07:09.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.427+0000 7f5e09600640 1 -- 172.21.15.86:0/3632967334 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df40b5da0 con 0x7f5e0c0cdcc0 2024-04-25T14:07:09.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.427+0000 7f5e13a00640 1 -- 172.21.15.86:0/3632967334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e0c073ff0 con 0x7f5e0c0cdcc0 2024-04-25T14:07:09.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.431+0000 7f5e13a00640 1 -- 172.21.15.86:0/3632967334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e0c069d00 con 0x7f5e0c0cdcc0 2024-04-25T14:07:09.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.431+0000 7f5e09600640 1 -- 172.21.15.86:0/3632967334 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7f5df40bd020 con 0x7f5e0c0cdcc0 2024-04-25T14:07:09.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.431+0000 7f5e09600640 1 --2- 172.21.15.86:0/3632967334 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f5de806f5a0 0x7f5de8071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:09.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.431+0000 7f5e09600640 1 -- 172.21.15.86:0/3632967334 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f5df4129fd0 con 0x7f5e0c0cdcc0 2024-04-25T14:07:09.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.431+0000 7f5e11000640 1 --2- 172.21.15.86:0/3632967334 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f5de806f5a0 0x7f5de8071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:09.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.431+0000 7f5e11000640 1 --2- 172.21.15.86:0/3632967334 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f5de806f5a0 0x7f5de8071ab0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c072200 tx=0x7f5dfc008040 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:09.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.435+0000 7f5e09600640 1 -- 172.21.15.86:0/3632967334 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5df40160d0 con 0x7f5e0c0cdcc0 2024-04-25T14:07:09.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.619+0000 7f5e13a00640 1 -- 172.21.15.86:0/3632967334 --> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f5e0c0737c0 con 0x7f5de806f5a0 2024-04-25T14:07:09.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.619+0000 7f5e09600640 1 -- 172.21.15.86:0/3632967334 <== mgr.6801 v2:172.21.15.86:6832/203029814 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+2958 (secure 0 0 0) 0x7f5e0c0737c0 con 0x7f5de806f5a0 2024-04-25T14:07:09.622 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {"cephfs": {"client.5945": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:172.21.15.86"}, "client.5951": {"hostname": "smithi155", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:192.168.0.1"}, "client.6834": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}, "client.6837": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}}}, "global_metrics": {"cephfs": {"client.5945": [[3, 19], [0, 0], [0, 0], [0, 10862852], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 452625], [12323397351675, 24]], "client.5951": [[22, 0], [0, 0], [0, 0], [0, 6006828], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 1501707], [12428818674318, 4]], "client.6834": [[2, 0], [0, 0], [0, 0], [0, 510044], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 510043], [0, 1]], "client.6837": [[2, 0], [0, 0], [0, 0], [0, 761751], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 761751], [0, 1]]}}, "metrics": {"delayed_ranks": [], "mds.0": {"client.5945": [], "client.5951": [], "client.6834": [], "client.6837": []}}} 2024-04-25T14:07:09.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.623+0000 7f5e13a00640 1 -- 172.21.15.86:0/3632967334 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f5de806f5a0 msgr2=0x7f5de8071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:09.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.623+0000 7f5e13a00640 1 --2- 172.21.15.86:0/3632967334 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f5de806f5a0 0x7f5de8071ab0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c072200 tx=0x7f5dfc008040 comp rx=0 tx=0).stop 2024-04-25T14:07:09.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.623+0000 7f5e13a00640 1 -- 172.21.15.86:0/3632967334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e0c0cdcc0 msgr2=0x7f5e0c15a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.623+0000 7f5e13a00640 1 --2- 172.21.15.86:0/3632967334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e0c0cdcc0 0x7f5e0c15a140 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f5df401aad0 tx=0x7f5df40191c0 comp rx=0 tx=0).stop 2024-04-25T14:07:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.623+0000 7f5e13a00640 1 -- 172.21.15.86:0/3632967334 shutdown_connections 2024-04-25T14:07:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.623+0000 7f5e13a00640 1 --2- 172.21.15.86:0/3632967334 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f5de806f5a0 0x7f5de8071ab0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.623+0000 7f5e13a00640 1 --2- 172.21.15.86:0/3632967334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e0c073290 0x7f5e0c0c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.623+0000 7f5e13a00640 1 --2- 172.21.15.86:0/3632967334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e0c0cdcc0 0x7f5e0c15a140 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.623+0000 7f5e13a00640 1 --2- 172.21.15.86:0/3632967334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e0c070330 0x7f5e0c0af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.623+0000 7f5e13a00640 1 -- 172.21.15.86:0/3632967334 >> 172.21.15.86:0/3632967334 conn(0x7f5e0c0b4650 msgr2=0x7f5e0c0b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.623+0000 7f5e13a00640 1 -- 172.21.15.86:0/3632967334 shutdown_connections 2024-04-25T14:07:09.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.623+0000 7f5e13a00640 1 -- 172.21.15.86:0/3632967334 wait complete. 2024-04-25T14:07:09.644 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:07:09.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.791+0000 7efde2400640 1 Processor -- start 2024-04-25T14:07:09.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.791+0000 7efde2400640 1 -- start start 2024-04-25T14:07:09.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.791+0000 7efde2400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efddc0cee30 0x7efddc06d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:09.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.791+0000 7efde2400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7efddc161c10 0x7efddc168640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:09.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.791+0000 7efde2400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efddc06dfd0 con 0x7efddc06dbc0 2024-04-25T14:07:09.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.791+0000 7efde2400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7efddc15ee70 con 0x7efddc0cee30 2024-04-25T14:07:09.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.791+0000 7efde2400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7efddc0cf680 con 0x7efddc161c10 2024-04-25T14:07:09.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.791+0000 7efde1000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efddc0cee30 0x7efddc06d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:09.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.791+0000 7efde1000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efddc0cee30 0x7efddc06d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37786/0 (socket says 172.21.15.86:37786) 2024-04-25T14:07:09.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.791+0000 7efde1000640 1 -- 172.21.15.86:0/3777549960 learned_addr learned my addr 172.21.15.86:0/3777549960 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:09.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.791+0000 7efddbe00640 1 --1- 172.21.15.86:0/3777549960 >> v1:172.21.15.86:6789/0 conn(0x7efddc06dbc0 0x7efddc161500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40370/0 (socket says 172.21.15.86:40370) 2024-04-25T14:07:09.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.791+0000 7efde1a00640 1 --2- 172.21.15.86:0/3777549960 >> v2:172.21.15.155:3300/0 conn(0x7efddc161c10 0x7efddc168640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:09.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.791+0000 7efde1000640 1 -- 172.21.15.86:0/3777549960 >> v2:172.21.15.155:3300/0 conn(0x7efddc161c10 msgr2=0x7efddc168640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:09.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.791+0000 7efde1000640 1 --2- 172.21.15.86:0/3777549960 >> v2:172.21.15.155:3300/0 conn(0x7efddc161c10 0x7efddc168640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:09.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.791+0000 7efde1000640 1 -- 172.21.15.86:0/3777549960 >> v1:172.21.15.86:6789/0 conn(0x7efddc06dbc0 legacy=0x7efddc161500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:09.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.791+0000 7efde1000640 1 -- 172.21.15.86:0/3777549960 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efddc15e0b0 con 0x7efddc0cee30 2024-04-25T14:07:09.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.791+0000 7efde1a00640 1 --2- 172.21.15.86:0/3777549960 >> v2:172.21.15.155:3300/0 conn(0x7efddc161c10 0x7efddc168640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:09.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.795+0000 7efde1000640 1 --2- 172.21.15.86:0/3777549960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efddc0cee30 0x7efddc06d680 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7efddc15bae0 tx=0x7efdd001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d80a8947d756a66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:09.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.795+0000 7efddb400640 1 -- 172.21.15.86:0/3777549960 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd00ab610 con 0x7efddc0cee30 2024-04-25T14:07:09.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.795+0000 7efddb400640 1 -- 172.21.15.86:0/3777549960 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd00b0070 con 0x7efddc0cee30 2024-04-25T14:07:09.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.795+0000 7efddb400640 1 -- 172.21.15.86:0/3777549960 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd00ab9d0 con 0x7efddc0cee30 2024-04-25T14:07:09.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.795+0000 7efde2400640 1 -- 172.21.15.86:0/3777549960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efddc0cee30 msgr2=0x7efddc06d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:09.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.795+0000 7efde2400640 1 --2- 172.21.15.86:0/3777549960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efddc0cee30 0x7efddc06d680 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7efddc15bae0 tx=0x7efdd001c520 comp rx=0 tx=0).stop 2024-04-25T14:07:09.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.795+0000 7efde2400640 1 -- 172.21.15.86:0/3777549960 shutdown_connections 2024-04-25T14:07:09.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.795+0000 7efde2400640 1 --2- 172.21.15.86:0/3777549960 >> v2:172.21.15.155:3300/0 conn(0x7efddc161c10 0x7efddc168640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:09.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.795+0000 7efde2400640 1 --2- 172.21.15.86:0/3777549960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efddc0cee30 0x7efddc06d680 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:09.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.795+0000 7efde2400640 1 -- 172.21.15.86:0/3777549960 >> 172.21.15.86:0/3777549960 conn(0x7efddc0730f0 msgr2=0x7efddc073500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:09.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.795+0000 7efde2400640 1 -- 172.21.15.86:0/3777549960 shutdown_connections 2024-04-25T14:07:09.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.795+0000 7efde2400640 1 -- 172.21.15.86:0/3777549960 wait complete. 2024-04-25T14:07:09.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.799+0000 7efde2400640 1 Processor -- start 2024-04-25T14:07:09.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.799+0000 7efde2400640 1 -- start start 2024-04-25T14:07:09.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.799+0000 7efde2400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efddc06dbc0 0x7efddc140690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:09.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.799+0000 7efde2400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efddc0cee30 0x7efddc163e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:09.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.799+0000 7efde2400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efddc161c10 0x7efddc1643d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:09.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.799+0000 7efde2400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efddc0b5b00 con 0x7efddc0cee30 2024-04-25T14:07:09.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.799+0000 7efde2400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efddc0b69a0 con 0x7efddc161c10 2024-04-25T14:07:09.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.799+0000 7efde2400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efddc0b5c80 con 0x7efddc06dbc0 2024-04-25T14:07:09.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.799+0000 7efddbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efddc0cee30 0x7efddc163e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:09.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.799+0000 7efddbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efddc0cee30 0x7efddc163e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37800/0 (socket says 172.21.15.86:37800) 2024-04-25T14:07:09.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.799+0000 7efddbe00640 1 -- 172.21.15.86:0/3411905490 learned_addr learned my addr 172.21.15.86:0/3411905490 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:09.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.799+0000 7efde1a00640 1 --2- 172.21.15.86:0/3411905490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efddc161c10 0x7efddc1643d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:09.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.799+0000 7efde1000640 1 --2- 172.21.15.86:0/3411905490 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efddc06dbc0 0x7efddc140690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:09.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.799+0000 7efddbe00640 1 -- 172.21.15.86:0/3411905490 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efddc06dbc0 msgr2=0x7efddc140690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.799+0000 7efddbe00640 1 --2- 172.21.15.86:0/3411905490 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efddc06dbc0 0x7efddc140690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.799+0000 7efddbe00640 1 -- 172.21.15.86:0/3411905490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efddc161c10 msgr2=0x7efddc1643d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.799+0000 7efddbe00640 1 --2- 172.21.15.86:0/3411905490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efddc161c10 0x7efddc1643d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.799+0000 7efddbe00640 1 -- 172.21.15.86:0/3411905490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efddc1633d0 con 0x7efddc0cee30 2024-04-25T14:07:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.803+0000 7efddbe00640 1 --2- 172.21.15.86:0/3411905490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efddc0cee30 0x7efddc163e90 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7efdc800c3a0 tx=0x7efdc800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.803+0000 7efdd8c00640 1 -- 172.21.15.86:0/3411905490 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc80074e0 con 0x7efddc0cee30 2024-04-25T14:07:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.803+0000 7efdd8c00640 1 -- 172.21.15.86:0/3411905490 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdc8007680 con 0x7efddc0cee30 2024-04-25T14:07:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.803+0000 7efdd8c00640 1 -- 172.21.15.86:0/3411905490 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc8004a30 con 0x7efddc0cee30 2024-04-25T14:07:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.803+0000 7efde2400640 1 -- 172.21.15.86:0/3411905490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efddc157730 con 0x7efddc0cee30 2024-04-25T14:07:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.803+0000 7efde2400640 1 -- 172.21.15.86:0/3411905490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efddc164f10 con 0x7efddc0cee30 2024-04-25T14:07:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.803+0000 7efde2400640 1 -- 172.21.15.86:0/3411905490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efddc076b80 con 0x7efddc0cee30 2024-04-25T14:07:09.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.803+0000 7efdd8c00640 1 -- 172.21.15.86:0/3411905490 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7efdc801a020 con 0x7efddc0cee30 2024-04-25T14:07:09.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.803+0000 7efdd8c00640 1 --2- 172.21.15.86:0/3411905490 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7efdbc06f570 0x7efdbc071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:09.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.803+0000 7efdd8c00640 1 -- 172.21.15.86:0/3411905490 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7efdc808ef20 con 0x7efddc0cee30 2024-04-25T14:07:09.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.807+0000 7efde1000640 1 --2- 172.21.15.86:0/3411905490 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7efdbc06f570 0x7efdbc071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:09.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.807+0000 7efde1000640 1 --2- 172.21.15.86:0/3411905490 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7efdbc06f570 0x7efdbc071a80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efdd00af000 tx=0x7efdd00a7000 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:09.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.811+0000 7efdd8c00640 1 -- 172.21.15.86:0/3411905490 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efddc06dfd0 con 0x7efddc0cee30 2024-04-25T14:07:10.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.999+0000 7efde2400640 1 -- 172.21.15.86:0/3411905490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efddc161640 con 0x7efddc0cee30 2024-04-25T14:07:10.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:09.999+0000 7efdd8c00640 1 -- 172.21.15.86:0/3411905490 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5189 (secure 0 0 0) 0x7efdc805ba50 con 0x7efddc0cee30 2024-04-25T14:07:10.001 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:07:10.001 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1210051860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1210051860},{"type":"v1","addr":"172.21.15.155:6833","nonce":1210051860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2868300438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2868300438},{"type":"v1","addr":"172.21.15.155:6835","nonce":2868300438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6600,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2682115891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2682115891},{"type":"v1","addr":"172.21.15.86:6835","nonce":2682115891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:06:44.910717+0000","modified":"2024-04-25T14:06:47.953301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6603},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6603":{"gid":6603,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/1088108173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1088108173},{"type":"v1","addr":"172.21.15.86:6837","nonce":1088108173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6603,"qdb_cluster":[6603]},"id":5}]} 2024-04-25T14:07:10.002 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 53 2024-04-25T14:07:10.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.003+0000 7efde2400640 1 -- 172.21.15.86:0/3411905490 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7efdbc06f570 msgr2=0x7efdbc071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:10.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.003+0000 7efde2400640 1 --2- 172.21.15.86:0/3411905490 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7efdbc06f570 0x7efdbc071a80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efdd00af000 tx=0x7efdd00a7000 comp rx=0 tx=0).stop 2024-04-25T14:07:10.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.003+0000 7efde2400640 1 -- 172.21.15.86:0/3411905490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efddc0cee30 msgr2=0x7efddc163e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:10.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.003+0000 7efde2400640 1 --2- 172.21.15.86:0/3411905490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efddc0cee30 0x7efddc163e90 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7efdc800c3a0 tx=0x7efdc800c860 comp rx=0 tx=0).stop 2024-04-25T14:07:10.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.003+0000 7efde2400640 1 -- 172.21.15.86:0/3411905490 shutdown_connections 2024-04-25T14:07:10.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.003+0000 7efde2400640 1 --2- 172.21.15.86:0/3411905490 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7efdbc06f570 0x7efdbc071a80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:10.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.003+0000 7efde2400640 1 --2- 172.21.15.86:0/3411905490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efddc161c10 0x7efddc1643d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:10.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.003+0000 7efde2400640 1 --2- 172.21.15.86:0/3411905490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efddc0cee30 0x7efddc163e90 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:10.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.003+0000 7efde2400640 1 --2- 172.21.15.86:0/3411905490 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efddc06dbc0 0x7efddc140690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:10.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.003+0000 7efde2400640 1 -- 172.21.15.86:0/3411905490 >> 172.21.15.86:0/3411905490 conn(0x7efddc0730f0 msgr2=0x7efddc0b1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:10.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.003+0000 7efde2400640 1 -- 172.21.15.86:0/3411905490 shutdown_connections 2024-04-25T14:07:10.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.003+0000 7efde2400640 1 -- 172.21.15.86:0/3411905490 wait complete. 2024-04-25T14:07:10.023 DEBUG:tasks.cephfs.test_mds_metrics:metrics={'version': 2, 'global_counters': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'counters': [], 'client_metadata': {'cephfs': {'client.5945': {'hostname': 'smithi086', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'kernel_version': '6.9.0-rc3-g806ec36b2112', 'IP': 'v1:172.21.15.86'}, 'client.5951': {'hostname': 'smithi155', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'kernel_version': '6.9.0-rc3-g806ec36b2112', 'IP': 'v1:192.168.0.1'}, 'client.6834': {'hostname': 'smithi086', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'IP': '172.21.15.86'}, 'client.6837': {'hostname': 'smithi086', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'IP': '172.21.15.86'}}}, 'global_metrics': {'cephfs': {'client.5945': [[3, 19], [0, 0], [0, 0], [0, 10862852], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 452625], [12323397351675, 24]], 'client.5951': [[22, 0], [0, 0], [0, 0], [0, 6006828], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 1501707], [12428818674318, 4]], 'client.6834': [[2, 0], [0, 0], [0, 0], [0, 510044], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 510043], [0, 1]], 'client.6837': [[2, 0], [0, 0], [0, 0], [0, 761751], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 761751], [0, 1]]}}, 'metrics': {'delayed_ranks': [], 'mds.0': {'client.5945': [], 'client.5951': [], 'client.6834': [], 'client.6837': []}}} 2024-04-25T14:07:10.023 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:07:10.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.167+0000 7f0b3c800640 1 Processor -- start 2024-04-25T14:07:10.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.171+0000 7f0b3c800640 1 -- start start 2024-04-25T14:07:10.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.171+0000 7f0b3c800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b3415c2a0 0x7f0b340ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:10.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.171+0000 7f0b3c800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0b3406fd50 0x7f0b34070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:10.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.171+0000 7f0b3c800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b340b0a60 con 0x7f0b3406c670 2024-04-25T14:07:10.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.171+0000 7f0b3c800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0b3406ef90 con 0x7f0b3406fd50 2024-04-25T14:07:10.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.171+0000 7f0b3c800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0b3406f270 con 0x7f0b3415c2a0 2024-04-25T14:07:10.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.171+0000 7f0b39400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0b3406fd50 0x7f0b34070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:10.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.171+0000 7f0b39400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0b3406fd50 0x7f0b34070160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37810/0 (socket says 172.21.15.86:37810) 2024-04-25T14:07:10.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.171+0000 7f0b39400640 1 -- 172.21.15.86:0/2987133247 learned_addr learned my addr 172.21.15.86:0/2987133247 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:10.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.171+0000 7f0b3a800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0b3406c670 0x7f0b3406ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40372/0 (socket says 172.21.15.86:40372) 2024-04-25T14:07:10.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.171+0000 7f0b39e00640 1 --2- 172.21.15.86:0/2987133247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b3415c2a0 0x7f0b340ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:10.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.171+0000 7f0b39400640 1 -- 172.21.15.86:0/2987133247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b3415c2a0 msgr2=0x7f0b340ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:10.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.171+0000 7f0b39400640 1 --2- 172.21.15.86:0/2987133247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b3415c2a0 0x7f0b340ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:10.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.171+0000 7f0b39400640 1 -- 172.21.15.86:0/2987133247 >> v1:172.21.15.86:6789/0 conn(0x7f0b3406c670 legacy=0x7f0b3406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:10.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.171+0000 7f0b39400640 1 -- 172.21.15.86:0/2987133247 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b3406f550 con 0x7f0b3406fd50 2024-04-25T14:07:10.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.175+0000 7f0b39400640 1 --2- 172.21.15.86:0/2987133247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b3406fd50 0x7f0b34070160 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f0b24009870 tx=0x7f0b2401d5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67005cf075aba0eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:10.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.175+0000 7f0b38a00640 1 -- 172.21.15.86:0/2987133247 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b240ab660 con 0x7f0b3406fd50 2024-04-25T14:07:10.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.175+0000 7f0b38a00640 1 -- 172.21.15.86:0/2987133247 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b240b0070 con 0x7f0b3406fd50 2024-04-25T14:07:10.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.175+0000 7f0b38a00640 1 -- 172.21.15.86:0/2987133247 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b240aba20 con 0x7f0b3406fd50 2024-04-25T14:07:10.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.175+0000 7f0b3c800640 1 -- 172.21.15.86:0/2987133247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b3406fd50 msgr2=0x7f0b34070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:10.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.175+0000 7f0b3c800640 1 --2- 172.21.15.86:0/2987133247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b3406fd50 0x7f0b34070160 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f0b24009870 tx=0x7f0b2401d5b0 comp rx=0 tx=0).stop 2024-04-25T14:07:10.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.175+0000 7f0b3c800640 1 -- 172.21.15.86:0/2987133247 shutdown_connections 2024-04-25T14:07:10.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.175+0000 7f0b3c800640 1 --2- 172.21.15.86:0/2987133247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b3406fd50 0x7f0b34070160 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:10.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.175+0000 7f0b3c800640 1 --2- 172.21.15.86:0/2987133247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b3415c2a0 0x7f0b340ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:10.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.175+0000 7f0b3c800640 1 -- 172.21.15.86:0/2987133247 >> 172.21.15.86:0/2987133247 conn(0x7f0b340b4650 msgr2=0x7f0b340b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:10.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.175+0000 7f0b3c800640 1 -- 172.21.15.86:0/2987133247 shutdown_connections 2024-04-25T14:07:10.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.175+0000 7f0b3c800640 1 -- 172.21.15.86:0/2987133247 wait complete. 2024-04-25T14:07:10.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.179+0000 7f0b3c800640 1 Processor -- start 2024-04-25T14:07:10.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.179+0000 7f0b3c800640 1 -- start start 2024-04-25T14:07:10.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.179+0000 7f0b3c800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b3406c670 0x7f0b341f8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:10.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.179+0000 7f0b3c800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b3406fd50 0x7f0b340c3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:10.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.179+0000 7f0b3c800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b3415c2a0 0x7f0b3409bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:10.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.179+0000 7f0b3c800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3415c830 con 0x7f0b3406c670 2024-04-25T14:07:10.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.179+0000 7f0b3c800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3415c6b0 con 0x7f0b3415c2a0 2024-04-25T14:07:10.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.179+0000 7f0b3c800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0b340ae670 con 0x7f0b3406fd50 2024-04-25T14:07:10.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.179+0000 7f0b39e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b3406c670 0x7f0b341f8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:10.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.179+0000 7f0b39e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b3406c670 0x7f0b341f8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37812/0 (socket says 172.21.15.86:37812) 2024-04-25T14:07:10.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.179+0000 7f0b39e00640 1 -- 172.21.15.86:0/584362099 learned_addr learned my addr 172.21.15.86:0/584362099 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:10.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.179+0000 7f0b39e00640 1 -- 172.21.15.86:0/584362099 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b3406fd50 msgr2=0x7f0b340c3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:10.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.179+0000 7f0b39e00640 1 --2- 172.21.15.86:0/584362099 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b3406fd50 0x7f0b340c3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:10.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.179+0000 7f0b39e00640 1 -- 172.21.15.86:0/584362099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b3415c2a0 msgr2=0x7f0b3409bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:10.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.179+0000 7f0b3a800640 1 --2- 172.21.15.86:0/584362099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b3415c2a0 0x7f0b3409bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:10.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.179+0000 7f0b39e00640 1 --2- 172.21.15.86:0/584362099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b3415c2a0 0x7f0b3409bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:10.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.179+0000 7f0b39e00640 1 -- 172.21.15.86:0/584362099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b340b1a00 con 0x7f0b3406c670 2024-04-25T14:07:10.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.179+0000 7f0b3a800640 1 --2- 172.21.15.86:0/584362099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b3415c2a0 0x7f0b3409bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:10.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.179+0000 7f0b39e00640 1 --2- 172.21.15.86:0/584362099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b3406c670 0x7f0b341f8010 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800d5f0 tx=0x7f0b2800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:10.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.179+0000 7f0b2e000640 1 -- 172.21.15.86:0/584362099 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28004240 con 0x7f0b3406c670 2024-04-25T14:07:10.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.179+0000 7f0b2e000640 1 -- 172.21.15.86:0/584362099 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b280043e0 con 0x7f0b3406c670 2024-04-25T14:07:10.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.179+0000 7f0b2e000640 1 -- 172.21.15.86:0/584362099 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28003840 con 0x7f0b3406c670 2024-04-25T14:07:10.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.179+0000 7f0b3c800640 1 -- 172.21.15.86:0/584362099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b3409b3e0 con 0x7f0b3406c670 2024-04-25T14:07:10.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.179+0000 7f0b3c800640 1 -- 172.21.15.86:0/584362099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b341f8840 con 0x7f0b3406c670 2024-04-25T14:07:10.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.183+0000 7f0b3c800640 1 -- 172.21.15.86:0/584362099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b34069d00 con 0x7f0b3406c670 2024-04-25T14:07:10.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.183+0000 7f0b2e000640 1 -- 172.21.15.86:0/584362099 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7f0b280039e0 con 0x7f0b3406c670 2024-04-25T14:07:10.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.183+0000 7f0b2e000640 1 --2- 172.21.15.86:0/584362099 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f0b1806f5a0 0x7f0b18071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:10.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.183+0000 7f0b2e000640 1 -- 172.21.15.86:0/584362099 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f0b2808fa20 con 0x7f0b3406c670 2024-04-25T14:07:10.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.183+0000 7f0b39400640 1 --2- 172.21.15.86:0/584362099 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f0b1806f5a0 0x7f0b18071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:10.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.183+0000 7f0b39400640 1 --2- 172.21.15.86:0/584362099 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f0b1806f5a0 0x7f0b18071ab0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0b240af000 tx=0x7f0b24004740 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:10.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.187+0000 7f0b2e000640 1 -- 172.21.15.86:0/584362099 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0b2805c4b0 con 0x7f0b3406c670 2024-04-25T14:07:10.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.375+0000 7f0b3c800640 1 -- 172.21.15.86:0/584362099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b34076bf0 con 0x7f0b3406c670 2024-04-25T14:07:10.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.375+0000 7f0b2e000640 1 -- 172.21.15.86:0/584362099 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5189 (secure 0 0 0) 0x7f0b2805bdb0 con 0x7f0b3406c670 2024-04-25T14:07:10.379 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:07:10.379 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1210051860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1210051860},{"type":"v1","addr":"172.21.15.155:6833","nonce":1210051860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2868300438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2868300438},{"type":"v1","addr":"172.21.15.155:6835","nonce":2868300438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6600,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2682115891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2682115891},{"type":"v1","addr":"172.21.15.86:6835","nonce":2682115891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:06:44.910717+0000","modified":"2024-04-25T14:06:47.953301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6603},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6603":{"gid":6603,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/1088108173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1088108173},{"type":"v1","addr":"172.21.15.86:6837","nonce":1088108173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6603,"qdb_cluster":[6603]},"id":5}]} 2024-04-25T14:07:10.379 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 53 2024-04-25T14:07:10.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.379+0000 7f0b3c800640 1 -- 172.21.15.86:0/584362099 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f0b1806f5a0 msgr2=0x7f0b18071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:10.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.379+0000 7f0b3c800640 1 --2- 172.21.15.86:0/584362099 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f0b1806f5a0 0x7f0b18071ab0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0b240af000 tx=0x7f0b24004740 comp rx=0 tx=0).stop 2024-04-25T14:07:10.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.379+0000 7f0b3c800640 1 -- 172.21.15.86:0/584362099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b3406c670 msgr2=0x7f0b341f8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:10.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.379+0000 7f0b3c800640 1 --2- 172.21.15.86:0/584362099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b3406c670 0x7f0b341f8010 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800d5f0 tx=0x7f0b2800dab0 comp rx=0 tx=0).stop 2024-04-25T14:07:10.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.379+0000 7f0b3c800640 1 -- 172.21.15.86:0/584362099 shutdown_connections 2024-04-25T14:07:10.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.379+0000 7f0b3c800640 1 --2- 172.21.15.86:0/584362099 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f0b1806f5a0 0x7f0b18071ab0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:10.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.379+0000 7f0b3c800640 1 --2- 172.21.15.86:0/584362099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b3415c2a0 0x7f0b3409bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:10.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.379+0000 7f0b3c800640 1 --2- 172.21.15.86:0/584362099 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b3406fd50 0x7f0b340c3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:10.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.379+0000 7f0b3c800640 1 --2- 172.21.15.86:0/584362099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b3406c670 0x7f0b341f8010 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:10.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.379+0000 7f0b3c800640 1 -- 172.21.15.86:0/584362099 >> 172.21.15.86:0/584362099 conn(0x7f0b340b4650 msgr2=0x7f0b3415b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:10.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.379+0000 7f0b3c800640 1 -- 172.21.15.86:0/584362099 shutdown_connections 2024-04-25T14:07:10.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.379+0000 7f0b3c800640 1 -- 172.21.15.86:0/584362099 wait complete. 2024-04-25T14:07:10.401 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2024-04-25T14:07:10.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.551+0000 7ff348000640 1 Processor -- start 2024-04-25T14:07:10.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.551+0000 7ff348000640 1 -- start start 2024-04-25T14:07:10.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.551+0000 7ff348000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff34015ec70 0x7ff3400d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:10.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.551+0000 7ff348000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff3400d1490 0x7ff34006d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:10.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.551+0000 7ff348000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3400cf8f0 con 0x7ff34015e860 2024-04-25T14:07:10.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.551+0000 7ff348000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff34006cfa0 con 0x7ff3400d1490 2024-04-25T14:07:10.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.551+0000 7ff348000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff34006c670 con 0x7ff34015ec70 2024-04-25T14:07:10.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.551+0000 7ff344c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff34015ec70 0x7ff3400d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:10.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.551+0000 7ff346000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff3400d1490 0x7ff34006d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:10.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.551+0000 7ff345600640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7ff34015e860 0x7ff3400ceea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:34792/0 (socket says 172.21.15.86:34792) 2024-04-25T14:07:10.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.551+0000 7ff345600640 1 -- 172.21.15.86:0/3746083549 learned_addr learned my addr 172.21.15.86:0/3746083549 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:07:10.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.551+0000 7ff344c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff34015ec70 0x7ff3400d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:54788/0 (socket says 172.21.15.86:54788) 2024-04-25T14:07:10.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.551+0000 7ff344c00640 1 -- 172.21.15.86:0/3746083549 >> v2:172.21.15.155:3300/0 conn(0x7ff3400d1490 msgr2=0x7ff34006d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:10.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.551+0000 7ff344c00640 1 --2- 172.21.15.86:0/3746083549 >> v2:172.21.15.155:3300/0 conn(0x7ff3400d1490 0x7ff34006d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:10.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.551+0000 7ff344c00640 1 -- 172.21.15.86:0/3746083549 >> v1:172.21.15.155:6789/0 conn(0x7ff34015e860 legacy=0x7ff3400ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:10.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.551+0000 7ff344c00640 1 -- 172.21.15.86:0/3746083549 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34006c950 con 0x7ff34015ec70 2024-04-25T14:07:10.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.555+0000 7ff344c00640 1 --2- 172.21.15.86:0/3746083549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff34015ec70 0x7ff3400d0f50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0097c0 tx=0x7ff32c01c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4607000c530d75e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:10.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.555+0000 7ff33fe00640 1 -- 172.21.15.86:0/3746083549 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c0b0070 con 0x7ff34015ec70 2024-04-25T14:07:10.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.555+0000 7ff33fe00640 1 -- 172.21.15.86:0/3746083549 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff32c0ab600 con 0x7ff34015ec70 2024-04-25T14:07:10.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.555+0000 7ff33fe00640 1 -- 172.21.15.86:0/3746083549 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c0ab9d0 con 0x7ff34015ec70 2024-04-25T14:07:10.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.555+0000 7ff348000640 1 -- 172.21.15.86:0/3746083549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff34015ec70 msgr2=0x7ff3400d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:10.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.555+0000 7ff348000640 1 --2- 172.21.15.86:0/3746083549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff34015ec70 0x7ff3400d0f50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0097c0 tx=0x7ff32c01c3e0 comp rx=0 tx=0).stop 2024-04-25T14:07:10.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.555+0000 7ff348000640 1 -- 172.21.15.86:0/3746083549 shutdown_connections 2024-04-25T14:07:10.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.555+0000 7ff348000640 1 --2- 172.21.15.86:0/3746083549 >> v2:172.21.15.155:3300/0 conn(0x7ff3400d1490 0x7ff34006d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:10.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.555+0000 7ff348000640 1 --2- 172.21.15.86:0/3746083549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff34015ec70 0x7ff3400d0f50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:10.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.555+0000 7ff348000640 1 -- 172.21.15.86:0/3746083549 >> 172.21.15.86:0/3746083549 conn(0x7ff3400d2230 msgr2=0x7ff3400d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:10.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.559+0000 7ff348000640 1 -- 172.21.15.86:0/3746083549 shutdown_connections 2024-04-25T14:07:10.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.559+0000 7ff348000640 1 -- 172.21.15.86:0/3746083549 wait complete. 2024-04-25T14:07:10.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.559+0000 7ff348000640 1 Processor -- start 2024-04-25T14:07:10.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.559+0000 7ff348000640 1 -- start start 2024-04-25T14:07:10.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.559+0000 7ff348000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3400d1490 0x7ff3400b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:10.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.559+0000 7ff348000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff34015e860 0x7ff340167710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:10.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.559+0000 7ff348000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff34015ec70 0x7ff340167c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:10.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.559+0000 7ff348000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff34016a720 con 0x7ff34015e860 2024-04-25T14:07:10.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.559+0000 7ff348000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff3400d18a0 con 0x7ff3400d1490 2024-04-25T14:07:10.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.559+0000 7ff348000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff34016a8a0 con 0x7ff34015ec70 2024-04-25T14:07:10.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.563+0000 7ff344c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff34015e860 0x7ff340167710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:10.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.563+0000 7ff344c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff34015e860 0x7ff340167710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37818/0 (socket says 172.21.15.86:37818) 2024-04-25T14:07:10.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.563+0000 7ff344c00640 1 -- 172.21.15.86:0/1503238549 learned_addr learned my addr 172.21.15.86:0/1503238549 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:10.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.563+0000 7ff345600640 1 --2- 172.21.15.86:0/1503238549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3400d1490 0x7ff3400b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:10.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.563+0000 7ff346000640 1 --2- 172.21.15.86:0/1503238549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff34015ec70 0x7ff340167c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:10.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.563+0000 7ff344c00640 1 -- 172.21.15.86:0/1503238549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff34015ec70 msgr2=0x7ff340167c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:10.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.563+0000 7ff344c00640 1 --2- 172.21.15.86:0/1503238549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff34015ec70 0x7ff340167c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:10.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.563+0000 7ff344c00640 1 -- 172.21.15.86:0/1503238549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3400d1490 msgr2=0x7ff3400b6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:10.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.563+0000 7ff344c00640 1 --2- 172.21.15.86:0/1503238549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3400d1490 0x7ff3400b6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:10.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.563+0000 7ff344c00640 1 -- 172.21.15.86:0/1503238549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff340159fe0 con 0x7ff34015e860 2024-04-25T14:07:10.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.563+0000 7ff344c00640 1 --2- 172.21.15.86:0/1503238549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff34015e860 0x7ff340167710 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0ae040 tx=0x7ff34006e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:10.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.563+0000 7ff33d600640 1 -- 172.21.15.86:0/1503238549 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c0ba070 con 0x7ff34015e860 2024-04-25T14:07:10.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.563+0000 7ff33d600640 1 -- 172.21.15.86:0/1503238549 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff32c0b43f0 con 0x7ff34015e860 2024-04-25T14:07:10.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.563+0000 7ff348000640 1 -- 172.21.15.86:0/1503238549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff34015a2c0 con 0x7ff34015e860 2024-04-25T14:07:10.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.563+0000 7ff33d600640 1 -- 172.21.15.86:0/1503238549 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c0b0040 con 0x7ff34015e860 2024-04-25T14:07:10.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.563+0000 7ff348000640 1 -- 172.21.15.86:0/1503238549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff34015d160 con 0x7ff34015e860 2024-04-25T14:07:10.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.563+0000 7ff348000640 1 -- 172.21.15.86:0/1503238549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff34015ced0 con 0x7ff34015e860 2024-04-25T14:07:10.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.563+0000 7ff33d600640 1 -- 172.21.15.86:0/1503238549 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7ff32c0bd070 con 0x7ff34015e860 2024-04-25T14:07:10.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.567+0000 7ff33d600640 1 --2- 172.21.15.86:0/1503238549 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7ff32006f5a0 0x7ff320071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:10.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.567+0000 7ff33d600640 1 -- 172.21.15.86:0/1503238549 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7ff32c128750 con 0x7ff34015e860 2024-04-25T14:07:10.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.567+0000 7ff345600640 1 --2- 172.21.15.86:0/1503238549 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7ff32006f5a0 0x7ff320071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:10.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.567+0000 7ff345600640 1 --2- 172.21.15.86:0/1503238549 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7ff32006f5a0 0x7ff320071ab0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff334000f70 tx=0x7ff334007e90 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:10.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.571+0000 7ff33d600640 1 -- 172.21.15.86:0/1503238549 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff32c016090 con 0x7ff34015e860 2024-04-25T14:07:10.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:10.759+0000 7ff348000640 1 -- 172.21.15.86:0/1503238549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) -- 0x7ff340074cf0 con 0x7ff34015e860 2024-04-25T14:07:11.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.127+0000 7ff33d600640 1 -- 172.21.15.86:0/1503238549 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v54) ==== 105+0+0 (secure 0 0 0) 0x7ff32c0f4a80 con 0x7ff34015e860 2024-04-25T14:07:11.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.131+0000 7ff348000640 1 -- 172.21.15.86:0/1503238549 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7ff32006f5a0 msgr2=0x7ff320071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:11.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.131+0000 7ff348000640 1 --2- 172.21.15.86:0/1503238549 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7ff32006f5a0 0x7ff320071ab0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff334000f70 tx=0x7ff334007e90 comp rx=0 tx=0).stop 2024-04-25T14:07:11.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.131+0000 7ff348000640 1 -- 172.21.15.86:0/1503238549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff34015e860 msgr2=0x7ff340167710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:11.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.131+0000 7ff348000640 1 --2- 172.21.15.86:0/1503238549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff34015e860 0x7ff340167710 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0ae040 tx=0x7ff34006e180 comp rx=0 tx=0).stop 2024-04-25T14:07:11.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.131+0000 7ff348000640 1 -- 172.21.15.86:0/1503238549 shutdown_connections 2024-04-25T14:07:11.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.131+0000 7ff348000640 1 --2- 172.21.15.86:0/1503238549 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7ff32006f5a0 0x7ff320071ab0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.131+0000 7ff348000640 1 --2- 172.21.15.86:0/1503238549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff34015ec70 0x7ff340167c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.131+0000 7ff348000640 1 --2- 172.21.15.86:0/1503238549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff34015e860 0x7ff340167710 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.131+0000 7ff348000640 1 --2- 172.21.15.86:0/1503238549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3400d1490 0x7ff3400b6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.131+0000 7ff348000640 1 -- 172.21.15.86:0/1503238549 >> 172.21.15.86:0/1503238549 conn(0x7ff3400d2230 msgr2=0x7ff34015fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:11.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.131+0000 7ff348000640 1 -- 172.21.15.86:0/1503238549 shutdown_connections 2024-04-25T14:07:11.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.131+0000 7ff348000640 1 -- 172.21.15.86:0/1503238549 wait complete. 2024-04-25T14:07:11.160 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:07:11.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.303+0000 7fa9df200640 1 Processor -- start 2024-04-25T14:07:11.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.303+0000 7fa9df200640 1 -- start start 2024-04-25T14:07:11.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.307+0000 7fa9df200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9d815c2a0 0x7fa9d80ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:11.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.307+0000 7fa9df200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa9d815feb0 0x7fa9d81602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:11.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.307+0000 7fa9df200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9d80b0a60 con 0x7fa9d806fd50 2024-04-25T14:07:11.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.307+0000 7fa9df200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa9d806ef90 con 0x7fa9d815feb0 2024-04-25T14:07:11.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.307+0000 7fa9df200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa9d806f270 con 0x7fa9d815c2a0 2024-04-25T14:07:11.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.307+0000 7fa9dd200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa9d815feb0 0x7fa9d81602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:11.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.307+0000 7fa9dd200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa9d815feb0 0x7fa9d81602c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37822/0 (socket says 172.21.15.86:37822) 2024-04-25T14:07:11.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.307+0000 7fa9dd200640 1 -- 172.21.15.86:0/2641094436 learned_addr learned my addr 172.21.15.86:0/2641094436 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:11.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.307+0000 7fa9d7e00640 1 --2- 172.21.15.86:0/2641094436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9d815c2a0 0x7fa9d80ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:11.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.307+0000 7fa9dd200640 1 -- 172.21.15.86:0/2641094436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9d815c2a0 msgr2=0x7fa9d80ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:11.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.307+0000 7fa9dd200640 1 --2- 172.21.15.86:0/2641094436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9d815c2a0 0x7fa9d80ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.307+0000 7fa9dd200640 1 -- 172.21.15.86:0/2641094436 >> v1:172.21.15.155:6789/0 conn(0x7fa9d806fd50 legacy=0x7fa9d8070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:11.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.307+0000 7fa9dd200640 1 -- 172.21.15.86:0/2641094436 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d806f550 con 0x7fa9d815feb0 2024-04-25T14:07:11.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.311+0000 7fa9dd200640 1 --2- 172.21.15.86:0/2641094436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9d815feb0 0x7fa9d81602c0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0009e80 tx=0x7fa9d001d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6049add98977822 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:11.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.311+0000 7fa9d7400640 1 -- 172.21.15.86:0/2641094436 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d00a56a0 con 0x7fa9d815feb0 2024-04-25T14:07:11.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.311+0000 7fa9d7400640 1 -- 172.21.15.86:0/2641094436 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9d00a5840 con 0x7fa9d815feb0 2024-04-25T14:07:11.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.311+0000 7fa9d7400640 1 -- 172.21.15.86:0/2641094436 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d00a5c10 con 0x7fa9d815feb0 2024-04-25T14:07:11.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.311+0000 7fa9df200640 1 -- 172.21.15.86:0/2641094436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9d815feb0 msgr2=0x7fa9d81602c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:11.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.311+0000 7fa9df200640 1 --2- 172.21.15.86:0/2641094436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9d815feb0 0x7fa9d81602c0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0009e80 tx=0x7fa9d001d3b0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.311+0000 7fa9df200640 1 -- 172.21.15.86:0/2641094436 shutdown_connections 2024-04-25T14:07:11.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.311+0000 7fa9df200640 1 --2- 172.21.15.86:0/2641094436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9d815feb0 0x7fa9d81602c0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.311+0000 7fa9df200640 1 --2- 172.21.15.86:0/2641094436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9d815c2a0 0x7fa9d80ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.311+0000 7fa9df200640 1 -- 172.21.15.86:0/2641094436 >> 172.21.15.86:0/2641094436 conn(0x7fa9d80b4650 msgr2=0x7fa9d80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:11.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.311+0000 7fa9df200640 1 -- 172.21.15.86:0/2641094436 shutdown_connections 2024-04-25T14:07:11.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.311+0000 7fa9df200640 1 -- 172.21.15.86:0/2641094436 wait complete. 2024-04-25T14:07:11.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.315+0000 7fa9df200640 1 Processor -- start 2024-04-25T14:07:11.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.315+0000 7fa9df200640 1 -- start start 2024-04-25T14:07:11.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.315+0000 7fa9df200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9d806fd50 0x7fa9d809aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:11.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.315+0000 7fa9df200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9d815c2a0 0x7fa9d8156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.315+0000 7fa9df200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9d815feb0 0x7fa9d81566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.315+0000 7fa9df200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d8160d90 con 0x7fa9d815feb0 2024-04-25T14:07:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.315+0000 7fa9df200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d8160c10 con 0x7fa9d806fd50 2024-04-25T14:07:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.315+0000 7fa9df200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa9d80ae670 con 0x7fa9d815c2a0 2024-04-25T14:07:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.315+0000 7fa9dd200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9d815feb0 0x7fa9d81566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.315+0000 7fa9dd200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9d815feb0 0x7fa9d81566d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37836/0 (socket says 172.21.15.86:37836) 2024-04-25T14:07:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.315+0000 7fa9dd200640 1 -- 172.21.15.86:0/3613817817 learned_addr learned my addr 172.21.15.86:0/3613817817 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.315+0000 7fa9d7e00640 1 --2- 172.21.15.86:0/3613817817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9d806fd50 0x7fa9d809aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.315+0000 7fa9cf400640 1 --2- 172.21.15.86:0/3613817817 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9d815c2a0 0x7fa9d8156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.315+0000 7fa9dd200640 1 -- 172.21.15.86:0/3613817817 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9d815c2a0 msgr2=0x7fa9d8156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.315+0000 7fa9dd200640 1 --2- 172.21.15.86:0/3613817817 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9d815c2a0 0x7fa9d8156190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.315+0000 7fa9dd200640 1 -- 172.21.15.86:0/3613817817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9d806fd50 msgr2=0x7fa9d809aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.315+0000 7fa9dd200640 1 --2- 172.21.15.86:0/3613817817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9d806fd50 0x7fa9d809aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.315+0000 7fa9dd200640 1 -- 172.21.15.86:0/3613817817 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d815a790 con 0x7fa9d815feb0 2024-04-25T14:07:11.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.315+0000 7fa9dd200640 1 --2- 172.21.15.86:0/3613817817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9d815feb0 0x7fa9d81566d0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fa9d001a580 tx=0x7fa9d00196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.315+0000 7fa9d4c00640 1 -- 172.21.15.86:0/3613817817 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d00bc030 con 0x7fa9d815feb0 2024-04-25T14:07:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.315+0000 7fa9d4c00640 1 -- 172.21.15.86:0/3613817817 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9d00b8470 con 0x7fa9d815feb0 2024-04-25T14:07:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.315+0000 7fa9df200640 1 -- 172.21.15.86:0/3613817817 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d815dd30 con 0x7fa9d815feb0 2024-04-25T14:07:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.315+0000 7fa9d4c00640 1 -- 172.21.15.86:0/3613817817 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0009510 con 0x7fa9d815feb0 2024-04-25T14:07:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.315+0000 7fa9df200640 1 -- 172.21.15.86:0/3613817817 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d815e010 con 0x7fa9d815feb0 2024-04-25T14:07:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.319+0000 7fa9df200640 1 -- 172.21.15.86:0/3613817817 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9d8069d00 con 0x7fa9d815feb0 2024-04-25T14:07:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.319+0000 7fa9d4c00640 1 -- 172.21.15.86:0/3613817817 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7fa9d00bf070 con 0x7fa9d815feb0 2024-04-25T14:07:11.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.319+0000 7fa9d4c00640 1 --2- 172.21.15.86:0/3613817817 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fa9b406f570 0x7fa9b4071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:11.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.319+0000 7fa9d4c00640 1 -- 172.21.15.86:0/3613817817 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7fa9d012a920 con 0x7fa9d815feb0 2024-04-25T14:07:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.319+0000 7fa9d7e00640 1 --2- 172.21.15.86:0/3613817817 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fa9b406f570 0x7fa9b4071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.319+0000 7fa9d7e00640 1 --2- 172.21.15.86:0/3613817817 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fa9b406f570 0x7fa9b4071a80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80b4200 tx=0x7fa9c8008040 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:11.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.323+0000 7fa9d4c00640 1 -- 172.21.15.86:0/3613817817 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa9d00c6070 con 0x7fa9d815feb0 2024-04-25T14:07:11.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.511+0000 7fa9df200640 1 -- 172.21.15.86:0/3613817817 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9d8076bf0 con 0x7fa9d815feb0 2024-04-25T14:07:11.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.511+0000 7fa9d4c00640 1 -- 172.21.15.86:0/3613817817 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5205 (secure 0 0 0) 0x7fa9d0016600 con 0x7fa9d815feb0 2024-04-25T14:07:11.513 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:07:11.514 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":55,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1210051860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1210051860},{"type":"v1","addr":"172.21.15.155:6833","nonce":1210051860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2868300438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2868300438},{"type":"v1","addr":"172.21.15.155:6835","nonce":2868300438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:06:44.910717+0000","modified":"2024-04-25T14:07:11.137141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6603,"mds_1":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"c","rank":1,"incarnation":55,"state":"up:creating","state_seq":1,"addr":"172.21.15.86:6835/2682115891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2682115891},{"type":"v1","addr":"172.21.15.86:6835","nonce":2682115891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6603":{"gid":6603,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/1088108173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1088108173},{"type":"v1","addr":"172.21.15.86:6837","nonce":1088108173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6603,"qdb_cluster":[6603]},"id":5}]} 2024-04-25T14:07:11.514 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 55 2024-04-25T14:07:11.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.515+0000 7fa9df200640 1 -- 172.21.15.86:0/3613817817 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fa9b406f570 msgr2=0x7fa9b4071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.515+0000 7fa9df200640 1 --2- 172.21.15.86:0/3613817817 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fa9b406f570 0x7fa9b4071a80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80b4200 tx=0x7fa9c8008040 comp rx=0 tx=0).stop 2024-04-25T14:07:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.515+0000 7fa9df200640 1 -- 172.21.15.86:0/3613817817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9d815feb0 msgr2=0x7fa9d81566d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.515+0000 7fa9df200640 1 --2- 172.21.15.86:0/3613817817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9d815feb0 0x7fa9d81566d0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fa9d001a580 tx=0x7fa9d00196f0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.515+0000 7fa9df200640 1 -- 172.21.15.86:0/3613817817 shutdown_connections 2024-04-25T14:07:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.515+0000 7fa9df200640 1 --2- 172.21.15.86:0/3613817817 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fa9b406f570 0x7fa9b4071a80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.515+0000 7fa9df200640 1 --2- 172.21.15.86:0/3613817817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9d815feb0 0x7fa9d81566d0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.515+0000 7fa9df200640 1 --2- 172.21.15.86:0/3613817817 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9d815c2a0 0x7fa9d8156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.515+0000 7fa9df200640 1 --2- 172.21.15.86:0/3613817817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9d806fd50 0x7fa9d809aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.515+0000 7fa9df200640 1 -- 172.21.15.86:0/3613817817 >> 172.21.15.86:0/3613817817 conn(0x7fa9d80b4650 msgr2=0x7fa9d80af0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.515+0000 7fa9df200640 1 -- 172.21.15.86:0/3613817817 shutdown_connections 2024-04-25T14:07:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:11.515+0000 7fa9df200640 1 -- 172.21.15.86:0/3613817817 wait complete. 2024-04-25T14:07:11.938 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:06:44.910717+0000', 'modified': '2024-04-25T14:07:11.137141+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 6603, 'mds_1': 6600}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6600': {'gid': 6600, 'name': 'c', 'rank': 1, 'incarnation': 55, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.86:6835/2682115891', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6834', 'nonce': 2682115891}, {'type': 'v1', 'addr': '172.21.15.86:6835', 'nonce': 2682115891}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6603': {'gid': 6603, 'name': 'a', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/1088108173', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 1088108173}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 1088108173}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6603, 'qdb_cluster': [6603]} 2024-04-25T14:07:11.938 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6600:up:creating 2024-04-25T14:07:12.939 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:07:13.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.095+0000 7f78cf400640 1 Processor -- start 2024-04-25T14:07:13.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.095+0000 7f78cf400640 1 -- start start 2024-04-25T14:07:13.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78cf400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f78c80cfca0 0x7f78c806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:13.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78cf400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78c8072bd0 con 0x7f78c80730b0 2024-04-25T14:07:13.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78cf400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78c806f930 con 0x7f78c80d16e0 2024-04-25T14:07:13.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78cf400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f78c8070470 con 0x7f78c80cfca0 2024-04-25T14:07:13.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78cd400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f78c80730b0 0x7f78c80734c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40378/0 (socket says 172.21.15.86:40378) 2024-04-25T14:07:13.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78cd400640 1 -- 172.21.15.86:0/3750671459 learned_addr learned my addr 172.21.15.86:0/3750671459 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:07:13.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78cca00640 1 --2- 172.21.15.86:0/3750671459 >> v2:172.21.15.155:3300/0 conn(0x7f78c80cfca0 0x7f78c806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:13.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78c7e00640 1 -- 172.21.15.86:0/3750671459 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 674222069 0 0) 0x7f78c8072bd0 con 0x7f78c80730b0 2024-04-25T14:07:13.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78c7e00640 1 -- 172.21.15.86:0/3750671459 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78ac003530 con 0x7f78c80730b0 2024-04-25T14:07:13.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78c7e00640 1 -- 172.21.15.86:0/3750671459 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1927934959 0 0) 0x7f78c806f930 con 0x7f78c80d16e0 2024-04-25T14:07:13.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78c7e00640 1 -- 172.21.15.86:0/3750671459 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78c8072bd0 con 0x7f78c80d16e0 2024-04-25T14:07:13.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78c7e00640 1 -- 172.21.15.86:0/3750671459 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 284311299 0 0) 0x7f78ac003530 con 0x7f78c80730b0 2024-04-25T14:07:13.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78c7e00640 1 -- 172.21.15.86:0/3750671459 >> v2:172.21.15.155:3300/0 conn(0x7f78c80cfca0 msgr2=0x7f78c806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:13.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78c7e00640 1 --2- 172.21.15.86:0/3750671459 >> v2:172.21.15.155:3300/0 conn(0x7f78c80cfca0 0x7f78c806f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78c7e00640 1 -- 172.21.15.86:0/3750671459 >> v1:172.21.15.155:6789/0 conn(0x7f78c80d16e0 legacy=0x7f78c80d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:13.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78c7e00640 1 -- 172.21.15.86:0/3750671459 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c806fbd0 con 0x7f78c80730b0 2024-04-25T14:07:13.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78c7e00640 1 -- 172.21.15.86:0/3750671459 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f78c0001a00 con 0x7f78c80730b0 2024-04-25T14:07:13.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78c7e00640 1 -- 172.21.15.86:0/3750671459 >> v1:172.21.15.86:6789/0 conn(0x7f78c80730b0 legacy=0x7f78c80734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:13.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78c7e00640 1 --2- 172.21.15.86:0/3750671459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78ac003bc0 0x7f78ac004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:13.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78c7e00640 1 --2- 172.21.15.86:0/3750671459 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78ac004810 0x7f78ac004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:13.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78c7e00640 1 --2- 172.21.15.86:0/3750671459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78ac005220 0x7f78ac00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:13.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78c7e00640 1 -- 172.21.15.86:0/3750671459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78c8070470 con 0x7f78ac005220 2024-04-25T14:07:13.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78c7e00640 1 -- 172.21.15.86:0/3750671459 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac00e3d0 con 0x7f78ac003bc0 2024-04-25T14:07:13.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78c7e00640 1 -- 172.21.15.86:0/3750671459 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f78ac00e5d0 con 0x7f78ac004810 2024-04-25T14:07:13.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78cd400640 1 --2- 172.21.15.86:0/3750671459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78ac005220 0x7f78ac00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:13.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78cca00640 1 --2- 172.21.15.86:0/3750671459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78ac003bc0 0x7f78ac004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:13.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78bfe00640 1 --2- 172.21.15.86:0/3750671459 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78ac004810 0x7f78ac004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:13.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78cd400640 1 -- 172.21.15.86:0/3750671459 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78ac004810 msgr2=0x7f78ac004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:13.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78cd400640 1 --2- 172.21.15.86:0/3750671459 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78ac004810 0x7f78ac004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78cd400640 1 -- 172.21.15.86:0/3750671459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78ac003bc0 msgr2=0x7f78ac004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:13.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78cd400640 1 --2- 172.21.15.86:0/3750671459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78ac003bc0 0x7f78ac004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78cd400640 1 -- 172.21.15.86:0/3750671459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78ac003530 con 0x7f78ac005220 2024-04-25T14:07:13.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.099+0000 7f78cca00640 1 --2- 172.21.15.86:0/3750671459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78ac003bc0 0x7f78ac004010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:13.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.103+0000 7f78cd400640 1 -- 172.21.15.86:0/3750671459 reap_dead start 2024-04-25T14:07:13.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.103+0000 7f78cd400640 1 --2- 172.21.15.86:0/3750671459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78ac005220 0x7f78ac00dbe0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f78c000a7e0 tx=0x7f78c001fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e25b6f5c3bfb13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:13.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.103+0000 7f78c7e00640 1 -- 172.21.15.86:0/3750671459 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c000a430 con 0x7f78ac005220 2024-04-25T14:07:13.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.103+0000 7f78c7e00640 1 -- 172.21.15.86:0/3750671459 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78c00aa670 con 0x7f78ac005220 2024-04-25T14:07:13.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.103+0000 7f78c7e00640 1 -- 172.21.15.86:0/3750671459 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c00aaa60 con 0x7f78ac005220 2024-04-25T14:07:13.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.103+0000 7f78cf400640 1 -- 172.21.15.86:0/3750671459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78ac005220 msgr2=0x7f78ac00dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:13.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.103+0000 7f78cf400640 1 --2- 172.21.15.86:0/3750671459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78ac005220 0x7f78ac00dbe0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f78c000a7e0 tx=0x7f78c001fa40 comp rx=0 tx=0).stop 2024-04-25T14:07:13.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.103+0000 7f78cf400640 1 -- 172.21.15.86:0/3750671459 shutdown_connections 2024-04-25T14:07:13.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.103+0000 7f78cf400640 1 --2- 172.21.15.86:0/3750671459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78ac005220 0x7f78ac00dbe0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.103+0000 7f78cf400640 1 -- 172.21.15.86:0/3750671459 >> 172.21.15.86:0/3750671459 conn(0x7f78c80b4650 msgr2=0x7f78c80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:13.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.103+0000 7f78cf400640 1 -- 172.21.15.86:0/3750671459 shutdown_connections 2024-04-25T14:07:13.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.103+0000 7f78cf400640 1 -- 172.21.15.86:0/3750671459 wait complete. 2024-04-25T14:07:13.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.107+0000 7f78cf400640 1 Processor -- start 2024-04-25T14:07:13.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.107+0000 7f78cf400640 1 -- start start 2024-04-25T14:07:13.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.107+0000 7f78cf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78ac005220 0x7f78c8158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:13.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.107+0000 7f78cf400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c80cad80 0x7f78c80ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:13.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.107+0000 7f78cf400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78c80cd160 0x7f78c80cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:13.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.107+0000 7f78cf400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78c806f3f0 con 0x7f78ac005220 2024-04-25T14:07:13.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.107+0000 7f78cf400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f78c8070190 con 0x7f78c80cad80 2024-04-25T14:07:13.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.107+0000 7f78cf400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f78c806f570 con 0x7f78c80cd160 2024-04-25T14:07:13.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.107+0000 7f78cca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78ac005220 0x7f78c8158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:13.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.107+0000 7f78cca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78ac005220 0x7f78c8158380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37842/0 (socket says 172.21.15.86:37842) 2024-04-25T14:07:13.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.107+0000 7f78cca00640 1 -- 172.21.15.86:0/1389284164 learned_addr learned my addr 172.21.15.86:0/1389284164 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:13.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.107+0000 7f78bfe00640 1 --2- 172.21.15.86:0/1389284164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c80cad80 0x7f78c80ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:13.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.107+0000 7f78cd400640 1 --2- 172.21.15.86:0/1389284164 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78c80cd160 0x7f78c80cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:13.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.107+0000 7f78cca00640 1 -- 172.21.15.86:0/1389284164 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78c80cd160 msgr2=0x7f78c80cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:13.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.107+0000 7f78cca00640 1 --2- 172.21.15.86:0/1389284164 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78c80cd160 0x7f78c80cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.107+0000 7f78cca00640 1 -- 172.21.15.86:0/1389284164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c80cad80 msgr2=0x7f78c80ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:13.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.107+0000 7f78cca00640 1 --2- 172.21.15.86:0/1389284164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c80cad80 0x7f78c80ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.107+0000 7f78cca00640 1 -- 172.21.15.86:0/1389284164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c815ff80 con 0x7f78ac005220 2024-04-25T14:07:13.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.107+0000 7f78cd400640 1 --2- 172.21.15.86:0/1389284164 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78c80cd160 0x7f78c80cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:13.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.111+0000 7f78cca00640 1 --2- 172.21.15.86:0/1389284164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78ac005220 0x7f78c8158380 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f78b8011560 tx=0x7f78b8011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:13.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.111+0000 7f78c5600640 1 -- 172.21.15.86:0/1389284164 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8015870 con 0x7f78ac005220 2024-04-25T14:07:13.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.111+0000 7f78c5600640 1 -- 172.21.15.86:0/1389284164 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b8015a10 con 0x7f78ac005220 2024-04-25T14:07:13.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.111+0000 7f78c5600640 1 -- 172.21.15.86:0/1389284164 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8015de0 con 0x7f78ac005220 2024-04-25T14:07:13.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.111+0000 7f78cf400640 1 -- 172.21.15.86:0/1389284164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c8160260 con 0x7f78ac005220 2024-04-25T14:07:13.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.111+0000 7f78cf400640 1 -- 172.21.15.86:0/1389284164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c81e5a70 con 0x7f78ac005220 2024-04-25T14:07:13.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.111+0000 7f78cf400640 1 -- 172.21.15.86:0/1389284164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c809f4b0 con 0x7f78ac005220 2024-04-25T14:07:13.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.111+0000 7f78c5600640 1 -- 172.21.15.86:0/1389284164 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7f78b801d400 con 0x7f78ac005220 2024-04-25T14:07:13.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.111+0000 7f78c5600640 1 --2- 172.21.15.86:0/1389284164 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f78a406f5a0 0x7f78a4071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:13.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.111+0000 7f78c5600640 1 -- 172.21.15.86:0/1389284164 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f78b808fff0 con 0x7f78ac005220 2024-04-25T14:07:13.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.111+0000 7f78bfe00640 1 --2- 172.21.15.86:0/1389284164 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f78a406f5a0 0x7f78a4071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.115+0000 7f78bfe00640 1 --2- 172.21.15.86:0/1389284164 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f78a406f5a0 0x7f78a4071ab0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f78c80cea50 tx=0x7f78b0009040 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:13.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.115+0000 7f78c5600640 1 -- 172.21.15.86:0/1389284164 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f78b8019030 con 0x7f78ac005220 2024-04-25T14:07:13.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.307+0000 7f78cf400640 1 -- 172.21.15.86:0/1389284164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78c8069d00 con 0x7f78ac005220 2024-04-25T14:07:13.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.307+0000 7f78c5600640 1 -- 172.21.15.86:0/1389284164 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5208 (secure 0 0 0) 0x7f78b805cae0 con 0x7f78ac005220 2024-04-25T14:07:13.310 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:07:13.310 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":56,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1210051860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1210051860},{"type":"v1","addr":"172.21.15.155:6833","nonce":1210051860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2868300438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2868300438},{"type":"v1","addr":"172.21.15.155:6835","nonce":2868300438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:06:44.910717+0000","modified":"2024-04-25T14:07:12.139089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6603,"mds_1":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"c","rank":1,"incarnation":55,"state":"up:active","state_seq":9,"addr":"172.21.15.86:6835/2682115891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2682115891},{"type":"v1","addr":"172.21.15.86:6835","nonce":2682115891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6603":{"gid":6603,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/1088108173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1088108173},{"type":"v1","addr":"172.21.15.86:6837","nonce":1088108173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6603,"qdb_cluster":[6600,6603]},"id":5}]} 2024-04-25T14:07:13.310 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 56 2024-04-25T14:07:13.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.311+0000 7f78cf400640 1 -- 172.21.15.86:0/1389284164 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f78a406f5a0 msgr2=0x7f78a4071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:13.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.311+0000 7f78cf400640 1 --2- 172.21.15.86:0/1389284164 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f78a406f5a0 0x7f78a4071ab0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f78c80cea50 tx=0x7f78b0009040 comp rx=0 tx=0).stop 2024-04-25T14:07:13.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.311+0000 7f78cf400640 1 -- 172.21.15.86:0/1389284164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78ac005220 msgr2=0x7f78c8158380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:13.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.311+0000 7f78cf400640 1 --2- 172.21.15.86:0/1389284164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78ac005220 0x7f78c8158380 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f78b8011560 tx=0x7f78b8011a20 comp rx=0 tx=0).stop 2024-04-25T14:07:13.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.311+0000 7f78cf400640 1 -- 172.21.15.86:0/1389284164 shutdown_connections 2024-04-25T14:07:13.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.311+0000 7f78cf400640 1 --2- 172.21.15.86:0/1389284164 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f78a406f5a0 0x7f78a4071ab0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.311+0000 7f78cf400640 1 --2- 172.21.15.86:0/1389284164 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78c80cd160 0x7f78c80cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.311+0000 7f78cf400640 1 --2- 172.21.15.86:0/1389284164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c80cad80 0x7f78c80ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.311+0000 7f78cf400640 1 --2- 172.21.15.86:0/1389284164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78ac005220 0x7f78c8158380 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.311+0000 7f78cf400640 1 -- 172.21.15.86:0/1389284164 >> 172.21.15.86:0/1389284164 conn(0x7f78c80b4650 msgr2=0x7f78c80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:13.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.311+0000 7f78cf400640 1 -- 172.21.15.86:0/1389284164 shutdown_connections 2024-04-25T14:07:13.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.311+0000 7f78cf400640 1 -- 172.21.15.86:0/1389284164 wait complete. 2024-04-25T14:07:13.735 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:06:44.910717+0000', 'modified': '2024-04-25T14:07:12.139089+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 6603, 'mds_1': 6600}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6600': {'gid': 6600, 'name': 'c', 'rank': 1, 'incarnation': 55, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.86:6835/2682115891', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6834', 'nonce': 2682115891}, {'type': 'v1', 'addr': '172.21.15.86:6835', 'nonce': 2682115891}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6603': {'gid': 6603, 'name': 'a', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/1088108173', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 1088108173}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 1088108173}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6603, 'qdb_cluster': [6600, 6603]} 2024-04-25T14:07:13.735 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2024-04-25T14:07:13.735 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T14:07:13.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.883+0000 7f780aa00640 1 Processor -- start 2024-04-25T14:07:13.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.883+0000 7f780aa00640 1 -- start start 2024-04-25T14:07:13.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.883+0000 7f780aa00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f78040d1c70 0x7f78040d2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:13.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.883+0000 7f780aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f78040b7480 0x7f78040b7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:13.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.883+0000 7f780aa00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7804160ec0 con 0x7f7804070e70 2024-04-25T14:07:13.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.883+0000 7f780aa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f78041611a0 con 0x7f78040b7480 2024-04-25T14:07:13.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.883+0000 7f780aa00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f78040d1190 con 0x7f78040d1c70 2024-04-25T14:07:13.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.883+0000 7f780a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f78040b7480 0x7f78040b7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:13.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.883+0000 7f780a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f78040b7480 0x7f78040b7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37844/0 (socket says 172.21.15.86:37844) 2024-04-25T14:07:13.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.883+0000 7f780a000640 1 -- 172.21.15.86:0/729037302 learned_addr learned my addr 172.21.15.86:0/729037302 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:13.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.883+0000 7f7809600640 1 --2- 172.21.15.86:0/729037302 >> v2:172.21.15.155:3300/0 conn(0x7f78040d1c70 0x7f78040d2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:13.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.883+0000 7f780a000640 1 -- 172.21.15.86:0/729037302 >> v2:172.21.15.155:3300/0 conn(0x7f78040d1c70 msgr2=0x7f78040d2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:13.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.883+0000 7f780a000640 1 --2- 172.21.15.86:0/729037302 >> v2:172.21.15.155:3300/0 conn(0x7f78040d1c70 0x7f78040d2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.887+0000 7f780a000640 1 -- 172.21.15.86:0/729037302 >> v1:172.21.15.155:6789/0 conn(0x7f7804070e70 legacy=0x7f7804071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:13.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.887+0000 7f780a000640 1 -- 172.21.15.86:0/729037302 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78040d1470 con 0x7f78040b7480 2024-04-25T14:07:13.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.887+0000 7f7809600640 1 --2- 172.21.15.86:0/729037302 >> v2:172.21.15.155:3300/0 conn(0x7f78040d1c70 0x7f78040d2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:13.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.887+0000 7f780a000640 1 --2- 172.21.15.86:0/729037302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78040b7480 0x7f78040b7890 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f7804074250 tx=0x7f77fc01c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8a15519cd497b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:13.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.887+0000 7f7803e00640 1 -- 172.21.15.86:0/729037302 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc0a5860 con 0x7f78040b7480 2024-04-25T14:07:13.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.887+0000 7f7803e00640 1 -- 172.21.15.86:0/729037302 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77fc0a5a00 con 0x7f78040b7480 2024-04-25T14:07:13.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.887+0000 7f7803e00640 1 -- 172.21.15.86:0/729037302 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc0a5dd0 con 0x7f78040b7480 2024-04-25T14:07:13.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.887+0000 7f780aa00640 1 -- 172.21.15.86:0/729037302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78040b7480 msgr2=0x7f78040b7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:13.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.887+0000 7f780aa00640 1 --2- 172.21.15.86:0/729037302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78040b7480 0x7f78040b7890 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f7804074250 tx=0x7f77fc01c980 comp rx=0 tx=0).stop 2024-04-25T14:07:13.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.891+0000 7f780aa00640 1 -- 172.21.15.86:0/729037302 shutdown_connections 2024-04-25T14:07:13.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.891+0000 7f780aa00640 1 --2- 172.21.15.86:0/729037302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78040b7480 0x7f78040b7890 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.891+0000 7f780aa00640 1 --2- 172.21.15.86:0/729037302 >> v2:172.21.15.155:3300/0 conn(0x7f78040d1c70 0x7f78040d2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.891+0000 7f780aa00640 1 -- 172.21.15.86:0/729037302 >> 172.21.15.86:0/729037302 conn(0x7f78040cfea0 msgr2=0x7f78040d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:13.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.891+0000 7f780aa00640 1 -- 172.21.15.86:0/729037302 shutdown_connections 2024-04-25T14:07:13.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.891+0000 7f780aa00640 1 -- 172.21.15.86:0/729037302 wait complete. 2024-04-25T14:07:13.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.891+0000 7f780aa00640 1 Processor -- start 2024-04-25T14:07:13.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.891+0000 7f780aa00640 1 -- start start 2024-04-25T14:07:13.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.891+0000 7f780aa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7804070e70 0x7f7804157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:13.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.891+0000 7f780aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78040b7480 0x7f78041549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:13.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.891+0000 7f780aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78040d1c70 0x7f780406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:13.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.891+0000 7f780aa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f780416c7d0 con 0x7f78040b7480 2024-04-25T14:07:13.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.891+0000 7f780aa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7804160bf0 con 0x7f78040d1c70 2024-04-25T14:07:13.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.891+0000 7f780aa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f780416c950 con 0x7f7804070e70 2024-04-25T14:07:13.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.895+0000 7f7808c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78040b7480 0x7f78041549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:13.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.895+0000 7f7809600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7804070e70 0x7f7804157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:13.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.895+0000 7f7808c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78040b7480 0x7f78041549c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37856/0 (socket says 172.21.15.86:37856) 2024-04-25T14:07:13.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.895+0000 7f7808c00640 1 -- 172.21.15.86:0/3049447809 learned_addr learned my addr 172.21.15.86:0/3049447809 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:13.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.895+0000 7f780a000640 1 --2- 172.21.15.86:0/3049447809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78040d1c70 0x7f780406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:13.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.895+0000 7f7808c00640 1 -- 172.21.15.86:0/3049447809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7804070e70 msgr2=0x7f7804157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:13.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.895+0000 7f7808c00640 1 --2- 172.21.15.86:0/3049447809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7804070e70 0x7f7804157cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.895+0000 7f7808c00640 1 -- 172.21.15.86:0/3049447809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78040d1c70 msgr2=0x7f780406d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:13.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.895+0000 7f7808c00640 1 --2- 172.21.15.86:0/3049447809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78040d1c70 0x7f780406d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.895+0000 7f7808c00640 1 -- 172.21.15.86:0/3049447809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78041585b0 con 0x7f78040b7480 2024-04-25T14:07:13.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.895+0000 7f7808c00640 1 --2- 172.21.15.86:0/3049447809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78040b7480 0x7f78041549c0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f77f000b410 tx=0x7f77f000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:13.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.895+0000 7f7801600640 1 -- 172.21.15.86:0/3049447809 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0015030 con 0x7f78040b7480 2024-04-25T14:07:13.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.895+0000 7f7801600640 1 -- 172.21.15.86:0/3049447809 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77f0004020 con 0x7f78040b7480 2024-04-25T14:07:13.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.895+0000 7f780aa00640 1 -- 172.21.15.86:0/3049447809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7804158890 con 0x7f78040b7480 2024-04-25T14:07:13.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.895+0000 7f7801600640 1 -- 172.21.15.86:0/3049447809 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0004410 con 0x7f78040b7480 2024-04-25T14:07:13.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.895+0000 7f780aa00640 1 -- 172.21.15.86:0/3049447809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7804158e50 con 0x7f78040b7480 2024-04-25T14:07:13.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.895+0000 7f7801600640 1 -- 172.21.15.86:0/3049447809 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7f77f0018070 con 0x7f78040b7480 2024-04-25T14:07:13.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.899+0000 7f7801600640 1 --2- 172.21.15.86:0/3049447809 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f77e406f5a0 0x7f77e4071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:13.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.899+0000 7f7801600640 1 -- 172.21.15.86:0/3049447809 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f77f008f340 con 0x7f78040b7480 2024-04-25T14:07:13.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.899+0000 7f7809600640 1 --2- 172.21.15.86:0/3049447809 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f77e406f5a0 0x7f77e4071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:13.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.899+0000 7f7813684000 1 Processor -- start 2024-04-25T14:07:13.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.899+0000 7f7813684000 1 -- start start 2024-04-25T14:07:13.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.899+0000 7f7809600640 1 --2- 172.21.15.86:0/3049447809 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f77e406f5a0 0x7f77e4071ab0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f78040c6ee0 tx=0x7f77f8008040 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:13.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.899+0000 7f7813684000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de9aa1dc20 0x55de9ab7bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:13.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.899+0000 7f7813684000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55de9aa09ea0 0x55de9ab7c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:13.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.899+0000 7f7813684000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de9a602280 0x55de9ab7c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:13.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.899+0000 7f7813684000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55de9a9eb160 con 0x55de9a602280 2024-04-25T14:07:13.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.899+0000 7f7813684000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55de9a980e20 con 0x55de9aa1dc20 2024-04-25T14:07:13.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.899+0000 7f7813684000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55de9a939a20 con 0x55de9aa09ea0 2024-04-25T14:07:13.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.899+0000 7f780a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de9a602280 0x55de9ab7c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:13.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.899+0000 7f780a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de9a602280 0x55de9ab7c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37862/0 (socket says 172.21.15.86:37862) 2024-04-25T14:07:13.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.899+0000 7f780a000640 1 -- 172.21.15.86:0/988853814 learned_addr learned my addr 172.21.15.86:0/988853814 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:13.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.899+0000 7f7809600640 1 --2- 172.21.15.86:0/988853814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de9aa1dc20 0x55de9ab7bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:13.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.899+0000 7f7808c00640 1 --2- 172.21.15.86:0/988853814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55de9aa09ea0 0x55de9ab7c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:13.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.899+0000 7f780a000640 1 -- 172.21.15.86:0/988853814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55de9aa09ea0 msgr2=0x55de9ab7c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:13.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.899+0000 7f780a000640 1 --2- 172.21.15.86:0/988853814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55de9aa09ea0 0x55de9ab7c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.899+0000 7f780a000640 1 -- 172.21.15.86:0/988853814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de9aa1dc20 msgr2=0x55de9ab7bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:13.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.899+0000 7f780a000640 1 --2- 172.21.15.86:0/988853814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de9aa1dc20 0x55de9ab7bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.899+0000 7f780a000640 1 -- 172.21.15.86:0/988853814 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de9a93e100 con 0x55de9a602280 2024-04-25T14:07:13.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.899+0000 7f7809600640 1 --2- 172.21.15.86:0/988853814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de9aa1dc20 0x55de9ab7bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:13.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.899+0000 7f780a000640 1 --2- 172.21.15.86:0/988853814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de9a602280 0x55de9ab7c960 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f77fc01a5b0 tx=0x7f77fc004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3242e98bc4dadd53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:13.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.899+0000 7f77f5c00640 1 -- 172.21.15.86:0/988853814 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc0a5860 con 0x55de9a602280 2024-04-25T14:07:13.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.899+0000 7f77f5c00640 1 -- 172.21.15.86:0/988853814 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77fc0a5a00 con 0x55de9a602280 2024-04-25T14:07:13.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.899+0000 7f77f5c00640 1 -- 172.21.15.86:0/988853814 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc0a5dd0 con 0x55de9a602280 2024-04-25T14:07:13.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.903+0000 7f7813684000 1 -- 172.21.15.86:0/988853814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de9a602280 msgr2=0x55de9ab7c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:13.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.903+0000 7f7813684000 1 --2- 172.21.15.86:0/988853814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de9a602280 0x55de9ab7c960 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f77fc01a5b0 tx=0x7f77fc004ac0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.903+0000 7f7813684000 1 -- 172.21.15.86:0/988853814 shutdown_connections 2024-04-25T14:07:13.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.903+0000 7f7813684000 1 --2- 172.21.15.86:0/988853814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de9a602280 0x55de9ab7c960 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.903+0000 7f7813684000 1 --2- 172.21.15.86:0/988853814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55de9aa09ea0 0x55de9ab7c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.903+0000 7f7813684000 1 --2- 172.21.15.86:0/988853814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de9aa1dc20 0x55de9ab7bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.903+0000 7f7813684000 1 -- 172.21.15.86:0/988853814 >> 172.21.15.86:0/988853814 conn(0x55de9a641ff0 msgr2=0x55de9ab314c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:13.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.903+0000 7f7813684000 1 -- 172.21.15.86:0/988853814 shutdown_connections 2024-04-25T14:07:13.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.903+0000 7f7813684000 1 -- 172.21.15.86:0/988853814 wait complete. 2024-04-25T14:07:13.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.903+0000 7f7813684000 1 -- start start 2024-04-25T14:07:13.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.903+0000 7f7813684000 1 Processor -- start 2024-04-25T14:07:13.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.903+0000 7f7813684000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de9a602280 0x55de9ab83070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:13.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.903+0000 7f7813684000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55de9aa09ea0 0x55de9ab835b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:13.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.903+0000 7f7813684000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de9aa1dc20 0x55de9ab3e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:13.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.903+0000 7f7813684000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55de9a98e220 con 0x55de9a602280 2024-04-25T14:07:13.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.903+0000 7f7813684000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55de9a924ea0 con 0x55de9aa1dc20 2024-04-25T14:07:13.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.903+0000 7f7813684000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55de9a97bb00 con 0x55de9aa09ea0 2024-04-25T14:07:13.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.903+0000 7f7813684000 10 client.0 _ll_register_callbacks cb 0x55de9a98ddc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:13.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.903+0000 7f7809600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de9a602280 0x55de9ab83070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:13.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.903+0000 7f7809600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de9a602280 0x55de9ab83070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37872/0 (socket says 172.21.15.86:37872) 2024-04-25T14:07:13.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.903+0000 7f7809600640 1 -- 172.21.15.86:0/2620030592 learned_addr learned my addr 172.21.15.86:0/2620030592 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:13.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.903+0000 7f780a000640 1 --2- 172.21.15.86:0/2620030592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de9aa1dc20 0x55de9ab3e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:13.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.903+0000 7f7808c00640 1 --2- 172.21.15.86:0/2620030592 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55de9aa09ea0 0x55de9ab835b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:13.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7809600640 1 -- 172.21.15.86:0/2620030592 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55de9aa09ea0 msgr2=0x55de9ab835b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:13.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7809600640 1 --2- 172.21.15.86:0/2620030592 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55de9aa09ea0 0x55de9ab835b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7809600640 1 -- 172.21.15.86:0/2620030592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de9aa1dc20 msgr2=0x55de9ab3e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:13.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7809600640 1 --2- 172.21.15.86:0/2620030592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de9aa1dc20 0x55de9ab3e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7809600640 1 -- 172.21.15.86:0/2620030592 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de9a920500 con 0x55de9a602280 2024-04-25T14:07:13.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7809600640 1 --2- 172.21.15.86:0/2620030592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de9a602280 0x55de9ab83070 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f77f8015800 tx=0x7f77f80046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa538309f8462af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:13.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f77ea000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:07:13.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f77ea000640 1 -- 172.21.15.86:0/2620030592 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f8024030 con 0x55de9a602280 2024-04-25T14:07:13.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7813684000 1 -- 172.21.15.86:0/2620030592 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77fc0a5a00 con 0x55de9a602280 2024-04-25T14:07:13.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f77ea000640 1 -- 172.21.15.86:0/2620030592 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f77f80139a0 con 0x55de9a602280 2024-04-25T14:07:13.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f77ea000640 1 -- 172.21.15.86:0/2620030592 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f77f8030050 con 0x55de9a602280 2024-04-25T14:07:13.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7813684000 10 client.6909 fetch_fsmap learned FSMap version 56 2024-04-25T14:07:13.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7813684000 1 -- 172.21.15.86:0/2620030592 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x55de9a906ea0 con 0x55de9a602280 2024-04-25T14:07:13.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f77ea000640 1 -- 172.21.15.86:0/2620030592 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77fc0a5a00 con 0x55de9a602280 2024-04-25T14:07:13.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f77ea000640 1 -- 172.21.15.86:0/2620030592 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 56) ==== 3514+0+0 (secure 0 0 0) 0x7f77f8033020 con 0x55de9a602280 2024-04-25T14:07:13.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7813684000 10 client.6909 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T14:07:13.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7813684000 10 client.6909 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T14:07:13.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7813684000 20 client.6909 populate_metadata read hostname 'smithi086' 2024-04-25T14:07:13.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7813684000 1 --2- 172.21.15.86:0/2620030592 >> [v2:172.21.15.86:6834/2682115891,v1:172.21.15.86:6835/2682115891] conn(0x55de9a6267d0 0x55de9ab3ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:13.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7813684000 4 client.6909 mds_command: new command op to 6600 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:07:13.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7813684000 1 -- 172.21.15.86:0/2620030592 --> [v2:172.21.15.86:6834/2682115891,v1:172.21.15.86:6835/2682115891] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de9a919be0 con 0x55de9a6267d0 2024-04-25T14:07:13.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7808c00640 1 --2- 172.21.15.86:0/2620030592 >> [v2:172.21.15.86:6834/2682115891,v1:172.21.15.86:6835/2682115891] conn(0x55de9a6267d0 0x55de9ab3ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:13.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7808c00640 1 --2- 172.21.15.86:0/2620030592 >> [v2:172.21.15.86:6834/2682115891,v1:172.21.15.86:6835/2682115891] conn(0x55de9a6267d0 0x55de9ab3ee30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=11f517669bd28b33 server_cookie=838892856876bbe8 in_seq=0 out_seq=0 2024-04-25T14:07:13.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f77ea000640 10 client.6909 ms_handle_connect on v2:172.21.15.86:6834/2682115891 2024-04-25T14:07:13.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f77ea000640 1 -- 172.21.15.86:0/2620030592 <== mds.1 v2:172.21.15.86:6834/2682115891 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55de9a919be0 con 0x55de9a6267d0 2024-04-25T14:07:13.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f77ea000640 10 client.6909 handle_command_reply: tid=0 2024-04-25T14:07:13.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7813684000 1 client.6909 shutdown 2024-04-25T14:07:13.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7813684000 1 -- 172.21.15.86:0/2620030592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de9a602280 msgr2=0x55de9ab83070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:13.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7813684000 1 --2- 172.21.15.86:0/2620030592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de9a602280 0x55de9ab83070 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f77f8015800 tx=0x7f77f80046d0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7813684000 1 -- 172.21.15.86:0/2620030592 shutdown_connections 2024-04-25T14:07:13.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7813684000 1 --2- 172.21.15.86:0/2620030592 >> [v2:172.21.15.86:6834/2682115891,v1:172.21.15.86:6835/2682115891] conn(0x55de9a6267d0 0x55de9ab3ee30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7813684000 1 --2- 172.21.15.86:0/2620030592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de9aa1dc20 0x55de9ab3e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7813684000 1 --2- 172.21.15.86:0/2620030592 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55de9aa09ea0 0x55de9ab835b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7813684000 1 --2- 172.21.15.86:0/2620030592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de9a602280 0x55de9ab83070 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f77ea000640 0 client.6909 ms_handle_reset on v2:172.21.15.86:6834/2682115891 2024-04-25T14:07:13.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7813684000 1 -- 172.21.15.86:0/2620030592 >> 172.21.15.86:0/2620030592 conn(0x55de9a641ff0 msgr2=0x55de9ab31c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:13.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7813684000 1 -- 172.21.15.86:0/2620030592 shutdown_connections 2024-04-25T14:07:13.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7813684000 1 -- 172.21.15.86:0/2620030592 wait complete. 2024-04-25T14:07:13.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.907+0000 7f7813684000 20 client.6909 trim_cache size 0 max 16384 2024-04-25T14:07:13.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.927+0000 7f7813684000 1 Processor -- start 2024-04-25T14:07:13.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.927+0000 7f7813684000 1 -- start start 2024-04-25T14:07:13.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.927+0000 7f7813684000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de9a602280 0x55de9ab82050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:13.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.927+0000 7f7813684000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55de9aa09ea0 0x55de9ab82590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:13.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.927+0000 7f7813684000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de9aa1dc20 0x55de9ab880b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:13.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.927+0000 7f7813684000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7804206200 con 0x55de9a602280 2024-04-25T14:07:13.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.927+0000 7f7813684000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f78040c7dd0 con 0x55de9aa1dc20 2024-04-25T14:07:13.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.927+0000 7f7813684000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7804204b60 con 0x55de9aa09ea0 2024-04-25T14:07:13.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.927+0000 7f7809600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de9a602280 0x55de9ab82050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:13.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.927+0000 7f7809600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de9a602280 0x55de9ab82050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37874/0 (socket says 172.21.15.86:37874) 2024-04-25T14:07:13.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.927+0000 7f7809600640 1 -- 172.21.15.86:0/1148060221 learned_addr learned my addr 172.21.15.86:0/1148060221 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:13.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.927+0000 7f7809600640 1 -- 172.21.15.86:0/1148060221 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55de9aa09ea0 msgr2=0x55de9ab82590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T14:07:13.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.927+0000 7f7809600640 1 --2- 172.21.15.86:0/1148060221 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55de9aa09ea0 0x55de9ab82590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.927+0000 7f7809600640 1 -- 172.21.15.86:0/1148060221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de9aa1dc20 msgr2=0x55de9ab880b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T14:07:13.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.927+0000 7f7809600640 1 --2- 172.21.15.86:0/1148060221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de9aa1dc20 0x55de9ab880b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.927+0000 7f7809600640 1 -- 172.21.15.86:0/1148060221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de9ab35840 con 0x55de9a602280 2024-04-25T14:07:13.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.931+0000 7f7809600640 1 --2- 172.21.15.86:0/1148060221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de9a602280 0x55de9ab82050 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f77f800fdc0 tx=0x7f77f801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a760c10ae4da3b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:13.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.931+0000 7f77f5200640 1 -- 172.21.15.86:0/1148060221 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f8024040 con 0x55de9a602280 2024-04-25T14:07:13.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.931+0000 7f77f5200640 1 -- 172.21.15.86:0/1148060221 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77f8019040 con 0x55de9a602280 2024-04-25T14:07:13.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.931+0000 7f77f5200640 1 -- 172.21.15.86:0/1148060221 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f8013e40 con 0x55de9a602280 2024-04-25T14:07:13.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.931+0000 7f7813684000 1 -- 172.21.15.86:0/1148060221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de9a602280 msgr2=0x55de9ab82050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:13.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.931+0000 7f7813684000 1 --2- 172.21.15.86:0/1148060221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de9a602280 0x55de9ab82050 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f77f800fdc0 tx=0x7f77f801b000 comp rx=0 tx=0).stop 2024-04-25T14:07:13.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.931+0000 7f7813684000 1 -- 172.21.15.86:0/1148060221 shutdown_connections 2024-04-25T14:07:13.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.931+0000 7f7813684000 1 --2- 172.21.15.86:0/1148060221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de9aa1dc20 0x55de9ab880b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.931+0000 7f7813684000 1 --2- 172.21.15.86:0/1148060221 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55de9aa09ea0 0x55de9ab82590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.931+0000 7f7813684000 1 --2- 172.21.15.86:0/1148060221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de9a602280 0x55de9ab82050 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.931+0000 7f7813684000 1 -- 172.21.15.86:0/1148060221 >> 172.21.15.86:0/1148060221 conn(0x55de9a641ff0 msgr2=0x55de9ab39040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:13.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.931+0000 7f7813684000 1 -- 172.21.15.86:0/1148060221 shutdown_connections 2024-04-25T14:07:13.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.931+0000 7f7813684000 1 -- 172.21.15.86:0/1148060221 wait complete. 2024-04-25T14:07:13.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.931+0000 7f7813684000 1 -- start start 2024-04-25T14:07:13.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.931+0000 7f7813684000 1 Processor -- start 2024-04-25T14:07:13.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.931+0000 7f7813684000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de9a602280 0x55de9ab86a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:13.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.931+0000 7f7813684000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de9aa09ea0 0x55de9ab86fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:13.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.931+0000 7f7813684000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55de9aa1dc20 0x55de9ab874e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:13.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.931+0000 7f7813684000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55de9a980130 con 0x55de9aa09ea0 2024-04-25T14:07:13.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.931+0000 7f7813684000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55de9a9167d0 con 0x55de9a602280 2024-04-25T14:07:13.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.931+0000 7f7813684000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55de9a985450 con 0x55de9aa1dc20 2024-04-25T14:07:13.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.931+0000 7f7813684000 10 client.0 _ll_register_callbacks cb 0x7f77f800a710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:13.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7808c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de9aa09ea0 0x55de9ab86fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:13.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7808c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de9aa09ea0 0x55de9ab86fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37880/0 (socket says 172.21.15.86:37880) 2024-04-25T14:07:13.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7808c00640 1 -- 172.21.15.86:0/1308959027 learned_addr learned my addr 172.21.15.86:0/1308959027 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:13.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7809600640 1 --2- 172.21.15.86:0/1308959027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de9a602280 0x55de9ab86a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:13.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f780a000640 1 --2- 172.21.15.86:0/1308959027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55de9aa1dc20 0x55de9ab874e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:13.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7808c00640 1 -- 172.21.15.86:0/1308959027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55de9aa1dc20 msgr2=0x55de9ab874e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:13.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7808c00640 1 --2- 172.21.15.86:0/1308959027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55de9aa1dc20 0x55de9ab874e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7808c00640 1 -- 172.21.15.86:0/1308959027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de9a602280 msgr2=0x55de9ab86a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:13.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7808c00640 1 --2- 172.21.15.86:0/1308959027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de9a602280 0x55de9ab86a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7808c00640 1 -- 172.21.15.86:0/1308959027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de9ab3e430 con 0x55de9aa09ea0 2024-04-25T14:07:13.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f780a000640 1 --2- 172.21.15.86:0/1308959027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55de9aa1dc20 0x55de9ab874e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:13.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7808c00640 1 --2- 172.21.15.86:0/1308959027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de9aa09ea0 0x55de9ab86fa0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f77f005b370 tx=0x7f77f008f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c091beac420435 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:13.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f77ea000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:07:13.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f77ea000640 1 -- 172.21.15.86:0/1308959027 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0057900 con 0x55de9aa09ea0 2024-04-25T14:07:13.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f77ea000640 1 -- 172.21.15.86:0/1308959027 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f77f005a8d0 con 0x55de9aa09ea0 2024-04-25T14:07:13.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7813684000 1 -- 172.21.15.86:0/1308959027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77f8019040 con 0x55de9aa09ea0 2024-04-25T14:07:13.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f77ea000640 1 -- 172.21.15.86:0/1308959027 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f77f0057aa0 con 0x55de9aa09ea0 2024-04-25T14:07:13.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7813684000 10 client.6915 fetch_fsmap learned FSMap version 56 2024-04-25T14:07:13.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7813684000 1 -- 172.21.15.86:0/1308959027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x55de9a939c70 con 0x55de9aa09ea0 2024-04-25T14:07:13.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f77ea000640 1 -- 172.21.15.86:0/1308959027 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77f8019040 con 0x55de9aa09ea0 2024-04-25T14:07:13.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f77ea000640 1 -- 172.21.15.86:0/1308959027 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 56) ==== 3514+0+0 (secure 0 0 0) 0x7f77f0022060 con 0x55de9aa09ea0 2024-04-25T14:07:13.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7813684000 10 client.6915 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T14:07:13.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7813684000 10 client.6915 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T14:07:13.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7813684000 20 client.6915 populate_metadata read hostname 'smithi086' 2024-04-25T14:07:13.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7813684000 1 --2- 172.21.15.86:0/1308959027 >> [v2:172.21.15.86:6834/2682115891,v1:172.21.15.86:6835/2682115891] conn(0x55de9a614830 0x55de9abf5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:13.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7813684000 4 client.6915 mds_command: new command op to 6600 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:07:13.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7813684000 1 -- 172.21.15.86:0/1308959027 --> [v2:172.21.15.86:6834/2682115891,v1:172.21.15.86:6835/2682115891] -- command(tid 0: {"prefix": "status"}) -- 0x55de9a98ea30 con 0x55de9a614830 2024-04-25T14:07:13.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7809600640 1 --2- 172.21.15.86:0/1308959027 >> [v2:172.21.15.86:6834/2682115891,v1:172.21.15.86:6835/2682115891] conn(0x55de9a614830 0x55de9abf5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:13.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7809600640 1 --2- 172.21.15.86:0/1308959027 >> [v2:172.21.15.86:6834/2682115891,v1:172.21.15.86:6835/2682115891] conn(0x55de9a614830 0x55de9abf5fd0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=264322faea4f381a server_cookie=942e98772d7f5af0 in_seq=0 out_seq=0 2024-04-25T14:07:13.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f77ea000640 10 client.6915 ms_handle_connect on v2:172.21.15.86:6834/2682115891 2024-04-25T14:07:13.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f77ea000640 1 -- 172.21.15.86:0/1308959027 <== mds.1 v2:172.21.15.86:6834/2682115891 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55de9a98ea30 con 0x55de9a614830 2024-04-25T14:07:13.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f77ea000640 10 client.6915 handle_command_reply: tid=0 2024-04-25T14:07:13.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7813684000 1 client.6915 shutdown 2024-04-25T14:07:13.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7813684000 1 -- 172.21.15.86:0/1308959027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de9aa09ea0 msgr2=0x55de9ab86fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:13.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7813684000 1 --2- 172.21.15.86:0/1308959027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de9aa09ea0 0x55de9ab86fa0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f77f005b370 tx=0x7f77f008f040 comp rx=0 tx=0).stop 2024-04-25T14:07:13.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7813684000 1 -- 172.21.15.86:0/1308959027 shutdown_connections 2024-04-25T14:07:13.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7813684000 1 --2- 172.21.15.86:0/1308959027 >> [v2:172.21.15.86:6834/2682115891,v1:172.21.15.86:6835/2682115891] conn(0x55de9a614830 0x55de9abf5fd0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7813684000 1 --2- 172.21.15.86:0/1308959027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55de9aa1dc20 0x55de9ab874e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7813684000 1 --2- 172.21.15.86:0/1308959027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de9aa09ea0 0x55de9ab86fa0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7813684000 1 --2- 172.21.15.86:0/1308959027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de9a602280 0x55de9ab86a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f77ea000640 0 client.6915 ms_handle_reset on v2:172.21.15.86:6834/2682115891 2024-04-25T14:07:13.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7813684000 1 -- 172.21.15.86:0/1308959027 >> 172.21.15.86:0/1308959027 conn(0x55de9a641ff0 msgr2=0x55de9ab81040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:13.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7813684000 1 -- 172.21.15.86:0/1308959027 shutdown_connections 2024-04-25T14:07:13.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7813684000 1 -- 172.21.15.86:0/1308959027 wait complete. 2024-04-25T14:07:13.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f7813684000 20 client.6915 trim_cache size 0 max 16384 2024-04-25T14:07:13.942 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:07:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb", 2024-04-25T14:07:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 1, 2024-04-25T14:07:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 6600, 2024-04-25T14:07:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T14:07:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T14:07:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T14:07:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 2.8017821980000002, 2024-04-25T14:07:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 56, 2024-04-25T14:07:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 75, 2024-04-25T14:07:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 75, 2024-04-25T14:07:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 34.24450367 2024-04-25T14:07:13.943 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:07:13.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f780aa00640 1 -- 172.21.15.86:0/3049447809 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f77e406f5a0 msgr2=0x7f77e4071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:13.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f780aa00640 1 --2- 172.21.15.86:0/3049447809 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f77e406f5a0 0x7f77e4071ab0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f78040c6ee0 tx=0x7f77f8008040 comp rx=0 tx=0).stop 2024-04-25T14:07:13.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f780aa00640 1 -- 172.21.15.86:0/3049447809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78040b7480 msgr2=0x7f78041549c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:13.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.935+0000 7f780aa00640 1 --2- 172.21.15.86:0/3049447809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78040b7480 0x7f78041549c0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f77f000b410 tx=0x7f77f000b8d0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.939+0000 7f780aa00640 1 -- 172.21.15.86:0/3049447809 shutdown_connections 2024-04-25T14:07:13.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.939+0000 7f780aa00640 1 --2- 172.21.15.86:0/3049447809 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f77e406f5a0 0x7f77e4071ab0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.939+0000 7f780aa00640 1 --2- 172.21.15.86:0/3049447809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78040d1c70 0x7f780406d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.939+0000 7f780aa00640 1 --2- 172.21.15.86:0/3049447809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78040b7480 0x7f78041549c0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.939+0000 7f780aa00640 1 --2- 172.21.15.86:0/3049447809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7804070e70 0x7f7804157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:13.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.939+0000 7f780aa00640 1 -- 172.21.15.86:0/3049447809 >> 172.21.15.86:0/3049447809 conn(0x7f78040cfea0 msgr2=0x7f78040b80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:13.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.939+0000 7f780aa00640 1 -- 172.21.15.86:0/3049447809 shutdown_connections 2024-04-25T14:07:13.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:13.939+0000 7f780aa00640 1 -- 172.21.15.86:0/3049447809 wait complete. 2024-04-25T14:07:13.958 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T14:07:14.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.103+0000 7f2643200640 1 Processor -- start 2024-04-25T14:07:14.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.103+0000 7f2643200640 1 -- start start 2024-04-25T14:07:14.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.107+0000 7f2643200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f263c070330 0x7f263c0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:14.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.107+0000 7f2643200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f263c0ce790 0x7f263c06f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:14.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.107+0000 7f2643200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f263c0d2030 con 0x7f263c0d1c20 2024-04-25T14:07:14.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.107+0000 7f2643200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f263c15e5c0 con 0x7f263c070330 2024-04-25T14:07:14.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.107+0000 7f2643200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f263c15e8a0 con 0x7f263c0ce790 2024-04-25T14:07:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.107+0000 7f2641200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f263c0ce790 0x7f263c06f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.107+0000 7f263be00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f263c070330 0x7f263c0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.107+0000 7f2633e00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f263c0d1c20 0x7f263c06ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:34830/0 (socket says 172.21.15.86:34830) 2024-04-25T14:07:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.107+0000 7f2641200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f263c0ce790 0x7f263c06f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:54910/0 (socket says 172.21.15.86:54910) 2024-04-25T14:07:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.107+0000 7f2633e00640 1 -- 172.21.15.86:0/730759829 learned_addr learned my addr 172.21.15.86:0/730759829 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:07:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.107+0000 7f2641200640 1 -- 172.21.15.86:0/730759829 >> v2:172.21.15.155:3300/0 conn(0x7f263c070330 msgr2=0x7f263c0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.107+0000 7f2641200640 1 --2- 172.21.15.86:0/730759829 >> v2:172.21.15.155:3300/0 conn(0x7f263c070330 0x7f263c0d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.107+0000 7f2641200640 1 -- 172.21.15.86:0/730759829 >> v1:172.21.15.155:6789/0 conn(0x7f263c0d1c20 legacy=0x7f263c06ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.107+0000 7f2641200640 1 -- 172.21.15.86:0/730759829 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263c15eb80 con 0x7f263c0ce790 2024-04-25T14:07:14.110 INFO:tasks.ceph.mds.a.smithi086.stderr:2024-04-25T14:07:14.107+0000 7f4bf8000640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-25T14:07:14.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.111+0000 7f2641200640 1 --2- 172.21.15.86:0/730759829 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f263c0ce790 0x7f263c06f440 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2634009d90 tx=0x7f263401c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41c72aa66e66900d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:14.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.111+0000 7f263b400640 1 -- 172.21.15.86:0/730759829 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26340b2070 con 0x7f263c0ce790 2024-04-25T14:07:14.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.111+0000 7f263b400640 1 -- 172.21.15.86:0/730759829 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26340a57a0 con 0x7f263c0ce790 2024-04-25T14:07:14.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.111+0000 7f263b400640 1 -- 172.21.15.86:0/730759829 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26340a5b70 con 0x7f263c0ce790 2024-04-25T14:07:14.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.111+0000 7f2643200640 1 -- 172.21.15.86:0/730759829 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f263c0ce790 msgr2=0x7f263c06f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:14.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.111+0000 7f2643200640 1 --2- 172.21.15.86:0/730759829 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f263c0ce790 0x7f263c06f440 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2634009d90 tx=0x7f263401c8e0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.111+0000 7f2643200640 1 -- 172.21.15.86:0/730759829 shutdown_connections 2024-04-25T14:07:14.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.111+0000 7f2643200640 1 --2- 172.21.15.86:0/730759829 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f263c0ce790 0x7f263c06f440 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.111+0000 7f2643200640 1 --2- 172.21.15.86:0/730759829 >> v2:172.21.15.155:3300/0 conn(0x7f263c070330 0x7f263c0d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.111+0000 7f2643200640 1 -- 172.21.15.86:0/730759829 >> 172.21.15.86:0/730759829 conn(0x7f263c0b4650 msgr2=0x7f263c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:14.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.111+0000 7f2643200640 1 -- 172.21.15.86:0/730759829 shutdown_connections 2024-04-25T14:07:14.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.111+0000 7f2643200640 1 -- 172.21.15.86:0/730759829 wait complete. 2024-04-25T14:07:14.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.115+0000 7f2643200640 1 Processor -- start 2024-04-25T14:07:14.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.115+0000 7f2643200640 1 -- start start 2024-04-25T14:07:14.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.115+0000 7f2643200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f263c070330 0x7f263c159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:14.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.115+0000 7f2643200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f263c0ce790 0x7f263c0ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:14.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.115+0000 7f2643200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f263c0d1c20 0x7f263c0ca820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:14.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.115+0000 7f2643200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f263c0708c0 con 0x7f263c0ce790 2024-04-25T14:07:14.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.115+0000 7f2643200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f263c070740 con 0x7f263c070330 2024-04-25T14:07:14.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.115+0000 7f2643200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f263c16afc0 con 0x7f263c0d1c20 2024-04-25T14:07:14.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.115+0000 7f2633e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f263c0ce790 0x7f263c0ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:14.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.115+0000 7f2633e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f263c0ce790 0x7f263c0ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37894/0 (socket says 172.21.15.86:37894) 2024-04-25T14:07:14.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.115+0000 7f2633e00640 1 -- 172.21.15.86:0/2755494389 learned_addr learned my addr 172.21.15.86:0/2755494389 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:14.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.115+0000 7f2641200640 1 --2- 172.21.15.86:0/2755494389 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f263c0d1c20 0x7f263c0ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:14.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.115+0000 7f263be00640 1 --2- 172.21.15.86:0/2755494389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f263c070330 0x7f263c159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:14.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.115+0000 7f2633e00640 1 -- 172.21.15.86:0/2755494389 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f263c0d1c20 msgr2=0x7f263c0ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:14.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.115+0000 7f2633e00640 1 --2- 172.21.15.86:0/2755494389 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f263c0d1c20 0x7f263c0ca820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.115+0000 7f2633e00640 1 -- 172.21.15.86:0/2755494389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f263c070330 msgr2=0x7f263c159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:14.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.115+0000 7f2633e00640 1 --2- 172.21.15.86:0/2755494389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f263c070330 0x7f263c159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.115+0000 7f2633e00640 1 -- 172.21.15.86:0/2755494389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263c159690 con 0x7f263c0ce790 2024-04-25T14:07:14.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.119+0000 7f2633e00640 1 --2- 172.21.15.86:0/2755494389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f263c0ce790 0x7f263c0ca2e0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f262400c3d0 tx=0x7f262400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:14.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.119+0000 7f2638c00640 1 -- 172.21.15.86:0/2755494389 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2624017070 con 0x7f263c0ce790 2024-04-25T14:07:14.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.119+0000 7f2638c00640 1 -- 172.21.15.86:0/2755494389 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2624005cf0 con 0x7f263c0ce790 2024-04-25T14:07:14.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.119+0000 7f2643200640 1 -- 172.21.15.86:0/2755494389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f263c0743b0 con 0x7f263c0ce790 2024-04-25T14:07:14.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.119+0000 7f2638c00640 1 -- 172.21.15.86:0/2755494389 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26240072d0 con 0x7f263c0ce790 2024-04-25T14:07:14.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.119+0000 7f2643200640 1 -- 172.21.15.86:0/2755494389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f263c0af620 con 0x7f263c0ce790 2024-04-25T14:07:14.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.119+0000 7f2638c00640 1 -- 172.21.15.86:0/2755494389 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7f262401a070 con 0x7f263c0ce790 2024-04-25T14:07:14.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.119+0000 7f2638c00640 1 --2- 172.21.15.86:0/2755494389 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f261806f5a0 0x7f2618071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:14.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.123+0000 7f2638c00640 1 -- 172.21.15.86:0/2755494389 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f262408e9c0 con 0x7f263c0ce790 2024-04-25T14:07:14.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.123+0000 7f263be00640 1 --2- 172.21.15.86:0/2755494389 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f261806f5a0 0x7f2618071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:14.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.123+0000 7f264a7e2000 1 Processor -- start 2024-04-25T14:07:14.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.123+0000 7f264a7e2000 1 -- start start 2024-04-25T14:07:14.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.123+0000 7f263be00640 1 --2- 172.21.15.86:0/2755494389 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f261806f5a0 0x7f2618071ab0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f263c072200 tx=0x7f262c009190 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:14.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.123+0000 7f264a7e2000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b162cc20 0x55f4b178f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:14.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.123+0000 7f264a7e2000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f4b1618ea0 0x55f4b178f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:14.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.123+0000 7f264a7e2000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4b1211280 0x55f4b178fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:14.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.123+0000 7f264a7e2000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f4b15fa160 con 0x55f4b162cc20 2024-04-25T14:07:14.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.123+0000 7f264a7e2000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f4b158fe20 con 0x55f4b1211280 2024-04-25T14:07:14.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.123+0000 7f264a7e2000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55f4b1548a20 con 0x55f4b1618ea0 2024-04-25T14:07:14.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.123+0000 7f263be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b162cc20 0x55f4b178f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:14.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.123+0000 7f263be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b162cc20 0x55f4b178f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37902/0 (socket says 172.21.15.86:37902) 2024-04-25T14:07:14.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.123+0000 7f263be00640 1 -- 172.21.15.86:0/1953449809 learned_addr learned my addr 172.21.15.86:0/1953449809 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:14.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.123+0000 7f2641200640 1 --2- 172.21.15.86:0/1953449809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4b1211280 0x55f4b178fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:14.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.123+0000 7f2633e00640 1 --2- 172.21.15.86:0/1953449809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f4b1618ea0 0x55f4b178f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:14.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.123+0000 7f263be00640 1 -- 172.21.15.86:0/1953449809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f4b1618ea0 msgr2=0x55f4b178f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:14.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.123+0000 7f263be00640 1 --2- 172.21.15.86:0/1953449809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f4b1618ea0 0x55f4b178f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.123+0000 7f263be00640 1 -- 172.21.15.86:0/1953449809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4b1211280 msgr2=0x55f4b178fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:14.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.123+0000 7f263be00640 1 --2- 172.21.15.86:0/1953449809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4b1211280 0x55f4b178fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.123+0000 7f263be00640 1 -- 172.21.15.86:0/1953449809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4b154d100 con 0x55f4b162cc20 2024-04-25T14:07:14.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.123+0000 7f2641200640 1 --2- 172.21.15.86:0/1953449809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4b1211280 0x55f4b178fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:14.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.123+0000 7f263be00640 1 --2- 172.21.15.86:0/1953449809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b162cc20 0x55f4b178f340 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f262c009fd0 tx=0x7f262c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03b630394d8cf98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:14.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.123+0000 7f2630c00640 1 -- 172.21.15.86:0/1953449809 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c025070 con 0x55f4b162cc20 2024-04-25T14:07:14.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.123+0000 7f2630c00640 1 -- 172.21.15.86:0/1953449809 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f262c01b070 con 0x55f4b162cc20 2024-04-25T14:07:14.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.123+0000 7f2630c00640 1 -- 172.21.15.86:0/1953449809 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c014da0 con 0x55f4b162cc20 2024-04-25T14:07:14.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f264a7e2000 1 -- 172.21.15.86:0/1953449809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b162cc20 msgr2=0x55f4b178f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:14.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f264a7e2000 1 --2- 172.21.15.86:0/1953449809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b162cc20 0x55f4b178f340 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f262c009fd0 tx=0x7f262c014000 comp rx=0 tx=0).stop 2024-04-25T14:07:14.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f264a7e2000 1 -- 172.21.15.86:0/1953449809 shutdown_connections 2024-04-25T14:07:14.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f264a7e2000 1 --2- 172.21.15.86:0/1953449809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4b1211280 0x55f4b178fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f264a7e2000 1 --2- 172.21.15.86:0/1953449809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f4b1618ea0 0x55f4b178f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f264a7e2000 1 --2- 172.21.15.86:0/1953449809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b162cc20 0x55f4b178f340 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f264a7e2000 1 -- 172.21.15.86:0/1953449809 >> 172.21.15.86:0/1953449809 conn(0x55f4b1250ff0 msgr2=0x55f4b17448d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:14.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f264a7e2000 1 -- 172.21.15.86:0/1953449809 shutdown_connections 2024-04-25T14:07:14.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f264a7e2000 1 -- 172.21.15.86:0/1953449809 wait complete. 2024-04-25T14:07:14.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f264a7e2000 1 -- start start 2024-04-25T14:07:14.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f264a7e2000 1 Processor -- start 2024-04-25T14:07:14.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f264a7e2000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f4b1211280 0x55f4b17947d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:14.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f264a7e2000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b1618ea0 0x55f4b1794d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:14.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f264a7e2000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4b162cc20 0x55f4b174f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:14.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f264a7e2000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f4b159d220 con 0x55f4b1618ea0 2024-04-25T14:07:14.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f264a7e2000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f4b1533ea0 con 0x55f4b162cc20 2024-04-25T14:07:14.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f264a7e2000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55f4b158ab00 con 0x55f4b1211280 2024-04-25T14:07:14.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f264a7e2000 10 client.0 _ll_register_callbacks cb 0x55f4b159cdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:14.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f2633e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b1618ea0 0x55f4b1794d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:14.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f2633e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b1618ea0 0x55f4b1794d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37918/0 (socket says 172.21.15.86:37918) 2024-04-25T14:07:14.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f2633e00640 1 -- 172.21.15.86:0/1595143570 learned_addr learned my addr 172.21.15.86:0/1595143570 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:14.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f263be00640 1 --2- 172.21.15.86:0/1595143570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f4b1211280 0x55f4b17947d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:14.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f2641200640 1 --2- 172.21.15.86:0/1595143570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4b162cc20 0x55f4b174f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:14.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f2633e00640 1 -- 172.21.15.86:0/1595143570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f4b1211280 msgr2=0x55f4b17947d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:14.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f2633e00640 1 --2- 172.21.15.86:0/1595143570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f4b1211280 0x55f4b17947d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f2633e00640 1 -- 172.21.15.86:0/1595143570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4b162cc20 msgr2=0x55f4b174f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:14.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f2633e00640 1 --2- 172.21.15.86:0/1595143570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4b162cc20 0x55f4b174f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f2633e00640 1 -- 172.21.15.86:0/1595143570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4b152f500 con 0x55f4b1618ea0 2024-04-25T14:07:14.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.127+0000 7f2641200640 1 --2- 172.21.15.86:0/1595143570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4b162cc20 0x55f4b174f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:14.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f2633e00640 1 --2- 172.21.15.86:0/1595143570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b1618ea0 0x55f4b1794d10 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f262405ae30 tx=0x7f262405b440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b39949495aef8b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:14.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f2629600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:07:14.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f2629600640 1 -- 172.21.15.86:0/1595143570 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262405aea0 con 0x55f4b1618ea0 2024-04-25T14:07:14.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f2629600640 1 -- 172.21.15.86:0/1595143570 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f262405a100 con 0x55f4b1618ea0 2024-04-25T14:07:14.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f264a7e2000 1 -- 172.21.15.86:0/1595143570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f262c01b070 con 0x55f4b1618ea0 2024-04-25T14:07:14.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f2629600640 1 -- 172.21.15.86:0/1595143570 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f262405f1a0 con 0x55f4b1618ea0 2024-04-25T14:07:14.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f264a7e2000 10 client.6924 fetch_fsmap learned FSMap version 56 2024-04-25T14:07:14.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f264a7e2000 1 -- 172.21.15.86:0/1595143570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x55f4b1515ea0 con 0x55f4b1618ea0 2024-04-25T14:07:14.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f2629600640 1 -- 172.21.15.86:0/1595143570 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f262c01b070 con 0x55f4b1618ea0 2024-04-25T14:07:14.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f2629600640 1 -- 172.21.15.86:0/1595143570 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 56) ==== 3514+0+0 (secure 0 0 0) 0x7f2624056810 con 0x55f4b1618ea0 2024-04-25T14:07:14.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f264a7e2000 10 client.6924 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T14:07:14.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f264a7e2000 10 client.6924 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T14:07:14.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f264a7e2000 20 client.6924 populate_metadata read hostname 'smithi086' 2024-04-25T14:07:14.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f264a7e2000 1 --2- 172.21.15.86:0/1595143570 >> [v2:172.21.15.86:6836/1088108173,v1:172.21.15.86:6837/1088108173] conn(0x55f4b12357d0 0x55f4b17501a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:14.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f264a7e2000 4 client.6924 mds_command: new command op to 6603 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:07:14.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f264a7e2000 1 -- 172.21.15.86:0/1595143570 --> [v2:172.21.15.86:6836/1088108173,v1:172.21.15.86:6837/1088108173] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4b1528be0 con 0x55f4b12357d0 2024-04-25T14:07:14.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f263be00640 1 --2- 172.21.15.86:0/1595143570 >> [v2:172.21.15.86:6836/1088108173,v1:172.21.15.86:6837/1088108173] conn(0x55f4b12357d0 0x55f4b17501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:14.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f2629600640 10 client.6924 ms_handle_connect on v2:172.21.15.86:6836/1088108173 2024-04-25T14:07:14.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f263be00640 1 --2- 172.21.15.86:0/1595143570 >> [v2:172.21.15.86:6836/1088108173,v1:172.21.15.86:6837/1088108173] conn(0x55f4b12357d0 0x55f4b17501a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c14b808e029f29cf server_cookie=a859bbdd95156067 in_seq=0 out_seq=0 2024-04-25T14:07:14.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f2629600640 1 -- 172.21.15.86:0/1595143570 <== mds.0 v2:172.21.15.86:6836/1088108173 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f4b1528be0 con 0x55f4b12357d0 2024-04-25T14:07:14.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f2629600640 10 client.6924 handle_command_reply: tid=0 2024-04-25T14:07:14.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f264a7e2000 1 client.6924 shutdown 2024-04-25T14:07:14.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f264a7e2000 1 -- 172.21.15.86:0/1595143570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b1618ea0 msgr2=0x55f4b1794d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:14.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f264a7e2000 1 --2- 172.21.15.86:0/1595143570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b1618ea0 0x55f4b1794d10 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f262405ae30 tx=0x7f262405b440 comp rx=0 tx=0).stop 2024-04-25T14:07:14.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f264a7e2000 1 -- 172.21.15.86:0/1595143570 shutdown_connections 2024-04-25T14:07:14.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f264a7e2000 1 --2- 172.21.15.86:0/1595143570 >> [v2:172.21.15.86:6836/1088108173,v1:172.21.15.86:6837/1088108173] conn(0x55f4b12357d0 0x55f4b17501a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f264a7e2000 1 --2- 172.21.15.86:0/1595143570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4b162cc20 0x55f4b174f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f264a7e2000 1 --2- 172.21.15.86:0/1595143570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b1618ea0 0x55f4b1794d10 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f2629600640 0 client.6924 ms_handle_reset on v2:172.21.15.86:6836/1088108173 2024-04-25T14:07:14.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f264a7e2000 1 --2- 172.21.15.86:0/1595143570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f4b1211280 0x55f4b17947d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f264a7e2000 1 -- 172.21.15.86:0/1595143570 >> 172.21.15.86:0/1595143570 conn(0x55f4b1250ff0 msgr2=0x55f4b1745300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:14.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f264a7e2000 1 -- 172.21.15.86:0/1595143570 shutdown_connections 2024-04-25T14:07:14.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f264a7e2000 1 -- 172.21.15.86:0/1595143570 wait complete. 2024-04-25T14:07:14.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.131+0000 7f264a7e2000 20 client.6924 trim_cache size 0 max 16384 2024-04-25T14:07:14.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.151+0000 7f264a7e2000 1 Processor -- start 2024-04-25T14:07:14.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.151+0000 7f264a7e2000 1 -- start start 2024-04-25T14:07:14.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.151+0000 7f264a7e2000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f4b1211280 0x55f4b17fcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:14.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.151+0000 7f264a7e2000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4b1618ea0 0x55f4b17fd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:14.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.151+0000 7f264a7e2000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b162cc20 0x55f4b17fd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:14.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.151+0000 7f264a7e2000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f263c202500 con 0x55f4b162cc20 2024-04-25T14:07:14.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.151+0000 7f264a7e2000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f263c0aecd0 con 0x55f4b1618ea0 2024-04-25T14:07:14.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.151+0000 7f264a7e2000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f263c0af320 con 0x55f4b1211280 2024-04-25T14:07:14.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.155+0000 7f2641200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b162cc20 0x55f4b17fd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:14.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.155+0000 7f263be00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f4b1211280 0x55f4b17fcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:14.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.155+0000 7f2641200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b162cc20 0x55f4b17fd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37930/0 (socket says 172.21.15.86:37930) 2024-04-25T14:07:14.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.155+0000 7f2641200640 1 -- 172.21.15.86:0/2145659786 learned_addr learned my addr 172.21.15.86:0/2145659786 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:14.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.155+0000 7f2633e00640 1 --2- 172.21.15.86:0/2145659786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4b1618ea0 0x55f4b17fd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:14.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.155+0000 7f2641200640 1 -- 172.21.15.86:0/2145659786 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f4b1211280 msgr2=0x55f4b17fcaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:14.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.155+0000 7f2641200640 1 --2- 172.21.15.86:0/2145659786 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f4b1211280 0x55f4b17fcaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.155+0000 7f2641200640 1 -- 172.21.15.86:0/2145659786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4b1618ea0 msgr2=0x55f4b17fd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:14.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.155+0000 7f2641200640 1 --2- 172.21.15.86:0/2145659786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4b1618ea0 0x55f4b17fd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.155+0000 7f2641200640 1 -- 172.21.15.86:0/2145659786 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4b1548c70 con 0x55f4b162cc20 2024-04-25T14:07:14.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.155+0000 7f2641200640 1 --2- 172.21.15.86:0/2145659786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b162cc20 0x55f4b17fd570 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f2634009d90 tx=0x7f2634004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf9be76aaa7f77aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:14.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.155+0000 7f262be00640 1 -- 172.21.15.86:0/2145659786 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26340b2040 con 0x55f4b162cc20 2024-04-25T14:07:14.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.155+0000 7f262be00640 1 -- 172.21.15.86:0/2145659786 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26340b3040 con 0x55f4b162cc20 2024-04-25T14:07:14.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.155+0000 7f262be00640 1 -- 172.21.15.86:0/2145659786 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26340a59a0 con 0x55f4b162cc20 2024-04-25T14:07:14.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.155+0000 7f264a7e2000 1 -- 172.21.15.86:0/2145659786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b162cc20 msgr2=0x55f4b17fd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:14.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.155+0000 7f264a7e2000 1 --2- 172.21.15.86:0/2145659786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b162cc20 0x55f4b17fd570 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f2634009d90 tx=0x7f2634004ac0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.155+0000 7f264a7e2000 1 -- 172.21.15.86:0/2145659786 shutdown_connections 2024-04-25T14:07:14.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.155+0000 7f264a7e2000 1 --2- 172.21.15.86:0/2145659786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b162cc20 0x55f4b17fd570 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.155+0000 7f264a7e2000 1 --2- 172.21.15.86:0/2145659786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4b1618ea0 0x55f4b17fd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.155+0000 7f264a7e2000 1 --2- 172.21.15.86:0/2145659786 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f4b1211280 0x55f4b17fcaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.155+0000 7f264a7e2000 1 -- 172.21.15.86:0/2145659786 >> 172.21.15.86:0/2145659786 conn(0x55f4b1250ff0 msgr2=0x55f4b17a4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:14.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.155+0000 7f264a7e2000 1 -- 172.21.15.86:0/2145659786 shutdown_connections 2024-04-25T14:07:14.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.155+0000 7f264a7e2000 1 -- 172.21.15.86:0/2145659786 wait complete. 2024-04-25T14:07:14.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.155+0000 7f264a7e2000 1 -- start start 2024-04-25T14:07:14.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.155+0000 7f264a7e2000 1 Processor -- start 2024-04-25T14:07:14.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f264a7e2000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b1211280 0x55f4b1800c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:14.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f264a7e2000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4b1618ea0 0x55f4b18011d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:14.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f264a7e2000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f4b162cc20 0x55f4b17bc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:14.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f264a7e2000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f4b158f130 con 0x55f4b1211280 2024-04-25T14:07:14.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f264a7e2000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f4b15257d0 con 0x55f4b1618ea0 2024-04-25T14:07:14.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f264a7e2000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55f4b1594450 con 0x55f4b162cc20 2024-04-25T14:07:14.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f264a7e2000 10 client.0 _ll_register_callbacks cb 0x7f26340b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:14.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f263be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b1211280 0x55f4b1800c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:14.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f263be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b1211280 0x55f4b1800c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37934/0 (socket says 172.21.15.86:37934) 2024-04-25T14:07:14.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f263be00640 1 -- 172.21.15.86:0/3553312519 learned_addr learned my addr 172.21.15.86:0/3553312519 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:14.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f2633e00640 1 --2- 172.21.15.86:0/3553312519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4b1618ea0 0x55f4b18011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:14.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f2641200640 1 --2- 172.21.15.86:0/3553312519 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f4b162cc20 0x55f4b17bc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:14.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f263be00640 1 -- 172.21.15.86:0/3553312519 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f4b162cc20 msgr2=0x55f4b17bc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:14.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f263be00640 1 --2- 172.21.15.86:0/3553312519 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f4b162cc20 0x55f4b17bc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f263be00640 1 -- 172.21.15.86:0/3553312519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4b1618ea0 msgr2=0x55f4b18011d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:14.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f263be00640 1 --2- 172.21.15.86:0/3553312519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4b1618ea0 0x55f4b18011d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f263be00640 1 -- 172.21.15.86:0/3553312519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4b1612180 con 0x55f4b1211280 2024-04-25T14:07:14.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f263be00640 1 --2- 172.21.15.86:0/3553312519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b1211280 0x55f4b1800c90 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f262c011dd0 tx=0x7f262c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64349a8ee24d47b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:14.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f2629600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:07:14.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f2629600640 1 -- 172.21.15.86:0/3553312519 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c038060 con 0x55f4b1211280 2024-04-25T14:07:14.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f2629600640 1 -- 172.21.15.86:0/3553312519 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f262c005ba0 con 0x55f4b1211280 2024-04-25T14:07:14.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f264a7e2000 1 -- 172.21.15.86:0/3553312519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26340b3040 con 0x55f4b1211280 2024-04-25T14:07:14.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f2629600640 1 -- 172.21.15.86:0/3553312519 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f262c03e050 con 0x55f4b1211280 2024-04-25T14:07:14.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f264a7e2000 10 client.6930 fetch_fsmap learned FSMap version 56 2024-04-25T14:07:14.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f264a7e2000 1 -- 172.21.15.86:0/3553312519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x55f4b12490c0 con 0x55f4b1211280 2024-04-25T14:07:14.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f2629600640 1 -- 172.21.15.86:0/3553312519 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26340b3040 con 0x55f4b1211280 2024-04-25T14:07:14.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f2629600640 1 -- 172.21.15.86:0/3553312519 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 56) ==== 3514+0+0 (secure 0 0 0) 0x7f262c041020 con 0x55f4b1211280 2024-04-25T14:07:14.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f264a7e2000 10 client.6930 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T14:07:14.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f264a7e2000 10 client.6930 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T14:07:14.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f264a7e2000 20 client.6930 populate_metadata read hostname 'smithi086' 2024-04-25T14:07:14.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f264a7e2000 1 --2- 172.21.15.86:0/3553312519 >> [v2:172.21.15.86:6836/1088108173,v1:172.21.15.86:6837/1088108173] conn(0x55f4b1223830 0x55f4b1809b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:14.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f264a7e2000 4 client.6930 mds_command: new command op to 6603 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:07:14.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f264a7e2000 1 -- 172.21.15.86:0/3553312519 --> [v2:172.21.15.86:6836/1088108173,v1:172.21.15.86:6837/1088108173] -- command(tid 0: {"prefix": "status"}) -- 0x55f4b159da30 con 0x55f4b1223830 2024-04-25T14:07:14.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f2633e00640 1 --2- 172.21.15.86:0/3553312519 >> [v2:172.21.15.86:6836/1088108173,v1:172.21.15.86:6837/1088108173] conn(0x55f4b1223830 0x55f4b1809b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:14.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f2633e00640 1 --2- 172.21.15.86:0/3553312519 >> [v2:172.21.15.86:6836/1088108173,v1:172.21.15.86:6837/1088108173] conn(0x55f4b1223830 0x55f4b1809b00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6dd00c5b34af81f server_cookie=5bb324ad2d8ec750 in_seq=0 out_seq=0 2024-04-25T14:07:14.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f2629600640 10 client.6930 ms_handle_connect on v2:172.21.15.86:6836/1088108173 2024-04-25T14:07:14.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f2629600640 1 -- 172.21.15.86:0/3553312519 <== mds.0 v2:172.21.15.86:6836/1088108173 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55f4b159da30 con 0x55f4b1223830 2024-04-25T14:07:14.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f2629600640 10 client.6930 handle_command_reply: tid=0 2024-04-25T14:07:14.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.159+0000 7f264a7e2000 1 client.6930 shutdown 2024-04-25T14:07:14.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.163+0000 7f264a7e2000 1 -- 172.21.15.86:0/3553312519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b1211280 msgr2=0x55f4b1800c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:14.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.163+0000 7f264a7e2000 1 --2- 172.21.15.86:0/3553312519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b1211280 0x55f4b1800c90 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f262c011dd0 tx=0x7f262c019000 comp rx=0 tx=0).stop 2024-04-25T14:07:14.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.163+0000 7f264a7e2000 1 -- 172.21.15.86:0/3553312519 shutdown_connections 2024-04-25T14:07:14.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.163+0000 7f264a7e2000 1 --2- 172.21.15.86:0/3553312519 >> [v2:172.21.15.86:6836/1088108173,v1:172.21.15.86:6837/1088108173] conn(0x55f4b1223830 0x55f4b1809b00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.163+0000 7f264a7e2000 1 --2- 172.21.15.86:0/3553312519 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f4b162cc20 0x55f4b17bc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.163+0000 7f264a7e2000 1 --2- 172.21.15.86:0/3553312519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4b1618ea0 0x55f4b18011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.163+0000 7f264a7e2000 1 --2- 172.21.15.86:0/3553312519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b1211280 0x55f4b1800c90 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.163+0000 7f264a7e2000 1 -- 172.21.15.86:0/3553312519 >> 172.21.15.86:0/3553312519 conn(0x55f4b1250ff0 msgr2=0x55f4b17448d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:14.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.163+0000 7f2629600640 0 client.6930 ms_handle_reset on v2:172.21.15.86:6836/1088108173 2024-04-25T14:07:14.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.163+0000 7f264a7e2000 1 -- 172.21.15.86:0/3553312519 shutdown_connections 2024-04-25T14:07:14.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.163+0000 7f264a7e2000 1 -- 172.21.15.86:0/3553312519 wait complete. 2024-04-25T14:07:14.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.163+0000 7f264a7e2000 20 client.6930 trim_cache size 0 max 16384 2024-04-25T14:07:14.171 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:07:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb", 2024-04-25T14:07:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-25T14:07:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 6603, 2024-04-25T14:07:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T14:07:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T14:07:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T14:07:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 28.621445531999999, 2024-04-25T14:07:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 56, 2024-04-25T14:07:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 75, 2024-04-25T14:07:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 75, 2024-04-25T14:07:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 34.466538104999998 2024-04-25T14:07:14.172 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:07:14.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.163+0000 7f2643200640 1 -- 172.21.15.86:0/2755494389 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f261806f5a0 msgr2=0x7f2618071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:14.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.163+0000 7f2643200640 1 --2- 172.21.15.86:0/2755494389 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f261806f5a0 0x7f2618071ab0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f263c072200 tx=0x7f262c009190 comp rx=0 tx=0).stop 2024-04-25T14:07:14.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.163+0000 7f2643200640 1 -- 172.21.15.86:0/2755494389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f263c0ce790 msgr2=0x7f263c0ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:14.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.163+0000 7f2643200640 1 --2- 172.21.15.86:0/2755494389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f263c0ce790 0x7f263c0ca2e0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f262400c3d0 tx=0x7f262400c890 comp rx=0 tx=0).stop 2024-04-25T14:07:14.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.163+0000 7f2643200640 1 -- 172.21.15.86:0/2755494389 shutdown_connections 2024-04-25T14:07:14.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.163+0000 7f2643200640 1 --2- 172.21.15.86:0/2755494389 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f261806f5a0 0x7f2618071ab0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.163+0000 7f2643200640 1 --2- 172.21.15.86:0/2755494389 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f263c0d1c20 0x7f263c0ca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.163+0000 7f2643200640 1 --2- 172.21.15.86:0/2755494389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f263c0ce790 0x7f263c0ca2e0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.163+0000 7f2643200640 1 --2- 172.21.15.86:0/2755494389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f263c070330 0x7f263c159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.163+0000 7f2643200640 1 -- 172.21.15.86:0/2755494389 >> 172.21.15.86:0/2755494389 conn(0x7f263c0b4650 msgr2=0x7f263c15d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:14.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.163+0000 7f2643200640 1 -- 172.21.15.86:0/2755494389 shutdown_connections 2024-04-25T14:07:14.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.163+0000 7f2643200640 1 -- 172.21.15.86:0/2755494389 wait complete. 2024-04-25T14:07:14.183 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:07:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.327+0000 7f449cc00640 1 Processor -- start 2024-04-25T14:07:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.327+0000 7f449cc00640 1 -- start start 2024-04-25T14:07:14.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.331+0000 7f449cc00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f449815c2a0 0x7f44980ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:14.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.331+0000 7f449cc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f449815feb0 0x7f44981602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:14.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.331+0000 7f449cc00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44980b0a60 con 0x7f449806fd50 2024-04-25T14:07:14.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.331+0000 7f449cc00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f449806ef90 con 0x7f449815c2a0 2024-04-25T14:07:14.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.331+0000 7f449cc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f449806f270 con 0x7f449815feb0 2024-04-25T14:07:14.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.331+0000 7f448fe00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f449806fd50 0x7f4498070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:34836/0 (socket says 172.21.15.86:34836) 2024-04-25T14:07:14.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.331+0000 7f4496800640 1 --2- 172.21.15.86:0/1663427139 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f449815feb0 0x7f44981602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:14.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.331+0000 7f448fe00640 1 -- 172.21.15.86:0/1663427139 learned_addr learned my addr 172.21.15.86:0/1663427139 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:07:14.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.331+0000 7f4495e00640 1 --2- 172.21.15.86:0/1663427139 >> v2:172.21.15.155:3300/0 conn(0x7f449815c2a0 0x7f44980ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:14.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.331+0000 7f4495400640 1 -- 172.21.15.86:0/1663427139 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3581334353 0 0) 0x7f44980b0a60 con 0x7f449806fd50 2024-04-25T14:07:14.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.331+0000 7f4495400640 1 -- 172.21.15.86:0/1663427139 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f447c005740 con 0x7f449806fd50 2024-04-25T14:07:14.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.331+0000 7f4495e00640 1 -- 172.21.15.86:0/1663427139 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f449815feb0 msgr2=0x7f44981602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:14.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.331+0000 7f4495e00640 1 --2- 172.21.15.86:0/1663427139 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f449815feb0 0x7f44981602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.331+0000 7f4495e00640 1 -- 172.21.15.86:0/1663427139 >> v1:172.21.15.155:6789/0 conn(0x7f449806fd50 legacy=0x7f4498070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:14.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.331+0000 7f4495e00640 1 -- 172.21.15.86:0/1663427139 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f449806f550 con 0x7f449815c2a0 2024-04-25T14:07:14.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.331+0000 7f4496800640 1 --2- 172.21.15.86:0/1663427139 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f449815feb0 0x7f44981602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:07:14.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.335+0000 7f4495e00640 1 --2- 172.21.15.86:0/1663427139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f449815c2a0 0x7f44980ae130 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f44880097c0 tx=0x7f448801ce40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7313d047e750cb19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:14.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.335+0000 7f4495400640 1 -- 172.21.15.86:0/1663427139 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44880b2070 con 0x7f449815c2a0 2024-04-25T14:07:14.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.335+0000 7f4495400640 1 -- 172.21.15.86:0/1663427139 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44880ad680 con 0x7f449815c2a0 2024-04-25T14:07:14.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.335+0000 7f4495400640 1 -- 172.21.15.86:0/1663427139 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44880ada50 con 0x7f449815c2a0 2024-04-25T14:07:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.335+0000 7f449cc00640 1 -- 172.21.15.86:0/1663427139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f449815c2a0 msgr2=0x7f44980ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.335+0000 7f449cc00640 1 --2- 172.21.15.86:0/1663427139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f449815c2a0 0x7f44980ae130 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f44880097c0 tx=0x7f448801ce40 comp rx=0 tx=0).stop 2024-04-25T14:07:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.335+0000 7f449cc00640 1 -- 172.21.15.86:0/1663427139 shutdown_connections 2024-04-25T14:07:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.335+0000 7f449cc00640 1 --2- 172.21.15.86:0/1663427139 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f449815feb0 0x7f44981602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.335+0000 7f449cc00640 1 --2- 172.21.15.86:0/1663427139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f449815c2a0 0x7f44980ae130 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.335+0000 7f449cc00640 1 -- 172.21.15.86:0/1663427139 >> 172.21.15.86:0/1663427139 conn(0x7f44980b4650 msgr2=0x7f44980b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.335+0000 7f449cc00640 1 -- 172.21.15.86:0/1663427139 shutdown_connections 2024-04-25T14:07:14.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.335+0000 7f449cc00640 1 -- 172.21.15.86:0/1663427139 wait complete. 2024-04-25T14:07:14.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.339+0000 7f449cc00640 1 Processor -- start 2024-04-25T14:07:14.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.339+0000 7f449cc00640 1 -- start start 2024-04-25T14:07:14.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.339+0000 7f449cc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f449806fd50 0x7f44980b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:14.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.339+0000 7f449cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f449815c2a0 0x7f44981570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:14.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.339+0000 7f449cc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f449815feb0 0x7f44981575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:14.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.339+0000 7f449cc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4498160d90 con 0x7f449815c2a0 2024-04-25T14:07:14.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.339+0000 7f449cc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4498160c10 con 0x7f449815feb0 2024-04-25T14:07:14.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.339+0000 7f449cc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f44980ae670 con 0x7f449806fd50 2024-04-25T14:07:14.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.339+0000 7f448fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f449815c2a0 0x7f44981570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:14.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.339+0000 7f448fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f449815c2a0 0x7f44981570b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37944/0 (socket says 172.21.15.86:37944) 2024-04-25T14:07:14.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.339+0000 7f448fe00640 1 -- 172.21.15.86:0/1719805145 learned_addr learned my addr 172.21.15.86:0/1719805145 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:14.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.339+0000 7f4496800640 1 --2- 172.21.15.86:0/1719805145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f449815feb0 0x7f44981575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:14.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.339+0000 7f4495e00640 1 --2- 172.21.15.86:0/1719805145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f449806fd50 0x7f44980b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:14.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.339+0000 7f448fe00640 1 -- 172.21.15.86:0/1719805145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f449806fd50 msgr2=0x7f44980b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:14.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.339+0000 7f448fe00640 1 --2- 172.21.15.86:0/1719805145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f449806fd50 0x7f44980b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.339+0000 7f448fe00640 1 -- 172.21.15.86:0/1719805145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f449815feb0 msgr2=0x7f44981575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:14.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.339+0000 7f448fe00640 1 --2- 172.21.15.86:0/1719805145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f449815feb0 0x7f44981575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.339+0000 7f448fe00640 1 -- 172.21.15.86:0/1719805145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44981f7440 con 0x7f449815c2a0 2024-04-25T14:07:14.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.339+0000 7f448fe00640 1 --2- 172.21.15.86:0/1719805145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f449815c2a0 0x7f44981570b0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f448000c3c0 tx=0x7f448000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:14.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.343+0000 7f448e000640 1 -- 172.21.15.86:0/1719805145 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4480017070 con 0x7f449815c2a0 2024-04-25T14:07:14.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.343+0000 7f448e000640 1 -- 172.21.15.86:0/1719805145 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4480005d10 con 0x7f449815c2a0 2024-04-25T14:07:14.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.343+0000 7f449cc00640 1 -- 172.21.15.86:0/1719805145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44980c41f0 con 0x7f449815c2a0 2024-04-25T14:07:14.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.343+0000 7f448e000640 1 -- 172.21.15.86:0/1719805145 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4480007310 con 0x7f449815c2a0 2024-04-25T14:07:14.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.343+0000 7f449cc00640 1 -- 172.21.15.86:0/1719805145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44981f7920 con 0x7f449815c2a0 2024-04-25T14:07:14.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.343+0000 7f449cc00640 1 -- 172.21.15.86:0/1719805145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4498069d00 con 0x7f449815c2a0 2024-04-25T14:07:14.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.343+0000 7f448e000640 1 -- 172.21.15.86:0/1719805145 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7f448001a070 con 0x7f449815c2a0 2024-04-25T14:07:14.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.343+0000 7f448e000640 1 --2- 172.21.15.86:0/1719805145 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f447406f570 0x7f4474071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:14.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.343+0000 7f448e000640 1 -- 172.21.15.86:0/1719805145 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f448008ebd0 con 0x7f449815c2a0 2024-04-25T14:07:14.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.343+0000 7f4495e00640 1 --2- 172.21.15.86:0/1719805145 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f447406f570 0x7f4474071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:14.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.347+0000 7f4495e00640 1 --2- 172.21.15.86:0/1719805145 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f447406f570 0x7f4474071a80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f44880af000 tx=0x7f4488004740 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:14.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.347+0000 7f448e000640 1 -- 172.21.15.86:0/1719805145 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f44980c41f0 con 0x7f449815c2a0 2024-04-25T14:07:14.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.539+0000 7f449cc00640 1 -- 172.21.15.86:0/1719805145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44981f76b0 con 0x7f449815c2a0 2024-04-25T14:07:14.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.543+0000 7f448e000640 1 -- 172.21.15.86:0/1719805145 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5208 (secure 0 0 0) 0x7f44981f7440 con 0x7f449815c2a0 2024-04-25T14:07:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:07:14.543 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":56,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1210051860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1210051860},{"type":"v1","addr":"172.21.15.155:6833","nonce":1210051860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2868300438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2868300438},{"type":"v1","addr":"172.21.15.155:6835","nonce":2868300438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:06:44.910717+0000","modified":"2024-04-25T14:07:12.139089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6603,"mds_1":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"c","rank":1,"incarnation":55,"state":"up:active","state_seq":9,"addr":"172.21.15.86:6835/2682115891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2682115891},{"type":"v1","addr":"172.21.15.86:6835","nonce":2682115891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6603":{"gid":6603,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/1088108173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1088108173},{"type":"v1","addr":"172.21.15.86:6837","nonce":1088108173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6603,"qdb_cluster":[6600,6603]},"id":5}]} 2024-04-25T14:07:14.544 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 56 2024-04-25T14:07:14.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.543+0000 7f449cc00640 1 -- 172.21.15.86:0/1719805145 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f447406f570 msgr2=0x7f4474071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:14.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.543+0000 7f449cc00640 1 --2- 172.21.15.86:0/1719805145 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f447406f570 0x7f4474071a80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f44880af000 tx=0x7f4488004740 comp rx=0 tx=0).stop 2024-04-25T14:07:14.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.543+0000 7f449cc00640 1 -- 172.21.15.86:0/1719805145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f449815c2a0 msgr2=0x7f44981570b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:14.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.543+0000 7f449cc00640 1 --2- 172.21.15.86:0/1719805145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f449815c2a0 0x7f44981570b0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f448000c3c0 tx=0x7f448000c7c0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.543+0000 7f449cc00640 1 -- 172.21.15.86:0/1719805145 shutdown_connections 2024-04-25T14:07:14.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.543+0000 7f449cc00640 1 --2- 172.21.15.86:0/1719805145 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f447406f570 0x7f4474071a80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.543+0000 7f449cc00640 1 --2- 172.21.15.86:0/1719805145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f449815feb0 0x7f44981575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.543+0000 7f449cc00640 1 --2- 172.21.15.86:0/1719805145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f449815c2a0 0x7f44981570b0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.543+0000 7f449cc00640 1 --2- 172.21.15.86:0/1719805145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f449806fd50 0x7f44980b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:14.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.543+0000 7f449cc00640 1 -- 172.21.15.86:0/1719805145 >> 172.21.15.86:0/1719805145 conn(0x7f44980b4650 msgr2=0x7f44980b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:14.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.543+0000 7f449cc00640 1 -- 172.21.15.86:0/1719805145 shutdown_connections 2024-04-25T14:07:14.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:14.543+0000 7f449cc00640 1 -- 172.21.15.86:0/1719805145 wait complete. 2024-04-25T14:07:14.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:14.565 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test_mds_metrics_0) 2024-04-25T14:07:14.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:14.582 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 test_mds_metrics_0) 2024-04-25T14:07:14.633 INFO:tasks.cephfs.test_mds_metrics:pinning directory test_mds_metrics_0 to rank 0 2024-04-25T14:07:14.633 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:14.633 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:14.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:14.683 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.0 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:14.759 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:14.760 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:14.760 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244711 s, 42.8 MB/s 2024-04-25T14:07:14.761 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:14.761 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:14.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:14.811 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.1 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:14.889 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:14.889 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:14.889 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253822 s, 41.3 MB/s 2024-04-25T14:07:14.890 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:14.890 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:14.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:14.939 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.2 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:15.010 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:15.011 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:15.011 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019361 s, 54.2 MB/s 2024-04-25T14:07:15.012 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:15.012 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:15.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:15.063 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.3 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:15.137 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:15.137 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:15.138 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219667 s, 47.7 MB/s 2024-04-25T14:07:15.139 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:15.139 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:15.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:15.191 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.4 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:15.265 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:15.265 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:15.265 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172657 s, 60.7 MB/s 2024-04-25T14:07:15.266 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:15.266 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:15.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:15.315 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.5 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026769 s, 39.2 MB/s 2024-04-25T14:07:15.399 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:15.399 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:15.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:15.451 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.6 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:15.526 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:16.002 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:16.002 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195679 s, 53.6 MB/s 2024-04-25T14:07:16.002 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:16.002 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:16.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:16.013 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.7 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:16.085 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:16.085 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:16.085 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210467 s, 49.8 MB/s 2024-04-25T14:07:16.086 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:16.087 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:16.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:16.144 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.8 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:16.222 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:16.222 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:16.222 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217869 s, 48.1 MB/s 2024-04-25T14:07:16.223 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:16.223 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:16.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:16.275 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.9 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:16.349 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:16.349 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:16.349 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212928 s, 49.2 MB/s 2024-04-25T14:07:16.350 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:16.350 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:16.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:16.399 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.10 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:16.466 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:16.466 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:16.466 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014323 s, 73.2 MB/s 2024-04-25T14:07:16.467 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:16.467 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:16.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:16.519 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.11 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:16.594 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:16.594 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:16.594 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188408 s, 55.7 MB/s 2024-04-25T14:07:16.595 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:16.596 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:16.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:16.647 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.12 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:16.719 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:16.720 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:16.720 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015609 s, 67.2 MB/s 2024-04-25T14:07:16.721 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:16.721 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:16.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:16.775 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.13 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:16.841 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:16.841 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:16.842 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143632 s, 73.0 MB/s 2024-04-25T14:07:16.843 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:16.843 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:16.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:16.899 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.14 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:16.971 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:16.971 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:16.971 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019606 s, 53.5 MB/s 2024-04-25T14:07:16.972 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:16.972 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:17.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:17.023 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.15 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:17.094 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:17.094 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:17.095 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192309 s, 54.5 MB/s 2024-04-25T14:07:17.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:17.096 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test_mds_metrics_1) 2024-04-25T14:07:17.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:17.152 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 test_mds_metrics_1) 2024-04-25T14:07:17.214 INFO:tasks.cephfs.test_mds_metrics:pinning directory test_mds_metrics_1 to rank 1 2024-04-25T14:07:17.215 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:17.215 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:17.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:17.271 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.0 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:17.340 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:17.341 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:17.341 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017783 s, 59.0 MB/s 2024-04-25T14:07:17.342 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:17.342 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:17.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:17.391 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.1 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:17.469 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:17.469 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:17.469 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222186 s, 47.2 MB/s 2024-04-25T14:07:17.471 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:17.471 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:17.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:17.523 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.2 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:17.596 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:17.596 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:17.596 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164411 s, 63.8 MB/s 2024-04-25T14:07:17.597 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:17.598 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:17.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:17.652 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.3 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:17.729 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:17.729 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:17.729 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218309 s, 48.0 MB/s 2024-04-25T14:07:17.730 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:17.730 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:17.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:17.779 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.4 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:17.850 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191701 s, 54.7 MB/s 2024-04-25T14:07:17.852 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:17.852 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:17.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:17.903 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.5 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164453 s, 63.8 MB/s 2024-04-25T14:07:17.973 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:17.973 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:18.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:18.027 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.6 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:18.097 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:18.097 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:18.097 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018049 s, 58.1 MB/s 2024-04-25T14:07:18.099 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:18.099 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:18.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:18.151 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.7 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:18.221 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:18.221 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:18.221 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017873 s, 58.7 MB/s 2024-04-25T14:07:18.222 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:18.222 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:18.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:18.275 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.8 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:18.340 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:18.340 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:18.340 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132922 s, 78.9 MB/s 2024-04-25T14:07:18.341 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:18.341 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:18.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:18.392 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.9 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:18.462 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:18.463 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:18.463 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193505 s, 54.2 MB/s 2024-04-25T14:07:18.464 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:18.464 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:18.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:18.515 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.10 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:18.586 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:18.587 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:18.587 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154258 s, 68.0 MB/s 2024-04-25T14:07:18.588 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:18.588 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:18.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:18.639 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.11 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:18.709 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:18.709 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:18.709 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017766 s, 59.0 MB/s 2024-04-25T14:07:18.710 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:18.710 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:18.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:18.763 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.12 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:18.837 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:18.838 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:18.838 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185045 s, 56.7 MB/s 2024-04-25T14:07:18.839 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:18.839 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:18.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:18.891 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.13 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:18.961 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:18.961 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:18.962 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188093 s, 55.7 MB/s 2024-04-25T14:07:18.963 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:18.963 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:19.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:19.015 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.14 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:19.087 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:19.087 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:19.087 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201454 s, 52.1 MB/s 2024-04-25T14:07:19.088 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:07:19.088 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:07:19.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:19.143 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.15 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:07:19.216 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:07:19.216 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:07:19.216 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203717 s, 51.5 MB/s 2024-04-25T14:07:19.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:19.217 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest/mnt.1 && exec find .) 2024-04-25T14:07:19.243 INFO:teuthology.orchestra.run.smithi155.stdout:. 2024-04-25T14:07:19.243 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0 2024-04-25T14:07:19.243 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.5 2024-04-25T14:07:19.243 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.7 2024-04-25T14:07:19.243 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.6 2024-04-25T14:07:19.243 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.10 2024-04-25T14:07:19.244 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.4 2024-04-25T14:07:19.244 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.12 2024-04-25T14:07:19.244 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.1 2024-04-25T14:07:19.244 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.9 2024-04-25T14:07:19.244 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.13 2024-04-25T14:07:19.244 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.0 2024-04-25T14:07:19.244 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.11 2024-04-25T14:07:19.244 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.8 2024-04-25T14:07:19.244 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.3 2024-04-25T14:07:19.245 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.2 2024-04-25T14:07:19.245 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.14 2024-04-25T14:07:19.245 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.15 2024-04-25T14:07:19.245 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1 2024-04-25T14:07:19.245 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.5 2024-04-25T14:07:19.245 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.7 2024-04-25T14:07:19.245 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.6 2024-04-25T14:07:19.245 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.10 2024-04-25T14:07:19.245 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.4 2024-04-25T14:07:19.245 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.12 2024-04-25T14:07:19.245 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.1 2024-04-25T14:07:19.245 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.9 2024-04-25T14:07:19.245 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.13 2024-04-25T14:07:19.246 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.0 2024-04-25T14:07:19.246 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.11 2024-04-25T14:07:19.246 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.8 2024-04-25T14:07:19.246 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.3 2024-04-25T14:07:19.246 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.2 2024-04-25T14:07:19.246 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.14 2024-04-25T14:07:19.246 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.15 2024-04-25T14:07:19.247 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:07:19.265 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:07:24.249 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:07:24.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.407+0000 7f872a000640 1 Processor -- start 2024-04-25T14:07:24.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.407+0000 7f872a000640 1 -- start start 2024-04-25T14:07:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f872a000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f872406fd50 0x7f8724070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f872a000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87240b0a60 con 0x7f872415c2a0 2024-04-25T14:07:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f872a000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87240ae840 con 0x7f872415feb0 2024-04-25T14:07:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f872a000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f872406ef90 con 0x7f872406fd50 2024-04-25T14:07:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f8723400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f872415c2a0 0x7f87240ae130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50738/0 (socket says 172.21.15.86:50738) 2024-04-25T14:07:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f8723400640 1 -- 172.21.15.86:0/2434775948 learned_addr learned my addr 172.21.15.86:0/2434775948 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:07:24.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f871aa00640 1 --2- 172.21.15.86:0/2434775948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f872406fd50 0x7f8724070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f8722a00640 1 -- 172.21.15.86:0/2434775948 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 167713473 0 0) 0x7f87240b0a60 con 0x7f872415c2a0 2024-04-25T14:07:24.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f8722a00640 1 -- 172.21.15.86:0/2434775948 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8708003530 con 0x7f872415c2a0 2024-04-25T14:07:24.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f8722a00640 1 -- 172.21.15.86:0/2434775948 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3691952477 0 0) 0x7f87240ae840 con 0x7f872415feb0 2024-04-25T14:07:24.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f8722a00640 1 -- 172.21.15.86:0/2434775948 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87240b0a60 con 0x7f872415feb0 2024-04-25T14:07:24.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f8722a00640 1 -- 172.21.15.86:0/2434775948 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4200964651 0 0) 0x7f8708003530 con 0x7f872415c2a0 2024-04-25T14:07:24.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f8722a00640 1 -- 172.21.15.86:0/2434775948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f872406fd50 msgr2=0x7f8724070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:24.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f8722a00640 1 --2- 172.21.15.86:0/2434775948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f872406fd50 0x7f8724070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f8722a00640 1 -- 172.21.15.86:0/2434775948 >> v1:172.21.15.155:6789/0 conn(0x7f872415feb0 legacy=0x7f87241602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f8722a00640 1 -- 172.21.15.86:0/2434775948 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f872406f270 con 0x7f872415c2a0 2024-04-25T14:07:24.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f8722a00640 1 -- 172.21.15.86:0/2434775948 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f8714001680 con 0x7f872415c2a0 2024-04-25T14:07:24.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f8722a00640 1 -- 172.21.15.86:0/2434775948 >> v1:172.21.15.86:6789/0 conn(0x7f872415c2a0 legacy=0x7f87240ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f8722a00640 1 --2- 172.21.15.86:0/2434775948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8708003bf0 0x7f8708004040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f8722a00640 1 --2- 172.21.15.86:0/2434775948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f872406fd50 0x7f8708004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f8722a00640 1 --2- 172.21.15.86:0/2434775948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8708004d50 0x7f87080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f8722a00640 1 -- 172.21.15.86:0/2434775948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f872406ef90 con 0x7f8708004d50 2024-04-25T14:07:24.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f8722a00640 1 -- 172.21.15.86:0/2434775948 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f870800bc00 con 0x7f8708003bf0 2024-04-25T14:07:24.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f8722a00640 1 -- 172.21.15.86:0/2434775948 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f870800e010 con 0x7f872406fd50 2024-04-25T14:07:24.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f8723e00640 1 --2- 172.21.15.86:0/2434775948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8708004d50 0x7f87080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f871aa00640 1 --2- 172.21.15.86:0/2434775948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f872406fd50 0x7f8708004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f8723400640 1 --2- 172.21.15.86:0/2434775948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8708003bf0 0x7f8708004040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f8723e00640 1 -- 172.21.15.86:0/2434775948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f872406fd50 msgr2=0x7f8708004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:24.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f8723e00640 1 --2- 172.21.15.86:0/2434775948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f872406fd50 0x7f8708004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f8723e00640 1 -- 172.21.15.86:0/2434775948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8708003bf0 msgr2=0x7f8708004040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:24.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f8723e00640 1 --2- 172.21.15.86:0/2434775948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8708003bf0 0x7f8708004040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.411+0000 7f8723e00640 1 -- 172.21.15.86:0/2434775948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8708003530 con 0x7f8708004d50 2024-04-25T14:07:24.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.415+0000 7f8723e00640 1 --2- 172.21.15.86:0/2434775948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8708004d50 0x7f87080051c0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f871c001fd0 tx=0x7f871c01f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130d7d48e902e74c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:24.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.415+0000 7f8722a00640 1 -- 172.21.15.86:0/2434775948 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c001c20 con 0x7f8708004d50 2024-04-25T14:07:24.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.415+0000 7f8722a00640 1 -- 172.21.15.86:0/2434775948 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f871c0a8cb0 con 0x7f8708004d50 2024-04-25T14:07:24.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.415+0000 7f8722a00640 1 -- 172.21.15.86:0/2434775948 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c0a8e60 con 0x7f8708004d50 2024-04-25T14:07:24.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.415+0000 7f872a000640 1 -- 172.21.15.86:0/2434775948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8708004d50 msgr2=0x7f87080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.415+0000 7f872a000640 1 --2- 172.21.15.86:0/2434775948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8708004d50 0x7f87080051c0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f871c001fd0 tx=0x7f871c01f070 comp rx=0 tx=0).stop 2024-04-25T14:07:24.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.415+0000 7f8723e00640 1 -- 172.21.15.86:0/2434775948 reap_dead start 2024-04-25T14:07:24.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.415+0000 7f872a000640 1 -- 172.21.15.86:0/2434775948 shutdown_connections 2024-04-25T14:07:24.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.415+0000 7f872a000640 1 -- 172.21.15.86:0/2434775948 >> 172.21.15.86:0/2434775948 conn(0x7f87240b4650 msgr2=0x7f87240b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:24.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.415+0000 7f872a000640 1 -- 172.21.15.86:0/2434775948 shutdown_connections 2024-04-25T14:07:24.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.415+0000 7f872a000640 1 -- 172.21.15.86:0/2434775948 wait complete. 2024-04-25T14:07:24.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.419+0000 7f872a000640 1 Processor -- start 2024-04-25T14:07:24.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.419+0000 7f872a000640 1 -- start start 2024-04-25T14:07:24.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.419+0000 7f872a000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8708004d50 0x7f872409de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.419+0000 7f872a000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87240b1da0 0x7f872409e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.419+0000 7f872a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87241f17b0 0x7f87241f1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.419+0000 7f872a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f87240d0fc0 con 0x7f87241f17b0 2024-04-25T14:07:24.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.419+0000 7f872a000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f872406f830 con 0x7f87240b1da0 2024-04-25T14:07:24.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.419+0000 7f872a000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f872415c6b0 con 0x7f8708004d50 2024-04-25T14:07:24.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.419+0000 7f8723e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87241f17b0 0x7f87241f1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.419+0000 7f8723e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87241f17b0 0x7f87241f1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44774/0 (socket says 172.21.15.86:44774) 2024-04-25T14:07:24.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.419+0000 7f8723e00640 1 -- 172.21.15.86:0/2373408048 learned_addr learned my addr 172.21.15.86:0/2373408048 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:24.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.419+0000 7f871aa00640 1 --2- 172.21.15.86:0/2373408048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87240b1da0 0x7f872409e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.419+0000 7f8723400640 1 --2- 172.21.15.86:0/2373408048 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8708004d50 0x7f872409de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.419+0000 7f8723e00640 1 -- 172.21.15.86:0/2373408048 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8708004d50 msgr2=0x7f872409de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.419+0000 7f8723e00640 1 --2- 172.21.15.86:0/2373408048 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8708004d50 0x7f872409de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.419+0000 7f8723e00640 1 -- 172.21.15.86:0/2373408048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87240b1da0 msgr2=0x7f872409e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.419+0000 7f8723e00640 1 --2- 172.21.15.86:0/2373408048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87240b1da0 0x7f872409e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.419+0000 7f8723e00640 1 -- 172.21.15.86:0/2373408048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87240b2510 con 0x7f87241f17b0 2024-04-25T14:07:24.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.419+0000 7f8723e00640 1 --2- 172.21.15.86:0/2373408048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87241f17b0 0x7f87241f1bc0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f871c001fd0 tx=0x7f8708005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:24.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.419+0000 7f871be00640 1 -- 172.21.15.86:0/2373408048 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c0bf070 con 0x7f87241f17b0 2024-04-25T14:07:24.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.419+0000 7f871be00640 1 -- 172.21.15.86:0/2373408048 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f871c001c00 con 0x7f87241f17b0 2024-04-25T14:07:24.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.419+0000 7f871be00640 1 -- 172.21.15.86:0/2373408048 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c0a8e60 con 0x7f87241f17b0 2024-04-25T14:07:24.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.419+0000 7f872a000640 1 -- 172.21.15.86:0/2373408048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87240b27f0 con 0x7f87241f17b0 2024-04-25T14:07:24.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.419+0000 7f872a000640 1 -- 172.21.15.86:0/2373408048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87240b2db0 con 0x7f87241f17b0 2024-04-25T14:07:24.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.423+0000 7f872a000640 1 -- 172.21.15.86:0/2373408048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87240b2280 con 0x7f87241f17b0 2024-04-25T14:07:24.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.423+0000 7f871be00640 1 -- 172.21.15.86:0/2373408048 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7f871c006b60 con 0x7f87241f17b0 2024-04-25T14:07:24.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.423+0000 7f871be00640 1 --2- 172.21.15.86:0/2373408048 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f870006f570 0x7f8700071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.423+0000 7f871be00640 1 -- 172.21.15.86:0/2373408048 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f871c12c4d0 con 0x7f87241f17b0 2024-04-25T14:07:24.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.423+0000 7f8723400640 1 --2- 172.21.15.86:0/2373408048 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f870006f570 0x7f8700071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.423+0000 7f8723400640 1 --2- 172.21.15.86:0/2373408048 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f870006f570 0x7f8700071a80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8714000c10 tx=0x7f8714009040 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.427+0000 7f871be00640 1 -- 172.21.15.86:0/2373408048 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f871c0c8020 con 0x7f87241f17b0 2024-04-25T14:07:24.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.607+0000 7f872a000640 1 -- 172.21.15.86:0/2373408048 --> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f8724069d00 con 0x7f870006f570 2024-04-25T14:07:24.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.607+0000 7f871be00640 1 -- 172.21.15.86:0/2373408048 <== mgr.6801 v2:172.21.15.86:6832/203029814 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3055 (secure 0 0 0) 0x7f8724069d00 con 0x7f870006f570 2024-04-25T14:07:24.608 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {"cephfs": {"client.5945": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:172.21.15.86"}, "client.5951": {"hostname": "smithi155", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:192.168.0.1"}, "client.6834": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}, "client.6837": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}}}, "global_metrics": {"cephfs": {"client.5945": [[415, 22], [0, 0], [0, 396500909], [0, 71840640], [34, 0], [0, 35], [35, 35], [0, 35], [0, 0], [32, 33554432], [0, 0], [0, 0], [0, 12390659], [275776813322337, 32], [0, 1122510], [133594044029089, 64]], "client.5951": [[48, 2], [0, 0], [0, 0], [0, 20679725], [6, 0], [0, 3], [3, 3], [0, 3], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 2297747], [50231033486578, 9]], "client.6834": [[2, 0], [0, 0], [0, 0], [0, 510044], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 510043], [0, 1]], "client.6837": [[2, 0], [0, 0], [0, 0], [0, 761751], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 761751], [0, 1]]}}, "metrics": {"delayed_ranks": [], "mds.0": {"client.5945": [], "client.5951": [], "client.6834": [], "client.6837": []}, "mds.1": {"client.5945": [], "client.5951": []}}} 2024-04-25T14:07:24.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.607+0000 7f872a000640 1 -- 172.21.15.86:0/2373408048 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f870006f570 msgr2=0x7f8700071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.607+0000 7f872a000640 1 --2- 172.21.15.86:0/2373408048 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f870006f570 0x7f8700071a80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8714000c10 tx=0x7f8714009040 comp rx=0 tx=0).stop 2024-04-25T14:07:24.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.607+0000 7f872a000640 1 -- 172.21.15.86:0/2373408048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87241f17b0 msgr2=0x7f87241f1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.607+0000 7f872a000640 1 --2- 172.21.15.86:0/2373408048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87241f17b0 0x7f87241f1bc0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f871c001fd0 tx=0x7f8708005840 comp rx=0 tx=0).stop 2024-04-25T14:07:24.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.607+0000 7f872a000640 1 -- 172.21.15.86:0/2373408048 shutdown_connections 2024-04-25T14:07:24.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.607+0000 7f872a000640 1 --2- 172.21.15.86:0/2373408048 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f870006f570 0x7f8700071a80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.607+0000 7f872a000640 1 --2- 172.21.15.86:0/2373408048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87241f17b0 0x7f87241f1bc0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.607+0000 7f872a000640 1 --2- 172.21.15.86:0/2373408048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87240b1da0 0x7f872409e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.607+0000 7f872a000640 1 --2- 172.21.15.86:0/2373408048 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8708004d50 0x7f872409de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.607+0000 7f872a000640 1 -- 172.21.15.86:0/2373408048 >> 172.21.15.86:0/2373408048 conn(0x7f87240b4650 msgr2=0x7f87240741d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:24.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.607+0000 7f872a000640 1 -- 172.21.15.86:0/2373408048 shutdown_connections 2024-04-25T14:07:24.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.607+0000 7f872a000640 1 -- 172.21.15.86:0/2373408048 wait complete. 2024-04-25T14:07:24.630 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:07:24.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.775+0000 7fc016600640 1 Processor -- start 2024-04-25T14:07:24.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.775+0000 7fc016600640 1 -- start start 2024-04-25T14:07:24.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.775+0000 7fc016600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0100d16e0 0x7fc0100d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.775+0000 7fc016600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc0100730b0 0x7fc0100734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.775+0000 7fc016600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0100d2030 con 0x7fc0100cfca0 2024-04-25T14:07:24.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.775+0000 7fc016600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc010070470 con 0x7fc0100730b0 2024-04-25T14:07:24.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.775+0000 7fc016600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc010072bd0 con 0x7fc0100d16e0 2024-04-25T14:07:24.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc00f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc0100cfca0 0x7fc01006f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50746/0 (socket says 172.21.15.86:50746) 2024-04-25T14:07:24.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc00f400640 1 -- 172.21.15.86:0/1124089612 learned_addr learned my addr 172.21.15.86:0/1124089612 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:07:24.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc00fe00640 1 --2- 172.21.15.86:0/1124089612 >> v2:172.21.15.155:3300/0 conn(0x7fc0100730b0 0x7fc0100734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc006a00640 1 --2- 172.21.15.86:0/1124089612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0100d16e0 0x7fc0100d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc00ea00640 1 -- 172.21.15.86:0/1124089612 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3018056307 0 0) 0x7fc000001120 con 0x7fc0100cfca0 2024-04-25T14:07:24.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc00ea00640 1 -- 172.21.15.86:0/1124089612 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbff4003530 con 0x7fc0100cfca0 2024-04-25T14:07:24.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc00ea00640 1 -- 172.21.15.86:0/1124089612 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 331088142 0 0) 0x7fbff4003530 con 0x7fc0100cfca0 2024-04-25T14:07:24.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc00ea00640 1 -- 172.21.15.86:0/1124089612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0100d16e0 msgr2=0x7fc0100d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:24.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc00ea00640 1 --2- 172.21.15.86:0/1124089612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0100d16e0 0x7fc0100d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc00ea00640 1 -- 172.21.15.86:0/1124089612 >> v2:172.21.15.155:3300/0 conn(0x7fc0100730b0 msgr2=0x7fc0100734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:24.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc00ea00640 1 --2- 172.21.15.86:0/1124089612 >> v2:172.21.15.155:3300/0 conn(0x7fc0100730b0 0x7fc0100734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc00ea00640 1 -- 172.21.15.86:0/1124089612 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01006fbd0 con 0x7fc0100cfca0 2024-04-25T14:07:24.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc00ea00640 1 -- 172.21.15.86:0/1124089612 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fc000001850 con 0x7fc0100cfca0 2024-04-25T14:07:24.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc00ea00640 1 -- 172.21.15.86:0/1124089612 >> v1:172.21.15.86:6789/0 conn(0x7fc0100cfca0 legacy=0x7fc01006f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc00ea00640 1 --2- 172.21.15.86:0/1124089612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0100d16e0 0x7fbff4003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc00ea00640 1 --2- 172.21.15.86:0/1124089612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff4003fb0 0x7fbff4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc00ea00640 1 --2- 172.21.15.86:0/1124089612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbff4004d50 0x7fbff4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc00ea00640 1 -- 172.21.15.86:0/1124089612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc010070470 con 0x7fbff4003fb0 2024-04-25T14:07:24.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc00ea00640 1 -- 172.21.15.86:0/1124089612 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc010072bd0 con 0x7fbff4004d50 2024-04-25T14:07:24.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc00ea00640 1 -- 172.21.15.86:0/1124089612 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbff400bab0 con 0x7fc0100d16e0 2024-04-25T14:07:24.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc006a00640 1 --2- 172.21.15.86:0/1124089612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff4003fb0 0x7fbff4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc00f400640 1 --2- 172.21.15.86:0/1124089612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0100d16e0 0x7fbff4003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc00fe00640 1 --2- 172.21.15.86:0/1124089612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbff4004d50 0x7fbff4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc006a00640 1 -- 172.21.15.86:0/1124089612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0100d16e0 msgr2=0x7fbff4003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:24.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc006a00640 1 --2- 172.21.15.86:0/1124089612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0100d16e0 0x7fbff4003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc006a00640 1 -- 172.21.15.86:0/1124089612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbff4004d50 msgr2=0x7fbff4005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:24.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc006a00640 1 --2- 172.21.15.86:0/1124089612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbff4004d50 0x7fbff4005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc006a00640 1 -- 172.21.15.86:0/1124089612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff4003530 con 0x7fbff4003fb0 2024-04-25T14:07:24.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc00f400640 1 --2- 172.21.15.86:0/1124089612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0100d16e0 0x7fbff4003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:24.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.779+0000 7fc00fe00640 1 --2- 172.21.15.86:0/1124089612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbff4004d50 0x7fbff4005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:24.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.783+0000 7fc006a00640 1 --2- 172.21.15.86:0/1124089612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff4003fb0 0x7fbff4004810 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fbff8009fc0 tx=0x7fbff801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83ccb559da5c7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:24.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.783+0000 7fc00ea00640 1 -- 172.21.15.86:0/1124089612 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff80b2070 con 0x7fbff4003fb0 2024-04-25T14:07:24.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.783+0000 7fc00ea00640 1 -- 172.21.15.86:0/1124089612 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbff80a58e0 con 0x7fbff4003fb0 2024-04-25T14:07:24.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.783+0000 7fc00ea00640 1 -- 172.21.15.86:0/1124089612 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff8009c10 con 0x7fbff4003fb0 2024-04-25T14:07:24.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.783+0000 7fc016600640 1 -- 172.21.15.86:0/1124089612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff4003fb0 msgr2=0x7fbff4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.783+0000 7fc016600640 1 --2- 172.21.15.86:0/1124089612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff4003fb0 0x7fbff4004810 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fbff8009fc0 tx=0x7fbff801cca0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.783+0000 7fc00fe00640 1 -- 172.21.15.86:0/1124089612 reap_dead start 2024-04-25T14:07:24.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.783+0000 7fc016600640 1 -- 172.21.15.86:0/1124089612 shutdown_connections 2024-04-25T14:07:24.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.783+0000 7fc016600640 1 -- 172.21.15.86:0/1124089612 >> 172.21.15.86:0/1124089612 conn(0x7fc0100b4650 msgr2=0x7fc0100b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:24.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.783+0000 7fc016600640 1 -- 172.21.15.86:0/1124089612 shutdown_connections 2024-04-25T14:07:24.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.783+0000 7fc016600640 1 -- 172.21.15.86:0/1124089612 wait complete. 2024-04-25T14:07:24.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.783+0000 7fc016600640 1 Processor -- start 2024-04-25T14:07:24.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.787+0000 7fc016600640 1 -- start start 2024-04-25T14:07:24.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.787+0000 7fc016600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff4003fb0 0x7fc01009d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.787+0000 7fc016600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc01009eed0 0x7fc01009b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.787+0000 7fc016600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc01009a620 0x7fc0101e3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.787+0000 7fc016600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0100d1af0 con 0x7fbff4003fb0 2024-04-25T14:07:24.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.787+0000 7fc016600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc010070190 con 0x7fc01009eed0 2024-04-25T14:07:24.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.787+0000 7fc016600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc0100d1c70 con 0x7fc01009a620 2024-04-25T14:07:24.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.787+0000 7fc00f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff4003fb0 0x7fc01009d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.787+0000 7fc00f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff4003fb0 0x7fc01009d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44780/0 (socket says 172.21.15.86:44780) 2024-04-25T14:07:24.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.787+0000 7fc00f400640 1 -- 172.21.15.86:0/4166417303 learned_addr learned my addr 172.21.15.86:0/4166417303 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:24.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.787+0000 7fc00fe00640 1 --2- 172.21.15.86:0/4166417303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc01009a620 0x7fc0101e3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.787+0000 7fc006a00640 1 --2- 172.21.15.86:0/4166417303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc01009eed0 0x7fc01009b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.787+0000 7fc00f400640 1 -- 172.21.15.86:0/4166417303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc01009a620 msgr2=0x7fc0101e3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.787+0000 7fc00f400640 1 --2- 172.21.15.86:0/4166417303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc01009a620 0x7fc0101e3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.787+0000 7fc00f400640 1 -- 172.21.15.86:0/4166417303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc01009eed0 msgr2=0x7fc01009b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.787+0000 7fc00f400640 1 --2- 172.21.15.86:0/4166417303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc01009eed0 0x7fc01009b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.787+0000 7fc00f400640 1 -- 172.21.15.86:0/4166417303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01009de30 con 0x7fbff4003fb0 2024-04-25T14:07:24.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.787+0000 7fc00f400640 1 --2- 172.21.15.86:0/4166417303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff4003fb0 0x7fc01009d7e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fc000011810 tx=0x7fc000011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:24.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.787+0000 7fc007e00640 1 -- 172.21.15.86:0/4166417303 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc000010040 con 0x7fbff4003fb0 2024-04-25T14:07:24.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.787+0000 7fc007e00640 1 -- 172.21.15.86:0/4166417303 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc00001a070 con 0x7fbff4003fb0 2024-04-25T14:07:24.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.787+0000 7fc016600640 1 -- 172.21.15.86:0/4166417303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc01009e110 con 0x7fbff4003fb0 2024-04-25T14:07:24.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.787+0000 7fc007e00640 1 -- 172.21.15.86:0/4166417303 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00000fb60 con 0x7fbff4003fb0 2024-04-25T14:07:24.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.787+0000 7fc016600640 1 -- 172.21.15.86:0/4166417303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc01009e6d0 con 0x7fbff4003fb0 2024-04-25T14:07:24.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.787+0000 7fc016600640 1 -- 172.21.15.86:0/4166417303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc01009f4b0 con 0x7fbff4003fb0 2024-04-25T14:07:24.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.791+0000 7fc007e00640 1 -- 172.21.15.86:0/4166417303 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7fc00000fd00 con 0x7fbff4003fb0 2024-04-25T14:07:24.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.791+0000 7fc007e00640 1 --2- 172.21.15.86:0/4166417303 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fbfec06f570 0x7fbfec071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.791+0000 7fc007e00640 1 -- 172.21.15.86:0/4166417303 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7fc000092880 con 0x7fbff4003fb0 2024-04-25T14:07:24.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.791+0000 7fc006a00640 1 --2- 172.21.15.86:0/4166417303 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fbfec06f570 0x7fbfec071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.791+0000 7fc006a00640 1 --2- 172.21.15.86:0/4166417303 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fbfec06f570 0x7fbfec071a80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbff80af000 tx=0x7fbff80a7040 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:24.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.795+0000 7fc007e00640 1 -- 172.21.15.86:0/4166417303 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc0100d2030 con 0x7fbff4003fb0 2024-04-25T14:07:24.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.983+0000 7fc016600640 1 -- 172.21.15.86:0/4166417303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc010069d00 con 0x7fbff4003fb0 2024-04-25T14:07:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.987+0000 7fc007e00640 1 -- 172.21.15.86:0/4166417303 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5209 (secure 0 0 0) 0x7fc00005f310 con 0x7fbff4003fb0 2024-04-25T14:07:24.987 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:07:24.987 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":57,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1210051860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1210051860},{"type":"v1","addr":"172.21.15.155:6833","nonce":1210051860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2868300438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2868300438},{"type":"v1","addr":"172.21.15.155:6835","nonce":2868300438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:06:44.910717+0000","modified":"2024-04-25T14:07:19.703362+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6603,"mds_1":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"c","rank":1,"incarnation":55,"state":"up:active","state_seq":9,"addr":"172.21.15.86:6835/2682115891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2682115891},{"type":"v1","addr":"172.21.15.86:6835","nonce":2682115891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6603":{"gid":6603,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/1088108173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1088108173},{"type":"v1","addr":"172.21.15.86:6837","nonce":1088108173}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6603,"qdb_cluster":[6603,6600]},"id":5}]} 2024-04-25T14:07:24.988 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 57 2024-04-25T14:07:24.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.987+0000 7fc016600640 1 -- 172.21.15.86:0/4166417303 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fbfec06f570 msgr2=0x7fbfec071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.987+0000 7fc016600640 1 --2- 172.21.15.86:0/4166417303 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fbfec06f570 0x7fbfec071a80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbff80af000 tx=0x7fbff80a7040 comp rx=0 tx=0).stop 2024-04-25T14:07:24.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.987+0000 7fc016600640 1 -- 172.21.15.86:0/4166417303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff4003fb0 msgr2=0x7fc01009d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.987+0000 7fc016600640 1 --2- 172.21.15.86:0/4166417303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff4003fb0 0x7fc01009d7e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fc000011810 tx=0x7fc000011cd0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.987+0000 7fc016600640 1 -- 172.21.15.86:0/4166417303 shutdown_connections 2024-04-25T14:07:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.987+0000 7fc016600640 1 --2- 172.21.15.86:0/4166417303 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fbfec06f570 0x7fbfec071a80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.987+0000 7fc016600640 1 --2- 172.21.15.86:0/4166417303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc01009a620 0x7fc0101e3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.987+0000 7fc016600640 1 --2- 172.21.15.86:0/4166417303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc01009eed0 0x7fc01009b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.987+0000 7fc016600640 1 --2- 172.21.15.86:0/4166417303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff4003fb0 0x7fc01009d7e0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.987+0000 7fc016600640 1 -- 172.21.15.86:0/4166417303 >> 172.21.15.86:0/4166417303 conn(0x7fc0100b4650 msgr2=0x7fc0100b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.987+0000 7fc016600640 1 -- 172.21.15.86:0/4166417303 shutdown_connections 2024-04-25T14:07:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:24.987+0000 7fc016600640 1 -- 172.21.15.86:0/4166417303 wait complete. 2024-04-25T14:07:25.009 DEBUG:tasks.cephfs.test_mds_metrics:metrics={'version': 2, 'global_counters': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'counters': [], 'client_metadata': {'cephfs': {'client.5945': {'hostname': 'smithi086', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'kernel_version': '6.9.0-rc3-g806ec36b2112', 'IP': 'v1:172.21.15.86'}, 'client.5951': {'hostname': 'smithi155', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'kernel_version': '6.9.0-rc3-g806ec36b2112', 'IP': 'v1:192.168.0.1'}, 'client.6834': {'hostname': 'smithi086', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'IP': '172.21.15.86'}, 'client.6837': {'hostname': 'smithi086', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'IP': '172.21.15.86'}}}, 'global_metrics': {'cephfs': {'client.5945': [[415, 22], [0, 0], [0, 396500909], [0, 71840640], [34, 0], [0, 35], [35, 35], [0, 35], [0, 0], [32, 33554432], [0, 0], [0, 0], [0, 12390659], [275776813322337, 32], [0, 1122510], [133594044029089, 64]], 'client.5951': [[48, 2], [0, 0], [0, 0], [0, 20679725], [6, 0], [0, 3], [3, 3], [0, 3], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 2297747], [50231033486578, 9]], 'client.6834': [[2, 0], [0, 0], [0, 0], [0, 510044], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 510043], [0, 1]], 'client.6837': [[2, 0], [0, 0], [0, 0], [0, 761751], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 761751], [0, 1]]}}, 'metrics': {'delayed_ranks': [], 'mds.0': {'client.5945': [], 'client.5951': [], 'client.6834': [], 'client.6837': []}, 'mds.1': {'client.5945': [], 'client.5951': []}}} 2024-04-25T14:07:25.009 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:07:25.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.155+0000 7f7f48a00640 1 Processor -- start 2024-04-25T14:07:25.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.155+0000 7f7f48a00640 1 -- start start 2024-04-25T14:07:25.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.155+0000 7f7f48a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f440d1200 0x7f7f440d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:25.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.155+0000 7f7f48a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f440d51a0 con 0x7f7f440b4440 2024-04-25T14:07:25.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.155+0000 7f7f48a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f440b4a30 con 0x7f7f440d3e70 2024-04-25T14:07:25.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.155+0000 7f7f48a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7f440b4d10 con 0x7f7f440d1200 2024-04-25T14:07:25.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.155+0000 7f7f41400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7f440b4440 0x7f7f440d4380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50762/0 (socket says 172.21.15.86:50762) 2024-04-25T14:07:25.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.155+0000 7f7f41400640 1 -- 172.21.15.86:0/1861176742 learned_addr learned my addr 172.21.15.86:0/1861176742 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:07:25.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.155+0000 7f7f41e00640 1 --2- 172.21.15.86:0/1861176742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f440d1200 0x7f7f440d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:25.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.155+0000 7f7f40a00640 1 -- 172.21.15.86:0/1861176742 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 889257612 0 0) 0x7f7f440d51a0 con 0x7f7f440b4440 2024-04-25T14:07:25.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.155+0000 7f7f40a00640 1 -- 172.21.15.86:0/1861176742 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f28003530 con 0x7f7f440b4440 2024-04-25T14:07:25.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.159+0000 7f7f40a00640 1 -- 172.21.15.86:0/1861176742 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2717550339 0 0) 0x7f7f440b4a30 con 0x7f7f440d3e70 2024-04-25T14:07:25.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.159+0000 7f7f40a00640 1 -- 172.21.15.86:0/1861176742 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f440d51a0 con 0x7f7f440d3e70 2024-04-25T14:07:25.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.159+0000 7f7f41e00640 1 -- 172.21.15.86:0/1861176742 >> v1:172.21.15.155:6789/0 conn(0x7f7f440d3e70 legacy=0x7f7f440d4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:25.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.159+0000 7f7f41e00640 1 -- 172.21.15.86:0/1861176742 >> v1:172.21.15.86:6789/0 conn(0x7f7f440b4440 legacy=0x7f7f440d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:25.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.159+0000 7f7f40a00640 1 -- 172.21.15.86:0/1861176742 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3856834635 0 0) 0x7f7f28003530 con 0x7f7f440b4440 2024-04-25T14:07:25.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.159+0000 7f7f41e00640 1 -- 172.21.15.86:0/1861176742 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4406d9c0 con 0x7f7f440d1200 2024-04-25T14:07:25.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.159+0000 7f7f41e00640 1 --2- 172.21.15.86:0/1861176742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f440d1200 0x7f7f440d1dd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7f340097f0 tx=0x7f7f3401c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a487d68f921229f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:25.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.159+0000 7f7f40a00640 1 -- 172.21.15.86:0/1861176742 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f340b0070 con 0x7f7f440d1200 2024-04-25T14:07:25.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.159+0000 7f7f40a00640 1 -- 172.21.15.86:0/1861176742 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f340ab400 con 0x7f7f440d1200 2024-04-25T14:07:25.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.159+0000 7f7f40a00640 1 -- 172.21.15.86:0/1861176742 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f30001680 con 0x7f7f440d1200 2024-04-25T14:07:25.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.163+0000 7f7f48a00640 1 -- 172.21.15.86:0/1861176742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f440d1200 msgr2=0x7f7f440d1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:25.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.163+0000 7f7f48a00640 1 --2- 172.21.15.86:0/1861176742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f440d1200 0x7f7f440d1dd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7f340097f0 tx=0x7f7f3401c370 comp rx=0 tx=0).stop 2024-04-25T14:07:25.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.163+0000 7f7f48a00640 1 -- 172.21.15.86:0/1861176742 shutdown_connections 2024-04-25T14:07:25.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.163+0000 7f7f48a00640 1 --2- 172.21.15.86:0/1861176742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f440d1200 0x7f7f440d1dd0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.163+0000 7f7f48a00640 1 -- 172.21.15.86:0/1861176742 >> 172.21.15.86:0/1861176742 conn(0x7f7f440b72d0 msgr2=0x7f7f440b76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:25.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.163+0000 7f7f48a00640 1 -- 172.21.15.86:0/1861176742 shutdown_connections 2024-04-25T14:07:25.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.163+0000 7f7f48a00640 1 -- 172.21.15.86:0/1861176742 wait complete. 2024-04-25T14:07:25.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.163+0000 7f7f48a00640 1 Processor -- start 2024-04-25T14:07:25.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.163+0000 7f7f48a00640 1 -- start start 2024-04-25T14:07:25.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.167+0000 7f7f48a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f440b4440 0x7f7f440c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:25.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.167+0000 7f7f48a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f440d1200 0x7f7f440c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:25.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.167+0000 7f7f48a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f440d3e70 0x7f7f44149310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:25.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.167+0000 7f7f48a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4416e870 con 0x7f7f440d1200 2024-04-25T14:07:25.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.167+0000 7f7f48a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4416e6f0 con 0x7f7f440d3e70 2024-04-25T14:07:25.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.167+0000 7f7f48a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7f4416e9f0 con 0x7f7f440b4440 2024-04-25T14:07:25.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.167+0000 7f7f41400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f440d1200 0x7f7f440c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:25.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.167+0000 7f7f41400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f440d1200 0x7f7f440c9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44788/0 (socket says 172.21.15.86:44788) 2024-04-25T14:07:25.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.167+0000 7f7f41400640 1 -- 172.21.15.86:0/2769398323 learned_addr learned my addr 172.21.15.86:0/2769398323 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:25.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.167+0000 7f7f42800640 1 --2- 172.21.15.86:0/2769398323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f440d3e70 0x7f7f44149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:25.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.167+0000 7f7f41e00640 1 --2- 172.21.15.86:0/2769398323 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f440b4440 0x7f7f440c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:25.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.167+0000 7f7f41400640 1 -- 172.21.15.86:0/2769398323 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f440b4440 msgr2=0x7f7f440c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:25.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.167+0000 7f7f41400640 1 --2- 172.21.15.86:0/2769398323 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f440b4440 0x7f7f440c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.167+0000 7f7f41400640 1 -- 172.21.15.86:0/2769398323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f440d3e70 msgr2=0x7f7f44149310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:25.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.167+0000 7f7f41400640 1 --2- 172.21.15.86:0/2769398323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f440d3e70 0x7f7f44149310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.167+0000 7f7f41400640 1 -- 172.21.15.86:0/2769398323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4415f810 con 0x7f7f440d1200 2024-04-25T14:07:25.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.167+0000 7f7f41400640 1 --2- 172.21.15.86:0/2769398323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f440d1200 0x7f7f440c9cb0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f7f30001ec0 tx=0x7f7f3000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:25.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.167+0000 7f7f3a000640 1 -- 172.21.15.86:0/2769398323 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f30017070 con 0x7f7f440d1200 2024-04-25T14:07:25.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.167+0000 7f7f3a000640 1 -- 172.21.15.86:0/2769398323 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f30005ae0 con 0x7f7f440d1200 2024-04-25T14:07:25.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.167+0000 7f7f48a00640 1 -- 172.21.15.86:0/2769398323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f440b5810 con 0x7f7f440d1200 2024-04-25T14:07:25.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.167+0000 7f7f3a000640 1 -- 172.21.15.86:0/2769398323 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f30007070 con 0x7f7f440d1200 2024-04-25T14:07:25.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.167+0000 7f7f48a00640 1 -- 172.21.15.86:0/2769398323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f440cbf80 con 0x7f7f440d1200 2024-04-25T14:07:25.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.167+0000 7f7f48a00640 1 -- 172.21.15.86:0/2769398323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f44160d40 con 0x7f7f440d1200 2024-04-25T14:07:25.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.167+0000 7f7f3a000640 1 -- 172.21.15.86:0/2769398323 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7f7f3001a070 con 0x7f7f440d1200 2024-04-25T14:07:25.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.171+0000 7f7f3a000640 1 --2- 172.21.15.86:0/2769398323 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f7f2406f570 0x7f7f24071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:25.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.171+0000 7f7f3a000640 1 -- 172.21.15.86:0/2769398323 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f7f3008ea10 con 0x7f7f440d1200 2024-04-25T14:07:25.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.171+0000 7f7f41e00640 1 --2- 172.21.15.86:0/2769398323 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f7f2406f570 0x7f7f24071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:25.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.171+0000 7f7f41e00640 1 --2- 172.21.15.86:0/2769398323 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f7f2406f570 0x7f7f24071a80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7f340ad000 tx=0x7f7f340a6000 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:25.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.175+0000 7f7f3a000640 1 -- 172.21.15.86:0/2769398323 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7f44160d40 con 0x7f7f440d1200 2024-04-25T14:07:25.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.367+0000 7f7f48a00640 1 -- 172.21.15.86:0/2769398323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f4406ef80 con 0x7f7f440d1200 2024-04-25T14:07:25.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.367+0000 7f7f3a000640 1 -- 172.21.15.86:0/2769398323 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5209 (secure 0 0 0) 0x7f7f3005af80 con 0x7f7f440d1200 2024-04-25T14:07:25.368 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:07:25.369 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":57,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1210051860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1210051860},{"type":"v1","addr":"172.21.15.155:6833","nonce":1210051860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2868300438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2868300438},{"type":"v1","addr":"172.21.15.155:6835","nonce":2868300438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:06:44.910717+0000","modified":"2024-04-25T14:07:19.703362+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6603,"mds_1":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"c","rank":1,"incarnation":55,"state":"up:active","state_seq":9,"addr":"172.21.15.86:6835/2682115891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2682115891},{"type":"v1","addr":"172.21.15.86:6835","nonce":2682115891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6603":{"gid":6603,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/1088108173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1088108173},{"type":"v1","addr":"172.21.15.86:6837","nonce":1088108173}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6603,"qdb_cluster":[6603,6600]},"id":5}]} 2024-04-25T14:07:25.369 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 57 2024-04-25T14:07:25.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.367+0000 7f7f48a00640 1 -- 172.21.15.86:0/2769398323 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f7f2406f570 msgr2=0x7f7f24071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:25.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.367+0000 7f7f48a00640 1 --2- 172.21.15.86:0/2769398323 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f7f2406f570 0x7f7f24071a80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7f340ad000 tx=0x7f7f340a6000 comp rx=0 tx=0).stop 2024-04-25T14:07:25.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.367+0000 7f7f48a00640 1 -- 172.21.15.86:0/2769398323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f440d1200 msgr2=0x7f7f440c9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:25.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.367+0000 7f7f48a00640 1 --2- 172.21.15.86:0/2769398323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f440d1200 0x7f7f440c9cb0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f7f30001ec0 tx=0x7f7f3000cbe0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.367+0000 7f7f48a00640 1 -- 172.21.15.86:0/2769398323 shutdown_connections 2024-04-25T14:07:25.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.367+0000 7f7f48a00640 1 --2- 172.21.15.86:0/2769398323 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f7f2406f570 0x7f7f24071a80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.367+0000 7f7f48a00640 1 --2- 172.21.15.86:0/2769398323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f440d3e70 0x7f7f44149310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.367+0000 7f7f48a00640 1 --2- 172.21.15.86:0/2769398323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f440d1200 0x7f7f440c9cb0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.367+0000 7f7f48a00640 1 --2- 172.21.15.86:0/2769398323 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f440b4440 0x7f7f440c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.367+0000 7f7f48a00640 1 -- 172.21.15.86:0/2769398323 >> 172.21.15.86:0/2769398323 conn(0x7f7f440b72d0 msgr2=0x7f7f440b6250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:25.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.367+0000 7f7f48a00640 1 -- 172.21.15.86:0/2769398323 shutdown_connections 2024-04-25T14:07:25.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.367+0000 7f7f48a00640 1 -- 172.21.15.86:0/2769398323 wait complete. 2024-04-25T14:07:25.390 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:07:25.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.535+0000 7f7ee8200640 1 Processor -- start 2024-04-25T14:07:25.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.535+0000 7f7ee8200640 1 -- start start 2024-04-25T14:07:25.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.535+0000 7f7ee8200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7ee00cf970 0x7f7ee0070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:25.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.535+0000 7f7ee8200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ee006e970 0x7f7ee00cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:25.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.535+0000 7f7ee8200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ee0071770 con 0x7f7ee0071360 2024-04-25T14:07:25.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.535+0000 7f7ee8200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7ee0163830 con 0x7f7ee00cf970 2024-04-25T14:07:25.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.535+0000 7f7ee8200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee006ef90 con 0x7f7ee006e970 2024-04-25T14:07:25.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.535+0000 7f7ee5800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7ee00cf970 0x7f7ee0070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:25.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.535+0000 7f7ee5800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7ee00cf970 0x7f7ee0070e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44804/0 (socket says 172.21.15.86:44804) 2024-04-25T14:07:25.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.535+0000 7f7ee5800640 1 -- 172.21.15.86:0/801766757 learned_addr learned my addr 172.21.15.86:0/801766757 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:25.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.535+0000 7f7ee6200640 1 --2- 172.21.15.86:0/801766757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ee006e970 0x7f7ee00cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:25.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.539+0000 7f7ee5800640 1 -- 172.21.15.86:0/801766757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ee006e970 msgr2=0x7f7ee00cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:25.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.539+0000 7f7ee5800640 1 --2- 172.21.15.86:0/801766757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ee006e970 0x7f7ee00cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.539+0000 7f7ee5800640 1 -- 172.21.15.86:0/801766757 >> v1:172.21.15.86:6789/0 conn(0x7f7ee0071360 legacy=0x7f7ee006e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:25.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.539+0000 7f7ee5800640 1 -- 172.21.15.86:0/801766757 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee0163a80 con 0x7f7ee00cf970 2024-04-25T14:07:25.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.539+0000 7f7ee6200640 1 --2- 172.21.15.86:0/801766757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ee006e970 0x7f7ee00cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:25.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.539+0000 7f7ee5800640 1 --2- 172.21.15.86:0/801766757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ee00cf970 0x7f7ee0070e20 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4009870 tx=0x7f7ed401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612da706f3ac474d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.539+0000 7f7edfe00640 1 -- 172.21.15.86:0/801766757 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed40ab650 con 0x7f7ee00cf970 2024-04-25T14:07:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.539+0000 7f7edfe00640 1 -- 172.21.15.86:0/801766757 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ed40b0070 con 0x7f7ee00cf970 2024-04-25T14:07:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.539+0000 7f7edfe00640 1 -- 172.21.15.86:0/801766757 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed40aba10 con 0x7f7ee00cf970 2024-04-25T14:07:25.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.539+0000 7f7ee8200640 1 -- 172.21.15.86:0/801766757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ee00cf970 msgr2=0x7f7ee0070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:25.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.539+0000 7f7ee8200640 1 --2- 172.21.15.86:0/801766757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ee00cf970 0x7f7ee0070e20 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4009870 tx=0x7f7ed401c560 comp rx=0 tx=0).stop 2024-04-25T14:07:25.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.543+0000 7f7ee8200640 1 -- 172.21.15.86:0/801766757 shutdown_connections 2024-04-25T14:07:25.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.543+0000 7f7ee8200640 1 --2- 172.21.15.86:0/801766757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ee006e970 0x7f7ee00cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.543+0000 7f7ee8200640 1 --2- 172.21.15.86:0/801766757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ee00cf970 0x7f7ee0070e20 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.543+0000 7f7ee8200640 1 -- 172.21.15.86:0/801766757 >> 172.21.15.86:0/801766757 conn(0x7f7ee00cffc0 msgr2=0x7f7ee00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:25.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.543+0000 7f7ee8200640 1 -- 172.21.15.86:0/801766757 shutdown_connections 2024-04-25T14:07:25.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.543+0000 7f7ee8200640 1 -- 172.21.15.86:0/801766757 wait complete. 2024-04-25T14:07:25.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.543+0000 7f7ee8200640 1 Processor -- start 2024-04-25T14:07:25.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.543+0000 7f7ee8200640 1 -- start start 2024-04-25T14:07:25.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.547+0000 7f7ee8200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ee006e970 0x7f7ee00cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:25.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.547+0000 7f7ee8200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ee0071360 0x7f7ee0150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:25.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.547+0000 7f7ee8200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ee00cf970 0x7f7ee00b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:25.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.547+0000 7f7ee8200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee016cbd0 con 0x7f7ee0071360 2024-04-25T14:07:25.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.547+0000 7f7ee8200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee00ced80 con 0x7f7ee00cf970 2024-04-25T14:07:25.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.547+0000 7f7ee8200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee016cd50 con 0x7f7ee006e970 2024-04-25T14:07:25.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.547+0000 7f7ee5800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ee006e970 0x7f7ee00cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:25.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.547+0000 7f7ee5800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ee006e970 0x7f7ee00cb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:41406/0 (socket says 172.21.15.86:41406) 2024-04-25T14:07:25.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.547+0000 7f7ee5800640 1 -- 172.21.15.86:0/2965457706 learned_addr learned my addr 172.21.15.86:0/2965457706 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:25.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.547+0000 7f7ee4e00640 1 --2- 172.21.15.86:0/2965457706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ee0071360 0x7f7ee0150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:25.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.547+0000 7f7ee6200640 1 --2- 172.21.15.86:0/2965457706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ee00cf970 0x7f7ee00b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:25.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.547+0000 7f7ee4e00640 1 -- 172.21.15.86:0/2965457706 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ee006e970 msgr2=0x7f7ee00cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:25.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.547+0000 7f7ee4e00640 1 --2- 172.21.15.86:0/2965457706 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ee006e970 0x7f7ee00cb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.547+0000 7f7ee4e00640 1 -- 172.21.15.86:0/2965457706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ee00cf970 msgr2=0x7f7ee00b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:25.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.547+0000 7f7ee4e00640 1 --2- 172.21.15.86:0/2965457706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ee00cf970 0x7f7ee00b3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.547+0000 7f7ee4e00640 1 -- 172.21.15.86:0/2965457706 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee00cb860 con 0x7f7ee0071360 2024-04-25T14:07:25.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.547+0000 7f7ee4e00640 1 --2- 172.21.15.86:0/2965457706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ee0071360 0x7f7ee0150a30 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00c3f0 tx=0x7f7ecc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:25.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.547+0000 7f7edd600640 1 -- 172.21.15.86:0/2965457706 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc017070 con 0x7f7ee0071360 2024-04-25T14:07:25.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.547+0000 7f7edd600640 1 -- 172.21.15.86:0/2965457706 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ecc003600 con 0x7f7ee0071360 2024-04-25T14:07:25.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.547+0000 7f7edd600640 1 -- 172.21.15.86:0/2965457706 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc0039f0 con 0x7f7ee0071360 2024-04-25T14:07:25.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.547+0000 7f7ee8200640 1 -- 172.21.15.86:0/2965457706 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee015ad60 con 0x7f7ee0071360 2024-04-25T14:07:25.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.547+0000 7f7ee8200640 1 -- 172.21.15.86:0/2965457706 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee00b63b0 con 0x7f7ee0071360 2024-04-25T14:07:25.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.547+0000 7f7ee8200640 1 -- 172.21.15.86:0/2965457706 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ee00720e0 con 0x7f7ee0071360 2024-04-25T14:07:25.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.551+0000 7f7edd600640 1 -- 172.21.15.86:0/2965457706 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7f7ecc01a070 con 0x7f7ee0071360 2024-04-25T14:07:25.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.551+0000 7f7edd600640 1 --2- 172.21.15.86:0/2965457706 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f7ec006f570 0x7f7ec0071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:25.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.551+0000 7f7edd600640 1 -- 172.21.15.86:0/2965457706 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f7ecc08e980 con 0x7f7ee0071360 2024-04-25T14:07:25.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.551+0000 7f7ee5800640 1 --2- 172.21.15.86:0/2965457706 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f7ec006f570 0x7f7ec0071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:25.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.551+0000 7f7ee5800640 1 --2- 172.21.15.86:0/2965457706 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f7ec006f570 0x7f7ec0071a80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7ed401c5a0 tx=0x7f7ed40a7000 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:25.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.555+0000 7f7edd600640 1 -- 172.21.15.86:0/2965457706 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7ee0071770 con 0x7f7ee0071360 2024-04-25T14:07:25.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.747+0000 7f7ee8200640 1 -- 172.21.15.86:0/2965457706 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ee00b94c0 con 0x7f7ee0071360 2024-04-25T14:07:25.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.747+0000 7f7edd600640 1 -- 172.21.15.86:0/2965457706 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5209 (secure 0 0 0) 0x7f7ecc05ae40 con 0x7f7ee0071360 2024-04-25T14:07:25.748 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:07:25.748 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":57,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1210051860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1210051860},{"type":"v1","addr":"172.21.15.155:6833","nonce":1210051860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2868300438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2868300438},{"type":"v1","addr":"172.21.15.155:6835","nonce":2868300438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:06:44.910717+0000","modified":"2024-04-25T14:07:19.703362+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6603,"mds_1":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"c","rank":1,"incarnation":55,"state":"up:active","state_seq":9,"addr":"172.21.15.86:6835/2682115891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2682115891},{"type":"v1","addr":"172.21.15.86:6835","nonce":2682115891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6603":{"gid":6603,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/1088108173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1088108173},{"type":"v1","addr":"172.21.15.86:6837","nonce":1088108173}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6603,"qdb_cluster":[6603,6600]},"id":5}]} 2024-04-25T14:07:25.748 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 57 2024-04-25T14:07:25.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.747+0000 7f7ee8200640 1 -- 172.21.15.86:0/2965457706 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f7ec006f570 msgr2=0x7f7ec0071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:25.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.747+0000 7f7ee8200640 1 --2- 172.21.15.86:0/2965457706 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f7ec006f570 0x7f7ec0071a80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7ed401c5a0 tx=0x7f7ed40a7000 comp rx=0 tx=0).stop 2024-04-25T14:07:25.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.747+0000 7f7ee8200640 1 -- 172.21.15.86:0/2965457706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ee0071360 msgr2=0x7f7ee0150a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:25.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.747+0000 7f7ee8200640 1 --2- 172.21.15.86:0/2965457706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ee0071360 0x7f7ee0150a30 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00c3f0 tx=0x7f7ecc00c8b0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.747+0000 7f7ee8200640 1 -- 172.21.15.86:0/2965457706 shutdown_connections 2024-04-25T14:07:25.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.747+0000 7f7ee8200640 1 --2- 172.21.15.86:0/2965457706 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f7ec006f570 0x7f7ec0071a80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.747+0000 7f7ee8200640 1 --2- 172.21.15.86:0/2965457706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ee00cf970 0x7f7ee00b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.747+0000 7f7ee8200640 1 --2- 172.21.15.86:0/2965457706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ee0071360 0x7f7ee0150a30 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.747+0000 7f7ee8200640 1 --2- 172.21.15.86:0/2965457706 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ee006e970 0x7f7ee00cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.747+0000 7f7ee8200640 1 -- 172.21.15.86:0/2965457706 >> 172.21.15.86:0/2965457706 conn(0x7f7ee00cffc0 msgr2=0x7f7ee00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:25.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.747+0000 7f7ee8200640 1 -- 172.21.15.86:0/2965457706 shutdown_connections 2024-04-25T14:07:25.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.747+0000 7f7ee8200640 1 -- 172.21.15.86:0/2965457706 wait complete. 2024-04-25T14:07:25.769 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:07:25.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.915+0000 7f82bac00640 1 Processor -- start 2024-04-25T14:07:25.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.915+0000 7f82bac00640 1 -- start start 2024-04-25T14:07:25.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.915+0000 7f82bac00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82b415fe00 0x7f82b4070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:25.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.915+0000 7f82bac00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f82b4071360 0x7f82b41638a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:25.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.915+0000 7f82bac00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82b4071770 con 0x7f82b4163de0 2024-04-25T14:07:25.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.915+0000 7f82bac00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f82b4160210 con 0x7f82b4071360 2024-04-25T14:07:25.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.915+0000 7f82bac00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f82b406e930 con 0x7f82b415fe00 2024-04-25T14:07:25.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.915+0000 7f82b3e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82b415fe00 0x7f82b4070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:25.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.915+0000 7f82b8c00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f82b4163de0 0x7f82b40cdde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:60532/0 (socket says 172.21.15.86:60532) 2024-04-25T14:07:25.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.915+0000 7f82b3e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82b415fe00 0x7f82b4070e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:41412/0 (socket says 172.21.15.86:41412) 2024-04-25T14:07:25.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.915+0000 7f82b8c00640 1 -- 172.21.15.86:0/1467800497 learned_addr learned my addr 172.21.15.86:0/1467800497 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:07:25.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.915+0000 7f82b3400640 1 --2- 172.21.15.86:0/1467800497 >> v2:172.21.15.155:3300/0 conn(0x7f82b4071360 0x7f82b41638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:25.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.915+0000 7f82b3400640 1 -- 172.21.15.86:0/1467800497 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82b415fe00 msgr2=0x7f82b4070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:25.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.915+0000 7f82b3400640 1 --2- 172.21.15.86:0/1467800497 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82b415fe00 0x7f82b4070e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.915+0000 7f82b3400640 1 -- 172.21.15.86:0/1467800497 >> v1:172.21.15.155:6789/0 conn(0x7f82b4163de0 legacy=0x7f82b40cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:25.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.915+0000 7f82b3400640 1 -- 172.21.15.86:0/1467800497 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b41641f0 con 0x7f82b4071360 2024-04-25T14:07:25.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.915+0000 7f82b2a00640 1 -- 172.21.15.86:0/1467800497 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3855238040 0 0) 0x7f82ac0014a0 con 0x7f82b4163de0 2024-04-25T14:07:25.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.919+0000 7f82b3400640 1 --2- 172.21.15.86:0/1467800497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82b4071360 0x7f82b41638a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f82a00097a0 tx=0x7f82a001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce726d5dbb0dd9c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:25.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.919+0000 7f82b2a00640 1 -- 172.21.15.86:0/1467800497 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a00b0070 con 0x7f82b4071360 2024-04-25T14:07:25.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.919+0000 7f82b2a00640 1 -- 172.21.15.86:0/1467800497 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82a00ab600 con 0x7f82b4071360 2024-04-25T14:07:25.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.919+0000 7f82b2a00640 1 -- 172.21.15.86:0/1467800497 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a00ab9d0 con 0x7f82b4071360 2024-04-25T14:07:25.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.919+0000 7f82bac00640 1 -- 172.21.15.86:0/1467800497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82b4071360 msgr2=0x7f82b41638a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:25.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.919+0000 7f82bac00640 1 --2- 172.21.15.86:0/1467800497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82b4071360 0x7f82b41638a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f82a00097a0 tx=0x7f82a001c480 comp rx=0 tx=0).stop 2024-04-25T14:07:25.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.919+0000 7f82bac00640 1 -- 172.21.15.86:0/1467800497 shutdown_connections 2024-04-25T14:07:25.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.919+0000 7f82bac00640 1 --2- 172.21.15.86:0/1467800497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82b4071360 0x7f82b41638a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.919+0000 7f82bac00640 1 --2- 172.21.15.86:0/1467800497 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82b415fe00 0x7f82b4070e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.919+0000 7f82bac00640 1 -- 172.21.15.86:0/1467800497 >> 172.21.15.86:0/1467800497 conn(0x7f82b40cffc0 msgr2=0x7f82b40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:25.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.919+0000 7f82bac00640 1 -- 172.21.15.86:0/1467800497 shutdown_connections 2024-04-25T14:07:25.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.919+0000 7f82bac00640 1 -- 172.21.15.86:0/1467800497 wait complete. 2024-04-25T14:07:25.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.923+0000 7f82bac00640 1 Processor -- start 2024-04-25T14:07:25.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.923+0000 7f82bac00640 1 -- start start 2024-04-25T14:07:25.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.923+0000 7f82bac00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82b4071360 0x7f82b4157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:25.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.923+0000 7f82bac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82b415fe00 0x7f82b40b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:25.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.923+0000 7f82bac00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82b4163de0 0x7f82b40cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:25.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.923+0000 7f82bac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82b416cc40 con 0x7f82b415fe00 2024-04-25T14:07:25.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.923+0000 7f82bac00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f82b4164760 con 0x7f82b4163de0 2024-04-25T14:07:25.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.923+0000 7f82bac00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f82b416cdc0 con 0x7f82b4071360 2024-04-25T14:07:25.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.923+0000 7f82b3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82b415fe00 0x7f82b40b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:25.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.923+0000 7f82b3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82b415fe00 0x7f82b40b61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44822/0 (socket says 172.21.15.86:44822) 2024-04-25T14:07:25.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.923+0000 7f82b3400640 1 -- 172.21.15.86:0/256914359 learned_addr learned my addr 172.21.15.86:0/256914359 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:25.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.923+0000 7f82b3e00640 1 --2- 172.21.15.86:0/256914359 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82b4071360 0x7f82b4157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:25.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.923+0000 7f82b8c00640 1 --2- 172.21.15.86:0/256914359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82b4163de0 0x7f82b40cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:25.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.923+0000 7f82b3400640 1 -- 172.21.15.86:0/256914359 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82b4071360 msgr2=0x7f82b4157530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:25.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.923+0000 7f82b3400640 1 --2- 172.21.15.86:0/256914359 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82b4071360 0x7f82b4157530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.923+0000 7f82b3400640 1 -- 172.21.15.86:0/256914359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82b4163de0 msgr2=0x7f82b40cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:25.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.923+0000 7f82b3400640 1 --2- 172.21.15.86:0/256914359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82b4163de0 0x7f82b40cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.923+0000 7f82b3400640 1 -- 172.21.15.86:0/256914359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b40cc350 con 0x7f82b415fe00 2024-04-25T14:07:25.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.923+0000 7f82b8c00640 1 --2- 172.21.15.86:0/256914359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82b4163de0 0x7f82b40cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:25.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.927+0000 7f82b3400640 1 --2- 172.21.15.86:0/256914359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82b415fe00 0x7f82b40b61a0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f82a001a0b0 tx=0x7f82a0019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:25.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.927+0000 7f82abe00640 1 -- 172.21.15.86:0/256914359 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a00b0040 con 0x7f82b415fe00 2024-04-25T14:07:25.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.927+0000 7f82abe00640 1 -- 172.21.15.86:0/256914359 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82a00ba070 con 0x7f82b415fe00 2024-04-25T14:07:25.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.927+0000 7f82bac00640 1 -- 172.21.15.86:0/256914359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b40cca30 con 0x7f82b415fe00 2024-04-25T14:07:25.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.927+0000 7f82abe00640 1 -- 172.21.15.86:0/256914359 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a00b4650 con 0x7f82b415fe00 2024-04-25T14:07:25.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.927+0000 7f82bac00640 1 -- 172.21.15.86:0/256914359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b40ccff0 con 0x7f82b415fe00 2024-04-25T14:07:25.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.927+0000 7f82bac00640 1 -- 172.21.15.86:0/256914359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82b40720e0 con 0x7f82b415fe00 2024-04-25T14:07:25.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.927+0000 7f82abe00640 1 -- 172.21.15.86:0/256914359 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7f82a00b47f0 con 0x7f82b415fe00 2024-04-25T14:07:25.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.927+0000 7f82abe00640 1 --2- 172.21.15.86:0/256914359 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f829406f5a0 0x7f8294071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:25.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.927+0000 7f82abe00640 1 -- 172.21.15.86:0/256914359 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f82a01292e0 con 0x7f82b415fe00 2024-04-25T14:07:25.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.927+0000 7f82b3e00640 1 --2- 172.21.15.86:0/256914359 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f829406f5a0 0x7f8294071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:25.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.931+0000 7f82b3e00640 1 --2- 172.21.15.86:0/256914359 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f829406f5a0 0x7f8294071ab0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f82b406d8a0 tx=0x7f82a4009190 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:25.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:25.935+0000 7f82abe00640 1 -- 172.21.15.86:0/256914359 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f82a00c4020 con 0x7f82b415fe00 2024-04-25T14:07:26.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.123+0000 7f82bac00640 1 -- 172.21.15.86:0/256914359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82b416aa10 con 0x7f82b415fe00 2024-04-25T14:07:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.123+0000 7f82abe00640 1 -- 172.21.15.86:0/256914359 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5209 (secure 0 0 0) 0x7f82a0016070 con 0x7f82b415fe00 2024-04-25T14:07:26.125 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:07:26.125 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":57,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1210051860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1210051860},{"type":"v1","addr":"172.21.15.155:6833","nonce":1210051860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2868300438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2868300438},{"type":"v1","addr":"172.21.15.155:6835","nonce":2868300438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:06:44.910717+0000","modified":"2024-04-25T14:07:19.703362+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6603,"mds_1":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"c","rank":1,"incarnation":55,"state":"up:active","state_seq":9,"addr":"172.21.15.86:6835/2682115891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2682115891},{"type":"v1","addr":"172.21.15.86:6835","nonce":2682115891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6603":{"gid":6603,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/1088108173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1088108173},{"type":"v1","addr":"172.21.15.86:6837","nonce":1088108173}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6603,"qdb_cluster":[6603,6600]},"id":5}]} 2024-04-25T14:07:26.126 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 57 2024-04-25T14:07:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.127+0000 7f82bac00640 1 -- 172.21.15.86:0/256914359 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f829406f5a0 msgr2=0x7f8294071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.127+0000 7f82bac00640 1 --2- 172.21.15.86:0/256914359 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f829406f5a0 0x7f8294071ab0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f82b406d8a0 tx=0x7f82a4009190 comp rx=0 tx=0).stop 2024-04-25T14:07:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.127+0000 7f82bac00640 1 -- 172.21.15.86:0/256914359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82b415fe00 msgr2=0x7f82b40b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.127+0000 7f82bac00640 1 --2- 172.21.15.86:0/256914359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82b415fe00 0x7f82b40b61a0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f82a001a0b0 tx=0x7f82a0019220 comp rx=0 tx=0).stop 2024-04-25T14:07:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.127+0000 7f82bac00640 1 -- 172.21.15.86:0/256914359 shutdown_connections 2024-04-25T14:07:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.127+0000 7f82bac00640 1 --2- 172.21.15.86:0/256914359 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f829406f5a0 0x7f8294071ab0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.127+0000 7f82bac00640 1 --2- 172.21.15.86:0/256914359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82b4163de0 0x7f82b40cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.127+0000 7f82bac00640 1 --2- 172.21.15.86:0/256914359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82b415fe00 0x7f82b40b61a0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.127+0000 7f82bac00640 1 --2- 172.21.15.86:0/256914359 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82b4071360 0x7f82b4157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.127+0000 7f82bac00640 1 -- 172.21.15.86:0/256914359 >> 172.21.15.86:0/256914359 conn(0x7f82b40cffc0 msgr2=0x7f82b415edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.127+0000 7f82bac00640 1 -- 172.21.15.86:0/256914359 shutdown_connections 2024-04-25T14:07:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.127+0000 7f82bac00640 1 -- 172.21.15.86:0/256914359 wait complete. 2024-04-25T14:07:26.148 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A INPUT -p tcp --dport 6836 -j REJECT -m comment --comment teuthology 2024-04-25T14:07:26.240 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A INPUT -p tcp --dport 6837 -j REJECT -m comment --comment teuthology 2024-04-25T14:07:26.261 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A OUTPUT -p tcp --sport 6834 -j REJECT -m comment --comment teuthology 2024-04-25T14:07:26.323 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A INPUT -p tcp --dport 6834 -j REJECT -m comment --comment teuthology 2024-04-25T14:07:26.387 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A OUTPUT -p tcp --sport 6835 -j REJECT -m comment --comment teuthology 2024-04-25T14:07:26.455 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A INPUT -p tcp --dport 6835 -j REJECT -m comment --comment teuthology 2024-04-25T14:07:26.522 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:07:26.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.715+0000 7fbe81400640 1 Processor -- start 2024-04-25T14:07:26.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.715+0000 7fbe81400640 1 -- start start 2024-04-25T14:07:26.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.719+0000 7fbe81400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbe7c0d1200 0x7fbe7c0d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:26.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.719+0000 7fbe81400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbe7c0b4440 0x7fbe7c0d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:26.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.719+0000 7fbe81400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe7c0b4a30 con 0x7fbe7c0d3e70 2024-04-25T14:07:26.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.719+0000 7fbe81400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbe7c0d2850 con 0x7fbe7c0b4440 2024-04-25T14:07:26.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.719+0000 7fbe81400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fbe7c0b4d10 con 0x7fbe7c0d1200 2024-04-25T14:07:26.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.719+0000 7fbe79e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbe7c0b4440 0x7fbe7c0d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:26.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.719+0000 7fbe79e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbe7c0b4440 0x7fbe7c0d2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44824/0 (socket says 172.21.15.86:44824) 2024-04-25T14:07:26.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.719+0000 7fbe79e00640 1 -- 172.21.15.86:0/3071195469 learned_addr learned my addr 172.21.15.86:0/3071195469 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:26.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.719+0000 7fbe79e00640 1 -- 172.21.15.86:0/3071195469 >> v2:172.21.15.155:3300/0 conn(0x7fbe7c0d1200 msgr2=0x7fbe7c0d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:26.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.719+0000 7fbe79e00640 1 --2- 172.21.15.86:0/3071195469 >> v2:172.21.15.155:3300/0 conn(0x7fbe7c0d1200 0x7fbe7c0d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:26.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.719+0000 7fbe79e00640 1 -- 172.21.15.86:0/3071195469 >> v1:172.21.15.155:6789/0 conn(0x7fbe7c0d3e70 legacy=0x7fbe7c0d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:26.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.719+0000 7fbe79e00640 1 -- 172.21.15.86:0/3071195469 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe7c06d9c0 con 0x7fbe7c0b4440 2024-04-25T14:07:26.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.719+0000 7fbe79e00640 1 --2- 172.21.15.86:0/3071195469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe7c0b4440 0x7fbe7c0d2310 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c06ef20 tx=0x7fbe6801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417cf1aa5dc922e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:26.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.723+0000 7fbe79400640 1 -- 172.21.15.86:0/3071195469 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe680b0070 con 0x7fbe7c0b4440 2024-04-25T14:07:26.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.723+0000 7fbe79400640 1 -- 172.21.15.86:0/3071195469 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe680ab600 con 0x7fbe7c0b4440 2024-04-25T14:07:26.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.723+0000 7fbe79400640 1 -- 172.21.15.86:0/3071195469 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe680ab9d0 con 0x7fbe7c0b4440 2024-04-25T14:07:26.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.723+0000 7fbe81400640 1 -- 172.21.15.86:0/3071195469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe7c0b4440 msgr2=0x7fbe7c0d2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:26.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.723+0000 7fbe81400640 1 --2- 172.21.15.86:0/3071195469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe7c0b4440 0x7fbe7c0d2310 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c06ef20 tx=0x7fbe6801c450 comp rx=0 tx=0).stop 2024-04-25T14:07:26.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.723+0000 7fbe81400640 1 -- 172.21.15.86:0/3071195469 shutdown_connections 2024-04-25T14:07:26.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.723+0000 7fbe81400640 1 --2- 172.21.15.86:0/3071195469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe7c0b4440 0x7fbe7c0d2310 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:26.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.723+0000 7fbe81400640 1 --2- 172.21.15.86:0/3071195469 >> v2:172.21.15.155:3300/0 conn(0x7fbe7c0d1200 0x7fbe7c0d1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:26.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.723+0000 7fbe81400640 1 -- 172.21.15.86:0/3071195469 >> 172.21.15.86:0/3071195469 conn(0x7fbe7c0b72d0 msgr2=0x7fbe7c0b76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:26.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.723+0000 7fbe81400640 1 -- 172.21.15.86:0/3071195469 shutdown_connections 2024-04-25T14:07:26.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.723+0000 7fbe81400640 1 -- 172.21.15.86:0/3071195469 wait complete. 2024-04-25T14:07:26.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.723+0000 7fbe81400640 1 Processor -- start 2024-04-25T14:07:26.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.723+0000 7fbe81400640 1 -- start start 2024-04-25T14:07:26.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.727+0000 7fbe81400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe7c0b4440 0x7fbe7c162d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:26.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.727+0000 7fbe81400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe7c0d1200 0x7fbe7c0ce760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:26.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.727+0000 7fbe81400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe7c0d3e70 0x7fbe7c0ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:26.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.727+0000 7fbe81400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7c16d050 con 0x7fbe7c0b4440 2024-04-25T14:07:26.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.727+0000 7fbe81400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7c16ced0 con 0x7fbe7c0d1200 2024-04-25T14:07:26.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.727+0000 7fbe81400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbe7c16d1d0 con 0x7fbe7c0d3e70 2024-04-25T14:07:26.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.727+0000 7fbe7a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe7c0b4440 0x7fbe7c162d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:26.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.727+0000 7fbe7a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe7c0b4440 0x7fbe7c162d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44836/0 (socket says 172.21.15.86:44836) 2024-04-25T14:07:26.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.727+0000 7fbe7a800640 1 -- 172.21.15.86:0/1263652236 learned_addr learned my addr 172.21.15.86:0/1263652236 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:26.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.727+0000 7fbe79e00640 1 --2- 172.21.15.86:0/1263652236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe7c0d1200 0x7fbe7c0ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:26.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.727+0000 7fbe7b200640 1 --2- 172.21.15.86:0/1263652236 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe7c0d3e70 0x7fbe7c0ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:26.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.727+0000 7fbe7a800640 1 -- 172.21.15.86:0/1263652236 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe7c0d3e70 msgr2=0x7fbe7c0ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:26.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.727+0000 7fbe7a800640 1 --2- 172.21.15.86:0/1263652236 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe7c0d3e70 0x7fbe7c0ceca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:26.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.727+0000 7fbe7a800640 1 -- 172.21.15.86:0/1263652236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe7c0d1200 msgr2=0x7fbe7c0ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.727+0000 7fbe7a800640 1 --2- 172.21.15.86:0/1263652236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe7c0d1200 0x7fbe7c0ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.727+0000 7fbe7a800640 1 -- 172.21.15.86:0/1263652236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe7c14bc10 con 0x7fbe7c0b4440 2024-04-25T14:07:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.727+0000 7fbe79e00640 1 --2- 172.21.15.86:0/1263652236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe7c0d1200 0x7fbe7c0ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.727+0000 7fbe7a800640 1 --2- 172.21.15.86:0/1263652236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe7c0b4440 0x7fbe7c162d90 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c00b420 tx=0x7fbe6c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.727+0000 7fbe72a00640 1 -- 172.21.15.86:0/1263652236 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c011830 con 0x7fbe7c0b4440 2024-04-25T14:07:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.727+0000 7fbe72a00640 1 -- 172.21.15.86:0/1263652236 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe6c016070 con 0x7fbe7c0b4440 2024-04-25T14:07:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.727+0000 7fbe81400640 1 -- 172.21.15.86:0/1263652236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe7c1588e0 con 0x7fbe7c0b4440 2024-04-25T14:07:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.727+0000 7fbe72a00640 1 -- 172.21.15.86:0/1263652236 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c011c30 con 0x7fbe7c0b4440 2024-04-25T14:07:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.727+0000 7fbe81400640 1 -- 172.21.15.86:0/1263652236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe7c157fb0 con 0x7fbe7c0b4440 2024-04-25T14:07:26.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.727+0000 7fbe81400640 1 -- 172.21.15.86:0/1263652236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe7c06ef80 con 0x7fbe7c0b4440 2024-04-25T14:07:26.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.731+0000 7fbe72a00640 1 -- 172.21.15.86:0/1263652236 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7fbe6c01b460 con 0x7fbe7c0b4440 2024-04-25T14:07:26.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.731+0000 7fbe72a00640 1 --2- 172.21.15.86:0/1263652236 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fbe5c06f570 0x7fbe5c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:26.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.731+0000 7fbe72a00640 1 -- 172.21.15.86:0/1263652236 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7fbe6c08f4b0 con 0x7fbe7c0b4440 2024-04-25T14:07:26.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.731+0000 7fbe79e00640 1 --2- 172.21.15.86:0/1263652236 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fbe5c06f570 0x7fbe5c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.731+0000 7fbe79e00640 1 --2- 172.21.15.86:0/1263652236 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fbe5c06f570 0x7fbe5c071a80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbe680ad000 tx=0x7fbe680a6000 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:26.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.735+0000 7fbe72a00640 1 -- 172.21.15.86:0/1263652236 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbe6c05bfe0 con 0x7fbe7c0b4440 2024-04-25T14:07:26.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.911+0000 7fbe81400640 1 -- 172.21.15.86:0/1263652236 --> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7fbe7c076bf0 con 0x7fbe5c06f570 2024-04-25T14:07:26.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.915+0000 7fbe72a00640 1 -- 172.21.15.86:0/1263652236 <== mgr.6801 v2:172.21.15.86:6832/203029814 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3055 (secure 0 0 0) 0x7fbe7c076bf0 con 0x7fbe5c06f570 2024-04-25T14:07:26.916 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {"cephfs": {"client.5945": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:172.21.15.86"}, "client.5951": {"hostname": "smithi155", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:192.168.0.1"}, "client.6834": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}, "client.6837": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}}}, "global_metrics": {"cephfs": {"client.5945": [[415, 22], [0, 0], [0, 396500909], [0, 71840640], [34, 0], [0, 35], [35, 35], [0, 35], [0, 0], [32, 33554432], [0, 0], [0, 0], [0, 12390659], [275776813322337, 32], [0, 1122510], [133594044029089, 64]], "client.5951": [[48, 2], [0, 0], [0, 0], [0, 20679725], [6, 0], [0, 3], [3, 3], [0, 3], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 2297747], [50231033486578, 9]], "client.6834": [[2, 0], [0, 0], [0, 0], [0, 510044], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 510043], [0, 1]], "client.6837": [[2, 0], [0, 0], [0, 0], [0, 761751], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 761751], [0, 1]]}}, "metrics": {"delayed_ranks": [], "mds.0": {"client.5945": [], "client.5951": [], "client.6834": [], "client.6837": []}, "mds.1": {"client.5945": [], "client.5951": []}}} 2024-04-25T14:07:26.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.915+0000 7fbe81400640 1 -- 172.21.15.86:0/1263652236 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fbe5c06f570 msgr2=0x7fbe5c071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:26.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.915+0000 7fbe81400640 1 --2- 172.21.15.86:0/1263652236 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fbe5c06f570 0x7fbe5c071a80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbe680ad000 tx=0x7fbe680a6000 comp rx=0 tx=0).stop 2024-04-25T14:07:26.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.915+0000 7fbe81400640 1 -- 172.21.15.86:0/1263652236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe7c0b4440 msgr2=0x7fbe7c162d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:26.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.915+0000 7fbe81400640 1 --2- 172.21.15.86:0/1263652236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe7c0b4440 0x7fbe7c162d90 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c00b420 tx=0x7fbe6c00b8e0 comp rx=0 tx=0).stop 2024-04-25T14:07:26.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.915+0000 7fbe81400640 1 -- 172.21.15.86:0/1263652236 shutdown_connections 2024-04-25T14:07:26.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.915+0000 7fbe81400640 1 --2- 172.21.15.86:0/1263652236 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fbe5c06f570 0x7fbe5c071a80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:26.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.915+0000 7fbe81400640 1 --2- 172.21.15.86:0/1263652236 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe7c0d3e70 0x7fbe7c0ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:26.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.915+0000 7fbe81400640 1 --2- 172.21.15.86:0/1263652236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe7c0d1200 0x7fbe7c0ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:26.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.915+0000 7fbe81400640 1 --2- 172.21.15.86:0/1263652236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe7c0b4440 0x7fbe7c162d90 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:26.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.915+0000 7fbe81400640 1 -- 172.21.15.86:0/1263652236 >> 172.21.15.86:0/1263652236 conn(0x7fbe7c0b72d0 msgr2=0x7fbe7c0b9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:26.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.915+0000 7fbe81400640 1 -- 172.21.15.86:0/1263652236 shutdown_connections 2024-04-25T14:07:26.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:26.915+0000 7fbe81400640 1 -- 172.21.15.86:0/1263652236 wait complete. 2024-04-25T14:07:27.938 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:07:28.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.095+0000 7f45de400640 1 Processor -- start 2024-04-25T14:07:28.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.095+0000 7f45de400640 1 -- start start 2024-04-25T14:07:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.095+0000 7f45de400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f45d806ebb0 0x7f45d8070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.095+0000 7f45de400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45d8071360 0x7f45d8163e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.095+0000 7f45de400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45d8071770 con 0x7f45d806e7a0 2024-04-25T14:07:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.095+0000 7f45de400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f45d80d1c10 con 0x7f45d806ebb0 2024-04-25T14:07:28.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.095+0000 7f45de400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f45d80d1ef0 con 0x7f45d8071360 2024-04-25T14:07:28.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.095+0000 7f45cea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f45d806ebb0 0x7f45d8070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:28.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.095+0000 7f45cea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f45d806ebb0 0x7f45d8070e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44846/0 (socket says 172.21.15.86:44846) 2024-04-25T14:07:28.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.095+0000 7f45cea00640 1 -- 172.21.15.86:0/2268866196 learned_addr learned my addr 172.21.15.86:0/2268866196 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:28.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.095+0000 7f45d7e00640 1 --2- 172.21.15.86:0/2268866196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45d8071360 0x7f45d8163e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:28.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.095+0000 7f45cea00640 1 -- 172.21.15.86:0/2268866196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45d8071360 msgr2=0x7f45d8163e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:28.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.095+0000 7f45cea00640 1 --2- 172.21.15.86:0/2268866196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45d8071360 0x7f45d8163e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:28.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.095+0000 7f45cea00640 1 -- 172.21.15.86:0/2268866196 >> v1:172.21.15.155:6789/0 conn(0x7f45d806e7a0 legacy=0x7f45d81602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:28.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.095+0000 7f45cea00640 1 -- 172.21.15.86:0/2268866196 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d81609b0 con 0x7f45d806ebb0 2024-04-25T14:07:28.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.099+0000 7f45cea00640 1 --2- 172.21.15.86:0/2268866196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d806ebb0 0x7f45d8070e20 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f45c8006fd0 tx=0x7f45c801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c4019f037f3813 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:28.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.099+0000 7f45d6a00640 1 -- 172.21.15.86:0/2268866196 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c80b0070 con 0x7f45d806ebb0 2024-04-25T14:07:28.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.099+0000 7f45d6a00640 1 -- 172.21.15.86:0/2268866196 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45c80ab600 con 0x7f45d806ebb0 2024-04-25T14:07:28.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.099+0000 7f45d6a00640 1 -- 172.21.15.86:0/2268866196 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c80ab9d0 con 0x7f45d806ebb0 2024-04-25T14:07:28.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.099+0000 7f45de400640 1 -- 172.21.15.86:0/2268866196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d806ebb0 msgr2=0x7f45d8070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:28.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.099+0000 7f45de400640 1 --2- 172.21.15.86:0/2268866196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d806ebb0 0x7f45d8070e20 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f45c8006fd0 tx=0x7f45c801c4b0 comp rx=0 tx=0).stop 2024-04-25T14:07:28.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.099+0000 7f45de400640 1 -- 172.21.15.86:0/2268866196 shutdown_connections 2024-04-25T14:07:28.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.099+0000 7f45de400640 1 --2- 172.21.15.86:0/2268866196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45d8071360 0x7f45d8163e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:28.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.099+0000 7f45de400640 1 --2- 172.21.15.86:0/2268866196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d806ebb0 0x7f45d8070e20 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:28.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.099+0000 7f45de400640 1 -- 172.21.15.86:0/2268866196 >> 172.21.15.86:0/2268866196 conn(0x7f45d80cffc0 msgr2=0x7f45d80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:28.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.099+0000 7f45de400640 1 -- 172.21.15.86:0/2268866196 shutdown_connections 2024-04-25T14:07:28.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.099+0000 7f45de400640 1 -- 172.21.15.86:0/2268866196 wait complete. 2024-04-25T14:07:28.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.103+0000 7f45de400640 1 Processor -- start 2024-04-25T14:07:28.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.103+0000 7f45de400640 1 -- start start 2024-04-25T14:07:28.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.103+0000 7f45de400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45d806e7a0 0x7f45d80b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:28.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.103+0000 7f45de400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d806ebb0 0x7f45d81564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:28.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.103+0000 7f45de400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45d8071360 0x7f45d8156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:28.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.103+0000 7f45de400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f45d816d5c0 con 0x7f45d806ebb0 2024-04-25T14:07:28.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.103+0000 7f45de400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f45d816d440 con 0x7f45d8071360 2024-04-25T14:07:28.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.103+0000 7f45de400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f45d816d740 con 0x7f45d806e7a0 2024-04-25T14:07:28.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.103+0000 7f45cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d806ebb0 0x7f45d81564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:28.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.103+0000 7f45cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d806ebb0 0x7f45d81564c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44854/0 (socket says 172.21.15.86:44854) 2024-04-25T14:07:28.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.103+0000 7f45cea00640 1 -- 172.21.15.86:0/3676319308 learned_addr learned my addr 172.21.15.86:0/3676319308 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:28.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.103+0000 7f45d7e00640 1 --2- 172.21.15.86:0/3676319308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45d8071360 0x7f45d8156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:28.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.103+0000 7f45d7400640 1 --2- 172.21.15.86:0/3676319308 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45d806e7a0 0x7f45d80b77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:28.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.103+0000 7f45cea00640 1 -- 172.21.15.86:0/3676319308 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45d806e7a0 msgr2=0x7f45d80b77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:28.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.103+0000 7f45cea00640 1 --2- 172.21.15.86:0/3676319308 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45d806e7a0 0x7f45d80b77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:28.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.103+0000 7f45cea00640 1 -- 172.21.15.86:0/3676319308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45d8071360 msgr2=0x7f45d8156a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:28.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.103+0000 7f45cea00640 1 --2- 172.21.15.86:0/3676319308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45d8071360 0x7f45d8156a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:28.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.103+0000 7f45cea00640 1 -- 172.21.15.86:0/3676319308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d80cb030 con 0x7f45d806ebb0 2024-04-25T14:07:28.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.103+0000 7f45cea00640 1 --2- 172.21.15.86:0/3676319308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d806ebb0 0x7f45d81564c0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f45c801a0e0 tx=0x7f45c8019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:28.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.103+0000 7f45cfe00640 1 -- 172.21.15.86:0/3676319308 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c80b9070 con 0x7f45d806ebb0 2024-04-25T14:07:28.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.103+0000 7f45cfe00640 1 -- 172.21.15.86:0/3676319308 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45c80ab9b0 con 0x7f45d806ebb0 2024-04-25T14:07:28.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.103+0000 7f45de400640 1 -- 172.21.15.86:0/3676319308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d8157260 con 0x7f45d806ebb0 2024-04-25T14:07:28.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.103+0000 7f45cfe00640 1 -- 172.21.15.86:0/3676319308 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c80b0040 con 0x7f45d806ebb0 2024-04-25T14:07:28.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.103+0000 7f45de400640 1 -- 172.21.15.86:0/3676319308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d80cd050 con 0x7f45d806ebb0 2024-04-25T14:07:28.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.107+0000 7f45de400640 1 -- 172.21.15.86:0/3676319308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45d80720e0 con 0x7f45d806ebb0 2024-04-25T14:07:28.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.107+0000 7f45cfe00640 1 -- 172.21.15.86:0/3676319308 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7f45c80bc070 con 0x7f45d806ebb0 2024-04-25T14:07:28.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.107+0000 7f45cfe00640 1 --2- 172.21.15.86:0/3676319308 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f45b406f5a0 0x7f45b4071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:28.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.107+0000 7f45cfe00640 1 -- 172.21.15.86:0/3676319308 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f45c8128910 con 0x7f45d806ebb0 2024-04-25T14:07:28.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.107+0000 7f45d7400640 1 --2- 172.21.15.86:0/3676319308 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f45b406f5a0 0x7f45b4071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:28.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.107+0000 7f45d7400640 1 --2- 172.21.15.86:0/3676319308 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f45b406f5a0 0x7f45b4071ab0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f45c00015b0 tx=0x7f45c0009040 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:28.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.111+0000 7f45cfe00640 1 -- 172.21.15.86:0/3676319308 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f45c8016130 con 0x7f45d806ebb0 2024-04-25T14:07:28.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.291+0000 7f45de400640 1 -- 172.21.15.86:0/3676319308 --> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f45d8157010 con 0x7f45b406f5a0 2024-04-25T14:07:28.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.295+0000 7f45cfe00640 1 -- 172.21.15.86:0/3676319308 <== mgr.6801 v2:172.21.15.86:6832/203029814 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3055 (secure 0 0 0) 0x7f45d8157010 con 0x7f45b406f5a0 2024-04-25T14:07:28.296 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {"cephfs": {"client.5945": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:172.21.15.86"}, "client.5951": {"hostname": "smithi155", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:192.168.0.1"}, "client.6834": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}, "client.6837": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}}}, "global_metrics": {"cephfs": {"client.5945": [[415, 22], [0, 0], [0, 396500909], [0, 71840640], [34, 0], [0, 35], [35, 35], [0, 35], [0, 0], [32, 33554432], [0, 0], [0, 0], [0, 12390659], [275776813322337, 32], [0, 1122510], [133594044029089, 64]], "client.5951": [[48, 2], [0, 0], [0, 0], [0, 20679725], [6, 0], [0, 3], [3, 3], [0, 3], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 2297747], [50231033486578, 9]], "client.6834": [[2, 0], [0, 0], [0, 0], [0, 510044], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 510043], [0, 1]], "client.6837": [[2, 0], [0, 0], [0, 0], [0, 761751], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 761751], [0, 1]]}}, "metrics": {"delayed_ranks": [], "mds.0": {"client.5945": [], "client.5951": [], "client.6834": [], "client.6837": []}, "mds.1": {"client.5945": [], "client.5951": []}}} 2024-04-25T14:07:28.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.295+0000 7f45de400640 1 -- 172.21.15.86:0/3676319308 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f45b406f5a0 msgr2=0x7f45b4071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:28.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.295+0000 7f45de400640 1 --2- 172.21.15.86:0/3676319308 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f45b406f5a0 0x7f45b4071ab0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f45c00015b0 tx=0x7f45c0009040 comp rx=0 tx=0).stop 2024-04-25T14:07:28.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.295+0000 7f45de400640 1 -- 172.21.15.86:0/3676319308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d806ebb0 msgr2=0x7f45d81564c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:28.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.295+0000 7f45de400640 1 --2- 172.21.15.86:0/3676319308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d806ebb0 0x7f45d81564c0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f45c801a0e0 tx=0x7f45c8019250 comp rx=0 tx=0).stop 2024-04-25T14:07:28.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.295+0000 7f45de400640 1 -- 172.21.15.86:0/3676319308 shutdown_connections 2024-04-25T14:07:28.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.295+0000 7f45de400640 1 --2- 172.21.15.86:0/3676319308 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f45b406f5a0 0x7f45b4071ab0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:28.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.295+0000 7f45de400640 1 --2- 172.21.15.86:0/3676319308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45d8071360 0x7f45d8156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:28.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.295+0000 7f45de400640 1 --2- 172.21.15.86:0/3676319308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d806ebb0 0x7f45d81564c0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:28.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.295+0000 7f45de400640 1 --2- 172.21.15.86:0/3676319308 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45d806e7a0 0x7f45d80b77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:28.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.295+0000 7f45de400640 1 -- 172.21.15.86:0/3676319308 >> 172.21.15.86:0/3676319308 conn(0x7f45d80cffc0 msgr2=0x7f45d80b5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:28.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.295+0000 7f45de400640 1 -- 172.21.15.86:0/3676319308 shutdown_connections 2024-04-25T14:07:28.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:28.295+0000 7f45de400640 1 -- 172.21.15.86:0/3676319308 wait complete. 2024-04-25T14:07:29.318 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:07:29.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.475+0000 7f8f99400640 1 Processor -- start 2024-04-25T14:07:29.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.475+0000 7f8f99400640 1 -- start start 2024-04-25T14:07:29.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f99400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8f940d16e0 0x7f8f940d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:29.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f99400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f940d2030 con 0x7f8f940cfca0 2024-04-25T14:07:29.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f99400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f94072bd0 con 0x7f8f940730b0 2024-04-25T14:07:29.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f99400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f8f94070470 con 0x7f8f940d16e0 2024-04-25T14:07:29.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f92800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8f940cfca0 0x7f8f9406f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41136/0 (socket says 172.21.15.86:41136) 2024-04-25T14:07:29.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f92800640 1 -- 172.21.15.86:0/3696866739 learned_addr learned my addr 172.21.15.86:0/3696866739 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:07:29.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f89e00640 1 --2- 172.21.15.86:0/3696866739 >> v2:172.21.15.155:3300/0 conn(0x7f8f940d16e0 0x7f8f940d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:29.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f91e00640 1 -- 172.21.15.86:0/3696866739 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2874843293 0 0) 0x7f8f84001120 con 0x7f8f940cfca0 2024-04-25T14:07:29.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f91e00640 1 -- 172.21.15.86:0/3696866739 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f78003530 con 0x7f8f940cfca0 2024-04-25T14:07:29.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f91e00640 1 -- 172.21.15.86:0/3696866739 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1196254675 0 0) 0x7f8f78003530 con 0x7f8f940cfca0 2024-04-25T14:07:29.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f91e00640 1 -- 172.21.15.86:0/3696866739 >> v2:172.21.15.155:3300/0 conn(0x7f8f940d16e0 msgr2=0x7f8f940d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:29.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f91e00640 1 --2- 172.21.15.86:0/3696866739 >> v2:172.21.15.155:3300/0 conn(0x7f8f940d16e0 0x7f8f940d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:29.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f91e00640 1 -- 172.21.15.86:0/3696866739 >> v1:172.21.15.155:6789/0 conn(0x7f8f940730b0 legacy=0x7f8f940734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:29.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f91e00640 1 -- 172.21.15.86:0/3696866739 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f9406fbd0 con 0x7f8f940cfca0 2024-04-25T14:07:29.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f91e00640 1 -- 172.21.15.86:0/3696866739 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f8f84001870 con 0x7f8f940cfca0 2024-04-25T14:07:29.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f91e00640 1 -- 172.21.15.86:0/3696866739 >> v1:172.21.15.86:6789/0 conn(0x7f8f940cfca0 legacy=0x7f8f9406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:29.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f91e00640 1 --2- 172.21.15.86:0/3696866739 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f78003a30 0x7f8f78003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:29.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f91e00640 1 --2- 172.21.15.86:0/3696866739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f78004810 0x7f8f78004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:29.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f91e00640 1 --2- 172.21.15.86:0/3696866739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f780051b0 0x7f8f7800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:29.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f91e00640 1 -- 172.21.15.86:0/3696866739 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f94070470 con 0x7f8f78004810 2024-04-25T14:07:29.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f91e00640 1 -- 172.21.15.86:0/3696866739 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7800c150 con 0x7f8f780051b0 2024-04-25T14:07:29.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f91e00640 1 -- 172.21.15.86:0/3696866739 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8f7800c350 con 0x7f8f78003a30 2024-04-25T14:07:29.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f89e00640 1 --2- 172.21.15.86:0/3696866739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f78004810 0x7f8f78004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:29.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f92800640 1 --2- 172.21.15.86:0/3696866739 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f78003a30 0x7f8f78003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:29.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f93200640 1 --2- 172.21.15.86:0/3696866739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f780051b0 0x7f8f7800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:29.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f89e00640 1 -- 172.21.15.86:0/3696866739 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f78003a30 msgr2=0x7f8f78003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:29.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f89e00640 1 --2- 172.21.15.86:0/3696866739 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f78003a30 0x7f8f78003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:29.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f89e00640 1 -- 172.21.15.86:0/3696866739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f780051b0 msgr2=0x7f8f7800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:29.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f89e00640 1 --2- 172.21.15.86:0/3696866739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f780051b0 0x7f8f7800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:29.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f89e00640 1 -- 172.21.15.86:0/3696866739 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f78003530 con 0x7f8f78004810 2024-04-25T14:07:29.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.479+0000 7f8f93200640 1 -- 172.21.15.86:0/3696866739 reap_dead start 2024-04-25T14:07:29.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.483+0000 7f8f89e00640 1 --2- 172.21.15.86:0/3696866739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f78004810 0x7f8f78004c20 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f8f78004790 tx=0x7f8f7c01ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a2523ce902b970 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:29.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.483+0000 7f8f91e00640 1 -- 172.21.15.86:0/3696866739 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c0b2070 con 0x7f8f78004810 2024-04-25T14:07:29.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.483+0000 7f8f91e00640 1 -- 172.21.15.86:0/3696866739 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f7c0a5600 con 0x7f8f78004810 2024-04-25T14:07:29.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.483+0000 7f8f91e00640 1 -- 172.21.15.86:0/3696866739 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c0096b0 con 0x7f8f78004810 2024-04-25T14:07:29.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.483+0000 7f8f99400640 1 -- 172.21.15.86:0/3696866739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f78004810 msgr2=0x7f8f78004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:29.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.483+0000 7f8f99400640 1 --2- 172.21.15.86:0/3696866739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f78004810 0x7f8f78004c20 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f8f78004790 tx=0x7f8f7c01ca10 comp rx=0 tx=0).stop 2024-04-25T14:07:29.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.483+0000 7f8f99400640 1 -- 172.21.15.86:0/3696866739 shutdown_connections 2024-04-25T14:07:29.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.483+0000 7f8f99400640 1 --2- 172.21.15.86:0/3696866739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f78004810 0x7f8f78004c20 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:29.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.483+0000 7f8f99400640 1 -- 172.21.15.86:0/3696866739 >> 172.21.15.86:0/3696866739 conn(0x7f8f940b4650 msgr2=0x7f8f940b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:29.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.483+0000 7f8f99400640 1 -- 172.21.15.86:0/3696866739 shutdown_connections 2024-04-25T14:07:29.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.483+0000 7f8f99400640 1 -- 172.21.15.86:0/3696866739 wait complete. 2024-04-25T14:07:29.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.487+0000 7f8f99400640 1 Processor -- start 2024-04-25T14:07:29.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.487+0000 7f8f99400640 1 -- start start 2024-04-25T14:07:29.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.487+0000 7f8f99400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f78004810 0x7f8f9415b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:29.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.487+0000 7f8f99400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f9415f2c0 0x7f8f940b1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:29.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.487+0000 7f8f99400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f940b1ca0 0x7f8f9409d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:29.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.487+0000 7f8f99400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f940d1af0 con 0x7f8f78004810 2024-04-25T14:07:29.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.487+0000 7f8f99400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8f94070190 con 0x7f8f9415f2c0 2024-04-25T14:07:29.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.487+0000 7f8f99400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8f940d1c70 con 0x7f8f940b1ca0 2024-04-25T14:07:29.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.487+0000 7f8f92800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f78004810 0x7f8f9415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:29.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.487+0000 7f8f92800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f78004810 0x7f8f9415b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48992/0 (socket says 172.21.15.86:48992) 2024-04-25T14:07:29.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.487+0000 7f8f92800640 1 -- 172.21.15.86:0/359447635 learned_addr learned my addr 172.21.15.86:0/359447635 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:29.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.487+0000 7f8f93200640 1 --2- 172.21.15.86:0/359447635 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f940b1ca0 0x7f8f9409d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:29.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.487+0000 7f8f89e00640 1 --2- 172.21.15.86:0/359447635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f9415f2c0 0x7f8f940b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:29.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.487+0000 7f8f92800640 1 -- 172.21.15.86:0/359447635 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f940b1ca0 msgr2=0x7f8f9409d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:29.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.487+0000 7f8f92800640 1 --2- 172.21.15.86:0/359447635 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f940b1ca0 0x7f8f9409d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:29.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.487+0000 7f8f92800640 1 -- 172.21.15.86:0/359447635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f9415f2c0 msgr2=0x7f8f940b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:29.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.487+0000 7f8f92800640 1 --2- 172.21.15.86:0/359447635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f9415f2c0 0x7f8f940b1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:29.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.487+0000 7f8f92800640 1 -- 172.21.15.86:0/359447635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f940b20b0 con 0x7f8f78004810 2024-04-25T14:07:29.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.487+0000 7f8f89e00640 1 --2- 172.21.15.86:0/359447635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f9415f2c0 0x7f8f940b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:29.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.487+0000 7f8f93200640 1 --2- 172.21.15.86:0/359447635 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f940b1ca0 0x7f8f9409d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:29.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.487+0000 7f8f92800640 1 --2- 172.21.15.86:0/359447635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f78004810 0x7f8f9415b200 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f8f84005ec0 tx=0x7f8f84007420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:29.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.487+0000 7f8f8b400640 1 -- 172.21.15.86:0/359447635 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8400e740 con 0x7f8f78004810 2024-04-25T14:07:29.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.487+0000 7f8f8b400640 1 -- 172.21.15.86:0/359447635 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f84018070 con 0x7f8f78004810 2024-04-25T14:07:29.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.487+0000 7f8f8b400640 1 -- 172.21.15.86:0/359447635 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8400eb20 con 0x7f8f78004810 2024-04-25T14:07:29.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.487+0000 7f8f99400640 1 -- 172.21.15.86:0/359447635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f9409bcc0 con 0x7f8f78004810 2024-04-25T14:07:29.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.487+0000 7f8f99400640 1 -- 172.21.15.86:0/359447635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f9409c4a0 con 0x7f8f78004810 2024-04-25T14:07:29.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.491+0000 7f8f99400640 1 -- 172.21.15.86:0/359447635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f9409f4b0 con 0x7f8f78004810 2024-04-25T14:07:29.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.491+0000 7f8f8b400640 1 -- 172.21.15.86:0/359447635 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7f8f8400ecc0 con 0x7f8f78004810 2024-04-25T14:07:29.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.491+0000 7f8f8b400640 1 --2- 172.21.15.86:0/359447635 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f8f7006f570 0x7f8f70071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:29.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.491+0000 7f8f8b400640 1 -- 172.21.15.86:0/359447635 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f8f8408f770 con 0x7f8f78004810 2024-04-25T14:07:29.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.491+0000 7f8f89e00640 1 --2- 172.21.15.86:0/359447635 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f8f7006f570 0x7f8f70071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:29.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.495+0000 7f8f89e00640 1 --2- 172.21.15.86:0/359447635 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f8f7006f570 0x7f8f70071a80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0af000 tx=0x7f8f7c008040 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:29.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.495+0000 7f8f8b400640 1 -- 172.21.15.86:0/359447635 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8f940b20b0 con 0x7f8f78004810 2024-04-25T14:07:29.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.675+0000 7f8f99400640 1 -- 172.21.15.86:0/359447635 --> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f8f94069d00 con 0x7f8f7006f570 2024-04-25T14:07:29.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.679+0000 7f8f8b400640 1 -- 172.21.15.86:0/359447635 <== mgr.6801 v2:172.21.15.86:6832/203029814 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3051 (secure 0 0 0) 0x7f8f94069d00 con 0x7f8f7006f570 2024-04-25T14:07:29.680 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {"cephfs": {"client.5945": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:172.21.15.86"}, "client.5951": {"hostname": "smithi155", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:192.168.0.1"}, "client.6834": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}, "client.6837": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}}}, "global_metrics": {"cephfs": {"client.5945": [[0, 0], [0, 0], [0, 396500909], [0, 71840640], [34, 0], [0, 35], [35, 35], [0, 35], [0, 0], [32, 33554432], [0, 0], [0, 0], [0, 12390659], [275776813322337, 32], [0, 1122510], [133594044029089, 64]], "client.5951": [[0, 0], [0, 0], [0, 0], [0, 20679725], [6, 0], [0, 3], [3, 3], [0, 3], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 2297747], [50231033486578, 9]], "client.6834": [[0, 0], [0, 0], [0, 0], [0, 510044], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 510043], [0, 1]], "client.6837": [[0, 0], [0, 0], [0, 0], [0, 761751], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 761751], [0, 1]]}}, "metrics": {"delayed_ranks": [], "mds.0": {"client.5945": [], "client.5951": [], "client.6834": [], "client.6837": []}, "mds.1": {"client.5945": [], "client.5951": []}}} 2024-04-25T14:07:29.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.679+0000 7f8f99400640 1 -- 172.21.15.86:0/359447635 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f8f7006f570 msgr2=0x7f8f70071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:29.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.679+0000 7f8f99400640 1 --2- 172.21.15.86:0/359447635 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f8f7006f570 0x7f8f70071a80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0af000 tx=0x7f8f7c008040 comp rx=0 tx=0).stop 2024-04-25T14:07:29.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.679+0000 7f8f99400640 1 -- 172.21.15.86:0/359447635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f78004810 msgr2=0x7f8f9415b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:29.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.679+0000 7f8f99400640 1 --2- 172.21.15.86:0/359447635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f78004810 0x7f8f9415b200 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f8f84005ec0 tx=0x7f8f84007420 comp rx=0 tx=0).stop 2024-04-25T14:07:29.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.679+0000 7f8f99400640 1 -- 172.21.15.86:0/359447635 shutdown_connections 2024-04-25T14:07:29.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.679+0000 7f8f99400640 1 --2- 172.21.15.86:0/359447635 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f8f7006f570 0x7f8f70071a80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:29.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.679+0000 7f8f99400640 1 --2- 172.21.15.86:0/359447635 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f940b1ca0 0x7f8f9409d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:29.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.679+0000 7f8f99400640 1 --2- 172.21.15.86:0/359447635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f9415f2c0 0x7f8f940b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:29.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.679+0000 7f8f99400640 1 --2- 172.21.15.86:0/359447635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f78004810 0x7f8f9415b200 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:29.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.679+0000 7f8f99400640 1 -- 172.21.15.86:0/359447635 >> 172.21.15.86:0/359447635 conn(0x7f8f940b4650 msgr2=0x7f8f9415c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:29.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.679+0000 7f8f99400640 1 -- 172.21.15.86:0/359447635 shutdown_connections 2024-04-25T14:07:29.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:29.679+0000 7f8f99400640 1 -- 172.21.15.86:0/359447635 wait complete. 2024-04-25T14:07:30.702 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:07:30.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.859+0000 7f3104200640 1 Processor -- start 2024-04-25T14:07:30.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.859+0000 7f3104200640 1 -- start start 2024-04-25T14:07:30.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.863+0000 7f3104200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f30fc0d16e0 0x7f30fc0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:30.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.863+0000 7f3104200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30fc0d2030 con 0x7f30fc0cfca0 2024-04-25T14:07:30.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.863+0000 7f3104200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30fc072bd0 con 0x7f30fc0730b0 2024-04-25T14:07:30.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.863+0000 7f3104200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f30fc070470 con 0x7f30fc0d16e0 2024-04-25T14:07:30.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.863+0000 7f3101800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f30fc0cfca0 0x7f30fc06f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41142/0 (socket says 172.21.15.86:41142) 2024-04-25T14:07:30.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.863+0000 7f3101800640 1 -- 172.21.15.86:0/3162987371 learned_addr learned my addr 172.21.15.86:0/3162987371 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:07:30.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.863+0000 7f3100e00640 1 --2- 172.21.15.86:0/3162987371 >> v2:172.21.15.86:3300/0 conn(0x7f30fc0d16e0 0x7f30fc0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:30.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.863+0000 7f3100e00640 1 -- 172.21.15.86:0/3162987371 >> v1:172.21.15.155:6789/0 conn(0x7f30fc0730b0 legacy=0x7f30fc0734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:30.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.863+0000 7f3100e00640 1 -- 172.21.15.86:0/3162987371 >> v1:172.21.15.86:6789/0 conn(0x7f30fc0cfca0 legacy=0x7f30fc06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:30.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.863+0000 7f3100e00640 1 -- 172.21.15.86:0/3162987371 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30fc06fbd0 con 0x7f30fc0d16e0 2024-04-25T14:07:30.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.867+0000 7f3100e00640 1 --2- 172.21.15.86:0/3162987371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30fc0d16e0 0x7f30fc0d1af0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f30e80097c0 tx=0x7f30e801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f50b48f59ba01d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:30.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.867+0000 7f30fbe00640 1 -- 172.21.15.86:0/3162987371 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e80b1070 con 0x7f30fc0d16e0 2024-04-25T14:07:30.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.867+0000 7f30fbe00640 1 -- 172.21.15.86:0/3162987371 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e80ac410 con 0x7f30fc0d16e0 2024-04-25T14:07:30.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.867+0000 7f30fbe00640 1 -- 172.21.15.86:0/3162987371 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30fc0743e0 con 0x7f30fc0d16e0 2024-04-25T14:07:30.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.867+0000 7f3104200640 1 -- 172.21.15.86:0/3162987371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30fc0d16e0 msgr2=0x7f30fc0d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:30.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.867+0000 7f3104200640 1 --2- 172.21.15.86:0/3162987371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30fc0d16e0 0x7f30fc0d1af0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f30e80097c0 tx=0x7f30e801c4a0 comp rx=0 tx=0).stop 2024-04-25T14:07:30.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.867+0000 7f3104200640 1 -- 172.21.15.86:0/3162987371 shutdown_connections 2024-04-25T14:07:30.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.867+0000 7f3104200640 1 --2- 172.21.15.86:0/3162987371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30fc0d16e0 0x7f30fc0d1af0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:30.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.867+0000 7f3104200640 1 -- 172.21.15.86:0/3162987371 >> 172.21.15.86:0/3162987371 conn(0x7f30fc0b4650 msgr2=0x7f30fc0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:30.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.867+0000 7f3104200640 1 -- 172.21.15.86:0/3162987371 shutdown_connections 2024-04-25T14:07:30.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.867+0000 7f3104200640 1 -- 172.21.15.86:0/3162987371 wait complete. 2024-04-25T14:07:30.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.871+0000 7f3104200640 1 Processor -- start 2024-04-25T14:07:30.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.871+0000 7f3104200640 1 -- start start 2024-04-25T14:07:30.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.871+0000 7f3104200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30fc0730b0 0x7f30fc15a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:30.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.871+0000 7f3104200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30fc0cfca0 0x7f30fc0995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:30.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.871+0000 7f3104200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30fc0d16e0 0x7f30fc099b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:30.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.871+0000 7f3104200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc16aa80 con 0x7f30fc0d16e0 2024-04-25T14:07:30.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.871+0000 7f3104200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc16a900 con 0x7f30fc0730b0 2024-04-25T14:07:30.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.871+0000 7f3104200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f30fc16ac00 con 0x7f30fc0cfca0 2024-04-25T14:07:30.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.871+0000 7f3102200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30fc0d16e0 0x7f30fc099b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:30.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.871+0000 7f3102200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30fc0d16e0 0x7f30fc099b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49016/0 (socket says 172.21.15.86:49016) 2024-04-25T14:07:30.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.871+0000 7f3102200640 1 -- 172.21.15.86:0/492195233 learned_addr learned my addr 172.21.15.86:0/492195233 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:30.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.871+0000 7f3101800640 1 --2- 172.21.15.86:0/492195233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30fc0730b0 0x7f30fc15a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:30.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.871+0000 7f3100e00640 1 --2- 172.21.15.86:0/492195233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30fc0cfca0 0x7f30fc0995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:30.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.871+0000 7f3102200640 1 -- 172.21.15.86:0/492195233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30fc0cfca0 msgr2=0x7f30fc0995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:30.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.871+0000 7f3102200640 1 --2- 172.21.15.86:0/492195233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30fc0cfca0 0x7f30fc0995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:30.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.871+0000 7f3102200640 1 -- 172.21.15.86:0/492195233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30fc0730b0 msgr2=0x7f30fc15a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:30.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.871+0000 7f3102200640 1 --2- 172.21.15.86:0/492195233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30fc0730b0 0x7f30fc15a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:30.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.871+0000 7f3102200640 1 -- 172.21.15.86:0/492195233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30fc15b110 con 0x7f30fc0d16e0 2024-04-25T14:07:30.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.871+0000 7f3102200640 1 --2- 172.21.15.86:0/492195233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30fc0d16e0 0x7f30fc099b10 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f30f400c760 tx=0x7f30f400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:30.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.871+0000 7f30f9600640 1 -- 172.21.15.86:0/492195233 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f4003b80 con 0x7f30fc0d16e0 2024-04-25T14:07:30.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.871+0000 7f30f9600640 1 -- 172.21.15.86:0/492195233 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30f4003d20 con 0x7f30fc0d16e0 2024-04-25T14:07:30.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.871+0000 7f3104200640 1 -- 172.21.15.86:0/492195233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30fc159b50 con 0x7f30fc0d16e0 2024-04-25T14:07:30.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.871+0000 7f30f9600640 1 -- 172.21.15.86:0/492195233 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f400aa90 con 0x7f30fc0d16e0 2024-04-25T14:07:30.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.871+0000 7f3104200640 1 -- 172.21.15.86:0/492195233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30fc0b2280 con 0x7f30fc0d16e0 2024-04-25T14:07:30.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.875+0000 7f3104200640 1 -- 172.21.15.86:0/492195233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30fc0b1ff0 con 0x7f30fc0d16e0 2024-04-25T14:07:30.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.875+0000 7f30f9600640 1 -- 172.21.15.86:0/492195233 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7f30f400ac30 con 0x7f30fc0d16e0 2024-04-25T14:07:30.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.875+0000 7f30f9600640 1 --2- 172.21.15.86:0/492195233 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f30dc06f570 0x7f30dc071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:30.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.875+0000 7f30f9600640 1 -- 172.21.15.86:0/492195233 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f30f408f4f0 con 0x7f30fc0d16e0 2024-04-25T14:07:30.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.875+0000 7f3101800640 1 --2- 172.21.15.86:0/492195233 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f30dc06f570 0x7f30dc071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:30.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.875+0000 7f3101800640 1 --2- 172.21.15.86:0/492195233 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f30dc06f570 0x7f30dc071a80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f30f0000f70 tx=0x7f30f0009040 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:30.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:30.879+0000 7f30f9600640 1 -- 172.21.15.86:0/492195233 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f30f405b940 con 0x7f30fc0d16e0 2024-04-25T14:07:31.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:31.059+0000 7f3104200640 1 -- 172.21.15.86:0/492195233 --> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f30fc09f4b0 con 0x7f30dc06f570 2024-04-25T14:07:31.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:31.063+0000 7f30f9600640 1 -- 172.21.15.86:0/492195233 <== mgr.6801 v2:172.21.15.86:6832/203029814 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3051 (secure 0 0 0) 0x7f30fc09f4b0 con 0x7f30dc06f570 2024-04-25T14:07:31.064 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {"cephfs": {"client.5945": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:172.21.15.86"}, "client.5951": {"hostname": "smithi155", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:192.168.0.1"}, "client.6834": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}, "client.6837": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}}}, "global_metrics": {"cephfs": {"client.5945": [[0, 0], [0, 0], [0, 396500909], [0, 71840640], [34, 0], [0, 35], [35, 35], [0, 35], [0, 0], [32, 33554432], [0, 0], [0, 0], [0, 12390659], [275776813322337, 32], [0, 1122510], [133594044029089, 64]], "client.5951": [[0, 0], [0, 0], [0, 0], [0, 20679725], [6, 0], [0, 3], [3, 3], [0, 3], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 2297747], [50231033486578, 9]], "client.6834": [[0, 0], [0, 0], [0, 0], [0, 510044], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 510043], [0, 1]], "client.6837": [[0, 0], [0, 0], [0, 0], [0, 761751], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 761751], [0, 1]]}}, "metrics": {"delayed_ranks": [], "mds.0": {"client.5945": [], "client.5951": [], "client.6834": [], "client.6837": []}, "mds.1": {"client.5945": [], "client.5951": []}}} 2024-04-25T14:07:31.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:31.063+0000 7f3104200640 1 -- 172.21.15.86:0/492195233 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f30dc06f570 msgr2=0x7f30dc071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:31.063+0000 7f3104200640 1 --2- 172.21.15.86:0/492195233 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f30dc06f570 0x7f30dc071a80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f30f0000f70 tx=0x7f30f0009040 comp rx=0 tx=0).stop 2024-04-25T14:07:31.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:31.063+0000 7f3104200640 1 -- 172.21.15.86:0/492195233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30fc0d16e0 msgr2=0x7f30fc099b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:31.063+0000 7f3104200640 1 --2- 172.21.15.86:0/492195233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30fc0d16e0 0x7f30fc099b10 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f30f400c760 tx=0x7f30f400cc20 comp rx=0 tx=0).stop 2024-04-25T14:07:31.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:31.063+0000 7f3104200640 1 -- 172.21.15.86:0/492195233 shutdown_connections 2024-04-25T14:07:31.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:31.063+0000 7f3104200640 1 --2- 172.21.15.86:0/492195233 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f30dc06f570 0x7f30dc071a80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:31.063+0000 7f3104200640 1 --2- 172.21.15.86:0/492195233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30fc0d16e0 0x7f30fc099b10 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:31.063+0000 7f3104200640 1 --2- 172.21.15.86:0/492195233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30fc0cfca0 0x7f30fc0995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:31.063+0000 7f3104200640 1 --2- 172.21.15.86:0/492195233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30fc0730b0 0x7f30fc15a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:31.063+0000 7f3104200640 1 -- 172.21.15.86:0/492195233 >> 172.21.15.86:0/492195233 conn(0x7f30fc0b4650 msgr2=0x7f30fc159160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:31.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:31.063+0000 7f3104200640 1 -- 172.21.15.86:0/492195233 shutdown_connections 2024-04-25T14:07:31.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:31.063+0000 7f3104200640 1 -- 172.21.15.86:0/492195233 wait complete. 2024-04-25T14:07:32.087 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:07:32.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.247+0000 7fa08aa00640 1 Processor -- start 2024-04-25T14:07:32.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.247+0000 7fa08aa00640 1 -- start start 2024-04-25T14:07:32.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.251+0000 7fa08aa00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa084070330 0x7fa0840d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.251+0000 7fa08aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa0840d1c20 0x7fa08406ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.251+0000 7fa08aa00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0840d2030 con 0x7fa08406f270 2024-04-25T14:07:32.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.251+0000 7fa08aa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa08415e5c0 con 0x7fa0840d1c20 2024-04-25T14:07:32.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.251+0000 7fa08aa00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa08415e8a0 con 0x7fa084070330 2024-04-25T14:07:32.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.251+0000 7fa083400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa0840d1c20 0x7fa08406ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.251+0000 7fa083400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa0840d1c20 0x7fa08406ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49026/0 (socket says 172.21.15.86:49026) 2024-04-25T14:07:32.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.251+0000 7fa083400640 1 -- 172.21.15.86:0/779056319 learned_addr learned my addr 172.21.15.86:0/779056319 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:32.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.251+0000 7fa083e00640 1 --2- 172.21.15.86:0/779056319 >> v2:172.21.15.155:3300/0 conn(0x7fa084070330 0x7fa0840d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.251+0000 7fa083400640 1 -- 172.21.15.86:0/779056319 >> v2:172.21.15.155:3300/0 conn(0x7fa084070330 msgr2=0x7fa0840d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:32.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.251+0000 7fa083400640 1 --2- 172.21.15.86:0/779056319 >> v2:172.21.15.155:3300/0 conn(0x7fa084070330 0x7fa0840d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.251+0000 7fa083400640 1 -- 172.21.15.86:0/779056319 >> v1:172.21.15.155:6789/0 conn(0x7fa08406f270 legacy=0x7fa08406f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.251+0000 7fa083400640 1 -- 172.21.15.86:0/779056319 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08415eb80 con 0x7fa0840d1c20 2024-04-25T14:07:32.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.251+0000 7fa083400640 1 --2- 172.21.15.86:0/779056319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0840d1c20 0x7fa08406ed30 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fa070009870 tx=0x7fa07001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e341f43473407 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:32.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.255+0000 7fa082a00640 1 -- 172.21.15.86:0/779056319 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0700b0070 con 0x7fa0840d1c20 2024-04-25T14:07:32.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.255+0000 7fa082a00640 1 -- 172.21.15.86:0/779056319 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0700ab600 con 0x7fa0840d1c20 2024-04-25T14:07:32.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.255+0000 7fa082a00640 1 -- 172.21.15.86:0/779056319 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0700ab9d0 con 0x7fa0840d1c20 2024-04-25T14:07:32.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.255+0000 7fa08aa00640 1 -- 172.21.15.86:0/779056319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0840d1c20 msgr2=0x7fa08406ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.255+0000 7fa08aa00640 1 --2- 172.21.15.86:0/779056319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0840d1c20 0x7fa08406ed30 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fa070009870 tx=0x7fa07001c490 comp rx=0 tx=0).stop 2024-04-25T14:07:32.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.255+0000 7fa08aa00640 1 -- 172.21.15.86:0/779056319 shutdown_connections 2024-04-25T14:07:32.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.255+0000 7fa08aa00640 1 --2- 172.21.15.86:0/779056319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0840d1c20 0x7fa08406ed30 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.255+0000 7fa08aa00640 1 --2- 172.21.15.86:0/779056319 >> v2:172.21.15.155:3300/0 conn(0x7fa084070330 0x7fa0840d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.255+0000 7fa08aa00640 1 -- 172.21.15.86:0/779056319 >> 172.21.15.86:0/779056319 conn(0x7fa0840b4650 msgr2=0x7fa0840b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:32.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.255+0000 7fa08aa00640 1 -- 172.21.15.86:0/779056319 shutdown_connections 2024-04-25T14:07:32.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.255+0000 7fa08aa00640 1 -- 172.21.15.86:0/779056319 wait complete. 2024-04-25T14:07:32.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.255+0000 7fa08aa00640 1 Processor -- start 2024-04-25T14:07:32.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.255+0000 7fa08aa00640 1 -- start start 2024-04-25T14:07:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.259+0000 7fa08aa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa08406f270 0x7fa084073290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.259+0000 7fa08aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa084070330 0x7fa08409a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.259+0000 7fa08aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0840d1c20 0x7fa0840b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.259+0000 7fa08aa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0840708c0 con 0x7fa084070330 2024-04-25T14:07:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.259+0000 7fa08aa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa084070740 con 0x7fa0840d1c20 2024-04-25T14:07:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.259+0000 7fa08aa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa08416afc0 con 0x7fa08406f270 2024-04-25T14:07:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.259+0000 7fa083400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa084070330 0x7fa08409a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.259+0000 7fa083400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa084070330 0x7fa08409a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49032/0 (socket says 172.21.15.86:49032) 2024-04-25T14:07:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.259+0000 7fa083400640 1 -- 172.21.15.86:0/2990241394 learned_addr learned my addr 172.21.15.86:0/2990241394 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.259+0000 7fa083e00640 1 --2- 172.21.15.86:0/2990241394 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa08406f270 0x7fa084073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.259+0000 7fa088a00640 1 --2- 172.21.15.86:0/2990241394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0840d1c20 0x7fa0840b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.259+0000 7fa083400640 1 -- 172.21.15.86:0/2990241394 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa08406f270 msgr2=0x7fa084073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.259+0000 7fa083400640 1 --2- 172.21.15.86:0/2990241394 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa08406f270 0x7fa084073290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.259+0000 7fa083400640 1 -- 172.21.15.86:0/2990241394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0840d1c20 msgr2=0x7fa0840b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.259+0000 7fa083400640 1 --2- 172.21.15.86:0/2990241394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0840d1c20 0x7fa0840b0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.259+0000 7fa083400640 1 -- 172.21.15.86:0/2990241394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0840737d0 con 0x7fa084070330 2024-04-25T14:07:32.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.259+0000 7fa083e00640 1 --2- 172.21.15.86:0/2990241394 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa08406f270 0x7fa084073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:32.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.259+0000 7fa083400640 1 --2- 172.21.15.86:0/2990241394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa084070330 0x7fa08409a870 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fa07001a0c0 tx=0x7fa070019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:32.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.259+0000 7fa07be00640 1 -- 172.21.15.86:0/2990241394 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0700ba020 con 0x7fa084070330 2024-04-25T14:07:32.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.259+0000 7fa07be00640 1 -- 172.21.15.86:0/2990241394 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0700b6620 con 0x7fa084070330 2024-04-25T14:07:32.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.259+0000 7fa08aa00640 1 -- 172.21.15.86:0/2990241394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa084073ce0 con 0x7fa084070330 2024-04-25T14:07:32.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.259+0000 7fa07be00640 1 -- 172.21.15.86:0/2990241394 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0700b0040 con 0x7fa084070330 2024-04-25T14:07:32.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.259+0000 7fa08aa00640 1 -- 172.21.15.86:0/2990241394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0840742a0 con 0x7fa084070330 2024-04-25T14:07:32.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.259+0000 7fa08aa00640 1 -- 172.21.15.86:0/2990241394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0840744e0 con 0x7fa084070330 2024-04-25T14:07:32.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.263+0000 7fa07be00640 1 -- 172.21.15.86:0/2990241394 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7fa0700bd020 con 0x7fa084070330 2024-04-25T14:07:32.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.263+0000 7fa07be00640 1 --2- 172.21.15.86:0/2990241394 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fa06406f5a0 0x7fa064071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.263+0000 7fa07be00640 1 -- 172.21.15.86:0/2990241394 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7fa070129950 con 0x7fa084070330 2024-04-25T14:07:32.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.263+0000 7fa083e00640 1 --2- 172.21.15.86:0/2990241394 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fa06406f5a0 0x7fa064071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.263+0000 7fa083e00640 1 --2- 172.21.15.86:0/2990241394 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fa06406f5a0 0x7fa064071ab0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa084072200 tx=0x7fa074008040 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:32.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.267+0000 7fa07be00640 1 -- 172.21.15.86:0/2990241394 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa0840744e0 con 0x7fa084070330 2024-04-25T14:07:32.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.443+0000 7fa08aa00640 1 -- 172.21.15.86:0/2990241394 --> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7fa084073a30 con 0x7fa06406f5a0 2024-04-25T14:07:32.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.447+0000 7fa07be00640 1 -- 172.21.15.86:0/2990241394 <== mgr.6801 v2:172.21.15.86:6832/203029814 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3051 (secure 0 0 0) 0x7fa084073a30 con 0x7fa06406f5a0 2024-04-25T14:07:32.447 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {"cephfs": {"client.5945": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:172.21.15.86"}, "client.5951": {"hostname": "smithi155", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:192.168.0.1"}, "client.6834": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}, "client.6837": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}}}, "global_metrics": {"cephfs": {"client.5945": [[0, 0], [0, 0], [0, 396500909], [0, 71840640], [34, 0], [0, 35], [35, 35], [0, 35], [0, 0], [32, 33554432], [0, 0], [0, 0], [0, 12390659], [275776813322337, 32], [0, 1122510], [133594044029089, 64]], "client.5951": [[0, 0], [0, 0], [0, 0], [0, 20679725], [6, 0], [0, 3], [3, 3], [0, 3], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 2297747], [50231033486578, 9]], "client.6834": [[0, 0], [0, 0], [0, 0], [0, 510044], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 510043], [0, 1]], "client.6837": [[0, 0], [0, 0], [0, 0], [0, 761751], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 761751], [0, 1]]}}, "metrics": {"delayed_ranks": [], "mds.0": {"client.5945": [], "client.5951": [], "client.6834": [], "client.6837": []}, "mds.1": {"client.5945": [], "client.5951": []}}} 2024-04-25T14:07:32.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.447+0000 7fa08aa00640 1 -- 172.21.15.86:0/2990241394 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fa06406f5a0 msgr2=0x7fa064071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.447+0000 7fa08aa00640 1 --2- 172.21.15.86:0/2990241394 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fa06406f5a0 0x7fa064071ab0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa084072200 tx=0x7fa074008040 comp rx=0 tx=0).stop 2024-04-25T14:07:32.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.447+0000 7fa08aa00640 1 -- 172.21.15.86:0/2990241394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa084070330 msgr2=0x7fa08409a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.447+0000 7fa08aa00640 1 --2- 172.21.15.86:0/2990241394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa084070330 0x7fa08409a870 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fa07001a0c0 tx=0x7fa070019230 comp rx=0 tx=0).stop 2024-04-25T14:07:32.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.447+0000 7fa08aa00640 1 -- 172.21.15.86:0/2990241394 shutdown_connections 2024-04-25T14:07:32.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.447+0000 7fa08aa00640 1 --2- 172.21.15.86:0/2990241394 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fa06406f5a0 0x7fa064071ab0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.447+0000 7fa08aa00640 1 --2- 172.21.15.86:0/2990241394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0840d1c20 0x7fa0840b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.447+0000 7fa08aa00640 1 --2- 172.21.15.86:0/2990241394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa084070330 0x7fa08409a870 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.447+0000 7fa08aa00640 1 --2- 172.21.15.86:0/2990241394 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa08406f270 0x7fa084073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.447+0000 7fa08aa00640 1 -- 172.21.15.86:0/2990241394 >> 172.21.15.86:0/2990241394 conn(0x7fa0840b4650 msgr2=0x7fa08415ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:32.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.447+0000 7fa08aa00640 1 -- 172.21.15.86:0/2990241394 shutdown_connections 2024-04-25T14:07:32.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:32.447+0000 7fa08aa00640 1 -- 172.21.15.86:0/2990241394 wait complete. 2024-04-25T14:07:33.469 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:07:33.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.631+0000 7f47b3200640 1 Processor -- start 2024-04-25T14:07:33.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.631+0000 7f47b3200640 1 -- start start 2024-04-25T14:07:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.631+0000 7f47b3200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47ac0d1c70 0x7f47ac0d2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:33.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.631+0000 7f47b3200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f47ac0b7480 0x7f47ac0b7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:33.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.631+0000 7f47b3200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47ac160ec0 con 0x7f47ac070e70 2024-04-25T14:07:33.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.631+0000 7f47b3200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f47ac1611a0 con 0x7f47ac0b7480 2024-04-25T14:07:33.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.631+0000 7f47b3200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f47ac0d1190 con 0x7f47ac0d1c70 2024-04-25T14:07:33.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.631+0000 7f47b1400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f47ac070e70 0x7f47ac071280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41148/0 (socket says 172.21.15.86:41148) 2024-04-25T14:07:33.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.631+0000 7f47b1400640 1 -- 172.21.15.86:0/4020566899 learned_addr learned my addr 172.21.15.86:0/4020566899 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:07:33.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.635+0000 7f47b1e00640 1 --2- 172.21.15.86:0/4020566899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47ac0d1c70 0x7f47ac0d2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:33.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.635+0000 7f47b2800640 1 --2- 172.21.15.86:0/4020566899 >> v2:172.21.15.155:3300/0 conn(0x7f47ac0b7480 0x7f47ac0b7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:33.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.635+0000 7f47b0a00640 1 -- 172.21.15.86:0/4020566899 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1390306110 0 0) 0x7f47ac160ec0 con 0x7f47ac070e70 2024-04-25T14:07:33.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.635+0000 7f47b0a00640 1 -- 172.21.15.86:0/4020566899 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4794003530 con 0x7f47ac070e70 2024-04-25T14:07:33.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.635+0000 7f47b1e00640 1 -- 172.21.15.86:0/4020566899 >> v2:172.21.15.155:3300/0 conn(0x7f47ac0b7480 msgr2=0x7f47ac0b7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:33.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.635+0000 7f47b1e00640 1 --2- 172.21.15.86:0/4020566899 >> v2:172.21.15.155:3300/0 conn(0x7f47ac0b7480 0x7f47ac0b7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:33.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.635+0000 7f47b1e00640 1 -- 172.21.15.86:0/4020566899 >> v1:172.21.15.86:6789/0 conn(0x7f47ac070e70 legacy=0x7f47ac071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:33.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.635+0000 7f47b0a00640 1 -- 172.21.15.86:0/4020566899 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 12058597 0 0) 0x7f4794003530 con 0x7f47ac070e70 2024-04-25T14:07:33.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.635+0000 7f47b1e00640 1 -- 172.21.15.86:0/4020566899 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47ac0d1470 con 0x7f47ac0d1c70 2024-04-25T14:07:33.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.635+0000 7f47b2800640 1 --2- 172.21.15.86:0/4020566899 >> v2:172.21.15.155:3300/0 conn(0x7f47ac0b7480 0x7f47ac0b7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:07:33.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.635+0000 7f47b1e00640 1 --2- 172.21.15.86:0/4020566899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47ac0d1c70 0x7f47ac0d2080 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f47a00097c0 tx=0x7f47a001c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ae5145bbfa3b64e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:33.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.639+0000 7f47b0a00640 1 -- 172.21.15.86:0/4020566899 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a00b0070 con 0x7f47ac0d1c70 2024-04-25T14:07:33.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.639+0000 7f47b0a00640 1 -- 172.21.15.86:0/4020566899 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47a00ab400 con 0x7f47ac0d1c70 2024-04-25T14:07:33.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.639+0000 7f47b0a00640 1 -- 172.21.15.86:0/4020566899 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c001680 con 0x7f47ac0d1c70 2024-04-25T14:07:33.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.639+0000 7f47b3200640 1 -- 172.21.15.86:0/4020566899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47ac0d1c70 msgr2=0x7f47ac0d2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:33.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.639+0000 7f47b3200640 1 --2- 172.21.15.86:0/4020566899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47ac0d1c70 0x7f47ac0d2080 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f47a00097c0 tx=0x7f47a001c390 comp rx=0 tx=0).stop 2024-04-25T14:07:33.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.639+0000 7f47b3200640 1 -- 172.21.15.86:0/4020566899 shutdown_connections 2024-04-25T14:07:33.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.639+0000 7f47b3200640 1 --2- 172.21.15.86:0/4020566899 >> v2:172.21.15.155:3300/0 conn(0x7f47ac0b7480 0x7f47ac0b7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:33.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.639+0000 7f47b3200640 1 --2- 172.21.15.86:0/4020566899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47ac0d1c70 0x7f47ac0d2080 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:33.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.639+0000 7f47b3200640 1 -- 172.21.15.86:0/4020566899 >> 172.21.15.86:0/4020566899 conn(0x7f47ac0cfea0 msgr2=0x7f47ac0d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:33.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.639+0000 7f47b3200640 1 -- 172.21.15.86:0/4020566899 shutdown_connections 2024-04-25T14:07:33.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.639+0000 7f47b3200640 1 -- 172.21.15.86:0/4020566899 wait complete. 2024-04-25T14:07:33.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.639+0000 7f47b3200640 1 Processor -- start 2024-04-25T14:07:33.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.639+0000 7f47b3200640 1 -- start start 2024-04-25T14:07:33.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.643+0000 7f47b3200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47ac070e70 0x7f47ac1524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:33.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.643+0000 7f47b3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ac0b7480 0x7f47ac15c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:33.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.643+0000 7f47b3200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47ac0d1c70 0x7f47ac15e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:33.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.643+0000 7f47b3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47ac16c7d0 con 0x7f47ac0b7480 2024-04-25T14:07:33.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.643+0000 7f47b3200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f47ac160bf0 con 0x7f47ac0d1c70 2024-04-25T14:07:33.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.643+0000 7f47b3200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f47ac16c950 con 0x7f47ac070e70 2024-04-25T14:07:33.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.643+0000 7f47b1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ac0b7480 0x7f47ac15c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:33.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.643+0000 7f47b1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ac0b7480 0x7f47ac15c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49044/0 (socket says 172.21.15.86:49044) 2024-04-25T14:07:33.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.643+0000 7f47b1400640 1 -- 172.21.15.86:0/3991666691 learned_addr learned my addr 172.21.15.86:0/3991666691 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:33.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.643+0000 7f47b1e00640 1 --2- 172.21.15.86:0/3991666691 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47ac070e70 0x7f47ac1524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:33.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.643+0000 7f47b2800640 1 --2- 172.21.15.86:0/3991666691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47ac0d1c70 0x7f47ac15e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:33.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.643+0000 7f47b1400640 1 -- 172.21.15.86:0/3991666691 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47ac070e70 msgr2=0x7f47ac1524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:33.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.643+0000 7f47b1400640 1 --2- 172.21.15.86:0/3991666691 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47ac070e70 0x7f47ac1524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:33.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.643+0000 7f47b1400640 1 -- 172.21.15.86:0/3991666691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47ac0d1c70 msgr2=0x7f47ac15e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:33.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.643+0000 7f47b1400640 1 --2- 172.21.15.86:0/3991666691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47ac0d1c70 0x7f47ac15e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:33.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.643+0000 7f47b1400640 1 -- 172.21.15.86:0/3991666691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47ac1537a0 con 0x7f47ac0b7480 2024-04-25T14:07:33.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.643+0000 7f47b1400640 1 --2- 172.21.15.86:0/3991666691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ac0b7480 0x7f47ac15c3a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f479c0017f0 tx=0x7f479c00cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:33.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.643+0000 7f47a6000640 1 -- 172.21.15.86:0/3991666691 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c00ce60 con 0x7f47ac0b7480 2024-04-25T14:07:33.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.643+0000 7f47a6000640 1 -- 172.21.15.86:0/3991666691 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f479c005ae0 con 0x7f47ac0b7480 2024-04-25T14:07:33.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.643+0000 7f47b3200640 1 -- 172.21.15.86:0/3991666691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47ac153a40 con 0x7f47ac0b7480 2024-04-25T14:07:33.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.643+0000 7f47a6000640 1 -- 172.21.15.86:0/3991666691 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c007070 con 0x7f47ac0b7480 2024-04-25T14:07:33.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.643+0000 7f47b3200640 1 -- 172.21.15.86:0/3991666691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47ac06d810 con 0x7f47ac0b7480 2024-04-25T14:07:33.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.643+0000 7f47b3200640 1 -- 172.21.15.86:0/3991666691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47ac076b80 con 0x7f47ac0b7480 2024-04-25T14:07:33.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.647+0000 7f47a6000640 1 -- 172.21.15.86:0/3991666691 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7f479c007210 con 0x7f47ac0b7480 2024-04-25T14:07:33.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.647+0000 7f47a6000640 1 --2- 172.21.15.86:0/3991666691 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f479006f5a0 0x7f4790071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:33.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.647+0000 7f47a6000640 1 -- 172.21.15.86:0/3991666691 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f479c08f350 con 0x7f47ac0b7480 2024-04-25T14:07:33.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.647+0000 7f47b1e00640 1 --2- 172.21.15.86:0/3991666691 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f479006f5a0 0x7f4790071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:33.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.647+0000 7f47b1e00640 1 --2- 172.21.15.86:0/3991666691 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f479006f5a0 0x7f4790071ab0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f47a00ad000 tx=0x7f47a0008040 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:33.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.651+0000 7f47a6000640 1 -- 172.21.15.86:0/3991666691 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f479c05be70 con 0x7f47ac0b7480 2024-04-25T14:07:33.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.839+0000 7f47b3200640 1 -- 172.21.15.86:0/3991666691 --> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f47ac152a20 con 0x7f479006f5a0 2024-04-25T14:07:33.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.839+0000 7f47a6000640 1 -- 172.21.15.86:0/3991666691 <== mgr.6801 v2:172.21.15.86:6832/203029814 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3051 (secure 0 0 0) 0x7f47ac152a20 con 0x7f479006f5a0 2024-04-25T14:07:33.841 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {"cephfs": {"client.5945": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:172.21.15.86"}, "client.5951": {"hostname": "smithi155", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:192.168.0.1"}, "client.6834": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}, "client.6837": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}}}, "global_metrics": {"cephfs": {"client.5945": [[0, 0], [0, 0], [0, 396500909], [0, 71840640], [34, 0], [0, 35], [35, 35], [0, 35], [0, 0], [32, 33554432], [0, 0], [0, 0], [0, 12390659], [275776813322337, 32], [0, 1122510], [133594044029089, 64]], "client.5951": [[0, 0], [0, 0], [0, 0], [0, 20679725], [6, 0], [0, 3], [3, 3], [0, 3], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 2297747], [50231033486578, 9]], "client.6834": [[0, 0], [0, 0], [0, 0], [0, 510044], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 510043], [0, 1]], "client.6837": [[0, 0], [0, 0], [0, 0], [0, 761751], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 761751], [0, 1]]}}, "metrics": {"delayed_ranks": [], "mds.0": {"client.5945": [], "client.5951": [], "client.6834": [], "client.6837": []}, "mds.1": {"client.5945": [], "client.5951": []}}} 2024-04-25T14:07:33.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.843+0000 7f47b3200640 1 -- 172.21.15.86:0/3991666691 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f479006f5a0 msgr2=0x7f4790071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:33.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.843+0000 7f47b3200640 1 --2- 172.21.15.86:0/3991666691 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f479006f5a0 0x7f4790071ab0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f47a00ad000 tx=0x7f47a0008040 comp rx=0 tx=0).stop 2024-04-25T14:07:33.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.843+0000 7f47b3200640 1 -- 172.21.15.86:0/3991666691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ac0b7480 msgr2=0x7f47ac15c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:33.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.843+0000 7f47b3200640 1 --2- 172.21.15.86:0/3991666691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ac0b7480 0x7f47ac15c3a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f479c0017f0 tx=0x7f479c00cb20 comp rx=0 tx=0).stop 2024-04-25T14:07:33.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.843+0000 7f47b3200640 1 -- 172.21.15.86:0/3991666691 shutdown_connections 2024-04-25T14:07:33.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.843+0000 7f47b3200640 1 --2- 172.21.15.86:0/3991666691 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f479006f5a0 0x7f4790071ab0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:33.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.843+0000 7f47b3200640 1 --2- 172.21.15.86:0/3991666691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47ac0d1c70 0x7f47ac15e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:33.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.843+0000 7f47b3200640 1 --2- 172.21.15.86:0/3991666691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ac0b7480 0x7f47ac15c3a0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:33.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.843+0000 7f47b3200640 1 --2- 172.21.15.86:0/3991666691 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47ac070e70 0x7f47ac1524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:33.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.843+0000 7f47b3200640 1 -- 172.21.15.86:0/3991666691 >> 172.21.15.86:0/3991666691 conn(0x7f47ac0cfea0 msgr2=0x7f47ac068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:33.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.843+0000 7f47b3200640 1 -- 172.21.15.86:0/3991666691 shutdown_connections 2024-04-25T14:07:33.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:33.843+0000 7f47b3200640 1 -- 172.21.15.86:0/3991666691 wait complete. 2024-04-25T14:07:34.864 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:07:35.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9f5800640 1 Processor -- start 2024-04-25T14:07:35.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9f5800640 1 -- start start 2024-04-25T14:07:35.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9f5800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9f0070330 0x7fa9f00d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:35.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9f5800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa9f00d1c20 0x7fa9f006ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:35.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9f5800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9f00d2030 con 0x7fa9f006f270 2024-04-25T14:07:35.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9f5800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa9f015e5c0 con 0x7fa9f00d1c20 2024-04-25T14:07:35.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9f5800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa9f015e8a0 con 0x7fa9f0070330 2024-04-25T14:07:35.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9ef400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa9f006f270 0x7fa9f006f680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41156/0 (socket says 172.21.15.86:41156) 2024-04-25T14:07:35.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9ef400640 1 -- 172.21.15.86:0/4210650038 learned_addr learned my addr 172.21.15.86:0/4210650038 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:07:35.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9eea00640 1 --2- 172.21.15.86:0/4210650038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9f0070330 0x7fa9f00d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:35.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9ee000640 1 --2- 172.21.15.86:0/4210650038 >> v2:172.21.15.155:3300/0 conn(0x7fa9f00d1c20 0x7fa9f006ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:35.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9ed600640 1 -- 172.21.15.86:0/4210650038 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2158931771 0 0) 0x7fa9e8001120 con 0x7fa9f006f270 2024-04-25T14:07:35.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9ed600640 1 -- 172.21.15.86:0/4210650038 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9d4003530 con 0x7fa9f006f270 2024-04-25T14:07:35.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9ed600640 1 -- 172.21.15.86:0/4210650038 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1685166673 0 0) 0x7fa9d4003530 con 0x7fa9f006f270 2024-04-25T14:07:35.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9ed600640 1 -- 172.21.15.86:0/4210650038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9f0070330 msgr2=0x7fa9f00d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:35.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9ed600640 1 --2- 172.21.15.86:0/4210650038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9f0070330 0x7fa9f00d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:35.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9ed600640 1 -- 172.21.15.86:0/4210650038 >> v2:172.21.15.155:3300/0 conn(0x7fa9f00d1c20 msgr2=0x7fa9f006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:35.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9ed600640 1 --2- 172.21.15.86:0/4210650038 >> v2:172.21.15.155:3300/0 conn(0x7fa9f00d1c20 0x7fa9f006ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:35.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9ed600640 1 -- 172.21.15.86:0/4210650038 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f015eb80 con 0x7fa9f006f270 2024-04-25T14:07:35.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9ed600640 1 -- 172.21.15.86:0/4210650038 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fa9e8001850 con 0x7fa9f006f270 2024-04-25T14:07:35.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9ed600640 1 -- 172.21.15.86:0/4210650038 >> v1:172.21.15.86:6789/0 conn(0x7fa9f006f270 legacy=0x7fa9f006f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:35.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9ed600640 1 --2- 172.21.15.86:0/4210650038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9d4003a30 0x7fa9d4003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:35.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9ed600640 1 --2- 172.21.15.86:0/4210650038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9d4004810 0x7fa9d4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:35.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9ed600640 1 --2- 172.21.15.86:0/4210650038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9f0070330 0x7fa9d4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:35.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9ed600640 1 -- 172.21.15.86:0/4210650038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f015e5c0 con 0x7fa9d4004810 2024-04-25T14:07:35.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9ed600640 1 -- 172.21.15.86:0/4210650038 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f015e8a0 con 0x7fa9d4003a30 2024-04-25T14:07:35.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9ed600640 1 -- 172.21.15.86:0/4210650038 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa9d400bad0 con 0x7fa9f0070330 2024-04-25T14:07:35.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9ee000640 1 --2- 172.21.15.86:0/4210650038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9d4004810 0x7fa9d4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:35.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9eea00640 1 --2- 172.21.15.86:0/4210650038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9d4003a30 0x7fa9d4003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:35.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9ef400640 1 --2- 172.21.15.86:0/4210650038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9f0070330 0x7fa9d4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:35.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9ee000640 1 -- 172.21.15.86:0/4210650038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9f0070330 msgr2=0x7fa9d4005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:35.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9ee000640 1 --2- 172.21.15.86:0/4210650038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9f0070330 0x7fa9d4005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:35.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9ee000640 1 -- 172.21.15.86:0/4210650038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9d4003a30 msgr2=0x7fa9d4003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:35.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.023+0000 7fa9ee000640 1 --2- 172.21.15.86:0/4210650038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9d4003a30 0x7fa9d4003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:35.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.027+0000 7fa9ee000640 1 -- 172.21.15.86:0/4210650038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d4003530 con 0x7fa9d4004810 2024-04-25T14:07:35.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.027+0000 7fa9eea00640 1 --2- 172.21.15.86:0/4210650038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9d4003a30 0x7fa9d4003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:35.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.027+0000 7fa9ef400640 1 --2- 172.21.15.86:0/4210650038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9f0070330 0x7fa9d4005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:35.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.027+0000 7fa9ee000640 1 --2- 172.21.15.86:0/4210650038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9d4004810 0x7fa9d4004c20 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009ae0 tx=0x7fa9dc01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ad26ffa704bf97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:35.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.027+0000 7fa9ed600640 1 -- 172.21.15.86:0/4210650038 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc0096b0 con 0x7fa9d4004810 2024-04-25T14:07:35.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.027+0000 7fa9ed600640 1 -- 172.21.15.86:0/4210650038 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9dc0a6490 con 0x7fa9d4004810 2024-04-25T14:07:35.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.027+0000 7fa9ed600640 1 -- 172.21.15.86:0/4210650038 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc0a6880 con 0x7fa9d4004810 2024-04-25T14:07:35.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.027+0000 7fa9f5800640 1 -- 172.21.15.86:0/4210650038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9d4004810 msgr2=0x7fa9d4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:35.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.027+0000 7fa9f5800640 1 --2- 172.21.15.86:0/4210650038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9d4004810 0x7fa9d4004c20 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009ae0 tx=0x7fa9dc01cab0 comp rx=0 tx=0).stop 2024-04-25T14:07:35.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.027+0000 7fa9ef400640 1 -- 172.21.15.86:0/4210650038 reap_dead start 2024-04-25T14:07:35.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.027+0000 7fa9f5800640 1 -- 172.21.15.86:0/4210650038 shutdown_connections 2024-04-25T14:07:35.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.027+0000 7fa9f5800640 1 -- 172.21.15.86:0/4210650038 >> 172.21.15.86:0/4210650038 conn(0x7fa9f00b4650 msgr2=0x7fa9f00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:35.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.027+0000 7fa9f5800640 1 -- 172.21.15.86:0/4210650038 shutdown_connections 2024-04-25T14:07:35.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.027+0000 7fa9f5800640 1 -- 172.21.15.86:0/4210650038 wait complete. 2024-04-25T14:07:35.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.031+0000 7fa9f5800640 1 Processor -- start 2024-04-25T14:07:35.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.031+0000 7fa9f5800640 1 -- start start 2024-04-25T14:07:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.031+0000 7fa9f5800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9d4004810 0x7fa9f009a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.031+0000 7fa9f5800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9f0160590 0x7fa9f01609a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.031+0000 7fa9f5800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9f0160ee0 0x7fa9f009e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.031+0000 7fa9f5800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f00708c0 con 0x7fa9f0160590 2024-04-25T14:07:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.031+0000 7fa9f5800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f0070740 con 0x7fa9d4004810 2024-04-25T14:07:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.031+0000 7fa9f5800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa9f0168fb0 con 0x7fa9f0160ee0 2024-04-25T14:07:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.031+0000 7fa9ee000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9f0160590 0x7fa9f01609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:35.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.031+0000 7fa9ee000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9f0160590 0x7fa9f01609a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49056/0 (socket says 172.21.15.86:49056) 2024-04-25T14:07:35.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.031+0000 7fa9ee000640 1 -- 172.21.15.86:0/3416856119 learned_addr learned my addr 172.21.15.86:0/3416856119 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:35.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.035+0000 7fa9ef400640 1 --2- 172.21.15.86:0/3416856119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9f0160ee0 0x7fa9f009e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:35.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.035+0000 7fa9eea00640 1 --2- 172.21.15.86:0/3416856119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9d4004810 0x7fa9f009a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:35.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.035+0000 7fa9ee000640 1 -- 172.21.15.86:0/3416856119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9f0160ee0 msgr2=0x7fa9f009e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:35.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.035+0000 7fa9ee000640 1 --2- 172.21.15.86:0/3416856119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9f0160ee0 0x7fa9f009e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:35.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.035+0000 7fa9ee000640 1 -- 172.21.15.86:0/3416856119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9d4004810 msgr2=0x7fa9f009a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:35.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.035+0000 7fa9ee000640 1 --2- 172.21.15.86:0/3416856119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9d4004810 0x7fa9f009a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:35.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.035+0000 7fa9ee000640 1 -- 172.21.15.86:0/3416856119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f009d020 con 0x7fa9f0160590 2024-04-25T14:07:35.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.035+0000 7fa9ee000640 1 --2- 172.21.15.86:0/3416856119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9f0160590 0x7fa9f01609a0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc01cf90 tx=0x7fa9d4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:35.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.035+0000 7fa9e6a00640 1 -- 172.21.15.86:0/3416856119 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc0a6c40 con 0x7fa9f0160590 2024-04-25T14:07:35.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.035+0000 7fa9e6a00640 1 -- 172.21.15.86:0/3416856119 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9dc0a6de0 con 0x7fa9f0160590 2024-04-25T14:07:35.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.035+0000 7fa9f5800640 1 -- 172.21.15.86:0/3416856119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9f009d300 con 0x7fa9f0160590 2024-04-25T14:07:35.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.035+0000 7fa9e6a00640 1 -- 172.21.15.86:0/3416856119 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc0a76b0 con 0x7fa9f0160590 2024-04-25T14:07:35.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.035+0000 7fa9f5800640 1 -- 172.21.15.86:0/3416856119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9f009d8c0 con 0x7fa9f0160590 2024-04-25T14:07:35.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.035+0000 7fa9f5800640 1 -- 172.21.15.86:0/3416856119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9f0069d00 con 0x7fa9f0160590 2024-04-25T14:07:35.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.035+0000 7fa9e6a00640 1 -- 172.21.15.86:0/3416856119 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7fa9dc0a7850 con 0x7fa9f0160590 2024-04-25T14:07:35.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.039+0000 7fa9e6a00640 1 --2- 172.21.15.86:0/3416856119 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fa9d006f570 0x7fa9d0071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:35.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.039+0000 7fa9e6a00640 1 -- 172.21.15.86:0/3416856119 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7fa9dc129f70 con 0x7fa9f0160590 2024-04-25T14:07:35.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.039+0000 7fa9eea00640 1 --2- 172.21.15.86:0/3416856119 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fa9d006f570 0x7fa9d0071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:35.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.039+0000 7fa9eea00640 1 --2- 172.21.15.86:0/3416856119 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fa9d006f570 0x7fa9d0071a80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0072200 tx=0x7fa9e000b040 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:35.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.043+0000 7fa9e6a00640 1 -- 172.21.15.86:0/3416856119 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa9dc0ac020 con 0x7fa9f0160590 2024-04-25T14:07:35.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.219+0000 7fa9f5800640 1 -- 172.21.15.86:0/3416856119 --> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7fa9f009cb60 con 0x7fa9d006f570 2024-04-25T14:07:35.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.223+0000 7fa9e6a00640 1 -- 172.21.15.86:0/3416856119 <== mgr.6801 v2:172.21.15.86:6832/203029814 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3051 (secure 0 0 0) 0x7fa9e00039a0 con 0x7fa9d006f570 2024-04-25T14:07:35.224 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {"cephfs": {"client.5945": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:172.21.15.86"}, "client.5951": {"hostname": "smithi155", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:192.168.0.1"}, "client.6834": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}, "client.6837": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}}}, "global_metrics": {"cephfs": {"client.5945": [[0, 0], [0, 0], [0, 396500909], [0, 71840640], [34, 0], [0, 35], [35, 35], [0, 35], [0, 0], [32, 33554432], [0, 0], [0, 0], [0, 12390659], [275776813322337, 32], [0, 1122510], [133594044029089, 64]], "client.5951": [[0, 0], [0, 0], [0, 0], [0, 20679725], [6, 0], [0, 3], [3, 3], [0, 3], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 2297747], [50231033486578, 9]], "client.6834": [[0, 0], [0, 0], [0, 0], [0, 510044], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 510043], [0, 1]], "client.6837": [[0, 0], [0, 0], [0, 0], [0, 761751], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 761751], [0, 1]]}}, "metrics": {"delayed_ranks": [], "mds.0": {"client.5945": [], "client.5951": [], "client.6834": [], "client.6837": []}, "mds.1": {"client.5945": [], "client.5951": []}}} 2024-04-25T14:07:35.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.223+0000 7fa9f5800640 1 -- 172.21.15.86:0/3416856119 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fa9d006f570 msgr2=0x7fa9d0071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:35.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.227+0000 7fa9f5800640 1 --2- 172.21.15.86:0/3416856119 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fa9d006f570 0x7fa9d0071a80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0072200 tx=0x7fa9e000b040 comp rx=0 tx=0).stop 2024-04-25T14:07:35.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.227+0000 7fa9f5800640 1 -- 172.21.15.86:0/3416856119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9f0160590 msgr2=0x7fa9f01609a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:35.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.227+0000 7fa9f5800640 1 --2- 172.21.15.86:0/3416856119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9f0160590 0x7fa9f01609a0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc01cf90 tx=0x7fa9d4005840 comp rx=0 tx=0).stop 2024-04-25T14:07:35.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.227+0000 7fa9f5800640 1 -- 172.21.15.86:0/3416856119 shutdown_connections 2024-04-25T14:07:35.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.227+0000 7fa9f5800640 1 --2- 172.21.15.86:0/3416856119 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fa9d006f570 0x7fa9d0071a80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:35.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.227+0000 7fa9f5800640 1 --2- 172.21.15.86:0/3416856119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9f0160ee0 0x7fa9f009e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:35.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.227+0000 7fa9f5800640 1 --2- 172.21.15.86:0/3416856119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9f0160590 0x7fa9f01609a0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:35.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.227+0000 7fa9f5800640 1 --2- 172.21.15.86:0/3416856119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9d4004810 0x7fa9f009a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:35.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.227+0000 7fa9f5800640 1 -- 172.21.15.86:0/3416856119 >> 172.21.15.86:0/3416856119 conn(0x7fa9f00b4650 msgr2=0x7fa9f0167c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:35.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.227+0000 7fa9f5800640 1 -- 172.21.15.86:0/3416856119 shutdown_connections 2024-04-25T14:07:35.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:35.227+0000 7fa9f5800640 1 -- 172.21.15.86:0/3416856119 wait complete. 2024-04-25T14:07:36.249 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:07:36.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.407+0000 7fed87600640 1 Processor -- start 2024-04-25T14:07:36.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.407+0000 7fed87600640 1 -- start start 2024-04-25T14:07:36.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.407+0000 7fed87600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed800cfca0 0x7fed8006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:36.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.407+0000 7fed87600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed80072bd0 con 0x7fed800d16e0 2024-04-25T14:07:36.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.407+0000 7fed87600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed8006f930 con 0x7fed800730b0 2024-04-25T14:07:36.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.407+0000 7fed87600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fed80070470 con 0x7fed800cfca0 2024-04-25T14:07:36.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.407+0000 7fed77e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fed800d16e0 0x7fed800d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41162/0 (socket says 172.21.15.86:41162) 2024-04-25T14:07:36.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.407+0000 7fed77e00640 1 -- 172.21.15.86:0/1181432854 learned_addr learned my addr 172.21.15.86:0/1181432854 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:07:36.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.407+0000 7fed84c00640 1 --2- 172.21.15.86:0/1181432854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed800cfca0 0x7fed8006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:36.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.407+0000 7fed7fe00640 1 -- 172.21.15.86:0/1181432854 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1010147682 0 0) 0x7fed80072bd0 con 0x7fed800d16e0 2024-04-25T14:07:36.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.407+0000 7fed7fe00640 1 -- 172.21.15.86:0/1181432854 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed64003530 con 0x7fed800d16e0 2024-04-25T14:07:36.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.407+0000 7fed7fe00640 1 -- 172.21.15.86:0/1181432854 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1106593727 0 0) 0x7fed8006f930 con 0x7fed800730b0 2024-04-25T14:07:36.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.407+0000 7fed7fe00640 1 -- 172.21.15.86:0/1181432854 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed80072bd0 con 0x7fed800730b0 2024-04-25T14:07:36.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.407+0000 7fed84c00640 1 -- 172.21.15.86:0/1181432854 >> v1:172.21.15.155:6789/0 conn(0x7fed800730b0 legacy=0x7fed800734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:36.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.407+0000 7fed84c00640 1 -- 172.21.15.86:0/1181432854 >> v1:172.21.15.86:6789/0 conn(0x7fed800d16e0 legacy=0x7fed800d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:36.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.411+0000 7fed84c00640 1 -- 172.21.15.86:0/1181432854 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8006fbd0 con 0x7fed800cfca0 2024-04-25T14:07:36.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.411+0000 7fed84c00640 1 --2- 172.21.15.86:0/1181432854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed800cfca0 0x7fed8006f3f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fed700097f0 tx=0x7fed7001c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb682f0cdc35045d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:36.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.411+0000 7fed7fe00640 1 -- 172.21.15.86:0/1181432854 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed800743e0 con 0x7fed800cfca0 2024-04-25T14:07:36.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.411+0000 7fed7fe00640 1 -- 172.21.15.86:0/1181432854 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed700b0070 con 0x7fed800cfca0 2024-04-25T14:07:36.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.411+0000 7fed7fe00640 1 -- 172.21.15.86:0/1181432854 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed700ab7f0 con 0x7fed800cfca0 2024-04-25T14:07:36.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.415+0000 7fed87600640 1 -- 172.21.15.86:0/1181432854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed800cfca0 msgr2=0x7fed8006f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:36.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.415+0000 7fed87600640 1 --2- 172.21.15.86:0/1181432854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed800cfca0 0x7fed8006f3f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fed700097f0 tx=0x7fed7001c4e0 comp rx=0 tx=0).stop 2024-04-25T14:07:36.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.415+0000 7fed87600640 1 -- 172.21.15.86:0/1181432854 shutdown_connections 2024-04-25T14:07:36.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.415+0000 7fed87600640 1 --2- 172.21.15.86:0/1181432854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed800cfca0 0x7fed8006f3f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:36.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.415+0000 7fed87600640 1 -- 172.21.15.86:0/1181432854 >> 172.21.15.86:0/1181432854 conn(0x7fed800b4650 msgr2=0x7fed800b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:36.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.415+0000 7fed87600640 1 -- 172.21.15.86:0/1181432854 shutdown_connections 2024-04-25T14:07:36.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.415+0000 7fed87600640 1 -- 172.21.15.86:0/1181432854 wait complete. 2024-04-25T14:07:36.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.415+0000 7fed87600640 1 Processor -- start 2024-04-25T14:07:36.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.415+0000 7fed87600640 1 -- start start 2024-04-25T14:07:36.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.415+0000 7fed87600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed800730b0 0x7fed801f9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:36.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.419+0000 7fed87600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed800cfca0 0x7fed8014bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:36.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.419+0000 7fed87600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed800d16e0 0x7fed8014c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:36.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.419+0000 7fed87600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed800ce0f0 con 0x7fed800cfca0 2024-04-25T14:07:36.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.419+0000 7fed87600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fed800cdf70 con 0x7fed800730b0 2024-04-25T14:07:36.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.419+0000 7fed87600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fed800ce270 con 0x7fed800d16e0 2024-04-25T14:07:36.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.419+0000 7fed77e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed800cfca0 0x7fed8014bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:36.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.419+0000 7fed77e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed800cfca0 0x7fed8014bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49064/0 (socket says 172.21.15.86:49064) 2024-04-25T14:07:36.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.419+0000 7fed77e00640 1 -- 172.21.15.86:0/1594322855 learned_addr learned my addr 172.21.15.86:0/1594322855 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:36.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.419+0000 7fed84c00640 1 --2- 172.21.15.86:0/1594322855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed800730b0 0x7fed801f9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:36.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.419+0000 7fed85600640 1 --2- 172.21.15.86:0/1594322855 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed800d16e0 0x7fed8014c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:36.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.419+0000 7fed77e00640 1 -- 172.21.15.86:0/1594322855 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed800d16e0 msgr2=0x7fed8014c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:36.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.419+0000 7fed77e00640 1 --2- 172.21.15.86:0/1594322855 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed800d16e0 0x7fed8014c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:36.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.419+0000 7fed77e00640 1 -- 172.21.15.86:0/1594322855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed800730b0 msgr2=0x7fed801f9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:36.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.419+0000 7fed77e00640 1 --2- 172.21.15.86:0/1594322855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed800730b0 0x7fed801f9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:36.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.419+0000 7fed77e00640 1 -- 172.21.15.86:0/1594322855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8015a760 con 0x7fed800cfca0 2024-04-25T14:07:36.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.419+0000 7fed85600640 1 --2- 172.21.15.86:0/1594322855 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed800d16e0 0x7fed8014c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:36.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.419+0000 7fed77e00640 1 --2- 172.21.15.86:0/1594322855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed800cfca0 0x7fed8014bb90 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fed6800c420 tx=0x7fed6800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:36.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.419+0000 7fed7d600640 1 -- 172.21.15.86:0/1594322855 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed68017070 con 0x7fed800cfca0 2024-04-25T14:07:36.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.419+0000 7fed7d600640 1 -- 172.21.15.86:0/1594322855 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed68005cf0 con 0x7fed800cfca0 2024-04-25T14:07:36.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.419+0000 7fed87600640 1 -- 172.21.15.86:0/1594322855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed801fc900 con 0x7fed800cfca0 2024-04-25T14:07:36.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.419+0000 7fed7d600640 1 -- 172.21.15.86:0/1594322855 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed68007310 con 0x7fed800cfca0 2024-04-25T14:07:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.419+0000 7fed87600640 1 -- 172.21.15.86:0/1594322855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed8015aa40 con 0x7fed800cfca0 2024-04-25T14:07:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.419+0000 7fed87600640 1 -- 172.21.15.86:0/1594322855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed8009f4b0 con 0x7fed800cfca0 2024-04-25T14:07:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.419+0000 7fed7d600640 1 -- 172.21.15.86:0/1594322855 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7fed6801a070 con 0x7fed800cfca0 2024-04-25T14:07:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.423+0000 7fed7d600640 1 --2- 172.21.15.86:0/1594322855 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fed5c06f5a0 0x7fed5c071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.423+0000 7fed7d600640 1 -- 172.21.15.86:0/1594322855 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7fed6808eb80 con 0x7fed800cfca0 2024-04-25T14:07:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.423+0000 7fed84c00640 1 --2- 172.21.15.86:0/1594322855 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fed5c06f5a0 0x7fed5c071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.423+0000 7fed84c00640 1 --2- 172.21.15.86:0/1594322855 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fed5c06f5a0 0x7fed5c071ab0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fed700af000 tx=0x7fed700b6040 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:36.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.427+0000 7fed7d600640 1 -- 172.21.15.86:0/1594322855 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fed6805afc0 con 0x7fed800cfca0 2024-04-25T14:07:36.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.607+0000 7fed87600640 1 -- 172.21.15.86:0/1594322855 --> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7fed80069d00 con 0x7fed5c06f5a0 2024-04-25T14:07:36.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.607+0000 7fed7d600640 1 -- 172.21.15.86:0/1594322855 <== mgr.6801 v2:172.21.15.86:6832/203029814 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3051 (secure 0 0 0) 0x7fed80069d00 con 0x7fed5c06f5a0 2024-04-25T14:07:36.609 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {"cephfs": {"client.5945": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:172.21.15.86"}, "client.5951": {"hostname": "smithi155", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:192.168.0.1"}, "client.6834": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}, "client.6837": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}}}, "global_metrics": {"cephfs": {"client.5945": [[0, 0], [0, 0], [0, 396500909], [0, 71840640], [34, 0], [0, 35], [35, 35], [0, 35], [0, 0], [32, 33554432], [0, 0], [0, 0], [0, 12390659], [275776813322337, 32], [0, 1122510], [133594044029089, 64]], "client.5951": [[0, 0], [0, 0], [0, 0], [0, 20679725], [6, 0], [0, 3], [3, 3], [0, 3], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 2297747], [50231033486578, 9]], "client.6834": [[0, 0], [0, 0], [0, 0], [0, 510044], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 510043], [0, 1]], "client.6837": [[0, 0], [0, 0], [0, 0], [0, 761751], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 761751], [0, 1]]}}, "metrics": {"delayed_ranks": [], "mds.0": {"client.5945": [], "client.5951": [], "client.6834": [], "client.6837": []}, "mds.1": {"client.5945": [], "client.5951": []}}} 2024-04-25T14:07:36.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.611+0000 7fed87600640 1 -- 172.21.15.86:0/1594322855 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fed5c06f5a0 msgr2=0x7fed5c071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:36.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.611+0000 7fed87600640 1 --2- 172.21.15.86:0/1594322855 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fed5c06f5a0 0x7fed5c071ab0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fed700af000 tx=0x7fed700b6040 comp rx=0 tx=0).stop 2024-04-25T14:07:36.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.611+0000 7fed87600640 1 -- 172.21.15.86:0/1594322855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed800cfca0 msgr2=0x7fed8014bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:36.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.611+0000 7fed87600640 1 --2- 172.21.15.86:0/1594322855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed800cfca0 0x7fed8014bb90 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fed6800c420 tx=0x7fed6800c8e0 comp rx=0 tx=0).stop 2024-04-25T14:07:36.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.611+0000 7fed87600640 1 -- 172.21.15.86:0/1594322855 shutdown_connections 2024-04-25T14:07:36.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.611+0000 7fed87600640 1 --2- 172.21.15.86:0/1594322855 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fed5c06f5a0 0x7fed5c071ab0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:36.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.611+0000 7fed87600640 1 --2- 172.21.15.86:0/1594322855 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed800d16e0 0x7fed8014c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:36.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.611+0000 7fed87600640 1 --2- 172.21.15.86:0/1594322855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed800cfca0 0x7fed8014bb90 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:36.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.611+0000 7fed87600640 1 --2- 172.21.15.86:0/1594322855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed800730b0 0x7fed801f9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:36.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.611+0000 7fed87600640 1 -- 172.21.15.86:0/1594322855 >> 172.21.15.86:0/1594322855 conn(0x7fed800b4650 msgr2=0x7fed80068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:36.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.611+0000 7fed87600640 1 -- 172.21.15.86:0/1594322855 shutdown_connections 2024-04-25T14:07:36.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:36.611+0000 7fed87600640 1 -- 172.21.15.86:0/1594322855 wait complete. 2024-04-25T14:07:37.632 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:07:37.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.791+0000 7f5389200640 1 Processor -- start 2024-04-25T14:07:37.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.791+0000 7f5389200640 1 -- start start 2024-04-25T14:07:37.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.795+0000 7f5389200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53840cfca0 0x7f538406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:37.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.795+0000 7f5389200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f53840d16e0 0x7f53840d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:37.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.795+0000 7f5389200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53840d2030 con 0x7f538406c670 2024-04-25T14:07:37.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.795+0000 7f5389200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f5384070470 con 0x7f53840d16e0 2024-04-25T14:07:37.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.795+0000 7f5389200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5384072bd0 con 0x7f53840cfca0 2024-04-25T14:07:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.795+0000 7f5381e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53840cfca0 0x7f538406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.795+0000 7f5381e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53840cfca0 0x7f538406f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:38592/0 (socket says 172.21.15.86:38592) 2024-04-25T14:07:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.795+0000 7f5381e00640 1 -- 172.21.15.86:0/1497867175 learned_addr learned my addr 172.21.15.86:0/1497867175 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.795+0000 7f537be00640 1 --2- 172.21.15.86:0/1497867175 >> v2:172.21.15.155:3300/0 conn(0x7f53840d16e0 0x7f53840d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.795+0000 7f5381e00640 1 -- 172.21.15.86:0/1497867175 >> v2:172.21.15.155:3300/0 conn(0x7f53840d16e0 msgr2=0x7f53840d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.795+0000 7f5381e00640 1 --2- 172.21.15.86:0/1497867175 >> v2:172.21.15.155:3300/0 conn(0x7f53840d16e0 0x7f53840d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.795+0000 7f5381e00640 1 -- 172.21.15.86:0/1497867175 >> v1:172.21.15.155:6789/0 conn(0x7f538406c670 legacy=0x7f538406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:37.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.795+0000 7f5381e00640 1 -- 172.21.15.86:0/1497867175 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f538406f930 con 0x7f53840cfca0 2024-04-25T14:07:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.799+0000 7f5381e00640 1 --2- 172.21.15.86:0/1497867175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53840cfca0 0x7f538406f3f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5374009870 tx=0x7f537401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f41a67c612c01dc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.799+0000 7f5381400640 1 -- 172.21.15.86:0/1497867175 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53740a7070 con 0x7f53840cfca0 2024-04-25T14:07:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.799+0000 7f5381400640 1 -- 172.21.15.86:0/1497867175 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53740b0070 con 0x7f53840cfca0 2024-04-25T14:07:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.799+0000 7f5381400640 1 -- 172.21.15.86:0/1497867175 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53740ab890 con 0x7f53840cfca0 2024-04-25T14:07:37.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.799+0000 7f5389200640 1 -- 172.21.15.86:0/1497867175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53840cfca0 msgr2=0x7f538406f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:37.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.799+0000 7f5389200640 1 --2- 172.21.15.86:0/1497867175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53840cfca0 0x7f538406f3f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5374009870 tx=0x7f537401c560 comp rx=0 tx=0).stop 2024-04-25T14:07:37.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.799+0000 7f5389200640 1 -- 172.21.15.86:0/1497867175 shutdown_connections 2024-04-25T14:07:37.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.799+0000 7f5389200640 1 --2- 172.21.15.86:0/1497867175 >> v2:172.21.15.155:3300/0 conn(0x7f53840d16e0 0x7f53840d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.799+0000 7f5389200640 1 --2- 172.21.15.86:0/1497867175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53840cfca0 0x7f538406f3f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.799+0000 7f5389200640 1 -- 172.21.15.86:0/1497867175 >> 172.21.15.86:0/1497867175 conn(0x7f53840b4650 msgr2=0x7f53840b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:37.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.799+0000 7f5389200640 1 -- 172.21.15.86:0/1497867175 shutdown_connections 2024-04-25T14:07:37.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.799+0000 7f5389200640 1 -- 172.21.15.86:0/1497867175 wait complete. 2024-04-25T14:07:37.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.799+0000 7f5389200640 1 Processor -- start 2024-04-25T14:07:37.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.803+0000 7f5389200640 1 -- start start 2024-04-25T14:07:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.803+0000 7f5389200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f538406c670 0x7f538415a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.803+0000 7f5389200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53840cfca0 0x7f53840b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.803+0000 7f5389200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53840d16e0 0x7f53841553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.803+0000 7f5389200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5384168060 con 0x7f538406c670 2024-04-25T14:07:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.803+0000 7f5389200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5384167ee0 con 0x7f53840d16e0 2024-04-25T14:07:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.803+0000 7f5389200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f53841681e0 con 0x7f53840cfca0 2024-04-25T14:07:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.803+0000 7f5381e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f538406c670 0x7f538415a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.803+0000 7f5381e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f538406c670 0x7f538415a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49068/0 (socket says 172.21.15.86:49068) 2024-04-25T14:07:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.803+0000 7f5381e00640 1 -- 172.21.15.86:0/3373248686 learned_addr learned my addr 172.21.15.86:0/3373248686 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.803+0000 7f537be00640 1 --2- 172.21.15.86:0/3373248686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53840cfca0 0x7f53840b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.803+0000 7f5382800640 1 --2- 172.21.15.86:0/3373248686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53840d16e0 0x7f53841553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:37.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.803+0000 7f5381e00640 1 -- 172.21.15.86:0/3373248686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53840cfca0 msgr2=0x7f53840b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:37.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.803+0000 7f5381e00640 1 --2- 172.21.15.86:0/3373248686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53840cfca0 0x7f53840b0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.803+0000 7f5381e00640 1 -- 172.21.15.86:0/3373248686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53840d16e0 msgr2=0x7f53841553c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:37.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.803+0000 7f5381e00640 1 --2- 172.21.15.86:0/3373248686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53840d16e0 0x7f53841553c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.803+0000 7f5381e00640 1 -- 172.21.15.86:0/3373248686 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53840b15d0 con 0x7f538406c670 2024-04-25T14:07:37.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.803+0000 7f5381e00640 1 --2- 172.21.15.86:0/3373248686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f538406c670 0x7f538415a710 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f53740ad040 tx=0x7f537401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:37.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.803+0000 7f537a000640 1 -- 172.21.15.86:0/3373248686 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53740be5f0 con 0x7f538406c670 2024-04-25T14:07:37.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.803+0000 7f537a000640 1 -- 172.21.15.86:0/3373248686 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53740be790 con 0x7f538406c670 2024-04-25T14:07:37.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.803+0000 7f537a000640 1 -- 172.21.15.86:0/3373248686 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53740beb80 con 0x7f538406c670 2024-04-25T14:07:37.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.803+0000 7f5389200640 1 -- 172.21.15.86:0/3373248686 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53840b18b0 con 0x7f538406c670 2024-04-25T14:07:37.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.803+0000 7f5389200640 1 -- 172.21.15.86:0/3373248686 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53840b1e70 con 0x7f538406c670 2024-04-25T14:07:37.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.803+0000 7f5389200640 1 -- 172.21.15.86:0/3373248686 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53840b20b0 con 0x7f538406c670 2024-04-25T14:07:37.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.807+0000 7f537a000640 1 -- 172.21.15.86:0/3373248686 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7f53740b43f0 con 0x7f538406c670 2024-04-25T14:07:37.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.807+0000 7f537a000640 1 --2- 172.21.15.86:0/3373248686 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f536006f5a0 0x7f5360071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:37.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.807+0000 7f537a000640 1 -- 172.21.15.86:0/3373248686 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f537412a4b0 con 0x7f538406c670 2024-04-25T14:07:37.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.807+0000 7f537be00640 1 --2- 172.21.15.86:0/3373248686 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f536006f5a0 0x7f5360071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:37.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.811+0000 7f537be00640 1 --2- 172.21.15.86:0/3373248686 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f536006f5a0 0x7f5360071ab0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f53840cc8d0 tx=0x7f536c008040 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:37.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.811+0000 7f537a000640 1 -- 172.21.15.86:0/3373248686 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f53740c4060 con 0x7f538406c670 2024-04-25T14:07:37.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.991+0000 7f5389200640 1 -- 172.21.15.86:0/3373248686 --> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f538409f4b0 con 0x7f536006f5a0 2024-04-25T14:07:37.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.991+0000 7f537a000640 1 -- 172.21.15.86:0/3373248686 <== mgr.6801 v2:172.21.15.86:6832/203029814 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3051 (secure 0 0 0) 0x7f538409f4b0 con 0x7f536006f5a0 2024-04-25T14:07:37.994 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {"cephfs": {"client.5945": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:172.21.15.86"}, "client.5951": {"hostname": "smithi155", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:192.168.0.1"}, "client.6834": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}, "client.6837": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}}}, "global_metrics": {"cephfs": {"client.5945": [[0, 0], [0, 0], [0, 396500909], [0, 71840640], [34, 0], [0, 35], [35, 35], [0, 35], [0, 0], [32, 33554432], [0, 0], [0, 0], [0, 12390659], [275776813322337, 32], [0, 1122510], [133594044029089, 64]], "client.5951": [[0, 0], [0, 0], [0, 0], [0, 20679725], [6, 0], [0, 3], [3, 3], [0, 3], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 2297747], [50231033486578, 9]], "client.6834": [[0, 0], [0, 0], [0, 0], [0, 510044], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 510043], [0, 1]], "client.6837": [[0, 0], [0, 0], [0, 0], [0, 761751], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 761751], [0, 1]]}}, "metrics": {"delayed_ranks": [], "mds.0": {"client.5945": [], "client.5951": [], "client.6834": [], "client.6837": []}, "mds.1": {"client.5945": [], "client.5951": []}}} 2024-04-25T14:07:37.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.995+0000 7f5389200640 1 -- 172.21.15.86:0/3373248686 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f536006f5a0 msgr2=0x7f5360071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:37.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.995+0000 7f5389200640 1 --2- 172.21.15.86:0/3373248686 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f536006f5a0 0x7f5360071ab0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f53840cc8d0 tx=0x7f536c008040 comp rx=0 tx=0).stop 2024-04-25T14:07:37.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.995+0000 7f5389200640 1 -- 172.21.15.86:0/3373248686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f538406c670 msgr2=0x7f538415a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:37.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.995+0000 7f5389200640 1 --2- 172.21.15.86:0/3373248686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f538406c670 0x7f538415a710 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f53740ad040 tx=0x7f537401ada0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.995+0000 7f5389200640 1 -- 172.21.15.86:0/3373248686 shutdown_connections 2024-04-25T14:07:37.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.995+0000 7f5389200640 1 --2- 172.21.15.86:0/3373248686 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f536006f5a0 0x7f5360071ab0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.995+0000 7f5389200640 1 --2- 172.21.15.86:0/3373248686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53840d16e0 0x7f53841553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.995+0000 7f5389200640 1 --2- 172.21.15.86:0/3373248686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53840cfca0 0x7f53840b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.995+0000 7f5389200640 1 --2- 172.21.15.86:0/3373248686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f538406c670 0x7f538415a710 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.995+0000 7f5389200640 1 -- 172.21.15.86:0/3373248686 >> 172.21.15.86:0/3373248686 conn(0x7f53840b4650 msgr2=0x7f5384068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:37.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.995+0000 7f5389200640 1 -- 172.21.15.86:0/3373248686 shutdown_connections 2024-04-25T14:07:37.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:37.995+0000 7f5389200640 1 -- 172.21.15.86:0/3373248686 wait complete. 2024-04-25T14:07:39.016 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:07:39.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.179+0000 7fc99f800640 1 Processor -- start 2024-04-25T14:07:39.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.179+0000 7fc99f800640 1 -- start start 2024-04-25T14:07:39.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.179+0000 7fc99f800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9980d16e0 0x7fc9980d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:39.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.179+0000 7fc99f800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9980d2030 con 0x7fc9980730b0 2024-04-25T14:07:39.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.179+0000 7fc99f800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc998072bd0 con 0x7fc9980cfca0 2024-04-25T14:07:39.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.179+0000 7fc99f800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc998070470 con 0x7fc9980d16e0 2024-04-25T14:07:39.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.179+0000 7fc99d800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc9980730b0 0x7fc9980734c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42906/0 (socket says 172.21.15.86:42906) 2024-04-25T14:07:39.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.179+0000 7fc99d800640 1 -- 172.21.15.86:0/1356352165 learned_addr learned my addr 172.21.15.86:0/1356352165 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:07:39.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.179+0000 7fc98fe00640 1 --2- 172.21.15.86:0/1356352165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9980d16e0 0x7fc9980d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:39.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.179+0000 7fc997e00640 1 -- 172.21.15.86:0/1356352165 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1182083617 0 0) 0x7fc9900014a0 con 0x7fc9980730b0 2024-04-25T14:07:39.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.179+0000 7fc997e00640 1 -- 172.21.15.86:0/1356352165 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc97c003530 con 0x7fc9980730b0 2024-04-25T14:07:39.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.179+0000 7fc997e00640 1 -- 172.21.15.86:0/1356352165 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1195826825 0 0) 0x7fc97c003530 con 0x7fc9980730b0 2024-04-25T14:07:39.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.179+0000 7fc997e00640 1 -- 172.21.15.86:0/1356352165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9980d16e0 msgr2=0x7fc9980d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:39.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.179+0000 7fc997e00640 1 --2- 172.21.15.86:0/1356352165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9980d16e0 0x7fc9980d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:39.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.179+0000 7fc997e00640 1 -- 172.21.15.86:0/1356352165 >> v1:172.21.15.155:6789/0 conn(0x7fc9980cfca0 legacy=0x7fc99806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:39.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.179+0000 7fc997e00640 1 -- 172.21.15.86:0/1356352165 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99806fbd0 con 0x7fc9980730b0 2024-04-25T14:07:39.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.179+0000 7fc997e00640 1 -- 172.21.15.86:0/1356352165 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fc990001bd0 con 0x7fc9980730b0 2024-04-25T14:07:39.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.179+0000 7fc997e00640 1 -- 172.21.15.86:0/1356352165 >> v1:172.21.15.86:6789/0 conn(0x7fc9980730b0 legacy=0x7fc9980734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:39.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.179+0000 7fc997e00640 1 --2- 172.21.15.86:0/1356352165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc97c003a60 0x7fc97c003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:39.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.179+0000 7fc997e00640 1 --2- 172.21.15.86:0/1356352165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9980d16e0 0x7fc97c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:39.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.179+0000 7fc997e00640 1 --2- 172.21.15.86:0/1356352165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc97c004d50 0x7fc97c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:39.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.179+0000 7fc997e00640 1 -- 172.21.15.86:0/1356352165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc998070470 con 0x7fc97c003a60 2024-04-25T14:07:39.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.179+0000 7fc997e00640 1 -- 172.21.15.86:0/1356352165 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c00bb80 con 0x7fc97c004d50 2024-04-25T14:07:39.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.183+0000 7fc997e00640 1 -- 172.21.15.86:0/1356352165 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc97c00df90 con 0x7fc9980d16e0 2024-04-25T14:07:39.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.183+0000 7fc99ce00640 1 --2- 172.21.15.86:0/1356352165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc97c003a60 0x7fc97c003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:39.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.183+0000 7fc98fe00640 1 --2- 172.21.15.86:0/1356352165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9980d16e0 0x7fc97c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:39.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.183+0000 7fc99d800640 1 --2- 172.21.15.86:0/1356352165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc97c004d50 0x7fc97c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:39.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.183+0000 7fc99ce00640 1 -- 172.21.15.86:0/1356352165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9980d16e0 msgr2=0x7fc97c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:39.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.183+0000 7fc99ce00640 1 --2- 172.21.15.86:0/1356352165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9980d16e0 0x7fc97c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:39.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.183+0000 7fc99ce00640 1 -- 172.21.15.86:0/1356352165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc97c004d50 msgr2=0x7fc97c005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:39.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.183+0000 7fc99ce00640 1 --2- 172.21.15.86:0/1356352165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc97c004d50 0x7fc97c005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:39.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.183+0000 7fc99ce00640 1 -- 172.21.15.86:0/1356352165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97c003530 con 0x7fc97c003a60 2024-04-25T14:07:39.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.183+0000 7fc98fe00640 1 --2- 172.21.15.86:0/1356352165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9980d16e0 0x7fc97c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:39.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.183+0000 7fc99ce00640 1 --2- 172.21.15.86:0/1356352165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc97c003a60 0x7fc97c003ef0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc988001750 tx=0x7fc98801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3bcd00c1313764 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:39.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.183+0000 7fc997e00640 1 -- 172.21.15.86:0/1356352165 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9880013a0 con 0x7fc97c003a60 2024-04-25T14:07:39.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.183+0000 7fc997e00640 1 -- 172.21.15.86:0/1356352165 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9880052f0 con 0x7fc97c003a60 2024-04-25T14:07:39.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.183+0000 7fc997e00640 1 -- 172.21.15.86:0/1356352165 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9980743e0 con 0x7fc97c003a60 2024-04-25T14:07:39.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.187+0000 7fc99f800640 1 -- 172.21.15.86:0/1356352165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc97c003a60 msgr2=0x7fc97c003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:39.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.187+0000 7fc99f800640 1 --2- 172.21.15.86:0/1356352165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc97c003a60 0x7fc97c003ef0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc988001750 tx=0x7fc98801c3e0 comp rx=0 tx=0).stop 2024-04-25T14:07:39.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.187+0000 7fc99d800640 1 -- 172.21.15.86:0/1356352165 reap_dead start 2024-04-25T14:07:39.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.187+0000 7fc99f800640 1 -- 172.21.15.86:0/1356352165 shutdown_connections 2024-04-25T14:07:39.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.187+0000 7fc99f800640 1 -- 172.21.15.86:0/1356352165 >> 172.21.15.86:0/1356352165 conn(0x7fc9980b4650 msgr2=0x7fc9980b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:39.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.187+0000 7fc99f800640 1 -- 172.21.15.86:0/1356352165 shutdown_connections 2024-04-25T14:07:39.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.187+0000 7fc99f800640 1 -- 172.21.15.86:0/1356352165 wait complete. 2024-04-25T14:07:39.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.187+0000 7fc99f800640 1 Processor -- start 2024-04-25T14:07:39.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.187+0000 7fc99f800640 1 -- start start 2024-04-25T14:07:39.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.191+0000 7fc99f800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc97c003a60 0x7fc9981f8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:39.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.191+0000 7fc99f800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9980cb0d0 0x7fc9981f61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:39.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.191+0000 7fc99f800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc99815ee60 0x7fc99815f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:39.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.191+0000 7fc99f800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc9980ce0f0 con 0x7fc99815ee60 2024-04-25T14:07:39.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.191+0000 7fc99f800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc9980cdf70 con 0x7fc9980cb0d0 2024-04-25T14:07:39.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.191+0000 7fc99f800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc9980ce270 con 0x7fc97c003a60 2024-04-25T14:07:39.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.191+0000 7fc99d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc99815ee60 0x7fc99815f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:39.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.191+0000 7fc99d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc99815ee60 0x7fc99815f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56420/0 (socket says 172.21.15.86:56420) 2024-04-25T14:07:39.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.191+0000 7fc99d800640 1 -- 172.21.15.86:0/2814655359 learned_addr learned my addr 172.21.15.86:0/2814655359 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:39.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.191+0000 7fc99ce00640 1 --2- 172.21.15.86:0/2814655359 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc97c003a60 0x7fc9981f8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:39.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.191+0000 7fc98fe00640 1 --2- 172.21.15.86:0/2814655359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9980cb0d0 0x7fc9981f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:39.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.191+0000 7fc99d800640 1 -- 172.21.15.86:0/2814655359 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc97c003a60 msgr2=0x7fc9981f8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:39.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.191+0000 7fc99d800640 1 --2- 172.21.15.86:0/2814655359 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc97c003a60 0x7fc9981f8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:39.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.191+0000 7fc99d800640 1 -- 172.21.15.86:0/2814655359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9980cb0d0 msgr2=0x7fc9981f61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:39.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.191+0000 7fc99d800640 1 --2- 172.21.15.86:0/2814655359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9980cb0d0 0x7fc9981f61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:39.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.191+0000 7fc99d800640 1 -- 172.21.15.86:0/2814655359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9981f6720 con 0x7fc99815ee60 2024-04-25T14:07:39.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.191+0000 7fc99ce00640 1 --2- 172.21.15.86:0/2814655359 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc97c003a60 0x7fc9981f8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:39.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.191+0000 7fc99d800640 1 --2- 172.21.15.86:0/2814655359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc99815ee60 0x7fc99815f270 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fc9980b3f10 tx=0x7fc990007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:39.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.191+0000 7fc995600640 1 -- 172.21.15.86:0/2814655359 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99000ea00 con 0x7fc99815ee60 2024-04-25T14:07:39.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.191+0000 7fc995600640 1 -- 172.21.15.86:0/2814655359 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc990019070 con 0x7fc99815ee60 2024-04-25T14:07:39.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.191+0000 7fc995600640 1 -- 172.21.15.86:0/2814655359 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99000ee20 con 0x7fc99815ee60 2024-04-25T14:07:39.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.191+0000 7fc99f800640 1 -- 172.21.15.86:0/2814655359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9981f9730 con 0x7fc99815ee60 2024-04-25T14:07:39.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.191+0000 7fc99f800640 1 -- 172.21.15.86:0/2814655359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9981f59e0 con 0x7fc99815ee60 2024-04-25T14:07:39.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.191+0000 7fc99f800640 1 -- 172.21.15.86:0/2814655359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9981f94a0 con 0x7fc99815ee60 2024-04-25T14:07:39.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.191+0000 7fc995600640 1 -- 172.21.15.86:0/2814655359 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7fc99001e460 con 0x7fc99815ee60 2024-04-25T14:07:39.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.195+0000 7fc995600640 1 --2- 172.21.15.86:0/2814655359 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fc97406f570 0x7fc974071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:39.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.195+0000 7fc995600640 1 -- 172.21.15.86:0/2814655359 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7fc99008fa70 con 0x7fc99815ee60 2024-04-25T14:07:39.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.195+0000 7fc99ce00640 1 --2- 172.21.15.86:0/2814655359 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fc97406f570 0x7fc974071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:39.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.195+0000 7fc99ce00640 1 --2- 172.21.15.86:0/2814655359 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fc97406f570 0x7fc974071a80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc998070710 tx=0x7fc9880011a0 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:39.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.199+0000 7fc995600640 1 -- 172.21.15.86:0/2814655359 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc9980d2030 con 0x7fc99815ee60 2024-04-25T14:07:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.379+0000 7fc99f800640 1 -- 172.21.15.86:0/2814655359 --> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7fc99809f4b0 con 0x7fc97406f570 2024-04-25T14:07:39.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.379+0000 7fc995600640 1 -- 172.21.15.86:0/2814655359 <== mgr.6801 v2:172.21.15.86:6832/203029814 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3052 (secure 0 0 0) 0x7fc99809f4b0 con 0x7fc97406f570 2024-04-25T14:07:39.381 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {"cephfs": {"client.5945": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:172.21.15.86"}, "client.5951": {"hostname": "smithi155", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:192.168.0.1"}, "client.6834": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}, "client.6837": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}}}, "global_metrics": {"cephfs": {"client.5945": [[0, 0], [0, 0], [0, 396500909], [0, 71840640], [34, 0], [0, 35], [35, 35], [0, 35], [0, 0], [32, 33554432], [0, 0], [0, 0], [0, 12390659], [275776813322337, 32], [0, 1122510], [133594044029089, 64]], "client.5951": [[0, 0], [0, 0], [0, 0], [0, 20679725], [6, 0], [0, 3], [3, 3], [0, 3], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 2297747], [50231033486578, 9]], "client.6834": [[0, 0], [0, 0], [0, 0], [0, 510044], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 510043], [0, 1]], "client.6837": [[0, 0], [0, 0], [0, 0], [0, 761751], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 761751], [0, 1]]}}, "metrics": {"delayed_ranks": [1], "mds.0": {"client.5945": [], "client.5951": [], "client.6834": [], "client.6837": []}, "mds.1": {"client.5945": [], "client.5951": []}}} 2024-04-25T14:07:39.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.383+0000 7fc99f800640 1 -- 172.21.15.86:0/2814655359 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fc97406f570 msgr2=0x7fc974071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:39.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.383+0000 7fc99f800640 1 --2- 172.21.15.86:0/2814655359 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fc97406f570 0x7fc974071a80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc998070710 tx=0x7fc9880011a0 comp rx=0 tx=0).stop 2024-04-25T14:07:39.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.383+0000 7fc99f800640 1 -- 172.21.15.86:0/2814655359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc99815ee60 msgr2=0x7fc99815f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:39.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.383+0000 7fc99f800640 1 --2- 172.21.15.86:0/2814655359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc99815ee60 0x7fc99815f270 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fc9980b3f10 tx=0x7fc990007bc0 comp rx=0 tx=0).stop 2024-04-25T14:07:39.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.383+0000 7fc99f800640 1 -- 172.21.15.86:0/2814655359 shutdown_connections 2024-04-25T14:07:39.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.383+0000 7fc99f800640 1 --2- 172.21.15.86:0/2814655359 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fc97406f570 0x7fc974071a80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:39.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.383+0000 7fc99f800640 1 --2- 172.21.15.86:0/2814655359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc99815ee60 0x7fc99815f270 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:39.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.383+0000 7fc99f800640 1 --2- 172.21.15.86:0/2814655359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9980cb0d0 0x7fc9981f61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:39.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.383+0000 7fc99f800640 1 --2- 172.21.15.86:0/2814655359 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc97c003a60 0x7fc9981f8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:39.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.383+0000 7fc99f800640 1 -- 172.21.15.86:0/2814655359 >> 172.21.15.86:0/2814655359 conn(0x7fc9980b4650 msgr2=0x7fc998162b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:39.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.383+0000 7fc99f800640 1 -- 172.21.15.86:0/2814655359 shutdown_connections 2024-04-25T14:07:39.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.383+0000 7fc99f800640 1 -- 172.21.15.86:0/2814655359 wait complete. 2024-04-25T14:07:39.403 DEBUG:tasks.cephfs.test_mds_metrics:metrics={'version': 2, 'global_counters': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'counters': [], 'client_metadata': {'cephfs': {'client.5945': {'hostname': 'smithi086', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'kernel_version': '6.9.0-rc3-g806ec36b2112', 'IP': 'v1:172.21.15.86'}, 'client.5951': {'hostname': 'smithi155', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'kernel_version': '6.9.0-rc3-g806ec36b2112', 'IP': 'v1:192.168.0.1'}, 'client.6834': {'hostname': 'smithi086', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'IP': '172.21.15.86'}, 'client.6837': {'hostname': 'smithi086', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'IP': '172.21.15.86'}}}, 'global_metrics': {'cephfs': {'client.5945': [[0, 0], [0, 0], [0, 396500909], [0, 71840640], [34, 0], [0, 35], [35, 35], [0, 35], [0, 0], [32, 33554432], [0, 0], [0, 0], [0, 12390659], [275776813322337, 32], [0, 1122510], [133594044029089, 64]], 'client.5951': [[0, 0], [0, 0], [0, 0], [0, 20679725], [6, 0], [0, 3], [3, 3], [0, 3], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 2297747], [50231033486578, 9]], 'client.6834': [[0, 0], [0, 0], [0, 0], [0, 510044], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 510043], [0, 1]], 'client.6837': [[0, 0], [0, 0], [0, 0], [0, 761751], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 761751], [0, 1]]}}, 'metrics': {'delayed_ranks': [1], 'mds.0': {'client.5945': [], 'client.5951': [], 'client.6834': [], 'client.6837': []}, 'mds.1': {'client.5945': [], 'client.5951': []}}} 2024-04-25T14:07:39.403 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:07:39.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.547+0000 7f2479800640 1 Processor -- start 2024-04-25T14:07:39.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.547+0000 7f2479800640 1 -- start start 2024-04-25T14:07:39.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.551+0000 7f2479800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2474070330 0x7f24740d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:39.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.551+0000 7f2479800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f24740ce790 0x7f247406f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:39.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.551+0000 7f2479800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24740d2030 con 0x7f24740d1c20 2024-04-25T14:07:39.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.551+0000 7f2479800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f247415e5c0 con 0x7f24740ce790 2024-04-25T14:07:39.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.551+0000 7f2479800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f247415e8a0 con 0x7f2474070330 2024-04-25T14:07:39.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.551+0000 7f2473400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f24740ce790 0x7f247406f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:39.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.551+0000 7f2473400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f24740ce790 0x7f247406f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56428/0 (socket says 172.21.15.86:56428) 2024-04-25T14:07:39.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.551+0000 7f2473400640 1 -- 172.21.15.86:0/776676055 learned_addr learned my addr 172.21.15.86:0/776676055 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:39.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.551+0000 7f246a000640 1 --1- 172.21.15.86:0/776676055 >> v1:172.21.15.86:6789/0 conn(0x7f24740d1c20 0x7f247406ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42914/0 (socket says 172.21.15.86:42914) 2024-04-25T14:07:39.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.551+0000 7f2472a00640 1 --2- 172.21.15.86:0/776676055 >> v2:172.21.15.155:3300/0 conn(0x7f2474070330 0x7f24740d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:39.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.551+0000 7f2473400640 1 -- 172.21.15.86:0/776676055 >> v2:172.21.15.155:3300/0 conn(0x7f2474070330 msgr2=0x7f24740d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:39.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.551+0000 7f2473400640 1 --2- 172.21.15.86:0/776676055 >> v2:172.21.15.155:3300/0 conn(0x7f2474070330 0x7f24740d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:39.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.551+0000 7f2473400640 1 -- 172.21.15.86:0/776676055 >> v1:172.21.15.86:6789/0 conn(0x7f24740d1c20 legacy=0x7f247406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:39.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.551+0000 7f2473400640 1 -- 172.21.15.86:0/776676055 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247415eb80 con 0x7f24740ce790 2024-04-25T14:07:39.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.555+0000 7f2473400640 1 --2- 172.21.15.86:0/776676055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24740ce790 0x7f247406f440 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f246c009870 tx=0x7f246c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c98a53dc0af918 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:39.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.555+0000 7f2472000640 1 -- 172.21.15.86:0/776676055 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c0b0070 con 0x7f24740ce790 2024-04-25T14:07:39.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.555+0000 7f2472000640 1 -- 172.21.15.86:0/776676055 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f246c0ab600 con 0x7f24740ce790 2024-04-25T14:07:39.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.555+0000 7f2472000640 1 -- 172.21.15.86:0/776676055 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c0ab9d0 con 0x7f24740ce790 2024-04-25T14:07:39.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.555+0000 7f2479800640 1 -- 172.21.15.86:0/776676055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24740ce790 msgr2=0x7f247406f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:39.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.555+0000 7f2479800640 1 --2- 172.21.15.86:0/776676055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24740ce790 0x7f247406f440 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f246c009870 tx=0x7f246c01c490 comp rx=0 tx=0).stop 2024-04-25T14:07:39.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.555+0000 7f2479800640 1 -- 172.21.15.86:0/776676055 shutdown_connections 2024-04-25T14:07:39.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.555+0000 7f2479800640 1 --2- 172.21.15.86:0/776676055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24740ce790 0x7f247406f440 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:39.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.555+0000 7f2479800640 1 --2- 172.21.15.86:0/776676055 >> v2:172.21.15.155:3300/0 conn(0x7f2474070330 0x7f24740d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:39.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.555+0000 7f2479800640 1 -- 172.21.15.86:0/776676055 >> 172.21.15.86:0/776676055 conn(0x7f24740b4650 msgr2=0x7f24740b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:39.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.555+0000 7f2479800640 1 -- 172.21.15.86:0/776676055 shutdown_connections 2024-04-25T14:07:39.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.555+0000 7f2479800640 1 -- 172.21.15.86:0/776676055 wait complete. 2024-04-25T14:07:39.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.555+0000 7f2479800640 1 Processor -- start 2024-04-25T14:07:39.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.559+0000 7f2479800640 1 -- start start 2024-04-25T14:07:39.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.559+0000 7f2479800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2474070330 0x7f247415b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:39.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.559+0000 7f2479800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24740ce790 0x7f24740c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:39.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.559+0000 7f2479800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24740d1c20 0x7f24740c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:39.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.559+0000 7f2479800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24740708c0 con 0x7f24740ce790 2024-04-25T14:07:39.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.559+0000 7f2479800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2474070740 con 0x7f24740d1c20 2024-04-25T14:07:39.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.559+0000 7f2479800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f247416afc0 con 0x7f2474070330 2024-04-25T14:07:39.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.559+0000 7f246a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24740ce790 0x7f24740c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:39.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.559+0000 7f246a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24740ce790 0x7f24740c7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56436/0 (socket says 172.21.15.86:56436) 2024-04-25T14:07:39.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.559+0000 7f246a000640 1 -- 172.21.15.86:0/2723506846 learned_addr learned my addr 172.21.15.86:0/2723506846 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:39.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.559+0000 7f2472a00640 1 --2- 172.21.15.86:0/2723506846 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2474070330 0x7f247415b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:39.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.559+0000 7f2473400640 1 --2- 172.21.15.86:0/2723506846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24740d1c20 0x7f24740c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:39.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.559+0000 7f246a000640 1 -- 172.21.15.86:0/2723506846 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2474070330 msgr2=0x7f247415b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:39.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.559+0000 7f246a000640 1 --2- 172.21.15.86:0/2723506846 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2474070330 0x7f247415b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:39.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.559+0000 7f246a000640 1 -- 172.21.15.86:0/2723506846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24740d1c20 msgr2=0x7f24740c9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:39.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.559+0000 7f246a000640 1 --2- 172.21.15.86:0/2723506846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24740d1c20 0x7f24740c9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:39.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.559+0000 7f246a000640 1 -- 172.21.15.86:0/2723506846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24740c4d60 con 0x7f24740ce790 2024-04-25T14:07:39.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.559+0000 7f246a000640 1 --2- 172.21.15.86:0/2723506846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24740ce790 0x7f24740c7f90 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f245c00c340 tx=0x7f245c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:39.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.559+0000 7f246b400640 1 -- 172.21.15.86:0/2723506846 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c0074e0 con 0x7f24740ce790 2024-04-25T14:07:39.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.559+0000 7f246b400640 1 -- 172.21.15.86:0/2723506846 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f245c007680 con 0x7f24740ce790 2024-04-25T14:07:39.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.559+0000 7f2479800640 1 -- 172.21.15.86:0/2723506846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24740c5040 con 0x7f24740ce790 2024-04-25T14:07:39.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.559+0000 7f246b400640 1 -- 172.21.15.86:0/2723506846 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c004a30 con 0x7f24740ce790 2024-04-25T14:07:39.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.559+0000 7f2479800640 1 -- 172.21.15.86:0/2723506846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24740c5600 con 0x7f24740ce790 2024-04-25T14:07:39.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.559+0000 7f2479800640 1 -- 172.21.15.86:0/2723506846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2474069d00 con 0x7f24740ce790 2024-04-25T14:07:39.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.563+0000 7f246b400640 1 -- 172.21.15.86:0/2723506846 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7f245c01a020 con 0x7f24740ce790 2024-04-25T14:07:39.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.563+0000 7f246b400640 1 --2- 172.21.15.86:0/2723506846 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f245006f5a0 0x7f2450071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:39.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.563+0000 7f246b400640 1 -- 172.21.15.86:0/2723506846 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f245c08eea0 con 0x7f24740ce790 2024-04-25T14:07:39.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.563+0000 7f2472a00640 1 --2- 172.21.15.86:0/2723506846 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f245006f5a0 0x7f2450071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:39.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.563+0000 7f2472a00640 1 --2- 172.21.15.86:0/2723506846 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f245006f5a0 0x7f2450071ab0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2474072200 tx=0x7f246400b040 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:39.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.567+0000 7f246b400640 1 -- 172.21.15.86:0/2723506846 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f24740d2030 con 0x7f24740ce790 2024-04-25T14:07:39.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.755+0000 7f2479800640 1 -- 172.21.15.86:0/2723506846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24740b4d10 con 0x7f24740ce790 2024-04-25T14:07:39.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.755+0000 7f246b400640 1 -- 172.21.15.86:0/2723506846 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5209 (secure 0 0 0) 0x7f245c05b250 con 0x7f24740ce790 2024-04-25T14:07:39.759 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:07:39.759 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":57,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1210051860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1210051860},{"type":"v1","addr":"172.21.15.155:6833","nonce":1210051860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2868300438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2868300438},{"type":"v1","addr":"172.21.15.155:6835","nonce":2868300438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:06:44.910717+0000","modified":"2024-04-25T14:07:19.703362+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6603,"mds_1":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"c","rank":1,"incarnation":55,"state":"up:active","state_seq":9,"addr":"172.21.15.86:6835/2682115891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2682115891},{"type":"v1","addr":"172.21.15.86:6835","nonce":2682115891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6603":{"gid":6603,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/1088108173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1088108173},{"type":"v1","addr":"172.21.15.86:6837","nonce":1088108173}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6603,"qdb_cluster":[6603,6600]},"id":5}]} 2024-04-25T14:07:39.759 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 57 2024-04-25T14:07:39.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.759+0000 7f2479800640 1 -- 172.21.15.86:0/2723506846 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f245006f5a0 msgr2=0x7f2450071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:39.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.759+0000 7f2479800640 1 --2- 172.21.15.86:0/2723506846 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f245006f5a0 0x7f2450071ab0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2474072200 tx=0x7f246400b040 comp rx=0 tx=0).stop 2024-04-25T14:07:39.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.759+0000 7f2479800640 1 -- 172.21.15.86:0/2723506846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24740ce790 msgr2=0x7f24740c7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:39.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.759+0000 7f2479800640 1 --2- 172.21.15.86:0/2723506846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24740ce790 0x7f24740c7f90 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f245c00c340 tx=0x7f245c00c800 comp rx=0 tx=0).stop 2024-04-25T14:07:39.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.759+0000 7f2479800640 1 -- 172.21.15.86:0/2723506846 shutdown_connections 2024-04-25T14:07:39.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.759+0000 7f2479800640 1 --2- 172.21.15.86:0/2723506846 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f245006f5a0 0x7f2450071ab0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:39.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.759+0000 7f2479800640 1 --2- 172.21.15.86:0/2723506846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24740d1c20 0x7f24740c9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:39.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.759+0000 7f2479800640 1 --2- 172.21.15.86:0/2723506846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24740ce790 0x7f24740c7f90 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:39.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.759+0000 7f2479800640 1 --2- 172.21.15.86:0/2723506846 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2474070330 0x7f247415b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:39.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.759+0000 7f2479800640 1 -- 172.21.15.86:0/2723506846 >> 172.21.15.86:0/2723506846 conn(0x7f24740b4650 msgr2=0x7f247415bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:39.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.759+0000 7f2479800640 1 -- 172.21.15.86:0/2723506846 shutdown_connections 2024-04-25T14:07:39.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:39.759+0000 7f2479800640 1 -- 172.21.15.86:0/2723506846 wait complete. 2024-04-25T14:07:39.781 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -D INPUT -p tcp --dport 6836 -j REJECT -m comment --comment teuthology 2024-04-25T14:07:39.803 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -D INPUT -p tcp --dport 6837 -j REJECT -m comment --comment teuthology 2024-04-25T14:07:39.867 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -D OUTPUT -p tcp --sport 6834 -j REJECT -m comment --comment teuthology 2024-04-25T14:07:39.930 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -D INPUT -p tcp --dport 6834 -j REJECT -m comment --comment teuthology 2024-04-25T14:07:39.994 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -D OUTPUT -p tcp --sport 6835 -j REJECT -m comment --comment teuthology 2024-04-25T14:07:40.058 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -D INPUT -p tcp --dport 6835 -j REJECT -m comment --comment teuthology 2024-04-25T14:07:40.123 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:07:40.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.311+0000 7f6435a00640 1 Processor -- start 2024-04-25T14:07:40.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.311+0000 7f6435a00640 1 -- start start 2024-04-25T14:07:40.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.315+0000 7f6435a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64300d1df0 0x7f64300cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:40.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.315+0000 7f6435a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f64300cdcc0 0x7f64300ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:40.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.315+0000 7f6435a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f643015e5c0 con 0x7f6430070330 2024-04-25T14:07:40.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.315+0000 7f6435a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f643006fd40 con 0x7f64300cdcc0 2024-04-25T14:07:40.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.315+0000 7f6435a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f643015e8a0 con 0x7f64300d1df0 2024-04-25T14:07:40.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.315+0000 7f642f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f64300cdcc0 0x7f64300ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:40.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.315+0000 7f642f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f64300cdcc0 0x7f64300ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56452/0 (socket says 172.21.15.86:56452) 2024-04-25T14:07:40.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.315+0000 7f642f400640 1 -- 172.21.15.86:0/3753715825 learned_addr learned my addr 172.21.15.86:0/3753715825 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:40.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.315+0000 7f6426000640 1 --2- 172.21.15.86:0/3753715825 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64300d1df0 0x7f64300cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:40.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.315+0000 7f642f400640 1 -- 172.21.15.86:0/3753715825 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64300d1df0 msgr2=0x7f64300cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:40.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.315+0000 7f642f400640 1 --2- 172.21.15.86:0/3753715825 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64300d1df0 0x7f64300cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:40.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.315+0000 7f642f400640 1 -- 172.21.15.86:0/3753715825 >> v1:172.21.15.155:6789/0 conn(0x7f6430070330 legacy=0x7f64300d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:40.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.315+0000 7f642f400640 1 -- 172.21.15.86:0/3753715825 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643015eb80 con 0x7f64300cdcc0 2024-04-25T14:07:40.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.319+0000 7f642f400640 1 --2- 172.21.15.86:0/3753715825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64300cdcc0 0x7f64300ce0d0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f6428009e50 tx=0x7f642801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19cac7bfb8d5599c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:40.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.319+0000 7f642e000640 1 -- 172.21.15.86:0/3753715825 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64280a5810 con 0x7f64300cdcc0 2024-04-25T14:07:40.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.319+0000 7f642e000640 1 -- 172.21.15.86:0/3753715825 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64280a59b0 con 0x7f64300cdcc0 2024-04-25T14:07:40.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.319+0000 7f642e000640 1 -- 172.21.15.86:0/3753715825 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64280a5d80 con 0x7f64300cdcc0 2024-04-25T14:07:40.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.319+0000 7f6435a00640 1 -- 172.21.15.86:0/3753715825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64300cdcc0 msgr2=0x7f64300ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:40.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.319+0000 7f6435a00640 1 --2- 172.21.15.86:0/3753715825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64300cdcc0 0x7f64300ce0d0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f6428009e50 tx=0x7f642801c970 comp rx=0 tx=0).stop 2024-04-25T14:07:40.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.319+0000 7f6435a00640 1 -- 172.21.15.86:0/3753715825 shutdown_connections 2024-04-25T14:07:40.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.319+0000 7f6435a00640 1 --2- 172.21.15.86:0/3753715825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64300cdcc0 0x7f64300ce0d0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:40.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.319+0000 7f6435a00640 1 --2- 172.21.15.86:0/3753715825 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64300d1df0 0x7f64300cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:40.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.319+0000 7f6435a00640 1 -- 172.21.15.86:0/3753715825 >> 172.21.15.86:0/3753715825 conn(0x7f64300b4650 msgr2=0x7f64300b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:40.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.319+0000 7f6435a00640 1 -- 172.21.15.86:0/3753715825 shutdown_connections 2024-04-25T14:07:40.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.319+0000 7f6435a00640 1 -- 172.21.15.86:0/3753715825 wait complete. 2024-04-25T14:07:40.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.319+0000 7f6435a00640 1 Processor -- start 2024-04-25T14:07:40.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.319+0000 7f6435a00640 1 -- start start 2024-04-25T14:07:40.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.323+0000 7f6435a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6430070330 0x7f643009b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:40.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.323+0000 7f6435a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64300cdcc0 0x7f64300c99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:40.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.323+0000 7f6435a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f643015b3f0 0x7f64300c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:40.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.323+0000 7f6435a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f643015f4b0 con 0x7f6430070330 2024-04-25T14:07:40.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.323+0000 7f6435a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f643015f330 con 0x7f64300cdcc0 2024-04-25T14:07:40.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.323+0000 7f6435a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6430070740 con 0x7f643015b3f0 2024-04-25T14:07:40.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.323+0000 7f642ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6430070330 0x7f643009b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:40.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.323+0000 7f642ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6430070330 0x7f643009b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56466/0 (socket says 172.21.15.86:56466) 2024-04-25T14:07:40.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.323+0000 7f642ea00640 1 -- 172.21.15.86:0/1551309560 learned_addr learned my addr 172.21.15.86:0/1551309560 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:40.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.323+0000 7f642f400640 1 --2- 172.21.15.86:0/1551309560 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f643015b3f0 0x7f64300c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:40.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.323+0000 7f642ea00640 1 -- 172.21.15.86:0/1551309560 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f643015b3f0 msgr2=0x7f64300c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:40.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.323+0000 7f642ea00640 1 --2- 172.21.15.86:0/1551309560 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f643015b3f0 0x7f64300c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:40.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.323+0000 7f6426000640 1 --2- 172.21.15.86:0/1551309560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64300cdcc0 0x7f64300c99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:40.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.323+0000 7f642ea00640 1 -- 172.21.15.86:0/1551309560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64300cdcc0 msgr2=0x7f64300c99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:40.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.323+0000 7f642ea00640 1 --2- 172.21.15.86:0/1551309560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64300cdcc0 0x7f64300c99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:40.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.323+0000 7f642ea00640 1 -- 172.21.15.86:0/1551309560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643009bae0 con 0x7f6430070330 2024-04-25T14:07:40.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.323+0000 7f642f400640 1 --2- 172.21.15.86:0/1551309560 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f643015b3f0 0x7f64300c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:40.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.323+0000 7f6426000640 1 --2- 172.21.15.86:0/1551309560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64300cdcc0 0x7f64300c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:40.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.323+0000 7f642ea00640 1 --2- 172.21.15.86:0/1551309560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6430070330 0x7f643009b2b0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f642000c370 tx=0x7f642000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:40.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.323+0000 7f6427400640 1 -- 172.21.15.86:0/1551309560 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64200052d0 con 0x7f6430070330 2024-04-25T14:07:40.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.323+0000 7f6427400640 1 -- 172.21.15.86:0/1551309560 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6420005470 con 0x7f6430070330 2024-04-25T14:07:40.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.323+0000 7f6435a00640 1 -- 172.21.15.86:0/1551309560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643009bdc0 con 0x7f6430070330 2024-04-25T14:07:40.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.323+0000 7f6427400640 1 -- 172.21.15.86:0/1551309560 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6420007260 con 0x7f6430070330 2024-04-25T14:07:40.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.323+0000 7f6435a00640 1 -- 172.21.15.86:0/1551309560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f643009c380 con 0x7f6430070330 2024-04-25T14:07:40.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.323+0000 7f6435a00640 1 -- 172.21.15.86:0/1551309560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6430069d00 con 0x7f6430070330 2024-04-25T14:07:40.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.327+0000 7f6427400640 1 -- 172.21.15.86:0/1551309560 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7f6420007400 con 0x7f6430070330 2024-04-25T14:07:40.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.327+0000 7f6427400640 1 --2- 172.21.15.86:0/1551309560 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f640c06f570 0x7f640c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:40.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.327+0000 7f6427400640 1 -- 172.21.15.86:0/1551309560 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f642008ecc0 con 0x7f6430070330 2024-04-25T14:07:40.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.327+0000 7f6426000640 1 --2- 172.21.15.86:0/1551309560 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f640c06f570 0x7f640c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:40.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.327+0000 7f6426000640 1 --2- 172.21.15.86:0/1551309560 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f640c06f570 0x7f640c071a80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f64300cbe80 tx=0x7f6418008040 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:40.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.331+0000 7f6427400640 1 -- 172.21.15.86:0/1551309560 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6420095080 con 0x7f6430070330 2024-04-25T14:07:40.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.511+0000 7f6435a00640 1 -- 172.21.15.86:0/1551309560 --> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f6430076bf0 con 0x7f640c06f570 2024-04-25T14:07:40.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.511+0000 7f6427400640 1 -- 172.21.15.86:0/1551309560 <== mgr.6801 v2:172.21.15.86:6832/203029814 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3052 (secure 0 0 0) 0x7f6430076bf0 con 0x7f640c06f570 2024-04-25T14:07:40.513 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {"cephfs": {"client.5945": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:172.21.15.86"}, "client.5951": {"hostname": "smithi155", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:192.168.0.1"}, "client.6834": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}, "client.6837": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}}}, "global_metrics": {"cephfs": {"client.5945": [[0, 0], [0, 0], [0, 396500909], [0, 71840640], [34, 0], [0, 35], [35, 35], [0, 35], [0, 0], [32, 33554432], [0, 0], [0, 0], [0, 12390659], [275776813322337, 32], [0, 1122510], [133594044029089, 64]], "client.5951": [[0, 0], [0, 0], [0, 0], [0, 20679725], [6, 0], [0, 3], [3, 3], [0, 3], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 2297747], [50231033486578, 9]], "client.6834": [[0, 0], [0, 0], [0, 0], [0, 510044], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 510043], [0, 1]], "client.6837": [[0, 0], [0, 0], [0, 0], [0, 761751], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 761751], [0, 1]]}}, "metrics": {"delayed_ranks": [1], "mds.0": {"client.5945": [], "client.5951": [], "client.6834": [], "client.6837": []}, "mds.1": {"client.5945": [], "client.5951": []}}} 2024-04-25T14:07:40.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.515+0000 7f6435a00640 1 -- 172.21.15.86:0/1551309560 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f640c06f570 msgr2=0x7f640c071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:40.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.515+0000 7f6435a00640 1 --2- 172.21.15.86:0/1551309560 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f640c06f570 0x7f640c071a80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f64300cbe80 tx=0x7f6418008040 comp rx=0 tx=0).stop 2024-04-25T14:07:40.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.515+0000 7f6435a00640 1 -- 172.21.15.86:0/1551309560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6430070330 msgr2=0x7f643009b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:40.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.515+0000 7f6435a00640 1 --2- 172.21.15.86:0/1551309560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6430070330 0x7f643009b2b0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f642000c370 tx=0x7f642000c830 comp rx=0 tx=0).stop 2024-04-25T14:07:40.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.515+0000 7f6435a00640 1 -- 172.21.15.86:0/1551309560 shutdown_connections 2024-04-25T14:07:40.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.515+0000 7f6435a00640 1 --2- 172.21.15.86:0/1551309560 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f640c06f570 0x7f640c071a80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:40.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.515+0000 7f6435a00640 1 --2- 172.21.15.86:0/1551309560 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f643015b3f0 0x7f64300c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:40.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.515+0000 7f6435a00640 1 --2- 172.21.15.86:0/1551309560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64300cdcc0 0x7f64300c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:40.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.515+0000 7f6435a00640 1 --2- 172.21.15.86:0/1551309560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6430070330 0x7f643009b2b0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:40.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.515+0000 7f6435a00640 1 -- 172.21.15.86:0/1551309560 >> 172.21.15.86:0/1551309560 conn(0x7f64300b4650 msgr2=0x7f64300af090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:40.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.515+0000 7f6435a00640 1 -- 172.21.15.86:0/1551309560 shutdown_connections 2024-04-25T14:07:40.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.515+0000 7f6435a00640 1 -- 172.21.15.86:0/1551309560 wait complete. 2024-04-25T14:07:40.535 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:07:40.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.683+0000 7ff3b4000640 1 Processor -- start 2024-04-25T14:07:40.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.683+0000 7ff3b4000640 1 -- start start 2024-04-25T14:07:40.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.683+0000 7ff3b4000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3ac0d1df0 0x7ff3ac0cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:40.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.683+0000 7ff3b4000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff3ac0cdcc0 0x7ff3ac0ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:40.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.683+0000 7ff3b4000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3ac15e5c0 con 0x7ff3ac070330 2024-04-25T14:07:40.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.683+0000 7ff3b4000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff3ac06fd40 con 0x7ff3ac0cdcc0 2024-04-25T14:07:40.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.683+0000 7ff3b4000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff3ac15e8a0 con 0x7ff3ac0d1df0 2024-04-25T14:07:40.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.683+0000 7ff3b2000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff3ac0cdcc0 0x7ff3ac0ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:40.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.683+0000 7ff3b2000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff3ac0cdcc0 0x7ff3ac0ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56470/0 (socket says 172.21.15.86:56470) 2024-04-25T14:07:40.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.683+0000 7ff3b2000640 1 -- 172.21.15.86:0/227961431 learned_addr learned my addr 172.21.15.86:0/227961431 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:40.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.683+0000 7ff3b0c00640 1 --2- 172.21.15.86:0/227961431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3ac0d1df0 0x7ff3ac0cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:40.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.683+0000 7ff3b2000640 1 -- 172.21.15.86:0/227961431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3ac0d1df0 msgr2=0x7ff3ac0cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:40.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.683+0000 7ff3b2000640 1 --2- 172.21.15.86:0/227961431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3ac0d1df0 0x7ff3ac0cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:40.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.683+0000 7ff3b2000640 1 -- 172.21.15.86:0/227961431 >> v1:172.21.15.155:6789/0 conn(0x7ff3ac070330 legacy=0x7ff3ac0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:40.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.683+0000 7ff3b2000640 1 -- 172.21.15.86:0/227961431 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3ac15eb80 con 0x7ff3ac0cdcc0 2024-04-25T14:07:40.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.687+0000 7ff3b2000640 1 --2- 172.21.15.86:0/227961431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3ac0cdcc0 0x7ff3ac0ce0d0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009e50 tx=0x7ff3a401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5ad87c850af942 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:40.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.687+0000 7ff3abe00640 1 -- 172.21.15.86:0/227961431 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a40a5810 con 0x7ff3ac0cdcc0 2024-04-25T14:07:40.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.687+0000 7ff3abe00640 1 -- 172.21.15.86:0/227961431 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3a40a59b0 con 0x7ff3ac0cdcc0 2024-04-25T14:07:40.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.687+0000 7ff3abe00640 1 -- 172.21.15.86:0/227961431 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a40a5d80 con 0x7ff3ac0cdcc0 2024-04-25T14:07:40.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.687+0000 7ff3b4000640 1 -- 172.21.15.86:0/227961431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3ac0cdcc0 msgr2=0x7ff3ac0ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:40.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.687+0000 7ff3b4000640 1 --2- 172.21.15.86:0/227961431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3ac0cdcc0 0x7ff3ac0ce0d0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009e50 tx=0x7ff3a401c970 comp rx=0 tx=0).stop 2024-04-25T14:07:40.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.687+0000 7ff3b4000640 1 -- 172.21.15.86:0/227961431 shutdown_connections 2024-04-25T14:07:40.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.687+0000 7ff3b4000640 1 --2- 172.21.15.86:0/227961431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3ac0cdcc0 0x7ff3ac0ce0d0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:40.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.687+0000 7ff3b4000640 1 --2- 172.21.15.86:0/227961431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3ac0d1df0 0x7ff3ac0cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:40.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.687+0000 7ff3b4000640 1 -- 172.21.15.86:0/227961431 >> 172.21.15.86:0/227961431 conn(0x7ff3ac0b4650 msgr2=0x7ff3ac0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:40.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.687+0000 7ff3b4000640 1 -- 172.21.15.86:0/227961431 shutdown_connections 2024-04-25T14:07:40.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.687+0000 7ff3b4000640 1 -- 172.21.15.86:0/227961431 wait complete. 2024-04-25T14:07:40.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.691+0000 7ff3b4000640 1 Processor -- start 2024-04-25T14:07:40.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.691+0000 7ff3b4000640 1 -- start start 2024-04-25T14:07:40.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.691+0000 7ff3b4000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3ac070330 0x7ff3ac0b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:40.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.691+0000 7ff3b4000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3ac0cdcc0 0x7ff3ac15ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:40.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.691+0000 7ff3b4000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3ac0ca420 0x7ff3ac0ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:40.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.691+0000 7ff3b4000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac15f4b0 con 0x7ff3ac070330 2024-04-25T14:07:40.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.691+0000 7ff3b4000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac15f330 con 0x7ff3ac0cdcc0 2024-04-25T14:07:40.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.691+0000 7ff3b4000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff3ac070740 con 0x7ff3ac0ca420 2024-04-25T14:07:40.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.691+0000 7ff3b1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3ac070330 0x7ff3ac0b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:40.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.691+0000 7ff3b1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3ac070330 0x7ff3ac0b1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56482/0 (socket says 172.21.15.86:56482) 2024-04-25T14:07:40.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.691+0000 7ff3b1600640 1 -- 172.21.15.86:0/1638939162 learned_addr learned my addr 172.21.15.86:0/1638939162 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:40.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.691+0000 7ff3b0c00640 1 --2- 172.21.15.86:0/1638939162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3ac0cdcc0 0x7ff3ac15ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:40.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.691+0000 7ff3b2000640 1 --2- 172.21.15.86:0/1638939162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3ac0ca420 0x7ff3ac0ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:40.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.691+0000 7ff3b1600640 1 -- 172.21.15.86:0/1638939162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3ac0ca420 msgr2=0x7ff3ac0ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:40.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.691+0000 7ff3b1600640 1 --2- 172.21.15.86:0/1638939162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3ac0ca420 0x7ff3ac0ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:40.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.691+0000 7ff3b1600640 1 -- 172.21.15.86:0/1638939162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3ac0cdcc0 msgr2=0x7ff3ac15ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:40.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.691+0000 7ff3b1600640 1 --2- 172.21.15.86:0/1638939162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3ac0cdcc0 0x7ff3ac15ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:40.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.691+0000 7ff3b1600640 1 -- 172.21.15.86:0/1638939162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3ac0cad70 con 0x7ff3ac070330 2024-04-25T14:07:40.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.691+0000 7ff3b1600640 1 --2- 172.21.15.86:0/1638939162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3ac070330 0x7ff3ac0b1dd0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000c390 tx=0x7ff3a000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:40.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.691+0000 7ff3a9600640 1 -- 172.21.15.86:0/1638939162 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0017070 con 0x7ff3ac070330 2024-04-25T14:07:40.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.691+0000 7ff3a9600640 1 -- 172.21.15.86:0/1638939162 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3a00052d0 con 0x7ff3ac070330 2024-04-25T14:07:40.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.691+0000 7ff3a9600640 1 -- 172.21.15.86:0/1638939162 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0003820 con 0x7ff3ac070330 2024-04-25T14:07:40.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.691+0000 7ff3b4000640 1 -- 172.21.15.86:0/1638939162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3ac1f9800 con 0x7ff3ac070330 2024-04-25T14:07:40.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.691+0000 7ff3b4000640 1 -- 172.21.15.86:0/1638939162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3ac0c6920 con 0x7ff3ac070330 2024-04-25T14:07:40.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.695+0000 7ff3b4000640 1 -- 172.21.15.86:0/1638939162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3ac069d00 con 0x7ff3ac070330 2024-04-25T14:07:40.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.695+0000 7ff3a9600640 1 -- 172.21.15.86:0/1638939162 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7ff3a001a070 con 0x7ff3ac070330 2024-04-25T14:07:40.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.695+0000 7ff3a9600640 1 --2- 172.21.15.86:0/1638939162 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7ff38c06f570 0x7ff38c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:40.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.695+0000 7ff3a9600640 1 -- 172.21.15.86:0/1638939162 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7ff3a008ee70 con 0x7ff3ac070330 2024-04-25T14:07:40.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.695+0000 7ff3b0c00640 1 --2- 172.21.15.86:0/1638939162 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7ff38c06f570 0x7ff38c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:40.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.695+0000 7ff3b0c00640 1 --2- 172.21.15.86:0/1638939162 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7ff38c06f570 0x7ff38c071a80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac074650 tx=0x7ff398008040 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:40.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.699+0000 7ff3a9600640 1 -- 172.21.15.86:0/1638939162 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff3a0021020 con 0x7ff3ac070330 2024-04-25T14:07:40.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.887+0000 7ff3b4000640 1 -- 172.21.15.86:0/1638939162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3ac076bf0 con 0x7ff3ac070330 2024-04-25T14:07:40.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.887+0000 7ff3a9600640 1 -- 172.21.15.86:0/1638939162 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5209 (secure 0 0 0) 0x7ff3ac0cad70 con 0x7ff3ac070330 2024-04-25T14:07:40.890 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:07:40.891 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":57,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1210051860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1210051860},{"type":"v1","addr":"172.21.15.155:6833","nonce":1210051860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2868300438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2868300438},{"type":"v1","addr":"172.21.15.155:6835","nonce":2868300438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:06:44.910717+0000","modified":"2024-04-25T14:07:19.703362+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6603,"mds_1":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"c","rank":1,"incarnation":55,"state":"up:active","state_seq":9,"addr":"172.21.15.86:6835/2682115891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2682115891},{"type":"v1","addr":"172.21.15.86:6835","nonce":2682115891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6603":{"gid":6603,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/1088108173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1088108173},{"type":"v1","addr":"172.21.15.86:6837","nonce":1088108173}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6603,"qdb_cluster":[6603,6600]},"id":5}]} 2024-04-25T14:07:40.891 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 57 2024-04-25T14:07:40.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.891+0000 7ff3b4000640 1 -- 172.21.15.86:0/1638939162 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7ff38c06f570 msgr2=0x7ff38c071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:40.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.891+0000 7ff3b4000640 1 --2- 172.21.15.86:0/1638939162 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7ff38c06f570 0x7ff38c071a80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac074650 tx=0x7ff398008040 comp rx=0 tx=0).stop 2024-04-25T14:07:40.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.891+0000 7ff3b4000640 1 -- 172.21.15.86:0/1638939162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3ac070330 msgr2=0x7ff3ac0b1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:40.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.891+0000 7ff3b4000640 1 --2- 172.21.15.86:0/1638939162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3ac070330 0x7ff3ac0b1dd0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000c390 tx=0x7ff3a000c850 comp rx=0 tx=0).stop 2024-04-25T14:07:40.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.891+0000 7ff3b4000640 1 -- 172.21.15.86:0/1638939162 shutdown_connections 2024-04-25T14:07:40.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.891+0000 7ff3b4000640 1 --2- 172.21.15.86:0/1638939162 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7ff38c06f570 0x7ff38c071a80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:40.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.891+0000 7ff3b4000640 1 --2- 172.21.15.86:0/1638939162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3ac0ca420 0x7ff3ac0ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:40.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.891+0000 7ff3b4000640 1 --2- 172.21.15.86:0/1638939162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3ac0cdcc0 0x7ff3ac15ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:40.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.891+0000 7ff3b4000640 1 --2- 172.21.15.86:0/1638939162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3ac070330 0x7ff3ac0b1dd0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:40.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.891+0000 7ff3b4000640 1 -- 172.21.15.86:0/1638939162 >> 172.21.15.86:0/1638939162 conn(0x7ff3ac0b4650 msgr2=0x7ff3ac0ae1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:40.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.891+0000 7ff3b4000640 1 -- 172.21.15.86:0/1638939162 shutdown_connections 2024-04-25T14:07:40.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:40.891+0000 7ff3b4000640 1 -- 172.21.15.86:0/1638939162 wait complete. 2024-04-25T14:07:41.913 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:07:42.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.067+0000 7fc065400640 1 Processor -- start 2024-04-25T14:07:42.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.067+0000 7fc065400640 1 -- start start 2024-04-25T14:07:42.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.067+0000 7fc065400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc060157f80 0x7fc0600aea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:42.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.067+0000 7fc065400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0600b0710 con 0x7fc06006f6d0 2024-04-25T14:07:42.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.067+0000 7fc065400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0600ae1c0 con 0x7fc060161b20 2024-04-25T14:07:42.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.067+0000 7fc065400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc0600aef70 con 0x7fc060157f80 2024-04-25T14:07:42.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.067+0000 7fc05fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc060157f80 0x7fc0600aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:42.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.067+0000 7fc05fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc060157f80 0x7fc0600aea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56492/0 (socket says 172.21.15.86:56492) 2024-04-25T14:07:42.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.067+0000 7fc05fe00640 1 -- 172.21.15.86:0/2579628797 learned_addr learned my addr 172.21.15.86:0/2579628797 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:42.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.067+0000 7fc057400640 1 --1- 172.21.15.86:0/2579628797 >> v1:172.21.15.86:6789/0 conn(0x7fc06006f6d0 0x7fc060161410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42916/0 (socket says 172.21.15.86:42916) 2024-04-25T14:07:42.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.067+0000 7fc05fe00640 1 -- 172.21.15.86:0/2579628797 >> v1:172.21.15.155:6789/0 conn(0x7fc060161b20 legacy=0x7fc06006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:42.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.067+0000 7fc05fe00640 1 -- 172.21.15.86:0/2579628797 >> v1:172.21.15.86:6789/0 conn(0x7fc06006f6d0 legacy=0x7fc060161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:42.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.067+0000 7fc05fe00640 1 -- 172.21.15.86:0/2579628797 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0600ae4a0 con 0x7fc060157f80 2024-04-25T14:07:42.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.071+0000 7fc05fe00640 1 --2- 172.21.15.86:0/2579628797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc060157f80 0x7fc0600aea30 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fc050009870 tx=0x7fc05001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f381c4b8cb315464 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:42.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.071+0000 7fc05f400640 1 -- 172.21.15.86:0/2579628797 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0500b0070 con 0x7fc060157f80 2024-04-25T14:07:42.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.071+0000 7fc05f400640 1 -- 172.21.15.86:0/2579628797 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0500ab600 con 0x7fc060157f80 2024-04-25T14:07:42.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.071+0000 7fc05f400640 1 -- 172.21.15.86:0/2579628797 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0500ab9d0 con 0x7fc060157f80 2024-04-25T14:07:42.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.071+0000 7fc065400640 1 -- 172.21.15.86:0/2579628797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc060157f80 msgr2=0x7fc0600aea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:42.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.071+0000 7fc065400640 1 --2- 172.21.15.86:0/2579628797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc060157f80 0x7fc0600aea30 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fc050009870 tx=0x7fc05001c4d0 comp rx=0 tx=0).stop 2024-04-25T14:07:42.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.071+0000 7fc065400640 1 -- 172.21.15.86:0/2579628797 shutdown_connections 2024-04-25T14:07:42.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.071+0000 7fc065400640 1 --2- 172.21.15.86:0/2579628797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc060157f80 0x7fc0600aea30 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:42.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.071+0000 7fc065400640 1 -- 172.21.15.86:0/2579628797 >> 172.21.15.86:0/2579628797 conn(0x7fc0600b4740 msgr2=0x7fc0600b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:42.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.071+0000 7fc065400640 1 -- 172.21.15.86:0/2579628797 shutdown_connections 2024-04-25T14:07:42.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.071+0000 7fc065400640 1 -- 172.21.15.86:0/2579628797 wait complete. 2024-04-25T14:07:42.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.075+0000 7fc065400640 1 Processor -- start 2024-04-25T14:07:42.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.075+0000 7fc065400640 1 -- start start 2024-04-25T14:07:42.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.075+0000 7fc065400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc06006f6d0 0x7fc060150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:42.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.075+0000 7fc065400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc060157f80 0x7fc06014f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:42.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.075+0000 7fc065400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc060146f70 0x7fc06014fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:42.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.075+0000 7fc065400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc06016c110 con 0x7fc060146f70 2024-04-25T14:07:42.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.075+0000 7fc065400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc06016bf90 con 0x7fc06006f6d0 2024-04-25T14:07:42.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.075+0000 7fc065400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc06016c290 con 0x7fc060157f80 2024-04-25T14:07:42.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.075+0000 7fc064a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc060146f70 0x7fc06014fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:42.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.075+0000 7fc064a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc060146f70 0x7fc06014fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56498/0 (socket says 172.21.15.86:56498) 2024-04-25T14:07:42.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.075+0000 7fc064a00640 1 -- 172.21.15.86:0/549602029 learned_addr learned my addr 172.21.15.86:0/549602029 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:42.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.075+0000 7fc057400640 1 --2- 172.21.15.86:0/549602029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc060157f80 0x7fc06014f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:42.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.075+0000 7fc05fe00640 1 --2- 172.21.15.86:0/549602029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc06006f6d0 0x7fc060150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:42.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.075+0000 7fc064a00640 1 -- 172.21.15.86:0/549602029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc060157f80 msgr2=0x7fc06014f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:42.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.075+0000 7fc064a00640 1 --2- 172.21.15.86:0/549602029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc060157f80 0x7fc06014f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:42.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.075+0000 7fc064a00640 1 -- 172.21.15.86:0/549602029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc06006f6d0 msgr2=0x7fc060150d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:42.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.075+0000 7fc064a00640 1 --2- 172.21.15.86:0/549602029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc06006f6d0 0x7fc060150d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:42.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.075+0000 7fc064a00640 1 -- 172.21.15.86:0/549602029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc060161350 con 0x7fc060146f70 2024-04-25T14:07:42.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.079+0000 7fc064a00640 1 --2- 172.21.15.86:0/549602029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc060146f70 0x7fc06014fa50 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fc05800c760 tx=0x7fc05800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:42.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.079+0000 7fc05cc00640 1 -- 172.21.15.86:0/549602029 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc058003b80 con 0x7fc060146f70 2024-04-25T14:07:42.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.079+0000 7fc05cc00640 1 -- 172.21.15.86:0/549602029 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc058003d20 con 0x7fc060146f70 2024-04-25T14:07:42.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.079+0000 7fc065400640 1 -- 172.21.15.86:0/549602029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc060161630 con 0x7fc060146f70 2024-04-25T14:07:42.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.079+0000 7fc05cc00640 1 -- 172.21.15.86:0/549602029 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05800aa30 con 0x7fc060146f70 2024-04-25T14:07:42.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.079+0000 7fc065400640 1 -- 172.21.15.86:0/549602029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc060152bd0 con 0x7fc060146f70 2024-04-25T14:07:42.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.079+0000 7fc065400640 1 -- 172.21.15.86:0/549602029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc060161b50 con 0x7fc060146f70 2024-04-25T14:07:42.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.079+0000 7fc05cc00640 1 -- 172.21.15.86:0/549602029 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7fc05800abd0 con 0x7fc060146f70 2024-04-25T14:07:42.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.079+0000 7fc05cc00640 1 --2- 172.21.15.86:0/549602029 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fc03c06f5a0 0x7fc03c071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:42.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.079+0000 7fc05cc00640 1 -- 172.21.15.86:0/549602029 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7fc0580901a0 con 0x7fc060146f70 2024-04-25T14:07:42.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.083+0000 7fc05fe00640 1 --2- 172.21.15.86:0/549602029 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fc03c06f5a0 0x7fc03c071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:42.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.083+0000 7fc05fe00640 1 --2- 172.21.15.86:0/549602029 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fc03c06f5a0 0x7fc03c071ab0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc0500ad000 tx=0x7fc050008040 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:42.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.083+0000 7fc05cc00640 1 -- 172.21.15.86:0/549602029 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc060161b50 con 0x7fc060146f70 2024-04-25T14:07:42.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.267+0000 7fc065400640 1 -- 172.21.15.86:0/549602029 --> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7fc060152e70 con 0x7fc03c06f5a0 2024-04-25T14:07:42.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.267+0000 7fc05cc00640 1 -- 172.21.15.86:0/549602029 <== mgr.6801 v2:172.21.15.86:6832/203029814 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3052 (secure 0 0 0) 0x7fc060152e70 con 0x7fc03c06f5a0 2024-04-25T14:07:42.269 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {"cephfs": {"client.5945": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:172.21.15.86"}, "client.5951": {"hostname": "smithi155", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:192.168.0.1"}, "client.6834": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}, "client.6837": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}}}, "global_metrics": {"cephfs": {"client.5945": [[0, 0], [0, 0], [0, 396500909], [0, 71840640], [34, 0], [0, 35], [35, 35], [0, 35], [0, 0], [32, 33554432], [0, 0], [0, 0], [0, 12390659], [275776813322337, 32], [0, 1122510], [133594044029089, 64]], "client.5951": [[0, 0], [0, 0], [0, 0], [0, 20679725], [6, 0], [0, 3], [3, 3], [0, 3], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 2297747], [50231033486578, 9]], "client.6834": [[0, 0], [0, 0], [0, 0], [0, 510044], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 510043], [0, 1]], "client.6837": [[0, 0], [0, 0], [0, 0], [0, 761751], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 761751], [0, 1]]}}, "metrics": {"delayed_ranks": [1], "mds.0": {"client.5945": [], "client.5951": [], "client.6834": [], "client.6837": []}, "mds.1": {"client.5945": [], "client.5951": []}}} 2024-04-25T14:07:42.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.271+0000 7fc065400640 1 -- 172.21.15.86:0/549602029 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fc03c06f5a0 msgr2=0x7fc03c071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:42.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.271+0000 7fc065400640 1 --2- 172.21.15.86:0/549602029 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fc03c06f5a0 0x7fc03c071ab0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc0500ad000 tx=0x7fc050008040 comp rx=0 tx=0).stop 2024-04-25T14:07:42.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.271+0000 7fc065400640 1 -- 172.21.15.86:0/549602029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc060146f70 msgr2=0x7fc06014fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:42.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.271+0000 7fc065400640 1 --2- 172.21.15.86:0/549602029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc060146f70 0x7fc06014fa50 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fc05800c760 tx=0x7fc05800cc20 comp rx=0 tx=0).stop 2024-04-25T14:07:42.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.271+0000 7fc065400640 1 -- 172.21.15.86:0/549602029 shutdown_connections 2024-04-25T14:07:42.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.271+0000 7fc065400640 1 --2- 172.21.15.86:0/549602029 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fc03c06f5a0 0x7fc03c071ab0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:42.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.271+0000 7fc065400640 1 --2- 172.21.15.86:0/549602029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc060146f70 0x7fc06014fa50 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:42.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.271+0000 7fc065400640 1 --2- 172.21.15.86:0/549602029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc060157f80 0x7fc06014f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:42.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.271+0000 7fc065400640 1 --2- 172.21.15.86:0/549602029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc06006f6d0 0x7fc060150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:42.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.271+0000 7fc065400640 1 -- 172.21.15.86:0/549602029 >> 172.21.15.86:0/549602029 conn(0x7fc0600b4740 msgr2=0x7fc0600b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:42.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.271+0000 7fc065400640 1 -- 172.21.15.86:0/549602029 shutdown_connections 2024-04-25T14:07:42.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.271+0000 7fc065400640 1 -- 172.21.15.86:0/549602029 wait complete. 2024-04-25T14:07:42.291 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:07:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.435+0000 7efdc4e00640 1 Processor -- start 2024-04-25T14:07:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.435+0000 7efdc4e00640 1 -- start start 2024-04-25T14:07:42.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.439+0000 7efdc4e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdc01642a0 0x7efdc00d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:42.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.439+0000 7efdc4e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efdc00b0860 0x7efdc00d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:42.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.439+0000 7efdc4e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdc00d2030 con 0x7efdc00d1c20 2024-04-25T14:07:42.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.439+0000 7efdc4e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7efdc00cea30 con 0x7efdc00b0860 2024-04-25T14:07:42.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.439+0000 7efdc4e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efdc00cde20 con 0x7efdc01642a0 2024-04-25T14:07:42.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.439+0000 7efdbe800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efdc00b0860 0x7efdc00d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:42.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.439+0000 7efdbe800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efdc00b0860 0x7efdc00d0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56506/0 (socket says 172.21.15.86:56506) 2024-04-25T14:07:42.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.439+0000 7efdbe800640 1 -- 172.21.15.86:0/251637254 learned_addr learned my addr 172.21.15.86:0/251637254 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:42.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.439+0000 7efdbde00640 1 --2- 172.21.15.86:0/251637254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdc01642a0 0x7efdc00d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:42.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.439+0000 7efdbe800640 1 -- 172.21.15.86:0/251637254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdc01642a0 msgr2=0x7efdc00d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:42.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.439+0000 7efdbe800640 1 --2- 172.21.15.86:0/251637254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdc01642a0 0x7efdc00d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:42.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.439+0000 7efdbe800640 1 -- 172.21.15.86:0/251637254 >> v1:172.21.15.155:6789/0 conn(0x7efdc00d1c20 legacy=0x7efdc00b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:42.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.439+0000 7efdbe800640 1 -- 172.21.15.86:0/251637254 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc00ce100 con 0x7efdc00b0860 2024-04-25T14:07:42.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.439+0000 7efdbde00640 1 --2- 172.21.15.86:0/251637254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdc01642a0 0x7efdc00d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:42.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.439+0000 7efdbe800640 1 --2- 172.21.15.86:0/251637254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc00b0860 0x7efdc00d0640 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009870 tx=0x7efdb801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397c611343a3940 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:42.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.443+0000 7efdbca00640 1 -- 172.21.15.86:0/251637254 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb80b0070 con 0x7efdc00b0860 2024-04-25T14:07:42.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.443+0000 7efdbca00640 1 -- 172.21.15.86:0/251637254 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdb80ab600 con 0x7efdc00b0860 2024-04-25T14:07:42.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.443+0000 7efdbca00640 1 -- 172.21.15.86:0/251637254 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb80ab9d0 con 0x7efdc00b0860 2024-04-25T14:07:42.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.443+0000 7efdc4e00640 1 -- 172.21.15.86:0/251637254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc00b0860 msgr2=0x7efdc00d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:42.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.443+0000 7efdc4e00640 1 --2- 172.21.15.86:0/251637254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc00b0860 0x7efdc00d0640 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009870 tx=0x7efdb801c4d0 comp rx=0 tx=0).stop 2024-04-25T14:07:42.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.443+0000 7efdc4e00640 1 -- 172.21.15.86:0/251637254 shutdown_connections 2024-04-25T14:07:42.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.443+0000 7efdc4e00640 1 --2- 172.21.15.86:0/251637254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc00b0860 0x7efdc00d0640 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:42.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.443+0000 7efdc4e00640 1 --2- 172.21.15.86:0/251637254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdc01642a0 0x7efdc00d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:42.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.443+0000 7efdc4e00640 1 -- 172.21.15.86:0/251637254 >> 172.21.15.86:0/251637254 conn(0x7efdc00b4650 msgr2=0x7efdc00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:42.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.443+0000 7efdc4e00640 1 -- 172.21.15.86:0/251637254 shutdown_connections 2024-04-25T14:07:42.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.443+0000 7efdc4e00640 1 -- 172.21.15.86:0/251637254 wait complete. 2024-04-25T14:07:42.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.443+0000 7efdc4e00640 1 Processor -- start 2024-04-25T14:07:42.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.443+0000 7efdc4e00640 1 -- start start 2024-04-25T14:07:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.447+0000 7efdc4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc00b0860 0x7efdc015a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.447+0000 7efdc4e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdc00d1c20 0x7efdc00ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.447+0000 7efdc4e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdc01642a0 0x7efdc00ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.447+0000 7efdc4e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efdc0164830 con 0x7efdc00b0860 2024-04-25T14:07:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.447+0000 7efdc4e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efdc01646b0 con 0x7efdc00d1c20 2024-04-25T14:07:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.447+0000 7efdc4e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efdc0160da0 con 0x7efdc01642a0 2024-04-25T14:07:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.447+0000 7efdbde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc00b0860 0x7efdc015a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.447+0000 7efdbde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc00b0860 0x7efdc015a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56520/0 (socket says 172.21.15.86:56520) 2024-04-25T14:07:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.447+0000 7efdbde00640 1 -- 172.21.15.86:0/1255194568 learned_addr learned my addr 172.21.15.86:0/1255194568 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.447+0000 7efdbd400640 1 --2- 172.21.15.86:0/1255194568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdc00d1c20 0x7efdc00ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.447+0000 7efdbe800640 1 --2- 172.21.15.86:0/1255194568 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdc01642a0 0x7efdc00ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.447+0000 7efdbde00640 1 -- 172.21.15.86:0/1255194568 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdc01642a0 msgr2=0x7efdc00ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.447+0000 7efdbde00640 1 --2- 172.21.15.86:0/1255194568 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdc01642a0 0x7efdc00ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.447+0000 7efdbde00640 1 -- 172.21.15.86:0/1255194568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdc00d1c20 msgr2=0x7efdc00ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.447+0000 7efdbde00640 1 --2- 172.21.15.86:0/1255194568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdc00d1c20 0x7efdc00ae300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.447+0000 7efdbde00640 1 -- 172.21.15.86:0/1255194568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc00af0d0 con 0x7efdc00b0860 2024-04-25T14:07:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.447+0000 7efdbe800640 1 --2- 172.21.15.86:0/1255194568 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdc01642a0 0x7efdc00ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.447+0000 7efdbd400640 1 --2- 172.21.15.86:0/1255194568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdc00d1c20 0x7efdc00ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.447+0000 7efdbde00640 1 --2- 172.21.15.86:0/1255194568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc00b0860 0x7efdc015a1a0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7efdb000cde0 tx=0x7efdb000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.447+0000 7efdb6000640 1 -- 172.21.15.86:0/1255194568 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb000f680 con 0x7efdc00b0860 2024-04-25T14:07:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.447+0000 7efdb6000640 1 -- 172.21.15.86:0/1255194568 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdb000f820 con 0x7efdc00b0860 2024-04-25T14:07:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.447+0000 7efdb6000640 1 -- 172.21.15.86:0/1255194568 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb000fc10 con 0x7efdc00b0860 2024-04-25T14:07:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.447+0000 7efdc4e00640 1 -- 172.21.15.86:0/1255194568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc0073530 con 0x7efdc00b0860 2024-04-25T14:07:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.447+0000 7efdc4e00640 1 -- 172.21.15.86:0/1255194568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc0073af0 con 0x7efdc00b0860 2024-04-25T14:07:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.447+0000 7efdc4e00640 1 -- 172.21.15.86:0/1255194568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdc0069d00 con 0x7efdc00b0860 2024-04-25T14:07:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.451+0000 7efdb6000640 1 -- 172.21.15.86:0/1255194568 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7efdb0018400 con 0x7efdc00b0860 2024-04-25T14:07:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.451+0000 7efdb6000640 1 --2- 172.21.15.86:0/1255194568 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7efda006f570 0x7efda0071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.451+0000 7efdb6000640 1 -- 172.21.15.86:0/1255194568 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7efdb00906e0 con 0x7efdc00b0860 2024-04-25T14:07:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.451+0000 7efdbd400640 1 --2- 172.21.15.86:0/1255194568 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7efda006f570 0x7efda0071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.451+0000 7efdbd400640 1 --2- 172.21.15.86:0/1255194568 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7efda006f570 0x7efda0071a80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efdc00b05d0 tx=0x7efdac009040 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.455+0000 7efdb6000640 1 -- 172.21.15.86:0/1255194568 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efdb0014030 con 0x7efdc00b0860 2024-04-25T14:07:42.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.643+0000 7efdc4e00640 1 -- 172.21.15.86:0/1255194568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efdc0076bf0 con 0x7efdc00b0860 2024-04-25T14:07:42.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.643+0000 7efdb6000640 1 -- 172.21.15.86:0/1255194568 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5209 (secure 0 0 0) 0x7efdb005ca10 con 0x7efdc00b0860 2024-04-25T14:07:42.646 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:07:42.646 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":57,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1210051860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1210051860},{"type":"v1","addr":"172.21.15.155:6833","nonce":1210051860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2868300438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2868300438},{"type":"v1","addr":"172.21.15.155:6835","nonce":2868300438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:06:44.910717+0000","modified":"2024-04-25T14:07:19.703362+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6603,"mds_1":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"c","rank":1,"incarnation":55,"state":"up:active","state_seq":9,"addr":"172.21.15.86:6835/2682115891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2682115891},{"type":"v1","addr":"172.21.15.86:6835","nonce":2682115891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6603":{"gid":6603,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/1088108173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1088108173},{"type":"v1","addr":"172.21.15.86:6837","nonce":1088108173}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6603,"qdb_cluster":[6603,6600]},"id":5}]} 2024-04-25T14:07:42.646 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 57 2024-04-25T14:07:42.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.647+0000 7efdc4e00640 1 -- 172.21.15.86:0/1255194568 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7efda006f570 msgr2=0x7efda0071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:42.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.647+0000 7efdc4e00640 1 --2- 172.21.15.86:0/1255194568 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7efda006f570 0x7efda0071a80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efdc00b05d0 tx=0x7efdac009040 comp rx=0 tx=0).stop 2024-04-25T14:07:42.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.647+0000 7efdc4e00640 1 -- 172.21.15.86:0/1255194568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc00b0860 msgr2=0x7efdc015a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:42.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.647+0000 7efdc4e00640 1 --2- 172.21.15.86:0/1255194568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc00b0860 0x7efdc015a1a0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7efdb000cde0 tx=0x7efdb000a650 comp rx=0 tx=0).stop 2024-04-25T14:07:42.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.647+0000 7efdc4e00640 1 -- 172.21.15.86:0/1255194568 shutdown_connections 2024-04-25T14:07:42.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.647+0000 7efdc4e00640 1 --2- 172.21.15.86:0/1255194568 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7efda006f570 0x7efda0071a80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:42.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.647+0000 7efdc4e00640 1 --2- 172.21.15.86:0/1255194568 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdc01642a0 0x7efdc00ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:42.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.647+0000 7efdc4e00640 1 --2- 172.21.15.86:0/1255194568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdc00d1c20 0x7efdc00ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:42.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.647+0000 7efdc4e00640 1 --2- 172.21.15.86:0/1255194568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc00b0860 0x7efdc015a1a0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:42.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.647+0000 7efdc4e00640 1 -- 172.21.15.86:0/1255194568 >> 172.21.15.86:0/1255194568 conn(0x7efdc00b4650 msgr2=0x7efdc00cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:42.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.647+0000 7efdc4e00640 1 -- 172.21.15.86:0/1255194568 shutdown_connections 2024-04-25T14:07:42.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:42.647+0000 7efdc4e00640 1 -- 172.21.15.86:0/1255194568 wait complete. 2024-04-25T14:07:43.669 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:07:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.827+0000 7f27d9600640 1 Processor -- start 2024-04-25T14:07:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.827+0000 7f27d9600640 1 -- start start 2024-04-25T14:07:43.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.827+0000 7f27d9600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f27d409ba30 0x7f27d415fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:43.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.827+0000 7f27d9600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27d41639c0 0x7f27d4163dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.827+0000 7f27d9600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27d40d2170 con 0x7f27d4164310 2024-04-25T14:07:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.827+0000 7f27d9600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f27d40d2450 con 0x7f27d409ba30 2024-04-25T14:07:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.827+0000 7f27d9600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f27d40d2730 con 0x7f27d41639c0 2024-04-25T14:07:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.827+0000 7f27d3e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f27d409ba30 0x7f27d415fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.827+0000 7f27d3e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f27d409ba30 0x7f27d415fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56534/0 (socket says 172.21.15.86:56534) 2024-04-25T14:07:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.827+0000 7f27d3e00640 1 -- 172.21.15.86:0/250749515 learned_addr learned my addr 172.21.15.86:0/250749515 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.831+0000 7f27d3e00640 1 -- 172.21.15.86:0/250749515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27d41639c0 msgr2=0x7f27d4163dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.831+0000 7f27d3e00640 1 --2- 172.21.15.86:0/250749515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27d41639c0 0x7f27d4163dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.831+0000 7f27d3e00640 1 -- 172.21.15.86:0/250749515 >> v1:172.21.15.86:6789/0 conn(0x7f27d4164310 legacy=0x7f27d40ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:43.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.831+0000 7f27d3e00640 1 -- 172.21.15.86:0/250749515 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d40d2a10 con 0x7f27d409ba30 2024-04-25T14:07:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.831+0000 7f27d3e00640 1 --2- 172.21.15.86:0/250749515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27d409ba30 0x7f27d415fc80 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f27c4009870 tx=0x7f27c401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3944d1d4c8d02a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.831+0000 7f27d2a00640 1 -- 172.21.15.86:0/250749515 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c40b0070 con 0x7f27d409ba30 2024-04-25T14:07:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.831+0000 7f27d2a00640 1 -- 172.21.15.86:0/250749515 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27c40ab600 con 0x7f27d409ba30 2024-04-25T14:07:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.831+0000 7f27d2a00640 1 -- 172.21.15.86:0/250749515 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c40ab9d0 con 0x7f27d409ba30 2024-04-25T14:07:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.831+0000 7f27d9600640 1 -- 172.21.15.86:0/250749515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27d409ba30 msgr2=0x7f27d415fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.831+0000 7f27d9600640 1 --2- 172.21.15.86:0/250749515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27d409ba30 0x7f27d415fc80 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f27c4009870 tx=0x7f27c401c560 comp rx=0 tx=0).stop 2024-04-25T14:07:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.831+0000 7f27d9600640 1 -- 172.21.15.86:0/250749515 shutdown_connections 2024-04-25T14:07:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.831+0000 7f27d9600640 1 --2- 172.21.15.86:0/250749515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27d41639c0 0x7f27d4163dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.831+0000 7f27d9600640 1 --2- 172.21.15.86:0/250749515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27d409ba30 0x7f27d415fc80 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.831+0000 7f27d9600640 1 -- 172.21.15.86:0/250749515 >> 172.21.15.86:0/250749515 conn(0x7f27d4073c00 msgr2=0x7f27d40b77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.835+0000 7f27d9600640 1 -- 172.21.15.86:0/250749515 shutdown_connections 2024-04-25T14:07:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.835+0000 7f27d9600640 1 -- 172.21.15.86:0/250749515 wait complete. 2024-04-25T14:07:43.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.835+0000 7f27d9600640 1 Processor -- start 2024-04-25T14:07:43.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.835+0000 7f27d9600640 1 -- start start 2024-04-25T14:07:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.835+0000 7f27d9600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27d409ba30 0x7f27d40cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.835+0000 7f27d9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27d41639c0 0x7f27d4148cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.835+0000 7f27d9600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27d4164310 0x7f27d4162c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.835+0000 7f27d9600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f27d416cee0 con 0x7f27d41639c0 2024-04-25T14:07:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.835+0000 7f27d9600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f27d416cd60 con 0x7f27d409ba30 2024-04-25T14:07:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.835+0000 7f27d9600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f27d416d060 con 0x7f27d4164310 2024-04-25T14:07:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.839+0000 7f27d3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27d41639c0 0x7f27d4148cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:43.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.839+0000 7f27d3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27d41639c0 0x7f27d4148cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56550/0 (socket says 172.21.15.86:56550) 2024-04-25T14:07:43.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.839+0000 7f27d3400640 1 -- 172.21.15.86:0/1997696590 learned_addr learned my addr 172.21.15.86:0/1997696590 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:43.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.839+0000 7f27d3400640 1 -- 172.21.15.86:0/1997696590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27d4164310 msgr2=0x7f27d4162c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:43.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.839+0000 7f27d3400640 1 --2- 172.21.15.86:0/1997696590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27d4164310 0x7f27d4162c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:43.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.839+0000 7f27d3400640 1 -- 172.21.15.86:0/1997696590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27d409ba30 msgr2=0x7f27d40cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:43.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.839+0000 7f27d3400640 1 --2- 172.21.15.86:0/1997696590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27d409ba30 0x7f27d40cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:43.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.839+0000 7f27d3400640 1 -- 172.21.15.86:0/1997696590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d40ccb10 con 0x7f27d41639c0 2024-04-25T14:07:43.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.839+0000 7f27d3400640 1 --2- 172.21.15.86:0/1997696590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27d41639c0 0x7f27d4148cb0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f27c000b410 tx=0x7f27c000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:43.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.839+0000 7f27cbe00640 1 -- 172.21.15.86:0/1997696590 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c0015020 con 0x7f27d41639c0 2024-04-25T14:07:43.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.839+0000 7f27cbe00640 1 -- 172.21.15.86:0/1997696590 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27c0011830 con 0x7f27d41639c0 2024-04-25T14:07:43.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.839+0000 7f27cbe00640 1 -- 172.21.15.86:0/1997696590 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c0011c20 con 0x7f27d41639c0 2024-04-25T14:07:43.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.839+0000 7f27d9600640 1 -- 172.21.15.86:0/1997696590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d40ccdf0 con 0x7f27d41639c0 2024-04-25T14:07:43.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.839+0000 7f27d9600640 1 -- 172.21.15.86:0/1997696590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d40cd3b0 con 0x7f27d41639c0 2024-04-25T14:07:43.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.839+0000 7f27d9600640 1 -- 172.21.15.86:0/1997696590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27d4076b80 con 0x7f27d41639c0 2024-04-25T14:07:43.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.839+0000 7f27cbe00640 1 -- 172.21.15.86:0/1997696590 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7f27c0018020 con 0x7f27d41639c0 2024-04-25T14:07:43.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.843+0000 7f27cbe00640 1 --2- 172.21.15.86:0/1997696590 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f27b406f5a0 0x7f27b4071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:43.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.843+0000 7f27cbe00640 1 -- 172.21.15.86:0/1997696590 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f27c008f740 con 0x7f27d41639c0 2024-04-25T14:07:43.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.843+0000 7f27d3e00640 1 --2- 172.21.15.86:0/1997696590 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f27b406f5a0 0x7f27b4071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:43.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.843+0000 7f27d3e00640 1 --2- 172.21.15.86:0/1997696590 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f27b406f5a0 0x7f27b4071ab0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f27c40ad000 tx=0x7f27c4008040 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:43.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:43.847+0000 7f27cbe00640 1 -- 172.21.15.86:0/1997696590 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f27c001f020 con 0x7f27d41639c0 2024-04-25T14:07:44.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.023+0000 7f27d9600640 1 -- 172.21.15.86:0/1997696590 --> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f27d40742d0 con 0x7f27b406f5a0 2024-04-25T14:07:44.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.027+0000 7f27cbe00640 1 -- 172.21.15.86:0/1997696590 <== mgr.6801 v2:172.21.15.86:6832/203029814 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3052 (secure 0 0 0) 0x7f27d40742d0 con 0x7f27b406f5a0 2024-04-25T14:07:44.027 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {"cephfs": {"client.5945": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:172.21.15.86"}, "client.5951": {"hostname": "smithi155", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:192.168.0.1"}, "client.6834": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}, "client.6837": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}}}, "global_metrics": {"cephfs": {"client.5945": [[0, 0], [0, 0], [0, 396500909], [0, 71840640], [34, 0], [0, 35], [35, 35], [0, 35], [0, 0], [32, 33554432], [0, 0], [0, 0], [0, 12390659], [275776813322337, 32], [0, 1122510], [133594044029089, 64]], "client.5951": [[0, 0], [0, 0], [0, 0], [0, 20679725], [6, 0], [0, 3], [3, 3], [0, 3], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 2297747], [50231033486578, 9]], "client.6834": [[0, 0], [0, 0], [0, 0], [0, 510044], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 510043], [0, 1]], "client.6837": [[0, 0], [0, 0], [0, 0], [0, 761751], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 761751], [0, 1]]}}, "metrics": {"delayed_ranks": [1], "mds.0": {"client.5945": [], "client.5951": [], "client.6834": [], "client.6837": []}, "mds.1": {"client.5945": [], "client.5951": []}}} 2024-04-25T14:07:44.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.027+0000 7f27d9600640 1 -- 172.21.15.86:0/1997696590 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f27b406f5a0 msgr2=0x7f27b4071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:44.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.027+0000 7f27d9600640 1 --2- 172.21.15.86:0/1997696590 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f27b406f5a0 0x7f27b4071ab0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f27c40ad000 tx=0x7f27c4008040 comp rx=0 tx=0).stop 2024-04-25T14:07:44.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.027+0000 7f27d9600640 1 -- 172.21.15.86:0/1997696590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27d41639c0 msgr2=0x7f27d4148cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:44.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.027+0000 7f27d9600640 1 --2- 172.21.15.86:0/1997696590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27d41639c0 0x7f27d4148cb0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f27c000b410 tx=0x7f27c000b8d0 comp rx=0 tx=0).stop 2024-04-25T14:07:44.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.027+0000 7f27d9600640 1 -- 172.21.15.86:0/1997696590 shutdown_connections 2024-04-25T14:07:44.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.027+0000 7f27d9600640 1 --2- 172.21.15.86:0/1997696590 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f27b406f5a0 0x7f27b4071ab0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:44.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.027+0000 7f27d9600640 1 --2- 172.21.15.86:0/1997696590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27d4164310 0x7f27d4162c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:44.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.027+0000 7f27d9600640 1 --2- 172.21.15.86:0/1997696590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27d41639c0 0x7f27d4148cb0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:44.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.027+0000 7f27d9600640 1 --2- 172.21.15.86:0/1997696590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27d409ba30 0x7f27d40cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:44.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.027+0000 7f27d9600640 1 -- 172.21.15.86:0/1997696590 >> 172.21.15.86:0/1997696590 conn(0x7f27d4073c00 msgr2=0x7f27d4070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.027+0000 7f27d9600640 1 -- 172.21.15.86:0/1997696590 shutdown_connections 2024-04-25T14:07:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.027+0000 7f27d9600640 1 -- 172.21.15.86:0/1997696590 wait complete. 2024-04-25T14:07:44.049 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:07:44.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.195+0000 7f0fdf600640 1 Processor -- start 2024-04-25T14:07:44.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.195+0000 7f0fdf600640 1 -- start start 2024-04-25T14:07:44.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.199+0000 7f0fdf600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fd80cfca0 0x7f0fd806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:44.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.199+0000 7f0fdf600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0fd80d16e0 0x7f0fd80d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:44.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.199+0000 7f0fdf600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0fd806c670 0x7f0fd806ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:44.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.199+0000 7f0fdf600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0fd8070470 con 0x7f0fd80d16e0 2024-04-25T14:07:44.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.199+0000 7f0fdf600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0fd80d2030 con 0x7f0fd806c670 2024-04-25T14:07:44.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.199+0000 7f0fdf600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0fd8072bd0 con 0x7f0fd80cfca0 2024-04-25T14:07:44.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.199+0000 7f0fd7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0fd80d16e0 0x7f0fd80d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:44.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.199+0000 7f0fd7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0fd80d16e0 0x7f0fd80d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56554/0 (socket says 172.21.15.86:56554) 2024-04-25T14:07:44.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.199+0000 7f0fd7e00640 1 -- 172.21.15.86:0/2289702723 learned_addr learned my addr 172.21.15.86:0/2289702723 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:44.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.199+0000 7f0fdcc00640 1 --2- 172.21.15.86:0/2289702723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fd80cfca0 0x7f0fd806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:44.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.199+0000 7f0fdd600640 1 --2- 172.21.15.86:0/2289702723 >> v2:172.21.15.155:3300/0 conn(0x7f0fd806c670 0x7f0fd806ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:44.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.199+0000 7f0fd7e00640 1 -- 172.21.15.86:0/2289702723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fd80cfca0 msgr2=0x7f0fd806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:44.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.199+0000 7f0fd7e00640 1 --2- 172.21.15.86:0/2289702723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fd80cfca0 0x7f0fd806f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:44.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.199+0000 7f0fd7e00640 1 -- 172.21.15.86:0/2289702723 >> v2:172.21.15.155:3300/0 conn(0x7f0fd806c670 msgr2=0x7f0fd806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:44.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.199+0000 7f0fd7e00640 1 --2- 172.21.15.86:0/2289702723 >> v2:172.21.15.155:3300/0 conn(0x7f0fd806c670 0x7f0fd806ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:44.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.199+0000 7f0fd7e00640 1 -- 172.21.15.86:0/2289702723 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd806f930 con 0x7f0fd80d16e0 2024-04-25T14:07:44.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.203+0000 7f0fd7e00640 1 --2- 172.21.15.86:0/2289702723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fd80d16e0 0x7f0fd80d1af0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f0fd806e060 tx=0x7f0fc401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90bd3c04ac6c06cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:44.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.203+0000 7f0fd7400640 1 -- 172.21.15.86:0/2289702723 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc40ab5f0 con 0x7f0fd80d16e0 2024-04-25T14:07:44.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.203+0000 7f0fd7400640 1 -- 172.21.15.86:0/2289702723 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fc40b0070 con 0x7f0fd80d16e0 2024-04-25T14:07:44.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.203+0000 7f0fd7400640 1 -- 172.21.15.86:0/2289702723 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc40ab9b0 con 0x7f0fd80d16e0 2024-04-25T14:07:44.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.203+0000 7f0fdf600640 1 -- 172.21.15.86:0/2289702723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fd80d16e0 msgr2=0x7f0fd80d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:44.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.203+0000 7f0fdf600640 1 --2- 172.21.15.86:0/2289702723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fd80d16e0 0x7f0fd80d1af0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f0fd806e060 tx=0x7f0fc401c520 comp rx=0 tx=0).stop 2024-04-25T14:07:44.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.203+0000 7f0fdf600640 1 -- 172.21.15.86:0/2289702723 shutdown_connections 2024-04-25T14:07:44.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.203+0000 7f0fdf600640 1 --2- 172.21.15.86:0/2289702723 >> v2:172.21.15.155:3300/0 conn(0x7f0fd806c670 0x7f0fd806ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:44.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.203+0000 7f0fdf600640 1 --2- 172.21.15.86:0/2289702723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fd80d16e0 0x7f0fd80d1af0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:44.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.203+0000 7f0fdf600640 1 --2- 172.21.15.86:0/2289702723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fd80cfca0 0x7f0fd806f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:44.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.203+0000 7f0fdf600640 1 -- 172.21.15.86:0/2289702723 >> 172.21.15.86:0/2289702723 conn(0x7f0fd80b4650 msgr2=0x7f0fd80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:44.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.203+0000 7f0fdf600640 1 -- 172.21.15.86:0/2289702723 shutdown_connections 2024-04-25T14:07:44.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.203+0000 7f0fdf600640 1 -- 172.21.15.86:0/2289702723 wait complete. 2024-04-25T14:07:44.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.207+0000 7f0fdf600640 1 Processor -- start 2024-04-25T14:07:44.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.207+0000 7f0fdf600640 1 -- start start 2024-04-25T14:07:44.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.207+0000 7f0fdf600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fd806c670 0x7f0fd80ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:44.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.207+0000 7f0fdf600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fd80cfca0 0x7f0fd81f9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:44.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.207+0000 7f0fdf600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fd80d16e0 0x7f0fd81fa250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:44.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.207+0000 7f0fdf600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd80738c0 con 0x7f0fd806c670 2024-04-25T14:07:44.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.207+0000 7f0fdf600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd806d3d0 con 0x7f0fd80d16e0 2024-04-25T14:07:44.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.207+0000 7f0fdf600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0fd8073a40 con 0x7f0fd80cfca0 2024-04-25T14:07:44.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.207+0000 7f0fdcc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fd806c670 0x7f0fd80ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:44.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.207+0000 7f0fdcc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fd806c670 0x7f0fd80ca7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56568/0 (socket says 172.21.15.86:56568) 2024-04-25T14:07:44.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.207+0000 7f0fdcc00640 1 -- 172.21.15.86:0/1070436155 learned_addr learned my addr 172.21.15.86:0/1070436155 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:44.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.207+0000 7f0fdcc00640 1 -- 172.21.15.86:0/1070436155 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fd80cfca0 msgr2=0x7f0fd81f9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:44.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.207+0000 7f0fdcc00640 1 --2- 172.21.15.86:0/1070436155 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fd80cfca0 0x7f0fd81f9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:44.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.207+0000 7f0fdcc00640 1 -- 172.21.15.86:0/1070436155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fd80d16e0 msgr2=0x7f0fd81fa250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:44.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.207+0000 7f0fdd600640 1 --2- 172.21.15.86:0/1070436155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fd80d16e0 0x7f0fd81fa250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:44.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.207+0000 7f0fdcc00640 1 --2- 172.21.15.86:0/1070436155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fd80d16e0 0x7f0fd81fa250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:44.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.207+0000 7f0fdcc00640 1 -- 172.21.15.86:0/1070436155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd80c9750 con 0x7f0fd806c670 2024-04-25T14:07:44.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.207+0000 7f0fdd600640 1 --2- 172.21.15.86:0/1070436155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fd80d16e0 0x7f0fd81fa250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:44.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.207+0000 7f0fdcc00640 1 --2- 172.21.15.86:0/1070436155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fd806c670 0x7f0fd80ca7f0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc00e840 tx=0x7f0fcc00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:44.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.207+0000 7f0fd4c00640 1 -- 172.21.15.86:0/1070436155 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc017070 con 0x7f0fd806c670 2024-04-25T14:07:44.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.207+0000 7f0fd4c00640 1 -- 172.21.15.86:0/1070436155 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fcc00c6d0 con 0x7f0fd806c670 2024-04-25T14:07:44.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.207+0000 7f0fd4c00640 1 -- 172.21.15.86:0/1070436155 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc00caa0 con 0x7f0fd806c670 2024-04-25T14:07:44.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.207+0000 7f0fdf600640 1 -- 172.21.15.86:0/1070436155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fd80c9a30 con 0x7f0fd806c670 2024-04-25T14:07:44.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.207+0000 7f0fdf600640 1 -- 172.21.15.86:0/1070436155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fd80c9ff0 con 0x7f0fd806c670 2024-04-25T14:07:44.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.211+0000 7f0fdf600640 1 -- 172.21.15.86:0/1070436155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fd809f4b0 con 0x7f0fd806c670 2024-04-25T14:07:44.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.211+0000 7f0fd4c00640 1 -- 172.21.15.86:0/1070436155 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7f0fcc01a070 con 0x7f0fd806c670 2024-04-25T14:07:44.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.211+0000 7f0fd4c00640 1 --2- 172.21.15.86:0/1070436155 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f0fb806f5a0 0x7f0fb8071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:44.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.211+0000 7f0fd4c00640 1 -- 172.21.15.86:0/1070436155 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f0fcc08f920 con 0x7f0fd806c670 2024-04-25T14:07:44.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.211+0000 7f0fd7e00640 1 --2- 172.21.15.86:0/1070436155 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f0fb806f5a0 0x7f0fb8071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:44.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.211+0000 7f0fd7e00640 1 --2- 172.21.15.86:0/1070436155 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f0fb806f5a0 0x7f0fb8071ab0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40af000 tx=0x7f0fc4004740 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:44.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.215+0000 7f0fd4c00640 1 -- 172.21.15.86:0/1070436155 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0fcc05c320 con 0x7f0fd806c670 2024-04-25T14:07:44.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.407+0000 7f0fdf600640 1 -- 172.21.15.86:0/1070436155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0fd8069d00 con 0x7f0fd806c670 2024-04-25T14:07:44.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.407+0000 7f0fd4c00640 1 -- 172.21.15.86:0/1070436155 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5209 (secure 0 0 0) 0x7f0fcc05bc20 con 0x7f0fd806c670 2024-04-25T14:07:44.409 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:07:44.410 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":57,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1210051860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1210051860},{"type":"v1","addr":"172.21.15.155:6833","nonce":1210051860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2868300438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2868300438},{"type":"v1","addr":"172.21.15.155:6835","nonce":2868300438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:06:44.910717+0000","modified":"2024-04-25T14:07:19.703362+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6603,"mds_1":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"c","rank":1,"incarnation":55,"state":"up:active","state_seq":9,"addr":"172.21.15.86:6835/2682115891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2682115891},{"type":"v1","addr":"172.21.15.86:6835","nonce":2682115891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6603":{"gid":6603,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/1088108173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1088108173},{"type":"v1","addr":"172.21.15.86:6837","nonce":1088108173}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6603,"qdb_cluster":[6603,6600]},"id":5}]} 2024-04-25T14:07:44.410 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 57 2024-04-25T14:07:44.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.411+0000 7f0fdf600640 1 -- 172.21.15.86:0/1070436155 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f0fb806f5a0 msgr2=0x7f0fb8071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:44.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.411+0000 7f0fdf600640 1 --2- 172.21.15.86:0/1070436155 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f0fb806f5a0 0x7f0fb8071ab0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40af000 tx=0x7f0fc4004740 comp rx=0 tx=0).stop 2024-04-25T14:07:44.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.411+0000 7f0fdf600640 1 -- 172.21.15.86:0/1070436155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fd806c670 msgr2=0x7f0fd80ca7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:44.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.411+0000 7f0fdf600640 1 --2- 172.21.15.86:0/1070436155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fd806c670 0x7f0fd80ca7f0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc00e840 tx=0x7f0fcc00ec40 comp rx=0 tx=0).stop 2024-04-25T14:07:44.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.411+0000 7f0fdf600640 1 -- 172.21.15.86:0/1070436155 shutdown_connections 2024-04-25T14:07:44.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.411+0000 7f0fdf600640 1 --2- 172.21.15.86:0/1070436155 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f0fb806f5a0 0x7f0fb8071ab0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:44.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.411+0000 7f0fdf600640 1 --2- 172.21.15.86:0/1070436155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fd80d16e0 0x7f0fd81fa250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:44.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.411+0000 7f0fdf600640 1 --2- 172.21.15.86:0/1070436155 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fd80cfca0 0x7f0fd81f9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:44.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.411+0000 7f0fdf600640 1 --2- 172.21.15.86:0/1070436155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fd806c670 0x7f0fd80ca7f0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:44.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.411+0000 7f0fdf600640 1 -- 172.21.15.86:0/1070436155 >> 172.21.15.86:0/1070436155 conn(0x7f0fd80b4650 msgr2=0x7f0fd8158520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:44.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.411+0000 7f0fdf600640 1 -- 172.21.15.86:0/1070436155 shutdown_connections 2024-04-25T14:07:44.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:44.411+0000 7f0fdf600640 1 -- 172.21.15.86:0/1070436155 wait complete. 2024-04-25T14:07:45.431 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:07:45.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.591+0000 7f66cd400640 1 Processor -- start 2024-04-25T14:07:45.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.591+0000 7f66cd400640 1 -- start start 2024-04-25T14:07:45.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.595+0000 7f66cd400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f66c80b3530 0x7f66c80d2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:45.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.595+0000 7f66cd400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66c80d27d0 0x7f66c80b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:45.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.595+0000 7f66cd400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66c80ccd60 con 0x7f66c80cc950 2024-04-25T14:07:45.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.595+0000 7f66cd400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f66c80b20d0 con 0x7f66c80b3530 2024-04-25T14:07:45.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.595+0000 7f66cd400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f66c80d2be0 con 0x7f66c80d27d0 2024-04-25T14:07:45.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.595+0000 7f66bfe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f66c80b3530 0x7f66c80d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:45.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.595+0000 7f66cca00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66c80d27d0 0x7f66c80b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:45.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.595+0000 7f66c7e00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f66c80cc950 0x7f66c806eb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:57696/0 (socket says 172.21.15.86:57696) 2024-04-25T14:07:45.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.595+0000 7f66c7e00640 1 -- 172.21.15.86:0/724916827 learned_addr learned my addr 172.21.15.86:0/724916827 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:07:45.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.595+0000 7f66cca00640 1 -- 172.21.15.86:0/724916827 >> v2:172.21.15.155:3300/0 conn(0x7f66c80b3530 msgr2=0x7f66c80d2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:45.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.595+0000 7f66cca00640 1 --2- 172.21.15.86:0/724916827 >> v2:172.21.15.155:3300/0 conn(0x7f66c80b3530 0x7f66c80d2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:45.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.595+0000 7f66cca00640 1 -- 172.21.15.86:0/724916827 >> v1:172.21.15.155:6789/0 conn(0x7f66c80cc950 legacy=0x7f66c806eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:45.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.595+0000 7f66cca00640 1 -- 172.21.15.86:0/724916827 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c80b17a0 con 0x7f66c80d27d0 2024-04-25T14:07:45.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.595+0000 7f66bfe00640 1 --2- 172.21.15.86:0/724916827 >> v2:172.21.15.155:3300/0 conn(0x7f66c80b3530 0x7f66c80d2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:07:45.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.599+0000 7f66cca00640 1 --2- 172.21.15.86:0/724916827 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66c80d27d0 0x7f66c80b2560 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f66c0009d90 tx=0x7f66c001c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4394ea10bcd6386 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:45.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.599+0000 7f66c7400640 1 -- 172.21.15.86:0/724916827 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c00b2070 con 0x7f66c80d27d0 2024-04-25T14:07:45.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.599+0000 7f66c7400640 1 -- 172.21.15.86:0/724916827 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66c00a5910 con 0x7f66c80d27d0 2024-04-25T14:07:45.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.599+0000 7f66c7400640 1 -- 172.21.15.86:0/724916827 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c00a5ce0 con 0x7f66c80d27d0 2024-04-25T14:07:45.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.599+0000 7f66cd400640 1 -- 172.21.15.86:0/724916827 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66c80d27d0 msgr2=0x7f66c80b2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:45.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.599+0000 7f66cd400640 1 --2- 172.21.15.86:0/724916827 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66c80d27d0 0x7f66c80b2560 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f66c0009d90 tx=0x7f66c001c850 comp rx=0 tx=0).stop 2024-04-25T14:07:45.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.599+0000 7f66cd400640 1 -- 172.21.15.86:0/724916827 shutdown_connections 2024-04-25T14:07:45.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.599+0000 7f66cd400640 1 --2- 172.21.15.86:0/724916827 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66c80d27d0 0x7f66c80b2560 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:45.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.599+0000 7f66cd400640 1 --2- 172.21.15.86:0/724916827 >> v2:172.21.15.155:3300/0 conn(0x7f66c80b3530 0x7f66c80d2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:45.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.599+0000 7f66cd400640 1 -- 172.21.15.86:0/724916827 >> 172.21.15.86:0/724916827 conn(0x7f66c809ba30 msgr2=0x7f66c80b71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:45.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.599+0000 7f66cd400640 1 -- 172.21.15.86:0/724916827 shutdown_connections 2024-04-25T14:07:45.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.599+0000 7f66cd400640 1 -- 172.21.15.86:0/724916827 wait complete. 2024-04-25T14:07:45.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.603+0000 7f66cd400640 1 Processor -- start 2024-04-25T14:07:45.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.603+0000 7f66cd400640 1 -- start start 2024-04-25T14:07:45.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.603+0000 7f66cd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66c80b3530 0x7f66c814e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:45.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.603+0000 7f66cd400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66c80cc950 0x7f66c815e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:45.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.603+0000 7f66cd400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66c80d27d0 0x7f66c815eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:45.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.603+0000 7f66cd400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f66c816ca20 con 0x7f66c80b3530 2024-04-25T14:07:45.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.603+0000 7f66cd400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f66c816cba0 con 0x7f66c80cc950 2024-04-25T14:07:45.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.603+0000 7f66cd400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f66c816cd20 con 0x7f66c80d27d0 2024-04-25T14:07:45.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.603+0000 7f66c7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66c80b3530 0x7f66c814e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:45.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.603+0000 7f66c7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66c80b3530 0x7f66c814e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56580/0 (socket says 172.21.15.86:56580) 2024-04-25T14:07:45.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.603+0000 7f66c7e00640 1 -- 172.21.15.86:0/167035536 learned_addr learned my addr 172.21.15.86:0/167035536 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:45.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.603+0000 7f66bfe00640 1 --2- 172.21.15.86:0/167035536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66c80cc950 0x7f66c815e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:45.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.603+0000 7f66cca00640 1 --2- 172.21.15.86:0/167035536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66c80d27d0 0x7f66c815eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:45.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.603+0000 7f66c7e00640 1 -- 172.21.15.86:0/167035536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66c80d27d0 msgr2=0x7f66c815eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:45.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.603+0000 7f66c7e00640 1 --2- 172.21.15.86:0/167035536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66c80d27d0 0x7f66c815eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:45.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.603+0000 7f66c7e00640 1 -- 172.21.15.86:0/167035536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66c80cc950 msgr2=0x7f66c815e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:45.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.603+0000 7f66c7e00640 1 --2- 172.21.15.86:0/167035536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66c80cc950 0x7f66c815e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:45.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.603+0000 7f66c7e00640 1 -- 172.21.15.86:0/167035536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c814ecf0 con 0x7f66c80b3530 2024-04-25T14:07:45.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.603+0000 7f66cca00640 1 --2- 172.21.15.86:0/167035536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66c80d27d0 0x7f66c815eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:45.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.603+0000 7f66c7e00640 1 --2- 172.21.15.86:0/167035536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66c80b3530 0x7f66c814e270 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f66b800c3d0 tx=0x7f66b800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:45.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.603+0000 7f66c4c00640 1 -- 172.21.15.86:0/167035536 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b8017070 con 0x7f66c80b3530 2024-04-25T14:07:45.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.603+0000 7f66c4c00640 1 -- 172.21.15.86:0/167035536 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b8005cf0 con 0x7f66c80b3530 2024-04-25T14:07:45.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.603+0000 7f66cd400640 1 -- 172.21.15.86:0/167035536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66c814efd0 con 0x7f66c80b3530 2024-04-25T14:07:45.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.603+0000 7f66c4c00640 1 -- 172.21.15.86:0/167035536 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b80072f0 con 0x7f66c80b3530 2024-04-25T14:07:45.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.603+0000 7f66cd400640 1 -- 172.21.15.86:0/167035536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66c814f590 con 0x7f66c80b3530 2024-04-25T14:07:45.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.607+0000 7f66cd400640 1 -- 172.21.15.86:0/167035536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66c8076b80 con 0x7f66c80b3530 2024-04-25T14:07:45.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.607+0000 7f66c4c00640 1 -- 172.21.15.86:0/167035536 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7f66b801a070 con 0x7f66c80b3530 2024-04-25T14:07:45.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.607+0000 7f66c4c00640 1 --2- 172.21.15.86:0/167035536 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f66a406f5a0 0x7f66a4071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:45.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.607+0000 7f66c4c00640 1 -- 172.21.15.86:0/167035536 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f66b808ea40 con 0x7f66c80b3530 2024-04-25T14:07:45.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.607+0000 7f66bfe00640 1 --2- 172.21.15.86:0/167035536 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f66a406f5a0 0x7f66a4071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:45.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.607+0000 7f66bfe00640 1 --2- 172.21.15.86:0/167035536 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f66a406f5a0 0x7f66a4071ab0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f66c806e970 tx=0x7f66b0009190 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:45.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.611+0000 7f66c4c00640 1 -- 172.21.15.86:0/167035536 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f66c80ccd60 con 0x7f66c80b3530 2024-04-25T14:07:45.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.791+0000 7f66cd400640 1 -- 172.21.15.86:0/167035536 --> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f66c80712d0 con 0x7f66a406f5a0 2024-04-25T14:07:45.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.791+0000 7f66c4c00640 1 -- 172.21.15.86:0/167035536 <== mgr.6801 v2:172.21.15.86:6832/203029814 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3055 (secure 0 0 0) 0x7f66c80712d0 con 0x7f66a406f5a0 2024-04-25T14:07:45.793 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {"cephfs": {"client.5945": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:172.21.15.86"}, "client.5951": {"hostname": "smithi155", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:192.168.0.1"}, "client.6834": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}, "client.6837": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}}}, "global_metrics": {"cephfs": {"client.5945": [[415, 22], [0, 0], [0, 396500909], [0, 71840640], [34, 0], [0, 35], [35, 35], [0, 35], [0, 0], [32, 33554432], [0, 0], [0, 0], [0, 12390659], [275776813322337, 32], [0, 1122510], [133594044029089, 64]], "client.5951": [[48, 2], [0, 0], [0, 0], [0, 20679725], [6, 0], [0, 3], [3, 3], [0, 3], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 2297747], [50231033486578, 9]], "client.6834": [[2, 0], [0, 0], [0, 0], [0, 510044], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 510043], [0, 1]], "client.6837": [[2, 0], [0, 0], [0, 0], [0, 761751], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 761751], [0, 1]]}}, "metrics": {"delayed_ranks": [], "mds.0": {"client.5945": [], "client.5951": [], "client.6834": [], "client.6837": []}, "mds.1": {"client.5945": [], "client.5951": []}}} 2024-04-25T14:07:45.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.795+0000 7f66cd400640 1 -- 172.21.15.86:0/167035536 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f66a406f5a0 msgr2=0x7f66a4071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:45.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.795+0000 7f66cd400640 1 --2- 172.21.15.86:0/167035536 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f66a406f5a0 0x7f66a4071ab0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f66c806e970 tx=0x7f66b0009190 comp rx=0 tx=0).stop 2024-04-25T14:07:45.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.795+0000 7f66cd400640 1 -- 172.21.15.86:0/167035536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66c80b3530 msgr2=0x7f66c814e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:45.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.795+0000 7f66cd400640 1 --2- 172.21.15.86:0/167035536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66c80b3530 0x7f66c814e270 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f66b800c3d0 tx=0x7f66b800c890 comp rx=0 tx=0).stop 2024-04-25T14:07:45.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.795+0000 7f66cd400640 1 -- 172.21.15.86:0/167035536 shutdown_connections 2024-04-25T14:07:45.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.795+0000 7f66cd400640 1 --2- 172.21.15.86:0/167035536 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f66a406f5a0 0x7f66a4071ab0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:45.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.795+0000 7f66cd400640 1 --2- 172.21.15.86:0/167035536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66c80d27d0 0x7f66c815eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:45.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.795+0000 7f66cd400640 1 --2- 172.21.15.86:0/167035536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66c80cc950 0x7f66c815e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:45.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.795+0000 7f66cd400640 1 --2- 172.21.15.86:0/167035536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66c80b3530 0x7f66c814e270 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:45.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.795+0000 7f66cd400640 1 -- 172.21.15.86:0/167035536 >> 172.21.15.86:0/167035536 conn(0x7f66c809ba30 msgr2=0x7f66c80b3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:45.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.795+0000 7f66cd400640 1 -- 172.21.15.86:0/167035536 shutdown_connections 2024-04-25T14:07:45.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.795+0000 7f66cd400640 1 -- 172.21.15.86:0/167035536 wait complete. 2024-04-25T14:07:45.815 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:07:45.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.959+0000 7f5db1000640 1 Processor -- start 2024-04-25T14:07:45.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.959+0000 7f5db1000640 1 -- start start 2024-04-25T14:07:45.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.963+0000 7f5db1000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5dac15c2a0 0x7f5dac0ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:45.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.963+0000 7f5db1000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dac0b0a60 con 0x7f5dac15feb0 2024-04-25T14:07:45.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.963+0000 7f5db1000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dac06ef90 con 0x7f5dac06fd50 2024-04-25T14:07:45.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.963+0000 7f5db1000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5dac06f270 con 0x7f5dac15c2a0 2024-04-25T14:07:45.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.963+0000 7f5daa800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5dac15feb0 0x7f5dac1602c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42924/0 (socket says 172.21.15.86:42924) 2024-04-25T14:07:45.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.963+0000 7f5daa800640 1 -- 172.21.15.86:0/1770001533 learned_addr learned my addr 172.21.15.86:0/1770001533 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:07:45.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.963+0000 7f5da9e00640 1 --2- 172.21.15.86:0/1770001533 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5dac15c2a0 0x7f5dac0ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:45.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.963+0000 7f5da9400640 1 -- 172.21.15.86:0/1770001533 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 641740911 0 0) 0x7f5dac0b0a60 con 0x7f5dac15feb0 2024-04-25T14:07:45.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.963+0000 7f5da9400640 1 -- 172.21.15.86:0/1770001533 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d90003450 con 0x7f5dac15feb0 2024-04-25T14:07:45.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.963+0000 7f5da9400640 1 -- 172.21.15.86:0/1770001533 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3581322928 0 0) 0x7f5dac06ef90 con 0x7f5dac06fd50 2024-04-25T14:07:45.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.963+0000 7f5da9400640 1 -- 172.21.15.86:0/1770001533 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5dac0b0a60 con 0x7f5dac06fd50 2024-04-25T14:07:45.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.963+0000 7f5da9e00640 1 -- 172.21.15.86:0/1770001533 >> v1:172.21.15.155:6789/0 conn(0x7f5dac06fd50 legacy=0x7f5dac070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:45.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.963+0000 7f5da9e00640 1 -- 172.21.15.86:0/1770001533 >> v1:172.21.15.86:6789/0 conn(0x7f5dac15feb0 legacy=0x7f5dac1602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:45.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.963+0000 7f5da9e00640 1 -- 172.21.15.86:0/1770001533 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dac06f550 con 0x7f5dac15c2a0 2024-04-25T14:07:45.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.967+0000 7f5da9e00640 1 --2- 172.21.15.86:0/1770001533 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5dac15c2a0 0x7f5dac0ae130 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0097f0 tx=0x7f5d9c01c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1020f64987b92e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:45.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.967+0000 7f5da9400640 1 -- 172.21.15.86:0/1770001533 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c0b0070 con 0x7f5dac15c2a0 2024-04-25T14:07:45.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.967+0000 7f5da9400640 1 -- 172.21.15.86:0/1770001533 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d9c0ab600 con 0x7f5dac15c2a0 2024-04-25T14:07:45.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.967+0000 7f5da9400640 1 -- 172.21.15.86:0/1770001533 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c0ab9d0 con 0x7f5dac15c2a0 2024-04-25T14:07:45.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.967+0000 7f5db1000640 1 -- 172.21.15.86:0/1770001533 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5dac15c2a0 msgr2=0x7f5dac0ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:45.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.967+0000 7f5db1000640 1 --2- 172.21.15.86:0/1770001533 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5dac15c2a0 0x7f5dac0ae130 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0097f0 tx=0x7f5d9c01c420 comp rx=0 tx=0).stop 2024-04-25T14:07:45.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.967+0000 7f5db1000640 1 -- 172.21.15.86:0/1770001533 shutdown_connections 2024-04-25T14:07:45.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.967+0000 7f5db1000640 1 --2- 172.21.15.86:0/1770001533 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5dac15c2a0 0x7f5dac0ae130 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:45.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.967+0000 7f5db1000640 1 -- 172.21.15.86:0/1770001533 >> 172.21.15.86:0/1770001533 conn(0x7f5dac0b4650 msgr2=0x7f5dac0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:45.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.967+0000 7f5db1000640 1 -- 172.21.15.86:0/1770001533 shutdown_connections 2024-04-25T14:07:45.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.967+0000 7f5db1000640 1 -- 172.21.15.86:0/1770001533 wait complete. 2024-04-25T14:07:45.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.967+0000 7f5db1000640 1 Processor -- start 2024-04-25T14:07:45.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.967+0000 7f5db1000640 1 -- start start 2024-04-25T14:07:45.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.971+0000 7f5db1000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5dac06fd50 0x7f5dac09a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:45.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.971+0000 7f5db1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dac15c2a0 0x7f5dac15b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:45.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.971+0000 7f5db1000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5dac15feb0 0x7f5dac15ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:45.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.971+0000 7f5db1000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac0ae7f0 con 0x7f5dac15c2a0 2024-04-25T14:07:45.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.971+0000 7f5db1000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac0ae670 con 0x7f5dac15feb0 2024-04-25T14:07:45.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.971+0000 7f5db1000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5dac16a200 con 0x7f5dac06fd50 2024-04-25T14:07:45.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.971+0000 7f5da1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dac15c2a0 0x7f5dac15b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:45.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.971+0000 7f5da1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dac15c2a0 0x7f5dac15b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56586/0 (socket says 172.21.15.86:56586) 2024-04-25T14:07:45.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.971+0000 7f5da1400640 1 -- 172.21.15.86:0/154864748 learned_addr learned my addr 172.21.15.86:0/154864748 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:07:45.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.971+0000 7f5da9e00640 1 --2- 172.21.15.86:0/154864748 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5dac06fd50 0x7f5dac09a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:45.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.971+0000 7f5daa800640 1 --2- 172.21.15.86:0/154864748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5dac15feb0 0x7f5dac15ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:45.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.971+0000 7f5da1400640 1 -- 172.21.15.86:0/154864748 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5dac06fd50 msgr2=0x7f5dac09a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:45.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.971+0000 7f5da1400640 1 --2- 172.21.15.86:0/154864748 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5dac06fd50 0x7f5dac09a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:45.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.971+0000 7f5da1400640 1 -- 172.21.15.86:0/154864748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5dac15feb0 msgr2=0x7f5dac15ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:45.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.971+0000 7f5da1400640 1 --2- 172.21.15.86:0/154864748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5dac15feb0 0x7f5dac15ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:45.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.971+0000 7f5da1400640 1 -- 172.21.15.86:0/154864748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dac15dcd0 con 0x7f5dac15c2a0 2024-04-25T14:07:45.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.971+0000 7f5daa800640 1 --2- 172.21.15.86:0/154864748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5dac15feb0 0x7f5dac15ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:45.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.971+0000 7f5da1400640 1 --2- 172.21.15.86:0/154864748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dac15c2a0 0x7f5dac15b050 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400c460 tx=0x7f5d9400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:45.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.971+0000 7f5da2a00640 1 -- 172.21.15.86:0/154864748 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d940052d0 con 0x7f5dac15c2a0 2024-04-25T14:07:45.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.971+0000 7f5da2a00640 1 -- 172.21.15.86:0/154864748 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d94005490 con 0x7f5dac15c2a0 2024-04-25T14:07:45.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.971+0000 7f5da2a00640 1 -- 172.21.15.86:0/154864748 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94012850 con 0x7f5dac15c2a0 2024-04-25T14:07:45.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.971+0000 7f5db1000640 1 -- 172.21.15.86:0/154864748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dac15dfb0 con 0x7f5dac15c2a0 2024-04-25T14:07:45.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.971+0000 7f5db1000640 1 -- 172.21.15.86:0/154864748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dac15e570 con 0x7f5dac15c2a0 2024-04-25T14:07:45.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.971+0000 7f5db1000640 1 -- 172.21.15.86:0/154864748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dac069d00 con 0x7f5dac15c2a0 2024-04-25T14:07:45.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.975+0000 7f5da2a00640 1 -- 172.21.15.86:0/154864748 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7f5d940129f0 con 0x7f5dac15c2a0 2024-04-25T14:07:45.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.975+0000 7f5da2a00640 1 --2- 172.21.15.86:0/154864748 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f5d8806f5a0 0x7f5d88071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:45.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.975+0000 7f5da2a00640 1 -- 172.21.15.86:0/154864748 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f5d9408fed0 con 0x7f5dac15c2a0 2024-04-25T14:07:45.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.975+0000 7f5da9e00640 1 --2- 172.21.15.86:0/154864748 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f5d8806f5a0 0x7f5d88071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:45.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.975+0000 7f5da9e00640 1 --2- 172.21.15.86:0/154864748 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f5d8806f5a0 0x7f5d88071ab0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0ad000 tx=0x7f5d9c008040 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:45.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:45.979+0000 7f5da2a00640 1 -- 172.21.15.86:0/154864748 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5d9405c950 con 0x7f5dac15c2a0 2024-04-25T14:07:46.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:46.171+0000 7f5db1000640 1 -- 172.21.15.86:0/154864748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5dac076bf0 con 0x7f5dac15c2a0 2024-04-25T14:07:46.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:46.171+0000 7f5da2a00640 1 -- 172.21.15.86:0/154864748 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5209 (secure 0 0 0) 0x7f5d9405c250 con 0x7f5dac15c2a0 2024-04-25T14:07:46.172 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:07:46.172 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":57,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1210051860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1210051860},{"type":"v1","addr":"172.21.15.155:6833","nonce":1210051860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2868300438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2868300438},{"type":"v1","addr":"172.21.15.155:6835","nonce":2868300438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:06:44.910717+0000","modified":"2024-04-25T14:07:19.703362+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6603,"mds_1":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"c","rank":1,"incarnation":55,"state":"up:active","state_seq":9,"addr":"172.21.15.86:6835/2682115891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2682115891},{"type":"v1","addr":"172.21.15.86:6835","nonce":2682115891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6603":{"gid":6603,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/1088108173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1088108173},{"type":"v1","addr":"172.21.15.86:6837","nonce":1088108173}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6603,"qdb_cluster":[6603,6600]},"id":5}]} 2024-04-25T14:07:46.172 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 57 2024-04-25T14:07:46.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:46.171+0000 7f5db1000640 1 -- 172.21.15.86:0/154864748 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f5d8806f5a0 msgr2=0x7f5d88071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:46.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:46.171+0000 7f5db1000640 1 --2- 172.21.15.86:0/154864748 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f5d8806f5a0 0x7f5d88071ab0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0ad000 tx=0x7f5d9c008040 comp rx=0 tx=0).stop 2024-04-25T14:07:46.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:46.171+0000 7f5db1000640 1 -- 172.21.15.86:0/154864748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dac15c2a0 msgr2=0x7f5dac15b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:46.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:46.171+0000 7f5db1000640 1 --2- 172.21.15.86:0/154864748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dac15c2a0 0x7f5dac15b050 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400c460 tx=0x7f5d9400c920 comp rx=0 tx=0).stop 2024-04-25T14:07:46.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:46.171+0000 7f5db1000640 1 -- 172.21.15.86:0/154864748 shutdown_connections 2024-04-25T14:07:46.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:46.171+0000 7f5db1000640 1 --2- 172.21.15.86:0/154864748 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f5d8806f5a0 0x7f5d88071ab0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:46.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:46.171+0000 7f5db1000640 1 --2- 172.21.15.86:0/154864748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5dac15feb0 0x7f5dac15ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:46.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:46.171+0000 7f5db1000640 1 --2- 172.21.15.86:0/154864748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dac15c2a0 0x7f5dac15b050 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:46.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:46.171+0000 7f5db1000640 1 --2- 172.21.15.86:0/154864748 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5dac06fd50 0x7f5dac09a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:46.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:46.171+0000 7f5db1000640 1 -- 172.21.15.86:0/154864748 >> 172.21.15.86:0/154864748 conn(0x7f5dac0b4650 msgr2=0x7f5dac1673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:46.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:46.171+0000 7f5db1000640 1 -- 172.21.15.86:0/154864748 shutdown_connections 2024-04-25T14:07:46.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:07:46.171+0000 7f5db1000640 1 -- 172.21.15.86:0/154864748 wait complete. 2024-04-25T14:07:46.193 DEBUG:tasks.cephfs.test_mds_metrics:metrics={'version': 2, 'global_counters': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'counters': [], 'client_metadata': {'cephfs': {'client.5945': {'hostname': 'smithi086', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'kernel_version': '6.9.0-rc3-g806ec36b2112', 'IP': 'v1:172.21.15.86'}, 'client.5951': {'hostname': 'smithi155', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'kernel_version': '6.9.0-rc3-g806ec36b2112', 'IP': 'v1:192.168.0.1'}, 'client.6834': {'hostname': 'smithi086', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'IP': '172.21.15.86'}, 'client.6837': {'hostname': 'smithi086', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'IP': '172.21.15.86'}}}, 'global_metrics': {'cephfs': {'client.5945': [[415, 22], [0, 0], [0, 396500909], [0, 71840640], [34, 0], [0, 35], [35, 35], [0, 35], [0, 0], [32, 33554432], [0, 0], [0, 0], [0, 12390659], [275776813322337, 32], [0, 1122510], [133594044029089, 64]], 'client.5951': [[48, 2], [0, 0], [0, 0], [0, 20679725], [6, 0], [0, 3], [3, 3], [0, 3], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 2297747], [50231033486578, 9]], 'client.6834': [[2, 0], [0, 0], [0, 0], [0, 510044], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 510043], [0, 1]], 'client.6837': [[2, 0], [0, 0], [0, 0], [0, 761751], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 761751], [0, 1]]}}, 'metrics': {'delayed_ranks': [], 'mds.0': {'client.5945': [], 'client.5951': [], 'client.6834': [], 'client.6837': []}, 'mds.1': {'client.5945': [], 'client.5951': []}}} 2024-04-25T14:07:46.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:46.194 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls) 2024-04-25T14:07:46.204 INFO:teuthology.orchestra.run.smithi086.stdout:test_mds_metrics_0 2024-04-25T14:07:46.204 INFO:teuthology.orchestra.run.smithi086.stdout:test_mds_metrics_1 2024-04-25T14:07:46.205 INFO:tasks.cephfs.test_mds_metrics:cleaning directory test_mds_metrics_0 2024-04-25T14:07:46.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:46.205 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -rf test_mds_metrics_0) 2024-04-25T14:07:46.301 INFO:tasks.cephfs.test_mds_metrics:cleaning directory test_mds_metrics_1 2024-04-25T14:07:46.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:46.302 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -rf test_mds_metrics_1) 2024-04-25T14:07:49.342 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:07:49.346 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:08:19.400 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:08:19.404 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:08:24.771 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable stats 2024-04-25T14:08:24.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.918+0000 7ff15f200640 1 Processor -- start 2024-04-25T14:08:24.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.918+0000 7ff15f200640 1 -- start start 2024-04-25T14:08:24.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff15f200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff15815c2a0 0x7ff1580ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:24.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff15f200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff15815feb0 0x7ff1581602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:24.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff15f200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1580b0a60 con 0x7ff15806fd50 2024-04-25T14:08:24.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff15f200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff15806ef90 con 0x7ff15815feb0 2024-04-25T14:08:24.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff15f200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff15806f270 con 0x7ff15815c2a0 2024-04-25T14:08:24.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff14f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff15806fd50 0x7ff158070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54126/0 (socket says 172.21.15.86:54126) 2024-04-25T14:08:24.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff14f400640 1 -- 172.21.15.86:0/677364999 learned_addr learned my addr 172.21.15.86:0/677364999 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:24.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff15d200640 1 --2- 172.21.15.86:0/677364999 >> v2:172.21.15.155:3300/0 conn(0x7ff15815feb0 0x7ff1581602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:24.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff157e00640 1 --2- 172.21.15.86:0/677364999 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff15815c2a0 0x7ff1580ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:24.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff157400640 1 -- 172.21.15.86:0/677364999 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3508337018 0 0) 0x7ff1580b0a60 con 0x7ff15806fd50 2024-04-25T14:08:24.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff157400640 1 -- 172.21.15.86:0/677364999 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff13c003530 con 0x7ff15806fd50 2024-04-25T14:08:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff157400640 1 -- 172.21.15.86:0/677364999 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2043527962 0 0) 0x7ff13c003530 con 0x7ff15806fd50 2024-04-25T14:08:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff157400640 1 -- 172.21.15.86:0/677364999 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff15815c2a0 msgr2=0x7ff1580ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff157400640 1 --2- 172.21.15.86:0/677364999 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff15815c2a0 0x7ff1580ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff157400640 1 -- 172.21.15.86:0/677364999 >> v2:172.21.15.155:3300/0 conn(0x7ff15815feb0 msgr2=0x7ff1581602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff157400640 1 --2- 172.21.15.86:0/677364999 >> v2:172.21.15.155:3300/0 conn(0x7ff15815feb0 0x7ff1581602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff157400640 1 -- 172.21.15.86:0/677364999 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff15806f550 con 0x7ff15806fd50 2024-04-25T14:08:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff157400640 1 -- 172.21.15.86:0/677364999 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7ff140001680 con 0x7ff15806fd50 2024-04-25T14:08:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff157400640 1 -- 172.21.15.86:0/677364999 >> v1:172.21.15.86:6789/0 conn(0x7ff15806fd50 legacy=0x7ff158070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff157400640 1 --2- 172.21.15.86:0/677364999 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff15815c2a0 0x7ff13c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff157400640 1 --2- 172.21.15.86:0/677364999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff13c003fb0 0x7ff13c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff157400640 1 --2- 172.21.15.86:0/677364999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff13c004d50 0x7ff13c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff157400640 1 -- 172.21.15.86:0/677364999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff15806ef90 con 0x7ff13c003fb0 2024-04-25T14:08:24.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff157400640 1 -- 172.21.15.86:0/677364999 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff15806f270 con 0x7ff13c004d50 2024-04-25T14:08:24.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff157400640 1 -- 172.21.15.86:0/677364999 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff13c00bb30 con 0x7ff15815c2a0 2024-04-25T14:08:24.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff14f400640 1 --2- 172.21.15.86:0/677364999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff13c003fb0 0x7ff13c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:24.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff15d200640 1 --2- 172.21.15.86:0/677364999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff13c004d50 0x7ff13c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:24.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff157e00640 1 --2- 172.21.15.86:0/677364999 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff15815c2a0 0x7ff13c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:24.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff14f400640 1 -- 172.21.15.86:0/677364999 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff15815c2a0 msgr2=0x7ff13c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:24.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff14f400640 1 --2- 172.21.15.86:0/677364999 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff15815c2a0 0x7ff13c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:24.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff14f400640 1 -- 172.21.15.86:0/677364999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff13c004d50 msgr2=0x7ff13c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:24.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff14f400640 1 --2- 172.21.15.86:0/677364999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff13c004d50 0x7ff13c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:24.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.922+0000 7ff14f400640 1 -- 172.21.15.86:0/677364999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13c003530 con 0x7ff13c003fb0 2024-04-25T14:08:24.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.926+0000 7ff14f400640 1 --2- 172.21.15.86:0/677364999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff13c003fb0 0x7ff13c004810 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7ff14000c2c0 tx=0x7ff14001d770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38e34fc2c3b0a36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:24.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.926+0000 7ff157400640 1 -- 172.21.15.86:0/677364999 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140007d60 con 0x7ff13c003fb0 2024-04-25T14:08:24.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.926+0000 7ff157400640 1 -- 172.21.15.86:0/677364999 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1400a9020 con 0x7ff13c003fb0 2024-04-25T14:08:24.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.926+0000 7ff157400640 1 -- 172.21.15.86:0/677364999 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1400a5b00 con 0x7ff13c003fb0 2024-04-25T14:08:24.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.926+0000 7ff15f200640 1 -- 172.21.15.86:0/677364999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff13c003fb0 msgr2=0x7ff13c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:24.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.926+0000 7ff15f200640 1 --2- 172.21.15.86:0/677364999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff13c003fb0 0x7ff13c004810 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7ff14000c2c0 tx=0x7ff14001d770 comp rx=0 tx=0).stop 2024-04-25T14:08:24.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.926+0000 7ff15d200640 1 -- 172.21.15.86:0/677364999 reap_dead start 2024-04-25T14:08:24.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.926+0000 7ff15f200640 1 -- 172.21.15.86:0/677364999 shutdown_connections 2024-04-25T14:08:24.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.926+0000 7ff15f200640 1 -- 172.21.15.86:0/677364999 >> 172.21.15.86:0/677364999 conn(0x7ff1580b4650 msgr2=0x7ff1580b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:24.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.926+0000 7ff15f200640 1 -- 172.21.15.86:0/677364999 shutdown_connections 2024-04-25T14:08:24.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.926+0000 7ff15f200640 1 -- 172.21.15.86:0/677364999 wait complete. 2024-04-25T14:08:24.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.930+0000 7ff15f200640 1 Processor -- start 2024-04-25T14:08:24.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.930+0000 7ff15f200640 1 -- start start 2024-04-25T14:08:24.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.930+0000 7ff15f200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff13c003fb0 0x7ff15815f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:24.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.930+0000 7ff15f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1580b0ec0 0x7ff1580c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:24.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.930+0000 7ff15f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1580ccc70 0x7ff1580cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:24.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.930+0000 7ff15f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff158160d90 con 0x7ff1580ccc70 2024-04-25T14:08:24.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.930+0000 7ff15f200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff158160c10 con 0x7ff1580b0ec0 2024-04-25T14:08:24.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.930+0000 7ff15f200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff1580ae130 con 0x7ff13c003fb0 2024-04-25T14:08:24.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.930+0000 7ff15d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1580ccc70 0x7ff1580cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:24.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.930+0000 7ff15d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1580ccc70 0x7ff1580cd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53536/0 (socket says 172.21.15.86:53536) 2024-04-25T14:08:24.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.930+0000 7ff15d200640 1 -- 172.21.15.86:0/3200103204 learned_addr learned my addr 172.21.15.86:0/3200103204 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:24.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.930+0000 7ff157e00640 1 --2- 172.21.15.86:0/3200103204 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff13c003fb0 0x7ff15815f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:24.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.930+0000 7ff14f400640 1 --2- 172.21.15.86:0/3200103204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1580b0ec0 0x7ff1580c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:24.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.930+0000 7ff15d200640 1 -- 172.21.15.86:0/3200103204 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff13c003fb0 msgr2=0x7ff15815f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:24.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.930+0000 7ff15d200640 1 --2- 172.21.15.86:0/3200103204 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff13c003fb0 0x7ff15815f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:24.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.930+0000 7ff15d200640 1 -- 172.21.15.86:0/3200103204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1580b0ec0 msgr2=0x7ff1580c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:24.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.930+0000 7ff15d200640 1 --2- 172.21.15.86:0/3200103204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1580b0ec0 0x7ff1580c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:24.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.930+0000 7ff15d200640 1 -- 172.21.15.86:0/3200103204 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1581594c0 con 0x7ff1580ccc70 2024-04-25T14:08:24.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.930+0000 7ff157e00640 1 --2- 172.21.15.86:0/3200103204 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff13c003fb0 0x7ff15815f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:24.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.930+0000 7ff15d200640 1 --2- 172.21.15.86:0/3200103204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1580ccc70 0x7ff1580cd080 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7ff1580729c0 tx=0x7ff15000af60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:24.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.930+0000 7ff154c00640 1 -- 172.21.15.86:0/3200103204 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff150003db0 con 0x7ff1580ccc70 2024-04-25T14:08:24.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.930+0000 7ff154c00640 1 -- 172.21.15.86:0/3200103204 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff15001b070 con 0x7ff1580ccc70 2024-04-25T14:08:24.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.930+0000 7ff15f200640 1 -- 172.21.15.86:0/3200103204 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1580cd5c0 con 0x7ff1580ccc70 2024-04-25T14:08:24.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.930+0000 7ff154c00640 1 -- 172.21.15.86:0/3200103204 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff150007b00 con 0x7ff1580ccc70 2024-04-25T14:08:24.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.930+0000 7ff15f200640 1 -- 172.21.15.86:0/3200103204 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff158163b30 con 0x7ff1580ccc70 2024-04-25T14:08:24.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.934+0000 7ff15f200640 1 -- 172.21.15.86:0/3200103204 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1581635c0 con 0x7ff1580ccc70 2024-04-25T14:08:24.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.934+0000 7ff154c00640 1 -- 172.21.15.86:0/3200103204 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 93318+0+0 (secure 0 0 0) 0x7ff150007ca0 con 0x7ff1580ccc70 2024-04-25T14:08:24.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.934+0000 7ff154c00640 1 --2- 172.21.15.86:0/3200103204 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7ff13406f570 0x7ff134071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:24.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.934+0000 7ff154c00640 1 -- 172.21.15.86:0/3200103204 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7ff1500902e0 con 0x7ff1580ccc70 2024-04-25T14:08:24.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.934+0000 7ff157e00640 1 --2- 172.21.15.86:0/3200103204 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7ff13406f570 0x7ff134071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:24.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.934+0000 7ff157e00640 1 --2- 172.21.15.86:0/3200103204 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7ff13406f570 0x7ff134071a80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff148009160 tx=0x7ff148008040 comp rx=0 tx=0).ready entity=mgr.6801 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:24.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:24.938+0000 7ff154c00640 1 -- 172.21.15.86:0/3200103204 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff1580cd5c0 con 0x7ff1580ccc70 2024-04-25T14:08:25.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.138+0000 7ff15f200640 1 -- 172.21.15.86:0/3200103204 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "stats"} v 0) -- 0x7ff158069d00 con 0x7ff1580ccc70 2024-04-25T14:08:25.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.186+0000 7ff154c00640 1 -- 172.21.15.86:0/3200103204 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "stats"}]=0 v23) ==== 85+0+0 (secure 0 0 0) 0x7ff1581594c0 con 0x7ff1580ccc70 2024-04-25T14:08:25.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.186+0000 7ff15f200640 1 -- 172.21.15.86:0/3200103204 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7ff13406f570 msgr2=0x7ff134071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:25.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.186+0000 7ff15f200640 1 --2- 172.21.15.86:0/3200103204 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7ff13406f570 0x7ff134071a80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff148009160 tx=0x7ff148008040 comp rx=0 tx=0).stop 2024-04-25T14:08:25.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.186+0000 7ff15f200640 1 -- 172.21.15.86:0/3200103204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1580ccc70 msgr2=0x7ff1580cd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:25.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.186+0000 7ff15f200640 1 --2- 172.21.15.86:0/3200103204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1580ccc70 0x7ff1580cd080 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7ff1580729c0 tx=0x7ff15000af60 comp rx=0 tx=0).stop 2024-04-25T14:08:25.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.186+0000 7ff15f200640 1 -- 172.21.15.86:0/3200103204 shutdown_connections 2024-04-25T14:08:25.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.186+0000 7ff15f200640 1 --2- 172.21.15.86:0/3200103204 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7ff13406f570 0x7ff134071a80 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff148009160 tx=0x7ff148008040 comp rx=0 tx=0).stop 2024-04-25T14:08:25.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.186+0000 7ff15f200640 1 --2- 172.21.15.86:0/3200103204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1580ccc70 0x7ff1580cd080 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:25.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.186+0000 7ff15f200640 1 --2- 172.21.15.86:0/3200103204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1580b0ec0 0x7ff1580c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:25.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.186+0000 7ff15f200640 1 --2- 172.21.15.86:0/3200103204 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff13c003fb0 0x7ff15815f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:25.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.186+0000 7ff15f200640 1 -- 172.21.15.86:0/3200103204 >> 172.21.15.86:0/3200103204 conn(0x7ff1580b4650 msgr2=0x7ff158070160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:25.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.190+0000 7ff15f200640 1 -- 172.21.15.86:0/3200103204 shutdown_connections 2024-04-25T14:08:25.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.190+0000 7ff15f200640 1 -- 172.21.15.86:0/3200103204 wait complete. 2024-04-25T14:08:25.213 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T14:08:25.214 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:08:25.329 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:08:25.441 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:08:25.440+0000 7f844abdb180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:08:25.448 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:25.446+0000 7f6d4ef21180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:08:25.452 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T14:08:25.453 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:08:25.453 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:08:25.453 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:08:25.462 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:08:25.463 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:08:25.515 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:08:25.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.706+0000 7f396da00640 1 Processor -- start 2024-04-25T14:08:25.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.706+0000 7f396da00640 1 -- start start 2024-04-25T14:08:25.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.710+0000 7f396da00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f39680cdd70 0x7f39680ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:25.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.710+0000 7f396da00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3968070e20 0x7f3968071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:25.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.710+0000 7f396da00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3968071770 con 0x7f396806e700 2024-04-25T14:08:25.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.710+0000 7f396da00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f396806d940 con 0x7f39680cdd70 2024-04-25T14:08:25.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.710+0000 7f396da00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f396806dc20 con 0x7f3968070e20 2024-04-25T14:08:25.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.710+0000 7f3966a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f39680cdd70 0x7f39680ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:25.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.710+0000 7f3966a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f39680cdd70 0x7f39680ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53554/0 (socket says 172.21.15.86:53554) 2024-04-25T14:08:25.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.710+0000 7f3966a00640 1 -- 172.21.15.86:0/1076944612 learned_addr learned my addr 172.21.15.86:0/1076944612 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:25.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.710+0000 7f395fe00640 1 --2- 172.21.15.86:0/1076944612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3968070e20 0x7f3968071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:25.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.710+0000 7f3966a00640 1 -- 172.21.15.86:0/1076944612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3968070e20 msgr2=0x7f3968071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:25.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.710+0000 7f3966a00640 1 --2- 172.21.15.86:0/1076944612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3968070e20 0x7f3968071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:25.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.710+0000 7f3966a00640 1 -- 172.21.15.86:0/1076944612 >> v1:172.21.15.155:6789/0 conn(0x7f396806e700 legacy=0x7f396806eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:25.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.710+0000 7f3966a00640 1 -- 172.21.15.86:0/1076944612 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f396806df00 con 0x7f39680cdd70 2024-04-25T14:08:25.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.710+0000 7f395fe00640 1 --2- 172.21.15.86:0/1076944612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3968070e20 0x7f3968071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:25.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.714+0000 7f3966a00640 1 --2- 172.21.15.86:0/1076944612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39680cdd70 0x7f39680ce180 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f3958009870 tx=0x7f395801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e230fe3a77d50d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.714+0000 7f3966000640 1 -- 172.21.15.86:0/1076944612 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39580b0070 con 0x7f39680cdd70 2024-04-25T14:08:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.714+0000 7f3966000640 1 -- 172.21.15.86:0/1076944612 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39580ab600 con 0x7f39680cdd70 2024-04-25T14:08:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.714+0000 7f3966000640 1 -- 172.21.15.86:0/1076944612 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39580ab9d0 con 0x7f39680cdd70 2024-04-25T14:08:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.714+0000 7f396da00640 1 -- 172.21.15.86:0/1076944612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39680cdd70 msgr2=0x7f39680ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.714+0000 7f396da00640 1 --2- 172.21.15.86:0/1076944612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39680cdd70 0x7f39680ce180 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f3958009870 tx=0x7f395801c4d0 comp rx=0 tx=0).stop 2024-04-25T14:08:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.714+0000 7f396da00640 1 -- 172.21.15.86:0/1076944612 shutdown_connections 2024-04-25T14:08:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.714+0000 7f396da00640 1 --2- 172.21.15.86:0/1076944612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3968070e20 0x7f3968071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.714+0000 7f396da00640 1 --2- 172.21.15.86:0/1076944612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39680cdd70 0x7f39680ce180 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.714+0000 7f396da00640 1 -- 172.21.15.86:0/1076944612 >> 172.21.15.86:0/1076944612 conn(0x7f39680cffc0 msgr2=0x7f39680d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.714+0000 7f396da00640 1 -- 172.21.15.86:0/1076944612 shutdown_connections 2024-04-25T14:08:25.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.714+0000 7f396da00640 1 -- 172.21.15.86:0/1076944612 wait complete. 2024-04-25T14:08:25.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.714+0000 7f396da00640 1 Processor -- start 2024-04-25T14:08:25.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.718+0000 7f396da00640 1 -- start start 2024-04-25T14:08:25.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.718+0000 7f396da00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f396806e700 0x7f39680cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:25.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.718+0000 7f396da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3968070e20 0x7f39680b74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:25.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.718+0000 7f396da00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39680cdd70 0x7f3968159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:25.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.718+0000 7f396da00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f396816cc00 con 0x7f3968070e20 2024-04-25T14:08:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.718+0000 7f396da00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3968164380 con 0x7f396806e700 2024-04-25T14:08:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.718+0000 7f396da00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f396816cd80 con 0x7f39680cdd70 2024-04-25T14:08:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.718+0000 7f395fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3968070e20 0x7f39680b74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.718+0000 7f395fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3968070e20 0x7f39680b74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53558/0 (socket says 172.21.15.86:53558) 2024-04-25T14:08:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.718+0000 7f395fe00640 1 -- 172.21.15.86:0/482768292 learned_addr learned my addr 172.21.15.86:0/482768292 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.718+0000 7f3966a00640 1 --2- 172.21.15.86:0/482768292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f396806e700 0x7f39680cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.718+0000 7f3967400640 1 --2- 172.21.15.86:0/482768292 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39680cdd70 0x7f3968159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.718+0000 7f395fe00640 1 -- 172.21.15.86:0/482768292 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39680cdd70 msgr2=0x7f3968159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.718+0000 7f395fe00640 1 --2- 172.21.15.86:0/482768292 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39680cdd70 0x7f3968159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.718+0000 7f395fe00640 1 -- 172.21.15.86:0/482768292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f396806e700 msgr2=0x7f39680cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.718+0000 7f395fe00640 1 --2- 172.21.15.86:0/482768292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f396806e700 0x7f39680cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.718+0000 7f395fe00640 1 -- 172.21.15.86:0/482768292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39680d2130 con 0x7f3968070e20 2024-04-25T14:08:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.718+0000 7f395fe00640 1 --2- 172.21.15.86:0/482768292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3968070e20 0x7f39680b74b0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f395000d7d0 tx=0x7f395000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.718+0000 7f395ea00640 1 -- 172.21.15.86:0/482768292 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3950018020 con 0x7f3968070e20 2024-04-25T14:08:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.718+0000 7f395ea00640 1 -- 172.21.15.86:0/482768292 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39500059a0 con 0x7f3968070e20 2024-04-25T14:08:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.718+0000 7f396da00640 1 -- 172.21.15.86:0/482768292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39680b89c0 con 0x7f3968070e20 2024-04-25T14:08:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.718+0000 7f395ea00640 1 -- 172.21.15.86:0/482768292 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3950005d70 con 0x7f3968070e20 2024-04-25T14:08:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.718+0000 7f396da00640 1 -- 172.21.15.86:0/482768292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39680b8f10 con 0x7f3968070e20 2024-04-25T14:08:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.718+0000 7f396da00640 1 -- 172.21.15.86:0/482768292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39680720e0 con 0x7f3968070e20 2024-04-25T14:08:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.722+0000 7f395ea00640 1 -- 172.21.15.86:0/482768292 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 93309+0+0 (secure 0 0 0) 0x7f3950004030 con 0x7f3968070e20 2024-04-25T14:08:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.722+0000 7f395ea00640 1 --2- 172.21.15.86:0/482768292 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f394406f520 0x7f3944071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.722+0000 7f3966a00640 1 -- 172.21.15.86:0/482768292 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f394406f520 msgr2=0x7f3944071a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/203029814 2024-04-25T14:08:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.722+0000 7f395ea00640 1 -- 172.21.15.86:0/482768292 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f39500904e0 con 0x7f3968070e20 2024-04-25T14:08:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.722+0000 7f3966a00640 1 --2- 172.21.15.86:0/482768292 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f394406f520 0x7f3944071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:08:25.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.730+0000 7f395ea00640 1 -- 172.21.15.86:0/482768292 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3950022020 con 0x7f3968070e20 2024-04-25T14:08:25.858 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:08:25.860+0000 7f844abdb180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:08:25.887 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:25.886+0000 7f6d4ef21180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:08:25.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.898+0000 7f396da00640 1 -- 172.21.15.86:0/482768292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3968163740 con 0x7f3968070e20 2024-04-25T14:08:25.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.898+0000 7f395ea00640 1 -- 172.21.15.86:0/482768292 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+17565 (secure 0 0 0) 0x7f395005d080 con 0x7f3968070e20 2024-04-25T14:08:25.902 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:08:25.902 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":75,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:07:03.055718+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:06:42.666742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T14:06:43.884074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2510298865":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/82848493":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/1793604306":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/3402941646":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/112097751":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/2567754949":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/529842883":"2024-04-26T14:07:03.055680+0000","172.21.15.86:6833/1025829557":"2024-04-26T14:07:03.055680+0000","172.21.15.86:6832/1025829557":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/759420784":"2024-04-26T14:07:03.055680+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:08:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.902+0000 7f396da00640 1 -- 172.21.15.86:0/482768292 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f394406f520 msgr2=0x7f3944071a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:08:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.902+0000 7f396da00640 1 --2- 172.21.15.86:0/482768292 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f394406f520 0x7f3944071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.902+0000 7f396da00640 1 -- 172.21.15.86:0/482768292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3968070e20 msgr2=0x7f39680b74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.902+0000 7f396da00640 1 --2- 172.21.15.86:0/482768292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3968070e20 0x7f39680b74b0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f395000d7d0 tx=0x7f395000dc90 comp rx=0 tx=0).stop 2024-04-25T14:08:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.902+0000 7f396da00640 1 -- 172.21.15.86:0/482768292 shutdown_connections 2024-04-25T14:08:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.902+0000 7f396da00640 1 --2- 172.21.15.86:0/482768292 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f394406f520 0x7f3944071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.902+0000 7f396da00640 1 --2- 172.21.15.86:0/482768292 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39680cdd70 0x7f3968159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.902+0000 7f396da00640 1 --2- 172.21.15.86:0/482768292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3968070e20 0x7f39680b74b0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.902+0000 7f396da00640 1 --2- 172.21.15.86:0/482768292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f396806e700 0x7f39680cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.902+0000 7f396da00640 1 -- 172.21.15.86:0/482768292 >> 172.21.15.86:0/482768292 conn(0x7f39680cffc0 msgr2=0x7f396815c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.902+0000 7f396da00640 1 -- 172.21.15.86:0/482768292 shutdown_connections 2024-04-25T14:08:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:25.902+0000 7f396da00640 1 -- 172.21.15.86:0/482768292 wait complete. 2024-04-25T14:08:25.925 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:06:42.666742+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:06:43.884074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:08:25.925 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:08:26.007 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:08:26.008+0000 7f844abdb180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:08:26.045 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:26.042+0000 7f6d4ef21180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:08:26.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.074+0000 7f0a7f000640 1 Processor -- start 2024-04-25T14:08:26.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.074+0000 7f0a7f000640 1 -- start start 2024-04-25T14:08:26.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.078+0000 7f0a7f000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0a780b39d0 0x7f0a780d2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:26.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.078+0000 7f0a7f000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a780b2560 con 0x7f0a780d2ba0 2024-04-25T14:08:26.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.078+0000 7f0a7f000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a780b3de0 con 0x7f0a780d2790 2024-04-25T14:08:26.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.078+0000 7f0a7f000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0a780d33d0 con 0x7f0a780b39d0 2024-04-25T14:08:26.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.078+0000 7f0a7d000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0a780d2ba0 0x7f0a7806de90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54142/0 (socket says 172.21.15.86:54142) 2024-04-25T14:08:26.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.078+0000 7f0a7d000640 1 -- 172.21.15.86:0/490850886 learned_addr learned my addr 172.21.15.86:0/490850886 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:26.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.078+0000 7f0a77e00640 1 --2- 172.21.15.86:0/490850886 >> v2:172.21.15.155:3300/0 conn(0x7f0a780b39d0 0x7f0a780d2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:26.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.078+0000 7f0a76a00640 1 -- 172.21.15.86:0/490850886 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 626422507 0 0) 0x7f0a780b2560 con 0x7f0a780d2ba0 2024-04-25T14:08:26.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.078+0000 7f0a76a00640 1 -- 172.21.15.86:0/490850886 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a5c003530 con 0x7f0a780d2ba0 2024-04-25T14:08:26.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.078+0000 7f0a76a00640 1 -- 172.21.15.86:0/490850886 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1202870224 0 0) 0x7f0a780b3de0 con 0x7f0a780d2790 2024-04-25T14:08:26.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.078+0000 7f0a76a00640 1 -- 172.21.15.86:0/490850886 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a780b2560 con 0x7f0a780d2790 2024-04-25T14:08:26.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.078+0000 7f0a77e00640 1 -- 172.21.15.86:0/490850886 >> v1:172.21.15.155:6789/0 conn(0x7f0a780d2790 legacy=0x7f0a7806d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:26.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.078+0000 7f0a77e00640 1 -- 172.21.15.86:0/490850886 >> v1:172.21.15.86:6789/0 conn(0x7f0a780d2ba0 legacy=0x7f0a7806de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:26.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.078+0000 7f0a77e00640 1 -- 172.21.15.86:0/490850886 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7806e5a0 con 0x7f0a780b39d0 2024-04-25T14:08:26.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.082+0000 7f0a77e00640 1 --2- 172.21.15.86:0/490850886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a780b39d0 0x7f0a780d2250 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0a680097c0 tx=0x7f0a6801ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4ee68620c57bd3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:26.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.082+0000 7f0a76a00640 1 -- 172.21.15.86:0/490850886 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a680b5070 con 0x7f0a780b39d0 2024-04-25T14:08:26.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.082+0000 7f0a76a00640 1 -- 172.21.15.86:0/490850886 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a680a8990 con 0x7f0a780b39d0 2024-04-25T14:08:26.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.082+0000 7f0a76a00640 1 -- 172.21.15.86:0/490850886 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a680a8d40 con 0x7f0a780b39d0 2024-04-25T14:08:26.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.082+0000 7f0a7f000640 1 -- 172.21.15.86:0/490850886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a780b39d0 msgr2=0x7f0a780d2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:26.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.082+0000 7f0a7f000640 1 --2- 172.21.15.86:0/490850886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a780b39d0 0x7f0a780d2250 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0a680097c0 tx=0x7f0a6801ec90 comp rx=0 tx=0).stop 2024-04-25T14:08:26.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.082+0000 7f0a7f000640 1 -- 172.21.15.86:0/490850886 shutdown_connections 2024-04-25T14:08:26.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.082+0000 7f0a7f000640 1 --2- 172.21.15.86:0/490850886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a780b39d0 0x7f0a780d2250 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:26.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.082+0000 7f0a7f000640 1 -- 172.21.15.86:0/490850886 >> 172.21.15.86:0/490850886 conn(0x7f0a7809ba30 msgr2=0x7f0a780b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:26.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.082+0000 7f0a7f000640 1 -- 172.21.15.86:0/490850886 shutdown_connections 2024-04-25T14:08:26.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.082+0000 7f0a7f000640 1 -- 172.21.15.86:0/490850886 wait complete. 2024-04-25T14:08:26.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.086+0000 7f0a7f000640 1 Processor -- start 2024-04-25T14:08:26.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.086+0000 7f0a7f000640 1 -- start start 2024-04-25T14:08:26.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.086+0000 7f0a7f000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a780b39d0 0x7f0a781567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:26.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.086+0000 7f0a7f000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a780d2790 0x7f0a780cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:26.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.086+0000 7f0a7f000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a780d2ba0 0x7f0a7815b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:26.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.086+0000 7f0a7f000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0a780b2280 con 0x7f0a780d2ba0 2024-04-25T14:08:26.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.086+0000 7f0a7f000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0a780b1fe0 con 0x7f0a780b39d0 2024-04-25T14:08:26.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.086+0000 7f0a7f000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0a7815cea0 con 0x7f0a780d2790 2024-04-25T14:08:26.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.086+0000 7f0a7d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a780d2ba0 0x7f0a7815b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:26.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.086+0000 7f0a77e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a780b39d0 0x7f0a781567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:26.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.086+0000 7f0a7d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a780d2ba0 0x7f0a7815b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53560/0 (socket says 172.21.15.86:53560) 2024-04-25T14:08:26.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.086+0000 7f0a7d000640 1 -- 172.21.15.86:0/2832385302 learned_addr learned my addr 172.21.15.86:0/2832385302 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:26.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.086+0000 7f0a77400640 1 --2- 172.21.15.86:0/2832385302 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a780d2790 0x7f0a780cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:26.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.086+0000 7f0a7d000640 1 -- 172.21.15.86:0/2832385302 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a780d2790 msgr2=0x7f0a780cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:26.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.086+0000 7f0a7d000640 1 --2- 172.21.15.86:0/2832385302 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a780d2790 0x7f0a780cabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:26.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.086+0000 7f0a7d000640 1 -- 172.21.15.86:0/2832385302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a780b39d0 msgr2=0x7f0a781567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:26.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.086+0000 7f0a7d000640 1 --2- 172.21.15.86:0/2832385302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a780b39d0 0x7f0a781567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:26.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.086+0000 7f0a7d000640 1 -- 172.21.15.86:0/2832385302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a780c9570 con 0x7f0a780d2ba0 2024-04-25T14:08:26.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.086+0000 7f0a77e00640 1 --2- 172.21.15.86:0/2832385302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a780b39d0 0x7f0a781567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:08:26.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.086+0000 7f0a7d000640 1 --2- 172.21.15.86:0/2832385302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a780d2ba0 0x7f0a7815b3a0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f0a7000cac0 tx=0x7f0a7000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:26.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.090+0000 7f0a6fe00640 1 -- 172.21.15.86:0/2832385302 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a70017070 con 0x7f0a780d2ba0 2024-04-25T14:08:26.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.090+0000 7f0a6fe00640 1 -- 172.21.15.86:0/2832385302 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a7000ac30 con 0x7f0a780d2ba0 2024-04-25T14:08:26.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.090+0000 7f0a6fe00640 1 -- 172.21.15.86:0/2832385302 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a70012400 con 0x7f0a780d2ba0 2024-04-25T14:08:26.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.090+0000 7f0a7f000640 1 -- 172.21.15.86:0/2832385302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a780c9850 con 0x7f0a780d2ba0 2024-04-25T14:08:26.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.090+0000 7f0a7f000640 1 -- 172.21.15.86:0/2832385302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a780c9e10 con 0x7f0a780d2ba0 2024-04-25T14:08:26.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.090+0000 7f0a7f000640 1 -- 172.21.15.86:0/2832385302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a780a0630 con 0x7f0a780d2ba0 2024-04-25T14:08:26.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.090+0000 7f0a6fe00640 1 -- 172.21.15.86:0/2832385302 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 93309+0+0 (secure 0 0 0) 0x7f0a7001a070 con 0x7f0a780d2ba0 2024-04-25T14:08:26.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.090+0000 7f0a6fe00640 1 --2- 172.21.15.86:0/2832385302 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f0a5806f520 0x7f0a58071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:26.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.090+0000 7f0a77e00640 1 -- 172.21.15.86:0/2832385302 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f0a5806f520 msgr2=0x7f0a58071a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/203029814 2024-04-25T14:08:26.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.090+0000 7f0a6fe00640 1 -- 172.21.15.86:0/2832385302 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f0a7008f440 con 0x7f0a780d2ba0 2024-04-25T14:08:26.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.090+0000 7f0a77e00640 1 --2- 172.21.15.86:0/2832385302 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f0a5806f520 0x7f0a58071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:08:26.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.094+0000 7f0a6fe00640 1 -- 172.21.15.86:0/2832385302 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0a7005b880 con 0x7f0a780d2ba0 2024-04-25T14:08:26.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.258+0000 7f0a7f000640 1 -- 172.21.15.86:0/2832385302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0a780b0700 con 0x7f0a780d2ba0 2024-04-25T14:08:26.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.258+0000 7f0a6fe00640 1 -- 172.21.15.86:0/2832385302 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) ==== 93+0+10 (secure 0 0 0) 0x7f0a7005fb80 con 0x7f0a780d2ba0 2024-04-25T14:08:26.259 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:08:26.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.258+0000 7f0a7f000640 1 -- 172.21.15.86:0/2832385302 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f0a5806f520 msgr2=0x7f0a58071a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:08:26.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.258+0000 7f0a7f000640 1 --2- 172.21.15.86:0/2832385302 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f0a5806f520 0x7f0a58071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:26.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.258+0000 7f0a7f000640 1 -- 172.21.15.86:0/2832385302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a780d2ba0 msgr2=0x7f0a7815b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:26.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.258+0000 7f0a7f000640 1 --2- 172.21.15.86:0/2832385302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a780d2ba0 0x7f0a7815b3a0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f0a7000cac0 tx=0x7f0a7000cf80 comp rx=0 tx=0).stop 2024-04-25T14:08:26.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.258+0000 7f0a7f000640 1 -- 172.21.15.86:0/2832385302 shutdown_connections 2024-04-25T14:08:26.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.258+0000 7f0a7f000640 1 --2- 172.21.15.86:0/2832385302 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f0a5806f520 0x7f0a58071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:26.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.258+0000 7f0a7f000640 1 --2- 172.21.15.86:0/2832385302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a780d2ba0 0x7f0a7815b3a0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:26.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.258+0000 7f0a7f000640 1 --2- 172.21.15.86:0/2832385302 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a780d2790 0x7f0a780cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:26.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.258+0000 7f0a7f000640 1 --2- 172.21.15.86:0/2832385302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a780b39d0 0x7f0a781567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:26.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.258+0000 7f0a7f000640 1 -- 172.21.15.86:0/2832385302 >> 172.21.15.86:0/2832385302 conn(0x7f0a7809ba30 msgr2=0x7f0a780af690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:26.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.258+0000 7f0a7f000640 1 -- 172.21.15.86:0/2832385302 shutdown_connections 2024-04-25T14:08:26.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.258+0000 7f0a7f000640 1 -- 172.21.15.86:0/2832385302 wait complete. 2024-04-25T14:08:26.281 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:08:26.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.430+0000 7f2592e00640 1 Processor -- start 2024-04-25T14:08:26.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.430+0000 7f2592e00640 1 -- start start 2024-04-25T14:08:26.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.430+0000 7f2592e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258c072f90 0x7f258c0d3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:26.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.430+0000 7f2592e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f258c0d4050 con 0x7f258c1441b0 2024-04-25T14:08:26.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.430+0000 7f2592e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f258c0d3d70 con 0x7f258c0d2f50 2024-04-25T14:08:26.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.430+0000 7f2592e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f258c0724b0 con 0x7f258c072f90 2024-04-25T14:08:26.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.430+0000 7f2583e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f258c1441b0 0x7f258c0d2840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54152/0 (socket says 172.21.15.86:54152) 2024-04-25T14:08:26.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.430+0000 7f2583e00640 1 -- 172.21.15.86:0/511449113 learned_addr learned my addr 172.21.15.86:0/511449113 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:26.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.430+0000 7f258be00640 1 --2- 172.21.15.86:0/511449113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258c072f90 0x7f258c0d3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:26.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.430+0000 7f258b400640 1 -- 172.21.15.86:0/511449113 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4028779858 0 0) 0x7f258c0d4050 con 0x7f258c1441b0 2024-04-25T14:08:26.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.430+0000 7f258b400640 1 -- 172.21.15.86:0/511449113 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2570003530 con 0x7f258c1441b0 2024-04-25T14:08:26.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.430+0000 7f258b400640 1 -- 172.21.15.86:0/511449113 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1521854292 0 0) 0x7f258c0d3d70 con 0x7f258c0d2f50 2024-04-25T14:08:26.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.430+0000 7f258b400640 1 -- 172.21.15.86:0/511449113 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f258c0d4050 con 0x7f258c0d2f50 2024-04-25T14:08:26.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.434+0000 7f258be00640 1 -- 172.21.15.86:0/511449113 >> v1:172.21.15.155:6789/0 conn(0x7f258c0d2f50 legacy=0x7f258c161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:26.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.434+0000 7f258be00640 1 -- 172.21.15.86:0/511449113 >> v1:172.21.15.86:6789/0 conn(0x7f258c1441b0 legacy=0x7f258c0d2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:26.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.434+0000 7f258be00640 1 -- 172.21.15.86:0/511449113 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258c072790 con 0x7f258c072f90 2024-04-25T14:08:26.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.438+0000 7f258be00640 1 --2- 172.21.15.86:0/511449113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258c072f90 0x7f258c0d3520 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f257c0097f0 tx=0x7f257c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d833d829069ce51c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:26.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.438+0000 7f258b400640 1 -- 172.21.15.86:0/511449113 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c0b2070 con 0x7f258c072f90 2024-04-25T14:08:26.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.438+0000 7f258b400640 1 -- 172.21.15.86:0/511449113 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f257c0a9070 con 0x7f258c072f90 2024-04-25T14:08:26.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.438+0000 7f258b400640 1 -- 172.21.15.86:0/511449113 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c0ad880 con 0x7f258c072f90 2024-04-25T14:08:26.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.438+0000 7f2592e00640 1 -- 172.21.15.86:0/511449113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258c072f90 msgr2=0x7f258c0d3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:26.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.438+0000 7f2592e00640 1 --2- 172.21.15.86:0/511449113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258c072f90 0x7f258c0d3520 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f257c0097f0 tx=0x7f257c01c450 comp rx=0 tx=0).stop 2024-04-25T14:08:26.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.438+0000 7f2592e00640 1 -- 172.21.15.86:0/511449113 shutdown_connections 2024-04-25T14:08:26.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.438+0000 7f2592e00640 1 --2- 172.21.15.86:0/511449113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258c072f90 0x7f258c0d3520 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:26.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.438+0000 7f2592e00640 1 -- 172.21.15.86:0/511449113 >> 172.21.15.86:0/511449113 conn(0x7f258c0b5100 msgr2=0x7f258c0b5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:26.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.438+0000 7f2592e00640 1 -- 172.21.15.86:0/511449113 shutdown_connections 2024-04-25T14:08:26.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.438+0000 7f2592e00640 1 -- 172.21.15.86:0/511449113 wait complete. 2024-04-25T14:08:26.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.442+0000 7f2592e00640 1 Processor -- start 2024-04-25T14:08:26.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.442+0000 7f2592e00640 1 -- start start 2024-04-25T14:08:26.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.442+0000 7f2592e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f258c072f90 0x7f258c157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:26.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.442+0000 7f2592e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258c0d2f50 0x7f258c145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:26.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.442+0000 7f2592e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f258c1441b0 0x7f258c159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:26.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.442+0000 7f2592e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f258c16be70 con 0x7f258c072f90 2024-04-25T14:08:26.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.442+0000 7f2592e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f258c16bcf0 con 0x7f258c1441b0 2024-04-25T14:08:26.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.442+0000 7f2592e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f258c16bff0 con 0x7f258c0d2f50 2024-04-25T14:08:26.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.442+0000 7f258be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f258c072f90 0x7f258c157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:26.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.442+0000 7f258be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f258c072f90 0x7f258c157fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53576/0 (socket says 172.21.15.86:53576) 2024-04-25T14:08:26.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.442+0000 7f258be00640 1 -- 172.21.15.86:0/2347608728 learned_addr learned my addr 172.21.15.86:0/2347608728 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:26.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.442+0000 7f2583e00640 1 --2- 172.21.15.86:0/2347608728 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258c0d2f50 0x7f258c145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:26.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.442+0000 7f2590e00640 1 --2- 172.21.15.86:0/2347608728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f258c1441b0 0x7f258c159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:26.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.442+0000 7f258be00640 1 -- 172.21.15.86:0/2347608728 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258c0d2f50 msgr2=0x7f258c145800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:26.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.442+0000 7f258be00640 1 --2- 172.21.15.86:0/2347608728 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258c0d2f50 0x7f258c145800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:26.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.442+0000 7f258be00640 1 -- 172.21.15.86:0/2347608728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f258c1441b0 msgr2=0x7f258c159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:26.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.442+0000 7f258be00640 1 --2- 172.21.15.86:0/2347608728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f258c1441b0 0x7f258c159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:26.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.442+0000 7f258be00640 1 -- 172.21.15.86:0/2347608728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258c13ee00 con 0x7f258c072f90 2024-04-25T14:08:26.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.442+0000 7f258be00640 1 --2- 172.21.15.86:0/2347608728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f258c072f90 0x7f258c157fd0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f257c0097f0 tx=0x7f257c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:26.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.442+0000 7f2588c00640 1 -- 172.21.15.86:0/2347608728 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c0b2050 con 0x7f258c072f90 2024-04-25T14:08:26.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.442+0000 7f2588c00640 1 -- 172.21.15.86:0/2347608728 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f257c0be070 con 0x7f258c072f90 2024-04-25T14:08:26.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.442+0000 7f2592e00640 1 -- 172.21.15.86:0/2347608728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f258c13f0e0 con 0x7f258c072f90 2024-04-25T14:08:26.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.442+0000 7f2588c00640 1 -- 172.21.15.86:0/2347608728 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c0b9690 con 0x7f258c072f90 2024-04-25T14:08:26.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.442+0000 7f2592e00640 1 -- 172.21.15.86:0/2347608728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f258c13f6a0 con 0x7f258c072f90 2024-04-25T14:08:26.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.446+0000 7f2592e00640 1 -- 172.21.15.86:0/2347608728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f258c072200 con 0x7f258c072f90 2024-04-25T14:08:26.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.446+0000 7f2588c00640 1 -- 172.21.15.86:0/2347608728 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 93309+0+0 (secure 0 0 0) 0x7f257c0ab070 con 0x7f258c072f90 2024-04-25T14:08:26.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.446+0000 7f2588c00640 1 --2- 172.21.15.86:0/2347608728 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f256806f520 0x7f2568071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:26.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.446+0000 7f2588c00640 1 -- 172.21.15.86:0/2347608728 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f257c12a4b0 con 0x7f258c072f90 2024-04-25T14:08:26.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.446+0000 7f2583e00640 1 -- 172.21.15.86:0/2347608728 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f256806f520 msgr2=0x7f2568071a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/203029814 2024-04-25T14:08:26.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.450+0000 7f2583e00640 1 --2- 172.21.15.86:0/2347608728 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f256806f520 0x7f2568071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:08:26.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.450+0000 7f2588c00640 1 -- 172.21.15.86:0/2347608728 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f257c0c8020 con 0x7f258c072f90 2024-04-25T14:08:26.479 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:08:26.480+0000 7f844abdb180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:08:26.538 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:08:26.540+0000 7f844abdb180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:08:26.543 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:26.542+0000 7f6d4ef21180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:08:26.593 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:08:26.592+0000 7f844abdb180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:08:26.604 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:26.602+0000 7f6d4ef21180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:08:26.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.610+0000 7f2592e00640 1 -- 172.21.15.86:0/2347608728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f258c076bf0 con 0x7f258c072f90 2024-04-25T14:08:26.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.610+0000 7f2588c00640 1 -- 172.21.15.86:0/2347608728 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v75) ==== 104+0+11 (secure 0 0 0) 0x7f257c0160d0 con 0x7f258c072f90 2024-04-25T14:08:26.613 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:08:26.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.614+0000 7f2592e00640 1 -- 172.21.15.86:0/2347608728 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f256806f520 msgr2=0x7f2568071a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:08:26.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.614+0000 7f2592e00640 1 --2- 172.21.15.86:0/2347608728 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f256806f520 0x7f2568071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:26.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.614+0000 7f2592e00640 1 -- 172.21.15.86:0/2347608728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f258c072f90 msgr2=0x7f258c157fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:26.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.614+0000 7f2592e00640 1 --2- 172.21.15.86:0/2347608728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f258c072f90 0x7f258c157fd0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f257c0097f0 tx=0x7f257c01ac90 comp rx=0 tx=0).stop 2024-04-25T14:08:26.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.614+0000 7f2592e00640 1 -- 172.21.15.86:0/2347608728 shutdown_connections 2024-04-25T14:08:26.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.614+0000 7f2592e00640 1 --2- 172.21.15.86:0/2347608728 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f256806f520 0x7f2568071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:26.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.614+0000 7f2592e00640 1 --2- 172.21.15.86:0/2347608728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f258c1441b0 0x7f258c159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:26.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.614+0000 7f2592e00640 1 --2- 172.21.15.86:0/2347608728 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258c0d2f50 0x7f258c145800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:26.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.614+0000 7f2592e00640 1 --2- 172.21.15.86:0/2347608728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f258c072f90 0x7f258c157fd0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:26.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.614+0000 7f2592e00640 1 -- 172.21.15.86:0/2347608728 >> 172.21.15.86:0/2347608728 conn(0x7f258c0b5100 msgr2=0x7f258c0b48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:26.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.614+0000 7f2592e00640 1 -- 172.21.15.86:0/2347608728 shutdown_connections 2024-04-25T14:08:26.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.614+0000 7f2592e00640 1 -- 172.21.15.86:0/2347608728 wait complete. 2024-04-25T14:08:26.636 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:08:26.667 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:26.666+0000 7f6d4ef21180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:08:26.680 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:08:26.680+0000 7f844abdb180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:08:26.737 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:08:26.736+0000 7f844abdb180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:08:26.757 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:26.754+0000 7f6d4ef21180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:08:26.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.786+0000 7fc2fdc00640 1 Processor -- start 2024-04-25T14:08:26.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.786+0000 7fc2fdc00640 1 -- start start 2024-04-25T14:08:26.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.790+0000 7fc2fdc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc2f8071530 0x7fc2f815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:26.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.790+0000 7fc2fdc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2f80cd8d0 con 0x7fc2f81616e0 2024-04-25T14:08:26.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.790+0000 7fc2fdc00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2f80ce970 con 0x7fc2f80d2c80 2024-04-25T14:08:26.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.790+0000 7fc2fdc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc2f80ce720 con 0x7fc2f8071530 2024-04-25T14:08:26.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.790+0000 7fc2f6000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc2f8071530 0x7fc2f815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:26.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.790+0000 7fc2f6000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc2f8071530 0x7fc2f815dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53586/0 (socket says 172.21.15.86:53586) 2024-04-25T14:08:26.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.790+0000 7fc2f6000640 1 -- 172.21.15.86:0/1470218206 learned_addr learned my addr 172.21.15.86:0/1470218206 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:26.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.790+0000 7fc2f6000640 1 -- 172.21.15.86:0/1470218206 >> v1:172.21.15.155:6789/0 conn(0x7fc2f80d2c80 legacy=0x7fc2f8070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:26.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.790+0000 7fc2f6000640 1 -- 172.21.15.86:0/1470218206 >> v1:172.21.15.86:6789/0 conn(0x7fc2f81616e0 legacy=0x7fc2f8161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:26.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.790+0000 7fc2f6000640 1 -- 172.21.15.86:0/1470218206 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f80d1be0 con 0x7fc2f8071530 2024-04-25T14:08:26.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.794+0000 7fc2f6000640 1 --2- 172.21.15.86:0/1470218206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f8071530 0x7fc2f815dc90 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4006fd0 tx=0x7fc2e401ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6fe4dee87665b56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:26.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.794+0000 7fc2f5600640 1 -- 172.21.15.86:0/1470218206 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e40b5070 con 0x7fc2f8071530 2024-04-25T14:08:26.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.794+0000 7fc2f5600640 1 -- 172.21.15.86:0/1470218206 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2e40a8a30 con 0x7fc2f8071530 2024-04-25T14:08:26.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.794+0000 7fc2f5600640 1 -- 172.21.15.86:0/1470218206 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e40a8e00 con 0x7fc2f8071530 2024-04-25T14:08:26.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.794+0000 7fc2fdc00640 1 -- 172.21.15.86:0/1470218206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f8071530 msgr2=0x7fc2f815dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:26.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.794+0000 7fc2fdc00640 1 --2- 172.21.15.86:0/1470218206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f8071530 0x7fc2f815dc90 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4006fd0 tx=0x7fc2e401ed30 comp rx=0 tx=0).stop 2024-04-25T14:08:26.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.794+0000 7fc2fdc00640 1 -- 172.21.15.86:0/1470218206 shutdown_connections 2024-04-25T14:08:26.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.794+0000 7fc2fdc00640 1 --2- 172.21.15.86:0/1470218206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f8071530 0x7fc2f815dc90 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:26.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.794+0000 7fc2fdc00640 1 -- 172.21.15.86:0/1470218206 >> 172.21.15.86:0/1470218206 conn(0x7fc2f80cffc0 msgr2=0x7fc2f80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:26.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.794+0000 7fc2fdc00640 1 -- 172.21.15.86:0/1470218206 shutdown_connections 2024-04-25T14:08:26.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.794+0000 7fc2fdc00640 1 -- 172.21.15.86:0/1470218206 wait complete. 2024-04-25T14:08:26.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.798+0000 7fc2fdc00640 1 Processor -- start 2024-04-25T14:08:26.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.798+0000 7fc2fdc00640 1 -- start start 2024-04-25T14:08:26.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.802+0000 7fc2fdc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2f80d2c80 0x7fc2f815f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:26.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.802+0000 7fc2fdc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f81616e0 0x7fc2f806e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:26.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.802+0000 7fc2fdc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2f81537f0 0x7fc2f80b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:26.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.802+0000 7fc2fdc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f80d2480 con 0x7fc2f81616e0 2024-04-25T14:08:26.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.802+0000 7fc2fdc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f80d21a0 con 0x7fc2f80d2c80 2024-04-25T14:08:26.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.802+0000 7fc2fdc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc2f80d29a0 con 0x7fc2f81537f0 2024-04-25T14:08:26.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.802+0000 7fc2f6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f81616e0 0x7fc2f806e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:26.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.802+0000 7fc2f6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f81616e0 0x7fc2f806e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53594/0 (socket says 172.21.15.86:53594) 2024-04-25T14:08:26.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.802+0000 7fc2f6000640 1 -- 172.21.15.86:0/3143072371 learned_addr learned my addr 172.21.15.86:0/3143072371 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:26.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.802+0000 7fc2f7400640 1 --2- 172.21.15.86:0/3143072371 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2f81537f0 0x7fc2f80b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:26.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.802+0000 7fc2f6a00640 1 --2- 172.21.15.86:0/3143072371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2f80d2c80 0x7fc2f815f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:26.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.802+0000 7fc2f6000640 1 -- 172.21.15.86:0/3143072371 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2f81537f0 msgr2=0x7fc2f80b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:26.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.802+0000 7fc2f6000640 1 --2- 172.21.15.86:0/3143072371 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2f81537f0 0x7fc2f80b1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:26.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.802+0000 7fc2f6000640 1 -- 172.21.15.86:0/3143072371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2f80d2c80 msgr2=0x7fc2f815f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:26.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.802+0000 7fc2f6000640 1 --2- 172.21.15.86:0/3143072371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2f80d2c80 0x7fc2f815f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:26.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.802+0000 7fc2f6000640 1 -- 172.21.15.86:0/3143072371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f806e5f0 con 0x7fc2f81616e0 2024-04-25T14:08:26.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.802+0000 7fc2f7400640 1 --2- 172.21.15.86:0/3143072371 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2f81537f0 0x7fc2f80b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:26.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.802+0000 7fc2f6a00640 1 --2- 172.21.15.86:0/3143072371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2f80d2c80 0x7fc2f815f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:26.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.802+0000 7fc2f6000640 1 --2- 172.21.15.86:0/3143072371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f81616e0 0x7fc2f806e0b0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fc2e401c960 tx=0x7fc2e401bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:26.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.802+0000 7fc2eea00640 1 -- 172.21.15.86:0/3143072371 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e40be070 con 0x7fc2f81616e0 2024-04-25T14:08:26.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.802+0000 7fc2eea00640 1 -- 172.21.15.86:0/3143072371 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2e4005ba0 con 0x7fc2f81616e0 2024-04-25T14:08:26.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.802+0000 7fc2fdc00640 1 -- 172.21.15.86:0/3143072371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f815ff00 con 0x7fc2f81616e0 2024-04-25T14:08:26.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.802+0000 7fc2eea00640 1 -- 172.21.15.86:0/3143072371 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e40b5070 con 0x7fc2f81616e0 2024-04-25T14:08:26.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.802+0000 7fc2fdc00640 1 -- 172.21.15.86:0/3143072371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f806d8b0 con 0x7fc2f81616e0 2024-04-25T14:08:26.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.802+0000 7fc2fdc00640 1 -- 172.21.15.86:0/3143072371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2f80720e0 con 0x7fc2f81616e0 2024-04-25T14:08:26.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.806+0000 7fc2eea00640 1 -- 172.21.15.86:0/3143072371 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 93309+0+0 (secure 0 0 0) 0x7fc2e40c1070 con 0x7fc2f81616e0 2024-04-25T14:08:26.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.806+0000 7fc2eea00640 1 --2- 172.21.15.86:0/3143072371 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fc2d806f550 0x7fc2d8071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:26.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.806+0000 7fc2eea00640 1 -- 172.21.15.86:0/3143072371 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7fc2e412c2b0 con 0x7fc2f81616e0 2024-04-25T14:08:26.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.806+0000 7fc2f6a00640 1 -- 172.21.15.86:0/3143072371 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fc2d806f550 msgr2=0x7fc2d8071a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/203029814 2024-04-25T14:08:26.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.806+0000 7fc2f6a00640 1 --2- 172.21.15.86:0/3143072371 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fc2d806f550 0x7fc2d8071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:08:26.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.810+0000 7fc2eea00640 1 -- 172.21.15.86:0/3143072371 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc2e4018890 con 0x7fc2f81616e0 2024-04-25T14:08:26.821 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:26.818+0000 7f6d4ef21180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:08:26.822 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:08:26.824+0000 7f844abdb180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:08:26.881 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:08:26.880+0000 7f844abdb180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:08:26.907 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:26.906+0000 7f6d4ef21180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:08:26.967 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:26.966+0000 7f6d4ef21180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:08:26.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.966+0000 7fc2fdc00640 1 -- 172.21.15.86:0/3143072371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc2f8076bf0 con 0x7fc2f81616e0 2024-04-25T14:08:26.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.966+0000 7fc2eea00640 1 -- 172.21.15.86:0/3143072371 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v75) ==== 100+0+11 (secure 0 0 0) 0x7fc2e40f8dd0 con 0x7fc2f81616e0 2024-04-25T14:08:26.970 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:08:26.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.970+0000 7fc2fdc00640 1 -- 172.21.15.86:0/3143072371 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fc2d806f550 msgr2=0x7fc2d8071a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:08:26.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.970+0000 7fc2fdc00640 1 --2- 172.21.15.86:0/3143072371 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fc2d806f550 0x7fc2d8071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:26.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.970+0000 7fc2fdc00640 1 -- 172.21.15.86:0/3143072371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f81616e0 msgr2=0x7fc2f806e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:26.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.970+0000 7fc2fdc00640 1 --2- 172.21.15.86:0/3143072371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f81616e0 0x7fc2f806e0b0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fc2e401c960 tx=0x7fc2e401bad0 comp rx=0 tx=0).stop 2024-04-25T14:08:26.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.970+0000 7fc2fdc00640 1 -- 172.21.15.86:0/3143072371 shutdown_connections 2024-04-25T14:08:26.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.970+0000 7fc2fdc00640 1 --2- 172.21.15.86:0/3143072371 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7fc2d806f550 0x7fc2d8071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:26.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.970+0000 7fc2fdc00640 1 --2- 172.21.15.86:0/3143072371 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2f81537f0 0x7fc2f80b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:26.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.970+0000 7fc2fdc00640 1 --2- 172.21.15.86:0/3143072371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f81616e0 0x7fc2f806e0b0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:26.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.970+0000 7fc2fdc00640 1 --2- 172.21.15.86:0/3143072371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2f80d2c80 0x7fc2f815f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:26.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.970+0000 7fc2fdc00640 1 -- 172.21.15.86:0/3143072371 >> 172.21.15.86:0/3143072371 conn(0x7fc2f80cffc0 msgr2=0x7fc2f815a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:26.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.970+0000 7fc2fdc00640 1 -- 172.21.15.86:0/3143072371 shutdown_connections 2024-04-25T14:08:26.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:26.970+0000 7fc2fdc00640 1 -- 172.21.15.86:0/3143072371 wait complete. 2024-04-25T14:08:26.992 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T14:08:27.033 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:08:27.032+0000 7f844abdb180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:08:27.091 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:08:27.092+0000 7f844abdb180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:08:27.127 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:27.126+0000 7f6d4ef21180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:08:27.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.182+0000 7eff1fa00640 1 Processor -- start 2024-04-25T14:08:27.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.182+0000 7eff1fa00640 1 -- start start 2024-04-25T14:08:27.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.182+0000 7eff1fa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7eff1815cc90 0x7eff18160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.182+0000 7eff1fa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff180af8b0 0x7eff180afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.182+0000 7eff1fa00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff180b3d60 con 0x7eff180b3950 2024-04-25T14:08:27.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.182+0000 7eff1fa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7eff180af140 con 0x7eff1815cc90 2024-04-25T14:08:27.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.182+0000 7eff1fa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7eff180af420 con 0x7eff180af8b0 2024-04-25T14:08:27.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.182+0000 7eff17e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7eff1815cc90 0x7eff18160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.182+0000 7eff17e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7eff1815cc90 0x7eff18160eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53598/0 (socket says 172.21.15.86:53598) 2024-04-25T14:08:27.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.182+0000 7eff17e00640 1 -- 172.21.15.86:0/4247960487 learned_addr learned my addr 172.21.15.86:0/4247960487 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:27.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.182+0000 7eff1da00640 1 --2- 172.21.15.86:0/4247960487 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff180af8b0 0x7eff180afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.186+0000 7eff17e00640 1 -- 172.21.15.86:0/4247960487 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff180af8b0 msgr2=0x7eff180afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:27.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.186+0000 7eff17e00640 1 --2- 172.21.15.86:0/4247960487 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff180af8b0 0x7eff180afcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.186+0000 7eff17e00640 1 -- 172.21.15.86:0/4247960487 >> v1:172.21.15.155:6789/0 conn(0x7eff180b3950 legacy=0x7eff181607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.186+0000 7eff17e00640 1 -- 172.21.15.86:0/4247960487 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff180b1d50 con 0x7eff1815cc90 2024-04-25T14:08:27.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.186+0000 7eff17e00640 1 --2- 172.21.15.86:0/4247960487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff1815cc90 0x7eff18160eb0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7eff04009870 tx=0x7eff0401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3033c4196c50709d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:27.190 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:27.186+0000 7f6d4ef21180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:08:27.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.190+0000 7eff17400640 1 -- 172.21.15.86:0/4247960487 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff040b1070 con 0x7eff1815cc90 2024-04-25T14:08:27.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.190+0000 7eff17400640 1 -- 172.21.15.86:0/4247960487 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff040a8050 con 0x7eff1815cc90 2024-04-25T14:08:27.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.190+0000 7eff17400640 1 -- 172.21.15.86:0/4247960487 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff040ac860 con 0x7eff1815cc90 2024-04-25T14:08:27.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.194+0000 7eff1fa00640 1 -- 172.21.15.86:0/4247960487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff1815cc90 msgr2=0x7eff18160eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.194+0000 7eff1fa00640 1 --2- 172.21.15.86:0/4247960487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff1815cc90 0x7eff18160eb0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7eff04009870 tx=0x7eff0401c4d0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.194+0000 7eff1fa00640 1 -- 172.21.15.86:0/4247960487 shutdown_connections 2024-04-25T14:08:27.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.194+0000 7eff1fa00640 1 --2- 172.21.15.86:0/4247960487 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff180af8b0 0x7eff180afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.194+0000 7eff1fa00640 1 --2- 172.21.15.86:0/4247960487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff1815cc90 0x7eff18160eb0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.194+0000 7eff1fa00640 1 -- 172.21.15.86:0/4247960487 >> 172.21.15.86:0/4247960487 conn(0x7eff1809ba30 msgr2=0x7eff180b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:27.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.194+0000 7eff1fa00640 1 -- 172.21.15.86:0/4247960487 shutdown_connections 2024-04-25T14:08:27.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.194+0000 7eff1fa00640 1 -- 172.21.15.86:0/4247960487 wait complete. 2024-04-25T14:08:27.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.194+0000 7eff1fa00640 1 Processor -- start 2024-04-25T14:08:27.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.194+0000 7eff1fa00640 1 -- start start 2024-04-25T14:08:27.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.198+0000 7eff1fa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff180af8b0 0x7eff180c3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.198+0000 7eff1fa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff180b3950 0x7eff180c3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.198+0000 7eff1fa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff1815cc90 0x7eff180d2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.198+0000 7eff1fa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7eff18169c60 con 0x7eff180b3950 2024-04-25T14:08:27.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.198+0000 7eff1fa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7eff18169ae0 con 0x7eff1815cc90 2024-04-25T14:08:27.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.198+0000 7eff1fa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7eff18169de0 con 0x7eff180af8b0 2024-04-25T14:08:27.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.198+0000 7eff17e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff180b3950 0x7eff180c3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.198+0000 7eff17e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff180b3950 0x7eff180c3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53608/0 (socket says 172.21.15.86:53608) 2024-04-25T14:08:27.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.198+0000 7eff17e00640 1 -- 172.21.15.86:0/2972421013 learned_addr learned my addr 172.21.15.86:0/2972421013 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:27.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.198+0000 7eff1d000640 1 --2- 172.21.15.86:0/2972421013 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff180af8b0 0x7eff180c3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.198+0000 7eff1da00640 1 --2- 172.21.15.86:0/2972421013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff1815cc90 0x7eff180d2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.198+0000 7eff1d000640 1 -- 172.21.15.86:0/2972421013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff1815cc90 msgr2=0x7eff180d2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.198+0000 7eff1d000640 1 --2- 172.21.15.86:0/2972421013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff1815cc90 0x7eff180d2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.198+0000 7eff1d000640 1 -- 172.21.15.86:0/2972421013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff180b3950 msgr2=0x7eff180c3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.198+0000 7eff1d000640 1 --2- 172.21.15.86:0/2972421013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff180b3950 0x7eff180c3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.198+0000 7eff1d000640 1 -- 172.21.15.86:0/2972421013 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff180d0a20 con 0x7eff180af8b0 2024-04-25T14:08:27.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.198+0000 7eff17e00640 1 --2- 172.21.15.86:0/2972421013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff180b3950 0x7eff180c3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:08:27.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.198+0000 7eff1da00640 1 --2- 172.21.15.86:0/2972421013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff1815cc90 0x7eff180d2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:08:27.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.198+0000 7eff1d000640 1 --2- 172.21.15.86:0/2972421013 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff180af8b0 0x7eff180c3720 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00c3c0 tx=0x7eff0c00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:27.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.198+0000 7eff14c00640 1 -- 172.21.15.86:0/2972421013 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c0052d0 con 0x7eff180af8b0 2024-04-25T14:08:27.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.198+0000 7eff14c00640 1 -- 172.21.15.86:0/2972421013 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff0c0054b0 con 0x7eff180af8b0 2024-04-25T14:08:27.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.198+0000 7eff1fa00640 1 -- 172.21.15.86:0/2972421013 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff180c5480 con 0x7eff180af8b0 2024-04-25T14:08:27.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.198+0000 7eff14c00640 1 -- 172.21.15.86:0/2972421013 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c006a70 con 0x7eff180af8b0 2024-04-25T14:08:27.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.198+0000 7eff1fa00640 1 -- 172.21.15.86:0/2972421013 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7eff181504e0 con 0x7eff180af8b0 2024-04-25T14:08:27.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.198+0000 7eff1fa00640 1 -- 172.21.15.86:0/2972421013 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff180d0bf0 con 0x7eff180af8b0 2024-04-25T14:08:27.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.202+0000 7eff14c00640 1 -- 172.21.15.86:0/2972421013 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 23) ==== 93309+0+0 (secure 0 0 0) 0x7eff0c006c10 con 0x7eff180af8b0 2024-04-25T14:08:27.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.202+0000 7eff14c00640 1 --2- 172.21.15.86:0/2972421013 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7efef806f520 0x7efef8071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.202+0000 7eff14c00640 1 -- 172.21.15.86:0/2972421013 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7eff0c08eb70 con 0x7eff180af8b0 2024-04-25T14:08:27.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.202+0000 7eff17e00640 1 -- 172.21.15.86:0/2972421013 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7efef806f520 msgr2=0x7efef8071a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/203029814 2024-04-25T14:08:27.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.202+0000 7eff17e00640 1 --2- 172.21.15.86:0/2972421013 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7efef806f520 0x7efef8071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:08:27.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.206+0000 7eff14c00640 1 -- 172.21.15.86:0/2972421013 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7eff0c05af20 con 0x7eff180af8b0 2024-04-25T14:08:27.333 INFO:tasks.ceph.mgr.x.smithi155.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:08:27.333 INFO:tasks.ceph.mgr.x.smithi155.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:08:27.333 INFO:tasks.ceph.mgr.x.smithi155.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:08:27.341 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:08:27.340+0000 7f844abdb180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:08:27.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.406+0000 7eff17e00640 1 -- 172.21.15.86:0/2972421013 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7efef806f520 msgr2=0x7efef8071a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/203029814 2024-04-25T14:08:27.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.406+0000 7eff17e00640 1 --2- 172.21.15.86:0/2972421013 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7efef806f520 0x7efef8071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:08:27.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.418+0000 7eff1fa00640 1 -- 172.21.15.86:0/2972421013 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7eff180a04c0 con 0x7eff180af8b0 2024-04-25T14:08:27.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.422+0000 7eff14c00640 1 -- 172.21.15.86:0/2972421013 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v75) ==== 81+0+581 (secure 0 0 0) 0x7eff0c05f220 con 0x7eff180af8b0 2024-04-25T14:08:27.424 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2510298865 2024-04-26T14:07:03.055680+0000 2024-04-25T14:08:27.424 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/82848493 2024-04-26T14:07:03.055680+0000 2024-04-25T14:08:27.424 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1793604306 2024-04-26T14:07:03.055680+0000 2024-04-25T14:08:27.424 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3402941646 2024-04-26T14:07:03.055680+0000 2024-04-25T14:08:27.424 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/112097751 2024-04-26T14:07:03.055680+0000 2024-04-25T14:08:27.424 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2567754949 2024-04-26T14:07:03.055680+0000 2024-04-25T14:08:27.424 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/529842883 2024-04-26T14:07:03.055680+0000 2024-04-25T14:08:27.424 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6833/1025829557 2024-04-26T14:07:03.055680+0000 2024-04-25T14:08:27.425 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6832/1025829557 2024-04-26T14:07:03.055680+0000 2024-04-25T14:08:27.425 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/759420784 2024-04-26T14:07:03.055680+0000 2024-04-25T14:08:27.426 INFO:teuthology.orchestra.run.smithi086.stderr:listed 10 entries 2024-04-25T14:08:27.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.422+0000 7eff1fa00640 1 -- 172.21.15.86:0/2972421013 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7efef806f520 msgr2=0x7efef8071a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:08:27.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.422+0000 7eff1fa00640 1 --2- 172.21.15.86:0/2972421013 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7efef806f520 0x7efef8071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.422+0000 7eff1fa00640 1 -- 172.21.15.86:0/2972421013 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff180af8b0 msgr2=0x7eff180c3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.422+0000 7eff1fa00640 1 --2- 172.21.15.86:0/2972421013 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff180af8b0 0x7eff180c3720 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00c3c0 tx=0x7eff0c00c880 comp rx=0 tx=0).stop 2024-04-25T14:08:27.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.422+0000 7eff1fa00640 1 -- 172.21.15.86:0/2972421013 shutdown_connections 2024-04-25T14:08:27.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.422+0000 7eff1fa00640 1 --2- 172.21.15.86:0/2972421013 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7efef806f520 0x7efef8071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.422+0000 7eff1fa00640 1 --2- 172.21.15.86:0/2972421013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff1815cc90 0x7eff180d2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.422+0000 7eff1fa00640 1 --2- 172.21.15.86:0/2972421013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff180b3950 0x7eff180c3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.422+0000 7eff1fa00640 1 --2- 172.21.15.86:0/2972421013 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff180af8b0 0x7eff180c3720 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.422+0000 7eff1fa00640 1 -- 172.21.15.86:0/2972421013 >> 172.21.15.86:0/2972421013 conn(0x7eff1809ba30 msgr2=0x7eff180687d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:27.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.422+0000 7eff1fa00640 1 -- 172.21.15.86:0/2972421013 shutdown_connections 2024-04-25T14:08:27.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.422+0000 7eff1fa00640 1 -- 172.21.15.86:0/2972421013 wait complete. 2024-04-25T14:08:27.436 INFO:tasks.ceph.mgr.y.smithi086.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:08:27.436 INFO:tasks.ceph.mgr.y.smithi086.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:08:27.436 INFO:tasks.ceph.mgr.y.smithi086.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:08:27.445 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T14:08:27.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:08:27.445 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T14:08:27.446 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:27.442+0000 7f6d4ef21180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:08:27.509 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T14:08:27.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:08:27.510 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T14:08:27.519 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:08:27.520+0000 7f844abdb180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:08:27.533 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:08:27.533 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:08:27.533 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:08:27.544 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:08:27.544 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:08:27.599 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:08:27.637 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:27.634+0000 7f6d4ef21180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:08:27.652 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:08:27.652+0000 7f844abdb180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:08:27.723 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:08:27.724+0000 7f844abdb180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:08:27.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.746+0000 7f2263200640 1 Processor -- start 2024-04-25T14:08:27.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.746+0000 7f2263200640 1 -- start start 2024-04-25T14:08:27.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.750+0000 7f2263200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f225c0d28f0 0x7f225c16abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.750+0000 7f2263200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f225c163940 con 0x7f225c0cdda0 2024-04-25T14:08:27.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.750+0000 7f2263200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f225c0d2f40 con 0x7f225c0cec60 2024-04-25T14:08:27.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.750+0000 7f2263200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f225c0d2d00 con 0x7f225c0d28f0 2024-04-25T14:08:27.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.750+0000 7f2253e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f225c0cdda0 0x7f225c0cf780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54164/0 (socket says 172.21.15.86:54164) 2024-04-25T14:08:27.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.750+0000 7f2253e00640 1 -- 172.21.15.86:0/7869472 learned_addr learned my addr 172.21.15.86:0/7869472 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:27.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.750+0000 7f2261200640 1 --2- 172.21.15.86:0/7869472 >> v2:172.21.15.86:3300/0 conn(0x7f225c0d28f0 0x7f225c16abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.750+0000 7f2261200640 1 -- 172.21.15.86:0/7869472 >> v1:172.21.15.155:6789/0 conn(0x7f225c0cec60 legacy=0x7f225c0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.750+0000 7f2261200640 1 -- 172.21.15.86:0/7869472 >> v1:172.21.15.86:6789/0 conn(0x7f225c0cdda0 legacy=0x7f225c0cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.750+0000 7f2261200640 1 -- 172.21.15.86:0/7869472 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225c0b3720 con 0x7f225c0d28f0 2024-04-25T14:08:27.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.754+0000 7f2261200640 1 --2- 172.21.15.86:0/7869472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f225c0d28f0 0x7f225c16abe0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f2254009d90 tx=0x7f225401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d6f304ba7f2b0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:27.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.754+0000 7f225b400640 1 -- 172.21.15.86:0/7869472 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22540b2070 con 0x7f225c0d28f0 2024-04-25T14:08:27.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.754+0000 7f225b400640 1 -- 172.21.15.86:0/7869472 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22540a5830 con 0x7f225c0d28f0 2024-04-25T14:08:27.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.754+0000 7f225b400640 1 -- 172.21.15.86:0/7869472 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22540a5c00 con 0x7f225c0d28f0 2024-04-25T14:08:27.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.754+0000 7f2263200640 1 -- 172.21.15.86:0/7869472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f225c0d28f0 msgr2=0x7f225c16abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.754+0000 7f2263200640 1 --2- 172.21.15.86:0/7869472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f225c0d28f0 0x7f225c16abe0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f2254009d90 tx=0x7f225401c970 comp rx=0 tx=0).stop 2024-04-25T14:08:27.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.754+0000 7f2263200640 1 -- 172.21.15.86:0/7869472 shutdown_connections 2024-04-25T14:08:27.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.754+0000 7f2263200640 1 --2- 172.21.15.86:0/7869472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f225c0d28f0 0x7f225c16abe0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.754+0000 7f2263200640 1 -- 172.21.15.86:0/7869472 >> 172.21.15.86:0/7869472 conn(0x7f225c09ba30 msgr2=0x7f225c0b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:27.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.754+0000 7f2263200640 1 -- 172.21.15.86:0/7869472 shutdown_connections 2024-04-25T14:08:27.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.754+0000 7f2263200640 1 -- 172.21.15.86:0/7869472 wait complete. 2024-04-25T14:08:27.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.758+0000 7f2263200640 1 Processor -- start 2024-04-25T14:08:27.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.758+0000 7f2263200640 1 -- start start 2024-04-25T14:08:27.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.758+0000 7f2263200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f225c0cdda0 0x7f225c158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.758+0000 7f2263200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f225c0cec60 0x7f225c0af140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.758+0000 7f2263200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f225c0d28f0 0x7f225c0af680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.758+0000 7f2263200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f225c16dcd0 con 0x7f225c0cdda0 2024-04-25T14:08:27.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.758+0000 7f2263200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f225c16db50 con 0x7f225c0cec60 2024-04-25T14:08:27.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.758+0000 7f2263200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f225c16de50 con 0x7f225c0d28f0 2024-04-25T14:08:27.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.758+0000 7f225be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f225c0cdda0 0x7f225c158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.758+0000 7f225be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f225c0cdda0 0x7f225c158a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53628/0 (socket says 172.21.15.86:53628) 2024-04-25T14:08:27.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.758+0000 7f225be00640 1 -- 172.21.15.86:0/269760228 learned_addr learned my addr 172.21.15.86:0/269760228 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:27.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.758+0000 7f2261200640 1 --2- 172.21.15.86:0/269760228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f225c0d28f0 0x7f225c0af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.758+0000 7f2253e00640 1 --2- 172.21.15.86:0/269760228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f225c0cec60 0x7f225c0af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.762+0000 7f225be00640 1 -- 172.21.15.86:0/269760228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f225c0d28f0 msgr2=0x7f225c0af680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.762+0000 7f225be00640 1 --2- 172.21.15.86:0/269760228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f225c0d28f0 0x7f225c0af680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.762+0000 7f225be00640 1 -- 172.21.15.86:0/269760228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f225c0cec60 msgr2=0x7f225c0af140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.762+0000 7f225be00640 1 --2- 172.21.15.86:0/269760228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f225c0cec60 0x7f225c0af140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.762+0000 7f225be00640 1 -- 172.21.15.86:0/269760228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225c158fc0 con 0x7f225c0cdda0 2024-04-25T14:08:27.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.762+0000 7f225be00640 1 --2- 172.21.15.86:0/269760228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f225c0cdda0 0x7f225c158a80 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f224c00c3f0 tx=0x7f224c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:27.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.762+0000 7f2258c00640 1 -- 172.21.15.86:0/269760228 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c0052d0 con 0x7f225c0cdda0 2024-04-25T14:08:27.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.762+0000 7f2258c00640 1 -- 172.21.15.86:0/269760228 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f224c0054b0 con 0x7f225c0cdda0 2024-04-25T14:08:27.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.762+0000 7f2258c00640 1 -- 172.21.15.86:0/269760228 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c006a70 con 0x7f225c0cdda0 2024-04-25T14:08:27.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.762+0000 7f2263200640 1 -- 172.21.15.86:0/269760228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f225c0c8ce0 con 0x7f225c0cdda0 2024-04-25T14:08:27.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.762+0000 7f2263200640 1 -- 172.21.15.86:0/269760228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f225c15a0a0 con 0x7f225c0cdda0 2024-04-25T14:08:27.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.762+0000 7f2263200640 1 -- 172.21.15.86:0/269760228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f225c0a04c0 con 0x7f225c0cdda0 2024-04-25T14:08:27.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.762+0000 7f2258c00640 1 -- 172.21.15.86:0/269760228 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 93309+0+0 (secure 0 0 0) 0x7f224c006c10 con 0x7f225c0cdda0 2024-04-25T14:08:27.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.762+0000 7f2258c00640 1 --2- 172.21.15.86:0/269760228 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f223806f520 0x7f2238071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.762+0000 7f2258c00640 1 -- 172.21.15.86:0/269760228 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f224c08ebb0 con 0x7f225c0cdda0 2024-04-25T14:08:27.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.762+0000 7f2253e00640 1 -- 172.21.15.86:0/269760228 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f223806f520 msgr2=0x7f2238071a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/203029814 2024-04-25T14:08:27.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.762+0000 7f2253e00640 1 --2- 172.21.15.86:0/269760228 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f223806f520 0x7f2238071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:08:27.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.770+0000 7f2258c00640 1 -- 172.21.15.86:0/269760228 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f224c05aff0 con 0x7f225c0cdda0 2024-04-25T14:08:27.774 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:27.774+0000 7f6d4ef21180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:08:27.849 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:27.846+0000 7f6d4ef21180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:08:27.873 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:08:27.872+0000 7f844abdb180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:08:27.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.930+0000 7f2263200640 1 -- 172.21.15.86:0/269760228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f225c076bf0 con 0x7f225c0cdda0 2024-04-25T14:08:27.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.930+0000 7f2258c00640 1 -- 172.21.15.86:0/269760228 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+17565 (secure 0 0 0) 0x7f224c05f2f0 con 0x7f225c0cdda0 2024-04-25T14:08:27.933 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:08:27.934 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":75,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:07:03.055718+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:06:42.666742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T14:06:43.884074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2510298865":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/82848493":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/1793604306":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/3402941646":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/112097751":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/2567754949":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/529842883":"2024-04-26T14:07:03.055680+0000","172.21.15.86:6833/1025829557":"2024-04-26T14:07:03.055680+0000","172.21.15.86:6832/1025829557":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/759420784":"2024-04-26T14:07:03.055680+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:08:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.934+0000 7f2263200640 1 -- 172.21.15.86:0/269760228 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f223806f520 msgr2=0x7f2238071a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:08:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.934+0000 7f2263200640 1 --2- 172.21.15.86:0/269760228 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f223806f520 0x7f2238071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.934+0000 7f2263200640 1 -- 172.21.15.86:0/269760228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f225c0cdda0 msgr2=0x7f225c158a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.934+0000 7f2263200640 1 --2- 172.21.15.86:0/269760228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f225c0cdda0 0x7f225c158a80 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f224c00c3f0 tx=0x7f224c00c8b0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.934+0000 7f2263200640 1 -- 172.21.15.86:0/269760228 shutdown_connections 2024-04-25T14:08:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.934+0000 7f2263200640 1 --2- 172.21.15.86:0/269760228 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f223806f520 0x7f2238071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.934+0000 7f2263200640 1 --2- 172.21.15.86:0/269760228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f225c0d28f0 0x7f225c0af680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.934+0000 7f2263200640 1 --2- 172.21.15.86:0/269760228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f225c0cec60 0x7f225c0af140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.934+0000 7f2263200640 1 --2- 172.21.15.86:0/269760228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f225c0cdda0 0x7f225c158a80 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.934+0000 7f2263200640 1 -- 172.21.15.86:0/269760228 >> 172.21.15.86:0/269760228 conn(0x7f225c09ba30 msgr2=0x7f225c15bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.934+0000 7f2263200640 1 -- 172.21.15.86:0/269760228 shutdown_connections 2024-04-25T14:08:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:27.934+0000 7f2263200640 1 -- 172.21.15.86:0/269760228 wait complete. 2024-04-25T14:08:27.956 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:06:42.666742+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:06:43.884074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:08:27.956 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:08:28.002 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:28.002+0000 7f6d4ef21180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:08:28.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.146+0000 7f7c9ba00640 1 Processor -- start 2024-04-25T14:08:28.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.146+0000 7f7c9ba00640 1 -- start start 2024-04-25T14:08:28.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.150+0000 7f7c9ba00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7c940d2c80 0x7f7c9415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.150+0000 7f7c9ba00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c94070e20 0x7f7c94071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.150+0000 7f7c9ba00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c94071770 con 0x7f7c940722f0 2024-04-25T14:08:28.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.150+0000 7f7c9ba00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7c940ce720 con 0x7f7c940d2c80 2024-04-25T14:08:28.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.150+0000 7f7c9ba00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7c940cd8d0 con 0x7f7c94070e20 2024-04-25T14:08:28.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.150+0000 7f7c99000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7c940d2c80 0x7f7c9415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.150+0000 7f7c99000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7c940d2c80 0x7f7c9415dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53636/0 (socket says 172.21.15.86:53636) 2024-04-25T14:08:28.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.150+0000 7f7c99000640 1 -- 172.21.15.86:0/4123277923 learned_addr learned my addr 172.21.15.86:0/4123277923 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:28.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.150+0000 7f7c93e00640 1 --2- 172.21.15.86:0/4123277923 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c94070e20 0x7f7c94071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.150+0000 7f7c99000640 1 -- 172.21.15.86:0/4123277923 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c94070e20 msgr2=0x7f7c94071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:28.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.150+0000 7f7c99000640 1 --2- 172.21.15.86:0/4123277923 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c94070e20 0x7f7c94071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.150+0000 7f7c99000640 1 -- 172.21.15.86:0/4123277923 >> v1:172.21.15.86:6789/0 conn(0x7f7c940722f0 legacy=0x7f7c94072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.150+0000 7f7c99000640 1 -- 172.21.15.86:0/4123277923 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c940ce970 con 0x7f7c940d2c80 2024-04-25T14:08:28.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.154+0000 7f7c99000640 1 --2- 172.21.15.86:0/4123277923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c940d2c80 0x7f7c9415dc90 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f7c88009870 tx=0x7f7c8801ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e71dc6770b4037b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:28.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.154+0000 7f7c93400640 1 -- 172.21.15.86:0/4123277923 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c880b5070 con 0x7f7c940d2c80 2024-04-25T14:08:28.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.154+0000 7f7c93400640 1 -- 172.21.15.86:0/4123277923 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c880a8a50 con 0x7f7c940d2c80 2024-04-25T14:08:28.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.154+0000 7f7c93400640 1 -- 172.21.15.86:0/4123277923 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c880a8e20 con 0x7f7c940d2c80 2024-04-25T14:08:28.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.154+0000 7f7c9ba00640 1 -- 172.21.15.86:0/4123277923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c940d2c80 msgr2=0x7f7c9415dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.154+0000 7f7c9ba00640 1 --2- 172.21.15.86:0/4123277923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c940d2c80 0x7f7c9415dc90 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f7c88009870 tx=0x7f7c8801ed50 comp rx=0 tx=0).stop 2024-04-25T14:08:28.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.154+0000 7f7c9ba00640 1 -- 172.21.15.86:0/4123277923 shutdown_connections 2024-04-25T14:08:28.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.154+0000 7f7c9ba00640 1 --2- 172.21.15.86:0/4123277923 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c94070e20 0x7f7c94071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.154+0000 7f7c9ba00640 1 --2- 172.21.15.86:0/4123277923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c940d2c80 0x7f7c9415dc90 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.154+0000 7f7c9ba00640 1 -- 172.21.15.86:0/4123277923 >> 172.21.15.86:0/4123277923 conn(0x7f7c940cffc0 msgr2=0x7f7c940d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:28.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.154+0000 7f7c9ba00640 1 -- 172.21.15.86:0/4123277923 shutdown_connections 2024-04-25T14:08:28.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.154+0000 7f7c9ba00640 1 -- 172.21.15.86:0/4123277923 wait complete. 2024-04-25T14:08:28.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.158+0000 7f7c9ba00640 1 Processor -- start 2024-04-25T14:08:28.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.158+0000 7f7c9ba00640 1 -- start start 2024-04-25T14:08:28.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.158+0000 7f7c9ba00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c94070e20 0x7f7c94154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.158+0000 7f7c9ba00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c940722f0 0x7f7c941f7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.158+0000 7f7c9ba00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c940d2c80 0x7f7c94148040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.158+0000 7f7c9ba00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c940d21a0 con 0x7f7c940d2c80 2024-04-25T14:08:28.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.158+0000 7f7c9ba00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7c940d1ec0 con 0x7f7c94070e20 2024-04-25T14:08:28.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.158+0000 7f7c9ba00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7c940d2480 con 0x7f7c940722f0 2024-04-25T14:08:28.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.158+0000 7f7c99a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c940d2c80 0x7f7c94148040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.158+0000 7f7c99000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c94070e20 0x7f7c94154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.158+0000 7f7c99a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c940d2c80 0x7f7c94148040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53638/0 (socket says 172.21.15.86:53638) 2024-04-25T14:08:28.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.158+0000 7f7c99a00640 1 -- 172.21.15.86:0/898501268 learned_addr learned my addr 172.21.15.86:0/898501268 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:28.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.158+0000 7f7c93e00640 1 --2- 172.21.15.86:0/898501268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c940722f0 0x7f7c941f7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.158+0000 7f7c99a00640 1 -- 172.21.15.86:0/898501268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c940722f0 msgr2=0x7f7c941f7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.158+0000 7f7c99a00640 1 --2- 172.21.15.86:0/898501268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c940722f0 0x7f7c941f7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.158+0000 7f7c99a00640 1 -- 172.21.15.86:0/898501268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c94070e20 msgr2=0x7f7c94154040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.158+0000 7f7c99a00640 1 --2- 172.21.15.86:0/898501268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c94070e20 0x7f7c94154040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.158+0000 7f7c99a00640 1 -- 172.21.15.86:0/898501268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9415ed40 con 0x7f7c940d2c80 2024-04-25T14:08:28.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.158+0000 7f7c99a00640 1 --2- 172.21.15.86:0/898501268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c940d2c80 0x7f7c94148040 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c00c760 tx=0x7f7c8c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:28.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.158+0000 7f7c90c00640 1 -- 172.21.15.86:0/898501268 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c003b80 con 0x7f7c940d2c80 2024-04-25T14:08:28.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.158+0000 7f7c90c00640 1 -- 172.21.15.86:0/898501268 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c8c003d20 con 0x7f7c940d2c80 2024-04-25T14:08:28.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.158+0000 7f7c9ba00640 1 -- 172.21.15.86:0/898501268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c9415f020 con 0x7f7c940d2c80 2024-04-25T14:08:28.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.158+0000 7f7c90c00640 1 -- 172.21.15.86:0/898501268 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c00aa90 con 0x7f7c940d2c80 2024-04-25T14:08:28.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.162+0000 7f7c9ba00640 1 -- 172.21.15.86:0/898501268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c9415f5e0 con 0x7f7c940d2c80 2024-04-25T14:08:28.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.162+0000 7f7c9ba00640 1 -- 172.21.15.86:0/898501268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c94076bf0 con 0x7f7c940d2c80 2024-04-25T14:08:28.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.162+0000 7f7c90c00640 1 -- 172.21.15.86:0/898501268 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 93309+0+0 (secure 0 0 0) 0x7f7c8c00ac30 con 0x7f7c940d2c80 2024-04-25T14:08:28.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.162+0000 7f7c90c00640 1 --2- 172.21.15.86:0/898501268 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f7c7406f550 0x7f7c74071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.162+0000 7f7c99000640 1 -- 172.21.15.86:0/898501268 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f7c7406f550 msgr2=0x7f7c74071a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/203029814 2024-04-25T14:08:28.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.162+0000 7f7c90c00640 1 -- 172.21.15.86:0/898501268 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f7c8c08f570 con 0x7f7c940d2c80 2024-04-25T14:08:28.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.162+0000 7f7c99000640 1 --2- 172.21.15.86:0/898501268 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f7c7406f550 0x7f7c74071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:08:28.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.166+0000 7f7c90c00640 1 -- 172.21.15.86:0/898501268 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7c94071770 con 0x7f7c940d2c80 2024-04-25T14:08:28.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.330+0000 7f7c9ba00640 1 -- 172.21.15.86:0/898501268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7c940d1470 con 0x7f7c940d2c80 2024-04-25T14:08:28.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.330+0000 7f7c90c00640 1 -- 172.21.15.86:0/898501268 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) ==== 93+0+10 (secure 0 0 0) 0x7f7c8c05b920 con 0x7f7c940d2c80 2024-04-25T14:08:28.332 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:08:28.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.330+0000 7f7c9ba00640 1 -- 172.21.15.86:0/898501268 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f7c7406f550 msgr2=0x7f7c74071a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:08:28.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.330+0000 7f7c9ba00640 1 --2- 172.21.15.86:0/898501268 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f7c7406f550 0x7f7c74071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.330+0000 7f7c9ba00640 1 -- 172.21.15.86:0/898501268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c940d2c80 msgr2=0x7f7c94148040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.330+0000 7f7c9ba00640 1 --2- 172.21.15.86:0/898501268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c940d2c80 0x7f7c94148040 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c00c760 tx=0x7f7c8c00cc20 comp rx=0 tx=0).stop 2024-04-25T14:08:28.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.330+0000 7f7c9ba00640 1 -- 172.21.15.86:0/898501268 shutdown_connections 2024-04-25T14:08:28.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.330+0000 7f7c9ba00640 1 --2- 172.21.15.86:0/898501268 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f7c7406f550 0x7f7c74071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.330+0000 7f7c9ba00640 1 --2- 172.21.15.86:0/898501268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c940d2c80 0x7f7c94148040 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.330+0000 7f7c9ba00640 1 --2- 172.21.15.86:0/898501268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c940722f0 0x7f7c941f7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.330+0000 7f7c9ba00640 1 --2- 172.21.15.86:0/898501268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c94070e20 0x7f7c94154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.330+0000 7f7c9ba00640 1 -- 172.21.15.86:0/898501268 >> 172.21.15.86:0/898501268 conn(0x7f7c940cffc0 msgr2=0x7f7c940b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:28.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.334+0000 7f7c9ba00640 1 -- 172.21.15.86:0/898501268 shutdown_connections 2024-04-25T14:08:28.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.334+0000 7f7c9ba00640 1 -- 172.21.15.86:0/898501268 wait complete. 2024-04-25T14:08:28.354 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:08:28.419 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:08:28.420+0000 7f844abdb180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:08:28.504 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:08:28.504+0000 7f844abdb180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:08:28.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.550+0000 7f764ea00640 1 Processor -- start 2024-04-25T14:08:28.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.550+0000 7f764ea00640 1 -- start start 2024-04-25T14:08:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.550+0000 7f764ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7648071530 0x7f764815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.554+0000 7f764ea00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76481616e0 0x7f7648161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.554+0000 7f764ea00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76480cd8d0 con 0x7f76480d2c80 2024-04-25T14:08:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.554+0000 7f764ea00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f76480ce720 con 0x7f7648071530 2024-04-25T14:08:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.554+0000 7f764ea00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f76480ce970 con 0x7f76481616e0 2024-04-25T14:08:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.554+0000 7f7647400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7648071530 0x7f764815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.554+0000 7f7647400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7648071530 0x7f764815dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53648/0 (socket says 172.21.15.86:53648) 2024-04-25T14:08:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.554+0000 7f7647400640 1 -- 172.21.15.86:0/3230194358 learned_addr learned my addr 172.21.15.86:0/3230194358 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.554+0000 7f764ca00640 1 --2- 172.21.15.86:0/3230194358 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76481616e0 0x7f7648161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.554+0000 7f7647400640 1 -- 172.21.15.86:0/3230194358 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76481616e0 msgr2=0x7f7648161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:28.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.554+0000 7f7647400640 1 --2- 172.21.15.86:0/3230194358 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76481616e0 0x7f7648161af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.554+0000 7f7647400640 1 -- 172.21.15.86:0/3230194358 >> v1:172.21.15.86:6789/0 conn(0x7f76480d2c80 legacy=0x7f7648070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.554+0000 7f7647400640 1 -- 172.21.15.86:0/3230194358 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76480d1be0 con 0x7f7648071530 2024-04-25T14:08:28.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.554+0000 7f7647400640 1 --2- 172.21.15.86:0/3230194358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7648071530 0x7f764815dc90 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f7634006fd0 tx=0x7f763401ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711e6603c0ff456f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:28.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.554+0000 7f7646a00640 1 -- 172.21.15.86:0/3230194358 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76340b5070 con 0x7f7648071530 2024-04-25T14:08:28.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.554+0000 7f7646a00640 1 -- 172.21.15.86:0/3230194358 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76340a8a30 con 0x7f7648071530 2024-04-25T14:08:28.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.554+0000 7f7646a00640 1 -- 172.21.15.86:0/3230194358 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76340a8e00 con 0x7f7648071530 2024-04-25T14:08:28.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.558+0000 7f764ea00640 1 -- 172.21.15.86:0/3230194358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7648071530 msgr2=0x7f764815dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.558+0000 7f764ea00640 1 --2- 172.21.15.86:0/3230194358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7648071530 0x7f764815dc90 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f7634006fd0 tx=0x7f763401ed30 comp rx=0 tx=0).stop 2024-04-25T14:08:28.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.558+0000 7f764ea00640 1 -- 172.21.15.86:0/3230194358 shutdown_connections 2024-04-25T14:08:28.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.558+0000 7f764ea00640 1 --2- 172.21.15.86:0/3230194358 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76481616e0 0x7f7648161af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.558+0000 7f764ea00640 1 --2- 172.21.15.86:0/3230194358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7648071530 0x7f764815dc90 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.558+0000 7f764ea00640 1 -- 172.21.15.86:0/3230194358 >> 172.21.15.86:0/3230194358 conn(0x7f76480cffc0 msgr2=0x7f76480d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:28.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.558+0000 7f764ea00640 1 -- 172.21.15.86:0/3230194358 shutdown_connections 2024-04-25T14:08:28.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.558+0000 7f764ea00640 1 -- 172.21.15.86:0/3230194358 wait complete. 2024-04-25T14:08:28.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.558+0000 7f764ea00640 1 Processor -- start 2024-04-25T14:08:28.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.558+0000 7f764ea00640 1 -- start start 2024-04-25T14:08:28.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.562+0000 7f764ea00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76480d2c80 0x7f764815e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.562+0000 7f764ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76481616e0 0x7f76480c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.562+0000 7f764ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7648160910 0x7f7648151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.562+0000 7f764ea00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f76480d2480 con 0x7f76481616e0 2024-04-25T14:08:28.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.562+0000 7f764ea00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f76480d21a0 con 0x7f7648160910 2024-04-25T14:08:28.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.562+0000 7f764ea00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f76480d29a0 con 0x7f76480d2c80 2024-04-25T14:08:28.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.562+0000 7f7647400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76481616e0 0x7f76480c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.562+0000 7f7647e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76480d2c80 0x7f764815e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.562+0000 7f764ca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7648160910 0x7f7648151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.562+0000 7f7647e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76480d2c80 0x7f764815e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:39388/0 (socket says 172.21.15.86:39388) 2024-04-25T14:08:28.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.562+0000 7f7647e00640 1 -- 172.21.15.86:0/2256348180 learned_addr learned my addr 172.21.15.86:0/2256348180 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:28.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.562+0000 7f7647400640 1 -- 172.21.15.86:0/2256348180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76480d2c80 msgr2=0x7f764815e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.562+0000 7f7647400640 1 --2- 172.21.15.86:0/2256348180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76480d2c80 0x7f764815e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.562+0000 7f7647400640 1 -- 172.21.15.86:0/2256348180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7648160910 msgr2=0x7f7648151650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.562+0000 7f7647400640 1 --2- 172.21.15.86:0/2256348180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7648160910 0x7f7648151650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.562+0000 7f7647400640 1 -- 172.21.15.86:0/2256348180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764815efb0 con 0x7f76481616e0 2024-04-25T14:08:28.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.562+0000 7f7647400640 1 --2- 172.21.15.86:0/2256348180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76481616e0 0x7f76480c9040 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f76480733a0 tx=0x7f763401bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:28.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.562+0000 7f763fe00640 1 -- 172.21.15.86:0/2256348180 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76340b5070 con 0x7f76481616e0 2024-04-25T14:08:28.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.562+0000 7f763fe00640 1 -- 172.21.15.86:0/2256348180 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7634003620 con 0x7f76481616e0 2024-04-25T14:08:28.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.562+0000 7f763fe00640 1 -- 172.21.15.86:0/2256348180 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634003a10 con 0x7f76481616e0 2024-04-25T14:08:28.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.562+0000 7f764ea00640 1 -- 172.21.15.86:0/2256348180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f764815f290 con 0x7f76481616e0 2024-04-25T14:08:28.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.562+0000 7f764ea00640 1 -- 172.21.15.86:0/2256348180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76480c9580 con 0x7f76481616e0 2024-04-25T14:08:28.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.562+0000 7f764ea00640 1 -- 172.21.15.86:0/2256348180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f764815f7b0 con 0x7f76481616e0 2024-04-25T14:08:28.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.566+0000 7f763fe00640 1 -- 172.21.15.86:0/2256348180 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 93309+0+0 (secure 0 0 0) 0x7f76340c0060 con 0x7f76481616e0 2024-04-25T14:08:28.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.566+0000 7f763fe00640 1 --2- 172.21.15.86:0/2256348180 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f762806f520 0x7f7628071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.566+0000 7f7647e00640 1 -- 172.21.15.86:0/2256348180 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f762806f520 msgr2=0x7f7628071a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/203029814 2024-04-25T14:08:28.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.566+0000 7f763fe00640 1 -- 172.21.15.86:0/2256348180 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f763412bfe0 con 0x7f76481616e0 2024-04-25T14:08:28.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.566+0000 7f7647e00640 1 --2- 172.21.15.86:0/2256348180 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f762806f520 0x7f7628071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:08:28.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.570+0000 7f763fe00640 1 -- 172.21.15.86:0/2256348180 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f764815f7b0 con 0x7f76481616e0 2024-04-25T14:08:28.578 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:28.574+0000 7f6d4ef21180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:08:28.665 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:28.662+0000 7f6d4ef21180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:08:28.668 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:08:28.668+0000 7f844abdb180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:08:28.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.730+0000 7f764ea00640 1 -- 172.21.15.86:0/2256348180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f76480720e0 con 0x7f76481616e0 2024-04-25T14:08:28.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.730+0000 7f763fe00640 1 -- 172.21.15.86:0/2256348180 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v75) ==== 104+0+11 (secure 0 0 0) 0x7f7634018890 con 0x7f76481616e0 2024-04-25T14:08:28.732 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:08:28.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.730+0000 7f764ea00640 1 -- 172.21.15.86:0/2256348180 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f762806f520 msgr2=0x7f7628071a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:08:28.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.730+0000 7f764ea00640 1 --2- 172.21.15.86:0/2256348180 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f762806f520 0x7f7628071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.730+0000 7f764ea00640 1 -- 172.21.15.86:0/2256348180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76481616e0 msgr2=0x7f76480c9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.730+0000 7f764ea00640 1 --2- 172.21.15.86:0/2256348180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76481616e0 0x7f76480c9040 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f76480733a0 tx=0x7f763401bad0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.734+0000 7f764ea00640 1 -- 172.21.15.86:0/2256348180 shutdown_connections 2024-04-25T14:08:28.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.734+0000 7f764ea00640 1 --2- 172.21.15.86:0/2256348180 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f762806f520 0x7f7628071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.734+0000 7f764ea00640 1 --2- 172.21.15.86:0/2256348180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7648160910 0x7f7648151650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.734+0000 7f764ea00640 1 --2- 172.21.15.86:0/2256348180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76481616e0 0x7f76480c9040 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.734+0000 7f764ea00640 1 --2- 172.21.15.86:0/2256348180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76480d2c80 0x7f764815e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.734+0000 7f764ea00640 1 -- 172.21.15.86:0/2256348180 >> 172.21.15.86:0/2256348180 conn(0x7f76480cffc0 msgr2=0x7f76480cbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:28.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.734+0000 7f764ea00640 1 -- 172.21.15.86:0/2256348180 shutdown_connections 2024-04-25T14:08:28.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.734+0000 7f764ea00640 1 -- 172.21.15.86:0/2256348180 wait complete. 2024-04-25T14:08:28.753 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:08:28.752+0000 7f844abdb180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:08:28.754 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:08:28.834 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:28.830+0000 7f6d4ef21180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:08:28.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.906+0000 7f9f9c200640 1 Processor -- start 2024-04-25T14:08:28.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.906+0000 7f9f9c200640 1 -- start start 2024-04-25T14:08:28.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.906+0000 7f9f9c200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9f940cdd70 0x7f9f940ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.906+0000 7f9f9c200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9f94070e20 0x7f9f94071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.906+0000 7f9f9c200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f94071770 con 0x7f9f9406e700 2024-04-25T14:08:28.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.906+0000 7f9f9c200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9f9406d940 con 0x7f9f940cdd70 2024-04-25T14:08:28.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.906+0000 7f9f9c200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9f9406dc20 con 0x7f9f94070e20 2024-04-25T14:08:28.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.906+0000 7f9f99800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9f940cdd70 0x7f9f940ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.906+0000 7f9f99800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9f940cdd70 0x7f9f940ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43204/0 (socket says 172.21.15.86:43204) 2024-04-25T14:08:28.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.906+0000 7f9f99800640 1 -- 172.21.15.86:0/2566403923 learned_addr learned my addr 172.21.15.86:0/2566403923 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:28.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.906+0000 7f9f98e00640 1 --2- 172.21.15.86:0/2566403923 >> v2:172.21.15.155:3300/0 conn(0x7f9f94070e20 0x7f9f94071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.906+0000 7f9f99800640 1 -- 172.21.15.86:0/2566403923 >> v2:172.21.15.155:3300/0 conn(0x7f9f94070e20 msgr2=0x7f9f94071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:28.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.906+0000 7f9f99800640 1 --2- 172.21.15.86:0/2566403923 >> v2:172.21.15.155:3300/0 conn(0x7f9f94070e20 0x7f9f94071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.906+0000 7f9f99800640 1 -- 172.21.15.86:0/2566403923 >> v1:172.21.15.155:6789/0 conn(0x7f9f9406e700 legacy=0x7f9f9406eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:28.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.906+0000 7f9f99800640 1 -- 172.21.15.86:0/2566403923 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f9406df00 con 0x7f9f940cdd70 2024-04-25T14:08:28.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.906+0000 7f9f98e00640 1 --2- 172.21.15.86:0/2566403923 >> v2:172.21.15.155:3300/0 conn(0x7f9f94070e20 0x7f9f94071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:28.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.910+0000 7f9f99800640 1 --2- 172.21.15.86:0/2566403923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f940cdd70 0x7f9f940ce180 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f9f94073230 tx=0x7f9f8801c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db465bfa636b3752 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:28.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.910+0000 7f9f93e00640 1 -- 172.21.15.86:0/2566403923 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f880b0070 con 0x7f9f940cdd70 2024-04-25T14:08:28.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.910+0000 7f9f93e00640 1 -- 172.21.15.86:0/2566403923 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f880ab600 con 0x7f9f940cdd70 2024-04-25T14:08:28.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.910+0000 7f9f93e00640 1 -- 172.21.15.86:0/2566403923 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f880ab9d0 con 0x7f9f940cdd70 2024-04-25T14:08:28.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.910+0000 7f9f9c200640 1 -- 172.21.15.86:0/2566403923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f940cdd70 msgr2=0x7f9f940ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.910+0000 7f9f9c200640 1 --2- 172.21.15.86:0/2566403923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f940cdd70 0x7f9f940ce180 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f9f94073230 tx=0x7f9f8801c270 comp rx=0 tx=0).stop 2024-04-25T14:08:28.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.910+0000 7f9f9c200640 1 -- 172.21.15.86:0/2566403923 shutdown_connections 2024-04-25T14:08:28.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.910+0000 7f9f9c200640 1 --2- 172.21.15.86:0/2566403923 >> v2:172.21.15.155:3300/0 conn(0x7f9f94070e20 0x7f9f94071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.910+0000 7f9f9c200640 1 --2- 172.21.15.86:0/2566403923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f940cdd70 0x7f9f940ce180 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.910+0000 7f9f9c200640 1 -- 172.21.15.86:0/2566403923 >> 172.21.15.86:0/2566403923 conn(0x7f9f940cffc0 msgr2=0x7f9f940d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:28.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.910+0000 7f9f9c200640 1 -- 172.21.15.86:0/2566403923 shutdown_connections 2024-04-25T14:08:28.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.910+0000 7f9f9c200640 1 -- 172.21.15.86:0/2566403923 wait complete. 2024-04-25T14:08:28.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.914+0000 7f9f9c200640 1 Processor -- start 2024-04-25T14:08:28.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.914+0000 7f9f9c200640 1 -- start start 2024-04-25T14:08:28.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.914+0000 7f9f9c200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f9406e700 0x7f9f940cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.918+0000 7f9f9c200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f94070e20 0x7f9f941510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.918+0000 7f9f9c200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f940cdd70 0x7f9f9415d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.918+0000 7f9f9c200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9416cc00 con 0x7f9f940cdd70 2024-04-25T14:08:28.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.918+0000 7f9f9c200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9f94164380 con 0x7f9f9406e700 2024-04-25T14:08:28.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.918+0000 7f9f9c200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9f9416cd80 con 0x7f9f94070e20 2024-04-25T14:08:28.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.918+0000 7f9f9a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f940cdd70 0x7f9f9415d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.918+0000 7f9f99800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f9406e700 0x7f9f940cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.918+0000 7f9f98e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f94070e20 0x7f9f941510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.918+0000 7f9f99800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f9406e700 0x7f9f940cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:42100/0 (socket says 172.21.15.86:42100) 2024-04-25T14:08:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.918+0000 7f9f99800640 1 -- 172.21.15.86:0/20174557 learned_addr learned my addr 172.21.15.86:0/20174557 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.918+0000 7f9f9a200640 1 -- 172.21.15.86:0/20174557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f94070e20 msgr2=0x7f9f941510f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.918+0000 7f9f9a200640 1 --2- 172.21.15.86:0/20174557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f94070e20 0x7f9f941510f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.918+0000 7f9f9a200640 1 -- 172.21.15.86:0/20174557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f9406e700 msgr2=0x7f9f940cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.918+0000 7f9f9a200640 1 --2- 172.21.15.86:0/20174557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f9406e700 0x7f9f940cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.918+0000 7f9f9a200640 1 -- 172.21.15.86:0/20174557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f940cc6d0 con 0x7f9f940cdd70 2024-04-25T14:08:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.918+0000 7f9f98e00640 1 --2- 172.21.15.86:0/20174557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f94070e20 0x7f9f941510f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:08:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.918+0000 7f9f99800640 1 --2- 172.21.15.86:0/20174557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f9406e700 0x7f9f940cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:08:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.918+0000 7f9f9a200640 1 --2- 172.21.15.86:0/20174557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f940cdd70 0x7f9f9415d750 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c00bdd0 tx=0x7f9f8c009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:28.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.918+0000 7f9f91600640 1 -- 172.21.15.86:0/20174557 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c019070 con 0x7f9f940cdd70 2024-04-25T14:08:28.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.918+0000 7f9f91600640 1 -- 172.21.15.86:0/20174557 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f8c00cdf0 con 0x7f9f940cdd70 2024-04-25T14:08:28.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.918+0000 7f9f9c200640 1 -- 172.21.15.86:0/20174557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f94151630 con 0x7f9f940cdd70 2024-04-25T14:08:28.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.918+0000 7f9f91600640 1 -- 172.21.15.86:0/20174557 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c014660 con 0x7f9f940cdd70 2024-04-25T14:08:28.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.918+0000 7f9f9c200640 1 -- 172.21.15.86:0/20174557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f940cb910 con 0x7f9f940cdd70 2024-04-25T14:08:28.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.918+0000 7f9f9c200640 1 -- 172.21.15.86:0/20174557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f940720e0 con 0x7f9f940cdd70 2024-04-25T14:08:28.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.922+0000 7f9f91600640 1 -- 172.21.15.86:0/20174557 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 93309+0+0 (secure 0 0 0) 0x7f9f8c01c070 con 0x7f9f940cdd70 2024-04-25T14:08:28.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.922+0000 7f9f91600640 1 --2- 172.21.15.86:0/20174557 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f9f7406f520 0x7f9f74071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.922+0000 7f9f91600640 1 -- 172.21.15.86:0/20174557 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f9f8c08f540 con 0x7f9f940cdd70 2024-04-25T14:08:28.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.922+0000 7f9f99800640 1 -- 172.21.15.86:0/20174557 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f9f7406f520 msgr2=0x7f9f74071a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/203029814 2024-04-25T14:08:28.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.922+0000 7f9f99800640 1 --2- 172.21.15.86:0/20174557 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f9f7406f520 0x7f9f74071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:08:28.924 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:28.922+0000 7f6d4ef21180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:08:28.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:28.926+0000 7f9f91600640 1 -- 172.21.15.86:0/20174557 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9f940cc6d0 con 0x7f9f940cdd70 2024-04-25T14:08:29.073 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:08:29.072+0000 7f844abdb180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:08:29.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.086+0000 7f9f9c200640 1 -- 172.21.15.86:0/20174557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9f94076bf0 con 0x7f9f940cdd70 2024-04-25T14:08:29.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.086+0000 7f9f91600640 1 -- 172.21.15.86:0/20174557 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v75) ==== 100+0+11 (secure 0 0 0) 0x7f9f8c05b9a0 con 0x7f9f940cdd70 2024-04-25T14:08:29.090 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:08:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.090+0000 7f9f9c200640 1 -- 172.21.15.86:0/20174557 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f9f7406f520 msgr2=0x7f9f74071a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:08:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.090+0000 7f9f9c200640 1 --2- 172.21.15.86:0/20174557 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f9f7406f520 0x7f9f74071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.090+0000 7f9f9c200640 1 -- 172.21.15.86:0/20174557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f940cdd70 msgr2=0x7f9f9415d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.090+0000 7f9f9c200640 1 --2- 172.21.15.86:0/20174557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f940cdd70 0x7f9f9415d750 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c00bdd0 tx=0x7f9f8c009ed0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.090+0000 7f9f9c200640 1 -- 172.21.15.86:0/20174557 shutdown_connections 2024-04-25T14:08:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.090+0000 7f9f9c200640 1 --2- 172.21.15.86:0/20174557 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f9f7406f520 0x7f9f74071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.090+0000 7f9f9c200640 1 --2- 172.21.15.86:0/20174557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f940cdd70 0x7f9f9415d750 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.090+0000 7f9f9c200640 1 --2- 172.21.15.86:0/20174557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f94070e20 0x7f9f941510f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.090+0000 7f9f9c200640 1 --2- 172.21.15.86:0/20174557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f9406e700 0x7f9f940cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.090+0000 7f9f9c200640 1 -- 172.21.15.86:0/20174557 >> 172.21.15.86:0/20174557 conn(0x7f9f940cffc0 msgr2=0x7f9f940d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.090+0000 7f9f9c200640 1 -- 172.21.15.86:0/20174557 shutdown_connections 2024-04-25T14:08:29.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.090+0000 7f9f9c200640 1 -- 172.21.15.86:0/20174557 wait complete. 2024-04-25T14:08:29.112 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T14:08:29.135 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:08:29.136+0000 7f844abdb180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:08:29.261 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:29.258+0000 7f6d4ef21180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:08:29.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.266+0000 7f5ef6e00640 1 Processor -- start 2024-04-25T14:08:29.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.266+0000 7f5ef6e00640 1 -- start start 2024-04-25T14:08:29.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.266+0000 7f5ef6e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5ef00cf970 0x7f5ef0070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.266+0000 7f5ef6e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ef0071770 con 0x7f5ef0071360 2024-04-25T14:08:29.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.266+0000 7f5ef6e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ef006ef90 con 0x7f5ef006e970 2024-04-25T14:08:29.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.266+0000 7f5ef6e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5ef0163830 con 0x7f5ef00cf970 2024-04-25T14:08:29.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.266+0000 7f5eefe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5ef00cf970 0x7f5ef0070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.266+0000 7f5eefe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5ef00cf970 0x7f5ef0070e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43236/0 (socket says 172.21.15.86:43236) 2024-04-25T14:08:29.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.266+0000 7f5eefe00640 1 -- 172.21.15.86:0/363555838 learned_addr learned my addr 172.21.15.86:0/363555838 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:29.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.266+0000 7f5ee7e00640 1 --1- 172.21.15.86:0/363555838 >> v1:172.21.15.86:6789/0 conn(0x7f5ef0071360 0x7f5ef006e260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:58380/0 (socket says 172.21.15.86:58380) 2024-04-25T14:08:29.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.266+0000 7f5eefe00640 1 -- 172.21.15.86:0/363555838 >> v1:172.21.15.155:6789/0 conn(0x7f5ef006e970 legacy=0x7f5ef00cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.266+0000 7f5eefe00640 1 -- 172.21.15.86:0/363555838 >> v1:172.21.15.86:6789/0 conn(0x7f5ef0071360 legacy=0x7f5ef006e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.266+0000 7f5eefe00640 1 -- 172.21.15.86:0/363555838 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef0163a80 con 0x7f5ef00cf970 2024-04-25T14:08:29.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.270+0000 7f5eefe00640 1 --2- 172.21.15.86:0/363555838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ef00cf970 0x7f5ef0070e20 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0009870 tx=0x7f5ee001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3caa510230b073c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:29.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.270+0000 7f5eef400640 1 -- 172.21.15.86:0/363555838 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee00ab650 con 0x7f5ef00cf970 2024-04-25T14:08:29.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.270+0000 7f5eef400640 1 -- 172.21.15.86:0/363555838 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ee00b0070 con 0x7f5ef00cf970 2024-04-25T14:08:29.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.270+0000 7f5eef400640 1 -- 172.21.15.86:0/363555838 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee00aba10 con 0x7f5ef00cf970 2024-04-25T14:08:29.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.270+0000 7f5ef6e00640 1 -- 172.21.15.86:0/363555838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ef00cf970 msgr2=0x7f5ef0070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.270+0000 7f5ef6e00640 1 --2- 172.21.15.86:0/363555838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ef00cf970 0x7f5ef0070e20 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0009870 tx=0x7f5ee001c560 comp rx=0 tx=0).stop 2024-04-25T14:08:29.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.270+0000 7f5ef6e00640 1 -- 172.21.15.86:0/363555838 shutdown_connections 2024-04-25T14:08:29.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.270+0000 7f5ef6e00640 1 --2- 172.21.15.86:0/363555838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ef00cf970 0x7f5ef0070e20 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.270+0000 7f5ef6e00640 1 -- 172.21.15.86:0/363555838 >> 172.21.15.86:0/363555838 conn(0x7f5ef00cffc0 msgr2=0x7f5ef00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:29.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.270+0000 7f5ef6e00640 1 -- 172.21.15.86:0/363555838 shutdown_connections 2024-04-25T14:08:29.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.274+0000 7f5ef6e00640 1 -- 172.21.15.86:0/363555838 wait complete. 2024-04-25T14:08:29.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.274+0000 7f5ef6e00640 1 Processor -- start 2024-04-25T14:08:29.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.274+0000 7f5ef6e00640 1 -- start start 2024-04-25T14:08:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.274+0000 7f5ef6e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ef006e970 0x7f5ef0159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.274+0000 7f5ef6e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ef0071360 0x7f5ef014ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.274+0000 7f5ef6e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ef00cf970 0x7f5ef014f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.274+0000 7f5ef6e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef016df70 con 0x7f5ef006e970 2024-04-25T14:08:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.274+0000 7f5ef6e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef016ddf0 con 0x7f5ef0071360 2024-04-25T14:08:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.274+0000 7f5ef6e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5ef016e0f0 con 0x7f5ef00cf970 2024-04-25T14:08:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.278+0000 7f5eefe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ef006e970 0x7f5ef0159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.278+0000 7f5eefe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ef006e970 0x7f5ef0159c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43238/0 (socket says 172.21.15.86:43238) 2024-04-25T14:08:29.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.278+0000 7f5eefe00640 1 -- 172.21.15.86:0/4289780717 learned_addr learned my addr 172.21.15.86:0/4289780717 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:29.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.278+0000 7f5ee7e00640 1 --2- 172.21.15.86:0/4289780717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ef0071360 0x7f5ef014ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.278+0000 7f5ef4e00640 1 --2- 172.21.15.86:0/4289780717 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ef00cf970 0x7f5ef014f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.278+0000 7f5eefe00640 1 -- 172.21.15.86:0/4289780717 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ef00cf970 msgr2=0x7f5ef014f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.278+0000 7f5eefe00640 1 --2- 172.21.15.86:0/4289780717 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ef00cf970 0x7f5ef014f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.278+0000 7f5eefe00640 1 -- 172.21.15.86:0/4289780717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ef0071360 msgr2=0x7f5ef014ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.278+0000 7f5eefe00640 1 --2- 172.21.15.86:0/4289780717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ef0071360 0x7f5ef014ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.278+0000 7f5eefe00640 1 -- 172.21.15.86:0/4289780717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef014efc0 con 0x7f5ef006e970 2024-04-25T14:08:29.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.278+0000 7f5eefe00640 1 --2- 172.21.15.86:0/4289780717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ef006e970 0x7f5ef0159c40 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f5ee001a190 tx=0x7f5ee001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:29.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.278+0000 7f5eecc00640 1 -- 172.21.15.86:0/4289780717 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee00abe60 con 0x7f5ef006e970 2024-04-25T14:08:29.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.278+0000 7f5eecc00640 1 -- 172.21.15.86:0/4289780717 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ee00b4820 con 0x7f5ef006e970 2024-04-25T14:08:29.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.278+0000 7f5ef6e00640 1 -- 172.21.15.86:0/4289780717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef0158f60 con 0x7f5ef006e970 2024-04-25T14:08:29.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.278+0000 7f5eecc00640 1 -- 172.21.15.86:0/4289780717 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee00b4c10 con 0x7f5ef006e970 2024-04-25T14:08:29.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.278+0000 7f5ef6e00640 1 -- 172.21.15.86:0/4289780717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef00b4250 con 0x7f5ef006e970 2024-04-25T14:08:29.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.278+0000 7f5ef6e00640 1 -- 172.21.15.86:0/4289780717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ef00720e0 con 0x7f5ef006e970 2024-04-25T14:08:29.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.278+0000 7f5eecc00640 1 -- 172.21.15.86:0/4289780717 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 93309+0+0 (secure 0 0 0) 0x7f5ee00bd390 con 0x7f5ef006e970 2024-04-25T14:08:29.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.282+0000 7f5eecc00640 1 --2- 172.21.15.86:0/4289780717 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f5ecc06f550 0x7f5ecc071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.282+0000 7f5ee7e00640 1 -- 172.21.15.86:0/4289780717 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f5ecc06f550 msgr2=0x7f5ecc071a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/203029814 2024-04-25T14:08:29.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.282+0000 7f5eecc00640 1 -- 172.21.15.86:0/4289780717 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6531+0+0 (secure 0 0 0) 0x7f5ee01297d0 con 0x7f5ef006e970 2024-04-25T14:08:29.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.282+0000 7f5ee7e00640 1 --2- 172.21.15.86:0/4289780717 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f5ecc06f550 0x7f5ecc071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:08:29.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.286+0000 7f5eecc00640 1 -- 172.21.15.86:0/4289780717 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5ee0016150 con 0x7f5ef006e970 2024-04-25T14:08:29.327 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:29.326+0000 7f6d4ef21180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:08:29.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.338+0000 7f5eecc00640 1 -- 172.21.15.86:0/4289780717 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 25) ==== 92918+0+0 (secure 0 0 0) 0x7f5ee01279b0 con 0x7f5ef006e970 2024-04-25T14:08:29.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.338+0000 7f5eecc00640 1 -- 172.21.15.86:0/4289780717 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f5ecc06f550 msgr2=0x7f5ecc071a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:08:29.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.338+0000 7f5eecc00640 1 --2- 172.21.15.86:0/4289780717 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f5ecc06f550 0x7f5ecc071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.407 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:29.406+0000 7f6cd1400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:08:29.407 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:29.406+0000 7f6cd1400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:08:29.407 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:29.406+0000 7f6cd1400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:08:29.407 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:29.406+0000 7f6cd1400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:08:29.407 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:29.406+0000 7f6cd1400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:08:29.407 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:29.406+0000 7f6ccd800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:08:29.407 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:29.406+0000 7f6ccd800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:08:29.407 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:29.406+0000 7f6ccd800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:08:29.407 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:29.406+0000 7f6ccd800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:08:29.407 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:29.406+0000 7f6ccd800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:08:29.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.474+0000 7f5ef6e00640 1 -- 172.21.15.86:0/4289780717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5ef0076bf0 con 0x7f5ef006e970 2024-04-25T14:08:29.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.474+0000 7f5eecc00640 1 -- 172.21.15.86:0/4289780717 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 18 entries v76) ==== 81+0+1049 (secure 0 0 0) 0x7f5ee01110a0 con 0x7f5ef006e970 2024-04-25T14:08:29.478 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2066326917 2024-04-26T14:08:29.333458+0000 2024-04-25T14:08:29.478 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6833/203029814 2024-04-26T14:08:29.333458+0000 2024-04-25T14:08:29.478 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6832/203029814 2024-04-26T14:08:29.333458+0000 2024-04-25T14:08:29.478 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3991804772 2024-04-26T14:08:29.333458+0000 2024-04-25T14:08:29.478 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2307768874 2024-04-26T14:08:29.333458+0000 2024-04-25T14:08:29.479 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2510298865 2024-04-26T14:07:03.055680+0000 2024-04-25T14:08:29.479 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3380489666 2024-04-26T14:08:29.333458+0000 2024-04-25T14:08:29.479 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/82848493 2024-04-26T14:07:03.055680+0000 2024-04-25T14:08:29.479 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2452803132 2024-04-26T14:08:29.333458+0000 2024-04-25T14:08:29.479 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1793604306 2024-04-26T14:07:03.055680+0000 2024-04-25T14:08:29.479 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3402941646 2024-04-26T14:07:03.055680+0000 2024-04-25T14:08:29.479 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/112097751 2024-04-26T14:07:03.055680+0000 2024-04-25T14:08:29.479 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2567754949 2024-04-26T14:07:03.055680+0000 2024-04-25T14:08:29.479 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2334647603 2024-04-26T14:08:29.333458+0000 2024-04-25T14:08:29.479 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/529842883 2024-04-26T14:07:03.055680+0000 2024-04-25T14:08:29.479 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6833/1025829557 2024-04-26T14:07:03.055680+0000 2024-04-25T14:08:29.479 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6832/1025829557 2024-04-26T14:07:03.055680+0000 2024-04-25T14:08:29.479 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/759420784 2024-04-26T14:07:03.055680+0000 2024-04-25T14:08:29.480 INFO:teuthology.orchestra.run.smithi086.stderr:listed 18 entries 2024-04-25T14:08:29.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.478+0000 7f5ef6e00640 1 -- 172.21.15.86:0/4289780717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ef006e970 msgr2=0x7f5ef0159c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.478+0000 7f5ef6e00640 1 --2- 172.21.15.86:0/4289780717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ef006e970 0x7f5ef0159c40 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f5ee001a190 tx=0x7f5ee001ada0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.478+0000 7f5ef6e00640 1 -- 172.21.15.86:0/4289780717 shutdown_connections 2024-04-25T14:08:29.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.478+0000 7f5ef6e00640 1 --2- 172.21.15.86:0/4289780717 >> [v2:172.21.15.86:6832/203029814,v1:172.21.15.86:6833/203029814] conn(0x7f5ecc06f550 0x7f5ecc071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.478+0000 7f5ef6e00640 1 --2- 172.21.15.86:0/4289780717 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ef00cf970 0x7f5ef014f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.478+0000 7f5ef6e00640 1 --2- 172.21.15.86:0/4289780717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ef0071360 0x7f5ef014ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.478+0000 7f5ef6e00640 1 --2- 172.21.15.86:0/4289780717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ef006e970 0x7f5ef0159c40 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.478+0000 7f5ef6e00640 1 -- 172.21.15.86:0/4289780717 >> 172.21.15.86:0/4289780717 conn(0x7f5ef00cffc0 msgr2=0x7f5ef0068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:29.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.478+0000 7f5ef6e00640 1 -- 172.21.15.86:0/4289780717 shutdown_connections 2024-04-25T14:08:29.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.478+0000 7f5ef6e00640 1 -- 172.21.15.86:0/4289780717 wait complete. 2024-04-25T14:08:29.500 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T14:08:29.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:08:29.500 DEBUG:teuthology.orchestra.run.smithi155:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T14:08:29.526 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T14:08:29.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:08:29.526 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T14:08:29.578 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:08:29.578 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:08:29.578 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:08:29.587 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2893 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T14:08:29.587 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:08:29.587 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:08:29.587 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:08:29.633 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4517 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T14:08:29.634 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:08:29.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.778+0000 7f4075c00640 1 Processor -- start 2024-04-25T14:08:29.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.778+0000 7f4075c00640 1 -- start start 2024-04-25T14:08:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.782+0000 7f4075c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4070071530 0x7f407015dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.782+0000 7f4075c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f40701616e0 0x7f4070161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.782+0000 7f4075c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40700cd8d0 con 0x7f40700d2c80 2024-04-25T14:08:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.782+0000 7f4075c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f40700ce720 con 0x7f40701616e0 2024-04-25T14:08:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.782+0000 7f4075c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f40700ce970 con 0x7f4070071530 2024-04-25T14:08:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.782+0000 7f406ea00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f40700d2c80 0x7f4070070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:58386/0 (socket says 172.21.15.86:58386) 2024-04-25T14:08:29.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.782+0000 7f406ea00640 1 -- 172.21.15.86:0/4156010001 learned_addr learned my addr 172.21.15.86:0/4156010001 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:29.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.782+0000 7f406f400640 1 --2- 172.21.15.86:0/4156010001 >> v2:172.21.15.86:3300/0 conn(0x7f40701616e0 0x7f4070161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.782+0000 7f4067e00640 1 --2- 172.21.15.86:0/4156010001 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4070071530 0x7f407015dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.782+0000 7f406e000640 1 -- 172.21.15.86:0/4156010001 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3865753583 0 0) 0x7f40700cd8d0 con 0x7f40700d2c80 2024-04-25T14:08:29.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.782+0000 7f406e000640 1 -- 172.21.15.86:0/4156010001 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4054003530 con 0x7f40700d2c80 2024-04-25T14:08:29.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.782+0000 7f406f400640 1 -- 172.21.15.86:0/4156010001 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4070071530 msgr2=0x7f407015dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:29.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.782+0000 7f406f400640 1 --2- 172.21.15.86:0/4156010001 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4070071530 0x7f407015dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.782+0000 7f406f400640 1 -- 172.21.15.86:0/4156010001 >> v1:172.21.15.86:6789/0 conn(0x7f40700d2c80 legacy=0x7f4070070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.782+0000 7f406f400640 1 -- 172.21.15.86:0/4156010001 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40700d1be0 con 0x7f40701616e0 2024-04-25T14:08:29.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.782+0000 7f4067e00640 1 --2- 172.21.15.86:0/4156010001 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4070071530 0x7f407015dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:29.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.786+0000 7f406f400640 1 --2- 172.21.15.86:0/4156010001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40701616e0 0x7f4070161af0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f4068009d90 tx=0x7f406801f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d794c3bd7137fb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:29.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.786+0000 7f406e000640 1 -- 172.21.15.86:0/4156010001 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40680b4070 con 0x7f40701616e0 2024-04-25T14:08:29.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.786+0000 7f406e000640 1 -- 172.21.15.86:0/4156010001 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40680af600 con 0x7f40701616e0 2024-04-25T14:08:29.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.786+0000 7f406e000640 1 -- 172.21.15.86:0/4156010001 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40680af9b0 con 0x7f40701616e0 2024-04-25T14:08:29.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.786+0000 7f4075c00640 1 -- 172.21.15.86:0/4156010001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40701616e0 msgr2=0x7f4070161af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.786+0000 7f4075c00640 1 --2- 172.21.15.86:0/4156010001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40701616e0 0x7f4070161af0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f4068009d90 tx=0x7f406801f150 comp rx=0 tx=0).stop 2024-04-25T14:08:29.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.786+0000 7f4075c00640 1 -- 172.21.15.86:0/4156010001 shutdown_connections 2024-04-25T14:08:29.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.786+0000 7f4075c00640 1 --2- 172.21.15.86:0/4156010001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40701616e0 0x7f4070161af0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.786+0000 7f4075c00640 1 --2- 172.21.15.86:0/4156010001 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4070071530 0x7f407015dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.786+0000 7f4075c00640 1 -- 172.21.15.86:0/4156010001 >> 172.21.15.86:0/4156010001 conn(0x7f40700cffc0 msgr2=0x7f40700d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:29.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.786+0000 7f4075c00640 1 -- 172.21.15.86:0/4156010001 shutdown_connections 2024-04-25T14:08:29.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.786+0000 7f4075c00640 1 -- 172.21.15.86:0/4156010001 wait complete. 2024-04-25T14:08:29.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.786+0000 7f4075c00640 1 Processor -- start 2024-04-25T14:08:29.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.790+0000 7f4075c00640 1 -- start start 2024-04-25T14:08:29.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.790+0000 7f4075c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40700d2c80 0x7f4070160ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.790+0000 7f4075c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40701616e0 0x7f4070151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.790+0000 7f4075c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40700b3ff0 0x7f40700b4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.790+0000 7f4075c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f40700d2480 con 0x7f40700d2c80 2024-04-25T14:08:29.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.790+0000 7f4075c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f40700d21a0 con 0x7f40700b3ff0 2024-04-25T14:08:29.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.790+0000 7f4075c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f40700d29a0 con 0x7f40701616e0 2024-04-25T14:08:29.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.790+0000 7f406ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40700d2c80 0x7f4070160ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.790+0000 7f406ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40700d2c80 0x7f4070160ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43362/0 (socket says 172.21.15.86:43362) 2024-04-25T14:08:29.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.790+0000 7f406ea00640 1 -- 172.21.15.86:0/3306538184 learned_addr learned my addr 172.21.15.86:0/3306538184 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:29.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.790+0000 7f406f400640 1 --2- 172.21.15.86:0/3306538184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40700b3ff0 0x7f40700b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.790+0000 7f4067e00640 1 --2- 172.21.15.86:0/3306538184 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40701616e0 0x7f4070151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.790+0000 7f406ea00640 1 -- 172.21.15.86:0/3306538184 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40701616e0 msgr2=0x7f4070151a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.790+0000 7f406ea00640 1 --2- 172.21.15.86:0/3306538184 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40701616e0 0x7f4070151a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.790+0000 7f406ea00640 1 -- 172.21.15.86:0/3306538184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40700b3ff0 msgr2=0x7f40700b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.790+0000 7f406ea00640 1 --2- 172.21.15.86:0/3306538184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40700b3ff0 0x7f40700b4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.790+0000 7f406ea00640 1 -- 172.21.15.86:0/3306538184 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40700b1650 con 0x7f40700d2c80 2024-04-25T14:08:29.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.790+0000 7f406ea00640 1 --2- 172.21.15.86:0/3306538184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40700d2c80 0x7f4070160ab0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f406000c3f0 tx=0x7f406000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:29.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.790+0000 7f4066a00640 1 -- 172.21.15.86:0/3306538184 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4060017070 con 0x7f40700d2c80 2024-04-25T14:08:29.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.790+0000 7f4066a00640 1 -- 172.21.15.86:0/3306538184 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4060005cf0 con 0x7f40700d2c80 2024-04-25T14:08:29.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.790+0000 7f4075c00640 1 -- 172.21.15.86:0/3306538184 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40700b1930 con 0x7f40700d2c80 2024-04-25T14:08:29.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.790+0000 7f4066a00640 1 -- 172.21.15.86:0/3306538184 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40600072f0 con 0x7f40700d2c80 2024-04-25T14:08:29.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.790+0000 7f4075c00640 1 -- 172.21.15.86:0/3306538184 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f407015e470 con 0x7f40700d2c80 2024-04-25T14:08:29.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.790+0000 7f4075c00640 1 -- 172.21.15.86:0/3306538184 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4070076bf0 con 0x7f40700d2c80 2024-04-25T14:08:29.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.794+0000 7f4066a00640 1 -- 172.21.15.86:0/3306538184 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 92918+0+0 (secure 0 0 0) 0x7f406001a070 con 0x7f40700d2c80 2024-04-25T14:08:29.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.794+0000 7f4066a00640 1 -- 172.21.15.86:0/3306538184 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6875+0+0 (secure 0 0 0) 0x7f406008e640 con 0x7f40700d2c80 2024-04-25T14:08:29.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.798+0000 7f4066a00640 1 -- 172.21.15.86:0/3306538184 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f406005b4f0 con 0x7f40700d2c80 2024-04-25T14:08:29.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.990+0000 7f4075c00640 1 -- 172.21.15.86:0/3306538184 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40700d1470 con 0x7f40700d2c80 2024-04-25T14:08:29.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.990+0000 7f4066a00640 1 -- 172.21.15.86:0/3306538184 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5209 (secure 0 0 0) 0x7f406005adf0 con 0x7f40700d2c80 2024-04-25T14:08:29.993 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:08:29.993 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":57,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1210051860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1210051860},{"type":"v1","addr":"172.21.15.155:6833","nonce":1210051860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2868300438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2868300438},{"type":"v1","addr":"172.21.15.155:6835","nonce":2868300438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:06:44.910717+0000","modified":"2024-04-25T14:07:19.703362+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6603,"mds_1":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"c","rank":1,"incarnation":55,"state":"up:active","state_seq":9,"addr":"172.21.15.86:6835/2682115891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2682115891},{"type":"v1","addr":"172.21.15.86:6835","nonce":2682115891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6603":{"gid":6603,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/1088108173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1088108173},{"type":"v1","addr":"172.21.15.86:6837","nonce":1088108173}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6603,"qdb_cluster":[6603,6600]},"id":5}]} 2024-04-25T14:08:29.993 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 57 2024-04-25T14:08:29.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.994+0000 7f4075c00640 1 -- 172.21.15.86:0/3306538184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40700d2c80 msgr2=0x7f4070160ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.994+0000 7f4075c00640 1 --2- 172.21.15.86:0/3306538184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40700d2c80 0x7f4070160ab0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f406000c3f0 tx=0x7f406000c8b0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.994+0000 7f4075c00640 1 -- 172.21.15.86:0/3306538184 shutdown_connections 2024-04-25T14:08:29.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.994+0000 7f4075c00640 1 --2- 172.21.15.86:0/3306538184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40700b3ff0 0x7f40700b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.994+0000 7f4075c00640 1 --2- 172.21.15.86:0/3306538184 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40701616e0 0x7f4070151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.994+0000 7f4075c00640 1 --2- 172.21.15.86:0/3306538184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40700d2c80 0x7f4070160ab0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.994+0000 7f4075c00640 1 -- 172.21.15.86:0/3306538184 >> 172.21.15.86:0/3306538184 conn(0x7f40700cffc0 msgr2=0x7f407015cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:29.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.994+0000 7f4075c00640 1 -- 172.21.15.86:0/3306538184 shutdown_connections 2024-04-25T14:08:29.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:29.994+0000 7f4075c00640 1 -- 172.21.15.86:0/3306538184 wait complete. 2024-04-25T14:08:30.015 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:08:30.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.158+0000 7f115be00640 1 Processor -- start 2024-04-25T14:08:30.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.158+0000 7f115be00640 1 -- start start 2024-04-25T14:08:30.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.162+0000 7f115be00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f11540cfca0 0x7f115406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:30.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.162+0000 7f115be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f11540730b0 0x7f11540734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:30.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.162+0000 7f115be00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1154072bd0 con 0x7f11540d16e0 2024-04-25T14:08:30.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.162+0000 7f115be00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1154070470 con 0x7f11540730b0 2024-04-25T14:08:30.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.162+0000 7f115be00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f115406f930 con 0x7f11540cfca0 2024-04-25T14:08:30.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.162+0000 7f1159e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f11540730b0 0x7f11540734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:30.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.162+0000 7f1159400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f11540cfca0 0x7f115406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:30.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.162+0000 7f1159e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f11540730b0 0x7f11540734c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43374/0 (socket says 172.21.15.86:43374) 2024-04-25T14:08:30.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.162+0000 7f1159e00640 1 -- 172.21.15.86:0/2535173481 learned_addr learned my addr 172.21.15.86:0/2535173481 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:30.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.162+0000 7f1159e00640 1 -- 172.21.15.86:0/2535173481 >> v2:172.21.15.155:3300/0 conn(0x7f11540cfca0 msgr2=0x7f115406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:30.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.162+0000 7f1159e00640 1 --2- 172.21.15.86:0/2535173481 >> v2:172.21.15.155:3300/0 conn(0x7f11540cfca0 0x7f115406f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:30.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.162+0000 7f1159e00640 1 -- 172.21.15.86:0/2535173481 >> v1:172.21.15.155:6789/0 conn(0x7f11540d16e0 legacy=0x7f11540d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:30.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.162+0000 7f1159e00640 1 -- 172.21.15.86:0/2535173481 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f115406fbd0 con 0x7f11540730b0 2024-04-25T14:08:30.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.166+0000 7f1159e00640 1 --2- 172.21.15.86:0/2535173481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11540730b0 0x7f11540734c0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f11540d22a0 tx=0x7f114c01ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a181662fe6b48b15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:30.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.166+0000 7f1153e00640 1 -- 172.21.15.86:0/2535173481 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c00b2d0 con 0x7f11540730b0 2024-04-25T14:08:30.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.166+0000 7f1153e00640 1 -- 172.21.15.86:0/2535173481 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f114c00b470 con 0x7f11540730b0 2024-04-25T14:08:30.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.166+0000 7f1153e00640 1 -- 172.21.15.86:0/2535173481 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c0a7bf0 con 0x7f11540730b0 2024-04-25T14:08:30.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.166+0000 7f115be00640 1 -- 172.21.15.86:0/2535173481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11540730b0 msgr2=0x7f11540734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:30.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.166+0000 7f115be00640 1 --2- 172.21.15.86:0/2535173481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11540730b0 0x7f11540734c0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f11540d22a0 tx=0x7f114c01ebf0 comp rx=0 tx=0).stop 2024-04-25T14:08:30.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.166+0000 7f115be00640 1 -- 172.21.15.86:0/2535173481 shutdown_connections 2024-04-25T14:08:30.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.166+0000 7f115be00640 1 --2- 172.21.15.86:0/2535173481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11540730b0 0x7f11540734c0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:30.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.166+0000 7f115be00640 1 --2- 172.21.15.86:0/2535173481 >> v2:172.21.15.155:3300/0 conn(0x7f11540cfca0 0x7f115406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:30.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.166+0000 7f115be00640 1 -- 172.21.15.86:0/2535173481 >> 172.21.15.86:0/2535173481 conn(0x7f11540b4650 msgr2=0x7f11540b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:30.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.166+0000 7f115be00640 1 -- 172.21.15.86:0/2535173481 shutdown_connections 2024-04-25T14:08:30.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.166+0000 7f115be00640 1 -- 172.21.15.86:0/2535173481 wait complete. 2024-04-25T14:08:30.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.170+0000 7f115be00640 1 Processor -- start 2024-04-25T14:08:30.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.170+0000 7f115be00640 1 -- start start 2024-04-25T14:08:30.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.170+0000 7f115be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11540730b0 0x7f1154159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:30.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.170+0000 7f115be00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11540cfca0 0x7f115415f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:30.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.170+0000 7f115be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11540d16e0 0x7f115415f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:30.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.170+0000 7f115be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11540ce0f0 con 0x7f11540730b0 2024-04-25T14:08:30.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.170+0000 7f115be00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f11540cdf70 con 0x7f11540d16e0 2024-04-25T14:08:30.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.170+0000 7f115be00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f11540ce270 con 0x7f11540cfca0 2024-04-25T14:08:30.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.170+0000 7f1159400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11540730b0 0x7f1154159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:30.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.170+0000 7f1159400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11540730b0 0x7f1154159130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43390/0 (socket says 172.21.15.86:43390) 2024-04-25T14:08:30.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.170+0000 7f1159400640 1 -- 172.21.15.86:0/854004894 learned_addr learned my addr 172.21.15.86:0/854004894 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:30.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.170+0000 7f1158a00640 1 --2- 172.21.15.86:0/854004894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11540cfca0 0x7f115415f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:30.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.170+0000 7f1159e00640 1 --2- 172.21.15.86:0/854004894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11540d16e0 0x7f115415f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:30.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.170+0000 7f1159400640 1 -- 172.21.15.86:0/854004894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11540cfca0 msgr2=0x7f115415f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:30.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.170+0000 7f1159400640 1 --2- 172.21.15.86:0/854004894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11540cfca0 0x7f115415f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:30.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.170+0000 7f1159400640 1 -- 172.21.15.86:0/854004894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11540d16e0 msgr2=0x7f115415f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:30.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.170+0000 7f1159400640 1 --2- 172.21.15.86:0/854004894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11540d16e0 0x7f115415f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:30.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.170+0000 7f1159400640 1 -- 172.21.15.86:0/854004894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11541f5670 con 0x7f11540730b0 2024-04-25T14:08:30.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.170+0000 7f1158a00640 1 --2- 172.21.15.86:0/854004894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11540cfca0 0x7f115415f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:08:30.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.170+0000 7f1159e00640 1 --2- 172.21.15.86:0/854004894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11540d16e0 0x7f115415f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:08:30.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.170+0000 7f1159400640 1 --2- 172.21.15.86:0/854004894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11540730b0 0x7f1154159130 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f114800e840 tx=0x7f114800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:30.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.170+0000 7f1151600640 1 -- 172.21.15.86:0/854004894 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1148017070 con 0x7f11540730b0 2024-04-25T14:08:30.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.170+0000 7f1151600640 1 -- 172.21.15.86:0/854004894 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f114800ee90 con 0x7f11540730b0 2024-04-25T14:08:30.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.170+0000 7f115be00640 1 -- 172.21.15.86:0/854004894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11540cb0a0 con 0x7f11540730b0 2024-04-25T14:08:30.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.170+0000 7f1151600640 1 -- 172.21.15.86:0/854004894 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114800cb60 con 0x7f11540730b0 2024-04-25T14:08:30.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.170+0000 7f115be00640 1 -- 172.21.15.86:0/854004894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1154159670 con 0x7f11540730b0 2024-04-25T14:08:30.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.174+0000 7f115be00640 1 -- 172.21.15.86:0/854004894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11541f5840 con 0x7f11540730b0 2024-04-25T14:08:30.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.174+0000 7f1151600640 1 -- 172.21.15.86:0/854004894 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 92918+0+0 (secure 0 0 0) 0x7f114801a070 con 0x7f11540730b0 2024-04-25T14:08:30.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.174+0000 7f1151600640 1 -- 172.21.15.86:0/854004894 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6875+0+0 (secure 0 0 0) 0x7f114808efe0 con 0x7f11540730b0 2024-04-25T14:08:30.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.178+0000 7f1151600640 1 -- 172.21.15.86:0/854004894 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f114805b820 con 0x7f11540730b0 2024-04-25T14:08:30.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.338+0000 7f1151600640 1 -- 172.21.15.86:0/854004894 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 26) ==== 93045+0+0 (secure 0 0 0) 0x7f114800cd00 con 0x7f11540730b0 2024-04-25T14:08:30.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.338+0000 7f1151600640 1 --2- 172.21.15.86:0/854004894 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f1134075280 0x7f11340776a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:30.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.338+0000 7f1158a00640 1 --2- 172.21.15.86:0/854004894 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f1134075280 0x7f11340776a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:30.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.342+0000 7f1158a00640 1 --2- 172.21.15.86:0/854004894 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f1134075280 0x7f11340776a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f115409c830 tx=0x7f1140009040 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:30.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.342+0000 7f115be00640 1 -- 172.21.15.86:0/854004894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f115409f4b0 con 0x7f11540730b0 2024-04-25T14:08:30.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.346+0000 7f1151600640 1 -- 172.21.15.86:0/854004894 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+18065 (secure 0 0 0) 0x7f11480798e0 con 0x7f11540730b0 2024-04-25T14:08:30.348 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:08:30.349 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":76,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:08:29.333499+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:06:42.666742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T14:06:43.884074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2066326917":"2024-04-26T14:08:29.333458+0000","172.21.15.86:6833/203029814":"2024-04-26T14:08:29.333458+0000","172.21.15.86:6832/203029814":"2024-04-26T14:08:29.333458+0000","172.21.15.86:0/3991804772":"2024-04-26T14:08:29.333458+0000","172.21.15.86:0/2307768874":"2024-04-26T14:08:29.333458+0000","172.21.15.86:0/2510298865":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/3380489666":"2024-04-26T14:08:29.333458+0000","172.21.15.86:0/82848493":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/2452803132":"2024-04-26T14:08:29.333458+0000","172.21.15.86:0/1793604306":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/3402941646":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/112097751":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/2567754949":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/2334647603":"2024-04-26T14:08:29.333458+0000","172.21.15.86:0/529842883":"2024-04-26T14:07:03.055680+0000","172.21.15.86:6833/1025829557":"2024-04-26T14:07:03.055680+0000","172.21.15.86:6832/1025829557":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/759420784":"2024-04-26T14:07:03.055680+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:08:30.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.350+0000 7f115be00640 1 -- 172.21.15.86:0/854004894 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f1134075280 msgr2=0x7f11340776a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:30.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.350+0000 7f115be00640 1 --2- 172.21.15.86:0/854004894 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f1134075280 0x7f11340776a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f115409c830 tx=0x7f1140009040 comp rx=0 tx=0).stop 2024-04-25T14:08:30.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.350+0000 7f115be00640 1 -- 172.21.15.86:0/854004894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11540730b0 msgr2=0x7f1154159130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:30.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.350+0000 7f115be00640 1 --2- 172.21.15.86:0/854004894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11540730b0 0x7f1154159130 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f114800e840 tx=0x7f114800ec40 comp rx=0 tx=0).stop 2024-04-25T14:08:30.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.350+0000 7f115be00640 1 -- 172.21.15.86:0/854004894 shutdown_connections 2024-04-25T14:08:30.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.350+0000 7f115be00640 1 --2- 172.21.15.86:0/854004894 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f1134075280 0x7f11340776a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:30.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.350+0000 7f115be00640 1 --2- 172.21.15.86:0/854004894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11540d16e0 0x7f115415f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:30.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.350+0000 7f115be00640 1 --2- 172.21.15.86:0/854004894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11540cfca0 0x7f115415f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:30.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.350+0000 7f115be00640 1 --2- 172.21.15.86:0/854004894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11540730b0 0x7f1154159130 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:30.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.350+0000 7f115be00640 1 -- 172.21.15.86:0/854004894 >> 172.21.15.86:0/854004894 conn(0x7f11540b4650 msgr2=0x7f11540b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:30.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.350+0000 7f115be00640 1 -- 172.21.15.86:0/854004894 shutdown_connections 2024-04-25T14:08:30.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.350+0000 7f115be00640 1 -- 172.21.15.86:0/854004894 wait complete. 2024-04-25T14:08:30.373 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:06:42.666742+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:06:43.884074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:08:30.374 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:08:30.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.522+0000 7fd5f7600640 1 Processor -- start 2024-04-25T14:08:30.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.522+0000 7fd5f7600640 1 -- start start 2024-04-25T14:08:30.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.522+0000 7fd5f7600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5f00d16e0 0x7fd5f00d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:30.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.522+0000 7fd5f7600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5f00d2030 con 0x7fd5f00730b0 2024-04-25T14:08:30.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.522+0000 7fd5f7600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5f0072bd0 con 0x7fd5f00cfca0 2024-04-25T14:08:30.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.522+0000 7fd5f7600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd5f0070470 con 0x7fd5f00d16e0 2024-04-25T14:08:30.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.522+0000 7fd5f5600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd5f00730b0 0x7fd5f00734c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:58390/0 (socket says 172.21.15.86:58390) 2024-04-25T14:08:30.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.522+0000 7fd5f5600640 1 -- 172.21.15.86:0/927527529 learned_addr learned my addr 172.21.15.86:0/927527529 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:30.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.522+0000 7fd5efe00640 1 --2- 172.21.15.86:0/927527529 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5f00d16e0 0x7fd5f00d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:30.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.522+0000 7fd5ef400640 1 -- 172.21.15.86:0/927527529 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2196335293 0 0) 0x7fd5e8001120 con 0x7fd5f00730b0 2024-04-25T14:08:30.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.522+0000 7fd5ef400640 1 -- 172.21.15.86:0/927527529 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5d4003530 con 0x7fd5f00730b0 2024-04-25T14:08:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.522+0000 7fd5ef400640 1 -- 172.21.15.86:0/927527529 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3359978065 0 0) 0x7fd5d4003530 con 0x7fd5f00730b0 2024-04-25T14:08:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.522+0000 7fd5ef400640 1 -- 172.21.15.86:0/927527529 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5f00d16e0 msgr2=0x7fd5f00d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.522+0000 7fd5ef400640 1 --2- 172.21.15.86:0/927527529 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5f00d16e0 0x7fd5f00d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.522+0000 7fd5ef400640 1 -- 172.21.15.86:0/927527529 >> v1:172.21.15.155:6789/0 conn(0x7fd5f00cfca0 legacy=0x7fd5f006f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.522+0000 7fd5ef400640 1 -- 172.21.15.86:0/927527529 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f006fbd0 con 0x7fd5f00730b0 2024-04-25T14:08:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.522+0000 7fd5ef400640 1 -- 172.21.15.86:0/927527529 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fd5e8001850 con 0x7fd5f00730b0 2024-04-25T14:08:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.526+0000 7fd5ef400640 1 -- 172.21.15.86:0/927527529 >> v1:172.21.15.86:6789/0 conn(0x7fd5f00730b0 legacy=0x7fd5f00734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.526+0000 7fd5ef400640 1 --2- 172.21.15.86:0/927527529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5d4003a60 0x7fd5d4003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.526+0000 7fd5ef400640 1 --2- 172.21.15.86:0/927527529 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5f00d16e0 0x7fd5d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.526+0000 7fd5ef400640 1 --2- 172.21.15.86:0/927527529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5d4004d50 0x7fd5d4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.526+0000 7fd5ef400640 1 -- 172.21.15.86:0/927527529 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f0070470 con 0x7fd5d4003a60 2024-04-25T14:08:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.526+0000 7fd5ef400640 1 -- 172.21.15.86:0/927527529 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d400bb00 con 0x7fd5d4004d50 2024-04-25T14:08:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.526+0000 7fd5ef400640 1 -- 172.21.15.86:0/927527529 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd5d400bd00 con 0x7fd5f00d16e0 2024-04-25T14:08:30.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.526+0000 7fd5f4c00640 1 --2- 172.21.15.86:0/927527529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5d4003a60 0x7fd5d4003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.526+0000 7fd5efe00640 1 --2- 172.21.15.86:0/927527529 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5f00d16e0 0x7fd5d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.526+0000 7fd5f5600640 1 --2- 172.21.15.86:0/927527529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5d4004d50 0x7fd5d4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.526+0000 7fd5f4c00640 1 -- 172.21.15.86:0/927527529 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5f00d16e0 msgr2=0x7fd5d4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.526+0000 7fd5f4c00640 1 --2- 172.21.15.86:0/927527529 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5f00d16e0 0x7fd5d4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.526+0000 7fd5f4c00640 1 -- 172.21.15.86:0/927527529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5d4004d50 msgr2=0x7fd5d4005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.526+0000 7fd5f4c00640 1 --2- 172.21.15.86:0/927527529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5d4004d50 0x7fd5d4005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.526+0000 7fd5f4c00640 1 -- 172.21.15.86:0/927527529 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d4003530 con 0x7fd5d4003a60 2024-04-25T14:08:30.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.526+0000 7fd5f4c00640 1 --2- 172.21.15.86:0/927527529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5d4003a60 0x7fd5d4003eb0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0070710 tx=0x7fd5e001c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f8169c4d48f6ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:30.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.526+0000 7fd5ef400640 1 -- 172.21.15.86:0/927527529 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0001c10 con 0x7fd5d4003a60 2024-04-25T14:08:30.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.526+0000 7fd5ef400640 1 -- 172.21.15.86:0/927527529 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5e00a6630 con 0x7fd5d4003a60 2024-04-25T14:08:30.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.526+0000 7fd5ef400640 1 -- 172.21.15.86:0/927527529 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f00743e0 con 0x7fd5d4003a60 2024-04-25T14:08:30.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.530+0000 7fd5f7600640 1 -- 172.21.15.86:0/927527529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5d4003a60 msgr2=0x7fd5d4003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:30.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.530+0000 7fd5f7600640 1 --2- 172.21.15.86:0/927527529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5d4003a60 0x7fd5d4003eb0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0070710 tx=0x7fd5e001c820 comp rx=0 tx=0).stop 2024-04-25T14:08:30.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.530+0000 7fd5f5600640 1 -- 172.21.15.86:0/927527529 reap_dead start 2024-04-25T14:08:30.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.530+0000 7fd5f7600640 1 -- 172.21.15.86:0/927527529 shutdown_connections 2024-04-25T14:08:30.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.530+0000 7fd5f7600640 1 -- 172.21.15.86:0/927527529 >> 172.21.15.86:0/927527529 conn(0x7fd5f00b4650 msgr2=0x7fd5f00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:30.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.530+0000 7fd5f7600640 1 -- 172.21.15.86:0/927527529 shutdown_connections 2024-04-25T14:08:30.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.530+0000 7fd5f7600640 1 -- 172.21.15.86:0/927527529 wait complete. 2024-04-25T14:08:30.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.530+0000 7fd5f7600640 1 Processor -- start 2024-04-25T14:08:30.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.530+0000 7fd5f7600640 1 -- start start 2024-04-25T14:08:30.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.534+0000 7fd5f7600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5d4003a60 0x7fd5f009a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:30.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.534+0000 7fd5f7600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5f0162d90 0x7fd5f015a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:30.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.534+0000 7fd5f7600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5f01631a0 0x7fd5f015a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:30.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.534+0000 7fd5f7600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f0168a70 con 0x7fd5f0162d90 2024-04-25T14:08:30.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.534+0000 7fd5f7600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f01688f0 con 0x7fd5f01631a0 2024-04-25T14:08:30.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.534+0000 7fd5f7600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd5f0168bf0 con 0x7fd5d4003a60 2024-04-25T14:08:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.534+0000 7fd5efe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5f0162d90 0x7fd5f015a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.534+0000 7fd5efe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5f0162d90 0x7fd5f015a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43404/0 (socket says 172.21.15.86:43404) 2024-04-25T14:08:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.534+0000 7fd5efe00640 1 -- 172.21.15.86:0/274860854 learned_addr learned my addr 172.21.15.86:0/274860854 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.534+0000 7fd5f5600640 1 --2- 172.21.15.86:0/274860854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5f01631a0 0x7fd5f015a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.534+0000 7fd5f4c00640 1 --2- 172.21.15.86:0/274860854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5d4003a60 0x7fd5f009a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.534+0000 7fd5efe00640 1 -- 172.21.15.86:0/274860854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5d4003a60 msgr2=0x7fd5f009a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.534+0000 7fd5efe00640 1 --2- 172.21.15.86:0/274860854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5d4003a60 0x7fd5f009a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.534+0000 7fd5efe00640 1 -- 172.21.15.86:0/274860854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5f01631a0 msgr2=0x7fd5f015a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.534+0000 7fd5efe00640 1 --2- 172.21.15.86:0/274860854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5f01631a0 0x7fd5f015a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.534+0000 7fd5efe00640 1 -- 172.21.15.86:0/274860854 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f0161880 con 0x7fd5f0162d90 2024-04-25T14:08:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.534+0000 7fd5efe00640 1 --2- 172.21.15.86:0/274860854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5f0162d90 0x7fd5f015a250 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0119d0 tx=0x7fd5dc011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.534+0000 7fd5ecc00640 1 -- 172.21.15.86:0/274860854 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc00fb50 con 0x7fd5f0162d90 2024-04-25T14:08:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.534+0000 7fd5ecc00640 1 -- 172.21.15.86:0/274860854 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5dc00fcf0 con 0x7fd5f0162d90 2024-04-25T14:08:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.534+0000 7fd5ecc00640 1 -- 172.21.15.86:0/274860854 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc00fea0 con 0x7fd5f0162d90 2024-04-25T14:08:30.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.534+0000 7fd5f7600640 1 -- 172.21.15.86:0/274860854 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5f0161b60 con 0x7fd5f0162d90 2024-04-25T14:08:30.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.534+0000 7fd5f7600640 1 -- 172.21.15.86:0/274860854 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5f0162120 con 0x7fd5f0162d90 2024-04-25T14:08:30.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.534+0000 7fd5f7600640 1 -- 172.21.15.86:0/274860854 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5f009f4b0 con 0x7fd5f0162d90 2024-04-25T14:08:30.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.538+0000 7fd5ecc00640 1 -- 172.21.15.86:0/274860854 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 93045+0+0 (secure 0 0 0) 0x7fd5dc012040 con 0x7fd5f0162d90 2024-04-25T14:08:30.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.538+0000 7fd5ecc00640 1 --2- 172.21.15.86:0/274860854 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fd5d006f250 0x7fd5d0071760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:30.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.538+0000 7fd5ecc00640 1 -- 172.21.15.86:0/274860854 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6875+0+0 (secure 0 0 0) 0x7fd5dc08f7f0 con 0x7fd5f0162d90 2024-04-25T14:08:30.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.538+0000 7fd5f4c00640 1 --2- 172.21.15.86:0/274860854 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fd5d006f250 0x7fd5d0071760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:30.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.538+0000 7fd5f4c00640 1 --2- 172.21.15.86:0/274860854 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fd5d006f250 0x7fd5d0071760 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00ad000 tx=0x7fd5e00b2040 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:30.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.542+0000 7fd5ecc00640 1 -- 172.21.15.86:0/274860854 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd5dc019030 con 0x7fd5f0162d90 2024-04-25T14:08:30.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.698+0000 7fd5f7600640 1 -- 172.21.15.86:0/274860854 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd5f0069d00 con 0x7fd5f0162d90 2024-04-25T14:08:30.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.698+0000 7fd5ecc00640 1 -- 172.21.15.86:0/274860854 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) ==== 93+0+10 (secure 0 0 0) 0x7fd5dc05c000 con 0x7fd5f0162d90 2024-04-25T14:08:30.701 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:08:30.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.702+0000 7fd5f7600640 1 -- 172.21.15.86:0/274860854 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fd5d006f250 msgr2=0x7fd5d0071760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:30.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.702+0000 7fd5f7600640 1 --2- 172.21.15.86:0/274860854 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fd5d006f250 0x7fd5d0071760 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00ad000 tx=0x7fd5e00b2040 comp rx=0 tx=0).stop 2024-04-25T14:08:30.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.702+0000 7fd5f7600640 1 -- 172.21.15.86:0/274860854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5f0162d90 msgr2=0x7fd5f015a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:30.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.702+0000 7fd5f7600640 1 --2- 172.21.15.86:0/274860854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5f0162d90 0x7fd5f015a250 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0119d0 tx=0x7fd5dc011e90 comp rx=0 tx=0).stop 2024-04-25T14:08:30.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.702+0000 7fd5f7600640 1 -- 172.21.15.86:0/274860854 shutdown_connections 2024-04-25T14:08:30.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.702+0000 7fd5f7600640 1 --2- 172.21.15.86:0/274860854 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fd5d006f250 0x7fd5d0071760 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:30.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.702+0000 7fd5f7600640 1 --2- 172.21.15.86:0/274860854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5f01631a0 0x7fd5f015a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:30.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.702+0000 7fd5f7600640 1 --2- 172.21.15.86:0/274860854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5f0162d90 0x7fd5f015a250 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:30.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.702+0000 7fd5f7600640 1 --2- 172.21.15.86:0/274860854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5d4003a60 0x7fd5f009a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:30.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.702+0000 7fd5f7600640 1 -- 172.21.15.86:0/274860854 >> 172.21.15.86:0/274860854 conn(0x7fd5f00b4650 msgr2=0x7fd5f015c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:30.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.702+0000 7fd5f7600640 1 -- 172.21.15.86:0/274860854 shutdown_connections 2024-04-25T14:08:30.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.702+0000 7fd5f7600640 1 -- 172.21.15.86:0/274860854 wait complete. 2024-04-25T14:08:30.723 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:08:30.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.870+0000 7fa0e7a00640 1 Processor -- start 2024-04-25T14:08:30.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.870+0000 7fa0e7a00640 1 -- start start 2024-04-25T14:08:30.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.874+0000 7fa0e7a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa0e00d1df0 0x7fa0e00cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:30.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.874+0000 7fa0e7a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0e00cdcc0 0x7fa0e00ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:30.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.874+0000 7fa0e7a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0e015e5c0 con 0x7fa0e0070330 2024-04-25T14:08:30.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.874+0000 7fa0e7a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa0e006fd40 con 0x7fa0e00d1df0 2024-04-25T14:08:30.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.874+0000 7fa0e7a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa0e015e8a0 con 0x7fa0e00cdcc0 2024-04-25T14:08:30.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.874+0000 7fa0d7e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa0e00d1df0 0x7fa0e00cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:30.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.874+0000 7fa0d7e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa0e00d1df0 0x7fa0e00cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:42258/0 (socket says 172.21.15.86:42258) 2024-04-25T14:08:30.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.874+0000 7fa0d7e00640 1 -- 172.21.15.86:0/3023769478 learned_addr learned my addr 172.21.15.86:0/3023769478 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:30.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.874+0000 7fa0e5000640 1 --1- 172.21.15.86:0/3023769478 >> v1:172.21.15.155:6789/0 conn(0x7fa0e0070330 0x7fa0e00d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:47894/0 (socket says 172.21.15.86:47894) 2024-04-25T14:08:30.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.874+0000 7fa0e5a00640 1 --2- 172.21.15.86:0/3023769478 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0e00cdcc0 0x7fa0e00ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:30.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.874+0000 7fa0d7e00640 1 -- 172.21.15.86:0/3023769478 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0e00cdcc0 msgr2=0x7fa0e00ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:30.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.874+0000 7fa0d7e00640 1 --2- 172.21.15.86:0/3023769478 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0e00cdcc0 0x7fa0e00ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:30.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.874+0000 7fa0d7e00640 1 -- 172.21.15.86:0/3023769478 >> v1:172.21.15.155:6789/0 conn(0x7fa0e0070330 legacy=0x7fa0e00d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:30.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.874+0000 7fa0d7e00640 1 -- 172.21.15.86:0/3023769478 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e015eb80 con 0x7fa0e00d1df0 2024-04-25T14:08:30.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.878+0000 7fa0d7e00640 1 --2- 172.21.15.86:0/3023769478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0e00d1df0 0x7fa0e00cd780 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8009870 tx=0x7fa0c801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4542254ade3534ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:30.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.878+0000 7fa0dfe00640 1 -- 172.21.15.86:0/3023769478 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c80ab680 con 0x7fa0e00d1df0 2024-04-25T14:08:30.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.878+0000 7fa0dfe00640 1 -- 172.21.15.86:0/3023769478 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0c80b0070 con 0x7fa0e00d1df0 2024-04-25T14:08:30.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.878+0000 7fa0dfe00640 1 -- 172.21.15.86:0/3023769478 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c80aba40 con 0x7fa0e00d1df0 2024-04-25T14:08:30.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.878+0000 7fa0e7a00640 1 -- 172.21.15.86:0/3023769478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0e00d1df0 msgr2=0x7fa0e00cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:30.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.878+0000 7fa0e7a00640 1 --2- 172.21.15.86:0/3023769478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0e00d1df0 0x7fa0e00cd780 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8009870 tx=0x7fa0c801c4a0 comp rx=0 tx=0).stop 2024-04-25T14:08:30.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.878+0000 7fa0e7a00640 1 -- 172.21.15.86:0/3023769478 shutdown_connections 2024-04-25T14:08:30.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.878+0000 7fa0e7a00640 1 --2- 172.21.15.86:0/3023769478 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0e00cdcc0 0x7fa0e00ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:30.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.878+0000 7fa0e7a00640 1 --2- 172.21.15.86:0/3023769478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0e00d1df0 0x7fa0e00cd780 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:30.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.878+0000 7fa0e7a00640 1 -- 172.21.15.86:0/3023769478 >> 172.21.15.86:0/3023769478 conn(0x7fa0e00b4650 msgr2=0x7fa0e00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:30.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.878+0000 7fa0e7a00640 1 -- 172.21.15.86:0/3023769478 shutdown_connections 2024-04-25T14:08:30.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.878+0000 7fa0e7a00640 1 -- 172.21.15.86:0/3023769478 wait complete. 2024-04-25T14:08:30.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.878+0000 7fa0e7a00640 1 Processor -- start 2024-04-25T14:08:30.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.878+0000 7fa0e7a00640 1 -- start start 2024-04-25T14:08:30.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.882+0000 7fa0e7a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0e0070330 0x7fa0e00cff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:30.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.882+0000 7fa0e7a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0e00cdcc0 0x7fa0e00c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:30.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.882+0000 7fa0e7a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e015cfb0 0x7fa0e015d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:30.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.882+0000 7fa0e7a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e015f4b0 con 0x7fa0e015cfb0 2024-04-25T14:08:30.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.882+0000 7fa0e7a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e015f330 con 0x7fa0e00cdcc0 2024-04-25T14:08:30.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.882+0000 7fa0e7a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa0e0070740 con 0x7fa0e0070330 2024-04-25T14:08:30.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.882+0000 7fa0e5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e015cfb0 0x7fa0e015d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:30.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.882+0000 7fa0e5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e015cfb0 0x7fa0e015d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43414/0 (socket says 172.21.15.86:43414) 2024-04-25T14:08:30.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.882+0000 7fa0e5a00640 1 -- 172.21.15.86:0/2863922008 learned_addr learned my addr 172.21.15.86:0/2863922008 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:30.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.882+0000 7fa0e5000640 1 --2- 172.21.15.86:0/2863922008 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0e0070330 0x7fa0e00cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:30.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.882+0000 7fa0d7e00640 1 --2- 172.21.15.86:0/2863922008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0e00cdcc0 0x7fa0e00c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:30.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.882+0000 7fa0e5a00640 1 -- 172.21.15.86:0/2863922008 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0e0070330 msgr2=0x7fa0e00cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:30.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.882+0000 7fa0e5a00640 1 --2- 172.21.15.86:0/2863922008 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0e0070330 0x7fa0e00cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:30.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.882+0000 7fa0e5a00640 1 -- 172.21.15.86:0/2863922008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0e00cdcc0 msgr2=0x7fa0e00c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:30.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.882+0000 7fa0e5a00640 1 --2- 172.21.15.86:0/2863922008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0e00cdcc0 0x7fa0e00c7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:30.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.882+0000 7fa0e5a00640 1 -- 172.21.15.86:0/2863922008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e014b500 con 0x7fa0e015cfb0 2024-04-25T14:08:30.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.882+0000 7fa0e5a00640 1 --2- 172.21.15.86:0/2863922008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e015cfb0 0x7fa0e015d3c0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800e7f0 tx=0x7fa0d800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:30.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.882+0000 7fa0dd600640 1 -- 172.21.15.86:0/2863922008 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8017070 con 0x7fa0e015cfb0 2024-04-25T14:08:30.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.882+0000 7fa0dd600640 1 -- 172.21.15.86:0/2863922008 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0d800c940 con 0x7fa0e015cfb0 2024-04-25T14:08:30.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.882+0000 7fa0dd600640 1 -- 172.21.15.86:0/2863922008 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d800eea0 con 0x7fa0e015cfb0 2024-04-25T14:08:30.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.882+0000 7fa0e7a00640 1 -- 172.21.15.86:0/2863922008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0e015d900 con 0x7fa0e015cfb0 2024-04-25T14:08:30.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.882+0000 7fa0e7a00640 1 -- 172.21.15.86:0/2863922008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0e00c8800 con 0x7fa0e015cfb0 2024-04-25T14:08:30.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.882+0000 7fa0e7a00640 1 -- 172.21.15.86:0/2863922008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0e00c8290 con 0x7fa0e015cfb0 2024-04-25T14:08:30.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.886+0000 7fa0dd600640 1 -- 172.21.15.86:0/2863922008 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 93045+0+0 (secure 0 0 0) 0x7fa0d801a070 con 0x7fa0e015cfb0 2024-04-25T14:08:30.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.886+0000 7fa0dd600640 1 --2- 172.21.15.86:0/2863922008 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fa0bc06f280 0x7fa0bc071790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:30.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.886+0000 7fa0dd600640 1 -- 172.21.15.86:0/2863922008 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6875+0+0 (secure 0 0 0) 0x7fa0d808f240 con 0x7fa0e015cfb0 2024-04-25T14:08:30.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.886+0000 7fa0e5000640 1 --2- 172.21.15.86:0/2863922008 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fa0bc06f280 0x7fa0bc071790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:30.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.886+0000 7fa0e5000640 1 --2- 172.21.15.86:0/2863922008 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fa0bc06f280 0x7fa0bc071790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0007fd0 tx=0x7fa0d0005b80 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:30.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:30.890+0000 7fa0dd600640 1 -- 172.21.15.86:0/2863922008 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa0e015d900 con 0x7fa0e015cfb0 2024-04-25T14:08:31.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.050+0000 7fa0e7a00640 1 -- 172.21.15.86:0/2863922008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa0e0069d00 con 0x7fa0e015cfb0 2024-04-25T14:08:31.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.050+0000 7fa0dd600640 1 -- 172.21.15.86:0/2863922008 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v76) ==== 104+0+11 (secure 0 0 0) 0x7fa0e014b500 con 0x7fa0e015cfb0 2024-04-25T14:08:31.051 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:08:31.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.050+0000 7fa0e7a00640 1 -- 172.21.15.86:0/2863922008 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fa0bc06f280 msgr2=0x7fa0bc071790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:31.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.050+0000 7fa0e7a00640 1 --2- 172.21.15.86:0/2863922008 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fa0bc06f280 0x7fa0bc071790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0007fd0 tx=0x7fa0d0005b80 comp rx=0 tx=0).stop 2024-04-25T14:08:31.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.050+0000 7fa0e7a00640 1 -- 172.21.15.86:0/2863922008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e015cfb0 msgr2=0x7fa0e015d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:31.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.050+0000 7fa0e7a00640 1 --2- 172.21.15.86:0/2863922008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e015cfb0 0x7fa0e015d3c0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800e7f0 tx=0x7fa0d800ecb0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.050+0000 7fa0e7a00640 1 -- 172.21.15.86:0/2863922008 shutdown_connections 2024-04-25T14:08:31.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.050+0000 7fa0e7a00640 1 --2- 172.21.15.86:0/2863922008 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fa0bc06f280 0x7fa0bc071790 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.050+0000 7fa0e7a00640 1 --2- 172.21.15.86:0/2863922008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e015cfb0 0x7fa0e015d3c0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.050+0000 7fa0e7a00640 1 --2- 172.21.15.86:0/2863922008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0e00cdcc0 0x7fa0e00c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.050+0000 7fa0e7a00640 1 --2- 172.21.15.86:0/2863922008 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0e0070330 0x7fa0e00cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.050+0000 7fa0e7a00640 1 -- 172.21.15.86:0/2863922008 >> 172.21.15.86:0/2863922008 conn(0x7fa0e00b4650 msgr2=0x7fa0e00b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:31.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.050+0000 7fa0e7a00640 1 -- 172.21.15.86:0/2863922008 shutdown_connections 2024-04-25T14:08:31.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.050+0000 7fa0e7a00640 1 -- 172.21.15.86:0/2863922008 wait complete. 2024-04-25T14:08:31.072 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:08:31.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.222+0000 7f9abac00640 1 Processor -- start 2024-04-25T14:08:31.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.222+0000 7f9abac00640 1 -- start start 2024-04-25T14:08:31.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.222+0000 7f9abac00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9ab415fc70 0x7f9ab4070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:31.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.222+0000 7f9abac00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9ab4071360 0x7f9ab4163810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:31.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.222+0000 7f9abac00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ab406f0a0 con 0x7f9ab4163d50 2024-04-25T14:08:31.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.222+0000 7f9abac00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9ab406e7f0 con 0x7f9ab4071360 2024-04-25T14:08:31.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.222+0000 7f9abac00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9ab4071770 con 0x7f9ab415fc70 2024-04-25T14:08:31.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.222+0000 7f9ab3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9ab4071360 0x7f9ab4163810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:31.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.222+0000 7f9ab3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9ab4071360 0x7f9ab4163810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43422/0 (socket says 172.21.15.86:43422) 2024-04-25T14:08:31.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.222+0000 7f9ab3400640 1 -- 172.21.15.86:0/3289172798 learned_addr learned my addr 172.21.15.86:0/3289172798 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:31.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.222+0000 7f9ab3e00640 1 --2- 172.21.15.86:0/3289172798 >> v2:172.21.15.155:3300/0 conn(0x7f9ab415fc70 0x7f9ab4070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:31.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.222+0000 7f9ab3400640 1 -- 172.21.15.86:0/3289172798 >> v2:172.21.15.155:3300/0 conn(0x7f9ab415fc70 msgr2=0x7f9ab4070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:31.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.222+0000 7f9ab3400640 1 --2- 172.21.15.86:0/3289172798 >> v2:172.21.15.155:3300/0 conn(0x7f9ab415fc70 0x7f9ab4070e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.222+0000 7f9ab3400640 1 -- 172.21.15.86:0/3289172798 >> v1:172.21.15.155:6789/0 conn(0x7f9ab4163d50 legacy=0x7f9ab40722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:31.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.222+0000 7f9ab3400640 1 -- 172.21.15.86:0/3289172798 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab4164160 con 0x7f9ab4071360 2024-04-25T14:08:31.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.226+0000 7f9ab3400640 1 --2- 172.21.15.86:0/3289172798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab4071360 0x7f9ab4163810 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4074330 tx=0x7f9aa001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5469987e6fb6e63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:31.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.226+0000 7f9ab2a00640 1 -- 172.21.15.86:0/3289172798 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa00ab640 con 0x7f9ab4071360 2024-04-25T14:08:31.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.226+0000 7f9ab2a00640 1 -- 172.21.15.86:0/3289172798 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa00b0070 con 0x7f9ab4071360 2024-04-25T14:08:31.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.226+0000 7f9ab2a00640 1 -- 172.21.15.86:0/3289172798 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa00aba00 con 0x7f9ab4071360 2024-04-25T14:08:31.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.226+0000 7f9abac00640 1 -- 172.21.15.86:0/3289172798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab4071360 msgr2=0x7f9ab4163810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:31.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.226+0000 7f9abac00640 1 --2- 172.21.15.86:0/3289172798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab4071360 0x7f9ab4163810 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4074330 tx=0x7f9aa001c520 comp rx=0 tx=0).stop 2024-04-25T14:08:31.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.226+0000 7f9abac00640 1 -- 172.21.15.86:0/3289172798 shutdown_connections 2024-04-25T14:08:31.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.226+0000 7f9abac00640 1 --2- 172.21.15.86:0/3289172798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab4071360 0x7f9ab4163810 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.226+0000 7f9abac00640 1 --2- 172.21.15.86:0/3289172798 >> v2:172.21.15.155:3300/0 conn(0x7f9ab415fc70 0x7f9ab4070e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.226+0000 7f9abac00640 1 -- 172.21.15.86:0/3289172798 >> 172.21.15.86:0/3289172798 conn(0x7f9ab40cffc0 msgr2=0x7f9ab40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:31.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.226+0000 7f9abac00640 1 -- 172.21.15.86:0/3289172798 shutdown_connections 2024-04-25T14:08:31.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.226+0000 7f9abac00640 1 -- 172.21.15.86:0/3289172798 wait complete. 2024-04-25T14:08:31.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.230+0000 7f9abac00640 1 Processor -- start 2024-04-25T14:08:31.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.230+0000 7f9abac00640 1 -- start start 2024-04-25T14:08:31.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.230+0000 7f9abac00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ab4071360 0x7f9ab41506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:31.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.230+0000 7f9abac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab415fc70 0x7f9ab4150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:31.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.230+0000 7f9abac00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ab4163d50 0x7f9ab40cef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:31.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.230+0000 7f9abac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab4160080 con 0x7f9ab415fc70 2024-04-25T14:08:31.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.230+0000 7f9abac00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab406e480 con 0x7f9ab4071360 2024-04-25T14:08:31.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.230+0000 7f9abac00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9ab416cb70 con 0x7f9ab4163d50 2024-04-25T14:08:31.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.230+0000 7f9ab3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab415fc70 0x7f9ab4150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:31.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.230+0000 7f9ab3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab415fc70 0x7f9ab4150be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43436/0 (socket says 172.21.15.86:43436) 2024-04-25T14:08:31.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.230+0000 7f9ab3400640 1 -- 172.21.15.86:0/546098670 learned_addr learned my addr 172.21.15.86:0/546098670 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:31.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.230+0000 7f9ab3e00640 1 --2- 172.21.15.86:0/546098670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ab4071360 0x7f9ab41506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:31.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.230+0000 7f9ab8c00640 1 --2- 172.21.15.86:0/546098670 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ab4163d50 0x7f9ab40cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:31.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.230+0000 7f9ab3400640 1 -- 172.21.15.86:0/546098670 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ab4163d50 msgr2=0x7f9ab40cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:31.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.230+0000 7f9ab3400640 1 --2- 172.21.15.86:0/546098670 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ab4163d50 0x7f9ab40cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.230+0000 7f9ab3400640 1 -- 172.21.15.86:0/546098670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ab4071360 msgr2=0x7f9ab41506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:31.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.230+0000 7f9ab3400640 1 --2- 172.21.15.86:0/546098670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ab4071360 0x7f9ab41506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.230+0000 7f9ab3400640 1 -- 172.21.15.86:0/546098670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab40cbcf0 con 0x7f9ab415fc70 2024-04-25T14:08:31.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.230+0000 7f9ab8c00640 1 --2- 172.21.15.86:0/546098670 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ab4163d50 0x7f9ab40cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:31.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.230+0000 7f9ab3400640 1 --2- 172.21.15.86:0/546098670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab415fc70 0x7f9ab4150be0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f9aa001ca00 tx=0x7f9aa001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:31.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.230+0000 7f9aabe00640 1 -- 172.21.15.86:0/546098670 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa00be5f0 con 0x7f9ab415fc70 2024-04-25T14:08:31.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.230+0000 7f9aabe00640 1 -- 172.21.15.86:0/546098670 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa00be7b0 con 0x7f9ab415fc70 2024-04-25T14:08:31.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.234+0000 7f9aabe00640 1 -- 172.21.15.86:0/546098670 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa00beba0 con 0x7f9ab415fc70 2024-04-25T14:08:31.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.234+0000 7f9abac00640 1 -- 172.21.15.86:0/546098670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab40cbfd0 con 0x7f9ab415fc70 2024-04-25T14:08:31.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.234+0000 7f9abac00640 1 -- 172.21.15.86:0/546098670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab40cc590 con 0x7f9ab415fc70 2024-04-25T14:08:31.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.234+0000 7f9abac00640 1 -- 172.21.15.86:0/546098670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ab40720e0 con 0x7f9ab415fc70 2024-04-25T14:08:31.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.234+0000 7f9aabe00640 1 -- 172.21.15.86:0/546098670 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 93045+0+0 (secure 0 0 0) 0x7f9aa00b43f0 con 0x7f9ab415fc70 2024-04-25T14:08:31.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.234+0000 7f9aabe00640 1 --2- 172.21.15.86:0/546098670 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f9a9406f250 0x7f9a94071760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:31.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.234+0000 7f9aabe00640 1 -- 172.21.15.86:0/546098670 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6875+0+0 (secure 0 0 0) 0x7f9aa0129bb0 con 0x7f9ab415fc70 2024-04-25T14:08:31.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.234+0000 7f9ab3e00640 1 --2- 172.21.15.86:0/546098670 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f9a9406f250 0x7f9a94071760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:31.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.238+0000 7f9ab3e00640 1 --2- 172.21.15.86:0/546098670 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f9a9406f250 0x7f9a94071760 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4003ab0 tx=0x7f9aa4009830 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:31.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.242+0000 7f9aabe00640 1 -- 172.21.15.86:0/546098670 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9aa0016110 con 0x7f9ab415fc70 2024-04-25T14:08:31.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.358+0000 7f9aabe00640 1 -- 172.21.15.86:0/546098670 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 27) ==== 93253+0+0 (secure 0 0 0) 0x7f9aa00f25e0 con 0x7f9ab415fc70 2024-04-25T14:08:31.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.406+0000 7f9abac00640 1 -- 172.21.15.86:0/546098670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9ab4076bf0 con 0x7f9ab415fc70 2024-04-25T14:08:31.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.406+0000 7f9aabe00640 1 -- 172.21.15.86:0/546098670 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v76) ==== 100+0+11 (secure 0 0 0) 0x7f9aa0122c50 con 0x7f9ab415fc70 2024-04-25T14:08:31.409 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:08:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.410+0000 7f9abac00640 1 -- 172.21.15.86:0/546098670 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f9a9406f250 msgr2=0x7f9a94071760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.410+0000 7f9abac00640 1 --2- 172.21.15.86:0/546098670 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f9a9406f250 0x7f9a94071760 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4003ab0 tx=0x7f9aa4009830 comp rx=0 tx=0).stop 2024-04-25T14:08:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.410+0000 7f9abac00640 1 -- 172.21.15.86:0/546098670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab415fc70 msgr2=0x7f9ab4150be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.410+0000 7f9abac00640 1 --2- 172.21.15.86:0/546098670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab415fc70 0x7f9ab4150be0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f9aa001ca00 tx=0x7f9aa001ad60 comp rx=0 tx=0).stop 2024-04-25T14:08:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.410+0000 7f9abac00640 1 -- 172.21.15.86:0/546098670 shutdown_connections 2024-04-25T14:08:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.410+0000 7f9abac00640 1 --2- 172.21.15.86:0/546098670 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f9a9406f250 0x7f9a94071760 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.410+0000 7f9abac00640 1 --2- 172.21.15.86:0/546098670 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ab4163d50 0x7f9ab40cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.410+0000 7f9abac00640 1 --2- 172.21.15.86:0/546098670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab415fc70 0x7f9ab4150be0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.410+0000 7f9abac00640 1 --2- 172.21.15.86:0/546098670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ab4071360 0x7f9ab41506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.410+0000 7f9abac00640 1 -- 172.21.15.86:0/546098670 >> 172.21.15.86:0/546098670 conn(0x7f9ab40cffc0 msgr2=0x7f9ab40722f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.410+0000 7f9abac00640 1 -- 172.21.15.86:0/546098670 shutdown_connections 2024-04-25T14:08:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.410+0000 7f9abac00640 1 -- 172.21.15.86:0/546098670 wait complete. 2024-04-25T14:08:31.431 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:08:31.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.574+0000 7f5a57200640 1 Processor -- start 2024-04-25T14:08:31.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.574+0000 7f5a57200640 1 -- start start 2024-04-25T14:08:31.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.578+0000 7f5a57200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a500cfca0 0x7f5a5006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:31.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.578+0000 7f5a57200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5a500d16e0 0x7f5a500d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:31.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.578+0000 7f5a57200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5a5006c670 0x7f5a5006ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:31.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.578+0000 7f5a57200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5a50070470 con 0x7f5a5006c670 2024-04-25T14:08:31.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.578+0000 7f5a57200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f5a500d2030 con 0x7f5a500d16e0 2024-04-25T14:08:31.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.578+0000 7f5a57200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5a50072bd0 con 0x7f5a500cfca0 2024-04-25T14:08:31.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.578+0000 7f5a55200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5a5006c670 0x7f5a5006ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:31.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.578+0000 7f5a55200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5a5006c670 0x7f5a5006ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43468/0 (socket says 172.21.15.86:43468) 2024-04-25T14:08:31.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.578+0000 7f5a55200640 1 -- 172.21.15.86:0/4059154691 learned_addr learned my addr 172.21.15.86:0/4059154691 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:31.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.578+0000 7f5a47400640 1 --2- 172.21.15.86:0/4059154691 >> v2:172.21.15.155:3300/0 conn(0x7f5a500d16e0 0x7f5a500d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:31.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.578+0000 7f5a4fe00640 1 --2- 172.21.15.86:0/4059154691 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a500cfca0 0x7f5a5006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:31.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.578+0000 7f5a55200640 1 -- 172.21.15.86:0/4059154691 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a500cfca0 msgr2=0x7f5a5006f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:31.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.578+0000 7f5a55200640 1 --2- 172.21.15.86:0/4059154691 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a500cfca0 0x7f5a5006f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.578+0000 7f5a55200640 1 -- 172.21.15.86:0/4059154691 >> v2:172.21.15.155:3300/0 conn(0x7f5a500d16e0 msgr2=0x7f5a500d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:31.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.578+0000 7f5a55200640 1 --2- 172.21.15.86:0/4059154691 >> v2:172.21.15.155:3300/0 conn(0x7f5a500d16e0 0x7f5a500d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.578+0000 7f5a55200640 1 -- 172.21.15.86:0/4059154691 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a5006f930 con 0x7f5a5006c670 2024-04-25T14:08:31.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.582+0000 7f5a55200640 1 --2- 172.21.15.86:0/4059154691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a5006c670 0x7f5a5006ca80 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f5a50070f40 tx=0x7f5a4801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32818cf8e496e215 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:31.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.582+0000 7f5a4f400640 1 -- 172.21.15.86:0/4059154691 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a480b0070 con 0x7f5a5006c670 2024-04-25T14:08:31.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.582+0000 7f5a4f400640 1 -- 172.21.15.86:0/4059154691 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a480ab600 con 0x7f5a5006c670 2024-04-25T14:08:31.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.582+0000 7f5a4f400640 1 -- 172.21.15.86:0/4059154691 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a480ab9d0 con 0x7f5a5006c670 2024-04-25T14:08:31.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.582+0000 7f5a57200640 1 -- 172.21.15.86:0/4059154691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a5006c670 msgr2=0x7f5a5006ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.582+0000 7f5a57200640 1 --2- 172.21.15.86:0/4059154691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a5006c670 0x7f5a5006ca80 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f5a50070f40 tx=0x7f5a4801c520 comp rx=0 tx=0).stop 2024-04-25T14:08:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.582+0000 7f5a57200640 1 -- 172.21.15.86:0/4059154691 shutdown_connections 2024-04-25T14:08:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.582+0000 7f5a57200640 1 --2- 172.21.15.86:0/4059154691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a5006c670 0x7f5a5006ca80 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.582+0000 7f5a57200640 1 --2- 172.21.15.86:0/4059154691 >> v2:172.21.15.155:3300/0 conn(0x7f5a500d16e0 0x7f5a500d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.582+0000 7f5a57200640 1 --2- 172.21.15.86:0/4059154691 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a500cfca0 0x7f5a5006f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.582+0000 7f5a57200640 1 -- 172.21.15.86:0/4059154691 >> 172.21.15.86:0/4059154691 conn(0x7f5a500b4650 msgr2=0x7f5a500b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.582+0000 7f5a57200640 1 -- 172.21.15.86:0/4059154691 shutdown_connections 2024-04-25T14:08:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.582+0000 7f5a57200640 1 -- 172.21.15.86:0/4059154691 wait complete. 2024-04-25T14:08:31.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.582+0000 7f5a57200640 1 Processor -- start 2024-04-25T14:08:31.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.582+0000 7f5a57200640 1 -- start start 2024-04-25T14:08:31.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.586+0000 7f5a57200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a5006c670 0x7f5a5006ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:31.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.586+0000 7f5a57200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a500cfca0 0x7f5a50073770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:31.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.586+0000 7f5a57200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a500d16e0 0x7f5a50156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:31.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.586+0000 7f5a57200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a500ce570 con 0x7f5a500d16e0 2024-04-25T14:08:31.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.586+0000 7f5a57200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5006d3d0 con 0x7f5a500cfca0 2024-04-25T14:08:31.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.586+0000 7f5a57200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5a500ce6f0 con 0x7f5a5006c670 2024-04-25T14:08:31.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.586+0000 7f5a55200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a500d16e0 0x7f5a50156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:31.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.586+0000 7f5a55200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a500d16e0 0x7f5a50156240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43478/0 (socket says 172.21.15.86:43478) 2024-04-25T14:08:31.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.586+0000 7f5a55200640 1 -- 172.21.15.86:0/3910303585 learned_addr learned my addr 172.21.15.86:0/3910303585 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:31.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.586+0000 7f5a4fe00640 1 --2- 172.21.15.86:0/3910303585 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a5006c670 0x7f5a5006ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:31.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.586+0000 7f5a47400640 1 --2- 172.21.15.86:0/3910303585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a500cfca0 0x7f5a50073770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:31.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.586+0000 7f5a55200640 1 -- 172.21.15.86:0/3910303585 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a5006c670 msgr2=0x7f5a5006ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:31.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.586+0000 7f5a55200640 1 --2- 172.21.15.86:0/3910303585 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a5006c670 0x7f5a5006ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.586+0000 7f5a55200640 1 -- 172.21.15.86:0/3910303585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a500cfca0 msgr2=0x7f5a50073770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:31.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.586+0000 7f5a55200640 1 --2- 172.21.15.86:0/3910303585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a500cfca0 0x7f5a50073770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.586+0000 7f5a55200640 1 -- 172.21.15.86:0/3910303585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a5015b350 con 0x7f5a500d16e0 2024-04-25T14:08:31.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.586+0000 7f5a47400640 1 --2- 172.21.15.86:0/3910303585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a500cfca0 0x7f5a50073770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:31.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.586+0000 7f5a55200640 1 --2- 172.21.15.86:0/3910303585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a500d16e0 0x7f5a50156240 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f5a4801ca00 tx=0x7f5a480192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:31.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.586+0000 7f5a4cc00640 1 -- 172.21.15.86:0/3910303585 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a480b0040 con 0x7f5a500d16e0 2024-04-25T14:08:31.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.586+0000 7f5a57200640 1 -- 172.21.15.86:0/3910303585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a500988e0 con 0x7f5a500d16e0 2024-04-25T14:08:31.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.586+0000 7f5a4cc00640 1 -- 172.21.15.86:0/3910303585 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a480b4420 con 0x7f5a500d16e0 2024-04-25T14:08:31.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.586+0000 7f5a4cc00640 1 -- 172.21.15.86:0/3910303585 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a480b4810 con 0x7f5a500d16e0 2024-04-25T14:08:31.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.586+0000 7f5a57200640 1 -- 172.21.15.86:0/3910303585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a5015a870 con 0x7f5a500d16e0 2024-04-25T14:08:31.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.586+0000 7f5a57200640 1 -- 172.21.15.86:0/3910303585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a5009f4b0 con 0x7f5a500d16e0 2024-04-25T14:08:31.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.590+0000 7f5a4cc00640 1 -- 172.21.15.86:0/3910303585 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 27) ==== 93253+0+0 (secure 0 0 0) 0x7f5a480b49b0 con 0x7f5a500d16e0 2024-04-25T14:08:31.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.590+0000 7f5a4cc00640 1 --2- 172.21.15.86:0/3910303585 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f5a2c06f4c0 0x7f5a2c0719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:31.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.590+0000 7f5a4cc00640 1 -- 172.21.15.86:0/3910303585 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6875+0+0 (secure 0 0 0) 0x7f5a48129340 con 0x7f5a500d16e0 2024-04-25T14:08:31.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.590+0000 7f5a4fe00640 1 --2- 172.21.15.86:0/3910303585 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f5a2c06f4c0 0x7f5a2c0719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:31.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.590+0000 7f5a4fe00640 1 --2- 172.21.15.86:0/3910303585 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f5a2c06f4c0 0x7f5a2c0719d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5a50070710 tx=0x7f5a4000b040 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:31.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.594+0000 7f5a4cc00640 1 -- 172.21.15.86:0/3910303585 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5a48016110 con 0x7f5a500d16e0 2024-04-25T14:08:31.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.786+0000 7f5a57200640 1 -- 172.21.15.86:0/3910303585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a50069d00 con 0x7f5a500d16e0 2024-04-25T14:08:31.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.786+0000 7f5a4cc00640 1 -- 172.21.15.86:0/3910303585 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5210 (secure 0 0 0) 0x7f5a480f5de0 con 0x7f5a500d16e0 2024-04-25T14:08:31.788 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:08:31.788 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":58,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1210051860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1210051860},{"type":"v1","addr":"172.21.15.155:6833","nonce":1210051860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2868300438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2868300438},{"type":"v1","addr":"172.21.15.155:6835","nonce":2868300438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:06:44.910717+0000","modified":"2024-04-25T14:08:29.703081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6603,"mds_1":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"c","rank":1,"incarnation":55,"state":"up:active","state_seq":9,"addr":"172.21.15.86:6835/2682115891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2682115891},{"type":"v1","addr":"172.21.15.86:6835","nonce":2682115891}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6603":{"gid":6603,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/1088108173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1088108173},{"type":"v1","addr":"172.21.15.86:6837","nonce":1088108173}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6603,"qdb_cluster":[6600,6603]},"id":5}]} 2024-04-25T14:08:31.788 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 58 2024-04-25T14:08:31.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.790+0000 7f5a57200640 1 -- 172.21.15.86:0/3910303585 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f5a2c06f4c0 msgr2=0x7f5a2c0719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.790+0000 7f5a57200640 1 --2- 172.21.15.86:0/3910303585 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f5a2c06f4c0 0x7f5a2c0719d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5a50070710 tx=0x7f5a4000b040 comp rx=0 tx=0).stop 2024-04-25T14:08:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.790+0000 7f5a57200640 1 -- 172.21.15.86:0/3910303585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a500d16e0 msgr2=0x7f5a50156240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.790+0000 7f5a57200640 1 --2- 172.21.15.86:0/3910303585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a500d16e0 0x7f5a50156240 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f5a4801ca00 tx=0x7f5a480192c0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.790+0000 7f5a57200640 1 -- 172.21.15.86:0/3910303585 shutdown_connections 2024-04-25T14:08:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.790+0000 7f5a57200640 1 --2- 172.21.15.86:0/3910303585 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f5a2c06f4c0 0x7f5a2c0719d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.790+0000 7f5a57200640 1 --2- 172.21.15.86:0/3910303585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a500d16e0 0x7f5a50156240 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.790+0000 7f5a57200640 1 --2- 172.21.15.86:0/3910303585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a500cfca0 0x7f5a50073770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.790+0000 7f5a57200640 1 --2- 172.21.15.86:0/3910303585 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a5006c670 0x7f5a5006ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.790+0000 7f5a57200640 1 -- 172.21.15.86:0/3910303585 >> 172.21.15.86:0/3910303585 conn(0x7f5a500b4650 msgr2=0x7f5a5015c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.790+0000 7f5a57200640 1 -- 172.21.15.86:0/3910303585 shutdown_connections 2024-04-25T14:08:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.790+0000 7f5a57200640 1 -- 172.21.15.86:0/3910303585 wait complete. 2024-04-25T14:08:31.811 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:08:31.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.954+0000 7fd368600640 1 Processor -- start 2024-04-25T14:08:31.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.954+0000 7fd368600640 1 -- start start 2024-04-25T14:08:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.958+0000 7fd368600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd36015faa0 0x7fd36006d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.958+0000 7fd368600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd3600aeea0 0x7fd3600af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.958+0000 7fd368600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd36006c670 con 0x7fd3600b59e0 2024-04-25T14:08:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.958+0000 7fd368600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd36006d560 con 0x7fd3600aeea0 2024-04-25T14:08:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.958+0000 7fd368600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd36006c950 con 0x7fd36015faa0 2024-04-25T14:08:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.958+0000 7fd365200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd3600b59e0 0x7fd3600b5df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:58394/0 (socket says 172.21.15.86:58394) 2024-04-25T14:08:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.958+0000 7fd365200640 1 -- 172.21.15.86:0/705579336 learned_addr learned my addr 172.21.15.86:0/705579336 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:31.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.958+0000 7fd365c00640 1 --2- 172.21.15.86:0/705579336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd36015faa0 0x7fd36006d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:31.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.958+0000 7fd366600640 1 --2- 172.21.15.86:0/705579336 >> v2:172.21.15.155:3300/0 conn(0x7fd3600aeea0 0x7fd3600af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:31.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.958+0000 7fd35fe00640 1 -- 172.21.15.86:0/705579336 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1319009704 0 0) 0x7fd36006c670 con 0x7fd3600b59e0 2024-04-25T14:08:31.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.958+0000 7fd35fe00640 1 -- 172.21.15.86:0/705579336 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd348003530 con 0x7fd3600b59e0 2024-04-25T14:08:31.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.958+0000 7fd365c00640 1 -- 172.21.15.86:0/705579336 >> v2:172.21.15.155:3300/0 conn(0x7fd3600aeea0 msgr2=0x7fd3600af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:31.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.958+0000 7fd365c00640 1 --2- 172.21.15.86:0/705579336 >> v2:172.21.15.155:3300/0 conn(0x7fd3600aeea0 0x7fd3600af2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.958+0000 7fd365c00640 1 -- 172.21.15.86:0/705579336 >> v1:172.21.15.86:6789/0 conn(0x7fd3600b59e0 legacy=0x7fd3600b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:31.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.958+0000 7fd35fe00640 1 -- 172.21.15.86:0/705579336 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 835816851 0 0) 0x7fd348003530 con 0x7fd3600b59e0 2024-04-25T14:08:31.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.958+0000 7fd365c00640 1 -- 172.21.15.86:0/705579336 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36006cc30 con 0x7fd36015faa0 2024-04-25T14:08:31.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.962+0000 7fd365c00640 1 --2- 172.21.15.86:0/705579336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd36015faa0 0x7fd36006d9f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd3540097c0 tx=0x7fd35401c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c555dbccfa4ecb2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:31.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.962+0000 7fd35fe00640 1 -- 172.21.15.86:0/705579336 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3540b0070 con 0x7fd36015faa0 2024-04-25T14:08:31.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.962+0000 7fd35fe00640 1 -- 172.21.15.86:0/705579336 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3540ab400 con 0x7fd36015faa0 2024-04-25T14:08:31.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.962+0000 7fd35fe00640 1 -- 172.21.15.86:0/705579336 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c001680 con 0x7fd36015faa0 2024-04-25T14:08:31.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.962+0000 7fd368600640 1 -- 172.21.15.86:0/705579336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd36015faa0 msgr2=0x7fd36006d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:31.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.962+0000 7fd368600640 1 --2- 172.21.15.86:0/705579336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd36015faa0 0x7fd36006d9f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd3540097c0 tx=0x7fd35401c3c0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.962+0000 7fd368600640 1 -- 172.21.15.86:0/705579336 shutdown_connections 2024-04-25T14:08:31.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.962+0000 7fd368600640 1 --2- 172.21.15.86:0/705579336 >> v2:172.21.15.155:3300/0 conn(0x7fd3600aeea0 0x7fd3600af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.962+0000 7fd368600640 1 --2- 172.21.15.86:0/705579336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd36015faa0 0x7fd36006d9f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.962+0000 7fd368600640 1 -- 172.21.15.86:0/705579336 >> 172.21.15.86:0/705579336 conn(0x7fd3600d2230 msgr2=0x7fd3600d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:31.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.962+0000 7fd368600640 1 -- 172.21.15.86:0/705579336 shutdown_connections 2024-04-25T14:08:31.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.962+0000 7fd368600640 1 -- 172.21.15.86:0/705579336 wait complete. 2024-04-25T14:08:31.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.966+0000 7fd368600640 1 Processor -- start 2024-04-25T14:08:31.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.966+0000 7fd368600640 1 -- start start 2024-04-25T14:08:31.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.966+0000 7fd368600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3600aeea0 0x7fd360140830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:31.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.966+0000 7fd368600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3600b59e0 0x7fd360159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.966+0000 7fd368600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd36015faa0 0x7fd360159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.966+0000 7fd368600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd360160030 con 0x7fd3600b59e0 2024-04-25T14:08:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.966+0000 7fd368600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd36015feb0 con 0x7fd36015faa0 2024-04-25T14:08:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.966+0000 7fd368600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd360070f30 con 0x7fd3600aeea0 2024-04-25T14:08:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.966+0000 7fd365200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3600b59e0 0x7fd360159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.966+0000 7fd365200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3600b59e0 0x7fd360159880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43480/0 (socket says 172.21.15.86:43480) 2024-04-25T14:08:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.966+0000 7fd365200640 1 -- 172.21.15.86:0/3772768617 learned_addr learned my addr 172.21.15.86:0/3772768617 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.966+0000 7fd366600640 1 --2- 172.21.15.86:0/3772768617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd36015faa0 0x7fd360159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.966+0000 7fd365c00640 1 --2- 172.21.15.86:0/3772768617 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3600aeea0 0x7fd360140830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.966+0000 7fd365200640 1 -- 172.21.15.86:0/3772768617 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3600aeea0 msgr2=0x7fd360140830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.966+0000 7fd365200640 1 --2- 172.21.15.86:0/3772768617 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3600aeea0 0x7fd360140830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.966+0000 7fd365200640 1 -- 172.21.15.86:0/3772768617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd36015faa0 msgr2=0x7fd360159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:31.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.966+0000 7fd365200640 1 --2- 172.21.15.86:0/3772768617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd36015faa0 0x7fd360159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:31.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.966+0000 7fd365200640 1 -- 172.21.15.86:0/3772768617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36015bc20 con 0x7fd3600b59e0 2024-04-25T14:08:31.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.966+0000 7fd365200640 1 --2- 172.21.15.86:0/3772768617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3600b59e0 0x7fd360159880 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0011f0 tx=0x7fd34c00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:31.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.966+0000 7fd35d600640 1 -- 172.21.15.86:0/3772768617 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c005ae0 con 0x7fd3600b59e0 2024-04-25T14:08:31.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.966+0000 7fd35d600640 1 -- 172.21.15.86:0/3772768617 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd34c005c80 con 0x7fd3600b59e0 2024-04-25T14:08:31.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.966+0000 7fd35d600640 1 -- 172.21.15.86:0/3772768617 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c007290 con 0x7fd3600b59e0 2024-04-25T14:08:31.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.966+0000 7fd368600640 1 -- 172.21.15.86:0/3772768617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd360141370 con 0x7fd3600b59e0 2024-04-25T14:08:31.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.966+0000 7fd368600640 1 -- 172.21.15.86:0/3772768617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd360141930 con 0x7fd3600b59e0 2024-04-25T14:08:31.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.970+0000 7fd368600640 1 -- 172.21.15.86:0/3772768617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd360164a60 con 0x7fd3600b59e0 2024-04-25T14:08:31.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.970+0000 7fd35d600640 1 -- 172.21.15.86:0/3772768617 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 27) ==== 93253+0+0 (secure 0 0 0) 0x7fd34c007430 con 0x7fd3600b59e0 2024-04-25T14:08:31.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.970+0000 7fd35d600640 1 --2- 172.21.15.86:0/3772768617 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fd34006f490 0x7fd3400719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:31.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.970+0000 7fd35d600640 1 -- 172.21.15.86:0/3772768617 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6875+0+0 (secure 0 0 0) 0x7fd34c08f4e0 con 0x7fd3600b59e0 2024-04-25T14:08:31.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.970+0000 7fd365c00640 1 --2- 172.21.15.86:0/3772768617 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fd34006f490 0x7fd3400719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:31.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.970+0000 7fd365c00640 1 --2- 172.21.15.86:0/3772768617 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fd34006f490 0x7fd3400719a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd3540ad000 tx=0x7fd3540a4f90 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:31.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:31.974+0000 7fd35d600640 1 -- 172.21.15.86:0/3772768617 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd34c007720 con 0x7fd3600b59e0 2024-04-25T14:08:32.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.134+0000 7fd368600640 1 -- 172.21.15.86:0/3772768617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd360074cf0 con 0x7fd3600b59e0 2024-04-25T14:08:32.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.134+0000 7fd35d600640 1 -- 172.21.15.86:0/3772768617 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+18065 (secure 0 0 0) 0x7fd34c05bf60 con 0x7fd3600b59e0 2024-04-25T14:08:32.136 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:08:32.136 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":76,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:08:29.333499+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:06:42.666742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T14:06:43.884074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2066326917":"2024-04-26T14:08:29.333458+0000","172.21.15.86:6833/203029814":"2024-04-26T14:08:29.333458+0000","172.21.15.86:6832/203029814":"2024-04-26T14:08:29.333458+0000","172.21.15.86:0/3991804772":"2024-04-26T14:08:29.333458+0000","172.21.15.86:0/2307768874":"2024-04-26T14:08:29.333458+0000","172.21.15.86:0/2510298865":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/3380489666":"2024-04-26T14:08:29.333458+0000","172.21.15.86:0/82848493":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/2452803132":"2024-04-26T14:08:29.333458+0000","172.21.15.86:0/1793604306":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/3402941646":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/112097751":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/2567754949":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/2334647603":"2024-04-26T14:08:29.333458+0000","172.21.15.86:0/529842883":"2024-04-26T14:07:03.055680+0000","172.21.15.86:6833/1025829557":"2024-04-26T14:07:03.055680+0000","172.21.15.86:6832/1025829557":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/759420784":"2024-04-26T14:07:03.055680+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:08:32.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.134+0000 7fd368600640 1 -- 172.21.15.86:0/3772768617 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fd34006f490 msgr2=0x7fd3400719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:32.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.134+0000 7fd368600640 1 --2- 172.21.15.86:0/3772768617 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fd34006f490 0x7fd3400719a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd3540ad000 tx=0x7fd3540a4f90 comp rx=0 tx=0).stop 2024-04-25T14:08:32.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.134+0000 7fd368600640 1 -- 172.21.15.86:0/3772768617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3600b59e0 msgr2=0x7fd360159880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:32.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.134+0000 7fd368600640 1 --2- 172.21.15.86:0/3772768617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3600b59e0 0x7fd360159880 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0011f0 tx=0x7fd34c00cbe0 comp rx=0 tx=0).stop 2024-04-25T14:08:32.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.134+0000 7fd368600640 1 -- 172.21.15.86:0/3772768617 shutdown_connections 2024-04-25T14:08:32.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.134+0000 7fd368600640 1 --2- 172.21.15.86:0/3772768617 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fd34006f490 0x7fd3400719a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:32.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.134+0000 7fd368600640 1 --2- 172.21.15.86:0/3772768617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd36015faa0 0x7fd360159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:32.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.134+0000 7fd368600640 1 --2- 172.21.15.86:0/3772768617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3600b59e0 0x7fd360159880 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:32.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.134+0000 7fd368600640 1 --2- 172.21.15.86:0/3772768617 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3600aeea0 0x7fd360140830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:32.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.134+0000 7fd368600640 1 -- 172.21.15.86:0/3772768617 >> 172.21.15.86:0/3772768617 conn(0x7fd3600d2230 msgr2=0x7fd3601622f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:32.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.134+0000 7fd368600640 1 -- 172.21.15.86:0/3772768617 shutdown_connections 2024-04-25T14:08:32.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.134+0000 7fd368600640 1 -- 172.21.15.86:0/3772768617 wait complete. 2024-04-25T14:08:32.158 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T14:08:32.158 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:08:32.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.302+0000 7f8bf7e00640 1 Processor -- start 2024-04-25T14:08:32.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.302+0000 7f8bf7e00640 1 -- start start 2024-04-25T14:08:32.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.306+0000 7f8bf7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8bf00cfca0 0x7f8bf006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:32.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.306+0000 7f8bf7e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bf0072bd0 con 0x7f8bf00d16e0 2024-04-25T14:08:32.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.306+0000 7f8bf7e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bf006f930 con 0x7f8bf00730b0 2024-04-25T14:08:32.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.306+0000 7f8bf7e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8bf0070470 con 0x7f8bf00cfca0 2024-04-25T14:08:32.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.306+0000 7f8bf5400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8bf00cfca0 0x7f8bf006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:32.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.306+0000 7f8bf4a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8bf00d16e0 0x7f8bf00d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:58406/0 (socket says 172.21.15.86:58406) 2024-04-25T14:08:32.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.306+0000 7f8bf4a00640 1 -- 172.21.15.86:0/880993716 learned_addr learned my addr 172.21.15.86:0/880993716 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:32.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.306+0000 7f8bf5400640 1 -- 172.21.15.86:0/880993716 >> v1:172.21.15.155:6789/0 conn(0x7f8bf00730b0 legacy=0x7f8bf00734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:32.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.306+0000 7f8bf5400640 1 -- 172.21.15.86:0/880993716 >> v1:172.21.15.86:6789/0 conn(0x7f8bf00d16e0 legacy=0x7f8bf00d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:32.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.306+0000 7f8bf5400640 1 -- 172.21.15.86:0/880993716 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf006fbd0 con 0x7f8bf00cfca0 2024-04-25T14:08:32.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.306+0000 7f8befe00640 1 -- 172.21.15.86:0/880993716 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2195514325 0 0) 0x7f8bf0072bd0 con 0x7f8bf00d16e0 2024-04-25T14:08:32.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.310+0000 7f8bf5400640 1 --2- 172.21.15.86:0/880993716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bf00cfca0 0x7f8bf006f3f0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f8be40097c0 tx=0x7f8be401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae9091d4f06ce6d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:32.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.310+0000 7f8befe00640 1 -- 172.21.15.86:0/880993716 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be40b1070 con 0x7f8bf00cfca0 2024-04-25T14:08:32.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.310+0000 7f8befe00640 1 -- 172.21.15.86:0/880993716 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8be40ac410 con 0x7f8bf00cfca0 2024-04-25T14:08:32.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.310+0000 7f8befe00640 1 -- 172.21.15.86:0/880993716 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf00743e0 con 0x7f8bf00cfca0 2024-04-25T14:08:32.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.310+0000 7f8bf7e00640 1 -- 172.21.15.86:0/880993716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bf00cfca0 msgr2=0x7f8bf006f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:32.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.310+0000 7f8bf7e00640 1 --2- 172.21.15.86:0/880993716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bf00cfca0 0x7f8bf006f3f0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f8be40097c0 tx=0x7f8be401c4a0 comp rx=0 tx=0).stop 2024-04-25T14:08:32.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.310+0000 7f8bf7e00640 1 -- 172.21.15.86:0/880993716 shutdown_connections 2024-04-25T14:08:32.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.310+0000 7f8bf7e00640 1 --2- 172.21.15.86:0/880993716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bf00cfca0 0x7f8bf006f3f0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:32.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.310+0000 7f8bf7e00640 1 -- 172.21.15.86:0/880993716 >> 172.21.15.86:0/880993716 conn(0x7f8bf00b4650 msgr2=0x7f8bf00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:32.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.310+0000 7f8bf7e00640 1 -- 172.21.15.86:0/880993716 shutdown_connections 2024-04-25T14:08:32.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.310+0000 7f8bf7e00640 1 -- 172.21.15.86:0/880993716 wait complete. 2024-04-25T14:08:32.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.310+0000 7f8bf7e00640 1 Processor -- start 2024-04-25T14:08:32.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.310+0000 7f8bf7e00640 1 -- start start 2024-04-25T14:08:32.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.314+0000 7f8bf7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bf00730b0 0x7f8bf009a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:32.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.314+0000 7f8bf7e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bf00cfca0 0x7f8bf015e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:32.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.314+0000 7f8bf7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bf00d16e0 0x7f8bf015d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:32.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.314+0000 7f8bf7e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf016aa80 con 0x7f8bf00d16e0 2024-04-25T14:08:32.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.314+0000 7f8bf7e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf016a900 con 0x7f8bf00730b0 2024-04-25T14:08:32.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.314+0000 7f8bf7e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8bf016ac00 con 0x7f8bf00cfca0 2024-04-25T14:08:32.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.314+0000 7f8bf5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bf00d16e0 0x7f8bf015d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:32.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.314+0000 7f8bf5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bf00d16e0 0x7f8bf015d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43496/0 (socket says 172.21.15.86:43496) 2024-04-25T14:08:32.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.314+0000 7f8bf5e00640 1 -- 172.21.15.86:0/255054404 learned_addr learned my addr 172.21.15.86:0/255054404 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:32.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.314+0000 7f8bf5e00640 1 -- 172.21.15.86:0/255054404 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bf00cfca0 msgr2=0x7f8bf015e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:32.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.314+0000 7f8bf5e00640 1 --2- 172.21.15.86:0/255054404 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bf00cfca0 0x7f8bf015e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:32.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.314+0000 7f8bf5e00640 1 -- 172.21.15.86:0/255054404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bf00730b0 msgr2=0x7f8bf009a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:32.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.314+0000 7f8bf5400640 1 --2- 172.21.15.86:0/255054404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bf00730b0 0x7f8bf009a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:32.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.314+0000 7f8bf5e00640 1 --2- 172.21.15.86:0/255054404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bf00730b0 0x7f8bf009a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:32.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.314+0000 7f8bf5e00640 1 -- 172.21.15.86:0/255054404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf015e0b0 con 0x7f8bf00d16e0 2024-04-25T14:08:32.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.314+0000 7f8bf5400640 1 --2- 172.21.15.86:0/255054404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bf00730b0 0x7f8bf009a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:32.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.314+0000 7f8bf5e00640 1 --2- 172.21.15.86:0/255054404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bf00d16e0 0x7f8bf015d150 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f8be800c760 tx=0x7f8be800cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:32.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.314+0000 7f8bed600640 1 -- 172.21.15.86:0/255054404 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be8003ba0 con 0x7f8bf00d16e0 2024-04-25T14:08:32.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.314+0000 7f8bed600640 1 -- 172.21.15.86:0/255054404 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8be8003d40 con 0x7f8bf00d16e0 2024-04-25T14:08:32.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.314+0000 7f8bed600640 1 -- 172.21.15.86:0/255054404 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be8005310 con 0x7f8bf00d16e0 2024-04-25T14:08:32.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.314+0000 7f8bf7e00640 1 -- 172.21.15.86:0/255054404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf009ae90 con 0x7f8bf00d16e0 2024-04-25T14:08:32.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.314+0000 7f8bf7e00640 1 -- 172.21.15.86:0/255054404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf009dd20 con 0x7f8bf00d16e0 2024-04-25T14:08:32.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.314+0000 7f8bf7e00640 1 -- 172.21.15.86:0/255054404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bf009f4b0 con 0x7f8bf00d16e0 2024-04-25T14:08:32.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.318+0000 7f8bed600640 1 -- 172.21.15.86:0/255054404 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 27) ==== 93253+0+0 (secure 0 0 0) 0x7f8be80054b0 con 0x7f8bf00d16e0 2024-04-25T14:08:32.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.318+0000 7f8bed600640 1 --2- 172.21.15.86:0/255054404 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f8bd006f490 0x7f8bd00719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:32.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.318+0000 7f8bed600640 1 -- 172.21.15.86:0/255054404 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6875+0+0 (secure 0 0 0) 0x7f8be808f6e0 con 0x7f8bf00d16e0 2024-04-25T14:08:32.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.318+0000 7f8bf5400640 1 --2- 172.21.15.86:0/255054404 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f8bd006f490 0x7f8bd00719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:32.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.318+0000 7f8bf5400640 1 --2- 172.21.15.86:0/255054404 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f8bd006f490 0x7f8bd00719a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8be40ae000 tx=0x7f8be40a6040 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:32.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.322+0000 7f8bed600640 1 -- 172.21.15.86:0/255054404 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8be805c050 con 0x7f8bf00d16e0 2024-04-25T14:08:32.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.514+0000 7f8bf7e00640 1 -- 172.21.15.86:0/255054404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bf0069d00 con 0x7f8bf00d16e0 2024-04-25T14:08:32.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.514+0000 7f8bed600640 1 -- 172.21.15.86:0/255054404 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5210 (secure 0 0 0) 0x7f8be80640d0 con 0x7f8bf00d16e0 2024-04-25T14:08:32.516 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:08:32.516 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":58,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1210051860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1210051860},{"type":"v1","addr":"172.21.15.155:6833","nonce":1210051860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2868300438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2868300438},{"type":"v1","addr":"172.21.15.155:6835","nonce":2868300438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:06:44.910717+0000","modified":"2024-04-25T14:08:29.703081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6603,"mds_1":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"c","rank":1,"incarnation":55,"state":"up:active","state_seq":9,"addr":"172.21.15.86:6835/2682115891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2682115891},{"type":"v1","addr":"172.21.15.86:6835","nonce":2682115891}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6603":{"gid":6603,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/1088108173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1088108173},{"type":"v1","addr":"172.21.15.86:6837","nonce":1088108173}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6603,"qdb_cluster":[6600,6603]},"id":5}]} 2024-04-25T14:08:32.517 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 58 2024-04-25T14:08:32.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.514+0000 7f8bf7e00640 1 -- 172.21.15.86:0/255054404 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f8bd006f490 msgr2=0x7f8bd00719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:32.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.514+0000 7f8bf7e00640 1 --2- 172.21.15.86:0/255054404 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f8bd006f490 0x7f8bd00719a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8be40ae000 tx=0x7f8be40a6040 comp rx=0 tx=0).stop 2024-04-25T14:08:32.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.514+0000 7f8bf7e00640 1 -- 172.21.15.86:0/255054404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bf00d16e0 msgr2=0x7f8bf015d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:32.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.514+0000 7f8bf7e00640 1 --2- 172.21.15.86:0/255054404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bf00d16e0 0x7f8bf015d150 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f8be800c760 tx=0x7f8be800cb60 comp rx=0 tx=0).stop 2024-04-25T14:08:32.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.518+0000 7f8bf7e00640 1 -- 172.21.15.86:0/255054404 shutdown_connections 2024-04-25T14:08:32.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.518+0000 7f8bf7e00640 1 --2- 172.21.15.86:0/255054404 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f8bd006f490 0x7f8bd00719a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:32.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.518+0000 7f8bf7e00640 1 --2- 172.21.15.86:0/255054404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bf00d16e0 0x7f8bf015d150 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:32.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.518+0000 7f8bf7e00640 1 --2- 172.21.15.86:0/255054404 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bf00cfca0 0x7f8bf015e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:32.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.518+0000 7f8bf7e00640 1 --2- 172.21.15.86:0/255054404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bf00730b0 0x7f8bf009a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:32.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.518+0000 7f8bf7e00640 1 -- 172.21.15.86:0/255054404 >> 172.21.15.86:0/255054404 conn(0x7f8bf00b4650 msgr2=0x7f8bf00734c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:32.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.518+0000 7f8bf7e00640 1 -- 172.21.15.86:0/255054404 shutdown_connections 2024-04-25T14:08:32.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.518+0000 7f8bf7e00640 1 -- 172.21.15.86:0/255054404 wait complete. 2024-04-25T14:08:32.538 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:08:32.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.682+0000 7f5992000640 1 Processor -- start 2024-04-25T14:08:32.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.682+0000 7f5992000640 1 -- start start 2024-04-25T14:08:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.682+0000 7f5992000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f598c071530 0x7f598c15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.682+0000 7f5992000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f598c1616e0 0x7f598c161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.682+0000 7f5992000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f598c0cd8d0 con 0x7f598c0d2c80 2024-04-25T14:08:32.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.682+0000 7f5992000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f598c0ce720 con 0x7f598c1616e0 2024-04-25T14:08:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.682+0000 7f5992000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f598c0ce970 con 0x7f598c071530 2024-04-25T14:08:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.682+0000 7f598b400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f598c0d2c80 0x7f598c070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:58416/0 (socket says 172.21.15.86:58416) 2024-04-25T14:08:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.682+0000 7f598b400640 1 -- 172.21.15.86:0/520672347 learned_addr learned my addr 172.21.15.86:0/520672347 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.682+0000 7f598be00640 1 --2- 172.21.15.86:0/520672347 >> v2:172.21.15.86:3300/0 conn(0x7f598c1616e0 0x7f598c161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:32.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.682+0000 7f5982a00640 1 --2- 172.21.15.86:0/520672347 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f598c071530 0x7f598c15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.682+0000 7f598be00640 1 -- 172.21.15.86:0/520672347 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f598c071530 msgr2=0x7f598c15dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.682+0000 7f598aa00640 1 -- 172.21.15.86:0/520672347 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4168532743 0 0) 0x7f598c0cd8d0 con 0x7f598c0d2c80 2024-04-25T14:08:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.682+0000 7f598be00640 1 --2- 172.21.15.86:0/520672347 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f598c071530 0x7f598c15dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.682+0000 7f598be00640 1 -- 172.21.15.86:0/520672347 >> v1:172.21.15.86:6789/0 conn(0x7f598c0d2c80 legacy=0x7f598c070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.682+0000 7f598be00640 1 -- 172.21.15.86:0/520672347 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f598c0d1be0 con 0x7f598c1616e0 2024-04-25T14:08:32.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.682+0000 7f5982a00640 1 --2- 172.21.15.86:0/520672347 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f598c071530 0x7f598c15dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:32.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.686+0000 7f598be00640 1 --2- 172.21.15.86:0/520672347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f598c1616e0 0x7f598c161af0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f5984009d90 tx=0x7f598401f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7394243fb760f94d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:32.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.686+0000 7f598aa00640 1 -- 172.21.15.86:0/520672347 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59840b4070 con 0x7f598c1616e0 2024-04-25T14:08:32.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.686+0000 7f598aa00640 1 -- 172.21.15.86:0/520672347 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59840af600 con 0x7f598c1616e0 2024-04-25T14:08:32.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.686+0000 7f598aa00640 1 -- 172.21.15.86:0/520672347 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59840af9d0 con 0x7f598c1616e0 2024-04-25T14:08:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.686+0000 7f5992000640 1 -- 172.21.15.86:0/520672347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f598c1616e0 msgr2=0x7f598c161af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.686+0000 7f5992000640 1 --2- 172.21.15.86:0/520672347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f598c1616e0 0x7f598c161af0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f5984009d90 tx=0x7f598401f150 comp rx=0 tx=0).stop 2024-04-25T14:08:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.686+0000 7f5992000640 1 -- 172.21.15.86:0/520672347 shutdown_connections 2024-04-25T14:08:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.686+0000 7f5992000640 1 --2- 172.21.15.86:0/520672347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f598c1616e0 0x7f598c161af0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.686+0000 7f5992000640 1 --2- 172.21.15.86:0/520672347 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f598c071530 0x7f598c15dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.686+0000 7f5992000640 1 -- 172.21.15.86:0/520672347 >> 172.21.15.86:0/520672347 conn(0x7f598c0cffc0 msgr2=0x7f598c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.686+0000 7f5992000640 1 -- 172.21.15.86:0/520672347 shutdown_connections 2024-04-25T14:08:32.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.686+0000 7f5992000640 1 -- 172.21.15.86:0/520672347 wait complete. 2024-04-25T14:08:32.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.690+0000 7f5992000640 1 Processor -- start 2024-04-25T14:08:32.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.690+0000 7f5992000640 1 -- start start 2024-04-25T14:08:32.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.690+0000 7f5992000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f598c0d2c80 0x7f598c06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:32.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.690+0000 7f5992000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f598c1616e0 0x7f598c15a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:32.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.690+0000 7f5992000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f598c0b2fa0 0x7f598c1537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:32.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.690+0000 7f5992000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f598c0d2480 con 0x7f598c0d2c80 2024-04-25T14:08:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.690+0000 7f5992000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f598c0d21a0 con 0x7f598c1616e0 2024-04-25T14:08:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.690+0000 7f5992000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f598c0d29a0 con 0x7f598c0b2fa0 2024-04-25T14:08:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.690+0000 7f598b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f598c0d2c80 0x7f598c06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.690+0000 7f598b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f598c0d2c80 0x7f598c06f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43502/0 (socket says 172.21.15.86:43502) 2024-04-25T14:08:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.690+0000 7f598b400640 1 -- 172.21.15.86:0/2103132123 learned_addr learned my addr 172.21.15.86:0/2103132123 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.694+0000 7f598be00640 1 --2- 172.21.15.86:0/2103132123 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f598c0b2fa0 0x7f598c1537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.694+0000 7f5982a00640 1 --2- 172.21.15.86:0/2103132123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f598c1616e0 0x7f598c15a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.694+0000 7f598b400640 1 -- 172.21.15.86:0/2103132123 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f598c0b2fa0 msgr2=0x7f598c1537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.694+0000 7f598b400640 1 --2- 172.21.15.86:0/2103132123 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f598c0b2fa0 0x7f598c1537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.694+0000 7f598b400640 1 -- 172.21.15.86:0/2103132123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f598c1616e0 msgr2=0x7f598c15a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.694+0000 7f598b400640 1 --2- 172.21.15.86:0/2103132123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f598c1616e0 0x7f598c15a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.694+0000 7f598b400640 1 -- 172.21.15.86:0/2103132123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f598c1f6900 con 0x7f598c0d2c80 2024-04-25T14:08:32.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.694+0000 7f598b400640 1 --2- 172.21.15.86:0/2103132123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f598c0d2c80 0x7f598c06f110 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f597c00c480 tx=0x7f597c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:32.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.694+0000 7f5983e00640 1 -- 172.21.15.86:0/2103132123 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597c018070 con 0x7f598c0d2c80 2024-04-25T14:08:32.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.694+0000 7f5983e00640 1 -- 172.21.15.86:0/2103132123 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f597c007500 con 0x7f598c0d2c80 2024-04-25T14:08:32.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.694+0000 7f5983e00640 1 -- 172.21.15.86:0/2103132123 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597c004a30 con 0x7f598c0d2c80 2024-04-25T14:08:32.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.694+0000 7f5992000640 1 -- 172.21.15.86:0/2103132123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f598c1f6be0 con 0x7f598c0d2c80 2024-04-25T14:08:32.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.694+0000 7f5992000640 1 -- 172.21.15.86:0/2103132123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f598c1f71a0 con 0x7f598c0d2c80 2024-04-25T14:08:32.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.694+0000 7f5992000640 1 -- 172.21.15.86:0/2103132123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f598c076bf0 con 0x7f598c0d2c80 2024-04-25T14:08:32.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.694+0000 7f5983e00640 1 -- 172.21.15.86:0/2103132123 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 27) ==== 93253+0+0 (secure 0 0 0) 0x7f597c01b070 con 0x7f598c0d2c80 2024-04-25T14:08:32.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.694+0000 7f5983e00640 1 --2- 172.21.15.86:0/2103132123 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f596806f490 0x7f59680719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:32.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.698+0000 7f5983e00640 1 -- 172.21.15.86:0/2103132123 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6875+0+0 (secure 0 0 0) 0x7f597c08eae0 con 0x7f598c0d2c80 2024-04-25T14:08:32.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.698+0000 7f5982a00640 1 --2- 172.21.15.86:0/2103132123 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f596806f490 0x7f59680719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:32.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.698+0000 7f5982a00640 1 --2- 172.21.15.86:0/2103132123 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f596806f490 0x7f59680719a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5974006fd0 tx=0x7f5974008040 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:32.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.702+0000 7f5983e00640 1 -- 172.21.15.86:0/2103132123 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f597c05b570 con 0x7f598c0d2c80 2024-04-25T14:08:32.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.890+0000 7f5992000640 1 -- 172.21.15.86:0/2103132123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f598c0d1470 con 0x7f598c0d2c80 2024-04-25T14:08:32.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.894+0000 7f5983e00640 1 -- 172.21.15.86:0/2103132123 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5210 (secure 0 0 0) 0x7f597c05ae70 con 0x7f598c0d2c80 2024-04-25T14:08:32.894 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:08:32.894 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":58,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1210051860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1210051860},{"type":"v1","addr":"172.21.15.155:6833","nonce":1210051860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2868300438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2868300438},{"type":"v1","addr":"172.21.15.155:6835","nonce":2868300438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:06:44.910717+0000","modified":"2024-04-25T14:08:29.703081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6603,"mds_1":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"c","rank":1,"incarnation":55,"state":"up:active","state_seq":9,"addr":"172.21.15.86:6835/2682115891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2682115891},{"type":"v1","addr":"172.21.15.86:6835","nonce":2682115891}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6603":{"gid":6603,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/1088108173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1088108173},{"type":"v1","addr":"172.21.15.86:6837","nonce":1088108173}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6603,"qdb_cluster":[6600,6603]},"id":5}]} 2024-04-25T14:08:32.895 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 58 2024-04-25T14:08:32.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.894+0000 7f5992000640 1 -- 172.21.15.86:0/2103132123 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f596806f490 msgr2=0x7f59680719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:32.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.894+0000 7f5992000640 1 --2- 172.21.15.86:0/2103132123 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f596806f490 0x7f59680719a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5974006fd0 tx=0x7f5974008040 comp rx=0 tx=0).stop 2024-04-25T14:08:32.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.894+0000 7f5992000640 1 -- 172.21.15.86:0/2103132123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f598c0d2c80 msgr2=0x7f598c06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:32.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.894+0000 7f5992000640 1 --2- 172.21.15.86:0/2103132123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f598c0d2c80 0x7f598c06f110 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f597c00c480 tx=0x7f597c00c940 comp rx=0 tx=0).stop 2024-04-25T14:08:32.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.894+0000 7f5992000640 1 -- 172.21.15.86:0/2103132123 shutdown_connections 2024-04-25T14:08:32.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.894+0000 7f5992000640 1 --2- 172.21.15.86:0/2103132123 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f596806f490 0x7f59680719a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:32.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.894+0000 7f5992000640 1 --2- 172.21.15.86:0/2103132123 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f598c0b2fa0 0x7f598c1537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:32.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.894+0000 7f5992000640 1 --2- 172.21.15.86:0/2103132123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f598c1616e0 0x7f598c15a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:32.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.894+0000 7f5992000640 1 --2- 172.21.15.86:0/2103132123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f598c0d2c80 0x7f598c06f110 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:32.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.894+0000 7f5992000640 1 -- 172.21.15.86:0/2103132123 >> 172.21.15.86:0/2103132123 conn(0x7f598c0cffc0 msgr2=0x7f598c0c99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:32.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.894+0000 7f5992000640 1 -- 172.21.15.86:0/2103132123 shutdown_connections 2024-04-25T14:08:32.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:32.894+0000 7f5992000640 1 -- 172.21.15.86:0/2103132123 wait complete. 2024-04-25T14:08:32.916 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:08:33.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.062+0000 7f2261800640 1 Processor -- start 2024-04-25T14:08:33.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.062+0000 7f2261800640 1 -- start start 2024-04-25T14:08:33.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.062+0000 7f2261800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f225c0d1df0 0x7f225c0cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:33.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.062+0000 7f2261800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f225c0cdcc0 0x7f225c0ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:33.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.062+0000 7f2261800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f225c15e5c0 con 0x7f225c070330 2024-04-25T14:08:33.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.062+0000 7f2261800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f225c06fd40 con 0x7f225c0d1df0 2024-04-25T14:08:33.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.062+0000 7f2261800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f225c15e8a0 con 0x7f225c0cdcc0 2024-04-25T14:08:33.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.062+0000 7f225b400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f225c0cdcc0 0x7f225c0ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:33.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.062+0000 7f225a000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f225c0d1df0 0x7f225c0cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:33.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.062+0000 7f225a000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f225c0d1df0 0x7f225c0cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:42340/0 (socket says 172.21.15.86:42340) 2024-04-25T14:08:33.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.062+0000 7f225b400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f225c0cdcc0 0x7f225c0ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:46120/0 (socket says 172.21.15.86:46120) 2024-04-25T14:08:33.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.062+0000 7f225a000640 1 -- 172.21.15.86:0/2900260325 learned_addr learned my addr 172.21.15.86:0/2900260325 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:33.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.062+0000 7f225b400640 1 -- 172.21.15.86:0/2900260325 >> v2:172.21.15.155:3300/0 conn(0x7f225c0d1df0 msgr2=0x7f225c0cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:33.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.062+0000 7f225b400640 1 --2- 172.21.15.86:0/2900260325 >> v2:172.21.15.155:3300/0 conn(0x7f225c0d1df0 0x7f225c0cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:33.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.062+0000 7f225b400640 1 -- 172.21.15.86:0/2900260325 >> v1:172.21.15.155:6789/0 conn(0x7f225c070330 legacy=0x7f225c0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:33.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.062+0000 7f225b400640 1 -- 172.21.15.86:0/2900260325 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225c15eb80 con 0x7f225c0cdcc0 2024-04-25T14:08:33.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.062+0000 7f225a000640 1 --2- 172.21.15.86:0/2900260325 >> v2:172.21.15.155:3300/0 conn(0x7f225c0d1df0 0x7f225c0cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:08:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.066+0000 7f225b400640 1 --2- 172.21.15.86:0/2900260325 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f225c0cdcc0 0x7f225c0ce0d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f22540097c0 tx=0x7f225401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85c970e2c100ddbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.066+0000 7f2259600640 1 -- 172.21.15.86:0/2900260325 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22540b1070 con 0x7f225c0cdcc0 2024-04-25T14:08:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.066+0000 7f2259600640 1 -- 172.21.15.86:0/2900260325 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22540ac620 con 0x7f225c0cdcc0 2024-04-25T14:08:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.066+0000 7f2259600640 1 -- 172.21.15.86:0/2900260325 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22540ac9f0 con 0x7f225c0cdcc0 2024-04-25T14:08:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.066+0000 7f2261800640 1 -- 172.21.15.86:0/2900260325 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f225c0cdcc0 msgr2=0x7f225c0ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.066+0000 7f2261800640 1 --2- 172.21.15.86:0/2900260325 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f225c0cdcc0 0x7f225c0ce0d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f22540097c0 tx=0x7f225401c3e0 comp rx=0 tx=0).stop 2024-04-25T14:08:33.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.066+0000 7f2261800640 1 -- 172.21.15.86:0/2900260325 shutdown_connections 2024-04-25T14:08:33.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.066+0000 7f2261800640 1 --2- 172.21.15.86:0/2900260325 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f225c0cdcc0 0x7f225c0ce0d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:33.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.066+0000 7f2261800640 1 --2- 172.21.15.86:0/2900260325 >> v2:172.21.15.155:3300/0 conn(0x7f225c0d1df0 0x7f225c0cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:33.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.066+0000 7f2261800640 1 -- 172.21.15.86:0/2900260325 >> 172.21.15.86:0/2900260325 conn(0x7f225c0b4650 msgr2=0x7f225c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:33.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.066+0000 7f2261800640 1 -- 172.21.15.86:0/2900260325 shutdown_connections 2024-04-25T14:08:33.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.066+0000 7f2261800640 1 -- 172.21.15.86:0/2900260325 wait complete. 2024-04-25T14:08:33.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.070+0000 7f2261800640 1 Processor -- start 2024-04-25T14:08:33.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.070+0000 7f2261800640 1 -- start start 2024-04-25T14:08:33.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.070+0000 7f2261800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f225c070330 0x7f225c0af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:33.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.070+0000 7f2261800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f225c0cdcc0 0x7f225c0cf610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:33.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.070+0000 7f2261800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f225c15b500 0x7f225c0bbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:33.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.070+0000 7f2261800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f225c15f4b0 con 0x7f225c0cdcc0 2024-04-25T14:08:33.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.070+0000 7f2261800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f225c15f330 con 0x7f225c070330 2024-04-25T14:08:33.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.070+0000 7f2261800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f225c070740 con 0x7f225c15b500 2024-04-25T14:08:33.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.070+0000 7f225a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f225c0cdcc0 0x7f225c0cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:33.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.070+0000 7f225a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f225c0cdcc0 0x7f225c0cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43504/0 (socket says 172.21.15.86:43504) 2024-04-25T14:08:33.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.070+0000 7f225a000640 1 -- 172.21.15.86:0/2215901467 learned_addr learned my addr 172.21.15.86:0/2215901467 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:33.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.070+0000 7f225b400640 1 --2- 172.21.15.86:0/2215901467 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f225c15b500 0x7f225c0bbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:33.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.070+0000 7f225aa00640 1 --2- 172.21.15.86:0/2215901467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f225c070330 0x7f225c0af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:33.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.070+0000 7f225a000640 1 -- 172.21.15.86:0/2215901467 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f225c15b500 msgr2=0x7f225c0bbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:33.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.070+0000 7f225a000640 1 --2- 172.21.15.86:0/2215901467 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f225c15b500 0x7f225c0bbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:33.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.070+0000 7f225a000640 1 -- 172.21.15.86:0/2215901467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f225c070330 msgr2=0x7f225c0af5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:33.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.070+0000 7f225a000640 1 --2- 172.21.15.86:0/2215901467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f225c070330 0x7f225c0af5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:33.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.070+0000 7f225a000640 1 -- 172.21.15.86:0/2215901467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225c0afaf0 con 0x7f225c0cdcc0 2024-04-25T14:08:33.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.074+0000 7f225a000640 1 --2- 172.21.15.86:0/2215901467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f225c0cdcc0 0x7f225c0cf610 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f224800e980 tx=0x7f224800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:33.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.074+0000 7f2252a00640 1 -- 172.21.15.86:0/2215901467 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2248017070 con 0x7f225c0cdcc0 2024-04-25T14:08:33.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.074+0000 7f2252a00640 1 -- 172.21.15.86:0/2215901467 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f224800ca00 con 0x7f225c0cdcc0 2024-04-25T14:08:33.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.074+0000 7f2261800640 1 -- 172.21.15.86:0/2215901467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f225c0ae510 con 0x7f225c0cdcc0 2024-04-25T14:08:33.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.074+0000 7f2252a00640 1 -- 172.21.15.86:0/2215901467 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224800cdf0 con 0x7f225c0cdcc0 2024-04-25T14:08:33.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.074+0000 7f2261800640 1 -- 172.21.15.86:0/2215901467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f225c0aead0 con 0x7f225c0cdcc0 2024-04-25T14:08:33.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.074+0000 7f2261800640 1 -- 172.21.15.86:0/2215901467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f225c0ae280 con 0x7f225c0cdcc0 2024-04-25T14:08:33.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.074+0000 7f2252a00640 1 -- 172.21.15.86:0/2215901467 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 27) ==== 93253+0+0 (secure 0 0 0) 0x7f224801a070 con 0x7f225c0cdcc0 2024-04-25T14:08:33.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.074+0000 7f2252a00640 1 --2- 172.21.15.86:0/2215901467 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f223c06f490 0x7f223c0719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:33.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.074+0000 7f2252a00640 1 -- 172.21.15.86:0/2215901467 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6875+0+0 (secure 0 0 0) 0x7f224808fee0 con 0x7f225c0cdcc0 2024-04-25T14:08:33.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.074+0000 7f225aa00640 1 --2- 172.21.15.86:0/2215901467 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f223c06f490 0x7f223c0719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:33.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.078+0000 7f225aa00640 1 --2- 172.21.15.86:0/2215901467 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f223c06f490 0x7f223c0719a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f225c072200 tx=0x7f224c009040 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:33.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.078+0000 7f2252a00640 1 -- 172.21.15.86:0/2215901467 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f224805c8d0 con 0x7f225c0cdcc0 2024-04-25T14:08:33.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.234+0000 7f2261800640 1 -- 172.21.15.86:0/2215901467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f225c069d00 con 0x7f225c0cdcc0 2024-04-25T14:08:33.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.238+0000 7f2252a00640 1 -- 172.21.15.86:0/2215901467 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+18065 (secure 0 0 0) 0x7f224805c1d0 con 0x7f225c0cdcc0 2024-04-25T14:08:33.239 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:08:33.240 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":76,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:08:29.333499+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:06:42.666742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T14:06:43.884074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2066326917":"2024-04-26T14:08:29.333458+0000","172.21.15.86:6833/203029814":"2024-04-26T14:08:29.333458+0000","172.21.15.86:6832/203029814":"2024-04-26T14:08:29.333458+0000","172.21.15.86:0/3991804772":"2024-04-26T14:08:29.333458+0000","172.21.15.86:0/2307768874":"2024-04-26T14:08:29.333458+0000","172.21.15.86:0/2510298865":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/3380489666":"2024-04-26T14:08:29.333458+0000","172.21.15.86:0/82848493":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/2452803132":"2024-04-26T14:08:29.333458+0000","172.21.15.86:0/1793604306":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/3402941646":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/112097751":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/2567754949":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/2334647603":"2024-04-26T14:08:29.333458+0000","172.21.15.86:0/529842883":"2024-04-26T14:07:03.055680+0000","172.21.15.86:6833/1025829557":"2024-04-26T14:07:03.055680+0000","172.21.15.86:6832/1025829557":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/759420784":"2024-04-26T14:07:03.055680+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:08:33.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.238+0000 7f2261800640 1 -- 172.21.15.86:0/2215901467 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f223c06f490 msgr2=0x7f223c0719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:33.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.238+0000 7f2261800640 1 --2- 172.21.15.86:0/2215901467 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f223c06f490 0x7f223c0719a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f225c072200 tx=0x7f224c009040 comp rx=0 tx=0).stop 2024-04-25T14:08:33.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.238+0000 7f2261800640 1 -- 172.21.15.86:0/2215901467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f225c0cdcc0 msgr2=0x7f225c0cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:33.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.238+0000 7f2261800640 1 --2- 172.21.15.86:0/2215901467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f225c0cdcc0 0x7f225c0cf610 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f224800e980 tx=0x7f224800ee40 comp rx=0 tx=0).stop 2024-04-25T14:08:33.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.238+0000 7f2261800640 1 -- 172.21.15.86:0/2215901467 shutdown_connections 2024-04-25T14:08:33.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.238+0000 7f2261800640 1 --2- 172.21.15.86:0/2215901467 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f223c06f490 0x7f223c0719a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:33.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.238+0000 7f2261800640 1 --2- 172.21.15.86:0/2215901467 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f225c15b500 0x7f225c0bbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:33.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.238+0000 7f2261800640 1 --2- 172.21.15.86:0/2215901467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f225c0cdcc0 0x7f225c0cf610 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:33.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.238+0000 7f2261800640 1 --2- 172.21.15.86:0/2215901467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f225c070330 0x7f225c0af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:33.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.238+0000 7f2261800640 1 -- 172.21.15.86:0/2215901467 >> 172.21.15.86:0/2215901467 conn(0x7f225c0b4650 msgr2=0x7f225c0c4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:33.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.238+0000 7f2261800640 1 -- 172.21.15.86:0/2215901467 shutdown_connections 2024-04-25T14:08:33.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.238+0000 7f2261800640 1 -- 172.21.15.86:0/2215901467 wait complete. 2024-04-25T14:08:33.261 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T14:08:33.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.414+0000 7f9c57000640 1 Processor -- start 2024-04-25T14:08:33.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.414+0000 7f9c57000640 1 -- start start 2024-04-25T14:08:33.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.414+0000 7f9c57000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c500d1df0 0x7f9c500cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:33.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.414+0000 7f9c57000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c5015e5c0 con 0x7f9c500cdcc0 2024-04-25T14:08:33.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.414+0000 7f9c57000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c5015e8a0 con 0x7f9c50070330 2024-04-25T14:08:33.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.414+0000 7f9c57000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9c5006fd40 con 0x7f9c500d1df0 2024-04-25T14:08:33.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.414+0000 7f9c55000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9c500cdcc0 0x7f9c500ce0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:58426/0 (socket says 172.21.15.86:58426) 2024-04-25T14:08:33.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.414+0000 7f9c55000640 1 -- 172.21.15.86:0/3324710472 learned_addr learned my addr 172.21.15.86:0/3324710472 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:33.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.414+0000 7f9c4f400640 1 --2- 172.21.15.86:0/3324710472 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c500d1df0 0x7f9c500cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:33.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.414+0000 7f9c4ea00640 1 -- 172.21.15.86:0/3324710472 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 972098418 0 0) 0x7f9c5015e5c0 con 0x7f9c500cdcc0 2024-04-25T14:08:33.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.414+0000 7f9c4ea00640 1 -- 172.21.15.86:0/3324710472 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c34003530 con 0x7f9c500cdcc0 2024-04-25T14:08:33.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.414+0000 7f9c4ea00640 1 -- 172.21.15.86:0/3324710472 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2085983578 0 0) 0x7f9c5015e8a0 con 0x7f9c50070330 2024-04-25T14:08:33.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.414+0000 7f9c4ea00640 1 -- 172.21.15.86:0/3324710472 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c5015e5c0 con 0x7f9c50070330 2024-04-25T14:08:33.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.414+0000 7f9c4f400640 1 -- 172.21.15.86:0/3324710472 >> v1:172.21.15.155:6789/0 conn(0x7f9c50070330 legacy=0x7f9c500d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:33.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.414+0000 7f9c4f400640 1 -- 172.21.15.86:0/3324710472 >> v1:172.21.15.86:6789/0 conn(0x7f9c500cdcc0 legacy=0x7f9c500ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:33.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.414+0000 7f9c4f400640 1 -- 172.21.15.86:0/3324710472 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5015eb80 con 0x7f9c500d1df0 2024-04-25T14:08:33.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.418+0000 7f9c4f400640 1 --2- 172.21.15.86:0/3324710472 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c500d1df0 0x7f9c500cd780 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c0097f0 tx=0x7f9c3c01c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c365f973aa2e95b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:33.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.418+0000 7f9c4ea00640 1 -- 172.21.15.86:0/3324710472 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c0b0070 con 0x7f9c500d1df0 2024-04-25T14:08:33.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.418+0000 7f9c4ea00640 1 -- 172.21.15.86:0/3324710472 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c3c0ab600 con 0x7f9c500d1df0 2024-04-25T14:08:33.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.418+0000 7f9c4ea00640 1 -- 172.21.15.86:0/3324710472 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c0ab9d0 con 0x7f9c500d1df0 2024-04-25T14:08:33.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.418+0000 7f9c57000640 1 -- 172.21.15.86:0/3324710472 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c500d1df0 msgr2=0x7f9c500cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:33.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.418+0000 7f9c57000640 1 --2- 172.21.15.86:0/3324710472 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c500d1df0 0x7f9c500cd780 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c0097f0 tx=0x7f9c3c01c420 comp rx=0 tx=0).stop 2024-04-25T14:08:33.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.418+0000 7f9c57000640 1 -- 172.21.15.86:0/3324710472 shutdown_connections 2024-04-25T14:08:33.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.418+0000 7f9c57000640 1 --2- 172.21.15.86:0/3324710472 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c500d1df0 0x7f9c500cd780 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:33.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.418+0000 7f9c57000640 1 -- 172.21.15.86:0/3324710472 >> 172.21.15.86:0/3324710472 conn(0x7f9c500b4650 msgr2=0x7f9c500b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:33.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.418+0000 7f9c57000640 1 -- 172.21.15.86:0/3324710472 shutdown_connections 2024-04-25T14:08:33.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.418+0000 7f9c57000640 1 -- 172.21.15.86:0/3324710472 wait complete. 2024-04-25T14:08:33.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.422+0000 7f9c57000640 1 Processor -- start 2024-04-25T14:08:33.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.422+0000 7f9c57000640 1 -- start start 2024-04-25T14:08:33.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.422+0000 7f9c57000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c50070330 0x7f9c5009e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:33.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.422+0000 7f9c57000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c500cdcc0 0x7f9c50155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:33.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.422+0000 7f9c57000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c50153e70 0x7f9c50156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:33.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.422+0000 7f9c57000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c500708c0 con 0x7f9c50070330 2024-04-25T14:08:33.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.422+0000 7f9c57000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9c50070740 con 0x7f9c500cdcc0 2024-04-25T14:08:33.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.422+0000 7f9c57000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9c5016bfd0 con 0x7f9c50153e70 2024-04-25T14:08:33.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.422+0000 7f9c4fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c50070330 0x7f9c5009e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:33.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.422+0000 7f9c4fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c50070330 0x7f9c5009e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43506/0 (socket says 172.21.15.86:43506) 2024-04-25T14:08:33.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.422+0000 7f9c4fe00640 1 -- 172.21.15.86:0/82440121 learned_addr learned my addr 172.21.15.86:0/82440121 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:33.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.422+0000 7f9c55000640 1 --2- 172.21.15.86:0/82440121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c50153e70 0x7f9c50156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:33.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.422+0000 7f9c4f400640 1 --2- 172.21.15.86:0/82440121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c500cdcc0 0x7f9c50155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:33.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.426+0000 7f9c4fe00640 1 -- 172.21.15.86:0/82440121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c50153e70 msgr2=0x7f9c50156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:33.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.426+0000 7f9c4fe00640 1 --2- 172.21.15.86:0/82440121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c50153e70 0x7f9c50156170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:33.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.426+0000 7f9c4fe00640 1 -- 172.21.15.86:0/82440121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c500cdcc0 msgr2=0x7f9c50155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:33.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.426+0000 7f9c4fe00640 1 --2- 172.21.15.86:0/82440121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c500cdcc0 0x7f9c50155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:33.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.426+0000 7f9c4fe00640 1 -- 172.21.15.86:0/82440121 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5009a5b0 con 0x7f9c50070330 2024-04-25T14:08:33.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.426+0000 7f9c4fe00640 1 --2- 172.21.15.86:0/82440121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c50070330 0x7f9c5009e210 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f9c4000c3f0 tx=0x7f9c4000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:33.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.426+0000 7f9c47e00640 1 -- 172.21.15.86:0/82440121 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c40017070 con 0x7f9c50070330 2024-04-25T14:08:33.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.426+0000 7f9c47e00640 1 -- 172.21.15.86:0/82440121 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c400052d0 con 0x7f9c50070330 2024-04-25T14:08:33.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.426+0000 7f9c57000640 1 -- 172.21.15.86:0/82440121 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c50155e60 con 0x7f9c50070330 2024-04-25T14:08:33.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.426+0000 7f9c47e00640 1 -- 172.21.15.86:0/82440121 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c40003840 con 0x7f9c50070330 2024-04-25T14:08:33.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.426+0000 7f9c57000640 1 -- 172.21.15.86:0/82440121 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c5009b630 con 0x7f9c50070330 2024-04-25T14:08:33.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.426+0000 7f9c57000640 1 -- 172.21.15.86:0/82440121 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c50069d00 con 0x7f9c50070330 2024-04-25T14:08:33.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.426+0000 7f9c47e00640 1 -- 172.21.15.86:0/82440121 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7f9c4001a070 con 0x7f9c50070330 2024-04-25T14:08:33.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.426+0000 7f9c47e00640 1 --2- 172.21.15.86:0/82440121 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f9c3006f520 0x7f9c30071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:33.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.426+0000 7f9c47e00640 1 -- 172.21.15.86:0/82440121 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6875+0+0 (secure 0 0 0) 0x7f9c4008ec90 con 0x7f9c50070330 2024-04-25T14:08:33.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.426+0000 7f9c4f400640 1 --2- 172.21.15.86:0/82440121 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f9c3006f520 0x7f9c30071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:33.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.430+0000 7f9c4f400640 1 --2- 172.21.15.86:0/82440121 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f9c3006f520 0x7f9c30071a30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c0ad000 tx=0x7f9c3c0a6000 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:33.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.434+0000 7f9c47e00640 1 -- 172.21.15.86:0/82440121 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9c40005480 con 0x7f9c50070330 2024-04-25T14:08:33.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.622+0000 7f9c57000640 1 -- 172.21.15.86:0/82440121 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9c5006d940 con 0x7f9c50070330 2024-04-25T14:08:33.624 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-25T14:08:33.622+0000 7fea29c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:08:33.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.622+0000 7f9c47e00640 1 -- 172.21.15.86:0/82440121 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v59) ==== 161+0+0 (secure 0 0 0) 0x7f9c4005aef0 con 0x7f9c50070330 2024-04-25T14:08:33.626 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T14:08:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.626+0000 7f9c57000640 1 -- 172.21.15.86:0/82440121 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f9c3006f520 msgr2=0x7f9c30071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.626+0000 7f9c57000640 1 --2- 172.21.15.86:0/82440121 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f9c3006f520 0x7f9c30071a30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c0ad000 tx=0x7f9c3c0a6000 comp rx=0 tx=0).stop 2024-04-25T14:08:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.626+0000 7f9c57000640 1 -- 172.21.15.86:0/82440121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c50070330 msgr2=0x7f9c5009e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.626+0000 7f9c57000640 1 --2- 172.21.15.86:0/82440121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c50070330 0x7f9c5009e210 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f9c4000c3f0 tx=0x7f9c4000c8b0 comp rx=0 tx=0).stop 2024-04-25T14:08:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.626+0000 7f9c57000640 1 -- 172.21.15.86:0/82440121 shutdown_connections 2024-04-25T14:08:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.626+0000 7f9c57000640 1 --2- 172.21.15.86:0/82440121 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f9c3006f520 0x7f9c30071a30 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c0ad000 tx=0x7f9c3c0a6000 comp rx=0 tx=0).stop 2024-04-25T14:08:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.626+0000 7f9c57000640 1 --2- 172.21.15.86:0/82440121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c50153e70 0x7f9c50156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.626+0000 7f9c57000640 1 --2- 172.21.15.86:0/82440121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c500cdcc0 0x7f9c50155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.626+0000 7f9c57000640 1 --2- 172.21.15.86:0/82440121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c50070330 0x7f9c5009e210 secure :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f9c4000c3f0 tx=0x7f9c4000c8b0 comp rx=0 tx=0).stop 2024-04-25T14:08:33.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.626+0000 7f9c57000640 1 -- 172.21.15.86:0/82440121 >> 172.21.15.86:0/82440121 conn(0x7f9c500b4650 msgr2=0x7f9c50073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.630+0000 7f9c57000640 1 -- 172.21.15.86:0/82440121 shutdown_connections 2024-04-25T14:08:33.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.630+0000 7f9c57000640 1 -- 172.21.15.86:0/82440121 wait complete. 2024-04-25T14:08:33.654 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T14:08:33.769 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-04-25T14:08:33.770 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-25T14:08:33.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.814+0000 7f9281a00640 1 Processor -- start 2024-04-25T14:08:33.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.814+0000 7f9281a00640 1 -- start start 2024-04-25T14:08:33.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.818+0000 7f9281a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f927c071530 0x7f927c15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:33.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.818+0000 7f9281a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f927c1616e0 0x7f927c161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:33.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.818+0000 7f9281a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f927c0cd8d0 con 0x7f927c0d2c80 2024-04-25T14:08:33.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.818+0000 7f9281a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f927c0ce720 con 0x7f927c1616e0 2024-04-25T14:08:33.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.818+0000 7f9281a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f927c0ce970 con 0x7f927c071530 2024-04-25T14:08:33.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.818+0000 7f927b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f927c1616e0 0x7f927c161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:33.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.818+0000 7f927b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f927c1616e0 0x7f927c161af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43530/0 (socket says 172.21.15.86:43530) 2024-04-25T14:08:33.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.818+0000 7f927b400640 1 -- 172.21.15.86:0/1861829694 learned_addr learned my addr 172.21.15.86:0/1861829694 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:33.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.818+0000 7f9273e00640 1 --2- 172.21.15.86:0/1861829694 >> v2:172.21.15.155:3300/0 conn(0x7f927c071530 0x7f927c15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:33.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.818+0000 7f927b400640 1 -- 172.21.15.86:0/1861829694 >> v2:172.21.15.155:3300/0 conn(0x7f927c071530 msgr2=0x7f927c15dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:33.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.818+0000 7f927b400640 1 --2- 172.21.15.86:0/1861829694 >> v2:172.21.15.155:3300/0 conn(0x7f927c071530 0x7f927c15dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:33.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.818+0000 7f927b400640 1 -- 172.21.15.86:0/1861829694 >> v1:172.21.15.155:6789/0 conn(0x7f927c0d2c80 legacy=0x7f927c070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:33.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.818+0000 7f927b400640 1 -- 172.21.15.86:0/1861829694 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f927c0d1be0 con 0x7f927c1616e0 2024-04-25T14:08:33.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.818+0000 7f927b400640 1 --2- 172.21.15.86:0/1861829694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f927c1616e0 0x7f927c161af0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f927c0b6070 tx=0x7f927401f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b1842cb3f4b73a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:33.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.818+0000 7f927a000640 1 -- 172.21.15.86:0/1861829694 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92740b4070 con 0x7f927c1616e0 2024-04-25T14:08:33.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.822+0000 7f927a000640 1 -- 172.21.15.86:0/1861829694 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92740af400 con 0x7f927c1616e0 2024-04-25T14:08:33.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.822+0000 7f927a000640 1 -- 172.21.15.86:0/1861829694 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92740af7d0 con 0x7f927c1616e0 2024-04-25T14:08:33.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.822+0000 7f9281a00640 1 -- 172.21.15.86:0/1861829694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f927c1616e0 msgr2=0x7f927c161af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:33.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.822+0000 7f9281a00640 1 --2- 172.21.15.86:0/1861829694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f927c1616e0 0x7f927c161af0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f927c0b6070 tx=0x7f927401f1e0 comp rx=0 tx=0).stop 2024-04-25T14:08:33.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.822+0000 7f9281a00640 1 -- 172.21.15.86:0/1861829694 shutdown_connections 2024-04-25T14:08:33.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.822+0000 7f9281a00640 1 --2- 172.21.15.86:0/1861829694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f927c1616e0 0x7f927c161af0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:33.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.822+0000 7f9281a00640 1 --2- 172.21.15.86:0/1861829694 >> v2:172.21.15.155:3300/0 conn(0x7f927c071530 0x7f927c15dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:33.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.822+0000 7f9281a00640 1 -- 172.21.15.86:0/1861829694 >> 172.21.15.86:0/1861829694 conn(0x7f927c0cffc0 msgr2=0x7f927c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:33.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.822+0000 7f9281a00640 1 -- 172.21.15.86:0/1861829694 shutdown_connections 2024-04-25T14:08:33.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.822+0000 7f9281a00640 1 -- 172.21.15.86:0/1861829694 wait complete. 2024-04-25T14:08:33.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.822+0000 7f9281a00640 1 Processor -- start 2024-04-25T14:08:33.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.822+0000 7f9281a00640 1 -- start start 2024-04-25T14:08:33.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.826+0000 7f9281a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f927c0d2c80 0x7f927c157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:33.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.826+0000 7f9281a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f927c1616e0 0x7f927c15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:33.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.826+0000 7f9281a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f927c15e1d0 0x7f927c15e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:33.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.826+0000 7f9281a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f927c0d2480 con 0x7f927c1616e0 2024-04-25T14:08:33.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.826+0000 7f9281a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f927c0d21a0 con 0x7f927c0d2c80 2024-04-25T14:08:33.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.826+0000 7f9281a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f927c0d29a0 con 0x7f927c15e1d0 2024-04-25T14:08:33.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.826+0000 7f9273e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f927c1616e0 0x7f927c15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:33.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.826+0000 7f9273e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f927c1616e0 0x7f927c15dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43544/0 (socket says 172.21.15.86:43544) 2024-04-25T14:08:33.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.826+0000 7f927aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f927c0d2c80 0x7f927c157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:33.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.826+0000 7f9273e00640 1 -- 172.21.15.86:0/3163394367 learned_addr learned my addr 172.21.15.86:0/3163394367 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:33.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.826+0000 7f927b400640 1 --2- 172.21.15.86:0/3163394367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f927c15e1d0 0x7f927c15e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:33.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.826+0000 7f9273e00640 1 -- 172.21.15.86:0/3163394367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f927c15e1d0 msgr2=0x7f927c15e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:33.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.826+0000 7f9273e00640 1 --2- 172.21.15.86:0/3163394367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f927c15e1d0 0x7f927c15e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:33.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.826+0000 7f9273e00640 1 -- 172.21.15.86:0/3163394367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f927c0d2c80 msgr2=0x7f927c157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:33.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.826+0000 7f9273e00640 1 --2- 172.21.15.86:0/3163394367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f927c0d2c80 0x7f927c157da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:33.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.826+0000 7f9273e00640 1 -- 172.21.15.86:0/3163394367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f927c15ed40 con 0x7f927c1616e0 2024-04-25T14:08:33.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.826+0000 7f9273e00640 1 --2- 172.21.15.86:0/3163394367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f927c1616e0 0x7f927c15dc90 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f926400e8a0 tx=0x7f926400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:33.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.826+0000 7f9272a00640 1 -- 172.21.15.86:0/3163394367 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9264017070 con 0x7f927c1616e0 2024-04-25T14:08:33.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.826+0000 7f9272a00640 1 -- 172.21.15.86:0/3163394367 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f926400c940 con 0x7f927c1616e0 2024-04-25T14:08:33.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.826+0000 7f9272a00640 1 -- 172.21.15.86:0/3163394367 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926400ccf0 con 0x7f927c1616e0 2024-04-25T14:08:33.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.826+0000 7f9281a00640 1 -- 172.21.15.86:0/3163394367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f927c15efb0 con 0x7f927c1616e0 2024-04-25T14:08:33.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.826+0000 7f9281a00640 1 -- 172.21.15.86:0/3163394367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f927c1582e0 con 0x7f927c1616e0 2024-04-25T14:08:33.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.826+0000 7f9281a00640 1 -- 172.21.15.86:0/3163394367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f927c1609c0 con 0x7f927c1616e0 2024-04-25T14:08:33.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.830+0000 7f9272a00640 1 -- 172.21.15.86:0/3163394367 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7f926401a070 con 0x7f927c1616e0 2024-04-25T14:08:33.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.830+0000 7f9272a00640 1 --2- 172.21.15.86:0/3163394367 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f925806f520 0x7f9258071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:33.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.830+0000 7f9272a00640 1 -- 172.21.15.86:0/3163394367 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7047+0+0 (secure 0 0 0) 0x7f926408fc30 con 0x7f927c1616e0 2024-04-25T14:08:33.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.830+0000 7f927aa00640 1 --2- 172.21.15.86:0/3163394367 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f925806f520 0x7f9258071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:33.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.830+0000 7f927aa00640 1 --2- 172.21.15.86:0/3163394367 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f925806f520 0x7f9258071a30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f926c001770 tx=0x7f926c009040 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:33.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:33.834+0000 7f9272a00640 1 -- 172.21.15.86:0/3163394367 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f926405be70 con 0x7f927c1616e0 2024-04-25T14:08:34.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.026+0000 7f9281a00640 1 -- 172.21.15.86:0/3163394367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f927c0720e0 con 0x7f927c1616e0 2024-04-25T14:08:34.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.626+0000 7f9272a00640 1 -- 172.21.15.86:0/3163394367 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v60) ==== 104+0+0 (secure 0 0 0) 0x7f9264060170 con 0x7f927c1616e0 2024-04-25T14:08:34.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.626+0000 7f9281a00640 1 -- 172.21.15.86:0/3163394367 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f925806f520 msgr2=0x7f9258071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:34.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.626+0000 7f9281a00640 1 --2- 172.21.15.86:0/3163394367 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f925806f520 0x7f9258071a30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f926c001770 tx=0x7f926c009040 comp rx=0 tx=0).stop 2024-04-25T14:08:34.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.626+0000 7f9281a00640 1 -- 172.21.15.86:0/3163394367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f927c1616e0 msgr2=0x7f927c15dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:34.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.626+0000 7f9281a00640 1 --2- 172.21.15.86:0/3163394367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f927c1616e0 0x7f927c15dc90 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f926400e8a0 tx=0x7f926400ed60 comp rx=0 tx=0).stop 2024-04-25T14:08:34.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.630+0000 7f9281a00640 1 -- 172.21.15.86:0/3163394367 shutdown_connections 2024-04-25T14:08:34.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.630+0000 7f9281a00640 1 --2- 172.21.15.86:0/3163394367 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f925806f520 0x7f9258071a30 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:34.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.630+0000 7f9281a00640 1 --2- 172.21.15.86:0/3163394367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f927c15e1d0 0x7f927c15e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:34.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.630+0000 7f9281a00640 1 --2- 172.21.15.86:0/3163394367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f927c1616e0 0x7f927c15dc90 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:34.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.630+0000 7f9281a00640 1 --2- 172.21.15.86:0/3163394367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f927c0d2c80 0x7f927c157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:34.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.630+0000 7f9281a00640 1 -- 172.21.15.86:0/3163394367 >> 172.21.15.86:0/3163394367 conn(0x7f927c0cffc0 msgr2=0x7f927c0c9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:34.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.630+0000 7f9281a00640 1 -- 172.21.15.86:0/3163394367 shutdown_connections 2024-04-25T14:08:34.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.630+0000 7f9281a00640 1 -- 172.21.15.86:0/3163394367 wait complete. 2024-04-25T14:08:34.652 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T14:08:34.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.802+0000 7fd520400640 1 Processor -- start 2024-04-25T14:08:34.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.802+0000 7fd520400640 1 -- start start 2024-04-25T14:08:34.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.802+0000 7fd520400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd51815e270 0x7fd51815e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:34.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.802+0000 7fd520400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5180b0b20 con 0x7fd5180b0540 2024-04-25T14:08:34.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.802+0000 7fd520400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd51806d220 con 0x7fd51815ebc0 2024-04-25T14:08:34.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.802+0000 7fd520400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd51806d870 con 0x7fd51815e270 2024-04-25T14:08:34.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.802+0000 7fd51d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd51815e270 0x7fd51815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:34.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.802+0000 7fd51d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd51815e270 0x7fd51815e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43556/0 (socket says 172.21.15.86:43556) 2024-04-25T14:08:34.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.802+0000 7fd51d000640 1 -- 172.21.15.86:0/1870940271 learned_addr learned my addr 172.21.15.86:0/1870940271 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:34.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.802+0000 7fd51da00640 1 --1- 172.21.15.86:0/1870940271 >> v1:172.21.15.86:6789/0 conn(0x7fd5180b0540 0x7fd5180ae2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:58454/0 (socket says 172.21.15.86:58454) 2024-04-25T14:08:34.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.802+0000 7fd51d000640 1 -- 172.21.15.86:0/1870940271 >> v1:172.21.15.155:6789/0 conn(0x7fd51815ebc0 legacy=0x7fd518167660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:34.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.802+0000 7fd51d000640 1 -- 172.21.15.86:0/1870940271 >> v1:172.21.15.86:6789/0 conn(0x7fd5180b0540 legacy=0x7fd5180ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:34.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.802+0000 7fd51d000640 1 -- 172.21.15.86:0/1870940271 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51806d500 con 0x7fd51815e270 2024-04-25T14:08:34.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.802+0000 7fd517e00640 1 -- 172.21.15.86:0/1870940271 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1936230147 0 0) 0x7fd5180b0b20 con 0x7fd5180b0540 2024-04-25T14:08:34.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.806+0000 7fd51d000640 1 --2- 172.21.15.86:0/1870940271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd51815e270 0x7fd51815e680 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fd504009870 tx=0x7fd50401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d4547ea09e7751 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:34.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.806+0000 7fd517e00640 1 -- 172.21.15.86:0/1870940271 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5040b0070 con 0x7fd51815e270 2024-04-25T14:08:34.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.806+0000 7fd517e00640 1 -- 172.21.15.86:0/1870940271 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5040ab600 con 0x7fd51815e270 2024-04-25T14:08:34.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.806+0000 7fd517e00640 1 -- 172.21.15.86:0/1870940271 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5040ab9d0 con 0x7fd51815e270 2024-04-25T14:08:34.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.806+0000 7fd520400640 1 -- 172.21.15.86:0/1870940271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd51815e270 msgr2=0x7fd51815e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:34.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.806+0000 7fd520400640 1 --2- 172.21.15.86:0/1870940271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd51815e270 0x7fd51815e680 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fd504009870 tx=0x7fd50401c4d0 comp rx=0 tx=0).stop 2024-04-25T14:08:34.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.806+0000 7fd520400640 1 -- 172.21.15.86:0/1870940271 shutdown_connections 2024-04-25T14:08:34.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.806+0000 7fd520400640 1 --2- 172.21.15.86:0/1870940271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd51815e270 0x7fd51815e680 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:34.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.806+0000 7fd520400640 1 -- 172.21.15.86:0/1870940271 >> 172.21.15.86:0/1870940271 conn(0x7fd5180b4a40 msgr2=0x7fd5180b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:34.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.806+0000 7fd520400640 1 -- 172.21.15.86:0/1870940271 shutdown_connections 2024-04-25T14:08:34.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.806+0000 7fd520400640 1 -- 172.21.15.86:0/1870940271 wait complete. 2024-04-25T14:08:34.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.810+0000 7fd520400640 1 Processor -- start 2024-04-25T14:08:34.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.810+0000 7fd520400640 1 -- start start 2024-04-25T14:08:34.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.810+0000 7fd520400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5180b0540 0x7fd518147810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:34.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.810+0000 7fd520400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd51815e270 0x7fd518144e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:34.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.810+0000 7fd520400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd51815ebc0 0x7fd518145340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:34.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.810+0000 7fd520400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd51816bf50 con 0x7fd51815ebc0 2024-04-25T14:08:34.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.810+0000 7fd520400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd51816bdd0 con 0x7fd5180b0540 2024-04-25T14:08:34.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.810+0000 7fd520400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd51816c0d0 con 0x7fd51815e270 2024-04-25T14:08:34.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.810+0000 7fd51e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd51815ebc0 0x7fd518145340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:34.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.810+0000 7fd51e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd51815ebc0 0x7fd518145340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43572/0 (socket says 172.21.15.86:43572) 2024-04-25T14:08:34.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.810+0000 7fd51e400640 1 -- 172.21.15.86:0/2330242879 learned_addr learned my addr 172.21.15.86:0/2330242879 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:34.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.814+0000 7fd51da00640 1 --2- 172.21.15.86:0/2330242879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5180b0540 0x7fd518147810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:34.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.814+0000 7fd51d000640 1 --2- 172.21.15.86:0/2330242879 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd51815e270 0x7fd518144e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:34.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.814+0000 7fd51e400640 1 -- 172.21.15.86:0/2330242879 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd51815e270 msgr2=0x7fd518144e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:34.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.814+0000 7fd51e400640 1 --2- 172.21.15.86:0/2330242879 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd51815e270 0x7fd518144e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:34.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.814+0000 7fd51e400640 1 -- 172.21.15.86:0/2330242879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5180b0540 msgr2=0x7fd518147810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:34.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.814+0000 7fd51e400640 1 --2- 172.21.15.86:0/2330242879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5180b0540 0x7fd518147810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:34.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.814+0000 7fd51e400640 1 -- 172.21.15.86:0/2330242879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd518149b70 con 0x7fd51815ebc0 2024-04-25T14:08:34.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.814+0000 7fd51d000640 1 --2- 172.21.15.86:0/2330242879 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd51815e270 0x7fd518144e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:34.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.814+0000 7fd51e400640 1 --2- 172.21.15.86:0/2330242879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd51815ebc0 0x7fd518145340 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fd51000c760 tx=0x7fd51000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:34.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.814+0000 7fd515600640 1 -- 172.21.15.86:0/2330242879 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd510003b80 con 0x7fd51815ebc0 2024-04-25T14:08:34.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.814+0000 7fd515600640 1 -- 172.21.15.86:0/2330242879 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd510003d20 con 0x7fd51815ebc0 2024-04-25T14:08:34.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.814+0000 7fd520400640 1 -- 172.21.15.86:0/2330242879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd518149e50 con 0x7fd51815ebc0 2024-04-25T14:08:34.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.814+0000 7fd515600640 1 -- 172.21.15.86:0/2330242879 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51000aa90 con 0x7fd51815ebc0 2024-04-25T14:08:34.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.814+0000 7fd520400640 1 -- 172.21.15.86:0/2330242879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd51814a410 con 0x7fd51815ebc0 2024-04-25T14:08:34.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.814+0000 7fd520400640 1 -- 172.21.15.86:0/2330242879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd518050750 con 0x7fd51815ebc0 2024-04-25T14:08:34.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.814+0000 7fd515600640 1 -- 172.21.15.86:0/2330242879 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7fd51000ac30 con 0x7fd51815ebc0 2024-04-25T14:08:34.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.814+0000 7fd515600640 1 --2- 172.21.15.86:0/2330242879 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fd4f806f520 0x7fd4f8071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:34.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.818+0000 7fd515600640 1 -- 172.21.15.86:0/2330242879 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7047+0+0 (secure 0 0 0) 0x7fd51008f7c0 con 0x7fd51815ebc0 2024-04-25T14:08:34.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.818+0000 7fd51da00640 1 --2- 172.21.15.86:0/2330242879 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fd4f806f520 0x7fd4f8071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:34.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.818+0000 7fd51da00640 1 --2- 172.21.15.86:0/2330242879 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fd4f806f520 0x7fd4f8071a30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd50c000f70 tx=0x7fd50c007d10 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:34.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.822+0000 7fd515600640 1 -- 172.21.15.86:0/2330242879 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd51005c0f0 con 0x7fd51815ebc0 2024-04-25T14:08:34.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:34.978+0000 7fd520400640 1 -- 172.21.15.86:0/2330242879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd518149890 con 0x7fd51815ebc0 2024-04-25T14:08:35.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.630+0000 7fd515600640 1 -- 172.21.15.86:0/2330242879 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v78) ==== 181+0+0 (secure 0 0 0) 0x7fd5100640e0 con 0x7fd51815ebc0 2024-04-25T14:08:35.633 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-04-25T14:08:35.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.634+0000 7fd520400640 1 -- 172.21.15.86:0/2330242879 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fd4f806f520 msgr2=0x7fd4f8071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:35.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.634+0000 7fd520400640 1 --2- 172.21.15.86:0/2330242879 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fd4f806f520 0x7fd4f8071a30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd50c000f70 tx=0x7fd50c007d10 comp rx=0 tx=0).stop 2024-04-25T14:08:35.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.634+0000 7fd520400640 1 -- 172.21.15.86:0/2330242879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd51815ebc0 msgr2=0x7fd518145340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:35.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.634+0000 7fd520400640 1 --2- 172.21.15.86:0/2330242879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd51815ebc0 0x7fd518145340 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fd51000c760 tx=0x7fd51000cc20 comp rx=0 tx=0).stop 2024-04-25T14:08:35.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.634+0000 7fd520400640 1 -- 172.21.15.86:0/2330242879 shutdown_connections 2024-04-25T14:08:35.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.634+0000 7fd520400640 1 --2- 172.21.15.86:0/2330242879 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fd4f806f520 0x7fd4f8071a30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:35.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.634+0000 7fd520400640 1 --2- 172.21.15.86:0/2330242879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd51815ebc0 0x7fd518145340 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:35.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.634+0000 7fd520400640 1 --2- 172.21.15.86:0/2330242879 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd51815e270 0x7fd518144e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:35.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.634+0000 7fd520400640 1 --2- 172.21.15.86:0/2330242879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5180b0540 0x7fd518147810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:35.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.634+0000 7fd520400640 1 -- 172.21.15.86:0/2330242879 >> 172.21.15.86:0/2330242879 conn(0x7fd5180b4a40 msgr2=0x7fd51815bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:35.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.634+0000 7fd520400640 1 -- 172.21.15.86:0/2330242879 shutdown_connections 2024-04-25T14:08:35.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.634+0000 7fd520400640 1 -- 172.21.15.86:0/2330242879 wait complete. 2024-04-25T14:08:35.661 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T14:08:35.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.814+0000 7f794c000640 1 Processor -- start 2024-04-25T14:08:35.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.814+0000 7f794c000640 1 -- start start 2024-04-25T14:08:35.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.814+0000 7f794c000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79440b0540 0x7f79440ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:35.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.814+0000 7f794c000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f794415e270 0x7f794415e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:35.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.814+0000 7f794c000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79440b0b20 con 0x7f794415ebc0 2024-04-25T14:08:35.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.814+0000 7f794c000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f794406d870 con 0x7f794415e270 2024-04-25T14:08:35.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.814+0000 7f794c000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f794406d220 con 0x7f79440b0540 2024-04-25T14:08:35.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.814+0000 7f794a000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f794415ebc0 0x7f79440733c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:58460/0 (socket says 172.21.15.86:58460) 2024-04-25T14:08:35.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.814+0000 7f794a000640 1 -- 172.21.15.86:0/611011713 learned_addr learned my addr 172.21.15.86:0/611011713 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:35.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.814+0000 7f7949600640 1 --2- 172.21.15.86:0/611011713 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79440b0540 0x7f79440ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:35.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.814+0000 7f7948c00640 1 --2- 172.21.15.86:0/611011713 >> v2:172.21.15.155:3300/0 conn(0x7f794415e270 0x7f794415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:35.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.814+0000 7f7943e00640 1 -- 172.21.15.86:0/611011713 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3805439543 0 0) 0x7f79440b0b20 con 0x7f794415ebc0 2024-04-25T14:08:35.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.814+0000 7f7943e00640 1 -- 172.21.15.86:0/611011713 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f792c003530 con 0x7f794415ebc0 2024-04-25T14:08:35.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.818+0000 7f7949600640 1 -- 172.21.15.86:0/611011713 >> v2:172.21.15.155:3300/0 conn(0x7f794415e270 msgr2=0x7f794415e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:35.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.818+0000 7f7949600640 1 --2- 172.21.15.86:0/611011713 >> v2:172.21.15.155:3300/0 conn(0x7f794415e270 0x7f794415e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:35.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.818+0000 7f7949600640 1 -- 172.21.15.86:0/611011713 >> v1:172.21.15.86:6789/0 conn(0x7f794415ebc0 legacy=0x7f79440733c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:35.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.818+0000 7f7949600640 1 -- 172.21.15.86:0/611011713 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f794406d500 con 0x7f79440b0540 2024-04-25T14:08:35.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.818+0000 7f7948c00640 1 --2- 172.21.15.86:0/611011713 >> v2:172.21.15.155:3300/0 conn(0x7f794415e270 0x7f794415e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:08:35.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.818+0000 7f7949600640 1 --2- 172.21.15.86:0/611011713 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79440b0540 0x7f79440ae2a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f79380097c0 tx=0x7f793801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acc9c2a7b0f0b958 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:35.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.818+0000 7f7943e00640 1 -- 172.21.15.86:0/611011713 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79380b0070 con 0x7f79440b0540 2024-04-25T14:08:35.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.818+0000 7f7943e00640 1 -- 172.21.15.86:0/611011713 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79380ab600 con 0x7f79440b0540 2024-04-25T14:08:35.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.818+0000 7f7943e00640 1 -- 172.21.15.86:0/611011713 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79380ab9b0 con 0x7f79440b0540 2024-04-25T14:08:35.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.822+0000 7f794c000640 1 -- 172.21.15.86:0/611011713 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79440b0540 msgr2=0x7f79440ae2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:35.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.822+0000 7f794c000640 1 --2- 172.21.15.86:0/611011713 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79440b0540 0x7f79440ae2a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f79380097c0 tx=0x7f793801c410 comp rx=0 tx=0).stop 2024-04-25T14:08:35.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.822+0000 7f794c000640 1 -- 172.21.15.86:0/611011713 shutdown_connections 2024-04-25T14:08:35.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.822+0000 7f794c000640 1 --2- 172.21.15.86:0/611011713 >> v2:172.21.15.155:3300/0 conn(0x7f794415e270 0x7f794415e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:35.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.822+0000 7f794c000640 1 --2- 172.21.15.86:0/611011713 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79440b0540 0x7f79440ae2a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:35.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.822+0000 7f794c000640 1 -- 172.21.15.86:0/611011713 >> 172.21.15.86:0/611011713 conn(0x7f79440b4a40 msgr2=0x7f79440b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:35.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.822+0000 7f794c000640 1 -- 172.21.15.86:0/611011713 shutdown_connections 2024-04-25T14:08:35.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.822+0000 7f794c000640 1 -- 172.21.15.86:0/611011713 wait complete. 2024-04-25T14:08:35.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.822+0000 7f794c000640 1 Processor -- start 2024-04-25T14:08:35.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.822+0000 7f794c000640 1 -- start start 2024-04-25T14:08:35.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.822+0000 7f794c000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79440b0540 0x7f794414cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:35.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.822+0000 7f794c000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f794415e270 0x7f79440b1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.822+0000 7f794c000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f794415ebc0 0x7f79440b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.826+0000 7f794c000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f794416b220 con 0x7f79440b0540 2024-04-25T14:08:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.826+0000 7f794c000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f794416b0a0 con 0x7f794415ebc0 2024-04-25T14:08:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.826+0000 7f794c000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f794416b3a0 con 0x7f794415e270 2024-04-25T14:08:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.826+0000 7f7949600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79440b0540 0x7f794414cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.826+0000 7f7949600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79440b0540 0x7f794414cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43578/0 (socket says 172.21.15.86:43578) 2024-04-25T14:08:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.826+0000 7f7949600640 1 -- 172.21.15.86:0/1573401662 learned_addr learned my addr 172.21.15.86:0/1573401662 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.826+0000 7f794a000640 1 --2- 172.21.15.86:0/1573401662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f794415ebc0 0x7f79440b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.826+0000 7f7948c00640 1 --2- 172.21.15.86:0/1573401662 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f794415e270 0x7f79440b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.826+0000 7f7949600640 1 -- 172.21.15.86:0/1573401662 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f794415e270 msgr2=0x7f79440b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.826+0000 7f7949600640 1 --2- 172.21.15.86:0/1573401662 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f794415e270 0x7f79440b1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:35.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.826+0000 7f7949600640 1 -- 172.21.15.86:0/1573401662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f794415ebc0 msgr2=0x7f79440b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:35.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.826+0000 7f7949600640 1 --2- 172.21.15.86:0/1573401662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f794415ebc0 0x7f79440b1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:35.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.826+0000 7f7949600640 1 -- 172.21.15.86:0/1573401662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79441fa3e0 con 0x7f79440b0540 2024-04-25T14:08:35.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.826+0000 7f7949600640 1 --2- 172.21.15.86:0/1573401662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79440b0540 0x7f794414cba0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f793801a040 tx=0x7f79380191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:35.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.826+0000 7f7941600640 1 -- 172.21.15.86:0/1573401662 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79380b8020 con 0x7f79440b0540 2024-04-25T14:08:35.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.826+0000 7f7941600640 1 -- 172.21.15.86:0/1573401662 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79380b4410 con 0x7f79440b0540 2024-04-25T14:08:35.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.826+0000 7f7941600640 1 -- 172.21.15.86:0/1573401662 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79380b0040 con 0x7f79440b0540 2024-04-25T14:08:35.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.826+0000 7f794c000640 1 -- 172.21.15.86:0/1573401662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f794414e970 con 0x7f79440b0540 2024-04-25T14:08:35.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.826+0000 7f794c000640 1 -- 172.21.15.86:0/1573401662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f794414ef30 con 0x7f79440b0540 2024-04-25T14:08:35.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.826+0000 7f794c000640 1 -- 172.21.15.86:0/1573401662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7944050750 con 0x7f79440b0540 2024-04-25T14:08:35.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.826+0000 7f7941600640 1 -- 172.21.15.86:0/1573401662 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7f79380bb020 con 0x7f79440b0540 2024-04-25T14:08:35.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.830+0000 7f7941600640 1 --2- 172.21.15.86:0/1573401662 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f792406f520 0x7f7924071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:35.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.830+0000 7f7941600640 1 -- 172.21.15.86:0/1573401662 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6609+0+0 (secure 0 0 0) 0x7f7938129360 con 0x7f79440b0540 2024-04-25T14:08:35.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.830+0000 7f7948c00640 1 --2- 172.21.15.86:0/1573401662 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f792406f520 0x7f7924071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:35.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.830+0000 7f7948c00640 1 --2- 172.21.15.86:0/1573401662 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f792406f520 0x7f7924071a30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7944153210 tx=0x7f7930009190 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:35.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.834+0000 7f7941600640 1 -- 172.21.15.86:0/1573401662 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f79380c2030 con 0x7f79440b0540 2024-04-25T14:08:35.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:35.990+0000 7f794c000640 1 -- 172.21.15.86:0/1573401662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7944157030 con 0x7f79440b0540 2024-04-25T14:08:36.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.634+0000 7f7941600640 1 -- 172.21.15.86:0/1573401662 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v79) ==== 169+0+0 (secure 0 0 0) 0x7f7938016090 con 0x7f79440b0540 2024-04-25T14:08:36.635 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-04-25T14:08:36.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.638+0000 7f794c000640 1 -- 172.21.15.86:0/1573401662 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f792406f520 msgr2=0x7f7924071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:36.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.638+0000 7f794c000640 1 --2- 172.21.15.86:0/1573401662 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f792406f520 0x7f7924071a30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7944153210 tx=0x7f7930009190 comp rx=0 tx=0).stop 2024-04-25T14:08:36.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.638+0000 7f794c000640 1 -- 172.21.15.86:0/1573401662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79440b0540 msgr2=0x7f794414cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:36.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.638+0000 7f794c000640 1 --2- 172.21.15.86:0/1573401662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79440b0540 0x7f794414cba0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f793801a040 tx=0x7f79380191b0 comp rx=0 tx=0).stop 2024-04-25T14:08:36.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.638+0000 7f794c000640 1 -- 172.21.15.86:0/1573401662 shutdown_connections 2024-04-25T14:08:36.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.638+0000 7f794c000640 1 --2- 172.21.15.86:0/1573401662 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f792406f520 0x7f7924071a30 secure :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7944153210 tx=0x7f7930009190 comp rx=0 tx=0).stop 2024-04-25T14:08:36.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.638+0000 7f794c000640 1 --2- 172.21.15.86:0/1573401662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f794415ebc0 0x7f79440b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:36.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.638+0000 7f794c000640 1 --2- 172.21.15.86:0/1573401662 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f794415e270 0x7f79440b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:36.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.638+0000 7f794c000640 1 --2- 172.21.15.86:0/1573401662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79440b0540 0x7f794414cba0 secure :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f793801a040 tx=0x7f79380191b0 comp rx=0 tx=0).stop 2024-04-25T14:08:36.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.638+0000 7f794c000640 1 -- 172.21.15.86:0/1573401662 >> 172.21.15.86:0/1573401662 conn(0x7f79440b4a40 msgr2=0x7f794415cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:36.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.638+0000 7f794c000640 1 -- 172.21.15.86:0/1573401662 shutdown_connections 2024-04-25T14:08:36.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.638+0000 7f794c000640 1 -- 172.21.15.86:0/1573401662 wait complete. 2024-04-25T14:08:36.666 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T14:08:36.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.814+0000 7f2407200640 1 Processor -- start 2024-04-25T14:08:36.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.814+0000 7f2407200640 1 -- start start 2024-04-25T14:08:36.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.814+0000 7f2407200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2400070330 0x7f24000d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.814+0000 7f2407200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f24000d1c20 0x7f240006ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.814+0000 7f2407200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f240006f270 0x7f240006f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.814+0000 7f2407200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f24000d2030 con 0x7f240006f270 2024-04-25T14:08:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.814+0000 7f2407200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f240015e5c0 con 0x7f24000d1c20 2024-04-25T14:08:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.814+0000 7f2407200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f240015e8a0 con 0x7f2400070330 2024-04-25T14:08:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.814+0000 7f2405200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f240006f270 0x7f240006f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.814+0000 7f2405200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f240006f270 0x7f240006f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43582/0 (socket says 172.21.15.86:43582) 2024-04-25T14:08:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.814+0000 7f2405200640 1 -- 172.21.15.86:0/2557001243 learned_addr learned my addr 172.21.15.86:0/2557001243 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.814+0000 7f23f7400640 1 --2- 172.21.15.86:0/2557001243 >> v2:172.21.15.155:3300/0 conn(0x7f24000d1c20 0x7f240006ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.814+0000 7f23ffe00640 1 --2- 172.21.15.86:0/2557001243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2400070330 0x7f24000d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.814+0000 7f2405200640 1 -- 172.21.15.86:0/2557001243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2400070330 msgr2=0x7f24000d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.814+0000 7f2405200640 1 --2- 172.21.15.86:0/2557001243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2400070330 0x7f24000d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.814+0000 7f2405200640 1 -- 172.21.15.86:0/2557001243 >> v2:172.21.15.155:3300/0 conn(0x7f24000d1c20 msgr2=0x7f240006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.814+0000 7f2405200640 1 --2- 172.21.15.86:0/2557001243 >> v2:172.21.15.155:3300/0 conn(0x7f24000d1c20 0x7f240006ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.814+0000 7f2405200640 1 -- 172.21.15.86:0/2557001243 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240015eb80 con 0x7f240006f270 2024-04-25T14:08:36.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.818+0000 7f2405200640 1 --2- 172.21.15.86:0/2557001243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f240006f270 0x7f240006f680 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f23f8009e50 tx=0x7f23f801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a411c4e4865bac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:36.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.818+0000 7f23ff400640 1 -- 172.21.15.86:0/2557001243 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f80a5790 con 0x7f240006f270 2024-04-25T14:08:36.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.818+0000 7f23ff400640 1 -- 172.21.15.86:0/2557001243 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23f80a5930 con 0x7f240006f270 2024-04-25T14:08:36.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.818+0000 7f23ff400640 1 -- 172.21.15.86:0/2557001243 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f80a5d00 con 0x7f240006f270 2024-04-25T14:08:36.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.818+0000 7f2407200640 1 -- 172.21.15.86:0/2557001243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f240006f270 msgr2=0x7f240006f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:36.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.818+0000 7f2407200640 1 --2- 172.21.15.86:0/2557001243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f240006f270 0x7f240006f680 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f23f8009e50 tx=0x7f23f801c930 comp rx=0 tx=0).stop 2024-04-25T14:08:36.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.818+0000 7f2407200640 1 -- 172.21.15.86:0/2557001243 shutdown_connections 2024-04-25T14:08:36.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.818+0000 7f2407200640 1 --2- 172.21.15.86:0/2557001243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f240006f270 0x7f240006f680 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:36.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.818+0000 7f2407200640 1 --2- 172.21.15.86:0/2557001243 >> v2:172.21.15.155:3300/0 conn(0x7f24000d1c20 0x7f240006ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:36.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.818+0000 7f2407200640 1 --2- 172.21.15.86:0/2557001243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2400070330 0x7f24000d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:36.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.818+0000 7f2407200640 1 -- 172.21.15.86:0/2557001243 >> 172.21.15.86:0/2557001243 conn(0x7f24000b4650 msgr2=0x7f24000b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:36.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.818+0000 7f2407200640 1 -- 172.21.15.86:0/2557001243 shutdown_connections 2024-04-25T14:08:36.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.818+0000 7f2407200640 1 -- 172.21.15.86:0/2557001243 wait complete. 2024-04-25T14:08:36.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.822+0000 7f2407200640 1 Processor -- start 2024-04-25T14:08:36.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.822+0000 7f2407200640 1 -- start start 2024-04-25T14:08:36.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.822+0000 7f2407200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f240006f270 0x7f240015a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:36.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.822+0000 7f2407200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2400070330 0x7f24000bbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:36.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.822+0000 7f2407200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24000d1c20 0x7f24000bc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:36.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.822+0000 7f2407200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f240015f4b0 con 0x7f240006f270 2024-04-25T14:08:36.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.822+0000 7f2407200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f240015f330 con 0x7f24000d1c20 2024-04-25T14:08:36.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.822+0000 7f2407200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2400070740 con 0x7f2400070330 2024-04-25T14:08:36.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.822+0000 7f23ffe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f240006f270 0x7f240015a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:36.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.822+0000 7f23ffe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f240006f270 0x7f240015a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43594/0 (socket says 172.21.15.86:43594) 2024-04-25T14:08:36.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.822+0000 7f23ffe00640 1 -- 172.21.15.86:0/3728108114 learned_addr learned my addr 172.21.15.86:0/3728108114 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:36.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.822+0000 7f23f7400640 1 --2- 172.21.15.86:0/3728108114 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2400070330 0x7f24000bbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:36.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.826+0000 7f2405200640 1 --2- 172.21.15.86:0/3728108114 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24000d1c20 0x7f24000bc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:36.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.826+0000 7f23ffe00640 1 -- 172.21.15.86:0/3728108114 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2400070330 msgr2=0x7f24000bbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:36.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.826+0000 7f23ffe00640 1 --2- 172.21.15.86:0/3728108114 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2400070330 0x7f24000bbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:36.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.826+0000 7f23ffe00640 1 -- 172.21.15.86:0/3728108114 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24000d1c20 msgr2=0x7f24000bc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:36.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.826+0000 7f23ffe00640 1 --2- 172.21.15.86:0/3728108114 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24000d1c20 0x7f24000bc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:36.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.826+0000 7f23ffe00640 1 -- 172.21.15.86:0/3728108114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24000bca50 con 0x7f240006f270 2024-04-25T14:08:36.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.826+0000 7f23ffe00640 1 --2- 172.21.15.86:0/3728108114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f240006f270 0x7f240015a6b0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f23f000d5f0 tx=0x7f23f000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:36.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.826+0000 7f23fcc00640 1 -- 172.21.15.86:0/3728108114 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f0004240 con 0x7f240006f270 2024-04-25T14:08:36.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.826+0000 7f23fcc00640 1 -- 172.21.15.86:0/3728108114 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23f00043e0 con 0x7f240006f270 2024-04-25T14:08:36.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.826+0000 7f2407200640 1 -- 172.21.15.86:0/3728108114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24000bcd30 con 0x7f240006f270 2024-04-25T14:08:36.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.826+0000 7f23fcc00640 1 -- 172.21.15.86:0/3728108114 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f0003620 con 0x7f240006f270 2024-04-25T14:08:36.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.826+0000 7f2407200640 1 -- 172.21.15.86:0/3728108114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24000bd2f0 con 0x7f240006f270 2024-04-25T14:08:36.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.826+0000 7f2407200640 1 -- 172.21.15.86:0/3728108114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2400069d00 con 0x7f240006f270 2024-04-25T14:08:36.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.826+0000 7f23fcc00640 1 -- 172.21.15.86:0/3728108114 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7f23f001b020 con 0x7f240006f270 2024-04-25T14:08:36.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.830+0000 7f23fcc00640 1 --2- 172.21.15.86:0/3728108114 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f23dc06f520 0x7f23dc071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:36.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.830+0000 7f23fcc00640 1 -- 172.21.15.86:0/3728108114 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6195+0+0 (secure 0 0 0) 0x7f23f008f200 con 0x7f240006f270 2024-04-25T14:08:36.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.830+0000 7f23f7400640 1 --2- 172.21.15.86:0/3728108114 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f23dc06f520 0x7f23dc071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:36.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.830+0000 7f23f7400640 1 --2- 172.21.15.86:0/3728108114 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f23dc06f520 0x7f23dc071a30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f240015b2f0 tx=0x7f23e8008040 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:36.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.834+0000 7f23fcc00640 1 -- 172.21.15.86:0/3728108114 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f23f005b790 con 0x7f240006f270 2024-04-25T14:08:36.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.994+0000 7f2407200640 1 -- 172.21.15.86:0/3728108114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f2400076bf0 con 0x7f240006f270 2024-04-25T14:08:36.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.998+0000 7f23fcc00640 1 -- 172.21.15.86:0/3728108114 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7f23f005fa90 con 0x7f240006f270 2024-04-25T14:08:37.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.998+0000 7f2407200640 1 -- 172.21.15.86:0/3728108114 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f23dc06f520 msgr2=0x7f23dc071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:37.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.998+0000 7f2407200640 1 --2- 172.21.15.86:0/3728108114 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f23dc06f520 0x7f23dc071a30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f240015b2f0 tx=0x7f23e8008040 comp rx=0 tx=0).stop 2024-04-25T14:08:37.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.998+0000 7f2407200640 1 -- 172.21.15.86:0/3728108114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f240006f270 msgr2=0x7f240015a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:37.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.998+0000 7f2407200640 1 --2- 172.21.15.86:0/3728108114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f240006f270 0x7f240015a6b0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f23f000d5f0 tx=0x7f23f000dab0 comp rx=0 tx=0).stop 2024-04-25T14:08:37.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.998+0000 7f2407200640 1 -- 172.21.15.86:0/3728108114 shutdown_connections 2024-04-25T14:08:37.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.998+0000 7f2407200640 1 --2- 172.21.15.86:0/3728108114 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f23dc06f520 0x7f23dc071a30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:37.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.998+0000 7f2407200640 1 --2- 172.21.15.86:0/3728108114 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24000d1c20 0x7f24000bc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:37.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.998+0000 7f2407200640 1 --2- 172.21.15.86:0/3728108114 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2400070330 0x7f24000bbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:37.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.998+0000 7f2407200640 1 --2- 172.21.15.86:0/3728108114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f240006f270 0x7f240015a6b0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:37.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.998+0000 7f2407200640 1 -- 172.21.15.86:0/3728108114 >> 172.21.15.86:0/3728108114 conn(0x7f24000b4650 msgr2=0x7f24000b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:37.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.998+0000 7f2407200640 1 -- 172.21.15.86:0/3728108114 shutdown_connections 2024-04-25T14:08:37.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:36.998+0000 7f2407200640 1 -- 172.21.15.86:0/3728108114 wait complete. 2024-04-25T14:08:37.021 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_delayed_metrics' 2024-04-25T14:08:37.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.166+0000 7fe896000640 1 Processor -- start 2024-04-25T14:08:37.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.166+0000 7fe896000640 1 -- start start 2024-04-25T14:08:37.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.166+0000 7fe896000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8900d2c80 0x7fe89015dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:37.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.166+0000 7fe896000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe890070e20 0x7fe890071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:37.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.166+0000 7fe896000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe8900722f0 0x7fe890072700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:37.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.166+0000 7fe896000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe8900ce720 con 0x7fe890070e20 2024-04-25T14:08:37.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.166+0000 7fe896000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe890071770 con 0x7fe8900722f0 2024-04-25T14:08:37.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.166+0000 7fe896000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe8900cd8d0 con 0x7fe8900d2c80 2024-04-25T14:08:37.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.166+0000 7fe886a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe890070e20 0x7fe890071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:37.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.166+0000 7fe886a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe890070e20 0x7fe890071230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43606/0 (socket says 172.21.15.86:43606) 2024-04-25T14:08:37.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.166+0000 7fe886a00640 1 -- 172.21.15.86:0/4061835723 learned_addr learned my addr 172.21.15.86:0/4061835723 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:37.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.166+0000 7fe88fe00640 1 --2- 172.21.15.86:0/4061835723 >> v2:172.21.15.155:3300/0 conn(0x7fe8900722f0 0x7fe890072700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:37.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.166+0000 7fe88f400640 1 --2- 172.21.15.86:0/4061835723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8900d2c80 0x7fe89015dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:37.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.166+0000 7fe886a00640 1 -- 172.21.15.86:0/4061835723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8900d2c80 msgr2=0x7fe89015dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:37.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.166+0000 7fe886a00640 1 --2- 172.21.15.86:0/4061835723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8900d2c80 0x7fe89015dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:37.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.166+0000 7fe886a00640 1 -- 172.21.15.86:0/4061835723 >> v2:172.21.15.155:3300/0 conn(0x7fe8900722f0 msgr2=0x7fe890072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:37.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.166+0000 7fe886a00640 1 --2- 172.21.15.86:0/4061835723 >> v2:172.21.15.155:3300/0 conn(0x7fe8900722f0 0x7fe890072700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:37.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.166+0000 7fe886a00640 1 -- 172.21.15.86:0/4061835723 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8900ce970 con 0x7fe890070e20 2024-04-25T14:08:37.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.166+0000 7fe88f400640 1 --2- 172.21.15.86:0/4061835723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8900d2c80 0x7fe89015dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:37.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.170+0000 7fe886a00640 1 --2- 172.21.15.86:0/4061835723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe890070e20 0x7fe890071230 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fe8900736f0 tx=0x7fe88001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555213f829f30bcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:37.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.170+0000 7fe88ea00640 1 -- 172.21.15.86:0/4061835723 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8800b5070 con 0x7fe890070e20 2024-04-25T14:08:37.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.170+0000 7fe88ea00640 1 -- 172.21.15.86:0/4061835723 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8800a89d0 con 0x7fe890070e20 2024-04-25T14:08:37.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.170+0000 7fe88ea00640 1 -- 172.21.15.86:0/4061835723 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8800a8da0 con 0x7fe890070e20 2024-04-25T14:08:37.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.170+0000 7fe896000640 1 -- 172.21.15.86:0/4061835723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe890070e20 msgr2=0x7fe890071230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:37.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.170+0000 7fe896000640 1 --2- 172.21.15.86:0/4061835723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe890070e20 0x7fe890071230 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fe8900736f0 tx=0x7fe88001ed10 comp rx=0 tx=0).stop 2024-04-25T14:08:37.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.170+0000 7fe896000640 1 -- 172.21.15.86:0/4061835723 shutdown_connections 2024-04-25T14:08:37.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.170+0000 7fe896000640 1 --2- 172.21.15.86:0/4061835723 >> v2:172.21.15.155:3300/0 conn(0x7fe8900722f0 0x7fe890072700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:37.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.170+0000 7fe896000640 1 --2- 172.21.15.86:0/4061835723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe890070e20 0x7fe890071230 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:37.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.170+0000 7fe896000640 1 --2- 172.21.15.86:0/4061835723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8900d2c80 0x7fe89015dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:37.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.170+0000 7fe896000640 1 -- 172.21.15.86:0/4061835723 >> 172.21.15.86:0/4061835723 conn(0x7fe8900cffc0 msgr2=0x7fe8900d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:37.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.170+0000 7fe896000640 1 -- 172.21.15.86:0/4061835723 shutdown_connections 2024-04-25T14:08:37.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.170+0000 7fe896000640 1 -- 172.21.15.86:0/4061835723 wait complete. 2024-04-25T14:08:37.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.174+0000 7fe896000640 1 Processor -- start 2024-04-25T14:08:37.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.174+0000 7fe896000640 1 -- start start 2024-04-25T14:08:37.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.174+0000 7fe896000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe890070e20 0x7fe8901607c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:37.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.174+0000 7fe896000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8900722f0 0x7fe8900c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:37.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.174+0000 7fe896000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8900d2c80 0x7fe890072700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:37.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.174+0000 7fe896000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8900d21a0 con 0x7fe890070e20 2024-04-25T14:08:37.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.174+0000 7fe896000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe8900d1ec0 con 0x7fe8900722f0 2024-04-25T14:08:37.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.174+0000 7fe896000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe8900d2480 con 0x7fe8900d2c80 2024-04-25T14:08:37.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.174+0000 7fe88f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe890070e20 0x7fe8901607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:37.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.174+0000 7fe88f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe890070e20 0x7fe8901607c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43618/0 (socket says 172.21.15.86:43618) 2024-04-25T14:08:37.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.174+0000 7fe88f400640 1 -- 172.21.15.86:0/2342959595 learned_addr learned my addr 172.21.15.86:0/2342959595 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:37.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.178+0000 7fe886a00640 1 --2- 172.21.15.86:0/2342959595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8900722f0 0x7fe8900c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:37.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.178+0000 7fe88fe00640 1 --2- 172.21.15.86:0/2342959595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8900d2c80 0x7fe890072700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:37.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.178+0000 7fe88f400640 1 -- 172.21.15.86:0/2342959595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8900d2c80 msgr2=0x7fe890072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:37.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.178+0000 7fe88f400640 1 --2- 172.21.15.86:0/2342959595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8900d2c80 0x7fe890072700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:37.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.178+0000 7fe88f400640 1 -- 172.21.15.86:0/2342959595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8900722f0 msgr2=0x7fe8900c8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:37.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.178+0000 7fe88f400640 1 --2- 172.21.15.86:0/2342959595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8900722f0 0x7fe8900c8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:37.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.178+0000 7fe88f400640 1 -- 172.21.15.86:0/2342959595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89015f2c0 con 0x7fe890070e20 2024-04-25T14:08:37.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.178+0000 7fe88f400640 1 --2- 172.21.15.86:0/2342959595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe890070e20 0x7fe8901607c0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe87800c7f0 tx=0x7fe87800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:37.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.178+0000 7fe887e00640 1 -- 172.21.15.86:0/2342959595 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878015070 con 0x7fe890070e20 2024-04-25T14:08:37.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.178+0000 7fe887e00640 1 -- 172.21.15.86:0/2342959595 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe878007820 con 0x7fe890070e20 2024-04-25T14:08:37.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.178+0000 7fe887e00640 1 -- 172.21.15.86:0/2342959595 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878007c10 con 0x7fe890070e20 2024-04-25T14:08:37.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.178+0000 7fe896000640 1 -- 172.21.15.86:0/2342959595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe89015f5a0 con 0x7fe890070e20 2024-04-25T14:08:37.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.178+0000 7fe896000640 1 -- 172.21.15.86:0/2342959595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe89015fb60 con 0x7fe890070e20 2024-04-25T14:08:37.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.178+0000 7fe896000640 1 -- 172.21.15.86:0/2342959595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8900720e0 con 0x7fe890070e20 2024-04-25T14:08:37.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.178+0000 7fe887e00640 1 -- 172.21.15.86:0/2342959595 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7fe878018070 con 0x7fe890070e20 2024-04-25T14:08:37.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.182+0000 7fe887e00640 1 --2- 172.21.15.86:0/2342959595 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fe86c06f550 0x7fe86c071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:37.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.182+0000 7fe887e00640 1 -- 172.21.15.86:0/2342959595 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6195+0+0 (secure 0 0 0) 0x7fe87808f840 con 0x7fe890070e20 2024-04-25T14:08:37.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.182+0000 7fe886a00640 1 --2- 172.21.15.86:0/2342959595 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fe86c06f550 0x7fe86c071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:37.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.182+0000 7fe886a00640 1 --2- 172.21.15.86:0/2342959595 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fe86c06f550 0x7fe86c071a60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe8800b2000 tx=0x7fe8800aa070 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:37.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.186+0000 7fe887e00640 1 -- 172.21.15.86:0/2342959595 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe87805bc30 con 0x7fe890070e20 2024-04-25T14:08:37.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.398+0000 7fe896000640 1 -- 172.21.15.86:0/2342959595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_delayed_metrics"]} v 0) -- 0x7fe890076bf0 con 0x7fe890070e20 2024-04-25T14:08:37.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.646+0000 7fe887e00640 1 -- 172.21.15.86:0/2342959595 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_delayed_metrics"]}]=0 v277) ==== 144+0+0 (secure 0 0 0) 0x7fe87805ff30 con 0x7fe890070e20 2024-04-25T14:08:37.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.646+0000 7fe896000640 1 -- 172.21.15.86:0/2342959595 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fe86c06f550 msgr2=0x7fe86c071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:37.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.646+0000 7fe896000640 1 --2- 172.21.15.86:0/2342959595 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fe86c06f550 0x7fe86c071a60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe8800b2000 tx=0x7fe8800aa070 comp rx=0 tx=0).stop 2024-04-25T14:08:37.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.646+0000 7fe896000640 1 -- 172.21.15.86:0/2342959595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe890070e20 msgr2=0x7fe8901607c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:37.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.646+0000 7fe896000640 1 --2- 172.21.15.86:0/2342959595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe890070e20 0x7fe8901607c0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe87800c7f0 tx=0x7fe87800ccb0 comp rx=0 tx=0).stop 2024-04-25T14:08:37.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.646+0000 7fe896000640 1 -- 172.21.15.86:0/2342959595 shutdown_connections 2024-04-25T14:08:37.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.646+0000 7fe896000640 1 --2- 172.21.15.86:0/2342959595 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fe86c06f550 0x7fe86c071a60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:37.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.646+0000 7fe896000640 1 --2- 172.21.15.86:0/2342959595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8900d2c80 0x7fe890072700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:37.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.646+0000 7fe896000640 1 --2- 172.21.15.86:0/2342959595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8900722f0 0x7fe8900c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:37.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.646+0000 7fe896000640 1 --2- 172.21.15.86:0/2342959595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe890070e20 0x7fe8901607c0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:37.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.646+0000 7fe896000640 1 -- 172.21.15.86:0/2342959595 >> 172.21.15.86:0/2342959595 conn(0x7fe8900cffc0 msgr2=0x7fe890159a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:37.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.650+0000 7fe896000640 1 -- 172.21.15.86:0/2342959595 shutdown_connections 2024-04-25T14:08:37.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.650+0000 7fe896000640 1 -- 172.21.15.86:0/2342959595 wait complete. 2024-04-25T14:08:37.670 INFO:tasks.cephfs_test_runner:test_delayed_metrics (tasks.cephfs.test_mds_metrics.TestMDSMetrics) ... ok 2024-04-25T14:08:37.670 INFO:tasks.cephfs_test_runner:Starting test: test_for_invalid_client_id (tasks.cephfs.test_mds_metrics.TestMDSMetrics) 2024-04-25T14:08:37.671 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:08:37.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.814+0000 7fbad5a00640 1 Processor -- start 2024-04-25T14:08:37.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.814+0000 7fbad5a00640 1 -- start start 2024-04-25T14:08:37.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.818+0000 7fbad5a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbad015faa0 0x7fbad006d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:37.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.818+0000 7fbad5a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbad00b59e0 0x7fbad00b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:37.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.818+0000 7fbad5a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbad00aeea0 0x7fbad00af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:37.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.818+0000 7fbad5a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbad006d560 con 0x7fbad00b59e0 2024-04-25T14:08:37.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.818+0000 7fbad5a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fbad00b6330 con 0x7fbad00aeea0 2024-04-25T14:08:37.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.818+0000 7fbad5a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbad006c670 con 0x7fbad015faa0 2024-04-25T14:08:37.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.818+0000 7fbace000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbad00b59e0 0x7fbad00b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:37.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.818+0000 7fbace000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbad00b59e0 0x7fbad00b5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43634/0 (socket says 172.21.15.86:43634) 2024-04-25T14:08:37.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.818+0000 7fbace000640 1 -- 172.21.15.86:0/4114743798 learned_addr learned my addr 172.21.15.86:0/4114743798 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:37.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.818+0000 7fbacf400640 1 --2- 172.21.15.86:0/4114743798 >> v2:172.21.15.155:3300/0 conn(0x7fbad00aeea0 0x7fbad00af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:37.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.818+0000 7fbace000640 1 -- 172.21.15.86:0/4114743798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbad015faa0 msgr2=0x7fbad006d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:37.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.818+0000 7fbacea00640 1 --2- 172.21.15.86:0/4114743798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbad015faa0 0x7fbad006d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:37.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.818+0000 7fbace000640 1 --2- 172.21.15.86:0/4114743798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbad015faa0 0x7fbad006d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:37.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.818+0000 7fbace000640 1 -- 172.21.15.86:0/4114743798 >> v2:172.21.15.155:3300/0 conn(0x7fbad00aeea0 msgr2=0x7fbad00af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:37.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.818+0000 7fbace000640 1 --2- 172.21.15.86:0/4114743798 >> v2:172.21.15.155:3300/0 conn(0x7fbad00aeea0 0x7fbad00af2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:37.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.818+0000 7fbace000640 1 -- 172.21.15.86:0/4114743798 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad006c950 con 0x7fbad00b59e0 2024-04-25T14:08:37.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.818+0000 7fbacf400640 1 --2- 172.21.15.86:0/4114743798 >> v2:172.21.15.155:3300/0 conn(0x7fbad00aeea0 0x7fbad00af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:37.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.818+0000 7fbacea00640 1 --2- 172.21.15.86:0/4114743798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbad015faa0 0x7fbad006d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:37.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.822+0000 7fbace000640 1 --2- 172.21.15.86:0/4114743798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbad00b59e0 0x7fbad00b5df0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fbad006d4d0 tx=0x7fbac001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc0a502ee20fe82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:37.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.822+0000 7fbacd600640 1 -- 172.21.15.86:0/4114743798 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac00a6040 con 0x7fbad00b59e0 2024-04-25T14:08:37.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.822+0000 7fbacd600640 1 -- 172.21.15.86:0/4114743798 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbac00af070 con 0x7fbad00b59e0 2024-04-25T14:08:37.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.822+0000 7fbacd600640 1 -- 172.21.15.86:0/4114743798 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac00aa860 con 0x7fbad00b59e0 2024-04-25T14:08:37.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.826+0000 7fbad5a00640 1 -- 172.21.15.86:0/4114743798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbad00b59e0 msgr2=0x7fbad00b5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:37.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.826+0000 7fbad5a00640 1 --2- 172.21.15.86:0/4114743798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbad00b59e0 0x7fbad00b5df0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fbad006d4d0 tx=0x7fbac001c520 comp rx=0 tx=0).stop 2024-04-25T14:08:37.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.826+0000 7fbad5a00640 1 -- 172.21.15.86:0/4114743798 shutdown_connections 2024-04-25T14:08:37.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.826+0000 7fbad5a00640 1 --2- 172.21.15.86:0/4114743798 >> v2:172.21.15.155:3300/0 conn(0x7fbad00aeea0 0x7fbad00af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:37.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.826+0000 7fbad5a00640 1 --2- 172.21.15.86:0/4114743798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbad00b59e0 0x7fbad00b5df0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:37.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.826+0000 7fbad5a00640 1 --2- 172.21.15.86:0/4114743798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbad015faa0 0x7fbad006d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:37.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.826+0000 7fbad5a00640 1 -- 172.21.15.86:0/4114743798 >> 172.21.15.86:0/4114743798 conn(0x7fbad00d2230 msgr2=0x7fbad00d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:37.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.826+0000 7fbad5a00640 1 -- 172.21.15.86:0/4114743798 shutdown_connections 2024-04-25T14:08:37.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.826+0000 7fbad5a00640 1 -- 172.21.15.86:0/4114743798 wait complete. 2024-04-25T14:08:37.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.826+0000 7fbad5a00640 1 Processor -- start 2024-04-25T14:08:37.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.830+0000 7fbad5a00640 1 -- start start 2024-04-25T14:08:37.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.830+0000 7fbad5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbad00aeea0 0x7fbad0144b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:37.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.830+0000 7fbad5a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbad00b59e0 0x7fbad015edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:37.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.830+0000 7fbad5a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbad015faa0 0x7fbad01420a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:37.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.830+0000 7fbad5a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbad015feb0 con 0x7fbad00aeea0 2024-04-25T14:08:37.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.830+0000 7fbad5a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbad006df30 con 0x7fbad00b59e0 2024-04-25T14:08:37.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.830+0000 7fbad5a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbad0160030 con 0x7fbad015faa0 2024-04-25T14:08:37.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.830+0000 7fbacea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbad00aeea0 0x7fbad0144b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:37.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.830+0000 7fbacea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbad00aeea0 0x7fbad0144b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43640/0 (socket says 172.21.15.86:43640) 2024-04-25T14:08:37.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.830+0000 7fbacea00640 1 -- 172.21.15.86:0/2610016045 learned_addr learned my addr 172.21.15.86:0/2610016045 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:37.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.830+0000 7fbacf400640 1 --2- 172.21.15.86:0/2610016045 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbad015faa0 0x7fbad01420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:37.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.830+0000 7fbace000640 1 --2- 172.21.15.86:0/2610016045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbad00b59e0 0x7fbad015edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:37.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.830+0000 7fbacea00640 1 -- 172.21.15.86:0/2610016045 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbad015faa0 msgr2=0x7fbad01420a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:37.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.830+0000 7fbacea00640 1 --2- 172.21.15.86:0/2610016045 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbad015faa0 0x7fbad01420a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:37.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.830+0000 7fbacea00640 1 -- 172.21.15.86:0/2610016045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbad00b59e0 msgr2=0x7fbad015edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:37.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.830+0000 7fbacea00640 1 --2- 172.21.15.86:0/2610016045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbad00b59e0 0x7fbad015edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:37.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.830+0000 7fbacea00640 1 -- 172.21.15.86:0/2610016045 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad015f2f0 con 0x7fbad00aeea0 2024-04-25T14:08:37.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.830+0000 7fbace000640 1 --2- 172.21.15.86:0/2610016045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbad00b59e0 0x7fbad015edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:37.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.830+0000 7fbacea00640 1 --2- 172.21.15.86:0/2610016045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbad00aeea0 0x7fbad0144b80 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fbab800c7f0 tx=0x7fbab800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:37.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.830+0000 7fbac6a00640 1 -- 172.21.15.86:0/2610016045 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab8015070 con 0x7fbad00aeea0 2024-04-25T14:08:37.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.830+0000 7fbac6a00640 1 -- 172.21.15.86:0/2610016045 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbab8007820 con 0x7fbad00aeea0 2024-04-25T14:08:37.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.830+0000 7fbac6a00640 1 -- 172.21.15.86:0/2610016045 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab8007c10 con 0x7fbad00aeea0 2024-04-25T14:08:37.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.830+0000 7fbad5a00640 1 -- 172.21.15.86:0/2610016045 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad015cc00 con 0x7fbad00aeea0 2024-04-25T14:08:37.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.830+0000 7fbad5a00640 1 -- 172.21.15.86:0/2610016045 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad0144380 con 0x7fbad00aeea0 2024-04-25T14:08:37.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.834+0000 7fbad5a00640 1 -- 172.21.15.86:0/2610016045 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad0074cf0 con 0x7fbad00aeea0 2024-04-25T14:08:37.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.834+0000 7fbac6a00640 1 -- 172.21.15.86:0/2610016045 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7fbab8018070 con 0x7fbad00aeea0 2024-04-25T14:08:37.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.834+0000 7fbac6a00640 1 --2- 172.21.15.86:0/2610016045 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fbaac06f550 0x7fbaac071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:37.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.834+0000 7fbac6a00640 1 -- 172.21.15.86:0/2610016045 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6195+0+0 (secure 0 0 0) 0x7fbab808f8d0 con 0x7fbad00aeea0 2024-04-25T14:08:37.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.834+0000 7fbace000640 1 --2- 172.21.15.86:0/2610016045 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fbaac06f550 0x7fbaac071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:37.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.834+0000 7fbace000640 1 --2- 172.21.15.86:0/2610016045 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fbaac06f550 0x7fbaac071a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbac00ae000 tx=0x7fbac0008040 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:37.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:37.838+0000 7fbac6a00640 1 -- 172.21.15.86:0/2610016045 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbab805c310 con 0x7fbad00aeea0 2024-04-25T14:08:38.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.006+0000 7fbad5a00640 1 -- 172.21.15.86:0/2610016045 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbad00b55b0 con 0x7fbad00aeea0 2024-04-25T14:08:38.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.006+0000 7fbac6a00640 1 -- 172.21.15.86:0/2610016045 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+14545 (secure 0 0 0) 0x7fbab805bc10 con 0x7fbad00aeea0 2024-04-25T14:08:38.010 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:08:38.010 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":79,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:08:36.637441+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6836/1088108173":"2024-04-26T14:08:33.628233+0000","172.21.15.86:6835/2682115891":"2024-04-26T14:08:33.628215+0000","172.21.15.86:6837/1088108173":"2024-04-26T14:08:33.628233+0000","172.21.15.86:6834/2682115891":"2024-04-26T14:08:33.628215+0000","172.21.15.86:0/2066326917":"2024-04-26T14:08:29.333458+0000","172.21.15.86:6833/203029814":"2024-04-26T14:08:29.333458+0000","172.21.15.86:6832/203029814":"2024-04-26T14:08:29.333458+0000","172.21.15.86:0/3991804772":"2024-04-26T14:08:29.333458+0000","172.21.15.86:0/2307768874":"2024-04-26T14:08:29.333458+0000","172.21.15.86:0/2510298865":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/3380489666":"2024-04-26T14:08:29.333458+0000","172.21.15.86:0/82848493":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/2452803132":"2024-04-26T14:08:29.333458+0000","172.21.15.86:0/1793604306":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/3402941646":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/112097751":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/2567754949":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/2334647603":"2024-04-26T14:08:29.333458+0000","172.21.15.86:0/529842883":"2024-04-26T14:07:03.055680+0000","172.21.15.86:6833/1025829557":"2024-04-26T14:07:03.055680+0000","172.21.15.86:6832/1025829557":"2024-04-26T14:07:03.055680+0000","172.21.15.86:0/759420784":"2024-04-26T14:07:03.055680+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:08:38.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.014+0000 7fbad5a00640 1 -- 172.21.15.86:0/2610016045 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fbaac06f550 msgr2=0x7fbaac071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:38.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.014+0000 7fbad5a00640 1 --2- 172.21.15.86:0/2610016045 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fbaac06f550 0x7fbaac071a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbac00ae000 tx=0x7fbac0008040 comp rx=0 tx=0).stop 2024-04-25T14:08:38.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.014+0000 7fbad5a00640 1 -- 172.21.15.86:0/2610016045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbad00aeea0 msgr2=0x7fbad0144b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:38.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.014+0000 7fbad5a00640 1 --2- 172.21.15.86:0/2610016045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbad00aeea0 0x7fbad0144b80 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fbab800c7f0 tx=0x7fbab800ccb0 comp rx=0 tx=0).stop 2024-04-25T14:08:38.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.014+0000 7fbad5a00640 1 -- 172.21.15.86:0/2610016045 shutdown_connections 2024-04-25T14:08:38.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.014+0000 7fbad5a00640 1 --2- 172.21.15.86:0/2610016045 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fbaac06f550 0x7fbaac071a60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:38.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.014+0000 7fbad5a00640 1 --2- 172.21.15.86:0/2610016045 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbad015faa0 0x7fbad01420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:38.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.014+0000 7fbad5a00640 1 --2- 172.21.15.86:0/2610016045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbad00b59e0 0x7fbad015edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:38.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.014+0000 7fbad5a00640 1 --2- 172.21.15.86:0/2610016045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbad00aeea0 0x7fbad0144b80 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:38.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.014+0000 7fbad5a00640 1 -- 172.21.15.86:0/2610016045 >> 172.21.15.86:0/2610016045 conn(0x7fbad00d2230 msgr2=0x7fbad0157af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:38.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.022+0000 7fbad5a00640 1 -- 172.21.15.86:0/2610016045 shutdown_connections 2024-04-25T14:08:38.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.022+0000 7fbad5a00640 1 -- 172.21.15.86:0/2610016045 wait complete. 2024-04-25T14:08:38.059 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:08:38.059 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:08:38.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.206+0000 7f165f000640 1 Processor -- start 2024-04-25T14:08:38.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.206+0000 7f165f000640 1 -- start start 2024-04-25T14:08:38.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.210+0000 7f165f000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f165815c2a0 0x7f16580ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:38.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.210+0000 7f165f000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16580b0a60 con 0x7f165806fd50 2024-04-25T14:08:38.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.210+0000 7f165f000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f165806ef90 con 0x7f165815feb0 2024-04-25T14:08:38.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.210+0000 7f165f000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f165806f270 con 0x7f165815c2a0 2024-04-25T14:08:38.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.210+0000 7f164fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f165806fd50 0x7f1658070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:58462/0 (socket says 172.21.15.86:58462) 2024-04-25T14:08:38.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.210+0000 7f164fe00640 1 -- 172.21.15.86:0/2756353061 learned_addr learned my addr 172.21.15.86:0/2756353061 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:38.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.210+0000 7f1657e00640 1 --2- 172.21.15.86:0/2756353061 >> v2:172.21.15.155:3300/0 conn(0x7f165815c2a0 0x7f16580ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:38.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.210+0000 7f1657400640 1 -- 172.21.15.86:0/2756353061 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 507313832 0 0) 0x7f16580b0a60 con 0x7f165806fd50 2024-04-25T14:08:38.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.210+0000 7f1657400640 1 -- 172.21.15.86:0/2756353061 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f163c003450 con 0x7f165806fd50 2024-04-25T14:08:38.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.210+0000 7f1657e00640 1 -- 172.21.15.86:0/2756353061 >> v1:172.21.15.155:6789/0 conn(0x7f165815feb0 legacy=0x7f16581602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:38.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.210+0000 7f1657e00640 1 -- 172.21.15.86:0/2756353061 >> v1:172.21.15.86:6789/0 conn(0x7f165806fd50 legacy=0x7f1658070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:38.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.210+0000 7f1657e00640 1 -- 172.21.15.86:0/2756353061 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f165806f550 con 0x7f165815c2a0 2024-04-25T14:08:38.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.214+0000 7f1657e00640 1 --2- 172.21.15.86:0/2756353061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f165815c2a0 0x7f16580ae130 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f16480097c0 tx=0x7f164801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68268fefd6cedcce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:38.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.214+0000 7f1657400640 1 -- 172.21.15.86:0/2756353061 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16480b0070 con 0x7f165815c2a0 2024-04-25T14:08:38.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.214+0000 7f1657400640 1 -- 172.21.15.86:0/2756353061 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16480ab600 con 0x7f165815c2a0 2024-04-25T14:08:38.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.214+0000 7f1657400640 1 -- 172.21.15.86:0/2756353061 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16480ab9b0 con 0x7f165815c2a0 2024-04-25T14:08:38.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.214+0000 7f165f000640 1 -- 172.21.15.86:0/2756353061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f165815c2a0 msgr2=0x7f16580ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:38.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.214+0000 7f165f000640 1 --2- 172.21.15.86:0/2756353061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f165815c2a0 0x7f16580ae130 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f16480097c0 tx=0x7f164801c3e0 comp rx=0 tx=0).stop 2024-04-25T14:08:38.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.214+0000 7f165f000640 1 -- 172.21.15.86:0/2756353061 shutdown_connections 2024-04-25T14:08:38.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.214+0000 7f165f000640 1 --2- 172.21.15.86:0/2756353061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f165815c2a0 0x7f16580ae130 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:38.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.214+0000 7f165f000640 1 -- 172.21.15.86:0/2756353061 >> 172.21.15.86:0/2756353061 conn(0x7f16580b4650 msgr2=0x7f16580b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:38.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.214+0000 7f165f000640 1 -- 172.21.15.86:0/2756353061 shutdown_connections 2024-04-25T14:08:38.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.214+0000 7f165f000640 1 -- 172.21.15.86:0/2756353061 wait complete. 2024-04-25T14:08:38.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.214+0000 7f165f000640 1 Processor -- start 2024-04-25T14:08:38.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.218+0000 7f165f000640 1 -- start start 2024-04-25T14:08:38.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.218+0000 7f165f000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f165806fd50 0x7f1658152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:38.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.218+0000 7f165f000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f165815c2a0 0x7f165815d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:38.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.218+0000 7f165f000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f165815feb0 0x7f1658096800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:38.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.218+0000 7f165f000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f16580ae7f0 con 0x7f165806fd50 2024-04-25T14:08:38.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.218+0000 7f165f000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f16580ae670 con 0x7f165815c2a0 2024-04-25T14:08:38.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.218+0000 7f165f000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f165816a200 con 0x7f165815feb0 2024-04-25T14:08:38.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.218+0000 7f1657e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f165806fd50 0x7f1658152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:38.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.218+0000 7f1657e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f165806fd50 0x7f1658152de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43650/0 (socket says 172.21.15.86:43650) 2024-04-25T14:08:38.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.218+0000 7f1657e00640 1 -- 172.21.15.86:0/3688839591 learned_addr learned my addr 172.21.15.86:0/3688839591 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:38.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.218+0000 7f164fe00640 1 --2- 172.21.15.86:0/3688839591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f165815c2a0 0x7f165815d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:38.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.218+0000 7f165d000640 1 --2- 172.21.15.86:0/3688839591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f165815feb0 0x7f1658096800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:38.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.218+0000 7f1657e00640 1 -- 172.21.15.86:0/3688839591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f165815feb0 msgr2=0x7f1658096800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:38.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.218+0000 7f1657e00640 1 --2- 172.21.15.86:0/3688839591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f165815feb0 0x7f1658096800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:38.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.218+0000 7f1657e00640 1 -- 172.21.15.86:0/3688839591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f165815c2a0 msgr2=0x7f165815d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:38.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.218+0000 7f1657e00640 1 --2- 172.21.15.86:0/3688839591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f165815c2a0 0x7f165815d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:38.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.218+0000 7f1657e00640 1 -- 172.21.15.86:0/3688839591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1658154d20 con 0x7f165806fd50 2024-04-25T14:08:38.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.218+0000 7f165d000640 1 --2- 172.21.15.86:0/3688839591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f165815feb0 0x7f1658096800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:38.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.218+0000 7f1657e00640 1 --2- 172.21.15.86:0/3688839591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f165806fd50 0x7f1658152de0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f16480097c0 tx=0x7f164801ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:38.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.218+0000 7f1654c00640 1 -- 172.21.15.86:0/3688839591 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16480bd070 con 0x7f165806fd50 2024-04-25T14:08:38.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.218+0000 7f1654c00640 1 -- 172.21.15.86:0/3688839591 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1648004030 con 0x7f165806fd50 2024-04-25T14:08:38.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.218+0000 7f165f000640 1 -- 172.21.15.86:0/3688839591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16581589c0 con 0x7f165806fd50 2024-04-25T14:08:38.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.218+0000 7f1654c00640 1 -- 172.21.15.86:0/3688839591 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16480b0040 con 0x7f165806fd50 2024-04-25T14:08:38.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.218+0000 7f165f000640 1 -- 172.21.15.86:0/3688839591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f165815bd40 con 0x7f165806fd50 2024-04-25T14:08:38.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.218+0000 7f165f000640 1 -- 172.21.15.86:0/3688839591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1658069d00 con 0x7f165806fd50 2024-04-25T14:08:38.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.222+0000 7f1654c00640 1 -- 172.21.15.86:0/3688839591 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7f16480c0070 con 0x7f165806fd50 2024-04-25T14:08:38.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.222+0000 7f1654c00640 1 --2- 172.21.15.86:0/3688839591 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f163406f550 0x7f1634071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:38.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.222+0000 7f1654c00640 1 -- 172.21.15.86:0/3688839591 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6195+0+0 (secure 0 0 0) 0x7f16481291a0 con 0x7f165806fd50 2024-04-25T14:08:38.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.222+0000 7f164fe00640 1 --2- 172.21.15.86:0/3688839591 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f163406f550 0x7f1634071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:38.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.222+0000 7f164fe00640 1 --2- 172.21.15.86:0/3688839591 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f163406f550 0x7f1634071a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f16400011f0 tx=0x7f1640009040 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:38.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.226+0000 7f1654c00640 1 -- 172.21.15.86:0/3688839591 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1648016090 con 0x7f165806fd50 2024-04-25T14:08:38.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.386+0000 7f165f000640 1 -- 172.21.15.86:0/3688839591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1658076bf0 con 0x7f165806fd50 2024-04-25T14:08:38.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.386+0000 7f1654c00640 1 -- 172.21.15.86:0/3688839591 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7f16480f5be0 con 0x7f165806fd50 2024-04-25T14:08:38.388 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:08:38.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.390+0000 7f165f000640 1 -- 172.21.15.86:0/3688839591 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f163406f550 msgr2=0x7f1634071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:38.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.390+0000 7f165f000640 1 --2- 172.21.15.86:0/3688839591 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f163406f550 0x7f1634071a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f16400011f0 tx=0x7f1640009040 comp rx=0 tx=0).stop 2024-04-25T14:08:38.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.390+0000 7f165f000640 1 -- 172.21.15.86:0/3688839591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f165806fd50 msgr2=0x7f1658152de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:38.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.390+0000 7f165f000640 1 --2- 172.21.15.86:0/3688839591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f165806fd50 0x7f1658152de0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f16480097c0 tx=0x7f164801ac20 comp rx=0 tx=0).stop 2024-04-25T14:08:38.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.390+0000 7f165f000640 1 -- 172.21.15.86:0/3688839591 shutdown_connections 2024-04-25T14:08:38.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.390+0000 7f165f000640 1 --2- 172.21.15.86:0/3688839591 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f163406f550 0x7f1634071a60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:38.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.390+0000 7f165f000640 1 --2- 172.21.15.86:0/3688839591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f165815feb0 0x7f1658096800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:38.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.390+0000 7f165f000640 1 --2- 172.21.15.86:0/3688839591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f165815c2a0 0x7f165815d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:38.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.390+0000 7f165f000640 1 --2- 172.21.15.86:0/3688839591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f165806fd50 0x7f1658152de0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:38.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.390+0000 7f165f000640 1 -- 172.21.15.86:0/3688839591 >> 172.21.15.86:0/3688839591 conn(0x7f16580b4650 msgr2=0x7f16580af800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:38.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.390+0000 7f165f000640 1 -- 172.21.15.86:0/3688839591 shutdown_connections 2024-04-25T14:08:38.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.390+0000 7f165f000640 1 -- 172.21.15.86:0/3688839591 wait complete. 2024-04-25T14:08:38.411 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_for_invalid_client_id' 2024-04-25T14:08:38.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.554+0000 7f7a23a00640 1 Processor -- start 2024-04-25T14:08:38.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.554+0000 7f7a23a00640 1 -- start start 2024-04-25T14:08:38.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.558+0000 7f7a23a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7a1c0740c0 0x7f7a1c1676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:38.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.558+0000 7f7a23a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a1c072bd0 con 0x7f7a1c0d16e0 2024-04-25T14:08:38.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.558+0000 7f7a23a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a1c06fbd0 con 0x7f7a1c0cfca0 2024-04-25T14:08:38.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.558+0000 7f7a23a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7a1c070470 con 0x7f7a1c0740c0 2024-04-25T14:08:38.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.558+0000 7f7a13e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7a1c0d16e0 0x7f7a1c0d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:58470/0 (socket says 172.21.15.86:58470) 2024-04-25T14:08:38.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.558+0000 7f7a13e00640 1 -- 172.21.15.86:0/4147043494 learned_addr learned my addr 172.21.15.86:0/4147043494 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:38.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.558+0000 7f7a21a00640 1 --2- 172.21.15.86:0/4147043494 >> v2:172.21.15.86:3300/0 conn(0x7f7a1c0740c0 0x7f7a1c1676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:38.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.558+0000 7f7a1be00640 1 -- 172.21.15.86:0/4147043494 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2267991504 0 0) 0x7f7a1c072bd0 con 0x7f7a1c0d16e0 2024-04-25T14:08:38.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.558+0000 7f7a1be00640 1 -- 172.21.15.86:0/4147043494 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a00005740 con 0x7f7a1c0d16e0 2024-04-25T14:08:38.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.558+0000 7f7a21a00640 1 -- 172.21.15.86:0/4147043494 >> v1:172.21.15.155:6789/0 conn(0x7f7a1c0cfca0 legacy=0x7f7a1c06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:38.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.558+0000 7f7a1be00640 1 -- 172.21.15.86:0/4147043494 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3736258303 0 0) 0x7f7a1c06fbd0 con 0x7f7a1c0cfca0 2024-04-25T14:08:38.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.558+0000 7f7a21a00640 1 -- 172.21.15.86:0/4147043494 >> v1:172.21.15.86:6789/0 conn(0x7f7a1c0d16e0 legacy=0x7f7a1c0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:38.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.558+0000 7f7a21a00640 1 -- 172.21.15.86:0/4147043494 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1c06feb0 con 0x7f7a1c0740c0 2024-04-25T14:08:38.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.562+0000 7f7a21a00640 1 --2- 172.21.15.86:0/4147043494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a1c0740c0 0x7f7a1c1676d0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f7a140097c0 tx=0x7f7a1401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decb014b485b7758 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:38.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.562+0000 7f7a1be00640 1 -- 172.21.15.86:0/4147043494 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a140b0070 con 0x7f7a1c0740c0 2024-04-25T14:08:38.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.562+0000 7f7a1be00640 1 -- 172.21.15.86:0/4147043494 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a140ab600 con 0x7f7a1c0740c0 2024-04-25T14:08:38.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.562+0000 7f7a1be00640 1 -- 172.21.15.86:0/4147043494 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a140ab9d0 con 0x7f7a1c0740c0 2024-04-25T14:08:38.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.562+0000 7f7a23a00640 1 -- 172.21.15.86:0/4147043494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a1c0740c0 msgr2=0x7f7a1c1676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:38.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.562+0000 7f7a23a00640 1 --2- 172.21.15.86:0/4147043494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a1c0740c0 0x7f7a1c1676d0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f7a140097c0 tx=0x7f7a1401c4a0 comp rx=0 tx=0).stop 2024-04-25T14:08:38.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.562+0000 7f7a23a00640 1 -- 172.21.15.86:0/4147043494 shutdown_connections 2024-04-25T14:08:38.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.562+0000 7f7a23a00640 1 --2- 172.21.15.86:0/4147043494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a1c0740c0 0x7f7a1c1676d0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:38.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.562+0000 7f7a23a00640 1 -- 172.21.15.86:0/4147043494 >> 172.21.15.86:0/4147043494 conn(0x7f7a1c0b4650 msgr2=0x7f7a1c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:38.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.562+0000 7f7a23a00640 1 -- 172.21.15.86:0/4147043494 shutdown_connections 2024-04-25T14:08:38.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.562+0000 7f7a23a00640 1 -- 172.21.15.86:0/4147043494 wait complete. 2024-04-25T14:08:38.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.566+0000 7f7a23a00640 1 Processor -- start 2024-04-25T14:08:38.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.566+0000 7f7a23a00640 1 -- start start 2024-04-25T14:08:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.566+0000 7f7a23a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a1c0740c0 0x7f7a1c15da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.566+0000 7f7a23a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a1c0cfca0 0x7f7a1c15dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.566+0000 7f7a23a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a1c0d16e0 0x7f7a1c15bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.566+0000 7f7a23a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c0ce900 con 0x7f7a1c0d16e0 2024-04-25T14:08:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.566+0000 7f7a23a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c0ce780 con 0x7f7a1c0740c0 2024-04-25T14:08:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.566+0000 7f7a23a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7a1c0cea80 con 0x7f7a1c0cfca0 2024-04-25T14:08:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.566+0000 7f7a21a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a1c0d16e0 0x7f7a1c15bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.566+0000 7f7a21a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a1c0d16e0 0x7f7a1c15bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43670/0 (socket says 172.21.15.86:43670) 2024-04-25T14:08:38.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.566+0000 7f7a21a00640 1 -- 172.21.15.86:0/2901252974 learned_addr learned my addr 172.21.15.86:0/2901252974 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:38.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.566+0000 7f7a21000640 1 --2- 172.21.15.86:0/2901252974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a1c0740c0 0x7f7a1c15da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:38.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.566+0000 7f7a13e00640 1 --2- 172.21.15.86:0/2901252974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a1c0cfca0 0x7f7a1c15dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:38.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.566+0000 7f7a21a00640 1 -- 172.21.15.86:0/2901252974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a1c0cfca0 msgr2=0x7f7a1c15dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:38.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.566+0000 7f7a21a00640 1 --2- 172.21.15.86:0/2901252974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a1c0cfca0 0x7f7a1c15dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:38.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.566+0000 7f7a21a00640 1 -- 172.21.15.86:0/2901252974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a1c0740c0 msgr2=0x7f7a1c15da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:38.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.566+0000 7f7a21a00640 1 --2- 172.21.15.86:0/2901252974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a1c0740c0 0x7f7a1c15da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:38.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.566+0000 7f7a21a00640 1 -- 172.21.15.86:0/2901252974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1c158000 con 0x7f7a1c0d16e0 2024-04-25T14:08:38.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.566+0000 7f7a21000640 1 --2- 172.21.15.86:0/2901252974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a1c0740c0 0x7f7a1c15da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:38.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.566+0000 7f7a21a00640 1 --2- 172.21.15.86:0/2901252974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a1c0d16e0 0x7f7a1c15bb40 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f7a140097c0 tx=0x7f7a1401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:38.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.566+0000 7f7a19600640 1 -- 172.21.15.86:0/2901252974 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a140b0050 con 0x7f7a1c0d16e0 2024-04-25T14:08:38.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.566+0000 7f7a19600640 1 -- 172.21.15.86:0/2901252974 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a140b1040 con 0x7f7a1c0d16e0 2024-04-25T14:08:38.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.566+0000 7f7a23a00640 1 -- 172.21.15.86:0/2901252974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a1c1582e0 con 0x7f7a1c0d16e0 2024-04-25T14:08:38.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.566+0000 7f7a19600640 1 -- 172.21.15.86:0/2901252974 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a14004020 con 0x7f7a1c0d16e0 2024-04-25T14:08:38.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.566+0000 7f7a23a00640 1 -- 172.21.15.86:0/2901252974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a1c15e830 con 0x7f7a1c0d16e0 2024-04-25T14:08:38.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.570+0000 7f7a23a00640 1 -- 172.21.15.86:0/2901252974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a1c155440 con 0x7f7a1c0d16e0 2024-04-25T14:08:38.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.570+0000 7f7a19600640 1 -- 172.21.15.86:0/2901252974 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7f7a140be020 con 0x7f7a1c0d16e0 2024-04-25T14:08:38.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.570+0000 7f7a19600640 1 --2- 172.21.15.86:0/2901252974 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f79f806f520 0x7f79f8071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:38.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.570+0000 7f7a19600640 1 -- 172.21.15.86:0/2901252974 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6195+0+0 (secure 0 0 0) 0x7f7a14129650 con 0x7f7a1c0d16e0 2024-04-25T14:08:38.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.570+0000 7f7a21000640 1 --2- 172.21.15.86:0/2901252974 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f79f806f520 0x7f79f8071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:38.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.570+0000 7f7a21000640 1 --2- 172.21.15.86:0/2901252974 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f79f806f520 0x7f79f8071a30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c0012f0 tx=0x7f7a0c009040 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:38.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.574+0000 7f7a19600640 1 -- 172.21.15.86:0/2901252974 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7a14016090 con 0x7f7a1c0d16e0 2024-04-25T14:08:38.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:38.790+0000 7f7a23a00640 1 -- 172.21.15.86:0/2901252974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_for_invalid_client_id"]} v 0) -- 0x7f7a1c15ea90 con 0x7f7a1c0d16e0 2024-04-25T14:08:39.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.654+0000 7f7a19600640 1 -- 172.21.15.86:0/2901252974 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_for_invalid_client_id"]}]=0 v279) ==== 153+0+0 (secure 0 0 0) 0x7f7a140f6130 con 0x7f7a1c0d16e0 2024-04-25T14:08:39.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.658+0000 7f7a23a00640 1 -- 172.21.15.86:0/2901252974 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f79f806f520 msgr2=0x7f79f8071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:39.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.658+0000 7f7a23a00640 1 --2- 172.21.15.86:0/2901252974 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f79f806f520 0x7f79f8071a30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c0012f0 tx=0x7f7a0c009040 comp rx=0 tx=0).stop 2024-04-25T14:08:39.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.658+0000 7f7a23a00640 1 -- 172.21.15.86:0/2901252974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a1c0d16e0 msgr2=0x7f7a1c15bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:39.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.658+0000 7f7a23a00640 1 --2- 172.21.15.86:0/2901252974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a1c0d16e0 0x7f7a1c15bb40 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f7a140097c0 tx=0x7f7a1401ace0 comp rx=0 tx=0).stop 2024-04-25T14:08:39.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.658+0000 7f7a23a00640 1 -- 172.21.15.86:0/2901252974 shutdown_connections 2024-04-25T14:08:39.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.658+0000 7f7a23a00640 1 --2- 172.21.15.86:0/2901252974 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f79f806f520 0x7f79f8071a30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:39.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.658+0000 7f7a23a00640 1 --2- 172.21.15.86:0/2901252974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a1c0d16e0 0x7f7a1c15bb40 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:39.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.658+0000 7f7a23a00640 1 --2- 172.21.15.86:0/2901252974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a1c0cfca0 0x7f7a1c15dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:39.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.658+0000 7f7a23a00640 1 --2- 172.21.15.86:0/2901252974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a1c0740c0 0x7f7a1c15da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:39.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.658+0000 7f7a23a00640 1 -- 172.21.15.86:0/2901252974 >> 172.21.15.86:0/2901252974 conn(0x7f7a1c0b4650 msgr2=0x7f7a1c0b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:39.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.658+0000 7f7a23a00640 1 -- 172.21.15.86:0/2901252974 shutdown_connections 2024-04-25T14:08:39.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.658+0000 7f7a23a00640 1 -- 172.21.15.86:0/2901252974 wait complete. 2024-04-25T14:08:39.683 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T14:08:39.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.834+0000 7f0211000640 1 Processor -- start 2024-04-25T14:08:39.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.834+0000 7f0211000640 1 -- start start 2024-04-25T14:08:39.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.834+0000 7f0211000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f020c06e7a0 0x7f020c1602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:39.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.834+0000 7f0211000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f020c0d1c10 con 0x7f020c1607e0 2024-04-25T14:08:39.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.834+0000 7f0211000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f020c0d1ef0 con 0x7f020c071530 2024-04-25T14:08:39.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.834+0000 7f0211000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f020c0d17d0 con 0x7f020c06e7a0 2024-04-25T14:08:39.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.834+0000 7f0201400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f020c1607e0 0x7f020c070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60034/0 (socket says 172.21.15.86:60034) 2024-04-25T14:08:39.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.834+0000 7f0201400640 1 -- 172.21.15.86:0/22909623 learned_addr learned my addr 172.21.15.86:0/22909623 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:39.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.838+0000 7f0209e00640 1 --2- 172.21.15.86:0/22909623 >> v2:172.21.15.155:3300/0 conn(0x7f020c06e7a0 0x7f020c1602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:39.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.838+0000 7f0209400640 1 -- 172.21.15.86:0/22909623 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1672571450 0 0) 0x7f020c0d1c10 con 0x7f020c1607e0 2024-04-25T14:08:39.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.838+0000 7f0209400640 1 -- 172.21.15.86:0/22909623 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01f0003530 con 0x7f020c1607e0 2024-04-25T14:08:39.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.838+0000 7f0209400640 1 -- 172.21.15.86:0/22909623 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4184300913 0 0) 0x7f01f0003530 con 0x7f020c1607e0 2024-04-25T14:08:39.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.838+0000 7f0209400640 1 -- 172.21.15.86:0/22909623 >> v2:172.21.15.155:3300/0 conn(0x7f020c06e7a0 msgr2=0x7f020c1602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:39.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.838+0000 7f0209400640 1 --2- 172.21.15.86:0/22909623 >> v2:172.21.15.155:3300/0 conn(0x7f020c06e7a0 0x7f020c1602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:39.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.838+0000 7f0209400640 1 -- 172.21.15.86:0/22909623 >> v1:172.21.15.155:6789/0 conn(0x7f020c071530 legacy=0x7f020c163e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:39.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.838+0000 7f0209400640 1 -- 172.21.15.86:0/22909623 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020c0d21d0 con 0x7f020c1607e0 2024-04-25T14:08:39.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.838+0000 7f0209400640 1 -- 172.21.15.86:0/22909623 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f01f4001820 con 0x7f020c1607e0 2024-04-25T14:08:39.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.838+0000 7f0209400640 1 -- 172.21.15.86:0/22909623 >> v1:172.21.15.86:6789/0 conn(0x7f020c1607e0 legacy=0x7f020c070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:39.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.838+0000 7f0209400640 1 --2- 172.21.15.86:0/22909623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01f0003a30 0x7f01f0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:39.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.838+0000 7f0209400640 1 --2- 172.21.15.86:0/22909623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01f0004810 0x7f01f0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:39.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.838+0000 7f0209400640 1 --2- 172.21.15.86:0/22909623 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01f0005360 0x7f01f000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:39.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.838+0000 7f0209400640 1 -- 172.21.15.86:0/22909623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f020c0d17d0 con 0x7f01f0004810 2024-04-25T14:08:39.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.838+0000 7f0209400640 1 -- 172.21.15.86:0/22909623 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f01f000c260 con 0x7f01f0003a30 2024-04-25T14:08:39.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.838+0000 7f0209400640 1 -- 172.21.15.86:0/22909623 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f01f000c460 con 0x7f01f0005360 2024-04-25T14:08:39.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.838+0000 7f0201400640 1 --2- 172.21.15.86:0/22909623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01f0004810 0x7f01f0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:39.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.838+0000 7f0209e00640 1 --2- 172.21.15.86:0/22909623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01f0003a30 0x7f01f0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:39.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.838+0000 7f020a800640 1 --2- 172.21.15.86:0/22909623 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01f0005360 0x7f01f000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:39.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.838+0000 7f0201400640 1 -- 172.21.15.86:0/22909623 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01f0005360 msgr2=0x7f01f000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:39.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.838+0000 7f0201400640 1 --2- 172.21.15.86:0/22909623 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01f0005360 0x7f01f000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:39.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.838+0000 7f0201400640 1 -- 172.21.15.86:0/22909623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01f0003a30 msgr2=0x7f01f0003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:39.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.838+0000 7f0201400640 1 --2- 172.21.15.86:0/22909623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01f0003a30 0x7f01f0003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:39.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.838+0000 7f0201400640 1 -- 172.21.15.86:0/22909623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f0003530 con 0x7f01f0004810 2024-04-25T14:08:39.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.838+0000 7f020a800640 1 -- 172.21.15.86:0/22909623 reap_dead start 2024-04-25T14:08:39.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.842+0000 7f0201400640 1 --2- 172.21.15.86:0/22909623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01f0004810 0x7f01f0004c20 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f01f4007fd0 tx=0x7f01f401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad743f60200b30dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:39.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.842+0000 7f0209400640 1 -- 172.21.15.86:0/22909623 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f4007d50 con 0x7f01f0004810 2024-04-25T14:08:39.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.842+0000 7f0209400640 1 -- 172.21.15.86:0/22909623 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01f40a9070 con 0x7f01f0004810 2024-04-25T14:08:39.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.842+0000 7f0209400640 1 -- 172.21.15.86:0/22909623 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f40074b0 con 0x7f01f0004810 2024-04-25T14:08:39.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.842+0000 7f0211000640 1 -- 172.21.15.86:0/22909623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01f0004810 msgr2=0x7f01f0004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:39.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.842+0000 7f0211000640 1 --2- 172.21.15.86:0/22909623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01f0004810 0x7f01f0004c20 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f01f4007fd0 tx=0x7f01f401cc60 comp rx=0 tx=0).stop 2024-04-25T14:08:39.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.842+0000 7f0211000640 1 -- 172.21.15.86:0/22909623 shutdown_connections 2024-04-25T14:08:39.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.842+0000 7f0211000640 1 --2- 172.21.15.86:0/22909623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01f0004810 0x7f01f0004c20 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:39.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.842+0000 7f0211000640 1 -- 172.21.15.86:0/22909623 >> 172.21.15.86:0/22909623 conn(0x7f020c0cffc0 msgr2=0x7f020c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:39.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.842+0000 7f0211000640 1 -- 172.21.15.86:0/22909623 shutdown_connections 2024-04-25T14:08:39.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.842+0000 7f0211000640 1 -- 172.21.15.86:0/22909623 wait complete. 2024-04-25T14:08:39.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.842+0000 7f0211000640 1 Processor -- start 2024-04-25T14:08:39.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.842+0000 7f0211000640 1 -- start start 2024-04-25T14:08:39.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.846+0000 7f0211000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01f0004810 0x7f020c0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:39.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.846+0000 7f0211000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f020c0cd110 0x7f020c14fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:39.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.846+0000 7f0211000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f020c0b36d0 0x7f020c0b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:39.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.846+0000 7f0211000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f020c163f90 con 0x7f01f0004810 2024-04-25T14:08:39.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.846+0000 7f0211000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f020c163e10 con 0x7f020c0b36d0 2024-04-25T14:08:39.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.846+0000 7f0211000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f020c164110 con 0x7f020c0cd110 2024-04-25T14:08:39.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.846+0000 7f0209e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01f0004810 0x7f020c0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:39.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.846+0000 7f0209e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01f0004810 0x7f020c0b6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48802/0 (socket says 172.21.15.86:48802) 2024-04-25T14:08:39.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.846+0000 7f0209e00640 1 -- 172.21.15.86:0/3343784738 learned_addr learned my addr 172.21.15.86:0/3343784738 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:39.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.846+0000 7f020a800640 1 --2- 172.21.15.86:0/3343784738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f020c0b36d0 0x7f020c0b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:39.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.846+0000 7f0201400640 1 --2- 172.21.15.86:0/3343784738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f020c0cd110 0x7f020c14fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:39.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.846+0000 7f0209e00640 1 -- 172.21.15.86:0/3343784738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f020c0cd110 msgr2=0x7f020c14fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:39.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.846+0000 7f0209e00640 1 --2- 172.21.15.86:0/3343784738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f020c0cd110 0x7f020c14fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:39.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.846+0000 7f0209e00640 1 -- 172.21.15.86:0/3343784738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f020c0b36d0 msgr2=0x7f020c0b3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:39.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.846+0000 7f0209e00640 1 --2- 172.21.15.86:0/3343784738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f020c0b36d0 0x7f020c0b3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:39.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.846+0000 7f0209e00640 1 -- 172.21.15.86:0/3343784738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020c0b4020 con 0x7f01f0004810 2024-04-25T14:08:39.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.846+0000 7f0209e00640 1 --2- 172.21.15.86:0/3343784738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01f0004810 0x7f020c0b6160 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f01fc011e30 tx=0x7f01fc00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:39.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.846+0000 7f0202a00640 1 -- 172.21.15.86:0/3343784738 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc015630 con 0x7f01f0004810 2024-04-25T14:08:39.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.846+0000 7f0202a00640 1 -- 172.21.15.86:0/3343784738 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01fc0157d0 con 0x7f01f0004810 2024-04-25T14:08:39.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.846+0000 7f0202a00640 1 -- 172.21.15.86:0/3343784738 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc015ba0 con 0x7f01f0004810 2024-04-25T14:08:39.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.846+0000 7f0211000640 1 -- 172.21.15.86:0/3343784738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f020c0b69b0 con 0x7f01f0004810 2024-04-25T14:08:39.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.846+0000 7f0211000640 1 -- 172.21.15.86:0/3343784738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f020c0b6f70 con 0x7f01f0004810 2024-04-25T14:08:39.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.846+0000 7f0211000640 1 -- 172.21.15.86:0/3343784738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f020c0720e0 con 0x7f01f0004810 2024-04-25T14:08:39.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.850+0000 7f0202a00640 1 -- 172.21.15.86:0/3343784738 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7f01fc01d400 con 0x7f01f0004810 2024-04-25T14:08:39.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.850+0000 7f0202a00640 1 --2- 172.21.15.86:0/3343784738 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f01e806f520 0x7f01e8071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:39.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.850+0000 7f0202a00640 1 -- 172.21.15.86:0/3343784738 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6195+0+0 (secure 0 0 0) 0x7f01fc090840 con 0x7f01f0004810 2024-04-25T14:08:39.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.850+0000 7f0201400640 1 --2- 172.21.15.86:0/3343784738 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f01e806f520 0x7f01e8071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:39.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.850+0000 7f0201400640 1 --2- 172.21.15.86:0/3343784738 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f01e806f520 0x7f01e8071a30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f01f40ab000 tx=0x7f01f40a7070 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:39.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:39.854+0000 7f0202a00640 1 -- 172.21.15.86:0/3343784738 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f020c0b4020 con 0x7f01f0004810 2024-04-25T14:08:40.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.010+0000 7f0211000640 1 -- 172.21.15.86:0/3343784738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f020c076bf0 con 0x7f01f0004810 2024-04-25T14:08:40.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.014+0000 7f0202a00640 1 -- 172.21.15.86:0/3343784738 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f01fc019030 con 0x7f01f0004810 2024-04-25T14:08:40.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.018+0000 7f0211000640 1 -- 172.21.15.86:0/3343784738 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f01e806f520 msgr2=0x7f01e8071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:40.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.018+0000 7f0211000640 1 --2- 172.21.15.86:0/3343784738 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f01e806f520 0x7f01e8071a30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f01f40ab000 tx=0x7f01f40a7070 comp rx=0 tx=0).stop 2024-04-25T14:08:40.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.018+0000 7f0211000640 1 -- 172.21.15.86:0/3343784738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01f0004810 msgr2=0x7f020c0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:40.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.018+0000 7f0211000640 1 --2- 172.21.15.86:0/3343784738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01f0004810 0x7f020c0b6160 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f01fc011e30 tx=0x7f01fc00f580 comp rx=0 tx=0).stop 2024-04-25T14:08:40.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.018+0000 7f0211000640 1 -- 172.21.15.86:0/3343784738 shutdown_connections 2024-04-25T14:08:40.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.018+0000 7f0211000640 1 --2- 172.21.15.86:0/3343784738 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f01e806f520 0x7f01e8071a30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:40.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.018+0000 7f0211000640 1 --2- 172.21.15.86:0/3343784738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f020c0b36d0 0x7f020c0b3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:40.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.018+0000 7f0211000640 1 --2- 172.21.15.86:0/3343784738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f020c0cd110 0x7f020c14fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:40.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.018+0000 7f0211000640 1 --2- 172.21.15.86:0/3343784738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01f0004810 0x7f020c0b6160 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:40.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.018+0000 7f0211000640 1 -- 172.21.15.86:0/3343784738 >> 172.21.15.86:0/3343784738 conn(0x7f020c0cffc0 msgr2=0x7f020c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:40.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.018+0000 7f0211000640 1 -- 172.21.15.86:0/3343784738 shutdown_connections 2024-04-25T14:08:40.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.018+0000 7f0211000640 1 -- 172.21.15.86:0/3343784738 wait complete. 2024-04-25T14:08:40.039 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T14:08:40.039 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:08:40.112 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:08:40.200 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T14:08:40.200 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:08:40.200 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:08:40.209 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2893 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T14:08:40.209 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:08:40.209 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:08:40.218 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4517 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T14:08:40.218 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:08:40.218 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:08:40.261 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2893 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T14:08:40.261 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:08:40.261 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:08:40.270 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4517 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T14:08:40.270 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T14:08:40.271 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:08:40.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.446+0000 7f4acea00640 1 Processor -- start 2024-04-25T14:08:40.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.446+0000 7f4acea00640 1 -- start start 2024-04-25T14:08:40.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.450+0000 7f4acea00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4ac80cfca0 0x7f4ac806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:40.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.450+0000 7f4acea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4ac80d16e0 0x7f4ac80d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:40.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.450+0000 7f4acea00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ac80d2030 con 0x7f4ac806c670 2024-04-25T14:08:40.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.450+0000 7f4acea00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4ac8070470 con 0x7f4ac80d16e0 2024-04-25T14:08:40.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.450+0000 7f4acea00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f4ac8072bd0 con 0x7f4ac80cfca0 2024-04-25T14:08:40.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.450+0000 7f4abf400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4ac80d16e0 0x7f4ac80d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:40.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.450+0000 7f4abf400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4ac80d16e0 0x7f4ac80d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48804/0 (socket says 172.21.15.86:48804) 2024-04-25T14:08:40.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.450+0000 7f4abf400640 1 -- 172.21.15.86:0/1349624938 learned_addr learned my addr 172.21.15.86:0/1349624938 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:40.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.450+0000 7f4ac7e00640 1 --2- 172.21.15.86:0/1349624938 >> v2:172.21.15.155:3300/0 conn(0x7f4ac80cfca0 0x7f4ac806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.450+0000 7f4abf400640 1 -- 172.21.15.86:0/1349624938 >> v2:172.21.15.155:3300/0 conn(0x7f4ac80cfca0 msgr2=0x7f4ac806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.450+0000 7f4abf400640 1 --2- 172.21.15.86:0/1349624938 >> v2:172.21.15.155:3300/0 conn(0x7f4ac80cfca0 0x7f4ac806f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.450+0000 7f4abf400640 1 -- 172.21.15.86:0/1349624938 >> v1:172.21.15.155:6789/0 conn(0x7f4ac806c670 legacy=0x7f4ac806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.450+0000 7f4abf400640 1 -- 172.21.15.86:0/1349624938 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac806f930 con 0x7f4ac80d16e0 2024-04-25T14:08:40.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.450+0000 7f4abf400640 1 --2- 172.21.15.86:0/1349624938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ac80d16e0 0x7f4ac80d1af0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f4ac80d22a0 tx=0x7f4ab001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a443f0c7bfdc65da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:40.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.450+0000 7f4ac7400640 1 -- 172.21.15.86:0/1349624938 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab00ab430 con 0x7f4ac80d16e0 2024-04-25T14:08:40.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.454+0000 7f4ac7400640 1 -- 172.21.15.86:0/1349624938 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ab00b0070 con 0x7f4ac80d16e0 2024-04-25T14:08:40.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.454+0000 7f4ac7400640 1 -- 172.21.15.86:0/1349624938 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab00ab7f0 con 0x7f4ac80d16e0 2024-04-25T14:08:40.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.454+0000 7f4acea00640 1 -- 172.21.15.86:0/1349624938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ac80d16e0 msgr2=0x7f4ac80d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:40.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.454+0000 7f4acea00640 1 --2- 172.21.15.86:0/1349624938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ac80d16e0 0x7f4ac80d1af0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f4ac80d22a0 tx=0x7f4ab001c520 comp rx=0 tx=0).stop 2024-04-25T14:08:40.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.454+0000 7f4acea00640 1 -- 172.21.15.86:0/1349624938 shutdown_connections 2024-04-25T14:08:40.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.454+0000 7f4acea00640 1 --2- 172.21.15.86:0/1349624938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ac80d16e0 0x7f4ac80d1af0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:40.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.454+0000 7f4acea00640 1 --2- 172.21.15.86:0/1349624938 >> v2:172.21.15.155:3300/0 conn(0x7f4ac80cfca0 0x7f4ac806f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:40.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.454+0000 7f4acea00640 1 -- 172.21.15.86:0/1349624938 >> 172.21.15.86:0/1349624938 conn(0x7f4ac80b4650 msgr2=0x7f4ac80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:40.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.454+0000 7f4acea00640 1 -- 172.21.15.86:0/1349624938 shutdown_connections 2024-04-25T14:08:40.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.454+0000 7f4acea00640 1 -- 172.21.15.86:0/1349624938 wait complete. 2024-04-25T14:08:40.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.454+0000 7f4acea00640 1 Processor -- start 2024-04-25T14:08:40.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.454+0000 7f4acea00640 1 -- start start 2024-04-25T14:08:40.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.458+0000 7f4acea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ac806c670 0x7f4ac81fd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:40.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.458+0000 7f4acea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ac80cfca0 0x7f4ac80afb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:40.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.458+0000 7f4acea00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ac80d16e0 0x7f4ac815a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:40.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.458+0000 7f4acea00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac816a070 con 0x7f4ac806c670 2024-04-25T14:08:40.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.458+0000 7f4acea00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac8169ef0 con 0x7f4ac80cfca0 2024-04-25T14:08:40.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.458+0000 7f4acea00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4ac816a1f0 con 0x7f4ac80d16e0 2024-04-25T14:08:40.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.458+0000 7f4ac7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ac806c670 0x7f4ac81fd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:40.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.458+0000 7f4ac7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ac806c670 0x7f4ac81fd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48806/0 (socket says 172.21.15.86:48806) 2024-04-25T14:08:40.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.458+0000 7f4ac7e00640 1 -- 172.21.15.86:0/1735774966 learned_addr learned my addr 172.21.15.86:0/1735774966 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:40.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.458+0000 7f4abf400640 1 --2- 172.21.15.86:0/1735774966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ac80cfca0 0x7f4ac80afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:40.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.458+0000 7f4acca00640 1 --2- 172.21.15.86:0/1735774966 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ac80d16e0 0x7f4ac815a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:40.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.458+0000 7f4ac7e00640 1 -- 172.21.15.86:0/1735774966 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ac80d16e0 msgr2=0x7f4ac815a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:40.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.458+0000 7f4ac7e00640 1 --2- 172.21.15.86:0/1735774966 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ac80d16e0 0x7f4ac815a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:40.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.458+0000 7f4ac7e00640 1 -- 172.21.15.86:0/1735774966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ac80cfca0 msgr2=0x7f4ac80afb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:40.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.458+0000 7f4ac7e00640 1 --2- 172.21.15.86:0/1735774966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ac80cfca0 0x7f4ac80afb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:40.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.458+0000 7f4ac7e00640 1 -- 172.21.15.86:0/1735774966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac80ca1a0 con 0x7f4ac806c670 2024-04-25T14:08:40.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.458+0000 7f4abf400640 1 --2- 172.21.15.86:0/1735774966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ac80cfca0 0x7f4ac80afb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:40.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.458+0000 7f4ac7e00640 1 --2- 172.21.15.86:0/1735774966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ac806c670 0x7f4ac81fd280 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800d5f0 tx=0x7f4ab800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:40.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.458+0000 7f4ac4c00640 1 -- 172.21.15.86:0/1735774966 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab80125f0 con 0x7f4ac806c670 2024-04-25T14:08:40.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.458+0000 7f4ac4c00640 1 -- 172.21.15.86:0/1735774966 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ab8019070 con 0x7f4ac806c670 2024-04-25T14:08:40.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.458+0000 7f4ac4c00640 1 -- 172.21.15.86:0/1735774966 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab80129d0 con 0x7f4ac806c670 2024-04-25T14:08:40.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.458+0000 7f4acea00640 1 -- 172.21.15.86:0/1735774966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ac80ca480 con 0x7f4ac806c670 2024-04-25T14:08:40.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.458+0000 7f4acea00640 1 -- 172.21.15.86:0/1735774966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ac80caa40 con 0x7f4ac806c670 2024-04-25T14:08:40.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.458+0000 7f4acea00640 1 -- 172.21.15.86:0/1735774966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ac809f4b0 con 0x7f4ac806c670 2024-04-25T14:08:40.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.462+0000 7f4ac4c00640 1 -- 172.21.15.86:0/1735774966 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7f4ab8012b70 con 0x7f4ac806c670 2024-04-25T14:08:40.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.462+0000 7f4ac4c00640 1 --2- 172.21.15.86:0/1735774966 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f4aa406f550 0x7f4aa4071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:40.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.462+0000 7f4ac4c00640 1 -- 172.21.15.86:0/1735774966 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6195+0+0 (secure 0 0 0) 0x7f4ab808f960 con 0x7f4ac806c670 2024-04-25T14:08:40.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.462+0000 7f4abf400640 1 --2- 172.21.15.86:0/1735774966 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f4aa406f550 0x7f4aa4071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:40.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.462+0000 7f4abf400640 1 --2- 172.21.15.86:0/1735774966 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f4aa406f550 0x7f4aa4071a60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00af000 tx=0x7f4ab00a7000 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:40.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.466+0000 7f4ac4c00640 1 -- 172.21.15.86:0/1735774966 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4ab8023020 con 0x7f4ac806c670 2024-04-25T14:08:40.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.654+0000 7f4acea00640 1 -- 172.21.15.86:0/1735774966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ac8069d00 con 0x7f4ac806c670 2024-04-25T14:08:40.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.658+0000 7f4ac4c00640 1 -- 172.21.15.86:0/1735774966 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+3875 (secure 0 0 0) 0x7f4ab805c4b0 con 0x7f4ac806c670 2024-04-25T14:08:40.658 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:08:40.658 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":60,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1210051860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1210051860},{"type":"v1","addr":"172.21.15.155:6833","nonce":1210051860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2868300438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2868300438},{"type":"v1","addr":"172.21.15.155:6835","nonce":2868300438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":7093,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3396400143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3396400143},{"type":"v1","addr":"172.21.15.86:6836","nonce":3396400143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":7281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1725840093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":1725840093},{"type":"v1","addr":"172.21.15.86:6837","nonce":1725840093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[]} 2024-04-25T14:08:40.658 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 60 2024-04-25T14:08:40.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.658+0000 7f4acea00640 1 -- 172.21.15.86:0/1735774966 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f4aa406f550 msgr2=0x7f4aa4071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:40.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.658+0000 7f4acea00640 1 --2- 172.21.15.86:0/1735774966 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f4aa406f550 0x7f4aa4071a60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00af000 tx=0x7f4ab00a7000 comp rx=0 tx=0).stop 2024-04-25T14:08:40.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.658+0000 7f4acea00640 1 -- 172.21.15.86:0/1735774966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ac806c670 msgr2=0x7f4ac81fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:40.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.658+0000 7f4acea00640 1 --2- 172.21.15.86:0/1735774966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ac806c670 0x7f4ac81fd280 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800d5f0 tx=0x7f4ab800dab0 comp rx=0 tx=0).stop 2024-04-25T14:08:40.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.658+0000 7f4acea00640 1 -- 172.21.15.86:0/1735774966 shutdown_connections 2024-04-25T14:08:40.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.658+0000 7f4acea00640 1 --2- 172.21.15.86:0/1735774966 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f4aa406f550 0x7f4aa4071a60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:40.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.658+0000 7f4acea00640 1 --2- 172.21.15.86:0/1735774966 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ac80d16e0 0x7f4ac815a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:40.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.658+0000 7f4acea00640 1 --2- 172.21.15.86:0/1735774966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ac80cfca0 0x7f4ac80afb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:40.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.658+0000 7f4acea00640 1 --2- 172.21.15.86:0/1735774966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ac806c670 0x7f4ac81fd280 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:40.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.658+0000 7f4acea00640 1 -- 172.21.15.86:0/1735774966 >> 172.21.15.86:0/1735774966 conn(0x7f4ac80b4650 msgr2=0x7f4ac815c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:40.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.658+0000 7f4acea00640 1 -- 172.21.15.86:0/1735774966 shutdown_connections 2024-04-25T14:08:40.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.658+0000 7f4acea00640 1 -- 172.21.15.86:0/1735774966 wait complete. 2024-04-25T14:08:40.680 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T14:08:40.680 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T14:08:40.680 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T14:08:40.681 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:08:40.681 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T14:08:40.681 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T14:08:40.681 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T14:08:40.681 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:08:40.681 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T14:08:40.681 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T14:08:40.682 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T14:08:40.682 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:08:40.682 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T14:08:40.682 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T14:08:40.682 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T14:08:40.682 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:08:40.698 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T14:08:40.698 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T14:08:40.700 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T14:08:40.700 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T14:08:40.710 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T14:08:40.710 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T14:08:40.711 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T14:08:40.711 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T14:08:40.748 INFO:tasks.ceph.mds.b:Started 2024-04-25T14:08:40.748 DEBUG:teuthology.parallel:result is None 2024-04-25T14:08:40.750 INFO:tasks.ceph.mds.d:Started 2024-04-25T14:08:40.750 DEBUG:teuthology.parallel:result is None 2024-04-25T14:08:40.759 INFO:tasks.ceph.mds.a:Started 2024-04-25T14:08:40.760 DEBUG:teuthology.parallel:result is None 2024-04-25T14:08:40.762 INFO:tasks.ceph.mds.c:Started 2024-04-25T14:08:40.762 DEBUG:teuthology.parallel:result is None 2024-04-25T14:08:40.762 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T14:08:40.853 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2024-04-25T14:08:40.856 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2024-04-25T14:08:40.859 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-25T14:08:40.864 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-04-25T14:08:40.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.918+0000 7f5c57800640 1 Processor -- start 2024-04-25T14:08:40.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.918+0000 7f5c57800640 1 -- start start 2024-04-25T14:08:40.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.922+0000 7f5c57800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5c50070330 0x7f5c500d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:40.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.922+0000 7f5c57800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c500d2030 con 0x7f5c500d1c20 2024-04-25T14:08:40.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.922+0000 7f5c57800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c5015e5c0 con 0x7f5c500ce790 2024-04-25T14:08:40.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.922+0000 7f5c57800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f5c5015e8a0 con 0x7f5c50070330 2024-04-25T14:08:40.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.922+0000 7f5c4fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5c500d1c20 0x7f5c5006ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60056/0 (socket says 172.21.15.86:60056) 2024-04-25T14:08:40.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.922+0000 7f5c4fe00640 1 -- 172.21.15.86:0/3937533863 learned_addr learned my addr 172.21.15.86:0/3937533863 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:40.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.922+0000 7f5c54e00640 1 --2- 172.21.15.86:0/3937533863 >> v2:172.21.15.155:3300/0 conn(0x7f5c50070330 0x7f5c500d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:40.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.922+0000 7f5c4f400640 1 -- 172.21.15.86:0/3937533863 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 403962680 0 0) 0x7f5c3c001120 con 0x7f5c500d1c20 2024-04-25T14:08:40.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.922+0000 7f5c4f400640 1 -- 172.21.15.86:0/3937533863 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c38003530 con 0x7f5c500d1c20 2024-04-25T14:08:40.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.922+0000 7f5c4f400640 1 -- 172.21.15.86:0/3937533863 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1559159156 0 0) 0x7f5c5015e5c0 con 0x7f5c500ce790 2024-04-25T14:08:40.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.922+0000 7f5c54e00640 1 -- 172.21.15.86:0/3937533863 >> v1:172.21.15.155:6789/0 conn(0x7f5c500ce790 legacy=0x7f5c5006f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:40.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.922+0000 7f5c54e00640 1 -- 172.21.15.86:0/3937533863 >> v1:172.21.15.86:6789/0 conn(0x7f5c500d1c20 legacy=0x7f5c5006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:40.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.922+0000 7f5c54e00640 1 -- 172.21.15.86:0/3937533863 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5015eb80 con 0x7f5c50070330 2024-04-25T14:08:40.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.926+0000 7f5c54e00640 1 --2- 172.21.15.86:0/3937533863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c50070330 0x7f5c500d16e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5c440097f0 tx=0x7f5c4401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11166521e639f502 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:40.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.926+0000 7f5c4f400640 1 -- 172.21.15.86:0/3937533863 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c440b0070 con 0x7f5c50070330 2024-04-25T14:08:40.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.926+0000 7f5c4f400640 1 -- 172.21.15.86:0/3937533863 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c440ab600 con 0x7f5c50070330 2024-04-25T14:08:40.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.926+0000 7f5c4f400640 1 -- 172.21.15.86:0/3937533863 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c440ab9d0 con 0x7f5c50070330 2024-04-25T14:08:40.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.926+0000 7f5c57800640 1 -- 172.21.15.86:0/3937533863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c50070330 msgr2=0x7f5c500d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:40.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.926+0000 7f5c57800640 1 --2- 172.21.15.86:0/3937533863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c50070330 0x7f5c500d16e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5c440097f0 tx=0x7f5c4401c410 comp rx=0 tx=0).stop 2024-04-25T14:08:40.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.926+0000 7f5c57800640 1 -- 172.21.15.86:0/3937533863 shutdown_connections 2024-04-25T14:08:40.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.926+0000 7f5c57800640 1 --2- 172.21.15.86:0/3937533863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c50070330 0x7f5c500d16e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:40.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.926+0000 7f5c57800640 1 -- 172.21.15.86:0/3937533863 >> 172.21.15.86:0/3937533863 conn(0x7f5c500b4650 msgr2=0x7f5c500b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:40.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.926+0000 7f5c57800640 1 -- 172.21.15.86:0/3937533863 shutdown_connections 2024-04-25T14:08:40.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.926+0000 7f5c57800640 1 -- 172.21.15.86:0/3937533863 wait complete. 2024-04-25T14:08:40.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.930+0000 7f5c57800640 1 Processor -- start 2024-04-25T14:08:40.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.930+0000 7f5c57800640 1 -- start start 2024-04-25T14:08:40.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.930+0000 7f5c57800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c50070330 0x7f5c50158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:40.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.930+0000 7f5c57800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c500ce790 0x7f5c500c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:40.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.930+0000 7f5c57800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c500d1c20 0x7f5c500c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:40.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.930+0000 7f5c57800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c500708c0 con 0x7f5c500d1c20 2024-04-25T14:08:40.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.930+0000 7f5c57800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5c50070740 con 0x7f5c500ce790 2024-04-25T14:08:40.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.930+0000 7f5c57800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5c5006fb50 con 0x7f5c50070330 2024-04-25T14:08:40.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.930+0000 7f5c55800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c500d1c20 0x7f5c500c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:40.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.930+0000 7f5c55800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c500d1c20 0x7f5c500c9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48844/0 (socket says 172.21.15.86:48844) 2024-04-25T14:08:40.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.930+0000 7f5c55800640 1 -- 172.21.15.86:0/2541648980 learned_addr learned my addr 172.21.15.86:0/2541648980 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:40.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.930+0000 7f5c54e00640 1 --2- 172.21.15.86:0/2541648980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c50070330 0x7f5c50158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:40.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.930+0000 7f5c4fe00640 1 --2- 172.21.15.86:0/2541648980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c500ce790 0x7f5c500c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:40.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.930+0000 7f5c54e00640 1 -- 172.21.15.86:0/2541648980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c500ce790 msgr2=0x7f5c500c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:40.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.930+0000 7f5c54e00640 1 --2- 172.21.15.86:0/2541648980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c500ce790 0x7f5c500c9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:40.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.930+0000 7f5c54e00640 1 -- 172.21.15.86:0/2541648980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c500d1c20 msgr2=0x7f5c500c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:40.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.930+0000 7f5c54e00640 1 --2- 172.21.15.86:0/2541648980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c500d1c20 0x7f5c500c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:40.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.930+0000 7f5c54e00640 1 -- 172.21.15.86:0/2541648980 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c500ca1a0 con 0x7f5c50070330 2024-04-25T14:08:40.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.930+0000 7f5c55800640 1 --2- 172.21.15.86:0/2541648980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c500d1c20 0x7f5c500c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:08:40.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.930+0000 7f5c54e00640 1 --2- 172.21.15.86:0/2541648980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c50070330 0x7f5c50158240 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5c50070b80 tx=0x7f5c4401ac50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:40.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.930+0000 7f5c4cc00640 1 -- 172.21.15.86:0/2541648980 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c440b0050 con 0x7f5c50070330 2024-04-25T14:08:40.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.930+0000 7f5c4cc00640 1 -- 172.21.15.86:0/2541648980 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c440bc070 con 0x7f5c50070330 2024-04-25T14:08:40.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.930+0000 7f5c4cc00640 1 -- 172.21.15.86:0/2541648980 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44006e20 con 0x7f5c50070330 2024-04-25T14:08:40.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.930+0000 7f5c57800640 1 -- 172.21.15.86:0/2541648980 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c500ca480 con 0x7f5c50070330 2024-04-25T14:08:40.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.934+0000 7f5c57800640 1 -- 172.21.15.86:0/2541648980 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c500caa40 con 0x7f5c50070330 2024-04-25T14:08:40.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.934+0000 7f5c57800640 1 -- 172.21.15.86:0/2541648980 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c50069d00 con 0x7f5c50070330 2024-04-25T14:08:40.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.934+0000 7f5c4cc00640 1 -- 172.21.15.86:0/2541648980 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7f5c440b5830 con 0x7f5c50070330 2024-04-25T14:08:40.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.934+0000 7f5c4cc00640 1 --2- 172.21.15.86:0/2541648980 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f5c3006f550 0x7f5c30071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:40.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.934+0000 7f5c4cc00640 1 -- 172.21.15.86:0/2541648980 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(79..79 src has 1..79) ==== 6195+0+0 (secure 0 0 0) 0x7f5c44129e10 con 0x7f5c50070330 2024-04-25T14:08:40.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.934+0000 7f5c4fe00640 1 --2- 172.21.15.86:0/2541648980 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f5c3006f550 0x7f5c30071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:40.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.938+0000 7f5c4fe00640 1 --2- 172.21.15.86:0/2541648980 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f5c3006f550 0x7f5c30071a60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5c501fca10 tx=0x7f5c3c009040 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:40.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:40.938+0000 7f5c4cc00640 1 -- 172.21.15.86:0/2541648980 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5c44016090 con 0x7f5c50070330 2024-04-25T14:08:41.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.106+0000 7f5c57800640 1 -- 172.21.15.86:0/2541648980 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f5c5006db10 con 0x7f5c50070330 2024-04-25T14:08:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.678+0000 7f5c4cc00640 1 -- 172.21.15.86:0/2541648980 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v81) ==== 97+0+0 (secure 0 0 0) 0x7f5c440f63a0 con 0x7f5c50070330 2024-04-25T14:08:41.680 INFO:teuthology.orchestra.run.smithi086.stderr: removed all blocklist entries 2024-04-25T14:08:41.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.682+0000 7f5c57800640 1 -- 172.21.15.86:0/2541648980 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f5c3006f550 msgr2=0x7f5c30071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:41.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.682+0000 7f5c57800640 1 --2- 172.21.15.86:0/2541648980 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f5c3006f550 0x7f5c30071a60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5c501fca10 tx=0x7f5c3c009040 comp rx=0 tx=0).stop 2024-04-25T14:08:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.682+0000 7f5c57800640 1 -- 172.21.15.86:0/2541648980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c50070330 msgr2=0x7f5c50158240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.682+0000 7f5c57800640 1 --2- 172.21.15.86:0/2541648980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c50070330 0x7f5c50158240 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5c50070b80 tx=0x7f5c4401ac50 comp rx=0 tx=0).stop 2024-04-25T14:08:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.682+0000 7f5c57800640 1 -- 172.21.15.86:0/2541648980 shutdown_connections 2024-04-25T14:08:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.682+0000 7f5c57800640 1 --2- 172.21.15.86:0/2541648980 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f5c3006f550 0x7f5c30071a60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.682+0000 7f5c57800640 1 --2- 172.21.15.86:0/2541648980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c500d1c20 0x7f5c500c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.682+0000 7f5c57800640 1 --2- 172.21.15.86:0/2541648980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c500ce790 0x7f5c500c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.682+0000 7f5c57800640 1 --2- 172.21.15.86:0/2541648980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c50070330 0x7f5c50158240 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.682+0000 7f5c57800640 1 -- 172.21.15.86:0/2541648980 >> 172.21.15.86:0/2541648980 conn(0x7f5c500b4650 msgr2=0x7f5c50072880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.682+0000 7f5c57800640 1 -- 172.21.15.86:0/2541648980 shutdown_connections 2024-04-25T14:08:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.682+0000 7f5c57800640 1 -- 172.21.15.86:0/2541648980 wait complete. 2024-04-25T14:08:41.707 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T14:08:41.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.854+0000 7f137f000640 1 Processor -- start 2024-04-25T14:08:41.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.854+0000 7f137f000640 1 -- start start 2024-04-25T14:08:41.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.854+0000 7f137f000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13780d16e0 0x7f13780d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:41.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.854+0000 7f137f000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f13780730b0 0x7f13780734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:41.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.854+0000 7f137f000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13780d2030 con 0x7f13780cfca0 2024-04-25T14:08:41.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.854+0000 7f137f000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1378070470 con 0x7f13780730b0 2024-04-25T14:08:41.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.854+0000 7f137f000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1378072bd0 con 0x7f13780d16e0 2024-04-25T14:08:41.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.854+0000 7f137d000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f13780730b0 0x7f13780734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:41.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.854+0000 7f136f400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13780d16e0 0x7f13780d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:41.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.854+0000 7f1377e00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f13780cfca0 0x7f137806f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:52388/0 (socket says 172.21.15.86:52388) 2024-04-25T14:08:41.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.854+0000 7f136f400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13780d16e0 0x7f13780d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:48994/0 (socket says 172.21.15.86:48994) 2024-04-25T14:08:41.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.854+0000 7f1377e00640 1 -- 172.21.15.86:0/2600122766 learned_addr learned my addr 172.21.15.86:0/2600122766 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:41.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.854+0000 7f137d000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f13780730b0 0x7f13780734c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:49952/0 (socket says 172.21.15.86:49952) 2024-04-25T14:08:41.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.854+0000 7f136f400640 1 -- 172.21.15.86:0/2600122766 >> v2:172.21.15.155:3300/0 conn(0x7f13780730b0 msgr2=0x7f13780734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:41.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.854+0000 7f136f400640 1 --2- 172.21.15.86:0/2600122766 >> v2:172.21.15.155:3300/0 conn(0x7f13780730b0 0x7f13780734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:41.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.854+0000 7f136f400640 1 -- 172.21.15.86:0/2600122766 >> v1:172.21.15.155:6789/0 conn(0x7f13780cfca0 legacy=0x7f137806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:41.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.854+0000 7f136f400640 1 -- 172.21.15.86:0/2600122766 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137806fbd0 con 0x7f13780d16e0 2024-04-25T14:08:41.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.854+0000 7f1377400640 1 -- 172.21.15.86:0/2600122766 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3476253010 0 0) 0x7f1368001120 con 0x7f13780cfca0 2024-04-25T14:08:41.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.854+0000 7f137d000640 1 --2- 172.21.15.86:0/2600122766 >> v2:172.21.15.155:3300/0 conn(0x7f13780730b0 0x7f13780734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:08:41.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.858+0000 7f136f400640 1 --2- 172.21.15.86:0/2600122766 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13780d16e0 0x7f13780d1af0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f13600097c0 tx=0x7f136001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d087668390b19bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:41.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.858+0000 7f1377400640 1 -- 172.21.15.86:0/2600122766 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13600b1070 con 0x7f13780d16e0 2024-04-25T14:08:41.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.858+0000 7f1377400640 1 -- 172.21.15.86:0/2600122766 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13600ac410 con 0x7f13780d16e0 2024-04-25T14:08:41.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.858+0000 7f1377400640 1 -- 172.21.15.86:0/2600122766 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13600ac7e0 con 0x7f13780d16e0 2024-04-25T14:08:41.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.858+0000 7f137f000640 1 -- 172.21.15.86:0/2600122766 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13780d16e0 msgr2=0x7f13780d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:41.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.858+0000 7f137f000640 1 --2- 172.21.15.86:0/2600122766 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13780d16e0 0x7f13780d1af0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f13600097c0 tx=0x7f136001c4a0 comp rx=0 tx=0).stop 2024-04-25T14:08:41.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.858+0000 7f137f000640 1 -- 172.21.15.86:0/2600122766 shutdown_connections 2024-04-25T14:08:41.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.858+0000 7f137f000640 1 --2- 172.21.15.86:0/2600122766 >> v2:172.21.15.155:3300/0 conn(0x7f13780730b0 0x7f13780734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:41.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.858+0000 7f137f000640 1 --2- 172.21.15.86:0/2600122766 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13780d16e0 0x7f13780d1af0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:41.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.858+0000 7f137f000640 1 -- 172.21.15.86:0/2600122766 >> 172.21.15.86:0/2600122766 conn(0x7f13780b4650 msgr2=0x7f13780b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:41.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.858+0000 7f137f000640 1 -- 172.21.15.86:0/2600122766 shutdown_connections 2024-04-25T14:08:41.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.862+0000 7f137f000640 1 -- 172.21.15.86:0/2600122766 wait complete. 2024-04-25T14:08:41.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.862+0000 7f137f000640 1 Processor -- start 2024-04-25T14:08:41.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.862+0000 7f137f000640 1 -- start start 2024-04-25T14:08:41.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.862+0000 7f137f000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13780730b0 0x7f137809a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:41.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.862+0000 7f137f000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13780cfca0 0x7f1378098aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:41.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.862+0000 7f137f000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13780d16e0 0x7f13780ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:41.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.862+0000 7f137f000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1378169a70 con 0x7f13780730b0 2024-04-25T14:08:41.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.862+0000 7f137f000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f13781698f0 con 0x7f13780cfca0 2024-04-25T14:08:41.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.862+0000 7f137f000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1378169bf0 con 0x7f13780d16e0 2024-04-25T14:08:41.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.862+0000 7f1377e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13780730b0 0x7f137809a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:41.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.862+0000 7f1377e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13780730b0 0x7f137809a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48848/0 (socket says 172.21.15.86:48848) 2024-04-25T14:08:41.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.862+0000 7f1377e00640 1 -- 172.21.15.86:0/1656990819 learned_addr learned my addr 172.21.15.86:0/1656990819 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:41.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.866+0000 7f136f400640 1 --2- 172.21.15.86:0/1656990819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13780cfca0 0x7f1378098aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:41.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.866+0000 7f137d000640 1 --2- 172.21.15.86:0/1656990819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13780d16e0 0x7f13780ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:41.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.866+0000 7f1377e00640 1 -- 172.21.15.86:0/1656990819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13780d16e0 msgr2=0x7f13780ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:41.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.866+0000 7f1377e00640 1 --2- 172.21.15.86:0/1656990819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13780d16e0 0x7f13780ca7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:41.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.866+0000 7f1377e00640 1 -- 172.21.15.86:0/1656990819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13780cfca0 msgr2=0x7f1378098aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:41.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.866+0000 7f1377e00640 1 --2- 172.21.15.86:0/1656990819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13780cfca0 0x7f1378098aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:41.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.866+0000 7f1377e00640 1 -- 172.21.15.86:0/1656990819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13780993f0 con 0x7f13780730b0 2024-04-25T14:08:41.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.866+0000 7f1377e00640 1 --2- 172.21.15.86:0/1656990819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13780730b0 0x7f137809a490 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f136800c450 tx=0x7f136800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:41.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.866+0000 7f1374c00640 1 -- 172.21.15.86:0/1656990819 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13680074e0 con 0x7f13780730b0 2024-04-25T14:08:41.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.866+0000 7f1374c00640 1 -- 172.21.15.86:0/1656990819 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13680047f0 con 0x7f13780730b0 2024-04-25T14:08:41.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.866+0000 7f1374c00640 1 -- 172.21.15.86:0/1656990819 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1368007680 con 0x7f13780730b0 2024-04-25T14:08:41.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.866+0000 7f137f000640 1 -- 172.21.15.86:0/1656990819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13780996d0 con 0x7f13780730b0 2024-04-25T14:08:41.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.866+0000 7f137f000640 1 -- 172.21.15.86:0/1656990819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1378099c90 con 0x7f13780730b0 2024-04-25T14:08:41.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.866+0000 7f137f000640 1 -- 172.21.15.86:0/1656990819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f137809f4b0 con 0x7f13780730b0 2024-04-25T14:08:41.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.866+0000 7f1374c00640 1 -- 172.21.15.86:0/1656990819 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7f1368011070 con 0x7f13780730b0 2024-04-25T14:08:41.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.870+0000 7f1374c00640 1 --2- 172.21.15.86:0/1656990819 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f135406f550 0x7f1354071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:41.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.870+0000 7f1374c00640 1 -- 172.21.15.86:0/1656990819 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5249+0+0 (secure 0 0 0) 0x7f136808ec90 con 0x7f13780730b0 2024-04-25T14:08:41.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.870+0000 7f136f400640 1 --2- 172.21.15.86:0/1656990819 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f135406f550 0x7f1354071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:41.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.870+0000 7f136f400640 1 --2- 172.21.15.86:0/1656990819 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f135406f550 0x7f1354071a60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f13600ae000 tx=0x7f1360008040 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:41.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:41.874+0000 7f1374c00640 1 -- 172.21.15.86:0/1656990819 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f13780d2030 con 0x7f13780730b0 2024-04-25T14:08:42.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.086+0000 7f137f000640 1 -- 172.21.15.86:0/1656990819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1378069d00 con 0x7f13780730b0 2024-04-25T14:08:42.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.086+0000 7f1374c00640 1 -- 172.21.15.86:0/1656990819 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v24) ==== 80+0+5238 (secure 0 0 0) 0x7f136805b5e0 con 0x7f13780730b0 2024-04-25T14:08:42.087 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:08:42.087 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:08:42.087 INFO:teuthology.orchestra.run.smithi086.stdout: "auth_dump": [ 2024-04-25T14:08:42.087 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:08:42.087 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.a", 2024-04-25T14:08:42.087 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmWk5fAxAAKEBKhWewpl23MUcvMi8qDg==", 2024-04-25T14:08:42.087 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:08:42.087 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:08:42.087 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:08:42.087 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:08:42.088 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:08:42.088 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:08:42.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:08:42.088 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:08:42.088 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.b", 2024-04-25T14:08:42.088 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmDhzOEBAABNys/NAEhucQ7A+HzQlzQw==", 2024-04-25T14:08:42.088 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:08:42.088 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:08:42.088 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:08:42.088 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:08:42.088 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:08:42.088 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:08:42.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:08:42.088 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:08:42.088 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.c", 2024-04-25T14:08:42.089 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmA/hNCxAAclDGbIjLfjqyXn0MwX/IkA==", 2024-04-25T14:08:42.089 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:08:42.089 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:08:42.089 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:08:42.089 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:08:42.089 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:08:42.089 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:08:42.089 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:08:42.089 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:08:42.089 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.d", 2024-04-25T14:08:42.089 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmemCcGBAAGrCx6C850qRJSWYjebxOBA==", 2024-04-25T14:08:42.089 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:08:42.089 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:08:42.089 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:08:42.090 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:08:42.090 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:08:42.090 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:08:42.090 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:08:42.090 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:08:42.090 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.0", 2024-04-25T14:08:42.090 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCOYipm9etgGxAAemg9CqXN0ShduDtDuKF2Zw==", 2024-04-25T14:08:42.090 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:08:42.090 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:08:42.090 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:08:42.090 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:08:42.090 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:08:42.090 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:08:42.090 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:08:42.090 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.1", 2024-04-25T14:08:42.091 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCRYipmBCiNJhAABAWHk8NEGVvqieL2klGhmA==", 2024-04-25T14:08:42.091 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:08:42.091 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:08:42.091 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:08:42.091 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:08:42.091 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:08:42.091 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:08:42.091 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:08:42.091 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.2", 2024-04-25T14:08:42.091 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCUYipm/XGYLxAAPXsw1svKseFzFiuDUjAsJQ==", 2024-04-25T14:08:42.091 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:08:42.091 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:08:42.091 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:08:42.091 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:08:42.092 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:08:42.092 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:08:42.092 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:08:42.092 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.3", 2024-04-25T14:08:42.092 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCXYipmlSRaORAAwpKyzZDdVYHWl2pAsQjmpw==", 2024-04-25T14:08:42.092 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:08:42.092 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:08:42.092 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:08:42.092 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:08:42.092 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:08:42.092 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:08:42.092 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:08:42.092 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.4", 2024-04-25T14:08:42.092 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCdYipmBI3FMBAAV2NdggfRSFnQvrcOC2YqiA==", 2024-04-25T14:08:42.093 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:08:42.093 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:08:42.093 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:08:42.093 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:08:42.093 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:08:42.093 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:08:42.093 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:08:42.093 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.5", 2024-04-25T14:08:42.093 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQChYipmctYxABAABwmOMgyYuOfRWD9KdHILng==", 2024-04-25T14:08:42.093 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:08:42.093 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:08:42.093 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:08:42.093 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:08:42.093 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:08:42.093 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:08:42.094 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:08:42.094 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.6", 2024-04-25T14:08:42.094 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCkYipmSL7TCRAAhFMWg1AQm54bB+x1GDcusA==", 2024-04-25T14:08:42.094 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:08:42.094 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:08:42.094 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:08:42.094 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:08:42.094 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:08:42.094 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:08:42.094 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:08:42.094 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.7", 2024-04-25T14:08:42.094 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCnYipmZ4F+FRAAEGaZN+byv/qJUFZb7WxTFg==", 2024-04-25T14:08:42.094 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:08:42.094 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:08:42.094 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:08:42.095 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:08:42.095 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:08:42.095 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:08:42.095 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:08:42.095 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.0", 2024-04-25T14:08:42.095 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipm5u4xHRAACDFe3Eb4KLrsdLj4eFe7mA==", 2024-04-25T14:08:42.095 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:08:42.095 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:08:42.095 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-25T14:08:42.095 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T14:08:42.095 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:08:42.095 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:08:42.095 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:08:42.095 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.1", 2024-04-25T14:08:42.096 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmDp1hJRAAQkBc5PdQoMTT7W4HahkTyg==", 2024-04-25T14:08:42.096 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:08:42.096 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:08:42.096 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-25T14:08:42.096 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw pool=fs2_data" 2024-04-25T14:08:42.096 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:08:42.096 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:08:42.096 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:08:42.096 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.2", 2024-04-25T14:08:42.096 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmhbhGIRAAxao/J0WY53IITLETFgwIVw==", 2024-04-25T14:08:42.096 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:08:42.096 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:08:42.096 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-25T14:08:42.096 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw pool=fs2_data" 2024-04-25T14:08:42.096 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:08:42.097 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:08:42.097 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:08:42.097 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.3", 2024-04-25T14:08:42.097 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmJsCBKRAAzXn2qDtblIpqt7U/TUUlyA==", 2024-04-25T14:08:42.097 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:08:42.097 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:08:42.097 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-25T14:08:42.097 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw pool=fs2_data" 2024-04-25T14:08:42.097 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:08:42.097 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:08:42.097 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:08:42.097 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.admin", 2024-04-25T14:08:42.097 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCKYipmTqbVIBAA1VpOtj24f2ibVEIT4EJwmw==", 2024-04-25T14:08:42.097 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:08:42.097 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-25T14:08:42.098 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:08:42.098 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:08:42.098 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:08:42.098 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:08:42.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:08:42.098 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:08:42.098 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.x", 2024-04-25T14:08:42.098 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCKYipmKMzAORAAtRyTNJ5HWi2XwuamoqwB0Q==", 2024-04-25T14:08:42.098 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:08:42.098 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-25T14:08:42.098 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-25T14:08:42.098 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:08:42.098 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:08:42.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:08:42.099 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:08:42.099 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.y", 2024-04-25T14:08:42.099 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCKYipmC7A7NRAAEwVNA0yxt3fwdHXrsV0dvg==", 2024-04-25T14:08:42.099 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:08:42.099 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-25T14:08:42.099 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-25T14:08:42.099 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:08:42.099 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:08:42.099 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:08:42.099 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-25T14:08:42.099 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:08:42.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.086+0000 7f137f000640 1 -- 172.21.15.86:0/1656990819 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f135406f550 msgr2=0x7f1354071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:42.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.086+0000 7f137f000640 1 --2- 172.21.15.86:0/1656990819 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f135406f550 0x7f1354071a60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f13600ae000 tx=0x7f1360008040 comp rx=0 tx=0).stop 2024-04-25T14:08:42.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.086+0000 7f137f000640 1 -- 172.21.15.86:0/1656990819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13780730b0 msgr2=0x7f137809a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:42.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.086+0000 7f137f000640 1 --2- 172.21.15.86:0/1656990819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13780730b0 0x7f137809a490 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f136800c450 tx=0x7f136800c910 comp rx=0 tx=0).stop 2024-04-25T14:08:42.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.086+0000 7f137f000640 1 -- 172.21.15.86:0/1656990819 shutdown_connections 2024-04-25T14:08:42.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.086+0000 7f137f000640 1 --2- 172.21.15.86:0/1656990819 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f135406f550 0x7f1354071a60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:42.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.086+0000 7f137f000640 1 --2- 172.21.15.86:0/1656990819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13780d16e0 0x7f13780ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:42.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.086+0000 7f137f000640 1 --2- 172.21.15.86:0/1656990819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13780cfca0 0x7f1378098aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:42.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.086+0000 7f137f000640 1 --2- 172.21.15.86:0/1656990819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13780730b0 0x7f137809a490 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:42.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.086+0000 7f137f000640 1 -- 172.21.15.86:0/1656990819 >> 172.21.15.86:0/1656990819 conn(0x7f13780b4650 msgr2=0x7f137815c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:42.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.086+0000 7f137f000640 1 -- 172.21.15.86:0/1656990819 shutdown_connections 2024-04-25T14:08:42.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.086+0000 7f137f000640 1 -- 172.21.15.86:0/1656990819 wait complete. 2024-04-25T14:08:42.109 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:08:42.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.254+0000 7f4c3ba00640 1 Processor -- start 2024-04-25T14:08:42.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.254+0000 7f4c3ba00640 1 -- start start 2024-04-25T14:08:42.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.254+0000 7f4c3ba00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4c34071530 0x7f4c340ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:42.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.254+0000 7f4c3ba00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4c3406e700 0x7f4c3406eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:42.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.254+0000 7f4c3ba00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c3406f050 con 0x7f4c340cdd70 2024-04-25T14:08:42.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.254+0000 7f4c3ba00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4c3406d940 con 0x7f4c3406e700 2024-04-25T14:08:42.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.254+0000 7f4c3ba00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f4c3406dc20 con 0x7f4c34071530 2024-04-25T14:08:42.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.254+0000 7f4c39000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f4c340cdd70 0x7f4c34070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60068/0 (socket says 172.21.15.86:60068) 2024-04-25T14:08:42.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.254+0000 7f4c39000640 1 -- 172.21.15.86:0/3146871362 learned_addr learned my addr 172.21.15.86:0/3146871362 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:42.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.254+0000 7f4c39a00640 1 --2- 172.21.15.86:0/3146871362 >> v2:172.21.15.86:3300/0 conn(0x7f4c3406e700 0x7f4c3406eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:42.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.254+0000 7f4c2be00640 1 --2- 172.21.15.86:0/3146871362 >> v2:172.21.15.155:3300/0 conn(0x7f4c34071530 0x7f4c340ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:42.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.258+0000 7f4c33e00640 1 -- 172.21.15.86:0/3146871362 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 245325549 0 0) 0x7f4c3406f050 con 0x7f4c340cdd70 2024-04-25T14:08:42.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.258+0000 7f4c33e00640 1 -- 172.21.15.86:0/3146871362 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c18003530 con 0x7f4c340cdd70 2024-04-25T14:08:42.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.258+0000 7f4c39a00640 1 -- 172.21.15.86:0/3146871362 >> v2:172.21.15.155:3300/0 conn(0x7f4c34071530 msgr2=0x7f4c340ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:42.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.258+0000 7f4c39a00640 1 --2- 172.21.15.86:0/3146871362 >> v2:172.21.15.155:3300/0 conn(0x7f4c34071530 0x7f4c340ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:42.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.258+0000 7f4c39a00640 1 -- 172.21.15.86:0/3146871362 >> v1:172.21.15.86:6789/0 conn(0x7f4c340cdd70 legacy=0x7f4c34070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:42.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.258+0000 7f4c39a00640 1 -- 172.21.15.86:0/3146871362 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c3406df00 con 0x7f4c3406e700 2024-04-25T14:08:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.258+0000 7f4c39a00640 1 --2- 172.21.15.86:0/3146871362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c3406e700 0x7f4c3406eb10 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f4c34073230 tx=0x7f4c2c01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb382364385a2a51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.258+0000 7f4c33e00640 1 -- 172.21.15.86:0/3146871362 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c0b2070 con 0x7f4c3406e700 2024-04-25T14:08:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.258+0000 7f4c33e00640 1 -- 172.21.15.86:0/3146871362 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c2c0a57b0 con 0x7f4c3406e700 2024-04-25T14:08:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.258+0000 7f4c33e00640 1 -- 172.21.15.86:0/3146871362 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c0a5b80 con 0x7f4c3406e700 2024-04-25T14:08:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.258+0000 7f4c3ba00640 1 -- 172.21.15.86:0/3146871362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c3406e700 msgr2=0x7f4c3406eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.258+0000 7f4c3ba00640 1 --2- 172.21.15.86:0/3146871362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c3406e700 0x7f4c3406eb10 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f4c34073230 tx=0x7f4c2c01c8a0 comp rx=0 tx=0).stop 2024-04-25T14:08:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.262+0000 7f4c3ba00640 1 -- 172.21.15.86:0/3146871362 shutdown_connections 2024-04-25T14:08:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.262+0000 7f4c3ba00640 1 --2- 172.21.15.86:0/3146871362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c3406e700 0x7f4c3406eb10 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.262+0000 7f4c3ba00640 1 --2- 172.21.15.86:0/3146871362 >> v2:172.21.15.155:3300/0 conn(0x7f4c34071530 0x7f4c340ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.262+0000 7f4c3ba00640 1 -- 172.21.15.86:0/3146871362 >> 172.21.15.86:0/3146871362 conn(0x7f4c340cffc0 msgr2=0x7f4c340d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.262+0000 7f4c3ba00640 1 -- 172.21.15.86:0/3146871362 shutdown_connections 2024-04-25T14:08:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.262+0000 7f4c3ba00640 1 -- 172.21.15.86:0/3146871362 wait complete. 2024-04-25T14:08:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.262+0000 7f4c3ba00640 1 Processor -- start 2024-04-25T14:08:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.262+0000 7f4c3ba00640 1 -- start start 2024-04-25T14:08:42.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.262+0000 7f4c3ba00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c3406e700 0x7f4c34154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:42.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.262+0000 7f4c3ba00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c340cdd70 0x7f4c34155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:42.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.266+0000 7f4c3ba00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c340b8300 0x7f4c340b8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:42.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.266+0000 7f4c3ba00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3416cc00 con 0x7f4c340b8300 2024-04-25T14:08:42.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.266+0000 7f4c3ba00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4c34164380 con 0x7f4c3406e700 2024-04-25T14:08:42.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.266+0000 7f4c3ba00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4c3416cd80 con 0x7f4c340cdd70 2024-04-25T14:08:42.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.266+0000 7f4c39a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c340b8300 0x7f4c340b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:42.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.266+0000 7f4c39a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c340b8300 0x7f4c340b8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48854/0 (socket says 172.21.15.86:48854) 2024-04-25T14:08:42.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.266+0000 7f4c39a00640 1 -- 172.21.15.86:0/2412734559 learned_addr learned my addr 172.21.15.86:0/2412734559 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:42.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.266+0000 7f4c39000640 1 --2- 172.21.15.86:0/2412734559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c3406e700 0x7f4c34154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:42.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.266+0000 7f4c2be00640 1 --2- 172.21.15.86:0/2412734559 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c340cdd70 0x7f4c34155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:42.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.266+0000 7f4c39a00640 1 -- 172.21.15.86:0/2412734559 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c340cdd70 msgr2=0x7f4c34155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:42.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.266+0000 7f4c39a00640 1 --2- 172.21.15.86:0/2412734559 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c340cdd70 0x7f4c34155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:42.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.266+0000 7f4c39a00640 1 -- 172.21.15.86:0/2412734559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c3406e700 msgr2=0x7f4c34154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:42.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.266+0000 7f4c39a00640 1 --2- 172.21.15.86:0/2412734559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c3406e700 0x7f4c34154dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:42.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.266+0000 7f4c39a00640 1 -- 172.21.15.86:0/2412734559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c3415c2b0 con 0x7f4c340b8300 2024-04-25T14:08:42.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.266+0000 7f4c39a00640 1 --2- 172.21.15.86:0/2412734559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c340b8300 0x7f4c340b8710 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0194b0 tx=0x7f4c2c019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:42.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.266+0000 7f4c31600640 1 -- 172.21.15.86:0/2412734559 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c0b2050 con 0x7f4c340b8300 2024-04-25T14:08:42.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.266+0000 7f4c31600640 1 -- 172.21.15.86:0/2412734559 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c2c0be070 con 0x7f4c340b8300 2024-04-25T14:08:42.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.266+0000 7f4c3ba00640 1 -- 172.21.15.86:0/2412734559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c3415c590 con 0x7f4c340b8300 2024-04-25T14:08:42.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.266+0000 7f4c31600640 1 -- 172.21.15.86:0/2412734559 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c0ad890 con 0x7f4c340b8300 2024-04-25T14:08:42.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.266+0000 7f4c3ba00640 1 -- 172.21.15.86:0/2412734559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c341608b0 con 0x7f4c340b8300 2024-04-25T14:08:42.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.266+0000 7f4c3ba00640 1 -- 172.21.15.86:0/2412734559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c340c99e0 con 0x7f4c340b8300 2024-04-25T14:08:42.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.266+0000 7f4c31600640 1 -- 172.21.15.86:0/2412734559 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7f4c2c0ada30 con 0x7f4c340b8300 2024-04-25T14:08:42.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.270+0000 7f4c31600640 1 --2- 172.21.15.86:0/2412734559 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f4c1006f520 0x7f4c10071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:42.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.270+0000 7f4c31600640 1 -- 172.21.15.86:0/2412734559 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5249+0+0 (secure 0 0 0) 0x7f4c2c12a1b0 con 0x7f4c340b8300 2024-04-25T14:08:42.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.270+0000 7f4c39000640 1 --2- 172.21.15.86:0/2412734559 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f4c1006f520 0x7f4c10071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:42.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.270+0000 7f4c39000640 1 --2- 172.21.15.86:0/2412734559 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f4c1006f520 0x7f4c10071a30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4c24007fd0 tx=0x7f4c24009040 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:42.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.274+0000 7f4c31600640 1 -- 172.21.15.86:0/2412734559 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4c340c99e0 con 0x7f4c340b8300 2024-04-25T14:08:42.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.430+0000 7f4c3ba00640 1 -- 172.21.15.86:0/2412734559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c340720e0 con 0x7f4c340b8300 2024-04-25T14:08:42.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.430+0000 7f4c31600640 1 -- 172.21.15.86:0/2412734559 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+13165 (secure 0 0 0) 0x7f4c2c114380 con 0x7f4c340b8300 2024-04-25T14:08:42.434 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:08:42.434 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":83,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:08:41.690287+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:08:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.434+0000 7f4c3ba00640 1 -- 172.21.15.86:0/2412734559 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f4c1006f520 msgr2=0x7f4c10071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.434+0000 7f4c3ba00640 1 --2- 172.21.15.86:0/2412734559 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f4c1006f520 0x7f4c10071a30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4c24007fd0 tx=0x7f4c24009040 comp rx=0 tx=0).stop 2024-04-25T14:08:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.434+0000 7f4c3ba00640 1 -- 172.21.15.86:0/2412734559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c340b8300 msgr2=0x7f4c340b8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.434+0000 7f4c3ba00640 1 --2- 172.21.15.86:0/2412734559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c340b8300 0x7f4c340b8710 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0194b0 tx=0x7f4c2c019640 comp rx=0 tx=0).stop 2024-04-25T14:08:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.434+0000 7f4c3ba00640 1 -- 172.21.15.86:0/2412734559 shutdown_connections 2024-04-25T14:08:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.434+0000 7f4c3ba00640 1 --2- 172.21.15.86:0/2412734559 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f4c1006f520 0x7f4c10071a30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.434+0000 7f4c3ba00640 1 --2- 172.21.15.86:0/2412734559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c340b8300 0x7f4c340b8710 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.434+0000 7f4c3ba00640 1 --2- 172.21.15.86:0/2412734559 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c340cdd70 0x7f4c34155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.434+0000 7f4c3ba00640 1 --2- 172.21.15.86:0/2412734559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c3406e700 0x7f4c34154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.434+0000 7f4c3ba00640 1 -- 172.21.15.86:0/2412734559 >> 172.21.15.86:0/2412734559 conn(0x7f4c340cffc0 msgr2=0x7f4c340d17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.434+0000 7f4c3ba00640 1 -- 172.21.15.86:0/2412734559 shutdown_connections 2024-04-25T14:08:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.434+0000 7f4c3ba00640 1 -- 172.21.15.86:0/2412734559 wait complete. 2024-04-25T14:08:42.456 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:08:42.456 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:08:42.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.602+0000 7f8806000640 1 Processor -- start 2024-04-25T14:08:42.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.602+0000 7f8806000640 1 -- start start 2024-04-25T14:08:42.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.606+0000 7f8806000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8800074840 0x7f88000af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:42.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.606+0000 7f8806000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88000af770 0x7f88000afb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:42.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.606+0000 7f8806000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8800072bf0 con 0x7f880015d790 2024-04-25T14:08:42.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.606+0000 7f8806000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8800072ed0 con 0x7f8800074840 2024-04-25T14:08:42.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.606+0000 7f8806000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f88000731b0 con 0x7f88000af770 2024-04-25T14:08:42.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.606+0000 7f87f7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8800074840 0x7f88000af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:42.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.606+0000 7f87f7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8800074840 0x7f88000af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48868/0 (socket says 172.21.15.86:48868) 2024-04-25T14:08:42.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.606+0000 7f87f7e00640 1 -- 172.21.15.86:0/3526845865 learned_addr learned my addr 172.21.15.86:0/3526845865 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:42.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.606+0000 7f8805600640 1 --2- 172.21.15.86:0/3526845865 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88000af770 0x7f88000afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:42.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.606+0000 7f87f7e00640 1 -- 172.21.15.86:0/3526845865 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88000af770 msgr2=0x7f88000afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:42.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.606+0000 7f87f7e00640 1 --2- 172.21.15.86:0/3526845865 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88000af770 0x7f88000afb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:42.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.606+0000 7f87f7e00640 1 -- 172.21.15.86:0/3526845865 >> v1:172.21.15.155:6789/0 conn(0x7f880015d790 legacy=0x7f8800074130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:42.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.606+0000 7f87f7e00640 1 -- 172.21.15.86:0/3526845865 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8800073490 con 0x7f8800074840 2024-04-25T14:08:42.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.606+0000 7f87ffe00640 1 -- 172.21.15.86:0/3526845865 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2981080053 0 0) 0x7f8800072bf0 con 0x7f880015d790 2024-04-25T14:08:42.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.606+0000 7f8805600640 1 --2- 172.21.15.86:0/3526845865 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88000af770 0x7f88000afb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:08:42.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.610+0000 7f87f7e00640 1 --2- 172.21.15.86:0/3526845865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8800074840 0x7f88000af230 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009870 tx=0x7f87e801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11431d9a55c0be9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:42.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.610+0000 7f87ffe00640 1 -- 172.21.15.86:0/3526845865 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e80aa5f0 con 0x7f8800074840 2024-04-25T14:08:42.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.610+0000 7f87ffe00640 1 -- 172.21.15.86:0/3526845865 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87e80af070 con 0x7f8800074840 2024-04-25T14:08:42.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.610+0000 7f87ffe00640 1 -- 172.21.15.86:0/3526845865 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e80aa9b0 con 0x7f8800074840 2024-04-25T14:08:42.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.610+0000 7f8806000640 1 -- 172.21.15.86:0/3526845865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8800074840 msgr2=0x7f88000af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:42.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.610+0000 7f8806000640 1 --2- 172.21.15.86:0/3526845865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8800074840 0x7f88000af230 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009870 tx=0x7f87e801c530 comp rx=0 tx=0).stop 2024-04-25T14:08:42.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.610+0000 7f8806000640 1 -- 172.21.15.86:0/3526845865 shutdown_connections 2024-04-25T14:08:42.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.610+0000 7f8806000640 1 --2- 172.21.15.86:0/3526845865 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88000af770 0x7f88000afb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:42.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.610+0000 7f8806000640 1 --2- 172.21.15.86:0/3526845865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8800074840 0x7f88000af230 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:42.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.610+0000 7f8806000640 1 -- 172.21.15.86:0/3526845865 >> 172.21.15.86:0/3526845865 conn(0x7f88000b4740 msgr2=0x7f88000b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:42.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.610+0000 7f8806000640 1 -- 172.21.15.86:0/3526845865 shutdown_connections 2024-04-25T14:08:42.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.610+0000 7f8806000640 1 -- 172.21.15.86:0/3526845865 wait complete. 2024-04-25T14:08:42.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.614+0000 7f8806000640 1 Processor -- start 2024-04-25T14:08:42.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.614+0000 7f8806000640 1 -- start start 2024-04-25T14:08:42.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.614+0000 7f8806000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88000af770 0x7f880015bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:42.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.614+0000 7f8806000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f880015d790 0x7f8800166980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:42.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.614+0000 7f8806000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8800153160 0x7f8800153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:42.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.614+0000 7f8806000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f880015dba0 con 0x7f8800153160 2024-04-25T14:08:42.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.614+0000 7f8806000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8800073c90 con 0x7f88000af770 2024-04-25T14:08:42.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.614+0000 7f8806000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f88000cfee0 con 0x7f880015d790 2024-04-25T14:08:42.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.614+0000 7f8805600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8800153160 0x7f8800153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:42.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.614+0000 7f8805600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8800153160 0x7f8800153570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48880/0 (socket says 172.21.15.86:48880) 2024-04-25T14:08:42.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.614+0000 7f8805600640 1 -- 172.21.15.86:0/1812233794 learned_addr learned my addr 172.21.15.86:0/1812233794 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:42.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.614+0000 7f87f7e00640 1 --2- 172.21.15.86:0/1812233794 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f880015d790 0x7f8800166980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:42.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.614+0000 7f8804c00640 1 --2- 172.21.15.86:0/1812233794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88000af770 0x7f880015bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:42.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.614+0000 7f8805600640 1 -- 172.21.15.86:0/1812233794 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f880015d790 msgr2=0x7f8800166980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:42.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.614+0000 7f8805600640 1 --2- 172.21.15.86:0/1812233794 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f880015d790 0x7f8800166980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:42.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.614+0000 7f8805600640 1 -- 172.21.15.86:0/1812233794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88000af770 msgr2=0x7f880015bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:42.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.614+0000 7f8805600640 1 --2- 172.21.15.86:0/1812233794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88000af770 0x7f880015bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:42.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.614+0000 7f8805600640 1 -- 172.21.15.86:0/1812233794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8800153ab0 con 0x7f8800153160 2024-04-25T14:08:42.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.614+0000 7f8805600640 1 --2- 172.21.15.86:0/1812233794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8800153160 0x7f8800153570 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f87f800efc0 tx=0x7f87f800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:42.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.614+0000 7f87fd600640 1 -- 172.21.15.86:0/1812233794 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8018030 con 0x7f8800153160 2024-04-25T14:08:42.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.614+0000 7f87fd600640 1 -- 172.21.15.86:0/1812233794 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f800cdf0 con 0x7f8800153160 2024-04-25T14:08:42.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.614+0000 7f8806000640 1 -- 172.21.15.86:0/1812233794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88001573b0 con 0x7f8800153160 2024-04-25T14:08:42.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.614+0000 7f87fd600640 1 -- 172.21.15.86:0/1812233794 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8003bc0 con 0x7f8800153160 2024-04-25T14:08:42.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.614+0000 7f8806000640 1 -- 172.21.15.86:0/1812233794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88000d0130 con 0x7f8800153160 2024-04-25T14:08:42.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.618+0000 7f8806000640 1 -- 172.21.15.86:0/1812233794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8800076b80 con 0x7f8800153160 2024-04-25T14:08:42.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.618+0000 7f87fd600640 1 -- 172.21.15.86:0/1812233794 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7f87f801b070 con 0x7f8800153160 2024-04-25T14:08:42.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.618+0000 7f87fd600640 1 --2- 172.21.15.86:0/1812233794 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f87dc06f520 0x7f87dc071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:42.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.618+0000 7f87fd600640 1 -- 172.21.15.86:0/1812233794 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5249+0+0 (secure 0 0 0) 0x7f87f808fd70 con 0x7f8800153160 2024-04-25T14:08:42.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.618+0000 7f8804c00640 1 --2- 172.21.15.86:0/1812233794 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f87dc06f520 0x7f87dc071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:42.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.622+0000 7f8804c00640 1 --2- 172.21.15.86:0/1812233794 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f87dc06f520 0x7f87dc071a30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f87f0000f70 tx=0x7f87f0009040 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:42.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.622+0000 7f87fd600640 1 -- 172.21.15.86:0/1812233794 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f87f8022070 con 0x7f8800153160 2024-04-25T14:08:42.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.790+0000 7f8806000640 1 -- 172.21.15.86:0/1812233794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f88000b2ab0 con 0x7f8800153160 2024-04-25T14:08:42.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.790+0000 7f87fd600640 1 -- 172.21.15.86:0/1812233794 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f8800153ab0 con 0x7f8800153160 2024-04-25T14:08:42.794 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:08:42.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.794+0000 7f8806000640 1 -- 172.21.15.86:0/1812233794 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f87dc06f520 msgr2=0x7f87dc071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:42.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.794+0000 7f8806000640 1 --2- 172.21.15.86:0/1812233794 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f87dc06f520 0x7f87dc071a30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f87f0000f70 tx=0x7f87f0009040 comp rx=0 tx=0).stop 2024-04-25T14:08:42.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.794+0000 7f8806000640 1 -- 172.21.15.86:0/1812233794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8800153160 msgr2=0x7f8800153570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:42.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.794+0000 7f8806000640 1 --2- 172.21.15.86:0/1812233794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8800153160 0x7f8800153570 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f87f800efc0 tx=0x7f87f800c490 comp rx=0 tx=0).stop 2024-04-25T14:08:42.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.794+0000 7f8806000640 1 -- 172.21.15.86:0/1812233794 shutdown_connections 2024-04-25T14:08:42.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.794+0000 7f8806000640 1 --2- 172.21.15.86:0/1812233794 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f87dc06f520 0x7f87dc071a30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:42.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.794+0000 7f8806000640 1 --2- 172.21.15.86:0/1812233794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8800153160 0x7f8800153570 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:42.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.794+0000 7f8806000640 1 --2- 172.21.15.86:0/1812233794 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f880015d790 0x7f8800166980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:42.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.794+0000 7f8806000640 1 --2- 172.21.15.86:0/1812233794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88000af770 0x7f880015bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:42.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.794+0000 7f8806000640 1 -- 172.21.15.86:0/1812233794 >> 172.21.15.86:0/1812233794 conn(0x7f88000b4740 msgr2=0x7f880015e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:42.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.794+0000 7f8806000640 1 -- 172.21.15.86:0/1812233794 shutdown_connections 2024-04-25T14:08:42.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.794+0000 7f8806000640 1 -- 172.21.15.86:0/1812233794 wait complete. 2024-04-25T14:08:42.816 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T14:08:42.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.962+0000 7fee15200640 1 Processor -- start 2024-04-25T14:08:42.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.962+0000 7fee15200640 1 -- start start 2024-04-25T14:08:42.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.962+0000 7fee15200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee10070330 0x7fee100d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:42.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.962+0000 7fee15200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fee100ce790 0x7fee1006f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:42.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.962+0000 7fee15200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee100d2030 con 0x7fee100d1c20 2024-04-25T14:08:42.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.962+0000 7fee15200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fee1015e5c0 con 0x7fee100ce790 2024-04-25T14:08:42.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.962+0000 7fee15200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fee1015e8a0 con 0x7fee10070330 2024-04-25T14:08:42.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.962+0000 7fee0d400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fee100d1c20 0x7fee1006ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60080/0 (socket says 172.21.15.86:60080) 2024-04-25T14:08:42.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.962+0000 7fee0d400640 1 -- 172.21.15.86:0/534200174 learned_addr learned my addr 172.21.15.86:0/534200174 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:42.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.962+0000 7fee0e800640 1 --2- 172.21.15.86:0/534200174 >> v2:172.21.15.155:3300/0 conn(0x7fee100ce790 0x7fee1006f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:42.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.962+0000 7fee0de00640 1 --2- 172.21.15.86:0/534200174 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee10070330 0x7fee100d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:42.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.962+0000 7fee0ca00640 1 -- 172.21.15.86:0/534200174 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 168300940 0 0) 0x7fedfc001120 con 0x7fee100d1c20 2024-04-25T14:08:42.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.962+0000 7fee0ca00640 1 -- 172.21.15.86:0/534200174 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedf4003530 con 0x7fee100d1c20 2024-04-25T14:08:42.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.962+0000 7fee0e800640 1 -- 172.21.15.86:0/534200174 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee10070330 msgr2=0x7fee100d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:42.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.962+0000 7fee0e800640 1 --2- 172.21.15.86:0/534200174 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee10070330 0x7fee100d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:42.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.962+0000 7fee0e800640 1 -- 172.21.15.86:0/534200174 >> v1:172.21.15.86:6789/0 conn(0x7fee100d1c20 legacy=0x7fee1006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:42.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.962+0000 7fee0ca00640 1 -- 172.21.15.86:0/534200174 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2124825447 0 0) 0x7fedf4003530 con 0x7fee100d1c20 2024-04-25T14:08:42.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.962+0000 7fee0e800640 1 -- 172.21.15.86:0/534200174 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee1015eb80 con 0x7fee100ce790 2024-04-25T14:08:42.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.966+0000 7fee0e800640 1 --2- 172.21.15.86:0/534200174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee100ce790 0x7fee1006f440 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fee080097f0 tx=0x7fee0801c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe8ef4323c4c4559 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:42.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.966+0000 7fee0ca00640 1 -- 172.21.15.86:0/534200174 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee080b0070 con 0x7fee100ce790 2024-04-25T14:08:42.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.966+0000 7fee0ca00640 1 -- 172.21.15.86:0/534200174 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee080ab400 con 0x7fee100ce790 2024-04-25T14:08:42.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.966+0000 7fee0ca00640 1 -- 172.21.15.86:0/534200174 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc001850 con 0x7fee100ce790 2024-04-25T14:08:42.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.966+0000 7fee15200640 1 -- 172.21.15.86:0/534200174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee100ce790 msgr2=0x7fee1006f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:42.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.966+0000 7fee15200640 1 --2- 172.21.15.86:0/534200174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee100ce790 0x7fee1006f440 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fee080097f0 tx=0x7fee0801c330 comp rx=0 tx=0).stop 2024-04-25T14:08:42.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.966+0000 7fee15200640 1 -- 172.21.15.86:0/534200174 shutdown_connections 2024-04-25T14:08:42.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.966+0000 7fee15200640 1 --2- 172.21.15.86:0/534200174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee100ce790 0x7fee1006f440 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:42.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.966+0000 7fee15200640 1 --2- 172.21.15.86:0/534200174 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee10070330 0x7fee100d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:42.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.966+0000 7fee15200640 1 -- 172.21.15.86:0/534200174 >> 172.21.15.86:0/534200174 conn(0x7fee100b4650 msgr2=0x7fee100b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:42.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.966+0000 7fee15200640 1 -- 172.21.15.86:0/534200174 shutdown_connections 2024-04-25T14:08:42.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.966+0000 7fee15200640 1 -- 172.21.15.86:0/534200174 wait complete. 2024-04-25T14:08:42.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.970+0000 7fee15200640 1 Processor -- start 2024-04-25T14:08:42.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.970+0000 7fee15200640 1 -- start start 2024-04-25T14:08:42.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.970+0000 7fee15200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee10070330 0x7fee1015a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:42.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.970+0000 7fee15200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee100ce790 0x7fee100731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:42.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.970+0000 7fee15200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee100d1c20 0x7fee10073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:42.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.970+0000 7fee15200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee100708c0 con 0x7fee10070330 2024-04-25T14:08:42.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.970+0000 7fee15200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fee10070740 con 0x7fee100ce790 2024-04-25T14:08:42.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.970+0000 7fee15200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fee1016afc0 con 0x7fee100d1c20 2024-04-25T14:08:42.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.970+0000 7fee0de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee10070330 0x7fee1015a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:42.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.970+0000 7fee0de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee10070330 0x7fee1015a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48886/0 (socket says 172.21.15.86:48886) 2024-04-25T14:08:42.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.970+0000 7fee0de00640 1 -- 172.21.15.86:0/2305221652 learned_addr learned my addr 172.21.15.86:0/2305221652 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:42.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.970+0000 7fee0e800640 1 --2- 172.21.15.86:0/2305221652 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee100d1c20 0x7fee10073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:42.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.970+0000 7fee0d400640 1 --2- 172.21.15.86:0/2305221652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee100ce790 0x7fee100731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:42.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.970+0000 7fee0de00640 1 -- 172.21.15.86:0/2305221652 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee100d1c20 msgr2=0x7fee10073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:42.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.970+0000 7fee0de00640 1 --2- 172.21.15.86:0/2305221652 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee100d1c20 0x7fee10073720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:42.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.970+0000 7fee0de00640 1 -- 172.21.15.86:0/2305221652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee100ce790 msgr2=0x7fee100731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:42.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.970+0000 7fee0de00640 1 --2- 172.21.15.86:0/2305221652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee100ce790 0x7fee100731e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:42.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.970+0000 7fee0de00640 1 -- 172.21.15.86:0/2305221652 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee1015c690 con 0x7fee10070330 2024-04-25T14:08:42.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.970+0000 7fee0d400640 1 --2- 172.21.15.86:0/2305221652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee100ce790 0x7fee100731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:42.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.970+0000 7fee0de00640 1 --2- 172.21.15.86:0/2305221652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee10070330 0x7fee1015a860 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fee00009cd0 tx=0x7fee0000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:42.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.970+0000 7fee06000640 1 -- 172.21.15.86:0/2305221652 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee00018020 con 0x7fee10070330 2024-04-25T14:08:42.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.974+0000 7fee06000640 1 -- 172.21.15.86:0/2305221652 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee00014480 con 0x7fee10070330 2024-04-25T14:08:42.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.974+0000 7fee06000640 1 -- 172.21.15.86:0/2305221652 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee00014850 con 0x7fee10070330 2024-04-25T14:08:42.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.974+0000 7fee15200640 1 -- 172.21.15.86:0/2305221652 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee1015c970 con 0x7fee10070330 2024-04-25T14:08:42.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.974+0000 7fee15200640 1 -- 172.21.15.86:0/2305221652 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee1015cf30 con 0x7fee10070330 2024-04-25T14:08:42.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.974+0000 7fee15200640 1 -- 172.21.15.86:0/2305221652 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee10069d00 con 0x7fee10070330 2024-04-25T14:08:42.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.974+0000 7fee06000640 1 -- 172.21.15.86:0/2305221652 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7fee0001b020 con 0x7fee10070330 2024-04-25T14:08:42.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.974+0000 7fee06000640 1 --2- 172.21.15.86:0/2305221652 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fedf006f550 0x7fedf0071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:42.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.974+0000 7fee06000640 1 -- 172.21.15.86:0/2305221652 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5249+0+0 (secure 0 0 0) 0x7fee000900e0 con 0x7fee10070330 2024-04-25T14:08:42.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.974+0000 7fee0d400640 1 --2- 172.21.15.86:0/2305221652 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fedf006f550 0x7fedf0071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:42.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.978+0000 7fee0d400640 1 --2- 172.21.15.86:0/2305221652 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fedf006f550 0x7fedf0071a60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fedfc000ff0 tx=0x7fedfc009040 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:42.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:42.982+0000 7fee06000640 1 -- 172.21.15.86:0/2305221652 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fee00022030 con 0x7fee10070330 2024-04-25T14:08:43.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.138+0000 7fee15200640 1 -- 172.21.15.86:0/2305221652 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fee100aea30 con 0x7fee10070330 2024-04-25T14:08:43.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.142+0000 7fee06000640 1 -- 172.21.15.86:0/2305221652 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) ==== 84+0+65 (secure 0 0 0) 0x7fee00010040 con 0x7fee10070330 2024-04-25T14:08:43.142 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:08:43.142 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-04-25T14:08:43.142 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:08:43.142 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-04-25T14:08:43.142 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-04-25T14:08:43.143 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:08:43.143 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-04-25T14:08:43.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.142+0000 7fee15200640 1 -- 172.21.15.86:0/2305221652 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fedf006f550 msgr2=0x7fedf0071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:43.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.142+0000 7fee15200640 1 --2- 172.21.15.86:0/2305221652 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fedf006f550 0x7fedf0071a60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fedfc000ff0 tx=0x7fedfc009040 comp rx=0 tx=0).stop 2024-04-25T14:08:43.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.142+0000 7fee15200640 1 -- 172.21.15.86:0/2305221652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee10070330 msgr2=0x7fee1015a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:43.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.142+0000 7fee15200640 1 --2- 172.21.15.86:0/2305221652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee10070330 0x7fee1015a860 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fee00009cd0 tx=0x7fee0000c490 comp rx=0 tx=0).stop 2024-04-25T14:08:43.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.142+0000 7fee15200640 1 -- 172.21.15.86:0/2305221652 shutdown_connections 2024-04-25T14:08:43.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.142+0000 7fee15200640 1 --2- 172.21.15.86:0/2305221652 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fedf006f550 0x7fedf0071a60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.142+0000 7fee15200640 1 --2- 172.21.15.86:0/2305221652 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee100d1c20 0x7fee10073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.142+0000 7fee15200640 1 --2- 172.21.15.86:0/2305221652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee100ce790 0x7fee100731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.142+0000 7fee15200640 1 --2- 172.21.15.86:0/2305221652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee10070330 0x7fee1015a860 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.142+0000 7fee15200640 1 -- 172.21.15.86:0/2305221652 >> 172.21.15.86:0/2305221652 conn(0x7fee100b4650 msgr2=0x7fee100c75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:43.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.142+0000 7fee15200640 1 -- 172.21.15.86:0/2305221652 shutdown_connections 2024-04-25T14:08:43.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.142+0000 7fee15200640 1 -- 172.21.15.86:0/2305221652 wait complete. 2024-04-25T14:08:43.164 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T14:08:43.164 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T14:08:43.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.314+0000 7f18b8000640 1 Processor -- start 2024-04-25T14:08:43.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.314+0000 7f18b8000640 1 -- start start 2024-04-25T14:08:43.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.314+0000 7f18b8000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f18b00d1b50 0x7f18b015dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.314+0000 7f18b8000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f18b00b1dd0 0x7f18b00b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.314+0000 7f18b8000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18b00b19d0 con 0x7f18b00d2230 2024-04-25T14:08:43.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.314+0000 7f18b8000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f18b006edc0 con 0x7f18b00d1b50 2024-04-25T14:08:43.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.314+0000 7f18b8000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f18b006e770 con 0x7f18b00b1dd0 2024-04-25T14:08:43.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.318+0000 7f18b5600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f18b00d1b50 0x7f18b015dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:43.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.318+0000 7f18b5600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f18b00d1b50 0x7f18b015dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48898/0 (socket says 172.21.15.86:48898) 2024-04-25T14:08:43.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.318+0000 7f18b5600640 1 -- 172.21.15.86:0/3516914101 learned_addr learned my addr 172.21.15.86:0/3516914101 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:43.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.318+0000 7f18b6000640 1 --2- 172.21.15.86:0/3516914101 >> v2:172.21.15.155:3300/0 conn(0x7f18b00b1dd0 0x7f18b00b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:43.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.318+0000 7f18b5600640 1 -- 172.21.15.86:0/3516914101 >> v2:172.21.15.155:3300/0 conn(0x7f18b00b1dd0 msgr2=0x7f18b00b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:43.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.318+0000 7f18b5600640 1 --2- 172.21.15.86:0/3516914101 >> v2:172.21.15.155:3300/0 conn(0x7f18b00b1dd0 0x7f18b00b21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.318+0000 7f18b5600640 1 -- 172.21.15.86:0/3516914101 >> v1:172.21.15.86:6789/0 conn(0x7f18b00d2230 legacy=0x7f18b00d2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:43.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.318+0000 7f18b5600640 1 -- 172.21.15.86:0/3516914101 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b006ea50 con 0x7f18b00d1b50 2024-04-25T14:08:43.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.318+0000 7f18afe00640 1 -- 172.21.15.86:0/3516914101 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3129510483 0 0) 0x7f18b00b19d0 con 0x7f18b00d2230 2024-04-25T14:08:43.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.318+0000 7f18b5600640 1 --2- 172.21.15.86:0/3516914101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18b00d1b50 0x7f18b015dc80 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f18b00747e0 tx=0x7f18a401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f465ba10b3b8983 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:43.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.318+0000 7f18afe00640 1 -- 172.21.15.86:0/3516914101 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a40a5730 con 0x7f18b00d1b50 2024-04-25T14:08:43.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.318+0000 7f18afe00640 1 -- 172.21.15.86:0/3516914101 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a40a58d0 con 0x7f18b00d1b50 2024-04-25T14:08:43.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.318+0000 7f18afe00640 1 -- 172.21.15.86:0/3516914101 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a40a5ca0 con 0x7f18b00d1b50 2024-04-25T14:08:43.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.322+0000 7f18b8000640 1 -- 172.21.15.86:0/3516914101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18b00d1b50 msgr2=0x7f18b015dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:43.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.322+0000 7f18b8000640 1 --2- 172.21.15.86:0/3516914101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18b00d1b50 0x7f18b015dc80 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f18b00747e0 tx=0x7f18a401c960 comp rx=0 tx=0).stop 2024-04-25T14:08:43.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.322+0000 7f18b8000640 1 -- 172.21.15.86:0/3516914101 shutdown_connections 2024-04-25T14:08:43.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.322+0000 7f18b8000640 1 --2- 172.21.15.86:0/3516914101 >> v2:172.21.15.155:3300/0 conn(0x7f18b00b1dd0 0x7f18b00b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.322+0000 7f18b8000640 1 --2- 172.21.15.86:0/3516914101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18b00d1b50 0x7f18b015dc80 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.322+0000 7f18b8000640 1 -- 172.21.15.86:0/3516914101 >> 172.21.15.86:0/3516914101 conn(0x7f18b0073c90 msgr2=0x7f18b00b77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:43.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.322+0000 7f18b8000640 1 -- 172.21.15.86:0/3516914101 shutdown_connections 2024-04-25T14:08:43.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.322+0000 7f18b8000640 1 -- 172.21.15.86:0/3516914101 wait complete. 2024-04-25T14:08:43.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.322+0000 7f18b8000640 1 Processor -- start 2024-04-25T14:08:43.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.322+0000 7f18b8000640 1 -- start start 2024-04-25T14:08:43.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.326+0000 7f18b8000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18b00b1dd0 0x7f18b0151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.326+0000 7f18b8000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18b00d1b50 0x7f18b015e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.326+0000 7f18b8000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18b00d2230 0x7f18b00b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.326+0000 7f18b8000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18b016ad60 con 0x7f18b00d1b50 2024-04-25T14:08:43.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.326+0000 7f18b8000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f18b016abe0 con 0x7f18b00b1dd0 2024-04-25T14:08:43.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.326+0000 7f18b8000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f18b016aee0 con 0x7f18b00d2230 2024-04-25T14:08:43.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.326+0000 7f18b4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18b00d1b50 0x7f18b015e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:43.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.326+0000 7f18b4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18b00d1b50 0x7f18b015e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48912/0 (socket says 172.21.15.86:48912) 2024-04-25T14:08:43.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.326+0000 7f18b4c00640 1 -- 172.21.15.86:0/2424773990 learned_addr learned my addr 172.21.15.86:0/2424773990 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:43.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.326+0000 7f18b4c00640 1 -- 172.21.15.86:0/2424773990 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18b00d2230 msgr2=0x7f18b00b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:43.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.326+0000 7f18b4c00640 1 --2- 172.21.15.86:0/2424773990 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18b00d2230 0x7f18b00b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.326+0000 7f18b4c00640 1 -- 172.21.15.86:0/2424773990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18b00b1dd0 msgr2=0x7f18b0151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:43.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.326+0000 7f18b4c00640 1 --2- 172.21.15.86:0/2424773990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18b00b1dd0 0x7f18b0151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.326+0000 7f18b4c00640 1 -- 172.21.15.86:0/2424773990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b015e9d0 con 0x7f18b00d1b50 2024-04-25T14:08:43.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.326+0000 7f18b4c00640 1 --2- 172.21.15.86:0/2424773990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18b00d1b50 0x7f18b015e490 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f189c001ec0 tx=0x7f189c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:43.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.326+0000 7f18ad600640 1 -- 172.21.15.86:0/2424773990 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c017070 con 0x7f18b00d1b50 2024-04-25T14:08:43.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.326+0000 7f18ad600640 1 -- 172.21.15.86:0/2424773990 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f189c003600 con 0x7f18b00d1b50 2024-04-25T14:08:43.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.326+0000 7f18b8000640 1 -- 172.21.15.86:0/2424773990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b01563d0 con 0x7f18b00d1b50 2024-04-25T14:08:43.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.326+0000 7f18ad600640 1 -- 172.21.15.86:0/2424773990 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c0039f0 con 0x7f18b00d1b50 2024-04-25T14:08:43.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.326+0000 7f18b8000640 1 -- 172.21.15.86:0/2424773990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b0156b20 con 0x7f18b00d1b50 2024-04-25T14:08:43.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.326+0000 7f18b8000640 1 -- 172.21.15.86:0/2424773990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18b0050750 con 0x7f18b00d1b50 2024-04-25T14:08:43.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.326+0000 7f18ad600640 1 -- 172.21.15.86:0/2424773990 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7f189c01a070 con 0x7f18b00d1b50 2024-04-25T14:08:43.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.330+0000 7f18ad600640 1 --2- 172.21.15.86:0/2424773990 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f189006f520 0x7f1890071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.330+0000 7f18ad600640 1 -- 172.21.15.86:0/2424773990 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5249+0+0 (secure 0 0 0) 0x7f189c08e690 con 0x7f18b00d1b50 2024-04-25T14:08:43.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.330+0000 7f18b5600640 1 --2- 172.21.15.86:0/2424773990 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f189006f520 0x7f1890071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:43.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.330+0000 7f18b5600640 1 --2- 172.21.15.86:0/2424773990 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f189006f520 0x7f1890071a30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f18a40ae000 tx=0x7f18a4004ac0 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:43.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.334+0000 7f18ad600640 1 -- 172.21.15.86:0/2424773990 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f189c05b620 con 0x7f18b00d1b50 2024-04-25T14:08:43.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.490+0000 7f18b8000640 1 -- 172.21.15.86:0/2424773990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f18b01565a0 con 0x7f18b00d1b50 2024-04-25T14:08:43.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.682+0000 7f18ad600640 1 -- 172.21.15.86:0/2424773990 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) ==== 138+0+0 (secure 0 0 0) 0x7f189c05af20 con 0x7f18b00d1b50 2024-04-25T14:08:43.683 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-04-25T14:08:43.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.682+0000 7f18b8000640 1 -- 172.21.15.86:0/2424773990 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f189006f520 msgr2=0x7f1890071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:43.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.682+0000 7f18b8000640 1 --2- 172.21.15.86:0/2424773990 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f189006f520 0x7f1890071a30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f18a40ae000 tx=0x7f18a4004ac0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.682+0000 7f18b8000640 1 -- 172.21.15.86:0/2424773990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18b00d1b50 msgr2=0x7f18b015e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:43.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.682+0000 7f18b8000640 1 --2- 172.21.15.86:0/2424773990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18b00d1b50 0x7f18b015e490 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f189c001ec0 tx=0x7f189c00c8e0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.682+0000 7f18b8000640 1 -- 172.21.15.86:0/2424773990 shutdown_connections 2024-04-25T14:08:43.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.682+0000 7f18b8000640 1 --2- 172.21.15.86:0/2424773990 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f189006f520 0x7f1890071a30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.682+0000 7f18b8000640 1 --2- 172.21.15.86:0/2424773990 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18b00d2230 0x7f18b00b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.682+0000 7f18b8000640 1 --2- 172.21.15.86:0/2424773990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18b00d1b50 0x7f18b015e490 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.682+0000 7f18b8000640 1 --2- 172.21.15.86:0/2424773990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18b00b1dd0 0x7f18b0151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.682+0000 7f18b8000640 1 -- 172.21.15.86:0/2424773990 >> 172.21.15.86:0/2424773990 conn(0x7f18b0073c90 msgr2=0x7f18b00b3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:43.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.682+0000 7f18b8000640 1 -- 172.21.15.86:0/2424773990 shutdown_connections 2024-04-25T14:08:43.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.682+0000 7f18b8000640 1 -- 172.21.15.86:0/2424773990 wait complete. 2024-04-25T14:08:43.710 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T14:08:43.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.862+0000 7f5cab800640 1 Processor -- start 2024-04-25T14:08:43.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.862+0000 7f5cab800640 1 -- start start 2024-04-25T14:08:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.862+0000 7f5cab800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5ca4070330 0x7f5ca40d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.862+0000 7f5cab800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca40d1c20 0x7f5ca406ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.862+0000 7f5cab800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ca40d2030 con 0x7f5ca406f270 2024-04-25T14:08:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.862+0000 7f5cab800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5ca415e5c0 con 0x7f5ca4070330 2024-04-25T14:08:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.862+0000 7f5cab800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5ca415e8a0 con 0x7f5ca40d1c20 2024-04-25T14:08:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.862+0000 7f5ca8e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5ca4070330 0x7f5ca40d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.862+0000 7f5ca9800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5ca406f270 0x7f5ca406f680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60094/0 (socket says 172.21.15.86:60094) 2024-04-25T14:08:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.862+0000 7f5ca8e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5ca4070330 0x7f5ca40d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48926/0 (socket says 172.21.15.86:48926) 2024-04-25T14:08:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.862+0000 7f5ca9800640 1 -- 172.21.15.86:0/3580527480 learned_addr learned my addr 172.21.15.86:0/3580527480 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:43.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.862+0000 7f5ca3e00640 1 --2- 172.21.15.86:0/3580527480 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca40d1c20 0x7f5ca406ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:43.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.862+0000 7f5ca8e00640 1 -- 172.21.15.86:0/3580527480 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca40d1c20 msgr2=0x7f5ca406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:43.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.866+0000 7f5ca8e00640 1 --2- 172.21.15.86:0/3580527480 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca40d1c20 0x7f5ca406ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.866+0000 7f5ca8e00640 1 -- 172.21.15.86:0/3580527480 >> v1:172.21.15.86:6789/0 conn(0x7f5ca406f270 legacy=0x7f5ca406f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:43.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.866+0000 7f5ca3400640 1 -- 172.21.15.86:0/3580527480 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 566789043 0 0) 0x7f5c9c001120 con 0x7f5ca406f270 2024-04-25T14:08:43.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.866+0000 7f5ca8e00640 1 -- 172.21.15.86:0/3580527480 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca415eb80 con 0x7f5ca4070330 2024-04-25T14:08:43.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.866+0000 7f5ca3e00640 1 --2- 172.21.15.86:0/3580527480 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca40d1c20 0x7f5ca406ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:43.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.866+0000 7f5ca8e00640 1 --2- 172.21.15.86:0/3580527480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ca4070330 0x7f5ca40d16e0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009d90 tx=0x7f5c9401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3609b1cf14cb0d90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:43.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.866+0000 7f5ca3400640 1 -- 172.21.15.86:0/3580527480 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c940b2070 con 0x7f5ca4070330 2024-04-25T14:08:43.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.866+0000 7f5ca3400640 1 -- 172.21.15.86:0/3580527480 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c940a57a0 con 0x7f5ca4070330 2024-04-25T14:08:43.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.866+0000 7f5ca3400640 1 -- 172.21.15.86:0/3580527480 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c940a5b70 con 0x7f5ca4070330 2024-04-25T14:08:43.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.866+0000 7f5cab800640 1 -- 172.21.15.86:0/3580527480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ca4070330 msgr2=0x7f5ca40d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:43.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.866+0000 7f5cab800640 1 --2- 172.21.15.86:0/3580527480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ca4070330 0x7f5ca40d16e0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009d90 tx=0x7f5c9401c8e0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.866+0000 7f5cab800640 1 -- 172.21.15.86:0/3580527480 shutdown_connections 2024-04-25T14:08:43.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.866+0000 7f5cab800640 1 --2- 172.21.15.86:0/3580527480 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca40d1c20 0x7f5ca406ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.866+0000 7f5cab800640 1 --2- 172.21.15.86:0/3580527480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ca4070330 0x7f5ca40d16e0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.870+0000 7f5cab800640 1 -- 172.21.15.86:0/3580527480 >> 172.21.15.86:0/3580527480 conn(0x7f5ca40b4650 msgr2=0x7f5ca40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:43.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.870+0000 7f5cab800640 1 -- 172.21.15.86:0/3580527480 shutdown_connections 2024-04-25T14:08:43.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.870+0000 7f5cab800640 1 -- 172.21.15.86:0/3580527480 wait complete. 2024-04-25T14:08:43.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.870+0000 7f5cab800640 1 Processor -- start 2024-04-25T14:08:43.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.870+0000 7f5cab800640 1 -- start start 2024-04-25T14:08:43.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.870+0000 7f5cab800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca406f270 0x7f5ca409a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.870+0000 7f5cab800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca4070330 0x7f5ca40af480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.870+0000 7f5cab800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ca40d1c20 0x7f5ca4099710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.870+0000 7f5cab800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca40708c0 con 0x7f5ca40d1c20 2024-04-25T14:08:43.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.870+0000 7f5cab800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca4070740 con 0x7f5ca406f270 2024-04-25T14:08:43.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.870+0000 7f5cab800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5ca40ceeb0 con 0x7f5ca4070330 2024-04-25T14:08:43.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.874+0000 7f5ca9800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ca40d1c20 0x7f5ca4099710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:43.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.874+0000 7f5ca9800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ca40d1c20 0x7f5ca4099710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48936/0 (socket says 172.21.15.86:48936) 2024-04-25T14:08:43.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.874+0000 7f5ca9800640 1 -- 172.21.15.86:0/1904290628 learned_addr learned my addr 172.21.15.86:0/1904290628 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:43.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.874+0000 7f5ca3e00640 1 --2- 172.21.15.86:0/1904290628 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca4070330 0x7f5ca40af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:43.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.874+0000 7f5ca8e00640 1 --2- 172.21.15.86:0/1904290628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca406f270 0x7f5ca409a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:43.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.874+0000 7f5ca9800640 1 -- 172.21.15.86:0/1904290628 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca4070330 msgr2=0x7f5ca40af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:43.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.874+0000 7f5ca9800640 1 --2- 172.21.15.86:0/1904290628 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca4070330 0x7f5ca40af480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.874+0000 7f5ca9800640 1 -- 172.21.15.86:0/1904290628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca406f270 msgr2=0x7f5ca409a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:43.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.874+0000 7f5ca9800640 1 --2- 172.21.15.86:0/1904290628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca406f270 0x7f5ca409a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.874+0000 7f5ca9800640 1 -- 172.21.15.86:0/1904290628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca4098170 con 0x7f5ca40d1c20 2024-04-25T14:08:43.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.874+0000 7f5ca9800640 1 --2- 172.21.15.86:0/1904290628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ca40d1c20 0x7f5ca4099710 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00c410 tx=0x7f5c9c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:43.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.874+0000 7f5ca0c00640 1 -- 172.21.15.86:0/1904290628 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c0074e0 con 0x7f5ca40d1c20 2024-04-25T14:08:43.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.874+0000 7f5ca0c00640 1 -- 172.21.15.86:0/1904290628 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c9c0047f0 con 0x7f5ca40d1c20 2024-04-25T14:08:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.874+0000 7f5ca0c00640 1 -- 172.21.15.86:0/1904290628 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c004c20 con 0x7f5ca40d1c20 2024-04-25T14:08:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.874+0000 7f5cab800640 1 -- 172.21.15.86:0/1904290628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca4098670 con 0x7f5ca40d1c20 2024-04-25T14:08:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.874+0000 7f5cab800640 1 -- 172.21.15.86:0/1904290628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca4098c30 con 0x7f5ca40d1c20 2024-04-25T14:08:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.874+0000 7f5cab800640 1 -- 172.21.15.86:0/1904290628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ca40983e0 con 0x7f5ca40d1c20 2024-04-25T14:08:43.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.878+0000 7f5ca0c00640 1 -- 172.21.15.86:0/1904290628 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7f5c9c011070 con 0x7f5ca40d1c20 2024-04-25T14:08:43.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.878+0000 7f5ca0c00640 1 --2- 172.21.15.86:0/1904290628 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f5c8406f550 0x7f5c84071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.878+0000 7f5ca0c00640 1 -- 172.21.15.86:0/1904290628 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5619+0+0 (secure 0 0 0) 0x7f5c9c08f350 con 0x7f5ca40d1c20 2024-04-25T14:08:43.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.878+0000 7f5ca8e00640 1 --2- 172.21.15.86:0/1904290628 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f5c8406f550 0x7f5c84071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:43.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.878+0000 7f5ca8e00640 1 --2- 172.21.15.86:0/1904290628 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f5c8406f550 0x7f5c84071a60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5c940af000 tx=0x7f5c940a7070 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:43.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:43.882+0000 7f5ca0c00640 1 -- 172.21.15.86:0/1904290628 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5ca40d2030 con 0x7f5ca40d1c20 2024-04-25T14:08:44.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.038+0000 7f5cab800640 1 -- 172.21.15.86:0/1904290628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5ca40c38d0 con 0x7f5ca40d1c20 2024-04-25T14:08:44.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.686+0000 7f5ca0c00640 1 -- 172.21.15.86:0/1904290628 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) ==== 170+0+0 (secure 0 0 0) 0x7f5c9c05bc10 con 0x7f5ca40d1c20 2024-04-25T14:08:44.686 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-04-25T14:08:44.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.686+0000 7f5cab800640 1 -- 172.21.15.86:0/1904290628 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f5c8406f550 msgr2=0x7f5c84071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:44.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.686+0000 7f5cab800640 1 --2- 172.21.15.86:0/1904290628 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f5c8406f550 0x7f5c84071a60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5c940af000 tx=0x7f5c940a7070 comp rx=0 tx=0).stop 2024-04-25T14:08:44.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.686+0000 7f5cab800640 1 -- 172.21.15.86:0/1904290628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ca40d1c20 msgr2=0x7f5ca4099710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:44.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.686+0000 7f5cab800640 1 --2- 172.21.15.86:0/1904290628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ca40d1c20 0x7f5ca4099710 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00c410 tx=0x7f5c9c00c810 comp rx=0 tx=0).stop 2024-04-25T14:08:44.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.686+0000 7f5cab800640 1 -- 172.21.15.86:0/1904290628 shutdown_connections 2024-04-25T14:08:44.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.686+0000 7f5cab800640 1 --2- 172.21.15.86:0/1904290628 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f5c8406f550 0x7f5c84071a60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:44.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.686+0000 7f5cab800640 1 --2- 172.21.15.86:0/1904290628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ca40d1c20 0x7f5ca4099710 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:44.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.686+0000 7f5cab800640 1 --2- 172.21.15.86:0/1904290628 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca4070330 0x7f5ca40af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:44.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.686+0000 7f5cab800640 1 --2- 172.21.15.86:0/1904290628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca406f270 0x7f5ca409a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:44.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.686+0000 7f5cab800640 1 -- 172.21.15.86:0/1904290628 >> 172.21.15.86:0/1904290628 conn(0x7f5ca40b4650 msgr2=0x7f5ca40b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:44.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.690+0000 7f5cab800640 1 -- 172.21.15.86:0/1904290628 shutdown_connections 2024-04-25T14:08:44.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.690+0000 7f5cab800640 1 -- 172.21.15.86:0/1904290628 wait complete. 2024-04-25T14:08:44.731 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T14:08:44.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.882+0000 7fd0ea600640 1 Processor -- start 2024-04-25T14:08:44.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.882+0000 7fd0ea600640 1 -- start start 2024-04-25T14:08:44.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.882+0000 7fd0ea600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0e40d10c0 0x7fd0e40cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:44.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.882+0000 7fd0ea600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd0e40d14d0 0x7fd0e40d0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:44.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.882+0000 7fd0ea600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0e40d19c0 con 0x7fd0e415fe90 2024-04-25T14:08:44.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.882+0000 7fd0ea600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd0e40d1ca0 con 0x7fd0e40d14d0 2024-04-25T14:08:44.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.882+0000 7fd0ea600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd0e4163bb0 con 0x7fd0e40d10c0 2024-04-25T14:08:44.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.882+0000 7fd0e3e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd0e40d14d0 0x7fd0e40d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:44.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.882+0000 7fd0e3e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd0e40d14d0 0x7fd0e40d0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48952/0 (socket says 172.21.15.86:48952) 2024-04-25T14:08:44.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.882+0000 7fd0e3e00640 1 -- 172.21.15.86:0/1831157198 learned_addr learned my addr 172.21.15.86:0/1831157198 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:44.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.886+0000 7fd0daa00640 1 --2- 172.21.15.86:0/1831157198 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0e40d10c0 0x7fd0e40cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:44.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.886+0000 7fd0e3e00640 1 -- 172.21.15.86:0/1831157198 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0e40d10c0 msgr2=0x7fd0e40cfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:44.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.886+0000 7fd0e3e00640 1 --2- 172.21.15.86:0/1831157198 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0e40d10c0 0x7fd0e40cfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:44.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.886+0000 7fd0e3e00640 1 -- 172.21.15.86:0/1831157198 >> v1:172.21.15.155:6789/0 conn(0x7fd0e415fe90 legacy=0x7fd0e41643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:44.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.886+0000 7fd0e3e00640 1 -- 172.21.15.86:0/1831157198 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e4163e90 con 0x7fd0e40d14d0 2024-04-25T14:08:44.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.886+0000 7fd0e3e00640 1 --2- 172.21.15.86:0/1831157198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0e40d14d0 0x7fd0e40d0310 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009870 tx=0x7fd0dc01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e7d07fdeb63bbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:44.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.886+0000 7fd0e2a00640 1 -- 172.21.15.86:0/1831157198 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc0b1070 con 0x7fd0e40d14d0 2024-04-25T14:08:44.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.886+0000 7fd0e2a00640 1 -- 172.21.15.86:0/1831157198 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0dc0a8050 con 0x7fd0e40d14d0 2024-04-25T14:08:44.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.886+0000 7fd0e2a00640 1 -- 172.21.15.86:0/1831157198 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc0ac860 con 0x7fd0e40d14d0 2024-04-25T14:08:44.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.890+0000 7fd0ea600640 1 -- 172.21.15.86:0/1831157198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0e40d14d0 msgr2=0x7fd0e40d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:44.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.890+0000 7fd0ea600640 1 --2- 172.21.15.86:0/1831157198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0e40d14d0 0x7fd0e40d0310 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009870 tx=0x7fd0dc01c4d0 comp rx=0 tx=0).stop 2024-04-25T14:08:44.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.890+0000 7fd0ea600640 1 -- 172.21.15.86:0/1831157198 shutdown_connections 2024-04-25T14:08:44.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.890+0000 7fd0ea600640 1 --2- 172.21.15.86:0/1831157198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0e40d14d0 0x7fd0e40d0310 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:44.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.890+0000 7fd0ea600640 1 --2- 172.21.15.86:0/1831157198 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0e40d10c0 0x7fd0e40cfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:44.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.890+0000 7fd0ea600640 1 -- 172.21.15.86:0/1831157198 >> 172.21.15.86:0/1831157198 conn(0x7fd0e40b5020 msgr2=0x7fd0e40b5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:44.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.890+0000 7fd0ea600640 1 -- 172.21.15.86:0/1831157198 shutdown_connections 2024-04-25T14:08:44.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.890+0000 7fd0ea600640 1 -- 172.21.15.86:0/1831157198 wait complete. 2024-04-25T14:08:44.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.890+0000 7fd0ea600640 1 Processor -- start 2024-04-25T14:08:44.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.890+0000 7fd0ea600640 1 -- start start 2024-04-25T14:08:44.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.894+0000 7fd0ea600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0e40d10c0 0x7fd0e41430a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:44.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.894+0000 7fd0ea600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0e40d14d0 0x7fd0e4145400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:44.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.894+0000 7fd0ea600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0e415fe90 0x7fd0e40c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:44.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.894+0000 7fd0ea600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e40b81d0 con 0x7fd0e40d10c0 2024-04-25T14:08:44.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.894+0000 7fd0ea600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e40b8050 con 0x7fd0e40d14d0 2024-04-25T14:08:44.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.894+0000 7fd0ea600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd0e416cd50 con 0x7fd0e415fe90 2024-04-25T14:08:44.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.894+0000 7fd0e3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0e40d10c0 0x7fd0e41430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:44.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.894+0000 7fd0e3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0e40d10c0 0x7fd0e41430a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48964/0 (socket says 172.21.15.86:48964) 2024-04-25T14:08:44.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.894+0000 7fd0e3400640 1 -- 172.21.15.86:0/537868253 learned_addr learned my addr 172.21.15.86:0/537868253 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:44.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.894+0000 7fd0e3e00640 1 --2- 172.21.15.86:0/537868253 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0e415fe90 0x7fd0e40c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:44.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.894+0000 7fd0daa00640 1 --2- 172.21.15.86:0/537868253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0e40d14d0 0x7fd0e4145400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:44.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.894+0000 7fd0e3400640 1 -- 172.21.15.86:0/537868253 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0e415fe90 msgr2=0x7fd0e40c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:44.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.894+0000 7fd0e3400640 1 --2- 172.21.15.86:0/537868253 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0e415fe90 0x7fd0e40c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:44.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.894+0000 7fd0e3400640 1 -- 172.21.15.86:0/537868253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0e40d14d0 msgr2=0x7fd0e4145400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:44.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.894+0000 7fd0e3400640 1 --2- 172.21.15.86:0/537868253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0e40d14d0 0x7fd0e4145400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:44.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.894+0000 7fd0e3400640 1 -- 172.21.15.86:0/537868253 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e40c90c0 con 0x7fd0e40d10c0 2024-04-25T14:08:44.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.894+0000 7fd0e3400640 1 --2- 172.21.15.86:0/537868253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0e40d10c0 0x7fd0e41430a0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fd0d400c730 tx=0x7fd0d400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:44.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.894+0000 7fd0dbe00640 1 -- 172.21.15.86:0/537868253 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4016020 con 0x7fd0e40d10c0 2024-04-25T14:08:44.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.894+0000 7fd0dbe00640 1 -- 172.21.15.86:0/537868253 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0d4006d60 con 0x7fd0e40d10c0 2024-04-25T14:08:44.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.894+0000 7fd0dbe00640 1 -- 172.21.15.86:0/537868253 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4003bc0 con 0x7fd0e40d10c0 2024-04-25T14:08:44.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.894+0000 7fd0ea600640 1 -- 172.21.15.86:0/537868253 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0e4162cc0 con 0x7fd0e40d10c0 2024-04-25T14:08:44.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.894+0000 7fd0ea600640 1 -- 172.21.15.86:0/537868253 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0e4163280 con 0x7fd0e40d10c0 2024-04-25T14:08:44.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.894+0000 7fd0ea600640 1 -- 172.21.15.86:0/537868253 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0e4072170 con 0x7fd0e40d10c0 2024-04-25T14:08:44.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.898+0000 7fd0dbe00640 1 -- 172.21.15.86:0/537868253 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7fd0d400a850 con 0x7fd0e40d10c0 2024-04-25T14:08:44.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.898+0000 7fd0dbe00640 1 --2- 172.21.15.86:0/537868253 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fd0c006f520 0x7fd0c0071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:44.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.898+0000 7fd0dbe00640 1 -- 172.21.15.86:0/537868253 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 6001+0+0 (secure 0 0 0) 0x7fd0d408fdf0 con 0x7fd0e40d10c0 2024-04-25T14:08:44.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.898+0000 7fd0daa00640 1 --2- 172.21.15.86:0/537868253 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fd0c006f520 0x7fd0c0071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:44.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.898+0000 7fd0daa00640 1 --2- 172.21.15.86:0/537868253 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fd0c006f520 0x7fd0c0071a30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd0e41626e0 tx=0x7fd0cc009190 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:44.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:44.902+0000 7fd0dbe00640 1 -- 172.21.15.86:0/537868253 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd0d4020020 con 0x7fd0e40d10c0 2024-04-25T14:08:45.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.090+0000 7fd0ea600640 1 -- 172.21.15.86:0/537868253 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd0e4076bf0 con 0x7fd0e40d10c0 2024-04-25T14:08:45.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.710+0000 7fd0dbe00640 1 -- 172.21.15.86:0/537868253 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v63) ==== 174+0+0 (secure 0 0 0) 0x7fd0d405c550 con 0x7fd0e40d10c0 2024-04-25T14:08:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 13 and data pool 14 2024-04-25T14:08:45.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.722+0000 7fd0ea600640 1 -- 172.21.15.86:0/537868253 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fd0c006f520 msgr2=0x7fd0c0071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:45.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.722+0000 7fd0ea600640 1 --2- 172.21.15.86:0/537868253 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fd0c006f520 0x7fd0c0071a30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd0e41626e0 tx=0x7fd0cc009190 comp rx=0 tx=0).stop 2024-04-25T14:08:45.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.722+0000 7fd0ea600640 1 -- 172.21.15.86:0/537868253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0e40d10c0 msgr2=0x7fd0e41430a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:45.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.722+0000 7fd0ea600640 1 --2- 172.21.15.86:0/537868253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0e40d10c0 0x7fd0e41430a0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fd0d400c730 tx=0x7fd0d400cbf0 comp rx=0 tx=0).stop 2024-04-25T14:08:45.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.726+0000 7fd0ea600640 1 -- 172.21.15.86:0/537868253 shutdown_connections 2024-04-25T14:08:45.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.726+0000 7fd0ea600640 1 --2- 172.21.15.86:0/537868253 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fd0c006f520 0x7fd0c0071a30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:45.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.726+0000 7fd0ea600640 1 --2- 172.21.15.86:0/537868253 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0e415fe90 0x7fd0e40c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:45.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.726+0000 7fd0ea600640 1 --2- 172.21.15.86:0/537868253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0e40d14d0 0x7fd0e4145400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:45.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.726+0000 7fd0ea600640 1 --2- 172.21.15.86:0/537868253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0e40d10c0 0x7fd0e41430a0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:45.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.726+0000 7fd0ea600640 1 -- 172.21.15.86:0/537868253 >> 172.21.15.86:0/537868253 conn(0x7fd0e40b5020 msgr2=0x7fd0e40d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:45.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.734+0000 7fd0ea600640 1 -- 172.21.15.86:0/537868253 shutdown_connections 2024-04-25T14:08:45.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.734+0000 7fd0ea600640 1 -- 172.21.15.86:0/537868253 wait complete. 2024-04-25T14:08:45.769 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:08:45.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.914+0000 7f3233e00640 1 Processor -- start 2024-04-25T14:08:45.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.914+0000 7f3233e00640 1 -- start start 2024-04-25T14:08:45.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.918+0000 7f3233e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f322c06fd50 0x7f322c070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:45.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.918+0000 7f3233e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f322c15feb0 0x7f322c1602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:45.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.918+0000 7f3233e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f322c0b0a60 con 0x7f322c15c2a0 2024-04-25T14:08:45.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.918+0000 7f3233e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f322c0ae840 con 0x7f322c15feb0 2024-04-25T14:08:45.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.918+0000 7f3233e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f322c06ef90 con 0x7f322c06fd50 2024-04-25T14:08:45.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.918+0000 7f3231e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f322c15feb0 0x7f322c1602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:45.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.918+0000 7f3231e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f322c15feb0 0x7f322c1602c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48968/0 (socket says 172.21.15.86:48968) 2024-04-25T14:08:45.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.918+0000 7f3231e00640 1 -- 172.21.15.86:0/1483435571 learned_addr learned my addr 172.21.15.86:0/1483435571 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:45.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.918+0000 7f3230a00640 1 --2- 172.21.15.86:0/1483435571 >> v2:172.21.15.155:3300/0 conn(0x7f322c06fd50 0x7f322c070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:45.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.918+0000 7f3231e00640 1 -- 172.21.15.86:0/1483435571 >> v2:172.21.15.155:3300/0 conn(0x7f322c06fd50 msgr2=0x7f322c070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:45.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.918+0000 7f3231e00640 1 --2- 172.21.15.86:0/1483435571 >> v2:172.21.15.155:3300/0 conn(0x7f322c06fd50 0x7f322c070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:45.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.918+0000 7f3231e00640 1 -- 172.21.15.86:0/1483435571 >> v1:172.21.15.86:6789/0 conn(0x7f322c15c2a0 legacy=0x7f322c0ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:45.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.918+0000 7f3231e00640 1 -- 172.21.15.86:0/1483435571 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322c06f270 con 0x7f322c15feb0 2024-04-25T14:08:45.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.918+0000 7f3231e00640 1 --2- 172.21.15.86:0/1483435571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f322c15feb0 0x7f322c1602c0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f322c074f40 tx=0x7f322401eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8450afc08944a269 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:45.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.922+0000 7f322be00640 1 -- 172.21.15.86:0/1483435571 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322400b2d0 con 0x7f322c15feb0 2024-04-25T14:08:45.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.922+0000 7f322be00640 1 -- 172.21.15.86:0/1483435571 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f322400b470 con 0x7f322c15feb0 2024-04-25T14:08:45.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.922+0000 7f322be00640 1 -- 172.21.15.86:0/1483435571 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32240a7ba0 con 0x7f322c15feb0 2024-04-25T14:08:45.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.922+0000 7f3233e00640 1 -- 172.21.15.86:0/1483435571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f322c15feb0 msgr2=0x7f322c1602c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:45.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.922+0000 7f3233e00640 1 --2- 172.21.15.86:0/1483435571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f322c15feb0 0x7f322c1602c0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f322c074f40 tx=0x7f322401eb30 comp rx=0 tx=0).stop 2024-04-25T14:08:45.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.922+0000 7f3233e00640 1 -- 172.21.15.86:0/1483435571 shutdown_connections 2024-04-25T14:08:45.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.922+0000 7f3233e00640 1 --2- 172.21.15.86:0/1483435571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f322c15feb0 0x7f322c1602c0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:45.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.922+0000 7f3233e00640 1 --2- 172.21.15.86:0/1483435571 >> v2:172.21.15.155:3300/0 conn(0x7f322c06fd50 0x7f322c070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:45.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.922+0000 7f3233e00640 1 -- 172.21.15.86:0/1483435571 >> 172.21.15.86:0/1483435571 conn(0x7f322c0b4650 msgr2=0x7f322c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:45.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.922+0000 7f3233e00640 1 -- 172.21.15.86:0/1483435571 shutdown_connections 2024-04-25T14:08:45.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.922+0000 7f3233e00640 1 -- 172.21.15.86:0/1483435571 wait complete. 2024-04-25T14:08:45.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.922+0000 7f3233e00640 1 Processor -- start 2024-04-25T14:08:45.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.922+0000 7f3233e00640 1 -- start start 2024-04-25T14:08:45.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.926+0000 7f3233e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f322c06fd50 0x7f322c0b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:45.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.926+0000 7f3233e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f322c15c2a0 0x7f322c1f97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:45.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.926+0000 7f3233e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f322c15feb0 0x7f322c073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:45.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.926+0000 7f3233e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f322c15c6b0 con 0x7f322c06fd50 2024-04-25T14:08:45.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.926+0000 7f3233e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f322c0d0fc0 con 0x7f322c15c2a0 2024-04-25T14:08:45.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.926+0000 7f3233e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f322c15c830 con 0x7f322c15feb0 2024-04-25T14:08:45.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.926+0000 7f3231400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f322c06fd50 0x7f322c0b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:45.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.926+0000 7f3231400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f322c06fd50 0x7f322c0b0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48984/0 (socket says 172.21.15.86:48984) 2024-04-25T14:08:45.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.926+0000 7f3231400640 1 -- 172.21.15.86:0/1724432556 learned_addr learned my addr 172.21.15.86:0/1724432556 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:45.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.926+0000 7f3231400640 1 -- 172.21.15.86:0/1724432556 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f322c15feb0 msgr2=0x7f322c073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:45.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.926+0000 7f3231e00640 1 --2- 172.21.15.86:0/1724432556 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f322c15feb0 0x7f322c073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:45.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.926+0000 7f3230a00640 1 --2- 172.21.15.86:0/1724432556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f322c15c2a0 0x7f322c1f97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:45.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.926+0000 7f3231400640 1 --2- 172.21.15.86:0/1724432556 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f322c15feb0 0x7f322c073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:45.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.926+0000 7f3231400640 1 -- 172.21.15.86:0/1724432556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f322c15c2a0 msgr2=0x7f322c1f97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:45.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.926+0000 7f3231400640 1 --2- 172.21.15.86:0/1724432556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f322c15c2a0 0x7f322c1f97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:45.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.926+0000 7f3231400640 1 -- 172.21.15.86:0/1724432556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322c0b14a0 con 0x7f322c06fd50 2024-04-25T14:08:45.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.926+0000 7f3231e00640 1 --2- 172.21.15.86:0/1724432556 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f322c15feb0 0x7f322c073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:45.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.926+0000 7f3230a00640 1 --2- 172.21.15.86:0/1724432556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f322c15c2a0 0x7f322c1f97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:45.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.926+0000 7f3231400640 1 --2- 172.21.15.86:0/1724432556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f322c06fd50 0x7f322c0b0e50 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f322000c340 tx=0x7f322000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:45.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.926+0000 7f3229600640 1 -- 172.21.15.86:0/1724432556 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32200052d0 con 0x7f322c06fd50 2024-04-25T14:08:45.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.926+0000 7f3229600640 1 -- 172.21.15.86:0/1724432556 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3220005470 con 0x7f322c06fd50 2024-04-25T14:08:45.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.926+0000 7f3233e00640 1 -- 172.21.15.86:0/1724432556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f322c1551b0 con 0x7f322c06fd50 2024-04-25T14:08:45.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.926+0000 7f3229600640 1 -- 172.21.15.86:0/1724432556 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3220007260 con 0x7f322c06fd50 2024-04-25T14:08:45.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.926+0000 7f3233e00640 1 -- 172.21.15.86:0/1724432556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f322c155770 con 0x7f322c06fd50 2024-04-25T14:08:45.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.926+0000 7f3233e00640 1 -- 172.21.15.86:0/1724432556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f322c069d00 con 0x7f322c06fd50 2024-04-25T14:08:45.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.930+0000 7f3229600640 1 -- 172.21.15.86:0/1724432556 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7f3220007400 con 0x7f322c06fd50 2024-04-25T14:08:45.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.930+0000 7f3229600640 1 --2- 172.21.15.86:0/1724432556 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f320c06f550 0x7f320c071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:45.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.930+0000 7f3229600640 1 -- 172.21.15.86:0/1724432556 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f322008ea10 con 0x7f322c06fd50 2024-04-25T14:08:45.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.930+0000 7f3230a00640 1 --2- 172.21.15.86:0/1724432556 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f320c06f550 0x7f320c071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:45.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.930+0000 7f3230a00640 1 --2- 172.21.15.86:0/1724432556 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f320c06f550 0x7f320c071a60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f322c0cf150 tx=0x7f3218008040 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:45.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:45.934+0000 7f3229600640 1 -- 172.21.15.86:0/1724432556 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f322005b660 con 0x7f322c06fd50 2024-04-25T14:08:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.090+0000 7f3233e00640 1 -- 172.21.15.86:0/1724432556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f322c076bf0 con 0x7f322c06fd50 2024-04-25T14:08:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.094+0000 7f3229600640 1 -- 172.21.15.86:0/1724432556 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+16898 (secure 0 0 0) 0x7f322005af60 con 0x7f322c06fd50 2024-04-25T14:08:46.094 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:08:46.095 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":86,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:08:45.096543+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:08:43.505988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T14:08:44.054628+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:08:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.094+0000 7f3233e00640 1 -- 172.21.15.86:0/1724432556 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f320c06f550 msgr2=0x7f320c071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.094+0000 7f3233e00640 1 --2- 172.21.15.86:0/1724432556 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f320c06f550 0x7f320c071a60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f322c0cf150 tx=0x7f3218008040 comp rx=0 tx=0).stop 2024-04-25T14:08:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.094+0000 7f3233e00640 1 -- 172.21.15.86:0/1724432556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f322c06fd50 msgr2=0x7f322c0b0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.094+0000 7f3233e00640 1 --2- 172.21.15.86:0/1724432556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f322c06fd50 0x7f322c0b0e50 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f322000c340 tx=0x7f322000c800 comp rx=0 tx=0).stop 2024-04-25T14:08:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.094+0000 7f3233e00640 1 -- 172.21.15.86:0/1724432556 shutdown_connections 2024-04-25T14:08:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.094+0000 7f3233e00640 1 --2- 172.21.15.86:0/1724432556 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f320c06f550 0x7f320c071a60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.094+0000 7f3233e00640 1 --2- 172.21.15.86:0/1724432556 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f322c15feb0 0x7f322c073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.094+0000 7f3233e00640 1 --2- 172.21.15.86:0/1724432556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f322c15c2a0 0x7f322c1f97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.094+0000 7f3233e00640 1 --2- 172.21.15.86:0/1724432556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f322c06fd50 0x7f322c0b0e50 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.094+0000 7f3233e00640 1 -- 172.21.15.86:0/1724432556 >> 172.21.15.86:0/1724432556 conn(0x7f322c0b4650 msgr2=0x7f322c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.094+0000 7f3233e00640 1 -- 172.21.15.86:0/1724432556 shutdown_connections 2024-04-25T14:08:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.094+0000 7f3233e00640 1 -- 172.21.15.86:0/1724432556 wait complete. 2024-04-25T14:08:46.117 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:08:46.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.262+0000 7fc399a00640 1 Processor -- start 2024-04-25T14:08:46.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.262+0000 7fc399a00640 1 -- start start 2024-04-25T14:08:46.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.266+0000 7fc399a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc394159d40 0x7fc39415d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:46.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.266+0000 7fc399a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc39406c670 0x7fc39406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:46.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.266+0000 7fc399a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc39415d5f0 con 0x7fc39415fc20 2024-04-25T14:08:46.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.266+0000 7fc399a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc39406cfc0 con 0x7fc39406c670 2024-04-25T14:08:46.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.266+0000 7fc399a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc39406ea40 con 0x7fc394159d40 2024-04-25T14:08:46.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.266+0000 7fc38b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc39406c670 0x7fc39406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:46.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.266+0000 7fc399000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc39415fc20 0x7fc394160030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60100/0 (socket says 172.21.15.86:60100) 2024-04-25T14:08:46.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.266+0000 7fc399000640 1 -- 172.21.15.86:0/375123621 learned_addr learned my addr 172.21.15.86:0/375123621 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:46.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.266+0000 7fc393e00640 1 --2- 172.21.15.86:0/375123621 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc394159d40 0x7fc39415d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:46.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.266+0000 7fc38b400640 1 -- 172.21.15.86:0/375123621 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc394159d40 msgr2=0x7fc39415d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:46.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.266+0000 7fc38b400640 1 --2- 172.21.15.86:0/375123621 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc394159d40 0x7fc39415d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:46.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.266+0000 7fc38b400640 1 -- 172.21.15.86:0/375123621 >> v1:172.21.15.86:6789/0 conn(0x7fc39415fc20 legacy=0x7fc394160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:46.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.266+0000 7fc38b400640 1 -- 172.21.15.86:0/375123621 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39406ed20 con 0x7fc39406c670 2024-04-25T14:08:46.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.266+0000 7fc393400640 1 -- 172.21.15.86:0/375123621 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 282002915 0 0) 0x7fc38c001120 con 0x7fc39415fc20 2024-04-25T14:08:46.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.270+0000 7fc38b400640 1 --2- 172.21.15.86:0/375123621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc39406c670 0x7fc39406ca80 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fc37c00ba60 tx=0x7fc37c01eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb90e78ffe67917 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:46.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.270+0000 7fc393400640 1 -- 172.21.15.86:0/375123621 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c0b4070 con 0x7fc39406c670 2024-04-25T14:08:46.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.270+0000 7fc393400640 1 -- 172.21.15.86:0/375123621 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc37c0a7aa0 con 0x7fc39406c670 2024-04-25T14:08:46.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.270+0000 7fc393400640 1 -- 172.21.15.86:0/375123621 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c0af440 con 0x7fc39406c670 2024-04-25T14:08:46.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.270+0000 7fc399a00640 1 -- 172.21.15.86:0/375123621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc39406c670 msgr2=0x7fc39406ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:46.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.270+0000 7fc399a00640 1 --2- 172.21.15.86:0/375123621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc39406c670 0x7fc39406ca80 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fc37c00ba60 tx=0x7fc37c01eae0 comp rx=0 tx=0).stop 2024-04-25T14:08:46.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.270+0000 7fc399a00640 1 -- 172.21.15.86:0/375123621 shutdown_connections 2024-04-25T14:08:46.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.270+0000 7fc399a00640 1 --2- 172.21.15.86:0/375123621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc39406c670 0x7fc39406ca80 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:46.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.270+0000 7fc399a00640 1 --2- 172.21.15.86:0/375123621 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc394159d40 0x7fc39415d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:46.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.270+0000 7fc399a00640 1 -- 172.21.15.86:0/375123621 >> 172.21.15.86:0/375123621 conn(0x7fc3940b4850 msgr2=0x7fc3940b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:46.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.270+0000 7fc399a00640 1 -- 172.21.15.86:0/375123621 shutdown_connections 2024-04-25T14:08:46.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.270+0000 7fc399a00640 1 -- 172.21.15.86:0/375123621 wait complete. 2024-04-25T14:08:46.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.270+0000 7fc399a00640 1 Processor -- start 2024-04-25T14:08:46.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.270+0000 7fc399a00640 1 -- start start 2024-04-25T14:08:46.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.274+0000 7fc399a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc39406c670 0x7fc3941f9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:46.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.274+0000 7fc399a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc394159d40 0x7fc394166780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:46.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.274+0000 7fc399a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc39415fc20 0x7fc394145d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:46.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.274+0000 7fc399a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc394169140 con 0x7fc394159d40 2024-04-25T14:08:46.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.274+0000 7fc399a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc394168fc0 con 0x7fc39415fc20 2024-04-25T14:08:46.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.274+0000 7fc399a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc3941692c0 con 0x7fc39406c670 2024-04-25T14:08:46.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.274+0000 7fc38b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc394159d40 0x7fc394166780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:46.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.274+0000 7fc38b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc394159d40 0x7fc394166780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49010/0 (socket says 172.21.15.86:49010) 2024-04-25T14:08:46.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.274+0000 7fc38b400640 1 -- 172.21.15.86:0/4145068701 learned_addr learned my addr 172.21.15.86:0/4145068701 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:46.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.274+0000 7fc393e00640 1 --2- 172.21.15.86:0/4145068701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc39406c670 0x7fc3941f9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:46.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.274+0000 7fc399000640 1 --2- 172.21.15.86:0/4145068701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc39415fc20 0x7fc394145d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:46.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.274+0000 7fc38b400640 1 -- 172.21.15.86:0/4145068701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc39406c670 msgr2=0x7fc3941f9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:46.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.274+0000 7fc38b400640 1 --2- 172.21.15.86:0/4145068701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc39406c670 0x7fc3941f9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:46.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.274+0000 7fc38b400640 1 -- 172.21.15.86:0/4145068701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc39415fc20 msgr2=0x7fc394145d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:46.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.274+0000 7fc38b400640 1 --2- 172.21.15.86:0/4145068701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc39415fc20 0x7fc394145d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:46.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.274+0000 7fc38b400640 1 -- 172.21.15.86:0/4145068701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3941fd190 con 0x7fc394159d40 2024-04-25T14:08:46.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.274+0000 7fc38b400640 1 --2- 172.21.15.86:0/4145068701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc394159d40 0x7fc394166780 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fc37c01c710 tx=0x7fc37c01b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:46.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.274+0000 7fc390c00640 1 -- 172.21.15.86:0/4145068701 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c0bd070 con 0x7fc394159d40 2024-04-25T14:08:46.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.274+0000 7fc390c00640 1 -- 172.21.15.86:0/4145068701 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc37c0af880 con 0x7fc394159d40 2024-04-25T14:08:46.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.274+0000 7fc399a00640 1 -- 172.21.15.86:0/4145068701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc394146290 con 0x7fc394159d40 2024-04-25T14:08:46.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.274+0000 7fc390c00640 1 -- 172.21.15.86:0/4145068701 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c0b4040 con 0x7fc394159d40 2024-04-25T14:08:46.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.274+0000 7fc399a00640 1 -- 172.21.15.86:0/4145068701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3941fa6b0 con 0x7fc394159d40 2024-04-25T14:08:46.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.274+0000 7fc399a00640 1 -- 172.21.15.86:0/4145068701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3941fa110 con 0x7fc394159d40 2024-04-25T14:08:46.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.278+0000 7fc390c00640 1 -- 172.21.15.86:0/4145068701 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7fc37c0c0070 con 0x7fc394159d40 2024-04-25T14:08:46.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.278+0000 7fc390c00640 1 --2- 172.21.15.86:0/4145068701 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fc37006f520 0x7fc370071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:46.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.278+0000 7fc390c00640 1 -- 172.21.15.86:0/4145068701 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7fc37c128f30 con 0x7fc394159d40 2024-04-25T14:08:46.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.278+0000 7fc393e00640 1 --2- 172.21.15.86:0/4145068701 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fc37006f520 0x7fc370071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:46.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.278+0000 7fc393e00640 1 --2- 172.21.15.86:0/4145068701 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fc37006f520 0x7fc370071a30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc384003fd0 tx=0x7fc3840039a0 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:46.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.282+0000 7fc390c00640 1 -- 172.21.15.86:0/4145068701 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc37c018760 con 0x7fc394159d40 2024-04-25T14:08:46.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.442+0000 7fc399a00640 1 -- 172.21.15.86:0/4145068701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc394076b80 con 0x7fc394159d40 2024-04-25T14:08:46.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.442+0000 7fc390c00640 1 -- 172.21.15.86:0/4145068701 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+16898 (secure 0 0 0) 0x7fc37c0f5ae0 con 0x7fc394159d40 2024-04-25T14:08:46.443 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:08:46.444 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":86,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:08:45.096543+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:08:43.505988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T14:08:44.054628+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:08:46.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.442+0000 7fc399a00640 1 -- 172.21.15.86:0/4145068701 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fc37006f520 msgr2=0x7fc370071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:46.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.442+0000 7fc399a00640 1 --2- 172.21.15.86:0/4145068701 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fc37006f520 0x7fc370071a30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc384003fd0 tx=0x7fc3840039a0 comp rx=0 tx=0).stop 2024-04-25T14:08:46.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.442+0000 7fc399a00640 1 -- 172.21.15.86:0/4145068701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc394159d40 msgr2=0x7fc394166780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:46.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.442+0000 7fc399a00640 1 --2- 172.21.15.86:0/4145068701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc394159d40 0x7fc394166780 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fc37c01c710 tx=0x7fc37c01b880 comp rx=0 tx=0).stop 2024-04-25T14:08:46.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.442+0000 7fc399a00640 1 -- 172.21.15.86:0/4145068701 shutdown_connections 2024-04-25T14:08:46.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.442+0000 7fc399a00640 1 --2- 172.21.15.86:0/4145068701 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fc37006f520 0x7fc370071a30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:46.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.442+0000 7fc399a00640 1 --2- 172.21.15.86:0/4145068701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc39415fc20 0x7fc394145d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:46.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.442+0000 7fc399a00640 1 --2- 172.21.15.86:0/4145068701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc394159d40 0x7fc394166780 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:46.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.442+0000 7fc399a00640 1 --2- 172.21.15.86:0/4145068701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc39406c670 0x7fc3941f9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:46.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.442+0000 7fc399a00640 1 -- 172.21.15.86:0/4145068701 >> 172.21.15.86:0/4145068701 conn(0x7fc3940b4850 msgr2=0x7fc39415af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:46.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.442+0000 7fc399a00640 1 -- 172.21.15.86:0/4145068701 shutdown_connections 2024-04-25T14:08:46.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.442+0000 7fc399a00640 1 -- 172.21.15.86:0/4145068701 wait complete. 2024-04-25T14:08:46.466 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T14:08:46.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.614+0000 7f1f6fe00640 1 Processor -- start 2024-04-25T14:08:46.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.614+0000 7f1f6fe00640 1 -- start start 2024-04-25T14:08:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.614+0000 7f1f6fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1f6815e860 0x7f1f680ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.614+0000 7f1f6fe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f680cf3e0 0x7f1f680d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.614+0000 7f1f6fe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1f6815ec70 0x7f1f680d1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.614+0000 7f1f6fe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1f6806cfa0 con 0x7f1f6815e860 2024-04-25T14:08:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.614+0000 7f1f6fe00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1f680d19d0 con 0x7f1f6815ec70 2024-04-25T14:08:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.614+0000 7f1f6fe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1f680cf8f0 con 0x7f1f680cf3e0 2024-04-25T14:08:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.614+0000 7f1f6d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1f6815e860 0x7f1f680ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.614+0000 7f1f6d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1f6815e860 0x7f1f680ceea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49016/0 (socket says 172.21.15.86:49016) 2024-04-25T14:08:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.614+0000 7f1f6d400640 1 -- 172.21.15.86:0/1224531256 learned_addr learned my addr 172.21.15.86:0/1224531256 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.614+0000 7f1f6de00640 1 --2- 172.21.15.86:0/1224531256 >> v2:172.21.15.155:3300/0 conn(0x7f1f6815ec70 0x7f1f680d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.614+0000 7f1f6ca00640 1 --2- 172.21.15.86:0/1224531256 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f680cf3e0 0x7f1f680d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.618+0000 7f1f6d400640 1 -- 172.21.15.86:0/1224531256 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f680cf3e0 msgr2=0x7f1f680d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.618+0000 7f1f6d400640 1 --2- 172.21.15.86:0/1224531256 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f680cf3e0 0x7f1f680d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.618+0000 7f1f6d400640 1 -- 172.21.15.86:0/1224531256 >> v2:172.21.15.155:3300/0 conn(0x7f1f6815ec70 msgr2=0x7f1f680d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.618+0000 7f1f6d400640 1 --2- 172.21.15.86:0/1224531256 >> v2:172.21.15.155:3300/0 conn(0x7f1f6815ec70 0x7f1f680d1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.618+0000 7f1f6d400640 1 -- 172.21.15.86:0/1224531256 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6806c670 con 0x7f1f6815e860 2024-04-25T14:08:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.618+0000 7f1f6d400640 1 --2- 172.21.15.86:0/1224531256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f6815e860 0x7f1f680ceea0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f1f680cf7f0 tx=0x7f1f5c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89b6e150645ef55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.618+0000 7f1f67e00640 1 -- 172.21.15.86:0/1224531256 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c0a57b0 con 0x7f1f6815e860 2024-04-25T14:08:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.618+0000 7f1f67e00640 1 -- 172.21.15.86:0/1224531256 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f5c0a5950 con 0x7f1f6815e860 2024-04-25T14:08:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.618+0000 7f1f67e00640 1 -- 172.21.15.86:0/1224531256 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c0a5d20 con 0x7f1f6815e860 2024-04-25T14:08:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.622+0000 7f1f6fe00640 1 -- 172.21.15.86:0/1224531256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f6815e860 msgr2=0x7f1f680ceea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.622+0000 7f1f6fe00640 1 --2- 172.21.15.86:0/1224531256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f6815e860 0x7f1f680ceea0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f1f680cf7f0 tx=0x7f1f5c01c930 comp rx=0 tx=0).stop 2024-04-25T14:08:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.622+0000 7f1f6fe00640 1 -- 172.21.15.86:0/1224531256 shutdown_connections 2024-04-25T14:08:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.622+0000 7f1f6fe00640 1 --2- 172.21.15.86:0/1224531256 >> v2:172.21.15.155:3300/0 conn(0x7f1f6815ec70 0x7f1f680d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.622+0000 7f1f6fe00640 1 --2- 172.21.15.86:0/1224531256 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f680cf3e0 0x7f1f680d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.622+0000 7f1f6fe00640 1 --2- 172.21.15.86:0/1224531256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f6815e860 0x7f1f680ceea0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.622+0000 7f1f6fe00640 1 -- 172.21.15.86:0/1224531256 >> 172.21.15.86:0/1224531256 conn(0x7f1f680d2230 msgr2=0x7f1f680d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.622+0000 7f1f6fe00640 1 -- 172.21.15.86:0/1224531256 shutdown_connections 2024-04-25T14:08:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.622+0000 7f1f6fe00640 1 -- 172.21.15.86:0/1224531256 wait complete. 2024-04-25T14:08:46.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.622+0000 7f1f6fe00640 1 Processor -- start 2024-04-25T14:08:46.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.622+0000 7f1f6fe00640 1 -- start start 2024-04-25T14:08:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.626+0000 7f1f6fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f680cf3e0 0x7f1f68144190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:46.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.626+0000 7f1f6fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f6815e860 0x7f1f68150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:46.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.626+0000 7f1f6fe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f6815ec70 0x7f1f68150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:46.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.626+0000 7f1f6fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6806ddc0 con 0x7f1f6815e860 2024-04-25T14:08:46.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.626+0000 7f1f6fe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6806dc40 con 0x7f1f680cf3e0 2024-04-25T14:08:46.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.626+0000 7f1f6fe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1f6806df40 con 0x7f1f6815ec70 2024-04-25T14:08:46.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.626+0000 7f1f6ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f6815e860 0x7f1f68150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:46.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.626+0000 7f1f6ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f6815e860 0x7f1f68150340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49018/0 (socket says 172.21.15.86:49018) 2024-04-25T14:08:46.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.626+0000 7f1f6ca00640 1 -- 172.21.15.86:0/1490647396 learned_addr learned my addr 172.21.15.86:0/1490647396 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:46.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.626+0000 7f1f6d400640 1 --2- 172.21.15.86:0/1490647396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f680cf3e0 0x7f1f68144190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:46.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.626+0000 7f1f6de00640 1 --2- 172.21.15.86:0/1490647396 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f6815ec70 0x7f1f68150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:46.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.626+0000 7f1f6ca00640 1 -- 172.21.15.86:0/1490647396 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f6815ec70 msgr2=0x7f1f68150880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:46.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.626+0000 7f1f6ca00640 1 --2- 172.21.15.86:0/1490647396 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f6815ec70 0x7f1f68150880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:46.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.626+0000 7f1f6ca00640 1 -- 172.21.15.86:0/1490647396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f680cf3e0 msgr2=0x7f1f68144190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:46.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.626+0000 7f1f6ca00640 1 --2- 172.21.15.86:0/1490647396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f680cf3e0 0x7f1f68144190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:46.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.626+0000 7f1f6ca00640 1 -- 172.21.15.86:0/1490647396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f680d2880 con 0x7f1f6815e860 2024-04-25T14:08:46.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.626+0000 7f1f6ca00640 1 --2- 172.21.15.86:0/1490647396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f6815e860 0x7f1f68150340 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f1f5400d620 tx=0x7f1f5400da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:46.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.626+0000 7f1f65600640 1 -- 172.21.15.86:0/1490647396 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f540125f0 con 0x7f1f6815e860 2024-04-25T14:08:46.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.626+0000 7f1f65600640 1 -- 172.21.15.86:0/1490647396 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f54019070 con 0x7f1f6815e860 2024-04-25T14:08:46.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.626+0000 7f1f65600640 1 -- 172.21.15.86:0/1490647396 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f540129b0 con 0x7f1f6815e860 2024-04-25T14:08:46.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.626+0000 7f1f6fe00640 1 -- 172.21.15.86:0/1490647396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f681446d0 con 0x7f1f6815e860 2024-04-25T14:08:46.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.626+0000 7f1f6fe00640 1 -- 172.21.15.86:0/1490647396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f6806f460 con 0x7f1f6815e860 2024-04-25T14:08:46.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.626+0000 7f1f6fe00640 1 -- 172.21.15.86:0/1490647396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f68074cf0 con 0x7f1f6815e860 2024-04-25T14:08:46.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.630+0000 7f1f65600640 1 -- 172.21.15.86:0/1490647396 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7f1f54012b50 con 0x7f1f6815e860 2024-04-25T14:08:46.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.630+0000 7f1f65600640 1 --2- 172.21.15.86:0/1490647396 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f1f4806f520 0x7f1f48071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:46.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.630+0000 7f1f65600640 1 -- 172.21.15.86:0/1490647396 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f1f5408f800 con 0x7f1f6815e860 2024-04-25T14:08:46.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.630+0000 7f1f6d400640 1 --2- 172.21.15.86:0/1490647396 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f1f4806f520 0x7f1f48071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:46.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.630+0000 7f1f6d400640 1 --2- 172.21.15.86:0/1490647396 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f1f4806f520 0x7f1f48071a30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0ae000 tx=0x7f1f5c0a7070 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:46.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.634+0000 7f1f65600640 1 -- 172.21.15.86:0/1490647396 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1f5405c4e0 con 0x7f1f6815e860 2024-04-25T14:08:46.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:46.830+0000 7f1f6fe00640 1 -- 172.21.15.86:0/1490647396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1f68158420 con 0x7f1f6815e860 2024-04-25T14:08:47.682 INFO:tasks.ceph.mds.a.smithi086.stderr:2024-04-25T14:08:47.682+0000 7fb6ff000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T14:08:47.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.726+0000 7f1f65600640 1 -- 172.21.15.86:0/1490647396 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v66) ==== 118+0+0 (secure 0 0 0) 0x7f1f5405bde0 con 0x7f1f6815e860 2024-04-25T14:08:47.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.730+0000 7f1f6fe00640 1 -- 172.21.15.86:0/1490647396 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f1f4806f520 msgr2=0x7f1f48071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:47.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.730+0000 7f1f6fe00640 1 --2- 172.21.15.86:0/1490647396 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f1f4806f520 0x7f1f48071a30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0ae000 tx=0x7f1f5c0a7070 comp rx=0 tx=0).stop 2024-04-25T14:08:47.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.730+0000 7f1f6fe00640 1 -- 172.21.15.86:0/1490647396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f6815e860 msgr2=0x7f1f68150340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:47.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.730+0000 7f1f6fe00640 1 --2- 172.21.15.86:0/1490647396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f6815e860 0x7f1f68150340 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f1f5400d620 tx=0x7f1f5400da20 comp rx=0 tx=0).stop 2024-04-25T14:08:47.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.730+0000 7f1f6fe00640 1 -- 172.21.15.86:0/1490647396 shutdown_connections 2024-04-25T14:08:47.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.730+0000 7f1f6fe00640 1 --2- 172.21.15.86:0/1490647396 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f1f4806f520 0x7f1f48071a30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:47.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.730+0000 7f1f6fe00640 1 --2- 172.21.15.86:0/1490647396 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f6815ec70 0x7f1f68150880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:47.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.730+0000 7f1f6fe00640 1 --2- 172.21.15.86:0/1490647396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f6815e860 0x7f1f68150340 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:47.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.730+0000 7f1f6fe00640 1 --2- 172.21.15.86:0/1490647396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f680cf3e0 0x7f1f68144190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:47.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.730+0000 7f1f6fe00640 1 -- 172.21.15.86:0/1490647396 >> 172.21.15.86:0/1490647396 conn(0x7f1f680d2230 msgr2=0x7f1f680b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:47.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.730+0000 7f1f6fe00640 1 -- 172.21.15.86:0/1490647396 shutdown_connections 2024-04-25T14:08:47.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.730+0000 7f1f6fe00640 1 -- 172.21.15.86:0/1490647396 wait complete. 2024-04-25T14:08:47.754 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T14:08:47.755 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T14:08:47.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.906+0000 7f694c600640 1 Processor -- start 2024-04-25T14:08:47.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.906+0000 7f694c600640 1 -- start start 2024-04-25T14:08:47.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.906+0000 7f694c600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f694415c2a0 0x7f69440ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:47.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.906+0000 7f694c600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f694415feb0 0x7f69441602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:47.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.906+0000 7f694c600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69440b0a60 con 0x7f694406fd50 2024-04-25T14:08:47.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.906+0000 7f694c600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f694406ef90 con 0x7f694415c2a0 2024-04-25T14:08:47.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.906+0000 7f694c600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f694406f270 con 0x7f694415feb0 2024-04-25T14:08:47.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.906+0000 7f6949200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f694406fd50 0x7f6944070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60116/0 (socket says 172.21.15.86:60116) 2024-04-25T14:08:47.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.906+0000 7f6949200640 1 -- 172.21.15.86:0/403813440 learned_addr learned my addr 172.21.15.86:0/403813440 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:47.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.906+0000 7f6949c00640 1 --2- 172.21.15.86:0/403813440 >> v2:172.21.15.155:3300/0 conn(0x7f694415c2a0 0x7f69440ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:47.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.906+0000 7f694a600640 1 --2- 172.21.15.86:0/403813440 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f694415feb0 0x7f69441602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:47.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.910+0000 7f6943e00640 1 -- 172.21.15.86:0/403813440 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3710270267 0 0) 0x7f69440b0a60 con 0x7f694406fd50 2024-04-25T14:08:47.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.910+0000 7f6943e00640 1 -- 172.21.15.86:0/403813440 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f692c003530 con 0x7f694406fd50 2024-04-25T14:08:47.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.910+0000 7f6949c00640 1 -- 172.21.15.86:0/403813440 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f694415feb0 msgr2=0x7f69441602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:47.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.910+0000 7f6949c00640 1 --2- 172.21.15.86:0/403813440 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f694415feb0 0x7f69441602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:47.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.910+0000 7f6949c00640 1 -- 172.21.15.86:0/403813440 >> v1:172.21.15.86:6789/0 conn(0x7f694406fd50 legacy=0x7f6944070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:47.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.910+0000 7f6949c00640 1 -- 172.21.15.86:0/403813440 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f694406f550 con 0x7f694415c2a0 2024-04-25T14:08:47.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.910+0000 7f694a600640 1 --2- 172.21.15.86:0/403813440 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f694415feb0 0x7f69441602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:08:47.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.910+0000 7f6949c00640 1 --2- 172.21.15.86:0/403813440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f694415c2a0 0x7f69440ae130 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f69380097c0 tx=0x7f693801ce40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c351b7af46a0fe6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:47.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.910+0000 7f6943e00640 1 -- 172.21.15.86:0/403813440 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69380b2070 con 0x7f694415c2a0 2024-04-25T14:08:47.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.910+0000 7f6943e00640 1 -- 172.21.15.86:0/403813440 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69380ad680 con 0x7f694415c2a0 2024-04-25T14:08:47.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.910+0000 7f6943e00640 1 -- 172.21.15.86:0/403813440 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69380ada50 con 0x7f694415c2a0 2024-04-25T14:08:47.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.914+0000 7f694c600640 1 -- 172.21.15.86:0/403813440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f694415c2a0 msgr2=0x7f69440ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:47.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.914+0000 7f694c600640 1 --2- 172.21.15.86:0/403813440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f694415c2a0 0x7f69440ae130 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f69380097c0 tx=0x7f693801ce40 comp rx=0 tx=0).stop 2024-04-25T14:08:47.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.914+0000 7f694c600640 1 -- 172.21.15.86:0/403813440 shutdown_connections 2024-04-25T14:08:47.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.914+0000 7f694c600640 1 --2- 172.21.15.86:0/403813440 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f694415feb0 0x7f69441602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:47.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.914+0000 7f694c600640 1 --2- 172.21.15.86:0/403813440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f694415c2a0 0x7f69440ae130 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:47.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.914+0000 7f694c600640 1 -- 172.21.15.86:0/403813440 >> 172.21.15.86:0/403813440 conn(0x7f69440b4650 msgr2=0x7f69440b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:47.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.914+0000 7f694c600640 1 -- 172.21.15.86:0/403813440 shutdown_connections 2024-04-25T14:08:47.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.914+0000 7f694c600640 1 -- 172.21.15.86:0/403813440 wait complete. 2024-04-25T14:08:47.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.914+0000 7f694c600640 1 Processor -- start 2024-04-25T14:08:47.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.914+0000 7f694c600640 1 -- start start 2024-04-25T14:08:47.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.918+0000 7f694c600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f694406fd50 0x7f69440c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:47.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.918+0000 7f694c600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f694415c2a0 0x7f69440b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:47.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.918+0000 7f694c600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f694415feb0 0x7f69440b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:47.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.918+0000 7f694c600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6944160d90 con 0x7f694406fd50 2024-04-25T14:08:47.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.918+0000 7f694c600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6944160c10 con 0x7f694415feb0 2024-04-25T14:08:47.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.918+0000 7f694c600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f69440ae670 con 0x7f694415c2a0 2024-04-25T14:08:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.918+0000 7f6949c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f694406fd50 0x7f69440c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.918+0000 7f6949c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f694406fd50 0x7f69440c3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49024/0 (socket says 172.21.15.86:49024) 2024-04-25T14:08:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.918+0000 7f6949c00640 1 -- 172.21.15.86:0/4089351731 learned_addr learned my addr 172.21.15.86:0/4089351731 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.918+0000 7f6949200640 1 --2- 172.21.15.86:0/4089351731 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f694415c2a0 0x7f69440b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.918+0000 7f694a600640 1 --2- 172.21.15.86:0/4089351731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f694415feb0 0x7f69440b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.918+0000 7f6949c00640 1 -- 172.21.15.86:0/4089351731 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f694415c2a0 msgr2=0x7f69440b24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.918+0000 7f6949c00640 1 --2- 172.21.15.86:0/4089351731 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f694415c2a0 0x7f69440b24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.918+0000 7f6949c00640 1 -- 172.21.15.86:0/4089351731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f694415feb0 msgr2=0x7f69440b2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.918+0000 7f6949c00640 1 --2- 172.21.15.86:0/4089351731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f694415feb0 0x7f69440b2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.918+0000 7f6949c00640 1 -- 172.21.15.86:0/4089351731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6944099ab0 con 0x7f694406fd50 2024-04-25T14:08:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.918+0000 7f694a600640 1 --2- 172.21.15.86:0/4089351731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f694415feb0 0x7f69440b2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.918+0000 7f6949200640 1 --2- 172.21.15.86:0/4089351731 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f694415c2a0 0x7f69440b24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.918+0000 7f6949c00640 1 --2- 172.21.15.86:0/4089351731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f694406fd50 0x7f69440c3e00 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f693801aa90 tx=0x7f693801ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:47.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.918+0000 7f6941600640 1 -- 172.21.15.86:0/4089351731 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69380bc020 con 0x7f694406fd50 2024-04-25T14:08:47.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.918+0000 7f6941600640 1 -- 172.21.15.86:0/4089351731 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69380adb70 con 0x7f694406fd50 2024-04-25T14:08:47.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.918+0000 7f694c600640 1 -- 172.21.15.86:0/4089351731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6944159930 con 0x7f694406fd50 2024-04-25T14:08:47.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.918+0000 7f6941600640 1 -- 172.21.15.86:0/4089351731 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69380b2040 con 0x7f694406fd50 2024-04-25T14:08:47.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.918+0000 7f694c600640 1 -- 172.21.15.86:0/4089351731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6944099d50 con 0x7f694406fd50 2024-04-25T14:08:47.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.918+0000 7f694c600640 1 -- 172.21.15.86:0/4089351731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6944069d00 con 0x7f694406fd50 2024-04-25T14:08:47.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.922+0000 7f6941600640 1 -- 172.21.15.86:0/4089351731 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7f69380bf020 con 0x7f694406fd50 2024-04-25T14:08:47.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.922+0000 7f6941600640 1 --2- 172.21.15.86:0/4089351731 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f692406f520 0x7f6924071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:47.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.922+0000 7f6941600640 1 -- 172.21.15.86:0/4089351731 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f69381294b0 con 0x7f694406fd50 2024-04-25T14:08:47.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.922+0000 7f6949200640 1 --2- 172.21.15.86:0/4089351731 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f692406f520 0x7f6924071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:47.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.922+0000 7f6949200640 1 --2- 172.21.15.86:0/4089351731 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f692406f520 0x7f6924071a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6944169230 tx=0x7f6930009040 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:47.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:47.926+0000 7f6941600640 1 -- 172.21.15.86:0/4089351731 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f69380c6030 con 0x7f694406fd50 2024-04-25T14:08:48.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.114+0000 7f694c600640 1 -- 172.21.15.86:0/4089351731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f694406eae0 con 0x7f694406fd50 2024-04-25T14:08:48.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.730+0000 7f6941600640 1 -- 172.21.15.86:0/4089351731 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v67) ==== 122+0+0 (secure 0 0 0) 0x7f6938016090 con 0x7f694406fd50 2024-04-25T14:08:48.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.730+0000 7f694c600640 1 -- 172.21.15.86:0/4089351731 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f692406f520 msgr2=0x7f6924071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:48.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.730+0000 7f694c600640 1 --2- 172.21.15.86:0/4089351731 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f692406f520 0x7f6924071a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6944169230 tx=0x7f6930009040 comp rx=0 tx=0).stop 2024-04-25T14:08:48.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.730+0000 7f694c600640 1 -- 172.21.15.86:0/4089351731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f694406fd50 msgr2=0x7f69440c3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:48.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.730+0000 7f694c600640 1 --2- 172.21.15.86:0/4089351731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f694406fd50 0x7f69440c3e00 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f693801aa90 tx=0x7f693801ac20 comp rx=0 tx=0).stop 2024-04-25T14:08:48.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.730+0000 7f694c600640 1 -- 172.21.15.86:0/4089351731 shutdown_connections 2024-04-25T14:08:48.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.730+0000 7f694c600640 1 --2- 172.21.15.86:0/4089351731 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f692406f520 0x7f6924071a30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:48.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.730+0000 7f694c600640 1 --2- 172.21.15.86:0/4089351731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f694415feb0 0x7f69440b2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:48.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.730+0000 7f694c600640 1 --2- 172.21.15.86:0/4089351731 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f694415c2a0 0x7f69440b24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:48.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.730+0000 7f694c600640 1 --2- 172.21.15.86:0/4089351731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f694406fd50 0x7f69440c3e00 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:48.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.730+0000 7f694c600640 1 -- 172.21.15.86:0/4089351731 >> 172.21.15.86:0/4089351731 conn(0x7f69440b4650 msgr2=0x7f6944154e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:48.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.730+0000 7f694c600640 1 -- 172.21.15.86:0/4089351731 shutdown_connections 2024-04-25T14:08:48.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.730+0000 7f694c600640 1 -- 172.21.15.86:0/4089351731 wait complete. 2024-04-25T14:08:48.757 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T14:08:48.757 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:08:48.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.906+0000 7f1254e00640 1 Processor -- start 2024-04-25T14:08:48.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.906+0000 7f1254e00640 1 -- start start 2024-04-25T14:08:48.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.910+0000 7f1254e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f12500cfca0 0x7f125006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:48.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.910+0000 7f1254e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1250072bd0 con 0x7f12500730b0 2024-04-25T14:08:48.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.910+0000 7f1254e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f125006f930 con 0x7f12500d16e0 2024-04-25T14:08:48.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.910+0000 7f1254e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1250070470 con 0x7f12500cfca0 2024-04-25T14:08:48.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.910+0000 7f124de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f12500cfca0 0x7f125006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:48.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.910+0000 7f124e800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f12500730b0 0x7f12500734c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49386/0 (socket says 172.21.15.86:49386) 2024-04-25T14:08:48.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.910+0000 7f124e800640 1 -- 172.21.15.86:0/506774337 learned_addr learned my addr 172.21.15.86:0/506774337 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:48.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.910+0000 7f124de00640 1 -- 172.21.15.86:0/506774337 >> v1:172.21.15.155:6789/0 conn(0x7f12500d16e0 legacy=0x7f12500d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:48.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.910+0000 7f124de00640 1 -- 172.21.15.86:0/506774337 >> v1:172.21.15.86:6789/0 conn(0x7f12500730b0 legacy=0x7f12500734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:48.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.910+0000 7f124de00640 1 -- 172.21.15.86:0/506774337 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125006fbd0 con 0x7f12500cfca0 2024-04-25T14:08:48.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.910+0000 7f124d400640 1 -- 172.21.15.86:0/506774337 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1250501776 0 0) 0x7f1250072bd0 con 0x7f12500730b0 2024-04-25T14:08:48.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.910+0000 7f124de00640 1 --2- 172.21.15.86:0/506774337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12500cfca0 0x7f125006f3f0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f12500d22a0 tx=0x7f124001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb3d00ee5732596 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:48.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.914+0000 7f124d400640 1 -- 172.21.15.86:0/506774337 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12400b2070 con 0x7f12500cfca0 2024-04-25T14:08:48.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.914+0000 7f124d400640 1 -- 172.21.15.86:0/506774337 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12400a58e0 con 0x7f12500cfca0 2024-04-25T14:08:48.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.914+0000 7f124d400640 1 -- 172.21.15.86:0/506774337 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12500743e0 con 0x7f12500cfca0 2024-04-25T14:08:48.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.914+0000 7f1254e00640 1 -- 172.21.15.86:0/506774337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12500cfca0 msgr2=0x7f125006f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:48.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.914+0000 7f1254e00640 1 --2- 172.21.15.86:0/506774337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12500cfca0 0x7f125006f3f0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f12500d22a0 tx=0x7f124001c910 comp rx=0 tx=0).stop 2024-04-25T14:08:48.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.914+0000 7f1254e00640 1 -- 172.21.15.86:0/506774337 shutdown_connections 2024-04-25T14:08:48.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.914+0000 7f1254e00640 1 --2- 172.21.15.86:0/506774337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12500cfca0 0x7f125006f3f0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:48.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.914+0000 7f1254e00640 1 -- 172.21.15.86:0/506774337 >> 172.21.15.86:0/506774337 conn(0x7f12500b4650 msgr2=0x7f12500b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:48.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.914+0000 7f1254e00640 1 -- 172.21.15.86:0/506774337 shutdown_connections 2024-04-25T14:08:48.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.914+0000 7f1254e00640 1 -- 172.21.15.86:0/506774337 wait complete. 2024-04-25T14:08:48.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.914+0000 7f1254e00640 1 Processor -- start 2024-04-25T14:08:48.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.914+0000 7f1254e00640 1 -- start start 2024-04-25T14:08:48.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.918+0000 7f1254e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12500730b0 0x7f1250099e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:48.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.918+0000 7f1254e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12500cfca0 0x7f125015aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:48.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.918+0000 7f1254e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12500d16e0 0x7f125015b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:48.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.918+0000 7f1254e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f125016aa80 con 0x7f12500d16e0 2024-04-25T14:08:48.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.918+0000 7f1254e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f125016a900 con 0x7f12500730b0 2024-04-25T14:08:48.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.918+0000 7f1254e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f125016ac00 con 0x7f12500cfca0 2024-04-25T14:08:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.918+0000 7f124e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12500d16e0 0x7f125015b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.918+0000 7f124e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12500d16e0 0x7f125015b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40262/0 (socket says 172.21.15.86:40262) 2024-04-25T14:08:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.918+0000 7f124e800640 1 -- 172.21.15.86:0/3458704612 learned_addr learned my addr 172.21.15.86:0/3458704612 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.918+0000 7f1245400640 1 --2- 172.21.15.86:0/3458704612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12500cfca0 0x7f125015aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.918+0000 7f124de00640 1 --2- 172.21.15.86:0/3458704612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12500730b0 0x7f1250099e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.918+0000 7f124e800640 1 -- 172.21.15.86:0/3458704612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12500cfca0 msgr2=0x7f125015aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.918+0000 7f124e800640 1 --2- 172.21.15.86:0/3458704612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12500cfca0 0x7f125015aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.918+0000 7f124e800640 1 -- 172.21.15.86:0/3458704612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12500730b0 msgr2=0x7f1250099e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.918+0000 7f124e800640 1 --2- 172.21.15.86:0/3458704612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12500730b0 0x7f1250099e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.918+0000 7f124e800640 1 -- 172.21.15.86:0/3458704612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12500cc1b0 con 0x7f12500d16e0 2024-04-25T14:08:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.918+0000 7f124e800640 1 --2- 172.21.15.86:0/3458704612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12500d16e0 0x7f125015b420 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f124800c410 tx=0x7f124800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.918+0000 7f1246a00640 1 -- 172.21.15.86:0/3458704612 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12480074e0 con 0x7f12500d16e0 2024-04-25T14:08:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.918+0000 7f1246a00640 1 -- 172.21.15.86:0/3458704612 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12480047f0 con 0x7f12500d16e0 2024-04-25T14:08:48.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.918+0000 7f1254e00640 1 -- 172.21.15.86:0/3458704612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f125015bc20 con 0x7f12500d16e0 2024-04-25T14:08:48.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.918+0000 7f1246a00640 1 -- 172.21.15.86:0/3458704612 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12480076a0 con 0x7f12500d16e0 2024-04-25T14:08:48.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.918+0000 7f1254e00640 1 -- 172.21.15.86:0/3458704612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f125015bec0 con 0x7f12500d16e0 2024-04-25T14:08:48.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.918+0000 7f1254e00640 1 -- 172.21.15.86:0/3458704612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f125009f4b0 con 0x7f12500d16e0 2024-04-25T14:08:48.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.922+0000 7f1246a00640 1 -- 172.21.15.86:0/3458704612 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7f1248011070 con 0x7f12500d16e0 2024-04-25T14:08:48.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.922+0000 7f1246a00640 1 --2- 172.21.15.86:0/3458704612 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f122c06f550 0x7f122c071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:48.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.922+0000 7f1246a00640 1 -- 172.21.15.86:0/3458704612 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f124808f010 con 0x7f12500d16e0 2024-04-25T14:08:48.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.922+0000 7f124de00640 1 --2- 172.21.15.86:0/3458704612 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f122c06f550 0x7f122c071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:48.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.922+0000 7f124de00640 1 --2- 172.21.15.86:0/3458704612 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f122c06f550 0x7f122c071a60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f12400af000 tx=0x7f12400a7070 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:48.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:48.926+0000 7f1246a00640 1 -- 172.21.15.86:0/3458704612 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f124805b570 con 0x7f12500d16e0 2024-04-25T14:08:49.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.114+0000 7f1254e00640 1 -- 172.21.15.86:0/3458704612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1250076bf0 con 0x7f12500d16e0 2024-04-25T14:08:49.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.118+0000 7f1246a00640 1 -- 172.21.15.86:0/3458704612 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5186 (secure 0 0 0) 0x7f124805f870 con 0x7f12500d16e0 2024-04-25T14:08:49.118 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:08:49.119 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7186,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/4265929298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4265929298},{"type":"v1","addr":"172.21.15.155:6833","nonce":4265929298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7189,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3254219000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3254219000},{"type":"v1","addr":"172.21.15.155:6835","nonce":3254219000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7195,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/193454084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":193454084},{"type":"v1","addr":"172.21.15.86:6837","nonce":193454084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:08:45.096500+0000","modified":"2024-04-25T14:08:48.121611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7371},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7371":{"gid":7371,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/1961103704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1961103704},{"type":"v1","addr":"172.21.15.86:6835","nonce":1961103704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7371,"qdb_cluster":[7371]},"id":6}]} 2024-04-25T14:08:49.119 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 67 2024-04-25T14:08:49.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.118+0000 7f1254e00640 1 -- 172.21.15.86:0/3458704612 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f122c06f550 msgr2=0x7f122c071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:49.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.118+0000 7f1254e00640 1 --2- 172.21.15.86:0/3458704612 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f122c06f550 0x7f122c071a60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f12400af000 tx=0x7f12400a7070 comp rx=0 tx=0).stop 2024-04-25T14:08:49.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.118+0000 7f1254e00640 1 -- 172.21.15.86:0/3458704612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12500d16e0 msgr2=0x7f125015b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:49.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.118+0000 7f1254e00640 1 --2- 172.21.15.86:0/3458704612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12500d16e0 0x7f125015b420 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f124800c410 tx=0x7f124800c810 comp rx=0 tx=0).stop 2024-04-25T14:08:49.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.118+0000 7f1254e00640 1 -- 172.21.15.86:0/3458704612 shutdown_connections 2024-04-25T14:08:49.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.118+0000 7f1254e00640 1 --2- 172.21.15.86:0/3458704612 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f122c06f550 0x7f122c071a60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:49.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.118+0000 7f1254e00640 1 --2- 172.21.15.86:0/3458704612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12500d16e0 0x7f125015b420 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:49.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.118+0000 7f1254e00640 1 --2- 172.21.15.86:0/3458704612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12500cfca0 0x7f125015aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:49.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.118+0000 7f1254e00640 1 --2- 172.21.15.86:0/3458704612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12500730b0 0x7f1250099e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:49.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.118+0000 7f1254e00640 1 -- 172.21.15.86:0/3458704612 >> 172.21.15.86:0/3458704612 conn(0x7f12500b4650 msgr2=0x7f1250154da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:49.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.118+0000 7f1254e00640 1 -- 172.21.15.86:0/3458704612 shutdown_connections 2024-04-25T14:08:49.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.118+0000 7f1254e00640 1 -- 172.21.15.86:0/3458704612 wait complete. 2024-04-25T14:08:49.140 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:08:49.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.286+0000 7fe6f2c00640 1 Processor -- start 2024-04-25T14:08:49.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.286+0000 7fe6f2c00640 1 -- start start 2024-04-25T14:08:49.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.286+0000 7fe6f2c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6ec06ef40 0x7fe6ec15c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:49.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.286+0000 7fe6f2c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe6ec0ae130 0x7fe6ec0ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:49.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.286+0000 7fe6f2c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6ec06d120 con 0x7fe6ec0d08c0 2024-04-25T14:08:49.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.286+0000 7fe6f2c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe6ec06d400 con 0x7fe6ec0ae130 2024-04-25T14:08:49.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.286+0000 7fe6f2c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe6ec06d6e0 con 0x7fe6ec06ef40 2024-04-25T14:08:49.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.286+0000 7fe6f2200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe6ec0ae130 0x7fe6ec0ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:49.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.286+0000 7fe6f2200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe6ec0ae130 0x7fe6ec0ae540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40274/0 (socket says 172.21.15.86:40274) 2024-04-25T14:08:49.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.286+0000 7fe6f2200640 1 -- 172.21.15.86:0/4182442746 learned_addr learned my addr 172.21.15.86:0/4182442746 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:49.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.286+0000 7fe6f0e00640 1 --2- 172.21.15.86:0/4182442746 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6ec06ef40 0x7fe6ec15c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:49.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.286+0000 7fe6f2200640 1 -- 172.21.15.86:0/4182442746 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6ec06ef40 msgr2=0x7fe6ec15c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:49.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.286+0000 7fe6f2200640 1 --2- 172.21.15.86:0/4182442746 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6ec06ef40 0x7fe6ec15c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:49.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.286+0000 7fe6f2200640 1 -- 172.21.15.86:0/4182442746 >> v1:172.21.15.155:6789/0 conn(0x7fe6ec0d08c0 legacy=0x7fe6ec06e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:49.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.286+0000 7fe6f2200640 1 -- 172.21.15.86:0/4182442746 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6ec06d9c0 con 0x7fe6ec0ae130 2024-04-25T14:08:49.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.290+0000 7fe6f2200640 1 --2- 172.21.15.86:0/4182442746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6ec0ae130 0x7fe6ec0ae540 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4009e50 tx=0x7fe6e401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dfad1fd83f5cc2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:49.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.290+0000 7fe6ebe00640 1 -- 172.21.15.86:0/4182442746 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e40a5700 con 0x7fe6ec0ae130 2024-04-25T14:08:49.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.290+0000 7fe6ebe00640 1 -- 172.21.15.86:0/4182442746 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6e40a58a0 con 0x7fe6ec0ae130 2024-04-25T14:08:49.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.290+0000 7fe6ebe00640 1 -- 172.21.15.86:0/4182442746 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e40a5c70 con 0x7fe6ec0ae130 2024-04-25T14:08:49.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.290+0000 7fe6f2c00640 1 -- 172.21.15.86:0/4182442746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6ec0ae130 msgr2=0x7fe6ec0ae540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:49.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.290+0000 7fe6f2c00640 1 --2- 172.21.15.86:0/4182442746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6ec0ae130 0x7fe6ec0ae540 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4009e50 tx=0x7fe6e401ca30 comp rx=0 tx=0).stop 2024-04-25T14:08:49.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.290+0000 7fe6f2c00640 1 -- 172.21.15.86:0/4182442746 shutdown_connections 2024-04-25T14:08:49.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.290+0000 7fe6f2c00640 1 --2- 172.21.15.86:0/4182442746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6ec0ae130 0x7fe6ec0ae540 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:49.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.290+0000 7fe6f2c00640 1 --2- 172.21.15.86:0/4182442746 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6ec06ef40 0x7fe6ec15c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:49.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.290+0000 7fe6f2c00640 1 -- 172.21.15.86:0/4182442746 >> 172.21.15.86:0/4182442746 conn(0x7fe6ec0b4850 msgr2=0x7fe6ec0b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:49.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.290+0000 7fe6f2c00640 1 -- 172.21.15.86:0/4182442746 shutdown_connections 2024-04-25T14:08:49.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.290+0000 7fe6f2c00640 1 -- 172.21.15.86:0/4182442746 wait complete. 2024-04-25T14:08:49.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.294+0000 7fe6f2c00640 1 Processor -- start 2024-04-25T14:08:49.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.294+0000 7fe6f2c00640 1 -- start start 2024-04-25T14:08:49.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.294+0000 7fe6f2c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6ec0ae130 0x7fe6ec072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:49.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.294+0000 7fe6f2c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6ec0d08c0 0x7fe6ec154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:49.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.294+0000 7fe6f2c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6ec157210 0x7fe6ec0b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:49.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.294+0000 7fe6f2c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec169700 con 0x7fe6ec0d08c0 2024-04-25T14:08:49.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.294+0000 7fe6f2c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec169580 con 0x7fe6ec157210 2024-04-25T14:08:49.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.294+0000 7fe6f2c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe6ec169880 con 0x7fe6ec0ae130 2024-04-25T14:08:49.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.294+0000 7fe6f0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6ec0d08c0 0x7fe6ec154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:49.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.294+0000 7fe6f0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6ec0d08c0 0x7fe6ec154c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40282/0 (socket says 172.21.15.86:40282) 2024-04-25T14:08:49.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.294+0000 7fe6f0e00640 1 -- 172.21.15.86:0/1211018361 learned_addr learned my addr 172.21.15.86:0/1211018361 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:49.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.294+0000 7fe6f1800640 1 --2- 172.21.15.86:0/1211018361 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6ec0ae130 0x7fe6ec072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:49.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.294+0000 7fe6f2200640 1 --2- 172.21.15.86:0/1211018361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6ec157210 0x7fe6ec0b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:49.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.298+0000 7fe6f0e00640 1 -- 172.21.15.86:0/1211018361 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6ec0ae130 msgr2=0x7fe6ec072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:49.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.298+0000 7fe6f0e00640 1 --2- 172.21.15.86:0/1211018361 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6ec0ae130 0x7fe6ec072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:49.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.298+0000 7fe6f0e00640 1 -- 172.21.15.86:0/1211018361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6ec157210 msgr2=0x7fe6ec0b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:49.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.298+0000 7fe6f0e00640 1 --2- 172.21.15.86:0/1211018361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6ec157210 0x7fe6ec0b0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:49.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.298+0000 7fe6f0e00640 1 -- 172.21.15.86:0/1211018361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6ec073340 con 0x7fe6ec0d08c0 2024-04-25T14:08:49.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.298+0000 7fe6f0e00640 1 --2- 172.21.15.86:0/1211018361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6ec0d08c0 0x7fe6ec154c60 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fe6d800d620 tx=0x7fe6d800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:49.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.298+0000 7fe6e9600640 1 -- 172.21.15.86:0/1211018361 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d800b620 con 0x7fe6ec0d08c0 2024-04-25T14:08:49.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.298+0000 7fe6e9600640 1 -- 172.21.15.86:0/1211018361 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6d8019070 con 0x7fe6ec0d08c0 2024-04-25T14:08:49.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.298+0000 7fe6f2c00640 1 -- 172.21.15.86:0/1211018361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6ec073620 con 0x7fe6ec0d08c0 2024-04-25T14:08:49.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.298+0000 7fe6e9600640 1 -- 172.21.15.86:0/1211018361 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d800b9e0 con 0x7fe6ec0d08c0 2024-04-25T14:08:49.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.298+0000 7fe6f2c00640 1 -- 172.21.15.86:0/1211018361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6ec15fdd0 con 0x7fe6ec0d08c0 2024-04-25T14:08:49.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.298+0000 7fe6f2c00640 1 -- 172.21.15.86:0/1211018361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6ec076b80 con 0x7fe6ec0d08c0 2024-04-25T14:08:49.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.298+0000 7fe6e9600640 1 -- 172.21.15.86:0/1211018361 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7fe6d800bb80 con 0x7fe6ec0d08c0 2024-04-25T14:08:49.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.298+0000 7fe6e9600640 1 --2- 172.21.15.86:0/1211018361 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fe6cc06f550 0x7fe6cc071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:49.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.298+0000 7fe6e9600640 1 -- 172.21.15.86:0/1211018361 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7fe6d808f8b0 con 0x7fe6ec0d08c0 2024-04-25T14:08:49.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.302+0000 7fe6f1800640 1 --2- 172.21.15.86:0/1211018361 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fe6cc06f550 0x7fe6cc071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:49.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.302+0000 7fe6f1800640 1 --2- 172.21.15.86:0/1211018361 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fe6cc06f550 0x7fe6cc071a60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec0b0d90 tx=0x7fe6e0009040 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:49.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.306+0000 7fe6e9600640 1 -- 172.21.15.86:0/1211018361 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe6d805bea0 con 0x7fe6ec0d08c0 2024-04-25T14:08:49.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.462+0000 7fe6f2c00640 1 -- 172.21.15.86:0/1211018361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6ec0730d0 con 0x7fe6ec0d08c0 2024-04-25T14:08:49.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.462+0000 7fe6e9600640 1 -- 172.21.15.86:0/1211018361 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+16885 (secure 0 0 0) 0x7fe6d80601a0 con 0x7fe6ec0d08c0 2024-04-25T14:08:49.466 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:08:49.466 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":87,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:08:46.714794+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:08:43.505988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T14:08:44.054628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:08:49.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.466+0000 7fe6f2c00640 1 -- 172.21.15.86:0/1211018361 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fe6cc06f550 msgr2=0x7fe6cc071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:49.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.466+0000 7fe6f2c00640 1 --2- 172.21.15.86:0/1211018361 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fe6cc06f550 0x7fe6cc071a60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec0b0d90 tx=0x7fe6e0009040 comp rx=0 tx=0).stop 2024-04-25T14:08:49.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.466+0000 7fe6f2c00640 1 -- 172.21.15.86:0/1211018361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6ec0d08c0 msgr2=0x7fe6ec154c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:49.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.466+0000 7fe6f2c00640 1 --2- 172.21.15.86:0/1211018361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6ec0d08c0 0x7fe6ec154c60 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fe6d800d620 tx=0x7fe6d800dae0 comp rx=0 tx=0).stop 2024-04-25T14:08:49.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.466+0000 7fe6f2c00640 1 -- 172.21.15.86:0/1211018361 shutdown_connections 2024-04-25T14:08:49.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.466+0000 7fe6f2c00640 1 --2- 172.21.15.86:0/1211018361 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fe6cc06f550 0x7fe6cc071a60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:49.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.466+0000 7fe6f2c00640 1 --2- 172.21.15.86:0/1211018361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6ec157210 0x7fe6ec0b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:49.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.466+0000 7fe6f2c00640 1 --2- 172.21.15.86:0/1211018361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6ec0d08c0 0x7fe6ec154c60 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:49.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.466+0000 7fe6f2c00640 1 --2- 172.21.15.86:0/1211018361 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6ec0ae130 0x7fe6ec072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:49.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.466+0000 7fe6f2c00640 1 -- 172.21.15.86:0/1211018361 >> 172.21.15.86:0/1211018361 conn(0x7fe6ec0b4850 msgr2=0x7fe6ec0cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:49.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.466+0000 7fe6f2c00640 1 -- 172.21.15.86:0/1211018361 shutdown_connections 2024-04-25T14:08:49.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.466+0000 7fe6f2c00640 1 -- 172.21.15.86:0/1211018361 wait complete. 2024-04-25T14:08:49.469 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:08:49.472 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:08:49.488 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T14:08:49.488 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:08:49.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.702+0000 7f2ade600640 1 Processor -- start 2024-04-25T14:08:49.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.702+0000 7f2ade600640 1 -- start start 2024-04-25T14:08:49.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.702+0000 7f2ade600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2ad80cf970 0x7f2ad8070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:49.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.702+0000 7f2ade600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ad806e970 0x7f2ad80cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:49.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.702+0000 7f2ade600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ad8071770 con 0x7f2ad8071360 2024-04-25T14:08:49.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.702+0000 7f2ade600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f2ad8163830 con 0x7f2ad80cf970 2024-04-25T14:08:49.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.702+0000 7f2ade600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2ad806ef90 con 0x7f2ad806e970 2024-04-25T14:08:49.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.702+0000 7f2acfe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2ad8071360 0x7f2ad806e260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49388/0 (socket says 172.21.15.86:49388) 2024-04-25T14:08:49.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.702+0000 7f2acfe00640 1 -- 172.21.15.86:0/157785773 learned_addr learned my addr 172.21.15.86:0/157785773 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:49.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.702+0000 7f2ad7e00640 1 --2- 172.21.15.86:0/157785773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ad806e970 0x7f2ad80cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:49.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.702+0000 7f2ad7400640 1 --2- 172.21.15.86:0/157785773 >> v2:172.21.15.155:3300/0 conn(0x7f2ad80cf970 0x7f2ad8070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:49.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.702+0000 7f2ad6a00640 1 -- 172.21.15.86:0/157785773 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2261188924 0 0) 0x7f2ac00012a0 con 0x7f2ad8071360 2024-04-25T14:08:49.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.706+0000 7f2ad6a00640 1 -- 172.21.15.86:0/157785773 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2abc003530 con 0x7f2ad8071360 2024-04-25T14:08:49.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.706+0000 7f2ad6a00640 1 -- 172.21.15.86:0/157785773 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2424914923 0 0) 0x7f2abc003530 con 0x7f2ad8071360 2024-04-25T14:08:49.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.706+0000 7f2ad7400640 1 -- 172.21.15.86:0/157785773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ad806e970 msgr2=0x7f2ad80cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:49.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.706+0000 7f2ad7400640 1 --2- 172.21.15.86:0/157785773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ad806e970 0x7f2ad80cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:49.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.706+0000 7f2ad7400640 1 -- 172.21.15.86:0/157785773 >> v1:172.21.15.86:6789/0 conn(0x7f2ad8071360 legacy=0x7f2ad806e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:49.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.706+0000 7f2ad7400640 1 -- 172.21.15.86:0/157785773 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad8163a80 con 0x7f2ad80cf970 2024-04-25T14:08:49.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.706+0000 7f2ad7e00640 1 --2- 172.21.15.86:0/157785773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ad806e970 0x7f2ad80cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:08:49.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.706+0000 7f2ad7400640 1 --2- 172.21.15.86:0/157785773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ad80cf970 0x7f2ad8070e20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2ac80097c0 tx=0x7f2ac801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae169ba6e4323309 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:49.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.706+0000 7f2ad6a00640 1 -- 172.21.15.86:0/157785773 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac80b0070 con 0x7f2ad80cf970 2024-04-25T14:08:49.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.706+0000 7f2ad6a00640 1 -- 172.21.15.86:0/157785773 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac80ab400 con 0x7f2ad80cf970 2024-04-25T14:08:49.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.706+0000 7f2ad6a00640 1 -- 172.21.15.86:0/157785773 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac00019d0 con 0x7f2ad80cf970 2024-04-25T14:08:49.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.710+0000 7f2ade600640 1 -- 172.21.15.86:0/157785773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ad80cf970 msgr2=0x7f2ad8070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:49.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.710+0000 7f2ade600640 1 --2- 172.21.15.86:0/157785773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ad80cf970 0x7f2ad8070e20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2ac80097c0 tx=0x7f2ac801c3c0 comp rx=0 tx=0).stop 2024-04-25T14:08:49.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.710+0000 7f2ade600640 1 -- 172.21.15.86:0/157785773 shutdown_connections 2024-04-25T14:08:49.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.710+0000 7f2ade600640 1 --2- 172.21.15.86:0/157785773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ad806e970 0x7f2ad80cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:49.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.710+0000 7f2ade600640 1 --2- 172.21.15.86:0/157785773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ad80cf970 0x7f2ad8070e20 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:49.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.710+0000 7f2ade600640 1 -- 172.21.15.86:0/157785773 >> 172.21.15.86:0/157785773 conn(0x7f2ad80cffc0 msgr2=0x7f2ad80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:49.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.710+0000 7f2ade600640 1 -- 172.21.15.86:0/157785773 shutdown_connections 2024-04-25T14:08:49.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.710+0000 7f2ade600640 1 -- 172.21.15.86:0/157785773 wait complete. 2024-04-25T14:08:49.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.710+0000 7f2ade600640 1 Processor -- start 2024-04-25T14:08:49.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.710+0000 7f2ade600640 1 -- start start 2024-04-25T14:08:49.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.714+0000 7f2ade600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ad806e970 0x7f2ad8155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:49.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.714+0000 7f2ade600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ad8071360 0x7f2ad81f9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:49.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.714+0000 7f2ade600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ad80cf970 0x7f2ad80b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:49.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.714+0000 7f2ade600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad816cbd0 con 0x7f2ad8071360 2024-04-25T14:08:49.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.714+0000 7f2ade600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad80ced80 con 0x7f2ad80cf970 2024-04-25T14:08:49.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.714+0000 7f2ade600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2ad816cd50 con 0x7f2ad806e970 2024-04-25T14:08:49.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.714+0000 7f2acfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ad8071360 0x7f2ad81f9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:49.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.714+0000 7f2acfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ad8071360 0x7f2ad81f9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40284/0 (socket says 172.21.15.86:40284) 2024-04-25T14:08:49.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.714+0000 7f2acfe00640 1 -- 172.21.15.86:0/144806473 learned_addr learned my addr 172.21.15.86:0/144806473 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:49.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.714+0000 7f2ad7400640 1 --2- 172.21.15.86:0/144806473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ad806e970 0x7f2ad8155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:49.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.714+0000 7f2ad7e00640 1 --2- 172.21.15.86:0/144806473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ad80cf970 0x7f2ad80b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:49.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.714+0000 7f2acfe00640 1 -- 172.21.15.86:0/144806473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ad806e970 msgr2=0x7f2ad8155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:49.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.714+0000 7f2acfe00640 1 --2- 172.21.15.86:0/144806473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ad806e970 0x7f2ad8155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:49.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.714+0000 7f2acfe00640 1 -- 172.21.15.86:0/144806473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ad80cf970 msgr2=0x7f2ad80b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:49.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.714+0000 7f2acfe00640 1 --2- 172.21.15.86:0/144806473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ad80cf970 0x7f2ad80b68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:49.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.714+0000 7f2acfe00640 1 -- 172.21.15.86:0/144806473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad80b7d70 con 0x7f2ad8071360 2024-04-25T14:08:49.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.714+0000 7f2acfe00640 1 --2- 172.21.15.86:0/144806473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ad8071360 0x7f2ad81f9cf0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000c7b0 tx=0x7f2ac000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:49.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.714+0000 7f2acf400640 1 -- 172.21.15.86:0/144806473 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0007270 con 0x7f2ad8071360 2024-04-25T14:08:49.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.714+0000 7f2acf400640 1 -- 172.21.15.86:0/144806473 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac0007430 con 0x7f2ad8071360 2024-04-25T14:08:49.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.714+0000 7f2ade600640 1 -- 172.21.15.86:0/144806473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad80b8050 con 0x7f2ad8071360 2024-04-25T14:08:49.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.714+0000 7f2acf400640 1 -- 172.21.15.86:0/144806473 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac00059f0 con 0x7f2ad8071360 2024-04-25T14:08:49.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.714+0000 7f2ade600640 1 -- 172.21.15.86:0/144806473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad81fa230 con 0x7f2ad8071360 2024-04-25T14:08:49.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.714+0000 7f2ade600640 1 -- 172.21.15.86:0/144806473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ad8156520 con 0x7f2ad8071360 2024-04-25T14:08:49.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.718+0000 7f2acf400640 1 -- 172.21.15.86:0/144806473 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7f2ac0005b90 con 0x7f2ad8071360 2024-04-25T14:08:49.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.718+0000 7f2acf400640 1 --2- 172.21.15.86:0/144806473 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f2ab406f520 0x7f2ab4071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:49.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.718+0000 7f2acf400640 1 -- 172.21.15.86:0/144806473 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f2ac008f260 con 0x7f2ad8071360 2024-04-25T14:08:49.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.718+0000 7f2ad7400640 1 --2- 172.21.15.86:0/144806473 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f2ab406f520 0x7f2ab4071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:49.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.718+0000 7f2ad7400640 1 --2- 172.21.15.86:0/144806473 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f2ab406f520 0x7f2ab4071a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2ac80ad000 tx=0x7f2ac8008040 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:49.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.722+0000 7f2acf400640 1 -- 172.21.15.86:0/144806473 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2ad8071770 con 0x7f2ad8071360 2024-04-25T14:08:49.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.878+0000 7f2ade600640 1 -- 172.21.15.86:0/144806473 --> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2ad80b7b20 con 0x7f2ab406f520 2024-04-25T14:08:49.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.882+0000 7f2acf400640 1 -- 172.21.15.86:0/144806473 <== mgr.7092 v2:172.21.15.86:6832/3589556609 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246763 (secure 0 0 0) 0x7f2ac80f7400 con 0x7f2ab406f520 2024-04-25T14:08:49.888 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:08:49.889 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:08:49.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.886+0000 7f2ade600640 1 -- 172.21.15.86:0/144806473 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f2ab406f520 msgr2=0x7f2ab4071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:49.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.886+0000 7f2ade600640 1 --2- 172.21.15.86:0/144806473 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f2ab406f520 0x7f2ab4071a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2ac80ad000 tx=0x7f2ac8008040 comp rx=0 tx=0).stop 2024-04-25T14:08:49.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.886+0000 7f2ade600640 1 -- 172.21.15.86:0/144806473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ad8071360 msgr2=0x7f2ad81f9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:49.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.886+0000 7f2ade600640 1 --2- 172.21.15.86:0/144806473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ad8071360 0x7f2ad81f9cf0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000c7b0 tx=0x7f2ac000cc70 comp rx=0 tx=0).stop 2024-04-25T14:08:49.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.886+0000 7f2ade600640 1 -- 172.21.15.86:0/144806473 shutdown_connections 2024-04-25T14:08:49.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.886+0000 7f2ade600640 1 --2- 172.21.15.86:0/144806473 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f2ab406f520 0x7f2ab4071a30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:49.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.886+0000 7f2ade600640 1 --2- 172.21.15.86:0/144806473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ad80cf970 0x7f2ad80b68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:49.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.886+0000 7f2ade600640 1 --2- 172.21.15.86:0/144806473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ad8071360 0x7f2ad81f9cf0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:49.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.886+0000 7f2ade600640 1 --2- 172.21.15.86:0/144806473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ad806e970 0x7f2ad8155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:49.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.886+0000 7f2ade600640 1 -- 172.21.15.86:0/144806473 >> 172.21.15.86:0/144806473 conn(0x7f2ad80cffc0 msgr2=0x7f2ad81596a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:49.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.886+0000 7f2ade600640 1 -- 172.21.15.86:0/144806473 shutdown_connections 2024-04-25T14:08:49.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:49.886+0000 7f2ade600640 1 -- 172.21.15.86:0/144806473 wait complete. 2024-04-25T14:08:49.910 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T14:08:49.359300+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":306,"num_read_kb":759,"num_write":149,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258792,"kb_used_data":9328,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715928,"statfs":{"total":773094113280,"available":772829110272,"internally_reserved":0,"allocated":9551872,"data_stored":5326762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.671432"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.700729+0000","last_change":"2024-04-25T14:08:44.700729+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.700729+0000","last_undegraded":"2024-04-25T14:08:44.700729+0000","last_fullsized":"2024-04-25T14:08:44.700729+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.695217+0000","last_change":"2024-04-25T14:08:44.695217+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.695217+0000","last_undegraded":"2024-04-25T14:08:44.695217+0000","last_fullsized":"2024-04-25T14:08:44.695217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.700715+0000","last_change":"2024-04-25T14:08:44.700715+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.700715+0000","last_undegraded":"2024-04-25T14:08:44.700715+0000","last_fullsized":"2024-04-25T14:08:44.700715+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.707535+0000","last_change":"2024-04-25T14:08:44.707535+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.707535+0000","last_undegraded":"2024-04-25T14:08:44.707535+0000","last_fullsized":"2024-04-25T14:08:44.707535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.696533+0000","last_change":"2024-04-25T14:08:44.696533+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.696533+0000","last_undegraded":"2024-04-25T14:08:44.696533+0000","last_fullsized":"2024-04-25T14:08:44.696533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.715602+0000","last_change":"2024-04-25T14:08:44.715602+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.715602+0000","last_undegraded":"2024-04-25T14:08:44.715602+0000","last_fullsized":"2024-04-25T14:08:44.715602+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.705515+0000","last_change":"2024-04-25T14:08:44.705515+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.705515+0000","last_undegraded":"2024-04-25T14:08:44.705515+0000","last_fullsized":"2024-04-25T14:08:44.705515+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.704048+0000","last_change":"2024-04-25T14:08:44.704048+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.704048+0000","last_undegraded":"2024-04-25T14:08:44.704048+0000","last_fullsized":"2024-04-25T14:08:44.704048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.701272+0000","last_change":"2024-04-25T14:08:44.701272+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.701272+0000","last_undegraded":"2024-04-25T14:08:44.701272+0000","last_fullsized":"2024-04-25T14:08:44.701272+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.709510+0000","last_change":"2024-04-25T14:08:44.709510+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.709510+0000","last_undegraded":"2024-04-25T14:08:44.709510+0000","last_fullsized":"2024-04-25T14:08:44.709510+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.699606+0000","last_change":"2024-04-25T14:08:44.699606+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.699606+0000","last_undegraded":"2024-04-25T14:08:44.699606+0000","last_fullsized":"2024-04-25T14:08:44.699606+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.711227+0000","last_change":"2024-04-25T14:08:44.711227+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.711227+0000","last_undegraded":"2024-04-25T14:08:44.711227+0000","last_fullsized":"2024-04-25T14:08:44.711227+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.702941+0000","last_change":"2024-04-25T14:08:44.702941+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.702941+0000","last_undegraded":"2024-04-25T14:08:44.702941+0000","last_fullsized":"2024-04-25T14:08:44.702941+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.699462+0000","last_change":"2024-04-25T14:08:44.699462+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.699462+0000","last_undegraded":"2024-04-25T14:08:44.699462+0000","last_fullsized":"2024-04-25T14:08:44.699462+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.702169+0000","last_change":"2024-04-25T14:08:44.702169+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.702169+0000","last_undegraded":"2024-04-25T14:08:44.702169+0000","last_fullsized":"2024-04-25T14:08:44.702169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.708333+0000","last_change":"2024-04-25T14:08:44.708333+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.708333+0000","last_undegraded":"2024-04-25T14:08:44.708333+0000","last_fullsized":"2024-04-25T14:08:44.708333+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.704663+0000","last_change":"2024-04-25T14:08:44.704663+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.704663+0000","last_undegraded":"2024-04-25T14:08:44.704663+0000","last_fullsized":"2024-04-25T14:08:44.704663+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.700154+0000","last_change":"2024-04-25T14:08:44.700154+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.700154+0000","last_undegraded":"2024-04-25T14:08:44.700154+0000","last_fullsized":"2024-04-25T14:08:44.700154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.699078+0000","last_change":"2024-04-25T14:08:44.699078+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.699078+0000","last_undegraded":"2024-04-25T14:08:44.699078+0000","last_fullsized":"2024-04-25T14:08:44.699078+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.707069+0000","last_change":"2024-04-25T14:08:44.707069+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.707069+0000","last_undegraded":"2024-04-25T14:08:44.707069+0000","last_fullsized":"2024-04-25T14:08:44.707069+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.700474+0000","last_change":"2024-04-25T14:08:44.700474+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.700474+0000","last_undegraded":"2024-04-25T14:08:44.700474+0000","last_fullsized":"2024-04-25T14:08:44.700474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.698892+0000","last_change":"2024-04-25T14:08:44.698892+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.698892+0000","last_undegraded":"2024-04-25T14:08:44.698892+0000","last_fullsized":"2024-04-25T14:08:44.698892+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.702950+0000","last_change":"2024-04-25T14:08:44.702950+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.702950+0000","last_undegraded":"2024-04-25T14:08:44.702950+0000","last_fullsized":"2024-04-25T14:08:44.702950+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.698720+0000","last_change":"2024-04-25T14:08:44.698720+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.698720+0000","last_undegraded":"2024-04-25T14:08:44.698720+0000","last_fullsized":"2024-04-25T14:08:44.698720+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.696352+0000","last_change":"2024-04-25T14:08:44.696352+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.696352+0000","last_undegraded":"2024-04-25T14:08:44.696352+0000","last_fullsized":"2024-04-25T14:08:44.696352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.700875+0000","last_change":"2024-04-25T14:08:44.700875+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.700875+0000","last_undegraded":"2024-04-25T14:08:44.700875+0000","last_fullsized":"2024-04-25T14:08:44.700875+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.709537+0000","last_change":"2024-04-25T14:08:44.709537+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.709537+0000","last_undegraded":"2024-04-25T14:08:44.709537+0000","last_fullsized":"2024-04-25T14:08:44.709537+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.702856+0000","last_change":"2024-04-25T14:08:44.702856+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.702856+0000","last_undegraded":"2024-04-25T14:08:44.702856+0000","last_fullsized":"2024-04-25T14:08:44.702856+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.704801+0000","last_change":"2024-04-25T14:08:44.704801+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.704801+0000","last_undegraded":"2024-04-25T14:08:44.704801+0000","last_fullsized":"2024-04-25T14:08:44.704801+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.695641+0000","last_change":"2024-04-25T14:08:44.695641+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.695641+0000","last_undegraded":"2024-04-25T14:08:44.695641+0000","last_fullsized":"2024-04-25T14:08:44.695641+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.695624+0000","last_change":"2024-04-25T14:08:44.695624+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.695624+0000","last_undegraded":"2024-04-25T14:08:44.695624+0000","last_fullsized":"2024-04-25T14:08:44.695624+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.707878+0000","last_change":"2024-04-25T14:08:44.707878+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.707878+0000","last_undegraded":"2024-04-25T14:08:44.707878+0000","last_fullsized":"2024-04-25T14:08:44.707878+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.701936+0000","last_change":"2024-04-25T14:08:44.701936+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.701936+0000","last_undegraded":"2024-04-25T14:08:44.701936+0000","last_fullsized":"2024-04-25T14:08:44.701936+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.728427+0000","last_change":"2024-04-25T14:08:44.725130+0000","last_active":"2024-04-25T14:08:44.728427+0000","last_peered":"2024-04-25T14:08:44.728427+0000","last_clean":"2024-04-25T14:08:44.728427+0000","last_became_active":"2024-04-25T14:08:44.724805+0000","last_became_peered":"2024-04-25T14:08:44.724805+0000","last_unstale":"2024-04-25T14:08:44.728427+0000","last_undegraded":"2024-04-25T14:08:44.728427+0000","last_fullsized":"2024-04-25T14:08:44.728427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:58.834978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.713037+0000","last_change":"2024-04-25T14:08:44.713037+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.713037+0000","last_undegraded":"2024-04-25T14:08:44.713037+0000","last_fullsized":"2024-04-25T14:08:44.713037+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.714929+0000","last_change":"2024-04-25T14:08:44.713248+0000","last_active":"2024-04-25T14:08:44.714929+0000","last_peered":"2024-04-25T14:08:44.714929+0000","last_clean":"2024-04-25T14:08:44.714929+0000","last_became_active":"2024-04-25T14:08:44.712237+0000","last_became_peered":"2024-04-25T14:08:44.712237+0000","last_unstale":"2024-04-25T14:08:44.714929+0000","last_undegraded":"2024-04-25T14:08:44.714929+0000","last_fullsized":"2024-04-25T14:08:44.714929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:14.643131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.704564+0000","last_change":"2024-04-25T14:08:44.704564+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.704564+0000","last_undegraded":"2024-04-25T14:08:44.704564+0000","last_fullsized":"2024-04-25T14:08:44.704564+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.728516+0000","last_change":"2024-04-25T14:08:44.725153+0000","last_active":"2024-04-25T14:08:44.728516+0000","last_peered":"2024-04-25T14:08:44.728516+0000","last_clean":"2024-04-25T14:08:44.728516+0000","last_became_active":"2024-04-25T14:08:44.724848+0000","last_became_peered":"2024-04-25T14:08:44.724848+0000","last_unstale":"2024-04-25T14:08:44.728516+0000","last_undegraded":"2024-04-25T14:08:44.728516+0000","last_fullsized":"2024-04-25T14:08:44.728516+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:28.574820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.704170+0000","last_change":"2024-04-25T14:08:44.704170+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.704170+0000","last_undegraded":"2024-04-25T14:08:44.704170+0000","last_fullsized":"2024-04-25T14:08:44.704170+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.721669+0000","last_change":"2024-04-25T14:08:44.720784+0000","last_active":"2024-04-25T14:08:44.721669+0000","last_peered":"2024-04-25T14:08:44.721669+0000","last_clean":"2024-04-25T14:08:44.721669+0000","last_became_active":"2024-04-25T14:08:44.720248+0000","last_became_peered":"2024-04-25T14:08:44.720248+0000","last_unstale":"2024-04-25T14:08:44.721669+0000","last_undegraded":"2024-04-25T14:08:44.721669+0000","last_fullsized":"2024-04-25T14:08:44.721669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:00.497301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.701864+0000","last_change":"2024-04-25T14:08:44.701864+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.701864+0000","last_undegraded":"2024-04-25T14:08:44.701864+0000","last_fullsized":"2024-04-25T14:08:44.701864+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.730802+0000","last_change":"2024-04-25T14:08:44.727021+0000","last_active":"2024-04-25T14:08:44.730802+0000","last_peered":"2024-04-25T14:08:44.730802+0000","last_clean":"2024-04-25T14:08:44.730802+0000","last_became_active":"2024-04-25T14:08:44.726079+0000","last_became_peered":"2024-04-25T14:08:44.726079+0000","last_unstale":"2024-04-25T14:08:44.730802+0000","last_undegraded":"2024-04-25T14:08:44.730802+0000","last_fullsized":"2024-04-25T14:08:44.730802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:18.045986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.701896+0000","last_change":"2024-04-25T14:08:44.701896+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.701896+0000","last_undegraded":"2024-04-25T14:08:44.701896+0000","last_fullsized":"2024-04-25T14:08:44.701896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.728800+0000","last_change":"2024-04-25T14:08:44.726268+0000","last_active":"2024-04-25T14:08:44.728800+0000","last_peered":"2024-04-25T14:08:44.728800+0000","last_clean":"2024-04-25T14:08:44.728800+0000","last_became_active":"2024-04-25T14:08:44.725839+0000","last_became_peered":"2024-04-25T14:08:44.725839+0000","last_unstale":"2024-04-25T14:08:44.728800+0000","last_undegraded":"2024-04-25T14:08:44.728800+0000","last_fullsized":"2024-04-25T14:08:44.728800+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:01.190413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.711108+0000","last_change":"2024-04-25T14:08:44.711108+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.711108+0000","last_undegraded":"2024-04-25T14:08:44.711108+0000","last_fullsized":"2024-04-25T14:08:44.711108+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.717869+0000","last_change":"2024-04-25T14:08:44.712147+0000","last_active":"2024-04-25T14:08:44.717869+0000","last_peered":"2024-04-25T14:08:44.717869+0000","last_clean":"2024-04-25T14:08:44.717869+0000","last_became_active":"2024-04-25T14:08:44.711799+0000","last_became_peered":"2024-04-25T14:08:44.711799+0000","last_unstale":"2024-04-25T14:08:44.717869+0000","last_undegraded":"2024-04-25T14:08:44.717869+0000","last_fullsized":"2024-04-25T14:08:44.717869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:28.574820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.702431+0000","last_change":"2024-04-25T14:08:44.702431+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.702431+0000","last_undegraded":"2024-04-25T14:08:44.702431+0000","last_fullsized":"2024-04-25T14:08:44.702431+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.726027+0000","last_change":"2024-04-25T14:08:44.723883+0000","last_active":"2024-04-25T14:08:44.726027+0000","last_peered":"2024-04-25T14:08:44.726027+0000","last_clean":"2024-04-25T14:08:44.726027+0000","last_became_active":"2024-04-25T14:08:44.723229+0000","last_became_peered":"2024-04-25T14:08:44.723229+0000","last_unstale":"2024-04-25T14:08:44.726027+0000","last_undegraded":"2024-04-25T14:08:44.726027+0000","last_fullsized":"2024-04-25T14:08:44.726027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:45.472144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.697949+0000","last_change":"2024-04-25T14:08:44.697949+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.697949+0000","last_undegraded":"2024-04-25T14:08:44.697949+0000","last_fullsized":"2024-04-25T14:08:44.697949+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.726787+0000","last_change":"2024-04-25T14:08:44.723981+0000","last_active":"2024-04-25T14:08:44.726787+0000","last_peered":"2024-04-25T14:08:44.726787+0000","last_clean":"2024-04-25T14:08:44.726787+0000","last_became_active":"2024-04-25T14:08:44.723657+0000","last_became_peered":"2024-04-25T14:08:44.723657+0000","last_unstale":"2024-04-25T14:08:44.726787+0000","last_undegraded":"2024-04-25T14:08:44.726787+0000","last_fullsized":"2024-04-25T14:08:44.726787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:51.117327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.705995+0000","last_change":"2024-04-25T14:08:44.705995+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.705995+0000","last_undegraded":"2024-04-25T14:08:44.705995+0000","last_fullsized":"2024-04-25T14:08:44.705995+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.720640+0000","last_change":"2024-04-25T14:08:44.718626+0000","last_active":"2024-04-25T14:08:44.720640+0000","last_peered":"2024-04-25T14:08:44.720640+0000","last_clean":"2024-04-25T14:08:44.720640+0000","last_became_active":"2024-04-25T14:08:44.717237+0000","last_became_peered":"2024-04-25T14:08:44.717237+0000","last_unstale":"2024-04-25T14:08:44.720640+0000","last_undegraded":"2024-04-25T14:08:44.720640+0000","last_fullsized":"2024-04-25T14:08:44.720640+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:39.391902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.698486+0000","last_change":"2024-04-25T14:08:44.698486+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.698486+0000","last_undegraded":"2024-04-25T14:08:44.698486+0000","last_fullsized":"2024-04-25T14:08:44.698486+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.726318+0000","last_change":"2024-04-25T14:08:44.723221+0000","last_active":"2024-04-25T14:08:44.726318+0000","last_peered":"2024-04-25T14:08:44.726318+0000","last_clean":"2024-04-25T14:08:44.726318+0000","last_became_active":"2024-04-25T14:08:44.722729+0000","last_became_peered":"2024-04-25T14:08:44.722729+0000","last_unstale":"2024-04-25T14:08:44.726318+0000","last_undegraded":"2024-04-25T14:08:44.726318+0000","last_fullsized":"2024-04-25T14:08:44.726318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:45.380736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.706660+0000","last_change":"2024-04-25T14:08:44.706660+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.706660+0000","last_undegraded":"2024-04-25T14:08:44.706660+0000","last_fullsized":"2024-04-25T14:08:44.706660+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.722005+0000","last_change":"2024-04-25T14:08:44.716853+0000","last_active":"2024-04-25T14:08:44.722005+0000","last_peered":"2024-04-25T14:08:44.722005+0000","last_clean":"2024-04-25T14:08:44.722005+0000","last_became_active":"2024-04-25T14:08:44.716479+0000","last_became_peered":"2024-04-25T14:08:44.716479+0000","last_unstale":"2024-04-25T14:08:44.722005+0000","last_undegraded":"2024-04-25T14:08:44.722005+0000","last_fullsized":"2024-04-25T14:08:44.722005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:04.500665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.701306+0000","last_change":"2024-04-25T14:08:44.701306+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.701306+0000","last_undegraded":"2024-04-25T14:08:44.701306+0000","last_fullsized":"2024-04-25T14:08:44.701306+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.731008+0000","last_change":"2024-04-25T14:08:44.726934+0000","last_active":"2024-04-25T14:08:44.731008+0000","last_peered":"2024-04-25T14:08:44.731008+0000","last_clean":"2024-04-25T14:08:44.731008+0000","last_became_active":"2024-04-25T14:08:44.726583+0000","last_became_peered":"2024-04-25T14:08:44.726583+0000","last_unstale":"2024-04-25T14:08:44.731008+0000","last_undegraded":"2024-04-25T14:08:44.731008+0000","last_fullsized":"2024-04-25T14:08:44.731008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:39.391902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.695980+0000","last_change":"2024-04-25T14:08:44.695980+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.695980+0000","last_undegraded":"2024-04-25T14:08:44.695980+0000","last_fullsized":"2024-04-25T14:08:44.695980+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.723891+0000","last_change":"2024-04-25T14:08:44.721083+0000","last_active":"2024-04-25T14:08:44.723891+0000","last_peered":"2024-04-25T14:08:44.723891+0000","last_clean":"2024-04-25T14:08:44.723891+0000","last_became_active":"2024-04-25T14:08:44.720775+0000","last_became_peered":"2024-04-25T14:08:44.720775+0000","last_unstale":"2024-04-25T14:08:44.723891+0000","last_undegraded":"2024-04-25T14:08:44.723891+0000","last_fullsized":"2024-04-25T14:08:44.723891+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:06.665120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.712919+0000","last_change":"2024-04-25T14:08:44.712919+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.712919+0000","last_undegraded":"2024-04-25T14:08:44.712919+0000","last_fullsized":"2024-04-25T14:08:44.712919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.725608+0000","last_change":"2024-04-25T14:08:44.723709+0000","last_active":"2024-04-25T14:08:44.725608+0000","last_peered":"2024-04-25T14:08:44.725608+0000","last_clean":"2024-04-25T14:08:44.725608+0000","last_became_active":"2024-04-25T14:08:44.723387+0000","last_became_peered":"2024-04-25T14:08:44.723387+0000","last_unstale":"2024-04-25T14:08:44.725608+0000","last_undegraded":"2024-04-25T14:08:44.725608+0000","last_fullsized":"2024-04-25T14:08:44.725608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:50.763491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.711073+0000","last_change":"2024-04-25T14:08:44.711073+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.711073+0000","last_undegraded":"2024-04-25T14:08:44.711073+0000","last_fullsized":"2024-04-25T14:08:44.711073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.729805+0000","last_change":"2024-04-25T14:08:44.728205+0000","last_active":"2024-04-25T14:08:44.729805+0000","last_peered":"2024-04-25T14:08:44.729805+0000","last_clean":"2024-04-25T14:08:44.729805+0000","last_became_active":"2024-04-25T14:08:44.727910+0000","last_became_peered":"2024-04-25T14:08:44.727910+0000","last_unstale":"2024-04-25T14:08:44.729805+0000","last_undegraded":"2024-04-25T14:08:44.729805+0000","last_fullsized":"2024-04-25T14:08:44.729805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:00.497301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.701371+0000","last_change":"2024-04-25T14:08:44.701371+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.701371+0000","last_undegraded":"2024-04-25T14:08:44.701371+0000","last_fullsized":"2024-04-25T14:08:44.701371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.715663+0000","last_change":"2024-04-25T14:08:44.713564+0000","last_active":"2024-04-25T14:08:44.715663+0000","last_peered":"2024-04-25T14:08:44.715663+0000","last_clean":"2024-04-25T14:08:44.715663+0000","last_became_active":"2024-04-25T14:08:44.712475+0000","last_became_peered":"2024-04-25T14:08:44.712475+0000","last_unstale":"2024-04-25T14:08:44.715663+0000","last_undegraded":"2024-04-25T14:08:44.715663+0000","last_fullsized":"2024-04-25T14:08:44.715663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:01.190413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.703954+0000","last_change":"2024-04-25T14:08:44.703954+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.703954+0000","last_undegraded":"2024-04-25T14:08:44.703954+0000","last_fullsized":"2024-04-25T14:08:44.703954+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.721141+0000","last_change":"2024-04-25T14:08:44.718093+0000","last_active":"2024-04-25T14:08:44.721141+0000","last_peered":"2024-04-25T14:08:44.721141+0000","last_clean":"2024-04-25T14:08:44.721141+0000","last_became_active":"2024-04-25T14:08:44.717768+0000","last_became_peered":"2024-04-25T14:08:44.717768+0000","last_unstale":"2024-04-25T14:08:44.721141+0000","last_undegraded":"2024-04-25T14:08:44.721141+0000","last_fullsized":"2024-04-25T14:08:44.721141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:10.155953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.700260+0000","last_change":"2024-04-25T14:08:44.700260+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.700260+0000","last_undegraded":"2024-04-25T14:08:44.700260+0000","last_fullsized":"2024-04-25T14:08:44.700260+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.720671+0000","last_change":"2024-04-25T14:08:44.717774+0000","last_active":"2024-04-25T14:08:44.720671+0000","last_peered":"2024-04-25T14:08:44.720671+0000","last_clean":"2024-04-25T14:08:44.720671+0000","last_became_active":"2024-04-25T14:08:44.717425+0000","last_became_peered":"2024-04-25T14:08:44.717425+0000","last_unstale":"2024-04-25T14:08:44.720671+0000","last_undegraded":"2024-04-25T14:08:44.720671+0000","last_fullsized":"2024-04-25T14:08:44.720671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:55.423236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.703371+0000","last_change":"2024-04-25T14:08:44.703371+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.703371+0000","last_undegraded":"2024-04-25T14:08:44.703371+0000","last_fullsized":"2024-04-25T14:08:44.703371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.703242+0000","last_change":"2024-04-25T14:08:44.700728+0000","last_active":"2024-04-25T14:08:44.703242+0000","last_peered":"2024-04-25T14:08:44.703242+0000","last_clean":"2024-04-25T14:08:44.703242+0000","last_became_active":"2024-04-25T14:08:44.700344+0000","last_became_peered":"2024-04-25T14:08:44.700344+0000","last_unstale":"2024-04-25T14:08:44.703242+0000","last_undegraded":"2024-04-25T14:08:44.703242+0000","last_fullsized":"2024-04-25T14:08:44.703242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:36:34.689927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.694919+0000","last_change":"2024-04-25T14:08:44.694919+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.694919+0000","last_undegraded":"2024-04-25T14:08:44.694919+0000","last_fullsized":"2024-04-25T14:08:44.694919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.727899+0000","last_change":"2024-04-25T14:08:44.725383+0000","last_active":"2024-04-25T14:08:44.727899+0000","last_peered":"2024-04-25T14:08:44.727899+0000","last_clean":"2024-04-25T14:08:44.727899+0000","last_became_active":"2024-04-25T14:08:44.724211+0000","last_became_peered":"2024-04-25T14:08:44.724211+0000","last_unstale":"2024-04-25T14:08:44.727899+0000","last_undegraded":"2024-04-25T14:08:44.727899+0000","last_fullsized":"2024-04-25T14:08:44.727899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:16.246992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.694866+0000","last_change":"2024-04-25T14:08:44.694866+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.694866+0000","last_undegraded":"2024-04-25T14:08:44.694866+0000","last_fullsized":"2024-04-25T14:08:44.694866+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.719715+0000","last_change":"2024-04-25T14:08:44.716954+0000","last_active":"2024-04-25T14:08:44.719715+0000","last_peered":"2024-04-25T14:08:44.719715+0000","last_clean":"2024-04-25T14:08:44.719715+0000","last_became_active":"2024-04-25T14:08:44.716583+0000","last_became_peered":"2024-04-25T14:08:44.716583+0000","last_unstale":"2024-04-25T14:08:44.719715+0000","last_undegraded":"2024-04-25T14:08:44.719715+0000","last_fullsized":"2024-04-25T14:08:44.719715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:31.019299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.705709+0000","last_change":"2024-04-25T14:08:44.705709+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.705709+0000","last_undegraded":"2024-04-25T14:08:44.705709+0000","last_fullsized":"2024-04-25T14:08:44.705709+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.727980+0000","last_change":"2024-04-25T14:08:44.725423+0000","last_active":"2024-04-25T14:08:44.727980+0000","last_peered":"2024-04-25T14:08:44.727980+0000","last_clean":"2024-04-25T14:08:44.727980+0000","last_became_active":"2024-04-25T14:08:44.725036+0000","last_became_peered":"2024-04-25T14:08:44.725036+0000","last_unstale":"2024-04-25T14:08:44.727980+0000","last_undegraded":"2024-04-25T14:08:44.727980+0000","last_fullsized":"2024-04-25T14:08:44.727980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:45.210808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.703583+0000","last_change":"2024-04-25T14:08:44.703583+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.703583+0000","last_undegraded":"2024-04-25T14:08:44.703583+0000","last_fullsized":"2024-04-25T14:08:44.703583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.720288+0000","last_change":"2024-04-25T14:08:44.717040+0000","last_active":"2024-04-25T14:08:44.720288+0000","last_peered":"2024-04-25T14:08:44.720288+0000","last_clean":"2024-04-25T14:08:44.720288+0000","last_became_active":"2024-04-25T14:08:44.716044+0000","last_became_peered":"2024-04-25T14:08:44.716044+0000","last_unstale":"2024-04-25T14:08:44.720288+0000","last_undegraded":"2024-04-25T14:08:44.720288+0000","last_fullsized":"2024-04-25T14:08:44.720288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:41.020517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.712548+0000","last_change":"2024-04-25T14:08:44.712548+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.712548+0000","last_undegraded":"2024-04-25T14:08:44.712548+0000","last_fullsized":"2024-04-25T14:08:44.712548+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.710784+0000","last_change":"2024-04-25T14:08:44.704043+0000","last_active":"2024-04-25T14:08:44.710784+0000","last_peered":"2024-04-25T14:08:44.710784+0000","last_clean":"2024-04-25T14:08:44.710784+0000","last_became_active":"2024-04-25T14:08:44.703697+0000","last_became_peered":"2024-04-25T14:08:44.703697+0000","last_unstale":"2024-04-25T14:08:44.710784+0000","last_undegraded":"2024-04-25T14:08:44.710784+0000","last_fullsized":"2024-04-25T14:08:44.710784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:42.844372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.704587+0000","last_change":"2024-04-25T14:08:44.704587+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.704587+0000","last_undegraded":"2024-04-25T14:08:44.704587+0000","last_fullsized":"2024-04-25T14:08:44.704587+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.722459+0000","last_change":"2024-04-25T14:08:44.721286+0000","last_active":"2024-04-25T14:08:44.722459+0000","last_peered":"2024-04-25T14:08:44.722459+0000","last_clean":"2024-04-25T14:08:44.722459+0000","last_became_active":"2024-04-25T14:08:44.720949+0000","last_became_peered":"2024-04-25T14:08:44.720949+0000","last_unstale":"2024-04-25T14:08:44.722459+0000","last_undegraded":"2024-04-25T14:08:44.722459+0000","last_fullsized":"2024-04-25T14:08:44.722459+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:58:08.650136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.695843+0000","last_change":"2024-04-25T14:08:44.695843+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.695843+0000","last_undegraded":"2024-04-25T14:08:44.695843+0000","last_fullsized":"2024-04-25T14:08:44.695843+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.728386+0000","last_change":"2024-04-25T14:08:44.725093+0000","last_active":"2024-04-25T14:08:44.728386+0000","last_peered":"2024-04-25T14:08:44.728386+0000","last_clean":"2024-04-25T14:08:44.728386+0000","last_became_active":"2024-04-25T14:08:44.724761+0000","last_became_peered":"2024-04-25T14:08:44.724761+0000","last_unstale":"2024-04-25T14:08:44.728386+0000","last_undegraded":"2024-04-25T14:08:44.728386+0000","last_fullsized":"2024-04-25T14:08:44.728386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:37.926762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.713062+0000","last_change":"2024-04-25T14:08:44.713062+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.713062+0000","last_undegraded":"2024-04-25T14:08:44.713062+0000","last_fullsized":"2024-04-25T14:08:44.713062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.723865+0000","last_change":"2024-04-25T14:08:44.721282+0000","last_active":"2024-04-25T14:08:44.723865+0000","last_peered":"2024-04-25T14:08:44.723865+0000","last_clean":"2024-04-25T14:08:44.723865+0000","last_became_active":"2024-04-25T14:08:44.721077+0000","last_became_peered":"2024-04-25T14:08:44.721077+0000","last_unstale":"2024-04-25T14:08:44.723865+0000","last_undegraded":"2024-04-25T14:08:44.723865+0000","last_fullsized":"2024-04-25T14:08:44.723865+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:07.051812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.698403+0000","last_change":"2024-04-25T14:08:44.698403+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.698403+0000","last_undegraded":"2024-04-25T14:08:44.698403+0000","last_fullsized":"2024-04-25T14:08:44.698403+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.704667+0000","last_change":"2024-04-25T14:08:44.701347+0000","last_active":"2024-04-25T14:08:44.704667+0000","last_peered":"2024-04-25T14:08:44.704667+0000","last_clean":"2024-04-25T14:08:44.704667+0000","last_became_active":"2024-04-25T14:08:44.700917+0000","last_became_peered":"2024-04-25T14:08:44.700917+0000","last_unstale":"2024-04-25T14:08:44.704667+0000","last_undegraded":"2024-04-25T14:08:44.704667+0000","last_fullsized":"2024-04-25T14:08:44.704667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:49.250431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.702541+0000","last_change":"2024-04-25T14:08:44.702541+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.702541+0000","last_undegraded":"2024-04-25T14:08:44.702541+0000","last_fullsized":"2024-04-25T14:08:44.702541+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.728341+0000","last_change":"2024-04-25T14:08:44.724767+0000","last_active":"2024-04-25T14:08:44.728341+0000","last_peered":"2024-04-25T14:08:44.728341+0000","last_clean":"2024-04-25T14:08:44.728341+0000","last_became_active":"2024-04-25T14:08:44.724442+0000","last_became_peered":"2024-04-25T14:08:44.724442+0000","last_unstale":"2024-04-25T14:08:44.728341+0000","last_undegraded":"2024-04-25T14:08:44.728341+0000","last_fullsized":"2024-04-25T14:08:44.728341+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:56.689895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.705244+0000","last_change":"2024-04-25T14:08:44.705244+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.705244+0000","last_undegraded":"2024-04-25T14:08:44.705244+0000","last_fullsized":"2024-04-25T14:08:44.705244+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.726004+0000","last_change":"2024-04-25T14:08:44.723570+0000","last_active":"2024-04-25T14:08:44.726004+0000","last_peered":"2024-04-25T14:08:44.726004+0000","last_clean":"2024-04-25T14:08:44.726004+0000","last_became_active":"2024-04-25T14:08:44.722656+0000","last_became_peered":"2024-04-25T14:08:44.722656+0000","last_unstale":"2024-04-25T14:08:44.726004+0000","last_undegraded":"2024-04-25T14:08:44.726004+0000","last_fullsized":"2024-04-25T14:08:44.726004+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:23.016722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.708517+0000","last_change":"2024-04-25T14:08:44.708517+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.708517+0000","last_undegraded":"2024-04-25T14:08:44.708517+0000","last_fullsized":"2024-04-25T14:08:44.708517+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.723217+0000","last_change":"2024-04-25T14:08:44.719472+0000","last_active":"2024-04-25T14:08:44.723217+0000","last_peered":"2024-04-25T14:08:44.723217+0000","last_clean":"2024-04-25T14:08:44.723217+0000","last_became_active":"2024-04-25T14:08:44.719159+0000","last_became_peered":"2024-04-25T14:08:44.719159+0000","last_unstale":"2024-04-25T14:08:44.723217+0000","last_undegraded":"2024-04-25T14:08:44.723217+0000","last_fullsized":"2024-04-25T14:08:44.723217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:15.094910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"76'100","reported_seq":329,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.711790+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:08:44.711790+0000","last_peered":"2024-04-25T14:08:44.711790+0000","last_clean":"2024-04-25T14:08:44.711790+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:08:44.711790+0000","last_undegraded":"2024-04-25T14:08:44.711790+0000","last_fullsized":"2024-04-25T14:08:44.711790+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":306,"num_read_kb":759,"num_write":149,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":306,"num_read_kb":759,"num_write":149,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574909,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32788,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339052,"statfs":{"total":96636764160,"available":96603189248,"internally_reserved":0,"allocated":1642496,"data_stored":1110288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1048576,"data_stored":519920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574911,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1048576,"data_stored":519920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1048576,"data_stored":519920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1036288,"data_stored":513253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1048576,"data_stored":519920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1048576,"data_stored":519920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574910,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32772,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339068,"statfs":{"total":96636764160,"available":96603205632,"internally_reserved":0,"allocated":1630208,"data_stored":1103621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:08:49.913 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:08:50.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.098+0000 7f69e2000640 1 Processor -- start 2024-04-25T14:08:50.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.098+0000 7f69e2000640 1 -- start start 2024-04-25T14:08:50.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.098+0000 7f69e2000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f69dc0d2c80 0x7f69dc15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:50.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.098+0000 7f69e2000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69dc070e20 0x7f69dc071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:50.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.098+0000 7f69e2000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f69dc0722f0 0x7f69dc072700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:50.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.098+0000 7f69e2000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f69dc0ce720 con 0x7f69dc0d2c80 2024-04-25T14:08:50.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.098+0000 7f69e2000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f69dc071770 con 0x7f69dc0722f0 2024-04-25T14:08:50.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.098+0000 7f69e2000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f69dc0cd8d0 con 0x7f69dc070e20 2024-04-25T14:08:50.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.098+0000 7f69db400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f69dc0d2c80 0x7f69dc15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:50.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.102+0000 7f69db400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f69dc0d2c80 0x7f69dc15dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40288/0 (socket says 172.21.15.86:40288) 2024-04-25T14:08:50.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.102+0000 7f69db400640 1 -- 172.21.15.86:0/3712630058 learned_addr learned my addr 172.21.15.86:0/3712630058 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:50.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.102+0000 7f69d2a00640 1 --2- 172.21.15.86:0/3712630058 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69dc070e20 0x7f69dc071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:50.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.102+0000 7f69dbe00640 1 --2- 172.21.15.86:0/3712630058 >> v2:172.21.15.155:3300/0 conn(0x7f69dc0722f0 0x7f69dc072700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:50.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.102+0000 7f69db400640 1 -- 172.21.15.86:0/3712630058 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69dc070e20 msgr2=0x7f69dc071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:50.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.102+0000 7f69db400640 1 --2- 172.21.15.86:0/3712630058 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69dc070e20 0x7f69dc071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:50.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.102+0000 7f69db400640 1 -- 172.21.15.86:0/3712630058 >> v2:172.21.15.155:3300/0 conn(0x7f69dc0722f0 msgr2=0x7f69dc072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:50.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.102+0000 7f69db400640 1 --2- 172.21.15.86:0/3712630058 >> v2:172.21.15.155:3300/0 conn(0x7f69dc0722f0 0x7f69dc072700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:50.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.102+0000 7f69db400640 1 -- 172.21.15.86:0/3712630058 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69dc0ce970 con 0x7f69dc0d2c80 2024-04-25T14:08:50.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.102+0000 7f69db400640 1 --2- 172.21.15.86:0/3712630058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69dc0d2c80 0x7f69dc15dc90 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f69dc0736c0 tx=0x7f69cc01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9690e1d52973ee34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:50.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.102+0000 7f69daa00640 1 -- 172.21.15.86:0/3712630058 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc0b5070 con 0x7f69dc0d2c80 2024-04-25T14:08:50.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.102+0000 7f69daa00640 1 -- 172.21.15.86:0/3712630058 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69cc0a89d0 con 0x7f69dc0d2c80 2024-04-25T14:08:50.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.102+0000 7f69daa00640 1 -- 172.21.15.86:0/3712630058 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc0a8da0 con 0x7f69dc0d2c80 2024-04-25T14:08:50.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.106+0000 7f69e2000640 1 -- 172.21.15.86:0/3712630058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69dc0d2c80 msgr2=0x7f69dc15dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:50.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.106+0000 7f69e2000640 1 --2- 172.21.15.86:0/3712630058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69dc0d2c80 0x7f69dc15dc90 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f69dc0736c0 tx=0x7f69cc01ed10 comp rx=0 tx=0).stop 2024-04-25T14:08:50.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.106+0000 7f69e2000640 1 -- 172.21.15.86:0/3712630058 shutdown_connections 2024-04-25T14:08:50.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.106+0000 7f69e2000640 1 --2- 172.21.15.86:0/3712630058 >> v2:172.21.15.155:3300/0 conn(0x7f69dc0722f0 0x7f69dc072700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:50.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.106+0000 7f69e2000640 1 --2- 172.21.15.86:0/3712630058 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69dc070e20 0x7f69dc071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:50.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.106+0000 7f69e2000640 1 --2- 172.21.15.86:0/3712630058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69dc0d2c80 0x7f69dc15dc90 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:50.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.106+0000 7f69e2000640 1 -- 172.21.15.86:0/3712630058 >> 172.21.15.86:0/3712630058 conn(0x7f69dc0cffc0 msgr2=0x7f69dc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:50.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.106+0000 7f69e2000640 1 -- 172.21.15.86:0/3712630058 shutdown_connections 2024-04-25T14:08:50.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.106+0000 7f69e2000640 1 -- 172.21.15.86:0/3712630058 wait complete. 2024-04-25T14:08:50.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.106+0000 7f69e2000640 1 Processor -- start 2024-04-25T14:08:50.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.106+0000 7f69e2000640 1 -- start start 2024-04-25T14:08:50.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.110+0000 7f69e2000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69dc070e20 0x7f69dc06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:50.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.110+0000 7f69e2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69dc0722f0 0x7f69dc1f0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:50.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.110+0000 7f69e2000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69dc0d2c80 0x7f69dc154c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:50.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.110+0000 7f69e2000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc0d21a0 con 0x7f69dc0722f0 2024-04-25T14:08:50.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.110+0000 7f69e2000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc0d1ec0 con 0x7f69dc0d2c80 2024-04-25T14:08:50.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.110+0000 7f69e2000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f69dc0d2480 con 0x7f69dc070e20 2024-04-25T14:08:50.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.110+0000 7f69d2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69dc0722f0 0x7f69dc1f0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:50.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.110+0000 7f69d2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69dc0722f0 0x7f69dc1f0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40290/0 (socket says 172.21.15.86:40290) 2024-04-25T14:08:50.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.110+0000 7f69d2a00640 1 -- 172.21.15.86:0/1446549233 learned_addr learned my addr 172.21.15.86:0/1446549233 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:50.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.110+0000 7f69db400640 1 --2- 172.21.15.86:0/1446549233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69dc070e20 0x7f69dc06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:50.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.110+0000 7f69dbe00640 1 --2- 172.21.15.86:0/1446549233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69dc0d2c80 0x7f69dc154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:50.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.110+0000 7f69d2a00640 1 -- 172.21.15.86:0/1446549233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69dc070e20 msgr2=0x7f69dc06d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:50.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.110+0000 7f69d2a00640 1 --2- 172.21.15.86:0/1446549233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69dc070e20 0x7f69dc06d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:50.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.110+0000 7f69d2a00640 1 -- 172.21.15.86:0/1446549233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69dc0d2c80 msgr2=0x7f69dc154c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:50.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.110+0000 7f69d2a00640 1 --2- 172.21.15.86:0/1446549233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69dc0d2c80 0x7f69dc154c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:50.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.110+0000 7f69d2a00640 1 -- 172.21.15.86:0/1446549233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69dc155180 con 0x7f69dc0722f0 2024-04-25T14:08:50.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.110+0000 7f69d2a00640 1 --2- 172.21.15.86:0/1446549233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69dc0722f0 0x7f69dc1f0c00 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f69c400d800 tx=0x7f69c400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:50.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.110+0000 7f69d3e00640 1 -- 172.21.15.86:0/1446549233 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c4009640 con 0x7f69dc0722f0 2024-04-25T14:08:50.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.110+0000 7f69d3e00640 1 -- 172.21.15.86:0/1446549233 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69c4019070 con 0x7f69dc0722f0 2024-04-25T14:08:50.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.110+0000 7f69d3e00640 1 -- 172.21.15.86:0/1446549233 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c4009a20 con 0x7f69dc0722f0 2024-04-25T14:08:50.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.110+0000 7f69e2000640 1 -- 172.21.15.86:0/1446549233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69dc0c9fe0 con 0x7f69dc0722f0 2024-04-25T14:08:50.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.110+0000 7f69e2000640 1 -- 172.21.15.86:0/1446549233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69dc0c9a20 con 0x7f69dc0722f0 2024-04-25T14:08:50.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.110+0000 7f69e2000640 1 -- 172.21.15.86:0/1446549233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69dc0c9490 con 0x7f69dc0722f0 2024-04-25T14:08:50.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.110+0000 7f69d3e00640 1 -- 172.21.15.86:0/1446549233 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7f69c4009bc0 con 0x7f69dc0722f0 2024-04-25T14:08:50.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.114+0000 7f69d3e00640 1 --2- 172.21.15.86:0/1446549233 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f69b806f550 0x7f69b8071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:50.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.114+0000 7f69d3e00640 1 -- 172.21.15.86:0/1446549233 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f69c408f840 con 0x7f69dc0722f0 2024-04-25T14:08:50.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.114+0000 7f69db400640 1 --2- 172.21.15.86:0/1446549233 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f69b806f550 0x7f69b8071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:50.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.114+0000 7f69db400640 1 --2- 172.21.15.86:0/1446549233 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f69b806f550 0x7f69b8071a60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f69cc0b2000 tx=0x7f69cc009040 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:50.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.118+0000 7f69d3e00640 1 -- 172.21.15.86:0/1446549233 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f69c405be30 con 0x7f69dc0722f0 2024-04-25T14:08:50.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.270+0000 7f69e2000640 1 -- 172.21.15.86:0/1446549233 --> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f69dc069d60 con 0x7f69b806f550 2024-04-25T14:08:50.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.278+0000 7f69d3e00640 1 -- 172.21.15.86:0/1446549233 <== mgr.7092 v2:172.21.15.86:6832/3589556609 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246763 (secure 0 0 0) 0x7f69cc0b0440 con 0x7f69b806f550 2024-04-25T14:08:50.281 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:08:50.283 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:08:50.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.282+0000 7f69e2000640 1 -- 172.21.15.86:0/1446549233 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f69b806f550 msgr2=0x7f69b8071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:50.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.282+0000 7f69e2000640 1 --2- 172.21.15.86:0/1446549233 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f69b806f550 0x7f69b8071a60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f69cc0b2000 tx=0x7f69cc009040 comp rx=0 tx=0).stop 2024-04-25T14:08:50.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.282+0000 7f69e2000640 1 -- 172.21.15.86:0/1446549233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69dc0722f0 msgr2=0x7f69dc1f0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:50.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.282+0000 7f69e2000640 1 --2- 172.21.15.86:0/1446549233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69dc0722f0 0x7f69dc1f0c00 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f69c400d800 tx=0x7f69c400dcc0 comp rx=0 tx=0).stop 2024-04-25T14:08:50.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.282+0000 7f69e2000640 1 -- 172.21.15.86:0/1446549233 shutdown_connections 2024-04-25T14:08:50.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.282+0000 7f69e2000640 1 --2- 172.21.15.86:0/1446549233 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f69b806f550 0x7f69b8071a60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:50.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.282+0000 7f69e2000640 1 --2- 172.21.15.86:0/1446549233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69dc0d2c80 0x7f69dc154c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:50.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.282+0000 7f69e2000640 1 --2- 172.21.15.86:0/1446549233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69dc0722f0 0x7f69dc1f0c00 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:50.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.282+0000 7f69e2000640 1 --2- 172.21.15.86:0/1446549233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69dc070e20 0x7f69dc06d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:50.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.282+0000 7f69e2000640 1 -- 172.21.15.86:0/1446549233 >> 172.21.15.86:0/1446549233 conn(0x7f69dc0cffc0 msgr2=0x7f69dc0cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:50.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.282+0000 7f69e2000640 1 -- 172.21.15.86:0/1446549233 shutdown_connections 2024-04-25T14:08:50.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.282+0000 7f69e2000640 1 -- 172.21.15.86:0/1446549233 wait complete. 2024-04-25T14:08:50.302 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T14:08:49.359300+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":306,"num_read_kb":759,"num_write":149,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258792,"kb_used_data":9328,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715928,"statfs":{"total":773094113280,"available":772829110272,"internally_reserved":0,"allocated":9551872,"data_stored":5326762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.671432"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.700729+0000","last_change":"2024-04-25T14:08:44.700729+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.700729+0000","last_undegraded":"2024-04-25T14:08:44.700729+0000","last_fullsized":"2024-04-25T14:08:44.700729+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.695217+0000","last_change":"2024-04-25T14:08:44.695217+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.695217+0000","last_undegraded":"2024-04-25T14:08:44.695217+0000","last_fullsized":"2024-04-25T14:08:44.695217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.700715+0000","last_change":"2024-04-25T14:08:44.700715+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.700715+0000","last_undegraded":"2024-04-25T14:08:44.700715+0000","last_fullsized":"2024-04-25T14:08:44.700715+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.707535+0000","last_change":"2024-04-25T14:08:44.707535+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.707535+0000","last_undegraded":"2024-04-25T14:08:44.707535+0000","last_fullsized":"2024-04-25T14:08:44.707535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.696533+0000","last_change":"2024-04-25T14:08:44.696533+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.696533+0000","last_undegraded":"2024-04-25T14:08:44.696533+0000","last_fullsized":"2024-04-25T14:08:44.696533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.715602+0000","last_change":"2024-04-25T14:08:44.715602+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.715602+0000","last_undegraded":"2024-04-25T14:08:44.715602+0000","last_fullsized":"2024-04-25T14:08:44.715602+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.705515+0000","last_change":"2024-04-25T14:08:44.705515+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.705515+0000","last_undegraded":"2024-04-25T14:08:44.705515+0000","last_fullsized":"2024-04-25T14:08:44.705515+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.704048+0000","last_change":"2024-04-25T14:08:44.704048+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.704048+0000","last_undegraded":"2024-04-25T14:08:44.704048+0000","last_fullsized":"2024-04-25T14:08:44.704048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.701272+0000","last_change":"2024-04-25T14:08:44.701272+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.701272+0000","last_undegraded":"2024-04-25T14:08:44.701272+0000","last_fullsized":"2024-04-25T14:08:44.701272+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.709510+0000","last_change":"2024-04-25T14:08:44.709510+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.709510+0000","last_undegraded":"2024-04-25T14:08:44.709510+0000","last_fullsized":"2024-04-25T14:08:44.709510+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.699606+0000","last_change":"2024-04-25T14:08:44.699606+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.699606+0000","last_undegraded":"2024-04-25T14:08:44.699606+0000","last_fullsized":"2024-04-25T14:08:44.699606+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.711227+0000","last_change":"2024-04-25T14:08:44.711227+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.711227+0000","last_undegraded":"2024-04-25T14:08:44.711227+0000","last_fullsized":"2024-04-25T14:08:44.711227+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.702941+0000","last_change":"2024-04-25T14:08:44.702941+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.702941+0000","last_undegraded":"2024-04-25T14:08:44.702941+0000","last_fullsized":"2024-04-25T14:08:44.702941+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.699462+0000","last_change":"2024-04-25T14:08:44.699462+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.699462+0000","last_undegraded":"2024-04-25T14:08:44.699462+0000","last_fullsized":"2024-04-25T14:08:44.699462+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.702169+0000","last_change":"2024-04-25T14:08:44.702169+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.702169+0000","last_undegraded":"2024-04-25T14:08:44.702169+0000","last_fullsized":"2024-04-25T14:08:44.702169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.708333+0000","last_change":"2024-04-25T14:08:44.708333+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.708333+0000","last_undegraded":"2024-04-25T14:08:44.708333+0000","last_fullsized":"2024-04-25T14:08:44.708333+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.704663+0000","last_change":"2024-04-25T14:08:44.704663+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.704663+0000","last_undegraded":"2024-04-25T14:08:44.704663+0000","last_fullsized":"2024-04-25T14:08:44.704663+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.700154+0000","last_change":"2024-04-25T14:08:44.700154+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.700154+0000","last_undegraded":"2024-04-25T14:08:44.700154+0000","last_fullsized":"2024-04-25T14:08:44.700154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.699078+0000","last_change":"2024-04-25T14:08:44.699078+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.699078+0000","last_undegraded":"2024-04-25T14:08:44.699078+0000","last_fullsized":"2024-04-25T14:08:44.699078+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.707069+0000","last_change":"2024-04-25T14:08:44.707069+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.707069+0000","last_undegraded":"2024-04-25T14:08:44.707069+0000","last_fullsized":"2024-04-25T14:08:44.707069+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.700474+0000","last_change":"2024-04-25T14:08:44.700474+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.700474+0000","last_undegraded":"2024-04-25T14:08:44.700474+0000","last_fullsized":"2024-04-25T14:08:44.700474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.698892+0000","last_change":"2024-04-25T14:08:44.698892+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.698892+0000","last_undegraded":"2024-04-25T14:08:44.698892+0000","last_fullsized":"2024-04-25T14:08:44.698892+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.702950+0000","last_change":"2024-04-25T14:08:44.702950+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.702950+0000","last_undegraded":"2024-04-25T14:08:44.702950+0000","last_fullsized":"2024-04-25T14:08:44.702950+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.698720+0000","last_change":"2024-04-25T14:08:44.698720+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.698720+0000","last_undegraded":"2024-04-25T14:08:44.698720+0000","last_fullsized":"2024-04-25T14:08:44.698720+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.696352+0000","last_change":"2024-04-25T14:08:44.696352+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.696352+0000","last_undegraded":"2024-04-25T14:08:44.696352+0000","last_fullsized":"2024-04-25T14:08:44.696352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.700875+0000","last_change":"2024-04-25T14:08:44.700875+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.700875+0000","last_undegraded":"2024-04-25T14:08:44.700875+0000","last_fullsized":"2024-04-25T14:08:44.700875+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.709537+0000","last_change":"2024-04-25T14:08:44.709537+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.709537+0000","last_undegraded":"2024-04-25T14:08:44.709537+0000","last_fullsized":"2024-04-25T14:08:44.709537+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.702856+0000","last_change":"2024-04-25T14:08:44.702856+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.702856+0000","last_undegraded":"2024-04-25T14:08:44.702856+0000","last_fullsized":"2024-04-25T14:08:44.702856+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.704801+0000","last_change":"2024-04-25T14:08:44.704801+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.704801+0000","last_undegraded":"2024-04-25T14:08:44.704801+0000","last_fullsized":"2024-04-25T14:08:44.704801+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.695641+0000","last_change":"2024-04-25T14:08:44.695641+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.695641+0000","last_undegraded":"2024-04-25T14:08:44.695641+0000","last_fullsized":"2024-04-25T14:08:44.695641+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.695624+0000","last_change":"2024-04-25T14:08:44.695624+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.695624+0000","last_undegraded":"2024-04-25T14:08:44.695624+0000","last_fullsized":"2024-04-25T14:08:44.695624+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.707878+0000","last_change":"2024-04-25T14:08:44.707878+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.707878+0000","last_undegraded":"2024-04-25T14:08:44.707878+0000","last_fullsized":"2024-04-25T14:08:44.707878+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.701936+0000","last_change":"2024-04-25T14:08:44.701936+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.701936+0000","last_undegraded":"2024-04-25T14:08:44.701936+0000","last_fullsized":"2024-04-25T14:08:44.701936+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.728427+0000","last_change":"2024-04-25T14:08:44.725130+0000","last_active":"2024-04-25T14:08:44.728427+0000","last_peered":"2024-04-25T14:08:44.728427+0000","last_clean":"2024-04-25T14:08:44.728427+0000","last_became_active":"2024-04-25T14:08:44.724805+0000","last_became_peered":"2024-04-25T14:08:44.724805+0000","last_unstale":"2024-04-25T14:08:44.728427+0000","last_undegraded":"2024-04-25T14:08:44.728427+0000","last_fullsized":"2024-04-25T14:08:44.728427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:58.834978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.713037+0000","last_change":"2024-04-25T14:08:44.713037+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.713037+0000","last_undegraded":"2024-04-25T14:08:44.713037+0000","last_fullsized":"2024-04-25T14:08:44.713037+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.714929+0000","last_change":"2024-04-25T14:08:44.713248+0000","last_active":"2024-04-25T14:08:44.714929+0000","last_peered":"2024-04-25T14:08:44.714929+0000","last_clean":"2024-04-25T14:08:44.714929+0000","last_became_active":"2024-04-25T14:08:44.712237+0000","last_became_peered":"2024-04-25T14:08:44.712237+0000","last_unstale":"2024-04-25T14:08:44.714929+0000","last_undegraded":"2024-04-25T14:08:44.714929+0000","last_fullsized":"2024-04-25T14:08:44.714929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:14.643131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.704564+0000","last_change":"2024-04-25T14:08:44.704564+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.704564+0000","last_undegraded":"2024-04-25T14:08:44.704564+0000","last_fullsized":"2024-04-25T14:08:44.704564+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.728516+0000","last_change":"2024-04-25T14:08:44.725153+0000","last_active":"2024-04-25T14:08:44.728516+0000","last_peered":"2024-04-25T14:08:44.728516+0000","last_clean":"2024-04-25T14:08:44.728516+0000","last_became_active":"2024-04-25T14:08:44.724848+0000","last_became_peered":"2024-04-25T14:08:44.724848+0000","last_unstale":"2024-04-25T14:08:44.728516+0000","last_undegraded":"2024-04-25T14:08:44.728516+0000","last_fullsized":"2024-04-25T14:08:44.728516+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:28.574820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.704170+0000","last_change":"2024-04-25T14:08:44.704170+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.704170+0000","last_undegraded":"2024-04-25T14:08:44.704170+0000","last_fullsized":"2024-04-25T14:08:44.704170+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.721669+0000","last_change":"2024-04-25T14:08:44.720784+0000","last_active":"2024-04-25T14:08:44.721669+0000","last_peered":"2024-04-25T14:08:44.721669+0000","last_clean":"2024-04-25T14:08:44.721669+0000","last_became_active":"2024-04-25T14:08:44.720248+0000","last_became_peered":"2024-04-25T14:08:44.720248+0000","last_unstale":"2024-04-25T14:08:44.721669+0000","last_undegraded":"2024-04-25T14:08:44.721669+0000","last_fullsized":"2024-04-25T14:08:44.721669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:00.497301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.701864+0000","last_change":"2024-04-25T14:08:44.701864+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.701864+0000","last_undegraded":"2024-04-25T14:08:44.701864+0000","last_fullsized":"2024-04-25T14:08:44.701864+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.730802+0000","last_change":"2024-04-25T14:08:44.727021+0000","last_active":"2024-04-25T14:08:44.730802+0000","last_peered":"2024-04-25T14:08:44.730802+0000","last_clean":"2024-04-25T14:08:44.730802+0000","last_became_active":"2024-04-25T14:08:44.726079+0000","last_became_peered":"2024-04-25T14:08:44.726079+0000","last_unstale":"2024-04-25T14:08:44.730802+0000","last_undegraded":"2024-04-25T14:08:44.730802+0000","last_fullsized":"2024-04-25T14:08:44.730802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:18.045986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.701896+0000","last_change":"2024-04-25T14:08:44.701896+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.701896+0000","last_undegraded":"2024-04-25T14:08:44.701896+0000","last_fullsized":"2024-04-25T14:08:44.701896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.728800+0000","last_change":"2024-04-25T14:08:44.726268+0000","last_active":"2024-04-25T14:08:44.728800+0000","last_peered":"2024-04-25T14:08:44.728800+0000","last_clean":"2024-04-25T14:08:44.728800+0000","last_became_active":"2024-04-25T14:08:44.725839+0000","last_became_peered":"2024-04-25T14:08:44.725839+0000","last_unstale":"2024-04-25T14:08:44.728800+0000","last_undegraded":"2024-04-25T14:08:44.728800+0000","last_fullsized":"2024-04-25T14:08:44.728800+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:01.190413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.711108+0000","last_change":"2024-04-25T14:08:44.711108+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.711108+0000","last_undegraded":"2024-04-25T14:08:44.711108+0000","last_fullsized":"2024-04-25T14:08:44.711108+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.717869+0000","last_change":"2024-04-25T14:08:44.712147+0000","last_active":"2024-04-25T14:08:44.717869+0000","last_peered":"2024-04-25T14:08:44.717869+0000","last_clean":"2024-04-25T14:08:44.717869+0000","last_became_active":"2024-04-25T14:08:44.711799+0000","last_became_peered":"2024-04-25T14:08:44.711799+0000","last_unstale":"2024-04-25T14:08:44.717869+0000","last_undegraded":"2024-04-25T14:08:44.717869+0000","last_fullsized":"2024-04-25T14:08:44.717869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:28.574820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.702431+0000","last_change":"2024-04-25T14:08:44.702431+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.702431+0000","last_undegraded":"2024-04-25T14:08:44.702431+0000","last_fullsized":"2024-04-25T14:08:44.702431+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.726027+0000","last_change":"2024-04-25T14:08:44.723883+0000","last_active":"2024-04-25T14:08:44.726027+0000","last_peered":"2024-04-25T14:08:44.726027+0000","last_clean":"2024-04-25T14:08:44.726027+0000","last_became_active":"2024-04-25T14:08:44.723229+0000","last_became_peered":"2024-04-25T14:08:44.723229+0000","last_unstale":"2024-04-25T14:08:44.726027+0000","last_undegraded":"2024-04-25T14:08:44.726027+0000","last_fullsized":"2024-04-25T14:08:44.726027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:45.472144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.697949+0000","last_change":"2024-04-25T14:08:44.697949+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.697949+0000","last_undegraded":"2024-04-25T14:08:44.697949+0000","last_fullsized":"2024-04-25T14:08:44.697949+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.726787+0000","last_change":"2024-04-25T14:08:44.723981+0000","last_active":"2024-04-25T14:08:44.726787+0000","last_peered":"2024-04-25T14:08:44.726787+0000","last_clean":"2024-04-25T14:08:44.726787+0000","last_became_active":"2024-04-25T14:08:44.723657+0000","last_became_peered":"2024-04-25T14:08:44.723657+0000","last_unstale":"2024-04-25T14:08:44.726787+0000","last_undegraded":"2024-04-25T14:08:44.726787+0000","last_fullsized":"2024-04-25T14:08:44.726787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:51.117327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.705995+0000","last_change":"2024-04-25T14:08:44.705995+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.705995+0000","last_undegraded":"2024-04-25T14:08:44.705995+0000","last_fullsized":"2024-04-25T14:08:44.705995+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.720640+0000","last_change":"2024-04-25T14:08:44.718626+0000","last_active":"2024-04-25T14:08:44.720640+0000","last_peered":"2024-04-25T14:08:44.720640+0000","last_clean":"2024-04-25T14:08:44.720640+0000","last_became_active":"2024-04-25T14:08:44.717237+0000","last_became_peered":"2024-04-25T14:08:44.717237+0000","last_unstale":"2024-04-25T14:08:44.720640+0000","last_undegraded":"2024-04-25T14:08:44.720640+0000","last_fullsized":"2024-04-25T14:08:44.720640+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:39.391902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.698486+0000","last_change":"2024-04-25T14:08:44.698486+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.698486+0000","last_undegraded":"2024-04-25T14:08:44.698486+0000","last_fullsized":"2024-04-25T14:08:44.698486+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.726318+0000","last_change":"2024-04-25T14:08:44.723221+0000","last_active":"2024-04-25T14:08:44.726318+0000","last_peered":"2024-04-25T14:08:44.726318+0000","last_clean":"2024-04-25T14:08:44.726318+0000","last_became_active":"2024-04-25T14:08:44.722729+0000","last_became_peered":"2024-04-25T14:08:44.722729+0000","last_unstale":"2024-04-25T14:08:44.726318+0000","last_undegraded":"2024-04-25T14:08:44.726318+0000","last_fullsized":"2024-04-25T14:08:44.726318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:45.380736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.706660+0000","last_change":"2024-04-25T14:08:44.706660+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.706660+0000","last_undegraded":"2024-04-25T14:08:44.706660+0000","last_fullsized":"2024-04-25T14:08:44.706660+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.722005+0000","last_change":"2024-04-25T14:08:44.716853+0000","last_active":"2024-04-25T14:08:44.722005+0000","last_peered":"2024-04-25T14:08:44.722005+0000","last_clean":"2024-04-25T14:08:44.722005+0000","last_became_active":"2024-04-25T14:08:44.716479+0000","last_became_peered":"2024-04-25T14:08:44.716479+0000","last_unstale":"2024-04-25T14:08:44.722005+0000","last_undegraded":"2024-04-25T14:08:44.722005+0000","last_fullsized":"2024-04-25T14:08:44.722005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:04.500665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.701306+0000","last_change":"2024-04-25T14:08:44.701306+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.701306+0000","last_undegraded":"2024-04-25T14:08:44.701306+0000","last_fullsized":"2024-04-25T14:08:44.701306+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.731008+0000","last_change":"2024-04-25T14:08:44.726934+0000","last_active":"2024-04-25T14:08:44.731008+0000","last_peered":"2024-04-25T14:08:44.731008+0000","last_clean":"2024-04-25T14:08:44.731008+0000","last_became_active":"2024-04-25T14:08:44.726583+0000","last_became_peered":"2024-04-25T14:08:44.726583+0000","last_unstale":"2024-04-25T14:08:44.731008+0000","last_undegraded":"2024-04-25T14:08:44.731008+0000","last_fullsized":"2024-04-25T14:08:44.731008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:39.391902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.695980+0000","last_change":"2024-04-25T14:08:44.695980+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.695980+0000","last_undegraded":"2024-04-25T14:08:44.695980+0000","last_fullsized":"2024-04-25T14:08:44.695980+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.723891+0000","last_change":"2024-04-25T14:08:44.721083+0000","last_active":"2024-04-25T14:08:44.723891+0000","last_peered":"2024-04-25T14:08:44.723891+0000","last_clean":"2024-04-25T14:08:44.723891+0000","last_became_active":"2024-04-25T14:08:44.720775+0000","last_became_peered":"2024-04-25T14:08:44.720775+0000","last_unstale":"2024-04-25T14:08:44.723891+0000","last_undegraded":"2024-04-25T14:08:44.723891+0000","last_fullsized":"2024-04-25T14:08:44.723891+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:06.665120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.712919+0000","last_change":"2024-04-25T14:08:44.712919+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.712919+0000","last_undegraded":"2024-04-25T14:08:44.712919+0000","last_fullsized":"2024-04-25T14:08:44.712919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.725608+0000","last_change":"2024-04-25T14:08:44.723709+0000","last_active":"2024-04-25T14:08:44.725608+0000","last_peered":"2024-04-25T14:08:44.725608+0000","last_clean":"2024-04-25T14:08:44.725608+0000","last_became_active":"2024-04-25T14:08:44.723387+0000","last_became_peered":"2024-04-25T14:08:44.723387+0000","last_unstale":"2024-04-25T14:08:44.725608+0000","last_undegraded":"2024-04-25T14:08:44.725608+0000","last_fullsized":"2024-04-25T14:08:44.725608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:50.763491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.711073+0000","last_change":"2024-04-25T14:08:44.711073+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.711073+0000","last_undegraded":"2024-04-25T14:08:44.711073+0000","last_fullsized":"2024-04-25T14:08:44.711073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.729805+0000","last_change":"2024-04-25T14:08:44.728205+0000","last_active":"2024-04-25T14:08:44.729805+0000","last_peered":"2024-04-25T14:08:44.729805+0000","last_clean":"2024-04-25T14:08:44.729805+0000","last_became_active":"2024-04-25T14:08:44.727910+0000","last_became_peered":"2024-04-25T14:08:44.727910+0000","last_unstale":"2024-04-25T14:08:44.729805+0000","last_undegraded":"2024-04-25T14:08:44.729805+0000","last_fullsized":"2024-04-25T14:08:44.729805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:00.497301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.701371+0000","last_change":"2024-04-25T14:08:44.701371+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.701371+0000","last_undegraded":"2024-04-25T14:08:44.701371+0000","last_fullsized":"2024-04-25T14:08:44.701371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.715663+0000","last_change":"2024-04-25T14:08:44.713564+0000","last_active":"2024-04-25T14:08:44.715663+0000","last_peered":"2024-04-25T14:08:44.715663+0000","last_clean":"2024-04-25T14:08:44.715663+0000","last_became_active":"2024-04-25T14:08:44.712475+0000","last_became_peered":"2024-04-25T14:08:44.712475+0000","last_unstale":"2024-04-25T14:08:44.715663+0000","last_undegraded":"2024-04-25T14:08:44.715663+0000","last_fullsized":"2024-04-25T14:08:44.715663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:01.190413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.703954+0000","last_change":"2024-04-25T14:08:44.703954+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.703954+0000","last_undegraded":"2024-04-25T14:08:44.703954+0000","last_fullsized":"2024-04-25T14:08:44.703954+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.721141+0000","last_change":"2024-04-25T14:08:44.718093+0000","last_active":"2024-04-25T14:08:44.721141+0000","last_peered":"2024-04-25T14:08:44.721141+0000","last_clean":"2024-04-25T14:08:44.721141+0000","last_became_active":"2024-04-25T14:08:44.717768+0000","last_became_peered":"2024-04-25T14:08:44.717768+0000","last_unstale":"2024-04-25T14:08:44.721141+0000","last_undegraded":"2024-04-25T14:08:44.721141+0000","last_fullsized":"2024-04-25T14:08:44.721141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:10.155953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.700260+0000","last_change":"2024-04-25T14:08:44.700260+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.700260+0000","last_undegraded":"2024-04-25T14:08:44.700260+0000","last_fullsized":"2024-04-25T14:08:44.700260+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.720671+0000","last_change":"2024-04-25T14:08:44.717774+0000","last_active":"2024-04-25T14:08:44.720671+0000","last_peered":"2024-04-25T14:08:44.720671+0000","last_clean":"2024-04-25T14:08:44.720671+0000","last_became_active":"2024-04-25T14:08:44.717425+0000","last_became_peered":"2024-04-25T14:08:44.717425+0000","last_unstale":"2024-04-25T14:08:44.720671+0000","last_undegraded":"2024-04-25T14:08:44.720671+0000","last_fullsized":"2024-04-25T14:08:44.720671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:55.423236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.703371+0000","last_change":"2024-04-25T14:08:44.703371+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.703371+0000","last_undegraded":"2024-04-25T14:08:44.703371+0000","last_fullsized":"2024-04-25T14:08:44.703371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.703242+0000","last_change":"2024-04-25T14:08:44.700728+0000","last_active":"2024-04-25T14:08:44.703242+0000","last_peered":"2024-04-25T14:08:44.703242+0000","last_clean":"2024-04-25T14:08:44.703242+0000","last_became_active":"2024-04-25T14:08:44.700344+0000","last_became_peered":"2024-04-25T14:08:44.700344+0000","last_unstale":"2024-04-25T14:08:44.703242+0000","last_undegraded":"2024-04-25T14:08:44.703242+0000","last_fullsized":"2024-04-25T14:08:44.703242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:36:34.689927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.694919+0000","last_change":"2024-04-25T14:08:44.694919+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.694919+0000","last_undegraded":"2024-04-25T14:08:44.694919+0000","last_fullsized":"2024-04-25T14:08:44.694919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.727899+0000","last_change":"2024-04-25T14:08:44.725383+0000","last_active":"2024-04-25T14:08:44.727899+0000","last_peered":"2024-04-25T14:08:44.727899+0000","last_clean":"2024-04-25T14:08:44.727899+0000","last_became_active":"2024-04-25T14:08:44.724211+0000","last_became_peered":"2024-04-25T14:08:44.724211+0000","last_unstale":"2024-04-25T14:08:44.727899+0000","last_undegraded":"2024-04-25T14:08:44.727899+0000","last_fullsized":"2024-04-25T14:08:44.727899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:16.246992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.694866+0000","last_change":"2024-04-25T14:08:44.694866+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.694866+0000","last_undegraded":"2024-04-25T14:08:44.694866+0000","last_fullsized":"2024-04-25T14:08:44.694866+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.719715+0000","last_change":"2024-04-25T14:08:44.716954+0000","last_active":"2024-04-25T14:08:44.719715+0000","last_peered":"2024-04-25T14:08:44.719715+0000","last_clean":"2024-04-25T14:08:44.719715+0000","last_became_active":"2024-04-25T14:08:44.716583+0000","last_became_peered":"2024-04-25T14:08:44.716583+0000","last_unstale":"2024-04-25T14:08:44.719715+0000","last_undegraded":"2024-04-25T14:08:44.719715+0000","last_fullsized":"2024-04-25T14:08:44.719715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:31.019299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.705709+0000","last_change":"2024-04-25T14:08:44.705709+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.705709+0000","last_undegraded":"2024-04-25T14:08:44.705709+0000","last_fullsized":"2024-04-25T14:08:44.705709+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.727980+0000","last_change":"2024-04-25T14:08:44.725423+0000","last_active":"2024-04-25T14:08:44.727980+0000","last_peered":"2024-04-25T14:08:44.727980+0000","last_clean":"2024-04-25T14:08:44.727980+0000","last_became_active":"2024-04-25T14:08:44.725036+0000","last_became_peered":"2024-04-25T14:08:44.725036+0000","last_unstale":"2024-04-25T14:08:44.727980+0000","last_undegraded":"2024-04-25T14:08:44.727980+0000","last_fullsized":"2024-04-25T14:08:44.727980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:45.210808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.703583+0000","last_change":"2024-04-25T14:08:44.703583+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.703583+0000","last_undegraded":"2024-04-25T14:08:44.703583+0000","last_fullsized":"2024-04-25T14:08:44.703583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.720288+0000","last_change":"2024-04-25T14:08:44.717040+0000","last_active":"2024-04-25T14:08:44.720288+0000","last_peered":"2024-04-25T14:08:44.720288+0000","last_clean":"2024-04-25T14:08:44.720288+0000","last_became_active":"2024-04-25T14:08:44.716044+0000","last_became_peered":"2024-04-25T14:08:44.716044+0000","last_unstale":"2024-04-25T14:08:44.720288+0000","last_undegraded":"2024-04-25T14:08:44.720288+0000","last_fullsized":"2024-04-25T14:08:44.720288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:41.020517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.712548+0000","last_change":"2024-04-25T14:08:44.712548+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.712548+0000","last_undegraded":"2024-04-25T14:08:44.712548+0000","last_fullsized":"2024-04-25T14:08:44.712548+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.710784+0000","last_change":"2024-04-25T14:08:44.704043+0000","last_active":"2024-04-25T14:08:44.710784+0000","last_peered":"2024-04-25T14:08:44.710784+0000","last_clean":"2024-04-25T14:08:44.710784+0000","last_became_active":"2024-04-25T14:08:44.703697+0000","last_became_peered":"2024-04-25T14:08:44.703697+0000","last_unstale":"2024-04-25T14:08:44.710784+0000","last_undegraded":"2024-04-25T14:08:44.710784+0000","last_fullsized":"2024-04-25T14:08:44.710784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:42.844372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.704587+0000","last_change":"2024-04-25T14:08:44.704587+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.704587+0000","last_undegraded":"2024-04-25T14:08:44.704587+0000","last_fullsized":"2024-04-25T14:08:44.704587+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.722459+0000","last_change":"2024-04-25T14:08:44.721286+0000","last_active":"2024-04-25T14:08:44.722459+0000","last_peered":"2024-04-25T14:08:44.722459+0000","last_clean":"2024-04-25T14:08:44.722459+0000","last_became_active":"2024-04-25T14:08:44.720949+0000","last_became_peered":"2024-04-25T14:08:44.720949+0000","last_unstale":"2024-04-25T14:08:44.722459+0000","last_undegraded":"2024-04-25T14:08:44.722459+0000","last_fullsized":"2024-04-25T14:08:44.722459+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:58:08.650136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.695843+0000","last_change":"2024-04-25T14:08:44.695843+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.695843+0000","last_undegraded":"2024-04-25T14:08:44.695843+0000","last_fullsized":"2024-04-25T14:08:44.695843+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.728386+0000","last_change":"2024-04-25T14:08:44.725093+0000","last_active":"2024-04-25T14:08:44.728386+0000","last_peered":"2024-04-25T14:08:44.728386+0000","last_clean":"2024-04-25T14:08:44.728386+0000","last_became_active":"2024-04-25T14:08:44.724761+0000","last_became_peered":"2024-04-25T14:08:44.724761+0000","last_unstale":"2024-04-25T14:08:44.728386+0000","last_undegraded":"2024-04-25T14:08:44.728386+0000","last_fullsized":"2024-04-25T14:08:44.728386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:37.926762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.713062+0000","last_change":"2024-04-25T14:08:44.713062+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.713062+0000","last_undegraded":"2024-04-25T14:08:44.713062+0000","last_fullsized":"2024-04-25T14:08:44.713062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.723865+0000","last_change":"2024-04-25T14:08:44.721282+0000","last_active":"2024-04-25T14:08:44.723865+0000","last_peered":"2024-04-25T14:08:44.723865+0000","last_clean":"2024-04-25T14:08:44.723865+0000","last_became_active":"2024-04-25T14:08:44.721077+0000","last_became_peered":"2024-04-25T14:08:44.721077+0000","last_unstale":"2024-04-25T14:08:44.723865+0000","last_undegraded":"2024-04-25T14:08:44.723865+0000","last_fullsized":"2024-04-25T14:08:44.723865+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:07.051812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.698403+0000","last_change":"2024-04-25T14:08:44.698403+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.698403+0000","last_undegraded":"2024-04-25T14:08:44.698403+0000","last_fullsized":"2024-04-25T14:08:44.698403+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.704667+0000","last_change":"2024-04-25T14:08:44.701347+0000","last_active":"2024-04-25T14:08:44.704667+0000","last_peered":"2024-04-25T14:08:44.704667+0000","last_clean":"2024-04-25T14:08:44.704667+0000","last_became_active":"2024-04-25T14:08:44.700917+0000","last_became_peered":"2024-04-25T14:08:44.700917+0000","last_unstale":"2024-04-25T14:08:44.704667+0000","last_undegraded":"2024-04-25T14:08:44.704667+0000","last_fullsized":"2024-04-25T14:08:44.704667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:49.250431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.702541+0000","last_change":"2024-04-25T14:08:44.702541+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.702541+0000","last_undegraded":"2024-04-25T14:08:44.702541+0000","last_fullsized":"2024-04-25T14:08:44.702541+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.728341+0000","last_change":"2024-04-25T14:08:44.724767+0000","last_active":"2024-04-25T14:08:44.728341+0000","last_peered":"2024-04-25T14:08:44.728341+0000","last_clean":"2024-04-25T14:08:44.728341+0000","last_became_active":"2024-04-25T14:08:44.724442+0000","last_became_peered":"2024-04-25T14:08:44.724442+0000","last_unstale":"2024-04-25T14:08:44.728341+0000","last_undegraded":"2024-04-25T14:08:44.728341+0000","last_fullsized":"2024-04-25T14:08:44.728341+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:56.689895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.705244+0000","last_change":"2024-04-25T14:08:44.705244+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.705244+0000","last_undegraded":"2024-04-25T14:08:44.705244+0000","last_fullsized":"2024-04-25T14:08:44.705244+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.726004+0000","last_change":"2024-04-25T14:08:44.723570+0000","last_active":"2024-04-25T14:08:44.726004+0000","last_peered":"2024-04-25T14:08:44.726004+0000","last_clean":"2024-04-25T14:08:44.726004+0000","last_became_active":"2024-04-25T14:08:44.722656+0000","last_became_peered":"2024-04-25T14:08:44.722656+0000","last_unstale":"2024-04-25T14:08:44.726004+0000","last_undegraded":"2024-04-25T14:08:44.726004+0000","last_fullsized":"2024-04-25T14:08:44.726004+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:23.016722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.708517+0000","last_change":"2024-04-25T14:08:44.708517+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.708517+0000","last_undegraded":"2024-04-25T14:08:44.708517+0000","last_fullsized":"2024-04-25T14:08:44.708517+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.723217+0000","last_change":"2024-04-25T14:08:44.719472+0000","last_active":"2024-04-25T14:08:44.723217+0000","last_peered":"2024-04-25T14:08:44.723217+0000","last_clean":"2024-04-25T14:08:44.723217+0000","last_became_active":"2024-04-25T14:08:44.719159+0000","last_became_peered":"2024-04-25T14:08:44.719159+0000","last_unstale":"2024-04-25T14:08:44.723217+0000","last_undegraded":"2024-04-25T14:08:44.723217+0000","last_fullsized":"2024-04-25T14:08:44.723217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:15.094910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"76'100","reported_seq":329,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.711790+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:08:44.711790+0000","last_peered":"2024-04-25T14:08:44.711790+0000","last_clean":"2024-04-25T14:08:44.711790+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:08:44.711790+0000","last_undegraded":"2024-04-25T14:08:44.711790+0000","last_fullsized":"2024-04-25T14:08:44.711790+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":306,"num_read_kb":759,"num_write":149,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":306,"num_read_kb":759,"num_write":149,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574909,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32788,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339052,"statfs":{"total":96636764160,"available":96603189248,"internally_reserved":0,"allocated":1642496,"data_stored":1110288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1048576,"data_stored":519920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574911,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1048576,"data_stored":519920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1048576,"data_stored":519920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1036288,"data_stored":513253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1048576,"data_stored":519920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1048576,"data_stored":519920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574910,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32772,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339068,"statfs":{"total":96636764160,"available":96603205632,"internally_reserved":0,"allocated":1630208,"data_stored":1103621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:08:50.306 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:08:50.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.490+0000 7ffa42200640 1 Processor -- start 2024-04-25T14:08:50.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.490+0000 7ffa42200640 1 -- start start 2024-04-25T14:08:50.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.490+0000 7ffa42200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffa3c070330 0x7ffa3c0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:50.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.490+0000 7ffa42200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa3c0d2030 con 0x7ffa3c0ce790 2024-04-25T14:08:50.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.490+0000 7ffa42200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa3c15e5c0 con 0x7ffa3c0d1c20 2024-04-25T14:08:50.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.490+0000 7ffa42200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ffa3c15e8a0 con 0x7ffa3c070330 2024-04-25T14:08:50.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.490+0000 7ffa3b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffa3c070330 0x7ffa3c0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:50.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.490+0000 7ffa3be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ffa3c0ce790 0x7ffa3c06f440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49392/0 (socket says 172.21.15.86:49392) 2024-04-25T14:08:50.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.490+0000 7ffa3b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffa3c070330 0x7ffa3c0d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40306/0 (socket says 172.21.15.86:40306) 2024-04-25T14:08:50.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.490+0000 7ffa3be00640 1 -- 172.21.15.86:0/224146 learned_addr learned my addr 172.21.15.86:0/224146 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:50.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.490+0000 7ffa3b400640 1 -- 172.21.15.86:0/224146 >> v1:172.21.15.155:6789/0 conn(0x7ffa3c0d1c20 legacy=0x7ffa3c06ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:50.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.490+0000 7ffa3b400640 1 -- 172.21.15.86:0/224146 >> v1:172.21.15.86:6789/0 conn(0x7ffa3c0ce790 legacy=0x7ffa3c06f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:50.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.490+0000 7ffa3b400640 1 -- 172.21.15.86:0/224146 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3c15eb80 con 0x7ffa3c070330 2024-04-25T14:08:50.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.494+0000 7ffa3b400640 1 --2- 172.21.15.86:0/224146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa3c070330 0x7ffa3c0d16e0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009dc0 tx=0x7ffa2c01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7f482e3ebdc98c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:50.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.494+0000 7ffa3aa00640 1 -- 172.21.15.86:0/224146 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c0b2070 con 0x7ffa3c070330 2024-04-25T14:08:50.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.494+0000 7ffa3aa00640 1 -- 172.21.15.86:0/224146 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa2c0a5850 con 0x7ffa3c070330 2024-04-25T14:08:50.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.494+0000 7ffa3aa00640 1 -- 172.21.15.86:0/224146 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c0a5c20 con 0x7ffa3c070330 2024-04-25T14:08:50.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.494+0000 7ffa42200640 1 -- 172.21.15.86:0/224146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa3c070330 msgr2=0x7ffa3c0d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:50.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.494+0000 7ffa42200640 1 --2- 172.21.15.86:0/224146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa3c070330 0x7ffa3c0d16e0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009dc0 tx=0x7ffa2c01c8c0 comp rx=0 tx=0).stop 2024-04-25T14:08:50.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.494+0000 7ffa42200640 1 -- 172.21.15.86:0/224146 shutdown_connections 2024-04-25T14:08:50.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.494+0000 7ffa42200640 1 --2- 172.21.15.86:0/224146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa3c070330 0x7ffa3c0d16e0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:50.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.494+0000 7ffa42200640 1 -- 172.21.15.86:0/224146 >> 172.21.15.86:0/224146 conn(0x7ffa3c0b4650 msgr2=0x7ffa3c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:50.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.494+0000 7ffa42200640 1 -- 172.21.15.86:0/224146 shutdown_connections 2024-04-25T14:08:50.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.494+0000 7ffa42200640 1 -- 172.21.15.86:0/224146 wait complete. 2024-04-25T14:08:50.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.498+0000 7ffa42200640 1 Processor -- start 2024-04-25T14:08:50.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.498+0000 7ffa42200640 1 -- start start 2024-04-25T14:08:50.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.498+0000 7ffa42200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa3c070330 0x7ffa3c0ae130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:50.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.498+0000 7ffa42200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa3c0ce790 0x7ffa3c0c2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:50.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.498+0000 7ffa42200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa3c0d1c20 0x7ffa3c0c3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:50.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.498+0000 7ffa42200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c0708c0 con 0x7ffa3c0d1c20 2024-04-25T14:08:50.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.498+0000 7ffa42200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c070740 con 0x7ffa3c0ce790 2024-04-25T14:08:50.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.498+0000 7ffa42200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ffa3c06fb50 con 0x7ffa3c070330 2024-04-25T14:08:50.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.498+0000 7ffa3be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa3c0d1c20 0x7ffa3c0c3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:50.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.498+0000 7ffa3be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa3c0d1c20 0x7ffa3c0c3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40322/0 (socket says 172.21.15.86:40322) 2024-04-25T14:08:50.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.498+0000 7ffa3be00640 1 -- 172.21.15.86:0/857572018 learned_addr learned my addr 172.21.15.86:0/857572018 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:50.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.502+0000 7ffa3b400640 1 --2- 172.21.15.86:0/857572018 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa3c070330 0x7ffa3c0ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:50.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.502+0000 7ffa32a00640 1 --2- 172.21.15.86:0/857572018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa3c0ce790 0x7ffa3c0c2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:50.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.502+0000 7ffa3be00640 1 -- 172.21.15.86:0/857572018 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa3c070330 msgr2=0x7ffa3c0ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:50.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.502+0000 7ffa3be00640 1 --2- 172.21.15.86:0/857572018 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa3c070330 0x7ffa3c0ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:50.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.502+0000 7ffa3be00640 1 -- 172.21.15.86:0/857572018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa3c0ce790 msgr2=0x7ffa3c0c2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:50.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.502+0000 7ffa3be00640 1 --2- 172.21.15.86:0/857572018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa3c0ce790 0x7ffa3c0c2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:50.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.502+0000 7ffa3be00640 1 -- 172.21.15.86:0/857572018 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3c0aea60 con 0x7ffa3c0d1c20 2024-04-25T14:08:50.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.502+0000 7ffa3be00640 1 --2- 172.21.15.86:0/857572018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa3c0d1c20 0x7ffa3c0c3040 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400c3c0 tx=0x7ffa3400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:50.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.502+0000 7ffa33e00640 1 -- 172.21.15.86:0/857572018 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa34017070 con 0x7ffa3c0d1c20 2024-04-25T14:08:50.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.502+0000 7ffa33e00640 1 -- 172.21.15.86:0/857572018 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa34005cf0 con 0x7ffa3c0d1c20 2024-04-25T14:08:50.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.502+0000 7ffa33e00640 1 -- 172.21.15.86:0/857572018 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa340072f0 con 0x7ffa3c0d1c20 2024-04-25T14:08:50.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.502+0000 7ffa42200640 1 -- 172.21.15.86:0/857572018 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa3c0aed40 con 0x7ffa3c0d1c20 2024-04-25T14:08:50.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.502+0000 7ffa42200640 1 -- 172.21.15.86:0/857572018 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa3c0af300 con 0x7ffa3c0d1c20 2024-04-25T14:08:50.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.502+0000 7ffa42200640 1 -- 172.21.15.86:0/857572018 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa3c069d00 con 0x7ffa3c0d1c20 2024-04-25T14:08:50.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.502+0000 7ffa33e00640 1 -- 172.21.15.86:0/857572018 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7ffa3401a070 con 0x7ffa3c0d1c20 2024-04-25T14:08:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.506+0000 7ffa33e00640 1 --2- 172.21.15.86:0/857572018 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7ffa1806f520 0x7ffa18071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.506+0000 7ffa33e00640 1 -- 172.21.15.86:0/857572018 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7ffa3408ea00 con 0x7ffa3c0d1c20 2024-04-25T14:08:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.506+0000 7ffa3b400640 1 --2- 172.21.15.86:0/857572018 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7ffa1806f520 0x7ffa18071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:50.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.506+0000 7ffa3b400640 1 --2- 172.21.15.86:0/857572018 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7ffa1806f520 0x7ffa18071a30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c0af000 tx=0x7ffa2c0a7070 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:50.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.510+0000 7ffa33e00640 1 -- 172.21.15.86:0/857572018 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffa3c0d2030 con 0x7ffa3c0d1c20 2024-04-25T14:08:50.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.662+0000 7ffa42200640 1 -- 172.21.15.86:0/857572018 --> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffa3c076bf0 con 0x7ffa1806f520 2024-04-25T14:08:50.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.670+0000 7ffa33e00640 1 -- 172.21.15.86:0/857572018 <== mgr.7092 v2:172.21.15.86:6832/3589556609 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246763 (secure 0 0 0) 0x7ffa3c076bf0 con 0x7ffa1806f520 2024-04-25T14:08:50.674 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:08:50.676 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:08:50.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.674+0000 7ffa42200640 1 -- 172.21.15.86:0/857572018 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7ffa1806f520 msgr2=0x7ffa18071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:50.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.674+0000 7ffa42200640 1 --2- 172.21.15.86:0/857572018 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7ffa1806f520 0x7ffa18071a30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c0af000 tx=0x7ffa2c0a7070 comp rx=0 tx=0).stop 2024-04-25T14:08:50.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.674+0000 7ffa42200640 1 -- 172.21.15.86:0/857572018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa3c0d1c20 msgr2=0x7ffa3c0c3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:50.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.674+0000 7ffa42200640 1 --2- 172.21.15.86:0/857572018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa3c0d1c20 0x7ffa3c0c3040 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400c3c0 tx=0x7ffa3400c880 comp rx=0 tx=0).stop 2024-04-25T14:08:50.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.674+0000 7ffa42200640 1 -- 172.21.15.86:0/857572018 shutdown_connections 2024-04-25T14:08:50.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.674+0000 7ffa42200640 1 --2- 172.21.15.86:0/857572018 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7ffa1806f520 0x7ffa18071a30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:50.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.674+0000 7ffa42200640 1 --2- 172.21.15.86:0/857572018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa3c0d1c20 0x7ffa3c0c3040 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:50.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.674+0000 7ffa42200640 1 --2- 172.21.15.86:0/857572018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa3c0ce790 0x7ffa3c0c2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:50.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.674+0000 7ffa42200640 1 --2- 172.21.15.86:0/857572018 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa3c070330 0x7ffa3c0ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:50.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.674+0000 7ffa42200640 1 -- 172.21.15.86:0/857572018 >> 172.21.15.86:0/857572018 conn(0x7ffa3c0b4650 msgr2=0x7ffa3c156c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:50.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.674+0000 7ffa42200640 1 -- 172.21.15.86:0/857572018 shutdown_connections 2024-04-25T14:08:50.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.674+0000 7ffa42200640 1 -- 172.21.15.86:0/857572018 wait complete. 2024-04-25T14:08:50.695 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T14:08:49.359300+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":306,"num_read_kb":759,"num_write":149,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258792,"kb_used_data":9328,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715928,"statfs":{"total":773094113280,"available":772829110272,"internally_reserved":0,"allocated":9551872,"data_stored":5326762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.671432"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.700729+0000","last_change":"2024-04-25T14:08:44.700729+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.700729+0000","last_undegraded":"2024-04-25T14:08:44.700729+0000","last_fullsized":"2024-04-25T14:08:44.700729+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.695217+0000","last_change":"2024-04-25T14:08:44.695217+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.695217+0000","last_undegraded":"2024-04-25T14:08:44.695217+0000","last_fullsized":"2024-04-25T14:08:44.695217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.700715+0000","last_change":"2024-04-25T14:08:44.700715+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.700715+0000","last_undegraded":"2024-04-25T14:08:44.700715+0000","last_fullsized":"2024-04-25T14:08:44.700715+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.707535+0000","last_change":"2024-04-25T14:08:44.707535+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.707535+0000","last_undegraded":"2024-04-25T14:08:44.707535+0000","last_fullsized":"2024-04-25T14:08:44.707535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.696533+0000","last_change":"2024-04-25T14:08:44.696533+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.696533+0000","last_undegraded":"2024-04-25T14:08:44.696533+0000","last_fullsized":"2024-04-25T14:08:44.696533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.715602+0000","last_change":"2024-04-25T14:08:44.715602+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.715602+0000","last_undegraded":"2024-04-25T14:08:44.715602+0000","last_fullsized":"2024-04-25T14:08:44.715602+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.705515+0000","last_change":"2024-04-25T14:08:44.705515+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.705515+0000","last_undegraded":"2024-04-25T14:08:44.705515+0000","last_fullsized":"2024-04-25T14:08:44.705515+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.704048+0000","last_change":"2024-04-25T14:08:44.704048+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.704048+0000","last_undegraded":"2024-04-25T14:08:44.704048+0000","last_fullsized":"2024-04-25T14:08:44.704048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.701272+0000","last_change":"2024-04-25T14:08:44.701272+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.701272+0000","last_undegraded":"2024-04-25T14:08:44.701272+0000","last_fullsized":"2024-04-25T14:08:44.701272+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.709510+0000","last_change":"2024-04-25T14:08:44.709510+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.709510+0000","last_undegraded":"2024-04-25T14:08:44.709510+0000","last_fullsized":"2024-04-25T14:08:44.709510+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.699606+0000","last_change":"2024-04-25T14:08:44.699606+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.699606+0000","last_undegraded":"2024-04-25T14:08:44.699606+0000","last_fullsized":"2024-04-25T14:08:44.699606+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.711227+0000","last_change":"2024-04-25T14:08:44.711227+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.711227+0000","last_undegraded":"2024-04-25T14:08:44.711227+0000","last_fullsized":"2024-04-25T14:08:44.711227+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.702941+0000","last_change":"2024-04-25T14:08:44.702941+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.702941+0000","last_undegraded":"2024-04-25T14:08:44.702941+0000","last_fullsized":"2024-04-25T14:08:44.702941+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.699462+0000","last_change":"2024-04-25T14:08:44.699462+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.699462+0000","last_undegraded":"2024-04-25T14:08:44.699462+0000","last_fullsized":"2024-04-25T14:08:44.699462+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.702169+0000","last_change":"2024-04-25T14:08:44.702169+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.702169+0000","last_undegraded":"2024-04-25T14:08:44.702169+0000","last_fullsized":"2024-04-25T14:08:44.702169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.708333+0000","last_change":"2024-04-25T14:08:44.708333+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.708333+0000","last_undegraded":"2024-04-25T14:08:44.708333+0000","last_fullsized":"2024-04-25T14:08:44.708333+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.704663+0000","last_change":"2024-04-25T14:08:44.704663+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.704663+0000","last_undegraded":"2024-04-25T14:08:44.704663+0000","last_fullsized":"2024-04-25T14:08:44.704663+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.700154+0000","last_change":"2024-04-25T14:08:44.700154+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.700154+0000","last_undegraded":"2024-04-25T14:08:44.700154+0000","last_fullsized":"2024-04-25T14:08:44.700154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.699078+0000","last_change":"2024-04-25T14:08:44.699078+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.699078+0000","last_undegraded":"2024-04-25T14:08:44.699078+0000","last_fullsized":"2024-04-25T14:08:44.699078+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.707069+0000","last_change":"2024-04-25T14:08:44.707069+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.707069+0000","last_undegraded":"2024-04-25T14:08:44.707069+0000","last_fullsized":"2024-04-25T14:08:44.707069+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.700474+0000","last_change":"2024-04-25T14:08:44.700474+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.700474+0000","last_undegraded":"2024-04-25T14:08:44.700474+0000","last_fullsized":"2024-04-25T14:08:44.700474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.698892+0000","last_change":"2024-04-25T14:08:44.698892+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.698892+0000","last_undegraded":"2024-04-25T14:08:44.698892+0000","last_fullsized":"2024-04-25T14:08:44.698892+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.702950+0000","last_change":"2024-04-25T14:08:44.702950+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.702950+0000","last_undegraded":"2024-04-25T14:08:44.702950+0000","last_fullsized":"2024-04-25T14:08:44.702950+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.698720+0000","last_change":"2024-04-25T14:08:44.698720+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.698720+0000","last_undegraded":"2024-04-25T14:08:44.698720+0000","last_fullsized":"2024-04-25T14:08:44.698720+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.696352+0000","last_change":"2024-04-25T14:08:44.696352+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.696352+0000","last_undegraded":"2024-04-25T14:08:44.696352+0000","last_fullsized":"2024-04-25T14:08:44.696352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.700875+0000","last_change":"2024-04-25T14:08:44.700875+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.700875+0000","last_undegraded":"2024-04-25T14:08:44.700875+0000","last_fullsized":"2024-04-25T14:08:44.700875+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.709537+0000","last_change":"2024-04-25T14:08:44.709537+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.709537+0000","last_undegraded":"2024-04-25T14:08:44.709537+0000","last_fullsized":"2024-04-25T14:08:44.709537+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.702856+0000","last_change":"2024-04-25T14:08:44.702856+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.702856+0000","last_undegraded":"2024-04-25T14:08:44.702856+0000","last_fullsized":"2024-04-25T14:08:44.702856+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.704801+0000","last_change":"2024-04-25T14:08:44.704801+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.704801+0000","last_undegraded":"2024-04-25T14:08:44.704801+0000","last_fullsized":"2024-04-25T14:08:44.704801+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.695641+0000","last_change":"2024-04-25T14:08:44.695641+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.695641+0000","last_undegraded":"2024-04-25T14:08:44.695641+0000","last_fullsized":"2024-04-25T14:08:44.695641+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.695624+0000","last_change":"2024-04-25T14:08:44.695624+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.695624+0000","last_undegraded":"2024-04-25T14:08:44.695624+0000","last_fullsized":"2024-04-25T14:08:44.695624+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.707878+0000","last_change":"2024-04-25T14:08:44.707878+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.707878+0000","last_undegraded":"2024-04-25T14:08:44.707878+0000","last_fullsized":"2024-04-25T14:08:44.707878+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.701936+0000","last_change":"2024-04-25T14:08:44.701936+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.701936+0000","last_undegraded":"2024-04-25T14:08:44.701936+0000","last_fullsized":"2024-04-25T14:08:44.701936+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.728427+0000","last_change":"2024-04-25T14:08:44.725130+0000","last_active":"2024-04-25T14:08:44.728427+0000","last_peered":"2024-04-25T14:08:44.728427+0000","last_clean":"2024-04-25T14:08:44.728427+0000","last_became_active":"2024-04-25T14:08:44.724805+0000","last_became_peered":"2024-04-25T14:08:44.724805+0000","last_unstale":"2024-04-25T14:08:44.728427+0000","last_undegraded":"2024-04-25T14:08:44.728427+0000","last_fullsized":"2024-04-25T14:08:44.728427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:58.834978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.713037+0000","last_change":"2024-04-25T14:08:44.713037+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.713037+0000","last_undegraded":"2024-04-25T14:08:44.713037+0000","last_fullsized":"2024-04-25T14:08:44.713037+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.714929+0000","last_change":"2024-04-25T14:08:44.713248+0000","last_active":"2024-04-25T14:08:44.714929+0000","last_peered":"2024-04-25T14:08:44.714929+0000","last_clean":"2024-04-25T14:08:44.714929+0000","last_became_active":"2024-04-25T14:08:44.712237+0000","last_became_peered":"2024-04-25T14:08:44.712237+0000","last_unstale":"2024-04-25T14:08:44.714929+0000","last_undegraded":"2024-04-25T14:08:44.714929+0000","last_fullsized":"2024-04-25T14:08:44.714929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:14.643131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.704564+0000","last_change":"2024-04-25T14:08:44.704564+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.704564+0000","last_undegraded":"2024-04-25T14:08:44.704564+0000","last_fullsized":"2024-04-25T14:08:44.704564+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.728516+0000","last_change":"2024-04-25T14:08:44.725153+0000","last_active":"2024-04-25T14:08:44.728516+0000","last_peered":"2024-04-25T14:08:44.728516+0000","last_clean":"2024-04-25T14:08:44.728516+0000","last_became_active":"2024-04-25T14:08:44.724848+0000","last_became_peered":"2024-04-25T14:08:44.724848+0000","last_unstale":"2024-04-25T14:08:44.728516+0000","last_undegraded":"2024-04-25T14:08:44.728516+0000","last_fullsized":"2024-04-25T14:08:44.728516+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:28.574820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.704170+0000","last_change":"2024-04-25T14:08:44.704170+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.704170+0000","last_undegraded":"2024-04-25T14:08:44.704170+0000","last_fullsized":"2024-04-25T14:08:44.704170+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.721669+0000","last_change":"2024-04-25T14:08:44.720784+0000","last_active":"2024-04-25T14:08:44.721669+0000","last_peered":"2024-04-25T14:08:44.721669+0000","last_clean":"2024-04-25T14:08:44.721669+0000","last_became_active":"2024-04-25T14:08:44.720248+0000","last_became_peered":"2024-04-25T14:08:44.720248+0000","last_unstale":"2024-04-25T14:08:44.721669+0000","last_undegraded":"2024-04-25T14:08:44.721669+0000","last_fullsized":"2024-04-25T14:08:44.721669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:00.497301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.701864+0000","last_change":"2024-04-25T14:08:44.701864+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.701864+0000","last_undegraded":"2024-04-25T14:08:44.701864+0000","last_fullsized":"2024-04-25T14:08:44.701864+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.730802+0000","last_change":"2024-04-25T14:08:44.727021+0000","last_active":"2024-04-25T14:08:44.730802+0000","last_peered":"2024-04-25T14:08:44.730802+0000","last_clean":"2024-04-25T14:08:44.730802+0000","last_became_active":"2024-04-25T14:08:44.726079+0000","last_became_peered":"2024-04-25T14:08:44.726079+0000","last_unstale":"2024-04-25T14:08:44.730802+0000","last_undegraded":"2024-04-25T14:08:44.730802+0000","last_fullsized":"2024-04-25T14:08:44.730802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:18.045986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.701896+0000","last_change":"2024-04-25T14:08:44.701896+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.701896+0000","last_undegraded":"2024-04-25T14:08:44.701896+0000","last_fullsized":"2024-04-25T14:08:44.701896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.728800+0000","last_change":"2024-04-25T14:08:44.726268+0000","last_active":"2024-04-25T14:08:44.728800+0000","last_peered":"2024-04-25T14:08:44.728800+0000","last_clean":"2024-04-25T14:08:44.728800+0000","last_became_active":"2024-04-25T14:08:44.725839+0000","last_became_peered":"2024-04-25T14:08:44.725839+0000","last_unstale":"2024-04-25T14:08:44.728800+0000","last_undegraded":"2024-04-25T14:08:44.728800+0000","last_fullsized":"2024-04-25T14:08:44.728800+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:01.190413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.711108+0000","last_change":"2024-04-25T14:08:44.711108+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.711108+0000","last_undegraded":"2024-04-25T14:08:44.711108+0000","last_fullsized":"2024-04-25T14:08:44.711108+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.717869+0000","last_change":"2024-04-25T14:08:44.712147+0000","last_active":"2024-04-25T14:08:44.717869+0000","last_peered":"2024-04-25T14:08:44.717869+0000","last_clean":"2024-04-25T14:08:44.717869+0000","last_became_active":"2024-04-25T14:08:44.711799+0000","last_became_peered":"2024-04-25T14:08:44.711799+0000","last_unstale":"2024-04-25T14:08:44.717869+0000","last_undegraded":"2024-04-25T14:08:44.717869+0000","last_fullsized":"2024-04-25T14:08:44.717869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:28.574820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.702431+0000","last_change":"2024-04-25T14:08:44.702431+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.702431+0000","last_undegraded":"2024-04-25T14:08:44.702431+0000","last_fullsized":"2024-04-25T14:08:44.702431+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.726027+0000","last_change":"2024-04-25T14:08:44.723883+0000","last_active":"2024-04-25T14:08:44.726027+0000","last_peered":"2024-04-25T14:08:44.726027+0000","last_clean":"2024-04-25T14:08:44.726027+0000","last_became_active":"2024-04-25T14:08:44.723229+0000","last_became_peered":"2024-04-25T14:08:44.723229+0000","last_unstale":"2024-04-25T14:08:44.726027+0000","last_undegraded":"2024-04-25T14:08:44.726027+0000","last_fullsized":"2024-04-25T14:08:44.726027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:45.472144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.697949+0000","last_change":"2024-04-25T14:08:44.697949+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.697949+0000","last_undegraded":"2024-04-25T14:08:44.697949+0000","last_fullsized":"2024-04-25T14:08:44.697949+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.726787+0000","last_change":"2024-04-25T14:08:44.723981+0000","last_active":"2024-04-25T14:08:44.726787+0000","last_peered":"2024-04-25T14:08:44.726787+0000","last_clean":"2024-04-25T14:08:44.726787+0000","last_became_active":"2024-04-25T14:08:44.723657+0000","last_became_peered":"2024-04-25T14:08:44.723657+0000","last_unstale":"2024-04-25T14:08:44.726787+0000","last_undegraded":"2024-04-25T14:08:44.726787+0000","last_fullsized":"2024-04-25T14:08:44.726787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:51.117327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.705995+0000","last_change":"2024-04-25T14:08:44.705995+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.705995+0000","last_undegraded":"2024-04-25T14:08:44.705995+0000","last_fullsized":"2024-04-25T14:08:44.705995+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.720640+0000","last_change":"2024-04-25T14:08:44.718626+0000","last_active":"2024-04-25T14:08:44.720640+0000","last_peered":"2024-04-25T14:08:44.720640+0000","last_clean":"2024-04-25T14:08:44.720640+0000","last_became_active":"2024-04-25T14:08:44.717237+0000","last_became_peered":"2024-04-25T14:08:44.717237+0000","last_unstale":"2024-04-25T14:08:44.720640+0000","last_undegraded":"2024-04-25T14:08:44.720640+0000","last_fullsized":"2024-04-25T14:08:44.720640+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:39.391902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.698486+0000","last_change":"2024-04-25T14:08:44.698486+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.698486+0000","last_undegraded":"2024-04-25T14:08:44.698486+0000","last_fullsized":"2024-04-25T14:08:44.698486+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.726318+0000","last_change":"2024-04-25T14:08:44.723221+0000","last_active":"2024-04-25T14:08:44.726318+0000","last_peered":"2024-04-25T14:08:44.726318+0000","last_clean":"2024-04-25T14:08:44.726318+0000","last_became_active":"2024-04-25T14:08:44.722729+0000","last_became_peered":"2024-04-25T14:08:44.722729+0000","last_unstale":"2024-04-25T14:08:44.726318+0000","last_undegraded":"2024-04-25T14:08:44.726318+0000","last_fullsized":"2024-04-25T14:08:44.726318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:45.380736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.706660+0000","last_change":"2024-04-25T14:08:44.706660+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.706660+0000","last_undegraded":"2024-04-25T14:08:44.706660+0000","last_fullsized":"2024-04-25T14:08:44.706660+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.722005+0000","last_change":"2024-04-25T14:08:44.716853+0000","last_active":"2024-04-25T14:08:44.722005+0000","last_peered":"2024-04-25T14:08:44.722005+0000","last_clean":"2024-04-25T14:08:44.722005+0000","last_became_active":"2024-04-25T14:08:44.716479+0000","last_became_peered":"2024-04-25T14:08:44.716479+0000","last_unstale":"2024-04-25T14:08:44.722005+0000","last_undegraded":"2024-04-25T14:08:44.722005+0000","last_fullsized":"2024-04-25T14:08:44.722005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:04.500665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.701306+0000","last_change":"2024-04-25T14:08:44.701306+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.701306+0000","last_undegraded":"2024-04-25T14:08:44.701306+0000","last_fullsized":"2024-04-25T14:08:44.701306+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.731008+0000","last_change":"2024-04-25T14:08:44.726934+0000","last_active":"2024-04-25T14:08:44.731008+0000","last_peered":"2024-04-25T14:08:44.731008+0000","last_clean":"2024-04-25T14:08:44.731008+0000","last_became_active":"2024-04-25T14:08:44.726583+0000","last_became_peered":"2024-04-25T14:08:44.726583+0000","last_unstale":"2024-04-25T14:08:44.731008+0000","last_undegraded":"2024-04-25T14:08:44.731008+0000","last_fullsized":"2024-04-25T14:08:44.731008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:39.391902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.695980+0000","last_change":"2024-04-25T14:08:44.695980+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.695980+0000","last_undegraded":"2024-04-25T14:08:44.695980+0000","last_fullsized":"2024-04-25T14:08:44.695980+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.723891+0000","last_change":"2024-04-25T14:08:44.721083+0000","last_active":"2024-04-25T14:08:44.723891+0000","last_peered":"2024-04-25T14:08:44.723891+0000","last_clean":"2024-04-25T14:08:44.723891+0000","last_became_active":"2024-04-25T14:08:44.720775+0000","last_became_peered":"2024-04-25T14:08:44.720775+0000","last_unstale":"2024-04-25T14:08:44.723891+0000","last_undegraded":"2024-04-25T14:08:44.723891+0000","last_fullsized":"2024-04-25T14:08:44.723891+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:06.665120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.712919+0000","last_change":"2024-04-25T14:08:44.712919+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.712919+0000","last_undegraded":"2024-04-25T14:08:44.712919+0000","last_fullsized":"2024-04-25T14:08:44.712919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.725608+0000","last_change":"2024-04-25T14:08:44.723709+0000","last_active":"2024-04-25T14:08:44.725608+0000","last_peered":"2024-04-25T14:08:44.725608+0000","last_clean":"2024-04-25T14:08:44.725608+0000","last_became_active":"2024-04-25T14:08:44.723387+0000","last_became_peered":"2024-04-25T14:08:44.723387+0000","last_unstale":"2024-04-25T14:08:44.725608+0000","last_undegraded":"2024-04-25T14:08:44.725608+0000","last_fullsized":"2024-04-25T14:08:44.725608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:50.763491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.711073+0000","last_change":"2024-04-25T14:08:44.711073+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.711073+0000","last_undegraded":"2024-04-25T14:08:44.711073+0000","last_fullsized":"2024-04-25T14:08:44.711073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.729805+0000","last_change":"2024-04-25T14:08:44.728205+0000","last_active":"2024-04-25T14:08:44.729805+0000","last_peered":"2024-04-25T14:08:44.729805+0000","last_clean":"2024-04-25T14:08:44.729805+0000","last_became_active":"2024-04-25T14:08:44.727910+0000","last_became_peered":"2024-04-25T14:08:44.727910+0000","last_unstale":"2024-04-25T14:08:44.729805+0000","last_undegraded":"2024-04-25T14:08:44.729805+0000","last_fullsized":"2024-04-25T14:08:44.729805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:00.497301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.701371+0000","last_change":"2024-04-25T14:08:44.701371+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.701371+0000","last_undegraded":"2024-04-25T14:08:44.701371+0000","last_fullsized":"2024-04-25T14:08:44.701371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.715663+0000","last_change":"2024-04-25T14:08:44.713564+0000","last_active":"2024-04-25T14:08:44.715663+0000","last_peered":"2024-04-25T14:08:44.715663+0000","last_clean":"2024-04-25T14:08:44.715663+0000","last_became_active":"2024-04-25T14:08:44.712475+0000","last_became_peered":"2024-04-25T14:08:44.712475+0000","last_unstale":"2024-04-25T14:08:44.715663+0000","last_undegraded":"2024-04-25T14:08:44.715663+0000","last_fullsized":"2024-04-25T14:08:44.715663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:01.190413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.703954+0000","last_change":"2024-04-25T14:08:44.703954+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.703954+0000","last_undegraded":"2024-04-25T14:08:44.703954+0000","last_fullsized":"2024-04-25T14:08:44.703954+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.721141+0000","last_change":"2024-04-25T14:08:44.718093+0000","last_active":"2024-04-25T14:08:44.721141+0000","last_peered":"2024-04-25T14:08:44.721141+0000","last_clean":"2024-04-25T14:08:44.721141+0000","last_became_active":"2024-04-25T14:08:44.717768+0000","last_became_peered":"2024-04-25T14:08:44.717768+0000","last_unstale":"2024-04-25T14:08:44.721141+0000","last_undegraded":"2024-04-25T14:08:44.721141+0000","last_fullsized":"2024-04-25T14:08:44.721141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:10.155953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.700260+0000","last_change":"2024-04-25T14:08:44.700260+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.700260+0000","last_undegraded":"2024-04-25T14:08:44.700260+0000","last_fullsized":"2024-04-25T14:08:44.700260+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.720671+0000","last_change":"2024-04-25T14:08:44.717774+0000","last_active":"2024-04-25T14:08:44.720671+0000","last_peered":"2024-04-25T14:08:44.720671+0000","last_clean":"2024-04-25T14:08:44.720671+0000","last_became_active":"2024-04-25T14:08:44.717425+0000","last_became_peered":"2024-04-25T14:08:44.717425+0000","last_unstale":"2024-04-25T14:08:44.720671+0000","last_undegraded":"2024-04-25T14:08:44.720671+0000","last_fullsized":"2024-04-25T14:08:44.720671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:55.423236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.703371+0000","last_change":"2024-04-25T14:08:44.703371+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.703371+0000","last_undegraded":"2024-04-25T14:08:44.703371+0000","last_fullsized":"2024-04-25T14:08:44.703371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.703242+0000","last_change":"2024-04-25T14:08:44.700728+0000","last_active":"2024-04-25T14:08:44.703242+0000","last_peered":"2024-04-25T14:08:44.703242+0000","last_clean":"2024-04-25T14:08:44.703242+0000","last_became_active":"2024-04-25T14:08:44.700344+0000","last_became_peered":"2024-04-25T14:08:44.700344+0000","last_unstale":"2024-04-25T14:08:44.703242+0000","last_undegraded":"2024-04-25T14:08:44.703242+0000","last_fullsized":"2024-04-25T14:08:44.703242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:36:34.689927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.694919+0000","last_change":"2024-04-25T14:08:44.694919+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.694919+0000","last_undegraded":"2024-04-25T14:08:44.694919+0000","last_fullsized":"2024-04-25T14:08:44.694919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.727899+0000","last_change":"2024-04-25T14:08:44.725383+0000","last_active":"2024-04-25T14:08:44.727899+0000","last_peered":"2024-04-25T14:08:44.727899+0000","last_clean":"2024-04-25T14:08:44.727899+0000","last_became_active":"2024-04-25T14:08:44.724211+0000","last_became_peered":"2024-04-25T14:08:44.724211+0000","last_unstale":"2024-04-25T14:08:44.727899+0000","last_undegraded":"2024-04-25T14:08:44.727899+0000","last_fullsized":"2024-04-25T14:08:44.727899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:16.246992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.694866+0000","last_change":"2024-04-25T14:08:44.694866+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.694866+0000","last_undegraded":"2024-04-25T14:08:44.694866+0000","last_fullsized":"2024-04-25T14:08:44.694866+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.719715+0000","last_change":"2024-04-25T14:08:44.716954+0000","last_active":"2024-04-25T14:08:44.719715+0000","last_peered":"2024-04-25T14:08:44.719715+0000","last_clean":"2024-04-25T14:08:44.719715+0000","last_became_active":"2024-04-25T14:08:44.716583+0000","last_became_peered":"2024-04-25T14:08:44.716583+0000","last_unstale":"2024-04-25T14:08:44.719715+0000","last_undegraded":"2024-04-25T14:08:44.719715+0000","last_fullsized":"2024-04-25T14:08:44.719715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:31.019299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.705709+0000","last_change":"2024-04-25T14:08:44.705709+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.705709+0000","last_undegraded":"2024-04-25T14:08:44.705709+0000","last_fullsized":"2024-04-25T14:08:44.705709+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.727980+0000","last_change":"2024-04-25T14:08:44.725423+0000","last_active":"2024-04-25T14:08:44.727980+0000","last_peered":"2024-04-25T14:08:44.727980+0000","last_clean":"2024-04-25T14:08:44.727980+0000","last_became_active":"2024-04-25T14:08:44.725036+0000","last_became_peered":"2024-04-25T14:08:44.725036+0000","last_unstale":"2024-04-25T14:08:44.727980+0000","last_undegraded":"2024-04-25T14:08:44.727980+0000","last_fullsized":"2024-04-25T14:08:44.727980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:45.210808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.703583+0000","last_change":"2024-04-25T14:08:44.703583+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.703583+0000","last_undegraded":"2024-04-25T14:08:44.703583+0000","last_fullsized":"2024-04-25T14:08:44.703583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.720288+0000","last_change":"2024-04-25T14:08:44.717040+0000","last_active":"2024-04-25T14:08:44.720288+0000","last_peered":"2024-04-25T14:08:44.720288+0000","last_clean":"2024-04-25T14:08:44.720288+0000","last_became_active":"2024-04-25T14:08:44.716044+0000","last_became_peered":"2024-04-25T14:08:44.716044+0000","last_unstale":"2024-04-25T14:08:44.720288+0000","last_undegraded":"2024-04-25T14:08:44.720288+0000","last_fullsized":"2024-04-25T14:08:44.720288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:41.020517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.712548+0000","last_change":"2024-04-25T14:08:44.712548+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.712548+0000","last_undegraded":"2024-04-25T14:08:44.712548+0000","last_fullsized":"2024-04-25T14:08:44.712548+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.710784+0000","last_change":"2024-04-25T14:08:44.704043+0000","last_active":"2024-04-25T14:08:44.710784+0000","last_peered":"2024-04-25T14:08:44.710784+0000","last_clean":"2024-04-25T14:08:44.710784+0000","last_became_active":"2024-04-25T14:08:44.703697+0000","last_became_peered":"2024-04-25T14:08:44.703697+0000","last_unstale":"2024-04-25T14:08:44.710784+0000","last_undegraded":"2024-04-25T14:08:44.710784+0000","last_fullsized":"2024-04-25T14:08:44.710784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:42.844372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.704587+0000","last_change":"2024-04-25T14:08:44.704587+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.704587+0000","last_undegraded":"2024-04-25T14:08:44.704587+0000","last_fullsized":"2024-04-25T14:08:44.704587+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.722459+0000","last_change":"2024-04-25T14:08:44.721286+0000","last_active":"2024-04-25T14:08:44.722459+0000","last_peered":"2024-04-25T14:08:44.722459+0000","last_clean":"2024-04-25T14:08:44.722459+0000","last_became_active":"2024-04-25T14:08:44.720949+0000","last_became_peered":"2024-04-25T14:08:44.720949+0000","last_unstale":"2024-04-25T14:08:44.722459+0000","last_undegraded":"2024-04-25T14:08:44.722459+0000","last_fullsized":"2024-04-25T14:08:44.722459+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:58:08.650136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.695843+0000","last_change":"2024-04-25T14:08:44.695843+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.695843+0000","last_undegraded":"2024-04-25T14:08:44.695843+0000","last_fullsized":"2024-04-25T14:08:44.695843+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.728386+0000","last_change":"2024-04-25T14:08:44.725093+0000","last_active":"2024-04-25T14:08:44.728386+0000","last_peered":"2024-04-25T14:08:44.728386+0000","last_clean":"2024-04-25T14:08:44.728386+0000","last_became_active":"2024-04-25T14:08:44.724761+0000","last_became_peered":"2024-04-25T14:08:44.724761+0000","last_unstale":"2024-04-25T14:08:44.728386+0000","last_undegraded":"2024-04-25T14:08:44.728386+0000","last_fullsized":"2024-04-25T14:08:44.728386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:37.926762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.713062+0000","last_change":"2024-04-25T14:08:44.713062+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.713062+0000","last_undegraded":"2024-04-25T14:08:44.713062+0000","last_fullsized":"2024-04-25T14:08:44.713062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.723865+0000","last_change":"2024-04-25T14:08:44.721282+0000","last_active":"2024-04-25T14:08:44.723865+0000","last_peered":"2024-04-25T14:08:44.723865+0000","last_clean":"2024-04-25T14:08:44.723865+0000","last_became_active":"2024-04-25T14:08:44.721077+0000","last_became_peered":"2024-04-25T14:08:44.721077+0000","last_unstale":"2024-04-25T14:08:44.723865+0000","last_undegraded":"2024-04-25T14:08:44.723865+0000","last_fullsized":"2024-04-25T14:08:44.723865+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:07.051812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.698403+0000","last_change":"2024-04-25T14:08:44.698403+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.698403+0000","last_undegraded":"2024-04-25T14:08:44.698403+0000","last_fullsized":"2024-04-25T14:08:44.698403+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.704667+0000","last_change":"2024-04-25T14:08:44.701347+0000","last_active":"2024-04-25T14:08:44.704667+0000","last_peered":"2024-04-25T14:08:44.704667+0000","last_clean":"2024-04-25T14:08:44.704667+0000","last_became_active":"2024-04-25T14:08:44.700917+0000","last_became_peered":"2024-04-25T14:08:44.700917+0000","last_unstale":"2024-04-25T14:08:44.704667+0000","last_undegraded":"2024-04-25T14:08:44.704667+0000","last_fullsized":"2024-04-25T14:08:44.704667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:49.250431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.702541+0000","last_change":"2024-04-25T14:08:44.702541+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.702541+0000","last_undegraded":"2024-04-25T14:08:44.702541+0000","last_fullsized":"2024-04-25T14:08:44.702541+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.728341+0000","last_change":"2024-04-25T14:08:44.724767+0000","last_active":"2024-04-25T14:08:44.728341+0000","last_peered":"2024-04-25T14:08:44.728341+0000","last_clean":"2024-04-25T14:08:44.728341+0000","last_became_active":"2024-04-25T14:08:44.724442+0000","last_became_peered":"2024-04-25T14:08:44.724442+0000","last_unstale":"2024-04-25T14:08:44.728341+0000","last_undegraded":"2024-04-25T14:08:44.728341+0000","last_fullsized":"2024-04-25T14:08:44.728341+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:56.689895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.705244+0000","last_change":"2024-04-25T14:08:44.705244+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.705244+0000","last_undegraded":"2024-04-25T14:08:44.705244+0000","last_fullsized":"2024-04-25T14:08:44.705244+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.726004+0000","last_change":"2024-04-25T14:08:44.723570+0000","last_active":"2024-04-25T14:08:44.726004+0000","last_peered":"2024-04-25T14:08:44.726004+0000","last_clean":"2024-04-25T14:08:44.726004+0000","last_became_active":"2024-04-25T14:08:44.722656+0000","last_became_peered":"2024-04-25T14:08:44.722656+0000","last_unstale":"2024-04-25T14:08:44.726004+0000","last_undegraded":"2024-04-25T14:08:44.726004+0000","last_fullsized":"2024-04-25T14:08:44.726004+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:23.016722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-04-25T14:08:44.708517+0000","last_change":"2024-04-25T14:08:44.708517+0000","last_active":"2024-04-25T14:08:44.686737+0000","last_peered":"2024-04-25T14:08:44.686737+0000","last_clean":"2024-04-25T14:08:44.686737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:08:44.708517+0000","last_undegraded":"2024-04-25T14:08:44.708517+0000","last_fullsized":"2024-04-25T14:08:44.708517+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.723217+0000","last_change":"2024-04-25T14:08:44.719472+0000","last_active":"2024-04-25T14:08:44.723217+0000","last_peered":"2024-04-25T14:08:44.723217+0000","last_clean":"2024-04-25T14:08:44.723217+0000","last_became_active":"2024-04-25T14:08:44.719159+0000","last_became_peered":"2024-04-25T14:08:44.719159+0000","last_unstale":"2024-04-25T14:08:44.723217+0000","last_undegraded":"2024-04-25T14:08:44.723217+0000","last_fullsized":"2024-04-25T14:08:44.723217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:15.094910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"76'100","reported_seq":329,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T14:08:44.711790+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:08:44.711790+0000","last_peered":"2024-04-25T14:08:44.711790+0000","last_clean":"2024-04-25T14:08:44.711790+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:08:44.711790+0000","last_undegraded":"2024-04-25T14:08:44.711790+0000","last_fullsized":"2024-04-25T14:08:44.711790+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":306,"num_read_kb":759,"num_write":149,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":306,"num_read_kb":759,"num_write":149,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574909,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32788,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339052,"statfs":{"total":96636764160,"available":96603189248,"internally_reserved":0,"allocated":1642496,"data_stored":1110288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1048576,"data_stored":519920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574911,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1048576,"data_stored":519920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1048576,"data_stored":519920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1036288,"data_stored":513253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1048576,"data_stored":519920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1048576,"data_stored":519920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574910,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32772,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339068,"statfs":{"total":96636764160,"available":96603205632,"internally_reserved":0,"allocated":1630208,"data_stored":1103621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:08:50.699 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3f is not active+clean 2024-04-25T14:08:50.699 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.700729+0000', 'last_change': '2024-04-25T14:08:44.700729+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.700729+0000', 'last_undegraded': '2024-04-25T14:08:44.700729+0000', 'last_fullsized': '2024-04-25T14:08:44.700729+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:08:50.699 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3e is not active+clean 2024-04-25T14:08:50.699 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.695217+0000', 'last_change': '2024-04-25T14:08:44.695217+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.695217+0000', 'last_undegraded': '2024-04-25T14:08:44.695217+0000', 'last_fullsized': '2024-04-25T14:08:44.695217+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:08:50.699 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3d is not active+clean 2024-04-25T14:08:50.700 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.700715+0000', 'last_change': '2024-04-25T14:08:44.700715+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.700715+0000', 'last_undegraded': '2024-04-25T14:08:44.700715+0000', 'last_fullsized': '2024-04-25T14:08:44.700715+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:08:50.700 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3c is not active+clean 2024-04-25T14:08:50.700 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.707535+0000', 'last_change': '2024-04-25T14:08:44.707535+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.707535+0000', 'last_undegraded': '2024-04-25T14:08:44.707535+0000', 'last_fullsized': '2024-04-25T14:08:44.707535+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:08:50.700 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3b is not active+clean 2024-04-25T14:08:50.700 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.696533+0000', 'last_change': '2024-04-25T14:08:44.696533+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.696533+0000', 'last_undegraded': '2024-04-25T14:08:44.696533+0000', 'last_fullsized': '2024-04-25T14:08:44.696533+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:08:50.700 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3a is not active+clean 2024-04-25T14:08:50.700 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.715602+0000', 'last_change': '2024-04-25T14:08:44.715602+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.715602+0000', 'last_undegraded': '2024-04-25T14:08:44.715602+0000', 'last_fullsized': '2024-04-25T14:08:44.715602+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:08:50.700 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.39 is not active+clean 2024-04-25T14:08:50.700 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.705515+0000', 'last_change': '2024-04-25T14:08:44.705515+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.705515+0000', 'last_undegraded': '2024-04-25T14:08:44.705515+0000', 'last_fullsized': '2024-04-25T14:08:44.705515+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:08:50.700 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.38 is not active+clean 2024-04-25T14:08:50.700 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.704048+0000', 'last_change': '2024-04-25T14:08:44.704048+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.704048+0000', 'last_undegraded': '2024-04-25T14:08:44.704048+0000', 'last_fullsized': '2024-04-25T14:08:44.704048+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:08:50.701 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.37 is not active+clean 2024-04-25T14:08:50.701 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.701272+0000', 'last_change': '2024-04-25T14:08:44.701272+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.701272+0000', 'last_undegraded': '2024-04-25T14:08:44.701272+0000', 'last_fullsized': '2024-04-25T14:08:44.701272+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:08:50.701 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.36 is not active+clean 2024-04-25T14:08:50.701 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.709510+0000', 'last_change': '2024-04-25T14:08:44.709510+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.709510+0000', 'last_undegraded': '2024-04-25T14:08:44.709510+0000', 'last_fullsized': '2024-04-25T14:08:44.709510+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:08:50.701 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.35 is not active+clean 2024-04-25T14:08:50.701 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.699606+0000', 'last_change': '2024-04-25T14:08:44.699606+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.699606+0000', 'last_undegraded': '2024-04-25T14:08:44.699606+0000', 'last_fullsized': '2024-04-25T14:08:44.699606+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:08:50.701 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.34 is not active+clean 2024-04-25T14:08:50.701 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.711227+0000', 'last_change': '2024-04-25T14:08:44.711227+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.711227+0000', 'last_undegraded': '2024-04-25T14:08:44.711227+0000', 'last_fullsized': '2024-04-25T14:08:44.711227+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:08:50.701 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.33 is not active+clean 2024-04-25T14:08:50.701 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.702941+0000', 'last_change': '2024-04-25T14:08:44.702941+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.702941+0000', 'last_undegraded': '2024-04-25T14:08:44.702941+0000', 'last_fullsized': '2024-04-25T14:08:44.702941+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:08:50.701 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.32 is not active+clean 2024-04-25T14:08:50.702 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.699462+0000', 'last_change': '2024-04-25T14:08:44.699462+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.699462+0000', 'last_undegraded': '2024-04-25T14:08:44.699462+0000', 'last_fullsized': '2024-04-25T14:08:44.699462+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:08:50.702 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.31 is not active+clean 2024-04-25T14:08:50.702 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.702169+0000', 'last_change': '2024-04-25T14:08:44.702169+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.702169+0000', 'last_undegraded': '2024-04-25T14:08:44.702169+0000', 'last_fullsized': '2024-04-25T14:08:44.702169+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:08:50.702 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.30 is not active+clean 2024-04-25T14:08:50.702 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.708333+0000', 'last_change': '2024-04-25T14:08:44.708333+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.708333+0000', 'last_undegraded': '2024-04-25T14:08:44.708333+0000', 'last_fullsized': '2024-04-25T14:08:44.708333+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:08:50.702 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2f is not active+clean 2024-04-25T14:08:50.702 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.704663+0000', 'last_change': '2024-04-25T14:08:44.704663+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.704663+0000', 'last_undegraded': '2024-04-25T14:08:44.704663+0000', 'last_fullsized': '2024-04-25T14:08:44.704663+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:08:50.702 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2e is not active+clean 2024-04-25T14:08:50.702 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.700154+0000', 'last_change': '2024-04-25T14:08:44.700154+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.700154+0000', 'last_undegraded': '2024-04-25T14:08:44.700154+0000', 'last_fullsized': '2024-04-25T14:08:44.700154+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:08:50.702 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2d is not active+clean 2024-04-25T14:08:50.703 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.699078+0000', 'last_change': '2024-04-25T14:08:44.699078+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.699078+0000', 'last_undegraded': '2024-04-25T14:08:44.699078+0000', 'last_fullsized': '2024-04-25T14:08:44.699078+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:08:50.703 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2c is not active+clean 2024-04-25T14:08:50.703 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.707069+0000', 'last_change': '2024-04-25T14:08:44.707069+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.707069+0000', 'last_undegraded': '2024-04-25T14:08:44.707069+0000', 'last_fullsized': '2024-04-25T14:08:44.707069+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:08:50.703 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2b is not active+clean 2024-04-25T14:08:50.703 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.700474+0000', 'last_change': '2024-04-25T14:08:44.700474+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.700474+0000', 'last_undegraded': '2024-04-25T14:08:44.700474+0000', 'last_fullsized': '2024-04-25T14:08:44.700474+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:08:50.703 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2a is not active+clean 2024-04-25T14:08:50.703 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.698892+0000', 'last_change': '2024-04-25T14:08:44.698892+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.698892+0000', 'last_undegraded': '2024-04-25T14:08:44.698892+0000', 'last_fullsized': '2024-04-25T14:08:44.698892+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:08:50.703 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.29 is not active+clean 2024-04-25T14:08:50.703 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.702950+0000', 'last_change': '2024-04-25T14:08:44.702950+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.702950+0000', 'last_undegraded': '2024-04-25T14:08:44.702950+0000', 'last_fullsized': '2024-04-25T14:08:44.702950+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:08:50.703 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.28 is not active+clean 2024-04-25T14:08:50.704 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.698720+0000', 'last_change': '2024-04-25T14:08:44.698720+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.698720+0000', 'last_undegraded': '2024-04-25T14:08:44.698720+0000', 'last_fullsized': '2024-04-25T14:08:44.698720+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:08:50.704 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.27 is not active+clean 2024-04-25T14:08:50.704 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.696352+0000', 'last_change': '2024-04-25T14:08:44.696352+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.696352+0000', 'last_undegraded': '2024-04-25T14:08:44.696352+0000', 'last_fullsized': '2024-04-25T14:08:44.696352+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:08:50.704 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.26 is not active+clean 2024-04-25T14:08:50.704 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.700875+0000', 'last_change': '2024-04-25T14:08:44.700875+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.700875+0000', 'last_undegraded': '2024-04-25T14:08:44.700875+0000', 'last_fullsized': '2024-04-25T14:08:44.700875+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:08:50.704 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.25 is not active+clean 2024-04-25T14:08:50.704 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.709537+0000', 'last_change': '2024-04-25T14:08:44.709537+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.709537+0000', 'last_undegraded': '2024-04-25T14:08:44.709537+0000', 'last_fullsized': '2024-04-25T14:08:44.709537+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:08:50.704 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.24 is not active+clean 2024-04-25T14:08:50.704 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.702856+0000', 'last_change': '2024-04-25T14:08:44.702856+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.702856+0000', 'last_undegraded': '2024-04-25T14:08:44.702856+0000', 'last_fullsized': '2024-04-25T14:08:44.702856+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:08:50.704 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.23 is not active+clean 2024-04-25T14:08:50.705 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.704801+0000', 'last_change': '2024-04-25T14:08:44.704801+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.704801+0000', 'last_undegraded': '2024-04-25T14:08:44.704801+0000', 'last_fullsized': '2024-04-25T14:08:44.704801+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:08:50.705 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.22 is not active+clean 2024-04-25T14:08:50.705 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.695641+0000', 'last_change': '2024-04-25T14:08:44.695641+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.695641+0000', 'last_undegraded': '2024-04-25T14:08:44.695641+0000', 'last_fullsized': '2024-04-25T14:08:44.695641+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:08:50.705 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.21 is not active+clean 2024-04-25T14:08:50.705 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.695624+0000', 'last_change': '2024-04-25T14:08:44.695624+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.695624+0000', 'last_undegraded': '2024-04-25T14:08:44.695624+0000', 'last_fullsized': '2024-04-25T14:08:44.695624+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:08:50.705 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.20 is not active+clean 2024-04-25T14:08:50.705 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.707878+0000', 'last_change': '2024-04-25T14:08:44.707878+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.707878+0000', 'last_undegraded': '2024-04-25T14:08:44.707878+0000', 'last_fullsized': '2024-04-25T14:08:44.707878+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:08:50.705 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1c is not active+clean 2024-04-25T14:08:50.705 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.701936+0000', 'last_change': '2024-04-25T14:08:44.701936+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.701936+0000', 'last_undegraded': '2024-04-25T14:08:44.701936+0000', 'last_fullsized': '2024-04-25T14:08:44.701936+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:08:50.705 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1d is not active+clean 2024-04-25T14:08:50.705 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.713037+0000', 'last_change': '2024-04-25T14:08:44.713037+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.713037+0000', 'last_undegraded': '2024-04-25T14:08:44.713037+0000', 'last_fullsized': '2024-04-25T14:08:44.713037+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:08:50.706 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1e is not active+clean 2024-04-25T14:08:50.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.704564+0000', 'last_change': '2024-04-25T14:08:44.704564+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.704564+0000', 'last_undegraded': '2024-04-25T14:08:44.704564+0000', 'last_fullsized': '2024-04-25T14:08:44.704564+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:08:50.706 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1f is not active+clean 2024-04-25T14:08:50.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.704170+0000', 'last_change': '2024-04-25T14:08:44.704170+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.704170+0000', 'last_undegraded': '2024-04-25T14:08:44.704170+0000', 'last_fullsized': '2024-04-25T14:08:44.704170+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:08:50.706 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.18 is not active+clean 2024-04-25T14:08:50.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.701864+0000', 'last_change': '2024-04-25T14:08:44.701864+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.701864+0000', 'last_undegraded': '2024-04-25T14:08:44.701864+0000', 'last_fullsized': '2024-04-25T14:08:44.701864+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:08:50.706 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.19 is not active+clean 2024-04-25T14:08:50.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.701896+0000', 'last_change': '2024-04-25T14:08:44.701896+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.701896+0000', 'last_undegraded': '2024-04-25T14:08:44.701896+0000', 'last_fullsized': '2024-04-25T14:08:44.701896+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:08:50.706 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1a is not active+clean 2024-04-25T14:08:50.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.711108+0000', 'last_change': '2024-04-25T14:08:44.711108+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.711108+0000', 'last_undegraded': '2024-04-25T14:08:44.711108+0000', 'last_fullsized': '2024-04-25T14:08:44.711108+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:08:50.706 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1b is not active+clean 2024-04-25T14:08:50.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.702431+0000', 'last_change': '2024-04-25T14:08:44.702431+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.702431+0000', 'last_undegraded': '2024-04-25T14:08:44.702431+0000', 'last_fullsized': '2024-04-25T14:08:44.702431+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:08:50.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.14 is not active+clean 2024-04-25T14:08:50.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.697949+0000', 'last_change': '2024-04-25T14:08:44.697949+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.697949+0000', 'last_undegraded': '2024-04-25T14:08:44.697949+0000', 'last_fullsized': '2024-04-25T14:08:44.697949+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:08:50.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.15 is not active+clean 2024-04-25T14:08:50.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.705995+0000', 'last_change': '2024-04-25T14:08:44.705995+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.705995+0000', 'last_undegraded': '2024-04-25T14:08:44.705995+0000', 'last_fullsized': '2024-04-25T14:08:44.705995+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:08:50.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.16 is not active+clean 2024-04-25T14:08:50.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.698486+0000', 'last_change': '2024-04-25T14:08:44.698486+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.698486+0000', 'last_undegraded': '2024-04-25T14:08:44.698486+0000', 'last_fullsized': '2024-04-25T14:08:44.698486+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:08:50.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.17 is not active+clean 2024-04-25T14:08:50.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.706660+0000', 'last_change': '2024-04-25T14:08:44.706660+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.706660+0000', 'last_undegraded': '2024-04-25T14:08:44.706660+0000', 'last_fullsized': '2024-04-25T14:08:44.706660+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:08:50.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.10 is not active+clean 2024-04-25T14:08:50.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.701306+0000', 'last_change': '2024-04-25T14:08:44.701306+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.701306+0000', 'last_undegraded': '2024-04-25T14:08:44.701306+0000', 'last_fullsized': '2024-04-25T14:08:44.701306+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:08:50.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.11 is not active+clean 2024-04-25T14:08:50.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.695980+0000', 'last_change': '2024-04-25T14:08:44.695980+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.695980+0000', 'last_undegraded': '2024-04-25T14:08:44.695980+0000', 'last_fullsized': '2024-04-25T14:08:44.695980+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:08:50.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.12 is not active+clean 2024-04-25T14:08:50.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.712919+0000', 'last_change': '2024-04-25T14:08:44.712919+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.712919+0000', 'last_undegraded': '2024-04-25T14:08:44.712919+0000', 'last_fullsized': '2024-04-25T14:08:44.712919+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:08:50.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.13 is not active+clean 2024-04-25T14:08:50.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.711073+0000', 'last_change': '2024-04-25T14:08:44.711073+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.711073+0000', 'last_undegraded': '2024-04-25T14:08:44.711073+0000', 'last_fullsized': '2024-04-25T14:08:44.711073+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:08:50.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.c is not active+clean 2024-04-25T14:08:50.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.701371+0000', 'last_change': '2024-04-25T14:08:44.701371+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.701371+0000', 'last_undegraded': '2024-04-25T14:08:44.701371+0000', 'last_fullsized': '2024-04-25T14:08:44.701371+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:08:50.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.d is not active+clean 2024-04-25T14:08:50.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.703954+0000', 'last_change': '2024-04-25T14:08:44.703954+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.703954+0000', 'last_undegraded': '2024-04-25T14:08:44.703954+0000', 'last_fullsized': '2024-04-25T14:08:44.703954+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:08:50.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.e is not active+clean 2024-04-25T14:08:50.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.700260+0000', 'last_change': '2024-04-25T14:08:44.700260+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.700260+0000', 'last_undegraded': '2024-04-25T14:08:44.700260+0000', 'last_fullsized': '2024-04-25T14:08:44.700260+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:08:50.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.8 is not active+clean 2024-04-25T14:08:50.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.703371+0000', 'last_change': '2024-04-25T14:08:44.703371+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.703371+0000', 'last_undegraded': '2024-04-25T14:08:44.703371+0000', 'last_fullsized': '2024-04-25T14:08:44.703371+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:08:50.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.9 is not active+clean 2024-04-25T14:08:50.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.694919+0000', 'last_change': '2024-04-25T14:08:44.694919+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.694919+0000', 'last_undegraded': '2024-04-25T14:08:44.694919+0000', 'last_fullsized': '2024-04-25T14:08:44.694919+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:08:50.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.a is not active+clean 2024-04-25T14:08:50.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.694866+0000', 'last_change': '2024-04-25T14:08:44.694866+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.694866+0000', 'last_undegraded': '2024-04-25T14:08:44.694866+0000', 'last_fullsized': '2024-04-25T14:08:44.694866+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:08:50.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.b is not active+clean 2024-04-25T14:08:50.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.705709+0000', 'last_change': '2024-04-25T14:08:44.705709+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.705709+0000', 'last_undegraded': '2024-04-25T14:08:44.705709+0000', 'last_fullsized': '2024-04-25T14:08:44.705709+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:08:50.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.4 is not active+clean 2024-04-25T14:08:50.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.703583+0000', 'last_change': '2024-04-25T14:08:44.703583+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.703583+0000', 'last_undegraded': '2024-04-25T14:08:44.703583+0000', 'last_fullsized': '2024-04-25T14:08:44.703583+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:08:50.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.5 is not active+clean 2024-04-25T14:08:50.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.712548+0000', 'last_change': '2024-04-25T14:08:44.712548+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.712548+0000', 'last_undegraded': '2024-04-25T14:08:44.712548+0000', 'last_fullsized': '2024-04-25T14:08:44.712548+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:08:50.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.6 is not active+clean 2024-04-25T14:08:50.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.704587+0000', 'last_change': '2024-04-25T14:08:44.704587+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.704587+0000', 'last_undegraded': '2024-04-25T14:08:44.704587+0000', 'last_fullsized': '2024-04-25T14:08:44.704587+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:08:50.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.7 is not active+clean 2024-04-25T14:08:50.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.695843+0000', 'last_change': '2024-04-25T14:08:44.695843+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.695843+0000', 'last_undegraded': '2024-04-25T14:08:44.695843+0000', 'last_fullsized': '2024-04-25T14:08:44.695843+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:08:50.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.0 is not active+clean 2024-04-25T14:08:50.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.713062+0000', 'last_change': '2024-04-25T14:08:44.713062+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.713062+0000', 'last_undegraded': '2024-04-25T14:08:44.713062+0000', 'last_fullsized': '2024-04-25T14:08:44.713062+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:08:50.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1 is not active+clean 2024-04-25T14:08:50.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.698403+0000', 'last_change': '2024-04-25T14:08:44.698403+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.698403+0000', 'last_undegraded': '2024-04-25T14:08:44.698403+0000', 'last_fullsized': '2024-04-25T14:08:44.698403+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:08:50.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2 is not active+clean 2024-04-25T14:08:50.711 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.702541+0000', 'last_change': '2024-04-25T14:08:44.702541+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.702541+0000', 'last_undegraded': '2024-04-25T14:08:44.702541+0000', 'last_fullsized': '2024-04-25T14:08:44.702541+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:08:50.711 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3 is not active+clean 2024-04-25T14:08:50.711 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.705244+0000', 'last_change': '2024-04-25T14:08:44.705244+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.705244+0000', 'last_undegraded': '2024-04-25T14:08:44.705244+0000', 'last_fullsized': '2024-04-25T14:08:44.705244+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:08:50.711 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.f is not active+clean 2024-04-25T14:08:50.711 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:08:44.708517+0000', 'last_change': '2024-04-25T14:08:44.708517+0000', 'last_active': '2024-04-25T14:08:44.686737+0000', 'last_peered': '2024-04-25T14:08:44.686737+0000', 'last_clean': '2024-04-25T14:08:44.686737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:08:44.708517+0000', 'last_undegraded': '2024-04-25T14:08:44.708517+0000', 'last_fullsized': '2024-04-25T14:08:44.708517+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'last_clean_scrub_stamp': '2024-04-25T14:08:44.686737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:08:50.711 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T14:08:50.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.882+0000 7f79b5c00640 1 Processor -- start 2024-04-25T14:08:50.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.882+0000 7f79b5c00640 1 -- start start 2024-04-25T14:08:50.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.886+0000 7f79b5c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f79b006f0f0 0x7f79b006f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:50.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.886+0000 7f79b5c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79b006d0a0 con 0x7f79b00ce570 2024-04-25T14:08:50.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.886+0000 7f79b5c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79b006d380 con 0x7f79b00c49c0 2024-04-25T14:08:50.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.886+0000 7f79b5c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f79b006dcb0 con 0x7f79b006f0f0 2024-04-25T14:08:50.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.886+0000 7f79af400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f79b006f0f0 0x7f79b006f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:50.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.886+0000 7f79aea00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f79b00ce570 0x7f79b00c42b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49408/0 (socket says 172.21.15.86:49408) 2024-04-25T14:08:50.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.886+0000 7f79aea00640 1 -- 172.21.15.86:0/4162690095 learned_addr learned my addr 172.21.15.86:0/4162690095 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:50.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.886+0000 7f79af400640 1 -- 172.21.15.86:0/4162690095 >> v1:172.21.15.155:6789/0 conn(0x7f79b00c49c0 legacy=0x7f79b006e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:50.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.886+0000 7f79af400640 1 -- 172.21.15.86:0/4162690095 >> v1:172.21.15.86:6789/0 conn(0x7f79b00ce570 legacy=0x7f79b00c42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:50.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.886+0000 7f79af400640 1 -- 172.21.15.86:0/4162690095 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b006d660 con 0x7f79b006f0f0 2024-04-25T14:08:50.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.886+0000 7f79ae000640 1 -- 172.21.15.86:0/4162690095 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4131242956 0 0) 0x7f79b006d0a0 con 0x7f79b00ce570 2024-04-25T14:08:50.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.886+0000 7f79af400640 1 --2- 172.21.15.86:0/4162690095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79b006f0f0 0x7f79b006f500 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f79a80097c0 tx=0x7f79a801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf8f5f25bd59150 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:50.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.890+0000 7f79ae000640 1 -- 172.21.15.86:0/4162690095 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a80b0070 con 0x7f79b006f0f0 2024-04-25T14:08:50.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.890+0000 7f79ae000640 1 -- 172.21.15.86:0/4162690095 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79a80ab600 con 0x7f79b006f0f0 2024-04-25T14:08:50.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.890+0000 7f79ae000640 1 -- 172.21.15.86:0/4162690095 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a80ab9d0 con 0x7f79b006f0f0 2024-04-25T14:08:50.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.890+0000 7f79b5c00640 1 -- 172.21.15.86:0/4162690095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79b006f0f0 msgr2=0x7f79b006f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:50.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.890+0000 7f79b5c00640 1 --2- 172.21.15.86:0/4162690095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79b006f0f0 0x7f79b006f500 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f79a80097c0 tx=0x7f79a801c3d0 comp rx=0 tx=0).stop 2024-04-25T14:08:50.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.890+0000 7f79b5c00640 1 -- 172.21.15.86:0/4162690095 shutdown_connections 2024-04-25T14:08:50.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.890+0000 7f79b5c00640 1 --2- 172.21.15.86:0/4162690095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79b006f0f0 0x7f79b006f500 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:50.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.890+0000 7f79b5c00640 1 -- 172.21.15.86:0/4162690095 >> 172.21.15.86:0/4162690095 conn(0x7f79b00b4650 msgr2=0x7f79b00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:50.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.890+0000 7f79b5c00640 1 -- 172.21.15.86:0/4162690095 shutdown_connections 2024-04-25T14:08:50.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.890+0000 7f79b5c00640 1 -- 172.21.15.86:0/4162690095 wait complete. 2024-04-25T14:08:50.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.890+0000 7f79b5c00640 1 Processor -- start 2024-04-25T14:08:50.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.890+0000 7f79b5c00640 1 -- start start 2024-04-25T14:08:50.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.894+0000 7f79b5c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79b006f0f0 0x7f79b00b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:50.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.894+0000 7f79b5c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79b00ce570 0x7f79b009aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:50.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.894+0000 7f79b5c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79b00b10c0 0x7f79b00c4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:50.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.894+0000 7f79b5c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79b00ceb00 con 0x7f79b00b10c0 2024-04-25T14:08:50.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.894+0000 7f79b5c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f79b00ce980 con 0x7f79b006f0f0 2024-04-25T14:08:50.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.894+0000 7f79b5c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f79b01729c0 con 0x7f79b00ce570 2024-04-25T14:08:50.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.894+0000 7f79af400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79b00b10c0 0x7f79b00c4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:50.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.894+0000 7f79af400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79b00b10c0 0x7f79b00c4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40332/0 (socket says 172.21.15.86:40332) 2024-04-25T14:08:50.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.894+0000 7f79af400640 1 -- 172.21.15.86:0/2696618070 learned_addr learned my addr 172.21.15.86:0/2696618070 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:50.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.894+0000 7f79af400640 1 -- 172.21.15.86:0/2696618070 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79b00ce570 msgr2=0x7f79b009aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:50.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.894+0000 7f79af400640 1 --2- 172.21.15.86:0/2696618070 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79b00ce570 0x7f79b009aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:50.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.894+0000 7f79af400640 1 -- 172.21.15.86:0/2696618070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79b006f0f0 msgr2=0x7f79b00b26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:50.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.894+0000 7f79af400640 1 --2- 172.21.15.86:0/2696618070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79b006f0f0 0x7f79b00b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:50.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.894+0000 7f79af400640 1 -- 172.21.15.86:0/2696618070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b00b1640 con 0x7f79b00b10c0 2024-04-25T14:08:50.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.894+0000 7f79af400640 1 --2- 172.21.15.86:0/2696618070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79b00b10c0 0x7f79b00c4ef0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f79a80097c0 tx=0x7f79a801ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:50.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.894+0000 7f79a7400640 1 -- 172.21.15.86:0/2696618070 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a80b0050 con 0x7f79b00b10c0 2024-04-25T14:08:50.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.894+0000 7f79a7400640 1 -- 172.21.15.86:0/2696618070 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79a80bc070 con 0x7f79b00b10c0 2024-04-25T14:08:50.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.894+0000 7f79a7400640 1 -- 172.21.15.86:0/2696618070 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a80b5830 con 0x7f79b00b10c0 2024-04-25T14:08:50.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.894+0000 7f79b5c00640 1 -- 172.21.15.86:0/2696618070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b00b1920 con 0x7f79b00b10c0 2024-04-25T14:08:50.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.894+0000 7f79b5c00640 1 -- 172.21.15.86:0/2696618070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b00b1ee0 con 0x7f79b00b10c0 2024-04-25T14:08:50.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.894+0000 7f79b5c00640 1 -- 172.21.15.86:0/2696618070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79b0069d00 con 0x7f79b00b10c0 2024-04-25T14:08:50.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.898+0000 7f79a7400640 1 -- 172.21.15.86:0/2696618070 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7f79a80b59d0 con 0x7f79b00b10c0 2024-04-25T14:08:50.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.898+0000 7f79a7400640 1 --2- 172.21.15.86:0/2696618070 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f798c06f550 0x7f798c071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:50.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.898+0000 7f79a7400640 1 -- 172.21.15.86:0/2696618070 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f79a8129eb0 con 0x7f79b00b10c0 2024-04-25T14:08:50.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.898+0000 7f79aea00640 1 --2- 172.21.15.86:0/2696618070 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f798c06f550 0x7f798c071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:50.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.898+0000 7f79aea00640 1 --2- 172.21.15.86:0/2696618070 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f798c06f550 0x7f798c071a60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f79a0001ca0 tx=0x7f79a0007cf0 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:50.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:50.902+0000 7f79a7400640 1 -- 172.21.15.86:0/2696618070 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f79a8016050 con 0x7f79b00b10c0 2024-04-25T14:08:51.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.114+0000 7f79b5c00640 1 -- 172.21.15.86:0/2696618070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f79b016ff80 con 0x7f79b00b10c0 2024-04-25T14:08:51.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.114+0000 7f79a7400640 1 -- 172.21.15.86:0/2696618070 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1034 (secure 0 0 0) 0x7f79a80f69e0 con 0x7f79b00b10c0 2024-04-25T14:08:51.117 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:08:51.117 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":325,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":87,"num_osds":8,"num_up_osds":8,"osd_up_since":1714053818,"num_in_osds":8,"osd_in_since":1714053808,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":265003008,"bytes_avail":772829110272,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":67,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7371}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-04-25T14:08:15.095679+0000","services":{}},"progress_events":{}} 2024-04-25T14:08:51.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.118+0000 7f79b5c00640 1 -- 172.21.15.86:0/2696618070 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f798c06f550 msgr2=0x7f798c071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:51.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.118+0000 7f79b5c00640 1 --2- 172.21.15.86:0/2696618070 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f798c06f550 0x7f798c071a60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f79a0001ca0 tx=0x7f79a0007cf0 comp rx=0 tx=0).stop 2024-04-25T14:08:51.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.118+0000 7f79b5c00640 1 -- 172.21.15.86:0/2696618070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79b00b10c0 msgr2=0x7f79b00c4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:51.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.118+0000 7f79b5c00640 1 --2- 172.21.15.86:0/2696618070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79b00b10c0 0x7f79b00c4ef0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f79a80097c0 tx=0x7f79a801ac10 comp rx=0 tx=0).stop 2024-04-25T14:08:51.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.118+0000 7f79b5c00640 1 -- 172.21.15.86:0/2696618070 shutdown_connections 2024-04-25T14:08:51.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.118+0000 7f79b5c00640 1 --2- 172.21.15.86:0/2696618070 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f798c06f550 0x7f798c071a60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:51.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.118+0000 7f79b5c00640 1 --2- 172.21.15.86:0/2696618070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79b00b10c0 0x7f79b00c4ef0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:51.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.118+0000 7f79b5c00640 1 --2- 172.21.15.86:0/2696618070 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79b00ce570 0x7f79b009aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:51.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.118+0000 7f79b5c00640 1 --2- 172.21.15.86:0/2696618070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79b006f0f0 0x7f79b00b26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:51.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.118+0000 7f79b5c00640 1 -- 172.21.15.86:0/2696618070 >> 172.21.15.86:0/2696618070 conn(0x7f79b00b4650 msgr2=0x7f79b00d0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:51.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.118+0000 7f79b5c00640 1 -- 172.21.15.86:0/2696618070 shutdown_connections 2024-04-25T14:08:51.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.118+0000 7f79b5c00640 1 -- 172.21.15.86:0/2696618070 wait complete. 2024-04-25T14:08:51.139 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T14:08:51.139 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:08:51.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.286+0000 7f8ecf000640 1 Processor -- start 2024-04-25T14:08:51.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.286+0000 7f8ecf000640 1 -- start start 2024-04-25T14:08:51.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.286+0000 7f8ecf000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ec80b6ee0 0x7f8ec80d06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:51.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.286+0000 7f8ecf000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8ec80729b0 0x7f8ec81614c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:51.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.286+0000 7f8ecf000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ec815df00 con 0x7f8ec80d0c00 2024-04-25T14:08:51.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.286+0000 7f8ecf000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8ec815e1e0 con 0x7f8ec80729b0 2024-04-25T14:08:51.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.286+0000 7f8ecf000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8ec815e4c0 con 0x7f8ec80b6ee0 2024-04-25T14:08:51.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.286+0000 7f8ece600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8ec80729b0 0x7f8ec81614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:51.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.286+0000 7f8ecd200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8ec80d0c00 0x7f8ec80722a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49410/0 (socket says 172.21.15.86:49410) 2024-04-25T14:08:51.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.286+0000 7f8ecd200640 1 -- 172.21.15.86:0/1962666549 learned_addr learned my addr 172.21.15.86:0/1962666549 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:51.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.286+0000 7f8ecdc00640 1 --2- 172.21.15.86:0/1962666549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ec80b6ee0 0x7f8ec80d06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:51.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.286+0000 7f8ec7e00640 1 -- 172.21.15.86:0/1962666549 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 755464761 0 0) 0x7f8ec815df00 con 0x7f8ec80d0c00 2024-04-25T14:08:51.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.286+0000 7f8ec7e00640 1 -- 172.21.15.86:0/1962666549 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8eb0003530 con 0x7f8ec80d0c00 2024-04-25T14:08:51.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.286+0000 7f8ece600640 1 -- 172.21.15.86:0/1962666549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ec80b6ee0 msgr2=0x7f8ec80d06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:51.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.286+0000 7f8ece600640 1 --2- 172.21.15.86:0/1962666549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ec80b6ee0 0x7f8ec80d06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:51.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.286+0000 7f8ece600640 1 -- 172.21.15.86:0/1962666549 >> v1:172.21.15.86:6789/0 conn(0x7f8ec80d0c00 legacy=0x7f8ec80722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:51.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.286+0000 7f8ece600640 1 -- 172.21.15.86:0/1962666549 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec815e7a0 con 0x7f8ec80729b0 2024-04-25T14:08:51.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.286+0000 7f8ecdc00640 1 --2- 172.21.15.86:0/1962666549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ec80b6ee0 0x7f8ec80d06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:51.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.290+0000 7f8ece600640 1 --2- 172.21.15.86:0/1962666549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ec80729b0 0x7f8ec81614c0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009d90 tx=0x7f8ec001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f824ae37ffc4d4db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:51.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.290+0000 7f8ec7e00640 1 -- 172.21.15.86:0/1962666549 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec00b2070 con 0x7f8ec80729b0 2024-04-25T14:08:51.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.290+0000 7f8ec7e00640 1 -- 172.21.15.86:0/1962666549 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ec00a57a0 con 0x7f8ec80729b0 2024-04-25T14:08:51.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.290+0000 7f8ec7e00640 1 -- 172.21.15.86:0/1962666549 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec00a5b70 con 0x7f8ec80729b0 2024-04-25T14:08:51.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.290+0000 7f8ecf000640 1 -- 172.21.15.86:0/1962666549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ec80729b0 msgr2=0x7f8ec81614c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:51.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.290+0000 7f8ecf000640 1 --2- 172.21.15.86:0/1962666549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ec80729b0 0x7f8ec81614c0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009d90 tx=0x7f8ec001c8e0 comp rx=0 tx=0).stop 2024-04-25T14:08:51.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.290+0000 7f8ecf000640 1 -- 172.21.15.86:0/1962666549 shutdown_connections 2024-04-25T14:08:51.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.290+0000 7f8ecf000640 1 --2- 172.21.15.86:0/1962666549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ec80729b0 0x7f8ec81614c0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:51.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.290+0000 7f8ecf000640 1 --2- 172.21.15.86:0/1962666549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ec80b6ee0 0x7f8ec80d06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:51.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.290+0000 7f8ecf000640 1 -- 172.21.15.86:0/1962666549 >> 172.21.15.86:0/1962666549 conn(0x7f8ec80b4f80 msgr2=0x7f8ec80b5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:51.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.290+0000 7f8ecf000640 1 -- 172.21.15.86:0/1962666549 shutdown_connections 2024-04-25T14:08:51.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.290+0000 7f8ecf000640 1 -- 172.21.15.86:0/1962666549 wait complete. 2024-04-25T14:08:51.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.294+0000 7f8ecf000640 1 Processor -- start 2024-04-25T14:08:51.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.294+0000 7f8ecf000640 1 -- start start 2024-04-25T14:08:51.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.294+0000 7f8ecf000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ec80b6ee0 0x7f8ec80b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:51.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.294+0000 7f8ecf000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ec80d0c00 0x7f8ec806e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.294+0000 7f8ecf000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ec806df70 0x7f8ec806ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.294+0000 7f8ecf000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec816a970 con 0x7f8ec80b6ee0 2024-04-25T14:08:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.294+0000 7f8ecf000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec80cff50 con 0x7f8ec80d0c00 2024-04-25T14:08:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.294+0000 7f8ecf000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8ec816aaf0 con 0x7f8ec806df70 2024-04-25T14:08:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.294+0000 7f8ecdc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ec80b6ee0 0x7f8ec80b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.294+0000 7f8ecdc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ec80b6ee0 0x7f8ec80b26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40344/0 (socket says 172.21.15.86:40344) 2024-04-25T14:08:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.294+0000 7f8ecdc00640 1 -- 172.21.15.86:0/1304270605 learned_addr learned my addr 172.21.15.86:0/1304270605 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.294+0000 7f8ecdc00640 1 -- 172.21.15.86:0/1304270605 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ec806df70 msgr2=0x7f8ec806ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.294+0000 7f8ecdc00640 1 --2- 172.21.15.86:0/1304270605 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ec806df70 0x7f8ec806ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.294+0000 7f8ecdc00640 1 -- 172.21.15.86:0/1304270605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ec80d0c00 msgr2=0x7f8ec806e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.294+0000 7f8ecdc00640 1 --2- 172.21.15.86:0/1304270605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ec80d0c00 0x7f8ec806e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.294+0000 7f8ecdc00640 1 -- 172.21.15.86:0/1304270605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec81597f0 con 0x7f8ec80b6ee0 2024-04-25T14:08:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.298+0000 7f8ecdc00640 1 --2- 172.21.15.86:0/1304270605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ec80b6ee0 0x7f8ec80b26a0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00d800 tx=0x7f8ebc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.298+0000 7f8ec5600640 1 -- 172.21.15.86:0/1304270605 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc009640 con 0x7f8ec80b6ee0 2024-04-25T14:08:51.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.298+0000 7f8ec5600640 1 -- 172.21.15.86:0/1304270605 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ebc019070 con 0x7f8ec80b6ee0 2024-04-25T14:08:51.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.298+0000 7f8ecf000640 1 -- 172.21.15.86:0/1304270605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec8159ad0 con 0x7f8ec80b6ee0 2024-04-25T14:08:51.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.298+0000 7f8ec5600640 1 -- 172.21.15.86:0/1304270605 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc009a00 con 0x7f8ec80b6ee0 2024-04-25T14:08:51.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.298+0000 7f8ecf000640 1 -- 172.21.15.86:0/1304270605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec815ef40 con 0x7f8ec80b6ee0 2024-04-25T14:08:51.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.298+0000 7f8ecf000640 1 -- 172.21.15.86:0/1304270605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ec8076b80 con 0x7f8ec80b6ee0 2024-04-25T14:08:51.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.298+0000 7f8ec5600640 1 -- 172.21.15.86:0/1304270605 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7f8ebc009ba0 con 0x7f8ec80b6ee0 2024-04-25T14:08:51.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.298+0000 7f8ec5600640 1 --2- 172.21.15.86:0/1304270605 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f8ea806f550 0x7f8ea8071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:51.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.298+0000 7f8ec5600640 1 -- 172.21.15.86:0/1304270605 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f8ebc08fa10 con 0x7f8ec80b6ee0 2024-04-25T14:08:51.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.298+0000 7f8ecd200640 1 --2- 172.21.15.86:0/1304270605 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f8ea806f550 0x7f8ea8071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:51.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.302+0000 7f8ecd200640 1 --2- 172.21.15.86:0/1304270605 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f8ea806f550 0x7f8ea8071a60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4007fd0 tx=0x7f8eb4007900 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:51.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.306+0000 7f8ec5600640 1 -- 172.21.15.86:0/1304270605 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8ebc05c5c0 con 0x7f8ec80b6ee0 2024-04-25T14:08:51.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.458+0000 7f8ecf000640 1 -- 172.21.15.86:0/1304270605 --> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8ec815c930 con 0x7f8ea806f550 2024-04-25T14:08:51.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.466+0000 7f8ec5600640 1 -- 172.21.15.86:0/1304270605 <== mgr.7092 v2:172.21.15.86:6832/3589556609 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252073 (secure 0 0 0) 0x7f8ec815c930 con 0x7f8ea806f550 2024-04-25T14:08:51.470 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:08:51.471 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:08:51.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.470+0000 7f8ecf000640 1 -- 172.21.15.86:0/1304270605 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f8ea806f550 msgr2=0x7f8ea8071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:51.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.470+0000 7f8ecf000640 1 --2- 172.21.15.86:0/1304270605 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f8ea806f550 0x7f8ea8071a60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4007fd0 tx=0x7f8eb4007900 comp rx=0 tx=0).stop 2024-04-25T14:08:51.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.470+0000 7f8ecf000640 1 -- 172.21.15.86:0/1304270605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ec80b6ee0 msgr2=0x7f8ec80b26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:51.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.470+0000 7f8ecf000640 1 --2- 172.21.15.86:0/1304270605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ec80b6ee0 0x7f8ec80b26a0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00d800 tx=0x7f8ebc00dcc0 comp rx=0 tx=0).stop 2024-04-25T14:08:51.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.470+0000 7f8ecf000640 1 -- 172.21.15.86:0/1304270605 shutdown_connections 2024-04-25T14:08:51.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.470+0000 7f8ecf000640 1 --2- 172.21.15.86:0/1304270605 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f8ea806f550 0x7f8ea8071a60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:51.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.470+0000 7f8ecf000640 1 --2- 172.21.15.86:0/1304270605 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ec806df70 0x7f8ec806ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:51.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.470+0000 7f8ecf000640 1 --2- 172.21.15.86:0/1304270605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ec80d0c00 0x7f8ec806e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:51.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.470+0000 7f8ecf000640 1 --2- 172.21.15.86:0/1304270605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ec80b6ee0 0x7f8ec80b26a0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:51.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.470+0000 7f8ecf000640 1 -- 172.21.15.86:0/1304270605 >> 172.21.15.86:0/1304270605 conn(0x7f8ec80b4f80 msgr2=0x7f8ec80b4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:51.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.470+0000 7f8ecf000640 1 -- 172.21.15.86:0/1304270605 shutdown_connections 2024-04-25T14:08:51.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:51.470+0000 7f8ecf000640 1 -- 172.21.15.86:0/1304270605 wait complete. 2024-04-25T14:08:51.491 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-04-25T14:08:51.359654+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":306,"num_read_kb":759,"num_write":194,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259052,"kb_used_data":9608,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715668,"statfs":{"total":773094113280,"available":772828844032,"internally_reserved":0,"allocated":9838592,"data_stored":5455788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.667977"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.725728+0000","last_change":"2024-04-25T14:08:45.724883+0000","last_active":"2024-04-25T14:08:46.725728+0000","last_peered":"2024-04-25T14:08:46.725728+0000","last_clean":"2024-04-25T14:08:46.725728+0000","last_became_active":"2024-04-25T14:08:45.724574+0000","last_became_peered":"2024-04-25T14:08:45.724574+0000","last_unstale":"2024-04-25T14:08:46.725728+0000","last_undegraded":"2024-04-25T14:08:46.725728+0000","last_fullsized":"2024-04-25T14:08:46.725728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:08.058416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.734645+0000","last_change":"2024-04-25T14:08:45.742271+0000","last_active":"2024-04-25T14:08:46.734645+0000","last_peered":"2024-04-25T14:08:46.734645+0000","last_clean":"2024-04-25T14:08:46.734645+0000","last_became_active":"2024-04-25T14:08:45.741928+0000","last_became_peered":"2024-04-25T14:08:45.741928+0000","last_unstale":"2024-04-25T14:08:46.734645+0000","last_undegraded":"2024-04-25T14:08:46.734645+0000","last_fullsized":"2024-04-25T14:08:46.734645+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:17.275325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.725242+0000","last_change":"2024-04-25T14:08:45.716258+0000","last_active":"2024-04-25T14:08:46.725242+0000","last_peered":"2024-04-25T14:08:46.725242+0000","last_clean":"2024-04-25T14:08:46.725242+0000","last_became_active":"2024-04-25T14:08:45.715956+0000","last_became_peered":"2024-04-25T14:08:45.715956+0000","last_unstale":"2024-04-25T14:08:46.725242+0000","last_undegraded":"2024-04-25T14:08:46.725242+0000","last_fullsized":"2024-04-25T14:08:46.725242+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:13.811808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913585+0000","last_change":"2024-04-25T14:08:45.737138+0000","last_active":"2024-04-25T14:08:46.913585+0000","last_peered":"2024-04-25T14:08:46.913585+0000","last_clean":"2024-04-25T14:08:46.913585+0000","last_became_active":"2024-04-25T14:08:45.736851+0000","last_became_peered":"2024-04-25T14:08:45.736851+0000","last_unstale":"2024-04-25T14:08:46.913585+0000","last_undegraded":"2024-04-25T14:08:46.913585+0000","last_fullsized":"2024-04-25T14:08:46.913585+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:54.212949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.731558+0000","last_change":"2024-04-25T14:08:45.731770+0000","last_active":"2024-04-25T14:08:46.731558+0000","last_peered":"2024-04-25T14:08:46.731558+0000","last_clean":"2024-04-25T14:08:46.731558+0000","last_became_active":"2024-04-25T14:08:45.731424+0000","last_became_peered":"2024-04-25T14:08:45.731424+0000","last_unstale":"2024-04-25T14:08:46.731558+0000","last_undegraded":"2024-04-25T14:08:46.731558+0000","last_fullsized":"2024-04-25T14:08:46.731558+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:54.212949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913518+0000","last_change":"2024-04-25T14:08:45.747879+0000","last_active":"2024-04-25T14:08:46.913518+0000","last_peered":"2024-04-25T14:08:46.913518+0000","last_clean":"2024-04-25T14:08:46.913518+0000","last_became_active":"2024-04-25T14:08:45.747571+0000","last_became_peered":"2024-04-25T14:08:45.747571+0000","last_unstale":"2024-04-25T14:08:46.913518+0000","last_undegraded":"2024-04-25T14:08:46.913518+0000","last_fullsized":"2024-04-25T14:08:46.913518+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:54.075986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.725703+0000","last_change":"2024-04-25T14:08:45.733715+0000","last_active":"2024-04-25T14:08:46.725703+0000","last_peered":"2024-04-25T14:08:46.725703+0000","last_clean":"2024-04-25T14:08:46.725703+0000","last_became_active":"2024-04-25T14:08:45.733358+0000","last_became_peered":"2024-04-25T14:08:45.733358+0000","last_unstale":"2024-04-25T14:08:46.725703+0000","last_undegraded":"2024-04-25T14:08:46.725703+0000","last_fullsized":"2024-04-25T14:08:46.725703+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:13.811808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913320+0000","last_change":"2024-04-25T14:08:45.733157+0000","last_active":"2024-04-25T14:08:46.913320+0000","last_peered":"2024-04-25T14:08:46.913320+0000","last_clean":"2024-04-25T14:08:46.913320+0000","last_became_active":"2024-04-25T14:08:45.732454+0000","last_became_peered":"2024-04-25T14:08:45.732454+0000","last_unstale":"2024-04-25T14:08:46.913320+0000","last_undegraded":"2024-04-25T14:08:46.913320+0000","last_fullsized":"2024-04-25T14:08:46.913320+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.062105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.732022+0000","last_change":"2024-04-25T14:08:45.726250+0000","last_active":"2024-04-25T14:08:46.732022+0000","last_peered":"2024-04-25T14:08:46.732022+0000","last_clean":"2024-04-25T14:08:46.732022+0000","last_became_active":"2024-04-25T14:08:45.725924+0000","last_became_peered":"2024-04-25T14:08:45.725924+0000","last_unstale":"2024-04-25T14:08:46.732022+0000","last_undegraded":"2024-04-25T14:08:46.732022+0000","last_fullsized":"2024-04-25T14:08:46.732022+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:17.275325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.726883+0000","last_change":"2024-04-25T14:08:45.743158+0000","last_active":"2024-04-25T14:08:46.726883+0000","last_peered":"2024-04-25T14:08:46.726883+0000","last_clean":"2024-04-25T14:08:46.726883+0000","last_became_active":"2024-04-25T14:08:45.742796+0000","last_became_peered":"2024-04-25T14:08:45.742796+0000","last_unstale":"2024-04-25T14:08:46.726883+0000","last_undegraded":"2024-04-25T14:08:46.726883+0000","last_fullsized":"2024-04-25T14:08:46.726883+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:16.586153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.731847+0000","last_change":"2024-04-25T14:08:45.719110+0000","last_active":"2024-04-25T14:08:46.731847+0000","last_peered":"2024-04-25T14:08:46.731847+0000","last_clean":"2024-04-25T14:08:46.731847+0000","last_became_active":"2024-04-25T14:08:45.718275+0000","last_became_peered":"2024-04-25T14:08:45.718275+0000","last_unstale":"2024-04-25T14:08:46.731847+0000","last_undegraded":"2024-04-25T14:08:46.731847+0000","last_fullsized":"2024-04-25T14:08:46.731847+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:53.715664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.726564+0000","last_change":"2024-04-25T14:08:45.737568+0000","last_active":"2024-04-25T14:08:46.726564+0000","last_peered":"2024-04-25T14:08:46.726564+0000","last_clean":"2024-04-25T14:08:46.726564+0000","last_became_active":"2024-04-25T14:08:45.736104+0000","last_became_peered":"2024-04-25T14:08:45.736104+0000","last_unstale":"2024-04-25T14:08:46.726564+0000","last_undegraded":"2024-04-25T14:08:46.726564+0000","last_fullsized":"2024-04-25T14:08:46.726564+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:01.114117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913398+0000","last_change":"2024-04-25T14:08:45.732768+0000","last_active":"2024-04-25T14:08:46.913398+0000","last_peered":"2024-04-25T14:08:46.913398+0000","last_clean":"2024-04-25T14:08:46.913398+0000","last_became_active":"2024-04-25T14:08:45.731990+0000","last_became_peered":"2024-04-25T14:08:45.731990+0000","last_unstale":"2024-04-25T14:08:46.913398+0000","last_undegraded":"2024-04-25T14:08:46.913398+0000","last_fullsized":"2024-04-25T14:08:46.913398+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:29.581424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.731350+0000","last_change":"2024-04-25T14:08:45.743210+0000","last_active":"2024-04-25T14:08:46.731350+0000","last_peered":"2024-04-25T14:08:46.731350+0000","last_clean":"2024-04-25T14:08:46.731350+0000","last_became_active":"2024-04-25T14:08:45.742458+0000","last_became_peered":"2024-04-25T14:08:45.742458+0000","last_unstale":"2024-04-25T14:08:46.731350+0000","last_undegraded":"2024-04-25T14:08:46.731350+0000","last_fullsized":"2024-04-25T14:08:46.731350+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:46.478748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913806+0000","last_change":"2024-04-25T14:08:45.747944+0000","last_active":"2024-04-25T14:08:46.913806+0000","last_peered":"2024-04-25T14:08:46.913806+0000","last_clean":"2024-04-25T14:08:46.913806+0000","last_became_active":"2024-04-25T14:08:45.747552+0000","last_became_peered":"2024-04-25T14:08:45.747552+0000","last_unstale":"2024-04-25T14:08:46.913806+0000","last_undegraded":"2024-04-25T14:08:46.913806+0000","last_fullsized":"2024-04-25T14:08:46.913806+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:05.956072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913927+0000","last_change":"2024-04-25T14:08:45.735744+0000","last_active":"2024-04-25T14:08:46.913927+0000","last_peered":"2024-04-25T14:08:46.913927+0000","last_clean":"2024-04-25T14:08:46.913927+0000","last_became_active":"2024-04-25T14:08:45.735434+0000","last_became_peered":"2024-04-25T14:08:45.735434+0000","last_unstale":"2024-04-25T14:08:46.913927+0000","last_undegraded":"2024-04-25T14:08:46.913927+0000","last_fullsized":"2024-04-25T14:08:46.913927+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:15.671992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914798+0000","last_change":"2024-04-25T14:08:45.724202+0000","last_active":"2024-04-25T14:08:46.914798+0000","last_peered":"2024-04-25T14:08:46.914798+0000","last_clean":"2024-04-25T14:08:46.914798+0000","last_became_active":"2024-04-25T14:08:45.723887+0000","last_became_peered":"2024-04-25T14:08:45.723887+0000","last_unstale":"2024-04-25T14:08:46.914798+0000","last_undegraded":"2024-04-25T14:08:46.914798+0000","last_fullsized":"2024-04-25T14:08:46.914798+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:53.822693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.725298+0000","last_change":"2024-04-25T14:08:45.731600+0000","last_active":"2024-04-25T14:08:46.725298+0000","last_peered":"2024-04-25T14:08:46.725298+0000","last_clean":"2024-04-25T14:08:46.725298+0000","last_became_active":"2024-04-25T14:08:45.731022+0000","last_became_peered":"2024-04-25T14:08:45.731022+0000","last_unstale":"2024-04-25T14:08:46.725298+0000","last_undegraded":"2024-04-25T14:08:46.725298+0000","last_fullsized":"2024-04-25T14:08:46.725298+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:05.507269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.731847+0000","last_change":"2024-04-25T14:08:45.739967+0000","last_active":"2024-04-25T14:08:46.731847+0000","last_peered":"2024-04-25T14:08:46.731847+0000","last_clean":"2024-04-25T14:08:46.731847+0000","last_became_active":"2024-04-25T14:08:45.739654+0000","last_became_peered":"2024-04-25T14:08:45.739654+0000","last_unstale":"2024-04-25T14:08:46.731847+0000","last_undegraded":"2024-04-25T14:08:46.731847+0000","last_fullsized":"2024-04-25T14:08:46.731847+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:05.956072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913757+0000","last_change":"2024-04-25T14:08:45.752989+0000","last_active":"2024-04-25T14:08:46.913757+0000","last_peered":"2024-04-25T14:08:46.913757+0000","last_clean":"2024-04-25T14:08:46.913757+0000","last_became_active":"2024-04-25T14:08:45.752664+0000","last_became_peered":"2024-04-25T14:08:45.752664+0000","last_unstale":"2024-04-25T14:08:46.913757+0000","last_undegraded":"2024-04-25T14:08:46.913757+0000","last_fullsized":"2024-04-25T14:08:46.913757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:11.361524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.731422+0000","last_change":"2024-04-25T14:08:45.716503+0000","last_active":"2024-04-25T14:08:46.731422+0000","last_peered":"2024-04-25T14:08:46.731422+0000","last_clean":"2024-04-25T14:08:46.731422+0000","last_became_active":"2024-04-25T14:08:45.716171+0000","last_became_peered":"2024-04-25T14:08:45.716171+0000","last_unstale":"2024-04-25T14:08:46.731422+0000","last_undegraded":"2024-04-25T14:08:46.731422+0000","last_fullsized":"2024-04-25T14:08:46.731422+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:40.398506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.731368+0000","last_change":"2024-04-25T14:08:45.743523+0000","last_active":"2024-04-25T14:08:46.731368+0000","last_peered":"2024-04-25T14:08:46.731368+0000","last_clean":"2024-04-25T14:08:46.731368+0000","last_became_active":"2024-04-25T14:08:45.742912+0000","last_became_peered":"2024-04-25T14:08:45.742912+0000","last_unstale":"2024-04-25T14:08:46.731368+0000","last_undegraded":"2024-04-25T14:08:46.731368+0000","last_fullsized":"2024-04-25T14:08:46.731368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:29.581424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914084+0000","last_change":"2024-04-25T14:08:45.747282+0000","last_active":"2024-04-25T14:08:46.914084+0000","last_peered":"2024-04-25T14:08:46.914084+0000","last_clean":"2024-04-25T14:08:46.914084+0000","last_became_active":"2024-04-25T14:08:45.746134+0000","last_became_peered":"2024-04-25T14:08:45.746134+0000","last_unstale":"2024-04-25T14:08:46.914084+0000","last_undegraded":"2024-04-25T14:08:46.914084+0000","last_fullsized":"2024-04-25T14:08:46.914084+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:54.212949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.724829+0000","last_change":"2024-04-25T14:08:45.715888+0000","last_active":"2024-04-25T14:08:46.724829+0000","last_peered":"2024-04-25T14:08:46.724829+0000","last_clean":"2024-04-25T14:08:46.724829+0000","last_became_active":"2024-04-25T14:08:45.715578+0000","last_became_peered":"2024-04-25T14:08:45.715578+0000","last_unstale":"2024-04-25T14:08:46.724829+0000","last_undegraded":"2024-04-25T14:08:46.724829+0000","last_fullsized":"2024-04-25T14:08:46.724829+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:48.690101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.734671+0000","last_change":"2024-04-25T14:08:45.731656+0000","last_active":"2024-04-25T14:08:46.734671+0000","last_peered":"2024-04-25T14:08:46.734671+0000","last_clean":"2024-04-25T14:08:46.734671+0000","last_became_active":"2024-04-25T14:08:45.731321+0000","last_became_peered":"2024-04-25T14:08:45.731321+0000","last_unstale":"2024-04-25T14:08:46.734671+0000","last_undegraded":"2024-04-25T14:08:46.734671+0000","last_fullsized":"2024-04-25T14:08:46.734671+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:06:47.578625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.731951+0000","last_change":"2024-04-25T14:08:45.730637+0000","last_active":"2024-04-25T14:08:46.731951+0000","last_peered":"2024-04-25T14:08:46.731951+0000","last_clean":"2024-04-25T14:08:46.731951+0000","last_became_active":"2024-04-25T14:08:45.729611+0000","last_became_peered":"2024-04-25T14:08:45.729611+0000","last_unstale":"2024-04-25T14:08:46.731951+0000","last_undegraded":"2024-04-25T14:08:46.731951+0000","last_fullsized":"2024-04-25T14:08:46.731951+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.062105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.726811+0000","last_change":"2024-04-25T14:08:45.734547+0000","last_active":"2024-04-25T14:08:46.726811+0000","last_peered":"2024-04-25T14:08:46.726811+0000","last_clean":"2024-04-25T14:08:46.726811+0000","last_became_active":"2024-04-25T14:08:45.732727+0000","last_became_peered":"2024-04-25T14:08:45.732727+0000","last_unstale":"2024-04-25T14:08:46.726811+0000","last_undegraded":"2024-04-25T14:08:46.726811+0000","last_fullsized":"2024-04-25T14:08:46.726811+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:23.823392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913911+0000","last_change":"2024-04-25T14:08:45.731440+0000","last_active":"2024-04-25T14:08:46.913911+0000","last_peered":"2024-04-25T14:08:46.913911+0000","last_clean":"2024-04-25T14:08:46.913911+0000","last_became_active":"2024-04-25T14:08:45.728923+0000","last_became_peered":"2024-04-25T14:08:45.728923+0000","last_unstale":"2024-04-25T14:08:46.913911+0000","last_undegraded":"2024-04-25T14:08:46.913911+0000","last_fullsized":"2024-04-25T14:08:46.913911+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:53.715664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.725785+0000","last_change":"2024-04-25T14:08:45.736550+0000","last_active":"2024-04-25T14:08:46.725785+0000","last_peered":"2024-04-25T14:08:46.725785+0000","last_clean":"2024-04-25T14:08:46.725785+0000","last_became_active":"2024-04-25T14:08:45.736223+0000","last_became_peered":"2024-04-25T14:08:45.736223+0000","last_unstale":"2024-04-25T14:08:46.725785+0000","last_undegraded":"2024-04-25T14:08:46.725785+0000","last_fullsized":"2024-04-25T14:08:46.725785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:05.507269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.731588+0000","last_change":"2024-04-25T14:08:45.725621+0000","last_active":"2024-04-25T14:08:46.731588+0000","last_peered":"2024-04-25T14:08:46.731588+0000","last_clean":"2024-04-25T14:08:46.731588+0000","last_became_active":"2024-04-25T14:08:45.724591+0000","last_became_peered":"2024-04-25T14:08:45.724591+0000","last_unstale":"2024-04-25T14:08:46.731588+0000","last_undegraded":"2024-04-25T14:08:46.731588+0000","last_fullsized":"2024-04-25T14:08:46.731588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:28.745662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.734502+0000","last_change":"2024-04-25T14:08:45.738736+0000","last_active":"2024-04-25T14:08:46.734502+0000","last_peered":"2024-04-25T14:08:46.734502+0000","last_clean":"2024-04-25T14:08:46.734502+0000","last_became_active":"2024-04-25T14:08:45.738475+0000","last_became_peered":"2024-04-25T14:08:45.738475+0000","last_unstale":"2024-04-25T14:08:46.734502+0000","last_undegraded":"2024-04-25T14:08:46.734502+0000","last_fullsized":"2024-04-25T14:08:46.734502+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:40.321904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913605+0000","last_change":"2024-04-25T14:08:45.736497+0000","last_active":"2024-04-25T14:08:46.913605+0000","last_peered":"2024-04-25T14:08:46.913605+0000","last_clean":"2024-04-25T14:08:46.913605+0000","last_became_active":"2024-04-25T14:08:45.736104+0000","last_became_peered":"2024-04-25T14:08:45.736104+0000","last_unstale":"2024-04-25T14:08:46.913605+0000","last_undegraded":"2024-04-25T14:08:46.913605+0000","last_fullsized":"2024-04-25T14:08:46.913605+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:43.083404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914177+0000","last_change":"2024-04-25T14:08:45.718764+0000","last_active":"2024-04-25T14:08:46.914177+0000","last_peered":"2024-04-25T14:08:46.914177+0000","last_clean":"2024-04-25T14:08:46.914177+0000","last_became_active":"2024-04-25T14:08:45.718406+0000","last_became_peered":"2024-04-25T14:08:45.718406+0000","last_unstale":"2024-04-25T14:08:46.914177+0000","last_undegraded":"2024-04-25T14:08:46.914177+0000","last_fullsized":"2024-04-25T14:08:46.914177+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:04.740266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"86'4","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914320+0000","last_change":"2024-04-25T14:08:44.725130+0000","last_active":"2024-04-25T14:08:46.914320+0000","last_peered":"2024-04-25T14:08:46.914320+0000","last_clean":"2024-04-25T14:08:46.914320+0000","last_became_active":"2024-04-25T14:08:44.724805+0000","last_became_peered":"2024-04-25T14:08:44.724805+0000","last_unstale":"2024-04-25T14:08:46.914320+0000","last_undegraded":"2024-04-25T14:08:46.914320+0000","last_fullsized":"2024-04-25T14:08:46.914320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:36:34.689927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914651+0000","last_change":"2024-04-25T14:08:45.730683+0000","last_active":"2024-04-25T14:08:46.914651+0000","last_peered":"2024-04-25T14:08:46.914651+0000","last_clean":"2024-04-25T14:08:46.914651+0000","last_became_active":"2024-04-25T14:08:45.727410+0000","last_became_peered":"2024-04-25T14:08:45.727410+0000","last_unstale":"2024-04-25T14:08:46.914651+0000","last_undegraded":"2024-04-25T14:08:46.914651+0000","last_fullsized":"2024-04-25T14:08:46.914651+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:30.451014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.732084+0000","last_change":"2024-04-25T14:08:44.713248+0000","last_active":"2024-04-25T14:08:46.732084+0000","last_peered":"2024-04-25T14:08:46.732084+0000","last_clean":"2024-04-25T14:08:46.732084+0000","last_became_active":"2024-04-25T14:08:44.712237+0000","last_became_peered":"2024-04-25T14:08:44.712237+0000","last_unstale":"2024-04-25T14:08:46.732084+0000","last_undegraded":"2024-04-25T14:08:46.732084+0000","last_fullsized":"2024-04-25T14:08:46.732084+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:15.094910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.726120+0000","last_change":"2024-04-25T14:08:45.743649+0000","last_active":"2024-04-25T14:08:46.726120+0000","last_peered":"2024-04-25T14:08:46.726120+0000","last_clean":"2024-04-25T14:08:46.726120+0000","last_became_active":"2024-04-25T14:08:45.743231+0000","last_became_peered":"2024-04-25T14:08:45.743231+0000","last_unstale":"2024-04-25T14:08:46.726120+0000","last_undegraded":"2024-04-25T14:08:46.726120+0000","last_fullsized":"2024-04-25T14:08:46.726120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:07.671723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"86'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.726066+0000","last_change":"2024-04-25T14:08:44.725153+0000","last_active":"2024-04-25T14:08:46.726066+0000","last_peered":"2024-04-25T14:08:46.726066+0000","last_clean":"2024-04-25T14:08:46.726066+0000","last_became_active":"2024-04-25T14:08:44.724848+0000","last_became_peered":"2024-04-25T14:08:44.724848+0000","last_unstale":"2024-04-25T14:08:46.726066+0000","last_undegraded":"2024-04-25T14:08:46.726066+0000","last_fullsized":"2024-04-25T14:08:46.726066+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:52.709060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914906+0000","last_change":"2024-04-25T14:08:45.734599+0000","last_active":"2024-04-25T14:08:46.914906+0000","last_peered":"2024-04-25T14:08:46.914906+0000","last_clean":"2024-04-25T14:08:46.914906+0000","last_became_active":"2024-04-25T14:08:45.734284+0000","last_became_peered":"2024-04-25T14:08:45.734284+0000","last_unstale":"2024-04-25T14:08:46.914906+0000","last_undegraded":"2024-04-25T14:08:46.914906+0000","last_fullsized":"2024-04-25T14:08:46.914906+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:16.471242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"86'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913973+0000","last_change":"2024-04-25T14:08:44.720784+0000","last_active":"2024-04-25T14:08:46.913973+0000","last_peered":"2024-04-25T14:08:46.913973+0000","last_clean":"2024-04-25T14:08:46.913973+0000","last_became_active":"2024-04-25T14:08:44.720248+0000","last_became_peered":"2024-04-25T14:08:44.720248+0000","last_unstale":"2024-04-25T14:08:46.913973+0000","last_undegraded":"2024-04-25T14:08:46.913973+0000","last_fullsized":"2024-04-25T14:08:46.913973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:08.924639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914120+0000","last_change":"2024-04-25T14:08:45.746974+0000","last_active":"2024-04-25T14:08:46.914120+0000","last_peered":"2024-04-25T14:08:46.914120+0000","last_clean":"2024-04-25T14:08:46.914120+0000","last_became_active":"2024-04-25T14:08:45.746647+0000","last_became_peered":"2024-04-25T14:08:45.746647+0000","last_unstale":"2024-04-25T14:08:46.914120+0000","last_undegraded":"2024-04-25T14:08:46.914120+0000","last_fullsized":"2024-04-25T14:08:46.914120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:40.321904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.731561+0000","last_change":"2024-04-25T14:08:44.727021+0000","last_active":"2024-04-25T14:08:46.731561+0000","last_peered":"2024-04-25T14:08:46.731561+0000","last_clean":"2024-04-25T14:08:46.731561+0000","last_became_active":"2024-04-25T14:08:44.726079+0000","last_became_peered":"2024-04-25T14:08:44.726079+0000","last_unstale":"2024-04-25T14:08:46.731561+0000","last_undegraded":"2024-04-25T14:08:46.731561+0000","last_fullsized":"2024-04-25T14:08:46.731561+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:18.196654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913987+0000","last_change":"2024-04-25T14:08:45.738838+0000","last_active":"2024-04-25T14:08:46.913987+0000","last_peered":"2024-04-25T14:08:46.913987+0000","last_clean":"2024-04-25T14:08:46.913987+0000","last_became_active":"2024-04-25T14:08:45.737972+0000","last_became_peered":"2024-04-25T14:08:45.737972+0000","last_unstale":"2024-04-25T14:08:46.913987+0000","last_undegraded":"2024-04-25T14:08:46.913987+0000","last_fullsized":"2024-04-25T14:08:46.913987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:29.581424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914639+0000","last_change":"2024-04-25T14:08:44.726268+0000","last_active":"2024-04-25T14:08:46.914639+0000","last_peered":"2024-04-25T14:08:46.914639+0000","last_clean":"2024-04-25T14:08:46.914639+0000","last_became_active":"2024-04-25T14:08:44.725839+0000","last_became_peered":"2024-04-25T14:08:44.725839+0000","last_unstale":"2024-04-25T14:08:46.914639+0000","last_undegraded":"2024-04-25T14:08:46.914639+0000","last_fullsized":"2024-04-25T14:08:46.914639+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:51.117327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913688+0000","last_change":"2024-04-25T14:08:45.744967+0000","last_active":"2024-04-25T14:08:46.913688+0000","last_peered":"2024-04-25T14:08:46.913688+0000","last_clean":"2024-04-25T14:08:46.913688+0000","last_became_active":"2024-04-25T14:08:45.741905+0000","last_became_peered":"2024-04-25T14:08:45.741905+0000","last_unstale":"2024-04-25T14:08:46.913688+0000","last_undegraded":"2024-04-25T14:08:46.913688+0000","last_fullsized":"2024-04-25T14:08:46.913688+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:01.848398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.724998+0000","last_change":"2024-04-25T14:08:44.712147+0000","last_active":"2024-04-25T14:08:46.724998+0000","last_peered":"2024-04-25T14:08:46.724998+0000","last_clean":"2024-04-25T14:08:46.724998+0000","last_became_active":"2024-04-25T14:08:44.711799+0000","last_became_peered":"2024-04-25T14:08:44.711799+0000","last_unstale":"2024-04-25T14:08:46.724998+0000","last_undegraded":"2024-04-25T14:08:46.724998+0000","last_fullsized":"2024-04-25T14:08:46.724998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:53.206346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913438+0000","last_change":"2024-04-25T14:08:45.732535+0000","last_active":"2024-04-25T14:08:46.913438+0000","last_peered":"2024-04-25T14:08:46.913438+0000","last_clean":"2024-04-25T14:08:46.913438+0000","last_became_active":"2024-04-25T14:08:45.731982+0000","last_became_peered":"2024-04-25T14:08:45.731982+0000","last_unstale":"2024-04-25T14:08:46.913438+0000","last_undegraded":"2024-04-25T14:08:46.913438+0000","last_fullsized":"2024-04-25T14:08:46.913438+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:40.398506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.725664+0000","last_change":"2024-04-25T14:08:44.723883+0000","last_active":"2024-04-25T14:08:46.725664+0000","last_peered":"2024-04-25T14:08:46.725664+0000","last_clean":"2024-04-25T14:08:46.725664+0000","last_became_active":"2024-04-25T14:08:44.723229+0000","last_became_peered":"2024-04-25T14:08:44.723229+0000","last_unstale":"2024-04-25T14:08:46.725664+0000","last_undegraded":"2024-04-25T14:08:46.725664+0000","last_fullsized":"2024-04-25T14:08:46.725664+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:53.206346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.731751+0000","last_change":"2024-04-25T14:08:45.735428+0000","last_active":"2024-04-25T14:08:46.731751+0000","last_peered":"2024-04-25T14:08:46.731751+0000","last_clean":"2024-04-25T14:08:46.731751+0000","last_became_active":"2024-04-25T14:08:45.733452+0000","last_became_peered":"2024-04-25T14:08:45.733452+0000","last_unstale":"2024-04-25T14:08:46.731751+0000","last_undegraded":"2024-04-25T14:08:46.731751+0000","last_fullsized":"2024-04-25T14:08:46.731751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:11.361524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.734755+0000","last_change":"2024-04-25T14:08:44.723981+0000","last_active":"2024-04-25T14:08:46.734755+0000","last_peered":"2024-04-25T14:08:46.734755+0000","last_clean":"2024-04-25T14:08:46.734755+0000","last_became_active":"2024-04-25T14:08:44.723657+0000","last_became_peered":"2024-04-25T14:08:44.723657+0000","last_unstale":"2024-04-25T14:08:46.734755+0000","last_undegraded":"2024-04-25T14:08:46.734755+0000","last_fullsized":"2024-04-25T14:08:46.734755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:45.472144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.725985+0000","last_change":"2024-04-25T14:08:45.726814+0000","last_active":"2024-04-25T14:08:46.725985+0000","last_peered":"2024-04-25T14:08:46.725985+0000","last_clean":"2024-04-25T14:08:46.725985+0000","last_became_active":"2024-04-25T14:08:45.726491+0000","last_became_peered":"2024-04-25T14:08:45.726491+0000","last_unstale":"2024-04-25T14:08:46.725985+0000","last_undegraded":"2024-04-25T14:08:46.725985+0000","last_fullsized":"2024-04-25T14:08:46.725985+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:08.058416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.725401+0000","last_change":"2024-04-25T14:08:44.718626+0000","last_active":"2024-04-25T14:08:46.725401+0000","last_peered":"2024-04-25T14:08:46.725401+0000","last_clean":"2024-04-25T14:08:46.725401+0000","last_became_active":"2024-04-25T14:08:44.717237+0000","last_became_peered":"2024-04-25T14:08:44.717237+0000","last_unstale":"2024-04-25T14:08:46.725401+0000","last_undegraded":"2024-04-25T14:08:46.725401+0000","last_fullsized":"2024-04-25T14:08:46.725401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:27.739059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.725391+0000","last_change":"2024-04-25T14:08:45.722720+0000","last_active":"2024-04-25T14:08:46.725391+0000","last_peered":"2024-04-25T14:08:46.725391+0000","last_clean":"2024-04-25T14:08:46.725391+0000","last_became_active":"2024-04-25T14:08:45.720917+0000","last_became_peered":"2024-04-25T14:08:45.720917+0000","last_unstale":"2024-04-25T14:08:46.725391+0000","last_undegraded":"2024-04-25T14:08:46.725391+0000","last_fullsized":"2024-04-25T14:08:46.725391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:07.671723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"86'3","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.731990+0000","last_change":"2024-04-25T14:08:44.723221+0000","last_active":"2024-04-25T14:08:46.731990+0000","last_peered":"2024-04-25T14:08:46.731990+0000","last_clean":"2024-04-25T14:08:46.731990+0000","last_became_active":"2024-04-25T14:08:44.722729+0000","last_became_peered":"2024-04-25T14:08:44.722729+0000","last_unstale":"2024-04-25T14:08:46.731990+0000","last_undegraded":"2024-04-25T14:08:46.731990+0000","last_fullsized":"2024-04-25T14:08:46.731990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:56.689895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913636+0000","last_change":"2024-04-25T14:08:45.730816+0000","last_active":"2024-04-25T14:08:46.913636+0000","last_peered":"2024-04-25T14:08:46.913636+0000","last_clean":"2024-04-25T14:08:46.913636+0000","last_became_active":"2024-04-25T14:08:45.729265+0000","last_became_peered":"2024-04-25T14:08:45.729265+0000","last_unstale":"2024-04-25T14:08:46.913636+0000","last_undegraded":"2024-04-25T14:08:46.913636+0000","last_fullsized":"2024-04-25T14:08:46.913636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:28.745662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"86'6","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914365+0000","last_change":"2024-04-25T14:08:44.716853+0000","last_active":"2024-04-25T14:08:46.914365+0000","last_peered":"2024-04-25T14:08:46.914365+0000","last_clean":"2024-04-25T14:08:46.914365+0000","last_became_active":"2024-04-25T14:08:44.716479+0000","last_became_peered":"2024-04-25T14:08:44.716479+0000","last_unstale":"2024-04-25T14:08:46.914365+0000","last_undegraded":"2024-04-25T14:08:46.914365+0000","last_fullsized":"2024-04-25T14:08:46.914365+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:11.127296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914199+0000","last_change":"2024-04-25T14:08:45.733994+0000","last_active":"2024-04-25T14:08:46.914199+0000","last_peered":"2024-04-25T14:08:46.914199+0000","last_clean":"2024-04-25T14:08:46.914199+0000","last_became_active":"2024-04-25T14:08:45.732848+0000","last_became_peered":"2024-04-25T14:08:45.732848+0000","last_unstale":"2024-04-25T14:08:46.914199+0000","last_undegraded":"2024-04-25T14:08:46.914199+0000","last_fullsized":"2024-04-25T14:08:46.914199+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:17.275325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"86'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.725901+0000","last_change":"2024-04-25T14:08:44.726934+0000","last_active":"2024-04-25T14:08:46.725901+0000","last_peered":"2024-04-25T14:08:46.725901+0000","last_clean":"2024-04-25T14:08:46.725901+0000","last_became_active":"2024-04-25T14:08:44.726583+0000","last_became_peered":"2024-04-25T14:08:44.726583+0000","last_unstale":"2024-04-25T14:08:46.725901+0000","last_undegraded":"2024-04-25T14:08:46.725901+0000","last_fullsized":"2024-04-25T14:08:46.725901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:10.354920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.731333+0000","last_change":"2024-04-25T14:08:45.727637+0000","last_active":"2024-04-25T14:08:46.731333+0000","last_peered":"2024-04-25T14:08:46.731333+0000","last_clean":"2024-04-25T14:08:46.731333+0000","last_became_active":"2024-04-25T14:08:45.727312+0000","last_became_peered":"2024-04-25T14:08:45.727312+0000","last_unstale":"2024-04-25T14:08:46.731333+0000","last_undegraded":"2024-04-25T14:08:46.731333+0000","last_fullsized":"2024-04-25T14:08:46.731333+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:24.023326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913920+0000","last_change":"2024-04-25T14:08:44.721083+0000","last_active":"2024-04-25T14:08:46.913920+0000","last_peered":"2024-04-25T14:08:46.913920+0000","last_clean":"2024-04-25T14:08:46.913920+0000","last_became_active":"2024-04-25T14:08:44.720775+0000","last_became_peered":"2024-04-25T14:08:44.720775+0000","last_unstale":"2024-04-25T14:08:46.913920+0000","last_undegraded":"2024-04-25T14:08:46.913920+0000","last_fullsized":"2024-04-25T14:08:46.913920+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:19.655489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.726394+0000","last_change":"2024-04-25T14:08:45.749537+0000","last_active":"2024-04-25T14:08:46.726394+0000","last_peered":"2024-04-25T14:08:46.726394+0000","last_clean":"2024-04-25T14:08:46.726394+0000","last_became_active":"2024-04-25T14:08:45.748947+0000","last_became_peered":"2024-04-25T14:08:45.748947+0000","last_unstale":"2024-04-25T14:08:46.726394+0000","last_undegraded":"2024-04-25T14:08:46.726394+0000","last_fullsized":"2024-04-25T14:08:46.726394+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:52.873960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913865+0000","last_change":"2024-04-25T14:08:44.723709+0000","last_active":"2024-04-25T14:08:46.913865+0000","last_peered":"2024-04-25T14:08:46.913865+0000","last_clean":"2024-04-25T14:08:46.913865+0000","last_became_active":"2024-04-25T14:08:44.723387+0000","last_became_peered":"2024-04-25T14:08:44.723387+0000","last_unstale":"2024-04-25T14:08:46.913865+0000","last_undegraded":"2024-04-25T14:08:46.913865+0000","last_fullsized":"2024-04-25T14:08:46.913865+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:37.926762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.726651+0000","last_change":"2024-04-25T14:08:45.728344+0000","last_active":"2024-04-25T14:08:46.726651+0000","last_peered":"2024-04-25T14:08:46.726651+0000","last_clean":"2024-04-25T14:08:46.726651+0000","last_became_active":"2024-04-25T14:08:45.728020+0000","last_became_peered":"2024-04-25T14:08:45.728020+0000","last_unstale":"2024-04-25T14:08:46.726651+0000","last_undegraded":"2024-04-25T14:08:46.726651+0000","last_fullsized":"2024-04-25T14:08:46.726651+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:42:36.456843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"86'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914225+0000","last_change":"2024-04-25T14:08:44.728205+0000","last_active":"2024-04-25T14:08:46.914225+0000","last_peered":"2024-04-25T14:08:46.914225+0000","last_clean":"2024-04-25T14:08:46.914225+0000","last_became_active":"2024-04-25T14:08:44.727910+0000","last_became_peered":"2024-04-25T14:08:44.727910+0000","last_unstale":"2024-04-25T14:08:46.914225+0000","last_undegraded":"2024-04-25T14:08:46.914225+0000","last_fullsized":"2024-04-25T14:08:46.914225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:56.689895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914357+0000","last_change":"2024-04-25T14:08:45.720863+0000","last_active":"2024-04-25T14:08:46.914357+0000","last_peered":"2024-04-25T14:08:46.914357+0000","last_clean":"2024-04-25T14:08:46.914357+0000","last_became_active":"2024-04-25T14:08:45.720540+0000","last_became_peered":"2024-04-25T14:08:45.720540+0000","last_unstale":"2024-04-25T14:08:46.914357+0000","last_undegraded":"2024-04-25T14:08:46.914357+0000","last_fullsized":"2024-04-25T14:08:46.914357+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:14.715491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914420+0000","last_change":"2024-04-25T14:08:44.713564+0000","last_active":"2024-04-25T14:08:46.914420+0000","last_peered":"2024-04-25T14:08:46.914420+0000","last_clean":"2024-04-25T14:08:46.914420+0000","last_became_active":"2024-04-25T14:08:44.712475+0000","last_became_peered":"2024-04-25T14:08:44.712475+0000","last_unstale":"2024-04-25T14:08:46.914420+0000","last_undegraded":"2024-04-25T14:08:46.914420+0000","last_fullsized":"2024-04-25T14:08:46.914420+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:22.832096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914098+0000","last_change":"2024-04-25T14:08:45.732385+0000","last_active":"2024-04-25T14:08:46.914098+0000","last_peered":"2024-04-25T14:08:46.914098+0000","last_clean":"2024-04-25T14:08:46.914098+0000","last_became_active":"2024-04-25T14:08:45.731872+0000","last_became_peered":"2024-04-25T14:08:45.731872+0000","last_unstale":"2024-04-25T14:08:46.914098+0000","last_undegraded":"2024-04-25T14:08:46.914098+0000","last_fullsized":"2024-04-25T14:08:46.914098+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:28.745662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"86'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.726265+0000","last_change":"2024-04-25T14:08:44.718093+0000","last_active":"2024-04-25T14:08:46.726265+0000","last_peered":"2024-04-25T14:08:46.726265+0000","last_clean":"2024-04-25T14:08:46.726265+0000","last_became_active":"2024-04-25T14:08:44.717768+0000","last_became_peered":"2024-04-25T14:08:44.717768+0000","last_unstale":"2024-04-25T14:08:46.726265+0000","last_undegraded":"2024-04-25T14:08:46.726265+0000","last_fullsized":"2024-04-25T14:08:46.726265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:04.949468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.731960+0000","last_change":"2024-04-25T14:08:45.734270+0000","last_active":"2024-04-25T14:08:46.731960+0000","last_peered":"2024-04-25T14:08:46.731960+0000","last_clean":"2024-04-25T14:08:46.731960+0000","last_became_active":"2024-04-25T14:08:45.733977+0000","last_became_peered":"2024-04-25T14:08:45.733977+0000","last_unstale":"2024-04-25T14:08:46.731960+0000","last_undegraded":"2024-04-25T14:08:46.731960+0000","last_fullsized":"2024-04-25T14:08:46.731960+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:40.321904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.732079+0000","last_change":"2024-04-25T14:08:44.717774+0000","last_active":"2024-04-25T14:08:46.732079+0000","last_peered":"2024-04-25T14:08:46.732079+0000","last_clean":"2024-04-25T14:08:46.732079+0000","last_became_active":"2024-04-25T14:08:44.717425+0000","last_became_peered":"2024-04-25T14:08:44.717425+0000","last_unstale":"2024-04-25T14:08:46.732079+0000","last_undegraded":"2024-04-25T14:08:46.732079+0000","last_fullsized":"2024-04-25T14:08:46.732079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:59.659739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913769+0000","last_change":"2024-04-25T14:08:45.732941+0000","last_active":"2024-04-25T14:08:46.913769+0000","last_peered":"2024-04-25T14:08:46.913769+0000","last_clean":"2024-04-25T14:08:46.913769+0000","last_became_active":"2024-04-25T14:08:45.732613+0000","last_became_peered":"2024-04-25T14:08:45.732613+0000","last_unstale":"2024-04-25T14:08:46.913769+0000","last_undegraded":"2024-04-25T14:08:46.913769+0000","last_fullsized":"2024-04-25T14:08:46.913769+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:49.711152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"86'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.734348+0000","last_change":"2024-04-25T14:08:44.700728+0000","last_active":"2024-04-25T14:08:46.734348+0000","last_peered":"2024-04-25T14:08:46.734348+0000","last_clean":"2024-04-25T14:08:46.734348+0000","last_became_active":"2024-04-25T14:08:44.700344+0000","last_became_peered":"2024-04-25T14:08:44.700344+0000","last_unstale":"2024-04-25T14:08:46.734348+0000","last_undegraded":"2024-04-25T14:08:46.734348+0000","last_fullsized":"2024-04-25T14:08:46.734348+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:10.155953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.734346+0000","last_change":"2024-04-25T14:08:45.736125+0000","last_active":"2024-04-25T14:08:46.734346+0000","last_peered":"2024-04-25T14:08:46.734346+0000","last_clean":"2024-04-25T14:08:46.734346+0000","last_became_active":"2024-04-25T14:08:45.735713+0000","last_became_peered":"2024-04-25T14:08:45.735713+0000","last_unstale":"2024-04-25T14:08:46.734346+0000","last_undegraded":"2024-04-25T14:08:46.734346+0000","last_fullsized":"2024-04-25T14:08:46.734346+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.062105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.726409+0000","last_change":"2024-04-25T14:08:44.725383+0000","last_active":"2024-04-25T14:08:46.726409+0000","last_peered":"2024-04-25T14:08:46.726409+0000","last_clean":"2024-04-25T14:08:46.726409+0000","last_became_active":"2024-04-25T14:08:44.724211+0000","last_became_peered":"2024-04-25T14:08:44.724211+0000","last_unstale":"2024-04-25T14:08:46.726409+0000","last_undegraded":"2024-04-25T14:08:46.726409+0000","last_fullsized":"2024-04-25T14:08:46.726409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:42.017831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.734379+0000","last_change":"2024-04-25T14:08:45.741432+0000","last_active":"2024-04-25T14:08:46.734379+0000","last_peered":"2024-04-25T14:08:46.734379+0000","last_clean":"2024-04-25T14:08:46.734379+0000","last_became_active":"2024-04-25T14:08:45.741069+0000","last_became_peered":"2024-04-25T14:08:45.741069+0000","last_unstale":"2024-04-25T14:08:46.734379+0000","last_undegraded":"2024-04-25T14:08:46.734379+0000","last_fullsized":"2024-04-25T14:08:46.734379+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:53.715664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.734576+0000","last_change":"2024-04-25T14:08:44.716954+0000","last_active":"2024-04-25T14:08:46.734576+0000","last_peered":"2024-04-25T14:08:46.734576+0000","last_clean":"2024-04-25T14:08:46.734576+0000","last_became_active":"2024-04-25T14:08:44.716583+0000","last_became_peered":"2024-04-25T14:08:44.716583+0000","last_unstale":"2024-04-25T14:08:46.734576+0000","last_undegraded":"2024-04-25T14:08:46.734576+0000","last_fullsized":"2024-04-25T14:08:46.734576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:41.020517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914184+0000","last_change":"2024-04-25T14:08:45.733159+0000","last_active":"2024-04-25T14:08:46.914184+0000","last_peered":"2024-04-25T14:08:46.914184+0000","last_clean":"2024-04-25T14:08:46.914184+0000","last_became_active":"2024-04-25T14:08:45.732850+0000","last_became_peered":"2024-04-25T14:08:45.732850+0000","last_unstale":"2024-04-25T14:08:46.914184+0000","last_undegraded":"2024-04-25T14:08:46.914184+0000","last_fullsized":"2024-04-25T14:08:46.914184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:01.325944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.726323+0000","last_change":"2024-04-25T14:08:44.725423+0000","last_active":"2024-04-25T14:08:46.726323+0000","last_peered":"2024-04-25T14:08:46.726323+0000","last_clean":"2024-04-25T14:08:46.726323+0000","last_became_active":"2024-04-25T14:08:44.725036+0000","last_became_peered":"2024-04-25T14:08:44.725036+0000","last_unstale":"2024-04-25T14:08:46.726323+0000","last_undegraded":"2024-04-25T14:08:46.726323+0000","last_fullsized":"2024-04-25T14:08:46.726323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:49.606537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914011+0000","last_change":"2024-04-25T14:08:45.731115+0000","last_active":"2024-04-25T14:08:46.914011+0000","last_peered":"2024-04-25T14:08:46.914011+0000","last_clean":"2024-04-25T14:08:46.914011+0000","last_became_active":"2024-04-25T14:08:45.730623+0000","last_became_peered":"2024-04-25T14:08:45.730623+0000","last_unstale":"2024-04-25T14:08:46.914011+0000","last_undegraded":"2024-04-25T14:08:46.914011+0000","last_fullsized":"2024-04-25T14:08:46.914011+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:40.398506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"86'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.726092+0000","last_change":"2024-04-25T14:08:44.717040+0000","last_active":"2024-04-25T14:08:46.726092+0000","last_peered":"2024-04-25T14:08:46.726092+0000","last_clean":"2024-04-25T14:08:46.726092+0000","last_became_active":"2024-04-25T14:08:44.716044+0000","last_became_peered":"2024-04-25T14:08:44.716044+0000","last_unstale":"2024-04-25T14:08:46.726092+0000","last_undegraded":"2024-04-25T14:08:46.726092+0000","last_fullsized":"2024-04-25T14:08:46.726092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:12.055502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914788+0000","last_change":"2024-04-25T14:08:45.718131+0000","last_active":"2024-04-25T14:08:46.914788+0000","last_peered":"2024-04-25T14:08:46.914788+0000","last_clean":"2024-04-25T14:08:46.914788+0000","last_became_active":"2024-04-25T14:08:45.717784+0000","last_became_peered":"2024-04-25T14:08:45.717784+0000","last_unstale":"2024-04-25T14:08:46.914788+0000","last_undegraded":"2024-04-25T14:08:46.914788+0000","last_fullsized":"2024-04-25T14:08:46.914788+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:37.700457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"87'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.732152+0000","last_change":"2024-04-25T14:08:44.704043+0000","last_active":"2024-04-25T14:08:46.732152+0000","last_peered":"2024-04-25T14:08:46.732152+0000","last_clean":"2024-04-25T14:08:46.732152+0000","last_became_active":"2024-04-25T14:08:44.703697+0000","last_became_peered":"2024-04-25T14:08:44.703697+0000","last_unstale":"2024-04-25T14:08:46.732152+0000","last_undegraded":"2024-04-25T14:08:46.732152+0000","last_fullsized":"2024-04-25T14:08:46.732152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:00.497301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914741+0000","last_change":"2024-04-25T14:08:45.739068+0000","last_active":"2024-04-25T14:08:46.914741+0000","last_peered":"2024-04-25T14:08:46.914741+0000","last_clean":"2024-04-25T14:08:46.914741+0000","last_became_active":"2024-04-25T14:08:45.738777+0000","last_became_peered":"2024-04-25T14:08:45.738777+0000","last_unstale":"2024-04-25T14:08:46.914741+0000","last_undegraded":"2024-04-25T14:08:46.914741+0000","last_fullsized":"2024-04-25T14:08:46.914741+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:47:40.495887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.732098+0000","last_change":"2024-04-25T14:08:44.721286+0000","last_active":"2024-04-25T14:08:46.732098+0000","last_peered":"2024-04-25T14:08:46.732098+0000","last_clean":"2024-04-25T14:08:46.732098+0000","last_became_active":"2024-04-25T14:08:44.720949+0000","last_became_peered":"2024-04-25T14:08:44.720949+0000","last_unstale":"2024-04-25T14:08:46.732098+0000","last_undegraded":"2024-04-25T14:08:46.732098+0000","last_fullsized":"2024-04-25T14:08:46.732098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:50.763491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.734687+0000","last_change":"2024-04-25T14:08:45.728752+0000","last_active":"2024-04-25T14:08:46.734687+0000","last_peered":"2024-04-25T14:08:46.734687+0000","last_clean":"2024-04-25T14:08:46.734687+0000","last_became_active":"2024-04-25T14:08:45.728433+0000","last_became_peered":"2024-04-25T14:08:45.728433+0000","last_unstale":"2024-04-25T14:08:46.734687+0000","last_undegraded":"2024-04-25T14:08:46.734687+0000","last_fullsized":"2024-04-25T14:08:46.734687+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:56.301529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914143+0000","last_change":"2024-04-25T14:08:44.725093+0000","last_active":"2024-04-25T14:08:46.914143+0000","last_peered":"2024-04-25T14:08:46.914143+0000","last_clean":"2024-04-25T14:08:46.914143+0000","last_became_active":"2024-04-25T14:08:44.724761+0000","last_became_peered":"2024-04-25T14:08:44.724761+0000","last_unstale":"2024-04-25T14:08:46.914143+0000","last_undegraded":"2024-04-25T14:08:46.914143+0000","last_fullsized":"2024-04-25T14:08:46.914143+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:41.595810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.726210+0000","last_change":"2024-04-25T14:08:45.739386+0000","last_active":"2024-04-25T14:08:46.726210+0000","last_peered":"2024-04-25T14:08:46.726210+0000","last_clean":"2024-04-25T14:08:46.726210+0000","last_became_active":"2024-04-25T14:08:45.738955+0000","last_became_peered":"2024-04-25T14:08:45.738955+0000","last_unstale":"2024-04-25T14:08:46.726210+0000","last_undegraded":"2024-04-25T14:08:46.726210+0000","last_fullsized":"2024-04-25T14:08:46.726210+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:52.341358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914289+0000","last_change":"2024-04-25T14:08:44.721282+0000","last_active":"2024-04-25T14:08:46.914289+0000","last_peered":"2024-04-25T14:08:46.914289+0000","last_clean":"2024-04-25T14:08:46.914289+0000","last_became_active":"2024-04-25T14:08:44.721077+0000","last_became_peered":"2024-04-25T14:08:44.721077+0000","last_unstale":"2024-04-25T14:08:46.914289+0000","last_undegraded":"2024-04-25T14:08:46.914289+0000","last_fullsized":"2024-04-25T14:08:46.914289+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:32.296318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.724991+0000","last_change":"2024-04-25T14:08:45.742804+0000","last_active":"2024-04-25T14:08:46.724991+0000","last_peered":"2024-04-25T14:08:46.724991+0000","last_clean":"2024-04-25T14:08:46.724991+0000","last_became_active":"2024-04-25T14:08:45.742516+0000","last_became_peered":"2024-04-25T14:08:45.742516+0000","last_unstale":"2024-04-25T14:08:46.724991+0000","last_undegraded":"2024-04-25T14:08:46.724991+0000","last_fullsized":"2024-04-25T14:08:46.724991+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:54.661962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.734401+0000","last_change":"2024-04-25T14:08:44.701347+0000","last_active":"2024-04-25T14:08:46.734401+0000","last_peered":"2024-04-25T14:08:46.734401+0000","last_clean":"2024-04-25T14:08:46.734401+0000","last_became_active":"2024-04-25T14:08:44.700917+0000","last_became_peered":"2024-04-25T14:08:44.700917+0000","last_unstale":"2024-04-25T14:08:46.734401+0000","last_undegraded":"2024-04-25T14:08:46.734401+0000","last_fullsized":"2024-04-25T14:08:46.734401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:23.016722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913784+0000","last_change":"2024-04-25T14:08:45.747902+0000","last_active":"2024-04-25T14:08:46.913784+0000","last_peered":"2024-04-25T14:08:46.913784+0000","last_clean":"2024-04-25T14:08:46.913784+0000","last_became_active":"2024-04-25T14:08:45.747592+0000","last_became_peered":"2024-04-25T14:08:45.747592+0000","last_unstale":"2024-04-25T14:08:46.913784+0000","last_undegraded":"2024-04-25T14:08:46.913784+0000","last_fullsized":"2024-04-25T14:08:46.913784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:46.478748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914146+0000","last_change":"2024-04-25T14:08:44.724767+0000","last_active":"2024-04-25T14:08:46.914146+0000","last_peered":"2024-04-25T14:08:46.914146+0000","last_clean":"2024-04-25T14:08:46.914146+0000","last_became_active":"2024-04-25T14:08:44.724442+0000","last_became_peered":"2024-04-25T14:08:44.724442+0000","last_unstale":"2024-04-25T14:08:46.914146+0000","last_undegraded":"2024-04-25T14:08:46.914146+0000","last_fullsized":"2024-04-25T14:08:46.914146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:48.142310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914222+0000","last_change":"2024-04-25T14:08:45.724582+0000","last_active":"2024-04-25T14:08:46.914222+0000","last_peered":"2024-04-25T14:08:46.914222+0000","last_clean":"2024-04-25T14:08:46.914222+0000","last_became_active":"2024-04-25T14:08:45.724295+0000","last_became_peered":"2024-04-25T14:08:45.724295+0000","last_unstale":"2024-04-25T14:08:46.914222+0000","last_undegraded":"2024-04-25T14:08:46.914222+0000","last_fullsized":"2024-04-25T14:08:46.914222+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:19.675356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.725933+0000","last_change":"2024-04-25T14:08:44.723570+0000","last_active":"2024-04-25T14:08:46.725933+0000","last_peered":"2024-04-25T14:08:46.725933+0000","last_clean":"2024-04-25T14:08:46.725933+0000","last_became_active":"2024-04-25T14:08:44.722656+0000","last_became_peered":"2024-04-25T14:08:44.722656+0000","last_unstale":"2024-04-25T14:08:46.725933+0000","last_undegraded":"2024-04-25T14:08:46.725933+0000","last_fullsized":"2024-04-25T14:08:46.725933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:27.739059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.726037+0000","last_change":"2024-04-25T14:08:45.736487+0000","last_active":"2024-04-25T14:08:46.726037+0000","last_peered":"2024-04-25T14:08:46.726037+0000","last_clean":"2024-04-25T14:08:46.726037+0000","last_became_active":"2024-04-25T14:08:45.736170+0000","last_became_peered":"2024-04-25T14:08:45.736170+0000","last_unstale":"2024-04-25T14:08:46.726037+0000","last_undegraded":"2024-04-25T14:08:46.726037+0000","last_fullsized":"2024-04-25T14:08:46.726037+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:20.662093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914019+0000","last_change":"2024-04-25T14:08:44.719472+0000","last_active":"2024-04-25T14:08:46.914019+0000","last_peered":"2024-04-25T14:08:46.914019+0000","last_clean":"2024-04-25T14:08:46.914019+0000","last_became_active":"2024-04-25T14:08:44.719159+0000","last_became_peered":"2024-04-25T14:08:44.719159+0000","last_unstale":"2024-04-25T14:08:46.914019+0000","last_undegraded":"2024-04-25T14:08:46.914019+0000","last_fullsized":"2024-04-25T14:08:46.914019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:58.834978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"76'100","reported_seq":333,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.725808+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:08:46.725808+0000","last_peered":"2024-04-25T14:08:46.725808+0000","last_clean":"2024-04-25T14:08:46.725808+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:08:46.725808+0000","last_undegraded":"2024-04-25T14:08:46.725808+0000","last_fullsized":"2024-04-25T14:08:46.725808+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":306,"num_read_kb":759,"num_write":149,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":306,"num_read_kb":759,"num_write":149,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32816,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1675264,"data_stored":1124203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574911,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32248,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339592,"statfs":{"total":96636764160,"available":96603742208,"internally_reserved":0,"allocated":1093632,"data_stored":535995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1073152,"data_stored":533779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574911,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32236,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339604,"statfs":{"total":96636764160,"available":96603754496,"internally_reserved":0,"allocated":1081344,"data_stored":533939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1073152,"data_stored":533779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574910,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1073152,"data_stored":533779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1077248,"data_stored":533849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574911,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32836,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339004,"statfs":{"total":96636764160,"available":96603140096,"internally_reserved":0,"allocated":1691648,"data_stored":1126465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:08:54.529 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:08:54.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.686+0000 7faf47600640 1 Processor -- start 2024-04-25T14:08:54.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.686+0000 7faf47600640 1 -- start start 2024-04-25T14:08:54.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.690+0000 7faf47600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7faf40070330 0x7faf400d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:54.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.690+0000 7faf47600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf400d1c20 0x7faf4006ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:54.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.690+0000 7faf47600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf400d2030 con 0x7faf4006f270 2024-04-25T14:08:54.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.690+0000 7faf47600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7faf4015e5c0 con 0x7faf40070330 2024-04-25T14:08:54.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.690+0000 7faf47600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7faf4015e8a0 con 0x7faf400d1c20 2024-04-25T14:08:54.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.690+0000 7faf45600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7faf4006f270 0x7faf4006f680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49414/0 (socket says 172.21.15.86:49414) 2024-04-25T14:08:54.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.690+0000 7faf45600640 1 -- 172.21.15.86:0/2578768474 learned_addr learned my addr 172.21.15.86:0/2578768474 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:54.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.690+0000 7faf44c00640 1 --2- 172.21.15.86:0/2578768474 >> v2:172.21.15.155:3300/0 conn(0x7faf40070330 0x7faf400d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:54.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.690+0000 7faf3fe00640 1 --2- 172.21.15.86:0/2578768474 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf400d1c20 0x7faf4006ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:54.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.690+0000 7faf3f400640 1 -- 172.21.15.86:0/2578768474 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3587516966 0 0) 0x7faf38001120 con 0x7faf4006f270 2024-04-25T14:08:54.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.690+0000 7faf3f400640 1 -- 172.21.15.86:0/2578768474 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf24003530 con 0x7faf4006f270 2024-04-25T14:08:54.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.690+0000 7faf3fe00640 1 -- 172.21.15.86:0/2578768474 >> v2:172.21.15.155:3300/0 conn(0x7faf40070330 msgr2=0x7faf400d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:54.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.690+0000 7faf3fe00640 1 --2- 172.21.15.86:0/2578768474 >> v2:172.21.15.155:3300/0 conn(0x7faf40070330 0x7faf400d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:54.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.690+0000 7faf3fe00640 1 -- 172.21.15.86:0/2578768474 >> v1:172.21.15.86:6789/0 conn(0x7faf4006f270 legacy=0x7faf4006f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:54.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.690+0000 7faf3fe00640 1 -- 172.21.15.86:0/2578768474 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4015eb80 con 0x7faf400d1c20 2024-04-25T14:08:54.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.690+0000 7faf44c00640 1 --2- 172.21.15.86:0/2578768474 >> v2:172.21.15.155:3300/0 conn(0x7faf40070330 0x7faf400d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:08:54.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.694+0000 7faf3fe00640 1 --2- 172.21.15.86:0/2578768474 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf400d1c20 0x7faf4006ed30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faf2c0097c0 tx=0x7faf2c01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa9382e1ca9ff5d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:54.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.694+0000 7faf3f400640 1 -- 172.21.15.86:0/2578768474 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c0b0070 con 0x7faf400d1c20 2024-04-25T14:08:54.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.694+0000 7faf3f400640 1 -- 172.21.15.86:0/2578768474 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf2c0ab600 con 0x7faf400d1c20 2024-04-25T14:08:54.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.694+0000 7faf3f400640 1 -- 172.21.15.86:0/2578768474 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c0ab9b0 con 0x7faf400d1c20 2024-04-25T14:08:54.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.694+0000 7faf47600640 1 -- 172.21.15.86:0/2578768474 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf400d1c20 msgr2=0x7faf4006ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:54.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.694+0000 7faf47600640 1 --2- 172.21.15.86:0/2578768474 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf400d1c20 0x7faf4006ed30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faf2c0097c0 tx=0x7faf2c01c410 comp rx=0 tx=0).stop 2024-04-25T14:08:54.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.694+0000 7faf47600640 1 -- 172.21.15.86:0/2578768474 shutdown_connections 2024-04-25T14:08:54.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.694+0000 7faf47600640 1 --2- 172.21.15.86:0/2578768474 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf400d1c20 0x7faf4006ed30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:54.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.694+0000 7faf47600640 1 --2- 172.21.15.86:0/2578768474 >> v2:172.21.15.155:3300/0 conn(0x7faf40070330 0x7faf400d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:54.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.694+0000 7faf47600640 1 -- 172.21.15.86:0/2578768474 >> 172.21.15.86:0/2578768474 conn(0x7faf400b4650 msgr2=0x7faf400b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:54.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.694+0000 7faf47600640 1 -- 172.21.15.86:0/2578768474 shutdown_connections 2024-04-25T14:08:54.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.694+0000 7faf47600640 1 -- 172.21.15.86:0/2578768474 wait complete. 2024-04-25T14:08:54.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.694+0000 7faf47600640 1 Processor -- start 2024-04-25T14:08:54.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.694+0000 7faf47600640 1 -- start start 2024-04-25T14:08:54.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.698+0000 7faf47600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf4006f270 0x7faf400c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:54.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.698+0000 7faf47600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf40070330 0x7faf400b1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:54.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.698+0000 7faf47600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf400d1c20 0x7faf4015b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:54.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.698+0000 7faf47600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faf400708c0 con 0x7faf400d1c20 2024-04-25T14:08:54.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.698+0000 7faf47600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faf40070740 con 0x7faf40070330 2024-04-25T14:08:54.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.698+0000 7faf47600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7faf4016afc0 con 0x7faf4006f270 2024-04-25T14:08:54.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.698+0000 7faf45600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf400d1c20 0x7faf4015b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:54.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.698+0000 7faf45600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf400d1c20 0x7faf4015b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40356/0 (socket says 172.21.15.86:40356) 2024-04-25T14:08:54.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.698+0000 7faf45600640 1 -- 172.21.15.86:0/1375984861 learned_addr learned my addr 172.21.15.86:0/1375984861 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:54.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.698+0000 7faf3fe00640 1 --2- 172.21.15.86:0/1375984861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf40070330 0x7faf400b1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:54.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.698+0000 7faf44c00640 1 --2- 172.21.15.86:0/1375984861 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf4006f270 0x7faf400c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:54.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.698+0000 7faf45600640 1 -- 172.21.15.86:0/1375984861 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf4006f270 msgr2=0x7faf400c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:54.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.698+0000 7faf45600640 1 --2- 172.21.15.86:0/1375984861 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf4006f270 0x7faf400c6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:54.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.698+0000 7faf45600640 1 -- 172.21.15.86:0/1375984861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf40070330 msgr2=0x7faf400b1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:54.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.698+0000 7faf45600640 1 --2- 172.21.15.86:0/1375984861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf40070330 0x7faf400b1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:54.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.698+0000 7faf45600640 1 -- 172.21.15.86:0/1375984861 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf400b0be0 con 0x7faf400d1c20 2024-04-25T14:08:54.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.698+0000 7faf45600640 1 --2- 172.21.15.86:0/1375984861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf400d1c20 0x7faf4015b070 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7faf3800c3c0 tx=0x7faf3800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:54.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.698+0000 7faf3cc00640 1 -- 172.21.15.86:0/1375984861 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38017070 con 0x7faf400d1c20 2024-04-25T14:08:54.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.698+0000 7faf3cc00640 1 -- 172.21.15.86:0/1375984861 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf38005cf0 con 0x7faf400d1c20 2024-04-25T14:08:54.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.698+0000 7faf3cc00640 1 -- 172.21.15.86:0/1375984861 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf380072d0 con 0x7faf400d1c20 2024-04-25T14:08:54.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.698+0000 7faf47600640 1 -- 172.21.15.86:0/1375984861 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf400b0ec0 con 0x7faf400d1c20 2024-04-25T14:08:54.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.698+0000 7faf47600640 1 -- 172.21.15.86:0/1375984861 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf400b1480 con 0x7faf400d1c20 2024-04-25T14:08:54.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.698+0000 7faf47600640 1 -- 172.21.15.86:0/1375984861 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf40069d00 con 0x7faf400d1c20 2024-04-25T14:08:54.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.702+0000 7faf3cc00640 1 -- 172.21.15.86:0/1375984861 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7faf3801a070 con 0x7faf400d1c20 2024-04-25T14:08:54.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.702+0000 7faf3cc00640 1 --2- 172.21.15.86:0/1375984861 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7faf2006f520 0x7faf20071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:54.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.702+0000 7faf3cc00640 1 -- 172.21.15.86:0/1375984861 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7faf3808e8f0 con 0x7faf400d1c20 2024-04-25T14:08:54.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.702+0000 7faf44c00640 1 --2- 172.21.15.86:0/1375984861 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7faf2006f520 0x7faf20071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:54.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.702+0000 7faf44c00640 1 --2- 172.21.15.86:0/1375984861 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7faf2006f520 0x7faf20071a30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faf30009bd0 tx=0x7faf3000b040 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:54.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.706+0000 7faf3cc00640 1 -- 172.21.15.86:0/1375984861 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faf400d2030 con 0x7faf400d1c20 2024-04-25T14:08:54.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.862+0000 7faf47600640 1 -- 172.21.15.86:0/1375984861 --> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faf40076bf0 con 0x7faf2006f520 2024-04-25T14:08:54.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.870+0000 7faf3cc00640 1 -- 172.21.15.86:0/1375984861 <== mgr.7092 v2:172.21.15.86:6832/3589556609 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252073 (secure 0 0 0) 0x7faf40076bf0 con 0x7faf2006f520 2024-04-25T14:08:54.873 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:08:54.874 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:08:54.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.874+0000 7faf47600640 1 -- 172.21.15.86:0/1375984861 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7faf2006f520 msgr2=0x7faf20071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:54.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.874+0000 7faf47600640 1 --2- 172.21.15.86:0/1375984861 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7faf2006f520 0x7faf20071a30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faf30009bd0 tx=0x7faf3000b040 comp rx=0 tx=0).stop 2024-04-25T14:08:54.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.874+0000 7faf47600640 1 -- 172.21.15.86:0/1375984861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf400d1c20 msgr2=0x7faf4015b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:54.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.874+0000 7faf47600640 1 --2- 172.21.15.86:0/1375984861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf400d1c20 0x7faf4015b070 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7faf3800c3c0 tx=0x7faf3800c880 comp rx=0 tx=0).stop 2024-04-25T14:08:54.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.874+0000 7faf47600640 1 -- 172.21.15.86:0/1375984861 shutdown_connections 2024-04-25T14:08:54.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.874+0000 7faf47600640 1 --2- 172.21.15.86:0/1375984861 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7faf2006f520 0x7faf20071a30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:54.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.874+0000 7faf47600640 1 --2- 172.21.15.86:0/1375984861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf400d1c20 0x7faf4015b070 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:54.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.874+0000 7faf47600640 1 --2- 172.21.15.86:0/1375984861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf40070330 0x7faf400b1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:54.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.874+0000 7faf47600640 1 --2- 172.21.15.86:0/1375984861 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf4006f270 0x7faf400c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:54.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.874+0000 7faf47600640 1 -- 172.21.15.86:0/1375984861 >> 172.21.15.86:0/1375984861 conn(0x7faf400b4650 msgr2=0x7faf400ce040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:54.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.874+0000 7faf47600640 1 -- 172.21.15.86:0/1375984861 shutdown_connections 2024-04-25T14:08:54.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:54.874+0000 7faf47600640 1 -- 172.21.15.86:0/1375984861 wait complete. 2024-04-25T14:08:54.895 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-04-25T14:08:53.360215+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":306,"num_read_kb":759,"num_write":194,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259052,"kb_used_data":9608,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715668,"statfs":{"total":773094113280,"available":772828844032,"internally_reserved":0,"allocated":9838592,"data_stored":5455788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001759"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.725728+0000","last_change":"2024-04-25T14:08:45.724883+0000","last_active":"2024-04-25T14:08:46.725728+0000","last_peered":"2024-04-25T14:08:46.725728+0000","last_clean":"2024-04-25T14:08:46.725728+0000","last_became_active":"2024-04-25T14:08:45.724574+0000","last_became_peered":"2024-04-25T14:08:45.724574+0000","last_unstale":"2024-04-25T14:08:46.725728+0000","last_undegraded":"2024-04-25T14:08:46.725728+0000","last_fullsized":"2024-04-25T14:08:46.725728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:08.058416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.734645+0000","last_change":"2024-04-25T14:08:45.742271+0000","last_active":"2024-04-25T14:08:46.734645+0000","last_peered":"2024-04-25T14:08:46.734645+0000","last_clean":"2024-04-25T14:08:46.734645+0000","last_became_active":"2024-04-25T14:08:45.741928+0000","last_became_peered":"2024-04-25T14:08:45.741928+0000","last_unstale":"2024-04-25T14:08:46.734645+0000","last_undegraded":"2024-04-25T14:08:46.734645+0000","last_fullsized":"2024-04-25T14:08:46.734645+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:17.275325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.725242+0000","last_change":"2024-04-25T14:08:45.716258+0000","last_active":"2024-04-25T14:08:46.725242+0000","last_peered":"2024-04-25T14:08:46.725242+0000","last_clean":"2024-04-25T14:08:46.725242+0000","last_became_active":"2024-04-25T14:08:45.715956+0000","last_became_peered":"2024-04-25T14:08:45.715956+0000","last_unstale":"2024-04-25T14:08:46.725242+0000","last_undegraded":"2024-04-25T14:08:46.725242+0000","last_fullsized":"2024-04-25T14:08:46.725242+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:13.811808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913585+0000","last_change":"2024-04-25T14:08:45.737138+0000","last_active":"2024-04-25T14:08:46.913585+0000","last_peered":"2024-04-25T14:08:46.913585+0000","last_clean":"2024-04-25T14:08:46.913585+0000","last_became_active":"2024-04-25T14:08:45.736851+0000","last_became_peered":"2024-04-25T14:08:45.736851+0000","last_unstale":"2024-04-25T14:08:46.913585+0000","last_undegraded":"2024-04-25T14:08:46.913585+0000","last_fullsized":"2024-04-25T14:08:46.913585+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:54.212949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.731558+0000","last_change":"2024-04-25T14:08:45.731770+0000","last_active":"2024-04-25T14:08:46.731558+0000","last_peered":"2024-04-25T14:08:46.731558+0000","last_clean":"2024-04-25T14:08:46.731558+0000","last_became_active":"2024-04-25T14:08:45.731424+0000","last_became_peered":"2024-04-25T14:08:45.731424+0000","last_unstale":"2024-04-25T14:08:46.731558+0000","last_undegraded":"2024-04-25T14:08:46.731558+0000","last_fullsized":"2024-04-25T14:08:46.731558+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:54.212949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913518+0000","last_change":"2024-04-25T14:08:45.747879+0000","last_active":"2024-04-25T14:08:46.913518+0000","last_peered":"2024-04-25T14:08:46.913518+0000","last_clean":"2024-04-25T14:08:46.913518+0000","last_became_active":"2024-04-25T14:08:45.747571+0000","last_became_peered":"2024-04-25T14:08:45.747571+0000","last_unstale":"2024-04-25T14:08:46.913518+0000","last_undegraded":"2024-04-25T14:08:46.913518+0000","last_fullsized":"2024-04-25T14:08:46.913518+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:54.075986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.725703+0000","last_change":"2024-04-25T14:08:45.733715+0000","last_active":"2024-04-25T14:08:46.725703+0000","last_peered":"2024-04-25T14:08:46.725703+0000","last_clean":"2024-04-25T14:08:46.725703+0000","last_became_active":"2024-04-25T14:08:45.733358+0000","last_became_peered":"2024-04-25T14:08:45.733358+0000","last_unstale":"2024-04-25T14:08:46.725703+0000","last_undegraded":"2024-04-25T14:08:46.725703+0000","last_fullsized":"2024-04-25T14:08:46.725703+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:13.811808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913320+0000","last_change":"2024-04-25T14:08:45.733157+0000","last_active":"2024-04-25T14:08:46.913320+0000","last_peered":"2024-04-25T14:08:46.913320+0000","last_clean":"2024-04-25T14:08:46.913320+0000","last_became_active":"2024-04-25T14:08:45.732454+0000","last_became_peered":"2024-04-25T14:08:45.732454+0000","last_unstale":"2024-04-25T14:08:46.913320+0000","last_undegraded":"2024-04-25T14:08:46.913320+0000","last_fullsized":"2024-04-25T14:08:46.913320+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.062105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.732022+0000","last_change":"2024-04-25T14:08:45.726250+0000","last_active":"2024-04-25T14:08:46.732022+0000","last_peered":"2024-04-25T14:08:46.732022+0000","last_clean":"2024-04-25T14:08:46.732022+0000","last_became_active":"2024-04-25T14:08:45.725924+0000","last_became_peered":"2024-04-25T14:08:45.725924+0000","last_unstale":"2024-04-25T14:08:46.732022+0000","last_undegraded":"2024-04-25T14:08:46.732022+0000","last_fullsized":"2024-04-25T14:08:46.732022+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:17.275325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.726883+0000","last_change":"2024-04-25T14:08:45.743158+0000","last_active":"2024-04-25T14:08:46.726883+0000","last_peered":"2024-04-25T14:08:46.726883+0000","last_clean":"2024-04-25T14:08:46.726883+0000","last_became_active":"2024-04-25T14:08:45.742796+0000","last_became_peered":"2024-04-25T14:08:45.742796+0000","last_unstale":"2024-04-25T14:08:46.726883+0000","last_undegraded":"2024-04-25T14:08:46.726883+0000","last_fullsized":"2024-04-25T14:08:46.726883+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:16.586153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.731847+0000","last_change":"2024-04-25T14:08:45.719110+0000","last_active":"2024-04-25T14:08:46.731847+0000","last_peered":"2024-04-25T14:08:46.731847+0000","last_clean":"2024-04-25T14:08:46.731847+0000","last_became_active":"2024-04-25T14:08:45.718275+0000","last_became_peered":"2024-04-25T14:08:45.718275+0000","last_unstale":"2024-04-25T14:08:46.731847+0000","last_undegraded":"2024-04-25T14:08:46.731847+0000","last_fullsized":"2024-04-25T14:08:46.731847+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:53.715664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.726564+0000","last_change":"2024-04-25T14:08:45.737568+0000","last_active":"2024-04-25T14:08:46.726564+0000","last_peered":"2024-04-25T14:08:46.726564+0000","last_clean":"2024-04-25T14:08:46.726564+0000","last_became_active":"2024-04-25T14:08:45.736104+0000","last_became_peered":"2024-04-25T14:08:45.736104+0000","last_unstale":"2024-04-25T14:08:46.726564+0000","last_undegraded":"2024-04-25T14:08:46.726564+0000","last_fullsized":"2024-04-25T14:08:46.726564+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:01.114117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913398+0000","last_change":"2024-04-25T14:08:45.732768+0000","last_active":"2024-04-25T14:08:46.913398+0000","last_peered":"2024-04-25T14:08:46.913398+0000","last_clean":"2024-04-25T14:08:46.913398+0000","last_became_active":"2024-04-25T14:08:45.731990+0000","last_became_peered":"2024-04-25T14:08:45.731990+0000","last_unstale":"2024-04-25T14:08:46.913398+0000","last_undegraded":"2024-04-25T14:08:46.913398+0000","last_fullsized":"2024-04-25T14:08:46.913398+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:29.581424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.731350+0000","last_change":"2024-04-25T14:08:45.743210+0000","last_active":"2024-04-25T14:08:46.731350+0000","last_peered":"2024-04-25T14:08:46.731350+0000","last_clean":"2024-04-25T14:08:46.731350+0000","last_became_active":"2024-04-25T14:08:45.742458+0000","last_became_peered":"2024-04-25T14:08:45.742458+0000","last_unstale":"2024-04-25T14:08:46.731350+0000","last_undegraded":"2024-04-25T14:08:46.731350+0000","last_fullsized":"2024-04-25T14:08:46.731350+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:46.478748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913806+0000","last_change":"2024-04-25T14:08:45.747944+0000","last_active":"2024-04-25T14:08:46.913806+0000","last_peered":"2024-04-25T14:08:46.913806+0000","last_clean":"2024-04-25T14:08:46.913806+0000","last_became_active":"2024-04-25T14:08:45.747552+0000","last_became_peered":"2024-04-25T14:08:45.747552+0000","last_unstale":"2024-04-25T14:08:46.913806+0000","last_undegraded":"2024-04-25T14:08:46.913806+0000","last_fullsized":"2024-04-25T14:08:46.913806+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:05.956072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913927+0000","last_change":"2024-04-25T14:08:45.735744+0000","last_active":"2024-04-25T14:08:46.913927+0000","last_peered":"2024-04-25T14:08:46.913927+0000","last_clean":"2024-04-25T14:08:46.913927+0000","last_became_active":"2024-04-25T14:08:45.735434+0000","last_became_peered":"2024-04-25T14:08:45.735434+0000","last_unstale":"2024-04-25T14:08:46.913927+0000","last_undegraded":"2024-04-25T14:08:46.913927+0000","last_fullsized":"2024-04-25T14:08:46.913927+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:15.671992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914798+0000","last_change":"2024-04-25T14:08:45.724202+0000","last_active":"2024-04-25T14:08:46.914798+0000","last_peered":"2024-04-25T14:08:46.914798+0000","last_clean":"2024-04-25T14:08:46.914798+0000","last_became_active":"2024-04-25T14:08:45.723887+0000","last_became_peered":"2024-04-25T14:08:45.723887+0000","last_unstale":"2024-04-25T14:08:46.914798+0000","last_undegraded":"2024-04-25T14:08:46.914798+0000","last_fullsized":"2024-04-25T14:08:46.914798+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:53.822693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.725298+0000","last_change":"2024-04-25T14:08:45.731600+0000","last_active":"2024-04-25T14:08:46.725298+0000","last_peered":"2024-04-25T14:08:46.725298+0000","last_clean":"2024-04-25T14:08:46.725298+0000","last_became_active":"2024-04-25T14:08:45.731022+0000","last_became_peered":"2024-04-25T14:08:45.731022+0000","last_unstale":"2024-04-25T14:08:46.725298+0000","last_undegraded":"2024-04-25T14:08:46.725298+0000","last_fullsized":"2024-04-25T14:08:46.725298+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:05.507269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.731847+0000","last_change":"2024-04-25T14:08:45.739967+0000","last_active":"2024-04-25T14:08:46.731847+0000","last_peered":"2024-04-25T14:08:46.731847+0000","last_clean":"2024-04-25T14:08:46.731847+0000","last_became_active":"2024-04-25T14:08:45.739654+0000","last_became_peered":"2024-04-25T14:08:45.739654+0000","last_unstale":"2024-04-25T14:08:46.731847+0000","last_undegraded":"2024-04-25T14:08:46.731847+0000","last_fullsized":"2024-04-25T14:08:46.731847+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:05.956072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913757+0000","last_change":"2024-04-25T14:08:45.752989+0000","last_active":"2024-04-25T14:08:46.913757+0000","last_peered":"2024-04-25T14:08:46.913757+0000","last_clean":"2024-04-25T14:08:46.913757+0000","last_became_active":"2024-04-25T14:08:45.752664+0000","last_became_peered":"2024-04-25T14:08:45.752664+0000","last_unstale":"2024-04-25T14:08:46.913757+0000","last_undegraded":"2024-04-25T14:08:46.913757+0000","last_fullsized":"2024-04-25T14:08:46.913757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:11.361524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.731422+0000","last_change":"2024-04-25T14:08:45.716503+0000","last_active":"2024-04-25T14:08:46.731422+0000","last_peered":"2024-04-25T14:08:46.731422+0000","last_clean":"2024-04-25T14:08:46.731422+0000","last_became_active":"2024-04-25T14:08:45.716171+0000","last_became_peered":"2024-04-25T14:08:45.716171+0000","last_unstale":"2024-04-25T14:08:46.731422+0000","last_undegraded":"2024-04-25T14:08:46.731422+0000","last_fullsized":"2024-04-25T14:08:46.731422+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:40.398506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.731368+0000","last_change":"2024-04-25T14:08:45.743523+0000","last_active":"2024-04-25T14:08:46.731368+0000","last_peered":"2024-04-25T14:08:46.731368+0000","last_clean":"2024-04-25T14:08:46.731368+0000","last_became_active":"2024-04-25T14:08:45.742912+0000","last_became_peered":"2024-04-25T14:08:45.742912+0000","last_unstale":"2024-04-25T14:08:46.731368+0000","last_undegraded":"2024-04-25T14:08:46.731368+0000","last_fullsized":"2024-04-25T14:08:46.731368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:29.581424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914084+0000","last_change":"2024-04-25T14:08:45.747282+0000","last_active":"2024-04-25T14:08:46.914084+0000","last_peered":"2024-04-25T14:08:46.914084+0000","last_clean":"2024-04-25T14:08:46.914084+0000","last_became_active":"2024-04-25T14:08:45.746134+0000","last_became_peered":"2024-04-25T14:08:45.746134+0000","last_unstale":"2024-04-25T14:08:46.914084+0000","last_undegraded":"2024-04-25T14:08:46.914084+0000","last_fullsized":"2024-04-25T14:08:46.914084+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:54.212949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.724829+0000","last_change":"2024-04-25T14:08:45.715888+0000","last_active":"2024-04-25T14:08:46.724829+0000","last_peered":"2024-04-25T14:08:46.724829+0000","last_clean":"2024-04-25T14:08:46.724829+0000","last_became_active":"2024-04-25T14:08:45.715578+0000","last_became_peered":"2024-04-25T14:08:45.715578+0000","last_unstale":"2024-04-25T14:08:46.724829+0000","last_undegraded":"2024-04-25T14:08:46.724829+0000","last_fullsized":"2024-04-25T14:08:46.724829+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:48.690101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.734671+0000","last_change":"2024-04-25T14:08:45.731656+0000","last_active":"2024-04-25T14:08:46.734671+0000","last_peered":"2024-04-25T14:08:46.734671+0000","last_clean":"2024-04-25T14:08:46.734671+0000","last_became_active":"2024-04-25T14:08:45.731321+0000","last_became_peered":"2024-04-25T14:08:45.731321+0000","last_unstale":"2024-04-25T14:08:46.734671+0000","last_undegraded":"2024-04-25T14:08:46.734671+0000","last_fullsized":"2024-04-25T14:08:46.734671+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:06:47.578625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.731951+0000","last_change":"2024-04-25T14:08:45.730637+0000","last_active":"2024-04-25T14:08:46.731951+0000","last_peered":"2024-04-25T14:08:46.731951+0000","last_clean":"2024-04-25T14:08:46.731951+0000","last_became_active":"2024-04-25T14:08:45.729611+0000","last_became_peered":"2024-04-25T14:08:45.729611+0000","last_unstale":"2024-04-25T14:08:46.731951+0000","last_undegraded":"2024-04-25T14:08:46.731951+0000","last_fullsized":"2024-04-25T14:08:46.731951+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.062105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.726811+0000","last_change":"2024-04-25T14:08:45.734547+0000","last_active":"2024-04-25T14:08:46.726811+0000","last_peered":"2024-04-25T14:08:46.726811+0000","last_clean":"2024-04-25T14:08:46.726811+0000","last_became_active":"2024-04-25T14:08:45.732727+0000","last_became_peered":"2024-04-25T14:08:45.732727+0000","last_unstale":"2024-04-25T14:08:46.726811+0000","last_undegraded":"2024-04-25T14:08:46.726811+0000","last_fullsized":"2024-04-25T14:08:46.726811+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:23.823392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913911+0000","last_change":"2024-04-25T14:08:45.731440+0000","last_active":"2024-04-25T14:08:46.913911+0000","last_peered":"2024-04-25T14:08:46.913911+0000","last_clean":"2024-04-25T14:08:46.913911+0000","last_became_active":"2024-04-25T14:08:45.728923+0000","last_became_peered":"2024-04-25T14:08:45.728923+0000","last_unstale":"2024-04-25T14:08:46.913911+0000","last_undegraded":"2024-04-25T14:08:46.913911+0000","last_fullsized":"2024-04-25T14:08:46.913911+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:53.715664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.725785+0000","last_change":"2024-04-25T14:08:45.736550+0000","last_active":"2024-04-25T14:08:46.725785+0000","last_peered":"2024-04-25T14:08:46.725785+0000","last_clean":"2024-04-25T14:08:46.725785+0000","last_became_active":"2024-04-25T14:08:45.736223+0000","last_became_peered":"2024-04-25T14:08:45.736223+0000","last_unstale":"2024-04-25T14:08:46.725785+0000","last_undegraded":"2024-04-25T14:08:46.725785+0000","last_fullsized":"2024-04-25T14:08:46.725785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:05.507269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.731588+0000","last_change":"2024-04-25T14:08:45.725621+0000","last_active":"2024-04-25T14:08:46.731588+0000","last_peered":"2024-04-25T14:08:46.731588+0000","last_clean":"2024-04-25T14:08:46.731588+0000","last_became_active":"2024-04-25T14:08:45.724591+0000","last_became_peered":"2024-04-25T14:08:45.724591+0000","last_unstale":"2024-04-25T14:08:46.731588+0000","last_undegraded":"2024-04-25T14:08:46.731588+0000","last_fullsized":"2024-04-25T14:08:46.731588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:28.745662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.734502+0000","last_change":"2024-04-25T14:08:45.738736+0000","last_active":"2024-04-25T14:08:46.734502+0000","last_peered":"2024-04-25T14:08:46.734502+0000","last_clean":"2024-04-25T14:08:46.734502+0000","last_became_active":"2024-04-25T14:08:45.738475+0000","last_became_peered":"2024-04-25T14:08:45.738475+0000","last_unstale":"2024-04-25T14:08:46.734502+0000","last_undegraded":"2024-04-25T14:08:46.734502+0000","last_fullsized":"2024-04-25T14:08:46.734502+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:40.321904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913605+0000","last_change":"2024-04-25T14:08:45.736497+0000","last_active":"2024-04-25T14:08:46.913605+0000","last_peered":"2024-04-25T14:08:46.913605+0000","last_clean":"2024-04-25T14:08:46.913605+0000","last_became_active":"2024-04-25T14:08:45.736104+0000","last_became_peered":"2024-04-25T14:08:45.736104+0000","last_unstale":"2024-04-25T14:08:46.913605+0000","last_undegraded":"2024-04-25T14:08:46.913605+0000","last_fullsized":"2024-04-25T14:08:46.913605+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:43.083404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914177+0000","last_change":"2024-04-25T14:08:45.718764+0000","last_active":"2024-04-25T14:08:46.914177+0000","last_peered":"2024-04-25T14:08:46.914177+0000","last_clean":"2024-04-25T14:08:46.914177+0000","last_became_active":"2024-04-25T14:08:45.718406+0000","last_became_peered":"2024-04-25T14:08:45.718406+0000","last_unstale":"2024-04-25T14:08:46.914177+0000","last_undegraded":"2024-04-25T14:08:46.914177+0000","last_fullsized":"2024-04-25T14:08:46.914177+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:04.740266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"86'4","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914320+0000","last_change":"2024-04-25T14:08:44.725130+0000","last_active":"2024-04-25T14:08:46.914320+0000","last_peered":"2024-04-25T14:08:46.914320+0000","last_clean":"2024-04-25T14:08:46.914320+0000","last_became_active":"2024-04-25T14:08:44.724805+0000","last_became_peered":"2024-04-25T14:08:44.724805+0000","last_unstale":"2024-04-25T14:08:46.914320+0000","last_undegraded":"2024-04-25T14:08:46.914320+0000","last_fullsized":"2024-04-25T14:08:46.914320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:36:34.689927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914651+0000","last_change":"2024-04-25T14:08:45.730683+0000","last_active":"2024-04-25T14:08:46.914651+0000","last_peered":"2024-04-25T14:08:46.914651+0000","last_clean":"2024-04-25T14:08:46.914651+0000","last_became_active":"2024-04-25T14:08:45.727410+0000","last_became_peered":"2024-04-25T14:08:45.727410+0000","last_unstale":"2024-04-25T14:08:46.914651+0000","last_undegraded":"2024-04-25T14:08:46.914651+0000","last_fullsized":"2024-04-25T14:08:46.914651+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:30.451014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.732084+0000","last_change":"2024-04-25T14:08:44.713248+0000","last_active":"2024-04-25T14:08:46.732084+0000","last_peered":"2024-04-25T14:08:46.732084+0000","last_clean":"2024-04-25T14:08:46.732084+0000","last_became_active":"2024-04-25T14:08:44.712237+0000","last_became_peered":"2024-04-25T14:08:44.712237+0000","last_unstale":"2024-04-25T14:08:46.732084+0000","last_undegraded":"2024-04-25T14:08:46.732084+0000","last_fullsized":"2024-04-25T14:08:46.732084+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:15.094910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.726120+0000","last_change":"2024-04-25T14:08:45.743649+0000","last_active":"2024-04-25T14:08:46.726120+0000","last_peered":"2024-04-25T14:08:46.726120+0000","last_clean":"2024-04-25T14:08:46.726120+0000","last_became_active":"2024-04-25T14:08:45.743231+0000","last_became_peered":"2024-04-25T14:08:45.743231+0000","last_unstale":"2024-04-25T14:08:46.726120+0000","last_undegraded":"2024-04-25T14:08:46.726120+0000","last_fullsized":"2024-04-25T14:08:46.726120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:07.671723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"86'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.726066+0000","last_change":"2024-04-25T14:08:44.725153+0000","last_active":"2024-04-25T14:08:46.726066+0000","last_peered":"2024-04-25T14:08:46.726066+0000","last_clean":"2024-04-25T14:08:46.726066+0000","last_became_active":"2024-04-25T14:08:44.724848+0000","last_became_peered":"2024-04-25T14:08:44.724848+0000","last_unstale":"2024-04-25T14:08:46.726066+0000","last_undegraded":"2024-04-25T14:08:46.726066+0000","last_fullsized":"2024-04-25T14:08:46.726066+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:52.709060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914906+0000","last_change":"2024-04-25T14:08:45.734599+0000","last_active":"2024-04-25T14:08:46.914906+0000","last_peered":"2024-04-25T14:08:46.914906+0000","last_clean":"2024-04-25T14:08:46.914906+0000","last_became_active":"2024-04-25T14:08:45.734284+0000","last_became_peered":"2024-04-25T14:08:45.734284+0000","last_unstale":"2024-04-25T14:08:46.914906+0000","last_undegraded":"2024-04-25T14:08:46.914906+0000","last_fullsized":"2024-04-25T14:08:46.914906+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:16.471242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"86'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913973+0000","last_change":"2024-04-25T14:08:44.720784+0000","last_active":"2024-04-25T14:08:46.913973+0000","last_peered":"2024-04-25T14:08:46.913973+0000","last_clean":"2024-04-25T14:08:46.913973+0000","last_became_active":"2024-04-25T14:08:44.720248+0000","last_became_peered":"2024-04-25T14:08:44.720248+0000","last_unstale":"2024-04-25T14:08:46.913973+0000","last_undegraded":"2024-04-25T14:08:46.913973+0000","last_fullsized":"2024-04-25T14:08:46.913973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:08.924639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914120+0000","last_change":"2024-04-25T14:08:45.746974+0000","last_active":"2024-04-25T14:08:46.914120+0000","last_peered":"2024-04-25T14:08:46.914120+0000","last_clean":"2024-04-25T14:08:46.914120+0000","last_became_active":"2024-04-25T14:08:45.746647+0000","last_became_peered":"2024-04-25T14:08:45.746647+0000","last_unstale":"2024-04-25T14:08:46.914120+0000","last_undegraded":"2024-04-25T14:08:46.914120+0000","last_fullsized":"2024-04-25T14:08:46.914120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:40.321904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.731561+0000","last_change":"2024-04-25T14:08:44.727021+0000","last_active":"2024-04-25T14:08:46.731561+0000","last_peered":"2024-04-25T14:08:46.731561+0000","last_clean":"2024-04-25T14:08:46.731561+0000","last_became_active":"2024-04-25T14:08:44.726079+0000","last_became_peered":"2024-04-25T14:08:44.726079+0000","last_unstale":"2024-04-25T14:08:46.731561+0000","last_undegraded":"2024-04-25T14:08:46.731561+0000","last_fullsized":"2024-04-25T14:08:46.731561+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:18.196654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913987+0000","last_change":"2024-04-25T14:08:45.738838+0000","last_active":"2024-04-25T14:08:46.913987+0000","last_peered":"2024-04-25T14:08:46.913987+0000","last_clean":"2024-04-25T14:08:46.913987+0000","last_became_active":"2024-04-25T14:08:45.737972+0000","last_became_peered":"2024-04-25T14:08:45.737972+0000","last_unstale":"2024-04-25T14:08:46.913987+0000","last_undegraded":"2024-04-25T14:08:46.913987+0000","last_fullsized":"2024-04-25T14:08:46.913987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:29.581424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914639+0000","last_change":"2024-04-25T14:08:44.726268+0000","last_active":"2024-04-25T14:08:46.914639+0000","last_peered":"2024-04-25T14:08:46.914639+0000","last_clean":"2024-04-25T14:08:46.914639+0000","last_became_active":"2024-04-25T14:08:44.725839+0000","last_became_peered":"2024-04-25T14:08:44.725839+0000","last_unstale":"2024-04-25T14:08:46.914639+0000","last_undegraded":"2024-04-25T14:08:46.914639+0000","last_fullsized":"2024-04-25T14:08:46.914639+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:51.117327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913688+0000","last_change":"2024-04-25T14:08:45.744967+0000","last_active":"2024-04-25T14:08:46.913688+0000","last_peered":"2024-04-25T14:08:46.913688+0000","last_clean":"2024-04-25T14:08:46.913688+0000","last_became_active":"2024-04-25T14:08:45.741905+0000","last_became_peered":"2024-04-25T14:08:45.741905+0000","last_unstale":"2024-04-25T14:08:46.913688+0000","last_undegraded":"2024-04-25T14:08:46.913688+0000","last_fullsized":"2024-04-25T14:08:46.913688+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:01.848398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.724998+0000","last_change":"2024-04-25T14:08:44.712147+0000","last_active":"2024-04-25T14:08:46.724998+0000","last_peered":"2024-04-25T14:08:46.724998+0000","last_clean":"2024-04-25T14:08:46.724998+0000","last_became_active":"2024-04-25T14:08:44.711799+0000","last_became_peered":"2024-04-25T14:08:44.711799+0000","last_unstale":"2024-04-25T14:08:46.724998+0000","last_undegraded":"2024-04-25T14:08:46.724998+0000","last_fullsized":"2024-04-25T14:08:46.724998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:53.206346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913438+0000","last_change":"2024-04-25T14:08:45.732535+0000","last_active":"2024-04-25T14:08:46.913438+0000","last_peered":"2024-04-25T14:08:46.913438+0000","last_clean":"2024-04-25T14:08:46.913438+0000","last_became_active":"2024-04-25T14:08:45.731982+0000","last_became_peered":"2024-04-25T14:08:45.731982+0000","last_unstale":"2024-04-25T14:08:46.913438+0000","last_undegraded":"2024-04-25T14:08:46.913438+0000","last_fullsized":"2024-04-25T14:08:46.913438+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:40.398506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.725664+0000","last_change":"2024-04-25T14:08:44.723883+0000","last_active":"2024-04-25T14:08:46.725664+0000","last_peered":"2024-04-25T14:08:46.725664+0000","last_clean":"2024-04-25T14:08:46.725664+0000","last_became_active":"2024-04-25T14:08:44.723229+0000","last_became_peered":"2024-04-25T14:08:44.723229+0000","last_unstale":"2024-04-25T14:08:46.725664+0000","last_undegraded":"2024-04-25T14:08:46.725664+0000","last_fullsized":"2024-04-25T14:08:46.725664+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:53.206346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.731751+0000","last_change":"2024-04-25T14:08:45.735428+0000","last_active":"2024-04-25T14:08:46.731751+0000","last_peered":"2024-04-25T14:08:46.731751+0000","last_clean":"2024-04-25T14:08:46.731751+0000","last_became_active":"2024-04-25T14:08:45.733452+0000","last_became_peered":"2024-04-25T14:08:45.733452+0000","last_unstale":"2024-04-25T14:08:46.731751+0000","last_undegraded":"2024-04-25T14:08:46.731751+0000","last_fullsized":"2024-04-25T14:08:46.731751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:11.361524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.734755+0000","last_change":"2024-04-25T14:08:44.723981+0000","last_active":"2024-04-25T14:08:46.734755+0000","last_peered":"2024-04-25T14:08:46.734755+0000","last_clean":"2024-04-25T14:08:46.734755+0000","last_became_active":"2024-04-25T14:08:44.723657+0000","last_became_peered":"2024-04-25T14:08:44.723657+0000","last_unstale":"2024-04-25T14:08:46.734755+0000","last_undegraded":"2024-04-25T14:08:46.734755+0000","last_fullsized":"2024-04-25T14:08:46.734755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:45.472144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.725985+0000","last_change":"2024-04-25T14:08:45.726814+0000","last_active":"2024-04-25T14:08:46.725985+0000","last_peered":"2024-04-25T14:08:46.725985+0000","last_clean":"2024-04-25T14:08:46.725985+0000","last_became_active":"2024-04-25T14:08:45.726491+0000","last_became_peered":"2024-04-25T14:08:45.726491+0000","last_unstale":"2024-04-25T14:08:46.725985+0000","last_undegraded":"2024-04-25T14:08:46.725985+0000","last_fullsized":"2024-04-25T14:08:46.725985+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:08.058416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.725401+0000","last_change":"2024-04-25T14:08:44.718626+0000","last_active":"2024-04-25T14:08:46.725401+0000","last_peered":"2024-04-25T14:08:46.725401+0000","last_clean":"2024-04-25T14:08:46.725401+0000","last_became_active":"2024-04-25T14:08:44.717237+0000","last_became_peered":"2024-04-25T14:08:44.717237+0000","last_unstale":"2024-04-25T14:08:46.725401+0000","last_undegraded":"2024-04-25T14:08:46.725401+0000","last_fullsized":"2024-04-25T14:08:46.725401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:27.739059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.725391+0000","last_change":"2024-04-25T14:08:45.722720+0000","last_active":"2024-04-25T14:08:46.725391+0000","last_peered":"2024-04-25T14:08:46.725391+0000","last_clean":"2024-04-25T14:08:46.725391+0000","last_became_active":"2024-04-25T14:08:45.720917+0000","last_became_peered":"2024-04-25T14:08:45.720917+0000","last_unstale":"2024-04-25T14:08:46.725391+0000","last_undegraded":"2024-04-25T14:08:46.725391+0000","last_fullsized":"2024-04-25T14:08:46.725391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:07.671723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"86'3","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.731990+0000","last_change":"2024-04-25T14:08:44.723221+0000","last_active":"2024-04-25T14:08:46.731990+0000","last_peered":"2024-04-25T14:08:46.731990+0000","last_clean":"2024-04-25T14:08:46.731990+0000","last_became_active":"2024-04-25T14:08:44.722729+0000","last_became_peered":"2024-04-25T14:08:44.722729+0000","last_unstale":"2024-04-25T14:08:46.731990+0000","last_undegraded":"2024-04-25T14:08:46.731990+0000","last_fullsized":"2024-04-25T14:08:46.731990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:56.689895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913636+0000","last_change":"2024-04-25T14:08:45.730816+0000","last_active":"2024-04-25T14:08:46.913636+0000","last_peered":"2024-04-25T14:08:46.913636+0000","last_clean":"2024-04-25T14:08:46.913636+0000","last_became_active":"2024-04-25T14:08:45.729265+0000","last_became_peered":"2024-04-25T14:08:45.729265+0000","last_unstale":"2024-04-25T14:08:46.913636+0000","last_undegraded":"2024-04-25T14:08:46.913636+0000","last_fullsized":"2024-04-25T14:08:46.913636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:28.745662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"86'6","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914365+0000","last_change":"2024-04-25T14:08:44.716853+0000","last_active":"2024-04-25T14:08:46.914365+0000","last_peered":"2024-04-25T14:08:46.914365+0000","last_clean":"2024-04-25T14:08:46.914365+0000","last_became_active":"2024-04-25T14:08:44.716479+0000","last_became_peered":"2024-04-25T14:08:44.716479+0000","last_unstale":"2024-04-25T14:08:46.914365+0000","last_undegraded":"2024-04-25T14:08:46.914365+0000","last_fullsized":"2024-04-25T14:08:46.914365+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:11.127296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914199+0000","last_change":"2024-04-25T14:08:45.733994+0000","last_active":"2024-04-25T14:08:46.914199+0000","last_peered":"2024-04-25T14:08:46.914199+0000","last_clean":"2024-04-25T14:08:46.914199+0000","last_became_active":"2024-04-25T14:08:45.732848+0000","last_became_peered":"2024-04-25T14:08:45.732848+0000","last_unstale":"2024-04-25T14:08:46.914199+0000","last_undegraded":"2024-04-25T14:08:46.914199+0000","last_fullsized":"2024-04-25T14:08:46.914199+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:17.275325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"86'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.725901+0000","last_change":"2024-04-25T14:08:44.726934+0000","last_active":"2024-04-25T14:08:46.725901+0000","last_peered":"2024-04-25T14:08:46.725901+0000","last_clean":"2024-04-25T14:08:46.725901+0000","last_became_active":"2024-04-25T14:08:44.726583+0000","last_became_peered":"2024-04-25T14:08:44.726583+0000","last_unstale":"2024-04-25T14:08:46.725901+0000","last_undegraded":"2024-04-25T14:08:46.725901+0000","last_fullsized":"2024-04-25T14:08:46.725901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:10.354920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.731333+0000","last_change":"2024-04-25T14:08:45.727637+0000","last_active":"2024-04-25T14:08:46.731333+0000","last_peered":"2024-04-25T14:08:46.731333+0000","last_clean":"2024-04-25T14:08:46.731333+0000","last_became_active":"2024-04-25T14:08:45.727312+0000","last_became_peered":"2024-04-25T14:08:45.727312+0000","last_unstale":"2024-04-25T14:08:46.731333+0000","last_undegraded":"2024-04-25T14:08:46.731333+0000","last_fullsized":"2024-04-25T14:08:46.731333+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:24.023326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913920+0000","last_change":"2024-04-25T14:08:44.721083+0000","last_active":"2024-04-25T14:08:46.913920+0000","last_peered":"2024-04-25T14:08:46.913920+0000","last_clean":"2024-04-25T14:08:46.913920+0000","last_became_active":"2024-04-25T14:08:44.720775+0000","last_became_peered":"2024-04-25T14:08:44.720775+0000","last_unstale":"2024-04-25T14:08:46.913920+0000","last_undegraded":"2024-04-25T14:08:46.913920+0000","last_fullsized":"2024-04-25T14:08:46.913920+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:19.655489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.726394+0000","last_change":"2024-04-25T14:08:45.749537+0000","last_active":"2024-04-25T14:08:46.726394+0000","last_peered":"2024-04-25T14:08:46.726394+0000","last_clean":"2024-04-25T14:08:46.726394+0000","last_became_active":"2024-04-25T14:08:45.748947+0000","last_became_peered":"2024-04-25T14:08:45.748947+0000","last_unstale":"2024-04-25T14:08:46.726394+0000","last_undegraded":"2024-04-25T14:08:46.726394+0000","last_fullsized":"2024-04-25T14:08:46.726394+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:52.873960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913865+0000","last_change":"2024-04-25T14:08:44.723709+0000","last_active":"2024-04-25T14:08:46.913865+0000","last_peered":"2024-04-25T14:08:46.913865+0000","last_clean":"2024-04-25T14:08:46.913865+0000","last_became_active":"2024-04-25T14:08:44.723387+0000","last_became_peered":"2024-04-25T14:08:44.723387+0000","last_unstale":"2024-04-25T14:08:46.913865+0000","last_undegraded":"2024-04-25T14:08:46.913865+0000","last_fullsized":"2024-04-25T14:08:46.913865+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:37.926762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.726651+0000","last_change":"2024-04-25T14:08:45.728344+0000","last_active":"2024-04-25T14:08:46.726651+0000","last_peered":"2024-04-25T14:08:46.726651+0000","last_clean":"2024-04-25T14:08:46.726651+0000","last_became_active":"2024-04-25T14:08:45.728020+0000","last_became_peered":"2024-04-25T14:08:45.728020+0000","last_unstale":"2024-04-25T14:08:46.726651+0000","last_undegraded":"2024-04-25T14:08:46.726651+0000","last_fullsized":"2024-04-25T14:08:46.726651+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:42:36.456843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"86'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914225+0000","last_change":"2024-04-25T14:08:44.728205+0000","last_active":"2024-04-25T14:08:46.914225+0000","last_peered":"2024-04-25T14:08:46.914225+0000","last_clean":"2024-04-25T14:08:46.914225+0000","last_became_active":"2024-04-25T14:08:44.727910+0000","last_became_peered":"2024-04-25T14:08:44.727910+0000","last_unstale":"2024-04-25T14:08:46.914225+0000","last_undegraded":"2024-04-25T14:08:46.914225+0000","last_fullsized":"2024-04-25T14:08:46.914225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:56.689895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914357+0000","last_change":"2024-04-25T14:08:45.720863+0000","last_active":"2024-04-25T14:08:46.914357+0000","last_peered":"2024-04-25T14:08:46.914357+0000","last_clean":"2024-04-25T14:08:46.914357+0000","last_became_active":"2024-04-25T14:08:45.720540+0000","last_became_peered":"2024-04-25T14:08:45.720540+0000","last_unstale":"2024-04-25T14:08:46.914357+0000","last_undegraded":"2024-04-25T14:08:46.914357+0000","last_fullsized":"2024-04-25T14:08:46.914357+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:14.715491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914420+0000","last_change":"2024-04-25T14:08:44.713564+0000","last_active":"2024-04-25T14:08:46.914420+0000","last_peered":"2024-04-25T14:08:46.914420+0000","last_clean":"2024-04-25T14:08:46.914420+0000","last_became_active":"2024-04-25T14:08:44.712475+0000","last_became_peered":"2024-04-25T14:08:44.712475+0000","last_unstale":"2024-04-25T14:08:46.914420+0000","last_undegraded":"2024-04-25T14:08:46.914420+0000","last_fullsized":"2024-04-25T14:08:46.914420+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:22.832096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914098+0000","last_change":"2024-04-25T14:08:45.732385+0000","last_active":"2024-04-25T14:08:46.914098+0000","last_peered":"2024-04-25T14:08:46.914098+0000","last_clean":"2024-04-25T14:08:46.914098+0000","last_became_active":"2024-04-25T14:08:45.731872+0000","last_became_peered":"2024-04-25T14:08:45.731872+0000","last_unstale":"2024-04-25T14:08:46.914098+0000","last_undegraded":"2024-04-25T14:08:46.914098+0000","last_fullsized":"2024-04-25T14:08:46.914098+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:28.745662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"86'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.726265+0000","last_change":"2024-04-25T14:08:44.718093+0000","last_active":"2024-04-25T14:08:46.726265+0000","last_peered":"2024-04-25T14:08:46.726265+0000","last_clean":"2024-04-25T14:08:46.726265+0000","last_became_active":"2024-04-25T14:08:44.717768+0000","last_became_peered":"2024-04-25T14:08:44.717768+0000","last_unstale":"2024-04-25T14:08:46.726265+0000","last_undegraded":"2024-04-25T14:08:46.726265+0000","last_fullsized":"2024-04-25T14:08:46.726265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:04.949468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.731960+0000","last_change":"2024-04-25T14:08:45.734270+0000","last_active":"2024-04-25T14:08:46.731960+0000","last_peered":"2024-04-25T14:08:46.731960+0000","last_clean":"2024-04-25T14:08:46.731960+0000","last_became_active":"2024-04-25T14:08:45.733977+0000","last_became_peered":"2024-04-25T14:08:45.733977+0000","last_unstale":"2024-04-25T14:08:46.731960+0000","last_undegraded":"2024-04-25T14:08:46.731960+0000","last_fullsized":"2024-04-25T14:08:46.731960+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:40.321904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.732079+0000","last_change":"2024-04-25T14:08:44.717774+0000","last_active":"2024-04-25T14:08:46.732079+0000","last_peered":"2024-04-25T14:08:46.732079+0000","last_clean":"2024-04-25T14:08:46.732079+0000","last_became_active":"2024-04-25T14:08:44.717425+0000","last_became_peered":"2024-04-25T14:08:44.717425+0000","last_unstale":"2024-04-25T14:08:46.732079+0000","last_undegraded":"2024-04-25T14:08:46.732079+0000","last_fullsized":"2024-04-25T14:08:46.732079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:59.659739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913769+0000","last_change":"2024-04-25T14:08:45.732941+0000","last_active":"2024-04-25T14:08:46.913769+0000","last_peered":"2024-04-25T14:08:46.913769+0000","last_clean":"2024-04-25T14:08:46.913769+0000","last_became_active":"2024-04-25T14:08:45.732613+0000","last_became_peered":"2024-04-25T14:08:45.732613+0000","last_unstale":"2024-04-25T14:08:46.913769+0000","last_undegraded":"2024-04-25T14:08:46.913769+0000","last_fullsized":"2024-04-25T14:08:46.913769+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:49.711152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"86'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.734348+0000","last_change":"2024-04-25T14:08:44.700728+0000","last_active":"2024-04-25T14:08:46.734348+0000","last_peered":"2024-04-25T14:08:46.734348+0000","last_clean":"2024-04-25T14:08:46.734348+0000","last_became_active":"2024-04-25T14:08:44.700344+0000","last_became_peered":"2024-04-25T14:08:44.700344+0000","last_unstale":"2024-04-25T14:08:46.734348+0000","last_undegraded":"2024-04-25T14:08:46.734348+0000","last_fullsized":"2024-04-25T14:08:46.734348+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:10.155953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.734346+0000","last_change":"2024-04-25T14:08:45.736125+0000","last_active":"2024-04-25T14:08:46.734346+0000","last_peered":"2024-04-25T14:08:46.734346+0000","last_clean":"2024-04-25T14:08:46.734346+0000","last_became_active":"2024-04-25T14:08:45.735713+0000","last_became_peered":"2024-04-25T14:08:45.735713+0000","last_unstale":"2024-04-25T14:08:46.734346+0000","last_undegraded":"2024-04-25T14:08:46.734346+0000","last_fullsized":"2024-04-25T14:08:46.734346+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.062105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.726409+0000","last_change":"2024-04-25T14:08:44.725383+0000","last_active":"2024-04-25T14:08:46.726409+0000","last_peered":"2024-04-25T14:08:46.726409+0000","last_clean":"2024-04-25T14:08:46.726409+0000","last_became_active":"2024-04-25T14:08:44.724211+0000","last_became_peered":"2024-04-25T14:08:44.724211+0000","last_unstale":"2024-04-25T14:08:46.726409+0000","last_undegraded":"2024-04-25T14:08:46.726409+0000","last_fullsized":"2024-04-25T14:08:46.726409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:42.017831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.734379+0000","last_change":"2024-04-25T14:08:45.741432+0000","last_active":"2024-04-25T14:08:46.734379+0000","last_peered":"2024-04-25T14:08:46.734379+0000","last_clean":"2024-04-25T14:08:46.734379+0000","last_became_active":"2024-04-25T14:08:45.741069+0000","last_became_peered":"2024-04-25T14:08:45.741069+0000","last_unstale":"2024-04-25T14:08:46.734379+0000","last_undegraded":"2024-04-25T14:08:46.734379+0000","last_fullsized":"2024-04-25T14:08:46.734379+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:53.715664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.734576+0000","last_change":"2024-04-25T14:08:44.716954+0000","last_active":"2024-04-25T14:08:46.734576+0000","last_peered":"2024-04-25T14:08:46.734576+0000","last_clean":"2024-04-25T14:08:46.734576+0000","last_became_active":"2024-04-25T14:08:44.716583+0000","last_became_peered":"2024-04-25T14:08:44.716583+0000","last_unstale":"2024-04-25T14:08:46.734576+0000","last_undegraded":"2024-04-25T14:08:46.734576+0000","last_fullsized":"2024-04-25T14:08:46.734576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:41.020517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914184+0000","last_change":"2024-04-25T14:08:45.733159+0000","last_active":"2024-04-25T14:08:46.914184+0000","last_peered":"2024-04-25T14:08:46.914184+0000","last_clean":"2024-04-25T14:08:46.914184+0000","last_became_active":"2024-04-25T14:08:45.732850+0000","last_became_peered":"2024-04-25T14:08:45.732850+0000","last_unstale":"2024-04-25T14:08:46.914184+0000","last_undegraded":"2024-04-25T14:08:46.914184+0000","last_fullsized":"2024-04-25T14:08:46.914184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:01.325944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.726323+0000","last_change":"2024-04-25T14:08:44.725423+0000","last_active":"2024-04-25T14:08:46.726323+0000","last_peered":"2024-04-25T14:08:46.726323+0000","last_clean":"2024-04-25T14:08:46.726323+0000","last_became_active":"2024-04-25T14:08:44.725036+0000","last_became_peered":"2024-04-25T14:08:44.725036+0000","last_unstale":"2024-04-25T14:08:46.726323+0000","last_undegraded":"2024-04-25T14:08:46.726323+0000","last_fullsized":"2024-04-25T14:08:46.726323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:49.606537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914011+0000","last_change":"2024-04-25T14:08:45.731115+0000","last_active":"2024-04-25T14:08:46.914011+0000","last_peered":"2024-04-25T14:08:46.914011+0000","last_clean":"2024-04-25T14:08:46.914011+0000","last_became_active":"2024-04-25T14:08:45.730623+0000","last_became_peered":"2024-04-25T14:08:45.730623+0000","last_unstale":"2024-04-25T14:08:46.914011+0000","last_undegraded":"2024-04-25T14:08:46.914011+0000","last_fullsized":"2024-04-25T14:08:46.914011+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:40.398506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"86'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.726092+0000","last_change":"2024-04-25T14:08:44.717040+0000","last_active":"2024-04-25T14:08:46.726092+0000","last_peered":"2024-04-25T14:08:46.726092+0000","last_clean":"2024-04-25T14:08:46.726092+0000","last_became_active":"2024-04-25T14:08:44.716044+0000","last_became_peered":"2024-04-25T14:08:44.716044+0000","last_unstale":"2024-04-25T14:08:46.726092+0000","last_undegraded":"2024-04-25T14:08:46.726092+0000","last_fullsized":"2024-04-25T14:08:46.726092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:12.055502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914788+0000","last_change":"2024-04-25T14:08:45.718131+0000","last_active":"2024-04-25T14:08:46.914788+0000","last_peered":"2024-04-25T14:08:46.914788+0000","last_clean":"2024-04-25T14:08:46.914788+0000","last_became_active":"2024-04-25T14:08:45.717784+0000","last_became_peered":"2024-04-25T14:08:45.717784+0000","last_unstale":"2024-04-25T14:08:46.914788+0000","last_undegraded":"2024-04-25T14:08:46.914788+0000","last_fullsized":"2024-04-25T14:08:46.914788+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:37.700457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"87'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.732152+0000","last_change":"2024-04-25T14:08:44.704043+0000","last_active":"2024-04-25T14:08:46.732152+0000","last_peered":"2024-04-25T14:08:46.732152+0000","last_clean":"2024-04-25T14:08:46.732152+0000","last_became_active":"2024-04-25T14:08:44.703697+0000","last_became_peered":"2024-04-25T14:08:44.703697+0000","last_unstale":"2024-04-25T14:08:46.732152+0000","last_undegraded":"2024-04-25T14:08:46.732152+0000","last_fullsized":"2024-04-25T14:08:46.732152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:00.497301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914741+0000","last_change":"2024-04-25T14:08:45.739068+0000","last_active":"2024-04-25T14:08:46.914741+0000","last_peered":"2024-04-25T14:08:46.914741+0000","last_clean":"2024-04-25T14:08:46.914741+0000","last_became_active":"2024-04-25T14:08:45.738777+0000","last_became_peered":"2024-04-25T14:08:45.738777+0000","last_unstale":"2024-04-25T14:08:46.914741+0000","last_undegraded":"2024-04-25T14:08:46.914741+0000","last_fullsized":"2024-04-25T14:08:46.914741+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:47:40.495887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.732098+0000","last_change":"2024-04-25T14:08:44.721286+0000","last_active":"2024-04-25T14:08:46.732098+0000","last_peered":"2024-04-25T14:08:46.732098+0000","last_clean":"2024-04-25T14:08:46.732098+0000","last_became_active":"2024-04-25T14:08:44.720949+0000","last_became_peered":"2024-04-25T14:08:44.720949+0000","last_unstale":"2024-04-25T14:08:46.732098+0000","last_undegraded":"2024-04-25T14:08:46.732098+0000","last_fullsized":"2024-04-25T14:08:46.732098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:50.763491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.734687+0000","last_change":"2024-04-25T14:08:45.728752+0000","last_active":"2024-04-25T14:08:46.734687+0000","last_peered":"2024-04-25T14:08:46.734687+0000","last_clean":"2024-04-25T14:08:46.734687+0000","last_became_active":"2024-04-25T14:08:45.728433+0000","last_became_peered":"2024-04-25T14:08:45.728433+0000","last_unstale":"2024-04-25T14:08:46.734687+0000","last_undegraded":"2024-04-25T14:08:46.734687+0000","last_fullsized":"2024-04-25T14:08:46.734687+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:56.301529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914143+0000","last_change":"2024-04-25T14:08:44.725093+0000","last_active":"2024-04-25T14:08:46.914143+0000","last_peered":"2024-04-25T14:08:46.914143+0000","last_clean":"2024-04-25T14:08:46.914143+0000","last_became_active":"2024-04-25T14:08:44.724761+0000","last_became_peered":"2024-04-25T14:08:44.724761+0000","last_unstale":"2024-04-25T14:08:46.914143+0000","last_undegraded":"2024-04-25T14:08:46.914143+0000","last_fullsized":"2024-04-25T14:08:46.914143+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:41.595810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.726210+0000","last_change":"2024-04-25T14:08:45.739386+0000","last_active":"2024-04-25T14:08:46.726210+0000","last_peered":"2024-04-25T14:08:46.726210+0000","last_clean":"2024-04-25T14:08:46.726210+0000","last_became_active":"2024-04-25T14:08:45.738955+0000","last_became_peered":"2024-04-25T14:08:45.738955+0000","last_unstale":"2024-04-25T14:08:46.726210+0000","last_undegraded":"2024-04-25T14:08:46.726210+0000","last_fullsized":"2024-04-25T14:08:46.726210+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:52.341358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914289+0000","last_change":"2024-04-25T14:08:44.721282+0000","last_active":"2024-04-25T14:08:46.914289+0000","last_peered":"2024-04-25T14:08:46.914289+0000","last_clean":"2024-04-25T14:08:46.914289+0000","last_became_active":"2024-04-25T14:08:44.721077+0000","last_became_peered":"2024-04-25T14:08:44.721077+0000","last_unstale":"2024-04-25T14:08:46.914289+0000","last_undegraded":"2024-04-25T14:08:46.914289+0000","last_fullsized":"2024-04-25T14:08:46.914289+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:32.296318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.724991+0000","last_change":"2024-04-25T14:08:45.742804+0000","last_active":"2024-04-25T14:08:46.724991+0000","last_peered":"2024-04-25T14:08:46.724991+0000","last_clean":"2024-04-25T14:08:46.724991+0000","last_became_active":"2024-04-25T14:08:45.742516+0000","last_became_peered":"2024-04-25T14:08:45.742516+0000","last_unstale":"2024-04-25T14:08:46.724991+0000","last_undegraded":"2024-04-25T14:08:46.724991+0000","last_fullsized":"2024-04-25T14:08:46.724991+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:54.661962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.734401+0000","last_change":"2024-04-25T14:08:44.701347+0000","last_active":"2024-04-25T14:08:46.734401+0000","last_peered":"2024-04-25T14:08:46.734401+0000","last_clean":"2024-04-25T14:08:46.734401+0000","last_became_active":"2024-04-25T14:08:44.700917+0000","last_became_peered":"2024-04-25T14:08:44.700917+0000","last_unstale":"2024-04-25T14:08:46.734401+0000","last_undegraded":"2024-04-25T14:08:46.734401+0000","last_fullsized":"2024-04-25T14:08:46.734401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:23.016722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.913784+0000","last_change":"2024-04-25T14:08:45.747902+0000","last_active":"2024-04-25T14:08:46.913784+0000","last_peered":"2024-04-25T14:08:46.913784+0000","last_clean":"2024-04-25T14:08:46.913784+0000","last_became_active":"2024-04-25T14:08:45.747592+0000","last_became_peered":"2024-04-25T14:08:45.747592+0000","last_unstale":"2024-04-25T14:08:46.913784+0000","last_undegraded":"2024-04-25T14:08:46.913784+0000","last_fullsized":"2024-04-25T14:08:46.913784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:46.478748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914146+0000","last_change":"2024-04-25T14:08:44.724767+0000","last_active":"2024-04-25T14:08:46.914146+0000","last_peered":"2024-04-25T14:08:46.914146+0000","last_clean":"2024-04-25T14:08:46.914146+0000","last_became_active":"2024-04-25T14:08:44.724442+0000","last_became_peered":"2024-04-25T14:08:44.724442+0000","last_unstale":"2024-04-25T14:08:46.914146+0000","last_undegraded":"2024-04-25T14:08:46.914146+0000","last_fullsized":"2024-04-25T14:08:46.914146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:48.142310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914222+0000","last_change":"2024-04-25T14:08:45.724582+0000","last_active":"2024-04-25T14:08:46.914222+0000","last_peered":"2024-04-25T14:08:46.914222+0000","last_clean":"2024-04-25T14:08:46.914222+0000","last_became_active":"2024-04-25T14:08:45.724295+0000","last_became_peered":"2024-04-25T14:08:45.724295+0000","last_unstale":"2024-04-25T14:08:46.914222+0000","last_undegraded":"2024-04-25T14:08:46.914222+0000","last_fullsized":"2024-04-25T14:08:46.914222+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:19.675356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.725933+0000","last_change":"2024-04-25T14:08:44.723570+0000","last_active":"2024-04-25T14:08:46.725933+0000","last_peered":"2024-04-25T14:08:46.725933+0000","last_clean":"2024-04-25T14:08:46.725933+0000","last_became_active":"2024-04-25T14:08:44.722656+0000","last_became_peered":"2024-04-25T14:08:44.722656+0000","last_unstale":"2024-04-25T14:08:46.725933+0000","last_undegraded":"2024-04-25T14:08:46.725933+0000","last_fullsized":"2024-04-25T14:08:46.725933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:27.739059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.726037+0000","last_change":"2024-04-25T14:08:45.736487+0000","last_active":"2024-04-25T14:08:46.726037+0000","last_peered":"2024-04-25T14:08:46.726037+0000","last_clean":"2024-04-25T14:08:46.726037+0000","last_became_active":"2024-04-25T14:08:45.736170+0000","last_became_peered":"2024-04-25T14:08:45.736170+0000","last_unstale":"2024-04-25T14:08:46.726037+0000","last_undegraded":"2024-04-25T14:08:46.726037+0000","last_fullsized":"2024-04-25T14:08:46.726037+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:44.686737+0000","last_clean_scrub_stamp":"2024-04-25T14:08:44.686737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:20.662093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.914019+0000","last_change":"2024-04-25T14:08:44.719472+0000","last_active":"2024-04-25T14:08:46.914019+0000","last_peered":"2024-04-25T14:08:46.914019+0000","last_clean":"2024-04-25T14:08:46.914019+0000","last_became_active":"2024-04-25T14:08:44.719159+0000","last_became_peered":"2024-04-25T14:08:44.719159+0000","last_unstale":"2024-04-25T14:08:46.914019+0000","last_undegraded":"2024-04-25T14:08:46.914019+0000","last_fullsized":"2024-04-25T14:08:46.914019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:08:43.680134+0000","last_clean_scrub_stamp":"2024-04-25T14:08:43.680134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:58.834978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"76'100","reported_seq":333,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T14:08:46.725808+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:08:46.725808+0000","last_peered":"2024-04-25T14:08:46.725808+0000","last_clean":"2024-04-25T14:08:46.725808+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:08:46.725808+0000","last_undegraded":"2024-04-25T14:08:46.725808+0000","last_fullsized":"2024-04-25T14:08:46.725808+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":306,"num_read_kb":759,"num_write":149,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":306,"num_read_kb":759,"num_write":149,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32816,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1675264,"data_stored":1124203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574911,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32248,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339592,"statfs":{"total":96636764160,"available":96603742208,"internally_reserved":0,"allocated":1093632,"data_stored":535995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1073152,"data_stored":533779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574911,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32236,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339604,"statfs":{"total":96636764160,"available":96603754496,"internally_reserved":0,"allocated":1081344,"data_stored":533939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1073152,"data_stored":533779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574910,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1073152,"data_stored":533779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1077248,"data_stored":533849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574911,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32836,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339004,"statfs":{"total":96636764160,"available":96603140096,"internally_reserved":0,"allocated":1691648,"data_stored":1126465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:08:54.898 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T14:08:54.898 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T14:08:55.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.090+0000 7fa103400640 1 Processor -- start 2024-04-25T14:08:55.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.090+0000 7fa103400640 1 -- start start 2024-04-25T14:08:55.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.090+0000 7fa103400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0fc0d16e0 0x7fa0fc0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:55.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.090+0000 7fa103400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0fc0d2030 con 0x7fa0fc0730b0 2024-04-25T14:08:55.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.090+0000 7fa103400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0fc072bd0 con 0x7fa0fc0cfca0 2024-04-25T14:08:55.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.090+0000 7fa103400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa0fc070470 con 0x7fa0fc0d16e0 2024-04-25T14:08:55.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.090+0000 7fa101400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa0fc0730b0 0x7fa0fc0734c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49422/0 (socket says 172.21.15.86:49422) 2024-04-25T14:08:55.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.090+0000 7fa101400640 1 -- 172.21.15.86:0/3945663637 learned_addr learned my addr 172.21.15.86:0/3945663637 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:55.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.090+0000 7fa0f3e00640 1 --2- 172.21.15.86:0/3945663637 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0fc0d16e0 0x7fa0fc0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:55.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.090+0000 7fa0fbe00640 1 -- 172.21.15.86:0/3945663637 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3108978786 0 0) 0x7fa0f40014a0 con 0x7fa0fc0730b0 2024-04-25T14:08:55.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.090+0000 7fa0fbe00640 1 -- 172.21.15.86:0/3945663637 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0e0003530 con 0x7fa0fc0730b0 2024-04-25T14:08:55.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.090+0000 7fa0fbe00640 1 -- 172.21.15.86:0/3945663637 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2764959934 0 0) 0x7fa0fc072bd0 con 0x7fa0fc0cfca0 2024-04-25T14:08:55.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.090+0000 7fa0fbe00640 1 -- 172.21.15.86:0/3945663637 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0f40014a0 con 0x7fa0fc0cfca0 2024-04-25T14:08:55.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.094+0000 7fa0f3e00640 1 -- 172.21.15.86:0/3945663637 >> v1:172.21.15.155:6789/0 conn(0x7fa0fc0cfca0 legacy=0x7fa0fc06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:55.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.094+0000 7fa0f3e00640 1 -- 172.21.15.86:0/3945663637 >> v1:172.21.15.86:6789/0 conn(0x7fa0fc0730b0 legacy=0x7fa0fc0734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:55.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.094+0000 7fa0fbe00640 1 -- 172.21.15.86:0/3945663637 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1233542282 0 0) 0x7fa0e0003530 con 0x7fa0fc0730b0 2024-04-25T14:08:55.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.094+0000 7fa0f3e00640 1 -- 172.21.15.86:0/3945663637 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0fc06fbd0 con 0x7fa0fc0d16e0 2024-04-25T14:08:55.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.094+0000 7fa0f3e00640 1 --2- 172.21.15.86:0/3945663637 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0fc0d16e0 0x7fa0fc0d1af0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc072da0 tx=0x7fa0e401c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=291893f874d39cc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:55.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.094+0000 7fa0fbe00640 1 -- 172.21.15.86:0/3945663637 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e40b1070 con 0x7fa0fc0d16e0 2024-04-25T14:08:55.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.094+0000 7fa0fbe00640 1 -- 172.21.15.86:0/3945663637 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0e40ac630 con 0x7fa0fc0d16e0 2024-04-25T14:08:55.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.094+0000 7fa0fbe00640 1 -- 172.21.15.86:0/3945663637 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e40a8050 con 0x7fa0fc0d16e0 2024-04-25T14:08:55.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.098+0000 7fa103400640 1 -- 172.21.15.86:0/3945663637 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0fc0d16e0 msgr2=0x7fa0fc0d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:55.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.098+0000 7fa103400640 1 --2- 172.21.15.86:0/3945663637 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0fc0d16e0 0x7fa0fc0d1af0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc072da0 tx=0x7fa0e401c4c0 comp rx=0 tx=0).stop 2024-04-25T14:08:55.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.098+0000 7fa103400640 1 -- 172.21.15.86:0/3945663637 shutdown_connections 2024-04-25T14:08:55.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.098+0000 7fa103400640 1 --2- 172.21.15.86:0/3945663637 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0fc0d16e0 0x7fa0fc0d1af0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:55.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.098+0000 7fa103400640 1 -- 172.21.15.86:0/3945663637 >> 172.21.15.86:0/3945663637 conn(0x7fa0fc0b4650 msgr2=0x7fa0fc0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:55.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.098+0000 7fa103400640 1 -- 172.21.15.86:0/3945663637 shutdown_connections 2024-04-25T14:08:55.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.098+0000 7fa103400640 1 -- 172.21.15.86:0/3945663637 wait complete. 2024-04-25T14:08:55.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.098+0000 7fa103400640 1 Processor -- start 2024-04-25T14:08:55.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.098+0000 7fa103400640 1 -- start start 2024-04-25T14:08:55.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.102+0000 7fa103400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0fc0730b0 0x7fa0fc0b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:55.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.102+0000 7fa103400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0fc0cfca0 0x7fa0fc09a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:55.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.102+0000 7fa103400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0fc0d16e0 0x7fa0fc0b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:55.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.102+0000 7fa103400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc16aa80 con 0x7fa0fc0730b0 2024-04-25T14:08:55.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.102+0000 7fa103400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc16a900 con 0x7fa0fc0d16e0 2024-04-25T14:08:55.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.102+0000 7fa103400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa0fc16ac00 con 0x7fa0fc0cfca0 2024-04-25T14:08:55.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.102+0000 7fa100a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0fc0730b0 0x7fa0fc0b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:55.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.102+0000 7fa100a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0fc0730b0 0x7fa0fc0b1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40364/0 (socket says 172.21.15.86:40364) 2024-04-25T14:08:55.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.102+0000 7fa100a00640 1 -- 172.21.15.86:0/1329134478 learned_addr learned my addr 172.21.15.86:0/1329134478 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:55.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.102+0000 7fa101400640 1 --2- 172.21.15.86:0/1329134478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0fc0d16e0 0x7fa0fc0b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:55.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.102+0000 7fa0f3e00640 1 --2- 172.21.15.86:0/1329134478 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0fc0cfca0 0x7fa0fc09a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:55.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.102+0000 7fa100a00640 1 -- 172.21.15.86:0/1329134478 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0fc0cfca0 msgr2=0x7fa0fc09a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:55.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.102+0000 7fa100a00640 1 --2- 172.21.15.86:0/1329134478 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0fc0cfca0 0x7fa0fc09a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:55.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.102+0000 7fa100a00640 1 -- 172.21.15.86:0/1329134478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0fc0d16e0 msgr2=0x7fa0fc0b0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:55.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.102+0000 7fa100a00640 1 --2- 172.21.15.86:0/1329134478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0fc0d16e0 0x7fa0fc0b0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:55.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.102+0000 7fa100a00640 1 -- 172.21.15.86:0/1329134478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0fc09ae70 con 0x7fa0fc0730b0 2024-04-25T14:08:55.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.102+0000 7fa100a00640 1 --2- 172.21.15.86:0/1329134478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0fc0730b0 0x7fa0fc0b1fb0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00c430 tx=0x7fa0ec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:55.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.102+0000 7fa0f9600640 1 -- 172.21.15.86:0/1329134478 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec017070 con 0x7fa0fc0730b0 2024-04-25T14:08:55.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.102+0000 7fa0f9600640 1 -- 172.21.15.86:0/1329134478 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0ec0052f0 con 0x7fa0fc0730b0 2024-04-25T14:08:55.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.102+0000 7fa103400640 1 -- 172.21.15.86:0/1329134478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0fc099bb0 con 0x7fa0fc0730b0 2024-04-25T14:08:55.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.102+0000 7fa0f9600640 1 -- 172.21.15.86:0/1329134478 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec003840 con 0x7fa0fc0730b0 2024-04-25T14:08:55.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.102+0000 7fa103400640 1 -- 172.21.15.86:0/1329134478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0fc099e50 con 0x7fa0fc0730b0 2024-04-25T14:08:55.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.102+0000 7fa103400640 1 -- 172.21.15.86:0/1329134478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0fc09f4b0 con 0x7fa0fc0730b0 2024-04-25T14:08:55.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.102+0000 7fa0f9600640 1 -- 172.21.15.86:0/1329134478 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7fa0ec01a070 con 0x7fa0fc0730b0 2024-04-25T14:08:55.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.106+0000 7fa0f9600640 1 --2- 172.21.15.86:0/1329134478 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fa0d806f520 0x7fa0d8071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:55.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.106+0000 7fa0f9600640 1 -- 172.21.15.86:0/1329134478 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7fa0ec08e9a0 con 0x7fa0fc0730b0 2024-04-25T14:08:55.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.106+0000 7fa0f3e00640 1 --2- 172.21.15.86:0/1329134478 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fa0d806f520 0x7fa0d8071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:55.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.106+0000 7fa0f3e00640 1 --2- 172.21.15.86:0/1329134478 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fa0d806f520 0x7fa0d8071a30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa0e40ae000 tx=0x7fa0e40a7000 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:55.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.110+0000 7fa0f9600640 1 -- 172.21.15.86:0/1329134478 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa0ec05af00 con 0x7fa0fc0730b0 2024-04-25T14:08:55.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.322+0000 7fa103400640 1 -- 172.21.15.86:0/1329134478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa0fc069d00 con 0x7fa0fc0730b0 2024-04-25T14:08:55.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.326+0000 7fa0f9600640 1 -- 172.21.15.86:0/1329134478 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v25) ==== 190+0+147 (secure 0 0 0) 0x7fa0ec05f200 con 0x7fa0fc0730b0 2024-04-25T14:08:55.327 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-04-25T14:08:55.328 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCLYipm5u4xHRAACDFe3Eb4KLrsdLj4eFe7mA== 2024-04-25T14:08:55.328 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-04-25T14:08:55.328 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-04-25T14:08:55.328 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T14:08:55.328 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.0 2024-04-25T14:08:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.326+0000 7fa103400640 1 -- 172.21.15.86:0/1329134478 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fa0d806f520 msgr2=0x7fa0d8071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.326+0000 7fa103400640 1 --2- 172.21.15.86:0/1329134478 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fa0d806f520 0x7fa0d8071a30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa0e40ae000 tx=0x7fa0e40a7000 comp rx=0 tx=0).stop 2024-04-25T14:08:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.326+0000 7fa103400640 1 -- 172.21.15.86:0/1329134478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0fc0730b0 msgr2=0x7fa0fc0b1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.326+0000 7fa103400640 1 --2- 172.21.15.86:0/1329134478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0fc0730b0 0x7fa0fc0b1fb0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00c430 tx=0x7fa0ec00c8f0 comp rx=0 tx=0).stop 2024-04-25T14:08:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.326+0000 7fa103400640 1 -- 172.21.15.86:0/1329134478 shutdown_connections 2024-04-25T14:08:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.326+0000 7fa103400640 1 --2- 172.21.15.86:0/1329134478 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fa0d806f520 0x7fa0d8071a30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.326+0000 7fa103400640 1 --2- 172.21.15.86:0/1329134478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0fc0d16e0 0x7fa0fc0b0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.326+0000 7fa103400640 1 --2- 172.21.15.86:0/1329134478 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0fc0cfca0 0x7fa0fc09a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.326+0000 7fa103400640 1 --2- 172.21.15.86:0/1329134478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0fc0730b0 0x7fa0fc0b1fb0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.326+0000 7fa103400640 1 -- 172.21.15.86:0/1329134478 >> 172.21.15.86:0/1329134478 conn(0x7fa0fc0b4650 msgr2=0x7fa0fc155c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.326+0000 7fa103400640 1 -- 172.21.15.86:0/1329134478 shutdown_connections 2024-04-25T14:08:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.326+0000 7fa103400640 1 -- 172.21.15.86:0/1329134478 wait complete. 2024-04-25T14:08:55.350 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:08:55.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.494+0000 7fe45dc00640 1 Processor -- start 2024-04-25T14:08:55.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.498+0000 7fe45dc00640 1 -- start start 2024-04-25T14:08:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.498+0000 7fe45dc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe45806fd50 0x7fe458070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.498+0000 7fe45dc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe45815feb0 0x7fe4581602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.498+0000 7fe45dc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4580b0a60 con 0x7fe45815c2a0 2024-04-25T14:08:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.498+0000 7fe45dc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe4580ae840 con 0x7fe45815feb0 2024-04-25T14:08:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.498+0000 7fe45dc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe45806ef90 con 0x7fe45806fd50 2024-04-25T14:08:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.498+0000 7fe456a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe45815c2a0 0x7fe4580ae130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49430/0 (socket says 172.21.15.86:49430) 2024-04-25T14:08:55.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.498+0000 7fe456a00640 1 -- 172.21.15.86:0/707331588 learned_addr learned my addr 172.21.15.86:0/707331588 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:55.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.498+0000 7fe457400640 1 --2- 172.21.15.86:0/707331588 >> v2:172.21.15.86:3300/0 conn(0x7fe45815feb0 0x7fe4581602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:55.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.498+0000 7fe456000640 1 --2- 172.21.15.86:0/707331588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe45806fd50 0x7fe458070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:55.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.498+0000 7fe457400640 1 -- 172.21.15.86:0/707331588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe45806fd50 msgr2=0x7fe458070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:55.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.498+0000 7fe457400640 1 --2- 172.21.15.86:0/707331588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe45806fd50 0x7fe458070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:55.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.498+0000 7fe457400640 1 -- 172.21.15.86:0/707331588 >> v1:172.21.15.86:6789/0 conn(0x7fe45815c2a0 legacy=0x7fe4580ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:55.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.498+0000 7fe457400640 1 -- 172.21.15.86:0/707331588 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45806f270 con 0x7fe45815feb0 2024-04-25T14:08:55.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.498+0000 7fe455600640 1 -- 172.21.15.86:0/707331588 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 483333740 0 0) 0x7fe4580b0a60 con 0x7fe45815c2a0 2024-04-25T14:08:55.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.498+0000 7fe456000640 1 --2- 172.21.15.86:0/707331588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe45806fd50 0x7fe458070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.502+0000 7fe457400640 1 --2- 172.21.15.86:0/707331588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe45815feb0 0x7fe4581602c0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fe450009d90 tx=0x7fe45001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c5e476e5206a8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.502+0000 7fe455600640 1 -- 172.21.15.86:0/707331588 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4500b2070 con 0x7fe45815feb0 2024-04-25T14:08:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.502+0000 7fe455600640 1 -- 172.21.15.86:0/707331588 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4500a5970 con 0x7fe45815feb0 2024-04-25T14:08:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.502+0000 7fe455600640 1 -- 172.21.15.86:0/707331588 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4500a5d40 con 0x7fe45815feb0 2024-04-25T14:08:55.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.502+0000 7fe45dc00640 1 -- 172.21.15.86:0/707331588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe45815feb0 msgr2=0x7fe4581602c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:55.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.502+0000 7fe45dc00640 1 --2- 172.21.15.86:0/707331588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe45815feb0 0x7fe4581602c0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fe450009d90 tx=0x7fe45001c8b0 comp rx=0 tx=0).stop 2024-04-25T14:08:55.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.502+0000 7fe45dc00640 1 -- 172.21.15.86:0/707331588 shutdown_connections 2024-04-25T14:08:55.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.502+0000 7fe45dc00640 1 --2- 172.21.15.86:0/707331588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe45815feb0 0x7fe4581602c0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:55.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.502+0000 7fe45dc00640 1 --2- 172.21.15.86:0/707331588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe45806fd50 0x7fe458070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:55.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.502+0000 7fe45dc00640 1 -- 172.21.15.86:0/707331588 >> 172.21.15.86:0/707331588 conn(0x7fe4580b4650 msgr2=0x7fe4580b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:55.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.502+0000 7fe45dc00640 1 -- 172.21.15.86:0/707331588 shutdown_connections 2024-04-25T14:08:55.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.502+0000 7fe45dc00640 1 -- 172.21.15.86:0/707331588 wait complete. 2024-04-25T14:08:55.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.506+0000 7fe45dc00640 1 Processor -- start 2024-04-25T14:08:55.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.506+0000 7fe45dc00640 1 -- start start 2024-04-25T14:08:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.506+0000 7fe45dc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe45806fd50 0x7fe45815e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.506+0000 7fe45dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe45815c2a0 0x7fe4580c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.506+0000 7fe45dc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe45815feb0 0x7fe4580b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.506+0000 7fe45dc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe45815c6b0 con 0x7fe45815c2a0 2024-04-25T14:08:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.506+0000 7fe45dc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe4580d0fc0 con 0x7fe45806fd50 2024-04-25T14:08:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.506+0000 7fe45dc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe45815c830 con 0x7fe45815feb0 2024-04-25T14:08:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.506+0000 7fe456000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe45815c2a0 0x7fe4580c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.506+0000 7fe456000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe45815c2a0 0x7fe4580c8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40386/0 (socket says 172.21.15.86:40386) 2024-04-25T14:08:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.506+0000 7fe456000640 1 -- 172.21.15.86:0/4114219112 learned_addr learned my addr 172.21.15.86:0/4114219112 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.506+0000 7fe456a00640 1 --2- 172.21.15.86:0/4114219112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe45806fd50 0x7fe45815e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.506+0000 7fe457400640 1 --2- 172.21.15.86:0/4114219112 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe45815feb0 0x7fe4580b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.506+0000 7fe456000640 1 -- 172.21.15.86:0/4114219112 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe45815feb0 msgr2=0x7fe4580b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.506+0000 7fe456000640 1 --2- 172.21.15.86:0/4114219112 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe45815feb0 0x7fe4580b0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:55.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.506+0000 7fe456000640 1 -- 172.21.15.86:0/4114219112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe45806fd50 msgr2=0x7fe45815e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:55.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.506+0000 7fe456000640 1 --2- 172.21.15.86:0/4114219112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe45806fd50 0x7fe45815e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:55.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.506+0000 7fe456000640 1 -- 172.21.15.86:0/4114219112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4580c9390 con 0x7fe45815c2a0 2024-04-25T14:08:55.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.506+0000 7fe456000640 1 --2- 172.21.15.86:0/4114219112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe45815c2a0 0x7fe4580c8e50 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fe44400d800 tx=0x7fe44400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:55.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.506+0000 7fe44ea00640 1 -- 172.21.15.86:0/4114219112 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe444018020 con 0x7fe45815c2a0 2024-04-25T14:08:55.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.506+0000 7fe44ea00640 1 -- 172.21.15.86:0/4114219112 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4440096c0 con 0x7fe45815c2a0 2024-04-25T14:08:55.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.506+0000 7fe45dc00640 1 -- 172.21.15.86:0/4114219112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe45815ec40 con 0x7fe45815c2a0 2024-04-25T14:08:55.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.506+0000 7fe44ea00640 1 -- 172.21.15.86:0/4114219112 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe444009ab0 con 0x7fe45815c2a0 2024-04-25T14:08:55.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.506+0000 7fe45dc00640 1 -- 172.21.15.86:0/4114219112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe45815f200 con 0x7fe45815c2a0 2024-04-25T14:08:55.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.510+0000 7fe45dc00640 1 -- 172.21.15.86:0/4114219112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe45815e970 con 0x7fe45815c2a0 2024-04-25T14:08:55.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.510+0000 7fe44ea00640 1 -- 172.21.15.86:0/4114219112 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7fe44401b020 con 0x7fe45815c2a0 2024-04-25T14:08:55.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.510+0000 7fe44ea00640 1 --2- 172.21.15.86:0/4114219112 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fe43806f520 0x7fe438071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:55.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.510+0000 7fe44ea00640 1 -- 172.21.15.86:0/4114219112 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7fe44408fb90 con 0x7fe45815c2a0 2024-04-25T14:08:55.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.510+0000 7fe456a00640 1 --2- 172.21.15.86:0/4114219112 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fe43806f520 0x7fe438071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:55.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.514+0000 7fe456a00640 1 --2- 172.21.15.86:0/4114219112 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fe43806f520 0x7fe438071a30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe448000f70 tx=0x7fe448007c10 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:55.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.514+0000 7fe44ea00640 1 -- 172.21.15.86:0/4114219112 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe444022020 con 0x7fe45815c2a0 2024-04-25T14:08:55.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.702+0000 7fe45dc00640 1 -- 172.21.15.86:0/4114219112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe458069d00 con 0x7fe45815c2a0 2024-04-25T14:08:55.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.702+0000 7fe44ea00640 1 -- 172.21.15.86:0/4114219112 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5186 (secure 0 0 0) 0x7fe44405c7d0 con 0x7fe45815c2a0 2024-04-25T14:08:55.706 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:08:55.706 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7186,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/4265929298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4265929298},{"type":"v1","addr":"172.21.15.155:6833","nonce":4265929298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7189,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3254219000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3254219000},{"type":"v1","addr":"172.21.15.155:6835","nonce":3254219000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7195,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/193454084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":193454084},{"type":"v1","addr":"172.21.15.86:6837","nonce":193454084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:08:45.096500+0000","modified":"2024-04-25T14:08:48.121611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7371},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7371":{"gid":7371,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/1961103704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1961103704},{"type":"v1","addr":"172.21.15.86:6835","nonce":1961103704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7371,"qdb_cluster":[7371]},"id":6}]} 2024-04-25T14:08:55.706 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 67 2024-04-25T14:08:55.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.706+0000 7fe45dc00640 1 -- 172.21.15.86:0/4114219112 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fe43806f520 msgr2=0x7fe438071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:55.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.706+0000 7fe45dc00640 1 --2- 172.21.15.86:0/4114219112 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fe43806f520 0x7fe438071a30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe448000f70 tx=0x7fe448007c10 comp rx=0 tx=0).stop 2024-04-25T14:08:55.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.706+0000 7fe45dc00640 1 -- 172.21.15.86:0/4114219112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe45815c2a0 msgr2=0x7fe4580c8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:55.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.706+0000 7fe45dc00640 1 --2- 172.21.15.86:0/4114219112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe45815c2a0 0x7fe4580c8e50 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fe44400d800 tx=0x7fe44400dcc0 comp rx=0 tx=0).stop 2024-04-25T14:08:55.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.706+0000 7fe45dc00640 1 -- 172.21.15.86:0/4114219112 shutdown_connections 2024-04-25T14:08:55.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.706+0000 7fe45dc00640 1 --2- 172.21.15.86:0/4114219112 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fe43806f520 0x7fe438071a30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:55.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.706+0000 7fe45dc00640 1 --2- 172.21.15.86:0/4114219112 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe45815feb0 0x7fe4580b0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:55.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.706+0000 7fe45dc00640 1 --2- 172.21.15.86:0/4114219112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe45815c2a0 0x7fe4580c8e50 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:55.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.706+0000 7fe45dc00640 1 --2- 172.21.15.86:0/4114219112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe45806fd50 0x7fe45815e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:55.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.706+0000 7fe45dc00640 1 -- 172.21.15.86:0/4114219112 >> 172.21.15.86:0/4114219112 conn(0x7fe4580b4650 msgr2=0x7fe458166aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:55.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.706+0000 7fe45dc00640 1 -- 172.21.15.86:0/4114219112 shutdown_connections 2024-04-25T14:08:55.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:55.706+0000 7fe45dc00640 1 -- 172.21.15.86:0/4114219112 wait complete. 2024-04-25T14:08:56.130 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:08:45.096500+0000', 'modified': '2024-04-25T14:08:48.121611+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7371}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7371': {'gid': 7371, 'name': 'a', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6835/1961103704', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6834', 'nonce': 1961103704}, {'type': 'v1', 'addr': '172.21.15.86:6835', 'nonce': 1961103704}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7371, 'qdb_cluster': [7371]} 2024-04-25T14:08:56.130 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T14:08:56.130 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T14:08:56.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.274+0000 7fca69a00640 1 Processor -- start 2024-04-25T14:08:56.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.274+0000 7fca69a00640 1 -- start start 2024-04-25T14:08:56.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.278+0000 7fca69a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fca640d16e0 0x7fca640d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.278+0000 7fca69a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca640730b0 0x7fca640734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.278+0000 7fca69a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca640d2030 con 0x7fca640cfca0 2024-04-25T14:08:56.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.278+0000 7fca69a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fca64070470 con 0x7fca640d16e0 2024-04-25T14:08:56.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.278+0000 7fca69a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fca64072bd0 con 0x7fca640730b0 2024-04-25T14:08:56.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.278+0000 7fca62a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fca640cfca0 0x7fca6406f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49434/0 (socket says 172.21.15.86:49434) 2024-04-25T14:08:56.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.278+0000 7fca62a00640 1 -- 172.21.15.86:0/2117819943 learned_addr learned my addr 172.21.15.86:0/2117819943 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:56.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.278+0000 7fca5a000640 1 --2- 172.21.15.86:0/2117819943 >> v2:172.21.15.86:3300/0 conn(0x7fca640d16e0 0x7fca640d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.278+0000 7fca63400640 1 --2- 172.21.15.86:0/2117819943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca640730b0 0x7fca640734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.278+0000 7fca62000640 1 -- 172.21.15.86:0/2117819943 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 373440359 0 0) 0x7fca54001120 con 0x7fca640cfca0 2024-04-25T14:08:56.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.278+0000 7fca62000640 1 -- 172.21.15.86:0/2117819943 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca48005740 con 0x7fca640cfca0 2024-04-25T14:08:56.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.278+0000 7fca5a000640 1 -- 172.21.15.86:0/2117819943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca640730b0 msgr2=0x7fca640734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:56.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.278+0000 7fca5a000640 1 --2- 172.21.15.86:0/2117819943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca640730b0 0x7fca640734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.278+0000 7fca5a000640 1 -- 172.21.15.86:0/2117819943 >> v1:172.21.15.86:6789/0 conn(0x7fca640cfca0 legacy=0x7fca6406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:56.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.278+0000 7fca5a000640 1 -- 172.21.15.86:0/2117819943 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6406fbd0 con 0x7fca640d16e0 2024-04-25T14:08:56.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.282+0000 7fca5a000640 1 --2- 172.21.15.86:0/2117819943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca640d16e0 0x7fca640d1af0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fca4c009d90 tx=0x7fca4c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f0d149e2b5572d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:56.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.282+0000 7fca62000640 1 -- 172.21.15.86:0/2117819943 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c0b2070 con 0x7fca640d16e0 2024-04-25T14:08:56.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.282+0000 7fca62000640 1 -- 172.21.15.86:0/2117819943 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca4c0a5940 con 0x7fca640d16e0 2024-04-25T14:08:56.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.282+0000 7fca62000640 1 -- 172.21.15.86:0/2117819943 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c0a5d10 con 0x7fca640d16e0 2024-04-25T14:08:56.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.282+0000 7fca69a00640 1 -- 172.21.15.86:0/2117819943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca640d16e0 msgr2=0x7fca640d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:56.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.282+0000 7fca69a00640 1 --2- 172.21.15.86:0/2117819943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca640d16e0 0x7fca640d1af0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fca4c009d90 tx=0x7fca4c01c970 comp rx=0 tx=0).stop 2024-04-25T14:08:56.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.282+0000 7fca69a00640 1 -- 172.21.15.86:0/2117819943 shutdown_connections 2024-04-25T14:08:56.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.282+0000 7fca69a00640 1 --2- 172.21.15.86:0/2117819943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca640730b0 0x7fca640734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.282+0000 7fca69a00640 1 --2- 172.21.15.86:0/2117819943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca640d16e0 0x7fca640d1af0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.282+0000 7fca69a00640 1 -- 172.21.15.86:0/2117819943 >> 172.21.15.86:0/2117819943 conn(0x7fca640b4650 msgr2=0x7fca640b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:56.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.282+0000 7fca69a00640 1 -- 172.21.15.86:0/2117819943 shutdown_connections 2024-04-25T14:08:56.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.282+0000 7fca69a00640 1 -- 172.21.15.86:0/2117819943 wait complete. 2024-04-25T14:08:56.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.286+0000 7fca69a00640 1 Processor -- start 2024-04-25T14:08:56.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.286+0000 7fca69a00640 1 -- start start 2024-04-25T14:08:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.286+0000 7fca69a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca640730b0 0x7fca6415f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.286+0000 7fca69a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca640cfca0 0x7fca6414df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.286+0000 7fca69a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca640d16e0 0x7fca6409caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.286+0000 7fca69a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca640ce170 con 0x7fca640d16e0 2024-04-25T14:08:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.286+0000 7fca69a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fca640cdff0 con 0x7fca640cfca0 2024-04-25T14:08:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.286+0000 7fca69a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fca640ce2f0 con 0x7fca640730b0 2024-04-25T14:08:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.286+0000 7fca63400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca640d16e0 0x7fca6409caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.286+0000 7fca63400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca640d16e0 0x7fca6409caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40408/0 (socket says 172.21.15.86:40408) 2024-04-25T14:08:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.286+0000 7fca63400640 1 -- 172.21.15.86:0/3872143649 learned_addr learned my addr 172.21.15.86:0/3872143649 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.286+0000 7fca63400640 1 -- 172.21.15.86:0/3872143649 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca640730b0 msgr2=0x7fca6415f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.286+0000 7fca63400640 1 --2- 172.21.15.86:0/3872143649 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca640730b0 0x7fca6415f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.286+0000 7fca63400640 1 -- 172.21.15.86:0/3872143649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca640cfca0 msgr2=0x7fca6414df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:56.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.286+0000 7fca63400640 1 --2- 172.21.15.86:0/3872143649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca640cfca0 0x7fca6414df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.286+0000 7fca63400640 1 -- 172.21.15.86:0/3872143649 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6415de90 con 0x7fca640d16e0 2024-04-25T14:08:56.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.286+0000 7fca63400640 1 --2- 172.21.15.86:0/3872143649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca640d16e0 0x7fca6409caf0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00e7f0 tx=0x7fca5c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:56.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.286+0000 7fca5b400640 1 -- 172.21.15.86:0/3872143649 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c017070 con 0x7fca640d16e0 2024-04-25T14:08:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.286+0000 7fca5b400640 1 -- 172.21.15.86:0/3872143649 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca5c00c940 con 0x7fca640d16e0 2024-04-25T14:08:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.286+0000 7fca5b400640 1 -- 172.21.15.86:0/3872143649 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c00eea0 con 0x7fca640d16e0 2024-04-25T14:08:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.286+0000 7fca69a00640 1 -- 172.21.15.86:0/3872143649 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca6415e170 con 0x7fca640d16e0 2024-04-25T14:08:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.286+0000 7fca69a00640 1 -- 172.21.15.86:0/3872143649 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca6415e730 con 0x7fca640d16e0 2024-04-25T14:08:56.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.290+0000 7fca5b400640 1 -- 172.21.15.86:0/3872143649 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7fca5c01a070 con 0x7fca640d16e0 2024-04-25T14:08:56.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.290+0000 7fca5b400640 1 --2- 172.21.15.86:0/3872143649 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fca4006f520 0x7fca40071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.290+0000 7fca5b400640 1 -- 172.21.15.86:0/3872143649 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7fca5c08f760 con 0x7fca640d16e0 2024-04-25T14:08:56.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca62a00640 1 --2- 172.21.15.86:0/3872143649 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fca4006f520 0x7fca40071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca71039000 1 Processor -- start 2024-04-25T14:08:56.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca71039000 1 -- start start 2024-04-25T14:08:56.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca62a00640 1 --2- 172.21.15.86:0/3872143649 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fca4006f520 0x7fca40071a30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fca54000f70 tx=0x7fca54001120 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:56.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca71039000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593e2a2dc20 0x5593e2b8bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca71039000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5593e2a19ea0 0x5593e2b8c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca71039000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593e2612280 0x5593e2b8c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca71039000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5593e29fb160 con 0x5593e2a2dc20 2024-04-25T14:08:56.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca71039000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5593e2990e20 con 0x5593e2612280 2024-04-25T14:08:56.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca71039000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5593e2949a20 con 0x5593e2a19ea0 2024-04-25T14:08:56.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca62a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593e2a2dc20 0x5593e2b8bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca62a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593e2a2dc20 0x5593e2b8bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40410/0 (socket says 172.21.15.86:40410) 2024-04-25T14:08:56.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca62a00640 1 -- 172.21.15.86:0/2058590262 learned_addr learned my addr 172.21.15.86:0/2058590262 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:56.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca63400640 1 --2- 172.21.15.86:0/2058590262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593e2612280 0x5593e2b8c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca5a000640 1 --2- 172.21.15.86:0/2058590262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5593e2a19ea0 0x5593e2b8c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca62a00640 1 -- 172.21.15.86:0/2058590262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5593e2a19ea0 msgr2=0x5593e2b8c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:56.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca62a00640 1 --2- 172.21.15.86:0/2058590262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5593e2a19ea0 0x5593e2b8c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca62a00640 1 -- 172.21.15.86:0/2058590262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593e2612280 msgr2=0x5593e2b8c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:56.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca62a00640 1 --2- 172.21.15.86:0/2058590262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593e2612280 0x5593e2b8c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca62a00640 1 -- 172.21.15.86:0/2058590262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593e294e100 con 0x5593e2a2dc20 2024-04-25T14:08:56.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca62a00640 1 --2- 172.21.15.86:0/2058590262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593e2a2dc20 0x5593e2b8bee0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fca54005d00 tx=0x7fca5400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f3ce6c251f0050 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:56.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca53400640 1 -- 172.21.15.86:0/2058590262 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca54016110 con 0x5593e2a2dc20 2024-04-25T14:08:56.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca53400640 1 -- 172.21.15.86:0/2058590262 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca54007560 con 0x5593e2a2dc20 2024-04-25T14:08:56.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca53400640 1 -- 172.21.15.86:0/2058590262 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca540162b0 con 0x5593e2a2dc20 2024-04-25T14:08:56.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca71039000 1 -- 172.21.15.86:0/2058590262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593e2a2dc20 msgr2=0x5593e2b8bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:56.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca71039000 1 --2- 172.21.15.86:0/2058590262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593e2a2dc20 0x5593e2b8bee0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fca54005d00 tx=0x7fca5400f000 comp rx=0 tx=0).stop 2024-04-25T14:08:56.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca71039000 1 -- 172.21.15.86:0/2058590262 shutdown_connections 2024-04-25T14:08:56.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca71039000 1 --2- 172.21.15.86:0/2058590262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593e2612280 0x5593e2b8c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca71039000 1 --2- 172.21.15.86:0/2058590262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5593e2a19ea0 0x5593e2b8c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca71039000 1 --2- 172.21.15.86:0/2058590262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593e2a2dc20 0x5593e2b8bee0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca71039000 1 -- 172.21.15.86:0/2058590262 >> 172.21.15.86:0/2058590262 conn(0x5593e2651ff0 msgr2=0x5593e2b414c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:56.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.294+0000 7fca71039000 1 -- 172.21.15.86:0/2058590262 shutdown_connections 2024-04-25T14:08:56.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.298+0000 7fca71039000 1 -- 172.21.15.86:0/2058590262 wait complete. 2024-04-25T14:08:56.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.298+0000 7fca71039000 1 -- start start 2024-04-25T14:08:56.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.298+0000 7fca71039000 1 Processor -- start 2024-04-25T14:08:56.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.298+0000 7fca71039000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593e2612280 0x5593e2b92e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.298+0000 7fca71039000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593e2a19ea0 0x5593e2b933a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.298+0000 7fca71039000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5593e2a2dc20 0x5593e2b4e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.298+0000 7fca71039000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5593e299e220 con 0x5593e2612280 2024-04-25T14:08:56.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.298+0000 7fca71039000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5593e2934ea0 con 0x5593e2a19ea0 2024-04-25T14:08:56.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.298+0000 7fca71039000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5593e298bb00 con 0x5593e2a2dc20 2024-04-25T14:08:56.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.298+0000 7fca71039000 10 client.0 _ll_register_callbacks cb 0x5593e299ddc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:08:56.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.298+0000 7fca62a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593e2612280 0x5593e2b92e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.298+0000 7fca62a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593e2612280 0x5593e2b92e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40414/0 (socket says 172.21.15.86:40414) 2024-04-25T14:08:56.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.298+0000 7fca62a00640 1 -- 172.21.15.86:0/1683195887 learned_addr learned my addr 172.21.15.86:0/1683195887 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:56.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.298+0000 7fca5a000640 1 --2- 172.21.15.86:0/1683195887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593e2a19ea0 0x5593e2b933a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.298+0000 7fca63400640 1 --2- 172.21.15.86:0/1683195887 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5593e2a2dc20 0x5593e2b4e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.298+0000 7fca62a00640 1 -- 172.21.15.86:0/1683195887 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5593e2a2dc20 msgr2=0x5593e2b4e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:56.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.298+0000 7fca62a00640 1 --2- 172.21.15.86:0/1683195887 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5593e2a2dc20 0x5593e2b4e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.298+0000 7fca62a00640 1 -- 172.21.15.86:0/1683195887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593e2a19ea0 msgr2=0x5593e2b933a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:56.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.298+0000 7fca62a00640 1 --2- 172.21.15.86:0/1683195887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593e2a19ea0 0x5593e2b933a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.298+0000 7fca62a00640 1 -- 172.21.15.86:0/1683195887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593e2930500 con 0x5593e2612280 2024-04-25T14:08:56.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.298+0000 7fca5a000640 1 --2- 172.21.15.86:0/1683195887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593e2a19ea0 0x5593e2b933a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:56.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.298+0000 7fca62a00640 1 --2- 172.21.15.86:0/1683195887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593e2612280 0x5593e2b92e60 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x5593e29cff70 tx=0x7fca54001330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c88942be37c013 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:56.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.298+0000 7fca47e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:08:56.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.298+0000 7fca47e00640 1 -- 172.21.15.86:0/1683195887 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca54007710 con 0x5593e2612280 2024-04-25T14:08:56.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.298+0000 7fca71039000 1 -- 172.21.15.86:0/1683195887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca54007560 con 0x5593e2612280 2024-04-25T14:08:56.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.298+0000 7fca47e00640 1 -- 172.21.15.86:0/1683195887 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7fca5402bc80 con 0x5593e2612280 2024-04-25T14:08:56.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca47e00640 1 -- 172.21.15.86:0/1683195887 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fca540047f0 con 0x5593e2612280 2024-04-25T14:08:56.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca71039000 10 client.7548 fetch_fsmap learned FSMap version 67 2024-04-25T14:08:56.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca71039000 1 -- 172.21.15.86:0/1683195887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x5593e2916ea0 con 0x5593e2612280 2024-04-25T14:08:56.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca47e00640 1 -- 172.21.15.86:0/1683195887 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca54007560 con 0x5593e2612280 2024-04-25T14:08:56.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca47e00640 1 -- 172.21.15.86:0/1683195887 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7fca54028070 con 0x5593e2612280 2024-04-25T14:08:56.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca71039000 10 client.7548 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T14:08:56.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca71039000 10 client.7548 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T14:08:56.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca71039000 20 client.7548 populate_metadata read hostname 'smithi086' 2024-04-25T14:08:56.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca71039000 1 --2- 172.21.15.86:0/1683195887 >> [v2:172.21.15.86:6834/1961103704,v1:172.21.15.86:6835/1961103704] conn(0x5593e26367d0 0x5593e2b4ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca71039000 4 client.7548 mds_command: new command op to 7371 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:08:56.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca71039000 1 -- 172.21.15.86:0/1683195887 --> [v2:172.21.15.86:6834/1961103704,v1:172.21.15.86:6835/1961103704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593e2929be0 con 0x5593e26367d0 2024-04-25T14:08:56.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca5a000640 1 --2- 172.21.15.86:0/1683195887 >> [v2:172.21.15.86:6834/1961103704,v1:172.21.15.86:6835/1961103704] conn(0x5593e26367d0 0x5593e2b4ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca5a000640 1 --2- 172.21.15.86:0/1683195887 >> [v2:172.21.15.86:6834/1961103704,v1:172.21.15.86:6835/1961103704] conn(0x5593e26367d0 0x5593e2b4ee00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9110acdb7748aa4b server_cookie=1d82b94aec9b509a in_seq=0 out_seq=0 2024-04-25T14:08:56.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca47e00640 10 client.7548 ms_handle_connect on v2:172.21.15.86:6834/1961103704 2024-04-25T14:08:56.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca47e00640 1 -- 172.21.15.86:0/1683195887 <== mds.0 v2:172.21.15.86:6834/1961103704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5593e2929be0 con 0x5593e26367d0 2024-04-25T14:08:56.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca47e00640 10 client.7548 handle_command_reply: tid=0 2024-04-25T14:08:56.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca71039000 1 client.7548 shutdown 2024-04-25T14:08:56.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca71039000 1 -- 172.21.15.86:0/1683195887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593e2612280 msgr2=0x5593e2b92e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:56.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca71039000 1 --2- 172.21.15.86:0/1683195887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593e2612280 0x5593e2b92e60 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x5593e29cff70 tx=0x7fca54001330 comp rx=0 tx=0).stop 2024-04-25T14:08:56.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca71039000 1 -- 172.21.15.86:0/1683195887 shutdown_connections 2024-04-25T14:08:56.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca71039000 1 --2- 172.21.15.86:0/1683195887 >> [v2:172.21.15.86:6834/1961103704,v1:172.21.15.86:6835/1961103704] conn(0x5593e26367d0 0x5593e2b4ee00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca71039000 1 --2- 172.21.15.86:0/1683195887 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5593e2a2dc20 0x5593e2b4e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca71039000 1 --2- 172.21.15.86:0/1683195887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593e2a19ea0 0x5593e2b933a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca47e00640 0 client.7548 ms_handle_reset on v2:172.21.15.86:6834/1961103704 2024-04-25T14:08:56.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca71039000 1 --2- 172.21.15.86:0/1683195887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593e2612280 0x5593e2b92e60 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca71039000 1 -- 172.21.15.86:0/1683195887 >> 172.21.15.86:0/1683195887 conn(0x5593e2651ff0 msgr2=0x5593e2b41c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:56.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca71039000 1 -- 172.21.15.86:0/1683195887 shutdown_connections 2024-04-25T14:08:56.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca71039000 1 -- 172.21.15.86:0/1683195887 wait complete. 2024-04-25T14:08:56.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.302+0000 7fca71039000 20 client.7548 trim_cache size 0 max 16384 2024-04-25T14:08:56.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.322+0000 7fca71039000 1 Processor -- start 2024-04-25T14:08:56.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.322+0000 7fca71039000 1 -- start start 2024-04-25T14:08:56.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.322+0000 7fca71039000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593e2612280 0x5593e2b97ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.322+0000 7fca71039000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5593e2a19ea0 0x5593e2b983e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.322+0000 7fca71039000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593e2a2dc20 0x5593e2ba1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.322+0000 7fca71039000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca642057b0 con 0x5593e2a2dc20 2024-04-25T14:08:56.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.322+0000 7fca71039000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fca640ca4f0 con 0x5593e2612280 2024-04-25T14:08:56.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.322+0000 7fca71039000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fca640cab40 con 0x5593e2a19ea0 2024-04-25T14:08:56.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.322+0000 7fca63400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593e2a2dc20 0x5593e2ba1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.322+0000 7fca63400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593e2a2dc20 0x5593e2ba1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40430/0 (socket says 172.21.15.86:40430) 2024-04-25T14:08:56.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.322+0000 7fca63400640 1 -- 172.21.15.86:0/2153451724 learned_addr learned my addr 172.21.15.86:0/2153451724 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:56.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.322+0000 7fca5a000640 1 --2- 172.21.15.86:0/2153451724 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5593e2a19ea0 0x5593e2b983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.322+0000 7fca62a00640 1 --2- 172.21.15.86:0/2153451724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593e2612280 0x5593e2b97ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.322+0000 7fca63400640 1 -- 172.21.15.86:0/2153451724 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5593e2a19ea0 msgr2=0x5593e2b983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:56.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.322+0000 7fca63400640 1 --2- 172.21.15.86:0/2153451724 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5593e2a19ea0 0x5593e2b983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.322+0000 7fca63400640 1 -- 172.21.15.86:0/2153451724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593e2612280 msgr2=0x5593e2b97ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:56.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.322+0000 7fca63400640 1 --2- 172.21.15.86:0/2153451724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593e2612280 0x5593e2b97ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.322+0000 7fca63400640 1 -- 172.21.15.86:0/2153451724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593e2b45840 con 0x5593e2a2dc20 2024-04-25T14:08:56.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.322+0000 7fca62a00640 1 --2- 172.21.15.86:0/2153451724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593e2612280 0x5593e2b97ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:56.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.322+0000 7fca5a000640 1 --2- 172.21.15.86:0/2153451724 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5593e2a19ea0 0x5593e2b983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:56.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.322+0000 7fca63400640 1 --2- 172.21.15.86:0/2153451724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593e2a2dc20 0x5593e2ba1ff0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fca5c057dc0 tx=0x7fca5c05b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7a65cc3d7c3333 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:56.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.322+0000 7fca52a00640 1 -- 172.21.15.86:0/2153451724 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c08b070 con 0x5593e2a2dc20 2024-04-25T14:08:56.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.322+0000 7fca52a00640 1 -- 172.21.15.86:0/2153451724 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca5c0600f0 con 0x5593e2a2dc20 2024-04-25T14:08:56.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.322+0000 7fca52a00640 1 -- 172.21.15.86:0/2153451724 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c073070 con 0x5593e2a2dc20 2024-04-25T14:08:56.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.326+0000 7fca71039000 1 -- 172.21.15.86:0/2153451724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593e2a2dc20 msgr2=0x5593e2ba1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:56.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.326+0000 7fca71039000 1 --2- 172.21.15.86:0/2153451724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593e2a2dc20 0x5593e2ba1ff0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fca5c057dc0 tx=0x7fca5c05b5b0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.326+0000 7fca71039000 1 -- 172.21.15.86:0/2153451724 shutdown_connections 2024-04-25T14:08:56.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.326+0000 7fca71039000 1 --2- 172.21.15.86:0/2153451724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593e2a2dc20 0x5593e2ba1ff0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.326+0000 7fca71039000 1 --2- 172.21.15.86:0/2153451724 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5593e2a19ea0 0x5593e2b983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.326+0000 7fca71039000 1 --2- 172.21.15.86:0/2153451724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593e2612280 0x5593e2b97ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.326+0000 7fca71039000 1 -- 172.21.15.86:0/2153451724 >> 172.21.15.86:0/2153451724 conn(0x5593e2651ff0 msgr2=0x5593e2b49040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:56.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.326+0000 7fca71039000 1 -- 172.21.15.86:0/2153451724 shutdown_connections 2024-04-25T14:08:56.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.326+0000 7fca71039000 1 -- 172.21.15.86:0/2153451724 wait complete. 2024-04-25T14:08:56.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.326+0000 7fca71039000 1 -- start start 2024-04-25T14:08:56.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.326+0000 7fca71039000 1 Processor -- start 2024-04-25T14:08:56.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.326+0000 7fca71039000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593e2612280 0x5593e2b967c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.326+0000 7fca71039000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5593e2a19ea0 0x5593e2b96d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.326+0000 7fca71039000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593e2a2dc20 0x5593e2b97240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.326+0000 7fca71039000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5593e2990130 con 0x5593e2a2dc20 2024-04-25T14:08:56.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.326+0000 7fca71039000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5593e29267d0 con 0x5593e2612280 2024-04-25T14:08:56.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.326+0000 7fca71039000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5593e2995450 con 0x5593e2a19ea0 2024-04-25T14:08:56.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.326+0000 7fca71039000 10 client.0 _ll_register_callbacks cb 0x7fca54005c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:08:56.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.326+0000 7fca63400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593e2a2dc20 0x5593e2b97240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.326+0000 7fca63400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593e2a2dc20 0x5593e2b97240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40446/0 (socket says 172.21.15.86:40446) 2024-04-25T14:08:56.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.326+0000 7fca63400640 1 -- 172.21.15.86:0/881305683 learned_addr learned my addr 172.21.15.86:0/881305683 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:56.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.326+0000 7fca5a000640 1 --2- 172.21.15.86:0/881305683 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5593e2a19ea0 0x5593e2b96d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.326+0000 7fca62a00640 1 --2- 172.21.15.86:0/881305683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593e2612280 0x5593e2b967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca63400640 1 -- 172.21.15.86:0/881305683 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5593e2a19ea0 msgr2=0x5593e2b96d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:56.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca63400640 1 --2- 172.21.15.86:0/881305683 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5593e2a19ea0 0x5593e2b96d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca63400640 1 -- 172.21.15.86:0/881305683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593e2612280 msgr2=0x5593e2b967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:56.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca63400640 1 --2- 172.21.15.86:0/881305683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593e2612280 0x5593e2b967c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca63400640 1 -- 172.21.15.86:0/881305683 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593e2b4e430 con 0x5593e2a2dc20 2024-04-25T14:08:56.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca63400640 1 --2- 172.21.15.86:0/881305683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593e2a2dc20 0x5593e2b97240 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fca5c075410 tx=0x7fca5c057d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ff3ec998e9ed05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:56.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca47e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:08:56.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca47e00640 1 -- 172.21.15.86:0/881305683 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c08b040 con 0x5593e2a2dc20 2024-04-25T14:08:56.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca47e00640 1 -- 172.21.15.86:0/881305683 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7fca5c086d90 con 0x5593e2a2dc20 2024-04-25T14:08:56.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca71039000 1 -- 172.21.15.86:0/881305683 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca5c0600f0 con 0x5593e2a2dc20 2024-04-25T14:08:56.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca47e00640 1 -- 172.21.15.86:0/881305683 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fca5c021040 con 0x5593e2a2dc20 2024-04-25T14:08:56.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca71039000 10 client.7554 fetch_fsmap learned FSMap version 67 2024-04-25T14:08:56.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca71039000 1 -- 172.21.15.86:0/881305683 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x5593e2949c70 con 0x5593e2a2dc20 2024-04-25T14:08:56.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca47e00640 1 -- 172.21.15.86:0/881305683 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca5c0600f0 con 0x5593e2a2dc20 2024-04-25T14:08:56.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca47e00640 1 -- 172.21.15.86:0/881305683 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7fca5c05afa0 con 0x5593e2a2dc20 2024-04-25T14:08:56.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca71039000 10 client.7554 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T14:08:56.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca71039000 10 client.7554 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T14:08:56.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca71039000 20 client.7554 populate_metadata read hostname 'smithi086' 2024-04-25T14:08:56.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca71039000 1 --2- 172.21.15.86:0/881305683 >> [v2:172.21.15.86:6834/1961103704,v1:172.21.15.86:6835/1961103704] conn(0x5593e2624830 0x5593e2c05dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca71039000 4 client.7554 mds_command: new command op to 7371 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:08:56.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca71039000 1 -- 172.21.15.86:0/881305683 --> [v2:172.21.15.86:6834/1961103704,v1:172.21.15.86:6835/1961103704] -- command(tid 0: {"prefix": "status"}) -- 0x5593e299ea30 con 0x5593e2624830 2024-04-25T14:08:56.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca62a00640 1 --2- 172.21.15.86:0/881305683 >> [v2:172.21.15.86:6834/1961103704,v1:172.21.15.86:6835/1961103704] conn(0x5593e2624830 0x5593e2c05dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca47e00640 10 client.7554 ms_handle_connect on v2:172.21.15.86:6834/1961103704 2024-04-25T14:08:56.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca62a00640 1 --2- 172.21.15.86:0/881305683 >> [v2:172.21.15.86:6834/1961103704,v1:172.21.15.86:6835/1961103704] conn(0x5593e2624830 0x5593e2c05dc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9124ac67b908f9e server_cookie=3392aeb577ce192a in_seq=0 out_seq=0 2024-04-25T14:08:56.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca47e00640 1 -- 172.21.15.86:0/881305683 <== mds.0 v2:172.21.15.86:6834/1961103704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5593e299ea30 con 0x5593e2624830 2024-04-25T14:08:56.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca47e00640 10 client.7554 handle_command_reply: tid=0 2024-04-25T14:08:56.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca71039000 1 client.7554 shutdown 2024-04-25T14:08:56.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca71039000 1 -- 172.21.15.86:0/881305683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593e2a2dc20 msgr2=0x5593e2b97240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:56.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca71039000 1 --2- 172.21.15.86:0/881305683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593e2a2dc20 0x5593e2b97240 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fca5c075410 tx=0x7fca5c057d30 comp rx=0 tx=0).stop 2024-04-25T14:08:56.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca71039000 1 -- 172.21.15.86:0/881305683 shutdown_connections 2024-04-25T14:08:56.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca71039000 1 --2- 172.21.15.86:0/881305683 >> [v2:172.21.15.86:6834/1961103704,v1:172.21.15.86:6835/1961103704] conn(0x5593e2624830 0x5593e2c05dc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca71039000 1 --2- 172.21.15.86:0/881305683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593e2a2dc20 0x5593e2b97240 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca71039000 1 --2- 172.21.15.86:0/881305683 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5593e2a19ea0 0x5593e2b96d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca71039000 1 --2- 172.21.15.86:0/881305683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593e2612280 0x5593e2b967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca71039000 1 -- 172.21.15.86:0/881305683 >> 172.21.15.86:0/881305683 conn(0x5593e2651ff0 msgr2=0x5593e2b497d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:56.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca47e00640 0 client.7554 ms_handle_reset on v2:172.21.15.86:6834/1961103704 2024-04-25T14:08:56.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca71039000 1 -- 172.21.15.86:0/881305683 shutdown_connections 2024-04-25T14:08:56.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca71039000 1 -- 172.21.15.86:0/881305683 wait complete. 2024-04-25T14:08:56.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca71039000 20 client.7554 trim_cache size 0 max 16384 2024-04-25T14:08:56.337 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:08:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb", 2024-04-25T14:08:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-25T14:08:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 7371, 2024-04-25T14:08:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T14:08:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T14:08:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T14:08:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 10.620708098, 2024-04-25T14:08:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 67, 2024-04-25T14:08:56.338 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 87, 2024-04-25T14:08:56.338 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 87, 2024-04-25T14:08:56.338 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 15.469520899000001 2024-04-25T14:08:56.338 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:08:56.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca69a00640 1 -- 172.21.15.86:0/3872143649 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fca4006f520 msgr2=0x7fca40071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:56.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca69a00640 1 --2- 172.21.15.86:0/3872143649 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fca4006f520 0x7fca40071a30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fca54000f70 tx=0x7fca54001120 comp rx=0 tx=0).stop 2024-04-25T14:08:56.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca69a00640 1 -- 172.21.15.86:0/3872143649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca640d16e0 msgr2=0x7fca6409caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:56.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.330+0000 7fca69a00640 1 --2- 172.21.15.86:0/3872143649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca640d16e0 0x7fca6409caf0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00e7f0 tx=0x7fca5c00ecb0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.334+0000 7fca69a00640 1 -- 172.21.15.86:0/3872143649 shutdown_connections 2024-04-25T14:08:56.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.334+0000 7fca69a00640 1 --2- 172.21.15.86:0/3872143649 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fca4006f520 0x7fca40071a30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.334+0000 7fca69a00640 1 --2- 172.21.15.86:0/3872143649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca640d16e0 0x7fca6409caf0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.334+0000 7fca69a00640 1 --2- 172.21.15.86:0/3872143649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca640cfca0 0x7fca6414df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.334+0000 7fca69a00640 1 --2- 172.21.15.86:0/3872143649 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca640730b0 0x7fca6415f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.334+0000 7fca69a00640 1 -- 172.21.15.86:0/3872143649 >> 172.21.15.86:0/3872143649 conn(0x7fca640b4650 msgr2=0x7fca6415b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:56.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.334+0000 7fca69a00640 1 -- 172.21.15.86:0/3872143649 shutdown_connections 2024-04-25T14:08:56.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.334+0000 7fca69a00640 1 -- 172.21.15.86:0/3872143649 wait complete. 2024-04-25T14:08:56.353 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T14:08:56.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7da0000640 1 Processor -- start 2024-04-25T14:08:56.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7da0000640 1 -- start start 2024-04-25T14:08:56.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7da0000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d981651d0 0x7f7d981675f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7da0000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d980b0a60 con 0x7f7d9806fd50 2024-04-25T14:08:56.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7da0000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d980ae840 con 0x7f7d9815c2a0 2024-04-25T14:08:56.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7da0000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7d9806ef90 con 0x7f7d981651d0 2024-04-25T14:08:56.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7d9cc00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7d9806fd50 0x7f7d98070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49448/0 (socket says 172.21.15.86:49448) 2024-04-25T14:08:56.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7d9cc00640 1 -- 172.21.15.86:0/377252235 learned_addr learned my addr 172.21.15.86:0/377252235 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:56.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7d9e000640 1 --2- 172.21.15.86:0/377252235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d981651d0 0x7f7d981675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7d97e00640 1 -- 172.21.15.86:0/377252235 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 383120138 0 0) 0x7f7d980b0a60 con 0x7f7d9806fd50 2024-04-25T14:08:56.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7d97e00640 1 -- 172.21.15.86:0/377252235 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d80003530 con 0x7f7d9806fd50 2024-04-25T14:08:56.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7d97e00640 1 -- 172.21.15.86:0/377252235 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2178167161 0 0) 0x7f7d980ae840 con 0x7f7d9815c2a0 2024-04-25T14:08:56.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7d97e00640 1 -- 172.21.15.86:0/377252235 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d980b0a60 con 0x7f7d9815c2a0 2024-04-25T14:08:56.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7d97e00640 1 -- 172.21.15.86:0/377252235 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2710826432 0 0) 0x7f7d80003530 con 0x7f7d9806fd50 2024-04-25T14:08:56.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7d97e00640 1 -- 172.21.15.86:0/377252235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d981651d0 msgr2=0x7f7d981675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:56.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7d97e00640 1 --2- 172.21.15.86:0/377252235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d981651d0 0x7f7d981675f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7d97e00640 1 -- 172.21.15.86:0/377252235 >> v1:172.21.15.155:6789/0 conn(0x7f7d9815c2a0 legacy=0x7f7d980ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:56.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7d97e00640 1 -- 172.21.15.86:0/377252235 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9806f270 con 0x7f7d9806fd50 2024-04-25T14:08:56.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7d97e00640 1 -- 172.21.15.86:0/377252235 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f7d84001600 con 0x7f7d9806fd50 2024-04-25T14:08:56.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7d97e00640 1 -- 172.21.15.86:0/377252235 >> v1:172.21.15.86:6789/0 conn(0x7f7d9806fd50 legacy=0x7f7d98070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:56.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7d97e00640 1 --2- 172.21.15.86:0/377252235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d981651d0 0x7f7d80003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7d97e00640 1 --2- 172.21.15.86:0/377252235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d800041a0 0x7f7d80004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7d97e00640 1 --2- 172.21.15.86:0/377252235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d80004d50 0x7f7d80005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7d97e00640 1 -- 172.21.15.86:0/377252235 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9806ef90 con 0x7f7d80004d50 2024-04-25T14:08:56.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7d97e00640 1 -- 172.21.15.86:0/377252235 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8000bc90 con 0x7f7d800041a0 2024-04-25T14:08:56.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7d97e00640 1 -- 172.21.15.86:0/377252235 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7d8000e0a0 con 0x7f7d981651d0 2024-04-25T14:08:56.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7d9e000640 1 --2- 172.21.15.86:0/377252235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d80004d50 0x7f7d80005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7d9d600640 1 --2- 172.21.15.86:0/377252235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d981651d0 0x7f7d80003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7d9cc00640 1 --2- 172.21.15.86:0/377252235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d800041a0 0x7f7d80004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7d9e000640 1 -- 172.21.15.86:0/377252235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d981651d0 msgr2=0x7f7d80003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:56.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7d9e000640 1 --2- 172.21.15.86:0/377252235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d981651d0 0x7f7d80003bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7d9e000640 1 -- 172.21.15.86:0/377252235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d800041a0 msgr2=0x7f7d80004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:56.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7d9e000640 1 --2- 172.21.15.86:0/377252235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d800041a0 0x7f7d80004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.502+0000 7f7d9e000640 1 -- 172.21.15.86:0/377252235 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d80003530 con 0x7f7d80004d50 2024-04-25T14:08:56.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.506+0000 7f7d9e000640 1 --2- 172.21.15.86:0/377252235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d80004d50 0x7f7d80005200 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000a220 tx=0x7f7d9001cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f6880ca0e57722 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:56.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.506+0000 7f7d97e00640 1 -- 172.21.15.86:0/377252235 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d900ab070 con 0x7f7d80004d50 2024-04-25T14:08:56.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.506+0000 7f7d97e00640 1 -- 172.21.15.86:0/377252235 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d900a5d00 con 0x7f7d80004d50 2024-04-25T14:08:56.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.506+0000 7f7d97e00640 1 -- 172.21.15.86:0/377252235 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9000adf0 con 0x7f7d80004d50 2024-04-25T14:08:56.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.506+0000 7f7da0000640 1 -- 172.21.15.86:0/377252235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d80004d50 msgr2=0x7f7d80005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:56.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.506+0000 7f7da0000640 1 --2- 172.21.15.86:0/377252235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d80004d50 0x7f7d80005200 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000a220 tx=0x7f7d9001cdf0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.506+0000 7f7d9e000640 1 -- 172.21.15.86:0/377252235 reap_dead start 2024-04-25T14:08:56.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.506+0000 7f7da0000640 1 -- 172.21.15.86:0/377252235 shutdown_connections 2024-04-25T14:08:56.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.506+0000 7f7da0000640 1 -- 172.21.15.86:0/377252235 >> 172.21.15.86:0/377252235 conn(0x7f7d980b4650 msgr2=0x7f7d980b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:56.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.506+0000 7f7da0000640 1 -- 172.21.15.86:0/377252235 shutdown_connections 2024-04-25T14:08:56.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.506+0000 7f7da0000640 1 -- 172.21.15.86:0/377252235 wait complete. 2024-04-25T14:08:56.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.510+0000 7f7da0000640 1 Processor -- start 2024-04-25T14:08:56.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.510+0000 7f7da0000640 1 -- start start 2024-04-25T14:08:56.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.510+0000 7f7da0000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d80004d50 0x7f7d9809ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.510+0000 7f7da0000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d98159000 0x7f7d98164000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.510+0000 7f7da0000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d98164540 0x7f7d9809b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.510+0000 7f7da0000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d980d0fc0 con 0x7f7d98164540 2024-04-25T14:08:56.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.510+0000 7f7da0000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9806f830 con 0x7f7d98159000 2024-04-25T14:08:56.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.510+0000 7f7da0000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7d9815c6b0 con 0x7f7d80004d50 2024-04-25T14:08:56.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.514+0000 7f7d9e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d98164540 0x7f7d9809b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.514+0000 7f7d9d600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d80004d50 0x7f7d9809ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.514+0000 7f7d9e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d98164540 0x7f7d9809b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40466/0 (socket says 172.21.15.86:40466) 2024-04-25T14:08:56.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.514+0000 7f7d9e000640 1 -- 172.21.15.86:0/4289591900 learned_addr learned my addr 172.21.15.86:0/4289591900 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:56.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.514+0000 7f7d9cc00640 1 --2- 172.21.15.86:0/4289591900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d98159000 0x7f7d98164000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.514+0000 7f7d9e000640 1 -- 172.21.15.86:0/4289591900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d80004d50 msgr2=0x7f7d9809ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:56.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.514+0000 7f7d9e000640 1 --2- 172.21.15.86:0/4289591900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d80004d50 0x7f7d9809ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.514+0000 7f7d9e000640 1 -- 172.21.15.86:0/4289591900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d98159000 msgr2=0x7f7d98164000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:56.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.514+0000 7f7d9e000640 1 --2- 172.21.15.86:0/4289591900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d98159000 0x7f7d98164000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.514+0000 7f7d9e000640 1 -- 172.21.15.86:0/4289591900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d98164950 con 0x7f7d98164540 2024-04-25T14:08:56.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.514+0000 7f7d9e000640 1 --2- 172.21.15.86:0/4289591900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d98164540 0x7f7d9809b2b0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f7d900b1040 tx=0x7f7d90019b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:56.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.514+0000 7f7d95600640 1 -- 172.21.15.86:0/4289591900 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d900bc020 con 0x7f7d98164540 2024-04-25T14:08:56.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.514+0000 7f7d95600640 1 -- 172.21.15.86:0/4289591900 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d900afe40 con 0x7f7d98164540 2024-04-25T14:08:56.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.514+0000 7f7d95600640 1 -- 172.21.15.86:0/4289591900 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d900ab040 con 0x7f7d98164540 2024-04-25T14:08:56.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.514+0000 7f7da0000640 1 -- 172.21.15.86:0/4289591900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d9809f1f0 con 0x7f7d98164540 2024-04-25T14:08:56.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.514+0000 7f7da0000640 1 -- 172.21.15.86:0/4289591900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d980b2b90 con 0x7f7d98164540 2024-04-25T14:08:56.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.514+0000 7f7da0000640 1 -- 172.21.15.86:0/4289591900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d98069d00 con 0x7f7d98164540 2024-04-25T14:08:56.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.514+0000 7f7d95600640 1 -- 172.21.15.86:0/4289591900 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7f7d900bf070 con 0x7f7d98164540 2024-04-25T14:08:56.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.518+0000 7f7d95600640 1 --2- 172.21.15.86:0/4289591900 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f7d7806f550 0x7f7d78071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.518+0000 7f7d95600640 1 -- 172.21.15.86:0/4289591900 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f7d901294e0 con 0x7f7d98164540 2024-04-25T14:08:56.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.518+0000 7f7d9d600640 1 --2- 172.21.15.86:0/4289591900 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f7d7806f550 0x7f7d78071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.518+0000 7f7d9d600640 1 --2- 172.21.15.86:0/4289591900 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f7d7806f550 0x7f7d78071a60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0014f0 tx=0x7f7d8c001400 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:56.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.522+0000 7f7d95600640 1 -- 172.21.15.86:0/4289591900 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7d98164950 con 0x7f7d98164540 2024-04-25T14:08:56.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.710+0000 7f7da0000640 1 -- 172.21.15.86:0/4289591900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7d98076bf0 con 0x7f7d98164540 2024-04-25T14:08:56.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.714+0000 7f7d95600640 1 -- 172.21.15.86:0/4289591900 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) ==== 53+0+73 (secure 0 0 0) 0x7f7d90016b00 con 0x7f7d98164540 2024-04-25T14:08:56.714 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T14:08:56.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.714+0000 7f7da0000640 1 -- 172.21.15.86:0/4289591900 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f7d7806f550 msgr2=0x7f7d78071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:56.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.714+0000 7f7da0000640 1 --2- 172.21.15.86:0/4289591900 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f7d7806f550 0x7f7d78071a60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0014f0 tx=0x7f7d8c001400 comp rx=0 tx=0).stop 2024-04-25T14:08:56.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.714+0000 7f7da0000640 1 -- 172.21.15.86:0/4289591900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d98164540 msgr2=0x7f7d9809b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:56.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.714+0000 7f7da0000640 1 --2- 172.21.15.86:0/4289591900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d98164540 0x7f7d9809b2b0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f7d900b1040 tx=0x7f7d90019b90 comp rx=0 tx=0).stop 2024-04-25T14:08:56.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.714+0000 7f7da0000640 1 -- 172.21.15.86:0/4289591900 shutdown_connections 2024-04-25T14:08:56.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.714+0000 7f7da0000640 1 --2- 172.21.15.86:0/4289591900 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f7d7806f550 0x7f7d78071a60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.714+0000 7f7da0000640 1 --2- 172.21.15.86:0/4289591900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d98164540 0x7f7d9809b2b0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.714+0000 7f7da0000640 1 --2- 172.21.15.86:0/4289591900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d98159000 0x7f7d98164000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.714+0000 7f7da0000640 1 --2- 172.21.15.86:0/4289591900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d80004d50 0x7f7d9809ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.714+0000 7f7da0000640 1 -- 172.21.15.86:0/4289591900 >> 172.21.15.86:0/4289591900 conn(0x7f7d980b4650 msgr2=0x7f7d9815a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:56.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.714+0000 7f7da0000640 1 -- 172.21.15.86:0/4289591900 shutdown_connections 2024-04-25T14:08:56.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56.714+0000 7f7da0000640 1 -- 172.21.15.86:0/4289591900 wait complete. 2024-04-25T14:08:56.736 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T14:08:56.736 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T14:08:56.736 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-25T14:08:56.736 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T14:08:56.736 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T14:08:56.736 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T14:08:56.737 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T14:08:56.737 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T14:08:56.737 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:08:56.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:08:56.737 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-25T14:08:56.747 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:08:56.747 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:08:56.747 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:08:56.747 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:08:56.747 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-25T14:08:56.747 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:08:56.747 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:08:56.747 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:08:56.747 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-25T14:08:56.747 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:08:56.748 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:08:56.748 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-25T14:08:56.748 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:08:56.748 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:08:56.748 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T14:08:56.748 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:08:56.748 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T14:08:56.748 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42128sec preferred_lft 42128sec 2024-04-25T14:08:56.748 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-25T14:08:56.748 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:08:56.748 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T14:08:56.748 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:dc:87:9c:96 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:08:56.748 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T14:08:56.748 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:08:56.749 INFO:teuthology.orchestra.run.smithi086.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T14:08:56.749 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T14:08:56.749 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T14:08:56.749 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:08:56.749 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T14:08:56.749 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:08:56.749 INFO:teuthology.orchestra.run.smithi086.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:08:56.749 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T14:08:56.749 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T14:08:56.749 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:08:56.749 INFO:teuthology.orchestra.run.smithi086.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:08:56.749 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T14:08:56.749 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T14:08:56.749 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:08:56.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:08:56.750 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T14:08:56.807 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-25T14:08:56.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:08:56.809 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-25T14:08:56.865 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-25T14:08:56.866 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T14:08:56.866 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T14:08:56.866 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T14:08:56.866 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T14:08:56.866 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T14:08:56.866 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T14:08:56.866 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:08:56.866 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:08:56.866 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:08:56.866 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T14:08:56.866 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:08:56.867 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T14:08:56.867 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T14:08:56.867 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T14:08:56.867 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T14:08:56.867 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T14:08:56.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:08:56.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:08:57.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:08:57.003 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T14:08:57.058 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T14:08:57.059 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T14:08:57.059 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:08:57.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:08:57.059 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-25T14:08:57.121 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:08:57.121 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T14:08:57.169 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:08:57.169 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:08:57.169 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T14:08:57.224 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T14:08:57.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:08:57.225 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:08:57.324 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:08:57.324 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T14:08:57.324 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:08:57.342 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:08:57.342 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T14:08:57.343 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:08:57.343 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:08:57.343 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:08:57.354 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:08:57.354 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T14:08:57.354 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:08:57.354 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:08:57.354 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:08:57.354 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790" 2024-04-25T14:08:57.354 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790:/ 2024-04-25T14:08:57.354 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb" will pass to kernel 2024-04-25T14:08:57.358 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T14:08:57.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:08:57.358 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T14:08:57.379 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:08:57.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:08:57.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:08:57.482 INFO:teuthology.orchestra.run.smithi086.stdout:client6500 2024-04-25T14:08:57.482 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:08:57.482 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:08:57.491 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T14:08:57.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.682+0000 7f673a200640 1 Processor -- start 2024-04-25T14:08:57.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.682+0000 7f673a200640 1 -- start start 2024-04-25T14:08:57.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.682+0000 7f673a200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67340b6ee0 0x7f67340d06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:57.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.682+0000 7f673a200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f673415df00 con 0x7f67340729b0 2024-04-25T14:08:57.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.682+0000 7f673a200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f673415e1e0 con 0x7f67340d0c00 2024-04-25T14:08:57.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.682+0000 7f673a200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f673415e4c0 con 0x7f67340b6ee0 2024-04-25T14:08:57.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.682+0000 7f6739800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f67340729b0 0x7f67341614c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49452/0 (socket says 172.21.15.86:49452) 2024-04-25T14:08:57.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.682+0000 7f6739800640 1 -- 172.21.15.86:0/147271968 learned_addr learned my addr 172.21.15.86:0/147271968 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:57.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.682+0000 7f6738e00640 1 --2- 172.21.15.86:0/147271968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67340b6ee0 0x7f67340d06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:57.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.682+0000 7f6733400640 1 -- 172.21.15.86:0/147271968 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2240941744 0 0) 0x7f673415df00 con 0x7f67340729b0 2024-04-25T14:08:57.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.682+0000 7f6733400640 1 -- 172.21.15.86:0/147271968 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f671c003530 con 0x7f67340729b0 2024-04-25T14:08:57.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.682+0000 7f6733400640 1 -- 172.21.15.86:0/147271968 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 675756815 0 0) 0x7f671c003530 con 0x7f67340729b0 2024-04-25T14:08:57.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.682+0000 7f6733400640 1 -- 172.21.15.86:0/147271968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67340b6ee0 msgr2=0x7f67340d06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:57.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.682+0000 7f6733400640 1 --2- 172.21.15.86:0/147271968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67340b6ee0 0x7f67340d06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:57.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.682+0000 7f6733400640 1 -- 172.21.15.86:0/147271968 >> v1:172.21.15.155:6789/0 conn(0x7f67340d0c00 legacy=0x7f67340722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:57.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.682+0000 7f6733400640 1 -- 172.21.15.86:0/147271968 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673415e7a0 con 0x7f67340729b0 2024-04-25T14:08:57.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.682+0000 7f6733400640 1 -- 172.21.15.86:0/147271968 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f672c001a00 con 0x7f67340729b0 2024-04-25T14:08:57.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.682+0000 7f6733400640 1 -- 172.21.15.86:0/147271968 >> v1:172.21.15.86:6789/0 conn(0x7f67340729b0 legacy=0x7f67341614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:57.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.682+0000 7f6733400640 1 --2- 172.21.15.86:0/147271968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f671c0039f0 0x7f671c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:57.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.682+0000 7f6733400640 1 --2- 172.21.15.86:0/147271968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f671c004810 0x7f671c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:57.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.682+0000 7f6733400640 1 --2- 172.21.15.86:0/147271968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67340b6ee0 0x7f671c0052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:57.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.682+0000 7f6733400640 1 -- 172.21.15.86:0/147271968 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f673415e4c0 con 0x7f671c004810 2024-04-25T14:08:57.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.682+0000 7f6733400640 1 -- 172.21.15.86:0/147271968 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f671c00bb90 con 0x7f671c0039f0 2024-04-25T14:08:57.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.682+0000 7f6733400640 1 -- 172.21.15.86:0/147271968 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f671c00bd90 con 0x7f67340b6ee0 2024-04-25T14:08:57.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.682+0000 7f6733e00640 1 --2- 172.21.15.86:0/147271968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f671c004810 0x7f671c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:57.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.686+0000 7f6738e00640 1 --2- 172.21.15.86:0/147271968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f671c0039f0 0x7f671c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:57.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.686+0000 7f6739800640 1 --2- 172.21.15.86:0/147271968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67340b6ee0 0x7f671c0052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:57.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.686+0000 7f6733e00640 1 -- 172.21.15.86:0/147271968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67340b6ee0 msgr2=0x7f671c0052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:57.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.686+0000 7f6733e00640 1 --2- 172.21.15.86:0/147271968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67340b6ee0 0x7f671c0052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:57.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.686+0000 7f6733e00640 1 -- 172.21.15.86:0/147271968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f671c0039f0 msgr2=0x7f671c003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:57.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.686+0000 7f6733e00640 1 --2- 172.21.15.86:0/147271968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f671c0039f0 0x7f671c003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:57.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.686+0000 7f6733e00640 1 -- 172.21.15.86:0/147271968 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671c003530 con 0x7f671c004810 2024-04-25T14:08:57.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.686+0000 7f6733e00640 1 --2- 172.21.15.86:0/147271968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f671c004810 0x7f671c004c20 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f6720001780 tx=0x7f672001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e1045b864667ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:57.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.686+0000 7f6733400640 1 -- 172.21.15.86:0/147271968 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67200013a0 con 0x7f671c004810 2024-04-25T14:08:57.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.686+0000 7f6733400640 1 -- 172.21.15.86:0/147271968 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67200052f0 con 0x7f671c004810 2024-04-25T14:08:57.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.686+0000 7f6733400640 1 -- 172.21.15.86:0/147271968 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720003840 con 0x7f671c004810 2024-04-25T14:08:57.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.686+0000 7f673a200640 1 -- 172.21.15.86:0/147271968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f671c004810 msgr2=0x7f671c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:57.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.686+0000 7f673a200640 1 --2- 172.21.15.86:0/147271968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f671c004810 0x7f671c004c20 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f6720001780 tx=0x7f672001c420 comp rx=0 tx=0).stop 2024-04-25T14:08:57.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.690+0000 7f6739800640 1 -- 172.21.15.86:0/147271968 reap_dead start 2024-04-25T14:08:57.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.690+0000 7f673a200640 1 -- 172.21.15.86:0/147271968 shutdown_connections 2024-04-25T14:08:57.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.690+0000 7f673a200640 1 -- 172.21.15.86:0/147271968 >> 172.21.15.86:0/147271968 conn(0x7f67340b4f80 msgr2=0x7f67340b5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:57.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.690+0000 7f673a200640 1 -- 172.21.15.86:0/147271968 shutdown_connections 2024-04-25T14:08:57.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.690+0000 7f673a200640 1 -- 172.21.15.86:0/147271968 wait complete. 2024-04-25T14:08:57.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.690+0000 7f673a200640 1 Processor -- start 2024-04-25T14:08:57.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.690+0000 7f673a200640 1 -- start start 2024-04-25T14:08:57.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.694+0000 7f673a200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f671c004810 0x7f67341fa320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:57.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.694+0000 7f673a200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f673415a4c0 0x7f6734168b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:57.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.694+0000 7f673a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6734169080 0x7f6734158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:57.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.694+0000 7f673a200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f67341621f0 con 0x7f6734169080 2024-04-25T14:08:57.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.694+0000 7f673a200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f67340cff50 con 0x7f673415a4c0 2024-04-25T14:08:57.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.694+0000 7f673a200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6734169f70 con 0x7f671c004810 2024-04-25T14:08:57.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.694+0000 7f6739800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6734169080 0x7f6734158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:57.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.694+0000 7f6739800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6734169080 0x7f6734158ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40482/0 (socket says 172.21.15.86:40482) 2024-04-25T14:08:57.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.694+0000 7f6739800640 1 -- 172.21.15.86:0/3746732442 learned_addr learned my addr 172.21.15.86:0/3746732442 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:57.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.694+0000 7f6738e00640 1 --2- 172.21.15.86:0/3746732442 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f671c004810 0x7f67341fa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:57.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.694+0000 7f6733e00640 1 --2- 172.21.15.86:0/3746732442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f673415a4c0 0x7f6734168b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:57.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.694+0000 7f6739800640 1 -- 172.21.15.86:0/3746732442 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f671c004810 msgr2=0x7f67341fa320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:57.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.694+0000 7f6739800640 1 --2- 172.21.15.86:0/3746732442 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f671c004810 0x7f67341fa320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:57.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.694+0000 7f6739800640 1 -- 172.21.15.86:0/3746732442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f673415a4c0 msgr2=0x7f6734168b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:57.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.694+0000 7f6739800640 1 --2- 172.21.15.86:0/3746732442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f673415a4c0 0x7f6734168b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:57.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.694+0000 7f6739800640 1 -- 172.21.15.86:0/3746732442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6734169490 con 0x7f6734169080 2024-04-25T14:08:57.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.694+0000 7f6739800640 1 --2- 172.21.15.86:0/3746732442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6734169080 0x7f6734158ac0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f673415dda0 tx=0x7f672c00e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:57.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.694+0000 7f6730c00640 1 -- 172.21.15.86:0/3746732442 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c00eb70 con 0x7f6734169080 2024-04-25T14:08:57.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.694+0000 7f6730c00640 1 -- 172.21.15.86:0/3746732442 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f672c00ed30 con 0x7f6734169080 2024-04-25T14:08:57.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.694+0000 7f673a200640 1 -- 172.21.15.86:0/3746732442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f673406ee80 con 0x7f6734169080 2024-04-25T14:08:57.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.694+0000 7f6730c00640 1 -- 172.21.15.86:0/3746732442 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c013420 con 0x7f6734169080 2024-04-25T14:08:57.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.694+0000 7f673a200640 1 -- 172.21.15.86:0/3746732442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f673406e0c0 con 0x7f6734169080 2024-04-25T14:08:57.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.694+0000 7f673a200640 1 -- 172.21.15.86:0/3746732442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6734076b80 con 0x7f6734169080 2024-04-25T14:08:57.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.694+0000 7f6730c00640 1 -- 172.21.15.86:0/3746732442 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7f672c01a020 con 0x7f6734169080 2024-04-25T14:08:57.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.698+0000 7f6730c00640 1 --2- 172.21.15.86:0/3746732442 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f671406f520 0x7f6714071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:57.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.698+0000 7f6730c00640 1 -- 172.21.15.86:0/3746732442 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f672c08f0f0 con 0x7f6734169080 2024-04-25T14:08:57.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.698+0000 7f6738e00640 1 --2- 172.21.15.86:0/3746732442 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f671406f520 0x7f6714071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:57.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.698+0000 7f6738e00640 1 --2- 172.21.15.86:0/3746732442 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f671406f520 0x7f6714071a30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f67340749a0 tx=0x7f6728008040 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:57.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.702+0000 7f6730c00640 1 -- 172.21.15.86:0/3746732442 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f672c021020 con 0x7f6734169080 2024-04-25T14:08:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.890+0000 7f673a200640 1 -- 172.21.15.86:0/3746732442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f67340b8080 con 0x7f6734169080 2024-04-25T14:08:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.894+0000 7f6730c00640 1 -- 172.21.15.86:0/3746732442 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) ==== 53+0+73 (secure 0 0 0) 0x7f6734169490 con 0x7f6734169080 2024-04-25T14:08:57.894 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T14:08:57.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.894+0000 7f673a200640 1 -- 172.21.15.86:0/3746732442 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f671406f520 msgr2=0x7f6714071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.894+0000 7f673a200640 1 --2- 172.21.15.86:0/3746732442 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f671406f520 0x7f6714071a30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f67340749a0 tx=0x7f6728008040 comp rx=0 tx=0).stop 2024-04-25T14:08:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.894+0000 7f673a200640 1 -- 172.21.15.86:0/3746732442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6734169080 msgr2=0x7f6734158ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.894+0000 7f673a200640 1 --2- 172.21.15.86:0/3746732442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6734169080 0x7f6734158ac0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f673415dda0 tx=0x7f672c00e160 comp rx=0 tx=0).stop 2024-04-25T14:08:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.894+0000 7f673a200640 1 -- 172.21.15.86:0/3746732442 shutdown_connections 2024-04-25T14:08:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.894+0000 7f673a200640 1 --2- 172.21.15.86:0/3746732442 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f671406f520 0x7f6714071a30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.894+0000 7f673a200640 1 --2- 172.21.15.86:0/3746732442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6734169080 0x7f6734158ac0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.894+0000 7f673a200640 1 --2- 172.21.15.86:0/3746732442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f673415a4c0 0x7f6734168b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.894+0000 7f673a200640 1 --2- 172.21.15.86:0/3746732442 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f671c004810 0x7f67341fa320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.894+0000 7f673a200640 1 -- 172.21.15.86:0/3746732442 >> 172.21.15.86:0/3746732442 conn(0x7f67340b4f80 msgr2=0x7f6734160220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.894+0000 7f673a200640 1 -- 172.21.15.86:0/3746732442 shutdown_connections 2024-04-25T14:08:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:57.894+0000 7f673a200640 1 -- 172.21.15.86:0/3746732442 wait complete. 2024-04-25T14:08:57.916 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T14:08:57.916 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T14:08:57.916 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2024-04-25T14:08:57.916 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T14:08:57.916 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T14:08:57.916 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T14:08:57.916 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T14:08:57.916 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T14:08:57.916 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:08:57.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:08:57.917 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2024-04-25T14:08:57.927 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:08:57.927 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:08:57.927 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:08:57.927 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:08:57.927 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2024-04-25T14:08:57.928 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:08:57.928 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:08:57.928 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2024-04-25T14:08:57.928 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f0 2024-04-25T14:08:57.928 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:08:57.928 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2024-04-25T14:08:57.928 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f1 2024-04-25T14:08:57.928 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:08:57.928 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2024-04-25T14:08:57.928 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T14:08:57.928 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2024-04-25T14:08:57.928 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T14:08:57.929 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 42120sec preferred_lft 42120sec 2024-04-25T14:08:57.929 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::ec4:7aff:fe88:6ded/64 scope link 2024-04-25T14:08:57.929 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:08:57.929 INFO:teuthology.orchestra.run.smithi155.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T14:08:57.929 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 02:42:b1:ff:60:50 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:08:57.929 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T14:08:57.929 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:08:57.929 INFO:teuthology.orchestra.run.smithi155.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T14:08:57.929 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T14:08:57.929 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T14:08:57.929 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:08:57.929 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T14:08:57.929 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:08:57.929 INFO:teuthology.orchestra.run.smithi155.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:08:57.930 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T14:08:57.930 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T14:08:57.930 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:08:57.930 INFO:teuthology.orchestra.run.smithi155.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:08:57.930 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T14:08:57.930 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T14:08:57.930 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:08:57.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:08:57.931 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T14:08:57.993 INFO:teuthology.orchestra.run.smithi155.stdout:1 2024-04-25T14:08:57.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:08:57.996 DEBUG:teuthology.orchestra.run.smithi155:> route 2024-04-25T14:08:58.049 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2024-04-25T14:08:58.050 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T14:08:58.050 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T14:08:58.050 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T14:08:58.050 INFO:teuthology.orchestra.run.smithi155.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T14:08:58.050 INFO:teuthology.orchestra.run.smithi155.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T14:08:58.050 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T14:08:58.050 INFO:teuthology.orchestra.run.smithi155.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:08:58.050 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:08:58.050 INFO:teuthology.orchestra.run.smithi155.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:08:58.050 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T14:08:58.050 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:08:58.051 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-04-25T14:08:58.051 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T14:08:58.051 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T14:08:58.051 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T14:08:58.051 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-04-25T14:08:58.149 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:08:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:08:58.192 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:08:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:08:58.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:08:58.199 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2024-04-25T14:08:58.208 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T14:08:58.208 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T14:08:58.208 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:08:58.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:08:58.209 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.0 up 2024-04-25T14:08:58.273 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:08:58.273 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T14:08:58.321 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:08:58.321 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:08:58.321 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T14:08:58.372 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T14:08:58.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:08:58.372 DEBUG:teuthology.orchestra.run.smithi155:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:08:58.473 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:08:58.474 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T14:08:58.474 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:08:58.491 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:08:58.491 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T14:08:58.492 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:08:58.492 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:08:58.492 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:08:58.502 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:08:58.502 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T14:08:58.502 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:08:58.502 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:08:58.502 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:08:58.503 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790" 2024-04-25T14:08:58.503 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790:/ 2024-04-25T14:08:58.503 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb" will pass to kernel 2024-04-25T14:08:58.506 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T14:08:58.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:08:58.507 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T14:08:58.526 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:08:58.615 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:08:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:08:58.618 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:08:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:08:58.624 INFO:teuthology.orchestra.run.smithi155.stdout:client7327 2024-04-25T14:08:58.625 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:08:58.625 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:08:58.634 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:08:58.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.786+0000 7f3162c00640 1 Processor -- start 2024-04-25T14:08:58.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.786+0000 7f3162c00640 1 -- start start 2024-04-25T14:08:58.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.786+0000 7f3162c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f315c0722a0 0x7f315c0726b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:58.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.786+0000 7f3162c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f315c1614c0 0x7f315c1618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:58.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.786+0000 7f3162c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f315c072bf0 con 0x7f315c0b6ee0 2024-04-25T14:08:58.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.786+0000 7f3162c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f315c15df00 con 0x7f315c1614c0 2024-04-25T14:08:58.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.786+0000 7f3162c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f315c15e1e0 con 0x7f315c0722a0 2024-04-25T14:08:58.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.786+0000 7f3162200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f315c1614c0 0x7f315c1618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:58.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.786+0000 7f3162200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f315c1614c0 0x7f315c1618d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38998/0 (socket says 172.21.15.86:38998) 2024-04-25T14:08:58.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.786+0000 7f3162200640 1 -- 172.21.15.86:0/1218705391 learned_addr learned my addr 172.21.15.86:0/1218705391 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:58.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.786+0000 7f3160e00640 1 --2- 172.21.15.86:0/1218705391 >> v2:172.21.15.155:3300/0 conn(0x7f315c0722a0 0x7f315c0726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:58.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.786+0000 7f3162200640 1 -- 172.21.15.86:0/1218705391 >> v2:172.21.15.155:3300/0 conn(0x7f315c0722a0 msgr2=0x7f315c0726b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:58.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.786+0000 7f3162200640 1 --2- 172.21.15.86:0/1218705391 >> v2:172.21.15.155:3300/0 conn(0x7f315c0722a0 0x7f315c0726b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:58.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.786+0000 7f3162200640 1 -- 172.21.15.86:0/1218705391 >> v1:172.21.15.155:6789/0 conn(0x7f315c0b6ee0 legacy=0x7f315c0d06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:58.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.786+0000 7f3162200640 1 -- 172.21.15.86:0/1218705391 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f315c15e4c0 con 0x7f315c1614c0 2024-04-25T14:08:58.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.786+0000 7f315be00640 1 -- 172.21.15.86:0/1218705391 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3562454940 0 0) 0x7f3150001120 con 0x7f315c0b6ee0 2024-04-25T14:08:58.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.790+0000 7f3162200640 1 --2- 172.21.15.86:0/1218705391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315c1614c0 0x7f315c1618d0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f3154009e50 tx=0x7f315401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a5e084a3da1f70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:58.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.790+0000 7f315be00640 1 -- 172.21.15.86:0/1218705391 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31540a5620 con 0x7f315c1614c0 2024-04-25T14:08:58.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.790+0000 7f315be00640 1 -- 172.21.15.86:0/1218705391 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31540a57c0 con 0x7f315c1614c0 2024-04-25T14:08:58.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.790+0000 7f315be00640 1 -- 172.21.15.86:0/1218705391 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31540a5b90 con 0x7f315c1614c0 2024-04-25T14:08:58.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.790+0000 7f3162c00640 1 -- 172.21.15.86:0/1218705391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315c1614c0 msgr2=0x7f315c1618d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:58.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.790+0000 7f3162c00640 1 --2- 172.21.15.86:0/1218705391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315c1614c0 0x7f315c1618d0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f3154009e50 tx=0x7f315401c960 comp rx=0 tx=0).stop 2024-04-25T14:08:58.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.790+0000 7f3162c00640 1 -- 172.21.15.86:0/1218705391 shutdown_connections 2024-04-25T14:08:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.790+0000 7f3162c00640 1 --2- 172.21.15.86:0/1218705391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315c1614c0 0x7f315c1618d0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.790+0000 7f3162c00640 1 --2- 172.21.15.86:0/1218705391 >> v2:172.21.15.155:3300/0 conn(0x7f315c0722a0 0x7f315c0726b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.790+0000 7f3162c00640 1 -- 172.21.15.86:0/1218705391 >> 172.21.15.86:0/1218705391 conn(0x7f315c0b4f80 msgr2=0x7f315c0b5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.790+0000 7f3162c00640 1 -- 172.21.15.86:0/1218705391 shutdown_connections 2024-04-25T14:08:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.790+0000 7f3162c00640 1 -- 172.21.15.86:0/1218705391 wait complete. 2024-04-25T14:08:58.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.794+0000 7f3162c00640 1 Processor -- start 2024-04-25T14:08:58.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.794+0000 7f3162c00640 1 -- start start 2024-04-25T14:08:58.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.794+0000 7f3162c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f315c0722a0 0x7f315c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:58.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.794+0000 7f3162c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f315c0b6ee0 0x7f315c140820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:58.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.794+0000 7f3162c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315c1614c0 0x7f315c06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:58.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.794+0000 7f3162c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f315c0cfd40 con 0x7f315c1614c0 2024-04-25T14:08:58.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.794+0000 7f3162c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f315c162220 con 0x7f315c0722a0 2024-04-25T14:08:58.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.794+0000 7f3162c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f315c0cfec0 con 0x7f315c0b6ee0 2024-04-25T14:08:58.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.794+0000 7f3162200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315c1614c0 0x7f315c06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:58.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.794+0000 7f3162200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315c1614c0 0x7f315c06dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39014/0 (socket says 172.21.15.86:39014) 2024-04-25T14:08:58.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.794+0000 7f3162200640 1 -- 172.21.15.86:0/2921786762 learned_addr learned my addr 172.21.15.86:0/2921786762 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:58.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.794+0000 7f3160e00640 1 --2- 172.21.15.86:0/2921786762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f315c0b6ee0 0x7f315c140820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:58.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.794+0000 7f3161800640 1 --2- 172.21.15.86:0/2921786762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f315c0722a0 0x7f315c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:58.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.794+0000 7f3162200640 1 -- 172.21.15.86:0/2921786762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f315c0b6ee0 msgr2=0x7f315c140820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:58.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.794+0000 7f3162200640 1 --2- 172.21.15.86:0/2921786762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f315c0b6ee0 0x7f315c140820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:58.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.794+0000 7f3162200640 1 -- 172.21.15.86:0/2921786762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f315c0722a0 msgr2=0x7f315c06d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:58.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.794+0000 7f3162200640 1 --2- 172.21.15.86:0/2921786762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f315c0722a0 0x7f315c06d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:58.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.794+0000 7f3162200640 1 -- 172.21.15.86:0/2921786762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f315c15c710 con 0x7f315c1614c0 2024-04-25T14:08:58.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.798+0000 7f3162200640 1 --2- 172.21.15.86:0/2921786762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315c1614c0 0x7f315c06dfd0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f3154009e50 tx=0x7f3154019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:58.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.798+0000 7f3159600640 1 -- 172.21.15.86:0/2921786762 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3154009510 con 0x7f315c1614c0 2024-04-25T14:08:58.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.798+0000 7f3159600640 1 -- 172.21.15.86:0/2921786762 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31540be070 con 0x7f315c1614c0 2024-04-25T14:08:58.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.798+0000 7f3162c00640 1 -- 172.21.15.86:0/2921786762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f315c15c9f0 con 0x7f315c1614c0 2024-04-25T14:08:58.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.798+0000 7f3159600640 1 -- 172.21.15.86:0/2921786762 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31540b88d0 con 0x7f315c1614c0 2024-04-25T14:08:58.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.798+0000 7f3162c00640 1 -- 172.21.15.86:0/2921786762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f315c141d90 con 0x7f315c1614c0 2024-04-25T14:08:58.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.798+0000 7f3162c00640 1 -- 172.21.15.86:0/2921786762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f315c076b80 con 0x7f315c1614c0 2024-04-25T14:08:58.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.798+0000 7f3159600640 1 -- 172.21.15.86:0/2921786762 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7f31540b8a70 con 0x7f315c1614c0 2024-04-25T14:08:58.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.798+0000 7f3159600640 1 --2- 172.21.15.86:0/2921786762 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f313c06f520 0x7f313c071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:58.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.798+0000 7f3159600640 1 -- 172.21.15.86:0/2921786762 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f315412a390 con 0x7f315c1614c0 2024-04-25T14:08:58.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.802+0000 7f3161800640 1 --2- 172.21.15.86:0/2921786762 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f313c06f520 0x7f313c071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:58.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.802+0000 7f3161800640 1 --2- 172.21.15.86:0/2921786762 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f313c06f520 0x7f313c071a30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3150007fd0 tx=0x7f3150009040 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:58.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.806+0000 7f3159600640 1 -- 172.21.15.86:0/2921786762 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f31540165e0 con 0x7f315c1614c0 2024-04-25T14:08:58.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.994+0000 7f3162c00640 1 -- 172.21.15.86:0/2921786762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f315c141b40 con 0x7f315c1614c0 2024-04-25T14:08:58.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.994+0000 7f3159600640 1 -- 172.21.15.86:0/2921786762 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5186 (secure 0 0 0) 0x7f31540f6990 con 0x7f315c1614c0 2024-04-25T14:08:58.996 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:08:58.996 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7186,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/4265929298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4265929298},{"type":"v1","addr":"172.21.15.155:6833","nonce":4265929298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7189,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3254219000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3254219000},{"type":"v1","addr":"172.21.15.155:6835","nonce":3254219000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7195,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/193454084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":193454084},{"type":"v1","addr":"172.21.15.86:6837","nonce":193454084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:08:45.096500+0000","modified":"2024-04-25T14:08:48.121611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7371},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7371":{"gid":7371,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/1961103704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1961103704},{"type":"v1","addr":"172.21.15.86:6835","nonce":1961103704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7371,"qdb_cluster":[7371]},"id":6}]} 2024-04-25T14:08:58.996 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 67 2024-04-25T14:08:58.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.998+0000 7f3162c00640 1 -- 172.21.15.86:0/2921786762 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f313c06f520 msgr2=0x7f313c071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:58.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.998+0000 7f3162c00640 1 --2- 172.21.15.86:0/2921786762 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f313c06f520 0x7f313c071a30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3150007fd0 tx=0x7f3150009040 comp rx=0 tx=0).stop 2024-04-25T14:08:58.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.998+0000 7f3162c00640 1 -- 172.21.15.86:0/2921786762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315c1614c0 msgr2=0x7f315c06dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:58.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.998+0000 7f3162c00640 1 --2- 172.21.15.86:0/2921786762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315c1614c0 0x7f315c06dfd0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f3154009e50 tx=0x7f3154019700 comp rx=0 tx=0).stop 2024-04-25T14:08:58.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.998+0000 7f3162c00640 1 -- 172.21.15.86:0/2921786762 shutdown_connections 2024-04-25T14:08:58.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.998+0000 7f3162c00640 1 --2- 172.21.15.86:0/2921786762 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f313c06f520 0x7f313c071a30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:58.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.998+0000 7f3162c00640 1 --2- 172.21.15.86:0/2921786762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315c1614c0 0x7f315c06dfd0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:58.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.998+0000 7f3162c00640 1 --2- 172.21.15.86:0/2921786762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f315c0b6ee0 0x7f315c140820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:58.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.998+0000 7f3162c00640 1 --2- 172.21.15.86:0/2921786762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f315c0722a0 0x7f315c06d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:58.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.998+0000 7f3162c00640 1 -- 172.21.15.86:0/2921786762 >> 172.21.15.86:0/2921786762 conn(0x7f315c0b4f80 msgr2=0x7f315c0b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:58.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.998+0000 7f3162c00640 1 -- 172.21.15.86:0/2921786762 shutdown_connections 2024-04-25T14:08:58.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:58.998+0000 7f3162c00640 1 -- 172.21.15.86:0/2921786762 wait complete. 2024-04-25T14:08:59.018 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable stats 2024-04-25T14:08:59.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.166+0000 7fe5b8400640 1 Processor -- start 2024-04-25T14:08:59.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.166+0000 7fe5b8400640 1 -- start start 2024-04-25T14:08:59.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.166+0000 7fe5b8400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5b00ae130 0x7fe5b006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.166+0000 7fe5b8400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe5b00ae540 0x7fe5b0070290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.166+0000 7fe5b8400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5b00b0a60 con 0x7fe5b015c2a0 2024-04-25T14:08:59.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.166+0000 7fe5b8400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe5b00d1df0 con 0x7fe5b00ae540 2024-04-25T14:08:59.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.166+0000 7fe5b8400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe5b006ef90 con 0x7fe5b00ae130 2024-04-25T14:08:59.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.166+0000 7fe5b6400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe5b00ae540 0x7fe5b0070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.166+0000 7fe5b5000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5b00ae130 0x7fe5b006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.166+0000 7fe5b5a00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fe5b015c2a0 0x7fe5b00d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:46694/0 (socket says 172.21.15.86:46694) 2024-04-25T14:08:59.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.166+0000 7fe5b5a00640 1 -- 172.21.15.86:0/3793116147 learned_addr learned my addr 172.21.15.86:0/3793116147 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:59.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.166+0000 7fe5afe00640 1 -- 172.21.15.86:0/3793116147 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 374774210 0 0) 0x7fe5b00b0a60 con 0x7fe5b015c2a0 2024-04-25T14:08:59.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.166+0000 7fe5afe00640 1 -- 172.21.15.86:0/3793116147 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe598003530 con 0x7fe5b015c2a0 2024-04-25T14:08:59.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.166+0000 7fe5b6400640 1 -- 172.21.15.86:0/3793116147 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5b00ae130 msgr2=0x7fe5b006fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:59.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.166+0000 7fe5b6400640 1 --2- 172.21.15.86:0/3793116147 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5b00ae130 0x7fe5b006fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.166+0000 7fe5b6400640 1 -- 172.21.15.86:0/3793116147 >> v1:172.21.15.155:6789/0 conn(0x7fe5b015c2a0 legacy=0x7fe5b00d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:59.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.166+0000 7fe5b6400640 1 -- 172.21.15.86:0/3793116147 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b006f270 con 0x7fe5b00ae540 2024-04-25T14:08:59.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.166+0000 7fe5b5000640 1 --2- 172.21.15.86:0/3793116147 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5b00ae130 0x7fe5b006fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:08:59.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.170+0000 7fe5b6400640 1 --2- 172.21.15.86:0/3793116147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5b00ae540 0x7fe5b0070290 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8009d90 tx=0x7fe5a801c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d173ee8edea1be7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.170+0000 7fe5afe00640 1 -- 172.21.15.86:0/3793116147 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a80b2070 con 0x7fe5b00ae540 2024-04-25T14:08:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.170+0000 7fe5afe00640 1 -- 172.21.15.86:0/3793116147 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5a80a57a0 con 0x7fe5b00ae540 2024-04-25T14:08:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.170+0000 7fe5afe00640 1 -- 172.21.15.86:0/3793116147 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a80a5b70 con 0x7fe5b00ae540 2024-04-25T14:08:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.170+0000 7fe5b8400640 1 -- 172.21.15.86:0/3793116147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5b00ae540 msgr2=0x7fe5b0070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.170+0000 7fe5b8400640 1 --2- 172.21.15.86:0/3793116147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5b00ae540 0x7fe5b0070290 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8009d90 tx=0x7fe5a801c8e0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.170+0000 7fe5b8400640 1 -- 172.21.15.86:0/3793116147 shutdown_connections 2024-04-25T14:08:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.170+0000 7fe5b8400640 1 --2- 172.21.15.86:0/3793116147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5b00ae540 0x7fe5b0070290 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.170+0000 7fe5b8400640 1 --2- 172.21.15.86:0/3793116147 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5b00ae130 0x7fe5b006fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.170+0000 7fe5b8400640 1 -- 172.21.15.86:0/3793116147 >> 172.21.15.86:0/3793116147 conn(0x7fe5b00b4650 msgr2=0x7fe5b00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:59.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.170+0000 7fe5b8400640 1 -- 172.21.15.86:0/3793116147 shutdown_connections 2024-04-25T14:08:59.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.170+0000 7fe5b8400640 1 -- 172.21.15.86:0/3793116147 wait complete. 2024-04-25T14:08:59.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.174+0000 7fe5b8400640 1 Processor -- start 2024-04-25T14:08:59.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.174+0000 7fe5b8400640 1 -- start start 2024-04-25T14:08:59.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.174+0000 7fe5b8400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5b00ae130 0x7fe5b01fdba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.174+0000 7fe5b8400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5b00ae540 0x7fe5b009a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.174+0000 7fe5b8400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5b015c2a0 0x7fe5b01587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.174+0000 7fe5b8400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b015c830 con 0x7fe5b015c2a0 2024-04-25T14:08:59.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.174+0000 7fe5b8400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b015c6b0 con 0x7fe5b00ae130 2024-04-25T14:08:59.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.174+0000 7fe5b8400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe5b01606c0 con 0x7fe5b00ae540 2024-04-25T14:08:59.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.174+0000 7fe5b6400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5b015c2a0 0x7fe5b01587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.174+0000 7fe5b6400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5b015c2a0 0x7fe5b01587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39026/0 (socket says 172.21.15.86:39026) 2024-04-25T14:08:59.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.174+0000 7fe5b6400640 1 -- 172.21.15.86:0/617257372 learned_addr learned my addr 172.21.15.86:0/617257372 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:59.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.174+0000 7fe5b5000640 1 --2- 172.21.15.86:0/617257372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5b00ae540 0x7fe5b009a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.174+0000 7fe5b5a00640 1 --2- 172.21.15.86:0/617257372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5b00ae130 0x7fe5b01fdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.174+0000 7fe5b6400640 1 -- 172.21.15.86:0/617257372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5b00ae540 msgr2=0x7fe5b009a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.174+0000 7fe5b6400640 1 --2- 172.21.15.86:0/617257372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5b00ae540 0x7fe5b009a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.174+0000 7fe5b6400640 1 -- 172.21.15.86:0/617257372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5b00ae130 msgr2=0x7fe5b01fdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.174+0000 7fe5b6400640 1 --2- 172.21.15.86:0/617257372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5b00ae130 0x7fe5b01fdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.174+0000 7fe5b6400640 1 -- 172.21.15.86:0/617257372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b00cfcd0 con 0x7fe5b015c2a0 2024-04-25T14:08:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.174+0000 7fe5b5a00640 1 --2- 172.21.15.86:0/617257372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5b00ae130 0x7fe5b01fdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.178+0000 7fe5b6400640 1 --2- 172.21.15.86:0/617257372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5b015c2a0 0x7fe5b01587a0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fe5a80b0040 tx=0x7fe5a8019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.178+0000 7fe5ad600640 1 -- 172.21.15.86:0/617257372 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a80b2040 con 0x7fe5b015c2a0 2024-04-25T14:08:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.178+0000 7fe5ad600640 1 -- 172.21.15.86:0/617257372 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5a80bf070 con 0x7fe5b015c2a0 2024-04-25T14:08:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.178+0000 7fe5ad600640 1 -- 172.21.15.86:0/617257372 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a80b86b0 con 0x7fe5b015c2a0 2024-04-25T14:08:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.178+0000 7fe5b8400640 1 -- 172.21.15.86:0/617257372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5b0155e50 con 0x7fe5b015c2a0 2024-04-25T14:08:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.178+0000 7fe5b8400640 1 -- 172.21.15.86:0/617257372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5b01563f0 con 0x7fe5b015c2a0 2024-04-25T14:08:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.178+0000 7fe5b8400640 1 -- 172.21.15.86:0/617257372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5b0069d00 con 0x7fe5b015c2a0 2024-04-25T14:08:59.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.178+0000 7fe5ad600640 1 -- 172.21.15.86:0/617257372 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 28) ==== 93309+0+0 (secure 0 0 0) 0x7fe5a80b8850 con 0x7fe5b015c2a0 2024-04-25T14:08:59.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.182+0000 7fe5ad600640 1 --2- 172.21.15.86:0/617257372 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fe59006f520 0x7fe590071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.182+0000 7fe5ad600640 1 -- 172.21.15.86:0/617257372 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7fe5a812a490 con 0x7fe5b015c2a0 2024-04-25T14:08:59.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.182+0000 7fe5b5a00640 1 --2- 172.21.15.86:0/617257372 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fe59006f520 0x7fe590071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.182+0000 7fe5b5a00640 1 --2- 172.21.15.86:0/617257372 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fe59006f520 0x7fe590071a30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4005890 tx=0x7fe5a4009040 comp rx=0 tx=0).ready entity=mgr.7092 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:59.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.186+0000 7fe5ad600640 1 -- 172.21.15.86:0/617257372 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe5a8016560 con 0x7fe5b015c2a0 2024-04-25T14:08:59.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.382+0000 7fe5b8400640 1 -- 172.21.15.86:0/617257372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "stats"} v 0) -- 0x7fe5b0154a40 con 0x7fe5b015c2a0 2024-04-25T14:08:59.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.394+0000 7fe5ad600640 1 -- 172.21.15.86:0/617257372 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "stats"}]=0 v29) ==== 84+0+0 (secure 0 0 0) 0x7fe5a80f70d0 con 0x7fe5b015c2a0 2024-04-25T14:08:59.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.394+0000 7fe5b8400640 1 -- 172.21.15.86:0/617257372 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fe59006f520 msgr2=0x7fe590071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:59.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.394+0000 7fe5b8400640 1 --2- 172.21.15.86:0/617257372 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fe59006f520 0x7fe590071a30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4005890 tx=0x7fe5a4009040 comp rx=0 tx=0).stop 2024-04-25T14:08:59.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.394+0000 7fe5b8400640 1 -- 172.21.15.86:0/617257372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5b015c2a0 msgr2=0x7fe5b01587a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:59.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.394+0000 7fe5b8400640 1 --2- 172.21.15.86:0/617257372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5b015c2a0 0x7fe5b01587a0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fe5a80b0040 tx=0x7fe5a8019680 comp rx=0 tx=0).stop 2024-04-25T14:08:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.394+0000 7fe5b8400640 1 -- 172.21.15.86:0/617257372 shutdown_connections 2024-04-25T14:08:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.394+0000 7fe5b8400640 1 --2- 172.21.15.86:0/617257372 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7fe59006f520 0x7fe590071a30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.394+0000 7fe5b8400640 1 --2- 172.21.15.86:0/617257372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5b015c2a0 0x7fe5b01587a0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.394+0000 7fe5b8400640 1 --2- 172.21.15.86:0/617257372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5b00ae540 0x7fe5b009a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.394+0000 7fe5b8400640 1 --2- 172.21.15.86:0/617257372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5b00ae130 0x7fe5b01fdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.394+0000 7fe5b8400640 1 -- 172.21.15.86:0/617257372 >> 172.21.15.86:0/617257372 conn(0x7fe5b00b4650 msgr2=0x7fe5b0072dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.394+0000 7fe5b8400640 1 -- 172.21.15.86:0/617257372 shutdown_connections 2024-04-25T14:08:59.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.394+0000 7fe5b8400640 1 -- 172.21.15.86:0/617257372 wait complete. 2024-04-25T14:08:59.420 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats --client_id abcd 2024-04-25T14:08:59.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.574+0000 7f7a36e00640 1 Processor -- start 2024-04-25T14:08:59.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.574+0000 7f7a36e00640 1 -- start start 2024-04-25T14:08:59.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.574+0000 7f7a36e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7a300ccab0 0x7f7a300d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.574+0000 7f7a36e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a300b79c0 0x7f7a300d2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.574+0000 7f7a36e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a300b3bd0 con 0x7f7a300cf9d0 2024-04-25T14:08:59.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.574+0000 7f7a36e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7a300cf190 con 0x7f7a300ccab0 2024-04-25T14:08:59.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.574+0000 7f7a36e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7a300b2620 con 0x7f7a300b79c0 2024-04-25T14:08:59.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.574+0000 7f7a2fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7a300ccab0 0x7f7a300d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.574+0000 7f7a2f400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a300b79c0 0x7f7a300d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.574+0000 7f7a34e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7a300cf9d0 0x7f7a301617f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:46134/0 (socket says 172.21.15.86:46134) 2024-04-25T14:08:59.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.574+0000 7f7a34e00640 1 -- 172.21.15.86:0/2118582191 learned_addr learned my addr 172.21.15.86:0/2118582191 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:08:59.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.574+0000 7f7a2f400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a300b79c0 0x7f7a300d2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:34220/0 (socket says 172.21.15.86:34220) 2024-04-25T14:08:59.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.578+0000 7f7a2f400640 1 -- 172.21.15.86:0/2118582191 >> v2:172.21.15.86:3300/0 conn(0x7f7a300ccab0 msgr2=0x7f7a300d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:59.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.578+0000 7f7a2f400640 1 --2- 172.21.15.86:0/2118582191 >> v2:172.21.15.86:3300/0 conn(0x7f7a300ccab0 0x7f7a300d23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.578+0000 7f7a2f400640 1 -- 172.21.15.86:0/2118582191 >> v1:172.21.15.86:6789/0 conn(0x7f7a300cf9d0 legacy=0x7f7a301617f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:59.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.578+0000 7f7a2f400640 1 -- 172.21.15.86:0/2118582191 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a300b1cf0 con 0x7f7a300b79c0 2024-04-25T14:08:59.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.578+0000 7f7a2f400640 1 --2- 172.21.15.86:0/2118582191 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a300b79c0 0x7f7a300d2910 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c0097c0 tx=0x7f7a1c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23a444215c593c65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:59.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.578+0000 7f7a2ea00640 1 -- 172.21.15.86:0/2118582191 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c0b0070 con 0x7f7a300b79c0 2024-04-25T14:08:59.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.578+0000 7f7a2ea00640 1 -- 172.21.15.86:0/2118582191 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a1c0ab600 con 0x7f7a300b79c0 2024-04-25T14:08:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.578+0000 7f7a2ea00640 1 -- 172.21.15.86:0/2118582191 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c0ab9d0 con 0x7f7a300b79c0 2024-04-25T14:08:59.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.582+0000 7f7a36e00640 1 -- 172.21.15.86:0/2118582191 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a300b79c0 msgr2=0x7f7a300d2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:59.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.582+0000 7f7a36e00640 1 --2- 172.21.15.86:0/2118582191 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a300b79c0 0x7f7a300d2910 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c0097c0 tx=0x7f7a1c01c4a0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.582+0000 7f7a36e00640 1 -- 172.21.15.86:0/2118582191 shutdown_connections 2024-04-25T14:08:59.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.582+0000 7f7a36e00640 1 --2- 172.21.15.86:0/2118582191 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a300b79c0 0x7f7a300d2910 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.582+0000 7f7a36e00640 1 --2- 172.21.15.86:0/2118582191 >> v2:172.21.15.86:3300/0 conn(0x7f7a300ccab0 0x7f7a300d23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.582+0000 7f7a36e00640 1 -- 172.21.15.86:0/2118582191 >> 172.21.15.86:0/2118582191 conn(0x7f7a3009ba30 msgr2=0x7f7a300b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:59.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.582+0000 7f7a36e00640 1 -- 172.21.15.86:0/2118582191 shutdown_connections 2024-04-25T14:08:59.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.582+0000 7f7a36e00640 1 -- 172.21.15.86:0/2118582191 wait complete. 2024-04-25T14:08:59.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.582+0000 7f7a36e00640 1 Processor -- start 2024-04-25T14:08:59.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.582+0000 7f7a36e00640 1 -- start start 2024-04-25T14:08:59.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.586+0000 7f7a36e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a300b79c0 0x7f7a30159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.586+0000 7f7a36e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a300ccab0 0x7f7a300cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.586+0000 7f7a36e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a300cf9d0 0x7f7a301fcc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.586+0000 7f7a36e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3016ab00 con 0x7f7a300cf9d0 2024-04-25T14:08:59.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.586+0000 7f7a36e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3016a980 con 0x7f7a300ccab0 2024-04-25T14:08:59.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.586+0000 7f7a36e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7a3016ac80 con 0x7f7a300b79c0 2024-04-25T14:08:59.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.586+0000 7f7a34e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a300cf9d0 0x7f7a301fcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.586+0000 7f7a34e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a300cf9d0 0x7f7a301fcc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39054/0 (socket says 172.21.15.86:39054) 2024-04-25T14:08:59.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.586+0000 7f7a34e00640 1 -- 172.21.15.86:0/2003789545 learned_addr learned my addr 172.21.15.86:0/2003789545 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:08:59.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.586+0000 7f7a2fe00640 1 --2- 172.21.15.86:0/2003789545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a300b79c0 0x7f7a30159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.586+0000 7f7a2f400640 1 --2- 172.21.15.86:0/2003789545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a300ccab0 0x7f7a300cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.586+0000 7f7a34e00640 1 -- 172.21.15.86:0/2003789545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a300b79c0 msgr2=0x7f7a30159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:59.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.586+0000 7f7a34e00640 1 --2- 172.21.15.86:0/2003789545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a300b79c0 0x7f7a30159610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.586+0000 7f7a34e00640 1 -- 172.21.15.86:0/2003789545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a300ccab0 msgr2=0x7f7a300cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:59.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.586+0000 7f7a34e00640 1 --2- 172.21.15.86:0/2003789545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a300ccab0 0x7f7a300cc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.586+0000 7f7a34e00640 1 -- 172.21.15.86:0/2003789545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a30160a30 con 0x7f7a300cf9d0 2024-04-25T14:08:59.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.586+0000 7f7a2fe00640 1 --2- 172.21.15.86:0/2003789545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a300b79c0 0x7f7a30159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:08:59.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.586+0000 7f7a2f400640 1 --2- 172.21.15.86:0/2003789545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a300ccab0 0x7f7a300cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:08:59.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.586+0000 7f7a34e00640 1 --2- 172.21.15.86:0/2003789545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a300cf9d0 0x7f7a301fcc40 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800ca50 tx=0x7f7a2800cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:59.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.586+0000 7f7a27e00640 1 -- 172.21.15.86:0/2003789545 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a28017070 con 0x7f7a300cf9d0 2024-04-25T14:08:59.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.586+0000 7f7a27e00640 1 -- 172.21.15.86:0/2003789545 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a2800abb0 con 0x7f7a300cf9d0 2024-04-25T14:08:59.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.586+0000 7f7a27e00640 1 -- 172.21.15.86:0/2003789545 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a28012400 con 0x7f7a300cf9d0 2024-04-25T14:08:59.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.586+0000 7f7a36e00640 1 -- 172.21.15.86:0/2003789545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a30160d10 con 0x7f7a300cf9d0 2024-04-25T14:08:59.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.586+0000 7f7a36e00640 1 -- 172.21.15.86:0/2003789545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a301fd920 con 0x7f7a300cf9d0 2024-04-25T14:08:59.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.586+0000 7f7a36e00640 1 -- 172.21.15.86:0/2003789545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a300a04c0 con 0x7f7a300cf9d0 2024-04-25T14:08:59.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.586+0000 7f7a27e00640 1 -- 172.21.15.86:0/2003789545 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 29) ==== 93318+0+0 (secure 0 0 0) 0x7f7a2801a070 con 0x7f7a300cf9d0 2024-04-25T14:08:59.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.590+0000 7f7a27e00640 1 --2- 172.21.15.86:0/2003789545 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f7a1006f570 0x7f7a10071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.590+0000 7f7a2fe00640 1 -- 172.21.15.86:0/2003789545 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f7a1006f570 msgr2=0x7f7a10071a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/3589556609 2024-04-25T14:08:59.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.590+0000 7f7a27e00640 1 -- 172.21.15.86:0/2003789545 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f7a2808f290 con 0x7f7a300cf9d0 2024-04-25T14:08:59.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.590+0000 7f7a2fe00640 1 --2- 172.21.15.86:0/2003789545 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f7a1006f570 0x7f7a10071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:08:59.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.594+0000 7f7a27e00640 1 -- 172.21.15.86:0/2003789545 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7a2805b7f0 con 0x7f7a300cf9d0 2024-04-25T14:08:59.646 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:08:59.648+0000 7f7ee6453180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:08:59.664 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:08:59.662+0000 7fb82707d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:08:59.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.778+0000 7f7a36e00640 1 -- 172.21.15.86:0/2003789545 --> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] -- mgr_command(tid 0: {"prefix": "fs perf stats", "client_id": "abcd", "target": ["mon-mgr", ""]}) -- 0x7f7a300d2f90 con 0x7f7a1006f570 2024-04-25T14:08:59.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.794+0000 7f7a2fe00640 1 -- 172.21.15.86:0/2003789545 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f7a1006f570 msgr2=0x7f7a10071a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/3589556609 2024-04-25T14:08:59.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:08:59.794+0000 7f7a2fe00640 1 --2- 172.21.15.86:0/2003789545 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f7a1006f570 0x7f7a10071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:09:00.062 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:00.064+0000 7f7ee6453180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:09:00.086 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:00.086+0000 7fb82707d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:09:00.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:00.198+0000 7f7a2fe00640 1 -- 172.21.15.86:0/2003789545 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f7a1006f570 msgr2=0x7f7a10071a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/3589556609 2024-04-25T14:09:00.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:00.198+0000 7f7a2fe00640 1 --2- 172.21.15.86:0/2003789545 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f7a1006f570 0x7f7a10071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T14:09:00.212 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:00.212+0000 7f7ee6453180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:09:00.236 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:00.234+0000 7fb82707d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:09:00.682 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:00.684+0000 7f7ee6453180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:09:00.709 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:00.706+0000 7fb82707d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:09:00.744 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:00.744+0000 7f7ee6453180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:09:00.770 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:00.770+0000 7fb82707d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:09:00.801 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:00.800+0000 7f7ee6453180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:09:00.827 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:00.826+0000 7fb82707d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:09:00.891 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:00.892+0000 7f7ee6453180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:09:00.916 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:00.914+0000 7fb82707d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:09:00.948 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:00.948+0000 7f7ee6453180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:09:00.973 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:00.970+0000 7fb82707d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:09:01.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:01.002+0000 7f7a2fe00640 1 -- 172.21.15.86:0/2003789545 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f7a1006f570 msgr2=0x7f7a10071a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/3589556609 2024-04-25T14:09:01.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:01.002+0000 7f7a2fe00640 1 --2- 172.21.15.86:0/2003789545 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f7a1006f570 0x7f7a10071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T14:09:01.032 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:01.032+0000 7f7ee6453180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:09:01.058 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:01.058+0000 7fb82707d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:09:01.091 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:01.092+0000 7f7ee6453180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:09:01.117 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:01.114+0000 7fb82707d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:09:01.243 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:01.244+0000 7f7ee6453180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:09:01.270 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:01.270+0000 7fb82707d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:09:01.301 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:01.304+0000 7f7ee6453180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:09:01.329 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:01.326+0000 7fb82707d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:09:01.540 INFO:tasks.ceph.mgr.x.smithi155.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:09:01.540 INFO:tasks.ceph.mgr.x.smithi155.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:09:01.540 INFO:tasks.ceph.mgr.x.smithi155.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:09:01.548 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:01.548+0000 7f7ee6453180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:09:01.569 INFO:tasks.ceph.mgr.y.smithi086.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:09:01.569 INFO:tasks.ceph.mgr.y.smithi086.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:09:01.569 INFO:tasks.ceph.mgr.y.smithi086.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:09:01.577 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:01.574+0000 7fb82707d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:09:01.696 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:01.696+0000 7f7ee6453180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:09:01.728 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:01.726+0000 7fb82707d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:09:01.826 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:01.828+0000 7f7ee6453180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:09:01.858 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:01.858+0000 7fb82707d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:09:01.897 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:01.896+0000 7f7ee6453180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:09:01.930 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:01.930+0000 7fb82707d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:09:02.045 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:02.044+0000 7f7ee6453180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:09:02.079 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:02.078+0000 7fb82707d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:09:02.587 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:02.588+0000 7f7ee6453180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:09:02.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:02.606+0000 7f7a2fe00640 1 -- 172.21.15.86:0/2003789545 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f7a1006f570 msgr2=0x7f7a10071a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/3589556609 2024-04-25T14:09:02.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:02.606+0000 7f7a2fe00640 1 --2- 172.21.15.86:0/2003789545 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f7a1006f570 0x7f7a10071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T14:09:02.624 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:02.622+0000 7fb82707d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:09:02.670 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:02.672+0000 7f7ee6453180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:09:02.709 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:02.706+0000 7fb82707d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:09:02.830 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:02.832+0000 7f7ee6453180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:09:02.872 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:02.870+0000 7fb82707d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:09:02.915 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:02.916+0000 7f7ee6453180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:09:02.956 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:02.954+0000 7fb82707d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:09:03.234 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:03.236+0000 7f7ee6453180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:09:03.276 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:03.274+0000 7fb82707d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:09:03.296 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:03.296+0000 7f7ee6453180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:09:03.339 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:03.338+0000 7fb82707d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:09:03.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:03.346+0000 7f7a27e00640 1 -- 172.21.15.86:0/2003789545 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 30) ==== 92927+0+0 (secure 0 0 0) 0x7f7a28057a80 con 0x7f7a300cf9d0 2024-04-25T14:09:03.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:03.350+0000 7f7a27e00640 1 -- 172.21.15.86:0/2003789545 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f7a1006f570 msgr2=0x7f7a10071a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:09:03.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:03.350+0000 7f7a27e00640 1 --2- 172.21.15.86:0/2003789545 >> [v2:172.21.15.86:6832/3589556609,v1:172.21.15.86:6833/3589556609] conn(0x7f7a1006f570 0x7f7a10071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:03.417 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:03.414+0000 7fb7a3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:03.417 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:03.414+0000 7fb7a3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:03.417 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:03.414+0000 7fb7a3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:03.417 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:03.414+0000 7fb7a3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:03.417 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:03.414+0000 7fb7a3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:03.417 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:03.414+0000 7fb7a7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:03.417 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:03.414+0000 7fb7a7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:03.417 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:03.414+0000 7fb7a7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:03.418 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:03.414+0000 7fb7a7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:03.418 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:03.414+0000 7fb7a7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:04.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.366+0000 7f7a27e00640 1 -- 172.21.15.86:0/2003789545 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mgrmap(e 31) ==== 93054+0+0 (secure 0 0 0) 0x7f7a2805ba50 con 0x7f7a300cf9d0 2024-04-25T14:09:04.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.370+0000 7f7a27e00640 1 --2- 172.21.15.86:0/2003789545 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f7a10077ac0 0x7f7a10079ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:04.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.370+0000 7f7a27e00640 1 -- 172.21.15.86:0/2003789545 --> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] -- mgr_command(tid 0: {"prefix": "fs perf stats", "client_id": "abcd", "target": ["mon-mgr", ""]}) -- 0x7f7a300d2f90 con 0x7f7a10077ac0 2024-04-25T14:09:04.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.370+0000 7f7a2fe00640 1 --2- 172.21.15.86:0/2003789545 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f7a10077ac0 0x7f7a10079ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:04.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.370+0000 7f7a2fe00640 1 --2- 172.21.15.86:0/2003789545 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f7a10077ac0 0x7f7a10079ee0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7a20009a70 tx=0x7f7a20006f90 comp rx=0 tx=0).ready entity=mgr.7587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:04.374 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:04.374+0000 7fb7b2c00640 -1 mgr.server reply reply (22) Invalid argument invalid client_id filter spec: abcd 2024-04-25T14:09:04.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.374+0000 7f7a27e00640 1 -- 172.21.15.86:0/2003789545 <== mgr.7587 v2:172.21.15.86:6832/132942210 1 ==== mgr_command_reply(tid 0: -22 invalid client_id filter spec: abcd) ==== 43+0+0 (secure 0 0 0) 0x7f7a300d2f90 con 0x7f7a10077ac0 2024-04-25T14:09:04.376 INFO:teuthology.orchestra.run.smithi086.stderr:Error EINVAL: invalid client_id filter spec: abcd 2024-04-25T14:09:04.389 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T14:09:04.389 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable stats 2024-04-25T14:09:04.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.542+0000 7f8247800640 1 Processor -- start 2024-04-25T14:09:04.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.542+0000 7f8247800640 1 -- start start 2024-04-25T14:09:04.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.542+0000 7f8247800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8240070330 0x7f82400d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:04.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.542+0000 7f8247800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f82400d1c20 0x7f824006ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:04.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.542+0000 7f8247800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82400d2030 con 0x7f824006f270 2024-04-25T14:09:04.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.542+0000 7f8247800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f824015e5c0 con 0x7f82400d1c20 2024-04-25T14:09:04.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.542+0000 7f8247800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f824015e8a0 con 0x7f8240070330 2024-04-25T14:09:04.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.542+0000 7f823fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f82400d1c20 0x7f824006ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:04.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.542+0000 7f8245800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f824006f270 0x7f824006f680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:46148/0 (socket says 172.21.15.86:46148) 2024-04-25T14:09:04.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.542+0000 7f8245800640 1 -- 172.21.15.86:0/1968729166 learned_addr learned my addr 172.21.15.86:0/1968729166 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:04.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.542+0000 7f8244e00640 1 --2- 172.21.15.86:0/1968729166 >> v2:172.21.15.155:3300/0 conn(0x7f8240070330 0x7f82400d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:04.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.542+0000 7f823fe00640 1 -- 172.21.15.86:0/1968729166 >> v2:172.21.15.155:3300/0 conn(0x7f8240070330 msgr2=0x7f82400d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:04.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.542+0000 7f823fe00640 1 --2- 172.21.15.86:0/1968729166 >> v2:172.21.15.155:3300/0 conn(0x7f8240070330 0x7f82400d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:04.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.542+0000 7f823fe00640 1 -- 172.21.15.86:0/1968729166 >> v1:172.21.15.86:6789/0 conn(0x7f824006f270 legacy=0x7f824006f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:04.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.542+0000 7f823fe00640 1 -- 172.21.15.86:0/1968729166 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824015eb80 con 0x7f82400d1c20 2024-04-25T14:09:04.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.546+0000 7f823fe00640 1 --2- 172.21.15.86:0/1968729166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82400d1c20 0x7f824006ed30 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f822c0097c0 tx=0x7f822c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ec6fb85d867da8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:04.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.546+0000 7f823f400640 1 -- 172.21.15.86:0/1968729166 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c0b0070 con 0x7f82400d1c20 2024-04-25T14:09:04.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.546+0000 7f823f400640 1 -- 172.21.15.86:0/1968729166 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f822c0ab600 con 0x7f82400d1c20 2024-04-25T14:09:04.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.546+0000 7f823f400640 1 -- 172.21.15.86:0/1968729166 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c0a4e60 con 0x7f82400d1c20 2024-04-25T14:09:04.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.546+0000 7f8247800640 1 -- 172.21.15.86:0/1968729166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82400d1c20 msgr2=0x7f824006ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:04.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.546+0000 7f8247800640 1 --2- 172.21.15.86:0/1968729166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82400d1c20 0x7f824006ed30 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f822c0097c0 tx=0x7f822c01c3d0 comp rx=0 tx=0).stop 2024-04-25T14:09:04.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.546+0000 7f8247800640 1 -- 172.21.15.86:0/1968729166 shutdown_connections 2024-04-25T14:09:04.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.546+0000 7f8247800640 1 --2- 172.21.15.86:0/1968729166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82400d1c20 0x7f824006ed30 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:04.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.546+0000 7f8247800640 1 --2- 172.21.15.86:0/1968729166 >> v2:172.21.15.155:3300/0 conn(0x7f8240070330 0x7f82400d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:04.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.546+0000 7f8247800640 1 -- 172.21.15.86:0/1968729166 >> 172.21.15.86:0/1968729166 conn(0x7f82400b4650 msgr2=0x7f82400b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:04.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.546+0000 7f8247800640 1 -- 172.21.15.86:0/1968729166 shutdown_connections 2024-04-25T14:09:04.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.546+0000 7f8247800640 1 -- 172.21.15.86:0/1968729166 wait complete. 2024-04-25T14:09:04.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.550+0000 7f8247800640 1 Processor -- start 2024-04-25T14:09:04.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.550+0000 7f8247800640 1 -- start start 2024-04-25T14:09:04.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.550+0000 7f8247800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f824006f270 0x7f824015b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:04.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.550+0000 7f8247800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8240070330 0x7f82400cf910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:04.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.550+0000 7f8247800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82400d1c20 0x7f82400bfb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:04.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.550+0000 7f8247800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82400708c0 con 0x7f8240070330 2024-04-25T14:09:04.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.550+0000 7f8247800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8240070740 con 0x7f82400d1c20 2024-04-25T14:09:04.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.550+0000 7f8247800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f824016afc0 con 0x7f824006f270 2024-04-25T14:09:04.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.550+0000 7f823fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8240070330 0x7f82400cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:04.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.550+0000 7f823fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8240070330 0x7f82400cf910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39182/0 (socket says 172.21.15.86:39182) 2024-04-25T14:09:04.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.550+0000 7f823fe00640 1 -- 172.21.15.86:0/2698435880 learned_addr learned my addr 172.21.15.86:0/2698435880 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:04.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.550+0000 7f8244e00640 1 --2- 172.21.15.86:0/2698435880 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f824006f270 0x7f824015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:04.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.550+0000 7f8245800640 1 --2- 172.21.15.86:0/2698435880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82400d1c20 0x7f82400bfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:04.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.554+0000 7f823fe00640 1 -- 172.21.15.86:0/2698435880 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f824006f270 msgr2=0x7f824015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:04.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.554+0000 7f823fe00640 1 --2- 172.21.15.86:0/2698435880 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f824006f270 0x7f824015b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:04.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.554+0000 7f823fe00640 1 -- 172.21.15.86:0/2698435880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82400d1c20 msgr2=0x7f82400bfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:04.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.554+0000 7f823fe00640 1 --2- 172.21.15.86:0/2698435880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82400d1c20 0x7f82400bfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:04.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.554+0000 7f823fe00640 1 -- 172.21.15.86:0/2698435880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824015b740 con 0x7f8240070330 2024-04-25T14:09:04.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.554+0000 7f8245800640 1 --2- 172.21.15.86:0/2698435880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82400d1c20 0x7f82400bfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:04.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.554+0000 7f823fe00640 1 --2- 172.21.15.86:0/2698435880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8240070330 0x7f82400cf910 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f822c018fe0 tx=0x7f822c019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:04.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.554+0000 7f823cc00640 1 -- 172.21.15.86:0/2698435880 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c0ba070 con 0x7f8240070330 2024-04-25T14:09:04.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.554+0000 7f823cc00640 1 -- 172.21.15.86:0/2698435880 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f822c0a4e60 con 0x7f8240070330 2024-04-25T14:09:04.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.554+0000 7f8247800640 1 -- 172.21.15.86:0/2698435880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82400b1080 con 0x7f8240070330 2024-04-25T14:09:04.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.554+0000 7f823cc00640 1 -- 172.21.15.86:0/2698435880 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c0b0040 con 0x7f8240070330 2024-04-25T14:09:04.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.554+0000 7f8247800640 1 -- 172.21.15.86:0/2698435880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82400b1640 con 0x7f8240070330 2024-04-25T14:09:04.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.554+0000 7f8247800640 1 -- 172.21.15.86:0/2698435880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8240069d00 con 0x7f8240070330 2024-04-25T14:09:04.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.554+0000 7f823cc00640 1 -- 172.21.15.86:0/2698435880 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 31) ==== 93054+0+0 (secure 0 0 0) 0x7f822c0bd070 con 0x7f8240070330 2024-04-25T14:09:04.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.554+0000 7f823cc00640 1 --2- 172.21.15.86:0/2698435880 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f822006f2a0 0x7f82200717b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:04.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.558+0000 7f823cc00640 1 -- 172.21.15.86:0/2698435880 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6445+0+0 (secure 0 0 0) 0x7f822c129880 con 0x7f8240070330 2024-04-25T14:09:04.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.558+0000 7f8244e00640 1 --2- 172.21.15.86:0/2698435880 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f822006f2a0 0x7f82200717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:04.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.558+0000 7f8244e00640 1 --2- 172.21.15.86:0/2698435880 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f822006f2a0 0x7f82200717b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8240072200 tx=0x7f8230008040 comp rx=0 tx=0).ready entity=mgr.7587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:04.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.562+0000 7f823cc00640 1 -- 172.21.15.86:0/2698435880 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f822c016050 con 0x7f8240070330 2024-04-25T14:09:04.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:04.758+0000 7f8247800640 1 -- 172.21.15.86:0/2698435880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "stats"} v 0) -- 0x7f8240076bf0 con 0x7f8240070330 2024-04-25T14:09:05.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.370+0000 7f823cc00640 1 -- 172.21.15.86:0/2698435880 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "stats"}]=0 v32) ==== 85+0+0 (secure 0 0 0) 0x7f822c0f6140 con 0x7f8240070330 2024-04-25T14:09:05.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.374+0000 7f8247800640 1 -- 172.21.15.86:0/2698435880 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f822006f2a0 msgr2=0x7f82200717b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:05.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.374+0000 7f8247800640 1 --2- 172.21.15.86:0/2698435880 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f822006f2a0 0x7f82200717b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8240072200 tx=0x7f8230008040 comp rx=0 tx=0).stop 2024-04-25T14:09:05.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.374+0000 7f8247800640 1 -- 172.21.15.86:0/2698435880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8240070330 msgr2=0x7f82400cf910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:05.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.374+0000 7f8247800640 1 --2- 172.21.15.86:0/2698435880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8240070330 0x7f82400cf910 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f822c018fe0 tx=0x7f822c019170 comp rx=0 tx=0).stop 2024-04-25T14:09:05.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.374+0000 7f8247800640 1 -- 172.21.15.86:0/2698435880 shutdown_connections 2024-04-25T14:09:05.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.374+0000 7f8247800640 1 --2- 172.21.15.86:0/2698435880 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f822006f2a0 0x7f82200717b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:05.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.374+0000 7f8247800640 1 --2- 172.21.15.86:0/2698435880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82400d1c20 0x7f82400bfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:05.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.374+0000 7f8247800640 1 --2- 172.21.15.86:0/2698435880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8240070330 0x7f82400cf910 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:05.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.374+0000 7f8247800640 1 --2- 172.21.15.86:0/2698435880 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f824006f270 0x7f824015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:05.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.374+0000 7f8247800640 1 -- 172.21.15.86:0/2698435880 >> 172.21.15.86:0/2698435880 conn(0x7f82400b4650 msgr2=0x7f82400c49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:05.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.374+0000 7f8247800640 1 -- 172.21.15.86:0/2698435880 shutdown_connections 2024-04-25T14:09:05.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.374+0000 7f8247800640 1 -- 172.21.15.86:0/2698435880 wait complete. 2024-04-25T14:09:05.399 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T14:09:05.399 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:09:05.504 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:09:05.596 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T14:09:05.596 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:09:05.596 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:09:05.596 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:09:05.605 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:09:05.605 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:09:05.630 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:05.632+0000 7fce4717e180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:09:05.633 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:05.630+0000 7f89ff72a180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:09:05.642 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:09:05.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.834+0000 7f8e09000640 1 Processor -- start 2024-04-25T14:09:05.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.834+0000 7f8e09000640 1 -- start start 2024-04-25T14:09:05.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.834+0000 7f8e09000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8e040cde40 0x7f8e040ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:05.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.834+0000 7f8e09000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e040b3d40 con 0x7f8e040b3930 2024-04-25T14:09:05.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.834+0000 7f8e09000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e040d2380 con 0x7f8e04167d20 2024-04-25T14:09:05.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.834+0000 7f8e09000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8e040d2660 con 0x7f8e040cde40 2024-04-25T14:09:05.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.834+0000 7f8e01400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8e040cde40 0x7f8e040ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:05.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.834+0000 7f8e01e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8e040b3930 0x7f8e040ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:46166/0 (socket says 172.21.15.86:46166) 2024-04-25T14:09:05.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.834+0000 7f8e01400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8e040cde40 0x7f8e040ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39202/0 (socket says 172.21.15.86:39202) 2024-04-25T14:09:05.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.834+0000 7f8e01e00640 1 -- 172.21.15.86:0/1350476458 learned_addr learned my addr 172.21.15.86:0/1350476458 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:05.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.834+0000 7f8e01400640 1 -- 172.21.15.86:0/1350476458 >> v1:172.21.15.155:6789/0 conn(0x7f8e04167d20 legacy=0x7f8e0416a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:05.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.834+0000 7f8e01400640 1 -- 172.21.15.86:0/1350476458 >> v1:172.21.15.86:6789/0 conn(0x7f8e040b3930 legacy=0x7f8e040ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:05.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.834+0000 7f8e01400640 1 -- 172.21.15.86:0/1350476458 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e040d2940 con 0x7f8e040cde40 2024-04-25T14:09:05.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.838+0000 7f8e01400640 1 --2- 172.21.15.86:0/1350476458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e040cde40 0x7f8e040ceec0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f8df00097c0 tx=0x7f8df001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7945cdf94695fb43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:05.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.838+0000 7f8e00a00640 1 -- 172.21.15.86:0/1350476458 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df00b0070 con 0x7f8e040cde40 2024-04-25T14:09:05.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.838+0000 7f8e00a00640 1 -- 172.21.15.86:0/1350476458 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8df00ab600 con 0x7f8e040cde40 2024-04-25T14:09:05.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.838+0000 7f8e00a00640 1 -- 172.21.15.86:0/1350476458 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df00ab9d0 con 0x7f8e040cde40 2024-04-25T14:09:05.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.838+0000 7f8e09000640 1 -- 172.21.15.86:0/1350476458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e040cde40 msgr2=0x7f8e040ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:05.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.838+0000 7f8e09000640 1 --2- 172.21.15.86:0/1350476458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e040cde40 0x7f8e040ceec0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f8df00097c0 tx=0x7f8df001c410 comp rx=0 tx=0).stop 2024-04-25T14:09:05.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.838+0000 7f8e09000640 1 -- 172.21.15.86:0/1350476458 shutdown_connections 2024-04-25T14:09:05.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.838+0000 7f8e09000640 1 --2- 172.21.15.86:0/1350476458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e040cde40 0x7f8e040ceec0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:05.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.838+0000 7f8e09000640 1 -- 172.21.15.86:0/1350476458 >> 172.21.15.86:0/1350476458 conn(0x7f8e0409ba30 msgr2=0x7f8e040b72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:05.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.838+0000 7f8e09000640 1 -- 172.21.15.86:0/1350476458 shutdown_connections 2024-04-25T14:09:05.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.838+0000 7f8e09000640 1 -- 172.21.15.86:0/1350476458 wait complete. 2024-04-25T14:09:05.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.842+0000 7f8e09000640 1 Processor -- start 2024-04-25T14:09:05.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.842+0000 7f8e09000640 1 -- start start 2024-04-25T14:09:05.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.842+0000 7f8e09000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e040b3930 0x7f8e0406c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:05.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.842+0000 7f8e09000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e040cde40 0x7f8e0406ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:05.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.842+0000 7f8e09000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e04167d20 0x7f8e04139ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:05.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.842+0000 7f8e09000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0416e610 con 0x7f8e04167d20 2024-04-25T14:09:05.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.842+0000 7f8e09000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0416e490 con 0x7f8e040cde40 2024-04-25T14:09:05.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.842+0000 7f8e09000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8e0416e790 con 0x7f8e040b3930 2024-04-25T14:09:05.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.842+0000 7f8e02800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e04167d20 0x7f8e04139ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:05.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.846+0000 7f8e02800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e04167d20 0x7f8e04139ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39208/0 (socket says 172.21.15.86:39208) 2024-04-25T14:09:05.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.846+0000 7f8e01400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e040cde40 0x7f8e0406ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:05.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.846+0000 7f8e01e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e040b3930 0x7f8e0406c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:05.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.846+0000 7f8e02800640 1 -- 172.21.15.86:0/1782019364 learned_addr learned my addr 172.21.15.86:0/1782019364 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:05.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.846+0000 7f8e02800640 1 -- 172.21.15.86:0/1782019364 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e040b3930 msgr2=0x7f8e0406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:05.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.846+0000 7f8e02800640 1 --2- 172.21.15.86:0/1782019364 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e040b3930 0x7f8e0406c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:05.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.846+0000 7f8e02800640 1 -- 172.21.15.86:0/1782019364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e040cde40 msgr2=0x7f8e0406ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:05.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.846+0000 7f8e02800640 1 --2- 172.21.15.86:0/1782019364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e040cde40 0x7f8e0406ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:05.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.846+0000 7f8e02800640 1 -- 172.21.15.86:0/1782019364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0413acb0 con 0x7f8e04167d20 2024-04-25T14:09:05.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.846+0000 7f8e02800640 1 --2- 172.21.15.86:0/1782019364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e04167d20 0x7f8e04139ea0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc00c760 tx=0x7f8dfc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:05.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.846+0000 7f8dfa000640 1 -- 172.21.15.86:0/1782019364 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc003b80 con 0x7f8e04167d20 2024-04-25T14:09:05.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.846+0000 7f8dfa000640 1 -- 172.21.15.86:0/1782019364 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dfc003d20 con 0x7f8e04167d20 2024-04-25T14:09:05.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.846+0000 7f8dfa000640 1 -- 172.21.15.86:0/1782019364 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc00aa90 con 0x7f8e04167d20 2024-04-25T14:09:05.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.846+0000 7f8e09000640 1 -- 172.21.15.86:0/1782019364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e04153270 con 0x7f8e04167d20 2024-04-25T14:09:05.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.846+0000 7f8e09000640 1 -- 172.21.15.86:0/1782019364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e0406d620 con 0x7f8e04167d20 2024-04-25T14:09:05.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.846+0000 7f8e09000640 1 -- 172.21.15.86:0/1782019364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e04152fe0 con 0x7f8e04167d20 2024-04-25T14:09:05.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.846+0000 7f8dfa000640 1 -- 172.21.15.86:0/1782019364 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 32) ==== 93253+0+0 (secure 0 0 0) 0x7f8dfc00ac30 con 0x7f8e04167d20 2024-04-25T14:09:05.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.850+0000 7f8dfa000640 1 --2- 172.21.15.86:0/1782019364 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f8de406f490 0x7f8de40719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:05.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.850+0000 7f8e01e00640 1 -- 172.21.15.86:0/1782019364 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f8de406f490 msgr2=0x7f8de40719a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/132942210 2024-04-25T14:09:05.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.850+0000 7f8dfa000640 1 -- 172.21.15.86:0/1782019364 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6445+0+0 (secure 0 0 0) 0x7f8dfc08f3e0 con 0x7f8e04167d20 2024-04-25T14:09:05.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.850+0000 7f8e01e00640 1 --2- 172.21.15.86:0/1782019364 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f8de406f490 0x7f8de40719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:09:05.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:05.854+0000 7f8dfa000640 1 -- 172.21.15.86:0/1782019364 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8dfc05b8b0 con 0x7f8e04167d20 2024-04-25T14:09:06.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.014+0000 7f8e09000640 1 -- 172.21.15.86:0/1782019364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8e040a05a0 con 0x7f8e04167d20 2024-04-25T14:09:06.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.014+0000 7f8dfa000640 1 -- 172.21.15.86:0/1782019364 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+17386 (secure 0 0 0) 0x7f8dfc05fbb0 con 0x7f8e04167d20 2024-04-25T14:09:06.017 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:06.017 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":88,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:09:03.346244+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:08:43.505988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T14:08:44.054628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/3132647616":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/1564187566":"2024-04-26T14:09:03.346201+0000","172.21.15.86:6833/3589556609":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/1099362620":"2024-04-26T14:09:03.346201+0000","172.21.15.86:6832/3589556609":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/3481243210":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/2509210906":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/3722630918":"2024-04-26T14:09:03.346201+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:09:06.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.018+0000 7f8e09000640 1 -- 172.21.15.86:0/1782019364 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f8de406f490 msgr2=0x7f8de40719a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:09:06.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.018+0000 7f8e09000640 1 --2- 172.21.15.86:0/1782019364 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f8de406f490 0x7f8de40719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:06.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.018+0000 7f8e09000640 1 -- 172.21.15.86:0/1782019364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e04167d20 msgr2=0x7f8e04139ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:06.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.018+0000 7f8e09000640 1 --2- 172.21.15.86:0/1782019364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e04167d20 0x7f8e04139ea0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc00c760 tx=0x7f8dfc00cc20 comp rx=0 tx=0).stop 2024-04-25T14:09:06.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.018+0000 7f8e09000640 1 -- 172.21.15.86:0/1782019364 shutdown_connections 2024-04-25T14:09:06.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.018+0000 7f8e09000640 1 --2- 172.21.15.86:0/1782019364 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f8de406f490 0x7f8de40719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:06.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.018+0000 7f8e09000640 1 --2- 172.21.15.86:0/1782019364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e04167d20 0x7f8e04139ea0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:06.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.018+0000 7f8e09000640 1 --2- 172.21.15.86:0/1782019364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e040cde40 0x7f8e0406ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:06.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.018+0000 7f8e09000640 1 --2- 172.21.15.86:0/1782019364 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e040b3930 0x7f8e0406c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:06.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.018+0000 7f8e09000640 1 -- 172.21.15.86:0/1782019364 >> 172.21.15.86:0/1782019364 conn(0x7f8e0409ba30 msgr2=0x7f8e041618b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:06.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.018+0000 7f8e09000640 1 -- 172.21.15.86:0/1782019364 shutdown_connections 2024-04-25T14:09:06.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.018+0000 7f8e09000640 1 -- 172.21.15.86:0/1782019364 wait complete. 2024-04-25T14:09:06.040 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:08:43.505988+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:08:44.054628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:09:06.040 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:09:06.048 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:06.048+0000 7fce4717e180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:09:06.070 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:06.070+0000 7f89ff72a180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:09:06.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.190+0000 7f2d64200640 1 Processor -- start 2024-04-25T14:09:06.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.190+0000 7f2d64200640 1 -- start start 2024-04-25T14:09:06.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.194+0000 7f2d64200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2d5c06c670 0x7f2d5c15de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:06.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.194+0000 7f2d64200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d5c161b40 con 0x7f2d5c0cef20 2024-04-25T14:09:06.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.194+0000 7f2d64200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d5c161e20 con 0x7f2d5c0cdf50 2024-04-25T14:09:06.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.194+0000 7f2d64200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f2d5c15e380 con 0x7f2d5c06c670 2024-04-25T14:09:06.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.194+0000 7f2d62200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2d5c0cef20 0x7f2d5c0cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:46180/0 (socket says 172.21.15.86:46180) 2024-04-25T14:09:06.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.194+0000 7f2d62200640 1 -- 172.21.15.86:0/2564408929 learned_addr learned my addr 172.21.15.86:0/2564408929 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:06.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.194+0000 7f2d61800640 1 --2- 172.21.15.86:0/2564408929 >> v2:172.21.15.155:3300/0 conn(0x7f2d5c06c670 0x7f2d5c15de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:06.197 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:06.196+0000 7fce4717e180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:09:06.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.194+0000 7f2d5be00640 1 -- 172.21.15.86:0/2564408929 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 344036395 0 0) 0x7f2d5c161b40 con 0x7f2d5c0cef20 2024-04-25T14:09:06.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.194+0000 7f2d5be00640 1 -- 172.21.15.86:0/2564408929 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d44003530 con 0x7f2d5c0cef20 2024-04-25T14:09:06.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.194+0000 7f2d61800640 1 -- 172.21.15.86:0/2564408929 >> v1:172.21.15.155:6789/0 conn(0x7f2d5c0cdf50 legacy=0x7f2d5c0ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:06.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.194+0000 7f2d61800640 1 -- 172.21.15.86:0/2564408929 >> v1:172.21.15.86:6789/0 conn(0x7f2d5c0cef20 legacy=0x7f2d5c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:06.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.194+0000 7f2d61800640 1 -- 172.21.15.86:0/2564408929 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d5c162100 con 0x7f2d5c06c670 2024-04-25T14:09:06.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.198+0000 7f2d61800640 1 --2- 172.21.15.86:0/2564408929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d5c06c670 0x7f2d5c15de40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2d500097c0 tx=0x7f2d5001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1b1c576557281cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:06.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.198+0000 7f2d5be00640 1 -- 172.21.15.86:0/2564408929 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d500b0070 con 0x7f2d5c06c670 2024-04-25T14:09:06.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.198+0000 7f2d5be00640 1 -- 172.21.15.86:0/2564408929 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d500ab600 con 0x7f2d5c06c670 2024-04-25T14:09:06.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.198+0000 7f2d5be00640 1 -- 172.21.15.86:0/2564408929 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d500ab9b0 con 0x7f2d5c06c670 2024-04-25T14:09:06.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.198+0000 7f2d64200640 1 -- 172.21.15.86:0/2564408929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d5c06c670 msgr2=0x7f2d5c15de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:06.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.198+0000 7f2d64200640 1 --2- 172.21.15.86:0/2564408929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d5c06c670 0x7f2d5c15de40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2d500097c0 tx=0x7f2d5001c410 comp rx=0 tx=0).stop 2024-04-25T14:09:06.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.198+0000 7f2d64200640 1 -- 172.21.15.86:0/2564408929 shutdown_connections 2024-04-25T14:09:06.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.198+0000 7f2d64200640 1 --2- 172.21.15.86:0/2564408929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d5c06c670 0x7f2d5c15de40 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:06.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.198+0000 7f2d64200640 1 -- 172.21.15.86:0/2564408929 >> 172.21.15.86:0/2564408929 conn(0x7f2d5c09ba30 msgr2=0x7f2d5c0b73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:06.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.198+0000 7f2d64200640 1 -- 172.21.15.86:0/2564408929 shutdown_connections 2024-04-25T14:09:06.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.198+0000 7f2d64200640 1 -- 172.21.15.86:0/2564408929 wait complete. 2024-04-25T14:09:06.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.202+0000 7f2d64200640 1 Processor -- start 2024-04-25T14:09:06.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.202+0000 7f2d64200640 1 -- start start 2024-04-25T14:09:06.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.202+0000 7f2d64200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d5c06c670 0x7f2d5c0ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:06.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.202+0000 7f2d64200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d5c0cdf50 0x7f2d5c152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:06.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.202+0000 7f2d64200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d5c0cef20 0x7f2d5c0cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:06.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.202+0000 7f2d64200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5c162750 con 0x7f2d5c0cdf50 2024-04-25T14:09:06.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.202+0000 7f2d64200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5c0b80f0 con 0x7f2d5c0cef20 2024-04-25T14:09:06.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.202+0000 7f2d64200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2d5c16c790 con 0x7f2d5c06c670 2024-04-25T14:09:06.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.202+0000 7f2d60e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d5c0cdf50 0x7f2d5c152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:06.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.202+0000 7f2d60e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d5c0cdf50 0x7f2d5c152520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39210/0 (socket says 172.21.15.86:39210) 2024-04-25T14:09:06.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.202+0000 7f2d60e00640 1 -- 172.21.15.86:0/2130352119 learned_addr learned my addr 172.21.15.86:0/2130352119 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:06.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.202+0000 7f2d62200640 1 --2- 172.21.15.86:0/2130352119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d5c0cef20 0x7f2d5c0cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:06.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.202+0000 7f2d61800640 1 --2- 172.21.15.86:0/2130352119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d5c06c670 0x7f2d5c0ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:06.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.202+0000 7f2d60e00640 1 -- 172.21.15.86:0/2130352119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d5c06c670 msgr2=0x7f2d5c0ca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:06.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.202+0000 7f2d60e00640 1 --2- 172.21.15.86:0/2130352119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d5c06c670 0x7f2d5c0ca5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:06.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.202+0000 7f2d60e00640 1 -- 172.21.15.86:0/2130352119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d5c0cef20 msgr2=0x7f2d5c0cbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:06.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.202+0000 7f2d60e00640 1 --2- 172.21.15.86:0/2130352119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d5c0cef20 0x7f2d5c0cbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:06.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.202+0000 7f2d60e00640 1 -- 172.21.15.86:0/2130352119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d5c157e10 con 0x7f2d5c0cdf50 2024-04-25T14:09:06.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.202+0000 7f2d62200640 1 --2- 172.21.15.86:0/2130352119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d5c0cef20 0x7f2d5c0cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:09:06.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.202+0000 7f2d60e00640 1 --2- 172.21.15.86:0/2130352119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d5c0cdf50 0x7f2d5c152520 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f2d4800c340 tx=0x7f2d4800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:06.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.206+0000 7f2d59600640 1 -- 172.21.15.86:0/2130352119 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d480052d0 con 0x7f2d5c0cdf50 2024-04-25T14:09:06.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.206+0000 7f2d59600640 1 -- 172.21.15.86:0/2130352119 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d48005470 con 0x7f2d5c0cdf50 2024-04-25T14:09:06.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.206+0000 7f2d59600640 1 -- 172.21.15.86:0/2130352119 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48007260 con 0x7f2d5c0cdf50 2024-04-25T14:09:06.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.206+0000 7f2d64200640 1 -- 172.21.15.86:0/2130352119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d5c158310 con 0x7f2d5c0cdf50 2024-04-25T14:09:06.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.206+0000 7f2d64200640 1 -- 172.21.15.86:0/2130352119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d5c1588d0 con 0x7f2d5c0cdf50 2024-04-25T14:09:06.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.206+0000 7f2d64200640 1 -- 172.21.15.86:0/2130352119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d5c158080 con 0x7f2d5c0cdf50 2024-04-25T14:09:06.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.206+0000 7f2d59600640 1 -- 172.21.15.86:0/2130352119 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 32) ==== 93253+0+0 (secure 0 0 0) 0x7f2d48007400 con 0x7f2d5c0cdf50 2024-04-25T14:09:06.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.210+0000 7f2d59600640 1 --2- 172.21.15.86:0/2130352119 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f2d3c06f490 0x7f2d3c0719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:06.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.210+0000 7f2d61800640 1 -- 172.21.15.86:0/2130352119 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f2d3c06f490 msgr2=0x7f2d3c0719a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/132942210 2024-04-25T14:09:06.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.210+0000 7f2d59600640 1 -- 172.21.15.86:0/2130352119 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6445+0+0 (secure 0 0 0) 0x7f2d4808e910 con 0x7f2d5c0cdf50 2024-04-25T14:09:06.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.210+0000 7f2d61800640 1 --2- 172.21.15.86:0/2130352119 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f2d3c06f490 0x7f2d3c0719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:09:06.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.214+0000 7f2d59600640 1 -- 172.21.15.86:0/2130352119 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2d4805af70 con 0x7f2d5c0cdf50 2024-04-25T14:09:06.228 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:06.226+0000 7f89ff72a180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:09:06.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.378+0000 7f2d64200640 1 -- 172.21.15.86:0/2130352119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2d5c0a0630 con 0x7f2d5c0cdf50 2024-04-25T14:09:06.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.378+0000 7f2d59600640 1 -- 172.21.15.86:0/2130352119 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f2d4805f270 con 0x7f2d5c0cdf50 2024-04-25T14:09:06.378 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:09:06.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.378+0000 7f2d64200640 1 -- 172.21.15.86:0/2130352119 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f2d3c06f490 msgr2=0x7f2d3c0719a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:09:06.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.378+0000 7f2d64200640 1 --2- 172.21.15.86:0/2130352119 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f2d3c06f490 0x7f2d3c0719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:06.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.378+0000 7f2d64200640 1 -- 172.21.15.86:0/2130352119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d5c0cdf50 msgr2=0x7f2d5c152520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:06.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.378+0000 7f2d64200640 1 --2- 172.21.15.86:0/2130352119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d5c0cdf50 0x7f2d5c152520 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f2d4800c340 tx=0x7f2d4800c800 comp rx=0 tx=0).stop 2024-04-25T14:09:06.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.378+0000 7f2d64200640 1 -- 172.21.15.86:0/2130352119 shutdown_connections 2024-04-25T14:09:06.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.378+0000 7f2d64200640 1 --2- 172.21.15.86:0/2130352119 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f2d3c06f490 0x7f2d3c0719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:06.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.378+0000 7f2d64200640 1 --2- 172.21.15.86:0/2130352119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d5c0cef20 0x7f2d5c0cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:06.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.378+0000 7f2d64200640 1 --2- 172.21.15.86:0/2130352119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d5c0cdf50 0x7f2d5c152520 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:06.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.378+0000 7f2d64200640 1 --2- 172.21.15.86:0/2130352119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d5c06c670 0x7f2d5c0ca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:06.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.378+0000 7f2d64200640 1 -- 172.21.15.86:0/2130352119 >> 172.21.15.86:0/2130352119 conn(0x7f2d5c09ba30 msgr2=0x7f2d5c0b73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:06.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.378+0000 7f2d64200640 1 -- 172.21.15.86:0/2130352119 shutdown_connections 2024-04-25T14:09:06.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.378+0000 7f2d64200640 1 -- 172.21.15.86:0/2130352119 wait complete. 2024-04-25T14:09:06.400 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:09:06.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.590+0000 7f4133800640 1 Processor -- start 2024-04-25T14:09:06.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.590+0000 7f4133800640 1 -- start start 2024-04-25T14:09:06.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.594+0000 7f4133800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f412c0cde90 0x7f412c0ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:06.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.594+0000 7f4133800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f412c15e5c0 con 0x7f412c070330 2024-04-25T14:09:06.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.594+0000 7f4133800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f412c15e8a0 con 0x7f412c0d1df0 2024-04-25T14:09:06.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.594+0000 7f4133800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f412c06fd40 con 0x7f412c0cde90 2024-04-25T14:09:06.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.594+0000 7f4131800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f412c0cde90 0x7f412c0ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:06.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.594+0000 7f4131800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f412c0cde90 0x7f412c0ce2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39214/0 (socket says 172.21.15.86:39214) 2024-04-25T14:09:06.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.594+0000 7f4131800640 1 -- 172.21.15.86:0/2457409343 learned_addr learned my addr 172.21.15.86:0/2457409343 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:06.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.594+0000 7f4131800640 1 -- 172.21.15.86:0/2457409343 >> v1:172.21.15.155:6789/0 conn(0x7f412c0d1df0 legacy=0x7f412c0cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:06.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.594+0000 7f4131800640 1 -- 172.21.15.86:0/2457409343 >> v1:172.21.15.86:6789/0 conn(0x7f412c070330 legacy=0x7f412c0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:06.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.594+0000 7f4131800640 1 -- 172.21.15.86:0/2457409343 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f412c15eb80 con 0x7f412c0cde90 2024-04-25T14:09:06.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.598+0000 7f4131800640 1 --2- 172.21.15.86:0/2457409343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f412c0cde90 0x7f412c0ce2a0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f4124009e80 tx=0x7f412401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f715ba178aa7fdd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:06.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.598+0000 7f412b400640 1 -- 172.21.15.86:0/2457409343 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41240a5720 con 0x7f412c0cde90 2024-04-25T14:09:06.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.598+0000 7f412b400640 1 -- 172.21.15.86:0/2457409343 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41240a58c0 con 0x7f412c0cde90 2024-04-25T14:09:06.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.598+0000 7f412b400640 1 -- 172.21.15.86:0/2457409343 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41240a5c90 con 0x7f412c0cde90 2024-04-25T14:09:06.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.598+0000 7f4133800640 1 -- 172.21.15.86:0/2457409343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f412c0cde90 msgr2=0x7f412c0ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:06.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.598+0000 7f4133800640 1 --2- 172.21.15.86:0/2457409343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f412c0cde90 0x7f412c0ce2a0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f4124009e80 tx=0x7f412401c950 comp rx=0 tx=0).stop 2024-04-25T14:09:06.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.598+0000 7f4133800640 1 -- 172.21.15.86:0/2457409343 shutdown_connections 2024-04-25T14:09:06.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.598+0000 7f4133800640 1 --2- 172.21.15.86:0/2457409343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f412c0cde90 0x7f412c0ce2a0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:06.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.598+0000 7f4133800640 1 -- 172.21.15.86:0/2457409343 >> 172.21.15.86:0/2457409343 conn(0x7f412c0b4650 msgr2=0x7f412c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:06.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.598+0000 7f4133800640 1 -- 172.21.15.86:0/2457409343 shutdown_connections 2024-04-25T14:09:06.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.598+0000 7f4133800640 1 -- 172.21.15.86:0/2457409343 wait complete. 2024-04-25T14:09:06.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.598+0000 7f4133800640 1 Processor -- start 2024-04-25T14:09:06.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.598+0000 7f4133800640 1 -- start start 2024-04-25T14:09:06.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.602+0000 7f4133800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f412c070330 0x7f412c09a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:06.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.602+0000 7f4133800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f412c0cde90 0x7f412c0c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:06.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.602+0000 7f4133800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f412c0ca190 0x7f412c156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:06.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.602+0000 7f4133800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f412c0708c0 con 0x7f412c0cde90 2024-04-25T14:09:06.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.602+0000 7f4133800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f412c070740 con 0x7f412c070330 2024-04-25T14:09:06.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.602+0000 7f4133800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f412c16bfd0 con 0x7f412c0ca190 2024-04-25T14:09:06.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.602+0000 7f412be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f412c0cde90 0x7f412c0c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:06.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.602+0000 7f4130e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f412c070330 0x7f412c09a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:06.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.602+0000 7f4131800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f412c0ca190 0x7f412c156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:06.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.602+0000 7f4130e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f412c070330 0x7f412c09a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:51046/0 (socket says 172.21.15.86:51046) 2024-04-25T14:09:06.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.602+0000 7f4130e00640 1 -- 172.21.15.86:0/1774995529 learned_addr learned my addr 172.21.15.86:0/1774995529 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:06.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.602+0000 7f412be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f412c0cde90 0x7f412c0c9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39228/0 (socket says 172.21.15.86:39228) 2024-04-25T14:09:06.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.602+0000 7f4130e00640 1 -- 172.21.15.86:0/1774995529 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f412c0ca190 msgr2=0x7f412c156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:06.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.602+0000 7f4130e00640 1 --2- 172.21.15.86:0/1774995529 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f412c0ca190 0x7f412c156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:06.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.602+0000 7f4130e00640 1 -- 172.21.15.86:0/1774995529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f412c0cde90 msgr2=0x7f412c0c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:06.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.602+0000 7f4130e00640 1 --2- 172.21.15.86:0/1774995529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f412c0cde90 0x7f412c0c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:06.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.602+0000 7f4130e00640 1 -- 172.21.15.86:0/1774995529 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f412c0caa10 con 0x7f412c070330 2024-04-25T14:09:06.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.602+0000 7f412be00640 1 --2- 172.21.15.86:0/1774995529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f412c0cde90 0x7f412c0c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:09:06.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.602+0000 7f4130e00640 1 --2- 172.21.15.86:0/1774995529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f412c070330 0x7f412c09a720 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f412000c390 tx=0x7f412000c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:06.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.602+0000 7f4128c00640 1 -- 172.21.15.86:0/1774995529 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4120017070 con 0x7f412c070330 2024-04-25T14:09:06.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.602+0000 7f4128c00640 1 -- 172.21.15.86:0/1774995529 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41200052d0 con 0x7f412c070330 2024-04-25T14:09:06.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.602+0000 7f4128c00640 1 -- 172.21.15.86:0/1774995529 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4120003820 con 0x7f412c070330 2024-04-25T14:09:06.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.602+0000 7f4133800640 1 -- 172.21.15.86:0/1774995529 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f412c0cacf0 con 0x7f412c070330 2024-04-25T14:09:06.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.602+0000 7f4133800640 1 -- 172.21.15.86:0/1774995529 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f412c0cf3e0 con 0x7f412c070330 2024-04-25T14:09:06.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.606+0000 7f4133800640 1 -- 172.21.15.86:0/1774995529 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f412c069d00 con 0x7f412c070330 2024-04-25T14:09:06.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.606+0000 7f4128c00640 1 -- 172.21.15.86:0/1774995529 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 32) ==== 93253+0+0 (secure 0 0 0) 0x7f412001a070 con 0x7f412c070330 2024-04-25T14:09:06.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.606+0000 7f4128c00640 1 --2- 172.21.15.86:0/1774995529 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f410c06f490 0x7f410c0719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:06.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.606+0000 7f4128c00640 1 -- 172.21.15.86:0/1774995529 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6445+0+0 (secure 0 0 0) 0x7f412008eab0 con 0x7f412c070330 2024-04-25T14:09:06.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.606+0000 7f412be00640 1 -- 172.21.15.86:0/1774995529 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f410c06f490 msgr2=0x7f410c0719a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/132942210 2024-04-25T14:09:06.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.606+0000 7f412be00640 1 --2- 172.21.15.86:0/1774995529 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f410c06f490 0x7f410c0719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:09:06.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.610+0000 7f4128c00640 1 -- 172.21.15.86:0/1774995529 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4120005480 con 0x7f412c070330 2024-04-25T14:09:06.670 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:06.672+0000 7fce4717e180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:09:06.717 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:06.714+0000 7f89ff72a180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:09:06.729 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:06.728+0000 7fce4717e180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:09:06.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.770+0000 7f4133800640 1 -- 172.21.15.86:0/1774995529 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f412c076bf0 con 0x7f412c070330 2024-04-25T14:09:06.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.770+0000 7f4128c00640 1 -- 172.21.15.86:0/1774995529 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f412005b080 con 0x7f412c070330 2024-04-25T14:09:06.772 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:09:06.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.774+0000 7f4133800640 1 -- 172.21.15.86:0/1774995529 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f410c06f490 msgr2=0x7f410c0719a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:09:06.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.774+0000 7f4133800640 1 --2- 172.21.15.86:0/1774995529 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f410c06f490 0x7f410c0719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:06.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.774+0000 7f4133800640 1 -- 172.21.15.86:0/1774995529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f412c070330 msgr2=0x7f412c09a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:06.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.774+0000 7f4133800640 1 --2- 172.21.15.86:0/1774995529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f412c070330 0x7f412c09a720 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f412000c390 tx=0x7f412000c850 comp rx=0 tx=0).stop 2024-04-25T14:09:06.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.774+0000 7f4133800640 1 -- 172.21.15.86:0/1774995529 shutdown_connections 2024-04-25T14:09:06.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.774+0000 7f4133800640 1 --2- 172.21.15.86:0/1774995529 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f410c06f490 0x7f410c0719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:06.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.774+0000 7f4133800640 1 --2- 172.21.15.86:0/1774995529 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f412c0ca190 0x7f412c156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:06.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.774+0000 7f4133800640 1 --2- 172.21.15.86:0/1774995529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f412c0cde90 0x7f412c0c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:06.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.774+0000 7f4133800640 1 --2- 172.21.15.86:0/1774995529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f412c070330 0x7f412c09a720 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:06.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.774+0000 7f4133800640 1 -- 172.21.15.86:0/1774995529 >> 172.21.15.86:0/1774995529 conn(0x7f412c0b4650 msgr2=0x7f412c0cb190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:06.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.774+0000 7f4133800640 1 -- 172.21.15.86:0/1774995529 shutdown_connections 2024-04-25T14:09:06.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.774+0000 7f4133800640 1 -- 172.21.15.86:0/1774995529 wait complete. 2024-04-25T14:09:06.778 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:06.778+0000 7f89ff72a180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:09:06.786 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:06.788+0000 7fce4717e180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:09:06.796 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:09:06.837 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:06.834+0000 7f89ff72a180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:09:06.874 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:06.876+0000 7fce4717e180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:09:06.930 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:06.930+0000 7f89ff72a180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:09:06.931 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:06.932+0000 7fce4717e180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:09:06.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.946+0000 7ff33d600640 1 Processor -- start 2024-04-25T14:09:06.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.946+0000 7ff33d600640 1 -- start start 2024-04-25T14:09:06.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.950+0000 7ff33d600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff338071530 0x7ff33815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:06.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.950+0000 7ff33d600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3381616e0 0x7ff338161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:06.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.950+0000 7ff33d600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3380cd8d0 con 0x7ff3380d2c80 2024-04-25T14:09:06.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.950+0000 7ff33d600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff3380ce720 con 0x7ff338071530 2024-04-25T14:09:06.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.950+0000 7ff33d600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff3380ce970 con 0x7ff3381616e0 2024-04-25T14:09:06.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.950+0000 7ff32fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff338071530 0x7ff33815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:06.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.950+0000 7ff32fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff338071530 0x7ff33815dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39236/0 (socket says 172.21.15.86:39236) 2024-04-25T14:09:06.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.950+0000 7ff32fe00640 1 -- 172.21.15.86:0/1529912850 learned_addr learned my addr 172.21.15.86:0/1529912850 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:06.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.950+0000 7ff337400640 1 --2- 172.21.15.86:0/1529912850 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3381616e0 0x7ff338161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:06.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.950+0000 7ff32fe00640 1 -- 172.21.15.86:0/1529912850 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3381616e0 msgr2=0x7ff338161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:06.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.950+0000 7ff32fe00640 1 --2- 172.21.15.86:0/1529912850 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3381616e0 0x7ff338161af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:06.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.950+0000 7ff32fe00640 1 -- 172.21.15.86:0/1529912850 >> v1:172.21.15.155:6789/0 conn(0x7ff3380d2c80 legacy=0x7ff338070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:06.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.950+0000 7ff32fe00640 1 -- 172.21.15.86:0/1529912850 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3380d1be0 con 0x7ff338071530 2024-04-25T14:09:06.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.954+0000 7ff32fe00640 1 --2- 172.21.15.86:0/1529912850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff338071530 0x7ff33815dc90 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7ff320006fd0 tx=0x7ff32001ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e08565f729e43c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:06.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.954+0000 7ff336000640 1 -- 172.21.15.86:0/1529912850 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3200b5070 con 0x7ff338071530 2024-04-25T14:09:06.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.954+0000 7ff336000640 1 -- 172.21.15.86:0/1529912850 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3200a8a30 con 0x7ff338071530 2024-04-25T14:09:06.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.954+0000 7ff336000640 1 -- 172.21.15.86:0/1529912850 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3200a8e00 con 0x7ff338071530 2024-04-25T14:09:06.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.954+0000 7ff33d600640 1 -- 172.21.15.86:0/1529912850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff338071530 msgr2=0x7ff33815dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:06.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.954+0000 7ff33d600640 1 --2- 172.21.15.86:0/1529912850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff338071530 0x7ff33815dc90 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7ff320006fd0 tx=0x7ff32001ed30 comp rx=0 tx=0).stop 2024-04-25T14:09:06.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.954+0000 7ff33d600640 1 -- 172.21.15.86:0/1529912850 shutdown_connections 2024-04-25T14:09:06.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.954+0000 7ff33d600640 1 --2- 172.21.15.86:0/1529912850 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3381616e0 0x7ff338161af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:06.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.954+0000 7ff33d600640 1 --2- 172.21.15.86:0/1529912850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff338071530 0x7ff33815dc90 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:06.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.954+0000 7ff33d600640 1 -- 172.21.15.86:0/1529912850 >> 172.21.15.86:0/1529912850 conn(0x7ff3380cffc0 msgr2=0x7ff3380d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:06.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.954+0000 7ff33d600640 1 -- 172.21.15.86:0/1529912850 shutdown_connections 2024-04-25T14:09:06.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.954+0000 7ff33d600640 1 -- 172.21.15.86:0/1529912850 wait complete. 2024-04-25T14:09:06.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.958+0000 7ff33d600640 1 Processor -- start 2024-04-25T14:09:06.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.958+0000 7ff33d600640 1 -- start start 2024-04-25T14:09:06.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.958+0000 7ff33d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3380d2c80 0x7ff33815e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:06.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.958+0000 7ff33d600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3381616e0 0x7ff3380c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:06.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.958+0000 7ff33d600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff338160910 0x7ff338151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:06.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.958+0000 7ff33d600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3380d2480 con 0x7ff3380d2c80 2024-04-25T14:09:06.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.958+0000 7ff33d600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff3380d21a0 con 0x7ff3381616e0 2024-04-25T14:09:06.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.958+0000 7ff33d600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff3380d29a0 con 0x7ff338160910 2024-04-25T14:09:06.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.958+0000 7ff336a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3380d2c80 0x7ff33815e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:06.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.958+0000 7ff336a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3380d2c80 0x7ff33815e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39244/0 (socket says 172.21.15.86:39244) 2024-04-25T14:09:06.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.958+0000 7ff336a00640 1 -- 172.21.15.86:0/807662226 learned_addr learned my addr 172.21.15.86:0/807662226 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:06.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.958+0000 7ff32fe00640 1 --2- 172.21.15.86:0/807662226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3381616e0 0x7ff3380c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:06.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.958+0000 7ff337400640 1 --2- 172.21.15.86:0/807662226 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff338160910 0x7ff338151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:06.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.958+0000 7ff336a00640 1 -- 172.21.15.86:0/807662226 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff338160910 msgr2=0x7ff338151650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:06.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.958+0000 7ff336a00640 1 --2- 172.21.15.86:0/807662226 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff338160910 0x7ff338151650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:06.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.958+0000 7ff336a00640 1 -- 172.21.15.86:0/807662226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3381616e0 msgr2=0x7ff3380c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:06.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.958+0000 7ff336a00640 1 --2- 172.21.15.86:0/807662226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3381616e0 0x7ff3380c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:06.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.958+0000 7ff336a00640 1 -- 172.21.15.86:0/807662226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff33815efb0 con 0x7ff3380d2c80 2024-04-25T14:09:06.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.958+0000 7ff336a00640 1 --2- 172.21.15.86:0/807662226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3380d2c80 0x7ff33815e6a0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7ff32800c3a0 tx=0x7ff32800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:06.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.958+0000 7ff32ea00640 1 -- 172.21.15.86:0/807662226 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3280052d0 con 0x7ff3380d2c80 2024-04-25T14:09:06.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.958+0000 7ff32ea00640 1 -- 172.21.15.86:0/807662226 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff328005470 con 0x7ff3380d2c80 2024-04-25T14:09:06.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.958+0000 7ff33d600640 1 -- 172.21.15.86:0/807662226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff33815f290 con 0x7ff3380d2c80 2024-04-25T14:09:06.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.958+0000 7ff32ea00640 1 -- 172.21.15.86:0/807662226 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff328007260 con 0x7ff3380d2c80 2024-04-25T14:09:06.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.958+0000 7ff33d600640 1 -- 172.21.15.86:0/807662226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3380c9580 con 0x7ff3380d2c80 2024-04-25T14:09:06.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.962+0000 7ff33d600640 1 -- 172.21.15.86:0/807662226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff33815f7b0 con 0x7ff3380d2c80 2024-04-25T14:09:06.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.962+0000 7ff32ea00640 1 -- 172.21.15.86:0/807662226 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 32) ==== 93253+0+0 (secure 0 0 0) 0x7ff328007400 con 0x7ff3380d2c80 2024-04-25T14:09:06.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.962+0000 7ff32ea00640 1 --2- 172.21.15.86:0/807662226 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7ff31406f490 0x7ff3140719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:06.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.962+0000 7ff32fe00640 1 -- 172.21.15.86:0/807662226 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7ff31406f490 msgr2=0x7ff3140719a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/132942210 2024-04-25T14:09:06.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.962+0000 7ff32ea00640 1 -- 172.21.15.86:0/807662226 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6445+0+0 (secure 0 0 0) 0x7ff32808ea30 con 0x7ff3380d2c80 2024-04-25T14:09:06.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.962+0000 7ff32fe00640 1 --2- 172.21.15.86:0/807662226 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7ff31406f490 0x7ff3140719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:09:06.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:06.970+0000 7ff32ea00640 1 -- 172.21.15.86:0/807662226 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff33815f7b0 con 0x7ff3380d2c80 2024-04-25T14:09:06.995 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:06.994+0000 7f89ff72a180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:09:07.017 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:07.016+0000 7fce4717e180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:09:07.076 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:07.076+0000 7fce4717e180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:09:07.082 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:07.082+0000 7f89ff72a180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:09:07.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.138+0000 7ff33d600640 1 -- 172.21.15.86:0/807662226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff3380720e0 con 0x7ff3380d2c80 2024-04-25T14:09:07.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.138+0000 7ff32ea00640 1 -- 172.21.15.86:0/807662226 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7ff32805af90 con 0x7ff3380d2c80 2024-04-25T14:09:07.141 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:09:07.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.142+0000 7ff33d600640 1 -- 172.21.15.86:0/807662226 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7ff31406f490 msgr2=0x7ff3140719a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:09:07.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.142+0000 7ff33d600640 1 --2- 172.21.15.86:0/807662226 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7ff31406f490 0x7ff3140719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:07.145 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:07.142+0000 7f89ff72a180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:09:07.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.142+0000 7ff33d600640 1 -- 172.21.15.86:0/807662226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3380d2c80 msgr2=0x7ff33815e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:07.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.142+0000 7ff33d600640 1 --2- 172.21.15.86:0/807662226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3380d2c80 0x7ff33815e6a0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7ff32800c3a0 tx=0x7ff32800c860 comp rx=0 tx=0).stop 2024-04-25T14:09:07.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.142+0000 7ff33d600640 1 -- 172.21.15.86:0/807662226 shutdown_connections 2024-04-25T14:09:07.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.142+0000 7ff33d600640 1 --2- 172.21.15.86:0/807662226 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7ff31406f490 0x7ff3140719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:07.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.142+0000 7ff33d600640 1 --2- 172.21.15.86:0/807662226 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff338160910 0x7ff338151650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:07.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.142+0000 7ff33d600640 1 --2- 172.21.15.86:0/807662226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3381616e0 0x7ff3380c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:07.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.142+0000 7ff33d600640 1 --2- 172.21.15.86:0/807662226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3380d2c80 0x7ff33815e6a0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:07.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.142+0000 7ff33d600640 1 -- 172.21.15.86:0/807662226 >> 172.21.15.86:0/807662226 conn(0x7ff3380cffc0 msgr2=0x7ff3380cbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:07.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.142+0000 7ff33d600640 1 -- 172.21.15.86:0/807662226 shutdown_connections 2024-04-25T14:09:07.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.142+0000 7ff33d600640 1 -- 172.21.15.86:0/807662226 wait complete. 2024-04-25T14:09:07.174 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T14:09:07.227 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:07.228+0000 7fce4717e180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:09:07.286 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:07.288+0000 7fce4717e180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:09:07.313 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:07.314+0000 7f89ff72a180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:09:07.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.366+0000 7f552c800640 1 Processor -- start 2024-04-25T14:09:07.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.366+0000 7f552c800640 1 -- start start 2024-04-25T14:09:07.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.366+0000 7f552c800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f55240d05e0 0x7f55240b6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:07.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.366+0000 7f552c800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55240d2420 con 0x7f55240b6dc0 2024-04-25T14:09:07.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.366+0000 7f552c800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55240d2700 con 0x7f55240ce530 2024-04-25T14:09:07.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.366+0000 7f552c800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f55240d29e0 con 0x7f55240d05e0 2024-04-25T14:09:07.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.366+0000 7f5529400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f55240d05e0 0x7f55240b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:07.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.366+0000 7f5529400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f55240d05e0 0x7f55240b6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39258/0 (socket says 172.21.15.86:39258) 2024-04-25T14:09:07.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.366+0000 7f5529400640 1 -- 172.21.15.86:0/3945342130 learned_addr learned my addr 172.21.15.86:0/3945342130 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:07.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.366+0000 7f552a800640 1 --1- 172.21.15.86:0/3945342130 >> v1:172.21.15.86:6789/0 conn(0x7f55240b6dc0 0x7f55240b71d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:46202/0 (socket says 172.21.15.86:46202) 2024-04-25T14:09:07.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.370+0000 7f5529400640 1 -- 172.21.15.86:0/3945342130 >> v1:172.21.15.155:6789/0 conn(0x7f55240ce530 legacy=0x7f55240cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:07.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.370+0000 7f5529400640 1 -- 172.21.15.86:0/3945342130 >> v1:172.21.15.86:6789/0 conn(0x7f55240b6dc0 legacy=0x7f55240b71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:07.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.370+0000 7f5529400640 1 -- 172.21.15.86:0/3945342130 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55240d2cc0 con 0x7f55240d05e0 2024-04-25T14:09:07.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.370+0000 7f5529400640 1 --2- 172.21.15.86:0/3945342130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55240d05e0 0x7f55240b6880 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f5514009870 tx=0x7f551401ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373a0c1c2afad1b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:07.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.370+0000 7f5528a00640 1 -- 172.21.15.86:0/3945342130 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55140b5070 con 0x7f55240d05e0 2024-04-25T14:09:07.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.370+0000 7f5528a00640 1 -- 172.21.15.86:0/3945342130 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55140a8a50 con 0x7f55240d05e0 2024-04-25T14:09:07.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.370+0000 7f5528a00640 1 -- 172.21.15.86:0/3945342130 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55140a8e20 con 0x7f55240d05e0 2024-04-25T14:09:07.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.374+0000 7f552c800640 1 -- 172.21.15.86:0/3945342130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55240d05e0 msgr2=0x7f55240b6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:07.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.374+0000 7f552c800640 1 --2- 172.21.15.86:0/3945342130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55240d05e0 0x7f55240b6880 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f5514009870 tx=0x7f551401ed50 comp rx=0 tx=0).stop 2024-04-25T14:09:07.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.374+0000 7f552c800640 1 -- 172.21.15.86:0/3945342130 shutdown_connections 2024-04-25T14:09:07.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.374+0000 7f552c800640 1 --2- 172.21.15.86:0/3945342130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55240d05e0 0x7f55240b6880 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:07.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.374+0000 7f552c800640 1 -- 172.21.15.86:0/3945342130 >> 172.21.15.86:0/3945342130 conn(0x7f55240b5020 msgr2=0x7f55240b5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:07.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.374+0000 7f552c800640 1 -- 172.21.15.86:0/3945342130 shutdown_connections 2024-04-25T14:09:07.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.374+0000 7f552c800640 1 -- 172.21.15.86:0/3945342130 wait complete. 2024-04-25T14:09:07.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.374+0000 7f552c800640 1 Processor -- start 2024-04-25T14:09:07.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.374+0000 7f552c800640 1 -- start start 2024-04-25T14:09:07.377 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:07.374+0000 7f89ff72a180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:09:07.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.378+0000 7f552c800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55240b6dc0 0x7f552414ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:07.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.378+0000 7f552c800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55240ce530 0x7f55240ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:07.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.378+0000 7f552c800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5524158c90 0x7f55240b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:07.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.378+0000 7f552c800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55240ce940 con 0x7f5524158c90 2024-04-25T14:09:07.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.378+0000 7f552c800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f552415df70 con 0x7f55240b6dc0 2024-04-25T14:09:07.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.378+0000 7f552c800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f55240ceac0 con 0x7f55240ce530 2024-04-25T14:09:07.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.378+0000 7f552a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5524158c90 0x7f55240b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:07.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.378+0000 7f552a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5524158c90 0x7f55240b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39268/0 (socket says 172.21.15.86:39268) 2024-04-25T14:09:07.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.378+0000 7f552a800640 1 -- 172.21.15.86:0/4239581196 learned_addr learned my addr 172.21.15.86:0/4239581196 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:07.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.378+0000 7f5529e00640 1 --2- 172.21.15.86:0/4239581196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55240b6dc0 0x7f552414ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:07.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.378+0000 7f5529400640 1 --2- 172.21.15.86:0/4239581196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55240ce530 0x7f55240ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:07.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.378+0000 7f552a800640 1 -- 172.21.15.86:0/4239581196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55240ce530 msgr2=0x7f55240ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:07.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.378+0000 7f552a800640 1 --2- 172.21.15.86:0/4239581196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55240ce530 0x7f55240ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:07.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.378+0000 7f552a800640 1 -- 172.21.15.86:0/4239581196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55240b6dc0 msgr2=0x7f552414ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:07.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.378+0000 7f552a800640 1 --2- 172.21.15.86:0/4239581196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55240b6dc0 0x7f552414ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:07.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.378+0000 7f552a800640 1 -- 172.21.15.86:0/4239581196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5524157bf0 con 0x7f5524158c90 2024-04-25T14:09:07.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.378+0000 7f552a800640 1 --2- 172.21.15.86:0/4239581196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5524158c90 0x7f55240b3f80 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f552000c700 tx=0x7f552000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:07.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.378+0000 7f551e000640 1 -- 172.21.15.86:0/4239581196 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5520018070 con 0x7f5524158c90 2024-04-25T14:09:07.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.378+0000 7f551e000640 1 -- 172.21.15.86:0/4239581196 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5520004d70 con 0x7f5524158c90 2024-04-25T14:09:07.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.378+0000 7f551e000640 1 -- 172.21.15.86:0/4239581196 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552000a800 con 0x7f5524158c90 2024-04-25T14:09:07.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.378+0000 7f552c800640 1 -- 172.21.15.86:0/4239581196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5524157ed0 con 0x7f5524158c90 2024-04-25T14:09:07.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.378+0000 7f552c800640 1 -- 172.21.15.86:0/4239581196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5524158490 con 0x7f5524158c90 2024-04-25T14:09:07.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.378+0000 7f552c800640 1 -- 172.21.15.86:0/4239581196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5524072170 con 0x7f5524158c90 2024-04-25T14:09:07.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.382+0000 7f551e000640 1 -- 172.21.15.86:0/4239581196 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 32) ==== 93253+0+0 (secure 0 0 0) 0x7f552000a9a0 con 0x7f5524158c90 2024-04-25T14:09:07.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.382+0000 7f551e000640 1 --2- 172.21.15.86:0/4239581196 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f550806f490 0x7f55080719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:07.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.382+0000 7f5529e00640 1 -- 172.21.15.86:0/4239581196 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f550806f490 msgr2=0x7f55080719a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/132942210 2024-04-25T14:09:07.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.382+0000 7f551e000640 1 -- 172.21.15.86:0/4239581196 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6445+0+0 (secure 0 0 0) 0x7f552008e890 con 0x7f5524158c90 2024-04-25T14:09:07.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.382+0000 7f5529e00640 1 --2- 172.21.15.86:0/4239581196 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f550806f490 0x7f55080719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:09:07.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.386+0000 7f551e000640 1 -- 172.21.15.86:0/4239581196 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f552005aef0 con 0x7f5524158c90 2024-04-25T14:09:07.529 INFO:tasks.ceph.mgr.x.smithi155.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:09:07.529 INFO:tasks.ceph.mgr.x.smithi155.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:09:07.530 INFO:tasks.ceph.mgr.x.smithi155.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:09:07.537 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:07.540+0000 7fce4717e180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:09:07.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.586+0000 7f5529e00640 1 -- 172.21.15.86:0/4239581196 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f550806f490 msgr2=0x7f55080719a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/132942210 2024-04-25T14:09:07.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.586+0000 7f5529e00640 1 --2- 172.21.15.86:0/4239581196 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f550806f490 0x7f55080719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:09:07.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.598+0000 7f552c800640 1 -- 172.21.15.86:0/4239581196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f55240ce2f0 con 0x7f5524158c90 2024-04-25T14:09:07.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.598+0000 7f551e000640 1 -- 172.21.15.86:0/4239581196 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v88) ==== 80+0+470 (secure 0 0 0) 0x7f552005f1f0 con 0x7f5524158c90 2024-04-25T14:09:07.601 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3132647616 2024-04-26T14:09:03.346201+0000 2024-04-25T14:09:07.602 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1564187566 2024-04-26T14:09:03.346201+0000 2024-04-25T14:09:07.602 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6833/3589556609 2024-04-26T14:09:03.346201+0000 2024-04-25T14:09:07.602 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1099362620 2024-04-26T14:09:03.346201+0000 2024-04-25T14:09:07.602 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6832/3589556609 2024-04-26T14:09:03.346201+0000 2024-04-25T14:09:07.602 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3481243210 2024-04-26T14:09:03.346201+0000 2024-04-25T14:09:07.602 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2509210906 2024-04-26T14:09:03.346201+0000 2024-04-25T14:09:07.602 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3722630918 2024-04-26T14:09:03.346201+0000 2024-04-25T14:09:07.602 INFO:teuthology.orchestra.run.smithi086.stderr:listed 8 entries 2024-04-25T14:09:07.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.602+0000 7f552c800640 1 -- 172.21.15.86:0/4239581196 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f550806f490 msgr2=0x7f55080719a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:09:07.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.602+0000 7f552c800640 1 --2- 172.21.15.86:0/4239581196 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f550806f490 0x7f55080719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:07.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.602+0000 7f552c800640 1 -- 172.21.15.86:0/4239581196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5524158c90 msgr2=0x7f55240b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:07.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.602+0000 7f552c800640 1 --2- 172.21.15.86:0/4239581196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5524158c90 0x7f55240b3f80 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f552000c700 tx=0x7f552000cbc0 comp rx=0 tx=0).stop 2024-04-25T14:09:07.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.602+0000 7f552c800640 1 -- 172.21.15.86:0/4239581196 shutdown_connections 2024-04-25T14:09:07.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.602+0000 7f552c800640 1 --2- 172.21.15.86:0/4239581196 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f550806f490 0x7f55080719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:07.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.602+0000 7f552c800640 1 --2- 172.21.15.86:0/4239581196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5524158c90 0x7f55240b3f80 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:07.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.602+0000 7f552c800640 1 --2- 172.21.15.86:0/4239581196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55240ce530 0x7f55240ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:07.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.602+0000 7f552c800640 1 --2- 172.21.15.86:0/4239581196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55240b6dc0 0x7f552414ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:07.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.602+0000 7f552c800640 1 -- 172.21.15.86:0/4239581196 >> 172.21.15.86:0/4239581196 conn(0x7f55240b5020 msgr2=0x7f55240b8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:07.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.602+0000 7f552c800640 1 -- 172.21.15.86:0/4239581196 shutdown_connections 2024-04-25T14:09:07.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.602+0000 7f552c800640 1 -- 172.21.15.86:0/4239581196 wait complete. 2024-04-25T14:09:07.623 INFO:tasks.ceph.mgr.y.smithi086.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:09:07.623 INFO:tasks.ceph.mgr.y.smithi086.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:09:07.623 INFO:tasks.ceph.mgr.y.smithi086.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:09:07.625 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T14:09:07.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:09:07.625 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T14:09:07.632 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:07.630+0000 7f89ff72a180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:09:07.651 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T14:09:07.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:09:07.652 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T14:09:07.685 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:07.688+0000 7fce4717e180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:09:07.702 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:09:07.702 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:09:07.702 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:09:07.711 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:09:07.711 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:09:07.761 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:09:07.786 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:07.786+0000 7f89ff72a180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:09:07.813 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:07.812+0000 7fce4717e180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:09:07.884 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:07.884+0000 7fce4717e180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:09:07.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.910+0000 7f63c9c00640 1 Processor -- start 2024-04-25T14:09:07.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.910+0000 7f63c9c00640 1 -- start start 2024-04-25T14:09:07.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.910+0000 7f63c9c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f63c40d1df0 0x7f63c40b0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:07.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.910+0000 7f63c9c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63c40b0690 0x7f63c40d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:07.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.910+0000 7f63c9c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63c40cde20 con 0x7f63c41642a0 2024-04-25T14:09:07.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.910+0000 7f63c9c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f63c40cea30 con 0x7f63c40d1df0 2024-04-25T14:09:07.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.910+0000 7f63c9c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f63c40ce100 con 0x7f63c40b0690 2024-04-25T14:09:07.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.910+0000 7f63c2a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f63c41642a0 0x7f63c40d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:46212/0 (socket says 172.21.15.86:46212) 2024-04-25T14:09:07.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.910+0000 7f63c2a00640 1 -- 172.21.15.86:0/4018149275 learned_addr learned my addr 172.21.15.86:0/4018149275 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:07.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.910+0000 7f63c2000640 1 --2- 172.21.15.86:0/4018149275 >> v2:172.21.15.155:3300/0 conn(0x7f63c40d1df0 0x7f63c40b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:07.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.910+0000 7f63c3400640 1 --2- 172.21.15.86:0/4018149275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63c40b0690 0x7f63c40d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:07.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.914+0000 7f63c1600640 1 -- 172.21.15.86:0/4018149275 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3445251047 0 0) 0x7f63c40cde20 con 0x7f63c41642a0 2024-04-25T14:09:07.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.914+0000 7f63c1600640 1 -- 172.21.15.86:0/4018149275 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f63a8003530 con 0x7f63c41642a0 2024-04-25T14:09:07.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.914+0000 7f63c2000640 1 -- 172.21.15.86:0/4018149275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63c40b0690 msgr2=0x7f63c40d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:07.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.914+0000 7f63c2000640 1 --2- 172.21.15.86:0/4018149275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63c40b0690 0x7f63c40d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:07.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.914+0000 7f63c2000640 1 -- 172.21.15.86:0/4018149275 >> v1:172.21.15.86:6789/0 conn(0x7f63c41642a0 legacy=0x7f63c40d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:07.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.914+0000 7f63c2000640 1 -- 172.21.15.86:0/4018149275 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c40ce3e0 con 0x7f63c40d1df0 2024-04-25T14:09:07.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.914+0000 7f63c2000640 1 --2- 172.21.15.86:0/4018149275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63c40d1df0 0x7f63c40b0150 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f63b00097c0 tx=0x7f63b001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16cf097e09b49178 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:07.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.914+0000 7f63c1600640 1 -- 172.21.15.86:0/4018149275 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b00b0070 con 0x7f63c40d1df0 2024-04-25T14:09:07.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.914+0000 7f63c1600640 1 -- 172.21.15.86:0/4018149275 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63b00ab600 con 0x7f63c40d1df0 2024-04-25T14:09:07.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.914+0000 7f63c1600640 1 -- 172.21.15.86:0/4018149275 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b00ab9d0 con 0x7f63c40d1df0 2024-04-25T14:09:07.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.918+0000 7f63c9c00640 1 -- 172.21.15.86:0/4018149275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63c40d1df0 msgr2=0x7f63c40b0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:07.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.918+0000 7f63c9c00640 1 --2- 172.21.15.86:0/4018149275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63c40d1df0 0x7f63c40b0150 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f63b00097c0 tx=0x7f63b001c410 comp rx=0 tx=0).stop 2024-04-25T14:09:07.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.918+0000 7f63c9c00640 1 -- 172.21.15.86:0/4018149275 shutdown_connections 2024-04-25T14:09:07.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.918+0000 7f63c9c00640 1 --2- 172.21.15.86:0/4018149275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63c40b0690 0x7f63c40d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:07.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.918+0000 7f63c9c00640 1 --2- 172.21.15.86:0/4018149275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63c40d1df0 0x7f63c40b0150 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:07.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.918+0000 7f63c9c00640 1 -- 172.21.15.86:0/4018149275 >> 172.21.15.86:0/4018149275 conn(0x7f63c40b4650 msgr2=0x7f63c40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:07.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.918+0000 7f63c9c00640 1 -- 172.21.15.86:0/4018149275 shutdown_connections 2024-04-25T14:09:07.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.918+0000 7f63c9c00640 1 -- 172.21.15.86:0/4018149275 wait complete. 2024-04-25T14:09:07.919 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:07.918+0000 7f89ff72a180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:09:07.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.918+0000 7f63c9c00640 1 Processor -- start 2024-04-25T14:09:07.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.918+0000 7f63c9c00640 1 -- start start 2024-04-25T14:09:07.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.922+0000 7f63c9c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63c40b0690 0x7f63c409ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:07.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.922+0000 7f63c9c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63c41642a0 0x7f63c4157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:07.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.922+0000 7f63c9c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63c4157c80 0x7f63c415b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:07.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.922+0000 7f63c9c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f63c4164830 con 0x7f63c4157c80 2024-04-25T14:09:07.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.922+0000 7f63c9c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f63c41646b0 con 0x7f63c41642a0 2024-04-25T14:09:07.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.922+0000 7f63c9c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f63c4160da0 con 0x7f63c40b0690 2024-04-25T14:09:07.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.922+0000 7f63c3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63c4157c80 0x7f63c415b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:07.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.922+0000 7f63c2a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63c40b0690 0x7f63c409ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:07.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.922+0000 7f63c2000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63c41642a0 0x7f63c4157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:07.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.922+0000 7f63c2a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63c40b0690 0x7f63c409ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:34418/0 (socket says 172.21.15.86:34418) 2024-04-25T14:09:07.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.922+0000 7f63c2000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63c41642a0 0x7f63c4157740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:51088/0 (socket says 172.21.15.86:51088) 2024-04-25T14:09:07.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.922+0000 7f63c2a00640 1 -- 172.21.15.86:0/2785926681 learned_addr learned my addr 172.21.15.86:0/2785926681 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:07.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.922+0000 7f63c3400640 1 -- 172.21.15.86:0/2785926681 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63c40b0690 msgr2=0x7f63c409ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:07.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.922+0000 7f63c3400640 1 --2- 172.21.15.86:0/2785926681 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63c40b0690 0x7f63c409ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:07.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.922+0000 7f63c3400640 1 -- 172.21.15.86:0/2785926681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63c41642a0 msgr2=0x7f63c4157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:07.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.922+0000 7f63c3400640 1 --2- 172.21.15.86:0/2785926681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63c41642a0 0x7f63c4157740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:07.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.922+0000 7f63c3400640 1 -- 172.21.15.86:0/2785926681 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c41fcc00 con 0x7f63c4157c80 2024-04-25T14:09:07.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.922+0000 7f63c3400640 1 --2- 172.21.15.86:0/2785926681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63c4157c80 0x7f63c415b2f0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f63bc00eb70 tx=0x7f63bc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:07.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.922+0000 7f63baa00640 1 -- 172.21.15.86:0/2785926681 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc018020 con 0x7f63c4157c80 2024-04-25T14:09:07.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.922+0000 7f63baa00640 1 -- 172.21.15.86:0/2785926681 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63bc00cd20 con 0x7f63c4157c80 2024-04-25T14:09:07.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.922+0000 7f63baa00640 1 -- 172.21.15.86:0/2785926681 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc0146c0 con 0x7f63c4157c80 2024-04-25T14:09:07.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.922+0000 7f63c9c00640 1 -- 172.21.15.86:0/2785926681 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c41fcee0 con 0x7f63c4157c80 2024-04-25T14:09:07.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.922+0000 7f63c9c00640 1 -- 172.21.15.86:0/2785926681 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c41fd4a0 con 0x7f63c4157c80 2024-04-25T14:09:07.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.922+0000 7f63c9c00640 1 -- 172.21.15.86:0/2785926681 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63c41581a0 con 0x7f63c4157c80 2024-04-25T14:09:07.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.926+0000 7f63baa00640 1 -- 172.21.15.86:0/2785926681 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 32) ==== 93253+0+0 (secure 0 0 0) 0x7f63bc01b020 con 0x7f63c4157c80 2024-04-25T14:09:07.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.926+0000 7f63baa00640 1 --2- 172.21.15.86:0/2785926681 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f63a406f4c0 0x7f63a40719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:07.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.926+0000 7f63baa00640 1 -- 172.21.15.86:0/2785926681 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6445+0+0 (secure 0 0 0) 0x7f63bc090800 con 0x7f63c4157c80 2024-04-25T14:09:07.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.926+0000 7f63c2a00640 1 -- 172.21.15.86:0/2785926681 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f63a406f4c0 msgr2=0x7f63a40719d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/132942210 2024-04-25T14:09:07.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.926+0000 7f63c2a00640 1 --2- 172.21.15.86:0/2785926681 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f63a406f4c0 0x7f63a40719d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:09:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:07.930+0000 7f63baa00640 1 -- 172.21.15.86:0/2785926681 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f63bc022030 con 0x7f63c4157c80 2024-04-25T14:09:07.994 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:07.994+0000 7f89ff72a180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:09:08.034 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:08.036+0000 7fce4717e180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:09:08.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.090+0000 7f63c9c00640 1 -- 172.21.15.86:0/2785926681 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f63c409f1a0 con 0x7f63c4157c80 2024-04-25T14:09:08.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.090+0000 7f63baa00640 1 -- 172.21.15.86:0/2785926681 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+17386 (secure 0 0 0) 0x7f63c409f1a0 con 0x7f63c4157c80 2024-04-25T14:09:08.093 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:08.093 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":88,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:09:03.346244+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:08:43.505988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T14:08:44.054628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/3132647616":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/1564187566":"2024-04-26T14:09:03.346201+0000","172.21.15.86:6833/3589556609":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/1099362620":"2024-04-26T14:09:03.346201+0000","172.21.15.86:6832/3589556609":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/3481243210":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/2509210906":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/3722630918":"2024-04-26T14:09:03.346201+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:09:08.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.094+0000 7f63c9c00640 1 -- 172.21.15.86:0/2785926681 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f63a406f4c0 msgr2=0x7f63a40719d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:09:08.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.094+0000 7f63c9c00640 1 --2- 172.21.15.86:0/2785926681 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f63a406f4c0 0x7f63a40719d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:08.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.094+0000 7f63c9c00640 1 -- 172.21.15.86:0/2785926681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63c4157c80 msgr2=0x7f63c415b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:08.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.094+0000 7f63c9c00640 1 --2- 172.21.15.86:0/2785926681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63c4157c80 0x7f63c415b2f0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f63bc00eb70 tx=0x7f63bc00c490 comp rx=0 tx=0).stop 2024-04-25T14:09:08.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.094+0000 7f63c9c00640 1 -- 172.21.15.86:0/2785926681 shutdown_connections 2024-04-25T14:09:08.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.094+0000 7f63c9c00640 1 --2- 172.21.15.86:0/2785926681 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f63a406f4c0 0x7f63a40719d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:08.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.094+0000 7f63c9c00640 1 --2- 172.21.15.86:0/2785926681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63c4157c80 0x7f63c415b2f0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:08.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.094+0000 7f63c9c00640 1 --2- 172.21.15.86:0/2785926681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63c41642a0 0x7f63c4157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:08.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.094+0000 7f63c9c00640 1 --2- 172.21.15.86:0/2785926681 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63c40b0690 0x7f63c409ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:08.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.094+0000 7f63c9c00640 1 -- 172.21.15.86:0/2785926681 >> 172.21.15.86:0/2785926681 conn(0x7f63c40b4650 msgr2=0x7f63c4161760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:08.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.094+0000 7f63c9c00640 1 -- 172.21.15.86:0/2785926681 shutdown_connections 2024-04-25T14:09:08.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.094+0000 7f63c9c00640 1 -- 172.21.15.86:0/2785926681 wait complete. 2024-04-25T14:09:08.116 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:08:43.505988+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:08:44.054628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:09:08.116 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:09:08.149 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:08.146+0000 7f89ff72a180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:09:08.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.298+0000 7f4f39000640 1 Processor -- start 2024-04-25T14:09:08.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.298+0000 7f4f39000640 1 -- start start 2024-04-25T14:09:08.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.302+0000 7f4f39000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4f34071530 0x7f4f3415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:08.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.302+0000 7f4f39000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f341616e0 0x7f4f34161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:08.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.302+0000 7f4f39000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f340cd8d0 con 0x7f4f340d2c80 2024-04-25T14:09:08.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.302+0000 7f4f39000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f4f340ce720 con 0x7f4f34071530 2024-04-25T14:09:08.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.302+0000 7f4f39000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4f340ce970 con 0x7f4f341616e0 2024-04-25T14:09:08.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.302+0000 7f4f2be00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4f34071530 0x7f4f3415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:08.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.302+0000 7f4f32800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f341616e0 0x7f4f34161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:08.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.302+0000 7f4f2be00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4f34071530 0x7f4f3415dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:51096/0 (socket says 172.21.15.86:51096) 2024-04-25T14:09:08.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.302+0000 7f4f2be00640 1 -- 172.21.15.86:0/3424785337 learned_addr learned my addr 172.21.15.86:0/3424785337 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.302+0000 7f4f2be00640 1 -- 172.21.15.86:0/3424785337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f341616e0 msgr2=0x7f4f34161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.302+0000 7f4f2be00640 1 --2- 172.21.15.86:0/3424785337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f341616e0 0x7f4f34161af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.302+0000 7f4f2be00640 1 -- 172.21.15.86:0/3424785337 >> v1:172.21.15.155:6789/0 conn(0x7f4f340d2c80 legacy=0x7f4f34070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.302+0000 7f4f2be00640 1 -- 172.21.15.86:0/3424785337 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f340d1be0 con 0x7f4f34071530 2024-04-25T14:09:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.302+0000 7f4f31400640 1 -- 172.21.15.86:0/3424785337 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3365652794 0 0) 0x7f4f340cd8d0 con 0x7f4f340d2c80 2024-04-25T14:09:08.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.306+0000 7f4f2be00640 1 --2- 172.21.15.86:0/3424785337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f34071530 0x7f4f3415dc90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c006fd0 tx=0x7f4f1c01ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d77804beb32289b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:08.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.306+0000 7f4f31400640 1 -- 172.21.15.86:0/3424785337 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c0b5070 con 0x7f4f34071530 2024-04-25T14:09:08.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.306+0000 7f4f31400640 1 -- 172.21.15.86:0/3424785337 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f1c0a8a30 con 0x7f4f34071530 2024-04-25T14:09:08.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.306+0000 7f4f31400640 1 -- 172.21.15.86:0/3424785337 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c0a8e00 con 0x7f4f34071530 2024-04-25T14:09:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.306+0000 7f4f39000640 1 -- 172.21.15.86:0/3424785337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f34071530 msgr2=0x7f4f3415dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.306+0000 7f4f39000640 1 --2- 172.21.15.86:0/3424785337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f34071530 0x7f4f3415dc90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c006fd0 tx=0x7f4f1c01ed30 comp rx=0 tx=0).stop 2024-04-25T14:09:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.306+0000 7f4f39000640 1 -- 172.21.15.86:0/3424785337 shutdown_connections 2024-04-25T14:09:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.306+0000 7f4f39000640 1 --2- 172.21.15.86:0/3424785337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f341616e0 0x7f4f34161af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.306+0000 7f4f39000640 1 --2- 172.21.15.86:0/3424785337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f34071530 0x7f4f3415dc90 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.306+0000 7f4f39000640 1 -- 172.21.15.86:0/3424785337 >> 172.21.15.86:0/3424785337 conn(0x7f4f340cffc0 msgr2=0x7f4f340d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.306+0000 7f4f39000640 1 -- 172.21.15.86:0/3424785337 shutdown_connections 2024-04-25T14:09:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.306+0000 7f4f39000640 1 -- 172.21.15.86:0/3424785337 wait complete. 2024-04-25T14:09:08.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.306+0000 7f4f39000640 1 Processor -- start 2024-04-25T14:09:08.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.306+0000 7f4f39000640 1 -- start start 2024-04-25T14:09:08.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.310+0000 7f4f39000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f340d2c80 0x7f4f34150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:08.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.310+0000 7f4f39000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f341616e0 0x7f4f341607d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:08.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.310+0000 7f4f39000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f34150d60 0x7f4f34156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:08.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.310+0000 7f4f39000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4f340d2480 con 0x7f4f34150d60 2024-04-25T14:09:08.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.310+0000 7f4f39000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4f340d21a0 con 0x7f4f341616e0 2024-04-25T14:09:08.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.310+0000 7f4f39000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4f340d29a0 con 0x7f4f340d2c80 2024-04-25T14:09:08.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.310+0000 7f4f31e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f340d2c80 0x7f4f34150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:08.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.310+0000 7f4f31e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f340d2c80 0x7f4f34150630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:34442/0 (socket says 172.21.15.86:34442) 2024-04-25T14:09:08.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.310+0000 7f4f31e00640 1 -- 172.21.15.86:0/4177814742 learned_addr learned my addr 172.21.15.86:0/4177814742 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:08.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.310+0000 7f4f2be00640 1 --2- 172.21.15.86:0/4177814742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f341616e0 0x7f4f341607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.310+0000 7f4f32800640 1 --2- 172.21.15.86:0/4177814742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f34150d60 0x7f4f34156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.310+0000 7f4f2be00640 1 -- 172.21.15.86:0/4177814742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f340d2c80 msgr2=0x7f4f34150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.310+0000 7f4f2be00640 1 --2- 172.21.15.86:0/4177814742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f340d2c80 0x7f4f34150630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.310+0000 7f4f2be00640 1 -- 172.21.15.86:0/4177814742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f34150d60 msgr2=0x7f4f34156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.310+0000 7f4f2be00640 1 --2- 172.21.15.86:0/4177814742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f34150d60 0x7f4f34156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.310+0000 7f4f2be00640 1 -- 172.21.15.86:0/4177814742 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f34150b70 con 0x7f4f341616e0 2024-04-25T14:09:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.310+0000 7f4f31e00640 1 --2- 172.21.15.86:0/4177814742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f340d2c80 0x7f4f34150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:09:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.310+0000 7f4f2be00640 1 --2- 172.21.15.86:0/4177814742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f341616e0 0x7f4f341607d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0af000 tx=0x7f4f1c01bad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:08.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.310+0000 7f4f2a000640 1 -- 172.21.15.86:0/4177814742 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c0be070 con 0x7f4f341616e0 2024-04-25T14:09:08.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.310+0000 7f4f2a000640 1 -- 172.21.15.86:0/4177814742 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f1c003620 con 0x7f4f341616e0 2024-04-25T14:09:08.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.310+0000 7f4f39000640 1 -- 172.21.15.86:0/4177814742 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f3415f730 con 0x7f4f341616e0 2024-04-25T14:09:08.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.310+0000 7f4f2a000640 1 -- 172.21.15.86:0/4177814742 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c0b5070 con 0x7f4f341616e0 2024-04-25T14:09:08.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.310+0000 7f4f39000640 1 -- 172.21.15.86:0/4177814742 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f3415fcf0 con 0x7f4f341616e0 2024-04-25T14:09:08.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.310+0000 7f4f39000640 1 -- 172.21.15.86:0/4177814742 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f3415f490 con 0x7f4f341616e0 2024-04-25T14:09:08.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.314+0000 7f4f2a000640 1 -- 172.21.15.86:0/4177814742 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 32) ==== 93253+0+0 (secure 0 0 0) 0x7f4f1c0c1070 con 0x7f4f341616e0 2024-04-25T14:09:08.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.314+0000 7f4f2a000640 1 --2- 172.21.15.86:0/4177814742 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f4f1006f4c0 0x7f4f100719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:08.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.314+0000 7f4f31e00640 1 -- 172.21.15.86:0/4177814742 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f4f1006f4c0 msgr2=0x7f4f100719d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/132942210 2024-04-25T14:09:08.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.314+0000 7f4f2a000640 1 -- 172.21.15.86:0/4177814742 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6445+0+0 (secure 0 0 0) 0x7f4f1c12c2b0 con 0x7f4f341616e0 2024-04-25T14:09:08.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.314+0000 7f4f31e00640 1 --2- 172.21.15.86:0/4177814742 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f4f1006f4c0 0x7f4f100719d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:09:08.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.318+0000 7f4f2a000640 1 -- 172.21.15.86:0/4177814742 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4f3415f490 con 0x7f4f341616e0 2024-04-25T14:09:08.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.478+0000 7f4f39000640 1 -- 172.21.15.86:0/4177814742 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4f340720e0 con 0x7f4f341616e0 2024-04-25T14:09:08.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.478+0000 7f4f2a000640 1 -- 172.21.15.86:0/4177814742 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f4f1c018890 con 0x7f4f341616e0 2024-04-25T14:09:08.481 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:09:08.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.482+0000 7f4f39000640 1 -- 172.21.15.86:0/4177814742 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f4f1006f4c0 msgr2=0x7f4f100719d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:09:08.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.482+0000 7f4f39000640 1 --2- 172.21.15.86:0/4177814742 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f4f1006f4c0 0x7f4f100719d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:08.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.482+0000 7f4f39000640 1 -- 172.21.15.86:0/4177814742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f341616e0 msgr2=0x7f4f341607d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:08.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.482+0000 7f4f39000640 1 --2- 172.21.15.86:0/4177814742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f341616e0 0x7f4f341607d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0af000 tx=0x7f4f1c01bad0 comp rx=0 tx=0).stop 2024-04-25T14:09:08.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.482+0000 7f4f39000640 1 -- 172.21.15.86:0/4177814742 shutdown_connections 2024-04-25T14:09:08.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.482+0000 7f4f39000640 1 --2- 172.21.15.86:0/4177814742 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f4f1006f4c0 0x7f4f100719d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:08.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.482+0000 7f4f39000640 1 --2- 172.21.15.86:0/4177814742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f34150d60 0x7f4f34156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:08.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.482+0000 7f4f39000640 1 --2- 172.21.15.86:0/4177814742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f341616e0 0x7f4f341607d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:08.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.482+0000 7f4f39000640 1 --2- 172.21.15.86:0/4177814742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f340d2c80 0x7f4f34150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:08.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.482+0000 7f4f39000640 1 -- 172.21.15.86:0/4177814742 >> 172.21.15.86:0/4177814742 conn(0x7f4f340cffc0 msgr2=0x7f4f341599c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:08.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.482+0000 7f4f39000640 1 -- 172.21.15.86:0/4177814742 shutdown_connections 2024-04-25T14:09:08.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.482+0000 7f4f39000640 1 -- 172.21.15.86:0/4177814742 wait complete. 2024-04-25T14:09:08.503 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:09:08.579 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:08.580+0000 7fce4717e180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:09:08.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.654+0000 7f5d6e000640 1 Processor -- start 2024-04-25T14:09:08.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.654+0000 7f5d6e000640 1 -- start start 2024-04-25T14:09:08.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.654+0000 7f5d6e000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d6806e970 0x7f5d6816abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:08.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.654+0000 7f5d6e000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d6806ef90 con 0x7f5d68071530 2024-04-25T14:09:08.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.654+0000 7f5d6e000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d68163a80 con 0x7f5d680cf970 2024-04-25T14:09:08.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.654+0000 7f5d6e000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5d680cedf0 con 0x7f5d6806e970 2024-04-25T14:09:08.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.654+0000 7f5d66a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5d68071530 0x7f5d6806e260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:46216/0 (socket says 172.21.15.86:46216) 2024-04-25T14:09:08.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.654+0000 7f5d66a00640 1 -- 172.21.15.86:0/851694995 learned_addr learned my addr 172.21.15.86:0/851694995 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:08.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.654+0000 7f5d67e00640 1 --2- 172.21.15.86:0/851694995 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d6806e970 0x7f5d6816abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:08.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.658+0000 7f5d66000640 1 -- 172.21.15.86:0/851694995 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1216903307 0 0) 0x7f5d6806ef90 con 0x7f5d68071530 2024-04-25T14:09:08.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.658+0000 7f5d66000640 1 -- 172.21.15.86:0/851694995 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d4c003530 con 0x7f5d68071530 2024-04-25T14:09:08.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.658+0000 7f5d66000640 1 -- 172.21.15.86:0/851694995 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 632796644 0 0) 0x7f5d68163a80 con 0x7f5d680cf970 2024-04-25T14:09:08.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.658+0000 7f5d66000640 1 -- 172.21.15.86:0/851694995 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d6806ef90 con 0x7f5d680cf970 2024-04-25T14:09:08.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.658+0000 7f5d67e00640 1 -- 172.21.15.86:0/851694995 >> v1:172.21.15.155:6789/0 conn(0x7f5d680cf970 legacy=0x7f5d68070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:08.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.658+0000 7f5d67e00640 1 -- 172.21.15.86:0/851694995 >> v1:172.21.15.86:6789/0 conn(0x7f5d68071530 legacy=0x7f5d6806e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:08.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.658+0000 7f5d67e00640 1 -- 172.21.15.86:0/851694995 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d68163d60 con 0x7f5d6806e970 2024-04-25T14:09:08.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.658+0000 7f5d67e00640 1 --2- 172.21.15.86:0/851694995 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d6806e970 0x7f5d6816abc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009dc0 tx=0x7f5d6001c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9844529d7a08e1c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:08.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.658+0000 7f5d66000640 1 -- 172.21.15.86:0/851694995 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d600b2070 con 0x7f5d6806e970 2024-04-25T14:09:08.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.658+0000 7f5d66000640 1 -- 172.21.15.86:0/851694995 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d600a5850 con 0x7f5d6806e970 2024-04-25T14:09:08.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.658+0000 7f5d66000640 1 -- 172.21.15.86:0/851694995 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d600a5c20 con 0x7f5d6806e970 2024-04-25T14:09:08.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.662+0000 7f5d6e000640 1 -- 172.21.15.86:0/851694995 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d6806e970 msgr2=0x7f5d6816abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:08.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.662+0000 7f5d6e000640 1 --2- 172.21.15.86:0/851694995 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d6806e970 0x7f5d6816abc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009dc0 tx=0x7f5d6001c990 comp rx=0 tx=0).stop 2024-04-25T14:09:08.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.662+0000 7f5d6e000640 1 -- 172.21.15.86:0/851694995 shutdown_connections 2024-04-25T14:09:08.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.662+0000 7f5d6e000640 1 --2- 172.21.15.86:0/851694995 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d6806e970 0x7f5d6816abc0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:08.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.662+0000 7f5d6e000640 1 -- 172.21.15.86:0/851694995 >> 172.21.15.86:0/851694995 conn(0x7f5d680cffc0 msgr2=0x7f5d680d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:08.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.662+0000 7f5d6e000640 1 -- 172.21.15.86:0/851694995 shutdown_connections 2024-04-25T14:09:08.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.662+0000 7f5d6e000640 1 -- 172.21.15.86:0/851694995 wait complete. 2024-04-25T14:09:08.664 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:08.664+0000 7fce4717e180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:09:08.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.662+0000 7f5d6e000640 1 Processor -- start 2024-04-25T14:09:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.662+0000 7f5d6e000640 1 -- start start 2024-04-25T14:09:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.666+0000 7f5d6e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d6806e970 0x7f5d680b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.666+0000 7f5d6e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d680cf970 0x7f5d680b8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.666+0000 7f5d6e000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d680b8870 0x7f5d681477b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.666+0000 7f5d6e000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6816dcb0 con 0x7f5d680cf970 2024-04-25T14:09:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.666+0000 7f5d6e000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6816db30 con 0x7f5d6806e970 2024-04-25T14:09:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.666+0000 7f5d6e000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5d6816de30 con 0x7f5d680b8870 2024-04-25T14:09:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.666+0000 7f5d66a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d680cf970 0x7f5d680b8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.666+0000 7f5d67400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d6806e970 0x7f5d680b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.666+0000 7f5d66a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d680cf970 0x7f5d680b8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39304/0 (socket says 172.21.15.86:39304) 2024-04-25T14:09:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.666+0000 7f5d66a00640 1 -- 172.21.15.86:0/3382802757 learned_addr learned my addr 172.21.15.86:0/3382802757 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.666+0000 7f5d67e00640 1 --2- 172.21.15.86:0/3382802757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d680b8870 0x7f5d681477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.666+0000 7f5d67400640 1 -- 172.21.15.86:0/3382802757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d680b8870 msgr2=0x7f5d681477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.666+0000 7f5d67400640 1 --2- 172.21.15.86:0/3382802757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d680b8870 0x7f5d681477b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.666+0000 7f5d67400640 1 -- 172.21.15.86:0/3382802757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d680cf970 msgr2=0x7f5d680b8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.666+0000 7f5d67400640 1 --2- 172.21.15.86:0/3382802757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d680cf970 0x7f5d680b8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.666+0000 7f5d67400640 1 -- 172.21.15.86:0/3382802757 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d680b8c80 con 0x7f5d6806e970 2024-04-25T14:09:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.666+0000 7f5d66a00640 1 --2- 172.21.15.86:0/3382802757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d680cf970 0x7f5d680b8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:09:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.666+0000 7f5d67400640 1 --2- 172.21.15.86:0/3382802757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d6806e970 0x7f5d680b25c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5d5800c340 tx=0x7f5d58001e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:08.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.666+0000 7f5d5f400640 1 -- 172.21.15.86:0/3382802757 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d58017070 con 0x7f5d6806e970 2024-04-25T14:09:08.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.666+0000 7f5d5f400640 1 -- 172.21.15.86:0/3382802757 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d580052f0 con 0x7f5d6806e970 2024-04-25T14:09:08.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.666+0000 7f5d5f400640 1 -- 172.21.15.86:0/3382802757 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d58003840 con 0x7f5d6806e970 2024-04-25T14:09:08.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.666+0000 7f5d6e000640 1 -- 172.21.15.86:0/3382802757 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d68148430 con 0x7f5d6806e970 2024-04-25T14:09:08.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.666+0000 7f5d6e000640 1 -- 172.21.15.86:0/3382802757 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d680affe0 con 0x7f5d6806e970 2024-04-25T14:09:08.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.666+0000 7f5d6e000640 1 -- 172.21.15.86:0/3382802757 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d680720e0 con 0x7f5d6806e970 2024-04-25T14:09:08.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.670+0000 7f5d5f400640 1 -- 172.21.15.86:0/3382802757 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 32) ==== 93253+0+0 (secure 0 0 0) 0x7f5d5801a070 con 0x7f5d6806e970 2024-04-25T14:09:08.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.670+0000 7f5d5f400640 1 --2- 172.21.15.86:0/3382802757 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f5d4806f490 0x7f5d480719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:08.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.670+0000 7f5d5f400640 1 -- 172.21.15.86:0/3382802757 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6445+0+0 (secure 0 0 0) 0x7f5d5808e960 con 0x7f5d6806e970 2024-04-25T14:09:08.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.670+0000 7f5d66a00640 1 -- 172.21.15.86:0/3382802757 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f5d4806f490 msgr2=0x7f5d480719a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/132942210 2024-04-25T14:09:08.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.670+0000 7f5d66a00640 1 --2- 172.21.15.86:0/3382802757 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f5d4806f490 0x7f5d480719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:09:08.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.674+0000 7f5d5f400640 1 -- 172.21.15.86:0/3382802757 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5d680b8c80 con 0x7f5d6806e970 2024-04-25T14:09:08.723 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:08.722+0000 7f89ff72a180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:09:08.811 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:08.810+0000 7f89ff72a180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:09:08.830 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:08.832+0000 7fce4717e180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:09:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.834+0000 7f5d6e000640 1 -- 172.21.15.86:0/3382802757 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5d680b94c0 con 0x7f5d6806e970 2024-04-25T14:09:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.838+0000 7f5d5f400640 1 -- 172.21.15.86:0/3382802757 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f5d5805b0d0 con 0x7f5d6806e970 2024-04-25T14:09:08.838 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:09:08.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.838+0000 7f5d6e000640 1 -- 172.21.15.86:0/3382802757 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f5d4806f490 msgr2=0x7f5d480719a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:09:08.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.838+0000 7f5d6e000640 1 --2- 172.21.15.86:0/3382802757 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f5d4806f490 0x7f5d480719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:08.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.838+0000 7f5d6e000640 1 -- 172.21.15.86:0/3382802757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d6806e970 msgr2=0x7f5d680b25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:08.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.838+0000 7f5d6e000640 1 --2- 172.21.15.86:0/3382802757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d6806e970 0x7f5d680b25c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5d5800c340 tx=0x7f5d58001e80 comp rx=0 tx=0).stop 2024-04-25T14:09:08.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.838+0000 7f5d6e000640 1 -- 172.21.15.86:0/3382802757 shutdown_connections 2024-04-25T14:09:08.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.838+0000 7f5d6e000640 1 --2- 172.21.15.86:0/3382802757 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f5d4806f490 0x7f5d480719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:08.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.838+0000 7f5d6e000640 1 --2- 172.21.15.86:0/3382802757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d680b8870 0x7f5d681477b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:08.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.838+0000 7f5d6e000640 1 --2- 172.21.15.86:0/3382802757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d680cf970 0x7f5d680b8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:08.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.838+0000 7f5d6e000640 1 --2- 172.21.15.86:0/3382802757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d6806e970 0x7f5d680b25c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:08.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.838+0000 7f5d6e000640 1 -- 172.21.15.86:0/3382802757 >> 172.21.15.86:0/3382802757 conn(0x7f5d680cffc0 msgr2=0x7f5d680cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:08.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.838+0000 7f5d6e000640 1 -- 172.21.15.86:0/3382802757 shutdown_connections 2024-04-25T14:09:08.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:08.838+0000 7f5d6e000640 1 -- 172.21.15.86:0/3382802757 wait complete. 2024-04-25T14:09:08.861 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:09:08.914 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:08.916+0000 7fce4717e180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:09:08.981 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:08.978+0000 7f89ff72a180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:09:09.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.010+0000 7fcc00a00640 1 Processor -- start 2024-04-25T14:09:09.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.010+0000 7fcc00a00640 1 -- start start 2024-04-25T14:09:09.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.014+0000 7fcc00a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcbfc0d2c80 0x7fcbfc15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.014+0000 7fcc00a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbfc1616e0 0x7fcbfc161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.014+0000 7fcc00a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbfc0cd8d0 con 0x7fcbfc070e20 2024-04-25T14:09:09.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.014+0000 7fcc00a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fcbfc0ce720 con 0x7fcbfc0d2c80 2024-04-25T14:09:09.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.014+0000 7fcc00a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcbfc0ce970 con 0x7fcbfc1616e0 2024-04-25T14:09:09.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.014+0000 7fcbf9400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fcbfc070e20 0x7fcbfc071230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42752/0 (socket says 172.21.15.86:42752) 2024-04-25T14:09:09.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.014+0000 7fcbf9400640 1 -- 172.21.15.86:0/2896133145 learned_addr learned my addr 172.21.15.86:0/2896133145 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:09.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.014+0000 7fcbfa800640 1 --2- 172.21.15.86:0/2896133145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbfc1616e0 0x7fcbfc161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.014+0000 7fcbf9e00640 1 --2- 172.21.15.86:0/2896133145 >> v2:172.21.15.155:3300/0 conn(0x7fcbfc0d2c80 0x7fcbfc15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.014+0000 7fcbf8a00640 1 -- 172.21.15.86:0/2896133145 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 701811316 0 0) 0x7fcbfc0cd8d0 con 0x7fcbfc070e20 2024-04-25T14:09:09.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.014+0000 7fcbf8a00640 1 -- 172.21.15.86:0/2896133145 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcbe0003530 con 0x7fcbfc070e20 2024-04-25T14:09:09.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.014+0000 7fcbfa800640 1 -- 172.21.15.86:0/2896133145 >> v2:172.21.15.155:3300/0 conn(0x7fcbfc0d2c80 msgr2=0x7fcbfc15dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:09.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.014+0000 7fcbfa800640 1 --2- 172.21.15.86:0/2896133145 >> v2:172.21.15.155:3300/0 conn(0x7fcbfc0d2c80 0x7fcbfc15dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.014+0000 7fcbfa800640 1 -- 172.21.15.86:0/2896133145 >> v1:172.21.15.86:6789/0 conn(0x7fcbfc070e20 legacy=0x7fcbfc071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:09.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.014+0000 7fcbfa800640 1 -- 172.21.15.86:0/2896133145 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbfc0d1be0 con 0x7fcbfc1616e0 2024-04-25T14:09:09.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.014+0000 7fcbf9e00640 1 --2- 172.21.15.86:0/2896133145 >> v2:172.21.15.155:3300/0 conn(0x7fcbfc0d2c80 0x7fcbfc15dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:09:09.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.018+0000 7fcbfa800640 1 --2- 172.21.15.86:0/2896133145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbfc1616e0 0x7fcbfc161af0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4009d90 tx=0x7fcbf401f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de7a8c9a0562e736 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:09.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.018+0000 7fcbf8a00640 1 -- 172.21.15.86:0/2896133145 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf40b4070 con 0x7fcbfc1616e0 2024-04-25T14:09:09.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.018+0000 7fcbf8a00640 1 -- 172.21.15.86:0/2896133145 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbf40af600 con 0x7fcbfc1616e0 2024-04-25T14:09:09.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.018+0000 7fcbf8a00640 1 -- 172.21.15.86:0/2896133145 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf40af9d0 con 0x7fcbfc1616e0 2024-04-25T14:09:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.018+0000 7fcc00a00640 1 -- 172.21.15.86:0/2896133145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbfc1616e0 msgr2=0x7fcbfc161af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.018+0000 7fcc00a00640 1 --2- 172.21.15.86:0/2896133145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbfc1616e0 0x7fcbfc161af0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4009d90 tx=0x7fcbf401f150 comp rx=0 tx=0).stop 2024-04-25T14:09:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.018+0000 7fcc00a00640 1 -- 172.21.15.86:0/2896133145 shutdown_connections 2024-04-25T14:09:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.018+0000 7fcc00a00640 1 --2- 172.21.15.86:0/2896133145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbfc1616e0 0x7fcbfc161af0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.018+0000 7fcc00a00640 1 --2- 172.21.15.86:0/2896133145 >> v2:172.21.15.155:3300/0 conn(0x7fcbfc0d2c80 0x7fcbfc15dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.018+0000 7fcc00a00640 1 -- 172.21.15.86:0/2896133145 >> 172.21.15.86:0/2896133145 conn(0x7fcbfc0cffc0 msgr2=0x7fcbfc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:09.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.018+0000 7fcc00a00640 1 -- 172.21.15.86:0/2896133145 shutdown_connections 2024-04-25T14:09:09.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.018+0000 7fcc00a00640 1 -- 172.21.15.86:0/2896133145 wait complete. 2024-04-25T14:09:09.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.022+0000 7fcc00a00640 1 Processor -- start 2024-04-25T14:09:09.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.022+0000 7fcc00a00640 1 -- start start 2024-04-25T14:09:09.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.022+0000 7fcc00a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbfc070e20 0x7fcbfc0b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.022+0000 7fcc00a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbfc0d2c80 0x7fcbfc1e94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.022+0000 7fcc00a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbfc1616e0 0x7fcbfc1f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.022+0000 7fcc00a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc0d2480 con 0x7fcbfc1616e0 2024-04-25T14:09:09.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.022+0000 7fcc00a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc0d21a0 con 0x7fcbfc070e20 2024-04-25T14:09:09.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.022+0000 7fcc00a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcbfc0d29a0 con 0x7fcbfc0d2c80 2024-04-25T14:09:09.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.022+0000 7fcbfa800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbfc1616e0 0x7fcbfc1f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.022+0000 7fcbfa800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbfc1616e0 0x7fcbfc1f40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57270/0 (socket says 172.21.15.86:57270) 2024-04-25T14:09:09.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.022+0000 7fcbfa800640 1 -- 172.21.15.86:0/3894161070 learned_addr learned my addr 172.21.15.86:0/3894161070 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:09.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.022+0000 7fcbf9400640 1 --2- 172.21.15.86:0/3894161070 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbfc0d2c80 0x7fcbfc1e94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.022+0000 7fcbf9e00640 1 --2- 172.21.15.86:0/3894161070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbfc070e20 0x7fcbfc0b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.022+0000 7fcbfa800640 1 -- 172.21.15.86:0/3894161070 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbfc0d2c80 msgr2=0x7fcbfc1e94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:09.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.022+0000 7fcbfa800640 1 --2- 172.21.15.86:0/3894161070 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbfc0d2c80 0x7fcbfc1e94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.022+0000 7fcbfa800640 1 -- 172.21.15.86:0/3894161070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbfc070e20 msgr2=0x7fcbfc0b20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:09.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.022+0000 7fcbfa800640 1 --2- 172.21.15.86:0/3894161070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbfc070e20 0x7fcbfc0b20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.022+0000 7fcbfa800640 1 -- 172.21.15.86:0/3894161070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbfc1e8bc0 con 0x7fcbfc1616e0 2024-04-25T14:09:09.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.022+0000 7fcbfa800640 1 --2- 172.21.15.86:0/3894161070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbfc1616e0 0x7fcbfc1f40c0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4009d90 tx=0x7fcbf401bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:09.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.022+0000 7fcbf2000640 1 -- 172.21.15.86:0/3894161070 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf40b4040 con 0x7fcbfc1616e0 2024-04-25T14:09:09.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.022+0000 7fcbf2000640 1 -- 172.21.15.86:0/3894161070 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbf40c1070 con 0x7fcbfc1616e0 2024-04-25T14:09:09.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.022+0000 7fcc00a00640 1 -- 172.21.15.86:0/3894161070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbfc1e7b20 con 0x7fcbfc1616e0 2024-04-25T14:09:09.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.022+0000 7fcbf2000640 1 -- 172.21.15.86:0/3894161070 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4009450 con 0x7fcbfc1616e0 2024-04-25T14:09:09.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.022+0000 7fcc00a00640 1 -- 172.21.15.86:0/3894161070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbfc1e80e0 con 0x7fcbfc1616e0 2024-04-25T14:09:09.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.026+0000 7fcc00a00640 1 -- 172.21.15.86:0/3894161070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbfc0720e0 con 0x7fcbfc1616e0 2024-04-25T14:09:09.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.026+0000 7fcbf2000640 1 -- 172.21.15.86:0/3894161070 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 32) ==== 93253+0+0 (secure 0 0 0) 0x7fcbf40ba6b0 con 0x7fcbfc1616e0 2024-04-25T14:09:09.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.026+0000 7fcbf2000640 1 --2- 172.21.15.86:0/3894161070 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7fcbdc06f490 0x7fcbdc0719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.026+0000 7fcbf2000640 1 -- 172.21.15.86:0/3894161070 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6445+0+0 (secure 0 0 0) 0x7fcbf412df30 con 0x7fcbfc1616e0 2024-04-25T14:09:09.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.026+0000 7fcbf9e00640 1 -- 172.21.15.86:0/3894161070 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7fcbdc06f490 msgr2=0x7fcbdc0719a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/132942210 2024-04-25T14:09:09.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.026+0000 7fcbf9e00640 1 --2- 172.21.15.86:0/3894161070 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7fcbdc06f490 0x7fcbdc0719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:09:09.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.030+0000 7fcbf2000640 1 -- 172.21.15.86:0/3894161070 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcbf4018cb0 con 0x7fcbfc1616e0 2024-04-25T14:09:09.071 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:09.070+0000 7f89ff72a180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:09:09.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.190+0000 7fcc00a00640 1 -- 172.21.15.86:0/3894161070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcbfc076bf0 con 0x7fcbfc1616e0 2024-04-25T14:09:09.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.190+0000 7fcbf2000640 1 -- 172.21.15.86:0/3894161070 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fcbf40fa2f0 con 0x7fcbfc1616e0 2024-04-25T14:09:09.192 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:09:09.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.194+0000 7fcc00a00640 1 -- 172.21.15.86:0/3894161070 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7fcbdc06f490 msgr2=0x7fcbdc0719a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:09:09.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.194+0000 7fcc00a00640 1 --2- 172.21.15.86:0/3894161070 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7fcbdc06f490 0x7fcbdc0719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.194+0000 7fcc00a00640 1 -- 172.21.15.86:0/3894161070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbfc1616e0 msgr2=0x7fcbfc1f40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:09.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.194+0000 7fcc00a00640 1 --2- 172.21.15.86:0/3894161070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbfc1616e0 0x7fcbfc1f40c0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4009d90 tx=0x7fcbf401bef0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.194+0000 7fcc00a00640 1 -- 172.21.15.86:0/3894161070 shutdown_connections 2024-04-25T14:09:09.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.194+0000 7fcc00a00640 1 --2- 172.21.15.86:0/3894161070 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7fcbdc06f490 0x7fcbdc0719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.194+0000 7fcc00a00640 1 --2- 172.21.15.86:0/3894161070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbfc1616e0 0x7fcbfc1f40c0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.194+0000 7fcc00a00640 1 --2- 172.21.15.86:0/3894161070 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbfc0d2c80 0x7fcbfc1e94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.194+0000 7fcc00a00640 1 --2- 172.21.15.86:0/3894161070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbfc070e20 0x7fcbfc0b20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.194+0000 7fcc00a00640 1 -- 172.21.15.86:0/3894161070 >> 172.21.15.86:0/3894161070 conn(0x7fcbfc0cffc0 msgr2=0x7fcbfc0cb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:09.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.194+0000 7fcc00a00640 1 -- 172.21.15.86:0/3894161070 shutdown_connections 2024-04-25T14:09:09.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.194+0000 7fcc00a00640 1 -- 172.21.15.86:0/3894161070 wait complete. 2024-04-25T14:09:09.215 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T14:09:09.235 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:09.236+0000 7fce4717e180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:09:09.297 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:09.296+0000 7fce4717e180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:09:09.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.366+0000 7f2e18600640 1 Processor -- start 2024-04-25T14:09:09.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.366+0000 7f2e18600640 1 -- start start 2024-04-25T14:09:09.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.366+0000 7f2e18600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2e100d2c80 0x7f2e1015dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.366+0000 7f2e18600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2e10070e20 0x7f2e10071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.366+0000 7f2e18600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e100722f0 0x7f2e10072700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.366+0000 7f2e18600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2e100ce720 con 0x7f2e10070e20 2024-04-25T14:09:09.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.366+0000 7f2e18600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f2e10071770 con 0x7f2e100d2c80 2024-04-25T14:09:09.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.366+0000 7f2e18600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2e100cd8d0 con 0x7f2e100722f0 2024-04-25T14:09:09.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.366+0000 7f2e15200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2e10070e20 0x7f2e10071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.366+0000 7f2e15200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2e10070e20 0x7f2e10071230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57278/0 (socket says 172.21.15.86:57278) 2024-04-25T14:09:09.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.366+0000 7f2e15200640 1 -- 172.21.15.86:0/1073499197 learned_addr learned my addr 172.21.15.86:0/1073499197 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:09.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.366+0000 7f2e16600640 1 --2- 172.21.15.86:0/1073499197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e100722f0 0x7f2e10072700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.366+0000 7f2e15c00640 1 --2- 172.21.15.86:0/1073499197 >> v2:172.21.15.155:3300/0 conn(0x7f2e100d2c80 0x7f2e1015dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.366+0000 7f2e15200640 1 -- 172.21.15.86:0/1073499197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e100722f0 msgr2=0x7f2e10072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:09.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.366+0000 7f2e15200640 1 --2- 172.21.15.86:0/1073499197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e100722f0 0x7f2e10072700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.366+0000 7f2e15200640 1 -- 172.21.15.86:0/1073499197 >> v2:172.21.15.155:3300/0 conn(0x7f2e100d2c80 msgr2=0x7f2e1015dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:09.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.366+0000 7f2e15200640 1 --2- 172.21.15.86:0/1073499197 >> v2:172.21.15.155:3300/0 conn(0x7f2e100d2c80 0x7f2e1015dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.366+0000 7f2e15200640 1 -- 172.21.15.86:0/1073499197 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e100ce970 con 0x7f2e10070e20 2024-04-25T14:09:09.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.370+0000 7f2e15200640 1 --2- 172.21.15.86:0/1073499197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e10070e20 0x7f2e10071230 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f2e100d11a0 tx=0x7f2dfc01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afca890af4aeffb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:09.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.370+0000 7f2e0fe00640 1 -- 172.21.15.86:0/1073499197 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc0b5070 con 0x7f2e10070e20 2024-04-25T14:09:09.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.370+0000 7f2e0fe00640 1 -- 172.21.15.86:0/1073499197 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dfc0a89d0 con 0x7f2e10070e20 2024-04-25T14:09:09.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.370+0000 7f2e0fe00640 1 -- 172.21.15.86:0/1073499197 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc0a8da0 con 0x7f2e10070e20 2024-04-25T14:09:09.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.370+0000 7f2e18600640 1 -- 172.21.15.86:0/1073499197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e10070e20 msgr2=0x7f2e10071230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:09.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.370+0000 7f2e18600640 1 --2- 172.21.15.86:0/1073499197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e10070e20 0x7f2e10071230 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f2e100d11a0 tx=0x7f2dfc01ed10 comp rx=0 tx=0).stop 2024-04-25T14:09:09.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.370+0000 7f2e18600640 1 -- 172.21.15.86:0/1073499197 shutdown_connections 2024-04-25T14:09:09.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.370+0000 7f2e18600640 1 --2- 172.21.15.86:0/1073499197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e100722f0 0x7f2e10072700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.370+0000 7f2e18600640 1 --2- 172.21.15.86:0/1073499197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e10070e20 0x7f2e10071230 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.370+0000 7f2e18600640 1 --2- 172.21.15.86:0/1073499197 >> v2:172.21.15.155:3300/0 conn(0x7f2e100d2c80 0x7f2e1015dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.374+0000 7f2e18600640 1 -- 172.21.15.86:0/1073499197 >> 172.21.15.86:0/1073499197 conn(0x7f2e100cffc0 msgr2=0x7f2e100d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:09.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.374+0000 7f2e18600640 1 -- 172.21.15.86:0/1073499197 shutdown_connections 2024-04-25T14:09:09.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.374+0000 7f2e18600640 1 -- 172.21.15.86:0/1073499197 wait complete. 2024-04-25T14:09:09.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.374+0000 7f2e18600640 1 Processor -- start 2024-04-25T14:09:09.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.374+0000 7f2e18600640 1 -- start start 2024-04-25T14:09:09.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.374+0000 7f2e18600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e10070e20 0x7f2e10155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.374+0000 7f2e18600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e100722f0 0x7f2e1015b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.374+0000 7f2e18600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e100d2c80 0x7f2e101f4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.374+0000 7f2e18600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e100d21a0 con 0x7f2e10070e20 2024-04-25T14:09:09.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.374+0000 7f2e18600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2e100d1ec0 con 0x7f2e100d2c80 2024-04-25T14:09:09.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.374+0000 7f2e18600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2e100d2480 con 0x7f2e100722f0 2024-04-25T14:09:09.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.378+0000 7f2e15c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e10070e20 0x7f2e10155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.378+0000 7f2e15c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e10070e20 0x7f2e10155610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57284/0 (socket says 172.21.15.86:57284) 2024-04-25T14:09:09.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.378+0000 7f2e15200640 1 --2- 172.21.15.86:0/2296178051 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e100722f0 0x7f2e1015b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.378+0000 7f2e15c00640 1 -- 172.21.15.86:0/2296178051 learned_addr learned my addr 172.21.15.86:0/2296178051 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:09.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.378+0000 7f2e16600640 1 --2- 172.21.15.86:0/2296178051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e100d2c80 0x7f2e101f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.378+0000 7f2e15200640 1 -- 172.21.15.86:0/2296178051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e100d2c80 msgr2=0x7f2e101f4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:09.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.378+0000 7f2e15200640 1 --2- 172.21.15.86:0/2296178051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e100d2c80 0x7f2e101f4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.378+0000 7f2e15200640 1 -- 172.21.15.86:0/2296178051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e10070e20 msgr2=0x7f2e10155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:09.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.378+0000 7f2e15200640 1 --2- 172.21.15.86:0/2296178051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e10070e20 0x7f2e10155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.378+0000 7f2e15200640 1 -- 172.21.15.86:0/2296178051 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e1015a890 con 0x7f2e100722f0 2024-04-25T14:09:09.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.378+0000 7f2e15c00640 1 --2- 172.21.15.86:0/2296178051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e10070e20 0x7f2e10155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:09:09.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.378+0000 7f2e15200640 1 --2- 172.21.15.86:0/2296178051 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e100722f0 0x7f2e1015b930 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc0af000 tx=0x7f2dfc01bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:09.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.378+0000 7f2e0d600640 1 -- 172.21.15.86:0/2296178051 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc0be070 con 0x7f2e100722f0 2024-04-25T14:09:09.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.378+0000 7f2e0d600640 1 -- 172.21.15.86:0/2296178051 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dfc004240 con 0x7f2e100722f0 2024-04-25T14:09:09.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.378+0000 7f2e0d600640 1 -- 172.21.15.86:0/2296178051 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc0b5040 con 0x7f2e100722f0 2024-04-25T14:09:09.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.378+0000 7f2e18600640 1 -- 172.21.15.86:0/2296178051 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e1015ab70 con 0x7f2e100722f0 2024-04-25T14:09:09.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.378+0000 7f2e18600640 1 -- 172.21.15.86:0/2296178051 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e1015b130 con 0x7f2e100722f0 2024-04-25T14:09:09.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.378+0000 7f2e18600640 1 -- 172.21.15.86:0/2296178051 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e100720e0 con 0x7f2e100722f0 2024-04-25T14:09:09.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.382+0000 7f2e0d600640 1 -- 172.21.15.86:0/2296178051 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 32) ==== 93253+0+0 (secure 0 0 0) 0x7f2dfc0c1070 con 0x7f2e100722f0 2024-04-25T14:09:09.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.382+0000 7f2e0d600640 1 --2- 172.21.15.86:0/2296178051 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f2df006f4e0 0x7f2df00719f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.382+0000 7f2e0d600640 1 -- 172.21.15.86:0/2296178051 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(88..88 src has 1..88) ==== 6445+0+0 (secure 0 0 0) 0x7f2dfc12be10 con 0x7f2e100722f0 2024-04-25T14:09:09.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.382+0000 7f2e15c00640 1 -- 172.21.15.86:0/2296178051 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f2df006f4e0 msgr2=0x7f2df00719f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/132942210 2024-04-25T14:09:09.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.382+0000 7f2e15c00640 1 --2- 172.21.15.86:0/2296178051 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f2df006f4e0 0x7f2df00719f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:09:09.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.386+0000 7f2e0d600640 1 -- 172.21.15.86:0/2296178051 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mgrmap(e 33) ==== 93253+0+0 (secure 0 0 0) 0x7f2dfc0f4680 con 0x7f2e100722f0 2024-04-25T14:09:09.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.390+0000 7f2e0d600640 1 -- 172.21.15.86:0/2296178051 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2dfc018870 con 0x7f2e100722f0 2024-04-25T14:09:09.410 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:09.414+0000 7f89ff72a180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:09:09.474 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:09.474+0000 7f89ff72a180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:09:09.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.486+0000 7f2e0d600640 1 -- 172.21.15.86:0/2296178051 <== mon.2 v2:172.21.15.155:3301/0 8 ==== mgrmap(e 34) ==== 92918+0+0 (secure 0 0 0) 0x7f2dfc127c30 con 0x7f2e100722f0 2024-04-25T14:09:09.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.486+0000 7f2e0d600640 1 -- 172.21.15.86:0/2296178051 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f2df006f4e0 msgr2=0x7f2df00719f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:09:09.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.486+0000 7f2e0d600640 1 --2- 172.21.15.86:0/2296178051 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f2df006f4e0 0x7f2df00719f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.562 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:09.562+0000 7f8981c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:09.562 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:09.562+0000 7f8981c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:09.562 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:09.562+0000 7f8981c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:09.562 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:09.562+0000 7f8981c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:09.562 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:09.562+0000 7f8981c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:09.563 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:09.562+0000 7f897e000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:09.563 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:09.562+0000 7f897e000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:09.563 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:09.562+0000 7f897e000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:09.563 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:09.562+0000 7f897e000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:09.563 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:09.562+0000 7f897e000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:09.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.594+0000 7f2e18600640 1 -- 172.21.15.86:0/2296178051 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2e10076bf0 con 0x7f2e100722f0 2024-04-25T14:09:09.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.594+0000 7f2e0d600640 1 -- 172.21.15.86:0/2296178051 <== mon.2 v2:172.21.15.155:3301/0 9 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v89) ==== 81+0+879 (secure 0 0 0) 0x7f2dfc1289e0 con 0x7f2e100722f0 2024-04-25T14:09:09.595 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2882289101 2024-04-26T14:09:09.480502+0000 2024-04-25T14:09:09.595 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6833/132942210 2024-04-26T14:09:09.480502+0000 2024-04-25T14:09:09.595 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6832/132942210 2024-04-26T14:09:09.480502+0000 2024-04-25T14:09:09.595 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3672669933 2024-04-26T14:09:09.480502+0000 2024-04-25T14:09:09.595 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/402603388 2024-04-26T14:09:09.480502+0000 2024-04-25T14:09:09.595 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3743735908 2024-04-26T14:09:09.480502+0000 2024-04-25T14:09:09.595 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3132647616 2024-04-26T14:09:03.346201+0000 2024-04-25T14:09:09.595 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/4104483212 2024-04-26T14:09:09.480502+0000 2024-04-25T14:09:09.595 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1564187566 2024-04-26T14:09:03.346201+0000 2024-04-25T14:09:09.595 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6833/3589556609 2024-04-26T14:09:03.346201+0000 2024-04-25T14:09:09.595 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1099362620 2024-04-26T14:09:03.346201+0000 2024-04-25T14:09:09.596 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6832/3589556609 2024-04-26T14:09:03.346201+0000 2024-04-25T14:09:09.596 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3481243210 2024-04-26T14:09:03.346201+0000 2024-04-25T14:09:09.596 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2509210906 2024-04-26T14:09:03.346201+0000 2024-04-25T14:09:09.596 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3722630918 2024-04-26T14:09:03.346201+0000 2024-04-25T14:09:09.596 INFO:teuthology.orchestra.run.smithi086.stderr:listed 15 entries 2024-04-25T14:09:09.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.594+0000 7f2e18600640 1 -- 172.21.15.86:0/2296178051 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e100722f0 msgr2=0x7f2e1015b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:09.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.594+0000 7f2e18600640 1 --2- 172.21.15.86:0/2296178051 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e100722f0 0x7f2e1015b930 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc0af000 tx=0x7f2dfc01bab0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.594+0000 7f2e18600640 1 -- 172.21.15.86:0/2296178051 shutdown_connections 2024-04-25T14:09:09.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.594+0000 7f2e18600640 1 --2- 172.21.15.86:0/2296178051 >> [v2:172.21.15.86:6832/132942210,v1:172.21.15.86:6833/132942210] conn(0x7f2df006f4e0 0x7f2df00719f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.594+0000 7f2e18600640 1 --2- 172.21.15.86:0/2296178051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e100d2c80 0x7f2e101f4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.594+0000 7f2e18600640 1 --2- 172.21.15.86:0/2296178051 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e100722f0 0x7f2e1015b930 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.594+0000 7f2e18600640 1 --2- 172.21.15.86:0/2296178051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e10070e20 0x7f2e10155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.594+0000 7f2e18600640 1 -- 172.21.15.86:0/2296178051 >> 172.21.15.86:0/2296178051 conn(0x7f2e100cffc0 msgr2=0x7f2e100d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:09.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.594+0000 7f2e18600640 1 -- 172.21.15.86:0/2296178051 shutdown_connections 2024-04-25T14:09:09.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.594+0000 7f2e18600640 1 -- 172.21.15.86:0/2296178051 wait complete. 2024-04-25T14:09:09.616 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T14:09:09.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:09:09.616 DEBUG:teuthology.orchestra.run.smithi155:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T14:09:09.641 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T14:09:09.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:09:09.642 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T14:09:09.693 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:09:09.694 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:09:09.694 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:09:09.703 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2893 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T14:09:09.703 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:09:09.703 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:09:09.703 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:09:09.750 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4517 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T14:09:09.750 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:09:09.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafaac00640 1 Processor -- start 2024-04-25T14:09:09.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafaac00640 1 -- start start 2024-04-25T14:09:09.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafaac00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fafa415c2a0 0x7fafa40ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafaac00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafa406fd50 0x7fafa4070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafaac00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafa40b0a60 con 0x7fafa406c670 2024-04-25T14:09:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafaac00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fafa406ef90 con 0x7fafa415c2a0 2024-04-25T14:09:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafaac00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fafa406f270 con 0x7fafa406fd50 2024-04-25T14:09:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafa8c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fafa406c670 0x7fafa406ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42756/0 (socket says 172.21.15.86:42756) 2024-04-25T14:09:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafa8c00640 1 -- 172.21.15.86:0/1343782484 learned_addr learned my addr 172.21.15.86:0/1343782484 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafa3e00640 1 --2- 172.21.15.86:0/1343782484 >> v2:172.21.15.155:3300/0 conn(0x7fafa415c2a0 0x7fafa40ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafa3400640 1 --2- 172.21.15.86:0/1343782484 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafa406fd50 0x7fafa4070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafa2a00640 1 -- 172.21.15.86:0/1343782484 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 42549170 0 0) 0x7fafa40b0a60 con 0x7fafa406c670 2024-04-25T14:09:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafa2a00640 1 -- 172.21.15.86:0/1343782484 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf88003530 con 0x7fafa406c670 2024-04-25T14:09:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafa3e00640 1 -- 172.21.15.86:0/1343782484 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafa406fd50 msgr2=0x7fafa4070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafa3e00640 1 --2- 172.21.15.86:0/1343782484 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafa406fd50 0x7fafa4070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafa3e00640 1 -- 172.21.15.86:0/1343782484 >> v1:172.21.15.86:6789/0 conn(0x7fafa406c670 legacy=0x7fafa406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafa3e00640 1 -- 172.21.15.86:0/1343782484 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa406f550 con 0x7fafa415c2a0 2024-04-25T14:09:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafa3400640 1 --2- 172.21.15.86:0/1343782484 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafa406fd50 0x7fafa4070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:09:09.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafa3e00640 1 --2- 172.21.15.86:0/1343782484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafa415c2a0 0x7fafa40ae130 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7faf940097c0 tx=0x7faf9401c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffc50867c2457e8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:09.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafa2a00640 1 -- 172.21.15.86:0/1343782484 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf940b0070 con 0x7fafa415c2a0 2024-04-25T14:09:09.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafa2a00640 1 -- 172.21.15.86:0/1343782484 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf940ab600 con 0x7fafa415c2a0 2024-04-25T14:09:09.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafa2a00640 1 -- 172.21.15.86:0/1343782484 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf940ab9d0 con 0x7fafa415c2a0 2024-04-25T14:09:09.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafaac00640 1 -- 172.21.15.86:0/1343782484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafa415c2a0 msgr2=0x7fafa40ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:09.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafaac00640 1 --2- 172.21.15.86:0/1343782484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafa415c2a0 0x7fafa40ae130 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7faf940097c0 tx=0x7faf9401c3e0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafaac00640 1 -- 172.21.15.86:0/1343782484 shutdown_connections 2024-04-25T14:09:09.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafaac00640 1 --2- 172.21.15.86:0/1343782484 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafa406fd50 0x7fafa4070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafaac00640 1 --2- 172.21.15.86:0/1343782484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafa415c2a0 0x7fafa40ae130 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafaac00640 1 -- 172.21.15.86:0/1343782484 >> 172.21.15.86:0/1343782484 conn(0x7fafa40b4650 msgr2=0x7fafa40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:09.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafaac00640 1 -- 172.21.15.86:0/1343782484 shutdown_connections 2024-04-25T14:09:09.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.902+0000 7fafaac00640 1 -- 172.21.15.86:0/1343782484 wait complete. 2024-04-25T14:09:09.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.906+0000 7fafaac00640 1 Processor -- start 2024-04-25T14:09:09.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.906+0000 7fafaac00640 1 -- start start 2024-04-25T14:09:09.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.906+0000 7fafaac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa406c670 0x7fafa40c0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.906+0000 7fafaac00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafa406fd50 0x7fafa4154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.906+0000 7fafaac00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafa415c2a0 0x7fafa4155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.906+0000 7fafaac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fafa415c830 con 0x7fafa406c670 2024-04-25T14:09:09.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.906+0000 7fafaac00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fafa415c6b0 con 0x7fafa415c2a0 2024-04-25T14:09:09.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.906+0000 7fafaac00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fafa40ae670 con 0x7fafa406fd50 2024-04-25T14:09:09.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.906+0000 7fafa3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa406c670 0x7fafa40c0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.906+0000 7fafa3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa406c670 0x7fafa40c0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57360/0 (socket says 172.21.15.86:57360) 2024-04-25T14:09:09.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.906+0000 7fafa3e00640 1 -- 172.21.15.86:0/3207534903 learned_addr learned my addr 172.21.15.86:0/3207534903 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:09.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.906+0000 7fafa3400640 1 --2- 172.21.15.86:0/3207534903 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafa406fd50 0x7fafa4154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.906+0000 7fafa8c00640 1 --2- 172.21.15.86:0/3207534903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafa415c2a0 0x7fafa4155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.906+0000 7fafa3e00640 1 -- 172.21.15.86:0/3207534903 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafa406fd50 msgr2=0x7fafa4154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:09.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.906+0000 7fafa3e00640 1 --2- 172.21.15.86:0/3207534903 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafa406fd50 0x7fafa4154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.906+0000 7fafa3e00640 1 -- 172.21.15.86:0/3207534903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafa415c2a0 msgr2=0x7fafa4155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:09.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.906+0000 7fafa3e00640 1 --2- 172.21.15.86:0/3207534903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafa415c2a0 0x7fafa4155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.906+0000 7fafa3e00640 1 -- 172.21.15.86:0/3207534903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa415b910 con 0x7fafa406c670 2024-04-25T14:09:09.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.906+0000 7fafa3400640 1 --2- 172.21.15.86:0/3207534903 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafa406fd50 0x7fafa4154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:09.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.906+0000 7fafa3e00640 1 --2- 172.21.15.86:0/3207534903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa406c670 0x7fafa40c0e60 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7faf9401a010 tx=0x7faf9401ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:09.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.906+0000 7faf9be00640 1 -- 172.21.15.86:0/3207534903 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf940b0050 con 0x7fafa406c670 2024-04-25T14:09:09.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.910+0000 7faf9be00640 1 -- 172.21.15.86:0/3207534903 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf940b1040 con 0x7fafa406c670 2024-04-25T14:09:09.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.910+0000 7faf9be00640 1 -- 172.21.15.86:0/3207534903 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf94006e20 con 0x7fafa406c670 2024-04-25T14:09:09.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.910+0000 7fafaac00640 1 -- 172.21.15.86:0/3207534903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa4072cd0 con 0x7fafa406c670 2024-04-25T14:09:09.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.910+0000 7fafaac00640 1 -- 172.21.15.86:0/3207534903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa40c3830 con 0x7fafa406c670 2024-04-25T14:09:09.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.910+0000 7fafaac00640 1 -- 172.21.15.86:0/3207534903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafa40d0320 con 0x7fafa406c670 2024-04-25T14:09:09.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.910+0000 7faf9be00640 1 -- 172.21.15.86:0/3207534903 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 34) ==== 92918+0+0 (secure 0 0 0) 0x7faf940be030 con 0x7fafa406c670 2024-04-25T14:09:09.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.910+0000 7faf9be00640 1 -- 172.21.15.86:0/3207534903 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6746+0+0 (secure 0 0 0) 0x7faf94128cd0 con 0x7fafa406c670 2024-04-25T14:09:09.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:09.914+0000 7faf9be00640 1 -- 172.21.15.86:0/3207534903 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fafa40d0320 con 0x7fafa406c670 2024-04-25T14:09:10.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.106+0000 7fafaac00640 1 -- 172.21.15.86:0/3207534903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafa4076bf0 con 0x7fafa406c670 2024-04-25T14:09:10.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.106+0000 7faf9be00640 1 -- 172.21.15.86:0/3207534903 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5186 (secure 0 0 0) 0x7faf94016090 con 0x7fafa406c670 2024-04-25T14:09:10.109 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:10.109 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7186,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/4265929298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4265929298},{"type":"v1","addr":"172.21.15.155:6833","nonce":4265929298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7189,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3254219000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3254219000},{"type":"v1","addr":"172.21.15.155:6835","nonce":3254219000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7195,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/193454084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":193454084},{"type":"v1","addr":"172.21.15.86:6837","nonce":193454084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:08:45.096500+0000","modified":"2024-04-25T14:08:48.121611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7371},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7371":{"gid":7371,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/1961103704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1961103704},{"type":"v1","addr":"172.21.15.86:6835","nonce":1961103704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7371,"qdb_cluster":[7371]},"id":6}]} 2024-04-25T14:09:10.109 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 67 2024-04-25T14:09:10.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.110+0000 7fafaac00640 1 -- 172.21.15.86:0/3207534903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa406c670 msgr2=0x7fafa40c0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:10.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.110+0000 7fafaac00640 1 --2- 172.21.15.86:0/3207534903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa406c670 0x7fafa40c0e60 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7faf9401a010 tx=0x7faf9401ac20 comp rx=0 tx=0).stop 2024-04-25T14:09:10.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.110+0000 7fafaac00640 1 -- 172.21.15.86:0/3207534903 shutdown_connections 2024-04-25T14:09:10.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.110+0000 7fafaac00640 1 --2- 172.21.15.86:0/3207534903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafa415c2a0 0x7fafa4155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:10.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.110+0000 7fafaac00640 1 --2- 172.21.15.86:0/3207534903 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafa406fd50 0x7fafa4154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:10.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.110+0000 7fafaac00640 1 --2- 172.21.15.86:0/3207534903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa406c670 0x7fafa40c0e60 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:10.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.110+0000 7fafaac00640 1 -- 172.21.15.86:0/3207534903 >> 172.21.15.86:0/3207534903 conn(0x7fafa40b4650 msgr2=0x7fafa40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:10.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.110+0000 7fafaac00640 1 -- 172.21.15.86:0/3207534903 shutdown_connections 2024-04-25T14:09:10.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.110+0000 7fafaac00640 1 -- 172.21.15.86:0/3207534903 wait complete. 2024-04-25T14:09:10.130 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:09:10.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.274+0000 7fdf61600640 1 Processor -- start 2024-04-25T14:09:10.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.274+0000 7fdf61600640 1 -- start start 2024-04-25T14:09:10.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.278+0000 7fdf61600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf5c0b4440 0x7fdf5c163920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:10.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.278+0000 7fdf61600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdf5c0d3e70 0x7fdf5c163e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:10.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.278+0000 7fdf61600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf5c0b4a30 con 0x7fdf5c0d1200 2024-04-25T14:09:10.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.278+0000 7fdf61600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdf5c0b4d10 con 0x7fdf5c0d3e70 2024-04-25T14:09:10.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.278+0000 7fdf61600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdf5c06e010 con 0x7fdf5c0b4440 2024-04-25T14:09:10.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.278+0000 7fdf5b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdf5c0d3e70 0x7fdf5c163e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:10.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.278+0000 7fdf5aa00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fdf5c0d1200 0x7fdf5c0d1dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42760/0 (socket says 172.21.15.86:42760) 2024-04-25T14:09:10.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.278+0000 7fdf5aa00640 1 -- 172.21.15.86:0/4216244712 learned_addr learned my addr 172.21.15.86:0/4216244712 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:10.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.278+0000 7fdf52000640 1 --2- 172.21.15.86:0/4216244712 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf5c0b4440 0x7fdf5c163920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:10.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.278+0000 7fdf5b400640 1 -- 172.21.15.86:0/4216244712 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf5c0b4440 msgr2=0x7fdf5c163920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:10.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.278+0000 7fdf5b400640 1 --2- 172.21.15.86:0/4216244712 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf5c0b4440 0x7fdf5c163920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:10.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.278+0000 7fdf5b400640 1 -- 172.21.15.86:0/4216244712 >> v1:172.21.15.86:6789/0 conn(0x7fdf5c0d1200 legacy=0x7fdf5c0d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:10.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.278+0000 7fdf5b400640 1 -- 172.21.15.86:0/4216244712 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5c06d9c0 con 0x7fdf5c0d3e70 2024-04-25T14:09:10.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.278+0000 7fdf5a000640 1 -- 172.21.15.86:0/4216244712 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1035520076 0 0) 0x7fdf5c0b4a30 con 0x7fdf5c0d1200 2024-04-25T14:09:10.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.282+0000 7fdf5b400640 1 --2- 172.21.15.86:0/4216244712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5c0d3e70 0x7fdf5c163e60 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009d90 tx=0x7fdf5401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636e7525c3e1bb93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:10.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.282+0000 7fdf5a000640 1 -- 172.21.15.86:0/4216244712 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf540b2070 con 0x7fdf5c0d3e70 2024-04-25T14:09:10.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.282+0000 7fdf5a000640 1 -- 172.21.15.86:0/4216244712 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf540a58b0 con 0x7fdf5c0d3e70 2024-04-25T14:09:10.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.282+0000 7fdf5a000640 1 -- 172.21.15.86:0/4216244712 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf540a5c80 con 0x7fdf5c0d3e70 2024-04-25T14:09:10.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.282+0000 7fdf61600640 1 -- 172.21.15.86:0/4216244712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5c0d3e70 msgr2=0x7fdf5c163e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:10.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.282+0000 7fdf61600640 1 --2- 172.21.15.86:0/4216244712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5c0d3e70 0x7fdf5c163e60 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009d90 tx=0x7fdf5401c8e0 comp rx=0 tx=0).stop 2024-04-25T14:09:10.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.282+0000 7fdf61600640 1 -- 172.21.15.86:0/4216244712 shutdown_connections 2024-04-25T14:09:10.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.282+0000 7fdf61600640 1 --2- 172.21.15.86:0/4216244712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5c0d3e70 0x7fdf5c163e60 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:10.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.282+0000 7fdf61600640 1 --2- 172.21.15.86:0/4216244712 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf5c0b4440 0x7fdf5c163920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:10.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.282+0000 7fdf61600640 1 -- 172.21.15.86:0/4216244712 >> 172.21.15.86:0/4216244712 conn(0x7fdf5c0b72d0 msgr2=0x7fdf5c0b76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:10.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.282+0000 7fdf61600640 1 -- 172.21.15.86:0/4216244712 shutdown_connections 2024-04-25T14:09:10.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.282+0000 7fdf61600640 1 -- 172.21.15.86:0/4216244712 wait complete. 2024-04-25T14:09:10.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.286+0000 7fdf61600640 1 Processor -- start 2024-04-25T14:09:10.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.286+0000 7fdf61600640 1 -- start start 2024-04-25T14:09:10.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.286+0000 7fdf61600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf5c0b4440 0x7fdf5c0cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:10.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.286+0000 7fdf61600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf5c0d1200 0x7fdf5c14ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:10.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.286+0000 7fdf61600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5c0d3e70 0x7fdf5c14ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:10.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.286+0000 7fdf61600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c16d650 con 0x7fdf5c0d3e70 2024-04-25T14:09:10.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.286+0000 7fdf61600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c16d4d0 con 0x7fdf5c0b4440 2024-04-25T14:09:10.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.286+0000 7fdf61600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdf5c16d7d0 con 0x7fdf5c0d1200 2024-04-25T14:09:10.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.286+0000 7fdf5b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5c0d3e70 0x7fdf5c14ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:10.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.286+0000 7fdf5b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5c0d3e70 0x7fdf5c14ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57376/0 (socket says 172.21.15.86:57376) 2024-04-25T14:09:10.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.286+0000 7fdf5b400640 1 -- 172.21.15.86:0/4287668457 learned_addr learned my addr 172.21.15.86:0/4287668457 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:10.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.286+0000 7fdf52000640 1 --2- 172.21.15.86:0/4287668457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf5c0d1200 0x7fdf5c14ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:10.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.286+0000 7fdf5aa00640 1 --2- 172.21.15.86:0/4287668457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf5c0b4440 0x7fdf5c0cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:10.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.286+0000 7fdf5b400640 1 -- 172.21.15.86:0/4287668457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf5c0d1200 msgr2=0x7fdf5c14ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:10.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.286+0000 7fdf5b400640 1 --2- 172.21.15.86:0/4287668457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf5c0d1200 0x7fdf5c14ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:10.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.286+0000 7fdf5b400640 1 -- 172.21.15.86:0/4287668457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf5c0b4440 msgr2=0x7fdf5c0cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:10.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.286+0000 7fdf5b400640 1 --2- 172.21.15.86:0/4287668457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf5c0b4440 0x7fdf5c0cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:10.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.286+0000 7fdf5b400640 1 -- 172.21.15.86:0/4287668457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5c150bb0 con 0x7fdf5c0d3e70 2024-04-25T14:09:10.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.286+0000 7fdf52000640 1 --2- 172.21.15.86:0/4287668457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf5c0d1200 0x7fdf5c14ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:10.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.286+0000 7fdf5aa00640 1 --2- 172.21.15.86:0/4287668457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf5c0b4440 0x7fdf5c0cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:10.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.286+0000 7fdf5b400640 1 --2- 172.21.15.86:0/4287668457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5c0d3e70 0x7fdf5c14ef40 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009d90 tx=0x7fdf54019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:10.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.286+0000 7fdf53400640 1 -- 172.21.15.86:0/4287668457 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf540b2050 con 0x7fdf5c0d3e70 2024-04-25T14:09:10.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.286+0000 7fdf53400640 1 -- 172.21.15.86:0/4287668457 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf540b3040 con 0x7fdf5c0d3e70 2024-04-25T14:09:10.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.286+0000 7fdf61600640 1 -- 172.21.15.86:0/4287668457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5c14cdc0 con 0x7fdf5c0d3e70 2024-04-25T14:09:10.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.286+0000 7fdf53400640 1 -- 172.21.15.86:0/4287668457 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf540ad850 con 0x7fdf5c0d3e70 2024-04-25T14:09:10.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.286+0000 7fdf61600640 1 -- 172.21.15.86:0/4287668457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf5c14d0a0 con 0x7fdf5c0d3e70 2024-04-25T14:09:10.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.290+0000 7fdf61600640 1 -- 172.21.15.86:0/4287668457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf5c160350 con 0x7fdf5c0d3e70 2024-04-25T14:09:10.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.290+0000 7fdf53400640 1 -- 172.21.15.86:0/4287668457 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 34) ==== 92918+0+0 (secure 0 0 0) 0x7fdf540c0020 con 0x7fdf5c0d3e70 2024-04-25T14:09:10.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.290+0000 7fdf53400640 1 -- 172.21.15.86:0/4287668457 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6746+0+0 (secure 0 0 0) 0x7fdf54129b00 con 0x7fdf5c0d3e70 2024-04-25T14:09:10.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.294+0000 7fdf53400640 1 -- 172.21.15.86:0/4287668457 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdf54016560 con 0x7fdf5c0d3e70 2024-04-25T14:09:10.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.454+0000 7fdf61600640 1 -- 172.21.15.86:0/4287668457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf5c06ef80 con 0x7fdf5c0d3e70 2024-04-25T14:09:10.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.454+0000 7fdf53400640 1 -- 172.21.15.86:0/4287668457 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+17823 (secure 0 0 0) 0x7fdf540f6940 con 0x7fdf5c0d3e70 2024-04-25T14:09:10.457 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:10.457 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:09:09.480535+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:08:43.505988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T14:08:44.054628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2882289101":"2024-04-26T14:09:09.480502+0000","172.21.15.86:6833/132942210":"2024-04-26T14:09:09.480502+0000","172.21.15.86:6832/132942210":"2024-04-26T14:09:09.480502+0000","172.21.15.86:0/3672669933":"2024-04-26T14:09:09.480502+0000","172.21.15.86:0/402603388":"2024-04-26T14:09:09.480502+0000","172.21.15.86:0/3743735908":"2024-04-26T14:09:09.480502+0000","172.21.15.86:0/3132647616":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/4104483212":"2024-04-26T14:09:09.480502+0000","172.21.15.86:0/1564187566":"2024-04-26T14:09:03.346201+0000","172.21.15.86:6833/3589556609":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/1099362620":"2024-04-26T14:09:03.346201+0000","172.21.15.86:6832/3589556609":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/3481243210":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/2509210906":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/3722630918":"2024-04-26T14:09:03.346201+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:09:10.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.458+0000 7fdf61600640 1 -- 172.21.15.86:0/4287668457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5c0d3e70 msgr2=0x7fdf5c14ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:10.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.458+0000 7fdf61600640 1 --2- 172.21.15.86:0/4287668457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5c0d3e70 0x7fdf5c14ef40 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009d90 tx=0x7fdf54019680 comp rx=0 tx=0).stop 2024-04-25T14:09:10.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.458+0000 7fdf61600640 1 -- 172.21.15.86:0/4287668457 shutdown_connections 2024-04-25T14:09:10.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.458+0000 7fdf61600640 1 --2- 172.21.15.86:0/4287668457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf5c0d3e70 0x7fdf5c14ef40 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:10.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.458+0000 7fdf61600640 1 --2- 172.21.15.86:0/4287668457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf5c0d1200 0x7fdf5c14ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:10.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.458+0000 7fdf61600640 1 --2- 172.21.15.86:0/4287668457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf5c0b4440 0x7fdf5c0cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:10.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.458+0000 7fdf61600640 1 -- 172.21.15.86:0/4287668457 >> 172.21.15.86:0/4287668457 conn(0x7fdf5c0b72d0 msgr2=0x7fdf5c1586d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:10.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.458+0000 7fdf61600640 1 -- 172.21.15.86:0/4287668457 shutdown_connections 2024-04-25T14:09:10.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.458+0000 7fdf61600640 1 -- 172.21.15.86:0/4287668457 wait complete. 2024-04-25T14:09:10.479 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:08:43.505988+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:08:44.054628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:09:10.479 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:09:10.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.638+0000 7f278c800640 1 Processor -- start 2024-04-25T14:09:10.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.638+0000 7f278c800640 1 -- start start 2024-04-25T14:09:10.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.642+0000 7f278c800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f27840d1df0 0x7f27840cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:10.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.642+0000 7f278c800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27840cdcc0 0x7f27840ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:10.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.642+0000 7f278c800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f278415e5c0 con 0x7f2784070330 2024-04-25T14:09:10.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.642+0000 7f278c800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f278406fd40 con 0x7f27840d1df0 2024-04-25T14:09:10.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.642+0000 7f278c800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f278415e8a0 con 0x7f27840cdcc0 2024-04-25T14:09:10.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.642+0000 7f278a800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27840cdcc0 0x7f27840ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:10.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.642+0000 7f2789400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f27840d1df0 0x7f27840cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:10.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.642+0000 7f2789e00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f2784070330 0x7f27840d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:54182/0 (socket says 172.21.15.86:54182) 2024-04-25T14:09:10.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.642+0000 7f2789e00640 1 -- 172.21.15.86:0/3796071573 learned_addr learned my addr 172.21.15.86:0/3796071573 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:10.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.642+0000 7f278a800640 1 -- 172.21.15.86:0/3796071573 >> v2:172.21.15.155:3300/0 conn(0x7f27840d1df0 msgr2=0x7f27840cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:10.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.642+0000 7f278a800640 1 --2- 172.21.15.86:0/3796071573 >> v2:172.21.15.155:3300/0 conn(0x7f27840d1df0 0x7f27840cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:10.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.642+0000 7f278a800640 1 -- 172.21.15.86:0/3796071573 >> v1:172.21.15.155:6789/0 conn(0x7f2784070330 legacy=0x7f27840d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:10.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.642+0000 7f278a800640 1 -- 172.21.15.86:0/3796071573 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278415eb80 con 0x7f27840cdcc0 2024-04-25T14:09:10.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.642+0000 7f2788a00640 1 -- 172.21.15.86:0/3796071573 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 606238246 0 0) 0x7f278415e5c0 con 0x7f2784070330 2024-04-25T14:09:10.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.642+0000 7f278a800640 1 --2- 172.21.15.86:0/3796071573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27840cdcc0 0x7f27840ce0d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2780009d90 tx=0x7f278001c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b58a43823a8612a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:10.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.642+0000 7f2788a00640 1 -- 172.21.15.86:0/3796071573 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27800b2070 con 0x7f27840cdcc0 2024-04-25T14:09:10.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.642+0000 7f2788a00640 1 -- 172.21.15.86:0/3796071573 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27800a5950 con 0x7f27840cdcc0 2024-04-25T14:09:10.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.642+0000 7f2788a00640 1 -- 172.21.15.86:0/3796071573 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27800a5d20 con 0x7f27840cdcc0 2024-04-25T14:09:10.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.642+0000 7f278c800640 1 -- 172.21.15.86:0/3796071573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27840cdcc0 msgr2=0x7f27840ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:10.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.642+0000 7f278c800640 1 --2- 172.21.15.86:0/3796071573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27840cdcc0 0x7f27840ce0d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2780009d90 tx=0x7f278001c8b0 comp rx=0 tx=0).stop 2024-04-25T14:09:10.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.642+0000 7f278c800640 1 -- 172.21.15.86:0/3796071573 shutdown_connections 2024-04-25T14:09:10.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.642+0000 7f278c800640 1 --2- 172.21.15.86:0/3796071573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27840cdcc0 0x7f27840ce0d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:10.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.642+0000 7f278c800640 1 --2- 172.21.15.86:0/3796071573 >> v2:172.21.15.155:3300/0 conn(0x7f27840d1df0 0x7f27840cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:10.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.642+0000 7f278c800640 1 -- 172.21.15.86:0/3796071573 >> 172.21.15.86:0/3796071573 conn(0x7f27840b4650 msgr2=0x7f27840b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:10.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.642+0000 7f278c800640 1 -- 172.21.15.86:0/3796071573 shutdown_connections 2024-04-25T14:09:10.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.642+0000 7f278c800640 1 -- 172.21.15.86:0/3796071573 wait complete. 2024-04-25T14:09:10.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.642+0000 7f278c800640 1 Processor -- start 2024-04-25T14:09:10.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.646+0000 7f278c800640 1 -- start start 2024-04-25T14:09:10.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.646+0000 7f278c800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2784070330 0x7f278415ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:10.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.646+0000 7f278c800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27840cdcc0 0x7f27840b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:10.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.646+0000 7f278c800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27840cf8c0 0x7f27840c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:10.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.646+0000 7f278c800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f278415f4b0 con 0x7f2784070330 2024-04-25T14:09:10.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.646+0000 7f278c800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f278415f330 con 0x7f27840cdcc0 2024-04-25T14:09:10.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.646+0000 7f278c800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2784070740 con 0x7f27840cf8c0 2024-04-25T14:09:10.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.646+0000 7f2789e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2784070330 0x7f278415ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:10.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.646+0000 7f2789e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2784070330 0x7f278415ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57378/0 (socket says 172.21.15.86:57378) 2024-04-25T14:09:10.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.646+0000 7f2789e00640 1 -- 172.21.15.86:0/3656808915 learned_addr learned my addr 172.21.15.86:0/3656808915 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:10.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.646+0000 7f2789400640 1 --2- 172.21.15.86:0/3656808915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27840cdcc0 0x7f27840b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:10.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.646+0000 7f278a800640 1 --2- 172.21.15.86:0/3656808915 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27840cf8c0 0x7f27840c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:10.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.646+0000 7f2789e00640 1 -- 172.21.15.86:0/3656808915 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27840cf8c0 msgr2=0x7f27840c6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:10.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.646+0000 7f2789e00640 1 --2- 172.21.15.86:0/3656808915 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27840cf8c0 0x7f27840c6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:10.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.646+0000 7f2789e00640 1 -- 172.21.15.86:0/3656808915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27840cdcc0 msgr2=0x7f27840b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:10.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.646+0000 7f2789e00640 1 --2- 172.21.15.86:0/3656808915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27840cdcc0 0x7f27840b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:10.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.646+0000 7f2789e00640 1 -- 172.21.15.86:0/3656808915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27840cfe60 con 0x7f2784070330 2024-04-25T14:09:10.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.646+0000 7f2789e00640 1 --2- 172.21.15.86:0/3656808915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2784070330 0x7f278415ab90 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f277800c480 tx=0x7f277800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:10.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.646+0000 7f277e000640 1 -- 172.21.15.86:0/3656808915 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778018070 con 0x7f2784070330 2024-04-25T14:09:10.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.646+0000 7f277e000640 1 -- 172.21.15.86:0/3656808915 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27780074e0 con 0x7f2784070330 2024-04-25T14:09:10.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.646+0000 7f278c800640 1 -- 172.21.15.86:0/3656808915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27840d0140 con 0x7f2784070330 2024-04-25T14:09:10.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.646+0000 7f277e000640 1 -- 172.21.15.86:0/3656808915 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27780049f0 con 0x7f2784070330 2024-04-25T14:09:10.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.646+0000 7f278c800640 1 -- 172.21.15.86:0/3656808915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27840d0700 con 0x7f2784070330 2024-04-25T14:09:10.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.646+0000 7f278c800640 1 -- 172.21.15.86:0/3656808915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2784069d00 con 0x7f2784070330 2024-04-25T14:09:10.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.650+0000 7f277e000640 1 -- 172.21.15.86:0/3656808915 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 35) ==== 93045+0+0 (secure 0 0 0) 0x7f277801b070 con 0x7f2784070330 2024-04-25T14:09:10.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.650+0000 7f277e000640 1 --2- 172.21.15.86:0/3656808915 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f276806f250 0x7f2768071760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:10.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.650+0000 7f277e000640 1 -- 172.21.15.86:0/3656808915 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6746+0+0 (secure 0 0 0) 0x7f277808e6a0 con 0x7f2784070330 2024-04-25T14:09:10.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.650+0000 7f2789400640 1 --2- 172.21.15.86:0/3656808915 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f276806f250 0x7f2768071760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:10.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.650+0000 7f2789400640 1 --2- 172.21.15.86:0/3656808915 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f276806f250 0x7f2768071760 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f27840afe60 tx=0x7f2774009190 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:10.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.654+0000 7f277e000640 1 -- 172.21.15.86:0/3656808915 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f277805ae30 con 0x7f2784070330 2024-04-25T14:09:10.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.814+0000 7f278c800640 1 -- 172.21.15.86:0/3656808915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2784076bf0 con 0x7f2784070330 2024-04-25T14:09:10.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.814+0000 7f277e000640 1 -- 172.21.15.86:0/3656808915 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f277805f130 con 0x7f2784070330 2024-04-25T14:09:10.815 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:09:10.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.814+0000 7f278c800640 1 -- 172.21.15.86:0/3656808915 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f276806f250 msgr2=0x7f2768071760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:10.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.814+0000 7f278c800640 1 --2- 172.21.15.86:0/3656808915 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f276806f250 0x7f2768071760 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f27840afe60 tx=0x7f2774009190 comp rx=0 tx=0).stop 2024-04-25T14:09:10.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.814+0000 7f278c800640 1 -- 172.21.15.86:0/3656808915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2784070330 msgr2=0x7f278415ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:10.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.814+0000 7f278c800640 1 --2- 172.21.15.86:0/3656808915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2784070330 0x7f278415ab90 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f277800c480 tx=0x7f277800c940 comp rx=0 tx=0).stop 2024-04-25T14:09:10.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.814+0000 7f278c800640 1 -- 172.21.15.86:0/3656808915 shutdown_connections 2024-04-25T14:09:10.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.814+0000 7f278c800640 1 --2- 172.21.15.86:0/3656808915 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f276806f250 0x7f2768071760 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:10.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.814+0000 7f278c800640 1 --2- 172.21.15.86:0/3656808915 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27840cf8c0 0x7f27840c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:10.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.814+0000 7f278c800640 1 --2- 172.21.15.86:0/3656808915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27840cdcc0 0x7f27840b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:10.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.814+0000 7f278c800640 1 --2- 172.21.15.86:0/3656808915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2784070330 0x7f278415ab90 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:10.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.814+0000 7f278c800640 1 -- 172.21.15.86:0/3656808915 >> 172.21.15.86:0/3656808915 conn(0x7f27840b4650 msgr2=0x7f27840aef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:10.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.814+0000 7f278c800640 1 -- 172.21.15.86:0/3656808915 shutdown_connections 2024-04-25T14:09:10.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.814+0000 7f278c800640 1 -- 172.21.15.86:0/3656808915 wait complete. 2024-04-25T14:09:10.836 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:09:10.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.986+0000 7f6baea00640 1 Processor -- start 2024-04-25T14:09:10.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.986+0000 7f6baea00640 1 -- start start 2024-04-25T14:09:10.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.986+0000 7f6baea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6ba8157f80 0x7f6ba80aea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:10.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.986+0000 7f6baea00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ba80b0710 con 0x7f6ba806f6d0 2024-04-25T14:09:10.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.986+0000 7f6baea00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ba80ae1c0 con 0x7f6ba8161b20 2024-04-25T14:09:10.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.986+0000 7f6baea00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6ba80aef70 con 0x7f6ba8157f80 2024-04-25T14:09:10.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.986+0000 7f6bad600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6ba8157f80 0x7f6ba80aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:10.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.986+0000 7f6bad600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6ba8157f80 0x7f6ba80aea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57384/0 (socket says 172.21.15.86:57384) 2024-04-25T14:09:10.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.986+0000 7f6bacc00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6ba806f6d0 0x7f6ba8161410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42776/0 (socket says 172.21.15.86:42776) 2024-04-25T14:09:10.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.986+0000 7f6bad600640 1 -- 172.21.15.86:0/1224800799 learned_addr learned my addr 172.21.15.86:0/1224800799 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:10.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.986+0000 7f6bad600640 1 -- 172.21.15.86:0/1224800799 >> v1:172.21.15.155:6789/0 conn(0x7f6ba8161b20 legacy=0x7f6ba806c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:10.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.986+0000 7f6bad600640 1 -- 172.21.15.86:0/1224800799 >> v1:172.21.15.86:6789/0 conn(0x7f6ba806f6d0 legacy=0x7f6ba8161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:10.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.986+0000 7f6ba7e00640 1 -- 172.21.15.86:0/1224800799 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2828392982 0 0) 0x7f6ba80b0710 con 0x7f6ba806f6d0 2024-04-25T14:09:10.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.986+0000 7f6bad600640 1 -- 172.21.15.86:0/1224800799 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba80ae4a0 con 0x7f6ba8157f80 2024-04-25T14:09:10.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.990+0000 7f6bad600640 1 --2- 172.21.15.86:0/1224800799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ba8157f80 0x7f6ba80aea30 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009e50 tx=0x7f6b9c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e0b04d552aa0d47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:10.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.990+0000 7f6ba7e00640 1 -- 172.21.15.86:0/1224800799 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c0a5660 con 0x7f6ba8157f80 2024-04-25T14:09:10.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.990+0000 7f6ba7e00640 1 -- 172.21.15.86:0/1224800799 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b9c0a5800 con 0x7f6ba8157f80 2024-04-25T14:09:10.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.990+0000 7f6ba7e00640 1 -- 172.21.15.86:0/1224800799 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c0a5bd0 con 0x7f6ba8157f80 2024-04-25T14:09:10.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.990+0000 7f6baea00640 1 -- 172.21.15.86:0/1224800799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ba8157f80 msgr2=0x7f6ba80aea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:10.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.990+0000 7f6baea00640 1 --2- 172.21.15.86:0/1224800799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ba8157f80 0x7f6ba80aea30 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009e50 tx=0x7f6b9c01c9a0 comp rx=0 tx=0).stop 2024-04-25T14:09:10.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.990+0000 7f6baea00640 1 -- 172.21.15.86:0/1224800799 shutdown_connections 2024-04-25T14:09:10.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.990+0000 7f6baea00640 1 --2- 172.21.15.86:0/1224800799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ba8157f80 0x7f6ba80aea30 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:10.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.990+0000 7f6baea00640 1 -- 172.21.15.86:0/1224800799 >> 172.21.15.86:0/1224800799 conn(0x7f6ba80b4740 msgr2=0x7f6ba80b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:10.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.990+0000 7f6baea00640 1 -- 172.21.15.86:0/1224800799 shutdown_connections 2024-04-25T14:09:10.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.990+0000 7f6baea00640 1 -- 172.21.15.86:0/1224800799 wait complete. 2024-04-25T14:09:10.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.994+0000 7f6baea00640 1 Processor -- start 2024-04-25T14:09:10.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.994+0000 7f6baea00640 1 -- start start 2024-04-25T14:09:10.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.994+0000 7f6baea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ba806f6d0 0x7f6ba814e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:10.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.994+0000 7f6baea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ba8157f80 0x7f6ba814ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:10.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.994+0000 7f6baea00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ba814fab0 0x7f6ba814bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:10.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.994+0000 7f6baea00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba816c110 con 0x7f6ba806f6d0 2024-04-25T14:09:10.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.994+0000 7f6baea00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba816bf90 con 0x7f6ba8157f80 2024-04-25T14:09:10.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.994+0000 7f6baea00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba816c290 con 0x7f6ba814fab0 2024-04-25T14:09:10.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.994+0000 7f6bad600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ba806f6d0 0x7f6ba814e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:10.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.994+0000 7f6bad600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ba806f6d0 0x7f6ba814e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57388/0 (socket says 172.21.15.86:57388) 2024-04-25T14:09:10.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.994+0000 7f6bad600640 1 -- 172.21.15.86:0/1711707485 learned_addr learned my addr 172.21.15.86:0/1711707485 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:10.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.994+0000 7f6bad600640 1 -- 172.21.15.86:0/1711707485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ba814fab0 msgr2=0x7f6ba814bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:10.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.994+0000 7f6bad600640 1 --2- 172.21.15.86:0/1711707485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ba814fab0 0x7f6ba814bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:10.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.994+0000 7f6bacc00640 1 --2- 172.21.15.86:0/1711707485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ba8157f80 0x7f6ba814ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:10.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.994+0000 7f6bad600640 1 -- 172.21.15.86:0/1711707485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ba8157f80 msgr2=0x7f6ba814ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:10.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.994+0000 7f6bad600640 1 --2- 172.21.15.86:0/1711707485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ba8157f80 0x7f6ba814ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:10.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.994+0000 7f6bad600640 1 -- 172.21.15.86:0/1711707485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba806edb0 con 0x7f6ba806f6d0 2024-04-25T14:09:10.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.994+0000 7f6bacc00640 1 --2- 172.21.15.86:0/1711707485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ba8157f80 0x7f6ba814ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:10.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.994+0000 7f6bad600640 1 --2- 172.21.15.86:0/1711707485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ba806f6d0 0x7f6ba814e470 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0195b0 tx=0x7f6b9c019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:10.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.994+0000 7f6ba5600640 1 -- 172.21.15.86:0/1711707485 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c009510 con 0x7f6ba806f6d0 2024-04-25T14:09:10.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.994+0000 7f6ba5600640 1 -- 172.21.15.86:0/1711707485 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b9c0be070 con 0x7f6ba806f6d0 2024-04-25T14:09:10.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.994+0000 7f6baea00640 1 -- 172.21.15.86:0/1711707485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba8150290 con 0x7f6ba806f6d0 2024-04-25T14:09:10.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.994+0000 7f6ba5600640 1 -- 172.21.15.86:0/1711707485 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c0b8ae0 con 0x7f6ba806f6d0 2024-04-25T14:09:10.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.994+0000 7f6baea00640 1 -- 172.21.15.86:0/1711707485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba814dc70 con 0x7f6ba806f6d0 2024-04-25T14:09:10.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.998+0000 7f6baea00640 1 -- 172.21.15.86:0/1711707485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba8076b80 con 0x7f6ba806f6d0 2024-04-25T14:09:10.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.998+0000 7f6ba5600640 1 -- 172.21.15.86:0/1711707485 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 35) ==== 93045+0+0 (secure 0 0 0) 0x7f6b9c0b8c80 con 0x7f6ba806f6d0 2024-04-25T14:09:11.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.998+0000 7f6ba5600640 1 --2- 172.21.15.86:0/1711707485 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f6b8806f280 0x7f6b88071790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:11.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.998+0000 7f6ba5600640 1 -- 172.21.15.86:0/1711707485 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6746+0+0 (secure 0 0 0) 0x7f6b9c12a130 con 0x7f6ba806f6d0 2024-04-25T14:09:11.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.998+0000 7f6bacc00640 1 --2- 172.21.15.86:0/1711707485 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f6b8806f280 0x7f6b88071790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:11.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:10.998+0000 7f6bacc00640 1 --2- 172.21.15.86:0/1711707485 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f6b8806f280 0x7f6b88071790 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80b2b80 tx=0x7f6b94009040 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:11.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.002+0000 7f6ba5600640 1 -- 172.21.15.86:0/1711707485 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6b9c0c7020 con 0x7f6ba806f6d0 2024-04-25T14:09:11.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.162+0000 7f6baea00640 1 -- 172.21.15.86:0/1711707485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6ba8070ba0 con 0x7f6ba806f6d0 2024-04-25T14:09:11.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.162+0000 7f6ba5600640 1 -- 172.21.15.86:0/1711707485 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f6b9c016620 con 0x7f6ba806f6d0 2024-04-25T14:09:11.163 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:09:11.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.162+0000 7f6baea00640 1 -- 172.21.15.86:0/1711707485 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f6b8806f280 msgr2=0x7f6b88071790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:11.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.162+0000 7f6baea00640 1 --2- 172.21.15.86:0/1711707485 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f6b8806f280 0x7f6b88071790 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80b2b80 tx=0x7f6b94009040 comp rx=0 tx=0).stop 2024-04-25T14:09:11.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.162+0000 7f6baea00640 1 -- 172.21.15.86:0/1711707485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ba806f6d0 msgr2=0x7f6ba814e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:11.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.162+0000 7f6baea00640 1 --2- 172.21.15.86:0/1711707485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ba806f6d0 0x7f6ba814e470 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0195b0 tx=0x7f6b9c019740 comp rx=0 tx=0).stop 2024-04-25T14:09:11.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.162+0000 7f6baea00640 1 -- 172.21.15.86:0/1711707485 shutdown_connections 2024-04-25T14:09:11.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.162+0000 7f6baea00640 1 --2- 172.21.15.86:0/1711707485 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f6b8806f280 0x7f6b88071790 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:11.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.162+0000 7f6baea00640 1 --2- 172.21.15.86:0/1711707485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ba814fab0 0x7f6ba814bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:11.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.162+0000 7f6baea00640 1 --2- 172.21.15.86:0/1711707485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ba8157f80 0x7f6ba814ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:11.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.162+0000 7f6baea00640 1 --2- 172.21.15.86:0/1711707485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ba806f6d0 0x7f6ba814e470 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:11.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.162+0000 7f6baea00640 1 -- 172.21.15.86:0/1711707485 >> 172.21.15.86:0/1711707485 conn(0x7f6ba80b4740 msgr2=0x7f6ba80b11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:11.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.162+0000 7f6baea00640 1 -- 172.21.15.86:0/1711707485 shutdown_connections 2024-04-25T14:09:11.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.162+0000 7f6baea00640 1 -- 172.21.15.86:0/1711707485 wait complete. 2024-04-25T14:09:11.185 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:09:11.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.338+0000 7f4c98a00640 1 Processor -- start 2024-04-25T14:09:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.338+0000 7f4c98a00640 1 -- start start 2024-04-25T14:09:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.338+0000 7f4c98a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4c940d1df0 0x7f4c940cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.338+0000 7f4c98a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c9415e5c0 con 0x7f4c940cdcc0 2024-04-25T14:09:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.338+0000 7f4c98a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c9415e8a0 con 0x7f4c94070330 2024-04-25T14:09:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.338+0000 7f4c98a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f4c9406fd40 con 0x7f4c940d1df0 2024-04-25T14:09:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.338+0000 7f4c92800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f4c940cdcc0 0x7f4c940ce0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42782/0 (socket says 172.21.15.86:42782) 2024-04-25T14:09:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.338+0000 7f4c92800640 1 -- 172.21.15.86:0/2830999235 learned_addr learned my addr 172.21.15.86:0/2830999235 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.338+0000 7f4c89400640 1 --2- 172.21.15.86:0/2830999235 >> v2:172.21.15.155:3300/0 conn(0x7f4c940d1df0 0x7f4c940cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.338+0000 7f4c91400640 1 -- 172.21.15.86:0/2830999235 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1941991063 0 0) 0x7f4c9415e5c0 con 0x7f4c940cdcc0 2024-04-25T14:09:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.338+0000 7f4c91400640 1 -- 172.21.15.86:0/2830999235 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c78003530 con 0x7f4c940cdcc0 2024-04-25T14:09:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.338+0000 7f4c89400640 1 -- 172.21.15.86:0/2830999235 >> v1:172.21.15.155:6789/0 conn(0x7f4c94070330 legacy=0x7f4c940d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.338+0000 7f4c89400640 1 -- 172.21.15.86:0/2830999235 >> v1:172.21.15.86:6789/0 conn(0x7f4c940cdcc0 legacy=0x7f4c940ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.338+0000 7f4c89400640 1 -- 172.21.15.86:0/2830999235 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c9415eb80 con 0x7f4c940d1df0 2024-04-25T14:09:11.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.342+0000 7f4c89400640 1 --2- 172.21.15.86:0/2830999235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c940d1df0 0x7f4c940cd780 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c0097f0 tx=0x7f4c7c01c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a30428763ff403e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:11.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.342+0000 7f4c91400640 1 -- 172.21.15.86:0/2830999235 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c0b0070 con 0x7f4c940d1df0 2024-04-25T14:09:11.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.342+0000 7f4c91400640 1 -- 172.21.15.86:0/2830999235 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c7c0ab600 con 0x7f4c940d1df0 2024-04-25T14:09:11.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.342+0000 7f4c91400640 1 -- 172.21.15.86:0/2830999235 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c0ab9d0 con 0x7f4c940d1df0 2024-04-25T14:09:11.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.342+0000 7f4c98a00640 1 -- 172.21.15.86:0/2830999235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c940d1df0 msgr2=0x7f4c940cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:11.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.342+0000 7f4c98a00640 1 --2- 172.21.15.86:0/2830999235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c940d1df0 0x7f4c940cd780 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c0097f0 tx=0x7f4c7c01c3e0 comp rx=0 tx=0).stop 2024-04-25T14:09:11.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.342+0000 7f4c98a00640 1 -- 172.21.15.86:0/2830999235 shutdown_connections 2024-04-25T14:09:11.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.342+0000 7f4c98a00640 1 --2- 172.21.15.86:0/2830999235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c940d1df0 0x7f4c940cd780 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:11.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.342+0000 7f4c98a00640 1 -- 172.21.15.86:0/2830999235 >> 172.21.15.86:0/2830999235 conn(0x7f4c940b4650 msgr2=0x7f4c940b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:11.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.342+0000 7f4c98a00640 1 -- 172.21.15.86:0/2830999235 shutdown_connections 2024-04-25T14:09:11.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.342+0000 7f4c98a00640 1 -- 172.21.15.86:0/2830999235 wait complete. 2024-04-25T14:09:11.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.346+0000 7f4c98a00640 1 Processor -- start 2024-04-25T14:09:11.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.346+0000 7f4c98a00640 1 -- start start 2024-04-25T14:09:11.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.346+0000 7f4c98a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c94070330 0x7f4c9409a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:11.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.346+0000 7f4c98a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c940cdcc0 0x7f4c94156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.346+0000 7f4c98a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c94154730 0x7f4c940b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.346+0000 7f4c98a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c940708c0 con 0x7f4c94154730 2024-04-25T14:09:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.346+0000 7f4c98a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4c94070740 con 0x7f4c94070330 2024-04-25T14:09:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.346+0000 7f4c98a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4c9416bfd0 con 0x7f4c940cdcc0 2024-04-25T14:09:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.346+0000 7f4c92800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c94154730 0x7f4c940b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.346+0000 7f4c92800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c94154730 0x7f4c940b1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57396/0 (socket says 172.21.15.86:57396) 2024-04-25T14:09:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.346+0000 7f4c92800640 1 -- 172.21.15.86:0/3880138597 learned_addr learned my addr 172.21.15.86:0/3880138597 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.346+0000 7f4c89400640 1 --2- 172.21.15.86:0/3880138597 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c940cdcc0 0x7f4c94156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.346+0000 7f4c91e00640 1 --2- 172.21.15.86:0/3880138597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c94070330 0x7f4c9409a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.346+0000 7f4c92800640 1 -- 172.21.15.86:0/3880138597 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c940cdcc0 msgr2=0x7f4c94156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.346+0000 7f4c92800640 1 --2- 172.21.15.86:0/3880138597 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c940cdcc0 0x7f4c94156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.346+0000 7f4c92800640 1 -- 172.21.15.86:0/3880138597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c94070330 msgr2=0x7f4c9409a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.346+0000 7f4c92800640 1 --2- 172.21.15.86:0/3880138597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c94070330 0x7f4c9409a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.346+0000 7f4c92800640 1 -- 172.21.15.86:0/3880138597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c9409ae90 con 0x7f4c94154730 2024-04-25T14:09:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.346+0000 7f4c92800640 1 --2- 172.21.15.86:0/3880138597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c94154730 0x7f4c940b1130 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00caa0 tx=0x7f4c8c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.346+0000 7f4c8aa00640 1 -- 172.21.15.86:0/3880138597 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c017070 con 0x7f4c94154730 2024-04-25T14:09:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.346+0000 7f4c8aa00640 1 -- 172.21.15.86:0/3880138597 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c8c00ac00 con 0x7f4c94154730 2024-04-25T14:09:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.346+0000 7f4c98a00640 1 -- 172.21.15.86:0/3880138597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c94156db0 con 0x7f4c94154730 2024-04-25T14:09:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.350+0000 7f4c8aa00640 1 -- 172.21.15.86:0/3880138597 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c012400 con 0x7f4c94154730 2024-04-25T14:09:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.350+0000 7f4c98a00640 1 -- 172.21.15.86:0/3880138597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c9415b5a0 con 0x7f4c94154730 2024-04-25T14:09:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.350+0000 7f4c98a00640 1 -- 172.21.15.86:0/3880138597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c9415b7e0 con 0x7f4c94154730 2024-04-25T14:09:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.350+0000 7f4c8aa00640 1 -- 172.21.15.86:0/3880138597 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 35) ==== 93045+0+0 (secure 0 0 0) 0x7f4c8c01a070 con 0x7f4c94154730 2024-04-25T14:09:11.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.350+0000 7f4c8aa00640 1 --2- 172.21.15.86:0/3880138597 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f4c7006f280 0x7f4c70071790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:11.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.350+0000 7f4c8aa00640 1 -- 172.21.15.86:0/3880138597 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6746+0+0 (secure 0 0 0) 0x7f4c8c08eff0 con 0x7f4c94154730 2024-04-25T14:09:11.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.350+0000 7f4c91e00640 1 --2- 172.21.15.86:0/3880138597 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f4c7006f280 0x7f4c70071790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:11.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.354+0000 7f4c91e00640 1 --2- 172.21.15.86:0/3880138597 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f4c7006f280 0x7f4c70071790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4c94072200 tx=0x7f4c84009040 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:11.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.354+0000 7f4c8aa00640 1 -- 172.21.15.86:0/3880138597 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4c9415b7e0 con 0x7f4c94154730 2024-04-25T14:09:11.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.518+0000 7f4c98a00640 1 -- 172.21.15.86:0/3880138597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4c94076bf0 con 0x7f4c94154730 2024-04-25T14:09:11.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.518+0000 7f4c8aa00640 1 -- 172.21.15.86:0/3880138597 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f4c8c05b710 con 0x7f4c94154730 2024-04-25T14:09:11.515 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:09:11.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.518+0000 7f4c98a00640 1 -- 172.21.15.86:0/3880138597 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f4c7006f280 msgr2=0x7f4c70071790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.518+0000 7f4c98a00640 1 --2- 172.21.15.86:0/3880138597 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f4c7006f280 0x7f4c70071790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4c94072200 tx=0x7f4c84009040 comp rx=0 tx=0).stop 2024-04-25T14:09:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.518+0000 7f4c98a00640 1 -- 172.21.15.86:0/3880138597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c94154730 msgr2=0x7f4c940b1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.518+0000 7f4c98a00640 1 --2- 172.21.15.86:0/3880138597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c94154730 0x7f4c940b1130 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00caa0 tx=0x7f4c8c00cf60 comp rx=0 tx=0).stop 2024-04-25T14:09:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.518+0000 7f4c98a00640 1 -- 172.21.15.86:0/3880138597 shutdown_connections 2024-04-25T14:09:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.518+0000 7f4c98a00640 1 --2- 172.21.15.86:0/3880138597 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f4c7006f280 0x7f4c70071790 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.518+0000 7f4c98a00640 1 --2- 172.21.15.86:0/3880138597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c94154730 0x7f4c940b1130 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.518+0000 7f4c98a00640 1 --2- 172.21.15.86:0/3880138597 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c940cdcc0 0x7f4c94156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.518+0000 7f4c98a00640 1 --2- 172.21.15.86:0/3880138597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c94070330 0x7f4c9409a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.518+0000 7f4c98a00640 1 -- 172.21.15.86:0/3880138597 >> 172.21.15.86:0/3880138597 conn(0x7f4c940b4650 msgr2=0x7f4c940b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.518+0000 7f4c98a00640 1 -- 172.21.15.86:0/3880138597 shutdown_connections 2024-04-25T14:09:11.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.518+0000 7f4c98a00640 1 -- 172.21.15.86:0/3880138597 wait complete. 2024-04-25T14:09:11.540 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:09:11.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.690+0000 7faf54600640 1 Processor -- start 2024-04-25T14:09:11.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.690+0000 7faf54600640 1 -- start start 2024-04-25T14:09:11.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.690+0000 7faf54600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7faf4c15c2a0 0x7faf4c0ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:11.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.690+0000 7faf54600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf4c0b0a60 con 0x7faf4c06fd50 2024-04-25T14:09:11.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.690+0000 7faf54600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf4c06ef90 con 0x7faf4c15feb0 2024-04-25T14:09:11.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.690+0000 7faf54600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7faf4c06f270 con 0x7faf4c15c2a0 2024-04-25T14:09:11.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.690+0000 7faf51c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7faf4c15c2a0 0x7faf4c0ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:11.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.694+0000 7faf51c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7faf4c15c2a0 0x7faf4c0ae130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57430/0 (socket says 172.21.15.86:57430) 2024-04-25T14:09:11.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.694+0000 7faf51200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7faf4c06fd50 0x7faf4c070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42794/0 (socket says 172.21.15.86:42794) 2024-04-25T14:09:11.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.694+0000 7faf51c00640 1 -- 172.21.15.86:0/430897888 learned_addr learned my addr 172.21.15.86:0/430897888 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:11.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.694+0000 7faf51c00640 1 -- 172.21.15.86:0/430897888 >> v1:172.21.15.155:6789/0 conn(0x7faf4c15feb0 legacy=0x7faf4c1602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:11.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.694+0000 7faf51c00640 1 -- 172.21.15.86:0/430897888 >> v1:172.21.15.86:6789/0 conn(0x7faf4c06fd50 legacy=0x7faf4c070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:11.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.694+0000 7faf4be00640 1 -- 172.21.15.86:0/430897888 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3795259364 0 0) 0x7faf4c06ef90 con 0x7faf4c15feb0 2024-04-25T14:09:11.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.694+0000 7faf51c00640 1 -- 172.21.15.86:0/430897888 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4c06f550 con 0x7faf4c15c2a0 2024-04-25T14:09:11.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.694+0000 7faf51c00640 1 --2- 172.21.15.86:0/430897888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf4c15c2a0 0x7faf4c0ae130 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7faf4000bb10 tx=0x7faf4001ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebfff75433913e7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:11.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.694+0000 7faf4be00640 1 -- 172.21.15.86:0/430897888 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf400b4070 con 0x7faf4c15c2a0 2024-04-25T14:09:11.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.694+0000 7faf4be00640 1 -- 172.21.15.86:0/430897888 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf400a7aa0 con 0x7faf4c15c2a0 2024-04-25T14:09:11.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.694+0000 7faf4be00640 1 -- 172.21.15.86:0/430897888 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf400af440 con 0x7faf4c15c2a0 2024-04-25T14:09:11.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.698+0000 7faf54600640 1 -- 172.21.15.86:0/430897888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf4c15c2a0 msgr2=0x7faf4c0ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:11.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.698+0000 7faf54600640 1 --2- 172.21.15.86:0/430897888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf4c15c2a0 0x7faf4c0ae130 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7faf4000bb10 tx=0x7faf4001ea30 comp rx=0 tx=0).stop 2024-04-25T14:09:11.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.698+0000 7faf54600640 1 -- 172.21.15.86:0/430897888 shutdown_connections 2024-04-25T14:09:11.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.698+0000 7faf54600640 1 --2- 172.21.15.86:0/430897888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf4c15c2a0 0x7faf4c0ae130 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:11.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.698+0000 7faf54600640 1 -- 172.21.15.86:0/430897888 >> 172.21.15.86:0/430897888 conn(0x7faf4c0b4650 msgr2=0x7faf4c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:11.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.698+0000 7faf54600640 1 -- 172.21.15.86:0/430897888 shutdown_connections 2024-04-25T14:09:11.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.698+0000 7faf54600640 1 -- 172.21.15.86:0/430897888 wait complete. 2024-04-25T14:09:11.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.698+0000 7faf54600640 1 Processor -- start 2024-04-25T14:09:11.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.698+0000 7faf54600640 1 -- start start 2024-04-25T14:09:11.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.702+0000 7faf54600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf4c06fd50 0x7faf4c158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:11.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.702+0000 7faf54600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf4c15c2a0 0x7faf4c09ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:11.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.702+0000 7faf54600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf4c15feb0 0x7faf4c1f6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:11.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.702+0000 7faf54600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faf4c15c6b0 con 0x7faf4c06fd50 2024-04-25T14:09:11.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.702+0000 7faf54600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faf4c0d0fc0 con 0x7faf4c15feb0 2024-04-25T14:09:11.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.702+0000 7faf54600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7faf4c15c830 con 0x7faf4c15c2a0 2024-04-25T14:09:11.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.702+0000 7faf51c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf4c06fd50 0x7faf4c158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:11.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.702+0000 7faf51c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf4c06fd50 0x7faf4c158a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57444/0 (socket says 172.21.15.86:57444) 2024-04-25T14:09:11.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.702+0000 7faf51c00640 1 -- 172.21.15.86:0/4164714093 learned_addr learned my addr 172.21.15.86:0/4164714093 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:11.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.702+0000 7faf51c00640 1 -- 172.21.15.86:0/4164714093 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf4c15c2a0 msgr2=0x7faf4c09ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:11.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.702+0000 7faf51c00640 1 --2- 172.21.15.86:0/4164714093 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf4c15c2a0 0x7faf4c09ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:11.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.702+0000 7faf51c00640 1 -- 172.21.15.86:0/4164714093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf4c15feb0 msgr2=0x7faf4c1f6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:11.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.702+0000 7faf51c00640 1 --2- 172.21.15.86:0/4164714093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf4c15feb0 0x7faf4c1f6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:11.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.702+0000 7faf51c00640 1 -- 172.21.15.86:0/4164714093 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4c09f1d0 con 0x7faf4c06fd50 2024-04-25T14:09:11.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.702+0000 7faf51c00640 1 --2- 172.21.15.86:0/4164714093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf4c06fd50 0x7faf4c158a10 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7faf4000bb10 tx=0x7faf4001d270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:11.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.702+0000 7faf49600640 1 -- 172.21.15.86:0/4164714093 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf400bf070 con 0x7faf4c06fd50 2024-04-25T14:09:11.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.702+0000 7faf49600640 1 -- 172.21.15.86:0/4164714093 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf400afcf0 con 0x7faf4c06fd50 2024-04-25T14:09:11.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.702+0000 7faf54600640 1 -- 172.21.15.86:0/4164714093 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf4c15a290 con 0x7faf4c06fd50 2024-04-25T14:09:11.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.702+0000 7faf49600640 1 -- 172.21.15.86:0/4164714093 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf400b4040 con 0x7faf4c06fd50 2024-04-25T14:09:11.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.702+0000 7faf54600640 1 -- 172.21.15.86:0/4164714093 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf4c0b17c0 con 0x7faf4c06fd50 2024-04-25T14:09:11.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.702+0000 7faf54600640 1 -- 172.21.15.86:0/4164714093 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf4c069d00 con 0x7faf4c06fd50 2024-04-25T14:09:11.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.702+0000 7faf49600640 1 -- 172.21.15.86:0/4164714093 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 35) ==== 93045+0+0 (secure 0 0 0) 0x7faf400c2070 con 0x7faf4c06fd50 2024-04-25T14:09:11.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.706+0000 7faf49600640 1 --2- 172.21.15.86:0/4164714093 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7faf2c06f280 0x7faf2c071790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:11.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.706+0000 7faf49600640 1 -- 172.21.15.86:0/4164714093 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6746+0+0 (secure 0 0 0) 0x7faf40129b70 con 0x7faf4c06fd50 2024-04-25T14:09:11.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.706+0000 7faf51200640 1 --2- 172.21.15.86:0/4164714093 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7faf2c06f280 0x7faf2c071790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:11.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.706+0000 7faf51200640 1 --2- 172.21.15.86:0/4164714093 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7faf2c06f280 0x7faf2c071790 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7faf38001ec0 tx=0x7faf38001c70 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:11.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.706+0000 7faf49600640 1 -- 172.21.15.86:0/4164714093 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faf40018740 con 0x7faf4c06fd50 2024-04-25T14:09:11.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.898+0000 7faf54600640 1 -- 172.21.15.86:0/4164714093 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf4c06d690 con 0x7faf4c06fd50 2024-04-25T14:09:11.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.898+0000 7faf49600640 1 -- 172.21.15.86:0/4164714093 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5186 (secure 0 0 0) 0x7faf400f69d0 con 0x7faf4c06fd50 2024-04-25T14:09:11.896 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:11.896 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7186,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/4265929298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4265929298},{"type":"v1","addr":"172.21.15.155:6833","nonce":4265929298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7189,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3254219000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3254219000},{"type":"v1","addr":"172.21.15.155:6835","nonce":3254219000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7195,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/193454084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":193454084},{"type":"v1","addr":"172.21.15.86:6837","nonce":193454084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:08:45.096500+0000","modified":"2024-04-25T14:08:48.121611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7371},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7371":{"gid":7371,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/1961103704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1961103704},{"type":"v1","addr":"172.21.15.86:6835","nonce":1961103704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7371,"qdb_cluster":[7371]},"id":6}]} 2024-04-25T14:09:11.897 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 67 2024-04-25T14:09:11.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.902+0000 7faf54600640 1 -- 172.21.15.86:0/4164714093 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7faf2c06f280 msgr2=0x7faf2c071790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:11.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.902+0000 7faf54600640 1 --2- 172.21.15.86:0/4164714093 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7faf2c06f280 0x7faf2c071790 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7faf38001ec0 tx=0x7faf38001c70 comp rx=0 tx=0).stop 2024-04-25T14:09:11.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.902+0000 7faf54600640 1 -- 172.21.15.86:0/4164714093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf4c06fd50 msgr2=0x7faf4c158a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:11.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.902+0000 7faf54600640 1 --2- 172.21.15.86:0/4164714093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf4c06fd50 0x7faf4c158a10 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7faf4000bb10 tx=0x7faf4001d270 comp rx=0 tx=0).stop 2024-04-25T14:09:11.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.902+0000 7faf54600640 1 -- 172.21.15.86:0/4164714093 shutdown_connections 2024-04-25T14:09:11.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.902+0000 7faf54600640 1 --2- 172.21.15.86:0/4164714093 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7faf2c06f280 0x7faf2c071790 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:11.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.902+0000 7faf54600640 1 --2- 172.21.15.86:0/4164714093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf4c15feb0 0x7faf4c1f6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:11.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.902+0000 7faf54600640 1 --2- 172.21.15.86:0/4164714093 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf4c15c2a0 0x7faf4c09ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:11.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.902+0000 7faf54600640 1 --2- 172.21.15.86:0/4164714093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf4c06fd50 0x7faf4c158a10 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:11.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.902+0000 7faf54600640 1 -- 172.21.15.86:0/4164714093 >> 172.21.15.86:0/4164714093 conn(0x7faf4c0b4650 msgr2=0x7faf4c072c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:11.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.902+0000 7faf54600640 1 -- 172.21.15.86:0/4164714093 shutdown_connections 2024-04-25T14:09:11.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:11.902+0000 7faf54600640 1 -- 172.21.15.86:0/4164714093 wait complete. 2024-04-25T14:09:11.917 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:09:12.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.066+0000 7f6981e00640 1 Processor -- start 2024-04-25T14:09:12.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.066+0000 7f6981e00640 1 -- start start 2024-04-25T14:09:12.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.066+0000 7f6981e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f697c0b69a0 0x7f697c0b6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:12.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.066+0000 7f6981e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f697c06ef30 0x7f697c06f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:12.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.066+0000 7f6981e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f697c161510 0x7f697c161920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:12.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.066+0000 7f6981e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f697c15ddd0 con 0x7f697c161510 2024-04-25T14:09:12.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.066+0000 7f6981e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f697c0b73f0 con 0x7f697c06ef30 2024-04-25T14:09:12.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.066+0000 7f6981e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f697c0b76d0 con 0x7f697c0b69a0 2024-04-25T14:09:12.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.066+0000 7f697b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f697c161510 0x7f697c161920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:12.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.066+0000 7f697b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f697c161510 0x7f697c161920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57448/0 (socket says 172.21.15.86:57448) 2024-04-25T14:09:12.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.066+0000 7f697b400640 1 -- 172.21.15.86:0/3790759685 learned_addr learned my addr 172.21.15.86:0/3790759685 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:12.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.066+0000 7f697aa00640 1 --2- 172.21.15.86:0/3790759685 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f697c0b69a0 0x7f697c0b6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:12.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.066+0000 7f6972000640 1 --2- 172.21.15.86:0/3790759685 >> v2:172.21.15.155:3300/0 conn(0x7f697c06ef30 0x7f697c06f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:12.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.066+0000 7f697b400640 1 -- 172.21.15.86:0/3790759685 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f697c0b69a0 msgr2=0x7f697c0b6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:12.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.066+0000 7f697b400640 1 --2- 172.21.15.86:0/3790759685 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f697c0b69a0 0x7f697c0b6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:12.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.066+0000 7f697b400640 1 -- 172.21.15.86:0/3790759685 >> v2:172.21.15.155:3300/0 conn(0x7f697c06ef30 msgr2=0x7f697c06f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:12.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.066+0000 7f697b400640 1 --2- 172.21.15.86:0/3790759685 >> v2:172.21.15.155:3300/0 conn(0x7f697c06ef30 0x7f697c06f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:12.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.066+0000 7f697b400640 1 -- 172.21.15.86:0/3790759685 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f697c06e730 con 0x7f697c161510 2024-04-25T14:09:12.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.066+0000 7f697aa00640 1 --2- 172.21.15.86:0/3790759685 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f697c0b69a0 0x7f697c0b6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:12.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.070+0000 7f697b400640 1 --2- 172.21.15.86:0/3790759685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f697c161510 0x7f697c161920 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f697c0b61c0 tx=0x7f697401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef8c9ef034383c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:12.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.070+0000 7f697a000640 1 -- 172.21.15.86:0/3790759685 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69740b2070 con 0x7f697c161510 2024-04-25T14:09:12.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.070+0000 7f697a000640 1 -- 172.21.15.86:0/3790759685 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69740a9070 con 0x7f697c161510 2024-04-25T14:09:12.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.070+0000 7f697a000640 1 -- 172.21.15.86:0/3790759685 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69740ad880 con 0x7f697c161510 2024-04-25T14:09:12.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.070+0000 7f6981e00640 1 -- 172.21.15.86:0/3790759685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f697c161510 msgr2=0x7f697c161920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:12.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.070+0000 7f6981e00640 1 --2- 172.21.15.86:0/3790759685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f697c161510 0x7f697c161920 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f697c0b61c0 tx=0x7f697401c490 comp rx=0 tx=0).stop 2024-04-25T14:09:12.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.070+0000 7f6981e00640 1 -- 172.21.15.86:0/3790759685 shutdown_connections 2024-04-25T14:09:12.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.070+0000 7f6981e00640 1 --2- 172.21.15.86:0/3790759685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f697c161510 0x7f697c161920 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:12.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.070+0000 7f6981e00640 1 --2- 172.21.15.86:0/3790759685 >> v2:172.21.15.155:3300/0 conn(0x7f697c06ef30 0x7f697c06f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:12.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.070+0000 7f6981e00640 1 --2- 172.21.15.86:0/3790759685 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f697c0b69a0 0x7f697c0b6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:12.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.070+0000 7f6981e00640 1 -- 172.21.15.86:0/3790759685 >> 172.21.15.86:0/3790759685 conn(0x7f697c0b5100 msgr2=0x7f697c0b5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:12.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.070+0000 7f6981e00640 1 -- 172.21.15.86:0/3790759685 shutdown_connections 2024-04-25T14:09:12.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.070+0000 7f6981e00640 1 -- 172.21.15.86:0/3790759685 wait complete. 2024-04-25T14:09:12.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.074+0000 7f6981e00640 1 Processor -- start 2024-04-25T14:09:12.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.074+0000 7f6981e00640 1 -- start start 2024-04-25T14:09:12.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.074+0000 7f6981e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f697c06ef30 0x7f697c14f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:12.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.074+0000 7f6981e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f697c0b69a0 0x7f697c152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:12.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.074+0000 7f6981e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f697c161510 0x7f697c152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:12.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.074+0000 7f6981e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f697c163ac0 con 0x7f697c0b69a0 2024-04-25T14:09:12.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.074+0000 7f6981e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f697c163940 con 0x7f697c161510 2024-04-25T14:09:12.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.074+0000 7f6981e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f697c163c40 con 0x7f697c06ef30 2024-04-25T14:09:12.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.074+0000 7f6972000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f697c0b69a0 0x7f697c152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:12.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.074+0000 7f6972000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f697c0b69a0 0x7f697c152a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57462/0 (socket says 172.21.15.86:57462) 2024-04-25T14:09:12.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.074+0000 7f6972000640 1 -- 172.21.15.86:0/1796819165 learned_addr learned my addr 172.21.15.86:0/1796819165 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:12.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.078+0000 7f697aa00640 1 --2- 172.21.15.86:0/1796819165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f697c06ef30 0x7f697c14f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:12.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.078+0000 7f697b400640 1 --2- 172.21.15.86:0/1796819165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f697c161510 0x7f697c152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:12.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.078+0000 7f6972000640 1 -- 172.21.15.86:0/1796819165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f697c06ef30 msgr2=0x7f697c14f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:12.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.078+0000 7f6972000640 1 --2- 172.21.15.86:0/1796819165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f697c06ef30 0x7f697c14f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:12.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.078+0000 7f6972000640 1 -- 172.21.15.86:0/1796819165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f697c161510 msgr2=0x7f697c152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:12.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.078+0000 7f6972000640 1 --2- 172.21.15.86:0/1796819165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f697c161510 0x7f697c152f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:12.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.078+0000 7f6972000640 1 -- 172.21.15.86:0/1796819165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f697c151e80 con 0x7f697c0b69a0 2024-04-25T14:09:12.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.078+0000 7f697aa00640 1 --2- 172.21.15.86:0/1796819165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f697c06ef30 0x7f697c14f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:12.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.078+0000 7f697b400640 1 --2- 172.21.15.86:0/1796819165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f697c161510 0x7f697c152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:12.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.078+0000 7f6972000640 1 --2- 172.21.15.86:0/1796819165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f697c0b69a0 0x7f697c152a50 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f696400d800 tx=0x7f696400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:12.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.078+0000 7f6973e00640 1 -- 172.21.15.86:0/1796819165 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6964018020 con 0x7f697c0b69a0 2024-04-25T14:09:12.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.078+0000 7f6973e00640 1 -- 172.21.15.86:0/1796819165 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6964009640 con 0x7f697c0b69a0 2024-04-25T14:09:12.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.078+0000 7f6973e00640 1 -- 172.21.15.86:0/1796819165 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6964009a30 con 0x7f697c0b69a0 2024-04-25T14:09:12.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.078+0000 7f6981e00640 1 -- 172.21.15.86:0/1796819165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f697c15bce0 con 0x7f697c0b69a0 2024-04-25T14:09:12.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.078+0000 7f6981e00640 1 -- 172.21.15.86:0/1796819165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f697c154b50 con 0x7f697c0b69a0 2024-04-25T14:09:12.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.078+0000 7f6981e00640 1 -- 172.21.15.86:0/1796819165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f697c072200 con 0x7f697c0b69a0 2024-04-25T14:09:12.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.078+0000 7f6973e00640 1 -- 172.21.15.86:0/1796819165 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 35) ==== 93045+0+0 (secure 0 0 0) 0x7f696401b020 con 0x7f697c0b69a0 2024-04-25T14:09:12.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.078+0000 7f6973e00640 1 --2- 172.21.15.86:0/1796819165 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f695806f280 0x7f6958071790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:12.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.078+0000 7f6973e00640 1 -- 172.21.15.86:0/1796819165 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6746+0+0 (secure 0 0 0) 0x7f696408f550 con 0x7f697c0b69a0 2024-04-25T14:09:12.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.082+0000 7f697aa00640 1 --2- 172.21.15.86:0/1796819165 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f695806f280 0x7f6958071790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:12.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.082+0000 7f697aa00640 1 --2- 172.21.15.86:0/1796819165 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f695806f280 0x7f6958071790 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f696c009540 tx=0x7f696c00b040 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:12.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.086+0000 7f6973e00640 1 -- 172.21.15.86:0/1796819165 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f696405c400 con 0x7f697c0b69a0 2024-04-25T14:09:12.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.242+0000 7f6981e00640 1 -- 172.21.15.86:0/1796819165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f697c076bf0 con 0x7f697c0b69a0 2024-04-25T14:09:12.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.246+0000 7f6973e00640 1 -- 172.21.15.86:0/1796819165 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+17823 (secure 0 0 0) 0x7f696405bd00 con 0x7f697c0b69a0 2024-04-25T14:09:12.242 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:12.243 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:09:09.480535+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:08:43.505988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T14:08:44.054628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2882289101":"2024-04-26T14:09:09.480502+0000","172.21.15.86:6833/132942210":"2024-04-26T14:09:09.480502+0000","172.21.15.86:6832/132942210":"2024-04-26T14:09:09.480502+0000","172.21.15.86:0/3672669933":"2024-04-26T14:09:09.480502+0000","172.21.15.86:0/402603388":"2024-04-26T14:09:09.480502+0000","172.21.15.86:0/3743735908":"2024-04-26T14:09:09.480502+0000","172.21.15.86:0/3132647616":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/4104483212":"2024-04-26T14:09:09.480502+0000","172.21.15.86:0/1564187566":"2024-04-26T14:09:03.346201+0000","172.21.15.86:6833/3589556609":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/1099362620":"2024-04-26T14:09:03.346201+0000","172.21.15.86:6832/3589556609":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/3481243210":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/2509210906":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/3722630918":"2024-04-26T14:09:03.346201+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:09:12.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.246+0000 7f6981e00640 1 -- 172.21.15.86:0/1796819165 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f695806f280 msgr2=0x7f6958071790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:12.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.246+0000 7f6981e00640 1 --2- 172.21.15.86:0/1796819165 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f695806f280 0x7f6958071790 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f696c009540 tx=0x7f696c00b040 comp rx=0 tx=0).stop 2024-04-25T14:09:12.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.246+0000 7f6981e00640 1 -- 172.21.15.86:0/1796819165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f697c0b69a0 msgr2=0x7f697c152a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:12.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.246+0000 7f6981e00640 1 --2- 172.21.15.86:0/1796819165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f697c0b69a0 0x7f697c152a50 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f696400d800 tx=0x7f696400dcc0 comp rx=0 tx=0).stop 2024-04-25T14:09:12.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.246+0000 7f6981e00640 1 -- 172.21.15.86:0/1796819165 shutdown_connections 2024-04-25T14:09:12.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.246+0000 7f6981e00640 1 --2- 172.21.15.86:0/1796819165 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f695806f280 0x7f6958071790 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:12.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.246+0000 7f6981e00640 1 --2- 172.21.15.86:0/1796819165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f697c161510 0x7f697c152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:12.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.246+0000 7f6981e00640 1 --2- 172.21.15.86:0/1796819165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f697c0b69a0 0x7f697c152a50 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:12.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.246+0000 7f6981e00640 1 --2- 172.21.15.86:0/1796819165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f697c06ef30 0x7f697c14f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:12.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.246+0000 7f6981e00640 1 -- 172.21.15.86:0/1796819165 >> 172.21.15.86:0/1796819165 conn(0x7f697c0b5100 msgr2=0x7f697c157330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:12.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.246+0000 7f6981e00640 1 -- 172.21.15.86:0/1796819165 shutdown_connections 2024-04-25T14:09:12.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.246+0000 7f6981e00640 1 -- 172.21.15.86:0/1796819165 wait complete. 2024-04-25T14:09:12.263 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T14:09:12.263 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:09:12.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.414+0000 7f57d2000640 1 Processor -- start 2024-04-25T14:09:12.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.414+0000 7f57d2000640 1 -- start start 2024-04-25T14:09:12.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.418+0000 7f57d2000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f57cc0cd090 0x7f57cc0cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:12.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.418+0000 7f57d2000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57cc0d2960 con 0x7f57cc0cd9e0 2024-04-25T14:09:12.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.418+0000 7f57d2000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57cc0d2c40 con 0x7f57cc0b3740 2024-04-25T14:09:12.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.418+0000 7f57d2000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f57cc06c670 con 0x7f57cc0cd090 2024-04-25T14:09:12.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.418+0000 7f57cbe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f57cc0cd9e0 0x7f57cc16a9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42802/0 (socket says 172.21.15.86:42802) 2024-04-25T14:09:12.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.418+0000 7f57cbe00640 1 -- 172.21.15.86:0/3714896375 learned_addr learned my addr 172.21.15.86:0/3714896375 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:12.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.418+0000 7f57c2a00640 1 --2- 172.21.15.86:0/3714896375 >> v2:172.21.15.155:3300/0 conn(0x7f57cc0cd090 0x7f57cc0cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:12.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.418+0000 7f57caa00640 1 -- 172.21.15.86:0/3714896375 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2375463441 0 0) 0x7f57cc0d2960 con 0x7f57cc0cd9e0 2024-04-25T14:09:12.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.418+0000 7f57caa00640 1 -- 172.21.15.86:0/3714896375 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57b0003530 con 0x7f57cc0cd9e0 2024-04-25T14:09:12.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.418+0000 7f57caa00640 1 -- 172.21.15.86:0/3714896375 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3765716325 0 0) 0x7f57cc0d2c40 con 0x7f57cc0b3740 2024-04-25T14:09:12.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.418+0000 7f57caa00640 1 -- 172.21.15.86:0/3714896375 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57cc0d2960 con 0x7f57cc0b3740 2024-04-25T14:09:12.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.418+0000 7f57c2a00640 1 -- 172.21.15.86:0/3714896375 >> v1:172.21.15.155:6789/0 conn(0x7f57cc0b3740 legacy=0x7f57cc0ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:12.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.418+0000 7f57c2a00640 1 -- 172.21.15.86:0/3714896375 >> v1:172.21.15.86:6789/0 conn(0x7f57cc0cd9e0 legacy=0x7f57cc16a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:12.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.418+0000 7f57c2a00640 1 -- 172.21.15.86:0/3714896375 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57cc0d2f20 con 0x7f57cc0cd090 2024-04-25T14:09:12.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.422+0000 7f57c2a00640 1 --2- 172.21.15.86:0/3714896375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57cc0cd090 0x7f57cc0cd4a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f57b40097c0 tx=0x7f57b401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6f669b7bb57242d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:12.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.422+0000 7f57caa00640 1 -- 172.21.15.86:0/3714896375 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b40b0070 con 0x7f57cc0cd090 2024-04-25T14:09:12.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.422+0000 7f57caa00640 1 -- 172.21.15.86:0/3714896375 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57b40ab600 con 0x7f57cc0cd090 2024-04-25T14:09:12.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.422+0000 7f57caa00640 1 -- 172.21.15.86:0/3714896375 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b40ab9b0 con 0x7f57cc0cd090 2024-04-25T14:09:12.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.422+0000 7f57d2000640 1 -- 172.21.15.86:0/3714896375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57cc0cd090 msgr2=0x7f57cc0cd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:12.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.422+0000 7f57d2000640 1 --2- 172.21.15.86:0/3714896375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57cc0cd090 0x7f57cc0cd4a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f57b40097c0 tx=0x7f57b401c4a0 comp rx=0 tx=0).stop 2024-04-25T14:09:12.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.422+0000 7f57d2000640 1 -- 172.21.15.86:0/3714896375 shutdown_connections 2024-04-25T14:09:12.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.422+0000 7f57d2000640 1 --2- 172.21.15.86:0/3714896375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57cc0cd090 0x7f57cc0cd4a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:12.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.422+0000 7f57d2000640 1 -- 172.21.15.86:0/3714896375 >> 172.21.15.86:0/3714896375 conn(0x7f57cc09ba30 msgr2=0x7f57cc0b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:12.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.422+0000 7f57d2000640 1 -- 172.21.15.86:0/3714896375 shutdown_connections 2024-04-25T14:09:12.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.422+0000 7f57d2000640 1 -- 172.21.15.86:0/3714896375 wait complete. 2024-04-25T14:09:12.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.426+0000 7f57d2000640 1 Processor -- start 2024-04-25T14:09:12.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.426+0000 7f57d2000640 1 -- start start 2024-04-25T14:09:12.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.426+0000 7f57d2000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57cc0b3740 0x7f57cc155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:12.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.426+0000 7f57d2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57cc0cd090 0x7f57cc1626a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:12.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.426+0000 7f57d2000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57cc0cd9e0 0x7f57cc160420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:12.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.426+0000 7f57d2000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc0b3ce0 con 0x7f57cc0cd090 2024-04-25T14:09:12.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.426+0000 7f57d2000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc0cfca0 con 0x7f57cc0cd9e0 2024-04-25T14:09:12.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.426+0000 7f57d2000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f57cc16e910 con 0x7f57cc0b3740 2024-04-25T14:09:12.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.426+0000 7f57c2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57cc0cd090 0x7f57cc1626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:12.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.426+0000 7f57c2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57cc0cd090 0x7f57cc1626a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57474/0 (socket says 172.21.15.86:57474) 2024-04-25T14:09:12.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.426+0000 7f57c2a00640 1 -- 172.21.15.86:0/3550991878 learned_addr learned my addr 172.21.15.86:0/3550991878 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:12.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.426+0000 7f57cb400640 1 --2- 172.21.15.86:0/3550991878 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57cc0b3740 0x7f57cc155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:12.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.426+0000 7f57cbe00640 1 --2- 172.21.15.86:0/3550991878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57cc0cd9e0 0x7f57cc160420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:12.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.426+0000 7f57c2a00640 1 -- 172.21.15.86:0/3550991878 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57cc0b3740 msgr2=0x7f57cc155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:12.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.426+0000 7f57c2a00640 1 --2- 172.21.15.86:0/3550991878 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57cc0b3740 0x7f57cc155230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:12.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.426+0000 7f57c2a00640 1 -- 172.21.15.86:0/3550991878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57cc0cd9e0 msgr2=0x7f57cc160420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:12.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.426+0000 7f57c2a00640 1 --2- 172.21.15.86:0/3550991878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57cc0cd9e0 0x7f57cc160420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:12.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.426+0000 7f57c2a00640 1 -- 172.21.15.86:0/3550991878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57cc162fc0 con 0x7f57cc0cd090 2024-04-25T14:09:12.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.426+0000 7f57c2a00640 1 --2- 172.21.15.86:0/3550991878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57cc0cd090 0x7f57cc1626a0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f57b40ae040 tx=0x7f57b4019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:12.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.426+0000 7f57c3e00640 1 -- 172.21.15.86:0/3550991878 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b40b0040 con 0x7f57cc0cd090 2024-04-25T14:09:12.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.426+0000 7f57c3e00640 1 -- 172.21.15.86:0/3550991878 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57b40ab400 con 0x7f57cc0cd090 2024-04-25T14:09:12.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.426+0000 7f57d2000640 1 -- 172.21.15.86:0/3550991878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57cc0c7b60 con 0x7f57cc0cd090 2024-04-25T14:09:12.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.426+0000 7f57c3e00640 1 -- 172.21.15.86:0/3550991878 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b4006e20 con 0x7f57cc0cd090 2024-04-25T14:09:12.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.426+0000 7f57d2000640 1 -- 172.21.15.86:0/3550991878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57cc0c80e0 con 0x7f57cc0cd090 2024-04-25T14:09:12.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.430+0000 7f57d2000640 1 -- 172.21.15.86:0/3550991878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57cc0a0630 con 0x7f57cc0cd090 2024-04-25T14:09:12.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.430+0000 7f57c3e00640 1 -- 172.21.15.86:0/3550991878 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 36) ==== 93253+0+0 (secure 0 0 0) 0x7f57b40bb030 con 0x7f57cc0cd090 2024-04-25T14:09:12.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.430+0000 7f57c3e00640 1 --2- 172.21.15.86:0/3550991878 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f57a806f4c0 0x7f57a80719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:12.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.430+0000 7f57c3e00640 1 -- 172.21.15.86:0/3550991878 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6746+0+0 (secure 0 0 0) 0x7f57b4128dc0 con 0x7f57cc0cd090 2024-04-25T14:09:12.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.430+0000 7f57cb400640 1 --2- 172.21.15.86:0/3550991878 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f57a806f4c0 0x7f57a80719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:12.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.430+0000 7f57cb400640 1 --2- 172.21.15.86:0/3550991878 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f57a806f4c0 0x7f57a80719d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f57cc073560 tx=0x7f57bc009040 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:12.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.434+0000 7f57c3e00640 1 -- 172.21.15.86:0/3550991878 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f57b40c2060 con 0x7f57cc0cd090 2024-04-25T14:09:12.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.626+0000 7f57d2000640 1 -- 172.21.15.86:0/3550991878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57cc076bf0 con 0x7f57cc0cd090 2024-04-25T14:09:12.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.626+0000 7f57c3e00640 1 -- 172.21.15.86:0/3550991878 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5186 (secure 0 0 0) 0x7f57b4016090 con 0x7f57cc0cd090 2024-04-25T14:09:12.624 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:12.625 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7186,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/4265929298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4265929298},{"type":"v1","addr":"172.21.15.155:6833","nonce":4265929298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7189,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3254219000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3254219000},{"type":"v1","addr":"172.21.15.155:6835","nonce":3254219000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7195,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/193454084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":193454084},{"type":"v1","addr":"172.21.15.86:6837","nonce":193454084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:08:45.096500+0000","modified":"2024-04-25T14:08:48.121611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7371},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7371":{"gid":7371,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/1961103704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1961103704},{"type":"v1","addr":"172.21.15.86:6835","nonce":1961103704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7371,"qdb_cluster":[7371]},"id":6}]} 2024-04-25T14:09:12.625 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 67 2024-04-25T14:09:12.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.630+0000 7f57d2000640 1 -- 172.21.15.86:0/3550991878 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f57a806f4c0 msgr2=0x7f57a80719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:12.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.630+0000 7f57d2000640 1 --2- 172.21.15.86:0/3550991878 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f57a806f4c0 0x7f57a80719d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f57cc073560 tx=0x7f57bc009040 comp rx=0 tx=0).stop 2024-04-25T14:09:12.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.630+0000 7f57d2000640 1 -- 172.21.15.86:0/3550991878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57cc0cd090 msgr2=0x7f57cc1626a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:12.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.630+0000 7f57d2000640 1 --2- 172.21.15.86:0/3550991878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57cc0cd090 0x7f57cc1626a0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f57b40ae040 tx=0x7f57b4019240 comp rx=0 tx=0).stop 2024-04-25T14:09:12.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.630+0000 7f57d2000640 1 -- 172.21.15.86:0/3550991878 shutdown_connections 2024-04-25T14:09:12.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.630+0000 7f57d2000640 1 --2- 172.21.15.86:0/3550991878 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f57a806f4c0 0x7f57a80719d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:12.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.630+0000 7f57d2000640 1 --2- 172.21.15.86:0/3550991878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57cc0cd9e0 0x7f57cc160420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:12.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.630+0000 7f57d2000640 1 --2- 172.21.15.86:0/3550991878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57cc0cd090 0x7f57cc1626a0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:12.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.630+0000 7f57d2000640 1 --2- 172.21.15.86:0/3550991878 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57cc0b3740 0x7f57cc155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:12.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.630+0000 7f57d2000640 1 -- 172.21.15.86:0/3550991878 >> 172.21.15.86:0/3550991878 conn(0x7f57cc09ba30 msgr2=0x7f57cc0ce810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:12.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.630+0000 7f57d2000640 1 -- 172.21.15.86:0/3550991878 shutdown_connections 2024-04-25T14:09:12.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.630+0000 7f57d2000640 1 -- 172.21.15.86:0/3550991878 wait complete. 2024-04-25T14:09:12.646 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:09:12.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.794+0000 7fbd94600640 1 Processor -- start 2024-04-25T14:09:12.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.798+0000 7fbd94600640 1 -- start start 2024-04-25T14:09:12.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.798+0000 7fbd94600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd8c0b3a50 0x7fbd8c168f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:12.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.798+0000 7fbd94600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd8c0cefe0 con 0x7fbd8c0d2b00 2024-04-25T14:09:12.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.798+0000 7fbd94600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd8c0b1d00 con 0x7fbd8c0b3640 2024-04-25T14:09:12.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.798+0000 7fbd94600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbd8c0b2910 con 0x7fbd8c0b3a50 2024-04-25T14:09:12.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.798+0000 7fbd91200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fbd8c0d2b00 0x7fbd8c1657f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42806/0 (socket says 172.21.15.86:42806) 2024-04-25T14:09:12.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.798+0000 7fbd91200640 1 -- 172.21.15.86:0/3461237750 learned_addr learned my addr 172.21.15.86:0/3461237750 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:12.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.798+0000 7fbd92600640 1 --2- 172.21.15.86:0/3461237750 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd8c0b3a50 0x7fbd8c168f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:12.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.798+0000 7fbd8be00640 1 -- 172.21.15.86:0/3461237750 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1895323787 0 0) 0x7fbd8c0cefe0 con 0x7fbd8c0d2b00 2024-04-25T14:09:12.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.798+0000 7fbd8be00640 1 -- 172.21.15.86:0/3461237750 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd74003530 con 0x7fbd8c0d2b00 2024-04-25T14:09:12.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.798+0000 7fbd8be00640 1 -- 172.21.15.86:0/3461237750 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2748165183 0 0) 0x7fbd8c0b1d00 con 0x7fbd8c0b3640 2024-04-25T14:09:12.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.798+0000 7fbd8be00640 1 -- 172.21.15.86:0/3461237750 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd8c0cefe0 con 0x7fbd8c0b3640 2024-04-25T14:09:12.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.798+0000 7fbd8be00640 1 -- 172.21.15.86:0/3461237750 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 550350047 0 0) 0x7fbd74003530 con 0x7fbd8c0d2b00 2024-04-25T14:09:12.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.798+0000 7fbd92600640 1 -- 172.21.15.86:0/3461237750 >> v1:172.21.15.155:6789/0 conn(0x7fbd8c0b3640 legacy=0x7fbd8c0d23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:12.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.798+0000 7fbd92600640 1 -- 172.21.15.86:0/3461237750 >> v1:172.21.15.86:6789/0 conn(0x7fbd8c0d2b00 legacy=0x7fbd8c1657f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:12.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.798+0000 7fbd92600640 1 -- 172.21.15.86:0/3461237750 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd8c0b1fe0 con 0x7fbd8c0b3a50 2024-04-25T14:09:12.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.802+0000 7fbd92600640 1 --2- 172.21.15.86:0/3461237750 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd8c0b3a50 0x7fbd8c168f20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009dc0 tx=0x7fbd8401c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=895806e7befccdd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:12.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.802+0000 7fbd8be00640 1 -- 172.21.15.86:0/3461237750 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd840b2070 con 0x7fbd8c0b3a50 2024-04-25T14:09:12.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.802+0000 7fbd8be00640 1 -- 172.21.15.86:0/3461237750 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd840a55e0 con 0x7fbd8c0b3a50 2024-04-25T14:09:12.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.802+0000 7fbd8be00640 1 -- 172.21.15.86:0/3461237750 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd78001680 con 0x7fbd8c0b3a50 2024-04-25T14:09:12.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.802+0000 7fbd94600640 1 -- 172.21.15.86:0/3461237750 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd8c0b3a50 msgr2=0x7fbd8c168f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:12.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.802+0000 7fbd94600640 1 --2- 172.21.15.86:0/3461237750 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd8c0b3a50 0x7fbd8c168f20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009dc0 tx=0x7fbd8401c8b0 comp rx=0 tx=0).stop 2024-04-25T14:09:12.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.802+0000 7fbd94600640 1 -- 172.21.15.86:0/3461237750 shutdown_connections 2024-04-25T14:09:12.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.802+0000 7fbd94600640 1 --2- 172.21.15.86:0/3461237750 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd8c0b3a50 0x7fbd8c168f20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:12.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.802+0000 7fbd94600640 1 -- 172.21.15.86:0/3461237750 >> 172.21.15.86:0/3461237750 conn(0x7fbd8c09ba30 msgr2=0x7fbd8c0b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:12.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.802+0000 7fbd94600640 1 -- 172.21.15.86:0/3461237750 shutdown_connections 2024-04-25T14:09:12.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.802+0000 7fbd94600640 1 -- 172.21.15.86:0/3461237750 wait complete. 2024-04-25T14:09:12.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.806+0000 7fbd94600640 1 Processor -- start 2024-04-25T14:09:12.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.806+0000 7fbd94600640 1 -- start start 2024-04-25T14:09:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.806+0000 7fbd94600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd8c0b3640 0x7fbd8c15a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.806+0000 7fbd94600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd8c0b3a50 0x7fbd8c09ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.806+0000 7fbd94600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd8c09d1f0 0x7fbd8c09f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.806+0000 7fbd94600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c0b2f20 con 0x7fbd8c0b3640 2024-04-25T14:09:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.806+0000 7fbd94600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c0b2da0 con 0x7fbd8c09d1f0 2024-04-25T14:09:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.806+0000 7fbd94600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbd8c16bc80 con 0x7fbd8c0b3a50 2024-04-25T14:09:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.806+0000 7fbd91c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd8c0b3640 0x7fbd8c15a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.806+0000 7fbd91c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd8c0b3640 0x7fbd8c15a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57476/0 (socket says 172.21.15.86:57476) 2024-04-25T14:09:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.806+0000 7fbd91c00640 1 -- 172.21.15.86:0/2705701801 learned_addr learned my addr 172.21.15.86:0/2705701801 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.806+0000 7fbd91c00640 1 -- 172.21.15.86:0/2705701801 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd8c0b3a50 msgr2=0x7fbd8c09ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.806+0000 7fbd91c00640 1 --2- 172.21.15.86:0/2705701801 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd8c0b3a50 0x7fbd8c09ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.806+0000 7fbd91c00640 1 -- 172.21.15.86:0/2705701801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd8c09d1f0 msgr2=0x7fbd8c09f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.806+0000 7fbd91c00640 1 --2- 172.21.15.86:0/2705701801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd8c09d1f0 0x7fbd8c09f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.806+0000 7fbd91c00640 1 -- 172.21.15.86:0/2705701801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd8c09da30 con 0x7fbd8c0b3640 2024-04-25T14:09:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.806+0000 7fbd91c00640 1 --2- 172.21.15.86:0/2705701801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd8c0b3640 0x7fbd8c15a550 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000c460 tx=0x7fbd8000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.806+0000 7fbd89600640 1 -- 172.21.15.86:0/2705701801 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd80017070 con 0x7fbd8c0b3640 2024-04-25T14:09:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.806+0000 7fbd89600640 1 -- 172.21.15.86:0/2705701801 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd800052f0 con 0x7fbd8c0b3640 2024-04-25T14:09:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.806+0000 7fbd94600640 1 -- 172.21.15.86:0/2705701801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd8c09dd10 con 0x7fbd8c0b3640 2024-04-25T14:09:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.806+0000 7fbd89600640 1 -- 172.21.15.86:0/2705701801 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd80003840 con 0x7fbd8c0b3640 2024-04-25T14:09:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.810+0000 7fbd94600640 1 -- 172.21.15.86:0/2705701801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd8c09e2d0 con 0x7fbd8c0b3640 2024-04-25T14:09:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.810+0000 7fbd94600640 1 -- 172.21.15.86:0/2705701801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd8c0a0630 con 0x7fbd8c0b3640 2024-04-25T14:09:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.810+0000 7fbd89600640 1 -- 172.21.15.86:0/2705701801 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 36) ==== 93253+0+0 (secure 0 0 0) 0x7fbd8001a070 con 0x7fbd8c0b3640 2024-04-25T14:09:12.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.810+0000 7fbd89600640 1 --2- 172.21.15.86:0/2705701801 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fbd6c06f4c0 0x7fbd6c0719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:12.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.810+0000 7fbd89600640 1 -- 172.21.15.86:0/2705701801 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6746+0+0 (secure 0 0 0) 0x7fbd8008ee90 con 0x7fbd8c0b3640 2024-04-25T14:09:12.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.810+0000 7fbd91200640 1 --2- 172.21.15.86:0/2705701801 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fbd6c06f4c0 0x7fbd6c0719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:12.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.814+0000 7fbd91200640 1 --2- 172.21.15.86:0/2705701801 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fbd6c06f4c0 0x7fbd6c0719d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbd78005890 tx=0x7fbd78000fd0 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:12.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:12.814+0000 7fbd89600640 1 -- 172.21.15.86:0/2705701801 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbd80021020 con 0x7fbd8c0b3640 2024-04-25T14:09:13.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.002+0000 7fbd94600640 1 -- 172.21.15.86:0/2705701801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd8c076bf0 con 0x7fbd8c0b3640 2024-04-25T14:09:13.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.006+0000 7fbd89600640 1 -- 172.21.15.86:0/2705701801 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5186 (secure 0 0 0) 0x7fbd8005ba00 con 0x7fbd8c0b3640 2024-04-25T14:09:13.002 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:13.003 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7186,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/4265929298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4265929298},{"type":"v1","addr":"172.21.15.155:6833","nonce":4265929298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7189,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3254219000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3254219000},{"type":"v1","addr":"172.21.15.155:6835","nonce":3254219000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7195,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/193454084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":193454084},{"type":"v1","addr":"172.21.15.86:6837","nonce":193454084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:08:45.096500+0000","modified":"2024-04-25T14:08:48.121611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7371},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7371":{"gid":7371,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/1961103704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1961103704},{"type":"v1","addr":"172.21.15.86:6835","nonce":1961103704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7371,"qdb_cluster":[7371]},"id":6}]} 2024-04-25T14:09:13.003 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 67 2024-04-25T14:09:13.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.006+0000 7fbd94600640 1 -- 172.21.15.86:0/2705701801 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fbd6c06f4c0 msgr2=0x7fbd6c0719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:13.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.006+0000 7fbd94600640 1 --2- 172.21.15.86:0/2705701801 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fbd6c06f4c0 0x7fbd6c0719d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbd78005890 tx=0x7fbd78000fd0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.006+0000 7fbd94600640 1 -- 172.21.15.86:0/2705701801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd8c0b3640 msgr2=0x7fbd8c15a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.006+0000 7fbd94600640 1 --2- 172.21.15.86:0/2705701801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd8c0b3640 0x7fbd8c15a550 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000c460 tx=0x7fbd8000c920 comp rx=0 tx=0).stop 2024-04-25T14:09:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.006+0000 7fbd94600640 1 -- 172.21.15.86:0/2705701801 shutdown_connections 2024-04-25T14:09:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.006+0000 7fbd94600640 1 --2- 172.21.15.86:0/2705701801 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fbd6c06f4c0 0x7fbd6c0719d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.006+0000 7fbd94600640 1 --2- 172.21.15.86:0/2705701801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd8c09d1f0 0x7fbd8c09f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.006+0000 7fbd94600640 1 --2- 172.21.15.86:0/2705701801 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd8c0b3a50 0x7fbd8c09ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.006+0000 7fbd94600640 1 --2- 172.21.15.86:0/2705701801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd8c0b3640 0x7fbd8c15a550 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.006+0000 7fbd94600640 1 -- 172.21.15.86:0/2705701801 >> 172.21.15.86:0/2705701801 conn(0x7fbd8c09ba30 msgr2=0x7fbd8c0cf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.006+0000 7fbd94600640 1 -- 172.21.15.86:0/2705701801 shutdown_connections 2024-04-25T14:09:13.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.006+0000 7fbd94600640 1 -- 172.21.15.86:0/2705701801 wait complete. 2024-04-25T14:09:13.024 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:09:13.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee12200640 1 Processor -- start 2024-04-25T14:09:13.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee12200640 1 -- start start 2024-04-25T14:09:13.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee12200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee0c070330 0x7fee0c0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:13.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee12200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fee0c0ce790 0x7fee0c06f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:13.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee12200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee0c0d2030 con 0x7fee0c0d1c20 2024-04-25T14:09:13.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee12200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fee0c15e5c0 con 0x7fee0c0ce790 2024-04-25T14:09:13.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee12200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fee0c15e8a0 con 0x7fee0c070330 2024-04-25T14:09:13.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0aa00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fee0c0d1c20 0x7fee0c06ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42816/0 (socket says 172.21.15.86:42816) 2024-04-25T14:09:13.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0aa00640 1 -- 172.21.15.86:0/1460229682 learned_addr learned my addr 172.21.15.86:0/1460229682 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:13.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0be00640 1 --2- 172.21.15.86:0/1460229682 >> v2:172.21.15.155:3300/0 conn(0x7fee0c0ce790 0x7fee0c06f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:13.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0b400640 1 --2- 172.21.15.86:0/1460229682 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee0c070330 0x7fee0c0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:13.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0a000640 1 -- 172.21.15.86:0/1460229682 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2319535401 0 0) 0x7fedf8001120 con 0x7fee0c0d1c20 2024-04-25T14:09:13.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0a000640 1 -- 172.21.15.86:0/1460229682 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedf0003530 con 0x7fee0c0d1c20 2024-04-25T14:09:13.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0a000640 1 -- 172.21.15.86:0/1460229682 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2768449181 0 0) 0x7fedf0003530 con 0x7fee0c0d1c20 2024-04-25T14:09:13.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0a000640 1 -- 172.21.15.86:0/1460229682 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee0c070330 msgr2=0x7fee0c0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:13.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0a000640 1 --2- 172.21.15.86:0/1460229682 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee0c070330 0x7fee0c0d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0a000640 1 -- 172.21.15.86:0/1460229682 >> v2:172.21.15.155:3300/0 conn(0x7fee0c0ce790 msgr2=0x7fee0c06f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:13.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0a000640 1 --2- 172.21.15.86:0/1460229682 >> v2:172.21.15.155:3300/0 conn(0x7fee0c0ce790 0x7fee0c06f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0a000640 1 -- 172.21.15.86:0/1460229682 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0c15eb80 con 0x7fee0c0d1c20 2024-04-25T14:09:13.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0a000640 1 -- 172.21.15.86:0/1460229682 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fedf8001850 con 0x7fee0c0d1c20 2024-04-25T14:09:13.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0a000640 1 -- 172.21.15.86:0/1460229682 >> v1:172.21.15.86:6789/0 conn(0x7fee0c0d1c20 legacy=0x7fee0c06ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:13.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0a000640 1 --2- 172.21.15.86:0/1460229682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fedf0003a30 0x7fedf0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:13.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0a000640 1 --2- 172.21.15.86:0/1460229682 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee0c070330 0x7fedf0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:13.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0a000640 1 --2- 172.21.15.86:0/1460229682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf0004d50 0x7fedf0005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:13.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0a000640 1 -- 172.21.15.86:0/1460229682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c15e5c0 con 0x7fedf0004d50 2024-04-25T14:09:13.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0a000640 1 -- 172.21.15.86:0/1460229682 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c15e8a0 con 0x7fedf0003a30 2024-04-25T14:09:13.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0a000640 1 -- 172.21.15.86:0/1460229682 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fedf000bb00 con 0x7fee0c070330 2024-04-25T14:09:13.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0be00640 1 --2- 172.21.15.86:0/1460229682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf0004d50 0x7fedf0005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:13.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0b400640 1 --2- 172.21.15.86:0/1460229682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fedf0003a30 0x7fedf0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:13.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0aa00640 1 --2- 172.21.15.86:0/1460229682 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee0c070330 0x7fedf0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:13.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0be00640 1 -- 172.21.15.86:0/1460229682 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee0c070330 msgr2=0x7fedf0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:13.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0be00640 1 --2- 172.21.15.86:0/1460229682 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee0c070330 0x7fedf0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0be00640 1 -- 172.21.15.86:0/1460229682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fedf0003a30 msgr2=0x7fedf0003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:13.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0be00640 1 --2- 172.21.15.86:0/1460229682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fedf0003a30 0x7fedf0003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0be00640 1 -- 172.21.15.86:0/1460229682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf0003530 con 0x7fedf0004d50 2024-04-25T14:09:13.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.174+0000 7fee0aa00640 1 --2- 172.21.15.86:0/1460229682 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee0c070330 0x7fedf0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:13.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.178+0000 7fee0be00640 1 --2- 172.21.15.86:0/1460229682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf0004d50 0x7fedf0005160 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fee04009f90 tx=0x7fee0401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ecf4544389aa45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:13.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.178+0000 7fee0a000640 1 -- 172.21.15.86:0/1460229682 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee040b2070 con 0x7fedf0004d50 2024-04-25T14:09:13.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.178+0000 7fee0a000640 1 -- 172.21.15.86:0/1460229682 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee040a5c90 con 0x7fedf0004d50 2024-04-25T14:09:13.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.178+0000 7fee0a000640 1 -- 172.21.15.86:0/1460229682 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee04009c10 con 0x7fedf0004d50 2024-04-25T14:09:13.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.178+0000 7fee12200640 1 -- 172.21.15.86:0/1460229682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf0004d50 msgr2=0x7fedf0005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:13.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.178+0000 7fee12200640 1 --2- 172.21.15.86:0/1460229682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf0004d50 0x7fedf0005160 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fee04009f90 tx=0x7fee0401cc80 comp rx=0 tx=0).stop 2024-04-25T14:09:13.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.178+0000 7fee0be00640 1 -- 172.21.15.86:0/1460229682 reap_dead start 2024-04-25T14:09:13.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.178+0000 7fee12200640 1 -- 172.21.15.86:0/1460229682 shutdown_connections 2024-04-25T14:09:13.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.178+0000 7fee12200640 1 -- 172.21.15.86:0/1460229682 >> 172.21.15.86:0/1460229682 conn(0x7fee0c0b4650 msgr2=0x7fee0c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:13.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.178+0000 7fee12200640 1 -- 172.21.15.86:0/1460229682 shutdown_connections 2024-04-25T14:09:13.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.178+0000 7fee12200640 1 -- 172.21.15.86:0/1460229682 wait complete. 2024-04-25T14:09:13.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.182+0000 7fee12200640 1 Processor -- start 2024-04-25T14:09:13.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.182+0000 7fee12200640 1 -- start start 2024-04-25T14:09:13.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.182+0000 7fee12200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fedf0004d50 0x7fee0c0c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.182+0000 7fee12200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee0c1f5bc0 0x7fee0c0cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.182+0000 7fee12200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee0c0cffa0 0x7fee0c1f9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.182+0000 7fee12200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c15f330 con 0x7fee0c1f5bc0 2024-04-25T14:09:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.182+0000 7fee12200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c06d170 con 0x7fee0c0cffa0 2024-04-25T14:09:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.182+0000 7fee12200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fee0c15f4b0 con 0x7fedf0004d50 2024-04-25T14:09:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.186+0000 7fee0aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee0c1f5bc0 0x7fee0c0cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.186+0000 7fee0aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee0c1f5bc0 0x7fee0c0cfa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57488/0 (socket says 172.21.15.86:57488) 2024-04-25T14:09:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.186+0000 7fee0aa00640 1 -- 172.21.15.86:0/122672122 learned_addr learned my addr 172.21.15.86:0/122672122 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.186+0000 7fee0b400640 1 --2- 172.21.15.86:0/122672122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fedf0004d50 0x7fee0c0c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.186+0000 7fee0be00640 1 --2- 172.21.15.86:0/122672122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee0c0cffa0 0x7fee0c1f9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.186+0000 7fee0aa00640 1 -- 172.21.15.86:0/122672122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fedf0004d50 msgr2=0x7fee0c0c9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.186+0000 7fee0aa00640 1 --2- 172.21.15.86:0/122672122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fedf0004d50 0x7fee0c0c9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.186+0000 7fee0aa00640 1 -- 172.21.15.86:0/122672122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee0c0cffa0 msgr2=0x7fee0c1f9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.186+0000 7fee0aa00640 1 --2- 172.21.15.86:0/122672122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee0c0cffa0 0x7fee0c1f9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.186+0000 7fee0aa00640 1 -- 172.21.15.86:0/122672122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0c0d03b0 con 0x7fee0c1f5bc0 2024-04-25T14:09:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.186+0000 7fee0aa00640 1 --2- 172.21.15.86:0/122672122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee0c1f5bc0 0x7fee0c0cfa60 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fedf8011630 tx=0x7fedf8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.186+0000 7fee03400640 1 -- 172.21.15.86:0/122672122 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8010040 con 0x7fee0c1f5bc0 2024-04-25T14:09:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.186+0000 7fee03400640 1 -- 172.21.15.86:0/122672122 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedf800f800 con 0x7fee0c1f5bc0 2024-04-25T14:09:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.186+0000 7fee03400640 1 -- 172.21.15.86:0/122672122 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf800fbb0 con 0x7fee0c1f5bc0 2024-04-25T14:09:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.186+0000 7fee12200640 1 -- 172.21.15.86:0/122672122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee0c0ca120 con 0x7fee0c1f5bc0 2024-04-25T14:09:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.186+0000 7fee12200640 1 -- 172.21.15.86:0/122672122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee0c0ca8f0 con 0x7fee0c1f5bc0 2024-04-25T14:09:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.186+0000 7fee12200640 1 -- 172.21.15.86:0/122672122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee0c0ca360 con 0x7fee0c1f5bc0 2024-04-25T14:09:13.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.186+0000 7fee03400640 1 -- 172.21.15.86:0/122672122 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 36) ==== 93253+0+0 (secure 0 0 0) 0x7fedf8020470 con 0x7fee0c1f5bc0 2024-04-25T14:09:13.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.190+0000 7fee03400640 1 --2- 172.21.15.86:0/122672122 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fedec06f4c0 0x7fedec0719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.190+0000 7fee03400640 1 -- 172.21.15.86:0/122672122 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6746+0+0 (secure 0 0 0) 0x7fedf80927d0 con 0x7fee0c1f5bc0 2024-04-25T14:09:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.190+0000 7fee0b400640 1 --2- 172.21.15.86:0/122672122 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fedec06f4c0 0x7fedec0719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.190+0000 7fee0b400640 1 --2- 172.21.15.86:0/122672122 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fedec06f4c0 0x7fedec0719d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fee0c072200 tx=0x7fedfc009190 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.194+0000 7fee03400640 1 -- 172.21.15.86:0/122672122 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fee0c0d03b0 con 0x7fee0c1f5bc0 2024-04-25T14:09:13.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.350+0000 7fee12200640 1 -- 172.21.15.86:0/122672122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fee0c069d00 con 0x7fee0c1f5bc0 2024-04-25T14:09:13.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.350+0000 7fee03400640 1 -- 172.21.15.86:0/122672122 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+17823 (secure 0 0 0) 0x7fee0c0d2030 con 0x7fee0c1f5bc0 2024-04-25T14:09:13.348 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:13.348 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:09:09.480535+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:08:43.505988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T14:08:44.054628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2882289101":"2024-04-26T14:09:09.480502+0000","172.21.15.86:6833/132942210":"2024-04-26T14:09:09.480502+0000","172.21.15.86:6832/132942210":"2024-04-26T14:09:09.480502+0000","172.21.15.86:0/3672669933":"2024-04-26T14:09:09.480502+0000","172.21.15.86:0/402603388":"2024-04-26T14:09:09.480502+0000","172.21.15.86:0/3743735908":"2024-04-26T14:09:09.480502+0000","172.21.15.86:0/3132647616":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/4104483212":"2024-04-26T14:09:09.480502+0000","172.21.15.86:0/1564187566":"2024-04-26T14:09:03.346201+0000","172.21.15.86:6833/3589556609":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/1099362620":"2024-04-26T14:09:03.346201+0000","172.21.15.86:6832/3589556609":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/3481243210":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/2509210906":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/3722630918":"2024-04-26T14:09:03.346201+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:09:13.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.354+0000 7fee12200640 1 -- 172.21.15.86:0/122672122 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fedec06f4c0 msgr2=0x7fedec0719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:13.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.354+0000 7fee12200640 1 --2- 172.21.15.86:0/122672122 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fedec06f4c0 0x7fedec0719d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fee0c072200 tx=0x7fedfc009190 comp rx=0 tx=0).stop 2024-04-25T14:09:13.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.354+0000 7fee12200640 1 -- 172.21.15.86:0/122672122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee0c1f5bc0 msgr2=0x7fee0c0cfa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:13.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.354+0000 7fee12200640 1 --2- 172.21.15.86:0/122672122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee0c1f5bc0 0x7fee0c0cfa60 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fedf8011630 tx=0x7fedf8011a30 comp rx=0 tx=0).stop 2024-04-25T14:09:13.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.354+0000 7fee12200640 1 -- 172.21.15.86:0/122672122 shutdown_connections 2024-04-25T14:09:13.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.354+0000 7fee12200640 1 --2- 172.21.15.86:0/122672122 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fedec06f4c0 0x7fedec0719d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.354+0000 7fee12200640 1 --2- 172.21.15.86:0/122672122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee0c0cffa0 0x7fee0c1f9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.354+0000 7fee12200640 1 --2- 172.21.15.86:0/122672122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee0c1f5bc0 0x7fee0c0cfa60 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.354+0000 7fee12200640 1 --2- 172.21.15.86:0/122672122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fedf0004d50 0x7fee0c0c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.354+0000 7fee12200640 1 -- 172.21.15.86:0/122672122 >> 172.21.15.86:0/122672122 conn(0x7fee0c0b4650 msgr2=0x7fee0c15c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:13.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.354+0000 7fee12200640 1 -- 172.21.15.86:0/122672122 shutdown_connections 2024-04-25T14:09:13.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.354+0000 7fee12200640 1 -- 172.21.15.86:0/122672122 wait complete. 2024-04-25T14:09:13.370 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T14:09:13.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.522+0000 7f62eac00640 1 Processor -- start 2024-04-25T14:09:13.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.522+0000 7f62eac00640 1 -- start start 2024-04-25T14:09:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.526+0000 7f62eac00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f62e406fc40 0x7f62e4070050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.526+0000 7f62eac00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62e4070590 con 0x7f62e4072880 2024-04-25T14:09:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.526+0000 7f62eac00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62e415c2c0 con 0x7f62e406d430 2024-04-25T14:09:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.526+0000 7f62eac00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f62e406f7e0 con 0x7f62e406fc40 2024-04-25T14:09:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.526+0000 7f62e9800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f62e4072880 0x7f62e40af480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42824/0 (socket says 172.21.15.86:42824) 2024-04-25T14:09:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.526+0000 7f62e9800640 1 -- 172.21.15.86:0/1717630390 learned_addr learned my addr 172.21.15.86:0/1717630390 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.526+0000 7f62e8e00640 1 --2- 172.21.15.86:0/1717630390 >> v2:172.21.15.155:3300/0 conn(0x7f62e406fc40 0x7f62e4070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.526+0000 7f62e3e00640 1 -- 172.21.15.86:0/1717630390 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2506743744 0 0) 0x7f62d8001120 con 0x7f62e4072880 2024-04-25T14:09:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.526+0000 7f62e3e00640 1 -- 172.21.15.86:0/1717630390 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62cc003530 con 0x7f62e4072880 2024-04-25T14:09:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.526+0000 7f62e3e00640 1 -- 172.21.15.86:0/1717630390 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4129758168 0 0) 0x7f62e415c2c0 con 0x7f62e406d430 2024-04-25T14:09:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.526+0000 7f62e3e00640 1 -- 172.21.15.86:0/1717630390 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62d8001120 con 0x7f62e406d430 2024-04-25T14:09:13.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.526+0000 7f62e3e00640 1 -- 172.21.15.86:0/1717630390 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1310715840 0 0) 0x7f62cc003530 con 0x7f62e4072880 2024-04-25T14:09:13.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.526+0000 7f62e8e00640 1 -- 172.21.15.86:0/1717630390 >> v1:172.21.15.155:6789/0 conn(0x7f62e406d430 legacy=0x7f62e406d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:13.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.526+0000 7f62e8e00640 1 -- 172.21.15.86:0/1717630390 >> v1:172.21.15.86:6789/0 conn(0x7f62e4072880 legacy=0x7f62e40af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:13.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.526+0000 7f62e8e00640 1 -- 172.21.15.86:0/1717630390 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e406df50 con 0x7f62e406fc40 2024-04-25T14:09:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.530+0000 7f62e8e00640 1 --2- 172.21.15.86:0/1717630390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62e406fc40 0x7f62e4070050 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f62d000a960 tx=0x7f62d001c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=128f1b841872d4d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.530+0000 7f62e3e00640 1 -- 172.21.15.86:0/1717630390 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d8001850 con 0x7f62e406fc40 2024-04-25T14:09:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.530+0000 7f62e3e00640 1 -- 172.21.15.86:0/1717630390 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62d00a6430 con 0x7f62e406fc40 2024-04-25T14:09:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.530+0000 7f62e3e00640 1 -- 172.21.15.86:0/1717630390 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d000b040 con 0x7f62e406fc40 2024-04-25T14:09:13.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.530+0000 7f62eac00640 1 -- 172.21.15.86:0/1717630390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62e406fc40 msgr2=0x7f62e4070050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:13.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.530+0000 7f62eac00640 1 --2- 172.21.15.86:0/1717630390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62e406fc40 0x7f62e4070050 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f62d000a960 tx=0x7f62d001c660 comp rx=0 tx=0).stop 2024-04-25T14:09:13.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.530+0000 7f62eac00640 1 -- 172.21.15.86:0/1717630390 shutdown_connections 2024-04-25T14:09:13.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.530+0000 7f62eac00640 1 --2- 172.21.15.86:0/1717630390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62e406fc40 0x7f62e4070050 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.530+0000 7f62eac00640 1 -- 172.21.15.86:0/1717630390 >> 172.21.15.86:0/1717630390 conn(0x7f62e40b4740 msgr2=0x7f62e40b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:13.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.530+0000 7f62eac00640 1 -- 172.21.15.86:0/1717630390 shutdown_connections 2024-04-25T14:09:13.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.530+0000 7f62eac00640 1 -- 172.21.15.86:0/1717630390 wait complete. 2024-04-25T14:09:13.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.534+0000 7f62eac00640 1 Processor -- start 2024-04-25T14:09:13.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.534+0000 7f62eac00640 1 -- start start 2024-04-25T14:09:13.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.534+0000 7f62eac00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62e406d430 0x7f62e4166600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:13.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.534+0000 7f62eac00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62e406fc40 0x7f62e4150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:13.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.534+0000 7f62eac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62e4072880 0x7f62e4152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:13.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.534+0000 7f62eac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62e406cf10 con 0x7f62e4072880 2024-04-25T14:09:13.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.534+0000 7f62eac00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f62e406ccb0 con 0x7f62e406fc40 2024-04-25T14:09:13.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.534+0000 7f62eac00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f62e4072c90 con 0x7f62e406d430 2024-04-25T14:09:13.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.534+0000 7f62ea200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62e4072880 0x7f62e4152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:13.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.534+0000 7f62ea200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62e4072880 0x7f62e4152f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57498/0 (socket says 172.21.15.86:57498) 2024-04-25T14:09:13.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.534+0000 7f62ea200640 1 -- 172.21.15.86:0/3585129612 learned_addr learned my addr 172.21.15.86:0/3585129612 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:13.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.534+0000 7f62e8e00640 1 --2- 172.21.15.86:0/3585129612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62e406fc40 0x7f62e4150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:13.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.534+0000 7f62e9800640 1 --2- 172.21.15.86:0/3585129612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62e406d430 0x7f62e4166600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:13.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.534+0000 7f62ea200640 1 -- 172.21.15.86:0/3585129612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62e406d430 msgr2=0x7f62e4166600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:13.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.534+0000 7f62ea200640 1 --2- 172.21.15.86:0/3585129612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62e406d430 0x7f62e4166600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.534+0000 7f62ea200640 1 -- 172.21.15.86:0/3585129612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62e406fc40 msgr2=0x7f62e4150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:13.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.534+0000 7f62ea200640 1 --2- 172.21.15.86:0/3585129612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62e406fc40 0x7f62e4150a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.534+0000 7f62ea200640 1 -- 172.21.15.86:0/3585129612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e41658a0 con 0x7f62e4072880 2024-04-25T14:09:13.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.534+0000 7f62e9800640 1 --2- 172.21.15.86:0/3585129612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62e406d430 0x7f62e4166600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:13.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.534+0000 7f62ea200640 1 --2- 172.21.15.86:0/3585129612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62e4072880 0x7f62e4152f60 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00c730 tx=0x7f62dc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:13.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.534+0000 7f62e1600640 1 -- 172.21.15.86:0/3585129612 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc003b80 con 0x7f62e4072880 2024-04-25T14:09:13.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.534+0000 7f62e1600640 1 -- 172.21.15.86:0/3585129612 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62dc003d20 con 0x7f62e4072880 2024-04-25T14:09:13.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.534+0000 7f62eac00640 1 -- 172.21.15.86:0/3585129612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e4167820 con 0x7f62e4072880 2024-04-25T14:09:13.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.534+0000 7f62e1600640 1 -- 172.21.15.86:0/3585129612 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc00aa90 con 0x7f62e4072880 2024-04-25T14:09:13.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.534+0000 7f62eac00640 1 -- 172.21.15.86:0/3585129612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e4165e00 con 0x7f62e4072880 2024-04-25T14:09:13.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.538+0000 7f62eac00640 1 -- 172.21.15.86:0/3585129612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62e4076b80 con 0x7f62e4072880 2024-04-25T14:09:13.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.538+0000 7f62e1600640 1 -- 172.21.15.86:0/3585129612 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7f62dc00ac30 con 0x7f62e4072880 2024-04-25T14:09:13.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.538+0000 7f62e1600640 1 --2- 172.21.15.86:0/3585129612 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f62c406f550 0x7f62c4071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:13.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.538+0000 7f62e1600640 1 -- 172.21.15.86:0/3585129612 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6746+0+0 (secure 0 0 0) 0x7f62dc08f750 con 0x7f62e4072880 2024-04-25T14:09:13.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.538+0000 7f62e9800640 1 --2- 172.21.15.86:0/3585129612 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f62c406f550 0x7f62c4071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:13.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.538+0000 7f62e9800640 1 --2- 172.21.15.86:0/3585129612 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f62c406f550 0x7f62c4071a60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f62d8001ec0 tx=0x7f62d8009040 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:13.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.542+0000 7f62e1600640 1 -- 172.21.15.86:0/3585129612 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f62e41658a0 con 0x7f62e4072880 2024-04-25T14:09:13.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.734+0000 7f62eac00640 1 -- 172.21.15.86:0/3585129612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f62e406f2c0 con 0x7f62e4072880 2024-04-25T14:09:13.732 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-25T14:09:13.734+0000 7fea29c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:09:13.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.738+0000 7f62e1600640 1 -- 172.21.15.86:0/3585129612 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v68) ==== 161+0+0 (secure 0 0 0) 0x7f62dc05c110 con 0x7f62e4072880 2024-04-25T14:09:13.734 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T14:09:13.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.738+0000 7f62eac00640 1 -- 172.21.15.86:0/3585129612 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f62c406f550 msgr2=0x7f62c4071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:13.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.738+0000 7f62eac00640 1 --2- 172.21.15.86:0/3585129612 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f62c406f550 0x7f62c4071a60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f62d8001ec0 tx=0x7f62d8009040 comp rx=0 tx=0).stop 2024-04-25T14:09:13.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.742+0000 7f62eac00640 1 -- 172.21.15.86:0/3585129612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62e4072880 msgr2=0x7f62e4152f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:13.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.742+0000 7f62eac00640 1 --2- 172.21.15.86:0/3585129612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62e4072880 0x7f62e4152f60 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00c730 tx=0x7f62dc00cbf0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.742+0000 7f62eac00640 1 -- 172.21.15.86:0/3585129612 shutdown_connections 2024-04-25T14:09:13.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.742+0000 7f62eac00640 1 --2- 172.21.15.86:0/3585129612 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f62c406f550 0x7f62c4071a60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.742+0000 7f62eac00640 1 --2- 172.21.15.86:0/3585129612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62e4072880 0x7f62e4152f60 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.742+0000 7f62eac00640 1 --2- 172.21.15.86:0/3585129612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62e406fc40 0x7f62e4150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.742+0000 7f62eac00640 1 --2- 172.21.15.86:0/3585129612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62e406d430 0x7f62e4166600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.742+0000 7f62eac00640 1 -- 172.21.15.86:0/3585129612 >> 172.21.15.86:0/3585129612 conn(0x7f62e40b4740 msgr2=0x7f62e406d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:13.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.742+0000 7f62eac00640 1 -- 172.21.15.86:0/3585129612 shutdown_connections 2024-04-25T14:09:13.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.742+0000 7f62eac00640 1 -- 172.21.15.86:0/3585129612 wait complete. 2024-04-25T14:09:13.745 INFO:tasks.ceph.mds.a.smithi086.stderr: -3556> 2024-04-25T14:08:47.682+0000 7fb6ff000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T14:09:13.760 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T14:09:13.861 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-25T14:09:13.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.914+0000 7fe7dc000640 1 Processor -- start 2024-04-25T14:09:13.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.914+0000 7fe7dc000640 1 -- start start 2024-04-25T14:09:13.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.918+0000 7fe7dc000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7d40b59e0 0x7fe7d40b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:13.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.918+0000 7fe7dc000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe7d40aeea0 0x7fe7d40af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:13.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.918+0000 7fe7dc000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7d40b6330 con 0x7fe7d415faa0 2024-04-25T14:09:13.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.918+0000 7fe7dc000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe7d406d560 con 0x7fe7d40aeea0 2024-04-25T14:09:13.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.918+0000 7fe7dc000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe7d406c670 con 0x7fe7d40b59e0 2024-04-25T14:09:13.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.918+0000 7fe7d9600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe7d415faa0 0x7fe7d406d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42840/0 (socket says 172.21.15.86:42840) 2024-04-25T14:09:13.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.918+0000 7fe7d9600640 1 -- 172.21.15.86:0/328106510 learned_addr learned my addr 172.21.15.86:0/328106510 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:13.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.918+0000 7fe7d8c00640 1 --2- 172.21.15.86:0/328106510 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7d40b59e0 0x7fe7d40b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:13.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.918+0000 7fe7da000640 1 --2- 172.21.15.86:0/328106510 >> v2:172.21.15.155:3300/0 conn(0x7fe7d40aeea0 0x7fe7d40af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:13.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.918+0000 7fe7d3e00640 1 -- 172.21.15.86:0/328106510 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 107567397 0 0) 0x7fe7c8001120 con 0x7fe7d415faa0 2024-04-25T14:09:13.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.918+0000 7fe7d3e00640 1 -- 172.21.15.86:0/328106510 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7bc003530 con 0x7fe7d415faa0 2024-04-25T14:09:13.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.918+0000 7fe7d8c00640 1 -- 172.21.15.86:0/328106510 >> v2:172.21.15.155:3300/0 conn(0x7fe7d40aeea0 msgr2=0x7fe7d40af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:13.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.918+0000 7fe7d8c00640 1 --2- 172.21.15.86:0/328106510 >> v2:172.21.15.155:3300/0 conn(0x7fe7d40aeea0 0x7fe7d40af2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.918+0000 7fe7d8c00640 1 -- 172.21.15.86:0/328106510 >> v1:172.21.15.86:6789/0 conn(0x7fe7d415faa0 legacy=0x7fe7d406d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:13.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.918+0000 7fe7d8c00640 1 -- 172.21.15.86:0/328106510 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d406c950 con 0x7fe7d40b59e0 2024-04-25T14:09:13.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.922+0000 7fe7d8c00640 1 --2- 172.21.15.86:0/328106510 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7d40b59e0 0x7fe7d40b5df0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe7c00097c0 tx=0x7fe7c001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdba76e042bf14cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:13.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.922+0000 7fe7d3e00640 1 -- 172.21.15.86:0/328106510 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c00aa620 con 0x7fe7d40b59e0 2024-04-25T14:09:13.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.922+0000 7fe7d3e00640 1 -- 172.21.15.86:0/328106510 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7c00af070 con 0x7fe7d40b59e0 2024-04-25T14:09:13.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.922+0000 7fe7d3e00640 1 -- 172.21.15.86:0/328106510 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c00aa9e0 con 0x7fe7d40b59e0 2024-04-25T14:09:13.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.922+0000 7fe7dc000640 1 -- 172.21.15.86:0/328106510 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7d40b59e0 msgr2=0x7fe7d40b5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:13.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.922+0000 7fe7dc000640 1 --2- 172.21.15.86:0/328106510 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7d40b59e0 0x7fe7d40b5df0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe7c00097c0 tx=0x7fe7c001c4a0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.922+0000 7fe7dc000640 1 -- 172.21.15.86:0/328106510 shutdown_connections 2024-04-25T14:09:13.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.922+0000 7fe7dc000640 1 --2- 172.21.15.86:0/328106510 >> v2:172.21.15.155:3300/0 conn(0x7fe7d40aeea0 0x7fe7d40af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.922+0000 7fe7dc000640 1 --2- 172.21.15.86:0/328106510 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7d40b59e0 0x7fe7d40b5df0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.922+0000 7fe7dc000640 1 -- 172.21.15.86:0/328106510 >> 172.21.15.86:0/328106510 conn(0x7fe7d40d2230 msgr2=0x7fe7d40d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:13.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.922+0000 7fe7dc000640 1 -- 172.21.15.86:0/328106510 shutdown_connections 2024-04-25T14:09:13.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.922+0000 7fe7dc000640 1 -- 172.21.15.86:0/328106510 wait complete. 2024-04-25T14:09:13.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.926+0000 7fe7dc000640 1 Processor -- start 2024-04-25T14:09:13.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.926+0000 7fe7dc000640 1 -- start start 2024-04-25T14:09:13.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.926+0000 7fe7dc000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7d40aeea0 0x7fe7d414b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:13.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.926+0000 7fe7dc000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7d40b59e0 0x7fe7d4159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:13.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.926+0000 7fe7dc000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7d415faa0 0x7fe7d40b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:13.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.926+0000 7fe7dc000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d4160030 con 0x7fe7d40aeea0 2024-04-25T14:09:13.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.926+0000 7fe7dc000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d415feb0 con 0x7fe7d40b59e0 2024-04-25T14:09:13.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.926+0000 7fe7dc000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe7d4070f30 con 0x7fe7d415faa0 2024-04-25T14:09:13.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.926+0000 7fe7da000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7d415faa0 0x7fe7d40b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:13.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.926+0000 7fe7d9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7d40aeea0 0x7fe7d414b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:13.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.926+0000 7fe7da000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7d415faa0 0x7fe7d40b0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:37604/0 (socket says 172.21.15.86:37604) 2024-04-25T14:09:13.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.926+0000 7fe7da000640 1 -- 172.21.15.86:0/912275289 learned_addr learned my addr 172.21.15.86:0/912275289 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:13.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.926+0000 7fe7d9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7d40aeea0 0x7fe7d414b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57524/0 (socket says 172.21.15.86:57524) 2024-04-25T14:09:13.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.926+0000 7fe7d8c00640 1 --2- 172.21.15.86:0/912275289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7d40b59e0 0x7fe7d4159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:13.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.930+0000 7fe7d9600640 1 -- 172.21.15.86:0/912275289 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7d415faa0 msgr2=0x7fe7d40b0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:13.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.930+0000 7fe7d9600640 1 --2- 172.21.15.86:0/912275289 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7d415faa0 0x7fe7d40b0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.930+0000 7fe7d9600640 1 -- 172.21.15.86:0/912275289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7d40b59e0 msgr2=0x7fe7d4159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:13.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.930+0000 7fe7d9600640 1 --2- 172.21.15.86:0/912275289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7d40b59e0 0x7fe7d4159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:13.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.930+0000 7fe7d9600640 1 -- 172.21.15.86:0/912275289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d414c590 con 0x7fe7d40aeea0 2024-04-25T14:09:13.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.930+0000 7fe7da000640 1 --2- 172.21.15.86:0/912275289 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7d415faa0 0x7fe7d40b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:09:13.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.930+0000 7fe7d9600640 1 --2- 172.21.15.86:0/912275289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7d40aeea0 0x7fe7d414b200 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800c3c0 tx=0x7fe7c800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:13.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.930+0000 7fe7d1600640 1 -- 172.21.15.86:0/912275289 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8017070 con 0x7fe7d40aeea0 2024-04-25T14:09:13.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.930+0000 7fe7d1600640 1 -- 172.21.15.86:0/912275289 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7c8005cf0 con 0x7fe7d40aeea0 2024-04-25T14:09:13.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.930+0000 7fe7d1600640 1 -- 172.21.15.86:0/912275289 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c80072f0 con 0x7fe7d40aeea0 2024-04-25T14:09:13.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.930+0000 7fe7dc000640 1 -- 172.21.15.86:0/912275289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d414c870 con 0x7fe7d40aeea0 2024-04-25T14:09:13.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.930+0000 7fe7dc000640 1 -- 172.21.15.86:0/912275289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d414ce30 con 0x7fe7d40aeea0 2024-04-25T14:09:13.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.930+0000 7fe7dc000640 1 -- 172.21.15.86:0/912275289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7d4074cf0 con 0x7fe7d40aeea0 2024-04-25T14:09:13.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.934+0000 7fe7d1600640 1 -- 172.21.15.86:0/912275289 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7fe7c801a070 con 0x7fe7d40aeea0 2024-04-25T14:09:13.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.934+0000 7fe7d1600640 1 --2- 172.21.15.86:0/912275289 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fe7b406f520 0x7fe7b4071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:13.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.934+0000 7fe7d1600640 1 -- 172.21.15.86:0/912275289 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6832+0+0 (secure 0 0 0) 0x7fe7c808eae0 con 0x7fe7d40aeea0 2024-04-25T14:09:13.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.934+0000 7fe7d8c00640 1 --2- 172.21.15.86:0/912275289 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fe7b406f520 0x7fe7b4071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:13.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.934+0000 7fe7d8c00640 1 --2- 172.21.15.86:0/912275289 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fe7b406f520 0x7fe7b4071a30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe7c00ae000 tx=0x7fe7c00a7000 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:13.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:13.938+0000 7fe7d1600640 1 -- 172.21.15.86:0/912275289 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe7d40b6330 con 0x7fe7d40aeea0 2024-04-25T14:09:14.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.130+0000 7fe7dc000640 1 -- 172.21.15.86:0/912275289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe7d415f120 con 0x7fe7d40aeea0 2024-04-25T14:09:14.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.738+0000 7fe7d1600640 1 -- 172.21.15.86:0/912275289 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v69) ==== 104+0+0 (secure 0 0 0) 0x7fe7c805ae10 con 0x7fe7d40aeea0 2024-04-25T14:09:14.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.742+0000 7fe7dc000640 1 -- 172.21.15.86:0/912275289 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fe7b406f520 msgr2=0x7fe7b4071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:14.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.742+0000 7fe7dc000640 1 --2- 172.21.15.86:0/912275289 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fe7b406f520 0x7fe7b4071a30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe7c00ae000 tx=0x7fe7c00a7000 comp rx=0 tx=0).stop 2024-04-25T14:09:14.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.742+0000 7fe7dc000640 1 -- 172.21.15.86:0/912275289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7d40aeea0 msgr2=0x7fe7d414b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:14.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.742+0000 7fe7dc000640 1 --2- 172.21.15.86:0/912275289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7d40aeea0 0x7fe7d414b200 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800c3c0 tx=0x7fe7c800c880 comp rx=0 tx=0).stop 2024-04-25T14:09:14.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.742+0000 7fe7dc000640 1 -- 172.21.15.86:0/912275289 shutdown_connections 2024-04-25T14:09:14.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.742+0000 7fe7dc000640 1 --2- 172.21.15.86:0/912275289 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fe7b406f520 0x7fe7b4071a30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:14.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.742+0000 7fe7dc000640 1 --2- 172.21.15.86:0/912275289 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7d415faa0 0x7fe7d40b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:14.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.742+0000 7fe7dc000640 1 --2- 172.21.15.86:0/912275289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7d40b59e0 0x7fe7d4159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:14.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.742+0000 7fe7dc000640 1 --2- 172.21.15.86:0/912275289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7d40aeea0 0x7fe7d414b200 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:14.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.742+0000 7fe7dc000640 1 -- 172.21.15.86:0/912275289 >> 172.21.15.86:0/912275289 conn(0x7fe7d40d2230 msgr2=0x7fe7d4163470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:14.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.742+0000 7fe7dc000640 1 -- 172.21.15.86:0/912275289 shutdown_connections 2024-04-25T14:09:14.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.742+0000 7fe7dc000640 1 -- 172.21.15.86:0/912275289 wait complete. 2024-04-25T14:09:14.761 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T14:09:14.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.914+0000 7fed04a00640 1 Processor -- start 2024-04-25T14:09:14.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.914+0000 7fed04a00640 1 -- start start 2024-04-25T14:09:14.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.918+0000 7fed04a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fed000701f0 0x7fed000729f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:14.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.918+0000 7fed04a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fed0006cd80 0x7fed00072f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:14.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.918+0000 7fed04a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed0006f430 con 0x7fed00070600 2024-04-25T14:09:14.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.918+0000 7fed04a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fed0006f1e0 con 0x7fed0006cd80 2024-04-25T14:09:14.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.918+0000 7fed04a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fed0006f710 con 0x7fed000701f0 2024-04-25T14:09:14.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.918+0000 7fecf5400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fed00070600 0x7fed0006c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42856/0 (socket says 172.21.15.86:42856) 2024-04-25T14:09:14.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.918+0000 7fecf5400640 1 -- 172.21.15.86:0/3062715099 learned_addr learned my addr 172.21.15.86:0/3062715099 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:14.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.918+0000 7fecfe800640 1 --2- 172.21.15.86:0/3062715099 >> v2:172.21.15.86:3300/0 conn(0x7fed0006cd80 0x7fed00072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:14.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.918+0000 7fecfde00640 1 --2- 172.21.15.86:0/3062715099 >> v2:172.21.15.155:3300/0 conn(0x7fed000701f0 0x7fed000729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:14.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.918+0000 7fecfd400640 1 -- 172.21.15.86:0/3062715099 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 527419650 0 0) 0x7fed0006f430 con 0x7fed00070600 2024-04-25T14:09:14.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.918+0000 7fecfd400640 1 -- 172.21.15.86:0/3062715099 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fece4003530 con 0x7fed00070600 2024-04-25T14:09:14.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.918+0000 7fecfe800640 1 -- 172.21.15.86:0/3062715099 >> v2:172.21.15.155:3300/0 conn(0x7fed000701f0 msgr2=0x7fed000729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:14.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.918+0000 7fecfe800640 1 --2- 172.21.15.86:0/3062715099 >> v2:172.21.15.155:3300/0 conn(0x7fed000701f0 0x7fed000729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:14.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.918+0000 7fecfe800640 1 -- 172.21.15.86:0/3062715099 >> v1:172.21.15.86:6789/0 conn(0x7fed00070600 legacy=0x7fed0006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:14.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.918+0000 7fecfe800640 1 -- 172.21.15.86:0/3062715099 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0006f9f0 con 0x7fed0006cd80 2024-04-25T14:09:14.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.922+0000 7fecfe800640 1 --2- 172.21.15.86:0/3062715099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed0006cd80 0x7fed00072f30 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fed0006f360 tx=0x7fecf801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48524d0d05393d8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:14.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.922+0000 7fecfd400640 1 -- 172.21.15.86:0/3062715099 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf80b2070 con 0x7fed0006cd80 2024-04-25T14:09:14.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.922+0000 7fecfd400640 1 -- 172.21.15.86:0/3062715099 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecf80a5760 con 0x7fed0006cd80 2024-04-25T14:09:14.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.922+0000 7fecfd400640 1 -- 172.21.15.86:0/3062715099 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf80a5b30 con 0x7fed0006cd80 2024-04-25T14:09:14.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.922+0000 7fed04a00640 1 -- 172.21.15.86:0/3062715099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed0006cd80 msgr2=0x7fed00072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:14.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.922+0000 7fed04a00640 1 --2- 172.21.15.86:0/3062715099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed0006cd80 0x7fed00072f30 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fed0006f360 tx=0x7fecf801c8a0 comp rx=0 tx=0).stop 2024-04-25T14:09:14.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.922+0000 7fed04a00640 1 -- 172.21.15.86:0/3062715099 shutdown_connections 2024-04-25T14:09:14.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.922+0000 7fed04a00640 1 --2- 172.21.15.86:0/3062715099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed0006cd80 0x7fed00072f30 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:14.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.922+0000 7fed04a00640 1 --2- 172.21.15.86:0/3062715099 >> v2:172.21.15.155:3300/0 conn(0x7fed000701f0 0x7fed000729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:14.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.922+0000 7fed04a00640 1 -- 172.21.15.86:0/3062715099 >> 172.21.15.86:0/3062715099 conn(0x7fed000b4a40 msgr2=0x7fed000b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:14.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.922+0000 7fed04a00640 1 -- 172.21.15.86:0/3062715099 shutdown_connections 2024-04-25T14:09:14.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.922+0000 7fed04a00640 1 -- 172.21.15.86:0/3062715099 wait complete. 2024-04-25T14:09:14.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.922+0000 7fed04a00640 1 Processor -- start 2024-04-25T14:09:14.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.922+0000 7fed04a00640 1 -- start start 2024-04-25T14:09:14.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.926+0000 7fed04a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed0006cd80 0x7fed0015f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:14.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.926+0000 7fed04a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed000701f0 0x7fed0015f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:14.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.926+0000 7fed04a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed00070600 0x7fed0015fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:14.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.926+0000 7fed04a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed000b36d0 con 0x7fed000701f0 2024-04-25T14:09:14.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.926+0000 7fed04a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fed0015d580 con 0x7fed00070600 2024-04-25T14:09:14.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.926+0000 7fed04a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fed000b3850 con 0x7fed0006cd80 2024-04-25T14:09:14.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.926+0000 7fecf5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed000701f0 0x7fed0015f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.926+0000 7fecf5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed000701f0 0x7fed0015f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57548/0 (socket says 172.21.15.86:57548) 2024-04-25T14:09:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.926+0000 7fecf5400640 1 -- 172.21.15.86:0/2596810000 learned_addr learned my addr 172.21.15.86:0/2596810000 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.926+0000 7fecfe800640 1 --2- 172.21.15.86:0/2596810000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed00070600 0x7fed0015fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.926+0000 7fecfde00640 1 --2- 172.21.15.86:0/2596810000 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed0006cd80 0x7fed0015f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.926+0000 7fecf5400640 1 -- 172.21.15.86:0/2596810000 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed0006cd80 msgr2=0x7fed0015f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.926+0000 7fecf5400640 1 --2- 172.21.15.86:0/2596810000 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed0006cd80 0x7fed0015f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.926+0000 7fecf5400640 1 -- 172.21.15.86:0/2596810000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed00070600 msgr2=0x7fed0015fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.926+0000 7fecf5400640 1 --2- 172.21.15.86:0/2596810000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed00070600 0x7fed0015fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.926+0000 7fecf5400640 1 -- 172.21.15.86:0/2596810000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed00150410 con 0x7fed000701f0 2024-04-25T14:09:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.926+0000 7fecfe800640 1 --2- 172.21.15.86:0/2596810000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed00070600 0x7fed0015fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.926+0000 7fecfde00640 1 --2- 172.21.15.86:0/2596810000 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed0006cd80 0x7fed0015f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.926+0000 7fecf5400640 1 --2- 172.21.15.86:0/2596810000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed000701f0 0x7fed0015f940 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fece800c3c0 tx=0x7fece800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.926+0000 7fecf6a00640 1 -- 172.21.15.86:0/2596810000 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece8017070 con 0x7fed000701f0 2024-04-25T14:09:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.926+0000 7fecf6a00640 1 -- 172.21.15.86:0/2596810000 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fece8005cf0 con 0x7fed000701f0 2024-04-25T14:09:14.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.926+0000 7fed04a00640 1 -- 172.21.15.86:0/2596810000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed00160730 con 0x7fed000701f0 2024-04-25T14:09:14.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.926+0000 7fecf6a00640 1 -- 172.21.15.86:0/2596810000 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece80072f0 con 0x7fed000701f0 2024-04-25T14:09:14.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.926+0000 7fed04a00640 1 -- 172.21.15.86:0/2596810000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed00150680 con 0x7fed000701f0 2024-04-25T14:09:14.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.926+0000 7fed04a00640 1 -- 172.21.15.86:0/2596810000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed00050750 con 0x7fed000701f0 2024-04-25T14:09:14.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.930+0000 7fecf6a00640 1 -- 172.21.15.86:0/2596810000 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7fece801a070 con 0x7fed000701f0 2024-04-25T14:09:14.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.930+0000 7fecf6a00640 1 --2- 172.21.15.86:0/2596810000 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fecdc06f550 0x7fecdc071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:14.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.930+0000 7fecf6a00640 1 -- 172.21.15.86:0/2596810000 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6832+0+0 (secure 0 0 0) 0x7fece808ec30 con 0x7fed000701f0 2024-04-25T14:09:14.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.930+0000 7fecfde00640 1 --2- 172.21.15.86:0/2596810000 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fecdc06f550 0x7fecdc071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:14.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.930+0000 7fecfde00640 1 --2- 172.21.15.86:0/2596810000 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fecdc06f550 0x7fecdc071a60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fecf00097a0 tx=0x7fecf0008040 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:14.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:14.934+0000 7fecf6a00640 1 -- 172.21.15.86:0/2596810000 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fece805b5b0 con 0x7fed000701f0 2024-04-25T14:09:15.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.090+0000 7fed04a00640 1 -- 172.21.15.86:0/2596810000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fed00075eb0 con 0x7fed000701f0 2024-04-25T14:09:15.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.750+0000 7fecf6a00640 1 -- 172.21.15.86:0/2596810000 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) ==== 181+0+0 (secure 0 0 0) 0x7fece805aeb0 con 0x7fed000701f0 2024-04-25T14:09:15.746 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-04-25T14:09:15.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.754+0000 7fed04a00640 1 -- 172.21.15.86:0/2596810000 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fecdc06f550 msgr2=0x7fecdc071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:15.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.754+0000 7fed04a00640 1 --2- 172.21.15.86:0/2596810000 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fecdc06f550 0x7fecdc071a60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fecf00097a0 tx=0x7fecf0008040 comp rx=0 tx=0).stop 2024-04-25T14:09:15.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.754+0000 7fed04a00640 1 -- 172.21.15.86:0/2596810000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed000701f0 msgr2=0x7fed0015f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:15.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.754+0000 7fed04a00640 1 --2- 172.21.15.86:0/2596810000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed000701f0 0x7fed0015f940 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fece800c3c0 tx=0x7fece800c880 comp rx=0 tx=0).stop 2024-04-25T14:09:15.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.754+0000 7fed04a00640 1 -- 172.21.15.86:0/2596810000 shutdown_connections 2024-04-25T14:09:15.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.754+0000 7fed04a00640 1 --2- 172.21.15.86:0/2596810000 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fecdc06f550 0x7fecdc071a60 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:15.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.754+0000 7fed04a00640 1 --2- 172.21.15.86:0/2596810000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed00070600 0x7fed0015fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:15.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.754+0000 7fed04a00640 1 --2- 172.21.15.86:0/2596810000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed000701f0 0x7fed0015f940 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:15.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.754+0000 7fed04a00640 1 --2- 172.21.15.86:0/2596810000 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed0006cd80 0x7fed0015f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:15.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.754+0000 7fed04a00640 1 -- 172.21.15.86:0/2596810000 >> 172.21.15.86:0/2596810000 conn(0x7fed000b4a40 msgr2=0x7fed000aeb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:15.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.754+0000 7fed04a00640 1 -- 172.21.15.86:0/2596810000 shutdown_connections 2024-04-25T14:09:15.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.754+0000 7fed04a00640 1 -- 172.21.15.86:0/2596810000 wait complete. 2024-04-25T14:09:15.775 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T14:09:15.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.930+0000 7f8934600640 1 Processor -- start 2024-04-25T14:09:15.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.930+0000 7f8934600640 1 -- start start 2024-04-25T14:09:15.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.930+0000 7f8934600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f892c0743c0 0x7f892c1605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:15.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.930+0000 7f8934600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f892c15ce60 con 0x7f892c15ca50 2024-04-25T14:09:15.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.930+0000 7f8934600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f892c072b90 con 0x7f892c0747d0 2024-04-25T14:09:15.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.930+0000 7f8934600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f892c072e70 con 0x7f892c0743c0 2024-04-25T14:09:15.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.930+0000 7f8931200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f892c0743c0 0x7f892c1605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:15.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.930+0000 7f8931c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f892c15ca50 0x7f892c073950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42858/0 (socket says 172.21.15.86:42858) 2024-04-25T14:09:15.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.930+0000 7f8931200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f892c0743c0 0x7f892c1605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57552/0 (socket says 172.21.15.86:57552) 2024-04-25T14:09:15.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.930+0000 7f8931c00640 1 -- 172.21.15.86:0/58454450 learned_addr learned my addr 172.21.15.86:0/58454450 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:15.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.930+0000 7f8931200640 1 -- 172.21.15.86:0/58454450 >> v1:172.21.15.155:6789/0 conn(0x7f892c0747d0 legacy=0x7f892c06e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:15.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.930+0000 7f8931200640 1 -- 172.21.15.86:0/58454450 >> v1:172.21.15.86:6789/0 conn(0x7f892c15ca50 legacy=0x7f892c073950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:15.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.930+0000 7f892be00640 1 -- 172.21.15.86:0/58454450 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3353390520 0 0) 0x7f892c15ce60 con 0x7f892c15ca50 2024-04-25T14:09:15.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.930+0000 7f8931200640 1 -- 172.21.15.86:0/58454450 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f892c073150 con 0x7f892c0743c0 2024-04-25T14:09:15.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.934+0000 7f8931200640 1 --2- 172.21.15.86:0/58454450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f892c0743c0 0x7f892c1605f0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f89180097c0 tx=0x7f891801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6710b0a955c3ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:15.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.934+0000 7f892be00640 1 -- 172.21.15.86:0/58454450 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89180b0070 con 0x7f892c0743c0 2024-04-25T14:09:15.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.934+0000 7f892be00640 1 -- 172.21.15.86:0/58454450 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89180ab600 con 0x7f892c0743c0 2024-04-25T14:09:15.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.934+0000 7f892be00640 1 -- 172.21.15.86:0/58454450 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89180ab9d0 con 0x7f892c0743c0 2024-04-25T14:09:15.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.934+0000 7f8934600640 1 -- 172.21.15.86:0/58454450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f892c0743c0 msgr2=0x7f892c1605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:15.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.934+0000 7f8934600640 1 --2- 172.21.15.86:0/58454450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f892c0743c0 0x7f892c1605f0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f89180097c0 tx=0x7f891801c4a0 comp rx=0 tx=0).stop 2024-04-25T14:09:15.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.934+0000 7f8934600640 1 -- 172.21.15.86:0/58454450 shutdown_connections 2024-04-25T14:09:15.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.934+0000 7f8934600640 1 --2- 172.21.15.86:0/58454450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f892c0743c0 0x7f892c1605f0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:15.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.934+0000 7f8934600640 1 -- 172.21.15.86:0/58454450 >> 172.21.15.86:0/58454450 conn(0x7f892c0b4a40 msgr2=0x7f892c0b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:15.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.934+0000 7f8934600640 1 -- 172.21.15.86:0/58454450 shutdown_connections 2024-04-25T14:09:15.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.934+0000 7f8934600640 1 -- 172.21.15.86:0/58454450 wait complete. 2024-04-25T14:09:15.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.938+0000 7f8934600640 1 Processor -- start 2024-04-25T14:09:15.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.938+0000 7f8934600640 1 -- start start 2024-04-25T14:09:15.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.938+0000 7f8934600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f892c0743c0 0x7f892c09ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:15.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.938+0000 7f8934600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f892c0747d0 0x7f892c153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:15.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.938+0000 7f8934600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f892c15ca50 0x7f892c1541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:15.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.938+0000 7f8934600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f892c16b540 con 0x7f892c0743c0 2024-04-25T14:09:15.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.938+0000 7f8934600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f892c16b3c0 con 0x7f892c0747d0 2024-04-25T14:09:15.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.938+0000 7f8934600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f892c16b6c0 con 0x7f892c15ca50 2024-04-25T14:09:15.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.938+0000 7f8931c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f892c0743c0 0x7f892c09ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:15.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.938+0000 7f8931c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f892c0743c0 0x7f892c09ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57560/0 (socket says 172.21.15.86:57560) 2024-04-25T14:09:15.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.938+0000 7f8931c00640 1 -- 172.21.15.86:0/31724381 learned_addr learned my addr 172.21.15.86:0/31724381 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:15.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.938+0000 7f8931200640 1 --2- 172.21.15.86:0/31724381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f892c0747d0 0x7f892c153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:15.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.938+0000 7f8932600640 1 --2- 172.21.15.86:0/31724381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f892c15ca50 0x7f892c1541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:15.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.938+0000 7f8931c00640 1 -- 172.21.15.86:0/31724381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f892c15ca50 msgr2=0x7f892c1541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:15.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.938+0000 7f8931c00640 1 --2- 172.21.15.86:0/31724381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f892c15ca50 0x7f892c1541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:15.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.938+0000 7f8931c00640 1 -- 172.21.15.86:0/31724381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f892c0747d0 msgr2=0x7f892c153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:15.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.938+0000 7f8931c00640 1 --2- 172.21.15.86:0/31724381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f892c0747d0 0x7f892c153ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:15.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.938+0000 7f8931c00640 1 -- 172.21.15.86:0/31724381 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f892c09eaf0 con 0x7f892c0743c0 2024-04-25T14:09:15.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.938+0000 7f8932600640 1 --2- 172.21.15.86:0/31724381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f892c15ca50 0x7f892c1541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:15.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.938+0000 7f8931c00640 1 --2- 172.21.15.86:0/31724381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f892c0743c0 0x7f892c09ce00 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f892000c410 tx=0x7f892000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:15.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.938+0000 7f8929600640 1 -- 172.21.15.86:0/31724381 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89200074e0 con 0x7f892c0743c0 2024-04-25T14:09:15.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.938+0000 7f8929600640 1 -- 172.21.15.86:0/31724381 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89200047f0 con 0x7f892c0743c0 2024-04-25T14:09:15.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.938+0000 7f8929600640 1 -- 172.21.15.86:0/31724381 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8920007680 con 0x7f892c0743c0 2024-04-25T14:09:15.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.938+0000 7f8934600640 1 -- 172.21.15.86:0/31724381 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f892c09edd0 con 0x7f892c0743c0 2024-04-25T14:09:15.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.942+0000 7f8934600640 1 -- 172.21.15.86:0/31724381 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f892c1f7dc0 con 0x7f892c0743c0 2024-04-25T14:09:15.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.942+0000 7f8934600640 1 -- 172.21.15.86:0/31724381 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f892c1f8050 con 0x7f892c0743c0 2024-04-25T14:09:15.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.942+0000 7f8929600640 1 -- 172.21.15.86:0/31724381 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7f8920011070 con 0x7f892c0743c0 2024-04-25T14:09:15.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.942+0000 7f8929600640 1 --2- 172.21.15.86:0/31724381 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f890c06f520 0x7f890c071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:15.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.942+0000 7f8929600640 1 -- 172.21.15.86:0/31724381 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6394+0+0 (secure 0 0 0) 0x7f892008f2e0 con 0x7f892c0743c0 2024-04-25T14:09:15.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.942+0000 7f8931200640 1 --2- 172.21.15.86:0/31724381 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f890c06f520 0x7f890c071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:15.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.942+0000 7f8931200640 1 --2- 172.21.15.86:0/31724381 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f890c06f520 0x7f890c071a30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f89180ad000 tx=0x7f8918008040 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:15.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:15.946+0000 7f8929600640 1 -- 172.21.15.86:0/31724381 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f892005b7b0 con 0x7f892c0743c0 2024-04-25T14:09:16.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.106+0000 7f8934600640 1 -- 172.21.15.86:0/31724381 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f892c1562f0 con 0x7f892c0743c0 2024-04-25T14:09:16.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.750+0000 7f8929600640 1 -- 172.21.15.86:0/31724381 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) ==== 169+0+0 (secure 0 0 0) 0x7f892005fab0 con 0x7f892c0743c0 2024-04-25T14:09:16.749 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-04-25T14:09:16.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.754+0000 7f8934600640 1 -- 172.21.15.86:0/31724381 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f890c06f520 msgr2=0x7f890c071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:16.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.754+0000 7f8934600640 1 --2- 172.21.15.86:0/31724381 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f890c06f520 0x7f890c071a30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f89180ad000 tx=0x7f8918008040 comp rx=0 tx=0).stop 2024-04-25T14:09:16.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.754+0000 7f8934600640 1 -- 172.21.15.86:0/31724381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f892c0743c0 msgr2=0x7f892c09ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:16.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.754+0000 7f8934600640 1 --2- 172.21.15.86:0/31724381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f892c0743c0 0x7f892c09ce00 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f892000c410 tx=0x7f892000c8d0 comp rx=0 tx=0).stop 2024-04-25T14:09:16.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.754+0000 7f8934600640 1 -- 172.21.15.86:0/31724381 shutdown_connections 2024-04-25T14:09:16.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.754+0000 7f8934600640 1 --2- 172.21.15.86:0/31724381 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f890c06f520 0x7f890c071a30 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:16.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.754+0000 7f8934600640 1 --2- 172.21.15.86:0/31724381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f892c15ca50 0x7f892c1541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:16.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.754+0000 7f8934600640 1 --2- 172.21.15.86:0/31724381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f892c0747d0 0x7f892c153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:16.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.754+0000 7f8934600640 1 --2- 172.21.15.86:0/31724381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f892c0743c0 0x7f892c09ce00 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:16.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.754+0000 7f8934600640 1 -- 172.21.15.86:0/31724381 >> 172.21.15.86:0/31724381 conn(0x7f892c0b4a40 msgr2=0x7f892c06cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:16.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.754+0000 7f8934600640 1 -- 172.21.15.86:0/31724381 shutdown_connections 2024-04-25T14:09:16.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.754+0000 7f8934600640 1 -- 172.21.15.86:0/31724381 wait complete. 2024-04-25T14:09:16.782 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T14:09:16.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.938+0000 7f6f0e600640 1 Processor -- start 2024-04-25T14:09:16.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.938+0000 7f6f0e600640 1 -- start start 2024-04-25T14:09:16.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.942+0000 7f6f0e600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6f0806f990 0x7f6f0806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:16.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.942+0000 7f6f0e600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6f081615c0 0x7f6f081619d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:16.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.942+0000 7f6f0e600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f080702e0 con 0x7f6f080b6530 2024-04-25T14:09:16.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.942+0000 7f6f0e600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6f081622f0 con 0x7f6f081615c0 2024-04-25T14:09:16.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.942+0000 7f6f0e600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f6f080d1730 con 0x7f6f0806f990 2024-04-25T14:09:16.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.942+0000 7f6effe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6f081615c0 0x7f6f081619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:16.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.942+0000 7f6effe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6f081615c0 0x7f6f081619d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57568/0 (socket says 172.21.15.86:57568) 2024-04-25T14:09:16.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.942+0000 7f6effe00640 1 -- 172.21.15.86:0/1944033907 learned_addr learned my addr 172.21.15.86:0/1944033907 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:16.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.942+0000 7f6f0d200640 1 --2- 172.21.15.86:0/1944033907 >> v2:172.21.15.155:3300/0 conn(0x7f6f0806f990 0x7f6f0806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:16.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.942+0000 7f6effe00640 1 -- 172.21.15.86:0/1944033907 >> v2:172.21.15.155:3300/0 conn(0x7f6f0806f990 msgr2=0x7f6f0806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:16.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.942+0000 7f6effe00640 1 --2- 172.21.15.86:0/1944033907 >> v2:172.21.15.155:3300/0 conn(0x7f6f0806f990 0x7f6f0806fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:16.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.942+0000 7f6effe00640 1 -- 172.21.15.86:0/1944033907 >> v1:172.21.15.155:6789/0 conn(0x7f6f080b6530 legacy=0x7f6f080b6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:16.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.942+0000 7f6effe00640 1 -- 172.21.15.86:0/1944033907 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f080d1a10 con 0x7f6f081615c0 2024-04-25T14:09:16.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.946+0000 7f6effe00640 1 --2- 172.21.15.86:0/1944033907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f081615c0 0x7f6f081619d0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f6f080739b0 tx=0x7f6ef001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dfee4dedacd085a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:16.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.946+0000 7f6f07e00640 1 -- 172.21.15.86:0/1944033907 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef00b2070 con 0x7f6f081615c0 2024-04-25T14:09:16.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.946+0000 7f6f07e00640 1 -- 172.21.15.86:0/1944033907 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ef00ad680 con 0x7f6f081615c0 2024-04-25T14:09:16.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.946+0000 7f6f07e00640 1 -- 172.21.15.86:0/1944033907 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef00ada50 con 0x7f6f081615c0 2024-04-25T14:09:16.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.946+0000 7f6f0e600640 1 -- 172.21.15.86:0/1944033907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f081615c0 msgr2=0x7f6f081619d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:16.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.946+0000 7f6f0e600640 1 --2- 172.21.15.86:0/1944033907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f081615c0 0x7f6f081619d0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f6f080739b0 tx=0x7f6ef001c490 comp rx=0 tx=0).stop 2024-04-25T14:09:16.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.946+0000 7f6f0e600640 1 -- 172.21.15.86:0/1944033907 shutdown_connections 2024-04-25T14:09:16.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.946+0000 7f6f0e600640 1 --2- 172.21.15.86:0/1944033907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f081615c0 0x7f6f081619d0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:16.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.946+0000 7f6f0e600640 1 --2- 172.21.15.86:0/1944033907 >> v2:172.21.15.155:3300/0 conn(0x7f6f0806f990 0x7f6f0806fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:16.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.946+0000 7f6f0e600640 1 -- 172.21.15.86:0/1944033907 >> 172.21.15.86:0/1944033907 conn(0x7f6f080cfea0 msgr2=0x7f6f080d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:16.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.946+0000 7f6f0e600640 1 -- 172.21.15.86:0/1944033907 shutdown_connections 2024-04-25T14:09:16.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.946+0000 7f6f0e600640 1 -- 172.21.15.86:0/1944033907 wait complete. 2024-04-25T14:09:16.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.950+0000 7f6f0e600640 1 Processor -- start 2024-04-25T14:09:16.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.950+0000 7f6f0e600640 1 -- start start 2024-04-25T14:09:16.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.950+0000 7f6f0e600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f0806f990 0x7f6f0806dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:16.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.950+0000 7f6f0e600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f080b6530 0x7f6f08160a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:16.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.950+0000 7f6f0e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f081615c0 0x7f6f0815e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:16.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.950+0000 7f6f0e600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0815de90 con 0x7f6f081615c0 2024-04-25T14:09:16.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.950+0000 7f6f0e600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0815dd10 con 0x7f6f0806f990 2024-04-25T14:09:16.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.950+0000 7f6f0e600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6f080d2630 con 0x7f6f080b6530 2024-04-25T14:09:16.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.950+0000 7f6f0dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f081615c0 0x7f6f0815e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:16.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.950+0000 7f6f0dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f081615c0 0x7f6f0815e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57572/0 (socket says 172.21.15.86:57572) 2024-04-25T14:09:16.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.950+0000 7f6f0dc00640 1 -- 172.21.15.86:0/2315695511 learned_addr learned my addr 172.21.15.86:0/2315695511 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:16.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.950+0000 7f6effe00640 1 --2- 172.21.15.86:0/2315695511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f080b6530 0x7f6f08160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:16.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.950+0000 7f6f0d200640 1 --2- 172.21.15.86:0/2315695511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f0806f990 0x7f6f0806dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:16.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.950+0000 7f6f0dc00640 1 -- 172.21.15.86:0/2315695511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f080b6530 msgr2=0x7f6f08160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:16.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.950+0000 7f6f0dc00640 1 --2- 172.21.15.86:0/2315695511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f080b6530 0x7f6f08160a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:16.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.950+0000 7f6f0dc00640 1 -- 172.21.15.86:0/2315695511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f0806f990 msgr2=0x7f6f0806dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:16.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.950+0000 7f6f0dc00640 1 --2- 172.21.15.86:0/2315695511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f0806f990 0x7f6f0806dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:16.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.950+0000 7f6f0dc00640 1 -- 172.21.15.86:0/2315695511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0806e5e0 con 0x7f6f081615c0 2024-04-25T14:09:16.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.950+0000 7f6f0dc00640 1 --2- 172.21.15.86:0/2315695511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f081615c0 0x7f6f0815e010 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000c760 tx=0x7f6f0000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:16.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.950+0000 7f6f05600640 1 -- 172.21.15.86:0/2315695511 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00003b80 con 0x7f6f081615c0 2024-04-25T14:09:16.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.950+0000 7f6f05600640 1 -- 172.21.15.86:0/2315695511 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f00003d20 con 0x7f6f081615c0 2024-04-25T14:09:16.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.950+0000 7f6f05600640 1 -- 172.21.15.86:0/2315695511 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0000aa90 con 0x7f6f081615c0 2024-04-25T14:09:16.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.950+0000 7f6f0e600640 1 -- 172.21.15.86:0/2315695511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f0806e8c0 con 0x7f6f081615c0 2024-04-25T14:09:16.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.950+0000 7f6f0e600640 1 -- 172.21.15.86:0/2315695511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f0806ee80 con 0x7f6f081615c0 2024-04-25T14:09:16.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.954+0000 7f6f0e600640 1 -- 172.21.15.86:0/2315695511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f08076b80 con 0x7f6f081615c0 2024-04-25T14:09:16.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.954+0000 7f6f05600640 1 -- 172.21.15.86:0/2315695511 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7f6f0000ac30 con 0x7f6f081615c0 2024-04-25T14:09:16.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.954+0000 7f6f05600640 1 --2- 172.21.15.86:0/2315695511 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f6ee406f520 0x7f6ee4071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:16.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.954+0000 7f6f05600640 1 -- 172.21.15.86:0/2315695511 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5980+0+0 (secure 0 0 0) 0x7f6f0008f240 con 0x7f6f081615c0 2024-04-25T14:09:16.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.954+0000 7f6f0d200640 1 --2- 172.21.15.86:0/2315695511 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f6ee406f520 0x7f6ee4071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:16.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.954+0000 7f6f0d200640 1 --2- 172.21.15.86:0/2315695511 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f6ee406f520 0x7f6ee4071a30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8009ac0 tx=0x7f6ef8009190 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:16.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:16.958+0000 7f6f05600640 1 -- 172.21.15.86:0/2315695511 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6f080702e0 con 0x7f6f081615c0 2024-04-25T14:09:17.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.114+0000 7f6f0e600640 1 -- 172.21.15.86:0/2315695511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f6f08069d00 con 0x7f6f081615c0 2024-04-25T14:09:17.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.118+0000 7f6f05600640 1 -- 172.21.15.86:0/2315695511 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7f6f0005b950 con 0x7f6f081615c0 2024-04-25T14:09:17.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.122+0000 7f6f0e600640 1 -- 172.21.15.86:0/2315695511 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f6ee406f520 msgr2=0x7f6ee4071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:17.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.122+0000 7f6f0e600640 1 --2- 172.21.15.86:0/2315695511 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f6ee406f520 0x7f6ee4071a30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8009ac0 tx=0x7f6ef8009190 comp rx=0 tx=0).stop 2024-04-25T14:09:17.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.122+0000 7f6f0e600640 1 -- 172.21.15.86:0/2315695511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f081615c0 msgr2=0x7f6f0815e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:17.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.122+0000 7f6f0e600640 1 --2- 172.21.15.86:0/2315695511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f081615c0 0x7f6f0815e010 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000c760 tx=0x7f6f0000cc20 comp rx=0 tx=0).stop 2024-04-25T14:09:17.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.122+0000 7f6f0e600640 1 -- 172.21.15.86:0/2315695511 shutdown_connections 2024-04-25T14:09:17.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.122+0000 7f6f0e600640 1 --2- 172.21.15.86:0/2315695511 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f6ee406f520 0x7f6ee4071a30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:17.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.122+0000 7f6f0e600640 1 --2- 172.21.15.86:0/2315695511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f081615c0 0x7f6f0815e010 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:17.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.122+0000 7f6f0e600640 1 --2- 172.21.15.86:0/2315695511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f080b6530 0x7f6f08160a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:17.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.122+0000 7f6f0e600640 1 --2- 172.21.15.86:0/2315695511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f0806f990 0x7f6f0806dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:17.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.122+0000 7f6f0e600640 1 -- 172.21.15.86:0/2315695511 >> 172.21.15.86:0/2315695511 conn(0x7f6f080cfea0 msgr2=0x7f6f080b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:17.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.122+0000 7f6f0e600640 1 -- 172.21.15.86:0/2315695511 shutdown_connections 2024-04-25T14:09:17.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.122+0000 7f6f0e600640 1 -- 172.21.15.86:0/2315695511 wait complete. 2024-04-25T14:09:17.139 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_for_invalid_client_id' 2024-04-25T14:09:17.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a24400640 1 Processor -- start 2024-04-25T14:09:17.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a24400640 1 -- start start 2024-04-25T14:09:17.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a24400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6a1c0cfca0 0x7f6a1c06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:17.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a24400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a1c0d16e0 0x7f6a1c0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:17.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a24400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a1c0d2030 con 0x7f6a1c06c670 2024-04-25T14:09:17.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a24400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f6a1c070470 con 0x7f6a1c0cfca0 2024-04-25T14:09:17.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a24400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6a1c072bd0 con 0x7f6a1c0d16e0 2024-04-25T14:09:17.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a22400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6a1c06c670 0x7f6a1c06ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42862/0 (socket says 172.21.15.86:42862) 2024-04-25T14:09:17.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a22400640 1 -- 172.21.15.86:0/3586321043 learned_addr learned my addr 172.21.15.86:0/3586321043 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:17.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a21000640 1 --2- 172.21.15.86:0/3586321043 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a1c0d16e0 0x7f6a1c0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:17.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a21a00640 1 --2- 172.21.15.86:0/3586321043 >> v2:172.21.15.155:3300/0 conn(0x7f6a1c0cfca0 0x7f6a1c06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:17.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a1be00640 1 -- 172.21.15.86:0/3586321043 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2552107230 0 0) 0x7f6a140014a0 con 0x7f6a1c06c670 2024-04-25T14:09:17.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a1be00640 1 -- 172.21.15.86:0/3586321043 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a04005740 con 0x7f6a1c06c670 2024-04-25T14:09:17.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a1be00640 1 -- 172.21.15.86:0/3586321043 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3123262603 0 0) 0x7f6a04005740 con 0x7f6a1c06c670 2024-04-25T14:09:17.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a1be00640 1 -- 172.21.15.86:0/3586321043 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a1c0d16e0 msgr2=0x7f6a1c0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:17.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a1be00640 1 --2- 172.21.15.86:0/3586321043 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a1c0d16e0 0x7f6a1c0d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:17.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a1be00640 1 -- 172.21.15.86:0/3586321043 >> v2:172.21.15.155:3300/0 conn(0x7f6a1c0cfca0 msgr2=0x7f6a1c06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:17.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a1be00640 1 --2- 172.21.15.86:0/3586321043 >> v2:172.21.15.155:3300/0 conn(0x7f6a1c0cfca0 0x7f6a1c06f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:17.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a1be00640 1 -- 172.21.15.86:0/3586321043 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a1c06f930 con 0x7f6a1c06c670 2024-04-25T14:09:17.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a1be00640 1 -- 172.21.15.86:0/3586321043 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f6a14001bd0 con 0x7f6a1c06c670 2024-04-25T14:09:17.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a21a00640 1 --2- 172.21.15.86:0/3586321043 >> v2:172.21.15.155:3300/0 conn(0x7f6a1c0cfca0 0x7f6a1c06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:09:17.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a1be00640 1 -- 172.21.15.86:0/3586321043 >> v1:172.21.15.86:6789/0 conn(0x7f6a1c06c670 legacy=0x7f6a1c06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:17.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a1be00640 1 --2- 172.21.15.86:0/3586321043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a04005c40 0x7f6a040060b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:17.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a1be00640 1 --2- 172.21.15.86:0/3586321043 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a1c0d16e0 0x7f6a04006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:17.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a1be00640 1 --2- 172.21.15.86:0/3586321043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a04006f60 0x7f6a04007370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:17.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a1be00640 1 -- 172.21.15.86:0/3586321043 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1c070470 con 0x7f6a04005c40 2024-04-25T14:09:17.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a1be00640 1 -- 172.21.15.86:0/3586321043 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1c072bd0 con 0x7f6a04006f60 2024-04-25T14:09:17.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a1be00640 1 -- 172.21.15.86:0/3586321043 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6a0400dc90 con 0x7f6a1c0d16e0 2024-04-25T14:09:17.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a21a00640 1 --2- 172.21.15.86:0/3586321043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a04005c40 0x7f6a040060b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:17.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a21000640 1 --2- 172.21.15.86:0/3586321043 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a1c0d16e0 0x7f6a04006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:17.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a22400640 1 --2- 172.21.15.86:0/3586321043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a04006f60 0x7f6a04007370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:17.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a21a00640 1 -- 172.21.15.86:0/3586321043 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a1c0d16e0 msgr2=0x7f6a04006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:17.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a21a00640 1 --2- 172.21.15.86:0/3586321043 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a1c0d16e0 0x7f6a04006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:17.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a21a00640 1 -- 172.21.15.86:0/3586321043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a04006f60 msgr2=0x7f6a04007370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:17.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a21a00640 1 --2- 172.21.15.86:0/3586321043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a04006f60 0x7f6a04007370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:17.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a21a00640 1 -- 172.21.15.86:0/3586321043 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a04005740 con 0x7f6a04005c40 2024-04-25T14:09:17.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.290+0000 7f6a22400640 1 --2- 172.21.15.86:0/3586321043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a04006f60 0x7f6a04007370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:17.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.294+0000 7f6a21a00640 1 --2- 172.21.15.86:0/3586321043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a04005c40 0x7f6a040060b0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f6a10009cb0 tx=0x7f6a1001cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef932e6f9154d32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:17.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.294+0000 7f6a1be00640 1 -- 172.21.15.86:0/3586321043 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a10009880 con 0x7f6a04005c40 2024-04-25T14:09:17.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.294+0000 7f6a1be00640 1 -- 172.21.15.86:0/3586321043 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a10004020 con 0x7f6a04005c40 2024-04-25T14:09:17.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.294+0000 7f6a1be00640 1 -- 172.21.15.86:0/3586321043 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a100043f0 con 0x7f6a04005c40 2024-04-25T14:09:17.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.294+0000 7f6a24400640 1 -- 172.21.15.86:0/3586321043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a04005c40 msgr2=0x7f6a040060b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:17.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.294+0000 7f6a24400640 1 --2- 172.21.15.86:0/3586321043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a04005c40 0x7f6a040060b0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f6a10009cb0 tx=0x7f6a1001cc50 comp rx=0 tx=0).stop 2024-04-25T14:09:17.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.294+0000 7f6a22400640 1 -- 172.21.15.86:0/3586321043 reap_dead start 2024-04-25T14:09:17.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.294+0000 7f6a24400640 1 -- 172.21.15.86:0/3586321043 shutdown_connections 2024-04-25T14:09:17.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.294+0000 7f6a24400640 1 -- 172.21.15.86:0/3586321043 >> 172.21.15.86:0/3586321043 conn(0x7f6a1c0b4650 msgr2=0x7f6a1c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:17.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.294+0000 7f6a24400640 1 -- 172.21.15.86:0/3586321043 shutdown_connections 2024-04-25T14:09:17.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.294+0000 7f6a24400640 1 -- 172.21.15.86:0/3586321043 wait complete. 2024-04-25T14:09:17.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.298+0000 7f6a24400640 1 Processor -- start 2024-04-25T14:09:17.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.298+0000 7f6a24400640 1 -- start start 2024-04-25T14:09:17.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.298+0000 7f6a24400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a04005c40 0x7f6a1c0af470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:17.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.298+0000 7f6a24400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a1c15d0f0 0x7f6a1c15eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:17.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.298+0000 7f6a24400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a1c15d500 0x7f6a1c15f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:17.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.298+0000 7f6a24400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1c0d1c70 con 0x7f6a04005c40 2024-04-25T14:09:17.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.298+0000 7f6a24400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1c0d1af0 con 0x7f6a1c15d500 2024-04-25T14:09:17.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.298+0000 7f6a24400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6a1c0d1df0 con 0x7f6a1c15d0f0 2024-04-25T14:09:17.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.298+0000 7f6a21a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a04005c40 0x7f6a1c0af470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:17.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.298+0000 7f6a21a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a04005c40 0x7f6a1c0af470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57584/0 (socket says 172.21.15.86:57584) 2024-04-25T14:09:17.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.298+0000 7f6a21a00640 1 -- 172.21.15.86:0/2461016345 learned_addr learned my addr 172.21.15.86:0/2461016345 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:17.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.302+0000 7f6a21000640 1 --2- 172.21.15.86:0/2461016345 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a1c15d0f0 0x7f6a1c15eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:17.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.302+0000 7f6a22400640 1 --2- 172.21.15.86:0/2461016345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a1c15d500 0x7f6a1c15f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:17.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.302+0000 7f6a21a00640 1 -- 172.21.15.86:0/2461016345 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a1c15d0f0 msgr2=0x7f6a1c15eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:17.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.302+0000 7f6a21a00640 1 --2- 172.21.15.86:0/2461016345 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a1c15d0f0 0x7f6a1c15eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:17.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.302+0000 7f6a21a00640 1 -- 172.21.15.86:0/2461016345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a1c15d500 msgr2=0x7f6a1c15f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:17.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.302+0000 7f6a21a00640 1 --2- 172.21.15.86:0/2461016345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a1c15d500 0x7f6a1c15f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:17.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.302+0000 7f6a21a00640 1 -- 172.21.15.86:0/2461016345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a1c0c4490 con 0x7f6a04005c40 2024-04-25T14:09:17.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.302+0000 7f6a21a00640 1 --2- 172.21.15.86:0/2461016345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a04005c40 0x7f6a1c0af470 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f6a100af040 tx=0x7f6a04007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:17.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.302+0000 7f6a19600640 1 -- 172.21.15.86:0/2461016345 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a10005ba0 con 0x7f6a04005c40 2024-04-25T14:09:17.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.302+0000 7f6a19600640 1 -- 172.21.15.86:0/2461016345 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a100bd070 con 0x7f6a04005c40 2024-04-25T14:09:17.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.302+0000 7f6a24400640 1 -- 172.21.15.86:0/2461016345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a1c160230 con 0x7f6a04005c40 2024-04-25T14:09:17.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.302+0000 7f6a19600640 1 -- 172.21.15.86:0/2461016345 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a100a5700 con 0x7f6a04005c40 2024-04-25T14:09:17.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.302+0000 7f6a24400640 1 -- 172.21.15.86:0/2461016345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a1c15fc70 con 0x7f6a04005c40 2024-04-25T14:09:17.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.302+0000 7f6a24400640 1 -- 172.21.15.86:0/2461016345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a1c09f4b0 con 0x7f6a04005c40 2024-04-25T14:09:17.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.302+0000 7f6a19600640 1 -- 172.21.15.86:0/2461016345 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7f6a100a58a0 con 0x7f6a04005c40 2024-04-25T14:09:17.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.302+0000 7f6a19600640 1 --2- 172.21.15.86:0/2461016345 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f69fc06f520 0x7f69fc071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:17.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.306+0000 7f6a19600640 1 -- 172.21.15.86:0/2461016345 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5980+0+0 (secure 0 0 0) 0x7f6a10129800 con 0x7f6a04005c40 2024-04-25T14:09:17.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.306+0000 7f6a21000640 1 --2- 172.21.15.86:0/2461016345 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f69fc06f520 0x7f69fc071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:17.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.306+0000 7f6a21000640 1 --2- 172.21.15.86:0/2461016345 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f69fc06f520 0x7f69fc071a30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c160a00 tx=0x7f6a08008040 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:17.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.310+0000 7f6a19600640 1 -- 172.21.15.86:0/2461016345 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6a100c7020 con 0x7f6a04005c40 2024-04-25T14:09:17.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.526+0000 7f6a24400640 1 -- 172.21.15.86:0/2461016345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_for_invalid_client_id"]} v 0) -- 0x7f6a1c076bf0 con 0x7f6a04005c40 2024-04-25T14:09:17.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.754+0000 7f6a19600640 1 -- 172.21.15.86:0/2461016345 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_for_invalid_client_id"]}]=0 v314) ==== 150+0+0 (secure 0 0 0) 0x7f6a1c0c4490 con 0x7f6a04005c40 2024-04-25T14:09:17.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.754+0000 7f6a24400640 1 -- 172.21.15.86:0/2461016345 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f69fc06f520 msgr2=0x7f69fc071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:17.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.754+0000 7f6a24400640 1 --2- 172.21.15.86:0/2461016345 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f69fc06f520 0x7f69fc071a30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c160a00 tx=0x7f6a08008040 comp rx=0 tx=0).stop 2024-04-25T14:09:17.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.754+0000 7f6a24400640 1 -- 172.21.15.86:0/2461016345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a04005c40 msgr2=0x7f6a1c0af470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:17.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.754+0000 7f6a24400640 1 --2- 172.21.15.86:0/2461016345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a04005c40 0x7f6a1c0af470 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f6a100af040 tx=0x7f6a04007a50 comp rx=0 tx=0).stop 2024-04-25T14:09:17.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.754+0000 7f6a24400640 1 -- 172.21.15.86:0/2461016345 shutdown_connections 2024-04-25T14:09:17.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.754+0000 7f6a24400640 1 --2- 172.21.15.86:0/2461016345 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f69fc06f520 0x7f69fc071a30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:17.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.754+0000 7f6a24400640 1 --2- 172.21.15.86:0/2461016345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a1c15d500 0x7f6a1c15f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:17.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.754+0000 7f6a24400640 1 --2- 172.21.15.86:0/2461016345 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a1c15d0f0 0x7f6a1c15eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:17.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.754+0000 7f6a24400640 1 --2- 172.21.15.86:0/2461016345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a04005c40 0x7f6a1c0af470 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:17.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.754+0000 7f6a24400640 1 -- 172.21.15.86:0/2461016345 >> 172.21.15.86:0/2461016345 conn(0x7f6a1c0b4650 msgr2=0x7f6a1c06edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:17.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.754+0000 7f6a24400640 1 -- 172.21.15.86:0/2461016345 shutdown_connections 2024-04-25T14:09:17.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.754+0000 7f6a24400640 1 -- 172.21.15.86:0/2461016345 wait complete. 2024-04-25T14:09:17.772 INFO:tasks.cephfs_test_runner:test_for_invalid_client_id (tasks.cephfs.test_mds_metrics.TestMDSMetrics) ... ok 2024-04-25T14:09:17.773 INFO:tasks.cephfs_test_runner:Starting test: test_for_invalid_client_ip (tasks.cephfs.test_mds_metrics.TestMDSMetrics) 2024-04-25T14:09:17.773 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:09:17.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.922+0000 7f3448c00640 1 Processor -- start 2024-04-25T14:09:17.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.922+0000 7f3448c00640 1 -- start start 2024-04-25T14:09:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3448c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f344406fd50 0x7f3444070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3448c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f344415feb0 0x7f34441602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3448c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34440b0a60 con 0x7f344415c2a0 2024-04-25T14:09:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3448c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f34440ae840 con 0x7f344415feb0 2024-04-25T14:09:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3448c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f344406ef90 con 0x7f344406fd50 2024-04-25T14:09:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3441e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f344415c2a0 0x7f34440ae130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42874/0 (socket says 172.21.15.86:42874) 2024-04-25T14:09:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3441e00640 1 -- 172.21.15.86:0/1585675501 learned_addr learned my addr 172.21.15.86:0/1585675501 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3442800640 1 --2- 172.21.15.86:0/1585675501 >> v2:172.21.15.155:3300/0 conn(0x7f344415feb0 0x7f34441602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f343be00640 1 --2- 172.21.15.86:0/1585675501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f344406fd50 0x7f3444070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3441400640 1 -- 172.21.15.86:0/1585675501 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 11739804 0 0) 0x7f34440b0a60 con 0x7f344415c2a0 2024-04-25T14:09:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3441400640 1 -- 172.21.15.86:0/1585675501 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3428003530 con 0x7f344415c2a0 2024-04-25T14:09:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3441400640 1 -- 172.21.15.86:0/1585675501 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 132387260 0 0) 0x7f3428003530 con 0x7f344415c2a0 2024-04-25T14:09:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3441400640 1 -- 172.21.15.86:0/1585675501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f344406fd50 msgr2=0x7f3444070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3441400640 1 --2- 172.21.15.86:0/1585675501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f344406fd50 0x7f3444070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3441400640 1 -- 172.21.15.86:0/1585675501 >> v2:172.21.15.155:3300/0 conn(0x7f344415feb0 msgr2=0x7f34441602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3441400640 1 --2- 172.21.15.86:0/1585675501 >> v2:172.21.15.155:3300/0 conn(0x7f344415feb0 0x7f34441602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3441400640 1 -- 172.21.15.86:0/1585675501 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f344406f270 con 0x7f344415c2a0 2024-04-25T14:09:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3441400640 1 -- 172.21.15.86:0/1585675501 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f3434001680 con 0x7f344415c2a0 2024-04-25T14:09:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3441400640 1 -- 172.21.15.86:0/1585675501 >> v1:172.21.15.86:6789/0 conn(0x7f344415c2a0 legacy=0x7f34440ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3441400640 1 --2- 172.21.15.86:0/1585675501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428003a30 0x7f3428003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3441400640 1 --2- 172.21.15.86:0/1585675501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3428004810 0x7f3428004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3441400640 1 --2- 172.21.15.86:0/1585675501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f344406fd50 0x7f3428005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3441400640 1 -- 172.21.15.86:0/1585675501 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34440ae840 con 0x7f3428003a30 2024-04-25T14:09:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3441400640 1 -- 172.21.15.86:0/1585675501 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f344406ef90 con 0x7f3428004810 2024-04-25T14:09:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3441400640 1 -- 172.21.15.86:0/1585675501 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f342800bad0 con 0x7f344406fd50 2024-04-25T14:09:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3441e00640 1 --2- 172.21.15.86:0/1585675501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428003a30 0x7f3428003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f343be00640 1 --2- 172.21.15.86:0/1585675501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3428004810 0x7f3428004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3442800640 1 --2- 172.21.15.86:0/1585675501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f344406fd50 0x7f3428005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3441e00640 1 -- 172.21.15.86:0/1585675501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f344406fd50 msgr2=0x7f3428005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3441e00640 1 --2- 172.21.15.86:0/1585675501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f344406fd50 0x7f3428005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3441e00640 1 -- 172.21.15.86:0/1585675501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3428004810 msgr2=0x7f3428004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3441e00640 1 --2- 172.21.15.86:0/1585675501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3428004810 0x7f3428004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3441e00640 1 -- 172.21.15.86:0/1585675501 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3428003530 con 0x7f3428003a30 2024-04-25T14:09:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f343be00640 1 --2- 172.21.15.86:0/1585675501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3428004810 0x7f3428004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.926+0000 7f3442800640 1 --2- 172.21.15.86:0/1585675501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f344406fd50 0x7f3428005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:17.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.930+0000 7f3441e00640 1 --2- 172.21.15.86:0/1585675501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428003a30 0x7f3428003ea0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f3444070bb0 tx=0x7f343401ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2518ca8a6909c7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:17.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.930+0000 7f3441400640 1 -- 172.21.15.86:0/1585675501 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3434007d60 con 0x7f3428003a30 2024-04-25T14:09:17.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.930+0000 7f3441400640 1 -- 172.21.15.86:0/1585675501 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34340a9020 con 0x7f3428003a30 2024-04-25T14:09:17.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.930+0000 7f3441400640 1 -- 172.21.15.86:0/1585675501 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34340a5b00 con 0x7f3428003a30 2024-04-25T14:09:17.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.930+0000 7f3448c00640 1 -- 172.21.15.86:0/1585675501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428003a30 msgr2=0x7f3428003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:17.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.930+0000 7f3448c00640 1 --2- 172.21.15.86:0/1585675501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428003a30 0x7f3428003ea0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f3444070bb0 tx=0x7f343401ccb0 comp rx=0 tx=0).stop 2024-04-25T14:09:17.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.930+0000 7f3442800640 1 -- 172.21.15.86:0/1585675501 reap_dead start 2024-04-25T14:09:17.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.930+0000 7f3448c00640 1 -- 172.21.15.86:0/1585675501 shutdown_connections 2024-04-25T14:09:17.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.930+0000 7f3448c00640 1 -- 172.21.15.86:0/1585675501 >> 172.21.15.86:0/1585675501 conn(0x7f34440b4650 msgr2=0x7f34440b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:17.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.930+0000 7f3448c00640 1 -- 172.21.15.86:0/1585675501 shutdown_connections 2024-04-25T14:09:17.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.930+0000 7f3448c00640 1 -- 172.21.15.86:0/1585675501 wait complete. 2024-04-25T14:09:17.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.934+0000 7f3448c00640 1 Processor -- start 2024-04-25T14:09:17.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.934+0000 7f3448c00640 1 -- start start 2024-04-25T14:09:17.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.934+0000 7f3448c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428003a30 0x7f3444156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:17.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.934+0000 7f3448c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f344415ad30 0x7f34440b3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:17.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.934+0000 7f3448c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34440744a0 0x7f34440b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:17.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.934+0000 7f3448c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f344415c830 con 0x7f3428003a30 2024-04-25T14:09:17.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.934+0000 7f3448c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f344415c6b0 con 0x7f344415ad30 2024-04-25T14:09:17.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.934+0000 7f3448c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3444160c10 con 0x7f34440744a0 2024-04-25T14:09:17.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.934+0000 7f3441e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428003a30 0x7f3444156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:17.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.934+0000 7f3441e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428003a30 0x7f3444156240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57598/0 (socket says 172.21.15.86:57598) 2024-04-25T14:09:17.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.934+0000 7f3441e00640 1 -- 172.21.15.86:0/3260346591 learned_addr learned my addr 172.21.15.86:0/3260346591 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:17.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.934+0000 7f3442800640 1 --2- 172.21.15.86:0/3260346591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34440744a0 0x7f34440b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:17.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.934+0000 7f343be00640 1 --2- 172.21.15.86:0/3260346591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f344415ad30 0x7f34440b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:17.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.934+0000 7f3441e00640 1 -- 172.21.15.86:0/3260346591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34440744a0 msgr2=0x7f34440b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:17.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.934+0000 7f3441e00640 1 --2- 172.21.15.86:0/3260346591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34440744a0 0x7f34440b35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:17.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.934+0000 7f3441e00640 1 -- 172.21.15.86:0/3260346591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f344415ad30 msgr2=0x7f34440b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:17.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.934+0000 7f3441e00640 1 --2- 172.21.15.86:0/3260346591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f344415ad30 0x7f34440b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:17.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.934+0000 7f3441e00640 1 -- 172.21.15.86:0/3260346591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34440b1110 con 0x7f3428003a30 2024-04-25T14:09:17.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.934+0000 7f3441e00640 1 --2- 172.21.15.86:0/3260346591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428003a30 0x7f3444156240 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f343401c720 tx=0x7f3428005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.934+0000 7f343a000640 1 -- 172.21.15.86:0/3260346591 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34340a5b00 con 0x7f3428003a30 2024-04-25T14:09:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.934+0000 7f343a000640 1 -- 172.21.15.86:0/3260346591 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34340b3070 con 0x7f3428003a30 2024-04-25T14:09:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.934+0000 7f343a000640 1 -- 172.21.15.86:0/3260346591 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34340047f0 con 0x7f3428003a30 2024-04-25T14:09:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.934+0000 7f3448c00640 1 -- 172.21.15.86:0/3260346591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34440b13f0 con 0x7f3428003a30 2024-04-25T14:09:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.934+0000 7f3448c00640 1 -- 172.21.15.86:0/3260346591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34441f9da0 con 0x7f3428003a30 2024-04-25T14:09:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.938+0000 7f3448c00640 1 -- 172.21.15.86:0/3260346591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3444069d00 con 0x7f3428003a30 2024-04-25T14:09:17.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.938+0000 7f343a000640 1 -- 172.21.15.86:0/3260346591 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7f3434004990 con 0x7f3428003a30 2024-04-25T14:09:17.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.938+0000 7f343a000640 1 --2- 172.21.15.86:0/3260346591 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f342006f550 0x7f3420071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:17.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.938+0000 7f343a000640 1 -- 172.21.15.86:0/3260346591 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5980+0+0 (secure 0 0 0) 0x7f3434129120 con 0x7f3428003a30 2024-04-25T14:09:17.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.938+0000 7f343be00640 1 --2- 172.21.15.86:0/3260346591 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f342006f550 0x7f3420071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:17.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.938+0000 7f343be00640 1 --2- 172.21.15.86:0/3260346591 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f342006f550 0x7f3420071a60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3444159730 tx=0x7f342c008040 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:17.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:17.942+0000 7f343a000640 1 -- 172.21.15.86:0/3260346591 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f34340169f0 con 0x7f3428003a30 2024-04-25T14:09:18.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.102+0000 7f3448c00640 1 -- 172.21.15.86:0/3260346591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3444076bf0 con 0x7f3428003a30 2024-04-25T14:09:18.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.102+0000 7f343a000640 1 -- 172.21.15.86:0/3260346591 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+14233 (secure 0 0 0) 0x7f34340fce20 con 0x7f3428003a30 2024-04-25T14:09:18.099 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:18.099 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":92,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:09:16.750819+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6835/1961103704":"2024-04-26T14:09:13.736901+0000","172.21.15.86:6834/1961103704":"2024-04-26T14:09:13.736901+0000","172.21.15.86:0/2882289101":"2024-04-26T14:09:09.480502+0000","172.21.15.86:6833/132942210":"2024-04-26T14:09:09.480502+0000","172.21.15.86:6832/132942210":"2024-04-26T14:09:09.480502+0000","172.21.15.86:0/3672669933":"2024-04-26T14:09:09.480502+0000","172.21.15.86:0/402603388":"2024-04-26T14:09:09.480502+0000","172.21.15.86:0/3743735908":"2024-04-26T14:09:09.480502+0000","172.21.15.86:0/3132647616":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/4104483212":"2024-04-26T14:09:09.480502+0000","172.21.15.86:0/1564187566":"2024-04-26T14:09:03.346201+0000","172.21.15.86:6833/3589556609":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/1099362620":"2024-04-26T14:09:03.346201+0000","172.21.15.86:6832/3589556609":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/3481243210":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/2509210906":"2024-04-26T14:09:03.346201+0000","172.21.15.86:0/3722630918":"2024-04-26T14:09:03.346201+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:09:18.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.102+0000 7f3448c00640 1 -- 172.21.15.86:0/3260346591 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f342006f550 msgr2=0x7f3420071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:18.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.102+0000 7f3448c00640 1 --2- 172.21.15.86:0/3260346591 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f342006f550 0x7f3420071a60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3444159730 tx=0x7f342c008040 comp rx=0 tx=0).stop 2024-04-25T14:09:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.102+0000 7f3448c00640 1 -- 172.21.15.86:0/3260346591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428003a30 msgr2=0x7f3444156240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.102+0000 7f3448c00640 1 --2- 172.21.15.86:0/3260346591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428003a30 0x7f3444156240 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f343401c720 tx=0x7f3428005840 comp rx=0 tx=0).stop 2024-04-25T14:09:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.102+0000 7f3448c00640 1 -- 172.21.15.86:0/3260346591 shutdown_connections 2024-04-25T14:09:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.102+0000 7f3448c00640 1 --2- 172.21.15.86:0/3260346591 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f342006f550 0x7f3420071a60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.102+0000 7f3448c00640 1 --2- 172.21.15.86:0/3260346591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34440744a0 0x7f34440b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.102+0000 7f3448c00640 1 --2- 172.21.15.86:0/3260346591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f344415ad30 0x7f34440b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.102+0000 7f3448c00640 1 --2- 172.21.15.86:0/3260346591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428003a30 0x7f3444156240 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.102+0000 7f3448c00640 1 -- 172.21.15.86:0/3260346591 >> 172.21.15.86:0/3260346591 conn(0x7f34440b4650 msgr2=0x7f344415a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.102+0000 7f3448c00640 1 -- 172.21.15.86:0/3260346591 shutdown_connections 2024-04-25T14:09:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.102+0000 7f3448c00640 1 -- 172.21.15.86:0/3260346591 wait complete. 2024-04-25T14:09:18.123 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:09:18.123 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:09:18.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.274+0000 7f1c32000640 1 Processor -- start 2024-04-25T14:09:18.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.274+0000 7f1c32000640 1 -- start start 2024-04-25T14:09:18.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.278+0000 7f1c32000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1c2c0d1df0 0x7f1c2c0b0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:18.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.278+0000 7f1c32000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c2c0b0690 0x7f1c2c0d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:18.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.278+0000 7f1c32000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c2c0cde20 con 0x7f1c2c1642a0 2024-04-25T14:09:18.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.278+0000 7f1c32000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1c2c0cea30 con 0x7f1c2c0d1df0 2024-04-25T14:09:18.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.278+0000 7f1c32000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1c2c0ce100 con 0x7f1c2c0b0690 2024-04-25T14:09:18.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.278+0000 7f1c22a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1c2c0d1df0 0x7f1c2c0b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:18.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.278+0000 7f1c22a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1c2c0d1df0 0x7f1c2c0b0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57606/0 (socket says 172.21.15.86:57606) 2024-04-25T14:09:18.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.278+0000 7f1c22a00640 1 -- 172.21.15.86:0/1333739718 learned_addr learned my addr 172.21.15.86:0/1333739718 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:18.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.278+0000 7f1c2be00640 1 --2- 172.21.15.86:0/1333739718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c2c0b0690 0x7f1c2c0d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:18.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.278+0000 7f1c22a00640 1 -- 172.21.15.86:0/1333739718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c2c0b0690 msgr2=0x7f1c2c0d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:18.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.278+0000 7f1c22a00640 1 --2- 172.21.15.86:0/1333739718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c2c0b0690 0x7f1c2c0d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:18.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.278+0000 7f1c22a00640 1 -- 172.21.15.86:0/1333739718 >> v1:172.21.15.155:6789/0 conn(0x7f1c2c1642a0 legacy=0x7f1c2c0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:18.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.278+0000 7f1c22a00640 1 -- 172.21.15.86:0/1333739718 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2c0ce3e0 con 0x7f1c2c0d1df0 2024-04-25T14:09:18.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.282+0000 7f1c22a00640 1 --2- 172.21.15.86:0/1333739718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c2c0d1df0 0x7f1c2c0b0150 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f1c14009870 tx=0x7f1c1401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943db14e6b95e3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:18.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.282+0000 7f1c2aa00640 1 -- 172.21.15.86:0/1333739718 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c140b0070 con 0x7f1c2c0d1df0 2024-04-25T14:09:18.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.282+0000 7f1c2aa00640 1 -- 172.21.15.86:0/1333739718 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c140ab600 con 0x7f1c2c0d1df0 2024-04-25T14:09:18.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.282+0000 7f1c2aa00640 1 -- 172.21.15.86:0/1333739718 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c140ab9d0 con 0x7f1c2c0d1df0 2024-04-25T14:09:18.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.282+0000 7f1c32000640 1 -- 172.21.15.86:0/1333739718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c2c0d1df0 msgr2=0x7f1c2c0b0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:18.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.282+0000 7f1c32000640 1 --2- 172.21.15.86:0/1333739718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c2c0d1df0 0x7f1c2c0b0150 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f1c14009870 tx=0x7f1c1401c4d0 comp rx=0 tx=0).stop 2024-04-25T14:09:18.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.282+0000 7f1c32000640 1 -- 172.21.15.86:0/1333739718 shutdown_connections 2024-04-25T14:09:18.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.282+0000 7f1c32000640 1 --2- 172.21.15.86:0/1333739718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c2c0b0690 0x7f1c2c0d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:18.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.282+0000 7f1c32000640 1 --2- 172.21.15.86:0/1333739718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c2c0d1df0 0x7f1c2c0b0150 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:18.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.282+0000 7f1c32000640 1 -- 172.21.15.86:0/1333739718 >> 172.21.15.86:0/1333739718 conn(0x7f1c2c0b4650 msgr2=0x7f1c2c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:18.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.282+0000 7f1c32000640 1 -- 172.21.15.86:0/1333739718 shutdown_connections 2024-04-25T14:09:18.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.282+0000 7f1c32000640 1 -- 172.21.15.86:0/1333739718 wait complete. 2024-04-25T14:09:18.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.282+0000 7f1c32000640 1 Processor -- start 2024-04-25T14:09:18.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.282+0000 7f1c32000640 1 -- start start 2024-04-25T14:09:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.286+0000 7f1c32000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c2c0b0690 0x7f1c2c157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.286+0000 7f1c32000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c2c1642a0 0x7f1c2c15f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.286+0000 7f1c32000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c2c15f7c0 0x7f1c2c0734a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.286+0000 7f1c32000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c164830 con 0x7f1c2c1642a0 2024-04-25T14:09:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.286+0000 7f1c32000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c1646b0 con 0x7f1c2c0b0690 2024-04-25T14:09:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.286+0000 7f1c32000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1c2c160da0 con 0x7f1c2c15f7c0 2024-04-25T14:09:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.286+0000 7f1c22a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c2c1642a0 0x7f1c2c15f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.286+0000 7f1c22a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c2c1642a0 0x7f1c2c15f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57614/0 (socket says 172.21.15.86:57614) 2024-04-25T14:09:18.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.286+0000 7f1c22a00640 1 -- 172.21.15.86:0/165113127 learned_addr learned my addr 172.21.15.86:0/165113127 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:18.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.286+0000 7f1c2be00640 1 --2- 172.21.15.86:0/165113127 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c2c15f7c0 0x7f1c2c0734a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:18.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.286+0000 7f1c2b400640 1 --2- 172.21.15.86:0/165113127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c2c0b0690 0x7f1c2c157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:18.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.286+0000 7f1c22a00640 1 -- 172.21.15.86:0/165113127 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c2c15f7c0 msgr2=0x7f1c2c0734a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:18.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.286+0000 7f1c22a00640 1 --2- 172.21.15.86:0/165113127 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c2c15f7c0 0x7f1c2c0734a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:18.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.286+0000 7f1c22a00640 1 -- 172.21.15.86:0/165113127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c2c0b0690 msgr2=0x7f1c2c157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:18.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.286+0000 7f1c22a00640 1 --2- 172.21.15.86:0/165113127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c2c0b0690 0x7f1c2c157130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:18.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.286+0000 7f1c22a00640 1 -- 172.21.15.86:0/165113127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2c15fbd0 con 0x7f1c2c1642a0 2024-04-25T14:09:18.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.286+0000 7f1c2b400640 1 --2- 172.21.15.86:0/165113127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c2c0b0690 0x7f1c2c157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:18.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.286+0000 7f1c22a00640 1 --2- 172.21.15.86:0/165113127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c2c1642a0 0x7f1c2c15f280 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f1c1401ab80 tx=0x7f1c14019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:18.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.286+0000 7f1c23e00640 1 -- 172.21.15.86:0/165113127 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c140ba070 con 0x7f1c2c1642a0 2024-04-25T14:09:18.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.286+0000 7f1c23e00640 1 -- 172.21.15.86:0/165113127 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c140b4420 con 0x7f1c2c1642a0 2024-04-25T14:09:18.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.286+0000 7f1c32000640 1 -- 172.21.15.86:0/165113127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c2c15b5a0 con 0x7f1c2c1642a0 2024-04-25T14:09:18.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.286+0000 7f1c23e00640 1 -- 172.21.15.86:0/165113127 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c140b0040 con 0x7f1c2c1642a0 2024-04-25T14:09:18.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.286+0000 7f1c32000640 1 -- 172.21.15.86:0/165113127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c2c160040 con 0x7f1c2c1642a0 2024-04-25T14:09:18.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.286+0000 7f1c32000640 1 -- 172.21.15.86:0/165113127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c2c15b7f0 con 0x7f1c2c1642a0 2024-04-25T14:09:18.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.290+0000 7f1c23e00640 1 -- 172.21.15.86:0/165113127 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7f1c140bd070 con 0x7f1c2c1642a0 2024-04-25T14:09:18.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.290+0000 7f1c23e00640 1 --2- 172.21.15.86:0/165113127 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f1c0806f550 0x7f1c08071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:18.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.290+0000 7f1c23e00640 1 -- 172.21.15.86:0/165113127 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5980+0+0 (secure 0 0 0) 0x7f1c14128540 con 0x7f1c2c1642a0 2024-04-25T14:09:18.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.290+0000 7f1c2b400640 1 --2- 172.21.15.86:0/165113127 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f1c0806f550 0x7f1c08071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:18.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.290+0000 7f1c2b400640 1 --2- 172.21.15.86:0/165113127 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f1c0806f550 0x7f1c08071a60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c070ac0 tx=0x7f1c1c009040 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:18.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.294+0000 7f1c23e00640 1 -- 172.21.15.86:0/165113127 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1c2c15fbd0 con 0x7f1c2c1642a0 2024-04-25T14:09:18.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.450+0000 7f1c32000640 1 -- 172.21.15.86:0/165113127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1c2c069d00 con 0x7f1c2c1642a0 2024-04-25T14:09:18.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.450+0000 7f1c23e00640 1 -- 172.21.15.86:0/165113127 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f1c14016150 con 0x7f1c2c1642a0 2024-04-25T14:09:18.448 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:09:18.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.454+0000 7f1c32000640 1 -- 172.21.15.86:0/165113127 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f1c0806f550 msgr2=0x7f1c08071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:18.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.454+0000 7f1c32000640 1 --2- 172.21.15.86:0/165113127 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f1c0806f550 0x7f1c08071a60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c070ac0 tx=0x7f1c1c009040 comp rx=0 tx=0).stop 2024-04-25T14:09:18.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.454+0000 7f1c32000640 1 -- 172.21.15.86:0/165113127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c2c1642a0 msgr2=0x7f1c2c15f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:18.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.454+0000 7f1c32000640 1 --2- 172.21.15.86:0/165113127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c2c1642a0 0x7f1c2c15f280 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f1c1401ab80 tx=0x7f1c14019270 comp rx=0 tx=0).stop 2024-04-25T14:09:18.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.454+0000 7f1c32000640 1 -- 172.21.15.86:0/165113127 shutdown_connections 2024-04-25T14:09:18.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.454+0000 7f1c32000640 1 --2- 172.21.15.86:0/165113127 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f1c0806f550 0x7f1c08071a60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:18.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.454+0000 7f1c32000640 1 --2- 172.21.15.86:0/165113127 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c2c15f7c0 0x7f1c2c0734a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:18.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.454+0000 7f1c32000640 1 --2- 172.21.15.86:0/165113127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c2c1642a0 0x7f1c2c15f280 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:18.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.454+0000 7f1c32000640 1 --2- 172.21.15.86:0/165113127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c2c0b0690 0x7f1c2c157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:18.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.454+0000 7f1c32000640 1 -- 172.21.15.86:0/165113127 >> 172.21.15.86:0/165113127 conn(0x7f1c2c0b4650 msgr2=0x7f1c2c161760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:18.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.454+0000 7f1c32000640 1 -- 172.21.15.86:0/165113127 shutdown_connections 2024-04-25T14:09:18.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.454+0000 7f1c32000640 1 -- 172.21.15.86:0/165113127 wait complete. 2024-04-25T14:09:18.469 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_for_invalid_client_ip' 2024-04-25T14:09:18.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.618+0000 7f2969000640 1 Processor -- start 2024-04-25T14:09:18.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.618+0000 7f2969000640 1 -- start start 2024-04-25T14:09:18.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2969000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2964070330 0x7f29640d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:18.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2969000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f29640ce790 0x7f296406f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:18.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2969000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29640d2030 con 0x7f29640d1c20 2024-04-25T14:09:18.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2969000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f296415e5c0 con 0x7f29640ce790 2024-04-25T14:09:18.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2969000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f296415e8a0 con 0x7f2964070330 2024-04-25T14:09:18.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2959400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f29640d1c20 0x7f296406ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42878/0 (socket says 172.21.15.86:42878) 2024-04-25T14:09:18.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2959400640 1 -- 172.21.15.86:0/2276506722 learned_addr learned my addr 172.21.15.86:0/2276506722 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:18.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2962800640 1 --2- 172.21.15.86:0/2276506722 >> v2:172.21.15.155:3300/0 conn(0x7f29640ce790 0x7f296406f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:18.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2961e00640 1 --2- 172.21.15.86:0/2276506722 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2964070330 0x7f29640d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:18.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2961400640 1 -- 172.21.15.86:0/2276506722 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2267394952 0 0) 0x7f294c001120 con 0x7f29640d1c20 2024-04-25T14:09:18.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2961400640 1 -- 172.21.15.86:0/2276506722 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2948003530 con 0x7f29640d1c20 2024-04-25T14:09:18.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2961400640 1 -- 172.21.15.86:0/2276506722 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 755710318 0 0) 0x7f2948003530 con 0x7f29640d1c20 2024-04-25T14:09:18.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2961400640 1 -- 172.21.15.86:0/2276506722 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2964070330 msgr2=0x7f29640d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:18.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2961400640 1 --2- 172.21.15.86:0/2276506722 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2964070330 0x7f29640d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:18.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2961400640 1 -- 172.21.15.86:0/2276506722 >> v2:172.21.15.155:3300/0 conn(0x7f29640ce790 msgr2=0x7f296406f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:18.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2961400640 1 --2- 172.21.15.86:0/2276506722 >> v2:172.21.15.155:3300/0 conn(0x7f29640ce790 0x7f296406f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:18.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2961400640 1 -- 172.21.15.86:0/2276506722 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f296415eb80 con 0x7f29640d1c20 2024-04-25T14:09:18.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2961400640 1 -- 172.21.15.86:0/2276506722 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f294c001850 con 0x7f29640d1c20 2024-04-25T14:09:18.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2961400640 1 -- 172.21.15.86:0/2276506722 >> v1:172.21.15.86:6789/0 conn(0x7f29640d1c20 legacy=0x7f296406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:18.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2961400640 1 --2- 172.21.15.86:0/2276506722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2948003a30 0x7f2948003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:18.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2961400640 1 --2- 172.21.15.86:0/2276506722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2948004810 0x7f2948004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:18.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2961400640 1 --2- 172.21.15.86:0/2276506722 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2964070330 0x7f2948005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:18.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2961400640 1 -- 172.21.15.86:0/2276506722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f296415e5c0 con 0x7f2948004810 2024-04-25T14:09:18.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2961400640 1 -- 172.21.15.86:0/2276506722 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f296415e8a0 con 0x7f2948003a30 2024-04-25T14:09:18.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2961400640 1 -- 172.21.15.86:0/2276506722 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f294800bad0 con 0x7f2964070330 2024-04-25T14:09:18.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2959400640 1 --2- 172.21.15.86:0/2276506722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2948004810 0x7f2948004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:18.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2962800640 1 --2- 172.21.15.86:0/2276506722 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2964070330 0x7f2948005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:18.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2961e00640 1 --2- 172.21.15.86:0/2276506722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2948003a30 0x7f2948003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:18.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2959400640 1 -- 172.21.15.86:0/2276506722 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2964070330 msgr2=0x7f2948005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:18.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2959400640 1 --2- 172.21.15.86:0/2276506722 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2964070330 0x7f2948005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:18.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2959400640 1 -- 172.21.15.86:0/2276506722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2948003a30 msgr2=0x7f2948003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:18.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2959400640 1 --2- 172.21.15.86:0/2276506722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2948003a30 0x7f2948003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:18.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.622+0000 7f2959400640 1 -- 172.21.15.86:0/2276506722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2948003530 con 0x7f2948004810 2024-04-25T14:09:18.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.626+0000 7f2959400640 1 --2- 172.21.15.86:0/2276506722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2948004810 0x7f2948004c20 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f294c00c4c0 tx=0x7f294c01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0c31902ce71bff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:18.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.626+0000 7f2961400640 1 -- 172.21.15.86:0/2276506722 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c00c0e0 con 0x7f2948004810 2024-04-25T14:09:18.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.626+0000 7f2961400640 1 -- 172.21.15.86:0/2276506722 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f294c0a9070 con 0x7f2948004810 2024-04-25T14:09:18.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.626+0000 7f2961400640 1 -- 172.21.15.86:0/2276506722 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c005d40 con 0x7f2948004810 2024-04-25T14:09:18.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.626+0000 7f2969000640 1 -- 172.21.15.86:0/2276506722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2948004810 msgr2=0x7f2948004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:18.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.626+0000 7f2969000640 1 --2- 172.21.15.86:0/2276506722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2948004810 0x7f2948004c20 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f294c00c4c0 tx=0x7f294c01cca0 comp rx=0 tx=0).stop 2024-04-25T14:09:18.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.626+0000 7f2962800640 1 -- 172.21.15.86:0/2276506722 reap_dead start 2024-04-25T14:09:18.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.626+0000 7f2969000640 1 -- 172.21.15.86:0/2276506722 shutdown_connections 2024-04-25T14:09:18.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.626+0000 7f2969000640 1 -- 172.21.15.86:0/2276506722 >> 172.21.15.86:0/2276506722 conn(0x7f29640b4650 msgr2=0x7f29640b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:18.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.626+0000 7f2969000640 1 -- 172.21.15.86:0/2276506722 shutdown_connections 2024-04-25T14:09:18.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.626+0000 7f2969000640 1 -- 172.21.15.86:0/2276506722 wait complete. 2024-04-25T14:09:18.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.630+0000 7f2969000640 1 Processor -- start 2024-04-25T14:09:18.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.630+0000 7f2969000640 1 -- start start 2024-04-25T14:09:18.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.630+0000 7f2969000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2948004810 0x7f2964161b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:18.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.630+0000 7f2969000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2964158e20 0x7f2964165660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:18.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.630+0000 7f2969000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2964160590 0x7f29641592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:18.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.630+0000 7f2969000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29640708c0 con 0x7f2964158e20 2024-04-25T14:09:18.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.630+0000 7f2969000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2964070740 con 0x7f2964160590 2024-04-25T14:09:18.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.630+0000 7f2969000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2964168fb0 con 0x7f2948004810 2024-04-25T14:09:18.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.630+0000 7f2959400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2964158e20 0x7f2964165660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:18.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.630+0000 7f2959400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2964158e20 0x7f2964165660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57636/0 (socket says 172.21.15.86:57636) 2024-04-25T14:09:18.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.630+0000 7f2959400640 1 -- 172.21.15.86:0/1656637159 learned_addr learned my addr 172.21.15.86:0/1656637159 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:18.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.630+0000 7f2962800640 1 --2- 172.21.15.86:0/1656637159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2964160590 0x7f29641592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:18.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.630+0000 7f2961e00640 1 --2- 172.21.15.86:0/1656637159 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2948004810 0x7f2964161b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:18.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.630+0000 7f2959400640 1 -- 172.21.15.86:0/1656637159 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2948004810 msgr2=0x7f2964161b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:18.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.630+0000 7f2959400640 1 --2- 172.21.15.86:0/1656637159 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2948004810 0x7f2964161b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:18.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.630+0000 7f2959400640 1 -- 172.21.15.86:0/1656637159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2964160590 msgr2=0x7f29641592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:18.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.630+0000 7f2959400640 1 --2- 172.21.15.86:0/1656637159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2964160590 0x7f29641592a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:18.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.630+0000 7f2959400640 1 -- 172.21.15.86:0/1656637159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2964160aa0 con 0x7f2964158e20 2024-04-25T14:09:18.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.630+0000 7f2959400640 1 --2- 172.21.15.86:0/1656637159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2964158e20 0x7f2964165660 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f294c01cad0 tx=0x7f2948005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:18.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.630+0000 7f295aa00640 1 -- 172.21.15.86:0/1656637159 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c001120 con 0x7f2964158e20 2024-04-25T14:09:18.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.630+0000 7f295aa00640 1 -- 172.21.15.86:0/1656637159 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f294c005d20 con 0x7f2964158e20 2024-04-25T14:09:18.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.630+0000 7f295aa00640 1 -- 172.21.15.86:0/1656637159 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c001830 con 0x7f2964158e20 2024-04-25T14:09:18.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.630+0000 7f2969000640 1 -- 172.21.15.86:0/1656637159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2964160d60 con 0x7f2964158e20 2024-04-25T14:09:18.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.630+0000 7f2969000640 1 -- 172.21.15.86:0/1656637159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2964161320 con 0x7f2964158e20 2024-04-25T14:09:18.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.634+0000 7f2969000640 1 -- 172.21.15.86:0/1656637159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2964069d00 con 0x7f2964158e20 2024-04-25T14:09:18.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.634+0000 7f295aa00640 1 -- 172.21.15.86:0/1656637159 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7f294c0a5780 con 0x7f2964158e20 2024-04-25T14:09:18.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.634+0000 7f295aa00640 1 --2- 172.21.15.86:0/1656637159 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f294006f520 0x7f2940071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:18.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.634+0000 7f295aa00640 1 -- 172.21.15.86:0/1656637159 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5980+0+0 (secure 0 0 0) 0x7f294c128380 con 0x7f2964158e20 2024-04-25T14:09:18.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.634+0000 7f2961e00640 1 --2- 172.21.15.86:0/1656637159 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f294006f520 0x7f2940071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:18.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.634+0000 7f2961e00640 1 --2- 172.21.15.86:0/1656637159 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f294006f520 0x7f2940071a30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2964072200 tx=0x7f2954009190 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:18.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.638+0000 7f295aa00640 1 -- 172.21.15.86:0/1656637159 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f29640d2030 con 0x7f2964158e20 2024-04-25T14:09:18.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:18.854+0000 7f2969000640 1 -- 172.21.15.86:0/1656637159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_for_invalid_client_ip"]} v 0) -- 0x7f296415b810 con 0x7f2964158e20 2024-04-25T14:09:19.528 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:09:19.532 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:09:19.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.762+0000 7f295aa00640 1 -- 172.21.15.86:0/1656637159 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_for_invalid_client_ip"]}]=0 v316) ==== 153+0+0 (secure 0 0 0) 0x7f294c0fd0d0 con 0x7f2964158e20 2024-04-25T14:09:19.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.766+0000 7f2969000640 1 -- 172.21.15.86:0/1656637159 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f294006f520 msgr2=0x7f2940071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:19.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.766+0000 7f2969000640 1 --2- 172.21.15.86:0/1656637159 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f294006f520 0x7f2940071a30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2964072200 tx=0x7f2954009190 comp rx=0 tx=0).stop 2024-04-25T14:09:19.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.766+0000 7f2969000640 1 -- 172.21.15.86:0/1656637159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2964158e20 msgr2=0x7f2964165660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:19.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.766+0000 7f2969000640 1 --2- 172.21.15.86:0/1656637159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2964158e20 0x7f2964165660 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f294c01cad0 tx=0x7f2948005840 comp rx=0 tx=0).stop 2024-04-25T14:09:19.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.766+0000 7f2969000640 1 -- 172.21.15.86:0/1656637159 shutdown_connections 2024-04-25T14:09:19.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.766+0000 7f2969000640 1 --2- 172.21.15.86:0/1656637159 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f294006f520 0x7f2940071a30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:19.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.766+0000 7f2969000640 1 --2- 172.21.15.86:0/1656637159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2964160590 0x7f29641592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:19.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.766+0000 7f2969000640 1 --2- 172.21.15.86:0/1656637159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2964158e20 0x7f2964165660 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:19.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.766+0000 7f2969000640 1 --2- 172.21.15.86:0/1656637159 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2948004810 0x7f2964161b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:19.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.766+0000 7f2969000640 1 -- 172.21.15.86:0/1656637159 >> 172.21.15.86:0/1656637159 conn(0x7f29640b4650 msgr2=0x7f29640742a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:19.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.766+0000 7f2969000640 1 -- 172.21.15.86:0/1656637159 shutdown_connections 2024-04-25T14:09:19.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.766+0000 7f2969000640 1 -- 172.21.15.86:0/1656637159 wait complete. 2024-04-25T14:09:19.786 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T14:09:19.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.942+0000 7f09efc00640 1 Processor -- start 2024-04-25T14:09:19.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.942+0000 7f09efc00640 1 -- start start 2024-04-25T14:09:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.942+0000 7f09efc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09e8070330 0x7f09e80d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.942+0000 7f09efc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f09e80d1c20 0x7f09e806ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.942+0000 7f09efc00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09e80d2030 con 0x7f09e806f270 2024-04-25T14:09:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.942+0000 7f09efc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f09e815e5c0 con 0x7f09e80d1c20 2024-04-25T14:09:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.942+0000 7f09efc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f09e815e8a0 con 0x7f09e8070330 2024-04-25T14:09:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.942+0000 7f09e7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f09e80d1c20 0x7f09e806ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.942+0000 7f09e7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f09e80d1c20 0x7f09e806ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33772/0 (socket says 172.21.15.86:33772) 2024-04-25T14:09:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.942+0000 7f09e7e00640 1 -- 172.21.15.86:0/4173454215 learned_addr learned my addr 172.21.15.86:0/4173454215 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.942+0000 7f09ed200640 1 --2- 172.21.15.86:0/4173454215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09e8070330 0x7f09e80d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.942+0000 7f09e7e00640 1 -- 172.21.15.86:0/4173454215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09e8070330 msgr2=0x7f09e80d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:19.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.942+0000 7f09e7e00640 1 --2- 172.21.15.86:0/4173454215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09e8070330 0x7f09e80d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:19.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.942+0000 7f09e7e00640 1 -- 172.21.15.86:0/4173454215 >> v1:172.21.15.155:6789/0 conn(0x7f09e806f270 legacy=0x7f09e806f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:19.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.942+0000 7f09e7e00640 1 -- 172.21.15.86:0/4173454215 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e815eb80 con 0x7f09e80d1c20 2024-04-25T14:09:19.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.946+0000 7f09e7e00640 1 --2- 172.21.15.86:0/4173454215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09e80d1c20 0x7f09e806ed30 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f09d40098a0 tx=0x7f09d401c290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31856f17186b97be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:19.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.946+0000 7f09e7400640 1 -- 172.21.15.86:0/4173454215 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d40b0070 con 0x7f09e80d1c20 2024-04-25T14:09:19.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.946+0000 7f09e7400640 1 -- 172.21.15.86:0/4173454215 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d40ab600 con 0x7f09e80d1c20 2024-04-25T14:09:19.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.946+0000 7f09e7400640 1 -- 172.21.15.86:0/4173454215 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d40ab9d0 con 0x7f09e80d1c20 2024-04-25T14:09:19.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.946+0000 7f09efc00640 1 -- 172.21.15.86:0/4173454215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09e80d1c20 msgr2=0x7f09e806ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:19.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.946+0000 7f09efc00640 1 --2- 172.21.15.86:0/4173454215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09e80d1c20 0x7f09e806ed30 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f09d40098a0 tx=0x7f09d401c290 comp rx=0 tx=0).stop 2024-04-25T14:09:19.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.946+0000 7f09efc00640 1 -- 172.21.15.86:0/4173454215 shutdown_connections 2024-04-25T14:09:19.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.946+0000 7f09efc00640 1 --2- 172.21.15.86:0/4173454215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09e80d1c20 0x7f09e806ed30 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:19.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.946+0000 7f09efc00640 1 --2- 172.21.15.86:0/4173454215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09e8070330 0x7f09e80d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:19.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.946+0000 7f09efc00640 1 -- 172.21.15.86:0/4173454215 >> 172.21.15.86:0/4173454215 conn(0x7f09e80b4650 msgr2=0x7f09e80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:19.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.946+0000 7f09efc00640 1 -- 172.21.15.86:0/4173454215 shutdown_connections 2024-04-25T14:09:19.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.946+0000 7f09efc00640 1 -- 172.21.15.86:0/4173454215 wait complete. 2024-04-25T14:09:19.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.950+0000 7f09efc00640 1 Processor -- start 2024-04-25T14:09:19.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.950+0000 7f09efc00640 1 -- start start 2024-04-25T14:09:19.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.950+0000 7f09efc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09e806f270 0x7f09e8073960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:19.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.950+0000 7f09efc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09e8070330 0x7f09e815a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:19.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.950+0000 7f09efc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09e80d1c20 0x7f09e815ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:19.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.950+0000 7f09efc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f09e80708c0 con 0x7f09e8070330 2024-04-25T14:09:19.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.950+0000 7f09efc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f09e8070740 con 0x7f09e806f270 2024-04-25T14:09:19.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.950+0000 7f09efc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f09e816afc0 con 0x7f09e80d1c20 2024-04-25T14:09:19.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.950+0000 7f09e7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09e8070330 0x7f09e815a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:19.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.950+0000 7f09e7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09e8070330 0x7f09e815a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33788/0 (socket says 172.21.15.86:33788) 2024-04-25T14:09:19.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.950+0000 7f09e7e00640 1 -- 172.21.15.86:0/1602993243 learned_addr learned my addr 172.21.15.86:0/1602993243 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:19.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.950+0000 7f09ed200640 1 --2- 172.21.15.86:0/1602993243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09e806f270 0x7f09e8073960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:19.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.950+0000 7f09edc00640 1 --2- 172.21.15.86:0/1602993243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09e80d1c20 0x7f09e815ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:19.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.950+0000 7f09e7e00640 1 -- 172.21.15.86:0/1602993243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09e80d1c20 msgr2=0x7f09e815ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:19.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.950+0000 7f09e7e00640 1 --2- 172.21.15.86:0/1602993243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09e80d1c20 0x7f09e815ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:19.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.950+0000 7f09e7e00640 1 -- 172.21.15.86:0/1602993243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09e806f270 msgr2=0x7f09e8073960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:19.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.950+0000 7f09e7e00640 1 --2- 172.21.15.86:0/1602993243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09e806f270 0x7f09e8073960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:19.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.950+0000 7f09e7e00640 1 -- 172.21.15.86:0/1602993243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e80c2a90 con 0x7f09e8070330 2024-04-25T14:09:19.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.950+0000 7f09edc00640 1 --2- 172.21.15.86:0/1602993243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09e80d1c20 0x7f09e815ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:19.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.954+0000 7f09e7e00640 1 --2- 172.21.15.86:0/1602993243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09e8070330 0x7f09e815a640 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f09d401cb30 tx=0x7f09d4019030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:19.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.954+0000 7f09e4c00640 1 -- 172.21.15.86:0/1602993243 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d40ba020 con 0x7f09e8070330 2024-04-25T14:09:19.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.954+0000 7f09e4c00640 1 -- 172.21.15.86:0/1602993243 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d40b6620 con 0x7f09e8070330 2024-04-25T14:09:19.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.954+0000 7f09e4c00640 1 -- 172.21.15.86:0/1602993243 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d40b0040 con 0x7f09e8070330 2024-04-25T14:09:19.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.954+0000 7f09efc00640 1 -- 172.21.15.86:0/1602993243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e80c3750 con 0x7f09e8070330 2024-04-25T14:09:19.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.954+0000 7f09efc00640 1 -- 172.21.15.86:0/1602993243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e80c3d10 con 0x7f09e8070330 2024-04-25T14:09:19.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.954+0000 7f09efc00640 1 -- 172.21.15.86:0/1602993243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e8069d00 con 0x7f09e8070330 2024-04-25T14:09:19.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.954+0000 7f09e4c00640 1 -- 172.21.15.86:0/1602993243 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7f09d40bd020 con 0x7f09e8070330 2024-04-25T14:09:19.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.954+0000 7f09e4c00640 1 --2- 172.21.15.86:0/1602993243 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f09c806f550 0x7f09c8071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:19.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.954+0000 7f09e4c00640 1 -- 172.21.15.86:0/1602993243 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5980+0+0 (secure 0 0 0) 0x7f09d41293f0 con 0x7f09e8070330 2024-04-25T14:09:19.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.954+0000 7f09ed200640 1 --2- 172.21.15.86:0/1602993243 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f09c806f550 0x7f09c8071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:19.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.958+0000 7f09ed200640 1 --2- 172.21.15.86:0/1602993243 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f09c806f550 0x7f09c8071a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f09e8072200 tx=0x7f09d8008040 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:19.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:19.962+0000 7f09e4c00640 1 -- 172.21.15.86:0/1602993243 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f09e80d2030 con 0x7f09e8070330 2024-04-25T14:09:20.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.114+0000 7f09efc00640 1 -- 172.21.15.86:0/1602993243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f09e8076bf0 con 0x7f09e8070330 2024-04-25T14:09:20.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.118+0000 7f09e4c00640 1 -- 172.21.15.86:0/1602993243 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f09d40f5fa0 con 0x7f09e8070330 2024-04-25T14:09:20.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.122+0000 7f09efc00640 1 -- 172.21.15.86:0/1602993243 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f09c806f550 msgr2=0x7f09c8071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:20.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.122+0000 7f09efc00640 1 --2- 172.21.15.86:0/1602993243 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f09c806f550 0x7f09c8071a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f09e8072200 tx=0x7f09d8008040 comp rx=0 tx=0).stop 2024-04-25T14:09:20.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.122+0000 7f09efc00640 1 -- 172.21.15.86:0/1602993243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09e8070330 msgr2=0x7f09e815a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:20.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.122+0000 7f09efc00640 1 --2- 172.21.15.86:0/1602993243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09e8070330 0x7f09e815a640 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f09d401cb30 tx=0x7f09d4019030 comp rx=0 tx=0).stop 2024-04-25T14:09:20.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.122+0000 7f09efc00640 1 -- 172.21.15.86:0/1602993243 shutdown_connections 2024-04-25T14:09:20.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.122+0000 7f09efc00640 1 --2- 172.21.15.86:0/1602993243 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f09c806f550 0x7f09c8071a60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:20.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.122+0000 7f09efc00640 1 --2- 172.21.15.86:0/1602993243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09e80d1c20 0x7f09e815ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:20.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.122+0000 7f09efc00640 1 --2- 172.21.15.86:0/1602993243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09e8070330 0x7f09e815a640 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:20.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.122+0000 7f09efc00640 1 --2- 172.21.15.86:0/1602993243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09e806f270 0x7f09e8073960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:20.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.122+0000 7f09efc00640 1 -- 172.21.15.86:0/1602993243 >> 172.21.15.86:0/1602993243 conn(0x7f09e80b4650 msgr2=0x7f09e80b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:20.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.122+0000 7f09efc00640 1 -- 172.21.15.86:0/1602993243 shutdown_connections 2024-04-25T14:09:20.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.122+0000 7f09efc00640 1 -- 172.21.15.86:0/1602993243 wait complete. 2024-04-25T14:09:20.139 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T14:09:20.139 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:09:20.244 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:09:20.343 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T14:09:20.344 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:09:20.344 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:09:20.353 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2893 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T14:09:20.353 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:09:20.353 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:09:20.362 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4517 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T14:09:20.362 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:09:20.362 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:09:20.401 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2893 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T14:09:20.401 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:09:20.401 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:09:20.417 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4517 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T14:09:20.417 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T14:09:20.417 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:09:20.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.590+0000 7fe73d400640 1 Processor -- start 2024-04-25T14:09:20.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.590+0000 7fe73d400640 1 -- start start 2024-04-25T14:09:20.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.594+0000 7fe73d400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7380d1df0 0x7fe7380cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:20.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.594+0000 7fe73d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe7380cdcc0 0x7fe7380ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:20.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.594+0000 7fe73d400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe73815e5c0 con 0x7fe738070330 2024-04-25T14:09:20.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.594+0000 7fe73d400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe73806fd40 con 0x7fe7380cdcc0 2024-04-25T14:09:20.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.594+0000 7fe73d400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe73815e8a0 con 0x7fe7380d1df0 2024-04-25T14:09:20.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.594+0000 7fe737200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe7380cdcc0 0x7fe7380ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:20.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.594+0000 7fe737200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe7380cdcc0 0x7fe7380ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33800/0 (socket says 172.21.15.86:33800) 2024-04-25T14:09:20.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.594+0000 7fe737200640 1 -- 172.21.15.86:0/2542313907 learned_addr learned my addr 172.21.15.86:0/2542313907 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:20.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.594+0000 7fe72de00640 1 --2- 172.21.15.86:0/2542313907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7380d1df0 0x7fe7380cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:20.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.594+0000 7fe737200640 1 -- 172.21.15.86:0/2542313907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7380d1df0 msgr2=0x7fe7380cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:20.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.594+0000 7fe737200640 1 --2- 172.21.15.86:0/2542313907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7380d1df0 0x7fe7380cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:20.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.594+0000 7fe737200640 1 -- 172.21.15.86:0/2542313907 >> v1:172.21.15.155:6789/0 conn(0x7fe738070330 legacy=0x7fe7380d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:20.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.594+0000 7fe737200640 1 -- 172.21.15.86:0/2542313907 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73815eb80 con 0x7fe7380cdcc0 2024-04-25T14:09:20.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.594+0000 7fe737200640 1 --2- 172.21.15.86:0/2542313907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7380cdcc0 0x7fe7380ce0d0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fe730009870 tx=0x7fe73001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e442fffbc45f6d79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:20.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.598+0000 7fe735e00640 1 -- 172.21.15.86:0/2542313907 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7300ab680 con 0x7fe7380cdcc0 2024-04-25T14:09:20.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.598+0000 7fe735e00640 1 -- 172.21.15.86:0/2542313907 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7300b0070 con 0x7fe7380cdcc0 2024-04-25T14:09:20.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.598+0000 7fe735e00640 1 -- 172.21.15.86:0/2542313907 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7300aba40 con 0x7fe7380cdcc0 2024-04-25T14:09:20.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.598+0000 7fe73d400640 1 -- 172.21.15.86:0/2542313907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7380cdcc0 msgr2=0x7fe7380ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:20.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.598+0000 7fe73d400640 1 --2- 172.21.15.86:0/2542313907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7380cdcc0 0x7fe7380ce0d0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fe730009870 tx=0x7fe73001c4a0 comp rx=0 tx=0).stop 2024-04-25T14:09:20.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.598+0000 7fe73d400640 1 -- 172.21.15.86:0/2542313907 shutdown_connections 2024-04-25T14:09:20.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.598+0000 7fe73d400640 1 --2- 172.21.15.86:0/2542313907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7380cdcc0 0x7fe7380ce0d0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:20.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.598+0000 7fe73d400640 1 --2- 172.21.15.86:0/2542313907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7380d1df0 0x7fe7380cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:20.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.598+0000 7fe73d400640 1 -- 172.21.15.86:0/2542313907 >> 172.21.15.86:0/2542313907 conn(0x7fe7380b4650 msgr2=0x7fe7380b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:20.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.598+0000 7fe73d400640 1 -- 172.21.15.86:0/2542313907 shutdown_connections 2024-04-25T14:09:20.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.598+0000 7fe73d400640 1 -- 172.21.15.86:0/2542313907 wait complete. 2024-04-25T14:09:20.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.598+0000 7fe73d400640 1 Processor -- start 2024-04-25T14:09:20.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.598+0000 7fe73d400640 1 -- start start 2024-04-25T14:09:20.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.602+0000 7fe73d400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe738070330 0x7fe7380c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:20.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.602+0000 7fe73d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7380cdcc0 0x7fe7380ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:20.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.602+0000 7fe73d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe738163a70 0x7fe7380c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:20.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.602+0000 7fe73d400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe73815f4b0 con 0x7fe738163a70 2024-04-25T14:09:20.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.602+0000 7fe73d400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe73815f330 con 0x7fe7380cdcc0 2024-04-25T14:09:20.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.602+0000 7fe73d400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe738070740 con 0x7fe738070330 2024-04-25T14:09:20.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.602+0000 7fe737200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe738163a70 0x7fe7380c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:20.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.602+0000 7fe737200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe738163a70 0x7fe7380c7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33814/0 (socket says 172.21.15.86:33814) 2024-04-25T14:09:20.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.602+0000 7fe737200640 1 -- 172.21.15.86:0/1652055044 learned_addr learned my addr 172.21.15.86:0/1652055044 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:20.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.602+0000 7fe736800640 1 --2- 172.21.15.86:0/1652055044 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe738070330 0x7fe7380c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:20.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.602+0000 7fe72de00640 1 --2- 172.21.15.86:0/1652055044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7380cdcc0 0x7fe7380ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:20.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.602+0000 7fe737200640 1 -- 172.21.15.86:0/1652055044 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe738070330 msgr2=0x7fe7380c90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:20.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.602+0000 7fe737200640 1 --2- 172.21.15.86:0/1652055044 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe738070330 0x7fe7380c90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:20.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.602+0000 7fe737200640 1 -- 172.21.15.86:0/1652055044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7380cdcc0 msgr2=0x7fe7380ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:20.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.602+0000 7fe737200640 1 --2- 172.21.15.86:0/1652055044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7380cdcc0 0x7fe7380ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:20.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.602+0000 7fe737200640 1 -- 172.21.15.86:0/1652055044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7380c9950 con 0x7fe738163a70 2024-04-25T14:09:20.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.602+0000 7fe737200640 1 --2- 172.21.15.86:0/1652055044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe738163a70 0x7fe7380c7ec0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fe73001a0d0 tx=0x7fe73001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:20.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.602+0000 7fe72f400640 1 -- 172.21.15.86:0/1652055044 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7300be5f0 con 0x7fe738163a70 2024-04-25T14:09:20.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.602+0000 7fe72f400640 1 -- 172.21.15.86:0/1652055044 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7300a7070 con 0x7fe738163a70 2024-04-25T14:09:20.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.602+0000 7fe73d400640 1 -- 172.21.15.86:0/1652055044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7380c9c30 con 0x7fe738163a70 2024-04-25T14:09:20.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.602+0000 7fe72f400640 1 -- 172.21.15.86:0/1652055044 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7300be990 con 0x7fe738163a70 2024-04-25T14:09:20.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.602+0000 7fe73d400640 1 -- 172.21.15.86:0/1652055044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7380ca1f0 con 0x7fe738163a70 2024-04-25T14:09:20.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.602+0000 7fe73d400640 1 -- 172.21.15.86:0/1652055044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe738069d00 con 0x7fe738163a70 2024-04-25T14:09:20.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.606+0000 7fe72f400640 1 -- 172.21.15.86:0/1652055044 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7fe7300b43f0 con 0x7fe738163a70 2024-04-25T14:09:20.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.606+0000 7fe72f400640 1 --2- 172.21.15.86:0/1652055044 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fe71406f520 0x7fe714071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:20.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.606+0000 7fe72f400640 1 -- 172.21.15.86:0/1652055044 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5980+0+0 (secure 0 0 0) 0x7fe730129780 con 0x7fe738163a70 2024-04-25T14:09:20.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.606+0000 7fe736800640 1 --2- 172.21.15.86:0/1652055044 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fe71406f520 0x7fe714071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:20.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.606+0000 7fe736800640 1 --2- 172.21.15.86:0/1652055044 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fe71406f520 0x7fe714071a30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe738072200 tx=0x7fe728009040 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:20.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.610+0000 7fe72f400640 1 -- 172.21.15.86:0/1652055044 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe7300c4060 con 0x7fe738163a70 2024-04-25T14:09:20.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.802+0000 7fe73d400640 1 -- 172.21.15.86:0/1652055044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe738073ae0 con 0x7fe738163a70 2024-04-25T14:09:20.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.802+0000 7fe72f400640 1 -- 172.21.15.86:0/1652055044 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+3872 (secure 0 0 0) 0x7fe730016150 con 0x7fe738163a70 2024-04-25T14:09:20.800 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:20.800 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":69,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7186,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/4265929298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4265929298},{"type":"v1","addr":"172.21.15.155:6833","nonce":4265929298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7189,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3254219000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3254219000},{"type":"v1","addr":"172.21.15.155:6835","nonce":3254219000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7195,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/193454084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":193454084},{"type":"v1","addr":"172.21.15.86:6837","nonce":193454084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7833,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2629889243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2629889243},{"type":"v1","addr":"172.21.15.86:6835","nonce":2629889243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[]} 2024-04-25T14:09:20.801 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 69 2024-04-25T14:09:20.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.806+0000 7fe73d400640 1 -- 172.21.15.86:0/1652055044 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fe71406f520 msgr2=0x7fe714071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:20.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.806+0000 7fe73d400640 1 --2- 172.21.15.86:0/1652055044 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fe71406f520 0x7fe714071a30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe738072200 tx=0x7fe728009040 comp rx=0 tx=0).stop 2024-04-25T14:09:20.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.806+0000 7fe73d400640 1 -- 172.21.15.86:0/1652055044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe738163a70 msgr2=0x7fe7380c7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:20.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.806+0000 7fe73d400640 1 --2- 172.21.15.86:0/1652055044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe738163a70 0x7fe7380c7ec0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fe73001a0d0 tx=0x7fe73001ace0 comp rx=0 tx=0).stop 2024-04-25T14:09:20.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.806+0000 7fe73d400640 1 -- 172.21.15.86:0/1652055044 shutdown_connections 2024-04-25T14:09:20.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.806+0000 7fe73d400640 1 --2- 172.21.15.86:0/1652055044 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fe71406f520 0x7fe714071a30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:20.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.806+0000 7fe73d400640 1 --2- 172.21.15.86:0/1652055044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe738163a70 0x7fe7380c7ec0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:20.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.806+0000 7fe73d400640 1 --2- 172.21.15.86:0/1652055044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7380cdcc0 0x7fe7380ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:20.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.806+0000 7fe73d400640 1 --2- 172.21.15.86:0/1652055044 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe738070330 0x7fe7380c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:20.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.806+0000 7fe73d400640 1 -- 172.21.15.86:0/1652055044 >> 172.21.15.86:0/1652055044 conn(0x7fe7380b4650 msgr2=0x7fe73815d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:20.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.806+0000 7fe73d400640 1 -- 172.21.15.86:0/1652055044 shutdown_connections 2024-04-25T14:09:20.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:20.806+0000 7fe73d400640 1 -- 172.21.15.86:0/1652055044 wait complete. 2024-04-25T14:09:20.823 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T14:09:20.823 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T14:09:20.823 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T14:09:20.823 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:09:20.824 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T14:09:20.824 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T14:09:20.824 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T14:09:20.824 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:09:20.824 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T14:09:20.824 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T14:09:20.825 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T14:09:20.825 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:09:20.825 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T14:09:20.825 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T14:09:20.825 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T14:09:20.825 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:09:20.839 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T14:09:20.839 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T14:09:20.844 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T14:09:20.845 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T14:09:20.846 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T14:09:20.846 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T14:09:20.846 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T14:09:20.847 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T14:09:20.887 INFO:tasks.ceph.mds.c:Started 2024-04-25T14:09:20.887 DEBUG:teuthology.parallel:result is None 2024-04-25T14:09:20.889 INFO:tasks.ceph.mds.a:Started 2024-04-25T14:09:20.889 DEBUG:teuthology.parallel:result is None 2024-04-25T14:09:20.892 INFO:tasks.ceph.mds.b:Started 2024-04-25T14:09:20.892 DEBUG:teuthology.parallel:result is None 2024-04-25T14:09:20.894 INFO:tasks.ceph.mds.d:Started 2024-04-25T14:09:20.894 DEBUG:teuthology.parallel:result is None 2024-04-25T14:09:20.894 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T14:09:20.987 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-25T14:09:20.987 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-04-25T14:09:21.001 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2024-04-25T14:09:21.002 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2024-04-25T14:09:21.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.054+0000 7f438dc00640 1 Processor -- start 2024-04-25T14:09:21.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.054+0000 7f438dc00640 1 -- start start 2024-04-25T14:09:21.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.058+0000 7f438dc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4388070330 0x7f43880d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:21.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.058+0000 7f438dc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43880d2030 con 0x7f43880ce790 2024-04-25T14:09:21.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.058+0000 7f438dc00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f438815e5c0 con 0x7f43880d1c20 2024-04-25T14:09:21.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.058+0000 7f438dc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f438815e8a0 con 0x7f4388070330 2024-04-25T14:09:21.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.058+0000 7f4386a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4388070330 0x7f43880d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:21.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.058+0000 7f4387400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f43880ce790 0x7f438806f440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34912/0 (socket says 172.21.15.86:34912) 2024-04-25T14:09:21.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.058+0000 7f4387400640 1 -- 172.21.15.86:0/3084746103 learned_addr learned my addr 172.21.15.86:0/3084746103 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:21.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.058+0000 7f4386000640 1 -- 172.21.15.86:0/3084746103 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3237075055 0 0) 0x7f4380001120 con 0x7f43880ce790 2024-04-25T14:09:21.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.058+0000 7f4386000640 1 -- 172.21.15.86:0/3084746103 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f436c003530 con 0x7f43880ce790 2024-04-25T14:09:21.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.058+0000 7f4386000640 1 -- 172.21.15.86:0/3084746103 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2291878229 0 0) 0x7f438815e5c0 con 0x7f43880d1c20 2024-04-25T14:09:21.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.058+0000 7f4386000640 1 -- 172.21.15.86:0/3084746103 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4380001120 con 0x7f43880d1c20 2024-04-25T14:09:21.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.058+0000 7f4386a00640 1 -- 172.21.15.86:0/3084746103 >> v1:172.21.15.155:6789/0 conn(0x7f43880d1c20 legacy=0x7f438806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:21.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.058+0000 7f4386a00640 1 -- 172.21.15.86:0/3084746103 >> v1:172.21.15.86:6789/0 conn(0x7f43880ce790 legacy=0x7f438806f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:21.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.058+0000 7f4386a00640 1 -- 172.21.15.86:0/3084746103 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f438815eb80 con 0x7f4388070330 2024-04-25T14:09:21.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.062+0000 7f4386a00640 1 --2- 172.21.15.86:0/3084746103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4388070330 0x7f43880d16e0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f4378009dc0 tx=0x7f437801c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40655e467c8c5ae2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:21.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.062+0000 7f4386000640 1 -- 172.21.15.86:0/3084746103 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43780b2070 con 0x7f4388070330 2024-04-25T14:09:21.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.062+0000 7f4386000640 1 -- 172.21.15.86:0/3084746103 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43780a5850 con 0x7f4388070330 2024-04-25T14:09:21.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.062+0000 7f4386000640 1 -- 172.21.15.86:0/3084746103 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43780a5c20 con 0x7f4388070330 2024-04-25T14:09:21.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.062+0000 7f438dc00640 1 -- 172.21.15.86:0/3084746103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4388070330 msgr2=0x7f43880d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:21.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.062+0000 7f438dc00640 1 --2- 172.21.15.86:0/3084746103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4388070330 0x7f43880d16e0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f4378009dc0 tx=0x7f437801c8c0 comp rx=0 tx=0).stop 2024-04-25T14:09:21.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.062+0000 7f438dc00640 1 -- 172.21.15.86:0/3084746103 shutdown_connections 2024-04-25T14:09:21.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.062+0000 7f438dc00640 1 --2- 172.21.15.86:0/3084746103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4388070330 0x7f43880d16e0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:21.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.062+0000 7f438dc00640 1 -- 172.21.15.86:0/3084746103 >> 172.21.15.86:0/3084746103 conn(0x7f43880b4650 msgr2=0x7f43880b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:21.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.062+0000 7f438dc00640 1 -- 172.21.15.86:0/3084746103 shutdown_connections 2024-04-25T14:09:21.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.062+0000 7f438dc00640 1 -- 172.21.15.86:0/3084746103 wait complete. 2024-04-25T14:09:21.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.066+0000 7f438dc00640 1 Processor -- start 2024-04-25T14:09:21.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.066+0000 7f438dc00640 1 -- start start 2024-04-25T14:09:21.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.066+0000 7f438dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4388070330 0x7f43880b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:21.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.066+0000 7f438dc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43880ce790 0x7f438815a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:21.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.066+0000 7f438dc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43880d1c20 0x7f43880af230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:21.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.066+0000 7f438dc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43880708c0 con 0x7f4388070330 2024-04-25T14:09:21.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.066+0000 7f438dc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4388070740 con 0x7f43880d1c20 2024-04-25T14:09:21.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.066+0000 7f438dc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f438806fb50 con 0x7f43880ce790 2024-04-25T14:09:21.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.066+0000 7f4386a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4388070330 0x7f43880b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:21.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.066+0000 7f4386a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4388070330 0x7f43880b1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33852/0 (socket says 172.21.15.86:33852) 2024-04-25T14:09:21.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.066+0000 7f4386a00640 1 -- 172.21.15.86:0/3823320167 learned_addr learned my addr 172.21.15.86:0/3823320167 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:21.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.066+0000 7f4387400640 1 --2- 172.21.15.86:0/3823320167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43880d1c20 0x7f43880af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:21.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.066+0000 7f437e000640 1 --2- 172.21.15.86:0/3823320167 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43880ce790 0x7f438815a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:21.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.066+0000 7f4386a00640 1 -- 172.21.15.86:0/3823320167 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43880ce790 msgr2=0x7f438815a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:21.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.066+0000 7f4386a00640 1 --2- 172.21.15.86:0/3823320167 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43880ce790 0x7f438815a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:21.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.066+0000 7f4386a00640 1 -- 172.21.15.86:0/3823320167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43880d1c20 msgr2=0x7f43880af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:21.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.066+0000 7f4386a00640 1 --2- 172.21.15.86:0/3823320167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43880d1c20 0x7f43880af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:21.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.066+0000 7f4386a00640 1 -- 172.21.15.86:0/3823320167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43880cb210 con 0x7f4388070330 2024-04-25T14:09:21.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.066+0000 7f437e000640 1 --2- 172.21.15.86:0/3823320167 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43880ce790 0x7f438815a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:21.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.066+0000 7f4386a00640 1 --2- 172.21.15.86:0/3823320167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4388070330 0x7f43880b1e90 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f43780194d0 tx=0x7f4378019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:21.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.066+0000 7f437f400640 1 -- 172.21.15.86:0/3823320167 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43780bd020 con 0x7f4388070330 2024-04-25T14:09:21.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.066+0000 7f437f400640 1 -- 172.21.15.86:0/3823320167 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43780b8470 con 0x7f4388070330 2024-04-25T14:09:21.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.066+0000 7f438dc00640 1 -- 172.21.15.86:0/3823320167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f438809e090 con 0x7f4388070330 2024-04-25T14:09:21.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.066+0000 7f437f400640 1 -- 172.21.15.86:0/3823320167 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43780b2040 con 0x7f4388070330 2024-04-25T14:09:21.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.066+0000 7f438dc00640 1 -- 172.21.15.86:0/3823320167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43880ca970 con 0x7f4388070330 2024-04-25T14:09:21.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.070+0000 7f438dc00640 1 -- 172.21.15.86:0/3823320167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4388069d00 con 0x7f4388070330 2024-04-25T14:09:21.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.070+0000 7f437f400640 1 -- 172.21.15.86:0/3823320167 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7f43780c0020 con 0x7f4388070330 2024-04-25T14:09:21.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.070+0000 7f437f400640 1 --2- 172.21.15.86:0/3823320167 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f436406f520 0x7f4364071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:21.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.070+0000 7f437f400640 1 -- 172.21.15.86:0/3823320167 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5980+0+0 (secure 0 0 0) 0x7f437812a320 con 0x7f4388070330 2024-04-25T14:09:21.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.070+0000 7f437e000640 1 --2- 172.21.15.86:0/3823320167 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f436406f520 0x7f4364071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:21.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.070+0000 7f437e000640 1 --2- 172.21.15.86:0/3823320167 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f436406f520 0x7f4364071a30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f438806a490 tx=0x7f4370009040 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:21.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.074+0000 7f437f400640 1 -- 172.21.15.86:0/3823320167 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f43780c7030 con 0x7f4388070330 2024-04-25T14:09:21.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.242+0000 7f438dc00640 1 -- 172.21.15.86:0/3823320167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f4388076bf0 con 0x7f4388070330 2024-04-25T14:09:21.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.774+0000 7f437f400640 1 -- 172.21.15.86:0/3823320167 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v94) ==== 97+0+0 (secure 0 0 0) 0x7f4378016540 con 0x7f4388070330 2024-04-25T14:09:21.773 INFO:teuthology.orchestra.run.smithi086.stderr: removed all blocklist entries 2024-04-25T14:09:21.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.778+0000 7f438dc00640 1 -- 172.21.15.86:0/3823320167 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f436406f520 msgr2=0x7f4364071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:21.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.778+0000 7f438dc00640 1 --2- 172.21.15.86:0/3823320167 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f436406f520 0x7f4364071a30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f438806a490 tx=0x7f4370009040 comp rx=0 tx=0).stop 2024-04-25T14:09:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.778+0000 7f438dc00640 1 -- 172.21.15.86:0/3823320167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4388070330 msgr2=0x7f43880b1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.778+0000 7f438dc00640 1 --2- 172.21.15.86:0/3823320167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4388070330 0x7f43880b1e90 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f43780194d0 tx=0x7f4378019660 comp rx=0 tx=0).stop 2024-04-25T14:09:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.778+0000 7f438dc00640 1 -- 172.21.15.86:0/3823320167 shutdown_connections 2024-04-25T14:09:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.778+0000 7f438dc00640 1 --2- 172.21.15.86:0/3823320167 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f436406f520 0x7f4364071a30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.778+0000 7f438dc00640 1 --2- 172.21.15.86:0/3823320167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43880d1c20 0x7f43880af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.778+0000 7f438dc00640 1 --2- 172.21.15.86:0/3823320167 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43880ce790 0x7f438815a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.778+0000 7f438dc00640 1 --2- 172.21.15.86:0/3823320167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4388070330 0x7f43880b1e90 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:21.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.778+0000 7f438dc00640 1 -- 172.21.15.86:0/3823320167 >> 172.21.15.86:0/3823320167 conn(0x7f43880b4650 msgr2=0x7f438815c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:21.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.778+0000 7f438dc00640 1 -- 172.21.15.86:0/3823320167 shutdown_connections 2024-04-25T14:09:21.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.778+0000 7f438dc00640 1 -- 172.21.15.86:0/3823320167 wait complete. 2024-04-25T14:09:21.800 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T14:09:21.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.950+0000 7f2003200640 1 Processor -- start 2024-04-25T14:09:21.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.950+0000 7f2003200640 1 -- start start 2024-04-25T14:09:21.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.950+0000 7f2003200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ffc15dc90 0x7f1ffc161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:21.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.950+0000 7f2003200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ffc0cd8d0 con 0x7f1ffc0d2c80 2024-04-25T14:09:21.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.950+0000 7f2003200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ffc0ce970 con 0x7f1ffc071530 2024-04-25T14:09:21.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.950+0000 7f2003200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1ffc0ce720 con 0x7f1ffc15dc90 2024-04-25T14:09:21.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.950+0000 7f1ffbe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1ffc0d2c80 0x7f1ffc070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34924/0 (socket says 172.21.15.86:34924) 2024-04-25T14:09:21.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.950+0000 7f1ffbe00640 1 -- 172.21.15.86:0/336957848 learned_addr learned my addr 172.21.15.86:0/336957848 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:21.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.950+0000 7f2001200640 1 --2- 172.21.15.86:0/336957848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ffc15dc90 0x7f1ffc161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:21.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.950+0000 7f1ffaa00640 1 -- 172.21.15.86:0/336957848 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1467051160 0 0) 0x7f1ffc0cd8d0 con 0x7f1ffc0d2c80 2024-04-25T14:09:21.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.950+0000 7f1ffaa00640 1 -- 172.21.15.86:0/336957848 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fe0003530 con 0x7f1ffc0d2c80 2024-04-25T14:09:21.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.950+0000 7f1ffaa00640 1 -- 172.21.15.86:0/336957848 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 176124616 0 0) 0x7f1fe0003530 con 0x7f1ffc0d2c80 2024-04-25T14:09:21.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.950+0000 7f1ffaa00640 1 -- 172.21.15.86:0/336957848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ffc15dc90 msgr2=0x7f1ffc161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:21.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.950+0000 7f1ffaa00640 1 --2- 172.21.15.86:0/336957848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ffc15dc90 0x7f1ffc161df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:21.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.950+0000 7f1ffaa00640 1 -- 172.21.15.86:0/336957848 >> v1:172.21.15.155:6789/0 conn(0x7f1ffc071530 legacy=0x7f1ffc1616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:21.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.950+0000 7f1ffaa00640 1 -- 172.21.15.86:0/336957848 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ffc0d1be0 con 0x7f1ffc0d2c80 2024-04-25T14:09:21.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.950+0000 7f1ffaa00640 1 -- 172.21.15.86:0/336957848 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f1fec001680 con 0x7f1ffc0d2c80 2024-04-25T14:09:21.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.950+0000 7f1ffaa00640 1 -- 172.21.15.86:0/336957848 >> v1:172.21.15.86:6789/0 conn(0x7f1ffc0d2c80 legacy=0x7f1ffc070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:21.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.950+0000 7f1ffaa00640 1 --2- 172.21.15.86:0/336957848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fe0003a60 0x7f1fe0003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:21.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.950+0000 7f1ffaa00640 1 --2- 172.21.15.86:0/336957848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fe0004810 0x7f1fe0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:21.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.950+0000 7f1ffaa00640 1 --2- 172.21.15.86:0/336957848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe00051d0 0x7f1fe000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:21.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.950+0000 7f1ffaa00640 1 -- 172.21.15.86:0/336957848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc0ce720 con 0x7f1fe00051d0 2024-04-25T14:09:21.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.950+0000 7f1ffaa00640 1 -- 172.21.15.86:0/336957848 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe000c130 con 0x7f1fe0004810 2024-04-25T14:09:21.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.950+0000 7f1ffaa00640 1 -- 172.21.15.86:0/336957848 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1fe000c330 con 0x7f1fe0003a60 2024-04-25T14:09:21.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.950+0000 7f2001200640 1 --2- 172.21.15.86:0/336957848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe00051d0 0x7f1fe000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:21.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.950+0000 7f1ffbe00640 1 --2- 172.21.15.86:0/336957848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fe0003a60 0x7f1fe0003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:21.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.950+0000 7f1ffb400640 1 --2- 172.21.15.86:0/336957848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fe0004810 0x7f1fe0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:21.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.954+0000 7f2001200640 1 -- 172.21.15.86:0/336957848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fe0003a60 msgr2=0x7f1fe0003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:21.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.954+0000 7f2001200640 1 --2- 172.21.15.86:0/336957848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1fe0003a60 0x7f1fe0003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:21.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.954+0000 7f2001200640 1 -- 172.21.15.86:0/336957848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fe0004810 msgr2=0x7f1fe0004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:21.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.954+0000 7f2001200640 1 --2- 172.21.15.86:0/336957848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fe0004810 0x7f1fe0004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:21.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.954+0000 7f2001200640 1 -- 172.21.15.86:0/336957848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe0003530 con 0x7f1fe00051d0 2024-04-25T14:09:21.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.954+0000 7f2001200640 1 --2- 172.21.15.86:0/336957848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe00051d0 0x7f1fe000b9d0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc074af0 tx=0x7f1ff401cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a259bf5adadf5361 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:21.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.954+0000 7f1ffaa00640 1 -- 172.21.15.86:0/336957848 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff40b2070 con 0x7f1fe00051d0 2024-04-25T14:09:21.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.954+0000 7f1ffaa00640 1 -- 172.21.15.86:0/336957848 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ff40a5b10 con 0x7f1fe00051d0 2024-04-25T14:09:21.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.954+0000 7f1ffaa00640 1 -- 172.21.15.86:0/336957848 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4009bd0 con 0x7f1fe00051d0 2024-04-25T14:09:21.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.954+0000 7f2003200640 1 -- 172.21.15.86:0/336957848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe00051d0 msgr2=0x7f1fe000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:21.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.954+0000 7f2003200640 1 --2- 172.21.15.86:0/336957848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe00051d0 0x7f1fe000b9d0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc074af0 tx=0x7f1ff401cac0 comp rx=0 tx=0).stop 2024-04-25T14:09:21.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.954+0000 7f2001200640 1 -- 172.21.15.86:0/336957848 reap_dead start 2024-04-25T14:09:21.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.954+0000 7f2003200640 1 -- 172.21.15.86:0/336957848 shutdown_connections 2024-04-25T14:09:21.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.954+0000 7f2003200640 1 -- 172.21.15.86:0/336957848 >> 172.21.15.86:0/336957848 conn(0x7f1ffc0cffc0 msgr2=0x7f1ffc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:21.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.954+0000 7f2003200640 1 -- 172.21.15.86:0/336957848 shutdown_connections 2024-04-25T14:09:21.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.958+0000 7f2003200640 1 -- 172.21.15.86:0/336957848 wait complete. 2024-04-25T14:09:21.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.958+0000 7f2003200640 1 Processor -- start 2024-04-25T14:09:21.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.958+0000 7f2003200640 1 -- start start 2024-04-25T14:09:21.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.958+0000 7f2003200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fe00051d0 0x7f1ffc14fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:21.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.958+0000 7f2003200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ffc0b55b0 0x7f1ffc06ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:21.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.958+0000 7f2003200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ffc06f1b0 0x7f1ffc15e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:21.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.958+0000 7f2003200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc0d2480 con 0x7f1ffc06f1b0 2024-04-25T14:09:21.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.958+0000 7f2003200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc0d21a0 con 0x7f1fe00051d0 2024-04-25T14:09:21.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.958+0000 7f2003200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1ffc0d29a0 con 0x7f1ffc0b55b0 2024-04-25T14:09:21.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.962+0000 7f2001200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ffc06f1b0 0x7f1ffc15e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:21.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.962+0000 7f2001200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ffc06f1b0 0x7f1ffc15e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33862/0 (socket says 172.21.15.86:33862) 2024-04-25T14:09:21.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.962+0000 7f2001200640 1 -- 172.21.15.86:0/3325549618 learned_addr learned my addr 172.21.15.86:0/3325549618 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:21.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.962+0000 7f1ffb400640 1 --2- 172.21.15.86:0/3325549618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ffc0b55b0 0x7f1ffc06ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:21.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.962+0000 7f1ffbe00640 1 --2- 172.21.15.86:0/3325549618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fe00051d0 0x7f1ffc14fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:21.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.962+0000 7f2001200640 1 -- 172.21.15.86:0/3325549618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ffc0b55b0 msgr2=0x7f1ffc06ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:21.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.962+0000 7f2001200640 1 --2- 172.21.15.86:0/3325549618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ffc0b55b0 0x7f1ffc06ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:21.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.962+0000 7f2001200640 1 -- 172.21.15.86:0/3325549618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fe00051d0 msgr2=0x7f1ffc14fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:21.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.962+0000 7f2001200640 1 --2- 172.21.15.86:0/3325549618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fe00051d0 0x7f1ffc14fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:21.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.962+0000 7f2001200640 1 -- 172.21.15.86:0/3325549618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ffc06f5c0 con 0x7f1ffc06f1b0 2024-04-25T14:09:21.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.962+0000 7f1ffbe00640 1 --2- 172.21.15.86:0/3325549618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fe00051d0 0x7f1ffc14fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:21.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.962+0000 7f2001200640 1 --2- 172.21.15.86:0/3325549618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ffc06f1b0 0x7f1ffc15e250 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40985f0 tx=0x7f1fe0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:21.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.962+0000 7f1ff3e00640 1 -- 172.21.15.86:0/3325549618 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff40b2070 con 0x7f1ffc06f1b0 2024-04-25T14:09:21.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.962+0000 7f1ff3e00640 1 -- 172.21.15.86:0/3325549618 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ff40bc070 con 0x7f1ffc06f1b0 2024-04-25T14:09:21.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.962+0000 7f2003200640 1 -- 172.21.15.86:0/3325549618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ffc1f99f0 con 0x7f1ffc06f1b0 2024-04-25T14:09:21.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.962+0000 7f1ff3e00640 1 -- 172.21.15.86:0/3325549618 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4009bd0 con 0x7f1ffc06f1b0 2024-04-25T14:09:21.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.962+0000 7f2003200640 1 -- 172.21.15.86:0/3325549618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ffc1f9c90 con 0x7f1ffc06f1b0 2024-04-25T14:09:21.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.962+0000 7f2003200640 1 -- 172.21.15.86:0/3325549618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ffc076bf0 con 0x7f1ffc06f1b0 2024-04-25T14:09:21.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.962+0000 7f1ff3e00640 1 -- 172.21.15.86:0/3325549618 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7f1ff40b7440 con 0x7f1ffc06f1b0 2024-04-25T14:09:21.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.966+0000 7f1ff3e00640 1 --2- 172.21.15.86:0/3325549618 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f1fdc06f550 0x7f1fdc071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:21.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.966+0000 7f1ff3e00640 1 -- 172.21.15.86:0/3325549618 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5249+0+0 (secure 0 0 0) 0x7f1ff4129d20 con 0x7f1ffc06f1b0 2024-04-25T14:09:21.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.966+0000 7f1ffbe00640 1 --2- 172.21.15.86:0/3325549618 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f1fdc06f550 0x7f1fdc071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:21.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.966+0000 7f1ffbe00640 1 --2- 172.21.15.86:0/3325549618 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f1fdc06f550 0x7f1fdc071a60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1fec001ec0 tx=0x7f1fec009040 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:21.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:21.970+0000 7f1ff3e00640 1 -- 172.21.15.86:0/3325549618 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1ff40c5020 con 0x7f1ffc06f1b0 2024-04-25T14:09:22.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.178+0000 7f2003200640 1 -- 172.21.15.86:0/3325549618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1ffc0d1470 con 0x7f1ffc06f1b0 2024-04-25T14:09:22.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.178+0000 7f1ff3e00640 1 -- 172.21.15.86:0/3325549618 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v25) ==== 80+0+5238 (secure 0 0 0) 0x7f1ffc06f5c0 con 0x7f1ffc06f1b0 2024-04-25T14:09:22.176 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:22.176 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:09:22.176 INFO:teuthology.orchestra.run.smithi086.stdout: "auth_dump": [ 2024-04-25T14:09:22.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:09:22.176 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.a", 2024-04-25T14:09:22.176 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmWk5fAxAAKEBKhWewpl23MUcvMi8qDg==", 2024-04-25T14:09:22.176 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:09:22.176 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:09:22.176 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:09:22.176 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:09:22.176 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:09:22.177 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:09:22.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:09:22.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:09:22.177 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.b", 2024-04-25T14:09:22.177 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmDhzOEBAABNys/NAEhucQ7A+HzQlzQw==", 2024-04-25T14:09:22.177 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:09:22.177 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:09:22.177 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:09:22.177 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:09:22.177 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:09:22.177 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:09:22.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:09:22.177 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:09:22.178 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.c", 2024-04-25T14:09:22.178 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmA/hNCxAAclDGbIjLfjqyXn0MwX/IkA==", 2024-04-25T14:09:22.178 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:09:22.178 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:09:22.178 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:09:22.178 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:09:22.178 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:09:22.178 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:09:22.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:09:22.178 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:09:22.178 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.d", 2024-04-25T14:09:22.178 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmemCcGBAAGrCx6C850qRJSWYjebxOBA==", 2024-04-25T14:09:22.178 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:09:22.178 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:09:22.179 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:09:22.179 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:09:22.179 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:09:22.179 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:09:22.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:09:22.179 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:09:22.179 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.0", 2024-04-25T14:09:22.179 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCOYipm9etgGxAAemg9CqXN0ShduDtDuKF2Zw==", 2024-04-25T14:09:22.179 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:09:22.179 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:09:22.179 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:09:22.179 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:09:22.179 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:09:22.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:09:22.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:09:22.180 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.1", 2024-04-25T14:09:22.180 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCRYipmBCiNJhAABAWHk8NEGVvqieL2klGhmA==", 2024-04-25T14:09:22.180 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:09:22.180 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:09:22.180 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:09:22.180 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:09:22.180 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:09:22.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:09:22.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:09:22.180 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.2", 2024-04-25T14:09:22.180 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCUYipm/XGYLxAAPXsw1svKseFzFiuDUjAsJQ==", 2024-04-25T14:09:22.180 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:09:22.181 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:09:22.181 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:09:22.181 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:09:22.181 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:09:22.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:09:22.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:09:22.181 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.3", 2024-04-25T14:09:22.181 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCXYipmlSRaORAAwpKyzZDdVYHWl2pAsQjmpw==", 2024-04-25T14:09:22.181 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:09:22.181 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:09:22.181 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:09:22.181 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:09:22.181 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:09:22.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:09:22.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:09:22.182 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.4", 2024-04-25T14:09:22.182 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCdYipmBI3FMBAAV2NdggfRSFnQvrcOC2YqiA==", 2024-04-25T14:09:22.182 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:09:22.182 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:09:22.182 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:09:22.182 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:09:22.182 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:09:22.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:09:22.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:09:22.182 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.5", 2024-04-25T14:09:22.182 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQChYipmctYxABAABwmOMgyYuOfRWD9KdHILng==", 2024-04-25T14:09:22.182 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:09:22.182 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:09:22.183 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:09:22.183 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:09:22.183 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:09:22.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:09:22.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:09:22.183 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.6", 2024-04-25T14:09:22.183 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCkYipmSL7TCRAAhFMWg1AQm54bB+x1GDcusA==", 2024-04-25T14:09:22.183 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:09:22.183 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:09:22.183 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:09:22.183 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:09:22.183 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:09:22.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:09:22.183 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:09:22.183 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.7", 2024-04-25T14:09:22.184 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCnYipmZ4F+FRAAEGaZN+byv/qJUFZb7WxTFg==", 2024-04-25T14:09:22.184 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:09:22.184 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:09:22.184 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:09:22.184 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:09:22.184 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:09:22.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:09:22.184 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:09:22.184 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.0", 2024-04-25T14:09:22.184 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipm5u4xHRAACDFe3Eb4KLrsdLj4eFe7mA==", 2024-04-25T14:09:22.184 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:09:22.185 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:09:22.185 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-25T14:09:22.185 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T14:09:22.185 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:09:22.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:09:22.185 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:09:22.185 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.1", 2024-04-25T14:09:22.185 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmDp1hJRAAQkBc5PdQoMTT7W4HahkTyg==", 2024-04-25T14:09:22.185 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:09:22.185 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:09:22.185 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-25T14:09:22.185 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw pool=fs2_data" 2024-04-25T14:09:22.186 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:09:22.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:09:22.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:09:22.186 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.2", 2024-04-25T14:09:22.186 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmhbhGIRAAxao/J0WY53IITLETFgwIVw==", 2024-04-25T14:09:22.186 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:09:22.186 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:09:22.186 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-25T14:09:22.186 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw pool=fs2_data" 2024-04-25T14:09:22.186 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:09:22.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:09:22.186 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:09:22.186 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.3", 2024-04-25T14:09:22.187 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmJsCBKRAAzXn2qDtblIpqt7U/TUUlyA==", 2024-04-25T14:09:22.187 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:09:22.187 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:09:22.187 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-25T14:09:22.187 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw pool=fs2_data" 2024-04-25T14:09:22.187 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:09:22.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:09:22.187 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:09:22.187 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.admin", 2024-04-25T14:09:22.187 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCKYipmTqbVIBAA1VpOtj24f2ibVEIT4EJwmw==", 2024-04-25T14:09:22.187 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:09:22.187 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-25T14:09:22.187 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:09:22.187 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:09:22.187 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:09:22.188 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:09:22.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:09:22.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:09:22.188 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.x", 2024-04-25T14:09:22.188 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCKYipmKMzAORAAtRyTNJ5HWi2XwuamoqwB0Q==", 2024-04-25T14:09:22.188 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:09:22.188 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-25T14:09:22.188 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-25T14:09:22.188 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:09:22.188 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:09:22.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:09:22.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:09:22.188 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.y", 2024-04-25T14:09:22.188 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCKYipmC7A7NRAAEwVNA0yxt3fwdHXrsV0dvg==", 2024-04-25T14:09:22.188 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:09:22.189 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-25T14:09:22.189 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-25T14:09:22.189 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:09:22.189 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:09:22.189 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:09:22.189 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-25T14:09:22.189 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:09:22.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.178+0000 7f2003200640 1 -- 172.21.15.86:0/3325549618 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f1fdc06f550 msgr2=0x7f1fdc071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:22.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.178+0000 7f2003200640 1 --2- 172.21.15.86:0/3325549618 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f1fdc06f550 0x7f1fdc071a60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1fec001ec0 tx=0x7f1fec009040 comp rx=0 tx=0).stop 2024-04-25T14:09:22.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.178+0000 7f2003200640 1 -- 172.21.15.86:0/3325549618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ffc06f1b0 msgr2=0x7f1ffc15e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:22.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.178+0000 7f2003200640 1 --2- 172.21.15.86:0/3325549618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ffc06f1b0 0x7f1ffc15e250 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40985f0 tx=0x7f1fe0005840 comp rx=0 tx=0).stop 2024-04-25T14:09:22.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.182+0000 7f2003200640 1 -- 172.21.15.86:0/3325549618 shutdown_connections 2024-04-25T14:09:22.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.182+0000 7f2003200640 1 --2- 172.21.15.86:0/3325549618 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f1fdc06f550 0x7f1fdc071a60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.182+0000 7f2003200640 1 --2- 172.21.15.86:0/3325549618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ffc06f1b0 0x7f1ffc15e250 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.182+0000 7f2003200640 1 --2- 172.21.15.86:0/3325549618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ffc0b55b0 0x7f1ffc06ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.182+0000 7f2003200640 1 --2- 172.21.15.86:0/3325549618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fe00051d0 0x7f1ffc14fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.182+0000 7f2003200640 1 -- 172.21.15.86:0/3325549618 >> 172.21.15.86:0/3325549618 conn(0x7f1ffc0cffc0 msgr2=0x7f1ffc0b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:22.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.182+0000 7f2003200640 1 -- 172.21.15.86:0/3325549618 shutdown_connections 2024-04-25T14:09:22.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.182+0000 7f2003200640 1 -- 172.21.15.86:0/3325549618 wait complete. 2024-04-25T14:09:22.198 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:09:22.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.346+0000 7f3ef4600640 1 Processor -- start 2024-04-25T14:09:22.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.346+0000 7f3ef4600640 1 -- start start 2024-04-25T14:09:22.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.350+0000 7f3ef4600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3eec070330 0x7f3eec0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.350+0000 7f3ef4600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3eec0d1c20 0x7f3eec06ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.350+0000 7f3ef4600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3eec0d2030 con 0x7f3eec06f270 2024-04-25T14:09:22.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.350+0000 7f3ef4600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3eec15e5c0 con 0x7f3eec070330 2024-04-25T14:09:22.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.350+0000 7f3ef4600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3eec15e8a0 con 0x7f3eec0d1c20 2024-04-25T14:09:22.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.350+0000 7f3ef1c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3eec070330 0x7f3eec0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.350+0000 7f3ef1c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3eec070330 0x7f3eec0d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33870/0 (socket says 172.21.15.86:33870) 2024-04-25T14:09:22.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.350+0000 7f3ef1c00640 1 -- 172.21.15.86:0/1328888648 learned_addr learned my addr 172.21.15.86:0/1328888648 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:22.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.350+0000 7f3ef1200640 1 --2- 172.21.15.86:0/1328888648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3eec0d1c20 0x7f3eec06ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.350+0000 7f3ef1c00640 1 -- 172.21.15.86:0/1328888648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3eec0d1c20 msgr2=0x7f3eec06ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:22.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.350+0000 7f3ef1c00640 1 --2- 172.21.15.86:0/1328888648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3eec0d1c20 0x7f3eec06ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.350+0000 7f3ef1c00640 1 -- 172.21.15.86:0/1328888648 >> v1:172.21.15.155:6789/0 conn(0x7f3eec06f270 legacy=0x7f3eec06f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:22.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.350+0000 7f3ef1c00640 1 -- 172.21.15.86:0/1328888648 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eec15eb80 con 0x7f3eec070330 2024-04-25T14:09:22.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.354+0000 7f3ef1c00640 1 --2- 172.21.15.86:0/1328888648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eec070330 0x7f3eec0d16e0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0009870 tx=0x7f3ee001c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a445718705302c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:22.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.354+0000 7f3eebe00640 1 -- 172.21.15.86:0/1328888648 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee00b0070 con 0x7f3eec070330 2024-04-25T14:09:22.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.354+0000 7f3eebe00640 1 -- 172.21.15.86:0/1328888648 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ee00ab600 con 0x7f3eec070330 2024-04-25T14:09:22.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.354+0000 7f3eebe00640 1 -- 172.21.15.86:0/1328888648 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee00ab9d0 con 0x7f3eec070330 2024-04-25T14:09:22.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.354+0000 7f3ef4600640 1 -- 172.21.15.86:0/1328888648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eec070330 msgr2=0x7f3eec0d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:22.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.354+0000 7f3ef4600640 1 --2- 172.21.15.86:0/1328888648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eec070330 0x7f3eec0d16e0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0009870 tx=0x7f3ee001c2b0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.354+0000 7f3ef4600640 1 -- 172.21.15.86:0/1328888648 shutdown_connections 2024-04-25T14:09:22.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.354+0000 7f3ef4600640 1 --2- 172.21.15.86:0/1328888648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3eec0d1c20 0x7f3eec06ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.354+0000 7f3ef4600640 1 --2- 172.21.15.86:0/1328888648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eec070330 0x7f3eec0d16e0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.354+0000 7f3ef4600640 1 -- 172.21.15.86:0/1328888648 >> 172.21.15.86:0/1328888648 conn(0x7f3eec0b4650 msgr2=0x7f3eec0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:22.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.354+0000 7f3ef4600640 1 -- 172.21.15.86:0/1328888648 shutdown_connections 2024-04-25T14:09:22.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.354+0000 7f3ef4600640 1 -- 172.21.15.86:0/1328888648 wait complete. 2024-04-25T14:09:22.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.354+0000 7f3ef4600640 1 Processor -- start 2024-04-25T14:09:22.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.354+0000 7f3ef4600640 1 -- start start 2024-04-25T14:09:22.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.358+0000 7f3ef4600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3eec06f270 0x7f3eec0b1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.358+0000 7f3ef4600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3eec070330 0x7f3eec0b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.358+0000 7f3ef4600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eec0d1c20 0x7f3eec072880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.358+0000 7f3ef4600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec0708c0 con 0x7f3eec0d1c20 2024-04-25T14:09:22.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.358+0000 7f3ef4600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec070740 con 0x7f3eec06f270 2024-04-25T14:09:22.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.358+0000 7f3ef4600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3eec16afc0 con 0x7f3eec070330 2024-04-25T14:09:22.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.358+0000 7f3ef2600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eec0d1c20 0x7f3eec072880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.358+0000 7f3ef2600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eec0d1c20 0x7f3eec072880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33882/0 (socket says 172.21.15.86:33882) 2024-04-25T14:09:22.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.358+0000 7f3ef2600640 1 -- 172.21.15.86:0/1064465138 learned_addr learned my addr 172.21.15.86:0/1064465138 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:22.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.358+0000 7f3ef1c00640 1 --2- 172.21.15.86:0/1064465138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3eec06f270 0x7f3eec0b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.358+0000 7f3ef1200640 1 --2- 172.21.15.86:0/1064465138 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3eec070330 0x7f3eec0b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.358+0000 7f3ef2600640 1 -- 172.21.15.86:0/1064465138 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3eec070330 msgr2=0x7f3eec0b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:22.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.358+0000 7f3ef2600640 1 --2- 172.21.15.86:0/1064465138 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3eec070330 0x7f3eec0b1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.358+0000 7f3ef2600640 1 -- 172.21.15.86:0/1064465138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3eec06f270 msgr2=0x7f3eec0b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:22.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.358+0000 7f3ef2600640 1 --2- 172.21.15.86:0/1064465138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3eec06f270 0x7f3eec0b1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.358+0000 7f3ef2600640 1 -- 172.21.15.86:0/1064465138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eec0b0a50 con 0x7f3eec0d1c20 2024-04-25T14:09:22.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.358+0000 7f3ef2600640 1 --2- 172.21.15.86:0/1064465138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eec0d1c20 0x7f3eec072880 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400ba10 tx=0x7f3ee400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:22.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.358+0000 7f3ee9600640 1 -- 172.21.15.86:0/1064465138 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4019070 con 0x7f3eec0d1c20 2024-04-25T14:09:22.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.358+0000 7f3ee9600640 1 -- 172.21.15.86:0/1064465138 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ee400cb40 con 0x7f3eec0d1c20 2024-04-25T14:09:22.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.358+0000 7f3ef4600640 1 -- 172.21.15.86:0/1064465138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eec0b0d30 con 0x7f3eec0d1c20 2024-04-25T14:09:22.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.358+0000 7f3ee9600640 1 -- 172.21.15.86:0/1064465138 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4014440 con 0x7f3eec0d1c20 2024-04-25T14:09:22.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.358+0000 7f3ef4600640 1 -- 172.21.15.86:0/1064465138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eec0b12f0 con 0x7f3eec0d1c20 2024-04-25T14:09:22.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.358+0000 7f3ef4600640 1 -- 172.21.15.86:0/1064465138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eec069d00 con 0x7f3eec0d1c20 2024-04-25T14:09:22.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.362+0000 7f3ee9600640 1 -- 172.21.15.86:0/1064465138 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7f3ee401c070 con 0x7f3eec0d1c20 2024-04-25T14:09:22.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.362+0000 7f3ee9600640 1 --2- 172.21.15.86:0/1064465138 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f3ecc06f550 0x7f3ecc071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.362+0000 7f3ee9600640 1 -- 172.21.15.86:0/1064465138 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5249+0+0 (secure 0 0 0) 0x7f3ee408eda0 con 0x7f3eec0d1c20 2024-04-25T14:09:22.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.362+0000 7f3ef1c00640 1 --2- 172.21.15.86:0/1064465138 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f3ecc06f550 0x7f3ecc071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.362+0000 7f3ef1c00640 1 --2- 172.21.15.86:0/1064465138 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f3ecc06f550 0x7f3ecc071a60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3ee00ad000 tx=0x7f3ee00a6000 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:22.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.366+0000 7f3ee9600640 1 -- 172.21.15.86:0/1064465138 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3ee405b6f0 con 0x7f3eec0d1c20 2024-04-25T14:09:22.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.522+0000 7f3ef4600640 1 -- 172.21.15.86:0/1064465138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3eec076bf0 con 0x7f3eec0d1c20 2024-04-25T14:09:22.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.522+0000 7f3ee9600640 1 -- 172.21.15.86:0/1064465138 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+13165 (secure 0 0 0) 0x7f3ee405f9f0 con 0x7f3eec0d1c20 2024-04-25T14:09:22.522 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:22.522 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":96,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:09:21.785770+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:09:22.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.526+0000 7f3ef4600640 1 -- 172.21.15.86:0/1064465138 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f3ecc06f550 msgr2=0x7f3ecc071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:22.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.526+0000 7f3ef4600640 1 --2- 172.21.15.86:0/1064465138 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f3ecc06f550 0x7f3ecc071a60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3ee00ad000 tx=0x7f3ee00a6000 comp rx=0 tx=0).stop 2024-04-25T14:09:22.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.526+0000 7f3ef4600640 1 -- 172.21.15.86:0/1064465138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eec0d1c20 msgr2=0x7f3eec072880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:22.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.526+0000 7f3ef4600640 1 --2- 172.21.15.86:0/1064465138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eec0d1c20 0x7f3eec072880 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400ba10 tx=0x7f3ee400bed0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.526+0000 7f3ef4600640 1 -- 172.21.15.86:0/1064465138 shutdown_connections 2024-04-25T14:09:22.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.526+0000 7f3ef4600640 1 --2- 172.21.15.86:0/1064465138 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f3ecc06f550 0x7f3ecc071a60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.526+0000 7f3ef4600640 1 --2- 172.21.15.86:0/1064465138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eec0d1c20 0x7f3eec072880 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.526+0000 7f3ef4600640 1 --2- 172.21.15.86:0/1064465138 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3eec070330 0x7f3eec0b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.526+0000 7f3ef4600640 1 --2- 172.21.15.86:0/1064465138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3eec06f270 0x7f3eec0b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.526+0000 7f3ef4600640 1 -- 172.21.15.86:0/1064465138 >> 172.21.15.86:0/1064465138 conn(0x7f3eec0b4650 msgr2=0x7f3eec0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:22.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.526+0000 7f3ef4600640 1 -- 172.21.15.86:0/1064465138 shutdown_connections 2024-04-25T14:09:22.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.526+0000 7f3ef4600640 1 -- 172.21.15.86:0/1064465138 wait complete. 2024-04-25T14:09:22.543 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:09:22.544 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:09:22.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.698+0000 7f5dbb600640 1 Processor -- start 2024-04-25T14:09:22.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.698+0000 7f5dbb600640 1 -- start start 2024-04-25T14:09:22.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.698+0000 7f5dbb600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5db4070330 0x7f5db40d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.698+0000 7f5dbb600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5db40d1c20 0x7f5db406ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.698+0000 7f5dbb600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5db40d2030 con 0x7f5db406f270 2024-04-25T14:09:22.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.698+0000 7f5dbb600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5db415e5c0 con 0x7f5db40d1c20 2024-04-25T14:09:22.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.698+0000 7f5dbb600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5db415e8a0 con 0x7f5db4070330 2024-04-25T14:09:22.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.698+0000 7f5dabe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5db40d1c20 0x7f5db406ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.698+0000 7f5db9600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5db406f270 0x7f5db406f680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34934/0 (socket says 172.21.15.86:34934) 2024-04-25T14:09:22.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.698+0000 7f5db9600640 1 -- 172.21.15.86:0/1869658548 learned_addr learned my addr 172.21.15.86:0/1869658548 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:22.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.698+0000 7f5db8c00640 1 --2- 172.21.15.86:0/1869658548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5db4070330 0x7f5db40d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.698+0000 7f5dabe00640 1 -- 172.21.15.86:0/1869658548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5db4070330 msgr2=0x7f5db40d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:22.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.698+0000 7f5dabe00640 1 --2- 172.21.15.86:0/1869658548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5db4070330 0x7f5db40d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.698+0000 7f5dabe00640 1 -- 172.21.15.86:0/1869658548 >> v1:172.21.15.86:6789/0 conn(0x7f5db406f270 legacy=0x7f5db406f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:22.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.698+0000 7f5db3e00640 1 -- 172.21.15.86:0/1869658548 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4197085668 0 0) 0x7f5dac0014a0 con 0x7f5db406f270 2024-04-25T14:09:22.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.698+0000 7f5dabe00640 1 -- 172.21.15.86:0/1869658548 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db415eb80 con 0x7f5db40d1c20 2024-04-25T14:09:22.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.702+0000 7f5dabe00640 1 --2- 172.21.15.86:0/1869658548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5db40d1c20 0x7f5db406ed30 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0097f0 tx=0x7f5d9c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8795e54665a9b080 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:22.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.702+0000 7f5db3e00640 1 -- 172.21.15.86:0/1869658548 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c0b0070 con 0x7f5db40d1c20 2024-04-25T14:09:22.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.702+0000 7f5db3e00640 1 -- 172.21.15.86:0/1869658548 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d9c0ab600 con 0x7f5db40d1c20 2024-04-25T14:09:22.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.702+0000 7f5db3e00640 1 -- 172.21.15.86:0/1869658548 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c0ab9d0 con 0x7f5db40d1c20 2024-04-25T14:09:22.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.702+0000 7f5dbb600640 1 -- 172.21.15.86:0/1869658548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5db40d1c20 msgr2=0x7f5db406ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:22.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.702+0000 7f5dbb600640 1 --2- 172.21.15.86:0/1869658548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5db40d1c20 0x7f5db406ed30 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0097f0 tx=0x7f5d9c01c410 comp rx=0 tx=0).stop 2024-04-25T14:09:22.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.702+0000 7f5dbb600640 1 -- 172.21.15.86:0/1869658548 shutdown_connections 2024-04-25T14:09:22.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.702+0000 7f5dbb600640 1 --2- 172.21.15.86:0/1869658548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5db40d1c20 0x7f5db406ed30 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.702+0000 7f5dbb600640 1 --2- 172.21.15.86:0/1869658548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5db4070330 0x7f5db40d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.702+0000 7f5dbb600640 1 -- 172.21.15.86:0/1869658548 >> 172.21.15.86:0/1869658548 conn(0x7f5db40b4650 msgr2=0x7f5db40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:22.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.702+0000 7f5dbb600640 1 -- 172.21.15.86:0/1869658548 shutdown_connections 2024-04-25T14:09:22.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.702+0000 7f5dbb600640 1 -- 172.21.15.86:0/1869658548 wait complete. 2024-04-25T14:09:22.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.706+0000 7f5dbb600640 1 Processor -- start 2024-04-25T14:09:22.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.706+0000 7f5dbb600640 1 -- start start 2024-04-25T14:09:22.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.706+0000 7f5dbb600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5db406f270 0x7f5db40c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.706+0000 7f5dbb600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5db4070330 0x7f5db40d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.706+0000 7f5dbb600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5db40d1c20 0x7f5db40c4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.706+0000 7f5dbb600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5db40708c0 con 0x7f5db4070330 2024-04-25T14:09:22.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.706+0000 7f5dbb600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5db4070740 con 0x7f5db40d1c20 2024-04-25T14:09:22.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.706+0000 7f5dbb600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5db416afc0 con 0x7f5db406f270 2024-04-25T14:09:22.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.706+0000 7f5dabe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5db4070330 0x7f5db40d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.706+0000 7f5dabe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5db4070330 0x7f5db40d02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33898/0 (socket says 172.21.15.86:33898) 2024-04-25T14:09:22.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.706+0000 7f5dabe00640 1 -- 172.21.15.86:0/2720863387 learned_addr learned my addr 172.21.15.86:0/2720863387 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:22.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.706+0000 7f5db8c00640 1 --2- 172.21.15.86:0/2720863387 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5db406f270 0x7f5db40c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.706+0000 7f5db9600640 1 --2- 172.21.15.86:0/2720863387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5db40d1c20 0x7f5db40c4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.706+0000 7f5dabe00640 1 -- 172.21.15.86:0/2720863387 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5db406f270 msgr2=0x7f5db40c64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:22.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.706+0000 7f5dabe00640 1 --2- 172.21.15.86:0/2720863387 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5db406f270 0x7f5db40c64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.706+0000 7f5dabe00640 1 -- 172.21.15.86:0/2720863387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5db40d1c20 msgr2=0x7f5db40c4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:22.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.706+0000 7f5dabe00640 1 --2- 172.21.15.86:0/2720863387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5db40d1c20 0x7f5db40c4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.706+0000 7f5dabe00640 1 -- 172.21.15.86:0/2720863387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db40c54f0 con 0x7f5db4070330 2024-04-25T14:09:22.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.710+0000 7f5dabe00640 1 --2- 172.21.15.86:0/2720863387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5db4070330 0x7f5db40d02c0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0097f0 tx=0x7f5d9c0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:22.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.710+0000 7f5db1600640 1 -- 172.21.15.86:0/2720863387 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c0ba020 con 0x7f5db4070330 2024-04-25T14:09:22.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.710+0000 7f5db1600640 1 -- 172.21.15.86:0/2720863387 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d9c0b4400 con 0x7f5db4070330 2024-04-25T14:09:22.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.710+0000 7f5dbb600640 1 -- 172.21.15.86:0/2720863387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db40c7e70 con 0x7f5db4070330 2024-04-25T14:09:22.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.710+0000 7f5db1600640 1 -- 172.21.15.86:0/2720863387 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c0b0040 con 0x7f5db4070330 2024-04-25T14:09:22.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.710+0000 7f5dbb600640 1 -- 172.21.15.86:0/2720863387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db40d0800 con 0x7f5db4070330 2024-04-25T14:09:22.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.710+0000 7f5dbb600640 1 -- 172.21.15.86:0/2720863387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5db4069d00 con 0x7f5db4070330 2024-04-25T14:09:22.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.710+0000 7f5db1600640 1 -- 172.21.15.86:0/2720863387 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7f5d9c0bd020 con 0x7f5db4070330 2024-04-25T14:09:22.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.710+0000 7f5db1600640 1 --2- 172.21.15.86:0/2720863387 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f5d9006f550 0x7f5d90071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.710+0000 7f5db1600640 1 -- 172.21.15.86:0/2720863387 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5249+0+0 (secure 0 0 0) 0x7f5d9c129330 con 0x7f5db4070330 2024-04-25T14:09:22.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.710+0000 7f5db8c00640 1 --2- 172.21.15.86:0/2720863387 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f5d9006f550 0x7f5d90071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.714+0000 7f5db8c00640 1 --2- 172.21.15.86:0/2720863387 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f5d9006f550 0x7f5d90071a60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5db4072200 tx=0x7f5da4008040 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:22.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.718+0000 7f5db1600640 1 -- 172.21.15.86:0/2720863387 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5d9c0c4030 con 0x7f5db4070330 2024-04-25T14:09:22.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.882+0000 7f5dbb600640 1 -- 172.21.15.86:0/2720863387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5db4076bf0 con 0x7f5db4070330 2024-04-25T14:09:22.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.882+0000 7f5db1600640 1 -- 172.21.15.86:0/2720863387 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7f5d9c016090 con 0x7f5db4070330 2024-04-25T14:09:22.878 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:09:22.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.882+0000 7f5dbb600640 1 -- 172.21.15.86:0/2720863387 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f5d9006f550 msgr2=0x7f5d90071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:22.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.882+0000 7f5dbb600640 1 --2- 172.21.15.86:0/2720863387 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f5d9006f550 0x7f5d90071a60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5db4072200 tx=0x7f5da4008040 comp rx=0 tx=0).stop 2024-04-25T14:09:22.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.882+0000 7f5dbb600640 1 -- 172.21.15.86:0/2720863387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5db4070330 msgr2=0x7f5db40d02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:22.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.882+0000 7f5dbb600640 1 --2- 172.21.15.86:0/2720863387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5db4070330 0x7f5db40d02c0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0097f0 tx=0x7f5d9c0191b0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.882+0000 7f5dbb600640 1 -- 172.21.15.86:0/2720863387 shutdown_connections 2024-04-25T14:09:22.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.882+0000 7f5dbb600640 1 --2- 172.21.15.86:0/2720863387 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f5d9006f550 0x7f5d90071a60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.882+0000 7f5dbb600640 1 --2- 172.21.15.86:0/2720863387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5db40d1c20 0x7f5db40c4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.882+0000 7f5dbb600640 1 --2- 172.21.15.86:0/2720863387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5db4070330 0x7f5db40d02c0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.882+0000 7f5dbb600640 1 --2- 172.21.15.86:0/2720863387 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5db406f270 0x7f5db40c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.882+0000 7f5dbb600640 1 -- 172.21.15.86:0/2720863387 >> 172.21.15.86:0/2720863387 conn(0x7f5db40b4650 msgr2=0x7f5db40c6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:22.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.882+0000 7f5dbb600640 1 -- 172.21.15.86:0/2720863387 shutdown_connections 2024-04-25T14:09:22.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:22.882+0000 7f5dbb600640 1 -- 172.21.15.86:0/2720863387 wait complete. 2024-04-25T14:09:22.900 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T14:09:23.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.050+0000 7f0728000640 1 Processor -- start 2024-04-25T14:09:23.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.050+0000 7f0728000640 1 -- start start 2024-04-25T14:09:23.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.050+0000 7f0728000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0720071530 0x7f072006e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:23.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.050+0000 7f0728000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f072006e7a0 0x7f07200cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:23.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.050+0000 7f0728000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f072006ef90 con 0x7f07200cf970 2024-04-25T14:09:23.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.050+0000 7f0728000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0720163830 con 0x7f072006e7a0 2024-04-25T14:09:23.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.050+0000 7f0728000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0720163a80 con 0x7f0720071530 2024-04-25T14:09:23.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.050+0000 7f0725600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f07200cf970 0x7f0720070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34946/0 (socket says 172.21.15.86:34946) 2024-04-25T14:09:23.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.050+0000 7f0725600640 1 -- 172.21.15.86:0/3981731348 learned_addr learned my addr 172.21.15.86:0/3981731348 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:23.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.050+0000 7f0726000640 1 --2- 172.21.15.86:0/3981731348 >> v2:172.21.15.155:3300/0 conn(0x7f072006e7a0 0x7f07200cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:23.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.050+0000 7f0724c00640 1 --2- 172.21.15.86:0/3981731348 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0720071530 0x7f072006e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:23.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.050+0000 7f071fe00640 1 -- 172.21.15.86:0/3981731348 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3685202943 0 0) 0x7f072006ef90 con 0x7f07200cf970 2024-04-25T14:09:23.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.050+0000 7f071fe00640 1 -- 172.21.15.86:0/3981731348 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0708003530 con 0x7f07200cf970 2024-04-25T14:09:23.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.050+0000 7f071fe00640 1 -- 172.21.15.86:0/3981731348 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1475685090 0 0) 0x7f0708003530 con 0x7f07200cf970 2024-04-25T14:09:23.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.050+0000 7f071fe00640 1 -- 172.21.15.86:0/3981731348 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0720071530 msgr2=0x7f072006e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:23.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.050+0000 7f071fe00640 1 --2- 172.21.15.86:0/3981731348 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0720071530 0x7f072006e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:23.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.050+0000 7f071fe00640 1 -- 172.21.15.86:0/3981731348 >> v2:172.21.15.155:3300/0 conn(0x7f072006e7a0 msgr2=0x7f07200cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:23.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.050+0000 7f071fe00640 1 --2- 172.21.15.86:0/3981731348 >> v2:172.21.15.155:3300/0 conn(0x7f072006e7a0 0x7f07200cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:23.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.050+0000 7f071fe00640 1 -- 172.21.15.86:0/3981731348 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0720163d60 con 0x7f07200cf970 2024-04-25T14:09:23.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.050+0000 7f071fe00640 1 -- 172.21.15.86:0/3981731348 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f0714001680 con 0x7f07200cf970 2024-04-25T14:09:23.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.050+0000 7f071fe00640 1 -- 172.21.15.86:0/3981731348 >> v1:172.21.15.86:6789/0 conn(0x7f07200cf970 legacy=0x7f0720070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:23.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.054+0000 7f071fe00640 1 --2- 172.21.15.86:0/3981731348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0708003a30 0x7f0708003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:23.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.054+0000 7f071fe00640 1 --2- 172.21.15.86:0/3981731348 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0708004810 0x7f0708004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:23.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.054+0000 7f071fe00640 1 --2- 172.21.15.86:0/3981731348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0708005160 0x7f070800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:23.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.054+0000 7f071fe00640 1 -- 172.21.15.86:0/3981731348 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0720163830 con 0x7f0708005160 2024-04-25T14:09:23.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.054+0000 7f071fe00640 1 -- 172.21.15.86:0/3981731348 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0720163a80 con 0x7f0708003a30 2024-04-25T14:09:23.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.054+0000 7f071fe00640 1 -- 172.21.15.86:0/3981731348 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f070800bf10 con 0x7f0708004810 2024-04-25T14:09:23.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.054+0000 7f0726000640 1 --2- 172.21.15.86:0/3981731348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0708005160 0x7f070800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:23.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.054+0000 7f0724c00640 1 --2- 172.21.15.86:0/3981731348 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0708004810 0x7f0708004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:23.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.054+0000 7f0725600640 1 --2- 172.21.15.86:0/3981731348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0708003a30 0x7f0708003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:23.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.054+0000 7f0726000640 1 -- 172.21.15.86:0/3981731348 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0708004810 msgr2=0x7f0708004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:23.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.054+0000 7f0726000640 1 --2- 172.21.15.86:0/3981731348 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0708004810 0x7f0708004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:23.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.054+0000 7f0726000640 1 -- 172.21.15.86:0/3981731348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0708003a30 msgr2=0x7f0708003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:23.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.054+0000 7f0726000640 1 --2- 172.21.15.86:0/3981731348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0708003a30 0x7f0708003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:23.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.054+0000 7f0726000640 1 -- 172.21.15.86:0/3981731348 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0708003530 con 0x7f0708005160 2024-04-25T14:09:23.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.054+0000 7f0726000640 1 --2- 172.21.15.86:0/3981731348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0708005160 0x7f070800b9d0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f0720074b10 tx=0x7f071801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606ff6c52ba5cf1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:23.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.054+0000 7f071fe00640 1 -- 172.21.15.86:0/3981731348 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07180b2070 con 0x7f0708005160 2024-04-25T14:09:23.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.054+0000 7f071fe00640 1 -- 172.21.15.86:0/3981731348 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07180a5c70 con 0x7f0708005160 2024-04-25T14:09:23.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.054+0000 7f071fe00640 1 -- 172.21.15.86:0/3981731348 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0718009c10 con 0x7f0708005160 2024-04-25T14:09:23.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.058+0000 7f0728000640 1 -- 172.21.15.86:0/3981731348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0708005160 msgr2=0x7f070800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:23.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.058+0000 7f0728000640 1 --2- 172.21.15.86:0/3981731348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0708005160 0x7f070800b9d0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f0720074b10 tx=0x7f071801cc60 comp rx=0 tx=0).stop 2024-04-25T14:09:23.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.058+0000 7f0726000640 1 -- 172.21.15.86:0/3981731348 reap_dead start 2024-04-25T14:09:23.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.058+0000 7f0728000640 1 -- 172.21.15.86:0/3981731348 shutdown_connections 2024-04-25T14:09:23.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.058+0000 7f0728000640 1 -- 172.21.15.86:0/3981731348 >> 172.21.15.86:0/3981731348 conn(0x7f07200cffc0 msgr2=0x7f07200d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:23.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.058+0000 7f0728000640 1 -- 172.21.15.86:0/3981731348 shutdown_connections 2024-04-25T14:09:23.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.058+0000 7f0728000640 1 -- 172.21.15.86:0/3981731348 wait complete. 2024-04-25T14:09:23.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.058+0000 7f0728000640 1 Processor -- start 2024-04-25T14:09:23.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.058+0000 7f0728000640 1 -- start start 2024-04-25T14:09:23.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.062+0000 7f0728000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07200cb640 0x7f0720149630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:23.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.062+0000 7f0728000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0720147cb0 0x7f0720160540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:23.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.062+0000 7f0728000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07201480c0 0x7f0720160a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:23.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.062+0000 7f0728000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f072006ed30 con 0x7f07201480c0 2024-04-25T14:09:23.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.062+0000 7f0728000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f072006ebb0 con 0x7f07200cb640 2024-04-25T14:09:23.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.062+0000 7f0728000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0720071530 con 0x7f0720147cb0 2024-04-25T14:09:23.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.062+0000 7f0726000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07201480c0 0x7f0720160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:23.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.062+0000 7f0726000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07201480c0 0x7f0720160a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33918/0 (socket says 172.21.15.86:33918) 2024-04-25T14:09:23.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.062+0000 7f0726000640 1 -- 172.21.15.86:0/3887732920 learned_addr learned my addr 172.21.15.86:0/3887732920 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:23.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.062+0000 7f0725600640 1 --2- 172.21.15.86:0/3887732920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07200cb640 0x7f0720149630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:23.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.062+0000 7f0724c00640 1 --2- 172.21.15.86:0/3887732920 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0720147cb0 0x7f0720160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:23.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.062+0000 7f0726000640 1 -- 172.21.15.86:0/3887732920 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0720147cb0 msgr2=0x7f0720160540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:23.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.062+0000 7f0726000640 1 --2- 172.21.15.86:0/3887732920 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0720147cb0 0x7f0720160540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:23.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.062+0000 7f0726000640 1 -- 172.21.15.86:0/3887732920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07200cb640 msgr2=0x7f0720149630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:23.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.062+0000 7f0726000640 1 --2- 172.21.15.86:0/3887732920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07200cb640 0x7f0720149630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:23.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.062+0000 7f0726000640 1 -- 172.21.15.86:0/3887732920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0720148590 con 0x7f07201480c0 2024-04-25T14:09:23.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.062+0000 7f0726000640 1 --2- 172.21.15.86:0/3887732920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07201480c0 0x7f0720160a80 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f0718002240 tx=0x7f0718009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:23.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.062+0000 7f071d600640 1 -- 172.21.15.86:0/3887732920 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07180b2040 con 0x7f07201480c0 2024-04-25T14:09:23.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.062+0000 7f071d600640 1 -- 172.21.15.86:0/3887732920 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07180adb50 con 0x7f07201480c0 2024-04-25T14:09:23.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.062+0000 7f0728000640 1 -- 172.21.15.86:0/3887732920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0720148870 con 0x7f07201480c0 2024-04-25T14:09:23.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.062+0000 7f071d600640 1 -- 172.21.15.86:0/3887732920 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0718009c10 con 0x7f07201480c0 2024-04-25T14:09:23.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.062+0000 7f0728000640 1 -- 172.21.15.86:0/3887732920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0720148e30 con 0x7f07201480c0 2024-04-25T14:09:23.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.062+0000 7f0728000640 1 -- 172.21.15.86:0/3887732920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07200720e0 con 0x7f07201480c0 2024-04-25T14:09:23.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.066+0000 7f071d600640 1 -- 172.21.15.86:0/3887732920 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7f07180bf060 con 0x7f07201480c0 2024-04-25T14:09:23.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.066+0000 7f071d600640 1 --2- 172.21.15.86:0/3887732920 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f070006f520 0x7f0700071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:23.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.066+0000 7f071d600640 1 -- 172.21.15.86:0/3887732920 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5249+0+0 (secure 0 0 0) 0x7f0718129120 con 0x7f07201480c0 2024-04-25T14:09:23.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.066+0000 7f0725600640 1 --2- 172.21.15.86:0/3887732920 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f070006f520 0x7f0700071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:23.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.066+0000 7f0725600640 1 --2- 172.21.15.86:0/3887732920 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f070006f520 0x7f0700071a30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0714000c10 tx=0x7f0714007900 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:23.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.070+0000 7f071d600640 1 -- 172.21.15.86:0/3887732920 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f07180a5e20 con 0x7f07201480c0 2024-04-25T14:09:23.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.226+0000 7f0728000640 1 -- 172.21.15.86:0/3887732920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f0720076bf0 con 0x7f07201480c0 2024-04-25T14:09:23.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.226+0000 7f071d600640 1 -- 172.21.15.86:0/3887732920 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v96) ==== 84+0+65 (secure 0 0 0) 0x7f0718016970 con 0x7f07201480c0 2024-04-25T14:09:23.224 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:23.224 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-04-25T14:09:23.225 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:09:23.225 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-04-25T14:09:23.225 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-04-25T14:09:23.225 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:09:23.225 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-04-25T14:09:23.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.230+0000 7f0728000640 1 -- 172.21.15.86:0/3887732920 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f070006f520 msgr2=0x7f0700071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:23.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.230+0000 7f0728000640 1 --2- 172.21.15.86:0/3887732920 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f070006f520 0x7f0700071a30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0714000c10 tx=0x7f0714007900 comp rx=0 tx=0).stop 2024-04-25T14:09:23.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.230+0000 7f0728000640 1 -- 172.21.15.86:0/3887732920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07201480c0 msgr2=0x7f0720160a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:23.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.230+0000 7f0728000640 1 --2- 172.21.15.86:0/3887732920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07201480c0 0x7f0720160a80 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f0718002240 tx=0x7f0718009750 comp rx=0 tx=0).stop 2024-04-25T14:09:23.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.230+0000 7f0728000640 1 -- 172.21.15.86:0/3887732920 shutdown_connections 2024-04-25T14:09:23.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.230+0000 7f0728000640 1 --2- 172.21.15.86:0/3887732920 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f070006f520 0x7f0700071a30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:23.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.230+0000 7f0728000640 1 --2- 172.21.15.86:0/3887732920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07201480c0 0x7f0720160a80 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:23.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.230+0000 7f0728000640 1 --2- 172.21.15.86:0/3887732920 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0720147cb0 0x7f0720160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:23.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.230+0000 7f0728000640 1 --2- 172.21.15.86:0/3887732920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07200cb640 0x7f0720149630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:23.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.230+0000 7f0728000640 1 -- 172.21.15.86:0/3887732920 >> 172.21.15.86:0/3887732920 conn(0x7f07200cffc0 msgr2=0x7f07200d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:23.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.230+0000 7f0728000640 1 -- 172.21.15.86:0/3887732920 shutdown_connections 2024-04-25T14:09:23.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.230+0000 7f0728000640 1 -- 172.21.15.86:0/3887732920 wait complete. 2024-04-25T14:09:23.246 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T14:09:23.246 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T14:09:23.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.398+0000 7f29aec00640 1 Processor -- start 2024-04-25T14:09:23.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.398+0000 7f29aec00640 1 -- start start 2024-04-25T14:09:23.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.402+0000 7f29aec00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f29a80d1a90 0x7f29a815b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:23.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.402+0000 7f29aec00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29a80d2360 0x7f29a80d2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:23.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.402+0000 7f29aec00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f29a815e120 0x7f29a815e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:23.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.402+0000 7f29aec00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f29a806e3e0 con 0x7f29a80d1a90 2024-04-25T14:09:23.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.402+0000 7f29aec00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f29a80d2cb0 con 0x7f29a815e120 2024-04-25T14:09:23.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.402+0000 7f29aec00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f29a806e610 con 0x7f29a80d2360 2024-04-25T14:09:23.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.402+0000 7f29a7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f29a80d1a90 0x7f29a815b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:23.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.402+0000 7f29a7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f29a80d1a90 0x7f29a815b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33934/0 (socket says 172.21.15.86:33934) 2024-04-25T14:09:23.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.402+0000 7f29a7e00640 1 -- 172.21.15.86:0/1822465332 learned_addr learned my addr 172.21.15.86:0/1822465332 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:23.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.402+0000 7f29a7400640 1 --2- 172.21.15.86:0/1822465332 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29a80d2360 0x7f29a80d2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:23.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.402+0000 7f29acc00640 1 --2- 172.21.15.86:0/1822465332 >> v2:172.21.15.155:3300/0 conn(0x7f29a815e120 0x7f29a815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:23.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.402+0000 7f29a7e00640 1 -- 172.21.15.86:0/1822465332 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29a80d2360 msgr2=0x7f29a80d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:23.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.402+0000 7f29a7e00640 1 --2- 172.21.15.86:0/1822465332 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29a80d2360 0x7f29a80d2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:23.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.402+0000 7f29a7e00640 1 -- 172.21.15.86:0/1822465332 >> v2:172.21.15.155:3300/0 conn(0x7f29a815e120 msgr2=0x7f29a815e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:23.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.402+0000 7f29a7e00640 1 --2- 172.21.15.86:0/1822465332 >> v2:172.21.15.155:3300/0 conn(0x7f29a815e120 0x7f29a815e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:23.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.402+0000 7f29a7e00640 1 -- 172.21.15.86:0/1822465332 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a806f0b0 con 0x7f29a80d1a90 2024-04-25T14:09:23.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.406+0000 7f29a7e00640 1 --2- 172.21.15.86:0/1822465332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a80d1a90 0x7f29a815b7d0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f29a80d1830 tx=0x7f299801c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a472119fe8006c79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:23.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.406+0000 7f29a6a00640 1 -- 172.21.15.86:0/1822465332 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29980a5800 con 0x7f29a80d1a90 2024-04-25T14:09:23.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.406+0000 7f29a6a00640 1 -- 172.21.15.86:0/1822465332 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29980a59a0 con 0x7f29a80d1a90 2024-04-25T14:09:23.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.406+0000 7f29a6a00640 1 -- 172.21.15.86:0/1822465332 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29980a5d70 con 0x7f29a80d1a90 2024-04-25T14:09:23.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.406+0000 7f29aec00640 1 -- 172.21.15.86:0/1822465332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a80d1a90 msgr2=0x7f29a815b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:23.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.406+0000 7f29aec00640 1 --2- 172.21.15.86:0/1822465332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a80d1a90 0x7f29a815b7d0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f29a80d1830 tx=0x7f299801c980 comp rx=0 tx=0).stop 2024-04-25T14:09:23.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.406+0000 7f29aec00640 1 -- 172.21.15.86:0/1822465332 shutdown_connections 2024-04-25T14:09:23.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.406+0000 7f29aec00640 1 --2- 172.21.15.86:0/1822465332 >> v2:172.21.15.155:3300/0 conn(0x7f29a815e120 0x7f29a815e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:23.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.406+0000 7f29aec00640 1 --2- 172.21.15.86:0/1822465332 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29a80d2360 0x7f29a80d2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:23.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.406+0000 7f29aec00640 1 --2- 172.21.15.86:0/1822465332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a80d1a90 0x7f29a815b7d0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:23.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.406+0000 7f29aec00640 1 -- 172.21.15.86:0/1822465332 >> 172.21.15.86:0/1822465332 conn(0x7f29a8073d30 msgr2=0x7f29a80b77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:23.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.406+0000 7f29aec00640 1 -- 172.21.15.86:0/1822465332 shutdown_connections 2024-04-25T14:09:23.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.406+0000 7f29aec00640 1 -- 172.21.15.86:0/1822465332 wait complete. 2024-04-25T14:09:23.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.406+0000 7f29aec00640 1 Processor -- start 2024-04-25T14:09:23.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.406+0000 7f29aec00640 1 -- start start 2024-04-25T14:09:23.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.410+0000 7f29aec00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29a80d1a90 0x7f29a815d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:23.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.410+0000 7f29aec00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29a80d2360 0x7f29a80cf810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:23.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.410+0000 7f29aec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a815e120 0x7f29a814aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:23.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.410+0000 7f29aec00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29a80b1d20 con 0x7f29a815e120 2024-04-25T14:09:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.410+0000 7f29aec00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f29a80b1ba0 con 0x7f29a80d1a90 2024-04-25T14:09:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.410+0000 7f29aec00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f29a80b1430 con 0x7f29a80d2360 2024-04-25T14:09:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.410+0000 7f29acc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a815e120 0x7f29a814aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.410+0000 7f29acc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a815e120 0x7f29a814aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33950/0 (socket says 172.21.15.86:33950) 2024-04-25T14:09:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.410+0000 7f29acc00640 1 -- 172.21.15.86:0/1185988671 learned_addr learned my addr 172.21.15.86:0/1185988671 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.410+0000 7f29a7e00640 1 --2- 172.21.15.86:0/1185988671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29a80d1a90 0x7f29a815d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.410+0000 7f29a7400640 1 --2- 172.21.15.86:0/1185988671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29a80d2360 0x7f29a80cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.410+0000 7f29acc00640 1 -- 172.21.15.86:0/1185988671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29a80d2360 msgr2=0x7f29a80cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.410+0000 7f29acc00640 1 --2- 172.21.15.86:0/1185988671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29a80d2360 0x7f29a80cf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:23.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.410+0000 7f29acc00640 1 -- 172.21.15.86:0/1185988671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29a80d1a90 msgr2=0x7f29a815d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:23.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.410+0000 7f29acc00640 1 --2- 172.21.15.86:0/1185988671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29a80d1a90 0x7f29a815d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:23.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.410+0000 7f29acc00640 1 -- 172.21.15.86:0/1185988671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a80b33d0 con 0x7f29a815e120 2024-04-25T14:09:23.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.410+0000 7f29a7e00640 1 --2- 172.21.15.86:0/1185988671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29a80d1a90 0x7f29a815d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:23.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.410+0000 7f29a7400640 1 --2- 172.21.15.86:0/1185988671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29a80d2360 0x7f29a80cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:23.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.410+0000 7f29acc00640 1 --2- 172.21.15.86:0/1185988671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a815e120 0x7f29a814aa60 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f29a000e7f0 tx=0x7f29a000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:23.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.410+0000 7f299fe00640 1 -- 172.21.15.86:0/1185988671 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a0017070 con 0x7f29a815e120 2024-04-25T14:09:23.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.410+0000 7f299fe00640 1 -- 172.21.15.86:0/1185988671 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a000c940 con 0x7f29a815e120 2024-04-25T14:09:23.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.410+0000 7f299fe00640 1 -- 172.21.15.86:0/1185988671 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a000eea0 con 0x7f29a815e120 2024-04-25T14:09:23.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.410+0000 7f29aec00640 1 -- 172.21.15.86:0/1185988671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a80b36b0 con 0x7f29a815e120 2024-04-25T14:09:23.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.410+0000 7f29aec00640 1 -- 172.21.15.86:0/1185988671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a80b3c70 con 0x7f29a815e120 2024-04-25T14:09:23.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.410+0000 7f29aec00640 1 -- 172.21.15.86:0/1185988671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29a8050750 con 0x7f29a815e120 2024-04-25T14:09:23.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.414+0000 7f299fe00640 1 -- 172.21.15.86:0/1185988671 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7f29a001a070 con 0x7f29a815e120 2024-04-25T14:09:23.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.414+0000 7f299fe00640 1 --2- 172.21.15.86:0/1185988671 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f298806f520 0x7f2988071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:23.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.414+0000 7f299fe00640 1 -- 172.21.15.86:0/1185988671 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5249+0+0 (secure 0 0 0) 0x7f29a008f3d0 con 0x7f29a815e120 2024-04-25T14:09:23.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.414+0000 7f29a7e00640 1 --2- 172.21.15.86:0/1185988671 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f298806f520 0x7f2988071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:23.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.414+0000 7f29a7e00640 1 --2- 172.21.15.86:0/1185988671 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f298806f520 0x7f2988071a30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f29980ae000 tx=0x7f29980a7070 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:23.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.418+0000 7f299fe00640 1 -- 172.21.15.86:0/1185988671 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f29a005c370 con 0x7f29a815e120 2024-04-25T14:09:23.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.574+0000 7f29aec00640 1 -- 172.21.15.86:0/1185988671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f29a8075eb0 con 0x7f29a815e120 2024-04-25T14:09:23.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.786+0000 7f299fe00640 1 -- 172.21.15.86:0/1185988671 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v97) ==== 138+0+0 (secure 0 0 0) 0x7f29a005bc70 con 0x7f29a815e120 2024-04-25T14:09:23.784 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-04-25T14:09:23.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.790+0000 7f29aec00640 1 -- 172.21.15.86:0/1185988671 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f298806f520 msgr2=0x7f2988071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:23.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.790+0000 7f29aec00640 1 --2- 172.21.15.86:0/1185988671 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f298806f520 0x7f2988071a30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f29980ae000 tx=0x7f29980a7070 comp rx=0 tx=0).stop 2024-04-25T14:09:23.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.790+0000 7f29aec00640 1 -- 172.21.15.86:0/1185988671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a815e120 msgr2=0x7f29a814aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:23.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.790+0000 7f29aec00640 1 --2- 172.21.15.86:0/1185988671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a815e120 0x7f29a814aa60 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f29a000e7f0 tx=0x7f29a000ecb0 comp rx=0 tx=0).stop 2024-04-25T14:09:23.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.790+0000 7f29aec00640 1 -- 172.21.15.86:0/1185988671 shutdown_connections 2024-04-25T14:09:23.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.790+0000 7f29aec00640 1 --2- 172.21.15.86:0/1185988671 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f298806f520 0x7f2988071a30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:23.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.790+0000 7f29aec00640 1 --2- 172.21.15.86:0/1185988671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a815e120 0x7f29a814aa60 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:23.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.790+0000 7f29aec00640 1 --2- 172.21.15.86:0/1185988671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29a80d2360 0x7f29a80cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:23.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.790+0000 7f29aec00640 1 --2- 172.21.15.86:0/1185988671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29a80d1a90 0x7f29a815d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:23.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.790+0000 7f29aec00640 1 -- 172.21.15.86:0/1185988671 >> 172.21.15.86:0/1185988671 conn(0x7f29a8073d30 msgr2=0x7f29a8158e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:23.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.790+0000 7f29aec00640 1 -- 172.21.15.86:0/1185988671 shutdown_connections 2024-04-25T14:09:23.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.790+0000 7f29aec00640 1 -- 172.21.15.86:0/1185988671 wait complete. 2024-04-25T14:09:23.813 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T14:09:23.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.966+0000 7f4fa7200640 1 Processor -- start 2024-04-25T14:09:23.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.966+0000 7f4fa7200640 1 -- start start 2024-04-25T14:09:23.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.970+0000 7f4fa7200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4fa8163610 0x7f4fa8163a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:23.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.970+0000 7f4fa7200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fa80b08c0 0x7f4fa80b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:23.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.970+0000 7f4fa7200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fa8163f60 con 0x7f4fa80d0fc0 2024-04-25T14:09:23.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.970+0000 7f4fa7200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4fa80b6c20 con 0x7f4fa8163610 2024-04-25T14:09:23.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.970+0000 7f4fa7200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4fa80b6e40 con 0x7f4fa80b08c0 2024-04-25T14:09:23.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.970+0000 7f4f9d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4fa8163610 0x7f4fa8163a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:23.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.970+0000 7f4f9d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4fa8163610 0x7f4fa8163a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33956/0 (socket says 172.21.15.86:33956) 2024-04-25T14:09:23.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.970+0000 7f4f9d400640 1 -- 172.21.15.86:0/1877593414 learned_addr learned my addr 172.21.15.86:0/1877593414 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:23.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.970+0000 7f4fa6800640 1 --2- 172.21.15.86:0/1877593414 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fa80b08c0 0x7f4fa80b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:23.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.970+0000 7f4f9d400640 1 -- 172.21.15.86:0/1877593414 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fa80b08c0 msgr2=0x7f4fa80b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:23.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.970+0000 7f4f9d400640 1 --2- 172.21.15.86:0/1877593414 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fa80b08c0 0x7f4fa80b0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:23.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.970+0000 7f4f9d400640 1 -- 172.21.15.86:0/1877593414 >> v1:172.21.15.155:6789/0 conn(0x7f4fa80d0fc0 legacy=0x7f4fa80b6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:23.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.970+0000 7f4f9d400640 1 -- 172.21.15.86:0/1877593414 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa80b7120 con 0x7f4fa8163610 2024-04-25T14:09:23.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.970+0000 7f4fa6800640 1 --2- 172.21.15.86:0/1877593414 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fa80b08c0 0x7f4fa80b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:23.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.974+0000 7f4f9d400640 1 --2- 172.21.15.86:0/1877593414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fa8163610 0x7f4fa8163a20 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009870 tx=0x7f4f9001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a139272dd9944e37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:23.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.974+0000 7f4fa5400640 1 -- 172.21.15.86:0/1877593414 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f900ab5f0 con 0x7f4fa8163610 2024-04-25T14:09:23.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.974+0000 7f4fa5400640 1 -- 172.21.15.86:0/1877593414 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f900b0070 con 0x7f4fa8163610 2024-04-25T14:09:23.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.974+0000 7f4fa5400640 1 -- 172.21.15.86:0/1877593414 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f900ab9b0 con 0x7f4fa8163610 2024-04-25T14:09:23.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.974+0000 7f4fa7200640 1 -- 172.21.15.86:0/1877593414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fa8163610 msgr2=0x7f4fa8163a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:23.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.974+0000 7f4fa7200640 1 --2- 172.21.15.86:0/1877593414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fa8163610 0x7f4fa8163a20 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009870 tx=0x7f4f9001c520 comp rx=0 tx=0).stop 2024-04-25T14:09:23.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.974+0000 7f4fa7200640 1 -- 172.21.15.86:0/1877593414 shutdown_connections 2024-04-25T14:09:23.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.974+0000 7f4fa7200640 1 --2- 172.21.15.86:0/1877593414 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fa80b08c0 0x7f4fa80b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:23.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.974+0000 7f4fa7200640 1 --2- 172.21.15.86:0/1877593414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fa8163610 0x7f4fa8163a20 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:23.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.974+0000 7f4fa7200640 1 -- 172.21.15.86:0/1877593414 >> 172.21.15.86:0/1877593414 conn(0x7f4fa8073ab0 msgr2=0x7f4fa806c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:23.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.974+0000 7f4fa7200640 1 -- 172.21.15.86:0/1877593414 shutdown_connections 2024-04-25T14:09:23.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.974+0000 7f4fa7200640 1 -- 172.21.15.86:0/1877593414 wait complete. 2024-04-25T14:09:23.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.978+0000 7f4fa7200640 1 Processor -- start 2024-04-25T14:09:23.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.978+0000 7f4fa7200640 1 -- start start 2024-04-25T14:09:23.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.978+0000 7f4fa7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fa80b08c0 0x7f4fa8140e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:23.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.978+0000 7f4fa7200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fa80d0fc0 0x7f4fa8158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:23.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.978+0000 7f4fa7200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fa8163610 0x7f4fa8144700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:23.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.978+0000 7f4fa7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa806fdb0 con 0x7f4fa80b08c0 2024-04-25T14:09:23.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.978+0000 7f4fa7200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa80d15e0 con 0x7f4fa80d0fc0 2024-04-25T14:09:23.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.978+0000 7f4fa7200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4fa806ff30 con 0x7f4fa8163610 2024-04-25T14:09:23.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.978+0000 7f4fa5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fa80b08c0 0x7f4fa8140e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:23.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.978+0000 7f4fa5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fa80b08c0 0x7f4fa8140e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33958/0 (socket says 172.21.15.86:33958) 2024-04-25T14:09:23.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.978+0000 7f4fa5e00640 1 -- 172.21.15.86:0/2202312713 learned_addr learned my addr 172.21.15.86:0/2202312713 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:23.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.978+0000 7f4f9d400640 1 --2- 172.21.15.86:0/2202312713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fa80d0fc0 0x7f4fa8158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:23.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.978+0000 7f4fa6800640 1 --2- 172.21.15.86:0/2202312713 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fa8163610 0x7f4fa8144700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:23.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.978+0000 7f4fa5e00640 1 -- 172.21.15.86:0/2202312713 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fa8163610 msgr2=0x7f4fa8144700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:23.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.978+0000 7f4fa5e00640 1 --2- 172.21.15.86:0/2202312713 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fa8163610 0x7f4fa8144700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:23.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.978+0000 7f4fa5e00640 1 -- 172.21.15.86:0/2202312713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fa80d0fc0 msgr2=0x7f4fa8158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:23.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.978+0000 7f4fa5e00640 1 --2- 172.21.15.86:0/2202312713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fa80d0fc0 0x7f4fa8158ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:23.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.978+0000 7f4fa5e00640 1 -- 172.21.15.86:0/2202312713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa8141b90 con 0x7f4fa80b08c0 2024-04-25T14:09:23.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.978+0000 7f4f9d400640 1 --2- 172.21.15.86:0/2202312713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fa80d0fc0 0x7f4fa8158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:23.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.978+0000 7f4fa5e00640 1 --2- 172.21.15.86:0/2202312713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fa80b08c0 0x7f4fa8140e10 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800b3e0 tx=0x7f4f9800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:23.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.978+0000 7f4f9ea00640 1 -- 172.21.15.86:0/2202312713 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98015020 con 0x7f4fa80b08c0 2024-04-25T14:09:23.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.978+0000 7f4f9ea00640 1 -- 172.21.15.86:0/2202312713 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f98011830 con 0x7f4fa80b08c0 2024-04-25T14:09:23.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.978+0000 7f4f9ea00640 1 -- 172.21.15.86:0/2202312713 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98011c60 con 0x7f4fa80b08c0 2024-04-25T14:09:23.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.978+0000 7f4fa7200640 1 -- 172.21.15.86:0/2202312713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa8141e70 con 0x7f4fa80b08c0 2024-04-25T14:09:23.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.978+0000 7f4fa7200640 1 -- 172.21.15.86:0/2202312713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa8142430 con 0x7f4fa80b08c0 2024-04-25T14:09:23.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.982+0000 7f4fa7200640 1 -- 172.21.15.86:0/2202312713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fa8076b80 con 0x7f4fa80b08c0 2024-04-25T14:09:23.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.982+0000 7f4f9ea00640 1 -- 172.21.15.86:0/2202312713 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7f4f98018020 con 0x7f4fa80b08c0 2024-04-25T14:09:23.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.982+0000 7f4f9ea00640 1 --2- 172.21.15.86:0/2202312713 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f4f8406f550 0x7f4f84071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:23.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.982+0000 7f4f9ea00640 1 -- 172.21.15.86:0/2202312713 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5619+0+0 (secure 0 0 0) 0x7f4f9808f490 con 0x7f4fa80b08c0 2024-04-25T14:09:23.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.982+0000 7f4f9d400640 1 --2- 172.21.15.86:0/2202312713 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f4f8406f550 0x7f4f84071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:23.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.986+0000 7f4f9d400640 1 --2- 172.21.15.86:0/2202312713 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f4f8406f550 0x7f4f84071a60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4f900af000 tx=0x7f4f900a7000 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:23.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:23.986+0000 7f4f9ea00640 1 -- 172.21.15.86:0/2202312713 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4f9801f030 con 0x7f4fa80b08c0 2024-04-25T14:09:24.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.146+0000 7f4fa7200640 1 -- 172.21.15.86:0/2202312713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f4fa8070b00 con 0x7f4fa80b08c0 2024-04-25T14:09:24.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.790+0000 7f4f9ea00640 1 -- 172.21.15.86:0/2202312713 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v98) ==== 170+0+0 (secure 0 0 0) 0x7f4f9805c3b0 con 0x7f4fa80b08c0 2024-04-25T14:09:24.787 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-04-25T14:09:24.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.794+0000 7f4fa7200640 1 -- 172.21.15.86:0/2202312713 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f4f8406f550 msgr2=0x7f4f84071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:24.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.794+0000 7f4fa7200640 1 --2- 172.21.15.86:0/2202312713 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f4f8406f550 0x7f4f84071a60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4f900af000 tx=0x7f4f900a7000 comp rx=0 tx=0).stop 2024-04-25T14:09:24.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.794+0000 7f4fa7200640 1 -- 172.21.15.86:0/2202312713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fa80b08c0 msgr2=0x7f4fa8140e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:24.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.794+0000 7f4fa7200640 1 --2- 172.21.15.86:0/2202312713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fa80b08c0 0x7f4fa8140e10 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800b3e0 tx=0x7f4f9800b8a0 comp rx=0 tx=0).stop 2024-04-25T14:09:24.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.798+0000 7f4fa7200640 1 -- 172.21.15.86:0/2202312713 shutdown_connections 2024-04-25T14:09:24.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.798+0000 7f4fa7200640 1 --2- 172.21.15.86:0/2202312713 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f4f8406f550 0x7f4f84071a60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:24.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.798+0000 7f4fa7200640 1 --2- 172.21.15.86:0/2202312713 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fa8163610 0x7f4fa8144700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:24.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.798+0000 7f4fa7200640 1 --2- 172.21.15.86:0/2202312713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fa80d0fc0 0x7f4fa8158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:24.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.798+0000 7f4fa7200640 1 --2- 172.21.15.86:0/2202312713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fa80b08c0 0x7f4fa8140e10 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:24.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.798+0000 7f4fa7200640 1 -- 172.21.15.86:0/2202312713 >> 172.21.15.86:0/2202312713 conn(0x7f4fa8073ab0 msgr2=0x7f4fa80b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:24.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.802+0000 7f4fa7200640 1 -- 172.21.15.86:0/2202312713 shutdown_connections 2024-04-25T14:09:24.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.802+0000 7f4fa7200640 1 -- 172.21.15.86:0/2202312713 wait complete. 2024-04-25T14:09:24.834 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T14:09:24.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.986+0000 7f605e400640 1 Processor -- start 2024-04-25T14:09:24.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.986+0000 7f605e400640 1 -- start start 2024-04-25T14:09:24.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.986+0000 7f605e400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f60580cde90 0x7f60580ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:24.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.986+0000 7f605e400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f605815e5c0 con 0x7f6058070330 2024-04-25T14:09:24.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.986+0000 7f605e400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f605815e8a0 con 0x7f60580d1df0 2024-04-25T14:09:24.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.986+0000 7f605e400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f605806fd40 con 0x7f60580cde90 2024-04-25T14:09:24.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.986+0000 7f6057400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6058070330 0x7f60580d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34956/0 (socket says 172.21.15.86:34956) 2024-04-25T14:09:24.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.986+0000 7f6057400640 1 -- 172.21.15.86:0/1272350489 learned_addr learned my addr 172.21.15.86:0/1272350489 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.986+0000 7f6057e00640 1 --2- 172.21.15.86:0/1272350489 >> v2:172.21.15.155:3300/0 conn(0x7f60580cde90 0x7f60580ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.986+0000 7f6056a00640 1 -- 172.21.15.86:0/1272350489 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2364009415 0 0) 0x7f605815e5c0 con 0x7f6058070330 2024-04-25T14:09:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.986+0000 7f6056a00640 1 -- 172.21.15.86:0/1272350489 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f603c003530 con 0x7f6058070330 2024-04-25T14:09:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.986+0000 7f6056a00640 1 -- 172.21.15.86:0/1272350489 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3792721270 0 0) 0x7f605815e8a0 con 0x7f60580d1df0 2024-04-25T14:09:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.986+0000 7f6056a00640 1 -- 172.21.15.86:0/1272350489 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f605815e5c0 con 0x7f60580d1df0 2024-04-25T14:09:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.986+0000 7f6056a00640 1 -- 172.21.15.86:0/1272350489 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 123048961 0 0) 0x7f603c003530 con 0x7f6058070330 2024-04-25T14:09:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.986+0000 7f6056a00640 1 -- 172.21.15.86:0/1272350489 >> v2:172.21.15.155:3300/0 conn(0x7f60580cde90 msgr2=0x7f60580ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.986+0000 7f6056a00640 1 --2- 172.21.15.86:0/1272350489 >> v2:172.21.15.155:3300/0 conn(0x7f60580cde90 0x7f60580ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.986+0000 7f6056a00640 1 -- 172.21.15.86:0/1272350489 >> v1:172.21.15.155:6789/0 conn(0x7f60580d1df0 legacy=0x7f60580cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.986+0000 7f6056a00640 1 -- 172.21.15.86:0/1272350489 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605815eb80 con 0x7f6058070330 2024-04-25T14:09:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.986+0000 7f6056a00640 1 -- 172.21.15.86:0/1272350489 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f6048001680 con 0x7f6058070330 2024-04-25T14:09:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.986+0000 7f6056a00640 1 -- 172.21.15.86:0/1272350489 >> v1:172.21.15.86:6789/0 conn(0x7f6058070330 legacy=0x7f60580d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.986+0000 7f6056a00640 1 --2- 172.21.15.86:0/1272350489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f603c003bf0 0x7f603c004040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:24.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.986+0000 7f6056a00640 1 --2- 172.21.15.86:0/1272350489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f603c004810 0x7f603c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:24.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.986+0000 7f6056a00640 1 --2- 172.21.15.86:0/1272350489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603c0052e0 0x7f603c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:24.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.986+0000 7f6056a00640 1 -- 172.21.15.86:0/1272350489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f603c00c240 con 0x7f603c0052e0 2024-04-25T14:09:24.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.986+0000 7f6056a00640 1 -- 172.21.15.86:0/1272350489 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f603c00c3f0 con 0x7f603c004810 2024-04-25T14:09:24.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.986+0000 7f6056a00640 1 -- 172.21.15.86:0/1272350489 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f603c00c5f0 con 0x7f603c003bf0 2024-04-25T14:09:24.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.990+0000 7f6057e00640 1 --2- 172.21.15.86:0/1272350489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603c0052e0 0x7f603c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:24.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.990+0000 7f604ea00640 1 --2- 172.21.15.86:0/1272350489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f603c004810 0x7f603c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:24.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.990+0000 7f6057400640 1 --2- 172.21.15.86:0/1272350489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f603c003bf0 0x7f603c004040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:24.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.990+0000 7f6057e00640 1 -- 172.21.15.86:0/1272350489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f603c003bf0 msgr2=0x7f603c004040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:24.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.990+0000 7f6057e00640 1 --2- 172.21.15.86:0/1272350489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f603c003bf0 0x7f603c004040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:24.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.990+0000 7f6057e00640 1 -- 172.21.15.86:0/1272350489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f603c004810 msgr2=0x7f603c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:24.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.990+0000 7f6057e00640 1 --2- 172.21.15.86:0/1272350489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f603c004810 0x7f603c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:24.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.990+0000 7f6057e00640 1 -- 172.21.15.86:0/1272350489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603c003530 con 0x7f603c0052e0 2024-04-25T14:09:24.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.990+0000 7f6057e00640 1 -- 172.21.15.86:0/1272350489 reap_dead start 2024-04-25T14:09:24.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.990+0000 7f6057e00640 1 --2- 172.21.15.86:0/1272350489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603c0052e0 0x7f603c00b9d0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f6050009f90 tx=0x7f605001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a43ffe1e39b071c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:24.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.990+0000 7f6056a00640 1 -- 172.21.15.86:0/1272350489 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6050009c10 con 0x7f603c0052e0 2024-04-25T14:09:24.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.990+0000 7f6056a00640 1 -- 172.21.15.86:0/1272350489 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60500a5c90 con 0x7f603c0052e0 2024-04-25T14:09:24.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.990+0000 7f6056a00640 1 -- 172.21.15.86:0/1272350489 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6050003be0 con 0x7f603c0052e0 2024-04-25T14:09:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.994+0000 7f605e400640 1 -- 172.21.15.86:0/1272350489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603c0052e0 msgr2=0x7f603c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.994+0000 7f605e400640 1 --2- 172.21.15.86:0/1272350489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603c0052e0 0x7f603c00b9d0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f6050009f90 tx=0x7f605001cc80 comp rx=0 tx=0).stop 2024-04-25T14:09:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.994+0000 7f605e400640 1 -- 172.21.15.86:0/1272350489 shutdown_connections 2024-04-25T14:09:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.994+0000 7f605e400640 1 --2- 172.21.15.86:0/1272350489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603c0052e0 0x7f603c00b9d0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.994+0000 7f605e400640 1 -- 172.21.15.86:0/1272350489 >> 172.21.15.86:0/1272350489 conn(0x7f60580b4650 msgr2=0x7f60580b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:24.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.994+0000 7f605e400640 1 -- 172.21.15.86:0/1272350489 shutdown_connections 2024-04-25T14:09:24.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.994+0000 7f605e400640 1 -- 172.21.15.86:0/1272350489 wait complete. 2024-04-25T14:09:24.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.994+0000 7f605e400640 1 Processor -- start 2024-04-25T14:09:24.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.994+0000 7f605e400640 1 -- start start 2024-04-25T14:09:24.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.998+0000 7f605e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603c0052e0 0x7f60581f8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:24.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.998+0000 7f605e400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60580cf8c0 0x7f6058159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:24.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.998+0000 7f605e400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f605815a140 0x7f60580c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:24.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.998+0000 7f605e400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f605815f330 con 0x7f603c0052e0 2024-04-25T14:09:24.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.998+0000 7f605e400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f605806d170 con 0x7f60580cf8c0 2024-04-25T14:09:24.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.998+0000 7f605e400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f605815f4b0 con 0x7f605815a140 2024-04-25T14:09:24.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.998+0000 7f6057400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603c0052e0 0x7f60581f8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:24.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.998+0000 7f6057400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603c0052e0 0x7f60581f8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33988/0 (socket says 172.21.15.86:33988) 2024-04-25T14:09:24.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.998+0000 7f6057400640 1 -- 172.21.15.86:0/3976652773 learned_addr learned my addr 172.21.15.86:0/3976652773 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:24.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.998+0000 7f604ea00640 1 --2- 172.21.15.86:0/3976652773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60580cf8c0 0x7f6058159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:24.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.998+0000 7f6057e00640 1 --2- 172.21.15.86:0/3976652773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f605815a140 0x7f60580c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:24.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.998+0000 7f6057400640 1 -- 172.21.15.86:0/3976652773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f605815a140 msgr2=0x7f60580c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:24.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.998+0000 7f6057400640 1 --2- 172.21.15.86:0/3976652773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f605815a140 0x7f60580c9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:24.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.998+0000 7f6057400640 1 -- 172.21.15.86:0/3976652773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60580cf8c0 msgr2=0x7f6058159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:24.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.998+0000 7f6057400640 1 --2- 172.21.15.86:0/3976652773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60580cf8c0 0x7f6058159c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:24.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.998+0000 7f6057400640 1 -- 172.21.15.86:0/3976652773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605815a550 con 0x7f603c0052e0 2024-04-25T14:09:24.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.998+0000 7f6057400640 1 --2- 172.21.15.86:0/3976652773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603c0052e0 0x7f60581f8c40 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f6048004e40 tx=0x7f6048007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:24.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.998+0000 7f604fe00640 1 -- 172.21.15.86:0/3976652773 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604800f7b0 con 0x7f603c0052e0 2024-04-25T14:09:24.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.998+0000 7f604fe00640 1 -- 172.21.15.86:0/3976652773 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6048018070 con 0x7f603c0052e0 2024-04-25T14:09:24.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.998+0000 7f605e400640 1 -- 172.21.15.86:0/3976652773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60580cf630 con 0x7f603c0052e0 2024-04-25T14:09:24.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.998+0000 7f604fe00640 1 -- 172.21.15.86:0/3976652773 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604800fbd0 con 0x7f603c0052e0 2024-04-25T14:09:24.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.998+0000 7f605e400640 1 -- 172.21.15.86:0/3976652773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f605815aab0 con 0x7f603c0052e0 2024-04-25T14:09:24.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:24.998+0000 7f605e400640 1 -- 172.21.15.86:0/3976652773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6058069d00 con 0x7f603c0052e0 2024-04-25T14:09:24.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:25.002+0000 7f604fe00640 1 -- 172.21.15.86:0/3976652773 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7f6048026430 con 0x7f603c0052e0 2024-04-25T14:09:24.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:25.002+0000 7f604fe00640 1 --2- 172.21.15.86:0/3976652773 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f603406f550 0x7f6034071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:24.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:25.002+0000 7f604fe00640 1 -- 172.21.15.86:0/3976652773 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6001+0+0 (secure 0 0 0) 0x7f604808f910 con 0x7f603c0052e0 2024-04-25T14:09:24.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:25.002+0000 7f604ea00640 1 --2- 172.21.15.86:0/3976652773 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f603406f550 0x7f6034071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:25.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:25.002+0000 7f604ea00640 1 --2- 172.21.15.86:0/3976652773 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f603406f550 0x7f6034071a60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f605815b440 tx=0x7f6040001400 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:25.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:25.006+0000 7f604fe00640 1 -- 172.21.15.86:0/3976652773 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f605815a550 con 0x7f603c0052e0 2024-04-25T14:09:25.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:25.194+0000 7f605e400640 1 -- 172.21.15.86:0/3976652773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6058076bf0 con 0x7f603c0052e0 2024-04-25T14:09:25.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:25.826+0000 7f604fe00640 1 -- 172.21.15.86:0/3976652773 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v72) ==== 174+0+0 (secure 0 0 0) 0x7f604805c730 con 0x7f603c0052e0 2024-04-25T14:09:25.831 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 15 and data pool 16 2024-04-25T14:09:25.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:25.838+0000 7f605e400640 1 -- 172.21.15.86:0/3976652773 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f603406f550 msgr2=0x7f6034071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:25.838+0000 7f605e400640 1 --2- 172.21.15.86:0/3976652773 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f603406f550 0x7f6034071a60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f605815b440 tx=0x7f6040001400 comp rx=0 tx=0).stop 2024-04-25T14:09:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:25.838+0000 7f605e400640 1 -- 172.21.15.86:0/3976652773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603c0052e0 msgr2=0x7f60581f8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:25.838+0000 7f605e400640 1 --2- 172.21.15.86:0/3976652773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603c0052e0 0x7f60581f8c40 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f6048004e40 tx=0x7f6048007430 comp rx=0 tx=0).stop 2024-04-25T14:09:25.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:25.838+0000 7f605e400640 1 -- 172.21.15.86:0/3976652773 shutdown_connections 2024-04-25T14:09:25.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:25.838+0000 7f605e400640 1 --2- 172.21.15.86:0/3976652773 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f603406f550 0x7f6034071a60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:25.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:25.838+0000 7f605e400640 1 --2- 172.21.15.86:0/3976652773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f605815a140 0x7f60580c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:25.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:25.838+0000 7f605e400640 1 --2- 172.21.15.86:0/3976652773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60580cf8c0 0x7f6058159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:25.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:25.838+0000 7f605e400640 1 --2- 172.21.15.86:0/3976652773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603c0052e0 0x7f60581f8c40 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:25.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:25.838+0000 7f605e400640 1 -- 172.21.15.86:0/3976652773 >> 172.21.15.86:0/3976652773 conn(0x7f60580b4650 msgr2=0x7f605815c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:25.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:25.838+0000 7f605e400640 1 -- 172.21.15.86:0/3976652773 shutdown_connections 2024-04-25T14:09:25.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:25.838+0000 7f605e400640 1 -- 172.21.15.86:0/3976652773 wait complete. 2024-04-25T14:09:25.871 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:09:26.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.018+0000 7faf3e600640 1 Processor -- start 2024-04-25T14:09:26.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.018+0000 7faf3e600640 1 -- start start 2024-04-25T14:09:26.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.022+0000 7faf3e600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7faf381642a0 0x7faf380d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:26.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.022+0000 7faf3e600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf380b0860 0x7faf380d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:26.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.022+0000 7faf3e600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf380d2030 con 0x7faf380d1c20 2024-04-25T14:09:26.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.022+0000 7faf3e600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7faf380cea30 con 0x7faf381642a0 2024-04-25T14:09:26.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.022+0000 7faf3e600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7faf380cde20 con 0x7faf380b0860 2024-04-25T14:09:26.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.022+0000 7faf37400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7faf381642a0 0x7faf380d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:26.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.022+0000 7faf37400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7faf381642a0 0x7faf380d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34004/0 (socket says 172.21.15.86:34004) 2024-04-25T14:09:26.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.022+0000 7faf37400640 1 -- 172.21.15.86:0/1220332883 learned_addr learned my addr 172.21.15.86:0/1220332883 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:26.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.022+0000 7faf36a00640 1 --1- 172.21.15.86:0/1220332883 >> v1:172.21.15.86:6789/0 conn(0x7faf380d1c20 0x7faf380b0150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34960/0 (socket says 172.21.15.86:34960) 2024-04-25T14:09:26.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.022+0000 7faf37e00640 1 --2- 172.21.15.86:0/1220332883 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf380b0860 0x7faf380d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:26.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.022+0000 7faf37400640 1 -- 172.21.15.86:0/1220332883 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf380b0860 msgr2=0x7faf380d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:26.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.022+0000 7faf37400640 1 --2- 172.21.15.86:0/1220332883 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf380b0860 0x7faf380d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:26.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.022+0000 7faf37400640 1 -- 172.21.15.86:0/1220332883 >> v1:172.21.15.86:6789/0 conn(0x7faf380d1c20 legacy=0x7faf380b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:26.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.022+0000 7faf36000640 1 -- 172.21.15.86:0/1220332883 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2947690756 0 0) 0x7faf24001120 con 0x7faf380d1c20 2024-04-25T14:09:26.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.022+0000 7faf37400640 1 -- 172.21.15.86:0/1220332883 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf380ce100 con 0x7faf381642a0 2024-04-25T14:09:26.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.026+0000 7faf37400640 1 --2- 172.21.15.86:0/1220332883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf381642a0 0x7faf380d16e0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7faf28009870 tx=0x7faf2801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9297cc640db7698 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:26.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.026+0000 7faf36000640 1 -- 172.21.15.86:0/1220332883 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf280b0070 con 0x7faf381642a0 2024-04-25T14:09:26.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.026+0000 7faf36000640 1 -- 172.21.15.86:0/1220332883 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf280ab600 con 0x7faf381642a0 2024-04-25T14:09:26.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.026+0000 7faf36000640 1 -- 172.21.15.86:0/1220332883 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf280ab9d0 con 0x7faf381642a0 2024-04-25T14:09:26.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.026+0000 7faf3e600640 1 -- 172.21.15.86:0/1220332883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf381642a0 msgr2=0x7faf380d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.026+0000 7faf3e600640 1 --2- 172.21.15.86:0/1220332883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf381642a0 0x7faf380d16e0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7faf28009870 tx=0x7faf2801c4d0 comp rx=0 tx=0).stop 2024-04-25T14:09:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.026+0000 7faf3e600640 1 -- 172.21.15.86:0/1220332883 shutdown_connections 2024-04-25T14:09:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.026+0000 7faf3e600640 1 --2- 172.21.15.86:0/1220332883 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf380b0860 0x7faf380d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.026+0000 7faf3e600640 1 --2- 172.21.15.86:0/1220332883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf381642a0 0x7faf380d16e0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.026+0000 7faf3e600640 1 -- 172.21.15.86:0/1220332883 >> 172.21.15.86:0/1220332883 conn(0x7faf380b4650 msgr2=0x7faf380b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.026+0000 7faf3e600640 1 -- 172.21.15.86:0/1220332883 shutdown_connections 2024-04-25T14:09:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.026+0000 7faf3e600640 1 -- 172.21.15.86:0/1220332883 wait complete. 2024-04-25T14:09:26.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.030+0000 7faf3e600640 1 Processor -- start 2024-04-25T14:09:26.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.030+0000 7faf3e600640 1 -- start start 2024-04-25T14:09:26.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.030+0000 7faf3e600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf380b0860 0x7faf3809a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:26.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.030+0000 7faf3e600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf380d1c20 0x7faf3815aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:26.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.030+0000 7faf3e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf381642a0 0x7faf380ae130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:26.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.030+0000 7faf3e600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faf38164830 con 0x7faf381642a0 2024-04-25T14:09:26.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.030+0000 7faf3e600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faf381646b0 con 0x7faf380b0860 2024-04-25T14:09:26.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.030+0000 7faf3e600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7faf38160da0 con 0x7faf380d1c20 2024-04-25T14:09:26.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.030+0000 7faf37e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf381642a0 0x7faf380ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:26.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.030+0000 7faf37e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf381642a0 0x7faf380ae130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34016/0 (socket says 172.21.15.86:34016) 2024-04-25T14:09:26.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.030+0000 7faf37e00640 1 -- 172.21.15.86:0/3673865320 learned_addr learned my addr 172.21.15.86:0/3673865320 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:26.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.030+0000 7faf36a00640 1 --2- 172.21.15.86:0/3673865320 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf380d1c20 0x7faf3815aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:26.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.030+0000 7faf37400640 1 --2- 172.21.15.86:0/3673865320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf380b0860 0x7faf3809a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:26.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.030+0000 7faf37e00640 1 -- 172.21.15.86:0/3673865320 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf380d1c20 msgr2=0x7faf3815aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:26.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.030+0000 7faf37e00640 1 --2- 172.21.15.86:0/3673865320 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf380d1c20 0x7faf3815aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:26.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.030+0000 7faf37e00640 1 -- 172.21.15.86:0/3673865320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf380b0860 msgr2=0x7faf3809a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:26.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.030+0000 7faf37e00640 1 --2- 172.21.15.86:0/3673865320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf380b0860 0x7faf3809a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:26.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.030+0000 7faf37e00640 1 -- 172.21.15.86:0/3673865320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3809a960 con 0x7faf381642a0 2024-04-25T14:09:26.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.030+0000 7faf37e00640 1 --2- 172.21.15.86:0/3673865320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf381642a0 0x7faf380ae130 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7faf30007d80 tx=0x7faf3000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:26.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.030+0000 7faf2f400640 1 -- 172.21.15.86:0/3673865320 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf30014420 con 0x7faf381642a0 2024-04-25T14:09:26.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.030+0000 7faf2f400640 1 -- 172.21.15.86:0/3673865320 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf30019070 con 0x7faf381642a0 2024-04-25T14:09:26.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.030+0000 7faf3e600640 1 -- 172.21.15.86:0/3673865320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf3815afa0 con 0x7faf381642a0 2024-04-25T14:09:26.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.030+0000 7faf2f400640 1 -- 172.21.15.86:0/3673865320 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf30014800 con 0x7faf381642a0 2024-04-25T14:09:26.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.030+0000 7faf3e600640 1 -- 172.21.15.86:0/3673865320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf38099660 con 0x7faf381642a0 2024-04-25T14:09:26.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.034+0000 7faf3e600640 1 -- 172.21.15.86:0/3673865320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf38069d00 con 0x7faf381642a0 2024-04-25T14:09:26.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.034+0000 7faf2f400640 1 -- 172.21.15.86:0/3673865320 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7faf300149a0 con 0x7faf381642a0 2024-04-25T14:09:26.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.034+0000 7faf2f400640 1 --2- 172.21.15.86:0/3673865320 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7faf1806f550 0x7faf18071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:26.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.034+0000 7faf2f400640 1 -- 172.21.15.86:0/3673865320 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7faf3008fdc0 con 0x7faf381642a0 2024-04-25T14:09:26.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.034+0000 7faf37400640 1 --2- 172.21.15.86:0/3673865320 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7faf1806f550 0x7faf18071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:26.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.034+0000 7faf37400640 1 --2- 172.21.15.86:0/3673865320 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7faf1806f550 0x7faf18071a60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faf280ad000 tx=0x7faf28008040 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:26.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.038+0000 7faf2f400640 1 -- 172.21.15.86:0/3673865320 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faf3005c320 con 0x7faf381642a0 2024-04-25T14:09:26.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.194+0000 7faf3e600640 1 -- 172.21.15.86:0/3673865320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faf38076bf0 con 0x7faf381642a0 2024-04-25T14:09:26.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.198+0000 7faf2f400640 1 -- 172.21.15.86:0/3673865320 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+16886 (secure 0 0 0) 0x7faf30060620 con 0x7faf381642a0 2024-04-25T14:09:26.194 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:26.194 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":99,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:09:25.198847+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:09:23.588241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T14:09:24.156880+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:09:26.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.198+0000 7faf3e600640 1 -- 172.21.15.86:0/3673865320 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7faf1806f550 msgr2=0x7faf18071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:26.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.198+0000 7faf3e600640 1 --2- 172.21.15.86:0/3673865320 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7faf1806f550 0x7faf18071a60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faf280ad000 tx=0x7faf28008040 comp rx=0 tx=0).stop 2024-04-25T14:09:26.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.198+0000 7faf3e600640 1 -- 172.21.15.86:0/3673865320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf381642a0 msgr2=0x7faf380ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:26.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.198+0000 7faf3e600640 1 --2- 172.21.15.86:0/3673865320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf381642a0 0x7faf380ae130 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7faf30007d80 tx=0x7faf3000e4b0 comp rx=0 tx=0).stop 2024-04-25T14:09:26.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.198+0000 7faf3e600640 1 -- 172.21.15.86:0/3673865320 shutdown_connections 2024-04-25T14:09:26.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.198+0000 7faf3e600640 1 --2- 172.21.15.86:0/3673865320 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7faf1806f550 0x7faf18071a60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:26.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.198+0000 7faf3e600640 1 --2- 172.21.15.86:0/3673865320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf381642a0 0x7faf380ae130 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:26.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.198+0000 7faf3e600640 1 --2- 172.21.15.86:0/3673865320 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf380d1c20 0x7faf3815aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:26.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.198+0000 7faf3e600640 1 --2- 172.21.15.86:0/3673865320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf380b0860 0x7faf3809a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:26.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.198+0000 7faf3e600640 1 -- 172.21.15.86:0/3673865320 >> 172.21.15.86:0/3673865320 conn(0x7faf380b4650 msgr2=0x7faf3806e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:26.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.198+0000 7faf3e600640 1 -- 172.21.15.86:0/3673865320 shutdown_connections 2024-04-25T14:09:26.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.198+0000 7faf3e600640 1 -- 172.21.15.86:0/3673865320 wait complete. 2024-04-25T14:09:26.216 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:09:26.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.366+0000 7f7080c00640 1 Processor -- start 2024-04-25T14:09:26.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.366+0000 7f7080c00640 1 -- start start 2024-04-25T14:09:26.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.366+0000 7f7080c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f707c0d1df0 0x7f707c0cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:26.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.366+0000 7f7080c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f707c15e5c0 con 0x7f707c0cdcc0 2024-04-25T14:09:26.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.366+0000 7f7080c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f707c15e8a0 con 0x7f707c070330 2024-04-25T14:09:26.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.366+0000 7f7080c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f707c06fd40 con 0x7f707c0d1df0 2024-04-25T14:09:26.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.366+0000 7f707a800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f707c0cdcc0 0x7f707c0ce0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34962/0 (socket says 172.21.15.86:34962) 2024-04-25T14:09:26.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.366+0000 7f707a800640 1 -- 172.21.15.86:0/3218128884 learned_addr learned my addr 172.21.15.86:0/3218128884 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:26.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.366+0000 7f7079400640 1 --2- 172.21.15.86:0/3218128884 >> v2:172.21.15.155:3300/0 conn(0x7f707c0d1df0 0x7f707c0cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:26.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.366+0000 7f7078a00640 1 -- 172.21.15.86:0/3218128884 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 226992943 0 0) 0x7f707c15e5c0 con 0x7f707c0cdcc0 2024-04-25T14:09:26.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.366+0000 7f7078a00640 1 -- 172.21.15.86:0/3218128884 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7060003530 con 0x7f707c0cdcc0 2024-04-25T14:09:26.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.366+0000 7f7079400640 1 -- 172.21.15.86:0/3218128884 >> v1:172.21.15.155:6789/0 conn(0x7f707c070330 legacy=0x7f707c0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:26.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.366+0000 7f7079400640 1 -- 172.21.15.86:0/3218128884 >> v1:172.21.15.86:6789/0 conn(0x7f707c0cdcc0 legacy=0x7f707c0ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:26.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.366+0000 7f7079400640 1 -- 172.21.15.86:0/3218128884 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f707c15eb80 con 0x7f707c0d1df0 2024-04-25T14:09:26.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.370+0000 7f7079400640 1 --2- 172.21.15.86:0/3218128884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f707c0d1df0 0x7f707c0cd780 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f70680097f0 tx=0x7f706801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e1d5006546d27fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:26.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.370+0000 7f7078a00640 1 -- 172.21.15.86:0/3218128884 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70680b0070 con 0x7f707c0d1df0 2024-04-25T14:09:26.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.370+0000 7f7078a00640 1 -- 172.21.15.86:0/3218128884 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70680ab600 con 0x7f707c0d1df0 2024-04-25T14:09:26.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.370+0000 7f7078a00640 1 -- 172.21.15.86:0/3218128884 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70680ab9d0 con 0x7f707c0d1df0 2024-04-25T14:09:26.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.370+0000 7f7080c00640 1 -- 172.21.15.86:0/3218128884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f707c0d1df0 msgr2=0x7f707c0cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:26.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.370+0000 7f7080c00640 1 --2- 172.21.15.86:0/3218128884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f707c0d1df0 0x7f707c0cd780 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f70680097f0 tx=0x7f706801c3e0 comp rx=0 tx=0).stop 2024-04-25T14:09:26.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.370+0000 7f7080c00640 1 -- 172.21.15.86:0/3218128884 shutdown_connections 2024-04-25T14:09:26.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.370+0000 7f7080c00640 1 --2- 172.21.15.86:0/3218128884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f707c0d1df0 0x7f707c0cd780 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:26.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.370+0000 7f7080c00640 1 -- 172.21.15.86:0/3218128884 >> 172.21.15.86:0/3218128884 conn(0x7f707c0b4650 msgr2=0x7f707c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:26.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.370+0000 7f7080c00640 1 -- 172.21.15.86:0/3218128884 shutdown_connections 2024-04-25T14:09:26.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.370+0000 7f7080c00640 1 -- 172.21.15.86:0/3218128884 wait complete. 2024-04-25T14:09:26.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.374+0000 7f7080c00640 1 Processor -- start 2024-04-25T14:09:26.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.374+0000 7f7080c00640 1 -- start start 2024-04-25T14:09:26.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.374+0000 7f7080c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707c070330 0x7f707c0c47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:26.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.374+0000 7f7080c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f707c0cdcc0 0x7f707c09aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:26.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.374+0000 7f7080c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f707c1566d0 0x7f707c158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:26.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.374+0000 7f7080c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f707c0708c0 con 0x7f707c070330 2024-04-25T14:09:26.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.374+0000 7f7080c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f707c070740 con 0x7f707c1566d0 2024-04-25T14:09:26.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.374+0000 7f7080c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f707c16bfd0 con 0x7f707c0cdcc0 2024-04-25T14:09:26.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.374+0000 7f7079e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707c070330 0x7f707c0c47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:26.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.374+0000 7f7079e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707c070330 0x7f707c0c47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34028/0 (socket says 172.21.15.86:34028) 2024-04-25T14:09:26.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.374+0000 7f7079e00640 1 -- 172.21.15.86:0/2036467695 learned_addr learned my addr 172.21.15.86:0/2036467695 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:26.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.374+0000 7f7079400640 1 --2- 172.21.15.86:0/2036467695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f707c0cdcc0 0x7f707c09aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:26.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.374+0000 7f707a800640 1 --2- 172.21.15.86:0/2036467695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f707c1566d0 0x7f707c158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:26.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.374+0000 7f7079e00640 1 -- 172.21.15.86:0/2036467695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f707c0cdcc0 msgr2=0x7f707c09aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:26.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.374+0000 7f7079e00640 1 --2- 172.21.15.86:0/2036467695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f707c0cdcc0 0x7f707c09aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:26.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.374+0000 7f7079e00640 1 -- 172.21.15.86:0/2036467695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f707c1566d0 msgr2=0x7f707c158790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:26.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.374+0000 7f7079e00640 1 --2- 172.21.15.86:0/2036467695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f707c1566d0 0x7f707c158790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:26.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.374+0000 7f7079e00640 1 -- 172.21.15.86:0/2036467695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f707c152780 con 0x7f707c070330 2024-04-25T14:09:26.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.374+0000 7f707a800640 1 --2- 172.21.15.86:0/2036467695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f707c1566d0 0x7f707c158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:26.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.378+0000 7f7079e00640 1 --2- 172.21.15.86:0/2036467695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707c070330 0x7f707c0c47f0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f706c00c370 tx=0x7f706c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:26.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.378+0000 7f7072000640 1 -- 172.21.15.86:0/2036467695 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706c0052d0 con 0x7f707c070330 2024-04-25T14:09:26.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.378+0000 7f7072000640 1 -- 172.21.15.86:0/2036467695 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f706c005470 con 0x7f707c070330 2024-04-25T14:09:26.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.378+0000 7f7072000640 1 -- 172.21.15.86:0/2036467695 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706c007260 con 0x7f707c070330 2024-04-25T14:09:26.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.378+0000 7f7080c00640 1 -- 172.21.15.86:0/2036467695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f707c155910 con 0x7f707c070330 2024-04-25T14:09:26.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.378+0000 7f7080c00640 1 -- 172.21.15.86:0/2036467695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f707c155ed0 con 0x7f707c070330 2024-04-25T14:09:26.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.378+0000 7f7080c00640 1 -- 172.21.15.86:0/2036467695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f707c069d00 con 0x7f707c070330 2024-04-25T14:09:26.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.378+0000 7f7072000640 1 -- 172.21.15.86:0/2036467695 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7f706c007400 con 0x7f707c070330 2024-04-25T14:09:26.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.382+0000 7f7072000640 1 --2- 172.21.15.86:0/2036467695 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f705c06f550 0x7f705c071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:26.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.382+0000 7f7072000640 1 -- 172.21.15.86:0/2036467695 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f706c08e900 con 0x7f707c070330 2024-04-25T14:09:26.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.382+0000 7f7079400640 1 --2- 172.21.15.86:0/2036467695 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f705c06f550 0x7f705c071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:26.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.382+0000 7f7079400640 1 --2- 172.21.15.86:0/2036467695 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f705c06f550 0x7f705c071a60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f70680ad000 tx=0x7f70680a6000 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:26.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.386+0000 7f7072000640 1 -- 172.21.15.86:0/2036467695 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f706c05af90 con 0x7f707c070330 2024-04-25T14:09:26.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.542+0000 7f7080c00640 1 -- 172.21.15.86:0/2036467695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f707c076bf0 con 0x7f707c070330 2024-04-25T14:09:26.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.546+0000 7f7072000640 1 -- 172.21.15.86:0/2036467695 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+16886 (secure 0 0 0) 0x7f706c05f290 con 0x7f707c070330 2024-04-25T14:09:26.542 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:26.542 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":99,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:09:25.198847+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:09:23.588241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T14:09:24.156880+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:09:26.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.546+0000 7f7080c00640 1 -- 172.21.15.86:0/2036467695 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f705c06f550 msgr2=0x7f705c071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:26.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.546+0000 7f7080c00640 1 --2- 172.21.15.86:0/2036467695 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f705c06f550 0x7f705c071a60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f70680ad000 tx=0x7f70680a6000 comp rx=0 tx=0).stop 2024-04-25T14:09:26.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.546+0000 7f7080c00640 1 -- 172.21.15.86:0/2036467695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707c070330 msgr2=0x7f707c0c47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:26.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.546+0000 7f7080c00640 1 --2- 172.21.15.86:0/2036467695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707c070330 0x7f707c0c47f0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f706c00c370 tx=0x7f706c00c830 comp rx=0 tx=0).stop 2024-04-25T14:09:26.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.546+0000 7f7080c00640 1 -- 172.21.15.86:0/2036467695 shutdown_connections 2024-04-25T14:09:26.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.546+0000 7f7080c00640 1 --2- 172.21.15.86:0/2036467695 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f705c06f550 0x7f705c071a60 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:26.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.546+0000 7f7080c00640 1 --2- 172.21.15.86:0/2036467695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f707c1566d0 0x7f707c158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:26.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.546+0000 7f7080c00640 1 --2- 172.21.15.86:0/2036467695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f707c0cdcc0 0x7f707c09aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:26.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.546+0000 7f7080c00640 1 --2- 172.21.15.86:0/2036467695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707c070330 0x7f707c0c47f0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:26.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.546+0000 7f7080c00640 1 -- 172.21.15.86:0/2036467695 >> 172.21.15.86:0/2036467695 conn(0x7f707c0b4650 msgr2=0x7f707c0b1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:26.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.546+0000 7f7080c00640 1 -- 172.21.15.86:0/2036467695 shutdown_connections 2024-04-25T14:09:26.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.546+0000 7f7080c00640 1 -- 172.21.15.86:0/2036467695 wait complete. 2024-04-25T14:09:26.564 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T14:09:26.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.718+0000 7fbd11600640 1 Processor -- start 2024-04-25T14:09:26.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.718+0000 7fbd11600640 1 -- start start 2024-04-25T14:09:26.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.718+0000 7fbd11600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd0c15dc10 0x7fbd0c0b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:26.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.718+0000 7fbd11600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbd0c0d2230 0x7fbd0c0d2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:26.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.718+0000 7fbd11600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd0c0d2b80 con 0x7fbd0c0cec50 2024-04-25T14:09:26.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.718+0000 7fbd11600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fbd0c0d15a0 con 0x7fbd0c0d2230 2024-04-25T14:09:26.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.718+0000 7fbd11600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbd0c0d1880 con 0x7fbd0c15dc10 2024-04-25T14:09:26.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.718+0000 7fbd0be00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd0c15dc10 0x7fbd0c0b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:26.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.718+0000 7fbd0b400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbd0c0d2230 0x7fbd0c0d2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:26.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.718+0000 7fbd0b400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbd0c0d2230 0x7fbd0c0d2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:43360/0 (socket says 172.21.15.86:43360) 2024-04-25T14:09:26.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.718+0000 7fbd10c00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fbd0c0cec50 0x7fbd0c0cf060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:41938/0 (socket says 172.21.15.86:41938) 2024-04-25T14:09:26.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.718+0000 7fbd0b400640 1 -- 172.21.15.86:0/2348214906 learned_addr learned my addr 172.21.15.86:0/2348214906 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:26.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.718+0000 7fbd0be00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd0c15dc10 0x7fbd0c0b34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:54900/0 (socket says 172.21.15.86:54900) 2024-04-25T14:09:26.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.722+0000 7fbd0b400640 1 -- 172.21.15.86:0/2348214906 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd0c15dc10 msgr2=0x7fbd0c0b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:26.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.722+0000 7fbd0b400640 1 --2- 172.21.15.86:0/2348214906 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd0c15dc10 0x7fbd0c0b34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:26.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.722+0000 7fbd0b400640 1 -- 172.21.15.86:0/2348214906 >> v1:172.21.15.155:6789/0 conn(0x7fbd0c0cec50 legacy=0x7fbd0c0cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:26.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.722+0000 7fbd0b400640 1 -- 172.21.15.86:0/2348214906 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd0c0d1b60 con 0x7fbd0c0d2230 2024-04-25T14:09:26.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.722+0000 7fbd0b400640 1 --2- 172.21.15.86:0/2348214906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd0c0d2230 0x7fbd0c0d2640 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8009870 tx=0x7fbcf801ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8d389a58c993b4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:26.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.722+0000 7fbd0aa00640 1 -- 172.21.15.86:0/2348214906 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf80b5070 con 0x7fbd0c0d2230 2024-04-25T14:09:26.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.722+0000 7fbd0aa00640 1 -- 172.21.15.86:0/2348214906 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcf80a8a50 con 0x7fbd0c0d2230 2024-04-25T14:09:26.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.722+0000 7fbd0aa00640 1 -- 172.21.15.86:0/2348214906 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf80a8e20 con 0x7fbd0c0d2230 2024-04-25T14:09:26.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.726+0000 7fbd11600640 1 -- 172.21.15.86:0/2348214906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd0c0d2230 msgr2=0x7fbd0c0d2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:26.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.726+0000 7fbd11600640 1 --2- 172.21.15.86:0/2348214906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd0c0d2230 0x7fbd0c0d2640 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8009870 tx=0x7fbcf801ed50 comp rx=0 tx=0).stop 2024-04-25T14:09:26.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.726+0000 7fbd11600640 1 -- 172.21.15.86:0/2348214906 shutdown_connections 2024-04-25T14:09:26.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.726+0000 7fbd11600640 1 --2- 172.21.15.86:0/2348214906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd0c0d2230 0x7fbd0c0d2640 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:26.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.726+0000 7fbd11600640 1 --2- 172.21.15.86:0/2348214906 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd0c15dc10 0x7fbd0c0b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:26.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.726+0000 7fbd11600640 1 -- 172.21.15.86:0/2348214906 >> 172.21.15.86:0/2348214906 conn(0x7fbd0c09ba30 msgr2=0x7fbd0c0b71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:26.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.726+0000 7fbd11600640 1 -- 172.21.15.86:0/2348214906 shutdown_connections 2024-04-25T14:09:26.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.726+0000 7fbd11600640 1 -- 172.21.15.86:0/2348214906 wait complete. 2024-04-25T14:09:26.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.726+0000 7fbd11600640 1 Processor -- start 2024-04-25T14:09:26.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.726+0000 7fbd11600640 1 -- start start 2024-04-25T14:09:26.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.730+0000 7fbd11600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd0c0cec50 0x7fbd0c09cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:26.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.730+0000 7fbd11600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd0c0d2230 0x7fbd0c070e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:26.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.730+0000 7fbd11600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd0c15dc10 0x7fbd0c153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:26.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.730+0000 7fbd11600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c15e020 con 0x7fbd0c0d2230 2024-04-25T14:09:26.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.730+0000 7fbd11600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c0d1290 con 0x7fbd0c0cec50 2024-04-25T14:09:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.730+0000 7fbd11600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbd0c0b39f0 con 0x7fbd0c15dc10 2024-04-25T14:09:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.730+0000 7fbd0b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd0c0d2230 0x7fbd0c070e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.730+0000 7fbd0b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd0c0d2230 0x7fbd0c070e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34030/0 (socket says 172.21.15.86:34030) 2024-04-25T14:09:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.730+0000 7fbd0b400640 1 -- 172.21.15.86:0/209761341 learned_addr learned my addr 172.21.15.86:0/209761341 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.730+0000 7fbd0be00640 1 --2- 172.21.15.86:0/209761341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd0c0cec50 0x7fbd0c09cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.730+0000 7fbd10c00640 1 --2- 172.21.15.86:0/209761341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd0c15dc10 0x7fbd0c153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.730+0000 7fbd0b400640 1 -- 172.21.15.86:0/209761341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd0c15dc10 msgr2=0x7fbd0c153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.730+0000 7fbd0b400640 1 --2- 172.21.15.86:0/209761341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd0c15dc10 0x7fbd0c153c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.730+0000 7fbd0b400640 1 -- 172.21.15.86:0/209761341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd0c0cec50 msgr2=0x7fbd0c09cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.730+0000 7fbd0b400640 1 --2- 172.21.15.86:0/209761341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd0c0cec50 0x7fbd0c09cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.730+0000 7fbd0b400640 1 -- 172.21.15.86:0/209761341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd0c09e7b0 con 0x7fbd0c0d2230 2024-04-25T14:09:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.730+0000 7fbd0b400640 1 --2- 172.21.15.86:0/209761341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd0c0d2230 0x7fbd0c070e90 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80b3040 tx=0x7fbcf801baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.730+0000 7fbd03e00640 1 -- 172.21.15.86:0/209761341 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf80b5070 con 0x7fbd0c0d2230 2024-04-25T14:09:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.730+0000 7fbd03e00640 1 -- 172.21.15.86:0/209761341 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcf80b0de0 con 0x7fbd0c0d2230 2024-04-25T14:09:26.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.730+0000 7fbd11600640 1 -- 172.21.15.86:0/209761341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd0c09d180 con 0x7fbd0c0d2230 2024-04-25T14:09:26.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.730+0000 7fbd03e00640 1 -- 172.21.15.86:0/209761341 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf80bb670 con 0x7fbd0c0d2230 2024-04-25T14:09:26.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.730+0000 7fbd11600640 1 -- 172.21.15.86:0/209761341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd0c1fa390 con 0x7fbd0c0d2230 2024-04-25T14:09:26.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.730+0000 7fbd11600640 1 -- 172.21.15.86:0/209761341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd0c076b80 con 0x7fbd0c0d2230 2024-04-25T14:09:26.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.730+0000 7fbd03e00640 1 -- 172.21.15.86:0/209761341 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7fbcf80c2060 con 0x7fbd0c0d2230 2024-04-25T14:09:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.734+0000 7fbd03e00640 1 --2- 172.21.15.86:0/209761341 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fbcec06f550 0x7fbcec071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.734+0000 7fbd03e00640 1 -- 172.21.15.86:0/209761341 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7fbcf812c820 con 0x7fbd0c0d2230 2024-04-25T14:09:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.734+0000 7fbd0be00640 1 --2- 172.21.15.86:0/209761341 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fbcec06f550 0x7fbcec071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.734+0000 7fbd0be00640 1 --2- 172.21.15.86:0/209761341 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fbcec06f550 0x7fbcec071a60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc009ac0 tx=0x7fbcfc009190 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.738+0000 7fbd03e00640 1 -- 172.21.15.86:0/209761341 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbcf80188b0 con 0x7fbd0c0d2230 2024-04-25T14:09:26.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:26.930+0000 7fbd11600640 1 -- 172.21.15.86:0/209761341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbd0c069d00 con 0x7fbd0c0d2230 2024-04-25T14:09:27.781 INFO:tasks.ceph.mds.b.smithi155.stderr:2024-04-25T14:09:27.779+0000 7f2d59600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T14:09:27.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:27.842+0000 7fbd03e00640 1 -- 172.21.15.86:0/209761341 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v75) ==== 118+0+0 (secure 0 0 0) 0x7fbcf80f8eb0 con 0x7fbd0c0d2230 2024-04-25T14:09:27.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:27.846+0000 7fbd11600640 1 -- 172.21.15.86:0/209761341 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fbcec06f550 msgr2=0x7fbcec071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:27.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:27.846+0000 7fbd11600640 1 --2- 172.21.15.86:0/209761341 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fbcec06f550 0x7fbcec071a60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc009ac0 tx=0x7fbcfc009190 comp rx=0 tx=0).stop 2024-04-25T14:09:27.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:27.846+0000 7fbd11600640 1 -- 172.21.15.86:0/209761341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd0c0d2230 msgr2=0x7fbd0c070e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:27.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:27.846+0000 7fbd11600640 1 --2- 172.21.15.86:0/209761341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd0c0d2230 0x7fbd0c070e90 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80b3040 tx=0x7fbcf801baf0 comp rx=0 tx=0).stop 2024-04-25T14:09:27.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:27.846+0000 7fbd11600640 1 -- 172.21.15.86:0/209761341 shutdown_connections 2024-04-25T14:09:27.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:27.846+0000 7fbd11600640 1 --2- 172.21.15.86:0/209761341 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fbcec06f550 0x7fbcec071a60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:27.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:27.846+0000 7fbd11600640 1 --2- 172.21.15.86:0/209761341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd0c15dc10 0x7fbd0c153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:27.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:27.846+0000 7fbd11600640 1 --2- 172.21.15.86:0/209761341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd0c0d2230 0x7fbd0c070e90 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:27.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:27.846+0000 7fbd11600640 1 --2- 172.21.15.86:0/209761341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd0c0cec50 0x7fbd0c09cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:27.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:27.846+0000 7fbd11600640 1 -- 172.21.15.86:0/209761341 >> 172.21.15.86:0/209761341 conn(0x7fbd0c09ba30 msgr2=0x7fbd0c15c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:27.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:27.846+0000 7fbd11600640 1 -- 172.21.15.86:0/209761341 shutdown_connections 2024-04-25T14:09:27.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:27.846+0000 7fbd11600640 1 -- 172.21.15.86:0/209761341 wait complete. 2024-04-25T14:09:27.867 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T14:09:27.867 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T14:09:28.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.026+0000 7f4699200640 1 Processor -- start 2024-04-25T14:09:28.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.026+0000 7f4699200640 1 -- start start 2024-04-25T14:09:28.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.026+0000 7f4699200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4694070330 0x7f46940d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:28.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.026+0000 7f4699200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f46940ce790 0x7f469406f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:28.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.026+0000 7f4699200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46940d2030 con 0x7f46940d1c20 2024-04-25T14:09:28.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.026+0000 7f4699200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f469415e5c0 con 0x7f46940ce790 2024-04-25T14:09:28.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.026+0000 7f4699200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f469415e8a0 con 0x7f4694070330 2024-04-25T14:09:28.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.026+0000 7f4691e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4694070330 0x7f46940d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.026+0000 7f4689400640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f46940d1c20 0x7f469406ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:41944/0 (socket says 172.21.15.86:41944) 2024-04-25T14:09:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.026+0000 7f4689400640 1 -- 172.21.15.86:0/3564413017 learned_addr learned my addr 172.21.15.86:0/3564413017 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.026+0000 7f4692800640 1 --2- 172.21.15.86:0/3564413017 >> v2:172.21.15.155:3300/0 conn(0x7f46940ce790 0x7f469406f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.026+0000 7f4691e00640 1 -- 172.21.15.86:0/3564413017 >> v2:172.21.15.155:3300/0 conn(0x7f46940ce790 msgr2=0x7f469406f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.026+0000 7f4691e00640 1 --2- 172.21.15.86:0/3564413017 >> v2:172.21.15.155:3300/0 conn(0x7f46940ce790 0x7f469406f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.026+0000 7f4691e00640 1 -- 172.21.15.86:0/3564413017 >> v1:172.21.15.155:6789/0 conn(0x7f46940d1c20 legacy=0x7f469406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.026+0000 7f4691e00640 1 -- 172.21.15.86:0/3564413017 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469415eb80 con 0x7f4694070330 2024-04-25T14:09:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.026+0000 7f4691400640 1 -- 172.21.15.86:0/3564413017 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 252540231 0 0) 0x7f467c001120 con 0x7f46940d1c20 2024-04-25T14:09:28.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.030+0000 7f4691e00640 1 --2- 172.21.15.86:0/3564413017 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4694070330 0x7f46940d16e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4684009dc0 tx=0x7f468401c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=731f2faf3c6c5590 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:28.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.030+0000 7f4691400640 1 -- 172.21.15.86:0/3564413017 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46840b2070 con 0x7f4694070330 2024-04-25T14:09:28.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.030+0000 7f4691400640 1 -- 172.21.15.86:0/3564413017 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46840a5780 con 0x7f4694070330 2024-04-25T14:09:28.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.030+0000 7f4691400640 1 -- 172.21.15.86:0/3564413017 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46840a5b50 con 0x7f4694070330 2024-04-25T14:09:28.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.030+0000 7f4699200640 1 -- 172.21.15.86:0/3564413017 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4694070330 msgr2=0x7f46940d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:28.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.030+0000 7f4699200640 1 --2- 172.21.15.86:0/3564413017 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4694070330 0x7f46940d16e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4684009dc0 tx=0x7f468401c8c0 comp rx=0 tx=0).stop 2024-04-25T14:09:28.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.030+0000 7f4699200640 1 -- 172.21.15.86:0/3564413017 shutdown_connections 2024-04-25T14:09:28.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.030+0000 7f4699200640 1 --2- 172.21.15.86:0/3564413017 >> v2:172.21.15.155:3300/0 conn(0x7f46940ce790 0x7f469406f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:28.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.030+0000 7f4699200640 1 --2- 172.21.15.86:0/3564413017 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4694070330 0x7f46940d16e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:28.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.030+0000 7f4699200640 1 -- 172.21.15.86:0/3564413017 >> 172.21.15.86:0/3564413017 conn(0x7f46940b4650 msgr2=0x7f46940b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:28.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.030+0000 7f4699200640 1 -- 172.21.15.86:0/3564413017 shutdown_connections 2024-04-25T14:09:28.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.030+0000 7f4699200640 1 -- 172.21.15.86:0/3564413017 wait complete. 2024-04-25T14:09:28.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.034+0000 7f4699200640 1 Processor -- start 2024-04-25T14:09:28.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.034+0000 7f4699200640 1 -- start start 2024-04-25T14:09:28.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.034+0000 7f4699200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4694070330 0x7f46940afb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:28.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.034+0000 7f4699200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46940ce790 0x7f46940738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:28.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.034+0000 7f4699200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46940d1c20 0x7f4694073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:28.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.034+0000 7f4699200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46940708c0 con 0x7f4694070330 2024-04-25T14:09:28.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.034+0000 7f4699200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4694070740 con 0x7f46940ce790 2024-04-25T14:09:28.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.034+0000 7f4699200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f469416afc0 con 0x7f46940d1c20 2024-04-25T14:09:28.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.034+0000 7f4691e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4694070330 0x7f46940afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:28.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.038+0000 7f4691e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4694070330 0x7f46940afb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34038/0 (socket says 172.21.15.86:34038) 2024-04-25T14:09:28.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.038+0000 7f4691e00640 1 -- 172.21.15.86:0/1792712713 learned_addr learned my addr 172.21.15.86:0/1792712713 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:28.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.038+0000 7f4692800640 1 --2- 172.21.15.86:0/1792712713 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46940d1c20 0x7f4694073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:28.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.038+0000 7f4689400640 1 --2- 172.21.15.86:0/1792712713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46940ce790 0x7f46940738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:28.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.038+0000 7f4691e00640 1 -- 172.21.15.86:0/1792712713 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46940d1c20 msgr2=0x7f4694073e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:28.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.038+0000 7f4691e00640 1 --2- 172.21.15.86:0/1792712713 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46940d1c20 0x7f4694073e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:28.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.038+0000 7f4691e00640 1 -- 172.21.15.86:0/1792712713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46940ce790 msgr2=0x7f46940738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:28.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.038+0000 7f4691e00640 1 --2- 172.21.15.86:0/1792712713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46940ce790 0x7f46940738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:28.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.038+0000 7f4691e00640 1 -- 172.21.15.86:0/1792712713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46940af0a0 con 0x7f4694070330 2024-04-25T14:09:28.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.038+0000 7f4689400640 1 --2- 172.21.15.86:0/1792712713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46940ce790 0x7f46940738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:28.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.038+0000 7f4691e00640 1 --2- 172.21.15.86:0/1792712713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4694070330 0x7f46940afb50 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f4684009dc0 tx=0x7f4684019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:28.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.038+0000 7f468aa00640 1 -- 172.21.15.86:0/1792712713 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46840b2040 con 0x7f4694070330 2024-04-25T14:09:28.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.038+0000 7f468aa00640 1 -- 172.21.15.86:0/1792712713 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46840bf070 con 0x7f4694070330 2024-04-25T14:09:28.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.038+0000 7f4699200640 1 -- 172.21.15.86:0/1792712713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46940c95e0 con 0x7f4694070330 2024-04-25T14:09:28.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.038+0000 7f468aa00640 1 -- 172.21.15.86:0/1792712713 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46840b8670 con 0x7f4694070330 2024-04-25T14:09:28.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.038+0000 7f4699200640 1 -- 172.21.15.86:0/1792712713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46940c9ba0 con 0x7f4694070330 2024-04-25T14:09:28.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.038+0000 7f4699200640 1 -- 172.21.15.86:0/1792712713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4694069d00 con 0x7f4694070330 2024-04-25T14:09:28.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.038+0000 7f468aa00640 1 -- 172.21.15.86:0/1792712713 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7f46840b8810 con 0x7f4694070330 2024-04-25T14:09:28.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.042+0000 7f468aa00640 1 --2- 172.21.15.86:0/1792712713 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f467006f550 0x7f4670071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:28.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.042+0000 7f468aa00640 1 -- 172.21.15.86:0/1792712713 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7f468412a480 con 0x7f4694070330 2024-04-25T14:09:28.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.042+0000 7f4689400640 1 --2- 172.21.15.86:0/1792712713 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f467006f550 0x7f4670071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:28.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.042+0000 7f4689400640 1 --2- 172.21.15.86:0/1792712713 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f467006f550 0x7f4670071a60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f46940af3f0 tx=0x7f467c007cd0 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:28.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.046+0000 7f468aa00640 1 -- 172.21.15.86:0/1792712713 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4684016540 con 0x7f4694070330 2024-04-25T14:09:28.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.234+0000 7f4699200640 1 -- 172.21.15.86:0/1792712713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f469415b7a0 con 0x7f4694070330 2024-04-25T14:09:28.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.846+0000 7f468aa00640 1 -- 172.21.15.86:0/1792712713 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v76) ==== 122+0+0 (secure 0 0 0) 0x7f46840f7030 con 0x7f4694070330 2024-04-25T14:09:28.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.850+0000 7f4699200640 1 -- 172.21.15.86:0/1792712713 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f467006f550 msgr2=0x7f4670071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:28.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.850+0000 7f4699200640 1 --2- 172.21.15.86:0/1792712713 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f467006f550 0x7f4670071a60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f46940af3f0 tx=0x7f467c007cd0 comp rx=0 tx=0).stop 2024-04-25T14:09:28.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.850+0000 7f4699200640 1 -- 172.21.15.86:0/1792712713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4694070330 msgr2=0x7f46940afb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:28.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.850+0000 7f4699200640 1 --2- 172.21.15.86:0/1792712713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4694070330 0x7f46940afb50 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f4684009dc0 tx=0x7f4684019660 comp rx=0 tx=0).stop 2024-04-25T14:09:28.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.850+0000 7f4699200640 1 -- 172.21.15.86:0/1792712713 shutdown_connections 2024-04-25T14:09:28.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.850+0000 7f4699200640 1 --2- 172.21.15.86:0/1792712713 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f467006f550 0x7f4670071a60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:28.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.850+0000 7f4699200640 1 --2- 172.21.15.86:0/1792712713 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46940d1c20 0x7f4694073e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.850+0000 7f4699200640 1 --2- 172.21.15.86:0/1792712713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46940ce790 0x7f46940738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.850+0000 7f4699200640 1 --2- 172.21.15.86:0/1792712713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4694070330 0x7f46940afb50 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.850+0000 7f4699200640 1 -- 172.21.15.86:0/1792712713 >> 172.21.15.86:0/1792712713 conn(0x7f46940b4650 msgr2=0x7f46940c6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.850+0000 7f4699200640 1 -- 172.21.15.86:0/1792712713 shutdown_connections 2024-04-25T14:09:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:28.850+0000 7f4699200640 1 -- 172.21.15.86:0/1792712713 wait complete. 2024-04-25T14:09:28.868 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T14:09:28.869 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:09:29.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.022+0000 7f2b91a00640 1 Processor -- start 2024-04-25T14:09:29.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.022+0000 7f2b91a00640 1 -- start start 2024-04-25T14:09:29.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.022+0000 7f2b91a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2b8c0cdd70 0x7f2b8c0ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:29.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.022+0000 7f2b91a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2b8c070e20 0x7f2b8c071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:29.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.022+0000 7f2b91a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b8c06e700 0x7f2b8c06eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:29.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.022+0000 7f2b91a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2b8c071770 con 0x7f2b8c070e20 2024-04-25T14:09:29.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.022+0000 7f2b91a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f2b8c06f050 con 0x7f2b8c0cdd70 2024-04-25T14:09:29.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.022+0000 7f2b91a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2b8c06d940 con 0x7f2b8c06e700 2024-04-25T14:09:29.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.022+0000 7f2b8a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2b8c070e20 0x7f2b8c071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:29.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.022+0000 7f2b8a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2b8c070e20 0x7f2b8c071230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58002/0 (socket says 172.21.15.86:58002) 2024-04-25T14:09:29.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.022+0000 7f2b8a000640 1 -- 172.21.15.86:0/2124609355 learned_addr learned my addr 172.21.15.86:0/2124609355 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:29.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.022+0000 7f2b8b400640 1 --2- 172.21.15.86:0/2124609355 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b8c06e700 0x7f2b8c06eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:29.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.022+0000 7f2b8aa00640 1 --2- 172.21.15.86:0/2124609355 >> v2:172.21.15.155:3300/0 conn(0x7f2b8c0cdd70 0x7f2b8c0ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:29.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.022+0000 7f2b8a000640 1 -- 172.21.15.86:0/2124609355 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b8c06e700 msgr2=0x7f2b8c06eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:29.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.022+0000 7f2b8a000640 1 --2- 172.21.15.86:0/2124609355 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b8c06e700 0x7f2b8c06eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.022+0000 7f2b8a000640 1 -- 172.21.15.86:0/2124609355 >> v2:172.21.15.155:3300/0 conn(0x7f2b8c0cdd70 msgr2=0x7f2b8c0ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:29.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.022+0000 7f2b8a000640 1 --2- 172.21.15.86:0/2124609355 >> v2:172.21.15.155:3300/0 conn(0x7f2b8c0cdd70 0x7f2b8c0ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.022+0000 7f2b8a000640 1 -- 172.21.15.86:0/2124609355 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8c06dc20 con 0x7f2b8c070e20 2024-04-25T14:09:29.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.022+0000 7f2b8aa00640 1 --2- 172.21.15.86:0/2124609355 >> v2:172.21.15.155:3300/0 conn(0x7f2b8c0cdd70 0x7f2b8c0ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:29.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.026+0000 7f2b8a000640 1 --2- 172.21.15.86:0/2124609355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b8c070e20 0x7f2b8c071230 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c074610 tx=0x7f2b7801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e23feea21da0fdb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:29.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.026+0000 7f2b89600640 1 -- 172.21.15.86:0/2124609355 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b780b0070 con 0x7f2b8c070e20 2024-04-25T14:09:29.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.026+0000 7f2b89600640 1 -- 172.21.15.86:0/2124609355 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b780ab600 con 0x7f2b8c070e20 2024-04-25T14:09:29.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.026+0000 7f2b89600640 1 -- 172.21.15.86:0/2124609355 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b780ab9d0 con 0x7f2b8c070e20 2024-04-25T14:09:29.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.026+0000 7f2b91a00640 1 -- 172.21.15.86:0/2124609355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b8c070e20 msgr2=0x7f2b8c071230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:29.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.026+0000 7f2b91a00640 1 --2- 172.21.15.86:0/2124609355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b8c070e20 0x7f2b8c071230 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c074610 tx=0x7f2b7801c490 comp rx=0 tx=0).stop 2024-04-25T14:09:29.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.026+0000 7f2b91a00640 1 -- 172.21.15.86:0/2124609355 shutdown_connections 2024-04-25T14:09:29.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.026+0000 7f2b91a00640 1 --2- 172.21.15.86:0/2124609355 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b8c06e700 0x7f2b8c06eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.026+0000 7f2b91a00640 1 --2- 172.21.15.86:0/2124609355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b8c070e20 0x7f2b8c071230 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.026+0000 7f2b91a00640 1 --2- 172.21.15.86:0/2124609355 >> v2:172.21.15.155:3300/0 conn(0x7f2b8c0cdd70 0x7f2b8c0ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.026+0000 7f2b91a00640 1 -- 172.21.15.86:0/2124609355 >> 172.21.15.86:0/2124609355 conn(0x7f2b8c0cffc0 msgr2=0x7f2b8c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:29.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.026+0000 7f2b91a00640 1 -- 172.21.15.86:0/2124609355 shutdown_connections 2024-04-25T14:09:29.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.026+0000 7f2b91a00640 1 -- 172.21.15.86:0/2124609355 wait complete. 2024-04-25T14:09:29.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.030+0000 7f2b91a00640 1 Processor -- start 2024-04-25T14:09:29.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.030+0000 7f2b91a00640 1 -- start start 2024-04-25T14:09:29.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.030+0000 7f2b91a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b8c06e700 0x7f2b8c1622c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:29.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.030+0000 7f2b91a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b8c070e20 0x7f2b8c153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:29.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.030+0000 7f2b91a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b8c0cdd70 0x7f2b8c0b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:29.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.030+0000 7f2b91a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c1642f0 con 0x7f2b8c070e20 2024-04-25T14:09:29.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.030+0000 7f2b91a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c164170 con 0x7f2b8c06e700 2024-04-25T14:09:29.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.030+0000 7f2b91a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2b8c164470 con 0x7f2b8c0cdd70 2024-04-25T14:09:29.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.030+0000 7f2b8a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b8c070e20 0x7f2b8c153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:29.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.030+0000 7f2b8a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b8c070e20 0x7f2b8c153d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58014/0 (socket says 172.21.15.86:58014) 2024-04-25T14:09:29.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.030+0000 7f2b8a000640 1 -- 172.21.15.86:0/2006006296 learned_addr learned my addr 172.21.15.86:0/2006006296 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:29.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.030+0000 7f2b8aa00640 1 --2- 172.21.15.86:0/2006006296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b8c06e700 0x7f2b8c1622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:29.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.030+0000 7f2b8b400640 1 --2- 172.21.15.86:0/2006006296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b8c0cdd70 0x7f2b8c0b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:29.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.034+0000 7f2b8a000640 1 -- 172.21.15.86:0/2006006296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b8c0cdd70 msgr2=0x7f2b8c0b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:29.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.034+0000 7f2b8a000640 1 --2- 172.21.15.86:0/2006006296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b8c0cdd70 0x7f2b8c0b3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.034+0000 7f2b8a000640 1 -- 172.21.15.86:0/2006006296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b8c06e700 msgr2=0x7f2b8c1622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:29.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.034+0000 7f2b8a000640 1 --2- 172.21.15.86:0/2006006296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b8c06e700 0x7f2b8c1622c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.034+0000 7f2b8a000640 1 -- 172.21.15.86:0/2006006296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8c154280 con 0x7f2b8c070e20 2024-04-25T14:09:29.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.034+0000 7f2b8a000640 1 --2- 172.21.15.86:0/2006006296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b8c070e20 0x7f2b8c153d40 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f2b7801c970 tx=0x7f2b78019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:29.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.034+0000 7f2b82a00640 1 -- 172.21.15.86:0/2006006296 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b780b0040 con 0x7f2b8c070e20 2024-04-25T14:09:29.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.034+0000 7f2b82a00640 1 -- 172.21.15.86:0/2006006296 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b78004220 con 0x7f2b8c070e20 2024-04-25T14:09:29.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.034+0000 7f2b82a00640 1 -- 172.21.15.86:0/2006006296 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b780ab7b0 con 0x7f2b8c070e20 2024-04-25T14:09:29.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.034+0000 7f2b91a00640 1 -- 172.21.15.86:0/2006006296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b8c161c80 con 0x7f2b8c070e20 2024-04-25T14:09:29.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.034+0000 7f2b91a00640 1 -- 172.21.15.86:0/2006006296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b8c0c7e50 con 0x7f2b8c070e20 2024-04-25T14:09:29.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.034+0000 7f2b91a00640 1 -- 172.21.15.86:0/2006006296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b8c0720e0 con 0x7f2b8c070e20 2024-04-25T14:09:29.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.034+0000 7f2b82a00640 1 -- 172.21.15.86:0/2006006296 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7f2b780bb060 con 0x7f2b8c070e20 2024-04-25T14:09:29.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.038+0000 7f2b82a00640 1 --2- 172.21.15.86:0/2006006296 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f2b6c06f520 0x7f2b6c071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:29.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.038+0000 7f2b82a00640 1 -- 172.21.15.86:0/2006006296 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7f2b78128600 con 0x7f2b8c070e20 2024-04-25T14:09:29.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.038+0000 7f2b8aa00640 1 --2- 172.21.15.86:0/2006006296 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f2b6c06f520 0x7f2b6c071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:29.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.038+0000 7f2b8aa00640 1 --2- 172.21.15.86:0/2006006296 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f2b6c06f520 0x7f2b6c071a30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c0045f0 tx=0x7f2b7c008040 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:29.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.042+0000 7f2b82a00640 1 -- 172.21.15.86:0/2006006296 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2b78016110 con 0x7f2b8c070e20 2024-04-25T14:09:29.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.230+0000 7f2b91a00640 1 -- 172.21.15.86:0/2006006296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b8c06f770 con 0x7f2b8c070e20 2024-04-25T14:09:29.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.230+0000 7f2b82a00640 1 -- 172.21.15.86:0/2006006296 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5189 (secure 0 0 0) 0x7f2b780fd120 con 0x7f2b8c070e20 2024-04-25T14:09:29.228 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:29.228 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":76,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1961914630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1961914630},{"type":"v1","addr":"172.21.15.86:6836","nonce":1961914630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2298829380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2298829380},{"type":"v1","addr":"172.21.15.155:6834","nonce":2298829380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7914,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3761704765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3761704765},{"type":"v1","addr":"172.21.15.86:6837","nonce":3761704765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:09:25.198798+0000","modified":"2024-04-25T14:09:28.237583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6835/1049360040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1049360040},{"type":"v1","addr":"172.21.15.155:6835","nonce":1049360040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7932,"qdb_cluster":[7932]},"id":7}]} 2024-04-25T14:09:29.228 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 76 2024-04-25T14:09:29.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.234+0000 7f2b91a00640 1 -- 172.21.15.86:0/2006006296 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f2b6c06f520 msgr2=0x7f2b6c071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:29.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.234+0000 7f2b91a00640 1 --2- 172.21.15.86:0/2006006296 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f2b6c06f520 0x7f2b6c071a30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c0045f0 tx=0x7f2b7c008040 comp rx=0 tx=0).stop 2024-04-25T14:09:29.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.234+0000 7f2b91a00640 1 -- 172.21.15.86:0/2006006296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b8c070e20 msgr2=0x7f2b8c153d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:29.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.234+0000 7f2b91a00640 1 --2- 172.21.15.86:0/2006006296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b8c070e20 0x7f2b8c153d40 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f2b7801c970 tx=0x7f2b78019230 comp rx=0 tx=0).stop 2024-04-25T14:09:29.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.234+0000 7f2b91a00640 1 -- 172.21.15.86:0/2006006296 shutdown_connections 2024-04-25T14:09:29.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.234+0000 7f2b91a00640 1 --2- 172.21.15.86:0/2006006296 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f2b6c06f520 0x7f2b6c071a30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.234+0000 7f2b91a00640 1 --2- 172.21.15.86:0/2006006296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b8c0cdd70 0x7f2b8c0b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.234+0000 7f2b91a00640 1 --2- 172.21.15.86:0/2006006296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b8c070e20 0x7f2b8c153d40 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.234+0000 7f2b91a00640 1 --2- 172.21.15.86:0/2006006296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b8c06e700 0x7f2b8c1622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.234+0000 7f2b91a00640 1 -- 172.21.15.86:0/2006006296 >> 172.21.15.86:0/2006006296 conn(0x7f2b8c0cffc0 msgr2=0x7f2b8c0b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:29.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.234+0000 7f2b91a00640 1 -- 172.21.15.86:0/2006006296 shutdown_connections 2024-04-25T14:09:29.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.234+0000 7f2b91a00640 1 -- 172.21.15.86:0/2006006296 wait complete. 2024-04-25T14:09:29.250 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:09:29.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.398+0000 7fcdff200640 1 Processor -- start 2024-04-25T14:09:29.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.398+0000 7fcdff200640 1 -- start start 2024-04-25T14:09:29.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.402+0000 7fcdff200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcdf80d16e0 0x7fcdf80d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:29.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.402+0000 7fcdff200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcdf80730b0 0x7fcdf80734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:29.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.402+0000 7fcdff200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdf80d2030 con 0x7fcdf80cfca0 2024-04-25T14:09:29.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.402+0000 7fcdff200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcdf8070470 con 0x7fcdf80730b0 2024-04-25T14:09:29.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.402+0000 7fcdff200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcdf8072bd0 con 0x7fcdf80d16e0 2024-04-25T14:09:29.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.402+0000 7fcdf7e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fcdf80cfca0 0x7fcdf806f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42860/0 (socket says 172.21.15.86:42860) 2024-04-25T14:09:29.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.402+0000 7fcdf7e00640 1 -- 172.21.15.86:0/2913969501 learned_addr learned my addr 172.21.15.86:0/2913969501 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:29.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.402+0000 7fcdfd200640 1 --2- 172.21.15.86:0/2913969501 >> v2:172.21.15.86:3300/0 conn(0x7fcdf80730b0 0x7fcdf80734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:29.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.402+0000 7fcdf7400640 1 -- 172.21.15.86:0/2913969501 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1644254327 0 0) 0x7fcde8001120 con 0x7fcdf80cfca0 2024-04-25T14:09:29.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.402+0000 7fcdef400640 1 --2- 172.21.15.86:0/2913969501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcdf80d16e0 0x7fcdf80d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:29.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.402+0000 7fcdf7400640 1 -- 172.21.15.86:0/2913969501 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcddc005740 con 0x7fcdf80cfca0 2024-04-25T14:09:29.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.402+0000 7fcdfd200640 1 -- 172.21.15.86:0/2913969501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcdf80d16e0 msgr2=0x7fcdf80d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:29.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.402+0000 7fcdfd200640 1 --2- 172.21.15.86:0/2913969501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcdf80d16e0 0x7fcdf80d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.402+0000 7fcdfd200640 1 -- 172.21.15.86:0/2913969501 >> v1:172.21.15.86:6789/0 conn(0x7fcdf80cfca0 legacy=0x7fcdf806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:29.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.402+0000 7fcdfd200640 1 -- 172.21.15.86:0/2913969501 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf806fbd0 con 0x7fcdf80730b0 2024-04-25T14:09:29.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.402+0000 7fcdef400640 1 --2- 172.21.15.86:0/2913969501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcdf80d16e0 0x7fcdf80d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:29.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.406+0000 7fcdfd200640 1 --2- 172.21.15.86:0/2913969501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdf80730b0 0x7fcdf80734c0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009d90 tx=0x7fcdf001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ede2289b7e30b9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:29.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.406+0000 7fcdf7400640 1 -- 172.21.15.86:0/2913969501 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf00b2070 con 0x7fcdf80730b0 2024-04-25T14:09:29.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.406+0000 7fcdf7400640 1 -- 172.21.15.86:0/2913969501 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdf00a5940 con 0x7fcdf80730b0 2024-04-25T14:09:29.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.406+0000 7fcdf7400640 1 -- 172.21.15.86:0/2913969501 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf00a5d10 con 0x7fcdf80730b0 2024-04-25T14:09:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.406+0000 7fcdff200640 1 -- 172.21.15.86:0/2913969501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdf80730b0 msgr2=0x7fcdf80734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.406+0000 7fcdff200640 1 --2- 172.21.15.86:0/2913969501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdf80730b0 0x7fcdf80734c0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009d90 tx=0x7fcdf001c970 comp rx=0 tx=0).stop 2024-04-25T14:09:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.406+0000 7fcdff200640 1 -- 172.21.15.86:0/2913969501 shutdown_connections 2024-04-25T14:09:29.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.406+0000 7fcdff200640 1 --2- 172.21.15.86:0/2913969501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdf80730b0 0x7fcdf80734c0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.406+0000 7fcdff200640 1 --2- 172.21.15.86:0/2913969501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcdf80d16e0 0x7fcdf80d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.406+0000 7fcdff200640 1 -- 172.21.15.86:0/2913969501 >> 172.21.15.86:0/2913969501 conn(0x7fcdf80b4650 msgr2=0x7fcdf80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:29.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.406+0000 7fcdff200640 1 -- 172.21.15.86:0/2913969501 shutdown_connections 2024-04-25T14:09:29.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.406+0000 7fcdff200640 1 -- 172.21.15.86:0/2913969501 wait complete. 2024-04-25T14:09:29.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.410+0000 7fcdff200640 1 Processor -- start 2024-04-25T14:09:29.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.410+0000 7fcdff200640 1 -- start start 2024-04-25T14:09:29.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.410+0000 7fcdff200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdf80730b0 0x7fcdf809a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:29.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.410+0000 7fcdff200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcdf80cfca0 0x7fcdf815b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:29.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.410+0000 7fcdff200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdf80d16e0 0x7fcdf80b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:29.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.410+0000 7fcdff200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf80ce130 con 0x7fcdf80d16e0 2024-04-25T14:09:29.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.410+0000 7fcdff200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf80cdfb0 con 0x7fcdf80730b0 2024-04-25T14:09:29.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.410+0000 7fcdff200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcdf80ce2b0 con 0x7fcdf80cfca0 2024-04-25T14:09:29.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.410+0000 7fcdfd200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdf80d16e0 0x7fcdf80b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:29.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.410+0000 7fcdfd200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdf80d16e0 0x7fcdf80b0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58024/0 (socket says 172.21.15.86:58024) 2024-04-25T14:09:29.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.410+0000 7fcdfd200640 1 -- 172.21.15.86:0/1338773319 learned_addr learned my addr 172.21.15.86:0/1338773319 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:29.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.410+0000 7fcdef400640 1 --2- 172.21.15.86:0/1338773319 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcdf80cfca0 0x7fcdf815b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:29.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.410+0000 7fcdf7e00640 1 --2- 172.21.15.86:0/1338773319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdf80730b0 0x7fcdf809a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:29.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.410+0000 7fcdfd200640 1 -- 172.21.15.86:0/1338773319 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcdf80cfca0 msgr2=0x7fcdf815b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:29.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.410+0000 7fcdfd200640 1 --2- 172.21.15.86:0/1338773319 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcdf80cfca0 0x7fcdf815b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.410+0000 7fcdfd200640 1 -- 172.21.15.86:0/1338773319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdf80730b0 msgr2=0x7fcdf809a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:29.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.410+0000 7fcdfd200640 1 --2- 172.21.15.86:0/1338773319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdf80730b0 0x7fcdf809a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.410+0000 7fcdfd200640 1 -- 172.21.15.86:0/1338773319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf809aeb0 con 0x7fcdf80d16e0 2024-04-25T14:09:29.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.410+0000 7fcdf7e00640 1 --2- 172.21.15.86:0/1338773319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdf80730b0 0x7fcdf809a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:29.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.410+0000 7fcdfd200640 1 --2- 172.21.15.86:0/1338773319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdf80d16e0 0x7fcdf80b0da0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00b0040 tx=0x7fcdf0019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:29.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.410+0000 7fcdf4c00640 1 -- 172.21.15.86:0/1338773319 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf00b2050 con 0x7fcdf80d16e0 2024-04-25T14:09:29.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.410+0000 7fcdf4c00640 1 -- 172.21.15.86:0/1338773319 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdf00be070 con 0x7fcdf80d16e0 2024-04-25T14:09:29.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.410+0000 7fcdf4c00640 1 -- 172.21.15.86:0/1338773319 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf00ada90 con 0x7fcdf80d16e0 2024-04-25T14:09:29.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.410+0000 7fcdff200640 1 -- 172.21.15.86:0/1338773319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdf815bd60 con 0x7fcdf80d16e0 2024-04-25T14:09:29.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.410+0000 7fcdff200640 1 -- 172.21.15.86:0/1338773319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdf8147dd0 con 0x7fcdf80d16e0 2024-04-25T14:09:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.414+0000 7fcdff200640 1 -- 172.21.15.86:0/1338773319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdf809f4b0 con 0x7fcdf80d16e0 2024-04-25T14:09:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.414+0000 7fcdf4c00640 1 -- 172.21.15.86:0/1338773319 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7fcdf00adc30 con 0x7fcdf80d16e0 2024-04-25T14:09:29.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.414+0000 7fcdf4c00640 1 --2- 172.21.15.86:0/1338773319 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fcdd406f550 0x7fcdd4071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:29.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.414+0000 7fcdf4c00640 1 -- 172.21.15.86:0/1338773319 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7fcdf012a170 con 0x7fcdf80d16e0 2024-04-25T14:09:29.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.414+0000 7fcdf7e00640 1 --2- 172.21.15.86:0/1338773319 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fcdd406f550 0x7fcdd4071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:29.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.414+0000 7fcdf7e00640 1 --2- 172.21.15.86:0/1338773319 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fcdd406f550 0x7fcdd4071a60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcde8005890 tx=0x7fcde8009040 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:29.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.418+0000 7fcdf4c00640 1 -- 172.21.15.86:0/1338773319 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcdf0016560 con 0x7fcdf80d16e0 2024-04-25T14:09:29.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.578+0000 7fcdff200640 1 -- 172.21.15.86:0/1338773319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcdf80734c0 con 0x7fcdf80d16e0 2024-04-25T14:09:29.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.578+0000 7fcdf4c00640 1 -- 172.21.15.86:0/1338773319 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+16875 (secure 0 0 0) 0x7fcdf00f6db0 con 0x7fcdf80d16e0 2024-04-25T14:09:29.577 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:29.577 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":100,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:09:26.827294+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:09:23.588241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T14:09:24.156880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:09:29.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.582+0000 7fcdff200640 1 -- 172.21.15.86:0/1338773319 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fcdd406f550 msgr2=0x7fcdd4071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:29.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.582+0000 7fcdff200640 1 --2- 172.21.15.86:0/1338773319 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fcdd406f550 0x7fcdd4071a60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcde8005890 tx=0x7fcde8009040 comp rx=0 tx=0).stop 2024-04-25T14:09:29.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.582+0000 7fcdff200640 1 -- 172.21.15.86:0/1338773319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdf80d16e0 msgr2=0x7fcdf80b0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:29.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.582+0000 7fcdff200640 1 --2- 172.21.15.86:0/1338773319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdf80d16e0 0x7fcdf80b0da0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00b0040 tx=0x7fcdf0019710 comp rx=0 tx=0).stop 2024-04-25T14:09:29.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.582+0000 7fcdff200640 1 -- 172.21.15.86:0/1338773319 shutdown_connections 2024-04-25T14:09:29.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.582+0000 7fcdff200640 1 --2- 172.21.15.86:0/1338773319 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fcdd406f550 0x7fcdd4071a60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.582+0000 7fcdff200640 1 --2- 172.21.15.86:0/1338773319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdf80d16e0 0x7fcdf80b0da0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.582+0000 7fcdff200640 1 --2- 172.21.15.86:0/1338773319 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcdf80cfca0 0x7fcdf815b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.582+0000 7fcdff200640 1 --2- 172.21.15.86:0/1338773319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdf80730b0 0x7fcdf809a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.582+0000 7fcdff200640 1 -- 172.21.15.86:0/1338773319 >> 172.21.15.86:0/1338773319 conn(0x7fcdf80b4650 msgr2=0x7fcdf8068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:29.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.582+0000 7fcdff200640 1 -- 172.21.15.86:0/1338773319 shutdown_connections 2024-04-25T14:09:29.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.582+0000 7fcdff200640 1 -- 172.21.15.86:0/1338773319 wait complete. 2024-04-25T14:09:29.600 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T14:09:29.600 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:09:29.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.750+0000 7f1e97c00640 1 Processor -- start 2024-04-25T14:09:29.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.750+0000 7f1e97c00640 1 -- start start 2024-04-25T14:09:29.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.750+0000 7f1e97c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e900d1df0 0x7f1e900cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:29.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.750+0000 7f1e97c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1e900cdcc0 0x7f1e900ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:29.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.750+0000 7f1e97c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e9015e5c0 con 0x7f1e90070330 2024-04-25T14:09:29.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.750+0000 7f1e97c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1e9006fd40 con 0x7f1e900cdcc0 2024-04-25T14:09:29.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.750+0000 7f1e97c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1e9015e8a0 con 0x7f1e900d1df0 2024-04-25T14:09:29.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.750+0000 7f1e95c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1e900cdcc0 0x7f1e900ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:29.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.750+0000 7f1e95c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1e900cdcc0 0x7f1e900ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58040/0 (socket says 172.21.15.86:58040) 2024-04-25T14:09:29.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.750+0000 7f1e95c00640 1 -- 172.21.15.86:0/2096497431 learned_addr learned my addr 172.21.15.86:0/2096497431 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:29.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.750+0000 7f1e87e00640 1 --2- 172.21.15.86:0/2096497431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e900d1df0 0x7f1e900cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:29.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.750+0000 7f1e95c00640 1 -- 172.21.15.86:0/2096497431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e900d1df0 msgr2=0x7f1e900cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:29.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.750+0000 7f1e95c00640 1 --2- 172.21.15.86:0/2096497431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e900d1df0 0x7f1e900cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.750+0000 7f1e95c00640 1 -- 172.21.15.86:0/2096497431 >> v1:172.21.15.155:6789/0 conn(0x7f1e90070330 legacy=0x7f1e900d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:29.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.750+0000 7f1e95c00640 1 -- 172.21.15.86:0/2096497431 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e9015eb80 con 0x7f1e900cdcc0 2024-04-25T14:09:29.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.754+0000 7f1e95c00640 1 --2- 172.21.15.86:0/2096497431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e900cdcc0 0x7f1e900ce0d0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f1e88009e50 tx=0x7f1e8801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5414b8b668842519 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:29.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.754+0000 7f1e8fe00640 1 -- 172.21.15.86:0/2096497431 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e880a5810 con 0x7f1e900cdcc0 2024-04-25T14:09:29.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.754+0000 7f1e8fe00640 1 -- 172.21.15.86:0/2096497431 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e880a59b0 con 0x7f1e900cdcc0 2024-04-25T14:09:29.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.754+0000 7f1e8fe00640 1 -- 172.21.15.86:0/2096497431 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e880a5d80 con 0x7f1e900cdcc0 2024-04-25T14:09:29.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.754+0000 7f1e97c00640 1 -- 172.21.15.86:0/2096497431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e900cdcc0 msgr2=0x7f1e900ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:29.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.754+0000 7f1e97c00640 1 --2- 172.21.15.86:0/2096497431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e900cdcc0 0x7f1e900ce0d0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f1e88009e50 tx=0x7f1e8801c970 comp rx=0 tx=0).stop 2024-04-25T14:09:29.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.754+0000 7f1e97c00640 1 -- 172.21.15.86:0/2096497431 shutdown_connections 2024-04-25T14:09:29.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.754+0000 7f1e97c00640 1 --2- 172.21.15.86:0/2096497431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e900cdcc0 0x7f1e900ce0d0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.754+0000 7f1e97c00640 1 --2- 172.21.15.86:0/2096497431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e900d1df0 0x7f1e900cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.754+0000 7f1e97c00640 1 -- 172.21.15.86:0/2096497431 >> 172.21.15.86:0/2096497431 conn(0x7f1e900b4650 msgr2=0x7f1e900b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:29.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.754+0000 7f1e97c00640 1 -- 172.21.15.86:0/2096497431 shutdown_connections 2024-04-25T14:09:29.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.754+0000 7f1e97c00640 1 -- 172.21.15.86:0/2096497431 wait complete. 2024-04-25T14:09:29.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.758+0000 7f1e97c00640 1 Processor -- start 2024-04-25T14:09:29.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.758+0000 7f1e97c00640 1 -- start start 2024-04-25T14:09:29.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.758+0000 7f1e97c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e90070330 0x7f1e9015aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:29.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.758+0000 7f1e97c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e900cdcc0 0x7f1e900af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:29.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.758+0000 7f1e97c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e9015b1c0 0x7f1e900af710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:29.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.758+0000 7f1e97c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e9015f4b0 con 0x7f1e900cdcc0 2024-04-25T14:09:29.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.758+0000 7f1e97c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1e9015f330 con 0x7f1e90070330 2024-04-25T14:09:29.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.758+0000 7f1e97c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1e90070740 con 0x7f1e9015b1c0 2024-04-25T14:09:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.758+0000 7f1e87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e900cdcc0 0x7f1e900af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.758+0000 7f1e87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e900cdcc0 0x7f1e900af1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58052/0 (socket says 172.21.15.86:58052) 2024-04-25T14:09:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.758+0000 7f1e87e00640 1 -- 172.21.15.86:0/3358817550 learned_addr learned my addr 172.21.15.86:0/3358817550 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.758+0000 7f1e95c00640 1 --2- 172.21.15.86:0/3358817550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e9015b1c0 0x7f1e900af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.758+0000 7f1e95200640 1 --2- 172.21.15.86:0/3358817550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e90070330 0x7f1e9015aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.758+0000 7f1e87e00640 1 -- 172.21.15.86:0/3358817550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e9015b1c0 msgr2=0x7f1e900af710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.758+0000 7f1e87e00640 1 --2- 172.21.15.86:0/3358817550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e9015b1c0 0x7f1e900af710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.758+0000 7f1e87e00640 1 -- 172.21.15.86:0/3358817550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e90070330 msgr2=0x7f1e9015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.758+0000 7f1e87e00640 1 --2- 172.21.15.86:0/3358817550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e90070330 0x7f1e9015aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.758+0000 7f1e87e00640 1 -- 172.21.15.86:0/3358817550 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e900c6740 con 0x7f1e900cdcc0 2024-04-25T14:09:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.758+0000 7f1e95c00640 1 --2- 172.21.15.86:0/3358817550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e9015b1c0 0x7f1e900af710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.758+0000 7f1e87e00640 1 --2- 172.21.15.86:0/3358817550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e900cdcc0 0x7f1e900af1d0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800d620 tx=0x7f1e7800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:29.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.758+0000 7f1e8d600640 1 -- 172.21.15.86:0/3358817550 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7800b620 con 0x7f1e900cdcc0 2024-04-25T14:09:29.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.758+0000 7f1e8d600640 1 -- 172.21.15.86:0/3358817550 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e78019070 con 0x7f1e900cdcc0 2024-04-25T14:09:29.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.758+0000 7f1e8d600640 1 -- 172.21.15.86:0/3358817550 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7800b9e0 con 0x7f1e900cdcc0 2024-04-25T14:09:29.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.758+0000 7f1e97c00640 1 -- 172.21.15.86:0/3358817550 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e9015afc0 con 0x7f1e900cdcc0 2024-04-25T14:09:29.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.758+0000 7f1e97c00640 1 -- 172.21.15.86:0/3358817550 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e900c6ca0 con 0x7f1e900cdcc0 2024-04-25T14:09:29.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.762+0000 7f1e97c00640 1 -- 172.21.15.86:0/3358817550 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e90069d00 con 0x7f1e900cdcc0 2024-04-25T14:09:29.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.762+0000 7f1e8d600640 1 -- 172.21.15.86:0/3358817550 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7f1e7800bb80 con 0x7f1e900cdcc0 2024-04-25T14:09:29.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.762+0000 7f1e8d600640 1 --2- 172.21.15.86:0/3358817550 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f1e6c06f550 0x7f1e6c071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:29.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.762+0000 7f1e8d600640 1 -- 172.21.15.86:0/3358817550 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7f1e7808fa70 con 0x7f1e900cdcc0 2024-04-25T14:09:29.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.762+0000 7f1e95200640 1 --2- 172.21.15.86:0/3358817550 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f1e6c06f550 0x7f1e6c071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:29.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.762+0000 7f1e95200640 1 --2- 172.21.15.86:0/3358817550 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f1e6c06f550 0x7f1e6c071a60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1e90072200 tx=0x7f1e80009040 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:29.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.766+0000 7f1e8d600640 1 -- 172.21.15.86:0/3358817550 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1e9015afc0 con 0x7f1e900cdcc0 2024-04-25T14:09:29.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.922+0000 7f1e97c00640 1 -- 172.21.15.86:0/3358817550 --> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1e90076bf0 con 0x7f1e6c06f550 2024-04-25T14:09:29.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.930+0000 7f1e8d600640 1 -- 172.21.15.86:0/3358817550 <== mgr.7650 v2:172.21.15.86:6832/4236874185 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246766 (secure 0 0 0) 0x7f1e90076bf0 con 0x7f1e6c06f550 2024-04-25T14:09:29.930 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:29.931 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:09:29.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.934+0000 7f1e97c00640 1 -- 172.21.15.86:0/3358817550 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f1e6c06f550 msgr2=0x7f1e6c071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:29.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.934+0000 7f1e97c00640 1 --2- 172.21.15.86:0/3358817550 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f1e6c06f550 0x7f1e6c071a60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1e90072200 tx=0x7f1e80009040 comp rx=0 tx=0).stop 2024-04-25T14:09:29.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.934+0000 7f1e97c00640 1 -- 172.21.15.86:0/3358817550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e900cdcc0 msgr2=0x7f1e900af1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:29.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.934+0000 7f1e97c00640 1 --2- 172.21.15.86:0/3358817550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e900cdcc0 0x7f1e900af1d0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800d620 tx=0x7f1e7800dae0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.934+0000 7f1e97c00640 1 -- 172.21.15.86:0/3358817550 shutdown_connections 2024-04-25T14:09:29.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.934+0000 7f1e97c00640 1 --2- 172.21.15.86:0/3358817550 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f1e6c06f550 0x7f1e6c071a60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.934+0000 7f1e97c00640 1 --2- 172.21.15.86:0/3358817550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e9015b1c0 0x7f1e900af710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.934+0000 7f1e97c00640 1 --2- 172.21.15.86:0/3358817550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e900cdcc0 0x7f1e900af1d0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.934+0000 7f1e97c00640 1 --2- 172.21.15.86:0/3358817550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e90070330 0x7f1e9015aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:29.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.934+0000 7f1e97c00640 1 -- 172.21.15.86:0/3358817550 >> 172.21.15.86:0/3358817550 conn(0x7f1e900b4650 msgr2=0x7f1e900ae250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:29.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.934+0000 7f1e97c00640 1 -- 172.21.15.86:0/3358817550 shutdown_connections 2024-04-25T14:09:29.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:29.934+0000 7f1e97c00640 1 -- 172.21.15.86:0/3358817550 wait complete. 2024-04-25T14:09:29.951 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T14:09:29.506130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":366,"num_read_kb":935,"num_write":161,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260076,"kb_used_data":10592,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714644,"statfs":{"total":773094113280,"available":772827795456,"internally_reserved":0,"allocated":10846208,"data_stored":5978918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.716486"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.811745+0000","last_change":"2024-04-25T14:09:24.811745+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.811745+0000","last_undegraded":"2024-04-25T14:09:24.811745+0000","last_fullsized":"2024-04-25T14:09:24.811745+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.805434+0000","last_change":"2024-04-25T14:09:24.805434+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.805434+0000","last_undegraded":"2024-04-25T14:09:24.805434+0000","last_fullsized":"2024-04-25T14:09:24.805434+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.806609+0000","last_change":"2024-04-25T14:09:24.806609+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.806609+0000","last_undegraded":"2024-04-25T14:09:24.806609+0000","last_fullsized":"2024-04-25T14:09:24.806609+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.814243+0000","last_change":"2024-04-25T14:09:24.814243+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.814243+0000","last_undegraded":"2024-04-25T14:09:24.814243+0000","last_fullsized":"2024-04-25T14:09:24.814243+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.796780+0000","last_change":"2024-04-25T14:09:24.796780+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.796780+0000","last_undegraded":"2024-04-25T14:09:24.796780+0000","last_fullsized":"2024-04-25T14:09:24.796780+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801913+0000","last_change":"2024-04-25T14:09:24.801913+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801913+0000","last_undegraded":"2024-04-25T14:09:24.801913+0000","last_fullsized":"2024-04-25T14:09:24.801913+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.805381+0000","last_change":"2024-04-25T14:09:24.805381+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.805381+0000","last_undegraded":"2024-04-25T14:09:24.805381+0000","last_fullsized":"2024-04-25T14:09:24.805381+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.805573+0000","last_change":"2024-04-25T14:09:24.805573+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.805573+0000","last_undegraded":"2024-04-25T14:09:24.805573+0000","last_fullsized":"2024-04-25T14:09:24.805573+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.802656+0000","last_change":"2024-04-25T14:09:24.802656+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.802656+0000","last_undegraded":"2024-04-25T14:09:24.802656+0000","last_fullsized":"2024-04-25T14:09:24.802656+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.796198+0000","last_change":"2024-04-25T14:09:24.796198+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.796198+0000","last_undegraded":"2024-04-25T14:09:24.796198+0000","last_fullsized":"2024-04-25T14:09:24.796198+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.797705+0000","last_change":"2024-04-25T14:09:24.797705+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.797705+0000","last_undegraded":"2024-04-25T14:09:24.797705+0000","last_fullsized":"2024-04-25T14:09:24.797705+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.803323+0000","last_change":"2024-04-25T14:09:24.803323+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.803323+0000","last_undegraded":"2024-04-25T14:09:24.803323+0000","last_fullsized":"2024-04-25T14:09:24.803323+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.814711+0000","last_change":"2024-04-25T14:09:24.814711+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.814711+0000","last_undegraded":"2024-04-25T14:09:24.814711+0000","last_fullsized":"2024-04-25T14:09:24.814711+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801188+0000","last_change":"2024-04-25T14:09:24.801188+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801188+0000","last_undegraded":"2024-04-25T14:09:24.801188+0000","last_fullsized":"2024-04-25T14:09:24.801188+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.800597+0000","last_change":"2024-04-25T14:09:24.800597+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.800597+0000","last_undegraded":"2024-04-25T14:09:24.800597+0000","last_fullsized":"2024-04-25T14:09:24.800597+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.799763+0000","last_change":"2024-04-25T14:09:24.799763+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.799763+0000","last_undegraded":"2024-04-25T14:09:24.799763+0000","last_fullsized":"2024-04-25T14:09:24.799763+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.806722+0000","last_change":"2024-04-25T14:09:24.806722+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.806722+0000","last_undegraded":"2024-04-25T14:09:24.806722+0000","last_fullsized":"2024-04-25T14:09:24.806722+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801800+0000","last_change":"2024-04-25T14:09:24.801800+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801800+0000","last_undegraded":"2024-04-25T14:09:24.801800+0000","last_fullsized":"2024-04-25T14:09:24.801800+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.805192+0000","last_change":"2024-04-25T14:09:24.805192+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.805192+0000","last_undegraded":"2024-04-25T14:09:24.805192+0000","last_fullsized":"2024-04-25T14:09:24.805192+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.802814+0000","last_change":"2024-04-25T14:09:24.802814+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.802814+0000","last_undegraded":"2024-04-25T14:09:24.802814+0000","last_fullsized":"2024-04-25T14:09:24.802814+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.800665+0000","last_change":"2024-04-25T14:09:24.800665+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.800665+0000","last_undegraded":"2024-04-25T14:09:24.800665+0000","last_fullsized":"2024-04-25T14:09:24.800665+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.797246+0000","last_change":"2024-04-25T14:09:24.797246+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.797246+0000","last_undegraded":"2024-04-25T14:09:24.797246+0000","last_fullsized":"2024-04-25T14:09:24.797246+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.804279+0000","last_change":"2024-04-25T14:09:24.804279+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.804279+0000","last_undegraded":"2024-04-25T14:09:24.804279+0000","last_fullsized":"2024-04-25T14:09:24.804279+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.795033+0000","last_change":"2024-04-25T14:09:24.795033+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.795033+0000","last_undegraded":"2024-04-25T14:09:24.795033+0000","last_fullsized":"2024-04-25T14:09:24.795033+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.803941+0000","last_change":"2024-04-25T14:09:24.803941+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.803941+0000","last_undegraded":"2024-04-25T14:09:24.803941+0000","last_fullsized":"2024-04-25T14:09:24.803941+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.803227+0000","last_change":"2024-04-25T14:09:24.803227+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.803227+0000","last_undegraded":"2024-04-25T14:09:24.803227+0000","last_fullsized":"2024-04-25T14:09:24.803227+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.797093+0000","last_change":"2024-04-25T14:09:24.797093+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.797093+0000","last_undegraded":"2024-04-25T14:09:24.797093+0000","last_fullsized":"2024-04-25T14:09:24.797093+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.808887+0000","last_change":"2024-04-25T14:09:24.808887+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.808887+0000","last_undegraded":"2024-04-25T14:09:24.808887+0000","last_fullsized":"2024-04-25T14:09:24.808887+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.800116+0000","last_change":"2024-04-25T14:09:24.800116+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.800116+0000","last_undegraded":"2024-04-25T14:09:24.800116+0000","last_fullsized":"2024-04-25T14:09:24.800116+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.805810+0000","last_change":"2024-04-25T14:09:24.805810+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.805810+0000","last_undegraded":"2024-04-25T14:09:24.805810+0000","last_fullsized":"2024-04-25T14:09:24.805810+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.804724+0000","last_change":"2024-04-25T14:09:24.804724+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.804724+0000","last_undegraded":"2024-04-25T14:09:24.804724+0000","last_fullsized":"2024-04-25T14:09:24.804724+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.805096+0000","last_change":"2024-04-25T14:09:24.805096+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.805096+0000","last_undegraded":"2024-04-25T14:09:24.805096+0000","last_fullsized":"2024-04-25T14:09:24.805096+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801802+0000","last_change":"2024-04-25T14:09:24.801802+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801802+0000","last_undegraded":"2024-04-25T14:09:24.801802+0000","last_fullsized":"2024-04-25T14:09:24.801802+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.823829+0000","last_change":"2024-04-25T14:09:24.817466+0000","last_active":"2024-04-25T14:09:24.823829+0000","last_peered":"2024-04-25T14:09:24.823829+0000","last_clean":"2024-04-25T14:09:24.823829+0000","last_became_active":"2024-04-25T14:09:24.817167+0000","last_became_peered":"2024-04-25T14:09:24.817167+0000","last_unstale":"2024-04-25T14:09:24.823829+0000","last_undegraded":"2024-04-25T14:09:24.823829+0000","last_fullsized":"2024-04-25T14:09:24.823829+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:02.374557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.803577+0000","last_change":"2024-04-25T14:09:24.803577+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.803577+0000","last_undegraded":"2024-04-25T14:09:24.803577+0000","last_fullsized":"2024-04-25T14:09:24.803577+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.824309+0000","last_change":"2024-04-25T14:09:24.819324+0000","last_active":"2024-04-25T14:09:24.824309+0000","last_peered":"2024-04-25T14:09:24.824309+0000","last_clean":"2024-04-25T14:09:24.824309+0000","last_became_active":"2024-04-25T14:09:24.818377+0000","last_became_peered":"2024-04-25T14:09:24.818377+0000","last_unstale":"2024-04-25T14:09:24.824309+0000","last_undegraded":"2024-04-25T14:09:24.824309+0000","last_fullsized":"2024-04-25T14:09:24.824309+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:38.935678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.802115+0000","last_change":"2024-04-25T14:09:24.802115+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.802115+0000","last_undegraded":"2024-04-25T14:09:24.802115+0000","last_fullsized":"2024-04-25T14:09:24.802115+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821745+0000","last_change":"2024-04-25T14:09:24.813915+0000","last_active":"2024-04-25T14:09:24.821745+0000","last_peered":"2024-04-25T14:09:24.821745+0000","last_clean":"2024-04-25T14:09:24.821745+0000","last_became_active":"2024-04-25T14:09:24.813610+0000","last_became_peered":"2024-04-25T14:09:24.813610+0000","last_unstale":"2024-04-25T14:09:24.821745+0000","last_undegraded":"2024-04-25T14:09:24.821745+0000","last_fullsized":"2024-04-25T14:09:24.821745+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:03.242163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.796164+0000","last_change":"2024-04-25T14:09:24.796164+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.796164+0000","last_undegraded":"2024-04-25T14:09:24.796164+0000","last_fullsized":"2024-04-25T14:09:24.796164+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.813538+0000","last_change":"2024-04-25T14:09:24.810780+0000","last_active":"2024-04-25T14:09:24.813538+0000","last_peered":"2024-04-25T14:09:24.813538+0000","last_clean":"2024-04-25T14:09:24.813538+0000","last_became_active":"2024-04-25T14:09:24.810462+0000","last_became_peered":"2024-04-25T14:09:24.810462+0000","last_unstale":"2024-04-25T14:09:24.813538+0000","last_undegraded":"2024-04-25T14:09:24.813538+0000","last_fullsized":"2024-04-25T14:09:24.813538+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:30.379959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.809107+0000","last_change":"2024-04-25T14:09:24.809107+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.809107+0000","last_undegraded":"2024-04-25T14:09:24.809107+0000","last_fullsized":"2024-04-25T14:09:24.809107+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.820528+0000","last_change":"2024-04-25T14:09:24.813955+0000","last_active":"2024-04-25T14:09:24.820528+0000","last_peered":"2024-04-25T14:09:24.820528+0000","last_clean":"2024-04-25T14:09:24.820528+0000","last_became_active":"2024-04-25T14:09:24.813645+0000","last_became_peered":"2024-04-25T14:09:24.813645+0000","last_unstale":"2024-04-25T14:09:24.820528+0000","last_undegraded":"2024-04-25T14:09:24.820528+0000","last_fullsized":"2024-04-25T14:09:24.820528+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:02.374557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801810+0000","last_change":"2024-04-25T14:09:24.801810+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801810+0000","last_undegraded":"2024-04-25T14:09:24.801810+0000","last_fullsized":"2024-04-25T14:09:24.801810+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821172+0000","last_change":"2024-04-25T14:09:24.814236+0000","last_active":"2024-04-25T14:09:24.821172+0000","last_peered":"2024-04-25T14:09:24.821172+0000","last_clean":"2024-04-25T14:09:24.821172+0000","last_became_active":"2024-04-25T14:09:24.813909+0000","last_became_peered":"2024-04-25T14:09:24.813909+0000","last_unstale":"2024-04-25T14:09:24.821172+0000","last_undegraded":"2024-04-25T14:09:24.821172+0000","last_fullsized":"2024-04-25T14:09:24.821172+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:57.671866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801284+0000","last_change":"2024-04-25T14:09:24.801284+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801284+0000","last_undegraded":"2024-04-25T14:09:24.801284+0000","last_fullsized":"2024-04-25T14:09:24.801284+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821282+0000","last_change":"2024-04-25T14:09:24.811288+0000","last_active":"2024-04-25T14:09:24.821282+0000","last_peered":"2024-04-25T14:09:24.821282+0000","last_clean":"2024-04-25T14:09:24.821282+0000","last_became_active":"2024-04-25T14:09:24.809633+0000","last_became_peered":"2024-04-25T14:09:24.809633+0000","last_unstale":"2024-04-25T14:09:24.821282+0000","last_undegraded":"2024-04-25T14:09:24.821282+0000","last_fullsized":"2024-04-25T14:09:24.821282+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:57.671866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.809114+0000","last_change":"2024-04-25T14:09:24.809114+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.809114+0000","last_undegraded":"2024-04-25T14:09:24.809114+0000","last_fullsized":"2024-04-25T14:09:24.809114+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.806864+0000","last_change":"2024-04-25T14:09:24.799767+0000","last_active":"2024-04-25T14:09:24.806864+0000","last_peered":"2024-04-25T14:09:24.806864+0000","last_clean":"2024-04-25T14:09:24.806864+0000","last_became_active":"2024-04-25T14:09:24.799440+0000","last_became_peered":"2024-04-25T14:09:24.799440+0000","last_unstale":"2024-04-25T14:09:24.806864+0000","last_undegraded":"2024-04-25T14:09:24.806864+0000","last_fullsized":"2024-04-25T14:09:24.806864+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:44.194963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.802420+0000","last_change":"2024-04-25T14:09:24.802420+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.802420+0000","last_undegraded":"2024-04-25T14:09:24.802420+0000","last_fullsized":"2024-04-25T14:09:24.802420+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821771+0000","last_change":"2024-04-25T14:09:24.813861+0000","last_active":"2024-04-25T14:09:24.821771+0000","last_peered":"2024-04-25T14:09:24.821771+0000","last_clean":"2024-04-25T14:09:24.821771+0000","last_became_active":"2024-04-25T14:09:24.813523+0000","last_became_peered":"2024-04-25T14:09:24.813523+0000","last_unstale":"2024-04-25T14:09:24.821771+0000","last_undegraded":"2024-04-25T14:09:24.821771+0000","last_fullsized":"2024-04-25T14:09:24.821771+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:03.406811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.795235+0000","last_change":"2024-04-25T14:09:24.795235+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.795235+0000","last_undegraded":"2024-04-25T14:09:24.795235+0000","last_fullsized":"2024-04-25T14:09:24.795235+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.823970+0000","last_change":"2024-04-25T14:09:24.819250+0000","last_active":"2024-04-25T14:09:24.823970+0000","last_peered":"2024-04-25T14:09:24.823970+0000","last_clean":"2024-04-25T14:09:24.823970+0000","last_became_active":"2024-04-25T14:09:24.818892+0000","last_became_peered":"2024-04-25T14:09:24.818892+0000","last_unstale":"2024-04-25T14:09:24.823970+0000","last_undegraded":"2024-04-25T14:09:24.823970+0000","last_fullsized":"2024-04-25T14:09:24.823970+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:12.433048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.799314+0000","last_change":"2024-04-25T14:09:24.799314+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.799314+0000","last_undegraded":"2024-04-25T14:09:24.799314+0000","last_fullsized":"2024-04-25T14:09:24.799314+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.812722+0000","last_change":"2024-04-25T14:09:24.808328+0000","last_active":"2024-04-25T14:09:24.812722+0000","last_peered":"2024-04-25T14:09:24.812722+0000","last_clean":"2024-04-25T14:09:24.812722+0000","last_became_active":"2024-04-25T14:09:24.807984+0000","last_became_peered":"2024-04-25T14:09:24.807984+0000","last_unstale":"2024-04-25T14:09:24.812722+0000","last_undegraded":"2024-04-25T14:09:24.812722+0000","last_fullsized":"2024-04-25T14:09:24.812722+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:50.180573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.795828+0000","last_change":"2024-04-25T14:09:24.795828+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.795828+0000","last_undegraded":"2024-04-25T14:09:24.795828+0000","last_fullsized":"2024-04-25T14:09:24.795828+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.819182+0000","last_change":"2024-04-25T14:09:24.817277+0000","last_active":"2024-04-25T14:09:24.819182+0000","last_peered":"2024-04-25T14:09:24.819182+0000","last_clean":"2024-04-25T14:09:24.819182+0000","last_became_active":"2024-04-25T14:09:24.816793+0000","last_became_peered":"2024-04-25T14:09:24.816793+0000","last_unstale":"2024-04-25T14:09:24.819182+0000","last_undegraded":"2024-04-25T14:09:24.819182+0000","last_fullsized":"2024-04-25T14:09:24.819182+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:47.152512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.795817+0000","last_change":"2024-04-25T14:09:24.795817+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.795817+0000","last_undegraded":"2024-04-25T14:09:24.795817+0000","last_fullsized":"2024-04-25T14:09:24.795817+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.811590+0000","last_change":"2024-04-25T14:09:24.808496+0000","last_active":"2024-04-25T14:09:24.811590+0000","last_peered":"2024-04-25T14:09:24.811590+0000","last_clean":"2024-04-25T14:09:24.811590+0000","last_became_active":"2024-04-25T14:09:24.807948+0000","last_became_peered":"2024-04-25T14:09:24.807948+0000","last_unstale":"2024-04-25T14:09:24.811590+0000","last_undegraded":"2024-04-25T14:09:24.811590+0000","last_fullsized":"2024-04-25T14:09:24.811590+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:57.671866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.804276+0000","last_change":"2024-04-25T14:09:24.804276+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.804276+0000","last_undegraded":"2024-04-25T14:09:24.804276+0000","last_fullsized":"2024-04-25T14:09:24.804276+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.815553+0000","last_change":"2024-04-25T14:09:24.810654+0000","last_active":"2024-04-25T14:09:24.815553+0000","last_peered":"2024-04-25T14:09:24.815553+0000","last_clean":"2024-04-25T14:09:24.815553+0000","last_became_active":"2024-04-25T14:09:24.810344+0000","last_became_peered":"2024-04-25T14:09:24.810344+0000","last_unstale":"2024-04-25T14:09:24.815553+0000","last_undegraded":"2024-04-25T14:09:24.815553+0000","last_fullsized":"2024-04-25T14:09:24.815553+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:50.180573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.800603+0000","last_change":"2024-04-25T14:09:24.800603+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.800603+0000","last_undegraded":"2024-04-25T14:09:24.800603+0000","last_fullsized":"2024-04-25T14:09:24.800603+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.823748+0000","last_change":"2024-04-25T14:09:24.817108+0000","last_active":"2024-04-25T14:09:24.823748+0000","last_peered":"2024-04-25T14:09:24.823748+0000","last_clean":"2024-04-25T14:09:24.823748+0000","last_became_active":"2024-04-25T14:09:24.816826+0000","last_became_peered":"2024-04-25T14:09:24.816826+0000","last_unstale":"2024-04-25T14:09:24.823748+0000","last_undegraded":"2024-04-25T14:09:24.823748+0000","last_fullsized":"2024-04-25T14:09:24.823748+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:36.790595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.803123+0000","last_change":"2024-04-25T14:09:24.803123+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.803123+0000","last_undegraded":"2024-04-25T14:09:24.803123+0000","last_fullsized":"2024-04-25T14:09:24.803123+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821222+0000","last_change":"2024-04-25T14:09:24.811340+0000","last_active":"2024-04-25T14:09:24.821222+0000","last_peered":"2024-04-25T14:09:24.821222+0000","last_clean":"2024-04-25T14:09:24.821222+0000","last_became_active":"2024-04-25T14:09:24.809685+0000","last_became_peered":"2024-04-25T14:09:24.809685+0000","last_unstale":"2024-04-25T14:09:24.821222+0000","last_undegraded":"2024-04-25T14:09:24.821222+0000","last_fullsized":"2024-04-25T14:09:24.821222+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:30.379959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.804614+0000","last_change":"2024-04-25T14:09:24.804614+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.804614+0000","last_undegraded":"2024-04-25T14:09:24.804614+0000","last_fullsized":"2024-04-25T14:09:24.804614+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.824699+0000","last_change":"2024-04-25T14:09:24.822761+0000","last_active":"2024-04-25T14:09:24.824699+0000","last_peered":"2024-04-25T14:09:24.824699+0000","last_clean":"2024-04-25T14:09:24.824699+0000","last_became_active":"2024-04-25T14:09:24.822428+0000","last_became_peered":"2024-04-25T14:09:24.822428+0000","last_unstale":"2024-04-25T14:09:24.824699+0000","last_undegraded":"2024-04-25T14:09:24.824699+0000","last_fullsized":"2024-04-25T14:09:24.824699+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:51.227996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.800253+0000","last_change":"2024-04-25T14:09:24.800253+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.800253+0000","last_undegraded":"2024-04-25T14:09:24.800253+0000","last_fullsized":"2024-04-25T14:09:24.800253+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821955+0000","last_change":"2024-04-25T14:09:24.817271+0000","last_active":"2024-04-25T14:09:24.821955+0000","last_peered":"2024-04-25T14:09:24.821955+0000","last_clean":"2024-04-25T14:09:24.821955+0000","last_became_active":"2024-04-25T14:09:24.816723+0000","last_became_peered":"2024-04-25T14:09:24.816723+0000","last_unstale":"2024-04-25T14:09:24.821955+0000","last_undegraded":"2024-04-25T14:09:24.821955+0000","last_fullsized":"2024-04-25T14:09:24.821955+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:29:29.089649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.796305+0000","last_change":"2024-04-25T14:09:24.796305+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.796305+0000","last_undegraded":"2024-04-25T14:09:24.796305+0000","last_fullsized":"2024-04-25T14:09:24.796305+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821149+0000","last_change":"2024-04-25T14:09:24.814155+0000","last_active":"2024-04-25T14:09:24.821149+0000","last_peered":"2024-04-25T14:09:24.821149+0000","last_clean":"2024-04-25T14:09:24.821149+0000","last_became_active":"2024-04-25T14:09:24.812762+0000","last_became_peered":"2024-04-25T14:09:24.812762+0000","last_unstale":"2024-04-25T14:09:24.821149+0000","last_undegraded":"2024-04-25T14:09:24.821149+0000","last_fullsized":"2024-04-25T14:09:24.821149+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:30.379959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.802445+0000","last_change":"2024-04-25T14:09:24.802445+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.802445+0000","last_undegraded":"2024-04-25T14:09:24.802445+0000","last_fullsized":"2024-04-25T14:09:24.802445+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.816847+0000","last_change":"2024-04-25T14:09:24.811205+0000","last_active":"2024-04-25T14:09:24.816847+0000","last_peered":"2024-04-25T14:09:24.816847+0000","last_clean":"2024-04-25T14:09:24.816847+0000","last_became_active":"2024-04-25T14:09:24.810270+0000","last_became_peered":"2024-04-25T14:09:24.810270+0000","last_unstale":"2024-04-25T14:09:24.816847+0000","last_undegraded":"2024-04-25T14:09:24.816847+0000","last_fullsized":"2024-04-25T14:09:24.816847+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:30.864192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.804489+0000","last_change":"2024-04-25T14:09:24.804489+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.804489+0000","last_undegraded":"2024-04-25T14:09:24.804489+0000","last_fullsized":"2024-04-25T14:09:24.804489+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.816656+0000","last_change":"2024-04-25T14:09:24.808594+0000","last_active":"2024-04-25T14:09:24.816656+0000","last_peered":"2024-04-25T14:09:24.816656+0000","last_clean":"2024-04-25T14:09:24.816656+0000","last_became_active":"2024-04-25T14:09:24.807904+0000","last_became_peered":"2024-04-25T14:09:24.807904+0000","last_unstale":"2024-04-25T14:09:24.816656+0000","last_undegraded":"2024-04-25T14:09:24.816656+0000","last_fullsized":"2024-04-25T14:09:24.816656+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:55.195611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.815279+0000","last_change":"2024-04-25T14:09:24.815279+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.815279+0000","last_undegraded":"2024-04-25T14:09:24.815279+0000","last_fullsized":"2024-04-25T14:09:24.815279+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.819227+0000","last_change":"2024-04-25T14:09:24.816525+0000","last_active":"2024-04-25T14:09:24.819227+0000","last_peered":"2024-04-25T14:09:24.819227+0000","last_clean":"2024-04-25T14:09:24.819227+0000","last_became_active":"2024-04-25T14:09:24.815546+0000","last_became_peered":"2024-04-25T14:09:24.815546+0000","last_unstale":"2024-04-25T14:09:24.819227+0000","last_undegraded":"2024-04-25T14:09:24.819227+0000","last_fullsized":"2024-04-25T14:09:24.819227+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:46.765820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801719+0000","last_change":"2024-04-25T14:09:24.801719+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801719+0000","last_undegraded":"2024-04-25T14:09:24.801719+0000","last_fullsized":"2024-04-25T14:09:24.801719+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.817516+0000","last_change":"2024-04-25T14:09:24.813608+0000","last_active":"2024-04-25T14:09:24.817516+0000","last_peered":"2024-04-25T14:09:24.817516+0000","last_clean":"2024-04-25T14:09:24.817516+0000","last_became_active":"2024-04-25T14:09:24.813300+0000","last_became_peered":"2024-04-25T14:09:24.813300+0000","last_unstale":"2024-04-25T14:09:24.817516+0000","last_undegraded":"2024-04-25T14:09:24.817516+0000","last_fullsized":"2024-04-25T14:09:24.817516+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:41.291113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.803987+0000","last_change":"2024-04-25T14:09:24.803987+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.803987+0000","last_undegraded":"2024-04-25T14:09:24.803987+0000","last_fullsized":"2024-04-25T14:09:24.803987+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821085+0000","last_change":"2024-04-25T14:09:24.818883+0000","last_active":"2024-04-25T14:09:24.821085+0000","last_peered":"2024-04-25T14:09:24.821085+0000","last_clean":"2024-04-25T14:09:24.821085+0000","last_became_active":"2024-04-25T14:09:24.817518+0000","last_became_peered":"2024-04-25T14:09:24.817518+0000","last_unstale":"2024-04-25T14:09:24.821085+0000","last_undegraded":"2024-04-25T14:09:24.821085+0000","last_fullsized":"2024-04-25T14:09:24.821085+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:03.242163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801652+0000","last_change":"2024-04-25T14:09:24.801652+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801652+0000","last_undegraded":"2024-04-25T14:09:24.801652+0000","last_fullsized":"2024-04-25T14:09:24.801652+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.816749+0000","last_change":"2024-04-25T14:09:24.814702+0000","last_active":"2024-04-25T14:09:24.816749+0000","last_peered":"2024-04-25T14:09:24.816749+0000","last_clean":"2024-04-25T14:09:24.816749+0000","last_became_active":"2024-04-25T14:09:24.807948+0000","last_became_peered":"2024-04-25T14:09:24.807948+0000","last_unstale":"2024-04-25T14:09:24.816749+0000","last_undegraded":"2024-04-25T14:09:24.816749+0000","last_fullsized":"2024-04-25T14:09:24.816749+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:36.733366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.814203+0000","last_change":"2024-04-25T14:09:24.814203+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.814203+0000","last_undegraded":"2024-04-25T14:09:24.814203+0000","last_fullsized":"2024-04-25T14:09:24.814203+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.819629+0000","last_change":"2024-04-25T14:09:24.817779+0000","last_active":"2024-04-25T14:09:24.819629+0000","last_peered":"2024-04-25T14:09:24.819629+0000","last_clean":"2024-04-25T14:09:24.819629+0000","last_became_active":"2024-04-25T14:09:24.817493+0000","last_became_peered":"2024-04-25T14:09:24.817493+0000","last_unstale":"2024-04-25T14:09:24.819629+0000","last_undegraded":"2024-04-25T14:09:24.819629+0000","last_fullsized":"2024-04-25T14:09:24.819629+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:59.756189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.796715+0000","last_change":"2024-04-25T14:09:24.796715+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.796715+0000","last_undegraded":"2024-04-25T14:09:24.796715+0000","last_fullsized":"2024-04-25T14:09:24.796715+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.820996+0000","last_change":"2024-04-25T14:09:24.818436+0000","last_active":"2024-04-25T14:09:24.820996+0000","last_peered":"2024-04-25T14:09:24.820996+0000","last_clean":"2024-04-25T14:09:24.820996+0000","last_became_active":"2024-04-25T14:09:24.818157+0000","last_became_peered":"2024-04-25T14:09:24.818157+0000","last_unstale":"2024-04-25T14:09:24.820996+0000","last_undegraded":"2024-04-25T14:09:24.820996+0000","last_fullsized":"2024-04-25T14:09:24.820996+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:03.406811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.802382+0000","last_change":"2024-04-25T14:09:24.802382+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.802382+0000","last_undegraded":"2024-04-25T14:09:24.802382+0000","last_fullsized":"2024-04-25T14:09:24.802382+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.824340+0000","last_change":"2024-04-25T14:09:24.820985+0000","last_active":"2024-04-25T14:09:24.824340+0000","last_peered":"2024-04-25T14:09:24.824340+0000","last_clean":"2024-04-25T14:09:24.824340+0000","last_became_active":"2024-04-25T14:09:24.819953+0000","last_became_peered":"2024-04-25T14:09:24.819953+0000","last_unstale":"2024-04-25T14:09:24.824340+0000","last_undegraded":"2024-04-25T14:09:24.824340+0000","last_fullsized":"2024-04-25T14:09:24.824340+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:17.719124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801234+0000","last_change":"2024-04-25T14:09:24.801234+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801234+0000","last_undegraded":"2024-04-25T14:09:24.801234+0000","last_fullsized":"2024-04-25T14:09:24.801234+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.824307+0000","last_change":"2024-04-25T14:09:24.822304+0000","last_active":"2024-04-25T14:09:24.824307+0000","last_peered":"2024-04-25T14:09:24.824307+0000","last_clean":"2024-04-25T14:09:24.824307+0000","last_became_active":"2024-04-25T14:09:24.821946+0000","last_became_peered":"2024-04-25T14:09:24.821946+0000","last_unstale":"2024-04-25T14:09:24.824307+0000","last_undegraded":"2024-04-25T14:09:24.824307+0000","last_fullsized":"2024-04-25T14:09:24.824307+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:28.243010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.800017+0000","last_change":"2024-04-25T14:09:24.800017+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.800017+0000","last_undegraded":"2024-04-25T14:09:24.800017+0000","last_fullsized":"2024-04-25T14:09:24.800017+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.816511+0000","last_change":"2024-04-25T14:09:24.811162+0000","last_active":"2024-04-25T14:09:24.816511+0000","last_peered":"2024-04-25T14:09:24.816511+0000","last_clean":"2024-04-25T14:09:24.816511+0000","last_became_active":"2024-04-25T14:09:24.810194+0000","last_became_peered":"2024-04-25T14:09:24.810194+0000","last_unstale":"2024-04-25T14:09:24.816511+0000","last_undegraded":"2024-04-25T14:09:24.816511+0000","last_fullsized":"2024-04-25T14:09:24.816511+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:44.601365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.795988+0000","last_change":"2024-04-25T14:09:24.795988+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.795988+0000","last_undegraded":"2024-04-25T14:09:24.795988+0000","last_fullsized":"2024-04-25T14:09:24.795988+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.823676+0000","last_change":"2024-04-25T14:09:24.817347+0000","last_active":"2024-04-25T14:09:24.823676+0000","last_peered":"2024-04-25T14:09:24.823676+0000","last_clean":"2024-04-25T14:09:24.823676+0000","last_became_active":"2024-04-25T14:09:24.815968+0000","last_became_peered":"2024-04-25T14:09:24.815968+0000","last_unstale":"2024-04-25T14:09:24.823676+0000","last_undegraded":"2024-04-25T14:09:24.823676+0000","last_fullsized":"2024-04-25T14:09:24.823676+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:18.027462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.795863+0000","last_change":"2024-04-25T14:09:24.795863+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.795863+0000","last_undegraded":"2024-04-25T14:09:24.795863+0000","last_fullsized":"2024-04-25T14:09:24.795863+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.819672+0000","last_change":"2024-04-25T14:09:24.816306+0000","last_active":"2024-04-25T14:09:24.819672+0000","last_peered":"2024-04-25T14:09:24.819672+0000","last_clean":"2024-04-25T14:09:24.819672+0000","last_became_active":"2024-04-25T14:09:24.816004+0000","last_became_peered":"2024-04-25T14:09:24.816004+0000","last_unstale":"2024-04-25T14:09:24.819672+0000","last_undegraded":"2024-04-25T14:09:24.819672+0000","last_fullsized":"2024-04-25T14:09:24.819672+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:40.598001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"89'107","reported_seq":377,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.800089+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:09:24.800089+0000","last_peered":"2024-04-25T14:09:24.800089+0000","last_clean":"2024-04-25T14:09:24.800089+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:09:24.800089+0000","last_undegraded":"2024-04-25T14:09:24.800089+0000","last_fullsized":"2024-04-25T14:09:24.800089+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":366,"num_read_kb":935,"num_write":161,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":366,"num_read_kb":935,"num_write":161,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574916,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32940,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338900,"statfs":{"total":96636764160,"available":96603033600,"internally_reserved":0,"allocated":1798144,"data_stored":1188474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":598106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1216512,"data_stored":604773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1216512,"data_stored":604773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574917,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":598106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574916,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1204224,"data_stored":598106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":598106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574917,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32948,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338892,"statfs":{"total":96636764160,"available":96603025408,"internally_reserved":0,"allocated":1798144,"data_stored":1188474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:09:29.955 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:09:30.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.142+0000 7f4e90800640 1 Processor -- start 2024-04-25T14:09:30.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.142+0000 7f4e90800640 1 -- start start 2024-04-25T14:09:30.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.146+0000 7f4e90800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e880d1df0 0x7f4e880cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:30.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.146+0000 7f4e90800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4e880cdcc0 0x7f4e880ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:30.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.146+0000 7f4e90800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e8815e5c0 con 0x7f4e88070330 2024-04-25T14:09:30.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.146+0000 7f4e90800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4e8806fd40 con 0x7f4e880cdcc0 2024-04-25T14:09:30.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.146+0000 7f4e90800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4e8815e8a0 con 0x7f4e880d1df0 2024-04-25T14:09:30.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.146+0000 7f4e8e800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4e880cdcc0 0x7f4e880ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:30.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.146+0000 7f4e8e800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4e880cdcc0 0x7f4e880ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58064/0 (socket says 172.21.15.86:58064) 2024-04-25T14:09:30.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.146+0000 7f4e8de00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f4e88070330 0x7f4e880d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42866/0 (socket says 172.21.15.86:42866) 2024-04-25T14:09:30.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.146+0000 7f4e8e800640 1 -- 172.21.15.86:0/1127765834 learned_addr learned my addr 172.21.15.86:0/1127765834 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:30.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.146+0000 7f4e8d400640 1 --2- 172.21.15.86:0/1127765834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e880d1df0 0x7f4e880cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:30.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.146+0000 7f4e8e800640 1 -- 172.21.15.86:0/1127765834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e880d1df0 msgr2=0x7f4e880cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:30.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.146+0000 7f4e8e800640 1 --2- 172.21.15.86:0/1127765834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e880d1df0 0x7f4e880cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:30.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.146+0000 7f4e8e800640 1 -- 172.21.15.86:0/1127765834 >> v1:172.21.15.86:6789/0 conn(0x7f4e88070330 legacy=0x7f4e880d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:30.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.146+0000 7f4e8e800640 1 -- 172.21.15.86:0/1127765834 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8815eb80 con 0x7f4e880cdcc0 2024-04-25T14:09:30.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.146+0000 7f4e8ca00640 1 -- 172.21.15.86:0/1127765834 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3698855076 0 0) 0x7f4e8815e5c0 con 0x7f4e88070330 2024-04-25T14:09:30.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.146+0000 7f4e8d400640 1 --2- 172.21.15.86:0/1127765834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e880d1df0 0x7f4e880cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:30.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.150+0000 7f4e8e800640 1 --2- 172.21.15.86:0/1127765834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e880cdcc0 0x7f4e880ce0d0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f4e84009e50 tx=0x7f4e8401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f893e6f78a0d562 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:30.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.150+0000 7f4e8ca00640 1 -- 172.21.15.86:0/1127765834 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e840a5810 con 0x7f4e880cdcc0 2024-04-25T14:09:30.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.150+0000 7f4e8ca00640 1 -- 172.21.15.86:0/1127765834 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e840a59b0 con 0x7f4e880cdcc0 2024-04-25T14:09:30.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.150+0000 7f4e8ca00640 1 -- 172.21.15.86:0/1127765834 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e840a5d80 con 0x7f4e880cdcc0 2024-04-25T14:09:30.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.150+0000 7f4e90800640 1 -- 172.21.15.86:0/1127765834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e880cdcc0 msgr2=0x7f4e880ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:30.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.150+0000 7f4e90800640 1 --2- 172.21.15.86:0/1127765834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e880cdcc0 0x7f4e880ce0d0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f4e84009e50 tx=0x7f4e8401c970 comp rx=0 tx=0).stop 2024-04-25T14:09:30.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.150+0000 7f4e90800640 1 -- 172.21.15.86:0/1127765834 shutdown_connections 2024-04-25T14:09:30.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.150+0000 7f4e90800640 1 --2- 172.21.15.86:0/1127765834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e880cdcc0 0x7f4e880ce0d0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:30.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.150+0000 7f4e90800640 1 --2- 172.21.15.86:0/1127765834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e880d1df0 0x7f4e880cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:30.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.150+0000 7f4e90800640 1 -- 172.21.15.86:0/1127765834 >> 172.21.15.86:0/1127765834 conn(0x7f4e880b4650 msgr2=0x7f4e880b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:30.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.150+0000 7f4e90800640 1 -- 172.21.15.86:0/1127765834 shutdown_connections 2024-04-25T14:09:30.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.150+0000 7f4e90800640 1 -- 172.21.15.86:0/1127765834 wait complete. 2024-04-25T14:09:30.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.150+0000 7f4e90800640 1 Processor -- start 2024-04-25T14:09:30.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.150+0000 7f4e90800640 1 -- start start 2024-04-25T14:09:30.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.154+0000 7f4e90800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e88070330 0x7f4e880d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:30.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.154+0000 7f4e90800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e880cdcc0 0x7f4e8815ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:30.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.154+0000 7f4e90800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e880c9d40 0x7f4e880ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:30.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.154+0000 7f4e90800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8815f4b0 con 0x7f4e880cdcc0 2024-04-25T14:09:30.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.154+0000 7f4e90800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8815f330 con 0x7f4e88070330 2024-04-25T14:09:30.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.154+0000 7f4e90800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4e88070740 con 0x7f4e880c9d40 2024-04-25T14:09:30.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.154+0000 7f4e8d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e880cdcc0 0x7f4e8815ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:30.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.154+0000 7f4e8d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e880cdcc0 0x7f4e8815ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58066/0 (socket says 172.21.15.86:58066) 2024-04-25T14:09:30.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.154+0000 7f4e8d400640 1 -- 172.21.15.86:0/3922354038 learned_addr learned my addr 172.21.15.86:0/3922354038 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:30.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.154+0000 7f4e8e800640 1 --2- 172.21.15.86:0/3922354038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e880c9d40 0x7f4e880ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:30.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.154+0000 7f4e8de00640 1 --2- 172.21.15.86:0/3922354038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e88070330 0x7f4e880d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:30.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.154+0000 7f4e8d400640 1 -- 172.21.15.86:0/3922354038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e880c9d40 msgr2=0x7f4e880ca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:30.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.154+0000 7f4e8d400640 1 --2- 172.21.15.86:0/3922354038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e880c9d40 0x7f4e880ca150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:30.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.154+0000 7f4e8d400640 1 -- 172.21.15.86:0/3922354038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e88070330 msgr2=0x7f4e880d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:30.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.154+0000 7f4e8d400640 1 --2- 172.21.15.86:0/3922354038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e88070330 0x7f4e880d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:30.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.154+0000 7f4e8d400640 1 -- 172.21.15.86:0/3922354038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8815b380 con 0x7f4e880cdcc0 2024-04-25T14:09:30.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.154+0000 7f4e8e800640 1 --2- 172.21.15.86:0/3922354038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e880c9d40 0x7f4e880ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:30.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.154+0000 7f4e8d400640 1 --2- 172.21.15.86:0/3922354038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e880cdcc0 0x7f4e8815ae40 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800d800 tx=0x7f4e7800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:30.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.154+0000 7f4e82000640 1 -- 172.21.15.86:0/3922354038 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78018020 con 0x7f4e880cdcc0 2024-04-25T14:09:30.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.154+0000 7f4e82000640 1 -- 172.21.15.86:0/3922354038 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e780096c0 con 0x7f4e880cdcc0 2024-04-25T14:09:30.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.154+0000 7f4e90800640 1 -- 172.21.15.86:0/3922354038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e880ca690 con 0x7f4e880cdcc0 2024-04-25T14:09:30.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.154+0000 7f4e82000640 1 -- 172.21.15.86:0/3922354038 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78009ab0 con 0x7f4e880cdcc0 2024-04-25T14:09:30.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.154+0000 7f4e90800640 1 -- 172.21.15.86:0/3922354038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e8815c440 con 0x7f4e880cdcc0 2024-04-25T14:09:30.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.154+0000 7f4e90800640 1 -- 172.21.15.86:0/3922354038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e8815c1b0 con 0x7f4e880cdcc0 2024-04-25T14:09:30.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.158+0000 7f4e82000640 1 -- 172.21.15.86:0/3922354038 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7f4e7801b020 con 0x7f4e880cdcc0 2024-04-25T14:09:30.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.158+0000 7f4e82000640 1 --2- 172.21.15.86:0/3922354038 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f4e6c06f520 0x7f4e6c071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:30.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.158+0000 7f4e82000640 1 -- 172.21.15.86:0/3922354038 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7f4e7808fb50 con 0x7f4e880cdcc0 2024-04-25T14:09:30.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.158+0000 7f4e8de00640 1 --2- 172.21.15.86:0/3922354038 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f4e6c06f520 0x7f4e6c071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:30.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.158+0000 7f4e8de00640 1 --2- 172.21.15.86:0/3922354038 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f4e6c06f520 0x7f4e6c071a30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c000f70 tx=0x7f4e7c007d10 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:30.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.162+0000 7f4e82000640 1 -- 172.21.15.86:0/3922354038 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4e78022020 con 0x7f4e880cdcc0 2024-04-25T14:09:30.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.314+0000 7f4e90800640 1 -- 172.21.15.86:0/3922354038 --> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4e88069d00 con 0x7f4e6c06f520 2024-04-25T14:09:30.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.322+0000 7f4e82000640 1 -- 172.21.15.86:0/3922354038 <== mgr.7650 v2:172.21.15.86:6832/4236874185 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246766 (secure 0 0 0) 0x7f4e88069d00 con 0x7f4e6c06f520 2024-04-25T14:09:30.321 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:30.323 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:09:30.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.326+0000 7f4e90800640 1 -- 172.21.15.86:0/3922354038 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f4e6c06f520 msgr2=0x7f4e6c071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:30.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.326+0000 7f4e90800640 1 --2- 172.21.15.86:0/3922354038 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f4e6c06f520 0x7f4e6c071a30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c000f70 tx=0x7f4e7c007d10 comp rx=0 tx=0).stop 2024-04-25T14:09:30.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.326+0000 7f4e90800640 1 -- 172.21.15.86:0/3922354038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e880cdcc0 msgr2=0x7f4e8815ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:30.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.326+0000 7f4e90800640 1 --2- 172.21.15.86:0/3922354038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e880cdcc0 0x7f4e8815ae40 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800d800 tx=0x7f4e7800dcc0 comp rx=0 tx=0).stop 2024-04-25T14:09:30.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.326+0000 7f4e90800640 1 -- 172.21.15.86:0/3922354038 shutdown_connections 2024-04-25T14:09:30.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.326+0000 7f4e90800640 1 --2- 172.21.15.86:0/3922354038 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f4e6c06f520 0x7f4e6c071a30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:30.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.326+0000 7f4e90800640 1 --2- 172.21.15.86:0/3922354038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e880c9d40 0x7f4e880ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:30.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.326+0000 7f4e90800640 1 --2- 172.21.15.86:0/3922354038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e880cdcc0 0x7f4e8815ae40 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:30.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.326+0000 7f4e90800640 1 --2- 172.21.15.86:0/3922354038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e88070330 0x7f4e880d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:30.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.326+0000 7f4e90800640 1 -- 172.21.15.86:0/3922354038 >> 172.21.15.86:0/3922354038 conn(0x7f4e880b4650 msgr2=0x7f4e880cb110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:30.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.326+0000 7f4e90800640 1 -- 172.21.15.86:0/3922354038 shutdown_connections 2024-04-25T14:09:30.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.326+0000 7f4e90800640 1 -- 172.21.15.86:0/3922354038 wait complete. 2024-04-25T14:09:30.343 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T14:09:29.506130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":366,"num_read_kb":935,"num_write":161,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260076,"kb_used_data":10592,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714644,"statfs":{"total":773094113280,"available":772827795456,"internally_reserved":0,"allocated":10846208,"data_stored":5978918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.716486"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.811745+0000","last_change":"2024-04-25T14:09:24.811745+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.811745+0000","last_undegraded":"2024-04-25T14:09:24.811745+0000","last_fullsized":"2024-04-25T14:09:24.811745+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.805434+0000","last_change":"2024-04-25T14:09:24.805434+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.805434+0000","last_undegraded":"2024-04-25T14:09:24.805434+0000","last_fullsized":"2024-04-25T14:09:24.805434+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.806609+0000","last_change":"2024-04-25T14:09:24.806609+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.806609+0000","last_undegraded":"2024-04-25T14:09:24.806609+0000","last_fullsized":"2024-04-25T14:09:24.806609+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.814243+0000","last_change":"2024-04-25T14:09:24.814243+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.814243+0000","last_undegraded":"2024-04-25T14:09:24.814243+0000","last_fullsized":"2024-04-25T14:09:24.814243+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.796780+0000","last_change":"2024-04-25T14:09:24.796780+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.796780+0000","last_undegraded":"2024-04-25T14:09:24.796780+0000","last_fullsized":"2024-04-25T14:09:24.796780+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801913+0000","last_change":"2024-04-25T14:09:24.801913+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801913+0000","last_undegraded":"2024-04-25T14:09:24.801913+0000","last_fullsized":"2024-04-25T14:09:24.801913+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.805381+0000","last_change":"2024-04-25T14:09:24.805381+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.805381+0000","last_undegraded":"2024-04-25T14:09:24.805381+0000","last_fullsized":"2024-04-25T14:09:24.805381+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.805573+0000","last_change":"2024-04-25T14:09:24.805573+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.805573+0000","last_undegraded":"2024-04-25T14:09:24.805573+0000","last_fullsized":"2024-04-25T14:09:24.805573+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.802656+0000","last_change":"2024-04-25T14:09:24.802656+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.802656+0000","last_undegraded":"2024-04-25T14:09:24.802656+0000","last_fullsized":"2024-04-25T14:09:24.802656+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.796198+0000","last_change":"2024-04-25T14:09:24.796198+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.796198+0000","last_undegraded":"2024-04-25T14:09:24.796198+0000","last_fullsized":"2024-04-25T14:09:24.796198+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.797705+0000","last_change":"2024-04-25T14:09:24.797705+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.797705+0000","last_undegraded":"2024-04-25T14:09:24.797705+0000","last_fullsized":"2024-04-25T14:09:24.797705+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.803323+0000","last_change":"2024-04-25T14:09:24.803323+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.803323+0000","last_undegraded":"2024-04-25T14:09:24.803323+0000","last_fullsized":"2024-04-25T14:09:24.803323+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.814711+0000","last_change":"2024-04-25T14:09:24.814711+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.814711+0000","last_undegraded":"2024-04-25T14:09:24.814711+0000","last_fullsized":"2024-04-25T14:09:24.814711+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801188+0000","last_change":"2024-04-25T14:09:24.801188+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801188+0000","last_undegraded":"2024-04-25T14:09:24.801188+0000","last_fullsized":"2024-04-25T14:09:24.801188+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.800597+0000","last_change":"2024-04-25T14:09:24.800597+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.800597+0000","last_undegraded":"2024-04-25T14:09:24.800597+0000","last_fullsized":"2024-04-25T14:09:24.800597+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.799763+0000","last_change":"2024-04-25T14:09:24.799763+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.799763+0000","last_undegraded":"2024-04-25T14:09:24.799763+0000","last_fullsized":"2024-04-25T14:09:24.799763+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.806722+0000","last_change":"2024-04-25T14:09:24.806722+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.806722+0000","last_undegraded":"2024-04-25T14:09:24.806722+0000","last_fullsized":"2024-04-25T14:09:24.806722+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801800+0000","last_change":"2024-04-25T14:09:24.801800+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801800+0000","last_undegraded":"2024-04-25T14:09:24.801800+0000","last_fullsized":"2024-04-25T14:09:24.801800+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.805192+0000","last_change":"2024-04-25T14:09:24.805192+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.805192+0000","last_undegraded":"2024-04-25T14:09:24.805192+0000","last_fullsized":"2024-04-25T14:09:24.805192+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.802814+0000","last_change":"2024-04-25T14:09:24.802814+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.802814+0000","last_undegraded":"2024-04-25T14:09:24.802814+0000","last_fullsized":"2024-04-25T14:09:24.802814+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.800665+0000","last_change":"2024-04-25T14:09:24.800665+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.800665+0000","last_undegraded":"2024-04-25T14:09:24.800665+0000","last_fullsized":"2024-04-25T14:09:24.800665+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.797246+0000","last_change":"2024-04-25T14:09:24.797246+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.797246+0000","last_undegraded":"2024-04-25T14:09:24.797246+0000","last_fullsized":"2024-04-25T14:09:24.797246+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.804279+0000","last_change":"2024-04-25T14:09:24.804279+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.804279+0000","last_undegraded":"2024-04-25T14:09:24.804279+0000","last_fullsized":"2024-04-25T14:09:24.804279+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.795033+0000","last_change":"2024-04-25T14:09:24.795033+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.795033+0000","last_undegraded":"2024-04-25T14:09:24.795033+0000","last_fullsized":"2024-04-25T14:09:24.795033+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.803941+0000","last_change":"2024-04-25T14:09:24.803941+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.803941+0000","last_undegraded":"2024-04-25T14:09:24.803941+0000","last_fullsized":"2024-04-25T14:09:24.803941+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.803227+0000","last_change":"2024-04-25T14:09:24.803227+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.803227+0000","last_undegraded":"2024-04-25T14:09:24.803227+0000","last_fullsized":"2024-04-25T14:09:24.803227+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.797093+0000","last_change":"2024-04-25T14:09:24.797093+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.797093+0000","last_undegraded":"2024-04-25T14:09:24.797093+0000","last_fullsized":"2024-04-25T14:09:24.797093+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.808887+0000","last_change":"2024-04-25T14:09:24.808887+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.808887+0000","last_undegraded":"2024-04-25T14:09:24.808887+0000","last_fullsized":"2024-04-25T14:09:24.808887+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.800116+0000","last_change":"2024-04-25T14:09:24.800116+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.800116+0000","last_undegraded":"2024-04-25T14:09:24.800116+0000","last_fullsized":"2024-04-25T14:09:24.800116+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.805810+0000","last_change":"2024-04-25T14:09:24.805810+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.805810+0000","last_undegraded":"2024-04-25T14:09:24.805810+0000","last_fullsized":"2024-04-25T14:09:24.805810+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.804724+0000","last_change":"2024-04-25T14:09:24.804724+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.804724+0000","last_undegraded":"2024-04-25T14:09:24.804724+0000","last_fullsized":"2024-04-25T14:09:24.804724+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.805096+0000","last_change":"2024-04-25T14:09:24.805096+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.805096+0000","last_undegraded":"2024-04-25T14:09:24.805096+0000","last_fullsized":"2024-04-25T14:09:24.805096+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801802+0000","last_change":"2024-04-25T14:09:24.801802+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801802+0000","last_undegraded":"2024-04-25T14:09:24.801802+0000","last_fullsized":"2024-04-25T14:09:24.801802+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.823829+0000","last_change":"2024-04-25T14:09:24.817466+0000","last_active":"2024-04-25T14:09:24.823829+0000","last_peered":"2024-04-25T14:09:24.823829+0000","last_clean":"2024-04-25T14:09:24.823829+0000","last_became_active":"2024-04-25T14:09:24.817167+0000","last_became_peered":"2024-04-25T14:09:24.817167+0000","last_unstale":"2024-04-25T14:09:24.823829+0000","last_undegraded":"2024-04-25T14:09:24.823829+0000","last_fullsized":"2024-04-25T14:09:24.823829+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:02.374557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.803577+0000","last_change":"2024-04-25T14:09:24.803577+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.803577+0000","last_undegraded":"2024-04-25T14:09:24.803577+0000","last_fullsized":"2024-04-25T14:09:24.803577+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.824309+0000","last_change":"2024-04-25T14:09:24.819324+0000","last_active":"2024-04-25T14:09:24.824309+0000","last_peered":"2024-04-25T14:09:24.824309+0000","last_clean":"2024-04-25T14:09:24.824309+0000","last_became_active":"2024-04-25T14:09:24.818377+0000","last_became_peered":"2024-04-25T14:09:24.818377+0000","last_unstale":"2024-04-25T14:09:24.824309+0000","last_undegraded":"2024-04-25T14:09:24.824309+0000","last_fullsized":"2024-04-25T14:09:24.824309+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:38.935678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.802115+0000","last_change":"2024-04-25T14:09:24.802115+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.802115+0000","last_undegraded":"2024-04-25T14:09:24.802115+0000","last_fullsized":"2024-04-25T14:09:24.802115+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821745+0000","last_change":"2024-04-25T14:09:24.813915+0000","last_active":"2024-04-25T14:09:24.821745+0000","last_peered":"2024-04-25T14:09:24.821745+0000","last_clean":"2024-04-25T14:09:24.821745+0000","last_became_active":"2024-04-25T14:09:24.813610+0000","last_became_peered":"2024-04-25T14:09:24.813610+0000","last_unstale":"2024-04-25T14:09:24.821745+0000","last_undegraded":"2024-04-25T14:09:24.821745+0000","last_fullsized":"2024-04-25T14:09:24.821745+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:03.242163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.796164+0000","last_change":"2024-04-25T14:09:24.796164+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.796164+0000","last_undegraded":"2024-04-25T14:09:24.796164+0000","last_fullsized":"2024-04-25T14:09:24.796164+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.813538+0000","last_change":"2024-04-25T14:09:24.810780+0000","last_active":"2024-04-25T14:09:24.813538+0000","last_peered":"2024-04-25T14:09:24.813538+0000","last_clean":"2024-04-25T14:09:24.813538+0000","last_became_active":"2024-04-25T14:09:24.810462+0000","last_became_peered":"2024-04-25T14:09:24.810462+0000","last_unstale":"2024-04-25T14:09:24.813538+0000","last_undegraded":"2024-04-25T14:09:24.813538+0000","last_fullsized":"2024-04-25T14:09:24.813538+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:30.379959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.809107+0000","last_change":"2024-04-25T14:09:24.809107+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.809107+0000","last_undegraded":"2024-04-25T14:09:24.809107+0000","last_fullsized":"2024-04-25T14:09:24.809107+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.820528+0000","last_change":"2024-04-25T14:09:24.813955+0000","last_active":"2024-04-25T14:09:24.820528+0000","last_peered":"2024-04-25T14:09:24.820528+0000","last_clean":"2024-04-25T14:09:24.820528+0000","last_became_active":"2024-04-25T14:09:24.813645+0000","last_became_peered":"2024-04-25T14:09:24.813645+0000","last_unstale":"2024-04-25T14:09:24.820528+0000","last_undegraded":"2024-04-25T14:09:24.820528+0000","last_fullsized":"2024-04-25T14:09:24.820528+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:02.374557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801810+0000","last_change":"2024-04-25T14:09:24.801810+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801810+0000","last_undegraded":"2024-04-25T14:09:24.801810+0000","last_fullsized":"2024-04-25T14:09:24.801810+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821172+0000","last_change":"2024-04-25T14:09:24.814236+0000","last_active":"2024-04-25T14:09:24.821172+0000","last_peered":"2024-04-25T14:09:24.821172+0000","last_clean":"2024-04-25T14:09:24.821172+0000","last_became_active":"2024-04-25T14:09:24.813909+0000","last_became_peered":"2024-04-25T14:09:24.813909+0000","last_unstale":"2024-04-25T14:09:24.821172+0000","last_undegraded":"2024-04-25T14:09:24.821172+0000","last_fullsized":"2024-04-25T14:09:24.821172+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:57.671866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801284+0000","last_change":"2024-04-25T14:09:24.801284+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801284+0000","last_undegraded":"2024-04-25T14:09:24.801284+0000","last_fullsized":"2024-04-25T14:09:24.801284+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821282+0000","last_change":"2024-04-25T14:09:24.811288+0000","last_active":"2024-04-25T14:09:24.821282+0000","last_peered":"2024-04-25T14:09:24.821282+0000","last_clean":"2024-04-25T14:09:24.821282+0000","last_became_active":"2024-04-25T14:09:24.809633+0000","last_became_peered":"2024-04-25T14:09:24.809633+0000","last_unstale":"2024-04-25T14:09:24.821282+0000","last_undegraded":"2024-04-25T14:09:24.821282+0000","last_fullsized":"2024-04-25T14:09:24.821282+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:57.671866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.809114+0000","last_change":"2024-04-25T14:09:24.809114+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.809114+0000","last_undegraded":"2024-04-25T14:09:24.809114+0000","last_fullsized":"2024-04-25T14:09:24.809114+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.806864+0000","last_change":"2024-04-25T14:09:24.799767+0000","last_active":"2024-04-25T14:09:24.806864+0000","last_peered":"2024-04-25T14:09:24.806864+0000","last_clean":"2024-04-25T14:09:24.806864+0000","last_became_active":"2024-04-25T14:09:24.799440+0000","last_became_peered":"2024-04-25T14:09:24.799440+0000","last_unstale":"2024-04-25T14:09:24.806864+0000","last_undegraded":"2024-04-25T14:09:24.806864+0000","last_fullsized":"2024-04-25T14:09:24.806864+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:44.194963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.802420+0000","last_change":"2024-04-25T14:09:24.802420+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.802420+0000","last_undegraded":"2024-04-25T14:09:24.802420+0000","last_fullsized":"2024-04-25T14:09:24.802420+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821771+0000","last_change":"2024-04-25T14:09:24.813861+0000","last_active":"2024-04-25T14:09:24.821771+0000","last_peered":"2024-04-25T14:09:24.821771+0000","last_clean":"2024-04-25T14:09:24.821771+0000","last_became_active":"2024-04-25T14:09:24.813523+0000","last_became_peered":"2024-04-25T14:09:24.813523+0000","last_unstale":"2024-04-25T14:09:24.821771+0000","last_undegraded":"2024-04-25T14:09:24.821771+0000","last_fullsized":"2024-04-25T14:09:24.821771+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:03.406811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.795235+0000","last_change":"2024-04-25T14:09:24.795235+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.795235+0000","last_undegraded":"2024-04-25T14:09:24.795235+0000","last_fullsized":"2024-04-25T14:09:24.795235+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.823970+0000","last_change":"2024-04-25T14:09:24.819250+0000","last_active":"2024-04-25T14:09:24.823970+0000","last_peered":"2024-04-25T14:09:24.823970+0000","last_clean":"2024-04-25T14:09:24.823970+0000","last_became_active":"2024-04-25T14:09:24.818892+0000","last_became_peered":"2024-04-25T14:09:24.818892+0000","last_unstale":"2024-04-25T14:09:24.823970+0000","last_undegraded":"2024-04-25T14:09:24.823970+0000","last_fullsized":"2024-04-25T14:09:24.823970+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:12.433048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.799314+0000","last_change":"2024-04-25T14:09:24.799314+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.799314+0000","last_undegraded":"2024-04-25T14:09:24.799314+0000","last_fullsized":"2024-04-25T14:09:24.799314+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.812722+0000","last_change":"2024-04-25T14:09:24.808328+0000","last_active":"2024-04-25T14:09:24.812722+0000","last_peered":"2024-04-25T14:09:24.812722+0000","last_clean":"2024-04-25T14:09:24.812722+0000","last_became_active":"2024-04-25T14:09:24.807984+0000","last_became_peered":"2024-04-25T14:09:24.807984+0000","last_unstale":"2024-04-25T14:09:24.812722+0000","last_undegraded":"2024-04-25T14:09:24.812722+0000","last_fullsized":"2024-04-25T14:09:24.812722+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:50.180573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.795828+0000","last_change":"2024-04-25T14:09:24.795828+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.795828+0000","last_undegraded":"2024-04-25T14:09:24.795828+0000","last_fullsized":"2024-04-25T14:09:24.795828+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.819182+0000","last_change":"2024-04-25T14:09:24.817277+0000","last_active":"2024-04-25T14:09:24.819182+0000","last_peered":"2024-04-25T14:09:24.819182+0000","last_clean":"2024-04-25T14:09:24.819182+0000","last_became_active":"2024-04-25T14:09:24.816793+0000","last_became_peered":"2024-04-25T14:09:24.816793+0000","last_unstale":"2024-04-25T14:09:24.819182+0000","last_undegraded":"2024-04-25T14:09:24.819182+0000","last_fullsized":"2024-04-25T14:09:24.819182+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:47.152512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.795817+0000","last_change":"2024-04-25T14:09:24.795817+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.795817+0000","last_undegraded":"2024-04-25T14:09:24.795817+0000","last_fullsized":"2024-04-25T14:09:24.795817+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.811590+0000","last_change":"2024-04-25T14:09:24.808496+0000","last_active":"2024-04-25T14:09:24.811590+0000","last_peered":"2024-04-25T14:09:24.811590+0000","last_clean":"2024-04-25T14:09:24.811590+0000","last_became_active":"2024-04-25T14:09:24.807948+0000","last_became_peered":"2024-04-25T14:09:24.807948+0000","last_unstale":"2024-04-25T14:09:24.811590+0000","last_undegraded":"2024-04-25T14:09:24.811590+0000","last_fullsized":"2024-04-25T14:09:24.811590+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:57.671866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.804276+0000","last_change":"2024-04-25T14:09:24.804276+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.804276+0000","last_undegraded":"2024-04-25T14:09:24.804276+0000","last_fullsized":"2024-04-25T14:09:24.804276+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.815553+0000","last_change":"2024-04-25T14:09:24.810654+0000","last_active":"2024-04-25T14:09:24.815553+0000","last_peered":"2024-04-25T14:09:24.815553+0000","last_clean":"2024-04-25T14:09:24.815553+0000","last_became_active":"2024-04-25T14:09:24.810344+0000","last_became_peered":"2024-04-25T14:09:24.810344+0000","last_unstale":"2024-04-25T14:09:24.815553+0000","last_undegraded":"2024-04-25T14:09:24.815553+0000","last_fullsized":"2024-04-25T14:09:24.815553+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:50.180573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.800603+0000","last_change":"2024-04-25T14:09:24.800603+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.800603+0000","last_undegraded":"2024-04-25T14:09:24.800603+0000","last_fullsized":"2024-04-25T14:09:24.800603+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.823748+0000","last_change":"2024-04-25T14:09:24.817108+0000","last_active":"2024-04-25T14:09:24.823748+0000","last_peered":"2024-04-25T14:09:24.823748+0000","last_clean":"2024-04-25T14:09:24.823748+0000","last_became_active":"2024-04-25T14:09:24.816826+0000","last_became_peered":"2024-04-25T14:09:24.816826+0000","last_unstale":"2024-04-25T14:09:24.823748+0000","last_undegraded":"2024-04-25T14:09:24.823748+0000","last_fullsized":"2024-04-25T14:09:24.823748+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:36.790595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.803123+0000","last_change":"2024-04-25T14:09:24.803123+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.803123+0000","last_undegraded":"2024-04-25T14:09:24.803123+0000","last_fullsized":"2024-04-25T14:09:24.803123+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821222+0000","last_change":"2024-04-25T14:09:24.811340+0000","last_active":"2024-04-25T14:09:24.821222+0000","last_peered":"2024-04-25T14:09:24.821222+0000","last_clean":"2024-04-25T14:09:24.821222+0000","last_became_active":"2024-04-25T14:09:24.809685+0000","last_became_peered":"2024-04-25T14:09:24.809685+0000","last_unstale":"2024-04-25T14:09:24.821222+0000","last_undegraded":"2024-04-25T14:09:24.821222+0000","last_fullsized":"2024-04-25T14:09:24.821222+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:30.379959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.804614+0000","last_change":"2024-04-25T14:09:24.804614+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.804614+0000","last_undegraded":"2024-04-25T14:09:24.804614+0000","last_fullsized":"2024-04-25T14:09:24.804614+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.824699+0000","last_change":"2024-04-25T14:09:24.822761+0000","last_active":"2024-04-25T14:09:24.824699+0000","last_peered":"2024-04-25T14:09:24.824699+0000","last_clean":"2024-04-25T14:09:24.824699+0000","last_became_active":"2024-04-25T14:09:24.822428+0000","last_became_peered":"2024-04-25T14:09:24.822428+0000","last_unstale":"2024-04-25T14:09:24.824699+0000","last_undegraded":"2024-04-25T14:09:24.824699+0000","last_fullsized":"2024-04-25T14:09:24.824699+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:51.227996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.800253+0000","last_change":"2024-04-25T14:09:24.800253+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.800253+0000","last_undegraded":"2024-04-25T14:09:24.800253+0000","last_fullsized":"2024-04-25T14:09:24.800253+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821955+0000","last_change":"2024-04-25T14:09:24.817271+0000","last_active":"2024-04-25T14:09:24.821955+0000","last_peered":"2024-04-25T14:09:24.821955+0000","last_clean":"2024-04-25T14:09:24.821955+0000","last_became_active":"2024-04-25T14:09:24.816723+0000","last_became_peered":"2024-04-25T14:09:24.816723+0000","last_unstale":"2024-04-25T14:09:24.821955+0000","last_undegraded":"2024-04-25T14:09:24.821955+0000","last_fullsized":"2024-04-25T14:09:24.821955+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:29:29.089649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.796305+0000","last_change":"2024-04-25T14:09:24.796305+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.796305+0000","last_undegraded":"2024-04-25T14:09:24.796305+0000","last_fullsized":"2024-04-25T14:09:24.796305+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821149+0000","last_change":"2024-04-25T14:09:24.814155+0000","last_active":"2024-04-25T14:09:24.821149+0000","last_peered":"2024-04-25T14:09:24.821149+0000","last_clean":"2024-04-25T14:09:24.821149+0000","last_became_active":"2024-04-25T14:09:24.812762+0000","last_became_peered":"2024-04-25T14:09:24.812762+0000","last_unstale":"2024-04-25T14:09:24.821149+0000","last_undegraded":"2024-04-25T14:09:24.821149+0000","last_fullsized":"2024-04-25T14:09:24.821149+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:30.379959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.802445+0000","last_change":"2024-04-25T14:09:24.802445+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.802445+0000","last_undegraded":"2024-04-25T14:09:24.802445+0000","last_fullsized":"2024-04-25T14:09:24.802445+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.816847+0000","last_change":"2024-04-25T14:09:24.811205+0000","last_active":"2024-04-25T14:09:24.816847+0000","last_peered":"2024-04-25T14:09:24.816847+0000","last_clean":"2024-04-25T14:09:24.816847+0000","last_became_active":"2024-04-25T14:09:24.810270+0000","last_became_peered":"2024-04-25T14:09:24.810270+0000","last_unstale":"2024-04-25T14:09:24.816847+0000","last_undegraded":"2024-04-25T14:09:24.816847+0000","last_fullsized":"2024-04-25T14:09:24.816847+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:30.864192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.804489+0000","last_change":"2024-04-25T14:09:24.804489+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.804489+0000","last_undegraded":"2024-04-25T14:09:24.804489+0000","last_fullsized":"2024-04-25T14:09:24.804489+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.816656+0000","last_change":"2024-04-25T14:09:24.808594+0000","last_active":"2024-04-25T14:09:24.816656+0000","last_peered":"2024-04-25T14:09:24.816656+0000","last_clean":"2024-04-25T14:09:24.816656+0000","last_became_active":"2024-04-25T14:09:24.807904+0000","last_became_peered":"2024-04-25T14:09:24.807904+0000","last_unstale":"2024-04-25T14:09:24.816656+0000","last_undegraded":"2024-04-25T14:09:24.816656+0000","last_fullsized":"2024-04-25T14:09:24.816656+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:55.195611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.815279+0000","last_change":"2024-04-25T14:09:24.815279+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.815279+0000","last_undegraded":"2024-04-25T14:09:24.815279+0000","last_fullsized":"2024-04-25T14:09:24.815279+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.819227+0000","last_change":"2024-04-25T14:09:24.816525+0000","last_active":"2024-04-25T14:09:24.819227+0000","last_peered":"2024-04-25T14:09:24.819227+0000","last_clean":"2024-04-25T14:09:24.819227+0000","last_became_active":"2024-04-25T14:09:24.815546+0000","last_became_peered":"2024-04-25T14:09:24.815546+0000","last_unstale":"2024-04-25T14:09:24.819227+0000","last_undegraded":"2024-04-25T14:09:24.819227+0000","last_fullsized":"2024-04-25T14:09:24.819227+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:46.765820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801719+0000","last_change":"2024-04-25T14:09:24.801719+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801719+0000","last_undegraded":"2024-04-25T14:09:24.801719+0000","last_fullsized":"2024-04-25T14:09:24.801719+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.817516+0000","last_change":"2024-04-25T14:09:24.813608+0000","last_active":"2024-04-25T14:09:24.817516+0000","last_peered":"2024-04-25T14:09:24.817516+0000","last_clean":"2024-04-25T14:09:24.817516+0000","last_became_active":"2024-04-25T14:09:24.813300+0000","last_became_peered":"2024-04-25T14:09:24.813300+0000","last_unstale":"2024-04-25T14:09:24.817516+0000","last_undegraded":"2024-04-25T14:09:24.817516+0000","last_fullsized":"2024-04-25T14:09:24.817516+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:41.291113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.803987+0000","last_change":"2024-04-25T14:09:24.803987+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.803987+0000","last_undegraded":"2024-04-25T14:09:24.803987+0000","last_fullsized":"2024-04-25T14:09:24.803987+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821085+0000","last_change":"2024-04-25T14:09:24.818883+0000","last_active":"2024-04-25T14:09:24.821085+0000","last_peered":"2024-04-25T14:09:24.821085+0000","last_clean":"2024-04-25T14:09:24.821085+0000","last_became_active":"2024-04-25T14:09:24.817518+0000","last_became_peered":"2024-04-25T14:09:24.817518+0000","last_unstale":"2024-04-25T14:09:24.821085+0000","last_undegraded":"2024-04-25T14:09:24.821085+0000","last_fullsized":"2024-04-25T14:09:24.821085+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:03.242163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801652+0000","last_change":"2024-04-25T14:09:24.801652+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801652+0000","last_undegraded":"2024-04-25T14:09:24.801652+0000","last_fullsized":"2024-04-25T14:09:24.801652+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.816749+0000","last_change":"2024-04-25T14:09:24.814702+0000","last_active":"2024-04-25T14:09:24.816749+0000","last_peered":"2024-04-25T14:09:24.816749+0000","last_clean":"2024-04-25T14:09:24.816749+0000","last_became_active":"2024-04-25T14:09:24.807948+0000","last_became_peered":"2024-04-25T14:09:24.807948+0000","last_unstale":"2024-04-25T14:09:24.816749+0000","last_undegraded":"2024-04-25T14:09:24.816749+0000","last_fullsized":"2024-04-25T14:09:24.816749+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:36.733366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.814203+0000","last_change":"2024-04-25T14:09:24.814203+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.814203+0000","last_undegraded":"2024-04-25T14:09:24.814203+0000","last_fullsized":"2024-04-25T14:09:24.814203+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.819629+0000","last_change":"2024-04-25T14:09:24.817779+0000","last_active":"2024-04-25T14:09:24.819629+0000","last_peered":"2024-04-25T14:09:24.819629+0000","last_clean":"2024-04-25T14:09:24.819629+0000","last_became_active":"2024-04-25T14:09:24.817493+0000","last_became_peered":"2024-04-25T14:09:24.817493+0000","last_unstale":"2024-04-25T14:09:24.819629+0000","last_undegraded":"2024-04-25T14:09:24.819629+0000","last_fullsized":"2024-04-25T14:09:24.819629+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:59.756189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.796715+0000","last_change":"2024-04-25T14:09:24.796715+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.796715+0000","last_undegraded":"2024-04-25T14:09:24.796715+0000","last_fullsized":"2024-04-25T14:09:24.796715+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.820996+0000","last_change":"2024-04-25T14:09:24.818436+0000","last_active":"2024-04-25T14:09:24.820996+0000","last_peered":"2024-04-25T14:09:24.820996+0000","last_clean":"2024-04-25T14:09:24.820996+0000","last_became_active":"2024-04-25T14:09:24.818157+0000","last_became_peered":"2024-04-25T14:09:24.818157+0000","last_unstale":"2024-04-25T14:09:24.820996+0000","last_undegraded":"2024-04-25T14:09:24.820996+0000","last_fullsized":"2024-04-25T14:09:24.820996+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:03.406811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.802382+0000","last_change":"2024-04-25T14:09:24.802382+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.802382+0000","last_undegraded":"2024-04-25T14:09:24.802382+0000","last_fullsized":"2024-04-25T14:09:24.802382+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.824340+0000","last_change":"2024-04-25T14:09:24.820985+0000","last_active":"2024-04-25T14:09:24.824340+0000","last_peered":"2024-04-25T14:09:24.824340+0000","last_clean":"2024-04-25T14:09:24.824340+0000","last_became_active":"2024-04-25T14:09:24.819953+0000","last_became_peered":"2024-04-25T14:09:24.819953+0000","last_unstale":"2024-04-25T14:09:24.824340+0000","last_undegraded":"2024-04-25T14:09:24.824340+0000","last_fullsized":"2024-04-25T14:09:24.824340+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:17.719124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801234+0000","last_change":"2024-04-25T14:09:24.801234+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801234+0000","last_undegraded":"2024-04-25T14:09:24.801234+0000","last_fullsized":"2024-04-25T14:09:24.801234+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.824307+0000","last_change":"2024-04-25T14:09:24.822304+0000","last_active":"2024-04-25T14:09:24.824307+0000","last_peered":"2024-04-25T14:09:24.824307+0000","last_clean":"2024-04-25T14:09:24.824307+0000","last_became_active":"2024-04-25T14:09:24.821946+0000","last_became_peered":"2024-04-25T14:09:24.821946+0000","last_unstale":"2024-04-25T14:09:24.824307+0000","last_undegraded":"2024-04-25T14:09:24.824307+0000","last_fullsized":"2024-04-25T14:09:24.824307+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:28.243010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.800017+0000","last_change":"2024-04-25T14:09:24.800017+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.800017+0000","last_undegraded":"2024-04-25T14:09:24.800017+0000","last_fullsized":"2024-04-25T14:09:24.800017+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.816511+0000","last_change":"2024-04-25T14:09:24.811162+0000","last_active":"2024-04-25T14:09:24.816511+0000","last_peered":"2024-04-25T14:09:24.816511+0000","last_clean":"2024-04-25T14:09:24.816511+0000","last_became_active":"2024-04-25T14:09:24.810194+0000","last_became_peered":"2024-04-25T14:09:24.810194+0000","last_unstale":"2024-04-25T14:09:24.816511+0000","last_undegraded":"2024-04-25T14:09:24.816511+0000","last_fullsized":"2024-04-25T14:09:24.816511+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:44.601365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.795988+0000","last_change":"2024-04-25T14:09:24.795988+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.795988+0000","last_undegraded":"2024-04-25T14:09:24.795988+0000","last_fullsized":"2024-04-25T14:09:24.795988+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.823676+0000","last_change":"2024-04-25T14:09:24.817347+0000","last_active":"2024-04-25T14:09:24.823676+0000","last_peered":"2024-04-25T14:09:24.823676+0000","last_clean":"2024-04-25T14:09:24.823676+0000","last_became_active":"2024-04-25T14:09:24.815968+0000","last_became_peered":"2024-04-25T14:09:24.815968+0000","last_unstale":"2024-04-25T14:09:24.823676+0000","last_undegraded":"2024-04-25T14:09:24.823676+0000","last_fullsized":"2024-04-25T14:09:24.823676+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:18.027462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.795863+0000","last_change":"2024-04-25T14:09:24.795863+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.795863+0000","last_undegraded":"2024-04-25T14:09:24.795863+0000","last_fullsized":"2024-04-25T14:09:24.795863+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.819672+0000","last_change":"2024-04-25T14:09:24.816306+0000","last_active":"2024-04-25T14:09:24.819672+0000","last_peered":"2024-04-25T14:09:24.819672+0000","last_clean":"2024-04-25T14:09:24.819672+0000","last_became_active":"2024-04-25T14:09:24.816004+0000","last_became_peered":"2024-04-25T14:09:24.816004+0000","last_unstale":"2024-04-25T14:09:24.819672+0000","last_undegraded":"2024-04-25T14:09:24.819672+0000","last_fullsized":"2024-04-25T14:09:24.819672+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:40.598001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"89'107","reported_seq":377,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.800089+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:09:24.800089+0000","last_peered":"2024-04-25T14:09:24.800089+0000","last_clean":"2024-04-25T14:09:24.800089+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:09:24.800089+0000","last_undegraded":"2024-04-25T14:09:24.800089+0000","last_fullsized":"2024-04-25T14:09:24.800089+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":366,"num_read_kb":935,"num_write":161,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":366,"num_read_kb":935,"num_write":161,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574916,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32940,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338900,"statfs":{"total":96636764160,"available":96603033600,"internally_reserved":0,"allocated":1798144,"data_stored":1188474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":598106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1216512,"data_stored":604773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1216512,"data_stored":604773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574917,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":598106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574916,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1204224,"data_stored":598106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":598106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574917,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32948,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338892,"statfs":{"total":96636764160,"available":96603025408,"internally_reserved":0,"allocated":1798144,"data_stored":1188474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:09:30.347 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:09:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.538+0000 7fb179400640 1 Processor -- start 2024-04-25T14:09:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.538+0000 7fb179400640 1 -- start start 2024-04-25T14:09:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.538+0000 7fb179400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb1741610d0 0x7fb1740b7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.538+0000 7fb179400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1740b2250 0x7fb1740b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.538+0000 7fb179400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1740b3750 con 0x7fb1740b2ba0 2024-04-25T14:09:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.538+0000 7fb179400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb1740b7f70 con 0x7fb1741610d0 2024-04-25T14:09:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.538+0000 7fb179400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb174161620 con 0x7fb1740b2250 2024-04-25T14:09:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.538+0000 7fb172800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb1741610d0 0x7fb1740b7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:30.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.538+0000 7fb173200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb1740b2ba0 0x7fb1740d15e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42876/0 (socket says 172.21.15.86:42876) 2024-04-25T14:09:30.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.538+0000 7fb173200640 1 -- 172.21.15.86:0/108102267 learned_addr learned my addr 172.21.15.86:0/108102267 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:30.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.538+0000 7fb169e00640 1 --2- 172.21.15.86:0/108102267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1740b2250 0x7fb1740b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:30.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.538+0000 7fb172800640 1 -- 172.21.15.86:0/108102267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1740b2250 msgr2=0x7fb1740b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:30.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.538+0000 7fb172800640 1 --2- 172.21.15.86:0/108102267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1740b2250 0x7fb1740b2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:30.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.538+0000 7fb172800640 1 -- 172.21.15.86:0/108102267 >> v1:172.21.15.86:6789/0 conn(0x7fb1740b2ba0 legacy=0x7fb1740d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:30.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.538+0000 7fb172800640 1 -- 172.21.15.86:0/108102267 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1740ce7b0 con 0x7fb1741610d0 2024-04-25T14:09:30.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.538+0000 7fb171e00640 1 -- 172.21.15.86:0/108102267 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3383338703 0 0) 0x7fb1740b3750 con 0x7fb1740b2ba0 2024-04-25T14:09:30.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.542+0000 7fb172800640 1 --2- 172.21.15.86:0/108102267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1741610d0 0x7fb1740b7a30 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fb164009dc0 tx=0x7fb16401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a96f985f160a07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:30.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.542+0000 7fb171e00640 1 -- 172.21.15.86:0/108102267 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1640b2070 con 0x7fb1741610d0 2024-04-25T14:09:30.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.542+0000 7fb171e00640 1 -- 172.21.15.86:0/108102267 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1640a5750 con 0x7fb1741610d0 2024-04-25T14:09:30.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.542+0000 7fb171e00640 1 -- 172.21.15.86:0/108102267 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1640a5b20 con 0x7fb1741610d0 2024-04-25T14:09:30.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.542+0000 7fb179400640 1 -- 172.21.15.86:0/108102267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1741610d0 msgr2=0x7fb1740b7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:30.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.542+0000 7fb179400640 1 --2- 172.21.15.86:0/108102267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1741610d0 0x7fb1740b7a30 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fb164009dc0 tx=0x7fb16401c890 comp rx=0 tx=0).stop 2024-04-25T14:09:30.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.542+0000 7fb179400640 1 -- 172.21.15.86:0/108102267 shutdown_connections 2024-04-25T14:09:30.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.542+0000 7fb179400640 1 --2- 172.21.15.86:0/108102267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1740b2250 0x7fb1740b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:30.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.542+0000 7fb179400640 1 --2- 172.21.15.86:0/108102267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1741610d0 0x7fb1740b7a30 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:30.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.542+0000 7fb179400640 1 -- 172.21.15.86:0/108102267 >> 172.21.15.86:0/108102267 conn(0x7fb17409ba30 msgr2=0x7fb1740b7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:30.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.542+0000 7fb179400640 1 -- 172.21.15.86:0/108102267 shutdown_connections 2024-04-25T14:09:30.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.542+0000 7fb179400640 1 -- 172.21.15.86:0/108102267 wait complete. 2024-04-25T14:09:30.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.546+0000 7fb179400640 1 Processor -- start 2024-04-25T14:09:30.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.546+0000 7fb179400640 1 -- start start 2024-04-25T14:09:30.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.546+0000 7fb179400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1740b2250 0x7fb17415c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:30.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.546+0000 7fb179400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1740b2ba0 0x7fb1740af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:30.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.546+0000 7fb179400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1741610d0 0x7fb1740c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:30.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.546+0000 7fb179400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb1740d2280 con 0x7fb1740b2ba0 2024-04-25T14:09:30.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.546+0000 7fb179400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb1740d2100 con 0x7fb1740b2250 2024-04-25T14:09:30.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.546+0000 7fb179400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb1740cf610 con 0x7fb1741610d0 2024-04-25T14:09:30.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.546+0000 7fb169e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1740b2ba0 0x7fb1740af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:30.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.546+0000 7fb169e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1740b2ba0 0x7fb1740af220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58080/0 (socket says 172.21.15.86:58080) 2024-04-25T14:09:30.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.546+0000 7fb169e00640 1 -- 172.21.15.86:0/1320306180 learned_addr learned my addr 172.21.15.86:0/1320306180 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:30.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.546+0000 7fb172800640 1 --2- 172.21.15.86:0/1320306180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1740b2250 0x7fb17415c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:30.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.550+0000 7fb173200640 1 --2- 172.21.15.86:0/1320306180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1741610d0 0x7fb1740c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:30.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.550+0000 7fb169e00640 1 -- 172.21.15.86:0/1320306180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1741610d0 msgr2=0x7fb1740c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:30.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.550+0000 7fb169e00640 1 --2- 172.21.15.86:0/1320306180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1741610d0 0x7fb1740c7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:30.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.550+0000 7fb169e00640 1 -- 172.21.15.86:0/1320306180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1740b2250 msgr2=0x7fb17415c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:30.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.550+0000 7fb169e00640 1 --2- 172.21.15.86:0/1320306180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1740b2250 0x7fb17415c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:30.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.550+0000 7fb169e00640 1 -- 172.21.15.86:0/1320306180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1740afea0 con 0x7fb1740b2ba0 2024-04-25T14:09:30.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.550+0000 7fb173200640 1 --2- 172.21.15.86:0/1320306180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1741610d0 0x7fb1740c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:30.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.550+0000 7fb169e00640 1 --2- 172.21.15.86:0/1320306180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1740b2ba0 0x7fb1740af220 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00d5d0 tx=0x7fb15c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:30.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.550+0000 7fb16b400640 1 -- 172.21.15.86:0/1320306180 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c0125f0 con 0x7fb1740b2ba0 2024-04-25T14:09:30.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.550+0000 7fb16b400640 1 -- 172.21.15.86:0/1320306180 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb15c019070 con 0x7fb1740b2ba0 2024-04-25T14:09:30.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.550+0000 7fb16b400640 1 -- 172.21.15.86:0/1320306180 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c0129b0 con 0x7fb1740b2ba0 2024-04-25T14:09:30.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.550+0000 7fb179400640 1 -- 172.21.15.86:0/1320306180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1740b0180 con 0x7fb1740b2ba0 2024-04-25T14:09:30.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.550+0000 7fb179400640 1 -- 172.21.15.86:0/1320306180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1740b0740 con 0x7fb1740b2ba0 2024-04-25T14:09:30.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.550+0000 7fb179400640 1 -- 172.21.15.86:0/1320306180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb174076bf0 con 0x7fb1740b2ba0 2024-04-25T14:09:30.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.550+0000 7fb16b400640 1 -- 172.21.15.86:0/1320306180 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7fb15c012b50 con 0x7fb1740b2ba0 2024-04-25T14:09:30.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.550+0000 7fb16b400640 1 --2- 172.21.15.86:0/1320306180 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fb15006f550 0x7fb150071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:30.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.554+0000 7fb16b400640 1 -- 172.21.15.86:0/1320306180 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7fb15c08f8e0 con 0x7fb1740b2ba0 2024-04-25T14:09:30.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.554+0000 7fb172800640 1 --2- 172.21.15.86:0/1320306180 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fb15006f550 0x7fb150071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:30.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.554+0000 7fb172800640 1 --2- 172.21.15.86:0/1320306180 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fb15006f550 0x7fb150071a60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb1640af000 tx=0x7fb1640a7070 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:30.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.558+0000 7fb16b400640 1 -- 172.21.15.86:0/1320306180 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb15c05c530 con 0x7fb1740b2ba0 2024-04-25T14:09:30.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.710+0000 7fb179400640 1 -- 172.21.15.86:0/1320306180 --> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb174074750 con 0x7fb15006f550 2024-04-25T14:09:30.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.718+0000 7fb16b400640 1 -- 172.21.15.86:0/1320306180 <== mgr.7650 v2:172.21.15.86:6832/4236874185 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246766 (secure 0 0 0) 0x7fb174074750 con 0x7fb15006f550 2024-04-25T14:09:30.717 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:30.718 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:09:30.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.718+0000 7fb179400640 1 -- 172.21.15.86:0/1320306180 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fb15006f550 msgr2=0x7fb150071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:30.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.718+0000 7fb179400640 1 --2- 172.21.15.86:0/1320306180 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fb15006f550 0x7fb150071a60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb1640af000 tx=0x7fb1640a7070 comp rx=0 tx=0).stop 2024-04-25T14:09:30.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.718+0000 7fb179400640 1 -- 172.21.15.86:0/1320306180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1740b2ba0 msgr2=0x7fb1740af220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:30.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.718+0000 7fb179400640 1 --2- 172.21.15.86:0/1320306180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1740b2ba0 0x7fb1740af220 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00d5d0 tx=0x7fb15c00da90 comp rx=0 tx=0).stop 2024-04-25T14:09:30.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.718+0000 7fb179400640 1 -- 172.21.15.86:0/1320306180 shutdown_connections 2024-04-25T14:09:30.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.718+0000 7fb179400640 1 --2- 172.21.15.86:0/1320306180 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fb15006f550 0x7fb150071a60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:30.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.718+0000 7fb179400640 1 --2- 172.21.15.86:0/1320306180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1741610d0 0x7fb1740c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:30.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.718+0000 7fb179400640 1 --2- 172.21.15.86:0/1320306180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1740b2ba0 0x7fb1740af220 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:30.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.718+0000 7fb179400640 1 --2- 172.21.15.86:0/1320306180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1740b2250 0x7fb17415c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:30.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.718+0000 7fb179400640 1 -- 172.21.15.86:0/1320306180 >> 172.21.15.86:0/1320306180 conn(0x7fb17409ba30 msgr2=0x7fb1740b7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:30.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.722+0000 7fb179400640 1 -- 172.21.15.86:0/1320306180 shutdown_connections 2024-04-25T14:09:30.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.722+0000 7fb179400640 1 -- 172.21.15.86:0/1320306180 wait complete. 2024-04-25T14:09:30.737 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T14:09:29.506130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":366,"num_read_kb":935,"num_write":161,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260076,"kb_used_data":10592,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714644,"statfs":{"total":773094113280,"available":772827795456,"internally_reserved":0,"allocated":10846208,"data_stored":5978918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.716486"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.811745+0000","last_change":"2024-04-25T14:09:24.811745+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.811745+0000","last_undegraded":"2024-04-25T14:09:24.811745+0000","last_fullsized":"2024-04-25T14:09:24.811745+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.805434+0000","last_change":"2024-04-25T14:09:24.805434+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.805434+0000","last_undegraded":"2024-04-25T14:09:24.805434+0000","last_fullsized":"2024-04-25T14:09:24.805434+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.806609+0000","last_change":"2024-04-25T14:09:24.806609+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.806609+0000","last_undegraded":"2024-04-25T14:09:24.806609+0000","last_fullsized":"2024-04-25T14:09:24.806609+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.814243+0000","last_change":"2024-04-25T14:09:24.814243+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.814243+0000","last_undegraded":"2024-04-25T14:09:24.814243+0000","last_fullsized":"2024-04-25T14:09:24.814243+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.796780+0000","last_change":"2024-04-25T14:09:24.796780+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.796780+0000","last_undegraded":"2024-04-25T14:09:24.796780+0000","last_fullsized":"2024-04-25T14:09:24.796780+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801913+0000","last_change":"2024-04-25T14:09:24.801913+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801913+0000","last_undegraded":"2024-04-25T14:09:24.801913+0000","last_fullsized":"2024-04-25T14:09:24.801913+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.805381+0000","last_change":"2024-04-25T14:09:24.805381+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.805381+0000","last_undegraded":"2024-04-25T14:09:24.805381+0000","last_fullsized":"2024-04-25T14:09:24.805381+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.805573+0000","last_change":"2024-04-25T14:09:24.805573+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.805573+0000","last_undegraded":"2024-04-25T14:09:24.805573+0000","last_fullsized":"2024-04-25T14:09:24.805573+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.802656+0000","last_change":"2024-04-25T14:09:24.802656+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.802656+0000","last_undegraded":"2024-04-25T14:09:24.802656+0000","last_fullsized":"2024-04-25T14:09:24.802656+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.796198+0000","last_change":"2024-04-25T14:09:24.796198+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.796198+0000","last_undegraded":"2024-04-25T14:09:24.796198+0000","last_fullsized":"2024-04-25T14:09:24.796198+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.797705+0000","last_change":"2024-04-25T14:09:24.797705+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.797705+0000","last_undegraded":"2024-04-25T14:09:24.797705+0000","last_fullsized":"2024-04-25T14:09:24.797705+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.803323+0000","last_change":"2024-04-25T14:09:24.803323+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.803323+0000","last_undegraded":"2024-04-25T14:09:24.803323+0000","last_fullsized":"2024-04-25T14:09:24.803323+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.814711+0000","last_change":"2024-04-25T14:09:24.814711+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.814711+0000","last_undegraded":"2024-04-25T14:09:24.814711+0000","last_fullsized":"2024-04-25T14:09:24.814711+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801188+0000","last_change":"2024-04-25T14:09:24.801188+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801188+0000","last_undegraded":"2024-04-25T14:09:24.801188+0000","last_fullsized":"2024-04-25T14:09:24.801188+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.800597+0000","last_change":"2024-04-25T14:09:24.800597+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.800597+0000","last_undegraded":"2024-04-25T14:09:24.800597+0000","last_fullsized":"2024-04-25T14:09:24.800597+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.799763+0000","last_change":"2024-04-25T14:09:24.799763+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.799763+0000","last_undegraded":"2024-04-25T14:09:24.799763+0000","last_fullsized":"2024-04-25T14:09:24.799763+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.806722+0000","last_change":"2024-04-25T14:09:24.806722+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.806722+0000","last_undegraded":"2024-04-25T14:09:24.806722+0000","last_fullsized":"2024-04-25T14:09:24.806722+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801800+0000","last_change":"2024-04-25T14:09:24.801800+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801800+0000","last_undegraded":"2024-04-25T14:09:24.801800+0000","last_fullsized":"2024-04-25T14:09:24.801800+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.805192+0000","last_change":"2024-04-25T14:09:24.805192+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.805192+0000","last_undegraded":"2024-04-25T14:09:24.805192+0000","last_fullsized":"2024-04-25T14:09:24.805192+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.802814+0000","last_change":"2024-04-25T14:09:24.802814+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.802814+0000","last_undegraded":"2024-04-25T14:09:24.802814+0000","last_fullsized":"2024-04-25T14:09:24.802814+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.800665+0000","last_change":"2024-04-25T14:09:24.800665+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.800665+0000","last_undegraded":"2024-04-25T14:09:24.800665+0000","last_fullsized":"2024-04-25T14:09:24.800665+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.797246+0000","last_change":"2024-04-25T14:09:24.797246+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.797246+0000","last_undegraded":"2024-04-25T14:09:24.797246+0000","last_fullsized":"2024-04-25T14:09:24.797246+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.804279+0000","last_change":"2024-04-25T14:09:24.804279+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.804279+0000","last_undegraded":"2024-04-25T14:09:24.804279+0000","last_fullsized":"2024-04-25T14:09:24.804279+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.795033+0000","last_change":"2024-04-25T14:09:24.795033+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.795033+0000","last_undegraded":"2024-04-25T14:09:24.795033+0000","last_fullsized":"2024-04-25T14:09:24.795033+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.803941+0000","last_change":"2024-04-25T14:09:24.803941+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.803941+0000","last_undegraded":"2024-04-25T14:09:24.803941+0000","last_fullsized":"2024-04-25T14:09:24.803941+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.803227+0000","last_change":"2024-04-25T14:09:24.803227+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.803227+0000","last_undegraded":"2024-04-25T14:09:24.803227+0000","last_fullsized":"2024-04-25T14:09:24.803227+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.797093+0000","last_change":"2024-04-25T14:09:24.797093+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.797093+0000","last_undegraded":"2024-04-25T14:09:24.797093+0000","last_fullsized":"2024-04-25T14:09:24.797093+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.808887+0000","last_change":"2024-04-25T14:09:24.808887+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.808887+0000","last_undegraded":"2024-04-25T14:09:24.808887+0000","last_fullsized":"2024-04-25T14:09:24.808887+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.800116+0000","last_change":"2024-04-25T14:09:24.800116+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.800116+0000","last_undegraded":"2024-04-25T14:09:24.800116+0000","last_fullsized":"2024-04-25T14:09:24.800116+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.805810+0000","last_change":"2024-04-25T14:09:24.805810+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.805810+0000","last_undegraded":"2024-04-25T14:09:24.805810+0000","last_fullsized":"2024-04-25T14:09:24.805810+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.804724+0000","last_change":"2024-04-25T14:09:24.804724+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.804724+0000","last_undegraded":"2024-04-25T14:09:24.804724+0000","last_fullsized":"2024-04-25T14:09:24.804724+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.805096+0000","last_change":"2024-04-25T14:09:24.805096+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.805096+0000","last_undegraded":"2024-04-25T14:09:24.805096+0000","last_fullsized":"2024-04-25T14:09:24.805096+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801802+0000","last_change":"2024-04-25T14:09:24.801802+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801802+0000","last_undegraded":"2024-04-25T14:09:24.801802+0000","last_fullsized":"2024-04-25T14:09:24.801802+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.823829+0000","last_change":"2024-04-25T14:09:24.817466+0000","last_active":"2024-04-25T14:09:24.823829+0000","last_peered":"2024-04-25T14:09:24.823829+0000","last_clean":"2024-04-25T14:09:24.823829+0000","last_became_active":"2024-04-25T14:09:24.817167+0000","last_became_peered":"2024-04-25T14:09:24.817167+0000","last_unstale":"2024-04-25T14:09:24.823829+0000","last_undegraded":"2024-04-25T14:09:24.823829+0000","last_fullsized":"2024-04-25T14:09:24.823829+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:02.374557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.803577+0000","last_change":"2024-04-25T14:09:24.803577+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.803577+0000","last_undegraded":"2024-04-25T14:09:24.803577+0000","last_fullsized":"2024-04-25T14:09:24.803577+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.824309+0000","last_change":"2024-04-25T14:09:24.819324+0000","last_active":"2024-04-25T14:09:24.824309+0000","last_peered":"2024-04-25T14:09:24.824309+0000","last_clean":"2024-04-25T14:09:24.824309+0000","last_became_active":"2024-04-25T14:09:24.818377+0000","last_became_peered":"2024-04-25T14:09:24.818377+0000","last_unstale":"2024-04-25T14:09:24.824309+0000","last_undegraded":"2024-04-25T14:09:24.824309+0000","last_fullsized":"2024-04-25T14:09:24.824309+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:38.935678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.802115+0000","last_change":"2024-04-25T14:09:24.802115+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.802115+0000","last_undegraded":"2024-04-25T14:09:24.802115+0000","last_fullsized":"2024-04-25T14:09:24.802115+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821745+0000","last_change":"2024-04-25T14:09:24.813915+0000","last_active":"2024-04-25T14:09:24.821745+0000","last_peered":"2024-04-25T14:09:24.821745+0000","last_clean":"2024-04-25T14:09:24.821745+0000","last_became_active":"2024-04-25T14:09:24.813610+0000","last_became_peered":"2024-04-25T14:09:24.813610+0000","last_unstale":"2024-04-25T14:09:24.821745+0000","last_undegraded":"2024-04-25T14:09:24.821745+0000","last_fullsized":"2024-04-25T14:09:24.821745+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:03.242163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.796164+0000","last_change":"2024-04-25T14:09:24.796164+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.796164+0000","last_undegraded":"2024-04-25T14:09:24.796164+0000","last_fullsized":"2024-04-25T14:09:24.796164+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.813538+0000","last_change":"2024-04-25T14:09:24.810780+0000","last_active":"2024-04-25T14:09:24.813538+0000","last_peered":"2024-04-25T14:09:24.813538+0000","last_clean":"2024-04-25T14:09:24.813538+0000","last_became_active":"2024-04-25T14:09:24.810462+0000","last_became_peered":"2024-04-25T14:09:24.810462+0000","last_unstale":"2024-04-25T14:09:24.813538+0000","last_undegraded":"2024-04-25T14:09:24.813538+0000","last_fullsized":"2024-04-25T14:09:24.813538+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:30.379959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.809107+0000","last_change":"2024-04-25T14:09:24.809107+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.809107+0000","last_undegraded":"2024-04-25T14:09:24.809107+0000","last_fullsized":"2024-04-25T14:09:24.809107+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.820528+0000","last_change":"2024-04-25T14:09:24.813955+0000","last_active":"2024-04-25T14:09:24.820528+0000","last_peered":"2024-04-25T14:09:24.820528+0000","last_clean":"2024-04-25T14:09:24.820528+0000","last_became_active":"2024-04-25T14:09:24.813645+0000","last_became_peered":"2024-04-25T14:09:24.813645+0000","last_unstale":"2024-04-25T14:09:24.820528+0000","last_undegraded":"2024-04-25T14:09:24.820528+0000","last_fullsized":"2024-04-25T14:09:24.820528+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:02.374557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801810+0000","last_change":"2024-04-25T14:09:24.801810+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801810+0000","last_undegraded":"2024-04-25T14:09:24.801810+0000","last_fullsized":"2024-04-25T14:09:24.801810+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821172+0000","last_change":"2024-04-25T14:09:24.814236+0000","last_active":"2024-04-25T14:09:24.821172+0000","last_peered":"2024-04-25T14:09:24.821172+0000","last_clean":"2024-04-25T14:09:24.821172+0000","last_became_active":"2024-04-25T14:09:24.813909+0000","last_became_peered":"2024-04-25T14:09:24.813909+0000","last_unstale":"2024-04-25T14:09:24.821172+0000","last_undegraded":"2024-04-25T14:09:24.821172+0000","last_fullsized":"2024-04-25T14:09:24.821172+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:57.671866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801284+0000","last_change":"2024-04-25T14:09:24.801284+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801284+0000","last_undegraded":"2024-04-25T14:09:24.801284+0000","last_fullsized":"2024-04-25T14:09:24.801284+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821282+0000","last_change":"2024-04-25T14:09:24.811288+0000","last_active":"2024-04-25T14:09:24.821282+0000","last_peered":"2024-04-25T14:09:24.821282+0000","last_clean":"2024-04-25T14:09:24.821282+0000","last_became_active":"2024-04-25T14:09:24.809633+0000","last_became_peered":"2024-04-25T14:09:24.809633+0000","last_unstale":"2024-04-25T14:09:24.821282+0000","last_undegraded":"2024-04-25T14:09:24.821282+0000","last_fullsized":"2024-04-25T14:09:24.821282+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:57.671866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.809114+0000","last_change":"2024-04-25T14:09:24.809114+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.809114+0000","last_undegraded":"2024-04-25T14:09:24.809114+0000","last_fullsized":"2024-04-25T14:09:24.809114+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.806864+0000","last_change":"2024-04-25T14:09:24.799767+0000","last_active":"2024-04-25T14:09:24.806864+0000","last_peered":"2024-04-25T14:09:24.806864+0000","last_clean":"2024-04-25T14:09:24.806864+0000","last_became_active":"2024-04-25T14:09:24.799440+0000","last_became_peered":"2024-04-25T14:09:24.799440+0000","last_unstale":"2024-04-25T14:09:24.806864+0000","last_undegraded":"2024-04-25T14:09:24.806864+0000","last_fullsized":"2024-04-25T14:09:24.806864+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:44.194963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.802420+0000","last_change":"2024-04-25T14:09:24.802420+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.802420+0000","last_undegraded":"2024-04-25T14:09:24.802420+0000","last_fullsized":"2024-04-25T14:09:24.802420+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821771+0000","last_change":"2024-04-25T14:09:24.813861+0000","last_active":"2024-04-25T14:09:24.821771+0000","last_peered":"2024-04-25T14:09:24.821771+0000","last_clean":"2024-04-25T14:09:24.821771+0000","last_became_active":"2024-04-25T14:09:24.813523+0000","last_became_peered":"2024-04-25T14:09:24.813523+0000","last_unstale":"2024-04-25T14:09:24.821771+0000","last_undegraded":"2024-04-25T14:09:24.821771+0000","last_fullsized":"2024-04-25T14:09:24.821771+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:03.406811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.795235+0000","last_change":"2024-04-25T14:09:24.795235+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.795235+0000","last_undegraded":"2024-04-25T14:09:24.795235+0000","last_fullsized":"2024-04-25T14:09:24.795235+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.823970+0000","last_change":"2024-04-25T14:09:24.819250+0000","last_active":"2024-04-25T14:09:24.823970+0000","last_peered":"2024-04-25T14:09:24.823970+0000","last_clean":"2024-04-25T14:09:24.823970+0000","last_became_active":"2024-04-25T14:09:24.818892+0000","last_became_peered":"2024-04-25T14:09:24.818892+0000","last_unstale":"2024-04-25T14:09:24.823970+0000","last_undegraded":"2024-04-25T14:09:24.823970+0000","last_fullsized":"2024-04-25T14:09:24.823970+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:12.433048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.799314+0000","last_change":"2024-04-25T14:09:24.799314+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.799314+0000","last_undegraded":"2024-04-25T14:09:24.799314+0000","last_fullsized":"2024-04-25T14:09:24.799314+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.812722+0000","last_change":"2024-04-25T14:09:24.808328+0000","last_active":"2024-04-25T14:09:24.812722+0000","last_peered":"2024-04-25T14:09:24.812722+0000","last_clean":"2024-04-25T14:09:24.812722+0000","last_became_active":"2024-04-25T14:09:24.807984+0000","last_became_peered":"2024-04-25T14:09:24.807984+0000","last_unstale":"2024-04-25T14:09:24.812722+0000","last_undegraded":"2024-04-25T14:09:24.812722+0000","last_fullsized":"2024-04-25T14:09:24.812722+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:50.180573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.795828+0000","last_change":"2024-04-25T14:09:24.795828+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.795828+0000","last_undegraded":"2024-04-25T14:09:24.795828+0000","last_fullsized":"2024-04-25T14:09:24.795828+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.819182+0000","last_change":"2024-04-25T14:09:24.817277+0000","last_active":"2024-04-25T14:09:24.819182+0000","last_peered":"2024-04-25T14:09:24.819182+0000","last_clean":"2024-04-25T14:09:24.819182+0000","last_became_active":"2024-04-25T14:09:24.816793+0000","last_became_peered":"2024-04-25T14:09:24.816793+0000","last_unstale":"2024-04-25T14:09:24.819182+0000","last_undegraded":"2024-04-25T14:09:24.819182+0000","last_fullsized":"2024-04-25T14:09:24.819182+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:47.152512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.795817+0000","last_change":"2024-04-25T14:09:24.795817+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.795817+0000","last_undegraded":"2024-04-25T14:09:24.795817+0000","last_fullsized":"2024-04-25T14:09:24.795817+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.811590+0000","last_change":"2024-04-25T14:09:24.808496+0000","last_active":"2024-04-25T14:09:24.811590+0000","last_peered":"2024-04-25T14:09:24.811590+0000","last_clean":"2024-04-25T14:09:24.811590+0000","last_became_active":"2024-04-25T14:09:24.807948+0000","last_became_peered":"2024-04-25T14:09:24.807948+0000","last_unstale":"2024-04-25T14:09:24.811590+0000","last_undegraded":"2024-04-25T14:09:24.811590+0000","last_fullsized":"2024-04-25T14:09:24.811590+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:57.671866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.804276+0000","last_change":"2024-04-25T14:09:24.804276+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.804276+0000","last_undegraded":"2024-04-25T14:09:24.804276+0000","last_fullsized":"2024-04-25T14:09:24.804276+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.815553+0000","last_change":"2024-04-25T14:09:24.810654+0000","last_active":"2024-04-25T14:09:24.815553+0000","last_peered":"2024-04-25T14:09:24.815553+0000","last_clean":"2024-04-25T14:09:24.815553+0000","last_became_active":"2024-04-25T14:09:24.810344+0000","last_became_peered":"2024-04-25T14:09:24.810344+0000","last_unstale":"2024-04-25T14:09:24.815553+0000","last_undegraded":"2024-04-25T14:09:24.815553+0000","last_fullsized":"2024-04-25T14:09:24.815553+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:50.180573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.800603+0000","last_change":"2024-04-25T14:09:24.800603+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.800603+0000","last_undegraded":"2024-04-25T14:09:24.800603+0000","last_fullsized":"2024-04-25T14:09:24.800603+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.823748+0000","last_change":"2024-04-25T14:09:24.817108+0000","last_active":"2024-04-25T14:09:24.823748+0000","last_peered":"2024-04-25T14:09:24.823748+0000","last_clean":"2024-04-25T14:09:24.823748+0000","last_became_active":"2024-04-25T14:09:24.816826+0000","last_became_peered":"2024-04-25T14:09:24.816826+0000","last_unstale":"2024-04-25T14:09:24.823748+0000","last_undegraded":"2024-04-25T14:09:24.823748+0000","last_fullsized":"2024-04-25T14:09:24.823748+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:36.790595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.803123+0000","last_change":"2024-04-25T14:09:24.803123+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.803123+0000","last_undegraded":"2024-04-25T14:09:24.803123+0000","last_fullsized":"2024-04-25T14:09:24.803123+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821222+0000","last_change":"2024-04-25T14:09:24.811340+0000","last_active":"2024-04-25T14:09:24.821222+0000","last_peered":"2024-04-25T14:09:24.821222+0000","last_clean":"2024-04-25T14:09:24.821222+0000","last_became_active":"2024-04-25T14:09:24.809685+0000","last_became_peered":"2024-04-25T14:09:24.809685+0000","last_unstale":"2024-04-25T14:09:24.821222+0000","last_undegraded":"2024-04-25T14:09:24.821222+0000","last_fullsized":"2024-04-25T14:09:24.821222+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:30.379959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.804614+0000","last_change":"2024-04-25T14:09:24.804614+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.804614+0000","last_undegraded":"2024-04-25T14:09:24.804614+0000","last_fullsized":"2024-04-25T14:09:24.804614+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.824699+0000","last_change":"2024-04-25T14:09:24.822761+0000","last_active":"2024-04-25T14:09:24.824699+0000","last_peered":"2024-04-25T14:09:24.824699+0000","last_clean":"2024-04-25T14:09:24.824699+0000","last_became_active":"2024-04-25T14:09:24.822428+0000","last_became_peered":"2024-04-25T14:09:24.822428+0000","last_unstale":"2024-04-25T14:09:24.824699+0000","last_undegraded":"2024-04-25T14:09:24.824699+0000","last_fullsized":"2024-04-25T14:09:24.824699+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:51.227996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.800253+0000","last_change":"2024-04-25T14:09:24.800253+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.800253+0000","last_undegraded":"2024-04-25T14:09:24.800253+0000","last_fullsized":"2024-04-25T14:09:24.800253+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821955+0000","last_change":"2024-04-25T14:09:24.817271+0000","last_active":"2024-04-25T14:09:24.821955+0000","last_peered":"2024-04-25T14:09:24.821955+0000","last_clean":"2024-04-25T14:09:24.821955+0000","last_became_active":"2024-04-25T14:09:24.816723+0000","last_became_peered":"2024-04-25T14:09:24.816723+0000","last_unstale":"2024-04-25T14:09:24.821955+0000","last_undegraded":"2024-04-25T14:09:24.821955+0000","last_fullsized":"2024-04-25T14:09:24.821955+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:29:29.089649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.796305+0000","last_change":"2024-04-25T14:09:24.796305+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.796305+0000","last_undegraded":"2024-04-25T14:09:24.796305+0000","last_fullsized":"2024-04-25T14:09:24.796305+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821149+0000","last_change":"2024-04-25T14:09:24.814155+0000","last_active":"2024-04-25T14:09:24.821149+0000","last_peered":"2024-04-25T14:09:24.821149+0000","last_clean":"2024-04-25T14:09:24.821149+0000","last_became_active":"2024-04-25T14:09:24.812762+0000","last_became_peered":"2024-04-25T14:09:24.812762+0000","last_unstale":"2024-04-25T14:09:24.821149+0000","last_undegraded":"2024-04-25T14:09:24.821149+0000","last_fullsized":"2024-04-25T14:09:24.821149+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:30.379959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.802445+0000","last_change":"2024-04-25T14:09:24.802445+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.802445+0000","last_undegraded":"2024-04-25T14:09:24.802445+0000","last_fullsized":"2024-04-25T14:09:24.802445+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.816847+0000","last_change":"2024-04-25T14:09:24.811205+0000","last_active":"2024-04-25T14:09:24.816847+0000","last_peered":"2024-04-25T14:09:24.816847+0000","last_clean":"2024-04-25T14:09:24.816847+0000","last_became_active":"2024-04-25T14:09:24.810270+0000","last_became_peered":"2024-04-25T14:09:24.810270+0000","last_unstale":"2024-04-25T14:09:24.816847+0000","last_undegraded":"2024-04-25T14:09:24.816847+0000","last_fullsized":"2024-04-25T14:09:24.816847+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:30.864192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.804489+0000","last_change":"2024-04-25T14:09:24.804489+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.804489+0000","last_undegraded":"2024-04-25T14:09:24.804489+0000","last_fullsized":"2024-04-25T14:09:24.804489+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.816656+0000","last_change":"2024-04-25T14:09:24.808594+0000","last_active":"2024-04-25T14:09:24.816656+0000","last_peered":"2024-04-25T14:09:24.816656+0000","last_clean":"2024-04-25T14:09:24.816656+0000","last_became_active":"2024-04-25T14:09:24.807904+0000","last_became_peered":"2024-04-25T14:09:24.807904+0000","last_unstale":"2024-04-25T14:09:24.816656+0000","last_undegraded":"2024-04-25T14:09:24.816656+0000","last_fullsized":"2024-04-25T14:09:24.816656+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:55.195611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.815279+0000","last_change":"2024-04-25T14:09:24.815279+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.815279+0000","last_undegraded":"2024-04-25T14:09:24.815279+0000","last_fullsized":"2024-04-25T14:09:24.815279+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.819227+0000","last_change":"2024-04-25T14:09:24.816525+0000","last_active":"2024-04-25T14:09:24.819227+0000","last_peered":"2024-04-25T14:09:24.819227+0000","last_clean":"2024-04-25T14:09:24.819227+0000","last_became_active":"2024-04-25T14:09:24.815546+0000","last_became_peered":"2024-04-25T14:09:24.815546+0000","last_unstale":"2024-04-25T14:09:24.819227+0000","last_undegraded":"2024-04-25T14:09:24.819227+0000","last_fullsized":"2024-04-25T14:09:24.819227+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:46.765820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801719+0000","last_change":"2024-04-25T14:09:24.801719+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801719+0000","last_undegraded":"2024-04-25T14:09:24.801719+0000","last_fullsized":"2024-04-25T14:09:24.801719+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.817516+0000","last_change":"2024-04-25T14:09:24.813608+0000","last_active":"2024-04-25T14:09:24.817516+0000","last_peered":"2024-04-25T14:09:24.817516+0000","last_clean":"2024-04-25T14:09:24.817516+0000","last_became_active":"2024-04-25T14:09:24.813300+0000","last_became_peered":"2024-04-25T14:09:24.813300+0000","last_unstale":"2024-04-25T14:09:24.817516+0000","last_undegraded":"2024-04-25T14:09:24.817516+0000","last_fullsized":"2024-04-25T14:09:24.817516+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:41.291113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.803987+0000","last_change":"2024-04-25T14:09:24.803987+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.803987+0000","last_undegraded":"2024-04-25T14:09:24.803987+0000","last_fullsized":"2024-04-25T14:09:24.803987+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821085+0000","last_change":"2024-04-25T14:09:24.818883+0000","last_active":"2024-04-25T14:09:24.821085+0000","last_peered":"2024-04-25T14:09:24.821085+0000","last_clean":"2024-04-25T14:09:24.821085+0000","last_became_active":"2024-04-25T14:09:24.817518+0000","last_became_peered":"2024-04-25T14:09:24.817518+0000","last_unstale":"2024-04-25T14:09:24.821085+0000","last_undegraded":"2024-04-25T14:09:24.821085+0000","last_fullsized":"2024-04-25T14:09:24.821085+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:03.242163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801652+0000","last_change":"2024-04-25T14:09:24.801652+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801652+0000","last_undegraded":"2024-04-25T14:09:24.801652+0000","last_fullsized":"2024-04-25T14:09:24.801652+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.816749+0000","last_change":"2024-04-25T14:09:24.814702+0000","last_active":"2024-04-25T14:09:24.816749+0000","last_peered":"2024-04-25T14:09:24.816749+0000","last_clean":"2024-04-25T14:09:24.816749+0000","last_became_active":"2024-04-25T14:09:24.807948+0000","last_became_peered":"2024-04-25T14:09:24.807948+0000","last_unstale":"2024-04-25T14:09:24.816749+0000","last_undegraded":"2024-04-25T14:09:24.816749+0000","last_fullsized":"2024-04-25T14:09:24.816749+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:36.733366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.814203+0000","last_change":"2024-04-25T14:09:24.814203+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.814203+0000","last_undegraded":"2024-04-25T14:09:24.814203+0000","last_fullsized":"2024-04-25T14:09:24.814203+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.819629+0000","last_change":"2024-04-25T14:09:24.817779+0000","last_active":"2024-04-25T14:09:24.819629+0000","last_peered":"2024-04-25T14:09:24.819629+0000","last_clean":"2024-04-25T14:09:24.819629+0000","last_became_active":"2024-04-25T14:09:24.817493+0000","last_became_peered":"2024-04-25T14:09:24.817493+0000","last_unstale":"2024-04-25T14:09:24.819629+0000","last_undegraded":"2024-04-25T14:09:24.819629+0000","last_fullsized":"2024-04-25T14:09:24.819629+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:59.756189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.796715+0000","last_change":"2024-04-25T14:09:24.796715+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.796715+0000","last_undegraded":"2024-04-25T14:09:24.796715+0000","last_fullsized":"2024-04-25T14:09:24.796715+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.820996+0000","last_change":"2024-04-25T14:09:24.818436+0000","last_active":"2024-04-25T14:09:24.820996+0000","last_peered":"2024-04-25T14:09:24.820996+0000","last_clean":"2024-04-25T14:09:24.820996+0000","last_became_active":"2024-04-25T14:09:24.818157+0000","last_became_peered":"2024-04-25T14:09:24.818157+0000","last_unstale":"2024-04-25T14:09:24.820996+0000","last_undegraded":"2024-04-25T14:09:24.820996+0000","last_fullsized":"2024-04-25T14:09:24.820996+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:03.406811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.802382+0000","last_change":"2024-04-25T14:09:24.802382+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.802382+0000","last_undegraded":"2024-04-25T14:09:24.802382+0000","last_fullsized":"2024-04-25T14:09:24.802382+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.824340+0000","last_change":"2024-04-25T14:09:24.820985+0000","last_active":"2024-04-25T14:09:24.824340+0000","last_peered":"2024-04-25T14:09:24.824340+0000","last_clean":"2024-04-25T14:09:24.824340+0000","last_became_active":"2024-04-25T14:09:24.819953+0000","last_became_peered":"2024-04-25T14:09:24.819953+0000","last_unstale":"2024-04-25T14:09:24.824340+0000","last_undegraded":"2024-04-25T14:09:24.824340+0000","last_fullsized":"2024-04-25T14:09:24.824340+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:17.719124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801234+0000","last_change":"2024-04-25T14:09:24.801234+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801234+0000","last_undegraded":"2024-04-25T14:09:24.801234+0000","last_fullsized":"2024-04-25T14:09:24.801234+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.824307+0000","last_change":"2024-04-25T14:09:24.822304+0000","last_active":"2024-04-25T14:09:24.824307+0000","last_peered":"2024-04-25T14:09:24.824307+0000","last_clean":"2024-04-25T14:09:24.824307+0000","last_became_active":"2024-04-25T14:09:24.821946+0000","last_became_peered":"2024-04-25T14:09:24.821946+0000","last_unstale":"2024-04-25T14:09:24.824307+0000","last_undegraded":"2024-04-25T14:09:24.824307+0000","last_fullsized":"2024-04-25T14:09:24.824307+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:28.243010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.800017+0000","last_change":"2024-04-25T14:09:24.800017+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.800017+0000","last_undegraded":"2024-04-25T14:09:24.800017+0000","last_fullsized":"2024-04-25T14:09:24.800017+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.816511+0000","last_change":"2024-04-25T14:09:24.811162+0000","last_active":"2024-04-25T14:09:24.816511+0000","last_peered":"2024-04-25T14:09:24.816511+0000","last_clean":"2024-04-25T14:09:24.816511+0000","last_became_active":"2024-04-25T14:09:24.810194+0000","last_became_peered":"2024-04-25T14:09:24.810194+0000","last_unstale":"2024-04-25T14:09:24.816511+0000","last_undegraded":"2024-04-25T14:09:24.816511+0000","last_fullsized":"2024-04-25T14:09:24.816511+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:44.601365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.795988+0000","last_change":"2024-04-25T14:09:24.795988+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.795988+0000","last_undegraded":"2024-04-25T14:09:24.795988+0000","last_fullsized":"2024-04-25T14:09:24.795988+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.823676+0000","last_change":"2024-04-25T14:09:24.817347+0000","last_active":"2024-04-25T14:09:24.823676+0000","last_peered":"2024-04-25T14:09:24.823676+0000","last_clean":"2024-04-25T14:09:24.823676+0000","last_became_active":"2024-04-25T14:09:24.815968+0000","last_became_peered":"2024-04-25T14:09:24.815968+0000","last_unstale":"2024-04-25T14:09:24.823676+0000","last_undegraded":"2024-04-25T14:09:24.823676+0000","last_fullsized":"2024-04-25T14:09:24.823676+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:18.027462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.795863+0000","last_change":"2024-04-25T14:09:24.795863+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.795863+0000","last_undegraded":"2024-04-25T14:09:24.795863+0000","last_fullsized":"2024-04-25T14:09:24.795863+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.819672+0000","last_change":"2024-04-25T14:09:24.816306+0000","last_active":"2024-04-25T14:09:24.819672+0000","last_peered":"2024-04-25T14:09:24.819672+0000","last_clean":"2024-04-25T14:09:24.819672+0000","last_became_active":"2024-04-25T14:09:24.816004+0000","last_became_peered":"2024-04-25T14:09:24.816004+0000","last_unstale":"2024-04-25T14:09:24.819672+0000","last_undegraded":"2024-04-25T14:09:24.819672+0000","last_fullsized":"2024-04-25T14:09:24.819672+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:40.598001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"89'107","reported_seq":377,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.800089+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:09:24.800089+0000","last_peered":"2024-04-25T14:09:24.800089+0000","last_clean":"2024-04-25T14:09:24.800089+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:09:24.800089+0000","last_undegraded":"2024-04-25T14:09:24.800089+0000","last_fullsized":"2024-04-25T14:09:24.800089+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":366,"num_read_kb":935,"num_write":161,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":366,"num_read_kb":935,"num_write":161,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574916,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32940,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338900,"statfs":{"total":96636764160,"available":96603033600,"internally_reserved":0,"allocated":1798144,"data_stored":1188474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":598106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1216512,"data_stored":604773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1216512,"data_stored":604773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574917,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":598106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574916,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1204224,"data_stored":598106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":598106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574917,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32948,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338892,"statfs":{"total":96636764160,"available":96603025408,"internally_reserved":0,"allocated":1798144,"data_stored":1188474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:09:30.741 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3f is not active+clean 2024-04-25T14:09:30.741 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.811745+0000', 'last_change': '2024-04-25T14:09:24.811745+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.811745+0000', 'last_undegraded': '2024-04-25T14:09:24.811745+0000', 'last_fullsized': '2024-04-25T14:09:24.811745+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:09:30.741 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3e is not active+clean 2024-04-25T14:09:30.741 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.805434+0000', 'last_change': '2024-04-25T14:09:24.805434+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.805434+0000', 'last_undegraded': '2024-04-25T14:09:24.805434+0000', 'last_fullsized': '2024-04-25T14:09:24.805434+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:09:30.741 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3d is not active+clean 2024-04-25T14:09:30.741 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.806609+0000', 'last_change': '2024-04-25T14:09:24.806609+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.806609+0000', 'last_undegraded': '2024-04-25T14:09:24.806609+0000', 'last_fullsized': '2024-04-25T14:09:24.806609+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:09:30.741 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3c is not active+clean 2024-04-25T14:09:30.741 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.814243+0000', 'last_change': '2024-04-25T14:09:24.814243+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.814243+0000', 'last_undegraded': '2024-04-25T14:09:24.814243+0000', 'last_fullsized': '2024-04-25T14:09:24.814243+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:09:30.741 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3b is not active+clean 2024-04-25T14:09:30.741 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.796780+0000', 'last_change': '2024-04-25T14:09:24.796780+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.796780+0000', 'last_undegraded': '2024-04-25T14:09:24.796780+0000', 'last_fullsized': '2024-04-25T14:09:24.796780+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:09:30.742 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3a is not active+clean 2024-04-25T14:09:30.742 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.801913+0000', 'last_change': '2024-04-25T14:09:24.801913+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.801913+0000', 'last_undegraded': '2024-04-25T14:09:24.801913+0000', 'last_fullsized': '2024-04-25T14:09:24.801913+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:09:30.742 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.39 is not active+clean 2024-04-25T14:09:30.742 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.805381+0000', 'last_change': '2024-04-25T14:09:24.805381+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.805381+0000', 'last_undegraded': '2024-04-25T14:09:24.805381+0000', 'last_fullsized': '2024-04-25T14:09:24.805381+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:09:30.742 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.38 is not active+clean 2024-04-25T14:09:30.742 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.805573+0000', 'last_change': '2024-04-25T14:09:24.805573+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.805573+0000', 'last_undegraded': '2024-04-25T14:09:24.805573+0000', 'last_fullsized': '2024-04-25T14:09:24.805573+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:09:30.742 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.37 is not active+clean 2024-04-25T14:09:30.742 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.802656+0000', 'last_change': '2024-04-25T14:09:24.802656+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.802656+0000', 'last_undegraded': '2024-04-25T14:09:24.802656+0000', 'last_fullsized': '2024-04-25T14:09:24.802656+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:09:30.742 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.36 is not active+clean 2024-04-25T14:09:30.742 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.796198+0000', 'last_change': '2024-04-25T14:09:24.796198+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.796198+0000', 'last_undegraded': '2024-04-25T14:09:24.796198+0000', 'last_fullsized': '2024-04-25T14:09:24.796198+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:09:30.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.35 is not active+clean 2024-04-25T14:09:30.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.797705+0000', 'last_change': '2024-04-25T14:09:24.797705+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.797705+0000', 'last_undegraded': '2024-04-25T14:09:24.797705+0000', 'last_fullsized': '2024-04-25T14:09:24.797705+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:09:30.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.34 is not active+clean 2024-04-25T14:09:30.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.803323+0000', 'last_change': '2024-04-25T14:09:24.803323+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.803323+0000', 'last_undegraded': '2024-04-25T14:09:24.803323+0000', 'last_fullsized': '2024-04-25T14:09:24.803323+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:09:30.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.33 is not active+clean 2024-04-25T14:09:30.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.814711+0000', 'last_change': '2024-04-25T14:09:24.814711+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.814711+0000', 'last_undegraded': '2024-04-25T14:09:24.814711+0000', 'last_fullsized': '2024-04-25T14:09:24.814711+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:09:30.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.32 is not active+clean 2024-04-25T14:09:30.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.801188+0000', 'last_change': '2024-04-25T14:09:24.801188+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.801188+0000', 'last_undegraded': '2024-04-25T14:09:24.801188+0000', 'last_fullsized': '2024-04-25T14:09:24.801188+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:09:30.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.31 is not active+clean 2024-04-25T14:09:30.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.800597+0000', 'last_change': '2024-04-25T14:09:24.800597+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.800597+0000', 'last_undegraded': '2024-04-25T14:09:24.800597+0000', 'last_fullsized': '2024-04-25T14:09:24.800597+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:09:30.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.30 is not active+clean 2024-04-25T14:09:30.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.799763+0000', 'last_change': '2024-04-25T14:09:24.799763+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.799763+0000', 'last_undegraded': '2024-04-25T14:09:24.799763+0000', 'last_fullsized': '2024-04-25T14:09:24.799763+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:09:30.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2f is not active+clean 2024-04-25T14:09:30.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.806722+0000', 'last_change': '2024-04-25T14:09:24.806722+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.806722+0000', 'last_undegraded': '2024-04-25T14:09:24.806722+0000', 'last_fullsized': '2024-04-25T14:09:24.806722+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:09:30.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2e is not active+clean 2024-04-25T14:09:30.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.801800+0000', 'last_change': '2024-04-25T14:09:24.801800+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.801800+0000', 'last_undegraded': '2024-04-25T14:09:24.801800+0000', 'last_fullsized': '2024-04-25T14:09:24.801800+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:09:30.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2d is not active+clean 2024-04-25T14:09:30.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.805192+0000', 'last_change': '2024-04-25T14:09:24.805192+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.805192+0000', 'last_undegraded': '2024-04-25T14:09:24.805192+0000', 'last_fullsized': '2024-04-25T14:09:24.805192+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:09:30.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2c is not active+clean 2024-04-25T14:09:30.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.802814+0000', 'last_change': '2024-04-25T14:09:24.802814+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.802814+0000', 'last_undegraded': '2024-04-25T14:09:24.802814+0000', 'last_fullsized': '2024-04-25T14:09:24.802814+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:09:30.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2b is not active+clean 2024-04-25T14:09:30.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.800665+0000', 'last_change': '2024-04-25T14:09:24.800665+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.800665+0000', 'last_undegraded': '2024-04-25T14:09:24.800665+0000', 'last_fullsized': '2024-04-25T14:09:24.800665+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:09:30.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2a is not active+clean 2024-04-25T14:09:30.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.797246+0000', 'last_change': '2024-04-25T14:09:24.797246+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.797246+0000', 'last_undegraded': '2024-04-25T14:09:24.797246+0000', 'last_fullsized': '2024-04-25T14:09:24.797246+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:09:30.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.29 is not active+clean 2024-04-25T14:09:30.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.804279+0000', 'last_change': '2024-04-25T14:09:24.804279+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.804279+0000', 'last_undegraded': '2024-04-25T14:09:24.804279+0000', 'last_fullsized': '2024-04-25T14:09:24.804279+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:09:30.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.28 is not active+clean 2024-04-25T14:09:30.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.795033+0000', 'last_change': '2024-04-25T14:09:24.795033+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.795033+0000', 'last_undegraded': '2024-04-25T14:09:24.795033+0000', 'last_fullsized': '2024-04-25T14:09:24.795033+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:09:30.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.27 is not active+clean 2024-04-25T14:09:30.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.803941+0000', 'last_change': '2024-04-25T14:09:24.803941+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.803941+0000', 'last_undegraded': '2024-04-25T14:09:24.803941+0000', 'last_fullsized': '2024-04-25T14:09:24.803941+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:09:30.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.26 is not active+clean 2024-04-25T14:09:30.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.803227+0000', 'last_change': '2024-04-25T14:09:24.803227+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.803227+0000', 'last_undegraded': '2024-04-25T14:09:24.803227+0000', 'last_fullsized': '2024-04-25T14:09:24.803227+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:09:30.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.25 is not active+clean 2024-04-25T14:09:30.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.797093+0000', 'last_change': '2024-04-25T14:09:24.797093+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.797093+0000', 'last_undegraded': '2024-04-25T14:09:24.797093+0000', 'last_fullsized': '2024-04-25T14:09:24.797093+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:09:30.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.24 is not active+clean 2024-04-25T14:09:30.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.808887+0000', 'last_change': '2024-04-25T14:09:24.808887+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.808887+0000', 'last_undegraded': '2024-04-25T14:09:24.808887+0000', 'last_fullsized': '2024-04-25T14:09:24.808887+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:09:30.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.23 is not active+clean 2024-04-25T14:09:30.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.800116+0000', 'last_change': '2024-04-25T14:09:24.800116+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.800116+0000', 'last_undegraded': '2024-04-25T14:09:24.800116+0000', 'last_fullsized': '2024-04-25T14:09:24.800116+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:09:30.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.22 is not active+clean 2024-04-25T14:09:30.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.805810+0000', 'last_change': '2024-04-25T14:09:24.805810+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.805810+0000', 'last_undegraded': '2024-04-25T14:09:24.805810+0000', 'last_fullsized': '2024-04-25T14:09:24.805810+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:09:30.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.21 is not active+clean 2024-04-25T14:09:30.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.804724+0000', 'last_change': '2024-04-25T14:09:24.804724+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.804724+0000', 'last_undegraded': '2024-04-25T14:09:24.804724+0000', 'last_fullsized': '2024-04-25T14:09:24.804724+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:09:30.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.20 is not active+clean 2024-04-25T14:09:30.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.805096+0000', 'last_change': '2024-04-25T14:09:24.805096+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.805096+0000', 'last_undegraded': '2024-04-25T14:09:24.805096+0000', 'last_fullsized': '2024-04-25T14:09:24.805096+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:09:30.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.0 is not active+clean 2024-04-25T14:09:30.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.801802+0000', 'last_change': '2024-04-25T14:09:24.801802+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.801802+0000', 'last_undegraded': '2024-04-25T14:09:24.801802+0000', 'last_fullsized': '2024-04-25T14:09:24.801802+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:09:30.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1 is not active+clean 2024-04-25T14:09:30.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.803577+0000', 'last_change': '2024-04-25T14:09:24.803577+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.803577+0000', 'last_undegraded': '2024-04-25T14:09:24.803577+0000', 'last_fullsized': '2024-04-25T14:09:24.803577+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:09:30.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2 is not active+clean 2024-04-25T14:09:30.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.802115+0000', 'last_change': '2024-04-25T14:09:24.802115+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.802115+0000', 'last_undegraded': '2024-04-25T14:09:24.802115+0000', 'last_fullsized': '2024-04-25T14:09:24.802115+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:09:30.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3 is not active+clean 2024-04-25T14:09:30.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.796164+0000', 'last_change': '2024-04-25T14:09:24.796164+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.796164+0000', 'last_undegraded': '2024-04-25T14:09:24.796164+0000', 'last_fullsized': '2024-04-25T14:09:24.796164+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:09:30.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.4 is not active+clean 2024-04-25T14:09:30.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.809107+0000', 'last_change': '2024-04-25T14:09:24.809107+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.809107+0000', 'last_undegraded': '2024-04-25T14:09:24.809107+0000', 'last_fullsized': '2024-04-25T14:09:24.809107+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:09:30.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.5 is not active+clean 2024-04-25T14:09:30.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.801810+0000', 'last_change': '2024-04-25T14:09:24.801810+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.801810+0000', 'last_undegraded': '2024-04-25T14:09:24.801810+0000', 'last_fullsized': '2024-04-25T14:09:24.801810+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:09:30.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.6 is not active+clean 2024-04-25T14:09:30.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.801284+0000', 'last_change': '2024-04-25T14:09:24.801284+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.801284+0000', 'last_undegraded': '2024-04-25T14:09:24.801284+0000', 'last_fullsized': '2024-04-25T14:09:24.801284+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:09:30.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.7 is not active+clean 2024-04-25T14:09:30.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.809114+0000', 'last_change': '2024-04-25T14:09:24.809114+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.809114+0000', 'last_undegraded': '2024-04-25T14:09:24.809114+0000', 'last_fullsized': '2024-04-25T14:09:24.809114+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:09:30.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.8 is not active+clean 2024-04-25T14:09:30.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.802420+0000', 'last_change': '2024-04-25T14:09:24.802420+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.802420+0000', 'last_undegraded': '2024-04-25T14:09:24.802420+0000', 'last_fullsized': '2024-04-25T14:09:24.802420+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:09:30.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.9 is not active+clean 2024-04-25T14:09:30.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.795235+0000', 'last_change': '2024-04-25T14:09:24.795235+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.795235+0000', 'last_undegraded': '2024-04-25T14:09:24.795235+0000', 'last_fullsized': '2024-04-25T14:09:24.795235+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:09:30.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.a is not active+clean 2024-04-25T14:09:30.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.799314+0000', 'last_change': '2024-04-25T14:09:24.799314+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.799314+0000', 'last_undegraded': '2024-04-25T14:09:24.799314+0000', 'last_fullsized': '2024-04-25T14:09:24.799314+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:09:30.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.b is not active+clean 2024-04-25T14:09:30.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.795828+0000', 'last_change': '2024-04-25T14:09:24.795828+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.795828+0000', 'last_undegraded': '2024-04-25T14:09:24.795828+0000', 'last_fullsized': '2024-04-25T14:09:24.795828+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:09:30.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.c is not active+clean 2024-04-25T14:09:30.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.795817+0000', 'last_change': '2024-04-25T14:09:24.795817+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.795817+0000', 'last_undegraded': '2024-04-25T14:09:24.795817+0000', 'last_fullsized': '2024-04-25T14:09:24.795817+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:09:30.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.d is not active+clean 2024-04-25T14:09:30.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.804276+0000', 'last_change': '2024-04-25T14:09:24.804276+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.804276+0000', 'last_undegraded': '2024-04-25T14:09:24.804276+0000', 'last_fullsized': '2024-04-25T14:09:24.804276+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:09:30.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.e is not active+clean 2024-04-25T14:09:30.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.800603+0000', 'last_change': '2024-04-25T14:09:24.800603+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.800603+0000', 'last_undegraded': '2024-04-25T14:09:24.800603+0000', 'last_fullsized': '2024-04-25T14:09:24.800603+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:09:30.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.f is not active+clean 2024-04-25T14:09:30.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.803123+0000', 'last_change': '2024-04-25T14:09:24.803123+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.803123+0000', 'last_undegraded': '2024-04-25T14:09:24.803123+0000', 'last_fullsized': '2024-04-25T14:09:24.803123+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:09:30.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.10 is not active+clean 2024-04-25T14:09:30.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.804614+0000', 'last_change': '2024-04-25T14:09:24.804614+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.804614+0000', 'last_undegraded': '2024-04-25T14:09:24.804614+0000', 'last_fullsized': '2024-04-25T14:09:24.804614+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:09:30.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.12 is not active+clean 2024-04-25T14:09:30.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.800253+0000', 'last_change': '2024-04-25T14:09:24.800253+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.800253+0000', 'last_undegraded': '2024-04-25T14:09:24.800253+0000', 'last_fullsized': '2024-04-25T14:09:24.800253+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:09:30.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.13 is not active+clean 2024-04-25T14:09:30.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.796305+0000', 'last_change': '2024-04-25T14:09:24.796305+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.796305+0000', 'last_undegraded': '2024-04-25T14:09:24.796305+0000', 'last_fullsized': '2024-04-25T14:09:24.796305+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:09:30.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.14 is not active+clean 2024-04-25T14:09:30.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.802445+0000', 'last_change': '2024-04-25T14:09:24.802445+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.802445+0000', 'last_undegraded': '2024-04-25T14:09:24.802445+0000', 'last_fullsized': '2024-04-25T14:09:24.802445+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:09:30.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.15 is not active+clean 2024-04-25T14:09:30.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.804489+0000', 'last_change': '2024-04-25T14:09:24.804489+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.804489+0000', 'last_undegraded': '2024-04-25T14:09:24.804489+0000', 'last_fullsized': '2024-04-25T14:09:24.804489+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:09:30.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.16 is not active+clean 2024-04-25T14:09:30.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.815279+0000', 'last_change': '2024-04-25T14:09:24.815279+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.815279+0000', 'last_undegraded': '2024-04-25T14:09:24.815279+0000', 'last_fullsized': '2024-04-25T14:09:24.815279+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:09:30.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.17 is not active+clean 2024-04-25T14:09:30.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.801719+0000', 'last_change': '2024-04-25T14:09:24.801719+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.801719+0000', 'last_undegraded': '2024-04-25T14:09:24.801719+0000', 'last_fullsized': '2024-04-25T14:09:24.801719+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:09:30.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.18 is not active+clean 2024-04-25T14:09:30.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.803987+0000', 'last_change': '2024-04-25T14:09:24.803987+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.803987+0000', 'last_undegraded': '2024-04-25T14:09:24.803987+0000', 'last_fullsized': '2024-04-25T14:09:24.803987+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:09:30.752 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.19 is not active+clean 2024-04-25T14:09:30.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.801652+0000', 'last_change': '2024-04-25T14:09:24.801652+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.801652+0000', 'last_undegraded': '2024-04-25T14:09:24.801652+0000', 'last_fullsized': '2024-04-25T14:09:24.801652+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:09:30.752 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1a is not active+clean 2024-04-25T14:09:30.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.814203+0000', 'last_change': '2024-04-25T14:09:24.814203+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.814203+0000', 'last_undegraded': '2024-04-25T14:09:24.814203+0000', 'last_fullsized': '2024-04-25T14:09:24.814203+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:09:30.752 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1b is not active+clean 2024-04-25T14:09:30.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.796715+0000', 'last_change': '2024-04-25T14:09:24.796715+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.796715+0000', 'last_undegraded': '2024-04-25T14:09:24.796715+0000', 'last_fullsized': '2024-04-25T14:09:24.796715+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:09:30.752 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1c is not active+clean 2024-04-25T14:09:30.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.802382+0000', 'last_change': '2024-04-25T14:09:24.802382+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.802382+0000', 'last_undegraded': '2024-04-25T14:09:24.802382+0000', 'last_fullsized': '2024-04-25T14:09:24.802382+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:09:30.752 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1d is not active+clean 2024-04-25T14:09:30.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.801234+0000', 'last_change': '2024-04-25T14:09:24.801234+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.801234+0000', 'last_undegraded': '2024-04-25T14:09:24.801234+0000', 'last_fullsized': '2024-04-25T14:09:24.801234+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:09:30.752 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1e is not active+clean 2024-04-25T14:09:30.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.800017+0000', 'last_change': '2024-04-25T14:09:24.800017+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.800017+0000', 'last_undegraded': '2024-04-25T14:09:24.800017+0000', 'last_fullsized': '2024-04-25T14:09:24.800017+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:09:30.753 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1f is not active+clean 2024-04-25T14:09:30.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.795988+0000', 'last_change': '2024-04-25T14:09:24.795988+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.795988+0000', 'last_undegraded': '2024-04-25T14:09:24.795988+0000', 'last_fullsized': '2024-04-25T14:09:24.795988+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:09:30.753 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.11 is not active+clean 2024-04-25T14:09:30.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:09:24.795863+0000', 'last_change': '2024-04-25T14:09:24.795863+0000', 'last_active': '2024-04-25T14:09:24.788787+0000', 'last_peered': '2024-04-25T14:09:24.788787+0000', 'last_clean': '2024-04-25T14:09:24.788787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:09:24.795863+0000', 'last_undegraded': '2024-04-25T14:09:24.795863+0000', 'last_fullsized': '2024-04-25T14:09:24.795863+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'last_clean_scrub_stamp': '2024-04-25T14:09:24.788787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:09:30.753 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T14:09:30.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.930+0000 7f31f7600640 1 Processor -- start 2024-04-25T14:09:30.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.930+0000 7f31f7600640 1 -- start start 2024-04-25T14:09:30.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.934+0000 7f31f7600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f31f00cfca0 0x7f31f006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:30.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.934+0000 7f31f7600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31f00730b0 0x7f31f00734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:30.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.934+0000 7f31f7600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31f0072bd0 con 0x7f31f00d16e0 2024-04-25T14:09:30.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.934+0000 7f31f7600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f31f0070470 con 0x7f31f00cfca0 2024-04-25T14:09:30.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.934+0000 7f31f7600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f31f006f930 con 0x7f31f00730b0 2024-04-25T14:09:30.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.934+0000 7f31f5600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31f00730b0 0x7f31f00734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:30.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.934+0000 7f31f4c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f31f00cfca0 0x7f31f006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:30.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.934+0000 7f31e7e00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f31f00d16e0 0x7f31f00d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:39314/0 (socket says 172.21.15.86:39314) 2024-04-25T14:09:30.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.934+0000 7f31e7e00640 1 -- 172.21.15.86:0/4170758983 learned_addr learned my addr 172.21.15.86:0/4170758983 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:30.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.934+0000 7f31f4c00640 1 -- 172.21.15.86:0/4170758983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31f00730b0 msgr2=0x7f31f00734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:30.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.934+0000 7f31f4c00640 1 --2- 172.21.15.86:0/4170758983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31f00730b0 0x7f31f00734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:30.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.934+0000 7f31f4c00640 1 -- 172.21.15.86:0/4170758983 >> v1:172.21.15.155:6789/0 conn(0x7f31f00d16e0 legacy=0x7f31f00d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:30.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.934+0000 7f31f4c00640 1 -- 172.21.15.86:0/4170758983 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f006fbd0 con 0x7f31f00cfca0 2024-04-25T14:09:30.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.938+0000 7f31f4c00640 1 --2- 172.21.15.86:0/4170758983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31f00cfca0 0x7f31f006f3f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f31e00097c0 tx=0x7f31e001e6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db2fc327ccaf52e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:30.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.938+0000 7f31efe00640 1 -- 172.21.15.86:0/4170758983 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e00b2600 con 0x7f31f00cfca0 2024-04-25T14:09:30.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.938+0000 7f31efe00640 1 -- 172.21.15.86:0/4170758983 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31e00b27a0 con 0x7f31f00cfca0 2024-04-25T14:09:30.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.938+0000 7f31efe00640 1 -- 172.21.15.86:0/4170758983 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e00b2b70 con 0x7f31f00cfca0 2024-04-25T14:09:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.938+0000 7f31f7600640 1 -- 172.21.15.86:0/4170758983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31f00cfca0 msgr2=0x7f31f006f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.938+0000 7f31f7600640 1 --2- 172.21.15.86:0/4170758983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31f00cfca0 0x7f31f006f3f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f31e00097c0 tx=0x7f31e001e6b0 comp rx=0 tx=0).stop 2024-04-25T14:09:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.938+0000 7f31f7600640 1 -- 172.21.15.86:0/4170758983 shutdown_connections 2024-04-25T14:09:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.938+0000 7f31f7600640 1 --2- 172.21.15.86:0/4170758983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31f00730b0 0x7f31f00734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.938+0000 7f31f7600640 1 --2- 172.21.15.86:0/4170758983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31f00cfca0 0x7f31f006f3f0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.938+0000 7f31f7600640 1 -- 172.21.15.86:0/4170758983 >> 172.21.15.86:0/4170758983 conn(0x7f31f00b4650 msgr2=0x7f31f00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.938+0000 7f31f7600640 1 -- 172.21.15.86:0/4170758983 shutdown_connections 2024-04-25T14:09:30.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.938+0000 7f31f7600640 1 -- 172.21.15.86:0/4170758983 wait complete. 2024-04-25T14:09:30.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.938+0000 7f31f7600640 1 Processor -- start 2024-04-25T14:09:30.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.938+0000 7f31f7600640 1 -- start start 2024-04-25T14:09:30.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.942+0000 7f31f7600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31f00730b0 0x7f31f0158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:30.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.942+0000 7f31f7600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31f00cfca0 0x7f31f015f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:30.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.942+0000 7f31f7600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31f00d16e0 0x7f31f015fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:30.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.942+0000 7f31f7600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f31f00ce0f0 con 0x7f31f00cfca0 2024-04-25T14:09:30.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.942+0000 7f31f7600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f31f00cdf70 con 0x7f31f00d16e0 2024-04-25T14:09:30.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.942+0000 7f31f7600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f31f00ce270 con 0x7f31f00730b0 2024-04-25T14:09:30.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.942+0000 7f31e7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31f00cfca0 0x7f31f015f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:30.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.942+0000 7f31e7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31f00cfca0 0x7f31f015f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58088/0 (socket says 172.21.15.86:58088) 2024-04-25T14:09:30.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.942+0000 7f31e7e00640 1 -- 172.21.15.86:0/1843199949 learned_addr learned my addr 172.21.15.86:0/1843199949 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:30.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.942+0000 7f31f5600640 1 --2- 172.21.15.86:0/1843199949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31f00d16e0 0x7f31f015fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:30.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.942+0000 7f31f4c00640 1 --2- 172.21.15.86:0/1843199949 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31f00730b0 0x7f31f0158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:30.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.942+0000 7f31e7e00640 1 -- 172.21.15.86:0/1843199949 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31f00730b0 msgr2=0x7f31f0158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:30.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.942+0000 7f31e7e00640 1 --2- 172.21.15.86:0/1843199949 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31f00730b0 0x7f31f0158e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:30.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.942+0000 7f31e7e00640 1 -- 172.21.15.86:0/1843199949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31f00d16e0 msgr2=0x7f31f015fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:30.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.942+0000 7f31e7e00640 1 --2- 172.21.15.86:0/1843199949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31f00d16e0 0x7f31f015fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:30.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.942+0000 7f31e7e00640 1 -- 172.21.15.86:0/1843199949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f015f7e0 con 0x7f31f00cfca0 2024-04-25T14:09:30.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.942+0000 7f31e7e00640 1 --2- 172.21.15.86:0/1843199949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31f00cfca0 0x7f31f015f2a0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f31d800c340 tx=0x7f31d800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:30.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.942+0000 7f31ed600640 1 -- 172.21.15.86:0/1843199949 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d8017070 con 0x7f31f00cfca0 2024-04-25T14:09:30.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.942+0000 7f31f7600640 1 -- 172.21.15.86:0/1843199949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31f0160310 con 0x7f31f00cfca0 2024-04-25T14:09:30.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.942+0000 7f31ed600640 1 -- 172.21.15.86:0/1843199949 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31d8005d10 con 0x7f31f00cfca0 2024-04-25T14:09:30.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.942+0000 7f31ed600640 1 -- 172.21.15.86:0/1843199949 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d8007310 con 0x7f31f00cfca0 2024-04-25T14:09:30.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.942+0000 7f31f7600640 1 -- 172.21.15.86:0/1843199949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31f015e4e0 con 0x7f31f00cfca0 2024-04-25T14:09:30.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.942+0000 7f31f7600640 1 -- 172.21.15.86:0/1843199949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31f009f4b0 con 0x7f31f00cfca0 2024-04-25T14:09:30.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.946+0000 7f31ed600640 1 -- 172.21.15.86:0/1843199949 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7f31d801a070 con 0x7f31f00cfca0 2024-04-25T14:09:30.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.946+0000 7f31ed600640 1 --2- 172.21.15.86:0/1843199949 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f31cc06f520 0x7f31cc071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:30.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.946+0000 7f31ed600640 1 -- 172.21.15.86:0/1843199949 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7f31d808e8b0 con 0x7f31f00cfca0 2024-04-25T14:09:30.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.946+0000 7f31f4c00640 1 --2- 172.21.15.86:0/1843199949 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f31cc06f520 0x7f31cc071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:30.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.946+0000 7f31f4c00640 1 --2- 172.21.15.86:0/1843199949 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f31cc06f520 0x7f31cc071a30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f31e00b1000 tx=0x7f31e00a9000 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:30.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:30.950+0000 7f31ed600640 1 -- 172.21.15.86:0/1843199949 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f31d805ae30 con 0x7f31f00cfca0 2024-04-25T14:09:31.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.162+0000 7f31f7600640 1 -- 172.21.15.86:0/1843199949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f31f0069d00 con 0x7f31f00cfca0 2024-04-25T14:09:31.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.162+0000 7f31ed600640 1 -- 172.21.15.86:0/1843199949 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1035 (secure 0 0 0) 0x7f31d805f130 con 0x7f31f00cfca0 2024-04-25T14:09:31.161 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:31.161 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":365,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":100,"num_osds":8,"num_up_osds":8,"osd_up_since":1714053818,"num_in_osds":8,"osd_in_since":1714053808,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":266317824,"bytes_avail":772827795456,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":76,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"b","status":"up:active","gid":7932}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-04-25T14:08:15.095679+0000","services":{}},"progress_events":{}} 2024-04-25T14:09:31.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.166+0000 7f31f7600640 1 -- 172.21.15.86:0/1843199949 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f31cc06f520 msgr2=0x7f31cc071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:31.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.166+0000 7f31f7600640 1 --2- 172.21.15.86:0/1843199949 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f31cc06f520 0x7f31cc071a30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f31e00b1000 tx=0x7f31e00a9000 comp rx=0 tx=0).stop 2024-04-25T14:09:31.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.166+0000 7f31f7600640 1 -- 172.21.15.86:0/1843199949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31f00cfca0 msgr2=0x7f31f015f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:31.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.166+0000 7f31f7600640 1 --2- 172.21.15.86:0/1843199949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31f00cfca0 0x7f31f015f2a0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f31d800c340 tx=0x7f31d800c800 comp rx=0 tx=0).stop 2024-04-25T14:09:31.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.166+0000 7f31f7600640 1 -- 172.21.15.86:0/1843199949 shutdown_connections 2024-04-25T14:09:31.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.166+0000 7f31f7600640 1 --2- 172.21.15.86:0/1843199949 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f31cc06f520 0x7f31cc071a30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:31.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.166+0000 7f31f7600640 1 --2- 172.21.15.86:0/1843199949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31f00d16e0 0x7f31f015fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:31.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.166+0000 7f31f7600640 1 --2- 172.21.15.86:0/1843199949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31f00cfca0 0x7f31f015f2a0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:31.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.166+0000 7f31f7600640 1 --2- 172.21.15.86:0/1843199949 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31f00730b0 0x7f31f0158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:31.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.166+0000 7f31f7600640 1 -- 172.21.15.86:0/1843199949 >> 172.21.15.86:0/1843199949 conn(0x7f31f00b4650 msgr2=0x7f31f00b22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:31.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.166+0000 7f31f7600640 1 -- 172.21.15.86:0/1843199949 shutdown_connections 2024-04-25T14:09:31.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.166+0000 7f31f7600640 1 -- 172.21.15.86:0/1843199949 wait complete. 2024-04-25T14:09:31.183 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T14:09:31.183 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:09:31.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.330+0000 7f8ca2e00640 1 Processor -- start 2024-04-25T14:09:31.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.330+0000 7f8ca2e00640 1 -- start start 2024-04-25T14:09:31.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.334+0000 7f8ca2e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8c9c070330 0x7f8c9c0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:31.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.334+0000 7f8ca2e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c9c0ce790 0x7f8c9c06f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:31.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.334+0000 7f8ca2e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c9c0d2030 con 0x7f8c9c0d1c20 2024-04-25T14:09:31.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.334+0000 7f8ca2e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8c9c15e5c0 con 0x7f8c9c070330 2024-04-25T14:09:31.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.334+0000 7f8ca2e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8c9c15e8a0 con 0x7f8c9c0ce790 2024-04-25T14:09:31.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.334+0000 7f8c9be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8c9c070330 0x7f8c9c0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:31.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.334+0000 7f8c9be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8c9c070330 0x7f8c9c0d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58096/0 (socket says 172.21.15.86:58096) 2024-04-25T14:09:31.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.334+0000 7f8c9be00640 1 -- 172.21.15.86:0/1968001747 learned_addr learned my addr 172.21.15.86:0/1968001747 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:31.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.334+0000 7f8ca0e00640 1 --2- 172.21.15.86:0/1968001747 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c9c0ce790 0x7f8c9c06f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:31.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.334+0000 7f8c9be00640 1 -- 172.21.15.86:0/1968001747 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c9c0ce790 msgr2=0x7f8c9c06f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:31.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.334+0000 7f8c9be00640 1 --2- 172.21.15.86:0/1968001747 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c9c0ce790 0x7f8c9c06f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:31.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.334+0000 7f8c9be00640 1 -- 172.21.15.86:0/1968001747 >> v1:172.21.15.155:6789/0 conn(0x7f8c9c0d1c20 legacy=0x7f8c9c06ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:31.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.334+0000 7f8c9be00640 1 -- 172.21.15.86:0/1968001747 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c9c15eb80 con 0x7f8c9c070330 2024-04-25T14:09:31.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.338+0000 7f8c9be00640 1 --2- 172.21.15.86:0/1968001747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c9c070330 0x7f8c9c0d16e0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009870 tx=0x7f8c8c01c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d1a69334daef4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:31.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.338+0000 7f8c9aa00640 1 -- 172.21.15.86:0/1968001747 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c0b0070 con 0x7f8c9c070330 2024-04-25T14:09:31.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.338+0000 7f8c9aa00640 1 -- 172.21.15.86:0/1968001747 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c8c0ab600 con 0x7f8c9c070330 2024-04-25T14:09:31.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.338+0000 7f8c9aa00640 1 -- 172.21.15.86:0/1968001747 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c0ab9d0 con 0x7f8c9c070330 2024-04-25T14:09:31.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.338+0000 7f8ca2e00640 1 -- 172.21.15.86:0/1968001747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c9c070330 msgr2=0x7f8c9c0d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:31.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.338+0000 7f8ca2e00640 1 --2- 172.21.15.86:0/1968001747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c9c070330 0x7f8c9c0d16e0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009870 tx=0x7f8c8c01c2b0 comp rx=0 tx=0).stop 2024-04-25T14:09:31.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.338+0000 7f8ca2e00640 1 -- 172.21.15.86:0/1968001747 shutdown_connections 2024-04-25T14:09:31.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.338+0000 7f8ca2e00640 1 --2- 172.21.15.86:0/1968001747 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c9c0ce790 0x7f8c9c06f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:31.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.338+0000 7f8ca2e00640 1 --2- 172.21.15.86:0/1968001747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c9c070330 0x7f8c9c0d16e0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:31.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.338+0000 7f8ca2e00640 1 -- 172.21.15.86:0/1968001747 >> 172.21.15.86:0/1968001747 conn(0x7f8c9c0b4650 msgr2=0x7f8c9c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:31.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.338+0000 7f8ca2e00640 1 -- 172.21.15.86:0/1968001747 shutdown_connections 2024-04-25T14:09:31.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.338+0000 7f8ca2e00640 1 -- 172.21.15.86:0/1968001747 wait complete. 2024-04-25T14:09:31.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.338+0000 7f8ca2e00640 1 Processor -- start 2024-04-25T14:09:31.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.342+0000 7f8ca2e00640 1 -- start start 2024-04-25T14:09:31.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.342+0000 7f8ca2e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c9c070330 0x7f8c9c1f9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:31.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.342+0000 7f8ca2e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c9c0ce790 0x7f8c9c0c2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:31.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.342+0000 7f8ca2e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c9c0d1c20 0x7f8c9c073ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:31.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.342+0000 7f8ca2e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9c0708c0 con 0x7f8c9c070330 2024-04-25T14:09:31.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.342+0000 7f8ca2e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9c070740 con 0x7f8c9c0d1c20 2024-04-25T14:09:31.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.342+0000 7f8ca2e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8c9c16afc0 con 0x7f8c9c0ce790 2024-04-25T14:09:31.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.342+0000 7f8c9be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c9c070330 0x7f8c9c1f9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:31.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.342+0000 7f8c9be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c9c070330 0x7f8c9c1f9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58098/0 (socket says 172.21.15.86:58098) 2024-04-25T14:09:31.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.342+0000 7f8c9be00640 1 -- 172.21.15.86:0/1789038586 learned_addr learned my addr 172.21.15.86:0/1789038586 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:31.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.342+0000 7f8ca0e00640 1 --2- 172.21.15.86:0/1789038586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c9c0d1c20 0x7f8c9c073ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:31.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.342+0000 7f8c9b400640 1 --2- 172.21.15.86:0/1789038586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c9c0ce790 0x7f8c9c0c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:31.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.342+0000 7f8c9be00640 1 -- 172.21.15.86:0/1789038586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c9c0ce790 msgr2=0x7f8c9c0c2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:31.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.342+0000 7f8c9be00640 1 --2- 172.21.15.86:0/1789038586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c9c0ce790 0x7f8c9c0c2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:31.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.342+0000 7f8c9be00640 1 -- 172.21.15.86:0/1789038586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c9c0d1c20 msgr2=0x7f8c9c073ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:31.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.342+0000 7f8c9be00640 1 --2- 172.21.15.86:0/1789038586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c9c0d1c20 0x7f8c9c073ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:31.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.342+0000 7f8c9be00640 1 -- 172.21.15.86:0/1789038586 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c9c0c2d90 con 0x7f8c9c070330 2024-04-25T14:09:31.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.342+0000 7f8c9b400640 1 --2- 172.21.15.86:0/1789038586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c9c0ce790 0x7f8c9c0c2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:31.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.342+0000 7f8c9be00640 1 --2- 172.21.15.86:0/1789038586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c9c070330 0x7f8c9c1f9690 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c0ae040 tx=0x7f8c8c01aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:31.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.342+0000 7f8c93e00640 1 -- 172.21.15.86:0/1789038586 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c0ba020 con 0x7f8c9c070330 2024-04-25T14:09:31.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.342+0000 7f8c93e00640 1 -- 172.21.15.86:0/1789038586 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c8c0b6620 con 0x7f8c9c070330 2024-04-25T14:09:31.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.342+0000 7f8c93e00640 1 -- 172.21.15.86:0/1789038586 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c0b0040 con 0x7f8c9c070330 2024-04-25T14:09:31.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.342+0000 7f8ca2e00640 1 -- 172.21.15.86:0/1789038586 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c9c0c3040 con 0x7f8c9c070330 2024-04-25T14:09:31.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.342+0000 7f8ca2e00640 1 -- 172.21.15.86:0/1789038586 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c9c0c3600 con 0x7f8c9c070330 2024-04-25T14:09:31.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.342+0000 7f8ca2e00640 1 -- 172.21.15.86:0/1789038586 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c9c069d00 con 0x7f8c9c070330 2024-04-25T14:09:31.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.346+0000 7f8c93e00640 1 -- 172.21.15.86:0/1789038586 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7f8c8c0bd020 con 0x7f8c9c070330 2024-04-25T14:09:31.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.346+0000 7f8c93e00640 1 --2- 172.21.15.86:0/1789038586 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f8c7c06f520 0x7f8c7c071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:31.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.346+0000 7f8c93e00640 1 -- 172.21.15.86:0/1789038586 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7f8c8c129800 con 0x7f8c9c070330 2024-04-25T14:09:31.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.346+0000 7f8c9b400640 1 --2- 172.21.15.86:0/1789038586 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f8c7c06f520 0x7f8c7c071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:31.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.346+0000 7f8c9b400640 1 --2- 172.21.15.86:0/1789038586 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f8c7c06f520 0x7f8c7c071a30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c15bcc0 tx=0x7f8c88008040 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:31.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.350+0000 7f8c93e00640 1 -- 172.21.15.86:0/1789038586 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8c8c0c4030 con 0x7f8c9c070330 2024-04-25T14:09:31.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.502+0000 7f8ca2e00640 1 -- 172.21.15.86:0/1789038586 --> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8c9c0c5900 con 0x7f8c7c06f520 2024-04-25T14:09:31.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.510+0000 7f8c93e00640 1 -- 172.21.15.86:0/1789038586 <== mgr.7650 v2:172.21.15.86:6832/4236874185 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246766 (secure 0 0 0) 0x7f8c9c0c5900 con 0x7f8c7c06f520 2024-04-25T14:09:31.509 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:31.510 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:09:31.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.514+0000 7f8ca2e00640 1 -- 172.21.15.86:0/1789038586 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f8c7c06f520 msgr2=0x7f8c7c071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:31.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.514+0000 7f8ca2e00640 1 --2- 172.21.15.86:0/1789038586 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f8c7c06f520 0x7f8c7c071a30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c15bcc0 tx=0x7f8c88008040 comp rx=0 tx=0).stop 2024-04-25T14:09:31.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.514+0000 7f8ca2e00640 1 -- 172.21.15.86:0/1789038586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c9c070330 msgr2=0x7f8c9c1f9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:31.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.514+0000 7f8ca2e00640 1 --2- 172.21.15.86:0/1789038586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c9c070330 0x7f8c9c1f9690 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c0ae040 tx=0x7f8c8c01aaf0 comp rx=0 tx=0).stop 2024-04-25T14:09:31.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.514+0000 7f8ca2e00640 1 -- 172.21.15.86:0/1789038586 shutdown_connections 2024-04-25T14:09:31.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.514+0000 7f8ca2e00640 1 --2- 172.21.15.86:0/1789038586 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f8c7c06f520 0x7f8c7c071a30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:31.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.514+0000 7f8ca2e00640 1 --2- 172.21.15.86:0/1789038586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c9c0d1c20 0x7f8c9c073ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:31.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.514+0000 7f8ca2e00640 1 --2- 172.21.15.86:0/1789038586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c9c0ce790 0x7f8c9c0c2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:31.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.514+0000 7f8ca2e00640 1 --2- 172.21.15.86:0/1789038586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c9c070330 0x7f8c9c1f9690 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:31.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.514+0000 7f8ca2e00640 1 -- 172.21.15.86:0/1789038586 >> 172.21.15.86:0/1789038586 conn(0x7f8c9c0b4650 msgr2=0x7f8c9c15a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:31.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.514+0000 7f8ca2e00640 1 -- 172.21.15.86:0/1789038586 shutdown_connections 2024-04-25T14:09:31.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:31.514+0000 7f8ca2e00640 1 -- 172.21.15.86:0/1789038586 wait complete. 2024-04-25T14:09:31.530 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T14:09:29.506130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":366,"num_read_kb":935,"num_write":161,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260076,"kb_used_data":10592,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714644,"statfs":{"total":773094113280,"available":772827795456,"internally_reserved":0,"allocated":10846208,"data_stored":5978918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.716486"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.811745+0000","last_change":"2024-04-25T14:09:24.811745+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.811745+0000","last_undegraded":"2024-04-25T14:09:24.811745+0000","last_fullsized":"2024-04-25T14:09:24.811745+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.805434+0000","last_change":"2024-04-25T14:09:24.805434+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.805434+0000","last_undegraded":"2024-04-25T14:09:24.805434+0000","last_fullsized":"2024-04-25T14:09:24.805434+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.806609+0000","last_change":"2024-04-25T14:09:24.806609+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.806609+0000","last_undegraded":"2024-04-25T14:09:24.806609+0000","last_fullsized":"2024-04-25T14:09:24.806609+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.814243+0000","last_change":"2024-04-25T14:09:24.814243+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.814243+0000","last_undegraded":"2024-04-25T14:09:24.814243+0000","last_fullsized":"2024-04-25T14:09:24.814243+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.796780+0000","last_change":"2024-04-25T14:09:24.796780+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.796780+0000","last_undegraded":"2024-04-25T14:09:24.796780+0000","last_fullsized":"2024-04-25T14:09:24.796780+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801913+0000","last_change":"2024-04-25T14:09:24.801913+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801913+0000","last_undegraded":"2024-04-25T14:09:24.801913+0000","last_fullsized":"2024-04-25T14:09:24.801913+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.805381+0000","last_change":"2024-04-25T14:09:24.805381+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.805381+0000","last_undegraded":"2024-04-25T14:09:24.805381+0000","last_fullsized":"2024-04-25T14:09:24.805381+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.805573+0000","last_change":"2024-04-25T14:09:24.805573+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.805573+0000","last_undegraded":"2024-04-25T14:09:24.805573+0000","last_fullsized":"2024-04-25T14:09:24.805573+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.802656+0000","last_change":"2024-04-25T14:09:24.802656+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.802656+0000","last_undegraded":"2024-04-25T14:09:24.802656+0000","last_fullsized":"2024-04-25T14:09:24.802656+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.796198+0000","last_change":"2024-04-25T14:09:24.796198+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.796198+0000","last_undegraded":"2024-04-25T14:09:24.796198+0000","last_fullsized":"2024-04-25T14:09:24.796198+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.797705+0000","last_change":"2024-04-25T14:09:24.797705+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.797705+0000","last_undegraded":"2024-04-25T14:09:24.797705+0000","last_fullsized":"2024-04-25T14:09:24.797705+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.803323+0000","last_change":"2024-04-25T14:09:24.803323+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.803323+0000","last_undegraded":"2024-04-25T14:09:24.803323+0000","last_fullsized":"2024-04-25T14:09:24.803323+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.814711+0000","last_change":"2024-04-25T14:09:24.814711+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.814711+0000","last_undegraded":"2024-04-25T14:09:24.814711+0000","last_fullsized":"2024-04-25T14:09:24.814711+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801188+0000","last_change":"2024-04-25T14:09:24.801188+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801188+0000","last_undegraded":"2024-04-25T14:09:24.801188+0000","last_fullsized":"2024-04-25T14:09:24.801188+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.800597+0000","last_change":"2024-04-25T14:09:24.800597+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.800597+0000","last_undegraded":"2024-04-25T14:09:24.800597+0000","last_fullsized":"2024-04-25T14:09:24.800597+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.799763+0000","last_change":"2024-04-25T14:09:24.799763+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.799763+0000","last_undegraded":"2024-04-25T14:09:24.799763+0000","last_fullsized":"2024-04-25T14:09:24.799763+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.806722+0000","last_change":"2024-04-25T14:09:24.806722+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.806722+0000","last_undegraded":"2024-04-25T14:09:24.806722+0000","last_fullsized":"2024-04-25T14:09:24.806722+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801800+0000","last_change":"2024-04-25T14:09:24.801800+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801800+0000","last_undegraded":"2024-04-25T14:09:24.801800+0000","last_fullsized":"2024-04-25T14:09:24.801800+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.805192+0000","last_change":"2024-04-25T14:09:24.805192+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.805192+0000","last_undegraded":"2024-04-25T14:09:24.805192+0000","last_fullsized":"2024-04-25T14:09:24.805192+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.802814+0000","last_change":"2024-04-25T14:09:24.802814+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.802814+0000","last_undegraded":"2024-04-25T14:09:24.802814+0000","last_fullsized":"2024-04-25T14:09:24.802814+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.800665+0000","last_change":"2024-04-25T14:09:24.800665+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.800665+0000","last_undegraded":"2024-04-25T14:09:24.800665+0000","last_fullsized":"2024-04-25T14:09:24.800665+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.797246+0000","last_change":"2024-04-25T14:09:24.797246+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.797246+0000","last_undegraded":"2024-04-25T14:09:24.797246+0000","last_fullsized":"2024-04-25T14:09:24.797246+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.804279+0000","last_change":"2024-04-25T14:09:24.804279+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.804279+0000","last_undegraded":"2024-04-25T14:09:24.804279+0000","last_fullsized":"2024-04-25T14:09:24.804279+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.795033+0000","last_change":"2024-04-25T14:09:24.795033+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.795033+0000","last_undegraded":"2024-04-25T14:09:24.795033+0000","last_fullsized":"2024-04-25T14:09:24.795033+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.803941+0000","last_change":"2024-04-25T14:09:24.803941+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.803941+0000","last_undegraded":"2024-04-25T14:09:24.803941+0000","last_fullsized":"2024-04-25T14:09:24.803941+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.803227+0000","last_change":"2024-04-25T14:09:24.803227+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.803227+0000","last_undegraded":"2024-04-25T14:09:24.803227+0000","last_fullsized":"2024-04-25T14:09:24.803227+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.797093+0000","last_change":"2024-04-25T14:09:24.797093+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.797093+0000","last_undegraded":"2024-04-25T14:09:24.797093+0000","last_fullsized":"2024-04-25T14:09:24.797093+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.808887+0000","last_change":"2024-04-25T14:09:24.808887+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.808887+0000","last_undegraded":"2024-04-25T14:09:24.808887+0000","last_fullsized":"2024-04-25T14:09:24.808887+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.800116+0000","last_change":"2024-04-25T14:09:24.800116+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.800116+0000","last_undegraded":"2024-04-25T14:09:24.800116+0000","last_fullsized":"2024-04-25T14:09:24.800116+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.805810+0000","last_change":"2024-04-25T14:09:24.805810+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.805810+0000","last_undegraded":"2024-04-25T14:09:24.805810+0000","last_fullsized":"2024-04-25T14:09:24.805810+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.804724+0000","last_change":"2024-04-25T14:09:24.804724+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.804724+0000","last_undegraded":"2024-04-25T14:09:24.804724+0000","last_fullsized":"2024-04-25T14:09:24.804724+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.805096+0000","last_change":"2024-04-25T14:09:24.805096+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.805096+0000","last_undegraded":"2024-04-25T14:09:24.805096+0000","last_fullsized":"2024-04-25T14:09:24.805096+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801802+0000","last_change":"2024-04-25T14:09:24.801802+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801802+0000","last_undegraded":"2024-04-25T14:09:24.801802+0000","last_fullsized":"2024-04-25T14:09:24.801802+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.823829+0000","last_change":"2024-04-25T14:09:24.817466+0000","last_active":"2024-04-25T14:09:24.823829+0000","last_peered":"2024-04-25T14:09:24.823829+0000","last_clean":"2024-04-25T14:09:24.823829+0000","last_became_active":"2024-04-25T14:09:24.817167+0000","last_became_peered":"2024-04-25T14:09:24.817167+0000","last_unstale":"2024-04-25T14:09:24.823829+0000","last_undegraded":"2024-04-25T14:09:24.823829+0000","last_fullsized":"2024-04-25T14:09:24.823829+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:02.374557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.803577+0000","last_change":"2024-04-25T14:09:24.803577+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.803577+0000","last_undegraded":"2024-04-25T14:09:24.803577+0000","last_fullsized":"2024-04-25T14:09:24.803577+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.824309+0000","last_change":"2024-04-25T14:09:24.819324+0000","last_active":"2024-04-25T14:09:24.824309+0000","last_peered":"2024-04-25T14:09:24.824309+0000","last_clean":"2024-04-25T14:09:24.824309+0000","last_became_active":"2024-04-25T14:09:24.818377+0000","last_became_peered":"2024-04-25T14:09:24.818377+0000","last_unstale":"2024-04-25T14:09:24.824309+0000","last_undegraded":"2024-04-25T14:09:24.824309+0000","last_fullsized":"2024-04-25T14:09:24.824309+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:38.935678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.802115+0000","last_change":"2024-04-25T14:09:24.802115+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.802115+0000","last_undegraded":"2024-04-25T14:09:24.802115+0000","last_fullsized":"2024-04-25T14:09:24.802115+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821745+0000","last_change":"2024-04-25T14:09:24.813915+0000","last_active":"2024-04-25T14:09:24.821745+0000","last_peered":"2024-04-25T14:09:24.821745+0000","last_clean":"2024-04-25T14:09:24.821745+0000","last_became_active":"2024-04-25T14:09:24.813610+0000","last_became_peered":"2024-04-25T14:09:24.813610+0000","last_unstale":"2024-04-25T14:09:24.821745+0000","last_undegraded":"2024-04-25T14:09:24.821745+0000","last_fullsized":"2024-04-25T14:09:24.821745+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:03.242163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.796164+0000","last_change":"2024-04-25T14:09:24.796164+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.796164+0000","last_undegraded":"2024-04-25T14:09:24.796164+0000","last_fullsized":"2024-04-25T14:09:24.796164+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.813538+0000","last_change":"2024-04-25T14:09:24.810780+0000","last_active":"2024-04-25T14:09:24.813538+0000","last_peered":"2024-04-25T14:09:24.813538+0000","last_clean":"2024-04-25T14:09:24.813538+0000","last_became_active":"2024-04-25T14:09:24.810462+0000","last_became_peered":"2024-04-25T14:09:24.810462+0000","last_unstale":"2024-04-25T14:09:24.813538+0000","last_undegraded":"2024-04-25T14:09:24.813538+0000","last_fullsized":"2024-04-25T14:09:24.813538+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:30.379959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.809107+0000","last_change":"2024-04-25T14:09:24.809107+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.809107+0000","last_undegraded":"2024-04-25T14:09:24.809107+0000","last_fullsized":"2024-04-25T14:09:24.809107+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.820528+0000","last_change":"2024-04-25T14:09:24.813955+0000","last_active":"2024-04-25T14:09:24.820528+0000","last_peered":"2024-04-25T14:09:24.820528+0000","last_clean":"2024-04-25T14:09:24.820528+0000","last_became_active":"2024-04-25T14:09:24.813645+0000","last_became_peered":"2024-04-25T14:09:24.813645+0000","last_unstale":"2024-04-25T14:09:24.820528+0000","last_undegraded":"2024-04-25T14:09:24.820528+0000","last_fullsized":"2024-04-25T14:09:24.820528+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:02.374557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801810+0000","last_change":"2024-04-25T14:09:24.801810+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801810+0000","last_undegraded":"2024-04-25T14:09:24.801810+0000","last_fullsized":"2024-04-25T14:09:24.801810+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821172+0000","last_change":"2024-04-25T14:09:24.814236+0000","last_active":"2024-04-25T14:09:24.821172+0000","last_peered":"2024-04-25T14:09:24.821172+0000","last_clean":"2024-04-25T14:09:24.821172+0000","last_became_active":"2024-04-25T14:09:24.813909+0000","last_became_peered":"2024-04-25T14:09:24.813909+0000","last_unstale":"2024-04-25T14:09:24.821172+0000","last_undegraded":"2024-04-25T14:09:24.821172+0000","last_fullsized":"2024-04-25T14:09:24.821172+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:57.671866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801284+0000","last_change":"2024-04-25T14:09:24.801284+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801284+0000","last_undegraded":"2024-04-25T14:09:24.801284+0000","last_fullsized":"2024-04-25T14:09:24.801284+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821282+0000","last_change":"2024-04-25T14:09:24.811288+0000","last_active":"2024-04-25T14:09:24.821282+0000","last_peered":"2024-04-25T14:09:24.821282+0000","last_clean":"2024-04-25T14:09:24.821282+0000","last_became_active":"2024-04-25T14:09:24.809633+0000","last_became_peered":"2024-04-25T14:09:24.809633+0000","last_unstale":"2024-04-25T14:09:24.821282+0000","last_undegraded":"2024-04-25T14:09:24.821282+0000","last_fullsized":"2024-04-25T14:09:24.821282+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:57.671866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.809114+0000","last_change":"2024-04-25T14:09:24.809114+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.809114+0000","last_undegraded":"2024-04-25T14:09:24.809114+0000","last_fullsized":"2024-04-25T14:09:24.809114+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.806864+0000","last_change":"2024-04-25T14:09:24.799767+0000","last_active":"2024-04-25T14:09:24.806864+0000","last_peered":"2024-04-25T14:09:24.806864+0000","last_clean":"2024-04-25T14:09:24.806864+0000","last_became_active":"2024-04-25T14:09:24.799440+0000","last_became_peered":"2024-04-25T14:09:24.799440+0000","last_unstale":"2024-04-25T14:09:24.806864+0000","last_undegraded":"2024-04-25T14:09:24.806864+0000","last_fullsized":"2024-04-25T14:09:24.806864+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:44.194963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.802420+0000","last_change":"2024-04-25T14:09:24.802420+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.802420+0000","last_undegraded":"2024-04-25T14:09:24.802420+0000","last_fullsized":"2024-04-25T14:09:24.802420+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821771+0000","last_change":"2024-04-25T14:09:24.813861+0000","last_active":"2024-04-25T14:09:24.821771+0000","last_peered":"2024-04-25T14:09:24.821771+0000","last_clean":"2024-04-25T14:09:24.821771+0000","last_became_active":"2024-04-25T14:09:24.813523+0000","last_became_peered":"2024-04-25T14:09:24.813523+0000","last_unstale":"2024-04-25T14:09:24.821771+0000","last_undegraded":"2024-04-25T14:09:24.821771+0000","last_fullsized":"2024-04-25T14:09:24.821771+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:03.406811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.795235+0000","last_change":"2024-04-25T14:09:24.795235+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.795235+0000","last_undegraded":"2024-04-25T14:09:24.795235+0000","last_fullsized":"2024-04-25T14:09:24.795235+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.823970+0000","last_change":"2024-04-25T14:09:24.819250+0000","last_active":"2024-04-25T14:09:24.823970+0000","last_peered":"2024-04-25T14:09:24.823970+0000","last_clean":"2024-04-25T14:09:24.823970+0000","last_became_active":"2024-04-25T14:09:24.818892+0000","last_became_peered":"2024-04-25T14:09:24.818892+0000","last_unstale":"2024-04-25T14:09:24.823970+0000","last_undegraded":"2024-04-25T14:09:24.823970+0000","last_fullsized":"2024-04-25T14:09:24.823970+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:12.433048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.799314+0000","last_change":"2024-04-25T14:09:24.799314+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.799314+0000","last_undegraded":"2024-04-25T14:09:24.799314+0000","last_fullsized":"2024-04-25T14:09:24.799314+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.812722+0000","last_change":"2024-04-25T14:09:24.808328+0000","last_active":"2024-04-25T14:09:24.812722+0000","last_peered":"2024-04-25T14:09:24.812722+0000","last_clean":"2024-04-25T14:09:24.812722+0000","last_became_active":"2024-04-25T14:09:24.807984+0000","last_became_peered":"2024-04-25T14:09:24.807984+0000","last_unstale":"2024-04-25T14:09:24.812722+0000","last_undegraded":"2024-04-25T14:09:24.812722+0000","last_fullsized":"2024-04-25T14:09:24.812722+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:50.180573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.795828+0000","last_change":"2024-04-25T14:09:24.795828+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.795828+0000","last_undegraded":"2024-04-25T14:09:24.795828+0000","last_fullsized":"2024-04-25T14:09:24.795828+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.819182+0000","last_change":"2024-04-25T14:09:24.817277+0000","last_active":"2024-04-25T14:09:24.819182+0000","last_peered":"2024-04-25T14:09:24.819182+0000","last_clean":"2024-04-25T14:09:24.819182+0000","last_became_active":"2024-04-25T14:09:24.816793+0000","last_became_peered":"2024-04-25T14:09:24.816793+0000","last_unstale":"2024-04-25T14:09:24.819182+0000","last_undegraded":"2024-04-25T14:09:24.819182+0000","last_fullsized":"2024-04-25T14:09:24.819182+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:47.152512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.795817+0000","last_change":"2024-04-25T14:09:24.795817+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.795817+0000","last_undegraded":"2024-04-25T14:09:24.795817+0000","last_fullsized":"2024-04-25T14:09:24.795817+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.811590+0000","last_change":"2024-04-25T14:09:24.808496+0000","last_active":"2024-04-25T14:09:24.811590+0000","last_peered":"2024-04-25T14:09:24.811590+0000","last_clean":"2024-04-25T14:09:24.811590+0000","last_became_active":"2024-04-25T14:09:24.807948+0000","last_became_peered":"2024-04-25T14:09:24.807948+0000","last_unstale":"2024-04-25T14:09:24.811590+0000","last_undegraded":"2024-04-25T14:09:24.811590+0000","last_fullsized":"2024-04-25T14:09:24.811590+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:57.671866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.804276+0000","last_change":"2024-04-25T14:09:24.804276+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.804276+0000","last_undegraded":"2024-04-25T14:09:24.804276+0000","last_fullsized":"2024-04-25T14:09:24.804276+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.815553+0000","last_change":"2024-04-25T14:09:24.810654+0000","last_active":"2024-04-25T14:09:24.815553+0000","last_peered":"2024-04-25T14:09:24.815553+0000","last_clean":"2024-04-25T14:09:24.815553+0000","last_became_active":"2024-04-25T14:09:24.810344+0000","last_became_peered":"2024-04-25T14:09:24.810344+0000","last_unstale":"2024-04-25T14:09:24.815553+0000","last_undegraded":"2024-04-25T14:09:24.815553+0000","last_fullsized":"2024-04-25T14:09:24.815553+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:50.180573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.800603+0000","last_change":"2024-04-25T14:09:24.800603+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.800603+0000","last_undegraded":"2024-04-25T14:09:24.800603+0000","last_fullsized":"2024-04-25T14:09:24.800603+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.823748+0000","last_change":"2024-04-25T14:09:24.817108+0000","last_active":"2024-04-25T14:09:24.823748+0000","last_peered":"2024-04-25T14:09:24.823748+0000","last_clean":"2024-04-25T14:09:24.823748+0000","last_became_active":"2024-04-25T14:09:24.816826+0000","last_became_peered":"2024-04-25T14:09:24.816826+0000","last_unstale":"2024-04-25T14:09:24.823748+0000","last_undegraded":"2024-04-25T14:09:24.823748+0000","last_fullsized":"2024-04-25T14:09:24.823748+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:36.790595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.803123+0000","last_change":"2024-04-25T14:09:24.803123+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.803123+0000","last_undegraded":"2024-04-25T14:09:24.803123+0000","last_fullsized":"2024-04-25T14:09:24.803123+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821222+0000","last_change":"2024-04-25T14:09:24.811340+0000","last_active":"2024-04-25T14:09:24.821222+0000","last_peered":"2024-04-25T14:09:24.821222+0000","last_clean":"2024-04-25T14:09:24.821222+0000","last_became_active":"2024-04-25T14:09:24.809685+0000","last_became_peered":"2024-04-25T14:09:24.809685+0000","last_unstale":"2024-04-25T14:09:24.821222+0000","last_undegraded":"2024-04-25T14:09:24.821222+0000","last_fullsized":"2024-04-25T14:09:24.821222+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:30.379959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.804614+0000","last_change":"2024-04-25T14:09:24.804614+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.804614+0000","last_undegraded":"2024-04-25T14:09:24.804614+0000","last_fullsized":"2024-04-25T14:09:24.804614+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.824699+0000","last_change":"2024-04-25T14:09:24.822761+0000","last_active":"2024-04-25T14:09:24.824699+0000","last_peered":"2024-04-25T14:09:24.824699+0000","last_clean":"2024-04-25T14:09:24.824699+0000","last_became_active":"2024-04-25T14:09:24.822428+0000","last_became_peered":"2024-04-25T14:09:24.822428+0000","last_unstale":"2024-04-25T14:09:24.824699+0000","last_undegraded":"2024-04-25T14:09:24.824699+0000","last_fullsized":"2024-04-25T14:09:24.824699+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:51.227996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.800253+0000","last_change":"2024-04-25T14:09:24.800253+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.800253+0000","last_undegraded":"2024-04-25T14:09:24.800253+0000","last_fullsized":"2024-04-25T14:09:24.800253+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821955+0000","last_change":"2024-04-25T14:09:24.817271+0000","last_active":"2024-04-25T14:09:24.821955+0000","last_peered":"2024-04-25T14:09:24.821955+0000","last_clean":"2024-04-25T14:09:24.821955+0000","last_became_active":"2024-04-25T14:09:24.816723+0000","last_became_peered":"2024-04-25T14:09:24.816723+0000","last_unstale":"2024-04-25T14:09:24.821955+0000","last_undegraded":"2024-04-25T14:09:24.821955+0000","last_fullsized":"2024-04-25T14:09:24.821955+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:29:29.089649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.796305+0000","last_change":"2024-04-25T14:09:24.796305+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.796305+0000","last_undegraded":"2024-04-25T14:09:24.796305+0000","last_fullsized":"2024-04-25T14:09:24.796305+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821149+0000","last_change":"2024-04-25T14:09:24.814155+0000","last_active":"2024-04-25T14:09:24.821149+0000","last_peered":"2024-04-25T14:09:24.821149+0000","last_clean":"2024-04-25T14:09:24.821149+0000","last_became_active":"2024-04-25T14:09:24.812762+0000","last_became_peered":"2024-04-25T14:09:24.812762+0000","last_unstale":"2024-04-25T14:09:24.821149+0000","last_undegraded":"2024-04-25T14:09:24.821149+0000","last_fullsized":"2024-04-25T14:09:24.821149+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:30.379959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.802445+0000","last_change":"2024-04-25T14:09:24.802445+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.802445+0000","last_undegraded":"2024-04-25T14:09:24.802445+0000","last_fullsized":"2024-04-25T14:09:24.802445+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.816847+0000","last_change":"2024-04-25T14:09:24.811205+0000","last_active":"2024-04-25T14:09:24.816847+0000","last_peered":"2024-04-25T14:09:24.816847+0000","last_clean":"2024-04-25T14:09:24.816847+0000","last_became_active":"2024-04-25T14:09:24.810270+0000","last_became_peered":"2024-04-25T14:09:24.810270+0000","last_unstale":"2024-04-25T14:09:24.816847+0000","last_undegraded":"2024-04-25T14:09:24.816847+0000","last_fullsized":"2024-04-25T14:09:24.816847+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:30.864192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.804489+0000","last_change":"2024-04-25T14:09:24.804489+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.804489+0000","last_undegraded":"2024-04-25T14:09:24.804489+0000","last_fullsized":"2024-04-25T14:09:24.804489+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.816656+0000","last_change":"2024-04-25T14:09:24.808594+0000","last_active":"2024-04-25T14:09:24.816656+0000","last_peered":"2024-04-25T14:09:24.816656+0000","last_clean":"2024-04-25T14:09:24.816656+0000","last_became_active":"2024-04-25T14:09:24.807904+0000","last_became_peered":"2024-04-25T14:09:24.807904+0000","last_unstale":"2024-04-25T14:09:24.816656+0000","last_undegraded":"2024-04-25T14:09:24.816656+0000","last_fullsized":"2024-04-25T14:09:24.816656+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:55.195611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.815279+0000","last_change":"2024-04-25T14:09:24.815279+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.815279+0000","last_undegraded":"2024-04-25T14:09:24.815279+0000","last_fullsized":"2024-04-25T14:09:24.815279+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.819227+0000","last_change":"2024-04-25T14:09:24.816525+0000","last_active":"2024-04-25T14:09:24.819227+0000","last_peered":"2024-04-25T14:09:24.819227+0000","last_clean":"2024-04-25T14:09:24.819227+0000","last_became_active":"2024-04-25T14:09:24.815546+0000","last_became_peered":"2024-04-25T14:09:24.815546+0000","last_unstale":"2024-04-25T14:09:24.819227+0000","last_undegraded":"2024-04-25T14:09:24.819227+0000","last_fullsized":"2024-04-25T14:09:24.819227+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:46.765820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801719+0000","last_change":"2024-04-25T14:09:24.801719+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801719+0000","last_undegraded":"2024-04-25T14:09:24.801719+0000","last_fullsized":"2024-04-25T14:09:24.801719+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.817516+0000","last_change":"2024-04-25T14:09:24.813608+0000","last_active":"2024-04-25T14:09:24.817516+0000","last_peered":"2024-04-25T14:09:24.817516+0000","last_clean":"2024-04-25T14:09:24.817516+0000","last_became_active":"2024-04-25T14:09:24.813300+0000","last_became_peered":"2024-04-25T14:09:24.813300+0000","last_unstale":"2024-04-25T14:09:24.817516+0000","last_undegraded":"2024-04-25T14:09:24.817516+0000","last_fullsized":"2024-04-25T14:09:24.817516+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:41.291113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.803987+0000","last_change":"2024-04-25T14:09:24.803987+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.803987+0000","last_undegraded":"2024-04-25T14:09:24.803987+0000","last_fullsized":"2024-04-25T14:09:24.803987+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.821085+0000","last_change":"2024-04-25T14:09:24.818883+0000","last_active":"2024-04-25T14:09:24.821085+0000","last_peered":"2024-04-25T14:09:24.821085+0000","last_clean":"2024-04-25T14:09:24.821085+0000","last_became_active":"2024-04-25T14:09:24.817518+0000","last_became_peered":"2024-04-25T14:09:24.817518+0000","last_unstale":"2024-04-25T14:09:24.821085+0000","last_undegraded":"2024-04-25T14:09:24.821085+0000","last_fullsized":"2024-04-25T14:09:24.821085+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:03.242163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801652+0000","last_change":"2024-04-25T14:09:24.801652+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801652+0000","last_undegraded":"2024-04-25T14:09:24.801652+0000","last_fullsized":"2024-04-25T14:09:24.801652+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.816749+0000","last_change":"2024-04-25T14:09:24.814702+0000","last_active":"2024-04-25T14:09:24.816749+0000","last_peered":"2024-04-25T14:09:24.816749+0000","last_clean":"2024-04-25T14:09:24.816749+0000","last_became_active":"2024-04-25T14:09:24.807948+0000","last_became_peered":"2024-04-25T14:09:24.807948+0000","last_unstale":"2024-04-25T14:09:24.816749+0000","last_undegraded":"2024-04-25T14:09:24.816749+0000","last_fullsized":"2024-04-25T14:09:24.816749+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:36.733366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.814203+0000","last_change":"2024-04-25T14:09:24.814203+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.814203+0000","last_undegraded":"2024-04-25T14:09:24.814203+0000","last_fullsized":"2024-04-25T14:09:24.814203+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.819629+0000","last_change":"2024-04-25T14:09:24.817779+0000","last_active":"2024-04-25T14:09:24.819629+0000","last_peered":"2024-04-25T14:09:24.819629+0000","last_clean":"2024-04-25T14:09:24.819629+0000","last_became_active":"2024-04-25T14:09:24.817493+0000","last_became_peered":"2024-04-25T14:09:24.817493+0000","last_unstale":"2024-04-25T14:09:24.819629+0000","last_undegraded":"2024-04-25T14:09:24.819629+0000","last_fullsized":"2024-04-25T14:09:24.819629+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:59.756189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.796715+0000","last_change":"2024-04-25T14:09:24.796715+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.796715+0000","last_undegraded":"2024-04-25T14:09:24.796715+0000","last_fullsized":"2024-04-25T14:09:24.796715+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.820996+0000","last_change":"2024-04-25T14:09:24.818436+0000","last_active":"2024-04-25T14:09:24.820996+0000","last_peered":"2024-04-25T14:09:24.820996+0000","last_clean":"2024-04-25T14:09:24.820996+0000","last_became_active":"2024-04-25T14:09:24.818157+0000","last_became_peered":"2024-04-25T14:09:24.818157+0000","last_unstale":"2024-04-25T14:09:24.820996+0000","last_undegraded":"2024-04-25T14:09:24.820996+0000","last_fullsized":"2024-04-25T14:09:24.820996+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:03.406811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.802382+0000","last_change":"2024-04-25T14:09:24.802382+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.802382+0000","last_undegraded":"2024-04-25T14:09:24.802382+0000","last_fullsized":"2024-04-25T14:09:24.802382+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.824340+0000","last_change":"2024-04-25T14:09:24.820985+0000","last_active":"2024-04-25T14:09:24.824340+0000","last_peered":"2024-04-25T14:09:24.824340+0000","last_clean":"2024-04-25T14:09:24.824340+0000","last_became_active":"2024-04-25T14:09:24.819953+0000","last_became_peered":"2024-04-25T14:09:24.819953+0000","last_unstale":"2024-04-25T14:09:24.824340+0000","last_undegraded":"2024-04-25T14:09:24.824340+0000","last_fullsized":"2024-04-25T14:09:24.824340+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:17.719124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.801234+0000","last_change":"2024-04-25T14:09:24.801234+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.801234+0000","last_undegraded":"2024-04-25T14:09:24.801234+0000","last_fullsized":"2024-04-25T14:09:24.801234+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.824307+0000","last_change":"2024-04-25T14:09:24.822304+0000","last_active":"2024-04-25T14:09:24.824307+0000","last_peered":"2024-04-25T14:09:24.824307+0000","last_clean":"2024-04-25T14:09:24.824307+0000","last_became_active":"2024-04-25T14:09:24.821946+0000","last_became_peered":"2024-04-25T14:09:24.821946+0000","last_unstale":"2024-04-25T14:09:24.824307+0000","last_undegraded":"2024-04-25T14:09:24.824307+0000","last_fullsized":"2024-04-25T14:09:24.824307+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:28.243010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.800017+0000","last_change":"2024-04-25T14:09:24.800017+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.800017+0000","last_undegraded":"2024-04-25T14:09:24.800017+0000","last_fullsized":"2024-04-25T14:09:24.800017+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.816511+0000","last_change":"2024-04-25T14:09:24.811162+0000","last_active":"2024-04-25T14:09:24.816511+0000","last_peered":"2024-04-25T14:09:24.816511+0000","last_clean":"2024-04-25T14:09:24.816511+0000","last_became_active":"2024-04-25T14:09:24.810194+0000","last_became_peered":"2024-04-25T14:09:24.810194+0000","last_unstale":"2024-04-25T14:09:24.816511+0000","last_undegraded":"2024-04-25T14:09:24.816511+0000","last_fullsized":"2024-04-25T14:09:24.816511+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:44.601365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.795988+0000","last_change":"2024-04-25T14:09:24.795988+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.795988+0000","last_undegraded":"2024-04-25T14:09:24.795988+0000","last_fullsized":"2024-04-25T14:09:24.795988+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.823676+0000","last_change":"2024-04-25T14:09:24.817347+0000","last_active":"2024-04-25T14:09:24.823676+0000","last_peered":"2024-04-25T14:09:24.823676+0000","last_clean":"2024-04-25T14:09:24.823676+0000","last_became_active":"2024-04-25T14:09:24.815968+0000","last_became_peered":"2024-04-25T14:09:24.815968+0000","last_unstale":"2024-04-25T14:09:24.823676+0000","last_undegraded":"2024-04-25T14:09:24.823676+0000","last_fullsized":"2024-04-25T14:09:24.823676+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:18.027462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-25T14:09:24.795863+0000","last_change":"2024-04-25T14:09:24.795863+0000","last_active":"2024-04-25T14:09:24.788787+0000","last_peered":"2024-04-25T14:09:24.788787+0000","last_clean":"2024-04-25T14:09:24.788787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:09:24.795863+0000","last_undegraded":"2024-04-25T14:09:24.795863+0000","last_fullsized":"2024-04-25T14:09:24.795863+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.819672+0000","last_change":"2024-04-25T14:09:24.816306+0000","last_active":"2024-04-25T14:09:24.819672+0000","last_peered":"2024-04-25T14:09:24.819672+0000","last_clean":"2024-04-25T14:09:24.819672+0000","last_became_active":"2024-04-25T14:09:24.816004+0000","last_became_peered":"2024-04-25T14:09:24.816004+0000","last_unstale":"2024-04-25T14:09:24.819672+0000","last_undegraded":"2024-04-25T14:09:24.819672+0000","last_fullsized":"2024-04-25T14:09:24.819672+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:40.598001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"89'107","reported_seq":377,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T14:09:24.800089+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:09:24.800089+0000","last_peered":"2024-04-25T14:09:24.800089+0000","last_clean":"2024-04-25T14:09:24.800089+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:09:24.800089+0000","last_undegraded":"2024-04-25T14:09:24.800089+0000","last_fullsized":"2024-04-25T14:09:24.800089+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":366,"num_read_kb":935,"num_write":161,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":366,"num_read_kb":935,"num_write":161,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574916,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32940,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338900,"statfs":{"total":96636764160,"available":96603033600,"internally_reserved":0,"allocated":1798144,"data_stored":1188474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":598106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1216512,"data_stored":604773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1216512,"data_stored":604773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574917,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":598106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574916,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1204224,"data_stored":598106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":598106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574917,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32948,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338892,"statfs":{"total":96636764160,"available":96603025408,"internally_reserved":0,"allocated":1798144,"data_stored":1188474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:09:34.534 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:09:34.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.698+0000 7fa157200640 1 Processor -- start 2024-04-25T14:09:34.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.698+0000 7fa157200640 1 -- start start 2024-04-25T14:09:34.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.698+0000 7fa157200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa15015c2a0 0x7fa1500ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.698+0000 7fa157200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa15006fd50 0x7fa150070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.698+0000 7fa157200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1500b0a60 con 0x7fa15006c670 2024-04-25T14:09:34.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.698+0000 7fa157200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa15006ef90 con 0x7fa15015c2a0 2024-04-25T14:09:34.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.698+0000 7fa157200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa15006f270 con 0x7fa15006fd50 2024-04-25T14:09:34.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.698+0000 7fa14fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa15015c2a0 0x7fa1500ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.698+0000 7fa14fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa15015c2a0 0x7fa1500ae130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58102/0 (socket says 172.21.15.86:58102) 2024-04-25T14:09:34.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.698+0000 7fa14fe00640 1 -- 172.21.15.86:0/3236122132 learned_addr learned my addr 172.21.15.86:0/3236122132 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:34.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.698+0000 7fa147400640 1 --2- 172.21.15.86:0/3236122132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa15006fd50 0x7fa150070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.698+0000 7fa14fe00640 1 -- 172.21.15.86:0/3236122132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa15006fd50 msgr2=0x7fa150070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:34.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.698+0000 7fa14fe00640 1 --2- 172.21.15.86:0/3236122132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa15006fd50 0x7fa150070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.698+0000 7fa14fe00640 1 -- 172.21.15.86:0/3236122132 >> v1:172.21.15.155:6789/0 conn(0x7fa15006c670 legacy=0x7fa15006ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:34.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.698+0000 7fa14fe00640 1 -- 172.21.15.86:0/3236122132 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15006f550 con 0x7fa15015c2a0 2024-04-25T14:09:34.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.702+0000 7fa14fe00640 1 --2- 172.21.15.86:0/3236122132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa15015c2a0 0x7fa1500ae130 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fa140009870 tx=0x7fa14001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed3100a2a38c647 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:34.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.702+0000 7fa14f400640 1 -- 172.21.15.86:0/3236122132 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1400ab660 con 0x7fa15015c2a0 2024-04-25T14:09:34.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.702+0000 7fa14f400640 1 -- 172.21.15.86:0/3236122132 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1400b0070 con 0x7fa15015c2a0 2024-04-25T14:09:34.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.702+0000 7fa14f400640 1 -- 172.21.15.86:0/3236122132 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1400aba20 con 0x7fa15015c2a0 2024-04-25T14:09:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.702+0000 7fa157200640 1 -- 172.21.15.86:0/3236122132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa15015c2a0 msgr2=0x7fa1500ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.702+0000 7fa157200640 1 --2- 172.21.15.86:0/3236122132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa15015c2a0 0x7fa1500ae130 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fa140009870 tx=0x7fa14001c4a0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.702+0000 7fa157200640 1 -- 172.21.15.86:0/3236122132 shutdown_connections 2024-04-25T14:09:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.702+0000 7fa157200640 1 --2- 172.21.15.86:0/3236122132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa15006fd50 0x7fa150070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.702+0000 7fa157200640 1 --2- 172.21.15.86:0/3236122132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa15015c2a0 0x7fa1500ae130 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.702+0000 7fa157200640 1 -- 172.21.15.86:0/3236122132 >> 172.21.15.86:0/3236122132 conn(0x7fa1500b4650 msgr2=0x7fa1500b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.702+0000 7fa157200640 1 -- 172.21.15.86:0/3236122132 shutdown_connections 2024-04-25T14:09:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.702+0000 7fa157200640 1 -- 172.21.15.86:0/3236122132 wait complete. 2024-04-25T14:09:34.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.706+0000 7fa157200640 1 Processor -- start 2024-04-25T14:09:34.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.706+0000 7fa157200640 1 -- start start 2024-04-25T14:09:34.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.706+0000 7fa157200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa15006c670 0x7fa15009a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.706+0000 7fa157200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa15006fd50 0x7fa1500b1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.706+0000 7fa157200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa15015c2a0 0x7fa1500b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.706+0000 7fa157200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa15015c830 con 0x7fa15015c2a0 2024-04-25T14:09:34.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.706+0000 7fa157200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa15015c6b0 con 0x7fa15006c670 2024-04-25T14:09:34.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.706+0000 7fa157200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa1500ae670 con 0x7fa15006fd50 2024-04-25T14:09:34.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.706+0000 7fa155200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa15015c2a0 0x7fa1500b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.706+0000 7fa155200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa15015c2a0 0x7fa1500b17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58106/0 (socket says 172.21.15.86:58106) 2024-04-25T14:09:34.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.706+0000 7fa155200640 1 -- 172.21.15.86:0/1934194088 learned_addr learned my addr 172.21.15.86:0/1934194088 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:34.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.706+0000 7fa14fe00640 1 --2- 172.21.15.86:0/1934194088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa15006c670 0x7fa15009a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.706+0000 7fa147400640 1 --2- 172.21.15.86:0/1934194088 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa15006fd50 0x7fa1500b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.706+0000 7fa155200640 1 -- 172.21.15.86:0/1934194088 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa15006fd50 msgr2=0x7fa1500b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:34.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.706+0000 7fa155200640 1 --2- 172.21.15.86:0/1934194088 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa15006fd50 0x7fa1500b1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.706+0000 7fa155200640 1 -- 172.21.15.86:0/1934194088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa15006c670 msgr2=0x7fa15009a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:34.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.706+0000 7fa155200640 1 --2- 172.21.15.86:0/1934194088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa15006c670 0x7fa15009a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.706+0000 7fa155200640 1 -- 172.21.15.86:0/1934194088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15014c5a0 con 0x7fa15015c2a0 2024-04-25T14:09:34.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.706+0000 7fa155200640 1 --2- 172.21.15.86:0/1934194088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa15015c2a0 0x7fa1500b17c0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fa14800c730 tx=0x7fa14800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:34.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.706+0000 7fa14cc00640 1 -- 172.21.15.86:0/1934194088 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa148003b80 con 0x7fa15015c2a0 2024-04-25T14:09:34.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.706+0000 7fa14cc00640 1 -- 172.21.15.86:0/1934194088 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa148003d20 con 0x7fa15015c2a0 2024-04-25T14:09:34.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.706+0000 7fa14cc00640 1 -- 172.21.15.86:0/1934194088 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14800aa30 con 0x7fa15015c2a0 2024-04-25T14:09:34.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.706+0000 7fa157200640 1 -- 172.21.15.86:0/1934194088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1500b3750 con 0x7fa15015c2a0 2024-04-25T14:09:34.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.710+0000 7fa157200640 1 -- 172.21.15.86:0/1934194088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1500740c0 con 0x7fa15015c2a0 2024-04-25T14:09:34.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.710+0000 7fa157200640 1 -- 172.21.15.86:0/1934194088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa150074320 con 0x7fa15015c2a0 2024-04-25T14:09:34.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.710+0000 7fa14cc00640 1 -- 172.21.15.86:0/1934194088 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7fa14800abd0 con 0x7fa15015c2a0 2024-04-25T14:09:34.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.710+0000 7fa14cc00640 1 --2- 172.21.15.86:0/1934194088 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fa12c06f550 0x7fa12c071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.710+0000 7fa14cc00640 1 -- 172.21.15.86:0/1934194088 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7fa14808f360 con 0x7fa15015c2a0 2024-04-25T14:09:34.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.710+0000 7fa14fe00640 1 --2- 172.21.15.86:0/1934194088 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fa12c06f550 0x7fa12c071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.714+0000 7fa14fe00640 1 --2- 172.21.15.86:0/1934194088 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fa12c06f550 0x7fa12c071a60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa1400af000 tx=0x7fa1400a7000 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:34.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.718+0000 7fa14cc00640 1 -- 172.21.15.86:0/1934194088 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa14805b950 con 0x7fa15015c2a0 2024-04-25T14:09:34.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.870+0000 7fa157200640 1 -- 172.21.15.86:0/1934194088 --> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa15014c7d0 con 0x7fa12c06f550 2024-04-25T14:09:34.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.878+0000 7fa14cc00640 1 -- 172.21.15.86:0/1934194088 <== mgr.7650 v2:172.21.15.86:6832/4236874185 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252182 (secure 0 0 0) 0x7fa15014c7d0 con 0x7fa12c06f550 2024-04-25T14:09:34.876 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:34.878 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:09:34.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.878+0000 7fa157200640 1 -- 172.21.15.86:0/1934194088 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fa12c06f550 msgr2=0x7fa12c071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:34.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.878+0000 7fa157200640 1 --2- 172.21.15.86:0/1934194088 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fa12c06f550 0x7fa12c071a60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa1400af000 tx=0x7fa1400a7000 comp rx=0 tx=0).stop 2024-04-25T14:09:34.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.878+0000 7fa157200640 1 -- 172.21.15.86:0/1934194088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa15015c2a0 msgr2=0x7fa1500b17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:34.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.878+0000 7fa157200640 1 --2- 172.21.15.86:0/1934194088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa15015c2a0 0x7fa1500b17c0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fa14800c730 tx=0x7fa14800cbf0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.882+0000 7fa157200640 1 -- 172.21.15.86:0/1934194088 shutdown_connections 2024-04-25T14:09:34.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.882+0000 7fa157200640 1 --2- 172.21.15.86:0/1934194088 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fa12c06f550 0x7fa12c071a60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.882+0000 7fa157200640 1 --2- 172.21.15.86:0/1934194088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa15015c2a0 0x7fa1500b17c0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.882+0000 7fa157200640 1 --2- 172.21.15.86:0/1934194088 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa15006fd50 0x7fa1500b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.882+0000 7fa157200640 1 --2- 172.21.15.86:0/1934194088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa15006c670 0x7fa15009a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.882+0000 7fa157200640 1 -- 172.21.15.86:0/1934194088 >> 172.21.15.86:0/1934194088 conn(0x7fa1500b4650 msgr2=0x7fa15015e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:34.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.882+0000 7fa157200640 1 -- 172.21.15.86:0/1934194088 shutdown_connections 2024-04-25T14:09:34.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:34.882+0000 7fa157200640 1 -- 172.21.15.86:0/1934194088 wait complete. 2024-04-25T14:09:34.897 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-04-25T14:09:33.511620+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":366,"num_read_kb":935,"num_write":206,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260396,"kb_used_data":10920,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714324,"statfs":{"total":773094113280,"available":772827467776,"internally_reserved":0,"allocated":11182080,"data_stored":6134612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.006646"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.844310+0000","last_change":"2024-04-25T14:09:25.843266+0000","last_active":"2024-04-25T14:09:26.844310+0000","last_peered":"2024-04-25T14:09:26.844310+0000","last_clean":"2024-04-25T14:09:26.844310+0000","last_became_active":"2024-04-25T14:09:25.842882+0000","last_became_peered":"2024-04-25T14:09:25.842882+0000","last_unstale":"2024-04-25T14:09:26.844310+0000","last_undegraded":"2024-04-25T14:09:26.844310+0000","last_fullsized":"2024-04-25T14:09:26.844310+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:16.716782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.220631+0000","last_change":"2024-04-25T14:09:25.834345+0000","last_active":"2024-04-25T14:09:27.220631+0000","last_peered":"2024-04-25T14:09:27.220631+0000","last_clean":"2024-04-25T14:09:27.220631+0000","last_became_active":"2024-04-25T14:09:25.834028+0000","last_became_peered":"2024-04-25T14:09:25.834028+0000","last_unstale":"2024-04-25T14:09:27.220631+0000","last_undegraded":"2024-04-25T14:09:27.220631+0000","last_fullsized":"2024-04-25T14:09:27.220631+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:41.427994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.843956+0000","last_change":"2024-04-25T14:09:25.840672+0000","last_active":"2024-04-25T14:09:26.843956+0000","last_peered":"2024-04-25T14:09:26.843956+0000","last_clean":"2024-04-25T14:09:26.843956+0000","last_became_active":"2024-04-25T14:09:25.840321+0000","last_became_peered":"2024-04-25T14:09:25.840321+0000","last_unstale":"2024-04-25T14:09:26.843956+0000","last_undegraded":"2024-04-25T14:09:26.843956+0000","last_fullsized":"2024-04-25T14:09:26.843956+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:38.517418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.837549+0000","last_change":"2024-04-25T14:09:25.833541+0000","last_active":"2024-04-25T14:09:26.837549+0000","last_peered":"2024-04-25T14:09:26.837549+0000","last_clean":"2024-04-25T14:09:26.837549+0000","last_became_active":"2024-04-25T14:09:25.833074+0000","last_became_peered":"2024-04-25T14:09:25.833074+0000","last_unstale":"2024-04-25T14:09:26.837549+0000","last_undegraded":"2024-04-25T14:09:26.837549+0000","last_fullsized":"2024-04-25T14:09:26.837549+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:59.777406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.837946+0000","last_change":"2024-04-25T14:09:25.844230+0000","last_active":"2024-04-25T14:09:26.837946+0000","last_peered":"2024-04-25T14:09:26.837946+0000","last_clean":"2024-04-25T14:09:26.837946+0000","last_became_active":"2024-04-25T14:09:25.843936+0000","last_became_peered":"2024-04-25T14:09:25.843936+0000","last_unstale":"2024-04-25T14:09:26.837946+0000","last_undegraded":"2024-04-25T14:09:26.837946+0000","last_fullsized":"2024-04-25T14:09:26.837946+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:34.178036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.220013+0000","last_change":"2024-04-25T14:09:25.846573+0000","last_active":"2024-04-25T14:09:27.220013+0000","last_peered":"2024-04-25T14:09:27.220013+0000","last_clean":"2024-04-25T14:09:27.220013+0000","last_became_active":"2024-04-25T14:09:25.845376+0000","last_became_peered":"2024-04-25T14:09:25.845376+0000","last_unstale":"2024-04-25T14:09:27.220013+0000","last_undegraded":"2024-04-25T14:09:27.220013+0000","last_fullsized":"2024-04-25T14:09:27.220013+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:23.185454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218617+0000","last_change":"2024-04-25T14:09:25.841805+0000","last_active":"2024-04-25T14:09:27.218617+0000","last_peered":"2024-04-25T14:09:27.218617+0000","last_clean":"2024-04-25T14:09:27.218617+0000","last_became_active":"2024-04-25T14:09:25.841485+0000","last_became_peered":"2024-04-25T14:09:25.841485+0000","last_unstale":"2024-04-25T14:09:27.218617+0000","last_undegraded":"2024-04-25T14:09:27.218617+0000","last_fullsized":"2024-04-25T14:09:27.218617+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:13.575081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218048+0000","last_change":"2024-04-25T14:09:25.825750+0000","last_active":"2024-04-25T14:09:27.218048+0000","last_peered":"2024-04-25T14:09:27.218048+0000","last_clean":"2024-04-25T14:09:27.218048+0000","last_became_active":"2024-04-25T14:09:25.825423+0000","last_became_peered":"2024-04-25T14:09:25.825423+0000","last_unstale":"2024-04-25T14:09:27.218048+0000","last_undegraded":"2024-04-25T14:09:27.218048+0000","last_fullsized":"2024-04-25T14:09:27.218048+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:28.792152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.220628+0000","last_change":"2024-04-25T14:09:25.835140+0000","last_active":"2024-04-25T14:09:27.220628+0000","last_peered":"2024-04-25T14:09:27.220628+0000","last_clean":"2024-04-25T14:09:27.220628+0000","last_became_active":"2024-04-25T14:09:25.834847+0000","last_became_peered":"2024-04-25T14:09:25.834847+0000","last_unstale":"2024-04-25T14:09:27.220628+0000","last_undegraded":"2024-04-25T14:09:27.220628+0000","last_fullsized":"2024-04-25T14:09:27.220628+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:44.842316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.839140+0000","last_change":"2024-04-25T14:09:25.840328+0000","last_active":"2024-04-25T14:09:26.839140+0000","last_peered":"2024-04-25T14:09:26.839140+0000","last_clean":"2024-04-25T14:09:26.839140+0000","last_became_active":"2024-04-25T14:09:25.839819+0000","last_became_peered":"2024-04-25T14:09:25.839819+0000","last_unstale":"2024-04-25T14:09:26.839140+0000","last_undegraded":"2024-04-25T14:09:26.839140+0000","last_fullsized":"2024-04-25T14:09:26.839140+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:41.427994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.838960+0000","last_change":"2024-04-25T14:09:25.834541+0000","last_active":"2024-04-25T14:09:26.838960+0000","last_peered":"2024-04-25T14:09:26.838960+0000","last_clean":"2024-04-25T14:09:26.838960+0000","last_became_active":"2024-04-25T14:09:25.834213+0000","last_became_peered":"2024-04-25T14:09:25.834213+0000","last_unstale":"2024-04-25T14:09:26.838960+0000","last_undegraded":"2024-04-25T14:09:26.838960+0000","last_fullsized":"2024-04-25T14:09:26.838960+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:34.178036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.219355+0000","last_change":"2024-04-25T14:09:25.834912+0000","last_active":"2024-04-25T14:09:27.219355+0000","last_peered":"2024-04-25T14:09:27.219355+0000","last_clean":"2024-04-25T14:09:27.219355+0000","last_became_active":"2024-04-25T14:09:25.833579+0000","last_became_peered":"2024-04-25T14:09:25.833579+0000","last_unstale":"2024-04-25T14:09:27.219355+0000","last_undegraded":"2024-04-25T14:09:27.219355+0000","last_fullsized":"2024-04-25T14:09:27.219355+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:44.842316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.837213+0000","last_change":"2024-04-25T14:09:25.835453+0000","last_active":"2024-04-25T14:09:26.837213+0000","last_peered":"2024-04-25T14:09:26.837213+0000","last_clean":"2024-04-25T14:09:26.837213+0000","last_became_active":"2024-04-25T14:09:25.835196+0000","last_became_peered":"2024-04-25T14:09:25.835196+0000","last_unstale":"2024-04-25T14:09:26.837213+0000","last_undegraded":"2024-04-25T14:09:26.837213+0000","last_fullsized":"2024-04-25T14:09:26.837213+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:44.842316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.219835+0000","last_change":"2024-04-25T14:09:25.837483+0000","last_active":"2024-04-25T14:09:27.219835+0000","last_peered":"2024-04-25T14:09:27.219835+0000","last_clean":"2024-04-25T14:09:27.219835+0000","last_became_active":"2024-04-25T14:09:25.837190+0000","last_became_peered":"2024-04-25T14:09:25.837190+0000","last_unstale":"2024-04-25T14:09:27.219835+0000","last_undegraded":"2024-04-25T14:09:27.219835+0000","last_fullsized":"2024-04-25T14:09:27.219835+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:41.427994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218643+0000","last_change":"2024-04-25T14:09:25.838370+0000","last_active":"2024-04-25T14:09:27.218643+0000","last_peered":"2024-04-25T14:09:27.218643+0000","last_clean":"2024-04-25T14:09:27.218643+0000","last_became_active":"2024-04-25T14:09:25.836444+0000","last_became_peered":"2024-04-25T14:09:25.836444+0000","last_unstale":"2024-04-25T14:09:27.218643+0000","last_undegraded":"2024-04-25T14:09:27.218643+0000","last_fullsized":"2024-04-25T14:09:27.218643+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:55.774042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218586+0000","last_change":"2024-04-25T14:09:25.834839+0000","last_active":"2024-04-25T14:09:27.218586+0000","last_peered":"2024-04-25T14:09:27.218586+0000","last_clean":"2024-04-25T14:09:27.218586+0000","last_became_active":"2024-04-25T14:09:25.834527+0000","last_became_peered":"2024-04-25T14:09:25.834527+0000","last_unstale":"2024-04-25T14:09:27.218586+0000","last_undegraded":"2024-04-25T14:09:27.218586+0000","last_fullsized":"2024-04-25T14:09:27.218586+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:29.813202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.844246+0000","last_change":"2024-04-25T14:09:25.840613+0000","last_active":"2024-04-25T14:09:26.844246+0000","last_peered":"2024-04-25T14:09:26.844246+0000","last_clean":"2024-04-25T14:09:26.844246+0000","last_became_active":"2024-04-25T14:09:25.840327+0000","last_became_peered":"2024-04-25T14:09:25.840327+0000","last_unstale":"2024-04-25T14:09:26.844246+0000","last_undegraded":"2024-04-25T14:09:26.844246+0000","last_fullsized":"2024-04-25T14:09:26.844246+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:38.965645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.220698+0000","last_change":"2024-04-25T14:09:25.842143+0000","last_active":"2024-04-25T14:09:27.220698+0000","last_peered":"2024-04-25T14:09:27.220698+0000","last_clean":"2024-04-25T14:09:27.220698+0000","last_became_active":"2024-04-25T14:09:25.841644+0000","last_became_peered":"2024-04-25T14:09:25.841644+0000","last_unstale":"2024-04-25T14:09:27.220698+0000","last_undegraded":"2024-04-25T14:09:27.220698+0000","last_fullsized":"2024-04-25T14:09:27.220698+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:59.777406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.844018+0000","last_change":"2024-04-25T14:09:25.846242+0000","last_active":"2024-04-25T14:09:26.844018+0000","last_peered":"2024-04-25T14:09:26.844018+0000","last_clean":"2024-04-25T14:09:26.844018+0000","last_became_active":"2024-04-25T14:09:25.845943+0000","last_became_peered":"2024-04-25T14:09:25.845943+0000","last_unstale":"2024-04-25T14:09:26.844018+0000","last_undegraded":"2024-04-25T14:09:26.844018+0000","last_fullsized":"2024-04-25T14:09:26.844018+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:14.306103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.219364+0000","last_change":"2024-04-25T14:09:25.836495+0000","last_active":"2024-04-25T14:09:27.219364+0000","last_peered":"2024-04-25T14:09:27.219364+0000","last_clean":"2024-04-25T14:09:27.219364+0000","last_became_active":"2024-04-25T14:09:25.836199+0000","last_became_peered":"2024-04-25T14:09:25.836199+0000","last_unstale":"2024-04-25T14:09:27.219364+0000","last_undegraded":"2024-04-25T14:09:27.219364+0000","last_fullsized":"2024-04-25T14:09:27.219364+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:41.427994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218217+0000","last_change":"2024-04-25T14:09:25.822789+0000","last_active":"2024-04-25T14:09:27.218217+0000","last_peered":"2024-04-25T14:09:27.218217+0000","last_clean":"2024-04-25T14:09:27.218217+0000","last_became_active":"2024-04-25T14:09:25.822057+0000","last_became_peered":"2024-04-25T14:09:25.822057+0000","last_unstale":"2024-04-25T14:09:27.218217+0000","last_undegraded":"2024-04-25T14:09:27.218217+0000","last_fullsized":"2024-04-25T14:09:27.218217+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:27.680675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.838765+0000","last_change":"2024-04-25T14:09:25.842344+0000","last_active":"2024-04-25T14:09:26.838765+0000","last_peered":"2024-04-25T14:09:26.838765+0000","last_clean":"2024-04-25T14:09:26.838765+0000","last_became_active":"2024-04-25T14:09:25.841984+0000","last_became_peered":"2024-04-25T14:09:25.841984+0000","last_unstale":"2024-04-25T14:09:26.838765+0000","last_undegraded":"2024-04-25T14:09:26.838765+0000","last_fullsized":"2024-04-25T14:09:26.838765+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:55.774042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.843785+0000","last_change":"2024-04-25T14:09:25.832941+0000","last_active":"2024-04-25T14:09:26.843785+0000","last_peered":"2024-04-25T14:09:26.843785+0000","last_clean":"2024-04-25T14:09:26.843785+0000","last_became_active":"2024-04-25T14:09:25.831754+0000","last_became_peered":"2024-04-25T14:09:25.831754+0000","last_unstale":"2024-04-25T14:09:26.843785+0000","last_undegraded":"2024-04-25T14:09:26.843785+0000","last_fullsized":"2024-04-25T14:09:26.843785+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:37.146242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.838841+0000","last_change":"2024-04-25T14:09:25.842650+0000","last_active":"2024-04-25T14:09:26.838841+0000","last_peered":"2024-04-25T14:09:26.838841+0000","last_clean":"2024-04-25T14:09:26.838841+0000","last_became_active":"2024-04-25T14:09:25.842228+0000","last_became_peered":"2024-04-25T14:09:25.842228+0000","last_unstale":"2024-04-25T14:09:26.838841+0000","last_undegraded":"2024-04-25T14:09:26.838841+0000","last_fullsized":"2024-04-25T14:09:26.838841+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:44.842316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.219770+0000","last_change":"2024-04-25T14:09:25.849028+0000","last_active":"2024-04-25T14:09:27.219770+0000","last_peered":"2024-04-25T14:09:27.219770+0000","last_clean":"2024-04-25T14:09:27.219770+0000","last_became_active":"2024-04-25T14:09:25.848342+0000","last_became_peered":"2024-04-25T14:09:25.848342+0000","last_unstale":"2024-04-25T14:09:27.219770+0000","last_undegraded":"2024-04-25T14:09:27.219770+0000","last_fullsized":"2024-04-25T14:09:27.219770+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:10.553064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.219724+0000","last_change":"2024-04-25T14:09:25.839960+0000","last_active":"2024-04-25T14:09:27.219724+0000","last_peered":"2024-04-25T14:09:27.219724+0000","last_clean":"2024-04-25T14:09:27.219724+0000","last_became_active":"2024-04-25T14:09:25.839616+0000","last_became_peered":"2024-04-25T14:09:25.839616+0000","last_unstale":"2024-04-25T14:09:27.219724+0000","last_undegraded":"2024-04-25T14:09:27.219724+0000","last_fullsized":"2024-04-25T14:09:27.219724+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:54.817541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.838977+0000","last_change":"2024-04-25T14:09:25.830310+0000","last_active":"2024-04-25T14:09:26.838977+0000","last_peered":"2024-04-25T14:09:26.838977+0000","last_clean":"2024-04-25T14:09:26.838977+0000","last_became_active":"2024-04-25T14:09:25.829870+0000","last_became_peered":"2024-04-25T14:09:25.829870+0000","last_unstale":"2024-04-25T14:09:26.838977+0000","last_undegraded":"2024-04-25T14:09:26.838977+0000","last_fullsized":"2024-04-25T14:09:26.838977+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:23.185454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.220145+0000","last_change":"2024-04-25T14:09:25.837526+0000","last_active":"2024-04-25T14:09:27.220145+0000","last_peered":"2024-04-25T14:09:27.220145+0000","last_clean":"2024-04-25T14:09:27.220145+0000","last_became_active":"2024-04-25T14:09:25.836424+0000","last_became_peered":"2024-04-25T14:09:25.836424+0000","last_unstale":"2024-04-25T14:09:27.220145+0000","last_undegraded":"2024-04-25T14:09:27.220145+0000","last_fullsized":"2024-04-25T14:09:27.220145+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:29.813202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218251+0000","last_change":"2024-04-25T14:09:25.832966+0000","last_active":"2024-04-25T14:09:27.218251+0000","last_peered":"2024-04-25T14:09:27.218251+0000","last_clean":"2024-04-25T14:09:27.218251+0000","last_became_active":"2024-04-25T14:09:25.832627+0000","last_became_peered":"2024-04-25T14:09:25.832627+0000","last_unstale":"2024-04-25T14:09:27.218251+0000","last_undegraded":"2024-04-25T14:09:27.218251+0000","last_fullsized":"2024-04-25T14:09:27.218251+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:36.403579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218690+0000","last_change":"2024-04-25T14:09:25.842931+0000","last_active":"2024-04-25T14:09:27.218690+0000","last_peered":"2024-04-25T14:09:27.218690+0000","last_clean":"2024-04-25T14:09:27.218690+0000","last_became_active":"2024-04-25T14:09:25.842575+0000","last_became_peered":"2024-04-25T14:09:25.842575+0000","last_unstale":"2024-04-25T14:09:27.218690+0000","last_undegraded":"2024-04-25T14:09:27.218690+0000","last_fullsized":"2024-04-25T14:09:27.218690+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:29.813202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218810+0000","last_change":"2024-04-25T14:09:25.847517+0000","last_active":"2024-04-25T14:09:27.218810+0000","last_peered":"2024-04-25T14:09:27.218810+0000","last_clean":"2024-04-25T14:09:27.218810+0000","last_became_active":"2024-04-25T14:09:25.845580+0000","last_became_peered":"2024-04-25T14:09:25.845580+0000","last_unstale":"2024-04-25T14:09:27.218810+0000","last_undegraded":"2024-04-25T14:09:27.218810+0000","last_fullsized":"2024-04-25T14:09:27.218810+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:34.178036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218146+0000","last_change":"2024-04-25T14:09:25.834428+0000","last_active":"2024-04-25T14:09:27.218146+0000","last_peered":"2024-04-25T14:09:27.218146+0000","last_clean":"2024-04-25T14:09:27.218146+0000","last_became_active":"2024-04-25T14:09:25.833728+0000","last_became_peered":"2024-04-25T14:09:25.833728+0000","last_unstale":"2024-04-25T14:09:27.218146+0000","last_undegraded":"2024-04-25T14:09:27.218146+0000","last_fullsized":"2024-04-25T14:09:27.218146+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:57.377375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.844196+0000","last_change":"2024-04-25T14:09:25.844409+0000","last_active":"2024-04-25T14:09:26.844196+0000","last_peered":"2024-04-25T14:09:26.844196+0000","last_clean":"2024-04-25T14:09:26.844196+0000","last_became_active":"2024-04-25T14:09:25.840896+0000","last_became_peered":"2024-04-25T14:09:25.840896+0000","last_unstale":"2024-04-25T14:09:26.844196+0000","last_undegraded":"2024-04-25T14:09:26.844196+0000","last_fullsized":"2024-04-25T14:09:26.844196+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:35.304378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"99'4","reported_seq":18,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.838169+0000","last_change":"2024-04-25T14:09:24.817466+0000","last_active":"2024-04-25T14:09:26.838169+0000","last_peered":"2024-04-25T14:09:26.838169+0000","last_clean":"2024-04-25T14:09:26.838169+0000","last_became_active":"2024-04-25T14:09:24.817167+0000","last_became_peered":"2024-04-25T14:09:24.817167+0000","last_unstale":"2024-04-25T14:09:26.838169+0000","last_undegraded":"2024-04-25T14:09:26.838169+0000","last_fullsized":"2024-04-25T14:09:26.838169+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:45.892361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218133+0000","last_change":"2024-04-25T14:09:25.838871+0000","last_active":"2024-04-25T14:09:27.218133+0000","last_peered":"2024-04-25T14:09:27.218133+0000","last_clean":"2024-04-25T14:09:27.218133+0000","last_became_active":"2024-04-25T14:09:25.838559+0000","last_became_peered":"2024-04-25T14:09:25.838559+0000","last_unstale":"2024-04-25T14:09:27.218133+0000","last_undegraded":"2024-04-25T14:09:27.218133+0000","last_fullsized":"2024-04-25T14:09:27.218133+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:20.423954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218490+0000","last_change":"2024-04-25T14:09:24.819324+0000","last_active":"2024-04-25T14:09:27.218490+0000","last_peered":"2024-04-25T14:09:27.218490+0000","last_clean":"2024-04-25T14:09:27.218490+0000","last_became_active":"2024-04-25T14:09:24.818377+0000","last_became_peered":"2024-04-25T14:09:24.818377+0000","last_unstale":"2024-04-25T14:09:27.218490+0000","last_undegraded":"2024-04-25T14:09:27.218490+0000","last_fullsized":"2024-04-25T14:09:27.218490+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:14.790627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218894+0000","last_change":"2024-04-25T14:09:25.845915+0000","last_active":"2024-04-25T14:09:27.218894+0000","last_peered":"2024-04-25T14:09:27.218894+0000","last_clean":"2024-04-25T14:09:27.218894+0000","last_became_active":"2024-04-25T14:09:25.845629+0000","last_became_peered":"2024-04-25T14:09:25.845629+0000","last_unstale":"2024-04-25T14:09:27.218894+0000","last_undegraded":"2024-04-25T14:09:27.218894+0000","last_fullsized":"2024-04-25T14:09:27.218894+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:55.774042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.219558+0000","last_change":"2024-04-25T14:09:24.813915+0000","last_active":"2024-04-25T14:09:27.219558+0000","last_peered":"2024-04-25T14:09:27.219558+0000","last_clean":"2024-04-25T14:09:27.219558+0000","last_became_active":"2024-04-25T14:09:24.813610+0000","last_became_peered":"2024-04-25T14:09:24.813610+0000","last_unstale":"2024-04-25T14:09:27.219558+0000","last_undegraded":"2024-04-25T14:09:27.219558+0000","last_fullsized":"2024-04-25T14:09:27.219558+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:02.374557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.838597+0000","last_change":"2024-04-25T14:09:25.845237+0000","last_active":"2024-04-25T14:09:26.838597+0000","last_peered":"2024-04-25T14:09:26.838597+0000","last_clean":"2024-04-25T14:09:26.838597+0000","last_became_active":"2024-04-25T14:09:25.844893+0000","last_became_peered":"2024-04-25T14:09:25.844893+0000","last_unstale":"2024-04-25T14:09:26.838597+0000","last_undegraded":"2024-04-25T14:09:26.838597+0000","last_fullsized":"2024-04-25T14:09:26.838597+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:14.156528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.838908+0000","last_change":"2024-04-25T14:09:24.810780+0000","last_active":"2024-04-25T14:09:26.838908+0000","last_peered":"2024-04-25T14:09:26.838908+0000","last_clean":"2024-04-25T14:09:26.838908+0000","last_became_active":"2024-04-25T14:09:24.810462+0000","last_became_peered":"2024-04-25T14:09:24.810462+0000","last_unstale":"2024-04-25T14:09:26.838908+0000","last_undegraded":"2024-04-25T14:09:26.838908+0000","last_fullsized":"2024-04-25T14:09:26.838908+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:44.194963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.837571+0000","last_change":"2024-04-25T14:09:25.848947+0000","last_active":"2024-04-25T14:09:26.837571+0000","last_peered":"2024-04-25T14:09:26.837571+0000","last_clean":"2024-04-25T14:09:26.837571+0000","last_became_active":"2024-04-25T14:09:25.847680+0000","last_became_peered":"2024-04-25T14:09:25.847680+0000","last_unstale":"2024-04-25T14:09:26.837571+0000","last_undegraded":"2024-04-25T14:09:26.837571+0000","last_fullsized":"2024-04-25T14:09:26.837571+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:48:20.597937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.219259+0000","last_change":"2024-04-25T14:09:24.813955+0000","last_active":"2024-04-25T14:09:27.219259+0000","last_peered":"2024-04-25T14:09:27.219259+0000","last_clean":"2024-04-25T14:09:27.219259+0000","last_became_active":"2024-04-25T14:09:24.813645+0000","last_became_peered":"2024-04-25T14:09:24.813645+0000","last_unstale":"2024-04-25T14:09:27.219259+0000","last_undegraded":"2024-04-25T14:09:27.219259+0000","last_fullsized":"2024-04-25T14:09:27.219259+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:45.892361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.844267+0000","last_change":"2024-04-25T14:09:25.843796+0000","last_active":"2024-04-25T14:09:26.844267+0000","last_peered":"2024-04-25T14:09:26.844267+0000","last_clean":"2024-04-25T14:09:26.844267+0000","last_became_active":"2024-04-25T14:09:25.843511+0000","last_became_peered":"2024-04-25T14:09:25.843511+0000","last_unstale":"2024-04-25T14:09:26.844267+0000","last_undegraded":"2024-04-25T14:09:26.844267+0000","last_fullsized":"2024-04-25T14:09:26.844267+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:45.592840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.838324+0000","last_change":"2024-04-25T14:09:24.814236+0000","last_active":"2024-04-25T14:09:26.838324+0000","last_peered":"2024-04-25T14:09:26.838324+0000","last_clean":"2024-04-25T14:09:26.838324+0000","last_became_active":"2024-04-25T14:09:24.813909+0000","last_became_peered":"2024-04-25T14:09:24.813909+0000","last_unstale":"2024-04-25T14:09:26.838324+0000","last_undegraded":"2024-04-25T14:09:26.838324+0000","last_fullsized":"2024-04-25T14:09:26.838324+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:03.406811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.220783+0000","last_change":"2024-04-25T14:09:25.836948+0000","last_active":"2024-04-25T14:09:27.220783+0000","last_peered":"2024-04-25T14:09:27.220783+0000","last_clean":"2024-04-25T14:09:27.220783+0000","last_became_active":"2024-04-25T14:09:25.836343+0000","last_became_peered":"2024-04-25T14:09:25.836343+0000","last_unstale":"2024-04-25T14:09:27.220783+0000","last_undegraded":"2024-04-25T14:09:27.220783+0000","last_fullsized":"2024-04-25T14:09:27.220783+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:54.817541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218955+0000","last_change":"2024-04-25T14:09:24.811288+0000","last_active":"2024-04-25T14:09:27.218955+0000","last_peered":"2024-04-25T14:09:27.218955+0000","last_clean":"2024-04-25T14:09:27.218955+0000","last_became_active":"2024-04-25T14:09:24.809633+0000","last_became_peered":"2024-04-25T14:09:24.809633+0000","last_unstale":"2024-04-25T14:09:27.218955+0000","last_undegraded":"2024-04-25T14:09:27.218955+0000","last_fullsized":"2024-04-25T14:09:27.218955+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:00.582546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.838062+0000","last_change":"2024-04-25T14:09:25.843764+0000","last_active":"2024-04-25T14:09:26.838062+0000","last_peered":"2024-04-25T14:09:26.838062+0000","last_clean":"2024-04-25T14:09:26.838062+0000","last_became_active":"2024-04-25T14:09:25.843524+0000","last_became_peered":"2024-04-25T14:09:25.843524+0000","last_unstale":"2024-04-25T14:09:26.838062+0000","last_undegraded":"2024-04-25T14:09:26.838062+0000","last_fullsized":"2024-04-25T14:09:26.838062+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:10.553064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.837079+0000","last_change":"2024-04-25T14:09:24.799767+0000","last_active":"2024-04-25T14:09:26.837079+0000","last_peered":"2024-04-25T14:09:26.837079+0000","last_clean":"2024-04-25T14:09:26.837079+0000","last_became_active":"2024-04-25T14:09:24.799440+0000","last_became_peered":"2024-04-25T14:09:24.799440+0000","last_unstale":"2024-04-25T14:09:26.837079+0000","last_undegraded":"2024-04-25T14:09:26.837079+0000","last_fullsized":"2024-04-25T14:09:26.837079+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:29:29.089649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.844024+0000","last_change":"2024-04-25T14:09:25.831573+0000","last_active":"2024-04-25T14:09:26.844024+0000","last_peered":"2024-04-25T14:09:26.844024+0000","last_clean":"2024-04-25T14:09:26.844024+0000","last_became_active":"2024-04-25T14:09:25.831260+0000","last_became_peered":"2024-04-25T14:09:25.831260+0000","last_unstale":"2024-04-25T14:09:26.844024+0000","last_undegraded":"2024-04-25T14:09:26.844024+0000","last_fullsized":"2024-04-25T14:09:26.844024+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:14.524379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.219776+0000","last_change":"2024-04-25T14:09:24.813861+0000","last_active":"2024-04-25T14:09:27.219776+0000","last_peered":"2024-04-25T14:09:27.219776+0000","last_clean":"2024-04-25T14:09:27.219776+0000","last_became_active":"2024-04-25T14:09:24.813523+0000","last_became_peered":"2024-04-25T14:09:24.813523+0000","last_unstale":"2024-04-25T14:09:27.219776+0000","last_undegraded":"2024-04-25T14:09:27.219776+0000","last_fullsized":"2024-04-25T14:09:27.219776+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:45.892361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.838082+0000","last_change":"2024-04-25T14:09:25.837769+0000","last_active":"2024-04-25T14:09:26.838082+0000","last_peered":"2024-04-25T14:09:26.838082+0000","last_clean":"2024-04-25T14:09:26.838082+0000","last_became_active":"2024-04-25T14:09:25.837467+0000","last_became_peered":"2024-04-25T14:09:25.837467+0000","last_unstale":"2024-04-25T14:09:26.838082+0000","last_undegraded":"2024-04-25T14:09:26.838082+0000","last_fullsized":"2024-04-25T14:09:26.838082+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:41.950449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.843843+0000","last_change":"2024-04-25T14:09:24.819250+0000","last_active":"2024-04-25T14:09:26.843843+0000","last_peered":"2024-04-25T14:09:26.843843+0000","last_clean":"2024-04-25T14:09:26.843843+0000","last_became_active":"2024-04-25T14:09:24.818892+0000","last_became_peered":"2024-04-25T14:09:24.818892+0000","last_unstale":"2024-04-25T14:09:26.843843+0000","last_undegraded":"2024-04-25T14:09:26.843843+0000","last_fullsized":"2024-04-25T14:09:26.843843+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:58.266420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218336+0000","last_change":"2024-04-25T14:09:25.831109+0000","last_active":"2024-04-25T14:09:27.218336+0000","last_peered":"2024-04-25T14:09:27.218336+0000","last_clean":"2024-04-25T14:09:27.218336+0000","last_became_active":"2024-04-25T14:09:25.830820+0000","last_became_peered":"2024-04-25T14:09:25.830820+0000","last_unstale":"2024-04-25T14:09:27.218336+0000","last_undegraded":"2024-04-25T14:09:27.218336+0000","last_fullsized":"2024-04-25T14:09:27.218336+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:34.764013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"99'3","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.219404+0000","last_change":"2024-04-25T14:09:24.808328+0000","last_active":"2024-04-25T14:09:27.219404+0000","last_peered":"2024-04-25T14:09:27.219404+0000","last_clean":"2024-04-25T14:09:27.219404+0000","last_became_active":"2024-04-25T14:09:24.807984+0000","last_became_peered":"2024-04-25T14:09:24.807984+0000","last_unstale":"2024-04-25T14:09:27.219404+0000","last_undegraded":"2024-04-25T14:09:27.219404+0000","last_fullsized":"2024-04-25T14:09:27.219404+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:11.900769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.838227+0000","last_change":"2024-04-25T14:09:25.841375+0000","last_active":"2024-04-25T14:09:26.838227+0000","last_peered":"2024-04-25T14:09:26.838227+0000","last_clean":"2024-04-25T14:09:26.838227+0000","last_became_active":"2024-04-25T14:09:25.839352+0000","last_became_peered":"2024-04-25T14:09:25.839352+0000","last_unstale":"2024-04-25T14:09:26.838227+0000","last_undegraded":"2024-04-25T14:09:26.838227+0000","last_fullsized":"2024-04-25T14:09:26.838227+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:29.813202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"99'6","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.220145+0000","last_change":"2024-04-25T14:09:24.817277+0000","last_active":"2024-04-25T14:09:27.220145+0000","last_peered":"2024-04-25T14:09:27.220145+0000","last_clean":"2024-04-25T14:09:27.220145+0000","last_became_active":"2024-04-25T14:09:24.816793+0000","last_became_peered":"2024-04-25T14:09:24.816793+0000","last_unstale":"2024-04-25T14:09:27.220145+0000","last_undegraded":"2024-04-25T14:09:27.220145+0000","last_fullsized":"2024-04-25T14:09:27.220145+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:30.379959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.838687+0000","last_change":"2024-04-25T14:09:25.843655+0000","last_active":"2024-04-25T14:09:26.838687+0000","last_peered":"2024-04-25T14:09:26.838687+0000","last_clean":"2024-04-25T14:09:26.838687+0000","last_became_active":"2024-04-25T14:09:25.843333+0000","last_became_peered":"2024-04-25T14:09:25.843333+0000","last_unstale":"2024-04-25T14:09:26.838687+0000","last_undegraded":"2024-04-25T14:09:26.838687+0000","last_fullsized":"2024-04-25T14:09:26.838687+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:13.575081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.838289+0000","last_change":"2024-04-25T14:09:24.808496+0000","last_active":"2024-04-25T14:09:26.838289+0000","last_peered":"2024-04-25T14:09:26.838289+0000","last_clean":"2024-04-25T14:09:26.838289+0000","last_became_active":"2024-04-25T14:09:24.807948+0000","last_became_peered":"2024-04-25T14:09:24.807948+0000","last_unstale":"2024-04-25T14:09:26.838289+0000","last_undegraded":"2024-04-25T14:09:26.838289+0000","last_fullsized":"2024-04-25T14:09:26.838289+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:17.719124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.220520+0000","last_change":"2024-04-25T14:09:25.833058+0000","last_active":"2024-04-25T14:09:27.220520+0000","last_peered":"2024-04-25T14:09:27.220520+0000","last_clean":"2024-04-25T14:09:27.220520+0000","last_became_active":"2024-04-25T14:09:25.832758+0000","last_became_peered":"2024-04-25T14:09:25.832758+0000","last_unstale":"2024-04-25T14:09:27.220520+0000","last_undegraded":"2024-04-25T14:09:27.220520+0000","last_fullsized":"2024-04-25T14:09:27.220520+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:55.774042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.837298+0000","last_change":"2024-04-25T14:09:24.810654+0000","last_active":"2024-04-25T14:09:26.837298+0000","last_peered":"2024-04-25T14:09:26.837298+0000","last_clean":"2024-04-25T14:09:26.837298+0000","last_became_active":"2024-04-25T14:09:24.810344+0000","last_became_peered":"2024-04-25T14:09:24.810344+0000","last_unstale":"2024-04-25T14:09:26.837298+0000","last_undegraded":"2024-04-25T14:09:26.837298+0000","last_fullsized":"2024-04-25T14:09:26.837298+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:11.900769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218406+0000","last_change":"2024-04-25T14:09:25.843051+0000","last_active":"2024-04-25T14:09:27.218406+0000","last_peered":"2024-04-25T14:09:27.218406+0000","last_clean":"2024-04-25T14:09:27.218406+0000","last_became_active":"2024-04-25T14:09:25.842775+0000","last_became_peered":"2024-04-25T14:09:25.842775+0000","last_unstale":"2024-04-25T14:09:27.218406+0000","last_undegraded":"2024-04-25T14:09:27.218406+0000","last_fullsized":"2024-04-25T14:09:27.218406+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:45.609319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218114+0000","last_change":"2024-04-25T14:09:24.817108+0000","last_active":"2024-04-25T14:09:27.218114+0000","last_peered":"2024-04-25T14:09:27.218114+0000","last_clean":"2024-04-25T14:09:27.218114+0000","last_became_active":"2024-04-25T14:09:24.816826+0000","last_became_peered":"2024-04-25T14:09:24.816826+0000","last_unstale":"2024-04-25T14:09:27.218114+0000","last_undegraded":"2024-04-25T14:09:27.218114+0000","last_fullsized":"2024-04-25T14:09:27.218114+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:28:29.351132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.844394+0000","last_change":"2024-04-25T14:09:25.839948+0000","last_active":"2024-04-25T14:09:26.844394+0000","last_peered":"2024-04-25T14:09:26.844394+0000","last_clean":"2024-04-25T14:09:26.844394+0000","last_became_active":"2024-04-25T14:09:25.839544+0000","last_became_peered":"2024-04-25T14:09:25.839544+0000","last_unstale":"2024-04-25T14:09:26.844394+0000","last_undegraded":"2024-04-25T14:09:26.844394+0000","last_fullsized":"2024-04-25T14:09:26.844394+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:29.806649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"99'8","reported_seq":22,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218585+0000","last_change":"2024-04-25T14:09:24.811340+0000","last_active":"2024-04-25T14:09:27.218585+0000","last_peered":"2024-04-25T14:09:27.218585+0000","last_clean":"2024-04-25T14:09:27.218585+0000","last_became_active":"2024-04-25T14:09:24.809685+0000","last_became_peered":"2024-04-25T14:09:24.809685+0000","last_unstale":"2024-04-25T14:09:27.218585+0000","last_undegraded":"2024-04-25T14:09:27.218585+0000","last_fullsized":"2024-04-25T14:09:27.218585+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:10.844004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218360+0000","last_change":"2024-04-25T14:09:25.834679+0000","last_active":"2024-04-25T14:09:27.218360+0000","last_peered":"2024-04-25T14:09:27.218360+0000","last_clean":"2024-04-25T14:09:27.218360+0000","last_became_active":"2024-04-25T14:09:25.834154+0000","last_became_peered":"2024-04-25T14:09:25.834154+0000","last_unstale":"2024-04-25T14:09:27.218360+0000","last_undegraded":"2024-04-25T14:09:27.218360+0000","last_fullsized":"2024-04-25T14:09:27.218360+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:53.913858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.220560+0000","last_change":"2024-04-25T14:09:24.822761+0000","last_active":"2024-04-25T14:09:27.220560+0000","last_peered":"2024-04-25T14:09:27.220560+0000","last_clean":"2024-04-25T14:09:27.220560+0000","last_became_active":"2024-04-25T14:09:24.822428+0000","last_became_peered":"2024-04-25T14:09:24.822428+0000","last_unstale":"2024-04-25T14:09:27.220560+0000","last_undegraded":"2024-04-25T14:09:27.220560+0000","last_fullsized":"2024-04-25T14:09:27.220560+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:03.242163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.220139+0000","last_change":"2024-04-25T14:09:25.843198+0000","last_active":"2024-04-25T14:09:27.220139+0000","last_peered":"2024-04-25T14:09:27.220139+0000","last_clean":"2024-04-25T14:09:27.220139+0000","last_became_active":"2024-04-25T14:09:25.842864+0000","last_became_peered":"2024-04-25T14:09:25.842864+0000","last_unstale":"2024-04-25T14:09:27.220139+0000","last_undegraded":"2024-04-25T14:09:27.220139+0000","last_fullsized":"2024-04-25T14:09:27.220139+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:41.950449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.219532+0000","last_change":"2024-04-25T14:09:24.817271+0000","last_active":"2024-04-25T14:09:27.219532+0000","last_peered":"2024-04-25T14:09:27.219532+0000","last_clean":"2024-04-25T14:09:27.219532+0000","last_became_active":"2024-04-25T14:09:24.816723+0000","last_became_peered":"2024-04-25T14:09:24.816723+0000","last_unstale":"2024-04-25T14:09:27.219532+0000","last_undegraded":"2024-04-25T14:09:27.219532+0000","last_fullsized":"2024-04-25T14:09:27.219532+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:13.148574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.838058+0000","last_change":"2024-04-25T14:09:25.849938+0000","last_active":"2024-04-25T14:09:26.838058+0000","last_peered":"2024-04-25T14:09:26.838058+0000","last_clean":"2024-04-25T14:09:26.838058+0000","last_became_active":"2024-04-25T14:09:25.849188+0000","last_became_peered":"2024-04-25T14:09:25.849188+0000","last_unstale":"2024-04-25T14:09:26.838058+0000","last_undegraded":"2024-04-25T14:09:26.838058+0000","last_fullsized":"2024-04-25T14:09:26.838058+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:23.185454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.838676+0000","last_change":"2024-04-25T14:09:24.814155+0000","last_active":"2024-04-25T14:09:26.838676+0000","last_peered":"2024-04-25T14:09:26.838676+0000","last_clean":"2024-04-25T14:09:26.838676+0000","last_became_active":"2024-04-25T14:09:24.812762+0000","last_became_peered":"2024-04-25T14:09:24.812762+0000","last_unstale":"2024-04-25T14:09:26.838676+0000","last_undegraded":"2024-04-25T14:09:26.838676+0000","last_fullsized":"2024-04-25T14:09:26.838676+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:03.242163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.843859+0000","last_change":"2024-04-25T14:09:25.832253+0000","last_active":"2024-04-25T14:09:26.843859+0000","last_peered":"2024-04-25T14:09:26.843859+0000","last_clean":"2024-04-25T14:09:26.843859+0000","last_became_active":"2024-04-25T14:09:25.831894+0000","last_became_peered":"2024-04-25T14:09:25.831894+0000","last_unstale":"2024-04-25T14:09:26.843859+0000","last_undegraded":"2024-04-25T14:09:26.843859+0000","last_fullsized":"2024-04-25T14:09:26.843859+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:47.246117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218361+0000","last_change":"2024-04-25T14:09:24.811205+0000","last_active":"2024-04-25T14:09:27.218361+0000","last_peered":"2024-04-25T14:09:27.218361+0000","last_clean":"2024-04-25T14:09:27.218361+0000","last_became_active":"2024-04-25T14:09:24.810270+0000","last_became_peered":"2024-04-25T14:09:24.810270+0000","last_unstale":"2024-04-25T14:09:27.218361+0000","last_undegraded":"2024-04-25T14:09:27.218361+0000","last_fullsized":"2024-04-25T14:09:27.218361+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:02.932797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.844121+0000","last_change":"2024-04-25T14:09:25.846671+0000","last_active":"2024-04-25T14:09:26.844121+0000","last_peered":"2024-04-25T14:09:26.844121+0000","last_clean":"2024-04-25T14:09:26.844121+0000","last_became_active":"2024-04-25T14:09:25.846343+0000","last_became_peered":"2024-04-25T14:09:25.846343+0000","last_unstale":"2024-04-25T14:09:26.844121+0000","last_undegraded":"2024-04-25T14:09:26.844121+0000","last_fullsized":"2024-04-25T14:09:26.844121+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:23.654263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218033+0000","last_change":"2024-04-25T14:09:24.808594+0000","last_active":"2024-04-25T14:09:27.218033+0000","last_peered":"2024-04-25T14:09:27.218033+0000","last_clean":"2024-04-25T14:09:27.218033+0000","last_became_active":"2024-04-25T14:09:24.807904+0000","last_became_peered":"2024-04-25T14:09:24.807904+0000","last_unstale":"2024-04-25T14:09:27.218033+0000","last_undegraded":"2024-04-25T14:09:27.218033+0000","last_fullsized":"2024-04-25T14:09:27.218033+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:49.025340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.837657+0000","last_change":"2024-04-25T14:09:25.834573+0000","last_active":"2024-04-25T14:09:26.837657+0000","last_peered":"2024-04-25T14:09:26.837657+0000","last_clean":"2024-04-25T14:09:26.837657+0000","last_became_active":"2024-04-25T14:09:25.834250+0000","last_became_peered":"2024-04-25T14:09:25.834250+0000","last_unstale":"2024-04-25T14:09:26.837657+0000","last_undegraded":"2024-04-25T14:09:26.837657+0000","last_fullsized":"2024-04-25T14:09:26.837657+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:54.817541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.219892+0000","last_change":"2024-04-25T14:09:24.816525+0000","last_active":"2024-04-25T14:09:27.219892+0000","last_peered":"2024-04-25T14:09:27.219892+0000","last_clean":"2024-04-25T14:09:27.219892+0000","last_became_active":"2024-04-25T14:09:24.815546+0000","last_became_peered":"2024-04-25T14:09:24.815546+0000","last_unstale":"2024-04-25T14:09:27.219892+0000","last_undegraded":"2024-04-25T14:09:27.219892+0000","last_fullsized":"2024-04-25T14:09:27.219892+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:12.397018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218537+0000","last_change":"2024-04-25T14:09:25.841620+0000","last_active":"2024-04-25T14:09:27.218537+0000","last_peered":"2024-04-25T14:09:27.218537+0000","last_clean":"2024-04-25T14:09:27.218537+0000","last_became_active":"2024-04-25T14:09:25.841321+0000","last_became_peered":"2024-04-25T14:09:25.841321+0000","last_unstale":"2024-04-25T14:09:27.218537+0000","last_undegraded":"2024-04-25T14:09:27.218537+0000","last_fullsized":"2024-04-25T14:09:27.218537+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:47.773774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218405+0000","last_change":"2024-04-25T14:09:24.813608+0000","last_active":"2024-04-25T14:09:27.218405+0000","last_peered":"2024-04-25T14:09:27.218405+0000","last_clean":"2024-04-25T14:09:27.218405+0000","last_became_active":"2024-04-25T14:09:24.813300+0000","last_became_peered":"2024-04-25T14:09:24.813300+0000","last_unstale":"2024-04-25T14:09:27.218405+0000","last_undegraded":"2024-04-25T14:09:27.218405+0000","last_fullsized":"2024-04-25T14:09:27.218405+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:21.121217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.843786+0000","last_change":"2024-04-25T14:09:25.835007+0000","last_active":"2024-04-25T14:09:26.843786+0000","last_peered":"2024-04-25T14:09:26.843786+0000","last_clean":"2024-04-25T14:09:26.843786+0000","last_became_active":"2024-04-25T14:09:25.832066+0000","last_became_peered":"2024-04-25T14:09:25.832066+0000","last_unstale":"2024-04-25T14:09:26.843786+0000","last_undegraded":"2024-04-25T14:09:26.843786+0000","last_fullsized":"2024-04-25T14:09:26.843786+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:13:32.841881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.839126+0000","last_change":"2024-04-25T14:09:24.818883+0000","last_active":"2024-04-25T14:09:26.839126+0000","last_peered":"2024-04-25T14:09:26.839126+0000","last_clean":"2024-04-25T14:09:26.839126+0000","last_became_active":"2024-04-25T14:09:24.817518+0000","last_became_peered":"2024-04-25T14:09:24.817518+0000","last_unstale":"2024-04-25T14:09:26.839126+0000","last_undegraded":"2024-04-25T14:09:26.839126+0000","last_fullsized":"2024-04-25T14:09:26.839126+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:00.582546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.219450+0000","last_change":"2024-04-25T14:09:25.840258+0000","last_active":"2024-04-25T14:09:27.219450+0000","last_peered":"2024-04-25T14:09:27.219450+0000","last_clean":"2024-04-25T14:09:27.219450+0000","last_became_active":"2024-04-25T14:09:25.839964+0000","last_became_peered":"2024-04-25T14:09:25.839964+0000","last_unstale":"2024-04-25T14:09:27.219450+0000","last_undegraded":"2024-04-25T14:09:27.219450+0000","last_fullsized":"2024-04-25T14:09:27.219450+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:59.777406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"100'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.844088+0000","last_change":"2024-04-25T14:09:24.814702+0000","last_active":"2024-04-25T14:09:26.844088+0000","last_peered":"2024-04-25T14:09:26.844088+0000","last_clean":"2024-04-25T14:09:26.844088+0000","last_became_active":"2024-04-25T14:09:24.807948+0000","last_became_peered":"2024-04-25T14:09:24.807948+0000","last_unstale":"2024-04-25T14:09:26.844088+0000","last_undegraded":"2024-04-25T14:09:26.844088+0000","last_fullsized":"2024-04-25T14:09:26.844088+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:43:21.822297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.837255+0000","last_change":"2024-04-25T14:09:25.833348+0000","last_active":"2024-04-25T14:09:26.837255+0000","last_peered":"2024-04-25T14:09:26.837255+0000","last_clean":"2024-04-25T14:09:26.837255+0000","last_became_active":"2024-04-25T14:09:25.832989+0000","last_became_peered":"2024-04-25T14:09:25.832989+0000","last_unstale":"2024-04-25T14:09:26.837255+0000","last_undegraded":"2024-04-25T14:09:26.837255+0000","last_fullsized":"2024-04-25T14:09:26.837255+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:41.427994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.220482+0000","last_change":"2024-04-25T14:09:24.817779+0000","last_active":"2024-04-25T14:09:27.220482+0000","last_peered":"2024-04-25T14:09:27.220482+0000","last_clean":"2024-04-25T14:09:27.220482+0000","last_became_active":"2024-04-25T14:09:24.817493+0000","last_became_peered":"2024-04-25T14:09:24.817493+0000","last_unstale":"2024-04-25T14:09:27.220482+0000","last_undegraded":"2024-04-25T14:09:27.220482+0000","last_fullsized":"2024-04-25T14:09:27.220482+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:03.406811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.838495+0000","last_change":"2024-04-25T14:09:25.851235+0000","last_active":"2024-04-25T14:09:26.838495+0000","last_peered":"2024-04-25T14:09:26.838495+0000","last_clean":"2024-04-25T14:09:26.838495+0000","last_became_active":"2024-04-25T14:09:25.850541+0000","last_became_peered":"2024-04-25T14:09:25.850541+0000","last_unstale":"2024-04-25T14:09:26.838495+0000","last_undegraded":"2024-04-25T14:09:26.838495+0000","last_fullsized":"2024-04-25T14:09:26.838495+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:12.908722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.838909+0000","last_change":"2024-04-25T14:09:24.818436+0000","last_active":"2024-04-25T14:09:26.838909+0000","last_peered":"2024-04-25T14:09:26.838909+0000","last_clean":"2024-04-25T14:09:26.838909+0000","last_became_active":"2024-04-25T14:09:24.818157+0000","last_became_peered":"2024-04-25T14:09:24.818157+0000","last_unstale":"2024-04-25T14:09:26.838909+0000","last_undegraded":"2024-04-25T14:09:26.838909+0000","last_fullsized":"2024-04-25T14:09:26.838909+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:10.844004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218719+0000","last_change":"2024-04-25T14:09:25.844544+0000","last_active":"2024-04-25T14:09:27.218719+0000","last_peered":"2024-04-25T14:09:27.218719+0000","last_clean":"2024-04-25T14:09:27.218719+0000","last_became_active":"2024-04-25T14:09:25.844228+0000","last_became_peered":"2024-04-25T14:09:25.844228+0000","last_unstale":"2024-04-25T14:09:27.218719+0000","last_undegraded":"2024-04-25T14:09:27.218719+0000","last_fullsized":"2024-04-25T14:09:27.218719+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:00.764143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218863+0000","last_change":"2024-04-25T14:09:24.820985+0000","last_active":"2024-04-25T14:09:27.218863+0000","last_peered":"2024-04-25T14:09:27.218863+0000","last_clean":"2024-04-25T14:09:27.218863+0000","last_became_active":"2024-04-25T14:09:24.819953+0000","last_became_peered":"2024-04-25T14:09:24.819953+0000","last_unstale":"2024-04-25T14:09:27.218863+0000","last_undegraded":"2024-04-25T14:09:27.218863+0000","last_fullsized":"2024-04-25T14:09:27.218863+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:44.194963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218634+0000","last_change":"2024-04-25T14:09:25.844551+0000","last_active":"2024-04-25T14:09:27.218634+0000","last_peered":"2024-04-25T14:09:27.218634+0000","last_clean":"2024-04-25T14:09:27.218634+0000","last_became_active":"2024-04-25T14:09:25.844241+0000","last_became_peered":"2024-04-25T14:09:25.844241+0000","last_unstale":"2024-04-25T14:09:27.218634+0000","last_undegraded":"2024-04-25T14:09:27.218634+0000","last_fullsized":"2024-04-25T14:09:27.218634+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:48.160466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218237+0000","last_change":"2024-04-25T14:09:24.822304+0000","last_active":"2024-04-25T14:09:27.218237+0000","last_peered":"2024-04-25T14:09:27.218237+0000","last_clean":"2024-04-25T14:09:27.218237+0000","last_became_active":"2024-04-25T14:09:24.821946+0000","last_became_peered":"2024-04-25T14:09:24.821946+0000","last_unstale":"2024-04-25T14:09:27.218237+0000","last_undegraded":"2024-04-25T14:09:27.218237+0000","last_fullsized":"2024-04-25T14:09:27.218237+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:21.696511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.219045+0000","last_change":"2024-04-25T14:09:25.841200+0000","last_active":"2024-04-25T14:09:27.219045+0000","last_peered":"2024-04-25T14:09:27.219045+0000","last_clean":"2024-04-25T14:09:27.219045+0000","last_became_active":"2024-04-25T14:09:25.840879+0000","last_became_peered":"2024-04-25T14:09:25.840879+0000","last_unstale":"2024-04-25T14:09:27.219045+0000","last_undegraded":"2024-04-25T14:09:27.219045+0000","last_fullsized":"2024-04-25T14:09:27.219045+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:44.842316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.219773+0000","last_change":"2024-04-25T14:09:24.811162+0000","last_active":"2024-04-25T14:09:27.219773+0000","last_peered":"2024-04-25T14:09:27.219773+0000","last_clean":"2024-04-25T14:09:27.219773+0000","last_became_active":"2024-04-25T14:09:24.810194+0000","last_became_peered":"2024-04-25T14:09:24.810194+0000","last_unstale":"2024-04-25T14:09:27.219773+0000","last_undegraded":"2024-04-25T14:09:27.219773+0000","last_fullsized":"2024-04-25T14:09:27.219773+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:57.671866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.839118+0000","last_change":"2024-04-25T14:09:25.830784+0000","last_active":"2024-04-25T14:09:26.839118+0000","last_peered":"2024-04-25T14:09:26.839118+0000","last_clean":"2024-04-25T14:09:26.839118+0000","last_became_active":"2024-04-25T14:09:25.829896+0000","last_became_peered":"2024-04-25T14:09:25.829896+0000","last_unstale":"2024-04-25T14:09:26.839118+0000","last_undegraded":"2024-04-25T14:09:26.839118+0000","last_fullsized":"2024-04-25T14:09:26.839118+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:41.950449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218516+0000","last_change":"2024-04-25T14:09:24.817347+0000","last_active":"2024-04-25T14:09:27.218516+0000","last_peered":"2024-04-25T14:09:27.218516+0000","last_clean":"2024-04-25T14:09:27.218516+0000","last_became_active":"2024-04-25T14:09:24.815968+0000","last_became_peered":"2024-04-25T14:09:24.815968+0000","last_unstale":"2024-04-25T14:09:27.218516+0000","last_undegraded":"2024-04-25T14:09:27.218516+0000","last_fullsized":"2024-04-25T14:09:27.218516+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:05:31.218027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:26.838326+0000","last_change":"2024-04-25T14:09:25.832559+0000","last_active":"2024-04-25T14:09:26.838326+0000","last_peered":"2024-04-25T14:09:26.838326+0000","last_clean":"2024-04-25T14:09:26.838326+0000","last_became_active":"2024-04-25T14:09:25.832253+0000","last_became_peered":"2024-04-25T14:09:25.832253+0000","last_unstale":"2024-04-25T14:09:26.838326+0000","last_undegraded":"2024-04-25T14:09:26.838326+0000","last_fullsized":"2024-04-25T14:09:26.838326+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:24.788787+0000","last_clean_scrub_stamp":"2024-04-25T14:09:24.788787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:46.900315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218596+0000","last_change":"2024-04-25T14:09:24.816306+0000","last_active":"2024-04-25T14:09:27.218596+0000","last_peered":"2024-04-25T14:09:27.218596+0000","last_clean":"2024-04-25T14:09:27.218596+0000","last_became_active":"2024-04-25T14:09:24.816004+0000","last_became_peered":"2024-04-25T14:09:24.816004+0000","last_unstale":"2024-04-25T14:09:27.218596+0000","last_undegraded":"2024-04-25T14:09:27.218596+0000","last_fullsized":"2024-04-25T14:09:27.218596+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:09:23.780834+0000","last_clean_scrub_stamp":"2024-04-25T14:09:23.780834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:11.119999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"89'107","reported_seq":381,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T14:09:27.218760+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:09:27.218760+0000","last_peered":"2024-04-25T14:09:27.218760+0000","last_clean":"2024-04-25T14:09:27.218760+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:09:27.218760+0000","last_undegraded":"2024-04-25T14:09:27.218760+0000","last_fullsized":"2024-04-25T14:09:27.218760+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":366,"num_read_kb":935,"num_write":161,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":366,"num_read_kb":935,"num_write":161,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574917,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32984,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338856,"statfs":{"total":96636764160,"available":96602988544,"internally_reserved":0,"allocated":1843200,"data_stored":1209160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574918,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1261568,"data_stored":620848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574919,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1241088,"data_stored":618632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574918,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1249280,"data_stored":618724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574918,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1245184,"data_stored":619214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1245184,"data_stored":618666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1245184,"data_stored":618666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574918,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32996,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338844,"statfs":{"total":96636764160,"available":96602976256,"internally_reserved":0,"allocated":1851392,"data_stored":1210702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:09:34.901 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T14:09:34.901 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T14:09:35.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.102+0000 7f49ea400640 1 Processor -- start 2024-04-25T14:09:35.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.102+0000 7f49ea400640 1 -- start start 2024-04-25T14:09:35.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.102+0000 7f49ea400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f49e406fd50 0x7f49e4070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.102+0000 7f49ea400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49e40b0a60 con 0x7f49e415feb0 2024-04-25T14:09:35.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.102+0000 7f49ea400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49e40ae840 con 0x7f49e415c2a0 2024-04-25T14:09:35.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.102+0000 7f49ea400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f49e406ef90 con 0x7f49e406fd50 2024-04-25T14:09:35.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.102+0000 7f49e2a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f49e406fd50 0x7f49e4070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.102+0000 7f49e2a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f49e406fd50 0x7f49e4070160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58112/0 (socket says 172.21.15.86:58112) 2024-04-25T14:09:35.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.102+0000 7f49e2a00640 1 -- 172.21.15.86:0/3538741223 learned_addr learned my addr 172.21.15.86:0/3538741223 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:35.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.102+0000 7f49e3e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f49e415feb0 0x7f49e41602c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42886/0 (socket says 172.21.15.86:42886) 2024-04-25T14:09:35.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.102+0000 7f49e2a00640 1 -- 172.21.15.86:0/3538741223 >> v1:172.21.15.155:6789/0 conn(0x7f49e415c2a0 legacy=0x7f49e40ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:35.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.102+0000 7f49e2a00640 1 -- 172.21.15.86:0/3538741223 >> v1:172.21.15.86:6789/0 conn(0x7f49e415feb0 legacy=0x7f49e41602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:35.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.102+0000 7f49e2a00640 1 -- 172.21.15.86:0/3538741223 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e406f270 con 0x7f49e406fd50 2024-04-25T14:09:35.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.102+0000 7f49e2000640 1 -- 172.21.15.86:0/3538741223 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3733523423 0 0) 0x7f49e40b0a60 con 0x7f49e415feb0 2024-04-25T14:09:35.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.106+0000 7f49e2a00640 1 --2- 172.21.15.86:0/3538741223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49e406fd50 0x7f49e4070160 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009870 tx=0x7f49d001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39270f45821bb883 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:35.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.106+0000 7f49e2000640 1 -- 172.21.15.86:0/3538741223 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d00b1070 con 0x7f49e406fd50 2024-04-25T14:09:35.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.106+0000 7f49e2000640 1 -- 172.21.15.86:0/3538741223 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d00ac410 con 0x7f49e406fd50 2024-04-25T14:09:35.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.106+0000 7f49e2000640 1 -- 172.21.15.86:0/3538741223 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d00ac7e0 con 0x7f49e406fd50 2024-04-25T14:09:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.106+0000 7f49ea400640 1 -- 172.21.15.86:0/3538741223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49e406fd50 msgr2=0x7f49e4070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.106+0000 7f49ea400640 1 --2- 172.21.15.86:0/3538741223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49e406fd50 0x7f49e4070160 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009870 tx=0x7f49d001c4a0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.106+0000 7f49ea400640 1 -- 172.21.15.86:0/3538741223 shutdown_connections 2024-04-25T14:09:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.106+0000 7f49ea400640 1 --2- 172.21.15.86:0/3538741223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49e406fd50 0x7f49e4070160 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.106+0000 7f49ea400640 1 -- 172.21.15.86:0/3538741223 >> 172.21.15.86:0/3538741223 conn(0x7f49e40b4650 msgr2=0x7f49e40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.106+0000 7f49ea400640 1 -- 172.21.15.86:0/3538741223 shutdown_connections 2024-04-25T14:09:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.106+0000 7f49ea400640 1 -- 172.21.15.86:0/3538741223 wait complete. 2024-04-25T14:09:35.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.110+0000 7f49ea400640 1 Processor -- start 2024-04-25T14:09:35.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.110+0000 7f49ea400640 1 -- start start 2024-04-25T14:09:35.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.110+0000 7f49ea400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49e406fd50 0x7f49e409a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.110+0000 7f49ea400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49e415c2a0 0x7f49e41549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.110+0000 7f49ea400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49e415feb0 0x7f49e4154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.110+0000 7f49ea400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f49e415c6b0 con 0x7f49e406fd50 2024-04-25T14:09:35.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.110+0000 7f49ea400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f49e40d0fc0 con 0x7f49e415feb0 2024-04-25T14:09:35.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.110+0000 7f49ea400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f49e415c830 con 0x7f49e415c2a0 2024-04-25T14:09:35.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.110+0000 7f49e3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49e406fd50 0x7f49e409a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.110+0000 7f49e3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49e406fd50 0x7f49e409a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58126/0 (socket says 172.21.15.86:58126) 2024-04-25T14:09:35.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.110+0000 7f49e3400640 1 -- 172.21.15.86:0/984729235 learned_addr learned my addr 172.21.15.86:0/984729235 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:35.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.110+0000 7f49e3e00640 1 --2- 172.21.15.86:0/984729235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49e415feb0 0x7f49e4154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.110+0000 7f49e2a00640 1 --2- 172.21.15.86:0/984729235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49e415c2a0 0x7f49e41549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.110+0000 7f49e3400640 1 -- 172.21.15.86:0/984729235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49e415c2a0 msgr2=0x7f49e41549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.110+0000 7f49e3400640 1 --2- 172.21.15.86:0/984729235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49e415c2a0 0x7f49e41549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.110+0000 7f49e3400640 1 -- 172.21.15.86:0/984729235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49e415feb0 msgr2=0x7f49e4154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.110+0000 7f49e3400640 1 --2- 172.21.15.86:0/984729235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49e415feb0 0x7f49e4154ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.110+0000 7f49e3400640 1 -- 172.21.15.86:0/984729235 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e40b1b20 con 0x7f49e406fd50 2024-04-25T14:09:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.110+0000 7f49e3400640 1 --2- 172.21.15.86:0/984729235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49e406fd50 0x7f49e409a520 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f49d400c3f0 tx=0x7f49d400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.110+0000 7f49db400640 1 -- 172.21.15.86:0/984729235 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d4017070 con 0x7f49e406fd50 2024-04-25T14:09:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.110+0000 7f49db400640 1 -- 172.21.15.86:0/984729235 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d40052d0 con 0x7f49e406fd50 2024-04-25T14:09:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.110+0000 7f49db400640 1 -- 172.21.15.86:0/984729235 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d4003840 con 0x7f49e406fd50 2024-04-25T14:09:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.110+0000 7f49ea400640 1 -- 172.21.15.86:0/984729235 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e40b1e00 con 0x7f49e406fd50 2024-04-25T14:09:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.110+0000 7f49ea400640 1 -- 172.21.15.86:0/984729235 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e40b23c0 con 0x7f49e406fd50 2024-04-25T14:09:35.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.114+0000 7f49ea400640 1 -- 172.21.15.86:0/984729235 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49e4069d00 con 0x7f49e406fd50 2024-04-25T14:09:35.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.114+0000 7f49db400640 1 -- 172.21.15.86:0/984729235 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7f49d401a070 con 0x7f49e406fd50 2024-04-25T14:09:35.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.114+0000 7f49db400640 1 --2- 172.21.15.86:0/984729235 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f49c406f520 0x7f49c4071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.114+0000 7f49db400640 1 -- 172.21.15.86:0/984729235 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7f49d408e870 con 0x7f49e406fd50 2024-04-25T14:09:35.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.114+0000 7f49e2a00640 1 --2- 172.21.15.86:0/984729235 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f49c406f520 0x7f49c4071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.118+0000 7f49e2a00640 1 --2- 172.21.15.86:0/984729235 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f49c406f520 0x7f49c4071a30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f49d00ae000 tx=0x7f49d00a6040 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:35.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.118+0000 7f49db400640 1 -- 172.21.15.86:0/984729235 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f49d4005480 con 0x7f49e406fd50 2024-04-25T14:09:35.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.330+0000 7f49ea400640 1 -- 172.21.15.86:0/984729235 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f49e4076bf0 con 0x7f49e406fd50 2024-04-25T14:09:35.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.334+0000 7f49db400640 1 -- 172.21.15.86:0/984729235 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v26) ==== 190+0+147 (secure 0 0 0) 0x7f49d405b550 con 0x7f49e406fd50 2024-04-25T14:09:35.331 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-04-25T14:09:35.331 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCLYipm5u4xHRAACDFe3Eb4KLrsdLj4eFe7mA== 2024-04-25T14:09:35.331 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-04-25T14:09:35.332 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-04-25T14:09:35.332 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T14:09:35.332 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.0 2024-04-25T14:09:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.338+0000 7f49ea400640 1 -- 172.21.15.86:0/984729235 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f49c406f520 msgr2=0x7f49c4071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.338+0000 7f49ea400640 1 --2- 172.21.15.86:0/984729235 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f49c406f520 0x7f49c4071a30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f49d00ae000 tx=0x7f49d00a6040 comp rx=0 tx=0).stop 2024-04-25T14:09:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.338+0000 7f49ea400640 1 -- 172.21.15.86:0/984729235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49e406fd50 msgr2=0x7f49e409a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.338+0000 7f49ea400640 1 --2- 172.21.15.86:0/984729235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49e406fd50 0x7f49e409a520 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f49d400c3f0 tx=0x7f49d400c8b0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.338+0000 7f49ea400640 1 -- 172.21.15.86:0/984729235 shutdown_connections 2024-04-25T14:09:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.338+0000 7f49ea400640 1 --2- 172.21.15.86:0/984729235 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f49c406f520 0x7f49c4071a30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.338+0000 7f49ea400640 1 --2- 172.21.15.86:0/984729235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49e415feb0 0x7f49e4154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.338+0000 7f49ea400640 1 --2- 172.21.15.86:0/984729235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49e415c2a0 0x7f49e41549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.338+0000 7f49ea400640 1 --2- 172.21.15.86:0/984729235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49e406fd50 0x7f49e409a520 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.338+0000 7f49ea400640 1 -- 172.21.15.86:0/984729235 >> 172.21.15.86:0/984729235 conn(0x7f49e40b4650 msgr2=0x7f49e41590e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.338+0000 7f49ea400640 1 -- 172.21.15.86:0/984729235 shutdown_connections 2024-04-25T14:09:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.338+0000 7f49ea400640 1 -- 172.21.15.86:0/984729235 wait complete. 2024-04-25T14:09:35.354 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:09:35.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.506+0000 7f66d9c00640 1 Processor -- start 2024-04-25T14:09:35.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.506+0000 7f66d9c00640 1 -- start start 2024-04-25T14:09:35.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.506+0000 7f66d9c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66d415c2a0 0x7f66d40ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.506+0000 7f66d9c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66d40b0a60 con 0x7f66d415feb0 2024-04-25T14:09:35.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.506+0000 7f66d9c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66d406ef90 con 0x7f66d406fd50 2024-04-25T14:09:35.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.506+0000 7f66d9c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f66d406f270 con 0x7f66d415c2a0 2024-04-25T14:09:35.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.506+0000 7f66d3400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f66d415feb0 0x7f66d41602c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42896/0 (socket says 172.21.15.86:42896) 2024-04-25T14:09:35.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.506+0000 7f66d3400640 1 -- 172.21.15.86:0/3434036411 learned_addr learned my addr 172.21.15.86:0/3434036411 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:35.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.510+0000 7f66d2a00640 1 --2- 172.21.15.86:0/3434036411 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66d415c2a0 0x7f66d40ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.510+0000 7f66d1600640 1 -- 172.21.15.86:0/3434036411 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1279630093 0 0) 0x7f66d40b0a60 con 0x7f66d415feb0 2024-04-25T14:09:35.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.510+0000 7f66d1600640 1 -- 172.21.15.86:0/3434036411 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66b8003450 con 0x7f66d415feb0 2024-04-25T14:09:35.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.510+0000 7f66d1600640 1 -- 172.21.15.86:0/3434036411 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2858497567 0 0) 0x7f66d406ef90 con 0x7f66d406fd50 2024-04-25T14:09:35.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.510+0000 7f66d1600640 1 -- 172.21.15.86:0/3434036411 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66d40b0a60 con 0x7f66d406fd50 2024-04-25T14:09:35.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.510+0000 7f66d1600640 1 -- 172.21.15.86:0/3434036411 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2272239924 0 0) 0x7f66b8003450 con 0x7f66d415feb0 2024-04-25T14:09:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.510+0000 7f66d1600640 1 -- 172.21.15.86:0/3434036411 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66d415c2a0 msgr2=0x7f66d40ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.510+0000 7f66d1600640 1 --2- 172.21.15.86:0/3434036411 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66d415c2a0 0x7f66d40ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.510+0000 7f66d1600640 1 -- 172.21.15.86:0/3434036411 >> v1:172.21.15.155:6789/0 conn(0x7f66d406fd50 legacy=0x7f66d4070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.510+0000 7f66d1600640 1 -- 172.21.15.86:0/3434036411 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d406f550 con 0x7f66d415feb0 2024-04-25T14:09:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.510+0000 7f66d1600640 1 -- 172.21.15.86:0/3434036411 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f66cc001a00 con 0x7f66d415feb0 2024-04-25T14:09:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.510+0000 7f66d1600640 1 -- 172.21.15.86:0/3434036411 >> v1:172.21.15.86:6789/0 conn(0x7f66d415feb0 legacy=0x7f66d41602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.510+0000 7f66d1600640 1 --2- 172.21.15.86:0/3434036411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66b8003ce0 0x7f66b8008860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.510+0000 7f66d1600640 1 --2- 172.21.15.86:0/3434036411 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66d415c2a0 0x7f66b800adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.510+0000 7f66d1600640 1 --2- 172.21.15.86:0/3434036411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66b800b2f0 0x7f66b800d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.510+0000 7f66d1600640 1 -- 172.21.15.86:0/3434036411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f66d406f270 con 0x7f66b800b2f0 2024-04-25T14:09:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.510+0000 7f66d1600640 1 -- 172.21.15.86:0/3434036411 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f66b800de10 con 0x7f66b8003ce0 2024-04-25T14:09:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.510+0000 7f66d1600640 1 -- 172.21.15.86:0/3434036411 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f66b800e010 con 0x7f66d415c2a0 2024-04-25T14:09:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.510+0000 7f66d3400640 1 --2- 172.21.15.86:0/3434036411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66b800b2f0 0x7f66b800d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.510+0000 7f66d2a00640 1 --2- 172.21.15.86:0/3434036411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66b8003ce0 0x7f66b8008860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.510+0000 7f66d2000640 1 --2- 172.21.15.86:0/3434036411 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66d415c2a0 0x7f66b800adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.510+0000 7f66d3400640 1 -- 172.21.15.86:0/3434036411 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66d415c2a0 msgr2=0x7f66b800adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:35.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.510+0000 7f66d3400640 1 --2- 172.21.15.86:0/3434036411 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66d415c2a0 0x7f66b800adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.510+0000 7f66d3400640 1 -- 172.21.15.86:0/3434036411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66b8003ce0 msgr2=0x7f66b8008860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:35.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.510+0000 7f66d3400640 1 --2- 172.21.15.86:0/3434036411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66b8003ce0 0x7f66b8008860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.510+0000 7f66d3400640 1 -- 172.21.15.86:0/3434036411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b8003450 con 0x7f66b800b2f0 2024-04-25T14:09:35.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.514+0000 7f66d3400640 1 --2- 172.21.15.86:0/3434036411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66b800b2f0 0x7f66b800d710 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00a7f0 tx=0x7f66cc01d320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2013909148de1f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:35.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.514+0000 7f66d1600640 1 -- 172.21.15.86:0/3434036411 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc00a410 con 0x7f66b800b2f0 2024-04-25T14:09:35.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.514+0000 7f66d1600640 1 -- 172.21.15.86:0/3434036411 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66cc0ad650 con 0x7f66b800b2f0 2024-04-25T14:09:35.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.514+0000 7f66d1600640 1 -- 172.21.15.86:0/3434036411 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc0ada40 con 0x7f66b800b2f0 2024-04-25T14:09:35.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.514+0000 7f66d9c00640 1 -- 172.21.15.86:0/3434036411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66b800b2f0 msgr2=0x7f66b800d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:35.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.514+0000 7f66d9c00640 1 --2- 172.21.15.86:0/3434036411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66b800b2f0 0x7f66b800d710 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00a7f0 tx=0x7f66cc01d320 comp rx=0 tx=0).stop 2024-04-25T14:09:35.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.514+0000 7f66d3400640 1 -- 172.21.15.86:0/3434036411 reap_dead start 2024-04-25T14:09:35.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.514+0000 7f66d9c00640 1 -- 172.21.15.86:0/3434036411 shutdown_connections 2024-04-25T14:09:35.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.514+0000 7f66d9c00640 1 -- 172.21.15.86:0/3434036411 >> 172.21.15.86:0/3434036411 conn(0x7f66d40b4650 msgr2=0x7f66d40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:35.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.514+0000 7f66d9c00640 1 -- 172.21.15.86:0/3434036411 shutdown_connections 2024-04-25T14:09:35.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.514+0000 7f66d9c00640 1 -- 172.21.15.86:0/3434036411 wait complete. 2024-04-25T14:09:35.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.514+0000 7f66d9c00640 1 Processor -- start 2024-04-25T14:09:35.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.514+0000 7f66d9c00640 1 -- start start 2024-04-25T14:09:35.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.518+0000 7f66d9c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66b800b2f0 0x7f66d41f6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.518+0000 7f66d9c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66d414fa10 0x7f66d409b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.518+0000 7f66d9c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66d41524e0 0x7f66d409b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.518+0000 7f66d9c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f66d415c6b0 con 0x7f66d41524e0 2024-04-25T14:09:35.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.518+0000 7f66d9c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f66d40d0fc0 con 0x7f66d414fa10 2024-04-25T14:09:35.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.518+0000 7f66d9c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f66d415c830 con 0x7f66b800b2f0 2024-04-25T14:09:35.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.518+0000 7f66d3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66d41524e0 0x7f66d409b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.518+0000 7f66d3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66d41524e0 0x7f66d409b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58146/0 (socket says 172.21.15.86:58146) 2024-04-25T14:09:35.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.518+0000 7f66d3400640 1 -- 172.21.15.86:0/1065827445 learned_addr learned my addr 172.21.15.86:0/1065827445 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:35.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.518+0000 7f66d2a00640 1 --2- 172.21.15.86:0/1065827445 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66b800b2f0 0x7f66d41f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.518+0000 7f66d2000640 1 --2- 172.21.15.86:0/1065827445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66d414fa10 0x7f66d409b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.518+0000 7f66d3400640 1 -- 172.21.15.86:0/1065827445 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66b800b2f0 msgr2=0x7f66d41f6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:35.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.518+0000 7f66d3400640 1 --2- 172.21.15.86:0/1065827445 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66b800b2f0 0x7f66d41f6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.518+0000 7f66d3400640 1 -- 172.21.15.86:0/1065827445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66d414fa10 msgr2=0x7f66d409b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:35.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.518+0000 7f66d3400640 1 --2- 172.21.15.86:0/1065827445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66d414fa10 0x7f66d409b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.518+0000 7f66d3400640 1 -- 172.21.15.86:0/1065827445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d40afec0 con 0x7f66d41524e0 2024-04-25T14:09:35.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.518+0000 7f66d2000640 1 --2- 172.21.15.86:0/1065827445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66d414fa10 0x7f66d409b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:35.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.518+0000 7f66d3400640 1 --2- 172.21.15.86:0/1065827445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66d41524e0 0x7f66d409b7f0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f66cc01d150 tx=0x7f66b8003740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:35.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.518+0000 7f66caa00640 1 -- 172.21.15.86:0/1065827445 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc0bc070 con 0x7f66d41524e0 2024-04-25T14:09:35.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.518+0000 7f66caa00640 1 -- 172.21.15.86:0/1065827445 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66cc0ad430 con 0x7f66d41524e0 2024-04-25T14:09:35.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.518+0000 7f66d9c00640 1 -- 172.21.15.86:0/1065827445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d40b2ec0 con 0x7f66d41524e0 2024-04-25T14:09:35.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.518+0000 7f66caa00640 1 -- 172.21.15.86:0/1065827445 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc001410 con 0x7f66d41524e0 2024-04-25T14:09:35.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.518+0000 7f66d9c00640 1 -- 172.21.15.86:0/1065827445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d40b3480 con 0x7f66d41524e0 2024-04-25T14:09:35.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.518+0000 7f66d9c00640 1 -- 172.21.15.86:0/1065827445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66d4069d00 con 0x7f66d41524e0 2024-04-25T14:09:35.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.522+0000 7f66caa00640 1 -- 172.21.15.86:0/1065827445 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7f66cc0bf070 con 0x7f66d41524e0 2024-04-25T14:09:35.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.522+0000 7f66caa00640 1 --2- 172.21.15.86:0/1065827445 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f66b406f6c0 0x7f66b4071bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.522+0000 7f66caa00640 1 -- 172.21.15.86:0/1065827445 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7f66cc1297e0 con 0x7f66d41524e0 2024-04-25T14:09:35.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.522+0000 7f66d2a00640 1 --2- 172.21.15.86:0/1065827445 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f66b406f6c0 0x7f66b4071bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.522+0000 7f66d2a00640 1 --2- 172.21.15.86:0/1065827445 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f66b406f6c0 0x7f66b4071bd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f66d40b4200 tx=0x7f66c4009190 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:35.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.526+0000 7f66caa00640 1 -- 172.21.15.86:0/1065827445 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f66cc017030 con 0x7f66d41524e0 2024-04-25T14:09:35.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.718+0000 7f66d9c00640 1 -- 172.21.15.86:0/1065827445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66d4076bf0 con 0x7f66d41524e0 2024-04-25T14:09:35.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.718+0000 7f66caa00640 1 -- 172.21.15.86:0/1065827445 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5189 (secure 0 0 0) 0x7f66cc0f6310 con 0x7f66d41524e0 2024-04-25T14:09:35.717 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:35.717 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":76,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1961914630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1961914630},{"type":"v1","addr":"172.21.15.86:6836","nonce":1961914630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2298829380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2298829380},{"type":"v1","addr":"172.21.15.155:6834","nonce":2298829380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7914,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3761704765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3761704765},{"type":"v1","addr":"172.21.15.86:6837","nonce":3761704765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:09:25.198798+0000","modified":"2024-04-25T14:09:28.237583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6835/1049360040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1049360040},{"type":"v1","addr":"172.21.15.155:6835","nonce":1049360040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7932,"qdb_cluster":[7932]},"id":7}]} 2024-04-25T14:09:35.717 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 76 2024-04-25T14:09:35.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.722+0000 7f66d9c00640 1 -- 172.21.15.86:0/1065827445 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f66b406f6c0 msgr2=0x7f66b4071bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:35.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.722+0000 7f66d9c00640 1 --2- 172.21.15.86:0/1065827445 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f66b406f6c0 0x7f66b4071bd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f66d40b4200 tx=0x7f66c4009190 comp rx=0 tx=0).stop 2024-04-25T14:09:35.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.722+0000 7f66d9c00640 1 -- 172.21.15.86:0/1065827445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66d41524e0 msgr2=0x7f66d409b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:35.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.722+0000 7f66d9c00640 1 --2- 172.21.15.86:0/1065827445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66d41524e0 0x7f66d409b7f0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f66cc01d150 tx=0x7f66b8003740 comp rx=0 tx=0).stop 2024-04-25T14:09:35.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.722+0000 7f66d9c00640 1 -- 172.21.15.86:0/1065827445 shutdown_connections 2024-04-25T14:09:35.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.722+0000 7f66d9c00640 1 --2- 172.21.15.86:0/1065827445 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f66b406f6c0 0x7f66b4071bd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.722+0000 7f66d9c00640 1 --2- 172.21.15.86:0/1065827445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66d41524e0 0x7f66d409b7f0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.722+0000 7f66d9c00640 1 --2- 172.21.15.86:0/1065827445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66d414fa10 0x7f66d409b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.722+0000 7f66d9c00640 1 --2- 172.21.15.86:0/1065827445 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66b800b2f0 0x7f66d41f6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.722+0000 7f66d9c00640 1 -- 172.21.15.86:0/1065827445 >> 172.21.15.86:0/1065827445 conn(0x7f66d40b4650 msgr2=0x7f66d415cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:35.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.722+0000 7f66d9c00640 1 -- 172.21.15.86:0/1065827445 shutdown_connections 2024-04-25T14:09:35.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:35.722+0000 7f66d9c00640 1 -- 172.21.15.86:0/1065827445 wait complete. 2024-04-25T14:09:36.141 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:09:25.198798+0000', 'modified': '2024-04-25T14:09:28.237583+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7932}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7932': {'gid': 7932, 'name': 'b', 'rank': 0, 'incarnation': 72, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.155:6835/1049360040', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6833', 'nonce': 1049360040}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 1049360040}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7932, 'qdb_cluster': [7932]} 2024-04-25T14:09:36.142 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T14:09:36.142 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T14:09:36.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.294+0000 7fbefba00640 1 Processor -- start 2024-04-25T14:09:36.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.294+0000 7fbefba00640 1 -- start start 2024-04-25T14:09:36.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.294+0000 7fbefba00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbef4070330 0x7fbef40d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:36.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.294+0000 7fbefba00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbef40d1c20 0x7fbef406ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:36.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.294+0000 7fbefba00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbef406f270 0x7fbef406f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:36.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.294+0000 7fbefba00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbef40d2030 con 0x7fbef4070330 2024-04-25T14:09:36.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.294+0000 7fbefba00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fbef415e5c0 con 0x7fbef406f270 2024-04-25T14:09:36.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.294+0000 7fbefba00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbef415e8a0 con 0x7fbef40d1c20 2024-04-25T14:09:36.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.294+0000 7fbef9000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbef4070330 0x7fbef40d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:36.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.294+0000 7fbef9000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbef4070330 0x7fbef40d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58160/0 (socket says 172.21.15.86:58160) 2024-04-25T14:09:36.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.294+0000 7fbef9000640 1 -- 172.21.15.86:0/3206954217 learned_addr learned my addr 172.21.15.86:0/3206954217 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:36.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.294+0000 7fbef3e00640 1 --2- 172.21.15.86:0/3206954217 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbef40d1c20 0x7fbef406ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:36.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.294+0000 7fbef9a00640 1 --2- 172.21.15.86:0/3206954217 >> v2:172.21.15.155:3300/0 conn(0x7fbef406f270 0x7fbef406f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:36.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.294+0000 7fbef9000640 1 -- 172.21.15.86:0/3206954217 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbef40d1c20 msgr2=0x7fbef406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:36.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.294+0000 7fbef9000640 1 --2- 172.21.15.86:0/3206954217 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbef40d1c20 0x7fbef406ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.294+0000 7fbef9000640 1 -- 172.21.15.86:0/3206954217 >> v2:172.21.15.155:3300/0 conn(0x7fbef406f270 msgr2=0x7fbef406f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:36.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.294+0000 7fbef9000640 1 --2- 172.21.15.86:0/3206954217 >> v2:172.21.15.155:3300/0 conn(0x7fbef406f270 0x7fbef406f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.294+0000 7fbef9000640 1 -- 172.21.15.86:0/3206954217 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef415eb80 con 0x7fbef4070330 2024-04-25T14:09:36.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.294+0000 7fbef9a00640 1 --2- 172.21.15.86:0/3206954217 >> v2:172.21.15.155:3300/0 conn(0x7fbef406f270 0x7fbef406f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:36.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.298+0000 7fbef9000640 1 --2- 172.21.15.86:0/3206954217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbef4070330 0x7fbef40d16e0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fbef406e9e0 tx=0x7fbee401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3692158e8742e1f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:36.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.298+0000 7fbef3400640 1 -- 172.21.15.86:0/3206954217 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee40ab600 con 0x7fbef4070330 2024-04-25T14:09:36.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.298+0000 7fbef3400640 1 -- 172.21.15.86:0/3206954217 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee40b0070 con 0x7fbef4070330 2024-04-25T14:09:36.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.298+0000 7fbef3400640 1 -- 172.21.15.86:0/3206954217 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee40ab9c0 con 0x7fbef4070330 2024-04-25T14:09:36.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.298+0000 7fbefba00640 1 -- 172.21.15.86:0/3206954217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbef4070330 msgr2=0x7fbef40d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:36.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.298+0000 7fbefba00640 1 --2- 172.21.15.86:0/3206954217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbef4070330 0x7fbef40d16e0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fbef406e9e0 tx=0x7fbee401c460 comp rx=0 tx=0).stop 2024-04-25T14:09:36.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.298+0000 7fbefba00640 1 -- 172.21.15.86:0/3206954217 shutdown_connections 2024-04-25T14:09:36.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.298+0000 7fbefba00640 1 --2- 172.21.15.86:0/3206954217 >> v2:172.21.15.155:3300/0 conn(0x7fbef406f270 0x7fbef406f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.298+0000 7fbefba00640 1 --2- 172.21.15.86:0/3206954217 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbef40d1c20 0x7fbef406ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.298+0000 7fbefba00640 1 --2- 172.21.15.86:0/3206954217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbef4070330 0x7fbef40d16e0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.298+0000 7fbefba00640 1 -- 172.21.15.86:0/3206954217 >> 172.21.15.86:0/3206954217 conn(0x7fbef40b4650 msgr2=0x7fbef40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:36.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.298+0000 7fbefba00640 1 -- 172.21.15.86:0/3206954217 shutdown_connections 2024-04-25T14:09:36.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.298+0000 7fbefba00640 1 -- 172.21.15.86:0/3206954217 wait complete. 2024-04-25T14:09:36.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.302+0000 7fbefba00640 1 Processor -- start 2024-04-25T14:09:36.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.302+0000 7fbefba00640 1 -- start start 2024-04-25T14:09:36.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.302+0000 7fbefba00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbef406f270 0x7fbef409aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:36.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.302+0000 7fbefba00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbef4070330 0x7fbef4147780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:36.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.302+0000 7fbefba00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbef40d1c20 0x7fbef4147cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:36.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.302+0000 7fbefba00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbef415f4b0 con 0x7fbef4070330 2024-04-25T14:09:36.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.302+0000 7fbefba00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbef415f330 con 0x7fbef40d1c20 2024-04-25T14:09:36.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.302+0000 7fbefba00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbef4070740 con 0x7fbef406f270 2024-04-25T14:09:36.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.302+0000 7fbef3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbef4070330 0x7fbef4147780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:36.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.306+0000 7fbef3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbef4070330 0x7fbef4147780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58166/0 (socket says 172.21.15.86:58166) 2024-04-25T14:09:36.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.306+0000 7fbef3e00640 1 -- 172.21.15.86:0/1862670575 learned_addr learned my addr 172.21.15.86:0/1862670575 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:36.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.306+0000 7fbef9000640 1 --2- 172.21.15.86:0/1862670575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbef406f270 0x7fbef409aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:36.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.306+0000 7fbef9a00640 1 --2- 172.21.15.86:0/1862670575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbef40d1c20 0x7fbef4147cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:36.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.306+0000 7fbef3e00640 1 -- 172.21.15.86:0/1862670575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbef406f270 msgr2=0x7fbef409aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:36.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.306+0000 7fbef3e00640 1 --2- 172.21.15.86:0/1862670575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbef406f270 0x7fbef409aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.306+0000 7fbef3e00640 1 -- 172.21.15.86:0/1862670575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbef40d1c20 msgr2=0x7fbef4147cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:36.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.306+0000 7fbef3e00640 1 --2- 172.21.15.86:0/1862670575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbef40d1c20 0x7fbef4147cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.306+0000 7fbef3e00640 1 -- 172.21.15.86:0/1862670575 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef4148540 con 0x7fbef4070330 2024-04-25T14:09:36.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.306+0000 7fbef3e00640 1 --2- 172.21.15.86:0/1862670575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbef4070330 0x7fbef4147780 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fbee000e8a0 tx=0x7fbee000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:36.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.306+0000 7fbef0c00640 1 -- 172.21.15.86:0/1862670575 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee0017070 con 0x7fbef4070330 2024-04-25T14:09:36.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.306+0000 7fbef0c00640 1 -- 172.21.15.86:0/1862670575 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee000c940 con 0x7fbef4070330 2024-04-25T14:09:36.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.306+0000 7fbefba00640 1 -- 172.21.15.86:0/1862670575 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef4159780 con 0x7fbef4070330 2024-04-25T14:09:36.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.306+0000 7fbef0c00640 1 -- 172.21.15.86:0/1862670575 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee000ccf0 con 0x7fbef4070330 2024-04-25T14:09:36.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.306+0000 7fbefba00640 1 -- 172.21.15.86:0/1862670575 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef40c8520 con 0x7fbef4070330 2024-04-25T14:09:36.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.310+0000 7fbef0c00640 1 -- 172.21.15.86:0/1862670575 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7fbee001a070 con 0x7fbef4070330 2024-04-25T14:09:36.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.310+0000 7fbef0c00640 1 --2- 172.21.15.86:0/1862670575 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fbed406f550 0x7fbed4071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:36.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.310+0000 7fbef0c00640 1 -- 172.21.15.86:0/1862670575 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7fbee008f400 con 0x7fbef4070330 2024-04-25T14:09:36.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.310+0000 7fbf03091000 1 Processor -- start 2024-04-25T14:09:36.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.310+0000 7fbef9000640 1 --2- 172.21.15.86:0/1862670575 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fbed406f550 0x7fbed4071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:36.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.310+0000 7fbf03091000 1 -- start start 2024-04-25T14:09:36.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.310+0000 7fbef9000640 1 --2- 172.21.15.86:0/1862670575 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fbed406f550 0x7fbed4071a60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbee40af000 tx=0x7fbee40a7000 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:36.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.310+0000 7fbf03091000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5629f8d1fc20 0x5629f8e7dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:36.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.310+0000 7fbf03091000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f8d0bea0 0x5629f8e7e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:36.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.310+0000 7fbf03091000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629f8904280 0x5629f8e7e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:36.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.310+0000 7fbf03091000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5629f8ced160 con 0x5629f8d0bea0 2024-04-25T14:09:36.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.310+0000 7fbf03091000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5629f8c82e20 con 0x5629f8904280 2024-04-25T14:09:36.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.310+0000 7fbf03091000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5629f8c3ba20 con 0x5629f8d1fc20 2024-04-25T14:09:36.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.310+0000 7fbef3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f8d0bea0 0x5629f8e7e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:36.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.310+0000 7fbef3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f8d0bea0 0x5629f8e7e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58174/0 (socket says 172.21.15.86:58174) 2024-04-25T14:09:36.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.310+0000 7fbef3e00640 1 -- 172.21.15.86:0/4158079313 learned_addr learned my addr 172.21.15.86:0/4158079313 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:36.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.310+0000 7fbef9000640 1 --2- 172.21.15.86:0/4158079313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5629f8d1fc20 0x5629f8e7dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:36.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.310+0000 7fbef9a00640 1 --2- 172.21.15.86:0/4158079313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629f8904280 0x5629f8e7e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:36.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.310+0000 7fbef3e00640 1 -- 172.21.15.86:0/4158079313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5629f8d1fc20 msgr2=0x5629f8e7dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:36.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.310+0000 7fbef3e00640 1 --2- 172.21.15.86:0/4158079313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5629f8d1fc20 0x5629f8e7dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.310+0000 7fbef3e00640 1 -- 172.21.15.86:0/4158079313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629f8904280 msgr2=0x5629f8e7e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:36.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.310+0000 7fbef3e00640 1 --2- 172.21.15.86:0/4158079313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629f8904280 0x5629f8e7e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.310+0000 7fbef3e00640 1 -- 172.21.15.86:0/4158079313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629f8c40100 con 0x5629f8d0bea0 2024-04-25T14:09:36.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.314+0000 7fbef3e00640 1 --2- 172.21.15.86:0/4158079313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f8d0bea0 0x5629f8e7e420 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fbee000e870 tx=0x7fbee00579f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d87f4967b81d26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:36.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.314+0000 7fbee9600640 1 -- 172.21.15.86:0/4158079313 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee0021040 con 0x5629f8d0bea0 2024-04-25T14:09:36.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.314+0000 7fbee9600640 1 -- 172.21.15.86:0/4158079313 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee005fd00 con 0x5629f8d0bea0 2024-04-25T14:09:36.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.314+0000 7fbee9600640 1 -- 172.21.15.86:0/4158079313 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee00647b0 con 0x5629f8d0bea0 2024-04-25T14:09:36.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.314+0000 7fbf03091000 1 -- 172.21.15.86:0/4158079313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f8d0bea0 msgr2=0x5629f8e7e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:36.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.314+0000 7fbf03091000 1 --2- 172.21.15.86:0/4158079313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f8d0bea0 0x5629f8e7e420 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fbee000e870 tx=0x7fbee00579f0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.314+0000 7fbf03091000 1 -- 172.21.15.86:0/4158079313 shutdown_connections 2024-04-25T14:09:36.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.314+0000 7fbf03091000 1 --2- 172.21.15.86:0/4158079313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629f8904280 0x5629f8e7e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.314+0000 7fbf03091000 1 --2- 172.21.15.86:0/4158079313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f8d0bea0 0x5629f8e7e420 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.314+0000 7fbf03091000 1 --2- 172.21.15.86:0/4158079313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5629f8d1fc20 0x5629f8e7dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.314+0000 7fbf03091000 1 -- 172.21.15.86:0/4158079313 >> 172.21.15.86:0/4158079313 conn(0x5629f8943ff0 msgr2=0x5629f8e334c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:36.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.314+0000 7fbf03091000 1 -- 172.21.15.86:0/4158079313 shutdown_connections 2024-04-25T14:09:36.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.314+0000 7fbf03091000 1 -- 172.21.15.86:0/4158079313 wait complete. 2024-04-25T14:09:36.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.314+0000 7fbf03091000 1 -- start start 2024-04-25T14:09:36.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.314+0000 7fbf03091000 1 Processor -- start 2024-04-25T14:09:36.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.314+0000 7fbf03091000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f8904280 0x5629f8e89270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:36.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.314+0000 7fbf03091000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629f8d0bea0 0x5629f8e897b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:36.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.314+0000 7fbf03091000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5629f8d1fc20 0x5629f8e444d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:36.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbf03091000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5629f8c90220 con 0x5629f8904280 2024-04-25T14:09:36.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbf03091000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5629f8c26ea0 con 0x5629f8d0bea0 2024-04-25T14:09:36.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbf03091000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5629f8c7db00 con 0x5629f8d1fc20 2024-04-25T14:09:36.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbf03091000 10 client.0 _ll_register_callbacks cb 0x5629f8c8fdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:36.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbef9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f8904280 0x5629f8e89270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:36.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbef9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f8904280 0x5629f8e89270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58184/0 (socket says 172.21.15.86:58184) 2024-04-25T14:09:36.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbef9000640 1 -- 172.21.15.86:0/2124819387 learned_addr learned my addr 172.21.15.86:0/2124819387 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:36.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbef3e00640 1 --2- 172.21.15.86:0/2124819387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629f8d0bea0 0x5629f8e897b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:36.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbef9a00640 1 --2- 172.21.15.86:0/2124819387 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5629f8d1fc20 0x5629f8e444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:36.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbef9000640 1 -- 172.21.15.86:0/2124819387 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5629f8d1fc20 msgr2=0x5629f8e444d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:36.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbef9000640 1 --2- 172.21.15.86:0/2124819387 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5629f8d1fc20 0x5629f8e444d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbef9000640 1 -- 172.21.15.86:0/2124819387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629f8d0bea0 msgr2=0x5629f8e897b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:36.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbef9000640 1 --2- 172.21.15.86:0/2124819387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629f8d0bea0 0x5629f8e897b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbef9000640 1 -- 172.21.15.86:0/2124819387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629f8c22500 con 0x5629f8904280 2024-04-25T14:09:36.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbef9000640 1 --2- 172.21.15.86:0/2124819387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f8904280 0x5629f8e89270 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fbee40ad040 tx=0x7fbee401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd5c4da43877963 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:36.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbede000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:09:36.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbede000640 1 -- 172.21.15.86:0/2124819387 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee40ce060 con 0x5629f8904280 2024-04-25T14:09:36.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbf03091000 1 -- 172.21.15.86:0/2124819387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbee005fd00 con 0x5629f8904280 2024-04-25T14:09:36.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbede000640 1 -- 172.21.15.86:0/2124819387 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7fbee40c0590 con 0x5629f8904280 2024-04-25T14:09:36.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbede000640 1 -- 172.21.15.86:0/2124819387 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fbee40d4050 con 0x5629f8904280 2024-04-25T14:09:36.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbf03091000 10 client.8088 fetch_fsmap learned FSMap version 76 2024-04-25T14:09:36.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbf03091000 1 -- 172.21.15.86:0/2124819387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x5629f8c08ea0 con 0x5629f8904280 2024-04-25T14:09:36.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbede000640 1 -- 172.21.15.86:0/2124819387 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee005fd00 con 0x5629f8904280 2024-04-25T14:09:36.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbede000640 1 -- 172.21.15.86:0/2124819387 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 76) ==== 3494+0+0 (secure 0 0 0) 0x7fbee40d7020 con 0x5629f8904280 2024-04-25T14:09:36.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbf03091000 10 client.8088 fetch_fsmap finished waiting for FSMap version 76 2024-04-25T14:09:36.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbf03091000 10 client.8088 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T14:09:36.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbf03091000 20 client.8088 populate_metadata read hostname 'smithi086' 2024-04-25T14:09:36.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbf03091000 1 --2- 172.21.15.86:0/2124819387 >> [v2:172.21.15.155:6833/1049360040,v1:172.21.15.155:6835/1049360040] conn(0x5629f89287d0 0x5629f8e45210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:36.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbf03091000 4 client.8088 mds_command: new command op to 7932 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:09:36.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbf03091000 1 -- 172.21.15.86:0/2124819387 --> [v2:172.21.15.155:6833/1049360040,v1:172.21.15.155:6835/1049360040] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629f8c1bbe0 con 0x5629f89287d0 2024-04-25T14:09:36.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbef3e00640 1 --2- 172.21.15.86:0/2124819387 >> [v2:172.21.15.155:6833/1049360040,v1:172.21.15.155:6835/1049360040] conn(0x5629f89287d0 0x5629f8e45210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:36.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbef3e00640 1 --2- 172.21.15.86:0/2124819387 >> [v2:172.21.15.155:6833/1049360040,v1:172.21.15.155:6835/1049360040] conn(0x5629f89287d0 0x5629f8e45210 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b033eb1500507500 server_cookie=961936744a29c0af in_seq=0 out_seq=0 2024-04-25T14:09:36.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.318+0000 7fbede000640 10 client.8088 ms_handle_connect on v2:172.21.15.155:6833/1049360040 2024-04-25T14:09:36.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.322+0000 7fbede000640 1 -- 172.21.15.86:0/2124819387 <== mds.0 v2:172.21.15.155:6833/1049360040 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5629f8c1bbe0 con 0x5629f89287d0 2024-04-25T14:09:36.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.322+0000 7fbede000640 10 client.8088 handle_command_reply: tid=0 2024-04-25T14:09:36.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.322+0000 7fbf03091000 1 client.8088 shutdown 2024-04-25T14:09:36.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.322+0000 7fbf03091000 1 -- 172.21.15.86:0/2124819387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f8904280 msgr2=0x5629f8e89270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:36.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.322+0000 7fbf03091000 1 --2- 172.21.15.86:0/2124819387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f8904280 0x5629f8e89270 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fbee40ad040 tx=0x7fbee401aca0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.322+0000 7fbf03091000 1 -- 172.21.15.86:0/2124819387 shutdown_connections 2024-04-25T14:09:36.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.322+0000 7fbf03091000 1 --2- 172.21.15.86:0/2124819387 >> [v2:172.21.15.155:6833/1049360040,v1:172.21.15.155:6835/1049360040] conn(0x5629f89287d0 0x5629f8e45210 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.322+0000 7fbf03091000 1 --2- 172.21.15.86:0/2124819387 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5629f8d1fc20 0x5629f8e444d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.322+0000 7fbf03091000 1 --2- 172.21.15.86:0/2124819387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629f8d0bea0 0x5629f8e897b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.322+0000 7fbf03091000 1 --2- 172.21.15.86:0/2124819387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f8904280 0x5629f8e89270 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.322+0000 7fbede000640 0 client.8088 ms_handle_reset on v2:172.21.15.155:6833/1049360040 2024-04-25T14:09:36.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.322+0000 7fbf03091000 1 -- 172.21.15.86:0/2124819387 >> 172.21.15.86:0/2124819387 conn(0x5629f8943ff0 msgr2=0x5629f8e33c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:36.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.322+0000 7fbf03091000 1 -- 172.21.15.86:0/2124819387 shutdown_connections 2024-04-25T14:09:36.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.322+0000 7fbf03091000 1 -- 172.21.15.86:0/2124819387 wait complete. 2024-04-25T14:09:36.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.322+0000 7fbf03091000 20 client.8088 trim_cache size 0 max 16384 2024-04-25T14:09:36.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.338+0000 7fbf03091000 1 Processor -- start 2024-04-25T14:09:36.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.338+0000 7fbf03091000 1 -- start start 2024-04-25T14:09:36.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbf03091000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f8904280 0x5629f8e8e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:36.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbf03091000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629f8d0bea0 0x5629f8e98400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:36.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbf03091000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5629f8d1fc20 0x5629f8e98940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:36.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbf03091000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbef41f77f0 con 0x5629f8904280 2024-04-25T14:09:36.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbf03091000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbef40d0480 con 0x5629f8d0bea0 2024-04-25T14:09:36.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbf03091000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbef40d0ad0 con 0x5629f8d1fc20 2024-04-25T14:09:36.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbef9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f8904280 0x5629f8e8e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:36.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbef9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f8904280 0x5629f8e8e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58198/0 (socket says 172.21.15.86:58198) 2024-04-25T14:09:36.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbef9000640 1 -- 172.21.15.86:0/2547577389 learned_addr learned my addr 172.21.15.86:0/2547577389 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:36.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbef9000640 1 -- 172.21.15.86:0/2547577389 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5629f8d1fc20 msgr2=0x5629f8e98940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:36.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbef9000640 1 --2- 172.21.15.86:0/2547577389 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5629f8d1fc20 0x5629f8e98940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbef9000640 1 -- 172.21.15.86:0/2547577389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629f8d0bea0 msgr2=0x5629f8e98400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:36.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbef9000640 1 --2- 172.21.15.86:0/2547577389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629f8d0bea0 0x5629f8e98400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbef9000640 1 -- 172.21.15.86:0/2547577389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629f8e44840 con 0x5629f8904280 2024-04-25T14:09:36.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbef9000640 1 --2- 172.21.15.86:0/2547577389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f8904280 0x5629f8e8e6d0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fbee4000c10 tx=0x7fbee40c5000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf499cfb969f7850 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:36.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbee8c00640 1 -- 172.21.15.86:0/2547577389 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee40c0590 con 0x5629f8904280 2024-04-25T14:09:36.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbee8c00640 1 -- 172.21.15.86:0/2547577389 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee40b0040 con 0x5629f8904280 2024-04-25T14:09:36.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbee8c00640 1 -- 172.21.15.86:0/2547577389 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee40c0d60 con 0x5629f8904280 2024-04-25T14:09:36.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbf03091000 1 -- 172.21.15.86:0/2547577389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f8904280 msgr2=0x5629f8e8e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:36.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbf03091000 1 --2- 172.21.15.86:0/2547577389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f8904280 0x5629f8e8e6d0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fbee4000c10 tx=0x7fbee40c5000 comp rx=0 tx=0).stop 2024-04-25T14:09:36.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbf03091000 1 -- 172.21.15.86:0/2547577389 shutdown_connections 2024-04-25T14:09:36.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbf03091000 1 --2- 172.21.15.86:0/2547577389 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5629f8d1fc20 0x5629f8e98940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbf03091000 1 --2- 172.21.15.86:0/2547577389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629f8d0bea0 0x5629f8e98400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbf03091000 1 --2- 172.21.15.86:0/2547577389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f8904280 0x5629f8e8e6d0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbf03091000 1 -- 172.21.15.86:0/2547577389 >> 172.21.15.86:0/2547577389 conn(0x5629f8943ff0 msgr2=0x5629f8e3b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:36.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbf03091000 1 -- 172.21.15.86:0/2547577389 shutdown_connections 2024-04-25T14:09:36.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbf03091000 1 -- 172.21.15.86:0/2547577389 wait complete. 2024-04-25T14:09:36.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbf03091000 1 -- start start 2024-04-25T14:09:36.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.342+0000 7fbf03091000 1 Processor -- start 2024-04-25T14:09:36.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbf03091000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f8904280 0x5629f8ef4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:36.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbf03091000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5629f8d0bea0 0x5629f8ef50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:36.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbf03091000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629f8d1fc20 0x5629f8e8cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:36.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbf03091000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5629f8c82130 con 0x5629f8904280 2024-04-25T14:09:36.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbf03091000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5629f8c187d0 con 0x5629f8d1fc20 2024-04-25T14:09:36.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbf03091000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5629f8c87450 con 0x5629f8d0bea0 2024-04-25T14:09:36.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbf03091000 10 client.0 _ll_register_callbacks cb 0x7fbee40090c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:36.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbef9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f8904280 0x5629f8ef4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:36.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbef9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f8904280 0x5629f8ef4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58212/0 (socket says 172.21.15.86:58212) 2024-04-25T14:09:36.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbef9000640 1 -- 172.21.15.86:0/3838162998 learned_addr learned my addr 172.21.15.86:0/3838162998 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:36.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbef3e00640 1 --2- 172.21.15.86:0/3838162998 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5629f8d0bea0 0x5629f8ef50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:36.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbef9a00640 1 --2- 172.21.15.86:0/3838162998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629f8d1fc20 0x5629f8e8cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:36.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbef9000640 1 -- 172.21.15.86:0/3838162998 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5629f8d0bea0 msgr2=0x5629f8ef50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:36.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbef9000640 1 --2- 172.21.15.86:0/3838162998 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5629f8d0bea0 0x5629f8ef50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbef9000640 1 -- 172.21.15.86:0/3838162998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629f8d1fc20 msgr2=0x5629f8e8cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:36.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbef9000640 1 --2- 172.21.15.86:0/3838162998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629f8d1fc20 0x5629f8e8cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbef9000640 1 -- 172.21.15.86:0/3838162998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629f8c3bc70 con 0x5629f8904280 2024-04-25T14:09:36.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbef9000640 1 --2- 172.21.15.86:0/3838162998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f8904280 0x5629f8ef4bb0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fbee40ad040 tx=0x7fbee4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab9aa5a5bd94361 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:36.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbede000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:09:36.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbede000640 1 -- 172.21.15.86:0/3838162998 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee40dc070 con 0x5629f8904280 2024-04-25T14:09:36.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbede000640 1 -- 172.21.15.86:0/3838162998 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7fbee40b5c30 con 0x5629f8904280 2024-04-25T14:09:36.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbf03091000 1 -- 172.21.15.86:0/3838162998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbee40b0040 con 0x5629f8904280 2024-04-25T14:09:36.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbede000640 1 -- 172.21.15.86:0/3838162998 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fbee40c9820 con 0x5629f8904280 2024-04-25T14:09:36.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbf03091000 10 client.8094 fetch_fsmap learned FSMap version 76 2024-04-25T14:09:36.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbf03091000 1 -- 172.21.15.86:0/3838162998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x5629f8d05180 con 0x5629f8904280 2024-04-25T14:09:36.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbede000640 1 -- 172.21.15.86:0/3838162998 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee40b0040 con 0x5629f8904280 2024-04-25T14:09:36.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbede000640 1 -- 172.21.15.86:0/3838162998 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 76) ==== 3494+0+0 (secure 0 0 0) 0x7fbee40d7070 con 0x5629f8904280 2024-04-25T14:09:36.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbf03091000 10 client.8094 fetch_fsmap finished waiting for FSMap version 76 2024-04-25T14:09:36.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbf03091000 10 client.8094 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T14:09:36.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbf03091000 20 client.8094 populate_metadata read hostname 'smithi086' 2024-04-25T14:09:36.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbf03091000 1 --2- 172.21.15.86:0/3838162998 >> [v2:172.21.15.155:6833/1049360040,v1:172.21.15.155:6835/1049360040] conn(0x5629f8916830 0x5629f8e8d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:36.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbf03091000 4 client.8094 mds_command: new command op to 7932 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:09:36.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbf03091000 1 -- 172.21.15.86:0/3838162998 --> [v2:172.21.15.155:6833/1049360040,v1:172.21.15.155:6835/1049360040] -- command(tid 0: {"prefix": "status"}) -- 0x5629f8c90a30 con 0x5629f8916830 2024-04-25T14:09:36.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbef3e00640 1 --2- 172.21.15.86:0/3838162998 >> [v2:172.21.15.155:6833/1049360040,v1:172.21.15.155:6835/1049360040] conn(0x5629f8916830 0x5629f8e8d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:36.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbede000640 10 client.8094 ms_handle_connect on v2:172.21.15.155:6833/1049360040 2024-04-25T14:09:36.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.346+0000 7fbef3e00640 1 --2- 172.21.15.86:0/3838162998 >> [v2:172.21.15.155:6833/1049360040,v1:172.21.15.155:6835/1049360040] conn(0x5629f8916830 0x5629f8e8d750 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71f9dcabce278ae0 server_cookie=8d2a174a58183761 in_seq=0 out_seq=0 2024-04-25T14:09:36.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.350+0000 7fbede000640 1 -- 172.21.15.86:0/3838162998 <== mds.0 v2:172.21.15.155:6833/1049360040 1 ==== command_reply(tid 0: 0 ) ==== 8+0+320 (crc 0 0 0) 0x5629f8c90a30 con 0x5629f8916830 2024-04-25T14:09:36.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.350+0000 7fbede000640 10 client.8094 handle_command_reply: tid=0 2024-04-25T14:09:36.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.350+0000 7fbf03091000 1 client.8094 shutdown 2024-04-25T14:09:36.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.350+0000 7fbf03091000 1 -- 172.21.15.86:0/3838162998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f8904280 msgr2=0x5629f8ef4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:36.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.350+0000 7fbf03091000 1 --2- 172.21.15.86:0/3838162998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f8904280 0x5629f8ef4bb0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fbee40ad040 tx=0x7fbee4008040 comp rx=0 tx=0).stop 2024-04-25T14:09:36.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.350+0000 7fbf03091000 1 -- 172.21.15.86:0/3838162998 shutdown_connections 2024-04-25T14:09:36.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.350+0000 7fbf03091000 1 --2- 172.21.15.86:0/3838162998 >> [v2:172.21.15.155:6833/1049360040,v1:172.21.15.155:6835/1049360040] conn(0x5629f8916830 0x5629f8e8d750 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.350+0000 7fbf03091000 1 --2- 172.21.15.86:0/3838162998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629f8d1fc20 0x5629f8e8cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.350+0000 7fbf03091000 1 --2- 172.21.15.86:0/3838162998 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5629f8d0bea0 0x5629f8ef50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.350+0000 7fbede000640 0 client.8094 ms_handle_reset on v2:172.21.15.155:6833/1049360040 2024-04-25T14:09:36.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.350+0000 7fbf03091000 1 --2- 172.21.15.86:0/3838162998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f8904280 0x5629f8ef4bb0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.350+0000 7fbf03091000 1 -- 172.21.15.86:0/3838162998 >> 172.21.15.86:0/3838162998 conn(0x5629f8943ff0 msgr2=0x5629f8e87420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:36.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.350+0000 7fbf03091000 1 -- 172.21.15.86:0/3838162998 shutdown_connections 2024-04-25T14:09:36.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.350+0000 7fbf03091000 1 -- 172.21.15.86:0/3838162998 wait complete. 2024-04-25T14:09:36.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.350+0000 7fbf03091000 20 client.8094 trim_cache size 0 max 16384 2024-04-25T14:09:36.349 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:09:36.349 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb", 2024-04-25T14:09:36.349 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-25T14:09:36.349 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 7932, 2024-04-25T14:09:36.349 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T14:09:36.349 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T14:09:36.349 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T14:09:36.349 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 10.51808275, 2024-04-25T14:09:36.349 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 76, 2024-04-25T14:09:36.350 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 100, 2024-04-25T14:09:36.350 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 100, 2024-04-25T14:09:36.350 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 15.340737673 2024-04-25T14:09:36.350 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:09:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.350+0000 7fbefba00640 1 -- 172.21.15.86:0/1862670575 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fbed406f550 msgr2=0x7fbed4071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.350+0000 7fbefba00640 1 --2- 172.21.15.86:0/1862670575 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fbed406f550 0x7fbed4071a60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbee40af000 tx=0x7fbee40a7000 comp rx=0 tx=0).stop 2024-04-25T14:09:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.350+0000 7fbefba00640 1 -- 172.21.15.86:0/1862670575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbef4070330 msgr2=0x7fbef4147780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.350+0000 7fbefba00640 1 --2- 172.21.15.86:0/1862670575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbef4070330 0x7fbef4147780 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fbee000e8a0 tx=0x7fbee000eca0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.350+0000 7fbefba00640 1 -- 172.21.15.86:0/1862670575 shutdown_connections 2024-04-25T14:09:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.350+0000 7fbefba00640 1 --2- 172.21.15.86:0/1862670575 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fbed406f550 0x7fbed4071a60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.350+0000 7fbefba00640 1 --2- 172.21.15.86:0/1862670575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbef40d1c20 0x7fbef4147cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.350+0000 7fbefba00640 1 --2- 172.21.15.86:0/1862670575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbef4070330 0x7fbef4147780 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.350+0000 7fbefba00640 1 --2- 172.21.15.86:0/1862670575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbef406f270 0x7fbef409aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.350+0000 7fbefba00640 1 -- 172.21.15.86:0/1862670575 >> 172.21.15.86:0/1862670575 conn(0x7fbef40b4650 msgr2=0x7fbef415b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.350+0000 7fbefba00640 1 -- 172.21.15.86:0/1862670575 shutdown_connections 2024-04-25T14:09:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.350+0000 7fbefba00640 1 -- 172.21.15.86:0/1862670575 wait complete. 2024-04-25T14:09:36.366 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T14:09:36.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.518+0000 7f8e6d600640 1 Processor -- start 2024-04-25T14:09:36.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.518+0000 7f8e6d600640 1 -- start start 2024-04-25T14:09:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.518+0000 7f8e6d600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8e680cfca0 0x7f8e6806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.518+0000 7f8e6d600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e680730b0 0x7f8e680734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.518+0000 7f8e6d600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e68072bd0 con 0x7f8e680d16e0 2024-04-25T14:09:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.518+0000 7f8e6d600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8e68070470 con 0x7f8e680cfca0 2024-04-25T14:09:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.518+0000 7f8e6d600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8e6806f930 con 0x7f8e680730b0 2024-04-25T14:09:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.518+0000 7f8e66a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8e680cfca0 0x7f8e6806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.518+0000 7f8e5e000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8e680d16e0 0x7f8e680d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42908/0 (socket says 172.21.15.86:42908) 2024-04-25T14:09:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.518+0000 7f8e66a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8e680cfca0 0x7f8e6806f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58214/0 (socket says 172.21.15.86:58214) 2024-04-25T14:09:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.518+0000 7f8e5e000640 1 -- 172.21.15.86:0/3779280399 learned_addr learned my addr 172.21.15.86:0/3779280399 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.518+0000 7f8e67400640 1 --2- 172.21.15.86:0/3779280399 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e680730b0 0x7f8e680734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.518+0000 7f8e66a00640 1 -- 172.21.15.86:0/3779280399 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e680730b0 msgr2=0x7f8e680734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.518+0000 7f8e66a00640 1 --2- 172.21.15.86:0/3779280399 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e680730b0 0x7f8e680734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.518+0000 7f8e66a00640 1 -- 172.21.15.86:0/3779280399 >> v1:172.21.15.86:6789/0 conn(0x7f8e680d16e0 legacy=0x7f8e680d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:36.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.518+0000 7f8e66a00640 1 -- 172.21.15.86:0/3779280399 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e6806fbd0 con 0x7f8e680cfca0 2024-04-25T14:09:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.522+0000 7f8e66a00640 1 --2- 172.21.15.86:0/3779280399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e680cfca0 0x7f8e6806f3f0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800ba60 tx=0x7f8e5801eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28cf0debbffa35bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.522+0000 7f8e66000640 1 -- 172.21.15.86:0/3779280399 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e580b4070 con 0x7f8e680cfca0 2024-04-25T14:09:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.522+0000 7f8e66000640 1 -- 172.21.15.86:0/3779280399 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e580a7b50 con 0x7f8e680cfca0 2024-04-25T14:09:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.522+0000 7f8e66000640 1 -- 172.21.15.86:0/3779280399 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e580af440 con 0x7f8e680cfca0 2024-04-25T14:09:36.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.522+0000 7f8e6d600640 1 -- 172.21.15.86:0/3779280399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e680cfca0 msgr2=0x7f8e6806f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:36.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.522+0000 7f8e6d600640 1 --2- 172.21.15.86:0/3779280399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e680cfca0 0x7f8e6806f3f0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800ba60 tx=0x7f8e5801eb70 comp rx=0 tx=0).stop 2024-04-25T14:09:36.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.522+0000 7f8e6d600640 1 -- 172.21.15.86:0/3779280399 shutdown_connections 2024-04-25T14:09:36.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.522+0000 7f8e6d600640 1 --2- 172.21.15.86:0/3779280399 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e680730b0 0x7f8e680734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.522+0000 7f8e6d600640 1 --2- 172.21.15.86:0/3779280399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e680cfca0 0x7f8e6806f3f0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.522+0000 7f8e6d600640 1 -- 172.21.15.86:0/3779280399 >> 172.21.15.86:0/3779280399 conn(0x7f8e680b4650 msgr2=0x7f8e680b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:36.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.522+0000 7f8e6d600640 1 -- 172.21.15.86:0/3779280399 shutdown_connections 2024-04-25T14:09:36.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.522+0000 7f8e6d600640 1 -- 172.21.15.86:0/3779280399 wait complete. 2024-04-25T14:09:36.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.526+0000 7f8e6d600640 1 Processor -- start 2024-04-25T14:09:36.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.526+0000 7f8e6d600640 1 -- start start 2024-04-25T14:09:36.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.526+0000 7f8e6d600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e680730b0 0x7f8e6809e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:36.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.526+0000 7f8e6d600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e680cfca0 0x7f8e6809b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:36.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.526+0000 7f8e6d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e680d16e0 0x7f8e6809b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:36.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.526+0000 7f8e6d600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e680ce0f0 con 0x7f8e680d16e0 2024-04-25T14:09:36.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.526+0000 7f8e6d600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8e680cdf70 con 0x7f8e680730b0 2024-04-25T14:09:36.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.526+0000 7f8e6d600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8e680ce270 con 0x7f8e680cfca0 2024-04-25T14:09:36.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.526+0000 7f8e67400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e680d16e0 0x7f8e6809b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:36.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.526+0000 7f8e67400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e680d16e0 0x7f8e6809b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58218/0 (socket says 172.21.15.86:58218) 2024-04-25T14:09:36.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.526+0000 7f8e67400640 1 -- 172.21.15.86:0/2110737870 learned_addr learned my addr 172.21.15.86:0/2110737870 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:36.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.526+0000 7f8e66a00640 1 --2- 172.21.15.86:0/2110737870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e680730b0 0x7f8e6809e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:36.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.526+0000 7f8e5e000640 1 --2- 172.21.15.86:0/2110737870 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e680cfca0 0x7f8e6809b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:36.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.526+0000 7f8e67400640 1 -- 172.21.15.86:0/2110737870 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e680cfca0 msgr2=0x7f8e6809b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:36.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.526+0000 7f8e67400640 1 --2- 172.21.15.86:0/2110737870 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e680cfca0 0x7f8e6809b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.526+0000 7f8e67400640 1 -- 172.21.15.86:0/2110737870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e680730b0 msgr2=0x7f8e6809e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:36.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.526+0000 7f8e67400640 1 --2- 172.21.15.86:0/2110737870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e680730b0 0x7f8e6809e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.526+0000 7f8e67400640 1 -- 172.21.15.86:0/2110737870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e6815f5e0 con 0x7f8e680d16e0 2024-04-25T14:09:36.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.526+0000 7f8e66a00640 1 --2- 172.21.15.86:0/2110737870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e680730b0 0x7f8e6809e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:36.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.526+0000 7f8e67400640 1 --2- 172.21.15.86:0/2110737870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e680d16e0 0x7f8e6809b7f0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f8e60007d80 tx=0x7f8e6000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:36.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.526+0000 7f8e5f400640 1 -- 172.21.15.86:0/2110737870 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e60014420 con 0x7f8e680d16e0 2024-04-25T14:09:36.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.526+0000 7f8e5f400640 1 -- 172.21.15.86:0/2110737870 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e60019070 con 0x7f8e680d16e0 2024-04-25T14:09:36.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.526+0000 7f8e6d600640 1 -- 172.21.15.86:0/2110737870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e6815dc30 con 0x7f8e680d16e0 2024-04-25T14:09:36.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.526+0000 7f8e5f400640 1 -- 172.21.15.86:0/2110737870 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e60014800 con 0x7f8e680d16e0 2024-04-25T14:09:36.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.530+0000 7f8e6d600640 1 -- 172.21.15.86:0/2110737870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e6815e1b0 con 0x7f8e680d16e0 2024-04-25T14:09:36.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.530+0000 7f8e6d600640 1 -- 172.21.15.86:0/2110737870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e6809f4b0 con 0x7f8e680d16e0 2024-04-25T14:09:36.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.530+0000 7f8e5f400640 1 -- 172.21.15.86:0/2110737870 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7f8e600149a0 con 0x7f8e680d16e0 2024-04-25T14:09:36.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.530+0000 7f8e5f400640 1 --2- 172.21.15.86:0/2110737870 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f8e4406f550 0x7f8e44071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:36.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.530+0000 7f8e5f400640 1 -- 172.21.15.86:0/2110737870 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7f8e6008fd20 con 0x7f8e680d16e0 2024-04-25T14:09:36.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.530+0000 7f8e66a00640 1 --2- 172.21.15.86:0/2110737870 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f8e4406f550 0x7f8e44071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:36.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.530+0000 7f8e66a00640 1 --2- 172.21.15.86:0/2110737870 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f8e4406f550 0x7f8e44071a60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8e580b1000 tx=0x7f8e580a9070 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:36.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.534+0000 7f8e5f400640 1 -- 172.21.15.86:0/2110737870 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8e6005ca00 con 0x7f8e680d16e0 2024-04-25T14:09:36.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.726+0000 7f8e6d600640 1 -- 172.21.15.86:0/2110737870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8e68069d00 con 0x7f8e680d16e0 2024-04-25T14:09:36.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.726+0000 7f8e5f400640 1 -- 172.21.15.86:0/2110737870 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v76) ==== 53+0+73 (secure 0 0 0) 0x7f8e6005c300 con 0x7f8e680d16e0 2024-04-25T14:09:36.725 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T14:09:36.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.730+0000 7f8e6d600640 1 -- 172.21.15.86:0/2110737870 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f8e4406f550 msgr2=0x7f8e44071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:36.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.730+0000 7f8e6d600640 1 --2- 172.21.15.86:0/2110737870 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f8e4406f550 0x7f8e44071a60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8e580b1000 tx=0x7f8e580a9070 comp rx=0 tx=0).stop 2024-04-25T14:09:36.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.730+0000 7f8e6d600640 1 -- 172.21.15.86:0/2110737870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e680d16e0 msgr2=0x7f8e6809b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:36.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.730+0000 7f8e6d600640 1 --2- 172.21.15.86:0/2110737870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e680d16e0 0x7f8e6809b7f0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f8e60007d80 tx=0x7f8e6000e4b0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.730+0000 7f8e6d600640 1 -- 172.21.15.86:0/2110737870 shutdown_connections 2024-04-25T14:09:36.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.730+0000 7f8e6d600640 1 --2- 172.21.15.86:0/2110737870 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f8e4406f550 0x7f8e44071a60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.730+0000 7f8e6d600640 1 --2- 172.21.15.86:0/2110737870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e680d16e0 0x7f8e6809b7f0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.730+0000 7f8e6d600640 1 --2- 172.21.15.86:0/2110737870 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e680cfca0 0x7f8e6809b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.730+0000 7f8e6d600640 1 --2- 172.21.15.86:0/2110737870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e680730b0 0x7f8e6809e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:36.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.730+0000 7f8e6d600640 1 -- 172.21.15.86:0/2110737870 >> 172.21.15.86:0/2110737870 conn(0x7f8e680b4650 msgr2=0x7f8e680b22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:36.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.730+0000 7f8e6d600640 1 -- 172.21.15.86:0/2110737870 shutdown_connections 2024-04-25T14:09:36.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36.730+0000 7f8e6d600640 1 -- 172.21.15.86:0/2110737870 wait complete. 2024-04-25T14:09:36.747 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T14:09:36.747 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T14:09:36.747 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-25T14:09:36.747 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T14:09:36.747 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T14:09:36.747 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T14:09:36.747 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T14:09:36.747 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T14:09:36.747 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:09:36.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:09:36.747 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-25T14:09:36.756 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:09:36.756 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:09:36.756 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:09:36.757 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:09:36.757 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-25T14:09:36.757 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:09:36.757 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:09:36.757 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:09:36.757 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-25T14:09:36.757 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:09:36.757 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:09:36.757 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-25T14:09:36.757 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:09:36.757 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:09:36.757 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T14:09:36.757 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:09:36.758 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T14:09:36.758 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42088sec preferred_lft 42088sec 2024-04-25T14:09:36.758 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-25T14:09:36.758 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:09:36.758 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T14:09:36.758 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:dc:87:9c:96 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:09:36.758 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T14:09:36.758 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:09:36.758 INFO:teuthology.orchestra.run.smithi086.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T14:09:36.758 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T14:09:36.758 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T14:09:36.758 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:09:36.758 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T14:09:36.759 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:09:36.759 INFO:teuthology.orchestra.run.smithi086.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:09:36.759 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T14:09:36.759 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T14:09:36.759 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:09:36.759 INFO:teuthology.orchestra.run.smithi086.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:09:36.759 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T14:09:36.759 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T14:09:36.759 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:09:36.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:09:36.760 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T14:09:36.818 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-25T14:09:36.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:09:36.821 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-25T14:09:36.872 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-25T14:09:36.872 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T14:09:36.872 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T14:09:36.872 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T14:09:36.872 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T14:09:36.872 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T14:09:36.872 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T14:09:36.872 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:09:36.872 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:09:36.872 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:09:36.872 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T14:09:36.873 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:09:36.873 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T14:09:36.873 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T14:09:36.873 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T14:09:36.873 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T14:09:36.873 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T14:09:36.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:09:36.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:09:37.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:09:37.006 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T14:09:37.015 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T14:09:37.015 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T14:09:37.015 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:09:37.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:09:37.015 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-25T14:09:37.076 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:09:37.076 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T14:09:37.125 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:09:37.125 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:09:37.125 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T14:09:37.172 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T14:09:37.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:09:37.172 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:09:37.272 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:09:37.272 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T14:09:37.273 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:09:37.290 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:09:37.290 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T14:09:37.290 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:09:37.291 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:09:37.291 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:09:37.301 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:09:37.301 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T14:09:37.301 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:09:37.301 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:09:37.301 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:09:37.301 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790" 2024-04-25T14:09:37.301 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790:/ 2024-04-25T14:09:37.301 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb" will pass to kernel 2024-04-25T14:09:37.305 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T14:09:37.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:09:37.305 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T14:09:37.326 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:09:37.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:09:37.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:09:37.444 INFO:teuthology.orchestra.run.smithi086.stdout:client6887 2024-04-25T14:09:37.445 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:09:37.445 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:09:37.454 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T14:09:37.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.646+0000 7fbcfe200640 1 Processor -- start 2024-04-25T14:09:37.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.646+0000 7fbcfe200640 1 -- start start 2024-04-25T14:09:37.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.646+0000 7fbcfe200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbcf80b7170 0x7fbcf80b7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:37.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.646+0000 7fbcfe200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcf806da40 0x7fbcf806de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:37.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.646+0000 7fbcfe200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbcf80b4160 0x7fbcf80b4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:37.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.646+0000 7fbcfe200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbcf80b7ac0 con 0x7fbcf80b7170 2024-04-25T14:09:37.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.646+0000 7fbcfe200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fbcf81639a0 con 0x7fbcf80b4160 2024-04-25T14:09:37.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.646+0000 7fbcfe200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf806e730 con 0x7fbcf806da40 2024-04-25T14:09:37.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.646+0000 7fbcf7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbcf80b7170 0x7fbcf80b7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:37.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.646+0000 7fbcf7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbcf80b7170 0x7fbcf80b7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58224/0 (socket says 172.21.15.86:58224) 2024-04-25T14:09:37.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.646+0000 7fbcf7400640 1 -- 172.21.15.86:0/3033051973 learned_addr learned my addr 172.21.15.86:0/3033051973 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:37.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.646+0000 7fbcf7e00640 1 --2- 172.21.15.86:0/3033051973 >> v2:172.21.15.155:3300/0 conn(0x7fbcf80b4160 0x7fbcf80b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:37.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.646+0000 7fbcf6a00640 1 --2- 172.21.15.86:0/3033051973 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcf806da40 0x7fbcf806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:37.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.646+0000 7fbcf7400640 1 -- 172.21.15.86:0/3033051973 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcf806da40 msgr2=0x7fbcf806de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:37.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.646+0000 7fbcf7400640 1 --2- 172.21.15.86:0/3033051973 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcf806da40 0x7fbcf806de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:37.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.646+0000 7fbcf7400640 1 -- 172.21.15.86:0/3033051973 >> v2:172.21.15.155:3300/0 conn(0x7fbcf80b4160 msgr2=0x7fbcf80b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:37.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.650+0000 7fbcf7400640 1 --2- 172.21.15.86:0/3033051973 >> v2:172.21.15.155:3300/0 conn(0x7fbcf80b4160 0x7fbcf80b4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:37.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.650+0000 7fbcf7400640 1 -- 172.21.15.86:0/3033051973 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf8163c40 con 0x7fbcf80b7170 2024-04-25T14:09:37.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.650+0000 7fbcf7e00640 1 --2- 172.21.15.86:0/3033051973 >> v2:172.21.15.155:3300/0 conn(0x7fbcf80b4160 0x7fbcf80b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:37.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.650+0000 7fbcf7400640 1 --2- 172.21.15.86:0/3033051973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcf80b7170 0x7fbcf80b7580 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fbcf808eb90 tx=0x7fbce801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c83b4f863ae3ab4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:37.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.650+0000 7fbcf6000640 1 -- 172.21.15.86:0/3033051973 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce80b1070 con 0x7fbcf80b7170 2024-04-25T14:09:37.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.650+0000 7fbcf6000640 1 -- 172.21.15.86:0/3033051973 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbce80ac630 con 0x7fbcf80b7170 2024-04-25T14:09:37.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.650+0000 7fbcf6000640 1 -- 172.21.15.86:0/3033051973 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce80a8050 con 0x7fbcf80b7170 2024-04-25T14:09:37.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.650+0000 7fbcfe200640 1 -- 172.21.15.86:0/3033051973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcf80b7170 msgr2=0x7fbcf80b7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:37.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.650+0000 7fbcfe200640 1 --2- 172.21.15.86:0/3033051973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcf80b7170 0x7fbcf80b7580 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fbcf808eb90 tx=0x7fbce801c520 comp rx=0 tx=0).stop 2024-04-25T14:09:37.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.654+0000 7fbcfe200640 1 -- 172.21.15.86:0/3033051973 shutdown_connections 2024-04-25T14:09:37.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.654+0000 7fbcfe200640 1 --2- 172.21.15.86:0/3033051973 >> v2:172.21.15.155:3300/0 conn(0x7fbcf80b4160 0x7fbcf80b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:37.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.654+0000 7fbcfe200640 1 --2- 172.21.15.86:0/3033051973 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcf806da40 0x7fbcf806de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:37.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.654+0000 7fbcfe200640 1 --2- 172.21.15.86:0/3033051973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcf80b7170 0x7fbcf80b7580 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:37.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.654+0000 7fbcfe200640 1 -- 172.21.15.86:0/3033051973 >> 172.21.15.86:0/3033051973 conn(0x7fbcf80d2340 msgr2=0x7fbcf80d2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:37.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.654+0000 7fbcfe200640 1 -- 172.21.15.86:0/3033051973 shutdown_connections 2024-04-25T14:09:37.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.654+0000 7fbcfe200640 1 -- 172.21.15.86:0/3033051973 wait complete. 2024-04-25T14:09:37.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.654+0000 7fbcfe200640 1 Processor -- start 2024-04-25T14:09:37.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.654+0000 7fbcfe200640 1 -- start start 2024-04-25T14:09:37.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.654+0000 7fbcfe200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcf806da40 0x7fbcf8159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:37.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.654+0000 7fbcfe200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcf80b4160 0x7fbcf81590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:37.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.654+0000 7fbcfe200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcf80b7170 0x7fbcf8159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:37.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.654+0000 7fbcfe200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf806e510 con 0x7fbcf806da40 2024-04-25T14:09:37.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.654+0000 7fbcfe200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf806e390 con 0x7fbcf80b7170 2024-04-25T14:09:37.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.654+0000 7fbcfe200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf8164720 con 0x7fbcf80b4160 2024-04-25T14:09:37.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.658+0000 7fbcf7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcf806da40 0x7fbcf8159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:37.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.658+0000 7fbcf7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcf806da40 0x7fbcf8159d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58230/0 (socket says 172.21.15.86:58230) 2024-04-25T14:09:37.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.658+0000 7fbcf7400640 1 -- 172.21.15.86:0/2203597946 learned_addr learned my addr 172.21.15.86:0/2203597946 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:37.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.658+0000 7fbcf6a00640 1 --2- 172.21.15.86:0/2203597946 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcf80b4160 0x7fbcf81590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:37.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.658+0000 7fbcf7e00640 1 --2- 172.21.15.86:0/2203597946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcf80b7170 0x7fbcf8159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:37.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.658+0000 7fbcf7400640 1 -- 172.21.15.86:0/2203597946 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcf80b4160 msgr2=0x7fbcf81590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:37.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.658+0000 7fbcf7400640 1 --2- 172.21.15.86:0/2203597946 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcf80b4160 0x7fbcf81590d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:37.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.658+0000 7fbcf7400640 1 -- 172.21.15.86:0/2203597946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcf80b7170 msgr2=0x7fbcf8159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:37.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.658+0000 7fbcf7400640 1 --2- 172.21.15.86:0/2203597946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcf80b7170 0x7fbcf8159610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:37.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.658+0000 7fbcf7400640 1 -- 172.21.15.86:0/2203597946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf80d04b0 con 0x7fbcf806da40 2024-04-25T14:09:37.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.658+0000 7fbcf7e00640 1 --2- 172.21.15.86:0/2203597946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcf80b7170 0x7fbcf8159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:37.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.658+0000 7fbcf7400640 1 --2- 172.21.15.86:0/2203597946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcf806da40 0x7fbcf8159d50 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fbce80af040 tx=0x7fbce80192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:37.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.658+0000 7fbcef400640 1 -- 172.21.15.86:0/2203597946 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce80b1070 con 0x7fbcf806da40 2024-04-25T14:09:37.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.658+0000 7fbcef400640 1 -- 172.21.15.86:0/2203597946 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbce80ab5e0 con 0x7fbcf806da40 2024-04-25T14:09:37.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.658+0000 7fbcfe200640 1 -- 172.21.15.86:0/2203597946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf8157920 con 0x7fbcf806da40 2024-04-25T14:09:37.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.658+0000 7fbcef400640 1 -- 172.21.15.86:0/2203597946 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce80a8040 con 0x7fbcf806da40 2024-04-25T14:09:37.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.658+0000 7fbcfe200640 1 -- 172.21.15.86:0/2203597946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf814a7f0 con 0x7fbcf806da40 2024-04-25T14:09:37.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.658+0000 7fbcfe200640 1 -- 172.21.15.86:0/2203597946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcf814a300 con 0x7fbcf806da40 2024-04-25T14:09:37.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.658+0000 7fbcef400640 1 -- 172.21.15.86:0/2203597946 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7fbce80be060 con 0x7fbcf806da40 2024-04-25T14:09:37.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.662+0000 7fbcef400640 1 --2- 172.21.15.86:0/2203597946 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fbcd806f550 0x7fbcd8071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:37.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.662+0000 7fbcef400640 1 -- 172.21.15.86:0/2203597946 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7fbce8128bd0 con 0x7fbcf806da40 2024-04-25T14:09:37.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.662+0000 7fbcf6a00640 1 --2- 172.21.15.86:0/2203597946 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fbcd806f550 0x7fbcd8071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:37.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.662+0000 7fbcf6a00640 1 --2- 172.21.15.86:0/2203597946 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fbcd806f550 0x7fbcd8071a60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbcf814ce10 tx=0x7fbce4008040 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:37.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.662+0000 7fbcef400640 1 -- 172.21.15.86:0/2203597946 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbce8016110 con 0x7fbcf806da40 2024-04-25T14:09:37.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.854+0000 7fbcfe200640 1 -- 172.21.15.86:0/2203597946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbcf806ee50 con 0x7fbcf806da40 2024-04-25T14:09:37.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.854+0000 7fbcef400640 1 -- 172.21.15.86:0/2203597946 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v76) ==== 53+0+73 (secure 0 0 0) 0x7fbce80f5920 con 0x7fbcf806da40 2024-04-25T14:09:37.853 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T14:09:37.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.858+0000 7fbcfe200640 1 -- 172.21.15.86:0/2203597946 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fbcd806f550 msgr2=0x7fbcd8071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:37.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.858+0000 7fbcfe200640 1 --2- 172.21.15.86:0/2203597946 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fbcd806f550 0x7fbcd8071a60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbcf814ce10 tx=0x7fbce4008040 comp rx=0 tx=0).stop 2024-04-25T14:09:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.858+0000 7fbcfe200640 1 -- 172.21.15.86:0/2203597946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcf806da40 msgr2=0x7fbcf8159d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.858+0000 7fbcfe200640 1 --2- 172.21.15.86:0/2203597946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcf806da40 0x7fbcf8159d50 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fbce80af040 tx=0x7fbce80192c0 comp rx=0 tx=0).stop 2024-04-25T14:09:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.858+0000 7fbcfe200640 1 -- 172.21.15.86:0/2203597946 shutdown_connections 2024-04-25T14:09:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.858+0000 7fbcfe200640 1 --2- 172.21.15.86:0/2203597946 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fbcd806f550 0x7fbcd8071a60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.858+0000 7fbcfe200640 1 --2- 172.21.15.86:0/2203597946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcf80b7170 0x7fbcf8159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.858+0000 7fbcfe200640 1 --2- 172.21.15.86:0/2203597946 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcf80b4160 0x7fbcf81590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.858+0000 7fbcfe200640 1 --2- 172.21.15.86:0/2203597946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcf806da40 0x7fbcf8159d50 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.858+0000 7fbcfe200640 1 -- 172.21.15.86:0/2203597946 >> 172.21.15.86:0/2203597946 conn(0x7fbcf80d2340 msgr2=0x7fbcf80ba5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.858+0000 7fbcfe200640 1 -- 172.21.15.86:0/2203597946 shutdown_connections 2024-04-25T14:09:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:37.858+0000 7fbcfe200640 1 -- 172.21.15.86:0/2203597946 wait complete. 2024-04-25T14:09:37.875 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T14:09:37.875 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T14:09:37.875 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2024-04-25T14:09:37.875 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T14:09:37.875 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T14:09:37.875 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T14:09:37.876 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T14:09:37.876 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T14:09:37.876 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:09:37.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:09:37.876 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2024-04-25T14:09:37.886 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:09:37.886 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:09:37.886 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:09:37.886 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:09:37.886 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2024-04-25T14:09:37.886 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:09:37.886 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:09:37.886 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2024-04-25T14:09:37.887 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f0 2024-04-25T14:09:37.887 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:09:37.887 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2024-04-25T14:09:37.887 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f1 2024-04-25T14:09:37.887 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:09:37.887 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2024-04-25T14:09:37.887 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T14:09:37.887 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2024-04-25T14:09:37.887 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T14:09:37.887 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 42080sec preferred_lft 42080sec 2024-04-25T14:09:37.887 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::ec4:7aff:fe88:6ded/64 scope link 2024-04-25T14:09:37.887 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:09:37.887 INFO:teuthology.orchestra.run.smithi155.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T14:09:37.887 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 02:42:b1:ff:60:50 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:09:37.887 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T14:09:37.887 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:09:37.888 INFO:teuthology.orchestra.run.smithi155.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T14:09:37.888 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T14:09:37.888 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T14:09:37.888 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:09:37.888 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T14:09:37.888 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:09:37.888 INFO:teuthology.orchestra.run.smithi155.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:09:37.888 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T14:09:37.888 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T14:09:37.888 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:09:37.888 INFO:teuthology.orchestra.run.smithi155.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:09:37.888 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T14:09:37.888 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T14:09:37.888 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:09:37.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:09:37.889 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T14:09:37.952 INFO:teuthology.orchestra.run.smithi155.stdout:1 2024-04-25T14:09:37.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:09:37.955 DEBUG:teuthology.orchestra.run.smithi155:> route 2024-04-25T14:09:38.009 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2024-04-25T14:09:38.009 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T14:09:38.009 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T14:09:38.009 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T14:09:38.009 INFO:teuthology.orchestra.run.smithi155.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T14:09:38.009 INFO:teuthology.orchestra.run.smithi155.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T14:09:38.009 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T14:09:38.009 INFO:teuthology.orchestra.run.smithi155.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:09:38.009 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:09:38.009 INFO:teuthology.orchestra.run.smithi155.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:09:38.009 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T14:09:38.010 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:09:38.010 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-04-25T14:09:38.010 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T14:09:38.010 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T14:09:38.010 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T14:09:38.010 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-04-25T14:09:38.105 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:09:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:09:38.149 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:09:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:09:38.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:09:38.156 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2024-04-25T14:09:38.164 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T14:09:38.164 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T14:09:38.165 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:09:38.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:09:38.165 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.0 up 2024-04-25T14:09:38.228 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:09:38.228 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T14:09:38.276 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:09:38.277 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:09:38.277 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T14:09:38.328 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T14:09:38.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:09:38.328 DEBUG:teuthology.orchestra.run.smithi155:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:09:38.429 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:09:38.430 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T14:09:38.430 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:09:38.447 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:09:38.447 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T14:09:38.447 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:09:38.448 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:09:38.448 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:09:38.457 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:09:38.457 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T14:09:38.457 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:09:38.457 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:09:38.457 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:09:38.458 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790" 2024-04-25T14:09:38.458 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790:/ 2024-04-25T14:09:38.458 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb" will pass to kernel 2024-04-25T14:09:38.461 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T14:09:38.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:09:38.462 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T14:09:38.481 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:09:38.574 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:09:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:09:38.578 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:09:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:09:38.584 INFO:teuthology.orchestra.run.smithi155.stdout:client7789 2024-04-25T14:09:38.584 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:09:38.584 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:09:38.636 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:09:38.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.790+0000 7f37fd800640 1 Processor -- start 2024-04-25T14:09:38.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.790+0000 7f37fd800640 1 -- start start 2024-04-25T14:09:38.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.790+0000 7f37fd800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f37f80cdd70 0x7f37f80ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:38.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.790+0000 7f37fd800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f37f806e700 0x7f37f806eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:38.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.790+0000 7f37fd800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37f806f050 con 0x7f37f8070e20 2024-04-25T14:09:38.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.790+0000 7f37fd800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f37f806d940 con 0x7f37f806e700 2024-04-25T14:09:38.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.790+0000 7f37fd800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f37f806dc20 con 0x7f37f80cdd70 2024-04-25T14:09:38.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.790+0000 7f37f7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f37f806e700 0x7f37f806eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:38.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.790+0000 7f37f7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f37f806e700 0x7f37f806eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48466/0 (socket says 172.21.15.86:48466) 2024-04-25T14:09:38.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.790+0000 7f37f7400640 1 -- 172.21.15.86:0/1550410885 learned_addr learned my addr 172.21.15.86:0/1550410885 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:38.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.790+0000 7f37f6a00640 1 --2- 172.21.15.86:0/1550410885 >> v2:172.21.15.155:3300/0 conn(0x7f37f80cdd70 0x7f37f80ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:38.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.790+0000 7f37f6000640 1 -- 172.21.15.86:0/1550410885 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 145179093 0 0) 0x7f37f806f050 con 0x7f37f8070e20 2024-04-25T14:09:38.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.790+0000 7f37f7400640 1 -- 172.21.15.86:0/1550410885 >> v2:172.21.15.155:3300/0 conn(0x7f37f80cdd70 msgr2=0x7f37f80ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:38.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.790+0000 7f37f7400640 1 --2- 172.21.15.86:0/1550410885 >> v2:172.21.15.155:3300/0 conn(0x7f37f80cdd70 0x7f37f80ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:38.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.790+0000 7f37f7400640 1 -- 172.21.15.86:0/1550410885 >> v1:172.21.15.155:6789/0 conn(0x7f37f8070e20 legacy=0x7f37f8071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:38.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.790+0000 7f37f7400640 1 -- 172.21.15.86:0/1550410885 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f806df00 con 0x7f37f806e700 2024-04-25T14:09:38.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.794+0000 7f37f7400640 1 --2- 172.21.15.86:0/1550410885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37f806e700 0x7f37f806eb10 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f37f8074610 tx=0x7f37f001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72563c912e17c27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:38.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.794+0000 7f37f6000640 1 -- 172.21.15.86:0/1550410885 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f00a5670 con 0x7f37f806e700 2024-04-25T14:09:38.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.794+0000 7f37f6000640 1 -- 172.21.15.86:0/1550410885 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37f00a5810 con 0x7f37f806e700 2024-04-25T14:09:38.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.794+0000 7f37f6000640 1 -- 172.21.15.86:0/1550410885 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f00a5be0 con 0x7f37f806e700 2024-04-25T14:09:38.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.794+0000 7f37fd800640 1 -- 172.21.15.86:0/1550410885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37f806e700 msgr2=0x7f37f806eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:38.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.794+0000 7f37fd800640 1 --2- 172.21.15.86:0/1550410885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37f806e700 0x7f37f806eb10 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f37f8074610 tx=0x7f37f001c960 comp rx=0 tx=0).stop 2024-04-25T14:09:38.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.798+0000 7f37fd800640 1 -- 172.21.15.86:0/1550410885 shutdown_connections 2024-04-25T14:09:38.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.798+0000 7f37fd800640 1 --2- 172.21.15.86:0/1550410885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37f806e700 0x7f37f806eb10 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:38.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.798+0000 7f37fd800640 1 --2- 172.21.15.86:0/1550410885 >> v2:172.21.15.155:3300/0 conn(0x7f37f80cdd70 0x7f37f80ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:38.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.798+0000 7f37fd800640 1 -- 172.21.15.86:0/1550410885 >> 172.21.15.86:0/1550410885 conn(0x7f37f80cffc0 msgr2=0x7f37f80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:38.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.798+0000 7f37fd800640 1 -- 172.21.15.86:0/1550410885 shutdown_connections 2024-04-25T14:09:38.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.798+0000 7f37fd800640 1 -- 172.21.15.86:0/1550410885 wait complete. 2024-04-25T14:09:38.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.798+0000 7f37fd800640 1 Processor -- start 2024-04-25T14:09:38.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.798+0000 7f37fd800640 1 -- start start 2024-04-25T14:09:38.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.798+0000 7f37fd800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37f806e700 0x7f37f8151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:38.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.798+0000 7f37fd800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37f8070e20 0x7f37f8151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:38.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.798+0000 7f37fd800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f80cdd70 0x7f37f8140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:38.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.798+0000 7f37fd800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37f816cc00 con 0x7f37f806e700 2024-04-25T14:09:38.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.798+0000 7f37fd800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f37f81643c0 con 0x7f37f80cdd70 2024-04-25T14:09:38.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.798+0000 7f37fd800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f37f816cd80 con 0x7f37f8070e20 2024-04-25T14:09:38.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.802+0000 7f37f6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37f806e700 0x7f37f8151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:38.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.802+0000 7f37f6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37f806e700 0x7f37f8151840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48472/0 (socket says 172.21.15.86:48472) 2024-04-25T14:09:38.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.802+0000 7f37f6a00640 1 -- 172.21.15.86:0/1635265161 learned_addr learned my addr 172.21.15.86:0/1635265161 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:38.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.802+0000 7f37f7400640 1 --2- 172.21.15.86:0/1635265161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f80cdd70 0x7f37f8140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:38.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.802+0000 7f37ee000640 1 --2- 172.21.15.86:0/1635265161 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37f8070e20 0x7f37f8151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:38.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.802+0000 7f37f6a00640 1 -- 172.21.15.86:0/1635265161 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37f8070e20 msgr2=0x7f37f8151d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:38.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.802+0000 7f37f6a00640 1 --2- 172.21.15.86:0/1635265161 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37f8070e20 0x7f37f8151d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:38.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.802+0000 7f37f6a00640 1 -- 172.21.15.86:0/1635265161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f80cdd70 msgr2=0x7f37f8140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:38.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.802+0000 7f37f6a00640 1 --2- 172.21.15.86:0/1635265161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f80cdd70 0x7f37f8140530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:38.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.802+0000 7f37f6a00640 1 -- 172.21.15.86:0/1635265161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f80ccec0 con 0x7f37f806e700 2024-04-25T14:09:38.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.802+0000 7f37f6a00640 1 --2- 172.21.15.86:0/1635265161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37f806e700 0x7f37f8151840 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f37e800e870 tx=0x7f37e800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:38.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.802+0000 7f37ef400640 1 -- 172.21.15.86:0/1635265161 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e8017070 con 0x7f37f806e700 2024-04-25T14:09:38.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.802+0000 7f37ef400640 1 -- 172.21.15.86:0/1635265161 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37e800c940 con 0x7f37f806e700 2024-04-25T14:09:38.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.802+0000 7f37fd800640 1 -- 172.21.15.86:0/1635265161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37f80cd1a0 con 0x7f37f806e700 2024-04-25T14:09:38.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.802+0000 7f37ef400640 1 -- 172.21.15.86:0/1635265161 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e800ccf0 con 0x7f37f806e700 2024-04-25T14:09:38.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.802+0000 7f37fd800640 1 -- 172.21.15.86:0/1635265161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37f8157b40 con 0x7f37f806e700 2024-04-25T14:09:38.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.802+0000 7f37fd800640 1 -- 172.21.15.86:0/1635265161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37f8157da0 con 0x7f37f806e700 2024-04-25T14:09:38.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.802+0000 7f37ef400640 1 -- 172.21.15.86:0/1635265161 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7f37e801a070 con 0x7f37f806e700 2024-04-25T14:09:38.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.806+0000 7f37ef400640 1 --2- 172.21.15.86:0/1635265161 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f37d406f520 0x7f37d4071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:38.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.806+0000 7f37ef400640 1 -- 172.21.15.86:0/1635265161 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7f37e808f740 con 0x7f37f806e700 2024-04-25T14:09:38.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.806+0000 7f37ee000640 1 --2- 172.21.15.86:0/1635265161 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f37d406f520 0x7f37d4071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:38.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.806+0000 7f37ee000640 1 --2- 172.21.15.86:0/1635265161 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f37d406f520 0x7f37d4071a30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f37f8156120 tx=0x7f37e0009040 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:38.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.810+0000 7f37ef400640 1 -- 172.21.15.86:0/1635265161 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f37e805bdd0 con 0x7f37f806e700 2024-04-25T14:09:38.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.998+0000 7f37fd800640 1 -- 172.21.15.86:0/1635265161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37f80720e0 con 0x7f37f806e700 2024-04-25T14:09:38.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:38.998+0000 7f37ef400640 1 -- 172.21.15.86:0/1635265161 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5189 (secure 0 0 0) 0x7f37e80600d0 con 0x7f37f806e700 2024-04-25T14:09:38.996 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:38.996 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":76,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1961914630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1961914630},{"type":"v1","addr":"172.21.15.86:6836","nonce":1961914630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2298829380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2298829380},{"type":"v1","addr":"172.21.15.155:6834","nonce":2298829380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7914,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3761704765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3761704765},{"type":"v1","addr":"172.21.15.86:6837","nonce":3761704765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:09:25.198798+0000","modified":"2024-04-25T14:09:28.237583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6835/1049360040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1049360040},{"type":"v1","addr":"172.21.15.155:6835","nonce":1049360040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7932,"qdb_cluster":[7932]},"id":7}]} 2024-04-25T14:09:38.996 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 76 2024-04-25T14:09:38.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.002+0000 7f37fd800640 1 -- 172.21.15.86:0/1635265161 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f37d406f520 msgr2=0x7f37d4071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:38.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.002+0000 7f37fd800640 1 --2- 172.21.15.86:0/1635265161 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f37d406f520 0x7f37d4071a30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f37f8156120 tx=0x7f37e0009040 comp rx=0 tx=0).stop 2024-04-25T14:09:38.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.002+0000 7f37fd800640 1 -- 172.21.15.86:0/1635265161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37f806e700 msgr2=0x7f37f8151840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:38.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.002+0000 7f37fd800640 1 --2- 172.21.15.86:0/1635265161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37f806e700 0x7f37f8151840 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f37e800e870 tx=0x7f37e800ec70 comp rx=0 tx=0).stop 2024-04-25T14:09:38.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.002+0000 7f37fd800640 1 -- 172.21.15.86:0/1635265161 shutdown_connections 2024-04-25T14:09:38.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.002+0000 7f37fd800640 1 --2- 172.21.15.86:0/1635265161 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f37d406f520 0x7f37d4071a30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:38.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.002+0000 7f37fd800640 1 --2- 172.21.15.86:0/1635265161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f80cdd70 0x7f37f8140530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:38.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.002+0000 7f37fd800640 1 --2- 172.21.15.86:0/1635265161 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37f8070e20 0x7f37f8151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:38.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.002+0000 7f37fd800640 1 --2- 172.21.15.86:0/1635265161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37f806e700 0x7f37f8151840 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:38.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.002+0000 7f37fd800640 1 -- 172.21.15.86:0/1635265161 >> 172.21.15.86:0/1635265161 conn(0x7f37f80cffc0 msgr2=0x7f37f806eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:38.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.002+0000 7f37fd800640 1 -- 172.21.15.86:0/1635265161 shutdown_connections 2024-04-25T14:09:38.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.002+0000 7f37fd800640 1 -- 172.21.15.86:0/1635265161 wait complete. 2024-04-25T14:09:39.018 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable stats 2024-04-25T14:09:39.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.170+0000 7f9287400640 1 Processor -- start 2024-04-25T14:09:39.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.170+0000 7f9287400640 1 -- start start 2024-04-25T14:09:39.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.174+0000 7f9287400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9280159aa0 0x7f9280159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:39.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.174+0000 7f9287400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92800b4c40 con 0x7f9280070750 2024-04-25T14:09:39.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.174+0000 7f9287400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f928015a800 con 0x7f928015a3f0 2024-04-25T14:09:39.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.174+0000 7f9287400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f92800b4ee0 con 0x7f9280159aa0 2024-04-25T14:09:39.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.174+0000 7f9284a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9280070750 0x7f92800b03b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41636/0 (socket says 172.21.15.86:41636) 2024-04-25T14:09:39.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.174+0000 7f9284a00640 1 -- 172.21.15.86:0/1883424591 learned_addr learned my addr 172.21.15.86:0/1883424591 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:39.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.174+0000 7f9277e00640 1 --2- 172.21.15.86:0/1883424591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9280159aa0 0x7f9280159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:39.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.174+0000 7f927fe00640 1 -- 172.21.15.86:0/1883424591 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1988582589 0 0) 0x7f92800b4c40 con 0x7f9280070750 2024-04-25T14:09:39.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.174+0000 7f927fe00640 1 -- 172.21.15.86:0/1883424591 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9264003530 con 0x7f9280070750 2024-04-25T14:09:39.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.174+0000 7f927fe00640 1 -- 172.21.15.86:0/1883424591 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2017188567 0 0) 0x7f928015a800 con 0x7f928015a3f0 2024-04-25T14:09:39.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.174+0000 7f927fe00640 1 -- 172.21.15.86:0/1883424591 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92800b4c40 con 0x7f928015a3f0 2024-04-25T14:09:39.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.174+0000 7f9277e00640 1 -- 172.21.15.86:0/1883424591 >> v1:172.21.15.155:6789/0 conn(0x7f928015a3f0 legacy=0x7f928006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:39.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.174+0000 7f9277e00640 1 -- 172.21.15.86:0/1883424591 >> v1:172.21.15.86:6789/0 conn(0x7f9280070750 legacy=0x7f92800b03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:39.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.174+0000 7f9277e00640 1 -- 172.21.15.86:0/1883424591 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92800b51c0 con 0x7f9280159aa0 2024-04-25T14:09:39.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.174+0000 7f9277e00640 1 --2- 172.21.15.86:0/1883424591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9280159aa0 0x7f9280159eb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f92680097f0 tx=0x7f926801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ade3e367614071fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:39.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.178+0000 7f927fe00640 1 -- 172.21.15.86:0/1883424591 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92680b0070 con 0x7f9280159aa0 2024-04-25T14:09:39.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.178+0000 7f927fe00640 1 -- 172.21.15.86:0/1883424591 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92680ab600 con 0x7f9280159aa0 2024-04-25T14:09:39.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.178+0000 7f927fe00640 1 -- 172.21.15.86:0/1883424591 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92680ab9d0 con 0x7f9280159aa0 2024-04-25T14:09:39.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.178+0000 7f9287400640 1 -- 172.21.15.86:0/1883424591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9280159aa0 msgr2=0x7f9280159eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:39.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.178+0000 7f9287400640 1 --2- 172.21.15.86:0/1883424591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9280159aa0 0x7f9280159eb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f92680097f0 tx=0x7f926801c450 comp rx=0 tx=0).stop 2024-04-25T14:09:39.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.178+0000 7f9287400640 1 -- 172.21.15.86:0/1883424591 shutdown_connections 2024-04-25T14:09:39.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.178+0000 7f9287400640 1 --2- 172.21.15.86:0/1883424591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9280159aa0 0x7f9280159eb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:39.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.178+0000 7f9287400640 1 -- 172.21.15.86:0/1883424591 >> 172.21.15.86:0/1883424591 conn(0x7f92800d19b0 msgr2=0x7f92800d1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:39.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.178+0000 7f9287400640 1 -- 172.21.15.86:0/1883424591 shutdown_connections 2024-04-25T14:09:39.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.178+0000 7f9287400640 1 -- 172.21.15.86:0/1883424591 wait complete. 2024-04-25T14:09:39.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.178+0000 7f9287400640 1 Processor -- start 2024-04-25T14:09:39.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.178+0000 7f9287400640 1 -- start start 2024-04-25T14:09:39.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.182+0000 7f9287400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9280070750 0x7f928014d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:39.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.182+0000 7f9287400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9280159aa0 0x7f9280150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:39.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.182+0000 7f9287400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f928015a3f0 0x7f9280153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:39.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.182+0000 7f9287400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f928016de90 con 0x7f9280070750 2024-04-25T14:09:39.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.182+0000 7f9287400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f928016dd10 con 0x7f9280159aa0 2024-04-25T14:09:39.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.182+0000 7f9287400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f928016e010 con 0x7f928015a3f0 2024-04-25T14:09:39.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.182+0000 7f9284a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9280070750 0x7f928014d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:39.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.182+0000 7f9284a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9280070750 0x7f928014d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48474/0 (socket says 172.21.15.86:48474) 2024-04-25T14:09:39.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.182+0000 7f9284a00640 1 -- 172.21.15.86:0/1637399145 learned_addr learned my addr 172.21.15.86:0/1637399145 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:39.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.182+0000 7f9285400640 1 --2- 172.21.15.86:0/1637399145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f928015a3f0 0x7f9280153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:39.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.182+0000 7f9277e00640 1 --2- 172.21.15.86:0/1637399145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9280159aa0 0x7f9280150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:39.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.182+0000 7f9284a00640 1 -- 172.21.15.86:0/1637399145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f928015a3f0 msgr2=0x7f9280153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:39.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.182+0000 7f9284a00640 1 --2- 172.21.15.86:0/1637399145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f928015a3f0 0x7f9280153af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:39.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.182+0000 7f9284a00640 1 -- 172.21.15.86:0/1637399145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9280159aa0 msgr2=0x7f9280150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:39.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.182+0000 7f9284a00640 1 --2- 172.21.15.86:0/1637399145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9280159aa0 0x7f9280150f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:39.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.182+0000 7f9284a00640 1 -- 172.21.15.86:0/1637399145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9280148140 con 0x7f9280070750 2024-04-25T14:09:39.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.182+0000 7f9284a00640 1 --2- 172.21.15.86:0/1637399145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9280070750 0x7f928014d760 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f927000c3e0 tx=0x7f927000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:39.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.182+0000 7f927d600640 1 -- 172.21.15.86:0/1637399145 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9270017070 con 0x7f9280070750 2024-04-25T14:09:39.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.182+0000 7f927d600640 1 -- 172.21.15.86:0/1637399145 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9270005cf0 con 0x7f9280070750 2024-04-25T14:09:39.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.182+0000 7f927d600640 1 -- 172.21.15.86:0/1637399145 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92700072d0 con 0x7f9280070750 2024-04-25T14:09:39.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.182+0000 7f9287400640 1 -- 172.21.15.86:0/1637399145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9280147b80 con 0x7f9280070750 2024-04-25T14:09:39.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.182+0000 7f9287400640 1 -- 172.21.15.86:0/1637399145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9280146dc0 con 0x7f9280070750 2024-04-25T14:09:39.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.182+0000 7f9287400640 1 -- 172.21.15.86:0/1637399145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92800748a0 con 0x7f9280070750 2024-04-25T14:09:39.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.186+0000 7f927d600640 1 -- 172.21.15.86:0/1637399145 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 37) ==== 93309+0+0 (secure 0 0 0) 0x7f927001a070 con 0x7f9280070750 2024-04-25T14:09:39.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.186+0000 7f927d600640 1 --2- 172.21.15.86:0/1637399145 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f925c06f520 0x7f925c071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:39.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.186+0000 7f927d600640 1 -- 172.21.15.86:0/1637399145 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7f927008e890 con 0x7f9280070750 2024-04-25T14:09:39.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.186+0000 7f9277e00640 1 --2- 172.21.15.86:0/1637399145 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f925c06f520 0x7f925c071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:39.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.186+0000 7f9277e00640 1 --2- 172.21.15.86:0/1637399145 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f925c06f520 0x7f925c071a30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f92800722e0 tx=0x7f92680a6000 comp rx=0 tx=0).ready entity=mgr.7650 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:39.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.190+0000 7f927d600640 1 -- 172.21.15.86:0/1637399145 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f927005ae90 con 0x7f9280070750 2024-04-25T14:09:39.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.390+0000 7f9287400640 1 -- 172.21.15.86:0/1637399145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "stats"} v 0) -- 0x7f9280076bf0 con 0x7f9280070750 2024-04-25T14:09:39.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.406+0000 7f927d600640 1 -- 172.21.15.86:0/1637399145 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "stats"}]=0 v38) ==== 84+0+0 (secure 0 0 0) 0x7f927005f190 con 0x7f9280070750 2024-04-25T14:09:39.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.406+0000 7f9287400640 1 -- 172.21.15.86:0/1637399145 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f925c06f520 msgr2=0x7f925c071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:39.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.406+0000 7f9287400640 1 --2- 172.21.15.86:0/1637399145 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f925c06f520 0x7f925c071a30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f92800722e0 tx=0x7f92680a6000 comp rx=0 tx=0).stop 2024-04-25T14:09:39.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.406+0000 7f9287400640 1 -- 172.21.15.86:0/1637399145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9280070750 msgr2=0x7f928014d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:39.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.406+0000 7f9287400640 1 --2- 172.21.15.86:0/1637399145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9280070750 0x7f928014d760 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f927000c3e0 tx=0x7f927000c8a0 comp rx=0 tx=0).stop 2024-04-25T14:09:39.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.406+0000 7f9287400640 1 -- 172.21.15.86:0/1637399145 shutdown_connections 2024-04-25T14:09:39.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.406+0000 7f9287400640 1 --2- 172.21.15.86:0/1637399145 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7f925c06f520 0x7f925c071a30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:39.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.406+0000 7f9287400640 1 --2- 172.21.15.86:0/1637399145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f928015a3f0 0x7f9280153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:39.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.406+0000 7f9287400640 1 --2- 172.21.15.86:0/1637399145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9280159aa0 0x7f9280150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:39.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.406+0000 7f9287400640 1 --2- 172.21.15.86:0/1637399145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9280070750 0x7f928014d760 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:39.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.406+0000 7f9287400640 1 -- 172.21.15.86:0/1637399145 >> 172.21.15.86:0/1637399145 conn(0x7f92800d19b0 msgr2=0x7f92800aef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:39.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.410+0000 7f9287400640 1 -- 172.21.15.86:0/1637399145 shutdown_connections 2024-04-25T14:09:39.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.410+0000 7f9287400640 1 -- 172.21.15.86:0/1637399145 wait complete. 2024-04-25T14:09:39.428 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats --client_ip 1.2.3 2024-04-25T14:09:39.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.586+0000 7fe8c2800640 1 Processor -- start 2024-04-25T14:09:39.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.586+0000 7fe8c2800640 1 -- start start 2024-04-25T14:09:39.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.590+0000 7fe8c2800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe8bc0d10c0 0x7fe8bc0cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:39.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.590+0000 7fe8c2800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8bc0d14d0 0x7fe8bc0d0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:39.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.590+0000 7fe8c2800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8bc0d19c0 con 0x7fe8bc15fe90 2024-04-25T14:09:39.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.590+0000 7fe8c2800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe8bc0d1ca0 con 0x7fe8bc0d10c0 2024-04-25T14:09:39.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.590+0000 7fe8c2800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe8bc163bb0 con 0x7fe8bc0d14d0 2024-04-25T14:09:39.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.590+0000 7fe8bb400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe8bc15fe90 0x7fe8bc1643b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41644/0 (socket says 172.21.15.86:41644) 2024-04-25T14:09:39.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.590+0000 7fe8bb400640 1 -- 172.21.15.86:0/295201740 learned_addr learned my addr 172.21.15.86:0/295201740 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:39.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.590+0000 7fe8baa00640 1 --2- 172.21.15.86:0/295201740 >> v2:172.21.15.155:3300/0 conn(0x7fe8bc0d10c0 0x7fe8bc0cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:39.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.590+0000 7fe8bbe00640 1 --2- 172.21.15.86:0/295201740 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8bc0d14d0 0x7fe8bc0d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:39.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.590+0000 7fe8ba000640 1 -- 172.21.15.86:0/295201740 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3301250534 0 0) 0x7fe8bc0d19c0 con 0x7fe8bc15fe90 2024-04-25T14:09:39.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.590+0000 7fe8ba000640 1 -- 172.21.15.86:0/295201740 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8a0003530 con 0x7fe8bc15fe90 2024-04-25T14:09:39.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.590+0000 7fe8baa00640 1 -- 172.21.15.86:0/295201740 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8bc0d14d0 msgr2=0x7fe8bc0d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:39.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.590+0000 7fe8baa00640 1 --2- 172.21.15.86:0/295201740 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8bc0d14d0 0x7fe8bc0d0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:39.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.590+0000 7fe8baa00640 1 -- 172.21.15.86:0/295201740 >> v1:172.21.15.86:6789/0 conn(0x7fe8bc15fe90 legacy=0x7fe8bc1643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:39.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.590+0000 7fe8baa00640 1 -- 172.21.15.86:0/295201740 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8bc163e90 con 0x7fe8bc0d10c0 2024-04-25T14:09:39.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.590+0000 7fe8bbe00640 1 --2- 172.21.15.86:0/295201740 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8bc0d14d0 0x7fe8bc0d0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:09:39.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.594+0000 7fe8baa00640 1 --2- 172.21.15.86:0/295201740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8bc0d10c0 0x7fe8bc0cfdd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80097c0 tx=0x7fe8a801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e09bb76a9be75dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:39.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.594+0000 7fe8ba000640 1 -- 172.21.15.86:0/295201740 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a80b0070 con 0x7fe8bc0d10c0 2024-04-25T14:09:39.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.594+0000 7fe8ba000640 1 -- 172.21.15.86:0/295201740 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8a80ab600 con 0x7fe8bc0d10c0 2024-04-25T14:09:39.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.594+0000 7fe8ba000640 1 -- 172.21.15.86:0/295201740 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a80ab9d0 con 0x7fe8bc0d10c0 2024-04-25T14:09:39.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.594+0000 7fe8c2800640 1 -- 172.21.15.86:0/295201740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8bc0d10c0 msgr2=0x7fe8bc0cfdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:39.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.594+0000 7fe8c2800640 1 --2- 172.21.15.86:0/295201740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8bc0d10c0 0x7fe8bc0cfdd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80097c0 tx=0x7fe8a801c410 comp rx=0 tx=0).stop 2024-04-25T14:09:39.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.594+0000 7fe8c2800640 1 -- 172.21.15.86:0/295201740 shutdown_connections 2024-04-25T14:09:39.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.594+0000 7fe8c2800640 1 --2- 172.21.15.86:0/295201740 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8bc0d14d0 0x7fe8bc0d0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:39.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.594+0000 7fe8c2800640 1 --2- 172.21.15.86:0/295201740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8bc0d10c0 0x7fe8bc0cfdd0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:39.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.594+0000 7fe8c2800640 1 -- 172.21.15.86:0/295201740 >> 172.21.15.86:0/295201740 conn(0x7fe8bc0b5020 msgr2=0x7fe8bc0b5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:39.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.594+0000 7fe8c2800640 1 -- 172.21.15.86:0/295201740 shutdown_connections 2024-04-25T14:09:39.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.594+0000 7fe8c2800640 1 -- 172.21.15.86:0/295201740 wait complete. 2024-04-25T14:09:39.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.594+0000 7fe8c2800640 1 Processor -- start 2024-04-25T14:09:39.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.594+0000 7fe8c2800640 1 -- start start 2024-04-25T14:09:39.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.598+0000 7fe8c2800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8bc0d10c0 0x7fe8bc15aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:39.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.598+0000 7fe8c2800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8bc0d14d0 0x7fe8bc0c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:39.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.598+0000 7fe8c2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8bc15fe90 0x7fe8bc162480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:39.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.598+0000 7fe8c2800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc0b81d0 con 0x7fe8bc15fe90 2024-04-25T14:09:39.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.598+0000 7fe8c2800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc0b8050 con 0x7fe8bc0d14d0 2024-04-25T14:09:39.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.598+0000 7fe8c2800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe8bc16cd50 con 0x7fe8bc0d10c0 2024-04-25T14:09:39.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.598+0000 7fe8bbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8bc15fe90 0x7fe8bc162480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:39.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.598+0000 7fe8bbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8bc15fe90 0x7fe8bc162480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48508/0 (socket says 172.21.15.86:48508) 2024-04-25T14:09:39.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.598+0000 7fe8bbe00640 1 -- 172.21.15.86:0/615812239 learned_addr learned my addr 172.21.15.86:0/615812239 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:39.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.598+0000 7fe8bb400640 1 --2- 172.21.15.86:0/615812239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8bc0d10c0 0x7fe8bc15aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:39.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.598+0000 7fe8baa00640 1 --2- 172.21.15.86:0/615812239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8bc0d14d0 0x7fe8bc0c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:39.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.598+0000 7fe8bbe00640 1 -- 172.21.15.86:0/615812239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8bc0d10c0 msgr2=0x7fe8bc15aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:39.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.598+0000 7fe8bbe00640 1 --2- 172.21.15.86:0/615812239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8bc0d10c0 0x7fe8bc15aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:39.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.598+0000 7fe8bbe00640 1 -- 172.21.15.86:0/615812239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8bc0d14d0 msgr2=0x7fe8bc0c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:39.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.598+0000 7fe8bbe00640 1 --2- 172.21.15.86:0/615812239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8bc0d14d0 0x7fe8bc0c96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:39.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.598+0000 7fe8bbe00640 1 -- 172.21.15.86:0/615812239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8bc0cdaa0 con 0x7fe8bc15fe90 2024-04-25T14:09:39.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.598+0000 7fe8bbe00640 1 --2- 172.21.15.86:0/615812239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8bc15fe90 0x7fe8bc162480 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4009e10 tx=0x7fe8b400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:39.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.598+0000 7fe8b3400640 1 -- 172.21.15.86:0/615812239 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4018020 con 0x7fe8bc15fe90 2024-04-25T14:09:39.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.598+0000 7fe8b3400640 1 -- 172.21.15.86:0/615812239 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8b4014480 con 0x7fe8bc15fe90 2024-04-25T14:09:39.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.598+0000 7fe8c2800640 1 -- 172.21.15.86:0/615812239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8bc0cdd80 con 0x7fe8bc15fe90 2024-04-25T14:09:39.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.598+0000 7fe8b3400640 1 -- 172.21.15.86:0/615812239 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4014850 con 0x7fe8bc15fe90 2024-04-25T14:09:39.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.598+0000 7fe8c2800640 1 -- 172.21.15.86:0/615812239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8bc0ce340 con 0x7fe8bc15fe90 2024-04-25T14:09:39.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.598+0000 7fe8c2800640 1 -- 172.21.15.86:0/615812239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8bc0cd5c0 con 0x7fe8bc15fe90 2024-04-25T14:09:39.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.602+0000 7fe8b3400640 1 -- 172.21.15.86:0/615812239 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 38) ==== 93318+0+0 (secure 0 0 0) 0x7fe8b401b020 con 0x7fe8bc15fe90 2024-04-25T14:09:39.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.602+0000 7fe8b3400640 1 --2- 172.21.15.86:0/615812239 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fe89c06f5a0 0x7fe89c071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:39.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.602+0000 7fe8b3400640 1 -- 172.21.15.86:0/615812239 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7fe8b4090300 con 0x7fe8bc15fe90 2024-04-25T14:09:39.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.602+0000 7fe8bb400640 1 -- 172.21.15.86:0/615812239 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fe89c06f5a0 msgr2=0x7fe89c071ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/4236874185 2024-04-25T14:09:39.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.602+0000 7fe8bb400640 1 --2- 172.21.15.86:0/615812239 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fe89c06f5a0 0x7fe89c071ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:09:39.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.606+0000 7fe8b3400640 1 -- 172.21.15.86:0/615812239 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe8bc0cd5c0 con 0x7fe8bc15fe90 2024-04-25T14:09:39.654 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:39.655+0000 7f500786d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:09:39.667 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:39.670+0000 7fa64827d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:09:39.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.790+0000 7fe8c2800640 1 -- 172.21.15.86:0/615812239 --> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] -- mgr_command(tid 0: {"prefix": "fs perf stats", "client_ip": "1.2.3", "target": ["mon-mgr", ""]}) -- 0x7fe8bc0cd7f0 con 0x7fe89c06f5a0 2024-04-25T14:09:39.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.806+0000 7fe8bb400640 1 -- 172.21.15.86:0/615812239 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fe89c06f5a0 msgr2=0x7fe89c071ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/4236874185 2024-04-25T14:09:39.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:39.806+0000 7fe8bb400640 1 --2- 172.21.15.86:0/615812239 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fe89c06f5a0 0x7fe89c071ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:09:40.073 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:40.071+0000 7f500786d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:09:40.085 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:40.090+0000 7fa64827d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:09:40.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:40.210+0000 7fe8bb400640 1 -- 172.21.15.86:0/615812239 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fe89c06f5a0 msgr2=0x7fe89c071ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/4236874185 2024-04-25T14:09:40.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:40.210+0000 7fe8bb400640 1 --2- 172.21.15.86:0/615812239 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fe89c06f5a0 0x7fe89c071ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T14:09:40.223 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:40.223+0000 7f500786d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:09:40.233 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:40.234+0000 7fa64827d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:09:40.695 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:40.695+0000 7f500786d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:09:40.703 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:40.706+0000 7fa64827d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:09:40.754 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:40.755+0000 7f500786d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:09:40.761 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:40.766+0000 7fa64827d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:09:40.813 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:40.815+0000 7f500786d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:09:40.819 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:40.822+0000 7fa64827d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:09:40.904 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:40.903+0000 7f500786d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:09:40.907 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:40.910+0000 7fa64827d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:09:40.961 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:40.959+0000 7f500786d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:09:40.963 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:40.966+0000 7fa64827d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:09:41.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:41.014+0000 7fe8bb400640 1 -- 172.21.15.86:0/615812239 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fe89c06f5a0 msgr2=0x7fe89c071ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/4236874185 2024-04-25T14:09:41.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:41.014+0000 7fe8bb400640 1 --2- 172.21.15.86:0/615812239 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fe89c06f5a0 0x7fe89c071ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T14:09:41.046 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:41.047+0000 7f500786d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:09:41.048 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:41.050+0000 7fa64827d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:09:41.106 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:41.107+0000 7f500786d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:09:41.107 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:41.110+0000 7fa64827d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:09:41.259 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:41.259+0000 7f500786d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:09:41.259 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:41.262+0000 7fa64827d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:09:41.317 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:41.322+0000 7fa64827d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:09:41.319 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:41.319+0000 7f500786d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:09:41.555 INFO:tasks.ceph.mgr.y.smithi086.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:09:41.555 INFO:tasks.ceph.mgr.y.smithi086.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:09:41.555 INFO:tasks.ceph.mgr.y.smithi086.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:09:41.559 INFO:tasks.ceph.mgr.x.smithi155.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:09:41.559 INFO:tasks.ceph.mgr.x.smithi155.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:09:41.559 INFO:tasks.ceph.mgr.x.smithi155.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:09:41.564 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:41.566+0000 7fa64827d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:09:41.567 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:41.567+0000 7f500786d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:09:41.709 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:41.714+0000 7fa64827d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:09:41.715 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:41.715+0000 7f500786d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:09:41.837 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:41.842+0000 7fa64827d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:09:41.845 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:41.847+0000 7f500786d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:09:41.910 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:41.914+0000 7fa64827d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:09:41.917 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:41.919+0000 7f500786d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:09:42.057 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:42.062+0000 7fa64827d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:09:42.066 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:42.067+0000 7f500786d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:09:42.596 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:42.598+0000 7fa64827d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:09:42.611 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:42.611+0000 7f500786d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:09:42.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:42.618+0000 7fe8bb400640 1 -- 172.21.15.86:0/615812239 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fe89c06f5a0 msgr2=0x7fe89c071ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/4236874185 2024-04-25T14:09:42.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:42.618+0000 7fe8bb400640 1 --2- 172.21.15.86:0/615812239 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fe89c06f5a0 0x7fe89c071ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T14:09:42.681 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:42.682+0000 7fa64827d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:09:42.696 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:42.695+0000 7f500786d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:09:42.843 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:42.846+0000 7fa64827d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:09:42.858 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:42.859+0000 7f500786d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:09:42.926 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:42.930+0000 7fa64827d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:09:42.942 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:42.943+0000 7f500786d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:09:43.244 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:43.246+0000 7fa64827d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:09:43.262 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:43.263+0000 7f500786d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:09:43.306 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:43.310+0000 7fa64827d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:09:43.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:43.318+0000 7fe8b3400640 1 -- 172.21.15.86:0/615812239 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 39) ==== 92927+0+0 (secure 0 0 0) 0x7fe8b4012050 con 0x7fe8bc15fe90 2024-04-25T14:09:43.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:43.318+0000 7fe8b3400640 1 -- 172.21.15.86:0/615812239 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fe89c06f5a0 msgr2=0x7fe89c071ab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:09:43.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:43.318+0000 7fe8b3400640 1 --2- 172.21.15.86:0/615812239 >> [v2:172.21.15.86:6832/4236874185,v1:172.21.15.86:6833/4236874185] conn(0x7fe89c06f5a0 0x7fe89c071ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:43.330 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:43.331+0000 7f500786d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:09:43.392 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:43.394+0000 7fa5c4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:43.392 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:43.394+0000 7fa5c4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:43.392 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:43.394+0000 7fa5c4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:43.393 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:43.394+0000 7fa5c4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:43.393 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:43.394+0000 7fa5c4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:43.393 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:43.394+0000 7fa5c8a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:43.393 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:43.394+0000 7fa5c8a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:43.393 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:43.394+0000 7fa5c8a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:43.393 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:43.394+0000 7fa5c8a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:43.393 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:43.394+0000 7fa5c8a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.326+0000 7fe8b3400640 1 -- 172.21.15.86:0/615812239 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mgrmap(e 40) ==== 93054+0+0 (secure 0 0 0) 0x7fe8b408e690 con 0x7fe8bc15fe90 2024-04-25T14:09:44.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.326+0000 7fe8b3400640 1 --2- 172.21.15.86:0/615812239 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7fe89c077ae0 0x7fe89c079f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:44.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.326+0000 7fe8b3400640 1 -- 172.21.15.86:0/615812239 --> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] -- mgr_command(tid 0: {"prefix": "fs perf stats", "client_ip": "1.2.3", "target": ["mon-mgr", ""]}) -- 0x7fe8b4006c00 con 0x7fe89c077ae0 2024-04-25T14:09:44.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.330+0000 7fe8bb400640 1 --2- 172.21.15.86:0/615812239 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7fe89c077ae0 0x7fe89c079f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:44.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.334+0000 7fe8bb400640 1 --2- 172.21.15.86:0/615812239 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7fe89c077ae0 0x7fe89c079f00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac007fd0 tx=0x7fe8ac009040 comp rx=0 tx=0).ready entity=mgr.7819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:44.332 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:44.334+0000 7fa5d3e00640 -1 mgr.server reply reply (22) Invalid argument invalid client_ip filter spec: 1.2.3 2024-04-25T14:09:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.334+0000 7fe8b3400640 1 -- 172.21.15.86:0/615812239 <== mgr.7819 v2:172.21.15.86:6832/436174229 1 ==== mgr_command_reply(tid 0: -22 invalid client_ip filter spec: 1.2.3) ==== 44+0+0 (secure 0 0 0) 0x7fe8b4006c00 con 0x7fe89c077ae0 2024-04-25T14:09:44.333 INFO:teuthology.orchestra.run.smithi086.stderr:Error EINVAL: invalid client_ip filter spec: 1.2.3 2024-04-25T14:09:44.344 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T14:09:44.345 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable stats 2024-04-25T14:09:44.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.502+0000 7feb15600640 1 Processor -- start 2024-04-25T14:09:44.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.502+0000 7feb15600640 1 -- start start 2024-04-25T14:09:44.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.502+0000 7feb15600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7feb1006f980 0x7feb1006cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:44.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.502+0000 7feb15600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb101627d0 con 0x7feb1006fd90 2024-04-25T14:09:44.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.502+0000 7feb15600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb10071780 con 0x7feb10147730 2024-04-25T14:09:44.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.502+0000 7feb15600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7feb10071a60 con 0x7feb1006f980 2024-04-25T14:09:44.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.502+0000 7feb0f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7feb1006fd90 0x7feb1006d000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41650/0 (socket says 172.21.15.86:41650) 2024-04-25T14:09:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.502+0000 7feb0f400640 1 -- 172.21.15.86:0/1516474090 learned_addr learned my addr 172.21.15.86:0/1516474090 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.502+0000 7feb06000640 1 --2- 172.21.15.86:0/1516474090 >> v2:172.21.15.155:3300/0 conn(0x7feb1006f980 0x7feb1006cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.506+0000 7feb0e000640 1 -- 172.21.15.86:0/1516474090 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2450827879 0 0) 0x7feb101627d0 con 0x7feb1006fd90 2024-04-25T14:09:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.506+0000 7feb0e000640 1 -- 172.21.15.86:0/1516474090 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feaf4003530 con 0x7feb1006fd90 2024-04-25T14:09:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.506+0000 7feb06000640 1 -- 172.21.15.86:0/1516474090 >> v1:172.21.15.155:6789/0 conn(0x7feb10147730 legacy=0x7feb1006f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.506+0000 7feb06000640 1 -- 172.21.15.86:0/1516474090 >> v1:172.21.15.86:6789/0 conn(0x7feb1006fd90 legacy=0x7feb1006d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.506+0000 7feb06000640 1 -- 172.21.15.86:0/1516474090 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb10071430 con 0x7feb1006f980 2024-04-25T14:09:44.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.506+0000 7feb06000640 1 --2- 172.21.15.86:0/1516474090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb1006f980 0x7feb1006cac0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7feaf80097c0 tx=0x7feaf801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef48ee887d28948a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:44.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.506+0000 7feb0e000640 1 -- 172.21.15.86:0/1516474090 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf80b0070 con 0x7feb1006f980 2024-04-25T14:09:44.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.506+0000 7feb0e000640 1 -- 172.21.15.86:0/1516474090 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaf80ab600 con 0x7feb1006f980 2024-04-25T14:09:44.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.506+0000 7feb0e000640 1 -- 172.21.15.86:0/1516474090 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf80ab9b0 con 0x7feb1006f980 2024-04-25T14:09:44.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.510+0000 7feb15600640 1 -- 172.21.15.86:0/1516474090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb1006f980 msgr2=0x7feb1006cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:44.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.510+0000 7feb15600640 1 --2- 172.21.15.86:0/1516474090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb1006f980 0x7feb1006cac0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7feaf80097c0 tx=0x7feaf801c4a0 comp rx=0 tx=0).stop 2024-04-25T14:09:44.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.510+0000 7feb15600640 1 -- 172.21.15.86:0/1516474090 shutdown_connections 2024-04-25T14:09:44.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.510+0000 7feb15600640 1 --2- 172.21.15.86:0/1516474090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb1006f980 0x7feb1006cac0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:44.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.510+0000 7feb15600640 1 -- 172.21.15.86:0/1516474090 >> 172.21.15.86:0/1516474090 conn(0x7feb100ce620 msgr2=0x7feb100cea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:44.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.510+0000 7feb15600640 1 -- 172.21.15.86:0/1516474090 shutdown_connections 2024-04-25T14:09:44.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.510+0000 7feb15600640 1 -- 172.21.15.86:0/1516474090 wait complete. 2024-04-25T14:09:44.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.510+0000 7feb15600640 1 Processor -- start 2024-04-25T14:09:44.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.510+0000 7feb15600640 1 -- start start 2024-04-25T14:09:44.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.514+0000 7feb15600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb1006f980 0x7feb10141890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:44.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.514+0000 7feb15600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb10147730 0x7feb1015d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:44.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.514+0000 7feb15600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb100b1cf0 0x7feb100b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:44.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.514+0000 7feb15600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb1016c950 con 0x7feb1006f980 2024-04-25T14:09:44.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.514+0000 7feb15600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7feb1016c7d0 con 0x7feb100b1cf0 2024-04-25T14:09:44.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.514+0000 7feb15600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7feb1016cad0 con 0x7feb10147730 2024-04-25T14:09:44.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.514+0000 7feb0ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb1006f980 0x7feb10141890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.514+0000 7feb0ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb1006f980 0x7feb10141890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48644/0 (socket says 172.21.15.86:48644) 2024-04-25T14:09:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.514+0000 7feb0ea00640 1 -- 172.21.15.86:0/2331378186 learned_addr learned my addr 172.21.15.86:0/2331378186 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.514+0000 7feb0f400640 1 --2- 172.21.15.86:0/2331378186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb100b1cf0 0x7feb100b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.514+0000 7feb06000640 1 --2- 172.21.15.86:0/2331378186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb10147730 0x7feb1015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.514+0000 7feb0ea00640 1 -- 172.21.15.86:0/2331378186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb10147730 msgr2=0x7feb1015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.514+0000 7feb0ea00640 1 --2- 172.21.15.86:0/2331378186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb10147730 0x7feb1015d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.514+0000 7feb0ea00640 1 -- 172.21.15.86:0/2331378186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb100b1cf0 msgr2=0x7feb100b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.514+0000 7feb0ea00640 1 --2- 172.21.15.86:0/2331378186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb100b1cf0 0x7feb100b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:44.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.514+0000 7feb0ea00640 1 -- 172.21.15.86:0/2331378186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb10143410 con 0x7feb1006f980 2024-04-25T14:09:44.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.514+0000 7feb0ea00640 1 --2- 172.21.15.86:0/2331378186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb1006f980 0x7feb10141890 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7feb0000c310 tx=0x7feb0000c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:44.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.514+0000 7feb07400640 1 -- 172.21.15.86:0/2331378186 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb000052d0 con 0x7feb1006f980 2024-04-25T14:09:44.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.514+0000 7feb07400640 1 -- 172.21.15.86:0/2331378186 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb00005470 con 0x7feb1006f980 2024-04-25T14:09:44.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.514+0000 7feb15600640 1 -- 172.21.15.86:0/2331378186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb100b6640 con 0x7feb1006f980 2024-04-25T14:09:44.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.514+0000 7feb07400640 1 -- 172.21.15.86:0/2331378186 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00007260 con 0x7feb1006f980 2024-04-25T14:09:44.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.514+0000 7feb15600640 1 -- 172.21.15.86:0/2331378186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb1015d890 con 0x7feb1006f980 2024-04-25T14:09:44.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.514+0000 7feb15600640 1 -- 172.21.15.86:0/2331378186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb10076bf0 con 0x7feb1006f980 2024-04-25T14:09:44.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.514+0000 7feb07400640 1 -- 172.21.15.86:0/2331378186 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 40) ==== 93054+0+0 (secure 0 0 0) 0x7feb00007400 con 0x7feb1006f980 2024-04-25T14:09:44.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.518+0000 7feb07400640 1 --2- 172.21.15.86:0/2331378186 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7feaec06f2a0 0x7feaec0717b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:44.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.518+0000 7feb07400640 1 -- 172.21.15.86:0/2331378186 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6445+0+0 (secure 0 0 0) 0x7feb0008e730 con 0x7feb1006f980 2024-04-25T14:09:44.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.518+0000 7feb06000640 1 --2- 172.21.15.86:0/2331378186 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7feaec06f2a0 0x7feaec0717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:44.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.518+0000 7feb06000640 1 --2- 172.21.15.86:0/2331378186 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7feaec06f2a0 0x7feaec0717b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7feaf80ad000 tx=0x7feaf8004740 comp rx=0 tx=0).ready entity=mgr.7819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:44.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.522+0000 7feb07400640 1 -- 172.21.15.86:0/2331378186 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7feb0005b5c0 con 0x7feb1006f980 2024-04-25T14:09:44.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:44.718+0000 7feb15600640 1 -- 172.21.15.86:0/2331378186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "stats"} v 0) -- 0x7feb10076350 con 0x7feb1006f980 2024-04-25T14:09:45.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.378+0000 7feb07400640 1 -- 172.21.15.86:0/2331378186 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "stats"}]=0 v41) ==== 85+0+0 (secure 0 0 0) 0x7feb0005aec0 con 0x7feb1006f980 2024-04-25T14:09:45.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.378+0000 7feb15600640 1 -- 172.21.15.86:0/2331378186 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7feaec06f2a0 msgr2=0x7feaec0717b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:45.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.378+0000 7feb15600640 1 --2- 172.21.15.86:0/2331378186 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7feaec06f2a0 0x7feaec0717b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7feaf80ad000 tx=0x7feaf8004740 comp rx=0 tx=0).stop 2024-04-25T14:09:45.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.378+0000 7feb15600640 1 -- 172.21.15.86:0/2331378186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb1006f980 msgr2=0x7feb10141890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:45.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.378+0000 7feb15600640 1 --2- 172.21.15.86:0/2331378186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb1006f980 0x7feb10141890 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7feb0000c310 tx=0x7feb0000c7d0 comp rx=0 tx=0).stop 2024-04-25T14:09:45.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.382+0000 7feb15600640 1 -- 172.21.15.86:0/2331378186 shutdown_connections 2024-04-25T14:09:45.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.382+0000 7feb15600640 1 --2- 172.21.15.86:0/2331378186 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7feaec06f2a0 0x7feaec0717b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:45.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.382+0000 7feb15600640 1 --2- 172.21.15.86:0/2331378186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb100b1cf0 0x7feb100b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:45.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.382+0000 7feb15600640 1 --2- 172.21.15.86:0/2331378186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb10147730 0x7feb1015d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:45.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.382+0000 7feb15600640 1 --2- 172.21.15.86:0/2331378186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb1006f980 0x7feb10141890 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:45.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.382+0000 7feb15600640 1 -- 172.21.15.86:0/2331378186 >> 172.21.15.86:0/2331378186 conn(0x7feb100ce620 msgr2=0x7feb100cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:45.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.382+0000 7feb15600640 1 -- 172.21.15.86:0/2331378186 shutdown_connections 2024-04-25T14:09:45.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.382+0000 7feb15600640 1 -- 172.21.15.86:0/2331378186 wait complete. 2024-04-25T14:09:45.402 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T14:09:45.402 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:09:45.468 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:09:45.524 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T14:09:45.524 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:09:45.525 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:09:45.525 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:09:45.534 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:09:45.534 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:09:45.590 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:09:45.628 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:45.627+0000 7f6dd142c180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:09:45.633 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:45.638+0000 7fb1a9257180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:09:45.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.786+0000 7f22c5a00640 1 Processor -- start 2024-04-25T14:09:45.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.786+0000 7f22c5a00640 1 -- start start 2024-04-25T14:09:45.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.786+0000 7f22c5a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f22c00cf970 0x7f22c0070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:45.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.786+0000 7f22c5a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c006e970 0x7f22c00cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:45.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.786+0000 7f22c5a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22c0071770 con 0x7f22c0071360 2024-04-25T14:09:45.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.786+0000 7f22c5a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f22c0163830 con 0x7f22c00cf970 2024-04-25T14:09:45.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.786+0000 7f22c5a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f22c006ef90 con 0x7f22c006e970 2024-04-25T14:09:45.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.786+0000 7f22be000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f22c0071360 0x7f22c006e260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41664/0 (socket says 172.21.15.86:41664) 2024-04-25T14:09:45.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.786+0000 7f22be000640 1 -- 172.21.15.86:0/3911919425 learned_addr learned my addr 172.21.15.86:0/3911919425 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:45.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.786+0000 7f22bea00640 1 --2- 172.21.15.86:0/3911919425 >> v2:172.21.15.155:3300/0 conn(0x7f22c00cf970 0x7f22c0070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:45.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.790+0000 7f22bf400640 1 --2- 172.21.15.86:0/3911919425 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c006e970 0x7f22c00cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:45.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.790+0000 7f22bd600640 1 -- 172.21.15.86:0/3911919425 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 988659883 0 0) 0x7f22ac0012a0 con 0x7f22c0071360 2024-04-25T14:09:45.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.790+0000 7f22bd600640 1 -- 172.21.15.86:0/3911919425 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22a4003530 con 0x7f22c0071360 2024-04-25T14:09:45.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.790+0000 7f22bea00640 1 -- 172.21.15.86:0/3911919425 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c006e970 msgr2=0x7f22c00cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:45.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.790+0000 7f22bea00640 1 --2- 172.21.15.86:0/3911919425 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c006e970 0x7f22c00cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:45.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.790+0000 7f22bea00640 1 -- 172.21.15.86:0/3911919425 >> v1:172.21.15.86:6789/0 conn(0x7f22c0071360 legacy=0x7f22c006e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:45.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.790+0000 7f22bea00640 1 -- 172.21.15.86:0/3911919425 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c0163a80 con 0x7f22c00cf970 2024-04-25T14:09:45.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.790+0000 7f22bf400640 1 --2- 172.21.15.86:0/3911919425 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c006e970 0x7f22c00cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:09:45.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.790+0000 7f22bea00640 1 --2- 172.21.15.86:0/3911919425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22c00cf970 0x7f22c0070e20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f22b00097c0 tx=0x7f22b001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14ad91bd4bc07024 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:45.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.790+0000 7f22bd600640 1 -- 172.21.15.86:0/3911919425 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b00b2070 con 0x7f22c00cf970 2024-04-25T14:09:45.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.790+0000 7f22bd600640 1 -- 172.21.15.86:0/3911919425 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22b00ad640 con 0x7f22c00cf970 2024-04-25T14:09:45.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.790+0000 7f22bd600640 1 -- 172.21.15.86:0/3911919425 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b00ad9f0 con 0x7f22c00cf970 2024-04-25T14:09:45.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.794+0000 7f22c5a00640 1 -- 172.21.15.86:0/3911919425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22c00cf970 msgr2=0x7f22c0070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:45.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.794+0000 7f22c5a00640 1 --2- 172.21.15.86:0/3911919425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22c00cf970 0x7f22c0070e20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f22b00097c0 tx=0x7f22b001c4a0 comp rx=0 tx=0).stop 2024-04-25T14:09:45.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.794+0000 7f22c5a00640 1 -- 172.21.15.86:0/3911919425 shutdown_connections 2024-04-25T14:09:45.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.794+0000 7f22c5a00640 1 --2- 172.21.15.86:0/3911919425 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c006e970 0x7f22c00cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:45.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.794+0000 7f22c5a00640 1 --2- 172.21.15.86:0/3911919425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22c00cf970 0x7f22c0070e20 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:45.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.794+0000 7f22c5a00640 1 -- 172.21.15.86:0/3911919425 >> 172.21.15.86:0/3911919425 conn(0x7f22c00cffc0 msgr2=0x7f22c00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:45.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.794+0000 7f22c5a00640 1 -- 172.21.15.86:0/3911919425 shutdown_connections 2024-04-25T14:09:45.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.794+0000 7f22c5a00640 1 -- 172.21.15.86:0/3911919425 wait complete. 2024-04-25T14:09:45.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.794+0000 7f22c5a00640 1 Processor -- start 2024-04-25T14:09:45.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.794+0000 7f22c5a00640 1 -- start start 2024-04-25T14:09:45.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.798+0000 7f22c5a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22c006e970 0x7f22c0155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:45.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.798+0000 7f22c5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c0071360 0x7f22c00b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:45.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.798+0000 7f22c5a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c00cf970 0x7f22c00b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:45.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.798+0000 7f22c5a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22c016cbd0 con 0x7f22c0071360 2024-04-25T14:09:45.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.798+0000 7f22c5a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f22c00ced80 con 0x7f22c006e970 2024-04-25T14:09:45.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.798+0000 7f22c5a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f22c016cd50 con 0x7f22c00cf970 2024-04-25T14:09:45.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.798+0000 7f22be000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c0071360 0x7f22c00b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:45.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.798+0000 7f22be000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c0071360 0x7f22c00b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48708/0 (socket says 172.21.15.86:48708) 2024-04-25T14:09:45.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.798+0000 7f22be000640 1 -- 172.21.15.86:0/941561992 learned_addr learned my addr 172.21.15.86:0/941561992 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:45.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.798+0000 7f22bea00640 1 --2- 172.21.15.86:0/941561992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22c006e970 0x7f22c0155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:45.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.798+0000 7f22bf400640 1 --2- 172.21.15.86:0/941561992 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c00cf970 0x7f22c00b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:45.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.798+0000 7f22bea00640 1 -- 172.21.15.86:0/941561992 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c00cf970 msgr2=0x7f22c00b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:45.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.798+0000 7f22bea00640 1 --2- 172.21.15.86:0/941561992 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c00cf970 0x7f22c00b6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:45.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.798+0000 7f22bea00640 1 -- 172.21.15.86:0/941561992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c0071360 msgr2=0x7f22c00b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:45.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.798+0000 7f22bea00640 1 --2- 172.21.15.86:0/941561992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c0071360 0x7f22c00b69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:45.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.798+0000 7f22bea00640 1 -- 172.21.15.86:0/941561992 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c00b89d0 con 0x7f22c006e970 2024-04-25T14:09:45.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.798+0000 7f22bea00640 1 --2- 172.21.15.86:0/941561992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22c006e970 0x7f22c0155730 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f22b00ac000 tx=0x7f22b0019240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:45.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.798+0000 7f22b6a00640 1 -- 172.21.15.86:0/941561992 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b00b2040 con 0x7f22c006e970 2024-04-25T14:09:45.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.798+0000 7f22b6a00640 1 -- 172.21.15.86:0/941561992 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22b00bc070 con 0x7f22c006e970 2024-04-25T14:09:45.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.798+0000 7f22c5a00640 1 -- 172.21.15.86:0/941561992 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22c00cd520 con 0x7f22c006e970 2024-04-25T14:09:45.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.798+0000 7f22b6a00640 1 -- 172.21.15.86:0/941561992 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b0006e20 con 0x7f22c006e970 2024-04-25T14:09:45.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.798+0000 7f22c5a00640 1 -- 172.21.15.86:0/941561992 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22c00b8f50 con 0x7f22c006e970 2024-04-25T14:09:45.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.798+0000 7f22c5a00640 1 -- 172.21.15.86:0/941561992 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22c00720e0 con 0x7f22c006e970 2024-04-25T14:09:45.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.802+0000 7f22b6a00640 1 -- 172.21.15.86:0/941561992 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 41) ==== 93253+0+0 (secure 0 0 0) 0x7f22b00b6610 con 0x7f22c006e970 2024-04-25T14:09:45.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.802+0000 7f22b6a00640 1 --2- 172.21.15.86:0/941561992 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f22a006f490 0x7f22a00719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:45.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.802+0000 7f22b6a00640 1 -- 172.21.15.86:0/941561992 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6445+0+0 (secure 0 0 0) 0x7f22b0129250 con 0x7f22c006e970 2024-04-25T14:09:45.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.802+0000 7f22be000640 1 -- 172.21.15.86:0/941561992 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f22a006f490 msgr2=0x7f22a00719a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/436174229 2024-04-25T14:09:45.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.802+0000 7f22be000640 1 --2- 172.21.15.86:0/941561992 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f22a006f490 0x7f22a00719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:09:45.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.806+0000 7f22b6a00640 1 -- 172.21.15.86:0/941561992 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f22b0016090 con 0x7f22c006e970 2024-04-25T14:09:45.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.966+0000 7f22c5a00640 1 -- 172.21.15.86:0/941561992 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22c00b94c0 con 0x7f22c006e970 2024-04-25T14:09:45.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.970+0000 7f22b6a00640 1 -- 172.21.15.86:0/941561992 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+17374 (secure 0 0 0) 0x7f22b00f5820 con 0x7f22c006e970 2024-04-25T14:09:45.967 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:45.967 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":101,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:09:43.313900+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:09:23.588241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T14:09:24.156880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2887621339":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/361929172":"2024-04-26T14:09:43.313864+0000","172.21.15.86:6833/4236874185":"2024-04-26T14:09:43.313864+0000","172.21.15.86:6832/4236874185":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/789182356":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/1532315242":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/1775030266":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/2226990349":"2024-04-26T14:09:43.313864+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:09:45.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.970+0000 7f22c5a00640 1 -- 172.21.15.86:0/941561992 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f22a006f490 msgr2=0x7f22a00719a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:09:45.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.970+0000 7f22c5a00640 1 --2- 172.21.15.86:0/941561992 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f22a006f490 0x7f22a00719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:45.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.970+0000 7f22c5a00640 1 -- 172.21.15.86:0/941561992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22c006e970 msgr2=0x7f22c0155730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:45.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.970+0000 7f22c5a00640 1 --2- 172.21.15.86:0/941561992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22c006e970 0x7f22c0155730 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f22b00ac000 tx=0x7f22b0019240 comp rx=0 tx=0).stop 2024-04-25T14:09:45.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.970+0000 7f22c5a00640 1 -- 172.21.15.86:0/941561992 shutdown_connections 2024-04-25T14:09:45.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.970+0000 7f22c5a00640 1 --2- 172.21.15.86:0/941561992 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f22a006f490 0x7f22a00719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:45.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.970+0000 7f22c5a00640 1 --2- 172.21.15.86:0/941561992 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c00cf970 0x7f22c00b6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:45.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.970+0000 7f22c5a00640 1 --2- 172.21.15.86:0/941561992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22c0071360 0x7f22c00b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:45.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.970+0000 7f22c5a00640 1 --2- 172.21.15.86:0/941561992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22c006e970 0x7f22c0155730 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:45.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.970+0000 7f22c5a00640 1 -- 172.21.15.86:0/941561992 >> 172.21.15.86:0/941561992 conn(0x7f22c00cffc0 msgr2=0x7f22c00b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:45.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.970+0000 7f22c5a00640 1 -- 172.21.15.86:0/941561992 shutdown_connections 2024-04-25T14:09:45.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:45.970+0000 7f22c5a00640 1 -- 172.21.15.86:0/941561992 wait complete. 2024-04-25T14:09:45.990 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:09:23.588241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:09:24.156880+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:09:45.990 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:09:46.049 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:46.051+0000 7f6dd142c180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:09:46.066 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:46.070+0000 7fb1a9257180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:09:46.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.146+0000 7f458a200640 1 Processor -- start 2024-04-25T14:09:46.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.146+0000 7f458a200640 1 -- start start 2024-04-25T14:09:46.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.150+0000 7f458a200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4584071530 0x7f45840ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:46.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.150+0000 7f458a200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f458406e700 0x7f458406eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:46.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.150+0000 7f458a200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f458406f050 con 0x7f45840cdd70 2024-04-25T14:09:46.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.150+0000 7f458a200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f458406d940 con 0x7f458406e700 2024-04-25T14:09:46.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.150+0000 7f458a200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f458406dc20 con 0x7f4584071530 2024-04-25T14:09:46.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.150+0000 7f4583e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f458406e700 0x7f458406eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:46.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.150+0000 7f4582a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4584071530 0x7f45840ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:46.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.150+0000 7f4583400640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f45840cdd70 0x7f4584070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:56296/0 (socket says 172.21.15.86:56296) 2024-04-25T14:09:46.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.150+0000 7f4582a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4584071530 0x7f45840ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:44750/0 (socket says 172.21.15.86:44750) 2024-04-25T14:09:46.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.150+0000 7f4583e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f458406e700 0x7f458406eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:48490/0 (socket says 172.21.15.86:48490) 2024-04-25T14:09:46.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.150+0000 7f4583400640 1 -- 172.21.15.86:0/3570711208 learned_addr learned my addr 172.21.15.86:0/3570711208 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:46.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.150+0000 7f4583e00640 1 -- 172.21.15.86:0/3570711208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4584071530 msgr2=0x7f45840ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:46.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.150+0000 7f4583e00640 1 --2- 172.21.15.86:0/3570711208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4584071530 0x7f45840ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.150+0000 7f4583e00640 1 -- 172.21.15.86:0/3570711208 >> v1:172.21.15.155:6789/0 conn(0x7f45840cdd70 legacy=0x7f4584070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:46.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.150+0000 7f4583e00640 1 -- 172.21.15.86:0/3570711208 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458406df00 con 0x7f458406e700 2024-04-25T14:09:46.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.154+0000 7f4583e00640 1 --2- 172.21.15.86:0/3570711208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f458406e700 0x7f458406eb10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f457c009d90 tx=0x7f457c01c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de8ce1096be45e04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:46.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.154+0000 7f4582000640 1 -- 172.21.15.86:0/3570711208 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c0b2070 con 0x7f458406e700 2024-04-25T14:09:46.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.154+0000 7f4582000640 1 -- 172.21.15.86:0/3570711208 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f457c0a57f0 con 0x7f458406e700 2024-04-25T14:09:46.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.154+0000 7f4582000640 1 -- 172.21.15.86:0/3570711208 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c0a5bc0 con 0x7f458406e700 2024-04-25T14:09:46.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.154+0000 7f458a200640 1 -- 172.21.15.86:0/3570711208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f458406e700 msgr2=0x7f458406eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:46.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.154+0000 7f458a200640 1 --2- 172.21.15.86:0/3570711208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f458406e700 0x7f458406eb10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f457c009d90 tx=0x7f457c01c8e0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.154+0000 7f458a200640 1 -- 172.21.15.86:0/3570711208 shutdown_connections 2024-04-25T14:09:46.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.154+0000 7f458a200640 1 --2- 172.21.15.86:0/3570711208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f458406e700 0x7f458406eb10 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.154+0000 7f458a200640 1 --2- 172.21.15.86:0/3570711208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4584071530 0x7f45840ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.154+0000 7f458a200640 1 -- 172.21.15.86:0/3570711208 >> 172.21.15.86:0/3570711208 conn(0x7f45840cffc0 msgr2=0x7f45840d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:46.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.154+0000 7f458a200640 1 -- 172.21.15.86:0/3570711208 shutdown_connections 2024-04-25T14:09:46.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.154+0000 7f458a200640 1 -- 172.21.15.86:0/3570711208 wait complete. 2024-04-25T14:09:46.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.154+0000 7f458a200640 1 Processor -- start 2024-04-25T14:09:46.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.154+0000 7f458a200640 1 -- start start 2024-04-25T14:09:46.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.158+0000 7f458a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f458406e700 0x7f458415cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:46.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.158+0000 7f458a200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45840cdd70 0x7f458414abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:46.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.158+0000 7f458a200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45840b7ac0 0x7f45840b7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:46.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.158+0000 7f458a200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f458416cc00 con 0x7f458406e700 2024-04-25T14:09:46.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.158+0000 7f458a200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4584164380 con 0x7f45840b7ac0 2024-04-25T14:09:46.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.158+0000 7f458a200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f458416cd80 con 0x7f45840cdd70 2024-04-25T14:09:46.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.158+0000 7f4583400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f458406e700 0x7f458415cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:46.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.158+0000 7f4583400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f458406e700 0x7f458415cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48724/0 (socket says 172.21.15.86:48724) 2024-04-25T14:09:46.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.158+0000 7f4583400640 1 -- 172.21.15.86:0/3166674595 learned_addr learned my addr 172.21.15.86:0/3166674595 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:46.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.158+0000 7f4582a00640 1 --2- 172.21.15.86:0/3166674595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45840cdd70 0x7f458414abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:46.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.158+0000 7f4583e00640 1 --2- 172.21.15.86:0/3166674595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45840b7ac0 0x7f45840b7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:46.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.158+0000 7f4583400640 1 -- 172.21.15.86:0/3166674595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45840cdd70 msgr2=0x7f458414abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:46.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.158+0000 7f4583400640 1 --2- 172.21.15.86:0/3166674595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45840cdd70 0x7f458414abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.158+0000 7f4583400640 1 -- 172.21.15.86:0/3166674595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45840b7ac0 msgr2=0x7f45840b7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:46.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.158+0000 7f4583400640 1 --2- 172.21.15.86:0/3166674595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45840b7ac0 0x7f45840b7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.158+0000 7f4583400640 1 -- 172.21.15.86:0/3166674595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45840b8410 con 0x7f458406e700 2024-04-25T14:09:46.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.158+0000 7f4583400640 1 --2- 172.21.15.86:0/3166674595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f458406e700 0x7f458415cb70 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f457400c370 tx=0x7f457400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:46.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.158+0000 7f457b400640 1 -- 172.21.15.86:0/3166674595 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4574017070 con 0x7f458406e700 2024-04-25T14:09:46.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.158+0000 7f457b400640 1 -- 172.21.15.86:0/3166674595 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4574005cf0 con 0x7f458406e700 2024-04-25T14:09:46.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.158+0000 7f458a200640 1 -- 172.21.15.86:0/3166674595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45840b8690 con 0x7f458406e700 2024-04-25T14:09:46.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.158+0000 7f457b400640 1 -- 172.21.15.86:0/3166674595 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45740072f0 con 0x7f458406e700 2024-04-25T14:09:46.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.158+0000 7f458a200640 1 -- 172.21.15.86:0/3166674595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45840b8c10 con 0x7f458406e700 2024-04-25T14:09:46.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.162+0000 7f458a200640 1 -- 172.21.15.86:0/3166674595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45840720e0 con 0x7f458406e700 2024-04-25T14:09:46.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.162+0000 7f457b400640 1 -- 172.21.15.86:0/3166674595 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 41) ==== 93253+0+0 (secure 0 0 0) 0x7f457401a070 con 0x7f458406e700 2024-04-25T14:09:46.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.162+0000 7f457b400640 1 --2- 172.21.15.86:0/3166674595 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f456406f490 0x7f45640719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:46.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.162+0000 7f457b400640 1 -- 172.21.15.86:0/3166674595 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6445+0+0 (secure 0 0 0) 0x7f457408e9b0 con 0x7f458406e700 2024-04-25T14:09:46.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.162+0000 7f4582a00640 1 -- 172.21.15.86:0/3166674595 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f456406f490 msgr2=0x7f45640719a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/436174229 2024-04-25T14:09:46.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.162+0000 7f4582a00640 1 --2- 172.21.15.86:0/3166674595 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f456406f490 0x7f45640719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:09:46.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.166+0000 7f457b400640 1 -- 172.21.15.86:0/3166674595 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f45840b8410 con 0x7f458406e700 2024-04-25T14:09:46.199 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:46.199+0000 7f6dd142c180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:09:46.221 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:46.222+0000 7fb1a9257180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:09:46.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.326+0000 7f458a200640 1 -- 172.21.15.86:0/3166674595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4584076bf0 con 0x7f458406e700 2024-04-25T14:09:46.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.326+0000 7f457b400640 1 -- 172.21.15.86:0/3166674595 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) ==== 93+0+10 (secure 0 0 0) 0x7f457405ae80 con 0x7f458406e700 2024-04-25T14:09:46.325 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:09:46.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.330+0000 7f458a200640 1 -- 172.21.15.86:0/3166674595 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f456406f490 msgr2=0x7f45640719a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:09:46.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.330+0000 7f458a200640 1 --2- 172.21.15.86:0/3166674595 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f456406f490 0x7f45640719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.330+0000 7f458a200640 1 -- 172.21.15.86:0/3166674595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f458406e700 msgr2=0x7f458415cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:46.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.330+0000 7f458a200640 1 --2- 172.21.15.86:0/3166674595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f458406e700 0x7f458415cb70 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f457400c370 tx=0x7f457400c830 comp rx=0 tx=0).stop 2024-04-25T14:09:46.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.330+0000 7f458a200640 1 -- 172.21.15.86:0/3166674595 shutdown_connections 2024-04-25T14:09:46.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.330+0000 7f458a200640 1 --2- 172.21.15.86:0/3166674595 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f456406f490 0x7f45640719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.330+0000 7f458a200640 1 --2- 172.21.15.86:0/3166674595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45840b7ac0 0x7f45840b7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.330+0000 7f458a200640 1 --2- 172.21.15.86:0/3166674595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45840cdd70 0x7f458414abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.330+0000 7f458a200640 1 --2- 172.21.15.86:0/3166674595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f458406e700 0x7f458415cb70 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.330+0000 7f458a200640 1 -- 172.21.15.86:0/3166674595 >> 172.21.15.86:0/3166674595 conn(0x7f45840cffc0 msgr2=0x7f4584070e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.330+0000 7f458a200640 1 -- 172.21.15.86:0/3166674595 shutdown_connections 2024-04-25T14:09:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.330+0000 7f458a200640 1 -- 172.21.15.86:0/3166674595 wait complete. 2024-04-25T14:09:46.354 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:09:46.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.510+0000 7ff4c4600640 1 Processor -- start 2024-04-25T14:09:46.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.510+0000 7ff4c4600640 1 -- start start 2024-04-25T14:09:46.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.510+0000 7ff4c4600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff4bc071530 0x7ff4bc163810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:46.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.510+0000 7ff4c4600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4bc163d50 0x7ff4bc0cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:46.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.510+0000 7ff4c4600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4bc06f0a0 con 0x7ff4bc15fc70 2024-04-25T14:09:46.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.510+0000 7ff4c4600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff4bc06e7f0 con 0x7ff4bc071530 2024-04-25T14:09:46.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.510+0000 7ff4c4600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff4bc164160 con 0x7ff4bc163d50 2024-04-25T14:09:46.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.510+0000 7ff4c1200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff4bc071530 0x7ff4bc163810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:46.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.510+0000 7ff4c1c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff4bc15fc70 0x7ff4bc070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41680/0 (socket says 172.21.15.86:41680) 2024-04-25T14:09:46.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.510+0000 7ff4c1c00640 1 -- 172.21.15.86:0/1651051536 learned_addr learned my addr 172.21.15.86:0/1651051536 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:46.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.510+0000 7ff4c2600640 1 --2- 172.21.15.86:0/1651051536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4bc163d50 0x7ff4bc0cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:46.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.510+0000 7ff4c1200640 1 -- 172.21.15.86:0/1651051536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4bc163d50 msgr2=0x7ff4bc0cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:46.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.510+0000 7ff4c1200640 1 --2- 172.21.15.86:0/1651051536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4bc163d50 0x7ff4bc0cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.510+0000 7ff4c1200640 1 -- 172.21.15.86:0/1651051536 >> v1:172.21.15.86:6789/0 conn(0x7ff4bc15fc70 legacy=0x7ff4bc070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:46.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.510+0000 7ff4c1200640 1 -- 172.21.15.86:0/1651051536 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4bc1646d0 con 0x7ff4bc071530 2024-04-25T14:09:46.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.510+0000 7ff4bbe00640 1 -- 172.21.15.86:0/1651051536 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 788273476 0 0) 0x7ff4b0001120 con 0x7ff4bc15fc70 2024-04-25T14:09:46.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.514+0000 7ff4c1200640 1 --2- 172.21.15.86:0/1651051536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4bc071530 0x7ff4bc163810 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80097c0 tx=0x7ff4a801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=193d0de989c7f16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:46.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.514+0000 7ff4bbe00640 1 -- 172.21.15.86:0/1651051536 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a80aa600 con 0x7ff4bc071530 2024-04-25T14:09:46.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.514+0000 7ff4bbe00640 1 -- 172.21.15.86:0/1651051536 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4a80af070 con 0x7ff4bc071530 2024-04-25T14:09:46.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.514+0000 7ff4bbe00640 1 -- 172.21.15.86:0/1651051536 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a80aa9c0 con 0x7ff4bc071530 2024-04-25T14:09:46.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.514+0000 7ff4c4600640 1 -- 172.21.15.86:0/1651051536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4bc071530 msgr2=0x7ff4bc163810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:46.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.514+0000 7ff4c4600640 1 --2- 172.21.15.86:0/1651051536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4bc071530 0x7ff4bc163810 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80097c0 tx=0x7ff4a801c4a0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.514+0000 7ff4c4600640 1 -- 172.21.15.86:0/1651051536 shutdown_connections 2024-04-25T14:09:46.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.514+0000 7ff4c4600640 1 --2- 172.21.15.86:0/1651051536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4bc163d50 0x7ff4bc0cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.514+0000 7ff4c4600640 1 --2- 172.21.15.86:0/1651051536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4bc071530 0x7ff4bc163810 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.514+0000 7ff4c4600640 1 -- 172.21.15.86:0/1651051536 >> 172.21.15.86:0/1651051536 conn(0x7ff4bc0cffc0 msgr2=0x7ff4bc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:46.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.514+0000 7ff4c4600640 1 -- 172.21.15.86:0/1651051536 shutdown_connections 2024-04-25T14:09:46.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.514+0000 7ff4c4600640 1 -- 172.21.15.86:0/1651051536 wait complete. 2024-04-25T14:09:46.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.518+0000 7ff4c4600640 1 Processor -- start 2024-04-25T14:09:46.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.518+0000 7ff4c4600640 1 -- start start 2024-04-25T14:09:46.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.518+0000 7ff4c4600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4bc15fc70 0x7ff4bc153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:46.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.518+0000 7ff4c4600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4bc163d50 0x7ff4bc153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:46.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.518+0000 7ff4c4600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4bc1fbcc0 0x7ff4bc1fc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:46.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.518+0000 7ff4c4600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc16cb70 con 0x7ff4bc15fc70 2024-04-25T14:09:46.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.518+0000 7ff4c4600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc160080 con 0x7ff4bc1fbcc0 2024-04-25T14:09:46.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.518+0000 7ff4c4600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff4bc16ccf0 con 0x7ff4bc163d50 2024-04-25T14:09:46.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.518+0000 7ff4c1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4bc15fc70 0x7ff4bc153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:46.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.518+0000 7ff4c1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4bc15fc70 0x7ff4bc153580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48738/0 (socket says 172.21.15.86:48738) 2024-04-25T14:09:46.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.518+0000 7ff4c1c00640 1 -- 172.21.15.86:0/89889145 learned_addr learned my addr 172.21.15.86:0/89889145 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:46.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.518+0000 7ff4c1200640 1 --2- 172.21.15.86:0/89889145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4bc163d50 0x7ff4bc153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:46.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.518+0000 7ff4c2600640 1 --2- 172.21.15.86:0/89889145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4bc1fbcc0 0x7ff4bc1fc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:46.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.522+0000 7ff4c1c00640 1 -- 172.21.15.86:0/89889145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4bc163d50 msgr2=0x7ff4bc153ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:46.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.522+0000 7ff4c1c00640 1 --2- 172.21.15.86:0/89889145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4bc163d50 0x7ff4bc153ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.522+0000 7ff4c1c00640 1 -- 172.21.15.86:0/89889145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4bc1fbcc0 msgr2=0x7ff4bc1fc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:46.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.522+0000 7ff4c1c00640 1 --2- 172.21.15.86:0/89889145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4bc1fbcc0 0x7ff4bc1fc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.522+0000 7ff4c1c00640 1 -- 172.21.15.86:0/89889145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4bc0ca7d0 con 0x7ff4bc15fc70 2024-04-25T14:09:46.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.522+0000 7ff4c1c00640 1 --2- 172.21.15.86:0/89889145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4bc15fc70 0x7ff4bc153580 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000c4b0 tx=0x7ff4b000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:46.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.522+0000 7ff4b9600640 1 -- 172.21.15.86:0/89889145 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0018070 con 0x7ff4bc15fc70 2024-04-25T14:09:46.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.522+0000 7ff4b9600640 1 -- 172.21.15.86:0/89889145 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b0007500 con 0x7ff4bc15fc70 2024-04-25T14:09:46.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.522+0000 7ff4c4600640 1 -- 172.21.15.86:0/89889145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4bc0caab0 con 0x7ff4bc15fc70 2024-04-25T14:09:46.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.522+0000 7ff4b9600640 1 -- 172.21.15.86:0/89889145 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0004a30 con 0x7ff4bc15fc70 2024-04-25T14:09:46.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.522+0000 7ff4c4600640 1 -- 172.21.15.86:0/89889145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4bc0cb070 con 0x7ff4bc15fc70 2024-04-25T14:09:46.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.522+0000 7ff4c4600640 1 -- 172.21.15.86:0/89889145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4bc0720e0 con 0x7ff4bc15fc70 2024-04-25T14:09:46.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.522+0000 7ff4b9600640 1 -- 172.21.15.86:0/89889145 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 41) ==== 93253+0+0 (secure 0 0 0) 0x7ff4b0004bd0 con 0x7ff4bc15fc70 2024-04-25T14:09:46.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.522+0000 7ff4b9600640 1 --2- 172.21.15.86:0/89889145 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7ff49c06f490 0x7ff49c0719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:46.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.526+0000 7ff4c1200640 1 -- 172.21.15.86:0/89889145 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7ff49c06f490 msgr2=0x7ff49c0719a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/436174229 2024-04-25T14:09:46.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.526+0000 7ff4b9600640 1 -- 172.21.15.86:0/89889145 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6445+0+0 (secure 0 0 0) 0x7ff4b008e790 con 0x7ff4bc15fc70 2024-04-25T14:09:46.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.526+0000 7ff4c1200640 1 --2- 172.21.15.86:0/89889145 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7ff49c06f490 0x7ff49c0719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:09:46.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.530+0000 7ff4b9600640 1 -- 172.21.15.86:0/89889145 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff4bc06f0a0 con 0x7ff4bc15fc70 2024-04-25T14:09:46.671 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:46.671+0000 7f6dd142c180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:09:46.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.686+0000 7ff4c4600640 1 -- 172.21.15.86:0/89889145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff4bc076bf0 con 0x7ff4bc15fc70 2024-04-25T14:09:46.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.690+0000 7ff4b9600640 1 -- 172.21.15.86:0/89889145 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) ==== 104+0+11 (secure 0 0 0) 0x7ff4b00633b0 con 0x7ff4bc15fc70 2024-04-25T14:09:46.686 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:09:46.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.690+0000 7ff4c4600640 1 -- 172.21.15.86:0/89889145 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7ff49c06f490 msgr2=0x7ff49c0719a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:09:46.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.690+0000 7ff4c4600640 1 --2- 172.21.15.86:0/89889145 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7ff49c06f490 0x7ff49c0719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.690+0000 7ff4c4600640 1 -- 172.21.15.86:0/89889145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4bc15fc70 msgr2=0x7ff4bc153580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:46.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.690+0000 7ff4c4600640 1 --2- 172.21.15.86:0/89889145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4bc15fc70 0x7ff4bc153580 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000c4b0 tx=0x7ff4b000c970 comp rx=0 tx=0).stop 2024-04-25T14:09:46.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.690+0000 7ff4c4600640 1 -- 172.21.15.86:0/89889145 shutdown_connections 2024-04-25T14:09:46.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.690+0000 7ff4c4600640 1 --2- 172.21.15.86:0/89889145 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7ff49c06f490 0x7ff49c0719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.690+0000 7ff4c4600640 1 --2- 172.21.15.86:0/89889145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4bc1fbcc0 0x7ff4bc1fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.690+0000 7ff4c4600640 1 --2- 172.21.15.86:0/89889145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4bc163d50 0x7ff4bc153ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.690+0000 7ff4c4600640 1 --2- 172.21.15.86:0/89889145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4bc15fc70 0x7ff4bc153580 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.690+0000 7ff4c4600640 1 -- 172.21.15.86:0/89889145 >> 172.21.15.86:0/89889145 conn(0x7ff4bc0cffc0 msgr2=0x7ff4bc160f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:46.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.690+0000 7ff4c4600640 1 -- 172.21.15.86:0/89889145 shutdown_connections 2024-04-25T14:09:46.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.690+0000 7ff4c4600640 1 -- 172.21.15.86:0/89889145 wait complete. 2024-04-25T14:09:46.708 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:09:46.719 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:46.722+0000 7fb1a9257180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:09:46.729 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:46.731+0000 7f6dd142c180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:09:46.781 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:46.786+0000 7fb1a9257180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:09:46.786 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:46.787+0000 7f6dd142c180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:09:46.840 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:46.842+0000 7fb1a9257180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:09:46.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.866+0000 7f783d000640 1 Processor -- start 2024-04-25T14:09:46.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.866+0000 7f783d000640 1 -- start start 2024-04-25T14:09:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.870+0000 7f783d000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78380d2c80 0x7f783815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.870+0000 7f783d000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78380cd8d0 con 0x7f78381616e0 2024-04-25T14:09:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.870+0000 7f783d000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78380ce970 con 0x7f7838070e20 2024-04-25T14:09:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.870+0000 7f783d000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f78380ce720 con 0x7f78380d2c80 2024-04-25T14:09:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.870+0000 7f7836800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f78381616e0 0x7f7838161af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41688/0 (socket says 172.21.15.86:41688) 2024-04-25T14:09:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.870+0000 7f7836800640 1 -- 172.21.15.86:0/1740348175 learned_addr learned my addr 172.21.15.86:0/1740348175 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.870+0000 7f7835e00640 1 --2- 172.21.15.86:0/1740348175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78380d2c80 0x7f783815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.870+0000 7f7835400640 1 -- 172.21.15.86:0/1740348175 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2896981921 0 0) 0x7f78380cd8d0 con 0x7f78381616e0 2024-04-25T14:09:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.870+0000 7f7835400640 1 -- 172.21.15.86:0/1740348175 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f781c003530 con 0x7f78381616e0 2024-04-25T14:09:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.870+0000 7f7835400640 1 -- 172.21.15.86:0/1740348175 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 494757142 0 0) 0x7f78380ce970 con 0x7f7838070e20 2024-04-25T14:09:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.870+0000 7f7835e00640 1 -- 172.21.15.86:0/1740348175 >> v1:172.21.15.155:6789/0 conn(0x7f7838070e20 legacy=0x7f7838071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.870+0000 7f7835e00640 1 -- 172.21.15.86:0/1740348175 >> v1:172.21.15.86:6789/0 conn(0x7f78381616e0 legacy=0x7f7838161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.870+0000 7f7835e00640 1 -- 172.21.15.86:0/1740348175 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78380d1be0 con 0x7f78380d2c80 2024-04-25T14:09:46.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.874+0000 7f7835e00640 1 --2- 172.21.15.86:0/1740348175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78380d2c80 0x7f783815dc90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f78280097f0 tx=0x7f782801ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6093f543a1481b52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:46.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.874+0000 7f7835400640 1 -- 172.21.15.86:0/1740348175 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78280b5070 con 0x7f78380d2c80 2024-04-25T14:09:46.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.874+0000 7f7835400640 1 -- 172.21.15.86:0/1740348175 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78280a89d0 con 0x7f78380d2c80 2024-04-25T14:09:46.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.874+0000 7f7835400640 1 -- 172.21.15.86:0/1740348175 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78280a8d80 con 0x7f78380d2c80 2024-04-25T14:09:46.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.874+0000 7f783d000640 1 -- 172.21.15.86:0/1740348175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78380d2c80 msgr2=0x7f783815dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:46.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.874+0000 7f783d000640 1 --2- 172.21.15.86:0/1740348175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78380d2c80 0x7f783815dc90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f78280097f0 tx=0x7f782801ecd0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.874+0000 7f783d000640 1 -- 172.21.15.86:0/1740348175 shutdown_connections 2024-04-25T14:09:46.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.874+0000 7f783d000640 1 --2- 172.21.15.86:0/1740348175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78380d2c80 0x7f783815dc90 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.874+0000 7f783d000640 1 -- 172.21.15.86:0/1740348175 >> 172.21.15.86:0/1740348175 conn(0x7f78380cffc0 msgr2=0x7f78380d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:46.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.874+0000 7f783d000640 1 -- 172.21.15.86:0/1740348175 shutdown_connections 2024-04-25T14:09:46.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.874+0000 7f783d000640 1 -- 172.21.15.86:0/1740348175 wait complete. 2024-04-25T14:09:46.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.878+0000 7f783d000640 1 Processor -- start 2024-04-25T14:09:46.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.878+0000 7f783d000640 1 -- start start 2024-04-25T14:09:46.873 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:46.875+0000 7f6dd142c180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:09:46.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.878+0000 7f783d000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7838070e20 0x7f7838158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:46.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.878+0000 7f783d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78380d2c80 0x7f783815e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:46.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.878+0000 7f783d000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78381616e0 0x7f78380c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:46.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.878+0000 7f783d000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78380d2480 con 0x7f78380d2c80 2024-04-25T14:09:46.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.878+0000 7f783d000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f78380d21a0 con 0x7f7838070e20 2024-04-25T14:09:46.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.878+0000 7f783d000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f78380d29a0 con 0x7f78381616e0 2024-04-25T14:09:46.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.878+0000 7f782fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78380d2c80 0x7f783815e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:46.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.878+0000 7f7836800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78381616e0 0x7f78380c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:46.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.878+0000 7f782fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78380d2c80 0x7f783815e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48740/0 (socket says 172.21.15.86:48740) 2024-04-25T14:09:46.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.878+0000 7f7835e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7838070e20 0x7f7838158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:46.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.878+0000 7f782fe00640 1 -- 172.21.15.86:0/2080441654 learned_addr learned my addr 172.21.15.86:0/2080441654 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.878+0000 7f7835e00640 1 -- 172.21.15.86:0/2080441654 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78381616e0 msgr2=0x7f78380c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.878+0000 7f7835e00640 1 --2- 172.21.15.86:0/2080441654 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78381616e0 0x7f78380c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.878+0000 7f7835e00640 1 -- 172.21.15.86:0/2080441654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78380d2c80 msgr2=0x7f783815e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.878+0000 7f7835e00640 1 --2- 172.21.15.86:0/2080441654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78380d2c80 0x7f783815e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.878+0000 7f7835e00640 1 -- 172.21.15.86:0/2080441654 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783815ff30 con 0x7f7838070e20 2024-04-25T14:09:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.878+0000 7f7836800640 1 --2- 172.21.15.86:0/2080441654 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78381616e0 0x7f78380c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:09:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.878+0000 7f782fe00640 1 --2- 172.21.15.86:0/2080441654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78380d2c80 0x7f783815e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:09:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.878+0000 7f7835e00640 1 --2- 172.21.15.86:0/2080441654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7838070e20 0x7f7838158c40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f78280af000 tx=0x7f782801ba70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.878+0000 7f782e000640 1 -- 172.21.15.86:0/2080441654 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78280b5040 con 0x7f7838070e20 2024-04-25T14:09:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.878+0000 7f782e000640 1 -- 172.21.15.86:0/2080441654 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78280c0070 con 0x7f7838070e20 2024-04-25T14:09:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.878+0000 7f782e000640 1 -- 172.21.15.86:0/2080441654 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78280b0920 con 0x7f7838070e20 2024-04-25T14:09:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.878+0000 7f783d000640 1 -- 172.21.15.86:0/2080441654 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7838160210 con 0x7f7838070e20 2024-04-25T14:09:46.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.878+0000 7f783d000640 1 -- 172.21.15.86:0/2080441654 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78381607d0 con 0x7f7838070e20 2024-04-25T14:09:46.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.882+0000 7f783d000640 1 -- 172.21.15.86:0/2080441654 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7838076bf0 con 0x7f7838070e20 2024-04-25T14:09:46.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.882+0000 7f782e000640 1 -- 172.21.15.86:0/2080441654 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 41) ==== 93253+0+0 (secure 0 0 0) 0x7f78280b0ac0 con 0x7f7838070e20 2024-04-25T14:09:46.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.882+0000 7f782e000640 1 --2- 172.21.15.86:0/2080441654 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f781406f490 0x7f78140719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:46.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.882+0000 7f782e000640 1 -- 172.21.15.86:0/2080441654 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6445+0+0 (secure 0 0 0) 0x7f782812c710 con 0x7f7838070e20 2024-04-25T14:09:46.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.882+0000 7f782fe00640 1 -- 172.21.15.86:0/2080441654 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f781406f490 msgr2=0x7f78140719a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/436174229 2024-04-25T14:09:46.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.882+0000 7f782fe00640 1 --2- 172.21.15.86:0/2080441654 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f781406f490 0x7f78140719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:09:46.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:46.890+0000 7f782e000640 1 -- 172.21.15.86:0/2080441654 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7828018830 con 0x7f7838070e20 2024-04-25T14:09:46.931 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:46.931+0000 7f6dd142c180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:09:46.932 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:46.934+0000 7fb1a9257180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:09:46.990 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:46.994+0000 7fb1a9257180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:09:47.016 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:47.015+0000 7f6dd142c180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:09:47.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.046+0000 7f783d000640 1 -- 172.21.15.86:0/2080441654 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f78380d1470 con 0x7f7838070e20 2024-04-25T14:09:47.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.046+0000 7f782e000640 1 -- 172.21.15.86:0/2080441654 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) ==== 100+0+11 (secure 0 0 0) 0x7f78280f8ce0 con 0x7f7838070e20 2024-04-25T14:09:47.044 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:09:47.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.050+0000 7f783d000640 1 -- 172.21.15.86:0/2080441654 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f781406f490 msgr2=0x7f78140719a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:09:47.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.050+0000 7f783d000640 1 --2- 172.21.15.86:0/2080441654 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f781406f490 0x7f78140719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.050+0000 7f783d000640 1 -- 172.21.15.86:0/2080441654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7838070e20 msgr2=0x7f7838158c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:47.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.050+0000 7f783d000640 1 --2- 172.21.15.86:0/2080441654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7838070e20 0x7f7838158c40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f78280af000 tx=0x7f782801ba70 comp rx=0 tx=0).stop 2024-04-25T14:09:47.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.050+0000 7f783d000640 1 -- 172.21.15.86:0/2080441654 shutdown_connections 2024-04-25T14:09:47.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.050+0000 7f783d000640 1 --2- 172.21.15.86:0/2080441654 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f781406f490 0x7f78140719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.050+0000 7f783d000640 1 --2- 172.21.15.86:0/2080441654 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78381616e0 0x7f78380c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.050+0000 7f783d000640 1 --2- 172.21.15.86:0/2080441654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78380d2c80 0x7f783815e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.050+0000 7f783d000640 1 --2- 172.21.15.86:0/2080441654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7838070e20 0x7f7838158c40 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.050+0000 7f783d000640 1 -- 172.21.15.86:0/2080441654 >> 172.21.15.86:0/2080441654 conn(0x7f78380cffc0 msgr2=0x7f78380b53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:47.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.050+0000 7f783d000640 1 -- 172.21.15.86:0/2080441654 shutdown_connections 2024-04-25T14:09:47.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.050+0000 7f783d000640 1 -- 172.21.15.86:0/2080441654 wait complete. 2024-04-25T14:09:47.067 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T14:09:47.075 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:47.075+0000 7f6dd142c180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:09:47.076 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:47.078+0000 7fb1a9257180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:09:47.138 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:47.142+0000 7fb1a9257180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:09:47.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.230+0000 7fb63fc00640 1 Processor -- start 2024-04-25T14:09:47.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.230+0000 7fb63fc00640 1 -- start start 2024-04-25T14:09:47.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.230+0000 7fb63fc00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb6380ccab0 0x7fb6380d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:47.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.230+0000 7fb63fc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb6380cf9d0 0x7fb6380d2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:47.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.230+0000 7fb63fc00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6380b3bd0 con 0x7fb6380b79c0 2024-04-25T14:09:47.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.230+0000 7fb63fc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb6380cf190 con 0x7fb6380cf9d0 2024-04-25T14:09:47.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.230+0000 7fb63fc00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb6380b2620 con 0x7fb6380ccab0 2024-04-25T14:09:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.230+0000 7fb63dc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb6380cf9d0 0x7fb6380d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.230+0000 7fb63dc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb6380cf9d0 0x7fb6380d2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48744/0 (socket says 172.21.15.86:48744) 2024-04-25T14:09:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.230+0000 7fb63dc00640 1 -- 172.21.15.86:0/3112980390 learned_addr learned my addr 172.21.15.86:0/3112980390 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:47.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.230+0000 7fb63d200640 1 --2- 172.21.15.86:0/3112980390 >> v2:172.21.15.155:3300/0 conn(0x7fb6380ccab0 0x7fb6380d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.230+0000 7fb63dc00640 1 -- 172.21.15.86:0/3112980390 >> v2:172.21.15.155:3300/0 conn(0x7fb6380ccab0 msgr2=0x7fb6380d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.230+0000 7fb63dc00640 1 --2- 172.21.15.86:0/3112980390 >> v2:172.21.15.155:3300/0 conn(0x7fb6380ccab0 0x7fb6380d23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.230+0000 7fb63dc00640 1 -- 172.21.15.86:0/3112980390 >> v1:172.21.15.155:6789/0 conn(0x7fb6380b79c0 legacy=0x7fb6381617f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:47.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.230+0000 7fb63dc00640 1 -- 172.21.15.86:0/3112980390 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6380b1cf0 con 0x7fb6380cf9d0 2024-04-25T14:09:47.229 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:47.227+0000 7f6dd142c180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:09:47.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.234+0000 7fb63dc00640 1 --2- 172.21.15.86:0/3112980390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6380cf9d0 0x7fb6380d2910 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fb638073230 tx=0x7fb63001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143e30c0acecef8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:47.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.234+0000 7fb637400640 1 -- 172.21.15.86:0/3112980390 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6300a56e0 con 0x7fb6380cf9d0 2024-04-25T14:09:47.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.234+0000 7fb637400640 1 -- 172.21.15.86:0/3112980390 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6300a5880 con 0x7fb6380cf9d0 2024-04-25T14:09:47.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.234+0000 7fb637400640 1 -- 172.21.15.86:0/3112980390 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6300a5c50 con 0x7fb6380cf9d0 2024-04-25T14:09:47.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.234+0000 7fb63fc00640 1 -- 172.21.15.86:0/3112980390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6380cf9d0 msgr2=0x7fb6380d2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:47.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.234+0000 7fb63fc00640 1 --2- 172.21.15.86:0/3112980390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6380cf9d0 0x7fb6380d2910 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fb638073230 tx=0x7fb63001c9f0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.234+0000 7fb63fc00640 1 -- 172.21.15.86:0/3112980390 shutdown_connections 2024-04-25T14:09:47.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.234+0000 7fb63fc00640 1 --2- 172.21.15.86:0/3112980390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6380cf9d0 0x7fb6380d2910 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.234+0000 7fb63fc00640 1 --2- 172.21.15.86:0/3112980390 >> v2:172.21.15.155:3300/0 conn(0x7fb6380ccab0 0x7fb6380d23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.234+0000 7fb63fc00640 1 -- 172.21.15.86:0/3112980390 >> 172.21.15.86:0/3112980390 conn(0x7fb63809ba30 msgr2=0x7fb6380b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:47.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.234+0000 7fb63fc00640 1 -- 172.21.15.86:0/3112980390 shutdown_connections 2024-04-25T14:09:47.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.234+0000 7fb63fc00640 1 -- 172.21.15.86:0/3112980390 wait complete. 2024-04-25T14:09:47.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.238+0000 7fb63fc00640 1 Processor -- start 2024-04-25T14:09:47.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.238+0000 7fb63fc00640 1 -- start start 2024-04-25T14:09:47.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.238+0000 7fb63fc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6380b79c0 0x7fb6380af140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:47.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.238+0000 7fb63fc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6380ccab0 0x7fb6380c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:47.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.238+0000 7fb63fc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6380cf9d0 0x7fb6380b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:47.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.238+0000 7fb63fc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb63816ab00 con 0x7fb6380b79c0 2024-04-25T14:09:47.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.238+0000 7fb63fc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb63816a980 con 0x7fb6380cf9d0 2024-04-25T14:09:47.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.238+0000 7fb63fc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb63816ac80 con 0x7fb6380ccab0 2024-04-25T14:09:47.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.238+0000 7fb63d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6380b79c0 0x7fb6380af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:47.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.238+0000 7fb63d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6380b79c0 0x7fb6380af140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48758/0 (socket says 172.21.15.86:48758) 2024-04-25T14:09:47.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.238+0000 7fb63d200640 1 -- 172.21.15.86:0/366972227 learned_addr learned my addr 172.21.15.86:0/366972227 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:47.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.238+0000 7fb63dc00640 1 --2- 172.21.15.86:0/366972227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6380cf9d0 0x7fb6380b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:47.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.238+0000 7fb637e00640 1 --2- 172.21.15.86:0/366972227 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6380ccab0 0x7fb6380c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:47.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.238+0000 7fb63d200640 1 -- 172.21.15.86:0/366972227 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6380ccab0 msgr2=0x7fb6380c5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:47.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.238+0000 7fb63d200640 1 --2- 172.21.15.86:0/366972227 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6380ccab0 0x7fb6380c5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.238+0000 7fb63d200640 1 -- 172.21.15.86:0/366972227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6380cf9d0 msgr2=0x7fb6380b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.238+0000 7fb63d200640 1 --2- 172.21.15.86:0/366972227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6380cf9d0 0x7fb6380b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.238+0000 7fb63d200640 1 -- 172.21.15.86:0/366972227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6380af990 con 0x7fb6380b79c0 2024-04-25T14:09:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.242+0000 7fb63d200640 1 --2- 172.21.15.86:0/366972227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6380b79c0 0x7fb6380af140 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00e8a0 tx=0x7fb62c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.242+0000 7fb634c00640 1 -- 172.21.15.86:0/366972227 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c017070 con 0x7fb6380b79c0 2024-04-25T14:09:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.242+0000 7fb634c00640 1 -- 172.21.15.86:0/366972227 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb62c00c940 con 0x7fb6380b79c0 2024-04-25T14:09:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.242+0000 7fb63fc00640 1 -- 172.21.15.86:0/366972227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6380afc70 con 0x7fb6380b79c0 2024-04-25T14:09:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.242+0000 7fb634c00640 1 -- 172.21.15.86:0/366972227 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c00ccf0 con 0x7fb6380b79c0 2024-04-25T14:09:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.242+0000 7fb63fc00640 1 -- 172.21.15.86:0/366972227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6380b09a0 con 0x7fb6380b79c0 2024-04-25T14:09:47.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.242+0000 7fb63fc00640 1 -- 172.21.15.86:0/366972227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6380a04c0 con 0x7fb6380b79c0 2024-04-25T14:09:47.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.242+0000 7fb634c00640 1 -- 172.21.15.86:0/366972227 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 41) ==== 93253+0+0 (secure 0 0 0) 0x7fb62c01a070 con 0x7fb6380b79c0 2024-04-25T14:09:47.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.242+0000 7fb634c00640 1 --2- 172.21.15.86:0/366972227 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7fb61806f4c0 0x7fb6180719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:47.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.242+0000 7fb637e00640 1 -- 172.21.15.86:0/366972227 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7fb61806f4c0 msgr2=0x7fb6180719d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/436174229 2024-04-25T14:09:47.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.242+0000 7fb634c00640 1 -- 172.21.15.86:0/366972227 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6445+0+0 (secure 0 0 0) 0x7fb62c08f990 con 0x7fb6380b79c0 2024-04-25T14:09:47.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.242+0000 7fb637e00640 1 --2- 172.21.15.86:0/366972227 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7fb61806f4c0 0x7fb6180719d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:09:47.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.254+0000 7fb634c00640 1 -- 172.21.15.86:0/366972227 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb62c05be70 con 0x7fb6380b79c0 2024-04-25T14:09:47.288 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:47.287+0000 7f6dd142c180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:09:47.301 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:47.302+0000 7fb1a9257180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:09:47.360 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:47.362+0000 7fb1a9257180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:09:47.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.422+0000 7fb63fc00640 1 -- 172.21.15.86:0/366972227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb6380d2f90 con 0x7fb6380b79c0 2024-04-25T14:09:47.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.422+0000 7fb634c00640 1 -- 172.21.15.86:0/366972227 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v101) ==== 80+0+468 (secure 0 0 0) 0x7fb62c060170 con 0x7fb6380b79c0 2024-04-25T14:09:47.420 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2887621339 2024-04-26T14:09:43.313864+0000 2024-04-25T14:09:47.420 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/361929172 2024-04-26T14:09:43.313864+0000 2024-04-25T14:09:47.420 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6833/4236874185 2024-04-26T14:09:43.313864+0000 2024-04-25T14:09:47.420 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6832/4236874185 2024-04-26T14:09:43.313864+0000 2024-04-25T14:09:47.420 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/789182356 2024-04-26T14:09:43.313864+0000 2024-04-25T14:09:47.420 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1532315242 2024-04-26T14:09:43.313864+0000 2024-04-25T14:09:47.420 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1775030266 2024-04-26T14:09:43.313864+0000 2024-04-25T14:09:47.420 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2226990349 2024-04-26T14:09:43.313864+0000 2024-04-25T14:09:47.421 INFO:teuthology.orchestra.run.smithi086.stderr:listed 8 entries 2024-04-25T14:09:47.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.426+0000 7fb63fc00640 1 -- 172.21.15.86:0/366972227 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7fb61806f4c0 msgr2=0x7fb6180719d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:09:47.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.426+0000 7fb63fc00640 1 --2- 172.21.15.86:0/366972227 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7fb61806f4c0 0x7fb6180719d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.426+0000 7fb63fc00640 1 -- 172.21.15.86:0/366972227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6380b79c0 msgr2=0x7fb6380af140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:47.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.426+0000 7fb63fc00640 1 --2- 172.21.15.86:0/366972227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6380b79c0 0x7fb6380af140 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00e8a0 tx=0x7fb62c00ed60 comp rx=0 tx=0).stop 2024-04-25T14:09:47.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.426+0000 7fb63fc00640 1 -- 172.21.15.86:0/366972227 shutdown_connections 2024-04-25T14:09:47.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.426+0000 7fb63fc00640 1 --2- 172.21.15.86:0/366972227 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7fb61806f4c0 0x7fb6180719d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.426+0000 7fb63fc00640 1 --2- 172.21.15.86:0/366972227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6380cf9d0 0x7fb6380b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.426+0000 7fb63fc00640 1 --2- 172.21.15.86:0/366972227 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6380ccab0 0x7fb6380c5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.426+0000 7fb63fc00640 1 --2- 172.21.15.86:0/366972227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6380b79c0 0x7fb6380af140 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.426+0000 7fb63fc00640 1 -- 172.21.15.86:0/366972227 >> 172.21.15.86:0/366972227 conn(0x7fb63809ba30 msgr2=0x7fb638168200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:47.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.426+0000 7fb63fc00640 1 -- 172.21.15.86:0/366972227 shutdown_connections 2024-04-25T14:09:47.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.426+0000 7fb63fc00640 1 -- 172.21.15.86:0/366972227 wait complete. 2024-04-25T14:09:47.443 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T14:09:47.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:09:47.443 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T14:09:47.472 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T14:09:47.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:09:47.472 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T14:09:47.529 INFO:tasks.ceph.mgr.x.smithi155.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:09:47.529 INFO:tasks.ceph.mgr.x.smithi155.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:09:47.529 INFO:tasks.ceph.mgr.x.smithi155.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:09:47.533 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:09:47.533 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:09:47.533 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:09:47.538 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:47.539+0000 7f6dd142c180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:09:47.542 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:09:47.542 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:09:47.593 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:09:47.607 INFO:tasks.ceph.mgr.y.smithi086.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:09:47.607 INFO:tasks.ceph.mgr.y.smithi086.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:09:47.607 INFO:tasks.ceph.mgr.y.smithi086.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:09:47.617 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:47.618+0000 7fb1a9257180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:09:47.687 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:47.687+0000 7f6dd142c180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:09:47.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.746+0000 7fba5a800640 1 Processor -- start 2024-04-25T14:09:47.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.746+0000 7fba5a800640 1 -- start start 2024-04-25T14:09:47.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.746+0000 7fba5a800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fba540cfca0 0x7fba5406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:47.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.746+0000 7fba5a800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba540d16e0 0x7fba540d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:47.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.746+0000 7fba5a800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fba5406c670 0x7fba5406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:47.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.746+0000 7fba5a800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fba54070470 con 0x7fba540cfca0 2024-04-25T14:09:47.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.746+0000 7fba5a800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fba540d2030 con 0x7fba5406c670 2024-04-25T14:09:47.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.746+0000 7fba5a800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fba54072bd0 con 0x7fba540d16e0 2024-04-25T14:09:47.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.746+0000 7fba53400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fba540cfca0 0x7fba5406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:47.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.746+0000 7fba53400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fba540cfca0 0x7fba5406f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48768/0 (socket says 172.21.15.86:48768) 2024-04-25T14:09:47.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.746+0000 7fba53400640 1 -- 172.21.15.86:0/844628894 learned_addr learned my addr 172.21.15.86:0/844628894 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:47.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.746+0000 7fba4be00640 1 --2- 172.21.15.86:0/844628894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba540d16e0 0x7fba540d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:47.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.746+0000 7fba53e00640 1 --2- 172.21.15.86:0/844628894 >> v2:172.21.15.155:3300/0 conn(0x7fba5406c670 0x7fba5406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:47.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.750+0000 7fba53400640 1 -- 172.21.15.86:0/844628894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba540d16e0 msgr2=0x7fba540d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:47.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.750+0000 7fba53400640 1 --2- 172.21.15.86:0/844628894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba540d16e0 0x7fba540d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.750+0000 7fba53400640 1 -- 172.21.15.86:0/844628894 >> v2:172.21.15.155:3300/0 conn(0x7fba5406c670 msgr2=0x7fba5406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:47.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.750+0000 7fba53400640 1 --2- 172.21.15.86:0/844628894 >> v2:172.21.15.155:3300/0 conn(0x7fba5406c670 0x7fba5406ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.750+0000 7fba53400640 1 -- 172.21.15.86:0/844628894 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba5406f930 con 0x7fba540cfca0 2024-04-25T14:09:47.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.750+0000 7fba53e00640 1 --2- 172.21.15.86:0/844628894 >> v2:172.21.15.155:3300/0 conn(0x7fba5406c670 0x7fba5406ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:47.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.750+0000 7fba53400640 1 --2- 172.21.15.86:0/844628894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba540cfca0 0x7fba5406f3f0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fba5406e060 tx=0x7fba4401ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee25625ff829562a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:47.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.750+0000 7fba52a00640 1 -- 172.21.15.86:0/844628894 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba440b4070 con 0x7fba540cfca0 2024-04-25T14:09:47.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.750+0000 7fba52a00640 1 -- 172.21.15.86:0/844628894 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba440a7a90 con 0x7fba540cfca0 2024-04-25T14:09:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.750+0000 7fba52a00640 1 -- 172.21.15.86:0/844628894 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba440a7e60 con 0x7fba540cfca0 2024-04-25T14:09:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.754+0000 7fba5a800640 1 -- 172.21.15.86:0/844628894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba540cfca0 msgr2=0x7fba5406f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:47.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.754+0000 7fba5a800640 1 --2- 172.21.15.86:0/844628894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba540cfca0 0x7fba5406f3f0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fba5406e060 tx=0x7fba4401ebf0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.754+0000 7fba5a800640 1 -- 172.21.15.86:0/844628894 shutdown_connections 2024-04-25T14:09:47.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.754+0000 7fba5a800640 1 --2- 172.21.15.86:0/844628894 >> v2:172.21.15.155:3300/0 conn(0x7fba5406c670 0x7fba5406ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.754+0000 7fba5a800640 1 --2- 172.21.15.86:0/844628894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba540d16e0 0x7fba540d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.754+0000 7fba5a800640 1 --2- 172.21.15.86:0/844628894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba540cfca0 0x7fba5406f3f0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.754+0000 7fba5a800640 1 -- 172.21.15.86:0/844628894 >> 172.21.15.86:0/844628894 conn(0x7fba540b4650 msgr2=0x7fba540b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:47.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.754+0000 7fba5a800640 1 -- 172.21.15.86:0/844628894 shutdown_connections 2024-04-25T14:09:47.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.754+0000 7fba5a800640 1 -- 172.21.15.86:0/844628894 wait complete. 2024-04-25T14:09:47.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.754+0000 7fba5a800640 1 Processor -- start 2024-04-25T14:09:47.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.754+0000 7fba5a800640 1 -- start start 2024-04-25T14:09:47.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.758+0000 7fba5a800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba5406c670 0x7fba5415d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:47.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.758+0000 7fba5a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba540cfca0 0x7fba541587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:47.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.758+0000 7fba5a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba540d16e0 0x7fba5415b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:47.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.758+0000 7fba5a800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba540ce570 con 0x7fba540d16e0 2024-04-25T14:09:47.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.758+0000 7fba5a800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fba5406d3d0 con 0x7fba540cfca0 2024-04-25T14:09:47.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.758+0000 7fba5a800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fba540ce6f0 con 0x7fba5406c670 2024-04-25T14:09:47.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.758+0000 7fba53e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba540d16e0 0x7fba5415b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:47.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.758+0000 7fba53e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba540d16e0 0x7fba5415b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48770/0 (socket says 172.21.15.86:48770) 2024-04-25T14:09:47.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.758+0000 7fba53e00640 1 -- 172.21.15.86:0/160833480 learned_addr learned my addr 172.21.15.86:0/160833480 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:47.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.758+0000 7fba4be00640 1 --2- 172.21.15.86:0/160833480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba540cfca0 0x7fba541587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:47.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.758+0000 7fba53400640 1 --2- 172.21.15.86:0/160833480 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba5406c670 0x7fba5415d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:47.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.758+0000 7fba53e00640 1 -- 172.21.15.86:0/160833480 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba5406c670 msgr2=0x7fba5415d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:47.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.758+0000 7fba53e00640 1 --2- 172.21.15.86:0/160833480 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba5406c670 0x7fba5415d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.758+0000 7fba53e00640 1 -- 172.21.15.86:0/160833480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba540cfca0 msgr2=0x7fba541587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:47.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.758+0000 7fba53e00640 1 --2- 172.21.15.86:0/160833480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba540cfca0 0x7fba541587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.758+0000 7fba53e00640 1 -- 172.21.15.86:0/160833480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba541fd200 con 0x7fba540d16e0 2024-04-25T14:09:47.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.758+0000 7fba53e00640 1 --2- 172.21.15.86:0/160833480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba540d16e0 0x7fba5415b120 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fba4c007e20 tx=0x7fba4c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:47.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.758+0000 7fba4b400640 1 -- 172.21.15.86:0/160833480 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c01a070 con 0x7fba540d16e0 2024-04-25T14:09:47.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.758+0000 7fba4b400640 1 -- 172.21.15.86:0/160833480 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba4c003ba0 con 0x7fba540d16e0 2024-04-25T14:09:47.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.758+0000 7fba4b400640 1 -- 172.21.15.86:0/160833480 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c00de80 con 0x7fba540d16e0 2024-04-25T14:09:47.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.758+0000 7fba5a800640 1 -- 172.21.15.86:0/160833480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba541fd4e0 con 0x7fba540d16e0 2024-04-25T14:09:47.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.758+0000 7fba5a800640 1 -- 172.21.15.86:0/160833480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba541fdaa0 con 0x7fba540d16e0 2024-04-25T14:09:47.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.758+0000 7fba5a800640 1 -- 172.21.15.86:0/160833480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba5409f4b0 con 0x7fba540d16e0 2024-04-25T14:09:47.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.758+0000 7fba4b400640 1 -- 172.21.15.86:0/160833480 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 41) ==== 93253+0+0 (secure 0 0 0) 0x7fba4c01d070 con 0x7fba540d16e0 2024-04-25T14:09:47.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.762+0000 7fba4b400640 1 --2- 172.21.15.86:0/160833480 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7fba3006f490 0x7fba300719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:47.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.762+0000 7fba4b400640 1 -- 172.21.15.86:0/160833480 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6445+0+0 (secure 0 0 0) 0x7fba4c08fa30 con 0x7fba540d16e0 2024-04-25T14:09:47.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.762+0000 7fba53400640 1 -- 172.21.15.86:0/160833480 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7fba3006f490 msgr2=0x7fba300719a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/436174229 2024-04-25T14:09:47.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.762+0000 7fba53400640 1 --2- 172.21.15.86:0/160833480 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7fba3006f490 0x7fba300719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:09:47.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.770+0000 7fba4b400640 1 -- 172.21.15.86:0/160833480 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fba4c010070 con 0x7fba540d16e0 2024-04-25T14:09:47.775 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:47.778+0000 7fb1a9257180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:09:47.814 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:47.815+0000 7f6dd142c180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:09:47.885 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:47.887+0000 7f6dd142c180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:09:47.903 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:47.906+0000 7fb1a9257180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:09:47.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.934+0000 7fba5a800640 1 -- 172.21.15.86:0/160833480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fba54069d00 con 0x7fba540d16e0 2024-04-25T14:09:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.934+0000 7fba4b400640 1 -- 172.21.15.86:0/160833480 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+17374 (secure 0 0 0) 0x7fba4c05bd70 con 0x7fba540d16e0 2024-04-25T14:09:47.931 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:47.932 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":101,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:09:43.313900+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:09:23.588241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T14:09:24.156880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2887621339":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/361929172":"2024-04-26T14:09:43.313864+0000","172.21.15.86:6833/4236874185":"2024-04-26T14:09:43.313864+0000","172.21.15.86:6832/4236874185":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/789182356":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/1532315242":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/1775030266":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/2226990349":"2024-04-26T14:09:43.313864+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:09:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.938+0000 7fba5a800640 1 -- 172.21.15.86:0/160833480 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7fba3006f490 msgr2=0x7fba300719a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:09:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.938+0000 7fba5a800640 1 --2- 172.21.15.86:0/160833480 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7fba3006f490 0x7fba300719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.938+0000 7fba5a800640 1 -- 172.21.15.86:0/160833480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba540d16e0 msgr2=0x7fba5415b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.938+0000 7fba5a800640 1 --2- 172.21.15.86:0/160833480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba540d16e0 0x7fba5415b120 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fba4c007e20 tx=0x7fba4c00d4a0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.938+0000 7fba5a800640 1 -- 172.21.15.86:0/160833480 shutdown_connections 2024-04-25T14:09:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.938+0000 7fba5a800640 1 --2- 172.21.15.86:0/160833480 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7fba3006f490 0x7fba300719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.938+0000 7fba5a800640 1 --2- 172.21.15.86:0/160833480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba540d16e0 0x7fba5415b120 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.938+0000 7fba5a800640 1 --2- 172.21.15.86:0/160833480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba540cfca0 0x7fba541587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.938+0000 7fba5a800640 1 --2- 172.21.15.86:0/160833480 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba5406c670 0x7fba5415d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.938+0000 7fba5a800640 1 -- 172.21.15.86:0/160833480 >> 172.21.15.86:0/160833480 conn(0x7fba540b4650 msgr2=0x7fba540b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.938+0000 7fba5a800640 1 -- 172.21.15.86:0/160833480 shutdown_connections 2024-04-25T14:09:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:47.938+0000 7fba5a800640 1 -- 172.21.15.86:0/160833480 wait complete. 2024-04-25T14:09:47.954 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:09:23.588241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:09:24.156880+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:09:47.955 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:09:47.978 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:47.982+0000 7fb1a9257180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:09:48.033 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:48.035+0000 7f6dd142c180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:09:48.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.130+0000 7f1e9c000640 1 Processor -- start 2024-04-25T14:09:48.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.130+0000 7f1e9c000640 1 -- start start 2024-04-25T14:09:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.134+0000 7f1e9c000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e940d1cc0 0x7f1e940d20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.134+0000 7f1e9c000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e94161770 con 0x7f1e940ce5a0 2024-04-25T14:09:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.134+0000 7f1e9c000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e94161a50 con 0x7f1e94071530 2024-04-25T14:09:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.134+0000 7f1e9c000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1e94162380 con 0x7f1e940d1cc0 2024-04-25T14:09:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.134+0000 7f1e99600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1e940ce5a0 0x7f1e94070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41692/0 (socket says 172.21.15.86:41692) 2024-04-25T14:09:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.134+0000 7f1e99600640 1 -- 172.21.15.86:0/3451915759 learned_addr learned my addr 172.21.15.86:0/3451915759 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.134+0000 7f1e9a000640 1 --2- 172.21.15.86:0/3451915759 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e940d1cc0 0x7f1e940d20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.134+0000 7f1e93e00640 1 -- 172.21.15.86:0/3451915759 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 493249990 0 0) 0x7f1e94161770 con 0x7f1e940ce5a0 2024-04-25T14:09:48.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.134+0000 7f1e93e00640 1 -- 172.21.15.86:0/3451915759 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e7c003530 con 0x7f1e940ce5a0 2024-04-25T14:09:48.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.134+0000 7f1e93e00640 1 -- 172.21.15.86:0/3451915759 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4248428217 0 0) 0x7f1e94161a50 con 0x7f1e94071530 2024-04-25T14:09:48.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.134+0000 7f1e9a000640 1 -- 172.21.15.86:0/3451915759 >> v1:172.21.15.155:6789/0 conn(0x7f1e94071530 legacy=0x7f1e940d15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:48.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.134+0000 7f1e9a000640 1 -- 172.21.15.86:0/3451915759 >> v1:172.21.15.86:6789/0 conn(0x7f1e940ce5a0 legacy=0x7f1e94070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:48.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.134+0000 7f1e9a000640 1 -- 172.21.15.86:0/3451915759 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e94161d30 con 0x7f1e940d1cc0 2024-04-25T14:09:48.132 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:48.134+0000 7fb1a9257180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:09:48.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.138+0000 7f1e9a000640 1 --2- 172.21.15.86:0/3451915759 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e940d1cc0 0x7f1e940d20d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c009dc0 tx=0x7f1e8c01c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e176bf30f02dd59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:48.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.138+0000 7f1e93e00640 1 -- 172.21.15.86:0/3451915759 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c0b2070 con 0x7f1e940d1cc0 2024-04-25T14:09:48.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.138+0000 7f1e93e00640 1 -- 172.21.15.86:0/3451915759 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e8c0a59e0 con 0x7f1e940d1cc0 2024-04-25T14:09:48.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.138+0000 7f1e93e00640 1 -- 172.21.15.86:0/3451915759 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c0a5db0 con 0x7f1e940d1cc0 2024-04-25T14:09:48.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.138+0000 7f1e9c000640 1 -- 172.21.15.86:0/3451915759 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e940d1cc0 msgr2=0x7f1e940d20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:48.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.138+0000 7f1e9c000640 1 --2- 172.21.15.86:0/3451915759 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e940d1cc0 0x7f1e940d20d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c009dc0 tx=0x7f1e8c01c960 comp rx=0 tx=0).stop 2024-04-25T14:09:48.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.138+0000 7f1e9c000640 1 -- 172.21.15.86:0/3451915759 shutdown_connections 2024-04-25T14:09:48.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.138+0000 7f1e9c000640 1 --2- 172.21.15.86:0/3451915759 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e940d1cc0 0x7f1e940d20d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.138+0000 7f1e9c000640 1 -- 172.21.15.86:0/3451915759 >> 172.21.15.86:0/3451915759 conn(0x7f1e940cffc0 msgr2=0x7f1e940d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:48.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.138+0000 7f1e9c000640 1 -- 172.21.15.86:0/3451915759 shutdown_connections 2024-04-25T14:09:48.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.138+0000 7f1e9c000640 1 -- 172.21.15.86:0/3451915759 wait complete. 2024-04-25T14:09:48.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.142+0000 7f1e9c000640 1 Processor -- start 2024-04-25T14:09:48.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.142+0000 7f1e9c000640 1 -- start start 2024-04-25T14:09:48.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.142+0000 7f1e9c000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e940ce5a0 0x7f1e941f7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.142+0000 7f1e9c000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e940d1cc0 0x7f1e941532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.142+0000 7f1e9c000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e941570d0 0x7f1e94156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.142+0000 7f1e9c000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e94158550 con 0x7f1e940ce5a0 2024-04-25T14:09:48.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.142+0000 7f1e9c000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1e941583d0 con 0x7f1e941570d0 2024-04-25T14:09:48.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.142+0000 7f1e9c000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1e9416c6a0 con 0x7f1e940d1cc0 2024-04-25T14:09:48.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.142+0000 7f1e99600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e940ce5a0 0x7f1e941f7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.142+0000 7f1e99600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e940ce5a0 0x7f1e941f7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48780/0 (socket says 172.21.15.86:48780) 2024-04-25T14:09:48.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.142+0000 7f1e99600640 1 -- 172.21.15.86:0/385736411 learned_addr learned my addr 172.21.15.86:0/385736411 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:48.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.142+0000 7f1e98c00640 1 --2- 172.21.15.86:0/385736411 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e940d1cc0 0x7f1e941532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.142+0000 7f1e9a000640 1 --2- 172.21.15.86:0/385736411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e941570d0 0x7f1e94156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.142+0000 7f1e99600640 1 -- 172.21.15.86:0/385736411 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e940d1cc0 msgr2=0x7f1e941532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:48.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.142+0000 7f1e99600640 1 --2- 172.21.15.86:0/385736411 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e940d1cc0 0x7f1e941532a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.142+0000 7f1e99600640 1 -- 172.21.15.86:0/385736411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e941570d0 msgr2=0x7f1e94156410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:48.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.142+0000 7f1e99600640 1 --2- 172.21.15.86:0/385736411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e941570d0 0x7f1e94156410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.142+0000 7f1e99600640 1 -- 172.21.15.86:0/385736411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e94154890 con 0x7f1e940ce5a0 2024-04-25T14:09:48.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.142+0000 7f1e99600640 1 --2- 172.21.15.86:0/385736411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e940ce5a0 0x7f1e941f7690 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f1e8800c3c0 tx=0x7f1e8800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:48.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.142+0000 7f1e91600640 1 -- 172.21.15.86:0/385736411 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88017070 con 0x7f1e940ce5a0 2024-04-25T14:09:48.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.142+0000 7f1e91600640 1 -- 172.21.15.86:0/385736411 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e88005d10 con 0x7f1e940ce5a0 2024-04-25T14:09:48.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.142+0000 7f1e9c000640 1 -- 172.21.15.86:0/385736411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e94155c30 con 0x7f1e940ce5a0 2024-04-25T14:09:48.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.142+0000 7f1e91600640 1 -- 172.21.15.86:0/385736411 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88007310 con 0x7f1e940ce5a0 2024-04-25T14:09:48.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.142+0000 7f1e9c000640 1 -- 172.21.15.86:0/385736411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e94154e10 con 0x7f1e940ce5a0 2024-04-25T14:09:48.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.146+0000 7f1e9c000640 1 -- 172.21.15.86:0/385736411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e94155050 con 0x7f1e940ce5a0 2024-04-25T14:09:48.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.146+0000 7f1e91600640 1 -- 172.21.15.86:0/385736411 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 41) ==== 93253+0+0 (secure 0 0 0) 0x7f1e8801a070 con 0x7f1e940ce5a0 2024-04-25T14:09:48.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.146+0000 7f1e91600640 1 --2- 172.21.15.86:0/385736411 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f1e7406f490 0x7f1e740719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.146+0000 7f1e98c00640 1 -- 172.21.15.86:0/385736411 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f1e7406f490 msgr2=0x7f1e740719a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/436174229 2024-04-25T14:09:48.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.146+0000 7f1e91600640 1 -- 172.21.15.86:0/385736411 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6445+0+0 (secure 0 0 0) 0x7f1e8808e8e0 con 0x7f1e940ce5a0 2024-04-25T14:09:48.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.146+0000 7f1e98c00640 1 --2- 172.21.15.86:0/385736411 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f1e7406f490 0x7f1e740719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:09:48.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.150+0000 7f1e91600640 1 -- 172.21.15.86:0/385736411 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1e8805af70 con 0x7f1e940ce5a0 2024-04-25T14:09:48.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.310+0000 7f1e9c000640 1 -- 172.21.15.86:0/385736411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1e940720e0 con 0x7f1e940ce5a0 2024-04-25T14:09:48.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.310+0000 7f1e91600640 1 -- 172.21.15.86:0/385736411 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) ==== 93+0+10 (secure 0 0 0) 0x7f1e8805f270 con 0x7f1e940ce5a0 2024-04-25T14:09:48.309 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:09:48.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.314+0000 7f1e9c000640 1 -- 172.21.15.86:0/385736411 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f1e7406f490 msgr2=0x7f1e740719a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:09:48.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.314+0000 7f1e9c000640 1 --2- 172.21.15.86:0/385736411 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f1e7406f490 0x7f1e740719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.314+0000 7f1e9c000640 1 -- 172.21.15.86:0/385736411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e940ce5a0 msgr2=0x7f1e941f7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:48.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.314+0000 7f1e9c000640 1 --2- 172.21.15.86:0/385736411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e940ce5a0 0x7f1e941f7690 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f1e8800c3c0 tx=0x7f1e8800c880 comp rx=0 tx=0).stop 2024-04-25T14:09:48.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.314+0000 7f1e9c000640 1 -- 172.21.15.86:0/385736411 shutdown_connections 2024-04-25T14:09:48.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.314+0000 7f1e9c000640 1 --2- 172.21.15.86:0/385736411 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f1e7406f490 0x7f1e740719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.314+0000 7f1e9c000640 1 --2- 172.21.15.86:0/385736411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e941570d0 0x7f1e94156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.314+0000 7f1e9c000640 1 --2- 172.21.15.86:0/385736411 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e940d1cc0 0x7f1e941532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.314+0000 7f1e9c000640 1 --2- 172.21.15.86:0/385736411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e940ce5a0 0x7f1e941f7690 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.314+0000 7f1e9c000640 1 -- 172.21.15.86:0/385736411 >> 172.21.15.86:0/385736411 conn(0x7f1e940cffc0 msgr2=0x7f1e940b8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:48.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.314+0000 7f1e9c000640 1 -- 172.21.15.86:0/385736411 shutdown_connections 2024-04-25T14:09:48.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.314+0000 7f1e9c000640 1 -- 172.21.15.86:0/385736411 wait complete. 2024-04-25T14:09:48.331 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:09:48.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.490+0000 7f6a34000640 1 Processor -- start 2024-04-25T14:09:48.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.490+0000 7f6a34000640 1 -- start start 2024-04-25T14:09:48.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.490+0000 7f6a34000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6a2c0d2c80 0x7f6a2c15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.490+0000 7f6a34000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6a2c070e20 0x7f6a2c071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.490+0000 7f6a34000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a2c071770 con 0x7f6a2c0722f0 2024-04-25T14:09:48.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.490+0000 7f6a34000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6a2c0ce720 con 0x7f6a2c070e20 2024-04-25T14:09:48.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.490+0000 7f6a34000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f6a2c0cd8d0 con 0x7f6a2c0d2c80 2024-04-25T14:09:48.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.490+0000 7f6a30c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6a2c070e20 0x7f6a2c071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.490+0000 7f6a30c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6a2c070e20 0x7f6a2c071230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48792/0 (socket says 172.21.15.86:48792) 2024-04-25T14:09:48.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.490+0000 7f6a30c00640 1 -- 172.21.15.86:0/2862453964 learned_addr learned my addr 172.21.15.86:0/2862453964 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:48.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.490+0000 7f6a31600640 1 --2- 172.21.15.86:0/2862453964 >> v2:172.21.15.155:3300/0 conn(0x7f6a2c0d2c80 0x7f6a2c15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.490+0000 7f6a30c00640 1 -- 172.21.15.86:0/2862453964 >> v2:172.21.15.155:3300/0 conn(0x7f6a2c0d2c80 msgr2=0x7f6a2c15dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:48.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.490+0000 7f6a30c00640 1 --2- 172.21.15.86:0/2862453964 >> v2:172.21.15.155:3300/0 conn(0x7f6a2c0d2c80 0x7f6a2c15dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.490+0000 7f6a30c00640 1 -- 172.21.15.86:0/2862453964 >> v1:172.21.15.86:6789/0 conn(0x7f6a2c0722f0 legacy=0x7f6a2c072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:48.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.490+0000 7f6a30c00640 1 -- 172.21.15.86:0/2862453964 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2c0ce970 con 0x7f6a2c070e20 2024-04-25T14:09:48.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.494+0000 7f6a30c00640 1 --2- 172.21.15.86:0/2862453964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a2c070e20 0x7f6a2c071230 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c0d11a0 tx=0x7f6a1801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af42648122a1d140 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:48.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.494+0000 7f6a2be00640 1 -- 172.21.15.86:0/2862453964 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a180b5070 con 0x7f6a2c070e20 2024-04-25T14:09:48.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.494+0000 7f6a2be00640 1 -- 172.21.15.86:0/2862453964 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a180a8a10 con 0x7f6a2c070e20 2024-04-25T14:09:48.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.494+0000 7f6a2be00640 1 -- 172.21.15.86:0/2862453964 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a180a8de0 con 0x7f6a2c070e20 2024-04-25T14:09:48.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.494+0000 7f6a34000640 1 -- 172.21.15.86:0/2862453964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a2c070e20 msgr2=0x7f6a2c071230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:48.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.494+0000 7f6a34000640 1 --2- 172.21.15.86:0/2862453964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a2c070e20 0x7f6a2c071230 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c0d11a0 tx=0x7f6a1801ed10 comp rx=0 tx=0).stop 2024-04-25T14:09:48.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.494+0000 7f6a34000640 1 -- 172.21.15.86:0/2862453964 shutdown_connections 2024-04-25T14:09:48.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.494+0000 7f6a34000640 1 --2- 172.21.15.86:0/2862453964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a2c070e20 0x7f6a2c071230 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.494+0000 7f6a34000640 1 --2- 172.21.15.86:0/2862453964 >> v2:172.21.15.155:3300/0 conn(0x7f6a2c0d2c80 0x7f6a2c15dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.494+0000 7f6a34000640 1 -- 172.21.15.86:0/2862453964 >> 172.21.15.86:0/2862453964 conn(0x7f6a2c0cffc0 msgr2=0x7f6a2c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:48.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.494+0000 7f6a34000640 1 -- 172.21.15.86:0/2862453964 shutdown_connections 2024-04-25T14:09:48.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.494+0000 7f6a34000640 1 -- 172.21.15.86:0/2862453964 wait complete. 2024-04-25T14:09:48.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.498+0000 7f6a34000640 1 Processor -- start 2024-04-25T14:09:48.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.498+0000 7f6a34000640 1 -- start start 2024-04-25T14:09:48.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.498+0000 7f6a34000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a2c070e20 0x7f6a2c15ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.498+0000 7f6a34000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a2c0722f0 0x7f6a2c06f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.498+0000 7f6a34000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a2c0d2c80 0x7f6a2c1522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.498+0000 7f6a34000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c0d21a0 con 0x7f6a2c070e20 2024-04-25T14:09:48.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.498+0000 7f6a34000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c0d1ec0 con 0x7f6a2c0722f0 2024-04-25T14:09:48.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.498+0000 7f6a34000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6a2c0d2480 con 0x7f6a2c0d2c80 2024-04-25T14:09:48.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.498+0000 7f6a31600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a2c070e20 0x7f6a2c15ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.498+0000 7f6a31600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a2c070e20 0x7f6a2c15ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48808/0 (socket says 172.21.15.86:48808) 2024-04-25T14:09:48.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.498+0000 7f6a31600640 1 -- 172.21.15.86:0/2487205967 learned_addr learned my addr 172.21.15.86:0/2487205967 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:48.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.498+0000 7f6a32000640 1 --2- 172.21.15.86:0/2487205967 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a2c0d2c80 0x7f6a2c1522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.498+0000 7f6a30c00640 1 --2- 172.21.15.86:0/2487205967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a2c0722f0 0x7f6a2c06f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.498+0000 7f6a31600640 1 -- 172.21.15.86:0/2487205967 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a2c0d2c80 msgr2=0x7f6a2c1522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:48.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.498+0000 7f6a31600640 1 --2- 172.21.15.86:0/2487205967 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a2c0d2c80 0x7f6a2c1522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.498+0000 7f6a31600640 1 -- 172.21.15.86:0/2487205967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a2c0722f0 msgr2=0x7f6a2c06f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:48.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.498+0000 7f6a31600640 1 --2- 172.21.15.86:0/2487205967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a2c0722f0 0x7f6a2c06f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.498+0000 7f6a31600640 1 -- 172.21.15.86:0/2487205967 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2c06e230 con 0x7f6a2c070e20 2024-04-25T14:09:48.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.498+0000 7f6a31600640 1 --2- 172.21.15.86:0/2487205967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a2c070e20 0x7f6a2c15ece0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000e870 tx=0x7f6a2000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:48.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.502+0000 7f6a29600640 1 -- 172.21.15.86:0/2487205967 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a20017070 con 0x7f6a2c070e20 2024-04-25T14:09:48.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.502+0000 7f6a29600640 1 -- 172.21.15.86:0/2487205967 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a2000c940 con 0x7f6a2c070e20 2024-04-25T14:09:48.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.502+0000 7f6a34000640 1 -- 172.21.15.86:0/2487205967 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a2c156af0 con 0x7f6a2c070e20 2024-04-25T14:09:48.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.502+0000 7f6a29600640 1 -- 172.21.15.86:0/2487205967 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2000ccf0 con 0x7f6a2c070e20 2024-04-25T14:09:48.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.502+0000 7f6a34000640 1 -- 172.21.15.86:0/2487205967 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a2c06e7b0 con 0x7f6a2c070e20 2024-04-25T14:09:48.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.502+0000 7f6a34000640 1 -- 172.21.15.86:0/2487205967 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a2c0720e0 con 0x7f6a2c070e20 2024-04-25T14:09:48.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.502+0000 7f6a29600640 1 -- 172.21.15.86:0/2487205967 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 41) ==== 93253+0+0 (secure 0 0 0) 0x7f6a2001a070 con 0x7f6a2c070e20 2024-04-25T14:09:48.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.502+0000 7f6a29600640 1 --2- 172.21.15.86:0/2487205967 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f6a0c06f490 0x7f6a0c0719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.502+0000 7f6a29600640 1 -- 172.21.15.86:0/2487205967 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6445+0+0 (secure 0 0 0) 0x7f6a2008f3b0 con 0x7f6a2c070e20 2024-04-25T14:09:48.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.502+0000 7f6a30c00640 1 -- 172.21.15.86:0/2487205967 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f6a0c06f490 msgr2=0x7f6a0c0719a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/436174229 2024-04-25T14:09:48.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.502+0000 7f6a30c00640 1 --2- 172.21.15.86:0/2487205967 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f6a0c06f490 0x7f6a0c0719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:09:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.506+0000 7f6a29600640 1 -- 172.21.15.86:0/2487205967 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6a2005b990 con 0x7f6a2c070e20 2024-04-25T14:09:48.585 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:48.587+0000 7f6dd142c180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:09:48.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.666+0000 7f6a34000640 1 -- 172.21.15.86:0/2487205967 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6a2c076bf0 con 0x7f6a2c070e20 2024-04-25T14:09:48.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.666+0000 7f6a29600640 1 -- 172.21.15.86:0/2487205967 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) ==== 104+0+11 (secure 0 0 0) 0x7f6a20060e20 con 0x7f6a2c070e20 2024-04-25T14:09:48.664 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:09:48.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.670+0000 7f6a34000640 1 -- 172.21.15.86:0/2487205967 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f6a0c06f490 msgr2=0x7f6a0c0719a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:09:48.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.670+0000 7f6a34000640 1 --2- 172.21.15.86:0/2487205967 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f6a0c06f490 0x7f6a0c0719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.670+0000 7f6a34000640 1 -- 172.21.15.86:0/2487205967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a2c070e20 msgr2=0x7f6a2c15ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:48.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.670+0000 7f6a34000640 1 --2- 172.21.15.86:0/2487205967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a2c070e20 0x7f6a2c15ece0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000e870 tx=0x7f6a2000ec70 comp rx=0 tx=0).stop 2024-04-25T14:09:48.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.670+0000 7f6a34000640 1 -- 172.21.15.86:0/2487205967 shutdown_connections 2024-04-25T14:09:48.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.670+0000 7f6a34000640 1 --2- 172.21.15.86:0/2487205967 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7f6a0c06f490 0x7f6a0c0719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.670+0000 7f6a34000640 1 --2- 172.21.15.86:0/2487205967 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a2c0d2c80 0x7f6a2c1522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.670+0000 7f6a34000640 1 --2- 172.21.15.86:0/2487205967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a2c0722f0 0x7f6a2c06f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.670+0000 7f6a34000640 1 --2- 172.21.15.86:0/2487205967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a2c070e20 0x7f6a2c15ece0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.670+0000 7f6a34000640 1 -- 172.21.15.86:0/2487205967 >> 172.21.15.86:0/2487205967 conn(0x7f6a2c0cffc0 msgr2=0x7f6a2c072700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:48.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.670+0000 7f6a34000640 1 -- 172.21.15.86:0/2487205967 shutdown_connections 2024-04-25T14:09:48.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.670+0000 7f6a34000640 1 -- 172.21.15.86:0/2487205967 wait complete. 2024-04-25T14:09:48.669 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:48.671+0000 7f6dd142c180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:09:48.687 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:09:48.700 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:48.702+0000 7fb1a9257180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:09:48.789 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:48.794+0000 7fb1a9257180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:09:48.834 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:48.835+0000 7f6dd142c180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:09:48.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.846+0000 7fce0b000640 1 Processor -- start 2024-04-25T14:09:48.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.846+0000 7fce0b000640 1 -- start start 2024-04-25T14:09:48.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.846+0000 7fce0b000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce040d2c80 0x7fce0415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.846+0000 7fce0b000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fce04070e20 0x7fce04071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.846+0000 7fce0b000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fce040722f0 0x7fce04072700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.846+0000 7fce0b000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fce040ce720 con 0x7fce04070e20 2024-04-25T14:09:48.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.846+0000 7fce0b000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fce04071770 con 0x7fce040722f0 2024-04-25T14:09:48.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.846+0000 7fce0b000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fce040cd8d0 con 0x7fce040d2c80 2024-04-25T14:09:48.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.846+0000 7fce03400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fce04070e20 0x7fce04071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.846+0000 7fce03400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fce04070e20 0x7fce04071230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43392/0 (socket says 172.21.15.86:43392) 2024-04-25T14:09:48.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.846+0000 7fce03400640 1 -- 172.21.15.86:0/4038271793 learned_addr learned my addr 172.21.15.86:0/4038271793 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:48.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.846+0000 7fce03e00640 1 --2- 172.21.15.86:0/4038271793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce040d2c80 0x7fce0415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.846+0000 7fce09000640 1 --2- 172.21.15.86:0/4038271793 >> v2:172.21.15.155:3300/0 conn(0x7fce040722f0 0x7fce04072700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.846+0000 7fce03400640 1 -- 172.21.15.86:0/4038271793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce040d2c80 msgr2=0x7fce0415dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:48.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.846+0000 7fce03400640 1 --2- 172.21.15.86:0/4038271793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce040d2c80 0x7fce0415dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.846+0000 7fce03400640 1 -- 172.21.15.86:0/4038271793 >> v2:172.21.15.155:3300/0 conn(0x7fce040722f0 msgr2=0x7fce04072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:48.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.846+0000 7fce03400640 1 --2- 172.21.15.86:0/4038271793 >> v2:172.21.15.155:3300/0 conn(0x7fce040722f0 0x7fce04072700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.846+0000 7fce03400640 1 -- 172.21.15.86:0/4038271793 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce040ce970 con 0x7fce04070e20 2024-04-25T14:09:48.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.846+0000 7fce03e00640 1 --2- 172.21.15.86:0/4038271793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce040d2c80 0x7fce0415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:48.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.850+0000 7fce03400640 1 --2- 172.21.15.86:0/4038271793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce04070e20 0x7fce04071230 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fce040736f0 tx=0x7fcdf001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4a528a33bb3ae2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:48.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.850+0000 7fce02a00640 1 -- 172.21.15.86:0/4038271793 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf00b5070 con 0x7fce04070e20 2024-04-25T14:09:48.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.850+0000 7fce02a00640 1 -- 172.21.15.86:0/4038271793 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdf00a89d0 con 0x7fce04070e20 2024-04-25T14:09:48.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.850+0000 7fce02a00640 1 -- 172.21.15.86:0/4038271793 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf00a8da0 con 0x7fce04070e20 2024-04-25T14:09:48.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.850+0000 7fce0b000640 1 -- 172.21.15.86:0/4038271793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce04070e20 msgr2=0x7fce04071230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:48.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.850+0000 7fce0b000640 1 --2- 172.21.15.86:0/4038271793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce04070e20 0x7fce04071230 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fce040736f0 tx=0x7fcdf001ed10 comp rx=0 tx=0).stop 2024-04-25T14:09:48.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.850+0000 7fce0b000640 1 -- 172.21.15.86:0/4038271793 shutdown_connections 2024-04-25T14:09:48.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.850+0000 7fce0b000640 1 --2- 172.21.15.86:0/4038271793 >> v2:172.21.15.155:3300/0 conn(0x7fce040722f0 0x7fce04072700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.850+0000 7fce0b000640 1 --2- 172.21.15.86:0/4038271793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce04070e20 0x7fce04071230 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.850+0000 7fce0b000640 1 --2- 172.21.15.86:0/4038271793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce040d2c80 0x7fce0415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.850+0000 7fce0b000640 1 -- 172.21.15.86:0/4038271793 >> 172.21.15.86:0/4038271793 conn(0x7fce040cffc0 msgr2=0x7fce040d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:48.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.850+0000 7fce0b000640 1 -- 172.21.15.86:0/4038271793 shutdown_connections 2024-04-25T14:09:48.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.850+0000 7fce0b000640 1 -- 172.21.15.86:0/4038271793 wait complete. 2024-04-25T14:09:48.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.854+0000 7fce0b000640 1 Processor -- start 2024-04-25T14:09:48.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.854+0000 7fce0b000640 1 -- start start 2024-04-25T14:09:48.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.854+0000 7fce0b000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce04070e20 0x7fce041526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.854+0000 7fce0b000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce040722f0 0x7fce04145640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.854+0000 7fce0b000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce040d2c80 0x7fce04145b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.854+0000 7fce0b000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce040d21a0 con 0x7fce040722f0 2024-04-25T14:09:48.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.854+0000 7fce0b000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fce040d1ec0 con 0x7fce04070e20 2024-04-25T14:09:48.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.854+0000 7fce0b000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fce040d2480 con 0x7fce040d2c80 2024-04-25T14:09:48.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.854+0000 7fce03400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce040722f0 0x7fce04145640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.854+0000 7fce03400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce040722f0 0x7fce04145640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43402/0 (socket says 172.21.15.86:43402) 2024-04-25T14:09:48.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.854+0000 7fce03400640 1 -- 172.21.15.86:0/4234069537 learned_addr learned my addr 172.21.15.86:0/4234069537 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:48.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.854+0000 7fce09000640 1 --2- 172.21.15.86:0/4234069537 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce040d2c80 0x7fce04145b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.854+0000 7fce03e00640 1 --2- 172.21.15.86:0/4234069537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce04070e20 0x7fce041526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.858+0000 7fce03400640 1 -- 172.21.15.86:0/4234069537 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce040d2c80 msgr2=0x7fce04145b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:48.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.858+0000 7fce03400640 1 --2- 172.21.15.86:0/4234069537 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce040d2c80 0x7fce04145b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.858+0000 7fce03400640 1 -- 172.21.15.86:0/4234069537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce04070e20 msgr2=0x7fce041526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:48.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.858+0000 7fce03400640 1 --2- 172.21.15.86:0/4234069537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce04070e20 0x7fce041526c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.858+0000 7fce03400640 1 -- 172.21.15.86:0/4234069537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce04152c00 con 0x7fce040722f0 2024-04-25T14:09:48.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.858+0000 7fce03400640 1 --2- 172.21.15.86:0/4234069537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce040722f0 0x7fce04145640 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fcdf001d3c0 tx=0x7fcdf001bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:48.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.858+0000 7fcdfbe00640 1 -- 172.21.15.86:0/4234069537 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf00b5040 con 0x7fce040722f0 2024-04-25T14:09:48.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.858+0000 7fcdfbe00640 1 -- 172.21.15.86:0/4234069537 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdf0004380 con 0x7fce040722f0 2024-04-25T14:09:48.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.858+0000 7fce0b000640 1 -- 172.21.15.86:0/4234069537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce04155260 con 0x7fce040722f0 2024-04-25T14:09:48.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.858+0000 7fcdfbe00640 1 -- 172.21.15.86:0/4234069537 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0003820 con 0x7fce040722f0 2024-04-25T14:09:48.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.858+0000 7fce0b000640 1 -- 172.21.15.86:0/4234069537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce04155820 con 0x7fce040722f0 2024-04-25T14:09:48.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.858+0000 7fce0b000640 1 -- 172.21.15.86:0/4234069537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce040720e0 con 0x7fce040722f0 2024-04-25T14:09:48.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.858+0000 7fcdfbe00640 1 -- 172.21.15.86:0/4234069537 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 41) ==== 93253+0+0 (secure 0 0 0) 0x7fcdf00039c0 con 0x7fce040722f0 2024-04-25T14:09:48.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.862+0000 7fcdfbe00640 1 --2- 172.21.15.86:0/4234069537 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7fcde406f4c0 0x7fcde40719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.862+0000 7fce03e00640 1 -- 172.21.15.86:0/4234069537 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7fcde406f4c0 msgr2=0x7fcde40719d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/436174229 2024-04-25T14:09:48.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.862+0000 7fcdfbe00640 1 -- 172.21.15.86:0/4234069537 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6445+0+0 (secure 0 0 0) 0x7fcdf012bf00 con 0x7fce040722f0 2024-04-25T14:09:48.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.862+0000 7fce03e00640 1 --2- 172.21.15.86:0/4234069537 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7fcde406f4c0 0x7fcde40719d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:09:48.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:48.866+0000 7fcdfbe00640 1 -- 172.21.15.86:0/4234069537 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcdf00bf060 con 0x7fce040722f0 2024-04-25T14:09:48.919 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:48.919+0000 7f6dd142c180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:09:48.958 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:48.962+0000 7fb1a9257180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:09:49.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.026+0000 7fce0b000640 1 -- 172.21.15.86:0/4234069537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fce04076bf0 con 0x7fce040722f0 2024-04-25T14:09:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.026+0000 7fcdfbe00640 1 -- 172.21.15.86:0/4234069537 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) ==== 100+0+11 (secure 0 0 0) 0x7fcdf0018870 con 0x7fce040722f0 2024-04-25T14:09:49.023 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:09:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.026+0000 7fce0b000640 1 -- 172.21.15.86:0/4234069537 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7fcde406f4c0 msgr2=0x7fcde40719d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:09:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.026+0000 7fce0b000640 1 --2- 172.21.15.86:0/4234069537 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7fcde406f4c0 0x7fcde40719d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.026+0000 7fce0b000640 1 -- 172.21.15.86:0/4234069537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce040722f0 msgr2=0x7fce04145640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.026+0000 7fce0b000640 1 --2- 172.21.15.86:0/4234069537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce040722f0 0x7fce04145640 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fcdf001d3c0 tx=0x7fcdf001bab0 comp rx=0 tx=0).stop 2024-04-25T14:09:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.026+0000 7fce0b000640 1 -- 172.21.15.86:0/4234069537 shutdown_connections 2024-04-25T14:09:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.026+0000 7fce0b000640 1 --2- 172.21.15.86:0/4234069537 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7fcde406f4c0 0x7fcde40719d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.026+0000 7fce0b000640 1 --2- 172.21.15.86:0/4234069537 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce040d2c80 0x7fce04145b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.026+0000 7fce0b000640 1 --2- 172.21.15.86:0/4234069537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce040722f0 0x7fce04145640 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.026+0000 7fce0b000640 1 --2- 172.21.15.86:0/4234069537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce04070e20 0x7fce041526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.026+0000 7fce0b000640 1 -- 172.21.15.86:0/4234069537 >> 172.21.15.86:0/4234069537 conn(0x7fce040cffc0 msgr2=0x7fce0415cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.026+0000 7fce0b000640 1 -- 172.21.15.86:0/4234069537 shutdown_connections 2024-04-25T14:09:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.030+0000 7fce0b000640 1 -- 172.21.15.86:0/4234069537 wait complete. 2024-04-25T14:09:49.045 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:49.046+0000 7fb1a9257180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:09:49.046 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T14:09:49.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.242+0000 7febe2600640 1 Processor -- start 2024-04-25T14:09:49.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.242+0000 7febe2600640 1 -- start start 2024-04-25T14:09:49.242 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:49.243+0000 7f6dd142c180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:09:49.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.246+0000 7febe2600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7febdc06df70 0x7febdc070d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:49.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.246+0000 7febe2600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febdc06e380 0x7febdc0712a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:49.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.246+0000 7febe2600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7febdc163630 0x7febdc163a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:49.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.246+0000 7febe2600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7febdc0722a0 con 0x7febdc163630 2024-04-25T14:09:49.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.246+0000 7febe2600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7febdc072e20 con 0x7febdc06df70 2024-04-25T14:09:49.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.246+0000 7febe2600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7febdc0724f0 con 0x7febdc06e380 2024-04-25T14:09:49.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.246+0000 7febe1c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7febdc163630 0x7febdc163a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:49.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.246+0000 7febe1c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7febdc163630 0x7febdc163a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43414/0 (socket says 172.21.15.86:43414) 2024-04-25T14:09:49.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.246+0000 7febe1c00640 1 -- 172.21.15.86:0/3779857039 learned_addr learned my addr 172.21.15.86:0/3779857039 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:49.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.246+0000 7febe1200640 1 --2- 172.21.15.86:0/3779857039 >> v2:172.21.15.155:3300/0 conn(0x7febdc06df70 0x7febdc070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:49.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.246+0000 7febd3e00640 1 --2- 172.21.15.86:0/3779857039 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febdc06e380 0x7febdc0712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:49.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.246+0000 7febe1c00640 1 -- 172.21.15.86:0/3779857039 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febdc06e380 msgr2=0x7febdc0712a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:49.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.246+0000 7febe1c00640 1 --2- 172.21.15.86:0/3779857039 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febdc06e380 0x7febdc0712a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:49.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.246+0000 7febe1c00640 1 -- 172.21.15.86:0/3779857039 >> v2:172.21.15.155:3300/0 conn(0x7febdc06df70 msgr2=0x7febdc070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:49.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.246+0000 7febe1c00640 1 --2- 172.21.15.86:0/3779857039 >> v2:172.21.15.155:3300/0 conn(0x7febdc06df70 0x7febdc070d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:49.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.246+0000 7febe1c00640 1 -- 172.21.15.86:0/3779857039 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febdc0727d0 con 0x7febdc163630 2024-04-25T14:09:49.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.246+0000 7febe1c00640 1 --2- 172.21.15.86:0/3779857039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febdc163630 0x7febdc163a40 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7febdc0742c0 tx=0x7febd401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca9e6e25a4669de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:49.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.250+0000 7febdbe00640 1 -- 172.21.15.86:0/3779857039 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd40a56b0 con 0x7febdc163630 2024-04-25T14:09:49.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.250+0000 7febdbe00640 1 -- 172.21.15.86:0/3779857039 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febd40a5850 con 0x7febdc163630 2024-04-25T14:09:49.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.250+0000 7febdbe00640 1 -- 172.21.15.86:0/3779857039 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd40a5c20 con 0x7febdc163630 2024-04-25T14:09:49.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.250+0000 7febe2600640 1 -- 172.21.15.86:0/3779857039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febdc163630 msgr2=0x7febdc163a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:49.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.250+0000 7febe2600640 1 --2- 172.21.15.86:0/3779857039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febdc163630 0x7febdc163a40 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7febdc0742c0 tx=0x7febd401c960 comp rx=0 tx=0).stop 2024-04-25T14:09:49.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.250+0000 7febe2600640 1 -- 172.21.15.86:0/3779857039 shutdown_connections 2024-04-25T14:09:49.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.250+0000 7febe2600640 1 --2- 172.21.15.86:0/3779857039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febdc163630 0x7febdc163a40 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:49.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.250+0000 7febe2600640 1 --2- 172.21.15.86:0/3779857039 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febdc06e380 0x7febdc0712a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:49.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.250+0000 7febe2600640 1 --2- 172.21.15.86:0/3779857039 >> v2:172.21.15.155:3300/0 conn(0x7febdc06df70 0x7febdc070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:49.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.250+0000 7febe2600640 1 -- 172.21.15.86:0/3779857039 >> 172.21.15.86:0/3779857039 conn(0x7febdc0d3330 msgr2=0x7febdc0d3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:49.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.250+0000 7febe2600640 1 -- 172.21.15.86:0/3779857039 shutdown_connections 2024-04-25T14:09:49.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.250+0000 7febe2600640 1 -- 172.21.15.86:0/3779857039 wait complete. 2024-04-25T14:09:49.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.250+0000 7febe2600640 1 Processor -- start 2024-04-25T14:09:49.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.250+0000 7febe2600640 1 -- start start 2024-04-25T14:09:49.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.254+0000 7febe2600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febdc06df70 0x7febdc0b6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:49.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.254+0000 7febe2600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febdc06e380 0x7febdc15bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:49.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.254+0000 7febe2600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febdc163630 0x7febdc0b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:49.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.254+0000 7febe2600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7febdc0d1b70 con 0x7febdc06e380 2024-04-25T14:09:49.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.254+0000 7febe2600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7febdc0d19f0 con 0x7febdc163630 2024-04-25T14:09:49.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.254+0000 7febe2600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7febdc0d1cf0 con 0x7febdc06df70 2024-04-25T14:09:49.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.254+0000 7febd3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febdc06e380 0x7febdc15bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:49.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.254+0000 7febe1c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febdc163630 0x7febdc0b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:49.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.254+0000 7febe1200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febdc06df70 0x7febdc0b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:49.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.254+0000 7febe1200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febdc06df70 0x7febdc0b6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:32910/0 (socket says 172.21.15.86:32910) 2024-04-25T14:09:49.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.254+0000 7febe1c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febdc163630 0x7febdc0b8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:35010/0 (socket says 172.21.15.86:35010) 2024-04-25T14:09:49.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.254+0000 7febe1200640 1 -- 172.21.15.86:0/1107075116 learned_addr learned my addr 172.21.15.86:0/1107075116 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:49.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.254+0000 7febd3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febdc06e380 0x7febdc15bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43430/0 (socket says 172.21.15.86:43430) 2024-04-25T14:09:49.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.254+0000 7febe1200640 1 -- 172.21.15.86:0/1107075116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febdc163630 msgr2=0x7febdc0b8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:49.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.254+0000 7febe1200640 1 --2- 172.21.15.86:0/1107075116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febdc163630 0x7febdc0b8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:49.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.254+0000 7febe1200640 1 -- 172.21.15.86:0/1107075116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febdc06e380 msgr2=0x7febdc15bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:49.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.254+0000 7febe1200640 1 --2- 172.21.15.86:0/1107075116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febdc06e380 0x7febdc15bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:49.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.254+0000 7febe1200640 1 -- 172.21.15.86:0/1107075116 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febdc0b6af0 con 0x7febdc06df70 2024-04-25T14:09:49.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.254+0000 7febe1200640 1 --2- 172.21.15.86:0/1107075116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febdc06df70 0x7febdc0b6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7febcc00e870 tx=0x7febcc00ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:49.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.254+0000 7febd9600640 1 -- 172.21.15.86:0/1107075116 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc017070 con 0x7febdc06df70 2024-04-25T14:09:49.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.254+0000 7febd9600640 1 -- 172.21.15.86:0/1107075116 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febcc00ee60 con 0x7febdc06df70 2024-04-25T14:09:49.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.254+0000 7febe2600640 1 -- 172.21.15.86:0/1107075116 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febdc0b6dd0 con 0x7febdc06df70 2024-04-25T14:09:49.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.254+0000 7febd9600640 1 -- 172.21.15.86:0/1107075116 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc004220 con 0x7febdc06df70 2024-04-25T14:09:49.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.254+0000 7febe2600640 1 -- 172.21.15.86:0/1107075116 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7febdc0b7d40 con 0x7febdc06df70 2024-04-25T14:09:49.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.258+0000 7febe2600640 1 -- 172.21.15.86:0/1107075116 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febdc076b80 con 0x7febdc06df70 2024-04-25T14:09:49.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.258+0000 7febd9600640 1 -- 172.21.15.86:0/1107075116 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 41) ==== 93253+0+0 (secure 0 0 0) 0x7febcc00c940 con 0x7febdc06df70 2024-04-25T14:09:49.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.258+0000 7febd9600640 1 --2- 172.21.15.86:0/1107075116 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7febb806f4c0 0x7febb80719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:49.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.258+0000 7febd9600640 1 -- 172.21.15.86:0/1107075116 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(101..101 src has 1..101) ==== 6445+0+0 (secure 0 0 0) 0x7febcc08f6b0 con 0x7febdc06df70 2024-04-25T14:09:49.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.258+0000 7febd3e00640 1 -- 172.21.15.86:0/1107075116 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7febb806f4c0 msgr2=0x7febb80719d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/436174229 2024-04-25T14:09:49.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.258+0000 7febd3e00640 1 --2- 172.21.15.86:0/1107075116 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7febb806f4c0 0x7febb80719d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:09:49.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.262+0000 7febd9600640 1 -- 172.21.15.86:0/1107075116 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7febcc05bc10 con 0x7febdc06df70 2024-04-25T14:09:49.304 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:09:49.303+0000 7f6dd142c180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:09:49.375 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:49.378+0000 7fb1a9257180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:09:49.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.398+0000 7febd9600640 1 -- 172.21.15.86:0/1107075116 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mgrmap(e 42) ==== 93253+0+0 (secure 0 0 0) 0x7febcc0043c0 con 0x7febdc06df70 2024-04-25T14:09:49.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.438+0000 7febe2600640 1 -- 172.21.15.86:0/1107075116 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7febdc0d30a0 con 0x7febdc06df70 2024-04-25T14:09:49.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.438+0000 7febd9600640 1 -- 172.21.15.86:0/1107075116 <== mon.2 v2:172.21.15.155:3301/0 8 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v101) ==== 80+0+468 (secure 0 0 0) 0x7febcc07a260 con 0x7febdc06df70 2024-04-25T14:09:49.436 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2887621339 2024-04-26T14:09:43.313864+0000 2024-04-25T14:09:49.436 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/361929172 2024-04-26T14:09:43.313864+0000 2024-04-25T14:09:49.436 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6833/4236874185 2024-04-26T14:09:43.313864+0000 2024-04-25T14:09:49.436 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6832/4236874185 2024-04-26T14:09:43.313864+0000 2024-04-25T14:09:49.436 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/789182356 2024-04-26T14:09:43.313864+0000 2024-04-25T14:09:49.437 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1532315242 2024-04-26T14:09:43.313864+0000 2024-04-25T14:09:49.437 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1775030266 2024-04-26T14:09:43.313864+0000 2024-04-25T14:09:49.437 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2226990349 2024-04-26T14:09:43.313864+0000 2024-04-25T14:09:49.437 INFO:teuthology.orchestra.run.smithi086.stderr:listed 8 entries 2024-04-25T14:09:49.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.442+0000 7febe2600640 1 -- 172.21.15.86:0/1107075116 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7febb806f4c0 msgr2=0x7febb80719d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:09:49.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.442+0000 7febe2600640 1 --2- 172.21.15.86:0/1107075116 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7febb806f4c0 0x7febb80719d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:49.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.442+0000 7febe2600640 1 -- 172.21.15.86:0/1107075116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febdc06df70 msgr2=0x7febdc0b6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:49.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.442+0000 7febe2600640 1 --2- 172.21.15.86:0/1107075116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febdc06df70 0x7febdc0b6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7febcc00e870 tx=0x7febcc00ed30 comp rx=0 tx=0).stop 2024-04-25T14:09:49.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.442+0000 7febe2600640 1 -- 172.21.15.86:0/1107075116 shutdown_connections 2024-04-25T14:09:49.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.442+0000 7febe2600640 1 --2- 172.21.15.86:0/1107075116 >> [v2:172.21.15.86:6832/436174229,v1:172.21.15.86:6833/436174229] conn(0x7febb806f4c0 0x7febb80719d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:49.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.442+0000 7febe2600640 1 --2- 172.21.15.86:0/1107075116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febdc163630 0x7febdc0b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:49.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.442+0000 7febe2600640 1 --2- 172.21.15.86:0/1107075116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febdc06e380 0x7febdc15bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:49.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.442+0000 7febe2600640 1 --2- 172.21.15.86:0/1107075116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febdc06df70 0x7febdc0b6070 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:49.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.442+0000 7febe2600640 1 -- 172.21.15.86:0/1107075116 >> 172.21.15.86:0/1107075116 conn(0x7febdc0d3330 msgr2=0x7febdc15d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:49.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.442+0000 7febe2600640 1 -- 172.21.15.86:0/1107075116 shutdown_connections 2024-04-25T14:09:49.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.442+0000 7febe2600640 1 -- 172.21.15.86:0/1107075116 wait complete. 2024-04-25T14:09:49.441 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:49.446+0000 7fb1a9257180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:09:49.463 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T14:09:49.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:09:49.464 DEBUG:teuthology.orchestra.run.smithi155:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T14:09:49.490 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T14:09:49.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:09:49.490 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T14:09:49.522 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:49.526+0000 7fb12ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:49.522 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:49.526+0000 7fb12ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:49.522 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:49.526+0000 7fb12ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:49.522 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:49.526+0000 7fb12ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:49.522 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:49.526+0000 7fb12ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:49.523 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:49.526+0000 7fb127000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:49.523 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:49.526+0000 7fb127000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:49.523 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:49.526+0000 7fb127000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:49.523 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:49.526+0000 7fb127000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:49.523 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:09:49.526+0000 7fb127000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:09:49.545 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:09:49.545 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:09:49.546 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:09:49.554 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2893 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T14:09:49.554 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:09:49.554 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:09:49.554 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:09:49.586 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:09:49.600 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:09:49.601 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4517 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T14:09:49.602 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:09:49.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.834+0000 7f46cfe00640 1 Processor -- start 2024-04-25T14:09:49.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.834+0000 7f46cfe00640 1 -- start start 2024-04-25T14:09:49.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.834+0000 7f46cfe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f46c8071530 0x7f46c806e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:49.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.834+0000 7f46cfe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46c806e7a0 0x7f46c80cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:49.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.834+0000 7f46cfe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46c806ef90 con 0x7f46c80cf970 2024-04-25T14:09:49.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.834+0000 7f46cfe00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f46c8163830 con 0x7f46c8071530 2024-04-25T14:09:49.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.834+0000 7f46cfe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f46c8163a80 con 0x7f46c806e7a0 2024-04-25T14:09:49.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.834+0000 7f46cd400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f46c80cf970 0x7f46c8070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34602/0 (socket says 172.21.15.86:34602) 2024-04-25T14:09:49.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.834+0000 7f46cd400640 1 -- 172.21.15.86:0/3333549517 learned_addr learned my addr 172.21.15.86:0/3333549517 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:49.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.834+0000 7f46cca00640 1 --2- 172.21.15.86:0/3333549517 >> v2:172.21.15.155:3300/0 conn(0x7f46c8071530 0x7f46c806e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:49.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.834+0000 7f46cde00640 1 --2- 172.21.15.86:0/3333549517 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46c806e7a0 0x7f46c80cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:49.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.834+0000 7f46c7e00640 1 -- 172.21.15.86:0/3333549517 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2312810678 0 0) 0x7f46c806ef90 con 0x7f46c80cf970 2024-04-25T14:09:49.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.834+0000 7f46c7e00640 1 -- 172.21.15.86:0/3333549517 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46b0003530 con 0x7f46c80cf970 2024-04-25T14:09:49.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.834+0000 7f46cca00640 1 -- 172.21.15.86:0/3333549517 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46c806e7a0 msgr2=0x7f46c80cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:49.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.834+0000 7f46cca00640 1 --2- 172.21.15.86:0/3333549517 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46c806e7a0 0x7f46c80cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:49.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.834+0000 7f46cca00640 1 -- 172.21.15.86:0/3333549517 >> v1:172.21.15.86:6789/0 conn(0x7f46c80cf970 legacy=0x7f46c8070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:49.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.838+0000 7f46cca00640 1 -- 172.21.15.86:0/3333549517 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c8163d60 con 0x7f46c8071530 2024-04-25T14:09:49.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.838+0000 7f46cde00640 1 --2- 172.21.15.86:0/3333549517 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46c806e7a0 0x7f46c80cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:09:49.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.838+0000 7f46cca00640 1 --2- 172.21.15.86:0/3333549517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46c8071530 0x7f46c806e260 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f46b40097a0 tx=0x7f46b401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13ef3a49adfacbd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:49.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.838+0000 7f46c7e00640 1 -- 172.21.15.86:0/3333549517 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b40a7070 con 0x7f46c8071530 2024-04-25T14:09:49.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.838+0000 7f46c7e00640 1 -- 172.21.15.86:0/3333549517 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46b40b0070 con 0x7f46c8071530 2024-04-25T14:09:49.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.838+0000 7f46c7e00640 1 -- 172.21.15.86:0/3333549517 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b40ab890 con 0x7f46c8071530 2024-04-25T14:09:49.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.842+0000 7f46cfe00640 1 -- 172.21.15.86:0/3333549517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46c8071530 msgr2=0x7f46c806e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:49.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.842+0000 7f46cfe00640 1 --2- 172.21.15.86:0/3333549517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46c8071530 0x7f46c806e260 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f46b40097a0 tx=0x7f46b401c450 comp rx=0 tx=0).stop 2024-04-25T14:09:49.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.842+0000 7f46cfe00640 1 -- 172.21.15.86:0/3333549517 shutdown_connections 2024-04-25T14:09:49.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.842+0000 7f46cfe00640 1 --2- 172.21.15.86:0/3333549517 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46c806e7a0 0x7f46c80cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:49.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.842+0000 7f46cfe00640 1 --2- 172.21.15.86:0/3333549517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46c8071530 0x7f46c806e260 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:49.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.842+0000 7f46cfe00640 1 -- 172.21.15.86:0/3333549517 >> 172.21.15.86:0/3333549517 conn(0x7f46c80cffc0 msgr2=0x7f46c80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:49.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.842+0000 7f46cfe00640 1 -- 172.21.15.86:0/3333549517 shutdown_connections 2024-04-25T14:09:49.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.842+0000 7f46cfe00640 1 -- 172.21.15.86:0/3333549517 wait complete. 2024-04-25T14:09:49.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.842+0000 7f46cfe00640 1 Processor -- start 2024-04-25T14:09:49.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.842+0000 7f46cfe00640 1 -- start start 2024-04-25T14:09:49.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.846+0000 7f46cfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46c806e7a0 0x7f46c8147500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:49.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.846+0000 7f46cfe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46c80cf970 0x7f46c80cce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:49.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.846+0000 7f46cfe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46c80b7df0 0x7f46c815ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:49.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.846+0000 7f46cfe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46c816cbd0 con 0x7f46c806e7a0 2024-04-25T14:09:49.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.846+0000 7f46cfe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f46c80ced80 con 0x7f46c80b7df0 2024-04-25T14:09:49.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.846+0000 7f46cfe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f46c816cd50 con 0x7f46c80cf970 2024-04-25T14:09:49.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.846+0000 7f46cd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46c806e7a0 0x7f46c8147500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:49.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.846+0000 7f46cd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46c806e7a0 0x7f46c8147500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43524/0 (socket says 172.21.15.86:43524) 2024-04-25T14:09:49.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.846+0000 7f46cd400640 1 -- 172.21.15.86:0/3342869983 learned_addr learned my addr 172.21.15.86:0/3342869983 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:49.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.846+0000 7f46cde00640 1 --2- 172.21.15.86:0/3342869983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46c80b7df0 0x7f46c815ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:49.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.846+0000 7f46cca00640 1 --2- 172.21.15.86:0/3342869983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46c80cf970 0x7f46c80cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:49.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.846+0000 7f46cd400640 1 -- 172.21.15.86:0/3342869983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46c80cf970 msgr2=0x7f46c80cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:49.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.846+0000 7f46cd400640 1 --2- 172.21.15.86:0/3342869983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46c80cf970 0x7f46c80cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:49.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.846+0000 7f46cd400640 1 -- 172.21.15.86:0/3342869983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46c80b7df0 msgr2=0x7f46c815ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:49.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.846+0000 7f46cd400640 1 --2- 172.21.15.86:0/3342869983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46c80b7df0 0x7f46c815ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:49.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.846+0000 7f46cd400640 1 -- 172.21.15.86:0/3342869983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c806d940 con 0x7f46c806e7a0 2024-04-25T14:09:49.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.846+0000 7f46cd400640 1 --2- 172.21.15.86:0/3342869983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46c806e7a0 0x7f46c8147500 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00c3f0 tx=0x7f46bc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:49.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.846+0000 7f46c5600640 1 -- 172.21.15.86:0/3342869983 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc017070 con 0x7f46c806e7a0 2024-04-25T14:09:49.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.846+0000 7f46c5600640 1 -- 172.21.15.86:0/3342869983 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46bc005cf0 con 0x7f46c806e7a0 2024-04-25T14:09:49.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.846+0000 7f46cfe00640 1 -- 172.21.15.86:0/3342869983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c80b84d0 con 0x7f46c806e7a0 2024-04-25T14:09:49.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.846+0000 7f46c5600640 1 -- 172.21.15.86:0/3342869983 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc0072f0 con 0x7f46c806e7a0 2024-04-25T14:09:49.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.846+0000 7f46cfe00640 1 -- 172.21.15.86:0/3342869983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c80b8a90 con 0x7f46c806e7a0 2024-04-25T14:09:49.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.846+0000 7f46cfe00640 1 -- 172.21.15.86:0/3342869983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46c8144b10 con 0x7f46c806e7a0 2024-04-25T14:09:49.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.850+0000 7f46c5600640 1 -- 172.21.15.86:0/3342869983 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 43) ==== 92918+0+0 (secure 0 0 0) 0x7f46bc01a070 con 0x7f46c806e7a0 2024-04-25T14:09:49.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.850+0000 7f46c5600640 1 -- 172.21.15.86:0/3342869983 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6746+0+0 (secure 0 0 0) 0x7f46bc08e480 con 0x7f46c806e7a0 2024-04-25T14:09:49.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:49.854+0000 7f46c5600640 1 -- 172.21.15.86:0/3342869983 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f46bc05adf0 con 0x7f46c806e7a0 2024-04-25T14:09:50.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.050+0000 7f46cfe00640 1 -- 172.21.15.86:0/3342869983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46c80720e0 con 0x7f46c806e7a0 2024-04-25T14:09:50.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.050+0000 7f46c5600640 1 -- 172.21.15.86:0/3342869983 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5189 (secure 0 0 0) 0x7f46bc05f0f0 con 0x7f46c806e7a0 2024-04-25T14:09:50.048 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:50.048 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":76,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1961914630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1961914630},{"type":"v1","addr":"172.21.15.86:6836","nonce":1961914630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2298829380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2298829380},{"type":"v1","addr":"172.21.15.155:6834","nonce":2298829380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7914,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3761704765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3761704765},{"type":"v1","addr":"172.21.15.86:6837","nonce":3761704765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:09:25.198798+0000","modified":"2024-04-25T14:09:28.237583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6835/1049360040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1049360040},{"type":"v1","addr":"172.21.15.155:6835","nonce":1049360040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7932,"qdb_cluster":[7932]},"id":7}]} 2024-04-25T14:09:50.048 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 76 2024-04-25T14:09:50.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.054+0000 7f46cfe00640 1 -- 172.21.15.86:0/3342869983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46c806e7a0 msgr2=0x7f46c8147500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:50.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.054+0000 7f46cfe00640 1 --2- 172.21.15.86:0/3342869983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46c806e7a0 0x7f46c8147500 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00c3f0 tx=0x7f46bc00c8b0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.054+0000 7f46cfe00640 1 -- 172.21.15.86:0/3342869983 shutdown_connections 2024-04-25T14:09:50.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.054+0000 7f46cfe00640 1 --2- 172.21.15.86:0/3342869983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46c80b7df0 0x7f46c815ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.054+0000 7f46cfe00640 1 --2- 172.21.15.86:0/3342869983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46c80cf970 0x7f46c80cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.054+0000 7f46cfe00640 1 --2- 172.21.15.86:0/3342869983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46c806e7a0 0x7f46c8147500 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.054+0000 7f46cfe00640 1 -- 172.21.15.86:0/3342869983 >> 172.21.15.86:0/3342869983 conn(0x7f46c80cffc0 msgr2=0x7f46c80b5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:50.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.054+0000 7f46cfe00640 1 -- 172.21.15.86:0/3342869983 shutdown_connections 2024-04-25T14:09:50.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.054+0000 7f46cfe00640 1 -- 172.21.15.86:0/3342869983 wait complete. 2024-04-25T14:09:50.070 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:09:50.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.222+0000 7ff4b9000640 1 Processor -- start 2024-04-25T14:09:50.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.222+0000 7ff4b9000640 1 -- start start 2024-04-25T14:09:50.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.222+0000 7ff4b9000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4b40cfca0 0x7ff4b406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:50.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.222+0000 7ff4b9000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff4b40d16e0 0x7ff4b40d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:50.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.222+0000 7ff4b9000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff4b406c670 0x7ff4b406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:50.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.222+0000 7ff4b9000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff4b4070470 con 0x7ff4b40d16e0 2024-04-25T14:09:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.222+0000 7ff4b9000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff4b40d2030 con 0x7ff4b406c670 2024-04-25T14:09:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.222+0000 7ff4b9000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff4b4072bd0 con 0x7ff4b40cfca0 2024-04-25T14:09:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.222+0000 7ff4b1400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff4b40d16e0 0x7ff4b40d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.222+0000 7ff4b1400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff4b40d16e0 0x7ff4b40d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43528/0 (socket says 172.21.15.86:43528) 2024-04-25T14:09:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.222+0000 7ff4b1400640 1 -- 172.21.15.86:0/1950748740 learned_addr learned my addr 172.21.15.86:0/1950748740 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.222+0000 7ff4b2800640 1 --2- 172.21.15.86:0/1950748740 >> v2:172.21.15.155:3300/0 conn(0x7ff4b406c670 0x7ff4b406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.222+0000 7ff4b1e00640 1 --2- 172.21.15.86:0/1950748740 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4b40cfca0 0x7ff4b406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.222+0000 7ff4b1400640 1 -- 172.21.15.86:0/1950748740 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4b40cfca0 msgr2=0x7ff4b406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.222+0000 7ff4b1400640 1 --2- 172.21.15.86:0/1950748740 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4b40cfca0 0x7ff4b406f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.222+0000 7ff4b1400640 1 -- 172.21.15.86:0/1950748740 >> v2:172.21.15.155:3300/0 conn(0x7ff4b406c670 msgr2=0x7ff4b406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.222+0000 7ff4b1400640 1 --2- 172.21.15.86:0/1950748740 >> v2:172.21.15.155:3300/0 conn(0x7ff4b406c670 0x7ff4b406ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.222+0000 7ff4b1400640 1 -- 172.21.15.86:0/1950748740 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b406f930 con 0x7ff4b40d16e0 2024-04-25T14:09:50.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.226+0000 7ff4b1400640 1 --2- 172.21.15.86:0/1950748740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4b40d16e0 0x7ff4b40d1af0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7ff4b406e060 tx=0x7ff4a001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe9b62be9c9faa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:50.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.226+0000 7ff4b0a00640 1 -- 172.21.15.86:0/1950748740 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a00ab5f0 con 0x7ff4b40d16e0 2024-04-25T14:09:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.226+0000 7ff4b0a00640 1 -- 172.21.15.86:0/1950748740 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4a00b0070 con 0x7ff4b40d16e0 2024-04-25T14:09:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.226+0000 7ff4b0a00640 1 -- 172.21.15.86:0/1950748740 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a00ab9b0 con 0x7ff4b40d16e0 2024-04-25T14:09:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.226+0000 7ff4b9000640 1 -- 172.21.15.86:0/1950748740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4b40d16e0 msgr2=0x7ff4b40d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.226+0000 7ff4b9000640 1 --2- 172.21.15.86:0/1950748740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4b40d16e0 0x7ff4b40d1af0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7ff4b406e060 tx=0x7ff4a001c520 comp rx=0 tx=0).stop 2024-04-25T14:09:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.226+0000 7ff4b9000640 1 -- 172.21.15.86:0/1950748740 shutdown_connections 2024-04-25T14:09:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.226+0000 7ff4b9000640 1 --2- 172.21.15.86:0/1950748740 >> v2:172.21.15.155:3300/0 conn(0x7ff4b406c670 0x7ff4b406ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.226+0000 7ff4b9000640 1 --2- 172.21.15.86:0/1950748740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4b40d16e0 0x7ff4b40d1af0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.226+0000 7ff4b9000640 1 --2- 172.21.15.86:0/1950748740 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4b40cfca0 0x7ff4b406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.226+0000 7ff4b9000640 1 -- 172.21.15.86:0/1950748740 >> 172.21.15.86:0/1950748740 conn(0x7ff4b40b4650 msgr2=0x7ff4b40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.226+0000 7ff4b9000640 1 -- 172.21.15.86:0/1950748740 shutdown_connections 2024-04-25T14:09:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.226+0000 7ff4b9000640 1 -- 172.21.15.86:0/1950748740 wait complete. 2024-04-25T14:09:50.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.230+0000 7ff4b9000640 1 Processor -- start 2024-04-25T14:09:50.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.230+0000 7ff4b9000640 1 -- start start 2024-04-25T14:09:50.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.230+0000 7ff4b9000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4b406c670 0x7ff4b41578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:50.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.230+0000 7ff4b9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4b40cfca0 0x7ff4b40cded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:50.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.230+0000 7ff4b9000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4b40d16e0 0x7ff4b40ce410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:50.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.230+0000 7ff4b9000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b40738c0 con 0x7ff4b40cfca0 2024-04-25T14:09:50.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.230+0000 7ff4b9000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b406d3d0 con 0x7ff4b406c670 2024-04-25T14:09:50.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.230+0000 7ff4b9000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff4b4073a40 con 0x7ff4b40d16e0 2024-04-25T14:09:50.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.230+0000 7ff4b1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4b40cfca0 0x7ff4b40cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:50.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.230+0000 7ff4b1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4b40cfca0 0x7ff4b40cded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43534/0 (socket says 172.21.15.86:43534) 2024-04-25T14:09:50.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.230+0000 7ff4b1400640 1 -- 172.21.15.86:0/503864863 learned_addr learned my addr 172.21.15.86:0/503864863 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:50.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.230+0000 7ff4b1e00640 1 --2- 172.21.15.86:0/503864863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4b406c670 0x7ff4b41578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:50.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.230+0000 7ff4b2800640 1 --2- 172.21.15.86:0/503864863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4b40d16e0 0x7ff4b40ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:50.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.230+0000 7ff4b1400640 1 -- 172.21.15.86:0/503864863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4b40d16e0 msgr2=0x7ff4b40ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:50.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.230+0000 7ff4b1400640 1 --2- 172.21.15.86:0/503864863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4b40d16e0 0x7ff4b40ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.230+0000 7ff4b1400640 1 -- 172.21.15.86:0/503864863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4b406c670 msgr2=0x7ff4b41578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:50.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.230+0000 7ff4b1400640 1 --2- 172.21.15.86:0/503864863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4b406c670 0x7ff4b41578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.230+0000 7ff4b1400640 1 -- 172.21.15.86:0/503864863 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b415ff50 con 0x7ff4b40cfca0 2024-04-25T14:09:50.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.230+0000 7ff4b1400640 1 --2- 172.21.15.86:0/503864863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4b40cfca0 0x7ff4b40cded0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7ff4a001ca00 tx=0x7ff4a001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:50.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.234+0000 7ff4aa000640 1 -- 172.21.15.86:0/503864863 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a00be3f0 con 0x7ff4b40cfca0 2024-04-25T14:09:50.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.234+0000 7ff4aa000640 1 -- 172.21.15.86:0/503864863 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4a00be5b0 con 0x7ff4b40cfca0 2024-04-25T14:09:50.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.234+0000 7ff4aa000640 1 -- 172.21.15.86:0/503864863 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a00be9a0 con 0x7ff4b40cfca0 2024-04-25T14:09:50.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.234+0000 7ff4b9000640 1 -- 172.21.15.86:0/503864863 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b409b450 con 0x7ff4b40cfca0 2024-04-25T14:09:50.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.234+0000 7ff4b9000640 1 -- 172.21.15.86:0/503864863 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b409ba10 con 0x7ff4b40cfca0 2024-04-25T14:09:50.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.234+0000 7ff4b9000640 1 -- 172.21.15.86:0/503864863 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4b409f4b0 con 0x7ff4b40cfca0 2024-04-25T14:09:50.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.234+0000 7ff4aa000640 1 -- 172.21.15.86:0/503864863 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 43) ==== 92918+0+0 (secure 0 0 0) 0x7ff4a00b43f0 con 0x7ff4b40cfca0 2024-04-25T14:09:50.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.234+0000 7ff4aa000640 1 -- 172.21.15.86:0/503864863 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6746+0+0 (secure 0 0 0) 0x7ff4a01293b0 con 0x7ff4b40cfca0 2024-04-25T14:09:50.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.238+0000 7ff4aa000640 1 -- 172.21.15.86:0/503864863 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff4a00c4060 con 0x7ff4b40cfca0 2024-04-25T14:09:50.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.398+0000 7ff4b9000640 1 -- 172.21.15.86:0/503864863 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4b415faf0 con 0x7ff4b40cfca0 2024-04-25T14:09:50.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.398+0000 7ff4aa000640 1 -- 172.21.15.86:0/503864863 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+17810 (secure 0 0 0) 0x7ff4a0016110 con 0x7ff4b40cfca0 2024-04-25T14:09:50.397 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:50.397 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":102,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:09:49.448189+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:09:23.588241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T14:09:24.156880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/3838196444":"2024-04-26T14:09:49.448138+0000","172.21.15.86:6833/436174229":"2024-04-26T14:09:49.448138+0000","172.21.15.86:0/129198987":"2024-04-26T14:09:49.448138+0000","172.21.15.86:0/2887621339":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/2356435633":"2024-04-26T14:09:49.448138+0000","172.21.15.86:6832/436174229":"2024-04-26T14:09:49.448138+0000","172.21.15.86:0/361929172":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/2077676736":"2024-04-26T14:09:49.448138+0000","172.21.15.86:6833/4236874185":"2024-04-26T14:09:43.313864+0000","172.21.15.86:6832/4236874185":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/746145476":"2024-04-26T14:09:49.448138+0000","172.21.15.86:0/789182356":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/1532315242":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/1775030266":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/2226990349":"2024-04-26T14:09:43.313864+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:09:50.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.402+0000 7ff4b9000640 1 -- 172.21.15.86:0/503864863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4b40cfca0 msgr2=0x7ff4b40cded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:50.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.402+0000 7ff4b9000640 1 --2- 172.21.15.86:0/503864863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4b40cfca0 0x7ff4b40cded0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7ff4a001ca00 tx=0x7ff4a001ad60 comp rx=0 tx=0).stop 2024-04-25T14:09:50.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.402+0000 7ff4b9000640 1 -- 172.21.15.86:0/503864863 shutdown_connections 2024-04-25T14:09:50.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.402+0000 7ff4b9000640 1 --2- 172.21.15.86:0/503864863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4b40d16e0 0x7ff4b40ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.402+0000 7ff4b9000640 1 --2- 172.21.15.86:0/503864863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4b40cfca0 0x7ff4b40cded0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.402+0000 7ff4b9000640 1 --2- 172.21.15.86:0/503864863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4b406c670 0x7ff4b41578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.402+0000 7ff4b9000640 1 -- 172.21.15.86:0/503864863 >> 172.21.15.86:0/503864863 conn(0x7ff4b40b4650 msgr2=0x7ff4b40afc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:50.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.402+0000 7ff4b9000640 1 -- 172.21.15.86:0/503864863 shutdown_connections 2024-04-25T14:09:50.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.402+0000 7ff4b9000640 1 -- 172.21.15.86:0/503864863 wait complete. 2024-04-25T14:09:50.418 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:09:23.588241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:09:24.156880+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:09:50.418 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:09:50.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.578+0000 7f14aac00640 1 Processor -- start 2024-04-25T14:09:50.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.578+0000 7f14aac00640 1 -- start start 2024-04-25T14:09:50.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f14aac00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f14a4070330 0x7f14a40d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:50.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f14aac00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14a40d2030 con 0x7f14a40ce790 2024-04-25T14:09:50.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f14aac00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14a415e5c0 con 0x7f14a40d1c20 2024-04-25T14:09:50.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f14aac00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f14a415e8a0 con 0x7f14a4070330 2024-04-25T14:09:50.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f14a8c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f14a40ce790 0x7f14a406f440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34608/0 (socket says 172.21.15.86:34608) 2024-04-25T14:09:50.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f14a8c00640 1 -- 172.21.15.86:0/216597102 learned_addr learned my addr 172.21.15.86:0/216597102 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:50.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f14a3e00640 1 --2- 172.21.15.86:0/216597102 >> v2:172.21.15.155:3300/0 conn(0x7f14a4070330 0x7f14a40d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f14a3400640 1 -- 172.21.15.86:0/216597102 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 945857387 0 0) 0x7f149c0014a0 con 0x7f14a40ce790 2024-04-25T14:09:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f14a3400640 1 -- 172.21.15.86:0/216597102 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1488003530 con 0x7f14a40ce790 2024-04-25T14:09:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f14a3400640 1 -- 172.21.15.86:0/216597102 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2373233309 0 0) 0x7f1488003530 con 0x7f14a40ce790 2024-04-25T14:09:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f14a3400640 1 -- 172.21.15.86:0/216597102 >> v2:172.21.15.155:3300/0 conn(0x7f14a4070330 msgr2=0x7f14a40d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f14a3400640 1 --2- 172.21.15.86:0/216597102 >> v2:172.21.15.155:3300/0 conn(0x7f14a4070330 0x7f14a40d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f14a3400640 1 -- 172.21.15.86:0/216597102 >> v1:172.21.15.155:6789/0 conn(0x7f14a40d1c20 legacy=0x7f14a406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f14a3400640 1 -- 172.21.15.86:0/216597102 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a415eb80 con 0x7f14a40ce790 2024-04-25T14:09:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f14a3400640 1 -- 172.21.15.86:0/216597102 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f149c001bd0 con 0x7f14a40ce790 2024-04-25T14:09:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f14a3400640 1 -- 172.21.15.86:0/216597102 >> v1:172.21.15.86:6789/0 conn(0x7f14a40ce790 legacy=0x7f14a406f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f14a3400640 1 --2- 172.21.15.86:0/216597102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1488003a30 0x7f1488003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f14a3400640 1 --2- 172.21.15.86:0/216597102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1488004810 0x7f1488004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:50.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f14a3400640 1 --2- 172.21.15.86:0/216597102 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1488005360 0x7f148800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:50.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f14a3400640 1 -- 172.21.15.86:0/216597102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14a415e8a0 con 0x7f1488004810 2024-04-25T14:09:50.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f14a3400640 1 -- 172.21.15.86:0/216597102 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f148800c260 con 0x7f1488003a30 2024-04-25T14:09:50.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f14a3400640 1 -- 172.21.15.86:0/216597102 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f148800c460 con 0x7f1488005360 2024-04-25T14:09:50.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f149b400640 1 --2- 172.21.15.86:0/216597102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1488004810 0x7f1488004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:50.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f14a8c00640 1 --2- 172.21.15.86:0/216597102 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1488005360 0x7f148800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:50.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f14a3e00640 1 --2- 172.21.15.86:0/216597102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1488003a30 0x7f1488003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:50.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f149b400640 1 -- 172.21.15.86:0/216597102 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1488005360 msgr2=0x7f148800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:50.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f149b400640 1 --2- 172.21.15.86:0/216597102 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1488005360 0x7f148800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f149b400640 1 -- 172.21.15.86:0/216597102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1488003a30 msgr2=0x7f1488003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:50.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f149b400640 1 --2- 172.21.15.86:0/216597102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1488003a30 0x7f1488003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f149b400640 1 -- 172.21.15.86:0/216597102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1488003530 con 0x7f1488004810 2024-04-25T14:09:50.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.582+0000 7f14a8c00640 1 -- 172.21.15.86:0/216597102 reap_dead start 2024-04-25T14:09:50.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.586+0000 7f149b400640 1 --2- 172.21.15.86:0/216597102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1488004810 0x7f1488004c20 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f14a4074ed0 tx=0x7f148c01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a870faca8100d801 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:50.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.586+0000 7f14a3400640 1 -- 172.21.15.86:0/216597102 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148c0013a0 con 0x7f1488004810 2024-04-25T14:09:50.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.586+0000 7f14a3400640 1 -- 172.21.15.86:0/216597102 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f148c0052f0 con 0x7f1488004810 2024-04-25T14:09:50.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.586+0000 7f14a3400640 1 -- 172.21.15.86:0/216597102 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148c003840 con 0x7f1488004810 2024-04-25T14:09:50.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.586+0000 7f14aac00640 1 -- 172.21.15.86:0/216597102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1488004810 msgr2=0x7f1488004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:50.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.586+0000 7f14aac00640 1 --2- 172.21.15.86:0/216597102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1488004810 0x7f1488004c20 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f14a4074ed0 tx=0x7f148c01c3a0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.586+0000 7f14aac00640 1 -- 172.21.15.86:0/216597102 shutdown_connections 2024-04-25T14:09:50.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.586+0000 7f14aac00640 1 --2- 172.21.15.86:0/216597102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1488004810 0x7f1488004c20 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.586+0000 7f14aac00640 1 -- 172.21.15.86:0/216597102 >> 172.21.15.86:0/216597102 conn(0x7f14a40b4650 msgr2=0x7f14a40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:50.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.586+0000 7f14aac00640 1 -- 172.21.15.86:0/216597102 shutdown_connections 2024-04-25T14:09:50.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.586+0000 7f14aac00640 1 -- 172.21.15.86:0/216597102 wait complete. 2024-04-25T14:09:50.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.590+0000 7f14aac00640 1 Processor -- start 2024-04-25T14:09:50.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.590+0000 7f14aac00640 1 -- start start 2024-04-25T14:09:50.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.590+0000 7f14aac00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1488004810 0x7f14a4160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:50.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.590+0000 7f14aac00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14a41f9b70 0x7f14a415a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:50.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.590+0000 7f14aac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a40b21d0 0x7f14a40b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:50.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.590+0000 7f14aac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14a40708c0 con 0x7f14a40b21d0 2024-04-25T14:09:50.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.590+0000 7f14aac00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f14a4070740 con 0x7f14a41f9b70 2024-04-25T14:09:50.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.590+0000 7f14aac00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f14a40cdf90 con 0x7f1488004810 2024-04-25T14:09:50.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.590+0000 7f14a8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a40b21d0 0x7f14a40b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:50.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.590+0000 7f14a8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a40b21d0 0x7f14a40b25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43552/0 (socket says 172.21.15.86:43552) 2024-04-25T14:09:50.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.590+0000 7f14a8c00640 1 -- 172.21.15.86:0/2588320193 learned_addr learned my addr 172.21.15.86:0/2588320193 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:50.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.590+0000 7f14a3e00640 1 --2- 172.21.15.86:0/2588320193 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1488004810 0x7f14a4160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:50.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.590+0000 7f149b400640 1 --2- 172.21.15.86:0/2588320193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14a41f9b70 0x7f14a415a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:50.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.590+0000 7f14a8c00640 1 -- 172.21.15.86:0/2588320193 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1488004810 msgr2=0x7f14a4160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:50.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.590+0000 7f14a8c00640 1 --2- 172.21.15.86:0/2588320193 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1488004810 0x7f14a4160280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.590+0000 7f14a8c00640 1 -- 172.21.15.86:0/2588320193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14a41f9b70 msgr2=0x7f14a415a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:50.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.590+0000 7f14a8c00640 1 --2- 172.21.15.86:0/2588320193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14a41f9b70 0x7f14a415a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.590+0000 7f14a8c00640 1 -- 172.21.15.86:0/2588320193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a40b2b20 con 0x7f14a40b21d0 2024-04-25T14:09:50.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.590+0000 7f14a3e00640 1 --2- 172.21.15.86:0/2588320193 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1488004810 0x7f14a4160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:50.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.590+0000 7f149b400640 1 --2- 172.21.15.86:0/2588320193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14a41f9b70 0x7f14a415a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:50.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.594+0000 7f14a8c00640 1 --2- 172.21.15.86:0/2588320193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a40b21d0 0x7f14a40b25e0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f14a415bd10 tx=0x7f149c00af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:50.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.594+0000 7f14a0c00640 1 -- 172.21.15.86:0/2588320193 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c007840 con 0x7f14a40b21d0 2024-04-25T14:09:50.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.594+0000 7f14a0c00640 1 -- 172.21.15.86:0/2588320193 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f149c019070 con 0x7f14a40b21d0 2024-04-25T14:09:50.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.594+0000 7f14a0c00640 1 -- 172.21.15.86:0/2588320193 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c0133f0 con 0x7f14a40b21d0 2024-04-25T14:09:50.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.594+0000 7f14aac00640 1 -- 172.21.15.86:0/2588320193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14a41fa290 con 0x7f14a40b21d0 2024-04-25T14:09:50.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.594+0000 7f14aac00640 1 -- 172.21.15.86:0/2588320193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14a41fa850 con 0x7f14a40b21d0 2024-04-25T14:09:50.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.594+0000 7f14aac00640 1 -- 172.21.15.86:0/2588320193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14a4069d00 con 0x7f14a40b21d0 2024-04-25T14:09:50.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.594+0000 7f14a0c00640 1 -- 172.21.15.86:0/2588320193 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 44) ==== 93045+0+0 (secure 0 0 0) 0x7f149c013590 con 0x7f14a40b21d0 2024-04-25T14:09:50.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.594+0000 7f14a0c00640 1 --2- 172.21.15.86:0/2588320193 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f148006f250 0x7f1480071760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:50.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.594+0000 7f14a0c00640 1 -- 172.21.15.86:0/2588320193 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6746+0+0 (secure 0 0 0) 0x7f149c08f8a0 con 0x7f14a40b21d0 2024-04-25T14:09:50.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.594+0000 7f14a3e00640 1 --2- 172.21.15.86:0/2588320193 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f148006f250 0x7f1480071760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:50.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.598+0000 7f14a3e00640 1 --2- 172.21.15.86:0/2588320193 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f148006f250 0x7f1480071760 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f14a40b3f10 tx=0x7f1494008040 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:50.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.602+0000 7f14a0c00640 1 -- 172.21.15.86:0/2588320193 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f14a40b2b20 con 0x7f14a40b21d0 2024-04-25T14:09:50.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.754+0000 7f14aac00640 1 -- 172.21.15.86:0/2588320193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f14a41fa020 con 0x7f14a40b21d0 2024-04-25T14:09:50.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.758+0000 7f14a0c00640 1 -- 172.21.15.86:0/2588320193 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) ==== 93+0+10 (secure 0 0 0) 0x7f14a40d2030 con 0x7f14a40b21d0 2024-04-25T14:09:50.754 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:09:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.758+0000 7f14aac00640 1 -- 172.21.15.86:0/2588320193 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f148006f250 msgr2=0x7f1480071760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.758+0000 7f14aac00640 1 --2- 172.21.15.86:0/2588320193 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f148006f250 0x7f1480071760 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f14a40b3f10 tx=0x7f1494008040 comp rx=0 tx=0).stop 2024-04-25T14:09:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.758+0000 7f14aac00640 1 -- 172.21.15.86:0/2588320193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a40b21d0 msgr2=0x7f14a40b25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.758+0000 7f14aac00640 1 --2- 172.21.15.86:0/2588320193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a40b21d0 0x7f14a40b25e0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f14a415bd10 tx=0x7f149c00af40 comp rx=0 tx=0).stop 2024-04-25T14:09:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.758+0000 7f14aac00640 1 -- 172.21.15.86:0/2588320193 shutdown_connections 2024-04-25T14:09:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.758+0000 7f14aac00640 1 --2- 172.21.15.86:0/2588320193 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f148006f250 0x7f1480071760 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.758+0000 7f14aac00640 1 --2- 172.21.15.86:0/2588320193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a40b21d0 0x7f14a40b25e0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.758+0000 7f14aac00640 1 --2- 172.21.15.86:0/2588320193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14a41f9b70 0x7f14a415a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.758+0000 7f14aac00640 1 --2- 172.21.15.86:0/2588320193 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1488004810 0x7f14a4160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.758+0000 7f14aac00640 1 -- 172.21.15.86:0/2588320193 >> 172.21.15.86:0/2588320193 conn(0x7f14a40b4650 msgr2=0x7f14a406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:50.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.758+0000 7f14aac00640 1 -- 172.21.15.86:0/2588320193 shutdown_connections 2024-04-25T14:09:50.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.758+0000 7f14aac00640 1 -- 172.21.15.86:0/2588320193 wait complete. 2024-04-25T14:09:50.775 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:09:50.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.926+0000 7f2cd2a00640 1 Processor -- start 2024-04-25T14:09:50.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.926+0000 7f2cd2a00640 1 -- start start 2024-04-25T14:09:50.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.930+0000 7f2cd2a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2ccc0d1df0 0x7f2ccc0cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:50.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.930+0000 7f2cd2a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ccc15e5c0 con 0x7f2ccc0cdcc0 2024-04-25T14:09:50.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.930+0000 7f2cd2a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ccc15e8a0 con 0x7f2ccc070330 2024-04-25T14:09:50.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.930+0000 7f2cd2a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2ccc06fd40 con 0x7f2ccc0d1df0 2024-04-25T14:09:50.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.930+0000 7f2cd0a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2ccc0cdcc0 0x7f2ccc0ce0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34610/0 (socket says 172.21.15.86:34610) 2024-04-25T14:09:50.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.930+0000 7f2cd0a00640 1 -- 172.21.15.86:0/2201923590 learned_addr learned my addr 172.21.15.86:0/2201923590 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:50.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.930+0000 7f2cc3400640 1 --2- 172.21.15.86:0/2201923590 >> v2:172.21.15.86:3300/0 conn(0x7f2ccc0d1df0 0x7f2ccc0cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:50.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.930+0000 7f2ccb400640 1 -- 172.21.15.86:0/2201923590 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1582857126 0 0) 0x7f2ccc15e5c0 con 0x7f2ccc0cdcc0 2024-04-25T14:09:50.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.930+0000 7f2ccb400640 1 -- 172.21.15.86:0/2201923590 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2cb0003530 con 0x7f2ccc0cdcc0 2024-04-25T14:09:50.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.930+0000 7f2cc3400640 1 -- 172.21.15.86:0/2201923590 >> v1:172.21.15.155:6789/0 conn(0x7f2ccc070330 legacy=0x7f2ccc0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:50.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.930+0000 7f2ccb400640 1 -- 172.21.15.86:0/2201923590 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3749053650 0 0) 0x7f2ccc15e8a0 con 0x7f2ccc070330 2024-04-25T14:09:50.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.930+0000 7f2cc3400640 1 -- 172.21.15.86:0/2201923590 >> v1:172.21.15.86:6789/0 conn(0x7f2ccc0cdcc0 legacy=0x7f2ccc0ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:50.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.930+0000 7f2cc3400640 1 -- 172.21.15.86:0/2201923590 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ccc15eb80 con 0x7f2ccc0d1df0 2024-04-25T14:09:50.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.934+0000 7f2cc3400640 1 --2- 172.21.15.86:0/2201923590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc0d1df0 0x7f2ccc0cd780 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f2cb40097f0 tx=0x7f2cb401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e647b592c2a83231 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:50.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.934+0000 7f2ccb400640 1 -- 172.21.15.86:0/2201923590 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb40b0070 con 0x7f2ccc0d1df0 2024-04-25T14:09:50.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.934+0000 7f2ccb400640 1 -- 172.21.15.86:0/2201923590 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cb40ab600 con 0x7f2ccc0d1df0 2024-04-25T14:09:50.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.934+0000 7f2ccb400640 1 -- 172.21.15.86:0/2201923590 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb40ab9d0 con 0x7f2ccc0d1df0 2024-04-25T14:09:50.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.934+0000 7f2cd2a00640 1 -- 172.21.15.86:0/2201923590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc0d1df0 msgr2=0x7f2ccc0cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:50.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.934+0000 7f2cd2a00640 1 --2- 172.21.15.86:0/2201923590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc0d1df0 0x7f2ccc0cd780 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f2cb40097f0 tx=0x7f2cb401c3e0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.934+0000 7f2cd2a00640 1 -- 172.21.15.86:0/2201923590 shutdown_connections 2024-04-25T14:09:50.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.934+0000 7f2cd2a00640 1 --2- 172.21.15.86:0/2201923590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc0d1df0 0x7f2ccc0cd780 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.934+0000 7f2cd2a00640 1 -- 172.21.15.86:0/2201923590 >> 172.21.15.86:0/2201923590 conn(0x7f2ccc0b4650 msgr2=0x7f2ccc0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:50.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.934+0000 7f2cd2a00640 1 -- 172.21.15.86:0/2201923590 shutdown_connections 2024-04-25T14:09:50.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.934+0000 7f2cd2a00640 1 -- 172.21.15.86:0/2201923590 wait complete. 2024-04-25T14:09:50.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.934+0000 7f2cd2a00640 1 Processor -- start 2024-04-25T14:09:50.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.934+0000 7f2cd2a00640 1 -- start start 2024-04-25T14:09:50.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.938+0000 7f2cd2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ccc070330 0x7f2ccc09a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:50.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.938+0000 7f2cd2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc0cdcc0 0x7f2ccc1503f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:50.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.938+0000 7f2cd2a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ccc15c970 0x7f2ccc150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:50.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.938+0000 7f2cd2a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc0708c0 con 0x7f2ccc0cdcc0 2024-04-25T14:09:50.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.938+0000 7f2cd2a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc070740 con 0x7f2ccc070330 2024-04-25T14:09:50.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.938+0000 7f2cd2a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2ccc16bfd0 con 0x7f2ccc15c970 2024-04-25T14:09:50.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.938+0000 7f2cc3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc0cdcc0 0x7f2ccc1503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:50.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.938+0000 7f2cc3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc0cdcc0 0x7f2ccc1503f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43568/0 (socket says 172.21.15.86:43568) 2024-04-25T14:09:50.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.938+0000 7f2cc3400640 1 -- 172.21.15.86:0/198255118 learned_addr learned my addr 172.21.15.86:0/198255118 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:50.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.938+0000 7f2ccbe00640 1 --2- 172.21.15.86:0/198255118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ccc070330 0x7f2ccc09a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:50.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.938+0000 7f2cd0a00640 1 --2- 172.21.15.86:0/198255118 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ccc15c970 0x7f2ccc150930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:50.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.938+0000 7f2cc3400640 1 -- 172.21.15.86:0/198255118 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ccc15c970 msgr2=0x7f2ccc150930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:50.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.938+0000 7f2cc3400640 1 --2- 172.21.15.86:0/198255118 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ccc15c970 0x7f2ccc150930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.938+0000 7f2cc3400640 1 -- 172.21.15.86:0/198255118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ccc070330 msgr2=0x7f2ccc09a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:50.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.938+0000 7f2cc3400640 1 --2- 172.21.15.86:0/198255118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ccc070330 0x7f2ccc09a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:50.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.938+0000 7f2cc3400640 1 -- 172.21.15.86:0/198255118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ccc09aa40 con 0x7f2ccc0cdcc0 2024-04-25T14:09:50.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.938+0000 7f2cc3400640 1 --2- 172.21.15.86:0/198255118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc0cdcc0 0x7f2ccc1503f0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f2cb40097f0 tx=0x7f2cb4019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:50.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.938+0000 7f2cc8c00640 1 -- 172.21.15.86:0/198255118 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb40bb070 con 0x7f2ccc0cdcc0 2024-04-25T14:09:50.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.938+0000 7f2cc8c00640 1 -- 172.21.15.86:0/198255118 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cb4004020 con 0x7f2ccc0cdcc0 2024-04-25T14:09:50.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.938+0000 7f2cc8c00640 1 -- 172.21.15.86:0/198255118 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb40b0040 con 0x7f2ccc0cdcc0 2024-04-25T14:09:50.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.938+0000 7f2cd2a00640 1 -- 172.21.15.86:0/198255118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ccc073fa0 con 0x7f2ccc0cdcc0 2024-04-25T14:09:50.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.938+0000 7f2cd2a00640 1 -- 172.21.15.86:0/198255118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ccc152260 con 0x7f2ccc0cdcc0 2024-04-25T14:09:50.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.942+0000 7f2cd2a00640 1 -- 172.21.15.86:0/198255118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ccc069d00 con 0x7f2ccc0cdcc0 2024-04-25T14:09:50.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.942+0000 7f2cc8c00640 1 -- 172.21.15.86:0/198255118 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 44) ==== 93045+0+0 (secure 0 0 0) 0x7f2cb40b3040 con 0x7f2ccc0cdcc0 2024-04-25T14:09:50.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.942+0000 7f2cc8c00640 1 --2- 172.21.15.86:0/198255118 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f2ca806f280 0x7f2ca8071790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:50.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.942+0000 7f2cc8c00640 1 -- 172.21.15.86:0/198255118 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6746+0+0 (secure 0 0 0) 0x7f2cb4129390 con 0x7f2ccc0cdcc0 2024-04-25T14:09:50.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.942+0000 7f2ccbe00640 1 --2- 172.21.15.86:0/198255118 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f2ca806f280 0x7f2ca8071790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.942+0000 7f2ccbe00640 1 --2- 172.21.15.86:0/198255118 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f2ca806f280 0x7f2ca8071790 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc0012f0 tx=0x7f2cbc009040 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:50.946+0000 7f2cc8c00640 1 -- 172.21.15.86:0/198255118 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2cb4016090 con 0x7f2ccc0cdcc0 2024-04-25T14:09:51.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.102+0000 7f2cd2a00640 1 -- 172.21.15.86:0/198255118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2ccc0afe60 con 0x7f2ccc0cdcc0 2024-04-25T14:09:51.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.102+0000 7f2cc8c00640 1 -- 172.21.15.86:0/198255118 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) ==== 104+0+11 (secure 0 0 0) 0x7f2cb40f5b20 con 0x7f2ccc0cdcc0 2024-04-25T14:09:51.100 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:09:51.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.106+0000 7f2cd2a00640 1 -- 172.21.15.86:0/198255118 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f2ca806f280 msgr2=0x7f2ca8071790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:51.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.106+0000 7f2cd2a00640 1 --2- 172.21.15.86:0/198255118 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f2ca806f280 0x7f2ca8071790 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc0012f0 tx=0x7f2cbc009040 comp rx=0 tx=0).stop 2024-04-25T14:09:51.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.106+0000 7f2cd2a00640 1 -- 172.21.15.86:0/198255118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc0cdcc0 msgr2=0x7f2ccc1503f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:51.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.106+0000 7f2cd2a00640 1 --2- 172.21.15.86:0/198255118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc0cdcc0 0x7f2ccc1503f0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f2cb40097f0 tx=0x7f2cb4019180 comp rx=0 tx=0).stop 2024-04-25T14:09:51.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.106+0000 7f2cd2a00640 1 -- 172.21.15.86:0/198255118 shutdown_connections 2024-04-25T14:09:51.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.106+0000 7f2cd2a00640 1 --2- 172.21.15.86:0/198255118 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f2ca806f280 0x7f2ca8071790 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:51.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.106+0000 7f2cd2a00640 1 --2- 172.21.15.86:0/198255118 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ccc15c970 0x7f2ccc150930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:51.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.106+0000 7f2cd2a00640 1 --2- 172.21.15.86:0/198255118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc0cdcc0 0x7f2ccc1503f0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:51.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.106+0000 7f2cd2a00640 1 --2- 172.21.15.86:0/198255118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ccc070330 0x7f2ccc09a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:51.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.106+0000 7f2cd2a00640 1 -- 172.21.15.86:0/198255118 >> 172.21.15.86:0/198255118 conn(0x7f2ccc0b4650 msgr2=0x7f2ccc15bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:51.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.106+0000 7f2cd2a00640 1 -- 172.21.15.86:0/198255118 shutdown_connections 2024-04-25T14:09:51.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.106+0000 7f2cd2a00640 1 -- 172.21.15.86:0/198255118 wait complete. 2024-04-25T14:09:51.121 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:09:51.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.274+0000 7f9475600640 1 Processor -- start 2024-04-25T14:09:51.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.274+0000 7f9475600640 1 -- start start 2024-04-25T14:09:51.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.274+0000 7f9475600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f947015cc90 0x7f9470160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:51.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.274+0000 7f9475600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94700b3d60 con 0x7f94700b3950 2024-04-25T14:09:51.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.274+0000 7f9475600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94700b1d50 con 0x7f94700af8b0 2024-04-25T14:09:51.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.274+0000 7f9475600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f94700af140 con 0x7f947015cc90 2024-04-25T14:09:51.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.274+0000 7f946e000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f947015cc90 0x7f9470160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:51.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.274+0000 7f946ea00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f94700b3950 0x7f94701607a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34614/0 (socket says 172.21.15.86:34614) 2024-04-25T14:09:51.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.274+0000 7f946ea00640 1 -- 172.21.15.86:0/706697548 learned_addr learned my addr 172.21.15.86:0/706697548 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:51.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.278+0000 7f946e000640 1 -- 172.21.15.86:0/706697548 >> v1:172.21.15.155:6789/0 conn(0x7f94700af8b0 legacy=0x7f94700afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:51.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.278+0000 7f946e000640 1 -- 172.21.15.86:0/706697548 >> v1:172.21.15.86:6789/0 conn(0x7f94700b3950 legacy=0x7f94701607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:51.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.278+0000 7f946d600640 1 -- 172.21.15.86:0/706697548 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2834195823 0 0) 0x7f94700b3d60 con 0x7f94700b3950 2024-04-25T14:09:51.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.278+0000 7f946e000640 1 -- 172.21.15.86:0/706697548 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f947015d0a0 con 0x7f947015cc90 2024-04-25T14:09:51.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.278+0000 7f946e000640 1 --2- 172.21.15.86:0/706697548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f947015cc90 0x7f9470160eb0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f94700b1f20 tx=0x7f945c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52fb3bac8c672b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:51.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.278+0000 7f946d600640 1 -- 172.21.15.86:0/706697548 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c0b0070 con 0x7f947015cc90 2024-04-25T14:09:51.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.278+0000 7f946d600640 1 -- 172.21.15.86:0/706697548 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f945c0ab600 con 0x7f947015cc90 2024-04-25T14:09:51.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.278+0000 7f946d600640 1 -- 172.21.15.86:0/706697548 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94700b0be0 con 0x7f947015cc90 2024-04-25T14:09:51.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.278+0000 7f9475600640 1 -- 172.21.15.86:0/706697548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f947015cc90 msgr2=0x7f9470160eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:51.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.278+0000 7f9475600640 1 --2- 172.21.15.86:0/706697548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f947015cc90 0x7f9470160eb0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f94700b1f20 tx=0x7f945c01c3d0 comp rx=0 tx=0).stop 2024-04-25T14:09:51.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.282+0000 7f9475600640 1 -- 172.21.15.86:0/706697548 shutdown_connections 2024-04-25T14:09:51.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.282+0000 7f9475600640 1 --2- 172.21.15.86:0/706697548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f947015cc90 0x7f9470160eb0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:51.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.282+0000 7f9475600640 1 -- 172.21.15.86:0/706697548 >> 172.21.15.86:0/706697548 conn(0x7f947009ba30 msgr2=0x7f94700b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:51.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.282+0000 7f9475600640 1 -- 172.21.15.86:0/706697548 shutdown_connections 2024-04-25T14:09:51.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.282+0000 7f9475600640 1 -- 172.21.15.86:0/706697548 wait complete. 2024-04-25T14:09:51.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.282+0000 7f9475600640 1 Processor -- start 2024-04-25T14:09:51.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.282+0000 7f9475600640 1 -- start start 2024-04-25T14:09:51.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.282+0000 7f9475600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94700af8b0 0x7f9470148ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:51.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.282+0000 7f9475600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94700b3950 0x7f94700d0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:51.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.282+0000 7f9475600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f947015cc90 0x7f94700d0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:51.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.282+0000 7f9475600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f947016ac70 con 0x7f947015cc90 2024-04-25T14:09:51.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.282+0000 7f9475600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f947016aaf0 con 0x7f94700af8b0 2024-04-25T14:09:51.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.282+0000 7f9475600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f947016adf0 con 0x7f94700b3950 2024-04-25T14:09:51.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.286+0000 7f946f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f947015cc90 0x7f94700d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:51.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.286+0000 7f946f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f947015cc90 0x7f94700d0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43582/0 (socket says 172.21.15.86:43582) 2024-04-25T14:09:51.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.286+0000 7f946f400640 1 -- 172.21.15.86:0/3496341262 learned_addr learned my addr 172.21.15.86:0/3496341262 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:51.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.286+0000 7f946e000640 1 --2- 172.21.15.86:0/3496341262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94700b3950 0x7f94700d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:51.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.286+0000 7f946ea00640 1 --2- 172.21.15.86:0/3496341262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94700af8b0 0x7f9470148ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:51.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.286+0000 7f946f400640 1 -- 172.21.15.86:0/3496341262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94700b3950 msgr2=0x7f94700d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:51.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.286+0000 7f946f400640 1 --2- 172.21.15.86:0/3496341262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94700b3950 0x7f94700d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:51.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.286+0000 7f946f400640 1 -- 172.21.15.86:0/3496341262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94700af8b0 msgr2=0x7f9470148ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:51.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.286+0000 7f946f400640 1 --2- 172.21.15.86:0/3496341262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94700af8b0 0x7f9470148ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:51.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.286+0000 7f946f400640 1 -- 172.21.15.86:0/3496341262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94700d2910 con 0x7f947015cc90 2024-04-25T14:09:51.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.286+0000 7f946f400640 1 --2- 172.21.15.86:0/3496341262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f947015cc90 0x7f94700d0980 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f946800b440 tx=0x7f946800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:51.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.286+0000 7f9466a00640 1 -- 172.21.15.86:0/3496341262 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9468015020 con 0x7f947015cc90 2024-04-25T14:09:51.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.286+0000 7f9466a00640 1 -- 172.21.15.86:0/3496341262 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9468011850 con 0x7f947015cc90 2024-04-25T14:09:51.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.286+0000 7f9475600640 1 -- 172.21.15.86:0/3496341262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94700d2bf0 con 0x7f947015cc90 2024-04-25T14:09:51.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.286+0000 7f9466a00640 1 -- 172.21.15.86:0/3496341262 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9468011c40 con 0x7f947015cc90 2024-04-25T14:09:51.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.286+0000 7f9475600640 1 -- 172.21.15.86:0/3496341262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f947014a720 con 0x7f947015cc90 2024-04-25T14:09:51.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.286+0000 7f9475600640 1 -- 172.21.15.86:0/3496341262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94700a04c0 con 0x7f947015cc90 2024-04-25T14:09:51.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.286+0000 7f9466a00640 1 -- 172.21.15.86:0/3496341262 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 44) ==== 93045+0+0 (secure 0 0 0) 0x7f9468018020 con 0x7f947015cc90 2024-04-25T14:09:51.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.290+0000 7f9466a00640 1 --2- 172.21.15.86:0/3496341262 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f945006f250 0x7f9450071760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:51.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.290+0000 7f9466a00640 1 -- 172.21.15.86:0/3496341262 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6746+0+0 (secure 0 0 0) 0x7f946808edb0 con 0x7f947015cc90 2024-04-25T14:09:51.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.290+0000 7f946ea00640 1 --2- 172.21.15.86:0/3496341262 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f945006f250 0x7f9450071760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:51.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.290+0000 7f946ea00640 1 --2- 172.21.15.86:0/3496341262 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f945006f250 0x7f9450071760 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9460001470 tx=0x7f9460009040 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:51.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.294+0000 7f9466a00640 1 -- 172.21.15.86:0/3496341262 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9468093070 con 0x7f947015cc90 2024-04-25T14:09:51.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.450+0000 7f9475600640 1 -- 172.21.15.86:0/3496341262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9470076bf0 con 0x7f947015cc90 2024-04-25T14:09:51.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.450+0000 7f9466a00640 1 -- 172.21.15.86:0/3496341262 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) ==== 100+0+11 (secure 0 0 0) 0x7f946801f030 con 0x7f947015cc90 2024-04-25T14:09:51.449 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:09:51.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.454+0000 7f9475600640 1 -- 172.21.15.86:0/3496341262 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f945006f250 msgr2=0x7f9450071760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:51.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.454+0000 7f9475600640 1 --2- 172.21.15.86:0/3496341262 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f945006f250 0x7f9450071760 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9460001470 tx=0x7f9460009040 comp rx=0 tx=0).stop 2024-04-25T14:09:51.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.454+0000 7f9475600640 1 -- 172.21.15.86:0/3496341262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f947015cc90 msgr2=0x7f94700d0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:51.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.454+0000 7f9475600640 1 --2- 172.21.15.86:0/3496341262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f947015cc90 0x7f94700d0980 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f946800b440 tx=0x7f946800b900 comp rx=0 tx=0).stop 2024-04-25T14:09:51.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.454+0000 7f9475600640 1 -- 172.21.15.86:0/3496341262 shutdown_connections 2024-04-25T14:09:51.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.454+0000 7f9475600640 1 --2- 172.21.15.86:0/3496341262 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f945006f250 0x7f9450071760 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:51.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.454+0000 7f9475600640 1 --2- 172.21.15.86:0/3496341262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f947015cc90 0x7f94700d0980 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:51.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.454+0000 7f9475600640 1 --2- 172.21.15.86:0/3496341262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94700b3950 0x7f94700d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:51.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.454+0000 7f9475600640 1 --2- 172.21.15.86:0/3496341262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94700af8b0 0x7f9470148ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:51.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.454+0000 7f9475600640 1 -- 172.21.15.86:0/3496341262 >> 172.21.15.86:0/3496341262 conn(0x7f947009ba30 msgr2=0x7f947006ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:51.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.454+0000 7f9475600640 1 -- 172.21.15.86:0/3496341262 shutdown_connections 2024-04-25T14:09:51.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.454+0000 7f9475600640 1 -- 172.21.15.86:0/3496341262 wait complete. 2024-04-25T14:09:51.472 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:09:51.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.626+0000 7fac2a600640 1 Processor -- start 2024-04-25T14:09:51.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.626+0000 7fac2a600640 1 -- start start 2024-04-25T14:09:51.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.630+0000 7fac2a600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fac240d16e0 0x7fac240d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:51.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.630+0000 7fac2a600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac240730b0 0x7fac240734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:51.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.630+0000 7fac2a600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac240d2030 con 0x7fac240cfca0 2024-04-25T14:09:51.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.630+0000 7fac2a600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fac24070470 con 0x7fac240d16e0 2024-04-25T14:09:51.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.630+0000 7fac2a600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fac24072bd0 con 0x7fac240730b0 2024-04-25T14:09:51.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.630+0000 7fac23400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fac240cfca0 0x7fac2406f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34630/0 (socket says 172.21.15.86:34630) 2024-04-25T14:09:51.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.630+0000 7fac1aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fac240d16e0 0x7fac240d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:51.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.630+0000 7fac23400640 1 -- 172.21.15.86:0/2269210107 learned_addr learned my addr 172.21.15.86:0/2269210107 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:51.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.630+0000 7fac23e00640 1 --2- 172.21.15.86:0/2269210107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac240730b0 0x7fac240734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:51.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.630+0000 7fac1aa00640 1 -- 172.21.15.86:0/2269210107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac240730b0 msgr2=0x7fac240734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:51.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.630+0000 7fac1aa00640 1 --2- 172.21.15.86:0/2269210107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac240730b0 0x7fac240734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:51.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.630+0000 7fac1aa00640 1 -- 172.21.15.86:0/2269210107 >> v1:172.21.15.86:6789/0 conn(0x7fac240cfca0 legacy=0x7fac2406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:51.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.630+0000 7fac22a00640 1 -- 172.21.15.86:0/2269210107 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1651510550 0 0) 0x7fac140014a0 con 0x7fac240cfca0 2024-04-25T14:09:51.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.630+0000 7fac1aa00640 1 -- 172.21.15.86:0/2269210107 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2406fbd0 con 0x7fac240d16e0 2024-04-25T14:09:51.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.630+0000 7fac1aa00640 1 --2- 172.21.15.86:0/2269210107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac240d16e0 0x7fac240d1af0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fac0c0097c0 tx=0x7fac0c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24d4b8cb80413e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:51.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.634+0000 7fac22a00640 1 -- 172.21.15.86:0/2269210107 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c0b1070 con 0x7fac240d16e0 2024-04-25T14:09:51.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.634+0000 7fac22a00640 1 -- 172.21.15.86:0/2269210107 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac0c0ac410 con 0x7fac240d16e0 2024-04-25T14:09:51.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.634+0000 7fac22a00640 1 -- 172.21.15.86:0/2269210107 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c0ac7e0 con 0x7fac240d16e0 2024-04-25T14:09:51.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.634+0000 7fac2a600640 1 -- 172.21.15.86:0/2269210107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac240d16e0 msgr2=0x7fac240d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:51.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.634+0000 7fac2a600640 1 --2- 172.21.15.86:0/2269210107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac240d16e0 0x7fac240d1af0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fac0c0097c0 tx=0x7fac0c01c4a0 comp rx=0 tx=0).stop 2024-04-25T14:09:51.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.634+0000 7fac2a600640 1 -- 172.21.15.86:0/2269210107 shutdown_connections 2024-04-25T14:09:51.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.634+0000 7fac2a600640 1 --2- 172.21.15.86:0/2269210107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac240730b0 0x7fac240734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:51.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.634+0000 7fac2a600640 1 --2- 172.21.15.86:0/2269210107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac240d16e0 0x7fac240d1af0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:51.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.634+0000 7fac2a600640 1 -- 172.21.15.86:0/2269210107 >> 172.21.15.86:0/2269210107 conn(0x7fac240b4650 msgr2=0x7fac240b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:51.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.634+0000 7fac2a600640 1 -- 172.21.15.86:0/2269210107 shutdown_connections 2024-04-25T14:09:51.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.634+0000 7fac2a600640 1 -- 172.21.15.86:0/2269210107 wait complete. 2024-04-25T14:09:51.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.634+0000 7fac2a600640 1 Processor -- start 2024-04-25T14:09:51.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.634+0000 7fac2a600640 1 -- start start 2024-04-25T14:09:51.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.638+0000 7fac2a600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac240730b0 0x7fac240afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:51.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.638+0000 7fac2a600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac240cfca0 0x7fac24159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:51.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.638+0000 7fac2a600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac240d16e0 0x7fac241f9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:51.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.638+0000 7fac2a600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac24169a70 con 0x7fac240cfca0 2024-04-25T14:09:51.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.638+0000 7fac2a600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fac241698f0 con 0x7fac240730b0 2024-04-25T14:09:51.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.638+0000 7fac2a600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fac24169bf0 con 0x7fac240d16e0 2024-04-25T14:09:51.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.638+0000 7fac1aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac240cfca0 0x7fac24159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:51.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.638+0000 7fac1aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac240cfca0 0x7fac24159d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43620/0 (socket says 172.21.15.86:43620) 2024-04-25T14:09:51.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.638+0000 7fac1aa00640 1 -- 172.21.15.86:0/3209046864 learned_addr learned my addr 172.21.15.86:0/3209046864 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:51.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.638+0000 7fac23400640 1 --2- 172.21.15.86:0/3209046864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac240730b0 0x7fac240afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:51.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.638+0000 7fac23e00640 1 --2- 172.21.15.86:0/3209046864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac240d16e0 0x7fac241f9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:51.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.638+0000 7fac1aa00640 1 -- 172.21.15.86:0/3209046864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac240d16e0 msgr2=0x7fac241f9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:51.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.638+0000 7fac1aa00640 1 --2- 172.21.15.86:0/3209046864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac240d16e0 0x7fac241f9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:51.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.638+0000 7fac1aa00640 1 -- 172.21.15.86:0/3209046864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac240730b0 msgr2=0x7fac240afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:51.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.638+0000 7fac1aa00640 1 --2- 172.21.15.86:0/3209046864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac240730b0 0x7fac240afd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:51.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.638+0000 7fac1aa00640 1 -- 172.21.15.86:0/3209046864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2409aea0 con 0x7fac240cfca0 2024-04-25T14:09:51.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.638+0000 7fac1aa00640 1 --2- 172.21.15.86:0/3209046864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac240cfca0 0x7fac24159d80 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fac0c0097c0 tx=0x7fac0c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:51.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.638+0000 7fac1be00640 1 -- 172.21.15.86:0/3209046864 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c0bc070 con 0x7fac240cfca0 2024-04-25T14:09:51.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.638+0000 7fac1be00640 1 -- 172.21.15.86:0/3209046864 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac0c0acd70 con 0x7fac240cfca0 2024-04-25T14:09:51.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.638+0000 7fac2a600640 1 -- 172.21.15.86:0/3209046864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac240aece0 con 0x7fac240cfca0 2024-04-25T14:09:51.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.638+0000 7fac1be00640 1 -- 172.21.15.86:0/3209046864 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c0b1040 con 0x7fac240cfca0 2024-04-25T14:09:51.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.638+0000 7fac2a600640 1 -- 172.21.15.86:0/3209046864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac240af2a0 con 0x7fac240cfca0 2024-04-25T14:09:51.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.638+0000 7fac2a600640 1 -- 172.21.15.86:0/3209046864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac2409f4b0 con 0x7fac240cfca0 2024-04-25T14:09:51.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.642+0000 7fac1be00640 1 -- 172.21.15.86:0/3209046864 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 44) ==== 93045+0+0 (secure 0 0 0) 0x7fac0c0bf070 con 0x7fac240cfca0 2024-04-25T14:09:51.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.642+0000 7fac1be00640 1 --2- 172.21.15.86:0/3209046864 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fac0006f280 0x7fac00071790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:51.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.642+0000 7fac1be00640 1 -- 172.21.15.86:0/3209046864 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6746+0+0 (secure 0 0 0) 0x7fac0c128230 con 0x7fac240cfca0 2024-04-25T14:09:51.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.642+0000 7fac23400640 1 --2- 172.21.15.86:0/3209046864 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fac0006f280 0x7fac00071790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:51.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.642+0000 7fac23400640 1 --2- 172.21.15.86:0/3209046864 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fac0006f280 0x7fac00071790 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fac140012f0 tx=0x7fac14001690 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:51.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.646+0000 7fac1be00640 1 -- 172.21.15.86:0/3209046864 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fac0c016090 con 0x7fac240cfca0 2024-04-25T14:09:51.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.834+0000 7fac2a600640 1 -- 172.21.15.86:0/3209046864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac24069d00 con 0x7fac240cfca0 2024-04-25T14:09:51.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.838+0000 7fac1be00640 1 -- 172.21.15.86:0/3209046864 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5189 (secure 0 0 0) 0x7fac0c0f4b50 con 0x7fac240cfca0 2024-04-25T14:09:51.833 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:51.833 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":76,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1961914630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1961914630},{"type":"v1","addr":"172.21.15.86:6836","nonce":1961914630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2298829380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2298829380},{"type":"v1","addr":"172.21.15.155:6834","nonce":2298829380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7914,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3761704765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3761704765},{"type":"v1","addr":"172.21.15.86:6837","nonce":3761704765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:09:25.198798+0000","modified":"2024-04-25T14:09:28.237583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6835/1049360040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1049360040},{"type":"v1","addr":"172.21.15.155:6835","nonce":1049360040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7932,"qdb_cluster":[7932]},"id":7}]} 2024-04-25T14:09:51.834 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 76 2024-04-25T14:09:51.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.838+0000 7fac2a600640 1 -- 172.21.15.86:0/3209046864 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fac0006f280 msgr2=0x7fac00071790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:51.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.838+0000 7fac2a600640 1 --2- 172.21.15.86:0/3209046864 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fac0006f280 0x7fac00071790 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fac140012f0 tx=0x7fac14001690 comp rx=0 tx=0).stop 2024-04-25T14:09:51.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.838+0000 7fac2a600640 1 -- 172.21.15.86:0/3209046864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac240cfca0 msgr2=0x7fac24159d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:51.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.838+0000 7fac2a600640 1 --2- 172.21.15.86:0/3209046864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac240cfca0 0x7fac24159d80 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fac0c0097c0 tx=0x7fac0c019240 comp rx=0 tx=0).stop 2024-04-25T14:09:51.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.838+0000 7fac2a600640 1 -- 172.21.15.86:0/3209046864 shutdown_connections 2024-04-25T14:09:51.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.838+0000 7fac2a600640 1 --2- 172.21.15.86:0/3209046864 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fac0006f280 0x7fac00071790 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:51.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.838+0000 7fac2a600640 1 --2- 172.21.15.86:0/3209046864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac240d16e0 0x7fac241f9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:51.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.838+0000 7fac2a600640 1 --2- 172.21.15.86:0/3209046864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac240cfca0 0x7fac24159d80 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:51.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.838+0000 7fac2a600640 1 --2- 172.21.15.86:0/3209046864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac240730b0 0x7fac240afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:51.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.838+0000 7fac2a600640 1 -- 172.21.15.86:0/3209046864 >> 172.21.15.86:0/3209046864 conn(0x7fac240b4650 msgr2=0x7fac2415c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:51.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.838+0000 7fac2a600640 1 -- 172.21.15.86:0/3209046864 shutdown_connections 2024-04-25T14:09:51.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:51.838+0000 7fac2a600640 1 -- 172.21.15.86:0/3209046864 wait complete. 2024-04-25T14:09:51.855 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:09:52.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.006+0000 7faa2b400640 1 Processor -- start 2024-04-25T14:09:52.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.006+0000 7faa2b400640 1 -- start start 2024-04-25T14:09:52.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.006+0000 7faa2b400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa240b08c0 0x7faa24167ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:52.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.006+0000 7faa2b400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa240b3d60 con 0x7faa2415cc90 2024-04-25T14:09:52.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.006+0000 7faa2b400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa240b1d50 con 0x7faa240b3950 2024-04-25T14:09:52.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.006+0000 7faa2b400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7faa240af140 con 0x7faa240b08c0 2024-04-25T14:09:52.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.006+0000 7faa23e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7faa2415cc90 0x7faa24160eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34646/0 (socket says 172.21.15.86:34646) 2024-04-25T14:09:52.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.006+0000 7faa23e00640 1 -- 172.21.15.86:0/44710281 learned_addr learned my addr 172.21.15.86:0/44710281 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:52.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.006+0000 7faa29400640 1 --2- 172.21.15.86:0/44710281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa240b08c0 0x7faa24167ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:52.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.006+0000 7faa23400640 1 -- 172.21.15.86:0/44710281 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1203087888 0 0) 0x7faa240b3d60 con 0x7faa2415cc90 2024-04-25T14:09:52.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.006+0000 7faa23400640 1 -- 172.21.15.86:0/44710281 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa0c003530 con 0x7faa2415cc90 2024-04-25T14:09:52.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.006+0000 7faa23400640 1 -- 172.21.15.86:0/44710281 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1433394605 0 0) 0x7faa240b1d50 con 0x7faa240b3950 2024-04-25T14:09:52.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.006+0000 7faa23400640 1 -- 172.21.15.86:0/44710281 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa240b3d60 con 0x7faa240b3950 2024-04-25T14:09:52.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.006+0000 7faa29400640 1 -- 172.21.15.86:0/44710281 >> v1:172.21.15.155:6789/0 conn(0x7faa240b3950 legacy=0x7faa241607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:52.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.006+0000 7faa29400640 1 -- 172.21.15.86:0/44710281 >> v1:172.21.15.86:6789/0 conn(0x7faa2415cc90 legacy=0x7faa24160eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:52.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.006+0000 7faa29400640 1 -- 172.21.15.86:0/44710281 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa2415d0a0 con 0x7faa240b08c0 2024-04-25T14:09:52.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.010+0000 7faa29400640 1 --2- 172.21.15.86:0/44710281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa240b08c0 0x7faa24167ad0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faa240af2c0 tx=0x7faa1c01c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a43681815c0927eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:52.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.010+0000 7faa23400640 1 -- 172.21.15.86:0/44710281 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c0b2070 con 0x7faa240b08c0 2024-04-25T14:09:52.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.010+0000 7faa23400640 1 -- 172.21.15.86:0/44710281 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa1c0a58d0 con 0x7faa240b08c0 2024-04-25T14:09:52.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.010+0000 7faa23400640 1 -- 172.21.15.86:0/44710281 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c0a5ca0 con 0x7faa240b08c0 2024-04-25T14:09:52.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.010+0000 7faa2b400640 1 -- 172.21.15.86:0/44710281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa240b08c0 msgr2=0x7faa24167ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:52.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.010+0000 7faa2b400640 1 --2- 172.21.15.86:0/44710281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa240b08c0 0x7faa24167ad0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faa240af2c0 tx=0x7faa1c01c900 comp rx=0 tx=0).stop 2024-04-25T14:09:52.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.010+0000 7faa2b400640 1 -- 172.21.15.86:0/44710281 shutdown_connections 2024-04-25T14:09:52.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.010+0000 7faa2b400640 1 --2- 172.21.15.86:0/44710281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa240b08c0 0x7faa24167ad0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.010+0000 7faa2b400640 1 -- 172.21.15.86:0/44710281 >> 172.21.15.86:0/44710281 conn(0x7faa2409ba30 msgr2=0x7faa240b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:52.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.010+0000 7faa2b400640 1 -- 172.21.15.86:0/44710281 shutdown_connections 2024-04-25T14:09:52.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.010+0000 7faa2b400640 1 -- 172.21.15.86:0/44710281 wait complete. 2024-04-25T14:09:52.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.014+0000 7faa2b400640 1 Processor -- start 2024-04-25T14:09:52.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.014+0000 7faa2b400640 1 -- start start 2024-04-25T14:09:52.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.014+0000 7faa2b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa240b08c0 0x7faa240d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.014+0000 7faa2b400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa240b3950 0x7faa2414fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.014+0000 7faa2b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa2415cc90 0x7faa2415dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.014+0000 7faa2b400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa2406ca00 con 0x7faa2415cc90 2024-04-25T14:09:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.014+0000 7faa2b400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faa2406c880 con 0x7faa240b08c0 2024-04-25T14:09:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.014+0000 7faa2b400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7faa2406cb80 con 0x7faa240b3950 2024-04-25T14:09:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.014+0000 7faa29400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa2415cc90 0x7faa2415dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.014+0000 7faa29400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa2415cc90 0x7faa2415dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43632/0 (socket says 172.21.15.86:43632) 2024-04-25T14:09:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.014+0000 7faa29400640 1 -- 172.21.15.86:0/490604696 learned_addr learned my addr 172.21.15.86:0/490604696 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.014+0000 7faa28a00640 1 --2- 172.21.15.86:0/490604696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa240b08c0 0x7faa240d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.014+0000 7faa23e00640 1 --2- 172.21.15.86:0/490604696 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa240b3950 0x7faa2414fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.014+0000 7faa29400640 1 -- 172.21.15.86:0/490604696 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa240b3950 msgr2=0x7faa2414fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.014+0000 7faa29400640 1 --2- 172.21.15.86:0/490604696 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa240b3950 0x7faa2414fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.014+0000 7faa29400640 1 -- 172.21.15.86:0/490604696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa240b08c0 msgr2=0x7faa240d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.014+0000 7faa29400640 1 --2- 172.21.15.86:0/490604696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa240b08c0 0x7faa240d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.014+0000 7faa29400640 1 -- 172.21.15.86:0/490604696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa240d00d0 con 0x7faa2415cc90 2024-04-25T14:09:52.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.014+0000 7faa29400640 1 --2- 172.21.15.86:0/490604696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa2415cc90 0x7faa2415dc30 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7faa1c009d90 tx=0x7faa1c0196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:52.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.014+0000 7faa20c00640 1 -- 172.21.15.86:0/490604696 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c0bd020 con 0x7faa2415cc90 2024-04-25T14:09:52.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.018+0000 7faa20c00640 1 -- 172.21.15.86:0/490604696 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa1c0b8470 con 0x7faa2415cc90 2024-04-25T14:09:52.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.018+0000 7faa2b400640 1 -- 172.21.15.86:0/490604696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa24146f80 con 0x7faa2415cc90 2024-04-25T14:09:52.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.018+0000 7faa20c00640 1 -- 172.21.15.86:0/490604696 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c0b2040 con 0x7faa2415cc90 2024-04-25T14:09:52.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.018+0000 7faa2b400640 1 -- 172.21.15.86:0/490604696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa240d0340 con 0x7faa2415cc90 2024-04-25T14:09:52.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.018+0000 7faa2b400640 1 -- 172.21.15.86:0/490604696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa240a04c0 con 0x7faa2415cc90 2024-04-25T14:09:52.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.018+0000 7faa20c00640 1 -- 172.21.15.86:0/490604696 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 44) ==== 93045+0+0 (secure 0 0 0) 0x7faa1c0c0020 con 0x7faa2415cc90 2024-04-25T14:09:52.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.018+0000 7faa20c00640 1 --2- 172.21.15.86:0/490604696 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7faa0406f250 0x7faa04071760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:52.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.018+0000 7faa20c00640 1 -- 172.21.15.86:0/490604696 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6746+0+0 (secure 0 0 0) 0x7faa1c129fa0 con 0x7faa2415cc90 2024-04-25T14:09:52.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.018+0000 7faa28a00640 1 --2- 172.21.15.86:0/490604696 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7faa0406f250 0x7faa04071760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:52.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.022+0000 7faa28a00640 1 --2- 172.21.15.86:0/490604696 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7faa0406f250 0x7faa04071760 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7faa18007fd0 tx=0x7faa18009040 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:52.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.022+0000 7faa20c00640 1 -- 172.21.15.86:0/490604696 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faa1c0c7030 con 0x7faa2415cc90 2024-04-25T14:09:52.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.178+0000 7faa2b400640 1 -- 172.21.15.86:0/490604696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa24076bf0 con 0x7faa2415cc90 2024-04-25T14:09:52.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.178+0000 7faa20c00640 1 -- 172.21.15.86:0/490604696 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+17810 (secure 0 0 0) 0x7faa24146f80 con 0x7faa2415cc90 2024-04-25T14:09:52.177 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:52.177 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":102,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:09:49.448189+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:09:23.588241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T14:09:24.156880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/3838196444":"2024-04-26T14:09:49.448138+0000","172.21.15.86:6833/436174229":"2024-04-26T14:09:49.448138+0000","172.21.15.86:0/129198987":"2024-04-26T14:09:49.448138+0000","172.21.15.86:0/2887621339":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/2356435633":"2024-04-26T14:09:49.448138+0000","172.21.15.86:6832/436174229":"2024-04-26T14:09:49.448138+0000","172.21.15.86:0/361929172":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/2077676736":"2024-04-26T14:09:49.448138+0000","172.21.15.86:6833/4236874185":"2024-04-26T14:09:43.313864+0000","172.21.15.86:6832/4236874185":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/746145476":"2024-04-26T14:09:49.448138+0000","172.21.15.86:0/789182356":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/1532315242":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/1775030266":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/2226990349":"2024-04-26T14:09:43.313864+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:09:52.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.182+0000 7faa2b400640 1 -- 172.21.15.86:0/490604696 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7faa0406f250 msgr2=0x7faa04071760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:52.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.182+0000 7faa2b400640 1 --2- 172.21.15.86:0/490604696 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7faa0406f250 0x7faa04071760 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7faa18007fd0 tx=0x7faa18009040 comp rx=0 tx=0).stop 2024-04-25T14:09:52.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.182+0000 7faa2b400640 1 -- 172.21.15.86:0/490604696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa2415cc90 msgr2=0x7faa2415dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:52.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.182+0000 7faa2b400640 1 --2- 172.21.15.86:0/490604696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa2415cc90 0x7faa2415dc30 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7faa1c009d90 tx=0x7faa1c0196a0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.182+0000 7faa2b400640 1 -- 172.21.15.86:0/490604696 shutdown_connections 2024-04-25T14:09:52.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.182+0000 7faa2b400640 1 --2- 172.21.15.86:0/490604696 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7faa0406f250 0x7faa04071760 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.182+0000 7faa2b400640 1 --2- 172.21.15.86:0/490604696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa2415cc90 0x7faa2415dc30 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.182+0000 7faa2b400640 1 --2- 172.21.15.86:0/490604696 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa240b3950 0x7faa2414fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.182+0000 7faa2b400640 1 --2- 172.21.15.86:0/490604696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa240b08c0 0x7faa240d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.182+0000 7faa2b400640 1 -- 172.21.15.86:0/490604696 >> 172.21.15.86:0/490604696 conn(0x7faa2409ba30 msgr2=0x7faa2406dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:52.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.182+0000 7faa2b400640 1 -- 172.21.15.86:0/490604696 shutdown_connections 2024-04-25T14:09:52.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.182+0000 7faa2b400640 1 -- 172.21.15.86:0/490604696 wait complete. 2024-04-25T14:09:52.199 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T14:09:52.199 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:09:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.350+0000 7f2d46a00640 1 Processor -- start 2024-04-25T14:09:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.350+0000 7f2d46a00640 1 -- start start 2024-04-25T14:09:52.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.350+0000 7f2d46a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2d400cfca0 0x7f2d4006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:52.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.350+0000 7f2d46a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2d400d16e0 0x7f2d400d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:52.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.350+0000 7f2d46a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d400d2030 con 0x7f2d4006c670 2024-04-25T14:09:52.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.350+0000 7f2d46a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2d40070470 con 0x7f2d400cfca0 2024-04-25T14:09:52.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.350+0000 7f2d46a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f2d40072bd0 con 0x7f2d400d16e0 2024-04-25T14:09:52.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.350+0000 7f2d3fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2d400cfca0 0x7f2d4006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:52.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.350+0000 7f2d3fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2d400cfca0 0x7f2d4006f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43638/0 (socket says 172.21.15.86:43638) 2024-04-25T14:09:52.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.350+0000 7f2d3fe00640 1 -- 172.21.15.86:0/1651072061 learned_addr learned my addr 172.21.15.86:0/1651072061 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:52.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.350+0000 7f2d3f400640 1 --2- 172.21.15.86:0/1651072061 >> v2:172.21.15.155:3300/0 conn(0x7f2d400d16e0 0x7f2d400d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:52.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.350+0000 7f2d3fe00640 1 -- 172.21.15.86:0/1651072061 >> v2:172.21.15.155:3300/0 conn(0x7f2d400d16e0 msgr2=0x7f2d400d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:52.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.350+0000 7f2d3fe00640 1 --2- 172.21.15.86:0/1651072061 >> v2:172.21.15.155:3300/0 conn(0x7f2d400d16e0 0x7f2d400d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.350+0000 7f2d3fe00640 1 -- 172.21.15.86:0/1651072061 >> v1:172.21.15.155:6789/0 conn(0x7f2d4006c670 legacy=0x7f2d4006ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:52.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.350+0000 7f2d3fe00640 1 -- 172.21.15.86:0/1651072061 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d4006f930 con 0x7f2d400cfca0 2024-04-25T14:09:52.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.354+0000 7f2d3fe00640 1 --2- 172.21.15.86:0/1651072061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d400cfca0 0x7f2d4006f3f0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f2d400729c0 tx=0x7f2d2801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f525d870bab63143 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:52.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.354+0000 7f2d3ea00640 1 -- 172.21.15.86:0/1651072061 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d280ab430 con 0x7f2d400cfca0 2024-04-25T14:09:52.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.354+0000 7f2d3ea00640 1 -- 172.21.15.86:0/1651072061 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d280b0070 con 0x7f2d400cfca0 2024-04-25T14:09:52.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.354+0000 7f2d3ea00640 1 -- 172.21.15.86:0/1651072061 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d280ab7f0 con 0x7f2d400cfca0 2024-04-25T14:09:52.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.354+0000 7f2d46a00640 1 -- 172.21.15.86:0/1651072061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d400cfca0 msgr2=0x7f2d4006f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:52.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.354+0000 7f2d46a00640 1 --2- 172.21.15.86:0/1651072061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d400cfca0 0x7f2d4006f3f0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f2d400729c0 tx=0x7f2d2801c520 comp rx=0 tx=0).stop 2024-04-25T14:09:52.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.354+0000 7f2d46a00640 1 -- 172.21.15.86:0/1651072061 shutdown_connections 2024-04-25T14:09:52.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.354+0000 7f2d46a00640 1 --2- 172.21.15.86:0/1651072061 >> v2:172.21.15.155:3300/0 conn(0x7f2d400d16e0 0x7f2d400d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.354+0000 7f2d46a00640 1 --2- 172.21.15.86:0/1651072061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d400cfca0 0x7f2d4006f3f0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.354+0000 7f2d46a00640 1 -- 172.21.15.86:0/1651072061 >> 172.21.15.86:0/1651072061 conn(0x7f2d400b4650 msgr2=0x7f2d400b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:52.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.354+0000 7f2d46a00640 1 -- 172.21.15.86:0/1651072061 shutdown_connections 2024-04-25T14:09:52.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.354+0000 7f2d46a00640 1 -- 172.21.15.86:0/1651072061 wait complete. 2024-04-25T14:09:52.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.358+0000 7f2d46a00640 1 Processor -- start 2024-04-25T14:09:52.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.358+0000 7f2d46a00640 1 -- start start 2024-04-25T14:09:52.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.358+0000 7f2d46a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d4006c670 0x7f2d4009a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:52.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.358+0000 7f2d46a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d400cfca0 0x7f2d40157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:52.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.358+0000 7f2d46a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d400d16e0 0x7f2d40157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:52.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.358+0000 7f2d46a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4016a070 con 0x7f2d400d16e0 2024-04-25T14:09:52.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.358+0000 7f2d46a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2d40169ef0 con 0x7f2d400cfca0 2024-04-25T14:09:52.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.358+0000 7f2d46a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2d4016a1f0 con 0x7f2d4006c670 2024-04-25T14:09:52.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.358+0000 7f2d44a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d400d16e0 0x7f2d40157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:52.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.358+0000 7f2d44a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d400d16e0 0x7f2d40157c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43640/0 (socket says 172.21.15.86:43640) 2024-04-25T14:09:52.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.358+0000 7f2d3fe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d4006c670 0x7f2d4009a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:52.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.358+0000 7f2d44a00640 1 -- 172.21.15.86:0/2331073372 learned_addr learned my addr 172.21.15.86:0/2331073372 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:52.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.358+0000 7f2d3f400640 1 --2- 172.21.15.86:0/2331073372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d400cfca0 0x7f2d40157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:52.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.358+0000 7f2d44a00640 1 -- 172.21.15.86:0/2331073372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d4006c670 msgr2=0x7f2d4009a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:52.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.358+0000 7f2d44a00640 1 --2- 172.21.15.86:0/2331073372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d4006c670 0x7f2d4009a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.358+0000 7f2d44a00640 1 -- 172.21.15.86:0/2331073372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d400cfca0 msgr2=0x7f2d40157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:52.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.358+0000 7f2d44a00640 1 --2- 172.21.15.86:0/2331073372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d400cfca0 0x7f2d40157730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.358+0000 7f2d44a00640 1 -- 172.21.15.86:0/2331073372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d4009ac60 con 0x7f2d400d16e0 2024-04-25T14:09:52.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.358+0000 7f2d44a00640 1 --2- 172.21.15.86:0/2331073372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d400d16e0 0x7f2d40157c70 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800c730 tx=0x7f2d3800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:52.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.358+0000 7f2d37e00640 1 -- 172.21.15.86:0/2331073372 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d38003b80 con 0x7f2d400d16e0 2024-04-25T14:09:52.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.358+0000 7f2d37e00640 1 -- 172.21.15.86:0/2331073372 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d38003d20 con 0x7f2d400d16e0 2024-04-25T14:09:52.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.362+0000 7f2d46a00640 1 -- 172.21.15.86:0/2331073372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d400c97c0 con 0x7f2d400d16e0 2024-04-25T14:09:52.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.362+0000 7f2d37e00640 1 -- 172.21.15.86:0/2331073372 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d38005310 con 0x7f2d400d16e0 2024-04-25T14:09:52.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.362+0000 7f2d46a00640 1 -- 172.21.15.86:0/2331073372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d400c9d80 con 0x7f2d400d16e0 2024-04-25T14:09:52.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.362+0000 7f2d46a00640 1 -- 172.21.15.86:0/2331073372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d4009f4b0 con 0x7f2d400d16e0 2024-04-25T14:09:52.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.362+0000 7f2d37e00640 1 -- 172.21.15.86:0/2331073372 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 44) ==== 93045+0+0 (secure 0 0 0) 0x7f2d380056c0 con 0x7f2d400d16e0 2024-04-25T14:09:52.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.362+0000 7f2d37e00640 1 --2- 172.21.15.86:0/2331073372 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f2d1c06f280 0x7f2d1c071790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:52.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.362+0000 7f2d37e00640 1 -- 172.21.15.86:0/2331073372 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6746+0+0 (secure 0 0 0) 0x7f2d3808f1c0 con 0x7f2d400d16e0 2024-04-25T14:09:52.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.362+0000 7f2d3fe00640 1 --2- 172.21.15.86:0/2331073372 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f2d1c06f280 0x7f2d1c071790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:52.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.362+0000 7f2d3fe00640 1 --2- 172.21.15.86:0/2331073372 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f2d1c06f280 0x7f2d1c071790 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2d280af000 tx=0x7f2d280a7000 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:52.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.366+0000 7f2d37e00640 1 -- 172.21.15.86:0/2331073372 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2d400d2030 con 0x7f2d400d16e0 2024-04-25T14:09:52.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.406+0000 7f2d37e00640 1 -- 172.21.15.86:0/2331073372 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 45) ==== 93253+0+0 (secure 0 0 0) 0x7f2d38057c70 con 0x7f2d400d16e0 2024-04-25T14:09:52.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.562+0000 7f2d46a00640 1 -- 172.21.15.86:0/2331073372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d40069d00 con 0x7f2d400d16e0 2024-04-25T14:09:52.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.562+0000 7f2d37e00640 1 -- 172.21.15.86:0/2331073372 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5189 (secure 0 0 0) 0x7f2d3807c010 con 0x7f2d400d16e0 2024-04-25T14:09:52.561 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:52.561 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":76,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1961914630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1961914630},{"type":"v1","addr":"172.21.15.86:6836","nonce":1961914630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2298829380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2298829380},{"type":"v1","addr":"172.21.15.155:6834","nonce":2298829380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7914,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3761704765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3761704765},{"type":"v1","addr":"172.21.15.86:6837","nonce":3761704765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:09:25.198798+0000","modified":"2024-04-25T14:09:28.237583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6835/1049360040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1049360040},{"type":"v1","addr":"172.21.15.155:6835","nonce":1049360040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7932,"qdb_cluster":[7932]},"id":7}]} 2024-04-25T14:09:52.561 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 76 2024-04-25T14:09:52.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.566+0000 7f2d46a00640 1 -- 172.21.15.86:0/2331073372 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f2d1c06f280 msgr2=0x7f2d1c071790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:52.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.566+0000 7f2d46a00640 1 --2- 172.21.15.86:0/2331073372 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f2d1c06f280 0x7f2d1c071790 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2d280af000 tx=0x7f2d280a7000 comp rx=0 tx=0).stop 2024-04-25T14:09:52.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.566+0000 7f2d46a00640 1 -- 172.21.15.86:0/2331073372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d400d16e0 msgr2=0x7f2d40157c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:52.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.566+0000 7f2d46a00640 1 --2- 172.21.15.86:0/2331073372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d400d16e0 0x7f2d40157c70 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800c730 tx=0x7f2d3800cbf0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.566+0000 7f2d46a00640 1 -- 172.21.15.86:0/2331073372 shutdown_connections 2024-04-25T14:09:52.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.566+0000 7f2d46a00640 1 --2- 172.21.15.86:0/2331073372 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f2d1c06f280 0x7f2d1c071790 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.566+0000 7f2d46a00640 1 --2- 172.21.15.86:0/2331073372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d400d16e0 0x7f2d40157c70 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.566+0000 7f2d46a00640 1 --2- 172.21.15.86:0/2331073372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d400cfca0 0x7f2d40157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.566+0000 7f2d46a00640 1 --2- 172.21.15.86:0/2331073372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d4006c670 0x7f2d4009a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.566+0000 7f2d46a00640 1 -- 172.21.15.86:0/2331073372 >> 172.21.15.86:0/2331073372 conn(0x7f2d400b4650 msgr2=0x7f2d400cbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:52.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.566+0000 7f2d46a00640 1 -- 172.21.15.86:0/2331073372 shutdown_connections 2024-04-25T14:09:52.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.566+0000 7f2d46a00640 1 -- 172.21.15.86:0/2331073372 wait complete. 2024-04-25T14:09:52.583 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:09:52.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.730+0000 7f7720400640 1 Processor -- start 2024-04-25T14:09:52.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.734+0000 7f7720400640 1 -- start start 2024-04-25T14:09:52.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.734+0000 7f7720400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f77180cfca0 0x7f771806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:52.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.734+0000 7f7720400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77180d16e0 0x7f77180d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:52.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.734+0000 7f7720400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f771806c670 0x7f771806ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:52.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.734+0000 7f7720400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7718070470 con 0x7f77180cfca0 2024-04-25T14:09:52.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.734+0000 7f7720400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f77180d2030 con 0x7f771806c670 2024-04-25T14:09:52.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.734+0000 7f7720400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7718072bd0 con 0x7f77180d16e0 2024-04-25T14:09:52.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.734+0000 7f771da00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f77180cfca0 0x7f771806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:52.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.734+0000 7f771da00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f77180cfca0 0x7f771806f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43642/0 (socket says 172.21.15.86:43642) 2024-04-25T14:09:52.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.734+0000 7f771da00640 1 -- 172.21.15.86:0/2653282088 learned_addr learned my addr 172.21.15.86:0/2653282088 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:52.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.734+0000 7f771e400640 1 --2- 172.21.15.86:0/2653282088 >> v2:172.21.15.155:3300/0 conn(0x7f771806c670 0x7f771806ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:52.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.734+0000 7f771d000640 1 --2- 172.21.15.86:0/2653282088 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77180d16e0 0x7f77180d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:52.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.734+0000 7f771da00640 1 -- 172.21.15.86:0/2653282088 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77180d16e0 msgr2=0x7f77180d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:52.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.734+0000 7f771da00640 1 --2- 172.21.15.86:0/2653282088 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77180d16e0 0x7f77180d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.734+0000 7f771da00640 1 -- 172.21.15.86:0/2653282088 >> v2:172.21.15.155:3300/0 conn(0x7f771806c670 msgr2=0x7f771806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.734+0000 7f771da00640 1 --2- 172.21.15.86:0/2653282088 >> v2:172.21.15.155:3300/0 conn(0x7f771806c670 0x7f771806ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.734+0000 7f771da00640 1 -- 172.21.15.86:0/2653282088 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771806f930 con 0x7f77180cfca0 2024-04-25T14:09:52.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.738+0000 7f771da00640 1 --2- 172.21.15.86:0/2653282088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77180cfca0 0x7f771806f3f0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f771806e060 tx=0x7f770c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2763e66b8284b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:52.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.738+0000 7f7717e00640 1 -- 172.21.15.86:0/2653282088 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c0ab5f0 con 0x7f77180cfca0 2024-04-25T14:09:52.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.738+0000 7f7717e00640 1 -- 172.21.15.86:0/2653282088 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f770c0b0070 con 0x7f77180cfca0 2024-04-25T14:09:52.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.738+0000 7f7717e00640 1 -- 172.21.15.86:0/2653282088 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c0ab9b0 con 0x7f77180cfca0 2024-04-25T14:09:52.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.738+0000 7f7720400640 1 -- 172.21.15.86:0/2653282088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77180cfca0 msgr2=0x7f771806f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:52.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.738+0000 7f7720400640 1 --2- 172.21.15.86:0/2653282088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77180cfca0 0x7f771806f3f0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f771806e060 tx=0x7f770c01c520 comp rx=0 tx=0).stop 2024-04-25T14:09:52.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.738+0000 7f7720400640 1 -- 172.21.15.86:0/2653282088 shutdown_connections 2024-04-25T14:09:52.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.738+0000 7f7720400640 1 --2- 172.21.15.86:0/2653282088 >> v2:172.21.15.155:3300/0 conn(0x7f771806c670 0x7f771806ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.738+0000 7f7720400640 1 --2- 172.21.15.86:0/2653282088 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77180d16e0 0x7f77180d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.738+0000 7f7720400640 1 --2- 172.21.15.86:0/2653282088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77180cfca0 0x7f771806f3f0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.738+0000 7f7720400640 1 -- 172.21.15.86:0/2653282088 >> 172.21.15.86:0/2653282088 conn(0x7f77180b4650 msgr2=0x7f77180b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:52.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.738+0000 7f7720400640 1 -- 172.21.15.86:0/2653282088 shutdown_connections 2024-04-25T14:09:52.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.738+0000 7f7720400640 1 -- 172.21.15.86:0/2653282088 wait complete. 2024-04-25T14:09:52.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.742+0000 7f7720400640 1 Processor -- start 2024-04-25T14:09:52.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.742+0000 7f7720400640 1 -- start start 2024-04-25T14:09:52.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.742+0000 7f7720400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f771806c670 0x7f77180ae130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:52.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.742+0000 7f7720400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77180cfca0 0x7f77181545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:52.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.742+0000 7f7720400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77180d16e0 0x7f771815f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:52.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.742+0000 7f7720400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f77180738c0 con 0x7f77180cfca0 2024-04-25T14:09:52.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.742+0000 7f7720400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f771806d3d0 con 0x7f77180d16e0 2024-04-25T14:09:52.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.742+0000 7f7720400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7718073a40 con 0x7f771806c670 2024-04-25T14:09:52.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.742+0000 7f771d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77180cfca0 0x7f77181545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:52.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.742+0000 7f771d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77180cfca0 0x7f77181545e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43656/0 (socket says 172.21.15.86:43656) 2024-04-25T14:09:52.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.742+0000 7f771d000640 1 -- 172.21.15.86:0/3522325757 learned_addr learned my addr 172.21.15.86:0/3522325757 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:52.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.742+0000 7f771da00640 1 --2- 172.21.15.86:0/3522325757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f771806c670 0x7f77180ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:52.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.742+0000 7f771e400640 1 --2- 172.21.15.86:0/3522325757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77180d16e0 0x7f771815f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:52.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.742+0000 7f771d000640 1 -- 172.21.15.86:0/3522325757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f771806c670 msgr2=0x7f77180ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:52.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.742+0000 7f771d000640 1 --2- 172.21.15.86:0/3522325757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f771806c670 0x7f77180ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.742+0000 7f771d000640 1 -- 172.21.15.86:0/3522325757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77180d16e0 msgr2=0x7f771815f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:52.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.742+0000 7f771d000640 1 --2- 172.21.15.86:0/3522325757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77180d16e0 0x7f771815f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.742+0000 7f771d000640 1 -- 172.21.15.86:0/3522325757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77180aeb50 con 0x7f77180cfca0 2024-04-25T14:09:52.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.742+0000 7f771d000640 1 --2- 172.21.15.86:0/3522325757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77180cfca0 0x7f77181545e0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f770400d5f0 tx=0x7f770400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:52.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.742+0000 7f7715600640 1 -- 172.21.15.86:0/3522325757 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77040125f0 con 0x7f77180cfca0 2024-04-25T14:09:52.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.742+0000 7f7715600640 1 -- 172.21.15.86:0/3522325757 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7704019070 con 0x7f77180cfca0 2024-04-25T14:09:52.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.742+0000 7f7720400640 1 -- 172.21.15.86:0/3522325757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77180aee30 con 0x7f77180cfca0 2024-04-25T14:09:52.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.742+0000 7f7715600640 1 -- 172.21.15.86:0/3522325757 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77040129b0 con 0x7f77180cfca0 2024-04-25T14:09:52.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.742+0000 7f7720400640 1 -- 172.21.15.86:0/3522325757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f771815eaf0 con 0x7f77180cfca0 2024-04-25T14:09:52.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.746+0000 7f7720400640 1 -- 172.21.15.86:0/3522325757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f771809f4b0 con 0x7f77180cfca0 2024-04-25T14:09:52.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.746+0000 7f7715600640 1 -- 172.21.15.86:0/3522325757 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 45) ==== 93253+0+0 (secure 0 0 0) 0x7f7704012b50 con 0x7f77180cfca0 2024-04-25T14:09:52.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.746+0000 7f7715600640 1 --2- 172.21.15.86:0/3522325757 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f76f806f4c0 0x7f76f80719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:52.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.746+0000 7f7715600640 1 -- 172.21.15.86:0/3522325757 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6746+0+0 (secure 0 0 0) 0x7f770408fa20 con 0x7f77180cfca0 2024-04-25T14:09:52.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.746+0000 7f771da00640 1 --2- 172.21.15.86:0/3522325757 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f76f806f4c0 0x7f76f80719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:52.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.746+0000 7f771da00640 1 --2- 172.21.15.86:0/3522325757 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f76f806f4c0 0x7f76f80719d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f770c0af000 tx=0x7f770c0a7000 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:52.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.750+0000 7f7715600640 1 -- 172.21.15.86:0/3522325757 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f770405be50 con 0x7f77180cfca0 2024-04-25T14:09:52.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.942+0000 7f7720400640 1 -- 172.21.15.86:0/3522325757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7718069d00 con 0x7f77180cfca0 2024-04-25T14:09:52.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.942+0000 7f7715600640 1 -- 172.21.15.86:0/3522325757 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5189 (secure 0 0 0) 0x7f7704060150 con 0x7f77180cfca0 2024-04-25T14:09:52.940 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:52.940 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":76,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1961914630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1961914630},{"type":"v1","addr":"172.21.15.86:6836","nonce":1961914630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2298829380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2298829380},{"type":"v1","addr":"172.21.15.155:6834","nonce":2298829380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7914,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3761704765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3761704765},{"type":"v1","addr":"172.21.15.86:6837","nonce":3761704765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:09:25.198798+0000","modified":"2024-04-25T14:09:28.237583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6835/1049360040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1049360040},{"type":"v1","addr":"172.21.15.155:6835","nonce":1049360040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7932,"qdb_cluster":[7932]},"id":7}]} 2024-04-25T14:09:52.940 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 76 2024-04-25T14:09:52.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.946+0000 7f7720400640 1 -- 172.21.15.86:0/3522325757 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f76f806f4c0 msgr2=0x7f76f80719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:52.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.946+0000 7f7720400640 1 --2- 172.21.15.86:0/3522325757 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f76f806f4c0 0x7f76f80719d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f770c0af000 tx=0x7f770c0a7000 comp rx=0 tx=0).stop 2024-04-25T14:09:52.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.946+0000 7f7720400640 1 -- 172.21.15.86:0/3522325757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77180cfca0 msgr2=0x7f77181545e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:52.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.946+0000 7f7720400640 1 --2- 172.21.15.86:0/3522325757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77180cfca0 0x7f77181545e0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f770400d5f0 tx=0x7f770400dab0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.946+0000 7f7720400640 1 -- 172.21.15.86:0/3522325757 shutdown_connections 2024-04-25T14:09:52.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.946+0000 7f7720400640 1 --2- 172.21.15.86:0/3522325757 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f76f806f4c0 0x7f76f80719d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.946+0000 7f7720400640 1 --2- 172.21.15.86:0/3522325757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77180d16e0 0x7f771815f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.946+0000 7f7720400640 1 --2- 172.21.15.86:0/3522325757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77180cfca0 0x7f77181545e0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.946+0000 7f7720400640 1 --2- 172.21.15.86:0/3522325757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f771806c670 0x7f77180ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:52.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.946+0000 7f7720400640 1 -- 172.21.15.86:0/3522325757 >> 172.21.15.86:0/3522325757 conn(0x7f77180b4650 msgr2=0x7f771815a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:52.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.946+0000 7f7720400640 1 -- 172.21.15.86:0/3522325757 shutdown_connections 2024-04-25T14:09:52.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:52.946+0000 7f7720400640 1 -- 172.21.15.86:0/3522325757 wait complete. 2024-04-25T14:09:52.962 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:09:53.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.110+0000 7efced800640 1 Processor -- start 2024-04-25T14:09:53.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.110+0000 7efced800640 1 -- start start 2024-04-25T14:09:53.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.114+0000 7efced800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efce80cfca0 0x7efce806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:53.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.114+0000 7efced800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efce8072bd0 con 0x7efce80730b0 2024-04-25T14:09:53.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.114+0000 7efced800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efce806f930 con 0x7efce80d16e0 2024-04-25T14:09:53.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.114+0000 7efced800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efce8070470 con 0x7efce80cfca0 2024-04-25T14:09:53.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.114+0000 7efce7400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7efce80730b0 0x7efce80734c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34658/0 (socket says 172.21.15.86:34658) 2024-04-25T14:09:53.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.114+0000 7efce7400640 1 -- 172.21.15.86:0/3496026843 learned_addr learned my addr 172.21.15.86:0/3496026843 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:53.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.114+0000 7efce6a00640 1 --2- 172.21.15.86:0/3496026843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efce80cfca0 0x7efce806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:53.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.114+0000 7efce6000640 1 -- 172.21.15.86:0/3496026843 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4081330075 0 0) 0x7efce8072bd0 con 0x7efce80730b0 2024-04-25T14:09:53.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.114+0000 7efce6000640 1 -- 172.21.15.86:0/3496026843 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efccc003530 con 0x7efce80730b0 2024-04-25T14:09:53.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.114+0000 7efce6000640 1 -- 172.21.15.86:0/3496026843 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 120481862 0 0) 0x7efce806f930 con 0x7efce80d16e0 2024-04-25T14:09:53.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.114+0000 7efce6000640 1 -- 172.21.15.86:0/3496026843 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efce8072bd0 con 0x7efce80d16e0 2024-04-25T14:09:53.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.114+0000 7efce6000640 1 -- 172.21.15.86:0/3496026843 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1218895699 0 0) 0x7efccc003530 con 0x7efce80730b0 2024-04-25T14:09:53.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.114+0000 7efce6a00640 1 -- 172.21.15.86:0/3496026843 >> v1:172.21.15.155:6789/0 conn(0x7efce80d16e0 legacy=0x7efce80d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:53.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.114+0000 7efce6a00640 1 -- 172.21.15.86:0/3496026843 >> v1:172.21.15.86:6789/0 conn(0x7efce80730b0 legacy=0x7efce80734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:53.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.114+0000 7efce6a00640 1 -- 172.21.15.86:0/3496026843 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efce806fbd0 con 0x7efce80cfca0 2024-04-25T14:09:53.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.118+0000 7efce6a00640 1 --2- 172.21.15.86:0/3496026843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efce80cfca0 0x7efce806f3f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efcd80097f0 tx=0x7efcd801c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19ecd95e31b28d10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:53.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.118+0000 7efce6000640 1 -- 172.21.15.86:0/3496026843 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd80b0070 con 0x7efce80cfca0 2024-04-25T14:09:53.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.118+0000 7efce6000640 1 -- 172.21.15.86:0/3496026843 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcd80ab400 con 0x7efce80cfca0 2024-04-25T14:09:53.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.118+0000 7efce6000640 1 -- 172.21.15.86:0/3496026843 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce80743e0 con 0x7efce80cfca0 2024-04-25T14:09:53.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.118+0000 7efced800640 1 -- 172.21.15.86:0/3496026843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efce80cfca0 msgr2=0x7efce806f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:53.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.118+0000 7efced800640 1 --2- 172.21.15.86:0/3496026843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efce80cfca0 0x7efce806f3f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efcd80097f0 tx=0x7efcd801c400 comp rx=0 tx=0).stop 2024-04-25T14:09:53.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.118+0000 7efced800640 1 -- 172.21.15.86:0/3496026843 shutdown_connections 2024-04-25T14:09:53.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.118+0000 7efced800640 1 --2- 172.21.15.86:0/3496026843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efce80cfca0 0x7efce806f3f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:53.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.118+0000 7efced800640 1 -- 172.21.15.86:0/3496026843 >> 172.21.15.86:0/3496026843 conn(0x7efce80b4650 msgr2=0x7efce80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:53.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.118+0000 7efced800640 1 -- 172.21.15.86:0/3496026843 shutdown_connections 2024-04-25T14:09:53.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.118+0000 7efced800640 1 -- 172.21.15.86:0/3496026843 wait complete. 2024-04-25T14:09:53.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.122+0000 7efced800640 1 Processor -- start 2024-04-25T14:09:53.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.122+0000 7efced800640 1 -- start start 2024-04-25T14:09:53.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.122+0000 7efced800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efce80730b0 0x7efce809a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:53.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.122+0000 7efced800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efce80cfca0 0x7efce815d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:53.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.122+0000 7efced800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efce80d16e0 0x7efce815d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:53.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.122+0000 7efced800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efce80ce0f0 con 0x7efce80d16e0 2024-04-25T14:09:53.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.122+0000 7efced800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efce80cdf70 con 0x7efce80cfca0 2024-04-25T14:09:53.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.122+0000 7efced800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efce80ce270 con 0x7efce80730b0 2024-04-25T14:09:53.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.122+0000 7efce7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efce80d16e0 0x7efce815d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:53.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.122+0000 7efce7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efce80d16e0 0x7efce815d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43662/0 (socket says 172.21.15.86:43662) 2024-04-25T14:09:53.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.122+0000 7efce6a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efce80730b0 0x7efce809a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:53.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.122+0000 7efce7400640 1 -- 172.21.15.86:0/2302937604 learned_addr learned my addr 172.21.15.86:0/2302937604 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:53.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.122+0000 7efcde000640 1 --2- 172.21.15.86:0/2302937604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efce80cfca0 0x7efce815d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:53.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.122+0000 7efce7400640 1 -- 172.21.15.86:0/2302937604 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efce80730b0 msgr2=0x7efce809a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:53.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.122+0000 7efce7400640 1 --2- 172.21.15.86:0/2302937604 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efce80730b0 0x7efce809a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:53.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.122+0000 7efce7400640 1 -- 172.21.15.86:0/2302937604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efce80cfca0 msgr2=0x7efce815d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:53.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.122+0000 7efce7400640 1 --2- 172.21.15.86:0/2302937604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efce80cfca0 0x7efce815d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:53.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.122+0000 7efce7400640 1 -- 172.21.15.86:0/2302937604 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efce815de60 con 0x7efce80d16e0 2024-04-25T14:09:53.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.122+0000 7efce7400640 1 --2- 172.21.15.86:0/2302937604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efce80d16e0 0x7efce815d6d0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7efce0002010 tx=0x7efce000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:53.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.122+0000 7efcdf400640 1 -- 172.21.15.86:0/2302937604 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce0012400 con 0x7efce80d16e0 2024-04-25T14:09:53.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.122+0000 7efcdf400640 1 -- 172.21.15.86:0/2302937604 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efce00125a0 con 0x7efce80d16e0 2024-04-25T14:09:53.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.122+0000 7efcdf400640 1 -- 172.21.15.86:0/2302937604 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce0012990 con 0x7efce80d16e0 2024-04-25T14:09:53.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.122+0000 7efced800640 1 -- 172.21.15.86:0/2302937604 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efce8156360 con 0x7efce80d16e0 2024-04-25T14:09:53.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.122+0000 7efced800640 1 -- 172.21.15.86:0/2302937604 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efce81f62d0 con 0x7efce80d16e0 2024-04-25T14:09:53.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.126+0000 7efced800640 1 -- 172.21.15.86:0/2302937604 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efce81f6530 con 0x7efce80d16e0 2024-04-25T14:09:53.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.126+0000 7efcdf400640 1 -- 172.21.15.86:0/2302937604 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 45) ==== 93253+0+0 (secure 0 0 0) 0x7efce0012b30 con 0x7efce80d16e0 2024-04-25T14:09:53.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.126+0000 7efcdf400640 1 --2- 172.21.15.86:0/2302937604 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7efcc406f490 0x7efcc40719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:53.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.126+0000 7efcdf400640 1 -- 172.21.15.86:0/2302937604 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6746+0+0 (secure 0 0 0) 0x7efce008ff30 con 0x7efce80d16e0 2024-04-25T14:09:53.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.126+0000 7efce6a00640 1 --2- 172.21.15.86:0/2302937604 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7efcc406f490 0x7efcc40719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:53.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.126+0000 7efce6a00640 1 --2- 172.21.15.86:0/2302937604 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7efcc406f490 0x7efcc40719a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efcd80ad000 tx=0x7efcd8008040 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:53.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.130+0000 7efcdf400640 1 -- 172.21.15.86:0/2302937604 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efce0012e20 con 0x7efce80d16e0 2024-04-25T14:09:53.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.286+0000 7efced800640 1 -- 172.21.15.86:0/2302937604 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efce809f4b0 con 0x7efce80d16e0 2024-04-25T14:09:53.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.290+0000 7efcdf400640 1 -- 172.21.15.86:0/2302937604 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+17810 (secure 0 0 0) 0x7efce005c360 con 0x7efce80d16e0 2024-04-25T14:09:53.286 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:53.287 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":102,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:09:49.448189+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:09:23.588241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T14:09:24.156880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/3838196444":"2024-04-26T14:09:49.448138+0000","172.21.15.86:6833/436174229":"2024-04-26T14:09:49.448138+0000","172.21.15.86:0/129198987":"2024-04-26T14:09:49.448138+0000","172.21.15.86:0/2887621339":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/2356435633":"2024-04-26T14:09:49.448138+0000","172.21.15.86:6832/436174229":"2024-04-26T14:09:49.448138+0000","172.21.15.86:0/361929172":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/2077676736":"2024-04-26T14:09:49.448138+0000","172.21.15.86:6833/4236874185":"2024-04-26T14:09:43.313864+0000","172.21.15.86:6832/4236874185":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/746145476":"2024-04-26T14:09:49.448138+0000","172.21.15.86:0/789182356":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/1532315242":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/1775030266":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/2226990349":"2024-04-26T14:09:43.313864+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:09:53.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.290+0000 7efced800640 1 -- 172.21.15.86:0/2302937604 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7efcc406f490 msgr2=0x7efcc40719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:53.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.290+0000 7efced800640 1 --2- 172.21.15.86:0/2302937604 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7efcc406f490 0x7efcc40719a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efcd80ad000 tx=0x7efcd8008040 comp rx=0 tx=0).stop 2024-04-25T14:09:53.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.290+0000 7efced800640 1 -- 172.21.15.86:0/2302937604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efce80d16e0 msgr2=0x7efce815d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:53.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.290+0000 7efced800640 1 --2- 172.21.15.86:0/2302937604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efce80d16e0 0x7efce815d6d0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7efce0002010 tx=0x7efce000a430 comp rx=0 tx=0).stop 2024-04-25T14:09:53.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.290+0000 7efced800640 1 -- 172.21.15.86:0/2302937604 shutdown_connections 2024-04-25T14:09:53.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.290+0000 7efced800640 1 --2- 172.21.15.86:0/2302937604 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7efcc406f490 0x7efcc40719a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:53.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.290+0000 7efced800640 1 --2- 172.21.15.86:0/2302937604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efce80d16e0 0x7efce815d6d0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:53.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.290+0000 7efced800640 1 --2- 172.21.15.86:0/2302937604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efce80cfca0 0x7efce815d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:53.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.290+0000 7efced800640 1 --2- 172.21.15.86:0/2302937604 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efce80730b0 0x7efce809a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:53.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.290+0000 7efced800640 1 -- 172.21.15.86:0/2302937604 >> 172.21.15.86:0/2302937604 conn(0x7efce80b4650 msgr2=0x7efce80cbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:53.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.290+0000 7efced800640 1 -- 172.21.15.86:0/2302937604 shutdown_connections 2024-04-25T14:09:53.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.290+0000 7efced800640 1 -- 172.21.15.86:0/2302937604 wait complete. 2024-04-25T14:09:53.308 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T14:09:53.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.458+0000 7f7842800640 1 Processor -- start 2024-04-25T14:09:53.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.458+0000 7f7842800640 1 -- start start 2024-04-25T14:09:53.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f7842800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f783c0cfca0 0x7f783c06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:53.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f7842800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f783c0730b0 0x7f783c0734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:53.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f7842800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f783c072bd0 con 0x7f783c0d16e0 2024-04-25T14:09:53.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f7842800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f783c070470 con 0x7f783c0730b0 2024-04-25T14:09:53.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f7842800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f783c06f930 con 0x7f783c0cfca0 2024-04-25T14:09:53.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f7832a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f783c0d16e0 0x7f783c0d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34662/0 (socket says 172.21.15.86:34662) 2024-04-25T14:09:53.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f7832a00640 1 -- 172.21.15.86:0/1044793408 learned_addr learned my addr 172.21.15.86:0/1044793408 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:53.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f783b400640 1 --2- 172.21.15.86:0/1044793408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f783c0cfca0 0x7f783c06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:53.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f783be00640 1 --2- 172.21.15.86:0/1044793408 >> v2:172.21.15.155:3300/0 conn(0x7f783c0730b0 0x7f783c0734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:53.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f783aa00640 1 -- 172.21.15.86:0/1044793408 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3652235479 0 0) 0x7f783c072bd0 con 0x7f783c0d16e0 2024-04-25T14:09:53.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f783aa00640 1 -- 172.21.15.86:0/1044793408 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7820003530 con 0x7f783c0d16e0 2024-04-25T14:09:53.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f783aa00640 1 -- 172.21.15.86:0/1044793408 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 851969278 0 0) 0x7f7820003530 con 0x7f783c0d16e0 2024-04-25T14:09:53.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f783aa00640 1 -- 172.21.15.86:0/1044793408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f783c0cfca0 msgr2=0x7f783c06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:53.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f783aa00640 1 --2- 172.21.15.86:0/1044793408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f783c0cfca0 0x7f783c06f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:53.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f783aa00640 1 -- 172.21.15.86:0/1044793408 >> v2:172.21.15.155:3300/0 conn(0x7f783c0730b0 msgr2=0x7f783c0734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:53.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f783aa00640 1 --2- 172.21.15.86:0/1044793408 >> v2:172.21.15.155:3300/0 conn(0x7f783c0730b0 0x7f783c0734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:53.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f783aa00640 1 -- 172.21.15.86:0/1044793408 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783c06fbd0 con 0x7f783c0d16e0 2024-04-25T14:09:53.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f783aa00640 1 -- 172.21.15.86:0/1044793408 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f7824001680 con 0x7f783c0d16e0 2024-04-25T14:09:53.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f783aa00640 1 -- 172.21.15.86:0/1044793408 >> v1:172.21.15.86:6789/0 conn(0x7f783c0d16e0 legacy=0x7f783c0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:53.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f783aa00640 1 --2- 172.21.15.86:0/1044793408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f783c0cfca0 0x7f7820003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:53.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f783aa00640 1 --2- 172.21.15.86:0/1044793408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7820003fb0 0x7f7820004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:53.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f783aa00640 1 --2- 172.21.15.86:0/1044793408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7820004d50 0x7f7820005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:53.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f783aa00640 1 -- 172.21.15.86:0/1044793408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f783c070470 con 0x7f7820003fb0 2024-04-25T14:09:53.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f783aa00640 1 -- 172.21.15.86:0/1044793408 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f783c06f930 con 0x7f7820004d50 2024-04-25T14:09:53.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f783aa00640 1 -- 172.21.15.86:0/1044793408 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f782000bab0 con 0x7f783c0cfca0 2024-04-25T14:09:53.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f7832a00640 1 --2- 172.21.15.86:0/1044793408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7820003fb0 0x7f7820004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:53.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f783b400640 1 --2- 172.21.15.86:0/1044793408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f783c0cfca0 0x7f7820003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:53.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f783be00640 1 --2- 172.21.15.86:0/1044793408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7820004d50 0x7f7820005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:53.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f7832a00640 1 -- 172.21.15.86:0/1044793408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f783c0cfca0 msgr2=0x7f7820003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:53.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f7832a00640 1 --2- 172.21.15.86:0/1044793408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f783c0cfca0 0x7f7820003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:53.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f7832a00640 1 -- 172.21.15.86:0/1044793408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7820004d50 msgr2=0x7f7820005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:53.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f7832a00640 1 --2- 172.21.15.86:0/1044793408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7820004d50 0x7f7820005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:53.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.462+0000 7f7832a00640 1 -- 172.21.15.86:0/1044793408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7820003530 con 0x7f7820003fb0 2024-04-25T14:09:53.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.466+0000 7f7832a00640 1 --2- 172.21.15.86:0/1044793408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7820003fb0 0x7f7820004810 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f782400c2c0 tx=0x7f782401f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3fef702c053b75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:53.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.466+0000 7f783aa00640 1 -- 172.21.15.86:0/1044793408 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7824007d60 con 0x7f7820003fb0 2024-04-25T14:09:53.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.466+0000 7f783aa00640 1 -- 172.21.15.86:0/1044793408 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7824005b00 con 0x7f7820003fb0 2024-04-25T14:09:53.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.466+0000 7f783aa00640 1 -- 172.21.15.86:0/1044793408 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7824007070 con 0x7f7820003fb0 2024-04-25T14:09:53.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.466+0000 7f7842800640 1 -- 172.21.15.86:0/1044793408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7820003fb0 msgr2=0x7f7820004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:53.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.466+0000 7f7842800640 1 --2- 172.21.15.86:0/1044793408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7820003fb0 0x7f7820004810 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f782400c2c0 tx=0x7f782401f4b0 comp rx=0 tx=0).stop 2024-04-25T14:09:53.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.466+0000 7f783be00640 1 -- 172.21.15.86:0/1044793408 reap_dead start 2024-04-25T14:09:53.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.466+0000 7f7842800640 1 -- 172.21.15.86:0/1044793408 shutdown_connections 2024-04-25T14:09:53.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.466+0000 7f7842800640 1 -- 172.21.15.86:0/1044793408 >> 172.21.15.86:0/1044793408 conn(0x7f783c0b4650 msgr2=0x7f783c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:53.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.466+0000 7f7842800640 1 -- 172.21.15.86:0/1044793408 shutdown_connections 2024-04-25T14:09:53.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.466+0000 7f7842800640 1 -- 172.21.15.86:0/1044793408 wait complete. 2024-04-25T14:09:53.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.470+0000 7f7842800640 1 Processor -- start 2024-04-25T14:09:53.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.470+0000 7f7842800640 1 -- start start 2024-04-25T14:09:53.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.470+0000 7f7842800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7820003fb0 0x7f783c1549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:53.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.470+0000 7f7842800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f783c0ca1c0 0x7f783c1f6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:53.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.470+0000 7f7842800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f783c1f6a40 0x7f783c151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:53.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.470+0000 7f7842800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f783c06f3f0 con 0x7f783c0ca1c0 2024-04-25T14:09:53.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.470+0000 7f7842800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f783c070190 con 0x7f7820003fb0 2024-04-25T14:09:53.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.470+0000 7f7842800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f783c06f570 con 0x7f783c1f6a40 2024-04-25T14:09:53.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.470+0000 7f7832a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f783c0ca1c0 0x7f783c1f6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:53.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.470+0000 7f7832a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f783c0ca1c0 0x7f783c1f6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43682/0 (socket says 172.21.15.86:43682) 2024-04-25T14:09:53.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.470+0000 7f7832a00640 1 -- 172.21.15.86:0/684903429 learned_addr learned my addr 172.21.15.86:0/684903429 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:53.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.470+0000 7f783b400640 1 --2- 172.21.15.86:0/684903429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7820003fb0 0x7f783c1549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:53.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.470+0000 7f783be00640 1 --2- 172.21.15.86:0/684903429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f783c1f6a40 0x7f783c151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:53.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.470+0000 7f7832a00640 1 -- 172.21.15.86:0/684903429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f783c1f6a40 msgr2=0x7f783c151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:53.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.470+0000 7f7832a00640 1 --2- 172.21.15.86:0/684903429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f783c1f6a40 0x7f783c151a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:53.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.470+0000 7f7832a00640 1 -- 172.21.15.86:0/684903429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7820003fb0 msgr2=0x7f783c1549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:53.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.470+0000 7f7832a00640 1 --2- 172.21.15.86:0/684903429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7820003fb0 0x7f783c1549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:53.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.470+0000 7f7832a00640 1 -- 172.21.15.86:0/684903429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783c0ca8a0 con 0x7f783c0ca1c0 2024-04-25T14:09:53.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.470+0000 7f7832a00640 1 --2- 172.21.15.86:0/684903429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f783c0ca1c0 0x7f783c1f6500 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f782401f4f0 tx=0x7f7820005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:53.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.474+0000 7f7833e00640 1 -- 172.21.15.86:0/684903429 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7824001c90 con 0x7f783c0ca1c0 2024-04-25T14:09:53.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.474+0000 7f7833e00640 1 -- 172.21.15.86:0/684903429 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7824005cb0 con 0x7f783c0ca1c0 2024-04-25T14:09:53.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.474+0000 7f7833e00640 1 -- 172.21.15.86:0/684903429 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7824005e60 con 0x7f783c0ca1c0 2024-04-25T14:09:53.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.474+0000 7f7842800640 1 -- 172.21.15.86:0/684903429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f783c0cab80 con 0x7f783c0ca1c0 2024-04-25T14:09:53.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.474+0000 7f7842800640 1 -- 172.21.15.86:0/684903429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f783c0cb140 con 0x7f783c0ca1c0 2024-04-25T14:09:53.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.474+0000 7f7842800640 1 -- 172.21.15.86:0/684903429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f783c09f4b0 con 0x7f783c0ca1c0 2024-04-25T14:09:53.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.474+0000 7f7833e00640 1 -- 172.21.15.86:0/684903429 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 45) ==== 93253+0+0 (secure 0 0 0) 0x7f78240049f0 con 0x7f783c0ca1c0 2024-04-25T14:09:53.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.474+0000 7f7833e00640 1 --2- 172.21.15.86:0/684903429 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f781806f430 0x7f7818071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:53.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.478+0000 7f7833e00640 1 -- 172.21.15.86:0/684903429 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6746+0+0 (secure 0 0 0) 0x7f782412c420 con 0x7f783c0ca1c0 2024-04-25T14:09:53.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.478+0000 7f783b400640 1 --2- 172.21.15.86:0/684903429 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f781806f430 0x7f7818071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:53.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.478+0000 7f783b400640 1 --2- 172.21.15.86:0/684903429 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f781806f430 0x7f7818071940 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f783c070710 tx=0x7f782c009190 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:53.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.482+0000 7f7833e00640 1 -- 172.21.15.86:0/684903429 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f78240191c0 con 0x7f783c0ca1c0 2024-04-25T14:09:53.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.670+0000 7f7842800640 1 -- 172.21.15.86:0/684903429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f783c069d00 con 0x7f783c0ca1c0 2024-04-25T14:09:53.668 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-25T14:09:53.670+0000 7fea29c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:09:53.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.674+0000 7f7833e00640 1 -- 172.21.15.86:0/684903429 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v77) ==== 161+0+0 (secure 0 0 0) 0x7f78240f8840 con 0x7f783c0ca1c0 2024-04-25T14:09:53.671 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T14:09:53.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.674+0000 7f7842800640 1 -- 172.21.15.86:0/684903429 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f781806f430 msgr2=0x7f7818071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:53.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.674+0000 7f7842800640 1 --2- 172.21.15.86:0/684903429 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f781806f430 0x7f7818071940 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f783c070710 tx=0x7f782c009190 comp rx=0 tx=0).stop 2024-04-25T14:09:53.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.674+0000 7f7842800640 1 -- 172.21.15.86:0/684903429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f783c0ca1c0 msgr2=0x7f783c1f6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:53.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.674+0000 7f7842800640 1 --2- 172.21.15.86:0/684903429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f783c0ca1c0 0x7f783c1f6500 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f782401f4f0 tx=0x7f7820005840 comp rx=0 tx=0).stop 2024-04-25T14:09:53.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.678+0000 7f7842800640 1 -- 172.21.15.86:0/684903429 shutdown_connections 2024-04-25T14:09:53.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.678+0000 7f7842800640 1 --2- 172.21.15.86:0/684903429 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f781806f430 0x7f7818071940 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:53.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.678+0000 7f7842800640 1 --2- 172.21.15.86:0/684903429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f783c1f6a40 0x7f783c151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:53.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.678+0000 7f7842800640 1 --2- 172.21.15.86:0/684903429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f783c0ca1c0 0x7f783c1f6500 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:53.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.678+0000 7f7842800640 1 --2- 172.21.15.86:0/684903429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7820003fb0 0x7f783c1549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:53.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.678+0000 7f7842800640 1 -- 172.21.15.86:0/684903429 >> 172.21.15.86:0/684903429 conn(0x7f783c0b4650 msgr2=0x7f783c15c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:53.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.678+0000 7f7842800640 1 -- 172.21.15.86:0/684903429 shutdown_connections 2024-04-25T14:09:53.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.678+0000 7f7842800640 1 -- 172.21.15.86:0/684903429 wait complete. 2024-04-25T14:09:53.682 INFO:tasks.ceph.mds.b.smithi155.stderr: -3549> 2024-04-25T14:09:27.779+0000 7f2d59600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T14:09:53.694 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T14:09:53.802 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2024-04-25T14:09:53.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.846+0000 7f690b200640 1 Processor -- start 2024-04-25T14:09:53.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.846+0000 7f690b200640 1 -- start start 2024-04-25T14:09:53.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.846+0000 7f690b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69040cde90 0x7f69040ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:53.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.846+0000 7f690b200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f690415e5c0 con 0x7f69040d1df0 2024-04-25T14:09:53.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.846+0000 7f690b200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f690415e8a0 con 0x7f6904070330 2024-04-25T14:09:53.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.846+0000 7f690b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f690406fd40 con 0x7f69040cde90 2024-04-25T14:09:53.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.850+0000 7f68fb400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f69040d1df0 0x7f69040cd780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34670/0 (socket says 172.21.15.86:34670) 2024-04-25T14:09:53.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.850+0000 7f68fb400640 1 -- 172.21.15.86:0/3473027687 learned_addr learned my addr 172.21.15.86:0/3473027687 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:53.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.850+0000 7f6909200640 1 --2- 172.21.15.86:0/3473027687 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69040cde90 0x7f69040ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:53.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.850+0000 7f6903400640 1 -- 172.21.15.86:0/3473027687 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4288273467 0 0) 0x7f690415e8a0 con 0x7f6904070330 2024-04-25T14:09:53.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.850+0000 7f6903400640 1 -- 172.21.15.86:0/3473027687 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f68e8003530 con 0x7f6904070330 2024-04-25T14:09:53.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.850+0000 7f6903400640 1 -- 172.21.15.86:0/3473027687 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4165134330 0 0) 0x7f690415e5c0 con 0x7f69040d1df0 2024-04-25T14:09:53.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.850+0000 7f6909200640 1 -- 172.21.15.86:0/3473027687 >> v1:172.21.15.155:6789/0 conn(0x7f6904070330 legacy=0x7f69040d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:53.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.850+0000 7f6909200640 1 -- 172.21.15.86:0/3473027687 >> v1:172.21.15.86:6789/0 conn(0x7f69040d1df0 legacy=0x7f69040cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:53.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.850+0000 7f6909200640 1 -- 172.21.15.86:0/3473027687 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f690415eb80 con 0x7f69040cde90 2024-04-25T14:09:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.850+0000 7f6909200640 1 --2- 172.21.15.86:0/3473027687 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69040cde90 0x7f69040ce2a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f68fc009dc0 tx=0x7f68fc01c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d11cb12320f3841a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.850+0000 7f6903400640 1 -- 172.21.15.86:0/3473027687 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc0b2070 con 0x7f69040cde90 2024-04-25T14:09:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.854+0000 7f6903400640 1 -- 172.21.15.86:0/3473027687 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68fc0a58a0 con 0x7f69040cde90 2024-04-25T14:09:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.854+0000 7f6903400640 1 -- 172.21.15.86:0/3473027687 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc0a5c70 con 0x7f69040cde90 2024-04-25T14:09:53.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.854+0000 7f690b200640 1 -- 172.21.15.86:0/3473027687 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69040cde90 msgr2=0x7f69040ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:53.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.854+0000 7f690b200640 1 --2- 172.21.15.86:0/3473027687 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69040cde90 0x7f69040ce2a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f68fc009dc0 tx=0x7f68fc01c8d0 comp rx=0 tx=0).stop 2024-04-25T14:09:53.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.854+0000 7f690b200640 1 -- 172.21.15.86:0/3473027687 shutdown_connections 2024-04-25T14:09:53.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.854+0000 7f690b200640 1 --2- 172.21.15.86:0/3473027687 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69040cde90 0x7f69040ce2a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:53.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.854+0000 7f690b200640 1 -- 172.21.15.86:0/3473027687 >> 172.21.15.86:0/3473027687 conn(0x7f69040b4650 msgr2=0x7f69040b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:53.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.854+0000 7f690b200640 1 -- 172.21.15.86:0/3473027687 shutdown_connections 2024-04-25T14:09:53.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.854+0000 7f690b200640 1 -- 172.21.15.86:0/3473027687 wait complete. 2024-04-25T14:09:53.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.854+0000 7f690b200640 1 Processor -- start 2024-04-25T14:09:53.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.854+0000 7f690b200640 1 -- start start 2024-04-25T14:09:53.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.858+0000 7f690b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6904070330 0x7f69040b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:53.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.858+0000 7f690b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69040cde90 0x7f6904154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:53.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.858+0000 7f690b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f690409ab70 0x7f69041f80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:53.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.858+0000 7f690b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f69040708c0 con 0x7f690409ab70 2024-04-25T14:09:53.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.858+0000 7f690b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6904070740 con 0x7f69040cde90 2024-04-25T14:09:53.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.858+0000 7f690b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f690416bfd0 con 0x7f6904070330 2024-04-25T14:09:53.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.858+0000 7f6909200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f690409ab70 0x7f69041f80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:53.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.858+0000 7f6909200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f690409ab70 0x7f69041f80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43696/0 (socket says 172.21.15.86:43696) 2024-04-25T14:09:53.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.858+0000 7f6909200640 1 -- 172.21.15.86:0/2388545704 learned_addr learned my addr 172.21.15.86:0/2388545704 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:53.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.858+0000 7f6903e00640 1 --2- 172.21.15.86:0/2388545704 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6904070330 0x7f69040b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:53.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.858+0000 7f68fb400640 1 --2- 172.21.15.86:0/2388545704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69040cde90 0x7f6904154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:53.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.858+0000 7f6909200640 1 -- 172.21.15.86:0/2388545704 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6904070330 msgr2=0x7f69040b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:53.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.858+0000 7f6909200640 1 --2- 172.21.15.86:0/2388545704 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6904070330 0x7f69040b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:53.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.858+0000 7f6909200640 1 -- 172.21.15.86:0/2388545704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69040cde90 msgr2=0x7f6904154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:53.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.858+0000 7f6909200640 1 --2- 172.21.15.86:0/2388545704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69040cde90 0x7f6904154760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:53.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.858+0000 7f6909200640 1 -- 172.21.15.86:0/2388545704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69040b2180 con 0x7f690409ab70 2024-04-25T14:09:53.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.858+0000 7f6909200640 1 --2- 172.21.15.86:0/2388545704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f690409ab70 0x7f69041f80e0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f68fc01cdb0 tx=0x7f68fc019670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:53.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.858+0000 7f6900c00640 1 -- 172.21.15.86:0/2388545704 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc0bd020 con 0x7f690409ab70 2024-04-25T14:09:53.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.858+0000 7f6900c00640 1 -- 172.21.15.86:0/2388545704 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68fc0b8670 con 0x7f690409ab70 2024-04-25T14:09:53.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.858+0000 7f690b200640 1 -- 172.21.15.86:0/2388545704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69040b2460 con 0x7f690409ab70 2024-04-25T14:09:53.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.858+0000 7f6900c00640 1 -- 172.21.15.86:0/2388545704 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc0b2040 con 0x7f690409ab70 2024-04-25T14:09:53.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.858+0000 7f690b200640 1 -- 172.21.15.86:0/2388545704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69041f7bc0 con 0x7f690409ab70 2024-04-25T14:09:53.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.858+0000 7f690b200640 1 -- 172.21.15.86:0/2388545704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6904069d00 con 0x7f690409ab70 2024-04-25T14:09:53.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.862+0000 7f6900c00640 1 -- 172.21.15.86:0/2388545704 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 45) ==== 93253+0+0 (secure 0 0 0) 0x7f68fc0c0020 con 0x7f690409ab70 2024-04-25T14:09:53.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.862+0000 7f6900c00640 1 --2- 172.21.15.86:0/2388545704 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f68e006f4c0 0x7f68e00719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:53.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.862+0000 7f6900c00640 1 -- 172.21.15.86:0/2388545704 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6832+0+0 (secure 0 0 0) 0x7f68fc12a500 con 0x7f690409ab70 2024-04-25T14:09:53.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.862+0000 7f6903e00640 1 --2- 172.21.15.86:0/2388545704 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f68e006f4c0 0x7f68e00719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:53.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.862+0000 7f6903e00640 1 --2- 172.21.15.86:0/2388545704 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f68e006f4c0 0x7f68e00719d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f68f4007fd0 tx=0x7f68f4009040 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:53.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:53.866+0000 7f6900c00640 1 -- 172.21.15.86:0/2388545704 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f68fc0c7030 con 0x7f690409ab70 2024-04-25T14:09:54.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.054+0000 7f690b200640 1 -- 172.21.15.86:0/2388545704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f69040b1ed0 con 0x7f690409ab70 2024-04-25T14:09:54.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.414+0000 7f6900c00640 1 -- 172.21.15.86:0/2388545704 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7f68fc0a5760 con 0x7f690409ab70 2024-04-25T14:09:54.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.674+0000 7f6900c00640 1 -- 172.21.15.86:0/2388545704 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v78) ==== 104+0+0 (secure 0 0 0) 0x7f68fc016580 con 0x7f690409ab70 2024-04-25T14:09:54.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.678+0000 7f690b200640 1 -- 172.21.15.86:0/2388545704 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f68e006f4c0 msgr2=0x7f68e00719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:54.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.678+0000 7f690b200640 1 --2- 172.21.15.86:0/2388545704 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f68e006f4c0 0x7f68e00719d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f68f4007fd0 tx=0x7f68f4009040 comp rx=0 tx=0).stop 2024-04-25T14:09:54.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.678+0000 7f690b200640 1 -- 172.21.15.86:0/2388545704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f690409ab70 msgr2=0x7f69041f80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:54.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.678+0000 7f690b200640 1 --2- 172.21.15.86:0/2388545704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f690409ab70 0x7f69041f80e0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f68fc01cdb0 tx=0x7f68fc019670 comp rx=0 tx=0).stop 2024-04-25T14:09:54.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.678+0000 7f690b200640 1 -- 172.21.15.86:0/2388545704 shutdown_connections 2024-04-25T14:09:54.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.678+0000 7f690b200640 1 --2- 172.21.15.86:0/2388545704 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f68e006f4c0 0x7f68e00719d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:54.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.678+0000 7f690b200640 1 --2- 172.21.15.86:0/2388545704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f690409ab70 0x7f69041f80e0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:54.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.678+0000 7f690b200640 1 --2- 172.21.15.86:0/2388545704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69040cde90 0x7f6904154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:54.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.678+0000 7f690b200640 1 --2- 172.21.15.86:0/2388545704 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6904070330 0x7f69040b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:54.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.678+0000 7f690b200640 1 -- 172.21.15.86:0/2388545704 >> 172.21.15.86:0/2388545704 conn(0x7f69040b4650 msgr2=0x7f69040aeeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:54.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.678+0000 7f690b200640 1 -- 172.21.15.86:0/2388545704 shutdown_connections 2024-04-25T14:09:54.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.678+0000 7f690b200640 1 -- 172.21.15.86:0/2388545704 wait complete. 2024-04-25T14:09:54.698 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T14:09:54.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.854+0000 7f023d600640 1 Processor -- start 2024-04-25T14:09:54.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.854+0000 7f023d600640 1 -- start start 2024-04-25T14:09:54.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.854+0000 7f023d600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02380b0540 0x7f02380ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:54.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.854+0000 7f023d600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f023815e270 0x7f023815e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:54.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.854+0000 7f023d600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02380b0b20 con 0x7f023815ebc0 2024-04-25T14:09:54.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.854+0000 7f023d600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f023806d870 con 0x7f023815e270 2024-04-25T14:09:54.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.854+0000 7f023d600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f023806d220 con 0x7f02380b0540 2024-04-25T14:09:54.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.854+0000 7f0236000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f023815e270 0x7f023815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:54.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.854+0000 7f0236a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02380b0540 0x7f02380ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:54.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.854+0000 7f0236000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f023815e270 0x7f023815e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:35300/0 (socket says 172.21.15.86:35300) 2024-04-25T14:09:54.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.854+0000 7f0236000640 1 -- 172.21.15.86:0/3520041713 learned_addr learned my addr 172.21.15.86:0/3520041713 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:54.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.854+0000 7f0236000640 1 -- 172.21.15.86:0/3520041713 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02380b0540 msgr2=0x7f02380ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:54.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.854+0000 7f0236000640 1 --2- 172.21.15.86:0/3520041713 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02380b0540 0x7f02380ae2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:54.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.854+0000 7f0236000640 1 -- 172.21.15.86:0/3520041713 >> v1:172.21.15.155:6789/0 conn(0x7f023815ebc0 legacy=0x7f02380733c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:54.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.854+0000 7f0236000640 1 -- 172.21.15.86:0/3520041713 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f023806d500 con 0x7f023815e270 2024-04-25T14:09:54.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.854+0000 7f0236a00640 1 --2- 172.21.15.86:0/3520041713 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02380b0540 0x7f02380ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:09:54.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.858+0000 7f0236000640 1 --2- 172.21.15.86:0/3520041713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f023815e270 0x7f023815e680 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0224009870 tx=0x7f022401c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cba36ce7fdfb946f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:54.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.858+0000 7f0235600640 1 -- 172.21.15.86:0/3520041713 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02240b0070 con 0x7f023815e270 2024-04-25T14:09:54.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.858+0000 7f0235600640 1 -- 172.21.15.86:0/3520041713 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02240ab600 con 0x7f023815e270 2024-04-25T14:09:54.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.858+0000 7f0235600640 1 -- 172.21.15.86:0/3520041713 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02240a4e60 con 0x7f023815e270 2024-04-25T14:09:54.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.858+0000 7f023d600640 1 -- 172.21.15.86:0/3520041713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f023815e270 msgr2=0x7f023815e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:54.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.858+0000 7f023d600640 1 --2- 172.21.15.86:0/3520041713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f023815e270 0x7f023815e680 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0224009870 tx=0x7f022401c4d0 comp rx=0 tx=0).stop 2024-04-25T14:09:54.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.858+0000 7f023d600640 1 -- 172.21.15.86:0/3520041713 shutdown_connections 2024-04-25T14:09:54.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.858+0000 7f023d600640 1 --2- 172.21.15.86:0/3520041713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f023815e270 0x7f023815e680 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:54.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.858+0000 7f023d600640 1 --2- 172.21.15.86:0/3520041713 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02380b0540 0x7f02380ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:54.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.858+0000 7f023d600640 1 -- 172.21.15.86:0/3520041713 >> 172.21.15.86:0/3520041713 conn(0x7f02380b4a40 msgr2=0x7f02380b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:54.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.858+0000 7f023d600640 1 -- 172.21.15.86:0/3520041713 shutdown_connections 2024-04-25T14:09:54.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.858+0000 7f023d600640 1 -- 172.21.15.86:0/3520041713 wait complete. 2024-04-25T14:09:54.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.862+0000 7f023d600640 1 Processor -- start 2024-04-25T14:09:54.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.862+0000 7f023d600640 1 -- start start 2024-04-25T14:09:54.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.862+0000 7f023d600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02380b0540 0x7f023815a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:54.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.862+0000 7f023d600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f023815e270 0x7f023814f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:54.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.862+0000 7f023d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023815ebc0 0x7f023814fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:54.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.862+0000 7f023d600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f023816b220 con 0x7f023815ebc0 2024-04-25T14:09:54.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.862+0000 7f023d600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f023816b0a0 con 0x7f023815e270 2024-04-25T14:09:54.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.862+0000 7f023d600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f023816b3a0 con 0x7f02380b0540 2024-04-25T14:09:54.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.862+0000 7f0237400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023815ebc0 0x7f023814fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:54.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.862+0000 7f0237400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023815ebc0 0x7f023814fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43712/0 (socket says 172.21.15.86:43712) 2024-04-25T14:09:54.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.862+0000 7f0237400640 1 -- 172.21.15.86:0/3032908599 learned_addr learned my addr 172.21.15.86:0/3032908599 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:54.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.866+0000 7f0236000640 1 --2- 172.21.15.86:0/3032908599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f023815e270 0x7f023814f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:54.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.866+0000 7f0236a00640 1 --2- 172.21.15.86:0/3032908599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02380b0540 0x7f023815a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:54.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.866+0000 7f0237400640 1 -- 172.21.15.86:0/3032908599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02380b0540 msgr2=0x7f023815a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:54.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.866+0000 7f0237400640 1 --2- 172.21.15.86:0/3032908599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02380b0540 0x7f023815a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:54.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.866+0000 7f0237400640 1 -- 172.21.15.86:0/3032908599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f023815e270 msgr2=0x7f023814f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:54.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.866+0000 7f0237400640 1 --2- 172.21.15.86:0/3032908599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f023815e270 0x7f023814f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:54.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.866+0000 7f0237400640 1 -- 172.21.15.86:0/3032908599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02380cac20 con 0x7f023815ebc0 2024-04-25T14:09:54.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.866+0000 7f0237400640 1 --2- 172.21.15.86:0/3032908599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023815ebc0 0x7f023814fd80 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f023000c730 tx=0x7f023000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:54.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.866+0000 7f022ea00640 1 -- 172.21.15.86:0/3032908599 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0230003b80 con 0x7f023815ebc0 2024-04-25T14:09:54.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.866+0000 7f022ea00640 1 -- 172.21.15.86:0/3032908599 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0230003d20 con 0x7f023815ebc0 2024-04-25T14:09:54.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.866+0000 7f022ea00640 1 -- 172.21.15.86:0/3032908599 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023000aa30 con 0x7f023815ebc0 2024-04-25T14:09:54.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.866+0000 7f023d600640 1 -- 172.21.15.86:0/3032908599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02380caf00 con 0x7f023815ebc0 2024-04-25T14:09:54.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.866+0000 7f023d600640 1 -- 172.21.15.86:0/3032908599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02380cb4c0 con 0x7f023815ebc0 2024-04-25T14:09:54.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.866+0000 7f023d600640 1 -- 172.21.15.86:0/3032908599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02380ca990 con 0x7f023815ebc0 2024-04-25T14:09:54.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.866+0000 7f022ea00640 1 -- 172.21.15.86:0/3032908599 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7f023000abd0 con 0x7f023815ebc0 2024-04-25T14:09:54.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.870+0000 7f022ea00640 1 --2- 172.21.15.86:0/3032908599 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f021806f520 0x7f0218071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:54.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.870+0000 7f022ea00640 1 -- 172.21.15.86:0/3032908599 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6832+0+0 (secure 0 0 0) 0x7f023008f690 con 0x7f023815ebc0 2024-04-25T14:09:54.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.870+0000 7f0236a00640 1 --2- 172.21.15.86:0/3032908599 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f021806f520 0x7f0218071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:54.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.870+0000 7f0236a00640 1 --2- 172.21.15.86:0/3032908599 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f021806f520 0x7f0218071a30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0228009580 tx=0x7f0228009190 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:54.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:54.874+0000 7f022ea00640 1 -- 172.21.15.86:0/3032908599 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f023005b940 con 0x7f023815ebc0 2024-04-25T14:09:55.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.030+0000 7f023d600640 1 -- 172.21.15.86:0/3032908599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0238075eb0 con 0x7f023815ebc0 2024-04-25T14:09:55.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.678+0000 7f022ea00640 1 -- 172.21.15.86:0/3032908599 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v104) ==== 181+0+0 (secure 0 0 0) 0x7f023005fc40 con 0x7f023815ebc0 2024-04-25T14:09:55.676 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-04-25T14:09:55.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.682+0000 7f023d600640 1 -- 172.21.15.86:0/3032908599 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f021806f520 msgr2=0x7f0218071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:55.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.682+0000 7f023d600640 1 --2- 172.21.15.86:0/3032908599 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f021806f520 0x7f0218071a30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0228009580 tx=0x7f0228009190 comp rx=0 tx=0).stop 2024-04-25T14:09:55.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.682+0000 7f023d600640 1 -- 172.21.15.86:0/3032908599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023815ebc0 msgr2=0x7f023814fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:55.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.682+0000 7f023d600640 1 --2- 172.21.15.86:0/3032908599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023815ebc0 0x7f023814fd80 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f023000c730 tx=0x7f023000cbf0 comp rx=0 tx=0).stop 2024-04-25T14:09:55.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.682+0000 7f023d600640 1 -- 172.21.15.86:0/3032908599 shutdown_connections 2024-04-25T14:09:55.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.682+0000 7f023d600640 1 --2- 172.21.15.86:0/3032908599 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f021806f520 0x7f0218071a30 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:55.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.682+0000 7f023d600640 1 --2- 172.21.15.86:0/3032908599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023815ebc0 0x7f023814fd80 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:55.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.682+0000 7f023d600640 1 --2- 172.21.15.86:0/3032908599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f023815e270 0x7f023814f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:55.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.682+0000 7f023d600640 1 --2- 172.21.15.86:0/3032908599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02380b0540 0x7f023815a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:55.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.682+0000 7f023d600640 1 -- 172.21.15.86:0/3032908599 >> 172.21.15.86:0/3032908599 conn(0x7f02380b4a40 msgr2=0x7f02380b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:55.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.682+0000 7f023d600640 1 -- 172.21.15.86:0/3032908599 shutdown_connections 2024-04-25T14:09:55.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.682+0000 7f023d600640 1 -- 172.21.15.86:0/3032908599 wait complete. 2024-04-25T14:09:55.701 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T14:09:55.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.854+0000 7f2786e00640 1 Processor -- start 2024-04-25T14:09:55.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.854+0000 7f2786e00640 1 -- start start 2024-04-25T14:09:55.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.858+0000 7f2786e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27800b0540 0x7f27800ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:55.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.858+0000 7f2786e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27800b0b20 con 0x7f278015e270 2024-04-25T14:09:55.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.858+0000 7f2786e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f278006d220 con 0x7f2780165650 2024-04-25T14:09:55.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.858+0000 7f2786e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f278006d870 con 0x7f27800b0540 2024-04-25T14:09:55.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.858+0000 7f2777400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f278015e270 0x7f278015e680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34684/0 (socket says 172.21.15.86:34684) 2024-04-25T14:09:55.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.858+0000 7f2777400640 1 -- 172.21.15.86:0/2736984240 learned_addr learned my addr 172.21.15.86:0/2736984240 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:55.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.858+0000 7f277fe00640 1 --2- 172.21.15.86:0/2736984240 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27800b0540 0x7f27800ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:55.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.858+0000 7f277f400640 1 -- 172.21.15.86:0/2736984240 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2834632061 0 0) 0x7f27800b0b20 con 0x7f278015e270 2024-04-25T14:09:55.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.858+0000 7f277f400640 1 -- 172.21.15.86:0/2736984240 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2764003530 con 0x7f278015e270 2024-04-25T14:09:55.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.858+0000 7f277f400640 1 -- 172.21.15.86:0/2736984240 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1912009054 0 0) 0x7f278006d220 con 0x7f2780165650 2024-04-25T14:09:55.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.858+0000 7f277f400640 1 -- 172.21.15.86:0/2736984240 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27800b0b20 con 0x7f2780165650 2024-04-25T14:09:55.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.858+0000 7f277fe00640 1 -- 172.21.15.86:0/2736984240 >> v1:172.21.15.155:6789/0 conn(0x7f2780165650 legacy=0x7f2780167a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:55.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.858+0000 7f277fe00640 1 -- 172.21.15.86:0/2736984240 >> v1:172.21.15.86:6789/0 conn(0x7f278015e270 legacy=0x7f278015e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:55.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.858+0000 7f277fe00640 1 -- 172.21.15.86:0/2736984240 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278006d500 con 0x7f27800b0540 2024-04-25T14:09:55.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.862+0000 7f277fe00640 1 --2- 172.21.15.86:0/2736984240 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27800b0540 0x7f27800ae2a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f27700097f0 tx=0x7f277001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddd047b232e8851d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:55.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.862+0000 7f277f400640 1 -- 172.21.15.86:0/2736984240 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27700b0070 con 0x7f27800b0540 2024-04-25T14:09:55.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.862+0000 7f277f400640 1 -- 172.21.15.86:0/2736984240 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27700ab600 con 0x7f27800b0540 2024-04-25T14:09:55.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.862+0000 7f277f400640 1 -- 172.21.15.86:0/2736984240 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27700ab9d0 con 0x7f27800b0540 2024-04-25T14:09:55.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.862+0000 7f2786e00640 1 -- 172.21.15.86:0/2736984240 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27800b0540 msgr2=0x7f27800ae2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:55.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.862+0000 7f2786e00640 1 --2- 172.21.15.86:0/2736984240 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27800b0540 0x7f27800ae2a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f27700097f0 tx=0x7f277001c450 comp rx=0 tx=0).stop 2024-04-25T14:09:55.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.862+0000 7f2786e00640 1 -- 172.21.15.86:0/2736984240 shutdown_connections 2024-04-25T14:09:55.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.862+0000 7f2786e00640 1 --2- 172.21.15.86:0/2736984240 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27800b0540 0x7f27800ae2a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:55.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.862+0000 7f2786e00640 1 -- 172.21.15.86:0/2736984240 >> 172.21.15.86:0/2736984240 conn(0x7f27800b4a40 msgr2=0x7f27800b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:55.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.862+0000 7f2786e00640 1 -- 172.21.15.86:0/2736984240 shutdown_connections 2024-04-25T14:09:55.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.862+0000 7f2786e00640 1 -- 172.21.15.86:0/2736984240 wait complete. 2024-04-25T14:09:55.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.862+0000 7f2786e00640 1 Processor -- start 2024-04-25T14:09:55.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.866+0000 7f2786e00640 1 -- start start 2024-04-25T14:09:55.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.866+0000 7f2786e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27800b0540 0x7f278015aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:55.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.866+0000 7f2786e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278015e270 0x7f2780153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:55.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.866+0000 7f2786e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2780165650 0x7f27801441d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:55.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.866+0000 7f2786e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f278016bd40 con 0x7f278015e270 2024-04-25T14:09:55.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.866+0000 7f2786e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f278016bbc0 con 0x7f2780165650 2024-04-25T14:09:55.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.866+0000 7f2786e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f278016bec0 con 0x7f27800b0540 2024-04-25T14:09:55.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.866+0000 7f2777400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278015e270 0x7f2780153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:55.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.866+0000 7f2777400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278015e270 0x7f2780153140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43722/0 (socket says 172.21.15.86:43722) 2024-04-25T14:09:55.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.866+0000 7f2777400640 1 -- 172.21.15.86:0/898169991 learned_addr learned my addr 172.21.15.86:0/898169991 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:55.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.866+0000 7f277fe00640 1 --2- 172.21.15.86:0/898169991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27800b0540 0x7f278015aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:55.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.866+0000 7f2777400640 1 -- 172.21.15.86:0/898169991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27800b0540 msgr2=0x7f278015aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:55.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.866+0000 7f2784e00640 1 --2- 172.21.15.86:0/898169991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2780165650 0x7f27801441d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:55.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.866+0000 7f2777400640 1 --2- 172.21.15.86:0/898169991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27800b0540 0x7f278015aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:55.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.866+0000 7f2777400640 1 -- 172.21.15.86:0/898169991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2780165650 msgr2=0x7f27801441d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:55.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.866+0000 7f2777400640 1 --2- 172.21.15.86:0/898169991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2780165650 0x7f27801441d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:55.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.866+0000 7f2777400640 1 -- 172.21.15.86:0/898169991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27801528d0 con 0x7f278015e270 2024-04-25T14:09:55.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.866+0000 7f277fe00640 1 --2- 172.21.15.86:0/898169991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27800b0540 0x7f278015aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:55.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.866+0000 7f2784e00640 1 --2- 172.21.15.86:0/898169991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2780165650 0x7f27801441d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:55.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.866+0000 7f2777400640 1 --2- 172.21.15.86:0/898169991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278015e270 0x7f2780153140 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f276800c460 tx=0x7f276800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:55.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.866+0000 7f277cc00640 1 -- 172.21.15.86:0/898169991 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2768017070 con 0x7f278015e270 2024-04-25T14:09:55.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.866+0000 7f277cc00640 1 -- 172.21.15.86:0/898169991 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2768005cf0 con 0x7f278015e270 2024-04-25T14:09:55.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.866+0000 7f2786e00640 1 -- 172.21.15.86:0/898169991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2780165d90 con 0x7f278015e270 2024-04-25T14:09:55.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.866+0000 7f277cc00640 1 -- 172.21.15.86:0/898169991 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2768007290 con 0x7f278015e270 2024-04-25T14:09:55.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.866+0000 7f2786e00640 1 -- 172.21.15.86:0/898169991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2780166350 con 0x7f278015e270 2024-04-25T14:09:55.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.866+0000 7f2786e00640 1 -- 172.21.15.86:0/898169991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2780050750 con 0x7f278015e270 2024-04-25T14:09:55.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.870+0000 7f277cc00640 1 -- 172.21.15.86:0/898169991 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7f276801a070 con 0x7f278015e270 2024-04-25T14:09:55.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.870+0000 7f277cc00640 1 --2- 172.21.15.86:0/898169991 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f275c06f520 0x7f275c071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:55.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.870+0000 7f277cc00640 1 -- 172.21.15.86:0/898169991 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6394+0+0 (secure 0 0 0) 0x7f276808eac0 con 0x7f278015e270 2024-04-25T14:09:55.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.870+0000 7f277fe00640 1 --2- 172.21.15.86:0/898169991 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f275c06f520 0x7f275c071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:55.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.870+0000 7f277fe00640 1 --2- 172.21.15.86:0/898169991 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f275c06f520 0x7f275c071a30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f27700ad000 tx=0x7f2770008040 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:55.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:55.874+0000 7f277cc00640 1 -- 172.21.15.86:0/898169991 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f27801528d0 con 0x7f278015e270 2024-04-25T14:09:56.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.034+0000 7f2786e00640 1 -- 172.21.15.86:0/898169991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2780153f50 con 0x7f278015e270 2024-04-25T14:09:56.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.682+0000 7f277cc00640 1 -- 172.21.15.86:0/898169991 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v105) ==== 169+0+0 (secure 0 0 0) 0x7f276805b550 con 0x7f278015e270 2024-04-25T14:09:56.677 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-04-25T14:09:56.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.686+0000 7f2786e00640 1 -- 172.21.15.86:0/898169991 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f275c06f520 msgr2=0x7f275c071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:56.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.686+0000 7f2786e00640 1 --2- 172.21.15.86:0/898169991 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f275c06f520 0x7f275c071a30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f27700ad000 tx=0x7f2770008040 comp rx=0 tx=0).stop 2024-04-25T14:09:56.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.686+0000 7f2786e00640 1 -- 172.21.15.86:0/898169991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278015e270 msgr2=0x7f2780153140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:56.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.686+0000 7f2786e00640 1 --2- 172.21.15.86:0/898169991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278015e270 0x7f2780153140 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f276800c460 tx=0x7f276800c920 comp rx=0 tx=0).stop 2024-04-25T14:09:56.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.686+0000 7f2786e00640 1 -- 172.21.15.86:0/898169991 shutdown_connections 2024-04-25T14:09:56.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.686+0000 7f2786e00640 1 --2- 172.21.15.86:0/898169991 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f275c06f520 0x7f275c071a30 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:56.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.686+0000 7f2786e00640 1 --2- 172.21.15.86:0/898169991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2780165650 0x7f27801441d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:56.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.686+0000 7f2786e00640 1 --2- 172.21.15.86:0/898169991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278015e270 0x7f2780153140 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:56.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.686+0000 7f2786e00640 1 --2- 172.21.15.86:0/898169991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27800b0540 0x7f278015aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:56.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.686+0000 7f2786e00640 1 -- 172.21.15.86:0/898169991 >> 172.21.15.86:0/898169991 conn(0x7f27800b4a40 msgr2=0x7f278015e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:56.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.686+0000 7f2786e00640 1 -- 172.21.15.86:0/898169991 shutdown_connections 2024-04-25T14:09:56.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.686+0000 7f2786e00640 1 -- 172.21.15.86:0/898169991 wait complete. 2024-04-25T14:09:56.709 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T14:09:56.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.862+0000 7f5bd2400640 1 Processor -- start 2024-04-25T14:09:56.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.862+0000 7f5bd2400640 1 -- start start 2024-04-25T14:09:56.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.862+0000 7f5bd2400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bcc0d16e0 0x7f5bcc0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:56.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.862+0000 7f5bd2400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5bcc0730b0 0x7f5bcc0734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:56.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.862+0000 7f5bd2400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bcc0d2030 con 0x7f5bcc0cfca0 2024-04-25T14:09:56.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.862+0000 7f5bd2400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5bcc070470 con 0x7f5bcc0730b0 2024-04-25T14:09:56.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.862+0000 7f5bd2400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5bcc072bd0 con 0x7f5bcc0d16e0 2024-04-25T14:09:56.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.862+0000 7f5bcb400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5bcc0cfca0 0x7f5bcc06f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34696/0 (socket says 172.21.15.86:34696) 2024-04-25T14:09:56.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.862+0000 7f5bcb400640 1 -- 172.21.15.86:0/1184554919 learned_addr learned my addr 172.21.15.86:0/1184554919 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:56.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.862+0000 7f5bcbe00640 1 --2- 172.21.15.86:0/1184554919 >> v2:172.21.15.86:3300/0 conn(0x7f5bcc0730b0 0x7f5bcc0734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:56.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.862+0000 7f5bc2a00640 1 --2- 172.21.15.86:0/1184554919 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bcc0d16e0 0x7f5bcc0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:56.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.862+0000 7f5bcbe00640 1 -- 172.21.15.86:0/1184554919 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bcc0d16e0 msgr2=0x7f5bcc0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:56.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.862+0000 7f5bcbe00640 1 --2- 172.21.15.86:0/1184554919 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bcc0d16e0 0x7f5bcc0d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:56.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.862+0000 7f5bcbe00640 1 -- 172.21.15.86:0/1184554919 >> v1:172.21.15.86:6789/0 conn(0x7f5bcc0cfca0 legacy=0x7f5bcc06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:56.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.862+0000 7f5bcbe00640 1 -- 172.21.15.86:0/1184554919 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bcc06fbd0 con 0x7f5bcc0730b0 2024-04-25T14:09:56.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.862+0000 7f5bcaa00640 1 -- 172.21.15.86:0/1184554919 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1781216625 0 0) 0x7f5bbc001120 con 0x7f5bcc0cfca0 2024-04-25T14:09:56.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.866+0000 7f5bcbe00640 1 --2- 172.21.15.86:0/1184554919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bcc0730b0 0x7f5bcc0734c0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009d90 tx=0x7f5bc401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585ca2b172093d0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:56.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.866+0000 7f5bcaa00640 1 -- 172.21.15.86:0/1184554919 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc40b2070 con 0x7f5bcc0730b0 2024-04-25T14:09:56.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.866+0000 7f5bcaa00640 1 -- 172.21.15.86:0/1184554919 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bc40a5940 con 0x7f5bcc0730b0 2024-04-25T14:09:56.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.866+0000 7f5bcaa00640 1 -- 172.21.15.86:0/1184554919 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc40a5d10 con 0x7f5bcc0730b0 2024-04-25T14:09:56.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.866+0000 7f5bd2400640 1 -- 172.21.15.86:0/1184554919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bcc0730b0 msgr2=0x7f5bcc0734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:56.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.866+0000 7f5bd2400640 1 --2- 172.21.15.86:0/1184554919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bcc0730b0 0x7f5bcc0734c0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009d90 tx=0x7f5bc401c970 comp rx=0 tx=0).stop 2024-04-25T14:09:56.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.866+0000 7f5bd2400640 1 -- 172.21.15.86:0/1184554919 shutdown_connections 2024-04-25T14:09:56.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.866+0000 7f5bd2400640 1 --2- 172.21.15.86:0/1184554919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bcc0730b0 0x7f5bcc0734c0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:56.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.866+0000 7f5bd2400640 1 --2- 172.21.15.86:0/1184554919 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bcc0d16e0 0x7f5bcc0d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:56.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.866+0000 7f5bd2400640 1 -- 172.21.15.86:0/1184554919 >> 172.21.15.86:0/1184554919 conn(0x7f5bcc0b4650 msgr2=0x7f5bcc0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:56.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.866+0000 7f5bd2400640 1 -- 172.21.15.86:0/1184554919 shutdown_connections 2024-04-25T14:09:56.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.866+0000 7f5bd2400640 1 -- 172.21.15.86:0/1184554919 wait complete. 2024-04-25T14:09:56.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.870+0000 7f5bd2400640 1 Processor -- start 2024-04-25T14:09:56.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.870+0000 7f5bd2400640 1 -- start start 2024-04-25T14:09:56.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.870+0000 7f5bd2400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bcc0730b0 0x7f5bcc1fdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:56.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.870+0000 7f5bd2400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bcc0cfca0 0x7f5bcc15fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:56.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.870+0000 7f5bd2400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bcc0d16e0 0x7f5bcc0af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:56.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.870+0000 7f5bd2400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc169a70 con 0x7f5bcc0d16e0 2024-04-25T14:09:56.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.870+0000 7f5bd2400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc1698f0 con 0x7f5bcc0730b0 2024-04-25T14:09:56.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.870+0000 7f5bd2400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5bcc169bf0 con 0x7f5bcc0cfca0 2024-04-25T14:09:56.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.870+0000 7f5bcbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bcc0d16e0 0x7f5bcc0af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:56.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.870+0000 7f5bcbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bcc0d16e0 0x7f5bcc0af2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43746/0 (socket says 172.21.15.86:43746) 2024-04-25T14:09:56.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.870+0000 7f5bcbe00640 1 -- 172.21.15.86:0/1444511889 learned_addr learned my addr 172.21.15.86:0/1444511889 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:56.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.870+0000 7f5bc2a00640 1 --2- 172.21.15.86:0/1444511889 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bcc0cfca0 0x7f5bcc15fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:56.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.870+0000 7f5bcb400640 1 --2- 172.21.15.86:0/1444511889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bcc0730b0 0x7f5bcc1fdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:56.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.870+0000 7f5bcbe00640 1 -- 172.21.15.86:0/1444511889 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bcc0cfca0 msgr2=0x7f5bcc15fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:56.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.870+0000 7f5bcbe00640 1 --2- 172.21.15.86:0/1444511889 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bcc0cfca0 0x7f5bcc15fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:56.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.870+0000 7f5bcbe00640 1 -- 172.21.15.86:0/1444511889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bcc0730b0 msgr2=0x7f5bcc1fdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:56.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.870+0000 7f5bcbe00640 1 --2- 172.21.15.86:0/1444511889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bcc0730b0 0x7f5bcc1fdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:56.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.870+0000 7f5bcbe00640 1 -- 172.21.15.86:0/1444511889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bcc15a1e0 con 0x7f5bcc0d16e0 2024-04-25T14:09:56.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.870+0000 7f5bcb400640 1 --2- 172.21.15.86:0/1444511889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bcc0730b0 0x7f5bcc1fdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:56.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.874+0000 7f5bcbe00640 1 --2- 172.21.15.86:0/1444511889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bcc0d16e0 0x7f5bcc0af2b0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f5bc401b020 tx=0x7f5bc4019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:56.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.874+0000 7f5bc3e00640 1 -- 172.21.15.86:0/1444511889 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc40b2050 con 0x7f5bcc0d16e0 2024-04-25T14:09:56.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.874+0000 7f5bc3e00640 1 -- 172.21.15.86:0/1444511889 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bc40be070 con 0x7f5bcc0d16e0 2024-04-25T14:09:56.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.874+0000 7f5bc3e00640 1 -- 172.21.15.86:0/1444511889 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc40ada50 con 0x7f5bcc0d16e0 2024-04-25T14:09:56.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.874+0000 7f5bd2400640 1 -- 172.21.15.86:0/1444511889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bcc1fe520 con 0x7f5bcc0d16e0 2024-04-25T14:09:56.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.874+0000 7f5bd2400640 1 -- 172.21.15.86:0/1444511889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bcc0cdef0 con 0x7f5bcc0d16e0 2024-04-25T14:09:56.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.874+0000 7f5bd2400640 1 -- 172.21.15.86:0/1444511889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bcc09f4b0 con 0x7f5bcc0d16e0 2024-04-25T14:09:56.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.874+0000 7f5bc3e00640 1 -- 172.21.15.86:0/1444511889 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7f5bc40adbf0 con 0x7f5bcc0d16e0 2024-04-25T14:09:56.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.874+0000 7f5bc3e00640 1 --2- 172.21.15.86:0/1444511889 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f5ba806f520 0x7f5ba8071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:56.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.874+0000 7f5bc3e00640 1 -- 172.21.15.86:0/1444511889 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5980+0+0 (secure 0 0 0) 0x7f5bc412a6f0 con 0x7f5bcc0d16e0 2024-04-25T14:09:56.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.874+0000 7f5bcb400640 1 --2- 172.21.15.86:0/1444511889 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f5ba806f520 0x7f5ba8071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:56.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.878+0000 7f5bcb400640 1 --2- 172.21.15.86:0/1444511889 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f5ba806f520 0x7f5ba8071a30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc000f70 tx=0x7f5bbc007cb0 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:56.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:56.882+0000 7f5bc3e00640 1 -- 172.21.15.86:0/1444511889 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5bc4016560 con 0x7f5bcc0d16e0 2024-04-25T14:09:57.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.046+0000 7f5bd2400640 1 -- 172.21.15.86:0/1444511889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f5bcc069d00 con 0x7f5bcc0d16e0 2024-04-25T14:09:57.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.050+0000 7f5bc3e00640 1 -- 172.21.15.86:0/1444511889 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f5bc40f7320 con 0x7f5bcc0d16e0 2024-04-25T14:09:57.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.054+0000 7f5bd2400640 1 -- 172.21.15.86:0/1444511889 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f5ba806f520 msgr2=0x7f5ba8071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:57.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.054+0000 7f5bd2400640 1 --2- 172.21.15.86:0/1444511889 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f5ba806f520 0x7f5ba8071a30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc000f70 tx=0x7f5bbc007cb0 comp rx=0 tx=0).stop 2024-04-25T14:09:57.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.054+0000 7f5bd2400640 1 -- 172.21.15.86:0/1444511889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bcc0d16e0 msgr2=0x7f5bcc0af2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:57.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.054+0000 7f5bd2400640 1 --2- 172.21.15.86:0/1444511889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bcc0d16e0 0x7f5bcc0af2b0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f5bc401b020 tx=0x7f5bc4019710 comp rx=0 tx=0).stop 2024-04-25T14:09:57.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.054+0000 7f5bd2400640 1 -- 172.21.15.86:0/1444511889 shutdown_connections 2024-04-25T14:09:57.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.054+0000 7f5bd2400640 1 --2- 172.21.15.86:0/1444511889 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f5ba806f520 0x7f5ba8071a30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:57.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.054+0000 7f5bd2400640 1 --2- 172.21.15.86:0/1444511889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bcc0d16e0 0x7f5bcc0af2b0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:57.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.054+0000 7f5bd2400640 1 --2- 172.21.15.86:0/1444511889 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bcc0cfca0 0x7f5bcc15fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:57.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.054+0000 7f5bd2400640 1 --2- 172.21.15.86:0/1444511889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bcc0730b0 0x7f5bcc1fdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:57.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.054+0000 7f5bd2400640 1 -- 172.21.15.86:0/1444511889 >> 172.21.15.86:0/1444511889 conn(0x7f5bcc0b4650 msgr2=0x7f5bcc09a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:57.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.054+0000 7f5bd2400640 1 -- 172.21.15.86:0/1444511889 shutdown_connections 2024-04-25T14:09:57.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.054+0000 7f5bd2400640 1 -- 172.21.15.86:0/1444511889 wait complete. 2024-04-25T14:09:57.070 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_for_invalid_client_ip' 2024-04-25T14:09:57.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.218+0000 7f6795a00640 1 Processor -- start 2024-04-25T14:09:57.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.218+0000 7f6795a00640 1 -- start start 2024-04-25T14:09:57.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.222+0000 7f6795a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67900d1df0 0x7f67900cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:57.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.222+0000 7f6795a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f679015e5c0 con 0x7f67900cdcc0 2024-04-25T14:09:57.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.222+0000 7f6795a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f679015e8a0 con 0x7f6790070330 2024-04-25T14:09:57.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.222+0000 7f6795a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f679006fd40 con 0x7f67900d1df0 2024-04-25T14:09:57.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.222+0000 7f678f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f67900cdcc0 0x7f67900ce0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34700/0 (socket says 172.21.15.86:34700) 2024-04-25T14:09:57.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.222+0000 7f678f400640 1 -- 172.21.15.86:0/3567350671 learned_addr learned my addr 172.21.15.86:0/3567350671 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:57.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.222+0000 7f678e000640 1 --2- 172.21.15.86:0/3567350671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67900d1df0 0x7f67900cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:57.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.222+0000 7f678d600640 1 -- 172.21.15.86:0/3567350671 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 534526680 0 0) 0x7f679015e5c0 con 0x7f67900cdcc0 2024-04-25T14:09:57.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.222+0000 7f678d600640 1 -- 172.21.15.86:0/3567350671 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6774003530 con 0x7f67900cdcc0 2024-04-25T14:09:57.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.222+0000 7f678d600640 1 -- 172.21.15.86:0/3567350671 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1369736445 0 0) 0x7f679015e8a0 con 0x7f6790070330 2024-04-25T14:09:57.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.222+0000 7f678d600640 1 -- 172.21.15.86:0/3567350671 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f679015e5c0 con 0x7f6790070330 2024-04-25T14:09:57.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.222+0000 7f678e000640 1 -- 172.21.15.86:0/3567350671 >> v1:172.21.15.155:6789/0 conn(0x7f6790070330 legacy=0x7f67900d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:57.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.222+0000 7f678e000640 1 -- 172.21.15.86:0/3567350671 >> v1:172.21.15.86:6789/0 conn(0x7f67900cdcc0 legacy=0x7f67900ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:57.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.222+0000 7f678e000640 1 -- 172.21.15.86:0/3567350671 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f679015eb80 con 0x7f67900d1df0 2024-04-25T14:09:57.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.226+0000 7f678e000640 1 --2- 172.21.15.86:0/3567350671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67900d1df0 0x7f67900cd780 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f677c0097f0 tx=0x7f677c01c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3df069e5ede6f22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:57.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.226+0000 7f678d600640 1 -- 172.21.15.86:0/3567350671 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c0b0070 con 0x7f67900d1df0 2024-04-25T14:09:57.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.226+0000 7f678d600640 1 -- 172.21.15.86:0/3567350671 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f677c0ab600 con 0x7f67900d1df0 2024-04-25T14:09:57.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.226+0000 7f678d600640 1 -- 172.21.15.86:0/3567350671 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c0ab9d0 con 0x7f67900d1df0 2024-04-25T14:09:57.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.226+0000 7f6795a00640 1 -- 172.21.15.86:0/3567350671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67900d1df0 msgr2=0x7f67900cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:57.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.226+0000 7f6795a00640 1 --2- 172.21.15.86:0/3567350671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67900d1df0 0x7f67900cd780 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f677c0097f0 tx=0x7f677c01c420 comp rx=0 tx=0).stop 2024-04-25T14:09:57.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.226+0000 7f6795a00640 1 -- 172.21.15.86:0/3567350671 shutdown_connections 2024-04-25T14:09:57.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.226+0000 7f6795a00640 1 --2- 172.21.15.86:0/3567350671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67900d1df0 0x7f67900cd780 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:57.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.226+0000 7f6795a00640 1 -- 172.21.15.86:0/3567350671 >> 172.21.15.86:0/3567350671 conn(0x7f67900b4650 msgr2=0x7f67900b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:57.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.226+0000 7f6795a00640 1 -- 172.21.15.86:0/3567350671 shutdown_connections 2024-04-25T14:09:57.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.226+0000 7f6795a00640 1 -- 172.21.15.86:0/3567350671 wait complete. 2024-04-25T14:09:57.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.226+0000 7f6795a00640 1 Processor -- start 2024-04-25T14:09:57.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.226+0000 7f6795a00640 1 -- start start 2024-04-25T14:09:57.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.230+0000 7f6795a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6790070330 0x7f67901fc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:57.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.230+0000 7f6795a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67900cdcc0 0x7f679009cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:57.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.230+0000 7f6795a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67901f97f0 0x7f679009d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:57.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.230+0000 7f6795a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f67900708c0 con 0x7f67900cdcc0 2024-04-25T14:09:57.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.230+0000 7f6795a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6790070740 con 0x7f67901f97f0 2024-04-25T14:09:57.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.230+0000 7f6795a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f679016bfd0 con 0x7f6790070330 2024-04-25T14:09:57.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.230+0000 7f678e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67900cdcc0 0x7f679009cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:57.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.230+0000 7f678e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67900cdcc0 0x7f679009cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43752/0 (socket says 172.21.15.86:43752) 2024-04-25T14:09:57.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.230+0000 7f678e000640 1 -- 172.21.15.86:0/2311463588 learned_addr learned my addr 172.21.15.86:0/2311463588 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:57.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.230+0000 7f678f400640 1 --2- 172.21.15.86:0/2311463588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67901f97f0 0x7f679009d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:57.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.230+0000 7f678e000640 1 -- 172.21.15.86:0/2311463588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6790070330 msgr2=0x7f67901fc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:57.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.230+0000 7f678ea00640 1 --2- 172.21.15.86:0/2311463588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6790070330 0x7f67901fc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:57.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.230+0000 7f678e000640 1 --2- 172.21.15.86:0/2311463588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6790070330 0x7f67901fc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:57.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.230+0000 7f678e000640 1 -- 172.21.15.86:0/2311463588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67901f97f0 msgr2=0x7f679009d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:57.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.230+0000 7f678e000640 1 --2- 172.21.15.86:0/2311463588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67901f97f0 0x7f679009d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:57.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.230+0000 7f678e000640 1 -- 172.21.15.86:0/2311463588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67900cdaf0 con 0x7f67900cdcc0 2024-04-25T14:09:57.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.230+0000 7f678f400640 1 --2- 172.21.15.86:0/2311463588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67901f97f0 0x7f679009d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:57.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.230+0000 7f678ea00640 1 --2- 172.21.15.86:0/2311463588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6790070330 0x7f67901fc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:57.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.230+0000 7f678e000640 1 --2- 172.21.15.86:0/2311463588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67900cdcc0 0x7f679009cac0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f677c009740 tx=0x7f677c0191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:57.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.230+0000 7f6786a00640 1 -- 172.21.15.86:0/2311463588 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c0b0040 con 0x7f67900cdcc0 2024-04-25T14:09:57.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.230+0000 7f6786a00640 1 -- 172.21.15.86:0/2311463588 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f677c0abe30 con 0x7f67900cdcc0 2024-04-25T14:09:57.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.230+0000 7f6786a00640 1 -- 172.21.15.86:0/2311463588 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c004220 con 0x7f67900cdcc0 2024-04-25T14:09:57.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.230+0000 7f6795a00640 1 -- 172.21.15.86:0/2311463588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67900ca960 con 0x7f67900cdcc0 2024-04-25T14:09:57.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.230+0000 7f6795a00640 1 -- 172.21.15.86:0/2311463588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67900caf20 con 0x7f67900cdcc0 2024-04-25T14:09:57.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.230+0000 7f6795a00640 1 -- 172.21.15.86:0/2311463588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f679015db60 con 0x7f67900cdcc0 2024-04-25T14:09:57.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.234+0000 7f6786a00640 1 -- 172.21.15.86:0/2311463588 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7f677c0b3040 con 0x7f67900cdcc0 2024-04-25T14:09:57.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.234+0000 7f6786a00640 1 --2- 172.21.15.86:0/2311463588 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f677006f520 0x7f6770071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:57.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.234+0000 7f6786a00640 1 -- 172.21.15.86:0/2311463588 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5980+0+0 (secure 0 0 0) 0x7f677c1298e0 con 0x7f67900cdcc0 2024-04-25T14:09:57.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.234+0000 7f678ea00640 1 --2- 172.21.15.86:0/2311463588 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f677006f520 0x7f6770071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:57.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.234+0000 7f678ea00640 1 --2- 172.21.15.86:0/2311463588 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f677006f520 0x7f6770071a30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6780001570 tx=0x7f6780009040 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:57.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.238+0000 7f6786a00640 1 -- 172.21.15.86:0/2311463588 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f677c0c4030 con 0x7f67900cdcc0 2024-04-25T14:09:57.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.446+0000 7f6795a00640 1 -- 172.21.15.86:0/2311463588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_for_invalid_client_ip"]} v 0) -- 0x7f6790069d00 con 0x7f67900cdcc0 2024-04-25T14:09:57.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.682+0000 7f6786a00640 1 -- 172.21.15.86:0/2311463588 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_for_invalid_client_ip"]}]=0 v350) ==== 150+0+0 (secure 0 0 0) 0x7f677c0bb030 con 0x7f67900cdcc0 2024-04-25T14:09:57.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.686+0000 7f6795a00640 1 -- 172.21.15.86:0/2311463588 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f677006f520 msgr2=0x7f6770071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:57.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.686+0000 7f6795a00640 1 --2- 172.21.15.86:0/2311463588 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f677006f520 0x7f6770071a30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6780001570 tx=0x7f6780009040 comp rx=0 tx=0).stop 2024-04-25T14:09:57.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.686+0000 7f6795a00640 1 -- 172.21.15.86:0/2311463588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67900cdcc0 msgr2=0x7f679009cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:57.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.686+0000 7f6795a00640 1 --2- 172.21.15.86:0/2311463588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67900cdcc0 0x7f679009cac0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f677c009740 tx=0x7f677c0191c0 comp rx=0 tx=0).stop 2024-04-25T14:09:57.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.686+0000 7f6795a00640 1 -- 172.21.15.86:0/2311463588 shutdown_connections 2024-04-25T14:09:57.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.686+0000 7f6795a00640 1 --2- 172.21.15.86:0/2311463588 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f677006f520 0x7f6770071a30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:57.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.686+0000 7f6795a00640 1 --2- 172.21.15.86:0/2311463588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67901f97f0 0x7f679009d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:57.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.686+0000 7f6795a00640 1 --2- 172.21.15.86:0/2311463588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67900cdcc0 0x7f679009cac0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:57.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.686+0000 7f6795a00640 1 --2- 172.21.15.86:0/2311463588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6790070330 0x7f67901fc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:57.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.686+0000 7f6795a00640 1 -- 172.21.15.86:0/2311463588 >> 172.21.15.86:0/2311463588 conn(0x7f67900b4650 msgr2=0x7f6790159060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:57.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.686+0000 7f6795a00640 1 -- 172.21.15.86:0/2311463588 shutdown_connections 2024-04-25T14:09:57.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.686+0000 7f6795a00640 1 -- 172.21.15.86:0/2311463588 wait complete. 2024-04-25T14:09:57.702 INFO:tasks.cephfs_test_runner:test_for_invalid_client_ip (tasks.cephfs.test_mds_metrics.TestMDSMetrics) ... ok 2024-04-25T14:09:57.702 INFO:tasks.cephfs_test_runner:Starting test: test_for_invalid_mds_rank (tasks.cephfs.test_mds_metrics.TestMDSMetrics) 2024-04-25T14:09:57.703 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:09:57.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.858+0000 7f11b2800640 1 Processor -- start 2024-04-25T14:09:57.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.858+0000 7f11b2800640 1 -- start start 2024-04-25T14:09:57.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.858+0000 7f11b2800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11ac070330 0x7f11ac0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:57.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.858+0000 7f11b2800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f11ac0d1c20 0x7f11ac06ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:57.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.858+0000 7f11b2800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11ac0d2030 con 0x7f11ac06f270 2024-04-25T14:09:57.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.858+0000 7f11b2800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f11ac15e5c0 con 0x7f11ac0d1c20 2024-04-25T14:09:57.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.858+0000 7f11b2800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f11ac15e8a0 con 0x7f11ac070330 2024-04-25T14:09:57.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.858+0000 7f11aaa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f11ac0d1c20 0x7f11ac06ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:57.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.858+0000 7f11aaa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f11ac0d1c20 0x7f11ac06ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43764/0 (socket says 172.21.15.86:43764) 2024-04-25T14:09:57.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.858+0000 7f11aaa00640 1 -- 172.21.15.86:0/3120304574 learned_addr learned my addr 172.21.15.86:0/3120304574 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:57.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.858+0000 7f11ab400640 1 --2- 172.21.15.86:0/3120304574 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11ac070330 0x7f11ac0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:57.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.858+0000 7f11aaa00640 1 -- 172.21.15.86:0/3120304574 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11ac070330 msgr2=0x7f11ac0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:57.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.858+0000 7f11aaa00640 1 --2- 172.21.15.86:0/3120304574 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11ac070330 0x7f11ac0d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:57.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.858+0000 7f11aaa00640 1 -- 172.21.15.86:0/3120304574 >> v1:172.21.15.155:6789/0 conn(0x7f11ac06f270 legacy=0x7f11ac06f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:57.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.858+0000 7f11aaa00640 1 -- 172.21.15.86:0/3120304574 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11ac15eb80 con 0x7f11ac0d1c20 2024-04-25T14:09:57.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.862+0000 7f11aaa00640 1 --2- 172.21.15.86:0/3120304574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11ac0d1c20 0x7f11ac06ed30 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f11980098a0 tx=0x7f119801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4029d5eaf4901db2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:57.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.862+0000 7f11aa000640 1 -- 172.21.15.86:0/3120304574 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11980b0070 con 0x7f11ac0d1c20 2024-04-25T14:09:57.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.862+0000 7f11aa000640 1 -- 172.21.15.86:0/3120304574 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11980ab600 con 0x7f11ac0d1c20 2024-04-25T14:09:57.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.862+0000 7f11aa000640 1 -- 172.21.15.86:0/3120304574 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11980ab9d0 con 0x7f11ac0d1c20 2024-04-25T14:09:57.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.862+0000 7f11b2800640 1 -- 172.21.15.86:0/3120304574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11ac0d1c20 msgr2=0x7f11ac06ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:57.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.862+0000 7f11b2800640 1 --2- 172.21.15.86:0/3120304574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11ac0d1c20 0x7f11ac06ed30 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f11980098a0 tx=0x7f119801c4b0 comp rx=0 tx=0).stop 2024-04-25T14:09:57.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.862+0000 7f11b2800640 1 -- 172.21.15.86:0/3120304574 shutdown_connections 2024-04-25T14:09:57.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.862+0000 7f11b2800640 1 --2- 172.21.15.86:0/3120304574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11ac0d1c20 0x7f11ac06ed30 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:57.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.862+0000 7f11b2800640 1 --2- 172.21.15.86:0/3120304574 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11ac070330 0x7f11ac0d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:57.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.862+0000 7f11b2800640 1 -- 172.21.15.86:0/3120304574 >> 172.21.15.86:0/3120304574 conn(0x7f11ac0b4650 msgr2=0x7f11ac0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:57.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.862+0000 7f11b2800640 1 -- 172.21.15.86:0/3120304574 shutdown_connections 2024-04-25T14:09:57.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.862+0000 7f11b2800640 1 -- 172.21.15.86:0/3120304574 wait complete. 2024-04-25T14:09:57.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.866+0000 7f11b2800640 1 Processor -- start 2024-04-25T14:09:57.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.866+0000 7f11b2800640 1 -- start start 2024-04-25T14:09:57.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.866+0000 7f11b2800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11ac06f270 0x7f11ac15a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:57.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.866+0000 7f11b2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11ac070330 0x7f11ac073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:57.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.866+0000 7f11b2800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11ac0d1c20 0x7f11ac0ae270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:57.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.866+0000 7f11b2800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac0708c0 con 0x7f11ac070330 2024-04-25T14:09:57.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.866+0000 7f11b2800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac070740 con 0x7f11ac06f270 2024-04-25T14:09:57.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.866+0000 7f11b2800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f11ac16afc0 con 0x7f11ac0d1c20 2024-04-25T14:09:57.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.866+0000 7f11aaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11ac070330 0x7f11ac073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:57.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.866+0000 7f11aaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11ac070330 0x7f11ac073bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43778/0 (socket says 172.21.15.86:43778) 2024-04-25T14:09:57.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.866+0000 7f11aaa00640 1 -- 172.21.15.86:0/1015105339 learned_addr learned my addr 172.21.15.86:0/1015105339 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:57.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.866+0000 7f11ab400640 1 --2- 172.21.15.86:0/1015105339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11ac06f270 0x7f11ac15a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:57.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.866+0000 7f11abe00640 1 --2- 172.21.15.86:0/1015105339 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11ac0d1c20 0x7f11ac0ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:57.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.866+0000 7f11aaa00640 1 -- 172.21.15.86:0/1015105339 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11ac0d1c20 msgr2=0x7f11ac0ae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:57.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.866+0000 7f11aaa00640 1 --2- 172.21.15.86:0/1015105339 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11ac0d1c20 0x7f11ac0ae270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:57.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.866+0000 7f11aaa00640 1 -- 172.21.15.86:0/1015105339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11ac06f270 msgr2=0x7f11ac15a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:57.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.866+0000 7f11aaa00640 1 --2- 172.21.15.86:0/1015105339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11ac06f270 0x7f11ac15a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:57.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.866+0000 7f11aaa00640 1 -- 172.21.15.86:0/1015105339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11ac0cfaf0 con 0x7f11ac070330 2024-04-25T14:09:57.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.870+0000 7f11ab400640 1 --2- 172.21.15.86:0/1015105339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11ac06f270 0x7f11ac15a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:57.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.870+0000 7f11abe00640 1 --2- 172.21.15.86:0/1015105339 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11ac0d1c20 0x7f11ac0ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:57.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.870+0000 7f11aaa00640 1 --2- 172.21.15.86:0/1015105339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11ac070330 0x7f11ac073bc0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f119801ab60 tx=0x7f1198019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:57.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.870+0000 7f11a3400640 1 -- 172.21.15.86:0/1015105339 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11980ba020 con 0x7f11ac070330 2024-04-25T14:09:57.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.870+0000 7f11a3400640 1 -- 172.21.15.86:0/1015105339 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11980b6620 con 0x7f11ac070330 2024-04-25T14:09:57.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.870+0000 7f11b2800640 1 -- 172.21.15.86:0/1015105339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11ac0cfdd0 con 0x7f11ac070330 2024-04-25T14:09:57.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.870+0000 7f11a3400640 1 -- 172.21.15.86:0/1015105339 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11980b0040 con 0x7f11ac070330 2024-04-25T14:09:57.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.870+0000 7f11b2800640 1 -- 172.21.15.86:0/1015105339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11ac0736a0 con 0x7f11ac070330 2024-04-25T14:09:57.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.870+0000 7f11b2800640 1 -- 172.21.15.86:0/1015105339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11ac069d00 con 0x7f11ac070330 2024-04-25T14:09:57.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.870+0000 7f11a3400640 1 -- 172.21.15.86:0/1015105339 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7f11980bd020 con 0x7f11ac070330 2024-04-25T14:09:57.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.870+0000 7f11a3400640 1 --2- 172.21.15.86:0/1015105339 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f118c06f520 0x7f118c071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:57.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.874+0000 7f11a3400640 1 -- 172.21.15.86:0/1015105339 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5980+0+0 (secure 0 0 0) 0x7f1198129ec0 con 0x7f11ac070330 2024-04-25T14:09:57.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.874+0000 7f11ab400640 1 --2- 172.21.15.86:0/1015105339 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f118c06f520 0x7f118c071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:57.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.874+0000 7f11ab400640 1 --2- 172.21.15.86:0/1015105339 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f118c06f520 0x7f118c071a30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f11ac072200 tx=0x7f119c00b040 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:57.874+0000 7f11a3400640 1 -- 172.21.15.86:0/1015105339 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1198016130 con 0x7f11ac070330 2024-04-25T14:09:58.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.034+0000 7f11b2800640 1 -- 172.21.15.86:0/1015105339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11ac0cf880 con 0x7f11ac070330 2024-04-25T14:09:58.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.034+0000 7f11a3400640 1 -- 172.21.15.86:0/1015105339 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+14233 (secure 0 0 0) 0x7f11980f6af0 con 0x7f11ac070330 2024-04-25T14:09:58.032 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:09:58.032 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":105,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:09:56.680658+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.155:6835/1049360040":"2024-04-26T14:09:53.673272+0000","172.21.15.86:0/3838196444":"2024-04-26T14:09:49.448138+0000","172.21.15.86:6833/436174229":"2024-04-26T14:09:49.448138+0000","172.21.15.86:0/129198987":"2024-04-26T14:09:49.448138+0000","172.21.15.86:0/2887621339":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/2356435633":"2024-04-26T14:09:49.448138+0000","172.21.15.86:6832/436174229":"2024-04-26T14:09:49.448138+0000","172.21.15.86:0/361929172":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/2077676736":"2024-04-26T14:09:49.448138+0000","172.21.15.86:6833/4236874185":"2024-04-26T14:09:43.313864+0000","172.21.15.86:6832/4236874185":"2024-04-26T14:09:43.313864+0000","172.21.15.155:6833/1049360040":"2024-04-26T14:09:53.673272+0000","172.21.15.86:0/746145476":"2024-04-26T14:09:49.448138+0000","172.21.15.86:0/789182356":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/1532315242":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/1775030266":"2024-04-26T14:09:43.313864+0000","172.21.15.86:0/2226990349":"2024-04-26T14:09:43.313864+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:09:58.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.038+0000 7f11b2800640 1 -- 172.21.15.86:0/1015105339 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f118c06f520 msgr2=0x7f118c071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:58.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.038+0000 7f11b2800640 1 --2- 172.21.15.86:0/1015105339 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f118c06f520 0x7f118c071a30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f11ac072200 tx=0x7f119c00b040 comp rx=0 tx=0).stop 2024-04-25T14:09:58.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.038+0000 7f11b2800640 1 -- 172.21.15.86:0/1015105339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11ac070330 msgr2=0x7f11ac073bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:58.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.038+0000 7f11b2800640 1 --2- 172.21.15.86:0/1015105339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11ac070330 0x7f11ac073bc0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f119801ab60 tx=0x7f1198019250 comp rx=0 tx=0).stop 2024-04-25T14:09:58.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.038+0000 7f11b2800640 1 -- 172.21.15.86:0/1015105339 shutdown_connections 2024-04-25T14:09:58.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.038+0000 7f11b2800640 1 --2- 172.21.15.86:0/1015105339 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f118c06f520 0x7f118c071a30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:58.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.038+0000 7f11b2800640 1 --2- 172.21.15.86:0/1015105339 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11ac0d1c20 0x7f11ac0ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:58.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.038+0000 7f11b2800640 1 --2- 172.21.15.86:0/1015105339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11ac070330 0x7f11ac073bc0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:58.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.038+0000 7f11b2800640 1 --2- 172.21.15.86:0/1015105339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11ac06f270 0x7f11ac15a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:58.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.038+0000 7f11b2800640 1 -- 172.21.15.86:0/1015105339 >> 172.21.15.86:0/1015105339 conn(0x7f11ac0b4650 msgr2=0x7f11ac0b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:58.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.038+0000 7f11b2800640 1 -- 172.21.15.86:0/1015105339 shutdown_connections 2024-04-25T14:09:58.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.038+0000 7f11b2800640 1 -- 172.21.15.86:0/1015105339 wait complete. 2024-04-25T14:09:58.054 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:09:58.055 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:09:58.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.206+0000 7fe2afa00640 1 Processor -- start 2024-04-25T14:09:58.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.206+0000 7fe2afa00640 1 -- start start 2024-04-25T14:09:58.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.210+0000 7fe2afa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe2a80d16e0 0x7fe2a80d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:58.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.210+0000 7fe2afa00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2a80d2030 con 0x7fe2a80730b0 2024-04-25T14:09:58.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.210+0000 7fe2afa00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2a8072bd0 con 0x7fe2a80cfca0 2024-04-25T14:09:58.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.210+0000 7fe2afa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe2a8070470 con 0x7fe2a80d16e0 2024-04-25T14:09:58.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.210+0000 7fe2a7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe2a80d16e0 0x7fe2a80d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:58.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.210+0000 7fe2a7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe2a80d16e0 0x7fe2a80d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43786/0 (socket says 172.21.15.86:43786) 2024-04-25T14:09:58.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.210+0000 7fe2a7e00640 1 -- 172.21.15.86:0/1143065715 learned_addr learned my addr 172.21.15.86:0/1143065715 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:58.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.210+0000 7fe2a7e00640 1 -- 172.21.15.86:0/1143065715 >> v1:172.21.15.155:6789/0 conn(0x7fe2a80cfca0 legacy=0x7fe2a806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:58.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.210+0000 7fe2a7e00640 1 -- 172.21.15.86:0/1143065715 >> v1:172.21.15.86:6789/0 conn(0x7fe2a80730b0 legacy=0x7fe2a80734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:58.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.210+0000 7fe2a7e00640 1 -- 172.21.15.86:0/1143065715 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a806fbd0 con 0x7fe2a80d16e0 2024-04-25T14:09:58.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.214+0000 7fe2a7e00640 1 --2- 172.21.15.86:0/1143065715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a80d16e0 0x7fe2a80d1af0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fe29400bb10 tx=0x7fe29401ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33417359cef31de4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:58.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.214+0000 7fe2a7400640 1 -- 172.21.15.86:0/1143065715 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2940b4070 con 0x7fe2a80d16e0 2024-04-25T14:09:58.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.214+0000 7fe2a7400640 1 -- 172.21.15.86:0/1143065715 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2940a7bd0 con 0x7fe2a80d16e0 2024-04-25T14:09:58.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.214+0000 7fe2a7400640 1 -- 172.21.15.86:0/1143065715 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a80743e0 con 0x7fe2a80d16e0 2024-04-25T14:09:58.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.214+0000 7fe2afa00640 1 -- 172.21.15.86:0/1143065715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a80d16e0 msgr2=0x7fe2a80d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:58.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.214+0000 7fe2afa00640 1 --2- 172.21.15.86:0/1143065715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a80d16e0 0x7fe2a80d1af0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fe29400bb10 tx=0x7fe29401ebf0 comp rx=0 tx=0).stop 2024-04-25T14:09:58.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.214+0000 7fe2afa00640 1 -- 172.21.15.86:0/1143065715 shutdown_connections 2024-04-25T14:09:58.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.214+0000 7fe2afa00640 1 --2- 172.21.15.86:0/1143065715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a80d16e0 0x7fe2a80d1af0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:58.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.214+0000 7fe2afa00640 1 -- 172.21.15.86:0/1143065715 >> 172.21.15.86:0/1143065715 conn(0x7fe2a80b4650 msgr2=0x7fe2a80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:58.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.214+0000 7fe2afa00640 1 -- 172.21.15.86:0/1143065715 shutdown_connections 2024-04-25T14:09:58.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.214+0000 7fe2afa00640 1 -- 172.21.15.86:0/1143065715 wait complete. 2024-04-25T14:09:58.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.218+0000 7fe2afa00640 1 Processor -- start 2024-04-25T14:09:58.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.218+0000 7fe2afa00640 1 -- start start 2024-04-25T14:09:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.218+0000 7fe2afa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2a80730b0 0x7fe2a8159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.218+0000 7fe2afa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2a80cfca0 0x7fe2a815d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.218+0000 7fe2afa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a80d16e0 0x7fe2a809a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.218+0000 7fe2afa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a80ce1b0 con 0x7fe2a80d16e0 2024-04-25T14:09:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.218+0000 7fe2afa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a80ce030 con 0x7fe2a80730b0 2024-04-25T14:09:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.218+0000 7fe2afa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe2a80ce330 con 0x7fe2a80cfca0 2024-04-25T14:09:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.218+0000 7fe2ada00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a80d16e0 0x7fe2a809a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.218+0000 7fe2ad000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2a80730b0 0x7fe2a8159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.218+0000 7fe2a7e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2a80cfca0 0x7fe2a815d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.218+0000 7fe2ad000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2a80730b0 0x7fe2a8159a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:35370/0 (socket says 172.21.15.86:35370) 2024-04-25T14:09:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.218+0000 7fe2ad000640 1 -- 172.21.15.86:0/1156615781 learned_addr learned my addr 172.21.15.86:0/1156615781 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.218+0000 7fe2ada00640 1 --2- 172.21.15.86:0/1156615781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a80d16e0 0x7fe2a809a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43788/0 (socket says 172.21.15.86:43788) 2024-04-25T14:09:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.222+0000 7fe2ada00640 1 -- 172.21.15.86:0/1156615781 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2a80cfca0 msgr2=0x7fe2a815d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.222+0000 7fe2ada00640 1 --2- 172.21.15.86:0/1156615781 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2a80cfca0 0x7fe2a815d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.222+0000 7fe2ada00640 1 -- 172.21.15.86:0/1156615781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2a80730b0 msgr2=0x7fe2a8159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.222+0000 7fe2ada00640 1 --2- 172.21.15.86:0/1156615781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2a80730b0 0x7fe2a8159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.222+0000 7fe2ada00640 1 -- 172.21.15.86:0/1156615781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a809adb0 con 0x7fe2a80d16e0 2024-04-25T14:09:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.222+0000 7fe2ada00640 1 --2- 172.21.15.86:0/1156615781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a80d16e0 0x7fe2a809a870 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8070c30 tx=0x7fe2a000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.222+0000 7fe2a4c00640 1 -- 172.21.15.86:0/1156615781 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0003ba0 con 0x7fe2a80d16e0 2024-04-25T14:09:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.222+0000 7fe2a4c00640 1 -- 172.21.15.86:0/1156615781 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2a0003d40 con 0x7fe2a80d16e0 2024-04-25T14:09:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.222+0000 7fe2afa00640 1 -- 172.21.15.86:0/1156615781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a80aec70 con 0x7fe2a80d16e0 2024-04-25T14:09:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.222+0000 7fe2a4c00640 1 -- 172.21.15.86:0/1156615781 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0005310 con 0x7fe2a80d16e0 2024-04-25T14:09:58.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.222+0000 7fe2afa00640 1 -- 172.21.15.86:0/1156615781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a80af230 con 0x7fe2a80d16e0 2024-04-25T14:09:58.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.222+0000 7fe2afa00640 1 -- 172.21.15.86:0/1156615781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2a815b930 con 0x7fe2a80d16e0 2024-04-25T14:09:58.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.222+0000 7fe2a4c00640 1 -- 172.21.15.86:0/1156615781 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7fe2a00054b0 con 0x7fe2a80d16e0 2024-04-25T14:09:58.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.226+0000 7fe2a4c00640 1 --2- 172.21.15.86:0/1156615781 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fe28806f550 0x7fe288071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:58.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.226+0000 7fe2a4c00640 1 -- 172.21.15.86:0/1156615781 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5980+0+0 (secure 0 0 0) 0x7fe2a008ef30 con 0x7fe2a80d16e0 2024-04-25T14:09:58.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.226+0000 7fe2ad000640 1 --2- 172.21.15.86:0/1156615781 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fe28806f550 0x7fe288071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:58.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.226+0000 7fe2ad000640 1 --2- 172.21.15.86:0/1156615781 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fe28806f550 0x7fe288071a60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe29c009980 tx=0x7fe29c0090e0 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:58.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.230+0000 7fe2a4c00640 1 -- 172.21.15.86:0/1156615781 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe2a0020020 con 0x7fe2a80d16e0 2024-04-25T14:09:58.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.386+0000 7fe2afa00640 1 -- 172.21.15.86:0/1156615781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe2a809f4b0 con 0x7fe2a80d16e0 2024-04-25T14:09:58.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.386+0000 7fe2a4c00640 1 -- 172.21.15.86:0/1156615781 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7fe2a80d2030 con 0x7fe2a80d16e0 2024-04-25T14:09:58.383 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:09:58.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.390+0000 7fe2afa00640 1 -- 172.21.15.86:0/1156615781 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fe28806f550 msgr2=0x7fe288071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:58.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.390+0000 7fe2afa00640 1 --2- 172.21.15.86:0/1156615781 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fe28806f550 0x7fe288071a60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe29c009980 tx=0x7fe29c0090e0 comp rx=0 tx=0).stop 2024-04-25T14:09:58.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.390+0000 7fe2afa00640 1 -- 172.21.15.86:0/1156615781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a80d16e0 msgr2=0x7fe2a809a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:58.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.390+0000 7fe2afa00640 1 --2- 172.21.15.86:0/1156615781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a80d16e0 0x7fe2a809a870 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8070c30 tx=0x7fe2a000cb90 comp rx=0 tx=0).stop 2024-04-25T14:09:58.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.390+0000 7fe2afa00640 1 -- 172.21.15.86:0/1156615781 shutdown_connections 2024-04-25T14:09:58.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.390+0000 7fe2afa00640 1 --2- 172.21.15.86:0/1156615781 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fe28806f550 0x7fe288071a60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:58.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.390+0000 7fe2afa00640 1 --2- 172.21.15.86:0/1156615781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a80d16e0 0x7fe2a809a870 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:58.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.390+0000 7fe2afa00640 1 --2- 172.21.15.86:0/1156615781 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2a80cfca0 0x7fe2a815d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:58.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.390+0000 7fe2afa00640 1 --2- 172.21.15.86:0/1156615781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2a80730b0 0x7fe2a8159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:58.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.390+0000 7fe2afa00640 1 -- 172.21.15.86:0/1156615781 >> 172.21.15.86:0/1156615781 conn(0x7fe2a80b4650 msgr2=0x7fe2a80b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:58.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.390+0000 7fe2afa00640 1 -- 172.21.15.86:0/1156615781 shutdown_connections 2024-04-25T14:09:58.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.390+0000 7fe2afa00640 1 -- 172.21.15.86:0/1156615781 wait complete. 2024-04-25T14:09:58.405 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_for_invalid_mds_rank' 2024-04-25T14:09:58.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.554+0000 7feb6d400640 1 Processor -- start 2024-04-25T14:09:58.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.554+0000 7feb6d400640 1 -- start start 2024-04-25T14:09:58.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.558+0000 7feb6d400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7feb680d16e0 0x7feb680d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:58.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.558+0000 7feb6d400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb680730b0 0x7feb680734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:58.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.558+0000 7feb6d400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb680d2030 con 0x7feb680cfca0 2024-04-25T14:09:58.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.558+0000 7feb6d400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7feb68070470 con 0x7feb680d16e0 2024-04-25T14:09:58.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.558+0000 7feb6d400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7feb68072bd0 con 0x7feb680730b0 2024-04-25T14:09:58.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.558+0000 7feb67200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb680730b0 0x7feb680734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:58.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.558+0000 7feb5de00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7feb680d16e0 0x7feb680d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:58.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.558+0000 7feb66800640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7feb680cfca0 0x7feb6806f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:53548/0 (socket says 172.21.15.86:53548) 2024-04-25T14:09:58.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.558+0000 7feb66800640 1 -- 172.21.15.86:0/35467224 learned_addr learned my addr 172.21.15.86:0/35467224 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:58.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.558+0000 7feb65e00640 1 -- 172.21.15.86:0/35467224 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4153455419 0 0) 0x7feb580014a0 con 0x7feb680cfca0 2024-04-25T14:09:58.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.558+0000 7feb65e00640 1 -- 172.21.15.86:0/35467224 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb4c003530 con 0x7feb680cfca0 2024-04-25T14:09:58.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.558+0000 7feb67200640 1 -- 172.21.15.86:0/35467224 >> v2:172.21.15.155:3300/0 conn(0x7feb680d16e0 msgr2=0x7feb680d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:58.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.558+0000 7feb67200640 1 --2- 172.21.15.86:0/35467224 >> v2:172.21.15.155:3300/0 conn(0x7feb680d16e0 0x7feb680d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:58.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.558+0000 7feb67200640 1 -- 172.21.15.86:0/35467224 >> v1:172.21.15.155:6789/0 conn(0x7feb680cfca0 legacy=0x7feb6806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:58.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.558+0000 7feb67200640 1 -- 172.21.15.86:0/35467224 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6806fbd0 con 0x7feb680730b0 2024-04-25T14:09:58.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.562+0000 7feb67200640 1 --2- 172.21.15.86:0/35467224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb680730b0 0x7feb680734c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7feb6000ba60 tx=0x7feb6001eb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ece6b05686ef01a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:58.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.562+0000 7feb65e00640 1 -- 172.21.15.86:0/35467224 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb600b4070 con 0x7feb680730b0 2024-04-25T14:09:58.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.562+0000 7feb65e00640 1 -- 172.21.15.86:0/35467224 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb600a7b50 con 0x7feb680730b0 2024-04-25T14:09:58.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.562+0000 7feb65e00640 1 -- 172.21.15.86:0/35467224 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb600af440 con 0x7feb680730b0 2024-04-25T14:09:58.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.562+0000 7feb6d400640 1 -- 172.21.15.86:0/35467224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb680730b0 msgr2=0x7feb680734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:58.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.562+0000 7feb6d400640 1 --2- 172.21.15.86:0/35467224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb680730b0 0x7feb680734c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7feb6000ba60 tx=0x7feb6001eb70 comp rx=0 tx=0).stop 2024-04-25T14:09:58.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.562+0000 7feb6d400640 1 -- 172.21.15.86:0/35467224 shutdown_connections 2024-04-25T14:09:58.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.562+0000 7feb6d400640 1 --2- 172.21.15.86:0/35467224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb680730b0 0x7feb680734c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:58.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.562+0000 7feb6d400640 1 --2- 172.21.15.86:0/35467224 >> v2:172.21.15.155:3300/0 conn(0x7feb680d16e0 0x7feb680d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:58.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.562+0000 7feb6d400640 1 -- 172.21.15.86:0/35467224 >> 172.21.15.86:0/35467224 conn(0x7feb680b4650 msgr2=0x7feb680b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:58.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.562+0000 7feb6d400640 1 -- 172.21.15.86:0/35467224 shutdown_connections 2024-04-25T14:09:58.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.562+0000 7feb6d400640 1 -- 172.21.15.86:0/35467224 wait complete. 2024-04-25T14:09:58.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.562+0000 7feb6d400640 1 Processor -- start 2024-04-25T14:09:58.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.562+0000 7feb6d400640 1 -- start start 2024-04-25T14:09:58.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.566+0000 7feb6d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb680730b0 0x7feb681550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:58.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.566+0000 7feb6d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb680cfca0 0x7feb681fdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:58.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.566+0000 7feb6d400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb680d16e0 0x7feb680b1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:58.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.566+0000 7feb6d400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb680ce130 con 0x7feb680730b0 2024-04-25T14:09:58.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.566+0000 7feb6d400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7feb680cdfb0 con 0x7feb680cfca0 2024-04-25T14:09:58.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.566+0000 7feb6d400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7feb680ce2b0 con 0x7feb680d16e0 2024-04-25T14:09:58.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.566+0000 7feb66800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb680730b0 0x7feb681550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:58.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.566+0000 7feb66800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb680730b0 0x7feb681550a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43790/0 (socket says 172.21.15.86:43790) 2024-04-25T14:09:58.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.566+0000 7feb66800640 1 -- 172.21.15.86:0/1517568225 learned_addr learned my addr 172.21.15.86:0/1517568225 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:58.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.566+0000 7feb67200640 1 --2- 172.21.15.86:0/1517568225 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb680d16e0 0x7feb680b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:58.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.566+0000 7feb5de00640 1 --2- 172.21.15.86:0/1517568225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb680cfca0 0x7feb681fdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:58.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.566+0000 7feb66800640 1 -- 172.21.15.86:0/1517568225 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb680d16e0 msgr2=0x7feb680b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:58.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.566+0000 7feb66800640 1 --2- 172.21.15.86:0/1517568225 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb680d16e0 0x7feb680b1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:58.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.566+0000 7feb66800640 1 -- 172.21.15.86:0/1517568225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb680cfca0 msgr2=0x7feb681fdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:58.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.566+0000 7feb66800640 1 --2- 172.21.15.86:0/1517568225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb680cfca0 0x7feb681fdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:58.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.566+0000 7feb66800640 1 -- 172.21.15.86:0/1517568225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6815b820 con 0x7feb680730b0 2024-04-25T14:09:58.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.566+0000 7feb5de00640 1 --2- 172.21.15.86:0/1517568225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb680cfca0 0x7feb681fdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:58.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.566+0000 7feb66800640 1 --2- 172.21.15.86:0/1517568225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb680730b0 0x7feb681550a0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7feb5800caa0 tx=0x7feb5800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:58.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.566+0000 7feb5f400640 1 -- 172.21.15.86:0/1517568225 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58017070 con 0x7feb680730b0 2024-04-25T14:09:58.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.566+0000 7feb5f400640 1 -- 172.21.15.86:0/1517568225 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb5800ac00 con 0x7feb680730b0 2024-04-25T14:09:58.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.566+0000 7feb5f400640 1 -- 172.21.15.86:0/1517568225 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58012400 con 0x7feb680730b0 2024-04-25T14:09:58.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.566+0000 7feb6d400640 1 -- 172.21.15.86:0/1517568225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6815bb00 con 0x7feb680730b0 2024-04-25T14:09:58.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.566+0000 7feb6d400640 1 -- 172.21.15.86:0/1517568225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb6815c0c0 con 0x7feb680730b0 2024-04-25T14:09:58.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.566+0000 7feb6d400640 1 -- 172.21.15.86:0/1517568225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb6809f4b0 con 0x7feb680730b0 2024-04-25T14:09:58.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.570+0000 7feb5f400640 1 -- 172.21.15.86:0/1517568225 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7feb5801a070 con 0x7feb680730b0 2024-04-25T14:09:58.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.570+0000 7feb5f400640 1 --2- 172.21.15.86:0/1517568225 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7feb4406f550 0x7feb44071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:58.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.570+0000 7feb5f400640 1 -- 172.21.15.86:0/1517568225 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5980+0+0 (secure 0 0 0) 0x7feb5808f2b0 con 0x7feb680730b0 2024-04-25T14:09:58.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.570+0000 7feb5de00640 1 --2- 172.21.15.86:0/1517568225 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7feb4406f550 0x7feb44071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:58.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.570+0000 7feb5de00640 1 --2- 172.21.15.86:0/1517568225 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7feb4406f550 0x7feb44071a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feb68160150 tx=0x7feb50008040 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:58.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.574+0000 7feb5f400640 1 -- 172.21.15.86:0/1517568225 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7feb680d2030 con 0x7feb680730b0 2024-04-25T14:09:58.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:58.786+0000 7feb6d400640 1 -- 172.21.15.86:0/1517568225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_for_invalid_mds_rank"]} v 0) -- 0x7feb68069d00 con 0x7feb680730b0 2024-04-25T14:09:59.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.690+0000 7feb5f400640 1 -- 172.21.15.86:0/1517568225 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_for_invalid_mds_rank"]}]=0 v352) ==== 152+0+0 (secure 0 0 0) 0x7feb5805b890 con 0x7feb680730b0 2024-04-25T14:09:59.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.694+0000 7feb6d400640 1 -- 172.21.15.86:0/1517568225 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7feb4406f550 msgr2=0x7feb44071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:59.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.694+0000 7feb6d400640 1 --2- 172.21.15.86:0/1517568225 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7feb4406f550 0x7feb44071a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feb68160150 tx=0x7feb50008040 comp rx=0 tx=0).stop 2024-04-25T14:09:59.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.694+0000 7feb6d400640 1 -- 172.21.15.86:0/1517568225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb680730b0 msgr2=0x7feb681550a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:59.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.694+0000 7feb6d400640 1 --2- 172.21.15.86:0/1517568225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb680730b0 0x7feb681550a0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7feb5800caa0 tx=0x7feb5800cf60 comp rx=0 tx=0).stop 2024-04-25T14:09:59.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.694+0000 7feb6d400640 1 -- 172.21.15.86:0/1517568225 shutdown_connections 2024-04-25T14:09:59.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.694+0000 7feb6d400640 1 --2- 172.21.15.86:0/1517568225 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7feb4406f550 0x7feb44071a60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:59.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.694+0000 7feb6d400640 1 --2- 172.21.15.86:0/1517568225 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb680d16e0 0x7feb680b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:59.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.694+0000 7feb6d400640 1 --2- 172.21.15.86:0/1517568225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb680cfca0 0x7feb681fdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:59.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.694+0000 7feb6d400640 1 --2- 172.21.15.86:0/1517568225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb680730b0 0x7feb681550a0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:59.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.694+0000 7feb6d400640 1 -- 172.21.15.86:0/1517568225 >> 172.21.15.86:0/1517568225 conn(0x7feb680b4650 msgr2=0x7feb68158100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:59.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.694+0000 7feb6d400640 1 -- 172.21.15.86:0/1517568225 shutdown_connections 2024-04-25T14:09:59.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.694+0000 7feb6d400640 1 -- 172.21.15.86:0/1517568225 wait complete. 2024-04-25T14:09:59.715 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T14:09:59.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.870+0000 7ff130c00640 1 Processor -- start 2024-04-25T14:09:59.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.870+0000 7ff130c00640 1 -- start start 2024-04-25T14:09:59.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.874+0000 7ff130c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff12c0cfca0 0x7ff12c06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:59.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.874+0000 7ff130c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff12c0730b0 0x7ff12c0734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:59.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.874+0000 7ff130c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff12c072bd0 con 0x7ff12c0d16e0 2024-04-25T14:09:59.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.874+0000 7ff130c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff12c070470 con 0x7ff12c0730b0 2024-04-25T14:09:59.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.874+0000 7ff130c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff12c06f930 con 0x7ff12c0cfca0 2024-04-25T14:09:59.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.874+0000 7ff12a800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff12c0730b0 0x7ff12c0734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:59.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.874+0000 7ff123e00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7ff12c0d16e0 0x7ff12c0d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:40634/0 (socket says 172.21.15.86:40634) 2024-04-25T14:09:59.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.874+0000 7ff123e00640 1 -- 172.21.15.86:0/3630955056 learned_addr learned my addr 172.21.15.86:0/3630955056 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:09:59.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.874+0000 7ff129e00640 1 --2- 172.21.15.86:0/3630955056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff12c0cfca0 0x7ff12c06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:59.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.874+0000 7ff129400640 1 -- 172.21.15.86:0/3630955056 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1246661667 0 0) 0x7ff12c072bd0 con 0x7ff12c0d16e0 2024-04-25T14:09:59.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.874+0000 7ff129400640 1 -- 172.21.15.86:0/3630955056 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff110003530 con 0x7ff12c0d16e0 2024-04-25T14:09:59.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.874+0000 7ff12a800640 1 -- 172.21.15.86:0/3630955056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff12c0cfca0 msgr2=0x7ff12c06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:59.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.874+0000 7ff12a800640 1 --2- 172.21.15.86:0/3630955056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff12c0cfca0 0x7ff12c06f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:59.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.874+0000 7ff12a800640 1 -- 172.21.15.86:0/3630955056 >> v1:172.21.15.155:6789/0 conn(0x7ff12c0d16e0 legacy=0x7ff12c0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:59.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.874+0000 7ff12a800640 1 -- 172.21.15.86:0/3630955056 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12c06fbd0 con 0x7ff12c0730b0 2024-04-25T14:09:59.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.874+0000 7ff129e00640 1 --2- 172.21.15.86:0/3630955056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff12c0cfca0 0x7ff12c06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:09:59.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.878+0000 7ff12a800640 1 --2- 172.21.15.86:0/3630955056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff12c0730b0 0x7ff12c0734c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff124009d90 tx=0x7ff12401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7ad5ffe904068c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:59.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.878+0000 7ff129400640 1 -- 172.21.15.86:0/3630955056 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1240b2070 con 0x7ff12c0730b0 2024-04-25T14:09:59.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.878+0000 7ff129400640 1 -- 172.21.15.86:0/3630955056 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1240a5940 con 0x7ff12c0730b0 2024-04-25T14:09:59.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.878+0000 7ff129400640 1 -- 172.21.15.86:0/3630955056 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1240a5d10 con 0x7ff12c0730b0 2024-04-25T14:09:59.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.878+0000 7ff130c00640 1 -- 172.21.15.86:0/3630955056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff12c0730b0 msgr2=0x7ff12c0734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:59.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.878+0000 7ff130c00640 1 --2- 172.21.15.86:0/3630955056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff12c0730b0 0x7ff12c0734c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff124009d90 tx=0x7ff12401c970 comp rx=0 tx=0).stop 2024-04-25T14:09:59.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.878+0000 7ff130c00640 1 -- 172.21.15.86:0/3630955056 shutdown_connections 2024-04-25T14:09:59.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.878+0000 7ff130c00640 1 --2- 172.21.15.86:0/3630955056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff12c0730b0 0x7ff12c0734c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:59.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.878+0000 7ff130c00640 1 --2- 172.21.15.86:0/3630955056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff12c0cfca0 0x7ff12c06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:59.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.878+0000 7ff130c00640 1 -- 172.21.15.86:0/3630955056 >> 172.21.15.86:0/3630955056 conn(0x7ff12c0b4650 msgr2=0x7ff12c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:59.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.878+0000 7ff130c00640 1 -- 172.21.15.86:0/3630955056 shutdown_connections 2024-04-25T14:09:59.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.878+0000 7ff130c00640 1 -- 172.21.15.86:0/3630955056 wait complete. 2024-04-25T14:09:59.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.882+0000 7ff130c00640 1 Processor -- start 2024-04-25T14:09:59.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.882+0000 7ff130c00640 1 -- start start 2024-04-25T14:09:59.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.882+0000 7ff130c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff12c0730b0 0x7ff12c1551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:59.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.882+0000 7ff130c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff12c0cfca0 0x7ff12c0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:59.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.882+0000 7ff130c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff12c0d16e0 0x7ff12c15d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:59.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.882+0000 7ff130c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c169a70 con 0x7ff12c0d16e0 2024-04-25T14:09:59.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.882+0000 7ff130c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c1698f0 con 0x7ff12c0cfca0 2024-04-25T14:09:59.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.882+0000 7ff130c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff12c169bf0 con 0x7ff12c0730b0 2024-04-25T14:09:59.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.882+0000 7ff12a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff12c0d16e0 0x7ff12c15d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:59.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.882+0000 7ff12a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff12c0d16e0 0x7ff12c15d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49424/0 (socket says 172.21.15.86:49424) 2024-04-25T14:09:59.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.882+0000 7ff12a800640 1 -- 172.21.15.86:0/1878789987 learned_addr learned my addr 172.21.15.86:0/1878789987 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:09:59.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.882+0000 7ff123e00640 1 --2- 172.21.15.86:0/1878789987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff12c0cfca0 0x7ff12c0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:59.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.882+0000 7ff129e00640 1 --2- 172.21.15.86:0/1878789987 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff12c0730b0 0x7ff12c1551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:59.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.882+0000 7ff12a800640 1 -- 172.21.15.86:0/1878789987 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff12c0730b0 msgr2=0x7ff12c1551a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:59.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.882+0000 7ff12a800640 1 --2- 172.21.15.86:0/1878789987 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff12c0730b0 0x7ff12c1551a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:59.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.882+0000 7ff12a800640 1 -- 172.21.15.86:0/1878789987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff12c0cfca0 msgr2=0x7ff12c0b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:59.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.882+0000 7ff12a800640 1 --2- 172.21.15.86:0/1878789987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff12c0cfca0 0x7ff12c0b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:59.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.882+0000 7ff12a800640 1 -- 172.21.15.86:0/1878789987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12c0cd200 con 0x7ff12c0d16e0 2024-04-25T14:09:59.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.882+0000 7ff12a800640 1 --2- 172.21.15.86:0/1878789987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff12c0d16e0 0x7ff12c15d300 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7ff1240b0040 tx=0x7ff124019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:59.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.886+0000 7ff122000640 1 -- 172.21.15.86:0/1878789987 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1240b2050 con 0x7ff12c0d16e0 2024-04-25T14:09:59.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.886+0000 7ff122000640 1 -- 172.21.15.86:0/1878789987 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1240be070 con 0x7ff12c0d16e0 2024-04-25T14:09:59.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.886+0000 7ff122000640 1 -- 172.21.15.86:0/1878789987 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1240ad850 con 0x7ff12c0d16e0 2024-04-25T14:09:59.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.886+0000 7ff130c00640 1 -- 172.21.15.86:0/1878789987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff12c0cd4e0 con 0x7ff12c0d16e0 2024-04-25T14:09:59.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.886+0000 7ff130c00640 1 -- 172.21.15.86:0/1878789987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff12c1556e0 con 0x7ff12c0d16e0 2024-04-25T14:09:59.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.886+0000 7ff130c00640 1 -- 172.21.15.86:0/1878789987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff12c0b2e60 con 0x7ff12c0d16e0 2024-04-25T14:09:59.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.886+0000 7ff122000640 1 -- 172.21.15.86:0/1878789987 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7ff1240ad9f0 con 0x7ff12c0d16e0 2024-04-25T14:09:59.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.886+0000 7ff122000640 1 --2- 172.21.15.86:0/1878789987 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7ff10806f550 0x7ff108071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:59.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.890+0000 7ff122000640 1 -- 172.21.15.86:0/1878789987 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5980+0+0 (secure 0 0 0) 0x7ff124129fa0 con 0x7ff12c0d16e0 2024-04-25T14:09:59.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.890+0000 7ff129e00640 1 --2- 172.21.15.86:0/1878789987 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7ff10806f550 0x7ff108071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:59.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.890+0000 7ff129e00640 1 --2- 172.21.15.86:0/1878789987 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7ff10806f550 0x7ff108071a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff11c009580 tx=0x7ff11c009190 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:59.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:09:59.894+0000 7ff122000640 1 -- 172.21.15.86:0/1878789987 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff124016560 con 0x7ff12c0d16e0 2024-04-25T14:10:00.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.050+0000 7ff130c00640 1 -- 172.21.15.86:0/1878789987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff12c069d00 con 0x7ff12c0d16e0 2024-04-25T14:10:00.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.054+0000 7ff122000640 1 -- 172.21.15.86:0/1878789987 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7ff1240f6610 con 0x7ff12c0d16e0 2024-04-25T14:10:00.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.054+0000 7ff130c00640 1 -- 172.21.15.86:0/1878789987 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7ff10806f550 msgr2=0x7ff108071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:00.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.054+0000 7ff130c00640 1 --2- 172.21.15.86:0/1878789987 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7ff10806f550 0x7ff108071a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff11c009580 tx=0x7ff11c009190 comp rx=0 tx=0).stop 2024-04-25T14:10:00.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.054+0000 7ff130c00640 1 -- 172.21.15.86:0/1878789987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff12c0d16e0 msgr2=0x7ff12c15d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:00.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.054+0000 7ff130c00640 1 --2- 172.21.15.86:0/1878789987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff12c0d16e0 0x7ff12c15d300 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7ff1240b0040 tx=0x7ff124019710 comp rx=0 tx=0).stop 2024-04-25T14:10:00.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.054+0000 7ff130c00640 1 -- 172.21.15.86:0/1878789987 shutdown_connections 2024-04-25T14:10:00.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.054+0000 7ff130c00640 1 --2- 172.21.15.86:0/1878789987 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7ff10806f550 0x7ff108071a60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:00.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.054+0000 7ff130c00640 1 --2- 172.21.15.86:0/1878789987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff12c0d16e0 0x7ff12c15d300 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:00.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.054+0000 7ff130c00640 1 --2- 172.21.15.86:0/1878789987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff12c0cfca0 0x7ff12c0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:00.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.054+0000 7ff130c00640 1 --2- 172.21.15.86:0/1878789987 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff12c0730b0 0x7ff12c1551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:00.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.054+0000 7ff130c00640 1 -- 172.21.15.86:0/1878789987 >> 172.21.15.86:0/1878789987 conn(0x7ff12c0b4650 msgr2=0x7ff12c1560e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:00.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.054+0000 7ff130c00640 1 -- 172.21.15.86:0/1878789987 shutdown_connections 2024-04-25T14:10:00.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.054+0000 7ff130c00640 1 -- 172.21.15.86:0/1878789987 wait complete. 2024-04-25T14:10:00.072 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T14:10:00.072 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:10:00.124 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:10:00.180 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T14:10:00.180 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:10:00.180 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:10:00.189 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2893 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T14:10:00.189 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:10:00.189 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:10:00.198 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4517 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T14:10:00.198 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:10:00.198 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:10:00.240 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2893 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T14:10:00.240 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:10:00.240 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:10:00.249 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4517 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T14:10:00.249 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T14:10:00.250 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:10:00.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.430+0000 7f3cd0c00640 1 Processor -- start 2024-04-25T14:10:00.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.430+0000 7f3cd0c00640 1 -- start start 2024-04-25T14:10:00.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.430+0000 7f3cd0c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3ccc0ce7a0 0x7f3ccc0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:00.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.430+0000 7f3cd0c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ccc0d1f60 0x7f3ccc0d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:00.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.430+0000 7f3cd0c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ccc0cf0f0 con 0x7f3ccc0d28b0 2024-04-25T14:10:00.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.430+0000 7f3cd0c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3ccc0cdfe0 con 0x7f3ccc0ce7a0 2024-04-25T14:10:00.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.430+0000 7f3cd0c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3ccc0d14c0 con 0x7f3ccc0d1f60 2024-04-25T14:10:00.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.430+0000 7f3ccbe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3ccc0d28b0 0x7f3ccc0cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41822/0 (socket says 172.21.15.86:41822) 2024-04-25T14:10:00.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.430+0000 7f3ccbe00640 1 -- 172.21.15.86:0/3514486259 learned_addr learned my addr 172.21.15.86:0/3514486259 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:00.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.430+0000 7f3cc2a00640 1 --2- 172.21.15.86:0/3514486259 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ccc0d1f60 0x7f3ccc0d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:00.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.430+0000 7f3ccb400640 1 --2- 172.21.15.86:0/3514486259 >> v2:172.21.15.155:3300/0 conn(0x7f3ccc0ce7a0 0x7f3ccc0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:00.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.430+0000 7f3ccaa00640 1 -- 172.21.15.86:0/3514486259 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 787487079 0 0) 0x7f3cc40014a0 con 0x7f3ccc0d28b0 2024-04-25T14:10:00.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.430+0000 7f3ccaa00640 1 -- 172.21.15.86:0/3514486259 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3cb0003530 con 0x7f3ccc0d28b0 2024-04-25T14:10:00.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.430+0000 7f3ccaa00640 1 -- 172.21.15.86:0/3514486259 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3976445580 0 0) 0x7f3cb0003530 con 0x7f3ccc0d28b0 2024-04-25T14:10:00.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.430+0000 7f3cc2a00640 1 -- 172.21.15.86:0/3514486259 >> v2:172.21.15.155:3300/0 conn(0x7f3ccc0ce7a0 msgr2=0x7f3ccc0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:00.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.430+0000 7f3cc2a00640 1 --2- 172.21.15.86:0/3514486259 >> v2:172.21.15.155:3300/0 conn(0x7f3ccc0ce7a0 0x7f3ccc0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:00.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.430+0000 7f3cc2a00640 1 -- 172.21.15.86:0/3514486259 >> v1:172.21.15.86:6789/0 conn(0x7f3ccc0d28b0 legacy=0x7f3ccc0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:00.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.430+0000 7f3cc2a00640 1 -- 172.21.15.86:0/3514486259 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ccc0ce220 con 0x7f3ccc0d1f60 2024-04-25T14:10:00.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.434+0000 7f3cc2a00640 1 --2- 172.21.15.86:0/3514486259 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ccc0d1f60 0x7f3ccc0d2370 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3cb40097c0 tx=0x7f3cb401c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1fba1d0568d8d97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:00.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.434+0000 7f3ccaa00640 1 -- 172.21.15.86:0/3514486259 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb40b0070 con 0x7f3ccc0d1f60 2024-04-25T14:10:00.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.434+0000 7f3ccaa00640 1 -- 172.21.15.86:0/3514486259 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cb40ab400 con 0x7f3ccc0d1f60 2024-04-25T14:10:00.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.434+0000 7f3ccaa00640 1 -- 172.21.15.86:0/3514486259 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4001bd0 con 0x7f3ccc0d1f60 2024-04-25T14:10:00.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.434+0000 7f3cd0c00640 1 -- 172.21.15.86:0/3514486259 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ccc0d1f60 msgr2=0x7f3ccc0d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:00.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.434+0000 7f3cd0c00640 1 --2- 172.21.15.86:0/3514486259 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ccc0d1f60 0x7f3ccc0d2370 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3cb40097c0 tx=0x7f3cb401c3c0 comp rx=0 tx=0).stop 2024-04-25T14:10:00.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.434+0000 7f3cd0c00640 1 -- 172.21.15.86:0/3514486259 shutdown_connections 2024-04-25T14:10:00.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.434+0000 7f3cd0c00640 1 --2- 172.21.15.86:0/3514486259 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ccc0d1f60 0x7f3ccc0d2370 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:00.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.434+0000 7f3cd0c00640 1 --2- 172.21.15.86:0/3514486259 >> v2:172.21.15.155:3300/0 conn(0x7f3ccc0ce7a0 0x7f3ccc0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:00.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.434+0000 7f3cd0c00640 1 -- 172.21.15.86:0/3514486259 >> 172.21.15.86:0/3514486259 conn(0x7f3ccc09ba30 msgr2=0x7f3ccc0b70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:00.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.434+0000 7f3cd0c00640 1 -- 172.21.15.86:0/3514486259 shutdown_connections 2024-04-25T14:10:00.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.434+0000 7f3cd0c00640 1 -- 172.21.15.86:0/3514486259 wait complete. 2024-04-25T14:10:00.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.438+0000 7f3cd0c00640 1 Processor -- start 2024-04-25T14:10:00.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.438+0000 7f3cd0c00640 1 -- start start 2024-04-25T14:10:00.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.438+0000 7f3cd0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ccc0ce7a0 0x7f3ccc14bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:00.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.438+0000 7f3cd0c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ccc0d1f60 0x7f3ccc15bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:00.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.438+0000 7f3cd0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ccc0d28b0 0x7f3ccc15f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:00.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.438+0000 7f3cd0c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ccc0b14e0 con 0x7f3ccc0d28b0 2024-04-25T14:10:00.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.438+0000 7f3cd0c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3ccc0b1360 con 0x7f3ccc0ce7a0 2024-04-25T14:10:00.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.438+0000 7f3cd0c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3ccc0d2cc0 con 0x7f3ccc0d1f60 2024-04-25T14:10:00.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.438+0000 7f3ccbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ccc0d28b0 0x7f3ccc15f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:00.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.438+0000 7f3ccbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ccc0d28b0 0x7f3ccc15f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49428/0 (socket says 172.21.15.86:49428) 2024-04-25T14:10:00.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.438+0000 7f3ccbe00640 1 -- 172.21.15.86:0/1605527856 learned_addr learned my addr 172.21.15.86:0/1605527856 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:00.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.438+0000 7f3cc2a00640 1 --2- 172.21.15.86:0/1605527856 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ccc0d1f60 0x7f3ccc15bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:00.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.438+0000 7f3ccb400640 1 --2- 172.21.15.86:0/1605527856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ccc0ce7a0 0x7f3ccc14bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:00.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.442+0000 7f3ccbe00640 1 -- 172.21.15.86:0/1605527856 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ccc0d1f60 msgr2=0x7f3ccc15bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:00.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.442+0000 7f3ccbe00640 1 --2- 172.21.15.86:0/1605527856 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ccc0d1f60 0x7f3ccc15bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:00.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.442+0000 7f3ccbe00640 1 -- 172.21.15.86:0/1605527856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ccc0ce7a0 msgr2=0x7f3ccc14bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:00.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.442+0000 7f3ccbe00640 1 --2- 172.21.15.86:0/1605527856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ccc0ce7a0 0x7f3ccc14bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:00.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.442+0000 7f3ccbe00640 1 -- 172.21.15.86:0/1605527856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ccc158360 con 0x7f3ccc0d28b0 2024-04-25T14:10:00.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.442+0000 7f3ccbe00640 1 --2- 172.21.15.86:0/1605527856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ccc0d28b0 0x7f3ccc15f490 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f3cc400cc40 tx=0x7f3cc400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:00.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.442+0000 7f3cc3e00640 1 -- 172.21.15.86:0/1605527856 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc400ce50 con 0x7f3ccc0d28b0 2024-04-25T14:10:00.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.442+0000 7f3cc3e00640 1 -- 172.21.15.86:0/1605527856 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cc4012400 con 0x7f3ccc0d28b0 2024-04-25T14:10:00.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.442+0000 7f3cd0c00640 1 -- 172.21.15.86:0/1605527856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ccc158640 con 0x7f3ccc0d28b0 2024-04-25T14:10:00.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.442+0000 7f3cc3e00640 1 -- 172.21.15.86:0/1605527856 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc40127f0 con 0x7f3ccc0d28b0 2024-04-25T14:10:00.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.442+0000 7f3cd0c00640 1 -- 172.21.15.86:0/1605527856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ccc158c00 con 0x7f3ccc0d28b0 2024-04-25T14:10:00.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.442+0000 7f3cd0c00640 1 -- 172.21.15.86:0/1605527856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ccc076b80 con 0x7f3ccc0d28b0 2024-04-25T14:10:00.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.442+0000 7f3cc3e00640 1 -- 172.21.15.86:0/1605527856 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7f3cc4012990 con 0x7f3ccc0d28b0 2024-04-25T14:10:00.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.446+0000 7f3cc3e00640 1 --2- 172.21.15.86:0/1605527856 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f3ca806f6c0 0x7f3ca8071bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:00.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.446+0000 7f3cc3e00640 1 -- 172.21.15.86:0/1605527856 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5980+0+0 (secure 0 0 0) 0x7f3cc408fbc0 con 0x7f3ccc0d28b0 2024-04-25T14:10:00.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.446+0000 7f3ccb400640 1 --2- 172.21.15.86:0/1605527856 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f3ca806f6c0 0x7f3ca8071bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:00.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.446+0000 7f3ccb400640 1 --2- 172.21.15.86:0/1605527856 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f3ca806f6c0 0x7f3ca8071bd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc073870 tx=0x7f3cbc009190 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:00.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.450+0000 7f3cc3e00640 1 -- 172.21.15.86:0/1605527856 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3ccc0cf0f0 con 0x7f3ccc0d28b0 2024-04-25T14:10:00.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.638+0000 7f3cd0c00640 1 -- 172.21.15.86:0/1605527856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ccc0cf400 con 0x7f3ccc0d28b0 2024-04-25T14:10:00.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.638+0000 7f3cc3e00640 1 -- 172.21.15.86:0/1605527856 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+3872 (secure 0 0 0) 0x7f3cc405c2c0 con 0x7f3ccc0d28b0 2024-04-25T14:10:00.637 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:00.637 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":78,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1961914630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1961914630},{"type":"v1","addr":"172.21.15.86:6836","nonce":1961914630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2298829380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2298829380},{"type":"v1","addr":"172.21.15.155:6834","nonce":2298829380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7914,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3761704765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3761704765},{"type":"v1","addr":"172.21.15.86:6837","nonce":3761704765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8041,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/274306463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":274306463},{"type":"v1","addr":"172.21.15.155:6835","nonce":274306463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[]} 2024-04-25T14:10:00.637 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 78 2024-04-25T14:10:00.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.642+0000 7f3cd0c00640 1 -- 172.21.15.86:0/1605527856 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f3ca806f6c0 msgr2=0x7f3ca8071bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:00.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.642+0000 7f3cd0c00640 1 --2- 172.21.15.86:0/1605527856 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f3ca806f6c0 0x7f3ca8071bd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc073870 tx=0x7f3cbc009190 comp rx=0 tx=0).stop 2024-04-25T14:10:00.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.642+0000 7f3cd0c00640 1 -- 172.21.15.86:0/1605527856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ccc0d28b0 msgr2=0x7f3ccc15f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:00.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.642+0000 7f3cd0c00640 1 --2- 172.21.15.86:0/1605527856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ccc0d28b0 0x7f3ccc15f490 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f3cc400cc40 tx=0x7f3cc400a430 comp rx=0 tx=0).stop 2024-04-25T14:10:00.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.642+0000 7f3cd0c00640 1 -- 172.21.15.86:0/1605527856 shutdown_connections 2024-04-25T14:10:00.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.642+0000 7f3cd0c00640 1 --2- 172.21.15.86:0/1605527856 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f3ca806f6c0 0x7f3ca8071bd0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:00.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.642+0000 7f3cd0c00640 1 --2- 172.21.15.86:0/1605527856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ccc0d28b0 0x7f3ccc15f490 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:00.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.642+0000 7f3cd0c00640 1 --2- 172.21.15.86:0/1605527856 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ccc0d1f60 0x7f3ccc15bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:00.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.642+0000 7f3cd0c00640 1 --2- 172.21.15.86:0/1605527856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ccc0ce7a0 0x7f3ccc14bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:00.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.642+0000 7f3cd0c00640 1 -- 172.21.15.86:0/1605527856 >> 172.21.15.86:0/1605527856 conn(0x7f3ccc09ba30 msgr2=0x7f3ccc070ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:00.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.642+0000 7f3cd0c00640 1 -- 172.21.15.86:0/1605527856 shutdown_connections 2024-04-25T14:10:00.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.642+0000 7f3cd0c00640 1 -- 172.21.15.86:0/1605527856 wait complete. 2024-04-25T14:10:00.659 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T14:10:00.659 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T14:10:00.659 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T14:10:00.659 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:10:00.659 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T14:10:00.659 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T14:10:00.659 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T14:10:00.659 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:10:00.660 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T14:10:00.660 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T14:10:00.660 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T14:10:00.660 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:10:00.660 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T14:10:00.660 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T14:10:00.660 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T14:10:00.661 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:10:00.671 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T14:10:00.671 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T14:10:00.672 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T14:10:00.672 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T14:10:00.676 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T14:10:00.676 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T14:10:00.682 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T14:10:00.682 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T14:10:00.718 INFO:tasks.ceph.mds.a:Started 2024-04-25T14:10:00.719 DEBUG:teuthology.parallel:result is None 2024-04-25T14:10:00.721 INFO:tasks.ceph.mds.c:Started 2024-04-25T14:10:00.721 DEBUG:teuthology.parallel:result is None 2024-04-25T14:10:00.727 INFO:tasks.ceph.mds.d:Started 2024-04-25T14:10:00.727 DEBUG:teuthology.parallel:result is None 2024-04-25T14:10:00.729 INFO:tasks.ceph.mds.b:Started 2024-04-25T14:10:00.729 DEBUG:teuthology.parallel:result is None 2024-04-25T14:10:00.729 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T14:10:00.819 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-04-25T14:10:00.820 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-25T14:10:00.833 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2024-04-25T14:10:00.835 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2024-04-25T14:10:00.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.890+0000 7fbc8fa00640 1 Processor -- start 2024-04-25T14:10:00.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.890+0000 7fbc8fa00640 1 -- start start 2024-04-25T14:10:00.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.894+0000 7fbc8fa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc88070330 0x7fbc880d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:00.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.894+0000 7fbc8fa00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbc880d1c20 0x7fbc8806ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.894+0000 7fbc8fa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbc8806f270 0x7fbc8806f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.894+0000 7fbc8fa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbc880d2030 con 0x7fbc8806f270 2024-04-25T14:10:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.894+0000 7fbc8fa00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fbc8815e5c0 con 0x7fbc880d1c20 2024-04-25T14:10:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.894+0000 7fbc8fa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbc8815e8a0 con 0x7fbc88070330 2024-04-25T14:10:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.894+0000 7fbc8da00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbc8806f270 0x7fbc8806f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.894+0000 7fbc8da00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbc8806f270 0x7fbc8806f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49458/0 (socket says 172.21.15.86:49458) 2024-04-25T14:10:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.894+0000 7fbc8da00640 1 -- 172.21.15.86:0/172294423 learned_addr learned my addr 172.21.15.86:0/172294423 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.894+0000 7fbc8d000640 1 --2- 172.21.15.86:0/172294423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc88070330 0x7fbc880d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.894+0000 7fbc7fe00640 1 --2- 172.21.15.86:0/172294423 >> v2:172.21.15.155:3300/0 conn(0x7fbc880d1c20 0x7fbc8806ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.894+0000 7fbc8da00640 1 -- 172.21.15.86:0/172294423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc88070330 msgr2=0x7fbc880d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.894+0000 7fbc8da00640 1 --2- 172.21.15.86:0/172294423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc88070330 0x7fbc880d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.894+0000 7fbc8da00640 1 -- 172.21.15.86:0/172294423 >> v2:172.21.15.155:3300/0 conn(0x7fbc880d1c20 msgr2=0x7fbc8806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.894+0000 7fbc8da00640 1 --2- 172.21.15.86:0/172294423 >> v2:172.21.15.155:3300/0 conn(0x7fbc880d1c20 0x7fbc8806ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.894+0000 7fbc8da00640 1 -- 172.21.15.86:0/172294423 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8815eb80 con 0x7fbc8806f270 2024-04-25T14:10:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.898+0000 7fbc8da00640 1 --2- 172.21.15.86:0/172294423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc8806f270 0x7fbc8806f680 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009e50 tx=0x7fbc8001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a0d6071604c525 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.898+0000 7fbc87e00640 1 -- 172.21.15.86:0/172294423 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc800a5790 con 0x7fbc8806f270 2024-04-25T14:10:00.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.898+0000 7fbc87e00640 1 -- 172.21.15.86:0/172294423 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc800a5930 con 0x7fbc8806f270 2024-04-25T14:10:00.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.898+0000 7fbc87e00640 1 -- 172.21.15.86:0/172294423 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc800a5d00 con 0x7fbc8806f270 2024-04-25T14:10:00.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.898+0000 7fbc8fa00640 1 -- 172.21.15.86:0/172294423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc8806f270 msgr2=0x7fbc8806f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:00.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.898+0000 7fbc8fa00640 1 --2- 172.21.15.86:0/172294423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc8806f270 0x7fbc8806f680 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009e50 tx=0x7fbc8001c930 comp rx=0 tx=0).stop 2024-04-25T14:10:00.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.898+0000 7fbc8fa00640 1 -- 172.21.15.86:0/172294423 shutdown_connections 2024-04-25T14:10:00.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.898+0000 7fbc8fa00640 1 --2- 172.21.15.86:0/172294423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc8806f270 0x7fbc8806f680 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:00.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.898+0000 7fbc8fa00640 1 --2- 172.21.15.86:0/172294423 >> v2:172.21.15.155:3300/0 conn(0x7fbc880d1c20 0x7fbc8806ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:00.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.898+0000 7fbc8fa00640 1 --2- 172.21.15.86:0/172294423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc88070330 0x7fbc880d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:00.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.898+0000 7fbc8fa00640 1 -- 172.21.15.86:0/172294423 >> 172.21.15.86:0/172294423 conn(0x7fbc880b4650 msgr2=0x7fbc880b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:00.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.898+0000 7fbc8fa00640 1 -- 172.21.15.86:0/172294423 shutdown_connections 2024-04-25T14:10:00.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.898+0000 7fbc8fa00640 1 -- 172.21.15.86:0/172294423 wait complete. 2024-04-25T14:10:00.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.898+0000 7fbc8fa00640 1 Processor -- start 2024-04-25T14:10:00.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.898+0000 7fbc8fa00640 1 -- start start 2024-04-25T14:10:00.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.902+0000 7fbc8fa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc8806f270 0x7fbc880c2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:00.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.902+0000 7fbc8fa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc88070330 0x7fbc8809a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:00.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.902+0000 7fbc8fa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc880d1c20 0x7fbc881585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:00.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.902+0000 7fbc8fa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8815f4b0 con 0x7fbc8806f270 2024-04-25T14:10:00.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.902+0000 7fbc8fa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8815f330 con 0x7fbc88070330 2024-04-25T14:10:00.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.902+0000 7fbc8fa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbc88070740 con 0x7fbc880d1c20 2024-04-25T14:10:00.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.902+0000 7fbc8d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc8806f270 0x7fbc880c2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:00.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.902+0000 7fbc8d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc8806f270 0x7fbc880c2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49464/0 (socket says 172.21.15.86:49464) 2024-04-25T14:10:00.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.902+0000 7fbc8d000640 1 -- 172.21.15.86:0/2583153209 learned_addr learned my addr 172.21.15.86:0/2583153209 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:00.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.902+0000 7fbc8d000640 1 -- 172.21.15.86:0/2583153209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc880d1c20 msgr2=0x7fbc881585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:00.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.902+0000 7fbc8d000640 1 --2- 172.21.15.86:0/2583153209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc880d1c20 0x7fbc881585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:00.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.902+0000 7fbc8d000640 1 -- 172.21.15.86:0/2583153209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc88070330 msgr2=0x7fbc8809a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:00.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.902+0000 7fbc8d000640 1 --2- 172.21.15.86:0/2583153209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc88070330 0x7fbc8809a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:00.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.902+0000 7fbc8d000640 1 -- 172.21.15.86:0/2583153209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8809ae90 con 0x7fbc8806f270 2024-04-25T14:10:00.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.902+0000 7fbc8d000640 1 --2- 172.21.15.86:0/2583153209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc8806f270 0x7fbc880c2c10 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fbc7800e870 tx=0x7fbc7800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.902+0000 7fbc85600640 1 -- 172.21.15.86:0/2583153209 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc78017070 con 0x7fbc8806f270 2024-04-25T14:10:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.902+0000 7fbc85600640 1 -- 172.21.15.86:0/2583153209 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc7800ee60 con 0x7fbc8806f270 2024-04-25T14:10:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.902+0000 7fbc85600640 1 -- 172.21.15.86:0/2583153209 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc78004240 con 0x7fbc8806f270 2024-04-25T14:10:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.902+0000 7fbc8fa00640 1 -- 172.21.15.86:0/2583153209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc880c4690 con 0x7fbc8806f270 2024-04-25T14:10:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.902+0000 7fbc8fa00640 1 -- 172.21.15.86:0/2583153209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc88159ca0 con 0x7fbc8806f270 2024-04-25T14:10:00.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.902+0000 7fbc8fa00640 1 -- 172.21.15.86:0/2583153209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc88069d00 con 0x7fbc8806f270 2024-04-25T14:10:00.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.906+0000 7fbc85600640 1 -- 172.21.15.86:0/2583153209 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7fbc7800c940 con 0x7fbc8806f270 2024-04-25T14:10:00.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.906+0000 7fbc85600640 1 --2- 172.21.15.86:0/2583153209 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fbc6406f520 0x7fbc64071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:00.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.906+0000 7fbc85600640 1 -- 172.21.15.86:0/2583153209 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5980+0+0 (secure 0 0 0) 0x7fbc7808f6d0 con 0x7fbc8806f270 2024-04-25T14:10:00.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.906+0000 7fbc7fe00640 1 --2- 172.21.15.86:0/2583153209 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fbc6406f520 0x7fbc64071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:00.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.906+0000 7fbc7fe00640 1 --2- 172.21.15.86:0/2583153209 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fbc6406f520 0x7fbc64071a30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbc70006aa0 tx=0x7fbc70006a30 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:00.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:00.910+0000 7fbc85600640 1 -- 172.21.15.86:0/2583153209 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbc7805c300 con 0x7fbc8806f270 2024-04-25T14:10:01.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.078+0000 7fbc8fa00640 1 -- 172.21.15.86:0/2583153209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fbc88076bf0 con 0x7fbc8806f270 2024-04-25T14:10:01.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.706+0000 7fbc85600640 1 -- 172.21.15.86:0/2583153209 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v107) ==== 97+0+0 (secure 0 0 0) 0x7fbc7805bc00 con 0x7fbc8806f270 2024-04-25T14:10:01.703 INFO:teuthology.orchestra.run.smithi086.stderr: removed all blocklist entries 2024-04-25T14:10:01.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.710+0000 7fbc8fa00640 1 -- 172.21.15.86:0/2583153209 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fbc6406f520 msgr2=0x7fbc64071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:01.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.710+0000 7fbc8fa00640 1 --2- 172.21.15.86:0/2583153209 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fbc6406f520 0x7fbc64071a30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbc70006aa0 tx=0x7fbc70006a30 comp rx=0 tx=0).stop 2024-04-25T14:10:01.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.710+0000 7fbc8fa00640 1 -- 172.21.15.86:0/2583153209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc8806f270 msgr2=0x7fbc880c2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:01.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.710+0000 7fbc8fa00640 1 --2- 172.21.15.86:0/2583153209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc8806f270 0x7fbc880c2c10 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fbc7800e870 tx=0x7fbc7800ed30 comp rx=0 tx=0).stop 2024-04-25T14:10:01.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.710+0000 7fbc8fa00640 1 -- 172.21.15.86:0/2583153209 shutdown_connections 2024-04-25T14:10:01.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.710+0000 7fbc8fa00640 1 --2- 172.21.15.86:0/2583153209 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fbc6406f520 0x7fbc64071a30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.710+0000 7fbc8fa00640 1 --2- 172.21.15.86:0/2583153209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc880d1c20 0x7fbc881585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.710+0000 7fbc8fa00640 1 --2- 172.21.15.86:0/2583153209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc88070330 0x7fbc8809a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.710+0000 7fbc8fa00640 1 --2- 172.21.15.86:0/2583153209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc8806f270 0x7fbc880c2c10 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.710+0000 7fbc8fa00640 1 -- 172.21.15.86:0/2583153209 >> 172.21.15.86:0/2583153209 conn(0x7fbc880b4650 msgr2=0x7fbc88073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:01.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.710+0000 7fbc8fa00640 1 -- 172.21.15.86:0/2583153209 shutdown_connections 2024-04-25T14:10:01.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.710+0000 7fbc8fa00640 1 -- 172.21.15.86:0/2583153209 wait complete. 2024-04-25T14:10:01.731 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T14:10:01.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.882+0000 7ff1f7800640 1 Processor -- start 2024-04-25T14:10:01.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.882+0000 7ff1f7800640 1 -- start start 2024-04-25T14:10:01.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1f7800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff1f00740c0 0x7ff1f01676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:01.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1f7800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1f0072bd0 con 0x7ff1f00cfca0 2024-04-25T14:10:01.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1f7800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1f006fbd0 con 0x7ff1f00d16e0 2024-04-25T14:10:01.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1f7800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff1f0070470 con 0x7ff1f00740c0 2024-04-25T14:10:01.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1f4e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff1f00cfca0 0x7ff1f006f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41844/0 (socket says 172.21.15.86:41844) 2024-04-25T14:10:01.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1f4e00640 1 -- 172.21.15.86:0/1780047813 learned_addr learned my addr 172.21.15.86:0/1780047813 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:01.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1f5800640 1 --2- 172.21.15.86:0/1780047813 >> v2:172.21.15.155:3300/0 conn(0x7ff1f00740c0 0x7ff1f01676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:01.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1efe00640 1 -- 172.21.15.86:0/1780047813 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3509816035 0 0) 0x7ff1f0072bd0 con 0x7ff1f00cfca0 2024-04-25T14:10:01.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1efe00640 1 -- 172.21.15.86:0/1780047813 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1d4003530 con 0x7ff1f00cfca0 2024-04-25T14:10:01.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1efe00640 1 -- 172.21.15.86:0/1780047813 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 478545078 0 0) 0x7ff1f006fbd0 con 0x7ff1f00d16e0 2024-04-25T14:10:01.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1efe00640 1 -- 172.21.15.86:0/1780047813 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1f0072bd0 con 0x7ff1f00d16e0 2024-04-25T14:10:01.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1efe00640 1 -- 172.21.15.86:0/1780047813 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1800692963 0 0) 0x7ff1d4003530 con 0x7ff1f00cfca0 2024-04-25T14:10:01.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1efe00640 1 -- 172.21.15.86:0/1780047813 >> v2:172.21.15.155:3300/0 conn(0x7ff1f00740c0 msgr2=0x7ff1f01676d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:01.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1efe00640 1 --2- 172.21.15.86:0/1780047813 >> v2:172.21.15.155:3300/0 conn(0x7ff1f00740c0 0x7ff1f01676d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1efe00640 1 -- 172.21.15.86:0/1780047813 >> v1:172.21.15.155:6789/0 conn(0x7ff1f00d16e0 legacy=0x7ff1f00d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:01.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1efe00640 1 -- 172.21.15.86:0/1780047813 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f006feb0 con 0x7ff1f00cfca0 2024-04-25T14:10:01.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1efe00640 1 -- 172.21.15.86:0/1780047813 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7ff1e00016a0 con 0x7ff1f00cfca0 2024-04-25T14:10:01.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1efe00640 1 -- 172.21.15.86:0/1780047813 >> v1:172.21.15.86:6789/0 conn(0x7ff1f00cfca0 legacy=0x7ff1f006f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:01.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1efe00640 1 --2- 172.21.15.86:0/1780047813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1d4003bc0 0x7ff1d4004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:01.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1efe00640 1 --2- 172.21.15.86:0/1780047813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1d4004810 0x7ff1d4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:01.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1efe00640 1 --2- 172.21.15.86:0/1780047813 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1d4005220 0x7ff1d400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:01.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1efe00640 1 -- 172.21.15.86:0/1780047813 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f0070470 con 0x7ff1d4003bc0 2024-04-25T14:10:01.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1efe00640 1 -- 172.21.15.86:0/1780047813 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d400c1c0 con 0x7ff1d4004810 2024-04-25T14:10:01.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1efe00640 1 -- 172.21.15.86:0/1780047813 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff1d400c3c0 con 0x7ff1d4005220 2024-04-25T14:10:01.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1f4e00640 1 --2- 172.21.15.86:0/1780047813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1d4003bc0 0x7ff1d4004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:01.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1e7e00640 1 --2- 172.21.15.86:0/1780047813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1d4004810 0x7ff1d4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:01.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1f5800640 1 --2- 172.21.15.86:0/1780047813 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1d4005220 0x7ff1d400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:01.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1f4e00640 1 -- 172.21.15.86:0/1780047813 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1d4005220 msgr2=0x7ff1d400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:01.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1f4e00640 1 --2- 172.21.15.86:0/1780047813 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1d4005220 0x7ff1d400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1f4e00640 1 -- 172.21.15.86:0/1780047813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1d4004810 msgr2=0x7ff1d4004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:01.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1f4e00640 1 --2- 172.21.15.86:0/1780047813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1d4004810 0x7ff1d4004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1f4e00640 1 -- 172.21.15.86:0/1780047813 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d4003530 con 0x7ff1d4003bc0 2024-04-25T14:10:01.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1f5800640 1 -- 172.21.15.86:0/1780047813 reap_dead start 2024-04-25T14:10:01.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.886+0000 7ff1e7e00640 1 --2- 172.21.15.86:0/1780047813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1d4004810 0x7ff1d4004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:01.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.890+0000 7ff1f4e00640 1 --2- 172.21.15.86:0/1780047813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1d4003bc0 0x7ff1d4004010 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0070c30 tx=0x7ff1e001cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11947f86122ec26b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:01.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.890+0000 7ff1efe00640 1 -- 172.21.15.86:0/1780047813 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0007cf0 con 0x7ff1d4003bc0 2024-04-25T14:10:01.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.890+0000 7ff1efe00640 1 -- 172.21.15.86:0/1780047813 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1e00a9020 con 0x7ff1d4003bc0 2024-04-25T14:10:01.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.890+0000 7ff1efe00640 1 -- 172.21.15.86:0/1780047813 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0005ae0 con 0x7ff1d4003bc0 2024-04-25T14:10:01.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.890+0000 7ff1f7800640 1 -- 172.21.15.86:0/1780047813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1d4003bc0 msgr2=0x7ff1d4004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:01.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.890+0000 7ff1f7800640 1 --2- 172.21.15.86:0/1780047813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1d4003bc0 0x7ff1d4004010 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0070c30 tx=0x7ff1e001cbc0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.890+0000 7ff1f7800640 1 -- 172.21.15.86:0/1780047813 shutdown_connections 2024-04-25T14:10:01.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.890+0000 7ff1f7800640 1 --2- 172.21.15.86:0/1780047813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1d4003bc0 0x7ff1d4004010 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.890+0000 7ff1f7800640 1 -- 172.21.15.86:0/1780047813 >> 172.21.15.86:0/1780047813 conn(0x7ff1f00b4650 msgr2=0x7ff1f00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:01.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.890+0000 7ff1f7800640 1 -- 172.21.15.86:0/1780047813 shutdown_connections 2024-04-25T14:10:01.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.890+0000 7ff1f7800640 1 -- 172.21.15.86:0/1780047813 wait complete. 2024-04-25T14:10:01.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.894+0000 7ff1f7800640 1 Processor -- start 2024-04-25T14:10:01.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.894+0000 7ff1f7800640 1 -- start start 2024-04-25T14:10:01.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.894+0000 7ff1f7800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1d4003bc0 0x7ff1f0156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:01.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.894+0000 7ff1f7800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1f01560a0 0x7ff1f00c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:01.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.894+0000 7ff1f7800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1f0159930 0x7ff1f0147780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:01.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.894+0000 7ff1f7800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f006d000 con 0x7ff1d4003bc0 2024-04-25T14:10:01.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.894+0000 7ff1f7800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f006ce80 con 0x7ff1f01560a0 2024-04-25T14:10:01.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.894+0000 7ff1f7800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f006d180 con 0x7ff1f0159930 2024-04-25T14:10:01.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.894+0000 7ff1f4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1d4003bc0 0x7ff1f0156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:01.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.894+0000 7ff1f4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1d4003bc0 0x7ff1f0156580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49482/0 (socket says 172.21.15.86:49482) 2024-04-25T14:10:01.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.894+0000 7ff1f4e00640 1 -- 172.21.15.86:0/1617524661 learned_addr learned my addr 172.21.15.86:0/1617524661 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:01.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.894+0000 7ff1e7e00640 1 --2- 172.21.15.86:0/1617524661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1f01560a0 0x7ff1f00c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:01.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.894+0000 7ff1f5800640 1 --2- 172.21.15.86:0/1617524661 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1f0159930 0x7ff1f0147780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:01.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.894+0000 7ff1f4e00640 1 -- 172.21.15.86:0/1617524661 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1f0159930 msgr2=0x7ff1f0147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:01.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.894+0000 7ff1f4e00640 1 --2- 172.21.15.86:0/1617524661 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1f0159930 0x7ff1f0147780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.894+0000 7ff1f4e00640 1 -- 172.21.15.86:0/1617524661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1f01560a0 msgr2=0x7ff1f00c5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:01.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.894+0000 7ff1f4e00640 1 --2- 172.21.15.86:0/1617524661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1f01560a0 0x7ff1f00c5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.894+0000 7ff1f4e00640 1 -- 172.21.15.86:0/1617524661 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f00b1f20 con 0x7ff1d4003bc0 2024-04-25T14:10:01.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.894+0000 7ff1f5800640 1 --2- 172.21.15.86:0/1617524661 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1f0159930 0x7ff1f0147780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:01.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.894+0000 7ff1f4e00640 1 --2- 172.21.15.86:0/1617524661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1d4003bc0 0x7ff1f0156580 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7ff1e001c630 tx=0x7ff1d4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:01.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.894+0000 7ff1ed600640 1 -- 172.21.15.86:0/1617524661 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e00049f0 con 0x7ff1d4003bc0 2024-04-25T14:10:01.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.894+0000 7ff1ed600640 1 -- 172.21.15.86:0/1617524661 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1e00b2030 con 0x7ff1d4003bc0 2024-04-25T14:10:01.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.894+0000 7ff1ed600640 1 -- 172.21.15.86:0/1617524661 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0004df0 con 0x7ff1d4003bc0 2024-04-25T14:10:01.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.894+0000 7ff1f7800640 1 -- 172.21.15.86:0/1617524661 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f00b2200 con 0x7ff1d4003bc0 2024-04-25T14:10:01.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.894+0000 7ff1f7800640 1 -- 172.21.15.86:0/1617524661 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f0156ea0 con 0x7ff1d4003bc0 2024-04-25T14:10:01.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.898+0000 7ff1f7800640 1 -- 172.21.15.86:0/1617524661 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f009f4b0 con 0x7ff1d4003bc0 2024-04-25T14:10:01.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.898+0000 7ff1ed600640 1 -- 172.21.15.86:0/1617524661 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7ff1e00a5780 con 0x7ff1d4003bc0 2024-04-25T14:10:01.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.898+0000 7ff1ed600640 1 --2- 172.21.15.86:0/1617524661 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7ff1cc06f550 0x7ff1cc071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:01.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.898+0000 7ff1ed600640 1 -- 172.21.15.86:0/1617524661 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5249+0+0 (secure 0 0 0) 0x7ff1e0128300 con 0x7ff1d4003bc0 2024-04-25T14:10:01.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.898+0000 7ff1e7e00640 1 --2- 172.21.15.86:0/1617524661 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7ff1cc06f550 0x7ff1cc071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:01.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.898+0000 7ff1e7e00640 1 --2- 172.21.15.86:0/1617524661 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7ff1cc06f550 0x7ff1cc071a60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8007fd0 tx=0x7ff1d8009040 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:01.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:01.902+0000 7ff1ed600640 1 -- 172.21.15.86:0/1617524661 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff1e00168d0 con 0x7ff1d4003bc0 2024-04-25T14:10:02.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.114+0000 7ff1f7800640 1 -- 172.21.15.86:0/1617524661 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7ff1f0069d00 con 0x7ff1d4003bc0 2024-04-25T14:10:02.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.114+0000 7ff1ed600640 1 -- 172.21.15.86:0/1617524661 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v26) ==== 80+0+5238 (secure 0 0 0) 0x7ff1e00fd280 con 0x7ff1d4003bc0 2024-04-25T14:10:02.110 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:02.110 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:10:02.111 INFO:teuthology.orchestra.run.smithi086.stdout: "auth_dump": [ 2024-04-25T14:10:02.111 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:02.111 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.a", 2024-04-25T14:10:02.111 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmWk5fAxAAKEBKhWewpl23MUcvMi8qDg==", 2024-04-25T14:10:02.111 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:02.111 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:10:02.111 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:10:02.111 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:10:02.111 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:02.111 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:02.111 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:02.111 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:02.111 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.b", 2024-04-25T14:10:02.112 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmDhzOEBAABNys/NAEhucQ7A+HzQlzQw==", 2024-04-25T14:10:02.112 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:02.112 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:10:02.112 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:10:02.112 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:10:02.112 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:02.112 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:02.112 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:02.112 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:02.112 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.c", 2024-04-25T14:10:02.112 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmA/hNCxAAclDGbIjLfjqyXn0MwX/IkA==", 2024-04-25T14:10:02.112 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:02.112 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:10:02.112 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:10:02.113 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:10:02.113 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:02.113 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:02.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:02.113 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:02.113 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.d", 2024-04-25T14:10:02.113 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmemCcGBAAGrCx6C850qRJSWYjebxOBA==", 2024-04-25T14:10:02.113 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:02.113 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:10:02.113 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:10:02.113 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:10:02.113 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:02.113 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:02.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:02.114 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:02.114 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.0", 2024-04-25T14:10:02.114 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCOYipm9etgGxAAemg9CqXN0ShduDtDuKF2Zw==", 2024-04-25T14:10:02.114 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:02.114 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:10:02.114 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:10:02.114 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:02.114 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:02.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:02.114 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:02.114 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.1", 2024-04-25T14:10:02.114 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCRYipmBCiNJhAABAWHk8NEGVvqieL2klGhmA==", 2024-04-25T14:10:02.114 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:02.115 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:10:02.115 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:10:02.115 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:02.115 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:02.115 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:02.115 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:02.115 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.2", 2024-04-25T14:10:02.115 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCUYipm/XGYLxAAPXsw1svKseFzFiuDUjAsJQ==", 2024-04-25T14:10:02.115 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:02.115 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:10:02.115 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:10:02.115 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:02.115 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:02.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:02.116 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:02.116 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.3", 2024-04-25T14:10:02.116 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCXYipmlSRaORAAwpKyzZDdVYHWl2pAsQjmpw==", 2024-04-25T14:10:02.116 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:02.116 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:10:02.116 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:10:02.116 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:02.116 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:02.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:02.116 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:02.116 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.4", 2024-04-25T14:10:02.116 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCdYipmBI3FMBAAV2NdggfRSFnQvrcOC2YqiA==", 2024-04-25T14:10:02.117 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:02.117 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:10:02.117 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:10:02.117 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:02.117 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:02.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:02.117 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:02.117 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.5", 2024-04-25T14:10:02.117 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQChYipmctYxABAABwmOMgyYuOfRWD9KdHILng==", 2024-04-25T14:10:02.117 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:02.117 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:10:02.117 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:10:02.117 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:02.117 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:02.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:02.118 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:02.118 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.6", 2024-04-25T14:10:02.118 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCkYipmSL7TCRAAhFMWg1AQm54bB+x1GDcusA==", 2024-04-25T14:10:02.118 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:02.118 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:10:02.118 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:10:02.118 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:02.118 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:02.118 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:02.118 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:02.118 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.7", 2024-04-25T14:10:02.118 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCnYipmZ4F+FRAAEGaZN+byv/qJUFZb7WxTFg==", 2024-04-25T14:10:02.118 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:02.118 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:10:02.119 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:10:02.119 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:02.119 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:02.119 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:02.119 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:02.119 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.0", 2024-04-25T14:10:02.119 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipm5u4xHRAACDFe3Eb4KLrsdLj4eFe7mA==", 2024-04-25T14:10:02.119 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:02.119 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:10:02.119 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-25T14:10:02.119 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T14:10:02.119 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:02.119 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:02.119 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:02.119 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.1", 2024-04-25T14:10:02.120 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmDp1hJRAAQkBc5PdQoMTT7W4HahkTyg==", 2024-04-25T14:10:02.120 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:02.120 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:10:02.120 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-25T14:10:02.120 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw pool=fs2_data" 2024-04-25T14:10:02.120 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:02.120 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:02.120 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:02.120 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.2", 2024-04-25T14:10:02.120 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmhbhGIRAAxao/J0WY53IITLETFgwIVw==", 2024-04-25T14:10:02.120 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:02.120 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:10:02.120 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-25T14:10:02.120 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw pool=fs2_data" 2024-04-25T14:10:02.120 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:02.121 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:02.121 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:02.121 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.3", 2024-04-25T14:10:02.121 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmJsCBKRAAzXn2qDtblIpqt7U/TUUlyA==", 2024-04-25T14:10:02.121 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:02.121 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:10:02.121 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-25T14:10:02.121 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw pool=fs2_data" 2024-04-25T14:10:02.121 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:02.121 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:02.121 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:02.121 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.admin", 2024-04-25T14:10:02.121 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCKYipmTqbVIBAA1VpOtj24f2ibVEIT4EJwmw==", 2024-04-25T14:10:02.121 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:02.122 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-25T14:10:02.122 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:10:02.122 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:10:02.122 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:02.122 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:02.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:02.122 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:02.122 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.x", 2024-04-25T14:10:02.122 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCKYipmKMzAORAAtRyTNJ5HWi2XwuamoqwB0Q==", 2024-04-25T14:10:02.122 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:02.122 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-25T14:10:02.122 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-25T14:10:02.122 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:02.122 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:02.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:02.123 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:02.123 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.y", 2024-04-25T14:10:02.123 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCKYipmC7A7NRAAEwVNA0yxt3fwdHXrsV0dvg==", 2024-04-25T14:10:02.123 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:02.123 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-25T14:10:02.123 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-25T14:10:02.123 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:02.123 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:02.123 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:02.123 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-25T14:10:02.123 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:10:02.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.114+0000 7ff1f7800640 1 -- 172.21.15.86:0/1617524661 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7ff1cc06f550 msgr2=0x7ff1cc071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:02.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.114+0000 7ff1f7800640 1 --2- 172.21.15.86:0/1617524661 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7ff1cc06f550 0x7ff1cc071a60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8007fd0 tx=0x7ff1d8009040 comp rx=0 tx=0).stop 2024-04-25T14:10:02.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.114+0000 7ff1f7800640 1 -- 172.21.15.86:0/1617524661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1d4003bc0 msgr2=0x7ff1f0156580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:02.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.114+0000 7ff1f7800640 1 --2- 172.21.15.86:0/1617524661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1d4003bc0 0x7ff1f0156580 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7ff1e001c630 tx=0x7ff1d4005840 comp rx=0 tx=0).stop 2024-04-25T14:10:02.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.114+0000 7ff1f7800640 1 -- 172.21.15.86:0/1617524661 shutdown_connections 2024-04-25T14:10:02.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.114+0000 7ff1f7800640 1 --2- 172.21.15.86:0/1617524661 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7ff1cc06f550 0x7ff1cc071a60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:02.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.114+0000 7ff1f7800640 1 --2- 172.21.15.86:0/1617524661 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1f0159930 0x7ff1f0147780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.114+0000 7ff1f7800640 1 --2- 172.21.15.86:0/1617524661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1f01560a0 0x7ff1f00c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.114+0000 7ff1f7800640 1 --2- 172.21.15.86:0/1617524661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1d4003bc0 0x7ff1f0156580 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.114+0000 7ff1f7800640 1 -- 172.21.15.86:0/1617524661 >> 172.21.15.86:0/1617524661 conn(0x7ff1f00b4650 msgr2=0x7ff1f00ae410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.114+0000 7ff1f7800640 1 -- 172.21.15.86:0/1617524661 shutdown_connections 2024-04-25T14:10:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.114+0000 7ff1f7800640 1 -- 172.21.15.86:0/1617524661 wait complete. 2024-04-25T14:10:02.131 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:10:02.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.278+0000 7f94c7a00640 1 Processor -- start 2024-04-25T14:10:02.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94c7a00640 1 -- start start 2024-04-25T14:10:02.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94c7a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94c00cde90 0x7f94c00ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:02.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94c7a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94c015e5c0 con 0x7f94c0070330 2024-04-25T14:10:02.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94c7a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94c015e8a0 con 0x7f94c00d1df0 2024-04-25T14:10:02.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94c7a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f94c006fd40 con 0x7f94c00cde90 2024-04-25T14:10:02.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94c5000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f94c0070330 0x7f94c00d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41852/0 (socket says 172.21.15.86:41852) 2024-04-25T14:10:02.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94c5000640 1 -- 172.21.15.86:0/2245650619 learned_addr learned my addr 172.21.15.86:0/2245650619 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94bfe00640 1 -- 172.21.15.86:0/2245650619 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4123893117 0 0) 0x7f94c015e5c0 con 0x7f94c0070330 2024-04-25T14:10:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94c5a00640 1 --2- 172.21.15.86:0/2245650619 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94c00cde90 0x7f94c00ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94bfe00640 1 -- 172.21.15.86:0/2245650619 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94a4003530 con 0x7f94c0070330 2024-04-25T14:10:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94bfe00640 1 -- 172.21.15.86:0/2245650619 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2187870193 0 0) 0x7f94a4003530 con 0x7f94c0070330 2024-04-25T14:10:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94bfe00640 1 -- 172.21.15.86:0/2245650619 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94c00cde90 msgr2=0x7f94c00ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94bfe00640 1 --2- 172.21.15.86:0/2245650619 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94c00cde90 0x7f94c00ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:02.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94bfe00640 1 -- 172.21.15.86:0/2245650619 >> v1:172.21.15.155:6789/0 conn(0x7f94c00d1df0 legacy=0x7f94c00cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:02.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94bfe00640 1 -- 172.21.15.86:0/2245650619 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c015eb80 con 0x7f94c0070330 2024-04-25T14:10:02.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94bfe00640 1 -- 172.21.15.86:0/2245650619 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f94b0001680 con 0x7f94c0070330 2024-04-25T14:10:02.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94bfe00640 1 -- 172.21.15.86:0/2245650619 >> v1:172.21.15.86:6789/0 conn(0x7f94c0070330 legacy=0x7f94c00d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:02.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94bfe00640 1 --2- 172.21.15.86:0/2245650619 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94c00cde90 0x7f94a4003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:02.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94bfe00640 1 --2- 172.21.15.86:0/2245650619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94a4003fe0 0x7f94a4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:02.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94bfe00640 1 --2- 172.21.15.86:0/2245650619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94a4004d50 0x7f94a40051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:02.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94bfe00640 1 -- 172.21.15.86:0/2245650619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f94a400bc50 con 0x7f94a4003fe0 2024-04-25T14:10:02.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94bfe00640 1 -- 172.21.15.86:0/2245650619 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f94a400be00 con 0x7f94a4004d50 2024-04-25T14:10:02.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94bfe00640 1 -- 172.21.15.86:0/2245650619 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f94a400c000 con 0x7f94c00cde90 2024-04-25T14:10:02.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94b7e00640 1 --2- 172.21.15.86:0/2245650619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94a4003fe0 0x7f94a4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:02.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94c5000640 1 --2- 172.21.15.86:0/2245650619 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94c00cde90 0x7f94a4003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:02.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94c5a00640 1 --2- 172.21.15.86:0/2245650619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94a4004d50 0x7f94a40051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:02.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94b7e00640 1 -- 172.21.15.86:0/2245650619 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94c00cde90 msgr2=0x7f94a4003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:02.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94b7e00640 1 --2- 172.21.15.86:0/2245650619 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94c00cde90 0x7f94a4003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:02.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94b7e00640 1 -- 172.21.15.86:0/2245650619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94a4004d50 msgr2=0x7f94a40051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:02.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94b7e00640 1 --2- 172.21.15.86:0/2245650619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94a4004d50 0x7f94a40051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:02.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94b7e00640 1 -- 172.21.15.86:0/2245650619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a4003530 con 0x7f94a4003fe0 2024-04-25T14:10:02.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.282+0000 7f94c5000640 1 --2- 172.21.15.86:0/2245650619 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94c00cde90 0x7f94a4003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:02.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.286+0000 7f94b7e00640 1 --2- 172.21.15.86:0/2245650619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94a4003fe0 0x7f94a4004810 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f94c0074ed0 tx=0x7f94a801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb8fb2c6a03000 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:02.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.286+0000 7f94bfe00640 1 -- 172.21.15.86:0/2245650619 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a80013a0 con 0x7f94a4003fe0 2024-04-25T14:10:02.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.286+0000 7f94bfe00640 1 -- 172.21.15.86:0/2245650619 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94a80052f0 con 0x7f94a4003fe0 2024-04-25T14:10:02.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.286+0000 7f94bfe00640 1 -- 172.21.15.86:0/2245650619 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a8003840 con 0x7f94a4003fe0 2024-04-25T14:10:02.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.286+0000 7f94c7a00640 1 -- 172.21.15.86:0/2245650619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94a4003fe0 msgr2=0x7f94a4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:02.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.286+0000 7f94c7a00640 1 --2- 172.21.15.86:0/2245650619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94a4003fe0 0x7f94a4004810 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f94c0074ed0 tx=0x7f94a801c390 comp rx=0 tx=0).stop 2024-04-25T14:10:02.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.286+0000 7f94c5a00640 1 -- 172.21.15.86:0/2245650619 reap_dead start 2024-04-25T14:10:02.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.286+0000 7f94c7a00640 1 -- 172.21.15.86:0/2245650619 shutdown_connections 2024-04-25T14:10:02.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.286+0000 7f94c7a00640 1 -- 172.21.15.86:0/2245650619 >> 172.21.15.86:0/2245650619 conn(0x7f94c00b4650 msgr2=0x7f94c00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:02.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.286+0000 7f94c7a00640 1 -- 172.21.15.86:0/2245650619 shutdown_connections 2024-04-25T14:10:02.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.286+0000 7f94c7a00640 1 -- 172.21.15.86:0/2245650619 wait complete. 2024-04-25T14:10:02.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.290+0000 7f94c7a00640 1 Processor -- start 2024-04-25T14:10:02.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.290+0000 7f94c7a00640 1 -- start start 2024-04-25T14:10:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.290+0000 7f94c7a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94a4003fe0 0x7f94c01566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.290+0000 7f94c7a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94c00b0990 0x7f94c015ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.290+0000 7f94c7a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94c015b160 0x7f94c009a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.290+0000 7f94c7a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f94c00708c0 con 0x7f94c00b0990 2024-04-25T14:10:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.290+0000 7f94c7a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f94c0070740 con 0x7f94a4003fe0 2024-04-25T14:10:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.290+0000 7f94c7a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f94c01697b0 con 0x7f94c015b160 2024-04-25T14:10:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.290+0000 7f94b7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94c00b0990 0x7f94c015ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.290+0000 7f94b7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94c00b0990 0x7f94c015ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49494/0 (socket says 172.21.15.86:49494) 2024-04-25T14:10:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.290+0000 7f94b7e00640 1 -- 172.21.15.86:0/2663212243 learned_addr learned my addr 172.21.15.86:0/2663212243 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.290+0000 7f94c5000640 1 --2- 172.21.15.86:0/2663212243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94a4003fe0 0x7f94c01566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.290+0000 7f94c5a00640 1 --2- 172.21.15.86:0/2663212243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94c015b160 0x7f94c009a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.290+0000 7f94b7e00640 1 -- 172.21.15.86:0/2663212243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94c015b160 msgr2=0x7f94c009a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:02.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.290+0000 7f94b7e00640 1 --2- 172.21.15.86:0/2663212243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94c015b160 0x7f94c009a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:02.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.294+0000 7f94b7e00640 1 -- 172.21.15.86:0/2663212243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94a4003fe0 msgr2=0x7f94c01566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:02.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.294+0000 7f94b7e00640 1 --2- 172.21.15.86:0/2663212243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94a4003fe0 0x7f94c01566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:02.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.294+0000 7f94b7e00640 1 -- 172.21.15.86:0/2663212243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c015b570 con 0x7f94c00b0990 2024-04-25T14:10:02.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.294+0000 7f94b7e00640 1 --2- 172.21.15.86:0/2663212243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94c00b0990 0x7f94c015ac20 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f94a801c1c0 tx=0x7f94a4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:02.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.294+0000 7f94bd600640 1 -- 172.21.15.86:0/2663212243 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a80ba020 con 0x7f94c00b0990 2024-04-25T14:10:02.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.294+0000 7f94bd600640 1 -- 172.21.15.86:0/2663212243 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94a8000ee0 con 0x7f94c00b0990 2024-04-25T14:10:02.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.294+0000 7f94bd600640 1 -- 172.21.15.86:0/2663212243 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a8007a40 con 0x7f94c00b0990 2024-04-25T14:10:02.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.294+0000 7f94c7a00640 1 -- 172.21.15.86:0/2663212243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c0156ee0 con 0x7f94c00b0990 2024-04-25T14:10:02.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.294+0000 7f94c7a00640 1 -- 172.21.15.86:0/2663212243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c00b2db0 con 0x7f94c00b0990 2024-04-25T14:10:02.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.294+0000 7f94c7a00640 1 -- 172.21.15.86:0/2663212243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94c0099a20 con 0x7f94c00b0990 2024-04-25T14:10:02.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.294+0000 7f94bd600640 1 -- 172.21.15.86:0/2663212243 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7f94a80bd020 con 0x7f94c00b0990 2024-04-25T14:10:02.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.294+0000 7f94bd600640 1 --2- 172.21.15.86:0/2663212243 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f949c06f550 0x7f949c071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:02.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.294+0000 7f94bd600640 1 -- 172.21.15.86:0/2663212243 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5249+0+0 (secure 0 0 0) 0x7f94a8129520 con 0x7f94c00b0990 2024-04-25T14:10:02.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.294+0000 7f94c5000640 1 --2- 172.21.15.86:0/2663212243 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f949c06f550 0x7f949c071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:02.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.298+0000 7f94c5000640 1 --2- 172.21.15.86:0/2663212243 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f949c06f550 0x7f949c071a60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f94b0000fc0 tx=0x7f94b0009040 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:02.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.302+0000 7f94bd600640 1 -- 172.21.15.86:0/2663212243 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f94a80c4030 con 0x7f94c00b0990 2024-04-25T14:10:02.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.458+0000 7f94c7a00640 1 -- 172.21.15.86:0/2663212243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94c0069d00 con 0x7f94c00b0990 2024-04-25T14:10:02.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.458+0000 7f94bd600640 1 -- 172.21.15.86:0/2663212243 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+13166 (secure 0 0 0) 0x7f94c015b570 con 0x7f94c00b0990 2024-04-25T14:10:02.457 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:02.457 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":109,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:10:01.720204+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:10:02.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.462+0000 7f94c7a00640 1 -- 172.21.15.86:0/2663212243 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f949c06f550 msgr2=0x7f949c071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:02.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.462+0000 7f94c7a00640 1 --2- 172.21.15.86:0/2663212243 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f949c06f550 0x7f949c071a60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f94b0000fc0 tx=0x7f94b0009040 comp rx=0 tx=0).stop 2024-04-25T14:10:02.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.462+0000 7f94c7a00640 1 -- 172.21.15.86:0/2663212243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94c00b0990 msgr2=0x7f94c015ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:02.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.462+0000 7f94c7a00640 1 --2- 172.21.15.86:0/2663212243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94c00b0990 0x7f94c015ac20 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f94a801c1c0 tx=0x7f94a4005840 comp rx=0 tx=0).stop 2024-04-25T14:10:02.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.462+0000 7f94c7a00640 1 -- 172.21.15.86:0/2663212243 shutdown_connections 2024-04-25T14:10:02.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.462+0000 7f94c7a00640 1 --2- 172.21.15.86:0/2663212243 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f949c06f550 0x7f949c071a60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:02.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.462+0000 7f94c7a00640 1 --2- 172.21.15.86:0/2663212243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94c015b160 0x7f94c009a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:02.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.462+0000 7f94c7a00640 1 --2- 172.21.15.86:0/2663212243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94c00b0990 0x7f94c015ac20 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:02.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.462+0000 7f94c7a00640 1 --2- 172.21.15.86:0/2663212243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94a4003fe0 0x7f94c01566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:02.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.462+0000 7f94c7a00640 1 -- 172.21.15.86:0/2663212243 >> 172.21.15.86:0/2663212243 conn(0x7f94c00b4650 msgr2=0x7f94c0073ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:02.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.462+0000 7f94c7a00640 1 -- 172.21.15.86:0/2663212243 shutdown_connections 2024-04-25T14:10:02.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.462+0000 7f94c7a00640 1 -- 172.21.15.86:0/2663212243 wait complete. 2024-04-25T14:10:02.478 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:10:02.479 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:10:02.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.634+0000 7f2400800640 1 Processor -- start 2024-04-25T14:10:02.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.634+0000 7f2400800640 1 -- start start 2024-04-25T14:10:02.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.634+0000 7f2400800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f23f8071530 0x7f23f806e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:02.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.634+0000 7f2400800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f23f806e7a0 0x7f23f80cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:02.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.634+0000 7f2400800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23f806ef90 con 0x7f23f80cf970 2024-04-25T14:10:02.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.634+0000 7f2400800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f23f8163830 con 0x7f23f806e7a0 2024-04-25T14:10:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.634+0000 7f2400800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f23f8163a80 con 0x7f23f8071530 2024-04-25T14:10:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.634+0000 7f23fde00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f23f80cf970 0x7f23f8070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41866/0 (socket says 172.21.15.86:41866) 2024-04-25T14:10:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.634+0000 7f23fde00640 1 -- 172.21.15.86:0/878753698 learned_addr learned my addr 172.21.15.86:0/878753698 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.634+0000 7f23fe800640 1 --2- 172.21.15.86:0/878753698 >> v2:172.21.15.86:3300/0 conn(0x7f23f806e7a0 0x7f23f80cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.634+0000 7f23fd400640 1 --2- 172.21.15.86:0/878753698 >> v2:172.21.15.155:3300/0 conn(0x7f23f8071530 0x7f23f806e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.634+0000 7f23fe800640 1 -- 172.21.15.86:0/878753698 >> v2:172.21.15.155:3300/0 conn(0x7f23f8071530 msgr2=0x7f23f806e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.634+0000 7f23fe800640 1 --2- 172.21.15.86:0/878753698 >> v2:172.21.15.155:3300/0 conn(0x7f23f8071530 0x7f23f806e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.634+0000 7f23fe800640 1 -- 172.21.15.86:0/878753698 >> v1:172.21.15.86:6789/0 conn(0x7f23f80cf970 legacy=0x7f23f8070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.634+0000 7f23fe800640 1 -- 172.21.15.86:0/878753698 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f8163d60 con 0x7f23f806e7a0 2024-04-25T14:10:02.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.638+0000 7f23fe800640 1 --2- 172.21.15.86:0/878753698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23f806e7a0 0x7f23f80cdde0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f23f81639b0 tx=0x7f23f401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8fc86d8921b1ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:02.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.638+0000 7f23fca00640 1 -- 172.21.15.86:0/878753698 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f40b2070 con 0x7f23f806e7a0 2024-04-25T14:10:02.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.638+0000 7f23fca00640 1 -- 172.21.15.86:0/878753698 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23f40a59e0 con 0x7f23f806e7a0 2024-04-25T14:10:02.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.638+0000 7f23fca00640 1 -- 172.21.15.86:0/878753698 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f40a5db0 con 0x7f23f806e7a0 2024-04-25T14:10:02.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.638+0000 7f2400800640 1 -- 172.21.15.86:0/878753698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23f806e7a0 msgr2=0x7f23f80cdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:02.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.638+0000 7f2400800640 1 --2- 172.21.15.86:0/878753698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23f806e7a0 0x7f23f80cdde0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f23f81639b0 tx=0x7f23f401c900 comp rx=0 tx=0).stop 2024-04-25T14:10:02.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.638+0000 7f2400800640 1 -- 172.21.15.86:0/878753698 shutdown_connections 2024-04-25T14:10:02.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.638+0000 7f2400800640 1 --2- 172.21.15.86:0/878753698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23f806e7a0 0x7f23f80cdde0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:02.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.638+0000 7f2400800640 1 --2- 172.21.15.86:0/878753698 >> v2:172.21.15.155:3300/0 conn(0x7f23f8071530 0x7f23f806e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:02.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.638+0000 7f2400800640 1 -- 172.21.15.86:0/878753698 >> 172.21.15.86:0/878753698 conn(0x7f23f80cffc0 msgr2=0x7f23f80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:02.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.638+0000 7f2400800640 1 -- 172.21.15.86:0/878753698 shutdown_connections 2024-04-25T14:10:02.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.638+0000 7f2400800640 1 -- 172.21.15.86:0/878753698 wait complete. 2024-04-25T14:10:02.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.642+0000 7f2400800640 1 Processor -- start 2024-04-25T14:10:02.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.642+0000 7f2400800640 1 -- start start 2024-04-25T14:10:02.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.642+0000 7f2400800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23f806e7a0 0x7f23f815ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:02.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.642+0000 7f2400800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23f80cf970 0x7f23f815f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:02.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.642+0000 7f2400800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23f8156ae0 0x7f23f815a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:02.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.642+0000 7f2400800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f23f816cbd0 con 0x7f23f80cf970 2024-04-25T14:10:02.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.642+0000 7f2400800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f23f80ced40 con 0x7f23f8156ae0 2024-04-25T14:10:02.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.642+0000 7f2400800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f23f816cd50 con 0x7f23f806e7a0 2024-04-25T14:10:02.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.642+0000 7f23fd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23f80cf970 0x7f23f815f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:02.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.642+0000 7f23fd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23f80cf970 0x7f23f815f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49512/0 (socket says 172.21.15.86:49512) 2024-04-25T14:10:02.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.642+0000 7f23fd400640 1 -- 172.21.15.86:0/610954252 learned_addr learned my addr 172.21.15.86:0/610954252 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:02.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.642+0000 7f23fe800640 1 --2- 172.21.15.86:0/610954252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23f8156ae0 0x7f23f815a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:02.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.642+0000 7f23fde00640 1 --2- 172.21.15.86:0/610954252 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23f806e7a0 0x7f23f815ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:02.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.642+0000 7f23fd400640 1 -- 172.21.15.86:0/610954252 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23f806e7a0 msgr2=0x7f23f815ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:02.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.642+0000 7f23fd400640 1 --2- 172.21.15.86:0/610954252 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23f806e7a0 0x7f23f815ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:02.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.642+0000 7f23fd400640 1 -- 172.21.15.86:0/610954252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23f8156ae0 msgr2=0x7f23f815a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:02.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.642+0000 7f23fd400640 1 --2- 172.21.15.86:0/610954252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23f8156ae0 0x7f23f815a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:02.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.642+0000 7f23fd400640 1 -- 172.21.15.86:0/610954252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f8157820 con 0x7f23f80cf970 2024-04-25T14:10:02.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.642+0000 7f23fe800640 1 --2- 172.21.15.86:0/610954252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23f8156ae0 0x7f23f815a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:02.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.642+0000 7f23fde00640 1 --2- 172.21.15.86:0/610954252 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23f806e7a0 0x7f23f815ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:02.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.646+0000 7f23fd400640 1 --2- 172.21.15.86:0/610954252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23f80cf970 0x7f23f815f0f0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f23e800e870 tx=0x7f23e800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:02.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.646+0000 7f23f2000640 1 -- 172.21.15.86:0/610954252 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e8017070 con 0x7f23f80cf970 2024-04-25T14:10:02.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.646+0000 7f23f2000640 1 -- 172.21.15.86:0/610954252 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23e800c940 con 0x7f23f80cf970 2024-04-25T14:10:02.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.646+0000 7f23f2000640 1 -- 172.21.15.86:0/610954252 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e800ccf0 con 0x7f23f80cf970 2024-04-25T14:10:02.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.646+0000 7f2400800640 1 -- 172.21.15.86:0/610954252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f80cd3d0 con 0x7f23f80cf970 2024-04-25T14:10:02.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.646+0000 7f2400800640 1 -- 172.21.15.86:0/610954252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f80b63b0 con 0x7f23f80cf970 2024-04-25T14:10:02.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.646+0000 7f2400800640 1 -- 172.21.15.86:0/610954252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23f80720e0 con 0x7f23f80cf970 2024-04-25T14:10:02.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.646+0000 7f23f2000640 1 -- 172.21.15.86:0/610954252 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7f23e801a070 con 0x7f23f80cf970 2024-04-25T14:10:02.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.646+0000 7f23f2000640 1 --2- 172.21.15.86:0/610954252 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f23dc06f550 0x7f23dc071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:02.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.646+0000 7f23f2000640 1 -- 172.21.15.86:0/610954252 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5249+0+0 (secure 0 0 0) 0x7f23e808f590 con 0x7f23f80cf970 2024-04-25T14:10:02.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.650+0000 7f23fde00640 1 --2- 172.21.15.86:0/610954252 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f23dc06f550 0x7f23dc071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:02.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.650+0000 7f23fde00640 1 --2- 172.21.15.86:0/610954252 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f23dc06f550 0x7f23dc071a60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f23ec007fd0 tx=0x7f23ec009040 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:02.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.654+0000 7f23f2000640 1 -- 172.21.15.86:0/610954252 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f23f8157820 con 0x7f23f80cf970 2024-04-25T14:10:02.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.814+0000 7f2400800640 1 -- 172.21.15.86:0/610954252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f23f8157f90 con 0x7f23f80cf970 2024-04-25T14:10:02.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.814+0000 7f23f2000640 1 -- 172.21.15.86:0/610954252 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) ==== 93+0+10 (secure 0 0 0) 0x7f23e805c5e0 con 0x7f23f80cf970 2024-04-25T14:10:02.812 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:10:02.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.818+0000 7f2400800640 1 -- 172.21.15.86:0/610954252 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f23dc06f550 msgr2=0x7f23dc071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:02.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.818+0000 7f2400800640 1 --2- 172.21.15.86:0/610954252 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f23dc06f550 0x7f23dc071a60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f23ec007fd0 tx=0x7f23ec009040 comp rx=0 tx=0).stop 2024-04-25T14:10:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.818+0000 7f2400800640 1 -- 172.21.15.86:0/610954252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23f80cf970 msgr2=0x7f23f815f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.818+0000 7f2400800640 1 --2- 172.21.15.86:0/610954252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23f80cf970 0x7f23f815f0f0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f23e800e870 tx=0x7f23e800ed30 comp rx=0 tx=0).stop 2024-04-25T14:10:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.818+0000 7f2400800640 1 -- 172.21.15.86:0/610954252 shutdown_connections 2024-04-25T14:10:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.818+0000 7f2400800640 1 --2- 172.21.15.86:0/610954252 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f23dc06f550 0x7f23dc071a60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.818+0000 7f2400800640 1 --2- 172.21.15.86:0/610954252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23f8156ae0 0x7f23f815a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.818+0000 7f2400800640 1 --2- 172.21.15.86:0/610954252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23f80cf970 0x7f23f815f0f0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.818+0000 7f2400800640 1 --2- 172.21.15.86:0/610954252 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23f806e7a0 0x7f23f815ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.818+0000 7f2400800640 1 -- 172.21.15.86:0/610954252 >> 172.21.15.86:0/610954252 conn(0x7f23f80cffc0 msgr2=0x7f23f80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.818+0000 7f2400800640 1 -- 172.21.15.86:0/610954252 shutdown_connections 2024-04-25T14:10:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.818+0000 7f2400800640 1 -- 172.21.15.86:0/610954252 wait complete. 2024-04-25T14:10:02.834 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T14:10:02.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.982+0000 7efc2d000640 1 Processor -- start 2024-04-25T14:10:02.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.982+0000 7efc2d000640 1 -- start start 2024-04-25T14:10:02.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.986+0000 7efc2d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efc280740c0 0x7efc281676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:02.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.986+0000 7efc2d000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc28072bd0 con 0x7efc280cfca0 2024-04-25T14:10:02.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.986+0000 7efc2d000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc2806fbd0 con 0x7efc280d16e0 2024-04-25T14:10:02.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.986+0000 7efc2d000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7efc28070470 con 0x7efc280740c0 2024-04-25T14:10:02.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.986+0000 7efc26800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efc280740c0 0x7efc281676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:02.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.986+0000 7efc26800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efc280740c0 0x7efc281676d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49520/0 (socket says 172.21.15.86:49520) 2024-04-25T14:10:02.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.986+0000 7efc26800640 1 -- 172.21.15.86:0/2246921002 learned_addr learned my addr 172.21.15.86:0/2246921002 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:02.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.986+0000 7efc26800640 1 -- 172.21.15.86:0/2246921002 >> v1:172.21.15.155:6789/0 conn(0x7efc280d16e0 legacy=0x7efc280d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:02.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.986+0000 7efc26800640 1 -- 172.21.15.86:0/2246921002 >> v1:172.21.15.86:6789/0 conn(0x7efc280cfca0 legacy=0x7efc2806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:02.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.986+0000 7efc26800640 1 -- 172.21.15.86:0/2246921002 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2806feb0 con 0x7efc280740c0 2024-04-25T14:10:02.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.986+0000 7efc26800640 1 --2- 172.21.15.86:0/2246921002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc280740c0 0x7efc281676d0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7efc20009870 tx=0x7efc2001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae32813332b9319d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:02.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.990+0000 7efc25400640 1 -- 172.21.15.86:0/2246921002 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc200ab430 con 0x7efc280740c0 2024-04-25T14:10:02.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.990+0000 7efc25400640 1 -- 172.21.15.86:0/2246921002 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc200b0070 con 0x7efc280740c0 2024-04-25T14:10:02.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.990+0000 7efc25400640 1 -- 172.21.15.86:0/2246921002 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc200ab7f0 con 0x7efc280740c0 2024-04-25T14:10:02.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.990+0000 7efc2d000640 1 -- 172.21.15.86:0/2246921002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc280740c0 msgr2=0x7efc281676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:02.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.990+0000 7efc2d000640 1 --2- 172.21.15.86:0/2246921002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc280740c0 0x7efc281676d0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7efc20009870 tx=0x7efc2001c520 comp rx=0 tx=0).stop 2024-04-25T14:10:02.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.990+0000 7efc2d000640 1 -- 172.21.15.86:0/2246921002 shutdown_connections 2024-04-25T14:10:02.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.990+0000 7efc2d000640 1 --2- 172.21.15.86:0/2246921002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc280740c0 0x7efc281676d0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:02.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.990+0000 7efc2d000640 1 -- 172.21.15.86:0/2246921002 >> 172.21.15.86:0/2246921002 conn(0x7efc280b4650 msgr2=0x7efc280b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:02.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.990+0000 7efc2d000640 1 -- 172.21.15.86:0/2246921002 shutdown_connections 2024-04-25T14:10:02.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.990+0000 7efc2d000640 1 -- 172.21.15.86:0/2246921002 wait complete. 2024-04-25T14:10:02.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.990+0000 7efc2d000640 1 Processor -- start 2024-04-25T14:10:02.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.990+0000 7efc2d000640 1 -- start start 2024-04-25T14:10:02.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.994+0000 7efc2d000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc280740c0 0x7efc28159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:02.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.994+0000 7efc2d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc280cfca0 0x7efc281fbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:02.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.994+0000 7efc2d000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efc280d16e0 0x7efc281fc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:02.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.994+0000 7efc2d000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efc2816ade0 con 0x7efc280cfca0 2024-04-25T14:10:02.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.994+0000 7efc2d000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efc2816ac60 con 0x7efc280740c0 2024-04-25T14:10:02.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.994+0000 7efc2d000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efc2816af60 con 0x7efc280d16e0 2024-04-25T14:10:02.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.994+0000 7efc1d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc280cfca0 0x7efc281fbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:02.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.994+0000 7efc1d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc280cfca0 0x7efc281fbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49526/0 (socket says 172.21.15.86:49526) 2024-04-25T14:10:02.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.994+0000 7efc1d400640 1 -- 172.21.15.86:0/4139091318 learned_addr learned my addr 172.21.15.86:0/4139091318 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:02.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.994+0000 7efc25e00640 1 --2- 172.21.15.86:0/4139091318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc280740c0 0x7efc28159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:02.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.994+0000 7efc26800640 1 --2- 172.21.15.86:0/4139091318 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efc280d16e0 0x7efc281fc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:02.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.994+0000 7efc1d400640 1 -- 172.21.15.86:0/4139091318 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efc280d16e0 msgr2=0x7efc281fc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:02.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.994+0000 7efc1d400640 1 --2- 172.21.15.86:0/4139091318 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efc280d16e0 0x7efc281fc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:02.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.994+0000 7efc1d400640 1 -- 172.21.15.86:0/4139091318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc280740c0 msgr2=0x7efc28159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:02.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.994+0000 7efc1d400640 1 --2- 172.21.15.86:0/4139091318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc280740c0 0x7efc28159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:02.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.994+0000 7efc1d400640 1 -- 172.21.15.86:0/4139091318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc28159690 con 0x7efc280cfca0 2024-04-25T14:10:02.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.994+0000 7efc25e00640 1 --2- 172.21.15.86:0/4139091318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc280740c0 0x7efc28159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:02.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.994+0000 7efc26800640 1 --2- 172.21.15.86:0/4139091318 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efc280d16e0 0x7efc281fc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:02.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.994+0000 7efc1d400640 1 --2- 172.21.15.86:0/4139091318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc280cfca0 0x7efc281fbe30 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7efc1000b410 tx=0x7efc1000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:02.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.994+0000 7efc1ea00640 1 -- 172.21.15.86:0/4139091318 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc10015030 con 0x7efc280cfca0 2024-04-25T14:10:02.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.994+0000 7efc1ea00640 1 -- 172.21.15.86:0/4139091318 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc10004020 con 0x7efc280cfca0 2024-04-25T14:10:02.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.994+0000 7efc2d000640 1 -- 172.21.15.86:0/4139091318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc281fb070 con 0x7efc280cfca0 2024-04-25T14:10:02.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.994+0000 7efc1ea00640 1 -- 172.21.15.86:0/4139091318 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc10004410 con 0x7efc280cfca0 2024-04-25T14:10:02.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.994+0000 7efc2d000640 1 -- 172.21.15.86:0/4139091318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc281fb630 con 0x7efc280cfca0 2024-04-25T14:10:02.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.994+0000 7efc2d000640 1 -- 172.21.15.86:0/4139091318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc281fae00 con 0x7efc280cfca0 2024-04-25T14:10:02.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.998+0000 7efc1ea00640 1 -- 172.21.15.86:0/4139091318 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7efc10018070 con 0x7efc280cfca0 2024-04-25T14:10:02.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.998+0000 7efc1ea00640 1 --2- 172.21.15.86:0/4139091318 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7efc0406f550 0x7efc04071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:02.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.998+0000 7efc1ea00640 1 -- 172.21.15.86:0/4139091318 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5249+0+0 (secure 0 0 0) 0x7efc1008ece0 con 0x7efc280cfca0 2024-04-25T14:10:02.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.998+0000 7efc25e00640 1 --2- 172.21.15.86:0/4139091318 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7efc0406f550 0x7efc04071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:02.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:02.998+0000 7efc25e00640 1 --2- 172.21.15.86:0/4139091318 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7efc0406f550 0x7efc04071a60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efc180012f0 tx=0x7efc18009040 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:02.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.002+0000 7efc1ea00640 1 -- 172.21.15.86:0/4139091318 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efc1005bd10 con 0x7efc280cfca0 2024-04-25T14:10:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.162+0000 7efc2d000640 1 -- 172.21.15.86:0/4139091318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7efc2809f4b0 con 0x7efc280cfca0 2024-04-25T14:10:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.162+0000 7efc1ea00640 1 -- 172.21.15.86:0/4139091318 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v109) ==== 84+0+65 (secure 0 0 0) 0x7efc1005b610 con 0x7efc280cfca0 2024-04-25T14:10:03.158 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:03.158 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-04-25T14:10:03.158 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:03.158 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-04-25T14:10:03.158 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-04-25T14:10:03.158 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:03.158 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-04-25T14:10:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.162+0000 7efc2d000640 1 -- 172.21.15.86:0/4139091318 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7efc0406f550 msgr2=0x7efc04071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.162+0000 7efc2d000640 1 --2- 172.21.15.86:0/4139091318 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7efc0406f550 0x7efc04071a60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efc180012f0 tx=0x7efc18009040 comp rx=0 tx=0).stop 2024-04-25T14:10:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.162+0000 7efc2d000640 1 -- 172.21.15.86:0/4139091318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc280cfca0 msgr2=0x7efc281fbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.162+0000 7efc2d000640 1 --2- 172.21.15.86:0/4139091318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc280cfca0 0x7efc281fbe30 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7efc1000b410 tx=0x7efc1000b8d0 comp rx=0 tx=0).stop 2024-04-25T14:10:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.162+0000 7efc2d000640 1 -- 172.21.15.86:0/4139091318 shutdown_connections 2024-04-25T14:10:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.162+0000 7efc2d000640 1 --2- 172.21.15.86:0/4139091318 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7efc0406f550 0x7efc04071a60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.162+0000 7efc2d000640 1 --2- 172.21.15.86:0/4139091318 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efc280d16e0 0x7efc281fc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.162+0000 7efc2d000640 1 --2- 172.21.15.86:0/4139091318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc280cfca0 0x7efc281fbe30 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.162+0000 7efc2d000640 1 --2- 172.21.15.86:0/4139091318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc280740c0 0x7efc28159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.162+0000 7efc2d000640 1 -- 172.21.15.86:0/4139091318 >> 172.21.15.86:0/4139091318 conn(0x7efc280b4650 msgr2=0x7efc280b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.162+0000 7efc2d000640 1 -- 172.21.15.86:0/4139091318 shutdown_connections 2024-04-25T14:10:03.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.162+0000 7efc2d000640 1 -- 172.21.15.86:0/4139091318 wait complete. 2024-04-25T14:10:03.179 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T14:10:03.179 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T14:10:03.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.334+0000 7fa560200640 1 Processor -- start 2024-04-25T14:10:03.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.334+0000 7fa560200640 1 -- start start 2024-04-25T14:10:03.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.334+0000 7fa560200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa5580743c0 0x7fa5581605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:03.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.334+0000 7fa560200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa5580747d0 0x7fa558160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:03.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.334+0000 7fa560200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa55815ce60 con 0x7fa55815ca50 2024-04-25T14:10:03.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.334+0000 7fa560200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa558072b90 con 0x7fa5580747d0 2024-04-25T14:10:03.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.334+0000 7fa560200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa558072e70 con 0x7fa5580743c0 2024-04-25T14:10:03.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.334+0000 7fa55e200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa5580747d0 0x7fa558160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:03.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.334+0000 7fa55e200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa5580747d0 0x7fa558160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49528/0 (socket says 172.21.15.86:49528) 2024-04-25T14:10:03.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.334+0000 7fa55e200640 1 -- 172.21.15.86:0/2783646442 learned_addr learned my addr 172.21.15.86:0/2783646442 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:03.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.334+0000 7fa55d800640 1 --1- 172.21.15.86:0/2783646442 >> v1:172.21.15.86:6789/0 conn(0x7fa55815ca50 0x7fa558073950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41884/0 (socket says 172.21.15.86:41884) 2024-04-25T14:10:03.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.334+0000 7fa55ce00640 1 --2- 172.21.15.86:0/2783646442 >> v2:172.21.15.155:3300/0 conn(0x7fa5580743c0 0x7fa5581605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:03.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.334+0000 7fa55e200640 1 -- 172.21.15.86:0/2783646442 >> v2:172.21.15.155:3300/0 conn(0x7fa5580743c0 msgr2=0x7fa5581605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:03.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.334+0000 7fa55e200640 1 --2- 172.21.15.86:0/2783646442 >> v2:172.21.15.155:3300/0 conn(0x7fa5580743c0 0x7fa5581605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:03.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.334+0000 7fa55e200640 1 -- 172.21.15.86:0/2783646442 >> v1:172.21.15.86:6789/0 conn(0x7fa55815ca50 legacy=0x7fa558073950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:03.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.334+0000 7fa55e200640 1 -- 172.21.15.86:0/2783646442 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa558073150 con 0x7fa5580747d0 2024-04-25T14:10:03.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.334+0000 7fa55ce00640 1 --2- 172.21.15.86:0/2783646442 >> v2:172.21.15.155:3300/0 conn(0x7fa5580743c0 0x7fa5581605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:03.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.338+0000 7fa55e200640 1 --2- 172.21.15.86:0/2783646442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5580747d0 0x7fa558160b30 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fa5580d2410 tx=0x7fa55001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afdd68d8a2a9d9aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:03.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.338+0000 7fa557e00640 1 -- 172.21.15.86:0/2783646442 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5500a56c0 con 0x7fa5580747d0 2024-04-25T14:10:03.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.338+0000 7fa557e00640 1 -- 172.21.15.86:0/2783646442 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5500a5860 con 0x7fa5580747d0 2024-04-25T14:10:03.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.338+0000 7fa557e00640 1 -- 172.21.15.86:0/2783646442 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5500a5c30 con 0x7fa5580747d0 2024-04-25T14:10:03.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.338+0000 7fa560200640 1 -- 172.21.15.86:0/2783646442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5580747d0 msgr2=0x7fa558160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:03.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.338+0000 7fa560200640 1 --2- 172.21.15.86:0/2783646442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5580747d0 0x7fa558160b30 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fa5580d2410 tx=0x7fa55001c9f0 comp rx=0 tx=0).stop 2024-04-25T14:10:03.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.338+0000 7fa560200640 1 -- 172.21.15.86:0/2783646442 shutdown_connections 2024-04-25T14:10:03.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.338+0000 7fa560200640 1 --2- 172.21.15.86:0/2783646442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5580747d0 0x7fa558160b30 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:03.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.338+0000 7fa560200640 1 --2- 172.21.15.86:0/2783646442 >> v2:172.21.15.155:3300/0 conn(0x7fa5580743c0 0x7fa5581605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:03.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.338+0000 7fa560200640 1 -- 172.21.15.86:0/2783646442 >> 172.21.15.86:0/2783646442 conn(0x7fa5580b4a40 msgr2=0x7fa5580b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:03.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.338+0000 7fa560200640 1 -- 172.21.15.86:0/2783646442 shutdown_connections 2024-04-25T14:10:03.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.338+0000 7fa560200640 1 -- 172.21.15.86:0/2783646442 wait complete. 2024-04-25T14:10:03.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.342+0000 7fa560200640 1 Processor -- start 2024-04-25T14:10:03.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.342+0000 7fa560200640 1 -- start start 2024-04-25T14:10:03.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.342+0000 7fa560200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5580743c0 0x7fa55815e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:03.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.342+0000 7fa560200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5580747d0 0x7fa55815eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:03.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.342+0000 7fa560200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa55815ca50 0x7fa558165a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:03.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.342+0000 7fa560200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa558167980 con 0x7fa5580747d0 2024-04-25T14:10:03.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.342+0000 7fa560200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa55806f190 con 0x7fa5580743c0 2024-04-25T14:10:03.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.342+0000 7fa560200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa558167b00 con 0x7fa55815ca50 2024-04-25T14:10:03.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.342+0000 7fa55ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5580747d0 0x7fa55815eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:03.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.342+0000 7fa55ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5580747d0 0x7fa55815eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49542/0 (socket says 172.21.15.86:49542) 2024-04-25T14:10:03.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.342+0000 7fa55ce00640 1 -- 172.21.15.86:0/848751294 learned_addr learned my addr 172.21.15.86:0/848751294 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:03.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.342+0000 7fa55d800640 1 --2- 172.21.15.86:0/848751294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5580743c0 0x7fa55815e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:03.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.342+0000 7fa55e200640 1 --2- 172.21.15.86:0/848751294 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa55815ca50 0x7fa558165a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:03.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.342+0000 7fa55ce00640 1 -- 172.21.15.86:0/848751294 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa55815ca50 msgr2=0x7fa558165a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:03.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.342+0000 7fa55ce00640 1 --2- 172.21.15.86:0/848751294 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa55815ca50 0x7fa558165a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:03.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.342+0000 7fa55ce00640 1 -- 172.21.15.86:0/848751294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5580743c0 msgr2=0x7fa55815e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:03.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.342+0000 7fa55ce00640 1 --2- 172.21.15.86:0/848751294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5580743c0 0x7fa55815e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:03.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.342+0000 7fa55ce00640 1 -- 172.21.15.86:0/848751294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa558169980 con 0x7fa5580747d0 2024-04-25T14:10:03.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.342+0000 7fa55ce00640 1 --2- 172.21.15.86:0/848751294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5580747d0 0x7fa55815eba0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fa54400c7f0 tx=0x7fa54400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:03.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.342+0000 7fa555600640 1 -- 172.21.15.86:0/848751294 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa544015070 con 0x7fa5580747d0 2024-04-25T14:10:03.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.342+0000 7fa555600640 1 -- 172.21.15.86:0/848751294 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa544007820 con 0x7fa5580747d0 2024-04-25T14:10:03.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.342+0000 7fa555600640 1 -- 172.21.15.86:0/848751294 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa544007c10 con 0x7fa5580747d0 2024-04-25T14:10:03.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.342+0000 7fa560200640 1 -- 172.21.15.86:0/848751294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa558169c60 con 0x7fa5580747d0 2024-04-25T14:10:03.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.346+0000 7fa560200640 1 -- 172.21.15.86:0/848751294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa55816a220 con 0x7fa5580747d0 2024-04-25T14:10:03.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.346+0000 7fa560200640 1 -- 172.21.15.86:0/848751294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa558050750 con 0x7fa5580747d0 2024-04-25T14:10:03.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.346+0000 7fa555600640 1 -- 172.21.15.86:0/848751294 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7fa544018070 con 0x7fa5580747d0 2024-04-25T14:10:03.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.346+0000 7fa555600640 1 --2- 172.21.15.86:0/848751294 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fa53806f550 0x7fa538071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:03.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.346+0000 7fa555600640 1 -- 172.21.15.86:0/848751294 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5249+0+0 (secure 0 0 0) 0x7fa54408f580 con 0x7fa5580747d0 2024-04-25T14:10:03.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.346+0000 7fa55d800640 1 --2- 172.21.15.86:0/848751294 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fa53806f550 0x7fa538071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:03.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.346+0000 7fa55d800640 1 --2- 172.21.15.86:0/848751294 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fa53806f550 0x7fa538071a60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa54c000f70 tx=0x7fa54c007e90 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:03.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.350+0000 7fa555600640 1 -- 172.21.15.86:0/848751294 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa54405bd20 con 0x7fa5580747d0 2024-04-25T14:10:03.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.506+0000 7fa560200640 1 -- 172.21.15.86:0/848751294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa55815f440 con 0x7fa5580747d0 2024-04-25T14:10:03.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.710+0000 7fa555600640 1 -- 172.21.15.86:0/848751294 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v110) ==== 138+0+0 (secure 0 0 0) 0x7fa544060020 con 0x7fa5580747d0 2024-04-25T14:10:03.708 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-04-25T14:10:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.714+0000 7fa560200640 1 -- 172.21.15.86:0/848751294 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fa53806f550 msgr2=0x7fa538071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.714+0000 7fa560200640 1 --2- 172.21.15.86:0/848751294 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fa53806f550 0x7fa538071a60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa54c000f70 tx=0x7fa54c007e90 comp rx=0 tx=0).stop 2024-04-25T14:10:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.714+0000 7fa560200640 1 -- 172.21.15.86:0/848751294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5580747d0 msgr2=0x7fa55815eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.714+0000 7fa560200640 1 --2- 172.21.15.86:0/848751294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5580747d0 0x7fa55815eba0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fa54400c7f0 tx=0x7fa54400ccb0 comp rx=0 tx=0).stop 2024-04-25T14:10:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.714+0000 7fa560200640 1 -- 172.21.15.86:0/848751294 shutdown_connections 2024-04-25T14:10:03.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.714+0000 7fa560200640 1 --2- 172.21.15.86:0/848751294 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fa53806f550 0x7fa538071a60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:03.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.714+0000 7fa560200640 1 --2- 172.21.15.86:0/848751294 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa55815ca50 0x7fa558165a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:03.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.714+0000 7fa560200640 1 --2- 172.21.15.86:0/848751294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5580747d0 0x7fa55815eba0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:03.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.714+0000 7fa560200640 1 --2- 172.21.15.86:0/848751294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5580743c0 0x7fa55815e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:03.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.714+0000 7fa560200640 1 -- 172.21.15.86:0/848751294 >> 172.21.15.86:0/848751294 conn(0x7fa5580b4a40 msgr2=0x7fa55806f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:03.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.714+0000 7fa560200640 1 -- 172.21.15.86:0/848751294 shutdown_connections 2024-04-25T14:10:03.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.714+0000 7fa560200640 1 -- 172.21.15.86:0/848751294 wait complete. 2024-04-25T14:10:03.735 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T14:10:03.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.890+0000 7f3988400640 1 Processor -- start 2024-04-25T14:10:03.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.890+0000 7f3988400640 1 -- start start 2024-04-25T14:10:03.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.894+0000 7f3988400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f398015ec70 0x7f39800d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:03.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.894+0000 7f3988400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f39800d1490 0x7f398006d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:03.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.894+0000 7f3988400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39800cf8f0 con 0x7f398015e860 2024-04-25T14:10:03.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.894+0000 7f3988400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f398006cfa0 con 0x7f39800d1490 2024-04-25T14:10:03.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.894+0000 7f3988400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f398006c670 con 0x7f398015ec70 2024-04-25T14:10:03.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.894+0000 7f3986400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f39800d1490 0x7f398006d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:03.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.894+0000 7f3986400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f39800d1490 0x7f398006d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49552/0 (socket says 172.21.15.86:49552) 2024-04-25T14:10:03.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.894+0000 7f3986400640 1 -- 172.21.15.86:0/1746338656 learned_addr learned my addr 172.21.15.86:0/1746338656 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:03.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.894+0000 7f3985000640 1 --2- 172.21.15.86:0/1746338656 >> v2:172.21.15.155:3300/0 conn(0x7f398015ec70 0x7f39800d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:03.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.894+0000 7f3986400640 1 -- 172.21.15.86:0/1746338656 >> v2:172.21.15.155:3300/0 conn(0x7f398015ec70 msgr2=0x7f39800d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:03.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.894+0000 7f3986400640 1 --2- 172.21.15.86:0/1746338656 >> v2:172.21.15.155:3300/0 conn(0x7f398015ec70 0x7f39800d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:03.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.894+0000 7f3986400640 1 -- 172.21.15.86:0/1746338656 >> v1:172.21.15.155:6789/0 conn(0x7f398015e860 legacy=0x7f39800ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:03.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.894+0000 7f3986400640 1 -- 172.21.15.86:0/1746338656 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f398006c950 con 0x7f39800d1490 2024-04-25T14:10:03.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.898+0000 7f3986400640 1 --2- 172.21.15.86:0/1746338656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39800d1490 0x7f398006d430 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f39800cec60 tx=0x7f397801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557f52f3de4234d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:03.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.898+0000 7f397fe00640 1 -- 172.21.15.86:0/1746338656 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39780a5780 con 0x7f39800d1490 2024-04-25T14:10:03.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.898+0000 7f397fe00640 1 -- 172.21.15.86:0/1746338656 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39780a5920 con 0x7f39800d1490 2024-04-25T14:10:03.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.898+0000 7f397fe00640 1 -- 172.21.15.86:0/1746338656 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39780a5cf0 con 0x7f39800d1490 2024-04-25T14:10:03.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.898+0000 7f3988400640 1 -- 172.21.15.86:0/1746338656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39800d1490 msgr2=0x7f398006d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:03.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.898+0000 7f3988400640 1 --2- 172.21.15.86:0/1746338656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39800d1490 0x7f398006d430 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f39800cec60 tx=0x7f397801c930 comp rx=0 tx=0).stop 2024-04-25T14:10:03.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.898+0000 7f3988400640 1 -- 172.21.15.86:0/1746338656 shutdown_connections 2024-04-25T14:10:03.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.898+0000 7f3988400640 1 --2- 172.21.15.86:0/1746338656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39800d1490 0x7f398006d430 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:03.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.898+0000 7f3988400640 1 --2- 172.21.15.86:0/1746338656 >> v2:172.21.15.155:3300/0 conn(0x7f398015ec70 0x7f39800d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:03.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.898+0000 7f3988400640 1 -- 172.21.15.86:0/1746338656 >> 172.21.15.86:0/1746338656 conn(0x7f39800d2230 msgr2=0x7f39800d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:03.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.898+0000 7f3988400640 1 -- 172.21.15.86:0/1746338656 shutdown_connections 2024-04-25T14:10:03.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.898+0000 7f3988400640 1 -- 172.21.15.86:0/1746338656 wait complete. 2024-04-25T14:10:03.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.902+0000 7f3988400640 1 Processor -- start 2024-04-25T14:10:03.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.902+0000 7f3988400640 1 -- start start 2024-04-25T14:10:03.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.902+0000 7f3988400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39800d1490 0x7f398014d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:03.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.902+0000 7f3988400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f398015e860 0x7f3980155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:03.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.902+0000 7f3988400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f398015ec70 0x7f39800b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:03.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.902+0000 7f3988400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f398016a720 con 0x7f39800d1490 2024-04-25T14:10:03.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.902+0000 7f3988400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f39800d18a0 con 0x7f398015e860 2024-04-25T14:10:03.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.902+0000 7f3988400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f398016a8a0 con 0x7f398015ec70 2024-04-25T14:10:03.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.902+0000 7f3985a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39800d1490 0x7f398014d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:03.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.902+0000 7f3985a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39800d1490 0x7f398014d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49564/0 (socket says 172.21.15.86:49564) 2024-04-25T14:10:03.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.902+0000 7f3985a00640 1 -- 172.21.15.86:0/1158103663 learned_addr learned my addr 172.21.15.86:0/1158103663 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:03.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.902+0000 7f3985000640 1 --2- 172.21.15.86:0/1158103663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f398015e860 0x7f3980155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:03.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.902+0000 7f3986400640 1 --2- 172.21.15.86:0/1158103663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f398015ec70 0x7f39800b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:03.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.902+0000 7f3985a00640 1 -- 172.21.15.86:0/1158103663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f398015ec70 msgr2=0x7f39800b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:03.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.902+0000 7f3985a00640 1 --2- 172.21.15.86:0/1158103663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f398015ec70 0x7f39800b24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:03.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.902+0000 7f3985a00640 1 -- 172.21.15.86:0/1158103663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f398015e860 msgr2=0x7f3980155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:03.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.902+0000 7f3985a00640 1 --2- 172.21.15.86:0/1158103663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f398015e860 0x7f3980155550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:03.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.902+0000 7f3985a00640 1 -- 172.21.15.86:0/1158103663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f398006f840 con 0x7f39800d1490 2024-04-25T14:10:03.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.902+0000 7f3985a00640 1 --2- 172.21.15.86:0/1158103663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39800d1490 0x7f398014d7b0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f397400c3a0 tx=0x7f397400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:03.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.902+0000 7f397d600640 1 -- 172.21.15.86:0/1158103663 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39740052d0 con 0x7f39800d1490 2024-04-25T14:10:03.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.902+0000 7f397d600640 1 -- 172.21.15.86:0/1158103663 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3974005470 con 0x7f39800d1490 2024-04-25T14:10:03.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.902+0000 7f3988400640 1 -- 172.21.15.86:0/1158103663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f398006fd40 con 0x7f39800d1490 2024-04-25T14:10:03.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.902+0000 7f397d600640 1 -- 172.21.15.86:0/1158103663 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3974007260 con 0x7f39800d1490 2024-04-25T14:10:03.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.902+0000 7f3988400640 1 -- 172.21.15.86:0/1158103663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39800b1cc0 con 0x7f39800d1490 2024-04-25T14:10:03.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.906+0000 7f3988400640 1 -- 172.21.15.86:0/1158103663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3980074cf0 con 0x7f39800d1490 2024-04-25T14:10:03.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.906+0000 7f397d600640 1 -- 172.21.15.86:0/1158103663 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7f3974007400 con 0x7f39800d1490 2024-04-25T14:10:03.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.906+0000 7f397d600640 1 --2- 172.21.15.86:0/1158103663 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f396006f550 0x7f3960071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:03.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.906+0000 7f397d600640 1 -- 172.21.15.86:0/1158103663 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5619+0+0 (secure 0 0 0) 0x7f397408efa0 con 0x7f39800d1490 2024-04-25T14:10:03.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.906+0000 7f3985000640 1 --2- 172.21.15.86:0/1158103663 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f396006f550 0x7f3960071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:03.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.906+0000 7f3985000640 1 --2- 172.21.15.86:0/1158103663 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f396006f550 0x7f3960071a60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3980154370 tx=0x7f396c008040 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:03.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:03.910+0000 7f397d600640 1 -- 172.21.15.86:0/1158103663 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f397405b860 con 0x7f39800d1490 2024-04-25T14:10:04.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.066+0000 7f3988400640 1 -- 172.21.15.86:0/1158103663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f398006a1b0 con 0x7f39800d1490 2024-04-25T14:10:04.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.718+0000 7f397d600640 1 -- 172.21.15.86:0/1158103663 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v111) ==== 170+0+0 (secure 0 0 0) 0x7f397405fb60 con 0x7f39800d1490 2024-04-25T14:10:04.717 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-04-25T14:10:04.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.722+0000 7f3988400640 1 -- 172.21.15.86:0/1158103663 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f396006f550 msgr2=0x7f3960071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.722+0000 7f3988400640 1 --2- 172.21.15.86:0/1158103663 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f396006f550 0x7f3960071a60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3980154370 tx=0x7f396c008040 comp rx=0 tx=0).stop 2024-04-25T14:10:04.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.722+0000 7f3988400640 1 -- 172.21.15.86:0/1158103663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39800d1490 msgr2=0x7f398014d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.722+0000 7f3988400640 1 --2- 172.21.15.86:0/1158103663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39800d1490 0x7f398014d7b0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f397400c3a0 tx=0x7f397400c860 comp rx=0 tx=0).stop 2024-04-25T14:10:04.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.726+0000 7f3988400640 1 -- 172.21.15.86:0/1158103663 shutdown_connections 2024-04-25T14:10:04.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.726+0000 7f3988400640 1 --2- 172.21.15.86:0/1158103663 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f396006f550 0x7f3960071a60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.726+0000 7f3988400640 1 --2- 172.21.15.86:0/1158103663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f398015ec70 0x7f39800b24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.726+0000 7f3988400640 1 --2- 172.21.15.86:0/1158103663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f398015e860 0x7f3980155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.726+0000 7f3988400640 1 --2- 172.21.15.86:0/1158103663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39800d1490 0x7f398014d7b0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.726+0000 7f3988400640 1 -- 172.21.15.86:0/1158103663 >> 172.21.15.86:0/1158103663 conn(0x7f39800d2230 msgr2=0x7f3980150280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:04.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.726+0000 7f3988400640 1 -- 172.21.15.86:0/1158103663 shutdown_connections 2024-04-25T14:10:04.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.726+0000 7f3988400640 1 -- 172.21.15.86:0/1158103663 wait complete. 2024-04-25T14:10:04.763 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T14:10:04.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.917+0000 7f556b600640 1 Processor -- start 2024-04-25T14:10:04.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.917+0000 7f556b600640 1 -- start start 2024-04-25T14:10:04.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.917+0000 7f556b600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5564070330 0x7f55640d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.917+0000 7f556b600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f55640d1c20 0x7f556406ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.917+0000 7f556b600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f556406f270 0x7f556406f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.917+0000 7f556b600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f55640d2030 con 0x7f5564070330 2024-04-25T14:10:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.917+0000 7f556b600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f556415e5c0 con 0x7f55640d1c20 2024-04-25T14:10:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.917+0000 7f556b600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f556415e8a0 con 0x7f556406f270 2024-04-25T14:10:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.917+0000 7f5568c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5564070330 0x7f55640d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.917+0000 7f5568c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5564070330 0x7f55640d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49576/0 (socket says 172.21.15.86:49576) 2024-04-25T14:10:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.917+0000 7f5568c00640 1 -- 172.21.15.86:0/1092542283 learned_addr learned my addr 172.21.15.86:0/1092542283 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.917+0000 7f555be00640 1 --2- 172.21.15.86:0/1092542283 >> v2:172.21.15.155:3300/0 conn(0x7f55640d1c20 0x7f556406ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.917+0000 7f5569600640 1 --2- 172.21.15.86:0/1092542283 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f556406f270 0x7f556406f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.917+0000 7f5568c00640 1 -- 172.21.15.86:0/1092542283 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f556406f270 msgr2=0x7f556406f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.917+0000 7f5568c00640 1 --2- 172.21.15.86:0/1092542283 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f556406f270 0x7f556406f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.917+0000 7f5568c00640 1 -- 172.21.15.86:0/1092542283 >> v2:172.21.15.155:3300/0 conn(0x7f55640d1c20 msgr2=0x7f556406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.917+0000 7f5568c00640 1 --2- 172.21.15.86:0/1092542283 >> v2:172.21.15.155:3300/0 conn(0x7f55640d1c20 0x7f556406ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.917+0000 7f5568c00640 1 -- 172.21.15.86:0/1092542283 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556415eb80 con 0x7f5564070330 2024-04-25T14:10:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.917+0000 7f555be00640 1 --2- 172.21.15.86:0/1092542283 >> v2:172.21.15.155:3300/0 conn(0x7f55640d1c20 0x7f556406ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:04.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.921+0000 7f5568c00640 1 --2- 172.21.15.86:0/1092542283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5564070330 0x7f55640d16e0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f556415bd40 tx=0x7f555401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247d281017ae6d5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:04.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.921+0000 7f5563e00640 1 -- 172.21.15.86:0/1092542283 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55540ab600 con 0x7f5564070330 2024-04-25T14:10:04.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.921+0000 7f5563e00640 1 -- 172.21.15.86:0/1092542283 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55540b0070 con 0x7f5564070330 2024-04-25T14:10:04.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.921+0000 7f5563e00640 1 -- 172.21.15.86:0/1092542283 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55540ab9c0 con 0x7f5564070330 2024-04-25T14:10:04.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.921+0000 7f556b600640 1 -- 172.21.15.86:0/1092542283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5564070330 msgr2=0x7f55640d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.921+0000 7f556b600640 1 --2- 172.21.15.86:0/1092542283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5564070330 0x7f55640d16e0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f556415bd40 tx=0x7f555401c460 comp rx=0 tx=0).stop 2024-04-25T14:10:04.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.921+0000 7f556b600640 1 -- 172.21.15.86:0/1092542283 shutdown_connections 2024-04-25T14:10:04.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.921+0000 7f556b600640 1 --2- 172.21.15.86:0/1092542283 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f556406f270 0x7f556406f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.921+0000 7f556b600640 1 --2- 172.21.15.86:0/1092542283 >> v2:172.21.15.155:3300/0 conn(0x7f55640d1c20 0x7f556406ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.921+0000 7f556b600640 1 --2- 172.21.15.86:0/1092542283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5564070330 0x7f55640d16e0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.921+0000 7f556b600640 1 -- 172.21.15.86:0/1092542283 >> 172.21.15.86:0/1092542283 conn(0x7f55640b4650 msgr2=0x7f55640b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:04.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.921+0000 7f556b600640 1 -- 172.21.15.86:0/1092542283 shutdown_connections 2024-04-25T14:10:04.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.921+0000 7f556b600640 1 -- 172.21.15.86:0/1092542283 wait complete. 2024-04-25T14:10:04.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.925+0000 7f556b600640 1 Processor -- start 2024-04-25T14:10:04.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.925+0000 7f556b600640 1 -- start start 2024-04-25T14:10:04.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.925+0000 7f556b600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f556406f270 0x7f55640cef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.925+0000 7f556b600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5564070330 0x7f55640cf480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.925+0000 7f556b600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55640d1c20 0x7f55641f93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.925+0000 7f556b600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f556415f4b0 con 0x7f556406f270 2024-04-25T14:10:04.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.925+0000 7f556b600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f556415f330 con 0x7f5564070330 2024-04-25T14:10:04.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.925+0000 7f556b600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5564070740 con 0x7f55640d1c20 2024-04-25T14:10:04.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.925+0000 7f5568c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f556406f270 0x7f55640cef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.925+0000 7f5568c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f556406f270 0x7f55640cef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49582/0 (socket says 172.21.15.86:49582) 2024-04-25T14:10:04.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.925+0000 7f5568c00640 1 -- 172.21.15.86:0/1445881303 learned_addr learned my addr 172.21.15.86:0/1445881303 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:04.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.925+0000 7f5568c00640 1 -- 172.21.15.86:0/1445881303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55640d1c20 msgr2=0x7f55641f93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.925+0000 7f5569600640 1 --2- 172.21.15.86:0/1445881303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55640d1c20 0x7f55641f93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.925+0000 7f555be00640 1 --2- 172.21.15.86:0/1445881303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5564070330 0x7f55640cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.925+0000 7f5568c00640 1 --2- 172.21.15.86:0/1445881303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55640d1c20 0x7f55641f93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.925+0000 7f5568c00640 1 -- 172.21.15.86:0/1445881303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5564070330 msgr2=0x7f55640cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.925+0000 7f5568c00640 1 --2- 172.21.15.86:0/1445881303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5564070330 0x7f55640cf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.925+0000 7f5568c00640 1 -- 172.21.15.86:0/1445881303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55641f9020 con 0x7f556406f270 2024-04-25T14:10:04.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.925+0000 7f5569600640 1 --2- 172.21.15.86:0/1445881303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55640d1c20 0x7f55641f93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:04.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.925+0000 7f555be00640 1 --2- 172.21.15.86:0/1445881303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5564070330 0x7f55640cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:04.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.929+0000 7f5568c00640 1 --2- 172.21.15.86:0/1445881303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f556406f270 0x7f55640cef40 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f555401ab10 tx=0x7f555401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:04.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.929+0000 7f5561600640 1 -- 172.21.15.86:0/1445881303 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55540be5f0 con 0x7f556406f270 2024-04-25T14:10:04.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.929+0000 7f5561600640 1 -- 172.21.15.86:0/1445881303 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55540be790 con 0x7f556406f270 2024-04-25T14:10:04.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.929+0000 7f5561600640 1 -- 172.21.15.86:0/1445881303 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55540beb80 con 0x7f556406f270 2024-04-25T14:10:04.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.929+0000 7f556b600640 1 -- 172.21.15.86:0/1445881303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55640d0840 con 0x7f556406f270 2024-04-25T14:10:04.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.929+0000 7f556b600640 1 -- 172.21.15.86:0/1445881303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55640cffa0 con 0x7f556406f270 2024-04-25T14:10:04.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.929+0000 7f556b600640 1 -- 172.21.15.86:0/1445881303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5564069d00 con 0x7f556406f270 2024-04-25T14:10:04.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.929+0000 7f5561600640 1 -- 172.21.15.86:0/1445881303 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7f55540aa070 con 0x7f556406f270 2024-04-25T14:10:04.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.933+0000 7f5561600640 1 --2- 172.21.15.86:0/1445881303 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f554006f550 0x7f5540071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.933+0000 7f5561600640 1 -- 172.21.15.86:0/1445881303 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6001+0+0 (secure 0 0 0) 0x7f55541299c0 con 0x7f556406f270 2024-04-25T14:10:04.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.933+0000 7f555be00640 1 --2- 172.21.15.86:0/1445881303 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f554006f550 0x7f5540071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.933+0000 7f555be00640 1 --2- 172.21.15.86:0/1445881303 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f554006f550 0x7f5540071a60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f556415de20 tx=0x7f554c008040 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:04.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:04.937+0000 7f5561600640 1 -- 172.21.15.86:0/1445881303 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f55641f9020 con 0x7f556406f270 2024-04-25T14:10:05.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.125+0000 7f556b600640 1 -- 172.21.15.86:0/1445881303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5564076bf0 con 0x7f556406f270 2024-04-25T14:10:05.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.741+0000 7f5561600640 1 -- 172.21.15.86:0/1445881303 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v81) ==== 174+0+0 (secure 0 0 0) 0x7f5554016110 con 0x7f556406f270 2024-04-25T14:10:05.747 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 17 and data pool 18 2024-04-25T14:10:05.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.749+0000 7f556b600640 1 -- 172.21.15.86:0/1445881303 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f554006f550 msgr2=0x7f5540071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:05.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.749+0000 7f556b600640 1 --2- 172.21.15.86:0/1445881303 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f554006f550 0x7f5540071a60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f556415de20 tx=0x7f554c008040 comp rx=0 tx=0).stop 2024-04-25T14:10:05.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.749+0000 7f556b600640 1 -- 172.21.15.86:0/1445881303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f556406f270 msgr2=0x7f55640cef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:05.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.749+0000 7f556b600640 1 --2- 172.21.15.86:0/1445881303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f556406f270 0x7f55640cef40 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f555401ab10 tx=0x7f555401aca0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.753+0000 7f556b600640 1 -- 172.21.15.86:0/1445881303 shutdown_connections 2024-04-25T14:10:05.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.753+0000 7f556b600640 1 --2- 172.21.15.86:0/1445881303 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f554006f550 0x7f5540071a60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.753+0000 7f556b600640 1 --2- 172.21.15.86:0/1445881303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55640d1c20 0x7f55641f93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.753+0000 7f556b600640 1 --2- 172.21.15.86:0/1445881303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5564070330 0x7f55640cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.753+0000 7f556b600640 1 --2- 172.21.15.86:0/1445881303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f556406f270 0x7f55640cef40 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.753+0000 7f556b600640 1 -- 172.21.15.86:0/1445881303 >> 172.21.15.86:0/1445881303 conn(0x7f55640b4650 msgr2=0x7f556415b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:05.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.753+0000 7f556b600640 1 -- 172.21.15.86:0/1445881303 shutdown_connections 2024-04-25T14:10:05.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.753+0000 7f556b600640 1 -- 172.21.15.86:0/1445881303 wait complete. 2024-04-25T14:10:05.785 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:10:05.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.937+0000 7f7e6b200640 1 Processor -- start 2024-04-25T14:10:05.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.937+0000 7f7e6b200640 1 -- start start 2024-04-25T14:10:05.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.937+0000 7f7e6b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e6c15d830 0x7f7e6c0d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.937+0000 7f7e6b200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7e6c15f8b0 0x7f7e6c160ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.937+0000 7f7e6b200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e6c0cf0b0 con 0x7f7e6c0d1380 2024-04-25T14:10:05.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.937+0000 7f7e6b200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f7e6c0cf350 con 0x7f7e6c15f8b0 2024-04-25T14:10:05.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.937+0000 7f7e6b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7e6c0cf630 con 0x7f7e6c15d830 2024-04-25T14:10:05.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.937+0000 7f7e61400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7e6c0d1380 0x7f7e6c1603a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41894/0 (socket says 172.21.15.86:41894) 2024-04-25T14:10:05.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.937+0000 7f7e61400640 1 -- 172.21.15.86:0/2764636489 learned_addr learned my addr 172.21.15.86:0/2764636489 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:05.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.937+0000 7f7e6a800640 1 --2- 172.21.15.86:0/2764636489 >> v2:172.21.15.155:3300/0 conn(0x7f7e6c15f8b0 0x7f7e6c160ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.937+0000 7f7e69e00640 1 --2- 172.21.15.86:0/2764636489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e6c15d830 0x7f7e6c0d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.937+0000 7f7e69400640 1 -- 172.21.15.86:0/2764636489 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4035732933 0 0) 0x7f7e6c0cf0b0 con 0x7f7e6c0d1380 2024-04-25T14:10:05.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.941+0000 7f7e69400640 1 -- 172.21.15.86:0/2764636489 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e50003530 con 0x7f7e6c0d1380 2024-04-25T14:10:05.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.941+0000 7f7e6a800640 1 -- 172.21.15.86:0/2764636489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e6c15d830 msgr2=0x7f7e6c0d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:05.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.941+0000 7f7e6a800640 1 --2- 172.21.15.86:0/2764636489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e6c15d830 0x7f7e6c0d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.941+0000 7f7e6a800640 1 -- 172.21.15.86:0/2764636489 >> v1:172.21.15.86:6789/0 conn(0x7f7e6c0d1380 legacy=0x7f7e6c1603a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:05.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.941+0000 7f7e69400640 1 -- 172.21.15.86:0/2764636489 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2761906618 0 0) 0x7f7e50003530 con 0x7f7e6c0d1380 2024-04-25T14:10:05.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.941+0000 7f7e6a800640 1 -- 172.21.15.86:0/2764636489 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e6c15e0a0 con 0x7f7e6c15f8b0 2024-04-25T14:10:05.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.941+0000 7f7e6a800640 1 --2- 172.21.15.86:0/2764636489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e6c15f8b0 0x7f7e6c160ab0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009d90 tx=0x7f7e6401c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e995ac331c65a17a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:05.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.941+0000 7f7e69400640 1 -- 172.21.15.86:0/2764636489 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e640b2070 con 0x7f7e6c15f8b0 2024-04-25T14:10:05.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.941+0000 7f7e69400640 1 -- 172.21.15.86:0/2764636489 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e640a5510 con 0x7f7e6c15f8b0 2024-04-25T14:10:05.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.941+0000 7f7e69400640 1 -- 172.21.15.86:0/2764636489 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c001680 con 0x7f7e6c15f8b0 2024-04-25T14:10:05.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.945+0000 7f7e6b200640 1 -- 172.21.15.86:0/2764636489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e6c15f8b0 msgr2=0x7f7e6c160ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:05.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.945+0000 7f7e6b200640 1 --2- 172.21.15.86:0/2764636489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e6c15f8b0 0x7f7e6c160ab0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009d90 tx=0x7f7e6401c800 comp rx=0 tx=0).stop 2024-04-25T14:10:05.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.945+0000 7f7e6b200640 1 -- 172.21.15.86:0/2764636489 shutdown_connections 2024-04-25T14:10:05.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.945+0000 7f7e6b200640 1 --2- 172.21.15.86:0/2764636489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e6c15f8b0 0x7f7e6c160ab0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.945+0000 7f7e6b200640 1 --2- 172.21.15.86:0/2764636489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e6c15d830 0x7f7e6c0d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.945+0000 7f7e6b200640 1 -- 172.21.15.86:0/2764636489 >> 172.21.15.86:0/2764636489 conn(0x7f7e6c06c840 msgr2=0x7f7e6c06cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:05.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.945+0000 7f7e6b200640 1 -- 172.21.15.86:0/2764636489 shutdown_connections 2024-04-25T14:10:05.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.945+0000 7f7e6b200640 1 -- 172.21.15.86:0/2764636489 wait complete. 2024-04-25T14:10:05.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.945+0000 7f7e6b200640 1 Processor -- start 2024-04-25T14:10:05.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.945+0000 7f7e6b200640 1 -- start start 2024-04-25T14:10:05.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.949+0000 7f7e6b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e6c0d1380 0x7f7e6c15bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.949+0000 7f7e6b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e6c15d830 0x7f7e6c1f8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.949+0000 7f7e6b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e6c15f8b0 0x7f7e6c1512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.949+0000 7f7e6b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c15dc40 con 0x7f7e6c15d830 2024-04-25T14:10:05.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.949+0000 7f7e6b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c0b7c40 con 0x7f7e6c0d1380 2024-04-25T14:10:05.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.949+0000 7f7e6b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7e6c1f9780 con 0x7f7e6c15f8b0 2024-04-25T14:10:05.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.949+0000 7f7e61400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e6c15d830 0x7f7e6c1f8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.949+0000 7f7e61400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e6c15d830 0x7f7e6c1f8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49596/0 (socket says 172.21.15.86:49596) 2024-04-25T14:10:05.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.949+0000 7f7e61400640 1 -- 172.21.15.86:0/1479424262 learned_addr learned my addr 172.21.15.86:0/1479424262 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:05.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.949+0000 7f7e69e00640 1 --2- 172.21.15.86:0/1479424262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e6c0d1380 0x7f7e6c15bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.949+0000 7f7e6a800640 1 --2- 172.21.15.86:0/1479424262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e6c15f8b0 0x7f7e6c1512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.949+0000 7f7e61400640 1 -- 172.21.15.86:0/1479424262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e6c15f8b0 msgr2=0x7f7e6c1512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:05.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.949+0000 7f7e61400640 1 --2- 172.21.15.86:0/1479424262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e6c15f8b0 0x7f7e6c1512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.949+0000 7f7e61400640 1 -- 172.21.15.86:0/1479424262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e6c0d1380 msgr2=0x7f7e6c15bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:05.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.949+0000 7f7e61400640 1 --2- 172.21.15.86:0/1479424262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e6c0d1380 0x7f7e6c15bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.949+0000 7f7e61400640 1 -- 172.21.15.86:0/1479424262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e6c150850 con 0x7f7e6c15d830 2024-04-25T14:10:05.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.949+0000 7f7e6a800640 1 --2- 172.21.15.86:0/1479424262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e6c15f8b0 0x7f7e6c1512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:05.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.949+0000 7f7e61400640 1 --2- 172.21.15.86:0/1479424262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e6c15d830 0x7f7e6c1f8d00 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0011f0 tx=0x7f7e5c00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:05.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.949+0000 7f7e62a00640 1 -- 172.21.15.86:0/1479424262 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c017070 con 0x7f7e6c15d830 2024-04-25T14:10:05.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.949+0000 7f7e62a00640 1 -- 172.21.15.86:0/1479424262 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e5c005b00 con 0x7f7e6c15d830 2024-04-25T14:10:05.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.949+0000 7f7e6b200640 1 -- 172.21.15.86:0/1479424262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e6c15c470 con 0x7f7e6c15d830 2024-04-25T14:10:05.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.949+0000 7f7e62a00640 1 -- 172.21.15.86:0/1479424262 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c007070 con 0x7f7e6c15d830 2024-04-25T14:10:05.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.949+0000 7f7e6b200640 1 -- 172.21.15.86:0/1479424262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e6c150dd0 con 0x7f7e6c15d830 2024-04-25T14:10:05.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.949+0000 7f7e6b200640 1 -- 172.21.15.86:0/1479424262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e6c076b80 con 0x7f7e6c15d830 2024-04-25T14:10:05.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.949+0000 7f7e62a00640 1 -- 172.21.15.86:0/1479424262 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7f7e5c01a070 con 0x7f7e6c15d830 2024-04-25T14:10:05.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.953+0000 7f7e62a00640 1 --2- 172.21.15.86:0/1479424262 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f7e4806f520 0x7f7e48071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.953+0000 7f7e62a00640 1 -- 172.21.15.86:0/1479424262 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6101+0+0 (secure 0 0 0) 0x7f7e5c08e870 con 0x7f7e6c15d830 2024-04-25T14:10:05.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.953+0000 7f7e69e00640 1 --2- 172.21.15.86:0/1479424262 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f7e4806f520 0x7f7e48071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.953+0000 7f7e69e00640 1 --2- 172.21.15.86:0/1479424262 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f7e4806f520 0x7f7e48071a30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7e54009a50 tx=0x7f7e54008040 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:05.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:05.957+0000 7f7e62a00640 1 -- 172.21.15.86:0/1479424262 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7e5c05ae70 con 0x7f7e6c15d830 2024-04-25T14:10:06.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.117+0000 7f7e6b200640 1 -- 172.21.15.86:0/1479424262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7e6c15c200 con 0x7f7e6c15d830 2024-04-25T14:10:06.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.117+0000 7f7e62a00640 1 -- 172.21.15.86:0/1479424262 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+16965 (secure 0 0 0) 0x7f7e5c05f170 con 0x7f7e6c15d830 2024-04-25T14:10:06.117 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:06.117 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":112,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:10:05.129522+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:10:03.520649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T14:10:04.080912+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:10:06.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.121+0000 7f7e6b200640 1 -- 172.21.15.86:0/1479424262 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f7e4806f520 msgr2=0x7f7e48071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.121+0000 7f7e6b200640 1 --2- 172.21.15.86:0/1479424262 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f7e4806f520 0x7f7e48071a30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7e54009a50 tx=0x7f7e54008040 comp rx=0 tx=0).stop 2024-04-25T14:10:06.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.121+0000 7f7e6b200640 1 -- 172.21.15.86:0/1479424262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e6c15d830 msgr2=0x7f7e6c1f8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.121+0000 7f7e6b200640 1 --2- 172.21.15.86:0/1479424262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e6c15d830 0x7f7e6c1f8d00 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0011f0 tx=0x7f7e5c00cbe0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.121+0000 7f7e6b200640 1 -- 172.21.15.86:0/1479424262 shutdown_connections 2024-04-25T14:10:06.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.121+0000 7f7e6b200640 1 --2- 172.21.15.86:0/1479424262 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f7e4806f520 0x7f7e48071a30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.121+0000 7f7e6b200640 1 --2- 172.21.15.86:0/1479424262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e6c15f8b0 0x7f7e6c1512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.121+0000 7f7e6b200640 1 --2- 172.21.15.86:0/1479424262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e6c15d830 0x7f7e6c1f8d00 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.121+0000 7f7e6b200640 1 --2- 172.21.15.86:0/1479424262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e6c0d1380 0x7f7e6c15bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.121+0000 7f7e6b200640 1 -- 172.21.15.86:0/1479424262 >> 172.21.15.86:0/1479424262 conn(0x7f7e6c06c840 msgr2=0x7f7e6c06cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:06.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.121+0000 7f7e6b200640 1 -- 172.21.15.86:0/1479424262 shutdown_connections 2024-04-25T14:10:06.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.121+0000 7f7e6b200640 1 -- 172.21.15.86:0/1479424262 wait complete. 2024-04-25T14:10:06.139 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:10:06.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.285+0000 7f439da00640 1 Processor -- start 2024-04-25T14:10:06.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.285+0000 7f439da00640 1 -- start start 2024-04-25T14:10:06.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.289+0000 7f439da00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f43980cfca0 0x7f439806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.289+0000 7f439da00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f43980730b0 0x7f43980734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.289+0000 7f439da00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4398072bd0 con 0x7f43980d16e0 2024-04-25T14:10:06.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.289+0000 7f439da00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4398070470 con 0x7f43980cfca0 2024-04-25T14:10:06.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.289+0000 7f439da00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f439806f930 con 0x7f43980730b0 2024-04-25T14:10:06.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.289+0000 7f438e000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f43980d16e0 0x7f43980d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41910/0 (socket says 172.21.15.86:41910) 2024-04-25T14:10:06.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.289+0000 7f438e000640 1 -- 172.21.15.86:0/3065375277 learned_addr learned my addr 172.21.15.86:0/3065375277 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:06.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.289+0000 7f4396a00640 1 --2- 172.21.15.86:0/3065375277 >> v2:172.21.15.86:3300/0 conn(0x7f43980cfca0 0x7f439806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.289+0000 7f4397400640 1 --2- 172.21.15.86:0/3065375277 >> v2:172.21.15.155:3300/0 conn(0x7f43980730b0 0x7f43980734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.289+0000 7f4396a00640 1 -- 172.21.15.86:0/3065375277 >> v2:172.21.15.155:3300/0 conn(0x7f43980730b0 msgr2=0x7f43980734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:06.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.289+0000 7f4396a00640 1 --2- 172.21.15.86:0/3065375277 >> v2:172.21.15.155:3300/0 conn(0x7f43980730b0 0x7f43980734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.289+0000 7f4396a00640 1 -- 172.21.15.86:0/3065375277 >> v1:172.21.15.86:6789/0 conn(0x7f43980d16e0 legacy=0x7f43980d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.289+0000 7f4396a00640 1 -- 172.21.15.86:0/3065375277 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439806fbd0 con 0x7f43980cfca0 2024-04-25T14:10:06.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.293+0000 7f4396a00640 1 --2- 172.21.15.86:0/3065375277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43980cfca0 0x7f439806f3f0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f43980cb8c0 tx=0x7f438801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1efd8cc63454a652 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:06.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.293+0000 7f4396000640 1 -- 172.21.15.86:0/3065375277 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43880b2070 con 0x7f43980cfca0 2024-04-25T14:10:06.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.293+0000 7f4396000640 1 -- 172.21.15.86:0/3065375277 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43880a5900 con 0x7f43980cfca0 2024-04-25T14:10:06.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.293+0000 7f4396000640 1 -- 172.21.15.86:0/3065375277 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43880a5cd0 con 0x7f43980cfca0 2024-04-25T14:10:06.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.293+0000 7f439da00640 1 -- 172.21.15.86:0/3065375277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43980cfca0 msgr2=0x7f439806f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.293+0000 7f439da00640 1 --2- 172.21.15.86:0/3065375277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43980cfca0 0x7f439806f3f0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f43980cb8c0 tx=0x7f438801c930 comp rx=0 tx=0).stop 2024-04-25T14:10:06.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.293+0000 7f439da00640 1 -- 172.21.15.86:0/3065375277 shutdown_connections 2024-04-25T14:10:06.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.293+0000 7f439da00640 1 --2- 172.21.15.86:0/3065375277 >> v2:172.21.15.155:3300/0 conn(0x7f43980730b0 0x7f43980734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.293+0000 7f439da00640 1 --2- 172.21.15.86:0/3065375277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43980cfca0 0x7f439806f3f0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.293+0000 7f439da00640 1 -- 172.21.15.86:0/3065375277 >> 172.21.15.86:0/3065375277 conn(0x7f43980b4650 msgr2=0x7f43980b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:06.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.293+0000 7f439da00640 1 -- 172.21.15.86:0/3065375277 shutdown_connections 2024-04-25T14:10:06.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.293+0000 7f439da00640 1 -- 172.21.15.86:0/3065375277 wait complete. 2024-04-25T14:10:06.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.297+0000 7f439da00640 1 Processor -- start 2024-04-25T14:10:06.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.297+0000 7f439da00640 1 -- start start 2024-04-25T14:10:06.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.297+0000 7f439da00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43980730b0 0x7f43981f9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.297+0000 7f439da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43980cfca0 0x7f439809a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.297+0000 7f439da00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43980d16e0 0x7f43980afa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.297+0000 7f439da00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4398169a70 con 0x7f43980cfca0 2024-04-25T14:10:06.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.297+0000 7f439da00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f43981698f0 con 0x7f43980d16e0 2024-04-25T14:10:06.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.297+0000 7f439da00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4398169bf0 con 0x7f43980730b0 2024-04-25T14:10:06.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.297+0000 7f438e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43980cfca0 0x7f439809a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.297+0000 7f438e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43980cfca0 0x7f439809a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49612/0 (socket says 172.21.15.86:49612) 2024-04-25T14:10:06.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.297+0000 7f438e000640 1 -- 172.21.15.86:0/774277981 learned_addr learned my addr 172.21.15.86:0/774277981 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:06.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.297+0000 7f4396a00640 1 --2- 172.21.15.86:0/774277981 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43980730b0 0x7f43981f9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.297+0000 7f4397400640 1 --2- 172.21.15.86:0/774277981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43980d16e0 0x7f43980afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.297+0000 7f438e000640 1 -- 172.21.15.86:0/774277981 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43980730b0 msgr2=0x7f43981f9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.297+0000 7f438e000640 1 --2- 172.21.15.86:0/774277981 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43980730b0 0x7f43981f9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.297+0000 7f438e000640 1 -- 172.21.15.86:0/774277981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43980d16e0 msgr2=0x7f43980afa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.297+0000 7f438e000640 1 --2- 172.21.15.86:0/774277981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43980d16e0 0x7f43980afa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.297+0000 7f438e000640 1 -- 172.21.15.86:0/774277981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43980ae970 con 0x7f43980cfca0 2024-04-25T14:10:06.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.297+0000 7f4396a00640 1 --2- 172.21.15.86:0/774277981 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43980730b0 0x7f43981f9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:06.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.297+0000 7f4397400640 1 --2- 172.21.15.86:0/774277981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43980d16e0 0x7f43980afa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:06.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.297+0000 7f438e000640 1 --2- 172.21.15.86:0/774277981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43980cfca0 0x7f439809a850 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f438000c340 tx=0x7f438000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:06.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.297+0000 7f438f400640 1 -- 172.21.15.86:0/774277981 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4380017070 con 0x7f43980cfca0 2024-04-25T14:10:06.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.297+0000 7f438f400640 1 -- 172.21.15.86:0/774277981 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4380005cf0 con 0x7f43980cfca0 2024-04-25T14:10:06.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.297+0000 7f438f400640 1 -- 172.21.15.86:0/774277981 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43800072f0 con 0x7f43980cfca0 2024-04-25T14:10:06.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.297+0000 7f439da00640 1 -- 172.21.15.86:0/774277981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43980aec50 con 0x7f43980cfca0 2024-04-25T14:10:06.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.297+0000 7f439da00640 1 -- 172.21.15.86:0/774277981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43980af210 con 0x7f43980cfca0 2024-04-25T14:10:06.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.301+0000 7f439da00640 1 -- 172.21.15.86:0/774277981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43980ae6e0 con 0x7f43980cfca0 2024-04-25T14:10:06.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.301+0000 7f438f400640 1 -- 172.21.15.86:0/774277981 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7f438001a070 con 0x7f43980cfca0 2024-04-25T14:10:06.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.301+0000 7f438f400640 1 --2- 172.21.15.86:0/774277981 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f437406f520 0x7f4374071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.301+0000 7f438f400640 1 -- 172.21.15.86:0/774277981 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6101+0+0 (secure 0 0 0) 0x7f438008e9d0 con 0x7f43980cfca0 2024-04-25T14:10:06.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.301+0000 7f4396a00640 1 --2- 172.21.15.86:0/774277981 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f437406f520 0x7f4374071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.301+0000 7f4396a00640 1 --2- 172.21.15.86:0/774277981 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f437406f520 0x7f4374071a30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f43880af000 tx=0x7f43880a7070 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:06.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.305+0000 7f438f400640 1 -- 172.21.15.86:0/774277981 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f438005b580 con 0x7f43980cfca0 2024-04-25T14:10:06.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.461+0000 7f439da00640 1 -- 172.21.15.86:0/774277981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f439809f4b0 con 0x7f43980cfca0 2024-04-25T14:10:06.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.465+0000 7f438f400640 1 -- 172.21.15.86:0/774277981 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+16965 (secure 0 0 0) 0x7f438005ae80 con 0x7f43980cfca0 2024-04-25T14:10:06.462 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:06.462 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":112,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:10:05.129522+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:10:03.520649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T14:10:04.080912+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:10:06.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.465+0000 7f439da00640 1 -- 172.21.15.86:0/774277981 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f437406f520 msgr2=0x7f4374071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.465+0000 7f439da00640 1 --2- 172.21.15.86:0/774277981 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f437406f520 0x7f4374071a30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f43880af000 tx=0x7f43880a7070 comp rx=0 tx=0).stop 2024-04-25T14:10:06.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.465+0000 7f439da00640 1 -- 172.21.15.86:0/774277981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43980cfca0 msgr2=0x7f439809a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.465+0000 7f439da00640 1 --2- 172.21.15.86:0/774277981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43980cfca0 0x7f439809a850 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f438000c340 tx=0x7f438000c800 comp rx=0 tx=0).stop 2024-04-25T14:10:06.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.465+0000 7f439da00640 1 -- 172.21.15.86:0/774277981 shutdown_connections 2024-04-25T14:10:06.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.465+0000 7f439da00640 1 --2- 172.21.15.86:0/774277981 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f437406f520 0x7f4374071a30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.465+0000 7f439da00640 1 --2- 172.21.15.86:0/774277981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43980d16e0 0x7f43980afa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.465+0000 7f439da00640 1 --2- 172.21.15.86:0/774277981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43980cfca0 0x7f439809a850 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.465+0000 7f439da00640 1 --2- 172.21.15.86:0/774277981 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43980730b0 0x7f43981f9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.465+0000 7f439da00640 1 -- 172.21.15.86:0/774277981 >> 172.21.15.86:0/774277981 conn(0x7f43980b4650 msgr2=0x7f439815c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:06.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.465+0000 7f439da00640 1 -- 172.21.15.86:0/774277981 shutdown_connections 2024-04-25T14:10:06.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.465+0000 7f439da00640 1 -- 172.21.15.86:0/774277981 wait complete. 2024-04-25T14:10:06.485 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T14:10:06.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.637+0000 7fb6b7000640 1 Processor -- start 2024-04-25T14:10:06.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.637+0000 7fb6b7000640 1 -- start start 2024-04-25T14:10:06.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.637+0000 7fb6b7000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb6b00cfca0 0x7fb6b006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.637+0000 7fb6b7000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb6b00730b0 0x7fb6b00734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.637+0000 7fb6b7000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6b0072bd0 con 0x7fb6b00d16e0 2024-04-25T14:10:06.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.637+0000 7fb6b7000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb6b0070470 con 0x7fb6b00730b0 2024-04-25T14:10:06.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.637+0000 7fb6b7000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb6b006f930 con 0x7fb6b00cfca0 2024-04-25T14:10:06.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.637+0000 7fb6b5000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb6b00730b0 0x7fb6b00734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.637+0000 7fb6b5000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb6b00730b0 0x7fb6b00734c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49614/0 (socket says 172.21.15.86:49614) 2024-04-25T14:10:06.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.637+0000 7fb6b5000640 1 -- 172.21.15.86:0/375484881 learned_addr learned my addr 172.21.15.86:0/375484881 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:06.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.641+0000 7fb6afe00640 1 --2- 172.21.15.86:0/375484881 >> v2:172.21.15.155:3300/0 conn(0x7fb6b00cfca0 0x7fb6b006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.641+0000 7fb6b5000640 1 -- 172.21.15.86:0/375484881 >> v2:172.21.15.155:3300/0 conn(0x7fb6b00cfca0 msgr2=0x7fb6b006f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:06.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.641+0000 7fb6b5000640 1 --2- 172.21.15.86:0/375484881 >> v2:172.21.15.155:3300/0 conn(0x7fb6b00cfca0 0x7fb6b006f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.641+0000 7fb6b5000640 1 -- 172.21.15.86:0/375484881 >> v1:172.21.15.86:6789/0 conn(0x7fb6b00d16e0 legacy=0x7fb6b00d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.641+0000 7fb6b5000640 1 -- 172.21.15.86:0/375484881 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b006fbd0 con 0x7fb6b00730b0 2024-04-25T14:10:06.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.641+0000 7fb6b5000640 1 --2- 172.21.15.86:0/375484881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6b00730b0 0x7fb6b00734c0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fb6b00d22a0 tx=0x7fb6a801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a976b7233c0a9422 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:06.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.641+0000 7fb6af400640 1 -- 172.21.15.86:0/375484881 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a80a57d0 con 0x7fb6b00730b0 2024-04-25T14:10:06.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.641+0000 7fb6af400640 1 -- 172.21.15.86:0/375484881 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6a80a5970 con 0x7fb6b00730b0 2024-04-25T14:10:06.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.641+0000 7fb6af400640 1 -- 172.21.15.86:0/375484881 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a80a5d40 con 0x7fb6b00730b0 2024-04-25T14:10:06.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.645+0000 7fb6b7000640 1 -- 172.21.15.86:0/375484881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6b00730b0 msgr2=0x7fb6b00734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.645+0000 7fb6b7000640 1 --2- 172.21.15.86:0/375484881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6b00730b0 0x7fb6b00734c0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fb6b00d22a0 tx=0x7fb6a801c9f0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.645+0000 7fb6b7000640 1 -- 172.21.15.86:0/375484881 shutdown_connections 2024-04-25T14:10:06.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.645+0000 7fb6b7000640 1 --2- 172.21.15.86:0/375484881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6b00730b0 0x7fb6b00734c0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.645+0000 7fb6b7000640 1 --2- 172.21.15.86:0/375484881 >> v2:172.21.15.155:3300/0 conn(0x7fb6b00cfca0 0x7fb6b006f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.645+0000 7fb6b7000640 1 -- 172.21.15.86:0/375484881 >> 172.21.15.86:0/375484881 conn(0x7fb6b00b4650 msgr2=0x7fb6b00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:06.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.645+0000 7fb6b7000640 1 -- 172.21.15.86:0/375484881 shutdown_connections 2024-04-25T14:10:06.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.645+0000 7fb6b7000640 1 -- 172.21.15.86:0/375484881 wait complete. 2024-04-25T14:10:06.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.645+0000 7fb6b7000640 1 Processor -- start 2024-04-25T14:10:06.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.645+0000 7fb6b7000640 1 -- start start 2024-04-25T14:10:06.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.645+0000 7fb6b7000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6b00730b0 0x7fb6b01536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.649+0000 7fb6b7000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6b00cfca0 0x7fb6b0157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.649+0000 7fb6b7000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6b00d16e0 0x7fb6b0099650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.649+0000 7fb6b7000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b0169a70 con 0x7fb6b00730b0 2024-04-25T14:10:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.649+0000 7fb6b7000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b01698f0 con 0x7fb6b00cfca0 2024-04-25T14:10:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.649+0000 7fb6b7000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb6b0169bf0 con 0x7fb6b00d16e0 2024-04-25T14:10:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.649+0000 7fb6afe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6b00730b0 0x7fb6b01536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.649+0000 7fb6afe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6b00730b0 0x7fb6b01536f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49630/0 (socket says 172.21.15.86:49630) 2024-04-25T14:10:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.649+0000 7fb6afe00640 1 -- 172.21.15.86:0/886341101 learned_addr learned my addr 172.21.15.86:0/886341101 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.649+0000 7fb6afe00640 1 -- 172.21.15.86:0/886341101 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6b00d16e0 msgr2=0x7fb6b0099650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.649+0000 7fb6afe00640 1 --2- 172.21.15.86:0/886341101 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6b00d16e0 0x7fb6b0099650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.649+0000 7fb6afe00640 1 -- 172.21.15.86:0/886341101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6b00cfca0 msgr2=0x7fb6b0157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.649+0000 7fb6afe00640 1 --2- 172.21.15.86:0/886341101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6b00cfca0 0x7fb6b0157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.649+0000 7fb6afe00640 1 -- 172.21.15.86:0/886341101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b0153170 con 0x7fb6b00730b0 2024-04-25T14:10:06.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.649+0000 7fb6afe00640 1 --2- 172.21.15.86:0/886341101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6b00730b0 0x7fb6b01536f0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000d600 tx=0x7fb6a000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:06.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.649+0000 7fb6acc00640 1 -- 172.21.15.86:0/886341101 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0004240 con 0x7fb6b00730b0 2024-04-25T14:10:06.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.649+0000 7fb6acc00640 1 -- 172.21.15.86:0/886341101 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6a00043e0 con 0x7fb6b00730b0 2024-04-25T14:10:06.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.649+0000 7fb6b7000640 1 -- 172.21.15.86:0/886341101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b00cb470 con 0x7fb6b00730b0 2024-04-25T14:10:06.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.649+0000 7fb6acc00640 1 -- 172.21.15.86:0/886341101 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0003840 con 0x7fb6b00730b0 2024-04-25T14:10:06.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.649+0000 7fb6b7000640 1 -- 172.21.15.86:0/886341101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b00cb710 con 0x7fb6b00730b0 2024-04-25T14:10:06.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.649+0000 7fb6b7000640 1 -- 172.21.15.86:0/886341101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6b009f4b0 con 0x7fb6b00730b0 2024-04-25T14:10:06.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.649+0000 7fb6acc00640 1 -- 172.21.15.86:0/886341101 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7fb6a00039e0 con 0x7fb6b00730b0 2024-04-25T14:10:06.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.653+0000 7fb6acc00640 1 --2- 172.21.15.86:0/886341101 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fb68c06f550 0x7fb68c071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.653+0000 7fb6acc00640 1 -- 172.21.15.86:0/886341101 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6101+0+0 (secure 0 0 0) 0x7fb6a008f7e0 con 0x7fb6b00730b0 2024-04-25T14:10:06.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.653+0000 7fb6a7400640 1 --2- 172.21.15.86:0/886341101 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fb68c06f550 0x7fb68c071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.653+0000 7fb6a7400640 1 --2- 172.21.15.86:0/886341101 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fb68c06f550 0x7fb68c071a60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb698000f70 tx=0x7fb6980079e0 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:06.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.657+0000 7fb6acc00640 1 -- 172.21.15.86:0/886341101 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb6a005c4c0 con 0x7fb6b00730b0 2024-04-25T14:10:06.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:06.857+0000 7fb6b7000640 1 -- 172.21.15.86:0/886341101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb6b0069d00 con 0x7fb6b00730b0 2024-04-25T14:10:07.708 INFO:tasks.ceph.mds.b.smithi155.stderr:2024-04-25T14:10:07.711+0000 7f4dac400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T14:10:07.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.749+0000 7fb6acc00640 1 -- 172.21.15.86:0/886341101 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v84) ==== 118+0+0 (secure 0 0 0) 0x7fb6a005bdc0 con 0x7fb6b00730b0 2024-04-25T14:10:07.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.753+0000 7fb6b7000640 1 -- 172.21.15.86:0/886341101 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fb68c06f550 msgr2=0x7fb68c071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:07.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.753+0000 7fb6b7000640 1 --2- 172.21.15.86:0/886341101 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fb68c06f550 0x7fb68c071a60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb698000f70 tx=0x7fb6980079e0 comp rx=0 tx=0).stop 2024-04-25T14:10:07.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.753+0000 7fb6b7000640 1 -- 172.21.15.86:0/886341101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6b00730b0 msgr2=0x7fb6b01536f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:07.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.753+0000 7fb6b7000640 1 --2- 172.21.15.86:0/886341101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6b00730b0 0x7fb6b01536f0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000d600 tx=0x7fb6a000dac0 comp rx=0 tx=0).stop 2024-04-25T14:10:07.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.753+0000 7fb6b7000640 1 -- 172.21.15.86:0/886341101 shutdown_connections 2024-04-25T14:10:07.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.753+0000 7fb6b7000640 1 --2- 172.21.15.86:0/886341101 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fb68c06f550 0x7fb68c071a60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:07.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.753+0000 7fb6b7000640 1 --2- 172.21.15.86:0/886341101 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6b00d16e0 0x7fb6b0099650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:07.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.753+0000 7fb6b7000640 1 --2- 172.21.15.86:0/886341101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6b00cfca0 0x7fb6b0157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:07.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.753+0000 7fb6b7000640 1 --2- 172.21.15.86:0/886341101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6b00730b0 0x7fb6b01536f0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:07.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.753+0000 7fb6b7000640 1 -- 172.21.15.86:0/886341101 >> 172.21.15.86:0/886341101 conn(0x7fb6b00b4650 msgr2=0x7fb6b00cbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:07.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.753+0000 7fb6b7000640 1 -- 172.21.15.86:0/886341101 shutdown_connections 2024-04-25T14:10:07.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.753+0000 7fb6b7000640 1 -- 172.21.15.86:0/886341101 wait complete. 2024-04-25T14:10:07.775 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T14:10:07.775 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T14:10:07.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.929+0000 7fbbef800640 1 Processor -- start 2024-04-25T14:10:07.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.929+0000 7fbbef800640 1 -- start start 2024-04-25T14:10:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.933+0000 7fbbef800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbbe815c2a0 0x7fbbe80ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.933+0000 7fbbef800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbbe806fd50 0x7fbbe8070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.933+0000 7fbbef800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbe80b0a60 con 0x7fbbe806c670 2024-04-25T14:10:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.933+0000 7fbbef800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fbbe806ef90 con 0x7fbbe806fd50 2024-04-25T14:10:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.933+0000 7fbbef800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbbe806f270 con 0x7fbbe815c2a0 2024-04-25T14:10:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.933+0000 7fbbece00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbbe815c2a0 0x7fbbe80ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.933+0000 7fbbdfe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbbe806fd50 0x7fbbe8070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.933+0000 7fbbdfe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbbe806fd50 0x7fbbe8070160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:53480/0 (socket says 172.21.15.86:53480) 2024-04-25T14:10:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.933+0000 7fbbdfe00640 1 -- 172.21.15.86:0/28545911 learned_addr learned my addr 172.21.15.86:0/28545911 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.933+0000 7fbbece00640 1 --2- 172.21.15.86:0/28545911 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbbe815c2a0 0x7fbbe80ae130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:40488/0 (socket says 172.21.15.86:40488) 2024-04-25T14:10:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.933+0000 7fbbdfe00640 1 -- 172.21.15.86:0/28545911 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbbe815c2a0 msgr2=0x7fbbe80ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.933+0000 7fbbdfe00640 1 --2- 172.21.15.86:0/28545911 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbbe815c2a0 0x7fbbe80ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.933+0000 7fbbdfe00640 1 -- 172.21.15.86:0/28545911 >> v1:172.21.15.155:6789/0 conn(0x7fbbe806c670 legacy=0x7fbbe806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.933+0000 7fbbdfe00640 1 -- 172.21.15.86:0/28545911 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe806f550 con 0x7fbbe806fd50 2024-04-25T14:10:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.933+0000 7fbbece00640 1 --2- 172.21.15.86:0/28545911 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbbe815c2a0 0x7fbbe80ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:10:07.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.937+0000 7fbbdfe00640 1 --2- 172.21.15.86:0/28545911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbe806fd50 0x7fbbe8070160 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000aa10 tx=0x7fbbd001d700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf267f74f45c8815 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:07.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.937+0000 7fbbe7e00640 1 -- 172.21.15.86:0/28545911 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd00ad070 con 0x7fbbe806fd50 2024-04-25T14:10:07.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.937+0000 7fbbe7e00640 1 -- 172.21.15.86:0/28545911 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbd00a7660 con 0x7fbbe806fd50 2024-04-25T14:10:07.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.937+0000 7fbbe7e00640 1 -- 172.21.15.86:0/28545911 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd000b040 con 0x7fbbe806fd50 2024-04-25T14:10:07.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.937+0000 7fbbef800640 1 -- 172.21.15.86:0/28545911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbe806fd50 msgr2=0x7fbbe8070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:07.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.937+0000 7fbbef800640 1 --2- 172.21.15.86:0/28545911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbe806fd50 0x7fbbe8070160 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000aa10 tx=0x7fbbd001d700 comp rx=0 tx=0).stop 2024-04-25T14:10:07.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.937+0000 7fbbef800640 1 -- 172.21.15.86:0/28545911 shutdown_connections 2024-04-25T14:10:07.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.937+0000 7fbbef800640 1 --2- 172.21.15.86:0/28545911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbe806fd50 0x7fbbe8070160 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:07.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.937+0000 7fbbef800640 1 --2- 172.21.15.86:0/28545911 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbbe815c2a0 0x7fbbe80ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:07.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.937+0000 7fbbef800640 1 -- 172.21.15.86:0/28545911 >> 172.21.15.86:0/28545911 conn(0x7fbbe80b4650 msgr2=0x7fbbe80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:07.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.937+0000 7fbbef800640 1 -- 172.21.15.86:0/28545911 shutdown_connections 2024-04-25T14:10:07.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.937+0000 7fbbef800640 1 -- 172.21.15.86:0/28545911 wait complete. 2024-04-25T14:10:07.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.941+0000 7fbbef800640 1 Processor -- start 2024-04-25T14:10:07.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.941+0000 7fbbef800640 1 -- start start 2024-04-25T14:10:07.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.941+0000 7fbbef800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbe806c670 0x7fbbe809a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:07.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.941+0000 7fbbef800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbbe806fd50 0x7fbbe8157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:07.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.941+0000 7fbbef800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbe815c2a0 0x7fbbe8164490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:07.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.941+0000 7fbbef800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe815c830 con 0x7fbbe806c670 2024-04-25T14:10:07.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.941+0000 7fbbef800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe815c6b0 con 0x7fbbe815c2a0 2024-04-25T14:10:07.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.941+0000 7fbbef800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbbe80ae670 con 0x7fbbe806fd50 2024-04-25T14:10:07.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.941+0000 7fbbece00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbe806c670 0x7fbbe809a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:07.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.941+0000 7fbbece00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbe806c670 0x7fbbe809a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49642/0 (socket says 172.21.15.86:49642) 2024-04-25T14:10:07.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.941+0000 7fbbece00640 1 -- 172.21.15.86:0/3698938634 learned_addr learned my addr 172.21.15.86:0/3698938634 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:07.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.941+0000 7fbbdfe00640 1 --2- 172.21.15.86:0/3698938634 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbbe806fd50 0x7fbbe8157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:07.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.941+0000 7fbbed800640 1 --2- 172.21.15.86:0/3698938634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbe815c2a0 0x7fbbe8164490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:07.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.941+0000 7fbbece00640 1 -- 172.21.15.86:0/3698938634 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbbe806fd50 msgr2=0x7fbbe8157bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:07.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.941+0000 7fbbece00640 1 --2- 172.21.15.86:0/3698938634 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbbe806fd50 0x7fbbe8157bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:07.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.941+0000 7fbbece00640 1 -- 172.21.15.86:0/3698938634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbe815c2a0 msgr2=0x7fbbe8164490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:07.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.941+0000 7fbbece00640 1 --2- 172.21.15.86:0/3698938634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbe815c2a0 0x7fbbe8164490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:07.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.941+0000 7fbbece00640 1 -- 172.21.15.86:0/3698938634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe80b2cc0 con 0x7fbbe806c670 2024-04-25T14:10:07.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.941+0000 7fbbed800640 1 --2- 172.21.15.86:0/3698938634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbe815c2a0 0x7fbbe8164490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:07.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.941+0000 7fbbece00640 1 --2- 172.21.15.86:0/3698938634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbe806c670 0x7fbbe809a9b0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800e900 tx=0x7fbbd800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:07.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.941+0000 7fbbe5600640 1 -- 172.21.15.86:0/3698938634 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8017070 con 0x7fbbe806c670 2024-04-25T14:10:07.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.941+0000 7fbbe5600640 1 -- 172.21.15.86:0/3698938634 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbd800ca00 con 0x7fbbe806c670 2024-04-25T14:10:07.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.941+0000 7fbbe5600640 1 -- 172.21.15.86:0/3698938634 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd800cdb0 con 0x7fbbe806c670 2024-04-25T14:10:07.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.941+0000 7fbbef800640 1 -- 172.21.15.86:0/3698938634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe80b31c0 con 0x7fbbe806c670 2024-04-25T14:10:07.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.941+0000 7fbbef800640 1 -- 172.21.15.86:0/3698938634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe80b3780 con 0x7fbbe806c670 2024-04-25T14:10:07.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.945+0000 7fbbef800640 1 -- 172.21.15.86:0/3698938634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbe8069d00 con 0x7fbbe806c670 2024-04-25T14:10:07.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.945+0000 7fbbe5600640 1 -- 172.21.15.86:0/3698938634 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7fbbd801a070 con 0x7fbbe806c670 2024-04-25T14:10:07.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.945+0000 7fbbe5600640 1 --2- 172.21.15.86:0/3698938634 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fbbc406f520 0x7fbbc4071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:07.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.945+0000 7fbbe5600640 1 -- 172.21.15.86:0/3698938634 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7fbbd808f3f0 con 0x7fbbe806c670 2024-04-25T14:10:07.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.945+0000 7fbbdfe00640 1 --2- 172.21.15.86:0/3698938634 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fbbc406f520 0x7fbbc4071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:07.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.949+0000 7fbbdfe00640 1 --2- 172.21.15.86:0/3698938634 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fbbc406f520 0x7fbbc4071a30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbbd00af000 tx=0x7fbbd00a8000 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:07.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:07.953+0000 7fbbe5600640 1 -- 172.21.15.86:0/3698938634 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbbd805b9f0 con 0x7fbbe806c670 2024-04-25T14:10:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.141+0000 7fbbef800640 1 -- 172.21.15.86:0/3698938634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fbbe8076bf0 con 0x7fbbe806c670 2024-04-25T14:10:08.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.753+0000 7fbbe5600640 1 -- 172.21.15.86:0/3698938634 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v85) ==== 122+0+0 (secure 0 0 0) 0x7fbbd805fcf0 con 0x7fbbe806c670 2024-04-25T14:10:08.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.757+0000 7fbbef800640 1 -- 172.21.15.86:0/3698938634 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fbbc406f520 msgr2=0x7fbbc4071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:08.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.757+0000 7fbbef800640 1 --2- 172.21.15.86:0/3698938634 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fbbc406f520 0x7fbbc4071a30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbbd00af000 tx=0x7fbbd00a8000 comp rx=0 tx=0).stop 2024-04-25T14:10:08.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.757+0000 7fbbef800640 1 -- 172.21.15.86:0/3698938634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbe806c670 msgr2=0x7fbbe809a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:08.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.757+0000 7fbbef800640 1 --2- 172.21.15.86:0/3698938634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbe806c670 0x7fbbe809a9b0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800e900 tx=0x7fbbd800edc0 comp rx=0 tx=0).stop 2024-04-25T14:10:08.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.757+0000 7fbbef800640 1 -- 172.21.15.86:0/3698938634 shutdown_connections 2024-04-25T14:10:08.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.757+0000 7fbbef800640 1 --2- 172.21.15.86:0/3698938634 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fbbc406f520 0x7fbbc4071a30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:08.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.757+0000 7fbbef800640 1 --2- 172.21.15.86:0/3698938634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbe815c2a0 0x7fbbe8164490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:08.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.757+0000 7fbbef800640 1 --2- 172.21.15.86:0/3698938634 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbbe806fd50 0x7fbbe8157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:08.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.757+0000 7fbbef800640 1 --2- 172.21.15.86:0/3698938634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbe806c670 0x7fbbe809a9b0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:08.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.757+0000 7fbbef800640 1 -- 172.21.15.86:0/3698938634 >> 172.21.15.86:0/3698938634 conn(0x7fbbe80b4650 msgr2=0x7fbbe815e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:08.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.757+0000 7fbbef800640 1 -- 172.21.15.86:0/3698938634 shutdown_connections 2024-04-25T14:10:08.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.757+0000 7fbbef800640 1 -- 172.21.15.86:0/3698938634 wait complete. 2024-04-25T14:10:08.777 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T14:10:08.778 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:10:08.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.929+0000 7f1923c00640 1 Processor -- start 2024-04-25T14:10:08.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.929+0000 7f1923c00640 1 -- start start 2024-04-25T14:10:08.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.933+0000 7f1923c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f191c0cfca0 0x7f191c06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:08.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.933+0000 7f1923c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f191c0730b0 0x7f191c0734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:08.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.933+0000 7f1923c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f191c072bd0 con 0x7f191c0d16e0 2024-04-25T14:10:08.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.933+0000 7f1923c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f191c070470 con 0x7f191c0730b0 2024-04-25T14:10:08.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.933+0000 7f1923c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f191c06f930 con 0x7f191c0cfca0 2024-04-25T14:10:08.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.933+0000 7f1913e00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f191c0d16e0 0x7f191c0d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:52328/0 (socket says 172.21.15.86:52328) 2024-04-25T14:10:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.933+0000 7f1913e00640 1 -- 172.21.15.86:0/3662753167 learned_addr learned my addr 172.21.15.86:0/3662753167 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.933+0000 7f1921c00640 1 --2- 172.21.15.86:0/3662753167 >> v2:172.21.15.155:3300/0 conn(0x7f191c0730b0 0x7f191c0734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.933+0000 7f1921200640 1 --2- 172.21.15.86:0/3662753167 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f191c0cfca0 0x7f191c06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.933+0000 7f191be00640 1 -- 172.21.15.86:0/3662753167 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3648772468 0 0) 0x7f191c072bd0 con 0x7f191c0d16e0 2024-04-25T14:10:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.933+0000 7f191be00640 1 -- 172.21.15.86:0/3662753167 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1900005740 con 0x7f191c0d16e0 2024-04-25T14:10:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.933+0000 7f1921c00640 1 -- 172.21.15.86:0/3662753167 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f191c0cfca0 msgr2=0x7f191c06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.933+0000 7f1921c00640 1 --2- 172.21.15.86:0/3662753167 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f191c0cfca0 0x7f191c06f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.933+0000 7f1921c00640 1 -- 172.21.15.86:0/3662753167 >> v1:172.21.15.155:6789/0 conn(0x7f191c0d16e0 legacy=0x7f191c0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.933+0000 7f1921c00640 1 -- 172.21.15.86:0/3662753167 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191c06fbd0 con 0x7f191c0730b0 2024-04-25T14:10:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.933+0000 7f1921200640 1 --2- 172.21.15.86:0/3662753167 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f191c0cfca0 0x7f191c06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:10:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.937+0000 7f1921c00640 1 --2- 172.21.15.86:0/3662753167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f191c0730b0 0x7f191c0734c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1914009d90 tx=0x7f191401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=624b438aed53129a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.937+0000 7f191be00640 1 -- 172.21.15.86:0/3662753167 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19140b2070 con 0x7f191c0730b0 2024-04-25T14:10:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.937+0000 7f191be00640 1 -- 172.21.15.86:0/3662753167 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19140a5940 con 0x7f191c0730b0 2024-04-25T14:10:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.937+0000 7f191be00640 1 -- 172.21.15.86:0/3662753167 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19140a5d10 con 0x7f191c0730b0 2024-04-25T14:10:08.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.937+0000 7f1923c00640 1 -- 172.21.15.86:0/3662753167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f191c0730b0 msgr2=0x7f191c0734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:08.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.937+0000 7f1923c00640 1 --2- 172.21.15.86:0/3662753167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f191c0730b0 0x7f191c0734c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1914009d90 tx=0x7f191401c970 comp rx=0 tx=0).stop 2024-04-25T14:10:08.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.937+0000 7f1923c00640 1 -- 172.21.15.86:0/3662753167 shutdown_connections 2024-04-25T14:10:08.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.937+0000 7f1923c00640 1 --2- 172.21.15.86:0/3662753167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f191c0730b0 0x7f191c0734c0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:08.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.937+0000 7f1923c00640 1 --2- 172.21.15.86:0/3662753167 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f191c0cfca0 0x7f191c06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:08.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.937+0000 7f1923c00640 1 -- 172.21.15.86:0/3662753167 >> 172.21.15.86:0/3662753167 conn(0x7f191c0b4650 msgr2=0x7f191c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:08.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.937+0000 7f1923c00640 1 -- 172.21.15.86:0/3662753167 shutdown_connections 2024-04-25T14:10:08.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.937+0000 7f1923c00640 1 -- 172.21.15.86:0/3662753167 wait complete. 2024-04-25T14:10:08.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.941+0000 7f1923c00640 1 Processor -- start 2024-04-25T14:10:08.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.941+0000 7f1923c00640 1 -- start start 2024-04-25T14:10:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.941+0000 7f1923c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f191c0730b0 0x7f191c1f9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.941+0000 7f1923c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f191c0cfca0 0x7f191c159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.941+0000 7f1923c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f191c0d16e0 0x7f191c0af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.941+0000 7f1923c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f191c0ce170 con 0x7f191c0cfca0 2024-04-25T14:10:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.941+0000 7f1923c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f191c0cdff0 con 0x7f191c0730b0 2024-04-25T14:10:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.941+0000 7f1923c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f191c0ce2f0 con 0x7f191c0d16e0 2024-04-25T14:10:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.941+0000 7f1913e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f191c0cfca0 0x7f191c159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.941+0000 7f1913e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f191c0cfca0 0x7f191c159270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53878/0 (socket says 172.21.15.86:53878) 2024-04-25T14:10:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.941+0000 7f1921200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f191c0730b0 0x7f191c1f9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.941+0000 7f1913e00640 1 -- 172.21.15.86:0/3446920383 learned_addr learned my addr 172.21.15.86:0/3446920383 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.941+0000 7f1921c00640 1 --2- 172.21.15.86:0/3446920383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f191c0d16e0 0x7f191c0af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.941+0000 7f1913e00640 1 -- 172.21.15.86:0/3446920383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f191c0d16e0 msgr2=0x7f191c0af540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.941+0000 7f1913e00640 1 --2- 172.21.15.86:0/3446920383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f191c0d16e0 0x7f191c0af540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.941+0000 7f1913e00640 1 -- 172.21.15.86:0/3446920383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f191c0730b0 msgr2=0x7f191c1f9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.941+0000 7f1913e00640 1 --2- 172.21.15.86:0/3446920383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f191c0730b0 0x7f191c1f9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.941+0000 7f1913e00640 1 -- 172.21.15.86:0/3446920383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191c15ecd0 con 0x7f191c0cfca0 2024-04-25T14:10:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.941+0000 7f1913e00640 1 --2- 172.21.15.86:0/3446920383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f191c0cfca0 0x7f191c159270 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f190400c3f0 tx=0x7f190400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.945+0000 7f1919600640 1 -- 172.21.15.86:0/3446920383 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1904017070 con 0x7f191c0cfca0 2024-04-25T14:10:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.945+0000 7f1919600640 1 -- 172.21.15.86:0/3446920383 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1904005cf0 con 0x7f191c0cfca0 2024-04-25T14:10:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.945+0000 7f1919600640 1 -- 172.21.15.86:0/3446920383 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19040072f0 con 0x7f191c0cfca0 2024-04-25T14:10:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.945+0000 7f1923c00640 1 -- 172.21.15.86:0/3446920383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f191c15efb0 con 0x7f191c0cfca0 2024-04-25T14:10:08.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.945+0000 7f1923c00640 1 -- 172.21.15.86:0/3446920383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f191c15f570 con 0x7f191c0cfca0 2024-04-25T14:10:08.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.945+0000 7f1923c00640 1 -- 172.21.15.86:0/3446920383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f191c09f4b0 con 0x7f191c0cfca0 2024-04-25T14:10:08.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.945+0000 7f1919600640 1 -- 172.21.15.86:0/3446920383 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7f190401a070 con 0x7f191c0cfca0 2024-04-25T14:10:08.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.945+0000 7f1919600640 1 --2- 172.21.15.86:0/3446920383 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f18f806f550 0x7f18f8071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:08.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.945+0000 7f1919600640 1 -- 172.21.15.86:0/3446920383 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f190408ebd0 con 0x7f191c0cfca0 2024-04-25T14:10:08.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.945+0000 7f1921200640 1 --2- 172.21.15.86:0/3446920383 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f18f806f550 0x7f18f8071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:08.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.949+0000 7f1921200640 1 --2- 172.21.15.86:0/3446920383 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f18f806f550 0x7f18f8071a60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f190c009580 tx=0x7f190c009190 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:08.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:08.949+0000 7f1919600640 1 -- 172.21.15.86:0/3446920383 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1904021020 con 0x7f191c0cfca0 2024-04-25T14:10:09.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.137+0000 7f1923c00640 1 -- 172.21.15.86:0/3446920383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f191c069d00 con 0x7f191c0cfca0 2024-04-25T14:10:09.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.141+0000 7f1919600640 1 -- 172.21.15.86:0/3446920383 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5183 (secure 0 0 0) 0x7f190405b3d0 con 0x7f191c0cfca0 2024-04-25T14:10:09.137 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:09.137 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":85,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/374465657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":374465657},{"type":"v1","addr":"172.21.15.86:6836","nonce":374465657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8107,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1466195534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":1466195534},{"type":"v1","addr":"172.21.15.86:6837","nonce":1466195534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8119,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2431580070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2431580070},{"type":"v1","addr":"172.21.15.155:6835","nonce":2431580070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:10:05.129470+0000","modified":"2024-04-25T14:10:08.145259+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8457":{"gid":8457,"name":"b","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6833/142240252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":142240252},{"type":"v1","addr":"172.21.15.155:6833","nonce":142240252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8457,"qdb_cluster":[8457]},"id":8}]} 2024-04-25T14:10:09.138 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 85 2024-04-25T14:10:09.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.141+0000 7f1923c00640 1 -- 172.21.15.86:0/3446920383 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f18f806f550 msgr2=0x7f18f8071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:09.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.141+0000 7f1923c00640 1 --2- 172.21.15.86:0/3446920383 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f18f806f550 0x7f18f8071a60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f190c009580 tx=0x7f190c009190 comp rx=0 tx=0).stop 2024-04-25T14:10:09.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.141+0000 7f1923c00640 1 -- 172.21.15.86:0/3446920383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f191c0cfca0 msgr2=0x7f191c159270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:09.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.141+0000 7f1923c00640 1 --2- 172.21.15.86:0/3446920383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f191c0cfca0 0x7f191c159270 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f190400c3f0 tx=0x7f190400c8b0 comp rx=0 tx=0).stop 2024-04-25T14:10:09.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.141+0000 7f1923c00640 1 -- 172.21.15.86:0/3446920383 shutdown_connections 2024-04-25T14:10:09.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.141+0000 7f1923c00640 1 --2- 172.21.15.86:0/3446920383 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f18f806f550 0x7f18f8071a60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.141+0000 7f1923c00640 1 --2- 172.21.15.86:0/3446920383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f191c0d16e0 0x7f191c0af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.141+0000 7f1923c00640 1 --2- 172.21.15.86:0/3446920383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f191c0cfca0 0x7f191c159270 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.141+0000 7f1923c00640 1 --2- 172.21.15.86:0/3446920383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f191c0730b0 0x7f191c1f9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.141+0000 7f1923c00640 1 -- 172.21.15.86:0/3446920383 >> 172.21.15.86:0/3446920383 conn(0x7f191c0b4650 msgr2=0x7f191c0ceeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.141+0000 7f1923c00640 1 -- 172.21.15.86:0/3446920383 shutdown_connections 2024-04-25T14:10:09.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.141+0000 7f1923c00640 1 -- 172.21.15.86:0/3446920383 wait complete. 2024-04-25T14:10:09.159 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:10:09.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.309+0000 7f8949000640 1 Processor -- start 2024-04-25T14:10:09.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.309+0000 7f8949000640 1 -- start start 2024-04-25T14:10:09.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.309+0000 7f8949000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f89440d1df0 0x7f89440cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:09.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.309+0000 7f8949000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f894415e5c0 con 0x7f89440cdcc0 2024-04-25T14:10:09.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.309+0000 7f8949000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f894415e8a0 con 0x7f8944070330 2024-04-25T14:10:09.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.309+0000 7f8949000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f894406fd40 con 0x7f89440d1df0 2024-04-25T14:10:09.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.309+0000 7f8942800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f89440cdcc0 0x7f89440ce0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51866/0 (socket says 172.21.15.86:51866) 2024-04-25T14:10:09.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.309+0000 7f8942800640 1 -- 172.21.15.86:0/2920583823 learned_addr learned my addr 172.21.15.86:0/2920583823 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:09.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.309+0000 7f8939400640 1 --2- 172.21.15.86:0/2920583823 >> v2:172.21.15.155:3300/0 conn(0x7f89440d1df0 0x7f89440cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:09.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.309+0000 7f8941400640 1 -- 172.21.15.86:0/2920583823 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4117918265 0 0) 0x7f894415e5c0 con 0x7f89440cdcc0 2024-04-25T14:10:09.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.309+0000 7f8941400640 1 -- 172.21.15.86:0/2920583823 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8928003530 con 0x7f89440cdcc0 2024-04-25T14:10:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.309+0000 7f8941400640 1 -- 172.21.15.86:0/2920583823 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2973089530 0 0) 0x7f894415e8a0 con 0x7f8944070330 2024-04-25T14:10:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.313+0000 7f8941400640 1 -- 172.21.15.86:0/2920583823 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f894415e5c0 con 0x7f8944070330 2024-04-25T14:10:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.313+0000 7f8941400640 1 -- 172.21.15.86:0/2920583823 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1780387304 0 0) 0x7f8928003530 con 0x7f89440cdcc0 2024-04-25T14:10:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.313+0000 7f8939400640 1 -- 172.21.15.86:0/2920583823 >> v1:172.21.15.155:6789/0 conn(0x7f8944070330 legacy=0x7f89440d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.313+0000 7f8939400640 1 -- 172.21.15.86:0/2920583823 >> v1:172.21.15.86:6789/0 conn(0x7f89440cdcc0 legacy=0x7f89440ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.313+0000 7f8939400640 1 -- 172.21.15.86:0/2920583823 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894415eb80 con 0x7f89440d1df0 2024-04-25T14:10:09.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.313+0000 7f8939400640 1 --2- 172.21.15.86:0/2920583823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89440d1df0 0x7f89440cd780 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f892c0097f0 tx=0x7f892c01c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9301995e50296a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:09.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.313+0000 7f8941400640 1 -- 172.21.15.86:0/2920583823 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c0b0070 con 0x7f89440d1df0 2024-04-25T14:10:09.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.313+0000 7f8941400640 1 -- 172.21.15.86:0/2920583823 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f892c0ab400 con 0x7f89440d1df0 2024-04-25T14:10:09.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.313+0000 7f8941400640 1 -- 172.21.15.86:0/2920583823 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c0a4e60 con 0x7f89440d1df0 2024-04-25T14:10:09.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.317+0000 7f8949000640 1 -- 172.21.15.86:0/2920583823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89440d1df0 msgr2=0x7f89440cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:09.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.317+0000 7f8949000640 1 --2- 172.21.15.86:0/2920583823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89440d1df0 0x7f89440cd780 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f892c0097f0 tx=0x7f892c01c300 comp rx=0 tx=0).stop 2024-04-25T14:10:09.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.317+0000 7f8949000640 1 -- 172.21.15.86:0/2920583823 shutdown_connections 2024-04-25T14:10:09.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.317+0000 7f8949000640 1 --2- 172.21.15.86:0/2920583823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89440d1df0 0x7f89440cd780 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:09.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.317+0000 7f8949000640 1 -- 172.21.15.86:0/2920583823 >> 172.21.15.86:0/2920583823 conn(0x7f89440b4650 msgr2=0x7f89440b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:09.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.317+0000 7f8949000640 1 -- 172.21.15.86:0/2920583823 shutdown_connections 2024-04-25T14:10:09.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.317+0000 7f8949000640 1 -- 172.21.15.86:0/2920583823 wait complete. 2024-04-25T14:10:09.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.317+0000 7f8949000640 1 Processor -- start 2024-04-25T14:10:09.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.317+0000 7f8949000640 1 -- start start 2024-04-25T14:10:09.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.317+0000 7f8949000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8944070330 0x7f89440c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:09.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.317+0000 7f8949000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89440cdcc0 0x7f8944153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:09.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.317+0000 7f8949000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8944158e60 0x7f8944153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:09.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.317+0000 7f8949000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f89440708c0 con 0x7f89440cdcc0 2024-04-25T14:10:09.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.317+0000 7f8949000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8944070740 con 0x7f8944158e60 2024-04-25T14:10:09.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.317+0000 7f8949000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f894416bfd0 con 0x7f8944070330 2024-04-25T14:10:09.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.321+0000 7f8939400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89440cdcc0 0x7f8944153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:09.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.321+0000 7f8939400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89440cdcc0 0x7f8944153a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53888/0 (socket says 172.21.15.86:53888) 2024-04-25T14:10:09.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.321+0000 7f8939400640 1 -- 172.21.15.86:0/518482150 learned_addr learned my addr 172.21.15.86:0/518482150 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:09.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.321+0000 7f8942800640 1 --2- 172.21.15.86:0/518482150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8944158e60 0x7f8944153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:09.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.321+0000 7f8941e00640 1 --2- 172.21.15.86:0/518482150 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8944070330 0x7f89440c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:09.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.321+0000 7f8939400640 1 -- 172.21.15.86:0/518482150 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8944070330 msgr2=0x7f89440c1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:09.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.321+0000 7f8939400640 1 --2- 172.21.15.86:0/518482150 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8944070330 0x7f89440c1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:09.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.321+0000 7f8939400640 1 -- 172.21.15.86:0/518482150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8944158e60 msgr2=0x7f8944153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:09.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.321+0000 7f8939400640 1 --2- 172.21.15.86:0/518482150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8944158e60 0x7f8944153f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:09.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.321+0000 7f8939400640 1 -- 172.21.15.86:0/518482150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89440b3800 con 0x7f89440cdcc0 2024-04-25T14:10:09.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.321+0000 7f8939400640 1 --2- 172.21.15.86:0/518482150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89440cdcc0 0x7f8944153a40 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f892c018f10 tx=0x7f892c0190a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:09.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.321+0000 7f893aa00640 1 -- 172.21.15.86:0/518482150 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c0bb070 con 0x7f89440cdcc0 2024-04-25T14:10:09.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.321+0000 7f893aa00640 1 -- 172.21.15.86:0/518482150 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f892c0a4e60 con 0x7f89440cdcc0 2024-04-25T14:10:09.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.321+0000 7f8949000640 1 -- 172.21.15.86:0/518482150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f894409e1a0 con 0x7f89440cdcc0 2024-04-25T14:10:09.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.321+0000 7f893aa00640 1 -- 172.21.15.86:0/518482150 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c0b0040 con 0x7f89440cdcc0 2024-04-25T14:10:09.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.321+0000 7f8949000640 1 -- 172.21.15.86:0/518482150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f894409a670 con 0x7f89440cdcc0 2024-04-25T14:10:09.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.321+0000 7f8949000640 1 -- 172.21.15.86:0/518482150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f894409a960 con 0x7f89440cdcc0 2024-04-25T14:10:09.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.321+0000 7f893aa00640 1 -- 172.21.15.86:0/518482150 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7f892c0be070 con 0x7f89440cdcc0 2024-04-25T14:10:09.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.325+0000 7f893aa00640 1 --2- 172.21.15.86:0/518482150 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f892006f520 0x7f8920071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:09.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.325+0000 7f893aa00640 1 -- 172.21.15.86:0/518482150 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f892c129a00 con 0x7f89440cdcc0 2024-04-25T14:10:09.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.325+0000 7f8941e00640 1 --2- 172.21.15.86:0/518482150 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f892006f520 0x7f8920071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:09.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.325+0000 7f8941e00640 1 --2- 172.21.15.86:0/518482150 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f892006f520 0x7f8920071a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8944072200 tx=0x7f8934009040 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:09.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.329+0000 7f893aa00640 1 -- 172.21.15.86:0/518482150 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f892c015ff0 con 0x7f89440cdcc0 2024-04-25T14:10:09.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.485+0000 7f8949000640 1 -- 172.21.15.86:0/518482150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f894409e370 con 0x7f89440cdcc0 2024-04-25T14:10:09.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.489+0000 7f893aa00640 1 -- 172.21.15.86:0/518482150 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+16952 (secure 0 0 0) 0x7f892c0f65d0 con 0x7f89440cdcc0 2024-04-25T14:10:09.485 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:09.486 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":113,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:10:06.724851+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:10:03.520649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T14:10:04.080912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:10:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.489+0000 7f8949000640 1 -- 172.21.15.86:0/518482150 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f892006f520 msgr2=0x7f8920071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.489+0000 7f8949000640 1 --2- 172.21.15.86:0/518482150 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f892006f520 0x7f8920071a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8944072200 tx=0x7f8934009040 comp rx=0 tx=0).stop 2024-04-25T14:10:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.489+0000 7f8949000640 1 -- 172.21.15.86:0/518482150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89440cdcc0 msgr2=0x7f8944153a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.489+0000 7f8949000640 1 --2- 172.21.15.86:0/518482150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89440cdcc0 0x7f8944153a40 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f892c018f10 tx=0x7f892c0190a0 comp rx=0 tx=0).stop 2024-04-25T14:10:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.489+0000 7f8949000640 1 -- 172.21.15.86:0/518482150 shutdown_connections 2024-04-25T14:10:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.489+0000 7f8949000640 1 --2- 172.21.15.86:0/518482150 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f892006f520 0x7f8920071a30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.489+0000 7f8949000640 1 --2- 172.21.15.86:0/518482150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8944158e60 0x7f8944153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:09.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.489+0000 7f8949000640 1 --2- 172.21.15.86:0/518482150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89440cdcc0 0x7f8944153a40 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:09.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.489+0000 7f8949000640 1 --2- 172.21.15.86:0/518482150 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8944070330 0x7f89440c1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:09.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.489+0000 7f8949000640 1 -- 172.21.15.86:0/518482150 >> 172.21.15.86:0/518482150 conn(0x7f89440b4650 msgr2=0x7f894415d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:09.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.489+0000 7f8949000640 1 -- 172.21.15.86:0/518482150 shutdown_connections 2024-04-25T14:10:09.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.489+0000 7f8949000640 1 -- 172.21.15.86:0/518482150 wait complete. 2024-04-25T14:10:09.507 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T14:10:09.507 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:10:09.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.657+0000 7f863f400640 1 Processor -- start 2024-04-25T14:10:09.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.657+0000 7f863f400640 1 -- start start 2024-04-25T14:10:09.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.657+0000 7f863f400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8638071530 0x7f863815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:09.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.657+0000 7f863f400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86380cd8d0 con 0x7f86381616e0 2024-04-25T14:10:09.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.657+0000 7f863f400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86380ce970 con 0x7f86380d2c80 2024-04-25T14:10:09.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.657+0000 7f863f400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f86380ce720 con 0x7f8638071530 2024-04-25T14:10:09.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.657+0000 7f863d400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f86381616e0 0x7f8638161af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51868/0 (socket says 172.21.15.86:51868) 2024-04-25T14:10:09.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.657+0000 7f863d400640 1 -- 172.21.15.86:0/2872583854 learned_addr learned my addr 172.21.15.86:0/2872583854 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:09.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.657+0000 7f8637e00640 1 --2- 172.21.15.86:0/2872583854 >> v2:172.21.15.155:3300/0 conn(0x7f8638071530 0x7f863815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:09.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.657+0000 7f8637400640 1 -- 172.21.15.86:0/2872583854 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2923597400 0 0) 0x7f86380cd8d0 con 0x7f86381616e0 2024-04-25T14:10:09.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.657+0000 7f8637400640 1 -- 172.21.15.86:0/2872583854 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f861c003530 con 0x7f86381616e0 2024-04-25T14:10:09.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.657+0000 7f8637400640 1 -- 172.21.15.86:0/2872583854 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1945970449 0 0) 0x7f861c003530 con 0x7f86381616e0 2024-04-25T14:10:09.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.657+0000 7f8637400640 1 -- 172.21.15.86:0/2872583854 >> v2:172.21.15.155:3300/0 conn(0x7f8638071530 msgr2=0x7f863815dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:09.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.657+0000 7f8637400640 1 --2- 172.21.15.86:0/2872583854 >> v2:172.21.15.155:3300/0 conn(0x7f8638071530 0x7f863815dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:09.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.657+0000 7f8637400640 1 -- 172.21.15.86:0/2872583854 >> v1:172.21.15.155:6789/0 conn(0x7f86380d2c80 legacy=0x7f8638070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:09.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.657+0000 7f8637400640 1 -- 172.21.15.86:0/2872583854 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86380d1be0 con 0x7f86381616e0 2024-04-25T14:10:09.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.661+0000 7f8637400640 1 -- 172.21.15.86:0/2872583854 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f8630001680 con 0x7f86381616e0 2024-04-25T14:10:09.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.661+0000 7f8637400640 1 -- 172.21.15.86:0/2872583854 >> v1:172.21.15.86:6789/0 conn(0x7f86381616e0 legacy=0x7f8638161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:09.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.661+0000 7f8637400640 1 --2- 172.21.15.86:0/2872583854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f861c003a30 0x7f861c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:09.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.661+0000 7f8637400640 1 --2- 172.21.15.86:0/2872583854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f861c004810 0x7f861c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:09.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.661+0000 7f8637400640 1 --2- 172.21.15.86:0/2872583854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f861c005230 0x7f861c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:09.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.661+0000 7f8637400640 1 -- 172.21.15.86:0/2872583854 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f86380ce720 con 0x7f861c003a30 2024-04-25T14:10:09.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.661+0000 7f8637400640 1 -- 172.21.15.86:0/2872583854 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f861c00c1b0 con 0x7f861c004810 2024-04-25T14:10:09.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.661+0000 7f8637400640 1 -- 172.21.15.86:0/2872583854 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f861c00c3b0 con 0x7f861c005230 2024-04-25T14:10:09.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.661+0000 7f863ca00640 1 --2- 172.21.15.86:0/2872583854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f861c003a30 0x7f861c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:09.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.661+0000 7f8637e00640 1 --2- 172.21.15.86:0/2872583854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f861c004810 0x7f861c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:09.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.661+0000 7f863d400640 1 --2- 172.21.15.86:0/2872583854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f861c005230 0x7f861c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:09.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.661+0000 7f863ca00640 1 -- 172.21.15.86:0/2872583854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f861c005230 msgr2=0x7f861c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:09.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.661+0000 7f863ca00640 1 --2- 172.21.15.86:0/2872583854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f861c005230 0x7f861c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:09.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.661+0000 7f863ca00640 1 -- 172.21.15.86:0/2872583854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f861c004810 msgr2=0x7f861c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:09.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.661+0000 7f863ca00640 1 --2- 172.21.15.86:0/2872583854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f861c004810 0x7f861c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:09.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.661+0000 7f863ca00640 1 -- 172.21.15.86:0/2872583854 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861c003530 con 0x7f861c003a30 2024-04-25T14:10:09.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.661+0000 7f863d400640 1 -- 172.21.15.86:0/2872583854 reap_dead start 2024-04-25T14:10:09.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.661+0000 7f863ca00640 1 --2- 172.21.15.86:0/2872583854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f861c003a30 0x7f861c003ea0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f8628001e70 tx=0x7f862801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af99e5e503236482 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:09.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.661+0000 7f8637400640 1 -- 172.21.15.86:0/2872583854 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628001a90 con 0x7f861c003a30 2024-04-25T14:10:09.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.661+0000 7f8637400640 1 -- 172.21.15.86:0/2872583854 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8628003b80 con 0x7f861c003a30 2024-04-25T14:10:09.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.665+0000 7f8637400640 1 -- 172.21.15.86:0/2872583854 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628005080 con 0x7f861c003a30 2024-04-25T14:10:09.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.665+0000 7f863f400640 1 -- 172.21.15.86:0/2872583854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f861c003a30 msgr2=0x7f861c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:09.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.665+0000 7f863f400640 1 --2- 172.21.15.86:0/2872583854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f861c003a30 0x7f861c003ea0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f8628001e70 tx=0x7f862801c760 comp rx=0 tx=0).stop 2024-04-25T14:10:09.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.665+0000 7f863f400640 1 -- 172.21.15.86:0/2872583854 shutdown_connections 2024-04-25T14:10:09.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.665+0000 7f863f400640 1 --2- 172.21.15.86:0/2872583854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f861c003a30 0x7f861c003ea0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:09.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.665+0000 7f863f400640 1 -- 172.21.15.86:0/2872583854 >> 172.21.15.86:0/2872583854 conn(0x7f86380cffc0 msgr2=0x7f86380d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:09.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.665+0000 7f863f400640 1 -- 172.21.15.86:0/2872583854 shutdown_connections 2024-04-25T14:10:09.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.665+0000 7f863f400640 1 -- 172.21.15.86:0/2872583854 wait complete. 2024-04-25T14:10:09.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.665+0000 7f863f400640 1 Processor -- start 2024-04-25T14:10:09.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.665+0000 7f863f400640 1 -- start start 2024-04-25T14:10:09.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.669+0000 7f863f400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f861c003a30 0x7f863814e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:09.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.669+0000 7f863f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8638159c80 0x7f863815a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:09.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.669+0000 7f863f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f863815f8b0 0x7f863815fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:09.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.669+0000 7f863f400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f86380d2480 con 0x7f8638159c80 2024-04-25T14:10:09.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.669+0000 7f863f400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f86380d21a0 con 0x7f863815f8b0 2024-04-25T14:10:09.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.669+0000 7f863f400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f86380d29a0 con 0x7f861c003a30 2024-04-25T14:10:09.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.669+0000 7f8637e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8638159c80 0x7f863815a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:09.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.669+0000 7f8637e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8638159c80 0x7f863815a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53908/0 (socket says 172.21.15.86:53908) 2024-04-25T14:10:09.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.669+0000 7f8637e00640 1 -- 172.21.15.86:0/3149620687 learned_addr learned my addr 172.21.15.86:0/3149620687 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:09.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.669+0000 7f863d400640 1 --2- 172.21.15.86:0/3149620687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f863815f8b0 0x7f863815fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:09.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.669+0000 7f863ca00640 1 --2- 172.21.15.86:0/3149620687 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f861c003a30 0x7f863814e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:09.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.669+0000 7f8637e00640 1 -- 172.21.15.86:0/3149620687 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f861c003a30 msgr2=0x7f863814e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:09.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.669+0000 7f8637e00640 1 --2- 172.21.15.86:0/3149620687 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f861c003a30 0x7f863814e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:09.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.669+0000 7f8637e00640 1 -- 172.21.15.86:0/3149620687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f863815f8b0 msgr2=0x7f863815fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:09.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.669+0000 7f8637e00640 1 --2- 172.21.15.86:0/3149620687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f863815f8b0 0x7f863815fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:09.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.669+0000 7f8637e00640 1 -- 172.21.15.86:0/3149620687 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8638160200 con 0x7f8638159c80 2024-04-25T14:10:09.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.669+0000 7f8637e00640 1 --2- 172.21.15.86:0/3149620687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8638159c80 0x7f863815a090 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f8624011890 tx=0x7f8624011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:09.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.669+0000 7f8634c00640 1 -- 172.21.15.86:0/3149620687 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624005bb0 con 0x7f8638159c80 2024-04-25T14:10:09.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.669+0000 7f8634c00640 1 -- 172.21.15.86:0/3149620687 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8624005d50 con 0x7f8638159c80 2024-04-25T14:10:09.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.669+0000 7f863f400640 1 -- 172.21.15.86:0/3149620687 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f863814eea0 con 0x7f8638159c80 2024-04-25T14:10:09.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.669+0000 7f8634c00640 1 -- 172.21.15.86:0/3149620687 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862400fa60 con 0x7f8638159c80 2024-04-25T14:10:09.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.669+0000 7f863f400640 1 -- 172.21.15.86:0/3149620687 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f863815d010 con 0x7f8638159c80 2024-04-25T14:10:09.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.669+0000 7f863f400640 1 -- 172.21.15.86:0/3149620687 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86380720e0 con 0x7f8638159c80 2024-04-25T14:10:09.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.673+0000 7f8634c00640 1 -- 172.21.15.86:0/3149620687 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7f8624012040 con 0x7f8638159c80 2024-04-25T14:10:09.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.673+0000 7f8634c00640 1 --2- 172.21.15.86:0/3149620687 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f861806f550 0x7f8618071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:09.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.673+0000 7f8634c00640 1 -- 172.21.15.86:0/3149620687 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f862408fba0 con 0x7f8638159c80 2024-04-25T14:10:09.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.673+0000 7f863ca00640 1 --2- 172.21.15.86:0/3149620687 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f861806f550 0x7f8618071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:09.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.673+0000 7f863ca00640 1 --2- 172.21.15.86:0/3149620687 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f861806f550 0x7f8618071a60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8628005000 tx=0x7f8628009040 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:09.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.677+0000 7f8634c00640 1 -- 172.21.15.86:0/3149620687 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8638160200 con 0x7f8638159c80 2024-04-25T14:10:09.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.833+0000 7f863f400640 1 -- 172.21.15.86:0/3149620687 --> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8638069d60 con 0x7f861806f550 2024-04-25T14:10:09.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.841+0000 7f8634c00640 1 -- 172.21.15.86:0/3149620687 <== mgr.8190 v2:172.21.15.86:6832/1354467416 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247091 (secure 0 0 0) 0x7f8628004c90 con 0x7f861806f550 2024-04-25T14:10:09.842 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:09.844 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:10:09.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.845+0000 7f863f400640 1 -- 172.21.15.86:0/3149620687 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f861806f550 msgr2=0x7f8618071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:09.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.845+0000 7f863f400640 1 --2- 172.21.15.86:0/3149620687 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f861806f550 0x7f8618071a60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8628005000 tx=0x7f8628009040 comp rx=0 tx=0).stop 2024-04-25T14:10:09.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.845+0000 7f863f400640 1 -- 172.21.15.86:0/3149620687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8638159c80 msgr2=0x7f863815a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:09.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.845+0000 7f863f400640 1 --2- 172.21.15.86:0/3149620687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8638159c80 0x7f863815a090 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f8624011890 tx=0x7f8624011d50 comp rx=0 tx=0).stop 2024-04-25T14:10:09.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.845+0000 7f863f400640 1 -- 172.21.15.86:0/3149620687 shutdown_connections 2024-04-25T14:10:09.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.845+0000 7f863f400640 1 --2- 172.21.15.86:0/3149620687 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f861806f550 0x7f8618071a60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:09.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.845+0000 7f863f400640 1 --2- 172.21.15.86:0/3149620687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f863815f8b0 0x7f863815fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:09.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.845+0000 7f863f400640 1 --2- 172.21.15.86:0/3149620687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8638159c80 0x7f863815a090 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:09.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.845+0000 7f863f400640 1 --2- 172.21.15.86:0/3149620687 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f861c003a30 0x7f863814e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:09.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.845+0000 7f863f400640 1 -- 172.21.15.86:0/3149620687 >> 172.21.15.86:0/3149620687 conn(0x7f86380cffc0 msgr2=0x7f86380cdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:09.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.845+0000 7f863f400640 1 -- 172.21.15.86:0/3149620687 shutdown_connections 2024-04-25T14:10:09.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:09.845+0000 7f863f400640 1 -- 172.21.15.86:0/3149620687 wait complete. 2024-04-25T14:10:09.863 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T14:10:09.474871+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":468,"num_read_kb":1272,"num_write":181,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261404,"kb_used_data":11928,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713316,"statfs":{"total":773094113280,"available":772826435584,"internally_reserved":0,"allocated":12214272,"data_stored":6671076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.761213"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735771+0000","last_change":"2024-04-25T14:10:04.735771+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735771+0000","last_undegraded":"2024-04-25T14:10:04.735771+0000","last_fullsized":"2024-04-25T14:10:04.735771+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735778+0000","last_change":"2024-04-25T14:10:04.735778+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735778+0000","last_undegraded":"2024-04-25T14:10:04.735778+0000","last_fullsized":"2024-04-25T14:10:04.735778+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735849+0000","last_change":"2024-04-25T14:10:04.735849+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735849+0000","last_undegraded":"2024-04-25T14:10:04.735849+0000","last_fullsized":"2024-04-25T14:10:04.735849+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.734374+0000","last_change":"2024-04-25T14:10:04.734374+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.734374+0000","last_undegraded":"2024-04-25T14:10:04.734374+0000","last_fullsized":"2024-04-25T14:10:04.734374+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.734611+0000","last_change":"2024-04-25T14:10:04.734611+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.734611+0000","last_undegraded":"2024-04-25T14:10:04.734611+0000","last_fullsized":"2024-04-25T14:10:04.734611+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.738244+0000","last_change":"2024-04-25T14:10:04.738244+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.738244+0000","last_undegraded":"2024-04-25T14:10:04.738244+0000","last_fullsized":"2024-04-25T14:10:04.738244+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.734697+0000","last_change":"2024-04-25T14:10:04.734697+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.734697+0000","last_undegraded":"2024-04-25T14:10:04.734697+0000","last_fullsized":"2024-04-25T14:10:04.734697+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733337+0000","last_change":"2024-04-25T14:10:04.733337+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733337+0000","last_undegraded":"2024-04-25T14:10:04.733337+0000","last_fullsized":"2024-04-25T14:10:04.733337+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.739896+0000","last_change":"2024-04-25T14:10:04.739896+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.739896+0000","last_undegraded":"2024-04-25T14:10:04.739896+0000","last_fullsized":"2024-04-25T14:10:04.739896+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726838+0000","last_change":"2024-04-25T14:10:04.726838+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726838+0000","last_undegraded":"2024-04-25T14:10:04.726838+0000","last_fullsized":"2024-04-25T14:10:04.726838+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735651+0000","last_change":"2024-04-25T14:10:04.735651+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735651+0000","last_undegraded":"2024-04-25T14:10:04.735651+0000","last_fullsized":"2024-04-25T14:10:04.735651+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.742843+0000","last_change":"2024-04-25T14:10:04.742843+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.742843+0000","last_undegraded":"2024-04-25T14:10:04.742843+0000","last_fullsized":"2024-04-25T14:10:04.742843+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.727264+0000","last_change":"2024-04-25T14:10:04.727264+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.727264+0000","last_undegraded":"2024-04-25T14:10:04.727264+0000","last_fullsized":"2024-04-25T14:10:04.727264+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733351+0000","last_change":"2024-04-25T14:10:04.733351+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733351+0000","last_undegraded":"2024-04-25T14:10:04.733351+0000","last_fullsized":"2024-04-25T14:10:04.733351+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.734380+0000","last_change":"2024-04-25T14:10:04.734380+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.734380+0000","last_undegraded":"2024-04-25T14:10:04.734380+0000","last_fullsized":"2024-04-25T14:10:04.734380+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735156+0000","last_change":"2024-04-25T14:10:04.735156+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735156+0000","last_undegraded":"2024-04-25T14:10:04.735156+0000","last_fullsized":"2024-04-25T14:10:04.735156+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.725975+0000","last_change":"2024-04-25T14:10:04.725975+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.725975+0000","last_undegraded":"2024-04-25T14:10:04.725975+0000","last_fullsized":"2024-04-25T14:10:04.725975+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733263+0000","last_change":"2024-04-25T14:10:04.733263+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733263+0000","last_undegraded":"2024-04-25T14:10:04.733263+0000","last_fullsized":"2024-04-25T14:10:04.733263+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.729768+0000","last_change":"2024-04-25T14:10:04.729768+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.729768+0000","last_undegraded":"2024-04-25T14:10:04.729768+0000","last_fullsized":"2024-04-25T14:10:04.729768+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.737805+0000","last_change":"2024-04-25T14:10:04.737805+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.737805+0000","last_undegraded":"2024-04-25T14:10:04.737805+0000","last_fullsized":"2024-04-25T14:10:04.737805+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735963+0000","last_change":"2024-04-25T14:10:04.735963+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735963+0000","last_undegraded":"2024-04-25T14:10:04.735963+0000","last_fullsized":"2024-04-25T14:10:04.735963+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733684+0000","last_change":"2024-04-25T14:10:04.733684+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733684+0000","last_undegraded":"2024-04-25T14:10:04.733684+0000","last_fullsized":"2024-04-25T14:10:04.733684+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.732828+0000","last_change":"2024-04-25T14:10:04.732828+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.732828+0000","last_undegraded":"2024-04-25T14:10:04.732828+0000","last_fullsized":"2024-04-25T14:10:04.732828+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.727648+0000","last_change":"2024-04-25T14:10:04.727648+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.727648+0000","last_undegraded":"2024-04-25T14:10:04.727648+0000","last_fullsized":"2024-04-25T14:10:04.727648+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.728155+0000","last_change":"2024-04-25T14:10:04.728155+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.728155+0000","last_undegraded":"2024-04-25T14:10:04.728155+0000","last_fullsized":"2024-04-25T14:10:04.728155+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735818+0000","last_change":"2024-04-25T14:10:04.735818+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735818+0000","last_undegraded":"2024-04-25T14:10:04.735818+0000","last_fullsized":"2024-04-25T14:10:04.735818+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.729202+0000","last_change":"2024-04-25T14:10:04.729202+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.729202+0000","last_undegraded":"2024-04-25T14:10:04.729202+0000","last_fullsized":"2024-04-25T14:10:04.729202+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.738433+0000","last_change":"2024-04-25T14:10:04.738433+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.738433+0000","last_undegraded":"2024-04-25T14:10:04.738433+0000","last_fullsized":"2024-04-25T14:10:04.738433+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.737523+0000","last_change":"2024-04-25T14:10:04.737523+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.737523+0000","last_undegraded":"2024-04-25T14:10:04.737523+0000","last_fullsized":"2024-04-25T14:10:04.737523+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.732817+0000","last_change":"2024-04-25T14:10:04.732817+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.732817+0000","last_undegraded":"2024-04-25T14:10:04.732817+0000","last_fullsized":"2024-04-25T14:10:04.732817+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726801+0000","last_change":"2024-04-25T14:10:04.726801+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726801+0000","last_undegraded":"2024-04-25T14:10:04.726801+0000","last_fullsized":"2024-04-25T14:10:04.726801+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726485+0000","last_change":"2024-04-25T14:10:04.726485+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726485+0000","last_undegraded":"2024-04-25T14:10:04.726485+0000","last_fullsized":"2024-04-25T14:10:04.726485+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726867+0000","last_change":"2024-04-25T14:10:04.726867+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726867+0000","last_undegraded":"2024-04-25T14:10:04.726867+0000","last_fullsized":"2024-04-25T14:10:04.726867+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.749691+0000","last_change":"2024-04-25T14:10:04.745986+0000","last_active":"2024-04-25T14:10:04.749691+0000","last_peered":"2024-04-25T14:10:04.749691+0000","last_clean":"2024-04-25T14:10:04.749691+0000","last_became_active":"2024-04-25T14:10:04.745516+0000","last_became_peered":"2024-04-25T14:10:04.745516+0000","last_unstale":"2024-04-25T14:10:04.749691+0000","last_undegraded":"2024-04-25T14:10:04.749691+0000","last_fullsized":"2024-04-25T14:10:04.749691+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:45.118281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735537+0000","last_change":"2024-04-25T14:10:04.735537+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735537+0000","last_undegraded":"2024-04-25T14:10:04.735537+0000","last_fullsized":"2024-04-25T14:10:04.735537+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.746437+0000","last_change":"2024-04-25T14:10:04.744056+0000","last_active":"2024-04-25T14:10:04.746437+0000","last_peered":"2024-04-25T14:10:04.746437+0000","last_clean":"2024-04-25T14:10:04.746437+0000","last_became_active":"2024-04-25T14:10:04.743755+0000","last_became_peered":"2024-04-25T14:10:04.743755+0000","last_unstale":"2024-04-25T14:10:04.746437+0000","last_undegraded":"2024-04-25T14:10:04.746437+0000","last_fullsized":"2024-04-25T14:10:04.746437+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:45.118281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.728128+0000","last_change":"2024-04-25T14:10:04.728128+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.728128+0000","last_undegraded":"2024-04-25T14:10:04.728128+0000","last_fullsized":"2024-04-25T14:10:04.728128+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.755092+0000","last_change":"2024-04-25T14:10:04.752833+0000","last_active":"2024-04-25T14:10:04.755092+0000","last_peered":"2024-04-25T14:10:04.755092+0000","last_clean":"2024-04-25T14:10:04.755092+0000","last_became_active":"2024-04-25T14:10:04.752483+0000","last_became_peered":"2024-04-25T14:10:04.752483+0000","last_unstale":"2024-04-25T14:10:04.755092+0000","last_undegraded":"2024-04-25T14:10:04.755092+0000","last_fullsized":"2024-04-25T14:10:04.755092+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:50.427076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.738654+0000","last_change":"2024-04-25T14:10:04.738654+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.738654+0000","last_undegraded":"2024-04-25T14:10:04.738654+0000","last_fullsized":"2024-04-25T14:10:04.738654+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.737624+0000","last_change":"2024-04-25T14:10:04.733242+0000","last_active":"2024-04-25T14:10:04.737624+0000","last_peered":"2024-04-25T14:10:04.737624+0000","last_clean":"2024-04-25T14:10:04.737624+0000","last_became_active":"2024-04-25T14:10:04.732929+0000","last_became_peered":"2024-04-25T14:10:04.732929+0000","last_unstale":"2024-04-25T14:10:04.737624+0000","last_undegraded":"2024-04-25T14:10:04.737624+0000","last_fullsized":"2024-04-25T14:10:04.737624+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:25.644454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.725539+0000","last_change":"2024-04-25T14:10:04.725539+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.725539+0000","last_undegraded":"2024-04-25T14:10:04.725539+0000","last_fullsized":"2024-04-25T14:10:04.725539+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.744498+0000","last_change":"2024-04-25T14:10:04.738920+0000","last_active":"2024-04-25T14:10:04.744498+0000","last_peered":"2024-04-25T14:10:04.744498+0000","last_clean":"2024-04-25T14:10:04.744498+0000","last_became_active":"2024-04-25T14:10:04.738531+0000","last_became_peered":"2024-04-25T14:10:04.738531+0000","last_unstale":"2024-04-25T14:10:04.744498+0000","last_undegraded":"2024-04-25T14:10:04.744498+0000","last_fullsized":"2024-04-25T14:10:04.744498+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:45.118281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733780+0000","last_change":"2024-04-25T14:10:04.733780+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733780+0000","last_undegraded":"2024-04-25T14:10:04.733780+0000","last_fullsized":"2024-04-25T14:10:04.733780+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.754194+0000","last_change":"2024-04-25T14:10:04.749193+0000","last_active":"2024-04-25T14:10:04.754194+0000","last_peered":"2024-04-25T14:10:04.754194+0000","last_clean":"2024-04-25T14:10:04.754194+0000","last_became_active":"2024-04-25T14:10:04.747285+0000","last_became_peered":"2024-04-25T14:10:04.747285+0000","last_unstale":"2024-04-25T14:10:04.754194+0000","last_undegraded":"2024-04-25T14:10:04.754194+0000","last_fullsized":"2024-04-25T14:10:04.754194+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:19:34.506848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.732355+0000","last_change":"2024-04-25T14:10:04.732355+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.732355+0000","last_undegraded":"2024-04-25T14:10:04.732355+0000","last_fullsized":"2024-04-25T14:10:04.732355+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.740654+0000","last_change":"2024-04-25T14:10:04.734364+0000","last_active":"2024-04-25T14:10:04.740654+0000","last_peered":"2024-04-25T14:10:04.740654+0000","last_clean":"2024-04-25T14:10:04.740654+0000","last_became_active":"2024-04-25T14:10:04.734026+0000","last_became_peered":"2024-04-25T14:10:04.734026+0000","last_unstale":"2024-04-25T14:10:04.740654+0000","last_undegraded":"2024-04-25T14:10:04.740654+0000","last_fullsized":"2024-04-25T14:10:04.740654+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:48.842742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726718+0000","last_change":"2024-04-25T14:10:04.726718+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726718+0000","last_undegraded":"2024-04-25T14:10:04.726718+0000","last_fullsized":"2024-04-25T14:10:04.726718+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.753241+0000","last_change":"2024-04-25T14:10:04.745258+0000","last_active":"2024-04-25T14:10:04.753241+0000","last_peered":"2024-04-25T14:10:04.753241+0000","last_clean":"2024-04-25T14:10:04.753241+0000","last_became_active":"2024-04-25T14:10:04.744962+0000","last_became_peered":"2024-04-25T14:10:04.744962+0000","last_unstale":"2024-04-25T14:10:04.753241+0000","last_undegraded":"2024-04-25T14:10:04.753241+0000","last_fullsized":"2024-04-25T14:10:04.753241+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:58.400437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.729428+0000","last_change":"2024-04-25T14:10:04.729428+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.729428+0000","last_undegraded":"2024-04-25T14:10:04.729428+0000","last_fullsized":"2024-04-25T14:10:04.729428+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.754546+0000","last_change":"2024-04-25T14:10:04.750916+0000","last_active":"2024-04-25T14:10:04.754546+0000","last_peered":"2024-04-25T14:10:04.754546+0000","last_clean":"2024-04-25T14:10:04.754546+0000","last_became_active":"2024-04-25T14:10:04.750562+0000","last_became_peered":"2024-04-25T14:10:04.750562+0000","last_unstale":"2024-04-25T14:10:04.754546+0000","last_undegraded":"2024-04-25T14:10:04.754546+0000","last_fullsized":"2024-04-25T14:10:04.754546+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:50.427076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726923+0000","last_change":"2024-04-25T14:10:04.726923+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726923+0000","last_undegraded":"2024-04-25T14:10:04.726923+0000","last_fullsized":"2024-04-25T14:10:04.726923+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.749495+0000","last_change":"2024-04-25T14:10:04.746574+0000","last_active":"2024-04-25T14:10:04.749495+0000","last_peered":"2024-04-25T14:10:04.749495+0000","last_clean":"2024-04-25T14:10:04.749495+0000","last_became_active":"2024-04-25T14:10:04.746266+0000","last_became_peered":"2024-04-25T14:10:04.746266+0000","last_unstale":"2024-04-25T14:10:04.749495+0000","last_undegraded":"2024-04-25T14:10:04.749495+0000","last_fullsized":"2024-04-25T14:10:04.749495+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:28.147163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.732111+0000","last_change":"2024-04-25T14:10:04.732111+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.732111+0000","last_undegraded":"2024-04-25T14:10:04.732111+0000","last_fullsized":"2024-04-25T14:10:04.732111+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.748958+0000","last_change":"2024-04-25T14:10:04.745704+0000","last_active":"2024-04-25T14:10:04.748958+0000","last_peered":"2024-04-25T14:10:04.748958+0000","last_clean":"2024-04-25T14:10:04.748958+0000","last_became_active":"2024-04-25T14:10:04.745401+0000","last_became_peered":"2024-04-25T14:10:04.745401+0000","last_unstale":"2024-04-25T14:10:04.748958+0000","last_undegraded":"2024-04-25T14:10:04.748958+0000","last_fullsized":"2024-04-25T14:10:04.748958+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:24.704172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.727533+0000","last_change":"2024-04-25T14:10:04.727533+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.727533+0000","last_undegraded":"2024-04-25T14:10:04.727533+0000","last_fullsized":"2024-04-25T14:10:04.727533+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.749695+0000","last_change":"2024-04-25T14:10:04.745297+0000","last_active":"2024-04-25T14:10:04.749695+0000","last_peered":"2024-04-25T14:10:04.749695+0000","last_clean":"2024-04-25T14:10:04.749695+0000","last_became_active":"2024-04-25T14:10:04.744890+0000","last_became_peered":"2024-04-25T14:10:04.744890+0000","last_unstale":"2024-04-25T14:10:04.749695+0000","last_undegraded":"2024-04-25T14:10:04.749695+0000","last_fullsized":"2024-04-25T14:10:04.749695+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:39.794816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.730347+0000","last_change":"2024-04-25T14:10:04.730347+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.730347+0000","last_undegraded":"2024-04-25T14:10:04.730347+0000","last_fullsized":"2024-04-25T14:10:04.730347+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.750158+0000","last_change":"2024-04-25T14:10:04.746252+0000","last_active":"2024-04-25T14:10:04.750158+0000","last_peered":"2024-04-25T14:10:04.750158+0000","last_clean":"2024-04-25T14:10:04.750158+0000","last_became_active":"2024-04-25T14:10:04.745299+0000","last_became_peered":"2024-04-25T14:10:04.745299+0000","last_unstale":"2024-04-25T14:10:04.750158+0000","last_undegraded":"2024-04-25T14:10:04.750158+0000","last_fullsized":"2024-04-25T14:10:04.750158+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:05.444701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.725553+0000","last_change":"2024-04-25T14:10:04.725553+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.725553+0000","last_undegraded":"2024-04-25T14:10:04.725553+0000","last_fullsized":"2024-04-25T14:10:04.725553+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.745756+0000","last_change":"2024-04-25T14:10:04.740086+0000","last_active":"2024-04-25T14:10:04.745756+0000","last_peered":"2024-04-25T14:10:04.745756+0000","last_clean":"2024-04-25T14:10:04.745756+0000","last_became_active":"2024-04-25T14:10:04.739141+0000","last_became_peered":"2024-04-25T14:10:04.739141+0000","last_unstale":"2024-04-25T14:10:04.745756+0000","last_undegraded":"2024-04-25T14:10:04.745756+0000","last_fullsized":"2024-04-25T14:10:04.745756+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:16.658408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.736954+0000","last_change":"2024-04-25T14:10:04.736954+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.736954+0000","last_undegraded":"2024-04-25T14:10:04.736954+0000","last_fullsized":"2024-04-25T14:10:04.736954+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.746391+0000","last_change":"2024-04-25T14:10:04.743146+0000","last_active":"2024-04-25T14:10:04.746391+0000","last_peered":"2024-04-25T14:10:04.746391+0000","last_clean":"2024-04-25T14:10:04.746391+0000","last_became_active":"2024-04-25T14:10:04.742805+0000","last_became_peered":"2024-04-25T14:10:04.742805+0000","last_unstale":"2024-04-25T14:10:04.746391+0000","last_undegraded":"2024-04-25T14:10:04.746391+0000","last_fullsized":"2024-04-25T14:10:04.746391+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:52.358091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.728072+0000","last_change":"2024-04-25T14:10:04.728072+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.728072+0000","last_undegraded":"2024-04-25T14:10:04.728072+0000","last_fullsized":"2024-04-25T14:10:04.728072+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.753810+0000","last_change":"2024-04-25T14:10:04.751976+0000","last_active":"2024-04-25T14:10:04.753810+0000","last_peered":"2024-04-25T14:10:04.753810+0000","last_clean":"2024-04-25T14:10:04.753810+0000","last_became_active":"2024-04-25T14:10:04.751610+0000","last_became_peered":"2024-04-25T14:10:04.751610+0000","last_unstale":"2024-04-25T14:10:04.753810+0000","last_undegraded":"2024-04-25T14:10:04.753810+0000","last_fullsized":"2024-04-25T14:10:04.753810+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:54.546901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735165+0000","last_change":"2024-04-25T14:10:04.735165+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735165+0000","last_undegraded":"2024-04-25T14:10:04.735165+0000","last_fullsized":"2024-04-25T14:10:04.735165+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.752176+0000","last_change":"2024-04-25T14:10:04.749720+0000","last_active":"2024-04-25T14:10:04.752176+0000","last_peered":"2024-04-25T14:10:04.752176+0000","last_clean":"2024-04-25T14:10:04.752176+0000","last_became_active":"2024-04-25T14:10:04.749327+0000","last_became_peered":"2024-04-25T14:10:04.749327+0000","last_unstale":"2024-04-25T14:10:04.752176+0000","last_undegraded":"2024-04-25T14:10:04.752176+0000","last_fullsized":"2024-04-25T14:10:04.752176+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:30.105615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.738871+0000","last_change":"2024-04-25T14:10:04.738871+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.738871+0000","last_undegraded":"2024-04-25T14:10:04.738871+0000","last_fullsized":"2024-04-25T14:10:04.738871+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.752711+0000","last_change":"2024-04-25T14:10:04.750043+0000","last_active":"2024-04-25T14:10:04.752711+0000","last_peered":"2024-04-25T14:10:04.752711+0000","last_clean":"2024-04-25T14:10:04.752711+0000","last_became_active":"2024-04-25T14:10:04.749722+0000","last_became_peered":"2024-04-25T14:10:04.749722+0000","last_unstale":"2024-04-25T14:10:04.752711+0000","last_undegraded":"2024-04-25T14:10:04.752711+0000","last_fullsized":"2024-04-25T14:10:04.752711+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:54.546901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726953+0000","last_change":"2024-04-25T14:10:04.726953+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726953+0000","last_undegraded":"2024-04-25T14:10:04.726953+0000","last_fullsized":"2024-04-25T14:10:04.726953+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.755134+0000","last_change":"2024-04-25T14:10:04.754055+0000","last_active":"2024-04-25T14:10:04.755134+0000","last_peered":"2024-04-25T14:10:04.755134+0000","last_clean":"2024-04-25T14:10:04.755134+0000","last_became_active":"2024-04-25T14:10:04.753740+0000","last_became_peered":"2024-04-25T14:10:04.753740+0000","last_unstale":"2024-04-25T14:10:04.755134+0000","last_undegraded":"2024-04-25T14:10:04.755134+0000","last_fullsized":"2024-04-25T14:10:04.755134+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:16.658408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.734586+0000","last_change":"2024-04-25T14:10:04.734586+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.734586+0000","last_undegraded":"2024-04-25T14:10:04.734586+0000","last_fullsized":"2024-04-25T14:10:04.734586+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.754869+0000","last_change":"2024-04-25T14:10:04.753046+0000","last_active":"2024-04-25T14:10:04.754869+0000","last_peered":"2024-04-25T14:10:04.754869+0000","last_clean":"2024-04-25T14:10:04.754869+0000","last_became_active":"2024-04-25T14:10:04.749881+0000","last_became_peered":"2024-04-25T14:10:04.749881+0000","last_unstale":"2024-04-25T14:10:04.754869+0000","last_undegraded":"2024-04-25T14:10:04.754869+0000","last_fullsized":"2024-04-25T14:10:04.754869+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:39.976208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.731633+0000","last_change":"2024-04-25T14:10:04.731633+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.731633+0000","last_undegraded":"2024-04-25T14:10:04.731633+0000","last_fullsized":"2024-04-25T14:10:04.731633+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.752192+0000","last_change":"2024-04-25T14:10:04.749068+0000","last_active":"2024-04-25T14:10:04.752192+0000","last_peered":"2024-04-25T14:10:04.752192+0000","last_clean":"2024-04-25T14:10:04.752192+0000","last_became_active":"2024-04-25T14:10:04.748714+0000","last_became_peered":"2024-04-25T14:10:04.748714+0000","last_unstale":"2024-04-25T14:10:04.752192+0000","last_undegraded":"2024-04-25T14:10:04.752192+0000","last_fullsized":"2024-04-25T14:10:04.752192+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:45.118281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.731871+0000","last_change":"2024-04-25T14:10:04.731871+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.731871+0000","last_undegraded":"2024-04-25T14:10:04.731871+0000","last_fullsized":"2024-04-25T14:10:04.731871+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.750229+0000","last_change":"2024-04-25T14:10:04.746533+0000","last_active":"2024-04-25T14:10:04.750229+0000","last_peered":"2024-04-25T14:10:04.750229+0000","last_clean":"2024-04-25T14:10:04.750229+0000","last_became_active":"2024-04-25T14:10:04.746152+0000","last_became_peered":"2024-04-25T14:10:04.746152+0000","last_unstale":"2024-04-25T14:10:04.750229+0000","last_undegraded":"2024-04-25T14:10:04.750229+0000","last_fullsized":"2024-04-25T14:10:04.750229+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:01.747339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726535+0000","last_change":"2024-04-25T14:10:04.726535+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726535+0000","last_undegraded":"2024-04-25T14:10:04.726535+0000","last_fullsized":"2024-04-25T14:10:04.726535+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.752736+0000","last_change":"2024-04-25T14:10:04.752102+0000","last_active":"2024-04-25T14:10:04.752736+0000","last_peered":"2024-04-25T14:10:04.752736+0000","last_clean":"2024-04-25T14:10:04.752736+0000","last_became_active":"2024-04-25T14:10:04.751649+0000","last_became_peered":"2024-04-25T14:10:04.751649+0000","last_unstale":"2024-04-25T14:10:04.752736+0000","last_undegraded":"2024-04-25T14:10:04.752736+0000","last_fullsized":"2024-04-25T14:10:04.752736+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.238156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.736386+0000","last_change":"2024-04-25T14:10:04.736386+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.736386+0000","last_undegraded":"2024-04-25T14:10:04.736386+0000","last_fullsized":"2024-04-25T14:10:04.736386+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.754240+0000","last_change":"2024-04-25T14:10:04.749248+0000","last_active":"2024-04-25T14:10:04.754240+0000","last_peered":"2024-04-25T14:10:04.754240+0000","last_clean":"2024-04-25T14:10:04.754240+0000","last_became_active":"2024-04-25T14:10:04.748828+0000","last_became_peered":"2024-04-25T14:10:04.748828+0000","last_unstale":"2024-04-25T14:10:04.754240+0000","last_undegraded":"2024-04-25T14:10:04.754240+0000","last_fullsized":"2024-04-25T14:10:04.754240+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:38.859877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726322+0000","last_change":"2024-04-25T14:10:04.726322+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726322+0000","last_undegraded":"2024-04-25T14:10:04.726322+0000","last_fullsized":"2024-04-25T14:10:04.726322+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.749775+0000","last_change":"2024-04-25T14:10:04.746371+0000","last_active":"2024-04-25T14:10:04.749775+0000","last_peered":"2024-04-25T14:10:04.749775+0000","last_clean":"2024-04-25T14:10:04.749775+0000","last_became_active":"2024-04-25T14:10:04.745733+0000","last_became_peered":"2024-04-25T14:10:04.745733+0000","last_unstale":"2024-04-25T14:10:04.749775+0000","last_undegraded":"2024-04-25T14:10:04.749775+0000","last_fullsized":"2024-04-25T14:10:04.749775+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:54.546901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733546+0000","last_change":"2024-04-25T14:10:04.733546+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733546+0000","last_undegraded":"2024-04-25T14:10:04.733546+0000","last_fullsized":"2024-04-25T14:10:04.733546+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.752120+0000","last_change":"2024-04-25T14:10:04.747682+0000","last_active":"2024-04-25T14:10:04.752120+0000","last_peered":"2024-04-25T14:10:04.752120+0000","last_clean":"2024-04-25T14:10:04.752120+0000","last_became_active":"2024-04-25T14:10:04.747380+0000","last_became_peered":"2024-04-25T14:10:04.747380+0000","last_unstale":"2024-04-25T14:10:04.752120+0000","last_undegraded":"2024-04-25T14:10:04.752120+0000","last_fullsized":"2024-04-25T14:10:04.752120+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.238156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.736394+0000","last_change":"2024-04-25T14:10:04.736394+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.736394+0000","last_undegraded":"2024-04-25T14:10:04.736394+0000","last_fullsized":"2024-04-25T14:10:04.736394+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.754913+0000","last_change":"2024-04-25T14:10:04.752523+0000","last_active":"2024-04-25T14:10:04.754913+0000","last_peered":"2024-04-25T14:10:04.754913+0000","last_clean":"2024-04-25T14:10:04.754913+0000","last_became_active":"2024-04-25T14:10:04.750781+0000","last_became_peered":"2024-04-25T14:10:04.750781+0000","last_unstale":"2024-04-25T14:10:04.754913+0000","last_undegraded":"2024-04-25T14:10:04.754913+0000","last_fullsized":"2024-04-25T14:10:04.754913+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:30.252303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733923+0000","last_change":"2024-04-25T14:10:04.733923+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733923+0000","last_undegraded":"2024-04-25T14:10:04.733923+0000","last_fullsized":"2024-04-25T14:10:04.733923+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.748268+0000","last_change":"2024-04-25T14:10:04.746161+0000","last_active":"2024-04-25T14:10:04.748268+0000","last_peered":"2024-04-25T14:10:04.748268+0000","last_clean":"2024-04-25T14:10:04.748268+0000","last_became_active":"2024-04-25T14:10:04.745423+0000","last_became_peered":"2024-04-25T14:10:04.745423+0000","last_unstale":"2024-04-25T14:10:04.748268+0000","last_undegraded":"2024-04-25T14:10:04.748268+0000","last_fullsized":"2024-04-25T14:10:04.748268+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.238156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.730645+0000","last_change":"2024-04-25T14:10:04.730645+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.730645+0000","last_undegraded":"2024-04-25T14:10:04.730645+0000","last_fullsized":"2024-04-25T14:10:04.730645+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.747112+0000","last_change":"2024-04-25T14:10:04.742349+0000","last_active":"2024-04-25T14:10:04.747112+0000","last_peered":"2024-04-25T14:10:04.747112+0000","last_clean":"2024-04-25T14:10:04.747112+0000","last_became_active":"2024-04-25T14:10:04.741292+0000","last_became_peered":"2024-04-25T14:10:04.741292+0000","last_unstale":"2024-04-25T14:10:04.747112+0000","last_undegraded":"2024-04-25T14:10:04.747112+0000","last_fullsized":"2024-04-25T14:10:04.747112+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:54.546901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733889+0000","last_change":"2024-04-25T14:10:04.733889+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733889+0000","last_undegraded":"2024-04-25T14:10:04.733889+0000","last_fullsized":"2024-04-25T14:10:04.733889+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.754166+0000","last_change":"2024-04-25T14:10:04.749122+0000","last_active":"2024-04-25T14:10:04.754166+0000","last_peered":"2024-04-25T14:10:04.754166+0000","last_clean":"2024-04-25T14:10:04.754166+0000","last_became_active":"2024-04-25T14:10:04.748630+0000","last_became_peered":"2024-04-25T14:10:04.748630+0000","last_unstale":"2024-04-25T14:10:04.754166+0000","last_undegraded":"2024-04-25T14:10:04.754166+0000","last_fullsized":"2024-04-25T14:10:04.754166+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:21.269555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726229+0000","last_change":"2024-04-25T14:10:04.726229+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726229+0000","last_undegraded":"2024-04-25T14:10:04.726229+0000","last_fullsized":"2024-04-25T14:10:04.726229+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.749405+0000","last_change":"2024-04-25T14:10:04.745286+0000","last_active":"2024-04-25T14:10:04.749405+0000","last_peered":"2024-04-25T14:10:04.749405+0000","last_clean":"2024-04-25T14:10:04.749405+0000","last_became_active":"2024-04-25T14:10:04.744949+0000","last_became_peered":"2024-04-25T14:10:04.744949+0000","last_unstale":"2024-04-25T14:10:04.749405+0000","last_undegraded":"2024-04-25T14:10:04.749405+0000","last_fullsized":"2024-04-25T14:10:04.749405+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:25.644454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.734790+0000","last_change":"2024-04-25T14:10:04.734790+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.734790+0000","last_undegraded":"2024-04-25T14:10:04.734790+0000","last_fullsized":"2024-04-25T14:10:04.734790+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.750270+0000","last_change":"2024-04-25T14:10:04.747339+0000","last_active":"2024-04-25T14:10:04.750270+0000","last_peered":"2024-04-25T14:10:04.750270+0000","last_clean":"2024-04-25T14:10:04.750270+0000","last_became_active":"2024-04-25T14:10:04.745494+0000","last_became_peered":"2024-04-25T14:10:04.745494+0000","last_unstale":"2024-04-25T14:10:04.750270+0000","last_undegraded":"2024-04-25T14:10:04.750270+0000","last_fullsized":"2024-04-25T14:10:04.750270+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:38.191463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"102'119","reported_seq":445,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.740092+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:10:04.740092+0000","last_peered":"2024-04-25T14:10:04.740092+0000","last_clean":"2024-04-25T14:10:04.740092+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:10:04.740092+0000","last_undegraded":"2024-04-25T14:10:04.740092+0000","last_fullsized":"2024-04-25T14:10:04.740092+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":468,"num_read_kb":1272,"num_write":181,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":468,"num_read_kb":1272,"num_write":181,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33108,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338732,"statfs":{"total":96636764160,"available":96602861568,"internally_reserved":0,"allocated":1966080,"data_stored":1273327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32524,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339316,"statfs":{"total":96636764160,"available":96603459584,"internally_reserved":0,"allocated":1372160,"data_stored":682959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1384448,"data_stored":689626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1372160,"data_stored":682959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1384448,"data_stored":689626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1384448,"data_stored":689626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1372160,"data_stored":682959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574924,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33116,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338724,"statfs":{"total":96636764160,"available":96602853376,"internally_reserved":0,"allocated":1978368,"data_stored":1279994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:10:09.867 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:10:10.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.053+0000 7ff6e0400640 1 Processor -- start 2024-04-25T14:10:10.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.053+0000 7ff6e0400640 1 -- start start 2024-04-25T14:10:10.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.057+0000 7ff6e0400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff6d8070330 0x7ff6d80d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:10.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.057+0000 7ff6e0400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff6d80d1c20 0x7ff6d806ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:10.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.057+0000 7ff6e0400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6d80d2030 con 0x7ff6d806f270 2024-04-25T14:10:10.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.057+0000 7ff6e0400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff6d815e5c0 con 0x7ff6d8070330 2024-04-25T14:10:10.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.057+0000 7ff6e0400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff6d815e8a0 con 0x7ff6d80d1c20 2024-04-25T14:10:10.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.057+0000 7ff6dda00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff6d8070330 0x7ff6d80d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:10.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.057+0000 7ff6dd000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff6d80d1c20 0x7ff6d806ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:10.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.057+0000 7ff6de400640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7ff6d806f270 0x7ff6d806f680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:52368/0 (socket says 172.21.15.86:52368) 2024-04-25T14:10:10.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.057+0000 7ff6de400640 1 -- 172.21.15.86:0/2655916092 learned_addr learned my addr 172.21.15.86:0/2655916092 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:10.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.057+0000 7ff6dda00640 1 -- 172.21.15.86:0/2655916092 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff6d80d1c20 msgr2=0x7ff6d806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:10.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.057+0000 7ff6dda00640 1 --2- 172.21.15.86:0/2655916092 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff6d80d1c20 0x7ff6d806ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:10.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.057+0000 7ff6dda00640 1 -- 172.21.15.86:0/2655916092 >> v1:172.21.15.155:6789/0 conn(0x7ff6d806f270 legacy=0x7ff6d806f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:10.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.057+0000 7ff6dda00640 1 -- 172.21.15.86:0/2655916092 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d815eb80 con 0x7ff6d8070330 2024-04-25T14:10:10.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.061+0000 7ff6dda00640 1 --2- 172.21.15.86:0/2655916092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6d8070330 0x7ff6d80d16e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0097c0 tx=0x7ff6cc01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=492a404cc51f0d42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:10.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.061+0000 7ff6d7e00640 1 -- 172.21.15.86:0/2655916092 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc0b0070 con 0x7ff6d8070330 2024-04-25T14:10:10.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.061+0000 7ff6d7e00640 1 -- 172.21.15.86:0/2655916092 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6cc0ab600 con 0x7ff6d8070330 2024-04-25T14:10:10.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.061+0000 7ff6d7e00640 1 -- 172.21.15.86:0/2655916092 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc0ab9d0 con 0x7ff6d8070330 2024-04-25T14:10:10.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.061+0000 7ff6e0400640 1 -- 172.21.15.86:0/2655916092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6d8070330 msgr2=0x7ff6d80d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:10.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.061+0000 7ff6e0400640 1 --2- 172.21.15.86:0/2655916092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6d8070330 0x7ff6d80d16e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0097c0 tx=0x7ff6cc01c410 comp rx=0 tx=0).stop 2024-04-25T14:10:10.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.061+0000 7ff6e0400640 1 -- 172.21.15.86:0/2655916092 shutdown_connections 2024-04-25T14:10:10.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.061+0000 7ff6e0400640 1 --2- 172.21.15.86:0/2655916092 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff6d80d1c20 0x7ff6d806ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:10.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.061+0000 7ff6e0400640 1 --2- 172.21.15.86:0/2655916092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6d8070330 0x7ff6d80d16e0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:10.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.061+0000 7ff6e0400640 1 -- 172.21.15.86:0/2655916092 >> 172.21.15.86:0/2655916092 conn(0x7ff6d80b4650 msgr2=0x7ff6d80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:10.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.061+0000 7ff6e0400640 1 -- 172.21.15.86:0/2655916092 shutdown_connections 2024-04-25T14:10:10.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.061+0000 7ff6e0400640 1 -- 172.21.15.86:0/2655916092 wait complete. 2024-04-25T14:10:10.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.065+0000 7ff6e0400640 1 Processor -- start 2024-04-25T14:10:10.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.065+0000 7ff6e0400640 1 -- start start 2024-04-25T14:10:10.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.065+0000 7ff6e0400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6d806f270 0x7ff6d8156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:10.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.065+0000 7ff6e0400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff6d8070330 0x7ff6d80b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:10.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.065+0000 7ff6e0400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6d80d1c20 0x7ff6d809a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:10.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.065+0000 7ff6e0400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d80708c0 con 0x7ff6d806f270 2024-04-25T14:10:10.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.065+0000 7ff6e0400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d8070740 con 0x7ff6d80d1c20 2024-04-25T14:10:10.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.065+0000 7ff6e0400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff6d816afc0 con 0x7ff6d8070330 2024-04-25T14:10:10.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.065+0000 7ff6dda00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6d806f270 0x7ff6d8156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:10.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.065+0000 7ff6dda00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6d806f270 0x7ff6d8156dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53916/0 (socket says 172.21.15.86:53916) 2024-04-25T14:10:10.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.065+0000 7ff6dda00640 1 -- 172.21.15.86:0/3015054176 learned_addr learned my addr 172.21.15.86:0/3015054176 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:10.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.065+0000 7ff6dd000640 1 --2- 172.21.15.86:0/3015054176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff6d8070330 0x7ff6d80b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:10.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.065+0000 7ff6de400640 1 --2- 172.21.15.86:0/3015054176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6d80d1c20 0x7ff6d809a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:10.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.065+0000 7ff6dda00640 1 -- 172.21.15.86:0/3015054176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff6d8070330 msgr2=0x7ff6d80b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:10.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.065+0000 7ff6dda00640 1 --2- 172.21.15.86:0/3015054176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff6d8070330 0x7ff6d80b0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:10.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.065+0000 7ff6dda00640 1 -- 172.21.15.86:0/3015054176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6d80d1c20 msgr2=0x7ff6d809a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:10.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.065+0000 7ff6dda00640 1 --2- 172.21.15.86:0/3015054176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6d80d1c20 0x7ff6d809a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:10.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.065+0000 7ff6dda00640 1 -- 172.21.15.86:0/3015054176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d80b24e0 con 0x7ff6d806f270 2024-04-25T14:10:10.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.065+0000 7ff6de400640 1 --2- 172.21.15.86:0/3015054176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6d80d1c20 0x7ff6d809a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:10.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.065+0000 7ff6dda00640 1 --2- 172.21.15.86:0/3015054176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6d806f270 0x7ff6d8156dc0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0ae040 tx=0x7ff6cc01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:10.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.065+0000 7ff6d5600640 1 -- 172.21.15.86:0/3015054176 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc0b0050 con 0x7ff6d806f270 2024-04-25T14:10:10.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.065+0000 7ff6d5600640 1 -- 172.21.15.86:0/3015054176 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6cc0bc070 con 0x7ff6d806f270 2024-04-25T14:10:10.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.065+0000 7ff6d5600640 1 -- 172.21.15.86:0/3015054176 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc0b5870 con 0x7ff6d806f270 2024-04-25T14:10:10.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.065+0000 7ff6e0400640 1 -- 172.21.15.86:0/3015054176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6d809ae90 con 0x7ff6d806f270 2024-04-25T14:10:10.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.065+0000 7ff6e0400640 1 -- 172.21.15.86:0/3015054176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6d80b1720 con 0x7ff6d806f270 2024-04-25T14:10:10.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.069+0000 7ff6e0400640 1 -- 172.21.15.86:0/3015054176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6d8069d00 con 0x7ff6d806f270 2024-04-25T14:10:10.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.069+0000 7ff6d5600640 1 -- 172.21.15.86:0/3015054176 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7ff6cc0b5a10 con 0x7ff6d806f270 2024-04-25T14:10:10.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.069+0000 7ff6d5600640 1 --2- 172.21.15.86:0/3015054176 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7ff6b806f550 0x7ff6b8071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:10.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.069+0000 7ff6d5600640 1 -- 172.21.15.86:0/3015054176 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7ff6cc129c30 con 0x7ff6d806f270 2024-04-25T14:10:10.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.069+0000 7ff6dd000640 1 --2- 172.21.15.86:0/3015054176 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7ff6b806f550 0x7ff6b8071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:10.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.069+0000 7ff6dd000640 1 --2- 172.21.15.86:0/3015054176 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7ff6b806f550 0x7ff6b8071a60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff6d80cff30 tx=0x7ff6c4009190 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:10.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.073+0000 7ff6d5600640 1 -- 172.21.15.86:0/3015054176 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff6cc0c6020 con 0x7ff6d806f270 2024-04-25T14:10:10.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.229+0000 7ff6e0400640 1 -- 172.21.15.86:0/3015054176 --> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff6d8076bf0 con 0x7ff6b806f550 2024-04-25T14:10:10.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.237+0000 7ff6d5600640 1 -- 172.21.15.86:0/3015054176 <== mgr.8190 v2:172.21.15.86:6832/1354467416 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247091 (secure 0 0 0) 0x7ff6d8076bf0 con 0x7ff6b806f550 2024-04-25T14:10:10.236 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:10.237 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:10:10.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.237+0000 7ff6e0400640 1 -- 172.21.15.86:0/3015054176 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7ff6b806f550 msgr2=0x7ff6b8071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:10.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.237+0000 7ff6e0400640 1 --2- 172.21.15.86:0/3015054176 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7ff6b806f550 0x7ff6b8071a60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff6d80cff30 tx=0x7ff6c4009190 comp rx=0 tx=0).stop 2024-04-25T14:10:10.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.237+0000 7ff6e0400640 1 -- 172.21.15.86:0/3015054176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6d806f270 msgr2=0x7ff6d8156dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:10.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.237+0000 7ff6e0400640 1 --2- 172.21.15.86:0/3015054176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6d806f270 0x7ff6d8156dc0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0ae040 tx=0x7ff6cc01ac50 comp rx=0 tx=0).stop 2024-04-25T14:10:10.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.237+0000 7ff6e0400640 1 -- 172.21.15.86:0/3015054176 shutdown_connections 2024-04-25T14:10:10.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.237+0000 7ff6e0400640 1 --2- 172.21.15.86:0/3015054176 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7ff6b806f550 0x7ff6b8071a60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:10.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.237+0000 7ff6e0400640 1 --2- 172.21.15.86:0/3015054176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6d80d1c20 0x7ff6d809a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:10.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.237+0000 7ff6e0400640 1 --2- 172.21.15.86:0/3015054176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff6d8070330 0x7ff6d80b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:10.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.237+0000 7ff6e0400640 1 --2- 172.21.15.86:0/3015054176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6d806f270 0x7ff6d8156dc0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:10.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.237+0000 7ff6e0400640 1 -- 172.21.15.86:0/3015054176 >> 172.21.15.86:0/3015054176 conn(0x7ff6d80b4650 msgr2=0x7ff6d80b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:10.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.241+0000 7ff6e0400640 1 -- 172.21.15.86:0/3015054176 shutdown_connections 2024-04-25T14:10:10.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.241+0000 7ff6e0400640 1 -- 172.21.15.86:0/3015054176 wait complete. 2024-04-25T14:10:10.257 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T14:10:09.474871+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":468,"num_read_kb":1272,"num_write":181,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261404,"kb_used_data":11928,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713316,"statfs":{"total":773094113280,"available":772826435584,"internally_reserved":0,"allocated":12214272,"data_stored":6671076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.761213"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735771+0000","last_change":"2024-04-25T14:10:04.735771+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735771+0000","last_undegraded":"2024-04-25T14:10:04.735771+0000","last_fullsized":"2024-04-25T14:10:04.735771+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735778+0000","last_change":"2024-04-25T14:10:04.735778+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735778+0000","last_undegraded":"2024-04-25T14:10:04.735778+0000","last_fullsized":"2024-04-25T14:10:04.735778+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735849+0000","last_change":"2024-04-25T14:10:04.735849+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735849+0000","last_undegraded":"2024-04-25T14:10:04.735849+0000","last_fullsized":"2024-04-25T14:10:04.735849+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.734374+0000","last_change":"2024-04-25T14:10:04.734374+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.734374+0000","last_undegraded":"2024-04-25T14:10:04.734374+0000","last_fullsized":"2024-04-25T14:10:04.734374+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.734611+0000","last_change":"2024-04-25T14:10:04.734611+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.734611+0000","last_undegraded":"2024-04-25T14:10:04.734611+0000","last_fullsized":"2024-04-25T14:10:04.734611+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.738244+0000","last_change":"2024-04-25T14:10:04.738244+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.738244+0000","last_undegraded":"2024-04-25T14:10:04.738244+0000","last_fullsized":"2024-04-25T14:10:04.738244+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.734697+0000","last_change":"2024-04-25T14:10:04.734697+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.734697+0000","last_undegraded":"2024-04-25T14:10:04.734697+0000","last_fullsized":"2024-04-25T14:10:04.734697+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733337+0000","last_change":"2024-04-25T14:10:04.733337+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733337+0000","last_undegraded":"2024-04-25T14:10:04.733337+0000","last_fullsized":"2024-04-25T14:10:04.733337+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.739896+0000","last_change":"2024-04-25T14:10:04.739896+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.739896+0000","last_undegraded":"2024-04-25T14:10:04.739896+0000","last_fullsized":"2024-04-25T14:10:04.739896+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726838+0000","last_change":"2024-04-25T14:10:04.726838+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726838+0000","last_undegraded":"2024-04-25T14:10:04.726838+0000","last_fullsized":"2024-04-25T14:10:04.726838+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735651+0000","last_change":"2024-04-25T14:10:04.735651+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735651+0000","last_undegraded":"2024-04-25T14:10:04.735651+0000","last_fullsized":"2024-04-25T14:10:04.735651+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.742843+0000","last_change":"2024-04-25T14:10:04.742843+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.742843+0000","last_undegraded":"2024-04-25T14:10:04.742843+0000","last_fullsized":"2024-04-25T14:10:04.742843+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.727264+0000","last_change":"2024-04-25T14:10:04.727264+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.727264+0000","last_undegraded":"2024-04-25T14:10:04.727264+0000","last_fullsized":"2024-04-25T14:10:04.727264+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733351+0000","last_change":"2024-04-25T14:10:04.733351+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733351+0000","last_undegraded":"2024-04-25T14:10:04.733351+0000","last_fullsized":"2024-04-25T14:10:04.733351+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.734380+0000","last_change":"2024-04-25T14:10:04.734380+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.734380+0000","last_undegraded":"2024-04-25T14:10:04.734380+0000","last_fullsized":"2024-04-25T14:10:04.734380+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735156+0000","last_change":"2024-04-25T14:10:04.735156+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735156+0000","last_undegraded":"2024-04-25T14:10:04.735156+0000","last_fullsized":"2024-04-25T14:10:04.735156+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.725975+0000","last_change":"2024-04-25T14:10:04.725975+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.725975+0000","last_undegraded":"2024-04-25T14:10:04.725975+0000","last_fullsized":"2024-04-25T14:10:04.725975+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733263+0000","last_change":"2024-04-25T14:10:04.733263+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733263+0000","last_undegraded":"2024-04-25T14:10:04.733263+0000","last_fullsized":"2024-04-25T14:10:04.733263+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.729768+0000","last_change":"2024-04-25T14:10:04.729768+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.729768+0000","last_undegraded":"2024-04-25T14:10:04.729768+0000","last_fullsized":"2024-04-25T14:10:04.729768+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.737805+0000","last_change":"2024-04-25T14:10:04.737805+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.737805+0000","last_undegraded":"2024-04-25T14:10:04.737805+0000","last_fullsized":"2024-04-25T14:10:04.737805+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735963+0000","last_change":"2024-04-25T14:10:04.735963+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735963+0000","last_undegraded":"2024-04-25T14:10:04.735963+0000","last_fullsized":"2024-04-25T14:10:04.735963+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733684+0000","last_change":"2024-04-25T14:10:04.733684+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733684+0000","last_undegraded":"2024-04-25T14:10:04.733684+0000","last_fullsized":"2024-04-25T14:10:04.733684+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.732828+0000","last_change":"2024-04-25T14:10:04.732828+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.732828+0000","last_undegraded":"2024-04-25T14:10:04.732828+0000","last_fullsized":"2024-04-25T14:10:04.732828+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.727648+0000","last_change":"2024-04-25T14:10:04.727648+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.727648+0000","last_undegraded":"2024-04-25T14:10:04.727648+0000","last_fullsized":"2024-04-25T14:10:04.727648+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.728155+0000","last_change":"2024-04-25T14:10:04.728155+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.728155+0000","last_undegraded":"2024-04-25T14:10:04.728155+0000","last_fullsized":"2024-04-25T14:10:04.728155+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735818+0000","last_change":"2024-04-25T14:10:04.735818+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735818+0000","last_undegraded":"2024-04-25T14:10:04.735818+0000","last_fullsized":"2024-04-25T14:10:04.735818+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.729202+0000","last_change":"2024-04-25T14:10:04.729202+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.729202+0000","last_undegraded":"2024-04-25T14:10:04.729202+0000","last_fullsized":"2024-04-25T14:10:04.729202+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.738433+0000","last_change":"2024-04-25T14:10:04.738433+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.738433+0000","last_undegraded":"2024-04-25T14:10:04.738433+0000","last_fullsized":"2024-04-25T14:10:04.738433+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.737523+0000","last_change":"2024-04-25T14:10:04.737523+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.737523+0000","last_undegraded":"2024-04-25T14:10:04.737523+0000","last_fullsized":"2024-04-25T14:10:04.737523+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.732817+0000","last_change":"2024-04-25T14:10:04.732817+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.732817+0000","last_undegraded":"2024-04-25T14:10:04.732817+0000","last_fullsized":"2024-04-25T14:10:04.732817+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726801+0000","last_change":"2024-04-25T14:10:04.726801+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726801+0000","last_undegraded":"2024-04-25T14:10:04.726801+0000","last_fullsized":"2024-04-25T14:10:04.726801+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726485+0000","last_change":"2024-04-25T14:10:04.726485+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726485+0000","last_undegraded":"2024-04-25T14:10:04.726485+0000","last_fullsized":"2024-04-25T14:10:04.726485+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726867+0000","last_change":"2024-04-25T14:10:04.726867+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726867+0000","last_undegraded":"2024-04-25T14:10:04.726867+0000","last_fullsized":"2024-04-25T14:10:04.726867+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.749691+0000","last_change":"2024-04-25T14:10:04.745986+0000","last_active":"2024-04-25T14:10:04.749691+0000","last_peered":"2024-04-25T14:10:04.749691+0000","last_clean":"2024-04-25T14:10:04.749691+0000","last_became_active":"2024-04-25T14:10:04.745516+0000","last_became_peered":"2024-04-25T14:10:04.745516+0000","last_unstale":"2024-04-25T14:10:04.749691+0000","last_undegraded":"2024-04-25T14:10:04.749691+0000","last_fullsized":"2024-04-25T14:10:04.749691+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:45.118281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735537+0000","last_change":"2024-04-25T14:10:04.735537+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735537+0000","last_undegraded":"2024-04-25T14:10:04.735537+0000","last_fullsized":"2024-04-25T14:10:04.735537+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.746437+0000","last_change":"2024-04-25T14:10:04.744056+0000","last_active":"2024-04-25T14:10:04.746437+0000","last_peered":"2024-04-25T14:10:04.746437+0000","last_clean":"2024-04-25T14:10:04.746437+0000","last_became_active":"2024-04-25T14:10:04.743755+0000","last_became_peered":"2024-04-25T14:10:04.743755+0000","last_unstale":"2024-04-25T14:10:04.746437+0000","last_undegraded":"2024-04-25T14:10:04.746437+0000","last_fullsized":"2024-04-25T14:10:04.746437+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:45.118281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.728128+0000","last_change":"2024-04-25T14:10:04.728128+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.728128+0000","last_undegraded":"2024-04-25T14:10:04.728128+0000","last_fullsized":"2024-04-25T14:10:04.728128+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.755092+0000","last_change":"2024-04-25T14:10:04.752833+0000","last_active":"2024-04-25T14:10:04.755092+0000","last_peered":"2024-04-25T14:10:04.755092+0000","last_clean":"2024-04-25T14:10:04.755092+0000","last_became_active":"2024-04-25T14:10:04.752483+0000","last_became_peered":"2024-04-25T14:10:04.752483+0000","last_unstale":"2024-04-25T14:10:04.755092+0000","last_undegraded":"2024-04-25T14:10:04.755092+0000","last_fullsized":"2024-04-25T14:10:04.755092+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:50.427076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.738654+0000","last_change":"2024-04-25T14:10:04.738654+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.738654+0000","last_undegraded":"2024-04-25T14:10:04.738654+0000","last_fullsized":"2024-04-25T14:10:04.738654+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.737624+0000","last_change":"2024-04-25T14:10:04.733242+0000","last_active":"2024-04-25T14:10:04.737624+0000","last_peered":"2024-04-25T14:10:04.737624+0000","last_clean":"2024-04-25T14:10:04.737624+0000","last_became_active":"2024-04-25T14:10:04.732929+0000","last_became_peered":"2024-04-25T14:10:04.732929+0000","last_unstale":"2024-04-25T14:10:04.737624+0000","last_undegraded":"2024-04-25T14:10:04.737624+0000","last_fullsized":"2024-04-25T14:10:04.737624+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:25.644454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.725539+0000","last_change":"2024-04-25T14:10:04.725539+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.725539+0000","last_undegraded":"2024-04-25T14:10:04.725539+0000","last_fullsized":"2024-04-25T14:10:04.725539+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.744498+0000","last_change":"2024-04-25T14:10:04.738920+0000","last_active":"2024-04-25T14:10:04.744498+0000","last_peered":"2024-04-25T14:10:04.744498+0000","last_clean":"2024-04-25T14:10:04.744498+0000","last_became_active":"2024-04-25T14:10:04.738531+0000","last_became_peered":"2024-04-25T14:10:04.738531+0000","last_unstale":"2024-04-25T14:10:04.744498+0000","last_undegraded":"2024-04-25T14:10:04.744498+0000","last_fullsized":"2024-04-25T14:10:04.744498+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:45.118281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733780+0000","last_change":"2024-04-25T14:10:04.733780+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733780+0000","last_undegraded":"2024-04-25T14:10:04.733780+0000","last_fullsized":"2024-04-25T14:10:04.733780+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.754194+0000","last_change":"2024-04-25T14:10:04.749193+0000","last_active":"2024-04-25T14:10:04.754194+0000","last_peered":"2024-04-25T14:10:04.754194+0000","last_clean":"2024-04-25T14:10:04.754194+0000","last_became_active":"2024-04-25T14:10:04.747285+0000","last_became_peered":"2024-04-25T14:10:04.747285+0000","last_unstale":"2024-04-25T14:10:04.754194+0000","last_undegraded":"2024-04-25T14:10:04.754194+0000","last_fullsized":"2024-04-25T14:10:04.754194+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:19:34.506848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.732355+0000","last_change":"2024-04-25T14:10:04.732355+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.732355+0000","last_undegraded":"2024-04-25T14:10:04.732355+0000","last_fullsized":"2024-04-25T14:10:04.732355+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.740654+0000","last_change":"2024-04-25T14:10:04.734364+0000","last_active":"2024-04-25T14:10:04.740654+0000","last_peered":"2024-04-25T14:10:04.740654+0000","last_clean":"2024-04-25T14:10:04.740654+0000","last_became_active":"2024-04-25T14:10:04.734026+0000","last_became_peered":"2024-04-25T14:10:04.734026+0000","last_unstale":"2024-04-25T14:10:04.740654+0000","last_undegraded":"2024-04-25T14:10:04.740654+0000","last_fullsized":"2024-04-25T14:10:04.740654+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:48.842742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726718+0000","last_change":"2024-04-25T14:10:04.726718+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726718+0000","last_undegraded":"2024-04-25T14:10:04.726718+0000","last_fullsized":"2024-04-25T14:10:04.726718+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.753241+0000","last_change":"2024-04-25T14:10:04.745258+0000","last_active":"2024-04-25T14:10:04.753241+0000","last_peered":"2024-04-25T14:10:04.753241+0000","last_clean":"2024-04-25T14:10:04.753241+0000","last_became_active":"2024-04-25T14:10:04.744962+0000","last_became_peered":"2024-04-25T14:10:04.744962+0000","last_unstale":"2024-04-25T14:10:04.753241+0000","last_undegraded":"2024-04-25T14:10:04.753241+0000","last_fullsized":"2024-04-25T14:10:04.753241+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:58.400437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.729428+0000","last_change":"2024-04-25T14:10:04.729428+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.729428+0000","last_undegraded":"2024-04-25T14:10:04.729428+0000","last_fullsized":"2024-04-25T14:10:04.729428+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.754546+0000","last_change":"2024-04-25T14:10:04.750916+0000","last_active":"2024-04-25T14:10:04.754546+0000","last_peered":"2024-04-25T14:10:04.754546+0000","last_clean":"2024-04-25T14:10:04.754546+0000","last_became_active":"2024-04-25T14:10:04.750562+0000","last_became_peered":"2024-04-25T14:10:04.750562+0000","last_unstale":"2024-04-25T14:10:04.754546+0000","last_undegraded":"2024-04-25T14:10:04.754546+0000","last_fullsized":"2024-04-25T14:10:04.754546+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:50.427076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726923+0000","last_change":"2024-04-25T14:10:04.726923+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726923+0000","last_undegraded":"2024-04-25T14:10:04.726923+0000","last_fullsized":"2024-04-25T14:10:04.726923+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.749495+0000","last_change":"2024-04-25T14:10:04.746574+0000","last_active":"2024-04-25T14:10:04.749495+0000","last_peered":"2024-04-25T14:10:04.749495+0000","last_clean":"2024-04-25T14:10:04.749495+0000","last_became_active":"2024-04-25T14:10:04.746266+0000","last_became_peered":"2024-04-25T14:10:04.746266+0000","last_unstale":"2024-04-25T14:10:04.749495+0000","last_undegraded":"2024-04-25T14:10:04.749495+0000","last_fullsized":"2024-04-25T14:10:04.749495+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:28.147163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.732111+0000","last_change":"2024-04-25T14:10:04.732111+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.732111+0000","last_undegraded":"2024-04-25T14:10:04.732111+0000","last_fullsized":"2024-04-25T14:10:04.732111+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.748958+0000","last_change":"2024-04-25T14:10:04.745704+0000","last_active":"2024-04-25T14:10:04.748958+0000","last_peered":"2024-04-25T14:10:04.748958+0000","last_clean":"2024-04-25T14:10:04.748958+0000","last_became_active":"2024-04-25T14:10:04.745401+0000","last_became_peered":"2024-04-25T14:10:04.745401+0000","last_unstale":"2024-04-25T14:10:04.748958+0000","last_undegraded":"2024-04-25T14:10:04.748958+0000","last_fullsized":"2024-04-25T14:10:04.748958+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:24.704172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.727533+0000","last_change":"2024-04-25T14:10:04.727533+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.727533+0000","last_undegraded":"2024-04-25T14:10:04.727533+0000","last_fullsized":"2024-04-25T14:10:04.727533+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.749695+0000","last_change":"2024-04-25T14:10:04.745297+0000","last_active":"2024-04-25T14:10:04.749695+0000","last_peered":"2024-04-25T14:10:04.749695+0000","last_clean":"2024-04-25T14:10:04.749695+0000","last_became_active":"2024-04-25T14:10:04.744890+0000","last_became_peered":"2024-04-25T14:10:04.744890+0000","last_unstale":"2024-04-25T14:10:04.749695+0000","last_undegraded":"2024-04-25T14:10:04.749695+0000","last_fullsized":"2024-04-25T14:10:04.749695+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:39.794816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.730347+0000","last_change":"2024-04-25T14:10:04.730347+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.730347+0000","last_undegraded":"2024-04-25T14:10:04.730347+0000","last_fullsized":"2024-04-25T14:10:04.730347+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.750158+0000","last_change":"2024-04-25T14:10:04.746252+0000","last_active":"2024-04-25T14:10:04.750158+0000","last_peered":"2024-04-25T14:10:04.750158+0000","last_clean":"2024-04-25T14:10:04.750158+0000","last_became_active":"2024-04-25T14:10:04.745299+0000","last_became_peered":"2024-04-25T14:10:04.745299+0000","last_unstale":"2024-04-25T14:10:04.750158+0000","last_undegraded":"2024-04-25T14:10:04.750158+0000","last_fullsized":"2024-04-25T14:10:04.750158+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:05.444701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.725553+0000","last_change":"2024-04-25T14:10:04.725553+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.725553+0000","last_undegraded":"2024-04-25T14:10:04.725553+0000","last_fullsized":"2024-04-25T14:10:04.725553+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.745756+0000","last_change":"2024-04-25T14:10:04.740086+0000","last_active":"2024-04-25T14:10:04.745756+0000","last_peered":"2024-04-25T14:10:04.745756+0000","last_clean":"2024-04-25T14:10:04.745756+0000","last_became_active":"2024-04-25T14:10:04.739141+0000","last_became_peered":"2024-04-25T14:10:04.739141+0000","last_unstale":"2024-04-25T14:10:04.745756+0000","last_undegraded":"2024-04-25T14:10:04.745756+0000","last_fullsized":"2024-04-25T14:10:04.745756+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:16.658408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.736954+0000","last_change":"2024-04-25T14:10:04.736954+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.736954+0000","last_undegraded":"2024-04-25T14:10:04.736954+0000","last_fullsized":"2024-04-25T14:10:04.736954+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.746391+0000","last_change":"2024-04-25T14:10:04.743146+0000","last_active":"2024-04-25T14:10:04.746391+0000","last_peered":"2024-04-25T14:10:04.746391+0000","last_clean":"2024-04-25T14:10:04.746391+0000","last_became_active":"2024-04-25T14:10:04.742805+0000","last_became_peered":"2024-04-25T14:10:04.742805+0000","last_unstale":"2024-04-25T14:10:04.746391+0000","last_undegraded":"2024-04-25T14:10:04.746391+0000","last_fullsized":"2024-04-25T14:10:04.746391+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:52.358091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.728072+0000","last_change":"2024-04-25T14:10:04.728072+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.728072+0000","last_undegraded":"2024-04-25T14:10:04.728072+0000","last_fullsized":"2024-04-25T14:10:04.728072+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.753810+0000","last_change":"2024-04-25T14:10:04.751976+0000","last_active":"2024-04-25T14:10:04.753810+0000","last_peered":"2024-04-25T14:10:04.753810+0000","last_clean":"2024-04-25T14:10:04.753810+0000","last_became_active":"2024-04-25T14:10:04.751610+0000","last_became_peered":"2024-04-25T14:10:04.751610+0000","last_unstale":"2024-04-25T14:10:04.753810+0000","last_undegraded":"2024-04-25T14:10:04.753810+0000","last_fullsized":"2024-04-25T14:10:04.753810+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:54.546901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735165+0000","last_change":"2024-04-25T14:10:04.735165+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735165+0000","last_undegraded":"2024-04-25T14:10:04.735165+0000","last_fullsized":"2024-04-25T14:10:04.735165+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.752176+0000","last_change":"2024-04-25T14:10:04.749720+0000","last_active":"2024-04-25T14:10:04.752176+0000","last_peered":"2024-04-25T14:10:04.752176+0000","last_clean":"2024-04-25T14:10:04.752176+0000","last_became_active":"2024-04-25T14:10:04.749327+0000","last_became_peered":"2024-04-25T14:10:04.749327+0000","last_unstale":"2024-04-25T14:10:04.752176+0000","last_undegraded":"2024-04-25T14:10:04.752176+0000","last_fullsized":"2024-04-25T14:10:04.752176+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:30.105615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.738871+0000","last_change":"2024-04-25T14:10:04.738871+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.738871+0000","last_undegraded":"2024-04-25T14:10:04.738871+0000","last_fullsized":"2024-04-25T14:10:04.738871+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.752711+0000","last_change":"2024-04-25T14:10:04.750043+0000","last_active":"2024-04-25T14:10:04.752711+0000","last_peered":"2024-04-25T14:10:04.752711+0000","last_clean":"2024-04-25T14:10:04.752711+0000","last_became_active":"2024-04-25T14:10:04.749722+0000","last_became_peered":"2024-04-25T14:10:04.749722+0000","last_unstale":"2024-04-25T14:10:04.752711+0000","last_undegraded":"2024-04-25T14:10:04.752711+0000","last_fullsized":"2024-04-25T14:10:04.752711+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:54.546901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726953+0000","last_change":"2024-04-25T14:10:04.726953+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726953+0000","last_undegraded":"2024-04-25T14:10:04.726953+0000","last_fullsized":"2024-04-25T14:10:04.726953+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.755134+0000","last_change":"2024-04-25T14:10:04.754055+0000","last_active":"2024-04-25T14:10:04.755134+0000","last_peered":"2024-04-25T14:10:04.755134+0000","last_clean":"2024-04-25T14:10:04.755134+0000","last_became_active":"2024-04-25T14:10:04.753740+0000","last_became_peered":"2024-04-25T14:10:04.753740+0000","last_unstale":"2024-04-25T14:10:04.755134+0000","last_undegraded":"2024-04-25T14:10:04.755134+0000","last_fullsized":"2024-04-25T14:10:04.755134+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:16.658408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.734586+0000","last_change":"2024-04-25T14:10:04.734586+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.734586+0000","last_undegraded":"2024-04-25T14:10:04.734586+0000","last_fullsized":"2024-04-25T14:10:04.734586+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.754869+0000","last_change":"2024-04-25T14:10:04.753046+0000","last_active":"2024-04-25T14:10:04.754869+0000","last_peered":"2024-04-25T14:10:04.754869+0000","last_clean":"2024-04-25T14:10:04.754869+0000","last_became_active":"2024-04-25T14:10:04.749881+0000","last_became_peered":"2024-04-25T14:10:04.749881+0000","last_unstale":"2024-04-25T14:10:04.754869+0000","last_undegraded":"2024-04-25T14:10:04.754869+0000","last_fullsized":"2024-04-25T14:10:04.754869+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:39.976208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.731633+0000","last_change":"2024-04-25T14:10:04.731633+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.731633+0000","last_undegraded":"2024-04-25T14:10:04.731633+0000","last_fullsized":"2024-04-25T14:10:04.731633+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.752192+0000","last_change":"2024-04-25T14:10:04.749068+0000","last_active":"2024-04-25T14:10:04.752192+0000","last_peered":"2024-04-25T14:10:04.752192+0000","last_clean":"2024-04-25T14:10:04.752192+0000","last_became_active":"2024-04-25T14:10:04.748714+0000","last_became_peered":"2024-04-25T14:10:04.748714+0000","last_unstale":"2024-04-25T14:10:04.752192+0000","last_undegraded":"2024-04-25T14:10:04.752192+0000","last_fullsized":"2024-04-25T14:10:04.752192+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:45.118281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.731871+0000","last_change":"2024-04-25T14:10:04.731871+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.731871+0000","last_undegraded":"2024-04-25T14:10:04.731871+0000","last_fullsized":"2024-04-25T14:10:04.731871+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.750229+0000","last_change":"2024-04-25T14:10:04.746533+0000","last_active":"2024-04-25T14:10:04.750229+0000","last_peered":"2024-04-25T14:10:04.750229+0000","last_clean":"2024-04-25T14:10:04.750229+0000","last_became_active":"2024-04-25T14:10:04.746152+0000","last_became_peered":"2024-04-25T14:10:04.746152+0000","last_unstale":"2024-04-25T14:10:04.750229+0000","last_undegraded":"2024-04-25T14:10:04.750229+0000","last_fullsized":"2024-04-25T14:10:04.750229+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:01.747339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726535+0000","last_change":"2024-04-25T14:10:04.726535+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726535+0000","last_undegraded":"2024-04-25T14:10:04.726535+0000","last_fullsized":"2024-04-25T14:10:04.726535+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.752736+0000","last_change":"2024-04-25T14:10:04.752102+0000","last_active":"2024-04-25T14:10:04.752736+0000","last_peered":"2024-04-25T14:10:04.752736+0000","last_clean":"2024-04-25T14:10:04.752736+0000","last_became_active":"2024-04-25T14:10:04.751649+0000","last_became_peered":"2024-04-25T14:10:04.751649+0000","last_unstale":"2024-04-25T14:10:04.752736+0000","last_undegraded":"2024-04-25T14:10:04.752736+0000","last_fullsized":"2024-04-25T14:10:04.752736+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.238156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.736386+0000","last_change":"2024-04-25T14:10:04.736386+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.736386+0000","last_undegraded":"2024-04-25T14:10:04.736386+0000","last_fullsized":"2024-04-25T14:10:04.736386+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.754240+0000","last_change":"2024-04-25T14:10:04.749248+0000","last_active":"2024-04-25T14:10:04.754240+0000","last_peered":"2024-04-25T14:10:04.754240+0000","last_clean":"2024-04-25T14:10:04.754240+0000","last_became_active":"2024-04-25T14:10:04.748828+0000","last_became_peered":"2024-04-25T14:10:04.748828+0000","last_unstale":"2024-04-25T14:10:04.754240+0000","last_undegraded":"2024-04-25T14:10:04.754240+0000","last_fullsized":"2024-04-25T14:10:04.754240+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:38.859877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726322+0000","last_change":"2024-04-25T14:10:04.726322+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726322+0000","last_undegraded":"2024-04-25T14:10:04.726322+0000","last_fullsized":"2024-04-25T14:10:04.726322+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.749775+0000","last_change":"2024-04-25T14:10:04.746371+0000","last_active":"2024-04-25T14:10:04.749775+0000","last_peered":"2024-04-25T14:10:04.749775+0000","last_clean":"2024-04-25T14:10:04.749775+0000","last_became_active":"2024-04-25T14:10:04.745733+0000","last_became_peered":"2024-04-25T14:10:04.745733+0000","last_unstale":"2024-04-25T14:10:04.749775+0000","last_undegraded":"2024-04-25T14:10:04.749775+0000","last_fullsized":"2024-04-25T14:10:04.749775+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:54.546901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733546+0000","last_change":"2024-04-25T14:10:04.733546+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733546+0000","last_undegraded":"2024-04-25T14:10:04.733546+0000","last_fullsized":"2024-04-25T14:10:04.733546+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.752120+0000","last_change":"2024-04-25T14:10:04.747682+0000","last_active":"2024-04-25T14:10:04.752120+0000","last_peered":"2024-04-25T14:10:04.752120+0000","last_clean":"2024-04-25T14:10:04.752120+0000","last_became_active":"2024-04-25T14:10:04.747380+0000","last_became_peered":"2024-04-25T14:10:04.747380+0000","last_unstale":"2024-04-25T14:10:04.752120+0000","last_undegraded":"2024-04-25T14:10:04.752120+0000","last_fullsized":"2024-04-25T14:10:04.752120+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.238156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.736394+0000","last_change":"2024-04-25T14:10:04.736394+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.736394+0000","last_undegraded":"2024-04-25T14:10:04.736394+0000","last_fullsized":"2024-04-25T14:10:04.736394+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.754913+0000","last_change":"2024-04-25T14:10:04.752523+0000","last_active":"2024-04-25T14:10:04.754913+0000","last_peered":"2024-04-25T14:10:04.754913+0000","last_clean":"2024-04-25T14:10:04.754913+0000","last_became_active":"2024-04-25T14:10:04.750781+0000","last_became_peered":"2024-04-25T14:10:04.750781+0000","last_unstale":"2024-04-25T14:10:04.754913+0000","last_undegraded":"2024-04-25T14:10:04.754913+0000","last_fullsized":"2024-04-25T14:10:04.754913+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:30.252303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733923+0000","last_change":"2024-04-25T14:10:04.733923+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733923+0000","last_undegraded":"2024-04-25T14:10:04.733923+0000","last_fullsized":"2024-04-25T14:10:04.733923+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.748268+0000","last_change":"2024-04-25T14:10:04.746161+0000","last_active":"2024-04-25T14:10:04.748268+0000","last_peered":"2024-04-25T14:10:04.748268+0000","last_clean":"2024-04-25T14:10:04.748268+0000","last_became_active":"2024-04-25T14:10:04.745423+0000","last_became_peered":"2024-04-25T14:10:04.745423+0000","last_unstale":"2024-04-25T14:10:04.748268+0000","last_undegraded":"2024-04-25T14:10:04.748268+0000","last_fullsized":"2024-04-25T14:10:04.748268+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.238156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.730645+0000","last_change":"2024-04-25T14:10:04.730645+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.730645+0000","last_undegraded":"2024-04-25T14:10:04.730645+0000","last_fullsized":"2024-04-25T14:10:04.730645+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.747112+0000","last_change":"2024-04-25T14:10:04.742349+0000","last_active":"2024-04-25T14:10:04.747112+0000","last_peered":"2024-04-25T14:10:04.747112+0000","last_clean":"2024-04-25T14:10:04.747112+0000","last_became_active":"2024-04-25T14:10:04.741292+0000","last_became_peered":"2024-04-25T14:10:04.741292+0000","last_unstale":"2024-04-25T14:10:04.747112+0000","last_undegraded":"2024-04-25T14:10:04.747112+0000","last_fullsized":"2024-04-25T14:10:04.747112+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:54.546901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733889+0000","last_change":"2024-04-25T14:10:04.733889+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733889+0000","last_undegraded":"2024-04-25T14:10:04.733889+0000","last_fullsized":"2024-04-25T14:10:04.733889+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.754166+0000","last_change":"2024-04-25T14:10:04.749122+0000","last_active":"2024-04-25T14:10:04.754166+0000","last_peered":"2024-04-25T14:10:04.754166+0000","last_clean":"2024-04-25T14:10:04.754166+0000","last_became_active":"2024-04-25T14:10:04.748630+0000","last_became_peered":"2024-04-25T14:10:04.748630+0000","last_unstale":"2024-04-25T14:10:04.754166+0000","last_undegraded":"2024-04-25T14:10:04.754166+0000","last_fullsized":"2024-04-25T14:10:04.754166+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:21.269555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726229+0000","last_change":"2024-04-25T14:10:04.726229+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726229+0000","last_undegraded":"2024-04-25T14:10:04.726229+0000","last_fullsized":"2024-04-25T14:10:04.726229+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.749405+0000","last_change":"2024-04-25T14:10:04.745286+0000","last_active":"2024-04-25T14:10:04.749405+0000","last_peered":"2024-04-25T14:10:04.749405+0000","last_clean":"2024-04-25T14:10:04.749405+0000","last_became_active":"2024-04-25T14:10:04.744949+0000","last_became_peered":"2024-04-25T14:10:04.744949+0000","last_unstale":"2024-04-25T14:10:04.749405+0000","last_undegraded":"2024-04-25T14:10:04.749405+0000","last_fullsized":"2024-04-25T14:10:04.749405+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:25.644454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.734790+0000","last_change":"2024-04-25T14:10:04.734790+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.734790+0000","last_undegraded":"2024-04-25T14:10:04.734790+0000","last_fullsized":"2024-04-25T14:10:04.734790+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.750270+0000","last_change":"2024-04-25T14:10:04.747339+0000","last_active":"2024-04-25T14:10:04.750270+0000","last_peered":"2024-04-25T14:10:04.750270+0000","last_clean":"2024-04-25T14:10:04.750270+0000","last_became_active":"2024-04-25T14:10:04.745494+0000","last_became_peered":"2024-04-25T14:10:04.745494+0000","last_unstale":"2024-04-25T14:10:04.750270+0000","last_undegraded":"2024-04-25T14:10:04.750270+0000","last_fullsized":"2024-04-25T14:10:04.750270+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:38.191463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"102'119","reported_seq":445,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.740092+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:10:04.740092+0000","last_peered":"2024-04-25T14:10:04.740092+0000","last_clean":"2024-04-25T14:10:04.740092+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:10:04.740092+0000","last_undegraded":"2024-04-25T14:10:04.740092+0000","last_fullsized":"2024-04-25T14:10:04.740092+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":468,"num_read_kb":1272,"num_write":181,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":468,"num_read_kb":1272,"num_write":181,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33108,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338732,"statfs":{"total":96636764160,"available":96602861568,"internally_reserved":0,"allocated":1966080,"data_stored":1273327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32524,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339316,"statfs":{"total":96636764160,"available":96603459584,"internally_reserved":0,"allocated":1372160,"data_stored":682959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1384448,"data_stored":689626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1372160,"data_stored":682959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1384448,"data_stored":689626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1384448,"data_stored":689626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1372160,"data_stored":682959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574924,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33116,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338724,"statfs":{"total":96636764160,"available":96602853376,"internally_reserved":0,"allocated":1978368,"data_stored":1279994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:10:10.260 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:10:10.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.453+0000 7f8808c00640 1 Processor -- start 2024-04-25T14:10:10.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.453+0000 7f8808c00640 1 -- start start 2024-04-25T14:10:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.453+0000 7f8808c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8804071530 0x7f880415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.453+0000 7f8808c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88040cd8d0 con 0x7f88040d2c80 2024-04-25T14:10:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.453+0000 7f8808c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88040ce970 con 0x7f88041616e0 2024-04-25T14:10:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.453+0000 7f8808c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f88040ce720 con 0x7f8804071530 2024-04-25T14:10:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.453+0000 7f8801e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f88040d2c80 0x7f8804070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51880/0 (socket says 172.21.15.86:51880) 2024-04-25T14:10:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.453+0000 7f8801e00640 1 -- 172.21.15.86:0/654608714 learned_addr learned my addr 172.21.15.86:0/654608714 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.453+0000 7f87f9400640 1 --2- 172.21.15.86:0/654608714 >> v2:172.21.15.86:3300/0 conn(0x7f8804071530 0x7f880415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.453+0000 7f87f9400640 1 -- 172.21.15.86:0/654608714 >> v1:172.21.15.155:6789/0 conn(0x7f88041616e0 legacy=0x7f8804161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.453+0000 7f87f9400640 1 -- 172.21.15.86:0/654608714 >> v1:172.21.15.86:6789/0 conn(0x7f88040d2c80 legacy=0x7f8804070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.453+0000 7f87f9400640 1 -- 172.21.15.86:0/654608714 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88040d1be0 con 0x7f8804071530 2024-04-25T14:10:10.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.457+0000 7f87f9400640 1 --2- 172.21.15.86:0/654608714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8804071530 0x7f880415dc90 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f87ec009de0 tx=0x7f87ec01f190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f2709a91b5f220 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:10.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.457+0000 7f8801400640 1 -- 172.21.15.86:0/654608714 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec0b4070 con 0x7f8804071530 2024-04-25T14:10:10.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.457+0000 7f8801400640 1 -- 172.21.15.86:0/654608714 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87ec0af600 con 0x7f8804071530 2024-04-25T14:10:10.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.457+0000 7f8801400640 1 -- 172.21.15.86:0/654608714 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec0af9d0 con 0x7f8804071530 2024-04-25T14:10:10.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.457+0000 7f8808c00640 1 -- 172.21.15.86:0/654608714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8804071530 msgr2=0x7f880415dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:10.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.457+0000 7f8808c00640 1 --2- 172.21.15.86:0/654608714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8804071530 0x7f880415dc90 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f87ec009de0 tx=0x7f87ec01f190 comp rx=0 tx=0).stop 2024-04-25T14:10:10.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.457+0000 7f8808c00640 1 -- 172.21.15.86:0/654608714 shutdown_connections 2024-04-25T14:10:10.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.457+0000 7f8808c00640 1 --2- 172.21.15.86:0/654608714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8804071530 0x7f880415dc90 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:10.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.457+0000 7f8808c00640 1 -- 172.21.15.86:0/654608714 >> 172.21.15.86:0/654608714 conn(0x7f88040cffc0 msgr2=0x7f88040d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:10.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.457+0000 7f8808c00640 1 -- 172.21.15.86:0/654608714 shutdown_connections 2024-04-25T14:10:10.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.457+0000 7f8808c00640 1 -- 172.21.15.86:0/654608714 wait complete. 2024-04-25T14:10:10.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.461+0000 7f8808c00640 1 Processor -- start 2024-04-25T14:10:10.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.461+0000 7f8808c00640 1 -- start start 2024-04-25T14:10:10.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.461+0000 7f8808c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88040d2c80 0x7f88040cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:10.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.461+0000 7f8808c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88041616e0 0x7f880414fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:10.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.461+0000 7f8808c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88040cc190 0x7f8804150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:10.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.461+0000 7f8808c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88040d2480 con 0x7f88040cc190 2024-04-25T14:10:10.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.461+0000 7f8808c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f88040d21a0 con 0x7f88040d2c80 2024-04-25T14:10:10.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.461+0000 7f8808c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f88040d29a0 con 0x7f88041616e0 2024-04-25T14:10:10.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.461+0000 7f8802800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88040cc190 0x7f8804150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:10.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.465+0000 7f8802800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88040cc190 0x7f8804150090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53934/0 (socket says 172.21.15.86:53934) 2024-04-25T14:10:10.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.465+0000 7f8802800640 1 -- 172.21.15.86:0/3741852216 learned_addr learned my addr 172.21.15.86:0/3741852216 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:10.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.465+0000 7f8801e00640 1 --2- 172.21.15.86:0/3741852216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88040d2c80 0x7f88040cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:10.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.465+0000 7f87f9400640 1 --2- 172.21.15.86:0/3741852216 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88041616e0 0x7f880414fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:10.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.465+0000 7f8802800640 1 -- 172.21.15.86:0/3741852216 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88041616e0 msgr2=0x7f880414fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:10.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.465+0000 7f8802800640 1 --2- 172.21.15.86:0/3741852216 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88041616e0 0x7f880414fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:10.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.465+0000 7f8802800640 1 -- 172.21.15.86:0/3741852216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88040d2c80 msgr2=0x7f88040cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:10.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.465+0000 7f8802800640 1 --2- 172.21.15.86:0/3741852216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88040d2c80 0x7f88040cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:10.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.465+0000 7f8802800640 1 -- 172.21.15.86:0/3741852216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88041fa770 con 0x7f88040cc190 2024-04-25T14:10:10.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.465+0000 7f8802800640 1 --2- 172.21.15.86:0/3741852216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88040cc190 0x7f8804150090 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00c310 tx=0x7f87fc00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:10.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.465+0000 7f87faa00640 1 -- 172.21.15.86:0/3741852216 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc017070 con 0x7f88040cc190 2024-04-25T14:10:10.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.465+0000 7f87faa00640 1 -- 172.21.15.86:0/3741852216 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87fc0052d0 con 0x7f88040cc190 2024-04-25T14:10:10.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.465+0000 7f87faa00640 1 -- 172.21.15.86:0/3741852216 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc003800 con 0x7f88040cc190 2024-04-25T14:10:10.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.465+0000 7f8808c00640 1 -- 172.21.15.86:0/3741852216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8804155c00 con 0x7f88040cc190 2024-04-25T14:10:10.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.465+0000 7f8808c00640 1 -- 172.21.15.86:0/3741852216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8804150e60 con 0x7f88040cc190 2024-04-25T14:10:10.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.465+0000 7f8808c00640 1 -- 172.21.15.86:0/3741852216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88040720e0 con 0x7f88040cc190 2024-04-25T14:10:10.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.465+0000 7f87faa00640 1 -- 172.21.15.86:0/3741852216 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7f87fc01a070 con 0x7f88040cc190 2024-04-25T14:10:10.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.469+0000 7f87faa00640 1 --2- 172.21.15.86:0/3741852216 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f87e006f520 0x7f87e0071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:10.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.469+0000 7f87faa00640 1 -- 172.21.15.86:0/3741852216 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f87fc08e640 con 0x7f88040cc190 2024-04-25T14:10:10.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.469+0000 7f8801e00640 1 --2- 172.21.15.86:0/3741852216 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f87e006f520 0x7f87e0071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:10.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.469+0000 7f8801e00640 1 --2- 172.21.15.86:0/3741852216 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f87e006f520 0x7f87e0071a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f87f4005890 tx=0x7f87f4009040 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:10.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.473+0000 7f87faa00640 1 -- 172.21.15.86:0/3741852216 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f87fc005480 con 0x7f88040cc190 2024-04-25T14:10:10.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.625+0000 7f8808c00640 1 -- 172.21.15.86:0/3741852216 --> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8804069d60 con 0x7f87e006f520 2024-04-25T14:10:10.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.633+0000 7f87faa00640 1 -- 172.21.15.86:0/3741852216 <== mgr.8190 v2:172.21.15.86:6832/1354467416 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247091 (secure 0 0 0) 0x7f87f40058c0 con 0x7f87e006f520 2024-04-25T14:10:10.633 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:10.635 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:10:10.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.637+0000 7f8808c00640 1 -- 172.21.15.86:0/3741852216 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f87e006f520 msgr2=0x7f87e0071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:10.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.637+0000 7f8808c00640 1 --2- 172.21.15.86:0/3741852216 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f87e006f520 0x7f87e0071a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f87f4005890 tx=0x7f87f4009040 comp rx=0 tx=0).stop 2024-04-25T14:10:10.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.637+0000 7f8808c00640 1 -- 172.21.15.86:0/3741852216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88040cc190 msgr2=0x7f8804150090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:10.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.637+0000 7f8808c00640 1 --2- 172.21.15.86:0/3741852216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88040cc190 0x7f8804150090 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00c310 tx=0x7f87fc00c7d0 comp rx=0 tx=0).stop 2024-04-25T14:10:10.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.637+0000 7f8808c00640 1 -- 172.21.15.86:0/3741852216 shutdown_connections 2024-04-25T14:10:10.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.637+0000 7f8808c00640 1 --2- 172.21.15.86:0/3741852216 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f87e006f520 0x7f87e0071a30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:10.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.637+0000 7f8808c00640 1 --2- 172.21.15.86:0/3741852216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88040cc190 0x7f8804150090 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:10.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.637+0000 7f8808c00640 1 --2- 172.21.15.86:0/3741852216 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88041616e0 0x7f880414fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:10.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.637+0000 7f8808c00640 1 --2- 172.21.15.86:0/3741852216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88040d2c80 0x7f88040cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:10.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.637+0000 7f8808c00640 1 -- 172.21.15.86:0/3741852216 >> 172.21.15.86:0/3741852216 conn(0x7f88040cffc0 msgr2=0x7f8804159c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:10.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.637+0000 7f8808c00640 1 -- 172.21.15.86:0/3741852216 shutdown_connections 2024-04-25T14:10:10.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.637+0000 7f8808c00640 1 -- 172.21.15.86:0/3741852216 wait complete. 2024-04-25T14:10:10.654 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T14:10:09.474871+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":468,"num_read_kb":1272,"num_write":181,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261404,"kb_used_data":11928,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713316,"statfs":{"total":773094113280,"available":772826435584,"internally_reserved":0,"allocated":12214272,"data_stored":6671076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.761213"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735771+0000","last_change":"2024-04-25T14:10:04.735771+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735771+0000","last_undegraded":"2024-04-25T14:10:04.735771+0000","last_fullsized":"2024-04-25T14:10:04.735771+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735778+0000","last_change":"2024-04-25T14:10:04.735778+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735778+0000","last_undegraded":"2024-04-25T14:10:04.735778+0000","last_fullsized":"2024-04-25T14:10:04.735778+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735849+0000","last_change":"2024-04-25T14:10:04.735849+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735849+0000","last_undegraded":"2024-04-25T14:10:04.735849+0000","last_fullsized":"2024-04-25T14:10:04.735849+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.734374+0000","last_change":"2024-04-25T14:10:04.734374+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.734374+0000","last_undegraded":"2024-04-25T14:10:04.734374+0000","last_fullsized":"2024-04-25T14:10:04.734374+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.734611+0000","last_change":"2024-04-25T14:10:04.734611+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.734611+0000","last_undegraded":"2024-04-25T14:10:04.734611+0000","last_fullsized":"2024-04-25T14:10:04.734611+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.738244+0000","last_change":"2024-04-25T14:10:04.738244+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.738244+0000","last_undegraded":"2024-04-25T14:10:04.738244+0000","last_fullsized":"2024-04-25T14:10:04.738244+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.734697+0000","last_change":"2024-04-25T14:10:04.734697+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.734697+0000","last_undegraded":"2024-04-25T14:10:04.734697+0000","last_fullsized":"2024-04-25T14:10:04.734697+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733337+0000","last_change":"2024-04-25T14:10:04.733337+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733337+0000","last_undegraded":"2024-04-25T14:10:04.733337+0000","last_fullsized":"2024-04-25T14:10:04.733337+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.739896+0000","last_change":"2024-04-25T14:10:04.739896+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.739896+0000","last_undegraded":"2024-04-25T14:10:04.739896+0000","last_fullsized":"2024-04-25T14:10:04.739896+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726838+0000","last_change":"2024-04-25T14:10:04.726838+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726838+0000","last_undegraded":"2024-04-25T14:10:04.726838+0000","last_fullsized":"2024-04-25T14:10:04.726838+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735651+0000","last_change":"2024-04-25T14:10:04.735651+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735651+0000","last_undegraded":"2024-04-25T14:10:04.735651+0000","last_fullsized":"2024-04-25T14:10:04.735651+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.742843+0000","last_change":"2024-04-25T14:10:04.742843+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.742843+0000","last_undegraded":"2024-04-25T14:10:04.742843+0000","last_fullsized":"2024-04-25T14:10:04.742843+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.727264+0000","last_change":"2024-04-25T14:10:04.727264+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.727264+0000","last_undegraded":"2024-04-25T14:10:04.727264+0000","last_fullsized":"2024-04-25T14:10:04.727264+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733351+0000","last_change":"2024-04-25T14:10:04.733351+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733351+0000","last_undegraded":"2024-04-25T14:10:04.733351+0000","last_fullsized":"2024-04-25T14:10:04.733351+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.734380+0000","last_change":"2024-04-25T14:10:04.734380+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.734380+0000","last_undegraded":"2024-04-25T14:10:04.734380+0000","last_fullsized":"2024-04-25T14:10:04.734380+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735156+0000","last_change":"2024-04-25T14:10:04.735156+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735156+0000","last_undegraded":"2024-04-25T14:10:04.735156+0000","last_fullsized":"2024-04-25T14:10:04.735156+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.725975+0000","last_change":"2024-04-25T14:10:04.725975+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.725975+0000","last_undegraded":"2024-04-25T14:10:04.725975+0000","last_fullsized":"2024-04-25T14:10:04.725975+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733263+0000","last_change":"2024-04-25T14:10:04.733263+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733263+0000","last_undegraded":"2024-04-25T14:10:04.733263+0000","last_fullsized":"2024-04-25T14:10:04.733263+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.729768+0000","last_change":"2024-04-25T14:10:04.729768+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.729768+0000","last_undegraded":"2024-04-25T14:10:04.729768+0000","last_fullsized":"2024-04-25T14:10:04.729768+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.737805+0000","last_change":"2024-04-25T14:10:04.737805+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.737805+0000","last_undegraded":"2024-04-25T14:10:04.737805+0000","last_fullsized":"2024-04-25T14:10:04.737805+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735963+0000","last_change":"2024-04-25T14:10:04.735963+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735963+0000","last_undegraded":"2024-04-25T14:10:04.735963+0000","last_fullsized":"2024-04-25T14:10:04.735963+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733684+0000","last_change":"2024-04-25T14:10:04.733684+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733684+0000","last_undegraded":"2024-04-25T14:10:04.733684+0000","last_fullsized":"2024-04-25T14:10:04.733684+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.732828+0000","last_change":"2024-04-25T14:10:04.732828+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.732828+0000","last_undegraded":"2024-04-25T14:10:04.732828+0000","last_fullsized":"2024-04-25T14:10:04.732828+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.727648+0000","last_change":"2024-04-25T14:10:04.727648+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.727648+0000","last_undegraded":"2024-04-25T14:10:04.727648+0000","last_fullsized":"2024-04-25T14:10:04.727648+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.728155+0000","last_change":"2024-04-25T14:10:04.728155+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.728155+0000","last_undegraded":"2024-04-25T14:10:04.728155+0000","last_fullsized":"2024-04-25T14:10:04.728155+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735818+0000","last_change":"2024-04-25T14:10:04.735818+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735818+0000","last_undegraded":"2024-04-25T14:10:04.735818+0000","last_fullsized":"2024-04-25T14:10:04.735818+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.729202+0000","last_change":"2024-04-25T14:10:04.729202+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.729202+0000","last_undegraded":"2024-04-25T14:10:04.729202+0000","last_fullsized":"2024-04-25T14:10:04.729202+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.738433+0000","last_change":"2024-04-25T14:10:04.738433+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.738433+0000","last_undegraded":"2024-04-25T14:10:04.738433+0000","last_fullsized":"2024-04-25T14:10:04.738433+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.737523+0000","last_change":"2024-04-25T14:10:04.737523+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.737523+0000","last_undegraded":"2024-04-25T14:10:04.737523+0000","last_fullsized":"2024-04-25T14:10:04.737523+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.732817+0000","last_change":"2024-04-25T14:10:04.732817+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.732817+0000","last_undegraded":"2024-04-25T14:10:04.732817+0000","last_fullsized":"2024-04-25T14:10:04.732817+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726801+0000","last_change":"2024-04-25T14:10:04.726801+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726801+0000","last_undegraded":"2024-04-25T14:10:04.726801+0000","last_fullsized":"2024-04-25T14:10:04.726801+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726485+0000","last_change":"2024-04-25T14:10:04.726485+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726485+0000","last_undegraded":"2024-04-25T14:10:04.726485+0000","last_fullsized":"2024-04-25T14:10:04.726485+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726867+0000","last_change":"2024-04-25T14:10:04.726867+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726867+0000","last_undegraded":"2024-04-25T14:10:04.726867+0000","last_fullsized":"2024-04-25T14:10:04.726867+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.749691+0000","last_change":"2024-04-25T14:10:04.745986+0000","last_active":"2024-04-25T14:10:04.749691+0000","last_peered":"2024-04-25T14:10:04.749691+0000","last_clean":"2024-04-25T14:10:04.749691+0000","last_became_active":"2024-04-25T14:10:04.745516+0000","last_became_peered":"2024-04-25T14:10:04.745516+0000","last_unstale":"2024-04-25T14:10:04.749691+0000","last_undegraded":"2024-04-25T14:10:04.749691+0000","last_fullsized":"2024-04-25T14:10:04.749691+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:45.118281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735537+0000","last_change":"2024-04-25T14:10:04.735537+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735537+0000","last_undegraded":"2024-04-25T14:10:04.735537+0000","last_fullsized":"2024-04-25T14:10:04.735537+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.746437+0000","last_change":"2024-04-25T14:10:04.744056+0000","last_active":"2024-04-25T14:10:04.746437+0000","last_peered":"2024-04-25T14:10:04.746437+0000","last_clean":"2024-04-25T14:10:04.746437+0000","last_became_active":"2024-04-25T14:10:04.743755+0000","last_became_peered":"2024-04-25T14:10:04.743755+0000","last_unstale":"2024-04-25T14:10:04.746437+0000","last_undegraded":"2024-04-25T14:10:04.746437+0000","last_fullsized":"2024-04-25T14:10:04.746437+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:45.118281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.728128+0000","last_change":"2024-04-25T14:10:04.728128+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.728128+0000","last_undegraded":"2024-04-25T14:10:04.728128+0000","last_fullsized":"2024-04-25T14:10:04.728128+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.755092+0000","last_change":"2024-04-25T14:10:04.752833+0000","last_active":"2024-04-25T14:10:04.755092+0000","last_peered":"2024-04-25T14:10:04.755092+0000","last_clean":"2024-04-25T14:10:04.755092+0000","last_became_active":"2024-04-25T14:10:04.752483+0000","last_became_peered":"2024-04-25T14:10:04.752483+0000","last_unstale":"2024-04-25T14:10:04.755092+0000","last_undegraded":"2024-04-25T14:10:04.755092+0000","last_fullsized":"2024-04-25T14:10:04.755092+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:50.427076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.738654+0000","last_change":"2024-04-25T14:10:04.738654+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.738654+0000","last_undegraded":"2024-04-25T14:10:04.738654+0000","last_fullsized":"2024-04-25T14:10:04.738654+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.737624+0000","last_change":"2024-04-25T14:10:04.733242+0000","last_active":"2024-04-25T14:10:04.737624+0000","last_peered":"2024-04-25T14:10:04.737624+0000","last_clean":"2024-04-25T14:10:04.737624+0000","last_became_active":"2024-04-25T14:10:04.732929+0000","last_became_peered":"2024-04-25T14:10:04.732929+0000","last_unstale":"2024-04-25T14:10:04.737624+0000","last_undegraded":"2024-04-25T14:10:04.737624+0000","last_fullsized":"2024-04-25T14:10:04.737624+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:25.644454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.725539+0000","last_change":"2024-04-25T14:10:04.725539+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.725539+0000","last_undegraded":"2024-04-25T14:10:04.725539+0000","last_fullsized":"2024-04-25T14:10:04.725539+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.744498+0000","last_change":"2024-04-25T14:10:04.738920+0000","last_active":"2024-04-25T14:10:04.744498+0000","last_peered":"2024-04-25T14:10:04.744498+0000","last_clean":"2024-04-25T14:10:04.744498+0000","last_became_active":"2024-04-25T14:10:04.738531+0000","last_became_peered":"2024-04-25T14:10:04.738531+0000","last_unstale":"2024-04-25T14:10:04.744498+0000","last_undegraded":"2024-04-25T14:10:04.744498+0000","last_fullsized":"2024-04-25T14:10:04.744498+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:45.118281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733780+0000","last_change":"2024-04-25T14:10:04.733780+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733780+0000","last_undegraded":"2024-04-25T14:10:04.733780+0000","last_fullsized":"2024-04-25T14:10:04.733780+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.754194+0000","last_change":"2024-04-25T14:10:04.749193+0000","last_active":"2024-04-25T14:10:04.754194+0000","last_peered":"2024-04-25T14:10:04.754194+0000","last_clean":"2024-04-25T14:10:04.754194+0000","last_became_active":"2024-04-25T14:10:04.747285+0000","last_became_peered":"2024-04-25T14:10:04.747285+0000","last_unstale":"2024-04-25T14:10:04.754194+0000","last_undegraded":"2024-04-25T14:10:04.754194+0000","last_fullsized":"2024-04-25T14:10:04.754194+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:19:34.506848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.732355+0000","last_change":"2024-04-25T14:10:04.732355+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.732355+0000","last_undegraded":"2024-04-25T14:10:04.732355+0000","last_fullsized":"2024-04-25T14:10:04.732355+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.740654+0000","last_change":"2024-04-25T14:10:04.734364+0000","last_active":"2024-04-25T14:10:04.740654+0000","last_peered":"2024-04-25T14:10:04.740654+0000","last_clean":"2024-04-25T14:10:04.740654+0000","last_became_active":"2024-04-25T14:10:04.734026+0000","last_became_peered":"2024-04-25T14:10:04.734026+0000","last_unstale":"2024-04-25T14:10:04.740654+0000","last_undegraded":"2024-04-25T14:10:04.740654+0000","last_fullsized":"2024-04-25T14:10:04.740654+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:48.842742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726718+0000","last_change":"2024-04-25T14:10:04.726718+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726718+0000","last_undegraded":"2024-04-25T14:10:04.726718+0000","last_fullsized":"2024-04-25T14:10:04.726718+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.753241+0000","last_change":"2024-04-25T14:10:04.745258+0000","last_active":"2024-04-25T14:10:04.753241+0000","last_peered":"2024-04-25T14:10:04.753241+0000","last_clean":"2024-04-25T14:10:04.753241+0000","last_became_active":"2024-04-25T14:10:04.744962+0000","last_became_peered":"2024-04-25T14:10:04.744962+0000","last_unstale":"2024-04-25T14:10:04.753241+0000","last_undegraded":"2024-04-25T14:10:04.753241+0000","last_fullsized":"2024-04-25T14:10:04.753241+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:58.400437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.729428+0000","last_change":"2024-04-25T14:10:04.729428+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.729428+0000","last_undegraded":"2024-04-25T14:10:04.729428+0000","last_fullsized":"2024-04-25T14:10:04.729428+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.754546+0000","last_change":"2024-04-25T14:10:04.750916+0000","last_active":"2024-04-25T14:10:04.754546+0000","last_peered":"2024-04-25T14:10:04.754546+0000","last_clean":"2024-04-25T14:10:04.754546+0000","last_became_active":"2024-04-25T14:10:04.750562+0000","last_became_peered":"2024-04-25T14:10:04.750562+0000","last_unstale":"2024-04-25T14:10:04.754546+0000","last_undegraded":"2024-04-25T14:10:04.754546+0000","last_fullsized":"2024-04-25T14:10:04.754546+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:50.427076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726923+0000","last_change":"2024-04-25T14:10:04.726923+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726923+0000","last_undegraded":"2024-04-25T14:10:04.726923+0000","last_fullsized":"2024-04-25T14:10:04.726923+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.749495+0000","last_change":"2024-04-25T14:10:04.746574+0000","last_active":"2024-04-25T14:10:04.749495+0000","last_peered":"2024-04-25T14:10:04.749495+0000","last_clean":"2024-04-25T14:10:04.749495+0000","last_became_active":"2024-04-25T14:10:04.746266+0000","last_became_peered":"2024-04-25T14:10:04.746266+0000","last_unstale":"2024-04-25T14:10:04.749495+0000","last_undegraded":"2024-04-25T14:10:04.749495+0000","last_fullsized":"2024-04-25T14:10:04.749495+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:28.147163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.732111+0000","last_change":"2024-04-25T14:10:04.732111+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.732111+0000","last_undegraded":"2024-04-25T14:10:04.732111+0000","last_fullsized":"2024-04-25T14:10:04.732111+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.748958+0000","last_change":"2024-04-25T14:10:04.745704+0000","last_active":"2024-04-25T14:10:04.748958+0000","last_peered":"2024-04-25T14:10:04.748958+0000","last_clean":"2024-04-25T14:10:04.748958+0000","last_became_active":"2024-04-25T14:10:04.745401+0000","last_became_peered":"2024-04-25T14:10:04.745401+0000","last_unstale":"2024-04-25T14:10:04.748958+0000","last_undegraded":"2024-04-25T14:10:04.748958+0000","last_fullsized":"2024-04-25T14:10:04.748958+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:24.704172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.727533+0000","last_change":"2024-04-25T14:10:04.727533+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.727533+0000","last_undegraded":"2024-04-25T14:10:04.727533+0000","last_fullsized":"2024-04-25T14:10:04.727533+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.749695+0000","last_change":"2024-04-25T14:10:04.745297+0000","last_active":"2024-04-25T14:10:04.749695+0000","last_peered":"2024-04-25T14:10:04.749695+0000","last_clean":"2024-04-25T14:10:04.749695+0000","last_became_active":"2024-04-25T14:10:04.744890+0000","last_became_peered":"2024-04-25T14:10:04.744890+0000","last_unstale":"2024-04-25T14:10:04.749695+0000","last_undegraded":"2024-04-25T14:10:04.749695+0000","last_fullsized":"2024-04-25T14:10:04.749695+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:39.794816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.730347+0000","last_change":"2024-04-25T14:10:04.730347+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.730347+0000","last_undegraded":"2024-04-25T14:10:04.730347+0000","last_fullsized":"2024-04-25T14:10:04.730347+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.750158+0000","last_change":"2024-04-25T14:10:04.746252+0000","last_active":"2024-04-25T14:10:04.750158+0000","last_peered":"2024-04-25T14:10:04.750158+0000","last_clean":"2024-04-25T14:10:04.750158+0000","last_became_active":"2024-04-25T14:10:04.745299+0000","last_became_peered":"2024-04-25T14:10:04.745299+0000","last_unstale":"2024-04-25T14:10:04.750158+0000","last_undegraded":"2024-04-25T14:10:04.750158+0000","last_fullsized":"2024-04-25T14:10:04.750158+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:05.444701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.725553+0000","last_change":"2024-04-25T14:10:04.725553+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.725553+0000","last_undegraded":"2024-04-25T14:10:04.725553+0000","last_fullsized":"2024-04-25T14:10:04.725553+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.745756+0000","last_change":"2024-04-25T14:10:04.740086+0000","last_active":"2024-04-25T14:10:04.745756+0000","last_peered":"2024-04-25T14:10:04.745756+0000","last_clean":"2024-04-25T14:10:04.745756+0000","last_became_active":"2024-04-25T14:10:04.739141+0000","last_became_peered":"2024-04-25T14:10:04.739141+0000","last_unstale":"2024-04-25T14:10:04.745756+0000","last_undegraded":"2024-04-25T14:10:04.745756+0000","last_fullsized":"2024-04-25T14:10:04.745756+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:16.658408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.736954+0000","last_change":"2024-04-25T14:10:04.736954+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.736954+0000","last_undegraded":"2024-04-25T14:10:04.736954+0000","last_fullsized":"2024-04-25T14:10:04.736954+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.746391+0000","last_change":"2024-04-25T14:10:04.743146+0000","last_active":"2024-04-25T14:10:04.746391+0000","last_peered":"2024-04-25T14:10:04.746391+0000","last_clean":"2024-04-25T14:10:04.746391+0000","last_became_active":"2024-04-25T14:10:04.742805+0000","last_became_peered":"2024-04-25T14:10:04.742805+0000","last_unstale":"2024-04-25T14:10:04.746391+0000","last_undegraded":"2024-04-25T14:10:04.746391+0000","last_fullsized":"2024-04-25T14:10:04.746391+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:52.358091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.728072+0000","last_change":"2024-04-25T14:10:04.728072+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.728072+0000","last_undegraded":"2024-04-25T14:10:04.728072+0000","last_fullsized":"2024-04-25T14:10:04.728072+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.753810+0000","last_change":"2024-04-25T14:10:04.751976+0000","last_active":"2024-04-25T14:10:04.753810+0000","last_peered":"2024-04-25T14:10:04.753810+0000","last_clean":"2024-04-25T14:10:04.753810+0000","last_became_active":"2024-04-25T14:10:04.751610+0000","last_became_peered":"2024-04-25T14:10:04.751610+0000","last_unstale":"2024-04-25T14:10:04.753810+0000","last_undegraded":"2024-04-25T14:10:04.753810+0000","last_fullsized":"2024-04-25T14:10:04.753810+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:54.546901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735165+0000","last_change":"2024-04-25T14:10:04.735165+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735165+0000","last_undegraded":"2024-04-25T14:10:04.735165+0000","last_fullsized":"2024-04-25T14:10:04.735165+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.752176+0000","last_change":"2024-04-25T14:10:04.749720+0000","last_active":"2024-04-25T14:10:04.752176+0000","last_peered":"2024-04-25T14:10:04.752176+0000","last_clean":"2024-04-25T14:10:04.752176+0000","last_became_active":"2024-04-25T14:10:04.749327+0000","last_became_peered":"2024-04-25T14:10:04.749327+0000","last_unstale":"2024-04-25T14:10:04.752176+0000","last_undegraded":"2024-04-25T14:10:04.752176+0000","last_fullsized":"2024-04-25T14:10:04.752176+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:30.105615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.738871+0000","last_change":"2024-04-25T14:10:04.738871+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.738871+0000","last_undegraded":"2024-04-25T14:10:04.738871+0000","last_fullsized":"2024-04-25T14:10:04.738871+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.752711+0000","last_change":"2024-04-25T14:10:04.750043+0000","last_active":"2024-04-25T14:10:04.752711+0000","last_peered":"2024-04-25T14:10:04.752711+0000","last_clean":"2024-04-25T14:10:04.752711+0000","last_became_active":"2024-04-25T14:10:04.749722+0000","last_became_peered":"2024-04-25T14:10:04.749722+0000","last_unstale":"2024-04-25T14:10:04.752711+0000","last_undegraded":"2024-04-25T14:10:04.752711+0000","last_fullsized":"2024-04-25T14:10:04.752711+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:54.546901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726953+0000","last_change":"2024-04-25T14:10:04.726953+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726953+0000","last_undegraded":"2024-04-25T14:10:04.726953+0000","last_fullsized":"2024-04-25T14:10:04.726953+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.755134+0000","last_change":"2024-04-25T14:10:04.754055+0000","last_active":"2024-04-25T14:10:04.755134+0000","last_peered":"2024-04-25T14:10:04.755134+0000","last_clean":"2024-04-25T14:10:04.755134+0000","last_became_active":"2024-04-25T14:10:04.753740+0000","last_became_peered":"2024-04-25T14:10:04.753740+0000","last_unstale":"2024-04-25T14:10:04.755134+0000","last_undegraded":"2024-04-25T14:10:04.755134+0000","last_fullsized":"2024-04-25T14:10:04.755134+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:16.658408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.734586+0000","last_change":"2024-04-25T14:10:04.734586+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.734586+0000","last_undegraded":"2024-04-25T14:10:04.734586+0000","last_fullsized":"2024-04-25T14:10:04.734586+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.754869+0000","last_change":"2024-04-25T14:10:04.753046+0000","last_active":"2024-04-25T14:10:04.754869+0000","last_peered":"2024-04-25T14:10:04.754869+0000","last_clean":"2024-04-25T14:10:04.754869+0000","last_became_active":"2024-04-25T14:10:04.749881+0000","last_became_peered":"2024-04-25T14:10:04.749881+0000","last_unstale":"2024-04-25T14:10:04.754869+0000","last_undegraded":"2024-04-25T14:10:04.754869+0000","last_fullsized":"2024-04-25T14:10:04.754869+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:39.976208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.731633+0000","last_change":"2024-04-25T14:10:04.731633+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.731633+0000","last_undegraded":"2024-04-25T14:10:04.731633+0000","last_fullsized":"2024-04-25T14:10:04.731633+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.752192+0000","last_change":"2024-04-25T14:10:04.749068+0000","last_active":"2024-04-25T14:10:04.752192+0000","last_peered":"2024-04-25T14:10:04.752192+0000","last_clean":"2024-04-25T14:10:04.752192+0000","last_became_active":"2024-04-25T14:10:04.748714+0000","last_became_peered":"2024-04-25T14:10:04.748714+0000","last_unstale":"2024-04-25T14:10:04.752192+0000","last_undegraded":"2024-04-25T14:10:04.752192+0000","last_fullsized":"2024-04-25T14:10:04.752192+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:45.118281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.731871+0000","last_change":"2024-04-25T14:10:04.731871+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.731871+0000","last_undegraded":"2024-04-25T14:10:04.731871+0000","last_fullsized":"2024-04-25T14:10:04.731871+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.750229+0000","last_change":"2024-04-25T14:10:04.746533+0000","last_active":"2024-04-25T14:10:04.750229+0000","last_peered":"2024-04-25T14:10:04.750229+0000","last_clean":"2024-04-25T14:10:04.750229+0000","last_became_active":"2024-04-25T14:10:04.746152+0000","last_became_peered":"2024-04-25T14:10:04.746152+0000","last_unstale":"2024-04-25T14:10:04.750229+0000","last_undegraded":"2024-04-25T14:10:04.750229+0000","last_fullsized":"2024-04-25T14:10:04.750229+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:01.747339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726535+0000","last_change":"2024-04-25T14:10:04.726535+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726535+0000","last_undegraded":"2024-04-25T14:10:04.726535+0000","last_fullsized":"2024-04-25T14:10:04.726535+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.752736+0000","last_change":"2024-04-25T14:10:04.752102+0000","last_active":"2024-04-25T14:10:04.752736+0000","last_peered":"2024-04-25T14:10:04.752736+0000","last_clean":"2024-04-25T14:10:04.752736+0000","last_became_active":"2024-04-25T14:10:04.751649+0000","last_became_peered":"2024-04-25T14:10:04.751649+0000","last_unstale":"2024-04-25T14:10:04.752736+0000","last_undegraded":"2024-04-25T14:10:04.752736+0000","last_fullsized":"2024-04-25T14:10:04.752736+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.238156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.736386+0000","last_change":"2024-04-25T14:10:04.736386+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.736386+0000","last_undegraded":"2024-04-25T14:10:04.736386+0000","last_fullsized":"2024-04-25T14:10:04.736386+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.754240+0000","last_change":"2024-04-25T14:10:04.749248+0000","last_active":"2024-04-25T14:10:04.754240+0000","last_peered":"2024-04-25T14:10:04.754240+0000","last_clean":"2024-04-25T14:10:04.754240+0000","last_became_active":"2024-04-25T14:10:04.748828+0000","last_became_peered":"2024-04-25T14:10:04.748828+0000","last_unstale":"2024-04-25T14:10:04.754240+0000","last_undegraded":"2024-04-25T14:10:04.754240+0000","last_fullsized":"2024-04-25T14:10:04.754240+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:38.859877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726322+0000","last_change":"2024-04-25T14:10:04.726322+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726322+0000","last_undegraded":"2024-04-25T14:10:04.726322+0000","last_fullsized":"2024-04-25T14:10:04.726322+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.749775+0000","last_change":"2024-04-25T14:10:04.746371+0000","last_active":"2024-04-25T14:10:04.749775+0000","last_peered":"2024-04-25T14:10:04.749775+0000","last_clean":"2024-04-25T14:10:04.749775+0000","last_became_active":"2024-04-25T14:10:04.745733+0000","last_became_peered":"2024-04-25T14:10:04.745733+0000","last_unstale":"2024-04-25T14:10:04.749775+0000","last_undegraded":"2024-04-25T14:10:04.749775+0000","last_fullsized":"2024-04-25T14:10:04.749775+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:54.546901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733546+0000","last_change":"2024-04-25T14:10:04.733546+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733546+0000","last_undegraded":"2024-04-25T14:10:04.733546+0000","last_fullsized":"2024-04-25T14:10:04.733546+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.752120+0000","last_change":"2024-04-25T14:10:04.747682+0000","last_active":"2024-04-25T14:10:04.752120+0000","last_peered":"2024-04-25T14:10:04.752120+0000","last_clean":"2024-04-25T14:10:04.752120+0000","last_became_active":"2024-04-25T14:10:04.747380+0000","last_became_peered":"2024-04-25T14:10:04.747380+0000","last_unstale":"2024-04-25T14:10:04.752120+0000","last_undegraded":"2024-04-25T14:10:04.752120+0000","last_fullsized":"2024-04-25T14:10:04.752120+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.238156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.736394+0000","last_change":"2024-04-25T14:10:04.736394+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.736394+0000","last_undegraded":"2024-04-25T14:10:04.736394+0000","last_fullsized":"2024-04-25T14:10:04.736394+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.754913+0000","last_change":"2024-04-25T14:10:04.752523+0000","last_active":"2024-04-25T14:10:04.754913+0000","last_peered":"2024-04-25T14:10:04.754913+0000","last_clean":"2024-04-25T14:10:04.754913+0000","last_became_active":"2024-04-25T14:10:04.750781+0000","last_became_peered":"2024-04-25T14:10:04.750781+0000","last_unstale":"2024-04-25T14:10:04.754913+0000","last_undegraded":"2024-04-25T14:10:04.754913+0000","last_fullsized":"2024-04-25T14:10:04.754913+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:30.252303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733923+0000","last_change":"2024-04-25T14:10:04.733923+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733923+0000","last_undegraded":"2024-04-25T14:10:04.733923+0000","last_fullsized":"2024-04-25T14:10:04.733923+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.748268+0000","last_change":"2024-04-25T14:10:04.746161+0000","last_active":"2024-04-25T14:10:04.748268+0000","last_peered":"2024-04-25T14:10:04.748268+0000","last_clean":"2024-04-25T14:10:04.748268+0000","last_became_active":"2024-04-25T14:10:04.745423+0000","last_became_peered":"2024-04-25T14:10:04.745423+0000","last_unstale":"2024-04-25T14:10:04.748268+0000","last_undegraded":"2024-04-25T14:10:04.748268+0000","last_fullsized":"2024-04-25T14:10:04.748268+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.238156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.730645+0000","last_change":"2024-04-25T14:10:04.730645+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.730645+0000","last_undegraded":"2024-04-25T14:10:04.730645+0000","last_fullsized":"2024-04-25T14:10:04.730645+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.747112+0000","last_change":"2024-04-25T14:10:04.742349+0000","last_active":"2024-04-25T14:10:04.747112+0000","last_peered":"2024-04-25T14:10:04.747112+0000","last_clean":"2024-04-25T14:10:04.747112+0000","last_became_active":"2024-04-25T14:10:04.741292+0000","last_became_peered":"2024-04-25T14:10:04.741292+0000","last_unstale":"2024-04-25T14:10:04.747112+0000","last_undegraded":"2024-04-25T14:10:04.747112+0000","last_fullsized":"2024-04-25T14:10:04.747112+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:54.546901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733889+0000","last_change":"2024-04-25T14:10:04.733889+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733889+0000","last_undegraded":"2024-04-25T14:10:04.733889+0000","last_fullsized":"2024-04-25T14:10:04.733889+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.754166+0000","last_change":"2024-04-25T14:10:04.749122+0000","last_active":"2024-04-25T14:10:04.754166+0000","last_peered":"2024-04-25T14:10:04.754166+0000","last_clean":"2024-04-25T14:10:04.754166+0000","last_became_active":"2024-04-25T14:10:04.748630+0000","last_became_peered":"2024-04-25T14:10:04.748630+0000","last_unstale":"2024-04-25T14:10:04.754166+0000","last_undegraded":"2024-04-25T14:10:04.754166+0000","last_fullsized":"2024-04-25T14:10:04.754166+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:21.269555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726229+0000","last_change":"2024-04-25T14:10:04.726229+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726229+0000","last_undegraded":"2024-04-25T14:10:04.726229+0000","last_fullsized":"2024-04-25T14:10:04.726229+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.749405+0000","last_change":"2024-04-25T14:10:04.745286+0000","last_active":"2024-04-25T14:10:04.749405+0000","last_peered":"2024-04-25T14:10:04.749405+0000","last_clean":"2024-04-25T14:10:04.749405+0000","last_became_active":"2024-04-25T14:10:04.744949+0000","last_became_peered":"2024-04-25T14:10:04.744949+0000","last_unstale":"2024-04-25T14:10:04.749405+0000","last_undegraded":"2024-04-25T14:10:04.749405+0000","last_fullsized":"2024-04-25T14:10:04.749405+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:25.644454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.734790+0000","last_change":"2024-04-25T14:10:04.734790+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.734790+0000","last_undegraded":"2024-04-25T14:10:04.734790+0000","last_fullsized":"2024-04-25T14:10:04.734790+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.750270+0000","last_change":"2024-04-25T14:10:04.747339+0000","last_active":"2024-04-25T14:10:04.750270+0000","last_peered":"2024-04-25T14:10:04.750270+0000","last_clean":"2024-04-25T14:10:04.750270+0000","last_became_active":"2024-04-25T14:10:04.745494+0000","last_became_peered":"2024-04-25T14:10:04.745494+0000","last_unstale":"2024-04-25T14:10:04.750270+0000","last_undegraded":"2024-04-25T14:10:04.750270+0000","last_fullsized":"2024-04-25T14:10:04.750270+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:38.191463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"102'119","reported_seq":445,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.740092+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:10:04.740092+0000","last_peered":"2024-04-25T14:10:04.740092+0000","last_clean":"2024-04-25T14:10:04.740092+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:10:04.740092+0000","last_undegraded":"2024-04-25T14:10:04.740092+0000","last_fullsized":"2024-04-25T14:10:04.740092+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":468,"num_read_kb":1272,"num_write":181,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":468,"num_read_kb":1272,"num_write":181,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33108,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338732,"statfs":{"total":96636764160,"available":96602861568,"internally_reserved":0,"allocated":1966080,"data_stored":1273327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32524,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339316,"statfs":{"total":96636764160,"available":96603459584,"internally_reserved":0,"allocated":1372160,"data_stored":682959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1384448,"data_stored":689626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1372160,"data_stored":682959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1384448,"data_stored":689626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1384448,"data_stored":689626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1372160,"data_stored":682959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574924,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33116,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338724,"statfs":{"total":96636764160,"available":96602853376,"internally_reserved":0,"allocated":1978368,"data_stored":1279994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:10:10.657 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3f is not active+clean 2024-04-25T14:10:10.658 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.735771+0000', 'last_change': '2024-04-25T14:10:04.735771+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.735771+0000', 'last_undegraded': '2024-04-25T14:10:04.735771+0000', 'last_fullsized': '2024-04-25T14:10:04.735771+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:10:10.658 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3e is not active+clean 2024-04-25T14:10:10.658 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.735778+0000', 'last_change': '2024-04-25T14:10:04.735778+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.735778+0000', 'last_undegraded': '2024-04-25T14:10:04.735778+0000', 'last_fullsized': '2024-04-25T14:10:04.735778+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:10:10.658 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3d is not active+clean 2024-04-25T14:10:10.658 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.735849+0000', 'last_change': '2024-04-25T14:10:04.735849+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.735849+0000', 'last_undegraded': '2024-04-25T14:10:04.735849+0000', 'last_fullsized': '2024-04-25T14:10:04.735849+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:10:10.658 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3c is not active+clean 2024-04-25T14:10:10.658 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.734374+0000', 'last_change': '2024-04-25T14:10:04.734374+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.734374+0000', 'last_undegraded': '2024-04-25T14:10:04.734374+0000', 'last_fullsized': '2024-04-25T14:10:04.734374+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:10:10.658 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3b is not active+clean 2024-04-25T14:10:10.658 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.734611+0000', 'last_change': '2024-04-25T14:10:04.734611+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.734611+0000', 'last_undegraded': '2024-04-25T14:10:04.734611+0000', 'last_fullsized': '2024-04-25T14:10:04.734611+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:10:10.659 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3a is not active+clean 2024-04-25T14:10:10.659 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.738244+0000', 'last_change': '2024-04-25T14:10:04.738244+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.738244+0000', 'last_undegraded': '2024-04-25T14:10:04.738244+0000', 'last_fullsized': '2024-04-25T14:10:04.738244+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:10:10.659 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.39 is not active+clean 2024-04-25T14:10:10.659 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.734697+0000', 'last_change': '2024-04-25T14:10:04.734697+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.734697+0000', 'last_undegraded': '2024-04-25T14:10:04.734697+0000', 'last_fullsized': '2024-04-25T14:10:04.734697+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:10:10.659 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.38 is not active+clean 2024-04-25T14:10:10.659 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.733337+0000', 'last_change': '2024-04-25T14:10:04.733337+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.733337+0000', 'last_undegraded': '2024-04-25T14:10:04.733337+0000', 'last_fullsized': '2024-04-25T14:10:04.733337+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:10:10.659 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.37 is not active+clean 2024-04-25T14:10:10.659 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.739896+0000', 'last_change': '2024-04-25T14:10:04.739896+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.739896+0000', 'last_undegraded': '2024-04-25T14:10:04.739896+0000', 'last_fullsized': '2024-04-25T14:10:04.739896+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:10:10.659 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.36 is not active+clean 2024-04-25T14:10:10.659 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.726838+0000', 'last_change': '2024-04-25T14:10:04.726838+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.726838+0000', 'last_undegraded': '2024-04-25T14:10:04.726838+0000', 'last_fullsized': '2024-04-25T14:10:04.726838+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:10:10.659 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.35 is not active+clean 2024-04-25T14:10:10.659 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.735651+0000', 'last_change': '2024-04-25T14:10:04.735651+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.735651+0000', 'last_undegraded': '2024-04-25T14:10:04.735651+0000', 'last_fullsized': '2024-04-25T14:10:04.735651+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:10:10.660 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.34 is not active+clean 2024-04-25T14:10:10.660 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.742843+0000', 'last_change': '2024-04-25T14:10:04.742843+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.742843+0000', 'last_undegraded': '2024-04-25T14:10:04.742843+0000', 'last_fullsized': '2024-04-25T14:10:04.742843+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:10:10.660 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.33 is not active+clean 2024-04-25T14:10:10.660 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.727264+0000', 'last_change': '2024-04-25T14:10:04.727264+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.727264+0000', 'last_undegraded': '2024-04-25T14:10:04.727264+0000', 'last_fullsized': '2024-04-25T14:10:04.727264+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:10:10.660 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.32 is not active+clean 2024-04-25T14:10:10.660 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.733351+0000', 'last_change': '2024-04-25T14:10:04.733351+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.733351+0000', 'last_undegraded': '2024-04-25T14:10:04.733351+0000', 'last_fullsized': '2024-04-25T14:10:04.733351+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:10:10.660 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.31 is not active+clean 2024-04-25T14:10:10.660 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.734380+0000', 'last_change': '2024-04-25T14:10:04.734380+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.734380+0000', 'last_undegraded': '2024-04-25T14:10:04.734380+0000', 'last_fullsized': '2024-04-25T14:10:04.734380+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:10:10.660 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.30 is not active+clean 2024-04-25T14:10:10.660 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.735156+0000', 'last_change': '2024-04-25T14:10:04.735156+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.735156+0000', 'last_undegraded': '2024-04-25T14:10:04.735156+0000', 'last_fullsized': '2024-04-25T14:10:04.735156+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:10:10.660 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2f is not active+clean 2024-04-25T14:10:10.660 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.725975+0000', 'last_change': '2024-04-25T14:10:04.725975+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.725975+0000', 'last_undegraded': '2024-04-25T14:10:04.725975+0000', 'last_fullsized': '2024-04-25T14:10:04.725975+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:10:10.661 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2e is not active+clean 2024-04-25T14:10:10.661 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.733263+0000', 'last_change': '2024-04-25T14:10:04.733263+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.733263+0000', 'last_undegraded': '2024-04-25T14:10:04.733263+0000', 'last_fullsized': '2024-04-25T14:10:04.733263+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:10:10.661 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2d is not active+clean 2024-04-25T14:10:10.661 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.729768+0000', 'last_change': '2024-04-25T14:10:04.729768+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.729768+0000', 'last_undegraded': '2024-04-25T14:10:04.729768+0000', 'last_fullsized': '2024-04-25T14:10:04.729768+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:10:10.661 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2c is not active+clean 2024-04-25T14:10:10.661 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.737805+0000', 'last_change': '2024-04-25T14:10:04.737805+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.737805+0000', 'last_undegraded': '2024-04-25T14:10:04.737805+0000', 'last_fullsized': '2024-04-25T14:10:04.737805+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:10:10.661 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2b is not active+clean 2024-04-25T14:10:10.661 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.735963+0000', 'last_change': '2024-04-25T14:10:04.735963+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.735963+0000', 'last_undegraded': '2024-04-25T14:10:04.735963+0000', 'last_fullsized': '2024-04-25T14:10:04.735963+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:10:10.661 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2a is not active+clean 2024-04-25T14:10:10.661 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.733684+0000', 'last_change': '2024-04-25T14:10:04.733684+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.733684+0000', 'last_undegraded': '2024-04-25T14:10:04.733684+0000', 'last_fullsized': '2024-04-25T14:10:04.733684+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:10:10.661 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.29 is not active+clean 2024-04-25T14:10:10.662 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.732828+0000', 'last_change': '2024-04-25T14:10:04.732828+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.732828+0000', 'last_undegraded': '2024-04-25T14:10:04.732828+0000', 'last_fullsized': '2024-04-25T14:10:04.732828+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:10:10.662 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.28 is not active+clean 2024-04-25T14:10:10.662 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.727648+0000', 'last_change': '2024-04-25T14:10:04.727648+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.727648+0000', 'last_undegraded': '2024-04-25T14:10:04.727648+0000', 'last_fullsized': '2024-04-25T14:10:04.727648+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:10:10.662 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.27 is not active+clean 2024-04-25T14:10:10.662 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.728155+0000', 'last_change': '2024-04-25T14:10:04.728155+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.728155+0000', 'last_undegraded': '2024-04-25T14:10:04.728155+0000', 'last_fullsized': '2024-04-25T14:10:04.728155+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:10:10.662 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.26 is not active+clean 2024-04-25T14:10:10.662 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.735818+0000', 'last_change': '2024-04-25T14:10:04.735818+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.735818+0000', 'last_undegraded': '2024-04-25T14:10:04.735818+0000', 'last_fullsized': '2024-04-25T14:10:04.735818+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:10:10.662 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.25 is not active+clean 2024-04-25T14:10:10.662 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.729202+0000', 'last_change': '2024-04-25T14:10:04.729202+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.729202+0000', 'last_undegraded': '2024-04-25T14:10:04.729202+0000', 'last_fullsized': '2024-04-25T14:10:04.729202+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:10:10.665 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.24 is not active+clean 2024-04-25T14:10:10.665 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.738433+0000', 'last_change': '2024-04-25T14:10:04.738433+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.738433+0000', 'last_undegraded': '2024-04-25T14:10:04.738433+0000', 'last_fullsized': '2024-04-25T14:10:04.738433+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:10:10.665 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.23 is not active+clean 2024-04-25T14:10:10.665 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.737523+0000', 'last_change': '2024-04-25T14:10:04.737523+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.737523+0000', 'last_undegraded': '2024-04-25T14:10:04.737523+0000', 'last_fullsized': '2024-04-25T14:10:04.737523+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:10:10.666 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.22 is not active+clean 2024-04-25T14:10:10.666 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.732817+0000', 'last_change': '2024-04-25T14:10:04.732817+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.732817+0000', 'last_undegraded': '2024-04-25T14:10:04.732817+0000', 'last_fullsized': '2024-04-25T14:10:04.732817+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:10:10.666 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.21 is not active+clean 2024-04-25T14:10:10.666 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.726801+0000', 'last_change': '2024-04-25T14:10:04.726801+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.726801+0000', 'last_undegraded': '2024-04-25T14:10:04.726801+0000', 'last_fullsized': '2024-04-25T14:10:04.726801+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:10:10.666 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.20 is not active+clean 2024-04-25T14:10:10.666 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.726485+0000', 'last_change': '2024-04-25T14:10:04.726485+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.726485+0000', 'last_undegraded': '2024-04-25T14:10:04.726485+0000', 'last_fullsized': '2024-04-25T14:10:04.726485+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:10:10.666 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1c is not active+clean 2024-04-25T14:10:10.666 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.726867+0000', 'last_change': '2024-04-25T14:10:04.726867+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.726867+0000', 'last_undegraded': '2024-04-25T14:10:04.726867+0000', 'last_fullsized': '2024-04-25T14:10:04.726867+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:10:10.666 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1d is not active+clean 2024-04-25T14:10:10.666 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.735537+0000', 'last_change': '2024-04-25T14:10:04.735537+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.735537+0000', 'last_undegraded': '2024-04-25T14:10:04.735537+0000', 'last_fullsized': '2024-04-25T14:10:04.735537+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:10:10.667 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1e is not active+clean 2024-04-25T14:10:10.667 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.728128+0000', 'last_change': '2024-04-25T14:10:04.728128+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.728128+0000', 'last_undegraded': '2024-04-25T14:10:04.728128+0000', 'last_fullsized': '2024-04-25T14:10:04.728128+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:10:10.667 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1f is not active+clean 2024-04-25T14:10:10.667 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.738654+0000', 'last_change': '2024-04-25T14:10:04.738654+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.738654+0000', 'last_undegraded': '2024-04-25T14:10:04.738654+0000', 'last_fullsized': '2024-04-25T14:10:04.738654+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:10:10.667 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.18 is not active+clean 2024-04-25T14:10:10.667 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.725539+0000', 'last_change': '2024-04-25T14:10:04.725539+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.725539+0000', 'last_undegraded': '2024-04-25T14:10:04.725539+0000', 'last_fullsized': '2024-04-25T14:10:04.725539+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:10:10.667 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.19 is not active+clean 2024-04-25T14:10:10.667 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.733780+0000', 'last_change': '2024-04-25T14:10:04.733780+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.733780+0000', 'last_undegraded': '2024-04-25T14:10:04.733780+0000', 'last_fullsized': '2024-04-25T14:10:04.733780+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:10:10.667 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1a is not active+clean 2024-04-25T14:10:10.667 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.732355+0000', 'last_change': '2024-04-25T14:10:04.732355+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.732355+0000', 'last_undegraded': '2024-04-25T14:10:04.732355+0000', 'last_fullsized': '2024-04-25T14:10:04.732355+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:10:10.667 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1b is not active+clean 2024-04-25T14:10:10.668 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.726718+0000', 'last_change': '2024-04-25T14:10:04.726718+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.726718+0000', 'last_undegraded': '2024-04-25T14:10:04.726718+0000', 'last_fullsized': '2024-04-25T14:10:04.726718+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:10:10.668 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.14 is not active+clean 2024-04-25T14:10:10.668 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.729428+0000', 'last_change': '2024-04-25T14:10:04.729428+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.729428+0000', 'last_undegraded': '2024-04-25T14:10:04.729428+0000', 'last_fullsized': '2024-04-25T14:10:04.729428+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:10:10.668 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.15 is not active+clean 2024-04-25T14:10:10.668 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.726923+0000', 'last_change': '2024-04-25T14:10:04.726923+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.726923+0000', 'last_undegraded': '2024-04-25T14:10:04.726923+0000', 'last_fullsized': '2024-04-25T14:10:04.726923+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:10:10.668 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.16 is not active+clean 2024-04-25T14:10:10.668 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.732111+0000', 'last_change': '2024-04-25T14:10:04.732111+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.732111+0000', 'last_undegraded': '2024-04-25T14:10:04.732111+0000', 'last_fullsized': '2024-04-25T14:10:04.732111+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:10:10.668 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.17 is not active+clean 2024-04-25T14:10:10.668 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.727533+0000', 'last_change': '2024-04-25T14:10:04.727533+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.727533+0000', 'last_undegraded': '2024-04-25T14:10:04.727533+0000', 'last_fullsized': '2024-04-25T14:10:04.727533+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:10:10.668 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.10 is not active+clean 2024-04-25T14:10:10.668 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.730347+0000', 'last_change': '2024-04-25T14:10:04.730347+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.730347+0000', 'last_undegraded': '2024-04-25T14:10:04.730347+0000', 'last_fullsized': '2024-04-25T14:10:04.730347+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:10:10.668 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.11 is not active+clean 2024-04-25T14:10:10.669 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.725553+0000', 'last_change': '2024-04-25T14:10:04.725553+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.725553+0000', 'last_undegraded': '2024-04-25T14:10:04.725553+0000', 'last_fullsized': '2024-04-25T14:10:04.725553+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:10:10.669 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.12 is not active+clean 2024-04-25T14:10:10.669 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.736954+0000', 'last_change': '2024-04-25T14:10:04.736954+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.736954+0000', 'last_undegraded': '2024-04-25T14:10:04.736954+0000', 'last_fullsized': '2024-04-25T14:10:04.736954+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:10:10.669 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.c is not active+clean 2024-04-25T14:10:10.669 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.728072+0000', 'last_change': '2024-04-25T14:10:04.728072+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.728072+0000', 'last_undegraded': '2024-04-25T14:10:04.728072+0000', 'last_fullsized': '2024-04-25T14:10:04.728072+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:10:10.669 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.d is not active+clean 2024-04-25T14:10:10.669 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.735165+0000', 'last_change': '2024-04-25T14:10:04.735165+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.735165+0000', 'last_undegraded': '2024-04-25T14:10:04.735165+0000', 'last_fullsized': '2024-04-25T14:10:04.735165+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:10:10.669 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.e is not active+clean 2024-04-25T14:10:10.669 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.738871+0000', 'last_change': '2024-04-25T14:10:04.738871+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.738871+0000', 'last_undegraded': '2024-04-25T14:10:04.738871+0000', 'last_fullsized': '2024-04-25T14:10:04.738871+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:10:10.669 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.f is not active+clean 2024-04-25T14:10:10.669 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.726953+0000', 'last_change': '2024-04-25T14:10:04.726953+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.726953+0000', 'last_undegraded': '2024-04-25T14:10:04.726953+0000', 'last_fullsized': '2024-04-25T14:10:04.726953+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:10:10.670 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.8 is not active+clean 2024-04-25T14:10:10.670 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.734586+0000', 'last_change': '2024-04-25T14:10:04.734586+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.734586+0000', 'last_undegraded': '2024-04-25T14:10:04.734586+0000', 'last_fullsized': '2024-04-25T14:10:04.734586+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:10:10.670 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.9 is not active+clean 2024-04-25T14:10:10.670 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.731633+0000', 'last_change': '2024-04-25T14:10:04.731633+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.731633+0000', 'last_undegraded': '2024-04-25T14:10:04.731633+0000', 'last_fullsized': '2024-04-25T14:10:04.731633+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:10:10.670 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.a is not active+clean 2024-04-25T14:10:10.670 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.731871+0000', 'last_change': '2024-04-25T14:10:04.731871+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.731871+0000', 'last_undegraded': '2024-04-25T14:10:04.731871+0000', 'last_fullsized': '2024-04-25T14:10:04.731871+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:10:10.670 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.b is not active+clean 2024-04-25T14:10:10.670 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.726535+0000', 'last_change': '2024-04-25T14:10:04.726535+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.726535+0000', 'last_undegraded': '2024-04-25T14:10:04.726535+0000', 'last_fullsized': '2024-04-25T14:10:04.726535+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:10:10.670 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.4 is not active+clean 2024-04-25T14:10:10.670 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.736386+0000', 'last_change': '2024-04-25T14:10:04.736386+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.736386+0000', 'last_undegraded': '2024-04-25T14:10:04.736386+0000', 'last_fullsized': '2024-04-25T14:10:04.736386+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:10:10.671 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.5 is not active+clean 2024-04-25T14:10:10.671 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.726322+0000', 'last_change': '2024-04-25T14:10:04.726322+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.726322+0000', 'last_undegraded': '2024-04-25T14:10:04.726322+0000', 'last_fullsized': '2024-04-25T14:10:04.726322+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:10:10.671 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.6 is not active+clean 2024-04-25T14:10:10.671 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.733546+0000', 'last_change': '2024-04-25T14:10:04.733546+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.733546+0000', 'last_undegraded': '2024-04-25T14:10:04.733546+0000', 'last_fullsized': '2024-04-25T14:10:04.733546+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:10:10.671 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.7 is not active+clean 2024-04-25T14:10:10.671 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.736394+0000', 'last_change': '2024-04-25T14:10:04.736394+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.736394+0000', 'last_undegraded': '2024-04-25T14:10:04.736394+0000', 'last_fullsized': '2024-04-25T14:10:04.736394+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:10:10.671 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.0 is not active+clean 2024-04-25T14:10:10.671 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.733923+0000', 'last_change': '2024-04-25T14:10:04.733923+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.733923+0000', 'last_undegraded': '2024-04-25T14:10:04.733923+0000', 'last_fullsized': '2024-04-25T14:10:04.733923+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:10:10.671 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1 is not active+clean 2024-04-25T14:10:10.671 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.730645+0000', 'last_change': '2024-04-25T14:10:04.730645+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.730645+0000', 'last_undegraded': '2024-04-25T14:10:04.730645+0000', 'last_fullsized': '2024-04-25T14:10:04.730645+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:10:10.672 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2 is not active+clean 2024-04-25T14:10:10.672 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.733889+0000', 'last_change': '2024-04-25T14:10:04.733889+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.733889+0000', 'last_undegraded': '2024-04-25T14:10:04.733889+0000', 'last_fullsized': '2024-04-25T14:10:04.733889+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:10:10.672 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3 is not active+clean 2024-04-25T14:10:10.672 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.726229+0000', 'last_change': '2024-04-25T14:10:04.726229+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.726229+0000', 'last_undegraded': '2024-04-25T14:10:04.726229+0000', 'last_fullsized': '2024-04-25T14:10:04.726229+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:10:10.672 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.13 is not active+clean 2024-04-25T14:10:10.672 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:04.734790+0000', 'last_change': '2024-04-25T14:10:04.734790+0000', 'last_active': '2024-04-25T14:10:04.713569+0000', 'last_peered': '2024-04-25T14:10:04.713569+0000', 'last_clean': '2024-04-25T14:10:04.713569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:04.734790+0000', 'last_undegraded': '2024-04-25T14:10:04.734790+0000', 'last_fullsized': '2024-04-25T14:10:04.734790+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:04.713569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:10:10.672 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T14:10:10.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.845+0000 7f59dda00640 1 Processor -- start 2024-04-25T14:10:10.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.845+0000 7f59dda00640 1 -- start start 2024-04-25T14:10:10.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.845+0000 7f59dda00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f59d80af6f0 0x7f59d80b6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:10.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.845+0000 7f59dda00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59d815e210 0x7f59d815e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:10.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.845+0000 7f59dda00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59d81584b0 con 0x7f59d806c670 2024-04-25T14:10:10.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.845+0000 7f59dda00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f59d80b5850 con 0x7f59d80af6f0 2024-04-25T14:10:10.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.845+0000 7f59dda00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f59d80b5290 con 0x7f59d815e210 2024-04-25T14:10:10.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.845+0000 7f59d6a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f59d80af6f0 0x7f59d80b6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:10.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.845+0000 7f59d6a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f59d80af6f0 0x7f59d80b6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53944/0 (socket says 172.21.15.86:53944) 2024-04-25T14:10:10.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.845+0000 7f59d6a00640 1 -- 172.21.15.86:0/3842962660 learned_addr learned my addr 172.21.15.86:0/3842962660 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:10.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.845+0000 7f59d7400640 1 --2- 172.21.15.86:0/3842962660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59d815e210 0x7f59d815e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:10.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.845+0000 7f59d6a00640 1 -- 172.21.15.86:0/3842962660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59d815e210 msgr2=0x7f59d815e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:10.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.845+0000 7f59d6a00640 1 --2- 172.21.15.86:0/3842962660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59d815e210 0x7f59d815e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:10.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.845+0000 7f59d6a00640 1 -- 172.21.15.86:0/3842962660 >> v1:172.21.15.155:6789/0 conn(0x7f59d806c670 legacy=0x7f59d806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:10.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.845+0000 7f59d6a00640 1 -- 172.21.15.86:0/3842962660 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d80b5570 con 0x7f59d80af6f0 2024-04-25T14:10:10.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.849+0000 7f59d6a00640 1 --2- 172.21.15.86:0/3842962660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d80af6f0 0x7f59d80b6b40 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009870 tx=0x7f59c801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa250d7e24b9c10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:10.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.849+0000 7f59d6000640 1 -- 172.21.15.86:0/3842962660 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c80ab5f0 con 0x7f59d80af6f0 2024-04-25T14:10:10.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.849+0000 7f59d6000640 1 -- 172.21.15.86:0/3842962660 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c80b0070 con 0x7f59d80af6f0 2024-04-25T14:10:10.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.849+0000 7f59d6000640 1 -- 172.21.15.86:0/3842962660 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c80ab9b0 con 0x7f59d80af6f0 2024-04-25T14:10:10.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.849+0000 7f59dda00640 1 -- 172.21.15.86:0/3842962660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d80af6f0 msgr2=0x7f59d80b6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:10.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.849+0000 7f59dda00640 1 --2- 172.21.15.86:0/3842962660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d80af6f0 0x7f59d80b6b40 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009870 tx=0x7f59c801c410 comp rx=0 tx=0).stop 2024-04-25T14:10:10.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.849+0000 7f59dda00640 1 -- 172.21.15.86:0/3842962660 shutdown_connections 2024-04-25T14:10:10.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.849+0000 7f59dda00640 1 --2- 172.21.15.86:0/3842962660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59d815e210 0x7f59d815e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:10.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.849+0000 7f59dda00640 1 --2- 172.21.15.86:0/3842962660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d80af6f0 0x7f59d80b6b40 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:10.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.849+0000 7f59dda00640 1 -- 172.21.15.86:0/3842962660 >> 172.21.15.86:0/3842962660 conn(0x7f59d80d1d70 msgr2=0x7f59d80d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:10.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.849+0000 7f59dda00640 1 -- 172.21.15.86:0/3842962660 shutdown_connections 2024-04-25T14:10:10.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.849+0000 7f59dda00640 1 -- 172.21.15.86:0/3842962660 wait complete. 2024-04-25T14:10:10.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.853+0000 7f59dda00640 1 Processor -- start 2024-04-25T14:10:10.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.853+0000 7f59dda00640 1 -- start start 2024-04-25T14:10:10.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.853+0000 7f59dda00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d806c670 0x7f59d81540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:10.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.853+0000 7f59dda00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59d80af6f0 0x7f59d80d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:10.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.853+0000 7f59dda00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59d815e210 0x7f59d81517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:10.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.853+0000 7f59dda00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f59d80afc80 con 0x7f59d806c670 2024-04-25T14:10:10.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.853+0000 7f59dda00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f59d80afb00 con 0x7f59d815e210 2024-04-25T14:10:10.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.853+0000 7f59dda00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f59d816ae20 con 0x7f59d80af6f0 2024-04-25T14:10:10.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.853+0000 7f59d6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d806c670 0x7f59d81540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:10.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.853+0000 7f59d6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d806c670 0x7f59d81540f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53946/0 (socket says 172.21.15.86:53946) 2024-04-25T14:10:10.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.853+0000 7f59d6a00640 1 -- 172.21.15.86:0/4016568892 learned_addr learned my addr 172.21.15.86:0/4016568892 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:10.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.853+0000 7f59d7400640 1 --2- 172.21.15.86:0/4016568892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59d815e210 0x7f59d81517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:10.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.853+0000 7f59ce000640 1 --2- 172.21.15.86:0/4016568892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59d80af6f0 0x7f59d80d0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:10.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.853+0000 7f59d6a00640 1 -- 172.21.15.86:0/4016568892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59d80af6f0 msgr2=0x7f59d80d0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:10.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.853+0000 7f59d6a00640 1 --2- 172.21.15.86:0/4016568892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59d80af6f0 0x7f59d80d0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:10.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.853+0000 7f59d6a00640 1 -- 172.21.15.86:0/4016568892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59d815e210 msgr2=0x7f59d81517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:10.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.853+0000 7f59d6a00640 1 --2- 172.21.15.86:0/4016568892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59d815e210 0x7f59d81517d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:10.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.857+0000 7f59d6a00640 1 -- 172.21.15.86:0/4016568892 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d8150730 con 0x7f59d806c670 2024-04-25T14:10:10.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.857+0000 7f59ce000640 1 --2- 172.21.15.86:0/4016568892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59d80af6f0 0x7f59d80d0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:10.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.857+0000 7f59d6a00640 1 --2- 172.21.15.86:0/4016568892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d806c670 0x7f59d81540f0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f59c80ad040 tx=0x7f59c801ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:10.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.857+0000 7f59cf400640 1 -- 172.21.15.86:0/4016568892 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c80be3f0 con 0x7f59d806c670 2024-04-25T14:10:10.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.857+0000 7f59cf400640 1 -- 172.21.15.86:0/4016568892 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c80be590 con 0x7f59d806c670 2024-04-25T14:10:10.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.857+0000 7f59dda00640 1 -- 172.21.15.86:0/4016568892 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59d8150a10 con 0x7f59d806c670 2024-04-25T14:10:10.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.857+0000 7f59cf400640 1 -- 172.21.15.86:0/4016568892 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c80be980 con 0x7f59d806c670 2024-04-25T14:10:10.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.857+0000 7f59dda00640 1 -- 172.21.15.86:0/4016568892 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59d8150fd0 con 0x7f59d806c670 2024-04-25T14:10:10.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.857+0000 7f59dda00640 1 -- 172.21.15.86:0/4016568892 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59d80a00f0 con 0x7f59d806c670 2024-04-25T14:10:10.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.857+0000 7f59cf400640 1 -- 172.21.15.86:0/4016568892 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7f59c80beb20 con 0x7f59d806c670 2024-04-25T14:10:10.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.861+0000 7f59cf400640 1 --2- 172.21.15.86:0/4016568892 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f59b406f520 0x7f59b4071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:10.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.861+0000 7f59cf400640 1 -- 172.21.15.86:0/4016568892 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f59c8129c30 con 0x7f59d806c670 2024-04-25T14:10:10.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.861+0000 7f59ce000640 1 --2- 172.21.15.86:0/4016568892 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f59b406f520 0x7f59b4071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:10.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.861+0000 7f59ce000640 1 --2- 172.21.15.86:0/4016568892 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f59b406f520 0x7f59b4071a30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f59d80b35e0 tx=0x7f59c0009040 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:10.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:10.865+0000 7f59cf400640 1 -- 172.21.15.86:0/4016568892 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f59c8016150 con 0x7f59d806c670 2024-04-25T14:10:11.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.077+0000 7f59dda00640 1 -- 172.21.15.86:0/4016568892 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f59d8073980 con 0x7f59d806c670 2024-04-25T14:10:11.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.077+0000 7f59cf400640 1 -- 172.21.15.86:0/4016568892 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1035 (secure 0 0 0) 0x7f59c80f62d0 con 0x7f59d806c670 2024-04-25T14:10:11.075 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:11.075 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":405,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1714053818,"num_in_osds":8,"osd_in_since":1714053808,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":267677696,"bytes_avail":772826435584,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":85,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"b","status":"up:active","gid":8457}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-04-25T14:08:15.095679+0000","services":{}},"progress_events":{}} 2024-04-25T14:10:11.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.077+0000 7f59dda00640 1 -- 172.21.15.86:0/4016568892 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f59b406f520 msgr2=0x7f59b4071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:11.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.077+0000 7f59dda00640 1 --2- 172.21.15.86:0/4016568892 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f59b406f520 0x7f59b4071a30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f59d80b35e0 tx=0x7f59c0009040 comp rx=0 tx=0).stop 2024-04-25T14:10:11.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.077+0000 7f59dda00640 1 -- 172.21.15.86:0/4016568892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d806c670 msgr2=0x7f59d81540f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:11.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.077+0000 7f59dda00640 1 --2- 172.21.15.86:0/4016568892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d806c670 0x7f59d81540f0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f59c80ad040 tx=0x7f59c801ac50 comp rx=0 tx=0).stop 2024-04-25T14:10:11.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.077+0000 7f59dda00640 1 -- 172.21.15.86:0/4016568892 shutdown_connections 2024-04-25T14:10:11.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.077+0000 7f59dda00640 1 --2- 172.21.15.86:0/4016568892 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f59b406f520 0x7f59b4071a30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:11.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.077+0000 7f59dda00640 1 --2- 172.21.15.86:0/4016568892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59d815e210 0x7f59d81517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:11.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.077+0000 7f59dda00640 1 --2- 172.21.15.86:0/4016568892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59d80af6f0 0x7f59d80d0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:11.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.077+0000 7f59dda00640 1 --2- 172.21.15.86:0/4016568892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d806c670 0x7f59d81540f0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:11.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.077+0000 7f59dda00640 1 -- 172.21.15.86:0/4016568892 >> 172.21.15.86:0/4016568892 conn(0x7f59d80d1d70 msgr2=0x7f59d8152640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:11.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.077+0000 7f59dda00640 1 -- 172.21.15.86:0/4016568892 shutdown_connections 2024-04-25T14:10:11.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.077+0000 7f59dda00640 1 -- 172.21.15.86:0/4016568892 wait complete. 2024-04-25T14:10:11.097 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T14:10:11.097 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:10:11.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.245+0000 7f15e0600640 1 Processor -- start 2024-04-25T14:10:11.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.245+0000 7f15e0600640 1 -- start start 2024-04-25T14:10:11.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.249+0000 7f15e0600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f15d815c2a0 0x7f15d80ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.249+0000 7f15e0600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15d806fd50 0x7f15d8070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.249+0000 7f15e0600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f15d806c670 0x7f15d806ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.249+0000 7f15e0600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f15d80b0a60 con 0x7f15d806c670 2024-04-25T14:10:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.249+0000 7f15e0600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f15d806ef90 con 0x7f15d815c2a0 2024-04-25T14:10:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.249+0000 7f15e0600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f15d806f270 con 0x7f15d806fd50 2024-04-25T14:10:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.249+0000 7f15de600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f15d806c670 0x7f15d806ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.249+0000 7f15de600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f15d806c670 0x7f15d806ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53962/0 (socket says 172.21.15.86:53962) 2024-04-25T14:10:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.249+0000 7f15de600640 1 -- 172.21.15.86:0/3651133296 learned_addr learned my addr 172.21.15.86:0/3651133296 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.249+0000 7f15dd200640 1 --2- 172.21.15.86:0/3651133296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15d806fd50 0x7f15d8070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.249+0000 7f15ddc00640 1 --2- 172.21.15.86:0/3651133296 >> v2:172.21.15.155:3300/0 conn(0x7f15d815c2a0 0x7f15d80ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.249+0000 7f15de600640 1 -- 172.21.15.86:0/3651133296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15d806fd50 msgr2=0x7f15d8070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.249+0000 7f15de600640 1 --2- 172.21.15.86:0/3651133296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15d806fd50 0x7f15d8070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.249+0000 7f15de600640 1 -- 172.21.15.86:0/3651133296 >> v2:172.21.15.155:3300/0 conn(0x7f15d815c2a0 msgr2=0x7f15d80ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.249+0000 7f15de600640 1 --2- 172.21.15.86:0/3651133296 >> v2:172.21.15.155:3300/0 conn(0x7f15d815c2a0 0x7f15d80ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.249+0000 7f15de600640 1 -- 172.21.15.86:0/3651133296 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d806f550 con 0x7f15d806c670 2024-04-25T14:10:11.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.249+0000 7f15ddc00640 1 --2- 172.21.15.86:0/3651133296 >> v2:172.21.15.155:3300/0 conn(0x7f15d815c2a0 0x7f15d80ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:11.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.253+0000 7f15de600640 1 --2- 172.21.15.86:0/3651133296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15d806c670 0x7f15d806ca80 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f15d8072200 tx=0x7f15d001eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72b98b5c1c31648 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:11.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.253+0000 7f15d7e00640 1 -- 172.21.15.86:0/3651133296 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d000b2d0 con 0x7f15d806c670 2024-04-25T14:10:11.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.253+0000 7f15d7e00640 1 -- 172.21.15.86:0/3651133296 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15d000b470 con 0x7f15d806c670 2024-04-25T14:10:11.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.253+0000 7f15d7e00640 1 -- 172.21.15.86:0/3651133296 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d00a7b20 con 0x7f15d806c670 2024-04-25T14:10:11.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.253+0000 7f15e0600640 1 -- 172.21.15.86:0/3651133296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15d806c670 msgr2=0x7f15d806ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:11.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.253+0000 7f15e0600640 1 --2- 172.21.15.86:0/3651133296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15d806c670 0x7f15d806ca80 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f15d8072200 tx=0x7f15d001eb30 comp rx=0 tx=0).stop 2024-04-25T14:10:11.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.253+0000 7f15e0600640 1 -- 172.21.15.86:0/3651133296 shutdown_connections 2024-04-25T14:10:11.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.253+0000 7f15e0600640 1 --2- 172.21.15.86:0/3651133296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15d806c670 0x7f15d806ca80 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:11.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.253+0000 7f15e0600640 1 --2- 172.21.15.86:0/3651133296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15d806fd50 0x7f15d8070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:11.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.253+0000 7f15e0600640 1 --2- 172.21.15.86:0/3651133296 >> v2:172.21.15.155:3300/0 conn(0x7f15d815c2a0 0x7f15d80ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:11.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.253+0000 7f15e0600640 1 -- 172.21.15.86:0/3651133296 >> 172.21.15.86:0/3651133296 conn(0x7f15d80b4650 msgr2=0x7f15d80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:11.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.253+0000 7f15e0600640 1 -- 172.21.15.86:0/3651133296 shutdown_connections 2024-04-25T14:10:11.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.253+0000 7f15e0600640 1 -- 172.21.15.86:0/3651133296 wait complete. 2024-04-25T14:10:11.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.253+0000 7f15e0600640 1 Processor -- start 2024-04-25T14:10:11.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.253+0000 7f15e0600640 1 -- start start 2024-04-25T14:10:11.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.257+0000 7f15e0600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15d806c670 0x7f15d809a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:11.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.257+0000 7f15e0600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15d806fd50 0x7f15d815afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:11.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.257+0000 7f15e0600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15d815c2a0 0x7f15d815b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:11.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.257+0000 7f15e0600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f15d815c6b0 con 0x7f15d806c670 2024-04-25T14:10:11.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.257+0000 7f15e0600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f15d806d3d0 con 0x7f15d806fd50 2024-04-25T14:10:11.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.257+0000 7f15e0600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f15d815c830 con 0x7f15d815c2a0 2024-04-25T14:10:11.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.257+0000 7f15ddc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15d806c670 0x7f15d809a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:11.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.257+0000 7f15ddc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15d806c670 0x7f15d809a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53978/0 (socket says 172.21.15.86:53978) 2024-04-25T14:10:11.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.257+0000 7f15ddc00640 1 -- 172.21.15.86:0/2506143037 learned_addr learned my addr 172.21.15.86:0/2506143037 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:11.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.257+0000 7f15dd200640 1 --2- 172.21.15.86:0/2506143037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15d806fd50 0x7f15d815afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:11.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.257+0000 7f15de600640 1 --2- 172.21.15.86:0/2506143037 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15d815c2a0 0x7f15d815b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:11.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.257+0000 7f15ddc00640 1 -- 172.21.15.86:0/2506143037 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15d815c2a0 msgr2=0x7f15d815b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:11.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.257+0000 7f15ddc00640 1 --2- 172.21.15.86:0/2506143037 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15d815c2a0 0x7f15d815b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:11.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.257+0000 7f15ddc00640 1 -- 172.21.15.86:0/2506143037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15d806fd50 msgr2=0x7f15d815afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:11.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.257+0000 7f15ddc00640 1 --2- 172.21.15.86:0/2506143037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15d806fd50 0x7f15d815afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:11.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.257+0000 7f15ddc00640 1 -- 172.21.15.86:0/2506143037 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d80ce5e0 con 0x7f15d806c670 2024-04-25T14:10:11.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.257+0000 7f15dd200640 1 --2- 172.21.15.86:0/2506143037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15d806fd50 0x7f15d815afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:11.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.257+0000 7f15de600640 1 --2- 172.21.15.86:0/2506143037 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15d815c2a0 0x7f15d815b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:11.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.257+0000 7f15ddc00640 1 --2- 172.21.15.86:0/2506143037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15d806c670 0x7f15d809a8e0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00c7f0 tx=0x7f15cc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:11.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.257+0000 7f15d5600640 1 -- 172.21.15.86:0/2506143037 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc015070 con 0x7f15d806c670 2024-04-25T14:10:11.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.257+0000 7f15d5600640 1 -- 172.21.15.86:0/2506143037 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15cc007820 con 0x7f15d806c670 2024-04-25T14:10:11.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.257+0000 7f15d5600640 1 -- 172.21.15.86:0/2506143037 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc007c10 con 0x7f15d806c670 2024-04-25T14:10:11.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.257+0000 7f15e0600640 1 -- 172.21.15.86:0/2506143037 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d8165190 con 0x7f15d806c670 2024-04-25T14:10:11.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.257+0000 7f15e0600640 1 -- 172.21.15.86:0/2506143037 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d8165750 con 0x7f15d806c670 2024-04-25T14:10:11.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.257+0000 7f15e0600640 1 -- 172.21.15.86:0/2506143037 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d8069d00 con 0x7f15d806c670 2024-04-25T14:10:11.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.261+0000 7f15d5600640 1 -- 172.21.15.86:0/2506143037 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7f15cc018070 con 0x7f15d806c670 2024-04-25T14:10:11.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.261+0000 7f15d5600640 1 --2- 172.21.15.86:0/2506143037 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f15b806f520 0x7f15b8071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:11.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.261+0000 7f15d5600640 1 -- 172.21.15.86:0/2506143037 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f15cc08f6c0 con 0x7f15d806c670 2024-04-25T14:10:11.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.261+0000 7f15dd200640 1 --2- 172.21.15.86:0/2506143037 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f15b806f520 0x7f15b8071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:11.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.261+0000 7f15dd200640 1 --2- 172.21.15.86:0/2506143037 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f15b806f520 0x7f15b8071a30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f15d815bb10 tx=0x7f15c4008040 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:11.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.265+0000 7f15d5600640 1 -- 172.21.15.86:0/2506143037 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f15cc05c290 con 0x7f15d806c670 2024-04-25T14:10:11.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.417+0000 7f15e0600640 1 -- 172.21.15.86:0/2506143037 --> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f15d80ce390 con 0x7f15b806f520 2024-04-25T14:10:11.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.425+0000 7f15d5600640 1 -- 172.21.15.86:0/2506143037 <== mgr.8190 v2:172.21.15.86:6832/1354467416 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247091 (secure 0 0 0) 0x7f15d80ce390 con 0x7f15b806f520 2024-04-25T14:10:11.426 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:11.427 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:10:11.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.429+0000 7f15e0600640 1 -- 172.21.15.86:0/2506143037 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f15b806f520 msgr2=0x7f15b8071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:11.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.429+0000 7f15e0600640 1 --2- 172.21.15.86:0/2506143037 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f15b806f520 0x7f15b8071a30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f15d815bb10 tx=0x7f15c4008040 comp rx=0 tx=0).stop 2024-04-25T14:10:11.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.429+0000 7f15e0600640 1 -- 172.21.15.86:0/2506143037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15d806c670 msgr2=0x7f15d809a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:11.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.429+0000 7f15e0600640 1 --2- 172.21.15.86:0/2506143037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15d806c670 0x7f15d809a8e0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00c7f0 tx=0x7f15cc00ccb0 comp rx=0 tx=0).stop 2024-04-25T14:10:11.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.429+0000 7f15e0600640 1 -- 172.21.15.86:0/2506143037 shutdown_connections 2024-04-25T14:10:11.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.429+0000 7f15e0600640 1 --2- 172.21.15.86:0/2506143037 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f15b806f520 0x7f15b8071a30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:11.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.429+0000 7f15e0600640 1 --2- 172.21.15.86:0/2506143037 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15d815c2a0 0x7f15d815b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:11.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.429+0000 7f15e0600640 1 --2- 172.21.15.86:0/2506143037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15d806fd50 0x7f15d815afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:11.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.429+0000 7f15e0600640 1 --2- 172.21.15.86:0/2506143037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15d806c670 0x7f15d809a8e0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:11.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.429+0000 7f15e0600640 1 -- 172.21.15.86:0/2506143037 >> 172.21.15.86:0/2506143037 conn(0x7f15d80b4650 msgr2=0x7f15d8072880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:11.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.429+0000 7f15e0600640 1 -- 172.21.15.86:0/2506143037 shutdown_connections 2024-04-25T14:10:11.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:11.429+0000 7f15e0600640 1 -- 172.21.15.86:0/2506143037 wait complete. 2024-04-25T14:10:11.447 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T14:10:09.474871+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":468,"num_read_kb":1272,"num_write":181,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261404,"kb_used_data":11928,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713316,"statfs":{"total":773094113280,"available":772826435584,"internally_reserved":0,"allocated":12214272,"data_stored":6671076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.761213"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735771+0000","last_change":"2024-04-25T14:10:04.735771+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735771+0000","last_undegraded":"2024-04-25T14:10:04.735771+0000","last_fullsized":"2024-04-25T14:10:04.735771+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735778+0000","last_change":"2024-04-25T14:10:04.735778+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735778+0000","last_undegraded":"2024-04-25T14:10:04.735778+0000","last_fullsized":"2024-04-25T14:10:04.735778+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735849+0000","last_change":"2024-04-25T14:10:04.735849+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735849+0000","last_undegraded":"2024-04-25T14:10:04.735849+0000","last_fullsized":"2024-04-25T14:10:04.735849+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.734374+0000","last_change":"2024-04-25T14:10:04.734374+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.734374+0000","last_undegraded":"2024-04-25T14:10:04.734374+0000","last_fullsized":"2024-04-25T14:10:04.734374+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.734611+0000","last_change":"2024-04-25T14:10:04.734611+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.734611+0000","last_undegraded":"2024-04-25T14:10:04.734611+0000","last_fullsized":"2024-04-25T14:10:04.734611+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.738244+0000","last_change":"2024-04-25T14:10:04.738244+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.738244+0000","last_undegraded":"2024-04-25T14:10:04.738244+0000","last_fullsized":"2024-04-25T14:10:04.738244+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.734697+0000","last_change":"2024-04-25T14:10:04.734697+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.734697+0000","last_undegraded":"2024-04-25T14:10:04.734697+0000","last_fullsized":"2024-04-25T14:10:04.734697+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733337+0000","last_change":"2024-04-25T14:10:04.733337+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733337+0000","last_undegraded":"2024-04-25T14:10:04.733337+0000","last_fullsized":"2024-04-25T14:10:04.733337+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.739896+0000","last_change":"2024-04-25T14:10:04.739896+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.739896+0000","last_undegraded":"2024-04-25T14:10:04.739896+0000","last_fullsized":"2024-04-25T14:10:04.739896+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726838+0000","last_change":"2024-04-25T14:10:04.726838+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726838+0000","last_undegraded":"2024-04-25T14:10:04.726838+0000","last_fullsized":"2024-04-25T14:10:04.726838+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735651+0000","last_change":"2024-04-25T14:10:04.735651+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735651+0000","last_undegraded":"2024-04-25T14:10:04.735651+0000","last_fullsized":"2024-04-25T14:10:04.735651+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.742843+0000","last_change":"2024-04-25T14:10:04.742843+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.742843+0000","last_undegraded":"2024-04-25T14:10:04.742843+0000","last_fullsized":"2024-04-25T14:10:04.742843+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.727264+0000","last_change":"2024-04-25T14:10:04.727264+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.727264+0000","last_undegraded":"2024-04-25T14:10:04.727264+0000","last_fullsized":"2024-04-25T14:10:04.727264+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733351+0000","last_change":"2024-04-25T14:10:04.733351+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733351+0000","last_undegraded":"2024-04-25T14:10:04.733351+0000","last_fullsized":"2024-04-25T14:10:04.733351+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.734380+0000","last_change":"2024-04-25T14:10:04.734380+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.734380+0000","last_undegraded":"2024-04-25T14:10:04.734380+0000","last_fullsized":"2024-04-25T14:10:04.734380+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735156+0000","last_change":"2024-04-25T14:10:04.735156+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735156+0000","last_undegraded":"2024-04-25T14:10:04.735156+0000","last_fullsized":"2024-04-25T14:10:04.735156+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.725975+0000","last_change":"2024-04-25T14:10:04.725975+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.725975+0000","last_undegraded":"2024-04-25T14:10:04.725975+0000","last_fullsized":"2024-04-25T14:10:04.725975+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733263+0000","last_change":"2024-04-25T14:10:04.733263+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733263+0000","last_undegraded":"2024-04-25T14:10:04.733263+0000","last_fullsized":"2024-04-25T14:10:04.733263+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.729768+0000","last_change":"2024-04-25T14:10:04.729768+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.729768+0000","last_undegraded":"2024-04-25T14:10:04.729768+0000","last_fullsized":"2024-04-25T14:10:04.729768+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.737805+0000","last_change":"2024-04-25T14:10:04.737805+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.737805+0000","last_undegraded":"2024-04-25T14:10:04.737805+0000","last_fullsized":"2024-04-25T14:10:04.737805+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735963+0000","last_change":"2024-04-25T14:10:04.735963+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735963+0000","last_undegraded":"2024-04-25T14:10:04.735963+0000","last_fullsized":"2024-04-25T14:10:04.735963+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733684+0000","last_change":"2024-04-25T14:10:04.733684+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733684+0000","last_undegraded":"2024-04-25T14:10:04.733684+0000","last_fullsized":"2024-04-25T14:10:04.733684+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.732828+0000","last_change":"2024-04-25T14:10:04.732828+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.732828+0000","last_undegraded":"2024-04-25T14:10:04.732828+0000","last_fullsized":"2024-04-25T14:10:04.732828+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.727648+0000","last_change":"2024-04-25T14:10:04.727648+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.727648+0000","last_undegraded":"2024-04-25T14:10:04.727648+0000","last_fullsized":"2024-04-25T14:10:04.727648+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.728155+0000","last_change":"2024-04-25T14:10:04.728155+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.728155+0000","last_undegraded":"2024-04-25T14:10:04.728155+0000","last_fullsized":"2024-04-25T14:10:04.728155+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735818+0000","last_change":"2024-04-25T14:10:04.735818+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735818+0000","last_undegraded":"2024-04-25T14:10:04.735818+0000","last_fullsized":"2024-04-25T14:10:04.735818+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.729202+0000","last_change":"2024-04-25T14:10:04.729202+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.729202+0000","last_undegraded":"2024-04-25T14:10:04.729202+0000","last_fullsized":"2024-04-25T14:10:04.729202+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.738433+0000","last_change":"2024-04-25T14:10:04.738433+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.738433+0000","last_undegraded":"2024-04-25T14:10:04.738433+0000","last_fullsized":"2024-04-25T14:10:04.738433+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.737523+0000","last_change":"2024-04-25T14:10:04.737523+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.737523+0000","last_undegraded":"2024-04-25T14:10:04.737523+0000","last_fullsized":"2024-04-25T14:10:04.737523+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.732817+0000","last_change":"2024-04-25T14:10:04.732817+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.732817+0000","last_undegraded":"2024-04-25T14:10:04.732817+0000","last_fullsized":"2024-04-25T14:10:04.732817+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726801+0000","last_change":"2024-04-25T14:10:04.726801+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726801+0000","last_undegraded":"2024-04-25T14:10:04.726801+0000","last_fullsized":"2024-04-25T14:10:04.726801+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726485+0000","last_change":"2024-04-25T14:10:04.726485+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726485+0000","last_undegraded":"2024-04-25T14:10:04.726485+0000","last_fullsized":"2024-04-25T14:10:04.726485+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726867+0000","last_change":"2024-04-25T14:10:04.726867+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726867+0000","last_undegraded":"2024-04-25T14:10:04.726867+0000","last_fullsized":"2024-04-25T14:10:04.726867+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.749691+0000","last_change":"2024-04-25T14:10:04.745986+0000","last_active":"2024-04-25T14:10:04.749691+0000","last_peered":"2024-04-25T14:10:04.749691+0000","last_clean":"2024-04-25T14:10:04.749691+0000","last_became_active":"2024-04-25T14:10:04.745516+0000","last_became_peered":"2024-04-25T14:10:04.745516+0000","last_unstale":"2024-04-25T14:10:04.749691+0000","last_undegraded":"2024-04-25T14:10:04.749691+0000","last_fullsized":"2024-04-25T14:10:04.749691+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:45.118281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735537+0000","last_change":"2024-04-25T14:10:04.735537+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735537+0000","last_undegraded":"2024-04-25T14:10:04.735537+0000","last_fullsized":"2024-04-25T14:10:04.735537+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.746437+0000","last_change":"2024-04-25T14:10:04.744056+0000","last_active":"2024-04-25T14:10:04.746437+0000","last_peered":"2024-04-25T14:10:04.746437+0000","last_clean":"2024-04-25T14:10:04.746437+0000","last_became_active":"2024-04-25T14:10:04.743755+0000","last_became_peered":"2024-04-25T14:10:04.743755+0000","last_unstale":"2024-04-25T14:10:04.746437+0000","last_undegraded":"2024-04-25T14:10:04.746437+0000","last_fullsized":"2024-04-25T14:10:04.746437+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:45.118281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.728128+0000","last_change":"2024-04-25T14:10:04.728128+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.728128+0000","last_undegraded":"2024-04-25T14:10:04.728128+0000","last_fullsized":"2024-04-25T14:10:04.728128+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.755092+0000","last_change":"2024-04-25T14:10:04.752833+0000","last_active":"2024-04-25T14:10:04.755092+0000","last_peered":"2024-04-25T14:10:04.755092+0000","last_clean":"2024-04-25T14:10:04.755092+0000","last_became_active":"2024-04-25T14:10:04.752483+0000","last_became_peered":"2024-04-25T14:10:04.752483+0000","last_unstale":"2024-04-25T14:10:04.755092+0000","last_undegraded":"2024-04-25T14:10:04.755092+0000","last_fullsized":"2024-04-25T14:10:04.755092+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:50.427076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.738654+0000","last_change":"2024-04-25T14:10:04.738654+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.738654+0000","last_undegraded":"2024-04-25T14:10:04.738654+0000","last_fullsized":"2024-04-25T14:10:04.738654+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.737624+0000","last_change":"2024-04-25T14:10:04.733242+0000","last_active":"2024-04-25T14:10:04.737624+0000","last_peered":"2024-04-25T14:10:04.737624+0000","last_clean":"2024-04-25T14:10:04.737624+0000","last_became_active":"2024-04-25T14:10:04.732929+0000","last_became_peered":"2024-04-25T14:10:04.732929+0000","last_unstale":"2024-04-25T14:10:04.737624+0000","last_undegraded":"2024-04-25T14:10:04.737624+0000","last_fullsized":"2024-04-25T14:10:04.737624+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:25.644454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.725539+0000","last_change":"2024-04-25T14:10:04.725539+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.725539+0000","last_undegraded":"2024-04-25T14:10:04.725539+0000","last_fullsized":"2024-04-25T14:10:04.725539+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.744498+0000","last_change":"2024-04-25T14:10:04.738920+0000","last_active":"2024-04-25T14:10:04.744498+0000","last_peered":"2024-04-25T14:10:04.744498+0000","last_clean":"2024-04-25T14:10:04.744498+0000","last_became_active":"2024-04-25T14:10:04.738531+0000","last_became_peered":"2024-04-25T14:10:04.738531+0000","last_unstale":"2024-04-25T14:10:04.744498+0000","last_undegraded":"2024-04-25T14:10:04.744498+0000","last_fullsized":"2024-04-25T14:10:04.744498+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:45.118281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733780+0000","last_change":"2024-04-25T14:10:04.733780+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733780+0000","last_undegraded":"2024-04-25T14:10:04.733780+0000","last_fullsized":"2024-04-25T14:10:04.733780+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.754194+0000","last_change":"2024-04-25T14:10:04.749193+0000","last_active":"2024-04-25T14:10:04.754194+0000","last_peered":"2024-04-25T14:10:04.754194+0000","last_clean":"2024-04-25T14:10:04.754194+0000","last_became_active":"2024-04-25T14:10:04.747285+0000","last_became_peered":"2024-04-25T14:10:04.747285+0000","last_unstale":"2024-04-25T14:10:04.754194+0000","last_undegraded":"2024-04-25T14:10:04.754194+0000","last_fullsized":"2024-04-25T14:10:04.754194+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:19:34.506848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.732355+0000","last_change":"2024-04-25T14:10:04.732355+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.732355+0000","last_undegraded":"2024-04-25T14:10:04.732355+0000","last_fullsized":"2024-04-25T14:10:04.732355+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.740654+0000","last_change":"2024-04-25T14:10:04.734364+0000","last_active":"2024-04-25T14:10:04.740654+0000","last_peered":"2024-04-25T14:10:04.740654+0000","last_clean":"2024-04-25T14:10:04.740654+0000","last_became_active":"2024-04-25T14:10:04.734026+0000","last_became_peered":"2024-04-25T14:10:04.734026+0000","last_unstale":"2024-04-25T14:10:04.740654+0000","last_undegraded":"2024-04-25T14:10:04.740654+0000","last_fullsized":"2024-04-25T14:10:04.740654+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:48.842742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726718+0000","last_change":"2024-04-25T14:10:04.726718+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726718+0000","last_undegraded":"2024-04-25T14:10:04.726718+0000","last_fullsized":"2024-04-25T14:10:04.726718+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.753241+0000","last_change":"2024-04-25T14:10:04.745258+0000","last_active":"2024-04-25T14:10:04.753241+0000","last_peered":"2024-04-25T14:10:04.753241+0000","last_clean":"2024-04-25T14:10:04.753241+0000","last_became_active":"2024-04-25T14:10:04.744962+0000","last_became_peered":"2024-04-25T14:10:04.744962+0000","last_unstale":"2024-04-25T14:10:04.753241+0000","last_undegraded":"2024-04-25T14:10:04.753241+0000","last_fullsized":"2024-04-25T14:10:04.753241+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:58.400437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.729428+0000","last_change":"2024-04-25T14:10:04.729428+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.729428+0000","last_undegraded":"2024-04-25T14:10:04.729428+0000","last_fullsized":"2024-04-25T14:10:04.729428+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.754546+0000","last_change":"2024-04-25T14:10:04.750916+0000","last_active":"2024-04-25T14:10:04.754546+0000","last_peered":"2024-04-25T14:10:04.754546+0000","last_clean":"2024-04-25T14:10:04.754546+0000","last_became_active":"2024-04-25T14:10:04.750562+0000","last_became_peered":"2024-04-25T14:10:04.750562+0000","last_unstale":"2024-04-25T14:10:04.754546+0000","last_undegraded":"2024-04-25T14:10:04.754546+0000","last_fullsized":"2024-04-25T14:10:04.754546+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:50.427076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726923+0000","last_change":"2024-04-25T14:10:04.726923+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726923+0000","last_undegraded":"2024-04-25T14:10:04.726923+0000","last_fullsized":"2024-04-25T14:10:04.726923+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.749495+0000","last_change":"2024-04-25T14:10:04.746574+0000","last_active":"2024-04-25T14:10:04.749495+0000","last_peered":"2024-04-25T14:10:04.749495+0000","last_clean":"2024-04-25T14:10:04.749495+0000","last_became_active":"2024-04-25T14:10:04.746266+0000","last_became_peered":"2024-04-25T14:10:04.746266+0000","last_unstale":"2024-04-25T14:10:04.749495+0000","last_undegraded":"2024-04-25T14:10:04.749495+0000","last_fullsized":"2024-04-25T14:10:04.749495+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:28.147163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.732111+0000","last_change":"2024-04-25T14:10:04.732111+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.732111+0000","last_undegraded":"2024-04-25T14:10:04.732111+0000","last_fullsized":"2024-04-25T14:10:04.732111+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.748958+0000","last_change":"2024-04-25T14:10:04.745704+0000","last_active":"2024-04-25T14:10:04.748958+0000","last_peered":"2024-04-25T14:10:04.748958+0000","last_clean":"2024-04-25T14:10:04.748958+0000","last_became_active":"2024-04-25T14:10:04.745401+0000","last_became_peered":"2024-04-25T14:10:04.745401+0000","last_unstale":"2024-04-25T14:10:04.748958+0000","last_undegraded":"2024-04-25T14:10:04.748958+0000","last_fullsized":"2024-04-25T14:10:04.748958+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:24.704172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.727533+0000","last_change":"2024-04-25T14:10:04.727533+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.727533+0000","last_undegraded":"2024-04-25T14:10:04.727533+0000","last_fullsized":"2024-04-25T14:10:04.727533+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.749695+0000","last_change":"2024-04-25T14:10:04.745297+0000","last_active":"2024-04-25T14:10:04.749695+0000","last_peered":"2024-04-25T14:10:04.749695+0000","last_clean":"2024-04-25T14:10:04.749695+0000","last_became_active":"2024-04-25T14:10:04.744890+0000","last_became_peered":"2024-04-25T14:10:04.744890+0000","last_unstale":"2024-04-25T14:10:04.749695+0000","last_undegraded":"2024-04-25T14:10:04.749695+0000","last_fullsized":"2024-04-25T14:10:04.749695+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:39.794816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.730347+0000","last_change":"2024-04-25T14:10:04.730347+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.730347+0000","last_undegraded":"2024-04-25T14:10:04.730347+0000","last_fullsized":"2024-04-25T14:10:04.730347+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.750158+0000","last_change":"2024-04-25T14:10:04.746252+0000","last_active":"2024-04-25T14:10:04.750158+0000","last_peered":"2024-04-25T14:10:04.750158+0000","last_clean":"2024-04-25T14:10:04.750158+0000","last_became_active":"2024-04-25T14:10:04.745299+0000","last_became_peered":"2024-04-25T14:10:04.745299+0000","last_unstale":"2024-04-25T14:10:04.750158+0000","last_undegraded":"2024-04-25T14:10:04.750158+0000","last_fullsized":"2024-04-25T14:10:04.750158+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:05.444701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.725553+0000","last_change":"2024-04-25T14:10:04.725553+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.725553+0000","last_undegraded":"2024-04-25T14:10:04.725553+0000","last_fullsized":"2024-04-25T14:10:04.725553+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.745756+0000","last_change":"2024-04-25T14:10:04.740086+0000","last_active":"2024-04-25T14:10:04.745756+0000","last_peered":"2024-04-25T14:10:04.745756+0000","last_clean":"2024-04-25T14:10:04.745756+0000","last_became_active":"2024-04-25T14:10:04.739141+0000","last_became_peered":"2024-04-25T14:10:04.739141+0000","last_unstale":"2024-04-25T14:10:04.745756+0000","last_undegraded":"2024-04-25T14:10:04.745756+0000","last_fullsized":"2024-04-25T14:10:04.745756+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:16.658408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.736954+0000","last_change":"2024-04-25T14:10:04.736954+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.736954+0000","last_undegraded":"2024-04-25T14:10:04.736954+0000","last_fullsized":"2024-04-25T14:10:04.736954+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.746391+0000","last_change":"2024-04-25T14:10:04.743146+0000","last_active":"2024-04-25T14:10:04.746391+0000","last_peered":"2024-04-25T14:10:04.746391+0000","last_clean":"2024-04-25T14:10:04.746391+0000","last_became_active":"2024-04-25T14:10:04.742805+0000","last_became_peered":"2024-04-25T14:10:04.742805+0000","last_unstale":"2024-04-25T14:10:04.746391+0000","last_undegraded":"2024-04-25T14:10:04.746391+0000","last_fullsized":"2024-04-25T14:10:04.746391+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:52.358091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.728072+0000","last_change":"2024-04-25T14:10:04.728072+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.728072+0000","last_undegraded":"2024-04-25T14:10:04.728072+0000","last_fullsized":"2024-04-25T14:10:04.728072+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.753810+0000","last_change":"2024-04-25T14:10:04.751976+0000","last_active":"2024-04-25T14:10:04.753810+0000","last_peered":"2024-04-25T14:10:04.753810+0000","last_clean":"2024-04-25T14:10:04.753810+0000","last_became_active":"2024-04-25T14:10:04.751610+0000","last_became_peered":"2024-04-25T14:10:04.751610+0000","last_unstale":"2024-04-25T14:10:04.753810+0000","last_undegraded":"2024-04-25T14:10:04.753810+0000","last_fullsized":"2024-04-25T14:10:04.753810+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:54.546901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.735165+0000","last_change":"2024-04-25T14:10:04.735165+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.735165+0000","last_undegraded":"2024-04-25T14:10:04.735165+0000","last_fullsized":"2024-04-25T14:10:04.735165+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.752176+0000","last_change":"2024-04-25T14:10:04.749720+0000","last_active":"2024-04-25T14:10:04.752176+0000","last_peered":"2024-04-25T14:10:04.752176+0000","last_clean":"2024-04-25T14:10:04.752176+0000","last_became_active":"2024-04-25T14:10:04.749327+0000","last_became_peered":"2024-04-25T14:10:04.749327+0000","last_unstale":"2024-04-25T14:10:04.752176+0000","last_undegraded":"2024-04-25T14:10:04.752176+0000","last_fullsized":"2024-04-25T14:10:04.752176+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:30.105615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.738871+0000","last_change":"2024-04-25T14:10:04.738871+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.738871+0000","last_undegraded":"2024-04-25T14:10:04.738871+0000","last_fullsized":"2024-04-25T14:10:04.738871+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.752711+0000","last_change":"2024-04-25T14:10:04.750043+0000","last_active":"2024-04-25T14:10:04.752711+0000","last_peered":"2024-04-25T14:10:04.752711+0000","last_clean":"2024-04-25T14:10:04.752711+0000","last_became_active":"2024-04-25T14:10:04.749722+0000","last_became_peered":"2024-04-25T14:10:04.749722+0000","last_unstale":"2024-04-25T14:10:04.752711+0000","last_undegraded":"2024-04-25T14:10:04.752711+0000","last_fullsized":"2024-04-25T14:10:04.752711+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:54.546901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726953+0000","last_change":"2024-04-25T14:10:04.726953+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726953+0000","last_undegraded":"2024-04-25T14:10:04.726953+0000","last_fullsized":"2024-04-25T14:10:04.726953+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.755134+0000","last_change":"2024-04-25T14:10:04.754055+0000","last_active":"2024-04-25T14:10:04.755134+0000","last_peered":"2024-04-25T14:10:04.755134+0000","last_clean":"2024-04-25T14:10:04.755134+0000","last_became_active":"2024-04-25T14:10:04.753740+0000","last_became_peered":"2024-04-25T14:10:04.753740+0000","last_unstale":"2024-04-25T14:10:04.755134+0000","last_undegraded":"2024-04-25T14:10:04.755134+0000","last_fullsized":"2024-04-25T14:10:04.755134+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:16.658408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.734586+0000","last_change":"2024-04-25T14:10:04.734586+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.734586+0000","last_undegraded":"2024-04-25T14:10:04.734586+0000","last_fullsized":"2024-04-25T14:10:04.734586+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.754869+0000","last_change":"2024-04-25T14:10:04.753046+0000","last_active":"2024-04-25T14:10:04.754869+0000","last_peered":"2024-04-25T14:10:04.754869+0000","last_clean":"2024-04-25T14:10:04.754869+0000","last_became_active":"2024-04-25T14:10:04.749881+0000","last_became_peered":"2024-04-25T14:10:04.749881+0000","last_unstale":"2024-04-25T14:10:04.754869+0000","last_undegraded":"2024-04-25T14:10:04.754869+0000","last_fullsized":"2024-04-25T14:10:04.754869+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:39.976208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.731633+0000","last_change":"2024-04-25T14:10:04.731633+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.731633+0000","last_undegraded":"2024-04-25T14:10:04.731633+0000","last_fullsized":"2024-04-25T14:10:04.731633+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.752192+0000","last_change":"2024-04-25T14:10:04.749068+0000","last_active":"2024-04-25T14:10:04.752192+0000","last_peered":"2024-04-25T14:10:04.752192+0000","last_clean":"2024-04-25T14:10:04.752192+0000","last_became_active":"2024-04-25T14:10:04.748714+0000","last_became_peered":"2024-04-25T14:10:04.748714+0000","last_unstale":"2024-04-25T14:10:04.752192+0000","last_undegraded":"2024-04-25T14:10:04.752192+0000","last_fullsized":"2024-04-25T14:10:04.752192+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:45.118281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.731871+0000","last_change":"2024-04-25T14:10:04.731871+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.731871+0000","last_undegraded":"2024-04-25T14:10:04.731871+0000","last_fullsized":"2024-04-25T14:10:04.731871+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.750229+0000","last_change":"2024-04-25T14:10:04.746533+0000","last_active":"2024-04-25T14:10:04.750229+0000","last_peered":"2024-04-25T14:10:04.750229+0000","last_clean":"2024-04-25T14:10:04.750229+0000","last_became_active":"2024-04-25T14:10:04.746152+0000","last_became_peered":"2024-04-25T14:10:04.746152+0000","last_unstale":"2024-04-25T14:10:04.750229+0000","last_undegraded":"2024-04-25T14:10:04.750229+0000","last_fullsized":"2024-04-25T14:10:04.750229+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:01.747339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726535+0000","last_change":"2024-04-25T14:10:04.726535+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726535+0000","last_undegraded":"2024-04-25T14:10:04.726535+0000","last_fullsized":"2024-04-25T14:10:04.726535+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.752736+0000","last_change":"2024-04-25T14:10:04.752102+0000","last_active":"2024-04-25T14:10:04.752736+0000","last_peered":"2024-04-25T14:10:04.752736+0000","last_clean":"2024-04-25T14:10:04.752736+0000","last_became_active":"2024-04-25T14:10:04.751649+0000","last_became_peered":"2024-04-25T14:10:04.751649+0000","last_unstale":"2024-04-25T14:10:04.752736+0000","last_undegraded":"2024-04-25T14:10:04.752736+0000","last_fullsized":"2024-04-25T14:10:04.752736+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.238156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.736386+0000","last_change":"2024-04-25T14:10:04.736386+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.736386+0000","last_undegraded":"2024-04-25T14:10:04.736386+0000","last_fullsized":"2024-04-25T14:10:04.736386+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.754240+0000","last_change":"2024-04-25T14:10:04.749248+0000","last_active":"2024-04-25T14:10:04.754240+0000","last_peered":"2024-04-25T14:10:04.754240+0000","last_clean":"2024-04-25T14:10:04.754240+0000","last_became_active":"2024-04-25T14:10:04.748828+0000","last_became_peered":"2024-04-25T14:10:04.748828+0000","last_unstale":"2024-04-25T14:10:04.754240+0000","last_undegraded":"2024-04-25T14:10:04.754240+0000","last_fullsized":"2024-04-25T14:10:04.754240+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:38.859877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726322+0000","last_change":"2024-04-25T14:10:04.726322+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726322+0000","last_undegraded":"2024-04-25T14:10:04.726322+0000","last_fullsized":"2024-04-25T14:10:04.726322+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.749775+0000","last_change":"2024-04-25T14:10:04.746371+0000","last_active":"2024-04-25T14:10:04.749775+0000","last_peered":"2024-04-25T14:10:04.749775+0000","last_clean":"2024-04-25T14:10:04.749775+0000","last_became_active":"2024-04-25T14:10:04.745733+0000","last_became_peered":"2024-04-25T14:10:04.745733+0000","last_unstale":"2024-04-25T14:10:04.749775+0000","last_undegraded":"2024-04-25T14:10:04.749775+0000","last_fullsized":"2024-04-25T14:10:04.749775+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:54.546901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733546+0000","last_change":"2024-04-25T14:10:04.733546+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733546+0000","last_undegraded":"2024-04-25T14:10:04.733546+0000","last_fullsized":"2024-04-25T14:10:04.733546+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.752120+0000","last_change":"2024-04-25T14:10:04.747682+0000","last_active":"2024-04-25T14:10:04.752120+0000","last_peered":"2024-04-25T14:10:04.752120+0000","last_clean":"2024-04-25T14:10:04.752120+0000","last_became_active":"2024-04-25T14:10:04.747380+0000","last_became_peered":"2024-04-25T14:10:04.747380+0000","last_unstale":"2024-04-25T14:10:04.752120+0000","last_undegraded":"2024-04-25T14:10:04.752120+0000","last_fullsized":"2024-04-25T14:10:04.752120+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.238156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.736394+0000","last_change":"2024-04-25T14:10:04.736394+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.736394+0000","last_undegraded":"2024-04-25T14:10:04.736394+0000","last_fullsized":"2024-04-25T14:10:04.736394+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.754913+0000","last_change":"2024-04-25T14:10:04.752523+0000","last_active":"2024-04-25T14:10:04.754913+0000","last_peered":"2024-04-25T14:10:04.754913+0000","last_clean":"2024-04-25T14:10:04.754913+0000","last_became_active":"2024-04-25T14:10:04.750781+0000","last_became_peered":"2024-04-25T14:10:04.750781+0000","last_unstale":"2024-04-25T14:10:04.754913+0000","last_undegraded":"2024-04-25T14:10:04.754913+0000","last_fullsized":"2024-04-25T14:10:04.754913+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:30.252303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733923+0000","last_change":"2024-04-25T14:10:04.733923+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733923+0000","last_undegraded":"2024-04-25T14:10:04.733923+0000","last_fullsized":"2024-04-25T14:10:04.733923+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.748268+0000","last_change":"2024-04-25T14:10:04.746161+0000","last_active":"2024-04-25T14:10:04.748268+0000","last_peered":"2024-04-25T14:10:04.748268+0000","last_clean":"2024-04-25T14:10:04.748268+0000","last_became_active":"2024-04-25T14:10:04.745423+0000","last_became_peered":"2024-04-25T14:10:04.745423+0000","last_unstale":"2024-04-25T14:10:04.748268+0000","last_undegraded":"2024-04-25T14:10:04.748268+0000","last_fullsized":"2024-04-25T14:10:04.748268+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.238156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.730645+0000","last_change":"2024-04-25T14:10:04.730645+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.730645+0000","last_undegraded":"2024-04-25T14:10:04.730645+0000","last_fullsized":"2024-04-25T14:10:04.730645+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.747112+0000","last_change":"2024-04-25T14:10:04.742349+0000","last_active":"2024-04-25T14:10:04.747112+0000","last_peered":"2024-04-25T14:10:04.747112+0000","last_clean":"2024-04-25T14:10:04.747112+0000","last_became_active":"2024-04-25T14:10:04.741292+0000","last_became_peered":"2024-04-25T14:10:04.741292+0000","last_unstale":"2024-04-25T14:10:04.747112+0000","last_undegraded":"2024-04-25T14:10:04.747112+0000","last_fullsized":"2024-04-25T14:10:04.747112+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:54.546901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.733889+0000","last_change":"2024-04-25T14:10:04.733889+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.733889+0000","last_undegraded":"2024-04-25T14:10:04.733889+0000","last_fullsized":"2024-04-25T14:10:04.733889+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.754166+0000","last_change":"2024-04-25T14:10:04.749122+0000","last_active":"2024-04-25T14:10:04.754166+0000","last_peered":"2024-04-25T14:10:04.754166+0000","last_clean":"2024-04-25T14:10:04.754166+0000","last_became_active":"2024-04-25T14:10:04.748630+0000","last_became_peered":"2024-04-25T14:10:04.748630+0000","last_unstale":"2024-04-25T14:10:04.754166+0000","last_undegraded":"2024-04-25T14:10:04.754166+0000","last_fullsized":"2024-04-25T14:10:04.754166+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:21.269555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.726229+0000","last_change":"2024-04-25T14:10:04.726229+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.726229+0000","last_undegraded":"2024-04-25T14:10:04.726229+0000","last_fullsized":"2024-04-25T14:10:04.726229+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.749405+0000","last_change":"2024-04-25T14:10:04.745286+0000","last_active":"2024-04-25T14:10:04.749405+0000","last_peered":"2024-04-25T14:10:04.749405+0000","last_clean":"2024-04-25T14:10:04.749405+0000","last_became_active":"2024-04-25T14:10:04.744949+0000","last_became_peered":"2024-04-25T14:10:04.744949+0000","last_unstale":"2024-04-25T14:10:04.749405+0000","last_undegraded":"2024-04-25T14:10:04.749405+0000","last_fullsized":"2024-04-25T14:10:04.749405+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:25.644454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-25T14:10:04.734790+0000","last_change":"2024-04-25T14:10:04.734790+0000","last_active":"2024-04-25T14:10:04.713569+0000","last_peered":"2024-04-25T14:10:04.713569+0000","last_clean":"2024-04-25T14:10:04.713569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:04.734790+0000","last_undegraded":"2024-04-25T14:10:04.734790+0000","last_fullsized":"2024-04-25T14:10:04.734790+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.750270+0000","last_change":"2024-04-25T14:10:04.747339+0000","last_active":"2024-04-25T14:10:04.750270+0000","last_peered":"2024-04-25T14:10:04.750270+0000","last_clean":"2024-04-25T14:10:04.750270+0000","last_became_active":"2024-04-25T14:10:04.745494+0000","last_became_peered":"2024-04-25T14:10:04.745494+0000","last_unstale":"2024-04-25T14:10:04.750270+0000","last_undegraded":"2024-04-25T14:10:04.750270+0000","last_fullsized":"2024-04-25T14:10:04.750270+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:38.191463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"102'119","reported_seq":445,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T14:10:04.740092+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:10:04.740092+0000","last_peered":"2024-04-25T14:10:04.740092+0000","last_clean":"2024-04-25T14:10:04.740092+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:10:04.740092+0000","last_undegraded":"2024-04-25T14:10:04.740092+0000","last_fullsized":"2024-04-25T14:10:04.740092+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":468,"num_read_kb":1272,"num_write":181,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":468,"num_read_kb":1272,"num_write":181,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33108,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338732,"statfs":{"total":96636764160,"available":96602861568,"internally_reserved":0,"allocated":1966080,"data_stored":1273327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32524,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339316,"statfs":{"total":96636764160,"available":96603459584,"internally_reserved":0,"allocated":1372160,"data_stored":682959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1384448,"data_stored":689626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1372160,"data_stored":682959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1384448,"data_stored":689626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1384448,"data_stored":689626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1372160,"data_stored":682959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574924,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33116,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338724,"statfs":{"total":96636764160,"available":96602853376,"internally_reserved":0,"allocated":1978368,"data_stored":1279994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:10:14.454 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:10:14.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.617+0000 7f2e45800640 1 Processor -- start 2024-04-25T14:10:14.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.617+0000 7f2e45800640 1 -- start start 2024-04-25T14:10:14.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.617+0000 7f2e45800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e40071530 0x7f2e4006e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:14.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.617+0000 7f2e45800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2e4006e7a0 0x7f2e400cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:14.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.617+0000 7f2e45800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e4006ef90 con 0x7f2e400cf970 2024-04-25T14:10:14.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.617+0000 7f2e45800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f2e40163830 con 0x7f2e4006e7a0 2024-04-25T14:10:14.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.617+0000 7f2e45800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2e40163a80 con 0x7f2e40071530 2024-04-25T14:10:14.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.617+0000 7f2e3ea00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2e400cf970 0x7f2e40070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51896/0 (socket says 172.21.15.86:51896) 2024-04-25T14:10:14.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.617+0000 7f2e3ea00640 1 -- 172.21.15.86:0/3596342186 learned_addr learned my addr 172.21.15.86:0/3596342186 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:14.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.617+0000 7f2e36000640 1 --2- 172.21.15.86:0/3596342186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e40071530 0x7f2e4006e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:14.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.617+0000 7f2e3f400640 1 --2- 172.21.15.86:0/3596342186 >> v2:172.21.15.155:3300/0 conn(0x7f2e4006e7a0 0x7f2e400cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:14.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.617+0000 7f2e3e000640 1 -- 172.21.15.86:0/3596342186 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1073926272 0 0) 0x7f2e4006ef90 con 0x7f2e400cf970 2024-04-25T14:10:14.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.617+0000 7f2e3e000640 1 -- 172.21.15.86:0/3596342186 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e24003530 con 0x7f2e400cf970 2024-04-25T14:10:14.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.621+0000 7f2e3e000640 1 -- 172.21.15.86:0/3596342186 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1346389451 0 0) 0x7f2e24003530 con 0x7f2e400cf970 2024-04-25T14:10:14.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.621+0000 7f2e36000640 1 -- 172.21.15.86:0/3596342186 >> v2:172.21.15.155:3300/0 conn(0x7f2e4006e7a0 msgr2=0x7f2e400cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:14.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.621+0000 7f2e36000640 1 --2- 172.21.15.86:0/3596342186 >> v2:172.21.15.155:3300/0 conn(0x7f2e4006e7a0 0x7f2e400cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:14.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.621+0000 7f2e36000640 1 -- 172.21.15.86:0/3596342186 >> v1:172.21.15.86:6789/0 conn(0x7f2e400cf970 legacy=0x7f2e40070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:14.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.621+0000 7f2e36000640 1 -- 172.21.15.86:0/3596342186 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e40163d60 con 0x7f2e40071530 2024-04-25T14:10:14.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.621+0000 7f2e36000640 1 --2- 172.21.15.86:0/3596342186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e40071530 0x7f2e4006e260 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2e280097a0 tx=0x7f2e2801c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c8026f810fd2eb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:14.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.621+0000 7f2e3e000640 1 -- 172.21.15.86:0/3596342186 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e280b0070 con 0x7f2e40071530 2024-04-25T14:10:14.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.621+0000 7f2e3e000640 1 -- 172.21.15.86:0/3596342186 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e280ab400 con 0x7f2e40071530 2024-04-25T14:10:14.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.621+0000 7f2e3e000640 1 -- 172.21.15.86:0/3596342186 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e30001680 con 0x7f2e40071530 2024-04-25T14:10:14.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.625+0000 7f2e45800640 1 -- 172.21.15.86:0/3596342186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e40071530 msgr2=0x7f2e4006e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:14.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.625+0000 7f2e45800640 1 --2- 172.21.15.86:0/3596342186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e40071530 0x7f2e4006e260 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2e280097a0 tx=0x7f2e2801c370 comp rx=0 tx=0).stop 2024-04-25T14:10:14.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.625+0000 7f2e45800640 1 -- 172.21.15.86:0/3596342186 shutdown_connections 2024-04-25T14:10:14.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.625+0000 7f2e45800640 1 --2- 172.21.15.86:0/3596342186 >> v2:172.21.15.155:3300/0 conn(0x7f2e4006e7a0 0x7f2e400cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:14.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.625+0000 7f2e45800640 1 --2- 172.21.15.86:0/3596342186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e40071530 0x7f2e4006e260 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:14.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.625+0000 7f2e45800640 1 -- 172.21.15.86:0/3596342186 >> 172.21.15.86:0/3596342186 conn(0x7f2e400cffc0 msgr2=0x7f2e400d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:14.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.625+0000 7f2e45800640 1 -- 172.21.15.86:0/3596342186 shutdown_connections 2024-04-25T14:10:14.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.625+0000 7f2e45800640 1 -- 172.21.15.86:0/3596342186 wait complete. 2024-04-25T14:10:14.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.625+0000 7f2e45800640 1 Processor -- start 2024-04-25T14:10:14.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.625+0000 7f2e45800640 1 -- start start 2024-04-25T14:10:14.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.629+0000 7f2e45800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e4006e7a0 0x7f2e40161f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:14.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.629+0000 7f2e45800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e400cf970 0x7f2e400cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:14.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.629+0000 7f2e45800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e401f94f0 0x7f2e401f9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:14.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.629+0000 7f2e45800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4016cbd0 con 0x7f2e401f94f0 2024-04-25T14:10:14.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.629+0000 7f2e45800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2e400ced80 con 0x7f2e400cf970 2024-04-25T14:10:14.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.629+0000 7f2e45800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2e4016cd50 con 0x7f2e4006e7a0 2024-04-25T14:10:14.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.629+0000 7f2e3f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e401f94f0 0x7f2e401f9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:14.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.629+0000 7f2e3f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e401f94f0 0x7f2e401f9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53994/0 (socket says 172.21.15.86:53994) 2024-04-25T14:10:14.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.629+0000 7f2e3f400640 1 -- 172.21.15.86:0/3479067166 learned_addr learned my addr 172.21.15.86:0/3479067166 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:14.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.629+0000 7f2e3ea00640 1 --2- 172.21.15.86:0/3479067166 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e4006e7a0 0x7f2e40161f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:14.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.629+0000 7f2e36000640 1 --2- 172.21.15.86:0/3479067166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e400cf970 0x7f2e400cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:14.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.629+0000 7f2e3f400640 1 -- 172.21.15.86:0/3479067166 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e4006e7a0 msgr2=0x7f2e40161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:14.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.629+0000 7f2e3f400640 1 --2- 172.21.15.86:0/3479067166 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e4006e7a0 0x7f2e40161f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:14.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.629+0000 7f2e3f400640 1 -- 172.21.15.86:0/3479067166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e400cf970 msgr2=0x7f2e400cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:14.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.629+0000 7f2e3f400640 1 --2- 172.21.15.86:0/3479067166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e400cf970 0x7f2e400cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:14.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.629+0000 7f2e3f400640 1 -- 172.21.15.86:0/3479067166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e401fa6b0 con 0x7f2e401f94f0 2024-04-25T14:10:14.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.629+0000 7f2e3f400640 1 --2- 172.21.15.86:0/3479067166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e401f94f0 0x7f2e401f9900 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800efc0 tx=0x7f2e3800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:14.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.629+0000 7f2e37400640 1 -- 172.21.15.86:0/3479067166 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38018020 con 0x7f2e401f94f0 2024-04-25T14:10:14.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.629+0000 7f2e37400640 1 -- 172.21.15.86:0/3479067166 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e38014480 con 0x7f2e401f94f0 2024-04-25T14:10:14.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.629+0000 7f2e37400640 1 -- 172.21.15.86:0/3479067166 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38009e60 con 0x7f2e401f94f0 2024-04-25T14:10:14.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.629+0000 7f2e45800640 1 -- 172.21.15.86:0/3479067166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e40157780 con 0x7f2e401f94f0 2024-04-25T14:10:14.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.629+0000 7f2e45800640 1 -- 172.21.15.86:0/3479067166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e40157d00 con 0x7f2e401f94f0 2024-04-25T14:10:14.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.629+0000 7f2e45800640 1 -- 172.21.15.86:0/3479067166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e401fa420 con 0x7f2e401f94f0 2024-04-25T14:10:14.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.633+0000 7f2e37400640 1 -- 172.21.15.86:0/3479067166 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7f2e3801b020 con 0x7f2e401f94f0 2024-04-25T14:10:14.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.633+0000 7f2e37400640 1 --2- 172.21.15.86:0/3479067166 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f2e1c06f520 0x7f2e1c071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:14.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.633+0000 7f2e37400640 1 -- 172.21.15.86:0/3479067166 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f2e38090710 con 0x7f2e401f94f0 2024-04-25T14:10:14.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.633+0000 7f2e3ea00640 1 --2- 172.21.15.86:0/3479067166 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f2e1c06f520 0x7f2e1c071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:14.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.633+0000 7f2e3ea00640 1 --2- 172.21.15.86:0/3479067166 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f2e1c06f520 0x7f2e1c071a30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2e30001330 tx=0x7f2e300012c0 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:14.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.637+0000 7f2e37400640 1 -- 172.21.15.86:0/3479067166 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2e38022030 con 0x7f2e401f94f0 2024-04-25T14:10:14.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.789+0000 7f2e45800640 1 -- 172.21.15.86:0/3479067166 --> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2e40069d60 con 0x7f2e1c06f520 2024-04-25T14:10:14.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.797+0000 7f2e37400640 1 -- 172.21.15.86:0/3479067166 <== mgr.8190 v2:172.21.15.86:6832/1354467416 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252491 (secure 0 0 0) 0x7f2e3000f870 con 0x7f2e1c06f520 2024-04-25T14:10:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:14.797 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:10:14.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.797+0000 7f2e45800640 1 -- 172.21.15.86:0/3479067166 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f2e1c06f520 msgr2=0x7f2e1c071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:14.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.797+0000 7f2e45800640 1 --2- 172.21.15.86:0/3479067166 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f2e1c06f520 0x7f2e1c071a30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2e30001330 tx=0x7f2e300012c0 comp rx=0 tx=0).stop 2024-04-25T14:10:14.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.797+0000 7f2e45800640 1 -- 172.21.15.86:0/3479067166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e401f94f0 msgr2=0x7f2e401f9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:14.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.797+0000 7f2e45800640 1 --2- 172.21.15.86:0/3479067166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e401f94f0 0x7f2e401f9900 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800efc0 tx=0x7f2e3800c490 comp rx=0 tx=0).stop 2024-04-25T14:10:14.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.801+0000 7f2e45800640 1 -- 172.21.15.86:0/3479067166 shutdown_connections 2024-04-25T14:10:14.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.801+0000 7f2e45800640 1 --2- 172.21.15.86:0/3479067166 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f2e1c06f520 0x7f2e1c071a30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:14.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.801+0000 7f2e45800640 1 --2- 172.21.15.86:0/3479067166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e401f94f0 0x7f2e401f9900 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:14.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.801+0000 7f2e45800640 1 --2- 172.21.15.86:0/3479067166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e400cf970 0x7f2e400cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:14.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.801+0000 7f2e45800640 1 --2- 172.21.15.86:0/3479067166 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e4006e7a0 0x7f2e40161f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:14.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.801+0000 7f2e45800640 1 -- 172.21.15.86:0/3479067166 >> 172.21.15.86:0/3479067166 conn(0x7f2e400cffc0 msgr2=0x7f2e400b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:14.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.801+0000 7f2e45800640 1 -- 172.21.15.86:0/3479067166 shutdown_connections 2024-04-25T14:10:14.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:14.801+0000 7f2e45800640 1 -- 172.21.15.86:0/3479067166 wait complete. 2024-04-25T14:10:14.817 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-04-25T14:10:13.475805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":468,"num_read_kb":1272,"num_write":226,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261728,"kb_used_data":12232,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712992,"statfs":{"total":773094113280,"available":772826103808,"internally_reserved":0,"allocated":12525568,"data_stored":6813436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001889"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.994837+0000","last_change":"2024-04-25T14:10:05.749089+0000","last_active":"2024-04-25T14:10:06.994837+0000","last_peered":"2024-04-25T14:10:06.994837+0000","last_clean":"2024-04-25T14:10:06.994837+0000","last_became_active":"2024-04-25T14:10:05.748767+0000","last_became_peered":"2024-04-25T14:10:05.748767+0000","last_unstale":"2024-04-25T14:10:06.994837+0000","last_undegraded":"2024-04-25T14:10:06.994837+0000","last_fullsized":"2024-04-25T14:10:06.994837+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:12.766663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.730292+0000","last_change":"2024-04-25T14:10:05.764581+0000","last_active":"2024-04-25T14:10:06.730292+0000","last_peered":"2024-04-25T14:10:06.730292+0000","last_clean":"2024-04-25T14:10:06.730292+0000","last_became_active":"2024-04-25T14:10:05.764227+0000","last_became_peered":"2024-04-25T14:10:05.764227+0000","last_unstale":"2024-04-25T14:10:06.730292+0000","last_undegraded":"2024-04-25T14:10:06.730292+0000","last_fullsized":"2024-04-25T14:10:06.730292+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:43.362784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.730433+0000","last_change":"2024-04-25T14:10:05.764908+0000","last_active":"2024-04-25T14:10:06.730433+0000","last_peered":"2024-04-25T14:10:06.730433+0000","last_clean":"2024-04-25T14:10:06.730433+0000","last_became_active":"2024-04-25T14:10:05.762400+0000","last_became_peered":"2024-04-25T14:10:05.762400+0000","last_unstale":"2024-04-25T14:10:06.730433+0000","last_undegraded":"2024-04-25T14:10:06.730433+0000","last_fullsized":"2024-04-25T14:10:06.730433+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:22.683983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.994101+0000","last_change":"2024-04-25T14:10:05.760755+0000","last_active":"2024-04-25T14:10:06.994101+0000","last_peered":"2024-04-25T14:10:06.994101+0000","last_clean":"2024-04-25T14:10:06.994101+0000","last_became_active":"2024-04-25T14:10:05.760447+0000","last_became_peered":"2024-04-25T14:10:05.760447+0000","last_unstale":"2024-04-25T14:10:06.994101+0000","last_undegraded":"2024-04-25T14:10:06.994101+0000","last_fullsized":"2024-04-25T14:10:06.994101+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:54.449161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.993113+0000","last_change":"2024-04-25T14:10:05.768093+0000","last_active":"2024-04-25T14:10:06.993113+0000","last_peered":"2024-04-25T14:10:06.993113+0000","last_clean":"2024-04-25T14:10:06.993113+0000","last_became_active":"2024-04-25T14:10:05.767772+0000","last_became_peered":"2024-04-25T14:10:05.767772+0000","last_unstale":"2024-04-25T14:10:06.993113+0000","last_undegraded":"2024-04-25T14:10:06.993113+0000","last_fullsized":"2024-04-25T14:10:06.993113+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:45.173029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.757267+0000","last_change":"2024-04-25T14:10:05.750464+0000","last_active":"2024-04-25T14:10:06.757267+0000","last_peered":"2024-04-25T14:10:06.757267+0000","last_clean":"2024-04-25T14:10:06.757267+0000","last_became_active":"2024-04-25T14:10:05.750152+0000","last_became_peered":"2024-04-25T14:10:05.750152+0000","last_unstale":"2024-04-25T14:10:06.757267+0000","last_undegraded":"2024-04-25T14:10:06.757267+0000","last_fullsized":"2024-04-25T14:10:06.757267+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:54.556554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.757148+0000","last_change":"2024-04-25T14:10:05.741457+0000","last_active":"2024-04-25T14:10:06.757148+0000","last_peered":"2024-04-25T14:10:06.757148+0000","last_clean":"2024-04-25T14:10:06.757148+0000","last_became_active":"2024-04-25T14:10:05.741088+0000","last_became_peered":"2024-04-25T14:10:05.741088+0000","last_unstale":"2024-04-25T14:10:06.757148+0000","last_undegraded":"2024-04-25T14:10:06.757148+0000","last_fullsized":"2024-04-25T14:10:06.757148+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:28.912310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.729297+0000","last_change":"2024-04-25T14:10:05.768776+0000","last_active":"2024-04-25T14:10:06.729297+0000","last_peered":"2024-04-25T14:10:06.729297+0000","last_clean":"2024-04-25T14:10:06.729297+0000","last_became_active":"2024-04-25T14:10:05.768412+0000","last_became_peered":"2024-04-25T14:10:05.768412+0000","last_unstale":"2024-04-25T14:10:06.729297+0000","last_undegraded":"2024-04-25T14:10:06.729297+0000","last_fullsized":"2024-04-25T14:10:06.729297+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:03.579045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.994474+0000","last_change":"2024-04-25T14:10:05.761897+0000","last_active":"2024-04-25T14:10:06.994474+0000","last_peered":"2024-04-25T14:10:06.994474+0000","last_clean":"2024-04-25T14:10:06.994474+0000","last_became_active":"2024-04-25T14:10:05.761554+0000","last_became_peered":"2024-04-25T14:10:05.761554+0000","last_unstale":"2024-04-25T14:10:06.994474+0000","last_undegraded":"2024-04-25T14:10:06.994474+0000","last_fullsized":"2024-04-25T14:10:06.994474+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:18.442200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.730846+0000","last_change":"2024-04-25T14:10:05.749580+0000","last_active":"2024-04-25T14:10:06.730846+0000","last_peered":"2024-04-25T14:10:06.730846+0000","last_clean":"2024-04-25T14:10:06.730846+0000","last_became_active":"2024-04-25T14:10:05.747961+0000","last_became_peered":"2024-04-25T14:10:05.747961+0000","last_unstale":"2024-04-25T14:10:06.730846+0000","last_undegraded":"2024-04-25T14:10:06.730846+0000","last_fullsized":"2024-04-25T14:10:06.730846+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:27.170899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.993398+0000","last_change":"2024-04-25T14:10:05.751938+0000","last_active":"2024-04-25T14:10:06.993398+0000","last_peered":"2024-04-25T14:10:06.993398+0000","last_clean":"2024-04-25T14:10:06.993398+0000","last_became_active":"2024-04-25T14:10:05.749328+0000","last_became_peered":"2024-04-25T14:10:05.749328+0000","last_unstale":"2024-04-25T14:10:06.993398+0000","last_undegraded":"2024-04-25T14:10:06.993398+0000","last_fullsized":"2024-04-25T14:10:06.993398+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:06.710017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.757504+0000","last_change":"2024-04-25T14:10:05.757941+0000","last_active":"2024-04-25T14:10:06.757504+0000","last_peered":"2024-04-25T14:10:06.757504+0000","last_clean":"2024-04-25T14:10:06.757504+0000","last_became_active":"2024-04-25T14:10:05.754205+0000","last_became_peered":"2024-04-25T14:10:05.754205+0000","last_unstale":"2024-04-25T14:10:06.757504+0000","last_undegraded":"2024-04-25T14:10:06.757504+0000","last_fullsized":"2024-04-25T14:10:06.757504+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:13:30.285723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.730392+0000","last_change":"2024-04-25T14:10:05.746420+0000","last_active":"2024-04-25T14:10:06.730392+0000","last_peered":"2024-04-25T14:10:06.730392+0000","last_clean":"2024-04-25T14:10:06.730392+0000","last_became_active":"2024-04-25T14:10:05.746107+0000","last_became_peered":"2024-04-25T14:10:05.746107+0000","last_unstale":"2024-04-25T14:10:06.730392+0000","last_undegraded":"2024-04-25T14:10:06.730392+0000","last_fullsized":"2024-04-25T14:10:06.730392+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:10.182930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.729581+0000","last_change":"2024-04-25T14:10:05.763427+0000","last_active":"2024-04-25T14:10:06.729581+0000","last_peered":"2024-04-25T14:10:06.729581+0000","last_clean":"2024-04-25T14:10:06.729581+0000","last_became_active":"2024-04-25T14:10:05.763102+0000","last_became_peered":"2024-04-25T14:10:05.763102+0000","last_unstale":"2024-04-25T14:10:06.729581+0000","last_undegraded":"2024-04-25T14:10:06.729581+0000","last_fullsized":"2024-04-25T14:10:06.729581+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:25.517622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.993125+0000","last_change":"2024-04-25T14:10:05.743959+0000","last_active":"2024-04-25T14:10:06.993125+0000","last_peered":"2024-04-25T14:10:06.993125+0000","last_clean":"2024-04-25T14:10:06.993125+0000","last_became_active":"2024-04-25T14:10:05.743129+0000","last_became_peered":"2024-04-25T14:10:05.743129+0000","last_unstale":"2024-04-25T14:10:06.993125+0000","last_undegraded":"2024-04-25T14:10:06.993125+0000","last_fullsized":"2024-04-25T14:10:06.993125+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:13.696298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.757040+0000","last_change":"2024-04-25T14:10:05.744727+0000","last_active":"2024-04-25T14:10:06.757040+0000","last_peered":"2024-04-25T14:10:06.757040+0000","last_clean":"2024-04-25T14:10:06.757040+0000","last_became_active":"2024-04-25T14:10:05.743865+0000","last_became_peered":"2024-04-25T14:10:05.743865+0000","last_unstale":"2024-04-25T14:10:06.757040+0000","last_undegraded":"2024-04-25T14:10:06.757040+0000","last_fullsized":"2024-04-25T14:10:06.757040+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:26.323868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.730542+0000","last_change":"2024-04-25T14:10:05.751095+0000","last_active":"2024-04-25T14:10:06.730542+0000","last_peered":"2024-04-25T14:10:06.730542+0000","last_clean":"2024-04-25T14:10:06.730542+0000","last_became_active":"2024-04-25T14:10:05.750731+0000","last_became_peered":"2024-04-25T14:10:05.750731+0000","last_unstale":"2024-04-25T14:10:06.730542+0000","last_undegraded":"2024-04-25T14:10:06.730542+0000","last_fullsized":"2024-04-25T14:10:06.730542+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:06.710017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.730461+0000","last_change":"2024-04-25T14:10:05.752169+0000","last_active":"2024-04-25T14:10:06.730461+0000","last_peered":"2024-04-25T14:10:06.730461+0000","last_clean":"2024-04-25T14:10:06.730461+0000","last_became_active":"2024-04-25T14:10:05.751242+0000","last_became_peered":"2024-04-25T14:10:05.751242+0000","last_unstale":"2024-04-25T14:10:06.730461+0000","last_undegraded":"2024-04-25T14:10:06.730461+0000","last_fullsized":"2024-04-25T14:10:06.730461+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:14.702136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.754643+0000","last_change":"2024-04-25T14:10:05.757362+0000","last_active":"2024-04-25T14:10:06.754643+0000","last_peered":"2024-04-25T14:10:06.754643+0000","last_clean":"2024-04-25T14:10:06.754643+0000","last_became_active":"2024-04-25T14:10:05.756210+0000","last_became_peered":"2024-04-25T14:10:05.756210+0000","last_unstale":"2024-04-25T14:10:06.754643+0000","last_undegraded":"2024-04-25T14:10:06.754643+0000","last_fullsized":"2024-04-25T14:10:06.754643+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:13.696298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.993162+0000","last_change":"2024-04-25T14:10:05.743654+0000","last_active":"2024-04-25T14:10:06.993162+0000","last_peered":"2024-04-25T14:10:06.993162+0000","last_clean":"2024-04-25T14:10:06.993162+0000","last_became_active":"2024-04-25T14:10:05.743314+0000","last_became_peered":"2024-04-25T14:10:05.743314+0000","last_unstale":"2024-04-25T14:10:06.993162+0000","last_undegraded":"2024-04-25T14:10:06.993162+0000","last_fullsized":"2024-04-25T14:10:06.993162+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:21.875230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.994161+0000","last_change":"2024-04-25T14:10:05.754814+0000","last_active":"2024-04-25T14:10:06.994161+0000","last_peered":"2024-04-25T14:10:06.994161+0000","last_clean":"2024-04-25T14:10:06.994161+0000","last_became_active":"2024-04-25T14:10:05.754106+0000","last_became_peered":"2024-04-25T14:10:05.754106+0000","last_unstale":"2024-04-25T14:10:06.994161+0000","last_undegraded":"2024-04-25T14:10:06.994161+0000","last_fullsized":"2024-04-25T14:10:06.994161+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:19.318885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.993248+0000","last_change":"2024-04-25T14:10:05.762736+0000","last_active":"2024-04-25T14:10:06.993248+0000","last_peered":"2024-04-25T14:10:06.993248+0000","last_clean":"2024-04-25T14:10:06.993248+0000","last_became_active":"2024-04-25T14:10:05.762318+0000","last_became_peered":"2024-04-25T14:10:05.762318+0000","last_unstale":"2024-04-25T14:10:06.993248+0000","last_undegraded":"2024-04-25T14:10:06.993248+0000","last_fullsized":"2024-04-25T14:10:06.993248+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:40.982158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.994204+0000","last_change":"2024-04-25T14:10:05.761096+0000","last_active":"2024-04-25T14:10:06.994204+0000","last_peered":"2024-04-25T14:10:06.994204+0000","last_clean":"2024-04-25T14:10:06.994204+0000","last_became_active":"2024-04-25T14:10:05.760796+0000","last_became_peered":"2024-04-25T14:10:05.760796+0000","last_unstale":"2024-04-25T14:10:06.994204+0000","last_undegraded":"2024-04-25T14:10:06.994204+0000","last_fullsized":"2024-04-25T14:10:06.994204+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:12.766663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.729883+0000","last_change":"2024-04-25T14:10:05.760783+0000","last_active":"2024-04-25T14:10:06.729883+0000","last_peered":"2024-04-25T14:10:06.729883+0000","last_clean":"2024-04-25T14:10:06.729883+0000","last_became_active":"2024-04-25T14:10:05.760456+0000","last_became_peered":"2024-04-25T14:10:05.760456+0000","last_unstale":"2024-04-25T14:10:06.729883+0000","last_undegraded":"2024-04-25T14:10:06.729883+0000","last_fullsized":"2024-04-25T14:10:06.729883+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:43.667564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.730415+0000","last_change":"2024-04-25T14:10:05.741716+0000","last_active":"2024-04-25T14:10:06.730415+0000","last_peered":"2024-04-25T14:10:06.730415+0000","last_clean":"2024-04-25T14:10:06.730415+0000","last_became_active":"2024-04-25T14:10:05.741408+0000","last_became_peered":"2024-04-25T14:10:05.741408+0000","last_unstale":"2024-04-25T14:10:06.730415+0000","last_undegraded":"2024-04-25T14:10:06.730415+0000","last_fullsized":"2024-04-25T14:10:06.730415+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:10.182930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.993234+0000","last_change":"2024-04-25T14:10:05.762001+0000","last_active":"2024-04-25T14:10:06.993234+0000","last_peered":"2024-04-25T14:10:06.993234+0000","last_clean":"2024-04-25T14:10:06.993234+0000","last_became_active":"2024-04-25T14:10:05.761692+0000","last_became_peered":"2024-04-25T14:10:05.761692+0000","last_unstale":"2024-04-25T14:10:06.993234+0000","last_undegraded":"2024-04-25T14:10:06.993234+0000","last_fullsized":"2024-04-25T14:10:06.993234+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:09.737984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.754675+0000","last_change":"2024-04-25T14:10:05.756484+0000","last_active":"2024-04-25T14:10:06.754675+0000","last_peered":"2024-04-25T14:10:06.754675+0000","last_clean":"2024-04-25T14:10:06.754675+0000","last_became_active":"2024-04-25T14:10:05.755756+0000","last_became_peered":"2024-04-25T14:10:05.755756+0000","last_unstale":"2024-04-25T14:10:06.754675+0000","last_undegraded":"2024-04-25T14:10:06.754675+0000","last_fullsized":"2024-04-25T14:10:06.754675+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:10.182930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.757498+0000","last_change":"2024-04-25T14:10:05.747575+0000","last_active":"2024-04-25T14:10:06.757498+0000","last_peered":"2024-04-25T14:10:06.757498+0000","last_clean":"2024-04-25T14:10:06.757498+0000","last_became_active":"2024-04-25T14:10:05.747253+0000","last_became_peered":"2024-04-25T14:10:05.747253+0000","last_unstale":"2024-04-25T14:10:06.757498+0000","last_undegraded":"2024-04-25T14:10:06.757498+0000","last_fullsized":"2024-04-25T14:10:06.757498+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:36.942319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.993428+0000","last_change":"2024-04-25T14:10:05.757230+0000","last_active":"2024-04-25T14:10:06.993428+0000","last_peered":"2024-04-25T14:10:06.993428+0000","last_clean":"2024-04-25T14:10:06.993428+0000","last_became_active":"2024-04-25T14:10:05.756900+0000","last_became_peered":"2024-04-25T14:10:05.756900+0000","last_unstale":"2024-04-25T14:10:06.993428+0000","last_undegraded":"2024-04-25T14:10:06.993428+0000","last_fullsized":"2024-04-25T14:10:06.993428+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:24.767098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.729745+0000","last_change":"2024-04-25T14:10:05.742534+0000","last_active":"2024-04-25T14:10:06.729745+0000","last_peered":"2024-04-25T14:10:06.729745+0000","last_clean":"2024-04-25T14:10:06.729745+0000","last_became_active":"2024-04-25T14:10:05.742200+0000","last_became_peered":"2024-04-25T14:10:05.742200+0000","last_unstale":"2024-04-25T14:10:06.729745+0000","last_undegraded":"2024-04-25T14:10:06.729745+0000","last_fullsized":"2024-04-25T14:10:06.729745+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:56.641564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.730568+0000","last_change":"2024-04-25T14:10:05.749962+0000","last_active":"2024-04-25T14:10:06.730568+0000","last_peered":"2024-04-25T14:10:06.730568+0000","last_clean":"2024-04-25T14:10:06.730568+0000","last_became_active":"2024-04-25T14:10:05.749601+0000","last_became_peered":"2024-04-25T14:10:05.749601+0000","last_unstale":"2024-04-25T14:10:06.730568+0000","last_undegraded":"2024-04-25T14:10:06.730568+0000","last_fullsized":"2024-04-25T14:10:06.730568+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:12.766663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.729902+0000","last_change":"2024-04-25T14:10:05.757998+0000","last_active":"2024-04-25T14:10:06.729902+0000","last_peered":"2024-04-25T14:10:06.729902+0000","last_clean":"2024-04-25T14:10:06.729902+0000","last_became_active":"2024-04-25T14:10:05.757378+0000","last_became_peered":"2024-04-25T14:10:05.757378+0000","last_unstale":"2024-04-25T14:10:06.729902+0000","last_undegraded":"2024-04-25T14:10:06.729902+0000","last_fullsized":"2024-04-25T14:10:06.729902+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:24.418653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.730839+0000","last_change":"2024-04-25T14:10:05.748658+0000","last_active":"2024-04-25T14:10:06.730839+0000","last_peered":"2024-04-25T14:10:06.730839+0000","last_clean":"2024-04-25T14:10:06.730839+0000","last_became_active":"2024-04-25T14:10:05.748321+0000","last_became_peered":"2024-04-25T14:10:05.748321+0000","last_unstale":"2024-04-25T14:10:06.730839+0000","last_undegraded":"2024-04-25T14:10:06.730839+0000","last_fullsized":"2024-04-25T14:10:06.730839+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:12.766663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"112'4","reported_seq":18,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.754828+0000","last_change":"2024-04-25T14:10:04.745986+0000","last_active":"2024-04-25T14:10:06.754828+0000","last_peered":"2024-04-25T14:10:06.754828+0000","last_clean":"2024-04-25T14:10:06.754828+0000","last_became_active":"2024-04-25T14:10:04.745516+0000","last_became_peered":"2024-04-25T14:10:04.745516+0000","last_unstale":"2024-04-25T14:10:06.754828+0000","last_undegraded":"2024-04-25T14:10:06.754828+0000","last_fullsized":"2024-04-25T14:10:06.754828+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:22.606342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.757509+0000","last_change":"2024-04-25T14:10:05.753060+0000","last_active":"2024-04-25T14:10:06.757509+0000","last_peered":"2024-04-25T14:10:06.757509+0000","last_clean":"2024-04-25T14:10:06.757509+0000","last_became_active":"2024-04-25T14:10:05.752765+0000","last_became_peered":"2024-04-25T14:10:05.752765+0000","last_unstale":"2024-04-25T14:10:06.757509+0000","last_undegraded":"2024-04-25T14:10:06.757509+0000","last_fullsized":"2024-04-25T14:10:06.757509+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:33.166238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.730946+0000","last_change":"2024-04-25T14:10:04.744056+0000","last_active":"2024-04-25T14:10:06.730946+0000","last_peered":"2024-04-25T14:10:06.730946+0000","last_clean":"2024-04-25T14:10:06.730946+0000","last_became_active":"2024-04-25T14:10:04.743755+0000","last_became_peered":"2024-04-25T14:10:04.743755+0000","last_unstale":"2024-04-25T14:10:06.730946+0000","last_undegraded":"2024-04-25T14:10:06.730946+0000","last_fullsized":"2024-04-25T14:10:06.730946+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:16.658408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.730280+0000","last_change":"2024-04-25T14:10:05.742925+0000","last_active":"2024-04-25T14:10:06.730280+0000","last_peered":"2024-04-25T14:10:06.730280+0000","last_clean":"2024-04-25T14:10:06.730280+0000","last_became_active":"2024-04-25T14:10:05.742551+0000","last_became_peered":"2024-04-25T14:10:05.742551+0000","last_unstale":"2024-04-25T14:10:06.730280+0000","last_undegraded":"2024-04-25T14:10:06.730280+0000","last_fullsized":"2024-04-25T14:10:06.730280+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:13.696298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.994387+0000","last_change":"2024-04-25T14:10:04.752833+0000","last_active":"2024-04-25T14:10:06.994387+0000","last_peered":"2024-04-25T14:10:06.994387+0000","last_clean":"2024-04-25T14:10:06.994387+0000","last_became_active":"2024-04-25T14:10:04.752483+0000","last_became_peered":"2024-04-25T14:10:04.752483+0000","last_unstale":"2024-04-25T14:10:06.994387+0000","last_undegraded":"2024-04-25T14:10:06.994387+0000","last_fullsized":"2024-04-25T14:10:06.994387+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:38.191463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.994615+0000","last_change":"2024-04-25T14:10:05.755355+0000","last_active":"2024-04-25T14:10:06.994615+0000","last_peered":"2024-04-25T14:10:06.994615+0000","last_clean":"2024-04-25T14:10:06.994615+0000","last_became_active":"2024-04-25T14:10:05.750001+0000","last_became_peered":"2024-04-25T14:10:05.750001+0000","last_unstale":"2024-04-25T14:10:06.994615+0000","last_undegraded":"2024-04-25T14:10:06.994615+0000","last_fullsized":"2024-04-25T14:10:06.994615+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:54.230885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.730775+0000","last_change":"2024-04-25T14:10:04.733242+0000","last_active":"2024-04-25T14:10:06.730775+0000","last_peered":"2024-04-25T14:10:06.730775+0000","last_clean":"2024-04-25T14:10:06.730775+0000","last_became_active":"2024-04-25T14:10:04.732929+0000","last_became_peered":"2024-04-25T14:10:04.732929+0000","last_unstale":"2024-04-25T14:10:06.730775+0000","last_undegraded":"2024-04-25T14:10:06.730775+0000","last_fullsized":"2024-04-25T14:10:06.730775+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:50.427076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.729842+0000","last_change":"2024-04-25T14:10:05.767605+0000","last_active":"2024-04-25T14:10:06.729842+0000","last_peered":"2024-04-25T14:10:06.729842+0000","last_clean":"2024-04-25T14:10:06.729842+0000","last_became_active":"2024-04-25T14:10:05.767319+0000","last_became_peered":"2024-04-25T14:10:05.767319+0000","last_unstale":"2024-04-25T14:10:06.729842+0000","last_undegraded":"2024-04-25T14:10:06.729842+0000","last_fullsized":"2024-04-25T14:10:06.729842+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:24.418653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.993309+0000","last_change":"2024-04-25T14:10:04.738920+0000","last_active":"2024-04-25T14:10:06.993309+0000","last_peered":"2024-04-25T14:10:06.993309+0000","last_clean":"2024-04-25T14:10:06.993309+0000","last_became_active":"2024-04-25T14:10:04.738531+0000","last_became_peered":"2024-04-25T14:10:04.738531+0000","last_unstale":"2024-04-25T14:10:06.993309+0000","last_undegraded":"2024-04-25T14:10:06.993309+0000","last_fullsized":"2024-04-25T14:10:06.993309+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:16.658408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.992992+0000","last_change":"2024-04-25T14:10:05.744263+0000","last_active":"2024-04-25T14:10:06.992992+0000","last_peered":"2024-04-25T14:10:06.992992+0000","last_clean":"2024-04-25T14:10:06.992992+0000","last_became_active":"2024-04-25T14:10:05.743653+0000","last_became_peered":"2024-04-25T14:10:05.743653+0000","last_unstale":"2024-04-25T14:10:06.992992+0000","last_undegraded":"2024-04-25T14:10:06.992992+0000","last_fullsized":"2024-04-25T14:10:06.992992+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:10.182930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.757410+0000","last_change":"2024-04-25T14:10:04.749193+0000","last_active":"2024-04-25T14:10:06.757410+0000","last_peered":"2024-04-25T14:10:06.757410+0000","last_clean":"2024-04-25T14:10:06.757410+0000","last_became_active":"2024-04-25T14:10:04.747285+0000","last_became_peered":"2024-04-25T14:10:04.747285+0000","last_unstale":"2024-04-25T14:10:06.757410+0000","last_undegraded":"2024-04-25T14:10:06.757410+0000","last_fullsized":"2024-04-25T14:10:06.757410+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:23.181559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.729775+0000","last_change":"2024-04-25T14:10:05.757133+0000","last_active":"2024-04-25T14:10:06.729775+0000","last_peered":"2024-04-25T14:10:06.729775+0000","last_clean":"2024-04-25T14:10:06.729775+0000","last_became_active":"2024-04-25T14:10:05.756371+0000","last_became_peered":"2024-04-25T14:10:05.756371+0000","last_unstale":"2024-04-25T14:10:06.729775+0000","last_undegraded":"2024-04-25T14:10:06.729775+0000","last_fullsized":"2024-04-25T14:10:06.729775+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:09.731431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.729965+0000","last_change":"2024-04-25T14:10:04.734364+0000","last_active":"2024-04-25T14:10:06.729965+0000","last_peered":"2024-04-25T14:10:06.729965+0000","last_clean":"2024-04-25T14:10:06.729965+0000","last_became_active":"2024-04-25T14:10:04.734026+0000","last_became_peered":"2024-04-25T14:10:04.734026+0000","last_unstale":"2024-04-25T14:10:06.729965+0000","last_undegraded":"2024-04-25T14:10:06.729965+0000","last_fullsized":"2024-04-25T14:10:06.729965+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:25.644454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.730470+0000","last_change":"2024-04-25T14:10:05.758989+0000","last_active":"2024-04-25T14:10:06.730470+0000","last_peered":"2024-04-25T14:10:06.730470+0000","last_clean":"2024-04-25T14:10:06.730470+0000","last_became_active":"2024-04-25T14:10:05.755225+0000","last_became_peered":"2024-04-25T14:10:05.755225+0000","last_unstale":"2024-04-25T14:10:06.730470+0000","last_undegraded":"2024-04-25T14:10:06.730470+0000","last_fullsized":"2024-04-25T14:10:06.730470+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:40.982158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.757214+0000","last_change":"2024-04-25T14:10:04.745258+0000","last_active":"2024-04-25T14:10:06.757214+0000","last_peered":"2024-04-25T14:10:06.757214+0000","last_clean":"2024-04-25T14:10:06.757214+0000","last_became_active":"2024-04-25T14:10:04.744962+0000","last_became_peered":"2024-04-25T14:10:04.744962+0000","last_unstale":"2024-04-25T14:10:06.757214+0000","last_undegraded":"2024-04-25T14:10:06.757214+0000","last_fullsized":"2024-04-25T14:10:06.757214+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:15.128465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.754610+0000","last_change":"2024-04-25T14:10:05.752050+0000","last_active":"2024-04-25T14:10:06.754610+0000","last_peered":"2024-04-25T14:10:06.754610+0000","last_clean":"2024-04-25T14:10:06.754610+0000","last_became_active":"2024-04-25T14:10:05.751083+0000","last_became_peered":"2024-04-25T14:10:05.751083+0000","last_unstale":"2024-04-25T14:10:06.754610+0000","last_undegraded":"2024-04-25T14:10:06.754610+0000","last_fullsized":"2024-04-25T14:10:06.754610+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:45.173029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.754866+0000","last_change":"2024-04-25T14:10:04.750916+0000","last_active":"2024-04-25T14:10:06.754866+0000","last_peered":"2024-04-25T14:10:06.754866+0000","last_clean":"2024-04-25T14:10:06.754866+0000","last_became_active":"2024-04-25T14:10:04.750562+0000","last_became_peered":"2024-04-25T14:10:04.750562+0000","last_unstale":"2024-04-25T14:10:06.754866+0000","last_undegraded":"2024-04-25T14:10:06.754866+0000","last_fullsized":"2024-04-25T14:10:06.754866+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:06.771754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.730495+0000","last_change":"2024-04-25T14:10:05.759690+0000","last_active":"2024-04-25T14:10:06.730495+0000","last_peered":"2024-04-25T14:10:06.730495+0000","last_clean":"2024-04-25T14:10:06.730495+0000","last_became_active":"2024-04-25T14:10:05.757315+0000","last_became_peered":"2024-04-25T14:10:05.757315+0000","last_unstale":"2024-04-25T14:10:06.730495+0000","last_undegraded":"2024-04-25T14:10:06.730495+0000","last_fullsized":"2024-04-25T14:10:06.730495+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:06.710017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.730159+0000","last_change":"2024-04-25T14:10:04.746574+0000","last_active":"2024-04-25T14:10:06.730159+0000","last_peered":"2024-04-25T14:10:06.730159+0000","last_clean":"2024-04-25T14:10:06.730159+0000","last_became_active":"2024-04-25T14:10:04.746266+0000","last_became_peered":"2024-04-25T14:10:04.746266+0000","last_unstale":"2024-04-25T14:10:06.730159+0000","last_undegraded":"2024-04-25T14:10:06.730159+0000","last_fullsized":"2024-04-25T14:10:06.730159+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:54.546901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.730644+0000","last_change":"2024-04-25T14:10:05.760752+0000","last_active":"2024-04-25T14:10:06.730644+0000","last_peered":"2024-04-25T14:10:06.730644+0000","last_clean":"2024-04-25T14:10:06.730644+0000","last_became_active":"2024-04-25T14:10:05.760369+0000","last_became_peered":"2024-04-25T14:10:05.760369+0000","last_unstale":"2024-04-25T14:10:06.730644+0000","last_undegraded":"2024-04-25T14:10:06.730644+0000","last_fullsized":"2024-04-25T14:10:06.730644+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:19.451703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"112'3","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.730536+0000","last_change":"2024-04-25T14:10:04.745704+0000","last_active":"2024-04-25T14:10:06.730536+0000","last_peered":"2024-04-25T14:10:06.730536+0000","last_clean":"2024-04-25T14:10:06.730536+0000","last_became_active":"2024-04-25T14:10:04.745401+0000","last_became_peered":"2024-04-25T14:10:04.745401+0000","last_unstale":"2024-04-25T14:10:06.730536+0000","last_undegraded":"2024-04-25T14:10:06.730536+0000","last_fullsized":"2024-04-25T14:10:06.730536+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:45.118281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.730542+0000","last_change":"2024-04-25T14:10:05.749962+0000","last_active":"2024-04-25T14:10:06.730542+0000","last_peered":"2024-04-25T14:10:06.730542+0000","last_clean":"2024-04-25T14:10:06.730542+0000","last_became_active":"2024-04-25T14:10:05.749687+0000","last_became_peered":"2024-04-25T14:10:05.749687+0000","last_unstale":"2024-04-25T14:10:06.730542+0000","last_undegraded":"2024-04-25T14:10:06.730542+0000","last_fullsized":"2024-04-25T14:10:06.730542+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:19.318885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"112'6","reported_seq":20,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.730103+0000","last_change":"2024-04-25T14:10:04.745297+0000","last_active":"2024-04-25T14:10:06.730103+0000","last_peered":"2024-04-25T14:10:06.730103+0000","last_clean":"2024-04-25T14:10:06.730103+0000","last_became_active":"2024-04-25T14:10:04.744890+0000","last_became_peered":"2024-04-25T14:10:04.744890+0000","last_unstale":"2024-04-25T14:10:06.730103+0000","last_undegraded":"2024-04-25T14:10:06.730103+0000","last_fullsized":"2024-04-25T14:10:06.730103+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:09.176717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.754882+0000","last_change":"2024-04-25T14:10:05.758201+0000","last_active":"2024-04-25T14:10:06.754882+0000","last_peered":"2024-04-25T14:10:06.754882+0000","last_clean":"2024-04-25T14:10:06.754882+0000","last_became_active":"2024-04-25T14:10:05.757883+0000","last_became_peered":"2024-04-25T14:10:05.757883+0000","last_unstale":"2024-04-25T14:10:06.754882+0000","last_undegraded":"2024-04-25T14:10:06.754882+0000","last_fullsized":"2024-04-25T14:10:06.754882+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:40.982158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.730075+0000","last_change":"2024-04-25T14:10:04.746252+0000","last_active":"2024-04-25T14:10:06.730075+0000","last_peered":"2024-04-25T14:10:06.730075+0000","last_clean":"2024-04-25T14:10:06.730075+0000","last_became_active":"2024-04-25T14:10:04.745299+0000","last_became_peered":"2024-04-25T14:10:04.745299+0000","last_unstale":"2024-04-25T14:10:06.730075+0000","last_undegraded":"2024-04-25T14:10:06.730075+0000","last_fullsized":"2024-04-25T14:10:06.730075+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:44.165336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.730295+0000","last_change":"2024-04-25T14:10:05.746319+0000","last_active":"2024-04-25T14:10:06.730295+0000","last_peered":"2024-04-25T14:10:06.730295+0000","last_clean":"2024-04-25T14:10:06.730295+0000","last_became_active":"2024-04-25T14:10:05.745988+0000","last_became_peered":"2024-04-25T14:10:05.745988+0000","last_unstale":"2024-04-25T14:10:06.730295+0000","last_undegraded":"2024-04-25T14:10:06.730295+0000","last_fullsized":"2024-04-25T14:10:06.730295+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:13.696298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.729914+0000","last_change":"2024-04-25T14:10:04.740086+0000","last_active":"2024-04-25T14:10:06.729914+0000","last_peered":"2024-04-25T14:10:06.729914+0000","last_clean":"2024-04-25T14:10:06.729914+0000","last_became_active":"2024-04-25T14:10:04.739141+0000","last_became_peered":"2024-04-25T14:10:04.739141+0000","last_unstale":"2024-04-25T14:10:06.729914+0000","last_undegraded":"2024-04-25T14:10:06.729914+0000","last_fullsized":"2024-04-25T14:10:06.729914+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:42.659871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.993124+0000","last_change":"2024-04-25T14:10:05.754789+0000","last_active":"2024-04-25T14:10:06.993124+0000","last_peered":"2024-04-25T14:10:06.993124+0000","last_clean":"2024-04-25T14:10:06.993124+0000","last_became_active":"2024-04-25T14:10:05.753790+0000","last_became_peered":"2024-04-25T14:10:05.753790+0000","last_unstale":"2024-04-25T14:10:06.993124+0000","last_undegraded":"2024-04-25T14:10:06.993124+0000","last_fullsized":"2024-04-25T14:10:06.993124+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:03.110236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.729496+0000","last_change":"2024-04-25T14:10:04.743146+0000","last_active":"2024-04-25T14:10:06.729496+0000","last_peered":"2024-04-25T14:10:06.729496+0000","last_clean":"2024-04-25T14:10:06.729496+0000","last_became_active":"2024-04-25T14:10:04.742805+0000","last_became_peered":"2024-04-25T14:10:04.742805+0000","last_unstale":"2024-04-25T14:10:06.729496+0000","last_undegraded":"2024-04-25T14:10:06.729496+0000","last_fullsized":"2024-04-25T14:10:06.729496+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:22.606342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.730208+0000","last_change":"2024-04-25T14:10:05.742441+0000","last_active":"2024-04-25T14:10:06.730208+0000","last_peered":"2024-04-25T14:10:06.730208+0000","last_clean":"2024-04-25T14:10:06.730208+0000","last_became_active":"2024-04-25T14:10:05.741608+0000","last_became_peered":"2024-04-25T14:10:05.741608+0000","last_unstale":"2024-04-25T14:10:06.730208+0000","last_undegraded":"2024-04-25T14:10:06.730208+0000","last_fullsized":"2024-04-25T14:10:06.730208+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:45.173029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.730965+0000","last_change":"2024-04-25T14:10:04.751976+0000","last_active":"2024-04-25T14:10:06.730965+0000","last_peered":"2024-04-25T14:10:06.730965+0000","last_clean":"2024-04-25T14:10:06.730965+0000","last_became_active":"2024-04-25T14:10:04.751610+0000","last_became_peered":"2024-04-25T14:10:04.751610+0000","last_unstale":"2024-04-25T14:10:06.730965+0000","last_undegraded":"2024-04-25T14:10:06.730965+0000","last_fullsized":"2024-04-25T14:10:06.730965+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.238156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.993697+0000","last_change":"2024-04-25T14:10:05.765178+0000","last_active":"2024-04-25T14:10:06.993697+0000","last_peered":"2024-04-25T14:10:06.993697+0000","last_clean":"2024-04-25T14:10:06.993697+0000","last_became_active":"2024-04-25T14:10:05.763157+0000","last_became_peered":"2024-04-25T14:10:05.763157+0000","last_unstale":"2024-04-25T14:10:06.993697+0000","last_undegraded":"2024-04-25T14:10:06.993697+0000","last_fullsized":"2024-04-25T14:10:06.993697+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:19.318885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.993358+0000","last_change":"2024-04-25T14:10:04.749720+0000","last_active":"2024-04-25T14:10:06.993358+0000","last_peered":"2024-04-25T14:10:06.993358+0000","last_clean":"2024-04-25T14:10:06.993358+0000","last_became_active":"2024-04-25T14:10:04.749327+0000","last_became_peered":"2024-04-25T14:10:04.749327+0000","last_unstale":"2024-04-25T14:10:06.993358+0000","last_undegraded":"2024-04-25T14:10:06.993358+0000","last_fullsized":"2024-04-25T14:10:06.993358+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:09.014692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.757754+0000","last_change":"2024-04-25T14:10:05.761451+0000","last_active":"2024-04-25T14:10:06.757754+0000","last_peered":"2024-04-25T14:10:06.757754+0000","last_clean":"2024-04-25T14:10:06.757754+0000","last_became_active":"2024-04-25T14:10:05.760214+0000","last_became_peered":"2024-04-25T14:10:05.760214+0000","last_unstale":"2024-04-25T14:10:06.757754+0000","last_undegraded":"2024-04-25T14:10:06.757754+0000","last_fullsized":"2024-04-25T14:10:06.757754+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:24.980264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.994423+0000","last_change":"2024-04-25T14:10:04.750043+0000","last_active":"2024-04-25T14:10:06.994423+0000","last_peered":"2024-04-25T14:10:06.994423+0000","last_clean":"2024-04-25T14:10:06.994423+0000","last_became_active":"2024-04-25T14:10:04.749722+0000","last_became_peered":"2024-04-25T14:10:04.749722+0000","last_unstale":"2024-04-25T14:10:06.994423+0000","last_undegraded":"2024-04-25T14:10:06.994423+0000","last_fullsized":"2024-04-25T14:10:06.994423+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:22.606342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.730630+0000","last_change":"2024-04-25T14:10:05.746550+0000","last_active":"2024-04-25T14:10:06.730630+0000","last_peered":"2024-04-25T14:10:06.730630+0000","last_clean":"2024-04-25T14:10:06.730630+0000","last_became_active":"2024-04-25T14:10:05.745589+0000","last_became_peered":"2024-04-25T14:10:05.745589+0000","last_unstale":"2024-04-25T14:10:06.730630+0000","last_undegraded":"2024-04-25T14:10:06.730630+0000","last_fullsized":"2024-04-25T14:10:06.730630+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:27.170899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.994333+0000","last_change":"2024-04-25T14:10:04.754055+0000","last_active":"2024-04-25T14:10:06.994333+0000","last_peered":"2024-04-25T14:10:06.994333+0000","last_clean":"2024-04-25T14:10:06.994333+0000","last_became_active":"2024-04-25T14:10:04.753740+0000","last_became_peered":"2024-04-25T14:10:04.753740+0000","last_unstale":"2024-04-25T14:10:06.994333+0000","last_undegraded":"2024-04-25T14:10:06.994333+0000","last_fullsized":"2024-04-25T14:10:06.994333+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:01.747339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.757074+0000","last_change":"2024-04-25T14:10:05.757744+0000","last_active":"2024-04-25T14:10:06.757074+0000","last_peered":"2024-04-25T14:10:06.757074+0000","last_clean":"2024-04-25T14:10:06.757074+0000","last_became_active":"2024-04-25T14:10:05.757443+0000","last_became_peered":"2024-04-25T14:10:05.757443+0000","last_unstale":"2024-04-25T14:10:06.757074+0000","last_undegraded":"2024-04-25T14:10:06.757074+0000","last_fullsized":"2024-04-25T14:10:06.757074+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:34.847336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.757140+0000","last_change":"2024-04-25T14:10:04.753046+0000","last_active":"2024-04-25T14:10:06.757140+0000","last_peered":"2024-04-25T14:10:06.757140+0000","last_clean":"2024-04-25T14:10:06.757140+0000","last_became_active":"2024-04-25T14:10:04.749881+0000","last_became_peered":"2024-04-25T14:10:04.749881+0000","last_unstale":"2024-04-25T14:10:06.757140+0000","last_undegraded":"2024-04-25T14:10:06.757140+0000","last_fullsized":"2024-04-25T14:10:06.757140+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:54.706060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.729387+0000","last_change":"2024-04-25T14:10:05.750837+0000","last_active":"2024-04-25T14:10:06.729387+0000","last_peered":"2024-04-25T14:10:06.729387+0000","last_clean":"2024-04-25T14:10:06.729387+0000","last_became_active":"2024-04-25T14:10:05.744626+0000","last_became_peered":"2024-04-25T14:10:05.744626+0000","last_unstale":"2024-04-25T14:10:06.729387+0000","last_undegraded":"2024-04-25T14:10:06.729387+0000","last_fullsized":"2024-04-25T14:10:06.729387+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:15.229160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.993857+0000","last_change":"2024-04-25T14:10:04.749068+0000","last_active":"2024-04-25T14:10:06.993857+0000","last_peered":"2024-04-25T14:10:06.993857+0000","last_clean":"2024-04-25T14:10:06.993857+0000","last_became_active":"2024-04-25T14:10:04.748714+0000","last_became_peered":"2024-04-25T14:10:04.748714+0000","last_unstale":"2024-04-25T14:10:06.993857+0000","last_undegraded":"2024-04-25T14:10:06.993857+0000","last_fullsized":"2024-04-25T14:10:06.993857+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:52.358091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.729939+0000","last_change":"2024-04-25T14:10:05.763453+0000","last_active":"2024-04-25T14:10:06.729939+0000","last_peered":"2024-04-25T14:10:06.729939+0000","last_clean":"2024-04-25T14:10:06.729939+0000","last_became_active":"2024-04-25T14:10:05.763115+0000","last_became_peered":"2024-04-25T14:10:05.763115+0000","last_unstale":"2024-04-25T14:10:06.729939+0000","last_undegraded":"2024-04-25T14:10:06.729939+0000","last_fullsized":"2024-04-25T14:10:06.729939+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:26.049922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.729588+0000","last_change":"2024-04-25T14:10:04.746533+0000","last_active":"2024-04-25T14:10:06.729588+0000","last_peered":"2024-04-25T14:10:06.729588+0000","last_clean":"2024-04-25T14:10:06.729588+0000","last_became_active":"2024-04-25T14:10:04.746152+0000","last_became_peered":"2024-04-25T14:10:04.746152+0000","last_unstale":"2024-04-25T14:10:06.729588+0000","last_undegraded":"2024-04-25T14:10:06.729588+0000","last_fullsized":"2024-04-25T14:10:06.729588+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:06.771754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.730465+0000","last_change":"2024-04-25T14:10:05.750713+0000","last_active":"2024-04-25T14:10:06.730465+0000","last_peered":"2024-04-25T14:10:06.730465+0000","last_clean":"2024-04-25T14:10:06.730465+0000","last_became_active":"2024-04-25T14:10:05.750421+0000","last_became_peered":"2024-04-25T14:10:05.750421+0000","last_unstale":"2024-04-25T14:10:06.730465+0000","last_undegraded":"2024-04-25T14:10:06.730465+0000","last_fullsized":"2024-04-25T14:10:06.730465+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:40.982158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.993416+0000","last_change":"2024-04-25T14:10:04.752102+0000","last_active":"2024-04-25T14:10:06.993416+0000","last_peered":"2024-04-25T14:10:06.993416+0000","last_clean":"2024-04-25T14:10:06.993416+0000","last_became_active":"2024-04-25T14:10:04.751649+0000","last_became_peered":"2024-04-25T14:10:04.751649+0000","last_unstale":"2024-04-25T14:10:06.993416+0000","last_undegraded":"2024-04-25T14:10:06.993416+0000","last_fullsized":"2024-04-25T14:10:06.993416+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:05.444701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.757710+0000","last_change":"2024-04-25T14:10:05.752033+0000","last_active":"2024-04-25T14:10:06.757710+0000","last_peered":"2024-04-25T14:10:06.757710+0000","last_clean":"2024-04-25T14:10:06.757710+0000","last_became_active":"2024-04-25T14:10:05.751536+0000","last_became_peered":"2024-04-25T14:10:05.751536+0000","last_unstale":"2024-04-25T14:10:06.757710+0000","last_undegraded":"2024-04-25T14:10:06.757710+0000","last_fullsized":"2024-04-25T14:10:06.757710+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:48.038206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.757626+0000","last_change":"2024-04-25T14:10:04.749248+0000","last_active":"2024-04-25T14:10:06.757626+0000","last_peered":"2024-04-25T14:10:06.757626+0000","last_clean":"2024-04-25T14:10:06.757626+0000","last_became_active":"2024-04-25T14:10:04.748828+0000","last_became_peered":"2024-04-25T14:10:04.748828+0000","last_unstale":"2024-04-25T14:10:06.757626+0000","last_undegraded":"2024-04-25T14:10:06.757626+0000","last_fullsized":"2024-04-25T14:10:06.757626+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:06.938959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.730265+0000","last_change":"2024-04-25T14:10:05.771359+0000","last_active":"2024-04-25T14:10:06.730265+0000","last_peered":"2024-04-25T14:10:06.730265+0000","last_clean":"2024-04-25T14:10:06.730265+0000","last_became_active":"2024-04-25T14:10:05.770992+0000","last_became_peered":"2024-04-25T14:10:05.770992+0000","last_unstale":"2024-04-25T14:10:06.730265+0000","last_undegraded":"2024-04-25T14:10:06.730265+0000","last_fullsized":"2024-04-25T14:10:06.730265+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:45.173029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"113'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.754863+0000","last_change":"2024-04-25T14:10:04.746371+0000","last_active":"2024-04-25T14:10:06.754863+0000","last_peered":"2024-04-25T14:10:06.754863+0000","last_clean":"2024-04-25T14:10:06.754863+0000","last_became_active":"2024-04-25T14:10:04.745733+0000","last_became_peered":"2024-04-25T14:10:04.745733+0000","last_unstale":"2024-04-25T14:10:06.754863+0000","last_undegraded":"2024-04-25T14:10:06.754863+0000","last_fullsized":"2024-04-25T14:10:06.754863+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:38.191463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.994304+0000","last_change":"2024-04-25T14:10:05.764083+0000","last_active":"2024-04-25T14:10:06.994304+0000","last_peered":"2024-04-25T14:10:06.994304+0000","last_clean":"2024-04-25T14:10:06.994304+0000","last_became_active":"2024-04-25T14:10:05.762497+0000","last_became_peered":"2024-04-25T14:10:05.762497+0000","last_unstale":"2024-04-25T14:10:06.994304+0000","last_undegraded":"2024-04-25T14:10:06.994304+0000","last_fullsized":"2024-04-25T14:10:06.994304+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:27.170899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.754786+0000","last_change":"2024-04-25T14:10:04.747682+0000","last_active":"2024-04-25T14:10:06.754786+0000","last_peered":"2024-04-25T14:10:06.754786+0000","last_clean":"2024-04-25T14:10:06.754786+0000","last_became_active":"2024-04-25T14:10:04.747380+0000","last_became_peered":"2024-04-25T14:10:04.747380+0000","last_unstale":"2024-04-25T14:10:06.754786+0000","last_undegraded":"2024-04-25T14:10:06.754786+0000","last_fullsized":"2024-04-25T14:10:06.754786+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:09.176717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.993340+0000","last_change":"2024-04-25T14:10:05.758465+0000","last_active":"2024-04-25T14:10:06.993340+0000","last_peered":"2024-04-25T14:10:06.993340+0000","last_clean":"2024-04-25T14:10:06.993340+0000","last_became_active":"2024-04-25T14:10:05.756747+0000","last_became_peered":"2024-04-25T14:10:05.756747+0000","last_unstale":"2024-04-25T14:10:06.993340+0000","last_undegraded":"2024-04-25T14:10:06.993340+0000","last_fullsized":"2024-04-25T14:10:06.993340+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:35.698824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.757621+0000","last_change":"2024-04-25T14:10:04.752523+0000","last_active":"2024-04-25T14:10:06.757621+0000","last_peered":"2024-04-25T14:10:06.757621+0000","last_clean":"2024-04-25T14:10:06.757621+0000","last_became_active":"2024-04-25T14:10:04.750781+0000","last_became_peered":"2024-04-25T14:10:04.750781+0000","last_unstale":"2024-04-25T14:10:06.757621+0000","last_undegraded":"2024-04-25T14:10:06.757621+0000","last_fullsized":"2024-04-25T14:10:06.757621+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:32.248751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.757240+0000","last_change":"2024-04-25T14:10:05.739561+0000","last_active":"2024-04-25T14:10:06.757240+0000","last_peered":"2024-04-25T14:10:06.757240+0000","last_clean":"2024-04-25T14:10:06.757240+0000","last_became_active":"2024-04-25T14:10:05.739231+0000","last_became_peered":"2024-04-25T14:10:05.739231+0000","last_unstale":"2024-04-25T14:10:06.757240+0000","last_undegraded":"2024-04-25T14:10:06.757240+0000","last_fullsized":"2024-04-25T14:10:06.757240+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:41.062002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.993458+0000","last_change":"2024-04-25T14:10:04.746161+0000","last_active":"2024-04-25T14:10:06.993458+0000","last_peered":"2024-04-25T14:10:06.993458+0000","last_clean":"2024-04-25T14:10:06.993458+0000","last_became_active":"2024-04-25T14:10:04.745423+0000","last_became_peered":"2024-04-25T14:10:04.745423+0000","last_unstale":"2024-04-25T14:10:06.993458+0000","last_undegraded":"2024-04-25T14:10:06.993458+0000","last_fullsized":"2024-04-25T14:10:06.993458+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:50.427076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.729472+0000","last_change":"2024-04-25T14:10:05.760842+0000","last_active":"2024-04-25T14:10:06.729472+0000","last_peered":"2024-04-25T14:10:06.729472+0000","last_clean":"2024-04-25T14:10:06.729472+0000","last_became_active":"2024-04-25T14:10:05.760432+0000","last_became_peered":"2024-04-25T14:10:05.760432+0000","last_unstale":"2024-04-25T14:10:06.729472+0000","last_undegraded":"2024-04-25T14:10:06.729472+0000","last_fullsized":"2024-04-25T14:10:06.729472+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:18.890427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.993060+0000","last_change":"2024-04-25T14:10:04.742349+0000","last_active":"2024-04-25T14:10:06.993060+0000","last_peered":"2024-04-25T14:10:06.993060+0000","last_clean":"2024-04-25T14:10:06.993060+0000","last_became_active":"2024-04-25T14:10:04.741292+0000","last_became_peered":"2024-04-25T14:10:04.741292+0000","last_unstale":"2024-04-25T14:10:06.993060+0000","last_undegraded":"2024-04-25T14:10:06.993060+0000","last_fullsized":"2024-04-25T14:10:06.993060+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:52.358091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.993942+0000","last_change":"2024-04-25T14:10:05.748767+0000","last_active":"2024-04-25T14:10:06.993942+0000","last_peered":"2024-04-25T14:10:06.993942+0000","last_clean":"2024-04-25T14:10:06.993942+0000","last_became_active":"2024-04-25T14:10:05.747683+0000","last_became_peered":"2024-04-25T14:10:05.747683+0000","last_unstale":"2024-04-25T14:10:06.993942+0000","last_undegraded":"2024-04-25T14:10:06.993942+0000","last_fullsized":"2024-04-25T14:10:06.993942+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:06.710017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.757274+0000","last_change":"2024-04-25T14:10:04.749122+0000","last_active":"2024-04-25T14:10:06.757274+0000","last_peered":"2024-04-25T14:10:06.757274+0000","last_clean":"2024-04-25T14:10:06.757274+0000","last_became_active":"2024-04-25T14:10:04.748630+0000","last_became_peered":"2024-04-25T14:10:04.748630+0000","last_unstale":"2024-04-25T14:10:06.757274+0000","last_undegraded":"2024-04-25T14:10:06.757274+0000","last_fullsized":"2024-04-25T14:10:06.757274+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:54.581504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.730585+0000","last_change":"2024-04-25T14:10:05.749103+0000","last_active":"2024-04-25T14:10:06.730585+0000","last_peered":"2024-04-25T14:10:06.730585+0000","last_clean":"2024-04-25T14:10:06.730585+0000","last_became_active":"2024-04-25T14:10:05.748822+0000","last_became_peered":"2024-04-25T14:10:05.748822+0000","last_unstale":"2024-04-25T14:10:06.730585+0000","last_undegraded":"2024-04-25T14:10:06.730585+0000","last_fullsized":"2024-04-25T14:10:06.730585+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:19.318885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.993431+0000","last_change":"2024-04-25T14:10:04.745286+0000","last_active":"2024-04-25T14:10:06.993431+0000","last_peered":"2024-04-25T14:10:06.993431+0000","last_clean":"2024-04-25T14:10:06.993431+0000","last_became_active":"2024-04-25T14:10:04.744949+0000","last_became_peered":"2024-04-25T14:10:04.744949+0000","last_unstale":"2024-04-25T14:10:06.993431+0000","last_undegraded":"2024-04-25T14:10:06.993431+0000","last_fullsized":"2024-04-25T14:10:06.993431+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:39.794816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.994400+0000","last_change":"2024-04-25T14:10:05.755078+0000","last_active":"2024-04-25T14:10:06.994400+0000","last_peered":"2024-04-25T14:10:06.994400+0000","last_clean":"2024-04-25T14:10:06.994400+0000","last_became_active":"2024-04-25T14:10:05.754511+0000","last_became_peered":"2024-04-25T14:10:05.754511+0000","last_unstale":"2024-04-25T14:10:06.994400+0000","last_undegraded":"2024-04-25T14:10:06.994400+0000","last_fullsized":"2024-04-25T14:10:06.994400+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:04.713569+0000","last_clean_scrub_stamp":"2024-04-25T14:10:04.713569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:17.071024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"112'8","reported_seq":22,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.729311+0000","last_change":"2024-04-25T14:10:04.747339+0000","last_active":"2024-04-25T14:10:06.729311+0000","last_peered":"2024-04-25T14:10:06.729311+0000","last_clean":"2024-04-25T14:10:06.729311+0000","last_became_active":"2024-04-25T14:10:04.745494+0000","last_became_peered":"2024-04-25T14:10:04.745494+0000","last_unstale":"2024-04-25T14:10:06.729311+0000","last_undegraded":"2024-04-25T14:10:06.729311+0000","last_fullsized":"2024-04-25T14:10:06.729311+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:03.705876+0000","last_clean_scrub_stamp":"2024-04-25T14:10:03.705876+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:23.410960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"102'119","reported_seq":449,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T14:10:06.993579+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:10:06.993579+0000","last_peered":"2024-04-25T14:10:06.993579+0000","last_clean":"2024-04-25T14:10:06.993579+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:10:06.993579+0000","last_undegraded":"2024-04-25T14:10:06.993579+0000","last_fullsized":"2024-04-25T14:10:06.993579+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":468,"num_read_kb":1272,"num_write":181,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":468,"num_read_kb":1272,"num_write":181,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574924,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33160,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338680,"statfs":{"total":96636764160,"available":96602808320,"internally_reserved":0,"allocated":2019328,"data_stored":1294629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1413120,"data_stored":704067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574926,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1417216,"data_stored":703645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574925,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1421312,"data_stored":705097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574925,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1425408,"data_stored":705131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1409024,"data_stored":703485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1413120,"data_stored":703507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33140,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338700,"statfs":{"total":96636764160,"available":96602828800,"internally_reserved":0,"allocated":2007040,"data_stored":1293875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:10:14.821 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T14:10:14.821 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T14:10:15.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.013+0000 7f5eb9000640 1 Processor -- start 2024-04-25T14:10:15.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.013+0000 7f5eb9000640 1 -- start start 2024-04-25T14:10:15.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.017+0000 7f5eb9000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5eb4072880 0x7f5eb40af480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.017+0000 7f5eb9000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5eb4070350 0x7f5eb406d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.017+0000 7f5eb9000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5eb415c2c0 con 0x7f5eb40af9c0 2024-04-25T14:10:15.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.017+0000 7f5eb9000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5eb40afdd0 con 0x7f5eb4072880 2024-04-25T14:10:15.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.017+0000 7f5eb9000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f5eb406f7e0 con 0x7f5eb4070350 2024-04-25T14:10:15.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.017+0000 7f5eb3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5eb4072880 0x7f5eb40af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.017+0000 7f5eb3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5eb4072880 0x7f5eb40af480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54006/0 (socket says 172.21.15.86:54006) 2024-04-25T14:10:15.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.017+0000 7f5eaaa00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5eb40af9c0 0x7f5eb406fc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51904/0 (socket says 172.21.15.86:51904) 2024-04-25T14:10:15.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.017+0000 7f5eb3400640 1 -- 172.21.15.86:0/3621561287 learned_addr learned my addr 172.21.15.86:0/3621561287 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:15.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.017+0000 7f5eb3e00640 1 --2- 172.21.15.86:0/3621561287 >> v2:172.21.15.155:3300/0 conn(0x7f5eb4070350 0x7f5eb406d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.017+0000 7f5eb3400640 1 -- 172.21.15.86:0/3621561287 >> v2:172.21.15.155:3300/0 conn(0x7f5eb4070350 msgr2=0x7f5eb406d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:15.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.017+0000 7f5eb3400640 1 --2- 172.21.15.86:0/3621561287 >> v2:172.21.15.155:3300/0 conn(0x7f5eb4070350 0x7f5eb406d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.017+0000 7f5eb3400640 1 -- 172.21.15.86:0/3621561287 >> v1:172.21.15.86:6789/0 conn(0x7f5eb40af9c0 legacy=0x7f5eb406fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:15.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.017+0000 7f5eb3400640 1 -- 172.21.15.86:0/3621561287 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb40afff0 con 0x7f5eb4072880 2024-04-25T14:10:15.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.021+0000 7f5eb3400640 1 --2- 172.21.15.86:0/3621561287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eb4072880 0x7f5eb40af480 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f5eb406edf0 tx=0x7f5ea401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd526917e9254 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:15.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.021+0000 7f5eb2a00640 1 -- 172.21.15.86:0/3621561287 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea40aa650 con 0x7f5eb4072880 2024-04-25T14:10:15.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.021+0000 7f5eb2a00640 1 -- 172.21.15.86:0/3621561287 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea40af070 con 0x7f5eb4072880 2024-04-25T14:10:15.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.021+0000 7f5eb2a00640 1 -- 172.21.15.86:0/3621561287 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea40aaa10 con 0x7f5eb4072880 2024-04-25T14:10:15.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.021+0000 7f5eb9000640 1 -- 172.21.15.86:0/3621561287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eb4072880 msgr2=0x7f5eb40af480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:15.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.021+0000 7f5eb9000640 1 --2- 172.21.15.86:0/3621561287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eb4072880 0x7f5eb40af480 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f5eb406edf0 tx=0x7f5ea401c4f0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.021+0000 7f5eb9000640 1 -- 172.21.15.86:0/3621561287 shutdown_connections 2024-04-25T14:10:15.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.021+0000 7f5eb9000640 1 --2- 172.21.15.86:0/3621561287 >> v2:172.21.15.155:3300/0 conn(0x7f5eb4070350 0x7f5eb406d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.021+0000 7f5eb9000640 1 --2- 172.21.15.86:0/3621561287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eb4072880 0x7f5eb40af480 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.021+0000 7f5eb9000640 1 -- 172.21.15.86:0/3621561287 >> 172.21.15.86:0/3621561287 conn(0x7f5eb40b4740 msgr2=0x7f5eb40b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:15.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.021+0000 7f5eb9000640 1 -- 172.21.15.86:0/3621561287 shutdown_connections 2024-04-25T14:10:15.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.021+0000 7f5eb9000640 1 -- 172.21.15.86:0/3621561287 wait complete. 2024-04-25T14:10:15.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.025+0000 7f5eb9000640 1 Processor -- start 2024-04-25T14:10:15.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.025+0000 7f5eb9000640 1 -- start start 2024-04-25T14:10:15.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.025+0000 7f5eb9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eb4072880 0x7f5eb4098d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.025+0000 7f5eb9000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5eb40af9c0 0x7f5eb4152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.025+0000 7f5eb9000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5eb41534f0 0x7f5eb414c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.025+0000 7f5eb9000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb4072c90 con 0x7f5eb4072880 2024-04-25T14:10:15.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.025+0000 7f5eb9000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb406cf10 con 0x7f5eb41534f0 2024-04-25T14:10:15.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.025+0000 7f5eb9000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5eb4072e10 con 0x7f5eb40af9c0 2024-04-25T14:10:15.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.025+0000 7f5eb3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eb4072880 0x7f5eb4098d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.025+0000 7f5eb3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eb4072880 0x7f5eb4098d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54010/0 (socket says 172.21.15.86:54010) 2024-04-25T14:10:15.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.025+0000 7f5eb3400640 1 -- 172.21.15.86:0/97560899 learned_addr learned my addr 172.21.15.86:0/97560899 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:15.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.025+0000 7f5eaaa00640 1 --2- 172.21.15.86:0/97560899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5eb40af9c0 0x7f5eb4152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.025+0000 7f5eb3e00640 1 --2- 172.21.15.86:0/97560899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5eb41534f0 0x7f5eb414c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.025+0000 7f5eb3400640 1 -- 172.21.15.86:0/97560899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5eb40af9c0 msgr2=0x7f5eb4152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:15.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.025+0000 7f5eb3400640 1 --2- 172.21.15.86:0/97560899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5eb40af9c0 0x7f5eb4152fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.025+0000 7f5eb3400640 1 -- 172.21.15.86:0/97560899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5eb41534f0 msgr2=0x7f5eb414c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:15.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.025+0000 7f5eb3400640 1 --2- 172.21.15.86:0/97560899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5eb41534f0 0x7f5eb414c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.025+0000 7f5eb3400640 1 -- 172.21.15.86:0/97560899 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb40b2020 con 0x7f5eb4072880 2024-04-25T14:10:15.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.025+0000 7f5eb3e00640 1 --2- 172.21.15.86:0/97560899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5eb41534f0 0x7f5eb414c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:15.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.025+0000 7f5eb3400640 1 --2- 172.21.15.86:0/97560899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eb4072880 0x7f5eb4098d80 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40ac040 tx=0x7f5ea401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:15.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.025+0000 7f5eabe00640 1 -- 172.21.15.86:0/97560899 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea40bd5f0 con 0x7f5eb4072880 2024-04-25T14:10:15.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.025+0000 7f5eabe00640 1 -- 172.21.15.86:0/97560899 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea40a6040 con 0x7f5eb4072880 2024-04-25T14:10:15.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.025+0000 7f5eb9000640 1 -- 172.21.15.86:0/97560899 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb4153900 con 0x7f5eb4072880 2024-04-25T14:10:15.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.025+0000 7f5eabe00640 1 -- 172.21.15.86:0/97560899 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea40bd9d0 con 0x7f5eb4072880 2024-04-25T14:10:15.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.025+0000 7f5eb9000640 1 -- 172.21.15.86:0/97560899 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb409aaa0 con 0x7f5eb4072880 2024-04-25T14:10:15.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.029+0000 7f5eb9000640 1 -- 172.21.15.86:0/97560899 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eb4076b80 con 0x7f5eb4072880 2024-04-25T14:10:15.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.029+0000 7f5eabe00640 1 -- 172.21.15.86:0/97560899 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7f5ea40b33f0 con 0x7f5eb4072880 2024-04-25T14:10:15.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.029+0000 7f5eabe00640 1 --2- 172.21.15.86:0/97560899 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f5e9006f550 0x7f5e90071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.029+0000 7f5eabe00640 1 -- 172.21.15.86:0/97560899 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f5ea4129780 con 0x7f5eb4072880 2024-04-25T14:10:15.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.029+0000 7f5eaaa00640 1 --2- 172.21.15.86:0/97560899 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f5e9006f550 0x7f5e90071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.029+0000 7f5eaaa00640 1 --2- 172.21.15.86:0/97560899 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f5e9006f550 0x7f5e90071a60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40b1d10 tx=0x7f5e9c007cf0 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:15.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.033+0000 7f5eabe00640 1 -- 172.21.15.86:0/97560899 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5eb4153900 con 0x7f5eb4072880 2024-04-25T14:10:15.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.245+0000 7f5eb9000640 1 -- 172.21.15.86:0/97560899 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f5eb406f2c0 con 0x7f5eb4072880 2024-04-25T14:10:15.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.249+0000 7f5eabe00640 1 -- 172.21.15.86:0/97560899 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v27) ==== 190+0+147 (secure 0 0 0) 0x7f5ea40c3060 con 0x7f5eb4072880 2024-04-25T14:10:15.247 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-04-25T14:10:15.247 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCLYipm5u4xHRAACDFe3Eb4KLrsdLj4eFe7mA== 2024-04-25T14:10:15.247 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-04-25T14:10:15.247 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-04-25T14:10:15.247 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T14:10:15.247 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.0 2024-04-25T14:10:15.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.253+0000 7f5eb9000640 1 -- 172.21.15.86:0/97560899 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f5e9006f550 msgr2=0x7f5e90071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:15.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.253+0000 7f5eb9000640 1 --2- 172.21.15.86:0/97560899 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f5e9006f550 0x7f5e90071a60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40b1d10 tx=0x7f5e9c007cf0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.253+0000 7f5eb9000640 1 -- 172.21.15.86:0/97560899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eb4072880 msgr2=0x7f5eb4098d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:15.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.253+0000 7f5eb9000640 1 --2- 172.21.15.86:0/97560899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eb4072880 0x7f5eb4098d80 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40ac040 tx=0x7f5ea401ad30 comp rx=0 tx=0).stop 2024-04-25T14:10:15.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.253+0000 7f5eb9000640 1 -- 172.21.15.86:0/97560899 shutdown_connections 2024-04-25T14:10:15.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.253+0000 7f5eb9000640 1 --2- 172.21.15.86:0/97560899 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f5e9006f550 0x7f5e90071a60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.253+0000 7f5eb9000640 1 --2- 172.21.15.86:0/97560899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5eb41534f0 0x7f5eb414c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.253+0000 7f5eb9000640 1 --2- 172.21.15.86:0/97560899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5eb40af9c0 0x7f5eb4152fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.253+0000 7f5eb9000640 1 --2- 172.21.15.86:0/97560899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eb4072880 0x7f5eb4098d80 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.253+0000 7f5eb9000640 1 -- 172.21.15.86:0/97560899 >> 172.21.15.86:0/97560899 conn(0x7f5eb40b4740 msgr2=0x7f5eb4073640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:15.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.253+0000 7f5eb9000640 1 -- 172.21.15.86:0/97560899 shutdown_connections 2024-04-25T14:10:15.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.253+0000 7f5eb9000640 1 -- 172.21.15.86:0/97560899 wait complete. 2024-04-25T14:10:15.269 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:10:15.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.417+0000 7f56a9a00640 1 Processor -- start 2024-04-25T14:10:15.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.417+0000 7f56a9a00640 1 -- start start 2024-04-25T14:10:15.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.421+0000 7f56a9a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f56a40cfca0 0x7f56a406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.421+0000 7f56a9a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f56a40730b0 0x7f56a40734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.421+0000 7f56a9a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56a4072bd0 con 0x7f56a40d16e0 2024-04-25T14:10:15.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.421+0000 7f56a9a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f56a4070470 con 0x7f56a40cfca0 2024-04-25T14:10:15.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.421+0000 7f56a9a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f56a406f930 con 0x7f56a40730b0 2024-04-25T14:10:15.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.421+0000 7f56a2a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f56a40cfca0 0x7f56a406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.421+0000 7f56a2a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f56a40cfca0 0x7f56a406f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54022/0 (socket says 172.21.15.86:54022) 2024-04-25T14:10:15.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.421+0000 7f56a2a00640 1 -- 172.21.15.86:0/3240832982 learned_addr learned my addr 172.21.15.86:0/3240832982 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:15.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.421+0000 7f569a000640 1 --1- 172.21.15.86:0/3240832982 >> v1:172.21.15.86:6789/0 conn(0x7f56a40d16e0 0x7f56a40d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51918/0 (socket says 172.21.15.86:51918) 2024-04-25T14:10:15.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.421+0000 7f56a3400640 1 --2- 172.21.15.86:0/3240832982 >> v2:172.21.15.155:3300/0 conn(0x7f56a40730b0 0x7f56a40734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.421+0000 7f56a2a00640 1 -- 172.21.15.86:0/3240832982 >> v2:172.21.15.155:3300/0 conn(0x7f56a40730b0 msgr2=0x7f56a40734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:15.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.421+0000 7f56a2a00640 1 --2- 172.21.15.86:0/3240832982 >> v2:172.21.15.155:3300/0 conn(0x7f56a40730b0 0x7f56a40734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.421+0000 7f56a2a00640 1 -- 172.21.15.86:0/3240832982 >> v1:172.21.15.86:6789/0 conn(0x7f56a40d16e0 legacy=0x7f56a40d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:15.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.421+0000 7f56a2a00640 1 -- 172.21.15.86:0/3240832982 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a406fbd0 con 0x7f56a40cfca0 2024-04-25T14:10:15.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.425+0000 7f56a2a00640 1 --2- 172.21.15.86:0/3240832982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a40cfca0 0x7f56a406f3f0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f56a40cb8c0 tx=0x7f569401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213e68250b55af18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:15.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.425+0000 7f56a2000640 1 -- 172.21.15.86:0/3240832982 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56940ab430 con 0x7f56a40cfca0 2024-04-25T14:10:15.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.425+0000 7f56a2000640 1 -- 172.21.15.86:0/3240832982 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56940b0070 con 0x7f56a40cfca0 2024-04-25T14:10:15.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.425+0000 7f56a2000640 1 -- 172.21.15.86:0/3240832982 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56940ab7f0 con 0x7f56a40cfca0 2024-04-25T14:10:15.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.425+0000 7f56a9a00640 1 -- 172.21.15.86:0/3240832982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a40cfca0 msgr2=0x7f56a406f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:15.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.425+0000 7f56a9a00640 1 --2- 172.21.15.86:0/3240832982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a40cfca0 0x7f56a406f3f0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f56a40cb8c0 tx=0x7f569401c520 comp rx=0 tx=0).stop 2024-04-25T14:10:15.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.425+0000 7f56a9a00640 1 -- 172.21.15.86:0/3240832982 shutdown_connections 2024-04-25T14:10:15.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.425+0000 7f56a9a00640 1 --2- 172.21.15.86:0/3240832982 >> v2:172.21.15.155:3300/0 conn(0x7f56a40730b0 0x7f56a40734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.425+0000 7f56a9a00640 1 --2- 172.21.15.86:0/3240832982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a40cfca0 0x7f56a406f3f0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.425+0000 7f56a9a00640 1 -- 172.21.15.86:0/3240832982 >> 172.21.15.86:0/3240832982 conn(0x7f56a40b4650 msgr2=0x7f56a40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:15.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.425+0000 7f56a9a00640 1 -- 172.21.15.86:0/3240832982 shutdown_connections 2024-04-25T14:10:15.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.425+0000 7f56a9a00640 1 -- 172.21.15.86:0/3240832982 wait complete. 2024-04-25T14:10:15.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.425+0000 7f56a9a00640 1 Processor -- start 2024-04-25T14:10:15.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.425+0000 7f56a9a00640 1 -- start start 2024-04-25T14:10:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.429+0000 7f56a9a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56a40730b0 0x7f56a4157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.429+0000 7f56a9a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56a40cfca0 0x7f56a4159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.429+0000 7f56a9a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a40d16e0 0x7f56a415a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.429+0000 7f56a9a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56a4167a60 con 0x7f56a40d16e0 2024-04-25T14:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.429+0000 7f56a9a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f56a41678e0 con 0x7f56a40cfca0 2024-04-25T14:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.429+0000 7f56a9a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f56a4167be0 con 0x7f56a40730b0 2024-04-25T14:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.429+0000 7f56a3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a40d16e0 0x7f56a415a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.429+0000 7f56a3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a40d16e0 0x7f56a415a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54038/0 (socket says 172.21.15.86:54038) 2024-04-25T14:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.429+0000 7f56a3400640 1 -- 172.21.15.86:0/3268425085 learned_addr learned my addr 172.21.15.86:0/3268425085 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.429+0000 7f569a000640 1 --2- 172.21.15.86:0/3268425085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56a40cfca0 0x7f56a4159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.429+0000 7f56a2a00640 1 --2- 172.21.15.86:0/3268425085 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56a40730b0 0x7f56a4157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.429+0000 7f56a3400640 1 -- 172.21.15.86:0/3268425085 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56a40730b0 msgr2=0x7f56a4157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.429+0000 7f56a3400640 1 --2- 172.21.15.86:0/3268425085 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56a40730b0 0x7f56a4157f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.429+0000 7f56a3400640 1 -- 172.21.15.86:0/3268425085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56a40cfca0 msgr2=0x7f56a4159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.429+0000 7f56a3400640 1 --2- 172.21.15.86:0/3268425085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56a40cfca0 0x7f56a4159fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.429+0000 7f56a3400640 1 -- 172.21.15.86:0/3268425085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a415c150 con 0x7f56a40d16e0 2024-04-25T14:10:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.429+0000 7f569a000640 1 --2- 172.21.15.86:0/3268425085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56a40cfca0 0x7f56a4159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.429+0000 7f56a2a00640 1 --2- 172.21.15.86:0/3268425085 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56a40730b0 0x7f56a4157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.429+0000 7f56a3400640 1 --2- 172.21.15.86:0/3268425085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a40d16e0 0x7f56a415a500 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f569c009fd0 tx=0x7f569c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.429+0000 7f569b400640 1 -- 172.21.15.86:0/3268425085 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c014420 con 0x7f56a40d16e0 2024-04-25T14:10:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.429+0000 7f569b400640 1 -- 172.21.15.86:0/3268425085 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f569c019070 con 0x7f56a40d16e0 2024-04-25T14:10:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.429+0000 7f56a9a00640 1 -- 172.21.15.86:0/3268425085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a40cae90 con 0x7f56a40d16e0 2024-04-25T14:10:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.429+0000 7f569b400640 1 -- 172.21.15.86:0/3268425085 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c014800 con 0x7f56a40d16e0 2024-04-25T14:10:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.429+0000 7f56a9a00640 1 -- 172.21.15.86:0/3268425085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a415be70 con 0x7f56a40d16e0 2024-04-25T14:10:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.429+0000 7f56a9a00640 1 -- 172.21.15.86:0/3268425085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56a409f4b0 con 0x7f56a40d16e0 2024-04-25T14:10:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.433+0000 7f569b400640 1 -- 172.21.15.86:0/3268425085 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7f569c0149a0 con 0x7f56a40d16e0 2024-04-25T14:10:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.433+0000 7f569b400640 1 --2- 172.21.15.86:0/3268425085 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f568006f550 0x7f5680071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.433+0000 7f569b400640 1 -- 172.21.15.86:0/3268425085 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f569c08fe40 con 0x7f56a40d16e0 2024-04-25T14:10:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.433+0000 7f56a2a00640 1 --2- 172.21.15.86:0/3268425085 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f568006f550 0x7f5680071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.433+0000 7f56a2a00640 1 --2- 172.21.15.86:0/3268425085 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f568006f550 0x7f5680071a60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f56940af000 tx=0x7f5694008040 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.437+0000 7f569b400640 1 -- 172.21.15.86:0/3268425085 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f569c05c9f0 con 0x7f56a40d16e0 2024-04-25T14:10:15.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.629+0000 7f56a9a00640 1 -- 172.21.15.86:0/3268425085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56a4069d00 con 0x7f56a40d16e0 2024-04-25T14:10:15.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.629+0000 7f569b400640 1 -- 172.21.15.86:0/3268425085 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5183 (secure 0 0 0) 0x7f569c05c2f0 con 0x7f56a40d16e0 2024-04-25T14:10:15.629 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:15.629 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":85,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/374465657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":374465657},{"type":"v1","addr":"172.21.15.86:6836","nonce":374465657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8107,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1466195534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":1466195534},{"type":"v1","addr":"172.21.15.86:6837","nonce":1466195534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8119,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2431580070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2431580070},{"type":"v1","addr":"172.21.15.155:6835","nonce":2431580070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:10:05.129470+0000","modified":"2024-04-25T14:10:08.145259+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8457":{"gid":8457,"name":"b","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6833/142240252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":142240252},{"type":"v1","addr":"172.21.15.155:6833","nonce":142240252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8457,"qdb_cluster":[8457]},"id":8}]} 2024-04-25T14:10:15.629 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 85 2024-04-25T14:10:15.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.633+0000 7f56a9a00640 1 -- 172.21.15.86:0/3268425085 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f568006f550 msgr2=0x7f5680071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:15.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.633+0000 7f56a9a00640 1 --2- 172.21.15.86:0/3268425085 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f568006f550 0x7f5680071a60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f56940af000 tx=0x7f5694008040 comp rx=0 tx=0).stop 2024-04-25T14:10:15.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.633+0000 7f56a9a00640 1 -- 172.21.15.86:0/3268425085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a40d16e0 msgr2=0x7f56a415a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:15.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.633+0000 7f56a9a00640 1 --2- 172.21.15.86:0/3268425085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a40d16e0 0x7f56a415a500 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f569c009fd0 tx=0x7f569c00e4b0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.633+0000 7f56a9a00640 1 -- 172.21.15.86:0/3268425085 shutdown_connections 2024-04-25T14:10:15.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.633+0000 7f56a9a00640 1 --2- 172.21.15.86:0/3268425085 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f568006f550 0x7f5680071a60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.633+0000 7f56a9a00640 1 --2- 172.21.15.86:0/3268425085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a40d16e0 0x7f56a415a500 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.633+0000 7f56a9a00640 1 --2- 172.21.15.86:0/3268425085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56a40cfca0 0x7f56a4159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.633+0000 7f56a9a00640 1 --2- 172.21.15.86:0/3268425085 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56a40730b0 0x7f56a4157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.633+0000 7f56a9a00640 1 -- 172.21.15.86:0/3268425085 >> 172.21.15.86:0/3268425085 conn(0x7f56a40b4650 msgr2=0x7f56a40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:15.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.633+0000 7f56a9a00640 1 -- 172.21.15.86:0/3268425085 shutdown_connections 2024-04-25T14:10:15.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:15.633+0000 7f56a9a00640 1 -- 172.21.15.86:0/3268425085 wait complete. 2024-04-25T14:10:16.053 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 85, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:10:05.129470+0000', 'modified': '2024-04-25T14:10:08.145259+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8457}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8457': {'gid': 8457, 'name': 'b', 'rank': 0, 'incarnation': 81, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.155:6833/142240252', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6832', 'nonce': 142240252}, {'type': 'v1', 'addr': '172.21.15.155:6833', 'nonce': 142240252}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8457, 'qdb_cluster': [8457]} 2024-04-25T14:10:16.053 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T14:10:16.054 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T14:10:16.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.205+0000 7efdedc00640 1 Processor -- start 2024-04-25T14:10:16.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.205+0000 7efdedc00640 1 -- start start 2024-04-25T14:10:16.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.205+0000 7efdedc00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7efde80d2940 0x7efde80b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:16.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.205+0000 7efdedc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efde80b05a0 0x7efde80b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:16.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.205+0000 7efdedc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efde8160340 con 0x7efde80b78c0 2024-04-25T14:10:16.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.205+0000 7efdedc00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7efde80b3550 con 0x7efde80d2940 2024-04-25T14:10:16.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.205+0000 7efdedc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efde80b3830 con 0x7efde80b05a0 2024-04-25T14:10:16.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.205+0000 7efde7e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7efde80b78c0 0x7efde80d2230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51926/0 (socket says 172.21.15.86:51926) 2024-04-25T14:10:16.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.205+0000 7efde7e00640 1 -- 172.21.15.86:0/1683545846 learned_addr learned my addr 172.21.15.86:0/1683545846 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:16.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.205+0000 7efde7400640 1 --2- 172.21.15.86:0/1683545846 >> v2:172.21.15.155:3300/0 conn(0x7efde80d2940 0x7efde80b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:16.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.205+0000 7efded200640 1 --2- 172.21.15.86:0/1683545846 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efde80b05a0 0x7efde80b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:16.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.209+0000 7efde6a00640 1 -- 172.21.15.86:0/1683545846 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 981809180 0 0) 0x7efdd8001120 con 0x7efde80b78c0 2024-04-25T14:10:16.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.209+0000 7efde6a00640 1 -- 172.21.15.86:0/1683545846 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdcc003530 con 0x7efde80b78c0 2024-04-25T14:10:16.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.209+0000 7efde7400640 1 -- 172.21.15.86:0/1683545846 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efde80b05a0 msgr2=0x7efde80b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:16.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.209+0000 7efde7400640 1 --2- 172.21.15.86:0/1683545846 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efde80b05a0 0x7efde80b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.209+0000 7efde7400640 1 -- 172.21.15.86:0/1683545846 >> v1:172.21.15.86:6789/0 conn(0x7efde80b78c0 legacy=0x7efde80d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:16.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.209+0000 7efde6a00640 1 -- 172.21.15.86:0/1683545846 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 821082456 0 0) 0x7efdcc003530 con 0x7efde80b78c0 2024-04-25T14:10:16.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.209+0000 7efde7400640 1 -- 172.21.15.86:0/1683545846 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde80b3b10 con 0x7efde80d2940 2024-04-25T14:10:16.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.209+0000 7efde7400640 1 --2- 172.21.15.86:0/1683545846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efde80d2940 0x7efde80b1bc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7efdd40097c0 tx=0x7efdd401c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fae8de55b11c694 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:16.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.209+0000 7efde6a00640 1 -- 172.21.15.86:0/1683545846 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd40b0070 con 0x7efde80d2940 2024-04-25T14:10:16.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.209+0000 7efde6a00640 1 -- 172.21.15.86:0/1683545846 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd40ab400 con 0x7efde80d2940 2024-04-25T14:10:16.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.209+0000 7efde6a00640 1 -- 172.21.15.86:0/1683545846 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd8001850 con 0x7efde80d2940 2024-04-25T14:10:16.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.213+0000 7efdedc00640 1 -- 172.21.15.86:0/1683545846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efde80d2940 msgr2=0x7efde80b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:16.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.213+0000 7efdedc00640 1 --2- 172.21.15.86:0/1683545846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efde80d2940 0x7efde80b1bc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7efdd40097c0 tx=0x7efdd401c330 comp rx=0 tx=0).stop 2024-04-25T14:10:16.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.213+0000 7efdedc00640 1 -- 172.21.15.86:0/1683545846 shutdown_connections 2024-04-25T14:10:16.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.213+0000 7efdedc00640 1 --2- 172.21.15.86:0/1683545846 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efde80b05a0 0x7efde80b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.213+0000 7efdedc00640 1 --2- 172.21.15.86:0/1683545846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efde80d2940 0x7efde80b1bc0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.213+0000 7efdedc00640 1 -- 172.21.15.86:0/1683545846 >> 172.21.15.86:0/1683545846 conn(0x7efde809ba30 msgr2=0x7efde80b71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:16.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.213+0000 7efdedc00640 1 -- 172.21.15.86:0/1683545846 shutdown_connections 2024-04-25T14:10:16.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.213+0000 7efdedc00640 1 -- 172.21.15.86:0/1683545846 wait complete. 2024-04-25T14:10:16.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.213+0000 7efdedc00640 1 Processor -- start 2024-04-25T14:10:16.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.213+0000 7efdedc00640 1 -- start start 2024-04-25T14:10:16.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.217+0000 7efdedc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efde80b05a0 0x7efde815d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:16.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.217+0000 7efdedc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efde80b78c0 0x7efde80d0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:16.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.217+0000 7efdedc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efde80d1900 0x7efde80d0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:16.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.217+0000 7efdedc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efde816ce00 con 0x7efde80b78c0 2024-04-25T14:10:16.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.217+0000 7efdedc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efde816cc80 con 0x7efde80d1900 2024-04-25T14:10:16.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.217+0000 7efdedc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efde816cf80 con 0x7efde80b05a0 2024-04-25T14:10:16.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.217+0000 7efde7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efde80b78c0 0x7efde80d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:16.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.217+0000 7efde7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efde80b78c0 0x7efde80d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54048/0 (socket says 172.21.15.86:54048) 2024-04-25T14:10:16.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.217+0000 7efde7400640 1 -- 172.21.15.86:0/3126724433 learned_addr learned my addr 172.21.15.86:0/3126724433 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:16.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.217+0000 7efde7e00640 1 --2- 172.21.15.86:0/3126724433 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efde80b05a0 0x7efde815d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:16.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.217+0000 7efded200640 1 --2- 172.21.15.86:0/3126724433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efde80d1900 0x7efde80d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:16.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.217+0000 7efde7400640 1 -- 172.21.15.86:0/3126724433 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efde80b05a0 msgr2=0x7efde815d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:16.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.217+0000 7efde7400640 1 --2- 172.21.15.86:0/3126724433 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efde80b05a0 0x7efde815d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.217+0000 7efde7400640 1 -- 172.21.15.86:0/3126724433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efde80d1900 msgr2=0x7efde80d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:16.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.217+0000 7efde7400640 1 --2- 172.21.15.86:0/3126724433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efde80d1900 0x7efde80d0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.217+0000 7efde7400640 1 -- 172.21.15.86:0/3126724433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde80d1020 con 0x7efde80b78c0 2024-04-25T14:10:16.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.217+0000 7efde7e00640 1 --2- 172.21.15.86:0/3126724433 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efde80b05a0 0x7efde815d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:16.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.217+0000 7efde7400640 1 --2- 172.21.15.86:0/3126724433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efde80b78c0 0x7efde80d0040 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7efdd401a9e0 tx=0x7efdd401ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:16.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.217+0000 7efddfe00640 1 -- 172.21.15.86:0/3126724433 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd40b0050 con 0x7efde80b78c0 2024-04-25T14:10:16.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.217+0000 7efddfe00640 1 -- 172.21.15.86:0/3126724433 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd40bc070 con 0x7efde80b78c0 2024-04-25T14:10:16.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.217+0000 7efdedc00640 1 -- 172.21.15.86:0/3126724433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde806c710 con 0x7efde80b78c0 2024-04-25T14:10:16.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.217+0000 7efddfe00640 1 -- 172.21.15.86:0/3126724433 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd40b78b0 con 0x7efde80b78c0 2024-04-25T14:10:16.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.217+0000 7efdedc00640 1 -- 172.21.15.86:0/3126724433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efde806c9b0 con 0x7efde80b78c0 2024-04-25T14:10:16.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.221+0000 7efddfe00640 1 -- 172.21.15.86:0/3126724433 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7efdd40b7a50 con 0x7efde80b78c0 2024-04-25T14:10:16.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.221+0000 7efddfe00640 1 --2- 172.21.15.86:0/3126724433 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7efdc806f520 0x7efdc8071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:16.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.221+0000 7efddfe00640 1 -- 172.21.15.86:0/3126724433 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7efdd4129aa0 con 0x7efde80b78c0 2024-04-25T14:10:16.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.221+0000 7efdf68a0000 1 Processor -- start 2024-04-25T14:10:16.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.221+0000 7efde7e00640 1 --2- 172.21.15.86:0/3126724433 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7efdc806f520 0x7efdc8071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:16.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.221+0000 7efdf68a0000 1 -- start start 2024-04-25T14:10:16.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.221+0000 7efde7e00640 1 --2- 172.21.15.86:0/3126724433 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7efdc806f520 0x7efdc8071a30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efdd8001ec0 tx=0x7efdd8009040 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:16.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efdf68a0000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555993ffc20 0x55559955dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:16.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efdf68a0000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555993ebea0 0x55559955e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:16.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efdf68a0000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555598fe4280 0x55559955e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:16.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efdf68a0000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5555993cd160 con 0x5555993ebea0 2024-04-25T14:10:16.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efdf68a0000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555599362e20 con 0x5555993ffc20 2024-04-25T14:10:16.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efdf68a0000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55559931ba20 con 0x555598fe4280 2024-04-25T14:10:16.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efde7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555993ebea0 0x55559955e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:16.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efde7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555993ebea0 0x55559955e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54064/0 (socket says 172.21.15.86:54064) 2024-04-25T14:10:16.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efde7400640 1 -- 172.21.15.86:0/2707615514 learned_addr learned my addr 172.21.15.86:0/2707615514 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:16.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efde7400640 1 -- 172.21.15.86:0/2707615514 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555598fe4280 msgr2=0x55559955e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:16.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efde7400640 1 --2- 172.21.15.86:0/2707615514 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555598fe4280 0x55559955e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efde7400640 1 -- 172.21.15.86:0/2707615514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555993ffc20 msgr2=0x55559955dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:16.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efde7400640 1 --2- 172.21.15.86:0/2707615514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555993ffc20 0x55559955dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efde7400640 1 -- 172.21.15.86:0/2707615514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555599320100 con 0x5555993ebea0 2024-04-25T14:10:16.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efde7400640 1 --2- 172.21.15.86:0/2707615514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555993ebea0 0x55559955e420 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7efdd401c8a0 tx=0x7efdd40f5970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b56fad089b8dcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:16.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efddcc00640 1 -- 172.21.15.86:0/2707615514 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd40f6020 con 0x5555993ebea0 2024-04-25T14:10:16.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efddcc00640 1 -- 172.21.15.86:0/2707615514 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd40b1040 con 0x5555993ebea0 2024-04-25T14:10:16.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efddcc00640 1 -- 172.21.15.86:0/2707615514 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd40feaf0 con 0x5555993ebea0 2024-04-25T14:10:16.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efdf68a0000 1 -- 172.21.15.86:0/2707615514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555993ebea0 msgr2=0x55559955e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:16.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efdf68a0000 1 --2- 172.21.15.86:0/2707615514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555993ebea0 0x55559955e420 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7efdd401c8a0 tx=0x7efdd40f5970 comp rx=0 tx=0).stop 2024-04-25T14:10:16.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efdf68a0000 1 -- 172.21.15.86:0/2707615514 shutdown_connections 2024-04-25T14:10:16.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efdf68a0000 1 --2- 172.21.15.86:0/2707615514 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555598fe4280 0x55559955e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efdf68a0000 1 --2- 172.21.15.86:0/2707615514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555993ebea0 0x55559955e420 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efdf68a0000 1 --2- 172.21.15.86:0/2707615514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555993ffc20 0x55559955dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efdf68a0000 1 -- 172.21.15.86:0/2707615514 >> 172.21.15.86:0/2707615514 conn(0x555599023ff0 msgr2=0x5555995134c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:16.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efdf68a0000 1 -- 172.21.15.86:0/2707615514 shutdown_connections 2024-04-25T14:10:16.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efdf68a0000 1 -- 172.21.15.86:0/2707615514 wait complete. 2024-04-25T14:10:16.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efdf68a0000 1 -- start start 2024-04-25T14:10:16.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.225+0000 7efdf68a0000 1 Processor -- start 2024-04-25T14:10:16.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efdf68a0000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555598fe4280 0x555599564c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:16.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efdf68a0000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555993ebea0 0x555599565190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:16.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efdf68a0000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5555993ffc20 0x5555995200c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:16.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efdf68a0000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555599370220 con 0x555598fe4280 2024-04-25T14:10:16.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efdf68a0000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555599306ea0 con 0x5555993ebea0 2024-04-25T14:10:16.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efdf68a0000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55559935db00 con 0x5555993ffc20 2024-04-25T14:10:16.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efdf68a0000 10 client.0 _ll_register_callbacks cb 0x55559936fdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:16.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efde7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555598fe4280 0x555599564c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:16.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efde7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555598fe4280 0x555599564c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54072/0 (socket says 172.21.15.86:54072) 2024-04-25T14:10:16.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efde7e00640 1 -- 172.21.15.86:0/2749181258 learned_addr learned my addr 172.21.15.86:0/2749181258 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:16.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efde7400640 1 --2- 172.21.15.86:0/2749181258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555993ebea0 0x555599565190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:16.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efded200640 1 --2- 172.21.15.86:0/2749181258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5555993ffc20 0x5555995200c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:16.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efde7e00640 1 -- 172.21.15.86:0/2749181258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5555993ffc20 msgr2=0x5555995200c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:16.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efde7e00640 1 --2- 172.21.15.86:0/2749181258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5555993ffc20 0x5555995200c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efde7e00640 1 -- 172.21.15.86:0/2749181258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555993ebea0 msgr2=0x555599565190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:16.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efde7e00640 1 --2- 172.21.15.86:0/2749181258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555993ebea0 0x555599565190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efde7e00640 1 -- 172.21.15.86:0/2749181258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555599302500 con 0x555598fe4280 2024-04-25T14:10:16.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efde7e00640 1 --2- 172.21.15.86:0/2749181258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555598fe4280 0x555599564c50 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7efdd80070d0 tx=0x7efdd80074d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71227c5759cfcb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:16.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efdd1600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:10:16.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efdd1600640 1 -- 172.21.15.86:0/2749181258 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd8007540 con 0x555598fe4280 2024-04-25T14:10:16.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efdd1600640 1 -- 172.21.15.86:0/2749181258 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7efdd80076e0 con 0x555598fe4280 2024-04-25T14:10:16.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efdf68a0000 1 -- 172.21.15.86:0/2749181258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdd40b1040 con 0x555598fe4280 2024-04-25T14:10:16.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efdd1600640 1 -- 172.21.15.86:0/2749181258 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7efdd8013420 con 0x555598fe4280 2024-04-25T14:10:16.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efdf68a0000 10 client.8613 fetch_fsmap learned FSMap version 85 2024-04-25T14:10:16.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efdf68a0000 1 -- 172.21.15.86:0/2749181258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) -- 0x5555992e8ea0 con 0x555598fe4280 2024-04-25T14:10:16.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efdd1600640 1 -- 172.21.15.86:0/2749181258 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd40b1040 con 0x555598fe4280 2024-04-25T14:10:16.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efdd1600640 1 -- 172.21.15.86:0/2749181258 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 85) ==== 3494+0+0 (secure 0 0 0) 0x7efdd802a050 con 0x555598fe4280 2024-04-25T14:10:16.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efdf68a0000 10 client.8613 fetch_fsmap finished waiting for FSMap version 85 2024-04-25T14:10:16.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efdf68a0000 10 client.8613 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T14:10:16.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efdf68a0000 20 client.8613 populate_metadata read hostname 'smithi086' 2024-04-25T14:10:16.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efdf68a0000 1 --2- 172.21.15.86:0/2749181258 >> [v2:172.21.15.155:6832/142240252,v1:172.21.15.155:6833/142240252] conn(0x5555990087d0 0x555599520e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:16.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efdf68a0000 4 client.8613 mds_command: new command op to 8457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:10:16.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efdf68a0000 1 -- 172.21.15.86:0/2749181258 --> [v2:172.21.15.155:6832/142240252,v1:172.21.15.155:6833/142240252] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555992fbbe0 con 0x5555990087d0 2024-04-25T14:10:16.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.229+0000 7efde7400640 1 --2- 172.21.15.86:0/2749181258 >> [v2:172.21.15.155:6832/142240252,v1:172.21.15.155:6833/142240252] conn(0x5555990087d0 0x555599520e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:16.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.233+0000 7efdd1600640 10 client.8613 ms_handle_connect on v2:172.21.15.155:6832/142240252 2024-04-25T14:10:16.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.233+0000 7efde7400640 1 --2- 172.21.15.86:0/2749181258 >> [v2:172.21.15.155:6832/142240252,v1:172.21.15.155:6833/142240252] conn(0x5555990087d0 0x555599520e30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95904fb594834fec server_cookie=731d0aaf22de61e8 in_seq=0 out_seq=0 2024-04-25T14:10:16.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.233+0000 7efdd1600640 1 -- 172.21.15.86:0/2749181258 <== mds.0 v2:172.21.15.155:6832/142240252 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5555992fbbe0 con 0x5555990087d0 2024-04-25T14:10:16.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.233+0000 7efdd1600640 10 client.8613 handle_command_reply: tid=0 2024-04-25T14:10:16.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.233+0000 7efdf68a0000 1 client.8613 shutdown 2024-04-25T14:10:16.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.233+0000 7efdf68a0000 1 -- 172.21.15.86:0/2749181258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555598fe4280 msgr2=0x555599564c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:16.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.233+0000 7efdf68a0000 1 --2- 172.21.15.86:0/2749181258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555598fe4280 0x555599564c50 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7efdd80070d0 tx=0x7efdd80074d0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.233+0000 7efdf68a0000 1 -- 172.21.15.86:0/2749181258 shutdown_connections 2024-04-25T14:10:16.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.233+0000 7efdf68a0000 1 --2- 172.21.15.86:0/2749181258 >> [v2:172.21.15.155:6832/142240252,v1:172.21.15.155:6833/142240252] conn(0x5555990087d0 0x555599520e30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.233+0000 7efdf68a0000 1 --2- 172.21.15.86:0/2749181258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5555993ffc20 0x5555995200c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.233+0000 7efdf68a0000 1 --2- 172.21.15.86:0/2749181258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555993ebea0 0x555599565190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.233+0000 7efdd1600640 0 client.8613 ms_handle_reset on v2:172.21.15.155:6832/142240252 2024-04-25T14:10:16.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.233+0000 7efdf68a0000 1 --2- 172.21.15.86:0/2749181258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555598fe4280 0x555599564c50 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.233+0000 7efdf68a0000 1 -- 172.21.15.86:0/2749181258 >> 172.21.15.86:0/2749181258 conn(0x555599023ff0 msgr2=0x555599513c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:16.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.233+0000 7efdf68a0000 1 -- 172.21.15.86:0/2749181258 shutdown_connections 2024-04-25T14:10:16.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.233+0000 7efdf68a0000 1 -- 172.21.15.86:0/2749181258 wait complete. 2024-04-25T14:10:16.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.233+0000 7efdf68a0000 20 client.8613 trim_cache size 0 max 16384 2024-04-25T14:10:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efdf68a0000 1 Processor -- start 2024-04-25T14:10:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efdf68a0000 1 -- start start 2024-04-25T14:10:16.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efdf68a0000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555598fe4280 0x555599569ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:16.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efdf68a0000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5555993ebea0 0x555599573de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:16.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efdf68a0000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555993ffc20 0x555599574320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:16.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efdf68a0000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efde8205180 con 0x555598fe4280 2024-04-25T14:10:16.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efdf68a0000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efde8151660 con 0x5555993ffc20 2024-04-25T14:10:16.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efdf68a0000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efde8204c10 con 0x5555993ebea0 2024-04-25T14:10:16.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efde7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555598fe4280 0x555599569ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:16.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efde7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555598fe4280 0x555599569ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54074/0 (socket says 172.21.15.86:54074) 2024-04-25T14:10:16.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efde7e00640 1 -- 172.21.15.86:0/3532959048 learned_addr learned my addr 172.21.15.86:0/3532959048 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:16.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efde7400640 1 --2- 172.21.15.86:0/3532959048 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5555993ebea0 0x555599573de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:16.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efded200640 1 --2- 172.21.15.86:0/3532959048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555993ffc20 0x555599574320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:16.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efde7e00640 1 -- 172.21.15.86:0/3532959048 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5555993ebea0 msgr2=0x555599573de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:16.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efde7e00640 1 --2- 172.21.15.86:0/3532959048 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5555993ebea0 0x555599573de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efde7e00640 1 -- 172.21.15.86:0/3532959048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555993ffc20 msgr2=0x555599574320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:16.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efde7e00640 1 --2- 172.21.15.86:0/3532959048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555993ffc20 0x555599574320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efde7e00640 1 -- 172.21.15.86:0/3532959048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555599517840 con 0x555598fe4280 2024-04-25T14:10:16.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efde7e00640 1 --2- 172.21.15.86:0/3532959048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555598fe4280 0x555599569ea0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7efdd8001bc0 tx=0x7efdd801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe8045bb34d1b72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:16.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efdd3e00640 1 -- 172.21.15.86:0/3532959048 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd8031030 con 0x555598fe4280 2024-04-25T14:10:16.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efdd3e00640 1 -- 172.21.15.86:0/3532959048 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd80075c0 con 0x555598fe4280 2024-04-25T14:10:16.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efdd3e00640 1 -- 172.21.15.86:0/3532959048 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd8013950 con 0x555598fe4280 2024-04-25T14:10:16.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efdf68a0000 1 -- 172.21.15.86:0/3532959048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555598fe4280 msgr2=0x555599569ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:16.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efdf68a0000 1 --2- 172.21.15.86:0/3532959048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555598fe4280 0x555599569ea0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7efdd8001bc0 tx=0x7efdd801d000 comp rx=0 tx=0).stop 2024-04-25T14:10:16.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efdf68a0000 1 -- 172.21.15.86:0/3532959048 shutdown_connections 2024-04-25T14:10:16.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efdf68a0000 1 --2- 172.21.15.86:0/3532959048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555993ffc20 0x555599574320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efdf68a0000 1 --2- 172.21.15.86:0/3532959048 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5555993ebea0 0x555599573de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efdf68a0000 1 --2- 172.21.15.86:0/3532959048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555598fe4280 0x555599569ea0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.253+0000 7efdf68a0000 1 -- 172.21.15.86:0/3532959048 >> 172.21.15.86:0/3532959048 conn(0x555599023ff0 msgr2=0x555599563a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:16.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.257+0000 7efdf68a0000 1 -- 172.21.15.86:0/3532959048 shutdown_connections 2024-04-25T14:10:16.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.257+0000 7efdf68a0000 1 -- 172.21.15.86:0/3532959048 wait complete. 2024-04-25T14:10:16.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.257+0000 7efdf68a0000 1 -- start start 2024-04-25T14:10:16.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.257+0000 7efdf68a0000 1 Processor -- start 2024-04-25T14:10:16.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.257+0000 7efdf68a0000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555598fe4280 0x5555995cff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:16.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.257+0000 7efdf68a0000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5555993ebea0 0x5555995d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:16.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.257+0000 7efdf68a0000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555993ffc20 0x555599568250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:16.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.257+0000 7efdf68a0000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555599362130 con 0x555598fe4280 2024-04-25T14:10:16.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.257+0000 7efdf68a0000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5555992f87d0 con 0x5555993ffc20 2024-04-25T14:10:16.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.257+0000 7efdf68a0000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x555599367450 con 0x5555993ebea0 2024-04-25T14:10:16.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.257+0000 7efdf68a0000 10 client.0 _ll_register_callbacks cb 0x7efdd8001790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:16.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.257+0000 7efde7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555598fe4280 0x5555995cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:16.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.257+0000 7efde7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555598fe4280 0x5555995cff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54078/0 (socket says 172.21.15.86:54078) 2024-04-25T14:10:16.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.257+0000 7efde7e00640 1 -- 172.21.15.86:0/1777585785 learned_addr learned my addr 172.21.15.86:0/1777585785 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:16.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.257+0000 7efded200640 1 --2- 172.21.15.86:0/1777585785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555993ffc20 0x555599568250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:16.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.257+0000 7efde7400640 1 --2- 172.21.15.86:0/1777585785 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5555993ebea0 0x5555995d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:16.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.257+0000 7efde7e00640 1 -- 172.21.15.86:0/1777585785 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5555993ebea0 msgr2=0x5555995d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:16.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.257+0000 7efde7e00640 1 --2- 172.21.15.86:0/1777585785 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5555993ebea0 0x5555995d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.257+0000 7efde7e00640 1 -- 172.21.15.86:0/1777585785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555993ffc20 msgr2=0x555599568250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:16.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.257+0000 7efde7e00640 1 --2- 172.21.15.86:0/1777585785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555993ffc20 0x555599568250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.257+0000 7efde7e00640 1 -- 172.21.15.86:0/1777585785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555599520430 con 0x555598fe4280 2024-04-25T14:10:16.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.257+0000 7efded200640 1 --2- 172.21.15.86:0/1777585785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555993ffc20 0x555599568250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:16.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.257+0000 7efde7400640 1 --2- 172.21.15.86:0/1777585785 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5555993ebea0 0x5555995d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:16.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.257+0000 7efde7e00640 1 --2- 172.21.15.86:0/1777585785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555598fe4280 0x5555995cff60 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7efdd8005fd0 tx=0x7efdd8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0eed644bcb1af91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:16.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.257+0000 7efdd1600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:10:16.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.257+0000 7efdd1600640 1 -- 172.21.15.86:0/1777585785 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd80047f0 con 0x555598fe4280 2024-04-25T14:10:16.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.257+0000 7efdd1600640 1 -- 172.21.15.86:0/1777585785 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7efdd8036c40 con 0x555598fe4280 2024-04-25T14:10:16.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.257+0000 7efdf68a0000 1 -- 172.21.15.86:0/1777585785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdd80075c0 con 0x555598fe4280 2024-04-25T14:10:16.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdd1600640 1 -- 172.21.15.86:0/1777585785 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7efdd8036e40 con 0x555598fe4280 2024-04-25T14:10:16.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdf68a0000 10 client.8619 fetch_fsmap learned FSMap version 85 2024-04-25T14:10:16.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdf68a0000 1 -- 172.21.15.86:0/1777585785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) -- 0x55559931bc70 con 0x555598fe4280 2024-04-25T14:10:16.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdd1600640 1 -- 172.21.15.86:0/1777585785 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd80075c0 con 0x555598fe4280 2024-04-25T14:10:16.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdd1600640 1 -- 172.21.15.86:0/1777585785 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 85) ==== 3494+0+0 (secure 0 0 0) 0x7efdd802a040 con 0x555598fe4280 2024-04-25T14:10:16.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdf68a0000 10 client.8619 fetch_fsmap finished waiting for FSMap version 85 2024-04-25T14:10:16.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdf68a0000 10 client.8619 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T14:10:16.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdf68a0000 20 client.8619 populate_metadata read hostname 'smithi086' 2024-04-25T14:10:16.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdf68a0000 1 --2- 172.21.15.86:0/1777585785 >> [v2:172.21.15.155:6832/142240252,v1:172.21.15.155:6833/142240252] conn(0x555598ff6830 0x555599568c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:16.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdf68a0000 4 client.8619 mds_command: new command op to 8457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:10:16.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdf68a0000 1 -- 172.21.15.86:0/1777585785 --> [v2:172.21.15.155:6832/142240252,v1:172.21.15.155:6833/142240252] -- command(tid 0: {"prefix": "status"}) -- 0x555599370a30 con 0x555598ff6830 2024-04-25T14:10:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efde7400640 1 --2- 172.21.15.86:0/1777585785 >> [v2:172.21.15.155:6832/142240252,v1:172.21.15.155:6833/142240252] conn(0x555598ff6830 0x555599568c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efde7400640 1 --2- 172.21.15.86:0/1777585785 >> [v2:172.21.15.155:6832/142240252,v1:172.21.15.155:6833/142240252] conn(0x555598ff6830 0x555599568c10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77b7665c7466af48 server_cookie=22aac971ea73f62a in_seq=0 out_seq=0 2024-04-25T14:10:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdd1600640 10 client.8619 ms_handle_connect on v2:172.21.15.155:6832/142240252 2024-04-25T14:10:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdd1600640 1 -- 172.21.15.86:0/1777585785 <== mds.0 v2:172.21.15.155:6832/142240252 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555599370a30 con 0x555598ff6830 2024-04-25T14:10:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdd1600640 10 client.8619 handle_command_reply: tid=0 2024-04-25T14:10:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdf68a0000 1 client.8619 shutdown 2024-04-25T14:10:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdf68a0000 1 -- 172.21.15.86:0/1777585785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555598fe4280 msgr2=0x5555995cff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdf68a0000 1 --2- 172.21.15.86:0/1777585785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555598fe4280 0x5555995cff60 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7efdd8005fd0 tx=0x7efdd8012000 comp rx=0 tx=0).stop 2024-04-25T14:10:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdf68a0000 1 -- 172.21.15.86:0/1777585785 shutdown_connections 2024-04-25T14:10:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdf68a0000 1 --2- 172.21.15.86:0/1777585785 >> [v2:172.21.15.155:6832/142240252,v1:172.21.15.155:6833/142240252] conn(0x555598ff6830 0x555599568c10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdf68a0000 1 --2- 172.21.15.86:0/1777585785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555993ffc20 0x555599568250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdf68a0000 1 --2- 172.21.15.86:0/1777585785 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5555993ebea0 0x5555995d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdf68a0000 1 --2- 172.21.15.86:0/1777585785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555598fe4280 0x5555995cff60 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdd1600640 0 client.8619 ms_handle_reset on v2:172.21.15.155:6832/142240252 2024-04-25T14:10:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdf68a0000 1 -- 172.21.15.86:0/1777585785 >> 172.21.15.86:0/1777585785 conn(0x555599023ff0 msgr2=0x55559951a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdf68a0000 1 -- 172.21.15.86:0/1777585785 shutdown_connections 2024-04-25T14:10:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdf68a0000 1 -- 172.21.15.86:0/1777585785 wait complete. 2024-04-25T14:10:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdf68a0000 20 client.8619 trim_cache size 0 max 16384 2024-04-25T14:10:16.263 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:10:16.263 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb", 2024-04-25T14:10:16.263 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-25T14:10:16.263 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 8457, 2024-04-25T14:10:16.263 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T14:10:16.263 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T14:10:16.263 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T14:10:16.263 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 10.522461005, 2024-04-25T14:10:16.263 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 85, 2024-04-25T14:10:16.263 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 113, 2024-04-25T14:10:16.263 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 113, 2024-04-25T14:10:16.263 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 15.422809663000001 2024-04-25T14:10:16.263 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:10:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdedc00640 1 -- 172.21.15.86:0/3126724433 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7efdc806f520 msgr2=0x7efdc8071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdedc00640 1 --2- 172.21.15.86:0/3126724433 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7efdc806f520 0x7efdc8071a30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efdd8001ec0 tx=0x7efdd8009040 comp rx=0 tx=0).stop 2024-04-25T14:10:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdedc00640 1 -- 172.21.15.86:0/3126724433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efde80b78c0 msgr2=0x7efde80d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdedc00640 1 --2- 172.21.15.86:0/3126724433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efde80b78c0 0x7efde80d0040 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7efdd401a9e0 tx=0x7efdd401ab70 comp rx=0 tx=0).stop 2024-04-25T14:10:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdedc00640 1 -- 172.21.15.86:0/3126724433 shutdown_connections 2024-04-25T14:10:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdedc00640 1 --2- 172.21.15.86:0/3126724433 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7efdc806f520 0x7efdc8071a30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdedc00640 1 --2- 172.21.15.86:0/3126724433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efde80d1900 0x7efde80d0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdedc00640 1 --2- 172.21.15.86:0/3126724433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efde80b78c0 0x7efde80d0040 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdedc00640 1 --2- 172.21.15.86:0/3126724433 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efde80b05a0 0x7efde815d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdedc00640 1 -- 172.21.15.86:0/3126724433 >> 172.21.15.86:0/3126724433 conn(0x7efde809ba30 msgr2=0x7efde806e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdedc00640 1 -- 172.21.15.86:0/3126724433 shutdown_connections 2024-04-25T14:10:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.261+0000 7efdedc00640 1 -- 172.21.15.86:0/3126724433 wait complete. 2024-04-25T14:10:16.280 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T14:10:16.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.429+0000 7fd6afe00640 1 Processor -- start 2024-04-25T14:10:16.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.429+0000 7fd6afe00640 1 -- start start 2024-04-25T14:10:16.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.429+0000 7fd6afe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd6a80d2420 0x7fd6a80d2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:16.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.429+0000 7fd6afe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd6a8070e20 0x7fd6a8071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:16.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.429+0000 7fd6afe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6a8071770 con 0x7fd6a80b6490 2024-04-25T14:10:16.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.429+0000 7fd6afe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd6a80d16f0 con 0x7fd6a80d2420 2024-04-25T14:10:16.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.429+0000 7fd6afe00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd6a8162480 con 0x7fd6a8070e20 2024-04-25T14:10:16.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.429+0000 7fd6ad400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd6a80d2420 0x7fd6a80d2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:16.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.429+0000 7fd6ad400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd6a80d2420 0x7fd6a80d2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54084/0 (socket says 172.21.15.86:54084) 2024-04-25T14:10:16.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.429+0000 7fd6ade00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd6a80b6490 0x7fd6a80b68a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51928/0 (socket says 172.21.15.86:51928) 2024-04-25T14:10:16.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.429+0000 7fd6ad400640 1 -- 172.21.15.86:0/3183722516 learned_addr learned my addr 172.21.15.86:0/3183722516 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:16.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.433+0000 7fd6aca00640 1 --2- 172.21.15.86:0/3183722516 >> v2:172.21.15.155:3300/0 conn(0x7fd6a8070e20 0x7fd6a8071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:16.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.433+0000 7fd6ad400640 1 -- 172.21.15.86:0/3183722516 >> v2:172.21.15.155:3300/0 conn(0x7fd6a8070e20 msgr2=0x7fd6a8071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:16.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.433+0000 7fd6ad400640 1 --2- 172.21.15.86:0/3183722516 >> v2:172.21.15.155:3300/0 conn(0x7fd6a8070e20 0x7fd6a8071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.433+0000 7fd6ad400640 1 -- 172.21.15.86:0/3183722516 >> v1:172.21.15.86:6789/0 conn(0x7fd6a80b6490 legacy=0x7fd6a80b68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:16.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.433+0000 7fd6ad400640 1 -- 172.21.15.86:0/3183722516 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a80d1940 con 0x7fd6a80d2420 2024-04-25T14:10:16.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.433+0000 7fd6ad400640 1 --2- 172.21.15.86:0/3183722516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6a80d2420 0x7fd6a80d2830 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8074610 tx=0x7fd69c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dfa979546036d3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:16.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.433+0000 7fd6a7e00640 1 -- 172.21.15.86:0/3183722516 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c0ab610 con 0x7fd6a80d2420 2024-04-25T14:10:16.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.433+0000 7fd6a7e00640 1 -- 172.21.15.86:0/3183722516 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd69c0b0070 con 0x7fd6a80d2420 2024-04-25T14:10:16.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.433+0000 7fd6a7e00640 1 -- 172.21.15.86:0/3183722516 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c0ab9d0 con 0x7fd6a80d2420 2024-04-25T14:10:16.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.437+0000 7fd6afe00640 1 -- 172.21.15.86:0/3183722516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6a80d2420 msgr2=0x7fd6a80d2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:16.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.437+0000 7fd6afe00640 1 --2- 172.21.15.86:0/3183722516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6a80d2420 0x7fd6a80d2830 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8074610 tx=0x7fd69c01c520 comp rx=0 tx=0).stop 2024-04-25T14:10:16.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.437+0000 7fd6afe00640 1 -- 172.21.15.86:0/3183722516 shutdown_connections 2024-04-25T14:10:16.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.437+0000 7fd6afe00640 1 --2- 172.21.15.86:0/3183722516 >> v2:172.21.15.155:3300/0 conn(0x7fd6a8070e20 0x7fd6a8071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.437+0000 7fd6afe00640 1 --2- 172.21.15.86:0/3183722516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6a80d2420 0x7fd6a80d2830 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.437+0000 7fd6afe00640 1 -- 172.21.15.86:0/3183722516 >> 172.21.15.86:0/3183722516 conn(0x7fd6a80cffc0 msgr2=0x7fd6a80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:16.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.437+0000 7fd6afe00640 1 -- 172.21.15.86:0/3183722516 shutdown_connections 2024-04-25T14:10:16.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.437+0000 7fd6afe00640 1 -- 172.21.15.86:0/3183722516 wait complete. 2024-04-25T14:10:16.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.437+0000 7fd6afe00640 1 Processor -- start 2024-04-25T14:10:16.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.437+0000 7fd6afe00640 1 -- start start 2024-04-25T14:10:16.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.437+0000 7fd6afe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6a8070e20 0x7fd6a80ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:16.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.437+0000 7fd6afe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6a80b6490 0x7fd6a80b7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:16.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.437+0000 7fd6afe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6a80d2420 0x7fd6a8156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:16.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.437+0000 7fd6afe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a8161310 con 0x7fd6a80d2420 2024-04-25T14:10:16.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.437+0000 7fd6afe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a80cdf30 con 0x7fd6a8070e20 2024-04-25T14:10:16.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.437+0000 7fd6afe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd6a80cb550 con 0x7fd6a80b6490 2024-04-25T14:10:16.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.441+0000 7fd6ade00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6a80d2420 0x7fd6a8156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:16.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.441+0000 7fd6ade00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6a80d2420 0x7fd6a8156350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54090/0 (socket says 172.21.15.86:54090) 2024-04-25T14:10:16.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.441+0000 7fd6ade00640 1 -- 172.21.15.86:0/818507793 learned_addr learned my addr 172.21.15.86:0/818507793 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:16.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.441+0000 7fd6ad400640 1 --2- 172.21.15.86:0/818507793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6a8070e20 0x7fd6a80ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:16.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.441+0000 7fd6aca00640 1 --2- 172.21.15.86:0/818507793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6a80b6490 0x7fd6a80b7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:16.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.441+0000 7fd6ade00640 1 -- 172.21.15.86:0/818507793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6a80b6490 msgr2=0x7fd6a80b7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:16.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.441+0000 7fd6ade00640 1 --2- 172.21.15.86:0/818507793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6a80b6490 0x7fd6a80b7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.441+0000 7fd6ade00640 1 -- 172.21.15.86:0/818507793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6a8070e20 msgr2=0x7fd6a80ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:16.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.441+0000 7fd6ade00640 1 --2- 172.21.15.86:0/818507793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6a8070e20 0x7fd6a80ccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.441+0000 7fd6ade00640 1 -- 172.21.15.86:0/818507793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a815b010 con 0x7fd6a80d2420 2024-04-25T14:10:16.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.441+0000 7fd6aca00640 1 --2- 172.21.15.86:0/818507793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6a80b6490 0x7fd6a80b7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:16.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.441+0000 7fd6ade00640 1 --2- 172.21.15.86:0/818507793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6a80d2420 0x7fd6a8156350 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000c760 tx=0x7fd6a000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:16.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.441+0000 7fd6a5600640 1 -- 172.21.15.86:0/818507793 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0017070 con 0x7fd6a80d2420 2024-04-25T14:10:16.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.441+0000 7fd6a5600640 1 -- 172.21.15.86:0/818507793 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6a000ce10 con 0x7fd6a80d2420 2024-04-25T14:10:16.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.441+0000 7fd6a5600640 1 -- 172.21.15.86:0/818507793 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a000a840 con 0x7fd6a80d2420 2024-04-25T14:10:16.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.441+0000 7fd6afe00640 1 -- 172.21.15.86:0/818507793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a80b72b0 con 0x7fd6a80d2420 2024-04-25T14:10:16.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.441+0000 7fd6afe00640 1 -- 172.21.15.86:0/818507793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a806e240 con 0x7fd6a80d2420 2024-04-25T14:10:16.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.441+0000 7fd6afe00640 1 -- 172.21.15.86:0/818507793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6a80cd8c0 con 0x7fd6a80d2420 2024-04-25T14:10:16.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.441+0000 7fd6a5600640 1 -- 172.21.15.86:0/818507793 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7fd6a001a070 con 0x7fd6a80d2420 2024-04-25T14:10:16.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.445+0000 7fd6a5600640 1 --2- 172.21.15.86:0/818507793 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fd68806f520 0x7fd688071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:16.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.445+0000 7fd6a5600640 1 -- 172.21.15.86:0/818507793 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7fd6a008ea00 con 0x7fd6a80d2420 2024-04-25T14:10:16.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.445+0000 7fd6ad400640 1 --2- 172.21.15.86:0/818507793 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fd68806f520 0x7fd688071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:16.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.445+0000 7fd6ad400640 1 --2- 172.21.15.86:0/818507793 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fd68806f520 0x7fd688071a30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0af000 tx=0x7fd69c0a7000 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:16.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.449+0000 7fd6a5600640 1 -- 172.21.15.86:0/818507793 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd6a8071770 con 0x7fd6a80d2420 2024-04-25T14:10:16.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.641+0000 7fd6afe00640 1 -- 172.21.15.86:0/818507793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd6a80b87f0 con 0x7fd6a80d2420 2024-04-25T14:10:16.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.641+0000 7fd6a5600640 1 -- 172.21.15.86:0/818507793 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v85) ==== 53+0+73 (secure 0 0 0) 0x7fd6a005b000 con 0x7fd6a80d2420 2024-04-25T14:10:16.639 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T14:10:16.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.641+0000 7fd6afe00640 1 -- 172.21.15.86:0/818507793 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fd68806f520 msgr2=0x7fd688071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:16.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.641+0000 7fd6afe00640 1 --2- 172.21.15.86:0/818507793 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fd68806f520 0x7fd688071a30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0af000 tx=0x7fd69c0a7000 comp rx=0 tx=0).stop 2024-04-25T14:10:16.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.641+0000 7fd6afe00640 1 -- 172.21.15.86:0/818507793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6a80d2420 msgr2=0x7fd6a8156350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:16.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.641+0000 7fd6afe00640 1 --2- 172.21.15.86:0/818507793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6a80d2420 0x7fd6a8156350 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000c760 tx=0x7fd6a000cc20 comp rx=0 tx=0).stop 2024-04-25T14:10:16.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.645+0000 7fd6afe00640 1 -- 172.21.15.86:0/818507793 shutdown_connections 2024-04-25T14:10:16.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.645+0000 7fd6afe00640 1 --2- 172.21.15.86:0/818507793 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fd68806f520 0x7fd688071a30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.645+0000 7fd6afe00640 1 --2- 172.21.15.86:0/818507793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6a80d2420 0x7fd6a8156350 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.645+0000 7fd6afe00640 1 --2- 172.21.15.86:0/818507793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6a80b6490 0x7fd6a80b7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.645+0000 7fd6afe00640 1 --2- 172.21.15.86:0/818507793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6a8070e20 0x7fd6a80ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:16.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.645+0000 7fd6afe00640 1 -- 172.21.15.86:0/818507793 >> 172.21.15.86:0/818507793 conn(0x7fd6a80cffc0 msgr2=0x7fd6a80b68a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:16.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.645+0000 7fd6afe00640 1 -- 172.21.15.86:0/818507793 shutdown_connections 2024-04-25T14:10:16.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16.645+0000 7fd6afe00640 1 -- 172.21.15.86:0/818507793 wait complete. 2024-04-25T14:10:16.661 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T14:10:16.661 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T14:10:16.661 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-25T14:10:16.661 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T14:10:16.661 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T14:10:16.661 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T14:10:16.661 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T14:10:16.661 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T14:10:16.661 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:10:16.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:16.662 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-25T14:10:16.672 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:10:16.672 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:10:16.672 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:10:16.672 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:16.672 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-25T14:10:16.672 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:16.672 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:10:16.672 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:10:16.672 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-25T14:10:16.672 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:10:16.672 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:10:16.672 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-25T14:10:16.672 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:10:16.672 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:10:16.673 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T14:10:16.673 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:10:16.673 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T14:10:16.673 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42048sec preferred_lft 42048sec 2024-04-25T14:10:16.673 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-25T14:10:16.673 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:16.673 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T14:10:16.673 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:dc:87:9c:96 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:10:16.673 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T14:10:16.673 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:16.673 INFO:teuthology.orchestra.run.smithi086.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T14:10:16.673 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T14:10:16.673 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T14:10:16.673 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:16.673 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T14:10:16.674 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:16.674 INFO:teuthology.orchestra.run.smithi086.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:10:16.674 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T14:10:16.674 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T14:10:16.674 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:16.674 INFO:teuthology.orchestra.run.smithi086.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:10:16.674 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T14:10:16.674 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T14:10:16.674 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:16.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:16.674 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T14:10:16.734 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-25T14:10:16.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:16.736 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-25T14:10:16.788 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-25T14:10:16.788 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T14:10:16.788 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T14:10:16.788 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T14:10:16.788 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T14:10:16.788 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T14:10:16.788 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T14:10:16.789 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:10:16.789 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:10:16.789 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:10:16.789 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T14:10:16.789 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:10:16.789 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T14:10:16.789 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T14:10:16.789 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T14:10:16.789 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T14:10:16.789 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T14:10:16.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:10:16.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:10:16.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:16.923 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T14:10:16.932 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T14:10:16.932 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T14:10:16.932 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:10:16.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:16.932 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-25T14:10:16.992 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:10:16.992 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T14:10:17.044 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:10:17.045 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:10:17.045 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T14:10:17.099 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T14:10:17.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:17.100 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:10:17.200 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:10:17.200 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T14:10:17.200 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:10:17.218 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:10:17.218 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T14:10:17.218 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:10:17.218 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:10:17.218 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:10:17.228 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:10:17.229 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T14:10:17.229 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:10:17.229 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:10:17.229 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:10:17.229 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790" 2024-04-25T14:10:17.229 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790:/ 2024-04-25T14:10:17.229 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb" will pass to kernel 2024-04-25T14:10:17.233 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T14:10:17.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:17.233 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T14:10:17.253 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:10:17.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:10:17.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:10:17.368 INFO:teuthology.orchestra.run.smithi086.stdout:client8631 2024-04-25T14:10:17.368 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:10:17.369 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:10:17.377 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T14:10:17.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.569+0000 7f90e3200640 1 Processor -- start 2024-04-25T14:10:17.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.569+0000 7f90e3200640 1 -- start start 2024-04-25T14:10:17.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.573+0000 7f90e3200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f90dc1642a0 0x7f90dc0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:17.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.573+0000 7f90e3200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f90dc0d1c20 0x7f90dc0b0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:17.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.573+0000 7f90e3200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90dc0b0690 0x7f90dc0d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:17.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.573+0000 7f90e3200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f90dc0cea30 con 0x7f90dc1642a0 2024-04-25T14:10:17.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.573+0000 7f90e3200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f90dc0d2030 con 0x7f90dc0d1c20 2024-04-25T14:10:17.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.573+0000 7f90e3200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f90dc0cde20 con 0x7f90dc0b0690 2024-04-25T14:10:17.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.573+0000 7f90dbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f90dc1642a0 0x7f90dc0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:17.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.573+0000 7f90dbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f90dc1642a0 0x7f90dc0d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54092/0 (socket says 172.21.15.86:54092) 2024-04-25T14:10:17.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.573+0000 7f90dbe00640 1 -- 172.21.15.86:0/2047187599 learned_addr learned my addr 172.21.15.86:0/2047187599 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:17.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.573+0000 7f90d3400640 1 --2- 172.21.15.86:0/2047187599 >> v2:172.21.15.155:3300/0 conn(0x7f90dc0d1c20 0x7f90dc0b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:17.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.573+0000 7f90e1200640 1 --2- 172.21.15.86:0/2047187599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90dc0b0690 0x7f90dc0d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:17.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.573+0000 7f90dbe00640 1 -- 172.21.15.86:0/2047187599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90dc0b0690 msgr2=0x7f90dc0d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:17.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.573+0000 7f90dbe00640 1 --2- 172.21.15.86:0/2047187599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90dc0b0690 0x7f90dc0d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:17.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.573+0000 7f90dbe00640 1 -- 172.21.15.86:0/2047187599 >> v2:172.21.15.155:3300/0 conn(0x7f90dc0d1c20 msgr2=0x7f90dc0b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:17.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.573+0000 7f90dbe00640 1 --2- 172.21.15.86:0/2047187599 >> v2:172.21.15.155:3300/0 conn(0x7f90dc0d1c20 0x7f90dc0b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:17.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.573+0000 7f90dbe00640 1 -- 172.21.15.86:0/2047187599 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90dc0ce100 con 0x7f90dc1642a0 2024-04-25T14:10:17.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.577+0000 7f90dbe00640 1 --2- 172.21.15.86:0/2047187599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90dc1642a0 0x7f90dc0d16e0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f90dc06df80 tx=0x7f90cc01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c48bd6d660dc08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:17.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.577+0000 7f90db400640 1 -- 172.21.15.86:0/2047187599 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc0a55e0 con 0x7f90dc1642a0 2024-04-25T14:10:17.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.577+0000 7f90db400640 1 -- 172.21.15.86:0/2047187599 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90cc0a5780 con 0x7f90dc1642a0 2024-04-25T14:10:17.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.577+0000 7f90db400640 1 -- 172.21.15.86:0/2047187599 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc0a5b50 con 0x7f90dc1642a0 2024-04-25T14:10:17.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.577+0000 7f90e3200640 1 -- 172.21.15.86:0/2047187599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90dc1642a0 msgr2=0x7f90dc0d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:17.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.577+0000 7f90e3200640 1 --2- 172.21.15.86:0/2047187599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90dc1642a0 0x7f90dc0d16e0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f90dc06df80 tx=0x7f90cc01c960 comp rx=0 tx=0).stop 2024-04-25T14:10:17.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.577+0000 7f90e3200640 1 -- 172.21.15.86:0/2047187599 shutdown_connections 2024-04-25T14:10:17.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.577+0000 7f90e3200640 1 --2- 172.21.15.86:0/2047187599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90dc0b0690 0x7f90dc0d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:17.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.577+0000 7f90e3200640 1 --2- 172.21.15.86:0/2047187599 >> v2:172.21.15.155:3300/0 conn(0x7f90dc0d1c20 0x7f90dc0b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:17.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.577+0000 7f90e3200640 1 --2- 172.21.15.86:0/2047187599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90dc1642a0 0x7f90dc0d16e0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:17.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.577+0000 7f90e3200640 1 -- 172.21.15.86:0/2047187599 >> 172.21.15.86:0/2047187599 conn(0x7f90dc0b4650 msgr2=0x7f90dc0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:17.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.577+0000 7f90e3200640 1 -- 172.21.15.86:0/2047187599 shutdown_connections 2024-04-25T14:10:17.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.577+0000 7f90e3200640 1 -- 172.21.15.86:0/2047187599 wait complete. 2024-04-25T14:10:17.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.577+0000 7f90e3200640 1 Processor -- start 2024-04-25T14:10:17.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.581+0000 7f90e3200640 1 -- start start 2024-04-25T14:10:17.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.581+0000 7f90e3200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90dc0b0690 0x7f90dc15a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:17.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.581+0000 7f90e3200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90dc0d1c20 0x7f90dc15a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:17.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.581+0000 7f90e3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90dc1642a0 0x7f90dc09a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:17.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.581+0000 7f90e3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc0d1110 con 0x7f90dc1642a0 2024-04-25T14:10:17.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.581+0000 7f90e3200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc0d0f90 con 0x7f90dc0d1c20 2024-04-25T14:10:17.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.581+0000 7f90e3200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f90dc1646b0 con 0x7f90dc0b0690 2024-04-25T14:10:17.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.581+0000 7f90e1200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90dc1642a0 0x7f90dc09a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:17.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.581+0000 7f90e1200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90dc1642a0 0x7f90dc09a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54104/0 (socket says 172.21.15.86:54104) 2024-04-25T14:10:17.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.581+0000 7f90e1200640 1 -- 172.21.15.86:0/100575749 learned_addr learned my addr 172.21.15.86:0/100575749 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:17.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.581+0000 7f90d3400640 1 --2- 172.21.15.86:0/100575749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90dc0d1c20 0x7f90dc15a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:17.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.581+0000 7f90dbe00640 1 --2- 172.21.15.86:0/100575749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90dc0b0690 0x7f90dc15a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:17.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.581+0000 7f90e1200640 1 -- 172.21.15.86:0/100575749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90dc0b0690 msgr2=0x7f90dc15a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:17.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.581+0000 7f90e1200640 1 --2- 172.21.15.86:0/100575749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90dc0b0690 0x7f90dc15a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:17.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.581+0000 7f90e1200640 1 -- 172.21.15.86:0/100575749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90dc0d1c20 msgr2=0x7f90dc15a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:17.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.581+0000 7f90e1200640 1 --2- 172.21.15.86:0/100575749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90dc0d1c20 0x7f90dc15a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:17.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.581+0000 7f90e1200640 1 -- 172.21.15.86:0/100575749 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90dc09a8b0 con 0x7f90dc1642a0 2024-04-25T14:10:17.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.581+0000 7f90e1200640 1 --2- 172.21.15.86:0/100575749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90dc1642a0 0x7f90dc09a370 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f90d4006e90 tx=0x7f90d400d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:17.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.581+0000 7f90d8c00640 1 -- 172.21.15.86:0/100575749 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d40125f0 con 0x7f90dc1642a0 2024-04-25T14:10:17.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.581+0000 7f90d8c00640 1 -- 172.21.15.86:0/100575749 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90d4019070 con 0x7f90dc1642a0 2024-04-25T14:10:17.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.581+0000 7f90d8c00640 1 -- 172.21.15.86:0/100575749 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d40129d0 con 0x7f90dc1642a0 2024-04-25T14:10:17.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.581+0000 7f90e3200640 1 -- 172.21.15.86:0/100575749 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90dc161510 con 0x7f90dc1642a0 2024-04-25T14:10:17.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.581+0000 7f90e3200640 1 -- 172.21.15.86:0/100575749 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90dc15e410 con 0x7f90dc1642a0 2024-04-25T14:10:17.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.581+0000 7f90e3200640 1 -- 172.21.15.86:0/100575749 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90dc09f4b0 con 0x7f90dc1642a0 2024-04-25T14:10:17.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.585+0000 7f90d8c00640 1 -- 172.21.15.86:0/100575749 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7f90d4012b70 con 0x7f90dc1642a0 2024-04-25T14:10:17.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.585+0000 7f90d8c00640 1 --2- 172.21.15.86:0/100575749 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f90b806f520 0x7f90b8071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:17.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.585+0000 7f90d8c00640 1 -- 172.21.15.86:0/100575749 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f90d408f910 con 0x7f90dc1642a0 2024-04-25T14:10:17.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.585+0000 7f90dbe00640 1 --2- 172.21.15.86:0/100575749 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f90b806f520 0x7f90b8071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:17.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.585+0000 7f90dbe00640 1 --2- 172.21.15.86:0/100575749 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f90b806f520 0x7f90b8071a30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f90cc0ae000 tx=0x7f90cc0a7070 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:17.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.589+0000 7f90d8c00640 1 -- 172.21.15.86:0/100575749 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f90d405c5f0 con 0x7f90dc1642a0 2024-04-25T14:10:17.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.781+0000 7f90e3200640 1 -- 172.21.15.86:0/100575749 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f90dc069d00 con 0x7f90dc1642a0 2024-04-25T14:10:17.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.781+0000 7f90d8c00640 1 -- 172.21.15.86:0/100575749 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v85) ==== 53+0+73 (secure 0 0 0) 0x7f90d405bef0 con 0x7f90dc1642a0 2024-04-25T14:10:17.781 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T14:10:17.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.785+0000 7f90e3200640 1 -- 172.21.15.86:0/100575749 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f90b806f520 msgr2=0x7f90b8071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:17.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.785+0000 7f90e3200640 1 --2- 172.21.15.86:0/100575749 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f90b806f520 0x7f90b8071a30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f90cc0ae000 tx=0x7f90cc0a7070 comp rx=0 tx=0).stop 2024-04-25T14:10:17.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.785+0000 7f90e3200640 1 -- 172.21.15.86:0/100575749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90dc1642a0 msgr2=0x7f90dc09a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:17.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.785+0000 7f90e3200640 1 --2- 172.21.15.86:0/100575749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90dc1642a0 0x7f90dc09a370 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f90d4006e90 tx=0x7f90d400d9e0 comp rx=0 tx=0).stop 2024-04-25T14:10:17.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.785+0000 7f90e3200640 1 -- 172.21.15.86:0/100575749 shutdown_connections 2024-04-25T14:10:17.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.785+0000 7f90e3200640 1 --2- 172.21.15.86:0/100575749 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f90b806f520 0x7f90b8071a30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:17.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.785+0000 7f90e3200640 1 --2- 172.21.15.86:0/100575749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90dc1642a0 0x7f90dc09a370 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:17.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.785+0000 7f90e3200640 1 --2- 172.21.15.86:0/100575749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90dc0d1c20 0x7f90dc15a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:17.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.785+0000 7f90e3200640 1 --2- 172.21.15.86:0/100575749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90dc0b0690 0x7f90dc15a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:17.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.785+0000 7f90e3200640 1 -- 172.21.15.86:0/100575749 >> 172.21.15.86:0/100575749 conn(0x7f90dc0b4650 msgr2=0x7f90dc0b1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:17.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.785+0000 7f90e3200640 1 -- 172.21.15.86:0/100575749 shutdown_connections 2024-04-25T14:10:17.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:17.785+0000 7f90e3200640 1 -- 172.21.15.86:0/100575749 wait complete. 2024-04-25T14:10:17.802 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T14:10:17.803 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T14:10:17.803 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2024-04-25T14:10:17.803 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T14:10:17.803 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T14:10:17.803 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T14:10:17.803 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T14:10:17.803 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T14:10:17.803 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:10:17.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:17.804 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2024-04-25T14:10:17.814 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:10:17.814 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:10:17.814 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:10:17.814 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:17.814 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2024-04-25T14:10:17.814 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:17.814 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:10:17.814 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2024-04-25T14:10:17.814 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f0 2024-04-25T14:10:17.814 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:10:17.815 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2024-04-25T14:10:17.815 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f1 2024-04-25T14:10:17.815 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:10:17.815 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2024-04-25T14:10:17.815 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T14:10:17.815 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2024-04-25T14:10:17.815 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T14:10:17.815 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 42040sec preferred_lft 42040sec 2024-04-25T14:10:17.815 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::ec4:7aff:fe88:6ded/64 scope link 2024-04-25T14:10:17.815 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:17.815 INFO:teuthology.orchestra.run.smithi155.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T14:10:17.815 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 02:42:b1:ff:60:50 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:10:17.816 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T14:10:17.816 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:17.816 INFO:teuthology.orchestra.run.smithi155.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T14:10:17.816 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T14:10:17.816 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T14:10:17.816 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:17.816 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T14:10:17.816 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:17.816 INFO:teuthology.orchestra.run.smithi155.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:10:17.816 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T14:10:17.816 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T14:10:17.816 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:17.817 INFO:teuthology.orchestra.run.smithi155.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:10:17.817 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T14:10:17.817 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T14:10:17.817 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:17.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:17.818 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T14:10:17.884 INFO:teuthology.orchestra.run.smithi155.stdout:1 2024-04-25T14:10:17.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:17.887 DEBUG:teuthology.orchestra.run.smithi155:> route 2024-04-25T14:10:17.940 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2024-04-25T14:10:17.940 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T14:10:17.940 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T14:10:17.940 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T14:10:17.940 INFO:teuthology.orchestra.run.smithi155.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T14:10:17.941 INFO:teuthology.orchestra.run.smithi155.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T14:10:17.941 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T14:10:17.941 INFO:teuthology.orchestra.run.smithi155.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:10:17.941 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:10:17.941 INFO:teuthology.orchestra.run.smithi155.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:10:17.941 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T14:10:17.942 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:10:17.942 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-04-25T14:10:17.942 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T14:10:17.942 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T14:10:17.942 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T14:10:17.942 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-04-25T14:10:18.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:10:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:10:18.084 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:10:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:10:18.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:18.092 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2024-04-25T14:10:18.100 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T14:10:18.100 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T14:10:18.101 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:10:18.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:18.102 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.0 up 2024-04-25T14:10:18.168 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:10:18.168 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T14:10:18.220 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:10:18.220 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:10:18.221 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T14:10:18.272 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T14:10:18.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:18.272 DEBUG:teuthology.orchestra.run.smithi155:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:10:18.372 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:10:18.372 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T14:10:18.372 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:10:18.390 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:10:18.390 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T14:10:18.390 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:10:18.390 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:10:18.390 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:10:18.400 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:10:18.400 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T14:10:18.400 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:10:18.400 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:10:18.400 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:10:18.400 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790" 2024-04-25T14:10:18.400 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790:/ 2024-04-25T14:10:18.401 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb" will pass to kernel 2024-04-25T14:10:18.404 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T14:10:18.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:18.405 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T14:10:18.424 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:10:18.510 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:10:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:10:18.513 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:10:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:10:18.520 INFO:teuthology.orchestra.run.smithi155.stdout:client7277 2024-04-25T14:10:18.520 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:10:18.520 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:10:18.529 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:10:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.685+0000 7fcfd6a00640 1 Processor -- start 2024-04-25T14:10:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.685+0000 7fcfd6a00640 1 -- start start 2024-04-25T14:10:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.689+0000 7fcfd6a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcfd00cfca0 0x7fcfd006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.689+0000 7fcfd6a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcfd00730b0 0x7fcfd00734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.689+0000 7fcfd6a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfd0072bd0 con 0x7fcfd00d16e0 2024-04-25T14:10:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.689+0000 7fcfd6a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fcfd0070470 con 0x7fcfd00730b0 2024-04-25T14:10:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.689+0000 7fcfd6a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcfd006f930 con 0x7fcfd00cfca0 2024-04-25T14:10:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.689+0000 7fcfc7e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fcfd00d16e0 0x7fcfd00d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51932/0 (socket says 172.21.15.86:51932) 2024-04-25T14:10:18.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.689+0000 7fcfc7e00640 1 -- 172.21.15.86:0/2619876112 learned_addr learned my addr 172.21.15.86:0/2619876112 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:18.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.689+0000 7fcfd4a00640 1 --2- 172.21.15.86:0/2619876112 >> v2:172.21.15.155:3300/0 conn(0x7fcfd00730b0 0x7fcfd00734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:18.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.689+0000 7fcfcfe00640 1 --2- 172.21.15.86:0/2619876112 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcfd00cfca0 0x7fcfd006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:18.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.689+0000 7fcfcf400640 1 -- 172.21.15.86:0/2619876112 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1496180706 0 0) 0x7fcfd0072bd0 con 0x7fcfd00d16e0 2024-04-25T14:10:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.689+0000 7fcfcf400640 1 -- 172.21.15.86:0/2619876112 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcfb4005740 con 0x7fcfd00d16e0 2024-04-25T14:10:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.689+0000 7fcfd4a00640 1 -- 172.21.15.86:0/2619876112 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcfd00cfca0 msgr2=0x7fcfd006f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.689+0000 7fcfd4a00640 1 --2- 172.21.15.86:0/2619876112 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcfd00cfca0 0x7fcfd006f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.689+0000 7fcfd4a00640 1 -- 172.21.15.86:0/2619876112 >> v1:172.21.15.86:6789/0 conn(0x7fcfd00d16e0 legacy=0x7fcfd00d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.689+0000 7fcfd4a00640 1 -- 172.21.15.86:0/2619876112 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd006fbd0 con 0x7fcfd00730b0 2024-04-25T14:10:18.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.693+0000 7fcfd4a00640 1 --2- 172.21.15.86:0/2619876112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfd00730b0 0x7fcfd00734c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8009d90 tx=0x7fcfc801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88051ab5ee2731ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:18.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.693+0000 7fcfcf400640 1 -- 172.21.15.86:0/2619876112 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc80b2070 con 0x7fcfd00730b0 2024-04-25T14:10:18.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.693+0000 7fcfcf400640 1 -- 172.21.15.86:0/2619876112 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfc80a5940 con 0x7fcfd00730b0 2024-04-25T14:10:18.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.693+0000 7fcfcf400640 1 -- 172.21.15.86:0/2619876112 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc80a5d10 con 0x7fcfd00730b0 2024-04-25T14:10:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.693+0000 7fcfd6a00640 1 -- 172.21.15.86:0/2619876112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfd00730b0 msgr2=0x7fcfd00734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.693+0000 7fcfd6a00640 1 --2- 172.21.15.86:0/2619876112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfd00730b0 0x7fcfd00734c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8009d90 tx=0x7fcfc801c970 comp rx=0 tx=0).stop 2024-04-25T14:10:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.693+0000 7fcfd6a00640 1 -- 172.21.15.86:0/2619876112 shutdown_connections 2024-04-25T14:10:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.693+0000 7fcfd6a00640 1 --2- 172.21.15.86:0/2619876112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfd00730b0 0x7fcfd00734c0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.693+0000 7fcfd6a00640 1 --2- 172.21.15.86:0/2619876112 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcfd00cfca0 0x7fcfd006f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.693+0000 7fcfd6a00640 1 -- 172.21.15.86:0/2619876112 >> 172.21.15.86:0/2619876112 conn(0x7fcfd00b4650 msgr2=0x7fcfd00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.693+0000 7fcfd6a00640 1 -- 172.21.15.86:0/2619876112 shutdown_connections 2024-04-25T14:10:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.693+0000 7fcfd6a00640 1 -- 172.21.15.86:0/2619876112 wait complete. 2024-04-25T14:10:18.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.693+0000 7fcfd6a00640 1 Processor -- start 2024-04-25T14:10:18.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.693+0000 7fcfd6a00640 1 -- start start 2024-04-25T14:10:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.697+0000 7fcfd6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfd00730b0 0x7fcfd0156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.697+0000 7fcfd6a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcfd00cfca0 0x7fcfd009a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.697+0000 7fcfd6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfd00d16e0 0x7fcfd015e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.697+0000 7fcfd6a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd00ce170 con 0x7fcfd00730b0 2024-04-25T14:10:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.697+0000 7fcfd6a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd00cdff0 con 0x7fcfd00d16e0 2024-04-25T14:10:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.697+0000 7fcfd6a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcfd00ce2f0 con 0x7fcfd00cfca0 2024-04-25T14:10:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.697+0000 7fcfcfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfd00730b0 0x7fcfd0156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.697+0000 7fcfcfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfd00730b0 0x7fcfd0156090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54114/0 (socket says 172.21.15.86:54114) 2024-04-25T14:10:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.697+0000 7fcfcfe00640 1 -- 172.21.15.86:0/2348560105 learned_addr learned my addr 172.21.15.86:0/2348560105 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:18.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.697+0000 7fcfc7e00640 1 --2- 172.21.15.86:0/2348560105 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcfd00cfca0 0x7fcfd009a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:18.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.697+0000 7fcfd4a00640 1 --2- 172.21.15.86:0/2348560105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfd00d16e0 0x7fcfd015e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:18.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.697+0000 7fcfcfe00640 1 -- 172.21.15.86:0/2348560105 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcfd00cfca0 msgr2=0x7fcfd009a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:18.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.697+0000 7fcfcfe00640 1 --2- 172.21.15.86:0/2348560105 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcfd00cfca0 0x7fcfd009a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:18.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.697+0000 7fcfcfe00640 1 -- 172.21.15.86:0/2348560105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfd00d16e0 msgr2=0x7fcfd015e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:18.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.697+0000 7fcfcfe00640 1 --2- 172.21.15.86:0/2348560105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfd00d16e0 0x7fcfd015e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:18.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.697+0000 7fcfcfe00640 1 -- 172.21.15.86:0/2348560105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd00b0f60 con 0x7fcfd00730b0 2024-04-25T14:10:18.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.697+0000 7fcfc7e00640 1 --2- 172.21.15.86:0/2348560105 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcfd00cfca0 0x7fcfd009a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:18.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.697+0000 7fcfd4a00640 1 --2- 172.21.15.86:0/2348560105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfd00d16e0 0x7fcfd015e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:18.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.697+0000 7fcfcfe00640 1 --2- 172.21.15.86:0/2348560105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfd00730b0 0x7fcfd0156090 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000e870 tx=0x7fcfc000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:18.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.697+0000 7fcfccc00640 1 -- 172.21.15.86:0/2348560105 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0017070 con 0x7fcfd00730b0 2024-04-25T14:10:18.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.697+0000 7fcfccc00640 1 -- 172.21.15.86:0/2348560105 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfc000c940 con 0x7fcfd00730b0 2024-04-25T14:10:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.697+0000 7fcfd6a00640 1 -- 172.21.15.86:0/2348560105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfd00b1240 con 0x7fcfd00730b0 2024-04-25T14:10:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.697+0000 7fcfccc00640 1 -- 172.21.15.86:0/2348560105 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc000cd10 con 0x7fcfd00730b0 2024-04-25T14:10:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.697+0000 7fcfd6a00640 1 -- 172.21.15.86:0/2348560105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfd00b1800 con 0x7fcfd00730b0 2024-04-25T14:10:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.697+0000 7fcfd6a00640 1 -- 172.21.15.86:0/2348560105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfd009f4b0 con 0x7fcfd00730b0 2024-04-25T14:10:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.701+0000 7fcfccc00640 1 -- 172.21.15.86:0/2348560105 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7fcfc001a070 con 0x7fcfd00730b0 2024-04-25T14:10:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.701+0000 7fcfccc00640 1 --2- 172.21.15.86:0/2348560105 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fcfac06f550 0x7fcfac071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.701+0000 7fcfccc00640 1 -- 172.21.15.86:0/2348560105 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7fcfc008f810 con 0x7fcfd00730b0 2024-04-25T14:10:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.701+0000 7fcfc7e00640 1 --2- 172.21.15.86:0/2348560105 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fcfac06f550 0x7fcfac071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.701+0000 7fcfc7e00640 1 --2- 172.21.15.86:0/2348560105 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fcfac06f550 0x7fcfac071a60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcfd009f480 tx=0x7fcfb8009040 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:18.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.705+0000 7fcfccc00640 1 -- 172.21.15.86:0/2348560105 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcfc005c460 con 0x7fcfd00730b0 2024-04-25T14:10:18.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.893+0000 7fcfd6a00640 1 -- 172.21.15.86:0/2348560105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcfd0069d00 con 0x7fcfd00730b0 2024-04-25T14:10:18.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.893+0000 7fcfccc00640 1 -- 172.21.15.86:0/2348560105 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5183 (secure 0 0 0) 0x7fcfc005bd60 con 0x7fcfd00730b0 2024-04-25T14:10:18.891 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:18.891 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":85,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/374465657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":374465657},{"type":"v1","addr":"172.21.15.86:6836","nonce":374465657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8107,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1466195534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":1466195534},{"type":"v1","addr":"172.21.15.86:6837","nonce":1466195534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8119,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2431580070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2431580070},{"type":"v1","addr":"172.21.15.155:6835","nonce":2431580070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:10:05.129470+0000","modified":"2024-04-25T14:10:08.145259+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8457":{"gid":8457,"name":"b","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6833/142240252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":142240252},{"type":"v1","addr":"172.21.15.155:6833","nonce":142240252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8457,"qdb_cluster":[8457]},"id":8}]} 2024-04-25T14:10:18.892 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 85 2024-04-25T14:10:18.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.897+0000 7fcfd6a00640 1 -- 172.21.15.86:0/2348560105 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fcfac06f550 msgr2=0x7fcfac071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:18.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.897+0000 7fcfd6a00640 1 --2- 172.21.15.86:0/2348560105 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fcfac06f550 0x7fcfac071a60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcfd009f480 tx=0x7fcfb8009040 comp rx=0 tx=0).stop 2024-04-25T14:10:18.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.897+0000 7fcfd6a00640 1 -- 172.21.15.86:0/2348560105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfd00730b0 msgr2=0x7fcfd0156090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:18.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.897+0000 7fcfd6a00640 1 --2- 172.21.15.86:0/2348560105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfd00730b0 0x7fcfd0156090 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000e870 tx=0x7fcfc000ec70 comp rx=0 tx=0).stop 2024-04-25T14:10:18.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.897+0000 7fcfd6a00640 1 -- 172.21.15.86:0/2348560105 shutdown_connections 2024-04-25T14:10:18.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.897+0000 7fcfd6a00640 1 --2- 172.21.15.86:0/2348560105 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fcfac06f550 0x7fcfac071a60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:18.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.897+0000 7fcfd6a00640 1 --2- 172.21.15.86:0/2348560105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfd00d16e0 0x7fcfd015e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:18.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.897+0000 7fcfd6a00640 1 --2- 172.21.15.86:0/2348560105 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcfd00cfca0 0x7fcfd009a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:18.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.897+0000 7fcfd6a00640 1 --2- 172.21.15.86:0/2348560105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfd00730b0 0x7fcfd0156090 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:18.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.897+0000 7fcfd6a00640 1 -- 172.21.15.86:0/2348560105 >> 172.21.15.86:0/2348560105 conn(0x7fcfd00b4650 msgr2=0x7fcfd006e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:18.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.897+0000 7fcfd6a00640 1 -- 172.21.15.86:0/2348560105 shutdown_connections 2024-04-25T14:10:18.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:18.897+0000 7fcfd6a00640 1 -- 172.21.15.86:0/2348560105 wait complete. 2024-04-25T14:10:18.913 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable stats 2024-04-25T14:10:19.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.065+0000 7f2260a00640 1 Processor -- start 2024-04-25T14:10:19.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.065+0000 7f2260a00640 1 -- start start 2024-04-25T14:10:19.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.065+0000 7f2260a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f225c15faa0 0x7f225c06d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:19.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.065+0000 7f2260a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f225c0b59e0 0x7f225c0b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:19.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.065+0000 7f2260a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f225c0b6330 con 0x7f225c0aeea0 2024-04-25T14:10:19.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.065+0000 7f2260a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f225c06d560 con 0x7f225c0b59e0 2024-04-25T14:10:19.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.065+0000 7f2260a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f225c06c670 con 0x7f225c15faa0 2024-04-25T14:10:19.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.065+0000 7f225a800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f225c0aeea0 0x7f225c0af2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48372/0 (socket says 172.21.15.86:48372) 2024-04-25T14:10:19.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.065+0000 7f225a800640 1 -- 172.21.15.86:0/2834689885 learned_addr learned my addr 172.21.15.86:0/2834689885 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:19.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.065+0000 7f2253e00640 1 --2- 172.21.15.86:0/2834689885 >> v2:172.21.15.155:3300/0 conn(0x7f225c0b59e0 0x7f225c0b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:19.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.065+0000 7f2259e00640 1 --2- 172.21.15.86:0/2834689885 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f225c15faa0 0x7f225c06d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:19.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.065+0000 7f2259400640 1 -- 172.21.15.86:0/2834689885 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3294924467 0 0) 0x7f22540014a0 con 0x7f225c0aeea0 2024-04-25T14:10:19.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.065+0000 7f2259400640 1 -- 172.21.15.86:0/2834689885 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2240003530 con 0x7f225c0aeea0 2024-04-25T14:10:19.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.065+0000 7f2253e00640 1 -- 172.21.15.86:0/2834689885 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f225c15faa0 msgr2=0x7f225c06d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:19.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.065+0000 7f2253e00640 1 --2- 172.21.15.86:0/2834689885 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f225c15faa0 0x7f225c06d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:19.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.065+0000 7f2253e00640 1 -- 172.21.15.86:0/2834689885 >> v1:172.21.15.86:6789/0 conn(0x7f225c0aeea0 legacy=0x7f225c0af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:19.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.065+0000 7f2253e00640 1 -- 172.21.15.86:0/2834689885 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225c06c950 con 0x7f225c0b59e0 2024-04-25T14:10:19.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.069+0000 7f2253e00640 1 --2- 172.21.15.86:0/2834689885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f225c0b59e0 0x7f225c0b5df0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f22440097c0 tx=0x7f224401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6cdc41ca2d91508 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:19.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.069+0000 7f2259400640 1 -- 172.21.15.86:0/2834689885 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22440aa600 con 0x7f225c0b59e0 2024-04-25T14:10:19.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.069+0000 7f2259400640 1 -- 172.21.15.86:0/2834689885 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22440af070 con 0x7f225c0b59e0 2024-04-25T14:10:19.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.069+0000 7f2259400640 1 -- 172.21.15.86:0/2834689885 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22440aa9a0 con 0x7f225c0b59e0 2024-04-25T14:10:19.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.069+0000 7f2260a00640 1 -- 172.21.15.86:0/2834689885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f225c0b59e0 msgr2=0x7f225c0b5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:19.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.069+0000 7f2260a00640 1 --2- 172.21.15.86:0/2834689885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f225c0b59e0 0x7f225c0b5df0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f22440097c0 tx=0x7f224401c4a0 comp rx=0 tx=0).stop 2024-04-25T14:10:19.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.069+0000 7f2260a00640 1 -- 172.21.15.86:0/2834689885 shutdown_connections 2024-04-25T14:10:19.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.069+0000 7f2260a00640 1 --2- 172.21.15.86:0/2834689885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f225c0b59e0 0x7f225c0b5df0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:19.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.069+0000 7f2260a00640 1 --2- 172.21.15.86:0/2834689885 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f225c15faa0 0x7f225c06d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:19.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.069+0000 7f2260a00640 1 -- 172.21.15.86:0/2834689885 >> 172.21.15.86:0/2834689885 conn(0x7f225c0d2230 msgr2=0x7f225c0d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:19.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.069+0000 7f2260a00640 1 -- 172.21.15.86:0/2834689885 shutdown_connections 2024-04-25T14:10:19.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.069+0000 7f2260a00640 1 -- 172.21.15.86:0/2834689885 wait complete. 2024-04-25T14:10:19.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.073+0000 7f2260a00640 1 Processor -- start 2024-04-25T14:10:19.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.073+0000 7f2260a00640 1 -- start start 2024-04-25T14:10:19.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.073+0000 7f2260a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f225c0aeea0 0x7f225c15b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:19.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.073+0000 7f2260a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f225c0b59e0 0x7f225c15dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:19.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.073+0000 7f2260a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f225c15faa0 0x7f225c154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:19.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.073+0000 7f2260a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f225c06df30 con 0x7f225c15faa0 2024-04-25T14:10:19.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.073+0000 7f2260a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f225c0af9c0 con 0x7f225c0b59e0 2024-04-25T14:10:19.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.073+0000 7f2260a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f225c15feb0 con 0x7f225c0aeea0 2024-04-25T14:10:19.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.073+0000 7f225a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f225c15faa0 0x7f225c154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:19.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.073+0000 7f225a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f225c15faa0 0x7f225c154ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49966/0 (socket says 172.21.15.86:49966) 2024-04-25T14:10:19.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.073+0000 7f225a800640 1 -- 172.21.15.86:0/1313424026 learned_addr learned my addr 172.21.15.86:0/1313424026 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:19.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.073+0000 7f2259e00640 1 --2- 172.21.15.86:0/1313424026 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f225c0aeea0 0x7f225c15b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:19.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.073+0000 7f2253e00640 1 --2- 172.21.15.86:0/1313424026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f225c0b59e0 0x7f225c15dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:19.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.073+0000 7f225a800640 1 -- 172.21.15.86:0/1313424026 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f225c0aeea0 msgr2=0x7f225c15b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:19.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.073+0000 7f225a800640 1 --2- 172.21.15.86:0/1313424026 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f225c0aeea0 0x7f225c15b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:19.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.073+0000 7f225a800640 1 -- 172.21.15.86:0/1313424026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f225c0b59e0 msgr2=0x7f225c15dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:19.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.073+0000 7f225a800640 1 --2- 172.21.15.86:0/1313424026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f225c0b59e0 0x7f225c15dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:19.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.073+0000 7f225a800640 1 -- 172.21.15.86:0/1313424026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225c1558d0 con 0x7f225c15faa0 2024-04-25T14:10:19.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.073+0000 7f2259e00640 1 --2- 172.21.15.86:0/1313424026 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f225c0aeea0 0x7f225c15b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:19.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.077+0000 7f225a800640 1 --2- 172.21.15.86:0/1313424026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f225c15faa0 0x7f225c154ba0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f225400cb10 tx=0x7f225400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:19.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.077+0000 7f2252000640 1 -- 172.21.15.86:0/1313424026 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2254017070 con 0x7f225c15faa0 2024-04-25T14:10:19.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.077+0000 7f2252000640 1 -- 172.21.15.86:0/1313424026 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f225400ac10 con 0x7f225c15faa0 2024-04-25T14:10:19.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.077+0000 7f2260a00640 1 -- 172.21.15.86:0/1313424026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f225c155bb0 con 0x7f225c15faa0 2024-04-25T14:10:19.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.077+0000 7f2252000640 1 -- 172.21.15.86:0/1313424026 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2254012400 con 0x7f225c15faa0 2024-04-25T14:10:19.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.077+0000 7f2260a00640 1 -- 172.21.15.86:0/1313424026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f225c15d5c0 con 0x7f225c15faa0 2024-04-25T14:10:19.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.077+0000 7f2260a00640 1 -- 172.21.15.86:0/1313424026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f225c074cf0 con 0x7f225c15faa0 2024-04-25T14:10:19.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.077+0000 7f2252000640 1 -- 172.21.15.86:0/1313424026 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 46) ==== 93309+0+0 (secure 0 0 0) 0x7f225401a070 con 0x7f225c15faa0 2024-04-25T14:10:19.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.077+0000 7f2252000640 1 --2- 172.21.15.86:0/1313424026 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f223806f520 0x7f2238071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:19.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.077+0000 7f2252000640 1 -- 172.21.15.86:0/1313424026 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f225408f610 con 0x7f225c15faa0 2024-04-25T14:10:19.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.081+0000 7f2259e00640 1 --2- 172.21.15.86:0/1313424026 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f223806f520 0x7f2238071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:19.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.081+0000 7f2259e00640 1 --2- 172.21.15.86:0/1313424026 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f223806f520 0x7f2238071a30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f224c009b10 tx=0x7f224c008040 comp rx=0 tx=0).ready entity=mgr.8190 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:19.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.081+0000 7f2252000640 1 -- 172.21.15.86:0/1313424026 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2254021020 con 0x7f225c15faa0 2024-04-25T14:10:19.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.281+0000 7f2260a00640 1 -- 172.21.15.86:0/1313424026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "stats"} v 0) -- 0x7f225c155660 con 0x7f225c15faa0 2024-04-25T14:10:19.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.549+0000 7f2252000640 1 -- 172.21.15.86:0/1313424026 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "stats"}]=0 v47) ==== 84+0+0 (secure 0 0 0) 0x7f225c0b6330 con 0x7f225c15faa0 2024-04-25T14:10:19.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.549+0000 7f2260a00640 1 -- 172.21.15.86:0/1313424026 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f223806f520 msgr2=0x7f2238071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:19.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.549+0000 7f2260a00640 1 --2- 172.21.15.86:0/1313424026 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f223806f520 0x7f2238071a30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f224c009b10 tx=0x7f224c008040 comp rx=0 tx=0).stop 2024-04-25T14:10:19.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.549+0000 7f2260a00640 1 -- 172.21.15.86:0/1313424026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f225c15faa0 msgr2=0x7f225c154ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:19.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.549+0000 7f2260a00640 1 --2- 172.21.15.86:0/1313424026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f225c15faa0 0x7f225c154ba0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f225400cb10 tx=0x7f225400a060 comp rx=0 tx=0).stop 2024-04-25T14:10:19.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.549+0000 7f2260a00640 1 -- 172.21.15.86:0/1313424026 shutdown_connections 2024-04-25T14:10:19.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.549+0000 7f2260a00640 1 --2- 172.21.15.86:0/1313424026 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7f223806f520 0x7f2238071a30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:19.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.549+0000 7f2260a00640 1 --2- 172.21.15.86:0/1313424026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f225c15faa0 0x7f225c154ba0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:19.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.549+0000 7f2260a00640 1 --2- 172.21.15.86:0/1313424026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f225c0b59e0 0x7f225c15dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:19.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.549+0000 7f2260a00640 1 --2- 172.21.15.86:0/1313424026 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f225c0aeea0 0x7f225c15b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:19.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.549+0000 7f2260a00640 1 -- 172.21.15.86:0/1313424026 >> 172.21.15.86:0/1313424026 conn(0x7f225c0d2230 msgr2=0x7f225c161d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:19.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.553+0000 7f2260a00640 1 -- 172.21.15.86:0/1313424026 shutdown_connections 2024-04-25T14:10:19.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.553+0000 7f2260a00640 1 -- 172.21.15.86:0/1313424026 wait complete. 2024-04-25T14:10:19.572 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats --mds_rank 1, 2024-04-25T14:10:19.707 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:10:19.710 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:10:19.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.733+0000 7fc83a600640 1 Processor -- start 2024-04-25T14:10:19.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.733+0000 7fc83a600640 1 -- start start 2024-04-25T14:10:19.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.733+0000 7fc83a600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8340cdd70 0x7fc8340ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:19.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.733+0000 7fc83a600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc83406e700 0x7fc83406eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:19.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.733+0000 7fc83a600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc83406f050 con 0x7fc834070e20 2024-04-25T14:10:19.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.733+0000 7fc83a600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc83406d940 con 0x7fc83406e700 2024-04-25T14:10:19.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.733+0000 7fc83a600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc83406dc20 con 0x7fc8340cdd70 2024-04-25T14:10:19.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.733+0000 7fc833e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc83406e700 0x7fc83406eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:19.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.733+0000 7fc832a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc834070e20 0x7fc834071230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48400/0 (socket says 172.21.15.86:48400) 2024-04-25T14:10:19.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.733+0000 7fc832a00640 1 -- 172.21.15.86:0/427440836 learned_addr learned my addr 172.21.15.86:0/427440836 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:19.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.733+0000 7fc833400640 1 --2- 172.21.15.86:0/427440836 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8340cdd70 0x7fc8340ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:19.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.733+0000 7fc832000640 1 -- 172.21.15.86:0/427440836 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1481117516 0 0) 0x7fc83406f050 con 0x7fc834070e20 2024-04-25T14:10:19.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.733+0000 7fc832000640 1 -- 172.21.15.86:0/427440836 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc818003530 con 0x7fc834070e20 2024-04-25T14:10:19.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.733+0000 7fc833e00640 1 -- 172.21.15.86:0/427440836 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8340cdd70 msgr2=0x7fc8340ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:19.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.733+0000 7fc833e00640 1 --2- 172.21.15.86:0/427440836 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8340cdd70 0x7fc8340ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:19.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.733+0000 7fc833e00640 1 -- 172.21.15.86:0/427440836 >> v1:172.21.15.86:6789/0 conn(0x7fc834070e20 legacy=0x7fc834071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:19.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.733+0000 7fc833e00640 1 -- 172.21.15.86:0/427440836 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83406df00 con 0x7fc83406e700 2024-04-25T14:10:19.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.741+0000 7fc833e00640 1 --2- 172.21.15.86:0/427440836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc83406e700 0x7fc83406eb10 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009d90 tx=0x7fc82c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c29073f22be10f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:19.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.741+0000 7fc832000640 1 -- 172.21.15.86:0/427440836 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c0b2070 con 0x7fc83406e700 2024-04-25T14:10:19.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.741+0000 7fc832000640 1 -- 172.21.15.86:0/427440836 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc82c0a57f0 con 0x7fc83406e700 2024-04-25T14:10:19.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.741+0000 7fc832000640 1 -- 172.21.15.86:0/427440836 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c0a5bc0 con 0x7fc83406e700 2024-04-25T14:10:19.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.741+0000 7fc83a600640 1 -- 172.21.15.86:0/427440836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc83406e700 msgr2=0x7fc83406eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:19.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.741+0000 7fc83a600640 1 --2- 172.21.15.86:0/427440836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc83406e700 0x7fc83406eb10 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009d90 tx=0x7fc82c01c8e0 comp rx=0 tx=0).stop 2024-04-25T14:10:19.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.741+0000 7fc83a600640 1 -- 172.21.15.86:0/427440836 shutdown_connections 2024-04-25T14:10:19.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.741+0000 7fc83a600640 1 --2- 172.21.15.86:0/427440836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc83406e700 0x7fc83406eb10 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:19.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.741+0000 7fc83a600640 1 --2- 172.21.15.86:0/427440836 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8340cdd70 0x7fc8340ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:19.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.741+0000 7fc83a600640 1 -- 172.21.15.86:0/427440836 >> 172.21.15.86:0/427440836 conn(0x7fc8340cffc0 msgr2=0x7fc8340d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:19.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.741+0000 7fc83a600640 1 -- 172.21.15.86:0/427440836 shutdown_connections 2024-04-25T14:10:19.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.741+0000 7fc83a600640 1 -- 172.21.15.86:0/427440836 wait complete. 2024-04-25T14:10:19.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.745+0000 7fc83a600640 1 Processor -- start 2024-04-25T14:10:19.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.745+0000 7fc83a600640 1 -- start start 2024-04-25T14:10:19.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.745+0000 7fc83a600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc83406e700 0x7fc8341517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:19.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.745+0000 7fc83a600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc834070e20 0x7fc834156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:19.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.745+0000 7fc83a600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8340cdd70 0x7fc834156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:19.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.745+0000 7fc83a600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc83416cc00 con 0x7fc834070e20 2024-04-25T14:10:19.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.745+0000 7fc83a600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc8341643c0 con 0x7fc83406e700 2024-04-25T14:10:19.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.745+0000 7fc83a600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc83416cd80 con 0x7fc8340cdd70 2024-04-25T14:10:19.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.745+0000 7fc832a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc834070e20 0x7fc834156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:19.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.745+0000 7fc832a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc834070e20 0x7fc834156790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49996/0 (socket says 172.21.15.86:49996) 2024-04-25T14:10:19.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.745+0000 7fc832a00640 1 -- 172.21.15.86:0/2204649348 learned_addr learned my addr 172.21.15.86:0/2204649348 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:19.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.745+0000 7fc833400640 1 --2- 172.21.15.86:0/2204649348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc83406e700 0x7fc8341517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:19.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.745+0000 7fc833e00640 1 --2- 172.21.15.86:0/2204649348 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8340cdd70 0x7fc834156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:19.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.745+0000 7fc832a00640 1 -- 172.21.15.86:0/2204649348 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8340cdd70 msgr2=0x7fc834156cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:19.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.745+0000 7fc832a00640 1 --2- 172.21.15.86:0/2204649348 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8340cdd70 0x7fc834156cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:19.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.745+0000 7fc832a00640 1 -- 172.21.15.86:0/2204649348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc83406e700 msgr2=0x7fc8341517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:19.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.745+0000 7fc832a00640 1 --2- 172.21.15.86:0/2204649348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc83406e700 0x7fc8341517d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:19.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.745+0000 7fc832a00640 1 -- 172.21.15.86:0/2204649348 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83415ad70 con 0x7fc834070e20 2024-04-25T14:10:19.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.745+0000 7fc832a00640 1 --2- 172.21.15.86:0/2204649348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc834070e20 0x7fc834156790 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fc82000c430 tx=0x7fc82000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:19.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.745+0000 7fc82b400640 1 -- 172.21.15.86:0/2204649348 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820017070 con 0x7fc834070e20 2024-04-25T14:10:19.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.745+0000 7fc82b400640 1 -- 172.21.15.86:0/2204649348 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc820007500 con 0x7fc834070e20 2024-04-25T14:10:19.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.745+0000 7fc82b400640 1 -- 172.21.15.86:0/2204649348 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8200059f0 con 0x7fc834070e20 2024-04-25T14:10:19.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.745+0000 7fc83a600640 1 -- 172.21.15.86:0/2204649348 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc83415f270 con 0x7fc834070e20 2024-04-25T14:10:19.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.745+0000 7fc83a600640 1 -- 172.21.15.86:0/2204649348 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc834153ad0 con 0x7fc834070e20 2024-04-25T14:10:19.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.749+0000 7fc83a600640 1 -- 172.21.15.86:0/2204649348 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc83415f4c0 con 0x7fc834070e20 2024-04-25T14:10:19.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.749+0000 7fc82b400640 1 -- 172.21.15.86:0/2204649348 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 47) ==== 93318+0+0 (secure 0 0 0) 0x7fc82001a070 con 0x7fc834070e20 2024-04-25T14:10:19.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.749+0000 7fc82b400640 1 --2- 172.21.15.86:0/2204649348 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fc81406f570 0x7fc814071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:19.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.749+0000 7fc833400640 1 -- 172.21.15.86:0/2204649348 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fc81406f570 msgr2=0x7fc814071a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/1354467416 2024-04-25T14:10:19.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.749+0000 7fc82b400640 1 -- 172.21.15.86:0/2204649348 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7fc82008e9a0 con 0x7fc834070e20 2024-04-25T14:10:19.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.749+0000 7fc833400640 1 --2- 172.21.15.86:0/2204649348 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fc81406f570 0x7fc814071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:10:19.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.753+0000 7fc82b400640 1 -- 172.21.15.86:0/2204649348 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc82005af90 con 0x7fc834070e20 2024-04-25T14:10:19.802 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:19.803+0000 7f090fa0c180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:10:19.816 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:19.817+0000 7f0f2e47d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:10:19.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.937+0000 7fc83a600640 1 -- 172.21.15.86:0/2204649348 --> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] -- mgr_command(tid 0: {"prefix": "fs perf stats", "mds_rank": "1,", "target": ["mon-mgr", ""]}) -- 0x7fc834069d60 con 0x7fc81406f570 2024-04-25T14:10:19.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.953+0000 7fc833400640 1 -- 172.21.15.86:0/2204649348 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fc81406f570 msgr2=0x7fc814071a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/1354467416 2024-04-25T14:10:19.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:19.953+0000 7fc833400640 1 --2- 172.21.15.86:0/2204649348 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fc81406f570 0x7fc814071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:10:20.219 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:20.219+0000 7f090fa0c180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:10:20.234 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:20.237+0000 7f0f2e47d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:10:20.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:20.357+0000 7fc833400640 1 -- 172.21.15.86:0/2204649348 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fc81406f570 msgr2=0x7fc814071a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/1354467416 2024-04-25T14:10:20.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:20.357+0000 7fc833400640 1 --2- 172.21.15.86:0/2204649348 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fc81406f570 0x7fc814071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T14:10:20.462 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:20.367+0000 7f090fa0c180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:10:20.463 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:20.385+0000 7f0f2e47d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:10:20.840 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:20.839+0000 7f090fa0c180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:10:20.853 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:20.857+0000 7f0f2e47d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:10:20.899 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:20.899+0000 7f090fa0c180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:10:20.912 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:20.913+0000 7f0f2e47d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:10:20.955 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:20.955+0000 7f090fa0c180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:10:20.967 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:20.969+0000 7f0f2e47d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:10:21.042 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:21.043+0000 7f090fa0c180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:10:21.054 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:21.057+0000 7f0f2e47d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:10:21.099 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:21.099+0000 7f090fa0c180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:10:21.110 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:21.113+0000 7f0f2e47d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:10:21.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:21.161+0000 7fc833400640 1 -- 172.21.15.86:0/2204649348 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fc81406f570 msgr2=0x7fc814071a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/1354467416 2024-04-25T14:10:21.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:21.161+0000 7fc833400640 1 --2- 172.21.15.86:0/2204649348 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fc81406f570 0x7fc814071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T14:10:21.183 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:21.183+0000 7f090fa0c180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:10:21.193 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:21.197+0000 7f0f2e47d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:10:21.242 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:21.243+0000 7f090fa0c180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:10:21.255 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:21.253+0000 7f0f2e47d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:10:21.393 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:21.395+0000 7f090fa0c180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:10:21.403 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:21.405+0000 7f0f2e47d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:10:21.452 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:21.451+0000 7f090fa0c180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:10:21.461 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:21.465+0000 7f0f2e47d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:10:21.695 INFO:tasks.ceph.mgr.x.smithi155.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:10:21.695 INFO:tasks.ceph.mgr.x.smithi155.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:10:21.695 INFO:tasks.ceph.mgr.x.smithi155.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:10:21.699 INFO:tasks.ceph.mgr.y.smithi086.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:10:21.700 INFO:tasks.ceph.mgr.y.smithi086.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:10:21.700 INFO:tasks.ceph.mgr.y.smithi086.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:10:21.700 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:21.699+0000 7f090fa0c180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:10:21.708 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:21.709+0000 7f0f2e47d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:10:21.854 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:21.855+0000 7f090fa0c180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:10:21.861 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:21.865+0000 7f0f2e47d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:10:21.979 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:21.979+0000 7f090fa0c180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:10:21.986 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:21.989+0000 7f0f2e47d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:10:22.049 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:22.051+0000 7f090fa0c180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:10:22.056 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:22.057+0000 7f0f2e47d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:10:22.196 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:22.195+0000 7f090fa0c180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:10:22.203 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:22.205+0000 7f0f2e47d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:10:22.738 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:22.739+0000 7f090fa0c180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:10:22.743 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:22.745+0000 7f0f2e47d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:10:22.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:22.765+0000 7fc833400640 1 -- 172.21.15.86:0/2204649348 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fc81406f570 msgr2=0x7fc814071a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/1354467416 2024-04-25T14:10:22.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:22.765+0000 7fc833400640 1 --2- 172.21.15.86:0/2204649348 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fc81406f570 0x7fc814071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T14:10:22.822 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:22.823+0000 7f090fa0c180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:10:22.827 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:22.829+0000 7f0f2e47d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:10:22.982 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:22.983+0000 7f090fa0c180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:10:22.986 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:22.989+0000 7f0f2e47d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:10:23.065 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:23.067+0000 7f090fa0c180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:10:23.069 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:23.073+0000 7f0f2e47d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:10:23.380 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:23.379+0000 7f090fa0c180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:10:23.386 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:23.389+0000 7f0f2e47d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:10:23.442 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:23.443+0000 7f090fa0c180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:10:23.447 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:23.449+0000 7f0f2e47d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:10:23.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:23.461+0000 7fc82b400640 1 -- 172.21.15.86:0/2204649348 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 48) ==== 92927+0+0 (secure 0 0 0) 0x7fc820057220 con 0x7fc834070e20 2024-04-25T14:10:23.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:23.461+0000 7fc82b400640 1 -- 172.21.15.86:0/2204649348 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fc81406f570 msgr2=0x7fc814071a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:10:23.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:23.461+0000 7fc82b400640 1 --2- 172.21.15.86:0/2204649348 >> [v2:172.21.15.86:6832/1354467416,v1:172.21.15.86:6833/1354467416] conn(0x7fc81406f570 0x7fc814071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:23.528 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:23.529+0000 7f0eaec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:10:23.528 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:23.529+0000 7f0eaec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:10:23.528 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:23.529+0000 7f0eaec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:10:23.528 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:23.529+0000 7f0eaec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:10:23.529 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:23.529+0000 7f0eaec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:10:23.529 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:23.529+0000 7f0eaa600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:10:23.529 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:23.529+0000 7f0eaa600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:10:23.529 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:23.529+0000 7f0eaa600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:10:23.529 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:23.529+0000 7f0eaa600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:10:23.529 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:23.529+0000 7f0eaa600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:10:24.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.465+0000 7fc82b400640 1 -- 172.21.15.86:0/2204649348 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mgrmap(e 49) ==== 93054+0+0 (secure 0 0 0) 0x7fc82007c540 con 0x7fc834070e20 2024-04-25T14:10:24.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.465+0000 7fc82b400640 1 --2- 172.21.15.86:0/2204649348 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fc8140383f0 0x7fc814079ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:24.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.465+0000 7fc82b400640 1 -- 172.21.15.86:0/2204649348 --> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] -- mgr_command(tid 0: {"prefix": "fs perf stats", "mds_rank": "1,", "target": ["mon-mgr", ""]}) -- 0x7fc820093e50 con 0x7fc8140383f0 2024-04-25T14:10:24.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.469+0000 7fc833400640 1 --2- 172.21.15.86:0/2204649348 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fc8140383f0 0x7fc814079ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.473+0000 7fc833400640 1 --2- 172.21.15.86:0/2204649348 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fc8140383f0 0x7fc814079ac0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc824009600 tx=0x7fc824009190 comp rx=0 tx=0).ready entity=mgr.8664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:24.473 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:24.477+0000 7f0eba000640 -1 mgr.server reply reply (22) Invalid argument invalid mds filter spec: 1, 2024-04-25T14:10:24.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.477+0000 7fc82b400640 1 -- 172.21.15.86:0/2204649348 <== mgr.8664 v2:172.21.15.86:6832/190090395 1 ==== mgr_command_reply(tid 0: -22 invalid mds filter spec: 1,) ==== 35+0+0 (secure 0 0 0) 0x7fc820093e50 con 0x7fc8140383f0 2024-04-25T14:10:24.473 INFO:teuthology.orchestra.run.smithi086.stderr:Error EINVAL: invalid mds filter spec: 1, 2024-04-25T14:10:24.485 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T14:10:24.486 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable stats 2024-04-25T14:10:24.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.641+0000 7f492bc00640 1 Processor -- start 2024-04-25T14:10:24.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.641+0000 7f492bc00640 1 -- start start 2024-04-25T14:10:24.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.641+0000 7f492bc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f49240cfca0 0x7f492406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:24.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.641+0000 7f492bc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49240d16e0 0x7f49240d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:24.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.641+0000 7f492bc00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f492406c670 0x7f492406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:24.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.641+0000 7f492bc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4924070470 con 0x7f49240cfca0 2024-04-25T14:10:24.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.641+0000 7f492bc00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f49240d2030 con 0x7f492406c670 2024-04-25T14:10:24.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.641+0000 7f492bc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4924072bd0 con 0x7f49240d16e0 2024-04-25T14:10:24.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.645+0000 7f4929200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f49240cfca0 0x7f492406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:24.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.645+0000 7f4929200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f49240cfca0 0x7f492406f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50100/0 (socket says 172.21.15.86:50100) 2024-04-25T14:10:24.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.645+0000 7f4929200640 1 -- 172.21.15.86:0/3147354731 learned_addr learned my addr 172.21.15.86:0/3147354731 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:24.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.645+0000 7f4929c00640 1 --2- 172.21.15.86:0/3147354731 >> v2:172.21.15.155:3300/0 conn(0x7f492406c670 0x7f492406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:24.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.645+0000 7f491be00640 1 --2- 172.21.15.86:0/3147354731 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49240d16e0 0x7f49240d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:24.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.645+0000 7f4929200640 1 -- 172.21.15.86:0/3147354731 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49240d16e0 msgr2=0x7f49240d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:24.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.645+0000 7f4929200640 1 --2- 172.21.15.86:0/3147354731 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49240d16e0 0x7f49240d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:24.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.645+0000 7f4929200640 1 -- 172.21.15.86:0/3147354731 >> v2:172.21.15.155:3300/0 conn(0x7f492406c670 msgr2=0x7f492406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:24.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.645+0000 7f4929200640 1 --2- 172.21.15.86:0/3147354731 >> v2:172.21.15.155:3300/0 conn(0x7f492406c670 0x7f492406ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:24.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.645+0000 7f4929200640 1 -- 172.21.15.86:0/3147354731 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f492406f930 con 0x7f49240cfca0 2024-04-25T14:10:24.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.645+0000 7f4929c00640 1 --2- 172.21.15.86:0/3147354731 >> v2:172.21.15.155:3300/0 conn(0x7f492406c670 0x7f492406ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:24.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.645+0000 7f4929200640 1 --2- 172.21.15.86:0/3147354731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49240cfca0 0x7f492406f3f0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f492406e060 tx=0x7f491401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd1e9acbd428c4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:24.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.645+0000 7f4923e00640 1 -- 172.21.15.86:0/3147354731 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49140ab5f0 con 0x7f49240cfca0 2024-04-25T14:10:24.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.645+0000 7f4923e00640 1 -- 172.21.15.86:0/3147354731 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49140b0070 con 0x7f49240cfca0 2024-04-25T14:10:24.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.645+0000 7f4923e00640 1 -- 172.21.15.86:0/3147354731 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49140ab9b0 con 0x7f49240cfca0 2024-04-25T14:10:24.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.649+0000 7f492bc00640 1 -- 172.21.15.86:0/3147354731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49240cfca0 msgr2=0x7f492406f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:24.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.649+0000 7f492bc00640 1 --2- 172.21.15.86:0/3147354731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49240cfca0 0x7f492406f3f0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f492406e060 tx=0x7f491401c520 comp rx=0 tx=0).stop 2024-04-25T14:10:24.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.649+0000 7f492bc00640 1 -- 172.21.15.86:0/3147354731 shutdown_connections 2024-04-25T14:10:24.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.649+0000 7f492bc00640 1 --2- 172.21.15.86:0/3147354731 >> v2:172.21.15.155:3300/0 conn(0x7f492406c670 0x7f492406ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:24.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.649+0000 7f492bc00640 1 --2- 172.21.15.86:0/3147354731 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49240d16e0 0x7f49240d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:24.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.649+0000 7f492bc00640 1 --2- 172.21.15.86:0/3147354731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49240cfca0 0x7f492406f3f0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:24.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.649+0000 7f492bc00640 1 -- 172.21.15.86:0/3147354731 >> 172.21.15.86:0/3147354731 conn(0x7f49240b4650 msgr2=0x7f49240b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:24.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.649+0000 7f492bc00640 1 -- 172.21.15.86:0/3147354731 shutdown_connections 2024-04-25T14:10:24.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.649+0000 7f492bc00640 1 -- 172.21.15.86:0/3147354731 wait complete. 2024-04-25T14:10:24.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.649+0000 7f492bc00640 1 Processor -- start 2024-04-25T14:10:24.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.649+0000 7f492bc00640 1 -- start start 2024-04-25T14:10:24.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.653+0000 7f492bc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f492406c670 0x7f49240c4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:24.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.653+0000 7f492bc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49240cfca0 0x7f4924156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:24.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.653+0000 7f492bc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49240d16e0 0x7f49240b1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:24.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.653+0000 7f492bc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f49240738c0 con 0x7f49240cfca0 2024-04-25T14:10:24.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.653+0000 7f492bc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f492406d3d0 con 0x7f49240d16e0 2024-04-25T14:10:24.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.653+0000 7f492bc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4924073a40 con 0x7f492406c670 2024-04-25T14:10:24.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.653+0000 7f491be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49240cfca0 0x7f4924156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:24.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.653+0000 7f491be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49240cfca0 0x7f4924156900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50108/0 (socket says 172.21.15.86:50108) 2024-04-25T14:10:24.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.653+0000 7f491be00640 1 -- 172.21.15.86:0/1099626029 learned_addr learned my addr 172.21.15.86:0/1099626029 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:24.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.653+0000 7f4929200640 1 --2- 172.21.15.86:0/1099626029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f492406c670 0x7f49240c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:24.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.653+0000 7f4929c00640 1 --2- 172.21.15.86:0/1099626029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49240d16e0 0x7f49240b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:24.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.653+0000 7f491be00640 1 -- 172.21.15.86:0/1099626029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f492406c670 msgr2=0x7f49240c4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:24.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.653+0000 7f491be00640 1 --2- 172.21.15.86:0/1099626029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f492406c670 0x7f49240c4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:24.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.653+0000 7f491be00640 1 -- 172.21.15.86:0/1099626029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49240d16e0 msgr2=0x7f49240b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:24.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.653+0000 7f491be00640 1 --2- 172.21.15.86:0/1099626029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49240d16e0 0x7f49240b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:24.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.653+0000 7f491be00640 1 -- 172.21.15.86:0/1099626029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f492415f070 con 0x7f49240cfca0 2024-04-25T14:10:24.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.653+0000 7f491be00640 1 --2- 172.21.15.86:0/1099626029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49240cfca0 0x7f4924156900 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f490c00d5f0 tx=0x7f490c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:24.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.653+0000 7f4921600640 1 -- 172.21.15.86:0/1099626029 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c0125f0 con 0x7f49240cfca0 2024-04-25T14:10:24.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.653+0000 7f4921600640 1 -- 172.21.15.86:0/1099626029 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f490c019070 con 0x7f49240cfca0 2024-04-25T14:10:24.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.653+0000 7f4921600640 1 -- 172.21.15.86:0/1099626029 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c0129b0 con 0x7f49240cfca0 2024-04-25T14:10:24.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.653+0000 7f492bc00640 1 -- 172.21.15.86:0/1099626029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f492415f350 con 0x7f49240cfca0 2024-04-25T14:10:24.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.653+0000 7f492bc00640 1 -- 172.21.15.86:0/1099626029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f492415d0f0 con 0x7f49240cfca0 2024-04-25T14:10:24.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.653+0000 7f492bc00640 1 -- 172.21.15.86:0/1099626029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f492409f4b0 con 0x7f49240cfca0 2024-04-25T14:10:24.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.657+0000 7f4921600640 1 -- 172.21.15.86:0/1099626029 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 49) ==== 93054+0+0 (secure 0 0 0) 0x7f490c012b50 con 0x7f49240cfca0 2024-04-25T14:10:24.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.657+0000 7f4921600640 1 --2- 172.21.15.86:0/1099626029 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f490006f2a0 0x7f49000717b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:24.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.657+0000 7f4921600640 1 -- 172.21.15.86:0/1099626029 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6445+0+0 (secure 0 0 0) 0x7f490c08f4e0 con 0x7f49240cfca0 2024-04-25T14:10:24.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.657+0000 7f4929200640 1 --2- 172.21.15.86:0/1099626029 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f490006f2a0 0x7f49000717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:24.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.657+0000 7f4929200640 1 --2- 172.21.15.86:0/1099626029 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f490006f2a0 0x7f49000717b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f49140af000 tx=0x7f49140a7000 comp rx=0 tx=0).ready entity=mgr.8664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:24.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.661+0000 7f4921600640 1 -- 172.21.15.86:0/1099626029 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f490c05bde0 con 0x7f49240cfca0 2024-04-25T14:10:24.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:24.857+0000 7f492bc00640 1 -- 172.21.15.86:0/1099626029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "stats"} v 0) -- 0x7f4924069d00 con 0x7f49240cfca0 2024-04-25T14:10:25.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.505+0000 7f4921600640 1 -- 172.21.15.86:0/1099626029 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "stats"}]=0 v50) ==== 85+0+0 (secure 0 0 0) 0x7f490c0600e0 con 0x7f49240cfca0 2024-04-25T14:10:25.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.509+0000 7f492bc00640 1 -- 172.21.15.86:0/1099626029 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f490006f2a0 msgr2=0x7f49000717b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:25.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.509+0000 7f492bc00640 1 --2- 172.21.15.86:0/1099626029 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f490006f2a0 0x7f49000717b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f49140af000 tx=0x7f49140a7000 comp rx=0 tx=0).stop 2024-04-25T14:10:25.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.509+0000 7f492bc00640 1 -- 172.21.15.86:0/1099626029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49240cfca0 msgr2=0x7f4924156900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:25.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.509+0000 7f492bc00640 1 --2- 172.21.15.86:0/1099626029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49240cfca0 0x7f4924156900 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f490c00d5f0 tx=0x7f490c00dab0 comp rx=0 tx=0).stop 2024-04-25T14:10:25.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.509+0000 7f492bc00640 1 -- 172.21.15.86:0/1099626029 shutdown_connections 2024-04-25T14:10:25.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.509+0000 7f492bc00640 1 --2- 172.21.15.86:0/1099626029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49240d16e0 0x7f49240b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:25.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.509+0000 7f492bc00640 1 --2- 172.21.15.86:0/1099626029 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f490006f2a0 0x7f49000717b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:25.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.509+0000 7f492bc00640 1 --2- 172.21.15.86:0/1099626029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49240cfca0 0x7f4924156900 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:25.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.509+0000 7f492bc00640 1 --2- 172.21.15.86:0/1099626029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f492406c670 0x7f49240c4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:25.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.509+0000 7f492bc00640 1 -- 172.21.15.86:0/1099626029 >> 172.21.15.86:0/1099626029 conn(0x7f49240b4650 msgr2=0x7f49240cd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:25.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.509+0000 7f492bc00640 1 -- 172.21.15.86:0/1099626029 shutdown_connections 2024-04-25T14:10:25.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.509+0000 7f492bc00640 1 -- 172.21.15.86:0/1099626029 wait complete. 2024-04-25T14:10:25.539 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T14:10:25.539 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:10:25.623 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:10:25.711 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T14:10:25.711 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:10:25.712 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:10:25.712 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:10:25.721 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:10:25.721 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:10:25.754 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:25.755+0000 7f3cba623180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:10:25.768 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:25.769+0000 7f6812189180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:10:25.773 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:10:25.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.965+0000 7f59bc000640 1 Processor -- start 2024-04-25T14:10:25.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.965+0000 7f59bc000640 1 -- start start 2024-04-25T14:10:25.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.969+0000 7f59bc000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f59b40d2c80 0x7f59b415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:25.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.969+0000 7f59bc000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59b4070e20 0x7f59b4071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:25.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.969+0000 7f59bc000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59b4071770 con 0x7f59b40722f0 2024-04-25T14:10:25.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.969+0000 7f59bc000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f59b40ce720 con 0x7f59b40d2c80 2024-04-25T14:10:25.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.969+0000 7f59bc000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f59b40cd8d0 con 0x7f59b4070e20 2024-04-25T14:10:25.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.969+0000 7f59ba000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f59b40722f0 0x7f59b4072700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48410/0 (socket says 172.21.15.86:48410) 2024-04-25T14:10:25.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.969+0000 7f59ba000640 1 -- 172.21.15.86:0/2784003424 learned_addr learned my addr 172.21.15.86:0/2784003424 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:25.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.969+0000 7f59b9600640 1 --2- 172.21.15.86:0/2784003424 >> v2:172.21.15.155:3300/0 conn(0x7f59b40d2c80 0x7f59b415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:25.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.969+0000 7f59b8c00640 1 --2- 172.21.15.86:0/2784003424 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59b4070e20 0x7f59b4071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:25.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.969+0000 7f59b3e00640 1 -- 172.21.15.86:0/2784003424 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1805635973 0 0) 0x7f59ac0014a0 con 0x7f59b40722f0 2024-04-25T14:10:25.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.969+0000 7f59b3e00640 1 -- 172.21.15.86:0/2784003424 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f599c003530 con 0x7f59b40722f0 2024-04-25T14:10:25.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.969+0000 7f59b9600640 1 -- 172.21.15.86:0/2784003424 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59b4070e20 msgr2=0x7f59b4071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:25.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.969+0000 7f59b9600640 1 --2- 172.21.15.86:0/2784003424 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59b4070e20 0x7f59b4071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:25.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.969+0000 7f59b9600640 1 -- 172.21.15.86:0/2784003424 >> v1:172.21.15.86:6789/0 conn(0x7f59b40722f0 legacy=0x7f59b4072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:25.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.969+0000 7f59b9600640 1 -- 172.21.15.86:0/2784003424 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b40ce970 con 0x7f59b40d2c80 2024-04-25T14:10:25.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.973+0000 7f59b9600640 1 --2- 172.21.15.86:0/2784003424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59b40d2c80 0x7f59b415dc90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f59a80097c0 tx=0x7f59a801ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=712d6d8af6318784 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:25.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.973+0000 7f59b3e00640 1 -- 172.21.15.86:0/2784003424 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a80b5070 con 0x7f59b40d2c80 2024-04-25T14:10:25.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.973+0000 7f59b3e00640 1 -- 172.21.15.86:0/2784003424 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a80a8990 con 0x7f59b40d2c80 2024-04-25T14:10:25.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.973+0000 7f59b3e00640 1 -- 172.21.15.86:0/2784003424 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a80a8d60 con 0x7f59b40d2c80 2024-04-25T14:10:25.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.973+0000 7f59bc000640 1 -- 172.21.15.86:0/2784003424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59b40d2c80 msgr2=0x7f59b415dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:25.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.973+0000 7f59bc000640 1 --2- 172.21.15.86:0/2784003424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59b40d2c80 0x7f59b415dc90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f59a80097c0 tx=0x7f59a801ec90 comp rx=0 tx=0).stop 2024-04-25T14:10:25.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.973+0000 7f59bc000640 1 -- 172.21.15.86:0/2784003424 shutdown_connections 2024-04-25T14:10:25.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.973+0000 7f59bc000640 1 --2- 172.21.15.86:0/2784003424 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59b4070e20 0x7f59b4071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:25.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.973+0000 7f59bc000640 1 --2- 172.21.15.86:0/2784003424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59b40d2c80 0x7f59b415dc90 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:25.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.973+0000 7f59bc000640 1 -- 172.21.15.86:0/2784003424 >> 172.21.15.86:0/2784003424 conn(0x7f59b40cffc0 msgr2=0x7f59b40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:25.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.973+0000 7f59bc000640 1 -- 172.21.15.86:0/2784003424 shutdown_connections 2024-04-25T14:10:25.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.973+0000 7f59bc000640 1 -- 172.21.15.86:0/2784003424 wait complete. 2024-04-25T14:10:25.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.977+0000 7f59bc000640 1 Processor -- start 2024-04-25T14:10:25.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.977+0000 7f59bc000640 1 -- start start 2024-04-25T14:10:25.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.977+0000 7f59bc000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59b4070e20 0x7f59b40c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:25.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.977+0000 7f59bc000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59b40722f0 0x7f59b414a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:25.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.977+0000 7f59bc000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59b40d2c80 0x7f59b414a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:25.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.977+0000 7f59bc000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f59b40d21a0 con 0x7f59b40d2c80 2024-04-25T14:10:25.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.977+0000 7f59bc000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f59b40d1ec0 con 0x7f59b40722f0 2024-04-25T14:10:25.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.977+0000 7f59bc000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f59b40d2480 con 0x7f59b4070e20 2024-04-25T14:10:25.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.977+0000 7f59ba000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59b40d2c80 0x7f59b414a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:25.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.977+0000 7f59b8c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59b40722f0 0x7f59b414a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:25.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.977+0000 7f59ba000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59b40d2c80 0x7f59b414a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50138/0 (socket says 172.21.15.86:50138) 2024-04-25T14:10:25.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.977+0000 7f59ba000640 1 -- 172.21.15.86:0/3543270220 learned_addr learned my addr 172.21.15.86:0/3543270220 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:25.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.977+0000 7f59b9600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59b4070e20 0x7f59b40c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:25.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.977+0000 7f59b8c00640 1 -- 172.21.15.86:0/3543270220 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59b4070e20 msgr2=0x7f59b40c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:25.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.977+0000 7f59b8c00640 1 --2- 172.21.15.86:0/3543270220 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59b4070e20 0x7f59b40c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:25.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.977+0000 7f59b8c00640 1 -- 172.21.15.86:0/3543270220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59b40d2c80 msgr2=0x7f59b414a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:25.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.977+0000 7f59b8c00640 1 --2- 172.21.15.86:0/3543270220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59b40d2c80 0x7f59b414a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:25.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.977+0000 7f59b8c00640 1 -- 172.21.15.86:0/3543270220 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b41604f0 con 0x7f59b40722f0 2024-04-25T14:10:25.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.977+0000 7f59ba000640 1 --2- 172.21.15.86:0/3543270220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59b40d2c80 0x7f59b414a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:10:25.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.977+0000 7f59b8c00640 1 --2- 172.21.15.86:0/3543270220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59b40722f0 0x7f59b414a400 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f59a000e8a0 tx=0x7f59a000eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:25.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.977+0000 7f59b1600640 1 -- 172.21.15.86:0/3543270220 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a0017070 con 0x7f59b40722f0 2024-04-25T14:10:25.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.977+0000 7f59b1600640 1 -- 172.21.15.86:0/3543270220 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a000c940 con 0x7f59b40722f0 2024-04-25T14:10:25.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.977+0000 7f59b1600640 1 -- 172.21.15.86:0/3543270220 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a000ccf0 con 0x7f59b40722f0 2024-04-25T14:10:25.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.977+0000 7f59bc000640 1 -- 172.21.15.86:0/3543270220 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b4160a70 con 0x7f59b40722f0 2024-04-25T14:10:25.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.977+0000 7f59bc000640 1 -- 172.21.15.86:0/3543270220 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b4160790 con 0x7f59b40722f0 2024-04-25T14:10:25.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.981+0000 7f59bc000640 1 -- 172.21.15.86:0/3543270220 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59b4076bf0 con 0x7f59b40722f0 2024-04-25T14:10:25.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.981+0000 7f59b1600640 1 -- 172.21.15.86:0/3543270220 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 50) ==== 93253+0+0 (secure 0 0 0) 0x7f59a001a070 con 0x7f59b40722f0 2024-04-25T14:10:25.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.981+0000 7f59b1600640 1 --2- 172.21.15.86:0/3543270220 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f599406f490 0x7f59940719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:25.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.981+0000 7f59b9600640 1 -- 172.21.15.86:0/3543270220 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f599406f490 msgr2=0x7f59940719a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/190090395 2024-04-25T14:10:25.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.981+0000 7f59b1600640 1 -- 172.21.15.86:0/3543270220 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6445+0+0 (secure 0 0 0) 0x7f59a008f4f0 con 0x7f59b40722f0 2024-04-25T14:10:25.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.981+0000 7f59b9600640 1 --2- 172.21.15.86:0/3543270220 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f599406f490 0x7f59940719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:10:25.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:25.985+0000 7f59b1600640 1 -- 172.21.15.86:0/3543270220 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f59a005ba50 con 0x7f59b40722f0 2024-04-25T14:10:26.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.149+0000 7f59bc000640 1 -- 172.21.15.86:0/3543270220 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59b40d1470 con 0x7f59b40722f0 2024-04-25T14:10:26.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.149+0000 7f59b1600640 1 -- 172.21.15.86:0/3543270220 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) ==== 74+0+17451 (secure 0 0 0) 0x7f59a005fd50 con 0x7f59b40722f0 2024-04-25T14:10:26.147 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:26.147 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":114,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:10:23.455025+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:10:03.520649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T14:10:04.080912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2052329939":"2024-04-26T14:10:23.454991+0000","172.21.15.86:0/1021944523":"2024-04-26T14:10:23.454991+0000","172.21.15.86:0/83615604":"2024-04-26T14:10:23.454991+0000","172.21.15.86:0/3719613294":"2024-04-26T14:10:23.454991+0000","172.21.15.86:0/1471025278":"2024-04-26T14:10:23.454991+0000","172.21.15.86:6833/1354467416":"2024-04-26T14:10:23.454991+0000","172.21.15.86:6832/1354467416":"2024-04-26T14:10:23.454991+0000","172.21.15.86:0/3000491277":"2024-04-26T14:10:23.454991+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:10:26.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.153+0000 7f59bc000640 1 -- 172.21.15.86:0/3543270220 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f599406f490 msgr2=0x7f59940719a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:10:26.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.153+0000 7f59bc000640 1 --2- 172.21.15.86:0/3543270220 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f599406f490 0x7f59940719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:26.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.153+0000 7f59bc000640 1 -- 172.21.15.86:0/3543270220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59b40722f0 msgr2=0x7f59b414a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:26.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.153+0000 7f59bc000640 1 --2- 172.21.15.86:0/3543270220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59b40722f0 0x7f59b414a400 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f59a000e8a0 tx=0x7f59a000eca0 comp rx=0 tx=0).stop 2024-04-25T14:10:26.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.153+0000 7f59bc000640 1 -- 172.21.15.86:0/3543270220 shutdown_connections 2024-04-25T14:10:26.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.153+0000 7f59bc000640 1 --2- 172.21.15.86:0/3543270220 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f599406f490 0x7f59940719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:26.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.153+0000 7f59bc000640 1 --2- 172.21.15.86:0/3543270220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59b40d2c80 0x7f59b414a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:26.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.153+0000 7f59bc000640 1 --2- 172.21.15.86:0/3543270220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59b40722f0 0x7f59b414a400 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:26.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.153+0000 7f59bc000640 1 --2- 172.21.15.86:0/3543270220 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59b4070e20 0x7f59b40c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:26.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.153+0000 7f59bc000640 1 -- 172.21.15.86:0/3543270220 >> 172.21.15.86:0/3543270220 conn(0x7f59b40cffc0 msgr2=0x7f59b415ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:26.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.153+0000 7f59bc000640 1 -- 172.21.15.86:0/3543270220 shutdown_connections 2024-04-25T14:10:26.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.153+0000 7f59bc000640 1 -- 172.21.15.86:0/3543270220 wait complete. 2024-04-25T14:10:26.171 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:10:03.520649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:10:04.080912+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:10:26.172 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:10:26.175 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:26.175+0000 7f3cba623180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:10:26.202 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:26.205+0000 7f6812189180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:10:26.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.329+0000 7f9bdaa00640 1 Processor -- start 2024-04-25T14:10:26.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.329+0000 7f9bdaa00640 1 -- start start 2024-04-25T14:10:26.326 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:26.327+0000 7f3cba623180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:10:26.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.329+0000 7f9bdaa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bd4071530 0x7f9bd415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:26.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.329+0000 7f9bdaa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9bd41616e0 0x7f9bd4161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:26.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.329+0000 7f9bdaa00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bd40cd8d0 con 0x7f9bd40d2c80 2024-04-25T14:10:26.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.329+0000 7f9bdaa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9bd40ce720 con 0x7f9bd41616e0 2024-04-25T14:10:26.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.329+0000 7f9bdaa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9bd40ce970 con 0x7f9bd4071530 2024-04-25T14:10:26.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.329+0000 7f9bd8a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9bd41616e0 0x7f9bd4161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:26.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.329+0000 7f9bd3e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9bd40d2c80 0x7f9bd4070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48424/0 (socket says 172.21.15.86:48424) 2024-04-25T14:10:26.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.329+0000 7f9bd3e00640 1 -- 172.21.15.86:0/2714520470 learned_addr learned my addr 172.21.15.86:0/2714520470 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:26.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.329+0000 7f9bd3400640 1 --2- 172.21.15.86:0/2714520470 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bd4071530 0x7f9bd415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:26.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.329+0000 7f9bd8a00640 1 -- 172.21.15.86:0/2714520470 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bd4071530 msgr2=0x7f9bd415dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:26.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.329+0000 7f9bd8a00640 1 --2- 172.21.15.86:0/2714520470 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bd4071530 0x7f9bd415dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:26.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.329+0000 7f9bd8a00640 1 -- 172.21.15.86:0/2714520470 >> v1:172.21.15.86:6789/0 conn(0x7f9bd40d2c80 legacy=0x7f9bd4070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:26.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.329+0000 7f9bd8a00640 1 -- 172.21.15.86:0/2714520470 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd40d1be0 con 0x7f9bd41616e0 2024-04-25T14:10:26.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.329+0000 7f9bd2a00640 1 -- 172.21.15.86:0/2714520470 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 425641924 0 0) 0x7f9bd40cd8d0 con 0x7f9bd40d2c80 2024-04-25T14:10:26.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.333+0000 7f9bd8a00640 1 --2- 172.21.15.86:0/2714520470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bd41616e0 0x7f9bd4161af0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc009d90 tx=0x7f9bcc01f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84590cbd5344b8ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:26.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.333+0000 7f9bd2a00640 1 -- 172.21.15.86:0/2714520470 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc0b4070 con 0x7f9bd41616e0 2024-04-25T14:10:26.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.333+0000 7f9bd2a00640 1 -- 172.21.15.86:0/2714520470 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bcc0af600 con 0x7f9bd41616e0 2024-04-25T14:10:26.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.333+0000 7f9bd2a00640 1 -- 172.21.15.86:0/2714520470 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc0af9d0 con 0x7f9bd41616e0 2024-04-25T14:10:26.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.333+0000 7f9bdaa00640 1 -- 172.21.15.86:0/2714520470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bd41616e0 msgr2=0x7f9bd4161af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:26.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.333+0000 7f9bdaa00640 1 --2- 172.21.15.86:0/2714520470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bd41616e0 0x7f9bd4161af0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc009d90 tx=0x7f9bcc01f150 comp rx=0 tx=0).stop 2024-04-25T14:10:26.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.333+0000 7f9bdaa00640 1 -- 172.21.15.86:0/2714520470 shutdown_connections 2024-04-25T14:10:26.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.333+0000 7f9bdaa00640 1 --2- 172.21.15.86:0/2714520470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bd41616e0 0x7f9bd4161af0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:26.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.333+0000 7f9bdaa00640 1 --2- 172.21.15.86:0/2714520470 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bd4071530 0x7f9bd415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:26.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.333+0000 7f9bdaa00640 1 -- 172.21.15.86:0/2714520470 >> 172.21.15.86:0/2714520470 conn(0x7f9bd40cffc0 msgr2=0x7f9bd40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:26.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.333+0000 7f9bdaa00640 1 -- 172.21.15.86:0/2714520470 shutdown_connections 2024-04-25T14:10:26.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.333+0000 7f9bdaa00640 1 -- 172.21.15.86:0/2714520470 wait complete. 2024-04-25T14:10:26.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.337+0000 7f9bdaa00640 1 Processor -- start 2024-04-25T14:10:26.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.337+0000 7f9bdaa00640 1 -- start start 2024-04-25T14:10:26.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.337+0000 7f9bdaa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bd40d2c80 0x7f9bd40ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:26.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.337+0000 7f9bdaa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bd41616e0 0x7f9bd415e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:26.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.337+0000 7f9bdaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bd414b550 0x7f9bd414b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:26.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.337+0000 7f9bdaa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd40d2480 con 0x7f9bd414b550 2024-04-25T14:10:26.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.337+0000 7f9bdaa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd40d21a0 con 0x7f9bd41616e0 2024-04-25T14:10:26.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.337+0000 7f9bdaa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9bd40d29a0 con 0x7f9bd40d2c80 2024-04-25T14:10:26.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.337+0000 7f9bd8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bd414b550 0x7f9bd414b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:26.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.337+0000 7f9bd8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bd414b550 0x7f9bd414b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50158/0 (socket says 172.21.15.86:50158) 2024-04-25T14:10:26.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.337+0000 7f9bd3e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bd40d2c80 0x7f9bd40ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:26.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.337+0000 7f9bd3400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bd41616e0 0x7f9bd415e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:26.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.337+0000 7f9bd8a00640 1 -- 172.21.15.86:0/3829628661 learned_addr learned my addr 172.21.15.86:0/3829628661 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:26.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.337+0000 7f9bd3400640 1 -- 172.21.15.86:0/3829628661 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bd40d2c80 msgr2=0x7f9bd40ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:26.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.337+0000 7f9bd3400640 1 --2- 172.21.15.86:0/3829628661 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bd40d2c80 0x7f9bd40ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:26.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.337+0000 7f9bd3400640 1 -- 172.21.15.86:0/3829628661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bd414b550 msgr2=0x7f9bd414b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:26.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.337+0000 7f9bd3400640 1 --2- 172.21.15.86:0/3829628661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bd414b550 0x7f9bd414b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:26.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.337+0000 7f9bd3400640 1 -- 172.21.15.86:0/3829628661 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd406e3a0 con 0x7f9bd41616e0 2024-04-25T14:10:26.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.337+0000 7f9bd8a00640 1 --2- 172.21.15.86:0/3829628661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bd414b550 0x7f9bd414b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:10:26.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.337+0000 7f9bd3400640 1 --2- 172.21.15.86:0/3829628661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bd41616e0 0x7f9bd415e620 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000e8d0 tx=0x7f9bc000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:26.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.341+0000 7f9bcbe00640 1 -- 172.21.15.86:0/3829628661 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0017070 con 0x7f9bd41616e0 2024-04-25T14:10:26.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.341+0000 7f9bcbe00640 1 -- 172.21.15.86:0/3829628661 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bc000c940 con 0x7f9bd41616e0 2024-04-25T14:10:26.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.341+0000 7f9bcbe00640 1 -- 172.21.15.86:0/3829628661 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc000ccf0 con 0x7f9bd41616e0 2024-04-25T14:10:26.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.341+0000 7f9bdaa00640 1 -- 172.21.15.86:0/3829628661 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd406e680 con 0x7f9bd41616e0 2024-04-25T14:10:26.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.341+0000 7f9bdaa00640 1 -- 172.21.15.86:0/3829628661 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd406ec40 con 0x7f9bd41616e0 2024-04-25T14:10:26.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.341+0000 7f9bdaa00640 1 -- 172.21.15.86:0/3829628661 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bd4076bf0 con 0x7f9bd41616e0 2024-04-25T14:10:26.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.341+0000 7f9bcbe00640 1 -- 172.21.15.86:0/3829628661 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 50) ==== 93253+0+0 (secure 0 0 0) 0x7f9bc001a070 con 0x7f9bd41616e0 2024-04-25T14:10:26.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.341+0000 7f9bcbe00640 1 --2- 172.21.15.86:0/3829628661 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f9bb406f4c0 0x7f9bb40719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:26.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.341+0000 7f9bd3e00640 1 -- 172.21.15.86:0/3829628661 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f9bb406f4c0 msgr2=0x7f9bb40719d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/190090395 2024-04-25T14:10:26.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.341+0000 7f9bcbe00640 1 -- 172.21.15.86:0/3829628661 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6445+0+0 (secure 0 0 0) 0x7f9bc008f490 con 0x7f9bd41616e0 2024-04-25T14:10:26.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.341+0000 7f9bd3e00640 1 --2- 172.21.15.86:0/3829628661 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f9bb406f4c0 0x7f9bb40719d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:10:26.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.345+0000 7f9bcbe00640 1 -- 172.21.15.86:0/3829628661 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9bc005bb00 con 0x7f9bd41616e0 2024-04-25T14:10:26.359 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:26.361+0000 7f6812189180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:10:26.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.505+0000 7f9bdaa00640 1 -- 172.21.15.86:0/3829628661 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9bd40d1470 con 0x7f9bd41616e0 2024-04-25T14:10:26.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.505+0000 7f9bcbe00640 1 -- 172.21.15.86:0/3829628661 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) ==== 93+0+10 (secure 0 0 0) 0x7f9bc005fe00 con 0x7f9bd41616e0 2024-04-25T14:10:26.503 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:10:26.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.509+0000 7f9bdaa00640 1 -- 172.21.15.86:0/3829628661 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f9bb406f4c0 msgr2=0x7f9bb40719d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:10:26.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.509+0000 7f9bdaa00640 1 --2- 172.21.15.86:0/3829628661 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f9bb406f4c0 0x7f9bb40719d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:26.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.509+0000 7f9bdaa00640 1 -- 172.21.15.86:0/3829628661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bd41616e0 msgr2=0x7f9bd415e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:26.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.509+0000 7f9bdaa00640 1 --2- 172.21.15.86:0/3829628661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bd41616e0 0x7f9bd415e620 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000e8d0 tx=0x7f9bc000ecd0 comp rx=0 tx=0).stop 2024-04-25T14:10:26.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.509+0000 7f9bdaa00640 1 -- 172.21.15.86:0/3829628661 shutdown_connections 2024-04-25T14:10:26.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.509+0000 7f9bdaa00640 1 --2- 172.21.15.86:0/3829628661 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f9bb406f4c0 0x7f9bb40719d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:26.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.509+0000 7f9bdaa00640 1 --2- 172.21.15.86:0/3829628661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bd414b550 0x7f9bd414b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:26.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.509+0000 7f9bdaa00640 1 --2- 172.21.15.86:0/3829628661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bd41616e0 0x7f9bd415e620 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:26.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.509+0000 7f9bdaa00640 1 --2- 172.21.15.86:0/3829628661 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bd40d2c80 0x7f9bd40ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:26.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.509+0000 7f9bdaa00640 1 -- 172.21.15.86:0/3829628661 >> 172.21.15.86:0/3829628661 conn(0x7f9bd40cffc0 msgr2=0x7f9bd415ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:26.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.509+0000 7f9bdaa00640 1 -- 172.21.15.86:0/3829628661 shutdown_connections 2024-04-25T14:10:26.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.509+0000 7f9bdaa00640 1 -- 172.21.15.86:0/3829628661 wait complete. 2024-04-25T14:10:26.526 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:10:26.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.721+0000 7f5e3aa00640 1 Processor -- start 2024-04-25T14:10:26.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.721+0000 7f5e3aa00640 1 -- start start 2024-04-25T14:10:26.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.725+0000 7f5e3aa00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5e34071530 0x7f5e340ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:26.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.725+0000 7f5e3aa00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e3406d940 con 0x7f5e3406e700 2024-04-25T14:10:26.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.725+0000 7f5e3aa00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e3406dc20 con 0x7f5e340cdd70 2024-04-25T14:10:26.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.725+0000 7f5e3aa00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f5e3406df00 con 0x7f5e34071530 2024-04-25T14:10:26.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.725+0000 7f5e38a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5e3406e700 0x7f5e3406eb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48440/0 (socket says 172.21.15.86:48440) 2024-04-25T14:10:26.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.725+0000 7f5e38a00640 1 -- 172.21.15.86:0/1382095567 learned_addr learned my addr 172.21.15.86:0/1382095567 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:26.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.725+0000 7f5e33400640 1 --2- 172.21.15.86:0/1382095567 >> v2:172.21.15.155:3300/0 conn(0x7f5e34071530 0x7f5e340ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:26.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.725+0000 7f5e32a00640 1 -- 172.21.15.86:0/1382095567 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2954495210 0 0) 0x7f5e3406d940 con 0x7f5e3406e700 2024-04-25T14:10:26.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.725+0000 7f5e32a00640 1 -- 172.21.15.86:0/1382095567 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e18003530 con 0x7f5e3406e700 2024-04-25T14:10:26.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.725+0000 7f5e32a00640 1 -- 172.21.15.86:0/1382095567 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2102674893 0 0) 0x7f5e3406dc20 con 0x7f5e340cdd70 2024-04-25T14:10:26.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.725+0000 7f5e32a00640 1 -- 172.21.15.86:0/1382095567 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e3406d940 con 0x7f5e340cdd70 2024-04-25T14:10:26.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.725+0000 7f5e32a00640 1 -- 172.21.15.86:0/1382095567 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 869874720 0 0) 0x7f5e18003530 con 0x7f5e3406e700 2024-04-25T14:10:26.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.725+0000 7f5e33400640 1 -- 172.21.15.86:0/1382095567 >> v1:172.21.15.155:6789/0 conn(0x7f5e340cdd70 legacy=0x7f5e34070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:26.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.725+0000 7f5e33400640 1 -- 172.21.15.86:0/1382095567 >> v1:172.21.15.86:6789/0 conn(0x7f5e3406e700 legacy=0x7f5e3406eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:26.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.725+0000 7f5e33400640 1 -- 172.21.15.86:0/1382095567 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e3406e1e0 con 0x7f5e34071530 2024-04-25T14:10:26.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.729+0000 7f5e33400640 1 --2- 172.21.15.86:0/1382095567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e34071530 0x7f5e340ce180 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5e200097a0 tx=0x7f5e2001c310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61b6882ba6734b8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.729+0000 7f5e32a00640 1 -- 172.21.15.86:0/1382095567 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e200b0070 con 0x7f5e34071530 2024-04-25T14:10:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.729+0000 7f5e32a00640 1 -- 172.21.15.86:0/1382095567 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e200ab400 con 0x7f5e34071530 2024-04-25T14:10:26.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.729+0000 7f5e32a00640 1 -- 172.21.15.86:0/1382095567 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c001a00 con 0x7f5e34071530 2024-04-25T14:10:26.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.729+0000 7f5e3aa00640 1 -- 172.21.15.86:0/1382095567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e34071530 msgr2=0x7f5e340ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:26.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.729+0000 7f5e3aa00640 1 --2- 172.21.15.86:0/1382095567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e34071530 0x7f5e340ce180 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5e200097a0 tx=0x7f5e2001c310 comp rx=0 tx=0).stop 2024-04-25T14:10:26.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.733+0000 7f5e3aa00640 1 -- 172.21.15.86:0/1382095567 shutdown_connections 2024-04-25T14:10:26.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.733+0000 7f5e3aa00640 1 --2- 172.21.15.86:0/1382095567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e34071530 0x7f5e340ce180 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:26.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.733+0000 7f5e3aa00640 1 -- 172.21.15.86:0/1382095567 >> 172.21.15.86:0/1382095567 conn(0x7f5e340cffc0 msgr2=0x7f5e340d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:26.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.733+0000 7f5e3aa00640 1 -- 172.21.15.86:0/1382095567 shutdown_connections 2024-04-25T14:10:26.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.733+0000 7f5e3aa00640 1 -- 172.21.15.86:0/1382095567 wait complete. 2024-04-25T14:10:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.733+0000 7f5e3aa00640 1 Processor -- start 2024-04-25T14:10:26.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.733+0000 7f5e3aa00640 1 -- start start 2024-04-25T14:10:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.737+0000 7f5e3aa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e3406e700 0x7f5e3415c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.737+0000 7f5e3aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e340cdd70 0x7f5e3414d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.737+0000 7f5e3aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e3414ff00 0x7f5e34162080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.737+0000 7f5e3aa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3416dc10 con 0x7f5e340cdd70 2024-04-25T14:10:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.737+0000 7f5e3aa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5e341643c0 con 0x7f5e3414ff00 2024-04-25T14:10:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.737+0000 7f5e3aa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5e3416dd90 con 0x7f5e3406e700 2024-04-25T14:10:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.737+0000 7f5e33400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e340cdd70 0x7f5e3414d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.737+0000 7f5e33400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e340cdd70 0x7f5e3414d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50164/0 (socket says 172.21.15.86:50164) 2024-04-25T14:10:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.737+0000 7f5e33400640 1 -- 172.21.15.86:0/4070992467 learned_addr learned my addr 172.21.15.86:0/4070992467 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.737+0000 7f5e33e00640 1 --2- 172.21.15.86:0/4070992467 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e3406e700 0x7f5e3415c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.737+0000 7f5e38a00640 1 --2- 172.21.15.86:0/4070992467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e3414ff00 0x7f5e34162080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.737+0000 7f5e33400640 1 -- 172.21.15.86:0/4070992467 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e3406e700 msgr2=0x7f5e3415c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.737+0000 7f5e33400640 1 --2- 172.21.15.86:0/4070992467 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e3406e700 0x7f5e3415c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.737+0000 7f5e33400640 1 -- 172.21.15.86:0/4070992467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e3414ff00 msgr2=0x7f5e34162080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:26.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.737+0000 7f5e33400640 1 --2- 172.21.15.86:0/4070992467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e3414ff00 0x7f5e34162080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.737+0000 7f5e33400640 1 -- 172.21.15.86:0/4070992467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e3414d900 con 0x7f5e340cdd70 2024-04-25T14:10:26.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.737+0000 7f5e33400640 1 --2- 172.21.15.86:0/4070992467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e340cdd70 0x7f5e3414d0b0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f5e20019f40 tx=0x7f5e200190b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:26.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.737+0000 7f5e2be00640 1 -- 172.21.15.86:0/4070992467 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e200b0040 con 0x7f5e340cdd70 2024-04-25T14:10:26.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.737+0000 7f5e2be00640 1 -- 172.21.15.86:0/4070992467 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e20005ba0 con 0x7f5e340cdd70 2024-04-25T14:10:26.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.737+0000 7f5e2be00640 1 -- 172.21.15.86:0/4070992467 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e20006dc0 con 0x7f5e340cdd70 2024-04-25T14:10:26.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.737+0000 7f5e3aa00640 1 -- 172.21.15.86:0/4070992467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e3415ce90 con 0x7f5e340cdd70 2024-04-25T14:10:26.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.737+0000 7f5e3aa00640 1 -- 172.21.15.86:0/4070992467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e340caa00 con 0x7f5e340cdd70 2024-04-25T14:10:26.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.737+0000 7f5e3aa00640 1 -- 172.21.15.86:0/4070992467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e340720e0 con 0x7f5e340cdd70 2024-04-25T14:10:26.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.741+0000 7f5e2be00640 1 -- 172.21.15.86:0/4070992467 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 50) ==== 93253+0+0 (secure 0 0 0) 0x7f5e20006030 con 0x7f5e340cdd70 2024-04-25T14:10:26.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.741+0000 7f5e2be00640 1 --2- 172.21.15.86:0/4070992467 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f5e1406f4c0 0x7f5e140719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:26.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.741+0000 7f5e33e00640 1 -- 172.21.15.86:0/4070992467 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f5e1406f4c0 msgr2=0x7f5e140719d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/190090395 2024-04-25T14:10:26.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.741+0000 7f5e2be00640 1 -- 172.21.15.86:0/4070992467 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6445+0+0 (secure 0 0 0) 0x7f5e20128ed0 con 0x7f5e340cdd70 2024-04-25T14:10:26.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.741+0000 7f5e33e00640 1 --2- 172.21.15.86:0/4070992467 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f5e1406f4c0 0x7f5e140719d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:10:26.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.745+0000 7f5e2be00640 1 -- 172.21.15.86:0/4070992467 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5e200c2060 con 0x7f5e340cdd70 2024-04-25T14:10:26.805 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:26.807+0000 7f3cba623180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:10:26.847 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:26.849+0000 7f6812189180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:10:26.863 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:26.863+0000 7f3cba623180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:10:26.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.905+0000 7f5e3aa00640 1 -- 172.21.15.86:0/4070992467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5e34076bf0 con 0x7f5e340cdd70 2024-04-25T14:10:26.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.905+0000 7f5e2be00640 1 -- 172.21.15.86:0/4070992467 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v114) ==== 104+0+11 (secure 0 0 0) 0x7f5e20015f90 con 0x7f5e340cdd70 2024-04-25T14:10:26.903 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:10:26.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.905+0000 7f5e3aa00640 1 -- 172.21.15.86:0/4070992467 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f5e1406f4c0 msgr2=0x7f5e140719d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:10:26.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.905+0000 7f5e3aa00640 1 --2- 172.21.15.86:0/4070992467 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f5e1406f4c0 0x7f5e140719d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:26.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.905+0000 7f5e3aa00640 1 -- 172.21.15.86:0/4070992467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e340cdd70 msgr2=0x7f5e3414d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:26.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.905+0000 7f5e3aa00640 1 --2- 172.21.15.86:0/4070992467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e340cdd70 0x7f5e3414d0b0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f5e20019f40 tx=0x7f5e200190b0 comp rx=0 tx=0).stop 2024-04-25T14:10:26.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.909+0000 7f5e3aa00640 1 -- 172.21.15.86:0/4070992467 shutdown_connections 2024-04-25T14:10:26.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.909+0000 7f5e3aa00640 1 --2- 172.21.15.86:0/4070992467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e3414ff00 0x7f5e34162080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:26.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.909+0000 7f5e3aa00640 1 --2- 172.21.15.86:0/4070992467 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f5e1406f4c0 0x7f5e140719d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:26.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.909+0000 7f5e3aa00640 1 --2- 172.21.15.86:0/4070992467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e340cdd70 0x7f5e3414d0b0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:26.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.909+0000 7f5e3aa00640 1 --2- 172.21.15.86:0/4070992467 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e3406e700 0x7f5e3415c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:26.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.909+0000 7f5e3aa00640 1 -- 172.21.15.86:0/4070992467 >> 172.21.15.86:0/4070992467 conn(0x7f5e340cffc0 msgr2=0x7f5e34070e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:26.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.909+0000 7f5e3aa00640 1 -- 172.21.15.86:0/4070992467 shutdown_connections 2024-04-25T14:10:26.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:26.909+0000 7f5e3aa00640 1 -- 172.21.15.86:0/4070992467 wait complete. 2024-04-25T14:10:26.907 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:26.909+0000 7f6812189180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:10:26.920 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:26.923+0000 7f3cba623180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:10:26.926 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:10:26.965 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:26.969+0000 7f6812189180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:10:27.007 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:27.007+0000 7f3cba623180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:10:27.056 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:27.057+0000 7f6812189180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:10:27.064 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:27.063+0000 7f3cba623180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:10:27.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.081+0000 7f8a21200640 1 Processor -- start 2024-04-25T14:10:27.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.081+0000 7f8a21200640 1 -- start start 2024-04-25T14:10:27.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.081+0000 7f8a21200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a1c0b25a0 0x7f8a1c09a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:27.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.081+0000 7f8a21200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a1c0cdb60 con 0x7f8a1c06d4b0 2024-04-25T14:10:27.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.081+0000 7f8a21200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a1c06dc00 con 0x7f8a1c0cb930 2024-04-25T14:10:27.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.081+0000 7f8a21200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8a1c0ce470 con 0x7f8a1c0b25a0 2024-04-25T14:10:27.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.081+0000 7f8a19e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8a1c06d4b0 0x7f8a1c0b1e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48446/0 (socket says 172.21.15.86:48446) 2024-04-25T14:10:27.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.081+0000 7f8a19e00640 1 -- 172.21.15.86:0/1745219933 learned_addr learned my addr 172.21.15.86:0/1745219933 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:27.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.081+0000 7f8a19400640 1 --2- 172.21.15.86:0/1745219933 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a1c0b25a0 0x7f8a1c09a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:27.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.081+0000 7f8a18a00640 1 -- 172.21.15.86:0/1745219933 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3484614255 0 0) 0x7f8a1c0cdb60 con 0x7f8a1c06d4b0 2024-04-25T14:10:27.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.105+0000 7f8a18a00640 1 -- 172.21.15.86:0/1745219933 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a00003510 con 0x7f8a1c06d4b0 2024-04-25T14:10:27.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.105+0000 7f8a18a00640 1 -- 172.21.15.86:0/1745219933 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 730612757 0 0) 0x7f8a1c06dc00 con 0x7f8a1c0cb930 2024-04-25T14:10:27.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.105+0000 7f8a18a00640 1 -- 172.21.15.86:0/1745219933 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a1c0cdb60 con 0x7f8a1c0cb930 2024-04-25T14:10:27.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.109+0000 7f8a19400640 1 -- 172.21.15.86:0/1745219933 >> v1:172.21.15.155:6789/0 conn(0x7f8a1c0cb930 legacy=0x7f8a1c0cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:27.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.109+0000 7f8a19400640 1 -- 172.21.15.86:0/1745219933 >> v1:172.21.15.86:6789/0 conn(0x7f8a1c06d4b0 legacy=0x7f8a1c0b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:27.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.109+0000 7f8a19400640 1 -- 172.21.15.86:0/1745219933 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1c0cde20 con 0x7f8a1c0b25a0 2024-04-25T14:10:27.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.109+0000 7f8a19400640 1 --2- 172.21.15.86:0/1745219933 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a1c0b25a0 0x7f8a1c09a360 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8a08007170 tx=0x7f8a0801c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f373d9faa81c675 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:27.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.109+0000 7f8a18a00640 1 -- 172.21.15.86:0/1745219933 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a080b0070 con 0x7f8a1c0b25a0 2024-04-25T14:10:27.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.109+0000 7f8a18a00640 1 -- 172.21.15.86:0/1745219933 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a08007ad0 con 0x7f8a1c0b25a0 2024-04-25T14:10:27.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.109+0000 7f8a18a00640 1 -- 172.21.15.86:0/1745219933 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a080090d0 con 0x7f8a1c0b25a0 2024-04-25T14:10:27.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.113+0000 7f8a21200640 1 -- 172.21.15.86:0/1745219933 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a1c0b25a0 msgr2=0x7f8a1c09a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:27.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.113+0000 7f8a21200640 1 --2- 172.21.15.86:0/1745219933 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a1c0b25a0 0x7f8a1c09a360 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8a08007170 tx=0x7f8a0801c390 comp rx=0 tx=0).stop 2024-04-25T14:10:27.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.113+0000 7f8a21200640 1 -- 172.21.15.86:0/1745219933 shutdown_connections 2024-04-25T14:10:27.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.113+0000 7f8a21200640 1 --2- 172.21.15.86:0/1745219933 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a1c0b25a0 0x7f8a1c09a360 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:27.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.113+0000 7f8a21200640 1 -- 172.21.15.86:0/1745219933 >> 172.21.15.86:0/1745219933 conn(0x7f8a1c0b6130 msgr2=0x7f8a1c0b6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:27.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.113+0000 7f8a21200640 1 -- 172.21.15.86:0/1745219933 shutdown_connections 2024-04-25T14:10:27.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.113+0000 7f8a21200640 1 -- 172.21.15.86:0/1745219933 wait complete. 2024-04-25T14:10:27.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.113+0000 7f8a21200640 1 Processor -- start 2024-04-25T14:10:27.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.113+0000 7f8a21200640 1 -- start start 2024-04-25T14:10:27.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.117+0000 7f8a21200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a1c06d4b0 0x7f8a1c0caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:27.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.117+0000 7f8a21200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a1c0b25a0 0x7f8a1c0caf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:27.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.117+0000 7f8a21200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a1c0cb930 0x7f8a1c0d0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:27.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.117+0000 7f8a21200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1c09acb0 con 0x7f8a1c0b25a0 2024-04-25T14:10:27.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.117+0000 7f8a21200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1c09ab30 con 0x7f8a1c0cb930 2024-04-25T14:10:27.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.117+0000 7f8a21200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8a1c17b690 con 0x7f8a1c06d4b0 2024-04-25T14:10:27.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.117+0000 7f8a19400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a1c0b25a0 0x7f8a1c0caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:27.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.117+0000 7f8a19400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a1c0b25a0 0x7f8a1c0caf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50180/0 (socket says 172.21.15.86:50180) 2024-04-25T14:10:27.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.117+0000 7f8a19400640 1 -- 172.21.15.86:0/2570934775 learned_addr learned my addr 172.21.15.86:0/2570934775 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:27.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.117+0000 7f8a19e00640 1 --2- 172.21.15.86:0/2570934775 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a1c06d4b0 0x7f8a1c0caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:27.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.117+0000 7f8a1a800640 1 --2- 172.21.15.86:0/2570934775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a1c0cb930 0x7f8a1c0d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:27.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.117+0000 7f8a19400640 1 -- 172.21.15.86:0/2570934775 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a1c06d4b0 msgr2=0x7f8a1c0caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:27.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.117+0000 7f8a19400640 1 --2- 172.21.15.86:0/2570934775 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a1c06d4b0 0x7f8a1c0caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:27.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.117+0000 7f8a19400640 1 -- 172.21.15.86:0/2570934775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a1c0cb930 msgr2=0x7f8a1c0d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:27.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.117+0000 7f8a19400640 1 --2- 172.21.15.86:0/2570934775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a1c0cb930 0x7f8a1c0d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:27.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.117+0000 7f8a19400640 1 -- 172.21.15.86:0/2570934775 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1c0c8ec0 con 0x7f8a1c0b25a0 2024-04-25T14:10:27.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.117+0000 7f8a19e00640 1 --2- 172.21.15.86:0/2570934775 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a1c06d4b0 0x7f8a1c0caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:10:27.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.117+0000 7f8a19400640 1 --2- 172.21.15.86:0/2570934775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a1c0b25a0 0x7f8a1c0caf90 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f8a0801aa40 tx=0x7f8a08019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:27.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.117+0000 7f8a12000640 1 -- 172.21.15.86:0/2570934775 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a080b0040 con 0x7f8a1c0b25a0 2024-04-25T14:10:27.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.117+0000 7f8a12000640 1 -- 172.21.15.86:0/2570934775 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a080b4430 con 0x7f8a1c0b25a0 2024-04-25T14:10:27.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.117+0000 7f8a21200640 1 -- 172.21.15.86:0/2570934775 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a1c0c91a0 con 0x7f8a1c0b25a0 2024-04-25T14:10:27.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.117+0000 7f8a12000640 1 -- 172.21.15.86:0/2570934775 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08006e20 con 0x7f8a1c0b25a0 2024-04-25T14:10:27.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.117+0000 7f8a21200640 1 -- 172.21.15.86:0/2570934775 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a1c0c9760 con 0x7f8a1c0b25a0 2024-04-25T14:10:27.117 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:27.117+0000 7f6812189180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:10:27.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.117+0000 7f8a21200640 1 -- 172.21.15.86:0/2570934775 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a1c069d00 con 0x7f8a1c0b25a0 2024-04-25T14:10:27.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.121+0000 7f8a12000640 1 -- 172.21.15.86:0/2570934775 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 50) ==== 93253+0+0 (secure 0 0 0) 0x7f8a080be060 con 0x7f8a1c0b25a0 2024-04-25T14:10:27.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.121+0000 7f8a12000640 1 --2- 172.21.15.86:0/2570934775 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f89fc06f490 0x7f89fc0719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:27.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.121+0000 7f8a19e00640 1 -- 172.21.15.86:0/2570934775 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f89fc06f490 msgr2=0x7f89fc0719a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/190090395 2024-04-25T14:10:27.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.121+0000 7f8a12000640 1 -- 172.21.15.86:0/2570934775 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6445+0+0 (secure 0 0 0) 0x7f8a08128ac0 con 0x7f8a1c0b25a0 2024-04-25T14:10:27.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.121+0000 7f8a19e00640 1 --2- 172.21.15.86:0/2570934775 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f89fc06f490 0x7f89fc0719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:10:27.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.125+0000 7f8a12000640 1 -- 172.21.15.86:0/2570934775 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8a08015f80 con 0x7f8a1c0b25a0 2024-04-25T14:10:27.150 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:27.151+0000 7f3cba623180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:10:27.203 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:27.205+0000 7f6812189180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:10:27.210 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:27.211+0000 7f3cba623180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:10:27.264 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:27.265+0000 7f6812189180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:10:27.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.289+0000 7f8a21200640 1 -- 172.21.15.86:0/2570934775 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8a1c076bf0 con 0x7f8a1c0b25a0 2024-04-25T14:10:27.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.289+0000 7f8a12000640 1 -- 172.21.15.86:0/2570934775 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v114) ==== 100+0+11 (secure 0 0 0) 0x7f8a080f5010 con 0x7f8a1c0b25a0 2024-04-25T14:10:27.286 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:10:27.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.289+0000 7f8a21200640 1 -- 172.21.15.86:0/2570934775 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f89fc06f490 msgr2=0x7f89fc0719a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:10:27.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.289+0000 7f8a21200640 1 --2- 172.21.15.86:0/2570934775 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f89fc06f490 0x7f89fc0719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:27.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.289+0000 7f8a21200640 1 -- 172.21.15.86:0/2570934775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a1c0b25a0 msgr2=0x7f8a1c0caf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:27.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.289+0000 7f8a21200640 1 --2- 172.21.15.86:0/2570934775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a1c0b25a0 0x7f8a1c0caf90 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f8a0801aa40 tx=0x7f8a08019130 comp rx=0 tx=0).stop 2024-04-25T14:10:27.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.289+0000 7f8a21200640 1 -- 172.21.15.86:0/2570934775 shutdown_connections 2024-04-25T14:10:27.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.289+0000 7f8a21200640 1 --2- 172.21.15.86:0/2570934775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a1c0cb930 0x7f8a1c0d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:27.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.289+0000 7f8a21200640 1 --2- 172.21.15.86:0/2570934775 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f89fc06f490 0x7f89fc0719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:27.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.289+0000 7f8a21200640 1 --2- 172.21.15.86:0/2570934775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a1c0b25a0 0x7f8a1c0caf90 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:27.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.289+0000 7f8a21200640 1 --2- 172.21.15.86:0/2570934775 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a1c06d4b0 0x7f8a1c0caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:27.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.289+0000 7f8a21200640 1 -- 172.21.15.86:0/2570934775 >> 172.21.15.86:0/2570934775 conn(0x7f8a1c0b6130 msgr2=0x7f8a1c068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:27.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.289+0000 7f8a21200640 1 -- 172.21.15.86:0/2570934775 shutdown_connections 2024-04-25T14:10:27.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.289+0000 7f8a21200640 1 -- 172.21.15.86:0/2570934775 wait complete. 2024-04-25T14:10:27.309 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T14:10:27.362 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:27.363+0000 7f3cba623180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:10:27.421 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:27.423+0000 7f3cba623180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:10:27.422 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:27.425+0000 7f6812189180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:10:27.482 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:27.485+0000 7f6812189180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:10:27.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.505+0000 7f25c0c00640 1 Processor -- start 2024-04-25T14:10:27.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.505+0000 7f25c0c00640 1 -- start start 2024-04-25T14:10:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.505+0000 7f25c0c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f25bc0d2c80 0x7f25bc15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.505+0000 7f25c0c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f25bc070e20 0x7f25bc071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.505+0000 7f25c0c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25bc071770 con 0x7f25bc0722f0 2024-04-25T14:10:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.505+0000 7f25c0c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f25bc0ce720 con 0x7f25bc0d2c80 2024-04-25T14:10:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.505+0000 7f25c0c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f25bc0cd8d0 con 0x7f25bc070e20 2024-04-25T14:10:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.505+0000 7f25b9e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f25bc0d2c80 0x7f25bc15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.505+0000 7f25b9e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f25bc0d2c80 0x7f25bc15dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50190/0 (socket says 172.21.15.86:50190) 2024-04-25T14:10:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.505+0000 7f25b9e00640 1 -- 172.21.15.86:0/1789146883 learned_addr learned my addr 172.21.15.86:0/1789146883 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.505+0000 7f25b9400640 1 --2- 172.21.15.86:0/1789146883 >> v2:172.21.15.155:3300/0 conn(0x7f25bc070e20 0x7f25bc071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.509+0000 7f25b9e00640 1 -- 172.21.15.86:0/1789146883 >> v2:172.21.15.155:3300/0 conn(0x7f25bc070e20 msgr2=0x7f25bc071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.509+0000 7f25b9e00640 1 --2- 172.21.15.86:0/1789146883 >> v2:172.21.15.155:3300/0 conn(0x7f25bc070e20 0x7f25bc071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.509+0000 7f25b9e00640 1 -- 172.21.15.86:0/1789146883 >> v1:172.21.15.155:6789/0 conn(0x7f25bc0722f0 legacy=0x7f25bc072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.509+0000 7f25b9e00640 1 -- 172.21.15.86:0/1789146883 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25bc0ce970 con 0x7f25bc0d2c80 2024-04-25T14:10:27.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.509+0000 7f25b9e00640 1 --2- 172.21.15.86:0/1789146883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25bc0d2c80 0x7f25bc15dc90 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f25bc0b6070 tx=0x7f25ac01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218be825ca3cf10c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:27.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.509+0000 7f25b8a00640 1 -- 172.21.15.86:0/1789146883 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ac0b5070 con 0x7f25bc0d2c80 2024-04-25T14:10:27.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.509+0000 7f25b8a00640 1 -- 172.21.15.86:0/1789146883 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25ac0a8a10 con 0x7f25bc0d2c80 2024-04-25T14:10:27.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.509+0000 7f25b8a00640 1 -- 172.21.15.86:0/1789146883 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ac0a8de0 con 0x7f25bc0d2c80 2024-04-25T14:10:27.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.513+0000 7f25c0c00640 1 -- 172.21.15.86:0/1789146883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25bc0d2c80 msgr2=0x7f25bc15dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:27.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.513+0000 7f25c0c00640 1 --2- 172.21.15.86:0/1789146883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25bc0d2c80 0x7f25bc15dc90 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f25bc0b6070 tx=0x7f25ac01ed10 comp rx=0 tx=0).stop 2024-04-25T14:10:27.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.513+0000 7f25c0c00640 1 -- 172.21.15.86:0/1789146883 shutdown_connections 2024-04-25T14:10:27.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.513+0000 7f25c0c00640 1 --2- 172.21.15.86:0/1789146883 >> v2:172.21.15.155:3300/0 conn(0x7f25bc070e20 0x7f25bc071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:27.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.513+0000 7f25c0c00640 1 --2- 172.21.15.86:0/1789146883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25bc0d2c80 0x7f25bc15dc90 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:27.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.513+0000 7f25c0c00640 1 -- 172.21.15.86:0/1789146883 >> 172.21.15.86:0/1789146883 conn(0x7f25bc0cffc0 msgr2=0x7f25bc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:27.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.513+0000 7f25c0c00640 1 -- 172.21.15.86:0/1789146883 shutdown_connections 2024-04-25T14:10:27.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.513+0000 7f25c0c00640 1 -- 172.21.15.86:0/1789146883 wait complete. 2024-04-25T14:10:27.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.513+0000 7f25c0c00640 1 Processor -- start 2024-04-25T14:10:27.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.513+0000 7f25c0c00640 1 -- start start 2024-04-25T14:10:27.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.517+0000 7f25c0c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25bc070e20 0x7f25bc0c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:27.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.517+0000 7f25c0c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25bc0722f0 0x7f25bc0b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:27.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.517+0000 7f25c0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25bc0d2c80 0x7f25bc0b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:27.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.517+0000 7f25c0c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc0d21a0 con 0x7f25bc0d2c80 2024-04-25T14:10:27.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.517+0000 7f25c0c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc0d1ec0 con 0x7f25bc070e20 2024-04-25T14:10:27.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.517+0000 7f25c0c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f25bc0d2480 con 0x7f25bc0722f0 2024-04-25T14:10:27.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.517+0000 7f25ba800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25bc0d2c80 0x7f25bc0b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:27.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.517+0000 7f25b9e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25bc070e20 0x7f25bc0c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:27.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.517+0000 7f25ba800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25bc0d2c80 0x7f25bc0b0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50192/0 (socket says 172.21.15.86:50192) 2024-04-25T14:10:27.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.517+0000 7f25b9400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25bc0722f0 0x7f25bc0b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:27.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.517+0000 7f25ba800640 1 -- 172.21.15.86:0/869687736 learned_addr learned my addr 172.21.15.86:0/869687736 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:27.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.517+0000 7f25ba800640 1 -- 172.21.15.86:0/869687736 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25bc0722f0 msgr2=0x7f25bc0b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:27.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.517+0000 7f25ba800640 1 --2- 172.21.15.86:0/869687736 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25bc0722f0 0x7f25bc0b08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:27.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.517+0000 7f25ba800640 1 -- 172.21.15.86:0/869687736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25bc070e20 msgr2=0x7f25bc0c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:27.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.517+0000 7f25ba800640 1 --2- 172.21.15.86:0/869687736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25bc070e20 0x7f25bc0c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:27.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.517+0000 7f25ba800640 1 -- 172.21.15.86:0/869687736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25bc0c9810 con 0x7f25bc0d2c80 2024-04-25T14:10:27.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.517+0000 7f25b9e00640 1 --2- 172.21.15.86:0/869687736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25bc070e20 0x7f25bc0c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:10:27.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.517+0000 7f25ba800640 1 --2- 172.21.15.86:0/869687736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25bc0d2c80 0x7f25bc0b0e20 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f25b400c760 tx=0x7f25b400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:27.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.517+0000 7f25b2000640 1 -- 172.21.15.86:0/869687736 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b4003b80 con 0x7f25bc0d2c80 2024-04-25T14:10:27.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.517+0000 7f25b2000640 1 -- 172.21.15.86:0/869687736 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25b4003d20 con 0x7f25bc0d2c80 2024-04-25T14:10:27.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.517+0000 7f25c0c00640 1 -- 172.21.15.86:0/869687736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25bc0b1590 con 0x7f25bc0d2c80 2024-04-25T14:10:27.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.517+0000 7f25b2000640 1 -- 172.21.15.86:0/869687736 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b400aa90 con 0x7f25bc0d2c80 2024-04-25T14:10:27.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.517+0000 7f25c0c00640 1 -- 172.21.15.86:0/869687736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25bc0b1b50 con 0x7f25bc0d2c80 2024-04-25T14:10:27.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.517+0000 7f25c0c00640 1 -- 172.21.15.86:0/869687736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25bc0720e0 con 0x7f25bc0d2c80 2024-04-25T14:10:27.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.521+0000 7f25b2000640 1 -- 172.21.15.86:0/869687736 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 50) ==== 93253+0+0 (secure 0 0 0) 0x7f25b400ac30 con 0x7f25bc0d2c80 2024-04-25T14:10:27.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.521+0000 7f25b2000640 1 --2- 172.21.15.86:0/869687736 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f259c06f490 0x7f259c0719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:27.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.521+0000 7f25b9e00640 1 -- 172.21.15.86:0/869687736 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f259c06f490 msgr2=0x7f259c0719a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/190090395 2024-04-25T14:10:27.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.521+0000 7f25b2000640 1 -- 172.21.15.86:0/869687736 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6445+0+0 (secure 0 0 0) 0x7f25b408f2b0 con 0x7f25bc0d2c80 2024-04-25T14:10:27.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.521+0000 7f25b9e00640 1 --2- 172.21.15.86:0/869687736 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f259c06f490 0x7f259c0719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:10:27.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.525+0000 7f25b2000640 1 -- 172.21.15.86:0/869687736 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f25bc071770 con 0x7f25bc0d2c80 2024-04-25T14:10:27.663 INFO:tasks.ceph.mgr.x.smithi155.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:10:27.663 INFO:tasks.ceph.mgr.x.smithi155.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:10:27.664 INFO:tasks.ceph.mgr.x.smithi155.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:10:27.672 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:27.675+0000 7f3cba623180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:10:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.725+0000 7f25b9e00640 1 -- 172.21.15.86:0/869687736 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f259c06f490 msgr2=0x7f259c0719a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/190090395 2024-04-25T14:10:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.725+0000 7f25b9e00640 1 --2- 172.21.15.86:0/869687736 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f259c06f490 0x7f259c0719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:10:27.753 INFO:tasks.ceph.mgr.y.smithi086.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:10:27.753 INFO:tasks.ceph.mgr.y.smithi086.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:10:27.753 INFO:tasks.ceph.mgr.y.smithi086.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:10:27.761 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:27.765+0000 7f6812189180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:10:27.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.769+0000 7f25c0c00640 1 -- 172.21.15.86:0/869687736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f25bc0c9fa0 con 0x7f25bc0d2c80 2024-04-25T14:10:27.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.769+0000 7f25b2000640 1 -- 172.21.15.86:0/869687736 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v114) ==== 80+0+468 (secure 0 0 0) 0x7f25b40620c0 con 0x7f25bc0d2c80 2024-04-25T14:10:27.768 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2052329939 2024-04-26T14:10:23.454991+0000 2024-04-25T14:10:27.768 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1021944523 2024-04-26T14:10:23.454991+0000 2024-04-25T14:10:27.768 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/83615604 2024-04-26T14:10:23.454991+0000 2024-04-25T14:10:27.768 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3719613294 2024-04-26T14:10:23.454991+0000 2024-04-25T14:10:27.768 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1471025278 2024-04-26T14:10:23.454991+0000 2024-04-25T14:10:27.768 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6833/1354467416 2024-04-26T14:10:23.454991+0000 2024-04-25T14:10:27.768 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6832/1354467416 2024-04-26T14:10:23.454991+0000 2024-04-25T14:10:27.768 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3000491277 2024-04-26T14:10:23.454991+0000 2024-04-25T14:10:27.768 INFO:teuthology.orchestra.run.smithi086.stderr:listed 8 entries 2024-04-25T14:10:27.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.773+0000 7f25c0c00640 1 -- 172.21.15.86:0/869687736 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f259c06f490 msgr2=0x7f259c0719a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:10:27.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.773+0000 7f25c0c00640 1 --2- 172.21.15.86:0/869687736 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f259c06f490 0x7f259c0719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:27.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.773+0000 7f25c0c00640 1 -- 172.21.15.86:0/869687736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25bc0d2c80 msgr2=0x7f25bc0b0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:27.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.773+0000 7f25c0c00640 1 --2- 172.21.15.86:0/869687736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25bc0d2c80 0x7f25bc0b0e20 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f25b400c760 tx=0x7f25b400cc20 comp rx=0 tx=0).stop 2024-04-25T14:10:27.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.773+0000 7f25c0c00640 1 -- 172.21.15.86:0/869687736 shutdown_connections 2024-04-25T14:10:27.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.773+0000 7f25c0c00640 1 --2- 172.21.15.86:0/869687736 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7f259c06f490 0x7f259c0719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:27.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.773+0000 7f25c0c00640 1 --2- 172.21.15.86:0/869687736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25bc0d2c80 0x7f25bc0b0e20 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:27.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.773+0000 7f25c0c00640 1 --2- 172.21.15.86:0/869687736 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25bc0722f0 0x7f25bc0b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:27.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.773+0000 7f25c0c00640 1 --2- 172.21.15.86:0/869687736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25bc070e20 0x7f25bc0c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:27.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.773+0000 7f25c0c00640 1 -- 172.21.15.86:0/869687736 >> 172.21.15.86:0/869687736 conn(0x7f25bc0cffc0 msgr2=0x7f25bc15c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:27.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.773+0000 7f25c0c00640 1 -- 172.21.15.86:0/869687736 shutdown_connections 2024-04-25T14:10:27.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:27.773+0000 7f25c0c00640 1 -- 172.21.15.86:0/869687736 wait complete. 2024-04-25T14:10:27.790 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T14:10:27.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:27.790 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T14:10:27.817 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T14:10:27.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:27.818 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T14:10:27.824 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:27.823+0000 7f3cba623180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:10:27.870 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:10:27.870 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:10:27.870 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:10:27.879 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:10:27.879 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:10:27.917 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:27.921+0000 7f6812189180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:10:27.928 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:10:27.952 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:27.951+0000 7f3cba623180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:10:28.024 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:28.027+0000 7f3cba623180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:10:28.048 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:28.049+0000 7f6812189180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:10:28.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.081+0000 7fcc7b200640 1 Processor -- start 2024-04-25T14:10:28.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.081+0000 7fcc7b200640 1 -- start start 2024-04-25T14:10:28.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.081+0000 7fcc7b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc740d1df0 0x7fcc740cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.081+0000 7fcc7b200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcc740cdcc0 0x7fcc740ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.081+0000 7fcc7b200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc7415e5c0 con 0x7fcc74070330 2024-04-25T14:10:28.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.081+0000 7fcc7b200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fcc7406fd40 con 0x7fcc740cdcc0 2024-04-25T14:10:28.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.081+0000 7fcc7b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcc7415e8a0 con 0x7fcc740d1df0 2024-04-25T14:10:28.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.081+0000 7fcc73400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc740d1df0 0x7fcc740cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.081+0000 7fcc79200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcc740cdcc0 0x7fcc740ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.081+0000 7fcc73400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc740d1df0 0x7fcc740cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:46958/0 (socket says 172.21.15.86:46958) 2024-04-25T14:10:28.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.081+0000 7fcc73400640 1 -- 172.21.15.86:0/2065196206 learned_addr learned my addr 172.21.15.86:0/2065196206 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:28.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.081+0000 7fcc73e00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fcc74070330 0x7fcc740d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:43332/0 (socket says 172.21.15.86:43332) 2024-04-25T14:10:28.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.081+0000 7fcc79200640 1 -- 172.21.15.86:0/2065196206 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc740d1df0 msgr2=0x7fcc740cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:28.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.081+0000 7fcc79200640 1 --2- 172.21.15.86:0/2065196206 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc740d1df0 0x7fcc740cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.081+0000 7fcc79200640 1 -- 172.21.15.86:0/2065196206 >> v1:172.21.15.155:6789/0 conn(0x7fcc74070330 legacy=0x7fcc740d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:28.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.081+0000 7fcc79200640 1 -- 172.21.15.86:0/2065196206 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7415eb80 con 0x7fcc740cdcc0 2024-04-25T14:10:28.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.081+0000 7fcc73400640 1 --2- 172.21.15.86:0/2065196206 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc740d1df0 0x7fcc740cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:10:28.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.085+0000 7fcc79200640 1 --2- 172.21.15.86:0/2065196206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc740cdcc0 0x7fcc740ce0d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009d90 tx=0x7fcc6c01c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f460ec8a3d130c58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:28.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.085+0000 7fcc72a00640 1 -- 172.21.15.86:0/2065196206 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c0b2070 con 0x7fcc740cdcc0 2024-04-25T14:10:28.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.085+0000 7fcc72a00640 1 -- 172.21.15.86:0/2065196206 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc6c0a5950 con 0x7fcc740cdcc0 2024-04-25T14:10:28.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.085+0000 7fcc72a00640 1 -- 172.21.15.86:0/2065196206 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c0a5d20 con 0x7fcc740cdcc0 2024-04-25T14:10:28.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.085+0000 7fcc7b200640 1 -- 172.21.15.86:0/2065196206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc740cdcc0 msgr2=0x7fcc740ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:28.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.085+0000 7fcc7b200640 1 --2- 172.21.15.86:0/2065196206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc740cdcc0 0x7fcc740ce0d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009d90 tx=0x7fcc6c01c8b0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.085+0000 7fcc7b200640 1 -- 172.21.15.86:0/2065196206 shutdown_connections 2024-04-25T14:10:28.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.085+0000 7fcc7b200640 1 --2- 172.21.15.86:0/2065196206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc740cdcc0 0x7fcc740ce0d0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.089+0000 7fcc7b200640 1 --2- 172.21.15.86:0/2065196206 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc740d1df0 0x7fcc740cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.089+0000 7fcc7b200640 1 -- 172.21.15.86:0/2065196206 >> 172.21.15.86:0/2065196206 conn(0x7fcc740b4650 msgr2=0x7fcc740b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:28.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.089+0000 7fcc7b200640 1 -- 172.21.15.86:0/2065196206 shutdown_connections 2024-04-25T14:10:28.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.089+0000 7fcc7b200640 1 -- 172.21.15.86:0/2065196206 wait complete. 2024-04-25T14:10:28.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.089+0000 7fcc7b200640 1 Processor -- start 2024-04-25T14:10:28.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.089+0000 7fcc7b200640 1 -- start start 2024-04-25T14:10:28.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.089+0000 7fcc7b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc74070330 0x7fcc7415b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.089+0000 7fcc7b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc740cdcc0 0x7fcc740c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.093+0000 7fcc7b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc740ceeb0 0x7fcc741f9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.093+0000 7fcc7b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7415f4b0 con 0x7fcc74070330 2024-04-25T14:10:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.093+0000 7fcc7b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7415f330 con 0x7fcc740cdcc0 2024-04-25T14:10:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.093+0000 7fcc7b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcc74070740 con 0x7fcc740ceeb0 2024-04-25T14:10:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.093+0000 7fcc73e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc74070330 0x7fcc7415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.093+0000 7fcc73e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc74070330 0x7fcc7415b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50194/0 (socket says 172.21.15.86:50194) 2024-04-25T14:10:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.093+0000 7fcc73e00640 1 -- 172.21.15.86:0/111917632 learned_addr learned my addr 172.21.15.86:0/111917632 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.093+0000 7fcc79200640 1 --2- 172.21.15.86:0/111917632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc740ceeb0 0x7fcc741f9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.093+0000 7fcc73400640 1 --2- 172.21.15.86:0/111917632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc740cdcc0 0x7fcc740c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.093+0000 7fcc73e00640 1 -- 172.21.15.86:0/111917632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc740ceeb0 msgr2=0x7fcc741f9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.093+0000 7fcc73e00640 1 --2- 172.21.15.86:0/111917632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc740ceeb0 0x7fcc741f9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.093+0000 7fcc73e00640 1 -- 172.21.15.86:0/111917632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc740cdcc0 msgr2=0x7fcc740c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.093+0000 7fcc73e00640 1 --2- 172.21.15.86:0/111917632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc740cdcc0 0x7fcc740c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.093+0000 7fcc73e00640 1 -- 172.21.15.86:0/111917632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc74073fe0 con 0x7fcc74070330 2024-04-25T14:10:28.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.093+0000 7fcc73e00640 1 --2- 172.21.15.86:0/111917632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc74070330 0x7fcc7415b200 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fcc6400c340 tx=0x7fcc6400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:28.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.093+0000 7fcc6be00640 1 -- 172.21.15.86:0/111917632 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc640074e0 con 0x7fcc74070330 2024-04-25T14:10:28.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.093+0000 7fcc6be00640 1 -- 172.21.15.86:0/111917632 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc64007680 con 0x7fcc74070330 2024-04-25T14:10:28.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.093+0000 7fcc7b200640 1 -- 172.21.15.86:0/111917632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc74073a20 con 0x7fcc74070330 2024-04-25T14:10:28.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.093+0000 7fcc6be00640 1 -- 172.21.15.86:0/111917632 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc64004a30 con 0x7fcc74070330 2024-04-25T14:10:28.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.093+0000 7fcc7b200640 1 -- 172.21.15.86:0/111917632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc740c7cd0 con 0x7fcc74070330 2024-04-25T14:10:28.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.093+0000 7fcc7b200640 1 -- 172.21.15.86:0/111917632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc74069d00 con 0x7fcc74070330 2024-04-25T14:10:28.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.093+0000 7fcc6be00640 1 -- 172.21.15.86:0/111917632 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 50) ==== 93253+0+0 (secure 0 0 0) 0x7fcc6401a020 con 0x7fcc74070330 2024-04-25T14:10:28.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.097+0000 7fcc6be00640 1 --2- 172.21.15.86:0/111917632 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fcc5406f4c0 0x7fcc540719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.097+0000 7fcc6be00640 1 -- 172.21.15.86:0/111917632 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6445+0+0 (secure 0 0 0) 0x7fcc6408eb70 con 0x7fcc74070330 2024-04-25T14:10:28.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.097+0000 7fcc73400640 1 -- 172.21.15.86:0/111917632 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fcc5406f4c0 msgr2=0x7fcc540719d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/190090395 2024-04-25T14:10:28.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.097+0000 7fcc73400640 1 --2- 172.21.15.86:0/111917632 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fcc5406f4c0 0x7fcc540719d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:10:28.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.101+0000 7fcc6be00640 1 -- 172.21.15.86:0/111917632 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcc6405b1d0 con 0x7fcc74070330 2024-04-25T14:10:28.123 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:28.125+0000 7f6812189180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:10:28.174 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:28.175+0000 7f3cba623180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:10:28.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.261+0000 7fcc7b200640 1 -- 172.21.15.86:0/111917632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcc74073070 con 0x7fcc74070330 2024-04-25T14:10:28.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.261+0000 7fcc6be00640 1 -- 172.21.15.86:0/111917632 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) ==== 74+0+17451 (secure 0 0 0) 0x7fcc6405f4d0 con 0x7fcc74070330 2024-04-25T14:10:28.258 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:28.259 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":114,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:10:23.455025+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:10:03.520649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T14:10:04.080912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2052329939":"2024-04-26T14:10:23.454991+0000","172.21.15.86:0/1021944523":"2024-04-26T14:10:23.454991+0000","172.21.15.86:0/83615604":"2024-04-26T14:10:23.454991+0000","172.21.15.86:0/3719613294":"2024-04-26T14:10:23.454991+0000","172.21.15.86:0/1471025278":"2024-04-26T14:10:23.454991+0000","172.21.15.86:6833/1354467416":"2024-04-26T14:10:23.454991+0000","172.21.15.86:6832/1354467416":"2024-04-26T14:10:23.454991+0000","172.21.15.86:0/3000491277":"2024-04-26T14:10:23.454991+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:10:28.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.261+0000 7fcc7b200640 1 -- 172.21.15.86:0/111917632 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fcc5406f4c0 msgr2=0x7fcc540719d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:10:28.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.261+0000 7fcc7b200640 1 --2- 172.21.15.86:0/111917632 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fcc5406f4c0 0x7fcc540719d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.261+0000 7fcc7b200640 1 -- 172.21.15.86:0/111917632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc74070330 msgr2=0x7fcc7415b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:28.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.261+0000 7fcc7b200640 1 --2- 172.21.15.86:0/111917632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc74070330 0x7fcc7415b200 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fcc6400c340 tx=0x7fcc6400c800 comp rx=0 tx=0).stop 2024-04-25T14:10:28.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.261+0000 7fcc7b200640 1 -- 172.21.15.86:0/111917632 shutdown_connections 2024-04-25T14:10:28.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.261+0000 7fcc7b200640 1 --2- 172.21.15.86:0/111917632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc740ceeb0 0x7fcc741f9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.261+0000 7fcc7b200640 1 --2- 172.21.15.86:0/111917632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc740cdcc0 0x7fcc740c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.261+0000 7fcc7b200640 1 --2- 172.21.15.86:0/111917632 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fcc5406f4c0 0x7fcc540719d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.261+0000 7fcc7b200640 1 --2- 172.21.15.86:0/111917632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc74070330 0x7fcc7415b200 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.261+0000 7fcc7b200640 1 -- 172.21.15.86:0/111917632 >> 172.21.15.86:0/111917632 conn(0x7fcc740b4650 msgr2=0x7fcc740ae130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:28.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.261+0000 7fcc7b200640 1 -- 172.21.15.86:0/111917632 shutdown_connections 2024-04-25T14:10:28.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.261+0000 7fcc7b200640 1 -- 172.21.15.86:0/111917632 wait complete. 2024-04-25T14:10:28.275 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:28.277+0000 7f6812189180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:10:28.281 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:10:03.520649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:10:04.080912+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:10:28.282 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:10:28.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.437+0000 7fea99400640 1 Processor -- start 2024-04-25T14:10:28.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.437+0000 7fea99400640 1 -- start start 2024-04-25T14:10:28.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.437+0000 7fea99400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fea94071530 0x7fea940d15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.437+0000 7fea99400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea94161770 con 0x7fea940d1af0 2024-04-25T14:10:28.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.437+0000 7fea99400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea94161a50 con 0x7fea940ce5a0 2024-04-25T14:10:28.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.437+0000 7fea99400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fea94162380 con 0x7fea94071530 2024-04-25T14:10:28.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.437+0000 7fea91e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fea94071530 0x7fea940d15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.437+0000 7fea93200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fea940d1af0 0x7fea940d1f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48460/0 (socket says 172.21.15.86:48460) 2024-04-25T14:10:28.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.437+0000 7fea93200640 1 -- 172.21.15.86:0/2356467957 learned_addr learned my addr 172.21.15.86:0/2356467957 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:28.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.437+0000 7fea91400640 1 -- 172.21.15.86:0/2356467957 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3458783438 0 0) 0x7fea94161a50 con 0x7fea940ce5a0 2024-04-25T14:10:28.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.437+0000 7fea91e00640 1 -- 172.21.15.86:0/2356467957 >> v1:172.21.15.155:6789/0 conn(0x7fea940ce5a0 legacy=0x7fea94070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:28.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.437+0000 7fea91e00640 1 -- 172.21.15.86:0/2356467957 >> v1:172.21.15.86:6789/0 conn(0x7fea940d1af0 legacy=0x7fea940d1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:28.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.437+0000 7fea91e00640 1 -- 172.21.15.86:0/2356467957 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea94161d30 con 0x7fea94071530 2024-04-25T14:10:28.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.441+0000 7fea91e00640 1 --2- 172.21.15.86:0/2356467957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea94071530 0x7fea940d15b0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fea800097c0 tx=0x7fea8001c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1baf6ddcc625dcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:28.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.441+0000 7fea91400640 1 -- 172.21.15.86:0/2356467957 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea800b0070 con 0x7fea94071530 2024-04-25T14:10:28.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.441+0000 7fea91400640 1 -- 172.21.15.86:0/2356467957 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea800ab600 con 0x7fea94071530 2024-04-25T14:10:28.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.441+0000 7fea91400640 1 -- 172.21.15.86:0/2356467957 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea800ab9d0 con 0x7fea94071530 2024-04-25T14:10:28.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.441+0000 7fea99400640 1 -- 172.21.15.86:0/2356467957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea94071530 msgr2=0x7fea940d15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:28.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.441+0000 7fea99400640 1 --2- 172.21.15.86:0/2356467957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea94071530 0x7fea940d15b0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fea800097c0 tx=0x7fea8001c330 comp rx=0 tx=0).stop 2024-04-25T14:10:28.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.441+0000 7fea99400640 1 -- 172.21.15.86:0/2356467957 shutdown_connections 2024-04-25T14:10:28.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.441+0000 7fea99400640 1 --2- 172.21.15.86:0/2356467957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea94071530 0x7fea940d15b0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.441+0000 7fea99400640 1 -- 172.21.15.86:0/2356467957 >> 172.21.15.86:0/2356467957 conn(0x7fea940cffc0 msgr2=0x7fea940d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:28.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.441+0000 7fea99400640 1 -- 172.21.15.86:0/2356467957 shutdown_connections 2024-04-25T14:10:28.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.441+0000 7fea99400640 1 -- 172.21.15.86:0/2356467957 wait complete. 2024-04-25T14:10:28.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.445+0000 7fea99400640 1 Processor -- start 2024-04-25T14:10:28.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.445+0000 7fea99400640 1 -- start start 2024-04-25T14:10:28.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.445+0000 7fea99400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea940ce5a0 0x7fea9414b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.445+0000 7fea99400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea940d1af0 0x7fea940c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.445+0000 7fea99400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea940c8f50 0x7fea941541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.445+0000 7fea99400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea940ce9b0 con 0x7fea940d1af0 2024-04-25T14:10:28.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.445+0000 7fea99400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fea9415ef80 con 0x7fea940ce5a0 2024-04-25T14:10:28.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.445+0000 7fea99400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fea940ceb30 con 0x7fea940c8f50 2024-04-25T14:10:28.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.445+0000 7fea91e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea940d1af0 0x7fea940c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.445+0000 7fea91e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea940d1af0 0x7fea940c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50204/0 (socket says 172.21.15.86:50204) 2024-04-25T14:10:28.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.445+0000 7fea91e00640 1 -- 172.21.15.86:0/3344011871 learned_addr learned my addr 172.21.15.86:0/3344011871 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:28.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.445+0000 7fea92800640 1 --2- 172.21.15.86:0/3344011871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea940ce5a0 0x7fea9414b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.445+0000 7fea93200640 1 --2- 172.21.15.86:0/3344011871 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea940c8f50 0x7fea941541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.445+0000 7fea91e00640 1 -- 172.21.15.86:0/3344011871 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea940c8f50 msgr2=0x7fea941541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:28.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.445+0000 7fea91e00640 1 --2- 172.21.15.86:0/3344011871 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea940c8f50 0x7fea941541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.445+0000 7fea91e00640 1 -- 172.21.15.86:0/3344011871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea940ce5a0 msgr2=0x7fea9414b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:28.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.445+0000 7fea91e00640 1 --2- 172.21.15.86:0/3344011871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea940ce5a0 0x7fea9414b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.445+0000 7fea91e00640 1 -- 172.21.15.86:0/3344011871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea940c9360 con 0x7fea940d1af0 2024-04-25T14:10:28.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.449+0000 7fea91e00640 1 --2- 172.21.15.86:0/3344011871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea940d1af0 0x7fea940c8a10 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fea800097c0 tx=0x7fea8001ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:28.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.449+0000 7fea8aa00640 1 -- 172.21.15.86:0/3344011871 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea800bd070 con 0x7fea940d1af0 2024-04-25T14:10:28.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.449+0000 7fea8aa00640 1 -- 172.21.15.86:0/3344011871 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea80004030 con 0x7fea940d1af0 2024-04-25T14:10:28.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.449+0000 7fea8aa00640 1 -- 172.21.15.86:0/3344011871 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea800b0040 con 0x7fea940d1af0 2024-04-25T14:10:28.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.449+0000 7fea99400640 1 -- 172.21.15.86:0/3344011871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea940b7240 con 0x7fea940d1af0 2024-04-25T14:10:28.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.449+0000 7fea99400640 1 -- 172.21.15.86:0/3344011871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea940b7800 con 0x7fea940d1af0 2024-04-25T14:10:28.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.449+0000 7fea99400640 1 -- 172.21.15.86:0/3344011871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea940b6fb0 con 0x7fea940d1af0 2024-04-25T14:10:28.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.449+0000 7fea8aa00640 1 -- 172.21.15.86:0/3344011871 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 50) ==== 93253+0+0 (secure 0 0 0) 0x7fea800aa750 con 0x7fea940d1af0 2024-04-25T14:10:28.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.449+0000 7fea8aa00640 1 --2- 172.21.15.86:0/3344011871 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fea7406f490 0x7fea740719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.449+0000 7fea92800640 1 -- 172.21.15.86:0/3344011871 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fea7406f490 msgr2=0x7fea740719a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/190090395 2024-04-25T14:10:28.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.449+0000 7fea8aa00640 1 -- 172.21.15.86:0/3344011871 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6445+0+0 (secure 0 0 0) 0x7fea80129860 con 0x7fea940d1af0 2024-04-25T14:10:28.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.449+0000 7fea92800640 1 --2- 172.21.15.86:0/3344011871 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fea7406f490 0x7fea740719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:10:28.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.453+0000 7fea8aa00640 1 -- 172.21.15.86:0/3344011871 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fea940c9360 con 0x7fea940d1af0 2024-04-25T14:10:28.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.617+0000 7fea99400640 1 -- 172.21.15.86:0/3344011871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fea940720e0 con 0x7fea940d1af0 2024-04-25T14:10:28.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.617+0000 7fea8aa00640 1 -- 172.21.15.86:0/3344011871 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) ==== 93+0+10 (secure 0 0 0) 0x7fea80015fb0 con 0x7fea940d1af0 2024-04-25T14:10:28.616 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:10:28.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.621+0000 7fea99400640 1 -- 172.21.15.86:0/3344011871 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fea7406f490 msgr2=0x7fea740719a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:10:28.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.621+0000 7fea99400640 1 --2- 172.21.15.86:0/3344011871 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fea7406f490 0x7fea740719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.621+0000 7fea99400640 1 -- 172.21.15.86:0/3344011871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea940d1af0 msgr2=0x7fea940c8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:28.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.621+0000 7fea99400640 1 --2- 172.21.15.86:0/3344011871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea940d1af0 0x7fea940c8a10 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fea800097c0 tx=0x7fea8001ab70 comp rx=0 tx=0).stop 2024-04-25T14:10:28.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.621+0000 7fea99400640 1 -- 172.21.15.86:0/3344011871 shutdown_connections 2024-04-25T14:10:28.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.621+0000 7fea99400640 1 --2- 172.21.15.86:0/3344011871 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea940c8f50 0x7fea941541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.621+0000 7fea99400640 1 --2- 172.21.15.86:0/3344011871 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fea7406f490 0x7fea740719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.621+0000 7fea99400640 1 --2- 172.21.15.86:0/3344011871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea940d1af0 0x7fea940c8a10 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.621+0000 7fea99400640 1 --2- 172.21.15.86:0/3344011871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea940ce5a0 0x7fea9414b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.621+0000 7fea99400640 1 -- 172.21.15.86:0/3344011871 >> 172.21.15.86:0/3344011871 conn(0x7fea940cffc0 msgr2=0x7fea940d1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:28.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.621+0000 7fea99400640 1 -- 172.21.15.86:0/3344011871 shutdown_connections 2024-04-25T14:10:28.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.621+0000 7fea99400640 1 -- 172.21.15.86:0/3344011871 wait complete. 2024-04-25T14:10:28.638 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:10:28.722 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:28.723+0000 7f3cba623180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:10:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.797+0000 7fb3b9200640 1 Processor -- start 2024-04-25T14:10:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.797+0000 7fb3b9200640 1 -- start start 2024-04-25T14:10:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.797+0000 7fb3b9200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb3b4071530 0x7fb3b40ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.797+0000 7fb3b9200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb3b406e700 0x7fb3b406eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.797+0000 7fb3b9200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3b406f050 con 0x7fb3b40cdd70 2024-04-25T14:10:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.797+0000 7fb3b9200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb3b406d940 con 0x7fb3b406e700 2024-04-25T14:10:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.797+0000 7fb3b9200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb3b406dc20 con 0x7fb3b4071530 2024-04-25T14:10:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.797+0000 7fb3b2800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb3b406e700 0x7fb3b406eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.797+0000 7fb3b1e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb3b40cdd70 0x7fb3b4070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39860/0 (socket says 172.21.15.86:39860) 2024-04-25T14:10:28.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.797+0000 7fb3b1e00640 1 -- 172.21.15.86:0/222728115 learned_addr learned my addr 172.21.15.86:0/222728115 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:28.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.797+0000 7fb3b2800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb3b406e700 0x7fb3b406eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57130/0 (socket says 172.21.15.86:57130) 2024-04-25T14:10:28.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.797+0000 7fb3b1400640 1 --2- 172.21.15.86:0/222728115 >> v2:172.21.15.155:3300/0 conn(0x7fb3b4071530 0x7fb3b40ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.797+0000 7fb3b2800640 1 -- 172.21.15.86:0/222728115 >> v2:172.21.15.155:3300/0 conn(0x7fb3b4071530 msgr2=0x7fb3b40ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:28.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.797+0000 7fb3b2800640 1 --2- 172.21.15.86:0/222728115 >> v2:172.21.15.155:3300/0 conn(0x7fb3b4071530 0x7fb3b40ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.797+0000 7fb3b2800640 1 -- 172.21.15.86:0/222728115 >> v1:172.21.15.86:6789/0 conn(0x7fb3b40cdd70 legacy=0x7fb3b4070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:28.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.797+0000 7fb3b2800640 1 -- 172.21.15.86:0/222728115 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b406df00 con 0x7fb3b406e700 2024-04-25T14:10:28.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.797+0000 7fb3b0a00640 1 -- 172.21.15.86:0/222728115 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4253575938 0 0) 0x7fb3b406f050 con 0x7fb3b40cdd70 2024-04-25T14:10:28.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.801+0000 7fb3b2800640 1 --2- 172.21.15.86:0/222728115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3b406e700 0x7fb3b406eb10 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4073230 tx=0x7fb3ac01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1a8033cd467c3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:28.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.801+0000 7fb3b0a00640 1 -- 172.21.15.86:0/222728115 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac0b2070 con 0x7fb3b406e700 2024-04-25T14:10:28.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.801+0000 7fb3b0a00640 1 -- 172.21.15.86:0/222728115 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3ac0a57b0 con 0x7fb3b406e700 2024-04-25T14:10:28.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.801+0000 7fb3b0a00640 1 -- 172.21.15.86:0/222728115 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac0a5b80 con 0x7fb3b406e700 2024-04-25T14:10:28.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.801+0000 7fb3b9200640 1 -- 172.21.15.86:0/222728115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3b406e700 msgr2=0x7fb3b406eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:28.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.801+0000 7fb3b9200640 1 --2- 172.21.15.86:0/222728115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3b406e700 0x7fb3b406eb10 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4073230 tx=0x7fb3ac01c8a0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.801+0000 7fb3b9200640 1 -- 172.21.15.86:0/222728115 shutdown_connections 2024-04-25T14:10:28.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.801+0000 7fb3b9200640 1 --2- 172.21.15.86:0/222728115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3b406e700 0x7fb3b406eb10 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.801+0000 7fb3b9200640 1 --2- 172.21.15.86:0/222728115 >> v2:172.21.15.155:3300/0 conn(0x7fb3b4071530 0x7fb3b40ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.801+0000 7fb3b9200640 1 -- 172.21.15.86:0/222728115 >> 172.21.15.86:0/222728115 conn(0x7fb3b40cffc0 msgr2=0x7fb3b40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:28.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.801+0000 7fb3b9200640 1 -- 172.21.15.86:0/222728115 shutdown_connections 2024-04-25T14:10:28.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.801+0000 7fb3b9200640 1 -- 172.21.15.86:0/222728115 wait complete. 2024-04-25T14:10:28.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.805+0000 7fb3b9200640 1 Processor -- start 2024-04-25T14:10:28.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.805+0000 7fb3b9200640 1 -- start start 2024-04-25T14:10:28.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.809+0000 7fb3b9200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3b406e700 0x7fb3b40b8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.809+0000 7fb3b9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3b40cdd70 0x7fb3b4162bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.809+0000 7fb3b9200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb3b41fd550 0x7fb3b41fcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.809+0000 7fb3b9200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b416cc00 con 0x7fb3b40cdd70 2024-04-25T14:10:28.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.809+0000 7fb3b9200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b4164380 con 0x7fb3b406e700 2024-04-25T14:10:28.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.809+0000 7fb3b9200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb3b416cd80 con 0x7fb3b41fd550 2024-04-25T14:10:28.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.809+0000 7fb3b1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3b40cdd70 0x7fb3b4162bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.809+0000 7fb3b1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3b40cdd70 0x7fb3b4162bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57138/0 (socket says 172.21.15.86:57138) 2024-04-25T14:10:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.809+0000 7fb3b1400640 1 -- 172.21.15.86:0/2831330069 learned_addr learned my addr 172.21.15.86:0/2831330069 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.809+0000 7fb3b1e00640 1 --2- 172.21.15.86:0/2831330069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3b406e700 0x7fb3b40b8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.809+0000 7fb3b2800640 1 --2- 172.21.15.86:0/2831330069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb3b41fd550 0x7fb3b41fcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.809+0000 7fb3b1400640 1 -- 172.21.15.86:0/2831330069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb3b41fd550 msgr2=0x7fb3b41fcaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.809+0000 7fb3b1400640 1 --2- 172.21.15.86:0/2831330069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb3b41fd550 0x7fb3b41fcaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.809+0000 7fb3b1400640 1 -- 172.21.15.86:0/2831330069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3b406e700 msgr2=0x7fb3b40b8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.809+0000 7fb3b1400640 1 --2- 172.21.15.86:0/2831330069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3b406e700 0x7fb3b40b8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.809+0000 7fb3b1400640 1 -- 172.21.15.86:0/2831330069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b40b7b30 con 0x7fb3b40cdd70 2024-04-25T14:10:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.809+0000 7fb3b1400640 1 --2- 172.21.15.86:0/2831330069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3b40cdd70 0x7fb3b4162bf0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000e840 tx=0x7fb3a000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.809+0000 7fb3aa000640 1 -- 172.21.15.86:0/2831330069 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0017070 con 0x7fb3b40cdd70 2024-04-25T14:10:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.809+0000 7fb3aa000640 1 -- 172.21.15.86:0/2831330069 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3a000c940 con 0x7fb3b40cdd70 2024-04-25T14:10:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.809+0000 7fb3b9200640 1 -- 172.21.15.86:0/2831330069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b40b7e10 con 0x7fb3b40cdd70 2024-04-25T14:10:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.809+0000 7fb3aa000640 1 -- 172.21.15.86:0/2831330069 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a000cd30 con 0x7fb3b40cdd70 2024-04-25T14:10:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.809+0000 7fb3b9200640 1 -- 172.21.15.86:0/2831330069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b40b83d0 con 0x7fb3b40cdd70 2024-04-25T14:10:28.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.813+0000 7fb3b9200640 1 -- 172.21.15.86:0/2831330069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3b40720e0 con 0x7fb3b40cdd70 2024-04-25T14:10:28.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.813+0000 7fb3aa000640 1 -- 172.21.15.86:0/2831330069 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 50) ==== 93253+0+0 (secure 0 0 0) 0x7fb3a001a070 con 0x7fb3b40cdd70 2024-04-25T14:10:28.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.813+0000 7fb3aa000640 1 --2- 172.21.15.86:0/2831330069 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fb39406f490 0x7fb3940719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.813+0000 7fb3aa000640 1 -- 172.21.15.86:0/2831330069 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6445+0+0 (secure 0 0 0) 0x7fb3a008f8a0 con 0x7fb3b40cdd70 2024-04-25T14:10:28.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.813+0000 7fb3b1e00640 1 -- 172.21.15.86:0/2831330069 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fb39406f490 msgr2=0x7fb3940719a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/190090395 2024-04-25T14:10:28.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.813+0000 7fb3b1e00640 1 --2- 172.21.15.86:0/2831330069 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fb39406f490 0x7fb3940719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:10:28.812 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:28.811+0000 7f3cba623180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:10:28.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.817+0000 7fb3aa000640 1 -- 172.21.15.86:0/2831330069 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb3a005be00 con 0x7fb3b40cdd70 2024-04-25T14:10:28.846 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:28.849+0000 7f6812189180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:10:28.931 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:28.933+0000 7f6812189180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:10:28.972 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:28.975+0000 7f3cba623180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:10:28.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.977+0000 7fb3b9200640 1 -- 172.21.15.86:0/2831330069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb3b4076bf0 con 0x7fb3b40cdd70 2024-04-25T14:10:28.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.977+0000 7fb3aa000640 1 -- 172.21.15.86:0/2831330069 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v114) ==== 104+0+11 (secure 0 0 0) 0x7fb3a0060100 con 0x7fb3b40cdd70 2024-04-25T14:10:28.975 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:10:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.981+0000 7fb3b9200640 1 -- 172.21.15.86:0/2831330069 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fb39406f490 msgr2=0x7fb3940719a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:10:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.981+0000 7fb3b9200640 1 --2- 172.21.15.86:0/2831330069 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fb39406f490 0x7fb3940719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.981+0000 7fb3b9200640 1 -- 172.21.15.86:0/2831330069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3b40cdd70 msgr2=0x7fb3b4162bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.981+0000 7fb3b9200640 1 --2- 172.21.15.86:0/2831330069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3b40cdd70 0x7fb3b4162bf0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000e840 tx=0x7fb3a000ed00 comp rx=0 tx=0).stop 2024-04-25T14:10:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.981+0000 7fb3b9200640 1 -- 172.21.15.86:0/2831330069 shutdown_connections 2024-04-25T14:10:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.981+0000 7fb3b9200640 1 --2- 172.21.15.86:0/2831330069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb3b41fd550 0x7fb3b41fcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.981+0000 7fb3b9200640 1 --2- 172.21.15.86:0/2831330069 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fb39406f490 0x7fb3940719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.981+0000 7fb3b9200640 1 --2- 172.21.15.86:0/2831330069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3b40cdd70 0x7fb3b4162bf0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.981+0000 7fb3b9200640 1 --2- 172.21.15.86:0/2831330069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3b406e700 0x7fb3b40b8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.981+0000 7fb3b9200640 1 -- 172.21.15.86:0/2831330069 >> 172.21.15.86:0/2831330069 conn(0x7fb3b40cffc0 msgr2=0x7fb3b40cf310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.981+0000 7fb3b9200640 1 -- 172.21.15.86:0/2831330069 shutdown_connections 2024-04-25T14:10:28.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:28.981+0000 7fb3b9200640 1 -- 172.21.15.86:0/2831330069 wait complete. 2024-04-25T14:10:28.997 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:10:29.056 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:29.059+0000 7f3cba623180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:10:29.097 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:29.101+0000 7f6812189180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:10:29.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.153+0000 7fb695800640 1 Processor -- start 2024-04-25T14:10:29.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.153+0000 7fb695800640 1 -- start start 2024-04-25T14:10:29.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.153+0000 7fb695800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb6900cf970 0x7fb690070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:29.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.153+0000 7fb695800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb690071360 0x7fb69006e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:29.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.153+0000 7fb695800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb690071770 con 0x7fb69006e7a0 2024-04-25T14:10:29.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.153+0000 7fb695800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb6900cedf0 con 0x7fb6900cf970 2024-04-25T14:10:29.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.153+0000 7fb695800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb690163830 con 0x7fb690071360 2024-04-25T14:10:29.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.153+0000 7fb68f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb69006e7a0 0x7fb6900722f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39866/0 (socket says 172.21.15.86:39866) 2024-04-25T14:10:29.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.153+0000 7fb68f400640 1 -- 172.21.15.86:0/229970638 learned_addr learned my addr 172.21.15.86:0/229970638 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:29.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.153+0000 7fb68ea00640 1 --2- 172.21.15.86:0/229970638 >> v2:172.21.15.155:3300/0 conn(0x7fb6900cf970 0x7fb690070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:29.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.153+0000 7fb68e000640 1 --2- 172.21.15.86:0/229970638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb690071360 0x7fb69006e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:29.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.153+0000 7fb68d600640 1 -- 172.21.15.86:0/229970638 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 674306105 0 0) 0x7fb6880014a0 con 0x7fb69006e7a0 2024-04-25T14:10:29.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.153+0000 7fb68d600640 1 -- 172.21.15.86:0/229970638 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb674003530 con 0x7fb69006e7a0 2024-04-25T14:10:29.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.153+0000 7fb68ea00640 1 -- 172.21.15.86:0/229970638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb690071360 msgr2=0x7fb69006e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:29.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.153+0000 7fb68ea00640 1 --2- 172.21.15.86:0/229970638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb690071360 0x7fb69006e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:29.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.153+0000 7fb68ea00640 1 -- 172.21.15.86:0/229970638 >> v1:172.21.15.86:6789/0 conn(0x7fb69006e7a0 legacy=0x7fb6900722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:29.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.153+0000 7fb68ea00640 1 -- 172.21.15.86:0/229970638 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb69006ef90 con 0x7fb6900cf970 2024-04-25T14:10:29.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.157+0000 7fb68ea00640 1 --2- 172.21.15.86:0/229970638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6900cf970 0x7fb690070e20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb6800097c0 tx=0x7fb68001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1f13684fea328df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:29.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.157+0000 7fb68d600640 1 -- 172.21.15.86:0/229970638 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6800b2070 con 0x7fb6900cf970 2024-04-25T14:10:29.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.157+0000 7fb68d600640 1 -- 172.21.15.86:0/229970638 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6800ad640 con 0x7fb6900cf970 2024-04-25T14:10:29.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.157+0000 7fb68d600640 1 -- 172.21.15.86:0/229970638 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6800ada10 con 0x7fb6900cf970 2024-04-25T14:10:29.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.157+0000 7fb695800640 1 -- 172.21.15.86:0/229970638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6900cf970 msgr2=0x7fb690070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:29.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.157+0000 7fb695800640 1 --2- 172.21.15.86:0/229970638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6900cf970 0x7fb690070e20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb6800097c0 tx=0x7fb68001c470 comp rx=0 tx=0).stop 2024-04-25T14:10:29.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.157+0000 7fb695800640 1 -- 172.21.15.86:0/229970638 shutdown_connections 2024-04-25T14:10:29.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.157+0000 7fb695800640 1 --2- 172.21.15.86:0/229970638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb690071360 0x7fb69006e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:29.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.157+0000 7fb695800640 1 --2- 172.21.15.86:0/229970638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6900cf970 0x7fb690070e20 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:29.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.157+0000 7fb695800640 1 -- 172.21.15.86:0/229970638 >> 172.21.15.86:0/229970638 conn(0x7fb6900cffc0 msgr2=0x7fb6900d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:29.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.157+0000 7fb695800640 1 -- 172.21.15.86:0/229970638 shutdown_connections 2024-04-25T14:10:29.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.157+0000 7fb695800640 1 -- 172.21.15.86:0/229970638 wait complete. 2024-04-25T14:10:29.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.161+0000 7fb695800640 1 Processor -- start 2024-04-25T14:10:29.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.161+0000 7fb695800640 1 -- start start 2024-04-25T14:10:29.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.161+0000 7fb695800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb69006e7a0 0x7fb6900b6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:29.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.161+0000 7fb695800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb690071360 0x7fb690150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:29.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.161+0000 7fb695800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6900cf970 0x7fb690159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:29.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.161+0000 7fb695800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb69006ed30 con 0x7fb690071360 2024-04-25T14:10:29.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.161+0000 7fb695800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb69006ebb0 con 0x7fb6900cf970 2024-04-25T14:10:29.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.161+0000 7fb695800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb69016cbd0 con 0x7fb69006e7a0 2024-04-25T14:10:29.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.161+0000 7fb68e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb690071360 0x7fb690150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:29.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.161+0000 7fb68e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb690071360 0x7fb690150a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57150/0 (socket says 172.21.15.86:57150) 2024-04-25T14:10:29.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.161+0000 7fb68e000640 1 -- 172.21.15.86:0/2573798171 learned_addr learned my addr 172.21.15.86:0/2573798171 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:29.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.161+0000 7fb68ea00640 1 --2- 172.21.15.86:0/2573798171 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb69006e7a0 0x7fb6900b6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:29.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.161+0000 7fb68f400640 1 --2- 172.21.15.86:0/2573798171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6900cf970 0x7fb690159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:29.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.165+0000 7fb68ea00640 1 -- 172.21.15.86:0/2573798171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6900cf970 msgr2=0x7fb690159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:29.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.165+0000 7fb68ea00640 1 --2- 172.21.15.86:0/2573798171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6900cf970 0x7fb690159350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:29.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.165+0000 7fb68ea00640 1 -- 172.21.15.86:0/2573798171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb690071360 msgr2=0x7fb690150a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:29.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.165+0000 7fb68ea00640 1 --2- 172.21.15.86:0/2573798171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb690071360 0x7fb690150a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:29.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.165+0000 7fb68ea00640 1 -- 172.21.15.86:0/2573798171 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb690151930 con 0x7fb69006e7a0 2024-04-25T14:10:29.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.165+0000 7fb68e000640 1 --2- 172.21.15.86:0/2573798171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb690071360 0x7fb690150a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:10:29.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.165+0000 7fb68f400640 1 --2- 172.21.15.86:0/2573798171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6900cf970 0x7fb690159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:10:29.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.165+0000 7fb68ea00640 1 --2- 172.21.15.86:0/2573798171 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb69006e7a0 0x7fb6900b6ba0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb6800ac000 tx=0x7fb680019210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:29.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.165+0000 7fb686a00640 1 -- 172.21.15.86:0/2573798171 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6800b2040 con 0x7fb69006e7a0 2024-04-25T14:10:29.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.165+0000 7fb686a00640 1 -- 172.21.15.86:0/2573798171 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6800bc070 con 0x7fb69006e7a0 2024-04-25T14:10:29.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.165+0000 7fb695800640 1 -- 172.21.15.86:0/2573798171 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6900cc3f0 con 0x7fb69006e7a0 2024-04-25T14:10:29.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.165+0000 7fb686a00640 1 -- 172.21.15.86:0/2573798171 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb680006e20 con 0x7fb69006e7a0 2024-04-25T14:10:29.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.165+0000 7fb695800640 1 -- 172.21.15.86:0/2573798171 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6900cc9b0 con 0x7fb69006e7a0 2024-04-25T14:10:29.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.165+0000 7fb695800640 1 -- 172.21.15.86:0/2573798171 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6900720e0 con 0x7fb69006e7a0 2024-04-25T14:10:29.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.165+0000 7fb686a00640 1 -- 172.21.15.86:0/2573798171 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 50) ==== 93253+0+0 (secure 0 0 0) 0x7fb6800b6670 con 0x7fb69006e7a0 2024-04-25T14:10:29.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.169+0000 7fb686a00640 1 --2- 172.21.15.86:0/2573798171 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fb67006f490 0x7fb6700719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:29.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.169+0000 7fb686a00640 1 -- 172.21.15.86:0/2573798171 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(114..114 src has 1..114) ==== 6445+0+0 (secure 0 0 0) 0x7fb6801292c0 con 0x7fb69006e7a0 2024-04-25T14:10:29.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.169+0000 7fb68e000640 1 -- 172.21.15.86:0/2573798171 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fb67006f490 msgr2=0x7fb6700719a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/190090395 2024-04-25T14:10:29.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.169+0000 7fb68e000640 1 --2- 172.21.15.86:0/2573798171 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fb67006f490 0x7fb6700719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:10:29.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.173+0000 7fb686a00640 1 -- 172.21.15.86:0/2573798171 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb690151930 con 0x7fb69006e7a0 2024-04-25T14:10:29.184 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:29.185+0000 7f6812189180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:10:29.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.341+0000 7fb695800640 1 -- 172.21.15.86:0/2573798171 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb6900b94c0 con 0x7fb69006e7a0 2024-04-25T14:10:29.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.341+0000 7fb686a00640 1 -- 172.21.15.86:0/2573798171 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v114) ==== 100+0+11 (secure 0 0 0) 0x7fb680016090 con 0x7fb69006e7a0 2024-04-25T14:10:29.338 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:10:29.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.341+0000 7fb695800640 1 -- 172.21.15.86:0/2573798171 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fb67006f490 msgr2=0x7fb6700719a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:10:29.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.341+0000 7fb695800640 1 --2- 172.21.15.86:0/2573798171 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fb67006f490 0x7fb6700719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:29.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.341+0000 7fb695800640 1 -- 172.21.15.86:0/2573798171 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb69006e7a0 msgr2=0x7fb6900b6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:29.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.341+0000 7fb695800640 1 --2- 172.21.15.86:0/2573798171 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb69006e7a0 0x7fb6900b6ba0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb6800ac000 tx=0x7fb680019210 comp rx=0 tx=0).stop 2024-04-25T14:10:29.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.341+0000 7fb695800640 1 -- 172.21.15.86:0/2573798171 shutdown_connections 2024-04-25T14:10:29.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.341+0000 7fb695800640 1 --2- 172.21.15.86:0/2573798171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6900cf970 0x7fb690159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:29.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.341+0000 7fb695800640 1 --2- 172.21.15.86:0/2573798171 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fb67006f490 0x7fb6700719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:29.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.341+0000 7fb695800640 1 --2- 172.21.15.86:0/2573798171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb690071360 0x7fb690150a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:29.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.341+0000 7fb695800640 1 --2- 172.21.15.86:0/2573798171 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb69006e7a0 0x7fb6900b6ba0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:29.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.341+0000 7fb695800640 1 -- 172.21.15.86:0/2573798171 >> 172.21.15.86:0/2573798171 conn(0x7fb6900cffc0 msgr2=0x7fb6900b4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:29.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.341+0000 7fb695800640 1 -- 172.21.15.86:0/2573798171 shutdown_connections 2024-04-25T14:10:29.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.341+0000 7fb695800640 1 -- 172.21.15.86:0/2573798171 wait complete. 2024-04-25T14:10:29.361 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T14:10:29.380 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:29.379+0000 7f3cba623180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:10:29.442 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:10:29.443+0000 7f3cba623180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:10:29.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.513+0000 7fac0c400640 1 Processor -- start 2024-04-25T14:10:29.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.513+0000 7fac0c400640 1 -- start start 2024-04-25T14:10:29.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.517+0000 7fac0c400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fac040ce5a0 0x7fac04070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:29.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.517+0000 7fac0c400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac04071770 con 0x7fac04071360 2024-04-25T14:10:29.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.517+0000 7fac0c400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac04073110 con 0x7fac040d25c0 2024-04-25T14:10:29.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.517+0000 7fac0c400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fac04162380 con 0x7fac040ce5a0 2024-04-25T14:10:29.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.517+0000 7fac09a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fac040ce5a0 0x7fac04070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:29.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.517+0000 7fac09000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fac04071360 0x7fac040722f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39870/0 (socket says 172.21.15.86:39870) 2024-04-25T14:10:29.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.517+0000 7fac09000640 1 -- 172.21.15.86:0/3216115574 learned_addr learned my addr 172.21.15.86:0/3216115574 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:29.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.517+0000 7fac09a00640 1 -- 172.21.15.86:0/3216115574 >> v1:172.21.15.155:6789/0 conn(0x7fac040d25c0 legacy=0x7fac04072a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:29.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.517+0000 7fac09a00640 1 -- 172.21.15.86:0/3216115574 >> v1:172.21.15.86:6789/0 conn(0x7fac04071360 legacy=0x7fac040722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:29.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.517+0000 7fac03e00640 1 -- 172.21.15.86:0/3216115574 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3964290776 0 0) 0x7fabf0001120 con 0x7fac04071360 2024-04-25T14:10:29.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.517+0000 7fac09a00640 1 -- 172.21.15.86:0/3216115574 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac04161770 con 0x7fac040ce5a0 2024-04-25T14:10:29.516 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:29.517+0000 7f6812189180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:10:29.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.521+0000 7fac09a00640 1 --2- 172.21.15.86:0/3216115574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac040ce5a0 0x7fac04070e20 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fabf8009740 tx=0x7fabf801c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58428a80101cea21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:29.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.521+0000 7fac03e00640 1 -- 172.21.15.86:0/3216115574 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf80b0070 con 0x7fac040ce5a0 2024-04-25T14:10:29.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.521+0000 7fac03e00640 1 -- 172.21.15.86:0/3216115574 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabf80ab600 con 0x7fac040ce5a0 2024-04-25T14:10:29.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.521+0000 7fac03e00640 1 -- 172.21.15.86:0/3216115574 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf80ab9d0 con 0x7fac040ce5a0 2024-04-25T14:10:29.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.521+0000 7fac0c400640 1 -- 172.21.15.86:0/3216115574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac040ce5a0 msgr2=0x7fac04070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.521+0000 7fac0c400640 1 --2- 172.21.15.86:0/3216115574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac040ce5a0 0x7fac04070e20 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fabf8009740 tx=0x7fabf801c2d0 comp rx=0 tx=0).stop 2024-04-25T14:10:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.521+0000 7fac0c400640 1 -- 172.21.15.86:0/3216115574 shutdown_connections 2024-04-25T14:10:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.521+0000 7fac0c400640 1 --2- 172.21.15.86:0/3216115574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac040ce5a0 0x7fac04070e20 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.521+0000 7fac0c400640 1 -- 172.21.15.86:0/3216115574 >> 172.21.15.86:0/3216115574 conn(0x7fac040cffc0 msgr2=0x7fac040d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.521+0000 7fac0c400640 1 -- 172.21.15.86:0/3216115574 shutdown_connections 2024-04-25T14:10:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.525+0000 7fac0c400640 1 -- 172.21.15.86:0/3216115574 wait complete. 2024-04-25T14:10:29.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.525+0000 7fac0c400640 1 Processor -- start 2024-04-25T14:10:29.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.525+0000 7fac0c400640 1 -- start start 2024-04-25T14:10:29.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.529+0000 7fac0c400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac04071360 0x7fac04146be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:29.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.529+0000 7fac0c400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac040ce5a0 0x7fac040b7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:29.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.529+0000 7fac0c400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac040d25c0 0x7fac04146350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:29.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.529+0000 7fac0c400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac04162010 con 0x7fac04071360 2024-04-25T14:10:29.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.529+0000 7fac0c400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fac04161d30 con 0x7fac040ce5a0 2024-04-25T14:10:29.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.529+0000 7fac0c400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fac0415ef80 con 0x7fac040d25c0 2024-04-25T14:10:29.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.529+0000 7fac09a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac04071360 0x7fac04146be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:29.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.529+0000 7fac09a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac04071360 0x7fac04146be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57170/0 (socket says 172.21.15.86:57170) 2024-04-25T14:10:29.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.529+0000 7fac09a00640 1 -- 172.21.15.86:0/3191832745 learned_addr learned my addr 172.21.15.86:0/3191832745 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:29.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.529+0000 7fac09000640 1 --2- 172.21.15.86:0/3191832745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac040ce5a0 0x7fac040b7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:29.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.529+0000 7fac0a400640 1 --2- 172.21.15.86:0/3191832745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac040d25c0 0x7fac04146350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:29.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.529+0000 7fac09a00640 1 -- 172.21.15.86:0/3191832745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac040d25c0 msgr2=0x7fac04146350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:29.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.529+0000 7fac09a00640 1 --2- 172.21.15.86:0/3191832745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac040d25c0 0x7fac04146350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:29.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.529+0000 7fac09a00640 1 -- 172.21.15.86:0/3191832745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac040ce5a0 msgr2=0x7fac040b7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:29.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.529+0000 7fac09a00640 1 --2- 172.21.15.86:0/3191832745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac040ce5a0 0x7fac040b7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:29.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.529+0000 7fac09a00640 1 -- 172.21.15.86:0/3191832745 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac040b7ab0 con 0x7fac04071360 2024-04-25T14:10:29.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.529+0000 7fac0a400640 1 --2- 172.21.15.86:0/3191832745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac040d25c0 0x7fac04146350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:29.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.529+0000 7fac09a00640 1 --2- 172.21.15.86:0/3191832745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac04071360 0x7fac04146be0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fabf8018ee0 tx=0x7fabf801ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:29.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.529+0000 7fac01600640 1 -- 172.21.15.86:0/3191832745 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf80b0050 con 0x7fac04071360 2024-04-25T14:10:29.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.529+0000 7fac01600640 1 -- 172.21.15.86:0/3191832745 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabf80bc070 con 0x7fac04071360 2024-04-25T14:10:29.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.529+0000 7fac0c400640 1 -- 172.21.15.86:0/3191832745 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac040c8880 con 0x7fac04071360 2024-04-25T14:10:29.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.529+0000 7fac01600640 1 -- 172.21.15.86:0/3191832745 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf80aabe0 con 0x7fac04071360 2024-04-25T14:10:29.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.529+0000 7fac0c400640 1 -- 172.21.15.86:0/3191832745 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac040b8060 con 0x7fac04071360 2024-04-25T14:10:29.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.533+0000 7fac0c400640 1 -- 172.21.15.86:0/3191832745 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac040720e0 con 0x7fac04071360 2024-04-25T14:10:29.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.533+0000 7fac01600640 1 -- 172.21.15.86:0/3191832745 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 51) ==== 93253+0+0 (secure 0 0 0) 0x7fabf80c1460 con 0x7fac04071360 2024-04-25T14:10:29.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.533+0000 7fac01600640 1 --2- 172.21.15.86:0/3191832745 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fabe406f490 0x7fabe40719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:29.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.533+0000 7fac09000640 1 -- 172.21.15.86:0/3191832745 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fabe406f490 msgr2=0x7fabe40719a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/190090395 2024-04-25T14:10:29.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.533+0000 7fac01600640 1 -- 172.21.15.86:0/3191832745 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6445+0+0 (secure 0 0 0) 0x7fabf812a270 con 0x7fac04071360 2024-04-25T14:10:29.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.533+0000 7fac09000640 1 --2- 172.21.15.86:0/3191832745 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fabe406f490 0x7fabe40719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:10:29.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.537+0000 7fac01600640 1 -- 172.21.15.86:0/3191832745 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fabf80c6020 con 0x7fac04071360 2024-04-25T14:10:29.583 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:29.585+0000 7f6812189180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:10:29.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.597+0000 7fac01600640 1 -- 172.21.15.86:0/3191832745 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 52) ==== 92918+0+0 (secure 0 0 0) 0x7fabf80f28a0 con 0x7fac04071360 2024-04-25T14:10:29.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.597+0000 7fac01600640 1 -- 172.21.15.86:0/3191832745 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fabe406f490 msgr2=0x7fabe40719a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:10:29.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.597+0000 7fac01600640 1 --2- 172.21.15.86:0/3191832745 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fabe406f490 0x7fabe40719a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:29.663 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:29.665+0000 7f6794600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:10:29.663 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:29.665+0000 7f6794600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:10:29.663 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:29.665+0000 7f6794600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:10:29.663 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:29.665+0000 7f6794600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:10:29.663 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:29.665+0000 7f6794600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:10:29.664 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:29.665+0000 7f6790a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:10:29.664 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:29.665+0000 7f6790a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:10:29.665 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:29.665+0000 7f6790a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:10:29.665 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:29.665+0000 7f6790a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:10:29.665 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:29.665+0000 7f6790a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:10:29.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.729+0000 7fac0c400640 1 -- 172.21.15.86:0/3191832745 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fac04076bf0 con 0x7fac04071360 2024-04-25T14:10:29.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.733+0000 7fac01600640 1 -- 172.21.15.86:0/3191832745 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v115) ==== 81+0+878 (secure 0 0 0) 0x7fabf8015f50 con 0x7fac04071360 2024-04-25T14:10:29.729 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1062432465 2024-04-26T14:10:29.591964+0000 2024-04-25T14:10:29.730 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6832/190090395 2024-04-26T14:10:29.591964+0000 2024-04-25T14:10:29.730 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1120094610 2024-04-26T14:10:29.591964+0000 2024-04-25T14:10:29.730 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2412030790 2024-04-26T14:10:29.591964+0000 2024-04-25T14:10:29.730 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/4291479632 2024-04-26T14:10:29.591964+0000 2024-04-25T14:10:29.730 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2052329939 2024-04-26T14:10:23.454991+0000 2024-04-25T14:10:29.730 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1021944523 2024-04-26T14:10:23.454991+0000 2024-04-25T14:10:29.730 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/83615604 2024-04-26T14:10:23.454991+0000 2024-04-25T14:10:29.730 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3719613294 2024-04-26T14:10:23.454991+0000 2024-04-25T14:10:29.730 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3734007180 2024-04-26T14:10:29.591964+0000 2024-04-25T14:10:29.730 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1471025278 2024-04-26T14:10:23.454991+0000 2024-04-25T14:10:29.730 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6833/1354467416 2024-04-26T14:10:23.454991+0000 2024-04-25T14:10:29.730 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6832/1354467416 2024-04-26T14:10:23.454991+0000 2024-04-25T14:10:29.731 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6833/190090395 2024-04-26T14:10:29.591964+0000 2024-04-25T14:10:29.731 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3000491277 2024-04-26T14:10:23.454991+0000 2024-04-25T14:10:29.732 INFO:teuthology.orchestra.run.smithi086.stderr:listed 15 entries 2024-04-25T14:10:29.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.733+0000 7fac0c400640 1 -- 172.21.15.86:0/3191832745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac04071360 msgr2=0x7fac04146be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:29.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.733+0000 7fac0c400640 1 --2- 172.21.15.86:0/3191832745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac04071360 0x7fac04146be0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fabf8018ee0 tx=0x7fabf801ab10 comp rx=0 tx=0).stop 2024-04-25T14:10:29.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.733+0000 7fac0c400640 1 -- 172.21.15.86:0/3191832745 shutdown_connections 2024-04-25T14:10:29.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.733+0000 7fac0c400640 1 --2- 172.21.15.86:0/3191832745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac040d25c0 0x7fac04146350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:29.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.733+0000 7fac0c400640 1 --2- 172.21.15.86:0/3191832745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac040ce5a0 0x7fac040b7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:29.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.733+0000 7fac0c400640 1 --2- 172.21.15.86:0/3191832745 >> [v2:172.21.15.86:6832/190090395,v1:172.21.15.86:6833/190090395] conn(0x7fabe406f490 0x7fabe40719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:29.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.733+0000 7fac0c400640 1 --2- 172.21.15.86:0/3191832745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac04071360 0x7fac04146be0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:29.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.733+0000 7fac0c400640 1 -- 172.21.15.86:0/3191832745 >> 172.21.15.86:0/3191832745 conn(0x7fac040cffc0 msgr2=0x7fac040b48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:29.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.733+0000 7fac0c400640 1 -- 172.21.15.86:0/3191832745 shutdown_connections 2024-04-25T14:10:29.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:29.733+0000 7fac0c400640 1 -- 172.21.15.86:0/3191832745 wait complete. 2024-04-25T14:10:29.752 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T14:10:29.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:29.753 DEBUG:teuthology.orchestra.run.smithi155:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T14:10:29.778 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T14:10:29.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:29.779 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T14:10:29.828 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:10:29.829 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:10:29.829 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:10:29.838 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2893 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T14:10:29.838 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:10:29.838 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:10:29.838 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:10:29.880 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4517 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T14:10:29.881 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:10:30.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.029+0000 7fb31a400640 1 Processor -- start 2024-04-25T14:10:30.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.029+0000 7fb31a400640 1 -- start start 2024-04-25T14:10:30.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.029+0000 7fb31a400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb31406ebb0 0x7fb314070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:30.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.029+0000 7fb31a400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb314071360 0x7fb314163e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:30.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.029+0000 7fb31a400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb314071770 con 0x7fb31406e7a0 2024-04-25T14:10:30.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.029+0000 7fb31a400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb3140d1c10 con 0x7fb31406ebb0 2024-04-25T14:10:30.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.029+0000 7fb31a400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb3140d1ef0 con 0x7fb314071360 2024-04-25T14:10:30.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.029+0000 7fb30aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb31406ebb0 0x7fb314070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:30.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.029+0000 7fb30aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb31406ebb0 0x7fb314070e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57282/0 (socket says 172.21.15.86:57282) 2024-04-25T14:10:30.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.029+0000 7fb30aa00640 1 -- 172.21.15.86:0/2035611098 learned_addr learned my addr 172.21.15.86:0/2035611098 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:30.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.033+0000 7fb313e00640 1 --2- 172.21.15.86:0/2035611098 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb314071360 0x7fb314163e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:30.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.033+0000 7fb30aa00640 1 -- 172.21.15.86:0/2035611098 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb314071360 msgr2=0x7fb314163e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:30.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.033+0000 7fb30aa00640 1 --2- 172.21.15.86:0/2035611098 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb314071360 0x7fb314163e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:30.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.033+0000 7fb30aa00640 1 -- 172.21.15.86:0/2035611098 >> v1:172.21.15.86:6789/0 conn(0x7fb31406e7a0 legacy=0x7fb3141602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:30.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.033+0000 7fb30aa00640 1 -- 172.21.15.86:0/2035611098 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3141609b0 con 0x7fb31406ebb0 2024-04-25T14:10:30.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.033+0000 7fb30aa00640 1 --2- 172.21.15.86:0/2035611098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb31406ebb0 0x7fb314070e20 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc006fd0 tx=0x7fb2fc01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638afc17e52e93a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:30.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.033+0000 7fb312a00640 1 -- 172.21.15.86:0/2035611098 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc0b0070 con 0x7fb31406ebb0 2024-04-25T14:10:30.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.033+0000 7fb312a00640 1 -- 172.21.15.86:0/2035611098 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2fc0ab600 con 0x7fb31406ebb0 2024-04-25T14:10:30.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.033+0000 7fb312a00640 1 -- 172.21.15.86:0/2035611098 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc0ab9d0 con 0x7fb31406ebb0 2024-04-25T14:10:30.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.037+0000 7fb31a400640 1 -- 172.21.15.86:0/2035611098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb31406ebb0 msgr2=0x7fb314070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:30.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.037+0000 7fb31a400640 1 --2- 172.21.15.86:0/2035611098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb31406ebb0 0x7fb314070e20 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc006fd0 tx=0x7fb2fc01c4b0 comp rx=0 tx=0).stop 2024-04-25T14:10:30.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.037+0000 7fb31a400640 1 -- 172.21.15.86:0/2035611098 shutdown_connections 2024-04-25T14:10:30.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.037+0000 7fb31a400640 1 --2- 172.21.15.86:0/2035611098 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb314071360 0x7fb314163e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:30.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.037+0000 7fb31a400640 1 --2- 172.21.15.86:0/2035611098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb31406ebb0 0x7fb314070e20 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:30.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.037+0000 7fb31a400640 1 -- 172.21.15.86:0/2035611098 >> 172.21.15.86:0/2035611098 conn(0x7fb3140cffc0 msgr2=0x7fb3140d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:30.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.037+0000 7fb31a400640 1 -- 172.21.15.86:0/2035611098 shutdown_connections 2024-04-25T14:10:30.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.037+0000 7fb31a400640 1 -- 172.21.15.86:0/2035611098 wait complete. 2024-04-25T14:10:30.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.037+0000 7fb31a400640 1 Processor -- start 2024-04-25T14:10:30.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.037+0000 7fb31a400640 1 -- start start 2024-04-25T14:10:30.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.041+0000 7fb31a400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb31406e7a0 0x7fb3140ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:30.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.041+0000 7fb31a400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb31406ebb0 0x7fb314156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:30.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.041+0000 7fb31a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb314071360 0x7fb3140c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:30.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.041+0000 7fb31a400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb31416d5c0 con 0x7fb314071360 2024-04-25T14:10:30.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.041+0000 7fb31a400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb31416d440 con 0x7fb31406ebb0 2024-04-25T14:10:30.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.041+0000 7fb31a400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb31416d740 con 0x7fb31406e7a0 2024-04-25T14:10:30.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.041+0000 7fb313e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb314071360 0x7fb3140c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:30.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.041+0000 7fb313e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb314071360 0x7fb3140c94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57286/0 (socket says 172.21.15.86:57286) 2024-04-25T14:10:30.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.041+0000 7fb313e00640 1 -- 172.21.15.86:0/1135397707 learned_addr learned my addr 172.21.15.86:0/1135397707 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:30.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.041+0000 7fb313400640 1 --2- 172.21.15.86:0/1135397707 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb31406e7a0 0x7fb3140ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:30.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.041+0000 7fb30aa00640 1 --2- 172.21.15.86:0/1135397707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb31406ebb0 0x7fb314156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:30.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.041+0000 7fb313e00640 1 -- 172.21.15.86:0/1135397707 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb31406e7a0 msgr2=0x7fb3140ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:30.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.041+0000 7fb313e00640 1 --2- 172.21.15.86:0/1135397707 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb31406e7a0 0x7fb3140ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:30.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.041+0000 7fb313e00640 1 -- 172.21.15.86:0/1135397707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb31406ebb0 msgr2=0x7fb314156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:30.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.041+0000 7fb313e00640 1 --2- 172.21.15.86:0/1135397707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb31406ebb0 0x7fb314156a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:30.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.041+0000 7fb313e00640 1 -- 172.21.15.86:0/1135397707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb314156f60 con 0x7fb314071360 2024-04-25T14:10:30.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.041+0000 7fb313e00640 1 --2- 172.21.15.86:0/1135397707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb314071360 0x7fb3140c94e0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00ef90 tx=0x7fb30c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:30.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.041+0000 7fb30be00640 1 -- 172.21.15.86:0/1135397707 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c018020 con 0x7fb314071360 2024-04-25T14:10:30.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.041+0000 7fb30be00640 1 -- 172.21.15.86:0/1135397707 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb30c014480 con 0x7fb314071360 2024-04-25T14:10:30.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.041+0000 7fb30be00640 1 -- 172.21.15.86:0/1135397707 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c009e60 con 0x7fb314071360 2024-04-25T14:10:30.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.041+0000 7fb31a400640 1 -- 172.21.15.86:0/1135397707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3140cabd0 con 0x7fb314071360 2024-04-25T14:10:30.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.041+0000 7fb31a400640 1 -- 172.21.15.86:0/1135397707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb31415df30 con 0x7fb314071360 2024-04-25T14:10:30.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.041+0000 7fb31a400640 1 -- 172.21.15.86:0/1135397707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3140720e0 con 0x7fb314071360 2024-04-25T14:10:30.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.041+0000 7fb30be00640 1 -- 172.21.15.86:0/1135397707 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 52) ==== 92918+0+0 (secure 0 0 0) 0x7fb30c01b020 con 0x7fb314071360 2024-04-25T14:10:30.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.045+0000 7fb30be00640 1 -- 172.21.15.86:0/1135397707 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6746+0+0 (secure 0 0 0) 0x7fb30c08fe70 con 0x7fb314071360 2024-04-25T14:10:30.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.049+0000 7fb30be00640 1 -- 172.21.15.86:0/1135397707 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb30c022020 con 0x7fb314071360 2024-04-25T14:10:30.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.237+0000 7fb31a400640 1 -- 172.21.15.86:0/1135397707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb31416a9d0 con 0x7fb314071360 2024-04-25T14:10:30.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.241+0000 7fb30be00640 1 -- 172.21.15.86:0/1135397707 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5183 (secure 0 0 0) 0x7fb30c010040 con 0x7fb314071360 2024-04-25T14:10:30.237 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:30.237 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":85,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/374465657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":374465657},{"type":"v1","addr":"172.21.15.86:6836","nonce":374465657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8107,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1466195534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":1466195534},{"type":"v1","addr":"172.21.15.86:6837","nonce":1466195534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8119,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2431580070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2431580070},{"type":"v1","addr":"172.21.15.155:6835","nonce":2431580070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:10:05.129470+0000","modified":"2024-04-25T14:10:08.145259+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8457":{"gid":8457,"name":"b","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6833/142240252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":142240252},{"type":"v1","addr":"172.21.15.155:6833","nonce":142240252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8457,"qdb_cluster":[8457]},"id":8}]} 2024-04-25T14:10:30.237 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 85 2024-04-25T14:10:30.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.241+0000 7fb31a400640 1 -- 172.21.15.86:0/1135397707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb314071360 msgr2=0x7fb3140c94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:30.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.241+0000 7fb31a400640 1 --2- 172.21.15.86:0/1135397707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb314071360 0x7fb3140c94e0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00ef90 tx=0x7fb30c00c490 comp rx=0 tx=0).stop 2024-04-25T14:10:30.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.241+0000 7fb31a400640 1 -- 172.21.15.86:0/1135397707 shutdown_connections 2024-04-25T14:10:30.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.241+0000 7fb31a400640 1 --2- 172.21.15.86:0/1135397707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb314071360 0x7fb3140c94e0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:30.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.241+0000 7fb31a400640 1 --2- 172.21.15.86:0/1135397707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb31406ebb0 0x7fb314156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:30.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.241+0000 7fb31a400640 1 --2- 172.21.15.86:0/1135397707 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb31406e7a0 0x7fb3140ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:30.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.241+0000 7fb31a400640 1 -- 172.21.15.86:0/1135397707 >> 172.21.15.86:0/1135397707 conn(0x7fb3140cffc0 msgr2=0x7fb3140b5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:30.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.241+0000 7fb31a400640 1 -- 172.21.15.86:0/1135397707 shutdown_connections 2024-04-25T14:10:30.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.241+0000 7fb31a400640 1 -- 172.21.15.86:0/1135397707 wait complete. 2024-04-25T14:10:30.260 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:10:30.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.409+0000 7f7acb200640 1 Processor -- start 2024-04-25T14:10:30.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.409+0000 7f7acb200640 1 -- start start 2024-04-25T14:10:30.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.409+0000 7f7acb200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ac40d2c80 0x7f7ac415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:30.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.409+0000 7f7acb200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7ac4070e20 0x7f7ac4071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:30.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.409+0000 7f7acb200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ac4071770 con 0x7f7ac40722f0 2024-04-25T14:10:30.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.409+0000 7f7acb200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f7ac40ce720 con 0x7f7ac4070e20 2024-04-25T14:10:30.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.409+0000 7f7acb200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7ac40cd8d0 con 0x7f7ac40d2c80 2024-04-25T14:10:30.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.409+0000 7f7ac9200640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f7ac40722f0 0x7f7ac4072700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:52848/0 (socket says 172.21.15.86:52848) 2024-04-25T14:10:30.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.409+0000 7f7abb400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7ac4070e20 0x7f7ac4071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:30.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.409+0000 7f7ac9200640 1 -- 172.21.15.86:0/3934934662 learned_addr learned my addr 172.21.15.86:0/3934934662 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:30.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.409+0000 7f7ac3e00640 1 --2- 172.21.15.86:0/3934934662 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ac40d2c80 0x7f7ac415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:30.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.413+0000 7f7ac3400640 1 -- 172.21.15.86:0/3934934662 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1519561290 0 0) 0x7f7abc0014a0 con 0x7f7ac40722f0 2024-04-25T14:10:30.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.413+0000 7f7ac3400640 1 -- 172.21.15.86:0/3934934662 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7aa8003530 con 0x7f7ac40722f0 2024-04-25T14:10:30.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.413+0000 7f7ac3e00640 1 -- 172.21.15.86:0/3934934662 >> v2:172.21.15.155:3300/0 conn(0x7f7ac4070e20 msgr2=0x7f7ac4071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:30.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.413+0000 7f7ac3e00640 1 --2- 172.21.15.86:0/3934934662 >> v2:172.21.15.155:3300/0 conn(0x7f7ac4070e20 0x7f7ac4071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:30.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.413+0000 7f7ac3e00640 1 -- 172.21.15.86:0/3934934662 >> v1:172.21.15.155:6789/0 conn(0x7f7ac40722f0 legacy=0x7f7ac4072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:30.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.413+0000 7f7ac3e00640 1 -- 172.21.15.86:0/3934934662 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac40ce970 con 0x7f7ac40d2c80 2024-04-25T14:10:30.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.413+0000 7f7abb400640 1 --2- 172.21.15.86:0/3934934662 >> v2:172.21.15.155:3300/0 conn(0x7f7ac4070e20 0x7f7ac4071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:10:30.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.413+0000 7f7ac3e00640 1 --2- 172.21.15.86:0/3934934662 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ac40d2c80 0x7f7ac415dc90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0097c0 tx=0x7f7aac01ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4848914887536933 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:30.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.413+0000 7f7ac3400640 1 -- 172.21.15.86:0/3934934662 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac0b5070 con 0x7f7ac40d2c80 2024-04-25T14:10:30.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.413+0000 7f7ac3400640 1 -- 172.21.15.86:0/3934934662 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aac0a8990 con 0x7f7ac40d2c80 2024-04-25T14:10:30.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.413+0000 7f7ac3400640 1 -- 172.21.15.86:0/3934934662 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac0a8d60 con 0x7f7ac40d2c80 2024-04-25T14:10:30.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.413+0000 7f7acb200640 1 -- 172.21.15.86:0/3934934662 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ac40d2c80 msgr2=0x7f7ac415dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:30.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.413+0000 7f7acb200640 1 --2- 172.21.15.86:0/3934934662 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ac40d2c80 0x7f7ac415dc90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0097c0 tx=0x7f7aac01ec90 comp rx=0 tx=0).stop 2024-04-25T14:10:30.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.417+0000 7f7acb200640 1 -- 172.21.15.86:0/3934934662 shutdown_connections 2024-04-25T14:10:30.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.417+0000 7f7acb200640 1 --2- 172.21.15.86:0/3934934662 >> v2:172.21.15.155:3300/0 conn(0x7f7ac4070e20 0x7f7ac4071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:30.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.417+0000 7f7acb200640 1 --2- 172.21.15.86:0/3934934662 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ac40d2c80 0x7f7ac415dc90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:30.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.417+0000 7f7acb200640 1 -- 172.21.15.86:0/3934934662 >> 172.21.15.86:0/3934934662 conn(0x7f7ac40cffc0 msgr2=0x7f7ac40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:30.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.417+0000 7f7acb200640 1 -- 172.21.15.86:0/3934934662 shutdown_connections 2024-04-25T14:10:30.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.417+0000 7f7acb200640 1 -- 172.21.15.86:0/3934934662 wait complete. 2024-04-25T14:10:30.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.417+0000 7f7acb200640 1 Processor -- start 2024-04-25T14:10:30.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.417+0000 7f7acb200640 1 -- start start 2024-04-25T14:10:30.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.417+0000 7f7acb200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ac4070e20 0x7f7ac40ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:30.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.417+0000 7f7acb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ac40722f0 0x7f7ac41539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:30.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.417+0000 7f7acb200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ac40d2c80 0x7f7ac415ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:30.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.417+0000 7f7acb200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac40d21a0 con 0x7f7ac40722f0 2024-04-25T14:10:30.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.417+0000 7f7acb200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac40d1ec0 con 0x7f7ac40d2c80 2024-04-25T14:10:30.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.417+0000 7f7acb200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7ac40d2480 con 0x7f7ac4070e20 2024-04-25T14:10:30.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.421+0000 7f7ac3e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ac4070e20 0x7f7ac40ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:30.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.421+0000 7f7ac9200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ac40d2c80 0x7f7ac415ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:30.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.421+0000 7f7ac3e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ac4070e20 0x7f7ac40ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:55184/0 (socket says 172.21.15.86:55184) 2024-04-25T14:10:30.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.421+0000 7f7ac3e00640 1 -- 172.21.15.86:0/2307455894 learned_addr learned my addr 172.21.15.86:0/2307455894 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:30.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.421+0000 7f7abb400640 1 --2- 172.21.15.86:0/2307455894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ac40722f0 0x7f7ac41539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:30.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.421+0000 7f7ac3e00640 1 -- 172.21.15.86:0/2307455894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ac40d2c80 msgr2=0x7f7ac415ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:30.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.421+0000 7f7ac3e00640 1 --2- 172.21.15.86:0/2307455894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ac40d2c80 0x7f7ac415ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:30.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.421+0000 7f7ac3e00640 1 -- 172.21.15.86:0/2307455894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ac40722f0 msgr2=0x7f7ac41539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:30.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.421+0000 7f7ac3e00640 1 --2- 172.21.15.86:0/2307455894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ac40722f0 0x7f7ac41539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:30.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.421+0000 7f7ac3e00640 1 -- 172.21.15.86:0/2307455894 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac40af370 con 0x7f7ac4070e20 2024-04-25T14:10:30.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.421+0000 7f7ac3e00640 1 --2- 172.21.15.86:0/2307455894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ac4070e20 0x7f7ac40ae8b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0097c0 tx=0x7f7aac01ba30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:30.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.421+0000 7f7ac0c00640 1 -- 172.21.15.86:0/2307455894 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac0c1070 con 0x7f7ac4070e20 2024-04-25T14:10:30.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.421+0000 7f7ac0c00640 1 -- 172.21.15.86:0/2307455894 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aac0b0640 con 0x7f7ac4070e20 2024-04-25T14:10:30.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.421+0000 7f7ac0c00640 1 -- 172.21.15.86:0/2307455894 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac0b5040 con 0x7f7ac4070e20 2024-04-25T14:10:30.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.421+0000 7f7acb200640 1 -- 172.21.15.86:0/2307455894 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ac40af650 con 0x7f7ac4070e20 2024-04-25T14:10:30.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.421+0000 7f7acb200640 1 -- 172.21.15.86:0/2307455894 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ac40afc10 con 0x7f7ac4070e20 2024-04-25T14:10:30.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.421+0000 7f7acb200640 1 -- 172.21.15.86:0/2307455894 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ac40720e0 con 0x7f7ac4070e20 2024-04-25T14:10:30.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.421+0000 7f7ac0c00640 1 -- 172.21.15.86:0/2307455894 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 52) ==== 92918+0+0 (secure 0 0 0) 0x7f7aac0b0a30 con 0x7f7ac4070e20 2024-04-25T14:10:30.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.425+0000 7f7ac0c00640 1 -- 172.21.15.86:0/2307455894 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(115..115 src has 1..115) ==== 6746+0+0 (secure 0 0 0) 0x7f7aac12be30 con 0x7f7ac4070e20 2024-04-25T14:10:30.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.429+0000 7f7ac0c00640 1 -- 172.21.15.86:0/2307455894 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7aac0187f0 con 0x7f7ac4070e20 2024-04-25T14:10:30.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.589+0000 7f7acb200640 1 -- 172.21.15.86:0/2307455894 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ac4076bf0 con 0x7f7ac4070e20 2024-04-25T14:10:30.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.589+0000 7f7ac0c00640 1 -- 172.21.15.86:0/2307455894 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+17889 (secure 0 0 0) 0x7f7aac0f8620 con 0x7f7ac4070e20 2024-04-25T14:10:30.587 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:30.587 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":115,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:10:29.591997+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:10:03.520649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T14:10:04.080912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1062432465":"2024-04-26T14:10:29.591964+0000","172.21.15.86:6832/190090395":"2024-04-26T14:10:29.591964+0000","172.21.15.86:0/1120094610":"2024-04-26T14:10:29.591964+0000","172.21.15.86:0/2412030790":"2024-04-26T14:10:29.591964+0000","172.21.15.86:0/4291479632":"2024-04-26T14:10:29.591964+0000","172.21.15.86:0/2052329939":"2024-04-26T14:10:23.454991+0000","172.21.15.86:0/1021944523":"2024-04-26T14:10:23.454991+0000","172.21.15.86:0/83615604":"2024-04-26T14:10:23.454991+0000","172.21.15.86:0/3719613294":"2024-04-26T14:10:23.454991+0000","172.21.15.86:0/3734007180":"2024-04-26T14:10:29.591964+0000","172.21.15.86:0/1471025278":"2024-04-26T14:10:23.454991+0000","172.21.15.86:6833/1354467416":"2024-04-26T14:10:23.454991+0000","172.21.15.86:6832/1354467416":"2024-04-26T14:10:23.454991+0000","172.21.15.86:6833/190090395":"2024-04-26T14:10:29.591964+0000","172.21.15.86:0/3000491277":"2024-04-26T14:10:23.454991+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:10:30.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.589+0000 7f7acb200640 1 -- 172.21.15.86:0/2307455894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ac4070e20 msgr2=0x7f7ac40ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:30.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.589+0000 7f7acb200640 1 --2- 172.21.15.86:0/2307455894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ac4070e20 0x7f7ac40ae8b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0097c0 tx=0x7f7aac01ba30 comp rx=0 tx=0).stop 2024-04-25T14:10:30.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.589+0000 7f7acb200640 1 -- 172.21.15.86:0/2307455894 shutdown_connections 2024-04-25T14:10:30.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.589+0000 7f7acb200640 1 --2- 172.21.15.86:0/2307455894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ac40d2c80 0x7f7ac415ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:30.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.589+0000 7f7acb200640 1 --2- 172.21.15.86:0/2307455894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ac40722f0 0x7f7ac41539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:30.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.589+0000 7f7acb200640 1 --2- 172.21.15.86:0/2307455894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ac4070e20 0x7f7ac40ae8b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:30.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.589+0000 7f7acb200640 1 -- 172.21.15.86:0/2307455894 >> 172.21.15.86:0/2307455894 conn(0x7f7ac40cffc0 msgr2=0x7f7ac40c8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:30.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.589+0000 7f7acb200640 1 -- 172.21.15.86:0/2307455894 shutdown_connections 2024-04-25T14:10:30.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.589+0000 7f7acb200640 1 -- 172.21.15.86:0/2307455894 wait complete. 2024-04-25T14:10:30.617 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:10:03.520649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:10:04.080912+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:10:30.617 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:10:30.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.773+0000 7fbfe4000640 1 Processor -- start 2024-04-25T14:10:30.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.773+0000 7fbfe4000640 1 -- start start 2024-04-25T14:10:30.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.773+0000 7fbfe4000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfdc070330 0x7fbfdc0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:30.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.773+0000 7fbfe4000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbfdc0ce790 0x7fbfdc06f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:30.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.773+0000 7fbfe4000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfdc0d2030 con 0x7fbfdc0d1c20 2024-04-25T14:10:30.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.773+0000 7fbfe4000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbfdc15e5c0 con 0x7fbfdc0ce790 2024-04-25T14:10:30.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.773+0000 7fbfe4000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbfdc15e8a0 con 0x7fbfdc070330 2024-04-25T14:10:30.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.773+0000 7fbfe2000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbfdc0ce790 0x7fbfdc06f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:30.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.773+0000 7fbfe2000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbfdc0ce790 0x7fbfdc06f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57302/0 (socket says 172.21.15.86:57302) 2024-04-25T14:10:30.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.773+0000 7fbfe2000640 1 -- 172.21.15.86:0/423958908 learned_addr learned my addr 172.21.15.86:0/423958908 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:30.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.773+0000 7fbfe1600640 1 --2- 172.21.15.86:0/423958908 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfdc070330 0x7fbfdc0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:30.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.773+0000 7fbfe2000640 1 -- 172.21.15.86:0/423958908 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfdc070330 msgr2=0x7fbfdc0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:30.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.773+0000 7fbfe2000640 1 --2- 172.21.15.86:0/423958908 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfdc070330 0x7fbfdc0d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:30.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.773+0000 7fbfe2000640 1 -- 172.21.15.86:0/423958908 >> v1:172.21.15.155:6789/0 conn(0x7fbfdc0d1c20 legacy=0x7fbfdc06ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:30.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.773+0000 7fbfe2000640 1 -- 172.21.15.86:0/423958908 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfdc15eb80 con 0x7fbfdc0ce790 2024-04-25T14:10:30.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.777+0000 7fbfe2000640 1 --2- 172.21.15.86:0/423958908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfdc0ce790 0x7fbfdc06f440 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4009e80 tx=0x7fbfd401c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e11b478607bdc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:30.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.777+0000 7fbfdbe00640 1 -- 172.21.15.86:0/423958908 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd40a5640 con 0x7fbfdc0ce790 2024-04-25T14:10:30.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.777+0000 7fbfdbe00640 1 -- 172.21.15.86:0/423958908 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfd40a57e0 con 0x7fbfdc0ce790 2024-04-25T14:10:30.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.777+0000 7fbfdbe00640 1 -- 172.21.15.86:0/423958908 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd40a5bb0 con 0x7fbfdc0ce790 2024-04-25T14:10:30.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.777+0000 7fbfe4000640 1 -- 172.21.15.86:0/423958908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfdc0ce790 msgr2=0x7fbfdc06f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:30.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.777+0000 7fbfe4000640 1 --2- 172.21.15.86:0/423958908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfdc0ce790 0x7fbfdc06f440 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4009e80 tx=0x7fbfd401c980 comp rx=0 tx=0).stop 2024-04-25T14:10:30.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.777+0000 7fbfe4000640 1 -- 172.21.15.86:0/423958908 shutdown_connections 2024-04-25T14:10:30.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.777+0000 7fbfe4000640 1 --2- 172.21.15.86:0/423958908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfdc0ce790 0x7fbfdc06f440 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:30.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.777+0000 7fbfe4000640 1 --2- 172.21.15.86:0/423958908 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfdc070330 0x7fbfdc0d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:30.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.777+0000 7fbfe4000640 1 -- 172.21.15.86:0/423958908 >> 172.21.15.86:0/423958908 conn(0x7fbfdc0b4650 msgr2=0x7fbfdc0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:30.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.777+0000 7fbfe4000640 1 -- 172.21.15.86:0/423958908 shutdown_connections 2024-04-25T14:10:30.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.777+0000 7fbfe4000640 1 -- 172.21.15.86:0/423958908 wait complete. 2024-04-25T14:10:30.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.781+0000 7fbfe4000640 1 Processor -- start 2024-04-25T14:10:30.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.781+0000 7fbfe4000640 1 -- start start 2024-04-25T14:10:30.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.781+0000 7fbfe4000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfdc070330 0x7fbfdc0c3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:30.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.781+0000 7fbfe4000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfdc0ce790 0x7fbfdc09a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:30.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.781+0000 7fbfe4000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfdc0d1c20 0x7fbfdc15bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:30.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.781+0000 7fbfe4000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbfdc0708c0 con 0x7fbfdc070330 2024-04-25T14:10:30.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.781+0000 7fbfe4000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbfdc070740 con 0x7fbfdc0ce790 2024-04-25T14:10:30.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.781+0000 7fbfe4000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbfdc16afc0 con 0x7fbfdc0d1c20 2024-04-25T14:10:30.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.781+0000 7fbfe1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfdc070330 0x7fbfdc0c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:30.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.781+0000 7fbfe1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfdc070330 0x7fbfdc0c3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57310/0 (socket says 172.21.15.86:57310) 2024-04-25T14:10:30.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.781+0000 7fbfe1600640 1 -- 172.21.15.86:0/1499630660 learned_addr learned my addr 172.21.15.86:0/1499630660 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:30.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.781+0000 7fbfe0c00640 1 --2- 172.21.15.86:0/1499630660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfdc0ce790 0x7fbfdc09a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:30.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.781+0000 7fbfe2000640 1 --2- 172.21.15.86:0/1499630660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfdc0d1c20 0x7fbfdc15bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:30.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.781+0000 7fbfe1600640 1 -- 172.21.15.86:0/1499630660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfdc0d1c20 msgr2=0x7fbfdc15bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:30.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.781+0000 7fbfe1600640 1 --2- 172.21.15.86:0/1499630660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfdc0d1c20 0x7fbfdc15bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:30.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.781+0000 7fbfe1600640 1 -- 172.21.15.86:0/1499630660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfdc0ce790 msgr2=0x7fbfdc09a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:30.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.781+0000 7fbfe1600640 1 --2- 172.21.15.86:0/1499630660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfdc0ce790 0x7fbfdc09a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:30.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.781+0000 7fbfe1600640 1 -- 172.21.15.86:0/1499630660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfdc09aeb0 con 0x7fbfdc070330 2024-04-25T14:10:30.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.781+0000 7fbfe1600640 1 --2- 172.21.15.86:0/1499630660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfdc070330 0x7fbfdc0c3710 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000d5d0 tx=0x7fbfd000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:30.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.781+0000 7fbfd9600640 1 -- 172.21.15.86:0/1499630660 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0004240 con 0x7fbfdc070330 2024-04-25T14:10:30.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.781+0000 7fbfd9600640 1 -- 172.21.15.86:0/1499630660 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfd00043e0 con 0x7fbfdc070330 2024-04-25T14:10:30.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.781+0000 7fbfd9600640 1 -- 172.21.15.86:0/1499630660 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0003840 con 0x7fbfdc070330 2024-04-25T14:10:30.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.781+0000 7fbfe4000640 1 -- 172.21.15.86:0/1499630660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfdc1fbec0 con 0x7fbfdc070330 2024-04-25T14:10:30.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.781+0000 7fbfe4000640 1 -- 172.21.15.86:0/1499630660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfdc0c8560 con 0x7fbfdc070330 2024-04-25T14:10:30.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.785+0000 7fbfe4000640 1 -- 172.21.15.86:0/1499630660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfdc069d00 con 0x7fbfdc070330 2024-04-25T14:10:30.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.785+0000 7fbfd9600640 1 -- 172.21.15.86:0/1499630660 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 53) ==== 93045+0+0 (secure 0 0 0) 0x7fbfd00039e0 con 0x7fbfdc070330 2024-04-25T14:10:30.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.785+0000 7fbfd9600640 1 --2- 172.21.15.86:0/1499630660 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fbfbc06f280 0x7fbfbc071790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:30.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.785+0000 7fbfd9600640 1 -- 172.21.15.86:0/1499630660 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6746+0+0 (secure 0 0 0) 0x7fbfd008f320 con 0x7fbfdc070330 2024-04-25T14:10:30.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.785+0000 7fbfe0c00640 1 --2- 172.21.15.86:0/1499630660 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fbfbc06f280 0x7fbfbc071790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:30.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.785+0000 7fbfe0c00640 1 --2- 172.21.15.86:0/1499630660 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fbfbc06f280 0x7fbfbc071790 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc15b1a0 tx=0x7fbfc8009040 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:30.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.789+0000 7fbfd9600640 1 -- 172.21.15.86:0/1499630660 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbfd005baf0 con 0x7fbfdc070330 2024-04-25T14:10:30.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.945+0000 7fbfe4000640 1 -- 172.21.15.86:0/1499630660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbfdc0b0b60 con 0x7fbfdc070330 2024-04-25T14:10:30.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.945+0000 7fbfd9600640 1 -- 172.21.15.86:0/1499630660 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v115) ==== 93+0+10 (secure 0 0 0) 0x7fbfd005fdf0 con 0x7fbfdc070330 2024-04-25T14:10:30.943 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:10:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.949+0000 7fbfe4000640 1 -- 172.21.15.86:0/1499630660 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fbfbc06f280 msgr2=0x7fbfbc071790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.949+0000 7fbfe4000640 1 --2- 172.21.15.86:0/1499630660 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fbfbc06f280 0x7fbfbc071790 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc15b1a0 tx=0x7fbfc8009040 comp rx=0 tx=0).stop 2024-04-25T14:10:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.949+0000 7fbfe4000640 1 -- 172.21.15.86:0/1499630660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfdc070330 msgr2=0x7fbfdc0c3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.949+0000 7fbfe4000640 1 --2- 172.21.15.86:0/1499630660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfdc070330 0x7fbfdc0c3710 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000d5d0 tx=0x7fbfd000da90 comp rx=0 tx=0).stop 2024-04-25T14:10:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.949+0000 7fbfe4000640 1 -- 172.21.15.86:0/1499630660 shutdown_connections 2024-04-25T14:10:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.949+0000 7fbfe4000640 1 --2- 172.21.15.86:0/1499630660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfdc0d1c20 0x7fbfdc15bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.949+0000 7fbfe4000640 1 --2- 172.21.15.86:0/1499630660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfdc0ce790 0x7fbfdc09a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.949+0000 7fbfe4000640 1 --2- 172.21.15.86:0/1499630660 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fbfbc06f280 0x7fbfbc071790 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.949+0000 7fbfe4000640 1 --2- 172.21.15.86:0/1499630660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfdc070330 0x7fbfdc0c3710 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.949+0000 7fbfe4000640 1 -- 172.21.15.86:0/1499630660 >> 172.21.15.86:0/1499630660 conn(0x7fbfdc0b4650 msgr2=0x7fbfdc0aef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.949+0000 7fbfe4000640 1 -- 172.21.15.86:0/1499630660 shutdown_connections 2024-04-25T14:10:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:30.949+0000 7fbfe4000640 1 -- 172.21.15.86:0/1499630660 wait complete. 2024-04-25T14:10:30.966 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:10:31.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.117+0000 7f14d5c00640 1 Processor -- start 2024-04-25T14:10:31.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.117+0000 7f14d5c00640 1 -- start start 2024-04-25T14:10:31.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.121+0000 7f14d5c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f14d0070330 0x7f14d00d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:31.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.121+0000 7f14d5c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14d00d2030 con 0x7f14d00d1c20 2024-04-25T14:10:31.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.121+0000 7f14d5c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14d015e5c0 con 0x7f14d00ce790 2024-04-25T14:10:31.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.121+0000 7f14d5c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f14d015e8a0 con 0x7f14d0070330 2024-04-25T14:10:31.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.121+0000 7f14ce000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f14d00d1c20 0x7f14d006ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39888/0 (socket says 172.21.15.86:39888) 2024-04-25T14:10:31.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.121+0000 7f14ce000640 1 -- 172.21.15.86:0/247967061 learned_addr learned my addr 172.21.15.86:0/247967061 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:31.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.121+0000 7f14cea00640 1 --2- 172.21.15.86:0/247967061 >> v2:172.21.15.155:3300/0 conn(0x7f14d0070330 0x7f14d00d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:31.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.121+0000 7f14cd600640 1 -- 172.21.15.86:0/247967061 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2742543396 0 0) 0x7f14bc001120 con 0x7f14d00d1c20 2024-04-25T14:10:31.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.121+0000 7f14cd600640 1 -- 172.21.15.86:0/247967061 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14b4003530 con 0x7f14d00d1c20 2024-04-25T14:10:31.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.121+0000 7f14cd600640 1 -- 172.21.15.86:0/247967061 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3809481187 0 0) 0x7f14d015e5c0 con 0x7f14d00ce790 2024-04-25T14:10:31.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.121+0000 7f14cd600640 1 -- 172.21.15.86:0/247967061 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14bc001120 con 0x7f14d00ce790 2024-04-25T14:10:31.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.121+0000 7f14cea00640 1 -- 172.21.15.86:0/247967061 >> v1:172.21.15.155:6789/0 conn(0x7f14d00ce790 legacy=0x7f14d006f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:31.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.121+0000 7f14cea00640 1 -- 172.21.15.86:0/247967061 >> v1:172.21.15.86:6789/0 conn(0x7f14d00d1c20 legacy=0x7f14d006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:31.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.121+0000 7f14cea00640 1 -- 172.21.15.86:0/247967061 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d015eb80 con 0x7f14d0070330 2024-04-25T14:10:31.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.125+0000 7f14cea00640 1 --2- 172.21.15.86:0/247967061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14d0070330 0x7f14d00d16e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009dc0 tx=0x7f14c001c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d47d565b0f6ef1c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:31.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.125+0000 7f14cd600640 1 -- 172.21.15.86:0/247967061 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c00b2070 con 0x7f14d0070330 2024-04-25T14:10:31.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.125+0000 7f14cd600640 1 -- 172.21.15.86:0/247967061 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14c00a5850 con 0x7f14d0070330 2024-04-25T14:10:31.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.125+0000 7f14cd600640 1 -- 172.21.15.86:0/247967061 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c00a5c20 con 0x7f14d0070330 2024-04-25T14:10:31.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.125+0000 7f14d5c00640 1 -- 172.21.15.86:0/247967061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14d0070330 msgr2=0x7f14d00d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:31.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.125+0000 7f14d5c00640 1 --2- 172.21.15.86:0/247967061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14d0070330 0x7f14d00d16e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009dc0 tx=0x7f14c001c8c0 comp rx=0 tx=0).stop 2024-04-25T14:10:31.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.125+0000 7f14d5c00640 1 -- 172.21.15.86:0/247967061 shutdown_connections 2024-04-25T14:10:31.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.125+0000 7f14d5c00640 1 --2- 172.21.15.86:0/247967061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14d0070330 0x7f14d00d16e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:31.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.125+0000 7f14d5c00640 1 -- 172.21.15.86:0/247967061 >> 172.21.15.86:0/247967061 conn(0x7f14d00b4650 msgr2=0x7f14d00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:31.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.125+0000 7f14d5c00640 1 -- 172.21.15.86:0/247967061 shutdown_connections 2024-04-25T14:10:31.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.125+0000 7f14d5c00640 1 -- 172.21.15.86:0/247967061 wait complete. 2024-04-25T14:10:31.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.125+0000 7f14d5c00640 1 Processor -- start 2024-04-25T14:10:31.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.125+0000 7f14d5c00640 1 -- start start 2024-04-25T14:10:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.129+0000 7f14d5c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14d0070330 0x7f14d009ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.129+0000 7f14d5c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14d00ce790 0x7f14d00c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.129+0000 7f14d5c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14d00d1c20 0x7f14d015a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.129+0000 7f14d5c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14d00708c0 con 0x7f14d0070330 2024-04-25T14:10:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.129+0000 7f14d5c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f14d0070740 con 0x7f14d00d1c20 2024-04-25T14:10:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.129+0000 7f14d5c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f14d006fb50 con 0x7f14d00ce790 2024-04-25T14:10:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.129+0000 7f14cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14d0070330 0x7f14d009ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.129+0000 7f14cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14d0070330 0x7f14d009ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57320/0 (socket says 172.21.15.86:57320) 2024-04-25T14:10:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.129+0000 7f14cea00640 1 -- 172.21.15.86:0/3243138200 learned_addr learned my addr 172.21.15.86:0/3243138200 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.129+0000 7f14cea00640 1 -- 172.21.15.86:0/3243138200 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14d00ce790 msgr2=0x7f14d00c3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.129+0000 7f14cea00640 1 --2- 172.21.15.86:0/3243138200 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14d00ce790 0x7f14d00c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.129+0000 7f14cea00640 1 -- 172.21.15.86:0/3243138200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14d00d1c20 msgr2=0x7f14d015a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.129+0000 7f14cf400640 1 --2- 172.21.15.86:0/3243138200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14d00d1c20 0x7f14d015a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.129+0000 7f14cea00640 1 --2- 172.21.15.86:0/3243138200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14d00d1c20 0x7f14d015a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.129+0000 7f14cea00640 1 -- 172.21.15.86:0/3243138200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d00c4460 con 0x7f14d0070330 2024-04-25T14:10:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.129+0000 7f14cf400640 1 --2- 172.21.15.86:0/3243138200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14d00d1c20 0x7f14d015a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.129+0000 7f14cea00640 1 --2- 172.21.15.86:0/3243138200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14d0070330 0x7f14d009ee70 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f14c00194d0 tx=0x7f14c0019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.129+0000 7f14c6a00640 1 -- 172.21.15.86:0/3243138200 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c00bd020 con 0x7f14d0070330 2024-04-25T14:10:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.129+0000 7f14c6a00640 1 -- 172.21.15.86:0/3243138200 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14c00b3040 con 0x7f14d0070330 2024-04-25T14:10:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.129+0000 7f14d5c00640 1 -- 172.21.15.86:0/3243138200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d0155bc0 con 0x7f14d0070330 2024-04-25T14:10:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.129+0000 7f14c6a00640 1 -- 172.21.15.86:0/3243138200 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c00b2070 con 0x7f14d0070330 2024-04-25T14:10:31.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.129+0000 7f14d5c00640 1 -- 172.21.15.86:0/3243138200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d01535d0 con 0x7f14d0070330 2024-04-25T14:10:31.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.129+0000 7f14d5c00640 1 -- 172.21.15.86:0/3243138200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14d0069d00 con 0x7f14d0070330 2024-04-25T14:10:31.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.133+0000 7f14c6a00640 1 -- 172.21.15.86:0/3243138200 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 53) ==== 93045+0+0 (secure 0 0 0) 0x7f14c00c0020 con 0x7f14d0070330 2024-04-25T14:10:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.133+0000 7f14c6a00640 1 --2- 172.21.15.86:0/3243138200 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f14b006f280 0x7f14b0071790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.133+0000 7f14c6a00640 1 -- 172.21.15.86:0/3243138200 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6746+0+0 (secure 0 0 0) 0x7f14c012a030 con 0x7f14d0070330 2024-04-25T14:10:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.133+0000 7f14ce000640 1 --2- 172.21.15.86:0/3243138200 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f14b006f280 0x7f14b0071790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.133+0000 7f14ce000640 1 --2- 172.21.15.86:0/3243138200 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f14b006f280 0x7f14b0071790 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f14bc007fd0 tx=0x7f14bc007900 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:31.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.137+0000 7f14c6a00640 1 -- 172.21.15.86:0/3243138200 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f14c00c7030 con 0x7f14d0070330 2024-04-25T14:10:31.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.293+0000 7f14d5c00640 1 -- 172.21.15.86:0/3243138200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f14d00ae130 con 0x7f14d0070330 2024-04-25T14:10:31.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.293+0000 7f14c6a00640 1 -- 172.21.15.86:0/3243138200 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v115) ==== 104+0+11 (secure 0 0 0) 0x7f14c0016540 con 0x7f14d0070330 2024-04-25T14:10:31.291 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:10:31.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.297+0000 7f14d5c00640 1 -- 172.21.15.86:0/3243138200 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f14b006f280 msgr2=0x7f14b0071790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:31.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.297+0000 7f14d5c00640 1 --2- 172.21.15.86:0/3243138200 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f14b006f280 0x7f14b0071790 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f14bc007fd0 tx=0x7f14bc007900 comp rx=0 tx=0).stop 2024-04-25T14:10:31.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.297+0000 7f14d5c00640 1 -- 172.21.15.86:0/3243138200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14d0070330 msgr2=0x7f14d009ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:31.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.297+0000 7f14d5c00640 1 --2- 172.21.15.86:0/3243138200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14d0070330 0x7f14d009ee70 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f14c00194d0 tx=0x7f14c0019660 comp rx=0 tx=0).stop 2024-04-25T14:10:31.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.297+0000 7f14d5c00640 1 -- 172.21.15.86:0/3243138200 shutdown_connections 2024-04-25T14:10:31.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.297+0000 7f14d5c00640 1 --2- 172.21.15.86:0/3243138200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14d00d1c20 0x7f14d015a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:31.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.297+0000 7f14d5c00640 1 --2- 172.21.15.86:0/3243138200 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14d00ce790 0x7f14d00c3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:31.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.297+0000 7f14d5c00640 1 --2- 172.21.15.86:0/3243138200 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f14b006f280 0x7f14b0071790 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:31.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.297+0000 7f14d5c00640 1 --2- 172.21.15.86:0/3243138200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14d0070330 0x7f14d009ee70 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:31.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.297+0000 7f14d5c00640 1 -- 172.21.15.86:0/3243138200 >> 172.21.15.86:0/3243138200 conn(0x7f14d00b4650 msgr2=0x7f14d00ae3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:31.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.297+0000 7f14d5c00640 1 -- 172.21.15.86:0/3243138200 shutdown_connections 2024-04-25T14:10:31.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.297+0000 7f14d5c00640 1 -- 172.21.15.86:0/3243138200 wait complete. 2024-04-25T14:10:31.313 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:10:31.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.465+0000 7ff1bcc00640 1 Processor -- start 2024-04-25T14:10:31.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.465+0000 7ff1bcc00640 1 -- start start 2024-04-25T14:10:31.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.465+0000 7ff1bcc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1b80d05e0 0x7ff1b80b6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:31.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.465+0000 7ff1bcc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff1b80b6dc0 0x7ff1b80b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:31.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.465+0000 7ff1bcc00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1b80d2420 con 0x7ff1b80ce530 2024-04-25T14:10:31.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.465+0000 7ff1bcc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff1b80d2700 con 0x7ff1b80b6dc0 2024-04-25T14:10:31.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.465+0000 7ff1bcc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff1b80d29e0 con 0x7ff1b80d05e0 2024-04-25T14:10:31.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.465+0000 7ff1b6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff1b80b6dc0 0x7ff1b80b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:31.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.465+0000 7ff1b6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff1b80b6dc0 0x7ff1b80b71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57328/0 (socket says 172.21.15.86:57328) 2024-04-25T14:10:31.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.465+0000 7ff1b6800640 1 -- 172.21.15.86:0/3104434946 learned_addr learned my addr 172.21.15.86:0/3104434946 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:31.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.465+0000 7ff1b5400640 1 --2- 172.21.15.86:0/3104434946 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1b80d05e0 0x7ff1b80b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:31.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.469+0000 7ff1b6800640 1 -- 172.21.15.86:0/3104434946 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1b80d05e0 msgr2=0x7ff1b80b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:31.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.469+0000 7ff1b6800640 1 --2- 172.21.15.86:0/3104434946 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1b80d05e0 0x7ff1b80b6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:31.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.469+0000 7ff1b6800640 1 -- 172.21.15.86:0/3104434946 >> v1:172.21.15.155:6789/0 conn(0x7ff1b80ce530 legacy=0x7ff1b80cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:31.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.469+0000 7ff1b6800640 1 -- 172.21.15.86:0/3104434946 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b80d2cc0 con 0x7ff1b80b6dc0 2024-04-25T14:10:31.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.469+0000 7ff1b6800640 1 --2- 172.21.15.86:0/3104434946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1b80b6dc0 0x7ff1b80b71d0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0009870 tx=0x7ff1b001ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7b7c33ca40e4fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:31.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.469+0000 7ff1b4a00640 1 -- 172.21.15.86:0/3104434946 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b00b5070 con 0x7ff1b80b6dc0 2024-04-25T14:10:31.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.469+0000 7ff1b4a00640 1 -- 172.21.15.86:0/3104434946 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1b00a8a50 con 0x7ff1b80b6dc0 2024-04-25T14:10:31.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.469+0000 7ff1b4a00640 1 -- 172.21.15.86:0/3104434946 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b00a8e20 con 0x7ff1b80b6dc0 2024-04-25T14:10:31.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.473+0000 7ff1bcc00640 1 -- 172.21.15.86:0/3104434946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1b80b6dc0 msgr2=0x7ff1b80b71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:31.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.473+0000 7ff1bcc00640 1 --2- 172.21.15.86:0/3104434946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1b80b6dc0 0x7ff1b80b71d0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0009870 tx=0x7ff1b001ed50 comp rx=0 tx=0).stop 2024-04-25T14:10:31.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.473+0000 7ff1bcc00640 1 -- 172.21.15.86:0/3104434946 shutdown_connections 2024-04-25T14:10:31.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.473+0000 7ff1bcc00640 1 --2- 172.21.15.86:0/3104434946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1b80b6dc0 0x7ff1b80b71d0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:31.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.473+0000 7ff1bcc00640 1 --2- 172.21.15.86:0/3104434946 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1b80d05e0 0x7ff1b80b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:31.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.473+0000 7ff1bcc00640 1 -- 172.21.15.86:0/3104434946 >> 172.21.15.86:0/3104434946 conn(0x7ff1b80b5020 msgr2=0x7ff1b80b5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:31.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.473+0000 7ff1bcc00640 1 -- 172.21.15.86:0/3104434946 shutdown_connections 2024-04-25T14:10:31.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.473+0000 7ff1bcc00640 1 -- 172.21.15.86:0/3104434946 wait complete. 2024-04-25T14:10:31.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.473+0000 7ff1bcc00640 1 Processor -- start 2024-04-25T14:10:31.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.473+0000 7ff1bcc00640 1 -- start start 2024-04-25T14:10:31.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.473+0000 7ff1bcc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1b80b6dc0 0x7ff1b80afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:31.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.473+0000 7ff1bcc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1b80ce530 0x7ff1b80cb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:31.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.473+0000 7ff1bcc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1b8150c50 0x7ff1b8160830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:31.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.473+0000 7ff1bcc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b80ce940 con 0x7ff1b80b6dc0 2024-04-25T14:10:31.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.473+0000 7ff1bcc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b815df70 con 0x7ff1b80ce530 2024-04-25T14:10:31.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.473+0000 7ff1bcc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff1b80ceac0 con 0x7ff1b8150c50 2024-04-25T14:10:31.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.477+0000 7ff1b5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1b80b6dc0 0x7ff1b80afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:31.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.477+0000 7ff1b5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1b80b6dc0 0x7ff1b80afaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57340/0 (socket says 172.21.15.86:57340) 2024-04-25T14:10:31.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.477+0000 7ff1b5e00640 1 -- 172.21.15.86:0/203237716 learned_addr learned my addr 172.21.15.86:0/203237716 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:31.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.477+0000 7ff1b5400640 1 --2- 172.21.15.86:0/203237716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1b80ce530 0x7ff1b80cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:31.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.477+0000 7ff1b6800640 1 --2- 172.21.15.86:0/203237716 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1b8150c50 0x7ff1b8160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:31.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.477+0000 7ff1b5e00640 1 -- 172.21.15.86:0/203237716 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1b8150c50 msgr2=0x7ff1b8160830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:31.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.477+0000 7ff1b5e00640 1 --2- 172.21.15.86:0/203237716 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1b8150c50 0x7ff1b8160830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:31.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.477+0000 7ff1b5e00640 1 -- 172.21.15.86:0/203237716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1b80ce530 msgr2=0x7ff1b80cb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:31.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.477+0000 7ff1b5e00640 1 --2- 172.21.15.86:0/203237716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1b80ce530 0x7ff1b80cb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:31.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.477+0000 7ff1b5e00640 1 -- 172.21.15.86:0/203237716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b80affe0 con 0x7ff1b80b6dc0 2024-04-25T14:10:31.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.477+0000 7ff1b5e00640 1 --2- 172.21.15.86:0/203237716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1b80b6dc0 0x7ff1b80afaa0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7ff1a800c730 tx=0x7ff1a800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:31.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.477+0000 7ff1ae000640 1 -- 172.21.15.86:0/203237716 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8016020 con 0x7ff1b80b6dc0 2024-04-25T14:10:31.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.477+0000 7ff1ae000640 1 -- 172.21.15.86:0/203237716 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1a8006d60 con 0x7ff1b80b6dc0 2024-04-25T14:10:31.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.477+0000 7ff1ae000640 1 -- 172.21.15.86:0/203237716 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8003bc0 con 0x7ff1b80b6dc0 2024-04-25T14:10:31.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.477+0000 7ff1bcc00640 1 -- 172.21.15.86:0/203237716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b815f4c0 con 0x7ff1b80b6dc0 2024-04-25T14:10:31.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.477+0000 7ff1bcc00640 1 -- 172.21.15.86:0/203237716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b815fa80 con 0x7ff1b80b6dc0 2024-04-25T14:10:31.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.477+0000 7ff1bcc00640 1 -- 172.21.15.86:0/203237716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1b8072170 con 0x7ff1b80b6dc0 2024-04-25T14:10:31.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.477+0000 7ff1ae000640 1 -- 172.21.15.86:0/203237716 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 53) ==== 93045+0+0 (secure 0 0 0) 0x7ff1a800a850 con 0x7ff1b80b6dc0 2024-04-25T14:10:31.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.481+0000 7ff1ae000640 1 --2- 172.21.15.86:0/203237716 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7ff19806f250 0x7ff198071760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:31.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.481+0000 7ff1ae000640 1 -- 172.21.15.86:0/203237716 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6746+0+0 (secure 0 0 0) 0x7ff1a808ee70 con 0x7ff1b80b6dc0 2024-04-25T14:10:31.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.481+0000 7ff1b5400640 1 --2- 172.21.15.86:0/203237716 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7ff19806f250 0x7ff198071760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:31.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.481+0000 7ff1b5400640 1 --2- 172.21.15.86:0/203237716 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7ff19806f250 0x7ff198071760 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80b0430 tx=0x7ff1a4009190 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:31.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.485+0000 7ff1ae000640 1 -- 172.21.15.86:0/203237716 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff1a8020020 con 0x7ff1b80b6dc0 2024-04-25T14:10:31.632 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:31.633+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:31.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.645+0000 7ff1bcc00640 1 -- 172.21.15.86:0/203237716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff1b815fcc0 con 0x7ff1b80b6dc0 2024-04-25T14:10:31.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.645+0000 7ff1ae000640 1 -- 172.21.15.86:0/203237716 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v115) ==== 100+0+11 (secure 0 0 0) 0x7ff1a805bd20 con 0x7ff1b80b6dc0 2024-04-25T14:10:31.642 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:10:31.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.645+0000 7ff1bcc00640 1 -- 172.21.15.86:0/203237716 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7ff19806f250 msgr2=0x7ff198071760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:31.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.645+0000 7ff1bcc00640 1 --2- 172.21.15.86:0/203237716 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7ff19806f250 0x7ff198071760 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80b0430 tx=0x7ff1a4009190 comp rx=0 tx=0).stop 2024-04-25T14:10:31.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.645+0000 7ff1bcc00640 1 -- 172.21.15.86:0/203237716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1b80b6dc0 msgr2=0x7ff1b80afaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:31.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.645+0000 7ff1bcc00640 1 --2- 172.21.15.86:0/203237716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1b80b6dc0 0x7ff1b80afaa0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7ff1a800c730 tx=0x7ff1a800cbf0 comp rx=0 tx=0).stop 2024-04-25T14:10:31.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.645+0000 7ff1bcc00640 1 -- 172.21.15.86:0/203237716 shutdown_connections 2024-04-25T14:10:31.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.645+0000 7ff1bcc00640 1 --2- 172.21.15.86:0/203237716 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1b8150c50 0x7ff1b8160830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:31.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.645+0000 7ff1bcc00640 1 --2- 172.21.15.86:0/203237716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1b80ce530 0x7ff1b80cb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:31.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.645+0000 7ff1bcc00640 1 --2- 172.21.15.86:0/203237716 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7ff19806f250 0x7ff198071760 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:31.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.645+0000 7ff1bcc00640 1 --2- 172.21.15.86:0/203237716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1b80b6dc0 0x7ff1b80afaa0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:31.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.645+0000 7ff1bcc00640 1 -- 172.21.15.86:0/203237716 >> 172.21.15.86:0/203237716 conn(0x7ff1b80b5020 msgr2=0x7ff1b80b5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:31.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.645+0000 7ff1bcc00640 1 -- 172.21.15.86:0/203237716 shutdown_connections 2024-04-25T14:10:31.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.645+0000 7ff1bcc00640 1 -- 172.21.15.86:0/203237716 wait complete. 2024-04-25T14:10:31.665 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:10:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.821+0000 7fe290a00640 1 Processor -- start 2024-04-25T14:10:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.821+0000 7fe290a00640 1 -- start start 2024-04-25T14:10:31.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.825+0000 7fe290a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe28c0b79c0 0x7fe28c0ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:31.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.825+0000 7fe290a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe28c0b3bd0 con 0x7fe28c0ccab0 2024-04-25T14:10:31.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.825+0000 7fe290a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe28c0b1cf0 con 0x7fe28c0d2ae0 2024-04-25T14:10:31.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.825+0000 7fe290a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe28c0b2620 con 0x7fe28c0b79c0 2024-04-25T14:10:31.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.825+0000 7fe289e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe28c0ccab0 0x7fe28c0d23d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39896/0 (socket says 172.21.15.86:39896) 2024-04-25T14:10:31.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.825+0000 7fe289e00640 1 -- 172.21.15.86:0/4255059063 learned_addr learned my addr 172.21.15.86:0/4255059063 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:31.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.825+0000 7fe28a800640 1 --2- 172.21.15.86:0/4255059063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe28c0b79c0 0x7fe28c0ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:31.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.825+0000 7fe288a00640 1 -- 172.21.15.86:0/4255059063 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1844760468 0 0) 0x7fe28c0b3bd0 con 0x7fe28c0ccab0 2024-04-25T14:10:31.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.825+0000 7fe288a00640 1 -- 172.21.15.86:0/4255059063 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe270003530 con 0x7fe28c0ccab0 2024-04-25T14:10:31.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.825+0000 7fe288a00640 1 -- 172.21.15.86:0/4255059063 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3801520358 0 0) 0x7fe28c0b1cf0 con 0x7fe28c0d2ae0 2024-04-25T14:10:31.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.825+0000 7fe288a00640 1 -- 172.21.15.86:0/4255059063 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe28c0b3bd0 con 0x7fe28c0d2ae0 2024-04-25T14:10:31.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.825+0000 7fe28a800640 1 -- 172.21.15.86:0/4255059063 >> v1:172.21.15.155:6789/0 conn(0x7fe28c0d2ae0 legacy=0x7fe28c0ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:31.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.825+0000 7fe28a800640 1 -- 172.21.15.86:0/4255059063 >> v1:172.21.15.86:6789/0 conn(0x7fe28c0ccab0 legacy=0x7fe28c0d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:31.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.825+0000 7fe28a800640 1 -- 172.21.15.86:0/4255059063 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe28c0b1fd0 con 0x7fe28c0b79c0 2024-04-25T14:10:31.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.829+0000 7fe28a800640 1 --2- 172.21.15.86:0/4255059063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe28c0b79c0 0x7fe28c0ce890 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe284009dc0 tx=0x7fe28401c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0031edb39b46229 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:31.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.829+0000 7fe288a00640 1 -- 172.21.15.86:0/4255059063 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2840b2070 con 0x7fe28c0b79c0 2024-04-25T14:10:31.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.829+0000 7fe288a00640 1 -- 172.21.15.86:0/4255059063 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2840a5960 con 0x7fe28c0b79c0 2024-04-25T14:10:31.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.829+0000 7fe288a00640 1 -- 172.21.15.86:0/4255059063 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2840a5d30 con 0x7fe28c0b79c0 2024-04-25T14:10:31.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.829+0000 7fe290a00640 1 -- 172.21.15.86:0/4255059063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe28c0b79c0 msgr2=0x7fe28c0ce890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:31.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.829+0000 7fe290a00640 1 --2- 172.21.15.86:0/4255059063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe28c0b79c0 0x7fe28c0ce890 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe284009dc0 tx=0x7fe28401c990 comp rx=0 tx=0).stop 2024-04-25T14:10:31.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.829+0000 7fe290a00640 1 -- 172.21.15.86:0/4255059063 shutdown_connections 2024-04-25T14:10:31.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.829+0000 7fe290a00640 1 --2- 172.21.15.86:0/4255059063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe28c0b79c0 0x7fe28c0ce890 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:31.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.829+0000 7fe290a00640 1 -- 172.21.15.86:0/4255059063 >> 172.21.15.86:0/4255059063 conn(0x7fe28c09ba30 msgr2=0x7fe28c0b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:31.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.829+0000 7fe290a00640 1 -- 172.21.15.86:0/4255059063 shutdown_connections 2024-04-25T14:10:31.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.829+0000 7fe290a00640 1 -- 172.21.15.86:0/4255059063 wait complete. 2024-04-25T14:10:31.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.833+0000 7fe290a00640 1 Processor -- start 2024-04-25T14:10:31.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.833+0000 7fe290a00640 1 -- start start 2024-04-25T14:10:31.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.833+0000 7fe290a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe28c0b79c0 0x7fe28c15c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:31.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.833+0000 7fe290a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe28c0ccab0 0x7fe28c14fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:31.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.833+0000 7fe290a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe28c06c950 0x7fe28c154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:31.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.833+0000 7fe290a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c16c530 con 0x7fe28c06c950 2024-04-25T14:10:31.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.833+0000 7fe290a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c16c3b0 con 0x7fe28c0b79c0 2024-04-25T14:10:31.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.833+0000 7fe290a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe28c16c6b0 con 0x7fe28c0ccab0 2024-04-25T14:10:31.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.833+0000 7fe28a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe28c06c950 0x7fe28c154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:31.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.833+0000 7fe28a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe28c06c950 0x7fe28c154b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57342/0 (socket says 172.21.15.86:57342) 2024-04-25T14:10:31.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.833+0000 7fe28a800640 1 -- 172.21.15.86:0/2603112272 learned_addr learned my addr 172.21.15.86:0/2603112272 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:31.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.833+0000 7fe289e00640 1 --2- 172.21.15.86:0/2603112272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe28c0b79c0 0x7fe28c15c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:31.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.833+0000 7fe289400640 1 --2- 172.21.15.86:0/2603112272 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe28c0ccab0 0x7fe28c14fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:31.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.833+0000 7fe28a800640 1 -- 172.21.15.86:0/2603112272 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe28c0ccab0 msgr2=0x7fe28c14fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:31.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.833+0000 7fe28a800640 1 --2- 172.21.15.86:0/2603112272 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe28c0ccab0 0x7fe28c14fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:31.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.833+0000 7fe28a800640 1 -- 172.21.15.86:0/2603112272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe28c0b79c0 msgr2=0x7fe28c15c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:31.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.833+0000 7fe28a800640 1 --2- 172.21.15.86:0/2603112272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe28c0b79c0 0x7fe28c15c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:31.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.833+0000 7fe28a800640 1 -- 172.21.15.86:0/2603112272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe28c06cef0 con 0x7fe28c06c950 2024-04-25T14:10:31.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.833+0000 7fe28a800640 1 --2- 172.21.15.86:0/2603112272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe28c06c950 0x7fe28c154b10 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fe28401a5c0 tx=0x7fe284019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:31.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.833+0000 7fe282000640 1 -- 172.21.15.86:0/2603112272 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2840b2050 con 0x7fe28c06c950 2024-04-25T14:10:31.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.833+0000 7fe282000640 1 -- 172.21.15.86:0/2603112272 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2840be070 con 0x7fe28c06c950 2024-04-25T14:10:31.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.833+0000 7fe282000640 1 -- 172.21.15.86:0/2603112272 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2840ad870 con 0x7fe28c06c950 2024-04-25T14:10:31.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.833+0000 7fe290a00640 1 -- 172.21.15.86:0/2603112272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe28c06d1d0 con 0x7fe28c06c950 2024-04-25T14:10:31.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.833+0000 7fe290a00640 1 -- 172.21.15.86:0/2603112272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe28c06d790 con 0x7fe28c06c950 2024-04-25T14:10:31.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.837+0000 7fe290a00640 1 -- 172.21.15.86:0/2603112272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe28c0d2f90 con 0x7fe28c06c950 2024-04-25T14:10:31.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.837+0000 7fe282000640 1 -- 172.21.15.86:0/2603112272 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 53) ==== 93045+0+0 (secure 0 0 0) 0x7fe2840ada10 con 0x7fe28c06c950 2024-04-25T14:10:31.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.837+0000 7fe282000640 1 --2- 172.21.15.86:0/2603112272 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fe26c06f250 0x7fe26c071760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:31.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.837+0000 7fe282000640 1 -- 172.21.15.86:0/2603112272 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6746+0+0 (secure 0 0 0) 0x7fe28412a2d0 con 0x7fe28c06c950 2024-04-25T14:10:31.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.837+0000 7fe289e00640 1 --2- 172.21.15.86:0/2603112272 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fe26c06f250 0x7fe26c071760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:31.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.837+0000 7fe289e00640 1 --2- 172.21.15.86:0/2603112272 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fe26c06f250 0x7fe26c071760 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe27c007fd0 tx=0x7fe27c009040 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:31.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:31.841+0000 7fe282000640 1 -- 172.21.15.86:0/2603112272 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe284016580 con 0x7fe28c06c950 2024-04-25T14:10:32.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.049+0000 7fe290a00640 1 -- 172.21.15.86:0/2603112272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe28c156cb0 con 0x7fe28c06c950 2024-04-25T14:10:32.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.049+0000 7fe282000640 1 -- 172.21.15.86:0/2603112272 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5183 (secure 0 0 0) 0x7fe2840f70b0 con 0x7fe28c06c950 2024-04-25T14:10:32.046 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:32.046 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":85,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/374465657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":374465657},{"type":"v1","addr":"172.21.15.86:6836","nonce":374465657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8107,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1466195534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":1466195534},{"type":"v1","addr":"172.21.15.86:6837","nonce":1466195534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8119,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2431580070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2431580070},{"type":"v1","addr":"172.21.15.155:6835","nonce":2431580070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:10:05.129470+0000","modified":"2024-04-25T14:10:08.145259+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8457":{"gid":8457,"name":"b","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6833/142240252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":142240252},{"type":"v1","addr":"172.21.15.155:6833","nonce":142240252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8457,"qdb_cluster":[8457]},"id":8}]} 2024-04-25T14:10:32.047 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 85 2024-04-25T14:10:32.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.049+0000 7fe290a00640 1 -- 172.21.15.86:0/2603112272 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fe26c06f250 msgr2=0x7fe26c071760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:32.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.049+0000 7fe290a00640 1 --2- 172.21.15.86:0/2603112272 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fe26c06f250 0x7fe26c071760 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe27c007fd0 tx=0x7fe27c009040 comp rx=0 tx=0).stop 2024-04-25T14:10:32.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.049+0000 7fe290a00640 1 -- 172.21.15.86:0/2603112272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe28c06c950 msgr2=0x7fe28c154b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:32.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.049+0000 7fe290a00640 1 --2- 172.21.15.86:0/2603112272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe28c06c950 0x7fe28c154b10 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fe28401a5c0 tx=0x7fe284019730 comp rx=0 tx=0).stop 2024-04-25T14:10:32.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.049+0000 7fe290a00640 1 -- 172.21.15.86:0/2603112272 shutdown_connections 2024-04-25T14:10:32.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.049+0000 7fe290a00640 1 --2- 172.21.15.86:0/2603112272 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fe26c06f250 0x7fe26c071760 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.049+0000 7fe290a00640 1 --2- 172.21.15.86:0/2603112272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe28c06c950 0x7fe28c154b10 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.049+0000 7fe290a00640 1 --2- 172.21.15.86:0/2603112272 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe28c0ccab0 0x7fe28c14fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.049+0000 7fe290a00640 1 --2- 172.21.15.86:0/2603112272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe28c0b79c0 0x7fe28c15c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.049+0000 7fe290a00640 1 -- 172.21.15.86:0/2603112272 >> 172.21.15.86:0/2603112272 conn(0x7fe28c09ba30 msgr2=0x7fe28c068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:32.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.049+0000 7fe290a00640 1 -- 172.21.15.86:0/2603112272 shutdown_connections 2024-04-25T14:10:32.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.049+0000 7fe290a00640 1 -- 172.21.15.86:0/2603112272 wait complete. 2024-04-25T14:10:32.069 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:10:32.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.225+0000 7fb015600640 1 Processor -- start 2024-04-25T14:10:32.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.225+0000 7fb015600640 1 -- start start 2024-04-25T14:10:32.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.225+0000 7fb015600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb010071530 0x7fb01015dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:32.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.225+0000 7fb015600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb0101616e0 0x7fb010161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:32.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.225+0000 7fb015600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0100cd8d0 con 0x7fb0100d2c80 2024-04-25T14:10:32.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.225+0000 7fb015600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb0100ce720 con 0x7fb010071530 2024-04-25T14:10:32.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.225+0000 7fb015600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb0100ce970 con 0x7fb0101616e0 2024-04-25T14:10:32.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.225+0000 7fb007e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb010071530 0x7fb01015dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:32.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.225+0000 7fb007e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb010071530 0x7fb01015dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57356/0 (socket says 172.21.15.86:57356) 2024-04-25T14:10:32.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.225+0000 7fb007e00640 1 -- 172.21.15.86:0/2928129846 learned_addr learned my addr 172.21.15.86:0/2928129846 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:32.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.225+0000 7fb00f400640 1 --2- 172.21.15.86:0/2928129846 >> v2:172.21.15.155:3300/0 conn(0x7fb0101616e0 0x7fb010161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:32.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.225+0000 7fb007e00640 1 -- 172.21.15.86:0/2928129846 >> v2:172.21.15.155:3300/0 conn(0x7fb0101616e0 msgr2=0x7fb010161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:32.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.225+0000 7fb007e00640 1 --2- 172.21.15.86:0/2928129846 >> v2:172.21.15.155:3300/0 conn(0x7fb0101616e0 0x7fb010161af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.225+0000 7fb007e00640 1 -- 172.21.15.86:0/2928129846 >> v1:172.21.15.155:6789/0 conn(0x7fb0100d2c80 legacy=0x7fb010070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:32.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.225+0000 7fb007e00640 1 -- 172.21.15.86:0/2928129846 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0100d1be0 con 0x7fb010071530 2024-04-25T14:10:32.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.229+0000 7fb007e00640 1 --2- 172.21.15.86:0/2928129846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb010071530 0x7fb01015dc90 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fb010074af0 tx=0x7faff801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3c690f5d3cc067 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:32.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.229+0000 7fb00e000640 1 -- 172.21.15.86:0/2928129846 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff80b5070 con 0x7fb010071530 2024-04-25T14:10:32.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.229+0000 7fb00e000640 1 -- 172.21.15.86:0/2928129846 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faff80a8a10 con 0x7fb010071530 2024-04-25T14:10:32.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.229+0000 7fb00e000640 1 -- 172.21.15.86:0/2928129846 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff80a8de0 con 0x7fb010071530 2024-04-25T14:10:32.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.229+0000 7fb015600640 1 -- 172.21.15.86:0/2928129846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb010071530 msgr2=0x7fb01015dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:32.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.229+0000 7fb015600640 1 --2- 172.21.15.86:0/2928129846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb010071530 0x7fb01015dc90 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fb010074af0 tx=0x7faff801ed10 comp rx=0 tx=0).stop 2024-04-25T14:10:32.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.229+0000 7fb015600640 1 -- 172.21.15.86:0/2928129846 shutdown_connections 2024-04-25T14:10:32.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.229+0000 7fb015600640 1 --2- 172.21.15.86:0/2928129846 >> v2:172.21.15.155:3300/0 conn(0x7fb0101616e0 0x7fb010161af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.229+0000 7fb015600640 1 --2- 172.21.15.86:0/2928129846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb010071530 0x7fb01015dc90 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.229+0000 7fb015600640 1 -- 172.21.15.86:0/2928129846 >> 172.21.15.86:0/2928129846 conn(0x7fb0100cffc0 msgr2=0x7fb0100d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:32.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.229+0000 7fb015600640 1 -- 172.21.15.86:0/2928129846 shutdown_connections 2024-04-25T14:10:32.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.229+0000 7fb015600640 1 -- 172.21.15.86:0/2928129846 wait complete. 2024-04-25T14:10:32.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.233+0000 7fb015600640 1 Processor -- start 2024-04-25T14:10:32.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.233+0000 7fb015600640 1 -- start start 2024-04-25T14:10:32.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.233+0000 7fb015600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0100d2c80 0x7fb01014e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:32.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.233+0000 7fb015600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0101616e0 0x7fb0100c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:32.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.233+0000 7fb015600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0100aeaa0 0x7fb0101f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:32.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.233+0000 7fb015600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0100d2480 con 0x7fb0100d2c80 2024-04-25T14:10:32.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.233+0000 7fb015600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb0100d21a0 con 0x7fb0101616e0 2024-04-25T14:10:32.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.233+0000 7fb015600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb0100d29a0 con 0x7fb0100aeaa0 2024-04-25T14:10:32.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.233+0000 7fb00ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0100d2c80 0x7fb01014e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:32.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.233+0000 7fb00ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0100d2c80 0x7fb01014e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57372/0 (socket says 172.21.15.86:57372) 2024-04-25T14:10:32.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.233+0000 7fb00ea00640 1 -- 172.21.15.86:0/3063354874 learned_addr learned my addr 172.21.15.86:0/3063354874 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:32.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.233+0000 7fb00f400640 1 --2- 172.21.15.86:0/3063354874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0100aeaa0 0x7fb0101f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:32.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.233+0000 7fb007e00640 1 --2- 172.21.15.86:0/3063354874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0101616e0 0x7fb0100c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:32.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.237+0000 7fb00ea00640 1 -- 172.21.15.86:0/3063354874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0100aeaa0 msgr2=0x7fb0101f63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:32.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.237+0000 7fb00ea00640 1 --2- 172.21.15.86:0/3063354874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0100aeaa0 0x7fb0101f63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.237+0000 7fb00ea00640 1 -- 172.21.15.86:0/3063354874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0101616e0 msgr2=0x7fb0100c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:32.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.237+0000 7fb00ea00640 1 --2- 172.21.15.86:0/3063354874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0101616e0 0x7fb0100c8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.237+0000 7fb00ea00640 1 -- 172.21.15.86:0/3063354874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01014ec40 con 0x7fb0100d2c80 2024-04-25T14:10:32.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.237+0000 7fb00ea00640 1 --2- 172.21.15.86:0/3063354874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0100d2c80 0x7fb01014e340 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fb00000c370 tx=0x7fb00000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:32.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.237+0000 7fb006a00640 1 -- 172.21.15.86:0/3063354874 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0000052d0 con 0x7fb0100d2c80 2024-04-25T14:10:32.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.237+0000 7fb006a00640 1 -- 172.21.15.86:0/3063354874 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb000005470 con 0x7fb0100d2c80 2024-04-25T14:10:32.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.237+0000 7fb015600640 1 -- 172.21.15.86:0/3063354874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb01014ef20 con 0x7fb0100d2c80 2024-04-25T14:10:32.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.237+0000 7fb006a00640 1 -- 172.21.15.86:0/3063354874 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000007260 con 0x7fb0100d2c80 2024-04-25T14:10:32.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.237+0000 7fb015600640 1 -- 172.21.15.86:0/3063354874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0100b3290 con 0x7fb0100d2c80 2024-04-25T14:10:32.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.237+0000 7fb015600640 1 -- 172.21.15.86:0/3063354874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0100720e0 con 0x7fb0100d2c80 2024-04-25T14:10:32.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.237+0000 7fb006a00640 1 -- 172.21.15.86:0/3063354874 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 53) ==== 93045+0+0 (secure 0 0 0) 0x7fb000007400 con 0x7fb0100d2c80 2024-04-25T14:10:32.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.241+0000 7fb006a00640 1 --2- 172.21.15.86:0/3063354874 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fafec06f250 0x7fafec071760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:32.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.241+0000 7fb006a00640 1 -- 172.21.15.86:0/3063354874 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6746+0+0 (secure 0 0 0) 0x7fb00008e650 con 0x7fb0100d2c80 2024-04-25T14:10:32.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.241+0000 7fb007e00640 1 --2- 172.21.15.86:0/3063354874 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fafec06f250 0x7fafec071760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:32.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.241+0000 7fb007e00640 1 --2- 172.21.15.86:0/3063354874 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fafec06f250 0x7fafec071760 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7faff80b2000 tx=0x7faff80aa070 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:32.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.245+0000 7fb006a00640 1 -- 172.21.15.86:0/3063354874 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb00005aeb0 con 0x7fb0100d2c80 2024-04-25T14:10:32.319 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:32.321+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:32.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.409+0000 7fb015600640 1 -- 172.21.15.86:0/3063354874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb010076bf0 con 0x7fb0100d2c80 2024-04-25T14:10:32.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.409+0000 7fb006a00640 1 -- 172.21.15.86:0/3063354874 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+17889 (secure 0 0 0) 0x7fb00005f1b0 con 0x7fb0100d2c80 2024-04-25T14:10:32.407 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:32.407 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":115,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:10:29.591997+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:10:03.520649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T14:10:04.080912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1062432465":"2024-04-26T14:10:29.591964+0000","172.21.15.86:6832/190090395":"2024-04-26T14:10:29.591964+0000","172.21.15.86:0/1120094610":"2024-04-26T14:10:29.591964+0000","172.21.15.86:0/2412030790":"2024-04-26T14:10:29.591964+0000","172.21.15.86:0/4291479632":"2024-04-26T14:10:29.591964+0000","172.21.15.86:0/2052329939":"2024-04-26T14:10:23.454991+0000","172.21.15.86:0/1021944523":"2024-04-26T14:10:23.454991+0000","172.21.15.86:0/83615604":"2024-04-26T14:10:23.454991+0000","172.21.15.86:0/3719613294":"2024-04-26T14:10:23.454991+0000","172.21.15.86:0/3734007180":"2024-04-26T14:10:29.591964+0000","172.21.15.86:0/1471025278":"2024-04-26T14:10:23.454991+0000","172.21.15.86:6833/1354467416":"2024-04-26T14:10:23.454991+0000","172.21.15.86:6832/1354467416":"2024-04-26T14:10:23.454991+0000","172.21.15.86:6833/190090395":"2024-04-26T14:10:29.591964+0000","172.21.15.86:0/3000491277":"2024-04-26T14:10:23.454991+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:10:32.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.413+0000 7fb015600640 1 -- 172.21.15.86:0/3063354874 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fafec06f250 msgr2=0x7fafec071760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:32.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.413+0000 7fb015600640 1 --2- 172.21.15.86:0/3063354874 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fafec06f250 0x7fafec071760 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7faff80b2000 tx=0x7faff80aa070 comp rx=0 tx=0).stop 2024-04-25T14:10:32.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.413+0000 7fb015600640 1 -- 172.21.15.86:0/3063354874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0100d2c80 msgr2=0x7fb01014e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:32.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.413+0000 7fb015600640 1 --2- 172.21.15.86:0/3063354874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0100d2c80 0x7fb01014e340 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fb00000c370 tx=0x7fb00000c830 comp rx=0 tx=0).stop 2024-04-25T14:10:32.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.413+0000 7fb015600640 1 -- 172.21.15.86:0/3063354874 shutdown_connections 2024-04-25T14:10:32.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.413+0000 7fb015600640 1 --2- 172.21.15.86:0/3063354874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0100aeaa0 0x7fb0101f63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.413+0000 7fb015600640 1 --2- 172.21.15.86:0/3063354874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0101616e0 0x7fb0100c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.413+0000 7fb015600640 1 --2- 172.21.15.86:0/3063354874 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fafec06f250 0x7fafec071760 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.413+0000 7fb015600640 1 --2- 172.21.15.86:0/3063354874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0100d2c80 0x7fb01014e340 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.413+0000 7fb015600640 1 -- 172.21.15.86:0/3063354874 >> 172.21.15.86:0/3063354874 conn(0x7fb0100cffc0 msgr2=0x7fb0100c9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:32.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.413+0000 7fb015600640 1 -- 172.21.15.86:0/3063354874 shutdown_connections 2024-04-25T14:10:32.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.413+0000 7fb015600640 1 -- 172.21.15.86:0/3063354874 wait complete. 2024-04-25T14:10:32.430 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T14:10:32.431 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:10:32.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.589+0000 7fd5adc00640 1 Processor -- start 2024-04-25T14:10:32.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.589+0000 7fd5adc00640 1 -- start start 2024-04-25T14:10:32.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.593+0000 7fd5adc00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd5a80d16e0 0x7fd5a80d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:32.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.593+0000 7fd5adc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5a80730b0 0x7fd5a80734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:32.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.593+0000 7fd5adc00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5a80d2030 con 0x7fd5a80cfca0 2024-04-25T14:10:32.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.593+0000 7fd5adc00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd5a8070470 con 0x7fd5a80d16e0 2024-04-25T14:10:32.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.593+0000 7fd5adc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd5a8072bd0 con 0x7fd5a80730b0 2024-04-25T14:10:32.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.593+0000 7fd5a7400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5a80730b0 0x7fd5a80734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:32.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.593+0000 7fd5a6a00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fd5a80cfca0 0x7fd5a806f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:52884/0 (socket says 172.21.15.86:52884) 2024-04-25T14:10:32.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.593+0000 7fd5a6000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd5a80d16e0 0x7fd5a80d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:32.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.593+0000 7fd5a6a00640 1 -- 172.21.15.86:0/2622888658 learned_addr learned my addr 172.21.15.86:0/2622888658 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:32.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.593+0000 7fd5a5600640 1 -- 172.21.15.86:0/2622888658 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2284021920 0 0) 0x7fd598001120 con 0x7fd5a80cfca0 2024-04-25T14:10:32.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.593+0000 7fd5a5600640 1 -- 172.21.15.86:0/2622888658 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd58c003530 con 0x7fd5a80cfca0 2024-04-25T14:10:32.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.593+0000 7fd5a7400640 1 -- 172.21.15.86:0/2622888658 >> v2:172.21.15.155:3300/0 conn(0x7fd5a80d16e0 msgr2=0x7fd5a80d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:32.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.593+0000 7fd5a7400640 1 --2- 172.21.15.86:0/2622888658 >> v2:172.21.15.155:3300/0 conn(0x7fd5a80d16e0 0x7fd5a80d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.593+0000 7fd5a7400640 1 -- 172.21.15.86:0/2622888658 >> v1:172.21.15.155:6789/0 conn(0x7fd5a80cfca0 legacy=0x7fd5a806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:32.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.593+0000 7fd5a7400640 1 -- 172.21.15.86:0/2622888658 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a806fbd0 con 0x7fd5a80730b0 2024-04-25T14:10:32.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.597+0000 7fd5a7400640 1 --2- 172.21.15.86:0/2622888658 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5a80730b0 0x7fd5a80734c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009d90 tx=0x7fd5a001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4abfb24bb3e54ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:32.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.597+0000 7fd5a5600640 1 -- 172.21.15.86:0/2622888658 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a00b2070 con 0x7fd5a80730b0 2024-04-25T14:10:32.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.597+0000 7fd5a5600640 1 -- 172.21.15.86:0/2622888658 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5a00a5940 con 0x7fd5a80730b0 2024-04-25T14:10:32.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.597+0000 7fd5a5600640 1 -- 172.21.15.86:0/2622888658 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a00a5cf0 con 0x7fd5a80730b0 2024-04-25T14:10:32.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.597+0000 7fd5adc00640 1 -- 172.21.15.86:0/2622888658 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5a80730b0 msgr2=0x7fd5a80734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:32.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.597+0000 7fd5adc00640 1 --2- 172.21.15.86:0/2622888658 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5a80730b0 0x7fd5a80734c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009d90 tx=0x7fd5a001c970 comp rx=0 tx=0).stop 2024-04-25T14:10:32.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.597+0000 7fd5adc00640 1 -- 172.21.15.86:0/2622888658 shutdown_connections 2024-04-25T14:10:32.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.597+0000 7fd5adc00640 1 --2- 172.21.15.86:0/2622888658 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5a80730b0 0x7fd5a80734c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.597+0000 7fd5adc00640 1 --2- 172.21.15.86:0/2622888658 >> v2:172.21.15.155:3300/0 conn(0x7fd5a80d16e0 0x7fd5a80d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.597+0000 7fd5adc00640 1 -- 172.21.15.86:0/2622888658 >> 172.21.15.86:0/2622888658 conn(0x7fd5a80b4650 msgr2=0x7fd5a80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:32.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.597+0000 7fd5adc00640 1 -- 172.21.15.86:0/2622888658 shutdown_connections 2024-04-25T14:10:32.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.597+0000 7fd5adc00640 1 -- 172.21.15.86:0/2622888658 wait complete. 2024-04-25T14:10:32.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.601+0000 7fd5adc00640 1 Processor -- start 2024-04-25T14:10:32.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.601+0000 7fd5adc00640 1 -- start start 2024-04-25T14:10:32.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.601+0000 7fd5adc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5a80730b0 0x7fd5a815b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:32.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.601+0000 7fd5adc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5a80cfca0 0x7fd5a80c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:32.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.601+0000 7fd5adc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5a80d16e0 0x7fd5a80c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:32.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.601+0000 7fd5adc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a8167aa0 con 0x7fd5a80d16e0 2024-04-25T14:10:32.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.601+0000 7fd5adc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a8167920 con 0x7fd5a80cfca0 2024-04-25T14:10:32.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.601+0000 7fd5adc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd5a8167c20 con 0x7fd5a80730b0 2024-04-25T14:10:32.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.601+0000 7fd5a7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5a80d16e0 0x7fd5a80c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:32.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.601+0000 7fd5a7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5a80d16e0 0x7fd5a80c98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57374/0 (socket says 172.21.15.86:57374) 2024-04-25T14:10:32.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.601+0000 7fd5a7400640 1 -- 172.21.15.86:0/1940531942 learned_addr learned my addr 172.21.15.86:0/1940531942 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:32.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.601+0000 7fd5a6a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5a80730b0 0x7fd5a815b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:32.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.601+0000 7fd5a6000640 1 --2- 172.21.15.86:0/1940531942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5a80cfca0 0x7fd5a80c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:32.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.601+0000 7fd5a7400640 1 -- 172.21.15.86:0/1940531942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5a80730b0 msgr2=0x7fd5a815b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:32.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.601+0000 7fd5a7400640 1 --2- 172.21.15.86:0/1940531942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5a80730b0 0x7fd5a815b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.601+0000 7fd5a7400640 1 -- 172.21.15.86:0/1940531942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5a80cfca0 msgr2=0x7fd5a80c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:32.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.601+0000 7fd5a7400640 1 --2- 172.21.15.86:0/1940531942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5a80cfca0 0x7fd5a80c93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.601+0000 7fd5a7400640 1 -- 172.21.15.86:0/1940531942 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a8097d50 con 0x7fd5a80d16e0 2024-04-25T14:10:32.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.601+0000 7fd5a7400640 1 --2- 172.21.15.86:0/1940531942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5a80d16e0 0x7fd5a80c98f0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fd5a001b020 tx=0x7fd5a0019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:32.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.601+0000 7fd59ea00640 1 -- 172.21.15.86:0/1940531942 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a00b2050 con 0x7fd5a80d16e0 2024-04-25T14:10:32.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.601+0000 7fd59ea00640 1 -- 172.21.15.86:0/1940531942 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5a00be070 con 0x7fd5a80d16e0 2024-04-25T14:10:32.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.601+0000 7fd59ea00640 1 -- 172.21.15.86:0/1940531942 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a00ad850 con 0x7fd5a80d16e0 2024-04-25T14:10:32.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.601+0000 7fd5adc00640 1 -- 172.21.15.86:0/1940531942 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a8097ff0 con 0x7fd5a80d16e0 2024-04-25T14:10:32.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.601+0000 7fd5adc00640 1 -- 172.21.15.86:0/1940531942 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a80ca130 con 0x7fd5a80d16e0 2024-04-25T14:10:32.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.605+0000 7fd5adc00640 1 -- 172.21.15.86:0/1940531942 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5a809f4b0 con 0x7fd5a80d16e0 2024-04-25T14:10:32.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.609+0000 7fd59ea00640 1 -- 172.21.15.86:0/1940531942 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7fd5a00ad9f0 con 0x7fd5a80d16e0 2024-04-25T14:10:32.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.609+0000 7fd59ea00640 1 --2- 172.21.15.86:0/1940531942 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fd58806f4c0 0x7fd5880719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:32.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.609+0000 7fd59ea00640 1 -- 172.21.15.86:0/1940531942 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6746+0+0 (secure 0 0 0) 0x7fd5a012b090 con 0x7fd5a80d16e0 2024-04-25T14:10:32.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.609+0000 7fd5a6a00640 1 --2- 172.21.15.86:0/1940531942 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fd58806f4c0 0x7fd5880719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:32.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.609+0000 7fd5a6a00640 1 --2- 172.21.15.86:0/1940531942 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fd58806f4c0 0x7fd5880719d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd598005890 tx=0x7fd598009040 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:32.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.613+0000 7fd59ea00640 1 -- 172.21.15.86:0/1940531942 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd5a0016560 con 0x7fd5a80d16e0 2024-04-25T14:10:32.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.809+0000 7fd5adc00640 1 -- 172.21.15.86:0/1940531942 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5a80c9ec0 con 0x7fd5a80d16e0 2024-04-25T14:10:32.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.809+0000 7fd59ea00640 1 -- 172.21.15.86:0/1940531942 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5183 (secure 0 0 0) 0x7fd5a00f7540 con 0x7fd5a80d16e0 2024-04-25T14:10:32.808 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:32.808 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":85,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/374465657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":374465657},{"type":"v1","addr":"172.21.15.86:6836","nonce":374465657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8107,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1466195534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":1466195534},{"type":"v1","addr":"172.21.15.86:6837","nonce":1466195534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8119,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2431580070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2431580070},{"type":"v1","addr":"172.21.15.155:6835","nonce":2431580070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:10:05.129470+0000","modified":"2024-04-25T14:10:08.145259+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8457":{"gid":8457,"name":"b","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6833/142240252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":142240252},{"type":"v1","addr":"172.21.15.155:6833","nonce":142240252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8457,"qdb_cluster":[8457]},"id":8}]} 2024-04-25T14:10:32.808 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 85 2024-04-25T14:10:32.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.813+0000 7fd5adc00640 1 -- 172.21.15.86:0/1940531942 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fd58806f4c0 msgr2=0x7fd5880719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:32.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.813+0000 7fd5adc00640 1 --2- 172.21.15.86:0/1940531942 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fd58806f4c0 0x7fd5880719d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd598005890 tx=0x7fd598009040 comp rx=0 tx=0).stop 2024-04-25T14:10:32.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.813+0000 7fd5adc00640 1 -- 172.21.15.86:0/1940531942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5a80d16e0 msgr2=0x7fd5a80c98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:32.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.813+0000 7fd5adc00640 1 --2- 172.21.15.86:0/1940531942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5a80d16e0 0x7fd5a80c98f0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fd5a001b020 tx=0x7fd5a0019710 comp rx=0 tx=0).stop 2024-04-25T14:10:32.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.813+0000 7fd5adc00640 1 -- 172.21.15.86:0/1940531942 shutdown_connections 2024-04-25T14:10:32.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.813+0000 7fd5adc00640 1 --2- 172.21.15.86:0/1940531942 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fd58806f4c0 0x7fd5880719d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.813+0000 7fd5adc00640 1 --2- 172.21.15.86:0/1940531942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5a80d16e0 0x7fd5a80c98f0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.813+0000 7fd5adc00640 1 --2- 172.21.15.86:0/1940531942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5a80cfca0 0x7fd5a80c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.813+0000 7fd5adc00640 1 --2- 172.21.15.86:0/1940531942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5a80730b0 0x7fd5a815b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.813+0000 7fd5adc00640 1 -- 172.21.15.86:0/1940531942 >> 172.21.15.86:0/1940531942 conn(0x7fd5a80b4650 msgr2=0x7fd5a80ae8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:32.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.813+0000 7fd5adc00640 1 -- 172.21.15.86:0/1940531942 shutdown_connections 2024-04-25T14:10:32.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.813+0000 7fd5adc00640 1 -- 172.21.15.86:0/1940531942 wait complete. 2024-04-25T14:10:32.830 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:10:32.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.985+0000 7f8f4d400640 1 Processor -- start 2024-04-25T14:10:32.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.985+0000 7f8f4d400640 1 -- start start 2024-04-25T14:10:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.989+0000 7f8f4d400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f4815dc90 0x7f8f48161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.989+0000 7f8f4d400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f480cd8d0 con 0x7f8f48071530 2024-04-25T14:10:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.989+0000 7f8f4d400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f480ce970 con 0x7f8f480d2c80 2024-04-25T14:10:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.989+0000 7f8f4d400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8f480ce720 con 0x7f8f4815dc90 2024-04-25T14:10:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.989+0000 7f8f45e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8f48071530 0x7f8f481616e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39906/0 (socket says 172.21.15.86:39906) 2024-04-25T14:10:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.989+0000 7f8f45e00640 1 -- 172.21.15.86:0/1925653785 learned_addr learned my addr 172.21.15.86:0/1925653785 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.989+0000 7f8f47200640 1 --2- 172.21.15.86:0/1925653785 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f4815dc90 0x7f8f48161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.989+0000 7f8f45400640 1 -- 172.21.15.86:0/1925653785 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 14119520 0 0) 0x7f8f480cd8d0 con 0x7f8f48071530 2024-04-25T14:10:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.989+0000 7f8f45400640 1 -- 172.21.15.86:0/1925653785 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f2c003530 con 0x7f8f48071530 2024-04-25T14:10:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.989+0000 7f8f45400640 1 -- 172.21.15.86:0/1925653785 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2393793342 0 0) 0x7f8f480ce970 con 0x7f8f480d2c80 2024-04-25T14:10:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.989+0000 7f8f45400640 1 -- 172.21.15.86:0/1925653785 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f480cd8d0 con 0x7f8f480d2c80 2024-04-25T14:10:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.989+0000 7f8f47200640 1 -- 172.21.15.86:0/1925653785 >> v1:172.21.15.155:6789/0 conn(0x7f8f480d2c80 legacy=0x7f8f48070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.989+0000 7f8f47200640 1 -- 172.21.15.86:0/1925653785 >> v1:172.21.15.86:6789/0 conn(0x7f8f48071530 legacy=0x7f8f481616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.989+0000 7f8f47200640 1 -- 172.21.15.86:0/1925653785 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f480d1be0 con 0x7f8f4815dc90 2024-04-25T14:10:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.993+0000 7f8f47200640 1 --2- 172.21.15.86:0/1925653785 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f4815dc90 0x7f8f48161df0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8f40009dc0 tx=0x7f8f4001f170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e664b74c23026a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.993+0000 7f8f45400640 1 -- 172.21.15.86:0/1925653785 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f400b4070 con 0x7f8f4815dc90 2024-04-25T14:10:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.993+0000 7f8f45400640 1 -- 172.21.15.86:0/1925653785 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f400af600 con 0x7f8f4815dc90 2024-04-25T14:10:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.993+0000 7f8f45400640 1 -- 172.21.15.86:0/1925653785 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f400af9b0 con 0x7f8f4815dc90 2024-04-25T14:10:32.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.993+0000 7f8f4d400640 1 -- 172.21.15.86:0/1925653785 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f4815dc90 msgr2=0x7f8f48161df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:32.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.993+0000 7f8f4d400640 1 --2- 172.21.15.86:0/1925653785 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f4815dc90 0x7f8f48161df0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8f40009dc0 tx=0x7f8f4001f170 comp rx=0 tx=0).stop 2024-04-25T14:10:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.993+0000 7f8f4d400640 1 -- 172.21.15.86:0/1925653785 shutdown_connections 2024-04-25T14:10:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.993+0000 7f8f4d400640 1 --2- 172.21.15.86:0/1925653785 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f4815dc90 0x7f8f48161df0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.993+0000 7f8f4d400640 1 -- 172.21.15.86:0/1925653785 >> 172.21.15.86:0/1925653785 conn(0x7f8f480cffc0 msgr2=0x7f8f480d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.993+0000 7f8f4d400640 1 -- 172.21.15.86:0/1925653785 shutdown_connections 2024-04-25T14:10:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.993+0000 7f8f4d400640 1 -- 172.21.15.86:0/1925653785 wait complete. 2024-04-25T14:10:32.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.993+0000 7f8f4d400640 1 Processor -- start 2024-04-25T14:10:32.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.997+0000 7f8f4d400640 1 -- start start 2024-04-25T14:10:32.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.997+0000 7f8f4d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f480d2c80 0x7f8f480b5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.997+0000 7f8f4d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f4815dc90 0x7f8f481f9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.997+0000 7f8f4d400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f480b0cf0 0x7f8f480b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.997+0000 7f8f4d400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f480d2480 con 0x7f8f480d2c80 2024-04-25T14:10:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.997+0000 7f8f4d400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8f480d21a0 con 0x7f8f4815dc90 2024-04-25T14:10:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.997+0000 7f8f4d400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8f480d29a0 con 0x7f8f480b0cf0 2024-04-25T14:10:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.997+0000 7f8f46800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f480d2c80 0x7f8f480b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.997+0000 7f8f46800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f480d2c80 0x7f8f480b5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57390/0 (socket says 172.21.15.86:57390) 2024-04-25T14:10:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.997+0000 7f8f46800640 1 -- 172.21.15.86:0/776901574 learned_addr learned my addr 172.21.15.86:0/776901574 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.997+0000 7f8f47200640 1 --2- 172.21.15.86:0/776901574 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f480b0cf0 0x7f8f480b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.997+0000 7f8f45e00640 1 --2- 172.21.15.86:0/776901574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f4815dc90 0x7f8f481f9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.997+0000 7f8f46800640 1 -- 172.21.15.86:0/776901574 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f480b0cf0 msgr2=0x7f8f480b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.997+0000 7f8f46800640 1 --2- 172.21.15.86:0/776901574 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f480b0cf0 0x7f8f480b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.997+0000 7f8f46800640 1 -- 172.21.15.86:0/776901574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f4815dc90 msgr2=0x7f8f481f9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:32.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.997+0000 7f8f46800640 1 --2- 172.21.15.86:0/776901574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f4815dc90 0x7f8f481f9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.997+0000 7f8f46800640 1 -- 172.21.15.86:0/776901574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f480b1640 con 0x7f8f480d2c80 2024-04-25T14:10:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.997+0000 7f8f47200640 1 --2- 172.21.15.86:0/776901574 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f480b0cf0 0x7f8f480b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.997+0000 7f8f45e00640 1 --2- 172.21.15.86:0/776901574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f4815dc90 0x7f8f481f9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.997+0000 7f8f46800640 1 --2- 172.21.15.86:0/776901574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f480d2c80 0x7f8f480b5260 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f8f3800c460 tx=0x7f8f3800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.997+0000 7f8f3ea00640 1 -- 172.21.15.86:0/776901574 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f380052d0 con 0x7f8f480d2c80 2024-04-25T14:10:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.997+0000 7f8f3ea00640 1 -- 172.21.15.86:0/776901574 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f38005490 con 0x7f8f480d2c80 2024-04-25T14:10:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.997+0000 7f8f4d400640 1 -- 172.21.15.86:0/776901574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f481f9950 con 0x7f8f480d2c80 2024-04-25T14:10:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.997+0000 7f8f3ea00640 1 -- 172.21.15.86:0/776901574 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38012850 con 0x7f8f480d2c80 2024-04-25T14:10:32.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.997+0000 7f8f4d400640 1 -- 172.21.15.86:0/776901574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f480aea30 con 0x7f8f480d2c80 2024-04-25T14:10:32.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:32.997+0000 7f8f4d400640 1 -- 172.21.15.86:0/776901574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f480720e0 con 0x7f8f480d2c80 2024-04-25T14:10:32.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.001+0000 7f8f3ea00640 1 -- 172.21.15.86:0/776901574 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7f8f380129f0 con 0x7f8f480d2c80 2024-04-25T14:10:32.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.001+0000 7f8f3ea00640 1 --2- 172.21.15.86:0/776901574 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f8f2806f4c0 0x7f8f280719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:32.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.001+0000 7f8f3ea00640 1 -- 172.21.15.86:0/776901574 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6746+0+0 (secure 0 0 0) 0x7f8f3808f300 con 0x7f8f480d2c80 2024-04-25T14:10:32.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.001+0000 7f8f45e00640 1 --2- 172.21.15.86:0/776901574 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f8f2806f4c0 0x7f8f280719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:32.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.001+0000 7f8f45e00640 1 --2- 172.21.15.86:0/776901574 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f8f2806f4c0 0x7f8f280719d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8f480b0520 tx=0x7f8f34009040 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:33.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.005+0000 7f8f3ea00640 1 -- 172.21.15.86:0/776901574 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8f480b1640 con 0x7f8f480d2c80 2024-04-25T14:10:33.012 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:33.013+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:33.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.209+0000 7f8f4d400640 1 -- 172.21.15.86:0/776901574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f48076bf0 con 0x7f8f480d2c80 2024-04-25T14:10:33.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.209+0000 7f8f3ea00640 1 -- 172.21.15.86:0/776901574 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5183 (secure 0 0 0) 0x7f8f3805be00 con 0x7f8f480d2c80 2024-04-25T14:10:33.207 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:33.208 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":85,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/374465657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":374465657},{"type":"v1","addr":"172.21.15.86:6836","nonce":374465657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8107,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1466195534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":1466195534},{"type":"v1","addr":"172.21.15.86:6837","nonce":1466195534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8119,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2431580070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2431580070},{"type":"v1","addr":"172.21.15.155:6835","nonce":2431580070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:10:05.129470+0000","modified":"2024-04-25T14:10:08.145259+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8457":{"gid":8457,"name":"b","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6833/142240252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":142240252},{"type":"v1","addr":"172.21.15.155:6833","nonce":142240252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8457,"qdb_cluster":[8457]},"id":8}]} 2024-04-25T14:10:33.208 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 85 2024-04-25T14:10:33.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.213+0000 7f8f4d400640 1 -- 172.21.15.86:0/776901574 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f8f2806f4c0 msgr2=0x7f8f280719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.213+0000 7f8f4d400640 1 --2- 172.21.15.86:0/776901574 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f8f2806f4c0 0x7f8f280719d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8f480b0520 tx=0x7f8f34009040 comp rx=0 tx=0).stop 2024-04-25T14:10:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.213+0000 7f8f4d400640 1 -- 172.21.15.86:0/776901574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f480d2c80 msgr2=0x7f8f480b5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.213+0000 7f8f4d400640 1 --2- 172.21.15.86:0/776901574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f480d2c80 0x7f8f480b5260 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f8f3800c460 tx=0x7f8f3800c920 comp rx=0 tx=0).stop 2024-04-25T14:10:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.213+0000 7f8f4d400640 1 -- 172.21.15.86:0/776901574 shutdown_connections 2024-04-25T14:10:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.213+0000 7f8f4d400640 1 --2- 172.21.15.86:0/776901574 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f480b0cf0 0x7f8f480b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.213+0000 7f8f4d400640 1 --2- 172.21.15.86:0/776901574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f4815dc90 0x7f8f481f9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.213+0000 7f8f4d400640 1 --2- 172.21.15.86:0/776901574 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f8f2806f4c0 0x7f8f280719d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.213+0000 7f8f4d400640 1 --2- 172.21.15.86:0/776901574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f480d2c80 0x7f8f480b5260 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.213+0000 7f8f4d400640 1 -- 172.21.15.86:0/776901574 >> 172.21.15.86:0/776901574 conn(0x7f8f480cffc0 msgr2=0x7f8f480d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.213+0000 7f8f4d400640 1 -- 172.21.15.86:0/776901574 shutdown_connections 2024-04-25T14:10:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.213+0000 7f8f4d400640 1 -- 172.21.15.86:0/776901574 wait complete. 2024-04-25T14:10:33.230 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:10:33.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.385+0000 7fac57000640 1 Processor -- start 2024-04-25T14:10:33.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.389+0000 7fac57000640 1 -- start start 2024-04-25T14:10:33.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.389+0000 7fac57000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fac500cc8c0 0x7fac500cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:33.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.389+0000 7fac57000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac500d14c0 con 0x7fac500cd210 2024-04-25T14:10:33.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.389+0000 7fac57000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac500ce220 con 0x7fac500ce7a0 2024-04-25T14:10:33.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.389+0000 7fac57000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fac500d1760 con 0x7fac500cc8c0 2024-04-25T14:10:33.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.389+0000 7fac56600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fac500cd210 0x7fac500cd620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39908/0 (socket says 172.21.15.86:39908) 2024-04-25T14:10:33.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.389+0000 7fac56600640 1 -- 172.21.15.86:0/3433920903 learned_addr learned my addr 172.21.15.86:0/3433920903 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:33.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.389+0000 7fac55200640 1 --2- 172.21.15.86:0/3433920903 >> v2:172.21.15.155:3300/0 conn(0x7fac500cc8c0 0x7fac500cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:33.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.389+0000 7fac4fe00640 1 -- 172.21.15.86:0/3433920903 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1695963160 0 0) 0x7fac500d14c0 con 0x7fac500cd210 2024-04-25T14:10:33.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.389+0000 7fac4fe00640 1 -- 172.21.15.86:0/3433920903 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac38003530 con 0x7fac500cd210 2024-04-25T14:10:33.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.389+0000 7fac4fe00640 1 -- 172.21.15.86:0/3433920903 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 908957898 0 0) 0x7fac500ce220 con 0x7fac500ce7a0 2024-04-25T14:10:33.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.389+0000 7fac4fe00640 1 -- 172.21.15.86:0/3433920903 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac500d14c0 con 0x7fac500ce7a0 2024-04-25T14:10:33.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.389+0000 7fac55200640 1 -- 172.21.15.86:0/3433920903 >> v1:172.21.15.155:6789/0 conn(0x7fac500ce7a0 legacy=0x7fac500cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:33.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.389+0000 7fac55200640 1 -- 172.21.15.86:0/3433920903 >> v1:172.21.15.86:6789/0 conn(0x7fac500cd210 legacy=0x7fac500cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:33.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.389+0000 7fac55200640 1 -- 172.21.15.86:0/3433920903 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac500d1a40 con 0x7fac500cc8c0 2024-04-25T14:10:33.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.393+0000 7fac55200640 1 --2- 172.21.15.86:0/3433920903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac500cc8c0 0x7fac500cccd0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fac3c0097c0 tx=0x7fac3c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d08de7c4e6b5b1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:33.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.393+0000 7fac4fe00640 1 -- 172.21.15.86:0/3433920903 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c0aa620 con 0x7fac500cc8c0 2024-04-25T14:10:33.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.393+0000 7fac4fe00640 1 -- 172.21.15.86:0/3433920903 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac3c0af070 con 0x7fac500cc8c0 2024-04-25T14:10:33.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.393+0000 7fac4fe00640 1 -- 172.21.15.86:0/3433920903 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c0aa9c0 con 0x7fac500cc8c0 2024-04-25T14:10:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.393+0000 7fac57000640 1 -- 172.21.15.86:0/3433920903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac500cc8c0 msgr2=0x7fac500cccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.393+0000 7fac57000640 1 --2- 172.21.15.86:0/3433920903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac500cc8c0 0x7fac500cccd0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fac3c0097c0 tx=0x7fac3c01c4a0 comp rx=0 tx=0).stop 2024-04-25T14:10:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.393+0000 7fac57000640 1 -- 172.21.15.86:0/3433920903 shutdown_connections 2024-04-25T14:10:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.393+0000 7fac57000640 1 --2- 172.21.15.86:0/3433920903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac500cc8c0 0x7fac500cccd0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.393+0000 7fac57000640 1 -- 172.21.15.86:0/3433920903 >> 172.21.15.86:0/3433920903 conn(0x7fac5009ba30 msgr2=0x7fac500b70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.393+0000 7fac57000640 1 -- 172.21.15.86:0/3433920903 shutdown_connections 2024-04-25T14:10:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.393+0000 7fac57000640 1 -- 172.21.15.86:0/3433920903 wait complete. 2024-04-25T14:10:33.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.397+0000 7fac57000640 1 Processor -- start 2024-04-25T14:10:33.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.397+0000 7fac57000640 1 -- start start 2024-04-25T14:10:33.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.397+0000 7fac57000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac500cc8c0 0x7fac501584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:33.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.397+0000 7fac57000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac500cd210 0x7fac5015e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:33.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.397+0000 7fac57000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac500ce7a0 0x7fac500b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:33.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.397+0000 7fac57000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac5016e4e0 con 0x7fac500cd210 2024-04-25T14:10:33.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.397+0000 7fac57000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fac500b3c10 con 0x7fac500cc8c0 2024-04-25T14:10:33.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.397+0000 7fac57000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fac5016e660 con 0x7fac500ce7a0 2024-04-25T14:10:33.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.397+0000 7fac55200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac500cd210 0x7fac5015e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:33.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.397+0000 7fac55200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac500cd210 0x7fac5015e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57404/0 (socket says 172.21.15.86:57404) 2024-04-25T14:10:33.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.397+0000 7fac55200640 1 -- 172.21.15.86:0/545119090 learned_addr learned my addr 172.21.15.86:0/545119090 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:33.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.397+0000 7fac55c00640 1 --2- 172.21.15.86:0/545119090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac500cc8c0 0x7fac501584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:33.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.397+0000 7fac56600640 1 --2- 172.21.15.86:0/545119090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac500ce7a0 0x7fac500b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:33.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.397+0000 7fac55200640 1 -- 172.21.15.86:0/545119090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac500ce7a0 msgr2=0x7fac500b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:33.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.397+0000 7fac55200640 1 --2- 172.21.15.86:0/545119090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac500ce7a0 0x7fac500b3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:33.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.397+0000 7fac55200640 1 -- 172.21.15.86:0/545119090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac500cc8c0 msgr2=0x7fac501584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:33.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.397+0000 7fac55200640 1 --2- 172.21.15.86:0/545119090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac500cc8c0 0x7fac501584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:33.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.397+0000 7fac55200640 1 -- 172.21.15.86:0/545119090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac5015c260 con 0x7fac500cd210 2024-04-25T14:10:33.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.401+0000 7fac55200640 1 --2- 172.21.15.86:0/545119090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac500cd210 0x7fac5015e650 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fac3c0ac040 tx=0x7fac3c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:33.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.401+0000 7fac4d600640 1 -- 172.21.15.86:0/545119090 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c0aabe0 con 0x7fac500cd210 2024-04-25T14:10:33.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.401+0000 7fac4d600640 1 -- 172.21.15.86:0/545119090 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac3c0aad80 con 0x7fac500cd210 2024-04-25T14:10:33.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.401+0000 7fac4d600640 1 -- 172.21.15.86:0/545119090 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c0be630 con 0x7fac500cd210 2024-04-25T14:10:33.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.401+0000 7fac57000640 1 -- 172.21.15.86:0/545119090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac5015c500 con 0x7fac500cd210 2024-04-25T14:10:33.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.401+0000 7fac57000640 1 -- 172.21.15.86:0/545119090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac5015cac0 con 0x7fac500cd210 2024-04-25T14:10:33.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.401+0000 7fac57000640 1 -- 172.21.15.86:0/545119090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac50076b80 con 0x7fac500cd210 2024-04-25T14:10:33.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.401+0000 7fac4d600640 1 -- 172.21.15.86:0/545119090 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7fac3c0bc020 con 0x7fac500cd210 2024-04-25T14:10:33.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.401+0000 7fac4d600640 1 --2- 172.21.15.86:0/545119090 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fac3006f4c0 0x7fac300719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:33.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.401+0000 7fac4d600640 1 -- 172.21.15.86:0/545119090 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6746+0+0 (secure 0 0 0) 0x7fac3c1298a0 con 0x7fac500cd210 2024-04-25T14:10:33.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.401+0000 7fac55c00640 1 --2- 172.21.15.86:0/545119090 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fac3006f4c0 0x7fac300719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:33.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.405+0000 7fac55c00640 1 --2- 172.21.15.86:0/545119090 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fac3006f4c0 0x7fac300719d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fac50073870 tx=0x7fac44009040 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:33.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.409+0000 7fac4d600640 1 -- 172.21.15.86:0/545119090 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fac3c016090 con 0x7fac500cd210 2024-04-25T14:10:33.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.573+0000 7fac57000640 1 -- 172.21.15.86:0/545119090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac5015bb40 con 0x7fac500cd210 2024-04-25T14:10:33.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.577+0000 7fac4d600640 1 -- 172.21.15.86:0/545119090 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+17889 (secure 0 0 0) 0x7fac3c0fe070 con 0x7fac500cd210 2024-04-25T14:10:33.573 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:33.574 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":115,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:10:29.591997+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:10:03.520649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T14:10:04.080912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1062432465":"2024-04-26T14:10:29.591964+0000","172.21.15.86:6832/190090395":"2024-04-26T14:10:29.591964+0000","172.21.15.86:0/1120094610":"2024-04-26T14:10:29.591964+0000","172.21.15.86:0/2412030790":"2024-04-26T14:10:29.591964+0000","172.21.15.86:0/4291479632":"2024-04-26T14:10:29.591964+0000","172.21.15.86:0/2052329939":"2024-04-26T14:10:23.454991+0000","172.21.15.86:0/1021944523":"2024-04-26T14:10:23.454991+0000","172.21.15.86:0/83615604":"2024-04-26T14:10:23.454991+0000","172.21.15.86:0/3719613294":"2024-04-26T14:10:23.454991+0000","172.21.15.86:0/3734007180":"2024-04-26T14:10:29.591964+0000","172.21.15.86:0/1471025278":"2024-04-26T14:10:23.454991+0000","172.21.15.86:6833/1354467416":"2024-04-26T14:10:23.454991+0000","172.21.15.86:6832/1354467416":"2024-04-26T14:10:23.454991+0000","172.21.15.86:6833/190090395":"2024-04-26T14:10:29.591964+0000","172.21.15.86:0/3000491277":"2024-04-26T14:10:23.454991+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:10:33.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.577+0000 7fac57000640 1 -- 172.21.15.86:0/545119090 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fac3006f4c0 msgr2=0x7fac300719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:33.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.577+0000 7fac57000640 1 --2- 172.21.15.86:0/545119090 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fac3006f4c0 0x7fac300719d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fac50073870 tx=0x7fac44009040 comp rx=0 tx=0).stop 2024-04-25T14:10:33.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.577+0000 7fac57000640 1 -- 172.21.15.86:0/545119090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac500cd210 msgr2=0x7fac5015e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:33.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.577+0000 7fac57000640 1 --2- 172.21.15.86:0/545119090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac500cd210 0x7fac5015e650 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fac3c0ac040 tx=0x7fac3c01ace0 comp rx=0 tx=0).stop 2024-04-25T14:10:33.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.577+0000 7fac57000640 1 -- 172.21.15.86:0/545119090 shutdown_connections 2024-04-25T14:10:33.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.577+0000 7fac57000640 1 --2- 172.21.15.86:0/545119090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac500ce7a0 0x7fac500b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:33.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.577+0000 7fac57000640 1 --2- 172.21.15.86:0/545119090 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fac3006f4c0 0x7fac300719d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:33.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.577+0000 7fac57000640 1 --2- 172.21.15.86:0/545119090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac500cd210 0x7fac5015e650 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:33.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.577+0000 7fac57000640 1 --2- 172.21.15.86:0/545119090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac500cc8c0 0x7fac501584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:33.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.577+0000 7fac57000640 1 -- 172.21.15.86:0/545119090 >> 172.21.15.86:0/545119090 conn(0x7fac5009ba30 msgr2=0x7fac5006f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:33.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.577+0000 7fac57000640 1 -- 172.21.15.86:0/545119090 shutdown_connections 2024-04-25T14:10:33.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.577+0000 7fac57000640 1 -- 172.21.15.86:0/545119090 wait complete. 2024-04-25T14:10:33.596 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T14:10:33.719 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:33.721+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:33.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.753+0000 7f779c800640 1 Processor -- start 2024-04-25T14:10:33.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.753+0000 7f779c800640 1 -- start start 2024-04-25T14:10:33.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.757+0000 7f779c800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f77940d2420 0x7f77940d2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:33.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.757+0000 7f779c800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f77940b6490 0x7f77940b68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:33.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.757+0000 7f779c800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7794162480 con 0x7f7794070e20 2024-04-25T14:10:33.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.757+0000 7f779c800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f77940d16f0 con 0x7f77940d2420 2024-04-25T14:10:33.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.757+0000 7f779c800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f77940d1940 con 0x7f77940b6490 2024-04-25T14:10:33.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.757+0000 7f7799e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f77940d2420 0x7f77940d2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:33.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.757+0000 7f7799400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7794070e20 0x7f7794071230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39912/0 (socket says 172.21.15.86:39912) 2024-04-25T14:10:33.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.757+0000 7f7799e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f77940d2420 0x7f77940d2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57416/0 (socket says 172.21.15.86:57416) 2024-04-25T14:10:33.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.757+0000 7f7799400640 1 -- 172.21.15.86:0/3122893390 learned_addr learned my addr 172.21.15.86:0/3122893390 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:33.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.757+0000 7f779a800640 1 --2- 172.21.15.86:0/3122893390 >> v2:172.21.15.155:3300/0 conn(0x7f77940b6490 0x7f77940b68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:33.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.757+0000 7f7799e00640 1 -- 172.21.15.86:0/3122893390 >> v2:172.21.15.155:3300/0 conn(0x7f77940b6490 msgr2=0x7f77940b68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:33.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.757+0000 7f7799e00640 1 --2- 172.21.15.86:0/3122893390 >> v2:172.21.15.155:3300/0 conn(0x7f77940b6490 0x7f77940b68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:33.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.757+0000 7f7799e00640 1 -- 172.21.15.86:0/3122893390 >> v1:172.21.15.86:6789/0 conn(0x7f7794070e20 legacy=0x7f7794071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:33.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.757+0000 7f7799e00640 1 -- 172.21.15.86:0/3122893390 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77940d1c20 con 0x7f77940d2420 2024-04-25T14:10:33.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.757+0000 7f7798a00640 1 -- 172.21.15.86:0/3122893390 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1055050456 0 0) 0x7f7794162480 con 0x7f7794070e20 2024-04-25T14:10:33.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.761+0000 7f7799e00640 1 --2- 172.21.15.86:0/3122893390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77940d2420 0x7f77940d2830 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f7794076550 tx=0x7f778801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc42f5652f454f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:33.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.761+0000 7f7798a00640 1 -- 172.21.15.86:0/3122893390 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77880b0070 con 0x7f77940d2420 2024-04-25T14:10:33.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.761+0000 7f7798a00640 1 -- 172.21.15.86:0/3122893390 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77880ab600 con 0x7f77940d2420 2024-04-25T14:10:33.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.761+0000 7f7798a00640 1 -- 172.21.15.86:0/3122893390 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77880ab9d0 con 0x7f77940d2420 2024-04-25T14:10:33.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.761+0000 7f779c800640 1 -- 172.21.15.86:0/3122893390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77940d2420 msgr2=0x7f77940d2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:33.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.761+0000 7f779c800640 1 --2- 172.21.15.86:0/3122893390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77940d2420 0x7f77940d2830 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f7794076550 tx=0x7f778801c480 comp rx=0 tx=0).stop 2024-04-25T14:10:33.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.761+0000 7f779c800640 1 -- 172.21.15.86:0/3122893390 shutdown_connections 2024-04-25T14:10:33.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.761+0000 7f779c800640 1 --2- 172.21.15.86:0/3122893390 >> v2:172.21.15.155:3300/0 conn(0x7f77940b6490 0x7f77940b68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:33.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.761+0000 7f779c800640 1 --2- 172.21.15.86:0/3122893390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77940d2420 0x7f77940d2830 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:33.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.761+0000 7f779c800640 1 -- 172.21.15.86:0/3122893390 >> 172.21.15.86:0/3122893390 conn(0x7f77940cffc0 msgr2=0x7f77940d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:33.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.761+0000 7f779c800640 1 -- 172.21.15.86:0/3122893390 shutdown_connections 2024-04-25T14:10:33.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.761+0000 7f779c800640 1 -- 172.21.15.86:0/3122893390 wait complete. 2024-04-25T14:10:33.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.761+0000 7f779c800640 1 Processor -- start 2024-04-25T14:10:33.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.761+0000 7f779c800640 1 -- start start 2024-04-25T14:10:33.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.765+0000 7f779c800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7794070e20 0x7f77940c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:33.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.765+0000 7f779c800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77940b6490 0x7f7794158890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:33.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.765+0000 7f779c800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77940d2420 0x7f77940b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:33.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.765+0000 7f779c800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7794161310 con 0x7f7794070e20 2024-04-25T14:10:33.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.765+0000 7f779c800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f77940cdf30 con 0x7f77940b6490 2024-04-25T14:10:33.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.765+0000 7f779c800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f779415cfb0 con 0x7f77940d2420 2024-04-25T14:10:33.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.765+0000 7f7799e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7794070e20 0x7f77940c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.765+0000 7f7799e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7794070e20 0x7f77940c91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57432/0 (socket says 172.21.15.86:57432) 2024-04-25T14:10:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.765+0000 7f7799400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77940b6490 0x7f7794158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.765+0000 7f7799e00640 1 -- 172.21.15.86:0/1761003202 learned_addr learned my addr 172.21.15.86:0/1761003202 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.765+0000 7f779a800640 1 --2- 172.21.15.86:0/1761003202 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77940d2420 0x7f77940b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.765+0000 7f7799e00640 1 -- 172.21.15.86:0/1761003202 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77940d2420 msgr2=0x7f77940b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.765+0000 7f7799e00640 1 --2- 172.21.15.86:0/1761003202 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77940d2420 0x7f77940b3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.765+0000 7f7799e00640 1 -- 172.21.15.86:0/1761003202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77940b6490 msgr2=0x7f7794158890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.765+0000 7f7799e00640 1 --2- 172.21.15.86:0/1761003202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77940b6490 0x7f7794158890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.765+0000 7f7799e00640 1 -- 172.21.15.86:0/1761003202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77940c96f0 con 0x7f7794070e20 2024-04-25T14:10:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.765+0000 7f7799400640 1 --2- 172.21.15.86:0/1761003202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77940b6490 0x7f7794158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:10:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.765+0000 7f7799e00640 1 --2- 172.21.15.86:0/1761003202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7794070e20 0x7f77940c91b0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f77880ae040 tx=0x7f778801acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.765+0000 7f778e000640 1 -- 172.21.15.86:0/1761003202 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77880b0050 con 0x7f7794070e20 2024-04-25T14:10:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.765+0000 7f778e000640 1 -- 172.21.15.86:0/1761003202 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77880bc070 con 0x7f7794070e20 2024-04-25T14:10:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.765+0000 7f779c800640 1 -- 172.21.15.86:0/1761003202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f779415c3a0 con 0x7f7794070e20 2024-04-25T14:10:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.765+0000 7f778e000640 1 -- 172.21.15.86:0/1761003202 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77880b5830 con 0x7f7794070e20 2024-04-25T14:10:33.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.765+0000 7f779c800640 1 -- 172.21.15.86:0/1761003202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f779415c960 con 0x7f7794070e20 2024-04-25T14:10:33.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.765+0000 7f779c800640 1 -- 172.21.15.86:0/1761003202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7794076bf0 con 0x7f7794070e20 2024-04-25T14:10:33.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.769+0000 7f778e000640 1 -- 172.21.15.86:0/1761003202 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7f77880b59d0 con 0x7f7794070e20 2024-04-25T14:10:33.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.769+0000 7f778e000640 1 --2- 172.21.15.86:0/1761003202 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f777806f4c0 0x7f77780719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:33.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.769+0000 7f778e000640 1 -- 172.21.15.86:0/1761003202 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6746+0+0 (secure 0 0 0) 0x7f7788129d20 con 0x7f7794070e20 2024-04-25T14:10:33.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.769+0000 7f7799400640 1 --2- 172.21.15.86:0/1761003202 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f777806f4c0 0x7f77780719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:33.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.769+0000 7f7799400640 1 --2- 172.21.15.86:0/1761003202 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f777806f4c0 0x7f77780719d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f77940b36d0 tx=0x7f7784009040 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:33.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.773+0000 7f778e000640 1 -- 172.21.15.86:0/1761003202 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f77880c6020 con 0x7f7794070e20 2024-04-25T14:10:33.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.973+0000 7f779c800640 1 -- 172.21.15.86:0/1761003202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f77940ce2f0 con 0x7f7794070e20 2024-04-25T14:10:33.973 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-25T14:10:33.977+0000 7fea29c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:10:33.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.977+0000 7f778e000640 1 -- 172.21.15.86:0/1761003202 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v86) ==== 161+0+0 (secure 0 0 0) 0x7f7788016070 con 0x7f7794070e20 2024-04-25T14:10:33.975 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T14:10:33.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.981+0000 7f779c800640 1 -- 172.21.15.86:0/1761003202 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f777806f4c0 msgr2=0x7f77780719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:33.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.981+0000 7f779c800640 1 --2- 172.21.15.86:0/1761003202 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f777806f4c0 0x7f77780719d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f77940b36d0 tx=0x7f7784009040 comp rx=0 tx=0).stop 2024-04-25T14:10:33.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.981+0000 7f779c800640 1 -- 172.21.15.86:0/1761003202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7794070e20 msgr2=0x7f77940c91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:33.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.981+0000 7f779c800640 1 --2- 172.21.15.86:0/1761003202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7794070e20 0x7f77940c91b0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f77880ae040 tx=0x7f778801acc0 comp rx=0 tx=0).stop 2024-04-25T14:10:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.981+0000 7f779c800640 1 -- 172.21.15.86:0/1761003202 shutdown_connections 2024-04-25T14:10:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.981+0000 7f779c800640 1 --2- 172.21.15.86:0/1761003202 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77940d2420 0x7f77940b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.981+0000 7f779c800640 1 --2- 172.21.15.86:0/1761003202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77940b6490 0x7f7794158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.981+0000 7f779c800640 1 --2- 172.21.15.86:0/1761003202 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f777806f4c0 0x7f77780719d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.981+0000 7f779c800640 1 --2- 172.21.15.86:0/1761003202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7794070e20 0x7f77940c91b0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.981+0000 7f779c800640 1 -- 172.21.15.86:0/1761003202 >> 172.21.15.86:0/1761003202 conn(0x7f77940cffc0 msgr2=0x7f77940d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.981+0000 7f779c800640 1 -- 172.21.15.86:0/1761003202 shutdown_connections 2024-04-25T14:10:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:33.981+0000 7f779c800640 1 -- 172.21.15.86:0/1761003202 wait complete. 2024-04-25T14:10:33.985 INFO:tasks.ceph.mds.b.smithi155.stderr: -3586> 2024-04-25T14:10:07.711+0000 7f4dac400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T14:10:34.000 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T14:10:34.106 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2024-04-25T14:10:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.161+0000 7f3e9ba00640 1 Processor -- start 2024-04-25T14:10:34.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.161+0000 7f3e9ba00640 1 -- start start 2024-04-25T14:10:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.161+0000 7f3e9ba00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3e94071530 0x7f3e94163e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.161+0000 7f3e9ba00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e940d1c10 con 0x7f3e9406ebb0 2024-04-25T14:10:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.161+0000 7f3e9ba00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e940d1ef0 con 0x7f3e9406e7a0 2024-04-25T14:10:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.161+0000 7f3e9ba00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3e941609b0 con 0x7f3e94071530 2024-04-25T14:10:34.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.165+0000 7f3e8be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3e9406ebb0 0x7f3e94070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39918/0 (socket says 172.21.15.86:39918) 2024-04-25T14:10:34.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.165+0000 7f3e8be00640 1 -- 172.21.15.86:0/3782330981 learned_addr learned my addr 172.21.15.86:0/3782330981 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:34.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.165+0000 7f3e99000640 1 --1- 172.21.15.86:0/3782330981 >> v1:172.21.15.155:6789/0 conn(0x7f3e9406e7a0 0x7f3e941602a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:52920/0 (socket says 172.21.15.86:52920) 2024-04-25T14:10:34.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.165+0000 7f3e99a00640 1 --2- 172.21.15.86:0/3782330981 >> v2:172.21.15.86:3300/0 conn(0x7f3e94071530 0x7f3e94163e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:34.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.165+0000 7f3e93e00640 1 -- 172.21.15.86:0/3782330981 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 916336506 0 0) 0x7f3e940d1ef0 con 0x7f3e9406e7a0 2024-04-25T14:10:34.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.165+0000 7f3e93e00640 1 -- 172.21.15.86:0/3782330981 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e78003530 con 0x7f3e9406e7a0 2024-04-25T14:10:34.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.165+0000 7f3e93e00640 1 -- 172.21.15.86:0/3782330981 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 743704988 0 0) 0x7f3e940d1c10 con 0x7f3e9406ebb0 2024-04-25T14:10:34.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.165+0000 7f3e93e00640 1 -- 172.21.15.86:0/3782330981 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e940d1ef0 con 0x7f3e9406ebb0 2024-04-25T14:10:34.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.165+0000 7f3e99a00640 1 -- 172.21.15.86:0/3782330981 >> v1:172.21.15.155:6789/0 conn(0x7f3e9406e7a0 legacy=0x7f3e941602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:34.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.165+0000 7f3e99a00640 1 -- 172.21.15.86:0/3782330981 >> v1:172.21.15.86:6789/0 conn(0x7f3e9406ebb0 legacy=0x7f3e94070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:34.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.165+0000 7f3e99a00640 1 -- 172.21.15.86:0/3782330981 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e940d17d0 con 0x7f3e94071530 2024-04-25T14:10:34.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.165+0000 7f3e99a00640 1 --2- 172.21.15.86:0/3782330981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e94071530 0x7f3e94163e10 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009d90 tx=0x7f3e8c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d831a47be6429b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:34.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.165+0000 7f3e93e00640 1 -- 172.21.15.86:0/3782330981 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c0b2070 con 0x7f3e94071530 2024-04-25T14:10:34.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.165+0000 7f3e93e00640 1 -- 172.21.15.86:0/3782330981 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e8c0a57a0 con 0x7f3e94071530 2024-04-25T14:10:34.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.165+0000 7f3e93e00640 1 -- 172.21.15.86:0/3782330981 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c0a5b70 con 0x7f3e94071530 2024-04-25T14:10:34.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.169+0000 7f3e9ba00640 1 -- 172.21.15.86:0/3782330981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e94071530 msgr2=0x7f3e94163e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:34.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.169+0000 7f3e9ba00640 1 --2- 172.21.15.86:0/3782330981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e94071530 0x7f3e94163e10 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009d90 tx=0x7f3e8c01c8e0 comp rx=0 tx=0).stop 2024-04-25T14:10:34.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.169+0000 7f3e9ba00640 1 -- 172.21.15.86:0/3782330981 shutdown_connections 2024-04-25T14:10:34.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.169+0000 7f3e9ba00640 1 --2- 172.21.15.86:0/3782330981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e94071530 0x7f3e94163e10 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:34.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.169+0000 7f3e9ba00640 1 -- 172.21.15.86:0/3782330981 >> 172.21.15.86:0/3782330981 conn(0x7f3e940cffc0 msgr2=0x7f3e940d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:34.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.169+0000 7f3e9ba00640 1 -- 172.21.15.86:0/3782330981 shutdown_connections 2024-04-25T14:10:34.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.169+0000 7f3e9ba00640 1 -- 172.21.15.86:0/3782330981 wait complete. 2024-04-25T14:10:34.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.169+0000 7f3e9ba00640 1 Processor -- start 2024-04-25T14:10:34.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.169+0000 7f3e9ba00640 1 -- start start 2024-04-25T14:10:34.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.173+0000 7f3e9ba00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9406e7a0 0x7f3e9414d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:34.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.173+0000 7f3e9ba00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e9406ebb0 0x7f3e940ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:34.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.173+0000 7f3e9ba00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e9414d970 0x7f3e94162750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:34.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.173+0000 7f3e9ba00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9416e7e0 con 0x7f3e9406e7a0 2024-04-25T14:10:34.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.173+0000 7f3e9ba00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9416e660 con 0x7f3e9406ebb0 2024-04-25T14:10:34.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.173+0000 7f3e9ba00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3e9416e960 con 0x7f3e9414d970 2024-04-25T14:10:34.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.173+0000 7f3e99000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9406e7a0 0x7f3e9414d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:34.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.173+0000 7f3e99000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9406e7a0 0x7f3e9414d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57460/0 (socket says 172.21.15.86:57460) 2024-04-25T14:10:34.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.173+0000 7f3e99000640 1 -- 172.21.15.86:0/981563122 learned_addr learned my addr 172.21.15.86:0/981563122 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:34.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.173+0000 7f3e8be00640 1 --2- 172.21.15.86:0/981563122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e9406ebb0 0x7f3e940ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:34.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.173+0000 7f3e99a00640 1 --2- 172.21.15.86:0/981563122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e9414d970 0x7f3e94162750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:34.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.173+0000 7f3e99000640 1 -- 172.21.15.86:0/981563122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e9414d970 msgr2=0x7f3e94162750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:34.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.173+0000 7f3e99000640 1 --2- 172.21.15.86:0/981563122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e9414d970 0x7f3e94162750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:34.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.173+0000 7f3e99000640 1 -- 172.21.15.86:0/981563122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e9406ebb0 msgr2=0x7f3e940ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:34.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.173+0000 7f3e99000640 1 --2- 172.21.15.86:0/981563122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e9406ebb0 0x7f3e940ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:34.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.173+0000 7f3e99000640 1 -- 172.21.15.86:0/981563122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e940caac0 con 0x7f3e9406e7a0 2024-04-25T14:10:34.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.173+0000 7f3e99000640 1 --2- 172.21.15.86:0/981563122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9406e7a0 0x7f3e9414d370 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f3e8400c420 tx=0x7f3e8400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:34.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.173+0000 7f3e91600640 1 -- 172.21.15.86:0/981563122 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e840074e0 con 0x7f3e9406e7a0 2024-04-25T14:10:34.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.173+0000 7f3e91600640 1 -- 172.21.15.86:0/981563122 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e84007680 con 0x7f3e9406e7a0 2024-04-25T14:10:34.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.173+0000 7f3e91600640 1 -- 172.21.15.86:0/981563122 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e84004a30 con 0x7f3e9406e7a0 2024-04-25T14:10:34.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.173+0000 7f3e9ba00640 1 -- 172.21.15.86:0/981563122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e94160c60 con 0x7f3e9406e7a0 2024-04-25T14:10:34.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.173+0000 7f3e9ba00640 1 -- 172.21.15.86:0/981563122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e94161f50 con 0x7f3e9406e7a0 2024-04-25T14:10:34.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.173+0000 7f3e9ba00640 1 -- 172.21.15.86:0/981563122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e940720e0 con 0x7f3e9406e7a0 2024-04-25T14:10:34.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.177+0000 7f3e91600640 1 -- 172.21.15.86:0/981563122 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7f3e84004bd0 con 0x7f3e9406e7a0 2024-04-25T14:10:34.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.177+0000 7f3e91600640 1 --2- 172.21.15.86:0/981563122 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f3e7006f490 0x7f3e700719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:34.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.177+0000 7f3e91600640 1 -- 172.21.15.86:0/981563122 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6832+0+0 (secure 0 0 0) 0x7f3e8408ebf0 con 0x7f3e9406e7a0 2024-04-25T14:10:34.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.177+0000 7f3e8be00640 1 --2- 172.21.15.86:0/981563122 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f3e7006f490 0x7f3e700719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:34.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.177+0000 7f3e8be00640 1 --2- 172.21.15.86:0/981563122 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f3e7006f490 0x7f3e700719a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3e940cf3d0 tx=0x7f3e7c009040 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:34.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.181+0000 7f3e91600640 1 -- 172.21.15.86:0/981563122 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3e8405b0f0 con 0x7f3e9406e7a0 2024-04-25T14:10:34.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.377+0000 7f3e9ba00640 1 -- 172.21.15.86:0/981563122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f3e94076bf0 con 0x7f3e9406e7a0 2024-04-25T14:10:34.423 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:34.425+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:34.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.977+0000 7f3e91600640 1 -- 172.21.15.86:0/981563122 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v87) ==== 104+0+0 (secure 0 0 0) 0x7f3e8405f3f0 con 0x7f3e9406e7a0 2024-04-25T14:10:34.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.981+0000 7f3e9ba00640 1 -- 172.21.15.86:0/981563122 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f3e7006f490 msgr2=0x7f3e700719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:34.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.981+0000 7f3e9ba00640 1 --2- 172.21.15.86:0/981563122 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f3e7006f490 0x7f3e700719a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3e940cf3d0 tx=0x7f3e7c009040 comp rx=0 tx=0).stop 2024-04-25T14:10:34.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.981+0000 7f3e9ba00640 1 -- 172.21.15.86:0/981563122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9406e7a0 msgr2=0x7f3e9414d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:34.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.981+0000 7f3e9ba00640 1 --2- 172.21.15.86:0/981563122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9406e7a0 0x7f3e9414d370 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f3e8400c420 tx=0x7f3e8400c8e0 comp rx=0 tx=0).stop 2024-04-25T14:10:34.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.981+0000 7f3e9ba00640 1 -- 172.21.15.86:0/981563122 shutdown_connections 2024-04-25T14:10:34.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.981+0000 7f3e9ba00640 1 --2- 172.21.15.86:0/981563122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e9414d970 0x7f3e94162750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:34.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.981+0000 7f3e9ba00640 1 --2- 172.21.15.86:0/981563122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e9406ebb0 0x7f3e940ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:34.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.981+0000 7f3e9ba00640 1 --2- 172.21.15.86:0/981563122 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f3e7006f490 0x7f3e700719a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.981+0000 7f3e9ba00640 1 --2- 172.21.15.86:0/981563122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9406e7a0 0x7f3e9414d370 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.981+0000 7f3e9ba00640 1 -- 172.21.15.86:0/981563122 >> 172.21.15.86:0/981563122 conn(0x7f3e940cffc0 msgr2=0x7f3e94157810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.981+0000 7f3e9ba00640 1 -- 172.21.15.86:0/981563122 shutdown_connections 2024-04-25T14:10:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:34.981+0000 7f3e9ba00640 1 -- 172.21.15.86:0/981563122 wait complete. 2024-04-25T14:10:34.999 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T14:10:35.128 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:35.129+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:35.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.161+0000 7fe33e000640 1 Processor -- start 2024-04-25T14:10:35.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.161+0000 7fe33e000640 1 -- start start 2024-04-25T14:10:35.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.161+0000 7fe33e000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3380d1aa0 0x7fe3380b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:35.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.161+0000 7fe33e000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe3380b2130 0x7fe3380b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:35.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.161+0000 7fe33e000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3380b34d0 con 0x7fe3380b77b0 2024-04-25T14:10:35.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.161+0000 7fe33e000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe3380d2ef0 con 0x7fe3380b2130 2024-04-25T14:10:35.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.161+0000 7fe33e000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe3380b37b0 con 0x7fe3380d1aa0 2024-04-25T14:10:35.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.165+0000 7fe33cc00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe3380b77b0 0x7fe3380d1390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39934/0 (socket says 172.21.15.86:39934) 2024-04-25T14:10:35.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.165+0000 7fe33cc00640 1 -- 172.21.15.86:0/1447327346 learned_addr learned my addr 172.21.15.86:0/1447327346 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:35.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.165+0000 7fe33d600640 1 --2- 172.21.15.86:0/1447327346 >> v2:172.21.15.155:3300/0 conn(0x7fe3380b2130 0x7fe3380b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:35.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.165+0000 7fe337e00640 1 --2- 172.21.15.86:0/1447327346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3380d1aa0 0x7fe3380b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:35.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.165+0000 7fe337400640 1 -- 172.21.15.86:0/1447327346 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2640503564 0 0) 0x7fe3380b34d0 con 0x7fe3380b77b0 2024-04-25T14:10:35.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.165+0000 7fe337400640 1 -- 172.21.15.86:0/1447327346 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe320003530 con 0x7fe3380b77b0 2024-04-25T14:10:35.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.165+0000 7fe33d600640 1 -- 172.21.15.86:0/1447327346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3380d1aa0 msgr2=0x7fe3380b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:35.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.165+0000 7fe33d600640 1 --2- 172.21.15.86:0/1447327346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3380d1aa0 0x7fe3380b1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:35.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.165+0000 7fe33d600640 1 -- 172.21.15.86:0/1447327346 >> v1:172.21.15.86:6789/0 conn(0x7fe3380b77b0 legacy=0x7fe3380d1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:35.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.165+0000 7fe33d600640 1 -- 172.21.15.86:0/1447327346 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3380b3a90 con 0x7fe3380b2130 2024-04-25T14:10:35.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.165+0000 7fe337e00640 1 --2- 172.21.15.86:0/1447327346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3380d1aa0 0x7fe3380b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:10:35.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.165+0000 7fe33d600640 1 --2- 172.21.15.86:0/1447327346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3380b2130 0x7fe3380b2540 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe330009d90 tx=0x7fe33001c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4518696176fca828 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:35.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.169+0000 7fe337400640 1 -- 172.21.15.86:0/1447327346 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3300b2070 con 0x7fe3380b2130 2024-04-25T14:10:35.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.169+0000 7fe337400640 1 -- 172.21.15.86:0/1447327346 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3300a5830 con 0x7fe3380b2130 2024-04-25T14:10:35.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.169+0000 7fe337400640 1 -- 172.21.15.86:0/1447327346 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3300a5c00 con 0x7fe3380b2130 2024-04-25T14:10:35.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.169+0000 7fe33e000640 1 -- 172.21.15.86:0/1447327346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3380b2130 msgr2=0x7fe3380b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:35.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.169+0000 7fe33e000640 1 --2- 172.21.15.86:0/1447327346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3380b2130 0x7fe3380b2540 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe330009d90 tx=0x7fe33001c970 comp rx=0 tx=0).stop 2024-04-25T14:10:35.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.169+0000 7fe33e000640 1 -- 172.21.15.86:0/1447327346 shutdown_connections 2024-04-25T14:10:35.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.169+0000 7fe33e000640 1 --2- 172.21.15.86:0/1447327346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3380b2130 0x7fe3380b2540 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:35.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.169+0000 7fe33e000640 1 --2- 172.21.15.86:0/1447327346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3380d1aa0 0x7fe3380b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:35.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.169+0000 7fe33e000640 1 -- 172.21.15.86:0/1447327346 >> 172.21.15.86:0/1447327346 conn(0x7fe33809ba30 msgr2=0x7fe3380b70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:35.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.169+0000 7fe33e000640 1 -- 172.21.15.86:0/1447327346 shutdown_connections 2024-04-25T14:10:35.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.169+0000 7fe33e000640 1 -- 172.21.15.86:0/1447327346 wait complete. 2024-04-25T14:10:35.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.169+0000 7fe33e000640 1 Processor -- start 2024-04-25T14:10:35.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.169+0000 7fe33e000640 1 -- start start 2024-04-25T14:10:35.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.173+0000 7fe33e000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3380b2130 0x7fe338157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:35.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.173+0000 7fe33e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3380b77b0 0x7fe33806d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:35.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.173+0000 7fe33e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3380d1aa0 0x7fe33806dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:35.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.173+0000 7fe33e000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe33816d4f0 con 0x7fe3380d1aa0 2024-04-25T14:10:35.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.173+0000 7fe33e000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe33816d370 con 0x7fe3380b77b0 2024-04-25T14:10:35.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.173+0000 7fe33e000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe33816d670 con 0x7fe3380b2130 2024-04-25T14:10:35.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.173+0000 7fe33d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3380d1aa0 0x7fe33806dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:35.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.173+0000 7fe33d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3380d1aa0 0x7fe33806dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57470/0 (socket says 172.21.15.86:57470) 2024-04-25T14:10:35.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.173+0000 7fe33d600640 1 -- 172.21.15.86:0/658794473 learned_addr learned my addr 172.21.15.86:0/658794473 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:35.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.173+0000 7fe33cc00640 1 --2- 172.21.15.86:0/658794473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3380b2130 0x7fe338157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:35.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.173+0000 7fe337e00640 1 --2- 172.21.15.86:0/658794473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3380b77b0 0x7fe33806d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:35.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.173+0000 7fe33d600640 1 -- 172.21.15.86:0/658794473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3380b2130 msgr2=0x7fe338157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:35.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.173+0000 7fe33d600640 1 --2- 172.21.15.86:0/658794473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3380b2130 0x7fe338157b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:35.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.173+0000 7fe33d600640 1 -- 172.21.15.86:0/658794473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3380b77b0 msgr2=0x7fe33806d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:35.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.173+0000 7fe33d600640 1 --2- 172.21.15.86:0/658794473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3380b77b0 0x7fe33806d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:35.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.173+0000 7fe33d600640 1 -- 172.21.15.86:0/658794473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe33815afb0 con 0x7fe3380d1aa0 2024-04-25T14:10:35.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.173+0000 7fe33d600640 1 --2- 172.21.15.86:0/658794473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3380d1aa0 0x7fe33806dd30 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fe33001b020 tx=0x7fe330019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:35.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.173+0000 7fe334c00640 1 -- 172.21.15.86:0/658794473 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3300b2050 con 0x7fe3380d1aa0 2024-04-25T14:10:35.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.173+0000 7fe334c00640 1 -- 172.21.15.86:0/658794473 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3300be070 con 0x7fe3380d1aa0 2024-04-25T14:10:35.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.173+0000 7fe33e000640 1 -- 172.21.15.86:0/658794473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe33815b290 con 0x7fe3380d1aa0 2024-04-25T14:10:35.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.173+0000 7fe334c00640 1 -- 172.21.15.86:0/658794473 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3300ad850 con 0x7fe3380d1aa0 2024-04-25T14:10:35.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.173+0000 7fe33e000640 1 -- 172.21.15.86:0/658794473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe33815b850 con 0x7fe3380d1aa0 2024-04-25T14:10:35.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.173+0000 7fe33e000640 1 -- 172.21.15.86:0/658794473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe33815c8b0 con 0x7fe3380d1aa0 2024-04-25T14:10:35.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.177+0000 7fe334c00640 1 -- 172.21.15.86:0/658794473 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7fe3300ad9f0 con 0x7fe3380d1aa0 2024-04-25T14:10:35.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.177+0000 7fe334c00640 1 --2- 172.21.15.86:0/658794473 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fe31806f490 0x7fe3180719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:35.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.177+0000 7fe334c00640 1 -- 172.21.15.86:0/658794473 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6832+0+0 (secure 0 0 0) 0x7fe33012a570 con 0x7fe3380d1aa0 2024-04-25T14:10:35.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.177+0000 7fe33cc00640 1 --2- 172.21.15.86:0/658794473 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fe31806f490 0x7fe3180719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:35.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.177+0000 7fe33cc00640 1 --2- 172.21.15.86:0/658794473 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fe31806f490 0x7fe3180719a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe32c007fd0 tx=0x7fe32c009040 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:35.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.181+0000 7fe334c00640 1 -- 172.21.15.86:0/658794473 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe330016560 con 0x7fe3380d1aa0 2024-04-25T14:10:35.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.353+0000 7fe33e000640 1 -- 172.21.15.86:0/658794473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe338076b80 con 0x7fe3380d1aa0 2024-04-25T14:10:35.852 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:35.853+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:35.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.981+0000 7fe334c00640 1 -- 172.21.15.86:0/658794473 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v117) ==== 181+0+0 (secure 0 0 0) 0x7fe3300f6a70 con 0x7fe3380d1aa0 2024-04-25T14:10:35.978 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-04-25T14:10:35.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.985+0000 7fe33e000640 1 -- 172.21.15.86:0/658794473 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fe31806f490 msgr2=0x7fe3180719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:35.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.985+0000 7fe33e000640 1 --2- 172.21.15.86:0/658794473 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fe31806f490 0x7fe3180719a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe32c007fd0 tx=0x7fe32c009040 comp rx=0 tx=0).stop 2024-04-25T14:10:35.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.985+0000 7fe33e000640 1 -- 172.21.15.86:0/658794473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3380d1aa0 msgr2=0x7fe33806dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:35.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.985+0000 7fe33e000640 1 --2- 172.21.15.86:0/658794473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3380d1aa0 0x7fe33806dd30 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fe33001b020 tx=0x7fe330019710 comp rx=0 tx=0).stop 2024-04-25T14:10:35.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.985+0000 7fe33e000640 1 -- 172.21.15.86:0/658794473 shutdown_connections 2024-04-25T14:10:35.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.985+0000 7fe33e000640 1 --2- 172.21.15.86:0/658794473 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fe31806f490 0x7fe3180719a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:35.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.985+0000 7fe33e000640 1 --2- 172.21.15.86:0/658794473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3380d1aa0 0x7fe33806dd30 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:35.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.985+0000 7fe33e000640 1 --2- 172.21.15.86:0/658794473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3380b77b0 0x7fe33806d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:35.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.985+0000 7fe33e000640 1 --2- 172.21.15.86:0/658794473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3380b2130 0x7fe338157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:35.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.985+0000 7fe33e000640 1 -- 172.21.15.86:0/658794473 >> 172.21.15.86:0/658794473 conn(0x7fe33809ba30 msgr2=0x7fe338070500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:35.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.985+0000 7fe33e000640 1 -- 172.21.15.86:0/658794473 shutdown_connections 2024-04-25T14:10:35.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:35.985+0000 7fe33e000640 1 -- 172.21.15.86:0/658794473 wait complete. 2024-04-25T14:10:36.005 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T14:10:36.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.169+0000 7f7832c00640 1 Processor -- start 2024-04-25T14:10:36.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.169+0000 7f7832c00640 1 -- start start 2024-04-25T14:10:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.173+0000 7f7832c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f782c06dd90 0x7f782c160020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.173+0000 7f7832c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f782c0b5160 0x7f782c0b5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.173+0000 7f7832c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f782c0705b0 con 0x7f782c15fc10 2024-04-25T14:10:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.173+0000 7f7832c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f782c1646d0 con 0x7f782c06dd90 2024-04-25T14:10:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.173+0000 7f7832c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f782c163be0 con 0x7f782c0b5160 2024-04-25T14:10:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.173+0000 7f782be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f782c15fc10 0x7f782c06d680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39948/0 (socket says 172.21.15.86:39948) 2024-04-25T14:10:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.173+0000 7f782be00640 1 -- 172.21.15.86:0/3765253250 learned_addr learned my addr 172.21.15.86:0/3765253250 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.173+0000 7f7830c00640 1 --2- 172.21.15.86:0/3765253250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f782c0b5160 0x7f782c0b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.173+0000 7f782b400640 1 --2- 172.21.15.86:0/3765253250 >> v2:172.21.15.155:3300/0 conn(0x7f782c06dd90 0x7f782c160020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:36.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.173+0000 7f782aa00640 1 -- 172.21.15.86:0/3765253250 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 653731900 0 0) 0x7f782c0705b0 con 0x7f782c15fc10 2024-04-25T14:10:36.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.173+0000 7f782aa00640 1 -- 172.21.15.86:0/3765253250 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7810003530 con 0x7f782c15fc10 2024-04-25T14:10:36.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.173+0000 7f7830c00640 1 -- 172.21.15.86:0/3765253250 >> v2:172.21.15.155:3300/0 conn(0x7f782c06dd90 msgr2=0x7f782c160020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:36.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.173+0000 7f7830c00640 1 --2- 172.21.15.86:0/3765253250 >> v2:172.21.15.155:3300/0 conn(0x7f782c06dd90 0x7f782c160020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.173+0000 7f7830c00640 1 -- 172.21.15.86:0/3765253250 >> v1:172.21.15.86:6789/0 conn(0x7f782c15fc10 legacy=0x7f782c06d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:36.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.173+0000 7f7830c00640 1 -- 172.21.15.86:0/3765253250 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f782c163e30 con 0x7f782c0b5160 2024-04-25T14:10:36.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.173+0000 7f782b400640 1 --2- 172.21.15.86:0/3765253250 >> v2:172.21.15.155:3300/0 conn(0x7f782c06dd90 0x7f782c160020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:10:36.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.177+0000 7f7830c00640 1 --2- 172.21.15.86:0/3765253250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f782c0b5160 0x7f782c0b5570 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7824009d90 tx=0x7f782401c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=540c104b77d4b522 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:36.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.177+0000 7f782aa00640 1 -- 172.21.15.86:0/3765253250 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78240b2070 con 0x7f782c0b5160 2024-04-25T14:10:36.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.177+0000 7f782aa00640 1 -- 172.21.15.86:0/3765253250 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78240a5890 con 0x7f782c0b5160 2024-04-25T14:10:36.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.177+0000 7f782aa00640 1 -- 172.21.15.86:0/3765253250 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78240a5c60 con 0x7f782c0b5160 2024-04-25T14:10:36.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.177+0000 7f7832c00640 1 -- 172.21.15.86:0/3765253250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f782c0b5160 msgr2=0x7f782c0b5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:36.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.177+0000 7f7832c00640 1 --2- 172.21.15.86:0/3765253250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f782c0b5160 0x7f782c0b5570 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7824009d90 tx=0x7f782401c8e0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.177+0000 7f7832c00640 1 -- 172.21.15.86:0/3765253250 shutdown_connections 2024-04-25T14:10:36.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.177+0000 7f7832c00640 1 --2- 172.21.15.86:0/3765253250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f782c0b5160 0x7f782c0b5570 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.177+0000 7f7832c00640 1 --2- 172.21.15.86:0/3765253250 >> v2:172.21.15.155:3300/0 conn(0x7f782c06dd90 0x7f782c160020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.177+0000 7f7832c00640 1 -- 172.21.15.86:0/3765253250 >> 172.21.15.86:0/3765253250 conn(0x7f782c073240 msgr2=0x7f782c073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:36.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.177+0000 7f7832c00640 1 -- 172.21.15.86:0/3765253250 shutdown_connections 2024-04-25T14:10:36.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.177+0000 7f7832c00640 1 -- 172.21.15.86:0/3765253250 wait complete. 2024-04-25T14:10:36.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.181+0000 7f7832c00640 1 Processor -- start 2024-04-25T14:10:36.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.181+0000 7f7832c00640 1 -- start start 2024-04-25T14:10:36.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.181+0000 7f7832c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f782c0b5160 0x7f782c1503f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:36.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.181+0000 7f7832c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f782c15fc10 0x7f782c0b1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:36.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.181+0000 7f7832c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f782c155610 0x7f782c15ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:36.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.181+0000 7f7832c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f782c16cce0 con 0x7f782c0b5160 2024-04-25T14:10:36.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.181+0000 7f7832c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f782c16cb60 con 0x7f782c15fc10 2024-04-25T14:10:36.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.181+0000 7f7832c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f782c16ce60 con 0x7f782c155610 2024-04-25T14:10:36.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.181+0000 7f782be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f782c0b5160 0x7f782c1503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:36.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.181+0000 7f782be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f782c0b5160 0x7f782c1503f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57478/0 (socket says 172.21.15.86:57478) 2024-04-25T14:10:36.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.181+0000 7f782be00640 1 -- 172.21.15.86:0/2243867290 learned_addr learned my addr 172.21.15.86:0/2243867290 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:36.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.181+0000 7f7830c00640 1 --2- 172.21.15.86:0/2243867290 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f782c155610 0x7f782c15ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:36.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.181+0000 7f782b400640 1 --2- 172.21.15.86:0/2243867290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f782c15fc10 0x7f782c0b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:36.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.181+0000 7f782be00640 1 -- 172.21.15.86:0/2243867290 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f782c155610 msgr2=0x7f782c15ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:36.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.181+0000 7f782be00640 1 --2- 172.21.15.86:0/2243867290 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f782c155610 0x7f782c15ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.181+0000 7f782be00640 1 -- 172.21.15.86:0/2243867290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f782c15fc10 msgr2=0x7f782c0b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:36.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.181+0000 7f782be00640 1 --2- 172.21.15.86:0/2243867290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f782c15fc10 0x7f782c0b1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.181+0000 7f782be00640 1 -- 172.21.15.86:0/2243867290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f782c150d60 con 0x7f782c0b5160 2024-04-25T14:10:36.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.181+0000 7f782be00640 1 --2- 172.21.15.86:0/2243867290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f782c0b5160 0x7f782c1503f0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f781c001210 tx=0x7f781c00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:36.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.181+0000 7f7823e00640 1 -- 172.21.15.86:0/2243867290 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c017070 con 0x7f782c0b5160 2024-04-25T14:10:36.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.181+0000 7f7823e00640 1 -- 172.21.15.86:0/2243867290 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f781c005d10 con 0x7f782c0b5160 2024-04-25T14:10:36.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.181+0000 7f7832c00640 1 -- 172.21.15.86:0/2243867290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f782c151040 con 0x7f782c0b5160 2024-04-25T14:10:36.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.181+0000 7f7823e00640 1 -- 172.21.15.86:0/2243867290 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c007310 con 0x7f782c0b5160 2024-04-25T14:10:36.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.185+0000 7f7832c00640 1 -- 172.21.15.86:0/2243867290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f782c151600 con 0x7f782c0b5160 2024-04-25T14:10:36.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.185+0000 7f7832c00640 1 -- 172.21.15.86:0/2243867290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f782c050750 con 0x7f782c0b5160 2024-04-25T14:10:36.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.185+0000 7f7823e00640 1 -- 172.21.15.86:0/2243867290 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7f781c01a070 con 0x7f782c0b5160 2024-04-25T14:10:36.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.185+0000 7f7823e00640 1 --2- 172.21.15.86:0/2243867290 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f780c06f490 0x7f780c0719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:36.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.185+0000 7f7823e00640 1 -- 172.21.15.86:0/2243867290 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6394+0+0 (secure 0 0 0) 0x7f781c08e950 con 0x7f782c0b5160 2024-04-25T14:10:36.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.185+0000 7f782b400640 1 --2- 172.21.15.86:0/2243867290 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f780c06f490 0x7f780c0719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:36.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.189+0000 7f782b400640 1 --2- 172.21.15.86:0/2243867290 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f780c06f490 0x7f780c0719a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f782c152f70 tx=0x7f7818009190 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:36.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.193+0000 7f7823e00640 1 -- 172.21.15.86:0/2243867290 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f781c05ae50 con 0x7f782c0b5160 2024-04-25T14:10:36.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.361+0000 7f7832c00640 1 -- 172.21.15.86:0/2243867290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f782c074150 con 0x7f782c0b5160 2024-04-25T14:10:36.592 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:36.593+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:36.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.993+0000 7f7823e00640 1 -- 172.21.15.86:0/2243867290 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v118) ==== 169+0+0 (secure 0 0 0) 0x7f781c05f150 con 0x7f782c0b5160 2024-04-25T14:10:36.990 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-04-25T14:10:36.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.997+0000 7f7832c00640 1 -- 172.21.15.86:0/2243867290 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f780c06f490 msgr2=0x7f780c0719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:36.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.997+0000 7f7832c00640 1 --2- 172.21.15.86:0/2243867290 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f780c06f490 0x7f780c0719a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f782c152f70 tx=0x7f7818009190 comp rx=0 tx=0).stop 2024-04-25T14:10:36.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.997+0000 7f7832c00640 1 -- 172.21.15.86:0/2243867290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f782c0b5160 msgr2=0x7f782c1503f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:36.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.997+0000 7f7832c00640 1 --2- 172.21.15.86:0/2243867290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f782c0b5160 0x7f782c1503f0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f781c001210 tx=0x7f781c00c820 comp rx=0 tx=0).stop 2024-04-25T14:10:36.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.997+0000 7f7832c00640 1 -- 172.21.15.86:0/2243867290 shutdown_connections 2024-04-25T14:10:36.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.997+0000 7f7832c00640 1 --2- 172.21.15.86:0/2243867290 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f782c155610 0x7f782c15ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.997+0000 7f7832c00640 1 --2- 172.21.15.86:0/2243867290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f782c15fc10 0x7f782c0b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.997+0000 7f7832c00640 1 --2- 172.21.15.86:0/2243867290 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f780c06f490 0x7f780c0719a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.997+0000 7f7832c00640 1 --2- 172.21.15.86:0/2243867290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f782c0b5160 0x7f782c1503f0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.997+0000 7f7832c00640 1 -- 172.21.15.86:0/2243867290 >> 172.21.15.86:0/2243867290 conn(0x7f782c073240 msgr2=0x7f782c0afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:36.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.997+0000 7f7832c00640 1 -- 172.21.15.86:0/2243867290 shutdown_connections 2024-04-25T14:10:36.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:36.997+0000 7f7832c00640 1 -- 172.21.15.86:0/2243867290 wait complete. 2024-04-25T14:10:37.020 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T14:10:37.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.181+0000 7f0beee00640 1 Processor -- start 2024-04-25T14:10:37.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.181+0000 7f0beee00640 1 -- start start 2024-04-25T14:10:37.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.181+0000 7f0beee00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0be8071530 0x7f0be80d15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.181+0000 7f0beee00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0be80d1af0 0x7f0be80d1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.181+0000 7f0beee00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0be8161770 con 0x7f0be80ce5a0 2024-04-25T14:10:37.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.181+0000 7f0beee00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0be8162380 con 0x7f0be80d1af0 2024-04-25T14:10:37.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.181+0000 7f0beee00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0be8161a50 con 0x7f0be8071530 2024-04-25T14:10:37.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.181+0000 7f0bece00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0be80d1af0 0x7f0be80d1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.181+0000 7f0be7e00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f0be80ce5a0 0x7f0be8070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:52926/0 (socket says 172.21.15.86:52926) 2024-04-25T14:10:37.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.181+0000 7f0be7e00640 1 -- 172.21.15.86:0/2720076284 learned_addr learned my addr 172.21.15.86:0/2720076284 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:37.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.181+0000 7f0bdf400640 1 --2- 172.21.15.86:0/2720076284 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0be8071530 0x7f0be80d15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.181+0000 7f0bece00640 1 -- 172.21.15.86:0/2720076284 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0be8071530 msgr2=0x7f0be80d15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:37.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.181+0000 7f0bece00640 1 --2- 172.21.15.86:0/2720076284 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0be8071530 0x7f0be80d15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.181+0000 7f0bece00640 1 -- 172.21.15.86:0/2720076284 >> v1:172.21.15.155:6789/0 conn(0x7f0be80ce5a0 legacy=0x7f0be8070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.181+0000 7f0bece00640 1 -- 172.21.15.86:0/2720076284 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be8161d30 con 0x7f0be80d1af0 2024-04-25T14:10:37.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.181+0000 7f0be7400640 1 -- 172.21.15.86:0/2720076284 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 498232055 0 0) 0x7f0be8161770 con 0x7f0be80ce5a0 2024-04-25T14:10:37.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.185+0000 7f0bece00640 1 --2- 172.21.15.86:0/2720076284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be80d1af0 0x7f0be80d1f00 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f0be0009d90 tx=0x7f0be001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c3998a441f0618 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:37.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.185+0000 7f0be7400640 1 -- 172.21.15.86:0/2720076284 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be00b2070 con 0x7f0be80d1af0 2024-04-25T14:10:37.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.185+0000 7f0be7400640 1 -- 172.21.15.86:0/2720076284 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0be00a5a00 con 0x7f0be80d1af0 2024-04-25T14:10:37.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.185+0000 7f0be7400640 1 -- 172.21.15.86:0/2720076284 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be00a5dd0 con 0x7f0be80d1af0 2024-04-25T14:10:37.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.185+0000 7f0beee00640 1 -- 172.21.15.86:0/2720076284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be80d1af0 msgr2=0x7f0be80d1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.185+0000 7f0beee00640 1 --2- 172.21.15.86:0/2720076284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be80d1af0 0x7f0be80d1f00 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f0be0009d90 tx=0x7f0be001c940 comp rx=0 tx=0).stop 2024-04-25T14:10:37.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.185+0000 7f0beee00640 1 -- 172.21.15.86:0/2720076284 shutdown_connections 2024-04-25T14:10:37.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.185+0000 7f0beee00640 1 --2- 172.21.15.86:0/2720076284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be80d1af0 0x7f0be80d1f00 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.185+0000 7f0beee00640 1 --2- 172.21.15.86:0/2720076284 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0be8071530 0x7f0be80d15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.185+0000 7f0beee00640 1 -- 172.21.15.86:0/2720076284 >> 172.21.15.86:0/2720076284 conn(0x7f0be80cffc0 msgr2=0x7f0be80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:37.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.185+0000 7f0beee00640 1 -- 172.21.15.86:0/2720076284 shutdown_connections 2024-04-25T14:10:37.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.185+0000 7f0beee00640 1 -- 172.21.15.86:0/2720076284 wait complete. 2024-04-25T14:10:37.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.189+0000 7f0beee00640 1 Processor -- start 2024-04-25T14:10:37.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.189+0000 7f0beee00640 1 -- start start 2024-04-25T14:10:37.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.189+0000 7f0beee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be80ce5a0 0x7f0be8154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.189+0000 7f0beee00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0be80d1af0 0x7f0be80caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.189+0000 7f0beee00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0be80c8c20 0x7f0be80c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.189+0000 7f0beee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0be80ceb30 con 0x7f0be80ce5a0 2024-04-25T14:10:37.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.189+0000 7f0beee00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0be80ce9b0 con 0x7f0be80c8c20 2024-04-25T14:10:37.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.189+0000 7f0beee00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0be81583d0 con 0x7f0be80d1af0 2024-04-25T14:10:37.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.189+0000 7f0be7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be80ce5a0 0x7f0be8154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.189+0000 7f0be7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be80ce5a0 0x7f0be8154020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57492/0 (socket says 172.21.15.86:57492) 2024-04-25T14:10:37.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.189+0000 7f0be7e00640 1 -- 172.21.15.86:0/978561194 learned_addr learned my addr 172.21.15.86:0/978561194 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:37.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.189+0000 7f0bdf400640 1 --2- 172.21.15.86:0/978561194 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0be80d1af0 0x7f0be80caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.189+0000 7f0bece00640 1 --2- 172.21.15.86:0/978561194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0be80c8c20 0x7f0be80c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.193+0000 7f0be7e00640 1 -- 172.21.15.86:0/978561194 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0be80d1af0 msgr2=0x7f0be80caf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.193+0000 7f0be7e00640 1 --2- 172.21.15.86:0/978561194 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0be80d1af0 0x7f0be80caf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.193+0000 7f0be7e00640 1 -- 172.21.15.86:0/978561194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0be80c8c20 msgr2=0x7f0be80c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.193+0000 7f0be7e00640 1 --2- 172.21.15.86:0/978561194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0be80c8c20 0x7f0be80c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.193+0000 7f0be7e00640 1 -- 172.21.15.86:0/978561194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be8168890 con 0x7f0be80ce5a0 2024-04-25T14:10:37.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.193+0000 7f0be7e00640 1 --2- 172.21.15.86:0/978561194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be80ce5a0 0x7f0be8154020 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800c440 tx=0x7f0bd800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:37.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.193+0000 7f0be4c00640 1 -- 172.21.15.86:0/978561194 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8018070 con 0x7f0be80ce5a0 2024-04-25T14:10:37.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.193+0000 7f0be4c00640 1 -- 172.21.15.86:0/978561194 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bd80074e0 con 0x7f0be80ce5a0 2024-04-25T14:10:37.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.193+0000 7f0be4c00640 1 -- 172.21.15.86:0/978561194 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8004a30 con 0x7f0be80ce5a0 2024-04-25T14:10:37.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.193+0000 7f0beee00640 1 -- 172.21.15.86:0/978561194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be8155240 con 0x7f0be80ce5a0 2024-04-25T14:10:37.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.193+0000 7f0beee00640 1 -- 172.21.15.86:0/978561194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be80b6160 con 0x7f0be80ce5a0 2024-04-25T14:10:37.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.193+0000 7f0beee00640 1 -- 172.21.15.86:0/978561194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0be80720e0 con 0x7f0be80ce5a0 2024-04-25T14:10:37.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.193+0000 7f0be4c00640 1 -- 172.21.15.86:0/978561194 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7f0bd8004bd0 con 0x7f0be80ce5a0 2024-04-25T14:10:37.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.197+0000 7f0be4c00640 1 --2- 172.21.15.86:0/978561194 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f0bc406f490 0x7f0bc40719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.197+0000 7f0be4c00640 1 -- 172.21.15.86:0/978561194 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5980+0+0 (secure 0 0 0) 0x7f0bd808e650 con 0x7f0be80ce5a0 2024-04-25T14:10:37.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.197+0000 7f0bdf400640 1 --2- 172.21.15.86:0/978561194 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f0bc406f490 0x7f0bc40719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.197+0000 7f0bdf400640 1 --2- 172.21.15.86:0/978561194 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f0bc406f490 0x7f0bc40719a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0be8157e00 tx=0x7f0bd0009190 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:37.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.201+0000 7f0be4c00640 1 -- 172.21.15.86:0/978561194 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0bd805ae10 con 0x7f0be80ce5a0 2024-04-25T14:10:37.322 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:37.325+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:37.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.373+0000 7f0beee00640 1 -- 172.21.15.86:0/978561194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f0be8076bf0 con 0x7f0be80ce5a0 2024-04-25T14:10:37.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.377+0000 7f0be4c00640 1 -- 172.21.15.86:0/978561194 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f0bd805f110 con 0x7f0be80ce5a0 2024-04-25T14:10:37.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.381+0000 7f0beee00640 1 -- 172.21.15.86:0/978561194 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f0bc406f490 msgr2=0x7f0bc40719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.381+0000 7f0beee00640 1 --2- 172.21.15.86:0/978561194 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f0bc406f490 0x7f0bc40719a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0be8157e00 tx=0x7f0bd0009190 comp rx=0 tx=0).stop 2024-04-25T14:10:37.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.381+0000 7f0beee00640 1 -- 172.21.15.86:0/978561194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be80ce5a0 msgr2=0x7f0be8154020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.381+0000 7f0beee00640 1 --2- 172.21.15.86:0/978561194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be80ce5a0 0x7f0be8154020 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800c440 tx=0x7f0bd800c900 comp rx=0 tx=0).stop 2024-04-25T14:10:37.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.381+0000 7f0beee00640 1 -- 172.21.15.86:0/978561194 shutdown_connections 2024-04-25T14:10:37.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.381+0000 7f0beee00640 1 --2- 172.21.15.86:0/978561194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0be80c8c20 0x7f0be80c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.381+0000 7f0beee00640 1 --2- 172.21.15.86:0/978561194 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0be80d1af0 0x7f0be80caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.381+0000 7f0beee00640 1 --2- 172.21.15.86:0/978561194 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f0bc406f490 0x7f0bc40719a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.381+0000 7f0beee00640 1 --2- 172.21.15.86:0/978561194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be80ce5a0 0x7f0be8154020 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.381+0000 7f0beee00640 1 -- 172.21.15.86:0/978561194 >> 172.21.15.86:0/978561194 conn(0x7f0be80cffc0 msgr2=0x7f0be80b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:37.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.381+0000 7f0beee00640 1 -- 172.21.15.86:0/978561194 shutdown_connections 2024-04-25T14:10:37.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.381+0000 7f0beee00640 1 -- 172.21.15.86:0/978561194 wait complete. 2024-04-25T14:10:37.399 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_for_invalid_mds_rank' 2024-04-25T14:10:37.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.553+0000 7f48cc600640 1 Processor -- start 2024-04-25T14:10:37.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.553+0000 7f48cc600640 1 -- start start 2024-04-25T14:10:37.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.557+0000 7f48cc600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f48c40cfca0 0x7f48c406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.557+0000 7f48cc600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f48c40d16e0 0x7f48c40d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.557+0000 7f48cc600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48c406c670 0x7f48c406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.557+0000 7f48cc600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f48c4070470 con 0x7f48c40d16e0 2024-04-25T14:10:37.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.557+0000 7f48cc600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f48c40d2030 con 0x7f48c40cfca0 2024-04-25T14:10:37.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.557+0000 7f48cc600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f48c4072bd0 con 0x7f48c406c670 2024-04-25T14:10:37.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.557+0000 7f48c9200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f48c40d16e0 0x7f48c40d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.557+0000 7f48c9200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f48c40d16e0 0x7f48c40d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57498/0 (socket says 172.21.15.86:57498) 2024-04-25T14:10:37.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.557+0000 7f48c9200640 1 -- 172.21.15.86:0/299508086 learned_addr learned my addr 172.21.15.86:0/299508086 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:37.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.557+0000 7f48ca600640 1 --2- 172.21.15.86:0/299508086 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48c406c670 0x7f48c406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.557+0000 7f48c9c00640 1 --2- 172.21.15.86:0/299508086 >> v2:172.21.15.155:3300/0 conn(0x7f48c40cfca0 0x7f48c406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.557+0000 7f48c9200640 1 -- 172.21.15.86:0/299508086 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48c406c670 msgr2=0x7f48c406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:37.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.557+0000 7f48c9200640 1 --2- 172.21.15.86:0/299508086 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48c406c670 0x7f48c406ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.557+0000 7f48c9200640 1 -- 172.21.15.86:0/299508086 >> v2:172.21.15.155:3300/0 conn(0x7f48c40cfca0 msgr2=0x7f48c406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:37.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.557+0000 7f48c9200640 1 --2- 172.21.15.86:0/299508086 >> v2:172.21.15.155:3300/0 conn(0x7f48c40cfca0 0x7f48c406f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.557+0000 7f48c9200640 1 -- 172.21.15.86:0/299508086 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c406f930 con 0x7f48c40d16e0 2024-04-25T14:10:37.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.561+0000 7f48c9200640 1 --2- 172.21.15.86:0/299508086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48c40d16e0 0x7f48c40d1af0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f48c40d22a0 tx=0x7f48b001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d106221291ee63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:37.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.561+0000 7f48c3e00640 1 -- 172.21.15.86:0/299508086 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b00ab5f0 con 0x7f48c40d16e0 2024-04-25T14:10:37.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.561+0000 7f48c3e00640 1 -- 172.21.15.86:0/299508086 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48b00b0070 con 0x7f48c40d16e0 2024-04-25T14:10:37.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.561+0000 7f48c3e00640 1 -- 172.21.15.86:0/299508086 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b00ab9b0 con 0x7f48c40d16e0 2024-04-25T14:10:37.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.561+0000 7f48cc600640 1 -- 172.21.15.86:0/299508086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48c40d16e0 msgr2=0x7f48c40d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.561+0000 7f48cc600640 1 --2- 172.21.15.86:0/299508086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48c40d16e0 0x7f48c40d1af0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f48c40d22a0 tx=0x7f48b001c520 comp rx=0 tx=0).stop 2024-04-25T14:10:37.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.561+0000 7f48cc600640 1 -- 172.21.15.86:0/299508086 shutdown_connections 2024-04-25T14:10:37.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.561+0000 7f48cc600640 1 --2- 172.21.15.86:0/299508086 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48c406c670 0x7f48c406ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.561+0000 7f48cc600640 1 --2- 172.21.15.86:0/299508086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48c40d16e0 0x7f48c40d1af0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.561+0000 7f48cc600640 1 --2- 172.21.15.86:0/299508086 >> v2:172.21.15.155:3300/0 conn(0x7f48c40cfca0 0x7f48c406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.561+0000 7f48cc600640 1 -- 172.21.15.86:0/299508086 >> 172.21.15.86:0/299508086 conn(0x7f48c40b4650 msgr2=0x7f48c40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:37.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.561+0000 7f48cc600640 1 -- 172.21.15.86:0/299508086 shutdown_connections 2024-04-25T14:10:37.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.561+0000 7f48cc600640 1 -- 172.21.15.86:0/299508086 wait complete. 2024-04-25T14:10:37.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.565+0000 7f48cc600640 1 Processor -- start 2024-04-25T14:10:37.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.565+0000 7f48cc600640 1 -- start start 2024-04-25T14:10:37.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.565+0000 7f48cc600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48c406c670 0x7f48c409a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.565+0000 7f48cc600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48c40cfca0 0x7f48c415bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.565+0000 7f48cc600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48c40d16e0 0x7f48c415c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.565+0000 7f48cc600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f48c40738c0 con 0x7f48c40cfca0 2024-04-25T14:10:37.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.565+0000 7f48cc600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f48c406d3d0 con 0x7f48c406c670 2024-04-25T14:10:37.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.565+0000 7f48cc600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f48c4073a40 con 0x7f48c40d16e0 2024-04-25T14:10:37.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.565+0000 7f48c9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48c40cfca0 0x7f48c415bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.565+0000 7f48c9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48c40cfca0 0x7f48c415bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57508/0 (socket says 172.21.15.86:57508) 2024-04-25T14:10:37.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.565+0000 7f48c9200640 1 -- 172.21.15.86:0/142683973 learned_addr learned my addr 172.21.15.86:0/142683973 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:37.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.565+0000 7f48c9c00640 1 --2- 172.21.15.86:0/142683973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48c406c670 0x7f48c409a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.565+0000 7f48ca600640 1 --2- 172.21.15.86:0/142683973 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48c40d16e0 0x7f48c415c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.565+0000 7f48c9200640 1 -- 172.21.15.86:0/142683973 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48c40d16e0 msgr2=0x7f48c415c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.565+0000 7f48c9200640 1 --2- 172.21.15.86:0/142683973 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48c40d16e0 0x7f48c415c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.565+0000 7f48c9200640 1 -- 172.21.15.86:0/142683973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48c406c670 msgr2=0x7f48c409a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.565+0000 7f48c9200640 1 --2- 172.21.15.86:0/142683973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48c406c670 0x7f48c409a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.565+0000 7f48c9200640 1 -- 172.21.15.86:0/142683973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c415a1f0 con 0x7f48c40cfca0 2024-04-25T14:10:37.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.565+0000 7f48ca600640 1 --2- 172.21.15.86:0/142683973 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48c40d16e0 0x7f48c415c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:37.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.565+0000 7f48c9c00640 1 --2- 172.21.15.86:0/142683973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48c406c670 0x7f48c409a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:37.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.565+0000 7f48c9200640 1 --2- 172.21.15.86:0/142683973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48c40cfca0 0x7f48c415bea0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f48b001abd0 tx=0x7f48b001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:37.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.565+0000 7f48c1600640 1 -- 172.21.15.86:0/142683973 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b00be3f0 con 0x7f48c40cfca0 2024-04-25T14:10:37.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.565+0000 7f48c1600640 1 -- 172.21.15.86:0/142683973 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48b00be590 con 0x7f48c40cfca0 2024-04-25T14:10:37.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.565+0000 7f48c1600640 1 -- 172.21.15.86:0/142683973 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b00be980 con 0x7f48c40cfca0 2024-04-25T14:10:37.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.565+0000 7f48cc600640 1 -- 172.21.15.86:0/142683973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48c415a490 con 0x7f48c40cfca0 2024-04-25T14:10:37.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.565+0000 7f48cc600640 1 -- 172.21.15.86:0/142683973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48c4152ea0 con 0x7f48c40cfca0 2024-04-25T14:10:37.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.569+0000 7f48cc600640 1 -- 172.21.15.86:0/142683973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48c4152bf0 con 0x7f48c40cfca0 2024-04-25T14:10:37.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.569+0000 7f48c1600640 1 -- 172.21.15.86:0/142683973 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7f48b00b43f0 con 0x7f48c40cfca0 2024-04-25T14:10:37.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.569+0000 7f48c1600640 1 --2- 172.21.15.86:0/142683973 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f48a406f490 0x7f48a40719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.569+0000 7f48c1600640 1 -- 172.21.15.86:0/142683973 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5980+0+0 (secure 0 0 0) 0x7f48b01293d0 con 0x7f48c40cfca0 2024-04-25T14:10:37.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.569+0000 7f48c9c00640 1 --2- 172.21.15.86:0/142683973 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f48a406f490 0x7f48a40719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.569+0000 7f48c9c00640 1 --2- 172.21.15.86:0/142683973 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f48a406f490 0x7f48a40719a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f48c4070710 tx=0x7f48b8008040 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:37.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.573+0000 7f48c1600640 1 -- 172.21.15.86:0/142683973 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f48b0016110 con 0x7f48c40cfca0 2024-04-25T14:10:37.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.801+0000 7f48cc600640 1 -- 172.21.15.86:0/142683973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_for_invalid_mds_rank"]} v 0) -- 0x7f48c409f4b0 con 0x7f48c40cfca0 2024-04-25T14:10:37.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.993+0000 7f48c1600640 1 -- 172.21.15.86:0/142683973 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_for_invalid_mds_rank"]}]=0 v386) ==== 149+0+0 (secure 0 0 0) 0x7f48b00fe150 con 0x7f48c40cfca0 2024-04-25T14:10:37.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.997+0000 7f48cc600640 1 -- 172.21.15.86:0/142683973 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f48a406f490 msgr2=0x7f48a40719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.997+0000 7f48cc600640 1 --2- 172.21.15.86:0/142683973 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f48a406f490 0x7f48a40719a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f48c4070710 tx=0x7f48b8008040 comp rx=0 tx=0).stop 2024-04-25T14:10:37.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.997+0000 7f48cc600640 1 -- 172.21.15.86:0/142683973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48c40cfca0 msgr2=0x7f48c415bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.997+0000 7f48cc600640 1 --2- 172.21.15.86:0/142683973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48c40cfca0 0x7f48c415bea0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f48b001abd0 tx=0x7f48b001ad60 comp rx=0 tx=0).stop 2024-04-25T14:10:37.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.997+0000 7f48cc600640 1 -- 172.21.15.86:0/142683973 shutdown_connections 2024-04-25T14:10:37.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.997+0000 7f48cc600640 1 --2- 172.21.15.86:0/142683973 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48c40d16e0 0x7f48c415c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.997+0000 7f48cc600640 1 --2- 172.21.15.86:0/142683973 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f48a406f490 0x7f48a40719a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.997+0000 7f48cc600640 1 --2- 172.21.15.86:0/142683973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48c40cfca0 0x7f48c415bea0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.997+0000 7f48cc600640 1 --2- 172.21.15.86:0/142683973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48c406c670 0x7f48c409a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.997+0000 7f48cc600640 1 -- 172.21.15.86:0/142683973 >> 172.21.15.86:0/142683973 conn(0x7f48c40b4650 msgr2=0x7f48c41566a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:37.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.997+0000 7f48cc600640 1 -- 172.21.15.86:0/142683973 shutdown_connections 2024-04-25T14:10:37.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:37.997+0000 7f48cc600640 1 -- 172.21.15.86:0/142683973 wait complete. 2024-04-25T14:10:38.015 INFO:tasks.cephfs_test_runner:test_for_invalid_mds_rank (tasks.cephfs.test_mds_metrics.TestMDSMetrics) ... ok 2024-04-25T14:10:38.016 INFO:tasks.cephfs_test_runner:Starting test: test_metrics_from_rank (tasks.cephfs.test_mds_metrics.TestMDSMetrics) 2024-04-25T14:10:38.016 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:10:38.049 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:38.053+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:38.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.169+0000 7fab7de00640 1 Processor -- start 2024-04-25T14:10:38.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.169+0000 7fab7de00640 1 -- start start 2024-04-25T14:10:38.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.169+0000 7fab7de00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab78070330 0x7fab780d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.169+0000 7fab7de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fab780d1c20 0x7fab7806ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.169+0000 7fab7de00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab780d2030 con 0x7fab7806f270 2024-04-25T14:10:38.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.169+0000 7fab7de00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fab7815e5c0 con 0x7fab780d1c20 2024-04-25T14:10:38.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.169+0000 7fab7de00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fab7815e8a0 con 0x7fab78070330 2024-04-25T14:10:38.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.169+0000 7fab76000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fab780d1c20 0x7fab7806ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.169+0000 7fab76000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fab780d1c20 0x7fab7806ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57516/0 (socket says 172.21.15.86:57516) 2024-04-25T14:10:38.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.169+0000 7fab76000640 1 -- 172.21.15.86:0/2632177474 learned_addr learned my addr 172.21.15.86:0/2632177474 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:38.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.169+0000 7fab76a00640 1 --2- 172.21.15.86:0/2632177474 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab78070330 0x7fab780d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.169+0000 7fab76000640 1 -- 172.21.15.86:0/2632177474 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab78070330 msgr2=0x7fab780d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:38.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.169+0000 7fab76000640 1 --2- 172.21.15.86:0/2632177474 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab78070330 0x7fab780d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.169+0000 7fab76000640 1 -- 172.21.15.86:0/2632177474 >> v1:172.21.15.155:6789/0 conn(0x7fab7806f270 legacy=0x7fab7806f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.169+0000 7fab76000640 1 -- 172.21.15.86:0/2632177474 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7815eb80 con 0x7fab780d1c20 2024-04-25T14:10:38.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.169+0000 7fab75600640 1 -- 172.21.15.86:0/2632177474 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1052140582 0 0) 0x7fab700014a0 con 0x7fab7806f270 2024-04-25T14:10:38.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.173+0000 7fab76000640 1 --2- 172.21.15.86:0/2632177474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab780d1c20 0x7fab7806ed30 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fab640098a0 tx=0x7fab6401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ace02382ab1f82b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:38.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.173+0000 7fab75600640 1 -- 172.21.15.86:0/2632177474 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab640b0070 con 0x7fab780d1c20 2024-04-25T14:10:38.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.173+0000 7fab75600640 1 -- 172.21.15.86:0/2632177474 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab640ab600 con 0x7fab780d1c20 2024-04-25T14:10:38.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.173+0000 7fab75600640 1 -- 172.21.15.86:0/2632177474 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab640ab9d0 con 0x7fab780d1c20 2024-04-25T14:10:38.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.173+0000 7fab7de00640 1 -- 172.21.15.86:0/2632177474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab780d1c20 msgr2=0x7fab7806ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.173+0000 7fab7de00640 1 --2- 172.21.15.86:0/2632177474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab780d1c20 0x7fab7806ed30 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fab640098a0 tx=0x7fab6401c4b0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.173+0000 7fab7de00640 1 -- 172.21.15.86:0/2632177474 shutdown_connections 2024-04-25T14:10:38.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.173+0000 7fab7de00640 1 --2- 172.21.15.86:0/2632177474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab780d1c20 0x7fab7806ed30 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.173+0000 7fab7de00640 1 --2- 172.21.15.86:0/2632177474 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab78070330 0x7fab780d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.173+0000 7fab7de00640 1 -- 172.21.15.86:0/2632177474 >> 172.21.15.86:0/2632177474 conn(0x7fab780b4650 msgr2=0x7fab780b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:38.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.173+0000 7fab7de00640 1 -- 172.21.15.86:0/2632177474 shutdown_connections 2024-04-25T14:10:38.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.173+0000 7fab7de00640 1 -- 172.21.15.86:0/2632177474 wait complete. 2024-04-25T14:10:38.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.177+0000 7fab7de00640 1 Processor -- start 2024-04-25T14:10:38.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.177+0000 7fab7de00640 1 -- start start 2024-04-25T14:10:38.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.177+0000 7fab7de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab7806f270 0x7fab7815ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.177+0000 7fab7de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab78070330 0x7fab78158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.177+0000 7fab7de00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab780d1c20 0x7fab781585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.177+0000 7fab7de00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fab780708c0 con 0x7fab78070330 2024-04-25T14:10:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.177+0000 7fab7de00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fab78070740 con 0x7fab7806f270 2024-04-25T14:10:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.177+0000 7fab7de00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fab7816afc0 con 0x7fab780d1c20 2024-04-25T14:10:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.177+0000 7fab76000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab78070330 0x7fab78158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.177+0000 7fab76000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab78070330 0x7fab78158090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57526/0 (socket says 172.21.15.86:57526) 2024-04-25T14:10:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.177+0000 7fab76000640 1 -- 172.21.15.86:0/2101648405 learned_addr learned my addr 172.21.15.86:0/2101648405 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.177+0000 7fab76a00640 1 --2- 172.21.15.86:0/2101648405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab7806f270 0x7fab7815ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.177+0000 7fab77400640 1 --2- 172.21.15.86:0/2101648405 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab780d1c20 0x7fab781585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.177+0000 7fab76a00640 1 -- 172.21.15.86:0/2101648405 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab780d1c20 msgr2=0x7fab781585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.177+0000 7fab76a00640 1 --2- 172.21.15.86:0/2101648405 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab780d1c20 0x7fab781585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.177+0000 7fab76a00640 1 -- 172.21.15.86:0/2101648405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab78070330 msgr2=0x7fab78158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.177+0000 7fab76a00640 1 --2- 172.21.15.86:0/2101648405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab78070330 0x7fab78158090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.177+0000 7fab76a00640 1 -- 172.21.15.86:0/2101648405 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab780cf2f0 con 0x7fab7806f270 2024-04-25T14:10:38.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.181+0000 7fab76000640 1 --2- 172.21.15.86:0/2101648405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab78070330 0x7fab78158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:10:38.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.181+0000 7fab76a00640 1 --2- 172.21.15.86:0/2101648405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab7806f270 0x7fab7815ac10 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fab6800e8a0 tx=0x7fab6800ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:38.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.181+0000 7fab6f400640 1 -- 172.21.15.86:0/2101648405 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab68017070 con 0x7fab7806f270 2024-04-25T14:10:38.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.181+0000 7fab6f400640 1 -- 172.21.15.86:0/2101648405 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab6800c940 con 0x7fab7806f270 2024-04-25T14:10:38.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.181+0000 7fab7de00640 1 -- 172.21.15.86:0/2101648405 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab780d0ea0 con 0x7fab7806f270 2024-04-25T14:10:38.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.181+0000 7fab6f400640 1 -- 172.21.15.86:0/2101648405 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6800cd10 con 0x7fab7806f270 2024-04-25T14:10:38.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.181+0000 7fab7de00640 1 -- 172.21.15.86:0/2101648405 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab78168490 con 0x7fab7806f270 2024-04-25T14:10:38.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.181+0000 7fab7de00640 1 -- 172.21.15.86:0/2101648405 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab78069d00 con 0x7fab7806f270 2024-04-25T14:10:38.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.181+0000 7fab6f400640 1 -- 172.21.15.86:0/2101648405 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7fab6801a070 con 0x7fab7806f270 2024-04-25T14:10:38.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.185+0000 7fab6f400640 1 --2- 172.21.15.86:0/2101648405 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fab5806f4c0 0x7fab580719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.185+0000 7fab6f400640 1 -- 172.21.15.86:0/2101648405 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5980+0+0 (secure 0 0 0) 0x7fab6808f280 con 0x7fab7806f270 2024-04-25T14:10:38.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.185+0000 7fab76000640 1 --2- 172.21.15.86:0/2101648405 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fab5806f4c0 0x7fab580719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.185+0000 7fab76000640 1 --2- 172.21.15.86:0/2101648405 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fab5806f4c0 0x7fab580719d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fab64000c10 tx=0x7fab640a6000 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:38.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.185+0000 7fab6f400640 1 -- 172.21.15.86:0/2101648405 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fab6805b9a0 con 0x7fab7806f270 2024-04-25T14:10:38.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.349+0000 7fab7de00640 1 -- 172.21.15.86:0/2101648405 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab78076bf0 con 0x7fab7806f270 2024-04-25T14:10:38.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.349+0000 7fab6f400640 1 -- 172.21.15.86:0/2101648405 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+14241 (secure 0 0 0) 0x7fab6805fca0 con 0x7fab7806f270 2024-04-25T14:10:38.348 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:38.348 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":118,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:10:36.993713+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.155:6832/142240252":"2024-04-26T14:10:33.978883+0000","172.21.15.86:0/1062432465":"2024-04-26T14:10:29.591964+0000","172.21.15.86:6832/190090395":"2024-04-26T14:10:29.591964+0000","172.21.15.86:0/1120094610":"2024-04-26T14:10:29.591964+0000","172.21.15.86:0/2412030790":"2024-04-26T14:10:29.591964+0000","172.21.15.86:0/4291479632":"2024-04-26T14:10:29.591964+0000","172.21.15.86:0/2052329939":"2024-04-26T14:10:23.454991+0000","172.21.15.155:6833/142240252":"2024-04-26T14:10:33.978883+0000","172.21.15.86:0/1021944523":"2024-04-26T14:10:23.454991+0000","172.21.15.86:0/83615604":"2024-04-26T14:10:23.454991+0000","172.21.15.86:0/3719613294":"2024-04-26T14:10:23.454991+0000","172.21.15.86:0/3734007180":"2024-04-26T14:10:29.591964+0000","172.21.15.86:0/1471025278":"2024-04-26T14:10:23.454991+0000","172.21.15.86:6833/1354467416":"2024-04-26T14:10:23.454991+0000","172.21.15.86:6832/1354467416":"2024-04-26T14:10:23.454991+0000","172.21.15.86:6833/190090395":"2024-04-26T14:10:29.591964+0000","172.21.15.86:0/3000491277":"2024-04-26T14:10:23.454991+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:10:38.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.353+0000 7fab7de00640 1 -- 172.21.15.86:0/2101648405 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fab5806f4c0 msgr2=0x7fab580719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.353+0000 7fab7de00640 1 --2- 172.21.15.86:0/2101648405 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fab5806f4c0 0x7fab580719d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fab64000c10 tx=0x7fab640a6000 comp rx=0 tx=0).stop 2024-04-25T14:10:38.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.353+0000 7fab7de00640 1 -- 172.21.15.86:0/2101648405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab7806f270 msgr2=0x7fab7815ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.353+0000 7fab7de00640 1 --2- 172.21.15.86:0/2101648405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab7806f270 0x7fab7815ac10 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fab6800e8a0 tx=0x7fab6800ed60 comp rx=0 tx=0).stop 2024-04-25T14:10:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.353+0000 7fab7de00640 1 -- 172.21.15.86:0/2101648405 shutdown_connections 2024-04-25T14:10:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.353+0000 7fab7de00640 1 --2- 172.21.15.86:0/2101648405 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab780d1c20 0x7fab781585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.353+0000 7fab7de00640 1 --2- 172.21.15.86:0/2101648405 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fab5806f4c0 0x7fab580719d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.353+0000 7fab7de00640 1 --2- 172.21.15.86:0/2101648405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab78070330 0x7fab78158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.353+0000 7fab7de00640 1 --2- 172.21.15.86:0/2101648405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab7806f270 0x7fab7815ac10 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.353+0000 7fab7de00640 1 -- 172.21.15.86:0/2101648405 >> 172.21.15.86:0/2101648405 conn(0x7fab780b4650 msgr2=0x7fab780c4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.353+0000 7fab7de00640 1 -- 172.21.15.86:0/2101648405 shutdown_connections 2024-04-25T14:10:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.353+0000 7fab7de00640 1 -- 172.21.15.86:0/2101648405 wait complete. 2024-04-25T14:10:38.371 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:10:38.371 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:10:38.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.569+0000 7fac73600640 1 Processor -- start 2024-04-25T14:10:38.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.569+0000 7fac73600640 1 -- start start 2024-04-25T14:10:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.569+0000 7fac73600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fac6c0ccab0 0x7fac6c0d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.569+0000 7fac73600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac6c0cf9d0 0x7fac6c0d2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.569+0000 7fac73600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac6c0b3bd0 con 0x7fac6c0b79c0 2024-04-25T14:10:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.569+0000 7fac73600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fac6c0cf190 con 0x7fac6c0ccab0 2024-04-25T14:10:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.569+0000 7fac73600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fac6c0b2620 con 0x7fac6c0cf9d0 2024-04-25T14:10:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.573+0000 7fac70c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fac6c0ccab0 0x7fac6c0d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.573+0000 7fac6be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fac6c0b79c0 0x7fac6c1617f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39962/0 (socket says 172.21.15.86:39962) 2024-04-25T14:10:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.573+0000 7fac6be00640 1 -- 172.21.15.86:0/1886441512 learned_addr learned my addr 172.21.15.86:0/1886441512 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:38.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.573+0000 7fac71600640 1 --2- 172.21.15.86:0/1886441512 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac6c0cf9d0 0x7fac6c0d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.573+0000 7fac70c00640 1 -- 172.21.15.86:0/1886441512 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac6c0cf9d0 msgr2=0x7fac6c0d2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:38.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.573+0000 7fac70c00640 1 --2- 172.21.15.86:0/1886441512 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac6c0cf9d0 0x7fac6c0d2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.573+0000 7fac70c00640 1 -- 172.21.15.86:0/1886441512 >> v1:172.21.15.86:6789/0 conn(0x7fac6c0b79c0 legacy=0x7fac6c1617f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.573+0000 7fac70c00640 1 -- 172.21.15.86:0/1886441512 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6c0b1cf0 con 0x7fac6c0ccab0 2024-04-25T14:10:38.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.573+0000 7fac70c00640 1 --2- 172.21.15.86:0/1886441512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac6c0ccab0 0x7fac6c0d23d0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fac600097a0 tx=0x7fac6001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef0b0c144063811d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:38.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.573+0000 7fac6b400640 1 -- 172.21.15.86:0/1886441512 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac600b0070 con 0x7fac6c0ccab0 2024-04-25T14:10:38.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.573+0000 7fac6b400640 1 -- 172.21.15.86:0/1886441512 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac600ab600 con 0x7fac6c0ccab0 2024-04-25T14:10:38.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.573+0000 7fac6b400640 1 -- 172.21.15.86:0/1886441512 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac600ab9d0 con 0x7fac6c0ccab0 2024-04-25T14:10:38.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.577+0000 7fac73600640 1 -- 172.21.15.86:0/1886441512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac6c0ccab0 msgr2=0x7fac6c0d23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.577+0000 7fac73600640 1 --2- 172.21.15.86:0/1886441512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac6c0ccab0 0x7fac6c0d23d0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fac600097a0 tx=0x7fac6001c480 comp rx=0 tx=0).stop 2024-04-25T14:10:38.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.577+0000 7fac73600640 1 -- 172.21.15.86:0/1886441512 shutdown_connections 2024-04-25T14:10:38.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.577+0000 7fac73600640 1 --2- 172.21.15.86:0/1886441512 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac6c0cf9d0 0x7fac6c0d2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.577+0000 7fac73600640 1 --2- 172.21.15.86:0/1886441512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac6c0ccab0 0x7fac6c0d23d0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.577+0000 7fac73600640 1 -- 172.21.15.86:0/1886441512 >> 172.21.15.86:0/1886441512 conn(0x7fac6c09ba30 msgr2=0x7fac6c0b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:38.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.577+0000 7fac73600640 1 -- 172.21.15.86:0/1886441512 shutdown_connections 2024-04-25T14:10:38.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.577+0000 7fac73600640 1 -- 172.21.15.86:0/1886441512 wait complete. 2024-04-25T14:10:38.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.577+0000 7fac73600640 1 Processor -- start 2024-04-25T14:10:38.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.577+0000 7fac73600640 1 -- start start 2024-04-25T14:10:38.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.581+0000 7fac73600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac6c0b79c0 0x7fac6c1437c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.581+0000 7fac73600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac6c0ccab0 0x7fac6c0b01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.581+0000 7fac73600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac6c0cf9d0 0x7fac6c0b0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.581+0000 7fac73600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c16ab00 con 0x7fac6c0ccab0 2024-04-25T14:10:38.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.581+0000 7fac73600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c16a980 con 0x7fac6c0b79c0 2024-04-25T14:10:38.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.581+0000 7fac73600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fac6c16ac80 con 0x7fac6c0cf9d0 2024-04-25T14:10:38.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.581+0000 7fac6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac6c0ccab0 0x7fac6c0b01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.581+0000 7fac6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac6c0ccab0 0x7fac6c0b01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57544/0 (socket says 172.21.15.86:57544) 2024-04-25T14:10:38.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.581+0000 7fac6be00640 1 -- 172.21.15.86:0/1116075982 learned_addr learned my addr 172.21.15.86:0/1116075982 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:38.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.581+0000 7fac70c00640 1 --2- 172.21.15.86:0/1116075982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac6c0b79c0 0x7fac6c1437c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.581+0000 7fac71600640 1 --2- 172.21.15.86:0/1116075982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac6c0cf9d0 0x7fac6c0b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.581+0000 7fac6be00640 1 -- 172.21.15.86:0/1116075982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac6c0cf9d0 msgr2=0x7fac6c0b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.581+0000 7fac6be00640 1 --2- 172.21.15.86:0/1116075982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac6c0cf9d0 0x7fac6c0b0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.581+0000 7fac6be00640 1 -- 172.21.15.86:0/1116075982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac6c0b79c0 msgr2=0x7fac6c1437c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.581+0000 7fac6be00640 1 --2- 172.21.15.86:0/1116075982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac6c0b79c0 0x7fac6c1437c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.581+0000 7fac6be00640 1 -- 172.21.15.86:0/1116075982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6c0afcf0 con 0x7fac6c0ccab0 2024-04-25T14:10:38.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.581+0000 7fac70c00640 1 --2- 172.21.15.86:0/1116075982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac6c0b79c0 0x7fac6c1437c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:38.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.581+0000 7fac6be00640 1 --2- 172.21.15.86:0/1116075982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac6c0ccab0 0x7fac6c0b01d0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fac5800c370 tx=0x7fac5800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:38.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.581+0000 7fac68c00640 1 -- 172.21.15.86:0/1116075982 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac58017070 con 0x7fac6c0ccab0 2024-04-25T14:10:38.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.581+0000 7fac68c00640 1 -- 172.21.15.86:0/1116075982 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac58005cf0 con 0x7fac6c0ccab0 2024-04-25T14:10:38.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.581+0000 7fac68c00640 1 -- 172.21.15.86:0/1116075982 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac580072f0 con 0x7fac6c0ccab0 2024-04-25T14:10:38.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.581+0000 7fac73600640 1 -- 172.21.15.86:0/1116075982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac6c0c63e0 con 0x7fac6c0ccab0 2024-04-25T14:10:38.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.581+0000 7fac73600640 1 -- 172.21.15.86:0/1116075982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac6c0c5e20 con 0x7fac6c0ccab0 2024-04-25T14:10:38.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.581+0000 7fac73600640 1 -- 172.21.15.86:0/1116075982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac6c0a04c0 con 0x7fac6c0ccab0 2024-04-25T14:10:38.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.585+0000 7fac68c00640 1 -- 172.21.15.86:0/1116075982 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7fac5801a070 con 0x7fac6c0ccab0 2024-04-25T14:10:38.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.585+0000 7fac68c00640 1 --2- 172.21.15.86:0/1116075982 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fac4c06f4c0 0x7fac4c0719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.585+0000 7fac68c00640 1 -- 172.21.15.86:0/1116075982 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5980+0+0 (secure 0 0 0) 0x7fac5808e680 con 0x7fac6c0ccab0 2024-04-25T14:10:38.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.585+0000 7fac70c00640 1 --2- 172.21.15.86:0/1116075982 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fac4c06f4c0 0x7fac4c0719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.585+0000 7fac70c00640 1 --2- 172.21.15.86:0/1116075982 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fac4c06f4c0 0x7fac4c0719d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fac600ad000 tx=0x7fac60008040 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:38.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.589+0000 7fac68c00640 1 -- 172.21.15.86:0/1116075982 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fac5805b510 con 0x7fac6c0ccab0 2024-04-25T14:10:38.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.757+0000 7fac73600640 1 -- 172.21.15.86:0/1116075982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fac6c0d2f90 con 0x7fac6c0ccab0 2024-04-25T14:10:38.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.757+0000 7fac68c00640 1 -- 172.21.15.86:0/1116075982 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) ==== 93+0+10 (secure 0 0 0) 0x7fac5805ae10 con 0x7fac6c0ccab0 2024-04-25T14:10:38.754 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:10:38.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.757+0000 7fac73600640 1 -- 172.21.15.86:0/1116075982 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fac4c06f4c0 msgr2=0x7fac4c0719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.757+0000 7fac73600640 1 --2- 172.21.15.86:0/1116075982 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fac4c06f4c0 0x7fac4c0719d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fac600ad000 tx=0x7fac60008040 comp rx=0 tx=0).stop 2024-04-25T14:10:38.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.757+0000 7fac73600640 1 -- 172.21.15.86:0/1116075982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac6c0ccab0 msgr2=0x7fac6c0b01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.757+0000 7fac73600640 1 --2- 172.21.15.86:0/1116075982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac6c0ccab0 0x7fac6c0b01d0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fac5800c370 tx=0x7fac5800c830 comp rx=0 tx=0).stop 2024-04-25T14:10:38.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.757+0000 7fac73600640 1 -- 172.21.15.86:0/1116075982 shutdown_connections 2024-04-25T14:10:38.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.757+0000 7fac73600640 1 --2- 172.21.15.86:0/1116075982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac6c0cf9d0 0x7fac6c0b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.757+0000 7fac73600640 1 --2- 172.21.15.86:0/1116075982 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fac4c06f4c0 0x7fac4c0719d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.757+0000 7fac73600640 1 --2- 172.21.15.86:0/1116075982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac6c0ccab0 0x7fac6c0b01d0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.757+0000 7fac73600640 1 --2- 172.21.15.86:0/1116075982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac6c0b79c0 0x7fac6c1437c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.757+0000 7fac73600640 1 -- 172.21.15.86:0/1116075982 >> 172.21.15.86:0/1116075982 conn(0x7fac6c09ba30 msgr2=0x7fac6c06ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:38.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.757+0000 7fac73600640 1 -- 172.21.15.86:0/1116075982 shutdown_connections 2024-04-25T14:10:38.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.757+0000 7fac73600640 1 -- 172.21.15.86:0/1116075982 wait complete. 2024-04-25T14:10:38.777 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_metrics_from_rank' 2024-04-25T14:10:38.781 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:38.781+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:38.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.933+0000 7f70e4600640 1 Processor -- start 2024-04-25T14:10:38.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.933+0000 7f70e4600640 1 -- start start 2024-04-25T14:10:38.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.937+0000 7f70e4600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f70dc0cfca0 0x7f70dc06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.937+0000 7f70e4600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f70dc0730b0 0x7f70dc0734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.937+0000 7f70e4600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70dc072bd0 con 0x7f70dc0d16e0 2024-04-25T14:10:38.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.937+0000 7f70e4600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f70dc070470 con 0x7f70dc0730b0 2024-04-25T14:10:38.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.937+0000 7f70e4600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f70dc06f930 con 0x7f70dc0cfca0 2024-04-25T14:10:38.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.937+0000 7f70e2600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f70dc0730b0 0x7f70dc0734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.937+0000 7f70e1200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f70dc0d16e0 0x7f70dc0d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36386/0 (socket says 172.21.15.86:36386) 2024-04-25T14:10:38.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.937+0000 7f70e1200640 1 -- 172.21.15.86:0/1935185003 learned_addr learned my addr 172.21.15.86:0/1935185003 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:38.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.937+0000 7f70e1c00640 1 --2- 172.21.15.86:0/1935185003 >> v2:172.21.15.155:3300/0 conn(0x7f70dc0cfca0 0x7f70dc06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.937+0000 7f70dbe00640 1 -- 172.21.15.86:0/1935185003 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1721032041 0 0) 0x7f70dc072bd0 con 0x7f70dc0d16e0 2024-04-25T14:10:38.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.937+0000 7f70dbe00640 1 -- 172.21.15.86:0/1935185003 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70c4003530 con 0x7f70dc0d16e0 2024-04-25T14:10:38.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.937+0000 7f70e2600640 1 -- 172.21.15.86:0/1935185003 >> v2:172.21.15.155:3300/0 conn(0x7f70dc0cfca0 msgr2=0x7f70dc06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:38.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.937+0000 7f70e2600640 1 --2- 172.21.15.86:0/1935185003 >> v2:172.21.15.155:3300/0 conn(0x7f70dc0cfca0 0x7f70dc06f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.937+0000 7f70e2600640 1 -- 172.21.15.86:0/1935185003 >> v1:172.21.15.86:6789/0 conn(0x7f70dc0d16e0 legacy=0x7f70dc0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.937+0000 7f70e2600640 1 -- 172.21.15.86:0/1935185003 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70dc06fbd0 con 0x7f70dc0730b0 2024-04-25T14:10:38.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.941+0000 7f70e2600640 1 --2- 172.21.15.86:0/1935185003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70dc0730b0 0x7f70dc0734c0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f70dc0d22a0 tx=0x7f70d401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d23abdafd40f4d02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:38.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.941+0000 7f70dbe00640 1 -- 172.21.15.86:0/1935185003 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d40b2070 con 0x7f70dc0730b0 2024-04-25T14:10:38.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.941+0000 7f70dbe00640 1 -- 172.21.15.86:0/1935185003 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70d40a5900 con 0x7f70dc0730b0 2024-04-25T14:10:38.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.941+0000 7f70dbe00640 1 -- 172.21.15.86:0/1935185003 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d40a5cb0 con 0x7f70dc0730b0 2024-04-25T14:10:38.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.941+0000 7f70e4600640 1 -- 172.21.15.86:0/1935185003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70dc0730b0 msgr2=0x7f70dc0734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.941+0000 7f70e4600640 1 --2- 172.21.15.86:0/1935185003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70dc0730b0 0x7f70dc0734c0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f70dc0d22a0 tx=0x7f70d401c930 comp rx=0 tx=0).stop 2024-04-25T14:10:38.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.941+0000 7f70e4600640 1 -- 172.21.15.86:0/1935185003 shutdown_connections 2024-04-25T14:10:38.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.941+0000 7f70e4600640 1 --2- 172.21.15.86:0/1935185003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70dc0730b0 0x7f70dc0734c0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.941+0000 7f70e4600640 1 --2- 172.21.15.86:0/1935185003 >> v2:172.21.15.155:3300/0 conn(0x7f70dc0cfca0 0x7f70dc06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.941+0000 7f70e4600640 1 -- 172.21.15.86:0/1935185003 >> 172.21.15.86:0/1935185003 conn(0x7f70dc0b4650 msgr2=0x7f70dc0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:38.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.941+0000 7f70e4600640 1 -- 172.21.15.86:0/1935185003 shutdown_connections 2024-04-25T14:10:38.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.941+0000 7f70e4600640 1 -- 172.21.15.86:0/1935185003 wait complete. 2024-04-25T14:10:38.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.941+0000 7f70e4600640 1 Processor -- start 2024-04-25T14:10:38.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.941+0000 7f70e4600640 1 -- start start 2024-04-25T14:10:38.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.945+0000 7f70e4600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70dc0730b0 0x7f70dc0c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.945+0000 7f70e4600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70dc0cfca0 0x7f70dc0b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.945+0000 7f70e4600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70dc0d16e0 0x7f70dc08f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.945+0000 7f70e4600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc169a70 con 0x7f70dc0730b0 2024-04-25T14:10:38.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.945+0000 7f70e4600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc1698f0 con 0x7f70dc0d16e0 2024-04-25T14:10:38.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.945+0000 7f70e4600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f70dc169bf0 con 0x7f70dc0cfca0 2024-04-25T14:10:38.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.945+0000 7f70e1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70dc0730b0 0x7f70dc0c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.945+0000 7f70e1200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70dc0cfca0 0x7f70dc0b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.945+0000 7f70e1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70dc0730b0 0x7f70dc0c98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53048/0 (socket says 172.21.15.86:53048) 2024-04-25T14:10:38.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.945+0000 7f70e1c00640 1 -- 172.21.15.86:0/3027037313 learned_addr learned my addr 172.21.15.86:0/3027037313 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:38.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.945+0000 7f70e2600640 1 --2- 172.21.15.86:0/3027037313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70dc0d16e0 0x7f70dc08f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.945+0000 7f70e1c00640 1 -- 172.21.15.86:0/3027037313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70dc0cfca0 msgr2=0x7f70dc0b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.945+0000 7f70e1c00640 1 --2- 172.21.15.86:0/3027037313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70dc0cfca0 0x7f70dc0b2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.945+0000 7f70e1c00640 1 -- 172.21.15.86:0/3027037313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70dc0d16e0 msgr2=0x7f70dc08f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.945+0000 7f70e1c00640 1 --2- 172.21.15.86:0/3027037313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70dc0d16e0 0x7f70dc08f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.945+0000 7f70e1c00640 1 -- 172.21.15.86:0/3027037313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70dc0ca2d0 con 0x7f70dc0730b0 2024-04-25T14:10:38.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.945+0000 7f70e1c00640 1 --2- 172.21.15.86:0/3027037313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70dc0730b0 0x7f70dc0c98f0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f70d000e870 tx=0x7f70d000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:38.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.945+0000 7f70d9600640 1 -- 172.21.15.86:0/3027037313 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d0017070 con 0x7f70dc0730b0 2024-04-25T14:10:38.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.945+0000 7f70d9600640 1 -- 172.21.15.86:0/3027037313 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70d000c940 con 0x7f70dc0730b0 2024-04-25T14:10:38.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.945+0000 7f70d9600640 1 -- 172.21.15.86:0/3027037313 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d000cd10 con 0x7f70dc0730b0 2024-04-25T14:10:38.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.945+0000 7f70e4600640 1 -- 172.21.15.86:0/3027037313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70dc0ca5b0 con 0x7f70dc0730b0 2024-04-25T14:10:38.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.945+0000 7f70e4600640 1 -- 172.21.15.86:0/3027037313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70dc0cab70 con 0x7f70dc0730b0 2024-04-25T14:10:38.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.945+0000 7f70e4600640 1 -- 172.21.15.86:0/3027037313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70dc09f4b0 con 0x7f70dc0730b0 2024-04-25T14:10:38.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.949+0000 7f70d9600640 1 -- 172.21.15.86:0/3027037313 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7f70d001a070 con 0x7f70dc0730b0 2024-04-25T14:10:38.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.949+0000 7f70d9600640 1 --2- 172.21.15.86:0/3027037313 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f70bc06f4c0 0x7f70bc0719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.949+0000 7f70d9600640 1 -- 172.21.15.86:0/3027037313 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5980+0+0 (secure 0 0 0) 0x7f70d008f1d0 con 0x7f70dc0730b0 2024-04-25T14:10:38.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.949+0000 7f70e1200640 1 --2- 172.21.15.86:0/3027037313 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f70bc06f4c0 0x7f70bc0719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.949+0000 7f70e1200640 1 --2- 172.21.15.86:0/3027037313 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f70bc06f4c0 0x7f70bc0719d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f70c8007fd0 tx=0x7f70c8009040 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:38.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:38.953+0000 7f70d9600640 1 -- 172.21.15.86:0/3027037313 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f70d005b970 con 0x7f70dc0730b0 2024-04-25T14:10:39.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:39.173+0000 7f70e4600640 1 -- 172.21.15.86:0/3027037313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_metrics_from_rank"]} v 0) -- 0x7f70dc069d00 con 0x7f70dc0730b0 2024-04-25T14:10:39.512 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:39.513+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:39.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.001+0000 7f70d9600640 1 -- 172.21.15.86:0/3027037313 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_metrics_from_rank"]}]=0 v388) ==== 149+0+0 (secure 0 0 0) 0x7f70d005fc70 con 0x7f70dc0730b0 2024-04-25T14:10:39.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.001+0000 7f70e4600640 1 -- 172.21.15.86:0/3027037313 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f70bc06f4c0 msgr2=0x7f70bc0719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:39.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.001+0000 7f70e4600640 1 --2- 172.21.15.86:0/3027037313 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f70bc06f4c0 0x7f70bc0719d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f70c8007fd0 tx=0x7f70c8009040 comp rx=0 tx=0).stop 2024-04-25T14:10:39.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.001+0000 7f70e4600640 1 -- 172.21.15.86:0/3027037313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70dc0730b0 msgr2=0x7f70dc0c98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:40.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.001+0000 7f70e4600640 1 --2- 172.21.15.86:0/3027037313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70dc0730b0 0x7f70dc0c98f0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f70d000e870 tx=0x7f70d000ec70 comp rx=0 tx=0).stop 2024-04-25T14:10:40.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.001+0000 7f70e4600640 1 -- 172.21.15.86:0/3027037313 shutdown_connections 2024-04-25T14:10:40.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.001+0000 7f70e4600640 1 --2- 172.21.15.86:0/3027037313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70dc0d16e0 0x7f70dc08f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:40.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.001+0000 7f70e4600640 1 --2- 172.21.15.86:0/3027037313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70dc0cfca0 0x7f70dc0b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:40.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.001+0000 7f70e4600640 1 --2- 172.21.15.86:0/3027037313 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f70bc06f4c0 0x7f70bc0719d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:40.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.001+0000 7f70e4600640 1 --2- 172.21.15.86:0/3027037313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70dc0730b0 0x7f70dc0c98f0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:40.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.001+0000 7f70e4600640 1 -- 172.21.15.86:0/3027037313 >> 172.21.15.86:0/3027037313 conn(0x7f70dc0b4650 msgr2=0x7f70dc0b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:40.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.001+0000 7f70e4600640 1 -- 172.21.15.86:0/3027037313 shutdown_connections 2024-04-25T14:10:40.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.001+0000 7f70e4600640 1 -- 172.21.15.86:0/3027037313 wait complete. 2024-04-25T14:10:40.019 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T14:10:40.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.181+0000 7f8023800640 1 Processor -- start 2024-04-25T14:10:40.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.181+0000 7f8023800640 1 -- start start 2024-04-25T14:10:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.181+0000 7f8023800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f801c0ce660 0x7f801c15de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.181+0000 7f8023800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f801c161b40 con 0x7f801c0cef20 2024-04-25T14:10:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.181+0000 7f8023800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f801c161e20 con 0x7f801c06c670 2024-04-25T14:10:40.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.181+0000 7f8023800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f801c15e380 con 0x7f801c0ce660 2024-04-25T14:10:40.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.185+0000 7f8021800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f801c0cef20 0x7f801c0cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36388/0 (socket says 172.21.15.86:36388) 2024-04-25T14:10:40.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.185+0000 7f8021800640 1 -- 172.21.15.86:0/187300472 learned_addr learned my addr 172.21.15.86:0/187300472 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:40.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.185+0000 7f8013e00640 1 --2- 172.21.15.86:0/187300472 >> v2:172.21.15.155:3300/0 conn(0x7f801c0ce660 0x7f801c15de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:40.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.185+0000 7f801be00640 1 -- 172.21.15.86:0/187300472 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1662325104 0 0) 0x7f801c161b40 con 0x7f801c0cef20 2024-04-25T14:10:40.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.185+0000 7f801be00640 1 -- 172.21.15.86:0/187300472 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8000003530 con 0x7f801c0cef20 2024-04-25T14:10:40.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.185+0000 7f8013e00640 1 -- 172.21.15.86:0/187300472 >> v1:172.21.15.155:6789/0 conn(0x7f801c06c670 legacy=0x7f801c0cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:40.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.185+0000 7f8013e00640 1 -- 172.21.15.86:0/187300472 >> v1:172.21.15.86:6789/0 conn(0x7f801c0cef20 legacy=0x7f801c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:40.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.185+0000 7f8013e00640 1 -- 172.21.15.86:0/187300472 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801c162100 con 0x7f801c0ce660 2024-04-25T14:10:40.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.185+0000 7f8013e00640 1 --2- 172.21.15.86:0/187300472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f801c0ce660 0x7f801c15de40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f80040097c0 tx=0x7f800401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a28ba033ef595d4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:40.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.189+0000 7f801be00640 1 -- 172.21.15.86:0/187300472 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80040b0070 con 0x7f801c0ce660 2024-04-25T14:10:40.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.189+0000 7f801be00640 1 -- 172.21.15.86:0/187300472 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80040ab600 con 0x7f801c0ce660 2024-04-25T14:10:40.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.189+0000 7f801be00640 1 -- 172.21.15.86:0/187300472 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80040ab9b0 con 0x7f801c0ce660 2024-04-25T14:10:40.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.189+0000 7f8023800640 1 -- 172.21.15.86:0/187300472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f801c0ce660 msgr2=0x7f801c15de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:40.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.189+0000 7f8023800640 1 --2- 172.21.15.86:0/187300472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f801c0ce660 0x7f801c15de40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f80040097c0 tx=0x7f800401c410 comp rx=0 tx=0).stop 2024-04-25T14:10:40.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.189+0000 7f8023800640 1 -- 172.21.15.86:0/187300472 shutdown_connections 2024-04-25T14:10:40.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.189+0000 7f8023800640 1 --2- 172.21.15.86:0/187300472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f801c0ce660 0x7f801c15de40 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:40.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.189+0000 7f8023800640 1 -- 172.21.15.86:0/187300472 >> 172.21.15.86:0/187300472 conn(0x7f801c09ba30 msgr2=0x7f801c0b73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:40.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.189+0000 7f8023800640 1 -- 172.21.15.86:0/187300472 shutdown_connections 2024-04-25T14:10:40.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.189+0000 7f8023800640 1 -- 172.21.15.86:0/187300472 wait complete. 2024-04-25T14:10:40.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.189+0000 7f8023800640 1 Processor -- start 2024-04-25T14:10:40.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.189+0000 7f8023800640 1 -- start start 2024-04-25T14:10:40.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.193+0000 7f8023800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f801c06c670 0x7f801c150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:40.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.193+0000 7f8023800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f801c0ce660 0x7f801c1517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:40.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.193+0000 7f8023800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801c0cef20 0x7f801c151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:40.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.193+0000 7f8023800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f801c162750 con 0x7f801c0cef20 2024-04-25T14:10:40.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.193+0000 7f8023800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f801c0b80f0 con 0x7f801c06c670 2024-04-25T14:10:40.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.193+0000 7f8023800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f801c16c790 con 0x7f801c0ce660 2024-04-25T14:10:40.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.193+0000 7f8020e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f801c06c670 0x7f801c150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:40.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.193+0000 7f8013e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f801c0ce660 0x7f801c1517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:40.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.193+0000 7f8021800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801c0cef20 0x7f801c151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:40.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.193+0000 7f8020e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f801c06c670 0x7f801c150f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:33680/0 (socket says 172.21.15.86:33680) 2024-04-25T14:10:40.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.193+0000 7f8020e00640 1 -- 172.21.15.86:0/1075176598 learned_addr learned my addr 172.21.15.86:0/1075176598 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:40.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.193+0000 7f8013e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f801c0ce660 0x7f801c1517e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:41878/0 (socket says 172.21.15.86:41878) 2024-04-25T14:10:40.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.193+0000 7f8020e00640 1 -- 172.21.15.86:0/1075176598 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f801c0ce660 msgr2=0x7f801c1517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:40.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.193+0000 7f8020e00640 1 --2- 172.21.15.86:0/1075176598 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f801c0ce660 0x7f801c1517e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:40.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.193+0000 7f8020e00640 1 -- 172.21.15.86:0/1075176598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801c0cef20 msgr2=0x7f801c151d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:40.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.193+0000 7f8020e00640 1 --2- 172.21.15.86:0/1075176598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801c0cef20 0x7f801c151d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:40.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.193+0000 7f8020e00640 1 -- 172.21.15.86:0/1075176598 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801c15b420 con 0x7f801c06c670 2024-04-25T14:10:40.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.193+0000 7f8013e00640 1 --2- 172.21.15.86:0/1075176598 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f801c0ce660 0x7f801c1517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:10:40.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.193+0000 7f8020e00640 1 --2- 172.21.15.86:0/1075176598 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f801c06c670 0x7f801c150f80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f800c00c340 tx=0x7f800c00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:40.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.193+0000 7f8019600640 1 -- 172.21.15.86:0/1075176598 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c0052d0 con 0x7f801c06c670 2024-04-25T14:10:40.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.193+0000 7f8019600640 1 -- 172.21.15.86:0/1075176598 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f800c005470 con 0x7f801c06c670 2024-04-25T14:10:40.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.193+0000 7f8023800640 1 -- 172.21.15.86:0/1075176598 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f801c15b700 con 0x7f801c06c670 2024-04-25T14:10:40.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.193+0000 7f8019600640 1 -- 172.21.15.86:0/1075176598 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c007260 con 0x7f801c06c670 2024-04-25T14:10:40.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.193+0000 7f8023800640 1 -- 172.21.15.86:0/1075176598 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f801c155ad0 con 0x7f801c06c670 2024-04-25T14:10:40.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.193+0000 7f8023800640 1 -- 172.21.15.86:0/1075176598 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f801c0a0630 con 0x7f801c06c670 2024-04-25T14:10:40.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.197+0000 7f8019600640 1 -- 172.21.15.86:0/1075176598 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7f800c007400 con 0x7f801c06c670 2024-04-25T14:10:40.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.197+0000 7f8019600640 1 --2- 172.21.15.86:0/1075176598 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f7ff806f4c0 0x7f7ff80719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:40.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.197+0000 7f8019600640 1 -- 172.21.15.86:0/1075176598 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5980+0+0 (secure 0 0 0) 0x7f800c08e810 con 0x7f801c06c670 2024-04-25T14:10:40.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.197+0000 7f8013e00640 1 --2- 172.21.15.86:0/1075176598 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f7ff806f4c0 0x7f7ff80719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:40.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.197+0000 7f8013e00640 1 --2- 172.21.15.86:0/1075176598 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f7ff806f4c0 0x7f7ff80719d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f80040ad000 tx=0x7f80040a6000 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:40.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.201+0000 7f8019600640 1 -- 172.21.15.86:0/1075176598 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f800c05afc0 con 0x7f801c06c670 2024-04-25T14:10:40.265 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:40.265+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:40.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.369+0000 7f8023800640 1 -- 172.21.15.86:0/1075176598 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f801c076bf0 con 0x7f801c06c670 2024-04-25T14:10:40.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.373+0000 7f8019600640 1 -- 172.21.15.86:0/1075176598 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7f800c05f2c0 con 0x7f801c06c670 2024-04-25T14:10:40.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.377+0000 7f8023800640 1 -- 172.21.15.86:0/1075176598 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f7ff806f4c0 msgr2=0x7f7ff80719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:40.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.377+0000 7f8023800640 1 --2- 172.21.15.86:0/1075176598 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f7ff806f4c0 0x7f7ff80719d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f80040ad000 tx=0x7f80040a6000 comp rx=0 tx=0).stop 2024-04-25T14:10:40.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.377+0000 7f8023800640 1 -- 172.21.15.86:0/1075176598 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f801c06c670 msgr2=0x7f801c150f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:40.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.377+0000 7f8023800640 1 --2- 172.21.15.86:0/1075176598 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f801c06c670 0x7f801c150f80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f800c00c340 tx=0x7f800c00c800 comp rx=0 tx=0).stop 2024-04-25T14:10:40.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.377+0000 7f8023800640 1 -- 172.21.15.86:0/1075176598 shutdown_connections 2024-04-25T14:10:40.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.377+0000 7f8023800640 1 --2- 172.21.15.86:0/1075176598 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f7ff806f4c0 0x7f7ff80719d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:40.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.377+0000 7f8023800640 1 --2- 172.21.15.86:0/1075176598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801c0cef20 0x7f801c151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:40.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.377+0000 7f8023800640 1 --2- 172.21.15.86:0/1075176598 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f801c0ce660 0x7f801c1517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:40.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.377+0000 7f8023800640 1 --2- 172.21.15.86:0/1075176598 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f801c06c670 0x7f801c150f80 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:40.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.377+0000 7f8023800640 1 -- 172.21.15.86:0/1075176598 >> 172.21.15.86:0/1075176598 conn(0x7f801c09ba30 msgr2=0x7f801c0b4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:40.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.377+0000 7f8023800640 1 -- 172.21.15.86:0/1075176598 shutdown_connections 2024-04-25T14:10:40.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.377+0000 7f8023800640 1 -- 172.21.15.86:0/1075176598 wait complete. 2024-04-25T14:10:40.396 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T14:10:40.396 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:10:40.463 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:10:40.527 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T14:10:40.527 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:10:40.527 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:10:40.536 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2893 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T14:10:40.536 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:10:40.536 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:10:40.545 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4517 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T14:10:40.545 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:10:40.545 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:10:40.589 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2893 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T14:10:40.589 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:10:40.589 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:10:40.598 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4517 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T14:10:40.598 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T14:10:40.598 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:10:40.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.793+0000 7f5e69800640 1 Processor -- start 2024-04-25T14:10:40.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.793+0000 7f5e69800640 1 -- start start 2024-04-25T14:10:40.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.793+0000 7f5e69800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5e64160120 0x7f5e640af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:40.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.793+0000 7f5e69800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e640d0f50 0x7f5e640d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:40.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.793+0000 7f5e69800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e640af780 con 0x7f5e640d18a0 2024-04-25T14:10:40.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.793+0000 7f5e69800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5e640d0240 con 0x7f5e64160120 2024-04-25T14:10:40.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.793+0000 7f5e69800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5e640b7fd0 con 0x7f5e640d0f50 2024-04-25T14:10:40.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.793+0000 7f5e63e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5e64160120 0x7f5e640af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:40.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.793+0000 7f5e63e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5e64160120 0x7f5e640af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53064/0 (socket says 172.21.15.86:53064) 2024-04-25T14:10:40.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.793+0000 7f5e63e00640 1 -- 172.21.15.86:0/3250378120 learned_addr learned my addr 172.21.15.86:0/3250378120 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:40.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.793+0000 7f5e63400640 1 --2- 172.21.15.86:0/3250378120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e640d0f50 0x7f5e640d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:40.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.793+0000 7f5e63e00640 1 -- 172.21.15.86:0/3250378120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e640d0f50 msgr2=0x7f5e640d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:40.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.793+0000 7f5e63e00640 1 --2- 172.21.15.86:0/3250378120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e640d0f50 0x7f5e640d1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:40.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.793+0000 7f5e63e00640 1 -- 172.21.15.86:0/3250378120 >> v1:172.21.15.155:6789/0 conn(0x7f5e640d18a0 legacy=0x7f5e640d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:40.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.793+0000 7f5e63e00640 1 -- 172.21.15.86:0/3250378120 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e640b7cf0 con 0x7f5e64160120 2024-04-25T14:10:40.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.797+0000 7f5e63e00640 1 --2- 172.21.15.86:0/3250378120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e64160120 0x7f5e640af240 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f5e540098a0 tx=0x7f5e5401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c48b60e6243f4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:40.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.797+0000 7f5e62a00640 1 -- 172.21.15.86:0/3250378120 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e540b0070 con 0x7f5e64160120 2024-04-25T14:10:40.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.797+0000 7f5e62a00640 1 -- 172.21.15.86:0/3250378120 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e540ab600 con 0x7f5e64160120 2024-04-25T14:10:40.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.797+0000 7f5e62a00640 1 -- 172.21.15.86:0/3250378120 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e540ab9d0 con 0x7f5e64160120 2024-04-25T14:10:40.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.797+0000 7f5e69800640 1 -- 172.21.15.86:0/3250378120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e64160120 msgr2=0x7f5e640af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:40.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.797+0000 7f5e69800640 1 --2- 172.21.15.86:0/3250378120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e64160120 0x7f5e640af240 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f5e540098a0 tx=0x7f5e5401c4b0 comp rx=0 tx=0).stop 2024-04-25T14:10:40.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.797+0000 7f5e69800640 1 -- 172.21.15.86:0/3250378120 shutdown_connections 2024-04-25T14:10:40.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.797+0000 7f5e69800640 1 --2- 172.21.15.86:0/3250378120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e640d0f50 0x7f5e640d1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:40.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.797+0000 7f5e69800640 1 --2- 172.21.15.86:0/3250378120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e64160120 0x7f5e640af240 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:40.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.797+0000 7f5e69800640 1 -- 172.21.15.86:0/3250378120 >> 172.21.15.86:0/3250378120 conn(0x7f5e6409ba30 msgr2=0x7f5e640b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:40.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.797+0000 7f5e69800640 1 -- 172.21.15.86:0/3250378120 shutdown_connections 2024-04-25T14:10:40.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.797+0000 7f5e69800640 1 -- 172.21.15.86:0/3250378120 wait complete. 2024-04-25T14:10:40.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.801+0000 7f5e69800640 1 Processor -- start 2024-04-25T14:10:40.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.801+0000 7f5e69800640 1 -- start start 2024-04-25T14:10:40.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.801+0000 7f5e69800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e640d0f50 0x7f5e64161350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:40.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.801+0000 7f5e69800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e640d18a0 0x7f5e64152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:40.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.801+0000 7f5e69800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e64160120 0x7f5e6406ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:40.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.801+0000 7f5e69800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6416cd60 con 0x7f5e64160120 2024-04-25T14:10:40.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.801+0000 7f5e69800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6416cbe0 con 0x7f5e640d0f50 2024-04-25T14:10:40.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.801+0000 7f5e69800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5e6416cee0 con 0x7f5e640d18a0 2024-04-25T14:10:40.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.801+0000 7f5e68e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e64160120 0x7f5e6406ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:40.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.801+0000 7f5e63e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e640d0f50 0x7f5e64161350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:40.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.805+0000 7f5e63e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e640d0f50 0x7f5e64161350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:33690/0 (socket says 172.21.15.86:33690) 2024-04-25T14:10:40.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.805+0000 7f5e63400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e640d18a0 0x7f5e64152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:40.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.805+0000 7f5e63e00640 1 -- 172.21.15.86:0/2281128338 learned_addr learned my addr 172.21.15.86:0/2281128338 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:40.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.805+0000 7f5e63e00640 1 -- 172.21.15.86:0/2281128338 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e640d18a0 msgr2=0x7f5e64152c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:40.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.805+0000 7f5e63e00640 1 --2- 172.21.15.86:0/2281128338 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e640d18a0 0x7f5e64152c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:40.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.805+0000 7f5e63e00640 1 -- 172.21.15.86:0/2281128338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e64160120 msgr2=0x7f5e6406ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:40.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.805+0000 7f5e63e00640 1 --2- 172.21.15.86:0/2281128338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e64160120 0x7f5e6406ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:40.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.805+0000 7f5e63e00640 1 -- 172.21.15.86:0/2281128338 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e641531d0 con 0x7f5e640d0f50 2024-04-25T14:10:40.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.805+0000 7f5e68e00640 1 --2- 172.21.15.86:0/2281128338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e64160120 0x7f5e6406ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:10:40.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.805+0000 7f5e63e00640 1 --2- 172.21.15.86:0/2281128338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e640d0f50 0x7f5e64161350 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5e5401a0e0 tx=0x7f5e54019250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:40.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.805+0000 7f5e5be00640 1 -- 172.21.15.86:0/2281128338 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e540ba070 con 0x7f5e640d0f50 2024-04-25T14:10:40.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.805+0000 7f5e5be00640 1 -- 172.21.15.86:0/2281128338 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e540b4420 con 0x7f5e640d0f50 2024-04-25T14:10:40.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.805+0000 7f5e5be00640 1 -- 172.21.15.86:0/2281128338 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e540b0040 con 0x7f5e640d0f50 2024-04-25T14:10:40.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.805+0000 7f5e69800640 1 -- 172.21.15.86:0/2281128338 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e64161ca0 con 0x7f5e640d0f50 2024-04-25T14:10:40.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.805+0000 7f5e69800640 1 -- 172.21.15.86:0/2281128338 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e64162260 con 0x7f5e640d0f50 2024-04-25T14:10:40.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.805+0000 7f5e69800640 1 -- 172.21.15.86:0/2281128338 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e64076b80 con 0x7f5e640d0f50 2024-04-25T14:10:40.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.805+0000 7f5e5be00640 1 -- 172.21.15.86:0/2281128338 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7f5e540b47f0 con 0x7f5e640d0f50 2024-04-25T14:10:40.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.809+0000 7f5e5be00640 1 --2- 172.21.15.86:0/2281128338 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f5e4406f490 0x7f5e440719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:40.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.809+0000 7f5e5be00640 1 -- 172.21.15.86:0/2281128338 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5980+0+0 (secure 0 0 0) 0x7f5e54128920 con 0x7f5e640d0f50 2024-04-25T14:10:40.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.809+0000 7f5e63400640 1 --2- 172.21.15.86:0/2281128338 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f5e4406f490 0x7f5e440719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:40.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.809+0000 7f5e63400640 1 --2- 172.21.15.86:0/2281128338 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f5e4406f490 0x7f5e440719a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5e640af6b0 tx=0x7f5e50009190 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:40.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:40.809+0000 7f5e5be00640 1 -- 172.21.15.86:0/2281128338 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5e540c3020 con 0x7f5e640d0f50 2024-04-25T14:10:41.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.009+0000 7f5e69800640 1 -- 172.21.15.86:0/2281128338 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e640b1c00 con 0x7f5e640d0f50 2024-04-25T14:10:41.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.009+0000 7f5e5be00640 1 -- 172.21.15.86:0/2281128338 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+3872 (secure 0 0 0) 0x7f5e641531d0 con 0x7f5e640d0f50 2024-04-25T14:10:41.008 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:41.009 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":87,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7472,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/1752009762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1752009762},{"type":"v1","addr":"172.21.15.155:6833","nonce":1752009762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":8104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/374465657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":374465657},{"type":"v1","addr":"172.21.15.86:6836","nonce":374465657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8107,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1466195534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":1466195534},{"type":"v1","addr":"172.21.15.86:6837","nonce":1466195534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8119,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2431580070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2431580070},{"type":"v1","addr":"172.21.15.155:6835","nonce":2431580070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[]} 2024-04-25T14:10:41.009 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 87 2024-04-25T14:10:41.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.013+0000 7f5e69800640 1 -- 172.21.15.86:0/2281128338 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f5e4406f490 msgr2=0x7f5e440719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:41.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.013+0000 7f5e69800640 1 --2- 172.21.15.86:0/2281128338 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f5e4406f490 0x7f5e440719a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5e640af6b0 tx=0x7f5e50009190 comp rx=0 tx=0).stop 2024-04-25T14:10:41.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.013+0000 7f5e69800640 1 -- 172.21.15.86:0/2281128338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e640d0f50 msgr2=0x7f5e64161350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:41.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.013+0000 7f5e69800640 1 --2- 172.21.15.86:0/2281128338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e640d0f50 0x7f5e64161350 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5e5401a0e0 tx=0x7f5e54019250 comp rx=0 tx=0).stop 2024-04-25T14:10:41.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.013+0000 7f5e69800640 1 -- 172.21.15.86:0/2281128338 shutdown_connections 2024-04-25T14:10:41.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.013+0000 7f5e69800640 1 --2- 172.21.15.86:0/2281128338 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f5e4406f490 0x7f5e440719a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.013+0000 7f5e69800640 1 --2- 172.21.15.86:0/2281128338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e64160120 0x7f5e6406ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.013+0000 7f5e69800640 1 --2- 172.21.15.86:0/2281128338 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e640d18a0 0x7f5e64152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.013+0000 7f5e69800640 1 --2- 172.21.15.86:0/2281128338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e640d0f50 0x7f5e64161350 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.013+0000 7f5e69800640 1 -- 172.21.15.86:0/2281128338 >> 172.21.15.86:0/2281128338 conn(0x7f5e6409ba30 msgr2=0x7f5e64155b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:41.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.013+0000 7f5e69800640 1 -- 172.21.15.86:0/2281128338 shutdown_connections 2024-04-25T14:10:41.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.013+0000 7f5e69800640 1 -- 172.21.15.86:0/2281128338 wait complete. 2024-04-25T14:10:41.022 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:41.025+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:41.032 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T14:10:41.032 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T14:10:41.032 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T14:10:41.032 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:10:41.032 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T14:10:41.033 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T14:10:41.033 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T14:10:41.033 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:10:41.033 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T14:10:41.033 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T14:10:41.033 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T14:10:41.033 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:10:41.034 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T14:10:41.034 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T14:10:41.034 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T14:10:41.034 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:10:41.047 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T14:10:41.047 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T14:10:41.048 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T14:10:41.048 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T14:10:41.051 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T14:10:41.051 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T14:10:41.057 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T14:10:41.057 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T14:10:41.091 INFO:tasks.ceph.mds.a:Started 2024-04-25T14:10:41.091 DEBUG:teuthology.parallel:result is None 2024-04-25T14:10:41.092 INFO:tasks.ceph.mds.c:Started 2024-04-25T14:10:41.092 DEBUG:teuthology.parallel:result is None 2024-04-25T14:10:41.103 INFO:tasks.ceph.mds.d:Started 2024-04-25T14:10:41.104 DEBUG:teuthology.parallel:result is None 2024-04-25T14:10:41.105 INFO:tasks.ceph.mds.b:Started 2024-04-25T14:10:41.106 DEBUG:teuthology.parallel:result is None 2024-04-25T14:10:41.106 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T14:10:41.194 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-25T14:10:41.195 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-04-25T14:10:41.206 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2024-04-25T14:10:41.206 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2024-04-25T14:10:41.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.273+0000 7f76a9e00640 1 Processor -- start 2024-04-25T14:10:41.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.273+0000 7f76a9e00640 1 -- start start 2024-04-25T14:10:41.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.277+0000 7f76a9e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f76a40ccab0 0x7f76a40d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:41.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.277+0000 7f76a9e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f76a40b79c0 0x7f76a40d2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:41.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.277+0000 7f76a9e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76a40cf9d0 0x7f76a41617f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:41.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.277+0000 7f76a9e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f76a40b2620 con 0x7f76a40ccab0 2024-04-25T14:10:41.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.277+0000 7f76a9e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f76a40b3bd0 con 0x7f76a40b79c0 2024-04-25T14:10:41.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.277+0000 7f76a9e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f76a40b1cf0 con 0x7f76a40cf9d0 2024-04-25T14:10:41.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.277+0000 7f76a2a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f76a40ccab0 0x7f76a40d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:41.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.277+0000 7f76a2a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f76a40ccab0 0x7f76a40d23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53136/0 (socket says 172.21.15.86:53136) 2024-04-25T14:10:41.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.277+0000 7f76a2a00640 1 -- 172.21.15.86:0/1713732561 learned_addr learned my addr 172.21.15.86:0/1713732561 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:41.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.277+0000 7f76a2000640 1 --2- 172.21.15.86:0/1713732561 >> v2:172.21.15.155:3300/0 conn(0x7f76a40b79c0 0x7f76a40d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:41.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.277+0000 7f76a3400640 1 --2- 172.21.15.86:0/1713732561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76a40cf9d0 0x7f76a41617f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:41.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.277+0000 7f76a2a00640 1 -- 172.21.15.86:0/1713732561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76a40cf9d0 msgr2=0x7f76a41617f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:41.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.277+0000 7f76a2a00640 1 --2- 172.21.15.86:0/1713732561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76a40cf9d0 0x7f76a41617f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.277+0000 7f76a2a00640 1 -- 172.21.15.86:0/1713732561 >> v2:172.21.15.155:3300/0 conn(0x7f76a40b79c0 msgr2=0x7f76a40d2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:41.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.277+0000 7f76a2a00640 1 --2- 172.21.15.86:0/1713732561 >> v2:172.21.15.155:3300/0 conn(0x7f76a40b79c0 0x7f76a40d2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.277+0000 7f76a2a00640 1 -- 172.21.15.86:0/1713732561 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a40b1fd0 con 0x7f76a40ccab0 2024-04-25T14:10:41.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.281+0000 7f76a2a00640 1 --2- 172.21.15.86:0/1713732561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76a40ccab0 0x7f76a40d23d0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f76a40739f0 tx=0x7f769401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8190459e48edc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:41.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.281+0000 7f76a1600640 1 -- 172.21.15.86:0/1713732561 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76940b0070 con 0x7f76a40ccab0 2024-04-25T14:10:41.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.281+0000 7f76a1600640 1 -- 172.21.15.86:0/1713732561 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76940ab600 con 0x7f76a40ccab0 2024-04-25T14:10:41.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.281+0000 7f76a1600640 1 -- 172.21.15.86:0/1713732561 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76940ab9d0 con 0x7f76a40ccab0 2024-04-25T14:10:41.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.281+0000 7f76a9e00640 1 -- 172.21.15.86:0/1713732561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76a40ccab0 msgr2=0x7f76a40d23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:41.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.281+0000 7f76a9e00640 1 --2- 172.21.15.86:0/1713732561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76a40ccab0 0x7f76a40d23d0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f76a40739f0 tx=0x7f769401c260 comp rx=0 tx=0).stop 2024-04-25T14:10:41.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.281+0000 7f76a9e00640 1 -- 172.21.15.86:0/1713732561 shutdown_connections 2024-04-25T14:10:41.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.281+0000 7f76a9e00640 1 --2- 172.21.15.86:0/1713732561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76a40cf9d0 0x7f76a41617f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.281+0000 7f76a9e00640 1 --2- 172.21.15.86:0/1713732561 >> v2:172.21.15.155:3300/0 conn(0x7f76a40b79c0 0x7f76a40d2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.281+0000 7f76a9e00640 1 --2- 172.21.15.86:0/1713732561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76a40ccab0 0x7f76a40d23d0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.281+0000 7f76a9e00640 1 -- 172.21.15.86:0/1713732561 >> 172.21.15.86:0/1713732561 conn(0x7f76a409ba30 msgr2=0x7f76a40b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:41.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.281+0000 7f76a9e00640 1 -- 172.21.15.86:0/1713732561 shutdown_connections 2024-04-25T14:10:41.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.281+0000 7f76a9e00640 1 -- 172.21.15.86:0/1713732561 wait complete. 2024-04-25T14:10:41.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.281+0000 7f76a9e00640 1 Processor -- start 2024-04-25T14:10:41.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.281+0000 7f76a9e00640 1 -- start start 2024-04-25T14:10:41.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.285+0000 7f76a9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76a40b79c0 0x7f76a41589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:41.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.285+0000 7f76a9e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76a40ccab0 0x7f76a415abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:41.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.285+0000 7f76a9e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76a40cf9d0 0x7f76a40d0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:41.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.285+0000 7f76a9e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f76a40ce510 con 0x7f76a40b79c0 2024-04-25T14:10:41.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.285+0000 7f76a9e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f76a40ce390 con 0x7f76a40ccab0 2024-04-25T14:10:41.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.285+0000 7f76a9e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f76a40ce690 con 0x7f76a40cf9d0 2024-04-25T14:10:41.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.285+0000 7f76a2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76a40b79c0 0x7f76a41589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:41.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.285+0000 7f76a2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76a40b79c0 0x7f76a41589a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53148/0 (socket says 172.21.15.86:53148) 2024-04-25T14:10:41.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.285+0000 7f76a2a00640 1 -- 172.21.15.86:0/561089569 learned_addr learned my addr 172.21.15.86:0/561089569 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:41.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.285+0000 7f76a3400640 1 --2- 172.21.15.86:0/561089569 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76a40cf9d0 0x7f76a40d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:41.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.285+0000 7f76a2000640 1 --2- 172.21.15.86:0/561089569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76a40ccab0 0x7f76a415abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:41.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.285+0000 7f76a2a00640 1 -- 172.21.15.86:0/561089569 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76a40cf9d0 msgr2=0x7f76a40d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:41.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.285+0000 7f76a2a00640 1 --2- 172.21.15.86:0/561089569 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76a40cf9d0 0x7f76a40d0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.285+0000 7f76a2a00640 1 -- 172.21.15.86:0/561089569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76a40ccab0 msgr2=0x7f76a415abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:41.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.285+0000 7f76a2a00640 1 --2- 172.21.15.86:0/561089569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76a40ccab0 0x7f76a415abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.285+0000 7f76a2a00640 1 -- 172.21.15.86:0/561089569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a4158ee0 con 0x7f76a40b79c0 2024-04-25T14:10:41.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.285+0000 7f76a2a00640 1 --2- 172.21.15.86:0/561089569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76a40b79c0 0x7f76a41589a0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f769401cb00 tx=0x7f7694019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:41.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.285+0000 7f769b400640 1 -- 172.21.15.86:0/561089569 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76940b9070 con 0x7f76a40b79c0 2024-04-25T14:10:41.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.285+0000 7f769b400640 1 -- 172.21.15.86:0/561089569 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76940ab7b0 con 0x7f76a40b79c0 2024-04-25T14:10:41.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.285+0000 7f76a9e00640 1 -- 172.21.15.86:0/561089569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a415f8b0 con 0x7f76a40b79c0 2024-04-25T14:10:41.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.285+0000 7f769b400640 1 -- 172.21.15.86:0/561089569 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76940b0040 con 0x7f76a40b79c0 2024-04-25T14:10:41.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.285+0000 7f76a9e00640 1 -- 172.21.15.86:0/561089569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a415fe70 con 0x7f76a40b79c0 2024-04-25T14:10:41.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.285+0000 7f76a9e00640 1 -- 172.21.15.86:0/561089569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a40a04c0 con 0x7f76a40b79c0 2024-04-25T14:10:41.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.289+0000 7f769b400640 1 -- 172.21.15.86:0/561089569 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7f76940bc070 con 0x7f76a40b79c0 2024-04-25T14:10:41.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.289+0000 7f769b400640 1 --2- 172.21.15.86:0/561089569 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f768406f490 0x7f76840719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:41.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.289+0000 7f769b400640 1 -- 172.21.15.86:0/561089569 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5980+0+0 (secure 0 0 0) 0x7f76941282f0 con 0x7f76a40b79c0 2024-04-25T14:10:41.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.289+0000 7f76a2000640 1 --2- 172.21.15.86:0/561089569 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f768406f490 0x7f76840719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:41.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.289+0000 7f76a2000640 1 --2- 172.21.15.86:0/561089569 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f768406f490 0x7f76840719a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f76a415a240 tx=0x7f7690009190 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:41.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.293+0000 7f769b400640 1 -- 172.21.15.86:0/561089569 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7694016110 con 0x7f76a40b79c0 2024-04-25T14:10:41.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:41.469+0000 7f76a9e00640 1 -- 172.21.15.86:0/561089569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f76a40d2f90 con 0x7f76a40b79c0 2024-04-25T14:10:41.790 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:41.793+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:42.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.017+0000 7f769b400640 1 -- 172.21.15.86:0/561089569 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v120) ==== 97+0+0 (secure 0 0 0) 0x7f76940f4880 con 0x7f76a40b79c0 2024-04-25T14:10:42.016 INFO:teuthology.orchestra.run.smithi086.stderr: removed all blocklist entries 2024-04-25T14:10:42.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.021+0000 7f76a9e00640 1 -- 172.21.15.86:0/561089569 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f768406f490 msgr2=0x7f76840719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:42.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.021+0000 7f76a9e00640 1 --2- 172.21.15.86:0/561089569 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f768406f490 0x7f76840719a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f76a415a240 tx=0x7f7690009190 comp rx=0 tx=0).stop 2024-04-25T14:10:42.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.021+0000 7f76a9e00640 1 -- 172.21.15.86:0/561089569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76a40b79c0 msgr2=0x7f76a41589a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:42.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.021+0000 7f76a9e00640 1 --2- 172.21.15.86:0/561089569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76a40b79c0 0x7f76a41589a0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f769401cb00 tx=0x7f7694019000 comp rx=0 tx=0).stop 2024-04-25T14:10:42.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.021+0000 7f76a9e00640 1 -- 172.21.15.86:0/561089569 shutdown_connections 2024-04-25T14:10:42.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.021+0000 7f76a9e00640 1 --2- 172.21.15.86:0/561089569 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76a40cf9d0 0x7f76a40d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.021+0000 7f76a9e00640 1 --2- 172.21.15.86:0/561089569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76a40ccab0 0x7f76a415abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.021+0000 7f76a9e00640 1 --2- 172.21.15.86:0/561089569 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f768406f490 0x7f76840719a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.021+0000 7f76a9e00640 1 --2- 172.21.15.86:0/561089569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76a40b79c0 0x7f76a41589a0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.021+0000 7f76a9e00640 1 -- 172.21.15.86:0/561089569 >> 172.21.15.86:0/561089569 conn(0x7f76a409ba30 msgr2=0x7f76a40ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:42.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.021+0000 7f76a9e00640 1 -- 172.21.15.86:0/561089569 shutdown_connections 2024-04-25T14:10:42.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.021+0000 7f76a9e00640 1 -- 172.21.15.86:0/561089569 wait complete. 2024-04-25T14:10:42.042 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T14:10:42.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.197+0000 7f4094800640 1 Processor -- start 2024-04-25T14:10:42.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.197+0000 7f4094800640 1 -- start start 2024-04-25T14:10:42.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.201+0000 7f4094800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f408c0b6db0 0x7f408c161510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:42.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.201+0000 7f4094800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f408c161a50 0x7f408c072410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:42.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.201+0000 7f4094800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f408c0b73f0 con 0x7f408c0b69a0 2024-04-25T14:10:42.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.201+0000 7f4094800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f408c0b71c0 con 0x7f408c161a50 2024-04-25T14:10:42.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.201+0000 7f4094800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f408c15ddd0 con 0x7f408c0b6db0 2024-04-25T14:10:42.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.201+0000 7f4091e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f408c0b69a0 0x7f408c06ef30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36416/0 (socket says 172.21.15.86:36416) 2024-04-25T14:10:42.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.201+0000 7f4091e00640 1 -- 172.21.15.86:0/1341992619 learned_addr learned my addr 172.21.15.86:0/1341992619 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:42.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.201+0000 7f4092800640 1 --2- 172.21.15.86:0/1341992619 >> v2:172.21.15.86:3300/0 conn(0x7f408c161a50 0x7f408c072410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:42.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.201+0000 7f4091400640 1 --2- 172.21.15.86:0/1341992619 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f408c0b6db0 0x7f408c161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:42.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.201+0000 7f4090a00640 1 -- 172.21.15.86:0/1341992619 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 489547516 0 0) 0x7f408c0b73f0 con 0x7f408c0b69a0 2024-04-25T14:10:42.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.201+0000 7f4090a00640 1 -- 172.21.15.86:0/1341992619 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4074003530 con 0x7f408c0b69a0 2024-04-25T14:10:42.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.201+0000 7f4092800640 1 -- 172.21.15.86:0/1341992619 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f408c0b6db0 msgr2=0x7f408c161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:42.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.201+0000 7f4092800640 1 --2- 172.21.15.86:0/1341992619 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f408c0b6db0 0x7f408c161510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.201+0000 7f4092800640 1 -- 172.21.15.86:0/1341992619 >> v1:172.21.15.86:6789/0 conn(0x7f408c0b69a0 legacy=0x7f408c06ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:42.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.201+0000 7f4092800640 1 -- 172.21.15.86:0/1341992619 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408c0b76d0 con 0x7f408c161a50 2024-04-25T14:10:42.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.201+0000 7f4092800640 1 --2- 172.21.15.86:0/1341992619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f408c161a50 0x7f408c072410 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f4088009d90 tx=0x7f408801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682f246bbd453caf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:42.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.205+0000 7f4090a00640 1 -- 172.21.15.86:0/1341992619 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40880b2070 con 0x7f408c161a50 2024-04-25T14:10:42.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.205+0000 7f4090a00640 1 -- 172.21.15.86:0/1341992619 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40880a58b0 con 0x7f408c161a50 2024-04-25T14:10:42.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.205+0000 7f4090a00640 1 -- 172.21.15.86:0/1341992619 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40880a5c80 con 0x7f408c161a50 2024-04-25T14:10:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.205+0000 7f4094800640 1 -- 172.21.15.86:0/1341992619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f408c161a50 msgr2=0x7f408c072410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.205+0000 7f4094800640 1 --2- 172.21.15.86:0/1341992619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f408c161a50 0x7f408c072410 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f4088009d90 tx=0x7f408801c8e0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.205+0000 7f4094800640 1 -- 172.21.15.86:0/1341992619 shutdown_connections 2024-04-25T14:10:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.205+0000 7f4094800640 1 --2- 172.21.15.86:0/1341992619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f408c161a50 0x7f408c072410 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.205+0000 7f4094800640 1 --2- 172.21.15.86:0/1341992619 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f408c0b6db0 0x7f408c161510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.205+0000 7f4094800640 1 -- 172.21.15.86:0/1341992619 >> 172.21.15.86:0/1341992619 conn(0x7f408c0b5100 msgr2=0x7f408c0b5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.205+0000 7f4094800640 1 -- 172.21.15.86:0/1341992619 shutdown_connections 2024-04-25T14:10:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.205+0000 7f4094800640 1 -- 172.21.15.86:0/1341992619 wait complete. 2024-04-25T14:10:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.205+0000 7f4094800640 1 Processor -- start 2024-04-25T14:10:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.205+0000 7f4094800640 1 -- start start 2024-04-25T14:10:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.209+0000 7f4094800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f408c0b69a0 0x7f408c13fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.209+0000 7f4094800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f408c0b6db0 0x7f408c154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.209+0000 7f4094800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f408c161a50 0x7f408c142380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.209+0000 7f4094800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f408c06f7c0 con 0x7f408c161a50 2024-04-25T14:10:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.209+0000 7f4094800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f408c06f640 con 0x7f408c0b69a0 2024-04-25T14:10:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.209+0000 7f4094800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f408c073100 con 0x7f408c0b6db0 2024-04-25T14:10:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.209+0000 7f4092800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f408c161a50 0x7f408c142380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.209+0000 7f4092800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f408c161a50 0x7f408c142380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53166/0 (socket says 172.21.15.86:53166) 2024-04-25T14:10:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.209+0000 7f4092800640 1 -- 172.21.15.86:0/3665321336 learned_addr learned my addr 172.21.15.86:0/3665321336 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.209+0000 7f4091e00640 1 --2- 172.21.15.86:0/3665321336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f408c0b69a0 0x7f408c13fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.209+0000 7f4091400640 1 --2- 172.21.15.86:0/3665321336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f408c0b6db0 0x7f408c154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.209+0000 7f4092800640 1 -- 172.21.15.86:0/3665321336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f408c0b6db0 msgr2=0x7f408c154e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.209+0000 7f4092800640 1 --2- 172.21.15.86:0/3665321336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f408c0b6db0 0x7f408c154e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.209+0000 7f4092800640 1 -- 172.21.15.86:0/3665321336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f408c0b69a0 msgr2=0x7f408c13fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.209+0000 7f4092800640 1 --2- 172.21.15.86:0/3665321336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f408c0b69a0 0x7f408c13fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.209+0000 7f4092800640 1 -- 172.21.15.86:0/3665321336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408c140060 con 0x7f408c161a50 2024-04-25T14:10:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.209+0000 7f4091400640 1 --2- 172.21.15.86:0/3665321336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f408c0b6db0 0x7f408c154e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.209+0000 7f4092800640 1 --2- 172.21.15.86:0/3665321336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f408c161a50 0x7f408c142380 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f408801af90 tx=0x7f4088019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.209+0000 7f4086000640 1 -- 172.21.15.86:0/3665321336 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40880b2050 con 0x7f408c161a50 2024-04-25T14:10:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.209+0000 7f4086000640 1 -- 172.21.15.86:0/3665321336 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40880be070 con 0x7f408c161a50 2024-04-25T14:10:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.209+0000 7f4094800640 1 -- 172.21.15.86:0/3665321336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f408c155b30 con 0x7f408c161a50 2024-04-25T14:10:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.209+0000 7f4086000640 1 -- 172.21.15.86:0/3665321336 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40880ad850 con 0x7f408c161a50 2024-04-25T14:10:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.209+0000 7f4094800640 1 -- 172.21.15.86:0/3665321336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f408c1560f0 con 0x7f408c161a50 2024-04-25T14:10:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.209+0000 7f4094800640 1 -- 172.21.15.86:0/3665321336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f408c072200 con 0x7f408c161a50 2024-04-25T14:10:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.213+0000 7f4086000640 1 -- 172.21.15.86:0/3665321336 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7f40880ad9f0 con 0x7f408c161a50 2024-04-25T14:10:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.213+0000 7f4086000640 1 --2- 172.21.15.86:0/3665321336 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f407006f490 0x7f40700719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.213+0000 7f4086000640 1 -- 172.21.15.86:0/3665321336 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5249+0+0 (secure 0 0 0) 0x7f408812a020 con 0x7f408c161a50 2024-04-25T14:10:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.213+0000 7f4091e00640 1 --2- 172.21.15.86:0/3665321336 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f407006f490 0x7f40700719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.213+0000 7f4091e00640 1 --2- 172.21.15.86:0/3665321336 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f407006f490 0x7f40700719a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4080005890 tx=0x7f4080009040 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:42.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.217+0000 7f4086000640 1 -- 172.21.15.86:0/3665321336 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4088016560 con 0x7f408c161a50 2024-04-25T14:10:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.441+0000 7f4094800640 1 -- 172.21.15.86:0/3665321336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f408c076bf0 con 0x7f408c161a50 2024-04-25T14:10:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.441+0000 7f4086000640 1 -- 172.21.15.86:0/3665321336 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v27) ==== 80+0+5238 (secure 0 0 0) 0x7f40880f7170 con 0x7f408c161a50 2024-04-25T14:10:42.438 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:42.438 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:10:42.438 INFO:teuthology.orchestra.run.smithi086.stdout: "auth_dump": [ 2024-04-25T14:10:42.438 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:42.438 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.a", 2024-04-25T14:10:42.438 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmWk5fAxAAKEBKhWewpl23MUcvMi8qDg==", 2024-04-25T14:10:42.438 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:42.438 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:10:42.438 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:10:42.438 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:10:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.b", 2024-04-25T14:10:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmDhzOEBAABNys/NAEhucQ7A+HzQlzQw==", 2024-04-25T14:10:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:10:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:10:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:10:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:42.439 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.c", 2024-04-25T14:10:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmA/hNCxAAclDGbIjLfjqyXn0MwX/IkA==", 2024-04-25T14:10:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:10:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:10:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:10:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.d", 2024-04-25T14:10:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmemCcGBAAGrCx6C850qRJSWYjebxOBA==", 2024-04-25T14:10:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:10:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:10:42.440 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:10:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.0", 2024-04-25T14:10:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCOYipm9etgGxAAemg9CqXN0ShduDtDuKF2Zw==", 2024-04-25T14:10:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:10:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:10:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:42.441 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.1", 2024-04-25T14:10:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCRYipmBCiNJhAABAWHk8NEGVvqieL2klGhmA==", 2024-04-25T14:10:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:10:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:10:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.2", 2024-04-25T14:10:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCUYipm/XGYLxAAPXsw1svKseFzFiuDUjAsJQ==", 2024-04-25T14:10:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:10:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:10:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:42.442 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.3", 2024-04-25T14:10:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCXYipmlSRaORAAwpKyzZDdVYHWl2pAsQjmpw==", 2024-04-25T14:10:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:10:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:10:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.4", 2024-04-25T14:10:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCdYipmBI3FMBAAV2NdggfRSFnQvrcOC2YqiA==", 2024-04-25T14:10:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:42.443 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:10:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:10:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.5", 2024-04-25T14:10:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQChYipmctYxABAABwmOMgyYuOfRWD9KdHILng==", 2024-04-25T14:10:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:10:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:10:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:42.444 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.6", 2024-04-25T14:10:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCkYipmSL7TCRAAhFMWg1AQm54bB+x1GDcusA==", 2024-04-25T14:10:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:10:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:10:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.7", 2024-04-25T14:10:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCnYipmZ4F+FRAAEGaZN+byv/qJUFZb7WxTFg==", 2024-04-25T14:10:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:10:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:10:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:42.445 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.0", 2024-04-25T14:10:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipm5u4xHRAACDFe3Eb4KLrsdLj4eFe7mA==", 2024-04-25T14:10:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:10:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-25T14:10:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T14:10:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.1", 2024-04-25T14:10:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmDp1hJRAAQkBc5PdQoMTT7W4HahkTyg==", 2024-04-25T14:10:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:42.446 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:10:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-25T14:10:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw pool=fs2_data" 2024-04-25T14:10:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.2", 2024-04-25T14:10:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmhbhGIRAAxao/J0WY53IITLETFgwIVw==", 2024-04-25T14:10:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:10:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-25T14:10:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw pool=fs2_data" 2024-04-25T14:10:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.3", 2024-04-25T14:10:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmJsCBKRAAzXn2qDtblIpqt7U/TUUlyA==", 2024-04-25T14:10:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:10:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-25T14:10:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw pool=fs2_data" 2024-04-25T14:10:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.admin", 2024-04-25T14:10:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCKYipmTqbVIBAA1VpOtj24f2ibVEIT4EJwmw==", 2024-04-25T14:10:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-25T14:10:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:10:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:10:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:42.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:42.449 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:42.449 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.x", 2024-04-25T14:10:42.449 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCKYipmKMzAORAAtRyTNJ5HWi2XwuamoqwB0Q==", 2024-04-25T14:10:42.449 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:42.449 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-25T14:10:42.449 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-25T14:10:42.449 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:42.449 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:42.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:10:42.449 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:42.449 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.y", 2024-04-25T14:10:42.449 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCKYipmC7A7NRAAEwVNA0yxt3fwdHXrsV0dvg==", 2024-04-25T14:10:42.449 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:10:42.449 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-25T14:10:42.450 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-25T14:10:42.450 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:10:42.450 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:42.450 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:42.450 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-25T14:10:42.450 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:10:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.441+0000 7f4094800640 1 -- 172.21.15.86:0/3665321336 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f407006f490 msgr2=0x7f40700719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.441+0000 7f4094800640 1 --2- 172.21.15.86:0/3665321336 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f407006f490 0x7f40700719a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4080005890 tx=0x7f4080009040 comp rx=0 tx=0).stop 2024-04-25T14:10:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.441+0000 7f4094800640 1 -- 172.21.15.86:0/3665321336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f408c161a50 msgr2=0x7f408c142380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.441+0000 7f4094800640 1 --2- 172.21.15.86:0/3665321336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f408c161a50 0x7f408c142380 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f408801af90 tx=0x7f4088019680 comp rx=0 tx=0).stop 2024-04-25T14:10:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.441+0000 7f4094800640 1 -- 172.21.15.86:0/3665321336 shutdown_connections 2024-04-25T14:10:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.441+0000 7f4094800640 1 --2- 172.21.15.86:0/3665321336 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f407006f490 0x7f40700719a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.441+0000 7f4094800640 1 --2- 172.21.15.86:0/3665321336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f408c161a50 0x7f408c142380 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.441+0000 7f4094800640 1 --2- 172.21.15.86:0/3665321336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f408c0b6db0 0x7f408c154e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.441+0000 7f4094800640 1 --2- 172.21.15.86:0/3665321336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f408c0b69a0 0x7f408c13fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.441+0000 7f4094800640 1 -- 172.21.15.86:0/3665321336 >> 172.21.15.86:0/3665321336 conn(0x7f408c0b5100 msgr2=0x7f408c160ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.441+0000 7f4094800640 1 -- 172.21.15.86:0/3665321336 shutdown_connections 2024-04-25T14:10:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.441+0000 7f4094800640 1 -- 172.21.15.86:0/3665321336 wait complete. 2024-04-25T14:10:42.460 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:10:42.547 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:42.549+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:42.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.617+0000 7fbb0e600640 1 Processor -- start 2024-04-25T14:10:42.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.617+0000 7fbb0e600640 1 -- start start 2024-04-25T14:10:42.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.617+0000 7fbb0e600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbb080b59e0 0x7fbb080b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:42.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.617+0000 7fbb0e600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbb080aeea0 0x7fbb080af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:42.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.617+0000 7fbb0e600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb080b6330 con 0x7fbb0815faa0 2024-04-25T14:10:42.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.617+0000 7fbb0e600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbb0806d560 con 0x7fbb080aeea0 2024-04-25T14:10:42.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.617+0000 7fbb0e600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fbb0806c670 con 0x7fbb080b59e0 2024-04-25T14:10:42.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.617+0000 7fbb07e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbb080aeea0 0x7fbb080af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:42.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.617+0000 7fbb06a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbb080b59e0 0x7fbb080b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:42.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.617+0000 7fbb07e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbb080aeea0 0x7fbb080af2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53174/0 (socket says 172.21.15.86:53174) 2024-04-25T14:10:42.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.617+0000 7fbb07e00640 1 -- 172.21.15.86:0/2690741048 learned_addr learned my addr 172.21.15.86:0/2690741048 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:42.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.617+0000 7fbb07400640 1 --1- 172.21.15.86:0/2690741048 >> v1:172.21.15.155:6789/0 conn(0x7fbb0815faa0 0x7fbb0806d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:54790/0 (socket says 172.21.15.86:54790) 2024-04-25T14:10:42.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.617+0000 7fbb07e00640 1 -- 172.21.15.86:0/2690741048 >> v2:172.21.15.155:3300/0 conn(0x7fbb080b59e0 msgr2=0x7fbb080b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:42.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.617+0000 7fbb07e00640 1 --2- 172.21.15.86:0/2690741048 >> v2:172.21.15.155:3300/0 conn(0x7fbb080b59e0 0x7fbb080b5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.617+0000 7fbb07e00640 1 -- 172.21.15.86:0/2690741048 >> v1:172.21.15.155:6789/0 conn(0x7fbb0815faa0 legacy=0x7fbb0806d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:42.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.617+0000 7fbb07e00640 1 -- 172.21.15.86:0/2690741048 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0806c950 con 0x7fbb080aeea0 2024-04-25T14:10:42.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.621+0000 7fbb07e00640 1 --2- 172.21.15.86:0/2690741048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb080aeea0 0x7fbb080af2b0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fbb080b7250 tx=0x7fbb0001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7dc086d9dee381e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:42.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.621+0000 7fbb06000640 1 -- 172.21.15.86:0/2690741048 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb000a58e0 con 0x7fbb080aeea0 2024-04-25T14:10:42.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.621+0000 7fbb06000640 1 -- 172.21.15.86:0/2690741048 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb000a5a80 con 0x7fbb080aeea0 2024-04-25T14:10:42.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.621+0000 7fbb06000640 1 -- 172.21.15.86:0/2690741048 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb000a5e50 con 0x7fbb080aeea0 2024-04-25T14:10:42.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.621+0000 7fbb0e600640 1 -- 172.21.15.86:0/2690741048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb080aeea0 msgr2=0x7fbb080af2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:42.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.621+0000 7fbb0e600640 1 --2- 172.21.15.86:0/2690741048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb080aeea0 0x7fbb080af2b0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fbb080b7250 tx=0x7fbb0001c9f0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.621+0000 7fbb0e600640 1 -- 172.21.15.86:0/2690741048 shutdown_connections 2024-04-25T14:10:42.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.621+0000 7fbb0e600640 1 --2- 172.21.15.86:0/2690741048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb080aeea0 0x7fbb080af2b0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.621+0000 7fbb0e600640 1 --2- 172.21.15.86:0/2690741048 >> v2:172.21.15.155:3300/0 conn(0x7fbb080b59e0 0x7fbb080b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.621+0000 7fbb0e600640 1 -- 172.21.15.86:0/2690741048 >> 172.21.15.86:0/2690741048 conn(0x7fbb080d2230 msgr2=0x7fbb080d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:42.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.621+0000 7fbb0e600640 1 -- 172.21.15.86:0/2690741048 shutdown_connections 2024-04-25T14:10:42.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.621+0000 7fbb0e600640 1 -- 172.21.15.86:0/2690741048 wait complete. 2024-04-25T14:10:42.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.625+0000 7fbb0e600640 1 Processor -- start 2024-04-25T14:10:42.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.625+0000 7fbb0e600640 1 -- start start 2024-04-25T14:10:42.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.625+0000 7fbb0e600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb080aeea0 0x7fbb081545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:42.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.625+0000 7fbb0e600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb080b59e0 0x7fbb08140510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:42.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.625+0000 7fbb0e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb0815faa0 0x7fbb08163470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:42.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.625+0000 7fbb0e600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb08160030 con 0x7fbb0815faa0 2024-04-25T14:10:42.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.625+0000 7fbb0e600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0815feb0 con 0x7fbb080aeea0 2024-04-25T14:10:42.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.625+0000 7fbb0e600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbb08070f30 con 0x7fbb080b59e0 2024-04-25T14:10:42.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.625+0000 7fbb07e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb0815faa0 0x7fbb08163470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:42.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.625+0000 7fbb07400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb080aeea0 0x7fbb081545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:42.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.625+0000 7fbb07e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb0815faa0 0x7fbb08163470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53176/0 (socket says 172.21.15.86:53176) 2024-04-25T14:10:42.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.625+0000 7fbb07e00640 1 -- 172.21.15.86:0/393765908 learned_addr learned my addr 172.21.15.86:0/393765908 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:42.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.625+0000 7fbb06a00640 1 --2- 172.21.15.86:0/393765908 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb080b59e0 0x7fbb08140510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:42.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.625+0000 7fbb07e00640 1 -- 172.21.15.86:0/393765908 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb080b59e0 msgr2=0x7fbb08140510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:42.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.625+0000 7fbb07e00640 1 --2- 172.21.15.86:0/393765908 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb080b59e0 0x7fbb08140510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.625+0000 7fbb07e00640 1 -- 172.21.15.86:0/393765908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb080aeea0 msgr2=0x7fbb081545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:42.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.625+0000 7fbb07e00640 1 --2- 172.21.15.86:0/393765908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb080aeea0 0x7fbb081545f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.625+0000 7fbb07e00640 1 -- 172.21.15.86:0/393765908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb08154b30 con 0x7fbb0815faa0 2024-04-25T14:10:42.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.629+0000 7fbb07e00640 1 --2- 172.21.15.86:0/393765908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb0815faa0 0x7fbb08163470 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fbb00019600 tx=0x7fbb00019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:42.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.629+0000 7fbaff400640 1 -- 172.21.15.86:0/393765908 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb000a58e0 con 0x7fbb0815faa0 2024-04-25T14:10:42.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.629+0000 7fbaff400640 1 -- 172.21.15.86:0/393765908 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb00007bd0 con 0x7fbb0815faa0 2024-04-25T14:10:42.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.629+0000 7fbaff400640 1 -- 172.21.15.86:0/393765908 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb00009510 con 0x7fbb0815faa0 2024-04-25T14:10:42.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.629+0000 7fbb0e600640 1 -- 172.21.15.86:0/393765908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb08159b30 con 0x7fbb0815faa0 2024-04-25T14:10:42.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.629+0000 7fbb0e600640 1 -- 172.21.15.86:0/393765908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb0815a0f0 con 0x7fbb0815faa0 2024-04-25T14:10:42.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.629+0000 7fbb0e600640 1 -- 172.21.15.86:0/393765908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb08074cf0 con 0x7fbb0815faa0 2024-04-25T14:10:42.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.629+0000 7fbaff400640 1 -- 172.21.15.86:0/393765908 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7fbb000c0020 con 0x7fbb0815faa0 2024-04-25T14:10:42.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.633+0000 7fbaff400640 1 --2- 172.21.15.86:0/393765908 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fbae806f490 0x7fbae80719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:42.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.633+0000 7fbaff400640 1 -- 172.21.15.86:0/393765908 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5249+0+0 (secure 0 0 0) 0x7fbb0012ab10 con 0x7fbb0815faa0 2024-04-25T14:10:42.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.633+0000 7fbb07400640 1 --2- 172.21.15.86:0/393765908 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fbae806f490 0x7fbae80719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:42.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.633+0000 7fbb07400640 1 --2- 172.21.15.86:0/393765908 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fbae806f490 0x7fbae80719a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8000f70 tx=0x7fbaf8007e90 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:42.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.637+0000 7fbaff400640 1 -- 172.21.15.86:0/393765908 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbb000165e0 con 0x7fbb0815faa0 2024-04-25T14:10:42.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.797+0000 7fbb0e600640 1 -- 172.21.15.86:0/393765908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbb080b55b0 con 0x7fbb0815faa0 2024-04-25T14:10:42.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.801+0000 7fbaff400640 1 -- 172.21.15.86:0/393765908 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) ==== 74+0+13174 (secure 0 0 0) 0x7fbb000f7620 con 0x7fbb0815faa0 2024-04-25T14:10:42.798 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:42.798 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":122,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:10:42.031706+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:10:42.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.801+0000 7fbb0e600640 1 -- 172.21.15.86:0/393765908 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fbae806f490 msgr2=0x7fbae80719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:42.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.801+0000 7fbb0e600640 1 --2- 172.21.15.86:0/393765908 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fbae806f490 0x7fbae80719a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8000f70 tx=0x7fbaf8007e90 comp rx=0 tx=0).stop 2024-04-25T14:10:42.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.801+0000 7fbb0e600640 1 -- 172.21.15.86:0/393765908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb0815faa0 msgr2=0x7fbb08163470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:42.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.801+0000 7fbb0e600640 1 --2- 172.21.15.86:0/393765908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb0815faa0 0x7fbb08163470 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fbb00019600 tx=0x7fbb00019790 comp rx=0 tx=0).stop 2024-04-25T14:10:42.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.801+0000 7fbb0e600640 1 -- 172.21.15.86:0/393765908 shutdown_connections 2024-04-25T14:10:42.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.801+0000 7fbb0e600640 1 --2- 172.21.15.86:0/393765908 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fbae806f490 0x7fbae80719a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.801+0000 7fbb0e600640 1 --2- 172.21.15.86:0/393765908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb0815faa0 0x7fbb08163470 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.801+0000 7fbb0e600640 1 --2- 172.21.15.86:0/393765908 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb080b59e0 0x7fbb08140510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.801+0000 7fbb0e600640 1 --2- 172.21.15.86:0/393765908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb080aeea0 0x7fbb081545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.801+0000 7fbb0e600640 1 -- 172.21.15.86:0/393765908 >> 172.21.15.86:0/393765908 conn(0x7fbb080d2230 msgr2=0x7fbb080b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:42.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.801+0000 7fbb0e600640 1 -- 172.21.15.86:0/393765908 shutdown_connections 2024-04-25T14:10:42.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.801+0000 7fbb0e600640 1 -- 172.21.15.86:0/393765908 wait complete. 2024-04-25T14:10:42.820 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:10:42.820 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:10:42.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.977+0000 7f7e5b400640 1 Processor -- start 2024-04-25T14:10:42.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.977+0000 7f7e5b400640 1 -- start start 2024-04-25T14:10:42.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.981+0000 7f7e5b400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e54070330 0x7f7e540d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:42.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.981+0000 7f7e5b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7e540ce790 0x7f7e5406f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:42.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.981+0000 7f7e5b400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e540d2030 con 0x7f7e540d1c20 2024-04-25T14:10:42.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.981+0000 7f7e5b400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7e5415e5c0 con 0x7f7e540ce790 2024-04-25T14:10:42.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.981+0000 7f7e5b400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7e5415e8a0 con 0x7f7e54070330 2024-04-25T14:10:42.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.981+0000 7f7e59400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7e540ce790 0x7f7e5406f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:42.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.981+0000 7f7e58a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e54070330 0x7f7e540d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:42.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.981+0000 7f7e59400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7e540ce790 0x7f7e5406f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53180/0 (socket says 172.21.15.86:53180) 2024-04-25T14:10:42.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.981+0000 7f7e59400640 1 -- 172.21.15.86:0/242108276 learned_addr learned my addr 172.21.15.86:0/242108276 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:42.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.981+0000 7f7e53e00640 1 --1- 172.21.15.86:0/242108276 >> v1:172.21.15.155:6789/0 conn(0x7f7e540d1c20 0x7f7e5406ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:54794/0 (socket says 172.21.15.86:54794) 2024-04-25T14:10:42.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.981+0000 7f7e59400640 1 -- 172.21.15.86:0/242108276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e54070330 msgr2=0x7f7e540d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:42.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.981+0000 7f7e59400640 1 --2- 172.21.15.86:0/242108276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e54070330 0x7f7e540d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.981+0000 7f7e59400640 1 -- 172.21.15.86:0/242108276 >> v1:172.21.15.155:6789/0 conn(0x7f7e540d1c20 legacy=0x7f7e5406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:42.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.981+0000 7f7e59400640 1 -- 172.21.15.86:0/242108276 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5415eb80 con 0x7f7e540ce790 2024-04-25T14:10:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.985+0000 7f7e59400640 1 --2- 172.21.15.86:0/242108276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e540ce790 0x7f7e5406f440 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c009e80 tx=0x7f7e4c01c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941f02ef2f798990 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.985+0000 7f7e53400640 1 -- 172.21.15.86:0/242108276 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c0a5640 con 0x7f7e540ce790 2024-04-25T14:10:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.985+0000 7f7e53400640 1 -- 172.21.15.86:0/242108276 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e4c0a57e0 con 0x7f7e540ce790 2024-04-25T14:10:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.985+0000 7f7e53400640 1 -- 172.21.15.86:0/242108276 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c0a5bb0 con 0x7f7e540ce790 2024-04-25T14:10:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.985+0000 7f7e5b400640 1 -- 172.21.15.86:0/242108276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e540ce790 msgr2=0x7f7e5406f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.985+0000 7f7e5b400640 1 --2- 172.21.15.86:0/242108276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e540ce790 0x7f7e5406f440 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c009e80 tx=0x7f7e4c01c980 comp rx=0 tx=0).stop 2024-04-25T14:10:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.985+0000 7f7e5b400640 1 -- 172.21.15.86:0/242108276 shutdown_connections 2024-04-25T14:10:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.985+0000 7f7e5b400640 1 --2- 172.21.15.86:0/242108276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e540ce790 0x7f7e5406f440 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.985+0000 7f7e5b400640 1 --2- 172.21.15.86:0/242108276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e54070330 0x7f7e540d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.985+0000 7f7e5b400640 1 -- 172.21.15.86:0/242108276 >> 172.21.15.86:0/242108276 conn(0x7f7e540b4650 msgr2=0x7f7e540b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:42.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.985+0000 7f7e5b400640 1 -- 172.21.15.86:0/242108276 shutdown_connections 2024-04-25T14:10:42.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.985+0000 7f7e5b400640 1 -- 172.21.15.86:0/242108276 wait complete. 2024-04-25T14:10:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.989+0000 7f7e5b400640 1 Processor -- start 2024-04-25T14:10:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.989+0000 7f7e5b400640 1 -- start start 2024-04-25T14:10:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.989+0000 7f7e5b400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e54070330 0x7f7e541f9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.989+0000 7f7e5b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e540ce790 0x7f7e5415b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.989+0000 7f7e5b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e540d1c20 0x7f7e540738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.989+0000 7f7e5b400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e540708c0 con 0x7f7e540d1c20 2024-04-25T14:10:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.989+0000 7f7e5b400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7e54070740 con 0x7f7e540ce790 2024-04-25T14:10:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.989+0000 7f7e5b400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7e5416afc0 con 0x7f7e54070330 2024-04-25T14:10:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.989+0000 7f7e59400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e540d1c20 0x7f7e540738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.989+0000 7f7e59400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e540d1c20 0x7f7e540738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53182/0 (socket says 172.21.15.86:53182) 2024-04-25T14:10:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.989+0000 7f7e59400640 1 -- 172.21.15.86:0/2823059503 learned_addr learned my addr 172.21.15.86:0/2823059503 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.989+0000 7f7e53e00640 1 --2- 172.21.15.86:0/2823059503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e540ce790 0x7f7e5415b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.989+0000 7f7e58a00640 1 --2- 172.21.15.86:0/2823059503 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e54070330 0x7f7e541f9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:42.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.989+0000 7f7e59400640 1 -- 172.21.15.86:0/2823059503 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e54070330 msgr2=0x7f7e541f9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:42.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.989+0000 7f7e59400640 1 --2- 172.21.15.86:0/2823059503 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e54070330 0x7f7e541f9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.989+0000 7f7e59400640 1 -- 172.21.15.86:0/2823059503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e540ce790 msgr2=0x7f7e5415b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:42.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.989+0000 7f7e59400640 1 --2- 172.21.15.86:0/2823059503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e540ce790 0x7f7e5415b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:42.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.989+0000 7f7e59400640 1 -- 172.21.15.86:0/2823059503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5415c010 con 0x7f7e540d1c20 2024-04-25T14:10:42.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.989+0000 7f7e59400640 1 --2- 172.21.15.86:0/2823059503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e540d1c20 0x7f7e540738d0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c0af040 tx=0x7f7e4c019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.989+0000 7f7e50c00640 1 -- 172.21.15.86:0/2823059503 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c009510 con 0x7f7e540d1c20 2024-04-25T14:10:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.989+0000 7f7e50c00640 1 -- 172.21.15.86:0/2823059503 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e4c0b8470 con 0x7f7e540d1c20 2024-04-25T14:10:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.989+0000 7f7e5b400640 1 -- 172.21.15.86:0/2823059503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e5415c2f0 con 0x7f7e540d1c20 2024-04-25T14:10:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.989+0000 7f7e50c00640 1 -- 172.21.15.86:0/2823059503 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c0b8860 con 0x7f7e540d1c20 2024-04-25T14:10:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.989+0000 7f7e5b400640 1 -- 172.21.15.86:0/2823059503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e5415c8b0 con 0x7f7e540d1c20 2024-04-25T14:10:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.993+0000 7f7e5b400640 1 -- 172.21.15.86:0/2823059503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e54069d00 con 0x7f7e540d1c20 2024-04-25T14:10:42.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.993+0000 7f7e50c00640 1 -- 172.21.15.86:0/2823059503 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7f7e4c0c0020 con 0x7f7e540d1c20 2024-04-25T14:10:42.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.993+0000 7f7e50c00640 1 --2- 172.21.15.86:0/2823059503 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f7e3406f4c0 0x7f7e340719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:42.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.993+0000 7f7e50c00640 1 -- 172.21.15.86:0/2823059503 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5249+0+0 (secure 0 0 0) 0x7f7e4c129bd0 con 0x7f7e540d1c20 2024-04-25T14:10:42.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.993+0000 7f7e58a00640 1 --2- 172.21.15.86:0/2823059503 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f7e3406f4c0 0x7f7e340719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:42.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.993+0000 7f7e58a00640 1 --2- 172.21.15.86:0/2823059503 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f7e3406f4c0 0x7f7e340719d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7e54072200 tx=0x7f7e48009190 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:42.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:42.997+0000 7f7e50c00640 1 -- 172.21.15.86:0/2823059503 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7e4c016600 con 0x7f7e540d1c20 2024-04-25T14:10:43.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.165+0000 7f7e5b400640 1 -- 172.21.15.86:0/2823059503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7e540aea30 con 0x7f7e540d1c20 2024-04-25T14:10:43.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.169+0000 7f7e50c00640 1 -- 172.21.15.86:0/2823059503 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) ==== 93+0+10 (secure 0 0 0) 0x7f7e4c0f6540 con 0x7f7e540d1c20 2024-04-25T14:10:43.165 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:10:43.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.169+0000 7f7e5b400640 1 -- 172.21.15.86:0/2823059503 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f7e3406f4c0 msgr2=0x7f7e340719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:43.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.169+0000 7f7e5b400640 1 --2- 172.21.15.86:0/2823059503 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f7e3406f4c0 0x7f7e340719d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7e54072200 tx=0x7f7e48009190 comp rx=0 tx=0).stop 2024-04-25T14:10:43.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.169+0000 7f7e5b400640 1 -- 172.21.15.86:0/2823059503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e540d1c20 msgr2=0x7f7e540738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:43.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.169+0000 7f7e5b400640 1 --2- 172.21.15.86:0/2823059503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e540d1c20 0x7f7e540738d0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c0af040 tx=0x7f7e4c019720 comp rx=0 tx=0).stop 2024-04-25T14:10:43.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.169+0000 7f7e5b400640 1 -- 172.21.15.86:0/2823059503 shutdown_connections 2024-04-25T14:10:43.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.169+0000 7f7e5b400640 1 --2- 172.21.15.86:0/2823059503 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f7e3406f4c0 0x7f7e340719d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:43.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.169+0000 7f7e5b400640 1 --2- 172.21.15.86:0/2823059503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e540d1c20 0x7f7e540738d0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:43.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.169+0000 7f7e5b400640 1 --2- 172.21.15.86:0/2823059503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e540ce790 0x7f7e5415b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:43.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.169+0000 7f7e5b400640 1 --2- 172.21.15.86:0/2823059503 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e54070330 0x7f7e541f9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:43.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.169+0000 7f7e5b400640 1 -- 172.21.15.86:0/2823059503 >> 172.21.15.86:0/2823059503 conn(0x7f7e540b4650 msgr2=0x7f7e54167ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:43.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.169+0000 7f7e5b400640 1 -- 172.21.15.86:0/2823059503 shutdown_connections 2024-04-25T14:10:43.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.169+0000 7f7e5b400640 1 -- 172.21.15.86:0/2823059503 wait complete. 2024-04-25T14:10:43.189 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T14:10:43.310 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:43.313+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:43.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.345+0000 7f7299200640 1 Processor -- start 2024-04-25T14:10:43.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.345+0000 7f7299200640 1 -- start start 2024-04-25T14:10:43.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.349+0000 7f7299200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7294071530 0x7f72940ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:43.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.349+0000 7f7299200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f729406e700 0x7f729406eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:43.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.349+0000 7f7299200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f729406f050 con 0x7f72940cdd70 2024-04-25T14:10:43.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.349+0000 7f7299200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f729406d940 con 0x7f729406e700 2024-04-25T14:10:43.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.349+0000 7f7299200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f729406dc20 con 0x7f7294071530 2024-04-25T14:10:43.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.349+0000 7f7292800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f729406e700 0x7f729406eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:43.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.349+0000 7f7292800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f729406e700 0x7f729406eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53188/0 (socket says 172.21.15.86:53188) 2024-04-25T14:10:43.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.349+0000 7f7292800640 1 -- 172.21.15.86:0/4104237631 learned_addr learned my addr 172.21.15.86:0/4104237631 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:43.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.349+0000 7f7289400640 1 --2- 172.21.15.86:0/4104237631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7294071530 0x7f72940ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:43.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.349+0000 7f7292800640 1 -- 172.21.15.86:0/4104237631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7294071530 msgr2=0x7f72940ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:43.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.349+0000 7f7292800640 1 --2- 172.21.15.86:0/4104237631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7294071530 0x7f72940ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:43.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.349+0000 7f7292800640 1 -- 172.21.15.86:0/4104237631 >> v1:172.21.15.155:6789/0 conn(0x7f72940cdd70 legacy=0x7f7294070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:43.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.349+0000 7f7292800640 1 -- 172.21.15.86:0/4104237631 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729406df00 con 0x7f729406e700 2024-04-25T14:10:43.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.353+0000 7f7292800640 1 --2- 172.21.15.86:0/4104237631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f729406e700 0x7f729406eb10 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f728c009e50 tx=0x7f728c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b81647e69d48867 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:43.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.353+0000 7f7291400640 1 -- 172.21.15.86:0/4104237631 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c0a56b0 con 0x7f729406e700 2024-04-25T14:10:43.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.353+0000 7f7291400640 1 -- 172.21.15.86:0/4104237631 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f728c0a5850 con 0x7f729406e700 2024-04-25T14:10:43.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.353+0000 7f7291400640 1 -- 172.21.15.86:0/4104237631 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c0a5c20 con 0x7f729406e700 2024-04-25T14:10:43.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.353+0000 7f7299200640 1 -- 172.21.15.86:0/4104237631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f729406e700 msgr2=0x7f729406eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:43.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.353+0000 7f7299200640 1 --2- 172.21.15.86:0/4104237631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f729406e700 0x7f729406eb10 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f728c009e50 tx=0x7f728c01c9a0 comp rx=0 tx=0).stop 2024-04-25T14:10:43.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.353+0000 7f7299200640 1 -- 172.21.15.86:0/4104237631 shutdown_connections 2024-04-25T14:10:43.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.353+0000 7f7299200640 1 --2- 172.21.15.86:0/4104237631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f729406e700 0x7f729406eb10 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:43.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.353+0000 7f7299200640 1 --2- 172.21.15.86:0/4104237631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7294071530 0x7f72940ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:43.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.353+0000 7f7299200640 1 -- 172.21.15.86:0/4104237631 >> 172.21.15.86:0/4104237631 conn(0x7f72940cffc0 msgr2=0x7f72940d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:43.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.353+0000 7f7299200640 1 -- 172.21.15.86:0/4104237631 shutdown_connections 2024-04-25T14:10:43.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.353+0000 7f7299200640 1 -- 172.21.15.86:0/4104237631 wait complete. 2024-04-25T14:10:43.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.357+0000 7f7299200640 1 Processor -- start 2024-04-25T14:10:43.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.357+0000 7f7299200640 1 -- start start 2024-04-25T14:10:43.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.357+0000 7f7299200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f729406e700 0x7f729414a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:43.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.357+0000 7f7299200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72940cdd70 0x7f729414aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:43.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.357+0000 7f7299200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7294154470 0x7f7294160b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:43.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.357+0000 7f7299200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f729416cc00 con 0x7f7294154470 2024-04-25T14:10:43.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.357+0000 7f7299200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7294164380 con 0x7f72940cdd70 2024-04-25T14:10:43.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.357+0000 7f7299200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f729416cd80 con 0x7f729406e700 2024-04-25T14:10:43.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.357+0000 7f7292800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7294154470 0x7f7294160b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:43.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.357+0000 7f7292800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7294154470 0x7f7294160b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53204/0 (socket says 172.21.15.86:53204) 2024-04-25T14:10:43.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.357+0000 7f7292800640 1 -- 172.21.15.86:0/2630831439 learned_addr learned my addr 172.21.15.86:0/2630831439 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:43.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.357+0000 7f7289400640 1 --2- 172.21.15.86:0/2630831439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72940cdd70 0x7f729414aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:43.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.357+0000 7f7291e00640 1 --2- 172.21.15.86:0/2630831439 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f729406e700 0x7f729414a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:43.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.357+0000 7f7292800640 1 -- 172.21.15.86:0/2630831439 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f729406e700 msgr2=0x7f729414a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:43.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.357+0000 7f7292800640 1 --2- 172.21.15.86:0/2630831439 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f729406e700 0x7f729414a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:43.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.357+0000 7f7292800640 1 -- 172.21.15.86:0/2630831439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72940cdd70 msgr2=0x7f729414aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:43.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.357+0000 7f7292800640 1 --2- 172.21.15.86:0/2630831439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72940cdd70 0x7f729414aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:43.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.357+0000 7f7292800640 1 -- 172.21.15.86:0/2630831439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729415f620 con 0x7f7294154470 2024-04-25T14:10:43.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.357+0000 7f7291e00640 1 --2- 172.21.15.86:0/2630831439 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f729406e700 0x7f729414a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:43.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.357+0000 7f7292800640 1 --2- 172.21.15.86:0/2630831439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7294154470 0x7f7294160b80 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f728c01a5d0 tx=0x7f728c019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:43.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.357+0000 7f728aa00640 1 -- 172.21.15.86:0/2630831439 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c009510 con 0x7f7294154470 2024-04-25T14:10:43.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.357+0000 7f728aa00640 1 -- 172.21.15.86:0/2630831439 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f728c0be070 con 0x7f7294154470 2024-04-25T14:10:43.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.357+0000 7f7299200640 1 -- 172.21.15.86:0/2630831439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f729415f060 con 0x7f7294154470 2024-04-25T14:10:43.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.357+0000 7f728aa00640 1 -- 172.21.15.86:0/2630831439 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c0b88b0 con 0x7f7294154470 2024-04-25T14:10:43.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.357+0000 7f7299200640 1 -- 172.21.15.86:0/2630831439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7294160380 con 0x7f7294154470 2024-04-25T14:10:43.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.361+0000 7f7299200640 1 -- 172.21.15.86:0/2630831439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72940720e0 con 0x7f7294154470 2024-04-25T14:10:43.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.361+0000 7f728aa00640 1 -- 172.21.15.86:0/2630831439 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7f728c0b8a50 con 0x7f7294154470 2024-04-25T14:10:43.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.361+0000 7f728aa00640 1 --2- 172.21.15.86:0/2630831439 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f727006f4c0 0x7f72700719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:43.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.361+0000 7f728aa00640 1 -- 172.21.15.86:0/2630831439 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5249+0+0 (secure 0 0 0) 0x7f728c129f80 con 0x7f7294154470 2024-04-25T14:10:43.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.361+0000 7f7291e00640 1 --2- 172.21.15.86:0/2630831439 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f727006f4c0 0x7f72700719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:43.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.361+0000 7f7291e00640 1 --2- 172.21.15.86:0/2630831439 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f727006f4c0 0x7f72700719d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7284001650 tx=0x7f7284009040 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:43.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.365+0000 7f728aa00640 1 -- 172.21.15.86:0/2630831439 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f728c016620 con 0x7f7294154470 2024-04-25T14:10:43.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.529+0000 7f7299200640 1 -- 172.21.15.86:0/2630831439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7294076bf0 con 0x7f7294154470 2024-04-25T14:10:43.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.529+0000 7f728aa00640 1 -- 172.21.15.86:0/2630831439 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v122) ==== 84+0+65 (secure 0 0 0) 0x7f728c0f6fc0 con 0x7f7294154470 2024-04-25T14:10:43.528 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:43.528 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-04-25T14:10:43.528 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:10:43.528 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-04-25T14:10:43.528 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-04-25T14:10:43.528 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:10:43.528 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-04-25T14:10:43.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.533+0000 7f7299200640 1 -- 172.21.15.86:0/2630831439 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f727006f4c0 msgr2=0x7f72700719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:43.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.533+0000 7f7299200640 1 --2- 172.21.15.86:0/2630831439 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f727006f4c0 0x7f72700719d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7284001650 tx=0x7f7284009040 comp rx=0 tx=0).stop 2024-04-25T14:10:43.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.533+0000 7f7299200640 1 -- 172.21.15.86:0/2630831439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7294154470 msgr2=0x7f7294160b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:43.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.533+0000 7f7299200640 1 --2- 172.21.15.86:0/2630831439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7294154470 0x7f7294160b80 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f728c01a5d0 tx=0x7f728c019740 comp rx=0 tx=0).stop 2024-04-25T14:10:43.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.533+0000 7f7299200640 1 -- 172.21.15.86:0/2630831439 shutdown_connections 2024-04-25T14:10:43.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.533+0000 7f7299200640 1 --2- 172.21.15.86:0/2630831439 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f727006f4c0 0x7f72700719d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:43.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.533+0000 7f7299200640 1 --2- 172.21.15.86:0/2630831439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7294154470 0x7f7294160b80 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:43.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.533+0000 7f7299200640 1 --2- 172.21.15.86:0/2630831439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72940cdd70 0x7f729414aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:43.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.533+0000 7f7299200640 1 --2- 172.21.15.86:0/2630831439 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f729406e700 0x7f729414a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:43.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.533+0000 7f7299200640 1 -- 172.21.15.86:0/2630831439 >> 172.21.15.86:0/2630831439 conn(0x7f72940cffc0 msgr2=0x7f72940d18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:43.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.533+0000 7f7299200640 1 -- 172.21.15.86:0/2630831439 shutdown_connections 2024-04-25T14:10:43.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.533+0000 7f7299200640 1 -- 172.21.15.86:0/2630831439 wait complete. 2024-04-25T14:10:43.550 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T14:10:43.551 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T14:10:43.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.709+0000 7f1733e00640 1 Processor -- start 2024-04-25T14:10:43.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.709+0000 7f1733e00640 1 -- start start 2024-04-25T14:10:43.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.713+0000 7f1733e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f172c0700a0 0x7f172c0b6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:43.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.713+0000 7f1733e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f172c15dbd0 0x7f172c1617b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:43.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.713+0000 7f1733e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f172c15dfe0 con 0x7f172c0d25c0 2024-04-25T14:10:43.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.713+0000 7f1733e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f172c0ce4e0 con 0x7f172c0700a0 2024-04-25T14:10:43.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.713+0000 7f1733e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f172c0ce730 con 0x7f172c15dbd0 2024-04-25T14:10:43.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.713+0000 7f1731e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f172c15dbd0 0x7f172c1617b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:43.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.713+0000 7f1730a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f172c0700a0 0x7f172c0b6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:43.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.713+0000 7f1730a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f172c0700a0 0x7f172c0b6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:33790/0 (socket says 172.21.15.86:33790) 2024-04-25T14:10:43.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.713+0000 7f1730a00640 1 -- 172.21.15.86:0/676750403 learned_addr learned my addr 172.21.15.86:0/676750403 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:43.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.713+0000 7f172be00640 1 -- 172.21.15.86:0/676750403 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2959114475 0 0) 0x7f172c15dfe0 con 0x7f172c0d25c0 2024-04-25T14:10:43.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.713+0000 7f1731e00640 1 -- 172.21.15.86:0/676750403 >> v2:172.21.15.155:3300/0 conn(0x7f172c0700a0 msgr2=0x7f172c0b6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:43.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.713+0000 7f1731e00640 1 --2- 172.21.15.86:0/676750403 >> v2:172.21.15.155:3300/0 conn(0x7f172c0700a0 0x7f172c0b6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:43.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.713+0000 7f1731e00640 1 -- 172.21.15.86:0/676750403 >> v1:172.21.15.155:6789/0 conn(0x7f172c0d25c0 legacy=0x7f172c06f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:43.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.713+0000 7f1731e00640 1 -- 172.21.15.86:0/676750403 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172c0cdd80 con 0x7f172c15dbd0 2024-04-25T14:10:43.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.717+0000 7f1731e00640 1 --2- 172.21.15.86:0/676750403 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f172c15dbd0 0x7f172c1617b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1724009d90 tx=0x7f172401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd8187a28fdd744a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:43.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.717+0000 7f172be00640 1 -- 172.21.15.86:0/676750403 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17240b2070 con 0x7f172c15dbd0 2024-04-25T14:10:43.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.717+0000 7f172be00640 1 -- 172.21.15.86:0/676750403 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17240a58b0 con 0x7f172c15dbd0 2024-04-25T14:10:43.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.717+0000 7f172be00640 1 -- 172.21.15.86:0/676750403 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17240a5c80 con 0x7f172c15dbd0 2024-04-25T14:10:43.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.717+0000 7f1733e00640 1 -- 172.21.15.86:0/676750403 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f172c15dbd0 msgr2=0x7f172c1617b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:43.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.717+0000 7f1733e00640 1 --2- 172.21.15.86:0/676750403 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f172c15dbd0 0x7f172c1617b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1724009d90 tx=0x7f172401c970 comp rx=0 tx=0).stop 2024-04-25T14:10:43.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.717+0000 7f1733e00640 1 -- 172.21.15.86:0/676750403 shutdown_connections 2024-04-25T14:10:43.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.717+0000 7f1733e00640 1 --2- 172.21.15.86:0/676750403 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f172c15dbd0 0x7f172c1617b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:43.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.717+0000 7f1733e00640 1 --2- 172.21.15.86:0/676750403 >> v2:172.21.15.155:3300/0 conn(0x7f172c0700a0 0x7f172c0b6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:43.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.717+0000 7f1733e00640 1 -- 172.21.15.86:0/676750403 >> 172.21.15.86:0/676750403 conn(0x7f172c0b5060 msgr2=0x7f172c0b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:43.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.717+0000 7f1733e00640 1 -- 172.21.15.86:0/676750403 shutdown_connections 2024-04-25T14:10:43.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.717+0000 7f1733e00640 1 -- 172.21.15.86:0/676750403 wait complete. 2024-04-25T14:10:43.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.721+0000 7f1733e00640 1 Processor -- start 2024-04-25T14:10:43.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.721+0000 7f1733e00640 1 -- start start 2024-04-25T14:10:43.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.721+0000 7f1733e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f172c0d25c0 0x7f172c0b8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:43.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.721+0000 7f1733e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f172c15dbd0 0x7f172c0b8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:43.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.721+0000 7f1733e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f172c156d90 0x7f172c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:43.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.721+0000 7f1733e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f172c16abb0 con 0x7f172c0d25c0 2024-04-25T14:10:43.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.721+0000 7f1733e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f172c16aa30 con 0x7f172c15dbd0 2024-04-25T14:10:43.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.721+0000 7f1733e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f172c16ad30 con 0x7f172c156d90 2024-04-25T14:10:43.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.721+0000 7f1731400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f172c0d25c0 0x7f172c0b8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:43.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.721+0000 7f1730a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f172c15dbd0 0x7f172c0b8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:43.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.721+0000 7f1730a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f172c15dbd0 0x7f172c0b8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:33792/0 (socket says 172.21.15.86:33792) 2024-04-25T14:10:43.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.721+0000 7f1730a00640 1 -- 172.21.15.86:0/1780214687 learned_addr learned my addr 172.21.15.86:0/1780214687 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:43.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.721+0000 7f1731e00640 1 --2- 172.21.15.86:0/1780214687 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f172c156d90 0x7f172c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:43.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.721+0000 7f1730a00640 1 -- 172.21.15.86:0/1780214687 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f172c156d90 msgr2=0x7f172c06d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:43.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.721+0000 7f1730a00640 1 --2- 172.21.15.86:0/1780214687 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f172c156d90 0x7f172c06d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:43.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.721+0000 7f1730a00640 1 -- 172.21.15.86:0/1780214687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f172c0d25c0 msgr2=0x7f172c0b8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:43.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.721+0000 7f1730a00640 1 --2- 172.21.15.86:0/1780214687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f172c0d25c0 0x7f172c0b8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:43.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.721+0000 7f1730a00640 1 -- 172.21.15.86:0/1780214687 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172c06f650 con 0x7f172c15dbd0 2024-04-25T14:10:43.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.721+0000 7f1730a00640 1 --2- 172.21.15.86:0/1780214687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f172c15dbd0 0x7f172c0b8670 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f171800e7f0 tx=0x7f171800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:43.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.721+0000 7f1729600640 1 -- 172.21.15.86:0/1780214687 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718017070 con 0x7f172c15dbd0 2024-04-25T14:10:43.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.721+0000 7f1729600640 1 -- 172.21.15.86:0/1780214687 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f171800c940 con 0x7f172c15dbd0 2024-04-25T14:10:43.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.721+0000 7f1733e00640 1 -- 172.21.15.86:0/1780214687 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f172c0cc6d0 con 0x7f172c15dbd0 2024-04-25T14:10:43.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.721+0000 7f1729600640 1 -- 172.21.15.86:0/1780214687 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171800cd30 con 0x7f172c15dbd0 2024-04-25T14:10:43.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.721+0000 7f1733e00640 1 -- 172.21.15.86:0/1780214687 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f172c0ccc90 con 0x7f172c15dbd0 2024-04-25T14:10:43.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.725+0000 7f1733e00640 1 -- 172.21.15.86:0/1780214687 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f172c050750 con 0x7f172c15dbd0 2024-04-25T14:10:43.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.725+0000 7f1729600640 1 -- 172.21.15.86:0/1780214687 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7f171801a070 con 0x7f172c15dbd0 2024-04-25T14:10:43.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.725+0000 7f1729600640 1 --2- 172.21.15.86:0/1780214687 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f170c06f490 0x7f170c0719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:43.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.725+0000 7f1729600640 1 -- 172.21.15.86:0/1780214687 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5249+0+0 (secure 0 0 0) 0x7f171808f2e0 con 0x7f172c15dbd0 2024-04-25T14:10:43.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.725+0000 7f1731400640 1 --2- 172.21.15.86:0/1780214687 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f170c06f490 0x7f170c0719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:43.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.725+0000 7f1731400640 1 --2- 172.21.15.86:0/1780214687 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f170c06f490 0x7f170c0719a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1720007fd0 tx=0x7f1720009040 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:43.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.729+0000 7f1729600640 1 -- 172.21.15.86:0/1780214687 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f171805bd60 con 0x7f172c15dbd0 2024-04-25T14:10:43.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:43.897+0000 7f1733e00640 1 -- 172.21.15.86:0/1780214687 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f172c0d1440 con 0x7f172c15dbd0 2024-04-25T14:10:44.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.029+0000 7f1729600640 1 -- 172.21.15.86:0/1780214687 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v123) ==== 138+0+0 (secure 0 0 0) 0x7f1718060060 con 0x7f172c15dbd0 2024-04-25T14:10:44.026 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-04-25T14:10:44.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.033+0000 7f1733e00640 1 -- 172.21.15.86:0/1780214687 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f170c06f490 msgr2=0x7f170c0719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:44.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.033+0000 7f1733e00640 1 --2- 172.21.15.86:0/1780214687 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f170c06f490 0x7f170c0719a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1720007fd0 tx=0x7f1720009040 comp rx=0 tx=0).stop 2024-04-25T14:10:44.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.033+0000 7f1733e00640 1 -- 172.21.15.86:0/1780214687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f172c15dbd0 msgr2=0x7f172c0b8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:44.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.033+0000 7f1733e00640 1 --2- 172.21.15.86:0/1780214687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f172c15dbd0 0x7f172c0b8670 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f171800e7f0 tx=0x7f171800ecb0 comp rx=0 tx=0).stop 2024-04-25T14:10:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.033+0000 7f1733e00640 1 -- 172.21.15.86:0/1780214687 shutdown_connections 2024-04-25T14:10:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.033+0000 7f1733e00640 1 --2- 172.21.15.86:0/1780214687 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f172c156d90 0x7f172c06d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.033+0000 7f1733e00640 1 --2- 172.21.15.86:0/1780214687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f172c15dbd0 0x7f172c0b8670 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.033+0000 7f1733e00640 1 --2- 172.21.15.86:0/1780214687 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f170c06f490 0x7f170c0719a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.033+0000 7f1733e00640 1 --2- 172.21.15.86:0/1780214687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f172c0d25c0 0x7f172c0b8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.033+0000 7f1733e00640 1 -- 172.21.15.86:0/1780214687 >> 172.21.15.86:0/1780214687 conn(0x7f172c0b5060 msgr2=0x7f172c15a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.033+0000 7f1733e00640 1 -- 172.21.15.86:0/1780214687 shutdown_connections 2024-04-25T14:10:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.033+0000 7f1733e00640 1 -- 172.21.15.86:0/1780214687 wait complete. 2024-04-25T14:10:44.051 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T14:10:44.072 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:44.073+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:44.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.217+0000 7f0f85c00640 1 Processor -- start 2024-04-25T14:10:44.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.217+0000 7f0f85c00640 1 -- start start 2024-04-25T14:10:44.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.217+0000 7f0f85c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f80070330 0x7f0f800d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:44.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.217+0000 7f0f85c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0f800ce790 0x7f0f8006f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:44.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.217+0000 7f0f85c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f800d2030 con 0x7f0f800d1c20 2024-04-25T14:10:44.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.217+0000 7f0f85c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0f8015e5c0 con 0x7f0f800ce790 2024-04-25T14:10:44.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.217+0000 7f0f85c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0f8015e8a0 con 0x7f0f80070330 2024-04-25T14:10:44.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.221+0000 7f0f7ea00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f80070330 0x7f0f800d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:44.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.221+0000 7f0f7ea00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f80070330 0x7f0f800d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:42054/0 (socket says 172.21.15.86:42054) 2024-04-25T14:10:44.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.221+0000 7f0f76000640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f0f800d1c20 0x7f0f8006ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:54806/0 (socket says 172.21.15.86:54806) 2024-04-25T14:10:44.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.221+0000 7f0f7ea00640 1 -- 172.21.15.86:0/455089044 learned_addr learned my addr 172.21.15.86:0/455089044 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:44.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.221+0000 7f0f7f400640 1 --2- 172.21.15.86:0/455089044 >> v2:172.21.15.155:3300/0 conn(0x7f0f800ce790 0x7f0f8006f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:44.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.221+0000 7f0f7e000640 1 -- 172.21.15.86:0/455089044 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4186072618 0 0) 0x7f0f68001120 con 0x7f0f800d1c20 2024-04-25T14:10:44.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.221+0000 7f0f7e000640 1 -- 172.21.15.86:0/455089044 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f64003530 con 0x7f0f800d1c20 2024-04-25T14:10:44.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.221+0000 7f0f7f400640 1 -- 172.21.15.86:0/455089044 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f80070330 msgr2=0x7f0f800d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:44.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.221+0000 7f0f7f400640 1 --2- 172.21.15.86:0/455089044 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f80070330 0x7f0f800d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:44.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.221+0000 7f0f7f400640 1 -- 172.21.15.86:0/455089044 >> v1:172.21.15.155:6789/0 conn(0x7f0f800d1c20 legacy=0x7f0f8006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:44.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.221+0000 7f0f7f400640 1 -- 172.21.15.86:0/455089044 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f8015eb80 con 0x7f0f800ce790 2024-04-25T14:10:44.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.221+0000 7f0f7ea00640 1 --2- 172.21.15.86:0/455089044 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f80070330 0x7f0f800d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:10:44.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.221+0000 7f0f7f400640 1 --2- 172.21.15.86:0/455089044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f800ce790 0x7f0f8006f440 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0f780097f0 tx=0x7f0f7801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4f44e73d2c6d056 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:44.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.221+0000 7f0f7e000640 1 -- 172.21.15.86:0/455089044 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f780b0070 con 0x7f0f800ce790 2024-04-25T14:10:44.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.225+0000 7f0f7e000640 1 -- 172.21.15.86:0/455089044 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f780ab600 con 0x7f0f800ce790 2024-04-25T14:10:44.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.225+0000 7f0f7e000640 1 -- 172.21.15.86:0/455089044 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f780ab9d0 con 0x7f0f800ce790 2024-04-25T14:10:44.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.225+0000 7f0f85c00640 1 -- 172.21.15.86:0/455089044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f800ce790 msgr2=0x7f0f8006f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:44.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.225+0000 7f0f85c00640 1 --2- 172.21.15.86:0/455089044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f800ce790 0x7f0f8006f440 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0f780097f0 tx=0x7f0f7801c410 comp rx=0 tx=0).stop 2024-04-25T14:10:44.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.225+0000 7f0f85c00640 1 -- 172.21.15.86:0/455089044 shutdown_connections 2024-04-25T14:10:44.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.225+0000 7f0f85c00640 1 --2- 172.21.15.86:0/455089044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f800ce790 0x7f0f8006f440 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:44.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.225+0000 7f0f85c00640 1 --2- 172.21.15.86:0/455089044 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f80070330 0x7f0f800d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:44.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.225+0000 7f0f85c00640 1 -- 172.21.15.86:0/455089044 >> 172.21.15.86:0/455089044 conn(0x7f0f800b4650 msgr2=0x7f0f800b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:44.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.225+0000 7f0f85c00640 1 -- 172.21.15.86:0/455089044 shutdown_connections 2024-04-25T14:10:44.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.225+0000 7f0f85c00640 1 -- 172.21.15.86:0/455089044 wait complete. 2024-04-25T14:10:44.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.225+0000 7f0f85c00640 1 Processor -- start 2024-04-25T14:10:44.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.225+0000 7f0f85c00640 1 -- start start 2024-04-25T14:10:44.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.229+0000 7f0f85c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f80070330 0x7f0f800c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:44.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.229+0000 7f0f85c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f800ce790 0x7f0f80073fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:44.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.229+0000 7f0f85c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f800d1c20 0x7f0f800b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:44.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.229+0000 7f0f85c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0f800708c0 con 0x7f0f800ce790 2024-04-25T14:10:44.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.229+0000 7f0f85c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0f80070740 con 0x7f0f800d1c20 2024-04-25T14:10:44.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.229+0000 7f0f85c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0f8016afc0 con 0x7f0f80070330 2024-04-25T14:10:44.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.229+0000 7f0f76000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f800ce790 0x7f0f80073fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:44.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.229+0000 7f0f76000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f800ce790 0x7f0f80073fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53226/0 (socket says 172.21.15.86:53226) 2024-04-25T14:10:44.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.229+0000 7f0f76000640 1 -- 172.21.15.86:0/2068980959 learned_addr learned my addr 172.21.15.86:0/2068980959 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:44.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.229+0000 7f0f7ea00640 1 --2- 172.21.15.86:0/2068980959 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f80070330 0x7f0f800c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:44.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.229+0000 7f0f7f400640 1 --2- 172.21.15.86:0/2068980959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f800d1c20 0x7f0f800b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:44.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.229+0000 7f0f76000640 1 -- 172.21.15.86:0/2068980959 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f80070330 msgr2=0x7f0f800c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:44.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.229+0000 7f0f76000640 1 --2- 172.21.15.86:0/2068980959 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f80070330 0x7f0f800c8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:44.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.229+0000 7f0f76000640 1 -- 172.21.15.86:0/2068980959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f800d1c20 msgr2=0x7f0f800b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:44.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.229+0000 7f0f76000640 1 --2- 172.21.15.86:0/2068980959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f800d1c20 0x7f0f800b1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:44.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.229+0000 7f0f76000640 1 -- 172.21.15.86:0/2068980959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f80168250 con 0x7f0f800ce790 2024-04-25T14:10:44.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.229+0000 7f0f76000640 1 --2- 172.21.15.86:0/2068980959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f800ce790 0x7f0f80073fc0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800c3c0 tx=0x7f0f6800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:44.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.229+0000 7f0f77400640 1 -- 172.21.15.86:0/2068980959 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f680074e0 con 0x7f0f800ce790 2024-04-25T14:10:44.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.229+0000 7f0f77400640 1 -- 172.21.15.86:0/2068980959 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f680047f0 con 0x7f0f800ce790 2024-04-25T14:10:44.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.229+0000 7f0f85c00640 1 -- 172.21.15.86:0/2068980959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f800734e0 con 0x7f0f800ce790 2024-04-25T14:10:44.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.229+0000 7f0f77400640 1 -- 172.21.15.86:0/2068980959 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f680076a0 con 0x7f0f800ce790 2024-04-25T14:10:44.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.229+0000 7f0f85c00640 1 -- 172.21.15.86:0/2068980959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f800c94f0 con 0x7f0f800ce790 2024-04-25T14:10:44.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.229+0000 7f0f85c00640 1 -- 172.21.15.86:0/2068980959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f80069d00 con 0x7f0f800ce790 2024-04-25T14:10:44.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.233+0000 7f0f77400640 1 -- 172.21.15.86:0/2068980959 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7f0f6801a020 con 0x7f0f800ce790 2024-04-25T14:10:44.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.233+0000 7f0f77400640 1 --2- 172.21.15.86:0/2068980959 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f0f5c06f4c0 0x7f0f5c0719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:44.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.233+0000 7f0f77400640 1 -- 172.21.15.86:0/2068980959 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5619+0+0 (secure 0 0 0) 0x7f0f6808f0d0 con 0x7f0f800ce790 2024-04-25T14:10:44.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.233+0000 7f0f7ea00640 1 --2- 172.21.15.86:0/2068980959 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f0f5c06f4c0 0x7f0f5c0719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:44.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.233+0000 7f0f7ea00640 1 --2- 172.21.15.86:0/2068980959 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f0f5c06f4c0 0x7f0f5c0719d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0f700072e0 tx=0x7f0f7000b040 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:44.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.237+0000 7f0f77400640 1 -- 172.21.15.86:0/2068980959 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0f68021070 con 0x7f0f800ce790 2024-04-25T14:10:44.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:44.405+0000 7f0f85c00640 1 -- 172.21.15.86:0/2068980959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f0f80167fe0 con 0x7f0f800ce790 2024-04-25T14:10:44.847 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:44.849+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:45.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.025+0000 7f0f77400640 1 -- 172.21.15.86:0/2068980959 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v124) ==== 170+0+0 (secure 0 0 0) 0x7f0f800d2030 con 0x7f0f800ce790 2024-04-25T14:10:45.024 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-04-25T14:10:45.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.029+0000 7f0f85c00640 1 -- 172.21.15.86:0/2068980959 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f0f5c06f4c0 msgr2=0x7f0f5c0719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:45.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.029+0000 7f0f85c00640 1 --2- 172.21.15.86:0/2068980959 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f0f5c06f4c0 0x7f0f5c0719d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0f700072e0 tx=0x7f0f7000b040 comp rx=0 tx=0).stop 2024-04-25T14:10:45.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.029+0000 7f0f85c00640 1 -- 172.21.15.86:0/2068980959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f800ce790 msgr2=0x7f0f80073fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:45.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.029+0000 7f0f85c00640 1 --2- 172.21.15.86:0/2068980959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f800ce790 0x7f0f80073fc0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800c3c0 tx=0x7f0f6800c880 comp rx=0 tx=0).stop 2024-04-25T14:10:45.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.029+0000 7f0f85c00640 1 -- 172.21.15.86:0/2068980959 shutdown_connections 2024-04-25T14:10:45.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.029+0000 7f0f85c00640 1 --2- 172.21.15.86:0/2068980959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f800d1c20 0x7f0f800b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:45.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.029+0000 7f0f85c00640 1 --2- 172.21.15.86:0/2068980959 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f0f5c06f4c0 0x7f0f5c0719d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:45.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.029+0000 7f0f85c00640 1 --2- 172.21.15.86:0/2068980959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f800ce790 0x7f0f80073fc0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:45.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.029+0000 7f0f85c00640 1 --2- 172.21.15.86:0/2068980959 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f80070330 0x7f0f800c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:45.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.029+0000 7f0f85c00640 1 -- 172.21.15.86:0/2068980959 >> 172.21.15.86:0/2068980959 conn(0x7f0f800b4650 msgr2=0x7f0f800b1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:45.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.029+0000 7f0f85c00640 1 -- 172.21.15.86:0/2068980959 shutdown_connections 2024-04-25T14:10:45.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.029+0000 7f0f85c00640 1 -- 172.21.15.86:0/2068980959 wait complete. 2024-04-25T14:10:45.068 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T14:10:45.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.273+0000 7f41e5c00640 1 Processor -- start 2024-04-25T14:10:45.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.273+0000 7f41e5c00640 1 -- start start 2024-04-25T14:10:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.277+0000 7f41e5c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f41e015ea20 0x7f41e01687e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.277+0000 7f41e5c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41e0072440 con 0x7f41e0161450 2024-04-25T14:10:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.277+0000 7f41e5c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41e0162140 con 0x7f41e0070d60 2024-04-25T14:10:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.277+0000 7f41e5c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f41e0162420 con 0x7f41e015ea20 2024-04-25T14:10:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.277+0000 7f41e5200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f41e015ea20 0x7f41e01687e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.277+0000 7f41df400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f41e0161450 0x7f41e0161860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36418/0 (socket says 172.21.15.86:36418) 2024-04-25T14:10:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.277+0000 7f41df400640 1 -- 172.21.15.86:0/2311315138 learned_addr learned my addr 172.21.15.86:0/2311315138 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:45.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.277+0000 7f41e5200640 1 -- 172.21.15.86:0/2311315138 >> v1:172.21.15.155:6789/0 conn(0x7f41e0070d60 legacy=0x7f41e0071170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:45.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.277+0000 7f41e5200640 1 -- 172.21.15.86:0/2311315138 >> v1:172.21.15.86:6789/0 conn(0x7f41e0161450 legacy=0x7f41e0161860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:45.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.277+0000 7f41e5200640 1 -- 172.21.15.86:0/2311315138 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e00726e0 con 0x7f41e015ea20 2024-04-25T14:10:45.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.281+0000 7f41e5200640 1 --2- 172.21.15.86:0/2311315138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e015ea20 0x7f41e01687e0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f41d8009d90 tx=0x7f41d801c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3456ad637f3aa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:45.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.281+0000 7f41dea00640 1 -- 172.21.15.86:0/2311315138 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d80b2070 con 0x7f41e015ea20 2024-04-25T14:10:45.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.281+0000 7f41dea00640 1 -- 172.21.15.86:0/2311315138 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41d80a5820 con 0x7f41e015ea20 2024-04-25T14:10:45.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.281+0000 7f41dea00640 1 -- 172.21.15.86:0/2311315138 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d80a5bf0 con 0x7f41e015ea20 2024-04-25T14:10:45.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.281+0000 7f41e5c00640 1 -- 172.21.15.86:0/2311315138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e015ea20 msgr2=0x7f41e01687e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:45.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.281+0000 7f41e5c00640 1 --2- 172.21.15.86:0/2311315138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e015ea20 0x7f41e01687e0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f41d8009d90 tx=0x7f41d801c850 comp rx=0 tx=0).stop 2024-04-25T14:10:45.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.281+0000 7f41e5c00640 1 -- 172.21.15.86:0/2311315138 shutdown_connections 2024-04-25T14:10:45.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.281+0000 7f41e5c00640 1 --2- 172.21.15.86:0/2311315138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e015ea20 0x7f41e01687e0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:45.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.281+0000 7f41e5c00640 1 -- 172.21.15.86:0/2311315138 >> 172.21.15.86:0/2311315138 conn(0x7f41e00d3330 msgr2=0x7f41e00d3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:45.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.281+0000 7f41e5c00640 1 -- 172.21.15.86:0/2311315138 shutdown_connections 2024-04-25T14:10:45.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.281+0000 7f41e5c00640 1 -- 172.21.15.86:0/2311315138 wait complete. 2024-04-25T14:10:45.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.281+0000 7f41e5c00640 1 Processor -- start 2024-04-25T14:10:45.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.281+0000 7f41e5c00640 1 -- start start 2024-04-25T14:10:45.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.285+0000 7f41e5c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e0070d60 0x7f41e0150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:45.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.285+0000 7f41e5c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41e015ea20 0x7f41e01f76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:45.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.285+0000 7f41e5c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41e0161450 0x7f41e01f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:45.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.285+0000 7f41e5c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f41e016bd70 con 0x7f41e0070d60 2024-04-25T14:10:45.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.285+0000 7f41e5c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f41e00ca200 con 0x7f41e015ea20 2024-04-25T14:10:45.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.285+0000 7f41e5c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f41e016bef0 con 0x7f41e0161450 2024-04-25T14:10:45.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.285+0000 7f41dfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e0070d60 0x7f41e0150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:45.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.285+0000 7f41dfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e0070d60 0x7f41e0150e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53244/0 (socket says 172.21.15.86:53244) 2024-04-25T14:10:45.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.285+0000 7f41dfe00640 1 -- 172.21.15.86:0/1867619508 learned_addr learned my addr 172.21.15.86:0/1867619508 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:45.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.285+0000 7f41df400640 1 --2- 172.21.15.86:0/1867619508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41e015ea20 0x7f41e01f76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:45.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.285+0000 7f41e5200640 1 --2- 172.21.15.86:0/1867619508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41e0161450 0x7f41e01f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:45.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.285+0000 7f41dfe00640 1 -- 172.21.15.86:0/1867619508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41e0161450 msgr2=0x7f41e01f7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:45.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.285+0000 7f41dfe00640 1 --2- 172.21.15.86:0/1867619508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41e0161450 0x7f41e01f7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:45.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.285+0000 7f41dfe00640 1 -- 172.21.15.86:0/1867619508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41e015ea20 msgr2=0x7f41e01f76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:45.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.285+0000 7f41dfe00640 1 --2- 172.21.15.86:0/1867619508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41e015ea20 0x7f41e01f76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:45.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.285+0000 7f41dfe00640 1 -- 172.21.15.86:0/1867619508 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e00cd2b0 con 0x7f41e0070d60 2024-04-25T14:10:45.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.285+0000 7f41dfe00640 1 --2- 172.21.15.86:0/1867619508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e0070d60 0x7f41e0150e70 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f41d000c390 tx=0x7f41d000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:45.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.285+0000 7f41d7e00640 1 -- 172.21.15.86:0/1867619508 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d0017070 con 0x7f41e0070d60 2024-04-25T14:10:45.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.285+0000 7f41d7e00640 1 -- 172.21.15.86:0/1867619508 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41d00052d0 con 0x7f41e0070d60 2024-04-25T14:10:45.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.285+0000 7f41e5c00640 1 -- 172.21.15.86:0/1867619508 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e00cc4f0 con 0x7f41e0070d60 2024-04-25T14:10:45.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.285+0000 7f41d7e00640 1 -- 172.21.15.86:0/1867619508 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d0003820 con 0x7f41e0070d60 2024-04-25T14:10:45.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.285+0000 7f41e5c00640 1 -- 172.21.15.86:0/1867619508 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e00ccab0 con 0x7f41e0070d60 2024-04-25T14:10:45.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.285+0000 7f41e5c00640 1 -- 172.21.15.86:0/1867619508 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41e0076b80 con 0x7f41e0070d60 2024-04-25T14:10:45.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.289+0000 7f41d7e00640 1 -- 172.21.15.86:0/1867619508 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7f41d001a070 con 0x7f41e0070d60 2024-04-25T14:10:45.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.289+0000 7f41d7e00640 1 --2- 172.21.15.86:0/1867619508 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f41c006f4c0 0x7f41c00719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:45.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.289+0000 7f41d7e00640 1 -- 172.21.15.86:0/1867619508 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6001+0+0 (secure 0 0 0) 0x7f41d008ea50 con 0x7f41e0070d60 2024-04-25T14:10:45.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.289+0000 7f41df400640 1 --2- 172.21.15.86:0/1867619508 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f41c006f4c0 0x7f41c00719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:45.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.289+0000 7f41df400640 1 --2- 172.21.15.86:0/1867619508 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f41c006f4c0 0x7f41c00719d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f41e00b92f0 tx=0x7f41cc009040 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:45.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.293+0000 7f41d7e00640 1 -- 172.21.15.86:0/1867619508 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f41d0021020 con 0x7f41e0070d60 2024-04-25T14:10:45.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:45.493+0000 7f41e5c00640 1 -- 172.21.15.86:0/1867619508 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f41e00c83a0 con 0x7f41e0070d60 2024-04-25T14:10:45.635 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:45.637+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:46.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.049+0000 7f41d7e00640 1 -- 172.21.15.86:0/1867619508 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v90) ==== 174+0+0 (secure 0 0 0) 0x7f41d0005480 con 0x7f41e0070d60 2024-04-25T14:10:46.049 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 19 and data pool 20 2024-04-25T14:10:46.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.061+0000 7f41e5c00640 1 -- 172.21.15.86:0/1867619508 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f41c006f4c0 msgr2=0x7f41c00719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:46.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.061+0000 7f41e5c00640 1 --2- 172.21.15.86:0/1867619508 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f41c006f4c0 0x7f41c00719d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f41e00b92f0 tx=0x7f41cc009040 comp rx=0 tx=0).stop 2024-04-25T14:10:46.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.061+0000 7f41e5c00640 1 -- 172.21.15.86:0/1867619508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e0070d60 msgr2=0x7f41e0150e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:46.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.061+0000 7f41e5c00640 1 --2- 172.21.15.86:0/1867619508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e0070d60 0x7f41e0150e70 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f41d000c390 tx=0x7f41d000c850 comp rx=0 tx=0).stop 2024-04-25T14:10:46.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.061+0000 7f41e5c00640 1 -- 172.21.15.86:0/1867619508 shutdown_connections 2024-04-25T14:10:46.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.061+0000 7f41e5c00640 1 --2- 172.21.15.86:0/1867619508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41e0161450 0x7f41e01f7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:46.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.061+0000 7f41e5c00640 1 --2- 172.21.15.86:0/1867619508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41e015ea20 0x7f41e01f76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:46.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.061+0000 7f41e5c00640 1 --2- 172.21.15.86:0/1867619508 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f41c006f4c0 0x7f41c00719d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:46.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.061+0000 7f41e5c00640 1 --2- 172.21.15.86:0/1867619508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e0070d60 0x7f41e0150e70 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:46.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.061+0000 7f41e5c00640 1 -- 172.21.15.86:0/1867619508 >> 172.21.15.86:0/1867619508 conn(0x7f41e00d3330 msgr2=0x7f41e00cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:46.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.065+0000 7f41e5c00640 1 -- 172.21.15.86:0/1867619508 shutdown_connections 2024-04-25T14:10:46.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.065+0000 7f41e5c00640 1 -- 172.21.15.86:0/1867619508 wait complete. 2024-04-25T14:10:46.095 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:10:46.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.249+0000 7fae14e00640 1 Processor -- start 2024-04-25T14:10:46.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.249+0000 7fae14e00640 1 -- start start 2024-04-25T14:10:46.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.253+0000 7fae14e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fae1006e970 0x7fae1016abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:46.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.253+0000 7fae14e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae1006ef90 con 0x7fae100cf970 2024-04-25T14:10:46.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.253+0000 7fae14e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae10163a80 con 0x7fae10071530 2024-04-25T14:10:46.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.253+0000 7fae14e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fae100cedf0 con 0x7fae1006e970 2024-04-25T14:10:46.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.253+0000 7fae0de00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fae100cf970 0x7fae10070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36430/0 (socket says 172.21.15.86:36430) 2024-04-25T14:10:46.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.253+0000 7fae0de00640 1 -- 172.21.15.86:0/3856547947 learned_addr learned my addr 172.21.15.86:0/3856547947 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:46.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.253+0000 7fae0e800640 1 --2- 172.21.15.86:0/3856547947 >> v2:172.21.15.155:3300/0 conn(0x7fae1006e970 0x7fae1016abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:46.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.253+0000 7fae0d400640 1 -- 172.21.15.86:0/3856547947 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 682246482 0 0) 0x7fae1006ef90 con 0x7fae100cf970 2024-04-25T14:10:46.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.253+0000 7fae0d400640 1 -- 172.21.15.86:0/3856547947 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fadf4003530 con 0x7fae100cf970 2024-04-25T14:10:46.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.253+0000 7fae0d400640 1 -- 172.21.15.86:0/3856547947 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4153908044 0 0) 0x7fae10163a80 con 0x7fae10071530 2024-04-25T14:10:46.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.253+0000 7fae0e800640 1 -- 172.21.15.86:0/3856547947 >> v1:172.21.15.155:6789/0 conn(0x7fae10071530 legacy=0x7fae1006e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:46.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.253+0000 7fae0e800640 1 -- 172.21.15.86:0/3856547947 >> v1:172.21.15.86:6789/0 conn(0x7fae100cf970 legacy=0x7fae10070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:46.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.253+0000 7fae0e800640 1 -- 172.21.15.86:0/3856547947 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae10163d60 con 0x7fae1006e970 2024-04-25T14:10:46.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.257+0000 7fae0e800640 1 --2- 172.21.15.86:0/3856547947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae1006e970 0x7fae1016abc0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fae08009d90 tx=0x7fae0801c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e5bbcbb9819d7ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:46.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.257+0000 7fae0d400640 1 -- 172.21.15.86:0/3856547947 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae00001680 con 0x7fae1006e970 2024-04-25T14:10:46.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.257+0000 7fae0d400640 1 -- 172.21.15.86:0/3856547947 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae080a6440 con 0x7fae1006e970 2024-04-25T14:10:46.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.257+0000 7fae0d400640 1 -- 172.21.15.86:0/3856547947 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae080a67f0 con 0x7fae1006e970 2024-04-25T14:10:46.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.257+0000 7fae14e00640 1 -- 172.21.15.86:0/3856547947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae1006e970 msgr2=0x7fae1016abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:46.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.257+0000 7fae14e00640 1 --2- 172.21.15.86:0/3856547947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae1006e970 0x7fae1016abc0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fae08009d90 tx=0x7fae0801c7b0 comp rx=0 tx=0).stop 2024-04-25T14:10:46.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.257+0000 7fae14e00640 1 -- 172.21.15.86:0/3856547947 shutdown_connections 2024-04-25T14:10:46.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.257+0000 7fae14e00640 1 --2- 172.21.15.86:0/3856547947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae1006e970 0x7fae1016abc0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:46.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.257+0000 7fae14e00640 1 -- 172.21.15.86:0/3856547947 >> 172.21.15.86:0/3856547947 conn(0x7fae100cffc0 msgr2=0x7fae100d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:46.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.257+0000 7fae14e00640 1 -- 172.21.15.86:0/3856547947 shutdown_connections 2024-04-25T14:10:46.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.257+0000 7fae14e00640 1 -- 172.21.15.86:0/3856547947 wait complete. 2024-04-25T14:10:46.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.257+0000 7fae14e00640 1 Processor -- start 2024-04-25T14:10:46.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.257+0000 7fae14e00640 1 -- start start 2024-04-25T14:10:46.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.261+0000 7fae14e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae1006e970 0x7fae100b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:46.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.261+0000 7fae14e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae100cf970 0x7fae10155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:46.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.261+0000 7fae14e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae100b1c70 0x7fae100b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:46.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.261+0000 7fae14e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae1016dcb0 con 0x7fae100cf970 2024-04-25T14:10:46.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.261+0000 7fae14e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fae1016db30 con 0x7fae1006e970 2024-04-25T14:10:46.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.261+0000 7fae14e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fae1016de30 con 0x7fae100b1c70 2024-04-25T14:10:46.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.261+0000 7fae07e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae100cf970 0x7fae10155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:46.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.261+0000 7fae0de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae1006e970 0x7fae100b02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:46.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.261+0000 7fae07e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae100cf970 0x7fae10155c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53254/0 (socket says 172.21.15.86:53254) 2024-04-25T14:10:46.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.261+0000 7fae07e00640 1 -- 172.21.15.86:0/1545005173 learned_addr learned my addr 172.21.15.86:0/1545005173 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:46.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.261+0000 7fae0e800640 1 --2- 172.21.15.86:0/1545005173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae100b1c70 0x7fae100b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:46.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.261+0000 7fae0de00640 1 -- 172.21.15.86:0/1545005173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae100b1c70 msgr2=0x7fae100b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:46.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.261+0000 7fae0de00640 1 --2- 172.21.15.86:0/1545005173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae100b1c70 0x7fae100b2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:46.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.261+0000 7fae0de00640 1 -- 172.21.15.86:0/1545005173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae100cf970 msgr2=0x7fae10155c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:46.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.261+0000 7fae0de00640 1 --2- 172.21.15.86:0/1545005173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae100cf970 0x7fae10155c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:46.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.261+0000 7fae0de00640 1 -- 172.21.15.86:0/1545005173 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae10158b80 con 0x7fae1006e970 2024-04-25T14:10:46.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.261+0000 7fae0de00640 1 --2- 172.21.15.86:0/1545005173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae1006e970 0x7fae100b02d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fae0000c700 tx=0x7fae0000cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:46.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.261+0000 7fae06000640 1 -- 172.21.15.86:0/1545005173 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae00018070 con 0x7fae1006e970 2024-04-25T14:10:46.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.261+0000 7fae06000640 1 -- 172.21.15.86:0/1545005173 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae000072f0 con 0x7fae1006e970 2024-04-25T14:10:46.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.261+0000 7fae14e00640 1 -- 172.21.15.86:0/1545005173 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae100b25c0 con 0x7fae1006e970 2024-04-25T14:10:46.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.261+0000 7fae06000640 1 -- 172.21.15.86:0/1545005173 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae000047f0 con 0x7fae1006e970 2024-04-25T14:10:46.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.261+0000 7fae14e00640 1 -- 172.21.15.86:0/1545005173 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae100b65e0 con 0x7fae1006e970 2024-04-25T14:10:46.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.261+0000 7fae14e00640 1 -- 172.21.15.86:0/1545005173 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae100720e0 con 0x7fae1006e970 2024-04-25T14:10:46.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.265+0000 7fae06000640 1 -- 172.21.15.86:0/1545005173 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7fae0001b070 con 0x7fae1006e970 2024-04-25T14:10:46.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.265+0000 7fae06000640 1 --2- 172.21.15.86:0/1545005173 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fadec06f490 0x7fadec0719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:46.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.265+0000 7fae06000640 1 -- 172.21.15.86:0/1545005173 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7fae0008e820 con 0x7fae1006e970 2024-04-25T14:10:46.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.265+0000 7fae07e00640 1 --2- 172.21.15.86:0/1545005173 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fadec06f490 0x7fadec0719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:46.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.265+0000 7fae07e00640 1 --2- 172.21.15.86:0/1545005173 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fadec06f490 0x7fadec0719a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fae100b5e00 tx=0x7fadf8009040 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:46.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.269+0000 7fae06000640 1 -- 172.21.15.86:0/1545005173 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fae100b25c0 con 0x7fae1006e970 2024-04-25T14:10:46.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.445+0000 7fae14e00640 1 -- 172.21.15.86:0/1545005173 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fae100b94c0 con 0x7fae1006e970 2024-04-25T14:10:46.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.445+0000 7fae06000640 1 -- 172.21.15.86:0/1545005173 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) ==== 74+0+16913 (secure 0 0 0) 0x7fae0005b060 con 0x7fae1006e970 2024-04-25T14:10:46.443 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:46.443 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:10:45.497727+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:10:43.913889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T14:10:44.419362+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:10:46.444 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:46.445+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:46.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.449+0000 7fae14e00640 1 -- 172.21.15.86:0/1545005173 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fadec06f490 msgr2=0x7fadec0719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:46.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.449+0000 7fae14e00640 1 --2- 172.21.15.86:0/1545005173 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fadec06f490 0x7fadec0719a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fae100b5e00 tx=0x7fadf8009040 comp rx=0 tx=0).stop 2024-04-25T14:10:46.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.449+0000 7fae14e00640 1 -- 172.21.15.86:0/1545005173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae1006e970 msgr2=0x7fae100b02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:46.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.449+0000 7fae14e00640 1 --2- 172.21.15.86:0/1545005173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae1006e970 0x7fae100b02d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fae0000c700 tx=0x7fae0000cbc0 comp rx=0 tx=0).stop 2024-04-25T14:10:46.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.449+0000 7fae14e00640 1 -- 172.21.15.86:0/1545005173 shutdown_connections 2024-04-25T14:10:46.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.449+0000 7fae14e00640 1 --2- 172.21.15.86:0/1545005173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae100b1c70 0x7fae100b2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:46.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.449+0000 7fae14e00640 1 --2- 172.21.15.86:0/1545005173 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fadec06f490 0x7fadec0719a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:46.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.449+0000 7fae14e00640 1 --2- 172.21.15.86:0/1545005173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae100cf970 0x7fae10155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:46.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.449+0000 7fae14e00640 1 --2- 172.21.15.86:0/1545005173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae1006e970 0x7fae100b02d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:46.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.449+0000 7fae14e00640 1 -- 172.21.15.86:0/1545005173 >> 172.21.15.86:0/1545005173 conn(0x7fae100cffc0 msgr2=0x7fae100b5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:46.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.449+0000 7fae14e00640 1 -- 172.21.15.86:0/1545005173 shutdown_connections 2024-04-25T14:10:46.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.449+0000 7fae14e00640 1 -- 172.21.15.86:0/1545005173 wait complete. 2024-04-25T14:10:46.466 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:10:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.621+0000 7f0534c00640 1 Processor -- start 2024-04-25T14:10:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.621+0000 7f0534c00640 1 -- start start 2024-04-25T14:10:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.625+0000 7f0534c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f05300d08c0 0x7f053015c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.625+0000 7f0534c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f053006e830 0x7f053006ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.625+0000 7f0534c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05300cf380 0x7f05300cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.625+0000 7f0534c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f053006f180 con 0x7f05300d08c0 2024-04-25T14:10:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.625+0000 7f0534c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f053006d120 con 0x7f053006e830 2024-04-25T14:10:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.625+0000 7f0534c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f053006d400 con 0x7f05300cf380 2024-04-25T14:10:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.625+0000 7f052f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f05300d08c0 0x7f053015c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.625+0000 7f052f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f05300d08c0 0x7f053015c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53258/0 (socket says 172.21.15.86:53258) 2024-04-25T14:10:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.625+0000 7f052f400640 1 -- 172.21.15.86:0/1827720289 learned_addr learned my addr 172.21.15.86:0/1827720289 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.625+0000 7f0527e00640 1 --2- 172.21.15.86:0/1827720289 >> v2:172.21.15.155:3300/0 conn(0x7f053006e830 0x7f053006ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.625+0000 7f052fe00640 1 --2- 172.21.15.86:0/1827720289 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05300cf380 0x7f05300cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.625+0000 7f052f400640 1 -- 172.21.15.86:0/1827720289 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05300cf380 msgr2=0x7f05300cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.625+0000 7f052f400640 1 --2- 172.21.15.86:0/1827720289 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05300cf380 0x7f05300cf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.625+0000 7f052f400640 1 -- 172.21.15.86:0/1827720289 >> v2:172.21.15.155:3300/0 conn(0x7f053006e830 msgr2=0x7f053006ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.625+0000 7f052f400640 1 --2- 172.21.15.86:0/1827720289 >> v2:172.21.15.155:3300/0 conn(0x7f053006e830 0x7f053006ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.625+0000 7f052f400640 1 -- 172.21.15.86:0/1827720289 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053006d6e0 con 0x7f05300d08c0 2024-04-25T14:10:46.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.625+0000 7f052f400640 1 --2- 172.21.15.86:0/1827720289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05300d08c0 0x7f053015c990 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f0530070330 tx=0x7f052001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2201d406c7cab5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:46.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.629+0000 7f052ea00640 1 -- 172.21.15.86:0/1827720289 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05200b0070 con 0x7f05300d08c0 2024-04-25T14:10:46.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.629+0000 7f052ea00640 1 -- 172.21.15.86:0/1827720289 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05200ab600 con 0x7f05300d08c0 2024-04-25T14:10:46.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.629+0000 7f052ea00640 1 -- 172.21.15.86:0/1827720289 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05200ab9d0 con 0x7f05300d08c0 2024-04-25T14:10:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.629+0000 7f0534c00640 1 -- 172.21.15.86:0/1827720289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05300d08c0 msgr2=0x7f053015c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.629+0000 7f0534c00640 1 --2- 172.21.15.86:0/1827720289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05300d08c0 0x7f053015c990 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f0530070330 tx=0x7f052001c520 comp rx=0 tx=0).stop 2024-04-25T14:10:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.629+0000 7f0534c00640 1 -- 172.21.15.86:0/1827720289 shutdown_connections 2024-04-25T14:10:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.629+0000 7f0534c00640 1 --2- 172.21.15.86:0/1827720289 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05300cf380 0x7f05300cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.629+0000 7f0534c00640 1 --2- 172.21.15.86:0/1827720289 >> v2:172.21.15.155:3300/0 conn(0x7f053006e830 0x7f053006ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.629+0000 7f0534c00640 1 --2- 172.21.15.86:0/1827720289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05300d08c0 0x7f053015c990 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.629+0000 7f0534c00640 1 -- 172.21.15.86:0/1827720289 >> 172.21.15.86:0/1827720289 conn(0x7f05300b4850 msgr2=0x7f05300b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.629+0000 7f0534c00640 1 -- 172.21.15.86:0/1827720289 shutdown_connections 2024-04-25T14:10:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.629+0000 7f0534c00640 1 -- 172.21.15.86:0/1827720289 wait complete. 2024-04-25T14:10:46.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.629+0000 7f0534c00640 1 Processor -- start 2024-04-25T14:10:46.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.629+0000 7f0534c00640 1 -- start start 2024-04-25T14:10:46.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.633+0000 7f0534c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f053006e830 0x7f0530073fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:46.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.633+0000 7f0534c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05300cf380 0x7f053009a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:46.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.633+0000 7f0534c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05300d08c0 0x7f053015fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:46.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.633+0000 7f0534c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05300d00e0 con 0x7f05300cf380 2024-04-25T14:10:46.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.633+0000 7f0534c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f053006f580 con 0x7f05300d08c0 2024-04-25T14:10:46.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.633+0000 7f0534c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f05300ae940 con 0x7f053006e830 2024-04-25T14:10:46.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.633+0000 7f0527e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05300cf380 0x7f053009a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:46.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.633+0000 7f052f400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f053006e830 0x7f0530073fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:46.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.633+0000 7f0527e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05300cf380 0x7f053009a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53270/0 (socket says 172.21.15.86:53270) 2024-04-25T14:10:46.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.633+0000 7f0527e00640 1 -- 172.21.15.86:0/3260233258 learned_addr learned my addr 172.21.15.86:0/3260233258 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:46.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.633+0000 7f052fe00640 1 --2- 172.21.15.86:0/3260233258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05300d08c0 0x7f053015fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:46.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.633+0000 7f0527e00640 1 -- 172.21.15.86:0/3260233258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f053006e830 msgr2=0x7f0530073fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:46.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.633+0000 7f0527e00640 1 --2- 172.21.15.86:0/3260233258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f053006e830 0x7f0530073fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:46.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.633+0000 7f0527e00640 1 -- 172.21.15.86:0/3260233258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05300d08c0 msgr2=0x7f053015fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:46.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.633+0000 7f0527e00640 1 --2- 172.21.15.86:0/3260233258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05300d08c0 0x7f053015fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:46.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.633+0000 7f0527e00640 1 -- 172.21.15.86:0/3260233258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053009abf0 con 0x7f05300cf380 2024-04-25T14:10:46.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.633+0000 7f0527e00640 1 --2- 172.21.15.86:0/3260233258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05300cf380 0x7f053009a6b0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f051800e8a0 tx=0x7f051800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:46.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.633+0000 7f0527400640 1 -- 172.21.15.86:0/3260233258 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0518017070 con 0x7f05300cf380 2024-04-25T14:10:46.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.633+0000 7f0527400640 1 -- 172.21.15.86:0/3260233258 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f051800c940 con 0x7f05300cf380 2024-04-25T14:10:46.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.633+0000 7f0527400640 1 -- 172.21.15.86:0/3260233258 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051800ccf0 con 0x7f05300cf380 2024-04-25T14:10:46.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.633+0000 7f0534c00640 1 -- 172.21.15.86:0/3260233258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0530072f40 con 0x7f05300cf380 2024-04-25T14:10:46.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.633+0000 7f0534c00640 1 -- 172.21.15.86:0/3260233258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0530073500 con 0x7f05300cf380 2024-04-25T14:10:46.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.633+0000 7f0534c00640 1 -- 172.21.15.86:0/3260233258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0530076b80 con 0x7f05300cf380 2024-04-25T14:10:46.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.637+0000 7f0527400640 1 -- 172.21.15.86:0/3260233258 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7f051801a070 con 0x7f05300cf380 2024-04-25T14:10:46.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.637+0000 7f0527400640 1 --2- 172.21.15.86:0/3260233258 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f050c06f4c0 0x7f050c0719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:46.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.637+0000 7f0527400640 1 -- 172.21.15.86:0/3260233258 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7f051808f3b0 con 0x7f05300cf380 2024-04-25T14:10:46.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.637+0000 7f052f400640 1 --2- 172.21.15.86:0/3260233258 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f050c06f4c0 0x7f050c0719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:46.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.637+0000 7f052f400640 1 --2- 172.21.15.86:0/3260233258 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f050c06f4c0 0x7f050c0719d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f05200ad000 tx=0x7f0520008040 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:46.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.641+0000 7f0527400640 1 -- 172.21.15.86:0/3260233258 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f051805b9d0 con 0x7f05300cf380 2024-04-25T14:10:46.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.809+0000 7f0534c00640 1 -- 172.21.15.86:0/3260233258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f053006f360 con 0x7f05300cf380 2024-04-25T14:10:46.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.813+0000 7f0527400640 1 -- 172.21.15.86:0/3260233258 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) ==== 74+0+16913 (secure 0 0 0) 0x7f051805fcd0 con 0x7f05300cf380 2024-04-25T14:10:46.809 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:46.809 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:10:45.497727+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:10:43.913889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T14:10:44.419362+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:10:46.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.813+0000 7f0534c00640 1 -- 172.21.15.86:0/3260233258 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f050c06f4c0 msgr2=0x7f050c0719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.813+0000 7f0534c00640 1 --2- 172.21.15.86:0/3260233258 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f050c06f4c0 0x7f050c0719d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f05200ad000 tx=0x7f0520008040 comp rx=0 tx=0).stop 2024-04-25T14:10:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.813+0000 7f0534c00640 1 -- 172.21.15.86:0/3260233258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05300cf380 msgr2=0x7f053009a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.813+0000 7f0534c00640 1 --2- 172.21.15.86:0/3260233258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05300cf380 0x7f053009a6b0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f051800e8a0 tx=0x7f051800eca0 comp rx=0 tx=0).stop 2024-04-25T14:10:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.813+0000 7f0534c00640 1 -- 172.21.15.86:0/3260233258 shutdown_connections 2024-04-25T14:10:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.813+0000 7f0534c00640 1 --2- 172.21.15.86:0/3260233258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05300d08c0 0x7f053015fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.813+0000 7f0534c00640 1 --2- 172.21.15.86:0/3260233258 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f050c06f4c0 0x7f050c0719d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.813+0000 7f0534c00640 1 --2- 172.21.15.86:0/3260233258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05300cf380 0x7f053009a6b0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.813+0000 7f0534c00640 1 --2- 172.21.15.86:0/3260233258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f053006e830 0x7f0530073fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.813+0000 7f0534c00640 1 -- 172.21.15.86:0/3260233258 >> 172.21.15.86:0/3260233258 conn(0x7f05300b4850 msgr2=0x7f05300b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.813+0000 7f0534c00640 1 -- 172.21.15.86:0/3260233258 shutdown_connections 2024-04-25T14:10:46.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.813+0000 7f0534c00640 1 -- 172.21.15.86:0/3260233258 wait complete. 2024-04-25T14:10:46.833 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T14:10:46.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.993+0000 7fd5d4200640 1 Processor -- start 2024-04-25T14:10:46.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.993+0000 7fd5d4200640 1 -- start start 2024-04-25T14:10:46.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.993+0000 7fd5d4200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd5cc0740c0 0x7fd5cc1676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:46.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.993+0000 7fd5d4200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5cc072bd0 con 0x7fd5cc0cfca0 2024-04-25T14:10:46.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.993+0000 7fd5d4200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5cc06fbd0 con 0x7fd5cc0d16e0 2024-04-25T14:10:46.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.993+0000 7fd5d4200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd5cc070470 con 0x7fd5cc0740c0 2024-04-25T14:10:46.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.993+0000 7fd5d1800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd5cc0cfca0 0x7fd5cc06f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36432/0 (socket says 172.21.15.86:36432) 2024-04-25T14:10:46.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.993+0000 7fd5d1800640 1 -- 172.21.15.86:0/1321597467 learned_addr learned my addr 172.21.15.86:0/1321597467 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:46.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.993+0000 7fd5d2200640 1 --2- 172.21.15.86:0/1321597467 >> v2:172.21.15.155:3300/0 conn(0x7fd5cc0740c0 0x7fd5cc1676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:46.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.993+0000 7fd5cbe00640 1 -- 172.21.15.86:0/1321597467 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3370721759 0 0) 0x7fd5cc072bd0 con 0x7fd5cc0cfca0 2024-04-25T14:10:46.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.993+0000 7fd5cbe00640 1 -- 172.21.15.86:0/1321597467 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5b4005740 con 0x7fd5cc0cfca0 2024-04-25T14:10:46.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.993+0000 7fd5cbe00640 1 -- 172.21.15.86:0/1321597467 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 356054864 0 0) 0x7fd5cc06fbd0 con 0x7fd5cc0d16e0 2024-04-25T14:10:46.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.993+0000 7fd5cbe00640 1 -- 172.21.15.86:0/1321597467 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5cc072bd0 con 0x7fd5cc0d16e0 2024-04-25T14:10:46.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.993+0000 7fd5d2200640 1 -- 172.21.15.86:0/1321597467 >> v1:172.21.15.155:6789/0 conn(0x7fd5cc0d16e0 legacy=0x7fd5cc0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:46.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.993+0000 7fd5d2200640 1 -- 172.21.15.86:0/1321597467 >> v1:172.21.15.86:6789/0 conn(0x7fd5cc0cfca0 legacy=0x7fd5cc06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:46.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.993+0000 7fd5cbe00640 1 -- 172.21.15.86:0/1321597467 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 727565294 0 0) 0x7fd5b4005740 con 0x7fd5cc0cfca0 2024-04-25T14:10:46.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.993+0000 7fd5d2200640 1 -- 172.21.15.86:0/1321597467 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5cc06feb0 con 0x7fd5cc0740c0 2024-04-25T14:10:46.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.997+0000 7fd5d2200640 1 --2- 172.21.15.86:0/1321597467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5cc0740c0 0x7fd5cc1676d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009d90 tx=0x7fd5c401c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a0ab1c9a0122f2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:46.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.997+0000 7fd5cbe00640 1 -- 172.21.15.86:0/1321597467 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c00016a0 con 0x7fd5cc0740c0 2024-04-25T14:10:46.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.997+0000 7fd5cbe00640 1 -- 172.21.15.86:0/1321597467 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5c40a64f0 con 0x7fd5cc0740c0 2024-04-25T14:10:46.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.997+0000 7fd5cbe00640 1 -- 172.21.15.86:0/1321597467 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c40a68a0 con 0x7fd5cc0740c0 2024-04-25T14:10:46.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.997+0000 7fd5d4200640 1 -- 172.21.15.86:0/1321597467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5cc0740c0 msgr2=0x7fd5cc1676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:46.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.997+0000 7fd5d4200640 1 --2- 172.21.15.86:0/1321597467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5cc0740c0 0x7fd5cc1676d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009d90 tx=0x7fd5c401c890 comp rx=0 tx=0).stop 2024-04-25T14:10:46.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.997+0000 7fd5d4200640 1 -- 172.21.15.86:0/1321597467 shutdown_connections 2024-04-25T14:10:46.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.997+0000 7fd5d4200640 1 --2- 172.21.15.86:0/1321597467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5cc0740c0 0x7fd5cc1676d0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:46.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.997+0000 7fd5d4200640 1 -- 172.21.15.86:0/1321597467 >> 172.21.15.86:0/1321597467 conn(0x7fd5cc0b4650 msgr2=0x7fd5cc0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:46.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.997+0000 7fd5d4200640 1 -- 172.21.15.86:0/1321597467 shutdown_connections 2024-04-25T14:10:46.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:46.997+0000 7fd5d4200640 1 -- 172.21.15.86:0/1321597467 wait complete. 2024-04-25T14:10:46.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.001+0000 7fd5d4200640 1 Processor -- start 2024-04-25T14:10:46.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.001+0000 7fd5d4200640 1 -- start start 2024-04-25T14:10:47.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.001+0000 7fd5d4200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5cc0740c0 0x7fd5cc159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:47.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.001+0000 7fd5d4200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5cc0cfca0 0x7fd5cc0ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:47.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.001+0000 7fd5d4200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5cc0d16e0 0x7fd5cc0ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:47.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.001+0000 7fd5d4200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc0ce900 con 0x7fd5cc0cfca0 2024-04-25T14:10:47.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.001+0000 7fd5d4200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc0ce780 con 0x7fd5cc0740c0 2024-04-25T14:10:47.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.001+0000 7fd5d4200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd5cc0cea80 con 0x7fd5cc0d16e0 2024-04-25T14:10:47.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.001+0000 7fd5d0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5cc0cfca0 0x7fd5cc0ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:47.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.001+0000 7fd5d1800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5cc0740c0 0x7fd5cc159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:47.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.001+0000 7fd5d1800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5cc0740c0 0x7fd5cc159630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:33898/0 (socket says 172.21.15.86:33898) 2024-04-25T14:10:47.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.001+0000 7fd5d0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5cc0cfca0 0x7fd5cc0ca270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53274/0 (socket says 172.21.15.86:53274) 2024-04-25T14:10:47.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.001+0000 7fd5d1800640 1 -- 172.21.15.86:0/3383226360 learned_addr learned my addr 172.21.15.86:0/3383226360 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:47.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.001+0000 7fd5d2200640 1 --2- 172.21.15.86:0/3383226360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5cc0d16e0 0x7fd5cc0ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:47.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.001+0000 7fd5d0e00640 1 -- 172.21.15.86:0/3383226360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5cc0d16e0 msgr2=0x7fd5cc0ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:47.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.001+0000 7fd5d0e00640 1 --2- 172.21.15.86:0/3383226360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5cc0d16e0 0x7fd5cc0ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:47.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.001+0000 7fd5d0e00640 1 -- 172.21.15.86:0/3383226360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5cc0740c0 msgr2=0x7fd5cc159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:47.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.001+0000 7fd5d0e00640 1 --2- 172.21.15.86:0/3383226360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5cc0740c0 0x7fd5cc159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:47.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.001+0000 7fd5d0e00640 1 -- 172.21.15.86:0/3383226360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5cc0cb4e0 con 0x7fd5cc0cfca0 2024-04-25T14:10:47.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.001+0000 7fd5d1800640 1 --2- 172.21.15.86:0/3383226360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5cc0740c0 0x7fd5cc159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:10:47.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.005+0000 7fd5d0e00640 1 --2- 172.21.15.86:0/3383226360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5cc0cfca0 0x7fd5cc0ca270 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800c3f0 tx=0x7fd5b800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:47.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.005+0000 7fd5c9600640 1 -- 172.21.15.86:0/3383226360 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8017070 con 0x7fd5cc0cfca0 2024-04-25T14:10:47.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.005+0000 7fd5c9600640 1 -- 172.21.15.86:0/3383226360 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5b80052d0 con 0x7fd5cc0cfca0 2024-04-25T14:10:47.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.005+0000 7fd5d4200640 1 -- 172.21.15.86:0/3383226360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5cc0cb7c0 con 0x7fd5cc0cfca0 2024-04-25T14:10:47.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.005+0000 7fd5c9600640 1 -- 172.21.15.86:0/3383226360 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8003840 con 0x7fd5cc0cfca0 2024-04-25T14:10:47.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.005+0000 7fd5d4200640 1 -- 172.21.15.86:0/3383226360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5cc15de80 con 0x7fd5cc0cfca0 2024-04-25T14:10:47.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.005+0000 7fd5d4200640 1 -- 172.21.15.86:0/3383226360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5cc09f4b0 con 0x7fd5cc0cfca0 2024-04-25T14:10:47.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.005+0000 7fd5c9600640 1 -- 172.21.15.86:0/3383226360 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7fd5b801a070 con 0x7fd5cc0cfca0 2024-04-25T14:10:47.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.005+0000 7fd5c9600640 1 --2- 172.21.15.86:0/3383226360 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fd5ac06f490 0x7fd5ac0719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:47.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.009+0000 7fd5c9600640 1 -- 172.21.15.86:0/3383226360 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7fd5b808e790 con 0x7fd5cc0cfca0 2024-04-25T14:10:47.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.009+0000 7fd5d1800640 1 --2- 172.21.15.86:0/3383226360 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fd5ac06f490 0x7fd5ac0719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:47.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.009+0000 7fd5d1800640 1 --2- 172.21.15.86:0/3383226360 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fd5ac06f490 0x7fd5ac0719a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0001ec0 tx=0x7fd5c00016e0 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:47.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.013+0000 7fd5c9600640 1 -- 172.21.15.86:0/3383226360 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd5b8005480 con 0x7fd5cc0cfca0 2024-04-25T14:10:47.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:47.221+0000 7fd5d4200640 1 -- 172.21.15.86:0/3383226360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd5cc069d00 con 0x7fd5cc0cfca0 2024-04-25T14:10:47.226 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:47.229+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:48.013 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:48.017+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:48.023 INFO:tasks.ceph.mds.b.smithi155.stderr:2024-04-25T14:10:48.023+0000 7f3bf3400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T14:10:48.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.061+0000 7fd5c9600640 1 -- 172.21.15.86:0/3383226360 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v93) ==== 118+0+0 (secure 0 0 0) 0x7fd5b805ae40 con 0x7fd5cc0cfca0 2024-04-25T14:10:48.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.065+0000 7fd5d4200640 1 -- 172.21.15.86:0/3383226360 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fd5ac06f490 msgr2=0x7fd5ac0719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:48.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.065+0000 7fd5d4200640 1 --2- 172.21.15.86:0/3383226360 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fd5ac06f490 0x7fd5ac0719a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0001ec0 tx=0x7fd5c00016e0 comp rx=0 tx=0).stop 2024-04-25T14:10:48.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.065+0000 7fd5d4200640 1 -- 172.21.15.86:0/3383226360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5cc0cfca0 msgr2=0x7fd5cc0ca270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:48.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.065+0000 7fd5d4200640 1 --2- 172.21.15.86:0/3383226360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5cc0cfca0 0x7fd5cc0ca270 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800c3f0 tx=0x7fd5b800c8b0 comp rx=0 tx=0).stop 2024-04-25T14:10:48.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.065+0000 7fd5d4200640 1 -- 172.21.15.86:0/3383226360 shutdown_connections 2024-04-25T14:10:48.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.065+0000 7fd5d4200640 1 --2- 172.21.15.86:0/3383226360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5cc0d16e0 0x7fd5cc0ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:48.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.065+0000 7fd5d4200640 1 --2- 172.21.15.86:0/3383226360 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fd5ac06f490 0x7fd5ac0719a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:48.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.065+0000 7fd5d4200640 1 --2- 172.21.15.86:0/3383226360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5cc0cfca0 0x7fd5cc0ca270 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:48.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.065+0000 7fd5d4200640 1 --2- 172.21.15.86:0/3383226360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5cc0740c0 0x7fd5cc159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:48.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.065+0000 7fd5d4200640 1 -- 172.21.15.86:0/3383226360 >> 172.21.15.86:0/3383226360 conn(0x7fd5cc0b4650 msgr2=0x7fd5cc0b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:48.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.065+0000 7fd5d4200640 1 -- 172.21.15.86:0/3383226360 shutdown_connections 2024-04-25T14:10:48.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.065+0000 7fd5d4200640 1 -- 172.21.15.86:0/3383226360 wait complete. 2024-04-25T14:10:48.082 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T14:10:48.083 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T14:10:48.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.245+0000 7f1648200640 1 Processor -- start 2024-04-25T14:10:48.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.245+0000 7f1648200640 1 -- start start 2024-04-25T14:10:48.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.245+0000 7f1648200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1640160e90 0x7f16400d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:48.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.245+0000 7f1648200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1640163a80 con 0x7f1640072630 2024-04-25T14:10:48.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.245+0000 7f1648200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1640163d60 con 0x7f164006e0e0 2024-04-25T14:10:48.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.245+0000 7f1648200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1640164040 con 0x7f1640160e90 2024-04-25T14:10:48.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.245+0000 7f1645800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1640160e90 0x7f16400d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:48.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.245+0000 7f1644e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1640072630 0x7f1640072a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36448/0 (socket says 172.21.15.86:36448) 2024-04-25T14:10:48.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.245+0000 7f1644e00640 1 -- 172.21.15.86:0/1901577862 learned_addr learned my addr 172.21.15.86:0/1901577862 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:48.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.249+0000 7f1645800640 1 -- 172.21.15.86:0/1901577862 >> v1:172.21.15.155:6789/0 conn(0x7f164006e0e0 legacy=0x7f164006e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:48.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.249+0000 7f163fe00640 1 -- 172.21.15.86:0/1901577862 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3801004627 0 0) 0x7f1640163d60 con 0x7f164006e0e0 2024-04-25T14:10:48.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.249+0000 7f1645800640 1 -- 172.21.15.86:0/1901577862 >> v1:172.21.15.86:6789/0 conn(0x7f1640072630 legacy=0x7f1640072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:48.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.249+0000 7f1645800640 1 -- 172.21.15.86:0/1901577862 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1640164320 con 0x7f1640160e90 2024-04-25T14:10:48.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.249+0000 7f1645800640 1 --2- 172.21.15.86:0/1901577862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1640160e90 0x7f16400d1b30 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f16340097c0 tx=0x7f163401c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8362bbc580a2a9e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:48.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.249+0000 7f163fe00640 1 -- 172.21.15.86:0/1901577862 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16340b0070 con 0x7f1640160e90 2024-04-25T14:10:48.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.249+0000 7f163fe00640 1 -- 172.21.15.86:0/1901577862 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16340ab600 con 0x7f1640160e90 2024-04-25T14:10:48.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.249+0000 7f163fe00640 1 -- 172.21.15.86:0/1901577862 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16340ab9d0 con 0x7f1640160e90 2024-04-25T14:10:48.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.253+0000 7f1648200640 1 -- 172.21.15.86:0/1901577862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1640160e90 msgr2=0x7f16400d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:48.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.253+0000 7f1648200640 1 --2- 172.21.15.86:0/1901577862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1640160e90 0x7f16400d1b30 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f16340097c0 tx=0x7f163401c330 comp rx=0 tx=0).stop 2024-04-25T14:10:48.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.253+0000 7f1648200640 1 -- 172.21.15.86:0/1901577862 shutdown_connections 2024-04-25T14:10:48.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.253+0000 7f1648200640 1 --2- 172.21.15.86:0/1901577862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1640160e90 0x7f16400d1b30 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:48.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.253+0000 7f1648200640 1 -- 172.21.15.86:0/1901577862 >> 172.21.15.86:0/1901577862 conn(0x7f16400d3570 msgr2=0x7f16400d3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:48.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.253+0000 7f1648200640 1 -- 172.21.15.86:0/1901577862 shutdown_connections 2024-04-25T14:10:48.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.253+0000 7f1648200640 1 -- 172.21.15.86:0/1901577862 wait complete. 2024-04-25T14:10:48.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.253+0000 7f1648200640 1 Processor -- start 2024-04-25T14:10:48.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.253+0000 7f1648200640 1 -- start start 2024-04-25T14:10:48.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.257+0000 7f1648200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f164006e0e0 0x7f16400c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:48.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.257+0000 7f1648200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1640072630 0x7f16401f6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:48.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.257+0000 7f1648200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1640160e90 0x7f16401f7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:48.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.257+0000 7f1648200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f16400cdf90 con 0x7f1640072630 2024-04-25T14:10:48.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.257+0000 7f1648200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f16400cde10 con 0x7f1640160e90 2024-04-25T14:10:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.257+0000 7f1648200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f164006ec00 con 0x7f164006e0e0 2024-04-25T14:10:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.257+0000 7f1644e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1640072630 0x7f16401f6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.257+0000 7f1644e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1640072630 0x7f16401f6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53284/0 (socket says 172.21.15.86:53284) 2024-04-25T14:10:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.257+0000 7f1644e00640 1 -- 172.21.15.86:0/1779483268 learned_addr learned my addr 172.21.15.86:0/1779483268 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.257+0000 7f1645800640 1 --2- 172.21.15.86:0/1779483268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f164006e0e0 0x7f16400c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.257+0000 7f1646200640 1 --2- 172.21.15.86:0/1779483268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1640160e90 0x7f16401f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.257+0000 7f1644e00640 1 -- 172.21.15.86:0/1779483268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f164006e0e0 msgr2=0x7f16400c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.257+0000 7f1644e00640 1 --2- 172.21.15.86:0/1779483268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f164006e0e0 0x7f16400c79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.257+0000 7f1644e00640 1 -- 172.21.15.86:0/1779483268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1640160e90 msgr2=0x7f16401f7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.257+0000 7f1644e00640 1 --2- 172.21.15.86:0/1779483268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1640160e90 0x7f16401f7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.257+0000 7f1644e00640 1 -- 172.21.15.86:0/1779483268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16400b20a0 con 0x7f1640072630 2024-04-25T14:10:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.257+0000 7f1644e00640 1 --2- 172.21.15.86:0/1779483268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1640072630 0x7f16401f6c30 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f162c009dc0 tx=0x7f162c00c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.257+0000 7f163d600640 1 -- 172.21.15.86:0/1779483268 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c019070 con 0x7f1640072630 2024-04-25T14:10:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.257+0000 7f163d600640 1 -- 172.21.15.86:0/1779483268 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f162c0078f0 con 0x7f1640072630 2024-04-25T14:10:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.257+0000 7f1648200640 1 -- 172.21.15.86:0/1779483268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16400b2380 con 0x7f1640072630 2024-04-25T14:10:48.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.257+0000 7f163d600640 1 -- 172.21.15.86:0/1779483268 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c007ce0 con 0x7f1640072630 2024-04-25T14:10:48.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.257+0000 7f1648200640 1 -- 172.21.15.86:0/1779483268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16401f7cb0 con 0x7f1640072630 2024-04-25T14:10:48.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.257+0000 7f1648200640 1 -- 172.21.15.86:0/1779483268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16400c8050 con 0x7f1640072630 2024-04-25T14:10:48.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.261+0000 7f163d600640 1 -- 172.21.15.86:0/1779483268 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7f162c01c070 con 0x7f1640072630 2024-04-25T14:10:48.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.261+0000 7f163d600640 1 --2- 172.21.15.86:0/1779483268 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f162006f490 0x7f16200719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:48.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.261+0000 7f163d600640 1 -- 172.21.15.86:0/1779483268 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f162c091170 con 0x7f1640072630 2024-04-25T14:10:48.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.261+0000 7f1645800640 1 --2- 172.21.15.86:0/1779483268 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f162006f490 0x7f16200719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:48.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.261+0000 7f1645800640 1 --2- 172.21.15.86:0/1779483268 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f162006f490 0x7f16200719a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f16340ad000 tx=0x7f16340a6000 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.265+0000 7f163d600640 1 -- 172.21.15.86:0/1779483268 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f162c05d790 con 0x7f1640072630 2024-04-25T14:10:48.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:48.473+0000 7f1648200640 1 -- 172.21.15.86:0/1779483268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1640072170 con 0x7f1640072630 2024-04-25T14:10:48.813 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:48.813+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:49.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.065+0000 7f163d600640 1 -- 172.21.15.86:0/1779483268 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v94) ==== 122+0+0 (secure 0 0 0) 0x7f162c061a90 con 0x7f1640072630 2024-04-25T14:10:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.065+0000 7f1648200640 1 -- 172.21.15.86:0/1779483268 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f162006f490 msgr2=0x7f16200719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.065+0000 7f1648200640 1 --2- 172.21.15.86:0/1779483268 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f162006f490 0x7f16200719a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f16340ad000 tx=0x7f16340a6000 comp rx=0 tx=0).stop 2024-04-25T14:10:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.065+0000 7f1648200640 1 -- 172.21.15.86:0/1779483268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1640072630 msgr2=0x7f16401f6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.065+0000 7f1648200640 1 --2- 172.21.15.86:0/1779483268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1640072630 0x7f16401f6c30 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f162c009dc0 tx=0x7f162c00c430 comp rx=0 tx=0).stop 2024-04-25T14:10:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.065+0000 7f1648200640 1 -- 172.21.15.86:0/1779483268 shutdown_connections 2024-04-25T14:10:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.065+0000 7f1648200640 1 --2- 172.21.15.86:0/1779483268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1640160e90 0x7f16401f7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.065+0000 7f1648200640 1 --2- 172.21.15.86:0/1779483268 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f162006f490 0x7f16200719a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.065+0000 7f1648200640 1 --2- 172.21.15.86:0/1779483268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1640072630 0x7f16401f6c30 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.065+0000 7f1648200640 1 --2- 172.21.15.86:0/1779483268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f164006e0e0 0x7f16400c79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.065+0000 7f1648200640 1 -- 172.21.15.86:0/1779483268 >> 172.21.15.86:0/1779483268 conn(0x7f16400d3570 msgr2=0x7f16400d3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:49.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.065+0000 7f1648200640 1 -- 172.21.15.86:0/1779483268 shutdown_connections 2024-04-25T14:10:49.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.065+0000 7f1648200640 1 -- 172.21.15.86:0/1779483268 wait complete. 2024-04-25T14:10:49.085 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T14:10:49.085 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:10:49.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.241+0000 7f68c7400640 1 Processor -- start 2024-04-25T14:10:49.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.241+0000 7f68c7400640 1 -- start start 2024-04-25T14:10:49.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.241+0000 7f68c7400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f68c00b3970 0x7f68c00d2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:49.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.241+0000 7f68c7400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68c00ceb60 con 0x7f68c00d28d0 2024-04-25T14:10:49.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.241+0000 7f68c7400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68c00cee40 con 0x7f68c00cf920 2024-04-25T14:10:49.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.241+0000 7f68c7400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f68c00b2eb0 con 0x7f68c00b3970 2024-04-25T14:10:49.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.241+0000 7f68c5400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f68c00d28d0 0x7f68c006c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54942/0 (socket says 172.21.15.86:54942) 2024-04-25T14:10:49.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.241+0000 7f68c5400640 1 -- 172.21.15.86:0/3901789348 learned_addr learned my addr 172.21.15.86:0/3901789348 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:49.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.241+0000 7f68bfe00640 1 --2- 172.21.15.86:0/3901789348 >> v2:172.21.15.155:3300/0 conn(0x7f68c00b3970 0x7f68c00d2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:49.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.245+0000 7f68bf400640 1 -- 172.21.15.86:0/3901789348 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2492553081 0 0) 0x7f68c00ceb60 con 0x7f68c00d28d0 2024-04-25T14:10:49.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.245+0000 7f68bf400640 1 -- 172.21.15.86:0/3901789348 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f68a8003530 con 0x7f68c00d28d0 2024-04-25T14:10:49.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.245+0000 7f68bfe00640 1 -- 172.21.15.86:0/3901789348 >> v1:172.21.15.155:6789/0 conn(0x7f68c00cf920 legacy=0x7f68c00b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:49.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.245+0000 7f68bf400640 1 -- 172.21.15.86:0/3901789348 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1679781201 0 0) 0x7f68c00cee40 con 0x7f68c00cf920 2024-04-25T14:10:49.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.245+0000 7f68bfe00640 1 -- 172.21.15.86:0/3901789348 >> v1:172.21.15.86:6789/0 conn(0x7f68c00d28d0 legacy=0x7f68c006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:49.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.245+0000 7f68bfe00640 1 -- 172.21.15.86:0/3901789348 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c00cf120 con 0x7f68c00b3970 2024-04-25T14:10:49.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.245+0000 7f68bfe00640 1 --2- 172.21.15.86:0/3901789348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68c00b3970 0x7f68c00d2390 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0097c0 tx=0x7f68ac01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acbac77b021458ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:49.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.245+0000 7f68bf400640 1 -- 172.21.15.86:0/3901789348 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac0b0070 con 0x7f68c00b3970 2024-04-25T14:10:49.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.245+0000 7f68bf400640 1 -- 172.21.15.86:0/3901789348 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68ac0ab600 con 0x7f68c00b3970 2024-04-25T14:10:49.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.245+0000 7f68bf400640 1 -- 172.21.15.86:0/3901789348 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac0ab9d0 con 0x7f68c00b3970 2024-04-25T14:10:49.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.249+0000 7f68c7400640 1 -- 172.21.15.86:0/3901789348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68c00b3970 msgr2=0x7f68c00d2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:49.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.249+0000 7f68c7400640 1 --2- 172.21.15.86:0/3901789348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68c00b3970 0x7f68c00d2390 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0097c0 tx=0x7f68ac01c4a0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.249+0000 7f68c7400640 1 -- 172.21.15.86:0/3901789348 shutdown_connections 2024-04-25T14:10:49.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.249+0000 7f68c7400640 1 --2- 172.21.15.86:0/3901789348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68c00b3970 0x7f68c00d2390 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.249+0000 7f68c7400640 1 -- 172.21.15.86:0/3901789348 >> 172.21.15.86:0/3901789348 conn(0x7f68c009ba30 msgr2=0x7f68c00b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:49.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.249+0000 7f68c7400640 1 -- 172.21.15.86:0/3901789348 shutdown_connections 2024-04-25T14:10:49.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.249+0000 7f68c7400640 1 -- 172.21.15.86:0/3901789348 wait complete. 2024-04-25T14:10:49.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.249+0000 7f68c7400640 1 Processor -- start 2024-04-25T14:10:49.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.249+0000 7f68c7400640 1 -- start start 2024-04-25T14:10:49.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.253+0000 7f68c7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68c00b3970 0x7f68c00c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:49.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.253+0000 7f68c7400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68c00cf920 0x7f68c00c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:49.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.253+0000 7f68c7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68c00d28d0 0x7f68c0169020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:49.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.253+0000 7f68c7400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f68c016c620 con 0x7f68c00b3970 2024-04-25T14:10:49.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.253+0000 7f68c7400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f68c00cfd30 con 0x7f68c00d28d0 2024-04-25T14:10:49.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.253+0000 7f68c7400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f68c016c7a0 con 0x7f68c00cf920 2024-04-25T14:10:49.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.253+0000 7f68c4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68c00b3970 0x7f68c00c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:49.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.253+0000 7f68c4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68c00b3970 0x7f68c00c83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46124/0 (socket says 172.21.15.86:46124) 2024-04-25T14:10:49.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.253+0000 7f68c4a00640 1 -- 172.21.15.86:0/2503692477 learned_addr learned my addr 172.21.15.86:0/2503692477 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:49.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.253+0000 7f68c5400640 1 --2- 172.21.15.86:0/2503692477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68c00d28d0 0x7f68c0169020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:49.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.253+0000 7f68bfe00640 1 --2- 172.21.15.86:0/2503692477 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68c00cf920 0x7f68c00c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:49.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.253+0000 7f68c4a00640 1 -- 172.21.15.86:0/2503692477 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68c00cf920 msgr2=0x7f68c00c6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:49.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.253+0000 7f68c4a00640 1 --2- 172.21.15.86:0/2503692477 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68c00cf920 0x7f68c00c6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.253+0000 7f68c4a00640 1 -- 172.21.15.86:0/2503692477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68c00d28d0 msgr2=0x7f68c0169020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:49.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.253+0000 7f68c4a00640 1 --2- 172.21.15.86:0/2503692477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68c00d28d0 0x7f68c0169020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.253+0000 7f68c4a00640 1 -- 172.21.15.86:0/2503692477 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c0166b00 con 0x7f68c00b3970 2024-04-25T14:10:49.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.253+0000 7f68c4a00640 1 --2- 172.21.15.86:0/2503692477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68c00b3970 0x7f68c00c83f0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f68b400c440 tx=0x7f68b400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:49.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.253+0000 7f68bcc00640 1 -- 172.21.15.86:0/2503692477 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b4005480 con 0x7f68c00b3970 2024-04-25T14:10:49.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.253+0000 7f68bcc00640 1 -- 172.21.15.86:0/2503692477 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68b4003600 con 0x7f68c00b3970 2024-04-25T14:10:49.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.253+0000 7f68bcc00640 1 -- 172.21.15.86:0/2503692477 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b40039d0 con 0x7f68c00b3970 2024-04-25T14:10:49.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.253+0000 7f68c7400640 1 -- 172.21.15.86:0/2503692477 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c0169560 con 0x7f68c00b3970 2024-04-25T14:10:49.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.253+0000 7f68c7400640 1 -- 172.21.15.86:0/2503692477 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c00c8e90 con 0x7f68c00b3970 2024-04-25T14:10:49.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.253+0000 7f68c7400640 1 -- 172.21.15.86:0/2503692477 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68c00a0630 con 0x7f68c00b3970 2024-04-25T14:10:49.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.257+0000 7f68bcc00640 1 -- 172.21.15.86:0/2503692477 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7f68b4003b70 con 0x7f68c00b3970 2024-04-25T14:10:49.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.257+0000 7f68bcc00640 1 --2- 172.21.15.86:0/2503692477 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f68a006f4c0 0x7f68a00719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:49.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.257+0000 7f68bcc00640 1 -- 172.21.15.86:0/2503692477 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f68b408ef70 con 0x7f68c00b3970 2024-04-25T14:10:49.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.257+0000 7f68bfe00640 1 --2- 172.21.15.86:0/2503692477 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f68a006f4c0 0x7f68a00719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:49.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.257+0000 7f68bfe00640 1 --2- 172.21.15.86:0/2503692477 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f68a006f4c0 0x7f68a00719d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0ad000 tx=0x7f68ac0a6000 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:49.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.261+0000 7f68bcc00640 1 -- 172.21.15.86:0/2503692477 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f68b405b5d0 con 0x7f68c00b3970 2024-04-25T14:10:49.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.461+0000 7f68c7400640 1 -- 172.21.15.86:0/2503692477 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68c00b28d0 con 0x7f68c00b3970 2024-04-25T14:10:49.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.461+0000 7f68bcc00640 1 -- 172.21.15.86:0/2503692477 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5186 (secure 0 0 0) 0x7f68b405f8d0 con 0x7f68c00b3970 2024-04-25T14:10:49.459 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:49.459 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":94,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2258481468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2258481468},{"type":"v1","addr":"172.21.15.155:6834","nonce":2258481468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":8976,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1064843151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1064843151},{"type":"v1","addr":"172.21.15.86:6835","nonce":1064843151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":8985,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1940755625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1940755625},{"type":"v1","addr":"172.21.15.86:6837","nonce":1940755625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:10:45.497688+0000","modified":"2024-04-25T14:10:48.475869+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8991":{"gid":8991,"name":"b","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6835/544985680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":544985680},{"type":"v1","addr":"172.21.15.155:6835","nonce":544985680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8991,"qdb_cluster":[8991]},"id":9}]} 2024-04-25T14:10:49.459 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 94 2024-04-25T14:10:49.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.465+0000 7f68c7400640 1 -- 172.21.15.86:0/2503692477 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f68a006f4c0 msgr2=0x7f68a00719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:49.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.465+0000 7f68c7400640 1 --2- 172.21.15.86:0/2503692477 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f68a006f4c0 0x7f68a00719d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0ad000 tx=0x7f68ac0a6000 comp rx=0 tx=0).stop 2024-04-25T14:10:49.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.465+0000 7f68c7400640 1 -- 172.21.15.86:0/2503692477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68c00b3970 msgr2=0x7f68c00c83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:49.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.465+0000 7f68c7400640 1 --2- 172.21.15.86:0/2503692477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68c00b3970 0x7f68c00c83f0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f68b400c440 tx=0x7f68b400c900 comp rx=0 tx=0).stop 2024-04-25T14:10:49.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.465+0000 7f68c7400640 1 -- 172.21.15.86:0/2503692477 shutdown_connections 2024-04-25T14:10:49.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.465+0000 7f68c7400640 1 --2- 172.21.15.86:0/2503692477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68c00d28d0 0x7f68c0169020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.465+0000 7f68c7400640 1 --2- 172.21.15.86:0/2503692477 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68c00cf920 0x7f68c00c6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.465+0000 7f68c7400640 1 --2- 172.21.15.86:0/2503692477 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f68a006f4c0 0x7f68a00719d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.465+0000 7f68c7400640 1 --2- 172.21.15.86:0/2503692477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68c00b3970 0x7f68c00c83f0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.465+0000 7f68c7400640 1 -- 172.21.15.86:0/2503692477 >> 172.21.15.86:0/2503692477 conn(0x7f68c009ba30 msgr2=0x7f68c006fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:49.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.465+0000 7f68c7400640 1 -- 172.21.15.86:0/2503692477 shutdown_connections 2024-04-25T14:10:49.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.465+0000 7f68c7400640 1 -- 172.21.15.86:0/2503692477 wait complete. 2024-04-25T14:10:49.482 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:10:49.611 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:49.613+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:49.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.637+0000 7f3a2fc00640 1 Processor -- start 2024-04-25T14:10:49.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.637+0000 7f3a2fc00640 1 -- start start 2024-04-25T14:10:49.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.637+0000 7f3a2fc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a280cfca0 0x7f3a2806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:49.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.641+0000 7f3a2fc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3a280730b0 0x7f3a280734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:49.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.641+0000 7f3a2fc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a28072bd0 con 0x7f3a280d16e0 2024-04-25T14:10:49.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.641+0000 7f3a2fc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3a28070470 con 0x7f3a280730b0 2024-04-25T14:10:49.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.641+0000 7f3a2fc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3a2806f930 con 0x7f3a280cfca0 2024-04-25T14:10:49.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.641+0000 7f3a27e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3a280d16e0 0x7f3a280d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54956/0 (socket says 172.21.15.86:54956) 2024-04-25T14:10:49.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.641+0000 7f3a27e00640 1 -- 172.21.15.86:0/1578211774 learned_addr learned my addr 172.21.15.86:0/1578211774 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:49.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.641+0000 7f3a2dc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3a280730b0 0x7f3a280734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:49.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.641+0000 7f3a2d200640 1 --2- 172.21.15.86:0/1578211774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a280cfca0 0x7f3a2806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:49.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.641+0000 7f3a2dc00640 1 -- 172.21.15.86:0/1578211774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a280cfca0 msgr2=0x7f3a2806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:49.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.641+0000 7f3a2dc00640 1 --2- 172.21.15.86:0/1578211774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a280cfca0 0x7f3a2806f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.641+0000 7f3a2dc00640 1 -- 172.21.15.86:0/1578211774 >> v1:172.21.15.86:6789/0 conn(0x7f3a280d16e0 legacy=0x7f3a280d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:49.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.641+0000 7f3a2dc00640 1 -- 172.21.15.86:0/1578211774 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2806fbd0 con 0x7f3a280730b0 2024-04-25T14:10:49.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.641+0000 7f3a27400640 1 -- 172.21.15.86:0/1578211774 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2477431319 0 0) 0x7f3a28072bd0 con 0x7f3a280d16e0 2024-04-25T14:10:49.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.641+0000 7f3a2dc00640 1 --2- 172.21.15.86:0/1578211774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a280730b0 0x7f3a280734c0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009d90 tx=0x7f3a2001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c13c5773bce5ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:49.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.645+0000 7f3a27400640 1 -- 172.21.15.86:0/1578211774 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a200b2070 con 0x7f3a280730b0 2024-04-25T14:10:49.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.645+0000 7f3a27400640 1 -- 172.21.15.86:0/1578211774 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a200a5940 con 0x7f3a280730b0 2024-04-25T14:10:49.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.645+0000 7f3a27400640 1 -- 172.21.15.86:0/1578211774 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a200a5d10 con 0x7f3a280730b0 2024-04-25T14:10:49.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.645+0000 7f3a2fc00640 1 -- 172.21.15.86:0/1578211774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a280730b0 msgr2=0x7f3a280734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:49.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.645+0000 7f3a2fc00640 1 --2- 172.21.15.86:0/1578211774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a280730b0 0x7f3a280734c0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009d90 tx=0x7f3a2001c970 comp rx=0 tx=0).stop 2024-04-25T14:10:49.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.645+0000 7f3a2fc00640 1 -- 172.21.15.86:0/1578211774 shutdown_connections 2024-04-25T14:10:49.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.645+0000 7f3a2fc00640 1 --2- 172.21.15.86:0/1578211774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a280730b0 0x7f3a280734c0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.645+0000 7f3a2fc00640 1 --2- 172.21.15.86:0/1578211774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a280cfca0 0x7f3a2806f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.645+0000 7f3a2fc00640 1 -- 172.21.15.86:0/1578211774 >> 172.21.15.86:0/1578211774 conn(0x7f3a280b4650 msgr2=0x7f3a280b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:49.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.645+0000 7f3a2fc00640 1 -- 172.21.15.86:0/1578211774 shutdown_connections 2024-04-25T14:10:49.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.645+0000 7f3a2fc00640 1 -- 172.21.15.86:0/1578211774 wait complete. 2024-04-25T14:10:49.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.645+0000 7f3a2fc00640 1 Processor -- start 2024-04-25T14:10:49.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.645+0000 7f3a2fc00640 1 -- start start 2024-04-25T14:10:49.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.649+0000 7f3a2fc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a280730b0 0x7f3a2815f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:49.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.649+0000 7f3a2fc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a280cfca0 0x7f3a28153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:49.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.649+0000 7f3a2fc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a280d16e0 0x7f3a281542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:49.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.649+0000 7f3a2fc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3a28169a70 con 0x7f3a280d16e0 2024-04-25T14:10:49.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.649+0000 7f3a2fc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3a281698f0 con 0x7f3a280cfca0 2024-04-25T14:10:49.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.649+0000 7f3a2fc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3a28169bf0 con 0x7f3a280730b0 2024-04-25T14:10:49.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.649+0000 7f3a2dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a280d16e0 0x7f3a281542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:49.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.649+0000 7f3a2d200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a280730b0 0x7f3a2815f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:49.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.649+0000 7f3a2dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a280d16e0 0x7f3a281542d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46150/0 (socket says 172.21.15.86:46150) 2024-04-25T14:10:49.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.649+0000 7f3a2d200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a280730b0 0x7f3a2815f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:47936/0 (socket says 172.21.15.86:47936) 2024-04-25T14:10:49.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.649+0000 7f3a2dc00640 1 -- 172.21.15.86:0/1494708127 learned_addr learned my addr 172.21.15.86:0/1494708127 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:49.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.649+0000 7f3a27e00640 1 --2- 172.21.15.86:0/1494708127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a280cfca0 0x7f3a28153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:49.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.649+0000 7f3a2dc00640 1 -- 172.21.15.86:0/1494708127 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a280730b0 msgr2=0x7f3a2815f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:49.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.649+0000 7f3a2dc00640 1 --2- 172.21.15.86:0/1494708127 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a280730b0 0x7f3a2815f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.649+0000 7f3a2dc00640 1 -- 172.21.15.86:0/1494708127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a280cfca0 msgr2=0x7f3a28153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:49.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.649+0000 7f3a2dc00640 1 --2- 172.21.15.86:0/1494708127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a280cfca0 0x7f3a28153d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.649+0000 7f3a2dc00640 1 -- 172.21.15.86:0/1494708127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a28099800 con 0x7f3a280d16e0 2024-04-25T14:10:49.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.649+0000 7f3a2dc00640 1 --2- 172.21.15.86:0/1494708127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a280d16e0 0x7f3a281542d0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f3a200b0040 tx=0x7f3a20019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:49.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.649+0000 7f3a24c00640 1 -- 172.21.15.86:0/1494708127 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a200b2050 con 0x7f3a280d16e0 2024-04-25T14:10:49.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.649+0000 7f3a24c00640 1 -- 172.21.15.86:0/1494708127 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a200be070 con 0x7f3a280d16e0 2024-04-25T14:10:49.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.649+0000 7f3a2fc00640 1 -- 172.21.15.86:0/1494708127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a280b2ef0 con 0x7f3a280d16e0 2024-04-25T14:10:49.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.649+0000 7f3a24c00640 1 -- 172.21.15.86:0/1494708127 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a200ada90 con 0x7f3a280d16e0 2024-04-25T14:10:49.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.649+0000 7f3a2fc00640 1 -- 172.21.15.86:0/1494708127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a2815a990 con 0x7f3a280d16e0 2024-04-25T14:10:49.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.649+0000 7f3a2fc00640 1 -- 172.21.15.86:0/1494708127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a2809f4b0 con 0x7f3a280d16e0 2024-04-25T14:10:49.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.653+0000 7f3a24c00640 1 -- 172.21.15.86:0/1494708127 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7f3a200adc30 con 0x7f3a280d16e0 2024-04-25T14:10:49.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.653+0000 7f3a24c00640 1 --2- 172.21.15.86:0/1494708127 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f3a0806f490 0x7f3a080719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:49.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.653+0000 7f3a24c00640 1 -- 172.21.15.86:0/1494708127 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f3a2012a290 con 0x7f3a280d16e0 2024-04-25T14:10:49.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.653+0000 7f3a2d200640 1 --2- 172.21.15.86:0/1494708127 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f3a0806f490 0x7f3a080719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:49.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.653+0000 7f3a2d200640 1 --2- 172.21.15.86:0/1494708127 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f3a0806f490 0x7f3a080719a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3a28070710 tx=0x7f3a1c009190 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:49.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.657+0000 7f3a24c00640 1 -- 172.21.15.86:0/1494708127 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3a20016560 con 0x7f3a280d16e0 2024-04-25T14:10:49.762 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:10:49.765 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:10:49.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.829+0000 7f3a2fc00640 1 -- 172.21.15.86:0/1494708127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a28069d00 con 0x7f3a280d16e0 2024-04-25T14:10:49.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.829+0000 7f3a24c00640 1 -- 172.21.15.86:0/1494708127 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+16900 (secure 0 0 0) 0x7f3a200f6ad0 con 0x7f3a280d16e0 2024-04-25T14:10:49.827 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":126,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:10:47.052575+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:10:43.913889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T14:10:44.419362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:10:49.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.833+0000 7f3a2fc00640 1 -- 172.21.15.86:0/1494708127 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f3a0806f490 msgr2=0x7f3a080719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:49.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.833+0000 7f3a2fc00640 1 --2- 172.21.15.86:0/1494708127 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f3a0806f490 0x7f3a080719a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3a28070710 tx=0x7f3a1c009190 comp rx=0 tx=0).stop 2024-04-25T14:10:49.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.833+0000 7f3a2fc00640 1 -- 172.21.15.86:0/1494708127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a280d16e0 msgr2=0x7f3a281542d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:49.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.833+0000 7f3a2fc00640 1 --2- 172.21.15.86:0/1494708127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a280d16e0 0x7f3a281542d0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f3a200b0040 tx=0x7f3a20019710 comp rx=0 tx=0).stop 2024-04-25T14:10:49.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.833+0000 7f3a2fc00640 1 -- 172.21.15.86:0/1494708127 shutdown_connections 2024-04-25T14:10:49.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.833+0000 7f3a2fc00640 1 --2- 172.21.15.86:0/1494708127 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f3a0806f490 0x7f3a080719a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.833+0000 7f3a2fc00640 1 --2- 172.21.15.86:0/1494708127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a280d16e0 0x7f3a281542d0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.833+0000 7f3a2fc00640 1 --2- 172.21.15.86:0/1494708127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a280cfca0 0x7f3a28153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.833+0000 7f3a2fc00640 1 --2- 172.21.15.86:0/1494708127 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a280730b0 0x7f3a2815f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:49.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.833+0000 7f3a2fc00640 1 -- 172.21.15.86:0/1494708127 >> 172.21.15.86:0/1494708127 conn(0x7f3a280b4650 msgr2=0x7f3a280ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:49.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.833+0000 7f3a2fc00640 1 -- 172.21.15.86:0/1494708127 shutdown_connections 2024-04-25T14:10:49.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:49.833+0000 7f3a2fc00640 1 -- 172.21.15.86:0/1494708127 wait complete. 2024-04-25T14:10:49.850 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T14:10:49.850 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:10:50.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.049+0000 7f357a400640 1 Processor -- start 2024-04-25T14:10:50.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.049+0000 7f357a400640 1 -- start start 2024-04-25T14:10:50.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.053+0000 7f357a400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35740b36f0 0x7f35740b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:50.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.053+0000 7f357a400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f357415bec0 0x7f357415c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:50.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.053+0000 7f357a400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35740b3b00 con 0x7f357406d780 2024-04-25T14:10:50.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.053+0000 7f357a400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f35740d1c40 con 0x7f357415bec0 2024-04-25T14:10:50.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.053+0000 7f357a400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f35740b1a90 con 0x7f35740b36f0 2024-04-25T14:10:50.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.053+0000 7f3579a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f357415bec0 0x7f357415c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:50.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.053+0000 7f3579a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f357415bec0 0x7f357415c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46160/0 (socket says 172.21.15.86:46160) 2024-04-25T14:10:50.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.053+0000 7f3579a00640 1 -- 172.21.15.86:0/801579974 learned_addr learned my addr 172.21.15.86:0/801579974 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:50.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.053+0000 7f3579000640 1 --2- 172.21.15.86:0/801579974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35740b36f0 0x7f35740b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:50.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.053+0000 7f3579a00640 1 -- 172.21.15.86:0/801579974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35740b36f0 msgr2=0x7f35740b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:50.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.053+0000 7f3579a00640 1 --2- 172.21.15.86:0/801579974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35740b36f0 0x7f35740b77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.053+0000 7f3573400640 1 -- 172.21.15.86:0/801579974 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 136476514 0 0) 0x7f35740b3b00 con 0x7f357406d780 2024-04-25T14:10:50.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.053+0000 7f3579a00640 1 -- 172.21.15.86:0/801579974 >> v1:172.21.15.155:6789/0 conn(0x7f357406d780 legacy=0x7f357406db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:50.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.053+0000 7f3579a00640 1 -- 172.21.15.86:0/801579974 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35740b2250 con 0x7f357415bec0 2024-04-25T14:10:50.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.057+0000 7f3579a00640 1 --2- 172.21.15.86:0/801579974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f357415bec0 0x7f357415c2d0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f356c009e50 tx=0x7f356c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf733a1f4e0fbd2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:50.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.057+0000 7f3573400640 1 -- 172.21.15.86:0/801579974 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356c0a5700 con 0x7f357415bec0 2024-04-25T14:10:50.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.057+0000 7f3573400640 1 -- 172.21.15.86:0/801579974 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f356c0a58a0 con 0x7f357415bec0 2024-04-25T14:10:50.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.057+0000 7f3573400640 1 -- 172.21.15.86:0/801579974 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356c0a5c70 con 0x7f357415bec0 2024-04-25T14:10:50.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.057+0000 7f357a400640 1 -- 172.21.15.86:0/801579974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f357415bec0 msgr2=0x7f357415c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:50.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.057+0000 7f357a400640 1 --2- 172.21.15.86:0/801579974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f357415bec0 0x7f357415c2d0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f356c009e50 tx=0x7f356c01ca30 comp rx=0 tx=0).stop 2024-04-25T14:10:50.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.057+0000 7f357a400640 1 -- 172.21.15.86:0/801579974 shutdown_connections 2024-04-25T14:10:50.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.057+0000 7f357a400640 1 --2- 172.21.15.86:0/801579974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f357415bec0 0x7f357415c2d0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.057+0000 7f357a400640 1 --2- 172.21.15.86:0/801579974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35740b36f0 0x7f35740b77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.057+0000 7f357a400640 1 -- 172.21.15.86:0/801579974 >> 172.21.15.86:0/801579974 conn(0x7f357409ba30 msgr2=0x7f35740b70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:50.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.057+0000 7f357a400640 1 -- 172.21.15.86:0/801579974 shutdown_connections 2024-04-25T14:10:50.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.057+0000 7f357a400640 1 -- 172.21.15.86:0/801579974 wait complete. 2024-04-25T14:10:50.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.057+0000 7f357a400640 1 Processor -- start 2024-04-25T14:10:50.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.057+0000 7f357a400640 1 -- start start 2024-04-25T14:10:50.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.061+0000 7f357a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f357406d780 0x7f3574167700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:50.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.061+0000 7f357a400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35740b36f0 0x7f357409d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:50.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.061+0000 7f357a400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f357415bec0 0x7f357409d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:50.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.061+0000 7f357a400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f357415cc20 con 0x7f357406d780 2024-04-25T14:10:50.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.061+0000 7f357a400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f35740d3140 con 0x7f35740b36f0 2024-04-25T14:10:50.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.061+0000 7f357a400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f35740d0650 con 0x7f357415bec0 2024-04-25T14:10:50.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.061+0000 7f3579000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f357406d780 0x7f3574167700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:50.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.061+0000 7f3579000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f357406d780 0x7f3574167700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46168/0 (socket says 172.21.15.86:46168) 2024-04-25T14:10:50.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.061+0000 7f3579000640 1 -- 172.21.15.86:0/17492787 learned_addr learned my addr 172.21.15.86:0/17492787 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:50.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.061+0000 7f3579a00640 1 --2- 172.21.15.86:0/17492787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f357415bec0 0x7f357409d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:50.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.061+0000 7f3573e00640 1 --2- 172.21.15.86:0/17492787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35740b36f0 0x7f357409d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:50.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.061+0000 7f3579000640 1 -- 172.21.15.86:0/17492787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f357415bec0 msgr2=0x7f357409d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:50.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.061+0000 7f3579000640 1 --2- 172.21.15.86:0/17492787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f357415bec0 0x7f357409d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.061+0000 7f3579000640 1 -- 172.21.15.86:0/17492787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35740b36f0 msgr2=0x7f357409d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:50.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.061+0000 7f3579000640 1 --2- 172.21.15.86:0/17492787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35740b36f0 0x7f357409d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.061+0000 7f3579000640 1 -- 172.21.15.86:0/17492787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35740ca790 con 0x7f357406d780 2024-04-25T14:10:50.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.061+0000 7f3579000640 1 --2- 172.21.15.86:0/17492787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f357406d780 0x7f3574167700 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f356800e870 tx=0x7f356800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:50.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.061+0000 7f3570c00640 1 -- 172.21.15.86:0/17492787 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3568017070 con 0x7f357406d780 2024-04-25T14:10:50.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.061+0000 7f3570c00640 1 -- 172.21.15.86:0/17492787 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f356800c940 con 0x7f357406d780 2024-04-25T14:10:50.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.061+0000 7f357a400640 1 -- 172.21.15.86:0/17492787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35740caa70 con 0x7f357406d780 2024-04-25T14:10:50.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.061+0000 7f3570c00640 1 -- 172.21.15.86:0/17492787 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356800cd10 con 0x7f357406d780 2024-04-25T14:10:50.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.061+0000 7f357a400640 1 -- 172.21.15.86:0/17492787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35740cb030 con 0x7f357406d780 2024-04-25T14:10:50.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.061+0000 7f357a400640 1 -- 172.21.15.86:0/17492787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3574076b80 con 0x7f357406d780 2024-04-25T14:10:50.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.065+0000 7f3570c00640 1 -- 172.21.15.86:0/17492787 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7f356801a070 con 0x7f357406d780 2024-04-25T14:10:50.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.065+0000 7f3570c00640 1 --2- 172.21.15.86:0/17492787 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f355406f490 0x7f35540719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:50.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.065+0000 7f3570c00640 1 -- 172.21.15.86:0/17492787 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f356808fa10 con 0x7f357406d780 2024-04-25T14:10:50.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.069+0000 7f3573e00640 1 --2- 172.21.15.86:0/17492787 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f355406f490 0x7f35540719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:50.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.069+0000 7f3573e00640 1 --2- 172.21.15.86:0/17492787 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f355406f490 0x7f35540719a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3560007fd0 tx=0x7f3560009040 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:50.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.069+0000 7f3570c00640 1 -- 172.21.15.86:0/17492787 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f356805c1d0 con 0x7f357406d780 2024-04-25T14:10:50.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.233+0000 7f357a400640 1 -- 172.21.15.86:0/17492787 --> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f35740cb4f0 con 0x7f355406f490 2024-04-25T14:10:50.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.241+0000 7f3570c00640 1 -- 172.21.15.86:0/17492787 <== mgr.8338 v2:172.21.15.86:6832/3964666460 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247083 (secure 0 0 0) 0x7f35740cb4f0 con 0x7f355406f490 2024-04-25T14:10:50.241 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:50.243 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:10:50.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.245+0000 7f357a400640 1 -- 172.21.15.86:0/17492787 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f355406f490 msgr2=0x7f35540719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:50.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.245+0000 7f357a400640 1 --2- 172.21.15.86:0/17492787 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f355406f490 0x7f35540719a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3560007fd0 tx=0x7f3560009040 comp rx=0 tx=0).stop 2024-04-25T14:10:50.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.245+0000 7f357a400640 1 -- 172.21.15.86:0/17492787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f357406d780 msgr2=0x7f3574167700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:50.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.245+0000 7f357a400640 1 --2- 172.21.15.86:0/17492787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f357406d780 0x7f3574167700 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f356800e870 tx=0x7f356800ec70 comp rx=0 tx=0).stop 2024-04-25T14:10:50.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.245+0000 7f357a400640 1 -- 172.21.15.86:0/17492787 shutdown_connections 2024-04-25T14:10:50.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.245+0000 7f357a400640 1 --2- 172.21.15.86:0/17492787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f357415bec0 0x7f357409d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.245+0000 7f357a400640 1 --2- 172.21.15.86:0/17492787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35740b36f0 0x7f357409d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.245+0000 7f357a400640 1 --2- 172.21.15.86:0/17492787 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f355406f490 0x7f35540719a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.245+0000 7f357a400640 1 --2- 172.21.15.86:0/17492787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f357406d780 0x7f3574167700 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.245+0000 7f357a400640 1 -- 172.21.15.86:0/17492787 >> 172.21.15.86:0/17492787 conn(0x7f357409ba30 msgr2=0x7f357406c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:50.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.245+0000 7f357a400640 1 -- 172.21.15.86:0/17492787 shutdown_connections 2024-04-25T14:10:50.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.245+0000 7f357a400640 1 -- 172.21.15.86:0/17492787 wait complete. 2024-04-25T14:10:50.264 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T14:10:49.615624+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":529,"num_read_kb":1389,"num_write":187,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2021,"ondisk_log_size":2021,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279328,"kb_used_data":13240,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754695392,"statfs":{"total":773094113280,"available":772808081408,"internally_reserved":0,"allocated":13557760,"data_stored":7349868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.588242"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.043709+0000","last_change":"2024-04-25T14:10:45.043709+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.043709+0000","last_undegraded":"2024-04-25T14:10:45.043709+0000","last_fullsized":"2024-04-25T14:10:45.043709+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.034723+0000","last_change":"2024-04-25T14:10:45.034723+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.034723+0000","last_undegraded":"2024-04-25T14:10:45.034723+0000","last_fullsized":"2024-04-25T14:10:45.034723+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038065+0000","last_change":"2024-04-25T14:10:45.038065+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038065+0000","last_undegraded":"2024-04-25T14:10:45.038065+0000","last_fullsized":"2024-04-25T14:10:45.038065+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.032697+0000","last_change":"2024-04-25T14:10:45.032697+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.032697+0000","last_undegraded":"2024-04-25T14:10:45.032697+0000","last_fullsized":"2024-04-25T14:10:45.032697+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038541+0000","last_change":"2024-04-25T14:10:45.038541+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038541+0000","last_undegraded":"2024-04-25T14:10:45.038541+0000","last_fullsized":"2024-04-25T14:10:45.038541+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.045986+0000","last_change":"2024-04-25T14:10:45.045986+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.045986+0000","last_undegraded":"2024-04-25T14:10:45.045986+0000","last_fullsized":"2024-04-25T14:10:45.045986+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.037964+0000","last_change":"2024-04-25T14:10:45.037964+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.037964+0000","last_undegraded":"2024-04-25T14:10:45.037964+0000","last_fullsized":"2024-04-25T14:10:45.037964+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.043120+0000","last_change":"2024-04-25T14:10:45.043120+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.043120+0000","last_undegraded":"2024-04-25T14:10:45.043120+0000","last_fullsized":"2024-04-25T14:10:45.043120+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.049467+0000","last_change":"2024-04-25T14:10:45.049467+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.049467+0000","last_undegraded":"2024-04-25T14:10:45.049467+0000","last_fullsized":"2024-04-25T14:10:45.049467+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039375+0000","last_change":"2024-04-25T14:10:45.039375+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039375+0000","last_undegraded":"2024-04-25T14:10:45.039375+0000","last_fullsized":"2024-04-25T14:10:45.039375+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.034076+0000","last_change":"2024-04-25T14:10:45.034076+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.034076+0000","last_undegraded":"2024-04-25T14:10:45.034076+0000","last_fullsized":"2024-04-25T14:10:45.034076+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.041407+0000","last_change":"2024-04-25T14:10:45.041407+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.041407+0000","last_undegraded":"2024-04-25T14:10:45.041407+0000","last_fullsized":"2024-04-25T14:10:45.041407+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.047463+0000","last_change":"2024-04-25T14:10:45.047463+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.047463+0000","last_undegraded":"2024-04-25T14:10:45.047463+0000","last_fullsized":"2024-04-25T14:10:45.047463+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.040256+0000","last_change":"2024-04-25T14:10:45.040256+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.040256+0000","last_undegraded":"2024-04-25T14:10:45.040256+0000","last_fullsized":"2024-04-25T14:10:45.040256+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.040028+0000","last_change":"2024-04-25T14:10:45.040028+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.040028+0000","last_undegraded":"2024-04-25T14:10:45.040028+0000","last_fullsized":"2024-04-25T14:10:45.040028+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.045366+0000","last_change":"2024-04-25T14:10:45.045366+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.045366+0000","last_undegraded":"2024-04-25T14:10:45.045366+0000","last_fullsized":"2024-04-25T14:10:45.045366+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.048856+0000","last_change":"2024-04-25T14:10:45.048856+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.048856+0000","last_undegraded":"2024-04-25T14:10:45.048856+0000","last_fullsized":"2024-04-25T14:10:45.048856+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.040344+0000","last_change":"2024-04-25T14:10:45.040344+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.040344+0000","last_undegraded":"2024-04-25T14:10:45.040344+0000","last_fullsized":"2024-04-25T14:10:45.040344+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039190+0000","last_change":"2024-04-25T14:10:45.039190+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039190+0000","last_undegraded":"2024-04-25T14:10:45.039190+0000","last_fullsized":"2024-04-25T14:10:45.039190+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038526+0000","last_change":"2024-04-25T14:10:45.038526+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038526+0000","last_undegraded":"2024-04-25T14:10:45.038526+0000","last_fullsized":"2024-04-25T14:10:45.038526+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.034790+0000","last_change":"2024-04-25T14:10:45.034790+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.034790+0000","last_undegraded":"2024-04-25T14:10:45.034790+0000","last_fullsized":"2024-04-25T14:10:45.034790+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.040708+0000","last_change":"2024-04-25T14:10:45.040708+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.040708+0000","last_undegraded":"2024-04-25T14:10:45.040708+0000","last_fullsized":"2024-04-25T14:10:45.040708+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.032697+0000","last_change":"2024-04-25T14:10:45.032697+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.032697+0000","last_undegraded":"2024-04-25T14:10:45.032697+0000","last_fullsized":"2024-04-25T14:10:45.032697+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.044889+0000","last_change":"2024-04-25T14:10:45.044889+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.044889+0000","last_undegraded":"2024-04-25T14:10:45.044889+0000","last_fullsized":"2024-04-25T14:10:45.044889+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039272+0000","last_change":"2024-04-25T14:10:45.039272+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039272+0000","last_undegraded":"2024-04-25T14:10:45.039272+0000","last_fullsized":"2024-04-25T14:10:45.039272+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038197+0000","last_change":"2024-04-25T14:10:45.038197+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038197+0000","last_undegraded":"2024-04-25T14:10:45.038197+0000","last_fullsized":"2024-04-25T14:10:45.038197+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038708+0000","last_change":"2024-04-25T14:10:45.038708+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038708+0000","last_undegraded":"2024-04-25T14:10:45.038708+0000","last_fullsized":"2024-04-25T14:10:45.038708+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039464+0000","last_change":"2024-04-25T14:10:45.039464+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039464+0000","last_undegraded":"2024-04-25T14:10:45.039464+0000","last_fullsized":"2024-04-25T14:10:45.039464+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.040768+0000","last_change":"2024-04-25T14:10:45.040768+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.040768+0000","last_undegraded":"2024-04-25T14:10:45.040768+0000","last_fullsized":"2024-04-25T14:10:45.040768+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038367+0000","last_change":"2024-04-25T14:10:45.038367+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038367+0000","last_undegraded":"2024-04-25T14:10:45.038367+0000","last_fullsized":"2024-04-25T14:10:45.038367+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.044360+0000","last_change":"2024-04-25T14:10:45.044360+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.044360+0000","last_undegraded":"2024-04-25T14:10:45.044360+0000","last_fullsized":"2024-04-25T14:10:45.044360+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039596+0000","last_change":"2024-04-25T14:10:45.039596+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039596+0000","last_undegraded":"2024-04-25T14:10:45.039596+0000","last_fullsized":"2024-04-25T14:10:45.039596+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.035461+0000","last_change":"2024-04-25T14:10:45.035461+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.035461+0000","last_undegraded":"2024-04-25T14:10:45.035461+0000","last_fullsized":"2024-04-25T14:10:45.035461+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.052380+0000","last_change":"2024-04-25T14:10:45.051471+0000","last_active":"2024-04-25T14:10:45.052380+0000","last_peered":"2024-04-25T14:10:45.052380+0000","last_clean":"2024-04-25T14:10:45.052380+0000","last_became_active":"2024-04-25T14:10:45.051055+0000","last_became_peered":"2024-04-25T14:10:45.051055+0000","last_unstale":"2024-04-25T14:10:45.052380+0000","last_undegraded":"2024-04-25T14:10:45.052380+0000","last_fullsized":"2024-04-25T14:10:45.052380+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:35.948249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039532+0000","last_change":"2024-04-25T14:10:45.039532+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039532+0000","last_undegraded":"2024-04-25T14:10:45.039532+0000","last_fullsized":"2024-04-25T14:10:45.039532+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.052927+0000","last_change":"2024-04-25T14:10:45.046859+0000","last_active":"2024-04-25T14:10:45.052927+0000","last_peered":"2024-04-25T14:10:45.052927+0000","last_clean":"2024-04-25T14:10:45.052927+0000","last_became_active":"2024-04-25T14:10:45.044618+0000","last_became_peered":"2024-04-25T14:10:45.044618+0000","last_unstale":"2024-04-25T14:10:45.052927+0000","last_undegraded":"2024-04-25T14:10:45.052927+0000","last_fullsized":"2024-04-25T14:10:45.052927+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:55.442843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.033422+0000","last_change":"2024-04-25T14:10:45.033422+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.033422+0000","last_undegraded":"2024-04-25T14:10:45.033422+0000","last_fullsized":"2024-04-25T14:10:45.033422+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.057542+0000","last_change":"2024-04-25T14:10:45.056867+0000","last_active":"2024-04-25T14:10:45.057542+0000","last_peered":"2024-04-25T14:10:45.057542+0000","last_clean":"2024-04-25T14:10:45.057542+0000","last_became_active":"2024-04-25T14:10:45.056470+0000","last_became_peered":"2024-04-25T14:10:45.056470+0000","last_unstale":"2024-04-25T14:10:45.057542+0000","last_undegraded":"2024-04-25T14:10:45.057542+0000","last_fullsized":"2024-04-25T14:10:45.057542+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:01.583932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.043935+0000","last_change":"2024-04-25T14:10:45.043935+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.043935+0000","last_undegraded":"2024-04-25T14:10:45.043935+0000","last_fullsized":"2024-04-25T14:10:45.043935+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.054827+0000","last_change":"2024-04-25T14:10:45.052367+0000","last_active":"2024-04-25T14:10:45.054827+0000","last_peered":"2024-04-25T14:10:45.054827+0000","last_clean":"2024-04-25T14:10:45.054827+0000","last_became_active":"2024-04-25T14:10:45.052055+0000","last_became_peered":"2024-04-25T14:10:45.052055+0000","last_unstale":"2024-04-25T14:10:45.054827+0000","last_undegraded":"2024-04-25T14:10:45.054827+0000","last_fullsized":"2024-04-25T14:10:45.054827+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:52.050283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.041991+0000","last_change":"2024-04-25T14:10:45.041991+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.041991+0000","last_undegraded":"2024-04-25T14:10:45.041991+0000","last_fullsized":"2024-04-25T14:10:45.041991+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.044829+0000","last_change":"2024-04-25T14:10:45.041897+0000","last_active":"2024-04-25T14:10:45.044829+0000","last_peered":"2024-04-25T14:10:45.044829+0000","last_clean":"2024-04-25T14:10:45.044829+0000","last_became_active":"2024-04-25T14:10:45.039679+0000","last_became_peered":"2024-04-25T14:10:45.039679+0000","last_unstale":"2024-04-25T14:10:45.044829+0000","last_undegraded":"2024-04-25T14:10:45.044829+0000","last_fullsized":"2024-04-25T14:10:45.044829+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:22.669468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.040465+0000","last_change":"2024-04-25T14:10:45.040465+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.040465+0000","last_undegraded":"2024-04-25T14:10:45.040465+0000","last_fullsized":"2024-04-25T14:10:45.040465+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.053034+0000","last_change":"2024-04-25T14:10:45.051135+0000","last_active":"2024-04-25T14:10:45.053034+0000","last_peered":"2024-04-25T14:10:45.053034+0000","last_clean":"2024-04-25T14:10:45.053034+0000","last_became_active":"2024-04-25T14:10:45.050463+0000","last_became_peered":"2024-04-25T14:10:45.050463+0000","last_unstale":"2024-04-25T14:10:45.053034+0000","last_undegraded":"2024-04-25T14:10:45.053034+0000","last_fullsized":"2024-04-25T14:10:45.053034+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:12.563128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039895+0000","last_change":"2024-04-25T14:10:45.039895+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039895+0000","last_undegraded":"2024-04-25T14:10:45.039895+0000","last_fullsized":"2024-04-25T14:10:45.039895+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.056835+0000","last_change":"2024-04-25T14:10:45.053969+0000","last_active":"2024-04-25T14:10:45.056835+0000","last_peered":"2024-04-25T14:10:45.056835+0000","last_clean":"2024-04-25T14:10:45.056835+0000","last_became_active":"2024-04-25T14:10:45.053634+0000","last_became_peered":"2024-04-25T14:10:45.053634+0000","last_unstale":"2024-04-25T14:10:45.056835+0000","last_undegraded":"2024-04-25T14:10:45.056835+0000","last_fullsized":"2024-04-25T14:10:45.056835+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:54.008821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.040667+0000","last_change":"2024-04-25T14:10:45.040667+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.040667+0000","last_undegraded":"2024-04-25T14:10:45.040667+0000","last_fullsized":"2024-04-25T14:10:45.040667+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.056301+0000","last_change":"2024-04-25T14:10:45.054900+0000","last_active":"2024-04-25T14:10:45.056301+0000","last_peered":"2024-04-25T14:10:45.056301+0000","last_clean":"2024-04-25T14:10:45.056301+0000","last_became_active":"2024-04-25T14:10:45.054599+0000","last_became_peered":"2024-04-25T14:10:45.054599+0000","last_unstale":"2024-04-25T14:10:45.056301+0000","last_undegraded":"2024-04-25T14:10:45.056301+0000","last_fullsized":"2024-04-25T14:10:45.056301+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:05.356606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039869+0000","last_change":"2024-04-25T14:10:45.039869+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039869+0000","last_undegraded":"2024-04-25T14:10:45.039869+0000","last_fullsized":"2024-04-25T14:10:45.039869+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.045456+0000","last_change":"2024-04-25T14:10:45.037687+0000","last_active":"2024-04-25T14:10:45.045456+0000","last_peered":"2024-04-25T14:10:45.045456+0000","last_clean":"2024-04-25T14:10:45.045456+0000","last_became_active":"2024-04-25T14:10:45.037370+0000","last_became_peered":"2024-04-25T14:10:45.037370+0000","last_unstale":"2024-04-25T14:10:45.045456+0000","last_undegraded":"2024-04-25T14:10:45.045456+0000","last_fullsized":"2024-04-25T14:10:45.045456+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:54.008821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.046751+0000","last_change":"2024-04-25T14:10:45.046751+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.046751+0000","last_undegraded":"2024-04-25T14:10:45.046751+0000","last_fullsized":"2024-04-25T14:10:45.046751+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.058741+0000","last_change":"2024-04-25T14:10:45.050926+0000","last_active":"2024-04-25T14:10:45.058741+0000","last_peered":"2024-04-25T14:10:45.058741+0000","last_clean":"2024-04-25T14:10:45.058741+0000","last_became_active":"2024-04-25T14:10:45.049733+0000","last_became_peered":"2024-04-25T14:10:45.049733+0000","last_unstale":"2024-04-25T14:10:45.058741+0000","last_undegraded":"2024-04-25T14:10:45.058741+0000","last_fullsized":"2024-04-25T14:10:45.058741+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:05.958831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.037539+0000","last_change":"2024-04-25T14:10:45.037539+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.037539+0000","last_undegraded":"2024-04-25T14:10:45.037539+0000","last_fullsized":"2024-04-25T14:10:45.037539+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.054475+0000","last_change":"2024-04-25T14:10:45.052411+0000","last_active":"2024-04-25T14:10:45.054475+0000","last_peered":"2024-04-25T14:10:45.054475+0000","last_clean":"2024-04-25T14:10:45.054475+0000","last_became_active":"2024-04-25T14:10:45.052020+0000","last_became_peered":"2024-04-25T14:10:45.052020+0000","last_unstale":"2024-04-25T14:10:45.054475+0000","last_undegraded":"2024-04-25T14:10:45.054475+0000","last_fullsized":"2024-04-25T14:10:45.054475+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:33.243994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.040238+0000","last_change":"2024-04-25T14:10:45.040238+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.040238+0000","last_undegraded":"2024-04-25T14:10:45.040238+0000","last_fullsized":"2024-04-25T14:10:45.040238+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.057096+0000","last_change":"2024-04-25T14:10:45.054962+0000","last_active":"2024-04-25T14:10:45.057096+0000","last_peered":"2024-04-25T14:10:45.057096+0000","last_clean":"2024-04-25T14:10:45.057096+0000","last_became_active":"2024-04-25T14:10:45.054639+0000","last_became_peered":"2024-04-25T14:10:45.054639+0000","last_unstale":"2024-04-25T14:10:45.057096+0000","last_undegraded":"2024-04-25T14:10:45.057096+0000","last_fullsized":"2024-04-25T14:10:45.057096+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:38.714815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038826+0000","last_change":"2024-04-25T14:10:45.038826+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038826+0000","last_undegraded":"2024-04-25T14:10:45.038826+0000","last_fullsized":"2024-04-25T14:10:45.038826+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.058736+0000","last_change":"2024-04-25T14:10:45.051737+0000","last_active":"2024-04-25T14:10:45.058736+0000","last_peered":"2024-04-25T14:10:45.058736+0000","last_clean":"2024-04-25T14:10:45.058736+0000","last_became_active":"2024-04-25T14:10:45.051337+0000","last_became_peered":"2024-04-25T14:10:45.051337+0000","last_unstale":"2024-04-25T14:10:45.058736+0000","last_undegraded":"2024-04-25T14:10:45.058736+0000","last_fullsized":"2024-04-25T14:10:45.058736+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:25.432658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039786+0000","last_change":"2024-04-25T14:10:45.039786+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039786+0000","last_undegraded":"2024-04-25T14:10:45.039786+0000","last_fullsized":"2024-04-25T14:10:45.039786+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.052338+0000","last_change":"2024-04-25T14:10:45.049002+0000","last_active":"2024-04-25T14:10:45.052338+0000","last_peered":"2024-04-25T14:10:45.052338+0000","last_clean":"2024-04-25T14:10:45.052338+0000","last_became_active":"2024-04-25T14:10:45.048755+0000","last_became_peered":"2024-04-25T14:10:45.048755+0000","last_unstale":"2024-04-25T14:10:45.052338+0000","last_undegraded":"2024-04-25T14:10:45.052338+0000","last_fullsized":"2024-04-25T14:10:45.052338+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:04.824306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.037596+0000","last_change":"2024-04-25T14:10:45.037596+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.037596+0000","last_undegraded":"2024-04-25T14:10:45.037596+0000","last_fullsized":"2024-04-25T14:10:45.037596+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.055500+0000","last_change":"2024-04-25T14:10:45.051770+0000","last_active":"2024-04-25T14:10:45.055500+0000","last_peered":"2024-04-25T14:10:45.055500+0000","last_clean":"2024-04-25T14:10:45.055500+0000","last_became_active":"2024-04-25T14:10:45.051446+0000","last_became_peered":"2024-04-25T14:10:45.051446+0000","last_unstale":"2024-04-25T14:10:45.055500+0000","last_undegraded":"2024-04-25T14:10:45.055500+0000","last_fullsized":"2024-04-25T14:10:45.055500+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:01.990667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038531+0000","last_change":"2024-04-25T14:10:45.038531+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038531+0000","last_undegraded":"2024-04-25T14:10:45.038531+0000","last_fullsized":"2024-04-25T14:10:45.038531+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.042874+0000","last_change":"2024-04-25T14:10:45.037175+0000","last_active":"2024-04-25T14:10:45.042874+0000","last_peered":"2024-04-25T14:10:45.042874+0000","last_clean":"2024-04-25T14:10:45.042874+0000","last_became_active":"2024-04-25T14:10:45.036881+0000","last_became_peered":"2024-04-25T14:10:45.036881+0000","last_unstale":"2024-04-25T14:10:45.042874+0000","last_undegraded":"2024-04-25T14:10:45.042874+0000","last_fullsized":"2024-04-25T14:10:45.042874+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:01.990667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.037392+0000","last_change":"2024-04-25T14:10:45.037392+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.037392+0000","last_undegraded":"2024-04-25T14:10:45.037392+0000","last_fullsized":"2024-04-25T14:10:45.037392+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.053697+0000","last_change":"2024-04-25T14:10:45.052421+0000","last_active":"2024-04-25T14:10:45.053697+0000","last_peered":"2024-04-25T14:10:45.053697+0000","last_clean":"2024-04-25T14:10:45.053697+0000","last_became_active":"2024-04-25T14:10:45.052078+0000","last_became_peered":"2024-04-25T14:10:45.052078+0000","last_unstale":"2024-04-25T14:10:45.053697+0000","last_undegraded":"2024-04-25T14:10:45.053697+0000","last_fullsized":"2024-04-25T14:10:45.053697+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:49.038115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039058+0000","last_change":"2024-04-25T14:10:45.039058+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039058+0000","last_undegraded":"2024-04-25T14:10:45.039058+0000","last_fullsized":"2024-04-25T14:10:45.039058+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.058211+0000","last_change":"2024-04-25T14:10:45.056493+0000","last_active":"2024-04-25T14:10:45.058211+0000","last_peered":"2024-04-25T14:10:45.058211+0000","last_clean":"2024-04-25T14:10:45.058211+0000","last_became_active":"2024-04-25T14:10:45.055271+0000","last_became_peered":"2024-04-25T14:10:45.055271+0000","last_unstale":"2024-04-25T14:10:45.058211+0000","last_undegraded":"2024-04-25T14:10:45.058211+0000","last_fullsized":"2024-04-25T14:10:45.058211+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:58.758388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.037958+0000","last_change":"2024-04-25T14:10:45.037958+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.037958+0000","last_undegraded":"2024-04-25T14:10:45.037958+0000","last_fullsized":"2024-04-25T14:10:45.037958+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.058425+0000","last_change":"2024-04-25T14:10:45.055677+0000","last_active":"2024-04-25T14:10:45.058425+0000","last_peered":"2024-04-25T14:10:45.058425+0000","last_clean":"2024-04-25T14:10:45.058425+0000","last_became_active":"2024-04-25T14:10:45.054262+0000","last_became_peered":"2024-04-25T14:10:45.054262+0000","last_unstale":"2024-04-25T14:10:45.058425+0000","last_undegraded":"2024-04-25T14:10:45.058425+0000","last_fullsized":"2024-04-25T14:10:45.058425+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:34.895882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038038+0000","last_change":"2024-04-25T14:10:45.038038+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038038+0000","last_undegraded":"2024-04-25T14:10:45.038038+0000","last_fullsized":"2024-04-25T14:10:45.038038+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.049582+0000","last_change":"2024-04-25T14:10:45.047266+0000","last_active":"2024-04-25T14:10:45.049582+0000","last_peered":"2024-04-25T14:10:45.049582+0000","last_clean":"2024-04-25T14:10:45.049582+0000","last_became_active":"2024-04-25T14:10:45.046928+0000","last_became_peered":"2024-04-25T14:10:45.046928+0000","last_unstale":"2024-04-25T14:10:45.049582+0000","last_undegraded":"2024-04-25T14:10:45.049582+0000","last_fullsized":"2024-04-25T14:10:45.049582+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:54.008821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.045070+0000","last_change":"2024-04-25T14:10:45.045070+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.045070+0000","last_undegraded":"2024-04-25T14:10:45.045070+0000","last_fullsized":"2024-04-25T14:10:45.045070+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.060466+0000","last_change":"2024-04-25T14:10:45.057160+0000","last_active":"2024-04-25T14:10:45.060466+0000","last_peered":"2024-04-25T14:10:45.060466+0000","last_clean":"2024-04-25T14:10:45.060466+0000","last_became_active":"2024-04-25T14:10:45.056871+0000","last_became_peered":"2024-04-25T14:10:45.056871+0000","last_unstale":"2024-04-25T14:10:45.060466+0000","last_undegraded":"2024-04-25T14:10:45.060466+0000","last_fullsized":"2024-04-25T14:10:45.060466+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:44.552534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038103+0000","last_change":"2024-04-25T14:10:45.038103+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038103+0000","last_undegraded":"2024-04-25T14:10:45.038103+0000","last_fullsized":"2024-04-25T14:10:45.038103+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.060444+0000","last_change":"2024-04-25T14:10:45.059841+0000","last_active":"2024-04-25T14:10:45.060444+0000","last_peered":"2024-04-25T14:10:45.060444+0000","last_clean":"2024-04-25T14:10:45.060444+0000","last_became_active":"2024-04-25T14:10:45.059609+0000","last_became_peered":"2024-04-25T14:10:45.059609+0000","last_unstale":"2024-04-25T14:10:45.060444+0000","last_undegraded":"2024-04-25T14:10:45.060444+0000","last_fullsized":"2024-04-25T14:10:45.060444+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:14.821225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.033610+0000","last_change":"2024-04-25T14:10:45.033610+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.033610+0000","last_undegraded":"2024-04-25T14:10:45.033610+0000","last_fullsized":"2024-04-25T14:10:45.033610+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.061732+0000","last_change":"2024-04-25T14:10:45.057442+0000","last_active":"2024-04-25T14:10:45.061732+0000","last_peered":"2024-04-25T14:10:45.061732+0000","last_clean":"2024-04-25T14:10:45.061732+0000","last_became_active":"2024-04-25T14:10:45.057077+0000","last_became_peered":"2024-04-25T14:10:45.057077+0000","last_unstale":"2024-04-25T14:10:45.061732+0000","last_undegraded":"2024-04-25T14:10:45.061732+0000","last_fullsized":"2024-04-25T14:10:45.061732+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:03.729472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.037513+0000","last_change":"2024-04-25T14:10:45.037513+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.037513+0000","last_undegraded":"2024-04-25T14:10:45.037513+0000","last_fullsized":"2024-04-25T14:10:45.037513+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.060351+0000","last_change":"2024-04-25T14:10:45.056735+0000","last_active":"2024-04-25T14:10:45.060351+0000","last_peered":"2024-04-25T14:10:45.060351+0000","last_clean":"2024-04-25T14:10:45.060351+0000","last_became_active":"2024-04-25T14:10:45.052530+0000","last_became_peered":"2024-04-25T14:10:45.052530+0000","last_unstale":"2024-04-25T14:10:45.060351+0000","last_undegraded":"2024-04-25T14:10:45.060351+0000","last_fullsized":"2024-04-25T14:10:45.060351+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:34.861278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.037754+0000","last_change":"2024-04-25T14:10:45.037754+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.037754+0000","last_undegraded":"2024-04-25T14:10:45.037754+0000","last_fullsized":"2024-04-25T14:10:45.037754+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.049696+0000","last_change":"2024-04-25T14:10:45.046573+0000","last_active":"2024-04-25T14:10:45.049696+0000","last_peered":"2024-04-25T14:10:45.049696+0000","last_clean":"2024-04-25T14:10:45.049696+0000","last_became_active":"2024-04-25T14:10:45.046226+0000","last_became_peered":"2024-04-25T14:10:45.046226+0000","last_unstale":"2024-04-25T14:10:45.049696+0000","last_undegraded":"2024-04-25T14:10:45.049696+0000","last_fullsized":"2024-04-25T14:10:45.049696+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:01.990667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039513+0000","last_change":"2024-04-25T14:10:45.039513+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039513+0000","last_undegraded":"2024-04-25T14:10:45.039513+0000","last_fullsized":"2024-04-25T14:10:45.039513+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.056936+0000","last_change":"2024-04-25T14:10:45.055809+0000","last_active":"2024-04-25T14:10:45.056936+0000","last_peered":"2024-04-25T14:10:45.056936+0000","last_clean":"2024-04-25T14:10:45.056936+0000","last_became_active":"2024-04-25T14:10:45.055480+0000","last_became_peered":"2024-04-25T14:10:45.055480+0000","last_unstale":"2024-04-25T14:10:45.056936+0000","last_undegraded":"2024-04-25T14:10:45.056936+0000","last_fullsized":"2024-04-25T14:10:45.056936+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:58.758388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.037351+0000","last_change":"2024-04-25T14:10:45.037351+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.037351+0000","last_undegraded":"2024-04-25T14:10:45.037351+0000","last_fullsized":"2024-04-25T14:10:45.037351+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.061791+0000","last_change":"2024-04-25T14:10:45.058112+0000","last_active":"2024-04-25T14:10:45.061791+0000","last_peered":"2024-04-25T14:10:45.061791+0000","last_clean":"2024-04-25T14:10:45.061791+0000","last_became_active":"2024-04-25T14:10:45.057825+0000","last_became_peered":"2024-04-25T14:10:45.057825+0000","last_unstale":"2024-04-25T14:10:45.061791+0000","last_undegraded":"2024-04-25T14:10:45.061791+0000","last_fullsized":"2024-04-25T14:10:45.061791+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:33.243994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.048890+0000","last_change":"2024-04-25T14:10:45.048890+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.048890+0000","last_undegraded":"2024-04-25T14:10:45.048890+0000","last_fullsized":"2024-04-25T14:10:45.048890+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.051870+0000","last_change":"2024-04-25T14:10:45.049671+0000","last_active":"2024-04-25T14:10:45.051870+0000","last_peered":"2024-04-25T14:10:45.051870+0000","last_clean":"2024-04-25T14:10:45.051870+0000","last_became_active":"2024-04-25T14:10:45.049375+0000","last_became_peered":"2024-04-25T14:10:45.049375+0000","last_unstale":"2024-04-25T14:10:45.051870+0000","last_undegraded":"2024-04-25T14:10:45.051870+0000","last_fullsized":"2024-04-25T14:10:45.051870+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:58.758388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.049911+0000","last_change":"2024-04-25T14:10:45.049911+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.049911+0000","last_undegraded":"2024-04-25T14:10:45.049911+0000","last_fullsized":"2024-04-25T14:10:45.049911+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.056549+0000","last_change":"2024-04-25T14:10:45.053154+0000","last_active":"2024-04-25T14:10:45.056549+0000","last_peered":"2024-04-25T14:10:45.056549+0000","last_clean":"2024-04-25T14:10:45.056549+0000","last_became_active":"2024-04-25T14:10:45.052842+0000","last_became_peered":"2024-04-25T14:10:45.052842+0000","last_unstale":"2024-04-25T14:10:45.056549+0000","last_undegraded":"2024-04-25T14:10:45.056549+0000","last_fullsized":"2024-04-25T14:10:45.056549+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:22.629958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.048918+0000","last_change":"2024-04-25T14:10:45.048918+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.048918+0000","last_undegraded":"2024-04-25T14:10:45.048918+0000","last_fullsized":"2024-04-25T14:10:45.048918+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.061784+0000","last_change":"2024-04-25T14:10:45.058009+0000","last_active":"2024-04-25T14:10:45.061784+0000","last_peered":"2024-04-25T14:10:45.061784+0000","last_clean":"2024-04-25T14:10:45.061784+0000","last_became_active":"2024-04-25T14:10:45.057724+0000","last_became_peered":"2024-04-25T14:10:45.057724+0000","last_unstale":"2024-04-25T14:10:45.061784+0000","last_undegraded":"2024-04-25T14:10:45.061784+0000","last_fullsized":"2024-04-25T14:10:45.061784+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:52.050283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.040213+0000","last_change":"2024-04-25T14:10:45.040213+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.040213+0000","last_undegraded":"2024-04-25T14:10:45.040213+0000","last_fullsized":"2024-04-25T14:10:45.040213+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.052969+0000","last_change":"2024-04-25T14:10:45.043256+0000","last_active":"2024-04-25T14:10:45.052969+0000","last_peered":"2024-04-25T14:10:45.052969+0000","last_clean":"2024-04-25T14:10:45.052969+0000","last_became_active":"2024-04-25T14:10:45.042731+0000","last_became_peered":"2024-04-25T14:10:45.042731+0000","last_unstale":"2024-04-25T14:10:45.052969+0000","last_undegraded":"2024-04-25T14:10:45.052969+0000","last_fullsized":"2024-04-25T14:10:45.052969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:20.290586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.037519+0000","last_change":"2024-04-25T14:10:45.037519+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.037519+0000","last_undegraded":"2024-04-25T14:10:45.037519+0000","last_fullsized":"2024-04-25T14:10:45.037519+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.056594+0000","last_change":"2024-04-25T14:10:45.053553+0000","last_active":"2024-04-25T14:10:45.056594+0000","last_peered":"2024-04-25T14:10:45.056594+0000","last_clean":"2024-04-25T14:10:45.056594+0000","last_became_active":"2024-04-25T14:10:45.053178+0000","last_became_peered":"2024-04-25T14:10:45.053178+0000","last_unstale":"2024-04-25T14:10:45.056594+0000","last_undegraded":"2024-04-25T14:10:45.056594+0000","last_fullsized":"2024-04-25T14:10:45.056594+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:55.116141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"124'2021","reported_seq":2403,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.602454+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:10:45.602454+0000","last_peered":"2024-04-25T14:10:45.602454+0000","last_clean":"2024-04-25T14:10:45.602454+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:10:45.602454+0000","last_undegraded":"2024-04-25T14:10:45.602454+0000","last_fullsized":"2024-04-25T14:10:45.602454+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":2021,"log_dups_size":0,"ondisk_log_size":2021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":529,"num_read_kb":1389,"num_write":187,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":529,"num_read_kb":1389,"num_write":187,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2021,"ondisk_log_size":2021,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574930,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41584,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330256,"statfs":{"total":96636764160,"available":96594182144,"internally_reserved":0,"allocated":2134016,"data_stored":1358180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32684,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1540096,"data_stored":767812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32684,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1540096,"data_stored":767812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574931,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1552384,"data_stored":774471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1552384,"data_stored":774471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574930,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1552384,"data_stored":774471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32684,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1540096,"data_stored":767812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574931,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41592,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330248,"statfs":{"total":96636764160,"available":96594173952,"internally_reserved":0,"allocated":2146304,"data_stored":1364839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:10:50.269 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:10:50.420 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:50.421+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:50.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.465+0000 7f1d33000640 1 Processor -- start 2024-04-25T14:10:50.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.465+0000 7f1d33000640 1 -- start start 2024-04-25T14:10:50.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.465+0000 7f1d33000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1d2c070750 0x7f1d2c0b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:50.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.465+0000 7f1d33000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1d2c159aa0 0x7f1d2c159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:50.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.465+0000 7f1d33000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d2c15a3f0 0x7f1d2c06c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:50.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.465+0000 7f1d33000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1d2c0b4c40 con 0x7f1d2c070750 2024-04-25T14:10:50.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.469+0000 7f1d33000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1d2c15a800 con 0x7f1d2c159aa0 2024-04-25T14:10:50.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.469+0000 7f1d33000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1d2c0b4ee0 con 0x7f1d2c15a3f0 2024-04-25T14:10:50.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.469+0000 7f1d2be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1d2c070750 0x7f1d2c0b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:50.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.469+0000 7f1d2be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1d2c070750 0x7f1d2c0b03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46174/0 (socket says 172.21.15.86:46174) 2024-04-25T14:10:50.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.469+0000 7f1d2be00640 1 -- 172.21.15.86:0/3856376966 learned_addr learned my addr 172.21.15.86:0/3856376966 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:50.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.469+0000 7f1d31000640 1 --2- 172.21.15.86:0/3856376966 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d2c15a3f0 0x7f1d2c06c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:50.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.469+0000 7f1d2b400640 1 --2- 172.21.15.86:0/3856376966 >> v2:172.21.15.155:3300/0 conn(0x7f1d2c159aa0 0x7f1d2c159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:50.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.469+0000 7f1d2be00640 1 -- 172.21.15.86:0/3856376966 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d2c15a3f0 msgr2=0x7f1d2c06c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:50.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.469+0000 7f1d2be00640 1 --2- 172.21.15.86:0/3856376966 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d2c15a3f0 0x7f1d2c06c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.469+0000 7f1d2be00640 1 -- 172.21.15.86:0/3856376966 >> v2:172.21.15.155:3300/0 conn(0x7f1d2c159aa0 msgr2=0x7f1d2c159eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:50.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.469+0000 7f1d2be00640 1 --2- 172.21.15.86:0/3856376966 >> v2:172.21.15.155:3300/0 conn(0x7f1d2c159aa0 0x7f1d2c159eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.469+0000 7f1d2be00640 1 -- 172.21.15.86:0/3856376966 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d2c0b51c0 con 0x7f1d2c070750 2024-04-25T14:10:50.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.469+0000 7f1d2be00640 1 --2- 172.21.15.86:0/3856376966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d2c070750 0x7f1d2c0b03b0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c069310 tx=0x7f1d1c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da02910a54f9bca6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:50.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.469+0000 7f1d2aa00640 1 -- 172.21.15.86:0/3856376966 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c0a56d0 con 0x7f1d2c070750 2024-04-25T14:10:50.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.469+0000 7f1d2aa00640 1 -- 172.21.15.86:0/3856376966 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d1c0a5870 con 0x7f1d2c070750 2024-04-25T14:10:50.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.469+0000 7f1d2aa00640 1 -- 172.21.15.86:0/3856376966 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c0a5c40 con 0x7f1d2c070750 2024-04-25T14:10:50.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.473+0000 7f1d33000640 1 -- 172.21.15.86:0/3856376966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d2c070750 msgr2=0x7f1d2c0b03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:50.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.473+0000 7f1d33000640 1 --2- 172.21.15.86:0/3856376966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d2c070750 0x7f1d2c0b03b0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c069310 tx=0x7f1d1c01c960 comp rx=0 tx=0).stop 2024-04-25T14:10:50.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.473+0000 7f1d33000640 1 -- 172.21.15.86:0/3856376966 shutdown_connections 2024-04-25T14:10:50.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.473+0000 7f1d33000640 1 --2- 172.21.15.86:0/3856376966 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d2c15a3f0 0x7f1d2c06c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.473+0000 7f1d33000640 1 --2- 172.21.15.86:0/3856376966 >> v2:172.21.15.155:3300/0 conn(0x7f1d2c159aa0 0x7f1d2c159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.473+0000 7f1d33000640 1 --2- 172.21.15.86:0/3856376966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d2c070750 0x7f1d2c0b03b0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.473+0000 7f1d33000640 1 -- 172.21.15.86:0/3856376966 >> 172.21.15.86:0/3856376966 conn(0x7f1d2c0d19b0 msgr2=0x7f1d2c0d1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:50.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.473+0000 7f1d33000640 1 -- 172.21.15.86:0/3856376966 shutdown_connections 2024-04-25T14:10:50.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.473+0000 7f1d33000640 1 -- 172.21.15.86:0/3856376966 wait complete. 2024-04-25T14:10:50.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.473+0000 7f1d33000640 1 Processor -- start 2024-04-25T14:10:50.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.473+0000 7f1d33000640 1 -- start start 2024-04-25T14:10:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.477+0000 7f1d33000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d2c070750 0x7f1d2c0b2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.477+0000 7f1d33000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d2c159aa0 0x7f1d2c1481d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.477+0000 7f1d33000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d2c15a3f0 0x7f1d2c152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.477+0000 7f1d33000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c070ce0 con 0x7f1d2c070750 2024-04-25T14:10:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.477+0000 7f1d33000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c070b60 con 0x7f1d2c159aa0 2024-04-25T14:10:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.477+0000 7f1d33000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1d2c163960 con 0x7f1d2c15a3f0 2024-04-25T14:10:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.477+0000 7f1d2be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d2c070750 0x7f1d2c0b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.477+0000 7f1d2be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d2c070750 0x7f1d2c0b2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46184/0 (socket says 172.21.15.86:46184) 2024-04-25T14:10:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.477+0000 7f1d2be00640 1 -- 172.21.15.86:0/3410797077 learned_addr learned my addr 172.21.15.86:0/3410797077 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.477+0000 7f1d2b400640 1 --2- 172.21.15.86:0/3410797077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d2c159aa0 0x7f1d2c1481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.477+0000 7f1d31000640 1 --2- 172.21.15.86:0/3410797077 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d2c15a3f0 0x7f1d2c152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.477+0000 7f1d2be00640 1 -- 172.21.15.86:0/3410797077 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d2c15a3f0 msgr2=0x7f1d2c152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.477+0000 7f1d2be00640 1 --2- 172.21.15.86:0/3410797077 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d2c15a3f0 0x7f1d2c152d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.477+0000 7f1d2be00640 1 -- 172.21.15.86:0/3410797077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d2c159aa0 msgr2=0x7f1d2c1481d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.477+0000 7f1d2be00640 1 --2- 172.21.15.86:0/3410797077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d2c159aa0 0x7f1d2c1481d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.477+0000 7f1d2be00640 1 -- 172.21.15.86:0/3410797077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d2c14a200 con 0x7f1d2c070750 2024-04-25T14:10:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.477+0000 7f1d2be00640 1 --2- 172.21.15.86:0/3410797077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d2c070750 0x7f1d2c0b2460 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c0af040 tx=0x7f1d1c019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.477+0000 7f1d23e00640 1 -- 172.21.15.86:0/3410797077 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c0bc020 con 0x7f1d2c070750 2024-04-25T14:10:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.477+0000 7f1d23e00640 1 -- 172.21.15.86:0/3410797077 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d1c0b8670 con 0x7f1d2c070750 2024-04-25T14:10:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.477+0000 7f1d33000640 1 -- 172.21.15.86:0/3410797077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d2c162d70 con 0x7f1d2c070750 2024-04-25T14:10:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.477+0000 7f1d23e00640 1 -- 172.21.15.86:0/3410797077 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c0b8a60 con 0x7f1d2c070750 2024-04-25T14:10:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.477+0000 7f1d33000640 1 -- 172.21.15.86:0/3410797077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d2c1536a0 con 0x7f1d2c070750 2024-04-25T14:10:50.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.477+0000 7f1d33000640 1 -- 172.21.15.86:0/3410797077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d2c0748a0 con 0x7f1d2c070750 2024-04-25T14:10:50.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.481+0000 7f1d23e00640 1 -- 172.21.15.86:0/3410797077 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7f1d1c0bf020 con 0x7f1d2c070750 2024-04-25T14:10:50.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.481+0000 7f1d23e00640 1 --2- 172.21.15.86:0/3410797077 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f1d0c06f490 0x7f1d0c0719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:50.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.481+0000 7f1d23e00640 1 -- 172.21.15.86:0/3410797077 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f1d1c12a2c0 con 0x7f1d2c070750 2024-04-25T14:10:50.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.481+0000 7f1d2b400640 1 --2- 172.21.15.86:0/3410797077 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f1d0c06f490 0x7f1d0c0719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:50.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.481+0000 7f1d2b400640 1 --2- 172.21.15.86:0/3410797077 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f1d0c06f490 0x7f1d0c0719a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c154c70 tx=0x7f1d18008040 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:50.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.485+0000 7f1d23e00640 1 -- 172.21.15.86:0/3410797077 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1d1c0c6030 con 0x7f1d2c070750 2024-04-25T14:10:50.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.649+0000 7f1d33000640 1 -- 172.21.15.86:0/3410797077 --> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1d2c153940 con 0x7f1d0c06f490 2024-04-25T14:10:50.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.657+0000 7f1d23e00640 1 -- 172.21.15.86:0/3410797077 <== mgr.8338 v2:172.21.15.86:6832/3964666460 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247083 (secure 0 0 0) 0x7f1d2c153940 con 0x7f1d0c06f490 2024-04-25T14:10:50.656 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:50.657 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:10:50.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.661+0000 7f1d33000640 1 -- 172.21.15.86:0/3410797077 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f1d0c06f490 msgr2=0x7f1d0c0719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:50.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.661+0000 7f1d33000640 1 --2- 172.21.15.86:0/3410797077 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f1d0c06f490 0x7f1d0c0719a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c154c70 tx=0x7f1d18008040 comp rx=0 tx=0).stop 2024-04-25T14:10:50.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.661+0000 7f1d33000640 1 -- 172.21.15.86:0/3410797077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d2c070750 msgr2=0x7f1d2c0b2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:50.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.661+0000 7f1d33000640 1 --2- 172.21.15.86:0/3410797077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d2c070750 0x7f1d2c0b2460 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c0af040 tx=0x7f1d1c019700 comp rx=0 tx=0).stop 2024-04-25T14:10:50.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.661+0000 7f1d33000640 1 -- 172.21.15.86:0/3410797077 shutdown_connections 2024-04-25T14:10:50.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.661+0000 7f1d33000640 1 --2- 172.21.15.86:0/3410797077 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d2c15a3f0 0x7f1d2c152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.661+0000 7f1d33000640 1 --2- 172.21.15.86:0/3410797077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d2c159aa0 0x7f1d2c1481d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.661+0000 7f1d33000640 1 --2- 172.21.15.86:0/3410797077 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f1d0c06f490 0x7f1d0c0719a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.661+0000 7f1d33000640 1 --2- 172.21.15.86:0/3410797077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d2c070750 0x7f1d2c0b2460 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.661+0000 7f1d33000640 1 -- 172.21.15.86:0/3410797077 >> 172.21.15.86:0/3410797077 conn(0x7f1d2c0d19b0 msgr2=0x7f1d2c0aee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:50.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.661+0000 7f1d33000640 1 -- 172.21.15.86:0/3410797077 shutdown_connections 2024-04-25T14:10:50.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.661+0000 7f1d33000640 1 -- 172.21.15.86:0/3410797077 wait complete. 2024-04-25T14:10:50.678 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T14:10:49.615624+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":529,"num_read_kb":1389,"num_write":187,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2021,"ondisk_log_size":2021,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279328,"kb_used_data":13240,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754695392,"statfs":{"total":773094113280,"available":772808081408,"internally_reserved":0,"allocated":13557760,"data_stored":7349868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.588242"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.043709+0000","last_change":"2024-04-25T14:10:45.043709+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.043709+0000","last_undegraded":"2024-04-25T14:10:45.043709+0000","last_fullsized":"2024-04-25T14:10:45.043709+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.034723+0000","last_change":"2024-04-25T14:10:45.034723+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.034723+0000","last_undegraded":"2024-04-25T14:10:45.034723+0000","last_fullsized":"2024-04-25T14:10:45.034723+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038065+0000","last_change":"2024-04-25T14:10:45.038065+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038065+0000","last_undegraded":"2024-04-25T14:10:45.038065+0000","last_fullsized":"2024-04-25T14:10:45.038065+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.032697+0000","last_change":"2024-04-25T14:10:45.032697+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.032697+0000","last_undegraded":"2024-04-25T14:10:45.032697+0000","last_fullsized":"2024-04-25T14:10:45.032697+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038541+0000","last_change":"2024-04-25T14:10:45.038541+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038541+0000","last_undegraded":"2024-04-25T14:10:45.038541+0000","last_fullsized":"2024-04-25T14:10:45.038541+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.045986+0000","last_change":"2024-04-25T14:10:45.045986+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.045986+0000","last_undegraded":"2024-04-25T14:10:45.045986+0000","last_fullsized":"2024-04-25T14:10:45.045986+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.037964+0000","last_change":"2024-04-25T14:10:45.037964+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.037964+0000","last_undegraded":"2024-04-25T14:10:45.037964+0000","last_fullsized":"2024-04-25T14:10:45.037964+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.043120+0000","last_change":"2024-04-25T14:10:45.043120+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.043120+0000","last_undegraded":"2024-04-25T14:10:45.043120+0000","last_fullsized":"2024-04-25T14:10:45.043120+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.049467+0000","last_change":"2024-04-25T14:10:45.049467+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.049467+0000","last_undegraded":"2024-04-25T14:10:45.049467+0000","last_fullsized":"2024-04-25T14:10:45.049467+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039375+0000","last_change":"2024-04-25T14:10:45.039375+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039375+0000","last_undegraded":"2024-04-25T14:10:45.039375+0000","last_fullsized":"2024-04-25T14:10:45.039375+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.034076+0000","last_change":"2024-04-25T14:10:45.034076+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.034076+0000","last_undegraded":"2024-04-25T14:10:45.034076+0000","last_fullsized":"2024-04-25T14:10:45.034076+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.041407+0000","last_change":"2024-04-25T14:10:45.041407+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.041407+0000","last_undegraded":"2024-04-25T14:10:45.041407+0000","last_fullsized":"2024-04-25T14:10:45.041407+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.047463+0000","last_change":"2024-04-25T14:10:45.047463+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.047463+0000","last_undegraded":"2024-04-25T14:10:45.047463+0000","last_fullsized":"2024-04-25T14:10:45.047463+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.040256+0000","last_change":"2024-04-25T14:10:45.040256+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.040256+0000","last_undegraded":"2024-04-25T14:10:45.040256+0000","last_fullsized":"2024-04-25T14:10:45.040256+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.040028+0000","last_change":"2024-04-25T14:10:45.040028+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.040028+0000","last_undegraded":"2024-04-25T14:10:45.040028+0000","last_fullsized":"2024-04-25T14:10:45.040028+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.045366+0000","last_change":"2024-04-25T14:10:45.045366+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.045366+0000","last_undegraded":"2024-04-25T14:10:45.045366+0000","last_fullsized":"2024-04-25T14:10:45.045366+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.048856+0000","last_change":"2024-04-25T14:10:45.048856+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.048856+0000","last_undegraded":"2024-04-25T14:10:45.048856+0000","last_fullsized":"2024-04-25T14:10:45.048856+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.040344+0000","last_change":"2024-04-25T14:10:45.040344+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.040344+0000","last_undegraded":"2024-04-25T14:10:45.040344+0000","last_fullsized":"2024-04-25T14:10:45.040344+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039190+0000","last_change":"2024-04-25T14:10:45.039190+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039190+0000","last_undegraded":"2024-04-25T14:10:45.039190+0000","last_fullsized":"2024-04-25T14:10:45.039190+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038526+0000","last_change":"2024-04-25T14:10:45.038526+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038526+0000","last_undegraded":"2024-04-25T14:10:45.038526+0000","last_fullsized":"2024-04-25T14:10:45.038526+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.034790+0000","last_change":"2024-04-25T14:10:45.034790+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.034790+0000","last_undegraded":"2024-04-25T14:10:45.034790+0000","last_fullsized":"2024-04-25T14:10:45.034790+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.040708+0000","last_change":"2024-04-25T14:10:45.040708+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.040708+0000","last_undegraded":"2024-04-25T14:10:45.040708+0000","last_fullsized":"2024-04-25T14:10:45.040708+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.032697+0000","last_change":"2024-04-25T14:10:45.032697+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.032697+0000","last_undegraded":"2024-04-25T14:10:45.032697+0000","last_fullsized":"2024-04-25T14:10:45.032697+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.044889+0000","last_change":"2024-04-25T14:10:45.044889+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.044889+0000","last_undegraded":"2024-04-25T14:10:45.044889+0000","last_fullsized":"2024-04-25T14:10:45.044889+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039272+0000","last_change":"2024-04-25T14:10:45.039272+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039272+0000","last_undegraded":"2024-04-25T14:10:45.039272+0000","last_fullsized":"2024-04-25T14:10:45.039272+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038197+0000","last_change":"2024-04-25T14:10:45.038197+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038197+0000","last_undegraded":"2024-04-25T14:10:45.038197+0000","last_fullsized":"2024-04-25T14:10:45.038197+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038708+0000","last_change":"2024-04-25T14:10:45.038708+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038708+0000","last_undegraded":"2024-04-25T14:10:45.038708+0000","last_fullsized":"2024-04-25T14:10:45.038708+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039464+0000","last_change":"2024-04-25T14:10:45.039464+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039464+0000","last_undegraded":"2024-04-25T14:10:45.039464+0000","last_fullsized":"2024-04-25T14:10:45.039464+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.040768+0000","last_change":"2024-04-25T14:10:45.040768+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.040768+0000","last_undegraded":"2024-04-25T14:10:45.040768+0000","last_fullsized":"2024-04-25T14:10:45.040768+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038367+0000","last_change":"2024-04-25T14:10:45.038367+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038367+0000","last_undegraded":"2024-04-25T14:10:45.038367+0000","last_fullsized":"2024-04-25T14:10:45.038367+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.044360+0000","last_change":"2024-04-25T14:10:45.044360+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.044360+0000","last_undegraded":"2024-04-25T14:10:45.044360+0000","last_fullsized":"2024-04-25T14:10:45.044360+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039596+0000","last_change":"2024-04-25T14:10:45.039596+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039596+0000","last_undegraded":"2024-04-25T14:10:45.039596+0000","last_fullsized":"2024-04-25T14:10:45.039596+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.035461+0000","last_change":"2024-04-25T14:10:45.035461+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.035461+0000","last_undegraded":"2024-04-25T14:10:45.035461+0000","last_fullsized":"2024-04-25T14:10:45.035461+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.052380+0000","last_change":"2024-04-25T14:10:45.051471+0000","last_active":"2024-04-25T14:10:45.052380+0000","last_peered":"2024-04-25T14:10:45.052380+0000","last_clean":"2024-04-25T14:10:45.052380+0000","last_became_active":"2024-04-25T14:10:45.051055+0000","last_became_peered":"2024-04-25T14:10:45.051055+0000","last_unstale":"2024-04-25T14:10:45.052380+0000","last_undegraded":"2024-04-25T14:10:45.052380+0000","last_fullsized":"2024-04-25T14:10:45.052380+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:35.948249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039532+0000","last_change":"2024-04-25T14:10:45.039532+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039532+0000","last_undegraded":"2024-04-25T14:10:45.039532+0000","last_fullsized":"2024-04-25T14:10:45.039532+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.052927+0000","last_change":"2024-04-25T14:10:45.046859+0000","last_active":"2024-04-25T14:10:45.052927+0000","last_peered":"2024-04-25T14:10:45.052927+0000","last_clean":"2024-04-25T14:10:45.052927+0000","last_became_active":"2024-04-25T14:10:45.044618+0000","last_became_peered":"2024-04-25T14:10:45.044618+0000","last_unstale":"2024-04-25T14:10:45.052927+0000","last_undegraded":"2024-04-25T14:10:45.052927+0000","last_fullsized":"2024-04-25T14:10:45.052927+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:55.442843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.033422+0000","last_change":"2024-04-25T14:10:45.033422+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.033422+0000","last_undegraded":"2024-04-25T14:10:45.033422+0000","last_fullsized":"2024-04-25T14:10:45.033422+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.057542+0000","last_change":"2024-04-25T14:10:45.056867+0000","last_active":"2024-04-25T14:10:45.057542+0000","last_peered":"2024-04-25T14:10:45.057542+0000","last_clean":"2024-04-25T14:10:45.057542+0000","last_became_active":"2024-04-25T14:10:45.056470+0000","last_became_peered":"2024-04-25T14:10:45.056470+0000","last_unstale":"2024-04-25T14:10:45.057542+0000","last_undegraded":"2024-04-25T14:10:45.057542+0000","last_fullsized":"2024-04-25T14:10:45.057542+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:01.583932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.043935+0000","last_change":"2024-04-25T14:10:45.043935+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.043935+0000","last_undegraded":"2024-04-25T14:10:45.043935+0000","last_fullsized":"2024-04-25T14:10:45.043935+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.054827+0000","last_change":"2024-04-25T14:10:45.052367+0000","last_active":"2024-04-25T14:10:45.054827+0000","last_peered":"2024-04-25T14:10:45.054827+0000","last_clean":"2024-04-25T14:10:45.054827+0000","last_became_active":"2024-04-25T14:10:45.052055+0000","last_became_peered":"2024-04-25T14:10:45.052055+0000","last_unstale":"2024-04-25T14:10:45.054827+0000","last_undegraded":"2024-04-25T14:10:45.054827+0000","last_fullsized":"2024-04-25T14:10:45.054827+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:52.050283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.041991+0000","last_change":"2024-04-25T14:10:45.041991+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.041991+0000","last_undegraded":"2024-04-25T14:10:45.041991+0000","last_fullsized":"2024-04-25T14:10:45.041991+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.044829+0000","last_change":"2024-04-25T14:10:45.041897+0000","last_active":"2024-04-25T14:10:45.044829+0000","last_peered":"2024-04-25T14:10:45.044829+0000","last_clean":"2024-04-25T14:10:45.044829+0000","last_became_active":"2024-04-25T14:10:45.039679+0000","last_became_peered":"2024-04-25T14:10:45.039679+0000","last_unstale":"2024-04-25T14:10:45.044829+0000","last_undegraded":"2024-04-25T14:10:45.044829+0000","last_fullsized":"2024-04-25T14:10:45.044829+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:22.669468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.040465+0000","last_change":"2024-04-25T14:10:45.040465+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.040465+0000","last_undegraded":"2024-04-25T14:10:45.040465+0000","last_fullsized":"2024-04-25T14:10:45.040465+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.053034+0000","last_change":"2024-04-25T14:10:45.051135+0000","last_active":"2024-04-25T14:10:45.053034+0000","last_peered":"2024-04-25T14:10:45.053034+0000","last_clean":"2024-04-25T14:10:45.053034+0000","last_became_active":"2024-04-25T14:10:45.050463+0000","last_became_peered":"2024-04-25T14:10:45.050463+0000","last_unstale":"2024-04-25T14:10:45.053034+0000","last_undegraded":"2024-04-25T14:10:45.053034+0000","last_fullsized":"2024-04-25T14:10:45.053034+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:12.563128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039895+0000","last_change":"2024-04-25T14:10:45.039895+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039895+0000","last_undegraded":"2024-04-25T14:10:45.039895+0000","last_fullsized":"2024-04-25T14:10:45.039895+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.056835+0000","last_change":"2024-04-25T14:10:45.053969+0000","last_active":"2024-04-25T14:10:45.056835+0000","last_peered":"2024-04-25T14:10:45.056835+0000","last_clean":"2024-04-25T14:10:45.056835+0000","last_became_active":"2024-04-25T14:10:45.053634+0000","last_became_peered":"2024-04-25T14:10:45.053634+0000","last_unstale":"2024-04-25T14:10:45.056835+0000","last_undegraded":"2024-04-25T14:10:45.056835+0000","last_fullsized":"2024-04-25T14:10:45.056835+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:54.008821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.040667+0000","last_change":"2024-04-25T14:10:45.040667+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.040667+0000","last_undegraded":"2024-04-25T14:10:45.040667+0000","last_fullsized":"2024-04-25T14:10:45.040667+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.056301+0000","last_change":"2024-04-25T14:10:45.054900+0000","last_active":"2024-04-25T14:10:45.056301+0000","last_peered":"2024-04-25T14:10:45.056301+0000","last_clean":"2024-04-25T14:10:45.056301+0000","last_became_active":"2024-04-25T14:10:45.054599+0000","last_became_peered":"2024-04-25T14:10:45.054599+0000","last_unstale":"2024-04-25T14:10:45.056301+0000","last_undegraded":"2024-04-25T14:10:45.056301+0000","last_fullsized":"2024-04-25T14:10:45.056301+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:05.356606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039869+0000","last_change":"2024-04-25T14:10:45.039869+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039869+0000","last_undegraded":"2024-04-25T14:10:45.039869+0000","last_fullsized":"2024-04-25T14:10:45.039869+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.045456+0000","last_change":"2024-04-25T14:10:45.037687+0000","last_active":"2024-04-25T14:10:45.045456+0000","last_peered":"2024-04-25T14:10:45.045456+0000","last_clean":"2024-04-25T14:10:45.045456+0000","last_became_active":"2024-04-25T14:10:45.037370+0000","last_became_peered":"2024-04-25T14:10:45.037370+0000","last_unstale":"2024-04-25T14:10:45.045456+0000","last_undegraded":"2024-04-25T14:10:45.045456+0000","last_fullsized":"2024-04-25T14:10:45.045456+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:54.008821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.046751+0000","last_change":"2024-04-25T14:10:45.046751+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.046751+0000","last_undegraded":"2024-04-25T14:10:45.046751+0000","last_fullsized":"2024-04-25T14:10:45.046751+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.058741+0000","last_change":"2024-04-25T14:10:45.050926+0000","last_active":"2024-04-25T14:10:45.058741+0000","last_peered":"2024-04-25T14:10:45.058741+0000","last_clean":"2024-04-25T14:10:45.058741+0000","last_became_active":"2024-04-25T14:10:45.049733+0000","last_became_peered":"2024-04-25T14:10:45.049733+0000","last_unstale":"2024-04-25T14:10:45.058741+0000","last_undegraded":"2024-04-25T14:10:45.058741+0000","last_fullsized":"2024-04-25T14:10:45.058741+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:05.958831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.037539+0000","last_change":"2024-04-25T14:10:45.037539+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.037539+0000","last_undegraded":"2024-04-25T14:10:45.037539+0000","last_fullsized":"2024-04-25T14:10:45.037539+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.054475+0000","last_change":"2024-04-25T14:10:45.052411+0000","last_active":"2024-04-25T14:10:45.054475+0000","last_peered":"2024-04-25T14:10:45.054475+0000","last_clean":"2024-04-25T14:10:45.054475+0000","last_became_active":"2024-04-25T14:10:45.052020+0000","last_became_peered":"2024-04-25T14:10:45.052020+0000","last_unstale":"2024-04-25T14:10:45.054475+0000","last_undegraded":"2024-04-25T14:10:45.054475+0000","last_fullsized":"2024-04-25T14:10:45.054475+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:33.243994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.040238+0000","last_change":"2024-04-25T14:10:45.040238+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.040238+0000","last_undegraded":"2024-04-25T14:10:45.040238+0000","last_fullsized":"2024-04-25T14:10:45.040238+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.057096+0000","last_change":"2024-04-25T14:10:45.054962+0000","last_active":"2024-04-25T14:10:45.057096+0000","last_peered":"2024-04-25T14:10:45.057096+0000","last_clean":"2024-04-25T14:10:45.057096+0000","last_became_active":"2024-04-25T14:10:45.054639+0000","last_became_peered":"2024-04-25T14:10:45.054639+0000","last_unstale":"2024-04-25T14:10:45.057096+0000","last_undegraded":"2024-04-25T14:10:45.057096+0000","last_fullsized":"2024-04-25T14:10:45.057096+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:38.714815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038826+0000","last_change":"2024-04-25T14:10:45.038826+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038826+0000","last_undegraded":"2024-04-25T14:10:45.038826+0000","last_fullsized":"2024-04-25T14:10:45.038826+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.058736+0000","last_change":"2024-04-25T14:10:45.051737+0000","last_active":"2024-04-25T14:10:45.058736+0000","last_peered":"2024-04-25T14:10:45.058736+0000","last_clean":"2024-04-25T14:10:45.058736+0000","last_became_active":"2024-04-25T14:10:45.051337+0000","last_became_peered":"2024-04-25T14:10:45.051337+0000","last_unstale":"2024-04-25T14:10:45.058736+0000","last_undegraded":"2024-04-25T14:10:45.058736+0000","last_fullsized":"2024-04-25T14:10:45.058736+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:25.432658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039786+0000","last_change":"2024-04-25T14:10:45.039786+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039786+0000","last_undegraded":"2024-04-25T14:10:45.039786+0000","last_fullsized":"2024-04-25T14:10:45.039786+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.052338+0000","last_change":"2024-04-25T14:10:45.049002+0000","last_active":"2024-04-25T14:10:45.052338+0000","last_peered":"2024-04-25T14:10:45.052338+0000","last_clean":"2024-04-25T14:10:45.052338+0000","last_became_active":"2024-04-25T14:10:45.048755+0000","last_became_peered":"2024-04-25T14:10:45.048755+0000","last_unstale":"2024-04-25T14:10:45.052338+0000","last_undegraded":"2024-04-25T14:10:45.052338+0000","last_fullsized":"2024-04-25T14:10:45.052338+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:04.824306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.037596+0000","last_change":"2024-04-25T14:10:45.037596+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.037596+0000","last_undegraded":"2024-04-25T14:10:45.037596+0000","last_fullsized":"2024-04-25T14:10:45.037596+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.055500+0000","last_change":"2024-04-25T14:10:45.051770+0000","last_active":"2024-04-25T14:10:45.055500+0000","last_peered":"2024-04-25T14:10:45.055500+0000","last_clean":"2024-04-25T14:10:45.055500+0000","last_became_active":"2024-04-25T14:10:45.051446+0000","last_became_peered":"2024-04-25T14:10:45.051446+0000","last_unstale":"2024-04-25T14:10:45.055500+0000","last_undegraded":"2024-04-25T14:10:45.055500+0000","last_fullsized":"2024-04-25T14:10:45.055500+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:01.990667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038531+0000","last_change":"2024-04-25T14:10:45.038531+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038531+0000","last_undegraded":"2024-04-25T14:10:45.038531+0000","last_fullsized":"2024-04-25T14:10:45.038531+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.042874+0000","last_change":"2024-04-25T14:10:45.037175+0000","last_active":"2024-04-25T14:10:45.042874+0000","last_peered":"2024-04-25T14:10:45.042874+0000","last_clean":"2024-04-25T14:10:45.042874+0000","last_became_active":"2024-04-25T14:10:45.036881+0000","last_became_peered":"2024-04-25T14:10:45.036881+0000","last_unstale":"2024-04-25T14:10:45.042874+0000","last_undegraded":"2024-04-25T14:10:45.042874+0000","last_fullsized":"2024-04-25T14:10:45.042874+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:01.990667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.037392+0000","last_change":"2024-04-25T14:10:45.037392+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.037392+0000","last_undegraded":"2024-04-25T14:10:45.037392+0000","last_fullsized":"2024-04-25T14:10:45.037392+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.053697+0000","last_change":"2024-04-25T14:10:45.052421+0000","last_active":"2024-04-25T14:10:45.053697+0000","last_peered":"2024-04-25T14:10:45.053697+0000","last_clean":"2024-04-25T14:10:45.053697+0000","last_became_active":"2024-04-25T14:10:45.052078+0000","last_became_peered":"2024-04-25T14:10:45.052078+0000","last_unstale":"2024-04-25T14:10:45.053697+0000","last_undegraded":"2024-04-25T14:10:45.053697+0000","last_fullsized":"2024-04-25T14:10:45.053697+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:49.038115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039058+0000","last_change":"2024-04-25T14:10:45.039058+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039058+0000","last_undegraded":"2024-04-25T14:10:45.039058+0000","last_fullsized":"2024-04-25T14:10:45.039058+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.058211+0000","last_change":"2024-04-25T14:10:45.056493+0000","last_active":"2024-04-25T14:10:45.058211+0000","last_peered":"2024-04-25T14:10:45.058211+0000","last_clean":"2024-04-25T14:10:45.058211+0000","last_became_active":"2024-04-25T14:10:45.055271+0000","last_became_peered":"2024-04-25T14:10:45.055271+0000","last_unstale":"2024-04-25T14:10:45.058211+0000","last_undegraded":"2024-04-25T14:10:45.058211+0000","last_fullsized":"2024-04-25T14:10:45.058211+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:58.758388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.037958+0000","last_change":"2024-04-25T14:10:45.037958+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.037958+0000","last_undegraded":"2024-04-25T14:10:45.037958+0000","last_fullsized":"2024-04-25T14:10:45.037958+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.058425+0000","last_change":"2024-04-25T14:10:45.055677+0000","last_active":"2024-04-25T14:10:45.058425+0000","last_peered":"2024-04-25T14:10:45.058425+0000","last_clean":"2024-04-25T14:10:45.058425+0000","last_became_active":"2024-04-25T14:10:45.054262+0000","last_became_peered":"2024-04-25T14:10:45.054262+0000","last_unstale":"2024-04-25T14:10:45.058425+0000","last_undegraded":"2024-04-25T14:10:45.058425+0000","last_fullsized":"2024-04-25T14:10:45.058425+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:34.895882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038038+0000","last_change":"2024-04-25T14:10:45.038038+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038038+0000","last_undegraded":"2024-04-25T14:10:45.038038+0000","last_fullsized":"2024-04-25T14:10:45.038038+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.049582+0000","last_change":"2024-04-25T14:10:45.047266+0000","last_active":"2024-04-25T14:10:45.049582+0000","last_peered":"2024-04-25T14:10:45.049582+0000","last_clean":"2024-04-25T14:10:45.049582+0000","last_became_active":"2024-04-25T14:10:45.046928+0000","last_became_peered":"2024-04-25T14:10:45.046928+0000","last_unstale":"2024-04-25T14:10:45.049582+0000","last_undegraded":"2024-04-25T14:10:45.049582+0000","last_fullsized":"2024-04-25T14:10:45.049582+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:54.008821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.045070+0000","last_change":"2024-04-25T14:10:45.045070+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.045070+0000","last_undegraded":"2024-04-25T14:10:45.045070+0000","last_fullsized":"2024-04-25T14:10:45.045070+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.060466+0000","last_change":"2024-04-25T14:10:45.057160+0000","last_active":"2024-04-25T14:10:45.060466+0000","last_peered":"2024-04-25T14:10:45.060466+0000","last_clean":"2024-04-25T14:10:45.060466+0000","last_became_active":"2024-04-25T14:10:45.056871+0000","last_became_peered":"2024-04-25T14:10:45.056871+0000","last_unstale":"2024-04-25T14:10:45.060466+0000","last_undegraded":"2024-04-25T14:10:45.060466+0000","last_fullsized":"2024-04-25T14:10:45.060466+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:44.552534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038103+0000","last_change":"2024-04-25T14:10:45.038103+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038103+0000","last_undegraded":"2024-04-25T14:10:45.038103+0000","last_fullsized":"2024-04-25T14:10:45.038103+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.060444+0000","last_change":"2024-04-25T14:10:45.059841+0000","last_active":"2024-04-25T14:10:45.060444+0000","last_peered":"2024-04-25T14:10:45.060444+0000","last_clean":"2024-04-25T14:10:45.060444+0000","last_became_active":"2024-04-25T14:10:45.059609+0000","last_became_peered":"2024-04-25T14:10:45.059609+0000","last_unstale":"2024-04-25T14:10:45.060444+0000","last_undegraded":"2024-04-25T14:10:45.060444+0000","last_fullsized":"2024-04-25T14:10:45.060444+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:14.821225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.033610+0000","last_change":"2024-04-25T14:10:45.033610+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.033610+0000","last_undegraded":"2024-04-25T14:10:45.033610+0000","last_fullsized":"2024-04-25T14:10:45.033610+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.061732+0000","last_change":"2024-04-25T14:10:45.057442+0000","last_active":"2024-04-25T14:10:45.061732+0000","last_peered":"2024-04-25T14:10:45.061732+0000","last_clean":"2024-04-25T14:10:45.061732+0000","last_became_active":"2024-04-25T14:10:45.057077+0000","last_became_peered":"2024-04-25T14:10:45.057077+0000","last_unstale":"2024-04-25T14:10:45.061732+0000","last_undegraded":"2024-04-25T14:10:45.061732+0000","last_fullsized":"2024-04-25T14:10:45.061732+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:03.729472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.037513+0000","last_change":"2024-04-25T14:10:45.037513+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.037513+0000","last_undegraded":"2024-04-25T14:10:45.037513+0000","last_fullsized":"2024-04-25T14:10:45.037513+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.060351+0000","last_change":"2024-04-25T14:10:45.056735+0000","last_active":"2024-04-25T14:10:45.060351+0000","last_peered":"2024-04-25T14:10:45.060351+0000","last_clean":"2024-04-25T14:10:45.060351+0000","last_became_active":"2024-04-25T14:10:45.052530+0000","last_became_peered":"2024-04-25T14:10:45.052530+0000","last_unstale":"2024-04-25T14:10:45.060351+0000","last_undegraded":"2024-04-25T14:10:45.060351+0000","last_fullsized":"2024-04-25T14:10:45.060351+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:34.861278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.037754+0000","last_change":"2024-04-25T14:10:45.037754+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.037754+0000","last_undegraded":"2024-04-25T14:10:45.037754+0000","last_fullsized":"2024-04-25T14:10:45.037754+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.049696+0000","last_change":"2024-04-25T14:10:45.046573+0000","last_active":"2024-04-25T14:10:45.049696+0000","last_peered":"2024-04-25T14:10:45.049696+0000","last_clean":"2024-04-25T14:10:45.049696+0000","last_became_active":"2024-04-25T14:10:45.046226+0000","last_became_peered":"2024-04-25T14:10:45.046226+0000","last_unstale":"2024-04-25T14:10:45.049696+0000","last_undegraded":"2024-04-25T14:10:45.049696+0000","last_fullsized":"2024-04-25T14:10:45.049696+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:01.990667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039513+0000","last_change":"2024-04-25T14:10:45.039513+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039513+0000","last_undegraded":"2024-04-25T14:10:45.039513+0000","last_fullsized":"2024-04-25T14:10:45.039513+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.056936+0000","last_change":"2024-04-25T14:10:45.055809+0000","last_active":"2024-04-25T14:10:45.056936+0000","last_peered":"2024-04-25T14:10:45.056936+0000","last_clean":"2024-04-25T14:10:45.056936+0000","last_became_active":"2024-04-25T14:10:45.055480+0000","last_became_peered":"2024-04-25T14:10:45.055480+0000","last_unstale":"2024-04-25T14:10:45.056936+0000","last_undegraded":"2024-04-25T14:10:45.056936+0000","last_fullsized":"2024-04-25T14:10:45.056936+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:58.758388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.037351+0000","last_change":"2024-04-25T14:10:45.037351+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.037351+0000","last_undegraded":"2024-04-25T14:10:45.037351+0000","last_fullsized":"2024-04-25T14:10:45.037351+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.061791+0000","last_change":"2024-04-25T14:10:45.058112+0000","last_active":"2024-04-25T14:10:45.061791+0000","last_peered":"2024-04-25T14:10:45.061791+0000","last_clean":"2024-04-25T14:10:45.061791+0000","last_became_active":"2024-04-25T14:10:45.057825+0000","last_became_peered":"2024-04-25T14:10:45.057825+0000","last_unstale":"2024-04-25T14:10:45.061791+0000","last_undegraded":"2024-04-25T14:10:45.061791+0000","last_fullsized":"2024-04-25T14:10:45.061791+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:33.243994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.048890+0000","last_change":"2024-04-25T14:10:45.048890+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.048890+0000","last_undegraded":"2024-04-25T14:10:45.048890+0000","last_fullsized":"2024-04-25T14:10:45.048890+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.051870+0000","last_change":"2024-04-25T14:10:45.049671+0000","last_active":"2024-04-25T14:10:45.051870+0000","last_peered":"2024-04-25T14:10:45.051870+0000","last_clean":"2024-04-25T14:10:45.051870+0000","last_became_active":"2024-04-25T14:10:45.049375+0000","last_became_peered":"2024-04-25T14:10:45.049375+0000","last_unstale":"2024-04-25T14:10:45.051870+0000","last_undegraded":"2024-04-25T14:10:45.051870+0000","last_fullsized":"2024-04-25T14:10:45.051870+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:58.758388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.049911+0000","last_change":"2024-04-25T14:10:45.049911+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.049911+0000","last_undegraded":"2024-04-25T14:10:45.049911+0000","last_fullsized":"2024-04-25T14:10:45.049911+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.056549+0000","last_change":"2024-04-25T14:10:45.053154+0000","last_active":"2024-04-25T14:10:45.056549+0000","last_peered":"2024-04-25T14:10:45.056549+0000","last_clean":"2024-04-25T14:10:45.056549+0000","last_became_active":"2024-04-25T14:10:45.052842+0000","last_became_peered":"2024-04-25T14:10:45.052842+0000","last_unstale":"2024-04-25T14:10:45.056549+0000","last_undegraded":"2024-04-25T14:10:45.056549+0000","last_fullsized":"2024-04-25T14:10:45.056549+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:22.629958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.048918+0000","last_change":"2024-04-25T14:10:45.048918+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.048918+0000","last_undegraded":"2024-04-25T14:10:45.048918+0000","last_fullsized":"2024-04-25T14:10:45.048918+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.061784+0000","last_change":"2024-04-25T14:10:45.058009+0000","last_active":"2024-04-25T14:10:45.061784+0000","last_peered":"2024-04-25T14:10:45.061784+0000","last_clean":"2024-04-25T14:10:45.061784+0000","last_became_active":"2024-04-25T14:10:45.057724+0000","last_became_peered":"2024-04-25T14:10:45.057724+0000","last_unstale":"2024-04-25T14:10:45.061784+0000","last_undegraded":"2024-04-25T14:10:45.061784+0000","last_fullsized":"2024-04-25T14:10:45.061784+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:52.050283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.040213+0000","last_change":"2024-04-25T14:10:45.040213+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.040213+0000","last_undegraded":"2024-04-25T14:10:45.040213+0000","last_fullsized":"2024-04-25T14:10:45.040213+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.052969+0000","last_change":"2024-04-25T14:10:45.043256+0000","last_active":"2024-04-25T14:10:45.052969+0000","last_peered":"2024-04-25T14:10:45.052969+0000","last_clean":"2024-04-25T14:10:45.052969+0000","last_became_active":"2024-04-25T14:10:45.042731+0000","last_became_peered":"2024-04-25T14:10:45.042731+0000","last_unstale":"2024-04-25T14:10:45.052969+0000","last_undegraded":"2024-04-25T14:10:45.052969+0000","last_fullsized":"2024-04-25T14:10:45.052969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:20.290586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.037519+0000","last_change":"2024-04-25T14:10:45.037519+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.037519+0000","last_undegraded":"2024-04-25T14:10:45.037519+0000","last_fullsized":"2024-04-25T14:10:45.037519+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.056594+0000","last_change":"2024-04-25T14:10:45.053553+0000","last_active":"2024-04-25T14:10:45.056594+0000","last_peered":"2024-04-25T14:10:45.056594+0000","last_clean":"2024-04-25T14:10:45.056594+0000","last_became_active":"2024-04-25T14:10:45.053178+0000","last_became_peered":"2024-04-25T14:10:45.053178+0000","last_unstale":"2024-04-25T14:10:45.056594+0000","last_undegraded":"2024-04-25T14:10:45.056594+0000","last_fullsized":"2024-04-25T14:10:45.056594+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:55.116141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"124'2021","reported_seq":2403,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.602454+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:10:45.602454+0000","last_peered":"2024-04-25T14:10:45.602454+0000","last_clean":"2024-04-25T14:10:45.602454+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:10:45.602454+0000","last_undegraded":"2024-04-25T14:10:45.602454+0000","last_fullsized":"2024-04-25T14:10:45.602454+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":2021,"log_dups_size":0,"ondisk_log_size":2021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":529,"num_read_kb":1389,"num_write":187,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":529,"num_read_kb":1389,"num_write":187,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2021,"ondisk_log_size":2021,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574930,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41584,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330256,"statfs":{"total":96636764160,"available":96594182144,"internally_reserved":0,"allocated":2134016,"data_stored":1358180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32684,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1540096,"data_stored":767812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32684,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1540096,"data_stored":767812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574931,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1552384,"data_stored":774471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1552384,"data_stored":774471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574930,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1552384,"data_stored":774471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32684,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1540096,"data_stored":767812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574931,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41592,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330248,"statfs":{"total":96636764160,"available":96594173952,"internally_reserved":0,"allocated":2146304,"data_stored":1364839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:10:50.681 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:10:50.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.877+0000 7fd471200640 1 Processor -- start 2024-04-25T14:10:50.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.877+0000 7fd471200640 1 -- start start 2024-04-25T14:10:50.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.881+0000 7fd471200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd46c0cf970 0x7fd46c070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.881+0000 7fd471200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd46c071360 0x7fd46c06e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.881+0000 7fd471200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd46c071770 con 0x7fd46c06e7a0 2024-04-25T14:10:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.881+0000 7fd471200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd46c0cedf0 con 0x7fd46c071360 2024-04-25T14:10:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.881+0000 7fd471200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd46c163830 con 0x7fd46c0cf970 2024-04-25T14:10:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.881+0000 7fd46a800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd46c06e7a0 0x7fd46c0722f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54958/0 (socket says 172.21.15.86:54958) 2024-04-25T14:10:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.881+0000 7fd46a800640 1 -- 172.21.15.86:0/689038226 learned_addr learned my addr 172.21.15.86:0/689038226 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.881+0000 7fd469400640 1 --2- 172.21.15.86:0/689038226 >> v2:172.21.15.155:3300/0 conn(0x7fd46c071360 0x7fd46c06e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.881+0000 7fd469e00640 1 --2- 172.21.15.86:0/689038226 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd46c0cf970 0x7fd46c070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.881+0000 7fd468a00640 1 -- 172.21.15.86:0/689038226 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 93722380 0 0) 0x7fd4640014a0 con 0x7fd46c06e7a0 2024-04-25T14:10:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.881+0000 7fd468a00640 1 -- 172.21.15.86:0/689038226 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd450003530 con 0x7fd46c06e7a0 2024-04-25T14:10:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.881+0000 7fd469400640 1 -- 172.21.15.86:0/689038226 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd46c0cf970 msgr2=0x7fd46c070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.881+0000 7fd469400640 1 --2- 172.21.15.86:0/689038226 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd46c0cf970 0x7fd46c070e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.881+0000 7fd469400640 1 -- 172.21.15.86:0/689038226 >> v1:172.21.15.86:6789/0 conn(0x7fd46c06e7a0 legacy=0x7fd46c0722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.881+0000 7fd469400640 1 -- 172.21.15.86:0/689038226 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd46c06ef90 con 0x7fd46c071360 2024-04-25T14:10:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.881+0000 7fd469e00640 1 --2- 172.21.15.86:0/689038226 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd46c0cf970 0x7fd46c070e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:10:50.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.885+0000 7fd469400640 1 --2- 172.21.15.86:0/689038226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd46c071360 0x7fd46c06e260 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd4580097a0 tx=0x7fd45801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1388acfb1606c73e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:50.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.885+0000 7fd468a00640 1 -- 172.21.15.86:0/689038226 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4580b0070 con 0x7fd46c071360 2024-04-25T14:10:50.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.885+0000 7fd468a00640 1 -- 172.21.15.86:0/689038226 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4580ab600 con 0x7fd46c071360 2024-04-25T14:10:50.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.885+0000 7fd468a00640 1 -- 172.21.15.86:0/689038226 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4580ab9d0 con 0x7fd46c071360 2024-04-25T14:10:50.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.885+0000 7fd471200640 1 -- 172.21.15.86:0/689038226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd46c071360 msgr2=0x7fd46c06e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:50.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.885+0000 7fd471200640 1 --2- 172.21.15.86:0/689038226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd46c071360 0x7fd46c06e260 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd4580097a0 tx=0x7fd45801c450 comp rx=0 tx=0).stop 2024-04-25T14:10:50.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.885+0000 7fd471200640 1 -- 172.21.15.86:0/689038226 shutdown_connections 2024-04-25T14:10:50.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.885+0000 7fd471200640 1 --2- 172.21.15.86:0/689038226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd46c071360 0x7fd46c06e260 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.885+0000 7fd471200640 1 --2- 172.21.15.86:0/689038226 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd46c0cf970 0x7fd46c070e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.885+0000 7fd471200640 1 -- 172.21.15.86:0/689038226 >> 172.21.15.86:0/689038226 conn(0x7fd46c0cffc0 msgr2=0x7fd46c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:50.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.885+0000 7fd471200640 1 -- 172.21.15.86:0/689038226 shutdown_connections 2024-04-25T14:10:50.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.885+0000 7fd471200640 1 -- 172.21.15.86:0/689038226 wait complete. 2024-04-25T14:10:50.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.889+0000 7fd471200640 1 Processor -- start 2024-04-25T14:10:50.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.889+0000 7fd471200640 1 -- start start 2024-04-25T14:10:50.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.889+0000 7fd471200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd46c06e7a0 0x7fd46c1507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:50.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.889+0000 7fd471200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd46c071360 0x7fd46c15d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:50.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.889+0000 7fd471200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd46c0cf970 0x7fd46c15da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:50.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.889+0000 7fd471200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c06ed30 con 0x7fd46c0cf970 2024-04-25T14:10:50.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.889+0000 7fd471200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c06ebb0 con 0x7fd46c071360 2024-04-25T14:10:50.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.889+0000 7fd471200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd46c16cbd0 con 0x7fd46c06e7a0 2024-04-25T14:10:50.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.889+0000 7fd46a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd46c0cf970 0x7fd46c15da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:50.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.889+0000 7fd46a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd46c0cf970 0x7fd46c15da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46198/0 (socket says 172.21.15.86:46198) 2024-04-25T14:10:50.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.889+0000 7fd46a800640 1 -- 172.21.15.86:0/2015531379 learned_addr learned my addr 172.21.15.86:0/2015531379 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:50.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.889+0000 7fd469400640 1 --2- 172.21.15.86:0/2015531379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd46c071360 0x7fd46c15d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:50.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.889+0000 7fd469e00640 1 --2- 172.21.15.86:0/2015531379 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd46c06e7a0 0x7fd46c1507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:50.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.889+0000 7fd46a800640 1 -- 172.21.15.86:0/2015531379 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd46c06e7a0 msgr2=0x7fd46c1507e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:50.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.889+0000 7fd46a800640 1 --2- 172.21.15.86:0/2015531379 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd46c06e7a0 0x7fd46c1507e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.889+0000 7fd46a800640 1 -- 172.21.15.86:0/2015531379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd46c071360 msgr2=0x7fd46c15d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:50.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.889+0000 7fd46a800640 1 --2- 172.21.15.86:0/2015531379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd46c071360 0x7fd46c15d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:50.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.889+0000 7fd46a800640 1 -- 172.21.15.86:0/2015531379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd46c0cc110 con 0x7fd46c0cf970 2024-04-25T14:10:50.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.889+0000 7fd469e00640 1 --2- 172.21.15.86:0/2015531379 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd46c06e7a0 0x7fd46c1507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:10:50.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.889+0000 7fd469400640 1 --2- 172.21.15.86:0/2015531379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd46c071360 0x7fd46c15d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:10:50.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.889+0000 7fd46a800640 1 --2- 172.21.15.86:0/2015531379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd46c0cf970 0x7fd46c15da40 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fd46400ca70 tx=0x7fd46400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:50.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.889+0000 7fd462000640 1 -- 172.21.15.86:0/2015531379 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd464017070 con 0x7fd46c0cf970 2024-04-25T14:10:50.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.889+0000 7fd462000640 1 -- 172.21.15.86:0/2015531379 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd46400ab70 con 0x7fd46c0cf970 2024-04-25T14:10:50.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.889+0000 7fd462000640 1 -- 172.21.15.86:0/2015531379 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd464012400 con 0x7fd46c0cf970 2024-04-25T14:10:50.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.889+0000 7fd471200640 1 -- 172.21.15.86:0/2015531379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd46c0cc3f0 con 0x7fd46c0cf970 2024-04-25T14:10:50.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.889+0000 7fd471200640 1 -- 172.21.15.86:0/2015531379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd46c0cc9b0 con 0x7fd46c0cf970 2024-04-25T14:10:50.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.893+0000 7fd471200640 1 -- 172.21.15.86:0/2015531379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd46c0720e0 con 0x7fd46c0cf970 2024-04-25T14:10:50.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.893+0000 7fd462000640 1 -- 172.21.15.86:0/2015531379 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7fd46401a070 con 0x7fd46c0cf970 2024-04-25T14:10:50.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.893+0000 7fd462000640 1 --2- 172.21.15.86:0/2015531379 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fd44c06f490 0x7fd44c0719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:50.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.893+0000 7fd462000640 1 -- 172.21.15.86:0/2015531379 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7fd46408f250 con 0x7fd46c0cf970 2024-04-25T14:10:50.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.893+0000 7fd469e00640 1 --2- 172.21.15.86:0/2015531379 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fd44c06f490 0x7fd44c0719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:50.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.893+0000 7fd469e00640 1 --2- 172.21.15.86:0/2015531379 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fd44c06f490 0x7fd44c0719a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009580 tx=0x7fd45c009190 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:50.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:50.897+0000 7fd462000640 1 -- 172.21.15.86:0/2015531379 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd46c071770 con 0x7fd46c0cf970 2024-04-25T14:10:51.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.057+0000 7fd471200640 1 -- 172.21.15.86:0/2015531379 --> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd46c069d60 con 0x7fd44c06f490 2024-04-25T14:10:51.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.065+0000 7fd462000640 1 -- 172.21.15.86:0/2015531379 <== mgr.8338 v2:172.21.15.86:6832/3964666460 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247083 (secure 0 0 0) 0x7fd45c011dd0 con 0x7fd44c06f490 2024-04-25T14:10:51.064 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:51.065 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:10:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.069+0000 7fd471200640 1 -- 172.21.15.86:0/2015531379 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fd44c06f490 msgr2=0x7fd44c0719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.069+0000 7fd471200640 1 --2- 172.21.15.86:0/2015531379 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fd44c06f490 0x7fd44c0719a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009580 tx=0x7fd45c009190 comp rx=0 tx=0).stop 2024-04-25T14:10:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.069+0000 7fd471200640 1 -- 172.21.15.86:0/2015531379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd46c0cf970 msgr2=0x7fd46c15da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.069+0000 7fd471200640 1 --2- 172.21.15.86:0/2015531379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd46c0cf970 0x7fd46c15da40 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fd46400ca70 tx=0x7fd46400cf30 comp rx=0 tx=0).stop 2024-04-25T14:10:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.069+0000 7fd471200640 1 -- 172.21.15.86:0/2015531379 shutdown_connections 2024-04-25T14:10:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.069+0000 7fd471200640 1 --2- 172.21.15.86:0/2015531379 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fd44c06f490 0x7fd44c0719a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.069+0000 7fd471200640 1 --2- 172.21.15.86:0/2015531379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd46c0cf970 0x7fd46c15da40 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.069+0000 7fd471200640 1 --2- 172.21.15.86:0/2015531379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd46c071360 0x7fd46c15d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.069+0000 7fd471200640 1 --2- 172.21.15.86:0/2015531379 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd46c06e7a0 0x7fd46c1507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.069+0000 7fd471200640 1 -- 172.21.15.86:0/2015531379 >> 172.21.15.86:0/2015531379 conn(0x7fd46c0cffc0 msgr2=0x7fd46c163000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.069+0000 7fd471200640 1 -- 172.21.15.86:0/2015531379 shutdown_connections 2024-04-25T14:10:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.069+0000 7fd471200640 1 -- 172.21.15.86:0/2015531379 wait complete. 2024-04-25T14:10:51.086 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T14:10:49.615624+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":529,"num_read_kb":1389,"num_write":187,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2021,"ondisk_log_size":2021,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279328,"kb_used_data":13240,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754695392,"statfs":{"total":773094113280,"available":772808081408,"internally_reserved":0,"allocated":13557760,"data_stored":7349868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.588242"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.043709+0000","last_change":"2024-04-25T14:10:45.043709+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.043709+0000","last_undegraded":"2024-04-25T14:10:45.043709+0000","last_fullsized":"2024-04-25T14:10:45.043709+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.034723+0000","last_change":"2024-04-25T14:10:45.034723+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.034723+0000","last_undegraded":"2024-04-25T14:10:45.034723+0000","last_fullsized":"2024-04-25T14:10:45.034723+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038065+0000","last_change":"2024-04-25T14:10:45.038065+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038065+0000","last_undegraded":"2024-04-25T14:10:45.038065+0000","last_fullsized":"2024-04-25T14:10:45.038065+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.032697+0000","last_change":"2024-04-25T14:10:45.032697+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.032697+0000","last_undegraded":"2024-04-25T14:10:45.032697+0000","last_fullsized":"2024-04-25T14:10:45.032697+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038541+0000","last_change":"2024-04-25T14:10:45.038541+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038541+0000","last_undegraded":"2024-04-25T14:10:45.038541+0000","last_fullsized":"2024-04-25T14:10:45.038541+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.045986+0000","last_change":"2024-04-25T14:10:45.045986+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.045986+0000","last_undegraded":"2024-04-25T14:10:45.045986+0000","last_fullsized":"2024-04-25T14:10:45.045986+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.037964+0000","last_change":"2024-04-25T14:10:45.037964+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.037964+0000","last_undegraded":"2024-04-25T14:10:45.037964+0000","last_fullsized":"2024-04-25T14:10:45.037964+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.043120+0000","last_change":"2024-04-25T14:10:45.043120+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.043120+0000","last_undegraded":"2024-04-25T14:10:45.043120+0000","last_fullsized":"2024-04-25T14:10:45.043120+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.049467+0000","last_change":"2024-04-25T14:10:45.049467+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.049467+0000","last_undegraded":"2024-04-25T14:10:45.049467+0000","last_fullsized":"2024-04-25T14:10:45.049467+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039375+0000","last_change":"2024-04-25T14:10:45.039375+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039375+0000","last_undegraded":"2024-04-25T14:10:45.039375+0000","last_fullsized":"2024-04-25T14:10:45.039375+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.034076+0000","last_change":"2024-04-25T14:10:45.034076+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.034076+0000","last_undegraded":"2024-04-25T14:10:45.034076+0000","last_fullsized":"2024-04-25T14:10:45.034076+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.041407+0000","last_change":"2024-04-25T14:10:45.041407+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.041407+0000","last_undegraded":"2024-04-25T14:10:45.041407+0000","last_fullsized":"2024-04-25T14:10:45.041407+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.047463+0000","last_change":"2024-04-25T14:10:45.047463+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.047463+0000","last_undegraded":"2024-04-25T14:10:45.047463+0000","last_fullsized":"2024-04-25T14:10:45.047463+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.040256+0000","last_change":"2024-04-25T14:10:45.040256+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.040256+0000","last_undegraded":"2024-04-25T14:10:45.040256+0000","last_fullsized":"2024-04-25T14:10:45.040256+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.040028+0000","last_change":"2024-04-25T14:10:45.040028+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.040028+0000","last_undegraded":"2024-04-25T14:10:45.040028+0000","last_fullsized":"2024-04-25T14:10:45.040028+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.045366+0000","last_change":"2024-04-25T14:10:45.045366+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.045366+0000","last_undegraded":"2024-04-25T14:10:45.045366+0000","last_fullsized":"2024-04-25T14:10:45.045366+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.048856+0000","last_change":"2024-04-25T14:10:45.048856+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.048856+0000","last_undegraded":"2024-04-25T14:10:45.048856+0000","last_fullsized":"2024-04-25T14:10:45.048856+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.040344+0000","last_change":"2024-04-25T14:10:45.040344+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.040344+0000","last_undegraded":"2024-04-25T14:10:45.040344+0000","last_fullsized":"2024-04-25T14:10:45.040344+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039190+0000","last_change":"2024-04-25T14:10:45.039190+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039190+0000","last_undegraded":"2024-04-25T14:10:45.039190+0000","last_fullsized":"2024-04-25T14:10:45.039190+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038526+0000","last_change":"2024-04-25T14:10:45.038526+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038526+0000","last_undegraded":"2024-04-25T14:10:45.038526+0000","last_fullsized":"2024-04-25T14:10:45.038526+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.034790+0000","last_change":"2024-04-25T14:10:45.034790+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.034790+0000","last_undegraded":"2024-04-25T14:10:45.034790+0000","last_fullsized":"2024-04-25T14:10:45.034790+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.040708+0000","last_change":"2024-04-25T14:10:45.040708+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.040708+0000","last_undegraded":"2024-04-25T14:10:45.040708+0000","last_fullsized":"2024-04-25T14:10:45.040708+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.032697+0000","last_change":"2024-04-25T14:10:45.032697+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.032697+0000","last_undegraded":"2024-04-25T14:10:45.032697+0000","last_fullsized":"2024-04-25T14:10:45.032697+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.044889+0000","last_change":"2024-04-25T14:10:45.044889+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.044889+0000","last_undegraded":"2024-04-25T14:10:45.044889+0000","last_fullsized":"2024-04-25T14:10:45.044889+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039272+0000","last_change":"2024-04-25T14:10:45.039272+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039272+0000","last_undegraded":"2024-04-25T14:10:45.039272+0000","last_fullsized":"2024-04-25T14:10:45.039272+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038197+0000","last_change":"2024-04-25T14:10:45.038197+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038197+0000","last_undegraded":"2024-04-25T14:10:45.038197+0000","last_fullsized":"2024-04-25T14:10:45.038197+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038708+0000","last_change":"2024-04-25T14:10:45.038708+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038708+0000","last_undegraded":"2024-04-25T14:10:45.038708+0000","last_fullsized":"2024-04-25T14:10:45.038708+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039464+0000","last_change":"2024-04-25T14:10:45.039464+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039464+0000","last_undegraded":"2024-04-25T14:10:45.039464+0000","last_fullsized":"2024-04-25T14:10:45.039464+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.040768+0000","last_change":"2024-04-25T14:10:45.040768+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.040768+0000","last_undegraded":"2024-04-25T14:10:45.040768+0000","last_fullsized":"2024-04-25T14:10:45.040768+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038367+0000","last_change":"2024-04-25T14:10:45.038367+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038367+0000","last_undegraded":"2024-04-25T14:10:45.038367+0000","last_fullsized":"2024-04-25T14:10:45.038367+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.044360+0000","last_change":"2024-04-25T14:10:45.044360+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.044360+0000","last_undegraded":"2024-04-25T14:10:45.044360+0000","last_fullsized":"2024-04-25T14:10:45.044360+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039596+0000","last_change":"2024-04-25T14:10:45.039596+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039596+0000","last_undegraded":"2024-04-25T14:10:45.039596+0000","last_fullsized":"2024-04-25T14:10:45.039596+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.035461+0000","last_change":"2024-04-25T14:10:45.035461+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.035461+0000","last_undegraded":"2024-04-25T14:10:45.035461+0000","last_fullsized":"2024-04-25T14:10:45.035461+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.052380+0000","last_change":"2024-04-25T14:10:45.051471+0000","last_active":"2024-04-25T14:10:45.052380+0000","last_peered":"2024-04-25T14:10:45.052380+0000","last_clean":"2024-04-25T14:10:45.052380+0000","last_became_active":"2024-04-25T14:10:45.051055+0000","last_became_peered":"2024-04-25T14:10:45.051055+0000","last_unstale":"2024-04-25T14:10:45.052380+0000","last_undegraded":"2024-04-25T14:10:45.052380+0000","last_fullsized":"2024-04-25T14:10:45.052380+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:35.948249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039532+0000","last_change":"2024-04-25T14:10:45.039532+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039532+0000","last_undegraded":"2024-04-25T14:10:45.039532+0000","last_fullsized":"2024-04-25T14:10:45.039532+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.052927+0000","last_change":"2024-04-25T14:10:45.046859+0000","last_active":"2024-04-25T14:10:45.052927+0000","last_peered":"2024-04-25T14:10:45.052927+0000","last_clean":"2024-04-25T14:10:45.052927+0000","last_became_active":"2024-04-25T14:10:45.044618+0000","last_became_peered":"2024-04-25T14:10:45.044618+0000","last_unstale":"2024-04-25T14:10:45.052927+0000","last_undegraded":"2024-04-25T14:10:45.052927+0000","last_fullsized":"2024-04-25T14:10:45.052927+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:55.442843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.033422+0000","last_change":"2024-04-25T14:10:45.033422+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.033422+0000","last_undegraded":"2024-04-25T14:10:45.033422+0000","last_fullsized":"2024-04-25T14:10:45.033422+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.057542+0000","last_change":"2024-04-25T14:10:45.056867+0000","last_active":"2024-04-25T14:10:45.057542+0000","last_peered":"2024-04-25T14:10:45.057542+0000","last_clean":"2024-04-25T14:10:45.057542+0000","last_became_active":"2024-04-25T14:10:45.056470+0000","last_became_peered":"2024-04-25T14:10:45.056470+0000","last_unstale":"2024-04-25T14:10:45.057542+0000","last_undegraded":"2024-04-25T14:10:45.057542+0000","last_fullsized":"2024-04-25T14:10:45.057542+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:01.583932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.043935+0000","last_change":"2024-04-25T14:10:45.043935+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.043935+0000","last_undegraded":"2024-04-25T14:10:45.043935+0000","last_fullsized":"2024-04-25T14:10:45.043935+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.054827+0000","last_change":"2024-04-25T14:10:45.052367+0000","last_active":"2024-04-25T14:10:45.054827+0000","last_peered":"2024-04-25T14:10:45.054827+0000","last_clean":"2024-04-25T14:10:45.054827+0000","last_became_active":"2024-04-25T14:10:45.052055+0000","last_became_peered":"2024-04-25T14:10:45.052055+0000","last_unstale":"2024-04-25T14:10:45.054827+0000","last_undegraded":"2024-04-25T14:10:45.054827+0000","last_fullsized":"2024-04-25T14:10:45.054827+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:52.050283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.041991+0000","last_change":"2024-04-25T14:10:45.041991+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.041991+0000","last_undegraded":"2024-04-25T14:10:45.041991+0000","last_fullsized":"2024-04-25T14:10:45.041991+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.044829+0000","last_change":"2024-04-25T14:10:45.041897+0000","last_active":"2024-04-25T14:10:45.044829+0000","last_peered":"2024-04-25T14:10:45.044829+0000","last_clean":"2024-04-25T14:10:45.044829+0000","last_became_active":"2024-04-25T14:10:45.039679+0000","last_became_peered":"2024-04-25T14:10:45.039679+0000","last_unstale":"2024-04-25T14:10:45.044829+0000","last_undegraded":"2024-04-25T14:10:45.044829+0000","last_fullsized":"2024-04-25T14:10:45.044829+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:22.669468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.040465+0000","last_change":"2024-04-25T14:10:45.040465+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.040465+0000","last_undegraded":"2024-04-25T14:10:45.040465+0000","last_fullsized":"2024-04-25T14:10:45.040465+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.053034+0000","last_change":"2024-04-25T14:10:45.051135+0000","last_active":"2024-04-25T14:10:45.053034+0000","last_peered":"2024-04-25T14:10:45.053034+0000","last_clean":"2024-04-25T14:10:45.053034+0000","last_became_active":"2024-04-25T14:10:45.050463+0000","last_became_peered":"2024-04-25T14:10:45.050463+0000","last_unstale":"2024-04-25T14:10:45.053034+0000","last_undegraded":"2024-04-25T14:10:45.053034+0000","last_fullsized":"2024-04-25T14:10:45.053034+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:12.563128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039895+0000","last_change":"2024-04-25T14:10:45.039895+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039895+0000","last_undegraded":"2024-04-25T14:10:45.039895+0000","last_fullsized":"2024-04-25T14:10:45.039895+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.056835+0000","last_change":"2024-04-25T14:10:45.053969+0000","last_active":"2024-04-25T14:10:45.056835+0000","last_peered":"2024-04-25T14:10:45.056835+0000","last_clean":"2024-04-25T14:10:45.056835+0000","last_became_active":"2024-04-25T14:10:45.053634+0000","last_became_peered":"2024-04-25T14:10:45.053634+0000","last_unstale":"2024-04-25T14:10:45.056835+0000","last_undegraded":"2024-04-25T14:10:45.056835+0000","last_fullsized":"2024-04-25T14:10:45.056835+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:54.008821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.040667+0000","last_change":"2024-04-25T14:10:45.040667+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.040667+0000","last_undegraded":"2024-04-25T14:10:45.040667+0000","last_fullsized":"2024-04-25T14:10:45.040667+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.056301+0000","last_change":"2024-04-25T14:10:45.054900+0000","last_active":"2024-04-25T14:10:45.056301+0000","last_peered":"2024-04-25T14:10:45.056301+0000","last_clean":"2024-04-25T14:10:45.056301+0000","last_became_active":"2024-04-25T14:10:45.054599+0000","last_became_peered":"2024-04-25T14:10:45.054599+0000","last_unstale":"2024-04-25T14:10:45.056301+0000","last_undegraded":"2024-04-25T14:10:45.056301+0000","last_fullsized":"2024-04-25T14:10:45.056301+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:05.356606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039869+0000","last_change":"2024-04-25T14:10:45.039869+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039869+0000","last_undegraded":"2024-04-25T14:10:45.039869+0000","last_fullsized":"2024-04-25T14:10:45.039869+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.045456+0000","last_change":"2024-04-25T14:10:45.037687+0000","last_active":"2024-04-25T14:10:45.045456+0000","last_peered":"2024-04-25T14:10:45.045456+0000","last_clean":"2024-04-25T14:10:45.045456+0000","last_became_active":"2024-04-25T14:10:45.037370+0000","last_became_peered":"2024-04-25T14:10:45.037370+0000","last_unstale":"2024-04-25T14:10:45.045456+0000","last_undegraded":"2024-04-25T14:10:45.045456+0000","last_fullsized":"2024-04-25T14:10:45.045456+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:54.008821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.046751+0000","last_change":"2024-04-25T14:10:45.046751+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.046751+0000","last_undegraded":"2024-04-25T14:10:45.046751+0000","last_fullsized":"2024-04-25T14:10:45.046751+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.058741+0000","last_change":"2024-04-25T14:10:45.050926+0000","last_active":"2024-04-25T14:10:45.058741+0000","last_peered":"2024-04-25T14:10:45.058741+0000","last_clean":"2024-04-25T14:10:45.058741+0000","last_became_active":"2024-04-25T14:10:45.049733+0000","last_became_peered":"2024-04-25T14:10:45.049733+0000","last_unstale":"2024-04-25T14:10:45.058741+0000","last_undegraded":"2024-04-25T14:10:45.058741+0000","last_fullsized":"2024-04-25T14:10:45.058741+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:05.958831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.037539+0000","last_change":"2024-04-25T14:10:45.037539+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.037539+0000","last_undegraded":"2024-04-25T14:10:45.037539+0000","last_fullsized":"2024-04-25T14:10:45.037539+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.054475+0000","last_change":"2024-04-25T14:10:45.052411+0000","last_active":"2024-04-25T14:10:45.054475+0000","last_peered":"2024-04-25T14:10:45.054475+0000","last_clean":"2024-04-25T14:10:45.054475+0000","last_became_active":"2024-04-25T14:10:45.052020+0000","last_became_peered":"2024-04-25T14:10:45.052020+0000","last_unstale":"2024-04-25T14:10:45.054475+0000","last_undegraded":"2024-04-25T14:10:45.054475+0000","last_fullsized":"2024-04-25T14:10:45.054475+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:33.243994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.040238+0000","last_change":"2024-04-25T14:10:45.040238+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.040238+0000","last_undegraded":"2024-04-25T14:10:45.040238+0000","last_fullsized":"2024-04-25T14:10:45.040238+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.057096+0000","last_change":"2024-04-25T14:10:45.054962+0000","last_active":"2024-04-25T14:10:45.057096+0000","last_peered":"2024-04-25T14:10:45.057096+0000","last_clean":"2024-04-25T14:10:45.057096+0000","last_became_active":"2024-04-25T14:10:45.054639+0000","last_became_peered":"2024-04-25T14:10:45.054639+0000","last_unstale":"2024-04-25T14:10:45.057096+0000","last_undegraded":"2024-04-25T14:10:45.057096+0000","last_fullsized":"2024-04-25T14:10:45.057096+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:38.714815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038826+0000","last_change":"2024-04-25T14:10:45.038826+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038826+0000","last_undegraded":"2024-04-25T14:10:45.038826+0000","last_fullsized":"2024-04-25T14:10:45.038826+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.058736+0000","last_change":"2024-04-25T14:10:45.051737+0000","last_active":"2024-04-25T14:10:45.058736+0000","last_peered":"2024-04-25T14:10:45.058736+0000","last_clean":"2024-04-25T14:10:45.058736+0000","last_became_active":"2024-04-25T14:10:45.051337+0000","last_became_peered":"2024-04-25T14:10:45.051337+0000","last_unstale":"2024-04-25T14:10:45.058736+0000","last_undegraded":"2024-04-25T14:10:45.058736+0000","last_fullsized":"2024-04-25T14:10:45.058736+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:25.432658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039786+0000","last_change":"2024-04-25T14:10:45.039786+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039786+0000","last_undegraded":"2024-04-25T14:10:45.039786+0000","last_fullsized":"2024-04-25T14:10:45.039786+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.052338+0000","last_change":"2024-04-25T14:10:45.049002+0000","last_active":"2024-04-25T14:10:45.052338+0000","last_peered":"2024-04-25T14:10:45.052338+0000","last_clean":"2024-04-25T14:10:45.052338+0000","last_became_active":"2024-04-25T14:10:45.048755+0000","last_became_peered":"2024-04-25T14:10:45.048755+0000","last_unstale":"2024-04-25T14:10:45.052338+0000","last_undegraded":"2024-04-25T14:10:45.052338+0000","last_fullsized":"2024-04-25T14:10:45.052338+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:04.824306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.037596+0000","last_change":"2024-04-25T14:10:45.037596+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.037596+0000","last_undegraded":"2024-04-25T14:10:45.037596+0000","last_fullsized":"2024-04-25T14:10:45.037596+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.055500+0000","last_change":"2024-04-25T14:10:45.051770+0000","last_active":"2024-04-25T14:10:45.055500+0000","last_peered":"2024-04-25T14:10:45.055500+0000","last_clean":"2024-04-25T14:10:45.055500+0000","last_became_active":"2024-04-25T14:10:45.051446+0000","last_became_peered":"2024-04-25T14:10:45.051446+0000","last_unstale":"2024-04-25T14:10:45.055500+0000","last_undegraded":"2024-04-25T14:10:45.055500+0000","last_fullsized":"2024-04-25T14:10:45.055500+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:01.990667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038531+0000","last_change":"2024-04-25T14:10:45.038531+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038531+0000","last_undegraded":"2024-04-25T14:10:45.038531+0000","last_fullsized":"2024-04-25T14:10:45.038531+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.042874+0000","last_change":"2024-04-25T14:10:45.037175+0000","last_active":"2024-04-25T14:10:45.042874+0000","last_peered":"2024-04-25T14:10:45.042874+0000","last_clean":"2024-04-25T14:10:45.042874+0000","last_became_active":"2024-04-25T14:10:45.036881+0000","last_became_peered":"2024-04-25T14:10:45.036881+0000","last_unstale":"2024-04-25T14:10:45.042874+0000","last_undegraded":"2024-04-25T14:10:45.042874+0000","last_fullsized":"2024-04-25T14:10:45.042874+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:01.990667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.037392+0000","last_change":"2024-04-25T14:10:45.037392+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.037392+0000","last_undegraded":"2024-04-25T14:10:45.037392+0000","last_fullsized":"2024-04-25T14:10:45.037392+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.053697+0000","last_change":"2024-04-25T14:10:45.052421+0000","last_active":"2024-04-25T14:10:45.053697+0000","last_peered":"2024-04-25T14:10:45.053697+0000","last_clean":"2024-04-25T14:10:45.053697+0000","last_became_active":"2024-04-25T14:10:45.052078+0000","last_became_peered":"2024-04-25T14:10:45.052078+0000","last_unstale":"2024-04-25T14:10:45.053697+0000","last_undegraded":"2024-04-25T14:10:45.053697+0000","last_fullsized":"2024-04-25T14:10:45.053697+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:49.038115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039058+0000","last_change":"2024-04-25T14:10:45.039058+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039058+0000","last_undegraded":"2024-04-25T14:10:45.039058+0000","last_fullsized":"2024-04-25T14:10:45.039058+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.058211+0000","last_change":"2024-04-25T14:10:45.056493+0000","last_active":"2024-04-25T14:10:45.058211+0000","last_peered":"2024-04-25T14:10:45.058211+0000","last_clean":"2024-04-25T14:10:45.058211+0000","last_became_active":"2024-04-25T14:10:45.055271+0000","last_became_peered":"2024-04-25T14:10:45.055271+0000","last_unstale":"2024-04-25T14:10:45.058211+0000","last_undegraded":"2024-04-25T14:10:45.058211+0000","last_fullsized":"2024-04-25T14:10:45.058211+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:58.758388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.037958+0000","last_change":"2024-04-25T14:10:45.037958+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.037958+0000","last_undegraded":"2024-04-25T14:10:45.037958+0000","last_fullsized":"2024-04-25T14:10:45.037958+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.058425+0000","last_change":"2024-04-25T14:10:45.055677+0000","last_active":"2024-04-25T14:10:45.058425+0000","last_peered":"2024-04-25T14:10:45.058425+0000","last_clean":"2024-04-25T14:10:45.058425+0000","last_became_active":"2024-04-25T14:10:45.054262+0000","last_became_peered":"2024-04-25T14:10:45.054262+0000","last_unstale":"2024-04-25T14:10:45.058425+0000","last_undegraded":"2024-04-25T14:10:45.058425+0000","last_fullsized":"2024-04-25T14:10:45.058425+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:34.895882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038038+0000","last_change":"2024-04-25T14:10:45.038038+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038038+0000","last_undegraded":"2024-04-25T14:10:45.038038+0000","last_fullsized":"2024-04-25T14:10:45.038038+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.049582+0000","last_change":"2024-04-25T14:10:45.047266+0000","last_active":"2024-04-25T14:10:45.049582+0000","last_peered":"2024-04-25T14:10:45.049582+0000","last_clean":"2024-04-25T14:10:45.049582+0000","last_became_active":"2024-04-25T14:10:45.046928+0000","last_became_peered":"2024-04-25T14:10:45.046928+0000","last_unstale":"2024-04-25T14:10:45.049582+0000","last_undegraded":"2024-04-25T14:10:45.049582+0000","last_fullsized":"2024-04-25T14:10:45.049582+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:54.008821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.045070+0000","last_change":"2024-04-25T14:10:45.045070+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.045070+0000","last_undegraded":"2024-04-25T14:10:45.045070+0000","last_fullsized":"2024-04-25T14:10:45.045070+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.060466+0000","last_change":"2024-04-25T14:10:45.057160+0000","last_active":"2024-04-25T14:10:45.060466+0000","last_peered":"2024-04-25T14:10:45.060466+0000","last_clean":"2024-04-25T14:10:45.060466+0000","last_became_active":"2024-04-25T14:10:45.056871+0000","last_became_peered":"2024-04-25T14:10:45.056871+0000","last_unstale":"2024-04-25T14:10:45.060466+0000","last_undegraded":"2024-04-25T14:10:45.060466+0000","last_fullsized":"2024-04-25T14:10:45.060466+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:44.552534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.038103+0000","last_change":"2024-04-25T14:10:45.038103+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.038103+0000","last_undegraded":"2024-04-25T14:10:45.038103+0000","last_fullsized":"2024-04-25T14:10:45.038103+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.060444+0000","last_change":"2024-04-25T14:10:45.059841+0000","last_active":"2024-04-25T14:10:45.060444+0000","last_peered":"2024-04-25T14:10:45.060444+0000","last_clean":"2024-04-25T14:10:45.060444+0000","last_became_active":"2024-04-25T14:10:45.059609+0000","last_became_peered":"2024-04-25T14:10:45.059609+0000","last_unstale":"2024-04-25T14:10:45.060444+0000","last_undegraded":"2024-04-25T14:10:45.060444+0000","last_fullsized":"2024-04-25T14:10:45.060444+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:14.821225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.033610+0000","last_change":"2024-04-25T14:10:45.033610+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.033610+0000","last_undegraded":"2024-04-25T14:10:45.033610+0000","last_fullsized":"2024-04-25T14:10:45.033610+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.061732+0000","last_change":"2024-04-25T14:10:45.057442+0000","last_active":"2024-04-25T14:10:45.061732+0000","last_peered":"2024-04-25T14:10:45.061732+0000","last_clean":"2024-04-25T14:10:45.061732+0000","last_became_active":"2024-04-25T14:10:45.057077+0000","last_became_peered":"2024-04-25T14:10:45.057077+0000","last_unstale":"2024-04-25T14:10:45.061732+0000","last_undegraded":"2024-04-25T14:10:45.061732+0000","last_fullsized":"2024-04-25T14:10:45.061732+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:03.729472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.037513+0000","last_change":"2024-04-25T14:10:45.037513+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.037513+0000","last_undegraded":"2024-04-25T14:10:45.037513+0000","last_fullsized":"2024-04-25T14:10:45.037513+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.060351+0000","last_change":"2024-04-25T14:10:45.056735+0000","last_active":"2024-04-25T14:10:45.060351+0000","last_peered":"2024-04-25T14:10:45.060351+0000","last_clean":"2024-04-25T14:10:45.060351+0000","last_became_active":"2024-04-25T14:10:45.052530+0000","last_became_peered":"2024-04-25T14:10:45.052530+0000","last_unstale":"2024-04-25T14:10:45.060351+0000","last_undegraded":"2024-04-25T14:10:45.060351+0000","last_fullsized":"2024-04-25T14:10:45.060351+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:34.861278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.037754+0000","last_change":"2024-04-25T14:10:45.037754+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.037754+0000","last_undegraded":"2024-04-25T14:10:45.037754+0000","last_fullsized":"2024-04-25T14:10:45.037754+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.049696+0000","last_change":"2024-04-25T14:10:45.046573+0000","last_active":"2024-04-25T14:10:45.049696+0000","last_peered":"2024-04-25T14:10:45.049696+0000","last_clean":"2024-04-25T14:10:45.049696+0000","last_became_active":"2024-04-25T14:10:45.046226+0000","last_became_peered":"2024-04-25T14:10:45.046226+0000","last_unstale":"2024-04-25T14:10:45.049696+0000","last_undegraded":"2024-04-25T14:10:45.049696+0000","last_fullsized":"2024-04-25T14:10:45.049696+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:01.990667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.039513+0000","last_change":"2024-04-25T14:10:45.039513+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.039513+0000","last_undegraded":"2024-04-25T14:10:45.039513+0000","last_fullsized":"2024-04-25T14:10:45.039513+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.056936+0000","last_change":"2024-04-25T14:10:45.055809+0000","last_active":"2024-04-25T14:10:45.056936+0000","last_peered":"2024-04-25T14:10:45.056936+0000","last_clean":"2024-04-25T14:10:45.056936+0000","last_became_active":"2024-04-25T14:10:45.055480+0000","last_became_peered":"2024-04-25T14:10:45.055480+0000","last_unstale":"2024-04-25T14:10:45.056936+0000","last_undegraded":"2024-04-25T14:10:45.056936+0000","last_fullsized":"2024-04-25T14:10:45.056936+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:58.758388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.037351+0000","last_change":"2024-04-25T14:10:45.037351+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.037351+0000","last_undegraded":"2024-04-25T14:10:45.037351+0000","last_fullsized":"2024-04-25T14:10:45.037351+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.061791+0000","last_change":"2024-04-25T14:10:45.058112+0000","last_active":"2024-04-25T14:10:45.061791+0000","last_peered":"2024-04-25T14:10:45.061791+0000","last_clean":"2024-04-25T14:10:45.061791+0000","last_became_active":"2024-04-25T14:10:45.057825+0000","last_became_peered":"2024-04-25T14:10:45.057825+0000","last_unstale":"2024-04-25T14:10:45.061791+0000","last_undegraded":"2024-04-25T14:10:45.061791+0000","last_fullsized":"2024-04-25T14:10:45.061791+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:33.243994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.048890+0000","last_change":"2024-04-25T14:10:45.048890+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.048890+0000","last_undegraded":"2024-04-25T14:10:45.048890+0000","last_fullsized":"2024-04-25T14:10:45.048890+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.051870+0000","last_change":"2024-04-25T14:10:45.049671+0000","last_active":"2024-04-25T14:10:45.051870+0000","last_peered":"2024-04-25T14:10:45.051870+0000","last_clean":"2024-04-25T14:10:45.051870+0000","last_became_active":"2024-04-25T14:10:45.049375+0000","last_became_peered":"2024-04-25T14:10:45.049375+0000","last_unstale":"2024-04-25T14:10:45.051870+0000","last_undegraded":"2024-04-25T14:10:45.051870+0000","last_fullsized":"2024-04-25T14:10:45.051870+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:58.758388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.049911+0000","last_change":"2024-04-25T14:10:45.049911+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.049911+0000","last_undegraded":"2024-04-25T14:10:45.049911+0000","last_fullsized":"2024-04-25T14:10:45.049911+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.056549+0000","last_change":"2024-04-25T14:10:45.053154+0000","last_active":"2024-04-25T14:10:45.056549+0000","last_peered":"2024-04-25T14:10:45.056549+0000","last_clean":"2024-04-25T14:10:45.056549+0000","last_became_active":"2024-04-25T14:10:45.052842+0000","last_became_peered":"2024-04-25T14:10:45.052842+0000","last_unstale":"2024-04-25T14:10:45.056549+0000","last_undegraded":"2024-04-25T14:10:45.056549+0000","last_fullsized":"2024-04-25T14:10:45.056549+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:22.629958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.048918+0000","last_change":"2024-04-25T14:10:45.048918+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.048918+0000","last_undegraded":"2024-04-25T14:10:45.048918+0000","last_fullsized":"2024-04-25T14:10:45.048918+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.061784+0000","last_change":"2024-04-25T14:10:45.058009+0000","last_active":"2024-04-25T14:10:45.061784+0000","last_peered":"2024-04-25T14:10:45.061784+0000","last_clean":"2024-04-25T14:10:45.061784+0000","last_became_active":"2024-04-25T14:10:45.057724+0000","last_became_peered":"2024-04-25T14:10:45.057724+0000","last_unstale":"2024-04-25T14:10:45.061784+0000","last_undegraded":"2024-04-25T14:10:45.061784+0000","last_fullsized":"2024-04-25T14:10:45.061784+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:52.050283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.040213+0000","last_change":"2024-04-25T14:10:45.040213+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.040213+0000","last_undegraded":"2024-04-25T14:10:45.040213+0000","last_fullsized":"2024-04-25T14:10:45.040213+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.052969+0000","last_change":"2024-04-25T14:10:45.043256+0000","last_active":"2024-04-25T14:10:45.052969+0000","last_peered":"2024-04-25T14:10:45.052969+0000","last_clean":"2024-04-25T14:10:45.052969+0000","last_became_active":"2024-04-25T14:10:45.042731+0000","last_became_peered":"2024-04-25T14:10:45.042731+0000","last_unstale":"2024-04-25T14:10:45.052969+0000","last_undegraded":"2024-04-25T14:10:45.052969+0000","last_fullsized":"2024-04-25T14:10:45.052969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:20.290586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"creating+peering","last_fresh":"2024-04-25T14:10:45.037519+0000","last_change":"2024-04-25T14:10:45.037519+0000","last_active":"2024-04-25T14:10:45.026666+0000","last_peered":"2024-04-25T14:10:45.026666+0000","last_clean":"2024-04-25T14:10:45.026666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:10:45.037519+0000","last_undegraded":"2024-04-25T14:10:45.037519+0000","last_fullsized":"2024-04-25T14:10:45.037519+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.056594+0000","last_change":"2024-04-25T14:10:45.053553+0000","last_active":"2024-04-25T14:10:45.056594+0000","last_peered":"2024-04-25T14:10:45.056594+0000","last_clean":"2024-04-25T14:10:45.056594+0000","last_became_active":"2024-04-25T14:10:45.053178+0000","last_became_peered":"2024-04-25T14:10:45.053178+0000","last_unstale":"2024-04-25T14:10:45.056594+0000","last_undegraded":"2024-04-25T14:10:45.056594+0000","last_fullsized":"2024-04-25T14:10:45.056594+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:55.116141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"124'2021","reported_seq":2403,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T14:10:45.602454+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:10:45.602454+0000","last_peered":"2024-04-25T14:10:45.602454+0000","last_clean":"2024-04-25T14:10:45.602454+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:10:45.602454+0000","last_undegraded":"2024-04-25T14:10:45.602454+0000","last_fullsized":"2024-04-25T14:10:45.602454+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":2021,"log_dups_size":0,"ondisk_log_size":2021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":529,"num_read_kb":1389,"num_write":187,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":529,"num_read_kb":1389,"num_write":187,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2021,"ondisk_log_size":2021,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574930,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41584,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330256,"statfs":{"total":96636764160,"available":96594182144,"internally_reserved":0,"allocated":2134016,"data_stored":1358180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32684,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1540096,"data_stored":767812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32684,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1540096,"data_stored":767812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574931,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1552384,"data_stored":774471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1552384,"data_stored":774471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574930,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1552384,"data_stored":774471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32684,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1540096,"data_stored":767812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574931,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41592,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330248,"statfs":{"total":96636764160,"available":96594173952,"internally_reserved":0,"allocated":2146304,"data_stored":1364839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:10:51.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3f is not active+clean 2024-04-25T14:10:51.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.043709+0000', 'last_change': '2024-04-25T14:10:45.043709+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.043709+0000', 'last_undegraded': '2024-04-25T14:10:45.043709+0000', 'last_fullsized': '2024-04-25T14:10:45.043709+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:10:51.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3e is not active+clean 2024-04-25T14:10:51.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.034723+0000', 'last_change': '2024-04-25T14:10:45.034723+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.034723+0000', 'last_undegraded': '2024-04-25T14:10:45.034723+0000', 'last_fullsized': '2024-04-25T14:10:45.034723+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:10:51.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3d is not active+clean 2024-04-25T14:10:51.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.038065+0000', 'last_change': '2024-04-25T14:10:45.038065+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.038065+0000', 'last_undegraded': '2024-04-25T14:10:45.038065+0000', 'last_fullsized': '2024-04-25T14:10:45.038065+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:10:51.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3c is not active+clean 2024-04-25T14:10:51.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.032697+0000', 'last_change': '2024-04-25T14:10:45.032697+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.032697+0000', 'last_undegraded': '2024-04-25T14:10:45.032697+0000', 'last_fullsized': '2024-04-25T14:10:45.032697+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:10:51.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3b is not active+clean 2024-04-25T14:10:51.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.038541+0000', 'last_change': '2024-04-25T14:10:45.038541+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.038541+0000', 'last_undegraded': '2024-04-25T14:10:45.038541+0000', 'last_fullsized': '2024-04-25T14:10:45.038541+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:10:51.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3a is not active+clean 2024-04-25T14:10:51.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.045986+0000', 'last_change': '2024-04-25T14:10:45.045986+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.045986+0000', 'last_undegraded': '2024-04-25T14:10:45.045986+0000', 'last_fullsized': '2024-04-25T14:10:45.045986+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:10:51.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.39 is not active+clean 2024-04-25T14:10:51.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.037964+0000', 'last_change': '2024-04-25T14:10:45.037964+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.037964+0000', 'last_undegraded': '2024-04-25T14:10:45.037964+0000', 'last_fullsized': '2024-04-25T14:10:45.037964+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:10:51.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.38 is not active+clean 2024-04-25T14:10:51.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.043120+0000', 'last_change': '2024-04-25T14:10:45.043120+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.043120+0000', 'last_undegraded': '2024-04-25T14:10:45.043120+0000', 'last_fullsized': '2024-04-25T14:10:45.043120+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:10:51.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.37 is not active+clean 2024-04-25T14:10:51.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.049467+0000', 'last_change': '2024-04-25T14:10:45.049467+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.049467+0000', 'last_undegraded': '2024-04-25T14:10:45.049467+0000', 'last_fullsized': '2024-04-25T14:10:45.049467+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:10:51.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.36 is not active+clean 2024-04-25T14:10:51.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.039375+0000', 'last_change': '2024-04-25T14:10:45.039375+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.039375+0000', 'last_undegraded': '2024-04-25T14:10:45.039375+0000', 'last_fullsized': '2024-04-25T14:10:45.039375+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:10:51.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.35 is not active+clean 2024-04-25T14:10:51.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.034076+0000', 'last_change': '2024-04-25T14:10:45.034076+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.034076+0000', 'last_undegraded': '2024-04-25T14:10:45.034076+0000', 'last_fullsized': '2024-04-25T14:10:45.034076+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:10:51.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.34 is not active+clean 2024-04-25T14:10:51.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.041407+0000', 'last_change': '2024-04-25T14:10:45.041407+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.041407+0000', 'last_undegraded': '2024-04-25T14:10:45.041407+0000', 'last_fullsized': '2024-04-25T14:10:45.041407+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:10:51.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.33 is not active+clean 2024-04-25T14:10:51.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.047463+0000', 'last_change': '2024-04-25T14:10:45.047463+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.047463+0000', 'last_undegraded': '2024-04-25T14:10:45.047463+0000', 'last_fullsized': '2024-04-25T14:10:45.047463+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:10:51.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.32 is not active+clean 2024-04-25T14:10:51.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.040256+0000', 'last_change': '2024-04-25T14:10:45.040256+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.040256+0000', 'last_undegraded': '2024-04-25T14:10:45.040256+0000', 'last_fullsized': '2024-04-25T14:10:45.040256+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:10:51.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.31 is not active+clean 2024-04-25T14:10:51.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.040028+0000', 'last_change': '2024-04-25T14:10:45.040028+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.040028+0000', 'last_undegraded': '2024-04-25T14:10:45.040028+0000', 'last_fullsized': '2024-04-25T14:10:45.040028+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:10:51.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.30 is not active+clean 2024-04-25T14:10:51.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.045366+0000', 'last_change': '2024-04-25T14:10:45.045366+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.045366+0000', 'last_undegraded': '2024-04-25T14:10:45.045366+0000', 'last_fullsized': '2024-04-25T14:10:45.045366+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:10:51.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2f is not active+clean 2024-04-25T14:10:51.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.048856+0000', 'last_change': '2024-04-25T14:10:45.048856+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.048856+0000', 'last_undegraded': '2024-04-25T14:10:45.048856+0000', 'last_fullsized': '2024-04-25T14:10:45.048856+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:10:51.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2e is not active+clean 2024-04-25T14:10:51.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.040344+0000', 'last_change': '2024-04-25T14:10:45.040344+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.040344+0000', 'last_undegraded': '2024-04-25T14:10:45.040344+0000', 'last_fullsized': '2024-04-25T14:10:45.040344+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:10:51.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2d is not active+clean 2024-04-25T14:10:51.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.039190+0000', 'last_change': '2024-04-25T14:10:45.039190+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.039190+0000', 'last_undegraded': '2024-04-25T14:10:45.039190+0000', 'last_fullsized': '2024-04-25T14:10:45.039190+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:10:51.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2c is not active+clean 2024-04-25T14:10:51.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.038526+0000', 'last_change': '2024-04-25T14:10:45.038526+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.038526+0000', 'last_undegraded': '2024-04-25T14:10:45.038526+0000', 'last_fullsized': '2024-04-25T14:10:45.038526+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:10:51.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2b is not active+clean 2024-04-25T14:10:51.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.034790+0000', 'last_change': '2024-04-25T14:10:45.034790+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.034790+0000', 'last_undegraded': '2024-04-25T14:10:45.034790+0000', 'last_fullsized': '2024-04-25T14:10:45.034790+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:10:51.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2a is not active+clean 2024-04-25T14:10:51.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.040708+0000', 'last_change': '2024-04-25T14:10:45.040708+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.040708+0000', 'last_undegraded': '2024-04-25T14:10:45.040708+0000', 'last_fullsized': '2024-04-25T14:10:45.040708+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:10:51.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.29 is not active+clean 2024-04-25T14:10:51.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.032697+0000', 'last_change': '2024-04-25T14:10:45.032697+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.032697+0000', 'last_undegraded': '2024-04-25T14:10:45.032697+0000', 'last_fullsized': '2024-04-25T14:10:45.032697+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:10:51.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.28 is not active+clean 2024-04-25T14:10:51.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.044889+0000', 'last_change': '2024-04-25T14:10:45.044889+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.044889+0000', 'last_undegraded': '2024-04-25T14:10:45.044889+0000', 'last_fullsized': '2024-04-25T14:10:45.044889+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:10:51.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.27 is not active+clean 2024-04-25T14:10:51.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.039272+0000', 'last_change': '2024-04-25T14:10:45.039272+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.039272+0000', 'last_undegraded': '2024-04-25T14:10:45.039272+0000', 'last_fullsized': '2024-04-25T14:10:45.039272+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:10:51.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.26 is not active+clean 2024-04-25T14:10:51.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.038197+0000', 'last_change': '2024-04-25T14:10:45.038197+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.038197+0000', 'last_undegraded': '2024-04-25T14:10:45.038197+0000', 'last_fullsized': '2024-04-25T14:10:45.038197+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:10:51.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.25 is not active+clean 2024-04-25T14:10:51.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.038708+0000', 'last_change': '2024-04-25T14:10:45.038708+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.038708+0000', 'last_undegraded': '2024-04-25T14:10:45.038708+0000', 'last_fullsized': '2024-04-25T14:10:45.038708+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:10:51.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.24 is not active+clean 2024-04-25T14:10:51.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.039464+0000', 'last_change': '2024-04-25T14:10:45.039464+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.039464+0000', 'last_undegraded': '2024-04-25T14:10:45.039464+0000', 'last_fullsized': '2024-04-25T14:10:45.039464+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:10:51.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.23 is not active+clean 2024-04-25T14:10:51.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.040768+0000', 'last_change': '2024-04-25T14:10:45.040768+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.040768+0000', 'last_undegraded': '2024-04-25T14:10:45.040768+0000', 'last_fullsized': '2024-04-25T14:10:45.040768+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:10:51.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.22 is not active+clean 2024-04-25T14:10:51.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.038367+0000', 'last_change': '2024-04-25T14:10:45.038367+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.038367+0000', 'last_undegraded': '2024-04-25T14:10:45.038367+0000', 'last_fullsized': '2024-04-25T14:10:45.038367+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:10:51.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.21 is not active+clean 2024-04-25T14:10:51.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.044360+0000', 'last_change': '2024-04-25T14:10:45.044360+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.044360+0000', 'last_undegraded': '2024-04-25T14:10:45.044360+0000', 'last_fullsized': '2024-04-25T14:10:45.044360+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:10:51.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.20 is not active+clean 2024-04-25T14:10:51.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.039596+0000', 'last_change': '2024-04-25T14:10:45.039596+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.039596+0000', 'last_undegraded': '2024-04-25T14:10:45.039596+0000', 'last_fullsized': '2024-04-25T14:10:45.039596+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:10:51.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.18 is not active+clean 2024-04-25T14:10:51.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.035461+0000', 'last_change': '2024-04-25T14:10:45.035461+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.035461+0000', 'last_undegraded': '2024-04-25T14:10:45.035461+0000', 'last_fullsized': '2024-04-25T14:10:45.035461+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:10:51.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.19 is not active+clean 2024-04-25T14:10:51.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.039532+0000', 'last_change': '2024-04-25T14:10:45.039532+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.039532+0000', 'last_undegraded': '2024-04-25T14:10:45.039532+0000', 'last_fullsized': '2024-04-25T14:10:45.039532+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:10:51.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1a is not active+clean 2024-04-25T14:10:51.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.033422+0000', 'last_change': '2024-04-25T14:10:45.033422+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.033422+0000', 'last_undegraded': '2024-04-25T14:10:45.033422+0000', 'last_fullsized': '2024-04-25T14:10:45.033422+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:10:51.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1b is not active+clean 2024-04-25T14:10:51.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.043935+0000', 'last_change': '2024-04-25T14:10:45.043935+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.043935+0000', 'last_undegraded': '2024-04-25T14:10:45.043935+0000', 'last_fullsized': '2024-04-25T14:10:45.043935+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:10:51.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1c is not active+clean 2024-04-25T14:10:51.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.041991+0000', 'last_change': '2024-04-25T14:10:45.041991+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.041991+0000', 'last_undegraded': '2024-04-25T14:10:45.041991+0000', 'last_fullsized': '2024-04-25T14:10:45.041991+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:10:51.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1d is not active+clean 2024-04-25T14:10:51.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.040465+0000', 'last_change': '2024-04-25T14:10:45.040465+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.040465+0000', 'last_undegraded': '2024-04-25T14:10:45.040465+0000', 'last_fullsized': '2024-04-25T14:10:45.040465+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:10:51.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1e is not active+clean 2024-04-25T14:10:51.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.039895+0000', 'last_change': '2024-04-25T14:10:45.039895+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.039895+0000', 'last_undegraded': '2024-04-25T14:10:45.039895+0000', 'last_fullsized': '2024-04-25T14:10:45.039895+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:10:51.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1f is not active+clean 2024-04-25T14:10:51.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.040667+0000', 'last_change': '2024-04-25T14:10:45.040667+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.040667+0000', 'last_undegraded': '2024-04-25T14:10:45.040667+0000', 'last_fullsized': '2024-04-25T14:10:45.040667+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:10:51.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.10 is not active+clean 2024-04-25T14:10:51.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.039869+0000', 'last_change': '2024-04-25T14:10:45.039869+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.039869+0000', 'last_undegraded': '2024-04-25T14:10:45.039869+0000', 'last_fullsized': '2024-04-25T14:10:45.039869+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:10:51.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.11 is not active+clean 2024-04-25T14:10:51.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.046751+0000', 'last_change': '2024-04-25T14:10:45.046751+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.046751+0000', 'last_undegraded': '2024-04-25T14:10:45.046751+0000', 'last_fullsized': '2024-04-25T14:10:45.046751+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:10:51.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.12 is not active+clean 2024-04-25T14:10:51.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.037539+0000', 'last_change': '2024-04-25T14:10:45.037539+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.037539+0000', 'last_undegraded': '2024-04-25T14:10:45.037539+0000', 'last_fullsized': '2024-04-25T14:10:45.037539+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:10:51.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.13 is not active+clean 2024-04-25T14:10:51.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.040238+0000', 'last_change': '2024-04-25T14:10:45.040238+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.040238+0000', 'last_undegraded': '2024-04-25T14:10:45.040238+0000', 'last_fullsized': '2024-04-25T14:10:45.040238+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:10:51.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.14 is not active+clean 2024-04-25T14:10:51.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.038826+0000', 'last_change': '2024-04-25T14:10:45.038826+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.038826+0000', 'last_undegraded': '2024-04-25T14:10:45.038826+0000', 'last_fullsized': '2024-04-25T14:10:45.038826+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:10:51.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.16 is not active+clean 2024-04-25T14:10:51.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.039786+0000', 'last_change': '2024-04-25T14:10:45.039786+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.039786+0000', 'last_undegraded': '2024-04-25T14:10:45.039786+0000', 'last_fullsized': '2024-04-25T14:10:45.039786+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:10:51.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.17 is not active+clean 2024-04-25T14:10:51.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.037596+0000', 'last_change': '2024-04-25T14:10:45.037596+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.037596+0000', 'last_undegraded': '2024-04-25T14:10:45.037596+0000', 'last_fullsized': '2024-04-25T14:10:45.037596+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:10:51.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.8 is not active+clean 2024-04-25T14:10:51.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.038531+0000', 'last_change': '2024-04-25T14:10:45.038531+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.038531+0000', 'last_undegraded': '2024-04-25T14:10:45.038531+0000', 'last_fullsized': '2024-04-25T14:10:45.038531+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:10:51.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.9 is not active+clean 2024-04-25T14:10:51.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.037392+0000', 'last_change': '2024-04-25T14:10:45.037392+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.037392+0000', 'last_undegraded': '2024-04-25T14:10:45.037392+0000', 'last_fullsized': '2024-04-25T14:10:45.037392+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:10:51.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.a is not active+clean 2024-04-25T14:10:51.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.039058+0000', 'last_change': '2024-04-25T14:10:45.039058+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.039058+0000', 'last_undegraded': '2024-04-25T14:10:45.039058+0000', 'last_fullsized': '2024-04-25T14:10:45.039058+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:10:51.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.b is not active+clean 2024-04-25T14:10:51.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.037958+0000', 'last_change': '2024-04-25T14:10:45.037958+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.037958+0000', 'last_undegraded': '2024-04-25T14:10:45.037958+0000', 'last_fullsized': '2024-04-25T14:10:45.037958+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:10:51.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.c is not active+clean 2024-04-25T14:10:51.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.038038+0000', 'last_change': '2024-04-25T14:10:45.038038+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.038038+0000', 'last_undegraded': '2024-04-25T14:10:45.038038+0000', 'last_fullsized': '2024-04-25T14:10:45.038038+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:10:51.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.d is not active+clean 2024-04-25T14:10:51.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.045070+0000', 'last_change': '2024-04-25T14:10:45.045070+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.045070+0000', 'last_undegraded': '2024-04-25T14:10:45.045070+0000', 'last_fullsized': '2024-04-25T14:10:45.045070+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:10:51.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.e is not active+clean 2024-04-25T14:10:51.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.038103+0000', 'last_change': '2024-04-25T14:10:45.038103+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.038103+0000', 'last_undegraded': '2024-04-25T14:10:45.038103+0000', 'last_fullsized': '2024-04-25T14:10:45.038103+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:10:51.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.f is not active+clean 2024-04-25T14:10:51.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.033610+0000', 'last_change': '2024-04-25T14:10:45.033610+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.033610+0000', 'last_undegraded': '2024-04-25T14:10:45.033610+0000', 'last_fullsized': '2024-04-25T14:10:45.033610+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:10:51.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.0 is not active+clean 2024-04-25T14:10:51.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.037513+0000', 'last_change': '2024-04-25T14:10:45.037513+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.037513+0000', 'last_undegraded': '2024-04-25T14:10:45.037513+0000', 'last_fullsized': '2024-04-25T14:10:45.037513+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:10:51.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1 is not active+clean 2024-04-25T14:10:51.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.037754+0000', 'last_change': '2024-04-25T14:10:45.037754+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.037754+0000', 'last_undegraded': '2024-04-25T14:10:45.037754+0000', 'last_fullsized': '2024-04-25T14:10:45.037754+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:10:51.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2 is not active+clean 2024-04-25T14:10:51.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.039513+0000', 'last_change': '2024-04-25T14:10:45.039513+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.039513+0000', 'last_undegraded': '2024-04-25T14:10:45.039513+0000', 'last_fullsized': '2024-04-25T14:10:45.039513+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:10:51.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3 is not active+clean 2024-04-25T14:10:51.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.037351+0000', 'last_change': '2024-04-25T14:10:45.037351+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.037351+0000', 'last_undegraded': '2024-04-25T14:10:45.037351+0000', 'last_fullsized': '2024-04-25T14:10:45.037351+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:10:51.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.4 is not active+clean 2024-04-25T14:10:51.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.048890+0000', 'last_change': '2024-04-25T14:10:45.048890+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.048890+0000', 'last_undegraded': '2024-04-25T14:10:45.048890+0000', 'last_fullsized': '2024-04-25T14:10:45.048890+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:10:51.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.5 is not active+clean 2024-04-25T14:10:51.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.049911+0000', 'last_change': '2024-04-25T14:10:45.049911+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.049911+0000', 'last_undegraded': '2024-04-25T14:10:45.049911+0000', 'last_fullsized': '2024-04-25T14:10:45.049911+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:10:51.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.6 is not active+clean 2024-04-25T14:10:51.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.048918+0000', 'last_change': '2024-04-25T14:10:45.048918+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.048918+0000', 'last_undegraded': '2024-04-25T14:10:45.048918+0000', 'last_fullsized': '2024-04-25T14:10:45.048918+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:10:51.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.7 is not active+clean 2024-04-25T14:10:51.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.040213+0000', 'last_change': '2024-04-25T14:10:45.040213+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.040213+0000', 'last_undegraded': '2024-04-25T14:10:45.040213+0000', 'last_fullsized': '2024-04-25T14:10:45.040213+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:10:51.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.15 is not active+clean 2024-04-25T14:10:51.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 124, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:10:45.037519+0000', 'last_change': '2024-04-25T14:10:45.037519+0000', 'last_active': '2024-04-25T14:10:45.026666+0000', 'last_peered': '2024-04-25T14:10:45.026666+0000', 'last_clean': '2024-04-25T14:10:45.026666+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:10:45.037519+0000', 'last_undegraded': '2024-04-25T14:10:45.037519+0000', 'last_fullsized': '2024-04-25T14:10:45.037519+0000', 'mapping_epoch': 124, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 124, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'last_clean_scrub_stamp': '2024-04-25T14:10:45.026666+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:10:51.102 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T14:10:51.235 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:51.237+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:51.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.289+0000 7fd235600640 1 Processor -- start 2024-04-25T14:10:51.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.289+0000 7fd235600640 1 -- start start 2024-04-25T14:10:51.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.289+0000 7fd235600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd23006e190 0x7fd2301606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:51.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.289+0000 7fd235600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd2300d2e40 0x7fd2301679d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:51.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.289+0000 7fd235600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd230160ff0 con 0x7fd230160be0 2024-04-25T14:10:51.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.289+0000 7fd235600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd23006ef10 con 0x7fd2300d2e40 2024-04-25T14:10:51.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.289+0000 7fd235600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd2300b1fa0 con 0x7fd23006e190 2024-04-25T14:10:51.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.289+0000 7fd22f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd2300d2e40 0x7fd2301679d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:51.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.289+0000 7fd22ea00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd23006e190 0x7fd2301606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:51.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.289+0000 7fd22f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd2300d2e40 0x7fd2301679d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46214/0 (socket says 172.21.15.86:46214) 2024-04-25T14:10:51.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.289+0000 7fd22f400640 1 -- 172.21.15.86:0/3414247883 learned_addr learned my addr 172.21.15.86:0/3414247883 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:51.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.293+0000 7fd22f400640 1 -- 172.21.15.86:0/3414247883 >> v2:172.21.15.155:3300/0 conn(0x7fd23006e190 msgr2=0x7fd2301606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:51.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.293+0000 7fd22f400640 1 --2- 172.21.15.86:0/3414247883 >> v2:172.21.15.155:3300/0 conn(0x7fd23006e190 0x7fd2301606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:51.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.293+0000 7fd22f400640 1 -- 172.21.15.86:0/3414247883 >> v1:172.21.15.155:6789/0 conn(0x7fd230160be0 legacy=0x7fd2300d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:51.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.293+0000 7fd22f400640 1 -- 172.21.15.86:0/3414247883 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2300b2280 con 0x7fd2300d2e40 2024-04-25T14:10:51.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.293+0000 7fd22d600640 1 -- 172.21.15.86:0/3414247883 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 181746645 0 0) 0x7fd21c001120 con 0x7fd230160be0 2024-04-25T14:10:51.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.293+0000 7fd22f400640 1 --2- 172.21.15.86:0/3414247883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2300d2e40 0x7fd2301679d0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fd23006f680 tx=0x7fd22801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb40e18034297272 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:51.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.293+0000 7fd22d600640 1 -- 172.21.15.86:0/3414247883 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2280a56c0 con 0x7fd2300d2e40 2024-04-25T14:10:51.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.293+0000 7fd22d600640 1 -- 172.21.15.86:0/3414247883 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2280a5860 con 0x7fd2300d2e40 2024-04-25T14:10:51.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.293+0000 7fd22d600640 1 -- 172.21.15.86:0/3414247883 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2280a5c30 con 0x7fd2300d2e40 2024-04-25T14:10:51.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.297+0000 7fd235600640 1 -- 172.21.15.86:0/3414247883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2300d2e40 msgr2=0x7fd2301679d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:51.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.297+0000 7fd235600640 1 --2- 172.21.15.86:0/3414247883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2300d2e40 0x7fd2301679d0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fd23006f680 tx=0x7fd22801c9f0 comp rx=0 tx=0).stop 2024-04-25T14:10:51.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.297+0000 7fd235600640 1 -- 172.21.15.86:0/3414247883 shutdown_connections 2024-04-25T14:10:51.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.297+0000 7fd235600640 1 --2- 172.21.15.86:0/3414247883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2300d2e40 0x7fd2301679d0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:51.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.297+0000 7fd235600640 1 --2- 172.21.15.86:0/3414247883 >> v2:172.21.15.155:3300/0 conn(0x7fd23006e190 0x7fd2301606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:51.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.297+0000 7fd235600640 1 -- 172.21.15.86:0/3414247883 >> 172.21.15.86:0/3414247883 conn(0x7fd23009ba30 msgr2=0x7fd2300b7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:51.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.297+0000 7fd235600640 1 -- 172.21.15.86:0/3414247883 shutdown_connections 2024-04-25T14:10:51.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.297+0000 7fd235600640 1 -- 172.21.15.86:0/3414247883 wait complete. 2024-04-25T14:10:51.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.297+0000 7fd235600640 1 Processor -- start 2024-04-25T14:10:51.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.297+0000 7fd235600640 1 -- start start 2024-04-25T14:10:51.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.301+0000 7fd235600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd23006e190 0x7fd2301f9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:51.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.301+0000 7fd235600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2300d2e40 0x7fd2300ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:51.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.301+0000 7fd235600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd230160be0 0x7fd2300cf470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:51.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.301+0000 7fd235600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd23006eb30 con 0x7fd2300d2e40 2024-04-25T14:10:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.301+0000 7fd235600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd23015e8c0 con 0x7fd230160be0 2024-04-25T14:10:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.301+0000 7fd235600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd23006ecb0 con 0x7fd23006e190 2024-04-25T14:10:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.301+0000 7fd22e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2300d2e40 0x7fd2300ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.301+0000 7fd22e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2300d2e40 0x7fd2300ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46216/0 (socket says 172.21.15.86:46216) 2024-04-25T14:10:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.301+0000 7fd22e000640 1 -- 172.21.15.86:0/2478825726 learned_addr learned my addr 172.21.15.86:0/2478825726 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.301+0000 7fd22f400640 1 --2- 172.21.15.86:0/2478825726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd230160be0 0x7fd2300cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.301+0000 7fd22ea00640 1 --2- 172.21.15.86:0/2478825726 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd23006e190 0x7fd2301f9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.301+0000 7fd22e000640 1 -- 172.21.15.86:0/2478825726 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd23006e190 msgr2=0x7fd2301f9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.301+0000 7fd22e000640 1 --2- 172.21.15.86:0/2478825726 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd23006e190 0x7fd2301f9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.301+0000 7fd22e000640 1 -- 172.21.15.86:0/2478825726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd230160be0 msgr2=0x7fd2300cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.301+0000 7fd22e000640 1 --2- 172.21.15.86:0/2478825726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd230160be0 0x7fd2300cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.301+0000 7fd22e000640 1 -- 172.21.15.86:0/2478825726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2300ce8f0 con 0x7fd2300d2e40 2024-04-25T14:10:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.301+0000 7fd22e000640 1 --2- 172.21.15.86:0/2478825726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2300d2e40 0x7fd2300ce3b0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fd21c001ec0 tx=0x7fd21c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.301+0000 7fd226a00640 1 -- 172.21.15.86:0/2478825726 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c017070 con 0x7fd2300d2e40 2024-04-25T14:10:51.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.301+0000 7fd226a00640 1 -- 172.21.15.86:0/2478825726 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd21c003600 con 0x7fd2300d2e40 2024-04-25T14:10:51.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.301+0000 7fd226a00640 1 -- 172.21.15.86:0/2478825726 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c0039f0 con 0x7fd2300d2e40 2024-04-25T14:10:51.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.301+0000 7fd235600640 1 -- 172.21.15.86:0/2478825726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2301fa720 con 0x7fd2300d2e40 2024-04-25T14:10:51.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.301+0000 7fd235600640 1 -- 172.21.15.86:0/2478825726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2301face0 con 0x7fd2300d2e40 2024-04-25T14:10:51.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.301+0000 7fd235600640 1 -- 172.21.15.86:0/2478825726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2301fa480 con 0x7fd2300d2e40 2024-04-25T14:10:51.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.305+0000 7fd226a00640 1 -- 172.21.15.86:0/2478825726 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7fd21c01a070 con 0x7fd2300d2e40 2024-04-25T14:10:51.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.305+0000 7fd226a00640 1 --2- 172.21.15.86:0/2478825726 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fd21006f490 0x7fd2100719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:51.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.305+0000 7fd226a00640 1 -- 172.21.15.86:0/2478825726 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7fd21c08e960 con 0x7fd2300d2e40 2024-04-25T14:10:51.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.305+0000 7fd22ea00640 1 --2- 172.21.15.86:0/2478825726 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fd21006f490 0x7fd2100719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:51.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.305+0000 7fd22ea00640 1 --2- 172.21.15.86:0/2478825726 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fd21006f490 0x7fd2100719a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd23015e260 tx=0x7fd220008040 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:51.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.309+0000 7fd226a00640 1 -- 172.21.15.86:0/2478825726 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd2301fa480 con 0x7fd2300d2e40 2024-04-25T14:10:51.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.545+0000 7fd235600640 1 -- 172.21.15.86:0/2478825726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fd2300d1e50 con 0x7fd2300d2e40 2024-04-25T14:10:51.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.545+0000 7fd226a00640 1 -- 172.21.15.86:0/2478825726 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1098 (secure 0 0 0) 0x7fd230160ff0 con 0x7fd2300d2e40 2024-04-25T14:10:51.541 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:51.541 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":445,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":126,"num_osds":8,"num_up_osds":8,"osd_up_since":1714053818,"num_in_osds":8,"osd_in_since":1714053808,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":286031872,"bytes_avail":772808081408,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085,"read_bytes_sec":1099,"read_op_per_sec":1,"write_op_per_sec":0},"fsmap":{"epoch":94,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"b","status":"up:active","gid":8991}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-04-25T14:08:15.095679+0000","services":{}},"progress_events":{}} 2024-04-25T14:10:51.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.545+0000 7fd235600640 1 -- 172.21.15.86:0/2478825726 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fd21006f490 msgr2=0x7fd2100719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:51.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.545+0000 7fd235600640 1 --2- 172.21.15.86:0/2478825726 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fd21006f490 0x7fd2100719a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd23015e260 tx=0x7fd220008040 comp rx=0 tx=0).stop 2024-04-25T14:10:51.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.545+0000 7fd235600640 1 -- 172.21.15.86:0/2478825726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2300d2e40 msgr2=0x7fd2300ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:51.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.545+0000 7fd235600640 1 --2- 172.21.15.86:0/2478825726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2300d2e40 0x7fd2300ce3b0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fd21c001ec0 tx=0x7fd21c00c8e0 comp rx=0 tx=0).stop 2024-04-25T14:10:51.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.545+0000 7fd235600640 1 -- 172.21.15.86:0/2478825726 shutdown_connections 2024-04-25T14:10:51.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.545+0000 7fd235600640 1 --2- 172.21.15.86:0/2478825726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd230160be0 0x7fd2300cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:51.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.545+0000 7fd235600640 1 --2- 172.21.15.86:0/2478825726 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fd21006f490 0x7fd2100719a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:51.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.545+0000 7fd235600640 1 --2- 172.21.15.86:0/2478825726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2300d2e40 0x7fd2300ce3b0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:51.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.545+0000 7fd235600640 1 --2- 172.21.15.86:0/2478825726 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd23006e190 0x7fd2301f9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:51.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.545+0000 7fd235600640 1 -- 172.21.15.86:0/2478825726 >> 172.21.15.86:0/2478825726 conn(0x7fd23009ba30 msgr2=0x7fd2300d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:51.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.545+0000 7fd235600640 1 -- 172.21.15.86:0/2478825726 shutdown_connections 2024-04-25T14:10:51.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.545+0000 7fd235600640 1 -- 172.21.15.86:0/2478825726 wait complete. 2024-04-25T14:10:51.564 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T14:10:51.564 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:10:51.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.721+0000 7f0c30a00640 1 Processor -- start 2024-04-25T14:10:51.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.721+0000 7f0c30a00640 1 -- start start 2024-04-25T14:10:51.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.721+0000 7f0c30a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2c0d1df0 0x7f0c2c0cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:51.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.721+0000 7f0c30a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c2c15e5c0 con 0x7f0c2c070330 2024-04-25T14:10:51.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.721+0000 7f0c30a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c2c15e8a0 con 0x7f0c2c0cdcc0 2024-04-25T14:10:51.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.721+0000 7f0c30a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0c2c06fd40 con 0x7f0c2c0d1df0 2024-04-25T14:10:51.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.721+0000 7f0c29e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0c2c070330 0x7f0c2c0d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54962/0 (socket says 172.21.15.86:54962) 2024-04-25T14:10:51.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.725+0000 7f0c29e00640 1 -- 172.21.15.86:0/3674763596 learned_addr learned my addr 172.21.15.86:0/3674763596 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:51.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.725+0000 7f0c23e00640 1 --2- 172.21.15.86:0/3674763596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2c0d1df0 0x7f0c2c0cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:51.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.725+0000 7f0c29400640 1 -- 172.21.15.86:0/3674763596 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1112666629 0 0) 0x7f0c2c15e5c0 con 0x7f0c2c070330 2024-04-25T14:10:51.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.725+0000 7f0c29400640 1 -- 172.21.15.86:0/3674763596 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c10003530 con 0x7f0c2c070330 2024-04-25T14:10:51.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.725+0000 7f0c29400640 1 -- 172.21.15.86:0/3674763596 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3076684407 0 0) 0x7f0c2c15e8a0 con 0x7f0c2c0cdcc0 2024-04-25T14:10:51.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.725+0000 7f0c29400640 1 -- 172.21.15.86:0/3674763596 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c2c15e5c0 con 0x7f0c2c0cdcc0 2024-04-25T14:10:51.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.725+0000 7f0c23e00640 1 -- 172.21.15.86:0/3674763596 >> v1:172.21.15.155:6789/0 conn(0x7f0c2c0cdcc0 legacy=0x7f0c2c0ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:51.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.725+0000 7f0c23e00640 1 -- 172.21.15.86:0/3674763596 >> v1:172.21.15.86:6789/0 conn(0x7f0c2c070330 legacy=0x7f0c2c0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:51.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.725+0000 7f0c23e00640 1 -- 172.21.15.86:0/3674763596 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2c15eb80 con 0x7f0c2c0d1df0 2024-04-25T14:10:51.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.729+0000 7f0c23e00640 1 --2- 172.21.15.86:0/3674763596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2c0d1df0 0x7f0c2c0cd780 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0c140097f0 tx=0x7f0c1401c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c0bd88e4b3df386 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:51.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.729+0000 7f0c29400640 1 -- 172.21.15.86:0/3674763596 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c140b0070 con 0x7f0c2c0d1df0 2024-04-25T14:10:51.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.729+0000 7f0c29400640 1 -- 172.21.15.86:0/3674763596 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c140ab600 con 0x7f0c2c0d1df0 2024-04-25T14:10:51.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.729+0000 7f0c29400640 1 -- 172.21.15.86:0/3674763596 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c140ab9b0 con 0x7f0c2c0d1df0 2024-04-25T14:10:51.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.729+0000 7f0c30a00640 1 -- 172.21.15.86:0/3674763596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2c0d1df0 msgr2=0x7f0c2c0cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:51.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.729+0000 7f0c30a00640 1 --2- 172.21.15.86:0/3674763596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2c0d1df0 0x7f0c2c0cd780 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0c140097f0 tx=0x7f0c1401c420 comp rx=0 tx=0).stop 2024-04-25T14:10:51.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.729+0000 7f0c30a00640 1 -- 172.21.15.86:0/3674763596 shutdown_connections 2024-04-25T14:10:51.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.729+0000 7f0c30a00640 1 --2- 172.21.15.86:0/3674763596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2c0d1df0 0x7f0c2c0cd780 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:51.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.729+0000 7f0c30a00640 1 -- 172.21.15.86:0/3674763596 >> 172.21.15.86:0/3674763596 conn(0x7f0c2c0b4650 msgr2=0x7f0c2c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:51.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.729+0000 7f0c30a00640 1 -- 172.21.15.86:0/3674763596 shutdown_connections 2024-04-25T14:10:51.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.729+0000 7f0c30a00640 1 -- 172.21.15.86:0/3674763596 wait complete. 2024-04-25T14:10:51.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.729+0000 7f0c30a00640 1 Processor -- start 2024-04-25T14:10:51.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.729+0000 7f0c30a00640 1 -- start start 2024-04-25T14:10:51.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.733+0000 7f0c30a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c2c070330 0x7f0c2c09eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:51.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.733+0000 7f0c30a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2c0cdcc0 0x7f0c2c0d0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:51.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.733+0000 7f0c30a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c2c156fb0 0x7f0c2c0d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:51.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.733+0000 7f0c30a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c0708c0 con 0x7f0c2c070330 2024-04-25T14:10:51.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.733+0000 7f0c30a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c070740 con 0x7f0c2c156fb0 2024-04-25T14:10:51.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.733+0000 7f0c30a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0c2c16bfd0 con 0x7f0c2c0cdcc0 2024-04-25T14:10:51.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.733+0000 7f0c29e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c2c070330 0x7f0c2c09eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:51.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.733+0000 7f0c29e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c2c070330 0x7f0c2c09eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46228/0 (socket says 172.21.15.86:46228) 2024-04-25T14:10:51.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.733+0000 7f0c29e00640 1 -- 172.21.15.86:0/2119835395 learned_addr learned my addr 172.21.15.86:0/2119835395 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:51.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.733+0000 7f0c23e00640 1 --2- 172.21.15.86:0/2119835395 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2c0cdcc0 0x7f0c2c0d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:51.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.733+0000 7f0c2a800640 1 --2- 172.21.15.86:0/2119835395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c2c156fb0 0x7f0c2c0d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:51.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.733+0000 7f0c29e00640 1 -- 172.21.15.86:0/2119835395 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2c0cdcc0 msgr2=0x7f0c2c0d0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:51.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.733+0000 7f0c29e00640 1 --2- 172.21.15.86:0/2119835395 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2c0cdcc0 0x7f0c2c0d0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:51.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.733+0000 7f0c29e00640 1 -- 172.21.15.86:0/2119835395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c2c156fb0 msgr2=0x7f0c2c0d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:51.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.733+0000 7f0c29e00640 1 --2- 172.21.15.86:0/2119835395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c2c156fb0 0x7f0c2c0d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:51.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.733+0000 7f0c29e00640 1 -- 172.21.15.86:0/2119835395 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2c09f070 con 0x7f0c2c070330 2024-04-25T14:10:51.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.733+0000 7f0c23e00640 1 --2- 172.21.15.86:0/2119835395 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2c0cdcc0 0x7f0c2c0d0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:10:51.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.733+0000 7f0c29e00640 1 --2- 172.21.15.86:0/2119835395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c2c070330 0x7f0c2c09eb30 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00c420 tx=0x7f0c1c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:51.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.733+0000 7f0c22000640 1 -- 172.21.15.86:0/2119835395 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c017070 con 0x7f0c2c070330 2024-04-25T14:10:51.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.733+0000 7f0c22000640 1 -- 172.21.15.86:0/2119835395 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c1c005cf0 con 0x7f0c2c070330 2024-04-25T14:10:51.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.733+0000 7f0c30a00640 1 -- 172.21.15.86:0/2119835395 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c2c15aa00 con 0x7f0c2c070330 2024-04-25T14:10:51.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.733+0000 7f0c22000640 1 -- 172.21.15.86:0/2119835395 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0072d0 con 0x7f0c2c070330 2024-04-25T14:10:51.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.733+0000 7f0c30a00640 1 -- 172.21.15.86:0/2119835395 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c2c0c9220 con 0x7f0c2c070330 2024-04-25T14:10:51.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.733+0000 7f0c30a00640 1 -- 172.21.15.86:0/2119835395 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c2c069d00 con 0x7f0c2c070330 2024-04-25T14:10:51.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.737+0000 7f0c22000640 1 -- 172.21.15.86:0/2119835395 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7f0c1c01a070 con 0x7f0c2c070330 2024-04-25T14:10:51.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.737+0000 7f0c22000640 1 --2- 172.21.15.86:0/2119835395 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f0c0806f490 0x7f0c080719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:51.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.737+0000 7f0c22000640 1 -- 172.21.15.86:0/2119835395 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7f0c1c08e760 con 0x7f0c2c070330 2024-04-25T14:10:51.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.737+0000 7f0c23e00640 1 --2- 172.21.15.86:0/2119835395 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f0c0806f490 0x7f0c080719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:51.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.737+0000 7f0c23e00640 1 --2- 172.21.15.86:0/2119835395 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f0c0806f490 0x7f0c080719a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0c140ad000 tx=0x7f0c140a6000 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.741+0000 7f0c22000640 1 -- 172.21.15.86:0/2119835395 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0c2c09f070 con 0x7f0c2c070330 2024-04-25T14:10:51.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.909+0000 7f0c30a00640 1 -- 172.21.15.86:0/2119835395 --> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0c2c0742a0 con 0x7f0c0806f490 2024-04-25T14:10:51.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.917+0000 7f0c22000640 1 -- 172.21.15.86:0/2119835395 <== mgr.8338 v2:172.21.15.86:6832/3964666460 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252481 (secure 0 0 0) 0x7f0c2c0742a0 con 0x7f0c0806f490 2024-04-25T14:10:51.915 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:51.917 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:10:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.917+0000 7f0c30a00640 1 -- 172.21.15.86:0/2119835395 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f0c0806f490 msgr2=0x7f0c080719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.917+0000 7f0c30a00640 1 --2- 172.21.15.86:0/2119835395 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f0c0806f490 0x7f0c080719a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0c140ad000 tx=0x7f0c140a6000 comp rx=0 tx=0).stop 2024-04-25T14:10:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.917+0000 7f0c30a00640 1 -- 172.21.15.86:0/2119835395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c2c070330 msgr2=0x7f0c2c09eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.917+0000 7f0c30a00640 1 --2- 172.21.15.86:0/2119835395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c2c070330 0x7f0c2c09eb30 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00c420 tx=0x7f0c1c00c8e0 comp rx=0 tx=0).stop 2024-04-25T14:10:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.917+0000 7f0c30a00640 1 -- 172.21.15.86:0/2119835395 shutdown_connections 2024-04-25T14:10:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.917+0000 7f0c30a00640 1 --2- 172.21.15.86:0/2119835395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c2c156fb0 0x7f0c2c0d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.917+0000 7f0c30a00640 1 --2- 172.21.15.86:0/2119835395 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2c0cdcc0 0x7f0c2c0d0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.917+0000 7f0c30a00640 1 --2- 172.21.15.86:0/2119835395 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f0c0806f490 0x7f0c080719a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.917+0000 7f0c30a00640 1 --2- 172.21.15.86:0/2119835395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c2c070330 0x7f0c2c09eb30 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.917+0000 7f0c30a00640 1 -- 172.21.15.86:0/2119835395 >> 172.21.15.86:0/2119835395 conn(0x7f0c2c0b4650 msgr2=0x7f0c2c072ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.921+0000 7f0c30a00640 1 -- 172.21.15.86:0/2119835395 shutdown_connections 2024-04-25T14:10:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:51.921+0000 7f0c30a00640 1 -- 172.21.15.86:0/2119835395 wait complete. 2024-04-25T14:10:51.938 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-04-25T14:10:51.615953+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":536,"num_read_kb":1396,"num_write":232,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2688,"ondisk_log_size":2688,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279616,"kb_used_data":13544,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754695104,"statfs":{"total":773094113280,"available":772807786496,"internally_reserved":0,"allocated":13869056,"data_stored":7492196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.585687"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.203013+0000","last_change":"2024-04-25T14:10:46.052795+0000","last_active":"2024-04-25T14:10:47.203013+0000","last_peered":"2024-04-25T14:10:47.203013+0000","last_clean":"2024-04-25T14:10:47.203013+0000","last_became_active":"2024-04-25T14:10:46.052501+0000","last_became_peered":"2024-04-25T14:10:46.052501+0000","last_unstale":"2024-04-25T14:10:47.203013+0000","last_undegraded":"2024-04-25T14:10:47.203013+0000","last_fullsized":"2024-04-25T14:10:47.203013+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:57.384121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.070065+0000","last_change":"2024-04-25T14:10:46.062961+0000","last_active":"2024-04-25T14:10:47.070065+0000","last_peered":"2024-04-25T14:10:47.070065+0000","last_clean":"2024-04-25T14:10:47.070065+0000","last_became_active":"2024-04-25T14:10:46.062564+0000","last_became_peered":"2024-04-25T14:10:46.062564+0000","last_unstale":"2024-04-25T14:10:47.070065+0000","last_undegraded":"2024-04-25T14:10:47.070065+0000","last_fullsized":"2024-04-25T14:10:47.070065+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:48.259748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.061394+0000","last_change":"2024-04-25T14:10:46.070470+0000","last_active":"2024-04-25T14:10:47.061394+0000","last_peered":"2024-04-25T14:10:47.061394+0000","last_clean":"2024-04-25T14:10:47.061394+0000","last_became_active":"2024-04-25T14:10:46.070060+0000","last_became_peered":"2024-04-25T14:10:46.070060+0000","last_unstale":"2024-04-25T14:10:47.061394+0000","last_undegraded":"2024-04-25T14:10:47.061394+0000","last_fullsized":"2024-04-25T14:10:47.061394+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:35.902293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.069855+0000","last_change":"2024-04-25T14:10:46.052043+0000","last_active":"2024-04-25T14:10:47.069855+0000","last_peered":"2024-04-25T14:10:47.069855+0000","last_clean":"2024-04-25T14:10:47.069855+0000","last_became_active":"2024-04-25T14:10:46.051748+0000","last_became_peered":"2024-04-25T14:10:46.051748+0000","last_unstale":"2024-04-25T14:10:47.069855+0000","last_undegraded":"2024-04-25T14:10:47.069855+0000","last_fullsized":"2024-04-25T14:10:47.069855+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:13.569540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.059645+0000","last_change":"2024-04-25T14:10:46.061616+0000","last_active":"2024-04-25T14:10:47.059645+0000","last_peered":"2024-04-25T14:10:47.059645+0000","last_clean":"2024-04-25T14:10:47.059645+0000","last_became_active":"2024-04-25T14:10:46.061296+0000","last_became_peered":"2024-04-25T14:10:46.061296+0000","last_unstale":"2024-04-25T14:10:47.059645+0000","last_undegraded":"2024-04-25T14:10:47.059645+0000","last_fullsized":"2024-04-25T14:10:47.059645+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:06.636965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.203067+0000","last_change":"2024-04-25T14:10:46.073895+0000","last_active":"2024-04-25T14:10:47.203067+0000","last_peered":"2024-04-25T14:10:47.203067+0000","last_clean":"2024-04-25T14:10:47.203067+0000","last_became_active":"2024-04-25T14:10:46.072233+0000","last_became_peered":"2024-04-25T14:10:46.072233+0000","last_unstale":"2024-04-25T14:10:47.203067+0000","last_undegraded":"2024-04-25T14:10:47.203067+0000","last_fullsized":"2024-04-25T14:10:47.203067+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:15.160432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060297+0000","last_change":"2024-04-25T14:10:46.074066+0000","last_active":"2024-04-25T14:10:47.060297+0000","last_peered":"2024-04-25T14:10:47.060297+0000","last_clean":"2024-04-25T14:10:47.060297+0000","last_became_active":"2024-04-25T14:10:46.073745+0000","last_became_peered":"2024-04-25T14:10:46.073745+0000","last_unstale":"2024-04-25T14:10:47.060297+0000","last_undegraded":"2024-04-25T14:10:47.060297+0000","last_fullsized":"2024-04-25T14:10:47.060297+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:57.190018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.067458+0000","last_change":"2024-04-25T14:10:46.072612+0000","last_active":"2024-04-25T14:10:47.067458+0000","last_peered":"2024-04-25T14:10:47.067458+0000","last_clean":"2024-04-25T14:10:47.067458+0000","last_became_active":"2024-04-25T14:10:46.072263+0000","last_became_peered":"2024-04-25T14:10:46.072263+0000","last_unstale":"2024-04-25T14:10:47.067458+0000","last_undegraded":"2024-04-25T14:10:47.067458+0000","last_fullsized":"2024-04-25T14:10:47.067458+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:21.296998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202939+0000","last_change":"2024-04-25T14:10:46.077239+0000","last_active":"2024-04-25T14:10:47.202939+0000","last_peered":"2024-04-25T14:10:47.202939+0000","last_clean":"2024-04-25T14:10:47.202939+0000","last_became_active":"2024-04-25T14:10:46.076966+0000","last_became_peered":"2024-04-25T14:10:46.076966+0000","last_unstale":"2024-04-25T14:10:47.202939+0000","last_undegraded":"2024-04-25T14:10:47.202939+0000","last_fullsized":"2024-04-25T14:10:47.202939+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:06.636965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.065101+0000","last_change":"2024-04-25T14:10:46.059381+0000","last_active":"2024-04-25T14:10:47.065101+0000","last_peered":"2024-04-25T14:10:47.065101+0000","last_clean":"2024-04-25T14:10:47.065101+0000","last_became_active":"2024-04-25T14:10:46.059060+0000","last_became_peered":"2024-04-25T14:10:46.059060+0000","last_unstale":"2024-04-25T14:10:47.065101+0000","last_undegraded":"2024-04-25T14:10:47.065101+0000","last_fullsized":"2024-04-25T14:10:47.065101+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:23.703727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.069934+0000","last_change":"2024-04-25T14:10:46.070967+0000","last_active":"2024-04-25T14:10:47.069934+0000","last_peered":"2024-04-25T14:10:47.069934+0000","last_clean":"2024-04-25T14:10:47.069934+0000","last_became_active":"2024-04-25T14:10:46.070649+0000","last_became_peered":"2024-04-25T14:10:46.070649+0000","last_unstale":"2024-04-25T14:10:47.069934+0000","last_undegraded":"2024-04-25T14:10:47.069934+0000","last_fullsized":"2024-04-25T14:10:47.069934+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:04.502348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202897+0000","last_change":"2024-04-25T14:10:46.062312+0000","last_active":"2024-04-25T14:10:47.202897+0000","last_peered":"2024-04-25T14:10:47.202897+0000","last_clean":"2024-04-25T14:10:47.202897+0000","last_became_active":"2024-04-25T14:10:46.062040+0000","last_became_peered":"2024-04-25T14:10:46.062040+0000","last_unstale":"2024-04-25T14:10:47.202897+0000","last_undegraded":"2024-04-25T14:10:47.202897+0000","last_fullsized":"2024-04-25T14:10:47.202897+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:07.483995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202667+0000","last_change":"2024-04-25T14:10:46.070884+0000","last_active":"2024-04-25T14:10:47.202667+0000","last_peered":"2024-04-25T14:10:47.202667+0000","last_clean":"2024-04-25T14:10:47.202667+0000","last_became_active":"2024-04-25T14:10:46.070638+0000","last_became_peered":"2024-04-25T14:10:46.070638+0000","last_unstale":"2024-04-25T14:10:47.202667+0000","last_undegraded":"2024-04-25T14:10:47.202667+0000","last_fullsized":"2024-04-25T14:10:47.202667+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:37.900506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.201995+0000","last_change":"2024-04-25T14:10:46.060384+0000","last_active":"2024-04-25T14:10:47.201995+0000","last_peered":"2024-04-25T14:10:47.201995+0000","last_clean":"2024-04-25T14:10:47.201995+0000","last_became_active":"2024-04-25T14:10:46.058137+0000","last_became_peered":"2024-04-25T14:10:46.058137+0000","last_unstale":"2024-04-25T14:10:47.201995+0000","last_undegraded":"2024-04-25T14:10:47.201995+0000","last_fullsized":"2024-04-25T14:10:47.201995+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:53.079759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.201769+0000","last_change":"2024-04-25T14:10:46.059716+0000","last_active":"2024-04-25T14:10:47.201769+0000","last_peered":"2024-04-25T14:10:47.201769+0000","last_clean":"2024-04-25T14:10:47.201769+0000","last_became_active":"2024-04-25T14:10:46.059418+0000","last_became_peered":"2024-04-25T14:10:46.059418+0000","last_unstale":"2024-04-25T14:10:47.201769+0000","last_undegraded":"2024-04-25T14:10:47.201769+0000","last_fullsized":"2024-04-25T14:10:47.201769+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:50.496026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.201912+0000","last_change":"2024-04-25T14:10:46.064411+0000","last_active":"2024-04-25T14:10:47.201912+0000","last_peered":"2024-04-25T14:10:47.201912+0000","last_clean":"2024-04-25T14:10:47.201912+0000","last_became_active":"2024-04-25T14:10:46.064098+0000","last_became_peered":"2024-04-25T14:10:46.064098+0000","last_unstale":"2024-04-25T14:10:47.201912+0000","last_undegraded":"2024-04-25T14:10:47.201912+0000","last_fullsized":"2024-04-25T14:10:47.201912+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:04:17.096454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.203099+0000","last_change":"2024-04-25T14:10:46.064728+0000","last_active":"2024-04-25T14:10:47.203099+0000","last_peered":"2024-04-25T14:10:47.203099+0000","last_clean":"2024-04-25T14:10:47.203099+0000","last_became_active":"2024-04-25T14:10:46.064402+0000","last_became_peered":"2024-04-25T14:10:46.064402+0000","last_unstale":"2024-04-25T14:10:47.203099+0000","last_undegraded":"2024-04-25T14:10:47.203099+0000","last_fullsized":"2024-04-25T14:10:47.203099+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:09.225406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.059826+0000","last_change":"2024-04-25T14:10:46.061910+0000","last_active":"2024-04-25T14:10:47.059826+0000","last_peered":"2024-04-25T14:10:47.059826+0000","last_clean":"2024-04-25T14:10:47.059826+0000","last_became_active":"2024-04-25T14:10:46.061626+0000","last_became_peered":"2024-04-25T14:10:46.061626+0000","last_unstale":"2024-04-25T14:10:47.059826+0000","last_undegraded":"2024-04-25T14:10:47.059826+0000","last_fullsized":"2024-04-25T14:10:47.059826+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:09.225406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060583+0000","last_change":"2024-04-25T14:10:46.078014+0000","last_active":"2024-04-25T14:10:47.060583+0000","last_peered":"2024-04-25T14:10:47.060583+0000","last_clean":"2024-04-25T14:10:47.060583+0000","last_became_active":"2024-04-25T14:10:46.073655+0000","last_became_peered":"2024-04-25T14:10:46.073655+0000","last_unstale":"2024-04-25T14:10:47.060583+0000","last_undegraded":"2024-04-25T14:10:47.060583+0000","last_fullsized":"2024-04-25T14:10:47.060583+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:06.636965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.059927+0000","last_change":"2024-04-25T14:10:46.075501+0000","last_active":"2024-04-25T14:10:47.059927+0000","last_peered":"2024-04-25T14:10:47.059927+0000","last_clean":"2024-04-25T14:10:47.059927+0000","last_became_active":"2024-04-25T14:10:46.075206+0000","last_became_peered":"2024-04-25T14:10:46.075206+0000","last_unstale":"2024-04-25T14:10:47.059927+0000","last_undegraded":"2024-04-25T14:10:47.059927+0000","last_fullsized":"2024-04-25T14:10:47.059927+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:38:21.375099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.069818+0000","last_change":"2024-04-25T14:10:46.069371+0000","last_active":"2024-04-25T14:10:47.069818+0000","last_peered":"2024-04-25T14:10:47.069818+0000","last_clean":"2024-04-25T14:10:47.069818+0000","last_became_active":"2024-04-25T14:10:46.068924+0000","last_became_peered":"2024-04-25T14:10:46.068924+0000","last_unstale":"2024-04-25T14:10:47.069818+0000","last_undegraded":"2024-04-25T14:10:47.069818+0000","last_fullsized":"2024-04-25T14:10:47.069818+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:56.449255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.201898+0000","last_change":"2024-04-25T14:10:46.051588+0000","last_active":"2024-04-25T14:10:47.201898+0000","last_peered":"2024-04-25T14:10:47.201898+0000","last_clean":"2024-04-25T14:10:47.201898+0000","last_became_active":"2024-04-25T14:10:46.051301+0000","last_became_peered":"2024-04-25T14:10:46.051301+0000","last_unstale":"2024-04-25T14:10:47.201898+0000","last_undegraded":"2024-04-25T14:10:47.201898+0000","last_fullsized":"2024-04-25T14:10:47.201898+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:47.023113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.069796+0000","last_change":"2024-04-25T14:10:46.063747+0000","last_active":"2024-04-25T14:10:47.069796+0000","last_peered":"2024-04-25T14:10:47.069796+0000","last_clean":"2024-04-25T14:10:47.069796+0000","last_became_active":"2024-04-25T14:10:46.063416+0000","last_became_peered":"2024-04-25T14:10:46.063416+0000","last_unstale":"2024-04-25T14:10:47.069796+0000","last_undegraded":"2024-04-25T14:10:47.069796+0000","last_fullsized":"2024-04-25T14:10:47.069796+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:21.296998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202071+0000","last_change":"2024-04-25T14:10:46.076309+0000","last_active":"2024-04-25T14:10:47.202071+0000","last_peered":"2024-04-25T14:10:47.202071+0000","last_clean":"2024-04-25T14:10:47.202071+0000","last_became_active":"2024-04-25T14:10:46.076027+0000","last_became_peered":"2024-04-25T14:10:46.076027+0000","last_unstale":"2024-04-25T14:10:47.202071+0000","last_undegraded":"2024-04-25T14:10:47.202071+0000","last_fullsized":"2024-04-25T14:10:47.202071+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:36.059775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.062071+0000","last_change":"2024-04-25T14:10:46.073491+0000","last_active":"2024-04-25T14:10:47.062071+0000","last_peered":"2024-04-25T14:10:47.062071+0000","last_clean":"2024-04-25T14:10:47.062071+0000","last_became_active":"2024-04-25T14:10:46.073143+0000","last_became_peered":"2024-04-25T14:10:46.073143+0000","last_unstale":"2024-04-25T14:10:47.062071+0000","last_undegraded":"2024-04-25T14:10:47.062071+0000","last_fullsized":"2024-04-25T14:10:47.062071+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:37.995319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.061886+0000","last_change":"2024-04-25T14:10:46.070326+0000","last_active":"2024-04-25T14:10:47.061886+0000","last_peered":"2024-04-25T14:10:47.061886+0000","last_clean":"2024-04-25T14:10:47.061886+0000","last_became_active":"2024-04-25T14:10:46.069774+0000","last_became_peered":"2024-04-25T14:10:46.069774+0000","last_unstale":"2024-04-25T14:10:47.061886+0000","last_undegraded":"2024-04-25T14:10:47.061886+0000","last_fullsized":"2024-04-25T14:10:47.061886+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:25.544185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.061109+0000","last_change":"2024-04-25T14:10:46.067155+0000","last_active":"2024-04-25T14:10:47.061109+0000","last_peered":"2024-04-25T14:10:47.061109+0000","last_clean":"2024-04-25T14:10:47.061109+0000","last_became_active":"2024-04-25T14:10:46.066844+0000","last_became_peered":"2024-04-25T14:10:46.066844+0000","last_unstale":"2024-04-25T14:10:47.061109+0000","last_undegraded":"2024-04-25T14:10:47.061109+0000","last_fullsized":"2024-04-25T14:10:47.061109+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:17.255415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.065139+0000","last_change":"2024-04-25T14:10:46.059739+0000","last_active":"2024-04-25T14:10:47.065139+0000","last_peered":"2024-04-25T14:10:47.065139+0000","last_clean":"2024-04-25T14:10:47.065139+0000","last_became_active":"2024-04-25T14:10:46.059042+0000","last_became_peered":"2024-04-25T14:10:46.059042+0000","last_unstale":"2024-04-25T14:10:47.065139+0000","last_undegraded":"2024-04-25T14:10:47.065139+0000","last_fullsized":"2024-04-25T14:10:47.065139+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:53.254490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.061419+0000","last_change":"2024-04-25T14:10:46.066163+0000","last_active":"2024-04-25T14:10:47.061419+0000","last_peered":"2024-04-25T14:10:47.061419+0000","last_clean":"2024-04-25T14:10:47.061419+0000","last_became_active":"2024-04-25T14:10:46.065719+0000","last_became_peered":"2024-04-25T14:10:46.065719+0000","last_unstale":"2024-04-25T14:10:47.061419+0000","last_undegraded":"2024-04-25T14:10:47.061419+0000","last_fullsized":"2024-04-25T14:10:47.061419+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:04.502348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.061753+0000","last_change":"2024-04-25T14:10:46.067769+0000","last_active":"2024-04-25T14:10:47.061753+0000","last_peered":"2024-04-25T14:10:47.061753+0000","last_clean":"2024-04-25T14:10:47.061753+0000","last_became_active":"2024-04-25T14:10:46.067441+0000","last_became_peered":"2024-04-25T14:10:46.067441+0000","last_unstale":"2024-04-25T14:10:47.061753+0000","last_undegraded":"2024-04-25T14:10:47.061753+0000","last_fullsized":"2024-04-25T14:10:47.061753+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:44:18.230978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.067634+0000","last_change":"2024-04-25T14:10:46.079701+0000","last_active":"2024-04-25T14:10:47.067634+0000","last_peered":"2024-04-25T14:10:47.067634+0000","last_clean":"2024-04-25T14:10:47.067634+0000","last_became_active":"2024-04-25T14:10:46.079271+0000","last_became_peered":"2024-04-25T14:10:46.079271+0000","last_unstale":"2024-04-25T14:10:47.067634+0000","last_undegraded":"2024-04-25T14:10:47.067634+0000","last_fullsized":"2024-04-25T14:10:47.067634+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:13.569540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.064920+0000","last_change":"2024-04-25T14:10:46.064405+0000","last_active":"2024-04-25T14:10:47.064920+0000","last_peered":"2024-04-25T14:10:47.064920+0000","last_clean":"2024-04-25T14:10:47.064920+0000","last_became_active":"2024-04-25T14:10:46.064018+0000","last_became_peered":"2024-04-25T14:10:46.064018+0000","last_unstale":"2024-04-25T14:10:47.064920+0000","last_undegraded":"2024-04-25T14:10:47.064920+0000","last_fullsized":"2024-04-25T14:10:47.064920+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:45:31.644211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.067467+0000","last_change":"2024-04-25T14:10:46.061993+0000","last_active":"2024-04-25T14:10:47.067467+0000","last_peered":"2024-04-25T14:10:47.067467+0000","last_clean":"2024-04-25T14:10:47.067467+0000","last_became_active":"2024-04-25T14:10:46.061608+0000","last_became_peered":"2024-04-25T14:10:46.061608+0000","last_unstale":"2024-04-25T14:10:47.067467+0000","last_undegraded":"2024-04-25T14:10:47.067467+0000","last_fullsized":"2024-04-25T14:10:47.067467+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:56.449255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"125'4","reported_seq":18,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.062014+0000","last_change":"2024-04-25T14:10:45.051471+0000","last_active":"2024-04-25T14:10:47.062014+0000","last_peered":"2024-04-25T14:10:47.062014+0000","last_clean":"2024-04-25T14:10:47.062014+0000","last_became_active":"2024-04-25T14:10:45.051055+0000","last_became_peered":"2024-04-25T14:10:45.051055+0000","last_unstale":"2024-04-25T14:10:47.062014+0000","last_undegraded":"2024-04-25T14:10:47.062014+0000","last_fullsized":"2024-04-25T14:10:47.062014+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:54.008821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.201854+0000","last_change":"2024-04-25T14:10:46.051756+0000","last_active":"2024-04-25T14:10:47.201854+0000","last_peered":"2024-04-25T14:10:47.201854+0000","last_clean":"2024-04-25T14:10:47.201854+0000","last_became_active":"2024-04-25T14:10:46.051417+0000","last_became_peered":"2024-04-25T14:10:46.051417+0000","last_unstale":"2024-04-25T14:10:47.201854+0000","last_undegraded":"2024-04-25T14:10:47.201854+0000","last_fullsized":"2024-04-25T14:10:47.201854+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:21.295254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060136+0000","last_change":"2024-04-25T14:10:45.046859+0000","last_active":"2024-04-25T14:10:47.060136+0000","last_peered":"2024-04-25T14:10:47.060136+0000","last_clean":"2024-04-25T14:10:47.060136+0000","last_became_active":"2024-04-25T14:10:45.044618+0000","last_became_peered":"2024-04-25T14:10:45.044618+0000","last_unstale":"2024-04-25T14:10:47.060136+0000","last_undegraded":"2024-04-25T14:10:47.060136+0000","last_fullsized":"2024-04-25T14:10:47.060136+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:35.020438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.069866+0000","last_change":"2024-04-25T14:10:46.064256+0000","last_active":"2024-04-25T14:10:47.069866+0000","last_peered":"2024-04-25T14:10:47.069866+0000","last_clean":"2024-04-25T14:10:47.069866+0000","last_became_active":"2024-04-25T14:10:46.063977+0000","last_became_peered":"2024-04-25T14:10:46.063977+0000","last_unstale":"2024-04-25T14:10:47.069866+0000","last_undegraded":"2024-04-25T14:10:47.069866+0000","last_fullsized":"2024-04-25T14:10:47.069866+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:35.902293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060739+0000","last_change":"2024-04-25T14:10:45.056867+0000","last_active":"2024-04-25T14:10:47.060739+0000","last_peered":"2024-04-25T14:10:47.060739+0000","last_clean":"2024-04-25T14:10:47.060739+0000","last_became_active":"2024-04-25T14:10:45.056470+0000","last_became_peered":"2024-04-25T14:10:45.056470+0000","last_unstale":"2024-04-25T14:10:47.060739+0000","last_undegraded":"2024-04-25T14:10:47.060739+0000","last_fullsized":"2024-04-25T14:10:47.060739+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:12.563128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.201990+0000","last_change":"2024-04-25T14:10:46.060833+0000","last_active":"2024-04-25T14:10:47.201990+0000","last_peered":"2024-04-25T14:10:47.201990+0000","last_clean":"2024-04-25T14:10:47.201990+0000","last_became_active":"2024-04-25T14:10:46.060494+0000","last_became_peered":"2024-04-25T14:10:46.060494+0000","last_unstale":"2024-04-25T14:10:47.201990+0000","last_undegraded":"2024-04-25T14:10:47.201990+0000","last_fullsized":"2024-04-25T14:10:47.201990+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:59.631982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.061133+0000","last_change":"2024-04-25T14:10:45.052367+0000","last_active":"2024-04-25T14:10:47.061133+0000","last_peered":"2024-04-25T14:10:47.061133+0000","last_clean":"2024-04-25T14:10:47.061133+0000","last_became_active":"2024-04-25T14:10:45.052055+0000","last_became_peered":"2024-04-25T14:10:45.052055+0000","last_unstale":"2024-04-25T14:10:47.061133+0000","last_undegraded":"2024-04-25T14:10:47.061133+0000","last_fullsized":"2024-04-25T14:10:47.061133+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:20.290586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.067790+0000","last_change":"2024-04-25T14:10:46.065476+0000","last_active":"2024-04-25T14:10:47.067790+0000","last_peered":"2024-04-25T14:10:47.067790+0000","last_clean":"2024-04-25T14:10:47.067790+0000","last_became_active":"2024-04-25T14:10:46.065194+0000","last_became_peered":"2024-04-25T14:10:46.065194+0000","last_unstale":"2024-04-25T14:10:47.067790+0000","last_undegraded":"2024-04-25T14:10:47.067790+0000","last_fullsized":"2024-04-25T14:10:47.067790+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:04.502348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.067639+0000","last_change":"2024-04-25T14:10:45.041897+0000","last_active":"2024-04-25T14:10:47.067639+0000","last_peered":"2024-04-25T14:10:47.067639+0000","last_clean":"2024-04-25T14:10:47.067639+0000","last_became_active":"2024-04-25T14:10:45.039679+0000","last_became_peered":"2024-04-25T14:10:45.039679+0000","last_unstale":"2024-04-25T14:10:47.067639+0000","last_undegraded":"2024-04-25T14:10:47.067639+0000","last_fullsized":"2024-04-25T14:10:47.067639+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:52.050283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.067660+0000","last_change":"2024-04-25T14:10:46.069194+0000","last_active":"2024-04-25T14:10:47.067660+0000","last_peered":"2024-04-25T14:10:47.067660+0000","last_clean":"2024-04-25T14:10:47.067660+0000","last_became_active":"2024-04-25T14:10:46.068852+0000","last_became_peered":"2024-04-25T14:10:46.068852+0000","last_unstale":"2024-04-25T14:10:47.067660+0000","last_undegraded":"2024-04-25T14:10:47.067660+0000","last_fullsized":"2024-04-25T14:10:47.067660+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:35.902293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.059566+0000","last_change":"2024-04-25T14:10:45.051135+0000","last_active":"2024-04-25T14:10:47.059566+0000","last_peered":"2024-04-25T14:10:47.059566+0000","last_clean":"2024-04-25T14:10:47.059566+0000","last_became_active":"2024-04-25T14:10:45.050463+0000","last_became_peered":"2024-04-25T14:10:45.050463+0000","last_unstale":"2024-04-25T14:10:47.059566+0000","last_undegraded":"2024-04-25T14:10:47.059566+0000","last_fullsized":"2024-04-25T14:10:47.059566+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:47.253337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060048+0000","last_change":"2024-04-25T14:10:46.075482+0000","last_active":"2024-04-25T14:10:47.060048+0000","last_peered":"2024-04-25T14:10:47.060048+0000","last_clean":"2024-04-25T14:10:47.060048+0000","last_became_active":"2024-04-25T14:10:46.075129+0000","last_became_peered":"2024-04-25T14:10:46.075129+0000","last_unstale":"2024-04-25T14:10:47.060048+0000","last_undegraded":"2024-04-25T14:10:47.060048+0000","last_fullsized":"2024-04-25T14:10:47.060048+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:10.598819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202161+0000","last_change":"2024-04-25T14:10:45.053969+0000","last_active":"2024-04-25T14:10:47.202161+0000","last_peered":"2024-04-25T14:10:47.202161+0000","last_clean":"2024-04-25T14:10:47.202161+0000","last_became_active":"2024-04-25T14:10:45.053634+0000","last_became_peered":"2024-04-25T14:10:45.053634+0000","last_unstale":"2024-04-25T14:10:47.202161+0000","last_undegraded":"2024-04-25T14:10:47.202161+0000","last_fullsized":"2024-04-25T14:10:47.202161+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:14.821225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060677+0000","last_change":"2024-04-25T14:10:46.070850+0000","last_active":"2024-04-25T14:10:47.060677+0000","last_peered":"2024-04-25T14:10:47.060677+0000","last_clean":"2024-04-25T14:10:47.060677+0000","last_became_active":"2024-04-25T14:10:46.070065+0000","last_became_peered":"2024-04-25T14:10:46.070065+0000","last_unstale":"2024-04-25T14:10:47.060677+0000","last_undegraded":"2024-04-25T14:10:47.060677+0000","last_fullsized":"2024-04-25T14:10:47.060677+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:15.160432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060624+0000","last_change":"2024-04-25T14:10:45.054900+0000","last_active":"2024-04-25T14:10:47.060624+0000","last_peered":"2024-04-25T14:10:47.060624+0000","last_clean":"2024-04-25T14:10:47.060624+0000","last_became_active":"2024-04-25T14:10:45.054599+0000","last_became_peered":"2024-04-25T14:10:45.054599+0000","last_unstale":"2024-04-25T14:10:47.060624+0000","last_undegraded":"2024-04-25T14:10:47.060624+0000","last_fullsized":"2024-04-25T14:10:47.060624+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:58.758388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.201755+0000","last_change":"2024-04-25T14:10:46.072348+0000","last_active":"2024-04-25T14:10:47.201755+0000","last_peered":"2024-04-25T14:10:47.201755+0000","last_clean":"2024-04-25T14:10:47.201755+0000","last_became_active":"2024-04-25T14:10:46.071967+0000","last_became_peered":"2024-04-25T14:10:46.071967+0000","last_unstale":"2024-04-25T14:10:47.201755+0000","last_undegraded":"2024-04-25T14:10:47.201755+0000","last_fullsized":"2024-04-25T14:10:47.201755+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:54.009394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.069987+0000","last_change":"2024-04-25T14:10:45.037687+0000","last_active":"2024-04-25T14:10:47.069987+0000","last_peered":"2024-04-25T14:10:47.069987+0000","last_clean":"2024-04-25T14:10:47.069987+0000","last_became_active":"2024-04-25T14:10:45.037370+0000","last_became_peered":"2024-04-25T14:10:45.037370+0000","last_unstale":"2024-04-25T14:10:47.069987+0000","last_undegraded":"2024-04-25T14:10:47.069987+0000","last_fullsized":"2024-04-25T14:10:47.069987+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:52.050283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202239+0000","last_change":"2024-04-25T14:10:46.069364+0000","last_active":"2024-04-25T14:10:47.202239+0000","last_peered":"2024-04-25T14:10:47.202239+0000","last_clean":"2024-04-25T14:10:47.202239+0000","last_became_active":"2024-04-25T14:10:46.068439+0000","last_became_peered":"2024-04-25T14:10:46.068439+0000","last_unstale":"2024-04-25T14:10:47.202239+0000","last_undegraded":"2024-04-25T14:10:47.202239+0000","last_fullsized":"2024-04-25T14:10:47.202239+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:52.172177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202823+0000","last_change":"2024-04-25T14:10:45.050926+0000","last_active":"2024-04-25T14:10:47.202823+0000","last_peered":"2024-04-25T14:10:47.202823+0000","last_clean":"2024-04-25T14:10:47.202823+0000","last_became_active":"2024-04-25T14:10:45.049733+0000","last_became_peered":"2024-04-25T14:10:45.049733+0000","last_unstale":"2024-04-25T14:10:47.202823+0000","last_undegraded":"2024-04-25T14:10:47.202823+0000","last_fullsized":"2024-04-25T14:10:47.202823+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:30.741453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.061105+0000","last_change":"2024-04-25T14:10:46.077586+0000","last_active":"2024-04-25T14:10:47.061105+0000","last_peered":"2024-04-25T14:10:47.061105+0000","last_clean":"2024-04-25T14:10:47.061105+0000","last_became_active":"2024-04-25T14:10:46.076789+0000","last_became_peered":"2024-04-25T14:10:46.076789+0000","last_unstale":"2024-04-25T14:10:47.061105+0000","last_undegraded":"2024-04-25T14:10:47.061105+0000","last_fullsized":"2024-04-25T14:10:47.061105+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:13.569540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"125'3","reported_seq":17,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060851+0000","last_change":"2024-04-25T14:10:45.052411+0000","last_active":"2024-04-25T14:10:47.060851+0000","last_peered":"2024-04-25T14:10:47.060851+0000","last_clean":"2024-04-25T14:10:47.060851+0000","last_became_active":"2024-04-25T14:10:45.052020+0000","last_became_peered":"2024-04-25T14:10:45.052020+0000","last_unstale":"2024-04-25T14:10:47.060851+0000","last_undegraded":"2024-04-25T14:10:47.060851+0000","last_fullsized":"2024-04-25T14:10:47.060851+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:34.895882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.061579+0000","last_change":"2024-04-25T14:10:46.068749+0000","last_active":"2024-04-25T14:10:47.061579+0000","last_peered":"2024-04-25T14:10:47.061579+0000","last_clean":"2024-04-25T14:10:47.061579+0000","last_became_active":"2024-04-25T14:10:46.068462+0000","last_became_peered":"2024-04-25T14:10:46.068462+0000","last_unstale":"2024-04-25T14:10:47.061579+0000","last_undegraded":"2024-04-25T14:10:47.061579+0000","last_fullsized":"2024-04-25T14:10:47.061579+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:48.259748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"125'6","reported_seq":20,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060606+0000","last_change":"2024-04-25T14:10:45.054962+0000","last_active":"2024-04-25T14:10:47.060606+0000","last_peered":"2024-04-25T14:10:47.060606+0000","last_clean":"2024-04-25T14:10:47.060606+0000","last_became_active":"2024-04-25T14:10:45.054639+0000","last_became_peered":"2024-04-25T14:10:45.054639+0000","last_unstale":"2024-04-25T14:10:47.060606+0000","last_undegraded":"2024-04-25T14:10:47.060606+0000","last_fullsized":"2024-04-25T14:10:47.060606+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:55.442843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.065230+0000","last_change":"2024-04-25T14:10:46.059974+0000","last_active":"2024-04-25T14:10:47.065230+0000","last_peered":"2024-04-25T14:10:47.065230+0000","last_clean":"2024-04-25T14:10:47.065230+0000","last_became_active":"2024-04-25T14:10:46.059461+0000","last_became_peered":"2024-04-25T14:10:46.059461+0000","last_unstale":"2024-04-25T14:10:47.065230+0000","last_undegraded":"2024-04-25T14:10:47.065230+0000","last_fullsized":"2024-04-25T14:10:47.065230+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:40:47.902487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202638+0000","last_change":"2024-04-25T14:10:45.051737+0000","last_active":"2024-04-25T14:10:47.202638+0000","last_peered":"2024-04-25T14:10:47.202638+0000","last_clean":"2024-04-25T14:10:47.202638+0000","last_became_active":"2024-04-25T14:10:45.051337+0000","last_became_peered":"2024-04-25T14:10:45.051337+0000","last_unstale":"2024-04-25T14:10:47.202638+0000","last_undegraded":"2024-04-25T14:10:47.202638+0000","last_fullsized":"2024-04-25T14:10:47.202638+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:56.972785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.061126+0000","last_change":"2024-04-25T14:10:46.081526+0000","last_active":"2024-04-25T14:10:47.061126+0000","last_peered":"2024-04-25T14:10:47.061126+0000","last_clean":"2024-04-25T14:10:47.061126+0000","last_became_active":"2024-04-25T14:10:46.081153+0000","last_became_peered":"2024-04-25T14:10:46.081153+0000","last_unstale":"2024-04-25T14:10:47.061126+0000","last_undegraded":"2024-04-25T14:10:47.061126+0000","last_fullsized":"2024-04-25T14:10:47.061126+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:10.598819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060831+0000","last_change":"2024-04-25T14:10:45.049002+0000","last_active":"2024-04-25T14:10:47.060831+0000","last_peered":"2024-04-25T14:10:47.060831+0000","last_clean":"2024-04-25T14:10:47.060831+0000","last_became_active":"2024-04-25T14:10:45.048755+0000","last_became_peered":"2024-04-25T14:10:45.048755+0000","last_unstale":"2024-04-25T14:10:47.060831+0000","last_undegraded":"2024-04-25T14:10:47.060831+0000","last_fullsized":"2024-04-25T14:10:47.060831+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:01.990667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.059895+0000","last_change":"2024-04-25T14:10:46.065779+0000","last_active":"2024-04-25T14:10:47.059895+0000","last_peered":"2024-04-25T14:10:47.059895+0000","last_clean":"2024-04-25T14:10:47.059895+0000","last_became_active":"2024-04-25T14:10:46.065454+0000","last_became_peered":"2024-04-25T14:10:46.065454+0000","last_unstale":"2024-04-25T14:10:47.059895+0000","last_undegraded":"2024-04-25T14:10:47.059895+0000","last_fullsized":"2024-04-25T14:10:47.059895+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:13.479334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"125'8","reported_seq":22,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202177+0000","last_change":"2024-04-25T14:10:45.051770+0000","last_active":"2024-04-25T14:10:47.202177+0000","last_peered":"2024-04-25T14:10:47.202177+0000","last_clean":"2024-04-25T14:10:47.202177+0000","last_became_active":"2024-04-25T14:10:45.051446+0000","last_became_peered":"2024-04-25T14:10:45.051446+0000","last_unstale":"2024-04-25T14:10:47.202177+0000","last_undegraded":"2024-04-25T14:10:47.202177+0000","last_fullsized":"2024-04-25T14:10:47.202177+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:55.116141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060459+0000","last_change":"2024-04-25T14:10:46.068019+0000","last_active":"2024-04-25T14:10:47.060459+0000","last_peered":"2024-04-25T14:10:47.060459+0000","last_clean":"2024-04-25T14:10:47.060459+0000","last_became_active":"2024-04-25T14:10:46.067686+0000","last_became_peered":"2024-04-25T14:10:46.067686+0000","last_unstale":"2024-04-25T14:10:47.060459+0000","last_undegraded":"2024-04-25T14:10:47.060459+0000","last_fullsized":"2024-04-25T14:10:47.060459+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:21.296998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.070063+0000","last_change":"2024-04-25T14:10:45.037175+0000","last_active":"2024-04-25T14:10:47.070063+0000","last_peered":"2024-04-25T14:10:47.070063+0000","last_clean":"2024-04-25T14:10:47.070063+0000","last_became_active":"2024-04-25T14:10:45.036881+0000","last_became_peered":"2024-04-25T14:10:45.036881+0000","last_unstale":"2024-04-25T14:10:47.070063+0000","last_undegraded":"2024-04-25T14:10:47.070063+0000","last_fullsized":"2024-04-25T14:10:47.070063+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:33.243994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060331+0000","last_change":"2024-04-25T14:10:46.072610+0000","last_active":"2024-04-25T14:10:47.060331+0000","last_peered":"2024-04-25T14:10:47.060331+0000","last_clean":"2024-04-25T14:10:47.060331+0000","last_became_active":"2024-04-25T14:10:46.072272+0000","last_became_peered":"2024-04-25T14:10:46.072272+0000","last_unstale":"2024-04-25T14:10:47.060331+0000","last_undegraded":"2024-04-25T14:10:47.060331+0000","last_fullsized":"2024-04-25T14:10:47.060331+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:59.236859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.061856+0000","last_change":"2024-04-25T14:10:45.052421+0000","last_active":"2024-04-25T14:10:47.061856+0000","last_peered":"2024-04-25T14:10:47.061856+0000","last_clean":"2024-04-25T14:10:47.061856+0000","last_became_active":"2024-04-25T14:10:45.052078+0000","last_became_peered":"2024-04-25T14:10:45.052078+0000","last_unstale":"2024-04-25T14:10:47.061856+0000","last_undegraded":"2024-04-25T14:10:47.061856+0000","last_fullsized":"2024-04-25T14:10:47.061856+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:22.669468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060764+0000","last_change":"2024-04-25T14:10:46.078401+0000","last_active":"2024-04-25T14:10:47.060764+0000","last_peered":"2024-04-25T14:10:47.060764+0000","last_clean":"2024-04-25T14:10:47.060764+0000","last_became_active":"2024-04-25T14:10:46.074147+0000","last_became_peered":"2024-04-25T14:10:46.074147+0000","last_unstale":"2024-04-25T14:10:47.060764+0000","last_undegraded":"2024-04-25T14:10:47.060764+0000","last_fullsized":"2024-04-25T14:10:47.060764+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:09.225406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202946+0000","last_change":"2024-04-25T14:10:45.056493+0000","last_active":"2024-04-25T14:10:47.202946+0000","last_peered":"2024-04-25T14:10:47.202946+0000","last_clean":"2024-04-25T14:10:47.202946+0000","last_became_active":"2024-04-25T14:10:45.055271+0000","last_became_peered":"2024-04-25T14:10:45.055271+0000","last_unstale":"2024-04-25T14:10:47.202946+0000","last_undegraded":"2024-04-25T14:10:47.202946+0000","last_fullsized":"2024-04-25T14:10:47.202946+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:22.629958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.065043+0000","last_change":"2024-04-25T14:10:46.071797+0000","last_active":"2024-04-25T14:10:47.065043+0000","last_peered":"2024-04-25T14:10:47.065043+0000","last_clean":"2024-04-25T14:10:47.065043+0000","last_became_active":"2024-04-25T14:10:46.071504+0000","last_became_peered":"2024-04-25T14:10:46.071504+0000","last_unstale":"2024-04-25T14:10:47.065043+0000","last_undegraded":"2024-04-25T14:10:47.065043+0000","last_fullsized":"2024-04-25T14:10:47.065043+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:41.808794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060046+0000","last_change":"2024-04-25T14:10:45.055677+0000","last_active":"2024-04-25T14:10:47.060046+0000","last_peered":"2024-04-25T14:10:47.060046+0000","last_clean":"2024-04-25T14:10:47.060046+0000","last_became_active":"2024-04-25T14:10:45.054262+0000","last_became_peered":"2024-04-25T14:10:45.054262+0000","last_unstale":"2024-04-25T14:10:47.060046+0000","last_undegraded":"2024-04-25T14:10:47.060046+0000","last_fullsized":"2024-04-25T14:10:47.060046+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:36.988908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.059967+0000","last_change":"2024-04-25T14:10:46.062938+0000","last_active":"2024-04-25T14:10:47.059967+0000","last_peered":"2024-04-25T14:10:47.059967+0000","last_clean":"2024-04-25T14:10:47.059967+0000","last_became_active":"2024-04-25T14:10:46.062652+0000","last_became_peered":"2024-04-25T14:10:46.062652+0000","last_unstale":"2024-04-25T14:10:47.059967+0000","last_undegraded":"2024-04-25T14:10:47.059967+0000","last_fullsized":"2024-04-25T14:10:47.059967+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:34.869650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.067776+0000","last_change":"2024-04-25T14:10:45.047266+0000","last_active":"2024-04-25T14:10:47.067776+0000","last_peered":"2024-04-25T14:10:47.067776+0000","last_clean":"2024-04-25T14:10:47.067776+0000","last_became_active":"2024-04-25T14:10:45.046928+0000","last_became_peered":"2024-04-25T14:10:45.046928+0000","last_unstale":"2024-04-25T14:10:47.067776+0000","last_undegraded":"2024-04-25T14:10:47.067776+0000","last_fullsized":"2024-04-25T14:10:47.067776+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:33.243994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.203086+0000","last_change":"2024-04-25T14:10:46.068140+0000","last_active":"2024-04-25T14:10:47.203086+0000","last_peered":"2024-04-25T14:10:47.203086+0000","last_clean":"2024-04-25T14:10:47.203086+0000","last_became_active":"2024-04-25T14:10:46.067559+0000","last_became_peered":"2024-04-25T14:10:46.067559+0000","last_unstale":"2024-04-25T14:10:47.203086+0000","last_undegraded":"2024-04-25T14:10:47.203086+0000","last_fullsized":"2024-04-25T14:10:47.203086+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:58.755296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.201997+0000","last_change":"2024-04-25T14:10:45.057160+0000","last_active":"2024-04-25T14:10:47.201997+0000","last_peered":"2024-04-25T14:10:47.201997+0000","last_clean":"2024-04-25T14:10:47.201997+0000","last_became_active":"2024-04-25T14:10:45.056871+0000","last_became_peered":"2024-04-25T14:10:45.056871+0000","last_unstale":"2024-04-25T14:10:47.201997+0000","last_undegraded":"2024-04-25T14:10:47.201997+0000","last_fullsized":"2024-04-25T14:10:47.201997+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:20.109193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060037+0000","last_change":"2024-04-25T14:10:46.068918+0000","last_active":"2024-04-25T14:10:47.060037+0000","last_peered":"2024-04-25T14:10:47.060037+0000","last_clean":"2024-04-25T14:10:47.060037+0000","last_became_active":"2024-04-25T14:10:46.068604+0000","last_became_peered":"2024-04-25T14:10:46.068604+0000","last_unstale":"2024-04-25T14:10:47.060037+0000","last_undegraded":"2024-04-25T14:10:47.060037+0000","last_fullsized":"2024-04-25T14:10:47.060037+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:17.255415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060420+0000","last_change":"2024-04-25T14:10:45.059841+0000","last_active":"2024-04-25T14:10:47.060420+0000","last_peered":"2024-04-25T14:10:47.060420+0000","last_clean":"2024-04-25T14:10:47.060420+0000","last_became_active":"2024-04-25T14:10:45.059609+0000","last_became_peered":"2024-04-25T14:10:45.059609+0000","last_unstale":"2024-04-25T14:10:47.060420+0000","last_undegraded":"2024-04-25T14:10:47.060420+0000","last_fullsized":"2024-04-25T14:10:47.060420+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:10.566680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.070012+0000","last_change":"2024-04-25T14:10:46.072628+0000","last_active":"2024-04-25T14:10:47.070012+0000","last_peered":"2024-04-25T14:10:47.070012+0000","last_clean":"2024-04-25T14:10:47.070012+0000","last_became_active":"2024-04-25T14:10:46.071215+0000","last_became_peered":"2024-04-25T14:10:46.071215+0000","last_unstale":"2024-04-25T14:10:47.070012+0000","last_undegraded":"2024-04-25T14:10:47.070012+0000","last_fullsized":"2024-04-25T14:10:47.070012+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:36.026849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202287+0000","last_change":"2024-04-25T14:10:45.057442+0000","last_active":"2024-04-25T14:10:47.202287+0000","last_peered":"2024-04-25T14:10:47.202287+0000","last_clean":"2024-04-25T14:10:47.202287+0000","last_became_active":"2024-04-25T14:10:45.057077+0000","last_became_peered":"2024-04-25T14:10:45.057077+0000","last_unstale":"2024-04-25T14:10:47.202287+0000","last_undegraded":"2024-04-25T14:10:47.202287+0000","last_fullsized":"2024-04-25T14:10:47.202287+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:38.714815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.059736+0000","last_change":"2024-04-25T14:10:46.075657+0000","last_active":"2024-04-25T14:10:47.059736+0000","last_peered":"2024-04-25T14:10:47.059736+0000","last_clean":"2024-04-25T14:10:47.059736+0000","last_became_active":"2024-04-25T14:10:46.075357+0000","last_became_peered":"2024-04-25T14:10:46.075357+0000","last_unstale":"2024-04-25T14:10:47.059736+0000","last_undegraded":"2024-04-25T14:10:47.059736+0000","last_fullsized":"2024-04-25T14:10:47.059736+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:15.160432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.203158+0000","last_change":"2024-04-25T14:10:45.056735+0000","last_active":"2024-04-25T14:10:47.203158+0000","last_peered":"2024-04-25T14:10:47.203158+0000","last_clean":"2024-04-25T14:10:47.203158+0000","last_became_active":"2024-04-25T14:10:45.052530+0000","last_became_peered":"2024-04-25T14:10:45.052530+0000","last_unstale":"2024-04-25T14:10:47.203158+0000","last_undegraded":"2024-04-25T14:10:47.203158+0000","last_fullsized":"2024-04-25T14:10:47.203158+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:32.672468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060737+0000","last_change":"2024-04-25T14:10:46.067306+0000","last_active":"2024-04-25T14:10:47.060737+0000","last_peered":"2024-04-25T14:10:47.060737+0000","last_clean":"2024-04-25T14:10:47.060737+0000","last_became_active":"2024-04-25T14:10:46.066990+0000","last_became_peered":"2024-04-25T14:10:46.066990+0000","last_unstale":"2024-04-25T14:10:47.060737+0000","last_undegraded":"2024-04-25T14:10:47.060737+0000","last_fullsized":"2024-04-25T14:10:47.060737+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:56.449255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"126'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.067866+0000","last_change":"2024-04-25T14:10:45.046573+0000","last_active":"2024-04-25T14:10:47.067866+0000","last_peered":"2024-04-25T14:10:47.067866+0000","last_clean":"2024-04-25T14:10:47.067866+0000","last_became_active":"2024-04-25T14:10:45.046226+0000","last_became_peered":"2024-04-25T14:10:45.046226+0000","last_unstale":"2024-04-25T14:10:47.067866+0000","last_undegraded":"2024-04-25T14:10:47.067866+0000","last_fullsized":"2024-04-25T14:10:47.067866+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:55.116141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060418+0000","last_change":"2024-04-25T14:10:46.076749+0000","last_active":"2024-04-25T14:10:47.060418+0000","last_peered":"2024-04-25T14:10:47.060418+0000","last_clean":"2024-04-25T14:10:47.060418+0000","last_became_active":"2024-04-25T14:10:46.074554+0000","last_became_peered":"2024-04-25T14:10:46.074554+0000","last_unstale":"2024-04-25T14:10:47.060418+0000","last_undegraded":"2024-04-25T14:10:47.060418+0000","last_fullsized":"2024-04-25T14:10:47.060418+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:34.869650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.067765+0000","last_change":"2024-04-25T14:10:45.055809+0000","last_active":"2024-04-25T14:10:47.067765+0000","last_peered":"2024-04-25T14:10:47.067765+0000","last_clean":"2024-04-25T14:10:47.067765+0000","last_became_active":"2024-04-25T14:10:45.055480+0000","last_became_peered":"2024-04-25T14:10:45.055480+0000","last_unstale":"2024-04-25T14:10:47.067765+0000","last_undegraded":"2024-04-25T14:10:47.067765+0000","last_fullsized":"2024-04-25T14:10:47.067765+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:22.629958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.065197+0000","last_change":"2024-04-25T14:10:46.065927+0000","last_active":"2024-04-25T14:10:47.065197+0000","last_peered":"2024-04-25T14:10:47.065197+0000","last_clean":"2024-04-25T14:10:47.065197+0000","last_became_active":"2024-04-25T14:10:46.065621+0000","last_became_peered":"2024-04-25T14:10:46.065621+0000","last_unstale":"2024-04-25T14:10:47.065197+0000","last_undegraded":"2024-04-25T14:10:47.065197+0000","last_fullsized":"2024-04-25T14:10:47.065197+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:11.942560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202551+0000","last_change":"2024-04-25T14:10:45.058112+0000","last_active":"2024-04-25T14:10:47.202551+0000","last_peered":"2024-04-25T14:10:47.202551+0000","last_clean":"2024-04-25T14:10:47.202551+0000","last_became_active":"2024-04-25T14:10:45.057825+0000","last_became_peered":"2024-04-25T14:10:45.057825+0000","last_unstale":"2024-04-25T14:10:47.202551+0000","last_undegraded":"2024-04-25T14:10:47.202551+0000","last_fullsized":"2024-04-25T14:10:47.202551+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:01.583932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202570+0000","last_change":"2024-04-25T14:10:46.069554+0000","last_active":"2024-04-25T14:10:47.202570+0000","last_peered":"2024-04-25T14:10:47.202570+0000","last_clean":"2024-04-25T14:10:47.202570+0000","last_became_active":"2024-04-25T14:10:46.069112+0000","last_became_peered":"2024-04-25T14:10:46.069112+0000","last_unstale":"2024-04-25T14:10:47.202570+0000","last_undegraded":"2024-04-25T14:10:47.202570+0000","last_fullsized":"2024-04-25T14:10:47.202570+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:27.987770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.069756+0000","last_change":"2024-04-25T14:10:45.049671+0000","last_active":"2024-04-25T14:10:47.069756+0000","last_peered":"2024-04-25T14:10:47.069756+0000","last_clean":"2024-04-25T14:10:47.069756+0000","last_became_active":"2024-04-25T14:10:45.049375+0000","last_became_peered":"2024-04-25T14:10:45.049375+0000","last_unstale":"2024-04-25T14:10:47.069756+0000","last_undegraded":"2024-04-25T14:10:47.069756+0000","last_fullsized":"2024-04-25T14:10:47.069756+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:03.729472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202837+0000","last_change":"2024-04-25T14:10:46.073752+0000","last_active":"2024-04-25T14:10:47.202837+0000","last_peered":"2024-04-25T14:10:47.202837+0000","last_clean":"2024-04-25T14:10:47.202837+0000","last_became_active":"2024-04-25T14:10:46.073446+0000","last_became_peered":"2024-04-25T14:10:46.073446+0000","last_unstale":"2024-04-25T14:10:47.202837+0000","last_undegraded":"2024-04-25T14:10:47.202837+0000","last_fullsized":"2024-04-25T14:10:47.202837+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:57.190018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060879+0000","last_change":"2024-04-25T14:10:45.053154+0000","last_active":"2024-04-25T14:10:47.060879+0000","last_peered":"2024-04-25T14:10:47.060879+0000","last_clean":"2024-04-25T14:10:47.060879+0000","last_became_active":"2024-04-25T14:10:45.052842+0000","last_became_peered":"2024-04-25T14:10:45.052842+0000","last_unstale":"2024-04-25T14:10:47.060879+0000","last_undegraded":"2024-04-25T14:10:47.060879+0000","last_fullsized":"2024-04-25T14:10:47.060879+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:35.020438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202856+0000","last_change":"2024-04-25T14:10:46.067406+0000","last_active":"2024-04-25T14:10:47.202856+0000","last_peered":"2024-04-25T14:10:47.202856+0000","last_clean":"2024-04-25T14:10:47.202856+0000","last_became_active":"2024-04-25T14:10:46.067024+0000","last_became_peered":"2024-04-25T14:10:46.067024+0000","last_unstale":"2024-04-25T14:10:47.202856+0000","last_undegraded":"2024-04-25T14:10:47.202856+0000","last_fullsized":"2024-04-25T14:10:47.202856+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:59.236859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202279+0000","last_change":"2024-04-25T14:10:45.058009+0000","last_active":"2024-04-25T14:10:47.202279+0000","last_peered":"2024-04-25T14:10:47.202279+0000","last_clean":"2024-04-25T14:10:47.202279+0000","last_became_active":"2024-04-25T14:10:45.057724+0000","last_became_peered":"2024-04-25T14:10:45.057724+0000","last_unstale":"2024-04-25T14:10:47.202279+0000","last_undegraded":"2024-04-25T14:10:47.202279+0000","last_fullsized":"2024-04-25T14:10:47.202279+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:19.174254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060840+0000","last_change":"2024-04-25T14:10:46.078777+0000","last_active":"2024-04-25T14:10:47.060840+0000","last_peered":"2024-04-25T14:10:47.060840+0000","last_clean":"2024-04-25T14:10:47.060840+0000","last_became_active":"2024-04-25T14:10:46.073967+0000","last_became_peered":"2024-04-25T14:10:46.073967+0000","last_unstale":"2024-04-25T14:10:47.060840+0000","last_undegraded":"2024-04-25T14:10:47.060840+0000","last_fullsized":"2024-04-25T14:10:47.060840+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:38:21.375099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.059887+0000","last_change":"2024-04-25T14:10:45.043256+0000","last_active":"2024-04-25T14:10:47.059887+0000","last_peered":"2024-04-25T14:10:47.059887+0000","last_clean":"2024-04-25T14:10:47.059887+0000","last_became_active":"2024-04-25T14:10:45.042731+0000","last_became_peered":"2024-04-25T14:10:45.042731+0000","last_unstale":"2024-04-25T14:10:47.059887+0000","last_undegraded":"2024-04-25T14:10:47.059887+0000","last_fullsized":"2024-04-25T14:10:47.059887+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:03.495936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.061539+0000","last_change":"2024-04-25T14:10:46.070796+0000","last_active":"2024-04-25T14:10:47.061539+0000","last_peered":"2024-04-25T14:10:47.061539+0000","last_clean":"2024-04-25T14:10:47.061539+0000","last_became_active":"2024-04-25T14:10:46.070503+0000","last_became_peered":"2024-04-25T14:10:46.070503+0000","last_unstale":"2024-04-25T14:10:47.061539+0000","last_undegraded":"2024-04-25T14:10:47.061539+0000","last_fullsized":"2024-04-25T14:10:47.061539+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:36.026849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060570+0000","last_change":"2024-04-25T14:10:45.053553+0000","last_active":"2024-04-25T14:10:47.060570+0000","last_peered":"2024-04-25T14:10:47.060570+0000","last_clean":"2024-04-25T14:10:47.060570+0000","last_became_active":"2024-04-25T14:10:45.053178+0000","last_became_peered":"2024-04-25T14:10:45.053178+0000","last_unstale":"2024-04-25T14:10:47.060570+0000","last_undegraded":"2024-04-25T14:10:47.060570+0000","last_fullsized":"2024-04-25T14:10:47.060570+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:03.495936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"126'2649","reported_seq":3042,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:50.606748+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:10:50.606748+0000","last_peered":"2024-04-25T14:10:50.606748+0000","last_clean":"2024-04-25T14:10:50.606748+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:10:50.606748+0000","last_undegraded":"2024-04-25T14:10:50.606748+0000","last_fullsized":"2024-04-25T14:10:50.606748+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":2649,"log_dups_size":0,"ondisk_log_size":2649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":536,"num_read_kb":1396,"num_write":187,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":536,"num_read_kb":1396,"num_write":187,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2649,"ondisk_log_size":2649,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574931,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41632,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330208,"statfs":{"total":96636764160,"available":96594132992,"internally_reserved":0,"allocated":2187264,"data_stored":1380400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574932,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1585152,"data_stored":789002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574933,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32724,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1581056,"data_stored":788352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32720,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1576960,"data_stored":788330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574932,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1593344,"data_stored":790012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574931,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1585152,"data_stored":788946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32724,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1581056,"data_stored":788400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574932,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41624,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330216,"statfs":{"total":96636764160,"available":96594141184,"internally_reserved":0,"allocated":2179072,"data_stored":1378754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:10:52.043 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:52.045+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:52.840 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:52.841+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:53.645 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:53.645+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:54.449 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:54.449+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:54.943 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:10:55.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.101+0000 7fbf39e00640 1 Processor -- start 2024-04-25T14:10:55.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.101+0000 7fbf39e00640 1 -- start start 2024-04-25T14:10:55.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.101+0000 7fbf39e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbf34070330 0x7fbf340d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:55.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.101+0000 7fbf39e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf340d2030 con 0x7fbf340ce790 2024-04-25T14:10:55.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.101+0000 7fbf39e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf3415e5c0 con 0x7fbf340d1c20 2024-04-25T14:10:55.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.101+0000 7fbf39e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fbf3415e8a0 con 0x7fbf34070330 2024-04-25T14:10:55.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.101+0000 7fbf33400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fbf340ce790 0x7fbf3406f440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54978/0 (socket says 172.21.15.86:54978) 2024-04-25T14:10:55.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.101+0000 7fbf33400640 1 -- 172.21.15.86:0/3542007026 learned_addr learned my addr 172.21.15.86:0/3542007026 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:55.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.101+0000 7fbf32a00640 1 --2- 172.21.15.86:0/3542007026 >> v2:172.21.15.155:3300/0 conn(0x7fbf34070330 0x7fbf340d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:55.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.101+0000 7fbf31600640 1 -- 172.21.15.86:0/3542007026 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2289213760 0 0) 0x7fbf2c0014a0 con 0x7fbf340ce790 2024-04-25T14:10:55.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.105+0000 7fbf31600640 1 -- 172.21.15.86:0/3542007026 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf18003530 con 0x7fbf340ce790 2024-04-25T14:10:55.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.105+0000 7fbf31600640 1 -- 172.21.15.86:0/3542007026 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 868641445 0 0) 0x7fbf18003530 con 0x7fbf340ce790 2024-04-25T14:10:55.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.105+0000 7fbf31600640 1 -- 172.21.15.86:0/3542007026 >> v2:172.21.15.155:3300/0 conn(0x7fbf34070330 msgr2=0x7fbf340d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:55.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.105+0000 7fbf31600640 1 --2- 172.21.15.86:0/3542007026 >> v2:172.21.15.155:3300/0 conn(0x7fbf34070330 0x7fbf340d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.105+0000 7fbf31600640 1 -- 172.21.15.86:0/3542007026 >> v1:172.21.15.155:6789/0 conn(0x7fbf340d1c20 legacy=0x7fbf3406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:55.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.105+0000 7fbf31600640 1 -- 172.21.15.86:0/3542007026 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3415eb80 con 0x7fbf340ce790 2024-04-25T14:10:55.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.105+0000 7fbf31600640 1 -- 172.21.15.86:0/3542007026 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fbf2c001bd0 con 0x7fbf340ce790 2024-04-25T14:10:55.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.105+0000 7fbf31600640 1 -- 172.21.15.86:0/3542007026 >> v1:172.21.15.86:6789/0 conn(0x7fbf340ce790 legacy=0x7fbf3406f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:55.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.105+0000 7fbf31600640 1 --2- 172.21.15.86:0/3542007026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf18003a30 0x7fbf18003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:55.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.105+0000 7fbf31600640 1 --2- 172.21.15.86:0/3542007026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf18004810 0x7fbf18004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:55.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.105+0000 7fbf31600640 1 --2- 172.21.15.86:0/3542007026 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf18005360 0x7fbf1800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:55.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.105+0000 7fbf31600640 1 -- 172.21.15.86:0/3542007026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3415e8a0 con 0x7fbf18003a30 2024-04-25T14:10:55.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.105+0000 7fbf31600640 1 -- 172.21.15.86:0/3542007026 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1800c260 con 0x7fbf18004810 2024-04-25T14:10:55.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.105+0000 7fbf31600640 1 -- 172.21.15.86:0/3542007026 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbf1800c460 con 0x7fbf18005360 2024-04-25T14:10:55.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.105+0000 7fbf32a00640 1 --2- 172.21.15.86:0/3542007026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf18003a30 0x7fbf18003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:55.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.105+0000 7fbf33400640 1 --2- 172.21.15.86:0/3542007026 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf18005360 0x7fbf1800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:55.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.105+0000 7fbf32000640 1 --2- 172.21.15.86:0/3542007026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf18004810 0x7fbf18004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:55.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.105+0000 7fbf32a00640 1 -- 172.21.15.86:0/3542007026 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf18005360 msgr2=0x7fbf1800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:55.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.105+0000 7fbf32a00640 1 --2- 172.21.15.86:0/3542007026 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf18005360 0x7fbf1800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.105+0000 7fbf32a00640 1 -- 172.21.15.86:0/3542007026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf18004810 msgr2=0x7fbf18004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:55.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.105+0000 7fbf32a00640 1 --2- 172.21.15.86:0/3542007026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf18004810 0x7fbf18004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.105+0000 7fbf32a00640 1 -- 172.21.15.86:0/3542007026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf18003530 con 0x7fbf18003a30 2024-04-25T14:10:55.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.105+0000 7fbf33400640 1 -- 172.21.15.86:0/3542007026 reap_dead start 2024-04-25T14:10:55.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.109+0000 7fbf32a00640 1 --2- 172.21.15.86:0/3542007026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf18003a30 0x7fbf18003ea0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fbf24009ab0 tx=0x7fbf2401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69bffb474f15ac3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:55.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.109+0000 7fbf31600640 1 -- 172.21.15.86:0/3542007026 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf240096b0 con 0x7fbf18003a30 2024-04-25T14:10:55.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.109+0000 7fbf31600640 1 -- 172.21.15.86:0/3542007026 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf240a6430 con 0x7fbf18003a30 2024-04-25T14:10:55.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.109+0000 7fbf31600640 1 -- 172.21.15.86:0/3542007026 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf240a6820 con 0x7fbf18003a30 2024-04-25T14:10:55.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.109+0000 7fbf39e00640 1 -- 172.21.15.86:0/3542007026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf18003a30 msgr2=0x7fbf18003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:55.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.109+0000 7fbf39e00640 1 --2- 172.21.15.86:0/3542007026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf18003a30 0x7fbf18003ea0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fbf24009ab0 tx=0x7fbf2401ca50 comp rx=0 tx=0).stop 2024-04-25T14:10:55.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.109+0000 7fbf39e00640 1 -- 172.21.15.86:0/3542007026 shutdown_connections 2024-04-25T14:10:55.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.109+0000 7fbf39e00640 1 --2- 172.21.15.86:0/3542007026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf18003a30 0x7fbf18003ea0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.109+0000 7fbf39e00640 1 -- 172.21.15.86:0/3542007026 >> 172.21.15.86:0/3542007026 conn(0x7fbf340b4650 msgr2=0x7fbf340b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:55.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.109+0000 7fbf39e00640 1 -- 172.21.15.86:0/3542007026 shutdown_connections 2024-04-25T14:10:55.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.109+0000 7fbf39e00640 1 -- 172.21.15.86:0/3542007026 wait complete. 2024-04-25T14:10:55.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.109+0000 7fbf39e00640 1 Processor -- start 2024-04-25T14:10:55.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.109+0000 7fbf39e00640 1 -- start start 2024-04-25T14:10:55.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.113+0000 7fbf39e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf18003a30 0x7fbf341fc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:55.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.113+0000 7fbf39e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf3415c300 0x7fbf341fcf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:55.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.113+0000 7fbf39e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf341fd4d0 0x7fbf340c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:55.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.113+0000 7fbf39e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf340708c0 con 0x7fbf3415c300 2024-04-25T14:10:55.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.113+0000 7fbf39e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbf34070740 con 0x7fbf341fd4d0 2024-04-25T14:10:55.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.113+0000 7fbf39e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbf340cdf90 con 0x7fbf18003a30 2024-04-25T14:10:55.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.113+0000 7fbf32000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf3415c300 0x7fbf341fcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:55.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.113+0000 7fbf32000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf3415c300 0x7fbf341fcf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46242/0 (socket says 172.21.15.86:46242) 2024-04-25T14:10:55.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.113+0000 7fbf32000640 1 -- 172.21.15.86:0/1784527702 learned_addr learned my addr 172.21.15.86:0/1784527702 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:55.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.113+0000 7fbf32a00640 1 --2- 172.21.15.86:0/1784527702 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf18003a30 0x7fbf341fc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:55.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.113+0000 7fbf33400640 1 --2- 172.21.15.86:0/1784527702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf341fd4d0 0x7fbf340c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:55.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.113+0000 7fbf32000640 1 -- 172.21.15.86:0/1784527702 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf18003a30 msgr2=0x7fbf341fc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:55.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.113+0000 7fbf32000640 1 --2- 172.21.15.86:0/1784527702 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf18003a30 0x7fbf341fc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.113+0000 7fbf32000640 1 -- 172.21.15.86:0/1784527702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf341fd4d0 msgr2=0x7fbf340c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:55.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.113+0000 7fbf32000640 1 --2- 172.21.15.86:0/1784527702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf341fd4d0 0x7fbf340c9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.113+0000 7fbf32000640 1 -- 172.21.15.86:0/1784527702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3415d000 con 0x7fbf3415c300 2024-04-25T14:10:55.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.113+0000 7fbf32000640 1 --2- 172.21.15.86:0/1784527702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf3415c300 0x7fbf341fcf90 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000f570 tx=0x7fbf2000f970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:55.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.113+0000 7fbf2b400640 1 -- 172.21.15.86:0/1784527702 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf200052e0 con 0x7fbf3415c300 2024-04-25T14:10:55.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.113+0000 7fbf2b400640 1 -- 172.21.15.86:0/1784527702 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf20018070 con 0x7fbf3415c300 2024-04-25T14:10:55.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.113+0000 7fbf39e00640 1 -- 172.21.15.86:0/1784527702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf340ceeb0 con 0x7fbf3415c300 2024-04-25T14:10:55.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.113+0000 7fbf2b400640 1 -- 172.21.15.86:0/1784527702 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf20003820 con 0x7fbf3415c300 2024-04-25T14:10:55.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.113+0000 7fbf39e00640 1 -- 172.21.15.86:0/1784527702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf340cf150 con 0x7fbf3415c300 2024-04-25T14:10:55.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.113+0000 7fbf39e00640 1 -- 172.21.15.86:0/1784527702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf34069d00 con 0x7fbf3415c300 2024-04-25T14:10:55.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.117+0000 7fbf2b400640 1 -- 172.21.15.86:0/1784527702 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7fbf200039c0 con 0x7fbf3415c300 2024-04-25T14:10:55.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.117+0000 7fbf2b400640 1 --2- 172.21.15.86:0/1784527702 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fbf1406f490 0x7fbf140719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:55.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.117+0000 7fbf2b400640 1 -- 172.21.15.86:0/1784527702 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7fbf2008efd0 con 0x7fbf3415c300 2024-04-25T14:10:55.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.117+0000 7fbf32a00640 1 --2- 172.21.15.86:0/1784527702 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fbf1406f490 0x7fbf140719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:55.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.117+0000 7fbf32a00640 1 --2- 172.21.15.86:0/1784527702 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fbf1406f490 0x7fbf140719a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbf240ad000 tx=0x7fbf240046d0 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:55.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.121+0000 7fbf2b400640 1 -- 172.21.15.86:0/1784527702 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbf2005bd40 con 0x7fbf3415c300 2024-04-25T14:10:55.255 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:10:55.257+0000 7f67ae000640 -1 client.8796: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8679:217bf30f-571a-4282-9b4b-c379a5aca2c4:v1:172.21.15.86:0/2412030790] 2024-04-25T14:10:55.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.281+0000 7fbf39e00640 1 -- 172.21.15.86:0/1784527702 --> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbf34076bf0 con 0x7fbf1406f490 2024-04-25T14:10:55.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.289+0000 7fbf2b400640 1 -- 172.21.15.86:0/1784527702 <== mgr.8338 v2:172.21.15.86:6832/3964666460 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252481 (secure 0 0 0) 0x7fbf34076bf0 con 0x7fbf1406f490 2024-04-25T14:10:55.288 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:55.290 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:10:55.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.293+0000 7fbf39e00640 1 -- 172.21.15.86:0/1784527702 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fbf1406f490 msgr2=0x7fbf140719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:55.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.293+0000 7fbf39e00640 1 --2- 172.21.15.86:0/1784527702 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fbf1406f490 0x7fbf140719a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbf240ad000 tx=0x7fbf240046d0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.293+0000 7fbf39e00640 1 -- 172.21.15.86:0/1784527702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf3415c300 msgr2=0x7fbf341fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:55.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.293+0000 7fbf39e00640 1 --2- 172.21.15.86:0/1784527702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf3415c300 0x7fbf341fcf90 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000f570 tx=0x7fbf2000f970 comp rx=0 tx=0).stop 2024-04-25T14:10:55.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.293+0000 7fbf39e00640 1 -- 172.21.15.86:0/1784527702 shutdown_connections 2024-04-25T14:10:55.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.293+0000 7fbf39e00640 1 --2- 172.21.15.86:0/1784527702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf341fd4d0 0x7fbf340c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.293+0000 7fbf39e00640 1 --2- 172.21.15.86:0/1784527702 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fbf1406f490 0x7fbf140719a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.293+0000 7fbf39e00640 1 --2- 172.21.15.86:0/1784527702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf3415c300 0x7fbf341fcf90 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.293+0000 7fbf39e00640 1 --2- 172.21.15.86:0/1784527702 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf18003a30 0x7fbf341fc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.293+0000 7fbf39e00640 1 -- 172.21.15.86:0/1784527702 >> 172.21.15.86:0/1784527702 conn(0x7fbf340b4650 msgr2=0x7fbf340b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:55.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.293+0000 7fbf39e00640 1 -- 172.21.15.86:0/1784527702 shutdown_connections 2024-04-25T14:10:55.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.293+0000 7fbf39e00640 1 -- 172.21.15.86:0/1784527702 wait complete. 2024-04-25T14:10:55.310 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-04-25T14:10:53.616542+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":536,"num_read_kb":1396,"num_write":232,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2688,"ondisk_log_size":2688,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279616,"kb_used_data":13544,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754695104,"statfs":{"total":773094113280,"available":772807786496,"internally_reserved":0,"allocated":13869056,"data_stored":7492196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001981"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.203013+0000","last_change":"2024-04-25T14:10:46.052795+0000","last_active":"2024-04-25T14:10:47.203013+0000","last_peered":"2024-04-25T14:10:47.203013+0000","last_clean":"2024-04-25T14:10:47.203013+0000","last_became_active":"2024-04-25T14:10:46.052501+0000","last_became_peered":"2024-04-25T14:10:46.052501+0000","last_unstale":"2024-04-25T14:10:47.203013+0000","last_undegraded":"2024-04-25T14:10:47.203013+0000","last_fullsized":"2024-04-25T14:10:47.203013+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:57.384121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.070065+0000","last_change":"2024-04-25T14:10:46.062961+0000","last_active":"2024-04-25T14:10:47.070065+0000","last_peered":"2024-04-25T14:10:47.070065+0000","last_clean":"2024-04-25T14:10:47.070065+0000","last_became_active":"2024-04-25T14:10:46.062564+0000","last_became_peered":"2024-04-25T14:10:46.062564+0000","last_unstale":"2024-04-25T14:10:47.070065+0000","last_undegraded":"2024-04-25T14:10:47.070065+0000","last_fullsized":"2024-04-25T14:10:47.070065+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:48.259748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.061394+0000","last_change":"2024-04-25T14:10:46.070470+0000","last_active":"2024-04-25T14:10:47.061394+0000","last_peered":"2024-04-25T14:10:47.061394+0000","last_clean":"2024-04-25T14:10:47.061394+0000","last_became_active":"2024-04-25T14:10:46.070060+0000","last_became_peered":"2024-04-25T14:10:46.070060+0000","last_unstale":"2024-04-25T14:10:47.061394+0000","last_undegraded":"2024-04-25T14:10:47.061394+0000","last_fullsized":"2024-04-25T14:10:47.061394+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:35.902293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.069855+0000","last_change":"2024-04-25T14:10:46.052043+0000","last_active":"2024-04-25T14:10:47.069855+0000","last_peered":"2024-04-25T14:10:47.069855+0000","last_clean":"2024-04-25T14:10:47.069855+0000","last_became_active":"2024-04-25T14:10:46.051748+0000","last_became_peered":"2024-04-25T14:10:46.051748+0000","last_unstale":"2024-04-25T14:10:47.069855+0000","last_undegraded":"2024-04-25T14:10:47.069855+0000","last_fullsized":"2024-04-25T14:10:47.069855+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:13.569540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.059645+0000","last_change":"2024-04-25T14:10:46.061616+0000","last_active":"2024-04-25T14:10:47.059645+0000","last_peered":"2024-04-25T14:10:47.059645+0000","last_clean":"2024-04-25T14:10:47.059645+0000","last_became_active":"2024-04-25T14:10:46.061296+0000","last_became_peered":"2024-04-25T14:10:46.061296+0000","last_unstale":"2024-04-25T14:10:47.059645+0000","last_undegraded":"2024-04-25T14:10:47.059645+0000","last_fullsized":"2024-04-25T14:10:47.059645+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:06.636965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.203067+0000","last_change":"2024-04-25T14:10:46.073895+0000","last_active":"2024-04-25T14:10:47.203067+0000","last_peered":"2024-04-25T14:10:47.203067+0000","last_clean":"2024-04-25T14:10:47.203067+0000","last_became_active":"2024-04-25T14:10:46.072233+0000","last_became_peered":"2024-04-25T14:10:46.072233+0000","last_unstale":"2024-04-25T14:10:47.203067+0000","last_undegraded":"2024-04-25T14:10:47.203067+0000","last_fullsized":"2024-04-25T14:10:47.203067+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:15.160432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060297+0000","last_change":"2024-04-25T14:10:46.074066+0000","last_active":"2024-04-25T14:10:47.060297+0000","last_peered":"2024-04-25T14:10:47.060297+0000","last_clean":"2024-04-25T14:10:47.060297+0000","last_became_active":"2024-04-25T14:10:46.073745+0000","last_became_peered":"2024-04-25T14:10:46.073745+0000","last_unstale":"2024-04-25T14:10:47.060297+0000","last_undegraded":"2024-04-25T14:10:47.060297+0000","last_fullsized":"2024-04-25T14:10:47.060297+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:57.190018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.067458+0000","last_change":"2024-04-25T14:10:46.072612+0000","last_active":"2024-04-25T14:10:47.067458+0000","last_peered":"2024-04-25T14:10:47.067458+0000","last_clean":"2024-04-25T14:10:47.067458+0000","last_became_active":"2024-04-25T14:10:46.072263+0000","last_became_peered":"2024-04-25T14:10:46.072263+0000","last_unstale":"2024-04-25T14:10:47.067458+0000","last_undegraded":"2024-04-25T14:10:47.067458+0000","last_fullsized":"2024-04-25T14:10:47.067458+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:21.296998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202939+0000","last_change":"2024-04-25T14:10:46.077239+0000","last_active":"2024-04-25T14:10:47.202939+0000","last_peered":"2024-04-25T14:10:47.202939+0000","last_clean":"2024-04-25T14:10:47.202939+0000","last_became_active":"2024-04-25T14:10:46.076966+0000","last_became_peered":"2024-04-25T14:10:46.076966+0000","last_unstale":"2024-04-25T14:10:47.202939+0000","last_undegraded":"2024-04-25T14:10:47.202939+0000","last_fullsized":"2024-04-25T14:10:47.202939+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:06.636965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.065101+0000","last_change":"2024-04-25T14:10:46.059381+0000","last_active":"2024-04-25T14:10:47.065101+0000","last_peered":"2024-04-25T14:10:47.065101+0000","last_clean":"2024-04-25T14:10:47.065101+0000","last_became_active":"2024-04-25T14:10:46.059060+0000","last_became_peered":"2024-04-25T14:10:46.059060+0000","last_unstale":"2024-04-25T14:10:47.065101+0000","last_undegraded":"2024-04-25T14:10:47.065101+0000","last_fullsized":"2024-04-25T14:10:47.065101+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:23.703727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.069934+0000","last_change":"2024-04-25T14:10:46.070967+0000","last_active":"2024-04-25T14:10:47.069934+0000","last_peered":"2024-04-25T14:10:47.069934+0000","last_clean":"2024-04-25T14:10:47.069934+0000","last_became_active":"2024-04-25T14:10:46.070649+0000","last_became_peered":"2024-04-25T14:10:46.070649+0000","last_unstale":"2024-04-25T14:10:47.069934+0000","last_undegraded":"2024-04-25T14:10:47.069934+0000","last_fullsized":"2024-04-25T14:10:47.069934+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:04.502348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202897+0000","last_change":"2024-04-25T14:10:46.062312+0000","last_active":"2024-04-25T14:10:47.202897+0000","last_peered":"2024-04-25T14:10:47.202897+0000","last_clean":"2024-04-25T14:10:47.202897+0000","last_became_active":"2024-04-25T14:10:46.062040+0000","last_became_peered":"2024-04-25T14:10:46.062040+0000","last_unstale":"2024-04-25T14:10:47.202897+0000","last_undegraded":"2024-04-25T14:10:47.202897+0000","last_fullsized":"2024-04-25T14:10:47.202897+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:07.483995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202667+0000","last_change":"2024-04-25T14:10:46.070884+0000","last_active":"2024-04-25T14:10:47.202667+0000","last_peered":"2024-04-25T14:10:47.202667+0000","last_clean":"2024-04-25T14:10:47.202667+0000","last_became_active":"2024-04-25T14:10:46.070638+0000","last_became_peered":"2024-04-25T14:10:46.070638+0000","last_unstale":"2024-04-25T14:10:47.202667+0000","last_undegraded":"2024-04-25T14:10:47.202667+0000","last_fullsized":"2024-04-25T14:10:47.202667+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:37.900506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.201995+0000","last_change":"2024-04-25T14:10:46.060384+0000","last_active":"2024-04-25T14:10:47.201995+0000","last_peered":"2024-04-25T14:10:47.201995+0000","last_clean":"2024-04-25T14:10:47.201995+0000","last_became_active":"2024-04-25T14:10:46.058137+0000","last_became_peered":"2024-04-25T14:10:46.058137+0000","last_unstale":"2024-04-25T14:10:47.201995+0000","last_undegraded":"2024-04-25T14:10:47.201995+0000","last_fullsized":"2024-04-25T14:10:47.201995+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:53.079759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.201769+0000","last_change":"2024-04-25T14:10:46.059716+0000","last_active":"2024-04-25T14:10:47.201769+0000","last_peered":"2024-04-25T14:10:47.201769+0000","last_clean":"2024-04-25T14:10:47.201769+0000","last_became_active":"2024-04-25T14:10:46.059418+0000","last_became_peered":"2024-04-25T14:10:46.059418+0000","last_unstale":"2024-04-25T14:10:47.201769+0000","last_undegraded":"2024-04-25T14:10:47.201769+0000","last_fullsized":"2024-04-25T14:10:47.201769+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:50.496026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.201912+0000","last_change":"2024-04-25T14:10:46.064411+0000","last_active":"2024-04-25T14:10:47.201912+0000","last_peered":"2024-04-25T14:10:47.201912+0000","last_clean":"2024-04-25T14:10:47.201912+0000","last_became_active":"2024-04-25T14:10:46.064098+0000","last_became_peered":"2024-04-25T14:10:46.064098+0000","last_unstale":"2024-04-25T14:10:47.201912+0000","last_undegraded":"2024-04-25T14:10:47.201912+0000","last_fullsized":"2024-04-25T14:10:47.201912+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:04:17.096454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.203099+0000","last_change":"2024-04-25T14:10:46.064728+0000","last_active":"2024-04-25T14:10:47.203099+0000","last_peered":"2024-04-25T14:10:47.203099+0000","last_clean":"2024-04-25T14:10:47.203099+0000","last_became_active":"2024-04-25T14:10:46.064402+0000","last_became_peered":"2024-04-25T14:10:46.064402+0000","last_unstale":"2024-04-25T14:10:47.203099+0000","last_undegraded":"2024-04-25T14:10:47.203099+0000","last_fullsized":"2024-04-25T14:10:47.203099+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:09.225406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.059826+0000","last_change":"2024-04-25T14:10:46.061910+0000","last_active":"2024-04-25T14:10:47.059826+0000","last_peered":"2024-04-25T14:10:47.059826+0000","last_clean":"2024-04-25T14:10:47.059826+0000","last_became_active":"2024-04-25T14:10:46.061626+0000","last_became_peered":"2024-04-25T14:10:46.061626+0000","last_unstale":"2024-04-25T14:10:47.059826+0000","last_undegraded":"2024-04-25T14:10:47.059826+0000","last_fullsized":"2024-04-25T14:10:47.059826+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:09.225406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060583+0000","last_change":"2024-04-25T14:10:46.078014+0000","last_active":"2024-04-25T14:10:47.060583+0000","last_peered":"2024-04-25T14:10:47.060583+0000","last_clean":"2024-04-25T14:10:47.060583+0000","last_became_active":"2024-04-25T14:10:46.073655+0000","last_became_peered":"2024-04-25T14:10:46.073655+0000","last_unstale":"2024-04-25T14:10:47.060583+0000","last_undegraded":"2024-04-25T14:10:47.060583+0000","last_fullsized":"2024-04-25T14:10:47.060583+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:06.636965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.059927+0000","last_change":"2024-04-25T14:10:46.075501+0000","last_active":"2024-04-25T14:10:47.059927+0000","last_peered":"2024-04-25T14:10:47.059927+0000","last_clean":"2024-04-25T14:10:47.059927+0000","last_became_active":"2024-04-25T14:10:46.075206+0000","last_became_peered":"2024-04-25T14:10:46.075206+0000","last_unstale":"2024-04-25T14:10:47.059927+0000","last_undegraded":"2024-04-25T14:10:47.059927+0000","last_fullsized":"2024-04-25T14:10:47.059927+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:38:21.375099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.069818+0000","last_change":"2024-04-25T14:10:46.069371+0000","last_active":"2024-04-25T14:10:47.069818+0000","last_peered":"2024-04-25T14:10:47.069818+0000","last_clean":"2024-04-25T14:10:47.069818+0000","last_became_active":"2024-04-25T14:10:46.068924+0000","last_became_peered":"2024-04-25T14:10:46.068924+0000","last_unstale":"2024-04-25T14:10:47.069818+0000","last_undegraded":"2024-04-25T14:10:47.069818+0000","last_fullsized":"2024-04-25T14:10:47.069818+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:56.449255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.201898+0000","last_change":"2024-04-25T14:10:46.051588+0000","last_active":"2024-04-25T14:10:47.201898+0000","last_peered":"2024-04-25T14:10:47.201898+0000","last_clean":"2024-04-25T14:10:47.201898+0000","last_became_active":"2024-04-25T14:10:46.051301+0000","last_became_peered":"2024-04-25T14:10:46.051301+0000","last_unstale":"2024-04-25T14:10:47.201898+0000","last_undegraded":"2024-04-25T14:10:47.201898+0000","last_fullsized":"2024-04-25T14:10:47.201898+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:47.023113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.069796+0000","last_change":"2024-04-25T14:10:46.063747+0000","last_active":"2024-04-25T14:10:47.069796+0000","last_peered":"2024-04-25T14:10:47.069796+0000","last_clean":"2024-04-25T14:10:47.069796+0000","last_became_active":"2024-04-25T14:10:46.063416+0000","last_became_peered":"2024-04-25T14:10:46.063416+0000","last_unstale":"2024-04-25T14:10:47.069796+0000","last_undegraded":"2024-04-25T14:10:47.069796+0000","last_fullsized":"2024-04-25T14:10:47.069796+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:21.296998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202071+0000","last_change":"2024-04-25T14:10:46.076309+0000","last_active":"2024-04-25T14:10:47.202071+0000","last_peered":"2024-04-25T14:10:47.202071+0000","last_clean":"2024-04-25T14:10:47.202071+0000","last_became_active":"2024-04-25T14:10:46.076027+0000","last_became_peered":"2024-04-25T14:10:46.076027+0000","last_unstale":"2024-04-25T14:10:47.202071+0000","last_undegraded":"2024-04-25T14:10:47.202071+0000","last_fullsized":"2024-04-25T14:10:47.202071+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:36.059775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.062071+0000","last_change":"2024-04-25T14:10:46.073491+0000","last_active":"2024-04-25T14:10:47.062071+0000","last_peered":"2024-04-25T14:10:47.062071+0000","last_clean":"2024-04-25T14:10:47.062071+0000","last_became_active":"2024-04-25T14:10:46.073143+0000","last_became_peered":"2024-04-25T14:10:46.073143+0000","last_unstale":"2024-04-25T14:10:47.062071+0000","last_undegraded":"2024-04-25T14:10:47.062071+0000","last_fullsized":"2024-04-25T14:10:47.062071+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:37.995319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.061886+0000","last_change":"2024-04-25T14:10:46.070326+0000","last_active":"2024-04-25T14:10:47.061886+0000","last_peered":"2024-04-25T14:10:47.061886+0000","last_clean":"2024-04-25T14:10:47.061886+0000","last_became_active":"2024-04-25T14:10:46.069774+0000","last_became_peered":"2024-04-25T14:10:46.069774+0000","last_unstale":"2024-04-25T14:10:47.061886+0000","last_undegraded":"2024-04-25T14:10:47.061886+0000","last_fullsized":"2024-04-25T14:10:47.061886+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:25.544185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.061109+0000","last_change":"2024-04-25T14:10:46.067155+0000","last_active":"2024-04-25T14:10:47.061109+0000","last_peered":"2024-04-25T14:10:47.061109+0000","last_clean":"2024-04-25T14:10:47.061109+0000","last_became_active":"2024-04-25T14:10:46.066844+0000","last_became_peered":"2024-04-25T14:10:46.066844+0000","last_unstale":"2024-04-25T14:10:47.061109+0000","last_undegraded":"2024-04-25T14:10:47.061109+0000","last_fullsized":"2024-04-25T14:10:47.061109+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:17.255415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.065139+0000","last_change":"2024-04-25T14:10:46.059739+0000","last_active":"2024-04-25T14:10:47.065139+0000","last_peered":"2024-04-25T14:10:47.065139+0000","last_clean":"2024-04-25T14:10:47.065139+0000","last_became_active":"2024-04-25T14:10:46.059042+0000","last_became_peered":"2024-04-25T14:10:46.059042+0000","last_unstale":"2024-04-25T14:10:47.065139+0000","last_undegraded":"2024-04-25T14:10:47.065139+0000","last_fullsized":"2024-04-25T14:10:47.065139+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:53.254490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.061419+0000","last_change":"2024-04-25T14:10:46.066163+0000","last_active":"2024-04-25T14:10:47.061419+0000","last_peered":"2024-04-25T14:10:47.061419+0000","last_clean":"2024-04-25T14:10:47.061419+0000","last_became_active":"2024-04-25T14:10:46.065719+0000","last_became_peered":"2024-04-25T14:10:46.065719+0000","last_unstale":"2024-04-25T14:10:47.061419+0000","last_undegraded":"2024-04-25T14:10:47.061419+0000","last_fullsized":"2024-04-25T14:10:47.061419+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:04.502348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.061753+0000","last_change":"2024-04-25T14:10:46.067769+0000","last_active":"2024-04-25T14:10:47.061753+0000","last_peered":"2024-04-25T14:10:47.061753+0000","last_clean":"2024-04-25T14:10:47.061753+0000","last_became_active":"2024-04-25T14:10:46.067441+0000","last_became_peered":"2024-04-25T14:10:46.067441+0000","last_unstale":"2024-04-25T14:10:47.061753+0000","last_undegraded":"2024-04-25T14:10:47.061753+0000","last_fullsized":"2024-04-25T14:10:47.061753+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:44:18.230978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.067634+0000","last_change":"2024-04-25T14:10:46.079701+0000","last_active":"2024-04-25T14:10:47.067634+0000","last_peered":"2024-04-25T14:10:47.067634+0000","last_clean":"2024-04-25T14:10:47.067634+0000","last_became_active":"2024-04-25T14:10:46.079271+0000","last_became_peered":"2024-04-25T14:10:46.079271+0000","last_unstale":"2024-04-25T14:10:47.067634+0000","last_undegraded":"2024-04-25T14:10:47.067634+0000","last_fullsized":"2024-04-25T14:10:47.067634+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:13.569540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.064920+0000","last_change":"2024-04-25T14:10:46.064405+0000","last_active":"2024-04-25T14:10:47.064920+0000","last_peered":"2024-04-25T14:10:47.064920+0000","last_clean":"2024-04-25T14:10:47.064920+0000","last_became_active":"2024-04-25T14:10:46.064018+0000","last_became_peered":"2024-04-25T14:10:46.064018+0000","last_unstale":"2024-04-25T14:10:47.064920+0000","last_undegraded":"2024-04-25T14:10:47.064920+0000","last_fullsized":"2024-04-25T14:10:47.064920+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:45:31.644211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.067467+0000","last_change":"2024-04-25T14:10:46.061993+0000","last_active":"2024-04-25T14:10:47.067467+0000","last_peered":"2024-04-25T14:10:47.067467+0000","last_clean":"2024-04-25T14:10:47.067467+0000","last_became_active":"2024-04-25T14:10:46.061608+0000","last_became_peered":"2024-04-25T14:10:46.061608+0000","last_unstale":"2024-04-25T14:10:47.067467+0000","last_undegraded":"2024-04-25T14:10:47.067467+0000","last_fullsized":"2024-04-25T14:10:47.067467+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:56.449255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"125'4","reported_seq":18,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.062014+0000","last_change":"2024-04-25T14:10:45.051471+0000","last_active":"2024-04-25T14:10:47.062014+0000","last_peered":"2024-04-25T14:10:47.062014+0000","last_clean":"2024-04-25T14:10:47.062014+0000","last_became_active":"2024-04-25T14:10:45.051055+0000","last_became_peered":"2024-04-25T14:10:45.051055+0000","last_unstale":"2024-04-25T14:10:47.062014+0000","last_undegraded":"2024-04-25T14:10:47.062014+0000","last_fullsized":"2024-04-25T14:10:47.062014+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:54.008821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.201854+0000","last_change":"2024-04-25T14:10:46.051756+0000","last_active":"2024-04-25T14:10:47.201854+0000","last_peered":"2024-04-25T14:10:47.201854+0000","last_clean":"2024-04-25T14:10:47.201854+0000","last_became_active":"2024-04-25T14:10:46.051417+0000","last_became_peered":"2024-04-25T14:10:46.051417+0000","last_unstale":"2024-04-25T14:10:47.201854+0000","last_undegraded":"2024-04-25T14:10:47.201854+0000","last_fullsized":"2024-04-25T14:10:47.201854+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:21.295254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060136+0000","last_change":"2024-04-25T14:10:45.046859+0000","last_active":"2024-04-25T14:10:47.060136+0000","last_peered":"2024-04-25T14:10:47.060136+0000","last_clean":"2024-04-25T14:10:47.060136+0000","last_became_active":"2024-04-25T14:10:45.044618+0000","last_became_peered":"2024-04-25T14:10:45.044618+0000","last_unstale":"2024-04-25T14:10:47.060136+0000","last_undegraded":"2024-04-25T14:10:47.060136+0000","last_fullsized":"2024-04-25T14:10:47.060136+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:35.020438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.069866+0000","last_change":"2024-04-25T14:10:46.064256+0000","last_active":"2024-04-25T14:10:47.069866+0000","last_peered":"2024-04-25T14:10:47.069866+0000","last_clean":"2024-04-25T14:10:47.069866+0000","last_became_active":"2024-04-25T14:10:46.063977+0000","last_became_peered":"2024-04-25T14:10:46.063977+0000","last_unstale":"2024-04-25T14:10:47.069866+0000","last_undegraded":"2024-04-25T14:10:47.069866+0000","last_fullsized":"2024-04-25T14:10:47.069866+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:35.902293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060739+0000","last_change":"2024-04-25T14:10:45.056867+0000","last_active":"2024-04-25T14:10:47.060739+0000","last_peered":"2024-04-25T14:10:47.060739+0000","last_clean":"2024-04-25T14:10:47.060739+0000","last_became_active":"2024-04-25T14:10:45.056470+0000","last_became_peered":"2024-04-25T14:10:45.056470+0000","last_unstale":"2024-04-25T14:10:47.060739+0000","last_undegraded":"2024-04-25T14:10:47.060739+0000","last_fullsized":"2024-04-25T14:10:47.060739+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:12.563128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.201990+0000","last_change":"2024-04-25T14:10:46.060833+0000","last_active":"2024-04-25T14:10:47.201990+0000","last_peered":"2024-04-25T14:10:47.201990+0000","last_clean":"2024-04-25T14:10:47.201990+0000","last_became_active":"2024-04-25T14:10:46.060494+0000","last_became_peered":"2024-04-25T14:10:46.060494+0000","last_unstale":"2024-04-25T14:10:47.201990+0000","last_undegraded":"2024-04-25T14:10:47.201990+0000","last_fullsized":"2024-04-25T14:10:47.201990+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:59.631982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.061133+0000","last_change":"2024-04-25T14:10:45.052367+0000","last_active":"2024-04-25T14:10:47.061133+0000","last_peered":"2024-04-25T14:10:47.061133+0000","last_clean":"2024-04-25T14:10:47.061133+0000","last_became_active":"2024-04-25T14:10:45.052055+0000","last_became_peered":"2024-04-25T14:10:45.052055+0000","last_unstale":"2024-04-25T14:10:47.061133+0000","last_undegraded":"2024-04-25T14:10:47.061133+0000","last_fullsized":"2024-04-25T14:10:47.061133+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:20.290586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.067790+0000","last_change":"2024-04-25T14:10:46.065476+0000","last_active":"2024-04-25T14:10:47.067790+0000","last_peered":"2024-04-25T14:10:47.067790+0000","last_clean":"2024-04-25T14:10:47.067790+0000","last_became_active":"2024-04-25T14:10:46.065194+0000","last_became_peered":"2024-04-25T14:10:46.065194+0000","last_unstale":"2024-04-25T14:10:47.067790+0000","last_undegraded":"2024-04-25T14:10:47.067790+0000","last_fullsized":"2024-04-25T14:10:47.067790+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:04.502348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.067639+0000","last_change":"2024-04-25T14:10:45.041897+0000","last_active":"2024-04-25T14:10:47.067639+0000","last_peered":"2024-04-25T14:10:47.067639+0000","last_clean":"2024-04-25T14:10:47.067639+0000","last_became_active":"2024-04-25T14:10:45.039679+0000","last_became_peered":"2024-04-25T14:10:45.039679+0000","last_unstale":"2024-04-25T14:10:47.067639+0000","last_undegraded":"2024-04-25T14:10:47.067639+0000","last_fullsized":"2024-04-25T14:10:47.067639+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:52.050283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.067660+0000","last_change":"2024-04-25T14:10:46.069194+0000","last_active":"2024-04-25T14:10:47.067660+0000","last_peered":"2024-04-25T14:10:47.067660+0000","last_clean":"2024-04-25T14:10:47.067660+0000","last_became_active":"2024-04-25T14:10:46.068852+0000","last_became_peered":"2024-04-25T14:10:46.068852+0000","last_unstale":"2024-04-25T14:10:47.067660+0000","last_undegraded":"2024-04-25T14:10:47.067660+0000","last_fullsized":"2024-04-25T14:10:47.067660+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:35.902293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.059566+0000","last_change":"2024-04-25T14:10:45.051135+0000","last_active":"2024-04-25T14:10:47.059566+0000","last_peered":"2024-04-25T14:10:47.059566+0000","last_clean":"2024-04-25T14:10:47.059566+0000","last_became_active":"2024-04-25T14:10:45.050463+0000","last_became_peered":"2024-04-25T14:10:45.050463+0000","last_unstale":"2024-04-25T14:10:47.059566+0000","last_undegraded":"2024-04-25T14:10:47.059566+0000","last_fullsized":"2024-04-25T14:10:47.059566+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:47.253337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060048+0000","last_change":"2024-04-25T14:10:46.075482+0000","last_active":"2024-04-25T14:10:47.060048+0000","last_peered":"2024-04-25T14:10:47.060048+0000","last_clean":"2024-04-25T14:10:47.060048+0000","last_became_active":"2024-04-25T14:10:46.075129+0000","last_became_peered":"2024-04-25T14:10:46.075129+0000","last_unstale":"2024-04-25T14:10:47.060048+0000","last_undegraded":"2024-04-25T14:10:47.060048+0000","last_fullsized":"2024-04-25T14:10:47.060048+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:10.598819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202161+0000","last_change":"2024-04-25T14:10:45.053969+0000","last_active":"2024-04-25T14:10:47.202161+0000","last_peered":"2024-04-25T14:10:47.202161+0000","last_clean":"2024-04-25T14:10:47.202161+0000","last_became_active":"2024-04-25T14:10:45.053634+0000","last_became_peered":"2024-04-25T14:10:45.053634+0000","last_unstale":"2024-04-25T14:10:47.202161+0000","last_undegraded":"2024-04-25T14:10:47.202161+0000","last_fullsized":"2024-04-25T14:10:47.202161+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:14.821225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060677+0000","last_change":"2024-04-25T14:10:46.070850+0000","last_active":"2024-04-25T14:10:47.060677+0000","last_peered":"2024-04-25T14:10:47.060677+0000","last_clean":"2024-04-25T14:10:47.060677+0000","last_became_active":"2024-04-25T14:10:46.070065+0000","last_became_peered":"2024-04-25T14:10:46.070065+0000","last_unstale":"2024-04-25T14:10:47.060677+0000","last_undegraded":"2024-04-25T14:10:47.060677+0000","last_fullsized":"2024-04-25T14:10:47.060677+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:15.160432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060624+0000","last_change":"2024-04-25T14:10:45.054900+0000","last_active":"2024-04-25T14:10:47.060624+0000","last_peered":"2024-04-25T14:10:47.060624+0000","last_clean":"2024-04-25T14:10:47.060624+0000","last_became_active":"2024-04-25T14:10:45.054599+0000","last_became_peered":"2024-04-25T14:10:45.054599+0000","last_unstale":"2024-04-25T14:10:47.060624+0000","last_undegraded":"2024-04-25T14:10:47.060624+0000","last_fullsized":"2024-04-25T14:10:47.060624+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:58.758388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.201755+0000","last_change":"2024-04-25T14:10:46.072348+0000","last_active":"2024-04-25T14:10:47.201755+0000","last_peered":"2024-04-25T14:10:47.201755+0000","last_clean":"2024-04-25T14:10:47.201755+0000","last_became_active":"2024-04-25T14:10:46.071967+0000","last_became_peered":"2024-04-25T14:10:46.071967+0000","last_unstale":"2024-04-25T14:10:47.201755+0000","last_undegraded":"2024-04-25T14:10:47.201755+0000","last_fullsized":"2024-04-25T14:10:47.201755+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:54.009394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.069987+0000","last_change":"2024-04-25T14:10:45.037687+0000","last_active":"2024-04-25T14:10:47.069987+0000","last_peered":"2024-04-25T14:10:47.069987+0000","last_clean":"2024-04-25T14:10:47.069987+0000","last_became_active":"2024-04-25T14:10:45.037370+0000","last_became_peered":"2024-04-25T14:10:45.037370+0000","last_unstale":"2024-04-25T14:10:47.069987+0000","last_undegraded":"2024-04-25T14:10:47.069987+0000","last_fullsized":"2024-04-25T14:10:47.069987+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:52.050283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202239+0000","last_change":"2024-04-25T14:10:46.069364+0000","last_active":"2024-04-25T14:10:47.202239+0000","last_peered":"2024-04-25T14:10:47.202239+0000","last_clean":"2024-04-25T14:10:47.202239+0000","last_became_active":"2024-04-25T14:10:46.068439+0000","last_became_peered":"2024-04-25T14:10:46.068439+0000","last_unstale":"2024-04-25T14:10:47.202239+0000","last_undegraded":"2024-04-25T14:10:47.202239+0000","last_fullsized":"2024-04-25T14:10:47.202239+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:07:52.172177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202823+0000","last_change":"2024-04-25T14:10:45.050926+0000","last_active":"2024-04-25T14:10:47.202823+0000","last_peered":"2024-04-25T14:10:47.202823+0000","last_clean":"2024-04-25T14:10:47.202823+0000","last_became_active":"2024-04-25T14:10:45.049733+0000","last_became_peered":"2024-04-25T14:10:45.049733+0000","last_unstale":"2024-04-25T14:10:47.202823+0000","last_undegraded":"2024-04-25T14:10:47.202823+0000","last_fullsized":"2024-04-25T14:10:47.202823+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:30.741453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.061105+0000","last_change":"2024-04-25T14:10:46.077586+0000","last_active":"2024-04-25T14:10:47.061105+0000","last_peered":"2024-04-25T14:10:47.061105+0000","last_clean":"2024-04-25T14:10:47.061105+0000","last_became_active":"2024-04-25T14:10:46.076789+0000","last_became_peered":"2024-04-25T14:10:46.076789+0000","last_unstale":"2024-04-25T14:10:47.061105+0000","last_undegraded":"2024-04-25T14:10:47.061105+0000","last_fullsized":"2024-04-25T14:10:47.061105+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:13.569540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"125'3","reported_seq":17,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060851+0000","last_change":"2024-04-25T14:10:45.052411+0000","last_active":"2024-04-25T14:10:47.060851+0000","last_peered":"2024-04-25T14:10:47.060851+0000","last_clean":"2024-04-25T14:10:47.060851+0000","last_became_active":"2024-04-25T14:10:45.052020+0000","last_became_peered":"2024-04-25T14:10:45.052020+0000","last_unstale":"2024-04-25T14:10:47.060851+0000","last_undegraded":"2024-04-25T14:10:47.060851+0000","last_fullsized":"2024-04-25T14:10:47.060851+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:34.895882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.061579+0000","last_change":"2024-04-25T14:10:46.068749+0000","last_active":"2024-04-25T14:10:47.061579+0000","last_peered":"2024-04-25T14:10:47.061579+0000","last_clean":"2024-04-25T14:10:47.061579+0000","last_became_active":"2024-04-25T14:10:46.068462+0000","last_became_peered":"2024-04-25T14:10:46.068462+0000","last_unstale":"2024-04-25T14:10:47.061579+0000","last_undegraded":"2024-04-25T14:10:47.061579+0000","last_fullsized":"2024-04-25T14:10:47.061579+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:48.259748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"125'6","reported_seq":20,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060606+0000","last_change":"2024-04-25T14:10:45.054962+0000","last_active":"2024-04-25T14:10:47.060606+0000","last_peered":"2024-04-25T14:10:47.060606+0000","last_clean":"2024-04-25T14:10:47.060606+0000","last_became_active":"2024-04-25T14:10:45.054639+0000","last_became_peered":"2024-04-25T14:10:45.054639+0000","last_unstale":"2024-04-25T14:10:47.060606+0000","last_undegraded":"2024-04-25T14:10:47.060606+0000","last_fullsized":"2024-04-25T14:10:47.060606+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:55.442843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.065230+0000","last_change":"2024-04-25T14:10:46.059974+0000","last_active":"2024-04-25T14:10:47.065230+0000","last_peered":"2024-04-25T14:10:47.065230+0000","last_clean":"2024-04-25T14:10:47.065230+0000","last_became_active":"2024-04-25T14:10:46.059461+0000","last_became_peered":"2024-04-25T14:10:46.059461+0000","last_unstale":"2024-04-25T14:10:47.065230+0000","last_undegraded":"2024-04-25T14:10:47.065230+0000","last_fullsized":"2024-04-25T14:10:47.065230+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:40:47.902487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202638+0000","last_change":"2024-04-25T14:10:45.051737+0000","last_active":"2024-04-25T14:10:47.202638+0000","last_peered":"2024-04-25T14:10:47.202638+0000","last_clean":"2024-04-25T14:10:47.202638+0000","last_became_active":"2024-04-25T14:10:45.051337+0000","last_became_peered":"2024-04-25T14:10:45.051337+0000","last_unstale":"2024-04-25T14:10:47.202638+0000","last_undegraded":"2024-04-25T14:10:47.202638+0000","last_fullsized":"2024-04-25T14:10:47.202638+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:56.972785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.061126+0000","last_change":"2024-04-25T14:10:46.081526+0000","last_active":"2024-04-25T14:10:47.061126+0000","last_peered":"2024-04-25T14:10:47.061126+0000","last_clean":"2024-04-25T14:10:47.061126+0000","last_became_active":"2024-04-25T14:10:46.081153+0000","last_became_peered":"2024-04-25T14:10:46.081153+0000","last_unstale":"2024-04-25T14:10:47.061126+0000","last_undegraded":"2024-04-25T14:10:47.061126+0000","last_fullsized":"2024-04-25T14:10:47.061126+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:10.598819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060831+0000","last_change":"2024-04-25T14:10:45.049002+0000","last_active":"2024-04-25T14:10:47.060831+0000","last_peered":"2024-04-25T14:10:47.060831+0000","last_clean":"2024-04-25T14:10:47.060831+0000","last_became_active":"2024-04-25T14:10:45.048755+0000","last_became_peered":"2024-04-25T14:10:45.048755+0000","last_unstale":"2024-04-25T14:10:47.060831+0000","last_undegraded":"2024-04-25T14:10:47.060831+0000","last_fullsized":"2024-04-25T14:10:47.060831+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:01.990667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.059895+0000","last_change":"2024-04-25T14:10:46.065779+0000","last_active":"2024-04-25T14:10:47.059895+0000","last_peered":"2024-04-25T14:10:47.059895+0000","last_clean":"2024-04-25T14:10:47.059895+0000","last_became_active":"2024-04-25T14:10:46.065454+0000","last_became_peered":"2024-04-25T14:10:46.065454+0000","last_unstale":"2024-04-25T14:10:47.059895+0000","last_undegraded":"2024-04-25T14:10:47.059895+0000","last_fullsized":"2024-04-25T14:10:47.059895+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:13.479334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"125'8","reported_seq":22,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202177+0000","last_change":"2024-04-25T14:10:45.051770+0000","last_active":"2024-04-25T14:10:47.202177+0000","last_peered":"2024-04-25T14:10:47.202177+0000","last_clean":"2024-04-25T14:10:47.202177+0000","last_became_active":"2024-04-25T14:10:45.051446+0000","last_became_peered":"2024-04-25T14:10:45.051446+0000","last_unstale":"2024-04-25T14:10:47.202177+0000","last_undegraded":"2024-04-25T14:10:47.202177+0000","last_fullsized":"2024-04-25T14:10:47.202177+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:55.116141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060459+0000","last_change":"2024-04-25T14:10:46.068019+0000","last_active":"2024-04-25T14:10:47.060459+0000","last_peered":"2024-04-25T14:10:47.060459+0000","last_clean":"2024-04-25T14:10:47.060459+0000","last_became_active":"2024-04-25T14:10:46.067686+0000","last_became_peered":"2024-04-25T14:10:46.067686+0000","last_unstale":"2024-04-25T14:10:47.060459+0000","last_undegraded":"2024-04-25T14:10:47.060459+0000","last_fullsized":"2024-04-25T14:10:47.060459+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:21.296998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.070063+0000","last_change":"2024-04-25T14:10:45.037175+0000","last_active":"2024-04-25T14:10:47.070063+0000","last_peered":"2024-04-25T14:10:47.070063+0000","last_clean":"2024-04-25T14:10:47.070063+0000","last_became_active":"2024-04-25T14:10:45.036881+0000","last_became_peered":"2024-04-25T14:10:45.036881+0000","last_unstale":"2024-04-25T14:10:47.070063+0000","last_undegraded":"2024-04-25T14:10:47.070063+0000","last_fullsized":"2024-04-25T14:10:47.070063+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:33.243994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060331+0000","last_change":"2024-04-25T14:10:46.072610+0000","last_active":"2024-04-25T14:10:47.060331+0000","last_peered":"2024-04-25T14:10:47.060331+0000","last_clean":"2024-04-25T14:10:47.060331+0000","last_became_active":"2024-04-25T14:10:46.072272+0000","last_became_peered":"2024-04-25T14:10:46.072272+0000","last_unstale":"2024-04-25T14:10:47.060331+0000","last_undegraded":"2024-04-25T14:10:47.060331+0000","last_fullsized":"2024-04-25T14:10:47.060331+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:59.236859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.061856+0000","last_change":"2024-04-25T14:10:45.052421+0000","last_active":"2024-04-25T14:10:47.061856+0000","last_peered":"2024-04-25T14:10:47.061856+0000","last_clean":"2024-04-25T14:10:47.061856+0000","last_became_active":"2024-04-25T14:10:45.052078+0000","last_became_peered":"2024-04-25T14:10:45.052078+0000","last_unstale":"2024-04-25T14:10:47.061856+0000","last_undegraded":"2024-04-25T14:10:47.061856+0000","last_fullsized":"2024-04-25T14:10:47.061856+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:22.669468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060764+0000","last_change":"2024-04-25T14:10:46.078401+0000","last_active":"2024-04-25T14:10:47.060764+0000","last_peered":"2024-04-25T14:10:47.060764+0000","last_clean":"2024-04-25T14:10:47.060764+0000","last_became_active":"2024-04-25T14:10:46.074147+0000","last_became_peered":"2024-04-25T14:10:46.074147+0000","last_unstale":"2024-04-25T14:10:47.060764+0000","last_undegraded":"2024-04-25T14:10:47.060764+0000","last_fullsized":"2024-04-25T14:10:47.060764+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:09.225406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202946+0000","last_change":"2024-04-25T14:10:45.056493+0000","last_active":"2024-04-25T14:10:47.202946+0000","last_peered":"2024-04-25T14:10:47.202946+0000","last_clean":"2024-04-25T14:10:47.202946+0000","last_became_active":"2024-04-25T14:10:45.055271+0000","last_became_peered":"2024-04-25T14:10:45.055271+0000","last_unstale":"2024-04-25T14:10:47.202946+0000","last_undegraded":"2024-04-25T14:10:47.202946+0000","last_fullsized":"2024-04-25T14:10:47.202946+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:22.629958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.065043+0000","last_change":"2024-04-25T14:10:46.071797+0000","last_active":"2024-04-25T14:10:47.065043+0000","last_peered":"2024-04-25T14:10:47.065043+0000","last_clean":"2024-04-25T14:10:47.065043+0000","last_became_active":"2024-04-25T14:10:46.071504+0000","last_became_peered":"2024-04-25T14:10:46.071504+0000","last_unstale":"2024-04-25T14:10:47.065043+0000","last_undegraded":"2024-04-25T14:10:47.065043+0000","last_fullsized":"2024-04-25T14:10:47.065043+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:41.808794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060046+0000","last_change":"2024-04-25T14:10:45.055677+0000","last_active":"2024-04-25T14:10:47.060046+0000","last_peered":"2024-04-25T14:10:47.060046+0000","last_clean":"2024-04-25T14:10:47.060046+0000","last_became_active":"2024-04-25T14:10:45.054262+0000","last_became_peered":"2024-04-25T14:10:45.054262+0000","last_unstale":"2024-04-25T14:10:47.060046+0000","last_undegraded":"2024-04-25T14:10:47.060046+0000","last_fullsized":"2024-04-25T14:10:47.060046+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:36.988908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.059967+0000","last_change":"2024-04-25T14:10:46.062938+0000","last_active":"2024-04-25T14:10:47.059967+0000","last_peered":"2024-04-25T14:10:47.059967+0000","last_clean":"2024-04-25T14:10:47.059967+0000","last_became_active":"2024-04-25T14:10:46.062652+0000","last_became_peered":"2024-04-25T14:10:46.062652+0000","last_unstale":"2024-04-25T14:10:47.059967+0000","last_undegraded":"2024-04-25T14:10:47.059967+0000","last_fullsized":"2024-04-25T14:10:47.059967+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:34.869650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.067776+0000","last_change":"2024-04-25T14:10:45.047266+0000","last_active":"2024-04-25T14:10:47.067776+0000","last_peered":"2024-04-25T14:10:47.067776+0000","last_clean":"2024-04-25T14:10:47.067776+0000","last_became_active":"2024-04-25T14:10:45.046928+0000","last_became_peered":"2024-04-25T14:10:45.046928+0000","last_unstale":"2024-04-25T14:10:47.067776+0000","last_undegraded":"2024-04-25T14:10:47.067776+0000","last_fullsized":"2024-04-25T14:10:47.067776+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:33.243994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.203086+0000","last_change":"2024-04-25T14:10:46.068140+0000","last_active":"2024-04-25T14:10:47.203086+0000","last_peered":"2024-04-25T14:10:47.203086+0000","last_clean":"2024-04-25T14:10:47.203086+0000","last_became_active":"2024-04-25T14:10:46.067559+0000","last_became_peered":"2024-04-25T14:10:46.067559+0000","last_unstale":"2024-04-25T14:10:47.203086+0000","last_undegraded":"2024-04-25T14:10:47.203086+0000","last_fullsized":"2024-04-25T14:10:47.203086+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:58.755296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.201997+0000","last_change":"2024-04-25T14:10:45.057160+0000","last_active":"2024-04-25T14:10:47.201997+0000","last_peered":"2024-04-25T14:10:47.201997+0000","last_clean":"2024-04-25T14:10:47.201997+0000","last_became_active":"2024-04-25T14:10:45.056871+0000","last_became_peered":"2024-04-25T14:10:45.056871+0000","last_unstale":"2024-04-25T14:10:47.201997+0000","last_undegraded":"2024-04-25T14:10:47.201997+0000","last_fullsized":"2024-04-25T14:10:47.201997+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:20.109193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060037+0000","last_change":"2024-04-25T14:10:46.068918+0000","last_active":"2024-04-25T14:10:47.060037+0000","last_peered":"2024-04-25T14:10:47.060037+0000","last_clean":"2024-04-25T14:10:47.060037+0000","last_became_active":"2024-04-25T14:10:46.068604+0000","last_became_peered":"2024-04-25T14:10:46.068604+0000","last_unstale":"2024-04-25T14:10:47.060037+0000","last_undegraded":"2024-04-25T14:10:47.060037+0000","last_fullsized":"2024-04-25T14:10:47.060037+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:17.255415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060420+0000","last_change":"2024-04-25T14:10:45.059841+0000","last_active":"2024-04-25T14:10:47.060420+0000","last_peered":"2024-04-25T14:10:47.060420+0000","last_clean":"2024-04-25T14:10:47.060420+0000","last_became_active":"2024-04-25T14:10:45.059609+0000","last_became_peered":"2024-04-25T14:10:45.059609+0000","last_unstale":"2024-04-25T14:10:47.060420+0000","last_undegraded":"2024-04-25T14:10:47.060420+0000","last_fullsized":"2024-04-25T14:10:47.060420+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:10.566680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.070012+0000","last_change":"2024-04-25T14:10:46.072628+0000","last_active":"2024-04-25T14:10:47.070012+0000","last_peered":"2024-04-25T14:10:47.070012+0000","last_clean":"2024-04-25T14:10:47.070012+0000","last_became_active":"2024-04-25T14:10:46.071215+0000","last_became_peered":"2024-04-25T14:10:46.071215+0000","last_unstale":"2024-04-25T14:10:47.070012+0000","last_undegraded":"2024-04-25T14:10:47.070012+0000","last_fullsized":"2024-04-25T14:10:47.070012+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:36.026849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202287+0000","last_change":"2024-04-25T14:10:45.057442+0000","last_active":"2024-04-25T14:10:47.202287+0000","last_peered":"2024-04-25T14:10:47.202287+0000","last_clean":"2024-04-25T14:10:47.202287+0000","last_became_active":"2024-04-25T14:10:45.057077+0000","last_became_peered":"2024-04-25T14:10:45.057077+0000","last_unstale":"2024-04-25T14:10:47.202287+0000","last_undegraded":"2024-04-25T14:10:47.202287+0000","last_fullsized":"2024-04-25T14:10:47.202287+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:38.714815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.059736+0000","last_change":"2024-04-25T14:10:46.075657+0000","last_active":"2024-04-25T14:10:47.059736+0000","last_peered":"2024-04-25T14:10:47.059736+0000","last_clean":"2024-04-25T14:10:47.059736+0000","last_became_active":"2024-04-25T14:10:46.075357+0000","last_became_peered":"2024-04-25T14:10:46.075357+0000","last_unstale":"2024-04-25T14:10:47.059736+0000","last_undegraded":"2024-04-25T14:10:47.059736+0000","last_fullsized":"2024-04-25T14:10:47.059736+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:15.160432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.203158+0000","last_change":"2024-04-25T14:10:45.056735+0000","last_active":"2024-04-25T14:10:47.203158+0000","last_peered":"2024-04-25T14:10:47.203158+0000","last_clean":"2024-04-25T14:10:47.203158+0000","last_became_active":"2024-04-25T14:10:45.052530+0000","last_became_peered":"2024-04-25T14:10:45.052530+0000","last_unstale":"2024-04-25T14:10:47.203158+0000","last_undegraded":"2024-04-25T14:10:47.203158+0000","last_fullsized":"2024-04-25T14:10:47.203158+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:32.672468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060737+0000","last_change":"2024-04-25T14:10:46.067306+0000","last_active":"2024-04-25T14:10:47.060737+0000","last_peered":"2024-04-25T14:10:47.060737+0000","last_clean":"2024-04-25T14:10:47.060737+0000","last_became_active":"2024-04-25T14:10:46.066990+0000","last_became_peered":"2024-04-25T14:10:46.066990+0000","last_unstale":"2024-04-25T14:10:47.060737+0000","last_undegraded":"2024-04-25T14:10:47.060737+0000","last_fullsized":"2024-04-25T14:10:47.060737+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:56.449255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"126'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.067866+0000","last_change":"2024-04-25T14:10:45.046573+0000","last_active":"2024-04-25T14:10:47.067866+0000","last_peered":"2024-04-25T14:10:47.067866+0000","last_clean":"2024-04-25T14:10:47.067866+0000","last_became_active":"2024-04-25T14:10:45.046226+0000","last_became_peered":"2024-04-25T14:10:45.046226+0000","last_unstale":"2024-04-25T14:10:47.067866+0000","last_undegraded":"2024-04-25T14:10:47.067866+0000","last_fullsized":"2024-04-25T14:10:47.067866+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:55.116141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060418+0000","last_change":"2024-04-25T14:10:46.076749+0000","last_active":"2024-04-25T14:10:47.060418+0000","last_peered":"2024-04-25T14:10:47.060418+0000","last_clean":"2024-04-25T14:10:47.060418+0000","last_became_active":"2024-04-25T14:10:46.074554+0000","last_became_peered":"2024-04-25T14:10:46.074554+0000","last_unstale":"2024-04-25T14:10:47.060418+0000","last_undegraded":"2024-04-25T14:10:47.060418+0000","last_fullsized":"2024-04-25T14:10:47.060418+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:34.869650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.067765+0000","last_change":"2024-04-25T14:10:45.055809+0000","last_active":"2024-04-25T14:10:47.067765+0000","last_peered":"2024-04-25T14:10:47.067765+0000","last_clean":"2024-04-25T14:10:47.067765+0000","last_became_active":"2024-04-25T14:10:45.055480+0000","last_became_peered":"2024-04-25T14:10:45.055480+0000","last_unstale":"2024-04-25T14:10:47.067765+0000","last_undegraded":"2024-04-25T14:10:47.067765+0000","last_fullsized":"2024-04-25T14:10:47.067765+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:22.629958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.065197+0000","last_change":"2024-04-25T14:10:46.065927+0000","last_active":"2024-04-25T14:10:47.065197+0000","last_peered":"2024-04-25T14:10:47.065197+0000","last_clean":"2024-04-25T14:10:47.065197+0000","last_became_active":"2024-04-25T14:10:46.065621+0000","last_became_peered":"2024-04-25T14:10:46.065621+0000","last_unstale":"2024-04-25T14:10:47.065197+0000","last_undegraded":"2024-04-25T14:10:47.065197+0000","last_fullsized":"2024-04-25T14:10:47.065197+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:11.942560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202551+0000","last_change":"2024-04-25T14:10:45.058112+0000","last_active":"2024-04-25T14:10:47.202551+0000","last_peered":"2024-04-25T14:10:47.202551+0000","last_clean":"2024-04-25T14:10:47.202551+0000","last_became_active":"2024-04-25T14:10:45.057825+0000","last_became_peered":"2024-04-25T14:10:45.057825+0000","last_unstale":"2024-04-25T14:10:47.202551+0000","last_undegraded":"2024-04-25T14:10:47.202551+0000","last_fullsized":"2024-04-25T14:10:47.202551+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:01.583932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202570+0000","last_change":"2024-04-25T14:10:46.069554+0000","last_active":"2024-04-25T14:10:47.202570+0000","last_peered":"2024-04-25T14:10:47.202570+0000","last_clean":"2024-04-25T14:10:47.202570+0000","last_became_active":"2024-04-25T14:10:46.069112+0000","last_became_peered":"2024-04-25T14:10:46.069112+0000","last_unstale":"2024-04-25T14:10:47.202570+0000","last_undegraded":"2024-04-25T14:10:47.202570+0000","last_fullsized":"2024-04-25T14:10:47.202570+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:27.987770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.069756+0000","last_change":"2024-04-25T14:10:45.049671+0000","last_active":"2024-04-25T14:10:47.069756+0000","last_peered":"2024-04-25T14:10:47.069756+0000","last_clean":"2024-04-25T14:10:47.069756+0000","last_became_active":"2024-04-25T14:10:45.049375+0000","last_became_peered":"2024-04-25T14:10:45.049375+0000","last_unstale":"2024-04-25T14:10:47.069756+0000","last_undegraded":"2024-04-25T14:10:47.069756+0000","last_fullsized":"2024-04-25T14:10:47.069756+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:03.729472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202837+0000","last_change":"2024-04-25T14:10:46.073752+0000","last_active":"2024-04-25T14:10:47.202837+0000","last_peered":"2024-04-25T14:10:47.202837+0000","last_clean":"2024-04-25T14:10:47.202837+0000","last_became_active":"2024-04-25T14:10:46.073446+0000","last_became_peered":"2024-04-25T14:10:46.073446+0000","last_unstale":"2024-04-25T14:10:47.202837+0000","last_undegraded":"2024-04-25T14:10:47.202837+0000","last_fullsized":"2024-04-25T14:10:47.202837+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:57.190018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060879+0000","last_change":"2024-04-25T14:10:45.053154+0000","last_active":"2024-04-25T14:10:47.060879+0000","last_peered":"2024-04-25T14:10:47.060879+0000","last_clean":"2024-04-25T14:10:47.060879+0000","last_became_active":"2024-04-25T14:10:45.052842+0000","last_became_peered":"2024-04-25T14:10:45.052842+0000","last_unstale":"2024-04-25T14:10:47.060879+0000","last_undegraded":"2024-04-25T14:10:47.060879+0000","last_fullsized":"2024-04-25T14:10:47.060879+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:35.020438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202856+0000","last_change":"2024-04-25T14:10:46.067406+0000","last_active":"2024-04-25T14:10:47.202856+0000","last_peered":"2024-04-25T14:10:47.202856+0000","last_clean":"2024-04-25T14:10:47.202856+0000","last_became_active":"2024-04-25T14:10:46.067024+0000","last_became_peered":"2024-04-25T14:10:46.067024+0000","last_unstale":"2024-04-25T14:10:47.202856+0000","last_undegraded":"2024-04-25T14:10:47.202856+0000","last_fullsized":"2024-04-25T14:10:47.202856+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:59.236859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.202279+0000","last_change":"2024-04-25T14:10:45.058009+0000","last_active":"2024-04-25T14:10:47.202279+0000","last_peered":"2024-04-25T14:10:47.202279+0000","last_clean":"2024-04-25T14:10:47.202279+0000","last_became_active":"2024-04-25T14:10:45.057724+0000","last_became_peered":"2024-04-25T14:10:45.057724+0000","last_unstale":"2024-04-25T14:10:47.202279+0000","last_undegraded":"2024-04-25T14:10:47.202279+0000","last_fullsized":"2024-04-25T14:10:47.202279+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:19.174254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060840+0000","last_change":"2024-04-25T14:10:46.078777+0000","last_active":"2024-04-25T14:10:47.060840+0000","last_peered":"2024-04-25T14:10:47.060840+0000","last_clean":"2024-04-25T14:10:47.060840+0000","last_became_active":"2024-04-25T14:10:46.073967+0000","last_became_peered":"2024-04-25T14:10:46.073967+0000","last_unstale":"2024-04-25T14:10:47.060840+0000","last_undegraded":"2024-04-25T14:10:47.060840+0000","last_fullsized":"2024-04-25T14:10:47.060840+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:38:21.375099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.059887+0000","last_change":"2024-04-25T14:10:45.043256+0000","last_active":"2024-04-25T14:10:47.059887+0000","last_peered":"2024-04-25T14:10:47.059887+0000","last_clean":"2024-04-25T14:10:47.059887+0000","last_became_active":"2024-04-25T14:10:45.042731+0000","last_became_peered":"2024-04-25T14:10:45.042731+0000","last_unstale":"2024-04-25T14:10:47.059887+0000","last_undegraded":"2024-04-25T14:10:47.059887+0000","last_fullsized":"2024-04-25T14:10:47.059887+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:03.495936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.061539+0000","last_change":"2024-04-25T14:10:46.070796+0000","last_active":"2024-04-25T14:10:47.061539+0000","last_peered":"2024-04-25T14:10:47.061539+0000","last_clean":"2024-04-25T14:10:47.061539+0000","last_became_active":"2024-04-25T14:10:46.070503+0000","last_became_peered":"2024-04-25T14:10:46.070503+0000","last_unstale":"2024-04-25T14:10:47.061539+0000","last_undegraded":"2024-04-25T14:10:47.061539+0000","last_fullsized":"2024-04-25T14:10:47.061539+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:45.026666+0000","last_clean_scrub_stamp":"2024-04-25T14:10:45.026666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:36.026849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:47.060570+0000","last_change":"2024-04-25T14:10:45.053553+0000","last_active":"2024-04-25T14:10:47.060570+0000","last_peered":"2024-04-25T14:10:47.060570+0000","last_clean":"2024-04-25T14:10:47.060570+0000","last_became_active":"2024-04-25T14:10:45.053178+0000","last_became_peered":"2024-04-25T14:10:45.053178+0000","last_unstale":"2024-04-25T14:10:47.060570+0000","last_undegraded":"2024-04-25T14:10:47.060570+0000","last_fullsized":"2024-04-25T14:10:47.060570+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:10:44.020254+0000","last_clean_scrub_stamp":"2024-04-25T14:10:44.020254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:03.495936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"126'2649","reported_seq":3042,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T14:10:50.606748+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:10:50.606748+0000","last_peered":"2024-04-25T14:10:50.606748+0000","last_clean":"2024-04-25T14:10:50.606748+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:10:50.606748+0000","last_undegraded":"2024-04-25T14:10:50.606748+0000","last_fullsized":"2024-04-25T14:10:50.606748+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":2649,"log_dups_size":0,"ondisk_log_size":2649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":536,"num_read_kb":1396,"num_write":187,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":536,"num_read_kb":1396,"num_write":187,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2649,"ondisk_log_size":2649,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574931,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41632,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330208,"statfs":{"total":96636764160,"available":96594132992,"internally_reserved":0,"allocated":2187264,"data_stored":1380400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574932,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1585152,"data_stored":789002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574933,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32724,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1581056,"data_stored":788352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32720,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1576960,"data_stored":788330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574932,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1593344,"data_stored":790012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574931,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1585152,"data_stored":788946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32724,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1581056,"data_stored":788400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574932,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41624,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330216,"statfs":{"total":96636764160,"available":96594141184,"internally_reserved":0,"allocated":2179072,"data_stored":1378754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:10:55.313 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T14:10:55.314 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T14:10:55.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.513+0000 7fdd11c00640 1 Processor -- start 2024-04-25T14:10:55.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.513+0000 7fdd11c00640 1 -- start start 2024-04-25T14:10:55.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.513+0000 7fdd11c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdd0c0cfca0 0x7fdd0c06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:55.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.513+0000 7fdd11c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd0c072bd0 con 0x7fdd0c0d16e0 2024-04-25T14:10:55.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.513+0000 7fdd11c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd0c06f930 con 0x7fdd0c0730b0 2024-04-25T14:10:55.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.513+0000 7fdd11c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdd0c070470 con 0x7fdd0c0cfca0 2024-04-25T14:10:55.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.513+0000 7fdd0a000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fdd0c0d16e0 0x7fdd0c0d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54994/0 (socket says 172.21.15.86:54994) 2024-04-25T14:10:55.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.513+0000 7fdd0a000640 1 -- 172.21.15.86:0/2000683131 learned_addr learned my addr 172.21.15.86:0/2000683131 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:55.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.513+0000 7fdd0b400640 1 --1- 172.21.15.86:0/2000683131 >> v1:172.21.15.155:6789/0 conn(0x7fdd0c0730b0 0x7fdd0c0734c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:58052/0 (socket says 172.21.15.86:58052) 2024-04-25T14:10:55.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.513+0000 7fdd0aa00640 1 --2- 172.21.15.86:0/2000683131 >> v2:172.21.15.86:3300/0 conn(0x7fdd0c0cfca0 0x7fdd0c06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:55.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.513+0000 7fdd09600640 1 -- 172.21.15.86:0/2000683131 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1834430144 0 0) 0x7fdd0c072bd0 con 0x7fdd0c0d16e0 2024-04-25T14:10:55.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.513+0000 7fdd09600640 1 -- 172.21.15.86:0/2000683131 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdcf0003530 con 0x7fdd0c0d16e0 2024-04-25T14:10:55.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.517+0000 7fdd09600640 1 -- 172.21.15.86:0/2000683131 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1802863292 0 0) 0x7fdd0c06f930 con 0x7fdd0c0730b0 2024-04-25T14:10:55.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.517+0000 7fdd09600640 1 -- 172.21.15.86:0/2000683131 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd0c072bd0 con 0x7fdd0c0730b0 2024-04-25T14:10:55.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.517+0000 7fdd0aa00640 1 -- 172.21.15.86:0/2000683131 >> v1:172.21.15.155:6789/0 conn(0x7fdd0c0730b0 legacy=0x7fdd0c0734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:55.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.517+0000 7fdd0aa00640 1 -- 172.21.15.86:0/2000683131 >> v1:172.21.15.86:6789/0 conn(0x7fdd0c0d16e0 legacy=0x7fdd0c0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:55.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.517+0000 7fdd0aa00640 1 -- 172.21.15.86:0/2000683131 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd0c06fbd0 con 0x7fdd0c0cfca0 2024-04-25T14:10:55.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.517+0000 7fdd0aa00640 1 --2- 172.21.15.86:0/2000683131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd0c0cfca0 0x7fdd0c06f3f0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0097c0 tx=0x7fdcfc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81cf38b68fee18f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:55.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.517+0000 7fdd09600640 1 -- 172.21.15.86:0/2000683131 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc0b1070 con 0x7fdd0c0cfca0 2024-04-25T14:10:55.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.517+0000 7fdd09600640 1 -- 172.21.15.86:0/2000683131 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcfc0ac410 con 0x7fdd0c0cfca0 2024-04-25T14:10:55.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.517+0000 7fdd09600640 1 -- 172.21.15.86:0/2000683131 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c0743e0 con 0x7fdd0c0cfca0 2024-04-25T14:10:55.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.521+0000 7fdd11c00640 1 -- 172.21.15.86:0/2000683131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd0c0cfca0 msgr2=0x7fdd0c06f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:55.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.521+0000 7fdd11c00640 1 --2- 172.21.15.86:0/2000683131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd0c0cfca0 0x7fdd0c06f3f0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0097c0 tx=0x7fdcfc01c4a0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.521+0000 7fdd11c00640 1 -- 172.21.15.86:0/2000683131 shutdown_connections 2024-04-25T14:10:55.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.521+0000 7fdd11c00640 1 --2- 172.21.15.86:0/2000683131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd0c0cfca0 0x7fdd0c06f3f0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.521+0000 7fdd11c00640 1 -- 172.21.15.86:0/2000683131 >> 172.21.15.86:0/2000683131 conn(0x7fdd0c0b4650 msgr2=0x7fdd0c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:55.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.521+0000 7fdd11c00640 1 -- 172.21.15.86:0/2000683131 shutdown_connections 2024-04-25T14:10:55.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.521+0000 7fdd11c00640 1 -- 172.21.15.86:0/2000683131 wait complete. 2024-04-25T14:10:55.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.521+0000 7fdd11c00640 1 Processor -- start 2024-04-25T14:10:55.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.521+0000 7fdd11c00640 1 -- start start 2024-04-25T14:10:55.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.525+0000 7fdd11c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd0c0730b0 0x7fdd0c09a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:55.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.525+0000 7fdd11c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd0c0cfca0 0x7fdd0c156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:55.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.525+0000 7fdd11c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd0c0d16e0 0x7fdd0c0b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:55.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.525+0000 7fdd11c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c16aa80 con 0x7fdd0c0cfca0 2024-04-25T14:10:55.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.525+0000 7fdd11c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c16a900 con 0x7fdd0c0730b0 2024-04-25T14:10:55.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.525+0000 7fdd11c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdd0c16ac00 con 0x7fdd0c0d16e0 2024-04-25T14:10:55.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.525+0000 7fdd0a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd0c0cfca0 0x7fdd0c156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:55.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.525+0000 7fdd0aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd0c0730b0 0x7fdd0c09a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:55.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.525+0000 7fdd0a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd0c0cfca0 0x7fdd0c156a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46260/0 (socket says 172.21.15.86:46260) 2024-04-25T14:10:55.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.525+0000 7fdd0a000640 1 -- 172.21.15.86:0/592256535 learned_addr learned my addr 172.21.15.86:0/592256535 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:55.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.525+0000 7fdd0b400640 1 --2- 172.21.15.86:0/592256535 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd0c0d16e0 0x7fdd0c0b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:55.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.525+0000 7fdd0a000640 1 -- 172.21.15.86:0/592256535 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd0c0d16e0 msgr2=0x7fdd0c0b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:55.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.525+0000 7fdd0a000640 1 --2- 172.21.15.86:0/592256535 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd0c0d16e0 0x7fdd0c0b1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.525+0000 7fdd0a000640 1 -- 172.21.15.86:0/592256535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd0c0730b0 msgr2=0x7fdd0c09a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:55.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.525+0000 7fdd0a000640 1 --2- 172.21.15.86:0/592256535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd0c0730b0 0x7fdd0c09a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.525+0000 7fdd0a000640 1 -- 172.21.15.86:0/592256535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd0c1573b0 con 0x7fdd0c0cfca0 2024-04-25T14:10:55.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.525+0000 7fdd0aa00640 1 --2- 172.21.15.86:0/592256535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd0c0730b0 0x7fdd0c09a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:10:55.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.525+0000 7fdd0a000640 1 --2- 172.21.15.86:0/592256535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd0c0cfca0 0x7fdd0c156a40 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800c420 tx=0x7fdcf800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:55.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.525+0000 7fdd02a00640 1 -- 172.21.15.86:0/592256535 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8017070 con 0x7fdd0c0cfca0 2024-04-25T14:10:55.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.525+0000 7fdd02a00640 1 -- 172.21.15.86:0/592256535 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcf8005cf0 con 0x7fdd0c0cfca0 2024-04-25T14:10:55.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.525+0000 7fdd11c00640 1 -- 172.21.15.86:0/592256535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd0c157690 con 0x7fdd0c0cfca0 2024-04-25T14:10:55.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.525+0000 7fdd02a00640 1 -- 172.21.15.86:0/592256535 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf80072d0 con 0x7fdd0c0cfca0 2024-04-25T14:10:55.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.525+0000 7fdd11c00640 1 -- 172.21.15.86:0/592256535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd0c0ca6c0 con 0x7fdd0c0cfca0 2024-04-25T14:10:55.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.525+0000 7fdd11c00640 1 -- 172.21.15.86:0/592256535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd0c09f4b0 con 0x7fdd0c0cfca0 2024-04-25T14:10:55.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.525+0000 7fdd02a00640 1 -- 172.21.15.86:0/592256535 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7fdcf801a070 con 0x7fdd0c0cfca0 2024-04-25T14:10:55.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.529+0000 7fdd02a00640 1 --2- 172.21.15.86:0/592256535 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fdcec06f4c0 0x7fdcec0719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:55.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.529+0000 7fdd02a00640 1 -- 172.21.15.86:0/592256535 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6101+0+0 (secure 0 0 0) 0x7fdcf808e610 con 0x7fdd0c0cfca0 2024-04-25T14:10:55.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.529+0000 7fdd0aa00640 1 --2- 172.21.15.86:0/592256535 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fdcec06f4c0 0x7fdcec0719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:55.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.529+0000 7fdd0aa00640 1 --2- 172.21.15.86:0/592256535 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fdcec06f4c0 0x7fdcec0719d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0ae000 tx=0x7fdcfc008040 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:55.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.533+0000 7fdd02a00640 1 -- 172.21.15.86:0/592256535 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdcf805adc0 con 0x7fdd0c0cfca0 2024-04-25T14:10:55.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.761+0000 7fdd11c00640 1 -- 172.21.15.86:0/592256535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fdd0c157bb0 con 0x7fdd0c0cfca0 2024-04-25T14:10:55.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.765+0000 7fdd02a00640 1 -- 172.21.15.86:0/592256535 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v28) ==== 190+0+147 (secure 0 0 0) 0x7fdcf8063400 con 0x7fdd0c0cfca0 2024-04-25T14:10:55.763 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-04-25T14:10:55.764 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCLYipm5u4xHRAACDFe3Eb4KLrsdLj4eFe7mA== 2024-04-25T14:10:55.764 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-04-25T14:10:55.764 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-04-25T14:10:55.764 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T14:10:55.764 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.0 2024-04-25T14:10:55.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.769+0000 7fdd11c00640 1 -- 172.21.15.86:0/592256535 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fdcec06f4c0 msgr2=0x7fdcec0719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:55.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.769+0000 7fdd11c00640 1 --2- 172.21.15.86:0/592256535 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fdcec06f4c0 0x7fdcec0719d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0ae000 tx=0x7fdcfc008040 comp rx=0 tx=0).stop 2024-04-25T14:10:55.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.769+0000 7fdd11c00640 1 -- 172.21.15.86:0/592256535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd0c0cfca0 msgr2=0x7fdd0c156a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:55.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.769+0000 7fdd11c00640 1 --2- 172.21.15.86:0/592256535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd0c0cfca0 0x7fdd0c156a40 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800c420 tx=0x7fdcf800c8e0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.769+0000 7fdd11c00640 1 -- 172.21.15.86:0/592256535 shutdown_connections 2024-04-25T14:10:55.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.769+0000 7fdd11c00640 1 --2- 172.21.15.86:0/592256535 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd0c0d16e0 0x7fdd0c0b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.769+0000 7fdd11c00640 1 --2- 172.21.15.86:0/592256535 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fdcec06f4c0 0x7fdcec0719d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.769+0000 7fdd11c00640 1 --2- 172.21.15.86:0/592256535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd0c0cfca0 0x7fdd0c156a40 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.769+0000 7fdd11c00640 1 --2- 172.21.15.86:0/592256535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd0c0730b0 0x7fdd0c09a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.769+0000 7fdd11c00640 1 -- 172.21.15.86:0/592256535 >> 172.21.15.86:0/592256535 conn(0x7fdd0c0b4650 msgr2=0x7fdd0c155030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:55.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.769+0000 7fdd11c00640 1 -- 172.21.15.86:0/592256535 shutdown_connections 2024-04-25T14:10:55.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.769+0000 7fdd11c00640 1 -- 172.21.15.86:0/592256535 wait complete. 2024-04-25T14:10:55.787 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:10:55.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.937+0000 7f52ddc00640 1 Processor -- start 2024-04-25T14:10:55.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.937+0000 7f52ddc00640 1 -- start start 2024-04-25T14:10:55.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.937+0000 7f52ddc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f52d80cfca0 0x7f52d806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:55.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.937+0000 7f52ddc00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f52d80d16e0 0x7f52d80d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:55.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.937+0000 7f52ddc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52d80d2030 con 0x7f52d806c670 2024-04-25T14:10:55.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.937+0000 7f52ddc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f52d8070470 con 0x7f52d80cfca0 2024-04-25T14:10:55.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.937+0000 7f52ddc00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f52d8072bd0 con 0x7f52d80d16e0 2024-04-25T14:10:55.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.941+0000 7f52d6a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f52d80cfca0 0x7f52d806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:55.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.941+0000 7f52d7400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f52d806c670 0x7f52d806ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54998/0 (socket says 172.21.15.86:54998) 2024-04-25T14:10:55.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.941+0000 7f52d7400640 1 -- 172.21.15.86:0/1508228824 learned_addr learned my addr 172.21.15.86:0/1508228824 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:55.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.941+0000 7f52ce000640 1 --2- 172.21.15.86:0/1508228824 >> v2:172.21.15.155:3300/0 conn(0x7f52d80d16e0 0x7f52d80d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:55.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.941+0000 7f52d6a00640 1 -- 172.21.15.86:0/1508228824 >> v2:172.21.15.155:3300/0 conn(0x7f52d80d16e0 msgr2=0x7f52d80d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:55.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.941+0000 7f52d6a00640 1 --2- 172.21.15.86:0/1508228824 >> v2:172.21.15.155:3300/0 conn(0x7f52d80d16e0 0x7f52d80d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.941+0000 7f52d6a00640 1 -- 172.21.15.86:0/1508228824 >> v1:172.21.15.86:6789/0 conn(0x7f52d806c670 legacy=0x7f52d806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:55.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.941+0000 7f52d6a00640 1 -- 172.21.15.86:0/1508228824 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d806f930 con 0x7f52d80cfca0 2024-04-25T14:10:55.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.941+0000 7f52d6000640 1 -- 172.21.15.86:0/1508228824 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 626554710 0 0) 0x7f52d00014a0 con 0x7f52d806c670 2024-04-25T14:10:55.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.941+0000 7f52d6a00640 1 --2- 172.21.15.86:0/1508228824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52d80cfca0 0x7f52d806f3f0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f52d80729c0 tx=0x7f52c801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d783b932b3f2a06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:55.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.941+0000 7f52d6000640 1 -- 172.21.15.86:0/1508228824 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c80b1070 con 0x7f52d80cfca0 2024-04-25T14:10:55.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.941+0000 7f52d6000640 1 -- 172.21.15.86:0/1508228824 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c80ac5f0 con 0x7f52d80cfca0 2024-04-25T14:10:55.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.941+0000 7f52d6000640 1 -- 172.21.15.86:0/1508228824 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c80ac9c0 con 0x7f52d80cfca0 2024-04-25T14:10:55.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.945+0000 7f52ddc00640 1 -- 172.21.15.86:0/1508228824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52d80cfca0 msgr2=0x7f52d806f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:55.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.945+0000 7f52ddc00640 1 --2- 172.21.15.86:0/1508228824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52d80cfca0 0x7f52d806f3f0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f52d80729c0 tx=0x7f52c801c480 comp rx=0 tx=0).stop 2024-04-25T14:10:55.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.945+0000 7f52ddc00640 1 -- 172.21.15.86:0/1508228824 shutdown_connections 2024-04-25T14:10:55.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.945+0000 7f52ddc00640 1 --2- 172.21.15.86:0/1508228824 >> v2:172.21.15.155:3300/0 conn(0x7f52d80d16e0 0x7f52d80d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.945+0000 7f52ddc00640 1 --2- 172.21.15.86:0/1508228824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52d80cfca0 0x7f52d806f3f0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.945+0000 7f52ddc00640 1 -- 172.21.15.86:0/1508228824 >> 172.21.15.86:0/1508228824 conn(0x7f52d80b4650 msgr2=0x7f52d80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:55.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.945+0000 7f52ddc00640 1 -- 172.21.15.86:0/1508228824 shutdown_connections 2024-04-25T14:10:55.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.945+0000 7f52ddc00640 1 -- 172.21.15.86:0/1508228824 wait complete. 2024-04-25T14:10:55.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.945+0000 7f52ddc00640 1 Processor -- start 2024-04-25T14:10:55.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.945+0000 7f52ddc00640 1 -- start start 2024-04-25T14:10:55.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.949+0000 7f52ddc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52d806c670 0x7f52d80ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:55.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.949+0000 7f52ddc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52d80cfca0 0x7f52d81f9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:55.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.949+0000 7f52ddc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52d80d16e0 0x7f52d8158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:55.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.949+0000 7f52ddc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f52d816a070 con 0x7f52d80cfca0 2024-04-25T14:10:55.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.949+0000 7f52ddc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f52d8169ef0 con 0x7f52d806c670 2024-04-25T14:10:55.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.949+0000 7f52ddc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f52d816a1f0 con 0x7f52d80d16e0 2024-04-25T14:10:55.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.949+0000 7f52ce000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52d80cfca0 0x7f52d81f9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:55.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.949+0000 7f52ce000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52d80cfca0 0x7f52d81f9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46300/0 (socket says 172.21.15.86:46300) 2024-04-25T14:10:55.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.949+0000 7f52ce000640 1 -- 172.21.15.86:0/3197911249 learned_addr learned my addr 172.21.15.86:0/3197911249 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:55.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.949+0000 7f52d7400640 1 --2- 172.21.15.86:0/3197911249 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52d80d16e0 0x7f52d8158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:55.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.949+0000 7f52d6a00640 1 --2- 172.21.15.86:0/3197911249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52d806c670 0x7f52d80ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:55.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.949+0000 7f52ce000640 1 -- 172.21.15.86:0/3197911249 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52d80d16e0 msgr2=0x7f52d8158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:55.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.949+0000 7f52ce000640 1 --2- 172.21.15.86:0/3197911249 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52d80d16e0 0x7f52d8158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.949+0000 7f52ce000640 1 -- 172.21.15.86:0/3197911249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52d806c670 msgr2=0x7f52d80ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:55.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.949+0000 7f52ce000640 1 --2- 172.21.15.86:0/3197911249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52d806c670 0x7f52d80ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:55.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.949+0000 7f52ce000640 1 -- 172.21.15.86:0/3197911249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d815b9d0 con 0x7f52d80cfca0 2024-04-25T14:10:55.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.949+0000 7f52ce000640 1 --2- 172.21.15.86:0/3197911249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52d80cfca0 0x7f52d81f9a70 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f52c000e840 tx=0x7f52c000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:55.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.949+0000 7f52cf400640 1 -- 172.21.15.86:0/3197911249 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c0017070 con 0x7f52d80cfca0 2024-04-25T14:10:55.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.949+0000 7f52cf400640 1 -- 172.21.15.86:0/3197911249 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c000ee90 con 0x7f52d80cfca0 2024-04-25T14:10:55.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.949+0000 7f52cf400640 1 -- 172.21.15.86:0/3197911249 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c000cb20 con 0x7f52d80cfca0 2024-04-25T14:10:55.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.949+0000 7f52ddc00640 1 -- 172.21.15.86:0/3197911249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52d815bcb0 con 0x7f52d80cfca0 2024-04-25T14:10:55.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.949+0000 7f52ddc00640 1 -- 172.21.15.86:0/3197911249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52d815c270 con 0x7f52d80cfca0 2024-04-25T14:10:55.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.949+0000 7f52ddc00640 1 -- 172.21.15.86:0/3197911249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52d809f4b0 con 0x7f52d80cfca0 2024-04-25T14:10:55.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.953+0000 7f52cf400640 1 -- 172.21.15.86:0/3197911249 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7f52c001a070 con 0x7f52d80cfca0 2024-04-25T14:10:55.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.953+0000 7f52cf400640 1 --2- 172.21.15.86:0/3197911249 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f52b406f490 0x7f52b40719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:55.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.953+0000 7f52cf400640 1 -- 172.21.15.86:0/3197911249 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 6144+0+0 (secure 0 0 0) 0x7f52c008f280 con 0x7f52d80cfca0 2024-04-25T14:10:55.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.953+0000 7f52d6a00640 1 --2- 172.21.15.86:0/3197911249 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f52b406f490 0x7f52b40719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:55.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.953+0000 7f52d6a00640 1 --2- 172.21.15.86:0/3197911249 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f52b406f490 0x7f52b40719a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f52c80ae000 tx=0x7f52c80a7000 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:55.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:55.957+0000 7f52cf400640 1 -- 172.21.15.86:0/3197911249 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f52c005b930 con 0x7f52d80cfca0 2024-04-25T14:10:56.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.145+0000 7f52ddc00640 1 -- 172.21.15.86:0/3197911249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52d8069d00 con 0x7f52d80cfca0 2024-04-25T14:10:56.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.145+0000 7f52cf400640 1 -- 172.21.15.86:0/3197911249 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5186 (secure 0 0 0) 0x7f52c005fc30 con 0x7f52d80cfca0 2024-04-25T14:10:56.142 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:56.142 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":94,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2258481468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2258481468},{"type":"v1","addr":"172.21.15.155:6834","nonce":2258481468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":8976,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1064843151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1064843151},{"type":"v1","addr":"172.21.15.86:6835","nonce":1064843151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":8985,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1940755625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1940755625},{"type":"v1","addr":"172.21.15.86:6837","nonce":1940755625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:10:45.497688+0000","modified":"2024-04-25T14:10:48.475869+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8991":{"gid":8991,"name":"b","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6835/544985680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":544985680},{"type":"v1","addr":"172.21.15.155:6835","nonce":544985680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8991,"qdb_cluster":[8991]},"id":9}]} 2024-04-25T14:10:56.142 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 94 2024-04-25T14:10:56.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.145+0000 7f52ddc00640 1 -- 172.21.15.86:0/3197911249 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f52b406f490 msgr2=0x7f52b40719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:56.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.145+0000 7f52ddc00640 1 --2- 172.21.15.86:0/3197911249 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f52b406f490 0x7f52b40719a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f52c80ae000 tx=0x7f52c80a7000 comp rx=0 tx=0).stop 2024-04-25T14:10:56.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.145+0000 7f52ddc00640 1 -- 172.21.15.86:0/3197911249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52d80cfca0 msgr2=0x7f52d81f9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:56.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.145+0000 7f52ddc00640 1 --2- 172.21.15.86:0/3197911249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52d80cfca0 0x7f52d81f9a70 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f52c000e840 tx=0x7f52c000ec40 comp rx=0 tx=0).stop 2024-04-25T14:10:56.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.145+0000 7f52ddc00640 1 -- 172.21.15.86:0/3197911249 shutdown_connections 2024-04-25T14:10:56.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.145+0000 7f52ddc00640 1 --2- 172.21.15.86:0/3197911249 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52d80d16e0 0x7f52d8158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.145+0000 7f52ddc00640 1 --2- 172.21.15.86:0/3197911249 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f52b406f490 0x7f52b40719a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.145+0000 7f52ddc00640 1 --2- 172.21.15.86:0/3197911249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52d80cfca0 0x7f52d81f9a70 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.145+0000 7f52ddc00640 1 --2- 172.21.15.86:0/3197911249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52d806c670 0x7f52d80ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.145+0000 7f52ddc00640 1 -- 172.21.15.86:0/3197911249 >> 172.21.15.86:0/3197911249 conn(0x7f52d80b4650 msgr2=0x7f52d809a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:56.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.145+0000 7f52ddc00640 1 -- 172.21.15.86:0/3197911249 shutdown_connections 2024-04-25T14:10:56.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.145+0000 7f52ddc00640 1 -- 172.21.15.86:0/3197911249 wait complete. 2024-04-25T14:10:56.567 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 94, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:10:45.497688+0000', 'modified': '2024-04-25T14:10:48.475869+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8991}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8991': {'gid': 8991, 'name': 'b', 'rank': 0, 'incarnation': 90, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.155:6835/544985680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6833', 'nonce': 544985680}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 544985680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8991, 'qdb_cluster': [8991]} 2024-04-25T14:10:56.567 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T14:10:56.567 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T14:10:56.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.721+0000 7f8c66800640 1 Processor -- start 2024-04-25T14:10:56.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.721+0000 7f8c66800640 1 -- start start 2024-04-25T14:10:56.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c66800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8c600d16e0 0x7f8c600d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c66800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c600730b0 0x7f8c600734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c66800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c600d2030 con 0x7f8c600cfca0 2024-04-25T14:10:56.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c66800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f8c60070470 con 0x7f8c600d16e0 2024-04-25T14:10:56.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c66800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8c60072bd0 con 0x7f8c600730b0 2024-04-25T14:10:56.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c5f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8c600cfca0 0x7f8c6006f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:55010/0 (socket says 172.21.15.86:55010) 2024-04-25T14:10:56.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c5f400640 1 -- 172.21.15.86:0/1946148852 learned_addr learned my addr 172.21.15.86:0/1946148852 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:56.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c5ea00640 1 --2- 172.21.15.86:0/1946148852 >> v2:172.21.15.155:3300/0 conn(0x7f8c600d16e0 0x7f8c600d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c5fe00640 1 --2- 172.21.15.86:0/1946148852 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c600730b0 0x7f8c600734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c5e000640 1 -- 172.21.15.86:0/1946148852 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 390441246 0 0) 0x7f8c500014a0 con 0x7f8c600cfca0 2024-04-25T14:10:56.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c5e000640 1 -- 172.21.15.86:0/1946148852 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c44003530 con 0x7f8c600cfca0 2024-04-25T14:10:56.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c5e000640 1 -- 172.21.15.86:0/1946148852 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 618115029 0 0) 0x7f8c44003530 con 0x7f8c600cfca0 2024-04-25T14:10:56.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c5e000640 1 -- 172.21.15.86:0/1946148852 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c600730b0 msgr2=0x7f8c600734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:56.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c5e000640 1 --2- 172.21.15.86:0/1946148852 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c600730b0 0x7f8c600734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c5e000640 1 -- 172.21.15.86:0/1946148852 >> v2:172.21.15.155:3300/0 conn(0x7f8c600d16e0 msgr2=0x7f8c600d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:56.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c5e000640 1 --2- 172.21.15.86:0/1946148852 >> v2:172.21.15.155:3300/0 conn(0x7f8c600d16e0 0x7f8c600d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c5e000640 1 -- 172.21.15.86:0/1946148852 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c6006fbd0 con 0x7f8c600cfca0 2024-04-25T14:10:56.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c5e000640 1 -- 172.21.15.86:0/1946148852 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f8c50001bd0 con 0x7f8c600cfca0 2024-04-25T14:10:56.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c5e000640 1 -- 172.21.15.86:0/1946148852 >> v1:172.21.15.86:6789/0 conn(0x7f8c600cfca0 legacy=0x7f8c6006f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:56.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c5e000640 1 --2- 172.21.15.86:0/1946148852 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c600730b0 0x7f8c44003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c5e000640 1 --2- 172.21.15.86:0/1946148852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c44003fb0 0x7f8c44004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c5e000640 1 --2- 172.21.15.86:0/1946148852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c44004d50 0x7f8c44005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c5e000640 1 -- 172.21.15.86:0/1946148852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c60070470 con 0x7f8c44003fb0 2024-04-25T14:10:56.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c5e000640 1 -- 172.21.15.86:0/1946148852 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8c60072bd0 con 0x7f8c44004d50 2024-04-25T14:10:56.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c5e000640 1 -- 172.21.15.86:0/1946148852 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8c4400bab0 con 0x7f8c600730b0 2024-04-25T14:10:56.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c5ea00640 1 --2- 172.21.15.86:0/1946148852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c44003fb0 0x7f8c44004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c5f400640 1 --2- 172.21.15.86:0/1946148852 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c600730b0 0x7f8c44003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c5fe00640 1 --2- 172.21.15.86:0/1946148852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c44004d50 0x7f8c44005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c5ea00640 1 -- 172.21.15.86:0/1946148852 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c600730b0 msgr2=0x7f8c44003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:56.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c5ea00640 1 --2- 172.21.15.86:0/1946148852 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c600730b0 0x7f8c44003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c5ea00640 1 -- 172.21.15.86:0/1946148852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c44004d50 msgr2=0x7f8c44005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:56.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c5ea00640 1 --2- 172.21.15.86:0/1946148852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c44004d50 0x7f8c44005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.725+0000 7f8c5ea00640 1 -- 172.21.15.86:0/1946148852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c44003530 con 0x7f8c44003fb0 2024-04-25T14:10:56.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.729+0000 7f8c5ea00640 1 --2- 172.21.15.86:0/1946148852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c44003fb0 0x7f8c44004810 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009ae0 tx=0x7f8c4c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd091cdece7f86d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:56.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.729+0000 7f8c5e000640 1 -- 172.21.15.86:0/1946148852 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c0b2070 con 0x7f8c44003fb0 2024-04-25T14:10:56.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.729+0000 7f8c5e000640 1 -- 172.21.15.86:0/1946148852 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c4c0a56a0 con 0x7f8c44003fb0 2024-04-25T14:10:56.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.729+0000 7f8c5e000640 1 -- 172.21.15.86:0/1946148852 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c0096b0 con 0x7f8c44003fb0 2024-04-25T14:10:56.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.729+0000 7f8c66800640 1 -- 172.21.15.86:0/1946148852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c44003fb0 msgr2=0x7f8c44004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:56.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.729+0000 7f8c66800640 1 --2- 172.21.15.86:0/1946148852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c44003fb0 0x7f8c44004810 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009ae0 tx=0x7f8c4c01cab0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.729+0000 7f8c5fe00640 1 -- 172.21.15.86:0/1946148852 reap_dead start 2024-04-25T14:10:56.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.729+0000 7f8c66800640 1 -- 172.21.15.86:0/1946148852 shutdown_connections 2024-04-25T14:10:56.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.729+0000 7f8c66800640 1 -- 172.21.15.86:0/1946148852 >> 172.21.15.86:0/1946148852 conn(0x7f8c600b4650 msgr2=0x7f8c600b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:56.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.729+0000 7f8c66800640 1 -- 172.21.15.86:0/1946148852 shutdown_connections 2024-04-25T14:10:56.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.729+0000 7f8c66800640 1 -- 172.21.15.86:0/1946148852 wait complete. 2024-04-25T14:10:56.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.733+0000 7f8c66800640 1 Processor -- start 2024-04-25T14:10:56.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.733+0000 7f8c66800640 1 -- start start 2024-04-25T14:10:56.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.733+0000 7f8c66800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c44003fb0 0x7f8c600ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.733+0000 7f8c66800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c6009a930 0x7f8c6015a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.733+0000 7f8c66800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c6015a6e0 0x7f8c60151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.733+0000 7f8c66800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c600734c0 con 0x7f8c6015a6e0 2024-04-25T14:10:56.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.733+0000 7f8c66800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8c60070190 con 0x7f8c44003fb0 2024-04-25T14:10:56.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.733+0000 7f8c66800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8c60073640 con 0x7f8c6009a930 2024-04-25T14:10:56.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.733+0000 7f8c5fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c6015a6e0 0x7f8c60151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.733+0000 7f8c5fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c6015a6e0 0x7f8c60151f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46320/0 (socket says 172.21.15.86:46320) 2024-04-25T14:10:56.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.733+0000 7f8c5fe00640 1 -- 172.21.15.86:0/2177899014 learned_addr learned my addr 172.21.15.86:0/2177899014 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:56.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.733+0000 7f8c5f400640 1 --2- 172.21.15.86:0/2177899014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c44003fb0 0x7f8c600ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.733+0000 7f8c5ea00640 1 --2- 172.21.15.86:0/2177899014 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c6009a930 0x7f8c6015a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.733+0000 7f8c5fe00640 1 -- 172.21.15.86:0/2177899014 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c6009a930 msgr2=0x7f8c6015a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:56.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.733+0000 7f8c5fe00640 1 --2- 172.21.15.86:0/2177899014 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c6009a930 0x7f8c6015a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.733+0000 7f8c5fe00640 1 -- 172.21.15.86:0/2177899014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c44003fb0 msgr2=0x7f8c600ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:56.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.733+0000 7f8c5fe00640 1 --2- 172.21.15.86:0/2177899014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c44003fb0 0x7f8c600ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.733+0000 7f8c5fe00640 1 -- 172.21.15.86:0/2177899014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c60154a40 con 0x7f8c6015a6e0 2024-04-25T14:10:56.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.733+0000 7f8c5fe00640 1 --2- 172.21.15.86:0/2177899014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c6015a6e0 0x7f8c60151f30 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f8c600b4220 tx=0x7f8c58011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:56.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.733+0000 7f8c57400640 1 -- 172.21.15.86:0/2177899014 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c58004030 con 0x7f8c6015a6e0 2024-04-25T14:10:56.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.733+0000 7f8c57400640 1 -- 172.21.15.86:0/2177899014 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c5801b070 con 0x7f8c6015a6e0 2024-04-25T14:10:56.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.733+0000 7f8c66800640 1 -- 172.21.15.86:0/2177899014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c600c9e80 con 0x7f8c6015a6e0 2024-04-25T14:10:56.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.733+0000 7f8c57400640 1 -- 172.21.15.86:0/2177899014 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c58004430 con 0x7f8c6015a6e0 2024-04-25T14:10:56.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.737+0000 7f8c66800640 1 -- 172.21.15.86:0/2177899014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c600ca440 con 0x7f8c6015a6e0 2024-04-25T14:10:56.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.737+0000 7f8c57400640 1 -- 172.21.15.86:0/2177899014 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7f8c58012040 con 0x7f8c6015a6e0 2024-04-25T14:10:56.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.737+0000 7f8c57400640 1 --2- 172.21.15.86:0/2177899014 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f8c4006f490 0x7f8c400719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.737+0000 7f8c57400640 1 -- 172.21.15.86:0/2177899014 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 6144+0+0 (secure 0 0 0) 0x7f8c5808f2a0 con 0x7f8c6015a6e0 2024-04-25T14:10:56.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.737+0000 7f8c5f400640 1 --2- 172.21.15.86:0/2177899014 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f8c4006f490 0x7f8c400719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.737+0000 7f8c6dea6000 1 Processor -- start 2024-04-25T14:10:56.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.737+0000 7f8c6dea6000 1 -- start start 2024-04-25T14:10:56.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c5f400640 1 --2- 172.21.15.86:0/2177899014 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f8c4006f490 0x7f8c400719a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8c50001680 tx=0x7f8c50009040 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:56.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c6dea6000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a13c8fc20 0x555a13dedee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c6dea6000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a13c7bea0 0x555a13dee420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c6dea6000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555a13874280 0x555a13dee960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c6dea6000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555a13c5d160 con 0x555a13c7bea0 2024-04-25T14:10:56.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c6dea6000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555a13bf2e20 con 0x555a13c8fc20 2024-04-25T14:10:56.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c6dea6000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x555a13baba20 con 0x555a13874280 2024-04-25T14:10:56.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c5ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a13c7bea0 0x555a13dee420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c5ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a13c7bea0 0x555a13dee420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46328/0 (socket says 172.21.15.86:46328) 2024-04-25T14:10:56.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c5ea00640 1 -- 172.21.15.86:0/2810453710 learned_addr learned my addr 172.21.15.86:0/2810453710 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:56.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c5fe00640 1 --2- 172.21.15.86:0/2810453710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555a13874280 0x555a13dee960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c5f400640 1 --2- 172.21.15.86:0/2810453710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a13c8fc20 0x555a13dedee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c5ea00640 1 -- 172.21.15.86:0/2810453710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555a13874280 msgr2=0x555a13dee960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:56.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c5ea00640 1 --2- 172.21.15.86:0/2810453710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555a13874280 0x555a13dee960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c5ea00640 1 -- 172.21.15.86:0/2810453710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a13c8fc20 msgr2=0x555a13dedee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:56.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c5ea00640 1 --2- 172.21.15.86:0/2810453710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a13c8fc20 0x555a13dedee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c5ea00640 1 -- 172.21.15.86:0/2810453710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a13bb0100 con 0x555a13c7bea0 2024-04-25T14:10:56.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c5f400640 1 --2- 172.21.15.86:0/2810453710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a13c8fc20 0x555a13dedee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:56.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c5ea00640 1 --2- 172.21.15.86:0/2810453710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a13c7bea0 0x555a13dee420 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c0b0040 tx=0x7f8c4c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f7cbebb0fd7f17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:56.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c4be00640 1 -- 172.21.15.86:0/2810453710 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c0b2040 con 0x555a13c7bea0 2024-04-25T14:10:56.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c4be00640 1 -- 172.21.15.86:0/2810453710 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c4c004220 con 0x555a13c7bea0 2024-04-25T14:10:56.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c4be00640 1 -- 172.21.15.86:0/2810453710 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c0a56a0 con 0x555a13c7bea0 2024-04-25T14:10:56.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c6dea6000 1 -- 172.21.15.86:0/2810453710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a13c7bea0 msgr2=0x555a13dee420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:56.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c6dea6000 1 --2- 172.21.15.86:0/2810453710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a13c7bea0 0x555a13dee420 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c0b0040 tx=0x7f8c4c004740 comp rx=0 tx=0).stop 2024-04-25T14:10:56.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c6dea6000 1 -- 172.21.15.86:0/2810453710 shutdown_connections 2024-04-25T14:10:56.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c6dea6000 1 --2- 172.21.15.86:0/2810453710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555a13874280 0x555a13dee960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c6dea6000 1 --2- 172.21.15.86:0/2810453710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a13c7bea0 0x555a13dee420 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c6dea6000 1 --2- 172.21.15.86:0/2810453710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a13c8fc20 0x555a13dedee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c6dea6000 1 -- 172.21.15.86:0/2810453710 >> 172.21.15.86:0/2810453710 conn(0x555a138b3ff0 msgr2=0x555a13da34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:56.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c6dea6000 1 -- 172.21.15.86:0/2810453710 shutdown_connections 2024-04-25T14:10:56.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.741+0000 7f8c6dea6000 1 -- 172.21.15.86:0/2810453710 wait complete. 2024-04-25T14:10:56.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c6dea6000 1 -- start start 2024-04-25T14:10:56.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c6dea6000 1 Processor -- start 2024-04-25T14:10:56.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c6dea6000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a13874280 0x555a13df5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c6dea6000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555a13c7bea0 0x555a13df55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c6dea6000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a13c8fc20 0x555a13db00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c6dea6000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555a13c00220 con 0x555a13c8fc20 2024-04-25T14:10:56.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c6dea6000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555a13b96ea0 con 0x555a13874280 2024-04-25T14:10:56.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c6dea6000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x555a13bedb00 con 0x555a13c7bea0 2024-04-25T14:10:56.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c6dea6000 10 client.0 _ll_register_callbacks cb 0x555a13bffdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:56.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c5fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a13c8fc20 0x555a13db00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c5fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a13c8fc20 0x555a13db00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46334/0 (socket says 172.21.15.86:46334) 2024-04-25T14:10:56.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c5fe00640 1 -- 172.21.15.86:0/3346369959 learned_addr learned my addr 172.21.15.86:0/3346369959 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:56.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c5f400640 1 --2- 172.21.15.86:0/3346369959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a13874280 0x555a13df5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c5ea00640 1 --2- 172.21.15.86:0/3346369959 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555a13c7bea0 0x555a13df55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c5fe00640 1 -- 172.21.15.86:0/3346369959 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555a13c7bea0 msgr2=0x555a13df55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:56.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c5fe00640 1 --2- 172.21.15.86:0/3346369959 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555a13c7bea0 0x555a13df55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c5fe00640 1 -- 172.21.15.86:0/3346369959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a13874280 msgr2=0x555a13df5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:56.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c5fe00640 1 --2- 172.21.15.86:0/3346369959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a13874280 0x555a13df5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c5fe00640 1 -- 172.21.15.86:0/3346369959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a13b92500 con 0x555a13c8fc20 2024-04-25T14:10:56.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c5fe00640 1 --2- 172.21.15.86:0/3346369959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a13c8fc20 0x555a13db00c0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f8c58058b10 tx=0x7f8c5805de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69138c01e04d329b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:56.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c48c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:10:56.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c48c00640 1 -- 172.21.15.86:0/3346369959 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c58057760 con 0x555a13c8fc20 2024-04-25T14:10:56.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c48c00640 1 -- 172.21.15.86:0/3346369959 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 6144+0+0 (secure 0 0 0) 0x7f8c580643e0 con 0x555a13c8fc20 2024-04-25T14:10:56.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c6dea6000 1 -- 172.21.15.86:0/3346369959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c4c004220 con 0x555a13c8fc20 2024-04-25T14:10:56.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c48c00640 1 -- 172.21.15.86:0/3346369959 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f8c5805b3b0 con 0x555a13c8fc20 2024-04-25T14:10:56.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c6dea6000 10 client.9153 fetch_fsmap learned FSMap version 94 2024-04-25T14:10:56.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c6dea6000 1 -- 172.21.15.86:0/3346369959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x555a13b78ea0 con 0x555a13c8fc20 2024-04-25T14:10:56.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c48c00640 1 -- 172.21.15.86:0/3346369959 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c4c004220 con 0x555a13c8fc20 2024-04-25T14:10:56.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c48c00640 1 -- 172.21.15.86:0/3346369959 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 94) ==== 3494+0+0 (secure 0 0 0) 0x7f8c5800fb90 con 0x555a13c8fc20 2024-04-25T14:10:56.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c6dea6000 10 client.9153 fetch_fsmap finished waiting for FSMap version 94 2024-04-25T14:10:56.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c6dea6000 10 client.9153 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T14:10:56.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c6dea6000 20 client.9153 populate_metadata read hostname 'smithi086' 2024-04-25T14:10:56.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c6dea6000 1 --2- 172.21.15.86:0/3346369959 >> [v2:172.21.15.155:6833/544985680,v1:172.21.15.155:6835/544985680] conn(0x555a138987d0 0x555a13db0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c6dea6000 4 client.9153 mds_command: new command op to 8991 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:10:56.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.745+0000 7f8c6dea6000 1 -- 172.21.15.86:0/3346369959 --> [v2:172.21.15.155:6833/544985680,v1:172.21.15.155:6835/544985680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a13b8bbe0 con 0x555a138987d0 2024-04-25T14:10:56.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.749+0000 7f8c5f400640 1 --2- 172.21.15.86:0/3346369959 >> [v2:172.21.15.155:6833/544985680,v1:172.21.15.155:6835/544985680] conn(0x555a138987d0 0x555a13db0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.749+0000 7f8c5f400640 1 --2- 172.21.15.86:0/3346369959 >> [v2:172.21.15.155:6833/544985680,v1:172.21.15.155:6835/544985680] conn(0x555a138987d0 0x555a13db0e00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40b3dcb1bdede8c6 server_cookie=aad812bbf40e5ac7 in_seq=0 out_seq=0 2024-04-25T14:10:56.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.749+0000 7f8c48c00640 10 client.9153 ms_handle_connect on v2:172.21.15.155:6833/544985680 2024-04-25T14:10:56.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.749+0000 7f8c48c00640 1 -- 172.21.15.86:0/3346369959 <== mds.0 v2:172.21.15.155:6833/544985680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555a13b8bbe0 con 0x555a138987d0 2024-04-25T14:10:56.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.749+0000 7f8c48c00640 10 client.9153 handle_command_reply: tid=0 2024-04-25T14:10:56.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.749+0000 7f8c6dea6000 1 client.9153 shutdown 2024-04-25T14:10:56.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.749+0000 7f8c6dea6000 1 -- 172.21.15.86:0/3346369959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a13c8fc20 msgr2=0x555a13db00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:56.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.749+0000 7f8c6dea6000 1 --2- 172.21.15.86:0/3346369959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a13c8fc20 0x555a13db00c0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f8c58058b10 tx=0x7f8c5805de60 comp rx=0 tx=0).stop 2024-04-25T14:10:56.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.749+0000 7f8c6dea6000 1 -- 172.21.15.86:0/3346369959 shutdown_connections 2024-04-25T14:10:56.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.749+0000 7f8c6dea6000 1 --2- 172.21.15.86:0/3346369959 >> [v2:172.21.15.155:6833/544985680,v1:172.21.15.155:6835/544985680] conn(0x555a138987d0 0x555a13db0e00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.749+0000 7f8c6dea6000 1 --2- 172.21.15.86:0/3346369959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a13c8fc20 0x555a13db00c0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.749+0000 7f8c6dea6000 1 --2- 172.21.15.86:0/3346369959 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555a13c7bea0 0x555a13df55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.749+0000 7f8c6dea6000 1 --2- 172.21.15.86:0/3346369959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a13874280 0x555a13df5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.749+0000 7f8c48c00640 0 client.9153 ms_handle_reset on v2:172.21.15.155:6833/544985680 2024-04-25T14:10:56.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.749+0000 7f8c6dea6000 1 -- 172.21.15.86:0/3346369959 >> 172.21.15.86:0/3346369959 conn(0x555a138b3ff0 msgr2=0x555a13da3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:56.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.749+0000 7f8c6dea6000 1 -- 172.21.15.86:0/3346369959 shutdown_connections 2024-04-25T14:10:56.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.749+0000 7f8c6dea6000 1 -- 172.21.15.86:0/3346369959 wait complete. 2024-04-25T14:10:56.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.749+0000 7f8c6dea6000 20 client.9153 trim_cache size 0 max 16384 2024-04-25T14:10:56.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.769+0000 7f8c6dea6000 1 Processor -- start 2024-04-25T14:10:56.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.769+0000 7f8c6dea6000 1 -- start start 2024-04-25T14:10:56.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.769+0000 7f8c6dea6000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a13874280 0x555a13df4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.769+0000 7f8c6dea6000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555a13c7bea0 0x555a13e05210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.769+0000 7f8c6dea6000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a13c8fc20 0x555a13e05750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.769+0000 7f8c6dea6000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c60202d40 con 0x555a13874280 2024-04-25T14:10:56.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.769+0000 7f8c6dea6000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8c60202220 con 0x555a13c8fc20 2024-04-25T14:10:56.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.769+0000 7f8c6dea6000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8c602026f0 con 0x555a13c7bea0 2024-04-25T14:10:56.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.769+0000 7f8c5f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a13874280 0x555a13df4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.769+0000 7f8c5f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a13874280 0x555a13df4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46346/0 (socket says 172.21.15.86:46346) 2024-04-25T14:10:56.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.769+0000 7f8c5f400640 1 -- 172.21.15.86:0/1460561215 learned_addr learned my addr 172.21.15.86:0/1460561215 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:56.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.769+0000 7f8c5ea00640 1 --2- 172.21.15.86:0/1460561215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555a13c7bea0 0x555a13e05210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.769+0000 7f8c5fe00640 1 --2- 172.21.15.86:0/1460561215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a13c8fc20 0x555a13e05750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.769+0000 7f8c5f400640 1 -- 172.21.15.86:0/1460561215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555a13c7bea0 msgr2=0x555a13e05210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:56.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.769+0000 7f8c5f400640 1 --2- 172.21.15.86:0/1460561215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555a13c7bea0 0x555a13e05210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.769+0000 7f8c5f400640 1 -- 172.21.15.86:0/1460561215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a13c8fc20 msgr2=0x555a13e05750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:56.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.769+0000 7f8c5f400640 1 --2- 172.21.15.86:0/1460561215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a13c8fc20 0x555a13e05750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.769+0000 7f8c5f400640 1 -- 172.21.15.86:0/1460561215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a13da7840 con 0x555a13874280 2024-04-25T14:10:56.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.769+0000 7f8c5ea00640 1 --2- 172.21.15.86:0/1460561215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555a13c7bea0 0x555a13e05210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:56.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.769+0000 7f8c5fe00640 1 --2- 172.21.15.86:0/1460561215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a13c8fc20 0x555a13e05750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:56.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.769+0000 7f8c5f400640 1 --2- 172.21.15.86:0/1460561215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a13874280 0x555a13df4440 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f8c50002240 tx=0x7f8c5000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab568b2ad065764 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:56.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.769+0000 7f8c4b400640 1 -- 172.21.15.86:0/1460561215 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5002d610 con 0x555a13874280 2024-04-25T14:10:56.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.769+0000 7f8c4b400640 1 -- 172.21.15.86:0/1460561215 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c5001b040 con 0x555a13874280 2024-04-25T14:10:56.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.769+0000 7f8c4b400640 1 -- 172.21.15.86:0/1460561215 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5002d9d0 con 0x555a13874280 2024-04-25T14:10:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c6dea6000 1 -- 172.21.15.86:0/1460561215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a13874280 msgr2=0x555a13df4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c6dea6000 1 --2- 172.21.15.86:0/1460561215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a13874280 0x555a13df4440 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f8c50002240 tx=0x7f8c5000f000 comp rx=0 tx=0).stop 2024-04-25T14:10:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c6dea6000 1 -- 172.21.15.86:0/1460561215 shutdown_connections 2024-04-25T14:10:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c6dea6000 1 --2- 172.21.15.86:0/1460561215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a13c8fc20 0x555a13e05750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c6dea6000 1 --2- 172.21.15.86:0/1460561215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555a13c7bea0 0x555a13e05210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c6dea6000 1 --2- 172.21.15.86:0/1460561215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a13874280 0x555a13df4440 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c6dea6000 1 -- 172.21.15.86:0/1460561215 >> 172.21.15.86:0/1460561215 conn(0x555a138b3ff0 msgr2=0x555a13dab040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c6dea6000 1 -- 172.21.15.86:0/1460561215 shutdown_connections 2024-04-25T14:10:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c6dea6000 1 -- 172.21.15.86:0/1460561215 wait complete. 2024-04-25T14:10:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c6dea6000 1 -- start start 2024-04-25T14:10:56.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c6dea6000 1 Processor -- start 2024-04-25T14:10:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c6dea6000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a13874280 0x555a13e607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c6dea6000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a13c7bea0 0x555a13e60ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c6dea6000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555a13c8fc20 0x555a13df8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c6dea6000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555a13bf2130 con 0x555a13c7bea0 2024-04-25T14:10:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c6dea6000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555a13b887d0 con 0x555a13874280 2024-04-25T14:10:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c6dea6000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x555a13bf7450 con 0x555a13c8fc20 2024-04-25T14:10:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c6dea6000 10 client.0 _ll_register_callbacks cb 0x7f8c5800f970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c5ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a13c7bea0 0x555a13e60ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c5ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a13c7bea0 0x555a13e60ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46350/0 (socket says 172.21.15.86:46350) 2024-04-25T14:10:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c5ea00640 1 -- 172.21.15.86:0/4106015082 learned_addr learned my addr 172.21.15.86:0/4106015082 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c5f400640 1 --2- 172.21.15.86:0/4106015082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a13874280 0x555a13e607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c5fe00640 1 --2- 172.21.15.86:0/4106015082 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555a13c8fc20 0x555a13df8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c5ea00640 1 -- 172.21.15.86:0/4106015082 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555a13c8fc20 msgr2=0x555a13df8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c5ea00640 1 --2- 172.21.15.86:0/4106015082 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555a13c8fc20 0x555a13df8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c5ea00640 1 -- 172.21.15.86:0/4106015082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a13874280 msgr2=0x555a13e607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c5ea00640 1 --2- 172.21.15.86:0/4106015082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a13874280 0x555a13e607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c5ea00640 1 -- 172.21.15.86:0/4106015082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a13db0430 con 0x555a13c7bea0 2024-04-25T14:10:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c5ea00640 1 --2- 172.21.15.86:0/4106015082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a13c7bea0 0x555a13e60ce0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c004780 tx=0x7f8c4c019850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a14a6ac7b44cc8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.773+0000 7f8c48c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:10:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c48c00640 1 -- 172.21.15.86:0/4106015082 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c0c6060 con 0x555a13c7bea0 2024-04-25T14:10:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c48c00640 1 -- 172.21.15.86:0/4106015082 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 6144+0+0 (secure 0 0 0) 0x7f8c4c0a58a0 con 0x555a13c7bea0 2024-04-25T14:10:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c6dea6000 1 -- 172.21.15.86:0/4106015082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c5001b040 con 0x555a13c7bea0 2024-04-25T14:10:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c48c00640 1 -- 172.21.15.86:0/4106015082 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f8c4c0cc050 con 0x555a13c7bea0 2024-04-25T14:10:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c6dea6000 10 client.9159 fetch_fsmap learned FSMap version 94 2024-04-25T14:10:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c6dea6000 1 -- 172.21.15.86:0/4106015082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x555a13babc70 con 0x555a13c7bea0 2024-04-25T14:10:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c48c00640 1 -- 172.21.15.86:0/4106015082 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c5001b040 con 0x555a13c7bea0 2024-04-25T14:10:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c48c00640 1 -- 172.21.15.86:0/4106015082 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 94) ==== 3494+0+0 (secure 0 0 0) 0x7f8c4c0cf020 con 0x555a13c7bea0 2024-04-25T14:10:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c6dea6000 10 client.9159 fetch_fsmap finished waiting for FSMap version 94 2024-04-25T14:10:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c6dea6000 10 client.9159 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T14:10:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c6dea6000 20 client.9159 populate_metadata read hostname 'smithi086' 2024-04-25T14:10:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c6dea6000 1 --2- 172.21.15.86:0/4106015082 >> [v2:172.21.15.155:6833/544985680,v1:172.21.15.155:6835/544985680] conn(0x555a13886830 0x555a13df94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c6dea6000 4 client.9159 mds_command: new command op to 8991 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:10:56.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c6dea6000 1 -- 172.21.15.86:0/4106015082 --> [v2:172.21.15.155:6833/544985680,v1:172.21.15.155:6835/544985680] -- command(tid 0: {"prefix": "status"}) -- 0x555a13c00a30 con 0x555a13886830 2024-04-25T14:10:56.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c5f400640 1 --2- 172.21.15.86:0/4106015082 >> [v2:172.21.15.155:6833/544985680,v1:172.21.15.155:6835/544985680] conn(0x555a13886830 0x555a13df94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c5f400640 1 --2- 172.21.15.86:0/4106015082 >> [v2:172.21.15.155:6833/544985680,v1:172.21.15.155:6835/544985680] conn(0x555a13886830 0x555a13df94c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ed027ce0c8a9edd server_cookie=26181a6ed87ed9d0 in_seq=0 out_seq=0 2024-04-25T14:10:56.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c48c00640 10 client.9159 ms_handle_connect on v2:172.21.15.155:6833/544985680 2024-04-25T14:10:56.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c48c00640 1 -- 172.21.15.86:0/4106015082 <== mds.0 v2:172.21.15.155:6833/544985680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x555a13c00a30 con 0x555a13886830 2024-04-25T14:10:56.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c48c00640 10 client.9159 handle_command_reply: tid=0 2024-04-25T14:10:56.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c6dea6000 1 client.9159 shutdown 2024-04-25T14:10:56.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c6dea6000 1 -- 172.21.15.86:0/4106015082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a13c7bea0 msgr2=0x555a13e60ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:56.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c6dea6000 1 --2- 172.21.15.86:0/4106015082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a13c7bea0 0x555a13e60ce0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c004780 tx=0x7f8c4c019850 comp rx=0 tx=0).stop 2024-04-25T14:10:56.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c6dea6000 1 -- 172.21.15.86:0/4106015082 shutdown_connections 2024-04-25T14:10:56.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c6dea6000 1 --2- 172.21.15.86:0/4106015082 >> [v2:172.21.15.155:6833/544985680,v1:172.21.15.155:6835/544985680] conn(0x555a13886830 0x555a13df94c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c6dea6000 1 --2- 172.21.15.86:0/4106015082 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555a13c8fc20 0x555a13df8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c6dea6000 1 --2- 172.21.15.86:0/4106015082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a13c7bea0 0x555a13e60ce0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c6dea6000 1 --2- 172.21.15.86:0/4106015082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a13874280 0x555a13e607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c48c00640 0 client.9159 ms_handle_reset on v2:172.21.15.155:6833/544985680 2024-04-25T14:10:56.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c6dea6000 1 -- 172.21.15.86:0/4106015082 >> 172.21.15.86:0/4106015082 conn(0x555a138b3ff0 msgr2=0x555a13e04200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:56.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c6dea6000 1 -- 172.21.15.86:0/4106015082 shutdown_connections 2024-04-25T14:10:56.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c6dea6000 1 -- 172.21.15.86:0/4106015082 wait complete. 2024-04-25T14:10:56.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c6dea6000 20 client.9159 trim_cache size 0 max 16384 2024-04-25T14:10:56.778 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:10:56.778 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb", 2024-04-25T14:10:56.778 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-25T14:10:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 8991, 2024-04-25T14:10:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T14:10:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T14:10:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T14:10:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 10.720540218, 2024-04-25T14:10:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 94, 2024-04-25T14:10:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 127, 2024-04-25T14:10:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 126, 2024-04-25T14:10:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 15.564892873 2024-04-25T14:10:56.779 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:10:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c66800640 1 -- 172.21.15.86:0/2177899014 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f8c4006f490 msgr2=0x7f8c400719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c66800640 1 --2- 172.21.15.86:0/2177899014 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f8c4006f490 0x7f8c400719a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8c50001680 tx=0x7f8c50009040 comp rx=0 tx=0).stop 2024-04-25T14:10:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c66800640 1 -- 172.21.15.86:0/2177899014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c6015a6e0 msgr2=0x7f8c60151f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c66800640 1 --2- 172.21.15.86:0/2177899014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c6015a6e0 0x7f8c60151f30 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f8c600b4220 tx=0x7f8c58011cf0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c66800640 1 -- 172.21.15.86:0/2177899014 shutdown_connections 2024-04-25T14:10:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c66800640 1 --2- 172.21.15.86:0/2177899014 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f8c4006f490 0x7f8c400719a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c66800640 1 --2- 172.21.15.86:0/2177899014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c6015a6e0 0x7f8c60151f30 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c66800640 1 --2- 172.21.15.86:0/2177899014 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c6009a930 0x7f8c6015a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c66800640 1 --2- 172.21.15.86:0/2177899014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c44003fb0 0x7f8c600ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c66800640 1 -- 172.21.15.86:0/2177899014 >> 172.21.15.86:0/2177899014 conn(0x7f8c600b4650 msgr2=0x7f8c600aebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c66800640 1 -- 172.21.15.86:0/2177899014 shutdown_connections 2024-04-25T14:10:56.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.777+0000 7f8c66800640 1 -- 172.21.15.86:0/2177899014 wait complete. 2024-04-25T14:10:56.795 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T14:10:56.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.945+0000 7fc655800640 1 Processor -- start 2024-04-25T14:10:56.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.945+0000 7fc655800640 1 -- start start 2024-04-25T14:10:56.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.945+0000 7fc655800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc65006c670 0x7fc65015de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.945+0000 7fc655800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6500cef20 0x7fc6500cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.945+0000 7fc655800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc650161b40 con 0x7fc6500cdf50 2024-04-25T14:10:56.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.945+0000 7fc655800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc65015e380 con 0x7fc65006c670 2024-04-25T14:10:56.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.945+0000 7fc655800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc650162750 con 0x7fc6500cef20 2024-04-25T14:10:56.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.945+0000 7fc64ea00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc65006c670 0x7fc65015de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.945+0000 7fc64f400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6500cef20 0x7fc6500cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.945+0000 7fc64f400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6500cef20 0x7fc6500cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:48126/0 (socket says 172.21.15.86:48126) 2024-04-25T14:10:56.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.945+0000 7fc64f400640 1 -- 172.21.15.86:0/1955299991 learned_addr learned my addr 172.21.15.86:0/1955299991 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:56.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.945+0000 7fc64ea00640 1 -- 172.21.15.86:0/1955299991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6500cef20 msgr2=0x7fc6500cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:56.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.945+0000 7fc64ea00640 1 --2- 172.21.15.86:0/1955299991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6500cef20 0x7fc6500cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.945+0000 7fc64ea00640 1 -- 172.21.15.86:0/1955299991 >> v1:172.21.15.155:6789/0 conn(0x7fc6500cdf50 legacy=0x7fc6500ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:56.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.945+0000 7fc64ea00640 1 -- 172.21.15.86:0/1955299991 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc650161e20 con 0x7fc65006c670 2024-04-25T14:10:56.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.945+0000 7fc64f400640 1 --2- 172.21.15.86:0/1955299991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6500cef20 0x7fc6500cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:10:56.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.949+0000 7fc64ea00640 1 --2- 172.21.15.86:0/1955299991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc65006c670 0x7fc65015de40 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc6400097c0 tx=0x7fc64001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=658c168d0a0a5c2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:56.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.949+0000 7fc64e000640 1 -- 172.21.15.86:0/1955299991 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6400b0070 con 0x7fc65006c670 2024-04-25T14:10:56.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.949+0000 7fc64e000640 1 -- 172.21.15.86:0/1955299991 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6400ab600 con 0x7fc65006c670 2024-04-25T14:10:56.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.949+0000 7fc64e000640 1 -- 172.21.15.86:0/1955299991 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6400ab9d0 con 0x7fc65006c670 2024-04-25T14:10:56.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.949+0000 7fc655800640 1 -- 172.21.15.86:0/1955299991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc65006c670 msgr2=0x7fc65015de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:56.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.949+0000 7fc655800640 1 --2- 172.21.15.86:0/1955299991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc65006c670 0x7fc65015de40 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc6400097c0 tx=0x7fc64001c410 comp rx=0 tx=0).stop 2024-04-25T14:10:56.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.949+0000 7fc655800640 1 -- 172.21.15.86:0/1955299991 shutdown_connections 2024-04-25T14:10:56.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.949+0000 7fc655800640 1 --2- 172.21.15.86:0/1955299991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6500cef20 0x7fc6500cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.949+0000 7fc655800640 1 --2- 172.21.15.86:0/1955299991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc65006c670 0x7fc65015de40 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.949+0000 7fc655800640 1 -- 172.21.15.86:0/1955299991 >> 172.21.15.86:0/1955299991 conn(0x7fc65009ba30 msgr2=0x7fc6500b73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:56.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.949+0000 7fc655800640 1 -- 172.21.15.86:0/1955299991 shutdown_connections 2024-04-25T14:10:56.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.949+0000 7fc655800640 1 -- 172.21.15.86:0/1955299991 wait complete. 2024-04-25T14:10:56.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.953+0000 7fc655800640 1 Processor -- start 2024-04-25T14:10:56.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.953+0000 7fc655800640 1 -- start start 2024-04-25T14:10:56.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.953+0000 7fc655800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc65006c670 0x7fc650156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.953+0000 7fc655800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6500cdf50 0x7fc650150c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.953+0000 7fc655800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6500cef20 0x7fc6501fadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.953+0000 7fc655800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc6500b8060 con 0x7fc6500cef20 2024-04-25T14:10:56.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.953+0000 7fc655800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc6500b7ee0 con 0x7fc6500cdf50 2024-04-25T14:10:56.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.953+0000 7fc655800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc65006ee70 con 0x7fc65006c670 2024-04-25T14:10:56.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.953+0000 7fc64f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6500cef20 0x7fc6501fadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.953+0000 7fc64ea00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc65006c670 0x7fc650156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.953+0000 7fc64f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6500cef20 0x7fc6501fadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46364/0 (socket says 172.21.15.86:46364) 2024-04-25T14:10:56.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.953+0000 7fc64f400640 1 -- 172.21.15.86:0/3002747608 learned_addr learned my addr 172.21.15.86:0/3002747608 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:56.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.957+0000 7fc646000640 1 --2- 172.21.15.86:0/3002747608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6500cdf50 0x7fc650150c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.957+0000 7fc64ea00640 1 -- 172.21.15.86:0/3002747608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6500cdf50 msgr2=0x7fc650150c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:56.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.957+0000 7fc64ea00640 1 --2- 172.21.15.86:0/3002747608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6500cdf50 0x7fc650150c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.957+0000 7fc64ea00640 1 -- 172.21.15.86:0/3002747608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6500cef20 msgr2=0x7fc6501fadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:56.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.957+0000 7fc64ea00640 1 --2- 172.21.15.86:0/3002747608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6500cef20 0x7fc6501fadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:56.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.957+0000 7fc64ea00640 1 -- 172.21.15.86:0/3002747608 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65015a2d0 con 0x7fc65006c670 2024-04-25T14:10:56.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.957+0000 7fc64f400640 1 --2- 172.21.15.86:0/3002747608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6500cef20 0x7fc6501fadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:10:56.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.957+0000 7fc64ea00640 1 --2- 172.21.15.86:0/3002747608 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc65006c670 0x7fc650156710 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc6400ab000 tx=0x7fc64001ac50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:56.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.957+0000 7fc647400640 1 -- 172.21.15.86:0/3002747608 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6400b0050 con 0x7fc65006c670 2024-04-25T14:10:56.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.957+0000 7fc647400640 1 -- 172.21.15.86:0/3002747608 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6400bc070 con 0x7fc65006c670 2024-04-25T14:10:56.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.957+0000 7fc655800640 1 -- 172.21.15.86:0/3002747608 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc65015a5b0 con 0x7fc65006c670 2024-04-25T14:10:56.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.957+0000 7fc647400640 1 -- 172.21.15.86:0/3002747608 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc640006e20 con 0x7fc65006c670 2024-04-25T14:10:56.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.957+0000 7fc655800640 1 -- 172.21.15.86:0/3002747608 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc65015ab70 con 0x7fc65006c670 2024-04-25T14:10:56.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.957+0000 7fc655800640 1 -- 172.21.15.86:0/3002747608 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6500a0630 con 0x7fc65006c670 2024-04-25T14:10:56.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.957+0000 7fc647400640 1 -- 172.21.15.86:0/3002747608 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7fc6400b5830 con 0x7fc65006c670 2024-04-25T14:10:56.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.961+0000 7fc647400640 1 --2- 172.21.15.86:0/3002747608 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fc62c06f4c0 0x7fc62c0719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:56.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.961+0000 7fc647400640 1 -- 172.21.15.86:0/3002747608 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(127..127 src has 1..127) ==== 6144+0+0 (secure 0 0 0) 0x7fc64012a6e0 con 0x7fc65006c670 2024-04-25T14:10:56.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.961+0000 7fc646000640 1 --2- 172.21.15.86:0/3002747608 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fc62c06f4c0 0x7fc62c0719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:56.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.961+0000 7fc646000640 1 --2- 172.21.15.86:0/3002747608 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fc62c06f4c0 0x7fc62c0719d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc65015d230 tx=0x7fc638009040 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:56.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:56.965+0000 7fc647400640 1 -- 172.21.15.86:0/3002747608 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc640016090 con 0x7fc65006c670 2024-04-25T14:10:57.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:57.157+0000 7fc655800640 1 -- 172.21.15.86:0/3002747608 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc650076bf0 con 0x7fc65006c670 2024-04-25T14:10:57.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:57.157+0000 7fc647400640 1 -- 172.21.15.86:0/3002747608 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v94) ==== 53+0+73 (secure 0 0 0) 0x7fc6400f72c0 con 0x7fc65006c670 2024-04-25T14:10:57.154 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T14:10:57.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:57.157+0000 7fc655800640 1 -- 172.21.15.86:0/3002747608 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fc62c06f4c0 msgr2=0x7fc62c0719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:57.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:57.157+0000 7fc655800640 1 --2- 172.21.15.86:0/3002747608 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fc62c06f4c0 0x7fc62c0719d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc65015d230 tx=0x7fc638009040 comp rx=0 tx=0).stop 2024-04-25T14:10:57.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:57.157+0000 7fc655800640 1 -- 172.21.15.86:0/3002747608 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc65006c670 msgr2=0x7fc650156710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:57.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:57.157+0000 7fc655800640 1 --2- 172.21.15.86:0/3002747608 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc65006c670 0x7fc650156710 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc6400ab000 tx=0x7fc64001ac50 comp rx=0 tx=0).stop 2024-04-25T14:10:57.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:57.157+0000 7fc655800640 1 -- 172.21.15.86:0/3002747608 shutdown_connections 2024-04-25T14:10:57.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:57.157+0000 7fc655800640 1 --2- 172.21.15.86:0/3002747608 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fc62c06f4c0 0x7fc62c0719d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:57.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:57.157+0000 7fc655800640 1 --2- 172.21.15.86:0/3002747608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6500cef20 0x7fc6501fadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:57.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:57.157+0000 7fc655800640 1 --2- 172.21.15.86:0/3002747608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6500cdf50 0x7fc650150c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:57.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:57.157+0000 7fc655800640 1 --2- 172.21.15.86:0/3002747608 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc65006c670 0x7fc650156710 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:57.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:57.157+0000 7fc655800640 1 -- 172.21.15.86:0/3002747608 >> 172.21.15.86:0/3002747608 conn(0x7fc65009ba30 msgr2=0x7fc65006fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:57.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:57.157+0000 7fc655800640 1 -- 172.21.15.86:0/3002747608 shutdown_connections 2024-04-25T14:10:57.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:57.157+0000 7fc655800640 1 -- 172.21.15.86:0/3002747608 wait complete. 2024-04-25T14:10:57.175 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T14:10:57.175 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T14:10:57.175 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-25T14:10:57.176 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T14:10:57.176 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T14:10:57.176 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T14:10:57.176 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T14:10:57.176 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T14:10:57.176 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:10:57.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:57.176 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-25T14:10:57.185 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:10:57.185 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:10:57.185 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:10:57.185 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:57.185 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-25T14:10:57.185 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:57.185 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:10:57.185 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:10:57.185 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-25T14:10:57.186 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:10:57.186 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:10:57.186 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-25T14:10:57.186 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:10:57.186 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:10:57.186 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T14:10:57.186 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:10:57.186 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T14:10:57.186 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42007sec preferred_lft 42007sec 2024-04-25T14:10:57.186 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-25T14:10:57.186 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:57.186 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T14:10:57.187 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:dc:87:9c:96 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:10:57.187 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T14:10:57.187 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:57.187 INFO:teuthology.orchestra.run.smithi086.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T14:10:57.187 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T14:10:57.187 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T14:10:57.187 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:57.187 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T14:10:57.187 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:57.187 INFO:teuthology.orchestra.run.smithi086.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:10:57.187 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T14:10:57.188 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T14:10:57.188 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:57.188 INFO:teuthology.orchestra.run.smithi086.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:10:57.188 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T14:10:57.188 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T14:10:57.188 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:57.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:57.189 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T14:10:57.253 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-25T14:10:57.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:57.256 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-25T14:10:57.307 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-25T14:10:57.308 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T14:10:57.308 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T14:10:57.308 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T14:10:57.308 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T14:10:57.308 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T14:10:57.308 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T14:10:57.308 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:10:57.308 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:10:57.308 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:10:57.308 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T14:10:57.309 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:10:57.309 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T14:10:57.309 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T14:10:57.309 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T14:10:57.309 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T14:10:57.309 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T14:10:57.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:10:57.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:10:57.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:57.446 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T14:10:57.455 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T14:10:57.456 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T14:10:57.456 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:10:57.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:57.456 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-25T14:10:57.515 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:10:57.515 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T14:10:57.564 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:10:57.564 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:10:57.564 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T14:10:57.612 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T14:10:57.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:57.612 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:10:57.716 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:10:57.716 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T14:10:57.716 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:10:57.734 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:10:57.734 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T14:10:57.734 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:10:57.734 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:10:57.734 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:10:57.745 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:10:57.745 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T14:10:57.745 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:10:57.745 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:10:57.745 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:10:57.746 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790" 2024-04-25T14:10:57.746 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790:/ 2024-04-25T14:10:57.746 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb" will pass to kernel 2024-04-25T14:10:57.749 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T14:10:57.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:57.750 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T14:10:57.770 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:10:57.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:10:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:10:57.878 INFO:teuthology.orchestra.run.smithi086.stdout:client8770 2024-04-25T14:10:57.879 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:10:57.879 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:10:57.887 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T14:10:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.089+0000 7ff584800640 1 Processor -- start 2024-04-25T14:10:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.089+0000 7ff584800640 1 -- start start 2024-04-25T14:10:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.089+0000 7ff584800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff57c0cfca0 0x7ff57c06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.089+0000 7ff584800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff57c0d16e0 0x7ff57c0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.089+0000 7ff584800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff57c06c670 0x7ff57c06ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.089+0000 7ff584800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff57c070470 con 0x7ff57c0d16e0 2024-04-25T14:10:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.089+0000 7ff584800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff57c0d2030 con 0x7ff57c0cfca0 2024-04-25T14:10:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.089+0000 7ff584800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff57c072bd0 con 0x7ff57c06c670 2024-04-25T14:10:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.089+0000 7ff581400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff57c0d16e0 0x7ff57c0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.089+0000 7ff581400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff57c0d16e0 0x7ff57c0d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46366/0 (socket says 172.21.15.86:46366) 2024-04-25T14:10:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.089+0000 7ff581400640 1 -- 172.21.15.86:0/1554870106 learned_addr learned my addr 172.21.15.86:0/1554870106 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.089+0000 7ff582800640 1 --2- 172.21.15.86:0/1554870106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff57c06c670 0x7ff57c06ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.089+0000 7ff581e00640 1 --2- 172.21.15.86:0/1554870106 >> v2:172.21.15.155:3300/0 conn(0x7ff57c0cfca0 0x7ff57c06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.089+0000 7ff581400640 1 -- 172.21.15.86:0/1554870106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff57c06c670 msgr2=0x7ff57c06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.089+0000 7ff581400640 1 --2- 172.21.15.86:0/1554870106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff57c06c670 0x7ff57c06ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.089+0000 7ff581400640 1 -- 172.21.15.86:0/1554870106 >> v2:172.21.15.155:3300/0 conn(0x7ff57c0cfca0 msgr2=0x7ff57c06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.089+0000 7ff581400640 1 --2- 172.21.15.86:0/1554870106 >> v2:172.21.15.155:3300/0 conn(0x7ff57c0cfca0 0x7ff57c06f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.089+0000 7ff581400640 1 -- 172.21.15.86:0/1554870106 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff57c06f930 con 0x7ff57c0d16e0 2024-04-25T14:10:58.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.093+0000 7ff581400640 1 --2- 172.21.15.86:0/1554870106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff57c0d16e0 0x7ff57c0d1af0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0d22a0 tx=0x7ff56c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d7b57584359f963 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:58.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.093+0000 7ff580a00640 1 -- 172.21.15.86:0/1554870106 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c0ab5f0 con 0x7ff57c0d16e0 2024-04-25T14:10:58.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.093+0000 7ff580a00640 1 -- 172.21.15.86:0/1554870106 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff56c0b0070 con 0x7ff57c0d16e0 2024-04-25T14:10:58.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.093+0000 7ff580a00640 1 -- 172.21.15.86:0/1554870106 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c0ab9b0 con 0x7ff57c0d16e0 2024-04-25T14:10:58.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.093+0000 7ff584800640 1 -- 172.21.15.86:0/1554870106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff57c0d16e0 msgr2=0x7ff57c0d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:58.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.093+0000 7ff584800640 1 --2- 172.21.15.86:0/1554870106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff57c0d16e0 0x7ff57c0d1af0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0d22a0 tx=0x7ff56c01c520 comp rx=0 tx=0).stop 2024-04-25T14:10:58.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.093+0000 7ff584800640 1 -- 172.21.15.86:0/1554870106 shutdown_connections 2024-04-25T14:10:58.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.093+0000 7ff584800640 1 --2- 172.21.15.86:0/1554870106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff57c06c670 0x7ff57c06ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:58.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.093+0000 7ff584800640 1 --2- 172.21.15.86:0/1554870106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff57c0d16e0 0x7ff57c0d1af0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:58.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.093+0000 7ff584800640 1 --2- 172.21.15.86:0/1554870106 >> v2:172.21.15.155:3300/0 conn(0x7ff57c0cfca0 0x7ff57c06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:58.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.093+0000 7ff584800640 1 -- 172.21.15.86:0/1554870106 >> 172.21.15.86:0/1554870106 conn(0x7ff57c0b4650 msgr2=0x7ff57c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:58.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.093+0000 7ff584800640 1 -- 172.21.15.86:0/1554870106 shutdown_connections 2024-04-25T14:10:58.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.093+0000 7ff584800640 1 -- 172.21.15.86:0/1554870106 wait complete. 2024-04-25T14:10:58.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.097+0000 7ff584800640 1 Processor -- start 2024-04-25T14:10:58.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.097+0000 7ff584800640 1 -- start start 2024-04-25T14:10:58.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.097+0000 7ff584800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff57c06c670 0x7ff57c09a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:58.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.097+0000 7ff584800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff57c0cfca0 0x7ff57c15bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:58.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.097+0000 7ff584800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff57c0d16e0 0x7ff57c15c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:58.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.097+0000 7ff584800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff57c0738c0 con 0x7ff57c0d16e0 2024-04-25T14:10:58.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.097+0000 7ff584800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff57c06d3d0 con 0x7ff57c06c670 2024-04-25T14:10:58.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.097+0000 7ff584800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff57c073a40 con 0x7ff57c0cfca0 2024-04-25T14:10:58.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.097+0000 7ff582800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff57c0d16e0 0x7ff57c15c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:58.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.097+0000 7ff582800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff57c0d16e0 0x7ff57c15c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46370/0 (socket says 172.21.15.86:46370) 2024-04-25T14:10:58.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.097+0000 7ff582800640 1 -- 172.21.15.86:0/4116464507 learned_addr learned my addr 172.21.15.86:0/4116464507 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:58.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.097+0000 7ff581e00640 1 --2- 172.21.15.86:0/4116464507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff57c06c670 0x7ff57c09a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:58.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.101+0000 7ff581400640 1 --2- 172.21.15.86:0/4116464507 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff57c0cfca0 0x7ff57c15bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:58.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.101+0000 7ff582800640 1 -- 172.21.15.86:0/4116464507 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff57c0cfca0 msgr2=0x7ff57c15bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:58.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.101+0000 7ff582800640 1 --2- 172.21.15.86:0/4116464507 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff57c0cfca0 0x7ff57c15bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:58.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.101+0000 7ff582800640 1 -- 172.21.15.86:0/4116464507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff57c06c670 msgr2=0x7ff57c09a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:58.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.101+0000 7ff582800640 1 --2- 172.21.15.86:0/4116464507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff57c06c670 0x7ff57c09a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:58.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.101+0000 7ff582800640 1 -- 172.21.15.86:0/4116464507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff57c15a1f0 con 0x7ff57c0d16e0 2024-04-25T14:10:58.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.101+0000 7ff582800640 1 --2- 172.21.15.86:0/4116464507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff57c0d16e0 0x7ff57c15c3e0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7ff57800efc0 tx=0x7ff57800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:58.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.101+0000 7ff576000640 1 -- 172.21.15.86:0/4116464507 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff578018020 con 0x7ff57c0d16e0 2024-04-25T14:10:58.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.101+0000 7ff576000640 1 -- 172.21.15.86:0/4116464507 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff578014480 con 0x7ff57c0d16e0 2024-04-25T14:10:58.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.101+0000 7ff576000640 1 -- 172.21.15.86:0/4116464507 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff578009e60 con 0x7ff57c0d16e0 2024-04-25T14:10:58.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.101+0000 7ff584800640 1 -- 172.21.15.86:0/4116464507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff57c15a490 con 0x7ff57c0d16e0 2024-04-25T14:10:58.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.101+0000 7ff584800640 1 -- 172.21.15.86:0/4116464507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff57c152ea0 con 0x7ff57c0d16e0 2024-04-25T14:10:58.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.101+0000 7ff584800640 1 -- 172.21.15.86:0/4116464507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff57c152bf0 con 0x7ff57c0d16e0 2024-04-25T14:10:58.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.101+0000 7ff576000640 1 -- 172.21.15.86:0/4116464507 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 54) ==== 93253+0+0 (secure 0 0 0) 0x7ff57801b020 con 0x7ff57c0d16e0 2024-04-25T14:10:58.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.105+0000 7ff576000640 1 --2- 172.21.15.86:0/4116464507 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7ff56006f490 0x7ff5600719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:58.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.105+0000 7ff576000640 1 -- 172.21.15.86:0/4116464507 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 6144+0+0 (secure 0 0 0) 0x7ff578090930 con 0x7ff57c0d16e0 2024-04-25T14:10:58.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.105+0000 7ff581e00640 1 --2- 172.21.15.86:0/4116464507 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7ff56006f490 0x7ff5600719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:58.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.105+0000 7ff581e00640 1 --2- 172.21.15.86:0/4116464507 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7ff56006f490 0x7ff5600719a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff57c070710 tx=0x7ff570009190 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:58.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.105+0000 7ff576000640 1 -- 172.21.15.86:0/4116464507 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff578022030 con 0x7ff57c0d16e0 2024-04-25T14:10:58.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.309+0000 7ff584800640 1 -- 172.21.15.86:0/4116464507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff57c09f4b0 con 0x7ff57c0d16e0 2024-04-25T14:10:58.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.309+0000 7ff576000640 1 -- 172.21.15.86:0/4116464507 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v94) ==== 53+0+73 (secure 0 0 0) 0x7ff578010040 con 0x7ff57c0d16e0 2024-04-25T14:10:58.306 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T14:10:58.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.309+0000 7ff584800640 1 -- 172.21.15.86:0/4116464507 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7ff56006f490 msgr2=0x7ff5600719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:58.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.309+0000 7ff584800640 1 --2- 172.21.15.86:0/4116464507 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7ff56006f490 0x7ff5600719a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff57c070710 tx=0x7ff570009190 comp rx=0 tx=0).stop 2024-04-25T14:10:58.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.309+0000 7ff584800640 1 -- 172.21.15.86:0/4116464507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff57c0d16e0 msgr2=0x7ff57c15c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:58.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.309+0000 7ff584800640 1 --2- 172.21.15.86:0/4116464507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff57c0d16e0 0x7ff57c15c3e0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7ff57800efc0 tx=0x7ff57800c490 comp rx=0 tx=0).stop 2024-04-25T14:10:58.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.309+0000 7ff584800640 1 -- 172.21.15.86:0/4116464507 shutdown_connections 2024-04-25T14:10:58.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.309+0000 7ff584800640 1 --2- 172.21.15.86:0/4116464507 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7ff56006f490 0x7ff5600719a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:58.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.309+0000 7ff584800640 1 --2- 172.21.15.86:0/4116464507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff57c0d16e0 0x7ff57c15c3e0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:58.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.309+0000 7ff584800640 1 --2- 172.21.15.86:0/4116464507 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff57c0cfca0 0x7ff57c15bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:58.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.309+0000 7ff584800640 1 --2- 172.21.15.86:0/4116464507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff57c06c670 0x7ff57c09a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:58.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.309+0000 7ff584800640 1 -- 172.21.15.86:0/4116464507 >> 172.21.15.86:0/4116464507 conn(0x7ff57c0b4650 msgr2=0x7ff57c1566a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:58.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.309+0000 7ff584800640 1 -- 172.21.15.86:0/4116464507 shutdown_connections 2024-04-25T14:10:58.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:58.309+0000 7ff584800640 1 -- 172.21.15.86:0/4116464507 wait complete. 2024-04-25T14:10:58.327 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T14:10:58.327 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T14:10:58.328 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2024-04-25T14:10:58.328 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T14:10:58.328 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T14:10:58.328 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T14:10:58.328 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T14:10:58.328 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T14:10:58.328 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:10:58.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:58.328 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2024-04-25T14:10:58.338 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:10:58.338 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:10:58.338 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:10:58.339 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:58.339 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2024-04-25T14:10:58.339 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:58.339 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:10:58.339 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2024-04-25T14:10:58.339 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f0 2024-04-25T14:10:58.339 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:10:58.339 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2024-04-25T14:10:58.339 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f1 2024-04-25T14:10:58.339 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:10:58.339 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2024-04-25T14:10:58.339 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T14:10:58.339 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2024-04-25T14:10:58.339 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T14:10:58.339 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 41999sec preferred_lft 41999sec 2024-04-25T14:10:58.340 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::ec4:7aff:fe88:6ded/64 scope link 2024-04-25T14:10:58.340 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:58.340 INFO:teuthology.orchestra.run.smithi155.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T14:10:58.340 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 02:42:b1:ff:60:50 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:10:58.340 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T14:10:58.340 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:58.340 INFO:teuthology.orchestra.run.smithi155.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T14:10:58.340 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T14:10:58.340 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T14:10:58.340 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:58.340 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T14:10:58.341 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:58.341 INFO:teuthology.orchestra.run.smithi155.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:10:58.341 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T14:10:58.341 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T14:10:58.341 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:58.341 INFO:teuthology.orchestra.run.smithi155.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:10:58.341 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T14:10:58.341 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T14:10:58.341 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:10:58.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:58.342 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T14:10:58.408 INFO:teuthology.orchestra.run.smithi155.stdout:1 2024-04-25T14:10:58.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:58.411 DEBUG:teuthology.orchestra.run.smithi155:> route 2024-04-25T14:10:58.465 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2024-04-25T14:10:58.465 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T14:10:58.465 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T14:10:58.465 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T14:10:58.465 INFO:teuthology.orchestra.run.smithi155.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T14:10:58.465 INFO:teuthology.orchestra.run.smithi155.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T14:10:58.465 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T14:10:58.465 INFO:teuthology.orchestra.run.smithi155.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:10:58.465 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:10:58.465 INFO:teuthology.orchestra.run.smithi155.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:10:58.466 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T14:10:58.466 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:10:58.466 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-04-25T14:10:58.466 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T14:10:58.466 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T14:10:58.466 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T14:10:58.466 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-04-25T14:10:58.561 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:10:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:10:58.610 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:10:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:10:58.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:58.618 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2024-04-25T14:10:58.627 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T14:10:58.627 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T14:10:58.627 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:10:58.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:58.628 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.0 up 2024-04-25T14:10:58.687 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:10:58.688 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T14:10:58.740 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:10:58.740 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:10:58.740 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T14:10:58.791 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T14:10:58.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:58.792 DEBUG:teuthology.orchestra.run.smithi155:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:10:58.893 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:10:58.893 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T14:10:58.893 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:10:58.911 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:10:58.911 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T14:10:58.911 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:10:58.911 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:10:58.911 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:10:58.921 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:10:58.921 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T14:10:58.921 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:10:58.921 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:10:58.921 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:10:58.921 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790" 2024-04-25T14:10:58.921 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790:/ 2024-04-25T14:10:58.921 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb" will pass to kernel 2024-04-25T14:10:58.925 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T14:10:58.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:58.925 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T14:10:58.944 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:10:59.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:10:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:10:59.037 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:10:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:10:59.044 INFO:teuthology.orchestra.run.smithi155.stdout:client8779 2024-04-25T14:10:59.044 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:10:59.044 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:10:59.096 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:10:59.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.257+0000 7ff4e9a00640 1 Processor -- start 2024-04-25T14:10:59.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.257+0000 7ff4e9a00640 1 -- start start 2024-04-25T14:10:59.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.257+0000 7ff4e9a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4e4070330 0x7ff4e40d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.257+0000 7ff4e9a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff4e40d1c20 0x7ff4e406ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.257+0000 7ff4e9a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4e40d2030 con 0x7ff4e406f270 2024-04-25T14:10:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.257+0000 7ff4e9a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff4e415e5c0 con 0x7ff4e40d1c20 2024-04-25T14:10:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.257+0000 7ff4e9a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e415e8a0 con 0x7ff4e4070330 2024-04-25T14:10:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.257+0000 7ff4da000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff4e40d1c20 0x7ff4e406ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.257+0000 7ff4da000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff4e40d1c20 0x7ff4e406ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44050/0 (socket says 172.21.15.86:44050) 2024-04-25T14:10:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.257+0000 7ff4da000640 1 -- 172.21.15.86:0/1273414481 learned_addr learned my addr 172.21.15.86:0/1273414481 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.257+0000 7ff4e2a00640 1 --2- 172.21.15.86:0/1273414481 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4e4070330 0x7ff4e40d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.257+0000 7ff4da000640 1 -- 172.21.15.86:0/1273414481 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4e4070330 msgr2=0x7ff4e40d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.257+0000 7ff4da000640 1 --2- 172.21.15.86:0/1273414481 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4e4070330 0x7ff4e40d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:59.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.257+0000 7ff4da000640 1 -- 172.21.15.86:0/1273414481 >> v1:172.21.15.155:6789/0 conn(0x7ff4e406f270 legacy=0x7ff4e406f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:59.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.257+0000 7ff4da000640 1 -- 172.21.15.86:0/1273414481 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e415eb80 con 0x7ff4e40d1c20 2024-04-25T14:10:59.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.261+0000 7ff4da000640 1 --2- 172.21.15.86:0/1273414481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4e40d1c20 0x7ff4e406ed30 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0098a0 tx=0x7ff4cc01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e16fac474f87169 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:59.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.261+0000 7ff4e2000640 1 -- 172.21.15.86:0/1273414481 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc0b0070 con 0x7ff4e40d1c20 2024-04-25T14:10:59.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.261+0000 7ff4e2000640 1 -- 172.21.15.86:0/1273414481 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4cc0ab600 con 0x7ff4e40d1c20 2024-04-25T14:10:59.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.261+0000 7ff4e2000640 1 -- 172.21.15.86:0/1273414481 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc0ab9d0 con 0x7ff4e40d1c20 2024-04-25T14:10:59.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.261+0000 7ff4e9a00640 1 -- 172.21.15.86:0/1273414481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4e40d1c20 msgr2=0x7ff4e406ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:59.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.261+0000 7ff4e9a00640 1 --2- 172.21.15.86:0/1273414481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4e40d1c20 0x7ff4e406ed30 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0098a0 tx=0x7ff4cc01c4b0 comp rx=0 tx=0).stop 2024-04-25T14:10:59.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.261+0000 7ff4e9a00640 1 -- 172.21.15.86:0/1273414481 shutdown_connections 2024-04-25T14:10:59.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.261+0000 7ff4e9a00640 1 --2- 172.21.15.86:0/1273414481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4e40d1c20 0x7ff4e406ed30 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:59.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.261+0000 7ff4e9a00640 1 --2- 172.21.15.86:0/1273414481 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4e4070330 0x7ff4e40d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:59.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.261+0000 7ff4e9a00640 1 -- 172.21.15.86:0/1273414481 >> 172.21.15.86:0/1273414481 conn(0x7ff4e40b4650 msgr2=0x7ff4e40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:59.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.261+0000 7ff4e9a00640 1 -- 172.21.15.86:0/1273414481 shutdown_connections 2024-04-25T14:10:59.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.261+0000 7ff4e9a00640 1 -- 172.21.15.86:0/1273414481 wait complete. 2024-04-25T14:10:59.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.265+0000 7ff4e9a00640 1 Processor -- start 2024-04-25T14:10:59.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.265+0000 7ff4e9a00640 1 -- start start 2024-04-25T14:10:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.265+0000 7ff4e9a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4e406f270 0x7ff4e415a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.265+0000 7ff4e9a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4e4070330 0x7ff4e4073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.265+0000 7ff4e9a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4e40d1c20 0x7ff4e40ae270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.265+0000 7ff4e9a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e40708c0 con 0x7ff4e406f270 2024-04-25T14:10:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.265+0000 7ff4e9a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e4070740 con 0x7ff4e4070330 2024-04-25T14:10:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.265+0000 7ff4e9a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e416afc0 con 0x7ff4e40d1c20 2024-04-25T14:10:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.265+0000 7ff4e2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4e406f270 0x7ff4e415a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.265+0000 7ff4e2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4e406f270 0x7ff4e415a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44054/0 (socket says 172.21.15.86:44054) 2024-04-25T14:10:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.265+0000 7ff4e2a00640 1 -- 172.21.15.86:0/2012213768 learned_addr learned my addr 172.21.15.86:0/2012213768 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.265+0000 7ff4e3400640 1 --2- 172.21.15.86:0/2012213768 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4e40d1c20 0x7ff4e40ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.265+0000 7ff4da000640 1 --2- 172.21.15.86:0/2012213768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4e4070330 0x7ff4e4073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.265+0000 7ff4e2a00640 1 -- 172.21.15.86:0/2012213768 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4e40d1c20 msgr2=0x7ff4e40ae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.265+0000 7ff4e2a00640 1 --2- 172.21.15.86:0/2012213768 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4e40d1c20 0x7ff4e40ae270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.265+0000 7ff4e2a00640 1 -- 172.21.15.86:0/2012213768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4e4070330 msgr2=0x7ff4e4073bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.265+0000 7ff4e2a00640 1 --2- 172.21.15.86:0/2012213768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4e4070330 0x7ff4e4073bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.265+0000 7ff4e2a00640 1 -- 172.21.15.86:0/2012213768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e40cfaf0 con 0x7ff4e406f270 2024-04-25T14:10:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.265+0000 7ff4da000640 1 --2- 172.21.15.86:0/2012213768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4e4070330 0x7ff4e4073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.265+0000 7ff4e2a00640 1 --2- 172.21.15.86:0/2012213768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4e406f270 0x7ff4e415a4a0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7ff4d400e870 tx=0x7ff4d400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.265+0000 7ff4db400640 1 -- 172.21.15.86:0/2012213768 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4017070 con 0x7ff4e406f270 2024-04-25T14:10:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.265+0000 7ff4db400640 1 -- 172.21.15.86:0/2012213768 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4d400c940 con 0x7ff4e406f270 2024-04-25T14:10:59.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.265+0000 7ff4db400640 1 -- 172.21.15.86:0/2012213768 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d400cd10 con 0x7ff4e406f270 2024-04-25T14:10:59.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.265+0000 7ff4e9a00640 1 -- 172.21.15.86:0/2012213768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e40cfdd0 con 0x7ff4e406f270 2024-04-25T14:10:59.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.265+0000 7ff4e9a00640 1 -- 172.21.15.86:0/2012213768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e40736a0 con 0x7ff4e406f270 2024-04-25T14:10:59.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.269+0000 7ff4e9a00640 1 -- 172.21.15.86:0/2012213768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e4069d00 con 0x7ff4e406f270 2024-04-25T14:10:59.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.269+0000 7ff4db400640 1 -- 172.21.15.86:0/2012213768 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 55) ==== 93309+0+0 (secure 0 0 0) 0x7ff4d401a070 con 0x7ff4e406f270 2024-04-25T14:10:59.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.269+0000 7ff4db400640 1 --2- 172.21.15.86:0/2012213768 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7ff4c006f520 0x7ff4c0071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:59.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.269+0000 7ff4db400640 1 -- 172.21.15.86:0/2012213768 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 6144+0+0 (secure 0 0 0) 0x7ff4d408f4a0 con 0x7ff4e406f270 2024-04-25T14:10:59.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.269+0000 7ff4da000640 1 --2- 172.21.15.86:0/2012213768 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7ff4c006f520 0x7ff4c0071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:59.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.269+0000 7ff4da000640 1 --2- 172.21.15.86:0/2012213768 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7ff4c006f520 0x7ff4c0071a30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0ad000 tx=0x7ff4cc0a6000 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:59.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.273+0000 7ff4db400640 1 -- 172.21.15.86:0/2012213768 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff4d405c0e0 con 0x7ff4e406f270 2024-04-25T14:10:59.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.465+0000 7ff4e9a00640 1 -- 172.21.15.86:0/2012213768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4e4076bf0 con 0x7ff4e406f270 2024-04-25T14:10:59.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.469+0000 7ff4db400640 1 -- 172.21.15.86:0/2012213768 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5186 (secure 0 0 0) 0x7ff4d405b9e0 con 0x7ff4e406f270 2024-04-25T14:10:59.465 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:10:59.465 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":94,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2258481468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2258481468},{"type":"v1","addr":"172.21.15.155:6834","nonce":2258481468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":8976,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1064843151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1064843151},{"type":"v1","addr":"172.21.15.86:6835","nonce":1064843151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":8985,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1940755625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1940755625},{"type":"v1","addr":"172.21.15.86:6837","nonce":1940755625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:10:45.497688+0000","modified":"2024-04-25T14:10:48.475869+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8991":{"gid":8991,"name":"b","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6835/544985680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":544985680},{"type":"v1","addr":"172.21.15.155:6835","nonce":544985680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8991,"qdb_cluster":[8991]},"id":9}]} 2024-04-25T14:10:59.465 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 94 2024-04-25T14:10:59.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.469+0000 7ff4e9a00640 1 -- 172.21.15.86:0/2012213768 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7ff4c006f520 msgr2=0x7ff4c0071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:59.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.469+0000 7ff4e9a00640 1 --2- 172.21.15.86:0/2012213768 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7ff4c006f520 0x7ff4c0071a30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0ad000 tx=0x7ff4cc0a6000 comp rx=0 tx=0).stop 2024-04-25T14:10:59.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.469+0000 7ff4e9a00640 1 -- 172.21.15.86:0/2012213768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4e406f270 msgr2=0x7ff4e415a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:59.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.469+0000 7ff4e9a00640 1 --2- 172.21.15.86:0/2012213768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4e406f270 0x7ff4e415a4a0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7ff4d400e870 tx=0x7ff4d400ec70 comp rx=0 tx=0).stop 2024-04-25T14:10:59.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.469+0000 7ff4e9a00640 1 -- 172.21.15.86:0/2012213768 shutdown_connections 2024-04-25T14:10:59.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.469+0000 7ff4e9a00640 1 --2- 172.21.15.86:0/2012213768 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4e40d1c20 0x7ff4e40ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:59.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.469+0000 7ff4e9a00640 1 --2- 172.21.15.86:0/2012213768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4e4070330 0x7ff4e4073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:59.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.469+0000 7ff4e9a00640 1 --2- 172.21.15.86:0/2012213768 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7ff4c006f520 0x7ff4c0071a30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:59.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.469+0000 7ff4e9a00640 1 --2- 172.21.15.86:0/2012213768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4e406f270 0x7ff4e415a4a0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:59.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.469+0000 7ff4e9a00640 1 -- 172.21.15.86:0/2012213768 >> 172.21.15.86:0/2012213768 conn(0x7ff4e40b4650 msgr2=0x7ff4e40b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:59.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.469+0000 7ff4e9a00640 1 -- 172.21.15.86:0/2012213768 shutdown_connections 2024-04-25T14:10:59.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.469+0000 7ff4e9a00640 1 -- 172.21.15.86:0/2012213768 wait complete. 2024-04-25T14:10:59.487 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable stats 2024-04-25T14:10:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.637+0000 7fa58b800640 1 Processor -- start 2024-04-25T14:10:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.637+0000 7fa58b800640 1 -- start start 2024-04-25T14:10:59.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa58b800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa584071530 0x7fa58415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:59.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa58b800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5840cd8d0 con 0x7fa5840d2c80 2024-04-25T14:10:59.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa58b800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5840ce970 con 0x7fa5841616e0 2024-04-25T14:10:59.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa58b800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa5840ce720 con 0x7fa584071530 2024-04-25T14:10:59.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa588e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa5840d2c80 0x7fa584070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42048/0 (socket says 172.21.15.86:42048) 2024-04-25T14:10:59.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa588e00640 1 -- 172.21.15.86:0/4128276386 learned_addr learned my addr 172.21.15.86:0/4128276386 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:10:59.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa583e00640 1 --2- 172.21.15.86:0/4128276386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa584071530 0x7fa58415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:59.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa583400640 1 -- 172.21.15.86:0/4128276386 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1240969824 0 0) 0x7fa5840cd8d0 con 0x7fa5840d2c80 2024-04-25T14:10:59.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa583400640 1 -- 172.21.15.86:0/4128276386 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa56c003530 con 0x7fa5840d2c80 2024-04-25T14:10:59.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa583400640 1 -- 172.21.15.86:0/4128276386 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 491437004 0 0) 0x7fa56c003530 con 0x7fa5840d2c80 2024-04-25T14:10:59.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa583400640 1 -- 172.21.15.86:0/4128276386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa584071530 msgr2=0x7fa58415dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:59.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa583400640 1 --2- 172.21.15.86:0/4128276386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa584071530 0x7fa58415dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa583400640 1 -- 172.21.15.86:0/4128276386 >> v1:172.21.15.155:6789/0 conn(0x7fa5841616e0 legacy=0x7fa584161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa583400640 1 -- 172.21.15.86:0/4128276386 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5840d1be0 con 0x7fa5840d2c80 2024-04-25T14:10:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa583400640 1 -- 172.21.15.86:0/4128276386 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fa578001680 con 0x7fa5840d2c80 2024-04-25T14:10:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa583400640 1 -- 172.21.15.86:0/4128276386 >> v1:172.21.15.86:6789/0 conn(0x7fa5840d2c80 legacy=0x7fa584070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa583400640 1 --2- 172.21.15.86:0/4128276386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa56c003a60 0x7fa56c003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa583400640 1 --2- 172.21.15.86:0/4128276386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa56c004810 0x7fa56c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa583400640 1 --2- 172.21.15.86:0/4128276386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa56c0051b0 0x7fa56c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa583400640 1 -- 172.21.15.86:0/4128276386 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5840ce720 con 0x7fa56c004810 2024-04-25T14:10:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa583400640 1 -- 172.21.15.86:0/4128276386 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c00bf10 con 0x7fa56c003a60 2024-04-25T14:10:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa583400640 1 -- 172.21.15.86:0/4128276386 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa56c00c110 con 0x7fa56c0051b0 2024-04-25T14:10:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa583e00640 1 --2- 172.21.15.86:0/4128276386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa56c004810 0x7fa56c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa588e00640 1 --2- 172.21.15.86:0/4128276386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa56c003a60 0x7fa56c003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa589800640 1 --2- 172.21.15.86:0/4128276386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa56c0051b0 0x7fa56c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa583e00640 1 -- 172.21.15.86:0/4128276386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa56c0051b0 msgr2=0x7fa56c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa583e00640 1 --2- 172.21.15.86:0/4128276386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa56c0051b0 0x7fa56c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:59.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa583e00640 1 -- 172.21.15.86:0/4128276386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa56c003a60 msgr2=0x7fa56c003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:59.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa583e00640 1 --2- 172.21.15.86:0/4128276386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa56c003a60 0x7fa56c003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:59.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.641+0000 7fa583e00640 1 -- 172.21.15.86:0/4128276386 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56c003530 con 0x7fa56c004810 2024-04-25T14:10:59.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.645+0000 7fa583e00640 1 --2- 172.21.15.86:0/4128276386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa56c004810 0x7fa56c004c20 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fa570009ac0 tx=0x7fa57001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242ee1bdf3122356 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:59.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.645+0000 7fa583400640 1 -- 172.21.15.86:0/4128276386 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa570009690 con 0x7fa56c004810 2024-04-25T14:10:59.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.645+0000 7fa583400640 1 -- 172.21.15.86:0/4128276386 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5700059a0 con 0x7fa56c004810 2024-04-25T14:10:59.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.645+0000 7fa583400640 1 -- 172.21.15.86:0/4128276386 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa570005d90 con 0x7fa56c004810 2024-04-25T14:10:59.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.645+0000 7fa58b800640 1 -- 172.21.15.86:0/4128276386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa56c004810 msgr2=0x7fa56c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:59.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.645+0000 7fa58b800640 1 --2- 172.21.15.86:0/4128276386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa56c004810 0x7fa56c004c20 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fa570009ac0 tx=0x7fa57001ca70 comp rx=0 tx=0).stop 2024-04-25T14:10:59.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.645+0000 7fa589800640 1 -- 172.21.15.86:0/4128276386 reap_dead start 2024-04-25T14:10:59.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.645+0000 7fa58b800640 1 -- 172.21.15.86:0/4128276386 shutdown_connections 2024-04-25T14:10:59.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.645+0000 7fa58b800640 1 -- 172.21.15.86:0/4128276386 >> 172.21.15.86:0/4128276386 conn(0x7fa5840cffc0 msgr2=0x7fa5840d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:59.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.645+0000 7fa58b800640 1 -- 172.21.15.86:0/4128276386 shutdown_connections 2024-04-25T14:10:59.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.645+0000 7fa58b800640 1 -- 172.21.15.86:0/4128276386 wait complete. 2024-04-25T14:10:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.649+0000 7fa58b800640 1 Processor -- start 2024-04-25T14:10:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.649+0000 7fa58b800640 1 -- start start 2024-04-25T14:10:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.649+0000 7fa58b800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa56c004810 0x7fa5840b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:59.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.649+0000 7fa58b800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa584152230 0x7fa5841f8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:59.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.649+0000 7fa58b800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa584153d10 0x7fa584154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:59.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.649+0000 7fa58b800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5840d2480 con 0x7fa56c004810 2024-04-25T14:10:59.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.649+0000 7fa58b800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa5840d21a0 con 0x7fa584153d10 2024-04-25T14:10:59.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.649+0000 7fa58b800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa5840d29a0 con 0x7fa584152230 2024-04-25T14:10:59.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.649+0000 7fa588e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa56c004810 0x7fa5840b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:59.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.649+0000 7fa588e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa56c004810 0x7fa5840b4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44076/0 (socket says 172.21.15.86:44076) 2024-04-25T14:10:59.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.649+0000 7fa588e00640 1 -- 172.21.15.86:0/901078830 learned_addr learned my addr 172.21.15.86:0/901078830 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:10:59.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.649+0000 7fa583e00640 1 --2- 172.21.15.86:0/901078830 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa584152230 0x7fa5841f8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:59.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.649+0000 7fa589800640 1 --2- 172.21.15.86:0/901078830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa584153d10 0x7fa584154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:59.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.653+0000 7fa588e00640 1 -- 172.21.15.86:0/901078830 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa584152230 msgr2=0x7fa5841f8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:59.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.653+0000 7fa588e00640 1 --2- 172.21.15.86:0/901078830 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa584152230 0x7fa5841f8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:59.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.653+0000 7fa588e00640 1 -- 172.21.15.86:0/901078830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa584153d10 msgr2=0x7fa584154120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:59.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.653+0000 7fa588e00640 1 --2- 172.21.15.86:0/901078830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa584153d10 0x7fa584154120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:59.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.653+0000 7fa588e00640 1 -- 172.21.15.86:0/901078830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5840b36a0 con 0x7fa56c004810 2024-04-25T14:10:59.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.653+0000 7fa588e00640 1 --2- 172.21.15.86:0/901078830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa56c004810 0x7fa5840b4740 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fa578012490 tx=0x7fa578012950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:59.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.653+0000 7fa580c00640 1 -- 172.21.15.86:0/901078830 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57801c070 con 0x7fa56c004810 2024-04-25T14:10:59.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.653+0000 7fa580c00640 1 -- 172.21.15.86:0/901078830 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa578014830 con 0x7fa56c004810 2024-04-25T14:10:59.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.653+0000 7fa580c00640 1 -- 172.21.15.86:0/901078830 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa578014c20 con 0x7fa56c004810 2024-04-25T14:10:59.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.653+0000 7fa58b800640 1 -- 172.21.15.86:0/901078830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5840b3980 con 0x7fa56c004810 2024-04-25T14:10:59.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.653+0000 7fa58b800640 1 -- 172.21.15.86:0/901078830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5840b3f40 con 0x7fa56c004810 2024-04-25T14:10:59.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.653+0000 7fa58b800640 1 -- 172.21.15.86:0/901078830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa58415ca90 con 0x7fa56c004810 2024-04-25T14:10:59.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.653+0000 7fa580c00640 1 -- 172.21.15.86:0/901078830 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 55) ==== 93309+0+0 (secure 0 0 0) 0x7fa5780153e0 con 0x7fa56c004810 2024-04-25T14:10:59.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.657+0000 7fa580c00640 1 --2- 172.21.15.86:0/901078830 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fa56406f520 0x7fa564071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:59.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.657+0000 7fa580c00640 1 -- 172.21.15.86:0/901078830 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 6144+0+0 (secure 0 0 0) 0x7fa5780920b0 con 0x7fa56c004810 2024-04-25T14:10:59.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.657+0000 7fa583e00640 1 --2- 172.21.15.86:0/901078830 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fa56406f520 0x7fa564071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:59.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.657+0000 7fa583e00640 1 --2- 172.21.15.86:0/901078830 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fa56406f520 0x7fa564071a30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa5700ae000 tx=0x7fa5700046d0 comp rx=0 tx=0).ready entity=mgr.8338 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:59.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.661+0000 7fa580c00640 1 -- 172.21.15.86:0/901078830 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa57805e610 con 0x7fa56c004810 2024-04-25T14:10:59.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:10:59.857+0000 7fa58b800640 1 -- 172.21.15.86:0/901078830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "stats"} v 0) -- 0x7fa584076bf0 con 0x7fa56c004810 2024-04-25T14:11:00.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.605+0000 7fa580c00640 1 -- 172.21.15.86:0/901078830 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "stats"}]=0 v56) ==== 84+0+0 (secure 0 0 0) 0x7fa578062910 con 0x7fa56c004810 2024-04-25T14:11:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.613+0000 7fa58b800640 1 -- 172.21.15.86:0/901078830 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fa56406f520 msgr2=0x7fa564071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.613+0000 7fa58b800640 1 --2- 172.21.15.86:0/901078830 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fa56406f520 0x7fa564071a30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa5700ae000 tx=0x7fa5700046d0 comp rx=0 tx=0).stop 2024-04-25T14:11:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.613+0000 7fa58b800640 1 -- 172.21.15.86:0/901078830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa56c004810 msgr2=0x7fa5840b4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.613+0000 7fa58b800640 1 --2- 172.21.15.86:0/901078830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa56c004810 0x7fa5840b4740 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fa578012490 tx=0x7fa578012950 comp rx=0 tx=0).stop 2024-04-25T14:11:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.613+0000 7fa58b800640 1 -- 172.21.15.86:0/901078830 shutdown_connections 2024-04-25T14:11:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.613+0000 7fa58b800640 1 --2- 172.21.15.86:0/901078830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa584153d10 0x7fa584154120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.613+0000 7fa58b800640 1 --2- 172.21.15.86:0/901078830 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa584152230 0x7fa5841f8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:00.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.613+0000 7fa58b800640 1 --2- 172.21.15.86:0/901078830 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7fa56406f520 0x7fa564071a30 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa5700ae000 tx=0x7fa5700046d0 comp rx=0 tx=0).stop 2024-04-25T14:11:00.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.613+0000 7fa58b800640 1 --2- 172.21.15.86:0/901078830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa56c004810 0x7fa5840b4740 secure :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fa578012490 tx=0x7fa578012950 comp rx=0 tx=0).stop 2024-04-25T14:11:00.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.613+0000 7fa58b800640 1 -- 172.21.15.86:0/901078830 >> 172.21.15.86:0/901078830 conn(0x7fa5840cffc0 msgr2=0x7fa584070e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:00.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.613+0000 7fa58b800640 1 -- 172.21.15.86:0/901078830 shutdown_connections 2024-04-25T14:11:00.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.613+0000 7fa58b800640 1 -- 172.21.15.86:0/901078830 wait complete. 2024-04-25T14:11:00.634 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:11:00.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.793+0000 7f7abb000640 1 Processor -- start 2024-04-25T14:11:00.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.793+0000 7f7abb000640 1 -- start start 2024-04-25T14:11:00.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.797+0000 7f7abb000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7ab40b7a30 0x7f7ab40b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:00.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.797+0000 7f7abb000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ab40b3750 con 0x7f7ab40d15e0 2024-04-25T14:11:00.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.797+0000 7f7abb000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ab40ce7b0 con 0x7f7ab41610d0 2024-04-25T14:11:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.797+0000 7f7abb000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f7ab4161620 con 0x7f7ab40b7a30 2024-04-25T14:11:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.797+0000 7f7ab3400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7ab40b7a30 0x7f7ab40b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.797+0000 7f7ab9000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7ab40d15e0 0x7f7ab40d19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42078/0 (socket says 172.21.15.86:42078) 2024-04-25T14:11:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.797+0000 7f7ab9000640 1 -- 172.21.15.86:0/2989700674 learned_addr learned my addr 172.21.15.86:0/2989700674 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.797+0000 7f7ab3e00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f7ab41610d0 0x7f7ab40b2250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:53264/0 (socket says 172.21.15.86:53264) 2024-04-25T14:11:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.797+0000 7f7ab2a00640 1 -- 172.21.15.86:0/2989700674 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2015696456 0 0) 0x7f7ab40ce7b0 con 0x7f7ab41610d0 2024-04-25T14:11:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.797+0000 7f7ab2a00640 1 -- 172.21.15.86:0/2989700674 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a98003530 con 0x7f7ab41610d0 2024-04-25T14:11:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.797+0000 7f7ab3400640 1 -- 172.21.15.86:0/2989700674 >> v1:172.21.15.155:6789/0 conn(0x7f7ab41610d0 legacy=0x7f7ab40b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.797+0000 7f7ab3400640 1 -- 172.21.15.86:0/2989700674 >> v1:172.21.15.86:6789/0 conn(0x7f7ab40d15e0 legacy=0x7f7ab40d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.797+0000 7f7ab2a00640 1 -- 172.21.15.86:0/2989700674 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 130044331 0 0) 0x7f7ab40b3750 con 0x7f7ab40d15e0 2024-04-25T14:11:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.797+0000 7f7ab3400640 1 -- 172.21.15.86:0/2989700674 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab40b39f0 con 0x7f7ab40b7a30 2024-04-25T14:11:00.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.797+0000 7f7ab3400640 1 --2- 172.21.15.86:0/2989700674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ab40b7a30 0x7f7ab40b2960 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7aa00097c0 tx=0x7f7aa001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8acf42082ee913a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:00.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.801+0000 7f7ab2a00640 1 -- 172.21.15.86:0/2989700674 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa00b0070 con 0x7f7ab40b7a30 2024-04-25T14:11:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.801+0000 7f7ab2a00640 1 -- 172.21.15.86:0/2989700674 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa00ab600 con 0x7f7ab40b7a30 2024-04-25T14:11:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.801+0000 7f7ab2a00640 1 -- 172.21.15.86:0/2989700674 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa00ab9d0 con 0x7f7ab40b7a30 2024-04-25T14:11:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.801+0000 7f7abb000640 1 -- 172.21.15.86:0/2989700674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ab40b7a30 msgr2=0x7f7ab40b2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.801+0000 7f7abb000640 1 --2- 172.21.15.86:0/2989700674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ab40b7a30 0x7f7ab40b2960 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7aa00097c0 tx=0x7f7aa001c4a0 comp rx=0 tx=0).stop 2024-04-25T14:11:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.801+0000 7f7abb000640 1 -- 172.21.15.86:0/2989700674 shutdown_connections 2024-04-25T14:11:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.801+0000 7f7abb000640 1 --2- 172.21.15.86:0/2989700674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ab40b7a30 0x7f7ab40b2960 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.801+0000 7f7abb000640 1 -- 172.21.15.86:0/2989700674 >> 172.21.15.86:0/2989700674 conn(0x7f7ab409ba30 msgr2=0x7f7ab40b7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.801+0000 7f7abb000640 1 -- 172.21.15.86:0/2989700674 shutdown_connections 2024-04-25T14:11:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.801+0000 7f7abb000640 1 -- 172.21.15.86:0/2989700674 wait complete. 2024-04-25T14:11:00.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.801+0000 7f7abb000640 1 Processor -- start 2024-04-25T14:11:00.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.801+0000 7f7abb000640 1 -- start start 2024-04-25T14:11:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.805+0000 7f7abb000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ab40b7a30 0x7f7ab4159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.805+0000 7f7abb000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ab40d15e0 0x7f7ab415e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.805+0000 7f7abb000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ab41610d0 0x7f7ab415be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.805+0000 7f7abb000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab40d2100 con 0x7f7ab41610d0 2024-04-25T14:11:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.805+0000 7f7abb000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab40b7e40 con 0x7f7ab40b7a30 2024-04-25T14:11:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.805+0000 7f7abb000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7ab40d2280 con 0x7f7ab40d15e0 2024-04-25T14:11:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.805+0000 7f7ab3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ab40b7a30 0x7f7ab4159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.805+0000 7f7ab9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ab41610d0 0x7f7ab415be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.805+0000 7f7ab3400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ab40d15e0 0x7f7ab415e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.805+0000 7f7ab3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ab40b7a30 0x7f7ab4159020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:47872/0 (socket says 172.21.15.86:47872) 2024-04-25T14:11:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.805+0000 7f7ab3e00640 1 -- 172.21.15.86:0/2529689835 learned_addr learned my addr 172.21.15.86:0/2529689835 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.805+0000 7f7ab3e00640 1 -- 172.21.15.86:0/2529689835 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ab40d15e0 msgr2=0x7f7ab415e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.805+0000 7f7ab3e00640 1 --2- 172.21.15.86:0/2529689835 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ab40d15e0 0x7f7ab415e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.805+0000 7f7ab3e00640 1 -- 172.21.15.86:0/2529689835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ab41610d0 msgr2=0x7f7ab415be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.805+0000 7f7ab3e00640 1 --2- 172.21.15.86:0/2529689835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ab41610d0 0x7f7ab415be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.805+0000 7f7ab3e00640 1 -- 172.21.15.86:0/2529689835 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab40c75a0 con 0x7f7ab40b7a30 2024-04-25T14:11:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.805+0000 7f7ab3e00640 1 --2- 172.21.15.86:0/2529689835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ab40b7a30 0x7f7ab4159020 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400c3c0 tx=0x7f7aa400c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.805+0000 7f7aabe00640 1 -- 172.21.15.86:0/2529689835 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa40074e0 con 0x7f7ab40b7a30 2024-04-25T14:11:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.805+0000 7f7aabe00640 1 -- 172.21.15.86:0/2529689835 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa40047f0 con 0x7f7ab40b7a30 2024-04-25T14:11:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.805+0000 7f7abb000640 1 -- 172.21.15.86:0/2529689835 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab40c7ad0 con 0x7f7ab40b7a30 2024-04-25T14:11:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.805+0000 7f7aabe00640 1 -- 172.21.15.86:0/2529689835 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa40076a0 con 0x7f7ab40b7a30 2024-04-25T14:11:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.805+0000 7f7abb000640 1 -- 172.21.15.86:0/2529689835 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab40c8090 con 0x7f7ab40b7a30 2024-04-25T14:11:00.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.805+0000 7f7abb000640 1 -- 172.21.15.86:0/2529689835 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ab40a05a0 con 0x7f7ab40b7a30 2024-04-25T14:11:00.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.809+0000 7f7aabe00640 1 -- 172.21.15.86:0/2529689835 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 56) ==== 93318+0+0 (secure 0 0 0) 0x7f7aa401a020 con 0x7f7ab40b7a30 2024-04-25T14:11:00.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.809+0000 7f7aabe00640 1 --2- 172.21.15.86:0/2529689835 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f7a9406f570 0x7f7a94071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:00.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.809+0000 7f7ab3400640 1 -- 172.21.15.86:0/2529689835 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f7a9406f570 msgr2=0x7f7a94071a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/3964666460 2024-04-25T14:11:00.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.809+0000 7f7aabe00640 1 -- 172.21.15.86:0/2529689835 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 6144+0+0 (secure 0 0 0) 0x7f7aa408edf0 con 0x7f7ab40b7a30 2024-04-25T14:11:00.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.809+0000 7f7ab3400640 1 --2- 172.21.15.86:0/2529689835 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f7a9406f570 0x7f7a94071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:11:00.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.813+0000 7f7aabe00640 1 -- 172.21.15.86:0/2529689835 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7aa405b350 con 0x7f7ab40b7a30 2024-04-25T14:11:00.861 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:00.863+0000 7eff89741180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:11:00.875 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:00.877+0000 7f5b2964a180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:11:00.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:00.993+0000 7f7abb000640 1 -- 172.21.15.86:0/2529689835 --> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f7ab40c91d0 con 0x7f7a9406f570 2024-04-25T14:11:01.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:01.013+0000 7f7ab3400640 1 -- 172.21.15.86:0/2529689835 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f7a9406f570 msgr2=0x7f7a94071a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/3964666460 2024-04-25T14:11:01.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:01.013+0000 7f7ab3400640 1 --2- 172.21.15.86:0/2529689835 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f7a9406f570 0x7f7a94071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:11:01.288 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:01.287+0000 7eff89741180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:11:01.297 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:01.301+0000 7f5b2964a180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:11:01.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:01.417+0000 7f7ab3400640 1 -- 172.21.15.86:0/2529689835 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f7a9406f570 msgr2=0x7f7a94071a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/3964666460 2024-04-25T14:11:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:01.417+0000 7f7ab3400640 1 --2- 172.21.15.86:0/2529689835 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f7a9406f570 0x7f7a94071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T14:11:01.437 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:01.439+0000 7eff89741180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:11:01.446 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:01.449+0000 7f5b2964a180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:11:01.912 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:01.915+0000 7eff89741180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:11:01.917 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:01.921+0000 7f5b2964a180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:11:01.971 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:01.971+0000 7eff89741180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:11:01.975 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:01.977+0000 7f5b2964a180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:11:02.028 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:02.027+0000 7eff89741180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:11:02.031 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:02.033+0000 7f5b2964a180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:11:02.121 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:02.125+0000 7f5b2964a180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:11:02.121 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:02.123+0000 7eff89741180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:11:02.177 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:02.181+0000 7f5b2964a180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:11:02.179 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:02.179+0000 7eff89741180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:11:02.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:02.221+0000 7f7ab3400640 1 -- 172.21.15.86:0/2529689835 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f7a9406f570 msgr2=0x7f7a94071a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/3964666460 2024-04-25T14:11:02.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:02.221+0000 7f7ab3400640 1 --2- 172.21.15.86:0/2529689835 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f7a9406f570 0x7f7a94071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T14:11:02.263 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:02.265+0000 7f5b2964a180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:11:02.264 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:02.263+0000 7eff89741180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:11:02.323 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:02.325+0000 7f5b2964a180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:11:02.324 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:02.327+0000 7eff89741180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:11:02.475 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:02.477+0000 7f5b2964a180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:11:02.478 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:02.479+0000 7eff89741180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:11:02.535 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:02.537+0000 7f5b2964a180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:11:02.537 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:02.539+0000 7eff89741180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:11:02.774 INFO:tasks.ceph.mgr.y.smithi086.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:11:02.774 INFO:tasks.ceph.mgr.y.smithi086.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:11:02.775 INFO:tasks.ceph.mgr.y.smithi086.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:11:02.778 INFO:tasks.ceph.mgr.x.smithi155.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:11:02.778 INFO:tasks.ceph.mgr.x.smithi155.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:11:02.778 INFO:tasks.ceph.mgr.x.smithi155.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:11:02.783 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:02.785+0000 7f5b2964a180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:11:02.787 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:02.787+0000 7eff89741180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:11:02.930 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:02.933+0000 7f5b2964a180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:11:02.937 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:02.939+0000 7eff89741180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:11:03.055 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:03.057+0000 7f5b2964a180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:11:03.063 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:03.063+0000 7eff89741180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:11:03.126 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:03.129+0000 7f5b2964a180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:11:03.135 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:03.135+0000 7eff89741180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:11:03.275 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:03.277+0000 7f5b2964a180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:11:03.284 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:03.287+0000 7eff89741180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:11:03.816 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:03.817+0000 7f5b2964a180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:11:03.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:03.825+0000 7f7ab3400640 1 -- 172.21.15.86:0/2529689835 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f7a9406f570 msgr2=0x7f7a94071a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/3964666460 2024-04-25T14:11:03.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:03.825+0000 7f7ab3400640 1 --2- 172.21.15.86:0/2529689835 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f7a9406f570 0x7f7a94071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T14:11:03.833 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:03.835+0000 7eff89741180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:11:03.901 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:03.905+0000 7f5b2964a180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:11:03.918 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:03.919+0000 7eff89741180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:11:04.060 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:04.065+0000 7f5b2964a180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:11:04.080 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:04.079+0000 7eff89741180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:11:04.144 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:04.149+0000 7f5b2964a180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:11:04.165 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:04.167+0000 7eff89741180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:11:04.463 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:04.465+0000 7f5b2964a180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:11:04.488 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:04.487+0000 7eff89741180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:11:04.525 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:04.529+0000 7f5b2964a180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:11:04.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:04.541+0000 7f7aabe00640 1 -- 172.21.15.86:0/2529689835 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mgrmap(e 57) ==== 92927+0+0 (secure 0 0 0) 0x7f7aa40575e0 con 0x7f7ab40b7a30 2024-04-25T14:11:04.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:04.541+0000 7f7aabe00640 1 -- 172.21.15.86:0/2529689835 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f7a9406f570 msgr2=0x7f7a94071a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:11:04.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:04.541+0000 7f7aabe00640 1 --2- 172.21.15.86:0/2529689835 >> [v2:172.21.15.86:6832/3964666460,v1:172.21.15.86:6833/3964666460] conn(0x7f7a9406f570 0x7f7a94071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:04.556 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:04.559+0000 7eff89741180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:11:04.602 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:04.605+0000 7f5aa5400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:04.603 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:04.605+0000 7f5aa5400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:04.603 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:04.605+0000 7f5aa5400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:04.603 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:04.605+0000 7f5aa5400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:04.603 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:04.605+0000 7f5aa5400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:04.603 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:04.605+0000 7f5aa9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:04.603 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:04.605+0000 7f5aa9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:04.603 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:04.605+0000 7f5aa9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:04.604 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:04.605+0000 7f5aa9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:04.604 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:04.605+0000 7f5aa9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:05.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:05.549+0000 7f7aabe00640 1 -- 172.21.15.86:0/2529689835 <== mon.1 v2:172.21.15.155:3300/0 8 ==== mgrmap(e 58) ==== 93054+0+0 (secure 0 0 0) 0x7f7aa40074e0 con 0x7f7ab40b7a30 2024-04-25T14:11:05.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:05.549+0000 7f7aabe00640 1 --2- 172.21.15.86:0/2529689835 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f7a94077b70 0x7f7a94079f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:05.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:05.549+0000 7f7aabe00640 1 -- 172.21.15.86:0/2529689835 --> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f7ab40c91d0 con 0x7f7a94077b70 2024-04-25T14:11:05.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:05.549+0000 7f7ab3400640 1 --2- 172.21.15.86:0/2529689835 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f7a94077b70 0x7f7a94079f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:05.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:05.553+0000 7f7ab3400640 1 --2- 172.21.15.86:0/2529689835 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f7a94077b70 0x7f7a94079f90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7aa00ad000 tx=0x7f7aa00a6000 comp rx=0 tx=0).ready entity=mgr.9198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:05.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:05.557+0000 7f7aabe00640 1 -- 172.21.15.86:0/2529689835 <== mgr.9198 v2:172.21.15.86:6832/2221276863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f7ab40c91d0 con 0x7f7a94077b70 2024-04-25T14:11:05.553 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {}, "global_metrics": {}, "metrics": {"delayed_ranks": []}} 2024-04-25T14:11:05.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:05.557+0000 7f7abb000640 1 -- 172.21.15.86:0/2529689835 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f7a94077b70 msgr2=0x7f7a94079f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:05.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:05.557+0000 7f7abb000640 1 --2- 172.21.15.86:0/2529689835 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f7a94077b70 0x7f7a94079f90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7aa00ad000 tx=0x7f7aa00a6000 comp rx=0 tx=0).stop 2024-04-25T14:11:05.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:05.557+0000 7f7abb000640 1 -- 172.21.15.86:0/2529689835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ab40b7a30 msgr2=0x7f7ab4159020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:05.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:05.557+0000 7f7abb000640 1 --2- 172.21.15.86:0/2529689835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ab40b7a30 0x7f7ab4159020 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400c3c0 tx=0x7f7aa400c880 comp rx=0 tx=0).stop 2024-04-25T14:11:05.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:05.557+0000 7f7ab9000640 1 -- 172.21.15.86:0/2529689835 reap_dead start 2024-04-25T14:11:05.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:05.557+0000 7f7abb000640 1 -- 172.21.15.86:0/2529689835 shutdown_connections 2024-04-25T14:11:05.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:05.557+0000 7f7abb000640 1 -- 172.21.15.86:0/2529689835 >> 172.21.15.86:0/2529689835 conn(0x7f7ab409ba30 msgr2=0x7f7ab4070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:05.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:05.557+0000 7f7abb000640 1 -- 172.21.15.86:0/2529689835 shutdown_connections 2024-04-25T14:11:05.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:05.557+0000 7f7abb000640 1 -- 172.21.15.86:0/2529689835 wait complete. 2024-04-25T14:11:06.580 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:11:06.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.745+0000 7f3ee5600640 1 Processor -- start 2024-04-25T14:11:06.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.745+0000 7f3ee5600640 1 -- start start 2024-04-25T14:11:06.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.749+0000 7f3ee5600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3ee01651d0 0x7f3ee01675f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:06.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.749+0000 7f3ee5600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ee00b0a60 con 0x7f3ee015c2a0 2024-04-25T14:11:06.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.749+0000 7f3ee5600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ee00ae840 con 0x7f3ee006fd50 2024-04-25T14:11:06.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.749+0000 7f3ee5600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3ee006ef90 con 0x7f3ee01651d0 2024-04-25T14:11:06.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.749+0000 7f3edea00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3ee015c2a0 0x7f3ee00ae130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42094/0 (socket says 172.21.15.86:42094) 2024-04-25T14:11:06.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.749+0000 7f3edea00640 1 -- 172.21.15.86:0/1854882548 learned_addr learned my addr 172.21.15.86:0/1854882548 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:06.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.749+0000 7f3edf400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3ee01651d0 0x7f3ee01675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:06.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.749+0000 7f3edd600640 1 -- 172.21.15.86:0/1854882548 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2096245041 0 0) 0x7f3ee00b0a60 con 0x7f3ee015c2a0 2024-04-25T14:11:06.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.749+0000 7f3edd600640 1 -- 172.21.15.86:0/1854882548 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ec4003530 con 0x7f3ee015c2a0 2024-04-25T14:11:06.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.749+0000 7f3edf400640 1 -- 172.21.15.86:0/1854882548 >> v1:172.21.15.155:6789/0 conn(0x7f3ee006fd50 legacy=0x7f3ee0070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:06.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.749+0000 7f3edf400640 1 -- 172.21.15.86:0/1854882548 >> v1:172.21.15.86:6789/0 conn(0x7f3ee015c2a0 legacy=0x7f3ee00ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:06.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.749+0000 7f3edf400640 1 -- 172.21.15.86:0/1854882548 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee006f270 con 0x7f3ee01651d0 2024-04-25T14:11:06.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.753+0000 7f3edf400640 1 --2- 172.21.15.86:0/1854882548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ee01651d0 0x7f3ee01675f0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f3ed800a960 tx=0x7f3ed801e8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a349b47ded5da4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:06.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.753+0000 7f3edd600640 1 -- 172.21.15.86:0/1854882548 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed80b4070 con 0x7f3ee01651d0 2024-04-25T14:11:06.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.753+0000 7f3edd600640 1 -- 172.21.15.86:0/1854882548 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ed80a7910 con 0x7f3ee01651d0 2024-04-25T14:11:06.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.753+0000 7f3edd600640 1 -- 172.21.15.86:0/1854882548 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed800b040 con 0x7f3ee01651d0 2024-04-25T14:11:06.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.753+0000 7f3ee5600640 1 -- 172.21.15.86:0/1854882548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ee01651d0 msgr2=0x7f3ee01675f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:06.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.753+0000 7f3ee5600640 1 --2- 172.21.15.86:0/1854882548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ee01651d0 0x7f3ee01675f0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f3ed800a960 tx=0x7f3ed801e8c0 comp rx=0 tx=0).stop 2024-04-25T14:11:06.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.753+0000 7f3ee5600640 1 -- 172.21.15.86:0/1854882548 shutdown_connections 2024-04-25T14:11:06.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.753+0000 7f3ee5600640 1 --2- 172.21.15.86:0/1854882548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ee01651d0 0x7f3ee01675f0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:06.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.753+0000 7f3ee5600640 1 -- 172.21.15.86:0/1854882548 >> 172.21.15.86:0/1854882548 conn(0x7f3ee00b4650 msgr2=0x7f3ee00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:06.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.753+0000 7f3ee5600640 1 -- 172.21.15.86:0/1854882548 shutdown_connections 2024-04-25T14:11:06.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.753+0000 7f3ee5600640 1 -- 172.21.15.86:0/1854882548 wait complete. 2024-04-25T14:11:06.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.757+0000 7f3ee5600640 1 Processor -- start 2024-04-25T14:11:06.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.757+0000 7f3ee5600640 1 -- start start 2024-04-25T14:11:06.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.757+0000 7f3ee5600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ee006fd50 0x7f3ee009a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:06.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.757+0000 7f3ee5600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ee015c2a0 0x7f3ee015f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:06.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.757+0000 7f3ee5600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ee01651d0 0x7f3ee01592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:06.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.757+0000 7f3ee5600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee015c830 con 0x7f3ee006fd50 2024-04-25T14:11:06.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.757+0000 7f3ee5600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee015c6b0 con 0x7f3ee01651d0 2024-04-25T14:11:06.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.757+0000 7f3ee5600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3ee0169b40 con 0x7f3ee015c2a0 2024-04-25T14:11:06.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.757+0000 7f3edea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ee006fd50 0x7f3ee009a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:06.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.757+0000 7f3edea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ee006fd50 0x7f3ee009a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44232/0 (socket says 172.21.15.86:44232) 2024-04-25T14:11:06.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.757+0000 7f3edea00640 1 -- 172.21.15.86:0/3826498577 learned_addr learned my addr 172.21.15.86:0/3826498577 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:06.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.757+0000 7f3ede000640 1 --2- 172.21.15.86:0/3826498577 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ee015c2a0 0x7f3ee015f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:06.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.757+0000 7f3edf400640 1 --2- 172.21.15.86:0/3826498577 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ee01651d0 0x7f3ee01592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:06.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.757+0000 7f3edea00640 1 -- 172.21.15.86:0/3826498577 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ee015c2a0 msgr2=0x7f3ee015f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:06.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.757+0000 7f3edea00640 1 --2- 172.21.15.86:0/3826498577 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ee015c2a0 0x7f3ee015f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.757+0000 7f3edea00640 1 -- 172.21.15.86:0/3826498577 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ee01651d0 msgr2=0x7f3ee01592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.757+0000 7f3edea00640 1 --2- 172.21.15.86:0/3826498577 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ee01651d0 0x7f3ee01592a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.757+0000 7f3edea00640 1 -- 172.21.15.86:0/3826498577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee0099a00 con 0x7f3ee006fd50 2024-04-25T14:11:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.757+0000 7f3edf400640 1 --2- 172.21.15.86:0/3826498577 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ee01651d0 0x7f3ee01592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.757+0000 7f3edea00640 1 --2- 172.21.15.86:0/3826498577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ee006fd50 0x7f3ee009a7c0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f3ed000cb10 tx=0x7f3ed000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.757+0000 7f3ed6a00640 1 -- 172.21.15.86:0/3826498577 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0017070 con 0x7f3ee006fd50 2024-04-25T14:11:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.757+0000 7f3ed6a00640 1 -- 172.21.15.86:0/3826498577 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ed000ac70 con 0x7f3ee006fd50 2024-04-25T14:11:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.757+0000 7f3ed6a00640 1 -- 172.21.15.86:0/3826498577 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0012640 con 0x7f3ee006fd50 2024-04-25T14:11:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.757+0000 7f3ee5600640 1 -- 172.21.15.86:0/3826498577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ee0099ce0 con 0x7f3ee006fd50 2024-04-25T14:11:06.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.757+0000 7f3ee5600640 1 -- 172.21.15.86:0/3826498577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee009a2a0 con 0x7f3ee006fd50 2024-04-25T14:11:06.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.761+0000 7f3ee5600640 1 -- 172.21.15.86:0/3826498577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ee0069d00 con 0x7f3ee006fd50 2024-04-25T14:11:06.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.761+0000 7f3ed6a00640 1 -- 172.21.15.86:0/3826498577 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 59) ==== 93262+0+0 (secure 0 0 0) 0x7f3ed001a070 con 0x7f3ee006fd50 2024-04-25T14:11:06.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.761+0000 7f3ed6a00640 1 --2- 172.21.15.86:0/3826498577 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f3ebc06f4e0 0x7f3ebc0719f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:06.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.761+0000 7f3ed6a00640 1 -- 172.21.15.86:0/3826498577 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6488+0+0 (secure 0 0 0) 0x7f3ed008f510 con 0x7f3ee006fd50 2024-04-25T14:11:06.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.761+0000 7f3ede000640 1 --2- 172.21.15.86:0/3826498577 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f3ebc06f4e0 0x7f3ebc0719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:06.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.761+0000 7f3ede000640 1 --2- 172.21.15.86:0/3826498577 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f3ebc06f4e0 0x7f3ebc0719f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0150cf0 tx=0x7f3ec8009040 comp rx=0 tx=0).ready entity=mgr.9198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:06.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.765+0000 7f3ed6a00640 1 -- 172.21.15.86:0/3826498577 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3ed005c000 con 0x7f3ee006fd50 2024-04-25T14:11:06.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.949+0000 7f3ee5600640 1 -- 172.21.15.86:0/3826498577 --> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f3ee0076bf0 con 0x7f3ebc06f4e0 2024-04-25T14:11:06.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.949+0000 7f3ed6a00640 1 -- 172.21.15.86:0/3826498577 <== mgr.9198 v2:172.21.15.86:6832/2221276863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f3ee0076bf0 con 0x7f3ebc06f4e0 2024-04-25T14:11:06.946 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {}, "global_metrics": {}, "metrics": {"delayed_ranks": []}} 2024-04-25T14:11:06.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.953+0000 7f3ee5600640 1 -- 172.21.15.86:0/3826498577 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f3ebc06f4e0 msgr2=0x7f3ebc0719f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:06.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.953+0000 7f3ee5600640 1 --2- 172.21.15.86:0/3826498577 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f3ebc06f4e0 0x7f3ebc0719f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0150cf0 tx=0x7f3ec8009040 comp rx=0 tx=0).stop 2024-04-25T14:11:06.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.953+0000 7f3ee5600640 1 -- 172.21.15.86:0/3826498577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ee006fd50 msgr2=0x7f3ee009a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:06.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.953+0000 7f3ee5600640 1 --2- 172.21.15.86:0/3826498577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ee006fd50 0x7f3ee009a7c0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f3ed000cb10 tx=0x7f3ed000a060 comp rx=0 tx=0).stop 2024-04-25T14:11:06.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.953+0000 7f3ee5600640 1 -- 172.21.15.86:0/3826498577 shutdown_connections 2024-04-25T14:11:06.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.953+0000 7f3ee5600640 1 --2- 172.21.15.86:0/3826498577 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f3ebc06f4e0 0x7f3ebc0719f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:06.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.953+0000 7f3ee5600640 1 --2- 172.21.15.86:0/3826498577 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ee01651d0 0x7f3ee01592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:06.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.953+0000 7f3ee5600640 1 --2- 172.21.15.86:0/3826498577 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ee015c2a0 0x7f3ee015f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:06.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.953+0000 7f3ee5600640 1 --2- 172.21.15.86:0/3826498577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ee006fd50 0x7f3ee009a7c0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:06.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.953+0000 7f3ee5600640 1 -- 172.21.15.86:0/3826498577 >> 172.21.15.86:0/3826498577 conn(0x7f3ee00b4650 msgr2=0x7f3ee0073320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:06.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.953+0000 7f3ee5600640 1 -- 172.21.15.86:0/3826498577 shutdown_connections 2024-04-25T14:11:06.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:06.953+0000 7f3ee5600640 1 -- 172.21.15.86:0/3826498577 wait complete. 2024-04-25T14:11:07.970 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:11:08.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.129+0000 7fdd4c800640 1 Processor -- start 2024-04-25T14:11:08.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.129+0000 7fdd4c800640 1 -- start start 2024-04-25T14:11:08.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.129+0000 7fdd4c800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdd440cfca0 0x7fdd4406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:08.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.129+0000 7fdd4c800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd440730b0 0x7fdd440734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:08.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.129+0000 7fdd4c800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd44072bd0 con 0x7fdd440d16e0 2024-04-25T14:11:08.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.129+0000 7fdd4c800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fdd44070470 con 0x7fdd440cfca0 2024-04-25T14:11:08.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.129+0000 7fdd4c800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdd4406f930 con 0x7fdd440730b0 2024-04-25T14:11:08.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.133+0000 7fdd49400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fdd440d16e0 0x7fdd440d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42100/0 (socket says 172.21.15.86:42100) 2024-04-25T14:11:08.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.133+0000 7fdd49400640 1 -- 172.21.15.86:0/1426673058 learned_addr learned my addr 172.21.15.86:0/1426673058 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.133+0000 7fdd4a800640 1 --2- 172.21.15.86:0/1426673058 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd440730b0 0x7fdd440734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.133+0000 7fdd49e00640 1 --2- 172.21.15.86:0/1426673058 >> v2:172.21.15.155:3300/0 conn(0x7fdd440cfca0 0x7fdd4406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.133+0000 7fdd48a00640 1 -- 172.21.15.86:0/1426673058 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 352669559 0 0) 0x7fdd44072bd0 con 0x7fdd440d16e0 2024-04-25T14:11:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.133+0000 7fdd48a00640 1 -- 172.21.15.86:0/1426673058 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd2c003530 con 0x7fdd440d16e0 2024-04-25T14:11:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.133+0000 7fdd48a00640 1 -- 172.21.15.86:0/1426673058 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1154295513 0 0) 0x7fdd2c003530 con 0x7fdd440d16e0 2024-04-25T14:11:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.133+0000 7fdd48a00640 1 -- 172.21.15.86:0/1426673058 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd440730b0 msgr2=0x7fdd440734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.133+0000 7fdd48a00640 1 --2- 172.21.15.86:0/1426673058 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd440730b0 0x7fdd440734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.133+0000 7fdd48a00640 1 -- 172.21.15.86:0/1426673058 >> v2:172.21.15.155:3300/0 conn(0x7fdd440cfca0 msgr2=0x7fdd4406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.133+0000 7fdd48a00640 1 --2- 172.21.15.86:0/1426673058 >> v2:172.21.15.155:3300/0 conn(0x7fdd440cfca0 0x7fdd4406f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.133+0000 7fdd48a00640 1 -- 172.21.15.86:0/1426673058 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd4406fbd0 con 0x7fdd440d16e0 2024-04-25T14:11:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.133+0000 7fdd48a00640 1 -- 172.21.15.86:0/1426673058 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fdd340016a0 con 0x7fdd440d16e0 2024-04-25T14:11:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.133+0000 7fdd48a00640 1 -- 172.21.15.86:0/1426673058 >> v1:172.21.15.86:6789/0 conn(0x7fdd440d16e0 legacy=0x7fdd440d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.133+0000 7fdd48a00640 1 --2- 172.21.15.86:0/1426673058 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd440730b0 0x7fdd2c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.133+0000 7fdd48a00640 1 --2- 172.21.15.86:0/1426673058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd2c003fb0 0x7fdd2c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.133+0000 7fdd48a00640 1 --2- 172.21.15.86:0/1426673058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd2c004d50 0x7fdd2c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:08.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.133+0000 7fdd48a00640 1 -- 172.21.15.86:0/1426673058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdd44070470 con 0x7fdd2c004d50 2024-04-25T14:11:08.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.133+0000 7fdd48a00640 1 -- 172.21.15.86:0/1426673058 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4406f930 con 0x7fdd2c003fb0 2024-04-25T14:11:08.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.133+0000 7fdd48a00640 1 -- 172.21.15.86:0/1426673058 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdd2c00bab0 con 0x7fdd440730b0 2024-04-25T14:11:08.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.133+0000 7fdd4a800640 1 --2- 172.21.15.86:0/1426673058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd2c004d50 0x7fdd2c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:08.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.133+0000 7fdd49e00640 1 --2- 172.21.15.86:0/1426673058 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd440730b0 0x7fdd2c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:08.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.133+0000 7fdd49400640 1 --2- 172.21.15.86:0/1426673058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd2c003fb0 0x7fdd2c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:08.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.133+0000 7fdd4a800640 1 -- 172.21.15.86:0/1426673058 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd440730b0 msgr2=0x7fdd2c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:08.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.133+0000 7fdd4a800640 1 --2- 172.21.15.86:0/1426673058 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd440730b0 0x7fdd2c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.133+0000 7fdd4a800640 1 -- 172.21.15.86:0/1426673058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd2c003fb0 msgr2=0x7fdd2c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:08.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.133+0000 7fdd4a800640 1 --2- 172.21.15.86:0/1426673058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd2c003fb0 0x7fdd2c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.133+0000 7fdd4a800640 1 -- 172.21.15.86:0/1426673058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd2c003530 con 0x7fdd2c004d50 2024-04-25T14:11:08.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.137+0000 7fdd4a800640 1 --2- 172.21.15.86:0/1426673058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd2c004d50 0x7fdd2c005160 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fdd40009f90 tx=0x7fdd4001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825cefcebd964e74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:08.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.137+0000 7fdd48a00640 1 -- 172.21.15.86:0/1426673058 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd400b2070 con 0x7fdd2c004d50 2024-04-25T14:11:08.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.137+0000 7fdd48a00640 1 -- 172.21.15.86:0/1426673058 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd400a5c70 con 0x7fdd2c004d50 2024-04-25T14:11:08.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.137+0000 7fdd48a00640 1 -- 172.21.15.86:0/1426673058 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd40009c10 con 0x7fdd2c004d50 2024-04-25T14:11:08.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.137+0000 7fdd4c800640 1 -- 172.21.15.86:0/1426673058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd2c004d50 msgr2=0x7fdd2c005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:08.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.137+0000 7fdd4c800640 1 --2- 172.21.15.86:0/1426673058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd2c004d50 0x7fdd2c005160 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fdd40009f90 tx=0x7fdd4001cc60 comp rx=0 tx=0).stop 2024-04-25T14:11:08.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.137+0000 7fdd4a800640 1 -- 172.21.15.86:0/1426673058 reap_dead start 2024-04-25T14:11:08.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.137+0000 7fdd4c800640 1 -- 172.21.15.86:0/1426673058 shutdown_connections 2024-04-25T14:11:08.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.137+0000 7fdd4c800640 1 -- 172.21.15.86:0/1426673058 >> 172.21.15.86:0/1426673058 conn(0x7fdd440b4650 msgr2=0x7fdd440b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:08.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.137+0000 7fdd4c800640 1 -- 172.21.15.86:0/1426673058 shutdown_connections 2024-04-25T14:11:08.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.137+0000 7fdd4c800640 1 -- 172.21.15.86:0/1426673058 wait complete. 2024-04-25T14:11:08.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.141+0000 7fdd4c800640 1 Processor -- start 2024-04-25T14:11:08.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.141+0000 7fdd4c800640 1 -- start start 2024-04-25T14:11:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.141+0000 7fdd4c800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd2c004d50 0x7fdd440cb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.141+0000 7fdd4c800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd4415b470 0x7fdd441ea570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.141+0000 7fdd4c800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd441eaab0 0x7fdd4409c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.141+0000 7fdd4c800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdd440734c0 con 0x7fdd2c004d50 2024-04-25T14:11:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.141+0000 7fdd4c800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdd44070190 con 0x7fdd4415b470 2024-04-25T14:11:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.141+0000 7fdd4c800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdd44073640 con 0x7fdd441eaab0 2024-04-25T14:11:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.141+0000 7fdd49e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd2c004d50 0x7fdd440cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.141+0000 7fdd49e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd2c004d50 0x7fdd440cb020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44240/0 (socket says 172.21.15.86:44240) 2024-04-25T14:11:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.141+0000 7fdd49e00640 1 -- 172.21.15.86:0/3726951419 learned_addr learned my addr 172.21.15.86:0/3726951419 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.141+0000 7fdd49400640 1 --2- 172.21.15.86:0/3726951419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd4415b470 0x7fdd441ea570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.141+0000 7fdd4a800640 1 --2- 172.21.15.86:0/3726951419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd441eaab0 0x7fdd4409c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.141+0000 7fdd49e00640 1 -- 172.21.15.86:0/3726951419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd441eaab0 msgr2=0x7fdd4409c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.141+0000 7fdd49e00640 1 --2- 172.21.15.86:0/3726951419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd441eaab0 0x7fdd4409c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.141+0000 7fdd49e00640 1 -- 172.21.15.86:0/3726951419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd4415b470 msgr2=0x7fdd441ea570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:08.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.141+0000 7fdd49e00640 1 --2- 172.21.15.86:0/3726951419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd4415b470 0x7fdd441ea570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.141+0000 7fdd49e00640 1 -- 172.21.15.86:0/3726951419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd441eaec0 con 0x7fdd2c004d50 2024-04-25T14:11:08.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.141+0000 7fdd49e00640 1 --2- 172.21.15.86:0/3726951419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd2c004d50 0x7fdd440cb020 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fdd380119a0 tx=0x7fdd38011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:08.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.141+0000 7fdd3e000640 1 -- 172.21.15.86:0/3726951419 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3800faf0 con 0x7fdd2c004d50 2024-04-25T14:11:08.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.141+0000 7fdd3e000640 1 -- 172.21.15.86:0/3726951419 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd3800fc90 con 0x7fdd2c004d50 2024-04-25T14:11:08.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.141+0000 7fdd3e000640 1 -- 172.21.15.86:0/3726951419 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd38014440 con 0x7fdd2c004d50 2024-04-25T14:11:08.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.141+0000 7fdd4c800640 1 -- 172.21.15.86:0/3726951419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd441f64d0 con 0x7fdd2c004d50 2024-04-25T14:11:08.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.141+0000 7fdd4c800640 1 -- 172.21.15.86:0/3726951419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd4409eca0 con 0x7fdd2c004d50 2024-04-25T14:11:08.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.145+0000 7fdd4c800640 1 -- 172.21.15.86:0/3726951419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd4409f4b0 con 0x7fdd2c004d50 2024-04-25T14:11:08.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.145+0000 7fdd3e000640 1 -- 172.21.15.86:0/3726951419 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 59) ==== 93262+0+0 (secure 0 0 0) 0x7fdd38012040 con 0x7fdd2c004d50 2024-04-25T14:11:08.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.145+0000 7fdd3e000640 1 --2- 172.21.15.86:0/3726951419 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fdd2806f4e0 0x7fdd280719f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:08.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.145+0000 7fdd3e000640 1 -- 172.21.15.86:0/3726951419 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6488+0+0 (secure 0 0 0) 0x7fdd3808fe10 con 0x7fdd2c004d50 2024-04-25T14:11:08.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.145+0000 7fdd49400640 1 --2- 172.21.15.86:0/3726951419 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fdd2806f4e0 0x7fdd280719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:08.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.145+0000 7fdd49400640 1 --2- 172.21.15.86:0/3726951419 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fdd2806f4e0 0x7fdd280719f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdd4409dbe0 tx=0x7fdd34009040 comp rx=0 tx=0).ready entity=mgr.9198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:08.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.149+0000 7fdd3e000640 1 -- 172.21.15.86:0/3726951419 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdd38093030 con 0x7fdd2c004d50 2024-04-25T14:11:08.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.329+0000 7fdd4c800640 1 -- 172.21.15.86:0/3726951419 --> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7fdd44069d00 con 0x7fdd2806f4e0 2024-04-25T14:11:08.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.329+0000 7fdd3e000640 1 -- 172.21.15.86:0/3726951419 <== mgr.9198 v2:172.21.15.86:6832/2221276863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fdd44069d00 con 0x7fdd2806f4e0 2024-04-25T14:11:08.327 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {}, "global_metrics": {}, "metrics": {"delayed_ranks": []}} 2024-04-25T14:11:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.333+0000 7fdd4c800640 1 -- 172.21.15.86:0/3726951419 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fdd2806f4e0 msgr2=0x7fdd280719f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.333+0000 7fdd4c800640 1 --2- 172.21.15.86:0/3726951419 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fdd2806f4e0 0x7fdd280719f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdd4409dbe0 tx=0x7fdd34009040 comp rx=0 tx=0).stop 2024-04-25T14:11:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.333+0000 7fdd4c800640 1 -- 172.21.15.86:0/3726951419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd2c004d50 msgr2=0x7fdd440cb020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.333+0000 7fdd4c800640 1 --2- 172.21.15.86:0/3726951419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd2c004d50 0x7fdd440cb020 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fdd380119a0 tx=0x7fdd38011e60 comp rx=0 tx=0).stop 2024-04-25T14:11:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.333+0000 7fdd4c800640 1 -- 172.21.15.86:0/3726951419 shutdown_connections 2024-04-25T14:11:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.333+0000 7fdd4c800640 1 --2- 172.21.15.86:0/3726951419 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fdd2806f4e0 0x7fdd280719f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.333+0000 7fdd4c800640 1 --2- 172.21.15.86:0/3726951419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd441eaab0 0x7fdd4409c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.333+0000 7fdd4c800640 1 --2- 172.21.15.86:0/3726951419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd4415b470 0x7fdd441ea570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.333+0000 7fdd4c800640 1 --2- 172.21.15.86:0/3726951419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd2c004d50 0x7fdd440cb020 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.333+0000 7fdd4c800640 1 -- 172.21.15.86:0/3726951419 >> 172.21.15.86:0/3726951419 conn(0x7fdd440b4650 msgr2=0x7fdd4415c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.333+0000 7fdd4c800640 1 -- 172.21.15.86:0/3726951419 shutdown_connections 2024-04-25T14:11:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:08.333+0000 7fdd4c800640 1 -- 172.21.15.86:0/3726951419 wait complete. 2024-04-25T14:11:09.350 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:11:09.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.509+0000 7f67b9c00640 1 Processor -- start 2024-04-25T14:11:09.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.509+0000 7f67b9c00640 1 -- start start 2024-04-25T14:11:09.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.509+0000 7f67b9c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67b40cfca0 0x7f67b406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.509+0000 7f67b9c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f67b40d16e0 0x7f67b40d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.509+0000 7f67b9c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67b40d2030 con 0x7f67b406c670 2024-04-25T14:11:09.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.509+0000 7f67b9c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f67b4070470 con 0x7f67b40d16e0 2024-04-25T14:11:09.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.509+0000 7f67b9c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f67b4072bd0 con 0x7f67b40cfca0 2024-04-25T14:11:09.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.509+0000 7f67b3400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f67b406c670 0x7f67b406ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34772/0 (socket says 172.21.15.86:34772) 2024-04-25T14:11:09.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.509+0000 7f67b3400640 1 -- 172.21.15.86:0/843770368 learned_addr learned my addr 172.21.15.86:0/843770368 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:09.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.509+0000 7f67b2000640 1 --2- 172.21.15.86:0/843770368 >> v2:172.21.15.155:3300/0 conn(0x7f67b40d16e0 0x7f67b40d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.509+0000 7f67b2a00640 1 --2- 172.21.15.86:0/843770368 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67b40cfca0 0x7f67b406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.509+0000 7f67b1600640 1 -- 172.21.15.86:0/843770368 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 856797566 0 0) 0x7f67ac0014a0 con 0x7f67b406c670 2024-04-25T14:11:09.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.509+0000 7f67b1600640 1 -- 172.21.15.86:0/843770368 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6798003530 con 0x7f67b406c670 2024-04-25T14:11:09.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.509+0000 7f67b2a00640 1 -- 172.21.15.86:0/843770368 >> v2:172.21.15.155:3300/0 conn(0x7f67b40d16e0 msgr2=0x7f67b40d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:09.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.509+0000 7f67b2a00640 1 --2- 172.21.15.86:0/843770368 >> v2:172.21.15.155:3300/0 conn(0x7f67b40d16e0 0x7f67b40d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.509+0000 7f67b2a00640 1 -- 172.21.15.86:0/843770368 >> v1:172.21.15.86:6789/0 conn(0x7f67b406c670 legacy=0x7f67b406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.509+0000 7f67b2a00640 1 -- 172.21.15.86:0/843770368 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b406f930 con 0x7f67b40cfca0 2024-04-25T14:11:09.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.509+0000 7f67b2000640 1 --2- 172.21.15.86:0/843770368 >> v2:172.21.15.155:3300/0 conn(0x7f67b40d16e0 0x7f67b40d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:11:09.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.513+0000 7f67b2a00640 1 --2- 172.21.15.86:0/843770368 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67b40cfca0 0x7f67b406f3f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f67a40097c0 tx=0x7f67a401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53cb1d95a8db6a43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:09.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.513+0000 7f67b1600640 1 -- 172.21.15.86:0/843770368 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a40b1070 con 0x7f67b40cfca0 2024-04-25T14:11:09.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.513+0000 7f67b1600640 1 -- 172.21.15.86:0/843770368 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67a40ac410 con 0x7f67b40cfca0 2024-04-25T14:11:09.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.513+0000 7f67b1600640 1 -- 172.21.15.86:0/843770368 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a40ac7e0 con 0x7f67b40cfca0 2024-04-25T14:11:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.513+0000 7f67b9c00640 1 -- 172.21.15.86:0/843770368 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67b40cfca0 msgr2=0x7f67b406f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.513+0000 7f67b9c00640 1 --2- 172.21.15.86:0/843770368 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67b40cfca0 0x7f67b406f3f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f67a40097c0 tx=0x7f67a401c4a0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.513+0000 7f67b9c00640 1 -- 172.21.15.86:0/843770368 shutdown_connections 2024-04-25T14:11:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.513+0000 7f67b9c00640 1 --2- 172.21.15.86:0/843770368 >> v2:172.21.15.155:3300/0 conn(0x7f67b40d16e0 0x7f67b40d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.513+0000 7f67b9c00640 1 --2- 172.21.15.86:0/843770368 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67b40cfca0 0x7f67b406f3f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.513+0000 7f67b9c00640 1 -- 172.21.15.86:0/843770368 >> 172.21.15.86:0/843770368 conn(0x7f67b40b4650 msgr2=0x7f67b40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.513+0000 7f67b9c00640 1 -- 172.21.15.86:0/843770368 shutdown_connections 2024-04-25T14:11:09.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.513+0000 7f67b9c00640 1 -- 172.21.15.86:0/843770368 wait complete. 2024-04-25T14:11:09.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.517+0000 7f67b9c00640 1 Processor -- start 2024-04-25T14:11:09.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.517+0000 7f67b9c00640 1 -- start start 2024-04-25T14:11:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.517+0000 7f67b9c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67b406c670 0x7f67b41590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.517+0000 7f67b9c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b40cfca0 0x7f67b4159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.517+0000 7f67b9c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67b40d16e0 0x7f67b409cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.517+0000 7f67b9c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f67b40cdee0 con 0x7f67b40cfca0 2024-04-25T14:11:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.517+0000 7f67b9c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f67b40cdd60 con 0x7f67b40d16e0 2024-04-25T14:11:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.517+0000 7f67b9c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f67b40ce060 con 0x7f67b406c670 2024-04-25T14:11:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.517+0000 7f67b2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b40cfca0 0x7f67b4159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.517+0000 7f67b2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b40cfca0 0x7f67b4159770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59588/0 (socket says 172.21.15.86:59588) 2024-04-25T14:11:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.517+0000 7f67b2000640 1 -- 172.21.15.86:0/558455518 learned_addr learned my addr 172.21.15.86:0/558455518 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.517+0000 7f67b2a00640 1 --2- 172.21.15.86:0/558455518 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67b406c670 0x7f67b41590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.517+0000 7f67b3400640 1 --2- 172.21.15.86:0/558455518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67b40d16e0 0x7f67b409cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.521+0000 7f67b2000640 1 -- 172.21.15.86:0/558455518 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67b406c670 msgr2=0x7f67b41590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.521+0000 7f67b2000640 1 --2- 172.21.15.86:0/558455518 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67b406c670 0x7f67b41590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.521+0000 7f67b2000640 1 -- 172.21.15.86:0/558455518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67b40d16e0 msgr2=0x7f67b409cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.521+0000 7f67b2000640 1 --2- 172.21.15.86:0/558455518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67b40d16e0 0x7f67b409cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.521+0000 7f67b2000640 1 -- 172.21.15.86:0/558455518 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b40b2cc0 con 0x7f67b40cfca0 2024-04-25T14:11:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.521+0000 7f67b2000640 1 --2- 172.21.15.86:0/558455518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b40cfca0 0x7f67b4159770 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f67a000e9b0 tx=0x7f67a000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.521+0000 7f67aaa00640 1 -- 172.21.15.86:0/558455518 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a0017070 con 0x7f67b40cfca0 2024-04-25T14:11:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.521+0000 7f67aaa00640 1 -- 172.21.15.86:0/558455518 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67a000ca00 con 0x7f67b40cfca0 2024-04-25T14:11:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.521+0000 7f67aaa00640 1 -- 172.21.15.86:0/558455518 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a000cdd0 con 0x7f67b40cfca0 2024-04-25T14:11:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.521+0000 7f67b9c00640 1 -- 172.21.15.86:0/558455518 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b40b2140 con 0x7f67b40cfca0 2024-04-25T14:11:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.521+0000 7f67b9c00640 1 -- 172.21.15.86:0/558455518 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b40b2700 con 0x7f67b40cfca0 2024-04-25T14:11:09.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.521+0000 7f67b9c00640 1 -- 172.21.15.86:0/558455518 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67b409f4b0 con 0x7f67b40cfca0 2024-04-25T14:11:09.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.521+0000 7f67aaa00640 1 -- 172.21.15.86:0/558455518 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 60) ==== 93318+0+0 (secure 0 0 0) 0x7f67a001a070 con 0x7f67b40cfca0 2024-04-25T14:11:09.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.521+0000 7f67aaa00640 1 --2- 172.21.15.86:0/558455518 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f679406f570 0x7f6794071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.525+0000 7f67aaa00640 1 -- 172.21.15.86:0/558455518 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6488+0+0 (secure 0 0 0) 0x7f67a008f9e0 con 0x7f67b40cfca0 2024-04-25T14:11:09.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.525+0000 7f67b2a00640 1 --2- 172.21.15.86:0/558455518 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f679406f570 0x7f6794071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.525+0000 7f67b2a00640 1 --2- 172.21.15.86:0/558455518 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f679406f570 0x7f6794071a80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f67a40ae000 tx=0x7f67a4008040 comp rx=0 tx=0).ready entity=mgr.9198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:09.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.529+0000 7f67aaa00640 1 -- 172.21.15.86:0/558455518 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f67a005c540 con 0x7f67b40cfca0 2024-04-25T14:11:09.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.709+0000 7f67b9c00640 1 -- 172.21.15.86:0/558455518 --> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f67b4069d00 con 0x7f679406f570 2024-04-25T14:11:09.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.709+0000 7f67aaa00640 1 -- 172.21.15.86:0/558455518 <== mgr.9198 v2:172.21.15.86:6832/2221276863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f67b4069d00 con 0x7f679406f570 2024-04-25T14:11:09.708 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {}, "global_metrics": {}, "metrics": {"delayed_ranks": []}} 2024-04-25T14:11:09.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.713+0000 7f67b9c00640 1 -- 172.21.15.86:0/558455518 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f679406f570 msgr2=0x7f6794071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.713+0000 7f67b9c00640 1 --2- 172.21.15.86:0/558455518 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f679406f570 0x7f6794071a80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f67a40ae000 tx=0x7f67a4008040 comp rx=0 tx=0).stop 2024-04-25T14:11:09.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.713+0000 7f67b9c00640 1 -- 172.21.15.86:0/558455518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b40cfca0 msgr2=0x7f67b4159770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.713+0000 7f67b9c00640 1 --2- 172.21.15.86:0/558455518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b40cfca0 0x7f67b4159770 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f67a000e9b0 tx=0x7f67a000ee70 comp rx=0 tx=0).stop 2024-04-25T14:11:09.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.713+0000 7f67b9c00640 1 -- 172.21.15.86:0/558455518 shutdown_connections 2024-04-25T14:11:09.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.713+0000 7f67b9c00640 1 --2- 172.21.15.86:0/558455518 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f679406f570 0x7f6794071a80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.713+0000 7f67b9c00640 1 --2- 172.21.15.86:0/558455518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67b40d16e0 0x7f67b409cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.713+0000 7f67b9c00640 1 --2- 172.21.15.86:0/558455518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b40cfca0 0x7f67b4159770 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.713+0000 7f67b9c00640 1 --2- 172.21.15.86:0/558455518 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67b406c670 0x7f67b41590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.713+0000 7f67b9c00640 1 -- 172.21.15.86:0/558455518 >> 172.21.15.86:0/558455518 conn(0x7f67b40b4650 msgr2=0x7f67b40af450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:09.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.713+0000 7f67b9c00640 1 -- 172.21.15.86:0/558455518 shutdown_connections 2024-04-25T14:11:09.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:09.713+0000 7f67b9c00640 1 -- 172.21.15.86:0/558455518 wait complete. 2024-04-25T14:11:10.730 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:11:10.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.885+0000 7f3367a00640 1 Processor -- start 2024-04-25T14:11:10.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.885+0000 7f3367a00640 1 -- start start 2024-04-25T14:11:10.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.885+0000 7f3367a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f33601651d0 0x7f33601675f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:10.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.885+0000 7f3367a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33600b0a60 con 0x7f336015c2a0 2024-04-25T14:11:10.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.885+0000 7f3367a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33600ae840 con 0x7f336006fd50 2024-04-25T14:11:10.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.885+0000 7f3367a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f336006ef90 con 0x7f33601651d0 2024-04-25T14:11:10.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.885+0000 7f3365000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f336015c2a0 0x7f33600ae130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34778/0 (socket says 172.21.15.86:34778) 2024-04-25T14:11:10.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.885+0000 7f3365000640 1 -- 172.21.15.86:0/2298241356 learned_addr learned my addr 172.21.15.86:0/2298241356 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:10.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.889+0000 7f3365a00640 1 --2- 172.21.15.86:0/2298241356 >> v2:172.21.15.155:3300/0 conn(0x7f33601651d0 0x7f33601675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:10.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.889+0000 7f335fe00640 1 -- 172.21.15.86:0/2298241356 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4011638733 0 0) 0x7f33600b0a60 con 0x7f336015c2a0 2024-04-25T14:11:10.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.889+0000 7f335fe00640 1 -- 172.21.15.86:0/2298241356 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3344004640 con 0x7f336015c2a0 2024-04-25T14:11:10.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.889+0000 7f335fe00640 1 -- 172.21.15.86:0/2298241356 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3976425241 0 0) 0x7f3344004640 con 0x7f336015c2a0 2024-04-25T14:11:10.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.889+0000 7f335fe00640 1 -- 172.21.15.86:0/2298241356 >> v2:172.21.15.155:3300/0 conn(0x7f33601651d0 msgr2=0x7f33601675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:10.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.889+0000 7f335fe00640 1 --2- 172.21.15.86:0/2298241356 >> v2:172.21.15.155:3300/0 conn(0x7f33601651d0 0x7f33601675f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:10.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.889+0000 7f335fe00640 1 -- 172.21.15.86:0/2298241356 >> v1:172.21.15.155:6789/0 conn(0x7f336006fd50 legacy=0x7f3360070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:10.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.889+0000 7f335fe00640 1 -- 172.21.15.86:0/2298241356 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336006f270 con 0x7f336015c2a0 2024-04-25T14:11:10.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.889+0000 7f335fe00640 1 -- 172.21.15.86:0/2298241356 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f3350001680 con 0x7f336015c2a0 2024-04-25T14:11:10.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.889+0000 7f335fe00640 1 -- 172.21.15.86:0/2298241356 >> v1:172.21.15.86:6789/0 conn(0x7f336015c2a0 legacy=0x7f33600ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:10.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.889+0000 7f335fe00640 1 --2- 172.21.15.86:0/2298241356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3344004b40 0x7f3344004fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:10.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.889+0000 7f335fe00640 1 --2- 172.21.15.86:0/2298241356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3344005920 0x7f3344005d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:10.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.889+0000 7f335fe00640 1 --2- 172.21.15.86:0/2298241356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3344006470 0x7f334400cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:10.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.889+0000 7f335fe00640 1 -- 172.21.15.86:0/2298241356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f336006ef90 con 0x7f3344006470 2024-04-25T14:11:10.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.889+0000 7f335fe00640 1 -- 172.21.15.86:0/2298241356 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f334400d370 con 0x7f3344005920 2024-04-25T14:11:10.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.889+0000 7f335fe00640 1 -- 172.21.15.86:0/2298241356 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f334400d570 con 0x7f3344004b40 2024-04-25T14:11:10.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.889+0000 7f3365a00640 1 --2- 172.21.15.86:0/2298241356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3344006470 0x7f334400cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:10.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.889+0000 7f3365000640 1 --2- 172.21.15.86:0/2298241356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3344004b40 0x7f3344004fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:10.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.889+0000 7f3357e00640 1 --2- 172.21.15.86:0/2298241356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3344005920 0x7f3344005d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:10.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.889+0000 7f3365a00640 1 -- 172.21.15.86:0/2298241356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3344004b40 msgr2=0x7f3344004fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:10.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.889+0000 7f3365a00640 1 --2- 172.21.15.86:0/2298241356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3344004b40 0x7f3344004fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:10.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.889+0000 7f3365a00640 1 -- 172.21.15.86:0/2298241356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3344005920 msgr2=0x7f3344005d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:10.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.889+0000 7f3365a00640 1 --2- 172.21.15.86:0/2298241356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3344005920 0x7f3344005d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:10.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.889+0000 7f3365a00640 1 -- 172.21.15.86:0/2298241356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3344004640 con 0x7f3344006470 2024-04-25T14:11:10.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.889+0000 7f3365000640 1 --2- 172.21.15.86:0/2298241356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3344004b40 0x7f3344004fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:10.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.893+0000 7f3365a00640 1 -- 172.21.15.86:0/2298241356 reap_dead start 2024-04-25T14:11:10.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.893+0000 7f3365a00640 1 --2- 172.21.15.86:0/2298241356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3344006470 0x7f334400cae0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f3358009fc0 tx=0x7f335801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865c054aed513e0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:10.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.893+0000 7f335fe00640 1 -- 172.21.15.86:0/2298241356 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3358009c10 con 0x7f3344006470 2024-04-25T14:11:10.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.893+0000 7f335fe00640 1 -- 172.21.15.86:0/2298241356 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33580a5cd0 con 0x7f3344006470 2024-04-25T14:11:10.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.893+0000 7f335fe00640 1 -- 172.21.15.86:0/2298241356 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3358003be0 con 0x7f3344006470 2024-04-25T14:11:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.893+0000 7f3367a00640 1 -- 172.21.15.86:0/2298241356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3344006470 msgr2=0x7f334400cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.893+0000 7f3367a00640 1 --2- 172.21.15.86:0/2298241356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3344006470 0x7f334400cae0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f3358009fc0 tx=0x7f335801ccc0 comp rx=0 tx=0).stop 2024-04-25T14:11:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.893+0000 7f3367a00640 1 -- 172.21.15.86:0/2298241356 shutdown_connections 2024-04-25T14:11:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.893+0000 7f3367a00640 1 --2- 172.21.15.86:0/2298241356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3344006470 0x7f334400cae0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.893+0000 7f3367a00640 1 -- 172.21.15.86:0/2298241356 >> 172.21.15.86:0/2298241356 conn(0x7f33600b4650 msgr2=0x7f33600b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.893+0000 7f3367a00640 1 -- 172.21.15.86:0/2298241356 shutdown_connections 2024-04-25T14:11:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.893+0000 7f3367a00640 1 -- 172.21.15.86:0/2298241356 wait complete. 2024-04-25T14:11:10.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.893+0000 7f3367a00640 1 Processor -- start 2024-04-25T14:11:10.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.897+0000 7f3367a00640 1 -- start start 2024-04-25T14:11:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.897+0000 7f3367a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3344006470 0x7f33601f25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.897+0000 7f3367a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33601f2ae0 0x7f3360156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.897+0000 7f3367a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33601566d0 0x7f33601f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.897+0000 7f3367a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33600d0fc0 con 0x7f3344006470 2024-04-25T14:11:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.897+0000 7f3367a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f336006f830 con 0x7f33601f2ae0 2024-04-25T14:11:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.897+0000 7f3367a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f336015c6b0 con 0x7f33601566d0 2024-04-25T14:11:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.897+0000 7f3365000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3344006470 0x7f33601f25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.897+0000 7f3365000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3344006470 0x7f33601f25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59602/0 (socket says 172.21.15.86:59602) 2024-04-25T14:11:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.897+0000 7f3365000640 1 -- 172.21.15.86:0/1655966351 learned_addr learned my addr 172.21.15.86:0/1655966351 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.897+0000 7f3365000640 1 -- 172.21.15.86:0/1655966351 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33601566d0 msgr2=0x7f33601f43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.897+0000 7f3365000640 1 --2- 172.21.15.86:0/1655966351 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33601566d0 0x7f33601f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.897+0000 7f3365000640 1 -- 172.21.15.86:0/1655966351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33601f2ae0 msgr2=0x7f3360156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.897+0000 7f3365000640 1 --2- 172.21.15.86:0/1655966351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33601f2ae0 0x7f3360156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.897+0000 7f3365000640 1 -- 172.21.15.86:0/1655966351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33601f3320 con 0x7f3344006470 2024-04-25T14:11:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.897+0000 7f3365000640 1 --2- 172.21.15.86:0/1655966351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3344006470 0x7f33601f25a0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f3350001b90 tx=0x7f33500119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.897+0000 7f335d600640 1 -- 172.21.15.86:0/1655966351 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3350010040 con 0x7f3344006470 2024-04-25T14:11:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.897+0000 7f335d600640 1 -- 172.21.15.86:0/1655966351 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f335001a070 con 0x7f3344006470 2024-04-25T14:11:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.897+0000 7f3367a00640 1 -- 172.21.15.86:0/1655966351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33601f3600 con 0x7f3344006470 2024-04-25T14:11:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.897+0000 7f335d600640 1 -- 172.21.15.86:0/1655966351 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33500139a0 con 0x7f3344006470 2024-04-25T14:11:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.897+0000 7f3367a00640 1 -- 172.21.15.86:0/1655966351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33601f3bc0 con 0x7f3344006470 2024-04-25T14:11:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.897+0000 7f3367a00640 1 -- 172.21.15.86:0/1655966351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3360069d00 con 0x7f3344006470 2024-04-25T14:11:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.901+0000 7f335d600640 1 -- 172.21.15.86:0/1655966351 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 60) ==== 93318+0+0 (secure 0 0 0) 0x7f3350013b40 con 0x7f3344006470 2024-04-25T14:11:10.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.901+0000 7f335d600640 1 --2- 172.21.15.86:0/1655966351 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f333c06f570 0x7f333c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:10.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.901+0000 7f335d600640 1 -- 172.21.15.86:0/1655966351 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6488+0+0 (secure 0 0 0) 0x7f3350092730 con 0x7f3344006470 2024-04-25T14:11:10.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.901+0000 7f3357e00640 1 --2- 172.21.15.86:0/1655966351 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f333c06f570 0x7f333c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:10.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.901+0000 7f3357e00640 1 --2- 172.21.15.86:0/1655966351 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f333c06f570 0x7f333c071a80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3348000c10 tx=0x7f3348009040 comp rx=0 tx=0).ready entity=mgr.9198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:10.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:10.905+0000 7f335d600640 1 -- 172.21.15.86:0/1655966351 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f335005f200 con 0x7f3344006470 2024-04-25T14:11:11.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.085+0000 7f3367a00640 1 -- 172.21.15.86:0/1655966351 --> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f3360076bf0 con 0x7f333c06f570 2024-04-25T14:11:11.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.085+0000 7f335d600640 1 -- 172.21.15.86:0/1655966351 <== mgr.9198 v2:172.21.15.86:6832/2221276863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+2957 (secure 0 0 0) 0x7f3360076bf0 con 0x7f333c06f570 2024-04-25T14:11:11.082 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {"cephfs": {"client.7715": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}, "client.8770": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:172.21.15.86"}, "client.8779": {"hostname": "smithi155", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:192.168.0.1"}, "client.9231": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}}}, "global_metrics": {"cephfs": {"client.7715": [[2, 0], [0, 0], [0, 0], [0, 474599], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 474599], [0, 1]], "client.8770": [[3, 19], [0, 0], [0, 0], [0, 16541833], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 689247], [13464462045498, 24]], "client.8779": [[22, 0], [0, 0], [0, 0], [0, 4738381], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 1184596], [8139524375000, 4]], "client.9231": [[2, 0], [0, 0], [0, 0], [0, 760975], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 760975], [0, 1]]}}, "metrics": {"delayed_ranks": [], "mds.0": {"client.7715": [], "client.8770": [], "client.8779": [], "client.9231": []}}} 2024-04-25T14:11:11.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.085+0000 7f3367a00640 1 -- 172.21.15.86:0/1655966351 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f333c06f570 msgr2=0x7f333c071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.085+0000 7f3367a00640 1 --2- 172.21.15.86:0/1655966351 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f333c06f570 0x7f333c071a80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3348000c10 tx=0x7f3348009040 comp rx=0 tx=0).stop 2024-04-25T14:11:11.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.085+0000 7f3367a00640 1 -- 172.21.15.86:0/1655966351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3344006470 msgr2=0x7f33601f25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.085+0000 7f3367a00640 1 --2- 172.21.15.86:0/1655966351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3344006470 0x7f33601f25a0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f3350001b90 tx=0x7f33500119f0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.085+0000 7f3367a00640 1 -- 172.21.15.86:0/1655966351 shutdown_connections 2024-04-25T14:11:11.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.085+0000 7f3367a00640 1 --2- 172.21.15.86:0/1655966351 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f333c06f570 0x7f333c071a80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.085+0000 7f3367a00640 1 --2- 172.21.15.86:0/1655966351 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33601566d0 0x7f33601f43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.085+0000 7f3367a00640 1 --2- 172.21.15.86:0/1655966351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33601f2ae0 0x7f3360156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.085+0000 7f3367a00640 1 --2- 172.21.15.86:0/1655966351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3344006470 0x7f33601f25a0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.085+0000 7f3367a00640 1 -- 172.21.15.86:0/1655966351 >> 172.21.15.86:0/1655966351 conn(0x7f33600b4650 msgr2=0x7f33601591a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:11.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.085+0000 7f3367a00640 1 -- 172.21.15.86:0/1655966351 shutdown_connections 2024-04-25T14:11:11.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.089+0000 7f3367a00640 1 -- 172.21.15.86:0/1655966351 wait complete. 2024-04-25T14:11:11.103 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:11:11.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.253+0000 7fd3b7800640 1 Processor -- start 2024-04-25T14:11:11.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.253+0000 7fd3b7800640 1 -- start start 2024-04-25T14:11:11.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3b7800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd3b00740c0 0x7fd3b01676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3b7800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3b0072bd0 con 0x7fd3b00cfca0 2024-04-25T14:11:11.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3b7800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3b006fbd0 con 0x7fd3b00d16e0 2024-04-25T14:11:11.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3b7800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd3b0070470 con 0x7fd3b00740c0 2024-04-25T14:11:11.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3b4e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd3b00cfca0 0x7fd3b006f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34794/0 (socket says 172.21.15.86:34794) 2024-04-25T14:11:11.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3b4e00640 1 -- 172.21.15.86:0/1933776197 learned_addr learned my addr 172.21.15.86:0/1933776197 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:11.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3b5800640 1 --2- 172.21.15.86:0/1933776197 >> v2:172.21.15.155:3300/0 conn(0x7fd3b00740c0 0x7fd3b01676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3af400640 1 -- 172.21.15.86:0/1933776197 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1306841286 0 0) 0x7fd3b0072bd0 con 0x7fd3b00cfca0 2024-04-25T14:11:11.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3af400640 1 -- 172.21.15.86:0/1933776197 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd394003530 con 0x7fd3b00cfca0 2024-04-25T14:11:11.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3af400640 1 -- 172.21.15.86:0/1933776197 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1460660496 0 0) 0x7fd394003530 con 0x7fd3b00cfca0 2024-04-25T14:11:11.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3af400640 1 -- 172.21.15.86:0/1933776197 >> v2:172.21.15.155:3300/0 conn(0x7fd3b00740c0 msgr2=0x7fd3b01676d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:11.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3af400640 1 --2- 172.21.15.86:0/1933776197 >> v2:172.21.15.155:3300/0 conn(0x7fd3b00740c0 0x7fd3b01676d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3af400640 1 -- 172.21.15.86:0/1933776197 >> v1:172.21.15.155:6789/0 conn(0x7fd3b00d16e0 legacy=0x7fd3b00d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3af400640 1 -- 172.21.15.86:0/1933776197 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b006feb0 con 0x7fd3b00cfca0 2024-04-25T14:11:11.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3af400640 1 -- 172.21.15.86:0/1933776197 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fd3a00016a0 con 0x7fd3b00cfca0 2024-04-25T14:11:11.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3b5800640 1 --2- 172.21.15.86:0/1933776197 >> v2:172.21.15.155:3300/0 conn(0x7fd3b00740c0 0x7fd3b01676d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:11:11.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3af400640 1 -- 172.21.15.86:0/1933776197 >> v1:172.21.15.86:6789/0 conn(0x7fd3b00cfca0 legacy=0x7fd3b006f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3af400640 1 --2- 172.21.15.86:0/1933776197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd394003a30 0x7fd394003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3af400640 1 --2- 172.21.15.86:0/1933776197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd394004810 0x7fd394004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3af400640 1 --2- 172.21.15.86:0/1933776197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd394005230 0x7fd39400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3af400640 1 -- 172.21.15.86:0/1933776197 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b0070470 con 0x7fd394005230 2024-04-25T14:11:11.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3af400640 1 -- 172.21.15.86:0/1933776197 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd39400c1c0 con 0x7fd394004810 2024-04-25T14:11:11.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3af400640 1 -- 172.21.15.86:0/1933776197 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd39400c3c0 con 0x7fd394003a30 2024-04-25T14:11:11.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3b5800640 1 --2- 172.21.15.86:0/1933776197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd394005230 0x7fd39400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3afe00640 1 --2- 172.21.15.86:0/1933776197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd394004810 0x7fd394004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3b4e00640 1 --2- 172.21.15.86:0/1933776197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd394003a30 0x7fd394003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3b5800640 1 -- 172.21.15.86:0/1933776197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd394003a30 msgr2=0x7fd394003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:11.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3b5800640 1 --2- 172.21.15.86:0/1933776197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd394003a30 0x7fd394003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3b5800640 1 -- 172.21.15.86:0/1933776197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd394004810 msgr2=0x7fd394004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:11.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3b5800640 1 --2- 172.21.15.86:0/1933776197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd394004810 0x7fd394004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.257+0000 7fd3b5800640 1 -- 172.21.15.86:0/1933776197 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd394003530 con 0x7fd394005230 2024-04-25T14:11:11.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.261+0000 7fd3b5800640 1 -- 172.21.15.86:0/1933776197 reap_dead start 2024-04-25T14:11:11.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.261+0000 7fd3b5800640 1 --2- 172.21.15.86:0/1933776197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd394005230 0x7fd39400b9d0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fd3a800e320 tx=0x7fd3a801ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516dc6a43662b187 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:11.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.261+0000 7fd3af400640 1 -- 172.21.15.86:0/1933776197 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8009de0 con 0x7fd394005230 2024-04-25T14:11:11.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.261+0000 7fd3af400640 1 -- 172.21.15.86:0/1933776197 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3a80a5e50 con 0x7fd394005230 2024-04-25T14:11:11.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.261+0000 7fd3af400640 1 -- 172.21.15.86:0/1933776197 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8003be0 con 0x7fd394005230 2024-04-25T14:11:11.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.261+0000 7fd3b7800640 1 -- 172.21.15.86:0/1933776197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd394005230 msgr2=0x7fd39400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.261+0000 7fd3b7800640 1 --2- 172.21.15.86:0/1933776197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd394005230 0x7fd39400b9d0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fd3a800e320 tx=0x7fd3a801ce40 comp rx=0 tx=0).stop 2024-04-25T14:11:11.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.261+0000 7fd3b7800640 1 -- 172.21.15.86:0/1933776197 shutdown_connections 2024-04-25T14:11:11.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.261+0000 7fd3b7800640 1 --2- 172.21.15.86:0/1933776197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd394005230 0x7fd39400b9d0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.261+0000 7fd3b7800640 1 -- 172.21.15.86:0/1933776197 >> 172.21.15.86:0/1933776197 conn(0x7fd3b00b4650 msgr2=0x7fd3b00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:11.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.261+0000 7fd3b7800640 1 -- 172.21.15.86:0/1933776197 shutdown_connections 2024-04-25T14:11:11.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.261+0000 7fd3b7800640 1 -- 172.21.15.86:0/1933776197 wait complete. 2024-04-25T14:11:11.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.265+0000 7fd3b7800640 1 Processor -- start 2024-04-25T14:11:11.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.265+0000 7fd3b7800640 1 -- start start 2024-04-25T14:11:11.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.265+0000 7fd3b7800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd394005230 0x7fd3b0158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.265+0000 7fd3b7800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3b01e7d40 0x7fd3b01f16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.265+0000 7fd3b7800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3b01f7740 0x7fd3b01f7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.265+0000 7fd3b7800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b006f570 con 0x7fd394005230 2024-04-25T14:11:11.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.265+0000 7fd3b7800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b006f3f0 con 0x7fd3b01e7d40 2024-04-25T14:11:11.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.265+0000 7fd3b7800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd3b006f6f0 con 0x7fd3b01f7740 2024-04-25T14:11:11.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.265+0000 7fd3b4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd394005230 0x7fd3b0158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.265+0000 7fd3b4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd394005230 0x7fd3b0158970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59616/0 (socket says 172.21.15.86:59616) 2024-04-25T14:11:11.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.265+0000 7fd3b4e00640 1 -- 172.21.15.86:0/114227274 learned_addr learned my addr 172.21.15.86:0/114227274 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:11.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.265+0000 7fd3b5800640 1 --2- 172.21.15.86:0/114227274 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3b01f7740 0x7fd3b01f7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.265+0000 7fd3afe00640 1 --2- 172.21.15.86:0/114227274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3b01e7d40 0x7fd3b01f16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.265+0000 7fd3b4e00640 1 -- 172.21.15.86:0/114227274 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3b01f7740 msgr2=0x7fd3b01f7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.265+0000 7fd3b4e00640 1 --2- 172.21.15.86:0/114227274 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3b01f7740 0x7fd3b01f7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.265+0000 7fd3b4e00640 1 -- 172.21.15.86:0/114227274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3b01e7d40 msgr2=0x7fd3b01f16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.265+0000 7fd3b4e00640 1 --2- 172.21.15.86:0/114227274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3b01e7d40 0x7fd3b01f16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.265+0000 7fd3b4e00640 1 -- 172.21.15.86:0/114227274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b01f8090 con 0x7fd394005230 2024-04-25T14:11:11.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.265+0000 7fd3b5800640 1 --2- 172.21.15.86:0/114227274 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3b01f7740 0x7fd3b01f7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:11.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.265+0000 7fd3afe00640 1 --2- 172.21.15.86:0/114227274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3b01e7d40 0x7fd3b01f16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:11.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.265+0000 7fd3b4e00640 1 --2- 172.21.15.86:0/114227274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd394005230 0x7fd3b0158970 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fd3a000e7b0 tx=0x7fd3a000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:11.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.265+0000 7fd3acc00640 1 -- 172.21.15.86:0/114227274 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a000ee60 con 0x7fd394005230 2024-04-25T14:11:11.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.265+0000 7fd3acc00640 1 -- 172.21.15.86:0/114227274 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3a0018070 con 0x7fd394005230 2024-04-25T14:11:11.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.265+0000 7fd3acc00640 1 -- 172.21.15.86:0/114227274 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a000cb10 con 0x7fd394005230 2024-04-25T14:11:11.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.265+0000 7fd3b7800640 1 -- 172.21.15.86:0/114227274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3b015bcc0 con 0x7fd394005230 2024-04-25T14:11:11.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.265+0000 7fd3b7800640 1 -- 172.21.15.86:0/114227274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3b015b390 con 0x7fd394005230 2024-04-25T14:11:11.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.269+0000 7fd3b7800640 1 -- 172.21.15.86:0/114227274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3b009f4b0 con 0x7fd394005230 2024-04-25T14:11:11.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.269+0000 7fd3acc00640 1 -- 172.21.15.86:0/114227274 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 60) ==== 93318+0+0 (secure 0 0 0) 0x7fd3a000ccb0 con 0x7fd394005230 2024-04-25T14:11:11.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.269+0000 7fd3acc00640 1 --2- 172.21.15.86:0/114227274 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fd39006f570 0x7fd390071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.269+0000 7fd3acc00640 1 -- 172.21.15.86:0/114227274 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6488+0+0 (secure 0 0 0) 0x7fd3a008f5c0 con 0x7fd394005230 2024-04-25T14:11:11.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.269+0000 7fd3afe00640 1 --2- 172.21.15.86:0/114227274 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fd39006f570 0x7fd390071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.269+0000 7fd3afe00640 1 --2- 172.21.15.86:0/114227274 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fd39006f570 0x7fd390071a80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd3b01e7670 tx=0x7fd39c009040 comp rx=0 tx=0).ready entity=mgr.9198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:11.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.273+0000 7fd3acc00640 1 -- 172.21.15.86:0/114227274 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd3b01f8090 con 0x7fd394005230 2024-04-25T14:11:11.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.465+0000 7fd3b7800640 1 -- 172.21.15.86:0/114227274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3b0069d00 con 0x7fd394005230 2024-04-25T14:11:11.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.465+0000 7fd3acc00640 1 -- 172.21.15.86:0/114227274 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5186 (secure 0 0 0) 0x7fd3a005c090 con 0x7fd394005230 2024-04-25T14:11:11.463 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:11.463 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":94,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2258481468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2258481468},{"type":"v1","addr":"172.21.15.155:6834","nonce":2258481468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":8976,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1064843151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1064843151},{"type":"v1","addr":"172.21.15.86:6835","nonce":1064843151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":8985,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1940755625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1940755625},{"type":"v1","addr":"172.21.15.86:6837","nonce":1940755625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:10:45.497688+0000","modified":"2024-04-25T14:10:48.475869+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8991":{"gid":8991,"name":"b","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6835/544985680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":544985680},{"type":"v1","addr":"172.21.15.155:6835","nonce":544985680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8991,"qdb_cluster":[8991]},"id":9}]} 2024-04-25T14:11:11.463 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 94 2024-04-25T14:11:11.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.469+0000 7fd3b7800640 1 -- 172.21.15.86:0/114227274 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fd39006f570 msgr2=0x7fd390071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.469+0000 7fd3b7800640 1 --2- 172.21.15.86:0/114227274 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fd39006f570 0x7fd390071a80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd3b01e7670 tx=0x7fd39c009040 comp rx=0 tx=0).stop 2024-04-25T14:11:11.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.469+0000 7fd3b7800640 1 -- 172.21.15.86:0/114227274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd394005230 msgr2=0x7fd3b0158970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.469+0000 7fd3b7800640 1 --2- 172.21.15.86:0/114227274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd394005230 0x7fd3b0158970 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fd3a000e7b0 tx=0x7fd3a000ec70 comp rx=0 tx=0).stop 2024-04-25T14:11:11.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.469+0000 7fd3b7800640 1 -- 172.21.15.86:0/114227274 shutdown_connections 2024-04-25T14:11:11.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.469+0000 7fd3b7800640 1 --2- 172.21.15.86:0/114227274 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fd39006f570 0x7fd390071a80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.469+0000 7fd3b7800640 1 --2- 172.21.15.86:0/114227274 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3b01f7740 0x7fd3b01f7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.469+0000 7fd3b7800640 1 --2- 172.21.15.86:0/114227274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3b01e7d40 0x7fd3b01f16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.469+0000 7fd3b7800640 1 --2- 172.21.15.86:0/114227274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd394005230 0x7fd3b0158970 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.469+0000 7fd3b7800640 1 -- 172.21.15.86:0/114227274 >> 172.21.15.86:0/114227274 conn(0x7fd3b00b4650 msgr2=0x7fd3b0072fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:11.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.469+0000 7fd3b7800640 1 -- 172.21.15.86:0/114227274 shutdown_connections 2024-04-25T14:11:11.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.469+0000 7fd3b7800640 1 -- 172.21.15.86:0/114227274 wait complete. 2024-04-25T14:11:11.484 DEBUG:tasks.cephfs.test_mds_metrics:metrics={'version': 2, 'global_counters': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'counters': [], 'client_metadata': {'cephfs': {'client.7715': {'hostname': 'smithi086', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'IP': '172.21.15.86'}, 'client.8770': {'hostname': 'smithi086', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'kernel_version': '6.9.0-rc3-g806ec36b2112', 'IP': 'v1:172.21.15.86'}, 'client.8779': {'hostname': 'smithi155', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'kernel_version': '6.9.0-rc3-g806ec36b2112', 'IP': 'v1:192.168.0.1'}, 'client.9231': {'hostname': 'smithi086', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'IP': '172.21.15.86'}}}, 'global_metrics': {'cephfs': {'client.7715': [[2, 0], [0, 0], [0, 0], [0, 474599], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 474599], [0, 1]], 'client.8770': [[3, 19], [0, 0], [0, 0], [0, 16541833], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 689247], [13464462045498, 24]], 'client.8779': [[22, 0], [0, 0], [0, 0], [0, 4738381], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 1184596], [8139524375000, 4]], 'client.9231': [[2, 0], [0, 0], [0, 0], [0, 760975], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 760975], [0, 1]]}}, 'metrics': {'delayed_ranks': [], 'mds.0': {'client.7715': [], 'client.8770': [], 'client.8779': [], 'client.9231': []}}} 2024-04-25T14:11:11.485 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable stats 2024-04-25T14:11:11.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.637+0000 7f1edd200640 1 Processor -- start 2024-04-25T14:11:11.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.637+0000 7f1edd200640 1 -- start start 2024-04-25T14:11:11.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.641+0000 7f1edd200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ed806fd50 0x7f1ed8070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.641+0000 7f1edd200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1ed815feb0 0x7f1ed81602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.641+0000 7f1edd200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ed80b0a60 con 0x7f1ed815c2a0 2024-04-25T14:11:11.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.641+0000 7f1edd200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1ed80ae840 con 0x7f1ed815feb0 2024-04-25T14:11:11.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.641+0000 7f1edd200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed806ef90 con 0x7f1ed806fd50 2024-04-25T14:11:11.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.641+0000 7f1ecd400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ed806fd50 0x7f1ed8070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.641+0000 7f1ed6800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1ed815feb0 0x7f1ed81602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.641+0000 7f1ed5e00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f1ed815c2a0 0x7f1ed80ae130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:49044/0 (socket says 172.21.15.86:49044) 2024-04-25T14:11:11.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.641+0000 7f1ed5e00640 1 -- 172.21.15.86:0/139039718 learned_addr learned my addr 172.21.15.86:0/139039718 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:11.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.641+0000 7f1ed5400640 1 -- 172.21.15.86:0/139039718 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1209277028 0 0) 0x7f1ed80b0a60 con 0x7f1ed815c2a0 2024-04-25T14:11:11.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.641+0000 7f1ed5400640 1 -- 172.21.15.86:0/139039718 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ebc003530 con 0x7f1ed815c2a0 2024-04-25T14:11:11.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.641+0000 7f1ed6800640 1 -- 172.21.15.86:0/139039718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ed806fd50 msgr2=0x7f1ed8070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:11.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.641+0000 7f1ed6800640 1 --2- 172.21.15.86:0/139039718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ed806fd50 0x7f1ed8070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.641+0000 7f1ed6800640 1 -- 172.21.15.86:0/139039718 >> v1:172.21.15.155:6789/0 conn(0x7f1ed815c2a0 legacy=0x7f1ed80ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.641+0000 7f1ed6800640 1 -- 172.21.15.86:0/139039718 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed806f270 con 0x7f1ed815feb0 2024-04-25T14:11:11.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.641+0000 7f1ecd400640 1 --2- 172.21.15.86:0/139039718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ed806fd50 0x7f1ed8070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:11:11.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.645+0000 7f1ed6800640 1 --2- 172.21.15.86:0/139039718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ed815feb0 0x7f1ed81602c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1ed00097c0 tx=0x7f1ed001c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5d1eda53b4dce28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:11.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.645+0000 7f1ed5400640 1 -- 172.21.15.86:0/139039718 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed00b2070 con 0x7f1ed815feb0 2024-04-25T14:11:11.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.645+0000 7f1ed5400640 1 -- 172.21.15.86:0/139039718 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ed00a9070 con 0x7f1ed815feb0 2024-04-25T14:11:11.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.645+0000 7f1ed5400640 1 -- 172.21.15.86:0/139039718 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed00ad860 con 0x7f1ed815feb0 2024-04-25T14:11:11.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.645+0000 7f1edd200640 1 -- 172.21.15.86:0/139039718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ed815feb0 msgr2=0x7f1ed81602c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.645+0000 7f1edd200640 1 --2- 172.21.15.86:0/139039718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ed815feb0 0x7f1ed81602c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1ed00097c0 tx=0x7f1ed001c3e0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.645+0000 7f1edd200640 1 -- 172.21.15.86:0/139039718 shutdown_connections 2024-04-25T14:11:11.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.645+0000 7f1edd200640 1 --2- 172.21.15.86:0/139039718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ed815feb0 0x7f1ed81602c0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.645+0000 7f1edd200640 1 --2- 172.21.15.86:0/139039718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ed806fd50 0x7f1ed8070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.645+0000 7f1edd200640 1 -- 172.21.15.86:0/139039718 >> 172.21.15.86:0/139039718 conn(0x7f1ed80b4650 msgr2=0x7f1ed80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:11.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.645+0000 7f1edd200640 1 -- 172.21.15.86:0/139039718 shutdown_connections 2024-04-25T14:11:11.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.645+0000 7f1edd200640 1 -- 172.21.15.86:0/139039718 wait complete. 2024-04-25T14:11:11.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.645+0000 7f1edd200640 1 Processor -- start 2024-04-25T14:11:11.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.645+0000 7f1edd200640 1 -- start start 2024-04-25T14:11:11.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.649+0000 7f1edd200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ed806fd50 0x7f1ed81679a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.649+0000 7f1edd200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ed815c2a0 0x7f1ed8167ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.649+0000 7f1edd200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ed815feb0 0x7f1ed815ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.649+0000 7f1edd200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed815c6b0 con 0x7f1ed815feb0 2024-04-25T14:11:11.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.649+0000 7f1edd200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed80d0fc0 con 0x7f1ed815c2a0 2024-04-25T14:11:11.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.649+0000 7f1edd200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed815c830 con 0x7f1ed806fd50 2024-04-25T14:11:11.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.649+0000 7f1ed6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ed815feb0 0x7f1ed815ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.649+0000 7f1ed6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ed815feb0 0x7f1ed815ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59628/0 (socket says 172.21.15.86:59628) 2024-04-25T14:11:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.649+0000 7f1ed6800640 1 -- 172.21.15.86:0/775764176 learned_addr learned my addr 172.21.15.86:0/775764176 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.649+0000 7f1ed5e00640 1 --2- 172.21.15.86:0/775764176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ed806fd50 0x7f1ed81679a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.649+0000 7f1ecd400640 1 --2- 172.21.15.86:0/775764176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ed815c2a0 0x7f1ed8167ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.649+0000 7f1ed6800640 1 -- 172.21.15.86:0/775764176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ed806fd50 msgr2=0x7f1ed81679a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.649+0000 7f1ed6800640 1 --2- 172.21.15.86:0/775764176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ed806fd50 0x7f1ed81679a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.649+0000 7f1ed6800640 1 -- 172.21.15.86:0/775764176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ed815c2a0 msgr2=0x7f1ed8167ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.649+0000 7f1ed6800640 1 --2- 172.21.15.86:0/775764176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ed815c2a0 0x7f1ed8167ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.649+0000 7f1ed6800640 1 -- 172.21.15.86:0/775764176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed8168670 con 0x7f1ed815feb0 2024-04-25T14:11:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.649+0000 7f1ed6800640 1 --2- 172.21.15.86:0/775764176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ed815feb0 0x7f1ed815ab40 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f1ed001aa90 tx=0x7f1ed0019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.649+0000 7f1ecea00640 1 -- 172.21.15.86:0/775764176 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed00b2040 con 0x7f1ed815feb0 2024-04-25T14:11:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.649+0000 7f1ecea00640 1 -- 172.21.15.86:0/775764176 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ed00bd070 con 0x7f1ed815feb0 2024-04-25T14:11:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.649+0000 7f1ecea00640 1 -- 172.21.15.86:0/775764176 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0003a20 con 0x7f1ed815feb0 2024-04-25T14:11:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.649+0000 7f1edd200640 1 -- 172.21.15.86:0/775764176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed8168950 con 0x7f1ed815feb0 2024-04-25T14:11:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.649+0000 7f1edd200640 1 -- 172.21.15.86:0/775764176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed815a620 con 0x7f1ed815feb0 2024-04-25T14:11:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.649+0000 7f1edd200640 1 -- 172.21.15.86:0/775764176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ed81660f0 con 0x7f1ed815feb0 2024-04-25T14:11:11.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.653+0000 7f1ecea00640 1 -- 172.21.15.86:0/775764176 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 60) ==== 93318+0+0 (secure 0 0 0) 0x7f1ed00b5070 con 0x7f1ed815feb0 2024-04-25T14:11:11.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.653+0000 7f1ecea00640 1 --2- 172.21.15.86:0/775764176 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f1eb406f5a0 0x7f1eb4071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.653+0000 7f1ecea00640 1 -- 172.21.15.86:0/775764176 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6488+0+0 (secure 0 0 0) 0x7f1ed0129510 con 0x7f1ed815feb0 2024-04-25T14:11:11.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.653+0000 7f1ed5e00640 1 --2- 172.21.15.86:0/775764176 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f1eb406f5a0 0x7f1eb4071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.653+0000 7f1ed5e00640 1 --2- 172.21.15.86:0/775764176 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f1eb406f5a0 0x7f1eb4071ab0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8005c10 tx=0x7f1ec8009040 comp rx=0 tx=0).ready entity=mgr.9198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:11.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.657+0000 7f1ecea00640 1 -- 172.21.15.86:0/775764176 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1ed0016090 con 0x7f1ed815feb0 2024-04-25T14:11:11.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:11.853+0000 7f1edd200640 1 -- 172.21.15.86:0/775764176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "stats"} v 0) -- 0x7f1ed8069d00 con 0x7f1ed815feb0 2024-04-25T14:11:12.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:12.617+0000 7f1ecea00640 1 -- 172.21.15.86:0/775764176 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "stats"}]=0 v61) ==== 85+0+0 (secure 0 0 0) 0x7f1ed00f57e0 con 0x7f1ed815feb0 2024-04-25T14:11:12.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:12.621+0000 7f1edd200640 1 -- 172.21.15.86:0/775764176 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f1eb406f5a0 msgr2=0x7f1eb4071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:12.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:12.621+0000 7f1edd200640 1 --2- 172.21.15.86:0/775764176 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f1eb406f5a0 0x7f1eb4071ab0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8005c10 tx=0x7f1ec8009040 comp rx=0 tx=0).stop 2024-04-25T14:11:12.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:12.621+0000 7f1edd200640 1 -- 172.21.15.86:0/775764176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ed815feb0 msgr2=0x7f1ed815ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:12.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:12.621+0000 7f1edd200640 1 --2- 172.21.15.86:0/775764176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ed815feb0 0x7f1ed815ab40 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f1ed001aa90 tx=0x7f1ed0019180 comp rx=0 tx=0).stop 2024-04-25T14:11:12.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:12.621+0000 7f1edd200640 1 -- 172.21.15.86:0/775764176 shutdown_connections 2024-04-25T14:11:12.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:12.621+0000 7f1edd200640 1 --2- 172.21.15.86:0/775764176 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f1eb406f5a0 0x7f1eb4071ab0 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8005c10 tx=0x7f1ec8009040 comp rx=0 tx=0).stop 2024-04-25T14:11:12.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:12.621+0000 7f1edd200640 1 --2- 172.21.15.86:0/775764176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ed815feb0 0x7f1ed815ab40 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:12.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:12.621+0000 7f1edd200640 1 --2- 172.21.15.86:0/775764176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ed815c2a0 0x7f1ed8167ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:12.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:12.621+0000 7f1edd200640 1 --2- 172.21.15.86:0/775764176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ed806fd50 0x7f1ed81679a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:12.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:12.621+0000 7f1edd200640 1 -- 172.21.15.86:0/775764176 >> 172.21.15.86:0/775764176 conn(0x7f1ed80b4650 msgr2=0x7f1ed8158d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:12.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:12.621+0000 7f1edd200640 1 -- 172.21.15.86:0/775764176 shutdown_connections 2024-04-25T14:11:12.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:12.621+0000 7f1edd200640 1 -- 172.21.15.86:0/775764176 wait complete. 2024-04-25T14:11:12.645 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T14:11:12.645 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:11:12.763 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:11:12.855 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T14:11:12.855 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:11:12.855 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:11:12.855 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:11:12.864 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:11:12.864 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:11:12.874 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:12.874+0000 7fdf0cf7e180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:11:12.883 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:12.885+0000 7f8b38049180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:11:12.891 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:11:13.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.081+0000 7fef46600640 1 Processor -- start 2024-04-25T14:11:13.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.081+0000 7fef46600640 1 -- start start 2024-04-25T14:11:13.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.085+0000 7fef46600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fef400cdd70 0x7fef400ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:13.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.085+0000 7fef46600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fef40070e20 0x7fef40071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:13.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.085+0000 7fef46600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef40071770 con 0x7fef4006e700 2024-04-25T14:11:13.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.085+0000 7fef46600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fef4006d940 con 0x7fef400cdd70 2024-04-25T14:11:13.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.085+0000 7fef46600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fef4006dc20 con 0x7fef40070e20 2024-04-25T14:11:13.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.085+0000 7fef3f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fef400cdd70 0x7fef400ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:13.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.085+0000 7fef3fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fef4006e700 0x7fef4006eb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34822/0 (socket says 172.21.15.86:34822) 2024-04-25T14:11:13.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.085+0000 7fef3fe00640 1 -- 172.21.15.86:0/771882846 learned_addr learned my addr 172.21.15.86:0/771882846 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:13.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.085+0000 7fef3f400640 1 --2- 172.21.15.86:0/771882846 >> v2:172.21.15.86:3300/0 conn(0x7fef400cdd70 0x7fef400ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59648/0 (socket says 172.21.15.86:59648) 2024-04-25T14:11:13.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.085+0000 7fef36a00640 1 --2- 172.21.15.86:0/771882846 >> v2:172.21.15.155:3300/0 conn(0x7fef40070e20 0x7fef40071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:13.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.085+0000 7fef3f400640 1 -- 172.21.15.86:0/771882846 >> v2:172.21.15.155:3300/0 conn(0x7fef40070e20 msgr2=0x7fef40071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:13.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.085+0000 7fef3f400640 1 --2- 172.21.15.86:0/771882846 >> v2:172.21.15.155:3300/0 conn(0x7fef40070e20 0x7fef40071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.085+0000 7fef3f400640 1 -- 172.21.15.86:0/771882846 >> v1:172.21.15.86:6789/0 conn(0x7fef4006e700 legacy=0x7fef4006eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:13.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.085+0000 7fef3ea00640 1 -- 172.21.15.86:0/771882846 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1464414063 0 0) 0x7fef380014a0 con 0x7fef4006e700 2024-04-25T14:11:13.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.085+0000 7fef3f400640 1 -- 172.21.15.86:0/771882846 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4006df00 con 0x7fef400cdd70 2024-04-25T14:11:13.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.089+0000 7fef3f400640 1 --2- 172.21.15.86:0/771882846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef400cdd70 0x7fef400ce180 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fef40073230 tx=0x7fef3001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a761fd0b882583 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:13.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.089+0000 7fef3ea00640 1 -- 172.21.15.86:0/771882846 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef300b0070 con 0x7fef400cdd70 2024-04-25T14:11:13.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.089+0000 7fef3ea00640 1 -- 172.21.15.86:0/771882846 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef300ab600 con 0x7fef400cdd70 2024-04-25T14:11:13.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.089+0000 7fef3ea00640 1 -- 172.21.15.86:0/771882846 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef300ab9d0 con 0x7fef400cdd70 2024-04-25T14:11:13.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.089+0000 7fef46600640 1 -- 172.21.15.86:0/771882846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef400cdd70 msgr2=0x7fef400ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:13.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.089+0000 7fef46600640 1 --2- 172.21.15.86:0/771882846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef400cdd70 0x7fef400ce180 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fef40073230 tx=0x7fef3001c3f0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.089+0000 7fef46600640 1 -- 172.21.15.86:0/771882846 shutdown_connections 2024-04-25T14:11:13.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.089+0000 7fef46600640 1 --2- 172.21.15.86:0/771882846 >> v2:172.21.15.155:3300/0 conn(0x7fef40070e20 0x7fef40071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.089+0000 7fef46600640 1 --2- 172.21.15.86:0/771882846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef400cdd70 0x7fef400ce180 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.089+0000 7fef46600640 1 -- 172.21.15.86:0/771882846 >> 172.21.15.86:0/771882846 conn(0x7fef400cffc0 msgr2=0x7fef400d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:13.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.089+0000 7fef46600640 1 -- 172.21.15.86:0/771882846 shutdown_connections 2024-04-25T14:11:13.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.089+0000 7fef46600640 1 -- 172.21.15.86:0/771882846 wait complete. 2024-04-25T14:11:13.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.093+0000 7fef46600640 1 Processor -- start 2024-04-25T14:11:13.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.093+0000 7fef46600640 1 -- start start 2024-04-25T14:11:13.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.097+0000 7fef46600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef4006e700 0x7fef401506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:13.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.097+0000 7fef46600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef40070e20 0x7fef40160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:13.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.097+0000 7fef46600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef400cdd70 0x7fef4015dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:13.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.097+0000 7fef46600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef4016cc00 con 0x7fef4006e700 2024-04-25T14:11:13.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.097+0000 7fef46600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fef40164380 con 0x7fef400cdd70 2024-04-25T14:11:13.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.097+0000 7fef46600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fef4016cd80 con 0x7fef40070e20 2024-04-25T14:11:13.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.097+0000 7fef3f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef4006e700 0x7fef401506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:13.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.097+0000 7fef3f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef4006e700 0x7fef401506e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59652/0 (socket says 172.21.15.86:59652) 2024-04-25T14:11:13.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.097+0000 7fef3f400640 1 -- 172.21.15.86:0/2733015326 learned_addr learned my addr 172.21.15.86:0/2733015326 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:13.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.097+0000 7fef36a00640 1 --2- 172.21.15.86:0/2733015326 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef40070e20 0x7fef40160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:13.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.097+0000 7fef3fe00640 1 --2- 172.21.15.86:0/2733015326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef400cdd70 0x7fef4015dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:13.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.097+0000 7fef3f400640 1 -- 172.21.15.86:0/2733015326 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef40070e20 msgr2=0x7fef40160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:13.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.097+0000 7fef3f400640 1 --2- 172.21.15.86:0/2733015326 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef40070e20 0x7fef40160cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.097+0000 7fef3f400640 1 -- 172.21.15.86:0/2733015326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef400cdd70 msgr2=0x7fef4015dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:13.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.097+0000 7fef3f400640 1 --2- 172.21.15.86:0/2733015326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef400cdd70 0x7fef4015dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.097+0000 7fef3f400640 1 -- 172.21.15.86:0/2733015326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef40150c20 con 0x7fef4006e700 2024-04-25T14:11:13.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.097+0000 7fef3f400640 1 --2- 172.21.15.86:0/2733015326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef4006e700 0x7fef401506e0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fef3001a020 tx=0x7fef3001ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:13.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.097+0000 7fef37e00640 1 -- 172.21.15.86:0/2733015326 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef300b0040 con 0x7fef4006e700 2024-04-25T14:11:13.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.097+0000 7fef37e00640 1 -- 172.21.15.86:0/2733015326 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef30004020 con 0x7fef4006e700 2024-04-25T14:11:13.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.097+0000 7fef46600640 1 -- 172.21.15.86:0/2733015326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef401510b0 con 0x7fef4006e700 2024-04-25T14:11:13.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.097+0000 7fef37e00640 1 -- 172.21.15.86:0/2733015326 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef30006e20 con 0x7fef4006e700 2024-04-25T14:11:13.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.097+0000 7fef46600640 1 -- 172.21.15.86:0/2733015326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef40151670 con 0x7fef4006e700 2024-04-25T14:11:13.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.097+0000 7fef46600640 1 -- 172.21.15.86:0/2733015326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef400720e0 con 0x7fef4006e700 2024-04-25T14:11:13.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.101+0000 7fef37e00640 1 -- 172.21.15.86:0/2733015326 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 61) ==== 93309+0+0 (secure 0 0 0) 0x7fef300bf060 con 0x7fef4006e700 2024-04-25T14:11:13.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.101+0000 7fef37e00640 1 --2- 172.21.15.86:0/2733015326 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fef1c06f550 0x7fef1c071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:13.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.101+0000 7fef36a00640 1 -- 172.21.15.86:0/2733015326 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fef1c06f550 msgr2=0x7fef1c071a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2221276863 2024-04-25T14:11:13.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.101+0000 7fef37e00640 1 -- 172.21.15.86:0/2733015326 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6488+0+0 (secure 0 0 0) 0x7fef30129210 con 0x7fef4006e700 2024-04-25T14:11:13.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.101+0000 7fef36a00640 1 --2- 172.21.15.86:0/2733015326 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fef1c06f550 0x7fef1c071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:11:13.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.105+0000 7fef37e00640 1 -- 172.21.15.86:0/2733015326 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fef30016070 con 0x7fef4006e700 2024-04-25T14:11:13.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.265+0000 7fef46600640 1 -- 172.21.15.86:0/2733015326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef40076bf0 con 0x7fef4006e700 2024-04-25T14:11:13.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.265+0000 7fef37e00640 1 -- 172.21.15.86:0/2733015326 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) ==== 74+0+17460 (secure 0 0 0) 0x7fef300f54e0 con 0x7fef4006e700 2024-04-25T14:11:13.263 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:13.263 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":128,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:11:04.532554+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:10:43.913889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T14:10:44.419362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/3961472441":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/287404215":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/889708173":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/3224392570":"2024-04-26T14:11:04.532513+0000","172.21.15.86:6833/3964666460":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/553620002":"2024-04-26T14:11:04.532513+0000","172.21.15.86:6832/3964666460":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/3726306847":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/2412030790":"2024-04-25T15:10:55.520329+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:11:13.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.269+0000 7fef46600640 1 -- 172.21.15.86:0/2733015326 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fef1c06f550 msgr2=0x7fef1c071a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:11:13.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.269+0000 7fef46600640 1 --2- 172.21.15.86:0/2733015326 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fef1c06f550 0x7fef1c071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.269+0000 7fef46600640 1 -- 172.21.15.86:0/2733015326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef4006e700 msgr2=0x7fef401506e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:13.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.269+0000 7fef46600640 1 --2- 172.21.15.86:0/2733015326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef4006e700 0x7fef401506e0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fef3001a020 tx=0x7fef3001ac30 comp rx=0 tx=0).stop 2024-04-25T14:11:13.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.269+0000 7fef46600640 1 -- 172.21.15.86:0/2733015326 shutdown_connections 2024-04-25T14:11:13.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.269+0000 7fef46600640 1 --2- 172.21.15.86:0/2733015326 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fef1c06f550 0x7fef1c071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.269+0000 7fef46600640 1 --2- 172.21.15.86:0/2733015326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef400cdd70 0x7fef4015dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.269+0000 7fef46600640 1 --2- 172.21.15.86:0/2733015326 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef40070e20 0x7fef40160cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.269+0000 7fef46600640 1 --2- 172.21.15.86:0/2733015326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef4006e700 0x7fef401506e0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.269+0000 7fef46600640 1 -- 172.21.15.86:0/2733015326 >> 172.21.15.86:0/2733015326 conn(0x7fef400cffc0 msgr2=0x7fef400ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:13.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.269+0000 7fef46600640 1 -- 172.21.15.86:0/2733015326 shutdown_connections 2024-04-25T14:11:13.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.269+0000 7fef46600640 1 -- 172.21.15.86:0/2733015326 wait complete. 2024-04-25T14:11:13.285 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:10:43.913889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:10:44.419362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:11:13.286 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:11:13.291 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:13.290+0000 7fdf0cf7e180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:11:13.321 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:13.325+0000 7f8b38049180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:11:13.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.441+0000 7f62daa00640 1 Processor -- start 2024-04-25T14:11:13.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.441+0000 7f62daa00640 1 -- start start 2024-04-25T14:11:13.441 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:13.442+0000 7fdf0cf7e180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:11:13.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.445+0000 7f62daa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62d415fc70 0x7f62d4070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:13.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.445+0000 7f62daa00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62d406f0a0 con 0x7f62d4071360 2024-04-25T14:11:13.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.445+0000 7f62daa00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62d4071770 con 0x7f62d4163f20 2024-04-25T14:11:13.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.445+0000 7f62daa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f62d406e7f0 con 0x7f62d415fc70 2024-04-25T14:11:13.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.445+0000 7f62cbe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f62d4071360 0x7f62d4163810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34836/0 (socket says 172.21.15.86:34836) 2024-04-25T14:11:13.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.445+0000 7f62cbe00640 1 -- 172.21.15.86:0/1463369222 learned_addr learned my addr 172.21.15.86:0/1463369222 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:13.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.445+0000 7f62d3e00640 1 --2- 172.21.15.86:0/1463369222 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62d415fc70 0x7f62d4070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:13.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.445+0000 7f62d3400640 1 -- 172.21.15.86:0/1463369222 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4214187729 0 0) 0x7f62bc001120 con 0x7f62d4071360 2024-04-25T14:11:13.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.445+0000 7f62d3400640 1 -- 172.21.15.86:0/1463369222 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62b8003530 con 0x7f62d4071360 2024-04-25T14:11:13.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.445+0000 7f62d3400640 1 -- 172.21.15.86:0/1463369222 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2818876625 0 0) 0x7f62cc0014a0 con 0x7f62d4163f20 2024-04-25T14:11:13.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.445+0000 7f62d3400640 1 -- 172.21.15.86:0/1463369222 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62bc001120 con 0x7f62d4163f20 2024-04-25T14:11:13.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.445+0000 7f62d3e00640 1 -- 172.21.15.86:0/1463369222 >> v1:172.21.15.155:6789/0 conn(0x7f62d4163f20 legacy=0x7f62d40cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:13.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.445+0000 7f62d3e00640 1 -- 172.21.15.86:0/1463369222 >> v1:172.21.15.86:6789/0 conn(0x7f62d4071360 legacy=0x7f62d4163810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:13.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.445+0000 7f62d3e00640 1 -- 172.21.15.86:0/1463369222 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d41646d0 con 0x7f62d415fc70 2024-04-25T14:11:13.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.449+0000 7f62d3e00640 1 --2- 172.21.15.86:0/1463369222 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62d415fc70 0x7f62d4070e20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f62c40097f0 tx=0x7f62c401c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a23abe10f47a4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:13.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.449+0000 7f62d3400640 1 -- 172.21.15.86:0/1463369222 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c40b1070 con 0x7f62d415fc70 2024-04-25T14:11:13.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.449+0000 7f62d3400640 1 -- 172.21.15.86:0/1463369222 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62c40a8050 con 0x7f62d415fc70 2024-04-25T14:11:13.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.449+0000 7f62d3400640 1 -- 172.21.15.86:0/1463369222 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c40ac840 con 0x7f62d415fc70 2024-04-25T14:11:13.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.449+0000 7f62daa00640 1 -- 172.21.15.86:0/1463369222 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62d415fc70 msgr2=0x7f62d4070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:13.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.449+0000 7f62daa00640 1 --2- 172.21.15.86:0/1463369222 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62d415fc70 0x7f62d4070e20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f62c40097f0 tx=0x7f62c401c4e0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.449+0000 7f62daa00640 1 -- 172.21.15.86:0/1463369222 shutdown_connections 2024-04-25T14:11:13.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.449+0000 7f62daa00640 1 --2- 172.21.15.86:0/1463369222 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62d415fc70 0x7f62d4070e20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.449+0000 7f62daa00640 1 -- 172.21.15.86:0/1463369222 >> 172.21.15.86:0/1463369222 conn(0x7f62d40cffc0 msgr2=0x7f62d40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:13.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.449+0000 7f62daa00640 1 -- 172.21.15.86:0/1463369222 shutdown_connections 2024-04-25T14:11:13.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.449+0000 7f62daa00640 1 -- 172.21.15.86:0/1463369222 wait complete. 2024-04-25T14:11:13.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.453+0000 7f62daa00640 1 Processor -- start 2024-04-25T14:11:13.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.453+0000 7f62daa00640 1 -- start start 2024-04-25T14:11:13.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.453+0000 7f62daa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62d4071360 0x7f62d414e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:13.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.453+0000 7f62daa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62d415fc70 0x7f62d40caf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:13.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.453+0000 7f62daa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d4163f20 0x7f62d40b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:13.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.453+0000 7f62daa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62d416db80 con 0x7f62d4163f20 2024-04-25T14:11:13.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.453+0000 7f62daa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f62d4160080 con 0x7f62d415fc70 2024-04-25T14:11:13.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.453+0000 7f62daa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f62d416dd00 con 0x7f62d4071360 2024-04-25T14:11:13.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.453+0000 7f62d8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d4163f20 0x7f62d40b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:13.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.453+0000 7f62d3e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62d4071360 0x7f62d414e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:13.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.453+0000 7f62d8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d4163f20 0x7f62d40b2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59658/0 (socket says 172.21.15.86:59658) 2024-04-25T14:11:13.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.453+0000 7f62d8a00640 1 -- 172.21.15.86:0/2706246099 learned_addr learned my addr 172.21.15.86:0/2706246099 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:13.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.453+0000 7f62cbe00640 1 --2- 172.21.15.86:0/2706246099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62d415fc70 0x7f62d40caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:13.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.457+0000 7f62d8a00640 1 -- 172.21.15.86:0/2706246099 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62d4071360 msgr2=0x7f62d414e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:13.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.457+0000 7f62d8a00640 1 --2- 172.21.15.86:0/2706246099 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62d4071360 0x7f62d414e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.457+0000 7f62d8a00640 1 -- 172.21.15.86:0/2706246099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62d415fc70 msgr2=0x7f62d40caf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:13.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.457+0000 7f62d8a00640 1 --2- 172.21.15.86:0/2706246099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62d415fc70 0x7f62d40caf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.457+0000 7f62d8a00640 1 -- 172.21.15.86:0/2706246099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d40cc950 con 0x7f62d4163f20 2024-04-25T14:11:13.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.457+0000 7f62d8a00640 1 --2- 172.21.15.86:0/2706246099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d4163f20 0x7f62d40b2fa0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00c730 tx=0x7f62cc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:13.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.457+0000 7f62d0c00640 1 -- 172.21.15.86:0/2706246099 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc003b80 con 0x7f62d4163f20 2024-04-25T14:11:13.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.457+0000 7f62d0c00640 1 -- 172.21.15.86:0/2706246099 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62cc003d20 con 0x7f62d4163f20 2024-04-25T14:11:13.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.457+0000 7f62daa00640 1 -- 172.21.15.86:0/2706246099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d40cc670 con 0x7f62d4163f20 2024-04-25T14:11:13.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.457+0000 7f62d0c00640 1 -- 172.21.15.86:0/2706246099 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc00aa90 con 0x7f62d4163f20 2024-04-25T14:11:13.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.457+0000 7f62daa00640 1 -- 172.21.15.86:0/2706246099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d40b6490 con 0x7f62d4163f20 2024-04-25T14:11:13.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.457+0000 7f62daa00640 1 -- 172.21.15.86:0/2706246099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62d40720e0 con 0x7f62d4163f20 2024-04-25T14:11:13.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.457+0000 7f62d0c00640 1 -- 172.21.15.86:0/2706246099 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 61) ==== 93309+0+0 (secure 0 0 0) 0x7f62cc00ac30 con 0x7f62d4163f20 2024-04-25T14:11:13.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.461+0000 7f62d0c00640 1 --2- 172.21.15.86:0/2706246099 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f62b006f550 0x7f62b0071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:13.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.461+0000 7f62d3e00640 1 -- 172.21.15.86:0/2706246099 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f62b006f550 msgr2=0x7f62b0071a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2221276863 2024-04-25T14:11:13.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.461+0000 7f62d0c00640 1 -- 172.21.15.86:0/2706246099 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6488+0+0 (secure 0 0 0) 0x7f62cc08f5b0 con 0x7f62d4163f20 2024-04-25T14:11:13.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.461+0000 7f62d3e00640 1 --2- 172.21.15.86:0/2706246099 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f62b006f550 0x7f62b0071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:11:13.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.465+0000 7f62d0c00640 1 -- 172.21.15.86:0/2706246099 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f62d4071770 con 0x7f62d4163f20 2024-04-25T14:11:13.479 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:13.481+0000 7f8b38049180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:11:13.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.625+0000 7f62daa00640 1 -- 172.21.15.86:0/2706246099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f62d4076bf0 con 0x7f62d4163f20 2024-04-25T14:11:13.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.625+0000 7f62d0c00640 1 -- 172.21.15.86:0/2706246099 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) ==== 93+0+10 (secure 0 0 0) 0x7f62cc064060 con 0x7f62d4163f20 2024-04-25T14:11:13.622 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:11:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.629+0000 7f62daa00640 1 -- 172.21.15.86:0/2706246099 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f62b006f550 msgr2=0x7f62b0071a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:11:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.629+0000 7f62daa00640 1 --2- 172.21.15.86:0/2706246099 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f62b006f550 0x7f62b0071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.629+0000 7f62daa00640 1 -- 172.21.15.86:0/2706246099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d4163f20 msgr2=0x7f62d40b2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.629+0000 7f62daa00640 1 --2- 172.21.15.86:0/2706246099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d4163f20 0x7f62d40b2fa0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00c730 tx=0x7f62cc00cbf0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.629+0000 7f62daa00640 1 -- 172.21.15.86:0/2706246099 shutdown_connections 2024-04-25T14:11:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.629+0000 7f62daa00640 1 --2- 172.21.15.86:0/2706246099 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f62b006f550 0x7f62b0071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.629+0000 7f62daa00640 1 --2- 172.21.15.86:0/2706246099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d4163f20 0x7f62d40b2fa0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.629+0000 7f62daa00640 1 --2- 172.21.15.86:0/2706246099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62d415fc70 0x7f62d40caf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.629+0000 7f62daa00640 1 --2- 172.21.15.86:0/2706246099 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62d4071360 0x7f62d414e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.629+0000 7f62daa00640 1 -- 172.21.15.86:0/2706246099 >> 172.21.15.86:0/2706246099 conn(0x7f62d40cffc0 msgr2=0x7f62d40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.629+0000 7f62daa00640 1 -- 172.21.15.86:0/2706246099 shutdown_connections 2024-04-25T14:11:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.629+0000 7f62daa00640 1 -- 172.21.15.86:0/2706246099 wait complete. 2024-04-25T14:11:13.645 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:11:13.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.841+0000 7f94d3200640 1 Processor -- start 2024-04-25T14:11:13.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.841+0000 7f94d3200640 1 -- start start 2024-04-25T14:11:13.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.845+0000 7f94d3200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94cc0d1530 0x7f94cc0d1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:13.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.845+0000 7f94d3200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f94cc06d8a0 0x7f94cc06dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:13.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.845+0000 7f94d3200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94cc164700 con 0x7f94cc072630 2024-04-25T14:11:13.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.845+0000 7f94d3200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f94cc163940 con 0x7f94cc06d8a0 2024-04-25T14:11:13.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.845+0000 7f94d3200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f94cc163c20 con 0x7f94cc0d1530 2024-04-25T14:11:13.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.845+0000 7f94d1200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f94cc06d8a0 0x7f94cc06dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:13.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.845+0000 7f94cbe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f94cc072630 0x7f94cc072a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34846/0 (socket says 172.21.15.86:34846) 2024-04-25T14:11:13.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.845+0000 7f94d1200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f94cc06d8a0 0x7f94cc06dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59664/0 (socket says 172.21.15.86:59664) 2024-04-25T14:11:13.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.845+0000 7f94cbe00640 1 -- 172.21.15.86:0/2355576390 learned_addr learned my addr 172.21.15.86:0/2355576390 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:13.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.845+0000 7f94c3e00640 1 --2- 172.21.15.86:0/2355576390 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94cc0d1530 0x7f94cc0d1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:13.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.845+0000 7f94d1200640 1 -- 172.21.15.86:0/2355576390 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94cc0d1530 msgr2=0x7f94cc0d1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:13.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.845+0000 7f94d1200640 1 --2- 172.21.15.86:0/2355576390 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94cc0d1530 0x7f94cc0d1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.845+0000 7f94d1200640 1 -- 172.21.15.86:0/2355576390 >> v1:172.21.15.86:6789/0 conn(0x7f94cc072630 legacy=0x7f94cc072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:13.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.845+0000 7f94d1200640 1 -- 172.21.15.86:0/2355576390 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94cc163f00 con 0x7f94cc06d8a0 2024-04-25T14:11:13.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.845+0000 7f94cb400640 1 -- 172.21.15.86:0/2355576390 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4112078765 0 0) 0x7f94cc164700 con 0x7f94cc072630 2024-04-25T14:11:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.849+0000 7f94d1200640 1 --2- 172.21.15.86:0/2355576390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94cc06d8a0 0x7f94cc06dcb0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f94c4009d90 tx=0x7f94c401f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a73cccf677315e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.849+0000 7f94cb400640 1 -- 172.21.15.86:0/2355576390 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c40b4070 con 0x7f94cc06d8a0 2024-04-25T14:11:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.849+0000 7f94cb400640 1 -- 172.21.15.86:0/2355576390 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94c40af600 con 0x7f94cc06d8a0 2024-04-25T14:11:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.849+0000 7f94cb400640 1 -- 172.21.15.86:0/2355576390 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c40af9d0 con 0x7f94cc06d8a0 2024-04-25T14:11:13.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.849+0000 7f94d3200640 1 -- 172.21.15.86:0/2355576390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94cc06d8a0 msgr2=0x7f94cc06dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:13.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.849+0000 7f94d3200640 1 --2- 172.21.15.86:0/2355576390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94cc06d8a0 0x7f94cc06dcb0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f94c4009d90 tx=0x7f94c401f150 comp rx=0 tx=0).stop 2024-04-25T14:11:13.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.849+0000 7f94d3200640 1 -- 172.21.15.86:0/2355576390 shutdown_connections 2024-04-25T14:11:13.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.849+0000 7f94d3200640 1 --2- 172.21.15.86:0/2355576390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94cc06d8a0 0x7f94cc06dcb0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.849+0000 7f94d3200640 1 --2- 172.21.15.86:0/2355576390 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94cc0d1530 0x7f94cc0d1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.849+0000 7f94d3200640 1 -- 172.21.15.86:0/2355576390 >> 172.21.15.86:0/2355576390 conn(0x7f94cc0d3570 msgr2=0x7f94cc0d3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:13.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.849+0000 7f94d3200640 1 -- 172.21.15.86:0/2355576390 shutdown_connections 2024-04-25T14:11:13.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.849+0000 7f94d3200640 1 -- 172.21.15.86:0/2355576390 wait complete. 2024-04-25T14:11:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.853+0000 7f94d3200640 1 Processor -- start 2024-04-25T14:11:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.853+0000 7f94d3200640 1 -- start start 2024-04-25T14:11:13.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.853+0000 7f94d3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94cc06d8a0 0x7f94cc0b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:13.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.853+0000 7f94d3200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94cc072630 0x7f94cc15edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:13.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.853+0000 7f94d3200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94cc0d1530 0x7f94cc1f64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:13.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.853+0000 7f94d3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f94cc0d1e80 con 0x7f94cc06d8a0 2024-04-25T14:11:13.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.853+0000 7f94d3200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f94cc0d2260 con 0x7f94cc0d1530 2024-04-25T14:11:13.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.853+0000 7f94d3200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f94cc0d2000 con 0x7f94cc072630 2024-04-25T14:11:13.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.853+0000 7f94cbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94cc06d8a0 0x7f94cc0b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:13.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.853+0000 7f94cbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94cc06d8a0 0x7f94cc0b8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59674/0 (socket says 172.21.15.86:59674) 2024-04-25T14:11:13.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.853+0000 7f94cbe00640 1 -- 172.21.15.86:0/3981750964 learned_addr learned my addr 172.21.15.86:0/3981750964 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:13.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.853+0000 7f94c3e00640 1 --2- 172.21.15.86:0/3981750964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94cc072630 0x7f94cc15edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:13.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.853+0000 7f94d1200640 1 --2- 172.21.15.86:0/3981750964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94cc0d1530 0x7f94cc1f64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:13.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.853+0000 7f94cbe00640 1 -- 172.21.15.86:0/3981750964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94cc072630 msgr2=0x7f94cc15edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:13.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.853+0000 7f94cbe00640 1 --2- 172.21.15.86:0/3981750964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94cc072630 0x7f94cc15edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.853+0000 7f94cbe00640 1 -- 172.21.15.86:0/3981750964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94cc0d1530 msgr2=0x7f94cc1f64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:13.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.853+0000 7f94cbe00640 1 --2- 172.21.15.86:0/3981750964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94cc0d1530 0x7f94cc1f64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.853+0000 7f94cbe00640 1 -- 172.21.15.86:0/3981750964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94cc0b79f0 con 0x7f94cc06d8a0 2024-04-25T14:11:13.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.853+0000 7f94cbe00640 1 --2- 172.21.15.86:0/3981750964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94cc06d8a0 0x7f94cc0b8a90 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00c480 tx=0x7f94bc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:13.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.853+0000 7f94c8c00640 1 -- 172.21.15.86:0/3981750964 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc018070 con 0x7f94cc06d8a0 2024-04-25T14:11:13.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.853+0000 7f94c8c00640 1 -- 172.21.15.86:0/3981750964 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94bc0074e0 con 0x7f94cc06d8a0 2024-04-25T14:11:13.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.853+0000 7f94c8c00640 1 -- 172.21.15.86:0/3981750964 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc0049f0 con 0x7f94cc06d8a0 2024-04-25T14:11:13.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.853+0000 7f94d3200640 1 -- 172.21.15.86:0/3981750964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94cc0b7cd0 con 0x7f94cc06d8a0 2024-04-25T14:11:13.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.853+0000 7f94d3200640 1 -- 172.21.15.86:0/3981750964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94cc0b8290 con 0x7f94cc06d8a0 2024-04-25T14:11:13.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.857+0000 7f94d3200640 1 -- 172.21.15.86:0/3981750964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94cc072170 con 0x7f94cc06d8a0 2024-04-25T14:11:13.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.857+0000 7f94c8c00640 1 -- 172.21.15.86:0/3981750964 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 61) ==== 93309+0+0 (secure 0 0 0) 0x7f94bc004b90 con 0x7f94cc06d8a0 2024-04-25T14:11:13.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.857+0000 7f94c8c00640 1 --2- 172.21.15.86:0/3981750964 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f94a806f550 0x7f94a8071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:13.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.857+0000 7f94c8c00640 1 -- 172.21.15.86:0/3981750964 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6488+0+0 (secure 0 0 0) 0x7f94bc08ea00 con 0x7f94cc06d8a0 2024-04-25T14:11:13.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.857+0000 7f94c3e00640 1 -- 172.21.15.86:0/3981750964 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f94a806f550 msgr2=0x7f94a8071a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2221276863 2024-04-25T14:11:13.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.857+0000 7f94c3e00640 1 --2- 172.21.15.86:0/3981750964 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f94a806f550 0x7f94a8071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:11:13.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:13.861+0000 7f94c8c00640 1 -- 172.21.15.86:0/3981750964 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f94bc05af00 con 0x7f94cc06d8a0 2024-04-25T14:11:13.915 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:13.914+0000 7fdf0cf7e180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:11:13.969 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:13.973+0000 7f8b38049180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:11:13.974 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:13.974+0000 7fdf0cf7e180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:11:14.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.021+0000 7f94d3200640 1 -- 172.21.15.86:0/3981750964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f94cc076bf0 con 0x7f94cc06d8a0 2024-04-25T14:11:14.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.021+0000 7f94c8c00640 1 -- 172.21.15.86:0/3981750964 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v128) ==== 104+0+11 (secure 0 0 0) 0x7f94bc05f200 con 0x7f94cc06d8a0 2024-04-25T14:11:14.019 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:11:14.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.025+0000 7f94d3200640 1 -- 172.21.15.86:0/3981750964 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f94a806f550 msgr2=0x7f94a8071a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:11:14.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.025+0000 7f94d3200640 1 --2- 172.21.15.86:0/3981750964 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f94a806f550 0x7f94a8071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.025+0000 7f94d3200640 1 -- 172.21.15.86:0/3981750964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94cc06d8a0 msgr2=0x7f94cc0b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:14.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.025+0000 7f94d3200640 1 --2- 172.21.15.86:0/3981750964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94cc06d8a0 0x7f94cc0b8a90 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00c480 tx=0x7f94bc00c940 comp rx=0 tx=0).stop 2024-04-25T14:11:14.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.025+0000 7f94d3200640 1 -- 172.21.15.86:0/3981750964 shutdown_connections 2024-04-25T14:11:14.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.025+0000 7f94d3200640 1 --2- 172.21.15.86:0/3981750964 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f94a806f550 0x7f94a8071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.025+0000 7f94d3200640 1 --2- 172.21.15.86:0/3981750964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94cc0d1530 0x7f94cc1f64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.025+0000 7f94d3200640 1 --2- 172.21.15.86:0/3981750964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94cc072630 0x7f94cc15edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.025+0000 7f94d3200640 1 --2- 172.21.15.86:0/3981750964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94cc06d8a0 0x7f94cc0b8a90 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.025+0000 7f94d3200640 1 -- 172.21.15.86:0/3981750964 >> 172.21.15.86:0/3981750964 conn(0x7f94cc0d3570 msgr2=0x7f94cc0c90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:14.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.025+0000 7f94d3200640 1 -- 172.21.15.86:0/3981750964 shutdown_connections 2024-04-25T14:11:14.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.025+0000 7f94d3200640 1 -- 172.21.15.86:0/3981750964 wait complete. 2024-04-25T14:11:14.029 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:14.033+0000 7f8b38049180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:11:14.030 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:14.030+0000 7fdf0cf7e180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:11:14.042 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:11:14.089 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:14.093+0000 7f8b38049180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:11:14.117 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:14.118+0000 7fdf0cf7e180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:11:14.174 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:14.174+0000 7fdf0cf7e180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:11:14.181 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:14.185+0000 7f8b38049180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:11:14.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.197+0000 7f4aa0e00640 1 Processor -- start 2024-04-25T14:11:14.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.197+0000 7f4aa0e00640 1 -- start start 2024-04-25T14:11:14.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.201+0000 7f4aa0e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a9c0d1df0 0x7f4a9c0cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:14.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.201+0000 7f4aa0e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4a9c0cdcc0 0x7f4a9c0ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:14.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.201+0000 7f4aa0e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a9c15e5c0 con 0x7f4a9c070330 2024-04-25T14:11:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.201+0000 7f4aa0e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f4a9c06fd40 con 0x7f4a9c0cdcc0 2024-04-25T14:11:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.201+0000 7f4aa0e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4a9c15e8a0 con 0x7f4a9c0d1df0 2024-04-25T14:11:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.201+0000 7f4a99e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f4a9c070330 0x7f4a9c0d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34860/0 (socket says 172.21.15.86:34860) 2024-04-25T14:11:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.201+0000 7f4a99e00640 1 -- 172.21.15.86:0/1090927288 learned_addr learned my addr 172.21.15.86:0/1090927288 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.201+0000 7f4a99400640 1 --2- 172.21.15.86:0/1090927288 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a9c0d1df0 0x7f4a9c0cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.201+0000 7f4a9a800640 1 --2- 172.21.15.86:0/1090927288 >> v2:172.21.15.155:3300/0 conn(0x7f4a9c0cdcc0 0x7f4a9c0ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.201+0000 7f4a98a00640 1 -- 172.21.15.86:0/1090927288 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1710400131 0 0) 0x7f4a9c15e5c0 con 0x7f4a9c070330 2024-04-25T14:11:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.201+0000 7f4a98a00640 1 -- 172.21.15.86:0/1090927288 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a80003530 con 0x7f4a9c070330 2024-04-25T14:11:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.201+0000 7f4a99400640 1 -- 172.21.15.86:0/1090927288 >> v2:172.21.15.155:3300/0 conn(0x7f4a9c0cdcc0 msgr2=0x7f4a9c0ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.201+0000 7f4a99400640 1 --2- 172.21.15.86:0/1090927288 >> v2:172.21.15.155:3300/0 conn(0x7f4a9c0cdcc0 0x7f4a9c0ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.201+0000 7f4a99400640 1 -- 172.21.15.86:0/1090927288 >> v1:172.21.15.86:6789/0 conn(0x7f4a9c070330 legacy=0x7f4a9c0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.201+0000 7f4a99400640 1 -- 172.21.15.86:0/1090927288 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9c15eb80 con 0x7f4a9c0d1df0 2024-04-25T14:11:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.201+0000 7f4a9a800640 1 --2- 172.21.15.86:0/1090927288 >> v2:172.21.15.155:3300/0 conn(0x7f4a9c0cdcc0 0x7f4a9c0ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:11:14.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.205+0000 7f4a99400640 1 --2- 172.21.15.86:0/1090927288 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a9c0d1df0 0x7f4a9c0cd780 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4a880097c0 tx=0x7f4a8801c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75c790752a9da386 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:14.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.205+0000 7f4a98a00640 1 -- 172.21.15.86:0/1090927288 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a880b1070 con 0x7f4a9c0d1df0 2024-04-25T14:11:14.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.205+0000 7f4a98a00640 1 -- 172.21.15.86:0/1090927288 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a880ac620 con 0x7f4a9c0d1df0 2024-04-25T14:11:14.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.205+0000 7f4a98a00640 1 -- 172.21.15.86:0/1090927288 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a880ac9f0 con 0x7f4a9c0d1df0 2024-04-25T14:11:14.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.205+0000 7f4aa0e00640 1 -- 172.21.15.86:0/1090927288 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a9c0d1df0 msgr2=0x7f4a9c0cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:14.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.205+0000 7f4aa0e00640 1 --2- 172.21.15.86:0/1090927288 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a9c0d1df0 0x7f4a9c0cd780 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4a880097c0 tx=0x7f4a8801c3e0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.205+0000 7f4aa0e00640 1 -- 172.21.15.86:0/1090927288 shutdown_connections 2024-04-25T14:11:14.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.205+0000 7f4aa0e00640 1 --2- 172.21.15.86:0/1090927288 >> v2:172.21.15.155:3300/0 conn(0x7f4a9c0cdcc0 0x7f4a9c0ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.205+0000 7f4aa0e00640 1 --2- 172.21.15.86:0/1090927288 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a9c0d1df0 0x7f4a9c0cd780 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.205+0000 7f4aa0e00640 1 -- 172.21.15.86:0/1090927288 >> 172.21.15.86:0/1090927288 conn(0x7f4a9c0b4650 msgr2=0x7f4a9c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:14.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.205+0000 7f4aa0e00640 1 -- 172.21.15.86:0/1090927288 shutdown_connections 2024-04-25T14:11:14.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.205+0000 7f4aa0e00640 1 -- 172.21.15.86:0/1090927288 wait complete. 2024-04-25T14:11:14.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.209+0000 7f4aa0e00640 1 Processor -- start 2024-04-25T14:11:14.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.209+0000 7f4aa0e00640 1 -- start start 2024-04-25T14:11:14.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.209+0000 7f4aa0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9c070330 0x7f4a9c157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:14.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.209+0000 7f4aa0e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a9c0cdcc0 0x7f4a9c157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:14.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.209+0000 7f4aa0e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a9c0b0df0 0x7f4a9c1fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:14.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.209+0000 7f4aa0e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c15f4b0 con 0x7f4a9c070330 2024-04-25T14:11:14.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.209+0000 7f4aa0e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c15f330 con 0x7f4a9c0b0df0 2024-04-25T14:11:14.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.209+0000 7f4aa0e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4a9c070740 con 0x7f4a9c0cdcc0 2024-04-25T14:11:14.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.209+0000 7f4a99e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9c070330 0x7f4a9c157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:14.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.209+0000 7f4a99e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9c070330 0x7f4a9c157610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59678/0 (socket says 172.21.15.86:59678) 2024-04-25T14:11:14.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.209+0000 7f4a99e00640 1 -- 172.21.15.86:0/2973278265 learned_addr learned my addr 172.21.15.86:0/2973278265 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:14.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.209+0000 7f4a99400640 1 --2- 172.21.15.86:0/2973278265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a9c0cdcc0 0x7f4a9c157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:14.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.209+0000 7f4a9a800640 1 --2- 172.21.15.86:0/2973278265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a9c0b0df0 0x7f4a9c1fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:14.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.209+0000 7f4a99e00640 1 -- 172.21.15.86:0/2973278265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a9c0cdcc0 msgr2=0x7f4a9c157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:14.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.209+0000 7f4a99e00640 1 --2- 172.21.15.86:0/2973278265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a9c0cdcc0 0x7f4a9c157b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.209+0000 7f4a99e00640 1 -- 172.21.15.86:0/2973278265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a9c0b0df0 msgr2=0x7f4a9c1fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:14.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.209+0000 7f4a99e00640 1 --2- 172.21.15.86:0/2973278265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a9c0b0df0 0x7f4a9c1fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.209+0000 7f4a99e00640 1 -- 172.21.15.86:0/2973278265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9c156850 con 0x7f4a9c070330 2024-04-25T14:11:14.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.209+0000 7f4a99e00640 1 --2- 172.21.15.86:0/2973278265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9c070330 0x7f4a9c157610 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c00c3f0 tx=0x7f4a8c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:14.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.209+0000 7f4a92000640 1 -- 172.21.15.86:0/2973278265 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c017070 con 0x7f4a9c070330 2024-04-25T14:11:14.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.209+0000 7f4a92000640 1 -- 172.21.15.86:0/2973278265 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a8c005d10 con 0x7f4a9c070330 2024-04-25T14:11:14.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.209+0000 7f4aa0e00640 1 -- 172.21.15.86:0/2973278265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a9c156b30 con 0x7f4a9c070330 2024-04-25T14:11:14.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.209+0000 7f4a92000640 1 -- 172.21.15.86:0/2973278265 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c007310 con 0x7f4a9c070330 2024-04-25T14:11:14.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.209+0000 7f4aa0e00640 1 -- 172.21.15.86:0/2973278265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a9c1f94f0 con 0x7f4a9c070330 2024-04-25T14:11:14.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.213+0000 7f4aa0e00640 1 -- 172.21.15.86:0/2973278265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a9c069d00 con 0x7f4a9c070330 2024-04-25T14:11:14.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.213+0000 7f4a92000640 1 -- 172.21.15.86:0/2973278265 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 61) ==== 93309+0+0 (secure 0 0 0) 0x7f4a8c01a070 con 0x7f4a9c070330 2024-04-25T14:11:14.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.213+0000 7f4a92000640 1 --2- 172.21.15.86:0/2973278265 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f4a7c06f550 0x7f4a7c071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:14.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.213+0000 7f4a99400640 1 -- 172.21.15.86:0/2973278265 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f4a7c06f550 msgr2=0x7f4a7c071a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2221276863 2024-04-25T14:11:14.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.213+0000 7f4a92000640 1 -- 172.21.15.86:0/2973278265 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6488+0+0 (secure 0 0 0) 0x7f4a8c08ea30 con 0x7f4a9c070330 2024-04-25T14:11:14.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.213+0000 7f4a99400640 1 --2- 172.21.15.86:0/2973278265 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f4a7c06f550 0x7f4a7c071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:11:14.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.221+0000 7f4a92000640 1 -- 172.21.15.86:0/2973278265 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4a8c05aee0 con 0x7f4a9c070330 2024-04-25T14:11:14.242 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:14.245+0000 7f8b38049180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:11:14.259 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:14.258+0000 7fdf0cf7e180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:11:14.318 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:14.318+0000 7fdf0cf7e180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:11:14.327 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:14.329+0000 7f8b38049180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:11:14.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.381+0000 7f4aa0e00640 1 -- 172.21.15.86:0/2973278265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4a9c0b3c90 con 0x7f4a9c070330 2024-04-25T14:11:14.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.381+0000 7f4a92000640 1 -- 172.21.15.86:0/2973278265 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v128) ==== 100+0+11 (secure 0 0 0) 0x7f4a8c05f1e0 con 0x7f4a9c070330 2024-04-25T14:11:14.379 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:11:14.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.385+0000 7f4aa0e00640 1 -- 172.21.15.86:0/2973278265 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f4a7c06f550 msgr2=0x7f4a7c071a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:11:14.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.385+0000 7f4aa0e00640 1 --2- 172.21.15.86:0/2973278265 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f4a7c06f550 0x7f4a7c071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.385+0000 7f4aa0e00640 1 -- 172.21.15.86:0/2973278265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9c070330 msgr2=0x7f4a9c157610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:14.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.385+0000 7f4aa0e00640 1 --2- 172.21.15.86:0/2973278265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9c070330 0x7f4a9c157610 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c00c3f0 tx=0x7f4a8c00c7f0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.385+0000 7f4aa0e00640 1 -- 172.21.15.86:0/2973278265 shutdown_connections 2024-04-25T14:11:14.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.385+0000 7f4aa0e00640 1 --2- 172.21.15.86:0/2973278265 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f4a7c06f550 0x7f4a7c071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.385+0000 7f4aa0e00640 1 --2- 172.21.15.86:0/2973278265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a9c0b0df0 0x7f4a9c1fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.385+0000 7f4aa0e00640 1 --2- 172.21.15.86:0/2973278265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a9c0cdcc0 0x7f4a9c157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.385+0000 7f4aa0e00640 1 --2- 172.21.15.86:0/2973278265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9c070330 0x7f4a9c157610 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.385+0000 7f4aa0e00640 1 -- 172.21.15.86:0/2973278265 >> 172.21.15.86:0/2973278265 conn(0x7f4a9c0b4650 msgr2=0x7f4a9c15d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:14.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.385+0000 7f4aa0e00640 1 -- 172.21.15.86:0/2973278265 shutdown_connections 2024-04-25T14:11:14.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.385+0000 7f4aa0e00640 1 -- 172.21.15.86:0/2973278265 wait complete. 2024-04-25T14:11:14.389 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:14.393+0000 7f8b38049180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:11:14.403 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T14:11:14.472 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:14.470+0000 7fdf0cf7e180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:11:14.531 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:14.530+0000 7fdf0cf7e180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:11:14.549 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:14.553+0000 7f8b38049180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:11:14.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.557+0000 7f4a8f800640 1 Processor -- start 2024-04-25T14:11:14.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.557+0000 7f4a8f800640 1 -- start start 2024-04-25T14:11:14.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.557+0000 7f4a8f800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4a880cdd70 0x7f4a880ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:14.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.557+0000 7f4a8f800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a8806e700 0x7f4a8806eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:14.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.557+0000 7f4a8f800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a8806f050 con 0x7f4a88070e20 2024-04-25T14:11:14.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.557+0000 7f4a8f800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f4a8806d940 con 0x7f4a880cdd70 2024-04-25T14:11:14.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.557+0000 7f4a8f800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4a8806dc20 con 0x7f4a8806e700 2024-04-25T14:11:14.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.557+0000 7f4a87e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f4a88070e20 0x7f4a88071230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34868/0 (socket says 172.21.15.86:34868) 2024-04-25T14:11:14.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.557+0000 7f4a87e00640 1 -- 172.21.15.86:0/623878590 learned_addr learned my addr 172.21.15.86:0/623878590 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:14.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.557+0000 7f4a8d800640 1 --2- 172.21.15.86:0/623878590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a8806e700 0x7f4a8806eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:14.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.557+0000 7f4a8ce00640 1 --2- 172.21.15.86:0/623878590 >> v2:172.21.15.155:3300/0 conn(0x7f4a880cdd70 0x7f4a880ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:14.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.557+0000 7f4a87400640 1 -- 172.21.15.86:0/623878590 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1329983882 0 0) 0x7f4a8806f050 con 0x7f4a88070e20 2024-04-25T14:11:14.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.557+0000 7f4a87400640 1 -- 172.21.15.86:0/623878590 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a70003530 con 0x7f4a88070e20 2024-04-25T14:11:14.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.561+0000 7f4a8d800640 1 -- 172.21.15.86:0/623878590 >> v2:172.21.15.155:3300/0 conn(0x7f4a880cdd70 msgr2=0x7f4a880ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:14.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.561+0000 7f4a8d800640 1 --2- 172.21.15.86:0/623878590 >> v2:172.21.15.155:3300/0 conn(0x7f4a880cdd70 0x7f4a880ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.561+0000 7f4a8d800640 1 -- 172.21.15.86:0/623878590 >> v1:172.21.15.86:6789/0 conn(0x7f4a88070e20 legacy=0x7f4a88071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:14.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.561+0000 7f4a8d800640 1 -- 172.21.15.86:0/623878590 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8806df00 con 0x7f4a8806e700 2024-04-25T14:11:14.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.561+0000 7f4a8d800640 1 --2- 172.21.15.86:0/623878590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a8806e700 0x7f4a8806eb10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4a80009d90 tx=0x7f4a8001c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0f54126a73f51ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:14.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.561+0000 7f4a87400640 1 -- 172.21.15.86:0/623878590 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a800b2070 con 0x7f4a8806e700 2024-04-25T14:11:14.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.561+0000 7f4a87400640 1 -- 172.21.15.86:0/623878590 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a800a57f0 con 0x7f4a8806e700 2024-04-25T14:11:14.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.561+0000 7f4a87400640 1 -- 172.21.15.86:0/623878590 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a800a5ba0 con 0x7f4a8806e700 2024-04-25T14:11:14.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.565+0000 7f4a8f800640 1 -- 172.21.15.86:0/623878590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a8806e700 msgr2=0x7f4a8806eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:14.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.565+0000 7f4a8f800640 1 --2- 172.21.15.86:0/623878590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a8806e700 0x7f4a8806eb10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4a80009d90 tx=0x7f4a8001c8e0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.565+0000 7f4a8f800640 1 -- 172.21.15.86:0/623878590 shutdown_connections 2024-04-25T14:11:14.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.565+0000 7f4a8f800640 1 --2- 172.21.15.86:0/623878590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a8806e700 0x7f4a8806eb10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.565+0000 7f4a8f800640 1 --2- 172.21.15.86:0/623878590 >> v2:172.21.15.155:3300/0 conn(0x7f4a880cdd70 0x7f4a880ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.565+0000 7f4a8f800640 1 -- 172.21.15.86:0/623878590 >> 172.21.15.86:0/623878590 conn(0x7f4a880cffc0 msgr2=0x7f4a880d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:14.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.565+0000 7f4a8f800640 1 -- 172.21.15.86:0/623878590 shutdown_connections 2024-04-25T14:11:14.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.565+0000 7f4a8f800640 1 -- 172.21.15.86:0/623878590 wait complete. 2024-04-25T14:11:14.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.569+0000 7f4a8f800640 1 Processor -- start 2024-04-25T14:11:14.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.569+0000 7f4a8f800640 1 -- start start 2024-04-25T14:11:14.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.569+0000 7f4a8f800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a8806e700 0x7f4a880b7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:14.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.569+0000 7f4a8f800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a88070e20 0x7f4a88160490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:14.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.569+0000 7f4a8f800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a880cdd70 0x7f4a881609d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:14.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.569+0000 7f4a8f800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8816cc00 con 0x7f4a88070e20 2024-04-25T14:11:14.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.569+0000 7f4a8f800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4a88164380 con 0x7f4a880cdd70 2024-04-25T14:11:14.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.569+0000 7f4a8f800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4a8816cd80 con 0x7f4a8806e700 2024-04-25T14:11:14.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.569+0000 7f4a87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a88070e20 0x7f4a88160490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:14.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.569+0000 7f4a8ce00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a8806e700 0x7f4a880b7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:14.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.569+0000 7f4a8d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a880cdd70 0x7f4a881609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:14.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.569+0000 7f4a8ce00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a8806e700 0x7f4a880b7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:58628/0 (socket says 172.21.15.86:58628) 2024-04-25T14:11:14.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.569+0000 7f4a8d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a880cdd70 0x7f4a881609d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:34648/0 (socket says 172.21.15.86:34648) 2024-04-25T14:11:14.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.569+0000 7f4a87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a88070e20 0x7f4a88160490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59682/0 (socket says 172.21.15.86:59682) 2024-04-25T14:11:14.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.569+0000 7f4a8ce00640 1 -- 172.21.15.86:0/921396170 learned_addr learned my addr 172.21.15.86:0/921396170 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:14.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.573+0000 7f4a8d800640 1 -- 172.21.15.86:0/921396170 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a8806e700 msgr2=0x7f4a880b7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:14.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.573+0000 7f4a8d800640 1 --2- 172.21.15.86:0/921396170 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a8806e700 0x7f4a880b7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.573+0000 7f4a8d800640 1 -- 172.21.15.86:0/921396170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a88070e20 msgr2=0x7f4a88160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:14.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.573+0000 7f4a8d800640 1 --2- 172.21.15.86:0/921396170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a88070e20 0x7f4a88160490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.573+0000 7f4a8d800640 1 -- 172.21.15.86:0/921396170 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a880b6200 con 0x7f4a880cdd70 2024-04-25T14:11:14.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.573+0000 7f4a8d800640 1 --2- 172.21.15.86:0/921396170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a880cdd70 0x7f4a881609d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4a800ac000 tx=0x7f4a80019680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:14.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.573+0000 7f4a84c00640 1 -- 172.21.15.86:0/921396170 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a800b2050 con 0x7f4a880cdd70 2024-04-25T14:11:14.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.573+0000 7f4a84c00640 1 -- 172.21.15.86:0/921396170 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a800b3040 con 0x7f4a880cdd70 2024-04-25T14:11:14.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.573+0000 7f4a84c00640 1 -- 172.21.15.86:0/921396170 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a800ad650 con 0x7f4a880cdd70 2024-04-25T14:11:14.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.573+0000 7f4a8f800640 1 -- 172.21.15.86:0/921396170 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a881630d0 con 0x7f4a880cdd70 2024-04-25T14:11:14.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.573+0000 7f4a8f800640 1 -- 172.21.15.86:0/921396170 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a88157970 con 0x7f4a880cdd70 2024-04-25T14:11:14.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.573+0000 7f4a8f800640 1 -- 172.21.15.86:0/921396170 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a880720e0 con 0x7f4a880cdd70 2024-04-25T14:11:14.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.573+0000 7f4a84c00640 1 -- 172.21.15.86:0/921396170 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 61) ==== 93309+0+0 (secure 0 0 0) 0x7f4a800c0020 con 0x7f4a880cdd70 2024-04-25T14:11:14.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.577+0000 7f4a84c00640 1 --2- 172.21.15.86:0/921396170 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f4a6806f520 0x7f4a68071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:14.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.577+0000 7f4a8ce00640 1 -- 172.21.15.86:0/921396170 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f4a6806f520 msgr2=0x7f4a68071a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2221276863 2024-04-25T14:11:14.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.577+0000 7f4a84c00640 1 -- 172.21.15.86:0/921396170 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6488+0+0 (secure 0 0 0) 0x7f4a80129b70 con 0x7f4a880cdd70 2024-04-25T14:11:14.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.577+0000 7f4a8ce00640 1 --2- 172.21.15.86:0/921396170 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f4a6806f520 0x7f4a68071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:11:14.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.581+0000 7f4a84c00640 1 -- 172.21.15.86:0/921396170 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4a800c7020 con 0x7f4a880cdd70 2024-04-25T14:11:14.615 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:14.617+0000 7f8b38049180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:11:14.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.753+0000 7f4a8f800640 1 -- 172.21.15.86:0/921396170 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4a8815d1e0 con 0x7f4a880cdd70 2024-04-25T14:11:14.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.753+0000 7f4a84c00640 1 -- 172.21.15.86:0/921396170 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 9 entries v128) ==== 80+0+525 (secure 0 0 0) 0x7f4a80016560 con 0x7f4a880cdd70 2024-04-25T14:11:14.751 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3961472441 2024-04-26T14:11:04.532513+0000 2024-04-25T14:11:14.751 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/287404215 2024-04-26T14:11:04.532513+0000 2024-04-25T14:11:14.751 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/889708173 2024-04-26T14:11:04.532513+0000 2024-04-25T14:11:14.751 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3224392570 2024-04-26T14:11:04.532513+0000 2024-04-25T14:11:14.751 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6833/3964666460 2024-04-26T14:11:04.532513+0000 2024-04-25T14:11:14.751 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/553620002 2024-04-26T14:11:04.532513+0000 2024-04-25T14:11:14.751 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6832/3964666460 2024-04-26T14:11:04.532513+0000 2024-04-25T14:11:14.751 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3726306847 2024-04-26T14:11:04.532513+0000 2024-04-25T14:11:14.751 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2412030790 2024-04-25T15:10:55.520329+0000 2024-04-25T14:11:14.752 INFO:teuthology.orchestra.run.smithi086.stderr:listed 9 entries 2024-04-25T14:11:14.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.757+0000 7f4a8f800640 1 -- 172.21.15.86:0/921396170 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f4a6806f520 msgr2=0x7f4a68071a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:11:14.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.757+0000 7f4a8f800640 1 --2- 172.21.15.86:0/921396170 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f4a6806f520 0x7f4a68071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.757+0000 7f4a8f800640 1 -- 172.21.15.86:0/921396170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a880cdd70 msgr2=0x7f4a881609d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:14.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.757+0000 7f4a8f800640 1 --2- 172.21.15.86:0/921396170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a880cdd70 0x7f4a881609d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4a800ac000 tx=0x7f4a80019680 comp rx=0 tx=0).stop 2024-04-25T14:11:14.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.757+0000 7f4a8f800640 1 -- 172.21.15.86:0/921396170 shutdown_connections 2024-04-25T14:11:14.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.757+0000 7f4a8f800640 1 --2- 172.21.15.86:0/921396170 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f4a6806f520 0x7f4a68071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.757+0000 7f4a8f800640 1 --2- 172.21.15.86:0/921396170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a880cdd70 0x7f4a881609d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.757+0000 7f4a8f800640 1 --2- 172.21.15.86:0/921396170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a88070e20 0x7f4a88160490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.757+0000 7f4a8f800640 1 --2- 172.21.15.86:0/921396170 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a8806e700 0x7f4a880b7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:14.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.757+0000 7f4a8f800640 1 -- 172.21.15.86:0/921396170 >> 172.21.15.86:0/921396170 conn(0x7f4a880cffc0 msgr2=0x7f4a88068880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:14.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.757+0000 7f4a8f800640 1 -- 172.21.15.86:0/921396170 shutdown_connections 2024-04-25T14:11:14.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:14.757+0000 7f4a8f800640 1 -- 172.21.15.86:0/921396170 wait complete. 2024-04-25T14:11:14.775 INFO:tasks.ceph.mgr.x.smithi155.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:11:14.775 INFO:tasks.ceph.mgr.x.smithi155.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:11:14.775 INFO:tasks.ceph.mgr.x.smithi155.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:11:14.784 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:14.786+0000 7fdf0cf7e180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:11:14.788 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T14:11:14.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:11:14.789 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T14:11:14.836 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T14:11:14.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:11:14.836 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T14:11:14.863 INFO:tasks.ceph.mgr.y.smithi086.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:11:14.863 INFO:tasks.ceph.mgr.y.smithi086.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:11:14.864 INFO:tasks.ceph.mgr.y.smithi086.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:11:14.872 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:11:14.873 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:11:14.873 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:11:14.873 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:14.877+0000 7f8b38049180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:11:14.882 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:11:14.882 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:11:14.932 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:11:14.935 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:14.934+0000 7fdf0cf7e180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:11:15.025 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:15.029+0000 7f8b38049180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:11:15.062 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:15.062+0000 7fdf0cf7e180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:11:15.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.085+0000 7fd822400640 1 Processor -- start 2024-04-25T14:11:15.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.085+0000 7fd822400640 1 -- start start 2024-04-25T14:11:15.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.085+0000 7fd822400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd81c0ccab0 0x7fd81c0d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:15.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.085+0000 7fd822400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd81c0b79c0 0x7fd81c0d2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:15.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.085+0000 7fd822400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd81c0cf9d0 0x7fd81c1617f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:15.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.085+0000 7fd822400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd81c0b2620 con 0x7fd81c0cf9d0 2024-04-25T14:11:15.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.085+0000 7fd822400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd81c0b3bd0 con 0x7fd81c0b79c0 2024-04-25T14:11:15.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.085+0000 7fd822400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd81c0b1cf0 con 0x7fd81c0ccab0 2024-04-25T14:11:15.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.085+0000 7fd81be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd81c0cf9d0 0x7fd81c1617f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:15.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.089+0000 7fd813e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd81c0b79c0 0x7fd81c0d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:15.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.089+0000 7fd81be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd81c0cf9d0 0x7fd81c1617f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59696/0 (socket says 172.21.15.86:59696) 2024-04-25T14:11:15.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.089+0000 7fd81be00640 1 -- 172.21.15.86:0/1788870904 learned_addr learned my addr 172.21.15.86:0/1788870904 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:15.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.089+0000 7fd81b400640 1 --2- 172.21.15.86:0/1788870904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd81c0ccab0 0x7fd81c0d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:15.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.089+0000 7fd81be00640 1 -- 172.21.15.86:0/1788870904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd81c0ccab0 msgr2=0x7fd81c0d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:15.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.089+0000 7fd81be00640 1 --2- 172.21.15.86:0/1788870904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd81c0ccab0 0x7fd81c0d23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.089+0000 7fd81be00640 1 -- 172.21.15.86:0/1788870904 >> v2:172.21.15.155:3300/0 conn(0x7fd81c0b79c0 msgr2=0x7fd81c0d2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:15.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.089+0000 7fd81be00640 1 --2- 172.21.15.86:0/1788870904 >> v2:172.21.15.155:3300/0 conn(0x7fd81c0b79c0 0x7fd81c0d2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.089+0000 7fd81be00640 1 -- 172.21.15.86:0/1788870904 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81c0b1fd0 con 0x7fd81c0cf9d0 2024-04-25T14:11:15.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.093+0000 7fd81be00640 1 --2- 172.21.15.86:0/1788870904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd81c0cf9d0 0x7fd81c1617f0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fd81c0b5050 tx=0x7fd81401c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278a74a4c7b690e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:15.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.093+0000 7fd81aa00640 1 -- 172.21.15.86:0/1788870904 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8140a64d0 con 0x7fd81c0cf9d0 2024-04-25T14:11:15.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.093+0000 7fd81aa00640 1 -- 172.21.15.86:0/1788870904 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8140a6670 con 0x7fd81c0cf9d0 2024-04-25T14:11:15.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.093+0000 7fd81aa00640 1 -- 172.21.15.86:0/1788870904 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8140a6a40 con 0x7fd81c0cf9d0 2024-04-25T14:11:15.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.093+0000 7fd822400640 1 -- 172.21.15.86:0/1788870904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd81c0cf9d0 msgr2=0x7fd81c1617f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:15.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.093+0000 7fd822400640 1 --2- 172.21.15.86:0/1788870904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd81c0cf9d0 0x7fd81c1617f0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fd81c0b5050 tx=0x7fd81401c730 comp rx=0 tx=0).stop 2024-04-25T14:11:15.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.093+0000 7fd822400640 1 -- 172.21.15.86:0/1788870904 shutdown_connections 2024-04-25T14:11:15.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.093+0000 7fd822400640 1 --2- 172.21.15.86:0/1788870904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd81c0cf9d0 0x7fd81c1617f0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.093+0000 7fd822400640 1 --2- 172.21.15.86:0/1788870904 >> v2:172.21.15.155:3300/0 conn(0x7fd81c0b79c0 0x7fd81c0d2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.093+0000 7fd822400640 1 --2- 172.21.15.86:0/1788870904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd81c0ccab0 0x7fd81c0d23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.093+0000 7fd822400640 1 -- 172.21.15.86:0/1788870904 >> 172.21.15.86:0/1788870904 conn(0x7fd81c09ba30 msgr2=0x7fd81c0b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:15.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.093+0000 7fd822400640 1 -- 172.21.15.86:0/1788870904 shutdown_connections 2024-04-25T14:11:15.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.093+0000 7fd822400640 1 -- 172.21.15.86:0/1788870904 wait complete. 2024-04-25T14:11:15.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.097+0000 7fd822400640 1 Processor -- start 2024-04-25T14:11:15.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.097+0000 7fd822400640 1 -- start start 2024-04-25T14:11:15.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.097+0000 7fd822400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd81c0b79c0 0x7fd81c160290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:15.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.097+0000 7fd822400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd81c0ccab0 0x7fd81c0cb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:15.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.097+0000 7fd822400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd81c0cf9d0 0x7fd81c15a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:15.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.097+0000 7fd822400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c0ce510 con 0x7fd81c0ccab0 2024-04-25T14:11:15.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.097+0000 7fd822400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c0ce390 con 0x7fd81c0cf9d0 2024-04-25T14:11:15.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.097+0000 7fd822400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd81c0ce690 con 0x7fd81c0b79c0 2024-04-25T14:11:15.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.097+0000 7fd813e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd81c0ccab0 0x7fd81c0cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:15.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.097+0000 7fd81b400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd81c0b79c0 0x7fd81c160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:15.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.097+0000 7fd81b400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd81c0b79c0 0x7fd81c160290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:58642/0 (socket says 172.21.15.86:58642) 2024-04-25T14:11:15.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.097+0000 7fd81b400640 1 -- 172.21.15.86:0/2681366171 learned_addr learned my addr 172.21.15.86:0/2681366171 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:15.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.097+0000 7fd813e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd81c0ccab0 0x7fd81c0cb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59708/0 (socket says 172.21.15.86:59708) 2024-04-25T14:11:15.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.097+0000 7fd81be00640 1 --2- 172.21.15.86:0/2681366171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd81c0cf9d0 0x7fd81c15a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:15.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.097+0000 7fd813e00640 1 -- 172.21.15.86:0/2681366171 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd81c0b79c0 msgr2=0x7fd81c160290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:15.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.097+0000 7fd813e00640 1 --2- 172.21.15.86:0/2681366171 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd81c0b79c0 0x7fd81c160290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.097+0000 7fd813e00640 1 -- 172.21.15.86:0/2681366171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd81c0cf9d0 msgr2=0x7fd81c15a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:15.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.097+0000 7fd813e00640 1 --2- 172.21.15.86:0/2681366171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd81c0cf9d0 0x7fd81c15a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.097+0000 7fd813e00640 1 -- 172.21.15.86:0/2681366171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81c0cb7b0 con 0x7fd81c0ccab0 2024-04-25T14:11:15.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.097+0000 7fd81b400640 1 --2- 172.21.15.86:0/2681366171 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd81c0b79c0 0x7fd81c160290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:11:15.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.097+0000 7fd813e00640 1 --2- 172.21.15.86:0/2681366171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd81c0ccab0 0x7fd81c0cb270 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fd80400e8a0 tx=0x7fd80400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:15.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.097+0000 7fd813400640 1 -- 172.21.15.86:0/2681366171 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804017070 con 0x7fd81c0ccab0 2024-04-25T14:11:15.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.101+0000 7fd813400640 1 -- 172.21.15.86:0/2681366171 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd804009e00 con 0x7fd81c0ccab0 2024-04-25T14:11:15.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.101+0000 7fd813400640 1 -- 172.21.15.86:0/2681366171 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80400cb40 con 0x7fd81c0ccab0 2024-04-25T14:11:15.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.101+0000 7fd822400640 1 -- 172.21.15.86:0/2681366171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd81c160c00 con 0x7fd81c0ccab0 2024-04-25T14:11:15.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.101+0000 7fd822400640 1 -- 172.21.15.86:0/2681366171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd81c0ca790 con 0x7fd81c0ccab0 2024-04-25T14:11:15.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.101+0000 7fd822400640 1 -- 172.21.15.86:0/2681366171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd81c160970 con 0x7fd81c0ccab0 2024-04-25T14:11:15.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.101+0000 7fd813400640 1 -- 172.21.15.86:0/2681366171 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 61) ==== 93309+0+0 (secure 0 0 0) 0x7fd80401a070 con 0x7fd81c0ccab0 2024-04-25T14:11:15.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.105+0000 7fd813400640 1 --2- 172.21.15.86:0/2681366171 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fd7f806f520 0x7fd7f8071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:15.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.105+0000 7fd813400640 1 -- 172.21.15.86:0/2681366171 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6488+0+0 (secure 0 0 0) 0x7fd80408f4a0 con 0x7fd81c0ccab0 2024-04-25T14:11:15.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.105+0000 7fd81b400640 1 -- 172.21.15.86:0/2681366171 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fd7f806f520 msgr2=0x7fd7f8071a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2221276863 2024-04-25T14:11:15.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.105+0000 7fd81b400640 1 --2- 172.21.15.86:0/2681366171 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fd7f806f520 0x7fd7f8071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:11:15.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.109+0000 7fd813400640 1 -- 172.21.15.86:0/2681366171 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd81c0cb7b0 con 0x7fd81c0ccab0 2024-04-25T14:11:15.133 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:15.134+0000 7fdf0cf7e180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:11:15.160 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:15.165+0000 7f8b38049180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:11:15.233 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:15.237+0000 7f8b38049180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:11:15.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.269+0000 7fd822400640 1 -- 172.21.15.86:0/2681366171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd81c0a04c0 con 0x7fd81c0ccab0 2024-04-25T14:11:15.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.269+0000 7fd813400640 1 -- 172.21.15.86:0/2681366171 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) ==== 74+0+17460 (secure 0 0 0) 0x7fd80405b920 con 0x7fd81c0ccab0 2024-04-25T14:11:15.267 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:15.267 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":128,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:11:04.532554+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:10:43.913889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T14:10:44.419362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/3961472441":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/287404215":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/889708173":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/3224392570":"2024-04-26T14:11:04.532513+0000","172.21.15.86:6833/3964666460":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/553620002":"2024-04-26T14:11:04.532513+0000","172.21.15.86:6832/3964666460":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/3726306847":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/2412030790":"2024-04-25T15:10:55.520329+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:11:15.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.273+0000 7fd822400640 1 -- 172.21.15.86:0/2681366171 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fd7f806f520 msgr2=0x7fd7f8071a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:11:15.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.273+0000 7fd822400640 1 --2- 172.21.15.86:0/2681366171 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fd7f806f520 0x7fd7f8071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.273+0000 7fd822400640 1 -- 172.21.15.86:0/2681366171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd81c0ccab0 msgr2=0x7fd81c0cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:15.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.273+0000 7fd822400640 1 --2- 172.21.15.86:0/2681366171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd81c0ccab0 0x7fd81c0cb270 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fd80400e8a0 tx=0x7fd80400ed60 comp rx=0 tx=0).stop 2024-04-25T14:11:15.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.273+0000 7fd822400640 1 -- 172.21.15.86:0/2681366171 shutdown_connections 2024-04-25T14:11:15.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.273+0000 7fd822400640 1 --2- 172.21.15.86:0/2681366171 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fd7f806f520 0x7fd7f8071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.273+0000 7fd822400640 1 --2- 172.21.15.86:0/2681366171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd81c0cf9d0 0x7fd81c15a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.273+0000 7fd822400640 1 --2- 172.21.15.86:0/2681366171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd81c0ccab0 0x7fd81c0cb270 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.273+0000 7fd822400640 1 --2- 172.21.15.86:0/2681366171 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd81c0b79c0 0x7fd81c160290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.273+0000 7fd822400640 1 -- 172.21.15.86:0/2681366171 >> 172.21.15.86:0/2681366171 conn(0x7fd81c09ba30 msgr2=0x7fd81c070fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:15.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.273+0000 7fd822400640 1 -- 172.21.15.86:0/2681366171 shutdown_connections 2024-04-25T14:11:15.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.273+0000 7fd822400640 1 -- 172.21.15.86:0/2681366171 wait complete. 2024-04-25T14:11:15.282 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:15.282+0000 7fdf0cf7e180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:11:15.290 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:10:43.913889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:10:44.419362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:11:15.291 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:11:15.388 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:15.393+0000 7f8b38049180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:11:15.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.445+0000 7ff2a3e00640 1 Processor -- start 2024-04-25T14:11:15.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.445+0000 7ff2a3e00640 1 -- start start 2024-04-25T14:11:15.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.449+0000 7ff2a3e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff29c0d2c80 0x7ff29c15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:15.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.449+0000 7ff2a3e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff29c070e20 0x7ff29c071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:15.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.449+0000 7ff2a3e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff29c071770 con 0x7ff29c0722f0 2024-04-25T14:11:15.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.449+0000 7ff2a3e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff29c0ce720 con 0x7ff29c070e20 2024-04-25T14:11:15.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.449+0000 7ff2a3e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff29c0cd8d0 con 0x7ff29c0d2c80 2024-04-25T14:11:15.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.449+0000 7ff2a1e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff29c0722f0 0x7ff29c072700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34882/0 (socket says 172.21.15.86:34882) 2024-04-25T14:11:15.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.449+0000 7ff2a1e00640 1 -- 172.21.15.86:0/1263375267 learned_addr learned my addr 172.21.15.86:0/1263375267 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:15.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.449+0000 7ff2a1400640 1 --2- 172.21.15.86:0/1263375267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff29c0d2c80 0x7ff29c15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:15.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.449+0000 7ff2a0a00640 1 --2- 172.21.15.86:0/1263375267 >> v2:172.21.15.155:3300/0 conn(0x7ff29c070e20 0x7ff29c071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:15.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.449+0000 7ff29be00640 1 -- 172.21.15.86:0/1263375267 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 219536636 0 0) 0x7ff2940014a0 con 0x7ff29c0722f0 2024-04-25T14:11:15.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.449+0000 7ff29be00640 1 -- 172.21.15.86:0/1263375267 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff284003530 con 0x7ff29c0722f0 2024-04-25T14:11:15.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.449+0000 7ff2a1400640 1 -- 172.21.15.86:0/1263375267 >> v2:172.21.15.155:3300/0 conn(0x7ff29c070e20 msgr2=0x7ff29c071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:15.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.449+0000 7ff2a1400640 1 --2- 172.21.15.86:0/1263375267 >> v2:172.21.15.155:3300/0 conn(0x7ff29c070e20 0x7ff29c071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.449+0000 7ff2a1400640 1 -- 172.21.15.86:0/1263375267 >> v1:172.21.15.86:6789/0 conn(0x7ff29c0722f0 legacy=0x7ff29c072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:15.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.449+0000 7ff2a1400640 1 -- 172.21.15.86:0/1263375267 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29c0ce970 con 0x7ff29c0d2c80 2024-04-25T14:11:15.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.449+0000 7ff2a0a00640 1 --2- 172.21.15.86:0/1263375267 >> v2:172.21.15.155:3300/0 conn(0x7ff29c070e20 0x7ff29c071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:11:15.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.453+0000 7ff2a1400640 1 --2- 172.21.15.86:0/1263375267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff29c0d2c80 0x7ff29c15dc90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff2900097c0 tx=0x7ff29001ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edfae8be2fa4472a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:15.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.453+0000 7ff29be00640 1 -- 172.21.15.86:0/1263375267 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2900b5070 con 0x7ff29c0d2c80 2024-04-25T14:11:15.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.453+0000 7ff29be00640 1 -- 172.21.15.86:0/1263375267 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2900a8990 con 0x7ff29c0d2c80 2024-04-25T14:11:15.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.453+0000 7ff29be00640 1 -- 172.21.15.86:0/1263375267 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2900a8d40 con 0x7ff29c0d2c80 2024-04-25T14:11:15.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.453+0000 7ff2a3e00640 1 -- 172.21.15.86:0/1263375267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff29c0d2c80 msgr2=0x7ff29c15dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:15.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.453+0000 7ff2a3e00640 1 --2- 172.21.15.86:0/1263375267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff29c0d2c80 0x7ff29c15dc90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff2900097c0 tx=0x7ff29001ec90 comp rx=0 tx=0).stop 2024-04-25T14:11:15.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.453+0000 7ff2a3e00640 1 -- 172.21.15.86:0/1263375267 shutdown_connections 2024-04-25T14:11:15.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.453+0000 7ff2a3e00640 1 --2- 172.21.15.86:0/1263375267 >> v2:172.21.15.155:3300/0 conn(0x7ff29c070e20 0x7ff29c071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.453+0000 7ff2a3e00640 1 --2- 172.21.15.86:0/1263375267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff29c0d2c80 0x7ff29c15dc90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.453+0000 7ff2a3e00640 1 -- 172.21.15.86:0/1263375267 >> 172.21.15.86:0/1263375267 conn(0x7ff29c0cffc0 msgr2=0x7ff29c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:15.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.453+0000 7ff2a3e00640 1 -- 172.21.15.86:0/1263375267 shutdown_connections 2024-04-25T14:11:15.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.453+0000 7ff2a3e00640 1 -- 172.21.15.86:0/1263375267 wait complete. 2024-04-25T14:11:15.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.453+0000 7ff2a3e00640 1 Processor -- start 2024-04-25T14:11:15.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.453+0000 7ff2a3e00640 1 -- start start 2024-04-25T14:11:15.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.457+0000 7ff2a3e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff29c070e20 0x7ff29c1fad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:15.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.457+0000 7ff2a3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff29c0722f0 0x7ff29c1529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:15.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.457+0000 7ff2a3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff29c0d2c80 0x7ff29c0cc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:15.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.457+0000 7ff2a3e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff29c0d21a0 con 0x7ff29c0722f0 2024-04-25T14:11:15.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.457+0000 7ff2a3e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff29c0d1ec0 con 0x7ff29c0d2c80 2024-04-25T14:11:15.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.457+0000 7ff2a3e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff29c0d2480 con 0x7ff29c070e20 2024-04-25T14:11:15.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.457+0000 7ff2a0a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff29c0722f0 0x7ff29c1529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:15.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.457+0000 7ff2a0a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff29c0722f0 0x7ff29c1529a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59720/0 (socket says 172.21.15.86:59720) 2024-04-25T14:11:15.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.457+0000 7ff2a0a00640 1 -- 172.21.15.86:0/2975411344 learned_addr learned my addr 172.21.15.86:0/2975411344 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:15.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.457+0000 7ff2a1e00640 1 --2- 172.21.15.86:0/2975411344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff29c0d2c80 0x7ff29c0cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:15.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.457+0000 7ff2a1400640 1 --2- 172.21.15.86:0/2975411344 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff29c070e20 0x7ff29c1fad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:15.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.457+0000 7ff2a0a00640 1 -- 172.21.15.86:0/2975411344 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff29c070e20 msgr2=0x7ff29c1fad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:15.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.457+0000 7ff2a0a00640 1 --2- 172.21.15.86:0/2975411344 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff29c070e20 0x7ff29c1fad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.457+0000 7ff2a0a00640 1 -- 172.21.15.86:0/2975411344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff29c0d2c80 msgr2=0x7ff29c0cc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:15.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.457+0000 7ff2a0a00640 1 --2- 172.21.15.86:0/2975411344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff29c0d2c80 0x7ff29c0cc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.457+0000 7ff2a0a00640 1 -- 172.21.15.86:0/2975411344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29c152ee0 con 0x7ff29c0722f0 2024-04-25T14:11:15.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.457+0000 7ff2a0a00640 1 --2- 172.21.15.86:0/2975411344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff29c0722f0 0x7ff29c1529a0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7ff28800e8d0 tx=0x7ff28800ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:15.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.457+0000 7ff299600640 1 -- 172.21.15.86:0/2975411344 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2880059a0 con 0x7ff29c0722f0 2024-04-25T14:11:15.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.457+0000 7ff299600640 1 -- 172.21.15.86:0/2975411344 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff288005b40 con 0x7ff29c0722f0 2024-04-25T14:11:15.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.457+0000 7ff2a3e00640 1 -- 172.21.15.86:0/2975411344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff29c14f820 con 0x7ff29c0722f0 2024-04-25T14:11:15.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.457+0000 7ff299600640 1 -- 172.21.15.86:0/2975411344 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28800c940 con 0x7ff29c0722f0 2024-04-25T14:11:15.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.457+0000 7ff2a3e00640 1 -- 172.21.15.86:0/2975411344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff29c14fde0 con 0x7ff29c0722f0 2024-04-25T14:11:15.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.457+0000 7ff2a3e00640 1 -- 172.21.15.86:0/2975411344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff29c0720e0 con 0x7ff29c0722f0 2024-04-25T14:11:15.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.461+0000 7ff299600640 1 -- 172.21.15.86:0/2975411344 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 61) ==== 93309+0+0 (secure 0 0 0) 0x7ff28800cae0 con 0x7ff29c0722f0 2024-04-25T14:11:15.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.461+0000 7ff299600640 1 --2- 172.21.15.86:0/2975411344 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7ff27c06f520 0x7ff27c071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:15.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.461+0000 7ff2a1400640 1 -- 172.21.15.86:0/2975411344 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7ff27c06f520 msgr2=0x7ff27c071a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2221276863 2024-04-25T14:11:15.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.461+0000 7ff299600640 1 -- 172.21.15.86:0/2975411344 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6488+0+0 (secure 0 0 0) 0x7ff28808f9d0 con 0x7ff29c0722f0 2024-04-25T14:11:15.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.461+0000 7ff2a1400640 1 --2- 172.21.15.86:0/2975411344 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7ff27c06f520 0x7ff27c071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:11:15.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.465+0000 7ff299600640 1 -- 172.21.15.86:0/2975411344 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff288017020 con 0x7ff29c0722f0 2024-04-25T14:11:15.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.625+0000 7ff2a3e00640 1 -- 172.21.15.86:0/2975411344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff29c161420 con 0x7ff29c0722f0 2024-04-25T14:11:15.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.625+0000 7ff299600640 1 -- 172.21.15.86:0/2975411344 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) ==== 93+0+10 (secure 0 0 0) 0x7ff28805be40 con 0x7ff29c0722f0 2024-04-25T14:11:15.623 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:11:15.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.629+0000 7ff2a3e00640 1 -- 172.21.15.86:0/2975411344 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7ff27c06f520 msgr2=0x7ff27c071a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:11:15.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.629+0000 7ff2a3e00640 1 --2- 172.21.15.86:0/2975411344 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7ff27c06f520 0x7ff27c071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.629+0000 7ff2a3e00640 1 -- 172.21.15.86:0/2975411344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff29c0722f0 msgr2=0x7ff29c1529a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:15.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.629+0000 7ff2a3e00640 1 --2- 172.21.15.86:0/2975411344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff29c0722f0 0x7ff29c1529a0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7ff28800e8d0 tx=0x7ff28800ed90 comp rx=0 tx=0).stop 2024-04-25T14:11:15.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.629+0000 7ff2a3e00640 1 -- 172.21.15.86:0/2975411344 shutdown_connections 2024-04-25T14:11:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.629+0000 7ff2a3e00640 1 --2- 172.21.15.86:0/2975411344 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7ff27c06f520 0x7ff27c071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.629+0000 7ff2a3e00640 1 --2- 172.21.15.86:0/2975411344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff29c0d2c80 0x7ff29c0cc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.629+0000 7ff2a3e00640 1 --2- 172.21.15.86:0/2975411344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff29c0722f0 0x7ff29c1529a0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.629+0000 7ff2a3e00640 1 --2- 172.21.15.86:0/2975411344 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff29c070e20 0x7ff29c1fad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.629+0000 7ff2a3e00640 1 -- 172.21.15.86:0/2975411344 >> 172.21.15.86:0/2975411344 conn(0x7ff29c0cffc0 msgr2=0x7ff29c15bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.629+0000 7ff2a3e00640 1 -- 172.21.15.86:0/2975411344 shutdown_connections 2024-04-25T14:11:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.629+0000 7ff2a3e00640 1 -- 172.21.15.86:0/2975411344 wait complete. 2024-04-25T14:11:15.646 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:11:15.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.805+0000 7f5015c00640 1 Processor -- start 2024-04-25T14:11:15.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.805+0000 7f5015c00640 1 -- start start 2024-04-25T14:11:15.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.805+0000 7f5015c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5010071530 0x7f50100ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:15.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.805+0000 7f5015c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f501006e700 0x7f501006eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:15.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.805+0000 7f5015c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f501006f050 con 0x7f50100cdd70 2024-04-25T14:11:15.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.805+0000 7f5015c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f501006d940 con 0x7f501006e700 2024-04-25T14:11:15.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.805+0000 7f5015c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f501006dc20 con 0x7f5010071530 2024-04-25T14:11:15.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.805+0000 7f500ea00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f50100cdd70 0x7f5010070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34884/0 (socket says 172.21.15.86:34884) 2024-04-25T14:11:15.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.805+0000 7f500ea00640 1 -- 172.21.15.86:0/699241735 learned_addr learned my addr 172.21.15.86:0/699241735 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:15.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.805+0000 7f500f400640 1 --2- 172.21.15.86:0/699241735 >> v2:172.21.15.86:3300/0 conn(0x7f501006e700 0x7f501006eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:15.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.805+0000 7f500e000640 1 --2- 172.21.15.86:0/699241735 >> v2:172.21.15.155:3300/0 conn(0x7f5010071530 0x7f50100ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:15.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.805+0000 7f500d600640 1 -- 172.21.15.86:0/699241735 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2074085468 0 0) 0x7f501006f050 con 0x7f50100cdd70 2024-04-25T14:11:15.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.805+0000 7f500d600640 1 -- 172.21.15.86:0/699241735 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ff4003530 con 0x7f50100cdd70 2024-04-25T14:11:15.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.809+0000 7f500f400640 1 -- 172.21.15.86:0/699241735 >> v2:172.21.15.155:3300/0 conn(0x7f5010071530 msgr2=0x7f50100ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:15.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.809+0000 7f500f400640 1 --2- 172.21.15.86:0/699241735 >> v2:172.21.15.155:3300/0 conn(0x7f5010071530 0x7f50100ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.809+0000 7f500f400640 1 -- 172.21.15.86:0/699241735 >> v1:172.21.15.86:6789/0 conn(0x7f50100cdd70 legacy=0x7f5010070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:15.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.809+0000 7f500f400640 1 -- 172.21.15.86:0/699241735 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f501006df00 con 0x7f501006e700 2024-04-25T14:11:15.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.809+0000 7f500e000640 1 --2- 172.21.15.86:0/699241735 >> v2:172.21.15.155:3300/0 conn(0x7f5010071530 0x7f50100ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:11:15.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.809+0000 7f500f400640 1 --2- 172.21.15.86:0/699241735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f501006e700 0x7f501006eb10 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f5010073230 tx=0x7f500801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222de54c5d9ad253 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:15.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.809+0000 7f500d600640 1 -- 172.21.15.86:0/699241735 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50080b2070 con 0x7f501006e700 2024-04-25T14:11:15.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.809+0000 7f500d600640 1 -- 172.21.15.86:0/699241735 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50080a57b0 con 0x7f501006e700 2024-04-25T14:11:15.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.809+0000 7f500d600640 1 -- 172.21.15.86:0/699241735 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50080a5b80 con 0x7f501006e700 2024-04-25T14:11:15.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.809+0000 7f5015c00640 1 -- 172.21.15.86:0/699241735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f501006e700 msgr2=0x7f501006eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:15.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.809+0000 7f5015c00640 1 --2- 172.21.15.86:0/699241735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f501006e700 0x7f501006eb10 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f5010073230 tx=0x7f500801c8a0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.813+0000 7f5015c00640 1 -- 172.21.15.86:0/699241735 shutdown_connections 2024-04-25T14:11:15.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.813+0000 7f5015c00640 1 --2- 172.21.15.86:0/699241735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f501006e700 0x7f501006eb10 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.813+0000 7f5015c00640 1 --2- 172.21.15.86:0/699241735 >> v2:172.21.15.155:3300/0 conn(0x7f5010071530 0x7f50100ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.813+0000 7f5015c00640 1 -- 172.21.15.86:0/699241735 >> 172.21.15.86:0/699241735 conn(0x7f50100cffc0 msgr2=0x7f50100d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:15.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.813+0000 7f5015c00640 1 -- 172.21.15.86:0/699241735 shutdown_connections 2024-04-25T14:11:15.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.813+0000 7f5015c00640 1 -- 172.21.15.86:0/699241735 wait complete. 2024-04-25T14:11:15.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.813+0000 7f5015c00640 1 Processor -- start 2024-04-25T14:11:15.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.813+0000 7f5015c00640 1 -- start start 2024-04-25T14:11:15.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.813+0000 7f5015c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f501006e700 0x7f50100c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:15.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.813+0000 7f5015c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50100cdd70 0x7f5010153870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:15.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.813+0000 7f5015c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50100b7ab0 0x7f50100b7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:15.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.813+0000 7f5015c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f501016cc00 con 0x7f501006e700 2024-04-25T14:11:15.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.813+0000 7f5015c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5010164380 con 0x7f50100cdd70 2024-04-25T14:11:15.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.813+0000 7f5015c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f501016cd80 con 0x7f50100b7ab0 2024-04-25T14:11:15.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.817+0000 7f500ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f501006e700 0x7f50100c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:15.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.817+0000 7f500ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f501006e700 0x7f50100c91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59748/0 (socket says 172.21.15.86:59748) 2024-04-25T14:11:15.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.817+0000 7f500ea00640 1 -- 172.21.15.86:0/802531055 learned_addr learned my addr 172.21.15.86:0/802531055 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:15.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.817+0000 7f500e000640 1 --2- 172.21.15.86:0/802531055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50100cdd70 0x7f5010153870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:15.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.817+0000 7f500f400640 1 --2- 172.21.15.86:0/802531055 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50100b7ab0 0x7f50100b7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:15.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.817+0000 7f500ea00640 1 -- 172.21.15.86:0/802531055 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50100b7ab0 msgr2=0x7f50100b7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:15.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.817+0000 7f500ea00640 1 --2- 172.21.15.86:0/802531055 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50100b7ab0 0x7f50100b7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.817+0000 7f500ea00640 1 -- 172.21.15.86:0/802531055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50100cdd70 msgr2=0x7f5010153870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:15.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.817+0000 7f500ea00640 1 --2- 172.21.15.86:0/802531055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50100cdd70 0x7f5010153870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.817+0000 7f500ea00640 1 -- 172.21.15.86:0/802531055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50100b8400 con 0x7f501006e700 2024-04-25T14:11:15.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.817+0000 7f500ea00640 1 --2- 172.21.15.86:0/802531055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f501006e700 0x7f50100c91d0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f500000c3f0 tx=0x7f500000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:15.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.817+0000 7f5006a00640 1 -- 172.21.15.86:0/802531055 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5000017070 con 0x7f501006e700 2024-04-25T14:11:15.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.817+0000 7f5006a00640 1 -- 172.21.15.86:0/802531055 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5000005cf0 con 0x7f501006e700 2024-04-25T14:11:15.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.817+0000 7f5015c00640 1 -- 172.21.15.86:0/802531055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50101602e0 con 0x7f501006e700 2024-04-25T14:11:15.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.817+0000 7f5006a00640 1 -- 172.21.15.86:0/802531055 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50000072f0 con 0x7f501006e700 2024-04-25T14:11:15.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.817+0000 7f5015c00640 1 -- 172.21.15.86:0/802531055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50100c9c20 con 0x7f501006e700 2024-04-25T14:11:15.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.817+0000 7f5015c00640 1 -- 172.21.15.86:0/802531055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50100720e0 con 0x7f501006e700 2024-04-25T14:11:15.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.817+0000 7f5006a00640 1 -- 172.21.15.86:0/802531055 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 61) ==== 93309+0+0 (secure 0 0 0) 0x7f500001a070 con 0x7f501006e700 2024-04-25T14:11:15.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.821+0000 7f5006a00640 1 --2- 172.21.15.86:0/802531055 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f4ff006f550 0x7f4ff0071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:15.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.821+0000 7f5006a00640 1 -- 172.21.15.86:0/802531055 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6488+0+0 (secure 0 0 0) 0x7f500008eab0 con 0x7f501006e700 2024-04-25T14:11:15.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.821+0000 7f500e000640 1 -- 172.21.15.86:0/802531055 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f4ff006f550 msgr2=0x7f4ff0071a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2221276863 2024-04-25T14:11:15.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.821+0000 7f500e000640 1 --2- 172.21.15.86:0/802531055 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f4ff006f550 0x7f4ff0071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:11:15.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.829+0000 7f5006a00640 1 -- 172.21.15.86:0/802531055 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f50100b8400 con 0x7f501006e700 2024-04-25T14:11:15.833 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:15.834+0000 7fdf0cf7e180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:11:15.917 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:15.918+0000 7fdf0cf7e180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:11:15.960 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:15.965+0000 7f8b38049180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:11:15.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.989+0000 7f5015c00640 1 -- 172.21.15.86:0/802531055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f501006f770 con 0x7f501006e700 2024-04-25T14:11:15.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.989+0000 7f5006a00640 1 -- 172.21.15.86:0/802531055 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v128) ==== 104+0+11 (secure 0 0 0) 0x7f500005af30 con 0x7f501006e700 2024-04-25T14:11:15.986 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:11:15.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.989+0000 7f5015c00640 1 -- 172.21.15.86:0/802531055 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f4ff006f550 msgr2=0x7f4ff0071a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:11:15.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.989+0000 7f5015c00640 1 --2- 172.21.15.86:0/802531055 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f4ff006f550 0x7f4ff0071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.989+0000 7f5015c00640 1 -- 172.21.15.86:0/802531055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f501006e700 msgr2=0x7f50100c91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:15.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.989+0000 7f5015c00640 1 --2- 172.21.15.86:0/802531055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f501006e700 0x7f50100c91d0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f500000c3f0 tx=0x7f500000c8b0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.989+0000 7f5015c00640 1 -- 172.21.15.86:0/802531055 shutdown_connections 2024-04-25T14:11:15.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.989+0000 7f5015c00640 1 --2- 172.21.15.86:0/802531055 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f4ff006f550 0x7f4ff0071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.989+0000 7f5015c00640 1 --2- 172.21.15.86:0/802531055 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50100b7ab0 0x7f50100b7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.989+0000 7f5015c00640 1 --2- 172.21.15.86:0/802531055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50100cdd70 0x7f5010153870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.989+0000 7f5015c00640 1 --2- 172.21.15.86:0/802531055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f501006e700 0x7f50100c91d0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:15.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.989+0000 7f5015c00640 1 -- 172.21.15.86:0/802531055 >> 172.21.15.86:0/802531055 conn(0x7f50100cffc0 msgr2=0x7f501015ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:15.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.989+0000 7f5015c00640 1 -- 172.21.15.86:0/802531055 shutdown_connections 2024-04-25T14:11:15.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:15.989+0000 7f5015c00640 1 -- 172.21.15.86:0/802531055 wait complete. 2024-04-25T14:11:16.009 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:11:16.050 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:16.053+0000 7f8b38049180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:11:16.078 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:16.078+0000 7fdf0cf7e180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:11:16.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.165+0000 7fc731a00640 1 Processor -- start 2024-04-25T14:11:16.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.165+0000 7fc731a00640 1 -- start start 2024-04-25T14:11:16.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.165+0000 7fc731a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc72c0cdd70 0x7fc72c0ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:16.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.165+0000 7fc731a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc72c070e20 0x7fc72c071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:16.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.165+0000 7fc731a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc72c071770 con 0x7fc72c06e700 2024-04-25T14:11:16.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.165+0000 7fc731a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc72c06d940 con 0x7fc72c0cdd70 2024-04-25T14:11:16.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.165+0000 7fc731a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc72c06dc20 con 0x7fc72c070e20 2024-04-25T14:11:16.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.165+0000 7fc72aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc72c0cdd70 0x7fc72c0ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:16.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.165+0000 7fc723e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc72c070e20 0x7fc72c071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:16.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.165+0000 7fc72b400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc72c06e700 0x7fc72c06eb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34890/0 (socket says 172.21.15.86:34890) 2024-04-25T14:11:16.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.165+0000 7fc72aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc72c0cdd70 0x7fc72c0ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59752/0 (socket says 172.21.15.86:59752) 2024-04-25T14:11:16.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.165+0000 7fc72b400640 1 -- 172.21.15.86:0/4171007292 learned_addr learned my addr 172.21.15.86:0/4171007292 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:16.164 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:16.162+0000 7fdf0cf7e180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:11:16.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.165+0000 7fc72aa00640 1 -- 172.21.15.86:0/4171007292 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc72c070e20 msgr2=0x7fc72c071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:16.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.165+0000 7fc72aa00640 1 --2- 172.21.15.86:0/4171007292 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc72c070e20 0x7fc72c071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.165+0000 7fc72aa00640 1 -- 172.21.15.86:0/4171007292 >> v1:172.21.15.86:6789/0 conn(0x7fc72c06e700 legacy=0x7fc72c06eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:16.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.165+0000 7fc72aa00640 1 -- 172.21.15.86:0/4171007292 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc72c06df00 con 0x7fc72c0cdd70 2024-04-25T14:11:16.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.169+0000 7fc72aa00640 1 --2- 172.21.15.86:0/4171007292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c0cdd70 0x7fc72c0ce180 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fc71c0097c0 tx=0x7fc71c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3867e1197672114 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:16.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.169+0000 7fc72a000640 1 -- 172.21.15.86:0/4171007292 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c0b0070 con 0x7fc72c0cdd70 2024-04-25T14:11:16.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.169+0000 7fc72a000640 1 -- 172.21.15.86:0/4171007292 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc71c0ab600 con 0x7fc72c0cdd70 2024-04-25T14:11:16.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.169+0000 7fc72a000640 1 -- 172.21.15.86:0/4171007292 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c0ab9d0 con 0x7fc72c0cdd70 2024-04-25T14:11:16.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.169+0000 7fc731a00640 1 -- 172.21.15.86:0/4171007292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c0cdd70 msgr2=0x7fc72c0ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:16.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.169+0000 7fc731a00640 1 --2- 172.21.15.86:0/4171007292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c0cdd70 0x7fc72c0ce180 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fc71c0097c0 tx=0x7fc71c01c410 comp rx=0 tx=0).stop 2024-04-25T14:11:16.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.169+0000 7fc731a00640 1 -- 172.21.15.86:0/4171007292 shutdown_connections 2024-04-25T14:11:16.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.169+0000 7fc731a00640 1 --2- 172.21.15.86:0/4171007292 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc72c070e20 0x7fc72c071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.169+0000 7fc731a00640 1 --2- 172.21.15.86:0/4171007292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c0cdd70 0x7fc72c0ce180 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.169+0000 7fc731a00640 1 -- 172.21.15.86:0/4171007292 >> 172.21.15.86:0/4171007292 conn(0x7fc72c0cffc0 msgr2=0x7fc72c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:16.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.169+0000 7fc731a00640 1 -- 172.21.15.86:0/4171007292 shutdown_connections 2024-04-25T14:11:16.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.169+0000 7fc731a00640 1 -- 172.21.15.86:0/4171007292 wait complete. 2024-04-25T14:11:16.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.173+0000 7fc731a00640 1 Processor -- start 2024-04-25T14:11:16.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.173+0000 7fc731a00640 1 -- start start 2024-04-25T14:11:16.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.173+0000 7fc731a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc72c06e700 0x7fc72c0ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:16.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.173+0000 7fc731a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc72c070e20 0x7fc72c1539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:16.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.173+0000 7fc731a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c0cdd70 0x7fc72c0cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:16.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.173+0000 7fc731a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c16cc00 con 0x7fc72c0cdd70 2024-04-25T14:11:16.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.173+0000 7fc731a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c164380 con 0x7fc72c070e20 2024-04-25T14:11:16.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.173+0000 7fc731a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc72c16cd80 con 0x7fc72c06e700 2024-04-25T14:11:16.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.173+0000 7fc72b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c0cdd70 0x7fc72c0cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:16.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.173+0000 7fc72b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c0cdd70 0x7fc72c0cb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59758/0 (socket says 172.21.15.86:59758) 2024-04-25T14:11:16.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.173+0000 7fc72b400640 1 -- 172.21.15.86:0/1465075989 learned_addr learned my addr 172.21.15.86:0/1465075989 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:16.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.173+0000 7fc723e00640 1 --2- 172.21.15.86:0/1465075989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc72c070e20 0x7fc72c1539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:16.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.173+0000 7fc72aa00640 1 --2- 172.21.15.86:0/1465075989 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc72c06e700 0x7fc72c0ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:16.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.173+0000 7fc72b400640 1 -- 172.21.15.86:0/1465075989 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc72c06e700 msgr2=0x7fc72c0ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:16.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.173+0000 7fc72b400640 1 --2- 172.21.15.86:0/1465075989 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc72c06e700 0x7fc72c0ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.173+0000 7fc72b400640 1 -- 172.21.15.86:0/1465075989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc72c070e20 msgr2=0x7fc72c1539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:16.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.173+0000 7fc72b400640 1 --2- 172.21.15.86:0/1465075989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc72c070e20 0x7fc72c1539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.173+0000 7fc72b400640 1 -- 172.21.15.86:0/1465075989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc72c0b2da0 con 0x7fc72c0cdd70 2024-04-25T14:11:16.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.173+0000 7fc72b400640 1 --2- 172.21.15.86:0/1465075989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c0cdd70 0x7fc72c0cb0a0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fc72400c9f0 tx=0x7fc72400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:16.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.173+0000 7fc722a00640 1 -- 172.21.15.86:0/1465075989 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724017070 con 0x7fc72c0cdd70 2024-04-25T14:11:16.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.177+0000 7fc722a00640 1 -- 172.21.15.86:0/1465075989 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc72400ab70 con 0x7fc72c0cdd70 2024-04-25T14:11:16.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.177+0000 7fc722a00640 1 -- 172.21.15.86:0/1465075989 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724012400 con 0x7fc72c0cdd70 2024-04-25T14:11:16.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.177+0000 7fc731a00640 1 -- 172.21.15.86:0/1465075989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc72c162e60 con 0x7fc72c0cdd70 2024-04-25T14:11:16.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.177+0000 7fc731a00640 1 -- 172.21.15.86:0/1465075989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc72c1625c0 con 0x7fc72c0cdd70 2024-04-25T14:11:16.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.177+0000 7fc731a00640 1 -- 172.21.15.86:0/1465075989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc72c0720e0 con 0x7fc72c0cdd70 2024-04-25T14:11:16.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.177+0000 7fc722a00640 1 -- 172.21.15.86:0/1465075989 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 61) ==== 93309+0+0 (secure 0 0 0) 0x7fc72401a070 con 0x7fc72c0cdd70 2024-04-25T14:11:16.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.181+0000 7fc722a00640 1 --2- 172.21.15.86:0/1465075989 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fc70806f520 0x7fc708071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:16.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.181+0000 7fc722a00640 1 -- 172.21.15.86:0/1465075989 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6488+0+0 (secure 0 0 0) 0x7fc72408f650 con 0x7fc72c0cdd70 2024-04-25T14:11:16.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.181+0000 7fc72aa00640 1 -- 172.21.15.86:0/1465075989 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fc70806f520 msgr2=0x7fc708071a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2221276863 2024-04-25T14:11:16.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.181+0000 7fc72aa00640 1 --2- 172.21.15.86:0/1465075989 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fc70806f520 0x7fc708071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:11:16.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.181+0000 7fc722a00640 1 -- 172.21.15.86:0/1465075989 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc72c071770 con 0x7fc72c0cdd70 2024-04-25T14:11:16.219 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:16.221+0000 7f8b38049180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:11:16.304 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:16.309+0000 7f8b38049180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:11:16.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.341+0000 7fc731a00640 1 -- 172.21.15.86:0/1465075989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc72c154f60 con 0x7fc72c0cdd70 2024-04-25T14:11:16.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.341+0000 7fc722a00640 1 -- 172.21.15.86:0/1465075989 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v128) ==== 100+0+11 (secure 0 0 0) 0x7fc724064280 con 0x7fc72c0cdd70 2024-04-25T14:11:16.338 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:11:16.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.341+0000 7fc731a00640 1 -- 172.21.15.86:0/1465075989 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fc70806f520 msgr2=0x7fc708071a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:11:16.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.341+0000 7fc731a00640 1 --2- 172.21.15.86:0/1465075989 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fc70806f520 0x7fc708071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.341+0000 7fc731a00640 1 -- 172.21.15.86:0/1465075989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c0cdd70 msgr2=0x7fc72c0cb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:16.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.341+0000 7fc731a00640 1 --2- 172.21.15.86:0/1465075989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c0cdd70 0x7fc72c0cb0a0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fc72400c9f0 tx=0x7fc72400ceb0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.341+0000 7fc731a00640 1 -- 172.21.15.86:0/1465075989 shutdown_connections 2024-04-25T14:11:16.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.341+0000 7fc731a00640 1 --2- 172.21.15.86:0/1465075989 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7fc70806f520 0x7fc708071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.341+0000 7fc731a00640 1 --2- 172.21.15.86:0/1465075989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc72c0cdd70 0x7fc72c0cb0a0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.341+0000 7fc731a00640 1 --2- 172.21.15.86:0/1465075989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc72c070e20 0x7fc72c1539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.341+0000 7fc731a00640 1 --2- 172.21.15.86:0/1465075989 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc72c06e700 0x7fc72c0ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.341+0000 7fc731a00640 1 -- 172.21.15.86:0/1465075989 >> 172.21.15.86:0/1465075989 conn(0x7fc72c0cffc0 msgr2=0x7fc72c1569d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:16.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.341+0000 7fc731a00640 1 -- 172.21.15.86:0/1465075989 shutdown_connections 2024-04-25T14:11:16.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.341+0000 7fc731a00640 1 -- 172.21.15.86:0/1465075989 wait complete. 2024-04-25T14:11:16.360 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T14:11:16.481 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:16.482+0000 7fdf0cf7e180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:11:16.543 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:16.542+0000 7fdf0cf7e180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:11:16.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.553+0000 7f1884400640 1 Processor -- start 2024-04-25T14:11:16.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.553+0000 7f1884400640 1 -- start start 2024-04-25T14:11:16.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.553+0000 7f1884400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f187c071530 0x7f187c15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:16.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.553+0000 7f1884400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f187c0cd8d0 con 0x7f187c0d2c80 2024-04-25T14:11:16.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.553+0000 7f1884400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f187c0ce970 con 0x7f187c1616e0 2024-04-25T14:11:16.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.553+0000 7f1884400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f187c0ce720 con 0x7f187c071530 2024-04-25T14:11:16.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.553+0000 7f1881a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f187c0d2c80 0x7f187c070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34904/0 (socket says 172.21.15.86:34904) 2024-04-25T14:11:16.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.553+0000 7f1881a00640 1 -- 172.21.15.86:0/399857325 learned_addr learned my addr 172.21.15.86:0/399857325 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:16.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.553+0000 7f1881000640 1 --2- 172.21.15.86:0/399857325 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f187c071530 0x7f187c15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:16.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.557+0000 7f187be00640 1 -- 172.21.15.86:0/399857325 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 735092160 0 0) 0x7f187c0cd8d0 con 0x7f187c0d2c80 2024-04-25T14:11:16.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.557+0000 7f187be00640 1 -- 172.21.15.86:0/399857325 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1864003530 con 0x7f187c0d2c80 2024-04-25T14:11:16.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.557+0000 7f187be00640 1 -- 172.21.15.86:0/399857325 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4027700499 0 0) 0x7f187c0ce970 con 0x7f187c1616e0 2024-04-25T14:11:16.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.557+0000 7f187be00640 1 -- 172.21.15.86:0/399857325 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f187c0cd8d0 con 0x7f187c1616e0 2024-04-25T14:11:16.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.557+0000 7f1881000640 1 -- 172.21.15.86:0/399857325 >> v1:172.21.15.155:6789/0 conn(0x7f187c1616e0 legacy=0x7f187c161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:16.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.557+0000 7f1881000640 1 -- 172.21.15.86:0/399857325 >> v1:172.21.15.86:6789/0 conn(0x7f187c0d2c80 legacy=0x7f187c070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:16.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.557+0000 7f1881000640 1 -- 172.21.15.86:0/399857325 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187c0d1be0 con 0x7f187c071530 2024-04-25T14:11:16.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.557+0000 7f1881000640 1 --2- 172.21.15.86:0/399857325 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f187c071530 0x7f187c15dc90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f18680097d0 tx=0x7f186801ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb2e019a4621a56a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:16.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.557+0000 7f187be00640 1 -- 172.21.15.86:0/399857325 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18680b5070 con 0x7f187c071530 2024-04-25T14:11:16.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.557+0000 7f187be00640 1 -- 172.21.15.86:0/399857325 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18680a89b0 con 0x7f187c071530 2024-04-25T14:11:16.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.557+0000 7f187be00640 1 -- 172.21.15.86:0/399857325 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18680a8d80 con 0x7f187c071530 2024-04-25T14:11:16.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.561+0000 7f1884400640 1 -- 172.21.15.86:0/399857325 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f187c071530 msgr2=0x7f187c15dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:16.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.561+0000 7f1884400640 1 --2- 172.21.15.86:0/399857325 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f187c071530 0x7f187c15dc90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f18680097d0 tx=0x7f186801ecb0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.561+0000 7f1884400640 1 -- 172.21.15.86:0/399857325 shutdown_connections 2024-04-25T14:11:16.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.561+0000 7f1884400640 1 --2- 172.21.15.86:0/399857325 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f187c071530 0x7f187c15dc90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.561+0000 7f1884400640 1 -- 172.21.15.86:0/399857325 >> 172.21.15.86:0/399857325 conn(0x7f187c0cffc0 msgr2=0x7f187c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:16.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.561+0000 7f1884400640 1 -- 172.21.15.86:0/399857325 shutdown_connections 2024-04-25T14:11:16.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.561+0000 7f1884400640 1 -- 172.21.15.86:0/399857325 wait complete. 2024-04-25T14:11:16.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.561+0000 7f1884400640 1 Processor -- start 2024-04-25T14:11:16.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.561+0000 7f1884400640 1 -- start start 2024-04-25T14:11:16.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.565+0000 7f1884400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f187c0d2c80 0x7f187c1473f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:16.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.565+0000 7f1884400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f187c1616e0 0x7f187c15c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:16.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.565+0000 7f1884400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f187c1556f0 0x7f187c157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:16.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.565+0000 7f1884400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f187c0d2480 con 0x7f187c0d2c80 2024-04-25T14:11:16.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.565+0000 7f1884400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f187c0d21a0 con 0x7f187c1616e0 2024-04-25T14:11:16.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.565+0000 7f1884400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f187c0d29a0 con 0x7f187c1556f0 2024-04-25T14:11:16.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.565+0000 7f1881a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f187c0d2c80 0x7f187c1473f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:16.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.565+0000 7f1881a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f187c0d2c80 0x7f187c1473f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59768/0 (socket says 172.21.15.86:59768) 2024-04-25T14:11:16.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.565+0000 7f1881a00640 1 -- 172.21.15.86:0/2909920196 learned_addr learned my addr 172.21.15.86:0/2909920196 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:16.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.565+0000 7f1881000640 1 --2- 172.21.15.86:0/2909920196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f187c1616e0 0x7f187c15c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:16.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.565+0000 7f1882400640 1 --2- 172.21.15.86:0/2909920196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f187c1556f0 0x7f187c157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:16.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.565+0000 7f1881000640 1 -- 172.21.15.86:0/2909920196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f187c1556f0 msgr2=0x7f187c157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:16.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.565+0000 7f1881000640 1 --2- 172.21.15.86:0/2909920196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f187c1556f0 0x7f187c157300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.565+0000 7f1881000640 1 -- 172.21.15.86:0/2909920196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f187c0d2c80 msgr2=0x7f187c1473f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:16.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.565+0000 7f1881000640 1 --2- 172.21.15.86:0/2909920196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f187c0d2c80 0x7f187c1473f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.565+0000 7f1881000640 1 -- 172.21.15.86:0/2909920196 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187c156260 con 0x7f187c1616e0 2024-04-25T14:11:16.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.565+0000 7f1881000640 1 --2- 172.21.15.86:0/2909920196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f187c1616e0 0x7f187c15c9f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f187c0733a0 tx=0x7f186801ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:16.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.565+0000 7f1879600640 1 -- 172.21.15.86:0/2909920196 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18680be070 con 0x7f187c1616e0 2024-04-25T14:11:16.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.565+0000 7f1879600640 1 -- 172.21.15.86:0/2909920196 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1868005bc0 con 0x7f187c1616e0 2024-04-25T14:11:16.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.565+0000 7f1884400640 1 -- 172.21.15.86:0/2909920196 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f187c156540 con 0x7f187c1616e0 2024-04-25T14:11:16.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.565+0000 7f1879600640 1 -- 172.21.15.86:0/2909920196 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18680b5040 con 0x7f187c1616e0 2024-04-25T14:11:16.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.565+0000 7f1884400640 1 -- 172.21.15.86:0/2909920196 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f187c156b00 con 0x7f187c1616e0 2024-04-25T14:11:16.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.565+0000 7f1884400640 1 -- 172.21.15.86:0/2909920196 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f187c076bf0 con 0x7f187c1616e0 2024-04-25T14:11:16.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.569+0000 7f1879600640 1 -- 172.21.15.86:0/2909920196 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 61) ==== 93309+0+0 (secure 0 0 0) 0x7f1868006030 con 0x7f187c1616e0 2024-04-25T14:11:16.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.569+0000 7f1879600640 1 --2- 172.21.15.86:0/2909920196 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f185c06f550 0x7f185c071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:16.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.569+0000 7f1881a00640 1 -- 172.21.15.86:0/2909920196 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f185c06f550 msgr2=0x7f185c071a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2221276863 2024-04-25T14:11:16.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.569+0000 7f1879600640 1 -- 172.21.15.86:0/2909920196 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6488+0+0 (secure 0 0 0) 0x7f186812bbf0 con 0x7f187c1616e0 2024-04-25T14:11:16.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.569+0000 7f1881a00640 1 --2- 172.21.15.86:0/2909920196 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f185c06f550 0x7f185c071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:11:16.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.573+0000 7f1879600640 1 -- 172.21.15.86:0/2909920196 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1868018810 con 0x7f187c1616e0 2024-04-25T14:11:16.639 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:16.641+0000 7f8b38049180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:11:16.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.641+0000 7f1879600640 1 -- 172.21.15.86:0/2909920196 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mgrmap(e 62) ==== 93309+0+0 (secure 0 0 0) 0x7f18680f42e0 con 0x7f187c1616e0 2024-04-25T14:11:16.703 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:16.705+0000 7f8b38049180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:11:16.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.721+0000 7f1879600640 1 -- 172.21.15.86:0/2909920196 <== mon.1 v2:172.21.15.155:3300/0 8 ==== mgrmap(e 63) ==== 92918+0+0 (secure 0 0 0) 0x7f18680dab10 con 0x7f187c1616e0 2024-04-25T14:11:16.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.721+0000 7f1879600640 1 -- 172.21.15.86:0/2909920196 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f185c06f550 msgr2=0x7f185c071a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:11:16.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.721+0000 7f1879600640 1 --2- 172.21.15.86:0/2909920196 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f185c06f550 0x7f185c071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.769+0000 7f1884400640 1 -- 172.21.15.86:0/2909920196 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f187c0d1470 con 0x7f187c1616e0 2024-04-25T14:11:16.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.769+0000 7f1879600640 1 -- 172.21.15.86:0/2909920196 <== mon.1 v2:172.21.15.155:3300/0 9 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v129) ==== 81+0+994 (secure 0 0 0) 0x7f18681128e0 con 0x7f187c1616e0 2024-04-25T14:11:16.767 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6833/2221276863 2024-04-26T14:11:16.711047+0000 2024-04-25T14:11:16.767 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6832/2221276863 2024-04-26T14:11:16.711047+0000 2024-04-25T14:11:16.767 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3342732660 2024-04-26T14:11:16.711047+0000 2024-04-25T14:11:16.767 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1871957865 2024-04-26T14:11:16.711047+0000 2024-04-25T14:11:16.767 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2615931231 2024-04-26T14:11:16.711047+0000 2024-04-25T14:11:16.767 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1695622468 2024-04-26T14:11:16.711047+0000 2024-04-25T14:11:16.767 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3537150562 2024-04-26T14:11:16.711047+0000 2024-04-25T14:11:16.767 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/506566715 2024-04-26T14:11:16.711047+0000 2024-04-25T14:11:16.767 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3961472441 2024-04-26T14:11:04.532513+0000 2024-04-25T14:11:16.768 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/287404215 2024-04-26T14:11:04.532513+0000 2024-04-25T14:11:16.768 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/889708173 2024-04-26T14:11:04.532513+0000 2024-04-25T14:11:16.768 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3224392570 2024-04-26T14:11:04.532513+0000 2024-04-25T14:11:16.768 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6833/3964666460 2024-04-26T14:11:04.532513+0000 2024-04-25T14:11:16.768 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/553620002 2024-04-26T14:11:04.532513+0000 2024-04-25T14:11:16.768 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6832/3964666460 2024-04-26T14:11:04.532513+0000 2024-04-25T14:11:16.768 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3726306847 2024-04-26T14:11:04.532513+0000 2024-04-25T14:11:16.768 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2412030790 2024-04-25T15:10:55.520329+0000 2024-04-25T14:11:16.768 INFO:teuthology.orchestra.run.smithi086.stderr:listed 17 entries 2024-04-25T14:11:16.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.773+0000 7f1884400640 1 -- 172.21.15.86:0/2909920196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f187c1616e0 msgr2=0x7f187c15c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:16.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.773+0000 7f1884400640 1 --2- 172.21.15.86:0/2909920196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f187c1616e0 0x7f187c15c9f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f187c0733a0 tx=0x7f186801ba50 comp rx=0 tx=0).stop 2024-04-25T14:11:16.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.773+0000 7f1884400640 1 -- 172.21.15.86:0/2909920196 shutdown_connections 2024-04-25T14:11:16.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.773+0000 7f1884400640 1 --2- 172.21.15.86:0/2909920196 >> [v2:172.21.15.86:6832/2221276863,v1:172.21.15.86:6833/2221276863] conn(0x7f185c06f550 0x7f185c071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.773+0000 7f1884400640 1 --2- 172.21.15.86:0/2909920196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f187c1556f0 0x7f187c157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.773+0000 7f1884400640 1 --2- 172.21.15.86:0/2909920196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f187c1616e0 0x7f187c15c9f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.773+0000 7f1884400640 1 --2- 172.21.15.86:0/2909920196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f187c0d2c80 0x7f187c1473f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:16.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.773+0000 7f1884400640 1 -- 172.21.15.86:0/2909920196 >> 172.21.15.86:0/2909920196 conn(0x7f187c0cffc0 msgr2=0x7f187c0cc060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:16.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.773+0000 7f1884400640 1 -- 172.21.15.86:0/2909920196 shutdown_connections 2024-04-25T14:11:16.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:16.773+0000 7f1884400640 1 -- 172.21.15.86:0/2909920196 wait complete. 2024-04-25T14:11:16.779 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:16.781+0000 7f8ab5e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:16.779 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:16.781+0000 7f8ab5e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:16.779 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:16.781+0000 7f8ab5e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:16.779 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:16.781+0000 7f8ab5e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:16.779 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:16.781+0000 7f8ab5e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:16.779 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:16.781+0000 7f8ab9a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:16.779 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:16.781+0000 7f8ab9a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:16.779 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:16.781+0000 7f8ab9a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:16.779 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:16.781+0000 7f8ab9a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:16.779 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:16.781+0000 7f8ab9a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:16.793 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T14:11:16.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:11:16.793 DEBUG:teuthology.orchestra.run.smithi155:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T14:11:16.818 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T14:11:16.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:11:16.818 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T14:11:16.872 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:11:16.872 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:11:16.872 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:11:16.881 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2893 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T14:11:16.881 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:11:16.882 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:11:16.882 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:11:16.924 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4517 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T14:11:16.924 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:11:17.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.073+0000 7fbd7ca00640 1 Processor -- start 2024-04-25T14:11:17.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.073+0000 7fbd7ca00640 1 -- start start 2024-04-25T14:11:17.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.077+0000 7fbd7ca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbd78168bf0 0x7fbd7816b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:17.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.077+0000 7fbd7ca00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd7806f0f0 con 0x7fbd78072630 2024-04-25T14:11:17.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.077+0000 7fbd7ca00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd78163a80 con 0x7fbd78160e90 2024-04-25T14:11:17.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.077+0000 7fbd7ca00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbd78163d60 con 0x7fbd78168bf0 2024-04-25T14:11:17.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.077+0000 7fbd76800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbd78168bf0 0x7fbd7816b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:17.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.077+0000 7fbd6fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fbd78072630 0x7fbd78072a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34918/0 (socket says 172.21.15.86:34918) 2024-04-25T14:11:17.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.077+0000 7fbd6fe00640 1 -- 172.21.15.86:0/1535222007 learned_addr learned my addr 172.21.15.86:0/1535222007 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:17.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.077+0000 7fbd76800640 1 -- 172.21.15.86:0/1535222007 >> v1:172.21.15.155:6789/0 conn(0x7fbd78160e90 legacy=0x7fbd780d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:17.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.077+0000 7fbd76800640 1 -- 172.21.15.86:0/1535222007 >> v1:172.21.15.86:6789/0 conn(0x7fbd78072630 legacy=0x7fbd78072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:17.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.077+0000 7fbd76800640 1 -- 172.21.15.86:0/1535222007 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd78164040 con 0x7fbd78168bf0 2024-04-25T14:11:17.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.081+0000 7fbd76800640 1 --2- 172.21.15.86:0/1535222007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd78168bf0 0x7fbd7816b010 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009d90 tx=0x7fbd7001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470dcdf19a616bca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:17.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.081+0000 7fbd75400640 1 -- 172.21.15.86:0/1535222007 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd700b2070 con 0x7fbd78168bf0 2024-04-25T14:11:17.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.081+0000 7fbd75400640 1 -- 172.21.15.86:0/1535222007 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd700a5830 con 0x7fbd78168bf0 2024-04-25T14:11:17.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.081+0000 7fbd75400640 1 -- 172.21.15.86:0/1535222007 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd700a5c00 con 0x7fbd78168bf0 2024-04-25T14:11:17.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.081+0000 7fbd7ca00640 1 -- 172.21.15.86:0/1535222007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd78168bf0 msgr2=0x7fbd7816b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:17.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.081+0000 7fbd7ca00640 1 --2- 172.21.15.86:0/1535222007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd78168bf0 0x7fbd7816b010 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009d90 tx=0x7fbd7001c970 comp rx=0 tx=0).stop 2024-04-25T14:11:17.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.081+0000 7fbd7ca00640 1 -- 172.21.15.86:0/1535222007 shutdown_connections 2024-04-25T14:11:17.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.081+0000 7fbd7ca00640 1 --2- 172.21.15.86:0/1535222007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd78168bf0 0x7fbd7816b010 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:17.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.081+0000 7fbd7ca00640 1 -- 172.21.15.86:0/1535222007 >> 172.21.15.86:0/1535222007 conn(0x7fbd780d3570 msgr2=0x7fbd780d3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:17.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.081+0000 7fbd7ca00640 1 -- 172.21.15.86:0/1535222007 shutdown_connections 2024-04-25T14:11:17.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.081+0000 7fbd7ca00640 1 -- 172.21.15.86:0/1535222007 wait complete. 2024-04-25T14:11:17.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.085+0000 7fbd7ca00640 1 Processor -- start 2024-04-25T14:11:17.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.085+0000 7fbd7ca00640 1 -- start start 2024-04-25T14:11:17.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.085+0000 7fbd7ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd78072630 0x7fbd781479d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:17.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.085+0000 7fbd7ca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd78160e90 0x7fbd7815b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:17.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.085+0000 7fbd7ca00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd78168bf0 0x7fbd7815bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:17.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.085+0000 7fbd7ca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7816e100 con 0x7fbd78072630 2024-04-25T14:11:17.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.085+0000 7fbd7ca00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7816df80 con 0x7fbd78160e90 2024-04-25T14:11:17.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.085+0000 7fbd7ca00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbd7816e280 con 0x7fbd78168bf0 2024-04-25T14:11:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.085+0000 7fbd75e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd78072630 0x7fbd781479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.085+0000 7fbd75e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd78072630 0x7fbd781479d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59878/0 (socket says 172.21.15.86:59878) 2024-04-25T14:11:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.085+0000 7fbd75e00640 1 -- 172.21.15.86:0/1993899944 learned_addr learned my addr 172.21.15.86:0/1993899944 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.085+0000 7fbd76800640 1 --2- 172.21.15.86:0/1993899944 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd78168bf0 0x7fbd7815bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.085+0000 7fbd6fe00640 1 --2- 172.21.15.86:0/1993899944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd78160e90 0x7fbd7815b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.085+0000 7fbd75e00640 1 -- 172.21.15.86:0/1993899944 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd78168bf0 msgr2=0x7fbd7815bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.085+0000 7fbd75e00640 1 --2- 172.21.15.86:0/1993899944 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd78168bf0 0x7fbd7815bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.085+0000 7fbd75e00640 1 -- 172.21.15.86:0/1993899944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd78160e90 msgr2=0x7fbd7815b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.085+0000 7fbd75e00640 1 --2- 172.21.15.86:0/1993899944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd78160e90 0x7fbd7815b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.085+0000 7fbd75e00640 1 -- 172.21.15.86:0/1993899944 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd78165080 con 0x7fbd78072630 2024-04-25T14:11:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.085+0000 7fbd75e00640 1 --2- 172.21.15.86:0/1993899944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd78072630 0x7fbd781479d0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800c370 tx=0x7fbd6800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.085+0000 7fbd6e000640 1 -- 172.21.15.86:0/1993899944 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd680052d0 con 0x7fbd78072630 2024-04-25T14:11:17.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.085+0000 7fbd6e000640 1 -- 172.21.15.86:0/1993899944 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd68005470 con 0x7fbd78072630 2024-04-25T14:11:17.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.085+0000 7fbd7ca00640 1 -- 172.21.15.86:0/1993899944 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd78165360 con 0x7fbd78072630 2024-04-25T14:11:17.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.085+0000 7fbd6e000640 1 -- 172.21.15.86:0/1993899944 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68007260 con 0x7fbd78072630 2024-04-25T14:11:17.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.085+0000 7fbd7ca00640 1 -- 172.21.15.86:0/1993899944 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd78165920 con 0x7fbd78072630 2024-04-25T14:11:17.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.089+0000 7fbd7ca00640 1 -- 172.21.15.86:0/1993899944 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd78072170 con 0x7fbd78072630 2024-04-25T14:11:17.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.089+0000 7fbd6e000640 1 -- 172.21.15.86:0/1993899944 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 63) ==== 92918+0+0 (secure 0 0 0) 0x7fbd68007400 con 0x7fbd78072630 2024-04-25T14:11:17.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.089+0000 7fbd6e000640 1 -- 172.21.15.86:0/1993899944 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6832+0+0 (secure 0 0 0) 0x7fbd6808e660 con 0x7fbd78072630 2024-04-25T14:11:17.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.093+0000 7fbd6e000640 1 -- 172.21.15.86:0/1993899944 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbd6805ae20 con 0x7fbd78072630 2024-04-25T14:11:17.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.285+0000 7fbd7ca00640 1 -- 172.21.15.86:0/1993899944 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd78157fd0 con 0x7fbd78072630 2024-04-25T14:11:17.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.285+0000 7fbd6e000640 1 -- 172.21.15.86:0/1993899944 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5186 (secure 0 0 0) 0x7fbd6805f120 con 0x7fbd78072630 2024-04-25T14:11:17.282 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:17.282 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":94,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2258481468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2258481468},{"type":"v1","addr":"172.21.15.155:6834","nonce":2258481468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":8976,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1064843151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1064843151},{"type":"v1","addr":"172.21.15.86:6835","nonce":1064843151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":8985,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1940755625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1940755625},{"type":"v1","addr":"172.21.15.86:6837","nonce":1940755625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:10:45.497688+0000","modified":"2024-04-25T14:10:48.475869+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8991":{"gid":8991,"name":"b","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6835/544985680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":544985680},{"type":"v1","addr":"172.21.15.155:6835","nonce":544985680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8991,"qdb_cluster":[8991]},"id":9}]} 2024-04-25T14:11:17.282 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 94 2024-04-25T14:11:17.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.285+0000 7fbd7ca00640 1 -- 172.21.15.86:0/1993899944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd78072630 msgr2=0x7fbd781479d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:17.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.285+0000 7fbd7ca00640 1 --2- 172.21.15.86:0/1993899944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd78072630 0x7fbd781479d0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800c370 tx=0x7fbd6800c830 comp rx=0 tx=0).stop 2024-04-25T14:11:17.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.285+0000 7fbd7ca00640 1 -- 172.21.15.86:0/1993899944 shutdown_connections 2024-04-25T14:11:17.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.285+0000 7fbd7ca00640 1 --2- 172.21.15.86:0/1993899944 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd78168bf0 0x7fbd7815bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:17.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.285+0000 7fbd7ca00640 1 --2- 172.21.15.86:0/1993899944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd78160e90 0x7fbd7815b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:17.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.285+0000 7fbd7ca00640 1 --2- 172.21.15.86:0/1993899944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd78072630 0x7fbd781479d0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:17.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.285+0000 7fbd7ca00640 1 -- 172.21.15.86:0/1993899944 >> 172.21.15.86:0/1993899944 conn(0x7fbd780d3570 msgr2=0x7fbd7806f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:17.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.285+0000 7fbd7ca00640 1 -- 172.21.15.86:0/1993899944 shutdown_connections 2024-04-25T14:11:17.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.285+0000 7fbd7ca00640 1 -- 172.21.15.86:0/1993899944 wait complete. 2024-04-25T14:11:17.304 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:11:17.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.453+0000 7f0d9d000640 1 Processor -- start 2024-04-25T14:11:17.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.453+0000 7f0d9d000640 1 -- start start 2024-04-25T14:11:17.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.453+0000 7f0d9d000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d9806c670 0x7f0d9815de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:17.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.453+0000 7f0d9d000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0d980cdf50 0x7f0d980ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:17.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.453+0000 7f0d9d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0d980d2600 0x7f0d980d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:17.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.453+0000 7f0d9d000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0d9815e380 con 0x7f0d980d2600 2024-04-25T14:11:17.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.453+0000 7f0d9d000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0d98162750 con 0x7f0d980cdf50 2024-04-25T14:11:17.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.453+0000 7f0d9d000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0d98161b40 con 0x7f0d9806c670 2024-04-25T14:11:17.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.453+0000 7f0d96800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0d980d2600 0x7f0d980d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:17.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.453+0000 7f0d96800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0d980d2600 0x7f0d980d2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59892/0 (socket says 172.21.15.86:59892) 2024-04-25T14:11:17.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.453+0000 7f0d96800640 1 -- 172.21.15.86:0/3344227742 learned_addr learned my addr 172.21.15.86:0/3344227742 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:17.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.453+0000 7f0d8d400640 1 --2- 172.21.15.86:0/3344227742 >> v2:172.21.15.155:3300/0 conn(0x7f0d980cdf50 0x7f0d980ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:17.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.453+0000 7f0d95e00640 1 --2- 172.21.15.86:0/3344227742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d9806c670 0x7f0d9815de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:17.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.453+0000 7f0d96800640 1 -- 172.21.15.86:0/3344227742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d9806c670 msgr2=0x7f0d9815de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:17.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.453+0000 7f0d96800640 1 --2- 172.21.15.86:0/3344227742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d9806c670 0x7f0d9815de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:17.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.453+0000 7f0d96800640 1 -- 172.21.15.86:0/3344227742 >> v2:172.21.15.155:3300/0 conn(0x7f0d980cdf50 msgr2=0x7f0d980ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:17.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.453+0000 7f0d96800640 1 --2- 172.21.15.86:0/3344227742 >> v2:172.21.15.155:3300/0 conn(0x7f0d980cdf50 0x7f0d980ce360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:17.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.453+0000 7f0d96800640 1 -- 172.21.15.86:0/3344227742 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d98161e20 con 0x7f0d980d2600 2024-04-25T14:11:17.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.457+0000 7f0d96800640 1 --2- 172.21.15.86:0/3344227742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d980d2600 0x7f0d980d2a10 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f0d980d2380 tx=0x7f0d9001c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d70d7fa2124b24c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:17.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.457+0000 7f0d95400640 1 -- 172.21.15.86:0/3344227742 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d900a5750 con 0x7f0d980d2600 2024-04-25T14:11:17.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.457+0000 7f0d95400640 1 -- 172.21.15.86:0/3344227742 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d900a58f0 con 0x7f0d980d2600 2024-04-25T14:11:17.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.457+0000 7f0d95400640 1 -- 172.21.15.86:0/3344227742 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d900a5cc0 con 0x7f0d980d2600 2024-04-25T14:11:17.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.457+0000 7f0d9d000640 1 -- 172.21.15.86:0/3344227742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d980d2600 msgr2=0x7f0d980d2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:17.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.457+0000 7f0d9d000640 1 --2- 172.21.15.86:0/3344227742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d980d2600 0x7f0d980d2a10 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f0d980d2380 tx=0x7f0d9001c8d0 comp rx=0 tx=0).stop 2024-04-25T14:11:17.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.457+0000 7f0d9d000640 1 -- 172.21.15.86:0/3344227742 shutdown_connections 2024-04-25T14:11:17.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.457+0000 7f0d9d000640 1 --2- 172.21.15.86:0/3344227742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d980d2600 0x7f0d980d2a10 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:17.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.457+0000 7f0d9d000640 1 --2- 172.21.15.86:0/3344227742 >> v2:172.21.15.155:3300/0 conn(0x7f0d980cdf50 0x7f0d980ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:17.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.457+0000 7f0d9d000640 1 --2- 172.21.15.86:0/3344227742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d9806c670 0x7f0d9815de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:17.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.457+0000 7f0d9d000640 1 -- 172.21.15.86:0/3344227742 >> 172.21.15.86:0/3344227742 conn(0x7f0d9809ba30 msgr2=0x7f0d980b73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:17.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.457+0000 7f0d9d000640 1 -- 172.21.15.86:0/3344227742 shutdown_connections 2024-04-25T14:11:17.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.457+0000 7f0d9d000640 1 -- 172.21.15.86:0/3344227742 wait complete. 2024-04-25T14:11:17.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.461+0000 7f0d9d000640 1 Processor -- start 2024-04-25T14:11:17.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.461+0000 7f0d9d000640 1 -- start start 2024-04-25T14:11:17.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.461+0000 7f0d9d000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d9806c670 0x7f0d98156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:17.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.461+0000 7f0d9d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d980cdf50 0x7f0d980cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:17.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.461+0000 7f0d9d000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d980d2600 0x7f0d980cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:17.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.461+0000 7f0d9d000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d980b7ee0 con 0x7f0d980cdf50 2024-04-25T14:11:17.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.461+0000 7f0d9d000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0d980ce8a0 con 0x7f0d9806c670 2024-04-25T14:11:17.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.461+0000 7f0d9d000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0d980b8060 con 0x7f0d980d2600 2024-04-25T14:11:17.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.461+0000 7f0d8d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d980cdf50 0x7f0d980cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:17.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.461+0000 7f0d8d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d980cdf50 0x7f0d980cac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59908/0 (socket says 172.21.15.86:59908) 2024-04-25T14:11:17.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.461+0000 7f0d8d400640 1 -- 172.21.15.86:0/1684002749 learned_addr learned my addr 172.21.15.86:0/1684002749 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:17.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.461+0000 7f0d95e00640 1 --2- 172.21.15.86:0/1684002749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d9806c670 0x7f0d98156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:17.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.461+0000 7f0d96800640 1 --2- 172.21.15.86:0/1684002749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d980d2600 0x7f0d980cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:17.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.465+0000 7f0d8d400640 1 -- 172.21.15.86:0/1684002749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d980d2600 msgr2=0x7f0d980cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:17.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.465+0000 7f0d8d400640 1 --2- 172.21.15.86:0/1684002749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d980d2600 0x7f0d980cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:17.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.465+0000 7f0d8d400640 1 -- 172.21.15.86:0/1684002749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d9806c670 msgr2=0x7f0d98156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:17.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.465+0000 7f0d8d400640 1 --2- 172.21.15.86:0/1684002749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d9806c670 0x7f0d98156c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:17.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.465+0000 7f0d8d400640 1 -- 172.21.15.86:0/1684002749 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d980c8ed0 con 0x7f0d980cdf50 2024-04-25T14:11:17.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.465+0000 7f0d8d400640 1 --2- 172.21.15.86:0/1684002749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d980cdf50 0x7f0d980cac70 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000d620 tx=0x7f0d8000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:17.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.465+0000 7f0d8ea00640 1 -- 172.21.15.86:0/1684002749 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8000b620 con 0x7f0d980cdf50 2024-04-25T14:11:17.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.465+0000 7f0d8ea00640 1 -- 172.21.15.86:0/1684002749 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d80019070 con 0x7f0d980cdf50 2024-04-25T14:11:17.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.465+0000 7f0d9d000640 1 -- 172.21.15.86:0/1684002749 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d980c8bf0 con 0x7f0d980cdf50 2024-04-25T14:11:17.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.465+0000 7f0d8ea00640 1 -- 172.21.15.86:0/1684002749 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8000b9e0 con 0x7f0d980cdf50 2024-04-25T14:11:17.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.465+0000 7f0d9d000640 1 -- 172.21.15.86:0/1684002749 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d980cbb20 con 0x7f0d980cdf50 2024-04-25T14:11:17.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.465+0000 7f0d9d000640 1 -- 172.21.15.86:0/1684002749 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d980a0630 con 0x7f0d980cdf50 2024-04-25T14:11:17.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.465+0000 7f0d8ea00640 1 -- 172.21.15.86:0/1684002749 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 63) ==== 92918+0+0 (secure 0 0 0) 0x7f0d8000bb80 con 0x7f0d980cdf50 2024-04-25T14:11:17.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.469+0000 7f0d8ea00640 1 -- 172.21.15.86:0/1684002749 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6832+0+0 (secure 0 0 0) 0x7f0d8008f570 con 0x7f0d980cdf50 2024-04-25T14:11:17.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.473+0000 7f0d8ea00640 1 -- 172.21.15.86:0/1684002749 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0d8005bd30 con 0x7f0d980cdf50 2024-04-25T14:11:17.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.633+0000 7f0d9d000640 1 -- 172.21.15.86:0/1684002749 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d98076bf0 con 0x7f0d980cdf50 2024-04-25T14:11:17.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.633+0000 7f0d8ea00640 1 -- 172.21.15.86:0/1684002749 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+17961 (secure 0 0 0) 0x7f0d80060030 con 0x7f0d980cdf50 2024-04-25T14:11:17.632 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:17.632 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":129,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:11:16.711086+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:10:43.913889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T14:10:44.419362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6833/2221276863":"2024-04-26T14:11:16.711047+0000","172.21.15.86:6832/2221276863":"2024-04-26T14:11:16.711047+0000","172.21.15.86:0/3342732660":"2024-04-26T14:11:16.711047+0000","172.21.15.86:0/1871957865":"2024-04-26T14:11:16.711047+0000","172.21.15.86:0/2615931231":"2024-04-26T14:11:16.711047+0000","172.21.15.86:0/1695622468":"2024-04-26T14:11:16.711047+0000","172.21.15.86:0/3537150562":"2024-04-26T14:11:16.711047+0000","172.21.15.86:0/506566715":"2024-04-26T14:11:16.711047+0000","172.21.15.86:0/3961472441":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/287404215":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/889708173":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/3224392570":"2024-04-26T14:11:04.532513+0000","172.21.15.86:6833/3964666460":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/553620002":"2024-04-26T14:11:04.532513+0000","172.21.15.86:6832/3964666460":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/3726306847":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/2412030790":"2024-04-25T15:10:55.520329+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:11:17.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.637+0000 7f0d9d000640 1 -- 172.21.15.86:0/1684002749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d980cdf50 msgr2=0x7f0d980cac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:17.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.637+0000 7f0d9d000640 1 --2- 172.21.15.86:0/1684002749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d980cdf50 0x7f0d980cac70 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000d620 tx=0x7f0d8000dae0 comp rx=0 tx=0).stop 2024-04-25T14:11:17.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.637+0000 7f0d9d000640 1 -- 172.21.15.86:0/1684002749 shutdown_connections 2024-04-25T14:11:17.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.637+0000 7f0d9d000640 1 --2- 172.21.15.86:0/1684002749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d980d2600 0x7f0d980cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:17.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.637+0000 7f0d9d000640 1 --2- 172.21.15.86:0/1684002749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d980cdf50 0x7f0d980cac70 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:17.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.637+0000 7f0d9d000640 1 --2- 172.21.15.86:0/1684002749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d9806c670 0x7f0d98156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:17.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.637+0000 7f0d9d000640 1 -- 172.21.15.86:0/1684002749 >> 172.21.15.86:0/1684002749 conn(0x7f0d9809ba30 msgr2=0x7f0d9815ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:17.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.637+0000 7f0d9d000640 1 -- 172.21.15.86:0/1684002749 shutdown_connections 2024-04-25T14:11:17.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.637+0000 7f0d9d000640 1 -- 172.21.15.86:0/1684002749 wait complete. 2024-04-25T14:11:17.656 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:10:43.913889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:10:44.419362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:11:17.656 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:11:17.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.817+0000 7f1003800640 1 Processor -- start 2024-04-25T14:11:17.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.817+0000 7f1003800640 1 -- start start 2024-04-25T14:11:17.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.821+0000 7f1003800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ffc0cfca0 0x7f0ffc06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:17.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.821+0000 7f1003800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0ffc0d16e0 0x7f0ffc0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:17.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.821+0000 7f1003800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ffc0d2030 con 0x7f0ffc06c670 2024-04-25T14:11:17.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.821+0000 7f1003800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0ffc070470 con 0x7f0ffc0d16e0 2024-04-25T14:11:17.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.821+0000 7f1003800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0ffc072bd0 con 0x7f0ffc0cfca0 2024-04-25T14:11:17.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.821+0000 7f1000e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ffc0cfca0 0x7f0ffc06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:17.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.821+0000 7f0ffbe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0ffc0d16e0 0x7f0ffc0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:17.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.821+0000 7f1001800640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f0ffc06c670 0x7f0ffc06ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:49084/0 (socket says 172.21.15.86:49084) 2024-04-25T14:11:17.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.821+0000 7f1001800640 1 -- 172.21.15.86:0/2515007428 learned_addr learned my addr 172.21.15.86:0/2515007428 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:17.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.821+0000 7f0ffb400640 1 -- 172.21.15.86:0/2515007428 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2533603937 0 0) 0x7f0ff40014a0 con 0x7f0ffc06c670 2024-04-25T14:11:17.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.821+0000 7f0ffb400640 1 -- 172.21.15.86:0/2515007428 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0fe4003530 con 0x7f0ffc06c670 2024-04-25T14:11:17.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.821+0000 7f1000e00640 1 -- 172.21.15.86:0/2515007428 >> v2:172.21.15.155:3300/0 conn(0x7f0ffc0d16e0 msgr2=0x7f0ffc0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:17.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.821+0000 7f1000e00640 1 --2- 172.21.15.86:0/2515007428 >> v2:172.21.15.155:3300/0 conn(0x7f0ffc0d16e0 0x7f0ffc0d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:17.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.821+0000 7f1000e00640 1 -- 172.21.15.86:0/2515007428 >> v1:172.21.15.155:6789/0 conn(0x7f0ffc06c670 legacy=0x7f0ffc06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:17.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.821+0000 7f1000e00640 1 -- 172.21.15.86:0/2515007428 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ffc06f930 con 0x7f0ffc0cfca0 2024-04-25T14:11:17.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.821+0000 7f0ffbe00640 1 --2- 172.21.15.86:0/2515007428 >> v2:172.21.15.155:3300/0 conn(0x7f0ffc0d16e0 0x7f0ffc0d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:11:17.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.825+0000 7f1000e00640 1 --2- 172.21.15.86:0/2515007428 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ffc0cfca0 0x7f0ffc06f3f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00097c0 tx=0x7f0ff001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c941ad99c8d519cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:17.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.825+0000 7f0ffb400640 1 -- 172.21.15.86:0/2515007428 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff00b1070 con 0x7f0ffc0cfca0 2024-04-25T14:11:17.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.825+0000 7f0ffb400640 1 -- 172.21.15.86:0/2515007428 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ff00ac410 con 0x7f0ffc0cfca0 2024-04-25T14:11:17.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.825+0000 7f0ffb400640 1 -- 172.21.15.86:0/2515007428 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff00ac7e0 con 0x7f0ffc0cfca0 2024-04-25T14:11:17.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.825+0000 7f1003800640 1 -- 172.21.15.86:0/2515007428 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ffc0cfca0 msgr2=0x7f0ffc06f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:17.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.825+0000 7f1003800640 1 --2- 172.21.15.86:0/2515007428 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ffc0cfca0 0x7f0ffc06f3f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00097c0 tx=0x7f0ff001c4a0 comp rx=0 tx=0).stop 2024-04-25T14:11:17.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.825+0000 7f1003800640 1 -- 172.21.15.86:0/2515007428 shutdown_connections 2024-04-25T14:11:17.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.825+0000 7f1003800640 1 --2- 172.21.15.86:0/2515007428 >> v2:172.21.15.155:3300/0 conn(0x7f0ffc0d16e0 0x7f0ffc0d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:17.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.825+0000 7f1003800640 1 --2- 172.21.15.86:0/2515007428 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ffc0cfca0 0x7f0ffc06f3f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:17.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.825+0000 7f1003800640 1 -- 172.21.15.86:0/2515007428 >> 172.21.15.86:0/2515007428 conn(0x7f0ffc0b4650 msgr2=0x7f0ffc0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:17.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.825+0000 7f1003800640 1 -- 172.21.15.86:0/2515007428 shutdown_connections 2024-04-25T14:11:17.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.825+0000 7f1003800640 1 -- 172.21.15.86:0/2515007428 wait complete. 2024-04-25T14:11:17.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.829+0000 7f1003800640 1 Processor -- start 2024-04-25T14:11:17.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.829+0000 7f1003800640 1 -- start start 2024-04-25T14:11:17.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.829+0000 7f1003800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ffc06c670 0x7f0ffc0cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:17.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.829+0000 7f1003800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ffc0cfca0 0x7f0ffc1f97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:17.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.829+0000 7f1003800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ffc0d16e0 0x7f0ffc0cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:17.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.829+0000 7f1003800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc16a070 con 0x7f0ffc0d16e0 2024-04-25T14:11:17.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.829+0000 7f1003800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc169ef0 con 0x7f0ffc06c670 2024-04-25T14:11:17.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.829+0000 7f1003800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0ffc16a1f0 con 0x7f0ffc0cfca0 2024-04-25T14:11:17.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.829+0000 7f1001800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ffc0d16e0 0x7f0ffc0cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:17.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.829+0000 7f1001800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ffc0d16e0 0x7f0ffc0cdf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59920/0 (socket says 172.21.15.86:59920) 2024-04-25T14:11:17.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.829+0000 7f1001800640 1 -- 172.21.15.86:0/3008170645 learned_addr learned my addr 172.21.15.86:0/3008170645 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:17.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.829+0000 7f1000e00640 1 --2- 172.21.15.86:0/3008170645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ffc06c670 0x7f0ffc0cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:17.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.829+0000 7f0ffbe00640 1 --2- 172.21.15.86:0/3008170645 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ffc0cfca0 0x7f0ffc1f97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:17.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.829+0000 7f1001800640 1 -- 172.21.15.86:0/3008170645 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ffc0cfca0 msgr2=0x7f0ffc1f97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:17.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.829+0000 7f1001800640 1 --2- 172.21.15.86:0/3008170645 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ffc0cfca0 0x7f0ffc1f97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:17.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.829+0000 7f1001800640 1 -- 172.21.15.86:0/3008170645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ffc06c670 msgr2=0x7f0ffc0cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:17.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.829+0000 7f1001800640 1 --2- 172.21.15.86:0/3008170645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ffc06c670 0x7f0ffc0cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:17.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.829+0000 7f1001800640 1 -- 172.21.15.86:0/3008170645 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ffc15bdb0 con 0x7f0ffc0d16e0 2024-04-25T14:11:17.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.829+0000 7f1001800640 1 --2- 172.21.15.86:0/3008170645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ffc0d16e0 0x7f0ffc0cdf60 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f0ff400caa0 tx=0x7f0ff400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:17.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.829+0000 7f0ff8c00640 1 -- 172.21.15.86:0/3008170645 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4017070 con 0x7f0ffc0d16e0 2024-04-25T14:11:17.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.829+0000 7f0ff8c00640 1 -- 172.21.15.86:0/3008170645 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ff400ac00 con 0x7f0ffc0d16e0 2024-04-25T14:11:17.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.829+0000 7f1003800640 1 -- 172.21.15.86:0/3008170645 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ffc160020 con 0x7f0ffc0d16e0 2024-04-25T14:11:17.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.829+0000 7f0ff8c00640 1 -- 172.21.15.86:0/3008170645 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4012400 con 0x7f0ffc0d16e0 2024-04-25T14:11:17.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.829+0000 7f1003800640 1 -- 172.21.15.86:0/3008170645 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ffc0cc260 con 0x7f0ffc0d16e0 2024-04-25T14:11:17.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.833+0000 7f1003800640 1 -- 172.21.15.86:0/3008170645 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ffc09f4b0 con 0x7f0ffc0d16e0 2024-04-25T14:11:17.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.833+0000 7f0ff8c00640 1 -- 172.21.15.86:0/3008170645 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 64) ==== 93045+0+0 (secure 0 0 0) 0x7f0ff401a070 con 0x7f0ffc0d16e0 2024-04-25T14:11:17.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.833+0000 7f0ff8c00640 1 --2- 172.21.15.86:0/3008170645 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f0fdc06f250 0x7f0fdc071760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:17.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.833+0000 7f0ff8c00640 1 -- 172.21.15.86:0/3008170645 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6832+0+0 (secure 0 0 0) 0x7f0ff408f0f0 con 0x7f0ffc0d16e0 2024-04-25T14:11:17.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.833+0000 7f1000e00640 1 --2- 172.21.15.86:0/3008170645 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f0fdc06f250 0x7f0fdc071760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:17.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.833+0000 7f1000e00640 1 --2- 172.21.15.86:0/3008170645 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f0fdc06f250 0x7f0fdc071760 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00ae000 tx=0x7f0ff00a6040 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:17.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.837+0000 7f0ff8c00640 1 -- 172.21.15.86:0/3008170645 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0ffc0d2030 con 0x7f0ffc0d16e0 2024-04-25T14:11:17.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.997+0000 7f1003800640 1 -- 172.21.15.86:0/3008170645 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0ffc069d00 con 0x7f0ffc0d16e0 2024-04-25T14:11:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.997+0000 7f0ff8c00640 1 -- 172.21.15.86:0/3008170645 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v129) ==== 93+0+10 (secure 0 0 0) 0x7f0ff405b7c0 con 0x7f0ffc0d16e0 2024-04-25T14:11:17.993 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:11:17.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.997+0000 7f1003800640 1 -- 172.21.15.86:0/3008170645 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f0fdc06f250 msgr2=0x7f0fdc071760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:17.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.997+0000 7f1003800640 1 --2- 172.21.15.86:0/3008170645 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f0fdc06f250 0x7f0fdc071760 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00ae000 tx=0x7f0ff00a6040 comp rx=0 tx=0).stop 2024-04-25T14:11:17.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.997+0000 7f1003800640 1 -- 172.21.15.86:0/3008170645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ffc0d16e0 msgr2=0x7f0ffc0cdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.997+0000 7f1003800640 1 --2- 172.21.15.86:0/3008170645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ffc0d16e0 0x7f0ffc0cdf60 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f0ff400caa0 tx=0x7f0ff400cf60 comp rx=0 tx=0).stop 2024-04-25T14:11:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.997+0000 7f1003800640 1 -- 172.21.15.86:0/3008170645 shutdown_connections 2024-04-25T14:11:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.997+0000 7f1003800640 1 --2- 172.21.15.86:0/3008170645 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f0fdc06f250 0x7f0fdc071760 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.997+0000 7f1003800640 1 --2- 172.21.15.86:0/3008170645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ffc0d16e0 0x7f0ffc0cdf60 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.997+0000 7f1003800640 1 --2- 172.21.15.86:0/3008170645 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ffc0cfca0 0x7f0ffc1f97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.997+0000 7f1003800640 1 --2- 172.21.15.86:0/3008170645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ffc06c670 0x7f0ffc0cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.997+0000 7f1003800640 1 -- 172.21.15.86:0/3008170645 >> 172.21.15.86:0/3008170645 conn(0x7f0ffc0b4650 msgr2=0x7f0ffc0afc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.997+0000 7f1003800640 1 -- 172.21.15.86:0/3008170645 shutdown_connections 2024-04-25T14:11:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:17.997+0000 7f1003800640 1 -- 172.21.15.86:0/3008170645 wait complete. 2024-04-25T14:11:18.014 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:11:18.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.169+0000 7fce72a00640 1 Processor -- start 2024-04-25T14:11:18.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.169+0000 7fce72a00640 1 -- start start 2024-04-25T14:11:18.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.173+0000 7fce72a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fce6c0d1df0 0x7fce6c0cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:18.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.173+0000 7fce72a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fce6c0cdcc0 0x7fce6c0ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:18.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.173+0000 7fce72a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce6c15e5c0 con 0x7fce6c070330 2024-04-25T14:11:18.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.173+0000 7fce72a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fce6c06fd40 con 0x7fce6c0d1df0 2024-04-25T14:11:18.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.173+0000 7fce72a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fce6c15e8a0 con 0x7fce6c0cdcc0 2024-04-25T14:11:18.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.173+0000 7fce63400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fce6c0d1df0 0x7fce6c0cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:18.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.173+0000 7fce63400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fce6c0d1df0 0x7fce6c0cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59932/0 (socket says 172.21.15.86:59932) 2024-04-25T14:11:18.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.173+0000 7fce6be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fce6c070330 0x7fce6c0d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34932/0 (socket says 172.21.15.86:34932) 2024-04-25T14:11:18.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.173+0000 7fce63400640 1 -- 172.21.15.86:0/3846536216 learned_addr learned my addr 172.21.15.86:0/3846536216 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:18.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.173+0000 7fce70a00640 1 --2- 172.21.15.86:0/3846536216 >> v2:172.21.15.155:3300/0 conn(0x7fce6c0cdcc0 0x7fce6c0ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:18.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.173+0000 7fce63400640 1 -- 172.21.15.86:0/3846536216 >> v2:172.21.15.155:3300/0 conn(0x7fce6c0cdcc0 msgr2=0x7fce6c0ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:18.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.173+0000 7fce63400640 1 --2- 172.21.15.86:0/3846536216 >> v2:172.21.15.155:3300/0 conn(0x7fce6c0cdcc0 0x7fce6c0ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.173+0000 7fce63400640 1 -- 172.21.15.86:0/3846536216 >> v1:172.21.15.86:6789/0 conn(0x7fce6c070330 legacy=0x7fce6c0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:18.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.173+0000 7fce63400640 1 -- 172.21.15.86:0/3846536216 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce6c15eb80 con 0x7fce6c0d1df0 2024-04-25T14:11:18.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.173+0000 7fce6b400640 1 -- 172.21.15.86:0/3846536216 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1939213471 0 0) 0x7fce6c15e5c0 con 0x7fce6c070330 2024-04-25T14:11:18.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.177+0000 7fce63400640 1 --2- 172.21.15.86:0/3846536216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce6c0d1df0 0x7fce6c0cd780 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fce54009870 tx=0x7fce5401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a4f8d4ab0557ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:18.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.177+0000 7fce6b400640 1 -- 172.21.15.86:0/3846536216 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce540ab640 con 0x7fce6c0d1df0 2024-04-25T14:11:18.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.177+0000 7fce6b400640 1 -- 172.21.15.86:0/3846536216 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce540b0070 con 0x7fce6c0d1df0 2024-04-25T14:11:18.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.177+0000 7fce6b400640 1 -- 172.21.15.86:0/3846536216 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce540aba00 con 0x7fce6c0d1df0 2024-04-25T14:11:18.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.177+0000 7fce72a00640 1 -- 172.21.15.86:0/3846536216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce6c0d1df0 msgr2=0x7fce6c0cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:18.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.177+0000 7fce72a00640 1 --2- 172.21.15.86:0/3846536216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce6c0d1df0 0x7fce6c0cd780 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fce54009870 tx=0x7fce5401c460 comp rx=0 tx=0).stop 2024-04-25T14:11:18.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.177+0000 7fce72a00640 1 -- 172.21.15.86:0/3846536216 shutdown_connections 2024-04-25T14:11:18.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.177+0000 7fce72a00640 1 --2- 172.21.15.86:0/3846536216 >> v2:172.21.15.155:3300/0 conn(0x7fce6c0cdcc0 0x7fce6c0ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.177+0000 7fce72a00640 1 --2- 172.21.15.86:0/3846536216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce6c0d1df0 0x7fce6c0cd780 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.177+0000 7fce72a00640 1 -- 172.21.15.86:0/3846536216 >> 172.21.15.86:0/3846536216 conn(0x7fce6c0b4650 msgr2=0x7fce6c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:18.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.177+0000 7fce72a00640 1 -- 172.21.15.86:0/3846536216 shutdown_connections 2024-04-25T14:11:18.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.177+0000 7fce72a00640 1 -- 172.21.15.86:0/3846536216 wait complete. 2024-04-25T14:11:18.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.177+0000 7fce72a00640 1 Processor -- start 2024-04-25T14:11:18.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.177+0000 7fce72a00640 1 -- start start 2024-04-25T14:11:18.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.181+0000 7fce72a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce6c070330 0x7fce6c0c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:18.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.181+0000 7fce72a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce6c0cdcc0 0x7fce6c073610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:18.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.181+0000 7fce72a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce6c073b50 0x7fce6c15d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:18.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.181+0000 7fce72a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c15f4b0 con 0x7fce6c070330 2024-04-25T14:11:18.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.181+0000 7fce72a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c15f330 con 0x7fce6c0cdcc0 2024-04-25T14:11:18.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.181+0000 7fce72a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fce6c070740 con 0x7fce6c073b50 2024-04-25T14:11:18.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.181+0000 7fce6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce6c070330 0x7fce6c0c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:18.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.181+0000 7fce6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce6c070330 0x7fce6c0c9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59936/0 (socket says 172.21.15.86:59936) 2024-04-25T14:11:18.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.181+0000 7fce6be00640 1 -- 172.21.15.86:0/3824027028 learned_addr learned my addr 172.21.15.86:0/3824027028 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:18.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.181+0000 7fce70a00640 1 --2- 172.21.15.86:0/3824027028 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce6c073b50 0x7fce6c15d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:18.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.181+0000 7fce63400640 1 --2- 172.21.15.86:0/3824027028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce6c0cdcc0 0x7fce6c073610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:18.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.181+0000 7fce6be00640 1 -- 172.21.15.86:0/3824027028 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce6c073b50 msgr2=0x7fce6c15d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:18.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.181+0000 7fce6be00640 1 --2- 172.21.15.86:0/3824027028 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce6c073b50 0x7fce6c15d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.181+0000 7fce6be00640 1 -- 172.21.15.86:0/3824027028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce6c0cdcc0 msgr2=0x7fce6c073610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:18.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.181+0000 7fce6be00640 1 --2- 172.21.15.86:0/3824027028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce6c0cdcc0 0x7fce6c073610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.181+0000 7fce6be00640 1 -- 172.21.15.86:0/3824027028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce6c0c9690 con 0x7fce6c070330 2024-04-25T14:11:18.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.181+0000 7fce6be00640 1 --2- 172.21.15.86:0/3824027028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce6c070330 0x7fce6c0c9150 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00c480 tx=0x7fce5c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:18.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.181+0000 7fce68c00640 1 -- 172.21.15.86:0/3824027028 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c018070 con 0x7fce6c070330 2024-04-25T14:11:18.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.181+0000 7fce68c00640 1 -- 172.21.15.86:0/3824027028 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce5c0074e0 con 0x7fce6c070330 2024-04-25T14:11:18.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.181+0000 7fce68c00640 1 -- 172.21.15.86:0/3824027028 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c0049f0 con 0x7fce6c070330 2024-04-25T14:11:18.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.181+0000 7fce72a00640 1 -- 172.21.15.86:0/3824027028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce6c159d20 con 0x7fce6c070330 2024-04-25T14:11:18.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.181+0000 7fce72a00640 1 -- 172.21.15.86:0/3824027028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce6c0745f0 con 0x7fce6c070330 2024-04-25T14:11:18.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.181+0000 7fce72a00640 1 -- 172.21.15.86:0/3824027028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce6c069d00 con 0x7fce6c070330 2024-04-25T14:11:18.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.185+0000 7fce68c00640 1 -- 172.21.15.86:0/3824027028 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 64) ==== 93045+0+0 (secure 0 0 0) 0x7fce5c004b90 con 0x7fce6c070330 2024-04-25T14:11:18.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.185+0000 7fce68c00640 1 --2- 172.21.15.86:0/3824027028 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fce4806f280 0x7fce48071790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:18.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.185+0000 7fce68c00640 1 -- 172.21.15.86:0/3824027028 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6832+0+0 (secure 0 0 0) 0x7fce5c08e660 con 0x7fce6c070330 2024-04-25T14:11:18.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.185+0000 7fce63400640 1 --2- 172.21.15.86:0/3824027028 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fce4806f280 0x7fce48071790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:18.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.185+0000 7fce63400640 1 --2- 172.21.15.86:0/3824027028 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fce4806f280 0x7fce48071790 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fce540af000 tx=0x7fce540a7000 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:18.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.189+0000 7fce68c00640 1 -- 172.21.15.86:0/3824027028 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fce6c0c9690 con 0x7fce6c070330 2024-04-25T14:11:18.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.345+0000 7fce72a00640 1 -- 172.21.15.86:0/3824027028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fce6c076bf0 con 0x7fce6c070330 2024-04-25T14:11:18.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.345+0000 7fce68c00640 1 -- 172.21.15.86:0/3824027028 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v129) ==== 104+0+11 (secure 0 0 0) 0x7fce5c05b410 con 0x7fce6c070330 2024-04-25T14:11:18.342 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:11:18.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.345+0000 7fce72a00640 1 -- 172.21.15.86:0/3824027028 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fce4806f280 msgr2=0x7fce48071790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:18.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.345+0000 7fce72a00640 1 --2- 172.21.15.86:0/3824027028 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fce4806f280 0x7fce48071790 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fce540af000 tx=0x7fce540a7000 comp rx=0 tx=0).stop 2024-04-25T14:11:18.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.345+0000 7fce72a00640 1 -- 172.21.15.86:0/3824027028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce6c070330 msgr2=0x7fce6c0c9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:18.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.345+0000 7fce72a00640 1 --2- 172.21.15.86:0/3824027028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce6c070330 0x7fce6c0c9150 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00c480 tx=0x7fce5c00c940 comp rx=0 tx=0).stop 2024-04-25T14:11:18.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.349+0000 7fce72a00640 1 -- 172.21.15.86:0/3824027028 shutdown_connections 2024-04-25T14:11:18.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.349+0000 7fce72a00640 1 --2- 172.21.15.86:0/3824027028 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fce4806f280 0x7fce48071790 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.349+0000 7fce72a00640 1 --2- 172.21.15.86:0/3824027028 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce6c073b50 0x7fce6c15d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.349+0000 7fce72a00640 1 --2- 172.21.15.86:0/3824027028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce6c0cdcc0 0x7fce6c073610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.349+0000 7fce72a00640 1 --2- 172.21.15.86:0/3824027028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce6c070330 0x7fce6c0c9150 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.349+0000 7fce72a00640 1 -- 172.21.15.86:0/3824027028 >> 172.21.15.86:0/3824027028 conn(0x7fce6c0b4650 msgr2=0x7fce6c0d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:18.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.349+0000 7fce72a00640 1 -- 172.21.15.86:0/3824027028 shutdown_connections 2024-04-25T14:11:18.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.349+0000 7fce72a00640 1 -- 172.21.15.86:0/3824027028 wait complete. 2024-04-25T14:11:18.363 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:11:18.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.513+0000 7f24ba200640 1 Processor -- start 2024-04-25T14:11:18.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.513+0000 7f24ba200640 1 -- start start 2024-04-25T14:11:18.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.517+0000 7f24ba200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f24b415ec70 0x7f24b40d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:18.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.517+0000 7f24ba200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24b40cf8f0 con 0x7f24b40d1490 2024-04-25T14:11:18.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.517+0000 7f24ba200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24b406c670 con 0x7f24b415e860 2024-04-25T14:11:18.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.517+0000 7f24ba200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f24b406cfa0 con 0x7f24b415ec70 2024-04-25T14:11:18.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.517+0000 7f24b2a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f24b415ec70 0x7f24b40d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:18.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.517+0000 7f24b3e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f24b40d1490 0x7f24b406d430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34948/0 (socket says 172.21.15.86:34948) 2024-04-25T14:11:18.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.517+0000 7f24b3e00640 1 -- 172.21.15.86:0/2056686557 learned_addr learned my addr 172.21.15.86:0/2056686557 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:18.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.517+0000 7f24b2a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f24b415ec70 0x7f24b40d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59950/0 (socket says 172.21.15.86:59950) 2024-04-25T14:11:18.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.517+0000 7f24b2a00640 1 -- 172.21.15.86:0/2056686557 >> v1:172.21.15.155:6789/0 conn(0x7f24b415e860 legacy=0x7f24b40ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:18.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.517+0000 7f24b2a00640 1 -- 172.21.15.86:0/2056686557 >> v1:172.21.15.86:6789/0 conn(0x7f24b40d1490 legacy=0x7f24b406d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:18.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.517+0000 7f24b2a00640 1 -- 172.21.15.86:0/2056686557 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b406c950 con 0x7f24b415ec70 2024-04-25T14:11:18.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.521+0000 7f24b2a00640 1 --2- 172.21.15.86:0/2056686557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24b415ec70 0x7f24b40d0f50 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f24a00097c0 tx=0x7f24a001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353246ac9ab36d3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:18.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.521+0000 7f24b2000640 1 -- 172.21.15.86:0/2056686557 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a00b0070 con 0x7f24b415ec70 2024-04-25T14:11:18.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.521+0000 7f24b2000640 1 -- 172.21.15.86:0/2056686557 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24a00ab600 con 0x7f24b415ec70 2024-04-25T14:11:18.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.521+0000 7f24b2000640 1 -- 172.21.15.86:0/2056686557 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a00ab9d0 con 0x7f24b415ec70 2024-04-25T14:11:18.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.521+0000 7f24ba200640 1 -- 172.21.15.86:0/2056686557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24b415ec70 msgr2=0x7f24b40d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:18.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.521+0000 7f24ba200640 1 --2- 172.21.15.86:0/2056686557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24b415ec70 0x7f24b40d0f50 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f24a00097c0 tx=0x7f24a001c3e0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.521+0000 7f24ba200640 1 -- 172.21.15.86:0/2056686557 shutdown_connections 2024-04-25T14:11:18.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.521+0000 7f24ba200640 1 --2- 172.21.15.86:0/2056686557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24b415ec70 0x7f24b40d0f50 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.521+0000 7f24ba200640 1 -- 172.21.15.86:0/2056686557 >> 172.21.15.86:0/2056686557 conn(0x7f24b40d2230 msgr2=0x7f24b40d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:18.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.521+0000 7f24ba200640 1 -- 172.21.15.86:0/2056686557 shutdown_connections 2024-04-25T14:11:18.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.521+0000 7f24ba200640 1 -- 172.21.15.86:0/2056686557 wait complete. 2024-04-25T14:11:18.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.525+0000 7f24ba200640 1 Processor -- start 2024-04-25T14:11:18.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.525+0000 7f24ba200640 1 -- start start 2024-04-25T14:11:18.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.525+0000 7f24ba200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24b40d1490 0x7f24b41fc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:18.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.525+0000 7f24ba200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f24b415e860 0x7f24b4153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:18.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.525+0000 7f24ba200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24b415ec70 0x7f24b415ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:18.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.525+0000 7f24ba200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24b40d18a0 con 0x7f24b415ec70 2024-04-25T14:11:18.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.525+0000 7f24ba200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f24b40b5770 con 0x7f24b40d1490 2024-04-25T14:11:18.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.525+0000 7f24ba200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f24b416bf40 con 0x7f24b415e860 2024-04-25T14:11:18.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.525+0000 7f24b3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24b415ec70 0x7f24b415ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:18.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.525+0000 7f24b3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24b415ec70 0x7f24b415ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59956/0 (socket says 172.21.15.86:59956) 2024-04-25T14:11:18.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.525+0000 7f24b3e00640 1 -- 172.21.15.86:0/2244051536 learned_addr learned my addr 172.21.15.86:0/2244051536 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:18.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.525+0000 7f24b2a00640 1 --2- 172.21.15.86:0/2244051536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f24b415e860 0x7f24b4153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:18.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.525+0000 7f24b3400640 1 --2- 172.21.15.86:0/2244051536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24b40d1490 0x7f24b41fc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:18.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.525+0000 7f24b3e00640 1 -- 172.21.15.86:0/2244051536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f24b415e860 msgr2=0x7f24b4153ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:18.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.525+0000 7f24b3e00640 1 --2- 172.21.15.86:0/2244051536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f24b415e860 0x7f24b4153ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.525+0000 7f24b3e00640 1 -- 172.21.15.86:0/2244051536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24b40d1490 msgr2=0x7f24b41fc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:18.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.525+0000 7f24b3e00640 1 --2- 172.21.15.86:0/2244051536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24b40d1490 0x7f24b41fc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.525+0000 7f24b3e00640 1 -- 172.21.15.86:0/2244051536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b406f060 con 0x7f24b415ec70 2024-04-25T14:11:18.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.525+0000 7f24b3400640 1 --2- 172.21.15.86:0/2244051536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24b40d1490 0x7f24b41fc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:18.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.525+0000 7f24b3e00640 1 --2- 172.21.15.86:0/2244051536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24b415ec70 0x7f24b415ca70 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f24ac00ca80 tx=0x7f24ac00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:18.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.525+0000 7f24ab400640 1 -- 172.21.15.86:0/2244051536 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ac017070 con 0x7f24b415ec70 2024-04-25T14:11:18.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.525+0000 7f24ab400640 1 -- 172.21.15.86:0/2244051536 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24ac00ac30 con 0x7f24b415ec70 2024-04-25T14:11:18.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.525+0000 7f24ab400640 1 -- 172.21.15.86:0/2244051536 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ac012600 con 0x7f24b415ec70 2024-04-25T14:11:18.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.525+0000 7f24ba200640 1 -- 172.21.15.86:0/2244051536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b406f340 con 0x7f24b415ec70 2024-04-25T14:11:18.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.525+0000 7f24ba200640 1 -- 172.21.15.86:0/2244051536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b41f99b0 con 0x7f24b415ec70 2024-04-25T14:11:18.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.529+0000 7f24ba200640 1 -- 172.21.15.86:0/2244051536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24b41f9c30 con 0x7f24b415ec70 2024-04-25T14:11:18.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.529+0000 7f24ab400640 1 -- 172.21.15.86:0/2244051536 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 64) ==== 93045+0+0 (secure 0 0 0) 0x7f24ac01a070 con 0x7f24b415ec70 2024-04-25T14:11:18.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.529+0000 7f24ab400640 1 --2- 172.21.15.86:0/2244051536 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f249406f250 0x7f2494071760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:18.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.529+0000 7f24ab400640 1 -- 172.21.15.86:0/2244051536 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6832+0+0 (secure 0 0 0) 0x7f24ac08f040 con 0x7f24b415ec70 2024-04-25T14:11:18.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.529+0000 7f24b3400640 1 --2- 172.21.15.86:0/2244051536 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f249406f250 0x7f2494071760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:18.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.529+0000 7f24b3400640 1 --2- 172.21.15.86:0/2244051536 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f249406f250 0x7f2494071760 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f24a4000f70 tx=0x7f24a4009040 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:18.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.533+0000 7f24ab400640 1 -- 172.21.15.86:0/2244051536 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f24ac00ae20 con 0x7f24b415ec70 2024-04-25T14:11:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.689+0000 7f24ba200640 1 -- 172.21.15.86:0/2244051536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f24b4074cf0 con 0x7f24b415ec70 2024-04-25T14:11:18.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.689+0000 7f24ab400640 1 -- 172.21.15.86:0/2244051536 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v129) ==== 100+0+11 (secure 0 0 0) 0x7f24ac05b710 con 0x7f24b415ec70 2024-04-25T14:11:18.688 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:11:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.693+0000 7f24ba200640 1 -- 172.21.15.86:0/2244051536 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f249406f250 msgr2=0x7f2494071760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.693+0000 7f24ba200640 1 --2- 172.21.15.86:0/2244051536 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f249406f250 0x7f2494071760 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f24a4000f70 tx=0x7f24a4009040 comp rx=0 tx=0).stop 2024-04-25T14:11:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.693+0000 7f24ba200640 1 -- 172.21.15.86:0/2244051536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24b415ec70 msgr2=0x7f24b415ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.693+0000 7f24ba200640 1 --2- 172.21.15.86:0/2244051536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24b415ec70 0x7f24b415ca70 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f24ac00ca80 tx=0x7f24ac00cf40 comp rx=0 tx=0).stop 2024-04-25T14:11:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.693+0000 7f24ba200640 1 -- 172.21.15.86:0/2244051536 shutdown_connections 2024-04-25T14:11:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.693+0000 7f24ba200640 1 --2- 172.21.15.86:0/2244051536 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f249406f250 0x7f2494071760 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.693+0000 7f24ba200640 1 --2- 172.21.15.86:0/2244051536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24b415ec70 0x7f24b415ca70 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.693+0000 7f24ba200640 1 --2- 172.21.15.86:0/2244051536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f24b415e860 0x7f24b4153ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.693+0000 7f24ba200640 1 --2- 172.21.15.86:0/2244051536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24b40d1490 0x7f24b41fc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.693+0000 7f24ba200640 1 -- 172.21.15.86:0/2244051536 >> 172.21.15.86:0/2244051536 conn(0x7f24b40d2230 msgr2=0x7f24b40b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:18.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.693+0000 7f24ba200640 1 -- 172.21.15.86:0/2244051536 shutdown_connections 2024-04-25T14:11:18.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.693+0000 7f24ba200640 1 -- 172.21.15.86:0/2244051536 wait complete. 2024-04-25T14:11:18.711 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:11:18.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.869+0000 7f3ca2800640 1 Processor -- start 2024-04-25T14:11:18.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.869+0000 7f3ca2800640 1 -- start start 2024-04-25T14:11:18.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.869+0000 7f3ca2800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3c9c071530 0x7f3c9c15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:18.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.869+0000 7f3ca2800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c9c1616e0 0x7f3c9c161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:18.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.869+0000 7f3ca2800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c9c0cd8d0 con 0x7f3c9c0d2c80 2024-04-25T14:11:18.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.869+0000 7f3ca2800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3c9c0ce720 con 0x7f3c9c071530 2024-04-25T14:11:18.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.869+0000 7f3ca2800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3c9c0ce970 con 0x7f3c9c1616e0 2024-04-25T14:11:18.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.869+0000 7f3c9aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3c9c071530 0x7f3c9c15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:18.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.869+0000 7f3c9b400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3c9c0d2c80 0x7f3c9c070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39064/0 (socket says 172.21.15.86:39064) 2024-04-25T14:11:18.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.869+0000 7f3c9b400640 1 -- 172.21.15.86:0/2197155030 learned_addr learned my addr 172.21.15.86:0/2197155030 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:18.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.869+0000 7f3c9aa00640 1 --2- 172.21.15.86:0/2197155030 >> v2:172.21.15.86:3300/0 conn(0x7f3c9c071530 0x7f3c9c15dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49524/0 (socket says 172.21.15.86:49524) 2024-04-25T14:11:18.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.869+0000 7f3c9be00640 1 --2- 172.21.15.86:0/2197155030 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c9c1616e0 0x7f3c9c161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:18.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.869+0000 7f3c9aa00640 1 -- 172.21.15.86:0/2197155030 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c9c1616e0 msgr2=0x7f3c9c161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:18.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.869+0000 7f3c9aa00640 1 --2- 172.21.15.86:0/2197155030 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c9c1616e0 0x7f3c9c161af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.869+0000 7f3c9aa00640 1 -- 172.21.15.86:0/2197155030 >> v1:172.21.15.86:6789/0 conn(0x7f3c9c0d2c80 legacy=0x7f3c9c070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:18.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.869+0000 7f3c9aa00640 1 -- 172.21.15.86:0/2197155030 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c9c0d1be0 con 0x7f3c9c071530 2024-04-25T14:11:18.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.873+0000 7f3c9aa00640 1 --2- 172.21.15.86:0/2197155030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c9c071530 0x7f3c9c15dc90 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f3c880097a0 tx=0x7f3c8801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99837538a538acb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:18.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.873+0000 7f3c9a000640 1 -- 172.21.15.86:0/2197155030 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c880b5070 con 0x7f3c9c071530 2024-04-25T14:11:18.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.873+0000 7f3c9a000640 1 -- 172.21.15.86:0/2197155030 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c880a8970 con 0x7f3c9c071530 2024-04-25T14:11:18.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.873+0000 7f3c9a000640 1 -- 172.21.15.86:0/2197155030 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c880a8d40 con 0x7f3c9c071530 2024-04-25T14:11:18.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.873+0000 7f3ca2800640 1 -- 172.21.15.86:0/2197155030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c9c071530 msgr2=0x7f3c9c15dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:18.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.873+0000 7f3ca2800640 1 --2- 172.21.15.86:0/2197155030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c9c071530 0x7f3c9c15dc90 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f3c880097a0 tx=0x7f3c8801ec70 comp rx=0 tx=0).stop 2024-04-25T14:11:18.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.873+0000 7f3ca2800640 1 -- 172.21.15.86:0/2197155030 shutdown_connections 2024-04-25T14:11:18.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.873+0000 7f3ca2800640 1 --2- 172.21.15.86:0/2197155030 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c9c1616e0 0x7f3c9c161af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.873+0000 7f3ca2800640 1 --2- 172.21.15.86:0/2197155030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c9c071530 0x7f3c9c15dc90 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.873+0000 7f3ca2800640 1 -- 172.21.15.86:0/2197155030 >> 172.21.15.86:0/2197155030 conn(0x7f3c9c0cffc0 msgr2=0x7f3c9c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:18.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.873+0000 7f3ca2800640 1 -- 172.21.15.86:0/2197155030 shutdown_connections 2024-04-25T14:11:18.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.873+0000 7f3ca2800640 1 -- 172.21.15.86:0/2197155030 wait complete. 2024-04-25T14:11:18.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.877+0000 7f3ca2800640 1 Processor -- start 2024-04-25T14:11:18.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.877+0000 7f3ca2800640 1 -- start start 2024-04-25T14:11:18.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.877+0000 7f3ca2800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c9c0d2c80 0x7f3c9c0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:18.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.877+0000 7f3ca2800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c9c1616e0 0x7f3c9c0c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:18.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.877+0000 7f3ca2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c9c0b4280 0x7f3c9c14d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:18.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.877+0000 7f3ca2800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c0d2480 con 0x7f3c9c0b4280 2024-04-25T14:11:18.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.877+0000 7f3ca2800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c0d21a0 con 0x7f3c9c1616e0 2024-04-25T14:11:18.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.877+0000 7f3ca2800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3c9c0d29a0 con 0x7f3c9c0d2c80 2024-04-25T14:11:18.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.877+0000 7f3c9be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c9c0b4280 0x7f3c9c14d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:18.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.877+0000 7f3c9be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c9c0b4280 0x7f3c9c14d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49526/0 (socket says 172.21.15.86:49526) 2024-04-25T14:11:18.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.877+0000 7f3c9be00640 1 -- 172.21.15.86:0/190636277 learned_addr learned my addr 172.21.15.86:0/190636277 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:18.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.877+0000 7f3c9b400640 1 --2- 172.21.15.86:0/190636277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c9c0d2c80 0x7f3c9c0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:18.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.877+0000 7f3c9aa00640 1 --2- 172.21.15.86:0/190636277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c9c1616e0 0x7f3c9c0c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:18.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.877+0000 7f3c9be00640 1 -- 172.21.15.86:0/190636277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c9c0d2c80 msgr2=0x7f3c9c0ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:18.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.877+0000 7f3c9be00640 1 --2- 172.21.15.86:0/190636277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c9c0d2c80 0x7f3c9c0ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.877+0000 7f3c9be00640 1 -- 172.21.15.86:0/190636277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c9c1616e0 msgr2=0x7f3c9c0c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:18.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.877+0000 7f3c9be00640 1 --2- 172.21.15.86:0/190636277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c9c1616e0 0x7f3c9c0c8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:18.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.877+0000 7f3c9be00640 1 -- 172.21.15.86:0/190636277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c9c14c3b0 con 0x7f3c9c0b4280 2024-04-25T14:11:18.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.877+0000 7f3c9b400640 1 --2- 172.21.15.86:0/190636277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c9c0d2c80 0x7f3c9c0ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:18.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.877+0000 7f3c9be00640 1 --2- 172.21.15.86:0/190636277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c9c0b4280 0x7f3c9c14d8c0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f3c94009fd0 tx=0x7f3c9400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:18.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.877+0000 7f3c93400640 1 -- 172.21.15.86:0/190636277 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94014420 con 0x7f3c9c0b4280 2024-04-25T14:11:18.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.877+0000 7f3c93400640 1 -- 172.21.15.86:0/190636277 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c94019070 con 0x7f3c9c0b4280 2024-04-25T14:11:18.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.877+0000 7f3ca2800640 1 -- 172.21.15.86:0/190636277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c9c0c9460 con 0x7f3c9c0b4280 2024-04-25T14:11:18.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.881+0000 7f3c93400640 1 -- 172.21.15.86:0/190636277 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94014800 con 0x7f3c9c0b4280 2024-04-25T14:11:18.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.881+0000 7f3ca2800640 1 -- 172.21.15.86:0/190636277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c9c14cb00 con 0x7f3c9c0b4280 2024-04-25T14:11:18.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.881+0000 7f3ca2800640 1 -- 172.21.15.86:0/190636277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c9c0720e0 con 0x7f3c9c0b4280 2024-04-25T14:11:18.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.881+0000 7f3c93400640 1 -- 172.21.15.86:0/190636277 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 65) ==== 93253+0+0 (secure 0 0 0) 0x7f3c940149a0 con 0x7f3c9c0b4280 2024-04-25T14:11:18.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.881+0000 7f3c93400640 1 --2- 172.21.15.86:0/190636277 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f3c7c06f490 0x7f3c7c0719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:18.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.881+0000 7f3c93400640 1 -- 172.21.15.86:0/190636277 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6832+0+0 (secure 0 0 0) 0x7f3c9408ffe0 con 0x7f3c9c0b4280 2024-04-25T14:11:18.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.881+0000 7f3c9b400640 1 --2- 172.21.15.86:0/190636277 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f3c7c06f490 0x7f3c7c0719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:18.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.881+0000 7f3c9b400640 1 --2- 172.21.15.86:0/190636277 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f3c7c06f490 0x7f3c7c0719a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c000f70 tx=0x7f3c8c009040 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:18.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:18.885+0000 7f3c93400640 1 -- 172.21.15.86:0/190636277 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3c9405c9a0 con 0x7f3c9c0b4280 2024-04-25T14:11:19.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.073+0000 7f3ca2800640 1 -- 172.21.15.86:0/190636277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c9c14c5b0 con 0x7f3c9c0b4280 2024-04-25T14:11:19.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.077+0000 7f3c93400640 1 -- 172.21.15.86:0/190636277 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5186 (secure 0 0 0) 0x7f3c9405c2a0 con 0x7f3c9c0b4280 2024-04-25T14:11:19.073 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:19.073 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":94,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2258481468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2258481468},{"type":"v1","addr":"172.21.15.155:6834","nonce":2258481468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":8976,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1064843151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1064843151},{"type":"v1","addr":"172.21.15.86:6835","nonce":1064843151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":8985,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1940755625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1940755625},{"type":"v1","addr":"172.21.15.86:6837","nonce":1940755625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:10:45.497688+0000","modified":"2024-04-25T14:10:48.475869+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8991":{"gid":8991,"name":"b","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6835/544985680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":544985680},{"type":"v1","addr":"172.21.15.155:6835","nonce":544985680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8991,"qdb_cluster":[8991]},"id":9}]} 2024-04-25T14:11:19.073 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 94 2024-04-25T14:11:19.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.077+0000 7f3ca2800640 1 -- 172.21.15.86:0/190636277 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f3c7c06f490 msgr2=0x7f3c7c0719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:19.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.077+0000 7f3ca2800640 1 --2- 172.21.15.86:0/190636277 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f3c7c06f490 0x7f3c7c0719a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c000f70 tx=0x7f3c8c009040 comp rx=0 tx=0).stop 2024-04-25T14:11:19.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.077+0000 7f3ca2800640 1 -- 172.21.15.86:0/190636277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c9c0b4280 msgr2=0x7f3c9c14d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:19.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.077+0000 7f3ca2800640 1 --2- 172.21.15.86:0/190636277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c9c0b4280 0x7f3c9c14d8c0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f3c94009fd0 tx=0x7f3c9400e4b0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.077+0000 7f3ca2800640 1 -- 172.21.15.86:0/190636277 shutdown_connections 2024-04-25T14:11:19.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.077+0000 7f3ca2800640 1 --2- 172.21.15.86:0/190636277 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f3c7c06f490 0x7f3c7c0719a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.077+0000 7f3ca2800640 1 --2- 172.21.15.86:0/190636277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c9c0b4280 0x7f3c9c14d8c0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.077+0000 7f3ca2800640 1 --2- 172.21.15.86:0/190636277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c9c1616e0 0x7f3c9c0c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.077+0000 7f3ca2800640 1 --2- 172.21.15.86:0/190636277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c9c0d2c80 0x7f3c9c0ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.077+0000 7f3ca2800640 1 -- 172.21.15.86:0/190636277 >> 172.21.15.86:0/190636277 conn(0x7f3c9c0cffc0 msgr2=0x7f3c9c0cb370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:19.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.077+0000 7f3ca2800640 1 -- 172.21.15.86:0/190636277 shutdown_connections 2024-04-25T14:11:19.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.077+0000 7f3ca2800640 1 -- 172.21.15.86:0/190636277 wait complete. 2024-04-25T14:11:19.093 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:11:19.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.241+0000 7f84cd600640 1 Processor -- start 2024-04-25T14:11:19.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.241+0000 7f84cd600640 1 -- start start 2024-04-25T14:11:19.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.245+0000 7f84cd600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84c80cfca0 0x7f84c806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:19.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.245+0000 7f84cd600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f84c80730b0 0x7f84c80734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:19.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.245+0000 7f84cd600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84c8072bd0 con 0x7f84c80d16e0 2024-04-25T14:11:19.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.245+0000 7f84cd600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f84c8070470 con 0x7f84c80730b0 2024-04-25T14:11:19.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.245+0000 7f84cd600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f84c806f930 con 0x7f84c80cfca0 2024-04-25T14:11:19.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.245+0000 7f84c7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f84c80730b0 0x7f84c80734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:19.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.245+0000 7f84be000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f84c80d16e0 0x7f84c80d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39078/0 (socket says 172.21.15.86:39078) 2024-04-25T14:11:19.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.245+0000 7f84be000640 1 -- 172.21.15.86:0/2738175946 learned_addr learned my addr 172.21.15.86:0/2738175946 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:19.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.245+0000 7f84c6a00640 1 --2- 172.21.15.86:0/2738175946 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84c80cfca0 0x7f84c806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:19.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.245+0000 7f84c7400640 1 -- 172.21.15.86:0/2738175946 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84c80cfca0 msgr2=0x7f84c806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:19.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.245+0000 7f84c7400640 1 --2- 172.21.15.86:0/2738175946 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84c80cfca0 0x7f84c806f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.245+0000 7f84c7400640 1 -- 172.21.15.86:0/2738175946 >> v1:172.21.15.86:6789/0 conn(0x7f84c80d16e0 legacy=0x7f84c80d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:19.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.245+0000 7f84c7400640 1 -- 172.21.15.86:0/2738175946 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c806fbd0 con 0x7f84c80730b0 2024-04-25T14:11:19.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.245+0000 7f84c6000640 1 -- 172.21.15.86:0/2738175946 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2183370876 0 0) 0x7f84c8072bd0 con 0x7f84c80d16e0 2024-04-25T14:11:19.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.249+0000 7f84c7400640 1 --2- 172.21.15.86:0/2738175946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84c80730b0 0x7f84c80734c0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f84c0009d90 tx=0x7f84c001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513e1d0cc4d366b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:19.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.249+0000 7f84c6000640 1 -- 172.21.15.86:0/2738175946 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c00b2070 con 0x7f84c80730b0 2024-04-25T14:11:19.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.249+0000 7f84c6000640 1 -- 172.21.15.86:0/2738175946 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84c00a5940 con 0x7f84c80730b0 2024-04-25T14:11:19.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.249+0000 7f84c6000640 1 -- 172.21.15.86:0/2738175946 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c00a5d10 con 0x7f84c80730b0 2024-04-25T14:11:19.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.249+0000 7f84cd600640 1 -- 172.21.15.86:0/2738175946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84c80730b0 msgr2=0x7f84c80734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:19.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.249+0000 7f84cd600640 1 --2- 172.21.15.86:0/2738175946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84c80730b0 0x7f84c80734c0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f84c0009d90 tx=0x7f84c001c970 comp rx=0 tx=0).stop 2024-04-25T14:11:19.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.249+0000 7f84cd600640 1 -- 172.21.15.86:0/2738175946 shutdown_connections 2024-04-25T14:11:19.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.249+0000 7f84cd600640 1 --2- 172.21.15.86:0/2738175946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84c80730b0 0x7f84c80734c0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.249+0000 7f84cd600640 1 --2- 172.21.15.86:0/2738175946 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84c80cfca0 0x7f84c806f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.249+0000 7f84cd600640 1 -- 172.21.15.86:0/2738175946 >> 172.21.15.86:0/2738175946 conn(0x7f84c80b4650 msgr2=0x7f84c80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:19.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.249+0000 7f84cd600640 1 -- 172.21.15.86:0/2738175946 shutdown_connections 2024-04-25T14:11:19.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.249+0000 7f84cd600640 1 -- 172.21.15.86:0/2738175946 wait complete. 2024-04-25T14:11:19.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.249+0000 7f84cd600640 1 Processor -- start 2024-04-25T14:11:19.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.249+0000 7f84cd600640 1 -- start start 2024-04-25T14:11:19.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.253+0000 7f84cd600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84c80730b0 0x7f84c80b17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:19.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.253+0000 7f84cd600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84c80cfca0 0x7f84c80b3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:19.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.253+0000 7f84cd600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84c80d16e0 0x7f84c80b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:19.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.253+0000 7f84cd600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84c8167ae0 con 0x7f84c80cfca0 2024-04-25T14:11:19.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.253+0000 7f84cd600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f84c8167960 con 0x7f84c80d16e0 2024-04-25T14:11:19.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.253+0000 7f84cd600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f84c8167c60 con 0x7f84c80730b0 2024-04-25T14:11:19.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.253+0000 7f84be000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84c80cfca0 0x7f84c80b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:19.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.253+0000 7f84be000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84c80cfca0 0x7f84c80b3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49544/0 (socket says 172.21.15.86:49544) 2024-04-25T14:11:19.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.253+0000 7f84be000640 1 -- 172.21.15.86:0/699103417 learned_addr learned my addr 172.21.15.86:0/699103417 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:19.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.253+0000 7f84c6a00640 1 --2- 172.21.15.86:0/699103417 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84c80730b0 0x7f84c80b17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:19.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.253+0000 7f84c7400640 1 --2- 172.21.15.86:0/699103417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84c80d16e0 0x7f84c80b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:19.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.253+0000 7f84be000640 1 -- 172.21.15.86:0/699103417 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84c80730b0 msgr2=0x7f84c80b17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:19.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.253+0000 7f84be000640 1 --2- 172.21.15.86:0/699103417 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84c80730b0 0x7f84c80b17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.253+0000 7f84be000640 1 -- 172.21.15.86:0/699103417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84c80d16e0 msgr2=0x7f84c80b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:19.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.253+0000 7f84be000640 1 --2- 172.21.15.86:0/699103417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84c80d16e0 0x7f84c80b38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.253+0000 7f84be000640 1 -- 172.21.15.86:0/699103417 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c8169e50 con 0x7f84c80cfca0 2024-04-25T14:11:19.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.253+0000 7f84be000640 1 --2- 172.21.15.86:0/699103417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84c80cfca0 0x7f84c80b3360 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f84b000c4b0 tx=0x7f84b000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:19.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.253+0000 7f84bf400640 1 -- 172.21.15.86:0/699103417 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b0018070 con 0x7f84c80cfca0 2024-04-25T14:11:19.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.253+0000 7f84bf400640 1 -- 172.21.15.86:0/699103417 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84b0007500 con 0x7f84c80cfca0 2024-04-25T14:11:19.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.253+0000 7f84bf400640 1 -- 172.21.15.86:0/699103417 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b0004a30 con 0x7f84c80cfca0 2024-04-25T14:11:19.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.253+0000 7f84cd600640 1 -- 172.21.15.86:0/699103417 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c816a130 con 0x7f84c80cfca0 2024-04-25T14:11:19.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.253+0000 7f84cd600640 1 -- 172.21.15.86:0/699103417 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c8154410 con 0x7f84c80cfca0 2024-04-25T14:11:19.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.253+0000 7f84cd600640 1 -- 172.21.15.86:0/699103417 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84c809f4b0 con 0x7f84c80cfca0 2024-04-25T14:11:19.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.257+0000 7f84bf400640 1 -- 172.21.15.86:0/699103417 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 65) ==== 93253+0+0 (secure 0 0 0) 0x7f84b0004bd0 con 0x7f84c80cfca0 2024-04-25T14:11:19.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.257+0000 7f84bf400640 1 --2- 172.21.15.86:0/699103417 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f84a406f4c0 0x7f84a40719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:19.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.257+0000 7f84bf400640 1 -- 172.21.15.86:0/699103417 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6832+0+0 (secure 0 0 0) 0x7f84b008e9a0 con 0x7f84c80cfca0 2024-04-25T14:11:19.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.257+0000 7f84c6a00640 1 --2- 172.21.15.86:0/699103417 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f84a406f4c0 0x7f84a40719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:19.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.257+0000 7f84c6a00640 1 --2- 172.21.15.86:0/699103417 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f84a406f4c0 0x7f84a40719d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f84c8070710 tx=0x7f84b8009190 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:19.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.261+0000 7f84bf400640 1 -- 172.21.15.86:0/699103417 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f84b005ad10 con 0x7f84c80cfca0 2024-04-25T14:11:19.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.417+0000 7f84cd600640 1 -- 172.21.15.86:0/699103417 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f84c816a300 con 0x7f84c80cfca0 2024-04-25T14:11:19.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.421+0000 7f84bf400640 1 -- 172.21.15.86:0/699103417 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+17961 (secure 0 0 0) 0x7f84b005f010 con 0x7f84c80cfca0 2024-04-25T14:11:19.417 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:19.417 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":129,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:11:16.711086+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:10:43.913889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T14:10:44.419362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6833/2221276863":"2024-04-26T14:11:16.711047+0000","172.21.15.86:6832/2221276863":"2024-04-26T14:11:16.711047+0000","172.21.15.86:0/3342732660":"2024-04-26T14:11:16.711047+0000","172.21.15.86:0/1871957865":"2024-04-26T14:11:16.711047+0000","172.21.15.86:0/2615931231":"2024-04-26T14:11:16.711047+0000","172.21.15.86:0/1695622468":"2024-04-26T14:11:16.711047+0000","172.21.15.86:0/3537150562":"2024-04-26T14:11:16.711047+0000","172.21.15.86:0/506566715":"2024-04-26T14:11:16.711047+0000","172.21.15.86:0/3961472441":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/287404215":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/889708173":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/3224392570":"2024-04-26T14:11:04.532513+0000","172.21.15.86:6833/3964666460":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/553620002":"2024-04-26T14:11:04.532513+0000","172.21.15.86:6832/3964666460":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/3726306847":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/2412030790":"2024-04-25T15:10:55.520329+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:11:19.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.421+0000 7f84cd600640 1 -- 172.21.15.86:0/699103417 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f84a406f4c0 msgr2=0x7f84a40719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:19.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.421+0000 7f84cd600640 1 --2- 172.21.15.86:0/699103417 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f84a406f4c0 0x7f84a40719d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f84c8070710 tx=0x7f84b8009190 comp rx=0 tx=0).stop 2024-04-25T14:11:19.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.421+0000 7f84cd600640 1 -- 172.21.15.86:0/699103417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84c80cfca0 msgr2=0x7f84c80b3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:19.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.421+0000 7f84cd600640 1 --2- 172.21.15.86:0/699103417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84c80cfca0 0x7f84c80b3360 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f84b000c4b0 tx=0x7f84b000c970 comp rx=0 tx=0).stop 2024-04-25T14:11:19.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.421+0000 7f84cd600640 1 -- 172.21.15.86:0/699103417 shutdown_connections 2024-04-25T14:11:19.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.421+0000 7f84cd600640 1 --2- 172.21.15.86:0/699103417 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f84a406f4c0 0x7f84a40719d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.421+0000 7f84cd600640 1 --2- 172.21.15.86:0/699103417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84c80d16e0 0x7f84c80b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.421+0000 7f84cd600640 1 --2- 172.21.15.86:0/699103417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84c80cfca0 0x7f84c80b3360 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.421+0000 7f84cd600640 1 --2- 172.21.15.86:0/699103417 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84c80730b0 0x7f84c80b17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.421+0000 7f84cd600640 1 -- 172.21.15.86:0/699103417 >> 172.21.15.86:0/699103417 conn(0x7f84c80b4650 msgr2=0x7f84c80cab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:19.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.421+0000 7f84cd600640 1 -- 172.21.15.86:0/699103417 shutdown_connections 2024-04-25T14:11:19.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.421+0000 7f84cd600640 1 -- 172.21.15.86:0/699103417 wait complete. 2024-04-25T14:11:19.439 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T14:11:19.439 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:11:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.589+0000 7fb93ea00640 1 Processor -- start 2024-04-25T14:11:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.589+0000 7fb93ea00640 1 -- start start 2024-04-25T14:11:19.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.589+0000 7fb93ea00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb93815e8e0 0x7fb93815f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:19.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.589+0000 7fb93ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb93815ecf0 0x7fb93815f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.589+0000 7fb93ea00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9380cf0e0 0x7fb9380cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.589+0000 7fb93ea00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb93815e450 con 0x7fb93815ecf0 2024-04-25T14:11:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.589+0000 7fb93ea00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb93806d4a0 con 0x7fb93815e8e0 2024-04-25T14:11:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.589+0000 7fb93ea00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb9380d0510 con 0x7fb9380cf0e0 2024-04-25T14:11:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.589+0000 7fb93cc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb93815ecf0 0x7fb93815f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.589+0000 7fb93cc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb93815ecf0 0x7fb93815f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49558/0 (socket says 172.21.15.86:49558) 2024-04-25T14:11:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.589+0000 7fb93cc00640 1 -- 172.21.15.86:0/2671742752 learned_addr learned my addr 172.21.15.86:0/2671742752 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.589+0000 7fb93d600640 1 --2- 172.21.15.86:0/2671742752 >> v2:172.21.15.155:3300/0 conn(0x7fb93815e8e0 0x7fb93815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.589+0000 7fb93cc00640 1 -- 172.21.15.86:0/2671742752 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9380cf0e0 msgr2=0x7fb9380cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.589+0000 7fb93e000640 1 --2- 172.21.15.86:0/2671742752 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9380cf0e0 0x7fb9380cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.589+0000 7fb93cc00640 1 --2- 172.21.15.86:0/2671742752 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9380cf0e0 0x7fb9380cf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.589+0000 7fb93cc00640 1 -- 172.21.15.86:0/2671742752 >> v2:172.21.15.155:3300/0 conn(0x7fb93815e8e0 msgr2=0x7fb93815f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.589+0000 7fb93cc00640 1 --2- 172.21.15.86:0/2671742752 >> v2:172.21.15.155:3300/0 conn(0x7fb93815e8e0 0x7fb93815f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.589+0000 7fb93cc00640 1 -- 172.21.15.86:0/2671742752 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9380d07f0 con 0x7fb93815ecf0 2024-04-25T14:11:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.589+0000 7fb93d600640 1 --2- 172.21.15.86:0/2671742752 >> v2:172.21.15.155:3300/0 conn(0x7fb93815e8e0 0x7fb93815f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.589+0000 7fb93e000640 1 --2- 172.21.15.86:0/2671742752 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9380cf0e0 0x7fb9380cf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.593+0000 7fb93cc00640 1 --2- 172.21.15.86:0/2671742752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb93815ecf0 0x7fb93815f6d0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fb938071a90 tx=0x7fb92401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780d071f5e6af0fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.593+0000 7fb937e00640 1 -- 172.21.15.86:0/2671742752 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9240ab5f0 con 0x7fb93815ecf0 2024-04-25T14:11:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.593+0000 7fb937e00640 1 -- 172.21.15.86:0/2671742752 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9240b0070 con 0x7fb93815ecf0 2024-04-25T14:11:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.593+0000 7fb937e00640 1 -- 172.21.15.86:0/2671742752 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9240ab9b0 con 0x7fb93815ecf0 2024-04-25T14:11:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.593+0000 7fb93ea00640 1 -- 172.21.15.86:0/2671742752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb93815ecf0 msgr2=0x7fb93815f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.593+0000 7fb93ea00640 1 --2- 172.21.15.86:0/2671742752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb93815ecf0 0x7fb93815f6d0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fb938071a90 tx=0x7fb92401c520 comp rx=0 tx=0).stop 2024-04-25T14:11:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.593+0000 7fb93ea00640 1 -- 172.21.15.86:0/2671742752 shutdown_connections 2024-04-25T14:11:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.593+0000 7fb93ea00640 1 --2- 172.21.15.86:0/2671742752 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9380cf0e0 0x7fb9380cf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.593+0000 7fb93ea00640 1 --2- 172.21.15.86:0/2671742752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb93815ecf0 0x7fb93815f6d0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.593+0000 7fb93ea00640 1 --2- 172.21.15.86:0/2671742752 >> v2:172.21.15.155:3300/0 conn(0x7fb93815e8e0 0x7fb93815f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.593+0000 7fb93ea00640 1 -- 172.21.15.86:0/2671742752 >> 172.21.15.86:0/2671742752 conn(0x7fb9380b4850 msgr2=0x7fb9380b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.593+0000 7fb93ea00640 1 -- 172.21.15.86:0/2671742752 shutdown_connections 2024-04-25T14:11:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.593+0000 7fb93ea00640 1 -- 172.21.15.86:0/2671742752 wait complete. 2024-04-25T14:11:19.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.597+0000 7fb93ea00640 1 Processor -- start 2024-04-25T14:11:19.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.597+0000 7fb93ea00640 1 -- start start 2024-04-25T14:11:19.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.597+0000 7fb93ea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9380cf0e0 0x7fb938150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:19.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.597+0000 7fb93ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb93815e8e0 0x7fb9380b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:19.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.597+0000 7fb93ea00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb93815ecf0 0x7fb9380b1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:19.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.597+0000 7fb93ea00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb9380d0c50 con 0x7fb93815e8e0 2024-04-25T14:11:19.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.597+0000 7fb93ea00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb9380d0ad0 con 0x7fb9380cf0e0 2024-04-25T14:11:19.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.597+0000 7fb93ea00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb93806ffe0 con 0x7fb93815ecf0 2024-04-25T14:11:19.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.597+0000 7fb93cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb93815e8e0 0x7fb9380b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:19.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.597+0000 7fb93cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb93815e8e0 0x7fb9380b0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49564/0 (socket says 172.21.15.86:49564) 2024-04-25T14:11:19.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.597+0000 7fb93cc00640 1 -- 172.21.15.86:0/1289102291 learned_addr learned my addr 172.21.15.86:0/1289102291 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:19.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.597+0000 7fb93d600640 1 --2- 172.21.15.86:0/1289102291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9380cf0e0 0x7fb938150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:19.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.597+0000 7fb93e000640 1 --2- 172.21.15.86:0/1289102291 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb93815ecf0 0x7fb9380b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:19.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.597+0000 7fb93cc00640 1 -- 172.21.15.86:0/1289102291 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb93815ecf0 msgr2=0x7fb9380b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:19.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.597+0000 7fb93cc00640 1 --2- 172.21.15.86:0/1289102291 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb93815ecf0 0x7fb9380b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.597+0000 7fb93cc00640 1 -- 172.21.15.86:0/1289102291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9380cf0e0 msgr2=0x7fb938150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:19.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.597+0000 7fb93cc00640 1 --2- 172.21.15.86:0/1289102291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9380cf0e0 0x7fb938150580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.597+0000 7fb93cc00640 1 -- 172.21.15.86:0/1289102291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9380b2580 con 0x7fb93815e8e0 2024-04-25T14:11:19.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.601+0000 7fb93cc00640 1 --2- 172.21.15.86:0/1289102291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb93815e8e0 0x7fb9380b0cc0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fb9240ad040 tx=0x7fb9240192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.601+0000 7fb935600640 1 -- 172.21.15.86:0/1289102291 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9240bf610 con 0x7fb93815e8e0 2024-04-25T14:11:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.601+0000 7fb935600640 1 -- 172.21.15.86:0/1289102291 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9240bf7d0 con 0x7fb93815e8e0 2024-04-25T14:11:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.601+0000 7fb93ea00640 1 -- 172.21.15.86:0/1289102291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9380b2860 con 0x7fb93815e8e0 2024-04-25T14:11:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.601+0000 7fb935600640 1 -- 172.21.15.86:0/1289102291 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9240bfbc0 con 0x7fb93815e8e0 2024-04-25T14:11:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.601+0000 7fb93ea00640 1 -- 172.21.15.86:0/1289102291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9380b2e20 con 0x7fb93815e8e0 2024-04-25T14:11:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.601+0000 7fb93ea00640 1 -- 172.21.15.86:0/1289102291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9380b22d0 con 0x7fb93815e8e0 2024-04-25T14:11:19.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.601+0000 7fb935600640 1 -- 172.21.15.86:0/1289102291 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 65) ==== 93253+0+0 (secure 0 0 0) 0x7fb9240bd020 con 0x7fb93815e8e0 2024-04-25T14:11:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.601+0000 7fb935600640 1 --2- 172.21.15.86:0/1289102291 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fb91806f490 0x7fb9180719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.601+0000 7fb935600640 1 -- 172.21.15.86:0/1289102291 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6832+0+0 (secure 0 0 0) 0x7fb92412a190 con 0x7fb93815e8e0 2024-04-25T14:11:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.601+0000 7fb93d600640 1 --2- 172.21.15.86:0/1289102291 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fb91806f490 0x7fb9180719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.605+0000 7fb93d600640 1 --2- 172.21.15.86:0/1289102291 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fb91806f490 0x7fb9180719a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb92c007510 tx=0x7fb92c0074a0 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:19.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.609+0000 7fb935600640 1 -- 172.21.15.86:0/1289102291 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb924016110 con 0x7fb93815e8e0 2024-04-25T14:11:19.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.801+0000 7fb93ea00640 1 -- 172.21.15.86:0/1289102291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb938076b80 con 0x7fb93815e8e0 2024-04-25T14:11:19.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.801+0000 7fb935600640 1 -- 172.21.15.86:0/1289102291 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5186 (secure 0 0 0) 0x7fb9240f6b60 con 0x7fb93815e8e0 2024-04-25T14:11:19.800 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:19.800 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":94,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2258481468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2258481468},{"type":"v1","addr":"172.21.15.155:6834","nonce":2258481468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":8976,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1064843151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1064843151},{"type":"v1","addr":"172.21.15.86:6835","nonce":1064843151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":8985,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1940755625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1940755625},{"type":"v1","addr":"172.21.15.86:6837","nonce":1940755625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:10:45.497688+0000","modified":"2024-04-25T14:10:48.475869+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8991":{"gid":8991,"name":"b","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6835/544985680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":544985680},{"type":"v1","addr":"172.21.15.155:6835","nonce":544985680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8991,"qdb_cluster":[8991]},"id":9}]} 2024-04-25T14:11:19.800 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 94 2024-04-25T14:11:19.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.805+0000 7fb93ea00640 1 -- 172.21.15.86:0/1289102291 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fb91806f490 msgr2=0x7fb9180719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:19.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.805+0000 7fb93ea00640 1 --2- 172.21.15.86:0/1289102291 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fb91806f490 0x7fb9180719a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb92c007510 tx=0x7fb92c0074a0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.805+0000 7fb93ea00640 1 -- 172.21.15.86:0/1289102291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb93815e8e0 msgr2=0x7fb9380b0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:19.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.805+0000 7fb93ea00640 1 --2- 172.21.15.86:0/1289102291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb93815e8e0 0x7fb9380b0cc0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fb9240ad040 tx=0x7fb9240192c0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.805+0000 7fb93ea00640 1 -- 172.21.15.86:0/1289102291 shutdown_connections 2024-04-25T14:11:19.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.805+0000 7fb93ea00640 1 --2- 172.21.15.86:0/1289102291 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fb91806f490 0x7fb9180719a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.805+0000 7fb93ea00640 1 --2- 172.21.15.86:0/1289102291 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb93815ecf0 0x7fb9380b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.805+0000 7fb93ea00640 1 --2- 172.21.15.86:0/1289102291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb93815e8e0 0x7fb9380b0cc0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.805+0000 7fb93ea00640 1 --2- 172.21.15.86:0/1289102291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9380cf0e0 0x7fb938150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.805+0000 7fb93ea00640 1 -- 172.21.15.86:0/1289102291 >> 172.21.15.86:0/1289102291 conn(0x7fb9380b4850 msgr2=0x7fb938073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:19.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.805+0000 7fb93ea00640 1 -- 172.21.15.86:0/1289102291 shutdown_connections 2024-04-25T14:11:19.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.805+0000 7fb93ea00640 1 -- 172.21.15.86:0/1289102291 wait complete. 2024-04-25T14:11:19.821 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:11:19.824 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:11:19.870 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:11:19.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.969+0000 7febc9600640 1 Processor -- start 2024-04-25T14:11:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.969+0000 7febc9600640 1 -- start start 2024-04-25T14:11:19.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc9600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7febc415c2a0 0x7febc40ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:19.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc9600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febc415feb0 0x7febc41602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:19.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc9600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febc40b0a60 con 0x7febc406fd50 2024-04-25T14:11:19.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc9600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7febc406ef90 con 0x7febc415c2a0 2024-04-25T14:11:19.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc9600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7febc406f270 con 0x7febc415feb0 2024-04-25T14:11:19.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febba000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7febc406fd50 0x7febc4070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39082/0 (socket says 172.21.15.86:39082) 2024-04-25T14:11:19.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febba000640 1 -- 172.21.15.86:0/2910863533 learned_addr learned my addr 172.21.15.86:0/2910863533 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:19.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc3400640 1 --2- 172.21.15.86:0/2910863533 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febc415feb0 0x7febc41602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:19.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc2a00640 1 --2- 172.21.15.86:0/2910863533 >> v2:172.21.15.155:3300/0 conn(0x7febc415c2a0 0x7febc40ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:19.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc2000640 1 -- 172.21.15.86:0/2910863533 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1104879790 0 0) 0x7febc40b0a60 con 0x7febc406fd50 2024-04-25T14:11:19.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc2000640 1 -- 172.21.15.86:0/2910863533 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feba8003530 con 0x7febc406fd50 2024-04-25T14:11:19.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc2000640 1 -- 172.21.15.86:0/2910863533 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 282639647 0 0) 0x7feba8003530 con 0x7febc406fd50 2024-04-25T14:11:19.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc2000640 1 -- 172.21.15.86:0/2910863533 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febc415feb0 msgr2=0x7febc41602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:19.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc2000640 1 --2- 172.21.15.86:0/2910863533 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febc415feb0 0x7febc41602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc2000640 1 -- 172.21.15.86:0/2910863533 >> v2:172.21.15.155:3300/0 conn(0x7febc415c2a0 msgr2=0x7febc40ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:19.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc2000640 1 --2- 172.21.15.86:0/2910863533 >> v2:172.21.15.155:3300/0 conn(0x7febc415c2a0 0x7febc40ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc2000640 1 -- 172.21.15.86:0/2910863533 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc406f550 con 0x7febc406fd50 2024-04-25T14:11:19.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc2000640 1 -- 172.21.15.86:0/2910863533 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7febac001680 con 0x7febc406fd50 2024-04-25T14:11:19.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc2000640 1 -- 172.21.15.86:0/2910863533 >> v1:172.21.15.86:6789/0 conn(0x7febc406fd50 legacy=0x7febc4070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:19.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc2000640 1 --2- 172.21.15.86:0/2910863533 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febc415feb0 0x7feba8003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:19.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc2000640 1 --2- 172.21.15.86:0/2910863533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feba8003fb0 0x7feba8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:19.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc2000640 1 --2- 172.21.15.86:0/2910863533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feba8004d50 0x7feba8005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:19.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc2000640 1 -- 172.21.15.86:0/2910863533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7febc406ef90 con 0x7feba8004d50 2024-04-25T14:11:19.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc2000640 1 -- 172.21.15.86:0/2910863533 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7febc406f270 con 0x7feba8003fb0 2024-04-25T14:11:19.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc2000640 1 -- 172.21.15.86:0/2910863533 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7feba800bb30 con 0x7febc415feb0 2024-04-25T14:11:19.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc2a00640 1 --2- 172.21.15.86:0/2910863533 >> v2:172.21.15.155:3300/0 conn(0x7febc415c2a0 0x7febc40ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:11:19.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc3400640 1 --2- 172.21.15.86:0/2910863533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feba8004d50 0x7feba8005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:19.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc3400640 1 -- 172.21.15.86:0/2910863533 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febc415feb0 msgr2=0x7feba8003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:19.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc3400640 1 --2- 172.21.15.86:0/2910863533 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febc415feb0 0x7feba8003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc3400640 1 -- 172.21.15.86:0/2910863533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feba8003fb0 msgr2=0x7feba8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:19.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc3400640 1 --2- 172.21.15.86:0/2910863533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feba8003fb0 0x7feba8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.973+0000 7febc3400640 1 -- 172.21.15.86:0/2910863533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba8003530 con 0x7feba8004d50 2024-04-25T14:11:19.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.977+0000 7febc3400640 1 --2- 172.21.15.86:0/2910863533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feba8004d50 0x7feba8005160 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7febbc009f90 tx=0x7febbc01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b433811f71f937a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:19.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.977+0000 7febc2000640 1 -- 172.21.15.86:0/2910863533 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc0b2070 con 0x7feba8004d50 2024-04-25T14:11:19.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.977+0000 7febc2000640 1 -- 172.21.15.86:0/2910863533 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febbc0a5c90 con 0x7feba8004d50 2024-04-25T14:11:19.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.977+0000 7febc2000640 1 -- 172.21.15.86:0/2910863533 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc009c10 con 0x7feba8004d50 2024-04-25T14:11:19.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.977+0000 7febc9600640 1 -- 172.21.15.86:0/2910863533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feba8004d50 msgr2=0x7feba8005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:19.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.977+0000 7febc9600640 1 --2- 172.21.15.86:0/2910863533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feba8004d50 0x7feba8005160 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7febbc009f90 tx=0x7febbc01cc80 comp rx=0 tx=0).stop 2024-04-25T14:11:19.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.977+0000 7febc3400640 1 -- 172.21.15.86:0/2910863533 reap_dead start 2024-04-25T14:11:19.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.977+0000 7febc9600640 1 -- 172.21.15.86:0/2910863533 shutdown_connections 2024-04-25T14:11:19.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.977+0000 7febc9600640 1 -- 172.21.15.86:0/2910863533 >> 172.21.15.86:0/2910863533 conn(0x7febc40b4650 msgr2=0x7febc40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:19.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.977+0000 7febc9600640 1 -- 172.21.15.86:0/2910863533 shutdown_connections 2024-04-25T14:11:19.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.977+0000 7febc9600640 1 -- 172.21.15.86:0/2910863533 wait complete. 2024-04-25T14:11:19.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.981+0000 7febc9600640 1 Processor -- start 2024-04-25T14:11:19.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.981+0000 7febc9600640 1 -- start start 2024-04-25T14:11:19.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.981+0000 7febc9600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feba8004d50 0x7febc40af700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:19.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.981+0000 7febc9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc40b3540 0x7febc4157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:19.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.981+0000 7febc9600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febc4157c50 0x7febc409a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:19.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.981+0000 7febc9600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7febc415c6b0 con 0x7febc40b3540 2024-04-25T14:11:19.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.981+0000 7febc9600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7febc40d0fc0 con 0x7feba8004d50 2024-04-25T14:11:19.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.981+0000 7febc9600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7febc415c830 con 0x7febc4157c50 2024-04-25T14:11:19.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.981+0000 7febba000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc40b3540 0x7febc4157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:19.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.981+0000 7febba000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc40b3540 0x7febc4157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49588/0 (socket says 172.21.15.86:49588) 2024-04-25T14:11:19.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.981+0000 7febba000640 1 -- 172.21.15.86:0/2543921597 learned_addr learned my addr 172.21.15.86:0/2543921597 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:19.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.981+0000 7febc3400640 1 --2- 172.21.15.86:0/2543921597 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febc4157c50 0x7febc409a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:19.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.981+0000 7febc2a00640 1 --2- 172.21.15.86:0/2543921597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feba8004d50 0x7febc40af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:19.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.981+0000 7febba000640 1 -- 172.21.15.86:0/2543921597 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febc4157c50 msgr2=0x7febc409a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:19.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.981+0000 7febba000640 1 --2- 172.21.15.86:0/2543921597 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febc4157c50 0x7febc409a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.981+0000 7febba000640 1 -- 172.21.15.86:0/2543921597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feba8004d50 msgr2=0x7febc40af700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:19.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.981+0000 7febba000640 1 --2- 172.21.15.86:0/2543921597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feba8004d50 0x7febc40af700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:19.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.981+0000 7febba000640 1 -- 172.21.15.86:0/2543921597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc4158060 con 0x7febc40b3540 2024-04-25T14:11:19.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.981+0000 7febba000640 1 --2- 172.21.15.86:0/2543921597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc40b3540 0x7febc4157710 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7febac00f430 tx=0x7febac00f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:19.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.981+0000 7febbb400640 1 -- 172.21.15.86:0/2543921597 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febac007d00 con 0x7febc40b3540 2024-04-25T14:11:19.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.981+0000 7febbb400640 1 -- 172.21.15.86:0/2543921597 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febac00a0d0 con 0x7febc40b3540 2024-04-25T14:11:19.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.981+0000 7febc9600640 1 -- 172.21.15.86:0/2543921597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc40740c0 con 0x7febc40b3540 2024-04-25T14:11:19.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.981+0000 7febbb400640 1 -- 172.21.15.86:0/2543921597 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febac00a540 con 0x7febc40b3540 2024-04-25T14:11:19.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.985+0000 7febc9600640 1 -- 172.21.15.86:0/2543921597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc4074680 con 0x7febc40b3540 2024-04-25T14:11:19.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.985+0000 7febc9600640 1 -- 172.21.15.86:0/2543921597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febc40b2e00 con 0x7febc40b3540 2024-04-25T14:11:19.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.985+0000 7febbb400640 1 -- 172.21.15.86:0/2543921597 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 65) ==== 93253+0+0 (secure 0 0 0) 0x7febac014070 con 0x7febc40b3540 2024-04-25T14:11:19.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.985+0000 7febbb400640 1 --2- 172.21.15.86:0/2543921597 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7feba006f4c0 0x7feba00719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:19.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.985+0000 7febbb400640 1 -- 172.21.15.86:0/2543921597 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6832+0+0 (secure 0 0 0) 0x7febac092260 con 0x7febc40b3540 2024-04-25T14:11:19.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.985+0000 7febc2a00640 1 --2- 172.21.15.86:0/2543921597 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7feba006f4c0 0x7feba00719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:19.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.985+0000 7febc2a00640 1 --2- 172.21.15.86:0/2543921597 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7feba006f4c0 0x7feba00719d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7febb40047a0 tx=0x7febb4008040 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:19.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:19.989+0000 7febbb400640 1 -- 172.21.15.86:0/2543921597 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7febac05ed20 con 0x7febc40b3540 2024-04-25T14:11:20.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.177+0000 7febc9600640 1 -- 172.21.15.86:0/2543921597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febc4069d00 con 0x7febc40b3540 2024-04-25T14:11:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.181+0000 7febbb400640 1 -- 172.21.15.86:0/2543921597 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5186 (secure 0 0 0) 0x7febac05e620 con 0x7febc40b3540 2024-04-25T14:11:20.177 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:20.177 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":94,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2258481468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2258481468},{"type":"v1","addr":"172.21.15.155:6834","nonce":2258481468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":8976,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1064843151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1064843151},{"type":"v1","addr":"172.21.15.86:6835","nonce":1064843151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":8985,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1940755625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1940755625},{"type":"v1","addr":"172.21.15.86:6837","nonce":1940755625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:10:45.497688+0000","modified":"2024-04-25T14:10:48.475869+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8991":{"gid":8991,"name":"b","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6835/544985680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":544985680},{"type":"v1","addr":"172.21.15.155:6835","nonce":544985680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8991,"qdb_cluster":[8991]},"id":9}]} 2024-04-25T14:11:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 94 2024-04-25T14:11:20.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.181+0000 7febc9600640 1 -- 172.21.15.86:0/2543921597 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7feba006f4c0 msgr2=0x7feba00719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:20.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.181+0000 7febc9600640 1 --2- 172.21.15.86:0/2543921597 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7feba006f4c0 0x7feba00719d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7febb40047a0 tx=0x7febb4008040 comp rx=0 tx=0).stop 2024-04-25T14:11:20.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.181+0000 7febc9600640 1 -- 172.21.15.86:0/2543921597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc40b3540 msgr2=0x7febc4157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:20.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.181+0000 7febc9600640 1 --2- 172.21.15.86:0/2543921597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc40b3540 0x7febc4157710 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7febac00f430 tx=0x7febac00f8f0 comp rx=0 tx=0).stop 2024-04-25T14:11:20.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.181+0000 7febc9600640 1 -- 172.21.15.86:0/2543921597 shutdown_connections 2024-04-25T14:11:20.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.181+0000 7febc9600640 1 --2- 172.21.15.86:0/2543921597 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7feba006f4c0 0x7feba00719d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:20.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.181+0000 7febc9600640 1 --2- 172.21.15.86:0/2543921597 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febc4157c50 0x7febc409a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:20.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.181+0000 7febc9600640 1 --2- 172.21.15.86:0/2543921597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc40b3540 0x7febc4157710 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:20.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.181+0000 7febc9600640 1 --2- 172.21.15.86:0/2543921597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feba8004d50 0x7febc40af700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:20.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.181+0000 7febc9600640 1 -- 172.21.15.86:0/2543921597 >> 172.21.15.86:0/2543921597 conn(0x7febc40b4650 msgr2=0x7febc415ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:20.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.181+0000 7febc9600640 1 -- 172.21.15.86:0/2543921597 shutdown_connections 2024-04-25T14:11:20.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.181+0000 7febc9600640 1 -- 172.21.15.86:0/2543921597 wait complete. 2024-04-25T14:11:20.198 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:11:20.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.389+0000 7fea43c00640 1 Processor -- start 2024-04-25T14:11:20.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.389+0000 7fea43c00640 1 -- start start 2024-04-25T14:11:20.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.393+0000 7fea43c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fea3c0cfca0 0x7fea3c06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:20.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.393+0000 7fea43c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fea3c0730b0 0x7fea3c0734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:20.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.393+0000 7fea43c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea3c072bd0 con 0x7fea3c0d16e0 2024-04-25T14:11:20.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.393+0000 7fea43c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fea3c070470 con 0x7fea3c0cfca0 2024-04-25T14:11:20.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.393+0000 7fea43c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fea3c06f930 con 0x7fea3c0730b0 2024-04-25T14:11:20.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.393+0000 7fea41200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fea3c0cfca0 0x7fea3c06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:20.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.393+0000 7fea41200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fea3c0cfca0 0x7fea3c06f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49592/0 (socket says 172.21.15.86:49592) 2024-04-25T14:11:20.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.393+0000 7fea41200640 1 -- 172.21.15.86:0/620389086 learned_addr learned my addr 172.21.15.86:0/620389086 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:20.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.393+0000 7fea41c00640 1 --2- 172.21.15.86:0/620389086 >> v2:172.21.15.155:3300/0 conn(0x7fea3c0730b0 0x7fea3c0734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:20.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.393+0000 7fea41200640 1 -- 172.21.15.86:0/620389086 >> v2:172.21.15.155:3300/0 conn(0x7fea3c0730b0 msgr2=0x7fea3c0734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:20.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.393+0000 7fea41200640 1 --2- 172.21.15.86:0/620389086 >> v2:172.21.15.155:3300/0 conn(0x7fea3c0730b0 0x7fea3c0734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:20.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.393+0000 7fea41200640 1 -- 172.21.15.86:0/620389086 >> v1:172.21.15.155:6789/0 conn(0x7fea3c0d16e0 legacy=0x7fea3c0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:20.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.393+0000 7fea41200640 1 -- 172.21.15.86:0/620389086 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3c06fbd0 con 0x7fea3c0cfca0 2024-04-25T14:11:20.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.397+0000 7fea41200640 1 --2- 172.21.15.86:0/620389086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea3c0cfca0 0x7fea3c06f3f0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0cb8c0 tx=0x7fea2c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471bb397bae9bfd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:20.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.397+0000 7fea3be00640 1 -- 172.21.15.86:0/620389086 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c0ab430 con 0x7fea3c0cfca0 2024-04-25T14:11:20.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.397+0000 7fea3be00640 1 -- 172.21.15.86:0/620389086 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea2c0b0070 con 0x7fea3c0cfca0 2024-04-25T14:11:20.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.397+0000 7fea3be00640 1 -- 172.21.15.86:0/620389086 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c0ab7f0 con 0x7fea3c0cfca0 2024-04-25T14:11:20.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.397+0000 7fea43c00640 1 -- 172.21.15.86:0/620389086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea3c0cfca0 msgr2=0x7fea3c06f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:20.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.397+0000 7fea43c00640 1 --2- 172.21.15.86:0/620389086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea3c0cfca0 0x7fea3c06f3f0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0cb8c0 tx=0x7fea2c01c520 comp rx=0 tx=0).stop 2024-04-25T14:11:20.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.397+0000 7fea43c00640 1 -- 172.21.15.86:0/620389086 shutdown_connections 2024-04-25T14:11:20.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.397+0000 7fea43c00640 1 --2- 172.21.15.86:0/620389086 >> v2:172.21.15.155:3300/0 conn(0x7fea3c0730b0 0x7fea3c0734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:20.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.397+0000 7fea43c00640 1 --2- 172.21.15.86:0/620389086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea3c0cfca0 0x7fea3c06f3f0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:20.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.397+0000 7fea43c00640 1 -- 172.21.15.86:0/620389086 >> 172.21.15.86:0/620389086 conn(0x7fea3c0b4650 msgr2=0x7fea3c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:20.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.397+0000 7fea43c00640 1 -- 172.21.15.86:0/620389086 shutdown_connections 2024-04-25T14:11:20.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.397+0000 7fea43c00640 1 -- 172.21.15.86:0/620389086 wait complete. 2024-04-25T14:11:20.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.397+0000 7fea43c00640 1 Processor -- start 2024-04-25T14:11:20.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.397+0000 7fea43c00640 1 -- start start 2024-04-25T14:11:20.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.401+0000 7fea43c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea3c0730b0 0x7fea3c0aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:20.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.401+0000 7fea43c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea3c0cfca0 0x7fea3c099bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:20.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.401+0000 7fea43c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3c0d16e0 0x7fea3c1541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:20.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.401+0000 7fea43c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c169a70 con 0x7fea3c0cfca0 2024-04-25T14:11:20.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.401+0000 7fea43c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c1698f0 con 0x7fea3c0730b0 2024-04-25T14:11:20.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.401+0000 7fea43c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fea3c169bf0 con 0x7fea3c0d16e0 2024-04-25T14:11:20.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.401+0000 7fea33e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea3c0cfca0 0x7fea3c099bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:20.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.401+0000 7fea33e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea3c0cfca0 0x7fea3c099bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49608/0 (socket says 172.21.15.86:49608) 2024-04-25T14:11:20.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.401+0000 7fea33e00640 1 -- 172.21.15.86:0/3510787166 learned_addr learned my addr 172.21.15.86:0/3510787166 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:20.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.401+0000 7fea41200640 1 --2- 172.21.15.86:0/3510787166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea3c0730b0 0x7fea3c0aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:20.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.401+0000 7fea41c00640 1 --2- 172.21.15.86:0/3510787166 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3c0d16e0 0x7fea3c1541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:20.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.401+0000 7fea33e00640 1 -- 172.21.15.86:0/3510787166 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3c0d16e0 msgr2=0x7fea3c1541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:20.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.401+0000 7fea33e00640 1 --2- 172.21.15.86:0/3510787166 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3c0d16e0 0x7fea3c1541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:20.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.401+0000 7fea33e00640 1 -- 172.21.15.86:0/3510787166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea3c0730b0 msgr2=0x7fea3c0aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:20.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.401+0000 7fea33e00640 1 --2- 172.21.15.86:0/3510787166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea3c0730b0 0x7fea3c0aec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:20.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.401+0000 7fea33e00640 1 -- 172.21.15.86:0/3510787166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3c0af840 con 0x7fea3c0cfca0 2024-04-25T14:11:20.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.401+0000 7fea41c00640 1 --2- 172.21.15.86:0/3510787166 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3c0d16e0 0x7fea3c1541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:20.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.401+0000 7fea33e00640 1 --2- 172.21.15.86:0/3510787166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea3c0cfca0 0x7fea3c099bf0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fea2400c370 tx=0x7fea2400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:20.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.401+0000 7fea39600640 1 -- 172.21.15.86:0/3510787166 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea240052d0 con 0x7fea3c0cfca0 2024-04-25T14:11:20.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.401+0000 7fea39600640 1 -- 172.21.15.86:0/3510787166 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea24005470 con 0x7fea3c0cfca0 2024-04-25T14:11:20.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.401+0000 7fea39600640 1 -- 172.21.15.86:0/3510787166 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea24007260 con 0x7fea3c0cfca0 2024-04-25T14:11:20.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.401+0000 7fea43c00640 1 -- 172.21.15.86:0/3510787166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea3c0afb20 con 0x7fea3c0cfca0 2024-04-25T14:11:20.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.401+0000 7fea43c00640 1 -- 172.21.15.86:0/3510787166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea3c0ae410 con 0x7fea3c0cfca0 2024-04-25T14:11:20.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.401+0000 7fea43c00640 1 -- 172.21.15.86:0/3510787166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea3c09f4b0 con 0x7fea3c0cfca0 2024-04-25T14:11:20.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.405+0000 7fea39600640 1 -- 172.21.15.86:0/3510787166 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 65) ==== 93253+0+0 (secure 0 0 0) 0x7fea24007400 con 0x7fea3c0cfca0 2024-04-25T14:11:20.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.405+0000 7fea39600640 1 --2- 172.21.15.86:0/3510787166 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fea1806f490 0x7fea180719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:20.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.405+0000 7fea39600640 1 -- 172.21.15.86:0/3510787166 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6832+0+0 (secure 0 0 0) 0x7fea2408ec50 con 0x7fea3c0cfca0 2024-04-25T14:11:20.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.405+0000 7fea41200640 1 --2- 172.21.15.86:0/3510787166 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fea1806f490 0x7fea180719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:20.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.405+0000 7fea41200640 1 --2- 172.21.15.86:0/3510787166 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fea1806f490 0x7fea180719a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0af000 tx=0x7fea2c0a7000 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:20.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.409+0000 7fea39600640 1 -- 172.21.15.86:0/3510787166 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fea2405afc0 con 0x7fea3c0cfca0 2024-04-25T14:11:20.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.565+0000 7fea43c00640 1 -- 172.21.15.86:0/3510787166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea3c069d00 con 0x7fea3c0cfca0 2024-04-25T14:11:20.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.565+0000 7fea39600640 1 -- 172.21.15.86:0/3510787166 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+17961 (secure 0 0 0) 0x7fea2405f2c0 con 0x7fea3c0cfca0 2024-04-25T14:11:20.564 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:20.564 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":129,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:11:16.711086+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:10:43.913889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T14:10:44.419362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6833/2221276863":"2024-04-26T14:11:16.711047+0000","172.21.15.86:6832/2221276863":"2024-04-26T14:11:16.711047+0000","172.21.15.86:0/3342732660":"2024-04-26T14:11:16.711047+0000","172.21.15.86:0/1871957865":"2024-04-26T14:11:16.711047+0000","172.21.15.86:0/2615931231":"2024-04-26T14:11:16.711047+0000","172.21.15.86:0/1695622468":"2024-04-26T14:11:16.711047+0000","172.21.15.86:0/3537150562":"2024-04-26T14:11:16.711047+0000","172.21.15.86:0/506566715":"2024-04-26T14:11:16.711047+0000","172.21.15.86:0/3961472441":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/287404215":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/889708173":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/3224392570":"2024-04-26T14:11:04.532513+0000","172.21.15.86:6833/3964666460":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/553620002":"2024-04-26T14:11:04.532513+0000","172.21.15.86:6832/3964666460":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/3726306847":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/2412030790":"2024-04-25T15:10:55.520329+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:11:20.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.569+0000 7fea43c00640 1 -- 172.21.15.86:0/3510787166 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fea1806f490 msgr2=0x7fea180719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:20.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.569+0000 7fea43c00640 1 --2- 172.21.15.86:0/3510787166 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fea1806f490 0x7fea180719a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0af000 tx=0x7fea2c0a7000 comp rx=0 tx=0).stop 2024-04-25T14:11:20.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.569+0000 7fea43c00640 1 -- 172.21.15.86:0/3510787166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea3c0cfca0 msgr2=0x7fea3c099bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:20.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.569+0000 7fea43c00640 1 --2- 172.21.15.86:0/3510787166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea3c0cfca0 0x7fea3c099bf0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fea2400c370 tx=0x7fea2400c830 comp rx=0 tx=0).stop 2024-04-25T14:11:20.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.569+0000 7fea43c00640 1 -- 172.21.15.86:0/3510787166 shutdown_connections 2024-04-25T14:11:20.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.569+0000 7fea43c00640 1 --2- 172.21.15.86:0/3510787166 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fea1806f490 0x7fea180719a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:20.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.569+0000 7fea43c00640 1 --2- 172.21.15.86:0/3510787166 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3c0d16e0 0x7fea3c1541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:20.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.569+0000 7fea43c00640 1 --2- 172.21.15.86:0/3510787166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea3c0cfca0 0x7fea3c099bf0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:20.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.569+0000 7fea43c00640 1 --2- 172.21.15.86:0/3510787166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea3c0730b0 0x7fea3c0aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:20.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.569+0000 7fea43c00640 1 -- 172.21.15.86:0/3510787166 >> 172.21.15.86:0/3510787166 conn(0x7fea3c0b4650 msgr2=0x7fea3c15c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:20.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.569+0000 7fea43c00640 1 -- 172.21.15.86:0/3510787166 shutdown_connections 2024-04-25T14:11:20.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.569+0000 7fea43c00640 1 -- 172.21.15.86:0/3510787166 wait complete. 2024-04-25T14:11:20.586 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T14:11:20.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.737+0000 7f0887c00640 1 Processor -- start 2024-04-25T14:11:20.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.737+0000 7f0887c00640 1 -- start start 2024-04-25T14:11:20.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.741+0000 7f0887c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f08800b3950 0x7f088015cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:20.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.741+0000 7f0887c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08801607a0 0x7f0880160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:20.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.741+0000 7f0887c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08800b3d60 con 0x7f08801610f0 2024-04-25T14:11:20.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.741+0000 7f0887c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0880161500 con 0x7f08800b3950 2024-04-25T14:11:20.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.741+0000 7f0887c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f08800af140 con 0x7f08801607a0 2024-04-25T14:11:20.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.741+0000 7f0885200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f08800b3950 0x7f088015cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:20.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.741+0000 7f0885200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f08800b3950 0x7f088015cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49618/0 (socket says 172.21.15.86:49618) 2024-04-25T14:11:20.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.741+0000 7f0885200640 1 -- 172.21.15.86:0/2216863486 learned_addr learned my addr 172.21.15.86:0/2216863486 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:20.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.741+0000 7f087fe00640 1 --2- 172.21.15.86:0/2216863486 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08801607a0 0x7f0880160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:20.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.741+0000 7f0885200640 1 -- 172.21.15.86:0/2216863486 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08801607a0 msgr2=0x7f0880160bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:20.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.741+0000 7f0885200640 1 --2- 172.21.15.86:0/2216863486 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08801607a0 0x7f0880160bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:20.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.741+0000 7f0885200640 1 -- 172.21.15.86:0/2216863486 >> v1:172.21.15.155:6789/0 conn(0x7f08801610f0 legacy=0x7f088006d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:20.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.741+0000 7f0885200640 1 -- 172.21.15.86:0/2216863486 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08800b1d50 con 0x7f08800b3950 2024-04-25T14:11:20.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.745+0000 7f0885200640 1 --2- 172.21.15.86:0/2216863486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08800b3950 0x7f088015cc90 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f0874009860 tx=0x7f087401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be819b72e40598d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.749+0000 7f087f400640 1 -- 172.21.15.86:0/2216863486 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08740b1070 con 0x7f08800b3950 2024-04-25T14:11:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.749+0000 7f087f400640 1 -- 172.21.15.86:0/2216863486 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08740ac620 con 0x7f08800b3950 2024-04-25T14:11:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.749+0000 7f087f400640 1 -- 172.21.15.86:0/2216863486 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08740ac9f0 con 0x7f08800b3950 2024-04-25T14:11:20.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.749+0000 7f0887c00640 1 -- 172.21.15.86:0/2216863486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08800b3950 msgr2=0x7f088015cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:20.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.749+0000 7f0887c00640 1 --2- 172.21.15.86:0/2216863486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08800b3950 0x7f088015cc90 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f0874009860 tx=0x7f087401c4b0 comp rx=0 tx=0).stop 2024-04-25T14:11:20.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.749+0000 7f0887c00640 1 -- 172.21.15.86:0/2216863486 shutdown_connections 2024-04-25T14:11:20.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.749+0000 7f0887c00640 1 --2- 172.21.15.86:0/2216863486 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08801607a0 0x7f0880160bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:20.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.749+0000 7f0887c00640 1 --2- 172.21.15.86:0/2216863486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08800b3950 0x7f088015cc90 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:20.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.749+0000 7f0887c00640 1 -- 172.21.15.86:0/2216863486 >> 172.21.15.86:0/2216863486 conn(0x7f088009ba30 msgr2=0x7f08800b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:20.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.749+0000 7f0887c00640 1 -- 172.21.15.86:0/2216863486 shutdown_connections 2024-04-25T14:11:20.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.749+0000 7f0887c00640 1 -- 172.21.15.86:0/2216863486 wait complete. 2024-04-25T14:11:20.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.753+0000 7f0887c00640 1 Processor -- start 2024-04-25T14:11:20.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.753+0000 7f0887c00640 1 -- start start 2024-04-25T14:11:20.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.753+0000 7f0887c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08800b3950 0x7f088014eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:20.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.753+0000 7f0887c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08801607a0 0x7f0880153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:20.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.753+0000 7f0887c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08801610f0 0x7f0880154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:20.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.753+0000 7f0887c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0880169e70 con 0x7f08800b3950 2024-04-25T14:11:20.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.753+0000 7f0887c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0880169cf0 con 0x7f08801607a0 2024-04-25T14:11:20.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.753+0000 7f0887c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0880169ff0 con 0x7f08801610f0 2024-04-25T14:11:20.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.753+0000 7f0885200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08800b3950 0x7f088014eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:20.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.753+0000 7f0885200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08800b3950 0x7f088014eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49620/0 (socket says 172.21.15.86:49620) 2024-04-25T14:11:20.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.753+0000 7f0885200640 1 -- 172.21.15.86:0/574139846 learned_addr learned my addr 172.21.15.86:0/574139846 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:20.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.753+0000 7f0885c00640 1 --2- 172.21.15.86:0/574139846 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08801610f0 0x7f0880154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:20.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.753+0000 7f087fe00640 1 --2- 172.21.15.86:0/574139846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08801607a0 0x7f0880153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:20.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.753+0000 7f0885200640 1 -- 172.21.15.86:0/574139846 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08801610f0 msgr2=0x7f0880154f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:20.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.753+0000 7f0885200640 1 --2- 172.21.15.86:0/574139846 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08801610f0 0x7f0880154f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:20.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.753+0000 7f0885200640 1 -- 172.21.15.86:0/574139846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08801607a0 msgr2=0x7f0880153230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:20.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.753+0000 7f0885200640 1 --2- 172.21.15.86:0/574139846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08801607a0 0x7f0880153230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:20.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.753+0000 7f0885200640 1 -- 172.21.15.86:0/574139846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0880152c90 con 0x7f08800b3950 2024-04-25T14:11:20.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.753+0000 7f0885200640 1 --2- 172.21.15.86:0/574139846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08800b3950 0x7f088014eac0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f08740af040 tx=0x7f0874019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:20.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.753+0000 7f087cc00640 1 -- 172.21.15.86:0/574139846 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08740bc070 con 0x7f08800b3950 2024-04-25T14:11:20.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.753+0000 7f087cc00640 1 -- 172.21.15.86:0/574139846 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08740b7430 con 0x7f08800b3950 2024-04-25T14:11:20.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.753+0000 7f0887c00640 1 -- 172.21.15.86:0/574139846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08800d0790 con 0x7f08800b3950 2024-04-25T14:11:20.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.753+0000 7f087cc00640 1 -- 172.21.15.86:0/574139846 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08740b1040 con 0x7f08800b3950 2024-04-25T14:11:20.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.753+0000 7f0887c00640 1 -- 172.21.15.86:0/574139846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08800d0d50 con 0x7f08800b3950 2024-04-25T14:11:20.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.757+0000 7f0887c00640 1 -- 172.21.15.86:0/574139846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08800a04c0 con 0x7f08800b3950 2024-04-25T14:11:20.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.757+0000 7f087cc00640 1 -- 172.21.15.86:0/574139846 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7f08740bf070 con 0x7f08800b3950 2024-04-25T14:11:20.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.757+0000 7f087cc00640 1 --2- 172.21.15.86:0/574139846 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f086006f520 0x7f0860071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:20.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.757+0000 7f087cc00640 1 -- 172.21.15.86:0/574139846 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6832+0+0 (secure 0 0 0) 0x7f0874129110 con 0x7f08800b3950 2024-04-25T14:11:20.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.757+0000 7f087fe00640 1 --2- 172.21.15.86:0/574139846 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f086006f520 0x7f0860071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:20.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.761+0000 7f087fe00640 1 --2- 172.21.15.86:0/574139846 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f086006f520 0x7f0860071a30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f08800715c0 tx=0x7f086c008040 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:20.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.765+0000 7f087cc00640 1 -- 172.21.15.86:0/574139846 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0874016130 con 0x7f08800b3950 2024-04-25T14:11:20.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.953+0000 7f0887c00640 1 -- 172.21.15.86:0/574139846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0880076bf0 con 0x7f08800b3950 2024-04-25T14:11:20.954 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-25T14:11:20.953+0000 7fea29c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:11:20.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.957+0000 7f087cc00640 1 -- 172.21.15.86:0/574139846 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v95) ==== 161+0+0 (secure 0 0 0) 0x7f08740f5460 con 0x7f08800b3950 2024-04-25T14:11:20.955 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T14:11:20.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.961+0000 7f0887c00640 1 -- 172.21.15.86:0/574139846 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f086006f520 msgr2=0x7f0860071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:20.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.961+0000 7f0887c00640 1 --2- 172.21.15.86:0/574139846 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f086006f520 0x7f0860071a30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f08800715c0 tx=0x7f086c008040 comp rx=0 tx=0).stop 2024-04-25T14:11:20.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.961+0000 7f0887c00640 1 -- 172.21.15.86:0/574139846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08800b3950 msgr2=0x7f088014eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:20.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.961+0000 7f0887c00640 1 --2- 172.21.15.86:0/574139846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08800b3950 0x7f088014eac0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f08740af040 tx=0x7f0874019250 comp rx=0 tx=0).stop 2024-04-25T14:11:20.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.961+0000 7f0887c00640 1 -- 172.21.15.86:0/574139846 shutdown_connections 2024-04-25T14:11:20.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.961+0000 7f0887c00640 1 --2- 172.21.15.86:0/574139846 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f086006f520 0x7f0860071a30 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:20.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.961+0000 7f0887c00640 1 --2- 172.21.15.86:0/574139846 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08801610f0 0x7f0880154f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:20.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.961+0000 7f0887c00640 1 --2- 172.21.15.86:0/574139846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08801607a0 0x7f0880153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:20.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.961+0000 7f0887c00640 1 --2- 172.21.15.86:0/574139846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08800b3950 0x7f088014eac0 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:20.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.961+0000 7f0887c00640 1 -- 172.21.15.86:0/574139846 >> 172.21.15.86:0/574139846 conn(0x7f088009ba30 msgr2=0x7f088015ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:20.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.961+0000 7f0887c00640 1 -- 172.21.15.86:0/574139846 shutdown_connections 2024-04-25T14:11:20.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:20.961+0000 7f0887c00640 1 -- 172.21.15.86:0/574139846 wait complete. 2024-04-25T14:11:20.967 INFO:tasks.ceph.mds.b.smithi155.stderr: -4705> 2024-04-25T14:10:48.023+0000 7f3bf3400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T14:11:20.978 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T14:11:21.089 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2024-04-25T14:11:21.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.129+0000 7f14ac400640 1 Processor -- start 2024-04-25T14:11:21.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.129+0000 7f14ac400640 1 -- start start 2024-04-25T14:11:21.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.133+0000 7f14ac400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f14a4070330 0x7f14a40d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:21.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.133+0000 7f14ac400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f14a40d1c20 0x7f14a406ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:21.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.133+0000 7f14ac400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14a40d2030 con 0x7f14a406f270 2024-04-25T14:11:21.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.133+0000 7f14ac400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f14a415e5c0 con 0x7f14a4070330 2024-04-25T14:11:21.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.133+0000 7f14ac400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f14a415e8a0 con 0x7f14a40d1c20 2024-04-25T14:11:21.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.133+0000 7f14a9a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f14a4070330 0x7f14a40d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:21.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.133+0000 7f14aa400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f14a406f270 0x7f14a406f680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39084/0 (socket says 172.21.15.86:39084) 2024-04-25T14:11:21.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.133+0000 7f14a9a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f14a4070330 0x7f14a40d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49624/0 (socket says 172.21.15.86:49624) 2024-04-25T14:11:21.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.133+0000 7f14aa400640 1 -- 172.21.15.86:0/2054928995 learned_addr learned my addr 172.21.15.86:0/2054928995 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:21.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.133+0000 7f14a9000640 1 --2- 172.21.15.86:0/2054928995 >> v2:172.21.15.155:3300/0 conn(0x7f14a40d1c20 0x7f14a406ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:21.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.133+0000 7f14a9a00640 1 -- 172.21.15.86:0/2054928995 >> v2:172.21.15.155:3300/0 conn(0x7f14a40d1c20 msgr2=0x7f14a406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:21.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.133+0000 7f14a9a00640 1 --2- 172.21.15.86:0/2054928995 >> v2:172.21.15.155:3300/0 conn(0x7f14a40d1c20 0x7f14a406ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:21.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.133+0000 7f14a9a00640 1 -- 172.21.15.86:0/2054928995 >> v1:172.21.15.86:6789/0 conn(0x7f14a406f270 legacy=0x7f14a406f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:21.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.133+0000 7f14a9a00640 1 -- 172.21.15.86:0/2054928995 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a415eb80 con 0x7f14a4070330 2024-04-25T14:11:21.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.133+0000 7f14a3e00640 1 -- 172.21.15.86:0/2054928995 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1230734571 0 0) 0x7f149c0014a0 con 0x7f14a406f270 2024-04-25T14:11:21.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.133+0000 7f14a9a00640 1 --2- 172.21.15.86:0/2054928995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a4070330 0x7f14a40d16e0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f14980097c0 tx=0x7f149801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1fbc9d84b15ec6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:21.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.137+0000 7f14a3e00640 1 -- 172.21.15.86:0/2054928995 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14980b0070 con 0x7f14a4070330 2024-04-25T14:11:21.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.137+0000 7f14a3e00640 1 -- 172.21.15.86:0/2054928995 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14980ab600 con 0x7f14a4070330 2024-04-25T14:11:21.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.137+0000 7f14a3e00640 1 -- 172.21.15.86:0/2054928995 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14980a4e60 con 0x7f14a4070330 2024-04-25T14:11:21.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.137+0000 7f14ac400640 1 -- 172.21.15.86:0/2054928995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a4070330 msgr2=0x7f14a40d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:21.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.137+0000 7f14ac400640 1 --2- 172.21.15.86:0/2054928995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a4070330 0x7f14a40d16e0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f14980097c0 tx=0x7f149801c3d0 comp rx=0 tx=0).stop 2024-04-25T14:11:21.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.137+0000 7f14ac400640 1 -- 172.21.15.86:0/2054928995 shutdown_connections 2024-04-25T14:11:21.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.137+0000 7f14ac400640 1 --2- 172.21.15.86:0/2054928995 >> v2:172.21.15.155:3300/0 conn(0x7f14a40d1c20 0x7f14a406ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:21.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.137+0000 7f14ac400640 1 --2- 172.21.15.86:0/2054928995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a4070330 0x7f14a40d16e0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:21.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.137+0000 7f14ac400640 1 -- 172.21.15.86:0/2054928995 >> 172.21.15.86:0/2054928995 conn(0x7f14a40b4650 msgr2=0x7f14a40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:21.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.137+0000 7f14ac400640 1 -- 172.21.15.86:0/2054928995 shutdown_connections 2024-04-25T14:11:21.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.137+0000 7f14ac400640 1 -- 172.21.15.86:0/2054928995 wait complete. 2024-04-25T14:11:21.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.137+0000 7f14ac400640 1 Processor -- start 2024-04-25T14:11:21.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.137+0000 7f14ac400640 1 -- start start 2024-04-25T14:11:21.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.141+0000 7f14ac400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14a406f270 0x7f14a409e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:21.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.141+0000 7f14ac400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a4070330 0x7f14a40c5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:21.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.141+0000 7f14ac400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14a40d1c20 0x7f14a409c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:21.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.141+0000 7f14ac400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14a40708c0 con 0x7f14a4070330 2024-04-25T14:11:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.141+0000 7f14ac400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f14a4070740 con 0x7f14a40d1c20 2024-04-25T14:11:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.141+0000 7f14ac400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f14a416afc0 con 0x7f14a406f270 2024-04-25T14:11:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.141+0000 7f14a9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a4070330 0x7f14a40c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.141+0000 7f14a9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a4070330 0x7f14a40c5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49638/0 (socket says 172.21.15.86:49638) 2024-04-25T14:11:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.141+0000 7f14a9000640 1 -- 172.21.15.86:0/1142507771 learned_addr learned my addr 172.21.15.86:0/1142507771 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.141+0000 7f14a9a00640 1 --2- 172.21.15.86:0/1142507771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14a406f270 0x7f14a409e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.141+0000 7f14aa400640 1 --2- 172.21.15.86:0/1142507771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14a40d1c20 0x7f14a409c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.141+0000 7f14a9000640 1 -- 172.21.15.86:0/1142507771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14a406f270 msgr2=0x7f14a409e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.141+0000 7f14a9000640 1 --2- 172.21.15.86:0/1142507771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14a406f270 0x7f14a409e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.141+0000 7f14a9000640 1 -- 172.21.15.86:0/1142507771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14a40d1c20 msgr2=0x7f14a409c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.141+0000 7f14a9000640 1 --2- 172.21.15.86:0/1142507771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14a40d1c20 0x7f14a409c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.141+0000 7f14a9000640 1 -- 172.21.15.86:0/1142507771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a409b6b0 con 0x7f14a4070330 2024-04-25T14:11:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.141+0000 7f14aa400640 1 --2- 172.21.15.86:0/1142507771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14a40d1c20 0x7f14a409c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:21.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.141+0000 7f14a9000640 1 --2- 172.21.15.86:0/1142507771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a4070330 0x7f14a40c5090 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f149000e870 tx=0x7f149000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:21.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.141+0000 7f14a1600640 1 -- 172.21.15.86:0/1142507771 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1490017070 con 0x7f14a4070330 2024-04-25T14:11:21.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.141+0000 7f14a1600640 1 -- 172.21.15.86:0/1142507771 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f149000ee60 con 0x7f14a4070330 2024-04-25T14:11:21.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.141+0000 7f14ac400640 1 -- 172.21.15.86:0/1142507771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14a409b990 con 0x7f14a4070330 2024-04-25T14:11:21.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.141+0000 7f14a1600640 1 -- 172.21.15.86:0/1142507771 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149000cb40 con 0x7f14a4070330 2024-04-25T14:11:21.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.141+0000 7f14ac400640 1 -- 172.21.15.86:0/1142507771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14a409bf50 con 0x7f14a4070330 2024-04-25T14:11:21.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.141+0000 7f14ac400640 1 -- 172.21.15.86:0/1142507771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14a4069d00 con 0x7f14a4070330 2024-04-25T14:11:21.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.145+0000 7f14a1600640 1 -- 172.21.15.86:0/1142507771 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7f149001a070 con 0x7f14a4070330 2024-04-25T14:11:21.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.145+0000 7f14a1600640 1 --2- 172.21.15.86:0/1142507771 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f148406f520 0x7f1484071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:21.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.145+0000 7f14a1600640 1 -- 172.21.15.86:0/1142507771 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6918+0+0 (secure 0 0 0) 0x7f149008fa70 con 0x7f14a4070330 2024-04-25T14:11:21.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.145+0000 7f14a9a00640 1 --2- 172.21.15.86:0/1142507771 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f148406f520 0x7f1484071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:21.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.145+0000 7f14a9a00640 1 --2- 172.21.15.86:0/1142507771 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f148406f520 0x7f1484071a30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f14980ad000 tx=0x7f14980a6000 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:21.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.149+0000 7f14a1600640 1 -- 172.21.15.86:0/1142507771 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f149005bd30 con 0x7f14a4070330 2024-04-25T14:11:21.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.337+0000 7f14ac400640 1 -- 172.21.15.86:0/1142507771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f14a40cc7c0 con 0x7f14a4070330 2024-04-25T14:11:21.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.961+0000 7f14a1600640 1 -- 172.21.15.86:0/1142507771 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v96) ==== 104+0+0 (secure 0 0 0) 0x7f1490060030 con 0x7f14a4070330 2024-04-25T14:11:21.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.961+0000 7f14ac400640 1 -- 172.21.15.86:0/1142507771 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f148406f520 msgr2=0x7f1484071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:21.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.961+0000 7f14ac400640 1 --2- 172.21.15.86:0/1142507771 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f148406f520 0x7f1484071a30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f14980ad000 tx=0x7f14980a6000 comp rx=0 tx=0).stop 2024-04-25T14:11:21.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.961+0000 7f14ac400640 1 -- 172.21.15.86:0/1142507771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a4070330 msgr2=0x7f14a40c5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:21.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.961+0000 7f14ac400640 1 --2- 172.21.15.86:0/1142507771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a4070330 0x7f14a40c5090 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f149000e870 tx=0x7f149000ed30 comp rx=0 tx=0).stop 2024-04-25T14:11:21.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.961+0000 7f14ac400640 1 -- 172.21.15.86:0/1142507771 shutdown_connections 2024-04-25T14:11:21.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.961+0000 7f14ac400640 1 --2- 172.21.15.86:0/1142507771 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f148406f520 0x7f1484071a30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:21.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.961+0000 7f14ac400640 1 --2- 172.21.15.86:0/1142507771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14a40d1c20 0x7f14a409c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:21.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.961+0000 7f14ac400640 1 --2- 172.21.15.86:0/1142507771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14a4070330 0x7f14a40c5090 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:21.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.961+0000 7f14ac400640 1 --2- 172.21.15.86:0/1142507771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14a406f270 0x7f14a409e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:21.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.961+0000 7f14ac400640 1 -- 172.21.15.86:0/1142507771 >> 172.21.15.86:0/1142507771 conn(0x7f14a40b4650 msgr2=0x7f14a40ae130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:21.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.961+0000 7f14ac400640 1 -- 172.21.15.86:0/1142507771 shutdown_connections 2024-04-25T14:11:21.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:21.961+0000 7f14ac400640 1 -- 172.21.15.86:0/1142507771 wait complete. 2024-04-25T14:11:21.981 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T14:11:22.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.137+0000 7f7830c00640 1 Processor -- start 2024-04-25T14:11:22.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.137+0000 7f7830c00640 1 -- start start 2024-04-25T14:11:22.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.137+0000 7f7830c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f782c166b00 0x7f782c168f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:22.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.137+0000 7f7830c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f782c1626c0 con 0x7f782c15e4a0 2024-04-25T14:11:22.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.137+0000 7f7830c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f782c161900 con 0x7f782c0b4f40 2024-04-25T14:11:22.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.137+0000 7f7830c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f782c15efc0 con 0x7f782c166b00 2024-04-25T14:11:22.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.137+0000 7f7821400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f782c15e4a0 0x7f782c15e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39096/0 (socket says 172.21.15.86:39096) 2024-04-25T14:11:22.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.137+0000 7f7821400640 1 -- 172.21.15.86:0/2496894287 learned_addr learned my addr 172.21.15.86:0/2496894287 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:22.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.137+0000 7f782a800640 1 --2- 172.21.15.86:0/2496894287 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f782c166b00 0x7f782c168f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.137+0000 7f7829400640 1 -- 172.21.15.86:0/2496894287 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 864765312 0 0) 0x7f782c1626c0 con 0x7f782c15e4a0 2024-04-25T14:11:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.137+0000 7f7829400640 1 -- 172.21.15.86:0/2496894287 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7810003530 con 0x7f782c15e4a0 2024-04-25T14:11:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.141+0000 7f7829400640 1 -- 172.21.15.86:0/2496894287 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2330930113 0 0) 0x7f782c161900 con 0x7f782c0b4f40 2024-04-25T14:11:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.141+0000 7f7829400640 1 -- 172.21.15.86:0/2496894287 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f782c1626c0 con 0x7f782c0b4f40 2024-04-25T14:11:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.141+0000 7f7829400640 1 -- 172.21.15.86:0/2496894287 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1249270218 0 0) 0x7f7810003530 con 0x7f782c15e4a0 2024-04-25T14:11:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.141+0000 7f7829400640 1 -- 172.21.15.86:0/2496894287 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f782c166b00 msgr2=0x7f782c168f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.141+0000 7f7829400640 1 --2- 172.21.15.86:0/2496894287 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f782c166b00 0x7f782c168f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.141+0000 7f7829400640 1 -- 172.21.15.86:0/2496894287 >> v1:172.21.15.155:6789/0 conn(0x7f782c0b4f40 legacy=0x7f782c0b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.141+0000 7f7829400640 1 -- 172.21.15.86:0/2496894287 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f782c161be0 con 0x7f782c15e4a0 2024-04-25T14:11:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.141+0000 7f7829400640 1 -- 172.21.15.86:0/2496894287 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f7814001600 con 0x7f782c15e4a0 2024-04-25T14:11:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.141+0000 7f7829400640 1 -- 172.21.15.86:0/2496894287 >> v1:172.21.15.86:6789/0 conn(0x7f782c15e4a0 legacy=0x7f782c15e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.141+0000 7f7829400640 1 --2- 172.21.15.86:0/2496894287 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f782c166b00 0x7f7810003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.141+0000 7f7829400640 1 --2- 172.21.15.86:0/2496894287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78100040e0 0x7f7810004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.141+0000 7f7829400640 1 --2- 172.21.15.86:0/2496894287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7810004d50 0x7f7810005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.141+0000 7f7829400640 1 -- 172.21.15.86:0/2496894287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f782c15efc0 con 0x7f7810004d50 2024-04-25T14:11:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.141+0000 7f7829400640 1 -- 172.21.15.86:0/2496894287 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f781000bb30 con 0x7f78100040e0 2024-04-25T14:11:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.141+0000 7f7829400640 1 -- 172.21.15.86:0/2496894287 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f781000bd30 con 0x7f782c166b00 2024-04-25T14:11:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.141+0000 7f782a800640 1 --2- 172.21.15.86:0/2496894287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7810004d50 0x7f7810005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.141+0000 7f7829e00640 1 --2- 172.21.15.86:0/2496894287 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f782c166b00 0x7f7810003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.141+0000 7f7821400640 1 --2- 172.21.15.86:0/2496894287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78100040e0 0x7f7810004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.141+0000 7f782a800640 1 -- 172.21.15.86:0/2496894287 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f782c166b00 msgr2=0x7f7810003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.141+0000 7f782a800640 1 --2- 172.21.15.86:0/2496894287 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f782c166b00 0x7f7810003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:22.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.141+0000 7f782a800640 1 -- 172.21.15.86:0/2496894287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78100040e0 msgr2=0x7f7810004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:22.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.141+0000 7f782a800640 1 --2- 172.21.15.86:0/2496894287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78100040e0 0x7f7810004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:22.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.141+0000 7f782a800640 1 -- 172.21.15.86:0/2496894287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7810003530 con 0x7f7810004d50 2024-04-25T14:11:22.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.141+0000 7f7821400640 1 --2- 172.21.15.86:0/2496894287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78100040e0 0x7f7810004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:22.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.141+0000 7f7829e00640 1 --2- 172.21.15.86:0/2496894287 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f782c166b00 0x7f7810003b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:22.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.141+0000 7f782a800640 1 --2- 172.21.15.86:0/2496894287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7810004d50 0x7f7810005160 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f7824009fc0 tx=0x7f782401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b4e0a729427724 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:22.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.141+0000 7f7829400640 1 -- 172.21.15.86:0/2496894287 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78240b2070 con 0x7f7810004d50 2024-04-25T14:11:22.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.145+0000 7f7829400640 1 -- 172.21.15.86:0/2496894287 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78240a5ba0 con 0x7f7810004d50 2024-04-25T14:11:22.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.145+0000 7f7829400640 1 -- 172.21.15.86:0/2496894287 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7824009c10 con 0x7f7810004d50 2024-04-25T14:11:22.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.145+0000 7f7830c00640 1 -- 172.21.15.86:0/2496894287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7810004d50 msgr2=0x7f7810005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:22.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.145+0000 7f7830c00640 1 --2- 172.21.15.86:0/2496894287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7810004d50 0x7f7810005160 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f7824009fc0 tx=0x7f782401cca0 comp rx=0 tx=0).stop 2024-04-25T14:11:22.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.145+0000 7f782a800640 1 -- 172.21.15.86:0/2496894287 reap_dead start 2024-04-25T14:11:22.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.145+0000 7f7830c00640 1 -- 172.21.15.86:0/2496894287 shutdown_connections 2024-04-25T14:11:22.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.145+0000 7f7830c00640 1 -- 172.21.15.86:0/2496894287 >> 172.21.15.86:0/2496894287 conn(0x7f782c0cff40 msgr2=0x7f782c0d0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:22.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.145+0000 7f7830c00640 1 -- 172.21.15.86:0/2496894287 shutdown_connections 2024-04-25T14:11:22.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.145+0000 7f7830c00640 1 -- 172.21.15.86:0/2496894287 wait complete. 2024-04-25T14:11:22.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.145+0000 7f7830c00640 1 Processor -- start 2024-04-25T14:11:22.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.145+0000 7f7830c00640 1 -- start start 2024-04-25T14:11:22.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.149+0000 7f7830c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7810004d50 0x7f782c150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:22.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.149+0000 7f7830c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f782c0cad40 0x7f782c15f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:22.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.149+0000 7f7830c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f782c15f6b0 0x7f782c0b7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:22.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.149+0000 7f7830c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f782c0d2600 con 0x7f782c0cad40 2024-04-25T14:11:22.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.149+0000 7f7830c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f782c1621a0 con 0x7f782c15f6b0 2024-04-25T14:11:22.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.149+0000 7f7830c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f782c0b92e0 con 0x7f7810004d50 2024-04-25T14:11:22.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.149+0000 7f7821400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f782c0cad40 0x7f782c15f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:22.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.149+0000 7f7821400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f782c0cad40 0x7f782c15f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49660/0 (socket says 172.21.15.86:49660) 2024-04-25T14:11:22.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.149+0000 7f7821400640 1 -- 172.21.15.86:0/4272658834 learned_addr learned my addr 172.21.15.86:0/4272658834 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:22.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.149+0000 7f7829e00640 1 --2- 172.21.15.86:0/4272658834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7810004d50 0x7f782c150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:22.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.149+0000 7f782a800640 1 --2- 172.21.15.86:0/4272658834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f782c15f6b0 0x7f782c0b7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:22.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.149+0000 7f7821400640 1 -- 172.21.15.86:0/4272658834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7810004d50 msgr2=0x7f782c150040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:22.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.149+0000 7f7821400640 1 --2- 172.21.15.86:0/4272658834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7810004d50 0x7f782c150040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:22.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.149+0000 7f7821400640 1 -- 172.21.15.86:0/4272658834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f782c15f6b0 msgr2=0x7f782c0b7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:22.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.149+0000 7f7821400640 1 --2- 172.21.15.86:0/4272658834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f782c15f6b0 0x7f782c0b7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:22.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.149+0000 7f7821400640 1 -- 172.21.15.86:0/4272658834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f782c15fac0 con 0x7f782c0cad40 2024-04-25T14:11:22.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.149+0000 7f7821400640 1 --2- 172.21.15.86:0/4272658834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f782c0cad40 0x7f782c15f170 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f7814011810 tx=0x7f7814011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.149+0000 7f7822a00640 1 -- 172.21.15.86:0/4272658834 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7814010040 con 0x7f782c0cad40 2024-04-25T14:11:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.149+0000 7f7822a00640 1 -- 172.21.15.86:0/4272658834 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f781401a070 con 0x7f782c0cad40 2024-04-25T14:11:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.149+0000 7f7830c00640 1 -- 172.21.15.86:0/4272658834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f782c0b6e10 con 0x7f782c0cad40 2024-04-25T14:11:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.149+0000 7f7822a00640 1 -- 172.21.15.86:0/4272658834 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781400fd60 con 0x7f782c0cad40 2024-04-25T14:11:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.149+0000 7f7830c00640 1 -- 172.21.15.86:0/4272658834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f782c0b73d0 con 0x7f782c0cad40 2024-04-25T14:11:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.149+0000 7f7830c00640 1 -- 172.21.15.86:0/4272658834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f782c06ef30 con 0x7f782c0cad40 2024-04-25T14:11:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.153+0000 7f7822a00640 1 -- 172.21.15.86:0/4272658834 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7f781401f460 con 0x7f782c0cad40 2024-04-25T14:11:22.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.153+0000 7f7822a00640 1 --2- 172.21.15.86:0/4272658834 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f780806f520 0x7f7808071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:22.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.153+0000 7f7822a00640 1 -- 172.21.15.86:0/4272658834 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6918+0+0 (secure 0 0 0) 0x7f7814092a20 con 0x7f782c0cad40 2024-04-25T14:11:22.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.153+0000 7f7829e00640 1 --2- 172.21.15.86:0/4272658834 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f780806f520 0x7f7808071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:22.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.153+0000 7f7829e00640 1 --2- 172.21.15.86:0/4272658834 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f780806f520 0x7f7808071a30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f781c007f60 tx=0x7f781c007ef0 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:22.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.157+0000 7f7822a00640 1 -- 172.21.15.86:0/4272658834 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f782c15fac0 con 0x7f782c0cad40 2024-04-25T14:11:22.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.313+0000 7f7830c00640 1 -- 172.21.15.86:0/4272658834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f782c050750 con 0x7f782c0cad40 2024-04-25T14:11:22.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.965+0000 7f7822a00640 1 -- 172.21.15.86:0/4272658834 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v131) ==== 181+0+0 (secure 0 0 0) 0x7f7814011e60 con 0x7f782c0cad40 2024-04-25T14:11:22.964 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-04-25T14:11:22.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.973+0000 7f7830c00640 1 -- 172.21.15.86:0/4272658834 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f780806f520 msgr2=0x7f7808071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:22.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.973+0000 7f7830c00640 1 --2- 172.21.15.86:0/4272658834 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f780806f520 0x7f7808071a30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f781c007f60 tx=0x7f781c007ef0 comp rx=0 tx=0).stop 2024-04-25T14:11:22.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.973+0000 7f7830c00640 1 -- 172.21.15.86:0/4272658834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f782c0cad40 msgr2=0x7f782c15f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:22.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.973+0000 7f7830c00640 1 --2- 172.21.15.86:0/4272658834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f782c0cad40 0x7f782c15f170 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f7814011810 tx=0x7f7814011c10 comp rx=0 tx=0).stop 2024-04-25T14:11:22.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.973+0000 7f7830c00640 1 -- 172.21.15.86:0/4272658834 shutdown_connections 2024-04-25T14:11:22.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.973+0000 7f7830c00640 1 --2- 172.21.15.86:0/4272658834 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f780806f520 0x7f7808071a30 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:22.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.973+0000 7f7830c00640 1 --2- 172.21.15.86:0/4272658834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f782c15f6b0 0x7f782c0b7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:22.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.973+0000 7f7830c00640 1 --2- 172.21.15.86:0/4272658834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f782c0cad40 0x7f782c15f170 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:22.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.973+0000 7f7830c00640 1 --2- 172.21.15.86:0/4272658834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7810004d50 0x7f782c150040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:22.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.973+0000 7f7830c00640 1 -- 172.21.15.86:0/4272658834 >> 172.21.15.86:0/4272658834 conn(0x7f782c0cff40 msgr2=0x7f782c0cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:22.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.973+0000 7f7830c00640 1 -- 172.21.15.86:0/4272658834 shutdown_connections 2024-04-25T14:11:22.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:22.973+0000 7f7830c00640 1 -- 172.21.15.86:0/4272658834 wait complete. 2024-04-25T14:11:22.993 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T14:11:23.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.149+0000 7f752aa00640 1 Processor -- start 2024-04-25T14:11:23.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.149+0000 7f752aa00640 1 -- start start 2024-04-25T14:11:23.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.149+0000 7f752aa00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f752415d960 0x7f7524161e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:23.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.149+0000 7f752aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f75240cdc20 0x7f75240ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:23.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.149+0000 7f752aa00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75241623c0 con 0x7f75240721f0 2024-04-25T14:11:23.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.149+0000 7f752aa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f752415dd70 con 0x7f75240cdc20 2024-04-25T14:11:23.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.149+0000 7f752aa00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f75240d30e0 con 0x7f752415d960 2024-04-25T14:11:23.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.149+0000 7f7528c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f75240cdc20 0x7f75240ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:23.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.149+0000 7f7528c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f75240cdc20 0x7f75240ce030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49674/0 (socket says 172.21.15.86:49674) 2024-04-25T14:11:23.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.149+0000 7f752a000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f75240721f0 0x7f7524072600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39110/0 (socket says 172.21.15.86:39110) 2024-04-25T14:11:23.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.149+0000 7f7528c00640 1 -- 172.21.15.86:0/4137519805 learned_addr learned my addr 172.21.15.86:0/4137519805 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:23.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.149+0000 7f7529600640 1 --2- 172.21.15.86:0/4137519805 >> v2:172.21.15.155:3300/0 conn(0x7f752415d960 0x7f7524161e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:23.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.149+0000 7f7528c00640 1 -- 172.21.15.86:0/4137519805 >> v2:172.21.15.155:3300/0 conn(0x7f752415d960 msgr2=0x7f7524161e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:23.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.149+0000 7f7528c00640 1 --2- 172.21.15.86:0/4137519805 >> v2:172.21.15.155:3300/0 conn(0x7f752415d960 0x7f7524161e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:23.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.149+0000 7f7528c00640 1 -- 172.21.15.86:0/4137519805 >> v1:172.21.15.86:6789/0 conn(0x7f75240721f0 legacy=0x7f7524072600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:23.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.149+0000 7f7528c00640 1 -- 172.21.15.86:0/4137519805 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75240ce570 con 0x7f75240cdc20 2024-04-25T14:11:23.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.149+0000 7f7523e00640 1 -- 172.21.15.86:0/4137519805 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3339778560 0 0) 0x7f75241623c0 con 0x7f75240721f0 2024-04-25T14:11:23.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.149+0000 7f7529600640 1 --2- 172.21.15.86:0/4137519805 >> v2:172.21.15.155:3300/0 conn(0x7f752415d960 0x7f7524161e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:23.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.153+0000 7f7528c00640 1 --2- 172.21.15.86:0/4137519805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75240cdc20 0x7f75240ce030 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f75240d4380 tx=0x7f751001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f78bc71815647e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:23.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.153+0000 7f7523e00640 1 -- 172.21.15.86:0/4137519805 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75100ab610 con 0x7f75240cdc20 2024-04-25T14:11:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.153+0000 7f7523e00640 1 -- 172.21.15.86:0/4137519805 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75100b0070 con 0x7f75240cdc20 2024-04-25T14:11:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.153+0000 7f7523e00640 1 -- 172.21.15.86:0/4137519805 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75100ab9d0 con 0x7f75240cdc20 2024-04-25T14:11:23.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.153+0000 7f752aa00640 1 -- 172.21.15.86:0/4137519805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75240cdc20 msgr2=0x7f75240ce030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:23.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.153+0000 7f752aa00640 1 --2- 172.21.15.86:0/4137519805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75240cdc20 0x7f75240ce030 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f75240d4380 tx=0x7f751001c520 comp rx=0 tx=0).stop 2024-04-25T14:11:23.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.153+0000 7f752aa00640 1 -- 172.21.15.86:0/4137519805 shutdown_connections 2024-04-25T14:11:23.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.153+0000 7f752aa00640 1 --2- 172.21.15.86:0/4137519805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75240cdc20 0x7f75240ce030 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:23.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.153+0000 7f752aa00640 1 --2- 172.21.15.86:0/4137519805 >> v2:172.21.15.155:3300/0 conn(0x7f752415d960 0x7f7524161e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:23.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.153+0000 7f752aa00640 1 -- 172.21.15.86:0/4137519805 >> 172.21.15.86:0/4137519805 conn(0x7f75240d3330 msgr2=0x7f75240d3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:23.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.153+0000 7f752aa00640 1 -- 172.21.15.86:0/4137519805 shutdown_connections 2024-04-25T14:11:23.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.153+0000 7f752aa00640 1 -- 172.21.15.86:0/4137519805 wait complete. 2024-04-25T14:11:23.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.157+0000 7f752aa00640 1 Processor -- start 2024-04-25T14:11:23.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.157+0000 7f752aa00640 1 -- start start 2024-04-25T14:11:23.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.157+0000 7f752aa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75240721f0 0x7f752414b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:23.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.157+0000 7f752aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75240cdc20 0x7f7524167d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:23.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.157+0000 7f752aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f752415d960 0x7f75240b80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:23.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.157+0000 7f752aa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f75240b61a0 con 0x7f752415d960 2024-04-25T14:11:23.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.157+0000 7f752aa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f75240d1bb0 con 0x7f75240cdc20 2024-04-25T14:11:23.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.157+0000 7f752aa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f752415f980 con 0x7f75240721f0 2024-04-25T14:11:23.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.157+0000 7f752a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f752415d960 0x7f75240b80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:23.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.157+0000 7f752a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f752415d960 0x7f75240b80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49678/0 (socket says 172.21.15.86:49678) 2024-04-25T14:11:23.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.157+0000 7f752a000640 1 -- 172.21.15.86:0/168663559 learned_addr learned my addr 172.21.15.86:0/168663559 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:23.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.157+0000 7f7528c00640 1 --2- 172.21.15.86:0/168663559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75240cdc20 0x7f7524167d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:23.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.157+0000 7f7529600640 1 --2- 172.21.15.86:0/168663559 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75240721f0 0x7f752414b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:23.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.157+0000 7f752a000640 1 -- 172.21.15.86:0/168663559 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75240721f0 msgr2=0x7f752414b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:23.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.157+0000 7f752a000640 1 --2- 172.21.15.86:0/168663559 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75240721f0 0x7f752414b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:23.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.157+0000 7f752a000640 1 -- 172.21.15.86:0/168663559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75240cdc20 msgr2=0x7f7524167d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:23.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.157+0000 7f752a000640 1 --2- 172.21.15.86:0/168663559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75240cdc20 0x7f7524167d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:23.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.157+0000 7f752a000640 1 -- 172.21.15.86:0/168663559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75240ca410 con 0x7f752415d960 2024-04-25T14:11:23.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.157+0000 7f7529600640 1 --2- 172.21.15.86:0/168663559 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75240721f0 0x7f752414b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:23.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.161+0000 7f752a000640 1 --2- 172.21.15.86:0/168663559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f752415d960 0x7f75240b80d0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f751c00c410 tx=0x7f751c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:23.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.161+0000 7f7521600640 1 -- 172.21.15.86:0/168663559 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c0074e0 con 0x7f752415d960 2024-04-25T14:11:23.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.161+0000 7f7521600640 1 -- 172.21.15.86:0/168663559 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f751c0047f0 con 0x7f752415d960 2024-04-25T14:11:23.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.161+0000 7f752aa00640 1 -- 172.21.15.86:0/168663559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75240ca6f0 con 0x7f752415d960 2024-04-25T14:11:23.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.161+0000 7f7521600640 1 -- 172.21.15.86:0/168663559 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c004be0 con 0x7f752415d960 2024-04-25T14:11:23.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.161+0000 7f752aa00640 1 -- 172.21.15.86:0/168663559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75240cacb0 con 0x7f752415d960 2024-04-25T14:11:23.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.161+0000 7f752aa00640 1 -- 172.21.15.86:0/168663559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7524076b80 con 0x7f752415d960 2024-04-25T14:11:23.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.161+0000 7f7521600640 1 -- 172.21.15.86:0/168663559 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7f751c011070 con 0x7f752415d960 2024-04-25T14:11:23.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.161+0000 7f7521600640 1 --2- 172.21.15.86:0/168663559 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f750406f520 0x7f7504071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:23.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.161+0000 7f7521600640 1 -- 172.21.15.86:0/168663559 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 6480+0+0 (secure 0 0 0) 0x7f751c08f240 con 0x7f752415d960 2024-04-25T14:11:23.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.161+0000 7f7529600640 1 --2- 172.21.15.86:0/168663559 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f750406f520 0x7f7504071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:23.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.165+0000 7f7529600640 1 --2- 172.21.15.86:0/168663559 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f750406f520 0x7f7504071a30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7518009aa0 tx=0x7f75180039a0 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:23.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.165+0000 7f7521600640 1 -- 172.21.15.86:0/168663559 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f751c05bdc0 con 0x7f752415d960 2024-04-25T14:11:23.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.325+0000 7f752aa00640 1 -- 172.21.15.86:0/168663559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f75240d1000 con 0x7f752415d960 2024-04-25T14:11:23.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.969+0000 7f7521600640 1 -- 172.21.15.86:0/168663559 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v132) ==== 169+0+0 (secure 0 0 0) 0x7f751c05b6c0 con 0x7f752415d960 2024-04-25T14:11:23.966 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-04-25T14:11:23.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.973+0000 7f752aa00640 1 -- 172.21.15.86:0/168663559 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f750406f520 msgr2=0x7f7504071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:23.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.973+0000 7f752aa00640 1 --2- 172.21.15.86:0/168663559 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f750406f520 0x7f7504071a30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7518009aa0 tx=0x7f75180039a0 comp rx=0 tx=0).stop 2024-04-25T14:11:23.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.973+0000 7f752aa00640 1 -- 172.21.15.86:0/168663559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f752415d960 msgr2=0x7f75240b80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:23.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.973+0000 7f752aa00640 1 --2- 172.21.15.86:0/168663559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f752415d960 0x7f75240b80d0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f751c00c410 tx=0x7f751c00c810 comp rx=0 tx=0).stop 2024-04-25T14:11:23.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.973+0000 7f752aa00640 1 -- 172.21.15.86:0/168663559 shutdown_connections 2024-04-25T14:11:23.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.973+0000 7f752aa00640 1 --2- 172.21.15.86:0/168663559 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f750406f520 0x7f7504071a30 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:23.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.973+0000 7f752aa00640 1 --2- 172.21.15.86:0/168663559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f752415d960 0x7f75240b80d0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:23.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.973+0000 7f752aa00640 1 --2- 172.21.15.86:0/168663559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75240cdc20 0x7f7524167d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:23.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.973+0000 7f752aa00640 1 --2- 172.21.15.86:0/168663559 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75240721f0 0x7f752414b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:23.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.973+0000 7f752aa00640 1 -- 172.21.15.86:0/168663559 >> 172.21.15.86:0/168663559 conn(0x7f75240d3330 msgr2=0x7f75240d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:23.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.973+0000 7f752aa00640 1 -- 172.21.15.86:0/168663559 shutdown_connections 2024-04-25T14:11:23.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:23.973+0000 7f752aa00640 1 -- 172.21.15.86:0/168663559 wait complete. 2024-04-25T14:11:23.993 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T14:11:24.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.145+0000 7fe9a7400640 1 Processor -- start 2024-04-25T14:11:24.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.145+0000 7fe9a7400640 1 -- start start 2024-04-25T14:11:24.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.145+0000 7fe9a7400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe9a015dc90 0x7fe9a0161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:24.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.145+0000 7fe9a7400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9a00cd8d0 con 0x7fe9a00d2c80 2024-04-25T14:11:24.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.145+0000 7fe9a7400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9a00ce970 con 0x7fe9a0071530 2024-04-25T14:11:24.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.145+0000 7fe9a7400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe9a00ce720 con 0x7fe9a015dc90 2024-04-25T14:11:24.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.145+0000 7fe9a4a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe9a00d2c80 0x7fe9a0070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39112/0 (socket says 172.21.15.86:39112) 2024-04-25T14:11:24.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.145+0000 7fe9a4a00640 1 -- 172.21.15.86:0/3692484947 learned_addr learned my addr 172.21.15.86:0/3692484947 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.149+0000 7fe9a5400640 1 --2- 172.21.15.86:0/3692484947 >> v2:172.21.15.155:3300/0 conn(0x7fe9a015dc90 0x7fe9a0161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.149+0000 7fe99f400640 1 -- 172.21.15.86:0/3692484947 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1882430791 0 0) 0x7fe9a00cd8d0 con 0x7fe9a00d2c80 2024-04-25T14:11:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.149+0000 7fe99f400640 1 -- 172.21.15.86:0/3692484947 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe984003530 con 0x7fe9a00d2c80 2024-04-25T14:11:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.149+0000 7fe99f400640 1 -- 172.21.15.86:0/3692484947 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 215224399 0 0) 0x7fe984003530 con 0x7fe9a00d2c80 2024-04-25T14:11:24.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.149+0000 7fe99f400640 1 -- 172.21.15.86:0/3692484947 >> v2:172.21.15.155:3300/0 conn(0x7fe9a015dc90 msgr2=0x7fe9a0161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:24.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.149+0000 7fe99f400640 1 --2- 172.21.15.86:0/3692484947 >> v2:172.21.15.155:3300/0 conn(0x7fe9a015dc90 0x7fe9a0161df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:24.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.149+0000 7fe99f400640 1 -- 172.21.15.86:0/3692484947 >> v1:172.21.15.155:6789/0 conn(0x7fe9a0071530 legacy=0x7fe9a01616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:24.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.149+0000 7fe99f400640 1 -- 172.21.15.86:0/3692484947 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a00d1be0 con 0x7fe9a00d2c80 2024-04-25T14:11:24.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.149+0000 7fe99f400640 1 -- 172.21.15.86:0/3692484947 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fe990001680 con 0x7fe9a00d2c80 2024-04-25T14:11:24.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.149+0000 7fe99f400640 1 -- 172.21.15.86:0/3692484947 >> v1:172.21.15.86:6789/0 conn(0x7fe9a00d2c80 legacy=0x7fe9a0070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:24.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.149+0000 7fe99f400640 1 --2- 172.21.15.86:0/3692484947 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe984003a30 0x7fe984003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:24.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.149+0000 7fe99f400640 1 --2- 172.21.15.86:0/3692484947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe984004810 0x7fe984004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:24.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.149+0000 7fe99f400640 1 --2- 172.21.15.86:0/3692484947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe984005230 0x7fe98400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:24.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.149+0000 7fe99f400640 1 -- 172.21.15.86:0/3692484947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a00ce720 con 0x7fe984004810 2024-04-25T14:11:24.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.149+0000 7fe99f400640 1 -- 172.21.15.86:0/3692484947 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe98400c1b0 con 0x7fe984005230 2024-04-25T14:11:24.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.149+0000 7fe99f400640 1 -- 172.21.15.86:0/3692484947 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe98400c3b0 con 0x7fe984003a30 2024-04-25T14:11:24.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.149+0000 7fe99fe00640 1 --2- 172.21.15.86:0/3692484947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe984004810 0x7fe984004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:24.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.149+0000 7fe9a4a00640 1 --2- 172.21.15.86:0/3692484947 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe984003a30 0x7fe984003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:24.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.149+0000 7fe9a5400640 1 --2- 172.21.15.86:0/3692484947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe984005230 0x7fe98400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:24.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.149+0000 7fe99fe00640 1 -- 172.21.15.86:0/3692484947 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe984003a30 msgr2=0x7fe984003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:24.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.149+0000 7fe99fe00640 1 --2- 172.21.15.86:0/3692484947 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe984003a30 0x7fe984003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:24.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.149+0000 7fe99fe00640 1 -- 172.21.15.86:0/3692484947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe984005230 msgr2=0x7fe98400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:24.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.149+0000 7fe99fe00640 1 --2- 172.21.15.86:0/3692484947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe984005230 0x7fe98400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:24.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.149+0000 7fe9a5400640 1 -- 172.21.15.86:0/3692484947 reap_dead start 2024-04-25T14:11:24.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.149+0000 7fe99fe00640 1 -- 172.21.15.86:0/3692484947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe984003530 con 0x7fe984004810 2024-04-25T14:11:24.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.153+0000 7fe99fe00640 1 --2- 172.21.15.86:0/3692484947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe984004810 0x7fe984004c20 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0017b0 tx=0x7fe98c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b8ba0324980405 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:24.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.153+0000 7fe99f400640 1 -- 172.21.15.86:0/3692484947 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c0013d0 con 0x7fe984004810 2024-04-25T14:11:24.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.153+0000 7fe99f400640 1 -- 172.21.15.86:0/3692484947 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe98c0052d0 con 0x7fe984004810 2024-04-25T14:11:24.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.153+0000 7fe99f400640 1 -- 172.21.15.86:0/3692484947 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c003650 con 0x7fe984004810 2024-04-25T14:11:24.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.153+0000 7fe9a7400640 1 -- 172.21.15.86:0/3692484947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe984004810 msgr2=0x7fe984004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:24.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.153+0000 7fe9a7400640 1 --2- 172.21.15.86:0/3692484947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe984004810 0x7fe984004c20 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0017b0 tx=0x7fe98c01c3e0 comp rx=0 tx=0).stop 2024-04-25T14:11:24.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.153+0000 7fe9a7400640 1 -- 172.21.15.86:0/3692484947 shutdown_connections 2024-04-25T14:11:24.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.153+0000 7fe9a7400640 1 --2- 172.21.15.86:0/3692484947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe984004810 0x7fe984004c20 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:24.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.153+0000 7fe9a7400640 1 -- 172.21.15.86:0/3692484947 >> 172.21.15.86:0/3692484947 conn(0x7fe9a00cffc0 msgr2=0x7fe9a00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:24.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.153+0000 7fe9a7400640 1 -- 172.21.15.86:0/3692484947 shutdown_connections 2024-04-25T14:11:24.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.153+0000 7fe9a7400640 1 -- 172.21.15.86:0/3692484947 wait complete. 2024-04-25T14:11:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.153+0000 7fe9a7400640 1 Processor -- start 2024-04-25T14:11:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.153+0000 7fe9a7400640 1 -- start start 2024-04-25T14:11:24.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.157+0000 7fe9a7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe984004810 0x7fe9a00b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:24.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.157+0000 7fe9a7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9a014e980 0x7fe9a00b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.157+0000 7fe9a7400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9a014ed90 0x7fe9a00b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.157+0000 7fe9a7400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a00d2480 con 0x7fe984004810 2024-04-25T14:11:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.157+0000 7fe9a7400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a00d21a0 con 0x7fe9a014e980 2024-04-25T14:11:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.157+0000 7fe9a7400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe9a00d29a0 con 0x7fe9a014ed90 2024-04-25T14:11:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.157+0000 7fe9a4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe984004810 0x7fe9a00b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.157+0000 7fe9a4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe984004810 0x7fe9a00b1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49694/0 (socket says 172.21.15.86:49694) 2024-04-25T14:11:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.157+0000 7fe9a4a00640 1 -- 172.21.15.86:0/3317031716 learned_addr learned my addr 172.21.15.86:0/3317031716 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.157+0000 7fe99fe00640 1 --2- 172.21.15.86:0/3317031716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9a014e980 0x7fe9a00b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.157+0000 7fe9a4a00640 1 -- 172.21.15.86:0/3317031716 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9a014ed90 msgr2=0x7fe9a00b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.157+0000 7fe9a4a00640 1 --2- 172.21.15.86:0/3317031716 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9a014ed90 0x7fe9a00b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.157+0000 7fe9a4a00640 1 -- 172.21.15.86:0/3317031716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9a014e980 msgr2=0x7fe9a00b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.157+0000 7fe9a4a00640 1 --2- 172.21.15.86:0/3317031716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9a014e980 0x7fe9a00b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.157+0000 7fe9a4a00640 1 -- 172.21.15.86:0/3317031716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a006f1d0 con 0x7fe984004810 2024-04-25T14:11:24.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.157+0000 7fe99fe00640 1 --2- 172.21.15.86:0/3317031716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9a014e980 0x7fe9a00b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:24.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.157+0000 7fe9a4a00640 1 --2- 172.21.15.86:0/3317031716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe984004810 0x7fe9a00b1c00 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fe990007290 tx=0x7fe990007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:24.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.157+0000 7fe99cc00640 1 -- 172.21.15.86:0/3317031716 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99000f730 con 0x7fe984004810 2024-04-25T14:11:24.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.157+0000 7fe99cc00640 1 -- 172.21.15.86:0/3317031716 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe990018070 con 0x7fe984004810 2024-04-25T14:11:24.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.157+0000 7fe9a7400640 1 -- 172.21.15.86:0/3317031716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a006e650 con 0x7fe984004810 2024-04-25T14:11:24.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.157+0000 7fe99cc00640 1 -- 172.21.15.86:0/3317031716 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99000fb30 con 0x7fe984004810 2024-04-25T14:11:24.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.157+0000 7fe9a7400640 1 -- 172.21.15.86:0/3317031716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a006ec10 con 0x7fe984004810 2024-04-25T14:11:24.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.157+0000 7fe9a7400640 1 -- 172.21.15.86:0/3317031716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9a00720e0 con 0x7fe984004810 2024-04-25T14:11:24.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.161+0000 7fe99cc00640 1 -- 172.21.15.86:0/3317031716 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7fe99000fcd0 con 0x7fe984004810 2024-04-25T14:11:24.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.161+0000 7fe99cc00640 1 --2- 172.21.15.86:0/3317031716 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fe98006f520 0x7fe980071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:24.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.161+0000 7fe99cc00640 1 -- 172.21.15.86:0/3317031716 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 6066+0+0 (secure 0 0 0) 0x7fe99008f5f0 con 0x7fe984004810 2024-04-25T14:11:24.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.161+0000 7fe99fe00640 1 --2- 172.21.15.86:0/3317031716 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fe98006f520 0x7fe980071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:24.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.161+0000 7fe99fe00640 1 --2- 172.21.15.86:0/3317031716 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fe98006f520 0x7fe980071a30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0ac000 tx=0x7fe98c0011f0 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.165+0000 7fe99cc00640 1 -- 172.21.15.86:0/3317031716 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe99005c250 con 0x7fe984004810 2024-04-25T14:11:24.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.321+0000 7fe9a7400640 1 -- 172.21.15.86:0/3317031716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fe9a0076bf0 con 0x7fe984004810 2024-04-25T14:11:24.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.325+0000 7fe99cc00640 1 -- 172.21.15.86:0/3317031716 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7fe99005bb50 con 0x7fe984004810 2024-04-25T14:11:24.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.329+0000 7fe9a7400640 1 -- 172.21.15.86:0/3317031716 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fe98006f520 msgr2=0x7fe980071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:24.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.329+0000 7fe9a7400640 1 --2- 172.21.15.86:0/3317031716 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fe98006f520 0x7fe980071a30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0ac000 tx=0x7fe98c0011f0 comp rx=0 tx=0).stop 2024-04-25T14:11:24.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.329+0000 7fe9a7400640 1 -- 172.21.15.86:0/3317031716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe984004810 msgr2=0x7fe9a00b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:24.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.329+0000 7fe9a7400640 1 --2- 172.21.15.86:0/3317031716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe984004810 0x7fe9a00b1c00 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fe990007290 tx=0x7fe990007750 comp rx=0 tx=0).stop 2024-04-25T14:11:24.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.329+0000 7fe9a7400640 1 -- 172.21.15.86:0/3317031716 shutdown_connections 2024-04-25T14:11:24.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.329+0000 7fe9a7400640 1 --2- 172.21.15.86:0/3317031716 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fe98006f520 0x7fe980071a30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:24.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.329+0000 7fe9a7400640 1 --2- 172.21.15.86:0/3317031716 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9a014ed90 0x7fe9a00b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:24.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.329+0000 7fe9a7400640 1 --2- 172.21.15.86:0/3317031716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9a014e980 0x7fe9a00b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:24.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.329+0000 7fe9a7400640 1 --2- 172.21.15.86:0/3317031716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe984004810 0x7fe9a00b1c00 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:24.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.329+0000 7fe9a7400640 1 -- 172.21.15.86:0/3317031716 >> 172.21.15.86:0/3317031716 conn(0x7fe9a00cffc0 msgr2=0x7fe9a00cbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:24.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.329+0000 7fe9a7400640 1 -- 172.21.15.86:0/3317031716 shutdown_connections 2024-04-25T14:11:24.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.329+0000 7fe9a7400640 1 -- 172.21.15.86:0/3317031716 wait complete. 2024-04-25T14:11:24.345 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_metrics_from_rank' 2024-04-25T14:11:24.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.493+0000 7fbc60000640 1 Processor -- start 2024-04-25T14:11:24.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.493+0000 7fbc60000640 1 -- start start 2024-04-25T14:11:24.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc60000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbc58070330 0x7fbc580d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:24.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc60000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc580d2030 con 0x7fbc580d1c20 2024-04-25T14:11:24.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc60000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc5815e5c0 con 0x7fbc580ce790 2024-04-25T14:11:24.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc60000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fbc5815e8a0 con 0x7fbc58070330 2024-04-25T14:11:24.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc5cc00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fbc580d1c20 0x7fbc5806ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39126/0 (socket says 172.21.15.86:39126) 2024-04-25T14:11:24.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc5cc00640 1 -- 172.21.15.86:0/1180991564 learned_addr learned my addr 172.21.15.86:0/1180991564 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:24.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc5d600640 1 --2- 172.21.15.86:0/1180991564 >> v2:172.21.15.155:3300/0 conn(0x7fbc58070330 0x7fbc580d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:24.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc57e00640 1 -- 172.21.15.86:0/1180991564 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3493664971 0 0) 0x7fbc44001120 con 0x7fbc580d1c20 2024-04-25T14:11:24.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc57e00640 1 -- 172.21.15.86:0/1180991564 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc40003530 con 0x7fbc580d1c20 2024-04-25T14:11:24.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc57e00640 1 -- 172.21.15.86:0/1180991564 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3283778321 0 0) 0x7fbc40003530 con 0x7fbc580d1c20 2024-04-25T14:11:24.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc57e00640 1 -- 172.21.15.86:0/1180991564 >> v2:172.21.15.155:3300/0 conn(0x7fbc58070330 msgr2=0x7fbc580d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:24.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc57e00640 1 --2- 172.21.15.86:0/1180991564 >> v2:172.21.15.155:3300/0 conn(0x7fbc58070330 0x7fbc580d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:24.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc57e00640 1 -- 172.21.15.86:0/1180991564 >> v1:172.21.15.155:6789/0 conn(0x7fbc580ce790 legacy=0x7fbc5806f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:24.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc57e00640 1 -- 172.21.15.86:0/1180991564 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5815eb80 con 0x7fbc580d1c20 2024-04-25T14:11:24.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc57e00640 1 -- 172.21.15.86:0/1180991564 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fbc44001850 con 0x7fbc580d1c20 2024-04-25T14:11:24.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc57e00640 1 -- 172.21.15.86:0/1180991564 >> v1:172.21.15.86:6789/0 conn(0x7fbc580d1c20 legacy=0x7fbc5806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:24.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc57e00640 1 --2- 172.21.15.86:0/1180991564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc40003a30 0x7fbc40003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:24.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc57e00640 1 --2- 172.21.15.86:0/1180991564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc40004810 0x7fbc40004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:24.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc57e00640 1 --2- 172.21.15.86:0/1180991564 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc40005360 0x7fbc4000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:24.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc57e00640 1 -- 172.21.15.86:0/1180991564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5815e8a0 con 0x7fbc40003a30 2024-04-25T14:11:24.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc57e00640 1 -- 172.21.15.86:0/1180991564 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4000c260 con 0x7fbc40004810 2024-04-25T14:11:24.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc57e00640 1 -- 172.21.15.86:0/1180991564 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbc4000c460 con 0x7fbc40005360 2024-04-25T14:11:24.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc5d600640 1 --2- 172.21.15.86:0/1180991564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc40003a30 0x7fbc40003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:24.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc5cc00640 1 --2- 172.21.15.86:0/1180991564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc40004810 0x7fbc40004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:24.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc5e000640 1 --2- 172.21.15.86:0/1180991564 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc40005360 0x7fbc4000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:24.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc5d600640 1 -- 172.21.15.86:0/1180991564 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc40005360 msgr2=0x7fbc4000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:24.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc5d600640 1 --2- 172.21.15.86:0/1180991564 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc40005360 0x7fbc4000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:24.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc5d600640 1 -- 172.21.15.86:0/1180991564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc40004810 msgr2=0x7fbc40004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:24.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc5d600640 1 --2- 172.21.15.86:0/1180991564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc40004810 0x7fbc40004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:24.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc5d600640 1 -- 172.21.15.86:0/1180991564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc40003530 con 0x7fbc40003a30 2024-04-25T14:11:24.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.497+0000 7fbc5e000640 1 -- 172.21.15.86:0/1180991564 reap_dead start 2024-04-25T14:11:24.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.501+0000 7fbc5d600640 1 --2- 172.21.15.86:0/1180991564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc40003a30 0x7fbc40003ea0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009ab0 tx=0x7fbc4c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4527551921f525 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:24.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.501+0000 7fbc57e00640 1 -- 172.21.15.86:0/1180991564 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c0096b0 con 0x7fbc40003a30 2024-04-25T14:11:24.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.501+0000 7fbc57e00640 1 -- 172.21.15.86:0/1180991564 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc4c0a6430 con 0x7fbc40003a30 2024-04-25T14:11:24.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.501+0000 7fbc57e00640 1 -- 172.21.15.86:0/1180991564 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c0a6820 con 0x7fbc40003a30 2024-04-25T14:11:24.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.501+0000 7fbc60000640 1 -- 172.21.15.86:0/1180991564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc40003a30 msgr2=0x7fbc40003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:24.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.501+0000 7fbc60000640 1 --2- 172.21.15.86:0/1180991564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc40003a30 0x7fbc40003ea0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009ab0 tx=0x7fbc4c01ca50 comp rx=0 tx=0).stop 2024-04-25T14:11:24.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.501+0000 7fbc60000640 1 -- 172.21.15.86:0/1180991564 shutdown_connections 2024-04-25T14:11:24.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.501+0000 7fbc60000640 1 --2- 172.21.15.86:0/1180991564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc40003a30 0x7fbc40003ea0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:24.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.501+0000 7fbc60000640 1 -- 172.21.15.86:0/1180991564 >> 172.21.15.86:0/1180991564 conn(0x7fbc580b4650 msgr2=0x7fbc580b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:24.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.501+0000 7fbc60000640 1 -- 172.21.15.86:0/1180991564 shutdown_connections 2024-04-25T14:11:24.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.501+0000 7fbc60000640 1 -- 172.21.15.86:0/1180991564 wait complete. 2024-04-25T14:11:24.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.505+0000 7fbc60000640 1 Processor -- start 2024-04-25T14:11:24.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.505+0000 7fbc60000640 1 -- start start 2024-04-25T14:11:24.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.505+0000 7fbc60000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc40003a30 0x7fbc581fd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:24.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.505+0000 7fbc60000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc580ca680 0x7fbc5809a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:24.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.505+0000 7fbc60000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc580cf3c0 0x7fbc5815ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:24.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.505+0000 7fbc60000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc580708c0 con 0x7fbc580cf3c0 2024-04-25T14:11:24.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.505+0000 7fbc60000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbc58070740 con 0x7fbc40003a30 2024-04-25T14:11:24.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.505+0000 7fbc60000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbc580cdf90 con 0x7fbc580ca680 2024-04-25T14:11:24.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.505+0000 7fbc5e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc580cf3c0 0x7fbc5815ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:24.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.505+0000 7fbc5e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc580cf3c0 0x7fbc5815ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49710/0 (socket says 172.21.15.86:49710) 2024-04-25T14:11:24.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.505+0000 7fbc5e000640 1 -- 172.21.15.86:0/1390198665 learned_addr learned my addr 172.21.15.86:0/1390198665 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:24.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.505+0000 7fbc5d600640 1 --2- 172.21.15.86:0/1390198665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc40003a30 0x7fbc581fd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:24.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.505+0000 7fbc5cc00640 1 --2- 172.21.15.86:0/1390198665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc580ca680 0x7fbc5809a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:24.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.509+0000 7fbc5e000640 1 -- 172.21.15.86:0/1390198665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc580ca680 msgr2=0x7fbc5809a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:24.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.509+0000 7fbc5e000640 1 --2- 172.21.15.86:0/1390198665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc580ca680 0x7fbc5809a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:24.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.509+0000 7fbc5e000640 1 -- 172.21.15.86:0/1390198665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc40003a30 msgr2=0x7fbc581fd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:24.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.509+0000 7fbc5e000640 1 --2- 172.21.15.86:0/1390198665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc40003a30 0x7fbc581fd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:24.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.509+0000 7fbc5e000640 1 -- 172.21.15.86:0/1390198665 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc580c95e0 con 0x7fbc580cf3c0 2024-04-25T14:11:24.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.509+0000 7fbc5e000640 1 --2- 172.21.15.86:0/1390198665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc580cf3c0 0x7fbc5815ff00 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fbc5815bd10 tx=0x7fbc5000ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:24.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.509+0000 7fbc55600640 1 -- 172.21.15.86:0/1390198665 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc50003b90 con 0x7fbc580cf3c0 2024-04-25T14:11:24.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.509+0000 7fbc55600640 1 -- 172.21.15.86:0/1390198665 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc50018070 con 0x7fbc580cf3c0 2024-04-25T14:11:24.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.509+0000 7fbc60000640 1 -- 172.21.15.86:0/1390198665 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc580c98c0 con 0x7fbc580cf3c0 2024-04-25T14:11:24.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.509+0000 7fbc55600640 1 -- 172.21.15.86:0/1390198665 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc500050d0 con 0x7fbc580cf3c0 2024-04-25T14:11:24.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.509+0000 7fbc60000640 1 -- 172.21.15.86:0/1390198665 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc580c9e80 con 0x7fbc580cf3c0 2024-04-25T14:11:24.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.509+0000 7fbc60000640 1 -- 172.21.15.86:0/1390198665 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc58069d00 con 0x7fbc580cf3c0 2024-04-25T14:11:24.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.509+0000 7fbc55600640 1 -- 172.21.15.86:0/1390198665 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7fbc50005270 con 0x7fbc580cf3c0 2024-04-25T14:11:24.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.513+0000 7fbc55600640 1 --2- 172.21.15.86:0/1390198665 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fbc3806f550 0x7fbc38071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:24.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.513+0000 7fbc55600640 1 -- 172.21.15.86:0/1390198665 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 6066+0+0 (secure 0 0 0) 0x7fbc5008f0d0 con 0x7fbc580cf3c0 2024-04-25T14:11:24.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.513+0000 7fbc5d600640 1 --2- 172.21.15.86:0/1390198665 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fbc3806f550 0x7fbc38071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:24.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.513+0000 7fbc5d600640 1 --2- 172.21.15.86:0/1390198665 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fbc3806f550 0x7fbc38071a60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0ad000 tx=0x7fbc4c0046d0 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:24.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.517+0000 7fbc55600640 1 -- 172.21.15.86:0/1390198665 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbc5005bd40 con 0x7fbc580cf3c0 2024-04-25T14:11:24.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.733+0000 7fbc60000640 1 -- 172.21.15.86:0/1390198665 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_metrics_from_rank"]} v 0) -- 0x7fbc58076bf0 con 0x7fbc580cf3c0 2024-04-25T14:11:24.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.969+0000 7fbc55600640 1 -- 172.21.15.86:0/1390198665 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_metrics_from_rank"]}]=0 v430) ==== 146+0+0 (secure 0 0 0) 0x7fbc5005b640 con 0x7fbc580cf3c0 2024-04-25T14:11:24.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.973+0000 7fbc60000640 1 -- 172.21.15.86:0/1390198665 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fbc3806f550 msgr2=0x7fbc38071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:24.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.973+0000 7fbc60000640 1 --2- 172.21.15.86:0/1390198665 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fbc3806f550 0x7fbc38071a60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0ad000 tx=0x7fbc4c0046d0 comp rx=0 tx=0).stop 2024-04-25T14:11:24.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.973+0000 7fbc60000640 1 -- 172.21.15.86:0/1390198665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc580cf3c0 msgr2=0x7fbc5815ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:24.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.973+0000 7fbc60000640 1 --2- 172.21.15.86:0/1390198665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc580cf3c0 0x7fbc5815ff00 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fbc5815bd10 tx=0x7fbc5000ead0 comp rx=0 tx=0).stop 2024-04-25T14:11:24.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.973+0000 7fbc60000640 1 -- 172.21.15.86:0/1390198665 shutdown_connections 2024-04-25T14:11:24.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.973+0000 7fbc60000640 1 --2- 172.21.15.86:0/1390198665 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fbc3806f550 0x7fbc38071a60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:24.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.973+0000 7fbc60000640 1 --2- 172.21.15.86:0/1390198665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc580cf3c0 0x7fbc5815ff00 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:24.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.973+0000 7fbc60000640 1 --2- 172.21.15.86:0/1390198665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc580ca680 0x7fbc5809a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:24.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.973+0000 7fbc60000640 1 --2- 172.21.15.86:0/1390198665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc40003a30 0x7fbc581fd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:24.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.973+0000 7fbc60000640 1 -- 172.21.15.86:0/1390198665 >> 172.21.15.86:0/1390198665 conn(0x7fbc580b4650 msgr2=0x7fbc5815b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:24.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.973+0000 7fbc60000640 1 -- 172.21.15.86:0/1390198665 shutdown_connections 2024-04-25T14:11:24.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:24.973+0000 7fbc60000640 1 -- 172.21.15.86:0/1390198665 wait complete. 2024-04-25T14:11:24.990 INFO:tasks.cephfs_test_runner:test_metrics_from_rank (tasks.cephfs.test_mds_metrics.TestMDSMetrics) ... ok 2024-04-25T14:11:24.991 INFO:tasks.cephfs_test_runner:Starting test: test_metrics_mds_grow (tasks.cephfs.test_mds_metrics.TestMDSMetrics) 2024-04-25T14:11:24.991 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:11:25.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.141+0000 7fc005400640 1 Processor -- start 2024-04-25T14:11:25.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.141+0000 7fc005400640 1 -- start start 2024-04-25T14:11:25.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.141+0000 7fc005400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc000166930 0x7fc000168d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:25.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.141+0000 7fc005400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc00006e7c0 con 0x7fc00006e9b0 2024-04-25T14:11:25.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.145+0000 7fc005400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc00015e500 con 0x7fc0000d2360 2024-04-25T14:11:25.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.145+0000 7fc005400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc00006c710 con 0x7fc000166930 2024-04-25T14:11:25.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.145+0000 7fbffe800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc00006e9b0 0x7fc00015ddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39128/0 (socket says 172.21.15.86:39128) 2024-04-25T14:11:25.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.145+0000 7fbffe800640 1 -- 172.21.15.86:0/886913323 learned_addr learned my addr 172.21.15.86:0/886913323 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:25.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.145+0000 7fbfff200640 1 --2- 172.21.15.86:0/886913323 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc000166930 0x7fc000168d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:25.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.145+0000 7fbffd400640 1 -- 172.21.15.86:0/886913323 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3728560729 0 0) 0x7fc00006e7c0 con 0x7fc00006e9b0 2024-04-25T14:11:25.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.145+0000 7fbffd400640 1 -- 172.21.15.86:0/886913323 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbfe4003530 con 0x7fc00006e9b0 2024-04-25T14:11:25.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.145+0000 7fbffd400640 1 -- 172.21.15.86:0/886913323 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3214192384 0 0) 0x7fc00015e500 con 0x7fc0000d2360 2024-04-25T14:11:25.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.145+0000 7fbffd400640 1 -- 172.21.15.86:0/886913323 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc00006e7c0 con 0x7fc0000d2360 2024-04-25T14:11:25.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.145+0000 7fbfff200640 1 -- 172.21.15.86:0/886913323 >> v1:172.21.15.155:6789/0 conn(0x7fc0000d2360 legacy=0x7fc0000d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:25.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.145+0000 7fbfff200640 1 -- 172.21.15.86:0/886913323 >> v1:172.21.15.86:6789/0 conn(0x7fc00006e9b0 legacy=0x7fc00015ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:25.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.145+0000 7fbfff200640 1 -- 172.21.15.86:0/886913323 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00006c9f0 con 0x7fc000166930 2024-04-25T14:11:25.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.145+0000 7fbfff200640 1 --2- 172.21.15.86:0/886913323 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc000166930 0x7fc000168d50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbff8009dc0 tx=0x7fbff801c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76dd6aa22993bca5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:25.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.149+0000 7fbffd400640 1 -- 172.21.15.86:0/886913323 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff80b2070 con 0x7fc000166930 2024-04-25T14:11:25.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.149+0000 7fbffd400640 1 -- 172.21.15.86:0/886913323 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbff80a5a50 con 0x7fc000166930 2024-04-25T14:11:25.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.149+0000 7fbffd400640 1 -- 172.21.15.86:0/886913323 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff80a5e20 con 0x7fc000166930 2024-04-25T14:11:25.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.149+0000 7fc005400640 1 -- 172.21.15.86:0/886913323 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc000166930 msgr2=0x7fc000168d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:25.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.149+0000 7fc005400640 1 --2- 172.21.15.86:0/886913323 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc000166930 0x7fc000168d50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbff8009dc0 tx=0x7fbff801c990 comp rx=0 tx=0).stop 2024-04-25T14:11:25.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.149+0000 7fc005400640 1 -- 172.21.15.86:0/886913323 shutdown_connections 2024-04-25T14:11:25.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.149+0000 7fc005400640 1 --2- 172.21.15.86:0/886913323 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc000166930 0x7fc000168d50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.149+0000 7fc005400640 1 -- 172.21.15.86:0/886913323 >> 172.21.15.86:0/886913323 conn(0x7fc00009ba30 msgr2=0x7fc0000b7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:25.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.149+0000 7fc005400640 1 -- 172.21.15.86:0/886913323 shutdown_connections 2024-04-25T14:11:25.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.149+0000 7fc005400640 1 -- 172.21.15.86:0/886913323 wait complete. 2024-04-25T14:11:25.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.149+0000 7fc005400640 1 Processor -- start 2024-04-25T14:11:25.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.149+0000 7fc005400640 1 -- start start 2024-04-25T14:11:25.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.153+0000 7fc005400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00006e9b0 0x7fc0001f8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:25.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.153+0000 7fc005400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0000d2360 0x7fc000142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:25.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.153+0000 7fc005400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc000166930 0x7fc0001433f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:25.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.153+0000 7fc005400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc00006f110 con 0x7fc000166930 2024-04-25T14:11:25.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.153+0000 7fc005400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc00006ef90 con 0x7fc00006e9b0 2024-04-25T14:11:25.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.153+0000 7fc005400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc00016bab0 con 0x7fc0000d2360 2024-04-25T14:11:25.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.153+0000 7fbfff200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc000166930 0x7fc0001433f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:25.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.153+0000 7fbfff200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc000166930 0x7fc0001433f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49724/0 (socket says 172.21.15.86:49724) 2024-04-25T14:11:25.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.153+0000 7fbfff200640 1 -- 172.21.15.86:0/1826225990 learned_addr learned my addr 172.21.15.86:0/1826225990 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:25.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.153+0000 7fbffe800640 1 --2- 172.21.15.86:0/1826225990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00006e9b0 0x7fc0001f8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:25.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.153+0000 7fbffde00640 1 --2- 172.21.15.86:0/1826225990 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0000d2360 0x7fc000142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:25.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.153+0000 7fbfff200640 1 -- 172.21.15.86:0/1826225990 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0000d2360 msgr2=0x7fc000142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:25.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.153+0000 7fbfff200640 1 --2- 172.21.15.86:0/1826225990 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0000d2360 0x7fc000142eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.153+0000 7fbfff200640 1 -- 172.21.15.86:0/1826225990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00006e9b0 msgr2=0x7fc0001f8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:25.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.153+0000 7fbfff200640 1 --2- 172.21.15.86:0/1826225990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00006e9b0 0x7fc0001f8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.153+0000 7fbfff200640 1 -- 172.21.15.86:0/1826225990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00014a340 con 0x7fc000166930 2024-04-25T14:11:25.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.153+0000 7fbffe800640 1 --2- 172.21.15.86:0/1826225990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00006e9b0 0x7fc0001f8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:25.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.153+0000 7fbfff200640 1 --2- 172.21.15.86:0/1826225990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc000166930 0x7fc0001433f0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fbff801ce70 tx=0x7fbff8019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:25.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.153+0000 7fbff6a00640 1 -- 172.21.15.86:0/1826225990 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff80b2040 con 0x7fc000166930 2024-04-25T14:11:25.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.153+0000 7fbff6a00640 1 -- 172.21.15.86:0/1826225990 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbff80bf070 con 0x7fc000166930 2024-04-25T14:11:25.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.153+0000 7fc005400640 1 -- 172.21.15.86:0/1826225990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc00015a7b0 con 0x7fc000166930 2024-04-25T14:11:25.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.153+0000 7fbff6a00640 1 -- 172.21.15.86:0/1826225990 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff8009450 con 0x7fc000166930 2024-04-25T14:11:25.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.153+0000 7fc005400640 1 -- 172.21.15.86:0/1826225990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0001441c0 con 0x7fc000166930 2024-04-25T14:11:25.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.153+0000 7fc005400640 1 -- 172.21.15.86:0/1826225990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc00015a520 con 0x7fc000166930 2024-04-25T14:11:25.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.157+0000 7fbff6a00640 1 -- 172.21.15.86:0/1826225990 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7fbff80adcc0 con 0x7fc000166930 2024-04-25T14:11:25.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.157+0000 7fbff6a00640 1 --2- 172.21.15.86:0/1826225990 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fbfe006f520 0x7fbfe0071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:25.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.157+0000 7fbff6a00640 1 -- 172.21.15.86:0/1826225990 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 6066+0+0 (secure 0 0 0) 0x7fbff812a960 con 0x7fc000166930 2024-04-25T14:11:25.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.157+0000 7fbffe800640 1 --2- 172.21.15.86:0/1826225990 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fbfe006f520 0x7fbfe0071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:25.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.157+0000 7fbffe800640 1 --2- 172.21.15.86:0/1826225990 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fbfe006f520 0x7fbfe0071a30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbff0000f70 tx=0x7fbff0009040 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:25.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.161+0000 7fbff6a00640 1 -- 172.21.15.86:0/1826225990 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc00014a340 con 0x7fc000166930 2024-04-25T14:11:25.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.317+0000 7fc005400640 1 -- 172.21.15.86:0/1826225990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0000a05a0 con 0x7fc000166930 2024-04-25T14:11:25.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.321+0000 7fbff6a00640 1 -- 172.21.15.86:0/1826225990 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) ==== 74+0+14365 (secure 0 0 0) 0x7fbff8016580 con 0x7fc000166930 2024-04-25T14:11:25.317 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:25.317 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":132,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:11:23.969536+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.155:6835/544985680":"2024-04-26T14:11:20.957729+0000","172.21.15.86:6833/2221276863":"2024-04-26T14:11:16.711047+0000","172.21.15.86:6832/2221276863":"2024-04-26T14:11:16.711047+0000","172.21.15.86:0/3342732660":"2024-04-26T14:11:16.711047+0000","172.21.15.155:6833/544985680":"2024-04-26T14:11:20.957729+0000","172.21.15.86:0/1871957865":"2024-04-26T14:11:16.711047+0000","172.21.15.86:0/2615931231":"2024-04-26T14:11:16.711047+0000","172.21.15.86:0/1695622468":"2024-04-26T14:11:16.711047+0000","172.21.15.86:0/3537150562":"2024-04-26T14:11:16.711047+0000","172.21.15.86:0/506566715":"2024-04-26T14:11:16.711047+0000","172.21.15.86:0/3961472441":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/287404215":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/889708173":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/3224392570":"2024-04-26T14:11:04.532513+0000","172.21.15.86:6833/3964666460":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/553620002":"2024-04-26T14:11:04.532513+0000","172.21.15.86:6832/3964666460":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/3726306847":"2024-04-26T14:11:04.532513+0000","172.21.15.86:0/2412030790":"2024-04-25T15:10:55.520329+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:11:25.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.321+0000 7fc005400640 1 -- 172.21.15.86:0/1826225990 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fbfe006f520 msgr2=0x7fbfe0071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:25.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.321+0000 7fc005400640 1 --2- 172.21.15.86:0/1826225990 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fbfe006f520 0x7fbfe0071a30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbff0000f70 tx=0x7fbff0009040 comp rx=0 tx=0).stop 2024-04-25T14:11:25.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.321+0000 7fc005400640 1 -- 172.21.15.86:0/1826225990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc000166930 msgr2=0x7fc0001433f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:25.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.321+0000 7fc005400640 1 --2- 172.21.15.86:0/1826225990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc000166930 0x7fc0001433f0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fbff801ce70 tx=0x7fbff8019730 comp rx=0 tx=0).stop 2024-04-25T14:11:25.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.321+0000 7fc005400640 1 -- 172.21.15.86:0/1826225990 shutdown_connections 2024-04-25T14:11:25.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.321+0000 7fc005400640 1 --2- 172.21.15.86:0/1826225990 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fbfe006f520 0x7fbfe0071a30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.321+0000 7fc005400640 1 --2- 172.21.15.86:0/1826225990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc000166930 0x7fc0001433f0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.321+0000 7fc005400640 1 --2- 172.21.15.86:0/1826225990 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0000d2360 0x7fc000142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.321+0000 7fc005400640 1 --2- 172.21.15.86:0/1826225990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00006e9b0 0x7fc0001f8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.321+0000 7fc005400640 1 -- 172.21.15.86:0/1826225990 >> 172.21.15.86:0/1826225990 conn(0x7fc00009ba30 msgr2=0x7fc00015b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:25.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.321+0000 7fc005400640 1 -- 172.21.15.86:0/1826225990 shutdown_connections 2024-04-25T14:11:25.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.321+0000 7fc005400640 1 -- 172.21.15.86:0/1826225990 wait complete. 2024-04-25T14:11:25.341 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:11:25.341 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:11:25.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.493+0000 7fcd57e00640 1 Processor -- start 2024-04-25T14:11:25.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.493+0000 7fcd57e00640 1 -- start start 2024-04-25T14:11:25.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.493+0000 7fcd57e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcd58157f80 0x7fcd580aea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:25.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.493+0000 7fcd57e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd5806f6d0 0x7fcd58161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:25.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.493+0000 7fcd57e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcd5806fae0 0x7fcd58161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:25.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.493+0000 7fcd57e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcd580aef70 con 0x7fcd58157f80 2024-04-25T14:11:25.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.493+0000 7fcd57e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fcd580b0710 con 0x7fcd5806fae0 2024-04-25T14:11:25.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.493+0000 7fcd57e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcd580ae1c0 con 0x7fcd5806f6d0 2024-04-25T14:11:25.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.497+0000 7fcd56a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcd58157f80 0x7fcd580aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:25.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.497+0000 7fcd56a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcd58157f80 0x7fcd580aea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49730/0 (socket says 172.21.15.86:49730) 2024-04-25T14:11:25.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.497+0000 7fcd56a00640 1 -- 172.21.15.86:0/4161683945 learned_addr learned my addr 172.21.15.86:0/4161683945 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:25.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.497+0000 7fcd56a00640 1 -- 172.21.15.86:0/4161683945 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd5806f6d0 msgr2=0x7fcd58161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:25.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.497+0000 7fcd57400640 1 --2- 172.21.15.86:0/4161683945 >> v2:172.21.15.155:3300/0 conn(0x7fcd5806fae0 0x7fcd58161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:25.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.497+0000 7fcd4e000640 1 --2- 172.21.15.86:0/4161683945 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd5806f6d0 0x7fcd58161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:25.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.497+0000 7fcd56a00640 1 --2- 172.21.15.86:0/4161683945 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd5806f6d0 0x7fcd58161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.497+0000 7fcd56a00640 1 -- 172.21.15.86:0/4161683945 >> v2:172.21.15.155:3300/0 conn(0x7fcd5806fae0 msgr2=0x7fcd58161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:25.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.497+0000 7fcd56a00640 1 --2- 172.21.15.86:0/4161683945 >> v2:172.21.15.155:3300/0 conn(0x7fcd5806fae0 0x7fcd58161950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.497+0000 7fcd56a00640 1 -- 172.21.15.86:0/4161683945 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd580ae4a0 con 0x7fcd58157f80 2024-04-25T14:11:25.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.497+0000 7fcd4e000640 1 --2- 172.21.15.86:0/4161683945 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd5806f6d0 0x7fcd58161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:25.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.497+0000 7fcd57400640 1 --2- 172.21.15.86:0/4161683945 >> v2:172.21.15.155:3300/0 conn(0x7fcd5806fae0 0x7fcd58161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:25.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.497+0000 7fcd56a00640 1 --2- 172.21.15.86:0/4161683945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd58157f80 0x7fcd580aea30 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fcd580b0920 tx=0x7fcd4801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0083db45a1ecf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:25.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.497+0000 7fcd56000640 1 -- 172.21.15.86:0/4161683945 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd480a5670 con 0x7fcd58157f80 2024-04-25T14:11:25.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.501+0000 7fcd56000640 1 -- 172.21.15.86:0/4161683945 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd480a5810 con 0x7fcd58157f80 2024-04-25T14:11:25.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.501+0000 7fcd56000640 1 -- 172.21.15.86:0/4161683945 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd480a5be0 con 0x7fcd58157f80 2024-04-25T14:11:25.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.501+0000 7fcd57e00640 1 -- 172.21.15.86:0/4161683945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd58157f80 msgr2=0x7fcd580aea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:25.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.501+0000 7fcd57e00640 1 --2- 172.21.15.86:0/4161683945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd58157f80 0x7fcd580aea30 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fcd580b0920 tx=0x7fcd4801c9f0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.501+0000 7fcd57e00640 1 -- 172.21.15.86:0/4161683945 shutdown_connections 2024-04-25T14:11:25.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.501+0000 7fcd57e00640 1 --2- 172.21.15.86:0/4161683945 >> v2:172.21.15.155:3300/0 conn(0x7fcd5806fae0 0x7fcd58161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.501+0000 7fcd57e00640 1 --2- 172.21.15.86:0/4161683945 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd5806f6d0 0x7fcd58161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.501+0000 7fcd57e00640 1 --2- 172.21.15.86:0/4161683945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd58157f80 0x7fcd580aea30 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.501+0000 7fcd57e00640 1 -- 172.21.15.86:0/4161683945 >> 172.21.15.86:0/4161683945 conn(0x7fcd580b4740 msgr2=0x7fcd580b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:25.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.501+0000 7fcd57e00640 1 -- 172.21.15.86:0/4161683945 shutdown_connections 2024-04-25T14:11:25.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.501+0000 7fcd57e00640 1 -- 172.21.15.86:0/4161683945 wait complete. 2024-04-25T14:11:25.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.501+0000 7fcd57e00640 1 Processor -- start 2024-04-25T14:11:25.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.501+0000 7fcd57e00640 1 -- start start 2024-04-25T14:11:25.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.505+0000 7fcd57e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd5806f6d0 0x7fcd5814fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:25.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.505+0000 7fcd57e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd5806fae0 0x7fcd580b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:25.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.505+0000 7fcd57e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd58157f80 0x7fcd580b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:25.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.505+0000 7fcd57e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5815de20 con 0x7fcd5806f6d0 2024-04-25T14:11:25.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.505+0000 7fcd57e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5806d090 con 0x7fcd5806fae0 2024-04-25T14:11:25.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.505+0000 7fcd57e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcd5815dfa0 con 0x7fcd58157f80 2024-04-25T14:11:25.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.505+0000 7fcd56a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd5806f6d0 0x7fcd5814fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:25.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.505+0000 7fcd56a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd5806f6d0 0x7fcd5814fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49732/0 (socket says 172.21.15.86:49732) 2024-04-25T14:11:25.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.505+0000 7fcd56a00640 1 -- 172.21.15.86:0/1769447925 learned_addr learned my addr 172.21.15.86:0/1769447925 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:25.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.505+0000 7fcd4e000640 1 --2- 172.21.15.86:0/1769447925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd5806fae0 0x7fcd580b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:25.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.505+0000 7fcd57400640 1 --2- 172.21.15.86:0/1769447925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd58157f80 0x7fcd580b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:25.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.505+0000 7fcd56a00640 1 -- 172.21.15.86:0/1769447925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd58157f80 msgr2=0x7fcd580b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:25.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.505+0000 7fcd56a00640 1 --2- 172.21.15.86:0/1769447925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd58157f80 0x7fcd580b2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.505+0000 7fcd56a00640 1 -- 172.21.15.86:0/1769447925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd5806fae0 msgr2=0x7fcd580b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:25.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.505+0000 7fcd56a00640 1 --2- 172.21.15.86:0/1769447925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd5806fae0 0x7fcd580b1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.505+0000 7fcd56a00640 1 -- 172.21.15.86:0/1769447925 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5809e8c0 con 0x7fcd5806f6d0 2024-04-25T14:11:25.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.505+0000 7fcd4e000640 1 --2- 172.21.15.86:0/1769447925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd5806fae0 0x7fcd580b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:25.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.505+0000 7fcd57400640 1 --2- 172.21.15.86:0/1769447925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd58157f80 0x7fcd580b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:25.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.505+0000 7fcd56a00640 1 --2- 172.21.15.86:0/1769447925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd5806f6d0 0x7fcd5814fa50 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fcd480af040 tx=0x7fcd48019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:25.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.505+0000 7fcd4f400640 1 -- 172.21.15.86:0/1769447925 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd48009510 con 0x7fcd5806f6d0 2024-04-25T14:11:25.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.505+0000 7fcd4f400640 1 -- 172.21.15.86:0/1769447925 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd480be070 con 0x7fcd5806f6d0 2024-04-25T14:11:25.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.505+0000 7fcd4f400640 1 -- 172.21.15.86:0/1769447925 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd480b8b00 con 0x7fcd5806f6d0 2024-04-25T14:11:25.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.505+0000 7fcd57e00640 1 -- 172.21.15.86:0/1769447925 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd5809e5e0 con 0x7fcd5806f6d0 2024-04-25T14:11:25.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.505+0000 7fcd57e00640 1 -- 172.21.15.86:0/1769447925 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd581fa2a0 con 0x7fcd5806f6d0 2024-04-25T14:11:25.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.505+0000 7fcd57e00640 1 -- 172.21.15.86:0/1769447925 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd58076b80 con 0x7fcd5806f6d0 2024-04-25T14:11:25.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.509+0000 7fcd4f400640 1 -- 172.21.15.86:0/1769447925 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7fcd480b8ca0 con 0x7fcd5806f6d0 2024-04-25T14:11:25.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.509+0000 7fcd4f400640 1 --2- 172.21.15.86:0/1769447925 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fcd3406f520 0x7fcd34071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:25.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.509+0000 7fcd4f400640 1 -- 172.21.15.86:0/1769447925 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 6066+0+0 (secure 0 0 0) 0x7fcd4812a930 con 0x7fcd5806f6d0 2024-04-25T14:11:25.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.509+0000 7fcd4e000640 1 --2- 172.21.15.86:0/1769447925 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fcd3406f520 0x7fcd34071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:25.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.509+0000 7fcd4e000640 1 --2- 172.21.15.86:0/1769447925 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fcd3406f520 0x7fcd34071a30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcd40006fd0 tx=0x7fcd40008040 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:25.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.513+0000 7fcd4f400640 1 -- 172.21.15.86:0/1769447925 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcd480165e0 con 0x7fcd5806f6d0 2024-04-25T14:11:25.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.669+0000 7fcd57e00640 1 -- 172.21.15.86:0/1769447925 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcd58068c20 con 0x7fcd5806f6d0 2024-04-25T14:11:25.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.669+0000 7fcd4f400640 1 -- 172.21.15.86:0/1769447925 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v132) ==== 93+0+10 (secure 0 0 0) 0x7fcd480f75a0 con 0x7fcd5806f6d0 2024-04-25T14:11:25.667 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:11:25.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.673+0000 7fcd57e00640 1 -- 172.21.15.86:0/1769447925 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fcd3406f520 msgr2=0x7fcd34071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:25.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.673+0000 7fcd57e00640 1 --2- 172.21.15.86:0/1769447925 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fcd3406f520 0x7fcd34071a30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcd40006fd0 tx=0x7fcd40008040 comp rx=0 tx=0).stop 2024-04-25T14:11:25.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.673+0000 7fcd57e00640 1 -- 172.21.15.86:0/1769447925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd5806f6d0 msgr2=0x7fcd5814fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:25.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.673+0000 7fcd57e00640 1 --2- 172.21.15.86:0/1769447925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd5806f6d0 0x7fcd5814fa50 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fcd480af040 tx=0x7fcd48019790 comp rx=0 tx=0).stop 2024-04-25T14:11:25.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.673+0000 7fcd57e00640 1 -- 172.21.15.86:0/1769447925 shutdown_connections 2024-04-25T14:11:25.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.673+0000 7fcd57e00640 1 --2- 172.21.15.86:0/1769447925 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fcd3406f520 0x7fcd34071a30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.673+0000 7fcd57e00640 1 --2- 172.21.15.86:0/1769447925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd58157f80 0x7fcd580b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.673+0000 7fcd57e00640 1 --2- 172.21.15.86:0/1769447925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd5806fae0 0x7fcd580b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.673+0000 7fcd57e00640 1 --2- 172.21.15.86:0/1769447925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd5806f6d0 0x7fcd5814fa50 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.673+0000 7fcd57e00640 1 -- 172.21.15.86:0/1769447925 >> 172.21.15.86:0/1769447925 conn(0x7fcd580b4740 msgr2=0x7fcd58153d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:25.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.673+0000 7fcd57e00640 1 -- 172.21.15.86:0/1769447925 shutdown_connections 2024-04-25T14:11:25.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.673+0000 7fcd57e00640 1 -- 172.21.15.86:0/1769447925 wait complete. 2024-04-25T14:11:25.689 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_metrics_mds_grow' 2024-04-25T14:11:25.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.837+0000 7fba9b200640 1 Processor -- start 2024-04-25T14:11:25.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba9b200640 1 -- start start 2024-04-25T14:11:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba9b200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fba940740c0 0x7fba941676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba9b200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba94072bd0 con 0x7fba940cfca0 2024-04-25T14:11:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba9b200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba9406fbd0 con 0x7fba940d16e0 2024-04-25T14:11:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba9b200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fba94070470 con 0x7fba940740c0 2024-04-25T14:11:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba93e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fba940cfca0 0x7fba9406f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39132/0 (socket says 172.21.15.86:39132) 2024-04-25T14:11:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba93e00640 1 -- 172.21.15.86:0/4179881833 learned_addr learned my addr 172.21.15.86:0/4179881833 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba99200640 1 --2- 172.21.15.86:0/4179881833 >> v2:172.21.15.155:3300/0 conn(0x7fba940740c0 0x7fba941676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba93400640 1 -- 172.21.15.86:0/4179881833 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 401660617 0 0) 0x7fba94072bd0 con 0x7fba940cfca0 2024-04-25T14:11:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba93400640 1 -- 172.21.15.86:0/4179881833 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba78003530 con 0x7fba940cfca0 2024-04-25T14:11:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba93400640 1 -- 172.21.15.86:0/4179881833 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 483491485 0 0) 0x7fba78003530 con 0x7fba940cfca0 2024-04-25T14:11:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba93400640 1 -- 172.21.15.86:0/4179881833 >> v2:172.21.15.155:3300/0 conn(0x7fba940740c0 msgr2=0x7fba941676d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:25.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba93400640 1 --2- 172.21.15.86:0/4179881833 >> v2:172.21.15.155:3300/0 conn(0x7fba940740c0 0x7fba941676d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba93400640 1 -- 172.21.15.86:0/4179881833 >> v1:172.21.15.155:6789/0 conn(0x7fba940d16e0 legacy=0x7fba940d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:25.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba93400640 1 -- 172.21.15.86:0/4179881833 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba9406feb0 con 0x7fba940cfca0 2024-04-25T14:11:25.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba93400640 1 -- 172.21.15.86:0/4179881833 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fba840016a0 con 0x7fba940cfca0 2024-04-25T14:11:25.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba93400640 1 -- 172.21.15.86:0/4179881833 >> v1:172.21.15.86:6789/0 conn(0x7fba940cfca0 legacy=0x7fba9406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:25.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba93400640 1 --2- 172.21.15.86:0/4179881833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba78003a30 0x7fba78003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:25.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba93400640 1 --2- 172.21.15.86:0/4179881833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba78004810 0x7fba78004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:25.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba93400640 1 --2- 172.21.15.86:0/4179881833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba78005230 0x7fba7800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:25.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba93400640 1 -- 172.21.15.86:0/4179881833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba94070470 con 0x7fba78005230 2024-04-25T14:11:25.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba93400640 1 -- 172.21.15.86:0/4179881833 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fba7800c1c0 con 0x7fba78003a30 2024-04-25T14:11:25.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba93400640 1 -- 172.21.15.86:0/4179881833 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fba7800c3c0 con 0x7fba78004810 2024-04-25T14:11:25.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba99200640 1 --2- 172.21.15.86:0/4179881833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba78005230 0x7fba7800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:25.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba93e00640 1 --2- 172.21.15.86:0/4179881833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba78003a30 0x7fba78003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:25.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba8b400640 1 --2- 172.21.15.86:0/4179881833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba78004810 0x7fba78004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:25.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba99200640 1 -- 172.21.15.86:0/4179881833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba78004810 msgr2=0x7fba78004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:25.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba99200640 1 --2- 172.21.15.86:0/4179881833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba78004810 0x7fba78004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba99200640 1 -- 172.21.15.86:0/4179881833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba78003a30 msgr2=0x7fba78003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:25.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba99200640 1 --2- 172.21.15.86:0/4179881833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba78003a30 0x7fba78003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba99200640 1 -- 172.21.15.86:0/4179881833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba78003530 con 0x7fba78005230 2024-04-25T14:11:25.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba93e00640 1 --2- 172.21.15.86:0/4179881833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba78003a30 0x7fba78003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:25.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.841+0000 7fba8b400640 1 --2- 172.21.15.86:0/4179881833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba78004810 0x7fba78004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.845+0000 7fba99200640 1 -- 172.21.15.86:0/4179881833 reap_dead start 2024-04-25T14:11:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.845+0000 7fba99200640 1 --2- 172.21.15.86:0/4179881833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba78005230 0x7fba7800b9d0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fba8c009f90 tx=0x7fba8c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d89d4c2b1b8c053 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.845+0000 7fba93400640 1 -- 172.21.15.86:0/4179881833 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c0b2070 con 0x7fba78005230 2024-04-25T14:11:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.845+0000 7fba93400640 1 -- 172.21.15.86:0/4179881833 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba8c0a5c90 con 0x7fba78005230 2024-04-25T14:11:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.845+0000 7fba93400640 1 -- 172.21.15.86:0/4179881833 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c009c10 con 0x7fba78005230 2024-04-25T14:11:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.845+0000 7fba9b200640 1 -- 172.21.15.86:0/4179881833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba78005230 msgr2=0x7fba7800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.845+0000 7fba9b200640 1 --2- 172.21.15.86:0/4179881833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba78005230 0x7fba7800b9d0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fba8c009f90 tx=0x7fba8c01cc80 comp rx=0 tx=0).stop 2024-04-25T14:11:25.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.845+0000 7fba9b200640 1 -- 172.21.15.86:0/4179881833 shutdown_connections 2024-04-25T14:11:25.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.845+0000 7fba9b200640 1 --2- 172.21.15.86:0/4179881833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba78005230 0x7fba7800b9d0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.845+0000 7fba9b200640 1 -- 172.21.15.86:0/4179881833 >> 172.21.15.86:0/4179881833 conn(0x7fba940b4650 msgr2=0x7fba940b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:25.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.845+0000 7fba9b200640 1 -- 172.21.15.86:0/4179881833 shutdown_connections 2024-04-25T14:11:25.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.845+0000 7fba9b200640 1 -- 172.21.15.86:0/4179881833 wait complete. 2024-04-25T14:11:25.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.849+0000 7fba9b200640 1 Processor -- start 2024-04-25T14:11:25.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.849+0000 7fba9b200640 1 -- start start 2024-04-25T14:11:25.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.849+0000 7fba9b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba78005230 0x7fba9415bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:25.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.849+0000 7fba9b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba941eacb0 0x7fba940af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:25.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.849+0000 7fba9b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba941eb0c0 0x7fba940afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:25.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.849+0000 7fba9b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba9406f570 con 0x7fba941eacb0 2024-04-25T14:11:25.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.849+0000 7fba9b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fba9406f3f0 con 0x7fba78005230 2024-04-25T14:11:25.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.849+0000 7fba9b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fba9406f6f0 con 0x7fba941eb0c0 2024-04-25T14:11:25.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.849+0000 7fba8b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba941eacb0 0x7fba940af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:25.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.849+0000 7fba8b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba941eacb0 0x7fba940af6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49748/0 (socket says 172.21.15.86:49748) 2024-04-25T14:11:25.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.849+0000 7fba8b400640 1 -- 172.21.15.86:0/153428945 learned_addr learned my addr 172.21.15.86:0/153428945 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:25.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.849+0000 7fba93e00640 1 --2- 172.21.15.86:0/153428945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba78005230 0x7fba9415bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:25.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.849+0000 7fba99200640 1 --2- 172.21.15.86:0/153428945 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba941eb0c0 0x7fba940afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:25.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.853+0000 7fba8b400640 1 -- 172.21.15.86:0/153428945 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba941eb0c0 msgr2=0x7fba940afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:25.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.853+0000 7fba8b400640 1 --2- 172.21.15.86:0/153428945 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba941eb0c0 0x7fba940afbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.853+0000 7fba8b400640 1 -- 172.21.15.86:0/153428945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba78005230 msgr2=0x7fba9415bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:25.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.853+0000 7fba8b400640 1 --2- 172.21.15.86:0/153428945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba78005230 0x7fba9415bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:25.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.853+0000 7fba8b400640 1 -- 172.21.15.86:0/153428945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba9415c0e0 con 0x7fba941eacb0 2024-04-25T14:11:25.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.853+0000 7fba8b400640 1 --2- 172.21.15.86:0/153428945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba941eacb0 0x7fba940af6a0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00e4b0 tx=0x7fba7c00e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:25.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.853+0000 7fba90c00640 1 -- 172.21.15.86:0/153428945 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7c0052e0 con 0x7fba941eacb0 2024-04-25T14:11:25.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.853+0000 7fba90c00640 1 -- 172.21.15.86:0/153428945 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba7c018070 con 0x7fba941eacb0 2024-04-25T14:11:25.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.853+0000 7fba9b200640 1 -- 172.21.15.86:0/153428945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba941f9e40 con 0x7fba941eacb0 2024-04-25T14:11:25.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.853+0000 7fba90c00640 1 -- 172.21.15.86:0/153428945 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7c005480 con 0x7fba941eacb0 2024-04-25T14:11:25.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.853+0000 7fba9b200640 1 -- 172.21.15.86:0/153428945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba941fa400 con 0x7fba941eacb0 2024-04-25T14:11:25.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.853+0000 7fba9b200640 1 -- 172.21.15.86:0/153428945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba9409f4b0 con 0x7fba941eacb0 2024-04-25T14:11:25.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.853+0000 7fba90c00640 1 -- 172.21.15.86:0/153428945 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7fba7c003820 con 0x7fba941eacb0 2024-04-25T14:11:25.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.853+0000 7fba90c00640 1 --2- 172.21.15.86:0/153428945 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fba7006f550 0x7fba70071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:25.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.853+0000 7fba90c00640 1 -- 172.21.15.86:0/153428945 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 6066+0+0 (secure 0 0 0) 0x7fba7c08eeb0 con 0x7fba941eacb0 2024-04-25T14:11:25.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.857+0000 7fba93e00640 1 --2- 172.21.15.86:0/153428945 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fba7006f550 0x7fba70071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:25.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.857+0000 7fba93e00640 1 --2- 172.21.15.86:0/153428945 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fba7006f550 0x7fba70071a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fba84001ec0 tx=0x7fba84009040 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:25.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:25.861+0000 7fba90c00640 1 -- 172.21.15.86:0/153428945 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fba7c05bb00 con 0x7fba941eacb0 2024-04-25T14:11:26.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:26.073+0000 7fba9b200640 1 -- 172.21.15.86:0/153428945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_metrics_mds_grow"]} v 0) -- 0x7fba94069d00 con 0x7fba941eacb0 2024-04-25T14:11:26.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:26.977+0000 7fba90c00640 1 -- 172.21.15.86:0/153428945 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_metrics_mds_grow"]}]=0 v432) ==== 148+0+0 (secure 0 0 0) 0x7fba7c05b400 con 0x7fba941eacb0 2024-04-25T14:11:26.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:26.981+0000 7fba9b200640 1 -- 172.21.15.86:0/153428945 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fba7006f550 msgr2=0x7fba70071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:26.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:26.981+0000 7fba9b200640 1 --2- 172.21.15.86:0/153428945 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fba7006f550 0x7fba70071a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fba84001ec0 tx=0x7fba84009040 comp rx=0 tx=0).stop 2024-04-25T14:11:26.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:26.981+0000 7fba9b200640 1 -- 172.21.15.86:0/153428945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba941eacb0 msgr2=0x7fba940af6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:26.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:26.981+0000 7fba9b200640 1 --2- 172.21.15.86:0/153428945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba941eacb0 0x7fba940af6a0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00e4b0 tx=0x7fba7c00e970 comp rx=0 tx=0).stop 2024-04-25T14:11:26.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:26.981+0000 7fba9b200640 1 -- 172.21.15.86:0/153428945 shutdown_connections 2024-04-25T14:11:26.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:26.981+0000 7fba9b200640 1 --2- 172.21.15.86:0/153428945 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fba7006f550 0x7fba70071a60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:26.981+0000 7fba9b200640 1 --2- 172.21.15.86:0/153428945 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba941eb0c0 0x7fba940afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:26.981+0000 7fba9b200640 1 --2- 172.21.15.86:0/153428945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba941eacb0 0x7fba940af6a0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:26.981+0000 7fba9b200640 1 --2- 172.21.15.86:0/153428945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba78005230 0x7fba9415bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:26.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:26.981+0000 7fba9b200640 1 -- 172.21.15.86:0/153428945 >> 172.21.15.86:0/153428945 conn(0x7fba940b4650 msgr2=0x7fba940b14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:26.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:26.981+0000 7fba9b200640 1 -- 172.21.15.86:0/153428945 shutdown_connections 2024-04-25T14:11:26.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:26.981+0000 7fba9b200640 1 -- 172.21.15.86:0/153428945 wait complete. 2024-04-25T14:11:27.003 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T14:11:27.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.157+0000 7fa247200640 1 Processor -- start 2024-04-25T14:11:27.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.157+0000 7fa247200640 1 -- start start 2024-04-25T14:11:27.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.161+0000 7fa247200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa2481615c0 0x7fa2481619d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:27.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.161+0000 7fa247200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2480b6530 0x7fa2480b6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:27.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.161+0000 7fa247200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2481622f0 con 0x7fa24806f990 2024-04-25T14:11:27.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.161+0000 7fa247200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa2480d1730 con 0x7fa2481615c0 2024-04-25T14:11:27.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.161+0000 7fa247200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa2480d1a10 con 0x7fa2480b6530 2024-04-25T14:11:27.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.161+0000 7fa245400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa2481615c0 0x7fa2481619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:27.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.161+0000 7fa246800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2480b6530 0x7fa2480b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:27.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.161+0000 7fa245400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa2481615c0 0x7fa2481619d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:32960/0 (socket says 172.21.15.86:32960) 2024-04-25T14:11:27.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.161+0000 7fa245400640 1 -- 172.21.15.86:0/3378157703 learned_addr learned my addr 172.21.15.86:0/3378157703 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:27.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.161+0000 7fa245e00640 1 --1- 172.21.15.86:0/3378157703 >> v1:172.21.15.155:6789/0 conn(0x7fa24806f990 0x7fa24806fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:33336/0 (socket says 172.21.15.86:33336) 2024-04-25T14:11:27.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.161+0000 7fa246800640 1 -- 172.21.15.86:0/3378157703 >> v2:172.21.15.155:3300/0 conn(0x7fa2481615c0 msgr2=0x7fa2481619d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:27.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.161+0000 7fa246800640 1 --2- 172.21.15.86:0/3378157703 >> v2:172.21.15.155:3300/0 conn(0x7fa2481615c0 0x7fa2481619d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.161+0000 7fa246800640 1 -- 172.21.15.86:0/3378157703 >> v1:172.21.15.155:6789/0 conn(0x7fa24806f990 legacy=0x7fa24806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:27.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.161+0000 7fa244a00640 1 -- 172.21.15.86:0/3378157703 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4165657806 0 0) 0x7fa2481622f0 con 0x7fa24806f990 2024-04-25T14:11:27.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.161+0000 7fa246800640 1 -- 172.21.15.86:0/3378157703 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2480d1cf0 con 0x7fa2480b6530 2024-04-25T14:11:27.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.161+0000 7fa245400640 1 --2- 172.21.15.86:0/3378157703 >> v2:172.21.15.155:3300/0 conn(0x7fa2481615c0 0x7fa2481619d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:11:27.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.165+0000 7fa246800640 1 --2- 172.21.15.86:0/3378157703 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2480b6530 0x7fa2480b6940 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa240009d90 tx=0x7fa24001c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25a868e810614c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:27.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.165+0000 7fa244a00640 1 -- 172.21.15.86:0/3378157703 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2400b2070 con 0x7fa2480b6530 2024-04-25T14:11:27.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.165+0000 7fa244a00640 1 -- 172.21.15.86:0/3378157703 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2400a57a0 con 0x7fa2480b6530 2024-04-25T14:11:27.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.165+0000 7fa244a00640 1 -- 172.21.15.86:0/3378157703 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2400a5b70 con 0x7fa2480b6530 2024-04-25T14:11:27.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.165+0000 7fa247200640 1 -- 172.21.15.86:0/3378157703 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2480b6530 msgr2=0x7fa2480b6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:27.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.165+0000 7fa247200640 1 --2- 172.21.15.86:0/3378157703 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2480b6530 0x7fa2480b6940 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa240009d90 tx=0x7fa24001c8e0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.165+0000 7fa247200640 1 -- 172.21.15.86:0/3378157703 shutdown_connections 2024-04-25T14:11:27.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.165+0000 7fa247200640 1 --2- 172.21.15.86:0/3378157703 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2480b6530 0x7fa2480b6940 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.165+0000 7fa247200640 1 --2- 172.21.15.86:0/3378157703 >> v2:172.21.15.155:3300/0 conn(0x7fa2481615c0 0x7fa2481619d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.165+0000 7fa247200640 1 -- 172.21.15.86:0/3378157703 >> 172.21.15.86:0/3378157703 conn(0x7fa2480cfea0 msgr2=0x7fa2480d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:27.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.165+0000 7fa247200640 1 -- 172.21.15.86:0/3378157703 shutdown_connections 2024-04-25T14:11:27.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.165+0000 7fa247200640 1 -- 172.21.15.86:0/3378157703 wait complete. 2024-04-25T14:11:27.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.169+0000 7fa247200640 1 Processor -- start 2024-04-25T14:11:27.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.169+0000 7fa247200640 1 -- start start 2024-04-25T14:11:27.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.169+0000 7fa247200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa24806f990 0x7fa2480cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:27.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.169+0000 7fa247200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2480b6530 0x7fa248151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:27.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.169+0000 7fa247200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2481615c0 0x7fa24815ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:27.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.169+0000 7fa247200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa24815df20 con 0x7fa2481615c0 2024-04-25T14:11:27.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.169+0000 7fa247200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa2480d2840 con 0x7fa24806f990 2024-04-25T14:11:27.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.169+0000 7fa247200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa24816b170 con 0x7fa2480b6530 2024-04-25T14:11:27.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.169+0000 7fa246800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2481615c0 0x7fa24815ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:27.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.169+0000 7fa246800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2481615c0 0x7fa24815ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49752/0 (socket says 172.21.15.86:49752) 2024-04-25T14:11:27.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.169+0000 7fa246800640 1 -- 172.21.15.86:0/2382653790 learned_addr learned my addr 172.21.15.86:0/2382653790 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:27.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.169+0000 7fa245e00640 1 --2- 172.21.15.86:0/2382653790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa24806f990 0x7fa2480cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:27.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.169+0000 7fa245400640 1 --2- 172.21.15.86:0/2382653790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2480b6530 0x7fa248151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:27.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.169+0000 7fa246800640 1 -- 172.21.15.86:0/2382653790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2480b6530 msgr2=0x7fa248151ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:27.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.169+0000 7fa246800640 1 --2- 172.21.15.86:0/2382653790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2480b6530 0x7fa248151ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.169+0000 7fa246800640 1 -- 172.21.15.86:0/2382653790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa24806f990 msgr2=0x7fa2480cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:27.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.169+0000 7fa246800640 1 --2- 172.21.15.86:0/2382653790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa24806f990 0x7fa2480cb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.169+0000 7fa246800640 1 -- 172.21.15.86:0/2382653790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2481555e0 con 0x7fa2481615c0 2024-04-25T14:11:27.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.169+0000 7fa246800640 1 --2- 172.21.15.86:0/2382653790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2481615c0 0x7fa24815ed00 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fa2400b0040 tx=0x7fa240019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:27.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.169+0000 7fa23e000640 1 -- 172.21.15.86:0/2382653790 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2400b2050 con 0x7fa2481615c0 2024-04-25T14:11:27.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.169+0000 7fa23e000640 1 -- 172.21.15.86:0/2382653790 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2400b3040 con 0x7fa2481615c0 2024-04-25T14:11:27.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.169+0000 7fa247200640 1 -- 172.21.15.86:0/2382653790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa24815e220 con 0x7fa2481615c0 2024-04-25T14:11:27.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.169+0000 7fa23e000640 1 -- 172.21.15.86:0/2382653790 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2400ad650 con 0x7fa2481615c0 2024-04-25T14:11:27.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.169+0000 7fa247200640 1 -- 172.21.15.86:0/2382653790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa248152220 con 0x7fa2481615c0 2024-04-25T14:11:27.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.173+0000 7fa247200640 1 -- 172.21.15.86:0/2382653790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa248076b80 con 0x7fa2481615c0 2024-04-25T14:11:27.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.173+0000 7fa23e000640 1 -- 172.21.15.86:0/2382653790 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7fa2400c0020 con 0x7fa2481615c0 2024-04-25T14:11:27.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.173+0000 7fa23e000640 1 --2- 172.21.15.86:0/2382653790 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa22806f520 0x7fa228071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:27.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.173+0000 7fa23e000640 1 -- 172.21.15.86:0/2382653790 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 6066+0+0 (secure 0 0 0) 0x7fa24012a5e0 con 0x7fa2481615c0 2024-04-25T14:11:27.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.173+0000 7fa245e00640 1 --2- 172.21.15.86:0/2382653790 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa22806f520 0x7fa228071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:27.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.177+0000 7fa245e00640 1 --2- 172.21.15.86:0/2382653790 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa22806f520 0x7fa228071a30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa238000f70 tx=0x7fa238007df0 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:27.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.177+0000 7fa23e000640 1 -- 172.21.15.86:0/2382653790 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa240016560 con 0x7fa2481615c0 2024-04-25T14:11:27.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.333+0000 7fa247200640 1 -- 172.21.15.86:0/2382653790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa248069d00 con 0x7fa2481615c0 2024-04-25T14:11:27.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.337+0000 7fa23e000640 1 -- 172.21.15.86:0/2382653790 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7fa2400f6e90 con 0x7fa2481615c0 2024-04-25T14:11:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.337+0000 7fa247200640 1 -- 172.21.15.86:0/2382653790 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa22806f520 msgr2=0x7fa228071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.337+0000 7fa247200640 1 --2- 172.21.15.86:0/2382653790 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa22806f520 0x7fa228071a30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa238000f70 tx=0x7fa238007df0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.337+0000 7fa247200640 1 -- 172.21.15.86:0/2382653790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2481615c0 msgr2=0x7fa24815ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.337+0000 7fa247200640 1 --2- 172.21.15.86:0/2382653790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2481615c0 0x7fa24815ed00 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fa2400b0040 tx=0x7fa240019680 comp rx=0 tx=0).stop 2024-04-25T14:11:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.337+0000 7fa247200640 1 -- 172.21.15.86:0/2382653790 shutdown_connections 2024-04-25T14:11:27.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.337+0000 7fa247200640 1 --2- 172.21.15.86:0/2382653790 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa22806f520 0x7fa228071a30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.337+0000 7fa247200640 1 --2- 172.21.15.86:0/2382653790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2481615c0 0x7fa24815ed00 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.337+0000 7fa247200640 1 --2- 172.21.15.86:0/2382653790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2480b6530 0x7fa248151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.337+0000 7fa247200640 1 --2- 172.21.15.86:0/2382653790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa24806f990 0x7fa2480cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.337+0000 7fa247200640 1 -- 172.21.15.86:0/2382653790 >> 172.21.15.86:0/2382653790 conn(0x7fa2480cfea0 msgr2=0x7fa2480b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:27.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.341+0000 7fa247200640 1 -- 172.21.15.86:0/2382653790 shutdown_connections 2024-04-25T14:11:27.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.341+0000 7fa247200640 1 -- 172.21.15.86:0/2382653790 wait complete. 2024-04-25T14:11:27.356 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T14:11:27.357 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:11:27.478 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:11:27.598 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T14:11:27.599 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:11:27.599 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:11:27.607 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2893 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T14:11:27.608 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:11:27.608 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:11:27.616 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4517 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T14:11:27.617 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:11:27.617 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:11:27.659 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2893 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T14:11:27.660 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:11:27.660 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:11:27.668 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4517 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T14:11:27.668 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T14:11:27.669 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:11:27.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.853+0000 7f0313e00640 1 Processor -- start 2024-04-25T14:11:27.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.853+0000 7f0313e00640 1 -- start start 2024-04-25T14:11:27.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.857+0000 7f0313e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f030c15c2a0 0x7f030c0ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:27.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.857+0000 7f0313e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f030c15feb0 0x7f030c1602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:27.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.857+0000 7f0313e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f030c0b0a60 con 0x7f030c06fd50 2024-04-25T14:11:27.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.857+0000 7f0313e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f030c06ef90 con 0x7f030c15c2a0 2024-04-25T14:11:27.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.857+0000 7f0313e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f030c06f270 con 0x7f030c15feb0 2024-04-25T14:11:27.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.857+0000 7f0311e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f030c15feb0 0x7f030c1602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:27.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.857+0000 7f0311400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f030c15c2a0 0x7f030c0ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:27.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.857+0000 7f0310a00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f030c06fd50 0x7f030c070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:33348/0 (socket says 172.21.15.86:33348) 2024-04-25T14:11:27.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.857+0000 7f0310a00640 1 -- 172.21.15.86:0/659988549 learned_addr learned my addr 172.21.15.86:0/659988549 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:27.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.857+0000 7f030be00640 1 -- 172.21.15.86:0/659988549 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3960903344 0 0) 0x7f030c0b0a60 con 0x7f030c06fd50 2024-04-25T14:11:27.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.857+0000 7f030be00640 1 -- 172.21.15.86:0/659988549 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02f4003530 con 0x7f030c06fd50 2024-04-25T14:11:27.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.857+0000 7f0311e00640 1 -- 172.21.15.86:0/659988549 >> v2:172.21.15.155:3300/0 conn(0x7f030c15c2a0 msgr2=0x7f030c0ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:27.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.857+0000 7f0311e00640 1 --2- 172.21.15.86:0/659988549 >> v2:172.21.15.155:3300/0 conn(0x7f030c15c2a0 0x7f030c0ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.857+0000 7f0311e00640 1 -- 172.21.15.86:0/659988549 >> v1:172.21.15.155:6789/0 conn(0x7f030c06fd50 legacy=0x7f030c070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:27.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.857+0000 7f0311e00640 1 -- 172.21.15.86:0/659988549 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030c06f550 con 0x7f030c15feb0 2024-04-25T14:11:27.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.857+0000 7f0311400640 1 --2- 172.21.15.86:0/659988549 >> v2:172.21.15.155:3300/0 conn(0x7f030c15c2a0 0x7f030c0ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:11:27.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.861+0000 7f0311e00640 1 --2- 172.21.15.86:0/659988549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f030c15feb0 0x7f030c1602c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0304009d90 tx=0x7f030401d310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9d26b042399acb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:27.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.861+0000 7f030be00640 1 -- 172.21.15.86:0/659988549 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03040b2070 con 0x7f030c15feb0 2024-04-25T14:11:27.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.861+0000 7f030be00640 1 -- 172.21.15.86:0/659988549 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03040a5910 con 0x7f030c15feb0 2024-04-25T14:11:27.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.861+0000 7f030be00640 1 -- 172.21.15.86:0/659988549 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03040a5cc0 con 0x7f030c15feb0 2024-04-25T14:11:27.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.861+0000 7f0313e00640 1 -- 172.21.15.86:0/659988549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f030c15feb0 msgr2=0x7f030c1602c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:27.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.861+0000 7f0313e00640 1 --2- 172.21.15.86:0/659988549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f030c15feb0 0x7f030c1602c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0304009d90 tx=0x7f030401d310 comp rx=0 tx=0).stop 2024-04-25T14:11:27.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.861+0000 7f0313e00640 1 -- 172.21.15.86:0/659988549 shutdown_connections 2024-04-25T14:11:27.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.861+0000 7f0313e00640 1 --2- 172.21.15.86:0/659988549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f030c15feb0 0x7f030c1602c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.861+0000 7f0313e00640 1 --2- 172.21.15.86:0/659988549 >> v2:172.21.15.155:3300/0 conn(0x7f030c15c2a0 0x7f030c0ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.861+0000 7f0313e00640 1 -- 172.21.15.86:0/659988549 >> 172.21.15.86:0/659988549 conn(0x7f030c0b4650 msgr2=0x7f030c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:27.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.861+0000 7f0313e00640 1 -- 172.21.15.86:0/659988549 shutdown_connections 2024-04-25T14:11:27.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.861+0000 7f0313e00640 1 -- 172.21.15.86:0/659988549 wait complete. 2024-04-25T14:11:27.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.865+0000 7f0313e00640 1 Processor -- start 2024-04-25T14:11:27.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.865+0000 7f0313e00640 1 -- start start 2024-04-25T14:11:27.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.865+0000 7f0313e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f030c06fd50 0x7f030c14ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:27.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.865+0000 7f0313e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f030c15c2a0 0x7f030c1fd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:27.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.865+0000 7f0313e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f030c15feb0 0x7f030c155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:27.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.865+0000 7f0313e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f030c160d90 con 0x7f030c06fd50 2024-04-25T14:11:27.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.865+0000 7f0313e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f030c160c10 con 0x7f030c15c2a0 2024-04-25T14:11:27.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.865+0000 7f0313e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f030c0ae670 con 0x7f030c15feb0 2024-04-25T14:11:27.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.865+0000 7f0311400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f030c06fd50 0x7f030c14ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:27.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.865+0000 7f0311400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f030c06fd50 0x7f030c14ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49760/0 (socket says 172.21.15.86:49760) 2024-04-25T14:11:27.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.865+0000 7f0311400640 1 -- 172.21.15.86:0/4099249094 learned_addr learned my addr 172.21.15.86:0/4099249094 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:27.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.865+0000 7f0311e00640 1 --2- 172.21.15.86:0/4099249094 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f030c15feb0 0x7f030c155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:27.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.865+0000 7f0310a00640 1 --2- 172.21.15.86:0/4099249094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f030c15c2a0 0x7f030c1fd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:27.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.865+0000 7f0311400640 1 -- 172.21.15.86:0/4099249094 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f030c15feb0 msgr2=0x7f030c155ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:27.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.865+0000 7f0311400640 1 --2- 172.21.15.86:0/4099249094 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f030c15feb0 0x7f030c155ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.865+0000 7f0311400640 1 -- 172.21.15.86:0/4099249094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f030c15c2a0 msgr2=0x7f030c1fd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:27.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.865+0000 7f0311400640 1 --2- 172.21.15.86:0/4099249094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f030c15c2a0 0x7f030c1fd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:27.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.865+0000 7f0311400640 1 -- 172.21.15.86:0/4099249094 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030c0cd6b0 con 0x7f030c06fd50 2024-04-25T14:11:27.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.865+0000 7f0311400640 1 --2- 172.21.15.86:0/4099249094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f030c06fd50 0x7f030c14ff60 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f02fc00e940 tx=0x7f02fc00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:27.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.865+0000 7f0309600640 1 -- 172.21.15.86:0/4099249094 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc016070 con 0x7f030c06fd50 2024-04-25T14:11:27.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.865+0000 7f0309600640 1 -- 172.21.15.86:0/4099249094 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02fc00cc40 con 0x7f030c06fd50 2024-04-25T14:11:27.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.865+0000 7f0313e00640 1 -- 172.21.15.86:0/4099249094 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f030c0cd950 con 0x7f030c06fd50 2024-04-25T14:11:27.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.865+0000 7f0309600640 1 -- 172.21.15.86:0/4099249094 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc00b040 con 0x7f030c06fd50 2024-04-25T14:11:27.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.865+0000 7f0313e00640 1 -- 172.21.15.86:0/4099249094 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f030c0cdc30 con 0x7f030c06fd50 2024-04-25T14:11:27.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.869+0000 7f0313e00640 1 -- 172.21.15.86:0/4099249094 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f030c069d00 con 0x7f030c06fd50 2024-04-25T14:11:27.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.869+0000 7f0309600640 1 -- 172.21.15.86:0/4099249094 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7f02fc019070 con 0x7f030c06fd50 2024-04-25T14:11:27.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.869+0000 7f0309600640 1 --2- 172.21.15.86:0/4099249094 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f02f006f520 0x7f02f0071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:27.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.869+0000 7f0309600640 1 -- 172.21.15.86:0/4099249094 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 6066+0+0 (secure 0 0 0) 0x7f02fc08f0b0 con 0x7f030c06fd50 2024-04-25T14:11:27.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.869+0000 7f0310a00640 1 --2- 172.21.15.86:0/4099249094 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f02f006f520 0x7f02f0071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:27.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.869+0000 7f0310a00640 1 --2- 172.21.15.86:0/4099249094 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f02f006f520 0x7f02f0071a30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0300007fd0 tx=0x7f0300009040 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:27.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:27.873+0000 7f0309600640 1 -- 172.21.15.86:0/4099249094 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f02fc05b5c0 con 0x7f030c06fd50 2024-04-25T14:11:28.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.065+0000 7f0313e00640 1 -- 172.21.15.86:0/4099249094 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f030c076bf0 con 0x7f030c06fd50 2024-04-25T14:11:28.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.065+0000 7f0309600640 1 -- 172.21.15.86:0/4099249094 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+3875 (secure 0 0 0) 0x7f02fc05f8c0 con 0x7f030c06fd50 2024-04-25T14:11:28.063 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:28.063 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":96,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2258481468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2258481468},{"type":"v1","addr":"172.21.15.155:6834","nonce":2258481468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":8976,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1064843151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1064843151},{"type":"v1","addr":"172.21.15.86:6835","nonce":1064843151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":8985,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1940755625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1940755625},{"type":"v1","addr":"172.21.15.86:6837","nonce":1940755625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":9046,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2535578744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2535578744},{"type":"v1","addr":"172.21.15.155:6835","nonce":2535578744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96}],"filesystems":[]} 2024-04-25T14:11:28.064 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 96 2024-04-25T14:11:28.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.069+0000 7f0313e00640 1 -- 172.21.15.86:0/4099249094 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f02f006f520 msgr2=0x7f02f0071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:28.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.069+0000 7f0313e00640 1 --2- 172.21.15.86:0/4099249094 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f02f006f520 0x7f02f0071a30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0300007fd0 tx=0x7f0300009040 comp rx=0 tx=0).stop 2024-04-25T14:11:28.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.069+0000 7f0313e00640 1 -- 172.21.15.86:0/4099249094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f030c06fd50 msgr2=0x7f030c14ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:28.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.069+0000 7f0313e00640 1 --2- 172.21.15.86:0/4099249094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f030c06fd50 0x7f030c14ff60 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f02fc00e940 tx=0x7f02fc00ee00 comp rx=0 tx=0).stop 2024-04-25T14:11:28.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.069+0000 7f0313e00640 1 -- 172.21.15.86:0/4099249094 shutdown_connections 2024-04-25T14:11:28.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.069+0000 7f0313e00640 1 --2- 172.21.15.86:0/4099249094 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f02f006f520 0x7f02f0071a30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:28.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.069+0000 7f0313e00640 1 --2- 172.21.15.86:0/4099249094 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f030c15feb0 0x7f030c155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:28.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.069+0000 7f0313e00640 1 --2- 172.21.15.86:0/4099249094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f030c15c2a0 0x7f030c1fd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:28.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.069+0000 7f0313e00640 1 --2- 172.21.15.86:0/4099249094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f030c06fd50 0x7f030c14ff60 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:28.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.069+0000 7f0313e00640 1 -- 172.21.15.86:0/4099249094 >> 172.21.15.86:0/4099249094 conn(0x7f030c0b4650 msgr2=0x7f030c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:28.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.069+0000 7f0313e00640 1 -- 172.21.15.86:0/4099249094 shutdown_connections 2024-04-25T14:11:28.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.069+0000 7f0313e00640 1 -- 172.21.15.86:0/4099249094 wait complete. 2024-04-25T14:11:28.085 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T14:11:28.085 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T14:11:28.085 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T14:11:28.085 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:11:28.086 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T14:11:28.086 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T14:11:28.086 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T14:11:28.086 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:11:28.086 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T14:11:28.086 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T14:11:28.087 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T14:11:28.087 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:11:28.087 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T14:11:28.087 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T14:11:28.087 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T14:11:28.087 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:11:28.097 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T14:11:28.097 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T14:11:28.098 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T14:11:28.099 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T14:11:28.105 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T14:11:28.105 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T14:11:28.111 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T14:11:28.111 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T14:11:28.142 INFO:tasks.ceph.mds.a:Started 2024-04-25T14:11:28.142 DEBUG:teuthology.parallel:result is None 2024-04-25T14:11:28.144 INFO:tasks.ceph.mds.c:Started 2024-04-25T14:11:28.144 DEBUG:teuthology.parallel:result is None 2024-04-25T14:11:28.154 INFO:tasks.ceph.mds.d:Started 2024-04-25T14:11:28.154 DEBUG:teuthology.parallel:result is None 2024-04-25T14:11:28.156 INFO:tasks.ceph.mds.b:Started 2024-04-25T14:11:28.156 DEBUG:teuthology.parallel:result is None 2024-04-25T14:11:28.156 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T14:11:28.241 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-25T14:11:28.243 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-04-25T14:11:28.260 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2024-04-25T14:11:28.260 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2024-04-25T14:11:28.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.317+0000 7fa0bc800640 1 Processor -- start 2024-04-25T14:11:28.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.317+0000 7fa0bc800640 1 -- start start 2024-04-25T14:11:28.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.317+0000 7fa0bc800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0b40cfca0 0x7fa0b406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:28.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.317+0000 7fa0bc800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0b4072bd0 con 0x7fa0b40d16e0 2024-04-25T14:11:28.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.321+0000 7fa0bc800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0b406f930 con 0x7fa0b40730b0 2024-04-25T14:11:28.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.321+0000 7fa0bc800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b4070470 con 0x7fa0b40cfca0 2024-04-25T14:11:28.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.321+0000 7fa0b9400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa0b40d16e0 0x7fa0b40d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39160/0 (socket says 172.21.15.86:39160) 2024-04-25T14:11:28.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.321+0000 7fa0b9400640 1 -- 172.21.15.86:0/3998431536 learned_addr learned my addr 172.21.15.86:0/3998431536 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:28.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.321+0000 7fa0b9e00640 1 --2- 172.21.15.86:0/3998431536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0b40cfca0 0x7fa0b406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:28.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.321+0000 7fa0b8a00640 1 -- 172.21.15.86:0/3998431536 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1744785851 0 0) 0x7fa0b4072bd0 con 0x7fa0b40d16e0 2024-04-25T14:11:28.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.321+0000 7fa0b8a00640 1 -- 172.21.15.86:0/3998431536 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa09c003530 con 0x7fa0b40d16e0 2024-04-25T14:11:28.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.321+0000 7fa0b8a00640 1 -- 172.21.15.86:0/3998431536 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3073017915 0 0) 0x7fa0b406f930 con 0x7fa0b40730b0 2024-04-25T14:11:28.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.321+0000 7fa0b8a00640 1 -- 172.21.15.86:0/3998431536 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0b4072bd0 con 0x7fa0b40730b0 2024-04-25T14:11:28.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.321+0000 7fa0b9e00640 1 -- 172.21.15.86:0/3998431536 >> v1:172.21.15.155:6789/0 conn(0x7fa0b40730b0 legacy=0x7fa0b40734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:28.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.321+0000 7fa0b9e00640 1 -- 172.21.15.86:0/3998431536 >> v1:172.21.15.86:6789/0 conn(0x7fa0b40d16e0 legacy=0x7fa0b40d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:28.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.321+0000 7fa0b9e00640 1 -- 172.21.15.86:0/3998431536 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b406fbd0 con 0x7fa0b40cfca0 2024-04-25T14:11:28.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.325+0000 7fa0b9e00640 1 --2- 172.21.15.86:0/3998431536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0b40cfca0 0x7fa0b406f3f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80097f0 tx=0x7fa0a801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8ed49d5c1635946 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:28.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.325+0000 7fa0b8a00640 1 -- 172.21.15.86:0/3998431536 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b40743e0 con 0x7fa0b40cfca0 2024-04-25T14:11:28.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.325+0000 7fa0b8a00640 1 -- 172.21.15.86:0/3998431536 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0a80b0070 con 0x7fa0b40cfca0 2024-04-25T14:11:28.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.325+0000 7fa0b8a00640 1 -- 172.21.15.86:0/3998431536 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a80ab810 con 0x7fa0b40cfca0 2024-04-25T14:11:28.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.325+0000 7fa0bc800640 1 -- 172.21.15.86:0/3998431536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0b40cfca0 msgr2=0x7fa0b406f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:28.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.325+0000 7fa0bc800640 1 --2- 172.21.15.86:0/3998431536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0b40cfca0 0x7fa0b406f3f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80097f0 tx=0x7fa0a801c4e0 comp rx=0 tx=0).stop 2024-04-25T14:11:28.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.325+0000 7fa0bc800640 1 -- 172.21.15.86:0/3998431536 shutdown_connections 2024-04-25T14:11:28.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.325+0000 7fa0bc800640 1 --2- 172.21.15.86:0/3998431536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0b40cfca0 0x7fa0b406f3f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:28.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.325+0000 7fa0bc800640 1 -- 172.21.15.86:0/3998431536 >> 172.21.15.86:0/3998431536 conn(0x7fa0b40b4650 msgr2=0x7fa0b40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:28.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.325+0000 7fa0bc800640 1 -- 172.21.15.86:0/3998431536 shutdown_connections 2024-04-25T14:11:28.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.325+0000 7fa0bc800640 1 -- 172.21.15.86:0/3998431536 wait complete. 2024-04-25T14:11:28.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.325+0000 7fa0bc800640 1 Processor -- start 2024-04-25T14:11:28.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.325+0000 7fa0bc800640 1 -- start start 2024-04-25T14:11:28.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.329+0000 7fa0bc800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0b40730b0 0x7fa0b40c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:28.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.329+0000 7fa0bc800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0b40cfca0 0x7fa0b409a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:28.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.329+0000 7fa0bc800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0b40d16e0 0x7fa0b40b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:28.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.329+0000 7fa0bc800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b416aa80 con 0x7fa0b40d16e0 2024-04-25T14:11:28.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.329+0000 7fa0bc800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b416a900 con 0x7fa0b40730b0 2024-04-25T14:11:28.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.329+0000 7fa0bc800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b416ac00 con 0x7fa0b40cfca0 2024-04-25T14:11:28.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.329+0000 7fa0ba800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0b40d16e0 0x7fa0b40b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:28.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.329+0000 7fa0ba800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0b40d16e0 0x7fa0b40b0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49792/0 (socket says 172.21.15.86:49792) 2024-04-25T14:11:28.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.329+0000 7fa0ba800640 1 -- 172.21.15.86:0/2076182334 learned_addr learned my addr 172.21.15.86:0/2076182334 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:28.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.329+0000 7fa0b9400640 1 --2- 172.21.15.86:0/2076182334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0b40cfca0 0x7fa0b409a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:28.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.329+0000 7fa0b9e00640 1 --2- 172.21.15.86:0/2076182334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0b40730b0 0x7fa0b40c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:28.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.329+0000 7fa0ba800640 1 -- 172.21.15.86:0/2076182334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0b40cfca0 msgr2=0x7fa0b409a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:28.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.329+0000 7fa0ba800640 1 --2- 172.21.15.86:0/2076182334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0b40cfca0 0x7fa0b409a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:28.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.329+0000 7fa0ba800640 1 -- 172.21.15.86:0/2076182334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0b40730b0 msgr2=0x7fa0b40c9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:28.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.329+0000 7fa0ba800640 1 --2- 172.21.15.86:0/2076182334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0b40730b0 0x7fa0b40c9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:28.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.329+0000 7fa0ba800640 1 -- 172.21.15.86:0/2076182334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b40ca480 con 0x7fa0b40d16e0 2024-04-25T14:11:28.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.329+0000 7fa0ba800640 1 --2- 172.21.15.86:0/2076182334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0b40d16e0 0x7fa0b40b0dc0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000c6b0 tx=0x7fa0b000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:28.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.329+0000 7fa0ae000640 1 -- 172.21.15.86:0/2076182334 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0003b80 con 0x7fa0b40d16e0 2024-04-25T14:11:28.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.329+0000 7fa0ae000640 1 -- 172.21.15.86:0/2076182334 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0b0003d40 con 0x7fa0b40d16e0 2024-04-25T14:11:28.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.329+0000 7fa0ae000640 1 -- 172.21.15.86:0/2076182334 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0005310 con 0x7fa0b40d16e0 2024-04-25T14:11:28.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.329+0000 7fa0bc800640 1 -- 172.21.15.86:0/2076182334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b40ca760 con 0x7fa0b40d16e0 2024-04-25T14:11:28.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.329+0000 7fa0bc800640 1 -- 172.21.15.86:0/2076182334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b40ae410 con 0x7fa0b40d16e0 2024-04-25T14:11:28.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.329+0000 7fa0bc800640 1 -- 172.21.15.86:0/2076182334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b409f4b0 con 0x7fa0b40d16e0 2024-04-25T14:11:28.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.333+0000 7fa0ae000640 1 -- 172.21.15.86:0/2076182334 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7fa0b00054b0 con 0x7fa0b40d16e0 2024-04-25T14:11:28.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.333+0000 7fa0ae000640 1 --2- 172.21.15.86:0/2076182334 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa09806f520 0x7fa098071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:28.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.333+0000 7fa0ae000640 1 -- 172.21.15.86:0/2076182334 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 6066+0+0 (secure 0 0 0) 0x7fa0b008fdf0 con 0x7fa0b40d16e0 2024-04-25T14:11:28.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.333+0000 7fa0b9e00640 1 --2- 172.21.15.86:0/2076182334 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa09806f520 0x7fa098071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:28.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.333+0000 7fa0b9e00640 1 --2- 172.21.15.86:0/2076182334 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa09806f520 0x7fa098071a30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80af000 tx=0x7fa0a80a7000 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:28.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.337+0000 7fa0ae000640 1 -- 172.21.15.86:0/2076182334 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa0b005c3f0 con 0x7fa0b40d16e0 2024-04-25T14:11:28.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.501+0000 7fa0bc800640 1 -- 172.21.15.86:0/2076182334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fa0b4069d00 con 0x7fa0b40d16e0 2024-04-25T14:11:28.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.993+0000 7fa0ae000640 1 -- 172.21.15.86:0/2076182334 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v134) ==== 97+0+0 (secure 0 0 0) 0x7fa0b00606f0 con 0x7fa0b40d16e0 2024-04-25T14:11:28.991 INFO:teuthology.orchestra.run.smithi086.stderr: removed all blocklist entries 2024-04-25T14:11:28.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.997+0000 7fa0bc800640 1 -- 172.21.15.86:0/2076182334 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa09806f520 msgr2=0x7fa098071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:28.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.997+0000 7fa0bc800640 1 --2- 172.21.15.86:0/2076182334 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa09806f520 0x7fa098071a30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80af000 tx=0x7fa0a80a7000 comp rx=0 tx=0).stop 2024-04-25T14:11:28.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.997+0000 7fa0bc800640 1 -- 172.21.15.86:0/2076182334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0b40d16e0 msgr2=0x7fa0b40b0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:28.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.997+0000 7fa0bc800640 1 --2- 172.21.15.86:0/2076182334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0b40d16e0 0x7fa0b40b0dc0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000c6b0 tx=0x7fa0b000cb70 comp rx=0 tx=0).stop 2024-04-25T14:11:28.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.997+0000 7fa0bc800640 1 -- 172.21.15.86:0/2076182334 shutdown_connections 2024-04-25T14:11:28.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.997+0000 7fa0bc800640 1 --2- 172.21.15.86:0/2076182334 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa09806f520 0x7fa098071a30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:28.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.997+0000 7fa0bc800640 1 --2- 172.21.15.86:0/2076182334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0b40d16e0 0x7fa0b40b0dc0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:28.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.997+0000 7fa0bc800640 1 --2- 172.21.15.86:0/2076182334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0b40cfca0 0x7fa0b409a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:28.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.997+0000 7fa0bc800640 1 --2- 172.21.15.86:0/2076182334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0b40730b0 0x7fa0b40c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:28.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.997+0000 7fa0bc800640 1 -- 172.21.15.86:0/2076182334 >> 172.21.15.86:0/2076182334 conn(0x7fa0b40b4650 msgr2=0x7fa0b415c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:28.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.997+0000 7fa0bc800640 1 -- 172.21.15.86:0/2076182334 shutdown_connections 2024-04-25T14:11:28.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:28.997+0000 7fa0bc800640 1 -- 172.21.15.86:0/2076182334 wait complete. 2024-04-25T14:11:29.016 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T14:11:29.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.165+0000 7fea02400640 1 Processor -- start 2024-04-25T14:11:29.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.165+0000 7fea02400640 1 -- start start 2024-04-25T14:11:29.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.165+0000 7fea02400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe9fc0740c0 0x7fe9fc1676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:29.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.169+0000 7fea02400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9fc072bd0 con 0x7fe9fc0cfca0 2024-04-25T14:11:29.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.169+0000 7fea02400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9fc06fbd0 con 0x7fe9fc0d16e0 2024-04-25T14:11:29.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.169+0000 7fea02400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe9fc070470 con 0x7fe9fc0740c0 2024-04-25T14:11:29.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.169+0000 7fe9fb400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe9fc0cfca0 0x7fe9fc06f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60406/0 (socket says 172.21.15.86:60406) 2024-04-25T14:11:29.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.169+0000 7fe9fb400640 1 -- 172.21.15.86:0/1656985428 learned_addr learned my addr 172.21.15.86:0/1656985428 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:29.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.169+0000 7fe9fbe00640 1 --2- 172.21.15.86:0/1656985428 >> v2:172.21.15.86:3300/0 conn(0x7fe9fc0740c0 0x7fe9fc1676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:29.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.169+0000 7fe9fbe00640 1 -- 172.21.15.86:0/1656985428 >> v1:172.21.15.155:6789/0 conn(0x7fe9fc0d16e0 legacy=0x7fe9fc0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:29.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.169+0000 7fe9fbe00640 1 -- 172.21.15.86:0/1656985428 >> v1:172.21.15.86:6789/0 conn(0x7fe9fc0cfca0 legacy=0x7fe9fc06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:29.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.169+0000 7fe9faa00640 1 -- 172.21.15.86:0/1656985428 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1465868384 0 0) 0x7fe9fc072bd0 con 0x7fe9fc0cfca0 2024-04-25T14:11:29.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.169+0000 7fe9fbe00640 1 -- 172.21.15.86:0/1656985428 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9fc06feb0 con 0x7fe9fc0740c0 2024-04-25T14:11:29.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.169+0000 7fe9fbe00640 1 --2- 172.21.15.86:0/1656985428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9fc0740c0 0x7fe9fc1676d0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009d90 tx=0x7fe9f401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5d0db6e4ec034a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:29.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.169+0000 7fe9faa00640 1 -- 172.21.15.86:0/1656985428 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f40b2070 con 0x7fe9fc0740c0 2024-04-25T14:11:29.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.169+0000 7fe9faa00640 1 -- 172.21.15.86:0/1656985428 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f40a5940 con 0x7fe9fc0740c0 2024-04-25T14:11:29.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.169+0000 7fe9faa00640 1 -- 172.21.15.86:0/1656985428 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f40a5d10 con 0x7fe9fc0740c0 2024-04-25T14:11:29.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.173+0000 7fea02400640 1 -- 172.21.15.86:0/1656985428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9fc0740c0 msgr2=0x7fe9fc1676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:29.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.173+0000 7fea02400640 1 --2- 172.21.15.86:0/1656985428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9fc0740c0 0x7fe9fc1676d0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009d90 tx=0x7fe9f401c970 comp rx=0 tx=0).stop 2024-04-25T14:11:29.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.173+0000 7fea02400640 1 -- 172.21.15.86:0/1656985428 shutdown_connections 2024-04-25T14:11:29.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.173+0000 7fea02400640 1 --2- 172.21.15.86:0/1656985428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9fc0740c0 0x7fe9fc1676d0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.173+0000 7fea02400640 1 -- 172.21.15.86:0/1656985428 >> 172.21.15.86:0/1656985428 conn(0x7fe9fc0b4650 msgr2=0x7fe9fc0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:29.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.173+0000 7fea02400640 1 -- 172.21.15.86:0/1656985428 shutdown_connections 2024-04-25T14:11:29.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.173+0000 7fea02400640 1 -- 172.21.15.86:0/1656985428 wait complete. 2024-04-25T14:11:29.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.173+0000 7fea02400640 1 Processor -- start 2024-04-25T14:11:29.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.173+0000 7fea02400640 1 -- start start 2024-04-25T14:11:29.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.177+0000 7fea02400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9fc0740c0 0x7fe9fc09e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:29.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.177+0000 7fea02400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9fc0cfca0 0x7fe9fc147780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:29.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.177+0000 7fea02400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9fc0d16e0 0x7fe9fc0ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:29.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.177+0000 7fea02400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc16ade0 con 0x7fe9fc0d16e0 2024-04-25T14:11:29.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.177+0000 7fea02400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc16ac60 con 0x7fe9fc0cfca0 2024-04-25T14:11:29.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.177+0000 7fea02400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe9fc16af60 con 0x7fe9fc0740c0 2024-04-25T14:11:29.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.177+0000 7fe9fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9fc0d16e0 0x7fe9fc0ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:29.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.177+0000 7fe9fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9fc0d16e0 0x7fe9fc0ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40806/0 (socket says 172.21.15.86:40806) 2024-04-25T14:11:29.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.177+0000 7fe9fbe00640 1 -- 172.21.15.86:0/1527241847 learned_addr learned my addr 172.21.15.86:0/1527241847 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:29.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.177+0000 7fe9f2a00640 1 --2- 172.21.15.86:0/1527241847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9fc0cfca0 0x7fe9fc147780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:29.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.177+0000 7fe9fb400640 1 --2- 172.21.15.86:0/1527241847 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9fc0740c0 0x7fe9fc09e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:29.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.177+0000 7fe9fbe00640 1 -- 172.21.15.86:0/1527241847 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9fc0740c0 msgr2=0x7fe9fc09e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:29.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.177+0000 7fe9fbe00640 1 --2- 172.21.15.86:0/1527241847 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9fc0740c0 0x7fe9fc09e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.177+0000 7fe9fbe00640 1 -- 172.21.15.86:0/1527241847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9fc0cfca0 msgr2=0x7fe9fc147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:29.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.177+0000 7fe9fbe00640 1 --2- 172.21.15.86:0/1527241847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9fc0cfca0 0x7fe9fc147780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.177+0000 7fe9fbe00640 1 -- 172.21.15.86:0/1527241847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9fc0cd400 con 0x7fe9fc0d16e0 2024-04-25T14:11:29.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.177+0000 7fe9fb400640 1 --2- 172.21.15.86:0/1527241847 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9fc0740c0 0x7fe9fc09e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:29.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.177+0000 7fe9fbe00640 1 --2- 172.21.15.86:0/1527241847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9fc0d16e0 0x7fe9fc0ce4a0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fe9f40b0040 tx=0x7fe9f4019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:29.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.177+0000 7fe9f3e00640 1 -- 172.21.15.86:0/1527241847 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f40b2040 con 0x7fe9fc0d16e0 2024-04-25T14:11:29.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.177+0000 7fe9f3e00640 1 -- 172.21.15.86:0/1527241847 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f40bf070 con 0x7fe9fc0d16e0 2024-04-25T14:11:29.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.177+0000 7fe9f3e00640 1 -- 172.21.15.86:0/1527241847 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f40ad920 con 0x7fe9fc0d16e0 2024-04-25T14:11:29.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.177+0000 7fea02400640 1 -- 172.21.15.86:0/1527241847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9fc15ea30 con 0x7fe9fc0d16e0 2024-04-25T14:11:29.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.177+0000 7fea02400640 1 -- 172.21.15.86:0/1527241847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9fc0cd6e0 con 0x7fe9fc0d16e0 2024-04-25T14:11:29.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.177+0000 7fea02400640 1 -- 172.21.15.86:0/1527241847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9fc09f4b0 con 0x7fe9fc0d16e0 2024-04-25T14:11:29.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.181+0000 7fe9f3e00640 1 -- 172.21.15.86:0/1527241847 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7fe9f40adac0 con 0x7fe9fc0d16e0 2024-04-25T14:11:29.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.181+0000 7fe9f3e00640 1 --2- 172.21.15.86:0/1527241847 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fe9d806f520 0x7fe9d8071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:29.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.181+0000 7fe9f3e00640 1 -- 172.21.15.86:0/1527241847 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5249+0+0 (secure 0 0 0) 0x7fe9f412a3f0 con 0x7fe9fc0d16e0 2024-04-25T14:11:29.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.181+0000 7fe9fb400640 1 --2- 172.21.15.86:0/1527241847 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fe9d806f520 0x7fe9d8071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:29.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.181+0000 7fe9fb400640 1 --2- 172.21.15.86:0/1527241847 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fe9d806f520 0x7fe9d8071a30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4000f70 tx=0x7fe9e4009040 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:29.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.185+0000 7fe9f3e00640 1 -- 172.21.15.86:0/1527241847 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe9f4016560 con 0x7fe9fc0d16e0 2024-04-25T14:11:29.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.393+0000 7fea02400640 1 -- 172.21.15.86:0/1527241847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe9fc076bf0 con 0x7fe9fc0d16e0 2024-04-25T14:11:29.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.393+0000 7fe9f3e00640 1 -- 172.21.15.86:0/1527241847 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v28) ==== 80+0+5238 (secure 0 0 0) 0x7fe9f40f7300 con 0x7fe9fc0d16e0 2024-04-25T14:11:29.392 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:29.392 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:11:29.392 INFO:teuthology.orchestra.run.smithi086.stdout: "auth_dump": [ 2024-04-25T14:11:29.392 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:11:29.392 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.a", 2024-04-25T14:11:29.392 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmWk5fAxAAKEBKhWewpl23MUcvMi8qDg==", 2024-04-25T14:11:29.392 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:11:29.393 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:11:29.393 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:11:29.393 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:11:29.393 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:11:29.393 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:11:29.393 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:11:29.393 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:11:29.393 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.b", 2024-04-25T14:11:29.393 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmDhzOEBAABNys/NAEhucQ7A+HzQlzQw==", 2024-04-25T14:11:29.393 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:11:29.393 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:11:29.393 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:11:29.393 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:11:29.394 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:11:29.394 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:11:29.394 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:11:29.394 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:11:29.394 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.c", 2024-04-25T14:11:29.394 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmA/hNCxAAclDGbIjLfjqyXn0MwX/IkA==", 2024-04-25T14:11:29.394 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:11:29.394 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:11:29.394 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:11:29.394 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:11:29.394 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:11:29.394 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:11:29.395 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:11:29.395 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:11:29.395 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.d", 2024-04-25T14:11:29.395 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmemCcGBAAGrCx6C850qRJSWYjebxOBA==", 2024-04-25T14:11:29.395 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:11:29.395 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:11:29.395 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:11:29.395 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:11:29.395 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:11:29.395 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:11:29.395 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:11:29.395 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:11:29.395 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.0", 2024-04-25T14:11:29.395 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCOYipm9etgGxAAemg9CqXN0ShduDtDuKF2Zw==", 2024-04-25T14:11:29.396 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:11:29.396 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:11:29.396 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:11:29.396 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:11:29.396 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:11:29.396 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:11:29.396 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:11:29.396 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.1", 2024-04-25T14:11:29.396 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCRYipmBCiNJhAABAWHk8NEGVvqieL2klGhmA==", 2024-04-25T14:11:29.396 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:11:29.396 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:11:29.396 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:11:29.396 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:11:29.396 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.2", 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCUYipm/XGYLxAAPXsw1svKseFzFiuDUjAsJQ==", 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.3", 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCXYipmlSRaORAAwpKyzZDdVYHWl2pAsQjmpw==", 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:11:29.398 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:11:29.398 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:11:29.398 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:11:29.398 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:11:29.398 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:11:29.398 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.4", 2024-04-25T14:11:29.398 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCdYipmBI3FMBAAV2NdggfRSFnQvrcOC2YqiA==", 2024-04-25T14:11:29.398 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:11:29.398 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:11:29.398 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:11:29.398 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:11:29.398 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:11:29.399 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:11:29.399 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:11:29.399 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.5", 2024-04-25T14:11:29.399 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQChYipmctYxABAABwmOMgyYuOfRWD9KdHILng==", 2024-04-25T14:11:29.399 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:11:29.399 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:11:29.400 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:11:29.400 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:11:29.400 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:11:29.400 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:11:29.400 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:11:29.400 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.6", 2024-04-25T14:11:29.400 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCkYipmSL7TCRAAhFMWg1AQm54bB+x1GDcusA==", 2024-04-25T14:11:29.400 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:11:29.400 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:11:29.400 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:11:29.400 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:11:29.400 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:11:29.401 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:11:29.401 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:11:29.401 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.7", 2024-04-25T14:11:29.401 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCnYipmZ4F+FRAAEGaZN+byv/qJUFZb7WxTFg==", 2024-04-25T14:11:29.401 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:11:29.401 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:11:29.401 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:11:29.401 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:11:29.401 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:11:29.401 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:11:29.401 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:11:29.402 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.0", 2024-04-25T14:11:29.402 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipm5u4xHRAACDFe3Eb4KLrsdLj4eFe7mA==", 2024-04-25T14:11:29.402 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:11:29.402 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:11:29.402 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-25T14:11:29.402 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T14:11:29.403 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:11:29.403 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:11:29.403 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:11:29.403 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.1", 2024-04-25T14:11:29.403 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmDp1hJRAAQkBc5PdQoMTT7W4HahkTyg==", 2024-04-25T14:11:29.404 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:11:29.404 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:11:29.404 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-25T14:11:29.404 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw pool=fs2_data" 2024-04-25T14:11:29.404 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:11:29.404 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:11:29.404 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:11:29.404 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.2", 2024-04-25T14:11:29.404 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmhbhGIRAAxao/J0WY53IITLETFgwIVw==", 2024-04-25T14:11:29.404 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:11:29.405 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:11:29.405 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-25T14:11:29.405 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw pool=fs2_data" 2024-04-25T14:11:29.405 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:11:29.405 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:11:29.405 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:11:29.405 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.3", 2024-04-25T14:11:29.405 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmJsCBKRAAzXn2qDtblIpqt7U/TUUlyA==", 2024-04-25T14:11:29.405 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:11:29.405 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:11:29.405 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-25T14:11:29.405 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw pool=fs2_data" 2024-04-25T14:11:29.406 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:11:29.406 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:11:29.406 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:11:29.406 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.admin", 2024-04-25T14:11:29.406 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCKYipmTqbVIBAA1VpOtj24f2ibVEIT4EJwmw==", 2024-04-25T14:11:29.406 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:11:29.406 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-25T14:11:29.406 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:11:29.406 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:11:29.406 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:11:29.406 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:11:29.407 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:11:29.407 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:11:29.407 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.x", 2024-04-25T14:11:29.407 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCKYipmKMzAORAAtRyTNJ5HWi2XwuamoqwB0Q==", 2024-04-25T14:11:29.407 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:11:29.407 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-25T14:11:29.407 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-25T14:11:29.407 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:11:29.408 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:11:29.408 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:11:29.408 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:11:29.408 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.y", 2024-04-25T14:11:29.408 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCKYipmC7A7NRAAEwVNA0yxt3fwdHXrsV0dvg==", 2024-04-25T14:11:29.408 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:11:29.408 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-25T14:11:29.409 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-25T14:11:29.409 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:11:29.409 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:11:29.409 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:11:29.409 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-25T14:11:29.409 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:11:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.397+0000 7fea02400640 1 -- 172.21.15.86:0/1527241847 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fe9d806f520 msgr2=0x7fe9d8071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.397+0000 7fea02400640 1 --2- 172.21.15.86:0/1527241847 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fe9d806f520 0x7fe9d8071a30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4000f70 tx=0x7fe9e4009040 comp rx=0 tx=0).stop 2024-04-25T14:11:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.397+0000 7fea02400640 1 -- 172.21.15.86:0/1527241847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9fc0d16e0 msgr2=0x7fe9fc0ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.397+0000 7fea02400640 1 --2- 172.21.15.86:0/1527241847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9fc0d16e0 0x7fe9fc0ce4a0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fe9f40b0040 tx=0x7fe9f4019710 comp rx=0 tx=0).stop 2024-04-25T14:11:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.397+0000 7fea02400640 1 -- 172.21.15.86:0/1527241847 shutdown_connections 2024-04-25T14:11:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.397+0000 7fea02400640 1 --2- 172.21.15.86:0/1527241847 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fe9d806f520 0x7fe9d8071a30 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.397+0000 7fea02400640 1 --2- 172.21.15.86:0/1527241847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9fc0d16e0 0x7fe9fc0ce4a0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.397+0000 7fea02400640 1 --2- 172.21.15.86:0/1527241847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9fc0cfca0 0x7fe9fc147780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.397+0000 7fea02400640 1 --2- 172.21.15.86:0/1527241847 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9fc0740c0 0x7fe9fc09e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.397+0000 7fea02400640 1 -- 172.21.15.86:0/1527241847 >> 172.21.15.86:0/1527241847 conn(0x7fe9fc0b4650 msgr2=0x7fe9fc0b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:29.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.397+0000 7fea02400640 1 -- 172.21.15.86:0/1527241847 shutdown_connections 2024-04-25T14:11:29.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.397+0000 7fea02400640 1 -- 172.21.15.86:0/1527241847 wait complete. 2024-04-25T14:11:29.415 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:11:29.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.565+0000 7fb1fba00640 1 Processor -- start 2024-04-25T14:11:29.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.565+0000 7fb1fba00640 1 -- start start 2024-04-25T14:11:29.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.569+0000 7fb1fba00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1f40cfca0 0x7fb1f406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:29.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.569+0000 7fb1fba00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb1f40d16e0 0x7fb1f40d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:29.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.569+0000 7fb1fba00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1f40d2030 con 0x7fb1f406c670 2024-04-25T14:11:29.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.569+0000 7fb1fba00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb1f4070470 con 0x7fb1f40d16e0 2024-04-25T14:11:29.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.569+0000 7fb1fba00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb1f4072bd0 con 0x7fb1f40cfca0 2024-04-25T14:11:29.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.569+0000 7fb1ebe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb1f40d16e0 0x7fb1f40d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:29.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.569+0000 7fb1f9000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1f40cfca0 0x7fb1f406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:29.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.569+0000 7fb1f9000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1f40cfca0 0x7fb1f406f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:42246/0 (socket says 172.21.15.86:42246) 2024-04-25T14:11:29.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.569+0000 7fb1ebe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb1f40d16e0 0x7fb1f40d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:51622/0 (socket says 172.21.15.86:51622) 2024-04-25T14:11:29.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.569+0000 7fb1f9000640 1 -- 172.21.15.86:0/1525689052 learned_addr learned my addr 172.21.15.86:0/1525689052 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:29.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.569+0000 7fb1f9a00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fb1f406c670 0x7fb1f406ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:46122/0 (socket says 172.21.15.86:46122) 2024-04-25T14:11:29.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.569+0000 7fb1f9000640 1 -- 172.21.15.86:0/1525689052 >> v2:172.21.15.155:3300/0 conn(0x7fb1f40d16e0 msgr2=0x7fb1f40d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:29.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.569+0000 7fb1f9000640 1 --2- 172.21.15.86:0/1525689052 >> v2:172.21.15.155:3300/0 conn(0x7fb1f40d16e0 0x7fb1f40d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.569+0000 7fb1f9000640 1 -- 172.21.15.86:0/1525689052 >> v1:172.21.15.155:6789/0 conn(0x7fb1f406c670 legacy=0x7fb1f406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:29.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.569+0000 7fb1f9000640 1 -- 172.21.15.86:0/1525689052 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f406f930 con 0x7fb1f40cfca0 2024-04-25T14:11:29.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.569+0000 7fb1ebe00640 1 --2- 172.21.15.86:0/1525689052 >> v2:172.21.15.155:3300/0 conn(0x7fb1f40d16e0 0x7fb1f40d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:11:29.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.573+0000 7fb1f9000640 1 --2- 172.21.15.86:0/1525689052 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1f40cfca0 0x7fb1f406f3f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4009870 tx=0x7fb1e401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9693d788ae09ce9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:29.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.573+0000 7fb1f3e00640 1 -- 172.21.15.86:0/1525689052 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e40b0070 con 0x7fb1f40cfca0 2024-04-25T14:11:29.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.573+0000 7fb1f3e00640 1 -- 172.21.15.86:0/1525689052 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1e40ab600 con 0x7fb1f40cfca0 2024-04-25T14:11:29.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.573+0000 7fb1f3e00640 1 -- 172.21.15.86:0/1525689052 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e40ab9d0 con 0x7fb1f40cfca0 2024-04-25T14:11:29.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.573+0000 7fb1fba00640 1 -- 172.21.15.86:0/1525689052 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1f40cfca0 msgr2=0x7fb1f406f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:29.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.573+0000 7fb1fba00640 1 --2- 172.21.15.86:0/1525689052 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1f40cfca0 0x7fb1f406f3f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4009870 tx=0x7fb1e401c560 comp rx=0 tx=0).stop 2024-04-25T14:11:29.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.573+0000 7fb1fba00640 1 -- 172.21.15.86:0/1525689052 shutdown_connections 2024-04-25T14:11:29.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.573+0000 7fb1fba00640 1 --2- 172.21.15.86:0/1525689052 >> v2:172.21.15.155:3300/0 conn(0x7fb1f40d16e0 0x7fb1f40d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.573+0000 7fb1fba00640 1 --2- 172.21.15.86:0/1525689052 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1f40cfca0 0x7fb1f406f3f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.573+0000 7fb1fba00640 1 -- 172.21.15.86:0/1525689052 >> 172.21.15.86:0/1525689052 conn(0x7fb1f40b4650 msgr2=0x7fb1f40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:29.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.573+0000 7fb1fba00640 1 -- 172.21.15.86:0/1525689052 shutdown_connections 2024-04-25T14:11:29.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.573+0000 7fb1fba00640 1 -- 172.21.15.86:0/1525689052 wait complete. 2024-04-25T14:11:29.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.577+0000 7fb1fba00640 1 Processor -- start 2024-04-25T14:11:29.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.577+0000 7fb1fba00640 1 -- start start 2024-04-25T14:11:29.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.577+0000 7fb1fba00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1f406c670 0x7fb1f40aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:29.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.577+0000 7fb1fba00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1f40cfca0 0x7fb1f4099480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:29.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.577+0000 7fb1fba00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1f40d16e0 0x7fb1f41f9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:29.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.577+0000 7fb1fba00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f40ce6f0 con 0x7fb1f40cfca0 2024-04-25T14:11:29.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.577+0000 7fb1fba00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f40ce570 con 0x7fb1f40d16e0 2024-04-25T14:11:29.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.577+0000 7fb1fba00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb1f40ce870 con 0x7fb1f406c670 2024-04-25T14:11:29.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.577+0000 7fb1ebe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1f40cfca0 0x7fb1f4099480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:29.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.577+0000 7fb1ebe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1f40cfca0 0x7fb1f4099480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40818/0 (socket says 172.21.15.86:40818) 2024-04-25T14:11:29.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.577+0000 7fb1ebe00640 1 -- 172.21.15.86:0/3090398281 learned_addr learned my addr 172.21.15.86:0/3090398281 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:29.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.577+0000 7fb1f9000640 1 --2- 172.21.15.86:0/3090398281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1f406c670 0x7fb1f40aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:29.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.577+0000 7fb1f9a00640 1 --2- 172.21.15.86:0/3090398281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1f40d16e0 0x7fb1f41f9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:29.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.577+0000 7fb1ebe00640 1 -- 172.21.15.86:0/3090398281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1f406c670 msgr2=0x7fb1f40aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:29.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.577+0000 7fb1ebe00640 1 --2- 172.21.15.86:0/3090398281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1f406c670 0x7fb1f40aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.577+0000 7fb1ebe00640 1 -- 172.21.15.86:0/3090398281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1f40d16e0 msgr2=0x7fb1f41f9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:29.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.577+0000 7fb1ebe00640 1 --2- 172.21.15.86:0/3090398281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1f40d16e0 0x7fb1f41f9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.577+0000 7fb1ebe00640 1 -- 172.21.15.86:0/3090398281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f415fc30 con 0x7fb1f40cfca0 2024-04-25T14:11:29.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.577+0000 7fb1ebe00640 1 --2- 172.21.15.86:0/3090398281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1f40cfca0 0x7fb1f4099480 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc010e30 tx=0x7fb1dc00e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:29.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.577+0000 7fb1f1600640 1 -- 172.21.15.86:0/3090398281 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc01e070 con 0x7fb1f40cfca0 2024-04-25T14:11:29.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.577+0000 7fb1f1600640 1 -- 172.21.15.86:0/3090398281 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1dc011860 con 0x7fb1f40cfca0 2024-04-25T14:11:29.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.577+0000 7fb1f1600640 1 -- 172.21.15.86:0/3090398281 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc011c30 con 0x7fb1f40cfca0 2024-04-25T14:11:29.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.577+0000 7fb1fba00640 1 -- 172.21.15.86:0/3090398281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f415ff10 con 0x7fb1f40cfca0 2024-04-25T14:11:29.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.577+0000 7fb1fba00640 1 -- 172.21.15.86:0/3090398281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f415e220 con 0x7fb1f40cfca0 2024-04-25T14:11:29.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.581+0000 7fb1fba00640 1 -- 172.21.15.86:0/3090398281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1f40af540 con 0x7fb1f40cfca0 2024-04-25T14:11:29.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.581+0000 7fb1f1600640 1 -- 172.21.15.86:0/3090398281 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7fb1dc021070 con 0x7fb1f40cfca0 2024-04-25T14:11:29.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.581+0000 7fb1f1600640 1 --2- 172.21.15.86:0/3090398281 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fb1d006f550 0x7fb1d0071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:29.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.581+0000 7fb1f1600640 1 -- 172.21.15.86:0/3090398281 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5249+0+0 (secure 0 0 0) 0x7fb1dc091c20 con 0x7fb1f40cfca0 2024-04-25T14:11:29.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.581+0000 7fb1f9000640 1 --2- 172.21.15.86:0/3090398281 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fb1d006f550 0x7fb1d0071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:29.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.581+0000 7fb1f9000640 1 --2- 172.21.15.86:0/3090398281 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fb1d006f550 0x7fb1d0071a60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40ad000 tx=0x7fb1e4008040 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:29.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.585+0000 7fb1f1600640 1 -- 172.21.15.86:0/3090398281 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb1dc05e560 con 0x7fb1f40cfca0 2024-04-25T14:11:29.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.745+0000 7fb1fba00640 1 -- 172.21.15.86:0/3090398281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1f409f4b0 con 0x7fb1f40cfca0 2024-04-25T14:11:29.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.749+0000 7fb1f1600640 1 -- 172.21.15.86:0/3090398281 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) ==== 74+0+13174 (secure 0 0 0) 0x7fb1dc062860 con 0x7fb1f40cfca0 2024-04-25T14:11:29.747 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:29.747 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":136,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:11:29.010712+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:11:29.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.749+0000 7fb1fba00640 1 -- 172.21.15.86:0/3090398281 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fb1d006f550 msgr2=0x7fb1d0071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:29.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.749+0000 7fb1fba00640 1 --2- 172.21.15.86:0/3090398281 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fb1d006f550 0x7fb1d0071a60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40ad000 tx=0x7fb1e4008040 comp rx=0 tx=0).stop 2024-04-25T14:11:29.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.749+0000 7fb1fba00640 1 -- 172.21.15.86:0/3090398281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1f40cfca0 msgr2=0x7fb1f4099480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:29.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.749+0000 7fb1fba00640 1 --2- 172.21.15.86:0/3090398281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1f40cfca0 0x7fb1f4099480 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc010e30 tx=0x7fb1dc00e730 comp rx=0 tx=0).stop 2024-04-25T14:11:29.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.749+0000 7fb1fba00640 1 -- 172.21.15.86:0/3090398281 shutdown_connections 2024-04-25T14:11:29.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.749+0000 7fb1fba00640 1 --2- 172.21.15.86:0/3090398281 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fb1d006f550 0x7fb1d0071a60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.749+0000 7fb1fba00640 1 --2- 172.21.15.86:0/3090398281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1f40d16e0 0x7fb1f41f9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.749+0000 7fb1fba00640 1 --2- 172.21.15.86:0/3090398281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1f40cfca0 0x7fb1f4099480 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.749+0000 7fb1fba00640 1 --2- 172.21.15.86:0/3090398281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1f406c670 0x7fb1f40aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.749+0000 7fb1fba00640 1 -- 172.21.15.86:0/3090398281 >> 172.21.15.86:0/3090398281 conn(0x7fb1f40b4650 msgr2=0x7fb1f40ae130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:29.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.749+0000 7fb1fba00640 1 -- 172.21.15.86:0/3090398281 shutdown_connections 2024-04-25T14:11:29.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.749+0000 7fb1fba00640 1 -- 172.21.15.86:0/3090398281 wait complete. 2024-04-25T14:11:29.766 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:11:29.767 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:11:29.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.921+0000 7f02d6200640 1 Processor -- start 2024-04-25T14:11:29.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.921+0000 7f02d6200640 1 -- start start 2024-04-25T14:11:29.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.921+0000 7f02d6200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02d015c2a0 0x7f02d00ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:29.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.921+0000 7f02d6200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f02d015feb0 0x7f02d01602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:29.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.921+0000 7f02d6200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02d00b0a60 con 0x7f02d006fd50 2024-04-25T14:11:29.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.921+0000 7f02d6200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f02d006ef90 con 0x7f02d015feb0 2024-04-25T14:11:29.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.921+0000 7f02d6200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f02d006f270 con 0x7f02d015c2a0 2024-04-25T14:11:29.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.921+0000 7f02cf400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02d015c2a0 0x7f02d00ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:29.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.921+0000 7f02cfe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f02d015feb0 0x7f02d01602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:29.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.921+0000 7f02c6a00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f02d006fd50 0x7f02d0070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:46138/0 (socket says 172.21.15.86:46138) 2024-04-25T14:11:29.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.921+0000 7f02c6a00640 1 -- 172.21.15.86:0/2913642792 learned_addr learned my addr 172.21.15.86:0/2913642792 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:29.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.921+0000 7f02cf400640 1 -- 172.21.15.86:0/2913642792 >> v2:172.21.15.155:3300/0 conn(0x7f02d015feb0 msgr2=0x7f02d01602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:29.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.921+0000 7f02cf400640 1 --2- 172.21.15.86:0/2913642792 >> v2:172.21.15.155:3300/0 conn(0x7f02d015feb0 0x7f02d01602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.921+0000 7f02cf400640 1 -- 172.21.15.86:0/2913642792 >> v1:172.21.15.155:6789/0 conn(0x7f02d006fd50 legacy=0x7f02d0070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:29.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.921+0000 7f02cf400640 1 -- 172.21.15.86:0/2913642792 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d006f550 con 0x7f02d015c2a0 2024-04-25T14:11:29.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.921+0000 7f02cea00640 1 -- 172.21.15.86:0/2913642792 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3703851672 0 0) 0x7f02d00b0a60 con 0x7f02d006fd50 2024-04-25T14:11:29.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.925+0000 7f02cf400640 1 --2- 172.21.15.86:0/2913642792 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02d015c2a0 0x7f02d00ae130 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f02c00097c0 tx=0x7f02c001ce40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dfdb18f1d333967 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:29.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.925+0000 7f02cea00640 1 -- 172.21.15.86:0/2913642792 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c00b2070 con 0x7f02d015c2a0 2024-04-25T14:11:29.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.925+0000 7f02cea00640 1 -- 172.21.15.86:0/2913642792 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02c00ad680 con 0x7f02d015c2a0 2024-04-25T14:11:29.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.925+0000 7f02cea00640 1 -- 172.21.15.86:0/2913642792 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c00ada50 con 0x7f02d015c2a0 2024-04-25T14:11:29.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.925+0000 7f02d6200640 1 -- 172.21.15.86:0/2913642792 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02d015c2a0 msgr2=0x7f02d00ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:29.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.925+0000 7f02d6200640 1 --2- 172.21.15.86:0/2913642792 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02d015c2a0 0x7f02d00ae130 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f02c00097c0 tx=0x7f02c001ce40 comp rx=0 tx=0).stop 2024-04-25T14:11:29.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.925+0000 7f02d6200640 1 -- 172.21.15.86:0/2913642792 shutdown_connections 2024-04-25T14:11:29.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.925+0000 7f02d6200640 1 --2- 172.21.15.86:0/2913642792 >> v2:172.21.15.155:3300/0 conn(0x7f02d015feb0 0x7f02d01602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.925+0000 7f02d6200640 1 --2- 172.21.15.86:0/2913642792 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02d015c2a0 0x7f02d00ae130 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.925+0000 7f02d6200640 1 -- 172.21.15.86:0/2913642792 >> 172.21.15.86:0/2913642792 conn(0x7f02d00b4650 msgr2=0x7f02d00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:29.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.925+0000 7f02d6200640 1 -- 172.21.15.86:0/2913642792 shutdown_connections 2024-04-25T14:11:29.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.925+0000 7f02d6200640 1 -- 172.21.15.86:0/2913642792 wait complete. 2024-04-25T14:11:29.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.929+0000 7f02d6200640 1 Processor -- start 2024-04-25T14:11:29.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.929+0000 7f02d6200640 1 -- start start 2024-04-25T14:11:29.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.929+0000 7f02d6200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02d006fd50 0x7f02d015f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:29.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.929+0000 7f02d6200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d015c2a0 0x7f02d009a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:29.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.929+0000 7f02d6200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02d015feb0 0x7f02d0157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:29.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.929+0000 7f02d6200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02d0160d90 con 0x7f02d015c2a0 2024-04-25T14:11:29.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.929+0000 7f02d6200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f02d0160c10 con 0x7f02d006fd50 2024-04-25T14:11:29.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.929+0000 7f02d6200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f02d00ae670 con 0x7f02d015feb0 2024-04-25T14:11:29.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.929+0000 7f02c6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d015c2a0 0x7f02d009a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:29.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.929+0000 7f02c6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d015c2a0 0x7f02d009a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40828/0 (socket says 172.21.15.86:40828) 2024-04-25T14:11:29.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.929+0000 7f02c6a00640 1 -- 172.21.15.86:0/2789017838 learned_addr learned my addr 172.21.15.86:0/2789017838 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:29.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.929+0000 7f02cf400640 1 --2- 172.21.15.86:0/2789017838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02d006fd50 0x7f02d015f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:29.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.929+0000 7f02cfe00640 1 --2- 172.21.15.86:0/2789017838 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02d015feb0 0x7f02d0157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:29.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.933+0000 7f02c6a00640 1 -- 172.21.15.86:0/2789017838 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02d015feb0 msgr2=0x7f02d0157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:29.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.933+0000 7f02c6a00640 1 --2- 172.21.15.86:0/2789017838 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02d015feb0 0x7f02d0157ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.933+0000 7f02c6a00640 1 -- 172.21.15.86:0/2789017838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02d006fd50 msgr2=0x7f02d015f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:29.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.933+0000 7f02c6a00640 1 --2- 172.21.15.86:0/2789017838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02d006fd50 0x7f02d015f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:29.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.933+0000 7f02c6a00640 1 -- 172.21.15.86:0/2789017838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d015d350 con 0x7f02d015c2a0 2024-04-25T14:11:29.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.933+0000 7f02cfe00640 1 --2- 172.21.15.86:0/2789017838 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02d015feb0 0x7f02d0157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:29.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.933+0000 7f02c6a00640 1 --2- 172.21.15.86:0/2789017838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d015c2a0 0x7f02d009a040 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f02b800c4b0 tx=0x7f02b800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:29.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.933+0000 7f02c7e00640 1 -- 172.21.15.86:0/2789017838 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b8018070 con 0x7f02d015c2a0 2024-04-25T14:11:29.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.933+0000 7f02c7e00640 1 -- 172.21.15.86:0/2789017838 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02b8007500 con 0x7f02d015c2a0 2024-04-25T14:11:29.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.933+0000 7f02c7e00640 1 -- 172.21.15.86:0/2789017838 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b8004a30 con 0x7f02d015c2a0 2024-04-25T14:11:29.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.933+0000 7f02d6200640 1 -- 172.21.15.86:0/2789017838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02d015d630 con 0x7f02d015c2a0 2024-04-25T14:11:29.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.933+0000 7f02d6200640 1 -- 172.21.15.86:0/2789017838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02d015dbf0 con 0x7f02d015c2a0 2024-04-25T14:11:29.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.933+0000 7f02d6200640 1 -- 172.21.15.86:0/2789017838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02d0069d00 con 0x7f02d015c2a0 2024-04-25T14:11:29.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.933+0000 7f02c7e00640 1 -- 172.21.15.86:0/2789017838 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7f02b801b360 con 0x7f02d015c2a0 2024-04-25T14:11:29.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.933+0000 7f02c7e00640 1 --2- 172.21.15.86:0/2789017838 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f02ac06f520 0x7f02ac071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:29.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.937+0000 7f02c7e00640 1 -- 172.21.15.86:0/2789017838 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5249+0+0 (secure 0 0 0) 0x7f02b808e7b0 con 0x7f02d015c2a0 2024-04-25T14:11:29.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.937+0000 7f02cf400640 1 --2- 172.21.15.86:0/2789017838 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f02ac06f520 0x7f02ac071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:29.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.937+0000 7f02cf400640 1 --2- 172.21.15.86:0/2789017838 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f02ac06f520 0x7f02ac071a30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f02d0070bb0 tx=0x7f02c0004740 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:29.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:29.937+0000 7f02c7e00640 1 -- 172.21.15.86:0/2789017838 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f02b805b0e0 con 0x7f02d015c2a0 2024-04-25T14:11:30.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.097+0000 7f02d6200640 1 -- 172.21.15.86:0/2789017838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f02d0076bf0 con 0x7f02d015c2a0 2024-04-25T14:11:30.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.097+0000 7f02c7e00640 1 -- 172.21.15.86:0/2789017838 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v136) ==== 93+0+10 (secure 0 0 0) 0x7f02b805f3e0 con 0x7f02d015c2a0 2024-04-25T14:11:30.096 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:11:30.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.101+0000 7f02d6200640 1 -- 172.21.15.86:0/2789017838 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f02ac06f520 msgr2=0x7f02ac071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:30.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.101+0000 7f02d6200640 1 --2- 172.21.15.86:0/2789017838 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f02ac06f520 0x7f02ac071a30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f02d0070bb0 tx=0x7f02c0004740 comp rx=0 tx=0).stop 2024-04-25T14:11:30.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.101+0000 7f02d6200640 1 -- 172.21.15.86:0/2789017838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d015c2a0 msgr2=0x7f02d009a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:30.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.101+0000 7f02d6200640 1 --2- 172.21.15.86:0/2789017838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d015c2a0 0x7f02d009a040 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f02b800c4b0 tx=0x7f02b800c970 comp rx=0 tx=0).stop 2024-04-25T14:11:30.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.101+0000 7f02d6200640 1 -- 172.21.15.86:0/2789017838 shutdown_connections 2024-04-25T14:11:30.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.101+0000 7f02d6200640 1 --2- 172.21.15.86:0/2789017838 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f02ac06f520 0x7f02ac071a30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.101+0000 7f02d6200640 1 --2- 172.21.15.86:0/2789017838 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02d015feb0 0x7f02d0157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.101+0000 7f02d6200640 1 --2- 172.21.15.86:0/2789017838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d015c2a0 0x7f02d009a040 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.101+0000 7f02d6200640 1 --2- 172.21.15.86:0/2789017838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02d006fd50 0x7f02d015f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.101+0000 7f02d6200640 1 -- 172.21.15.86:0/2789017838 >> 172.21.15.86:0/2789017838 conn(0x7f02d00b4650 msgr2=0x7f02d0074310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:30.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.101+0000 7f02d6200640 1 -- 172.21.15.86:0/2789017838 shutdown_connections 2024-04-25T14:11:30.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.101+0000 7f02d6200640 1 -- 172.21.15.86:0/2789017838 wait complete. 2024-04-25T14:11:30.118 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T14:11:30.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.265+0000 7f2f1be00640 1 Processor -- start 2024-04-25T14:11:30.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.269+0000 7f2f1be00640 1 -- start start 2024-04-25T14:11:30.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.269+0000 7f2f1be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2f1c0b5500 0x7f2f1c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:30.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.269+0000 7f2f1be00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f1c0af700 0x7f2f1c0afb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:30.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.269+0000 7f2f1be00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f1c0b5910 con 0x7f2f1c06d680 2024-04-25T14:11:30.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.269+0000 7f2f1be00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2f1c149a40 con 0x7f2f1c0b5500 2024-04-25T14:11:30.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.269+0000 7f2f1be00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2f1c070ce0 con 0x7f2f1c0af700 2024-04-25T14:11:30.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.269+0000 7f2f1aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2f1c0b5500 0x7f2f1c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:30.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.269+0000 7f2f1aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2f1c0b5500 0x7f2f1c0707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40844/0 (socket says 172.21.15.86:40844) 2024-04-25T14:11:30.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.269+0000 7f2f1aa00640 1 -- 172.21.15.86:0/2183459246 learned_addr learned my addr 172.21.15.86:0/2183459246 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:30.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.269+0000 7f2f1b400640 1 --2- 172.21.15.86:0/2183459246 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f1c0af700 0x7f2f1c0afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:30.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.269+0000 7f2f1aa00640 1 -- 172.21.15.86:0/2183459246 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f1c0af700 msgr2=0x7f2f1c0afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:30.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.269+0000 7f2f1aa00640 1 --2- 172.21.15.86:0/2183459246 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f1c0af700 0x7f2f1c0afb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.269+0000 7f2f1aa00640 1 -- 172.21.15.86:0/2183459246 >> v1:172.21.15.155:6789/0 conn(0x7f2f1c06d680 legacy=0x7f2f1c06da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:30.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.269+0000 7f2f1aa00640 1 -- 172.21.15.86:0/2183459246 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f1c0b5bc0 con 0x7f2f1c0b5500 2024-04-25T14:11:30.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.269+0000 7f2f1b400640 1 --2- 172.21.15.86:0/2183459246 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f1c0af700 0x7f2f1c0afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:30.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.273+0000 7f2f1aa00640 1 --2- 172.21.15.86:0/2183459246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f1c0b5500 0x7f2f1c0707a0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f2f04009870 tx=0x7f2f0401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c9dcf26269f040c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:30.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.273+0000 7f2f1a000640 1 -- 172.21.15.86:0/2183459246 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f040aa610 con 0x7f2f1c0b5500 2024-04-25T14:11:30.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.273+0000 7f2f1a000640 1 -- 172.21.15.86:0/2183459246 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f040af070 con 0x7f2f1c0b5500 2024-04-25T14:11:30.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.273+0000 7f2f1a000640 1 -- 172.21.15.86:0/2183459246 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f040aa9d0 con 0x7f2f1c0b5500 2024-04-25T14:11:30.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.273+0000 7f2f1be00640 1 -- 172.21.15.86:0/2183459246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f1c0b5500 msgr2=0x7f2f1c0707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:30.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.273+0000 7f2f1be00640 1 --2- 172.21.15.86:0/2183459246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f1c0b5500 0x7f2f1c0707a0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f2f04009870 tx=0x7f2f0401c4f0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.273+0000 7f2f1be00640 1 -- 172.21.15.86:0/2183459246 shutdown_connections 2024-04-25T14:11:30.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.273+0000 7f2f1be00640 1 --2- 172.21.15.86:0/2183459246 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f1c0af700 0x7f2f1c0afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.273+0000 7f2f1be00640 1 --2- 172.21.15.86:0/2183459246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f1c0b5500 0x7f2f1c0707a0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.273+0000 7f2f1be00640 1 -- 172.21.15.86:0/2183459246 >> 172.21.15.86:0/2183459246 conn(0x7f2f1c0730f0 msgr2=0x7f2f1c073500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:30.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.273+0000 7f2f1be00640 1 -- 172.21.15.86:0/2183459246 shutdown_connections 2024-04-25T14:11:30.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.273+0000 7f2f1be00640 1 -- 172.21.15.86:0/2183459246 wait complete. 2024-04-25T14:11:30.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.277+0000 7f2f1be00640 1 Processor -- start 2024-04-25T14:11:30.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.277+0000 7f2f1be00640 1 -- start start 2024-04-25T14:11:30.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.277+0000 7f2f1be00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f1c06d680 0x7f2f1c15a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:30.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.277+0000 7f2f1be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f1c0af700 0x7f2f1c0d0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:30.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.277+0000 7f2f1be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f1c0b5500 0x7f2f1c13fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:30.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.277+0000 7f2f1be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1c16c7d0 con 0x7f2f1c0af700 2024-04-25T14:11:30.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.277+0000 7f2f1be00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1c0b6180 con 0x7f2f1c0b5500 2024-04-25T14:11:30.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.277+0000 7f2f1be00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2f1c16c950 con 0x7f2f1c06d680 2024-04-25T14:11:30.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.277+0000 7f2f12000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f1c0af700 0x7f2f1c0d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:30.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.277+0000 7f2f12000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f1c0af700 0x7f2f1c0d0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40846/0 (socket says 172.21.15.86:40846) 2024-04-25T14:11:30.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.277+0000 7f2f12000640 1 -- 172.21.15.86:0/1063175797 learned_addr learned my addr 172.21.15.86:0/1063175797 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:30.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.277+0000 7f2f1aa00640 1 --2- 172.21.15.86:0/1063175797 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f1c06d680 0x7f2f1c15a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:30.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.277+0000 7f2f1b400640 1 --2- 172.21.15.86:0/1063175797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f1c0b5500 0x7f2f1c13fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:30.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.277+0000 7f2f12000640 1 -- 172.21.15.86:0/1063175797 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f1c06d680 msgr2=0x7f2f1c15a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:30.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.277+0000 7f2f12000640 1 --2- 172.21.15.86:0/1063175797 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f1c06d680 0x7f2f1c15a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.277+0000 7f2f12000640 1 -- 172.21.15.86:0/1063175797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f1c0b5500 msgr2=0x7f2f1c13fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:30.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.277+0000 7f2f12000640 1 --2- 172.21.15.86:0/1063175797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f1c0b5500 0x7f2f1c13fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.277+0000 7f2f12000640 1 -- 172.21.15.86:0/1063175797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f1c0ce520 con 0x7f2f1c0af700 2024-04-25T14:11:30.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.277+0000 7f2f12000640 1 --2- 172.21.15.86:0/1063175797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f1c0af700 0x7f2f1c0d0190 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00b410 tx=0x7f2f0c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:30.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.277+0000 7f2f13e00640 1 -- 172.21.15.86:0/1063175797 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c015030 con 0x7f2f1c0af700 2024-04-25T14:11:30.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.277+0000 7f2f13e00640 1 -- 172.21.15.86:0/1063175797 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f0c004020 con 0x7f2f1c0af700 2024-04-25T14:11:30.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.277+0000 7f2f13e00640 1 -- 172.21.15.86:0/1063175797 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c004410 con 0x7f2f1c0af700 2024-04-25T14:11:30.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.277+0000 7f2f1be00640 1 -- 172.21.15.86:0/1063175797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f1c0ce7c0 con 0x7f2f1c0af700 2024-04-25T14:11:30.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.277+0000 7f2f1be00640 1 -- 172.21.15.86:0/1063175797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f1c0ced80 con 0x7f2f1c0af700 2024-04-25T14:11:30.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.281+0000 7f2f1be00640 1 -- 172.21.15.86:0/1063175797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f1c076b80 con 0x7f2f1c0af700 2024-04-25T14:11:30.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.281+0000 7f2f13e00640 1 -- 172.21.15.86:0/1063175797 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7f2f0c018070 con 0x7f2f1c0af700 2024-04-25T14:11:30.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.281+0000 7f2f13e00640 1 --2- 172.21.15.86:0/1063175797 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f2ef806f550 0x7f2ef8071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:30.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.281+0000 7f2f13e00640 1 -- 172.21.15.86:0/1063175797 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5249+0+0 (secure 0 0 0) 0x7f2f0c08ecb0 con 0x7f2f1c0af700 2024-04-25T14:11:30.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.281+0000 7f2f1aa00640 1 --2- 172.21.15.86:0/1063175797 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f2ef806f550 0x7f2ef8071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:30.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.285+0000 7f2f1aa00640 1 --2- 172.21.15.86:0/1063175797 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f2ef806f550 0x7f2ef8071a60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2f040ae000 tx=0x7f2f040a7000 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:30.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.289+0000 7f2f13e00640 1 -- 172.21.15.86:0/1063175797 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2f1c0ce520 con 0x7f2f1c0af700 2024-04-25T14:11:30.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.445+0000 7f2f1be00640 1 -- 172.21.15.86:0/1063175797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2f1c06e1b0 con 0x7f2f1c0af700 2024-04-25T14:11:30.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.445+0000 7f2f13e00640 1 -- 172.21.15.86:0/1063175797 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v136) ==== 84+0+65 (secure 0 0 0) 0x7f2f0c05b600 con 0x7f2f1c0af700 2024-04-25T14:11:30.442 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:30.442 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-04-25T14:11:30.442 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:11:30.442 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-04-25T14:11:30.442 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-04-25T14:11:30.442 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:11:30.442 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-04-25T14:11:30.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.445+0000 7f2f1be00640 1 -- 172.21.15.86:0/1063175797 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f2ef806f550 msgr2=0x7f2ef8071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:30.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.445+0000 7f2f1be00640 1 --2- 172.21.15.86:0/1063175797 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f2ef806f550 0x7f2ef8071a60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2f040ae000 tx=0x7f2f040a7000 comp rx=0 tx=0).stop 2024-04-25T14:11:30.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.445+0000 7f2f1be00640 1 -- 172.21.15.86:0/1063175797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f1c0af700 msgr2=0x7f2f1c0d0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:30.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.445+0000 7f2f1be00640 1 --2- 172.21.15.86:0/1063175797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f1c0af700 0x7f2f1c0d0190 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00b410 tx=0x7f2f0c00b8d0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.445+0000 7f2f1be00640 1 -- 172.21.15.86:0/1063175797 shutdown_connections 2024-04-25T14:11:30.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.445+0000 7f2f1be00640 1 --2- 172.21.15.86:0/1063175797 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f2ef806f550 0x7f2ef8071a60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.445+0000 7f2f1be00640 1 --2- 172.21.15.86:0/1063175797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f1c0b5500 0x7f2f1c13fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.445+0000 7f2f1be00640 1 --2- 172.21.15.86:0/1063175797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f1c0af700 0x7f2f1c0d0190 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.445+0000 7f2f1be00640 1 --2- 172.21.15.86:0/1063175797 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f1c06d680 0x7f2f1c15a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.445+0000 7f2f1be00640 1 -- 172.21.15.86:0/1063175797 >> 172.21.15.86:0/1063175797 conn(0x7f2f1c0730f0 msgr2=0x7f2f1c1486d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:30.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.445+0000 7f2f1be00640 1 -- 172.21.15.86:0/1063175797 shutdown_connections 2024-04-25T14:11:30.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.445+0000 7f2f1be00640 1 -- 172.21.15.86:0/1063175797 wait complete. 2024-04-25T14:11:30.463 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T14:11:30.464 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T14:11:30.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.617+0000 7f9d98e00640 1 Processor -- start 2024-04-25T14:11:30.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.617+0000 7f9d98e00640 1 -- start start 2024-04-25T14:11:30.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d98e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d9406d360 0x7f9d9406d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:30.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d98e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d94149800 con 0x7f9d94072e40 2024-04-25T14:11:30.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d98e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d940b4f60 con 0x7f9d94160d10 2024-04-25T14:11:30.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d98e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9d940b6c10 con 0x7f9d9406d360 2024-04-25T14:11:30.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d91400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9d94072e40 0x7f9d94073250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60420/0 (socket says 172.21.15.86:60420) 2024-04-25T14:11:30.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d91400640 1 -- 172.21.15.86:0/664970213 learned_addr learned my addr 172.21.15.86:0/664970213 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:30.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d92800640 1 --2- 172.21.15.86:0/664970213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d9406d360 0x7f9d9406d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:30.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d90a00640 1 -- 172.21.15.86:0/664970213 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3436705110 0 0) 0x7f9d94149800 con 0x7f9d94072e40 2024-04-25T14:11:30.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d90a00640 1 -- 172.21.15.86:0/664970213 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d78003530 con 0x7f9d94072e40 2024-04-25T14:11:30.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d90a00640 1 -- 172.21.15.86:0/664970213 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2379190486 0 0) 0x7f9d78003530 con 0x7f9d94072e40 2024-04-25T14:11:30.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d90a00640 1 -- 172.21.15.86:0/664970213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d9406d360 msgr2=0x7f9d9406d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:30.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d90a00640 1 --2- 172.21.15.86:0/664970213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d9406d360 0x7f9d9406d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d90a00640 1 -- 172.21.15.86:0/664970213 >> v1:172.21.15.155:6789/0 conn(0x7f9d94160d10 legacy=0x7f9d940b4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:30.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d90a00640 1 -- 172.21.15.86:0/664970213 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9406f010 con 0x7f9d94072e40 2024-04-25T14:11:30.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d90a00640 1 -- 172.21.15.86:0/664970213 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f9d80001680 con 0x7f9d94072e40 2024-04-25T14:11:30.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d90a00640 1 -- 172.21.15.86:0/664970213 >> v1:172.21.15.86:6789/0 conn(0x7f9d94072e40 legacy=0x7f9d94073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:30.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d90a00640 1 --2- 172.21.15.86:0/664970213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d9406d360 0x7f9d78003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:30.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d90a00640 1 --2- 172.21.15.86:0/664970213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d78003f90 0x7f9d78004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:30.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d90a00640 1 --2- 172.21.15.86:0/664970213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d78004d50 0x7f9d78005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:30.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d90a00640 1 -- 172.21.15.86:0/664970213 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7800bb60 con 0x7f9d78003f90 2024-04-25T14:11:30.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d90a00640 1 -- 172.21.15.86:0/664970213 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7800bd10 con 0x7f9d78004d50 2024-04-25T14:11:30.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d90a00640 1 -- 172.21.15.86:0/664970213 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9d7800bf10 con 0x7f9d9406d360 2024-04-25T14:11:30.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d91400640 1 --2- 172.21.15.86:0/664970213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d78003f90 0x7f9d78004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:30.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d91e00640 1 --2- 172.21.15.86:0/664970213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d9406d360 0x7f9d78003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:30.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d92800640 1 --2- 172.21.15.86:0/664970213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d78004d50 0x7f9d78005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:30.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d91400640 1 -- 172.21.15.86:0/664970213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d9406d360 msgr2=0x7f9d78003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:30.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d91400640 1 --2- 172.21.15.86:0/664970213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d9406d360 0x7f9d78003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d91400640 1 -- 172.21.15.86:0/664970213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d78004d50 msgr2=0x7f9d78005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:30.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d91400640 1 --2- 172.21.15.86:0/664970213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d78004d50 0x7f9d78005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.621+0000 7f9d91400640 1 -- 172.21.15.86:0/664970213 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d78003530 con 0x7f9d78003f90 2024-04-25T14:11:30.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.625+0000 7f9d91400640 1 --2- 172.21.15.86:0/664970213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d78003f90 0x7f9d78004810 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000c210 tx=0x7f9d8001f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b9024729a5546e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:30.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.625+0000 7f9d90a00640 1 -- 172.21.15.86:0/664970213 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d80007d20 con 0x7f9d78003f90 2024-04-25T14:11:30.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.625+0000 7f9d90a00640 1 -- 172.21.15.86:0/664970213 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d80005ae0 con 0x7f9d78003f90 2024-04-25T14:11:30.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.625+0000 7f9d90a00640 1 -- 172.21.15.86:0/664970213 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d80007070 con 0x7f9d78003f90 2024-04-25T14:11:30.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.625+0000 7f9d98e00640 1 -- 172.21.15.86:0/664970213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d78003f90 msgr2=0x7f9d78004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:30.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.625+0000 7f9d98e00640 1 --2- 172.21.15.86:0/664970213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d78003f90 0x7f9d78004810 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000c210 tx=0x7f9d8001f3b0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.625+0000 7f9d92800640 1 -- 172.21.15.86:0/664970213 reap_dead start 2024-04-25T14:11:30.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.625+0000 7f9d98e00640 1 -- 172.21.15.86:0/664970213 shutdown_connections 2024-04-25T14:11:30.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.625+0000 7f9d98e00640 1 -- 172.21.15.86:0/664970213 >> 172.21.15.86:0/664970213 conn(0x7f9d940d17e0 msgr2=0x7f9d940d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:30.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.625+0000 7f9d98e00640 1 -- 172.21.15.86:0/664970213 shutdown_connections 2024-04-25T14:11:30.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.625+0000 7f9d98e00640 1 -- 172.21.15.86:0/664970213 wait complete. 2024-04-25T14:11:30.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.629+0000 7f9d98e00640 1 Processor -- start 2024-04-25T14:11:30.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.629+0000 7f9d98e00640 1 -- start start 2024-04-25T14:11:30.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.629+0000 7f9d98e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d78003f90 0x7f9d941f60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:30.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.629+0000 7f9d98e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d940d0590 0x7f9d940c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:30.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.629+0000 7f9d98e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d940d09a0 0x7f9d940c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:30.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.629+0000 7f9d98e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9406cb60 con 0x7f9d940d0590 2024-04-25T14:11:30.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.629+0000 7f9d98e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9406ced0 con 0x7f9d940d09a0 2024-04-25T14:11:30.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.629+0000 7f9d98e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9d94161120 con 0x7f9d78003f90 2024-04-25T14:11:30.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.629+0000 7f9d91400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d940d0590 0x7f9d940c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:30.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.629+0000 7f9d91400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d940d0590 0x7f9d940c67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40868/0 (socket says 172.21.15.86:40868) 2024-04-25T14:11:30.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.629+0000 7f9d91400640 1 -- 172.21.15.86:0/3291564652 learned_addr learned my addr 172.21.15.86:0/3291564652 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:30.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.629+0000 7f9d92800640 1 --2- 172.21.15.86:0/3291564652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d940d09a0 0x7f9d940c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:30.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.629+0000 7f9d91e00640 1 --2- 172.21.15.86:0/3291564652 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d78003f90 0x7f9d941f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:30.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.629+0000 7f9d91400640 1 -- 172.21.15.86:0/3291564652 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d78003f90 msgr2=0x7f9d941f60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:30.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.629+0000 7f9d91400640 1 --2- 172.21.15.86:0/3291564652 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d78003f90 0x7f9d941f60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.629+0000 7f9d91400640 1 -- 172.21.15.86:0/3291564652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d940d09a0 msgr2=0x7f9d940c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:30.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.629+0000 7f9d91400640 1 --2- 172.21.15.86:0/3291564652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d940d09a0 0x7f9d940c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:30.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.629+0000 7f9d91400640 1 -- 172.21.15.86:0/3291564652 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d94138d10 con 0x7f9d940d0590 2024-04-25T14:11:30.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.629+0000 7f9d91e00640 1 --2- 172.21.15.86:0/3291564652 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d78003f90 0x7f9d941f60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:30.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.629+0000 7f9d92800640 1 --2- 172.21.15.86:0/3291564652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d940d09a0 0x7f9d940c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:30.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.629+0000 7f9d91400640 1 --2- 172.21.15.86:0/3291564652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d940d0590 0x7f9d940c67c0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000c210 tx=0x7f9d78005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:30.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.633+0000 7f9d8a000640 1 -- 172.21.15.86:0/3291564652 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d80007d20 con 0x7f9d940d0590 2024-04-25T14:11:30.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.633+0000 7f9d8a000640 1 -- 172.21.15.86:0/3291564652 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d800047f0 con 0x7f9d940d0590 2024-04-25T14:11:30.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.633+0000 7f9d98e00640 1 -- 172.21.15.86:0/3291564652 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d940d1170 con 0x7f9d940d0590 2024-04-25T14:11:30.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.633+0000 7f9d8a000640 1 -- 172.21.15.86:0/3291564652 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d80004bc0 con 0x7f9d940d0590 2024-04-25T14:11:30.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.633+0000 7f9d98e00640 1 -- 172.21.15.86:0/3291564652 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d941eac60 con 0x7f9d940d0590 2024-04-25T14:11:30.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.633+0000 7f9d98e00640 1 -- 172.21.15.86:0/3291564652 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d941eaa50 con 0x7f9d940d0590 2024-04-25T14:11:30.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.633+0000 7f9d8a000640 1 -- 172.21.15.86:0/3291564652 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7f9d80007070 con 0x7f9d940d0590 2024-04-25T14:11:30.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.633+0000 7f9d8a000640 1 --2- 172.21.15.86:0/3291564652 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f9d7406f520 0x7f9d74071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:30.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.633+0000 7f9d8a000640 1 -- 172.21.15.86:0/3291564652 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5249+0+0 (secure 0 0 0) 0x7f9d8012d140 con 0x7f9d940d0590 2024-04-25T14:11:30.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.633+0000 7f9d91e00640 1 --2- 172.21.15.86:0/3291564652 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f9d7406f520 0x7f9d74071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:30.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.637+0000 7f9d91e00640 1 --2- 172.21.15.86:0/3291564652 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f9d7406f520 0x7f9d74071a30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9d84000f90 tx=0x7f9d84009040 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:30.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.637+0000 7f9d8a000640 1 -- 172.21.15.86:0/3291564652 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9d800ca020 con 0x7f9d940d0590 2024-04-25T14:11:30.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:30.797+0000 7f9d98e00640 1 -- 172.21.15.86:0/3291564652 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f9d940d0f00 con 0x7f9d940d0590 2024-04-25T14:11:31.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.565+0000 7f9d8a000640 1 -- 172.21.15.86:0/3291564652 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v137) ==== 138+0+0 (secure 0 0 0) 0x7f9d800b7070 con 0x7f9d940d0590 2024-04-25T14:11:31.561 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-04-25T14:11:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.565+0000 7f9d98e00640 1 -- 172.21.15.86:0/3291564652 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f9d7406f520 msgr2=0x7f9d74071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.565+0000 7f9d98e00640 1 --2- 172.21.15.86:0/3291564652 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f9d7406f520 0x7f9d74071a30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9d84000f90 tx=0x7f9d84009040 comp rx=0 tx=0).stop 2024-04-25T14:11:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.565+0000 7f9d98e00640 1 -- 172.21.15.86:0/3291564652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d940d0590 msgr2=0x7f9d940c67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.565+0000 7f9d98e00640 1 --2- 172.21.15.86:0/3291564652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d940d0590 0x7f9d940c67c0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000c210 tx=0x7f9d78005840 comp rx=0 tx=0).stop 2024-04-25T14:11:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.565+0000 7f9d98e00640 1 -- 172.21.15.86:0/3291564652 shutdown_connections 2024-04-25T14:11:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.565+0000 7f9d98e00640 1 --2- 172.21.15.86:0/3291564652 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f9d7406f520 0x7f9d74071a30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.565+0000 7f9d98e00640 1 --2- 172.21.15.86:0/3291564652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d940d09a0 0x7f9d940c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:31.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.565+0000 7f9d98e00640 1 --2- 172.21.15.86:0/3291564652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d940d0590 0x7f9d940c67c0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:31.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.565+0000 7f9d98e00640 1 --2- 172.21.15.86:0/3291564652 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d78003f90 0x7f9d941f60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:31.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.565+0000 7f9d98e00640 1 -- 172.21.15.86:0/3291564652 >> 172.21.15.86:0/3291564652 conn(0x7f9d940d17e0 msgr2=0x7f9d94160300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:31.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.565+0000 7f9d98e00640 1 -- 172.21.15.86:0/3291564652 shutdown_connections 2024-04-25T14:11:31.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.565+0000 7f9d98e00640 1 -- 172.21.15.86:0/3291564652 wait complete. 2024-04-25T14:11:31.593 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T14:11:31.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.753+0000 7f109a400640 1 Processor -- start 2024-04-25T14:11:31.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.753+0000 7f109a400640 1 -- start start 2024-04-25T14:11:31.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.753+0000 7f109a400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f10940d1df0 0x7f10940cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:31.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.753+0000 7f109a400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f10940cdcc0 0x7f10940ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:31.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.753+0000 7f109a400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f109415e5c0 con 0x7f1094070330 2024-04-25T14:11:31.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.753+0000 7f109a400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f109406fd40 con 0x7f10940d1df0 2024-04-25T14:11:31.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.753+0000 7f109a400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f109415e8a0 con 0x7f10940cdcc0 2024-04-25T14:11:31.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.753+0000 7f108aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f10940d1df0 0x7f10940cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:31.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.753+0000 7f108aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f10940d1df0 0x7f10940cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40872/0 (socket says 172.21.15.86:40872) 2024-04-25T14:11:31.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.753+0000 7f108aa00640 1 -- 172.21.15.86:0/863972345 learned_addr learned my addr 172.21.15.86:0/863972345 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:31.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.757+0000 7f1093e00640 1 --2- 172.21.15.86:0/863972345 >> v2:172.21.15.155:3300/0 conn(0x7f10940cdcc0 0x7f10940ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:31.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.757+0000 7f108aa00640 1 -- 172.21.15.86:0/863972345 >> v2:172.21.15.155:3300/0 conn(0x7f10940cdcc0 msgr2=0x7f10940ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:31.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.757+0000 7f108aa00640 1 --2- 172.21.15.86:0/863972345 >> v2:172.21.15.155:3300/0 conn(0x7f10940cdcc0 0x7f10940ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:31.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.757+0000 7f108aa00640 1 -- 172.21.15.86:0/863972345 >> v1:172.21.15.155:6789/0 conn(0x7f1094070330 legacy=0x7f10940d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:31.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.757+0000 7f108aa00640 1 -- 172.21.15.86:0/863972345 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f109415eb80 con 0x7f10940d1df0 2024-04-25T14:11:31.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.757+0000 7f108aa00640 1 --2- 172.21.15.86:0/863972345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10940d1df0 0x7f10940cd780 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f1084009870 tx=0x7f108401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=993f70b443055a6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:31.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.757+0000 7f1092a00640 1 -- 172.21.15.86:0/863972345 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10840ab640 con 0x7f10940d1df0 2024-04-25T14:11:31.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.757+0000 7f1092a00640 1 -- 172.21.15.86:0/863972345 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10840b0070 con 0x7f10940d1df0 2024-04-25T14:11:31.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.757+0000 7f1092a00640 1 -- 172.21.15.86:0/863972345 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10840aba00 con 0x7f10940d1df0 2024-04-25T14:11:31.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.761+0000 7f109a400640 1 -- 172.21.15.86:0/863972345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10940d1df0 msgr2=0x7f10940cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:31.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.761+0000 7f109a400640 1 --2- 172.21.15.86:0/863972345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10940d1df0 0x7f10940cd780 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f1084009870 tx=0x7f108401c460 comp rx=0 tx=0).stop 2024-04-25T14:11:31.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.761+0000 7f109a400640 1 -- 172.21.15.86:0/863972345 shutdown_connections 2024-04-25T14:11:31.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.761+0000 7f109a400640 1 --2- 172.21.15.86:0/863972345 >> v2:172.21.15.155:3300/0 conn(0x7f10940cdcc0 0x7f10940ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:31.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.761+0000 7f109a400640 1 --2- 172.21.15.86:0/863972345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10940d1df0 0x7f10940cd780 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:31.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.761+0000 7f109a400640 1 -- 172.21.15.86:0/863972345 >> 172.21.15.86:0/863972345 conn(0x7f10940b4650 msgr2=0x7f10940b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:31.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.761+0000 7f109a400640 1 -- 172.21.15.86:0/863972345 shutdown_connections 2024-04-25T14:11:31.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.761+0000 7f109a400640 1 -- 172.21.15.86:0/863972345 wait complete. 2024-04-25T14:11:31.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.761+0000 7f109a400640 1 Processor -- start 2024-04-25T14:11:31.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.761+0000 7f109a400640 1 -- start start 2024-04-25T14:11:31.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.765+0000 7f109a400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1094070330 0x7f10940c16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:31.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.765+0000 7f109a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10940cdcc0 0x7f1094159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:31.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.765+0000 7f109a400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10940c3c90 0x7f1094072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:31.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.765+0000 7f109a400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f109415f4b0 con 0x7f10940cdcc0 2024-04-25T14:11:31.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.765+0000 7f109a400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f109415f330 con 0x7f10940c3c90 2024-04-25T14:11:31.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.765+0000 7f109a400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1094070740 con 0x7f1094070330 2024-04-25T14:11:31.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.765+0000 7f108aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10940cdcc0 0x7f1094159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:31.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.765+0000 7f108aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10940cdcc0 0x7f1094159e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40874/0 (socket says 172.21.15.86:40874) 2024-04-25T14:11:31.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.765+0000 7f108aa00640 1 -- 172.21.15.86:0/1288732341 learned_addr learned my addr 172.21.15.86:0/1288732341 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:31.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.765+0000 7f1093e00640 1 --2- 172.21.15.86:0/1288732341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10940c3c90 0x7f1094072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:31.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.765+0000 7f1093400640 1 --2- 172.21.15.86:0/1288732341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1094070330 0x7f10940c16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:31.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.765+0000 7f108aa00640 1 -- 172.21.15.86:0/1288732341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1094070330 msgr2=0x7f10940c16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:31.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.765+0000 7f108aa00640 1 --2- 172.21.15.86:0/1288732341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1094070330 0x7f10940c16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:31.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.765+0000 7f108aa00640 1 -- 172.21.15.86:0/1288732341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10940c3c90 msgr2=0x7f1094072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:31.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.765+0000 7f108aa00640 1 --2- 172.21.15.86:0/1288732341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10940c3c90 0x7f1094072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:31.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.765+0000 7f108aa00640 1 -- 172.21.15.86:0/1288732341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10940c1ef0 con 0x7f10940cdcc0 2024-04-25T14:11:31.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.765+0000 7f1093400640 1 --2- 172.21.15.86:0/1288732341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1094070330 0x7f10940c16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:31.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.765+0000 7f108aa00640 1 --2- 172.21.15.86:0/1288732341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10940cdcc0 0x7f1094159e30 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f10840ad040 tx=0x7f1084019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:31.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.765+0000 7f108be00640 1 -- 172.21.15.86:0/1288732341 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10840bf410 con 0x7f10940cdcc0 2024-04-25T14:11:31.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.765+0000 7f108be00640 1 -- 172.21.15.86:0/1288732341 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10840bf5d0 con 0x7f10940cdcc0 2024-04-25T14:11:31.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.765+0000 7f108be00640 1 -- 172.21.15.86:0/1288732341 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10840bf9c0 con 0x7f10940cdcc0 2024-04-25T14:11:31.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.765+0000 7f109a400640 1 -- 172.21.15.86:0/1288732341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f109415b750 con 0x7f10940cdcc0 2024-04-25T14:11:31.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.765+0000 7f109a400640 1 -- 172.21.15.86:0/1288732341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10940af8b0 con 0x7f10940cdcc0 2024-04-25T14:11:31.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.765+0000 7f109a400640 1 -- 172.21.15.86:0/1288732341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10940cd040 con 0x7f10940cdcc0 2024-04-25T14:11:31.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.769+0000 7f108be00640 1 -- 172.21.15.86:0/1288732341 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7f10840bd020 con 0x7f10940cdcc0 2024-04-25T14:11:31.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.769+0000 7f108be00640 1 --2- 172.21.15.86:0/1288732341 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f107006f520 0x7f1070071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:31.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.769+0000 7f108be00640 1 -- 172.21.15.86:0/1288732341 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5619+0+0 (secure 0 0 0) 0x7f1084129460 con 0x7f10940cdcc0 2024-04-25T14:11:31.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.769+0000 7f1093400640 1 --2- 172.21.15.86:0/1288732341 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f107006f520 0x7f1070071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:31.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.769+0000 7f1093400640 1 --2- 172.21.15.86:0/1288732341 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f107006f520 0x7f1070071a30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1094072200 tx=0x7f107c009040 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:31.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.773+0000 7f108be00640 1 -- 172.21.15.86:0/1288732341 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f10840c6070 con 0x7f10940cdcc0 2024-04-25T14:11:31.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:31.929+0000 7f109a400640 1 -- 172.21.15.86:0/1288732341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1094069d00 con 0x7f10940cdcc0 2024-04-25T14:11:32.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.573+0000 7f108be00640 1 -- 172.21.15.86:0/1288732341 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v138) ==== 170+0+0 (secure 0 0 0) 0x7f1084016110 con 0x7f10940cdcc0 2024-04-25T14:11:32.570 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-04-25T14:11:32.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.577+0000 7f109a400640 1 -- 172.21.15.86:0/1288732341 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f107006f520 msgr2=0x7f1070071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:32.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.577+0000 7f109a400640 1 --2- 172.21.15.86:0/1288732341 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f107006f520 0x7f1070071a30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1094072200 tx=0x7f107c009040 comp rx=0 tx=0).stop 2024-04-25T14:11:32.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.577+0000 7f109a400640 1 -- 172.21.15.86:0/1288732341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10940cdcc0 msgr2=0x7f1094159e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:32.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.577+0000 7f109a400640 1 --2- 172.21.15.86:0/1288732341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10940cdcc0 0x7f1094159e30 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f10840ad040 tx=0x7f1084019200 comp rx=0 tx=0).stop 2024-04-25T14:11:32.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.577+0000 7f109a400640 1 -- 172.21.15.86:0/1288732341 shutdown_connections 2024-04-25T14:11:32.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.577+0000 7f109a400640 1 --2- 172.21.15.86:0/1288732341 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f107006f520 0x7f1070071a30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:32.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.577+0000 7f109a400640 1 --2- 172.21.15.86:0/1288732341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10940c3c90 0x7f1094072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:32.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.577+0000 7f109a400640 1 --2- 172.21.15.86:0/1288732341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10940cdcc0 0x7f1094159e30 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:32.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.577+0000 7f109a400640 1 --2- 172.21.15.86:0/1288732341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1094070330 0x7f10940c16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:32.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.577+0000 7f109a400640 1 -- 172.21.15.86:0/1288732341 >> 172.21.15.86:0/1288732341 conn(0x7f10940b4650 msgr2=0x7f10940b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:32.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.577+0000 7f109a400640 1 -- 172.21.15.86:0/1288732341 shutdown_connections 2024-04-25T14:11:32.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.577+0000 7f109a400640 1 -- 172.21.15.86:0/1288732341 wait complete. 2024-04-25T14:11:32.611 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T14:11:32.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.765+0000 7f9601e00640 1 Processor -- start 2024-04-25T14:11:32.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.765+0000 7f9601e00640 1 -- start start 2024-04-25T14:11:32.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.769+0000 7f9601e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f95fc15cc90 0x7f95fc160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:32.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.769+0000 7f9601e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95fc0af8b0 0x7f95fc0afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:32.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.769+0000 7f9601e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95fc0b3d60 con 0x7f95fc0b3950 2024-04-25T14:11:32.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.769+0000 7f9601e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f95fc0af140 con 0x7f95fc15cc90 2024-04-25T14:11:32.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.769+0000 7f9601e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f95fc0af420 con 0x7f95fc0af8b0 2024-04-25T14:11:32.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.769+0000 7f95faa00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f95fc0b3950 0x7f95fc1607a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60426/0 (socket says 172.21.15.86:60426) 2024-04-25T14:11:32.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.769+0000 7f95faa00640 1 -- 172.21.15.86:0/2703631094 learned_addr learned my addr 172.21.15.86:0/2703631094 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:32.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.769+0000 7f95f2000640 1 --2- 172.21.15.86:0/2703631094 >> v2:172.21.15.155:3300/0 conn(0x7f95fc15cc90 0x7f95fc160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:32.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.769+0000 7f95fb400640 1 --2- 172.21.15.86:0/2703631094 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95fc0af8b0 0x7f95fc0afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:32.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.769+0000 7f95fa000640 1 -- 172.21.15.86:0/2703631094 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1330549247 0 0) 0x7f95fc0b3d60 con 0x7f95fc0b3950 2024-04-25T14:11:32.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.769+0000 7f95fa000640 1 -- 172.21.15.86:0/2703631094 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95e0003530 con 0x7f95fc0b3950 2024-04-25T14:11:32.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.769+0000 7f95f2000640 1 -- 172.21.15.86:0/2703631094 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95fc0af8b0 msgr2=0x7f95fc0afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:32.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.769+0000 7f95f2000640 1 --2- 172.21.15.86:0/2703631094 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95fc0af8b0 0x7f95fc0afcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:32.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.769+0000 7f95f2000640 1 -- 172.21.15.86:0/2703631094 >> v1:172.21.15.86:6789/0 conn(0x7f95fc0b3950 legacy=0x7f95fc1607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:32.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.769+0000 7f95f2000640 1 -- 172.21.15.86:0/2703631094 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95fc0b1d50 con 0x7f95fc15cc90 2024-04-25T14:11:32.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.773+0000 7f95f2000640 1 --2- 172.21.15.86:0/2703631094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95fc15cc90 0x7f95fc160eb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f95e40097c0 tx=0x7f95e401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d45671659d1a5dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:32.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.773+0000 7f95fa000640 1 -- 172.21.15.86:0/2703631094 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e40b0070 con 0x7f95fc15cc90 2024-04-25T14:11:32.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.773+0000 7f95fa000640 1 -- 172.21.15.86:0/2703631094 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95e40ab600 con 0x7f95fc15cc90 2024-04-25T14:11:32.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.773+0000 7f95fa000640 1 -- 172.21.15.86:0/2703631094 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e40ab9b0 con 0x7f95fc15cc90 2024-04-25T14:11:32.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.773+0000 7f9601e00640 1 -- 172.21.15.86:0/2703631094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95fc15cc90 msgr2=0x7f95fc160eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:32.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.773+0000 7f9601e00640 1 --2- 172.21.15.86:0/2703631094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95fc15cc90 0x7f95fc160eb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f95e40097c0 tx=0x7f95e401c410 comp rx=0 tx=0).stop 2024-04-25T14:11:32.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.773+0000 7f9601e00640 1 -- 172.21.15.86:0/2703631094 shutdown_connections 2024-04-25T14:11:32.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.773+0000 7f9601e00640 1 --2- 172.21.15.86:0/2703631094 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95fc0af8b0 0x7f95fc0afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:32.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.773+0000 7f9601e00640 1 --2- 172.21.15.86:0/2703631094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95fc15cc90 0x7f95fc160eb0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:32.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.773+0000 7f9601e00640 1 -- 172.21.15.86:0/2703631094 >> 172.21.15.86:0/2703631094 conn(0x7f95fc09ba30 msgr2=0x7f95fc0b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:32.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.773+0000 7f9601e00640 1 -- 172.21.15.86:0/2703631094 shutdown_connections 2024-04-25T14:11:32.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.773+0000 7f9601e00640 1 -- 172.21.15.86:0/2703631094 wait complete. 2024-04-25T14:11:32.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.777+0000 7f9601e00640 1 Processor -- start 2024-04-25T14:11:32.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.777+0000 7f9601e00640 1 -- start start 2024-04-25T14:11:32.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.777+0000 7f9601e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95fc0af8b0 0x7f95fc15e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:32.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.777+0000 7f9601e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95fc0b3950 0x7f95fc14c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:32.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.777+0000 7f9601e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95fc15cc90 0x7f95fc159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:32.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.777+0000 7f9601e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc169c60 con 0x7f95fc0af8b0 2024-04-25T14:11:32.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.777+0000 7f9601e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc169ae0 con 0x7f95fc15cc90 2024-04-25T14:11:32.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.777+0000 7f9601e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f95fc169de0 con 0x7f95fc0b3950 2024-04-25T14:11:32.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.777+0000 7f95faa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95fc0af8b0 0x7f95fc15e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:32.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.777+0000 7f95faa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95fc0af8b0 0x7f95fc15e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40884/0 (socket says 172.21.15.86:40884) 2024-04-25T14:11:32.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.777+0000 7f95faa00640 1 -- 172.21.15.86:0/3412658427 learned_addr learned my addr 172.21.15.86:0/3412658427 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:32.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.777+0000 7f95faa00640 1 -- 172.21.15.86:0/3412658427 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95fc0b3950 msgr2=0x7f95fc14c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:32.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.777+0000 7f95faa00640 1 --2- 172.21.15.86:0/3412658427 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95fc0b3950 0x7f95fc14c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:32.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.777+0000 7f95faa00640 1 -- 172.21.15.86:0/3412658427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95fc15cc90 msgr2=0x7f95fc159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:32.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.777+0000 7f95faa00640 1 --2- 172.21.15.86:0/3412658427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95fc15cc90 0x7f95fc159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:32.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.777+0000 7f95faa00640 1 -- 172.21.15.86:0/3412658427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95fc15fd20 con 0x7f95fc0af8b0 2024-04-25T14:11:32.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.777+0000 7f95faa00640 1 --2- 172.21.15.86:0/3412658427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95fc0af8b0 0x7f95fc15e790 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00c370 tx=0x7f95ec00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:32.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.777+0000 7f95f3e00640 1 -- 172.21.15.86:0/3412658427 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec017070 con 0x7f95fc0af8b0 2024-04-25T14:11:32.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.777+0000 7f9601e00640 1 -- 172.21.15.86:0/3412658427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95fc15dcb0 con 0x7f95fc0af8b0 2024-04-25T14:11:32.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.777+0000 7f95f3e00640 1 -- 172.21.15.86:0/3412658427 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95ec0074e0 con 0x7f95fc0af8b0 2024-04-25T14:11:32.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.777+0000 7f95f3e00640 1 -- 172.21.15.86:0/3412658427 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec0059f0 con 0x7f95fc0af8b0 2024-04-25T14:11:32.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.777+0000 7f9601e00640 1 -- 172.21.15.86:0/3412658427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95fc160000 con 0x7f95fc0af8b0 2024-04-25T14:11:32.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.781+0000 7f9601e00640 1 -- 172.21.15.86:0/3412658427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95fc0a04c0 con 0x7f95fc0af8b0 2024-04-25T14:11:32.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.781+0000 7f95f3e00640 1 -- 172.21.15.86:0/3412658427 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7f95ec01a070 con 0x7f95fc0af8b0 2024-04-25T14:11:32.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.781+0000 7f95f3e00640 1 --2- 172.21.15.86:0/3412658427 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f95d806f550 0x7f95d8071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:32.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.781+0000 7f95f3e00640 1 -- 172.21.15.86:0/3412658427 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 6001+0+0 (secure 0 0 0) 0x7f95ec08e990 con 0x7f95fc0af8b0 2024-04-25T14:11:32.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.781+0000 7f95f2000640 1 --2- 172.21.15.86:0/3412658427 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f95d806f550 0x7f95d8071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:32.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.781+0000 7f95f2000640 1 --2- 172.21.15.86:0/3412658427 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f95d806f550 0x7f95d8071a60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f95e40ad000 tx=0x7f95e4004740 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:32.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.785+0000 7f95f3e00640 1 -- 172.21.15.86:0/3412658427 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f95ec05b5c0 con 0x7f95fc0af8b0 2024-04-25T14:11:32.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:32.977+0000 7f9601e00640 1 -- 172.21.15.86:0/3412658427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f95fc076bf0 con 0x7f95fc0af8b0 2024-04-25T14:11:33.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.577+0000 7f95f3e00640 1 -- 172.21.15.86:0/3412658427 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v99) ==== 174+0+0 (secure 0 0 0) 0x7f95ec08e070 con 0x7f95fc0af8b0 2024-04-25T14:11:33.573 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 21 and data pool 22 2024-04-25T14:11:33.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.577+0000 7f9601e00640 1 -- 172.21.15.86:0/3412658427 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f95d806f550 msgr2=0x7f95d8071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:33.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.577+0000 7f9601e00640 1 --2- 172.21.15.86:0/3412658427 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f95d806f550 0x7f95d8071a60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f95e40ad000 tx=0x7f95e4004740 comp rx=0 tx=0).stop 2024-04-25T14:11:33.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.577+0000 7f9601e00640 1 -- 172.21.15.86:0/3412658427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95fc0af8b0 msgr2=0x7f95fc15e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:33.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.577+0000 7f9601e00640 1 --2- 172.21.15.86:0/3412658427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95fc0af8b0 0x7f95fc15e790 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00c370 tx=0x7f95ec00c830 comp rx=0 tx=0).stop 2024-04-25T14:11:33.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.577+0000 7f9601e00640 1 -- 172.21.15.86:0/3412658427 shutdown_connections 2024-04-25T14:11:33.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.577+0000 7f9601e00640 1 --2- 172.21.15.86:0/3412658427 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f95d806f550 0x7f95d8071a60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:33.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.581+0000 7f9601e00640 1 --2- 172.21.15.86:0/3412658427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95fc15cc90 0x7f95fc159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:33.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.581+0000 7f9601e00640 1 --2- 172.21.15.86:0/3412658427 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95fc0b3950 0x7f95fc14c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:33.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.581+0000 7f9601e00640 1 --2- 172.21.15.86:0/3412658427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95fc0af8b0 0x7f95fc15e790 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:33.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.581+0000 7f9601e00640 1 -- 172.21.15.86:0/3412658427 >> 172.21.15.86:0/3412658427 conn(0x7f95fc09ba30 msgr2=0x7f95fc1587c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:33.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.581+0000 7f9601e00640 1 -- 172.21.15.86:0/3412658427 shutdown_connections 2024-04-25T14:11:33.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.581+0000 7f9601e00640 1 -- 172.21.15.86:0/3412658427 wait complete. 2024-04-25T14:11:33.627 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:11:33.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.777+0000 7f3f48000640 1 Processor -- start 2024-04-25T14:11:33.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.777+0000 7f3f48000640 1 -- start start 2024-04-25T14:11:33.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.781+0000 7f3f48000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f400d1df0 0x7f3f400cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:33.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.781+0000 7f3f48000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3f400cdcc0 0x7f3f400ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:33.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.781+0000 7f3f48000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f4015e5c0 con 0x7f3f40070330 2024-04-25T14:11:33.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.781+0000 7f3f48000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3f4006fd40 con 0x7f3f400cdcc0 2024-04-25T14:11:33.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.781+0000 7f3f48000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3f4015e8a0 con 0x7f3f400d1df0 2024-04-25T14:11:33.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.781+0000 7f3f46000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3f400cdcc0 0x7f3f400ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:33.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.781+0000 7f3f44c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f400d1df0 0x7f3f400cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:33.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.781+0000 7f3f44c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f400d1df0 0x7f3f400cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:42344/0 (socket says 172.21.15.86:42344) 2024-04-25T14:11:33.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.781+0000 7f3f44c00640 1 -- 172.21.15.86:0/2685658157 learned_addr learned my addr 172.21.15.86:0/2685658157 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:33.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.781+0000 7f3f45600640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f3f40070330 0x7f3f400d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:46176/0 (socket says 172.21.15.86:46176) 2024-04-25T14:11:33.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.781+0000 7f3f44c00640 1 -- 172.21.15.86:0/2685658157 >> v2:172.21.15.155:3300/0 conn(0x7f3f400cdcc0 msgr2=0x7f3f400ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:33.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.781+0000 7f3f44c00640 1 --2- 172.21.15.86:0/2685658157 >> v2:172.21.15.155:3300/0 conn(0x7f3f400cdcc0 0x7f3f400ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:33.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.781+0000 7f3f44c00640 1 -- 172.21.15.86:0/2685658157 >> v1:172.21.15.155:6789/0 conn(0x7f3f40070330 legacy=0x7f3f400d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:33.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.781+0000 7f3f44c00640 1 -- 172.21.15.86:0/2685658157 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4015eb80 con 0x7f3f400d1df0 2024-04-25T14:11:33.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.781+0000 7f3f46000640 1 --2- 172.21.15.86:0/2685658157 >> v2:172.21.15.155:3300/0 conn(0x7f3f400cdcc0 0x7f3f400ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:11:33.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.785+0000 7f3f44c00640 1 --2- 172.21.15.86:0/2685658157 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f400d1df0 0x7f3f400cd780 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c009870 tx=0x7f3f2c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8ac4e396dd8c354 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:33.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.785+0000 7f3f3fe00640 1 -- 172.21.15.86:0/2685658157 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c0ab680 con 0x7f3f400d1df0 2024-04-25T14:11:33.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.785+0000 7f3f3fe00640 1 -- 172.21.15.86:0/2685658157 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f2c0b0070 con 0x7f3f400d1df0 2024-04-25T14:11:33.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.785+0000 7f3f3fe00640 1 -- 172.21.15.86:0/2685658157 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c0aba40 con 0x7f3f400d1df0 2024-04-25T14:11:33.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.785+0000 7f3f48000640 1 -- 172.21.15.86:0/2685658157 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f400d1df0 msgr2=0x7f3f400cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:33.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.785+0000 7f3f48000640 1 --2- 172.21.15.86:0/2685658157 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f400d1df0 0x7f3f400cd780 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c009870 tx=0x7f3f2c01c4a0 comp rx=0 tx=0).stop 2024-04-25T14:11:33.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.785+0000 7f3f48000640 1 -- 172.21.15.86:0/2685658157 shutdown_connections 2024-04-25T14:11:33.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.785+0000 7f3f48000640 1 --2- 172.21.15.86:0/2685658157 >> v2:172.21.15.155:3300/0 conn(0x7f3f400cdcc0 0x7f3f400ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:33.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.785+0000 7f3f48000640 1 --2- 172.21.15.86:0/2685658157 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f400d1df0 0x7f3f400cd780 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:33.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.785+0000 7f3f48000640 1 -- 172.21.15.86:0/2685658157 >> 172.21.15.86:0/2685658157 conn(0x7f3f400b4650 msgr2=0x7f3f400b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:33.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.785+0000 7f3f48000640 1 -- 172.21.15.86:0/2685658157 shutdown_connections 2024-04-25T14:11:33.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.785+0000 7f3f48000640 1 -- 172.21.15.86:0/2685658157 wait complete. 2024-04-25T14:11:33.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.789+0000 7f3f48000640 1 Processor -- start 2024-04-25T14:11:33.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.789+0000 7f3f48000640 1 -- start start 2024-04-25T14:11:33.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.789+0000 7f3f48000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f40070330 0x7f3f400c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:33.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.789+0000 7f3f48000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f400cdcc0 0x7f3f40159f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:33.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.789+0000 7f3f48000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f4015cf10 0x7f3f4015d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:33.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.789+0000 7f3f48000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4015f4b0 con 0x7f3f40070330 2024-04-25T14:11:33.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.789+0000 7f3f48000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4015f330 con 0x7f3f400cdcc0 2024-04-25T14:11:33.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.789+0000 7f3f48000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3f40070740 con 0x7f3f4015cf10 2024-04-25T14:11:33.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.789+0000 7f3f45600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f40070330 0x7f3f400c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:33.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.789+0000 7f3f45600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f40070330 0x7f3f400c98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40888/0 (socket says 172.21.15.86:40888) 2024-04-25T14:11:33.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.789+0000 7f3f45600640 1 -- 172.21.15.86:0/7049857 learned_addr learned my addr 172.21.15.86:0/7049857 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:33.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.789+0000 7f3f44c00640 1 --2- 172.21.15.86:0/7049857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f400cdcc0 0x7f3f40159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:33.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.789+0000 7f3f46000640 1 --2- 172.21.15.86:0/7049857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f4015cf10 0x7f3f4015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:33.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.789+0000 7f3f45600640 1 -- 172.21.15.86:0/7049857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f4015cf10 msgr2=0x7f3f4015d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:33.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.789+0000 7f3f45600640 1 --2- 172.21.15.86:0/7049857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f4015cf10 0x7f3f4015d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:33.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.789+0000 7f3f45600640 1 -- 172.21.15.86:0/7049857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f400cdcc0 msgr2=0x7f3f40159f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:33.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.789+0000 7f3f45600640 1 --2- 172.21.15.86:0/7049857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f400cdcc0 0x7f3f40159f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:33.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.789+0000 7f3f45600640 1 -- 172.21.15.86:0/7049857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4015d860 con 0x7f3f40070330 2024-04-25T14:11:33.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.789+0000 7f3f44c00640 1 --2- 172.21.15.86:0/7049857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f400cdcc0 0x7f3f40159f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:11:33.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.793+0000 7f3f45600640 1 --2- 172.21.15.86:0/7049857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f40070330 0x7f3f400c98b0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f3f3400c370 tx=0x7f3f3400c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:33.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.793+0000 7f3f3d600640 1 -- 172.21.15.86:0/7049857 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f340074e0 con 0x7f3f40070330 2024-04-25T14:11:33.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.793+0000 7f3f3d600640 1 -- 172.21.15.86:0/7049857 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f34007680 con 0x7f3f40070330 2024-04-25T14:11:33.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.793+0000 7f3f48000640 1 -- 172.21.15.86:0/7049857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f4015b040 con 0x7f3f40070330 2024-04-25T14:11:33.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.793+0000 7f3f3d600640 1 -- 172.21.15.86:0/7049857 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f34004a30 con 0x7f3f40070330 2024-04-25T14:11:33.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.793+0000 7f3f48000640 1 -- 172.21.15.86:0/7049857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f4015b600 con 0x7f3f40070330 2024-04-25T14:11:33.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.793+0000 7f3f48000640 1 -- 172.21.15.86:0/7049857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f40069d00 con 0x7f3f40070330 2024-04-25T14:11:33.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.793+0000 7f3f3d600640 1 -- 172.21.15.86:0/7049857 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7f3f3401a020 con 0x7f3f40070330 2024-04-25T14:11:33.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.793+0000 7f3f3d600640 1 --2- 172.21.15.86:0/7049857 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f3f2006f520 0x7f3f20071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:33.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.797+0000 7f3f3d600640 1 -- 172.21.15.86:0/7049857 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 6101+0+0 (secure 0 0 0) 0x7f3f3408ecc0 con 0x7f3f40070330 2024-04-25T14:11:33.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.797+0000 7f3f44c00640 1 --2- 172.21.15.86:0/7049857 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f3f2006f520 0x7f3f20071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:33.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.797+0000 7f3f44c00640 1 --2- 172.21.15.86:0/7049857 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f3f2006f520 0x7f3f20071a30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c0af000 tx=0x7f3f2c0a7000 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:33.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.801+0000 7f3f3d600640 1 -- 172.21.15.86:0/7049857 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3f34095080 con 0x7f3f40070330 2024-04-25T14:11:33.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.957+0000 7f3f48000640 1 -- 172.21.15.86:0/7049857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f40076bf0 con 0x7f3f40070330 2024-04-25T14:11:33.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.961+0000 7f3f3d600640 1 -- 172.21.15.86:0/7049857 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) ==== 74+0+16905 (secure 0 0 0) 0x7f3f4015d860 con 0x7f3f40070330 2024-04-25T14:11:33.957 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:33.957 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":139,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:11:32.981579+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:11:30.810321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T14:11:31.944888+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:11:33.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.961+0000 7f3f48000640 1 -- 172.21.15.86:0/7049857 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f3f2006f520 msgr2=0x7f3f20071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:33.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.961+0000 7f3f48000640 1 --2- 172.21.15.86:0/7049857 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f3f2006f520 0x7f3f20071a30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c0af000 tx=0x7f3f2c0a7000 comp rx=0 tx=0).stop 2024-04-25T14:11:33.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.961+0000 7f3f48000640 1 -- 172.21.15.86:0/7049857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f40070330 msgr2=0x7f3f400c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:33.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.961+0000 7f3f48000640 1 --2- 172.21.15.86:0/7049857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f40070330 0x7f3f400c98b0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f3f3400c370 tx=0x7f3f3400c770 comp rx=0 tx=0).stop 2024-04-25T14:11:33.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.961+0000 7f3f48000640 1 -- 172.21.15.86:0/7049857 shutdown_connections 2024-04-25T14:11:33.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.961+0000 7f3f48000640 1 --2- 172.21.15.86:0/7049857 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f3f2006f520 0x7f3f20071a30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:33.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.961+0000 7f3f48000640 1 --2- 172.21.15.86:0/7049857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f4015cf10 0x7f3f4015d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:33.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.961+0000 7f3f48000640 1 --2- 172.21.15.86:0/7049857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f400cdcc0 0x7f3f40159f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:33.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.961+0000 7f3f48000640 1 --2- 172.21.15.86:0/7049857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f40070330 0x7f3f400c98b0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:33.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.961+0000 7f3f48000640 1 -- 172.21.15.86:0/7049857 >> 172.21.15.86:0/7049857 conn(0x7f3f400b4650 msgr2=0x7f3f400c7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:33.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.961+0000 7f3f48000640 1 -- 172.21.15.86:0/7049857 shutdown_connections 2024-04-25T14:11:33.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:33.961+0000 7f3f48000640 1 -- 172.21.15.86:0/7049857 wait complete. 2024-04-25T14:11:33.979 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:11:34.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.129+0000 7f4f0f400640 1 Processor -- start 2024-04-25T14:11:34.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.129+0000 7f4f0f400640 1 -- start start 2024-04-25T14:11:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.129+0000 7f4f0f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4f080d1df0 0x7f4f080cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.129+0000 7f4f0f400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f080cdcc0 0x7f4f080ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.129+0000 7f4f0f400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f0815e5c0 con 0x7f4f08070330 2024-04-25T14:11:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.129+0000 7f4f0f400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4f0806fd40 con 0x7f4f080d1df0 2024-04-25T14:11:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.129+0000 7f4f0f400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4f0815e8a0 con 0x7f4f080cdcc0 2024-04-25T14:11:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.129+0000 7f4f07e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4f080d1df0 0x7f4f080cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.129+0000 7f4f0ca00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f4f08070330 0x7f4f080d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60436/0 (socket says 172.21.15.86:60436) 2024-04-25T14:11:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.129+0000 7f4f0ca00640 1 -- 172.21.15.86:0/2554008922 learned_addr learned my addr 172.21.15.86:0/2554008922 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:34.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.133+0000 7f4f0d400640 1 --2- 172.21.15.86:0/2554008922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f080cdcc0 0x7f4f080ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.133+0000 7f4f07e00640 1 -- 172.21.15.86:0/2554008922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f080cdcc0 msgr2=0x7f4f080ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.133+0000 7f4f07e00640 1 --2- 172.21.15.86:0/2554008922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f080cdcc0 0x7f4f080ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.133+0000 7f4f07e00640 1 -- 172.21.15.86:0/2554008922 >> v1:172.21.15.86:6789/0 conn(0x7f4f08070330 legacy=0x7f4f080d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.133+0000 7f4f07e00640 1 -- 172.21.15.86:0/2554008922 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0815eb80 con 0x7f4f080d1df0 2024-04-25T14:11:34.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.133+0000 7f4f07e00640 1 --2- 172.21.15.86:0/2554008922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f080d1df0 0x7f4f080cd780 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40097c0 tx=0x7f4ef401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13db9d2d1ee9cc07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:34.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.133+0000 7f4f07400640 1 -- 172.21.15.86:0/2554008922 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef40b1070 con 0x7f4f080d1df0 2024-04-25T14:11:34.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.133+0000 7f4f07400640 1 -- 172.21.15.86:0/2554008922 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef40ac620 con 0x7f4f080d1df0 2024-04-25T14:11:34.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.133+0000 7f4f07400640 1 -- 172.21.15.86:0/2554008922 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef40ac9f0 con 0x7f4f080d1df0 2024-04-25T14:11:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.137+0000 7f4f0f400640 1 -- 172.21.15.86:0/2554008922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f080d1df0 msgr2=0x7f4f080cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.137+0000 7f4f0f400640 1 --2- 172.21.15.86:0/2554008922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f080d1df0 0x7f4f080cd780 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40097c0 tx=0x7f4ef401c3e0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.137+0000 7f4f0f400640 1 -- 172.21.15.86:0/2554008922 shutdown_connections 2024-04-25T14:11:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.137+0000 7f4f0f400640 1 --2- 172.21.15.86:0/2554008922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f080cdcc0 0x7f4f080ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.137+0000 7f4f0f400640 1 --2- 172.21.15.86:0/2554008922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f080d1df0 0x7f4f080cd780 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.137+0000 7f4f0f400640 1 -- 172.21.15.86:0/2554008922 >> 172.21.15.86:0/2554008922 conn(0x7f4f080b4650 msgr2=0x7f4f080b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.137+0000 7f4f0f400640 1 -- 172.21.15.86:0/2554008922 shutdown_connections 2024-04-25T14:11:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.137+0000 7f4f0f400640 1 -- 172.21.15.86:0/2554008922 wait complete. 2024-04-25T14:11:34.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.137+0000 7f4f0f400640 1 Processor -- start 2024-04-25T14:11:34.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.137+0000 7f4f0f400640 1 -- start start 2024-04-25T14:11:34.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.141+0000 7f4f0f400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f08070330 0x7f4f080c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:34.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.141+0000 7f4f0f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f080cdcc0 0x7f4f080ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:34.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.141+0000 7f4f0f400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f08158fc0 0x7f4f08073f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:34.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.141+0000 7f4f0f400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0815f4b0 con 0x7f4f080cdcc0 2024-04-25T14:11:34.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.141+0000 7f4f0f400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0815f330 con 0x7f4f08158fc0 2024-04-25T14:11:34.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.141+0000 7f4f0f400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4f08070740 con 0x7f4f08070330 2024-04-25T14:11:34.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.141+0000 7f4f07e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f080cdcc0 0x7f4f080ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:34.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.141+0000 7f4f07e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f080cdcc0 0x7f4f080ae230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40906/0 (socket says 172.21.15.86:40906) 2024-04-25T14:11:34.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.141+0000 7f4f07e00640 1 -- 172.21.15.86:0/2807006666 learned_addr learned my addr 172.21.15.86:0/2807006666 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:34.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.141+0000 7f4f0ca00640 1 --2- 172.21.15.86:0/2807006666 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f08070330 0x7f4f080c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:34.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.141+0000 7f4f0d400640 1 --2- 172.21.15.86:0/2807006666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f08158fc0 0x7f4f08073f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:34.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.141+0000 7f4f07e00640 1 -- 172.21.15.86:0/2807006666 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f08070330 msgr2=0x7f4f080c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:34.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.141+0000 7f4f07e00640 1 --2- 172.21.15.86:0/2807006666 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f08070330 0x7f4f080c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.141+0000 7f4f07e00640 1 -- 172.21.15.86:0/2807006666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f08158fc0 msgr2=0x7f4f08073f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:34.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.141+0000 7f4f07e00640 1 --2- 172.21.15.86:0/2807006666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f08158fc0 0x7f4f08073f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.141+0000 7f4f07e00640 1 -- 172.21.15.86:0/2807006666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f080ae770 con 0x7f4f080cdcc0 2024-04-25T14:11:34.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.141+0000 7f4f0ca00640 1 --2- 172.21.15.86:0/2807006666 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f08070330 0x7f4f080c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:34.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.141+0000 7f4f0d400640 1 --2- 172.21.15.86:0/2807006666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f08158fc0 0x7f4f08073f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:34.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.141+0000 7f4f07e00640 1 --2- 172.21.15.86:0/2807006666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f080cdcc0 0x7f4f080ae230 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40097c0 tx=0x7f4ef4019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:34.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.141+0000 7f4f04c00640 1 -- 172.21.15.86:0/2807006666 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef40bb070 con 0x7f4f080cdcc0 2024-04-25T14:11:34.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.141+0000 7f4f04c00640 1 -- 172.21.15.86:0/2807006666 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef40b6460 con 0x7f4f080cdcc0 2024-04-25T14:11:34.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.141+0000 7f4f0f400640 1 -- 172.21.15.86:0/2807006666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f080c7600 con 0x7f4f080cdcc0 2024-04-25T14:11:34.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.141+0000 7f4f04c00640 1 -- 172.21.15.86:0/2807006666 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef40b1040 con 0x7f4f080cdcc0 2024-04-25T14:11:34.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.141+0000 7f4f0f400640 1 -- 172.21.15.86:0/2807006666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f08073750 con 0x7f4f080cdcc0 2024-04-25T14:11:34.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.141+0000 7f4f0f400640 1 -- 172.21.15.86:0/2807006666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f08073290 con 0x7f4f080cdcc0 2024-04-25T14:11:34.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.141+0000 7f4f04c00640 1 -- 172.21.15.86:0/2807006666 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7f4ef40be070 con 0x7f4f080cdcc0 2024-04-25T14:11:34.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.145+0000 7f4f04c00640 1 --2- 172.21.15.86:0/2807006666 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f4ee806f520 0x7f4ee8071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:34.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.145+0000 7f4f04c00640 1 -- 172.21.15.86:0/2807006666 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 6101+0+0 (secure 0 0 0) 0x7f4ef4129cb0 con 0x7f4f080cdcc0 2024-04-25T14:11:34.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.145+0000 7f4f0ca00640 1 --2- 172.21.15.86:0/2807006666 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f4ee806f520 0x7f4ee8071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:34.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.145+0000 7f4f0ca00640 1 --2- 172.21.15.86:0/2807006666 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f4ee806f520 0x7f4ee8071a30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80012f0 tx=0x7f4ef8009040 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:34.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.149+0000 7f4f04c00640 1 -- 172.21.15.86:0/2807006666 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4f08073290 con 0x7f4f080cdcc0 2024-04-25T14:11:34.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.305+0000 7f4f0f400640 1 -- 172.21.15.86:0/2807006666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f08069d00 con 0x7f4f080cdcc0 2024-04-25T14:11:34.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.309+0000 7f4f04c00640 1 -- 172.21.15.86:0/2807006666 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) ==== 74+0+16905 (secure 0 0 0) 0x7f4ef4016090 con 0x7f4f080cdcc0 2024-04-25T14:11:34.304 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:34.304 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":139,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:11:32.981579+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:11:30.810321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T14:11:31.944888+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:11:34.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.309+0000 7f4f0f400640 1 -- 172.21.15.86:0/2807006666 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f4ee806f520 msgr2=0x7f4ee8071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:34.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.309+0000 7f4f0f400640 1 --2- 172.21.15.86:0/2807006666 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f4ee806f520 0x7f4ee8071a30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80012f0 tx=0x7f4ef8009040 comp rx=0 tx=0).stop 2024-04-25T14:11:34.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.309+0000 7f4f0f400640 1 -- 172.21.15.86:0/2807006666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f080cdcc0 msgr2=0x7f4f080ae230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:34.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.309+0000 7f4f0f400640 1 --2- 172.21.15.86:0/2807006666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f080cdcc0 0x7f4f080ae230 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40097c0 tx=0x7f4ef4019180 comp rx=0 tx=0).stop 2024-04-25T14:11:34.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.309+0000 7f4f0f400640 1 -- 172.21.15.86:0/2807006666 shutdown_connections 2024-04-25T14:11:34.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.309+0000 7f4f0f400640 1 --2- 172.21.15.86:0/2807006666 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f4ee806f520 0x7f4ee8071a30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.309+0000 7f4f0f400640 1 --2- 172.21.15.86:0/2807006666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f08158fc0 0x7f4f08073f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.309+0000 7f4f0f400640 1 --2- 172.21.15.86:0/2807006666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f080cdcc0 0x7f4f080ae230 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.309+0000 7f4f0f400640 1 --2- 172.21.15.86:0/2807006666 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f08070330 0x7f4f080c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.309+0000 7f4f0f400640 1 -- 172.21.15.86:0/2807006666 >> 172.21.15.86:0/2807006666 conn(0x7f4f080b4650 msgr2=0x7f4f080c4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:34.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.309+0000 7f4f0f400640 1 -- 172.21.15.86:0/2807006666 shutdown_connections 2024-04-25T14:11:34.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.309+0000 7f4f0f400640 1 -- 172.21.15.86:0/2807006666 wait complete. 2024-04-25T14:11:34.326 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T14:11:34.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.477+0000 7f12e2a00640 1 Processor -- start 2024-04-25T14:11:34.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.477+0000 7f12e2a00640 1 -- start start 2024-04-25T14:11:34.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.481+0000 7f12e2a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f12dc0740c0 0x7f12dc1676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:34.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.481+0000 7f12e2a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12dc072bd0 con 0x7f12dc0cfca0 2024-04-25T14:11:34.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.481+0000 7f12e2a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12dc06fbd0 con 0x7f12dc0d16e0 2024-04-25T14:11:34.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.481+0000 7f12e2a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f12dc070470 con 0x7f12dc0740c0 2024-04-25T14:11:34.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.481+0000 7f12dbe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f12dc0cfca0 0x7f12dc06f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60438/0 (socket says 172.21.15.86:60438) 2024-04-25T14:11:34.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.481+0000 7f12dbe00640 1 -- 172.21.15.86:0/2255381358 learned_addr learned my addr 172.21.15.86:0/2255381358 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:34.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.481+0000 7f12e0a00640 1 --2- 172.21.15.86:0/2255381358 >> v2:172.21.15.155:3300/0 conn(0x7f12dc0740c0 0x7f12dc1676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:34.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.481+0000 7f12daa00640 1 -- 172.21.15.86:0/2255381358 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1549313885 0 0) 0x7f12dc072bd0 con 0x7f12dc0cfca0 2024-04-25T14:11:34.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.481+0000 7f12daa00640 1 -- 172.21.15.86:0/2255381358 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12c0005740 con 0x7f12dc0cfca0 2024-04-25T14:11:34.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.481+0000 7f12daa00640 1 -- 172.21.15.86:0/2255381358 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4135963304 0 0) 0x7f12dc06fbd0 con 0x7f12dc0d16e0 2024-04-25T14:11:34.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.481+0000 7f12daa00640 1 -- 172.21.15.86:0/2255381358 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12dc072bd0 con 0x7f12dc0d16e0 2024-04-25T14:11:34.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.481+0000 7f12daa00640 1 -- 172.21.15.86:0/2255381358 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3958969242 0 0) 0x7f12c0005740 con 0x7f12dc0cfca0 2024-04-25T14:11:34.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.481+0000 7f12e0a00640 1 -- 172.21.15.86:0/2255381358 >> v1:172.21.15.155:6789/0 conn(0x7f12dc0d16e0 legacy=0x7f12dc0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:34.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.481+0000 7f12e0a00640 1 -- 172.21.15.86:0/2255381358 >> v1:172.21.15.86:6789/0 conn(0x7f12dc0cfca0 legacy=0x7f12dc06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:34.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.481+0000 7f12e0a00640 1 -- 172.21.15.86:0/2255381358 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12dc06feb0 con 0x7f12dc0740c0 2024-04-25T14:11:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.485+0000 7f12e0a00640 1 --2- 172.21.15.86:0/2255381358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12dc0740c0 0x7f12dc1676d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f12d40097c0 tx=0x7f12d401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87f797a7802ee3ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.485+0000 7f12daa00640 1 -- 172.21.15.86:0/2255381358 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d40b0070 con 0x7f12dc0740c0 2024-04-25T14:11:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.485+0000 7f12daa00640 1 -- 172.21.15.86:0/2255381358 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12d40ab400 con 0x7f12dc0740c0 2024-04-25T14:11:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.485+0000 7f12daa00640 1 -- 172.21.15.86:0/2255381358 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc001a20 con 0x7f12dc0740c0 2024-04-25T14:11:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.485+0000 7f12e2a00640 1 -- 172.21.15.86:0/2255381358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12dc0740c0 msgr2=0x7f12dc1676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.485+0000 7f12e2a00640 1 --2- 172.21.15.86:0/2255381358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12dc0740c0 0x7f12dc1676d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f12d40097c0 tx=0x7f12d401c3c0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.485+0000 7f12e2a00640 1 -- 172.21.15.86:0/2255381358 shutdown_connections 2024-04-25T14:11:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.485+0000 7f12e2a00640 1 --2- 172.21.15.86:0/2255381358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12dc0740c0 0x7f12dc1676d0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.485+0000 7f12e2a00640 1 -- 172.21.15.86:0/2255381358 >> 172.21.15.86:0/2255381358 conn(0x7f12dc0b4650 msgr2=0x7f12dc0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.485+0000 7f12e2a00640 1 -- 172.21.15.86:0/2255381358 shutdown_connections 2024-04-25T14:11:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.485+0000 7f12e2a00640 1 -- 172.21.15.86:0/2255381358 wait complete. 2024-04-25T14:11:34.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.489+0000 7f12e2a00640 1 Processor -- start 2024-04-25T14:11:34.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.489+0000 7f12e2a00640 1 -- start start 2024-04-25T14:11:34.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.489+0000 7f12e2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12dc0740c0 0x7f12dc155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:34.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.489+0000 7f12e2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12dc0cfca0 0x7f12dc153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:34.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.489+0000 7f12e2a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12dc0d16e0 0x7f12dc159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:34.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.489+0000 7f12e2a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc0ce900 con 0x7f12dc0740c0 2024-04-25T14:11:34.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.489+0000 7f12e2a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc0ce780 con 0x7f12dc0cfca0 2024-04-25T14:11:34.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.489+0000 7f12e2a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f12dc0cea80 con 0x7f12dc0d16e0 2024-04-25T14:11:34.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.489+0000 7f12dbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12dc0740c0 0x7f12dc155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:34.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.489+0000 7f12dbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12dc0740c0 0x7f12dc155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40922/0 (socket says 172.21.15.86:40922) 2024-04-25T14:11:34.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.489+0000 7f12dbe00640 1 -- 172.21.15.86:0/371624070 learned_addr learned my addr 172.21.15.86:0/371624070 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:34.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.489+0000 7f12e0a00640 1 --2- 172.21.15.86:0/371624070 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12dc0d16e0 0x7f12dc159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:34.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.489+0000 7f12db400640 1 --2- 172.21.15.86:0/371624070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12dc0cfca0 0x7f12dc153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:34.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.489+0000 7f12dbe00640 1 -- 172.21.15.86:0/371624070 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12dc0d16e0 msgr2=0x7f12dc159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:34.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.489+0000 7f12dbe00640 1 --2- 172.21.15.86:0/371624070 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12dc0d16e0 0x7f12dc159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.489+0000 7f12dbe00640 1 -- 172.21.15.86:0/371624070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12dc0cfca0 msgr2=0x7f12dc153f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:34.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.489+0000 7f12dbe00640 1 --2- 172.21.15.86:0/371624070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12dc0cfca0 0x7f12dc153f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:34.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.489+0000 7f12dbe00640 1 -- 172.21.15.86:0/371624070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12dc154450 con 0x7f12dc0740c0 2024-04-25T14:11:34.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.489+0000 7f12dbe00640 1 --2- 172.21.15.86:0/371624070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12dc0740c0 0x7f12dc155a00 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f12cc00cc20 tx=0x7f12cc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:34.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.489+0000 7f12d3e00640 1 -- 172.21.15.86:0/371624070 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc017070 con 0x7f12dc0740c0 2024-04-25T14:11:34.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.489+0000 7f12d3e00640 1 -- 172.21.15.86:0/371624070 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12cc00ce30 con 0x7f12dc0740c0 2024-04-25T14:11:34.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.489+0000 7f12e2a00640 1 -- 172.21.15.86:0/371624070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12dc154960 con 0x7f12dc0740c0 2024-04-25T14:11:34.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.489+0000 7f12d3e00640 1 -- 172.21.15.86:0/371624070 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc012620 con 0x7f12dc0740c0 2024-04-25T14:11:34.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.489+0000 7f12e2a00640 1 -- 172.21.15.86:0/371624070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12dc154f20 con 0x7f12dc0740c0 2024-04-25T14:11:34.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.489+0000 7f12e2a00640 1 -- 172.21.15.86:0/371624070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12dc09f4b0 con 0x7f12dc0740c0 2024-04-25T14:11:34.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.493+0000 7f12d3e00640 1 -- 172.21.15.86:0/371624070 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7f12cc01a070 con 0x7f12dc0740c0 2024-04-25T14:11:34.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.493+0000 7f12d3e00640 1 --2- 172.21.15.86:0/371624070 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f12bc06f550 0x7f12bc071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:34.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.493+0000 7f12d3e00640 1 -- 172.21.15.86:0/371624070 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 6101+0+0 (secure 0 0 0) 0x7f12cc08f2f0 con 0x7f12dc0740c0 2024-04-25T14:11:34.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.493+0000 7f12db400640 1 --2- 172.21.15.86:0/371624070 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f12bc06f550 0x7f12bc071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:34.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.493+0000 7f12db400640 1 --2- 172.21.15.86:0/371624070 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f12bc06f550 0x7f12bc071a60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f12dc153e10 tx=0x7f12c8009040 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:34.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.497+0000 7f12d3e00640 1 -- 172.21.15.86:0/371624070 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f12cc05b980 con 0x7f12dc0740c0 2024-04-25T14:11:34.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:34.697+0000 7f12e2a00640 1 -- 172.21.15.86:0/371624070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f12dc069d00 con 0x7f12dc0740c0 2024-04-25T14:11:35.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.601+0000 7f12d3e00640 1 -- 172.21.15.86:0/371624070 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v102) ==== 118+0+0 (secure 0 0 0) 0x7f12cc05fc80 con 0x7f12dc0740c0 2024-04-25T14:11:35.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.605+0000 7f12e2a00640 1 -- 172.21.15.86:0/371624070 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f12bc06f550 msgr2=0x7f12bc071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:35.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.605+0000 7f12e2a00640 1 --2- 172.21.15.86:0/371624070 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f12bc06f550 0x7f12bc071a60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f12dc153e10 tx=0x7f12c8009040 comp rx=0 tx=0).stop 2024-04-25T14:11:35.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.605+0000 7f12e2a00640 1 -- 172.21.15.86:0/371624070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12dc0740c0 msgr2=0x7f12dc155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:35.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.605+0000 7f12e2a00640 1 --2- 172.21.15.86:0/371624070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12dc0740c0 0x7f12dc155a00 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f12cc00cc20 tx=0x7f12cc00a430 comp rx=0 tx=0).stop 2024-04-25T14:11:35.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.605+0000 7f12e2a00640 1 -- 172.21.15.86:0/371624070 shutdown_connections 2024-04-25T14:11:35.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.605+0000 7f12e2a00640 1 --2- 172.21.15.86:0/371624070 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f12bc06f550 0x7f12bc071a60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.605+0000 7f12e2a00640 1 --2- 172.21.15.86:0/371624070 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12dc0d16e0 0x7f12dc159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.605+0000 7f12e2a00640 1 --2- 172.21.15.86:0/371624070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12dc0cfca0 0x7f12dc153f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.605+0000 7f12e2a00640 1 --2- 172.21.15.86:0/371624070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12dc0740c0 0x7f12dc155a00 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.605+0000 7f12e2a00640 1 -- 172.21.15.86:0/371624070 >> 172.21.15.86:0/371624070 conn(0x7f12dc0b4650 msgr2=0x7f12dc0cc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:35.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.605+0000 7f12e2a00640 1 -- 172.21.15.86:0/371624070 shutdown_connections 2024-04-25T14:11:35.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.605+0000 7f12e2a00640 1 -- 172.21.15.86:0/371624070 wait complete. 2024-04-25T14:11:35.625 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T14:11:35.625 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T14:11:35.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.781+0000 7fca3b200640 1 Processor -- start 2024-04-25T14:11:35.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.781+0000 7fca3b200640 1 -- start start 2024-04-25T14:11:35.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca3b200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fca34070330 0x7fca340d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:35.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca3b200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca340d2030 con 0x7fca340ce790 2024-04-25T14:11:35.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca3b200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca3415e5c0 con 0x7fca340d1c20 2024-04-25T14:11:35.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca3b200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fca3415e8a0 con 0x7fca34070330 2024-04-25T14:11:35.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca39200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fca340ce790 0x7fca3406f440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60442/0 (socket says 172.21.15.86:60442) 2024-04-25T14:11:35.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca39200640 1 -- 172.21.15.86:0/1436371363 learned_addr learned my addr 172.21.15.86:0/1436371363 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:35.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca33e00640 1 --2- 172.21.15.86:0/1436371363 >> v2:172.21.15.155:3300/0 conn(0x7fca34070330 0x7fca340d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:35.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca33400640 1 -- 172.21.15.86:0/1436371363 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1200844684 0 0) 0x7fca2c0014a0 con 0x7fca340ce790 2024-04-25T14:11:35.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca33400640 1 -- 172.21.15.86:0/1436371363 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca18003530 con 0x7fca340ce790 2024-04-25T14:11:35.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca33400640 1 -- 172.21.15.86:0/1436371363 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 949364225 0 0) 0x7fca18003530 con 0x7fca340ce790 2024-04-25T14:11:35.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca33400640 1 -- 172.21.15.86:0/1436371363 >> v2:172.21.15.155:3300/0 conn(0x7fca34070330 msgr2=0x7fca340d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:35.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca33400640 1 --2- 172.21.15.86:0/1436371363 >> v2:172.21.15.155:3300/0 conn(0x7fca34070330 0x7fca340d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca33400640 1 -- 172.21.15.86:0/1436371363 >> v1:172.21.15.155:6789/0 conn(0x7fca340d1c20 legacy=0x7fca3406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:35.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca33400640 1 -- 172.21.15.86:0/1436371363 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3415eb80 con 0x7fca340ce790 2024-04-25T14:11:35.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca33400640 1 -- 172.21.15.86:0/1436371363 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fca2c001bd0 con 0x7fca340ce790 2024-04-25T14:11:35.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca33400640 1 -- 172.21.15.86:0/1436371363 >> v1:172.21.15.86:6789/0 conn(0x7fca340ce790 legacy=0x7fca3406f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:35.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca33400640 1 --2- 172.21.15.86:0/1436371363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca18003a30 0x7fca18003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:35.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca33400640 1 --2- 172.21.15.86:0/1436371363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca18004810 0x7fca18004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:35.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca33400640 1 --2- 172.21.15.86:0/1436371363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca18005360 0x7fca1800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:35.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca33400640 1 -- 172.21.15.86:0/1436371363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca3415e8a0 con 0x7fca18004810 2024-04-25T14:11:35.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca33400640 1 -- 172.21.15.86:0/1436371363 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fca1800c260 con 0x7fca18003a30 2024-04-25T14:11:35.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca33400640 1 -- 172.21.15.86:0/1436371363 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fca1800c460 con 0x7fca18005360 2024-04-25T14:11:35.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca2b400640 1 --2- 172.21.15.86:0/1436371363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca18004810 0x7fca18004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:35.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca39200640 1 --2- 172.21.15.86:0/1436371363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca18005360 0x7fca1800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:35.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca33e00640 1 --2- 172.21.15.86:0/1436371363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca18003a30 0x7fca18003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:35.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca2b400640 1 -- 172.21.15.86:0/1436371363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca18005360 msgr2=0x7fca1800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:35.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca2b400640 1 --2- 172.21.15.86:0/1436371363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca18005360 0x7fca1800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca2b400640 1 -- 172.21.15.86:0/1436371363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca18003a30 msgr2=0x7fca18003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:35.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca2b400640 1 --2- 172.21.15.86:0/1436371363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca18003a30 0x7fca18003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca2b400640 1 -- 172.21.15.86:0/1436371363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca18003530 con 0x7fca18004810 2024-04-25T14:11:35.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca39200640 1 --2- 172.21.15.86:0/1436371363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca18005360 0x7fca1800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:35.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca33e00640 1 --2- 172.21.15.86:0/1436371363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca18003a30 0x7fca18003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:35.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.785+0000 7fca39200640 1 -- 172.21.15.86:0/1436371363 reap_dead start 2024-04-25T14:11:35.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.789+0000 7fca2b400640 1 --2- 172.21.15.86:0/1436371363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca18004810 0x7fca18004c20 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fca34074ed0 tx=0x7fca1c01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d01969552c244d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:35.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.789+0000 7fca33400640 1 -- 172.21.15.86:0/1436371363 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0013a0 con 0x7fca18004810 2024-04-25T14:11:35.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.789+0000 7fca33400640 1 -- 172.21.15.86:0/1436371363 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca1c0052f0 con 0x7fca18004810 2024-04-25T14:11:35.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.789+0000 7fca33400640 1 -- 172.21.15.86:0/1436371363 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c003840 con 0x7fca18004810 2024-04-25T14:11:35.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.789+0000 7fca3b200640 1 -- 172.21.15.86:0/1436371363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca18004810 msgr2=0x7fca18004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:35.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.789+0000 7fca3b200640 1 --2- 172.21.15.86:0/1436371363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca18004810 0x7fca18004c20 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fca34074ed0 tx=0x7fca1c01c3a0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.789+0000 7fca3b200640 1 -- 172.21.15.86:0/1436371363 shutdown_connections 2024-04-25T14:11:35.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.789+0000 7fca3b200640 1 --2- 172.21.15.86:0/1436371363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca18004810 0x7fca18004c20 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.789+0000 7fca3b200640 1 -- 172.21.15.86:0/1436371363 >> 172.21.15.86:0/1436371363 conn(0x7fca340b4650 msgr2=0x7fca340b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:35.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.789+0000 7fca3b200640 1 -- 172.21.15.86:0/1436371363 shutdown_connections 2024-04-25T14:11:35.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.789+0000 7fca3b200640 1 -- 172.21.15.86:0/1436371363 wait complete. 2024-04-25T14:11:35.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.793+0000 7fca3b200640 1 Processor -- start 2024-04-25T14:11:35.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.793+0000 7fca3b200640 1 -- start start 2024-04-25T14:11:35.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.793+0000 7fca3b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca18004810 0x7fca341f7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:35.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.793+0000 7fca3b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca34072880 0x7fca341583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:35.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.793+0000 7fca3b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca341fb580 0x7fca341fb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:35.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.793+0000 7fca3b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca340708c0 con 0x7fca341fb580 2024-04-25T14:11:35.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.793+0000 7fca3b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fca34070740 con 0x7fca34072880 2024-04-25T14:11:35.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.793+0000 7fca3b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fca340cdf90 con 0x7fca18004810 2024-04-25T14:11:35.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.793+0000 7fca39200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca341fb580 0x7fca341fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:35.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.793+0000 7fca39200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca341fb580 0x7fca341fb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40946/0 (socket says 172.21.15.86:40946) 2024-04-25T14:11:35.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.793+0000 7fca39200640 1 -- 172.21.15.86:0/924910171 learned_addr learned my addr 172.21.15.86:0/924910171 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:35.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.793+0000 7fca2b400640 1 --2- 172.21.15.86:0/924910171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca34072880 0x7fca341583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:35.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.793+0000 7fca33e00640 1 --2- 172.21.15.86:0/924910171 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca18004810 0x7fca341f7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:35.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.793+0000 7fca39200640 1 -- 172.21.15.86:0/924910171 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca18004810 msgr2=0x7fca341f7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:35.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.793+0000 7fca39200640 1 --2- 172.21.15.86:0/924910171 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca18004810 0x7fca341f7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.793+0000 7fca39200640 1 -- 172.21.15.86:0/924910171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca34072880 msgr2=0x7fca341583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:35.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.793+0000 7fca39200640 1 --2- 172.21.15.86:0/924910171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca34072880 0x7fca341583c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:35.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.793+0000 7fca39200640 1 -- 172.21.15.86:0/924910171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca341fbed0 con 0x7fca341fb580 2024-04-25T14:11:35.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.793+0000 7fca2b400640 1 --2- 172.21.15.86:0/924910171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca34072880 0x7fca341583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:35.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.797+0000 7fca39200640 1 --2- 172.21.15.86:0/924910171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca341fb580 0x7fca341fb990 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fca340cc200 tx=0x7fca2c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:35.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.797+0000 7fca30c00640 1 -- 172.21.15.86:0/924910171 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c00eb90 con 0x7fca341fb580 2024-04-25T14:11:35.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.797+0000 7fca30c00640 1 -- 172.21.15.86:0/924910171 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca2c019070 con 0x7fca341fb580 2024-04-25T14:11:35.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.797+0000 7fca3b200640 1 -- 172.21.15.86:0/924910171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca34157c00 con 0x7fca341fb580 2024-04-25T14:11:35.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.797+0000 7fca30c00640 1 -- 172.21.15.86:0/924910171 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c013420 con 0x7fca341fb580 2024-04-25T14:11:35.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.797+0000 7fca3b200640 1 -- 172.21.15.86:0/924910171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca34161850 con 0x7fca341fb580 2024-04-25T14:11:35.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.797+0000 7fca3b200640 1 -- 172.21.15.86:0/924910171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca34069d00 con 0x7fca341fb580 2024-04-25T14:11:35.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.797+0000 7fca30c00640 1 -- 172.21.15.86:0/924910171 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7fca2c0135c0 con 0x7fca341fb580 2024-04-25T14:11:35.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.797+0000 7fca30c00640 1 --2- 172.21.15.86:0/924910171 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fca1006f550 0x7fca10071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:35.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.797+0000 7fca30c00640 1 -- 172.21.15.86:0/924910171 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7fca2c08f890 con 0x7fca341fb580 2024-04-25T14:11:35.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.797+0000 7fca33e00640 1 --2- 172.21.15.86:0/924910171 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fca1006f550 0x7fca10071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:35.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.801+0000 7fca33e00640 1 --2- 172.21.15.86:0/924910171 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fca1006f550 0x7fca10071a60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fca34072200 tx=0x7fca24008040 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:35.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.805+0000 7fca30c00640 1 -- 172.21.15.86:0/924910171 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fca341fbed0 con 0x7fca341fb580 2024-04-25T14:11:35.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:35.993+0000 7fca3b200640 1 -- 172.21.15.86:0/924910171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fca34076bf0 con 0x7fca341fb580 2024-04-25T14:11:36.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.609+0000 7fca30c00640 1 -- 172.21.15.86:0/924910171 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v103) ==== 122+0+0 (secure 0 0 0) 0x7fca340d2030 con 0x7fca341fb580 2024-04-25T14:11:36.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.613+0000 7fca3b200640 1 -- 172.21.15.86:0/924910171 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fca1006f550 msgr2=0x7fca10071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:36.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.613+0000 7fca3b200640 1 --2- 172.21.15.86:0/924910171 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fca1006f550 0x7fca10071a60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fca34072200 tx=0x7fca24008040 comp rx=0 tx=0).stop 2024-04-25T14:11:36.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.613+0000 7fca3b200640 1 -- 172.21.15.86:0/924910171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca341fb580 msgr2=0x7fca341fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:36.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.613+0000 7fca3b200640 1 --2- 172.21.15.86:0/924910171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca341fb580 0x7fca341fb990 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fca340cc200 tx=0x7fca2c00e530 comp rx=0 tx=0).stop 2024-04-25T14:11:36.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.613+0000 7fca3b200640 1 -- 172.21.15.86:0/924910171 shutdown_connections 2024-04-25T14:11:36.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.613+0000 7fca3b200640 1 --2- 172.21.15.86:0/924910171 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fca1006f550 0x7fca10071a60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.613+0000 7fca3b200640 1 --2- 172.21.15.86:0/924910171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca341fb580 0x7fca341fb990 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.613+0000 7fca3b200640 1 --2- 172.21.15.86:0/924910171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca34072880 0x7fca341583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.613+0000 7fca3b200640 1 --2- 172.21.15.86:0/924910171 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca18004810 0x7fca341f7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.613+0000 7fca3b200640 1 -- 172.21.15.86:0/924910171 >> 172.21.15.86:0/924910171 conn(0x7fca340b4650 msgr2=0x7fca34155180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:36.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.613+0000 7fca3b200640 1 -- 172.21.15.86:0/924910171 shutdown_connections 2024-04-25T14:11:36.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.613+0000 7fca3b200640 1 -- 172.21.15.86:0/924910171 wait complete. 2024-04-25T14:11:36.633 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T14:11:36.633 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:11:36.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.785+0000 7fa84bc00640 1 Processor -- start 2024-04-25T14:11:36.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.785+0000 7fa84bc00640 1 -- start start 2024-04-25T14:11:36.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.789+0000 7fa84bc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa8440cdda0 0x7fa8440cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:36.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.789+0000 7fa84bc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8440d18e0 0x7fa8440d1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:36.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.789+0000 7fa84bc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa844163940 con 0x7fa8440cec60 2024-04-25T14:11:36.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.789+0000 7fa84bc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa8440cfcc0 con 0x7fa8440cdda0 2024-04-25T14:11:36.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.789+0000 7fa84bc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa8440d2f40 con 0x7fa8440d18e0 2024-04-25T14:11:36.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.789+0000 7fa849200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa8440cec60 0x7fa8440cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60454/0 (socket says 172.21.15.86:60454) 2024-04-25T14:11:36.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.789+0000 7fa849200640 1 -- 172.21.15.86:0/2783067556 learned_addr learned my addr 172.21.15.86:0/2783067556 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:36.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.789+0000 7fa843e00640 1 --2- 172.21.15.86:0/2783067556 >> v2:172.21.15.86:3300/0 conn(0x7fa8440cdda0 0x7fa8440cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:36.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.789+0000 7fa849c00640 1 --2- 172.21.15.86:0/2783067556 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8440d18e0 0x7fa8440d1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:36.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.789+0000 7fa843400640 1 -- 172.21.15.86:0/2783067556 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3630322298 0 0) 0x7fa844163940 con 0x7fa8440cec60 2024-04-25T14:11:36.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.789+0000 7fa843400640 1 -- 172.21.15.86:0/2783067556 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa82c003530 con 0x7fa8440cec60 2024-04-25T14:11:36.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.789+0000 7fa843e00640 1 -- 172.21.15.86:0/2783067556 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8440d18e0 msgr2=0x7fa8440d1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:36.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.789+0000 7fa843e00640 1 --2- 172.21.15.86:0/2783067556 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8440d18e0 0x7fa8440d1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.789+0000 7fa843e00640 1 -- 172.21.15.86:0/2783067556 >> v1:172.21.15.86:6789/0 conn(0x7fa8440cec60 legacy=0x7fa8440cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:36.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.789+0000 7fa843e00640 1 -- 172.21.15.86:0/2783067556 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8440b3720 con 0x7fa8440cdda0 2024-04-25T14:11:36.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.793+0000 7fa843e00640 1 --2- 172.21.15.86:0/2783067556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8440cdda0 0x7fa8440cf780 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fa8300097c0 tx=0x7fa83001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2ea76ac52bfabe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:36.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.793+0000 7fa843400640 1 -- 172.21.15.86:0/2783067556 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8300b0070 con 0x7fa8440cdda0 2024-04-25T14:11:36.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.793+0000 7fa843400640 1 -- 172.21.15.86:0/2783067556 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8300ab600 con 0x7fa8440cdda0 2024-04-25T14:11:36.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.793+0000 7fa843400640 1 -- 172.21.15.86:0/2783067556 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8300ab9d0 con 0x7fa8440cdda0 2024-04-25T14:11:36.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.793+0000 7fa84bc00640 1 -- 172.21.15.86:0/2783067556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8440cdda0 msgr2=0x7fa8440cf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:36.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.793+0000 7fa84bc00640 1 --2- 172.21.15.86:0/2783067556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8440cdda0 0x7fa8440cf780 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fa8300097c0 tx=0x7fa83001c4a0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.793+0000 7fa84bc00640 1 -- 172.21.15.86:0/2783067556 shutdown_connections 2024-04-25T14:11:36.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.793+0000 7fa84bc00640 1 --2- 172.21.15.86:0/2783067556 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8440d18e0 0x7fa8440d1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.793+0000 7fa84bc00640 1 --2- 172.21.15.86:0/2783067556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8440cdda0 0x7fa8440cf780 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.793+0000 7fa84bc00640 1 -- 172.21.15.86:0/2783067556 >> 172.21.15.86:0/2783067556 conn(0x7fa84409ba30 msgr2=0x7fa8440b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:36.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.793+0000 7fa84bc00640 1 -- 172.21.15.86:0/2783067556 shutdown_connections 2024-04-25T14:11:36.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.793+0000 7fa84bc00640 1 -- 172.21.15.86:0/2783067556 wait complete. 2024-04-25T14:11:36.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.797+0000 7fa84bc00640 1 Processor -- start 2024-04-25T14:11:36.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.797+0000 7fa84bc00640 1 -- start start 2024-04-25T14:11:36.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.797+0000 7fa84bc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8440cdda0 0x7fa844147680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:36.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.797+0000 7fa84bc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8440cec60 0x7fa84406fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:36.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.797+0000 7fa84bc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8440d18e0 0x7fa844070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:36.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.797+0000 7fa84bc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa84416cbf0 con 0x7fa8440cec60 2024-04-25T14:11:36.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.797+0000 7fa84bc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa8440d2c50 con 0x7fa8440cdda0 2024-04-25T14:11:36.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.797+0000 7fa84bc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa84416cd70 con 0x7fa8440d18e0 2024-04-25T14:11:36.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.797+0000 7fa843e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8440cec60 0x7fa84406fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:36.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.797+0000 7fa843e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8440cec60 0x7fa84406fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40952/0 (socket says 172.21.15.86:40952) 2024-04-25T14:11:36.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.797+0000 7fa843e00640 1 -- 172.21.15.86:0/1783942659 learned_addr learned my addr 172.21.15.86:0/1783942659 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:36.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.797+0000 7fa849200640 1 --2- 172.21.15.86:0/1783942659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8440cdda0 0x7fa844147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:36.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.797+0000 7fa849c00640 1 --2- 172.21.15.86:0/1783942659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8440d18e0 0x7fa844070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:36.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.797+0000 7fa843e00640 1 -- 172.21.15.86:0/1783942659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8440d18e0 msgr2=0x7fa844070040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:36.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.797+0000 7fa843e00640 1 --2- 172.21.15.86:0/1783942659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8440d18e0 0x7fa844070040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.797+0000 7fa843e00640 1 -- 172.21.15.86:0/1783942659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8440cdda0 msgr2=0x7fa844147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:36.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.797+0000 7fa843e00640 1 --2- 172.21.15.86:0/1783942659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8440cdda0 0x7fa844147680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.797+0000 7fa843e00640 1 -- 172.21.15.86:0/1783942659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa844160f00 con 0x7fa8440cec60 2024-04-25T14:11:36.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.797+0000 7fa849c00640 1 --2- 172.21.15.86:0/1783942659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8440d18e0 0x7fa844070040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:36.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.797+0000 7fa843e00640 1 --2- 172.21.15.86:0/1783942659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8440cec60 0x7fa84406fb00 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fa8300ae040 tx=0x7fa830019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:36.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.797+0000 7fa840c00640 1 -- 172.21.15.86:0/1783942659 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8300b0040 con 0x7fa8440cec60 2024-04-25T14:11:36.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.797+0000 7fa840c00640 1 -- 172.21.15.86:0/1783942659 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8300b9070 con 0x7fa8440cec60 2024-04-25T14:11:36.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.797+0000 7fa84bc00640 1 -- 172.21.15.86:0/1783942659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8441611e0 con 0x7fa8440cec60 2024-04-25T14:11:36.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.797+0000 7fa840c00640 1 -- 172.21.15.86:0/1783942659 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8300b4670 con 0x7fa8440cec60 2024-04-25T14:11:36.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.797+0000 7fa84bc00640 1 -- 172.21.15.86:0/1783942659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8441617a0 con 0x7fa8440cec60 2024-04-25T14:11:36.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.801+0000 7fa84bc00640 1 -- 172.21.15.86:0/1783942659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8440a04c0 con 0x7fa8440cec60 2024-04-25T14:11:36.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.801+0000 7fa840c00640 1 -- 172.21.15.86:0/1783942659 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7fa8300b4810 con 0x7fa8440cec60 2024-04-25T14:11:36.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.801+0000 7fa840c00640 1 --2- 172.21.15.86:0/1783942659 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa82406f550 0x7fa824071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:36.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.801+0000 7fa840c00640 1 -- 172.21.15.86:0/1783942659 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7fa830129360 con 0x7fa8440cec60 2024-04-25T14:11:36.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.801+0000 7fa849200640 1 --2- 172.21.15.86:0/1783942659 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa82406f550 0x7fa824071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:36.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.801+0000 7fa849200640 1 --2- 172.21.15.86:0/1783942659 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa82406f550 0x7fa824071a60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa838001370 tx=0x7fa838009040 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:36.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.805+0000 7fa840c00640 1 -- 172.21.15.86:0/1783942659 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa830016090 con 0x7fa8440cec60 2024-04-25T14:11:36.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.993+0000 7fa84bc00640 1 -- 172.21.15.86:0/1783942659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa844076bf0 con 0x7fa8440cec60 2024-04-25T14:11:36.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.993+0000 7fa840c00640 1 -- 172.21.15.86:0/1783942659 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5187 (secure 0 0 0) 0x7fa8300f5e90 con 0x7fa8440cec60 2024-04-25T14:11:36.991 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:36.991 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":103,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/1828317917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1828317917},{"type":"v1","addr":"172.21.15.155:6834","nonce":1828317917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":9124,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1389714390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1389714390},{"type":"v1","addr":"172.21.15.86:6837","nonce":1389714390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":9136,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/91793677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":91793677},{"type":"v1","addr":"172.21.15.155:6835","nonce":91793677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:11:32.981538+0000","modified":"2024-04-25T14:11:35.996552+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9546},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9546":{"gid":9546,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/2571655765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2571655765},{"type":"v1","addr":"172.21.15.86:6835","nonce":2571655765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9546,"qdb_cluster":[9546]},"id":10}]} 2024-04-25T14:11:36.991 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 103 2024-04-25T14:11:36.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.997+0000 7fa84bc00640 1 -- 172.21.15.86:0/1783942659 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa82406f550 msgr2=0x7fa824071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:36.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.997+0000 7fa84bc00640 1 --2- 172.21.15.86:0/1783942659 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa82406f550 0x7fa824071a60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa838001370 tx=0x7fa838009040 comp rx=0 tx=0).stop 2024-04-25T14:11:36.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.997+0000 7fa84bc00640 1 -- 172.21.15.86:0/1783942659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8440cec60 msgr2=0x7fa84406fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:36.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.997+0000 7fa84bc00640 1 --2- 172.21.15.86:0/1783942659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8440cec60 0x7fa84406fb00 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fa8300ae040 tx=0x7fa830019240 comp rx=0 tx=0).stop 2024-04-25T14:11:36.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.997+0000 7fa84bc00640 1 -- 172.21.15.86:0/1783942659 shutdown_connections 2024-04-25T14:11:36.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.997+0000 7fa84bc00640 1 --2- 172.21.15.86:0/1783942659 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa82406f550 0x7fa824071a60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.997+0000 7fa84bc00640 1 --2- 172.21.15.86:0/1783942659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8440d18e0 0x7fa844070040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.997+0000 7fa84bc00640 1 --2- 172.21.15.86:0/1783942659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8440cec60 0x7fa84406fb00 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.997+0000 7fa84bc00640 1 --2- 172.21.15.86:0/1783942659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8440cdda0 0x7fa844147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:36.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.997+0000 7fa84bc00640 1 -- 172.21.15.86:0/1783942659 >> 172.21.15.86:0/1783942659 conn(0x7fa84409ba30 msgr2=0x7fa8440ccc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:36.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.997+0000 7fa84bc00640 1 -- 172.21.15.86:0/1783942659 shutdown_connections 2024-04-25T14:11:36.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:36.997+0000 7fa84bc00640 1 -- 172.21.15.86:0/1783942659 wait complete. 2024-04-25T14:11:37.013 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:11:37.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.161+0000 7fcb7ec00640 1 Processor -- start 2024-04-25T14:11:37.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.161+0000 7fcb7ec00640 1 -- start start 2024-04-25T14:11:37.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.165+0000 7fcb7ec00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcb7815c2a0 0x7fcb780ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:37.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.165+0000 7fcb7ec00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcb7815feb0 0x7fcb781602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:37.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.165+0000 7fcb7ec00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb780b0a60 con 0x7fcb7806fd50 2024-04-25T14:11:37.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.165+0000 7fcb7ec00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcb7806ef90 con 0x7fcb7815c2a0 2024-04-25T14:11:37.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.165+0000 7fcb7ec00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fcb7806f270 con 0x7fcb7815feb0 2024-04-25T14:11:37.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.165+0000 7fcb6f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fcb7806fd50 0x7fcb78070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60462/0 (socket says 172.21.15.86:60462) 2024-04-25T14:11:37.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.165+0000 7fcb77e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcb7815c2a0 0x7fcb780ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:37.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.165+0000 7fcb6f400640 1 -- 172.21.15.86:0/3303733494 learned_addr learned my addr 172.21.15.86:0/3303733494 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:37.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.165+0000 7fcb7cc00640 1 --2- 172.21.15.86:0/3303733494 >> v2:172.21.15.155:3300/0 conn(0x7fcb7815feb0 0x7fcb781602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:37.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.165+0000 7fcb77e00640 1 -- 172.21.15.86:0/3303733494 >> v2:172.21.15.155:3300/0 conn(0x7fcb7815feb0 msgr2=0x7fcb781602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:37.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.165+0000 7fcb77e00640 1 --2- 172.21.15.86:0/3303733494 >> v2:172.21.15.155:3300/0 conn(0x7fcb7815feb0 0x7fcb781602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:37.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.165+0000 7fcb77e00640 1 -- 172.21.15.86:0/3303733494 >> v1:172.21.15.86:6789/0 conn(0x7fcb7806fd50 legacy=0x7fcb78070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:37.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.165+0000 7fcb77e00640 1 -- 172.21.15.86:0/3303733494 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb7806f550 con 0x7fcb7815c2a0 2024-04-25T14:11:37.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.165+0000 7fcb7cc00640 1 --2- 172.21.15.86:0/3303733494 >> v2:172.21.15.155:3300/0 conn(0x7fcb7815feb0 0x7fcb781602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:37.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.165+0000 7fcb77400640 1 -- 172.21.15.86:0/3303733494 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 662205118 0 0) 0x7fcb780b0a60 con 0x7fcb7806fd50 2024-04-25T14:11:37.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.169+0000 7fcb77e00640 1 --2- 172.21.15.86:0/3303733494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb7815c2a0 0x7fcb780ae130 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fcb780708a0 tx=0x7fcb6801ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cafb8bd26546830c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:37.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.169+0000 7fcb77400640 1 -- 172.21.15.86:0/3303733494 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb680b2070 con 0x7fcb7815c2a0 2024-04-25T14:11:37.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.169+0000 7fcb77400640 1 -- 172.21.15.86:0/3303733494 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb680a9070 con 0x7fcb7815c2a0 2024-04-25T14:11:37.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.169+0000 7fcb77400640 1 -- 172.21.15.86:0/3303733494 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb680ad880 con 0x7fcb7815c2a0 2024-04-25T14:11:37.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.169+0000 7fcb7ec00640 1 -- 172.21.15.86:0/3303733494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb7815c2a0 msgr2=0x7fcb780ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:37.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.169+0000 7fcb7ec00640 1 --2- 172.21.15.86:0/3303733494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb7815c2a0 0x7fcb780ae130 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fcb780708a0 tx=0x7fcb6801ce20 comp rx=0 tx=0).stop 2024-04-25T14:11:37.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.169+0000 7fcb7ec00640 1 -- 172.21.15.86:0/3303733494 shutdown_connections 2024-04-25T14:11:37.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.169+0000 7fcb7ec00640 1 --2- 172.21.15.86:0/3303733494 >> v2:172.21.15.155:3300/0 conn(0x7fcb7815feb0 0x7fcb781602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:37.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.169+0000 7fcb7ec00640 1 --2- 172.21.15.86:0/3303733494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb7815c2a0 0x7fcb780ae130 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:37.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.169+0000 7fcb7ec00640 1 -- 172.21.15.86:0/3303733494 >> 172.21.15.86:0/3303733494 conn(0x7fcb780b4650 msgr2=0x7fcb780b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:37.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.169+0000 7fcb7ec00640 1 -- 172.21.15.86:0/3303733494 shutdown_connections 2024-04-25T14:11:37.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.169+0000 7fcb7ec00640 1 -- 172.21.15.86:0/3303733494 wait complete. 2024-04-25T14:11:37.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.169+0000 7fcb7ec00640 1 Processor -- start 2024-04-25T14:11:37.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.169+0000 7fcb7ec00640 1 -- start start 2024-04-25T14:11:37.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.173+0000 7fcb7ec00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb7806fd50 0x7fcb7815ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:37.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.173+0000 7fcb7ec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb7815c2a0 0x7fcb781533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:37.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.173+0000 7fcb7ec00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb7815feb0 0x7fcb7809a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:37.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.173+0000 7fcb7ec00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcb78160d90 con 0x7fcb7815c2a0 2024-04-25T14:11:37.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.173+0000 7fcb7ec00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcb78160c10 con 0x7fcb7815feb0 2024-04-25T14:11:37.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.173+0000 7fcb7ec00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcb780ae670 con 0x7fcb7806fd50 2024-04-25T14:11:37.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.173+0000 7fcb6f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb7815c2a0 0x7fcb781533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:37.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.173+0000 7fcb6f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb7815c2a0 0x7fcb781533a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40968/0 (socket says 172.21.15.86:40968) 2024-04-25T14:11:37.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.173+0000 7fcb6f400640 1 -- 172.21.15.86:0/1898362073 learned_addr learned my addr 172.21.15.86:0/1898362073 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:37.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.173+0000 7fcb7cc00640 1 --2- 172.21.15.86:0/1898362073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb7815feb0 0x7fcb7809a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:37.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.173+0000 7fcb77e00640 1 --2- 172.21.15.86:0/1898362073 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb7806fd50 0x7fcb7815ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:37.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.173+0000 7fcb6f400640 1 -- 172.21.15.86:0/1898362073 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb7806fd50 msgr2=0x7fcb7815ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:37.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.173+0000 7fcb6f400640 1 --2- 172.21.15.86:0/1898362073 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb7806fd50 0x7fcb7815ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:37.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.173+0000 7fcb6f400640 1 -- 172.21.15.86:0/1898362073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb7815feb0 msgr2=0x7fcb7809a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:37.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.173+0000 7fcb6f400640 1 --2- 172.21.15.86:0/1898362073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb7815feb0 0x7fcb7809a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:37.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.173+0000 7fcb6f400640 1 -- 172.21.15.86:0/1898362073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb780ae3a0 con 0x7fcb7815c2a0 2024-04-25T14:11:37.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.173+0000 7fcb6f400640 1 --2- 172.21.15.86:0/1898362073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb7815c2a0 0x7fcb781533a0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000c450 tx=0x7fcb6000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:37.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.173+0000 7fcb74c00640 1 -- 172.21.15.86:0/1898362073 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb60018070 con 0x7fcb7815c2a0 2024-04-25T14:11:37.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.173+0000 7fcb74c00640 1 -- 172.21.15.86:0/1898362073 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb600074e0 con 0x7fcb7815c2a0 2024-04-25T14:11:37.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.173+0000 7fcb7ec00640 1 -- 172.21.15.86:0/1898362073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb7815f750 con 0x7fcb7815c2a0 2024-04-25T14:11:37.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.173+0000 7fcb74c00640 1 -- 172.21.15.86:0/1898362073 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb600049f0 con 0x7fcb7815c2a0 2024-04-25T14:11:37.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.173+0000 7fcb7ec00640 1 -- 172.21.15.86:0/1898362073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb7815f1a0 con 0x7fcb7815c2a0 2024-04-25T14:11:37.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.173+0000 7fcb7ec00640 1 -- 172.21.15.86:0/1898362073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb78069d00 con 0x7fcb7815c2a0 2024-04-25T14:11:37.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.177+0000 7fcb74c00640 1 -- 172.21.15.86:0/1898362073 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7fcb6001b070 con 0x7fcb7815c2a0 2024-04-25T14:11:37.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.177+0000 7fcb74c00640 1 --2- 172.21.15.86:0/1898362073 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fcb5406f550 0x7fcb54071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:37.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.177+0000 7fcb74c00640 1 -- 172.21.15.86:0/1898362073 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7fcb6008e910 con 0x7fcb7815c2a0 2024-04-25T14:11:37.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.177+0000 7fcb77e00640 1 --2- 172.21.15.86:0/1898362073 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fcb5406f550 0x7fcb54071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:37.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.177+0000 7fcb77e00640 1 --2- 172.21.15.86:0/1898362073 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fcb5406f550 0x7fcb54071a60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcb680af000 tx=0x7fcb68008040 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:37.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.181+0000 7fcb74c00640 1 -- 172.21.15.86:0/1898362073 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcb6005b550 con 0x7fcb7815c2a0 2024-04-25T14:11:37.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.341+0000 7fcb7ec00640 1 -- 172.21.15.86:0/1898362073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb78076bf0 con 0x7fcb7815c2a0 2024-04-25T14:11:37.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.341+0000 7fcb74c00640 1 -- 172.21.15.86:0/1898362073 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) ==== 74+0+16892 (secure 0 0 0) 0x7fcb6005ae50 con 0x7fcb7815c2a0 2024-04-25T14:11:37.339 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:37.339 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":140,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:11:34.592046+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:11:30.810321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T14:11:31.944888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:11:37.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.345+0000 7fcb7ec00640 1 -- 172.21.15.86:0/1898362073 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fcb5406f550 msgr2=0x7fcb54071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:37.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.345+0000 7fcb7ec00640 1 --2- 172.21.15.86:0/1898362073 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fcb5406f550 0x7fcb54071a60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcb680af000 tx=0x7fcb68008040 comp rx=0 tx=0).stop 2024-04-25T14:11:37.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.345+0000 7fcb7ec00640 1 -- 172.21.15.86:0/1898362073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb7815c2a0 msgr2=0x7fcb781533a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:37.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.345+0000 7fcb7ec00640 1 --2- 172.21.15.86:0/1898362073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb7815c2a0 0x7fcb781533a0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000c450 tx=0x7fcb6000c910 comp rx=0 tx=0).stop 2024-04-25T14:11:37.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.345+0000 7fcb7ec00640 1 -- 172.21.15.86:0/1898362073 shutdown_connections 2024-04-25T14:11:37.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.345+0000 7fcb7ec00640 1 --2- 172.21.15.86:0/1898362073 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fcb5406f550 0x7fcb54071a60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:37.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.345+0000 7fcb7ec00640 1 --2- 172.21.15.86:0/1898362073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb7815feb0 0x7fcb7809a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:37.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.345+0000 7fcb7ec00640 1 --2- 172.21.15.86:0/1898362073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb7815c2a0 0x7fcb781533a0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:37.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.345+0000 7fcb7ec00640 1 --2- 172.21.15.86:0/1898362073 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb7806fd50 0x7fcb7815ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:37.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.345+0000 7fcb7ec00640 1 -- 172.21.15.86:0/1898362073 >> 172.21.15.86:0/1898362073 conn(0x7fcb780b4650 msgr2=0x7fcb7816b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:37.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.345+0000 7fcb7ec00640 1 -- 172.21.15.86:0/1898362073 shutdown_connections 2024-04-25T14:11:37.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.345+0000 7fcb7ec00640 1 -- 172.21.15.86:0/1898362073 wait complete. 2024-04-25T14:11:37.361 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T14:11:37.361 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:11:37.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.513+0000 7f2769400640 1 Processor -- start 2024-04-25T14:11:37.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.513+0000 7f2769400640 1 -- start start 2024-04-25T14:11:37.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2769400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f27640cfca0 0x7f276406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:37.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2769400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2764072bd0 con 0x7f27640d16e0 2024-04-25T14:11:37.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2769400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f276406f930 con 0x7f27640730b0 2024-04-25T14:11:37.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2769400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f2764070470 con 0x7f27640cfca0 2024-04-25T14:11:37.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2761e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f27640d16e0 0x7f27640d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60466/0 (socket says 172.21.15.86:60466) 2024-04-25T14:11:37.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2761e00640 1 -- 172.21.15.86:0/593633485 learned_addr learned my addr 172.21.15.86:0/593633485 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:37.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2762800640 1 --2- 172.21.15.86:0/593633485 >> v2:172.21.15.155:3300/0 conn(0x7f27640cfca0 0x7f276406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:37.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2761400640 1 -- 172.21.15.86:0/593633485 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1584168993 0 0) 0x7f2764072bd0 con 0x7f27640d16e0 2024-04-25T14:11:37.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2761400640 1 -- 172.21.15.86:0/593633485 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2748003530 con 0x7f27640d16e0 2024-04-25T14:11:37.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2761400640 1 -- 172.21.15.86:0/593633485 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 932109966 0 0) 0x7f2748003530 con 0x7f27640d16e0 2024-04-25T14:11:37.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2761400640 1 -- 172.21.15.86:0/593633485 >> v2:172.21.15.155:3300/0 conn(0x7f27640cfca0 msgr2=0x7f276406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:37.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2761400640 1 --2- 172.21.15.86:0/593633485 >> v2:172.21.15.155:3300/0 conn(0x7f27640cfca0 0x7f276406f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:37.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2761400640 1 -- 172.21.15.86:0/593633485 >> v1:172.21.15.155:6789/0 conn(0x7f27640730b0 legacy=0x7f27640734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:37.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2761400640 1 -- 172.21.15.86:0/593633485 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276406fbd0 con 0x7f27640d16e0 2024-04-25T14:11:37.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2761400640 1 -- 172.21.15.86:0/593633485 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f2750001680 con 0x7f27640d16e0 2024-04-25T14:11:37.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2761400640 1 -- 172.21.15.86:0/593633485 >> v1:172.21.15.86:6789/0 conn(0x7f27640d16e0 legacy=0x7f27640d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:37.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2761400640 1 --2- 172.21.15.86:0/593633485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2748003a30 0x7f2748003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:37.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2761400640 1 --2- 172.21.15.86:0/593633485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2748004810 0x7f2748004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:37.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2761400640 1 --2- 172.21.15.86:0/593633485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2748005230 0x7f274800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:37.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2761400640 1 -- 172.21.15.86:0/593633485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2764070470 con 0x7f2748004810 2024-04-25T14:11:37.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2761400640 1 -- 172.21.15.86:0/593633485 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f274800c1c0 con 0x7f2748003a30 2024-04-25T14:11:37.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2761400640 1 -- 172.21.15.86:0/593633485 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f274800c3c0 con 0x7f2748005230 2024-04-25T14:11:37.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2762800640 1 --2- 172.21.15.86:0/593633485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2748003a30 0x7f2748003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:37.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2761e00640 1 --2- 172.21.15.86:0/593633485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2748004810 0x7f2748004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:37.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2763200640 1 --2- 172.21.15.86:0/593633485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2748005230 0x7f274800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:37.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2761e00640 1 -- 172.21.15.86:0/593633485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2748005230 msgr2=0x7f274800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:37.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2761e00640 1 --2- 172.21.15.86:0/593633485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2748005230 0x7f274800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:37.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2761e00640 1 -- 172.21.15.86:0/593633485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2748003a30 msgr2=0x7f2748003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:37.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2761e00640 1 --2- 172.21.15.86:0/593633485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2748003a30 0x7f2748003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:37.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2761e00640 1 -- 172.21.15.86:0/593633485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2748003530 con 0x7f2748004810 2024-04-25T14:11:37.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2762800640 1 --2- 172.21.15.86:0/593633485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2748003a30 0x7f2748003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:11:37.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.517+0000 7f2763200640 1 -- 172.21.15.86:0/593633485 reap_dead start 2024-04-25T14:11:37.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.521+0000 7f2761e00640 1 --2- 172.21.15.86:0/593633485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2748004810 0x7f2748004c20 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f275000c210 tx=0x7f275001f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764d6a794fdeb537 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:37.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.521+0000 7f2761400640 1 -- 172.21.15.86:0/593633485 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2750007d20 con 0x7f2748004810 2024-04-25T14:11:37.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.521+0000 7f2761400640 1 -- 172.21.15.86:0/593633485 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2750005ae0 con 0x7f2748004810 2024-04-25T14:11:37.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.521+0000 7f2761400640 1 -- 172.21.15.86:0/593633485 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2750007070 con 0x7f2748004810 2024-04-25T14:11:37.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.521+0000 7f2769400640 1 -- 172.21.15.86:0/593633485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2748004810 msgr2=0x7f2748004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:37.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.521+0000 7f2769400640 1 --2- 172.21.15.86:0/593633485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2748004810 0x7f2748004c20 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f275000c210 tx=0x7f275001f370 comp rx=0 tx=0).stop 2024-04-25T14:11:37.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.521+0000 7f2769400640 1 -- 172.21.15.86:0/593633485 shutdown_connections 2024-04-25T14:11:37.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.521+0000 7f2769400640 1 --2- 172.21.15.86:0/593633485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2748004810 0x7f2748004c20 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:37.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.521+0000 7f2769400640 1 -- 172.21.15.86:0/593633485 >> 172.21.15.86:0/593633485 conn(0x7f27640b4650 msgr2=0x7f27640b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:37.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.521+0000 7f2769400640 1 -- 172.21.15.86:0/593633485 shutdown_connections 2024-04-25T14:11:37.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.521+0000 7f2769400640 1 -- 172.21.15.86:0/593633485 wait complete. 2024-04-25T14:11:37.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.525+0000 7f2769400640 1 Processor -- start 2024-04-25T14:11:37.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.525+0000 7f2769400640 1 -- start start 2024-04-25T14:11:37.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.525+0000 7f2769400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2748004810 0x7f27641f4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:37.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.525+0000 7f2769400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27641f5310 0x7f27641f3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:37.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.525+0000 7f2769400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2764159d00 0x7f276415a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:37.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.525+0000 7f2769400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f276406f3f0 con 0x7f2764159d00 2024-04-25T14:11:37.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.525+0000 7f2769400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2764070190 con 0x7f27641f5310 2024-04-25T14:11:37.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.525+0000 7f2769400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f276406f570 con 0x7f2748004810 2024-04-25T14:11:37.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.525+0000 7f2763200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2764159d00 0x7f276415a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:37.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.525+0000 7f2763200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2764159d00 0x7f276415a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40976/0 (socket says 172.21.15.86:40976) 2024-04-25T14:11:37.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.525+0000 7f2763200640 1 -- 172.21.15.86:0/403672771 learned_addr learned my addr 172.21.15.86:0/403672771 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:37.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.525+0000 7f2761e00640 1 --2- 172.21.15.86:0/403672771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27641f5310 0x7f27641f3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:37.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.525+0000 7f2762800640 1 --2- 172.21.15.86:0/403672771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2748004810 0x7f27641f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:37.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.525+0000 7f2763200640 1 -- 172.21.15.86:0/403672771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2748004810 msgr2=0x7f27641f4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:37.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.525+0000 7f2763200640 1 --2- 172.21.15.86:0/403672771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2748004810 0x7f27641f4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:37.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.525+0000 7f2763200640 1 -- 172.21.15.86:0/403672771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27641f5310 msgr2=0x7f27641f3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:37.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.525+0000 7f2763200640 1 --2- 172.21.15.86:0/403672771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27641f5310 0x7f27641f3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:37.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.525+0000 7f2763200640 1 -- 172.21.15.86:0/403672771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276415a650 con 0x7f2764159d00 2024-04-25T14:11:37.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.525+0000 7f2761e00640 1 --2- 172.21.15.86:0/403672771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27641f5310 0x7f27641f3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:37.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.525+0000 7f2763200640 1 --2- 172.21.15.86:0/403672771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2764159d00 0x7f276415a110 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f276406dd30 tx=0x7f275c00fd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:37.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.525+0000 7f275aa00640 1 -- 172.21.15.86:0/403672771 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c003bb0 con 0x7f2764159d00 2024-04-25T14:11:37.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.525+0000 7f275aa00640 1 -- 172.21.15.86:0/403672771 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f275c018070 con 0x7f2764159d00 2024-04-25T14:11:37.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.525+0000 7f275aa00640 1 -- 172.21.15.86:0/403672771 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c0050d0 con 0x7f2764159d00 2024-04-25T14:11:37.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.525+0000 7f2769400640 1 -- 172.21.15.86:0/403672771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27641f58b0 con 0x7f2764159d00 2024-04-25T14:11:37.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.525+0000 7f2769400640 1 -- 172.21.15.86:0/403672771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27641f5e70 con 0x7f2764159d00 2024-04-25T14:11:37.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.529+0000 7f2769400640 1 -- 172.21.15.86:0/403672771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f276409ee30 con 0x7f2764159d00 2024-04-25T14:11:37.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.529+0000 7f275aa00640 1 -- 172.21.15.86:0/403672771 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7f275c005270 con 0x7f2764159d00 2024-04-25T14:11:37.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.529+0000 7f275aa00640 1 --2- 172.21.15.86:0/403672771 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f274406f520 0x7f2744071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:37.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.529+0000 7f275aa00640 1 -- 172.21.15.86:0/403672771 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f275c08f6b0 con 0x7f2764159d00 2024-04-25T14:11:37.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.529+0000 7f2762800640 1 --2- 172.21.15.86:0/403672771 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f274406f520 0x7f2744071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:37.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.529+0000 7f2762800640 1 --2- 172.21.15.86:0/403672771 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f274406f520 0x7f2744071a30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f276406b910 tx=0x7f2754008040 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:37.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.533+0000 7f275aa00640 1 -- 172.21.15.86:0/403672771 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f276415a650 con 0x7f2764159d00 2024-04-25T14:11:37.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.689+0000 7f2769400640 1 -- 172.21.15.86:0/403672771 --> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f276409f4b0 con 0x7f274406f520 2024-04-25T14:11:37.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.697+0000 7f275aa00640 1 -- 172.21.15.86:0/403672771 <== mgr.9282 v2:172.21.15.86:6832/264890479 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247100 (secure 0 0 0) 0x7f276409f4b0 con 0x7f274406f520 2024-04-25T14:11:37.697 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:37.699 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:11:37.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.701+0000 7f2769400640 1 -- 172.21.15.86:0/403672771 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f274406f520 msgr2=0x7f2744071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:37.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.701+0000 7f2769400640 1 --2- 172.21.15.86:0/403672771 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f274406f520 0x7f2744071a30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f276406b910 tx=0x7f2754008040 comp rx=0 tx=0).stop 2024-04-25T14:11:37.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.701+0000 7f2769400640 1 -- 172.21.15.86:0/403672771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2764159d00 msgr2=0x7f276415a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:37.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.701+0000 7f2769400640 1 --2- 172.21.15.86:0/403672771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2764159d00 0x7f276415a110 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f276406dd30 tx=0x7f275c00fd20 comp rx=0 tx=0).stop 2024-04-25T14:11:37.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.701+0000 7f2769400640 1 -- 172.21.15.86:0/403672771 shutdown_connections 2024-04-25T14:11:37.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.701+0000 7f2769400640 1 --2- 172.21.15.86:0/403672771 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f274406f520 0x7f2744071a30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:37.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.701+0000 7f2769400640 1 --2- 172.21.15.86:0/403672771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2764159d00 0x7f276415a110 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:37.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.701+0000 7f2769400640 1 --2- 172.21.15.86:0/403672771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27641f5310 0x7f27641f3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:37.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.701+0000 7f2769400640 1 --2- 172.21.15.86:0/403672771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2748004810 0x7f27641f4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:37.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.701+0000 7f2769400640 1 -- 172.21.15.86:0/403672771 >> 172.21.15.86:0/403672771 conn(0x7f27640b4650 msgr2=0x7f27640cafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:37.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.701+0000 7f2769400640 1 -- 172.21.15.86:0/403672771 shutdown_connections 2024-04-25T14:11:37.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.701+0000 7f2769400640 1 -- 172.21.15.86:0/403672771 wait complete. 2024-04-25T14:11:37.718 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T14:11:36.734296+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":717,"num_read_kb":1927,"num_write":223,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3277,"ondisk_log_size":3277,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":255768,"kb_used_data":14636,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754718952,"statfs":{"total":773094113280,"available":772832206848,"internally_reserved":0,"allocated":14987264,"data_stored":8077777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.166599"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.583286+0000","last_change":"2024-04-25T14:11:32.583286+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.583286+0000","last_undegraded":"2024-04-25T14:11:32.583286+0000","last_fullsized":"2024-04-25T14:11:32.583286+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.593406+0000","last_change":"2024-04-25T14:11:32.593406+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.593406+0000","last_undegraded":"2024-04-25T14:11:32.593406+0000","last_fullsized":"2024-04-25T14:11:32.593406+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.597461+0000","last_change":"2024-04-25T14:11:32.597461+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.597461+0000","last_undegraded":"2024-04-25T14:11:32.597461+0000","last_fullsized":"2024-04-25T14:11:32.597461+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.595145+0000","last_change":"2024-04-25T14:11:32.595145+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.595145+0000","last_undegraded":"2024-04-25T14:11:32.595145+0000","last_fullsized":"2024-04-25T14:11:32.595145+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.590150+0000","last_change":"2024-04-25T14:11:32.590150+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.590150+0000","last_undegraded":"2024-04-25T14:11:32.590150+0000","last_fullsized":"2024-04-25T14:11:32.590150+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.588606+0000","last_change":"2024-04-25T14:11:32.588606+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.588606+0000","last_undegraded":"2024-04-25T14:11:32.588606+0000","last_fullsized":"2024-04-25T14:11:32.588606+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.582342+0000","last_change":"2024-04-25T14:11:32.582342+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.582342+0000","last_undegraded":"2024-04-25T14:11:32.582342+0000","last_fullsized":"2024-04-25T14:11:32.582342+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.583423+0000","last_change":"2024-04-25T14:11:32.583423+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.583423+0000","last_undegraded":"2024-04-25T14:11:32.583423+0000","last_fullsized":"2024-04-25T14:11:32.583423+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.588120+0000","last_change":"2024-04-25T14:11:32.588120+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.588120+0000","last_undegraded":"2024-04-25T14:11:32.588120+0000","last_fullsized":"2024-04-25T14:11:32.588120+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.585963+0000","last_change":"2024-04-25T14:11:32.585963+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.585963+0000","last_undegraded":"2024-04-25T14:11:32.585963+0000","last_fullsized":"2024-04-25T14:11:32.585963+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.588205+0000","last_change":"2024-04-25T14:11:32.588205+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.588205+0000","last_undegraded":"2024-04-25T14:11:32.588205+0000","last_fullsized":"2024-04-25T14:11:32.588205+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.593307+0000","last_change":"2024-04-25T14:11:32.593307+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.593307+0000","last_undegraded":"2024-04-25T14:11:32.593307+0000","last_fullsized":"2024-04-25T14:11:32.593307+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.589718+0000","last_change":"2024-04-25T14:11:32.589718+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.589718+0000","last_undegraded":"2024-04-25T14:11:32.589718+0000","last_fullsized":"2024-04-25T14:11:32.589718+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.596485+0000","last_change":"2024-04-25T14:11:32.596485+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.596485+0000","last_undegraded":"2024-04-25T14:11:32.596485+0000","last_fullsized":"2024-04-25T14:11:32.596485+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.600818+0000","last_change":"2024-04-25T14:11:32.600818+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.600818+0000","last_undegraded":"2024-04-25T14:11:32.600818+0000","last_fullsized":"2024-04-25T14:11:32.600818+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.586259+0000","last_change":"2024-04-25T14:11:32.586259+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.586259+0000","last_undegraded":"2024-04-25T14:11:32.586259+0000","last_fullsized":"2024-04-25T14:11:32.586259+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.594205+0000","last_change":"2024-04-25T14:11:32.594205+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.594205+0000","last_undegraded":"2024-04-25T14:11:32.594205+0000","last_fullsized":"2024-04-25T14:11:32.594205+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.588827+0000","last_change":"2024-04-25T14:11:32.588827+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.588827+0000","last_undegraded":"2024-04-25T14:11:32.588827+0000","last_fullsized":"2024-04-25T14:11:32.588827+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.588488+0000","last_change":"2024-04-25T14:11:32.588488+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.588488+0000","last_undegraded":"2024-04-25T14:11:32.588488+0000","last_fullsized":"2024-04-25T14:11:32.588488+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.585021+0000","last_change":"2024-04-25T14:11:32.585021+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.585021+0000","last_undegraded":"2024-04-25T14:11:32.585021+0000","last_fullsized":"2024-04-25T14:11:32.585021+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.589265+0000","last_change":"2024-04-25T14:11:32.589265+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.589265+0000","last_undegraded":"2024-04-25T14:11:32.589265+0000","last_fullsized":"2024-04-25T14:11:32.589265+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.581795+0000","last_change":"2024-04-25T14:11:32.581795+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.581795+0000","last_undegraded":"2024-04-25T14:11:32.581795+0000","last_fullsized":"2024-04-25T14:11:32.581795+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.583620+0000","last_change":"2024-04-25T14:11:32.583620+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.583620+0000","last_undegraded":"2024-04-25T14:11:32.583620+0000","last_fullsized":"2024-04-25T14:11:32.583620+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.586990+0000","last_change":"2024-04-25T14:11:32.586990+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.586990+0000","last_undegraded":"2024-04-25T14:11:32.586990+0000","last_fullsized":"2024-04-25T14:11:32.586990+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.580738+0000","last_change":"2024-04-25T14:11:32.580738+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.580738+0000","last_undegraded":"2024-04-25T14:11:32.580738+0000","last_fullsized":"2024-04-25T14:11:32.580738+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.581641+0000","last_change":"2024-04-25T14:11:32.581641+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.581641+0000","last_undegraded":"2024-04-25T14:11:32.581641+0000","last_fullsized":"2024-04-25T14:11:32.581641+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.586189+0000","last_change":"2024-04-25T14:11:32.586189+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.586189+0000","last_undegraded":"2024-04-25T14:11:32.586189+0000","last_fullsized":"2024-04-25T14:11:32.586189+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.588712+0000","last_change":"2024-04-25T14:11:32.588712+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.588712+0000","last_undegraded":"2024-04-25T14:11:32.588712+0000","last_fullsized":"2024-04-25T14:11:32.588712+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.582773+0000","last_change":"2024-04-25T14:11:32.582773+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.582773+0000","last_undegraded":"2024-04-25T14:11:32.582773+0000","last_fullsized":"2024-04-25T14:11:32.582773+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.587206+0000","last_change":"2024-04-25T14:11:32.587206+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.587206+0000","last_undegraded":"2024-04-25T14:11:32.587206+0000","last_fullsized":"2024-04-25T14:11:32.587206+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.584884+0000","last_change":"2024-04-25T14:11:32.584884+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.584884+0000","last_undegraded":"2024-04-25T14:11:32.584884+0000","last_fullsized":"2024-04-25T14:11:32.584884+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.582726+0000","last_change":"2024-04-25T14:11:32.582726+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.582726+0000","last_undegraded":"2024-04-25T14:11:32.582726+0000","last_fullsized":"2024-04-25T14:11:32.582726+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.596501+0000","last_change":"2024-04-25T14:11:32.596501+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.596501+0000","last_undegraded":"2024-04-25T14:11:32.596501+0000","last_fullsized":"2024-04-25T14:11:32.596501+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.594681+0000","last_change":"2024-04-25T14:11:32.591452+0000","last_active":"2024-04-25T14:11:32.594681+0000","last_peered":"2024-04-25T14:11:32.594681+0000","last_clean":"2024-04-25T14:11:32.594681+0000","last_became_active":"2024-04-25T14:11:32.591159+0000","last_became_peered":"2024-04-25T14:11:32.591159+0000","last_unstale":"2024-04-25T14:11:32.594681+0000","last_undegraded":"2024-04-25T14:11:32.594681+0000","last_fullsized":"2024-04-25T14:11:32.594681+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:16.063389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.596966+0000","last_change":"2024-04-25T14:11:32.596966+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.596966+0000","last_undegraded":"2024-04-25T14:11:32.596966+0000","last_fullsized":"2024-04-25T14:11:32.596966+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.606856+0000","last_change":"2024-04-25T14:11:32.603965+0000","last_active":"2024-04-25T14:11:32.606856+0000","last_peered":"2024-04-25T14:11:32.606856+0000","last_clean":"2024-04-25T14:11:32.606856+0000","last_became_active":"2024-04-25T14:11:32.602019+0000","last_became_peered":"2024-04-25T14:11:32.602019+0000","last_unstale":"2024-04-25T14:11:32.606856+0000","last_undegraded":"2024-04-25T14:11:32.606856+0000","last_fullsized":"2024-04-25T14:11:32.606856+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:44.182089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.582818+0000","last_change":"2024-04-25T14:11:32.582818+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.582818+0000","last_undegraded":"2024-04-25T14:11:32.582818+0000","last_fullsized":"2024-04-25T14:11:32.582818+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.606798+0000","last_change":"2024-04-25T14:11:32.601614+0000","last_active":"2024-04-25T14:11:32.606798+0000","last_peered":"2024-04-25T14:11:32.606798+0000","last_clean":"2024-04-25T14:11:32.606798+0000","last_became_active":"2024-04-25T14:11:32.601092+0000","last_became_peered":"2024-04-25T14:11:32.601092+0000","last_unstale":"2024-04-25T14:11:32.606798+0000","last_undegraded":"2024-04-25T14:11:32.606798+0000","last_fullsized":"2024-04-25T14:11:32.606798+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:16.063389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.587090+0000","last_change":"2024-04-25T14:11:32.587090+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.587090+0000","last_undegraded":"2024-04-25T14:11:32.587090+0000","last_fullsized":"2024-04-25T14:11:32.587090+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.605321+0000","last_change":"2024-04-25T14:11:32.602711+0000","last_active":"2024-04-25T14:11:32.605321+0000","last_peered":"2024-04-25T14:11:32.605321+0000","last_clean":"2024-04-25T14:11:32.605321+0000","last_became_active":"2024-04-25T14:11:32.602368+0000","last_became_peered":"2024-04-25T14:11:32.602368+0000","last_unstale":"2024-04-25T14:11:32.605321+0000","last_undegraded":"2024-04-25T14:11:32.605321+0000","last_fullsized":"2024-04-25T14:11:32.605321+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:12.830306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.584817+0000","last_change":"2024-04-25T14:11:32.584817+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.584817+0000","last_undegraded":"2024-04-25T14:11:32.584817+0000","last_fullsized":"2024-04-25T14:11:32.584817+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.601151+0000","last_change":"2024-04-25T14:11:32.598778+0000","last_active":"2024-04-25T14:11:32.601151+0000","last_peered":"2024-04-25T14:11:32.601151+0000","last_clean":"2024-04-25T14:11:32.601151+0000","last_became_active":"2024-04-25T14:11:32.598190+0000","last_became_peered":"2024-04-25T14:11:32.598190+0000","last_unstale":"2024-04-25T14:11:32.601151+0000","last_undegraded":"2024-04-25T14:11:32.601151+0000","last_fullsized":"2024-04-25T14:11:32.601151+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:55.757498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.592800+0000","last_change":"2024-04-25T14:11:32.592800+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.592800+0000","last_undegraded":"2024-04-25T14:11:32.592800+0000","last_fullsized":"2024-04-25T14:11:32.592800+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.605858+0000","last_change":"2024-04-25T14:11:32.605097+0000","last_active":"2024-04-25T14:11:32.605858+0000","last_peered":"2024-04-25T14:11:32.605858+0000","last_clean":"2024-04-25T14:11:32.605858+0000","last_became_active":"2024-04-25T14:11:32.604770+0000","last_became_peered":"2024-04-25T14:11:32.604770+0000","last_unstale":"2024-04-25T14:11:32.605858+0000","last_undegraded":"2024-04-25T14:11:32.605858+0000","last_fullsized":"2024-04-25T14:11:32.605858+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:44.182089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.600464+0000","last_change":"2024-04-25T14:11:32.600464+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.600464+0000","last_undegraded":"2024-04-25T14:11:32.600464+0000","last_fullsized":"2024-04-25T14:11:32.600464+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.606597+0000","last_change":"2024-04-25T14:11:32.605101+0000","last_active":"2024-04-25T14:11:32.606597+0000","last_peered":"2024-04-25T14:11:32.606597+0000","last_clean":"2024-04-25T14:11:32.606597+0000","last_became_active":"2024-04-25T14:11:32.604386+0000","last_became_peered":"2024-04-25T14:11:32.604386+0000","last_unstale":"2024-04-25T14:11:32.606597+0000","last_undegraded":"2024-04-25T14:11:32.606597+0000","last_fullsized":"2024-04-25T14:11:32.606597+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:00.011427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.583992+0000","last_change":"2024-04-25T14:11:32.583992+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.583992+0000","last_undegraded":"2024-04-25T14:11:32.583992+0000","last_fullsized":"2024-04-25T14:11:32.583992+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.594681+0000","last_change":"2024-04-25T14:11:32.591107+0000","last_active":"2024-04-25T14:11:32.594681+0000","last_peered":"2024-04-25T14:11:32.594681+0000","last_clean":"2024-04-25T14:11:32.594681+0000","last_became_active":"2024-04-25T14:11:32.590762+0000","last_became_peered":"2024-04-25T14:11:32.590762+0000","last_unstale":"2024-04-25T14:11:32.594681+0000","last_undegraded":"2024-04-25T14:11:32.594681+0000","last_fullsized":"2024-04-25T14:11:32.594681+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:03.664590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.581752+0000","last_change":"2024-04-25T14:11:32.581752+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.581752+0000","last_undegraded":"2024-04-25T14:11:32.581752+0000","last_fullsized":"2024-04-25T14:11:32.581752+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.606645+0000","last_change":"2024-04-25T14:11:32.602319+0000","last_active":"2024-04-25T14:11:32.606645+0000","last_peered":"2024-04-25T14:11:32.606645+0000","last_clean":"2024-04-25T14:11:32.606645+0000","last_became_active":"2024-04-25T14:11:32.601994+0000","last_became_peered":"2024-04-25T14:11:32.601994+0000","last_unstale":"2024-04-25T14:11:32.606645+0000","last_undegraded":"2024-04-25T14:11:32.606645+0000","last_fullsized":"2024-04-25T14:11:32.606645+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:31.082017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.587696+0000","last_change":"2024-04-25T14:11:32.587696+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.587696+0000","last_undegraded":"2024-04-25T14:11:32.587696+0000","last_fullsized":"2024-04-25T14:11:32.587696+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.605694+0000","last_change":"2024-04-25T14:11:32.599420+0000","last_active":"2024-04-25T14:11:32.605694+0000","last_peered":"2024-04-25T14:11:32.605694+0000","last_clean":"2024-04-25T14:11:32.605694+0000","last_became_active":"2024-04-25T14:11:32.599133+0000","last_became_peered":"2024-04-25T14:11:32.599133+0000","last_unstale":"2024-04-25T14:11:32.605694+0000","last_undegraded":"2024-04-25T14:11:32.605694+0000","last_fullsized":"2024-04-25T14:11:32.605694+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:57.130911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.588360+0000","last_change":"2024-04-25T14:11:32.588360+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.588360+0000","last_undegraded":"2024-04-25T14:11:32.588360+0000","last_fullsized":"2024-04-25T14:11:32.588360+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.597686+0000","last_change":"2024-04-25T14:11:32.596904+0000","last_active":"2024-04-25T14:11:32.597686+0000","last_peered":"2024-04-25T14:11:32.597686+0000","last_clean":"2024-04-25T14:11:32.597686+0000","last_became_active":"2024-04-25T14:11:32.595555+0000","last_became_peered":"2024-04-25T14:11:32.595555+0000","last_unstale":"2024-04-25T14:11:32.597686+0000","last_undegraded":"2024-04-25T14:11:32.597686+0000","last_fullsized":"2024-04-25T14:11:32.597686+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:26:39.588788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.582115+0000","last_change":"2024-04-25T14:11:32.582115+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.582115+0000","last_undegraded":"2024-04-25T14:11:32.582115+0000","last_fullsized":"2024-04-25T14:11:32.582115+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.594949+0000","last_change":"2024-04-25T14:11:32.593779+0000","last_active":"2024-04-25T14:11:32.594949+0000","last_peered":"2024-04-25T14:11:32.594949+0000","last_clean":"2024-04-25T14:11:32.594949+0000","last_became_active":"2024-04-25T14:11:32.593185+0000","last_became_peered":"2024-04-25T14:11:32.593185+0000","last_unstale":"2024-04-25T14:11:32.594949+0000","last_undegraded":"2024-04-25T14:11:32.594949+0000","last_fullsized":"2024-04-25T14:11:32.594949+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:31.082017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.600313+0000","last_change":"2024-04-25T14:11:32.600313+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.600313+0000","last_undegraded":"2024-04-25T14:11:32.600313+0000","last_fullsized":"2024-04-25T14:11:32.600313+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.606874+0000","last_change":"2024-04-25T14:11:32.602817+0000","last_active":"2024-04-25T14:11:32.606874+0000","last_peered":"2024-04-25T14:11:32.606874+0000","last_clean":"2024-04-25T14:11:32.606874+0000","last_became_active":"2024-04-25T14:11:32.602507+0000","last_became_peered":"2024-04-25T14:11:32.602507+0000","last_unstale":"2024-04-25T14:11:32.606874+0000","last_undegraded":"2024-04-25T14:11:32.606874+0000","last_fullsized":"2024-04-25T14:11:32.606874+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:57.130911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.585777+0000","last_change":"2024-04-25T14:11:32.585777+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.585777+0000","last_undegraded":"2024-04-25T14:11:32.585777+0000","last_fullsized":"2024-04-25T14:11:32.585777+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.607043+0000","last_change":"2024-04-25T14:11:32.604165+0000","last_active":"2024-04-25T14:11:32.607043+0000","last_peered":"2024-04-25T14:11:32.607043+0000","last_clean":"2024-04-25T14:11:32.607043+0000","last_became_active":"2024-04-25T14:11:32.603846+0000","last_became_peered":"2024-04-25T14:11:32.603846+0000","last_unstale":"2024-04-25T14:11:32.607043+0000","last_undegraded":"2024-04-25T14:11:32.607043+0000","last_fullsized":"2024-04-25T14:11:32.607043+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:51.825452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.596371+0000","last_change":"2024-04-25T14:11:32.596371+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.596371+0000","last_undegraded":"2024-04-25T14:11:32.596371+0000","last_fullsized":"2024-04-25T14:11:32.596371+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.605403+0000","last_change":"2024-04-25T14:11:32.598249+0000","last_active":"2024-04-25T14:11:32.605403+0000","last_peered":"2024-04-25T14:11:32.605403+0000","last_clean":"2024-04-25T14:11:32.605403+0000","last_became_active":"2024-04-25T14:11:32.596737+0000","last_became_peered":"2024-04-25T14:11:32.596737+0000","last_unstale":"2024-04-25T14:11:32.605403+0000","last_undegraded":"2024-04-25T14:11:32.605403+0000","last_fullsized":"2024-04-25T14:11:32.605403+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:03.787508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.586319+0000","last_change":"2024-04-25T14:11:32.586319+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.586319+0000","last_undegraded":"2024-04-25T14:11:32.586319+0000","last_fullsized":"2024-04-25T14:11:32.586319+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.600618+0000","last_change":"2024-04-25T14:11:32.598361+0000","last_active":"2024-04-25T14:11:32.600618+0000","last_peered":"2024-04-25T14:11:32.600618+0000","last_clean":"2024-04-25T14:11:32.600618+0000","last_became_active":"2024-04-25T14:11:32.597623+0000","last_became_peered":"2024-04-25T14:11:32.597623+0000","last_unstale":"2024-04-25T14:11:32.600618+0000","last_undegraded":"2024-04-25T14:11:32.600618+0000","last_fullsized":"2024-04-25T14:11:32.600618+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:47:20.782498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.586726+0000","last_change":"2024-04-25T14:11:32.586726+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.586726+0000","last_undegraded":"2024-04-25T14:11:32.586726+0000","last_fullsized":"2024-04-25T14:11:32.586726+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.606881+0000","last_change":"2024-04-25T14:11:32.605007+0000","last_active":"2024-04-25T14:11:32.606881+0000","last_peered":"2024-04-25T14:11:32.606881+0000","last_clean":"2024-04-25T14:11:32.606881+0000","last_became_active":"2024-04-25T14:11:32.604678+0000","last_became_peered":"2024-04-25T14:11:32.604678+0000","last_unstale":"2024-04-25T14:11:32.606881+0000","last_undegraded":"2024-04-25T14:11:32.606881+0000","last_fullsized":"2024-04-25T14:11:32.606881+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:09.032042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.581119+0000","last_change":"2024-04-25T14:11:32.581119+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.581119+0000","last_undegraded":"2024-04-25T14:11:32.581119+0000","last_fullsized":"2024-04-25T14:11:32.581119+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.606097+0000","last_change":"2024-04-25T14:11:32.603829+0000","last_active":"2024-04-25T14:11:32.606097+0000","last_peered":"2024-04-25T14:11:32.606097+0000","last_clean":"2024-04-25T14:11:32.606097+0000","last_became_active":"2024-04-25T14:11:32.603424+0000","last_became_peered":"2024-04-25T14:11:32.603424+0000","last_unstale":"2024-04-25T14:11:32.606097+0000","last_undegraded":"2024-04-25T14:11:32.606097+0000","last_fullsized":"2024-04-25T14:11:32.606097+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:03.664590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.579652+0000","last_change":"2024-04-25T14:11:32.579652+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.579652+0000","last_undegraded":"2024-04-25T14:11:32.579652+0000","last_fullsized":"2024-04-25T14:11:32.579652+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.607006+0000","last_change":"2024-04-25T14:11:32.603572+0000","last_active":"2024-04-25T14:11:32.607006+0000","last_peered":"2024-04-25T14:11:32.607006+0000","last_clean":"2024-04-25T14:11:32.607006+0000","last_became_active":"2024-04-25T14:11:32.603270+0000","last_became_peered":"2024-04-25T14:11:32.603270+0000","last_unstale":"2024-04-25T14:11:32.607006+0000","last_undegraded":"2024-04-25T14:11:32.607006+0000","last_fullsized":"2024-04-25T14:11:32.607006+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:14.883395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.589786+0000","last_change":"2024-04-25T14:11:32.589786+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.589786+0000","last_undegraded":"2024-04-25T14:11:32.589786+0000","last_fullsized":"2024-04-25T14:11:32.589786+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.598593+0000","last_change":"2024-04-25T14:11:32.593507+0000","last_active":"2024-04-25T14:11:32.598593+0000","last_peered":"2024-04-25T14:11:32.598593+0000","last_clean":"2024-04-25T14:11:32.598593+0000","last_became_active":"2024-04-25T14:11:32.593173+0000","last_became_peered":"2024-04-25T14:11:32.593173+0000","last_unstale":"2024-04-25T14:11:32.598593+0000","last_undegraded":"2024-04-25T14:11:32.598593+0000","last_fullsized":"2024-04-25T14:11:32.598593+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:30.080066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.599854+0000","last_change":"2024-04-25T14:11:32.599854+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.599854+0000","last_undegraded":"2024-04-25T14:11:32.599854+0000","last_fullsized":"2024-04-25T14:11:32.599854+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.607359+0000","last_change":"2024-04-25T14:11:32.606046+0000","last_active":"2024-04-25T14:11:32.607359+0000","last_peered":"2024-04-25T14:11:32.607359+0000","last_clean":"2024-04-25T14:11:32.607359+0000","last_became_active":"2024-04-25T14:11:32.605755+0000","last_became_peered":"2024-04-25T14:11:32.605755+0000","last_unstale":"2024-04-25T14:11:32.607359+0000","last_undegraded":"2024-04-25T14:11:32.607359+0000","last_fullsized":"2024-04-25T14:11:32.607359+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:07.907191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.585978+0000","last_change":"2024-04-25T14:11:32.585978+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.585978+0000","last_undegraded":"2024-04-25T14:11:32.585978+0000","last_fullsized":"2024-04-25T14:11:32.585978+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.597673+0000","last_change":"2024-04-25T14:11:32.591966+0000","last_active":"2024-04-25T14:11:32.597673+0000","last_peered":"2024-04-25T14:11:32.597673+0000","last_clean":"2024-04-25T14:11:32.597673+0000","last_became_active":"2024-04-25T14:11:32.591633+0000","last_became_peered":"2024-04-25T14:11:32.591633+0000","last_unstale":"2024-04-25T14:11:32.597673+0000","last_undegraded":"2024-04-25T14:11:32.597673+0000","last_fullsized":"2024-04-25T14:11:32.597673+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:28.560866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.595836+0000","last_change":"2024-04-25T14:11:32.595836+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.595836+0000","last_undegraded":"2024-04-25T14:11:32.595836+0000","last_fullsized":"2024-04-25T14:11:32.595836+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.596752+0000","last_change":"2024-04-25T14:11:32.590551+0000","last_active":"2024-04-25T14:11:32.596752+0000","last_peered":"2024-04-25T14:11:32.596752+0000","last_clean":"2024-04-25T14:11:32.596752+0000","last_became_active":"2024-04-25T14:11:32.590233+0000","last_became_peered":"2024-04-25T14:11:32.590233+0000","last_unstale":"2024-04-25T14:11:32.596752+0000","last_undegraded":"2024-04-25T14:11:32.596752+0000","last_fullsized":"2024-04-25T14:11:32.596752+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:01.692524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.585051+0000","last_change":"2024-04-25T14:11:32.585051+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.585051+0000","last_undegraded":"2024-04-25T14:11:32.585051+0000","last_fullsized":"2024-04-25T14:11:32.585051+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.606893+0000","last_change":"2024-04-25T14:11:32.602907+0000","last_active":"2024-04-25T14:11:32.606893+0000","last_peered":"2024-04-25T14:11:32.606893+0000","last_clean":"2024-04-25T14:11:32.606893+0000","last_became_active":"2024-04-25T14:11:32.602600+0000","last_became_peered":"2024-04-25T14:11:32.602600+0000","last_unstale":"2024-04-25T14:11:32.606893+0000","last_undegraded":"2024-04-25T14:11:32.606893+0000","last_fullsized":"2024-04-25T14:11:32.606893+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:00.011427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.591752+0000","last_change":"2024-04-25T14:11:32.591752+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.591752+0000","last_undegraded":"2024-04-25T14:11:32.591752+0000","last_fullsized":"2024-04-25T14:11:32.591752+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.599321+0000","last_change":"2024-04-25T14:11:32.595961+0000","last_active":"2024-04-25T14:11:32.599321+0000","last_peered":"2024-04-25T14:11:32.599321+0000","last_clean":"2024-04-25T14:11:32.599321+0000","last_became_active":"2024-04-25T14:11:32.595641+0000","last_became_peered":"2024-04-25T14:11:32.595641+0000","last_unstale":"2024-04-25T14:11:32.599321+0000","last_undegraded":"2024-04-25T14:11:32.599321+0000","last_fullsized":"2024-04-25T14:11:32.599321+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:07.907191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.580561+0000","last_change":"2024-04-25T14:11:32.580561+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.580561+0000","last_undegraded":"2024-04-25T14:11:32.580561+0000","last_fullsized":"2024-04-25T14:11:32.580561+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.606072+0000","last_change":"2024-04-25T14:11:32.603997+0000","last_active":"2024-04-25T14:11:32.606072+0000","last_peered":"2024-04-25T14:11:32.606072+0000","last_clean":"2024-04-25T14:11:32.606072+0000","last_became_active":"2024-04-25T14:11:32.603630+0000","last_became_peered":"2024-04-25T14:11:32.603630+0000","last_unstale":"2024-04-25T14:11:32.606072+0000","last_undegraded":"2024-04-25T14:11:32.606072+0000","last_fullsized":"2024-04-25T14:11:32.606072+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:57.877724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.585864+0000","last_change":"2024-04-25T14:11:32.585864+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.585864+0000","last_undegraded":"2024-04-25T14:11:32.585864+0000","last_fullsized":"2024-04-25T14:11:32.585864+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.594581+0000","last_change":"2024-04-25T14:11:32.590839+0000","last_active":"2024-04-25T14:11:32.594581+0000","last_peered":"2024-04-25T14:11:32.594581+0000","last_clean":"2024-04-25T14:11:32.594581+0000","last_became_active":"2024-04-25T14:11:32.590349+0000","last_became_peered":"2024-04-25T14:11:32.590349+0000","last_unstale":"2024-04-25T14:11:32.594581+0000","last_undegraded":"2024-04-25T14:11:32.594581+0000","last_fullsized":"2024-04-25T14:11:32.594581+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:57.877724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.580907+0000","last_change":"2024-04-25T14:11:32.580907+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.580907+0000","last_undegraded":"2024-04-25T14:11:32.580907+0000","last_fullsized":"2024-04-25T14:11:32.580907+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.595070+0000","last_change":"2024-04-25T14:11:32.592951+0000","last_active":"2024-04-25T14:11:32.595070+0000","last_peered":"2024-04-25T14:11:32.595070+0000","last_clean":"2024-04-25T14:11:32.595070+0000","last_became_active":"2024-04-25T14:11:32.592568+0000","last_became_peered":"2024-04-25T14:11:32.592568+0000","last_unstale":"2024-04-25T14:11:32.595070+0000","last_undegraded":"2024-04-25T14:11:32.595070+0000","last_fullsized":"2024-04-25T14:11:32.595070+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:55.757498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.590620+0000","last_change":"2024-04-25T14:11:32.590620+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.590620+0000","last_undegraded":"2024-04-25T14:11:32.590620+0000","last_fullsized":"2024-04-25T14:11:32.590620+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.600664+0000","last_change":"2024-04-25T14:11:32.598702+0000","last_active":"2024-04-25T14:11:32.600664+0000","last_peered":"2024-04-25T14:11:32.600664+0000","last_clean":"2024-04-25T14:11:32.600664+0000","last_became_active":"2024-04-25T14:11:32.598372+0000","last_became_peered":"2024-04-25T14:11:32.598372+0000","last_unstale":"2024-04-25T14:11:32.600664+0000","last_undegraded":"2024-04-25T14:11:32.600664+0000","last_fullsized":"2024-04-25T14:11:32.600664+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:10.168462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.600010+0000","last_change":"2024-04-25T14:11:32.600010+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.600010+0000","last_undegraded":"2024-04-25T14:11:32.600010+0000","last_fullsized":"2024-04-25T14:11:32.600010+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.601509+0000","last_change":"2024-04-25T14:11:32.596091+0000","last_active":"2024-04-25T14:11:32.601509+0000","last_peered":"2024-04-25T14:11:32.601509+0000","last_clean":"2024-04-25T14:11:32.601509+0000","last_became_active":"2024-04-25T14:11:32.595711+0000","last_became_peered":"2024-04-25T14:11:32.595711+0000","last_unstale":"2024-04-25T14:11:32.601509+0000","last_undegraded":"2024-04-25T14:11:32.601509+0000","last_fullsized":"2024-04-25T14:11:32.601509+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:21.401742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.582257+0000","last_change":"2024-04-25T14:11:32.582257+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.582257+0000","last_undegraded":"2024-04-25T14:11:32.582257+0000","last_fullsized":"2024-04-25T14:11:32.582257+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.602463+0000","last_change":"2024-04-25T14:11:32.599842+0000","last_active":"2024-04-25T14:11:32.602463+0000","last_peered":"2024-04-25T14:11:32.602463+0000","last_clean":"2024-04-25T14:11:32.602463+0000","last_became_active":"2024-04-25T14:11:32.599226+0000","last_became_peered":"2024-04-25T14:11:32.599226+0000","last_unstale":"2024-04-25T14:11:32.602463+0000","last_undegraded":"2024-04-25T14:11:32.602463+0000","last_fullsized":"2024-04-25T14:11:32.602463+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:03.664590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.583391+0000","last_change":"2024-04-25T14:11:32.583391+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.583391+0000","last_undegraded":"2024-04-25T14:11:32.583391+0000","last_fullsized":"2024-04-25T14:11:32.583391+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.598499+0000","last_change":"2024-04-25T14:11:32.593191+0000","last_active":"2024-04-25T14:11:32.598499+0000","last_peered":"2024-04-25T14:11:32.598499+0000","last_clean":"2024-04-25T14:11:32.598499+0000","last_became_active":"2024-04-25T14:11:32.592628+0000","last_became_peered":"2024-04-25T14:11:32.592628+0000","last_unstale":"2024-04-25T14:11:32.598499+0000","last_undegraded":"2024-04-25T14:11:32.598499+0000","last_fullsized":"2024-04-25T14:11:32.598499+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:20.285653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"129'3277","reported_seq":3750,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.591126+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:11:32.591126+0000","last_peered":"2024-04-25T14:11:32.591126+0000","last_clean":"2024-04-25T14:11:32.591126+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:11:32.591126+0000","last_undegraded":"2024-04-25T14:11:32.591126+0000","last_fullsized":"2024-04-25T14:11:32.591126+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":3277,"log_dups_size":0,"ondisk_log_size":3277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":717,"num_read_kb":1927,"num_write":223,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":717,"num_read_kb":1927,"num_write":223,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3277,"ondisk_log_size":3277,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574939,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29296,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342544,"statfs":{"total":96636764160,"available":96606765056,"internally_reserved":0,"allocated":2326528,"data_stored":1456665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574940,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1732608,"data_stored":866297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32860,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338980,"statfs":{"total":96636764160,"available":96603115520,"internally_reserved":0,"allocated":1720320,"data_stored":859630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574940,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32860,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338980,"statfs":{"total":96636764160,"available":96603115520,"internally_reserved":0,"allocated":1720320,"data_stored":859630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32860,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338980,"statfs":{"total":96636764160,"available":96603115520,"internally_reserved":0,"allocated":1720320,"data_stored":859630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574939,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32860,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338980,"statfs":{"total":96636764160,"available":96603115520,"internally_reserved":0,"allocated":1720320,"data_stored":859630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1732608,"data_stored":866297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29280,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342560,"statfs":{"total":96636764160,"available":96606781440,"internally_reserved":0,"allocated":2314240,"data_stored":1449998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:11:37.722 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:11:37.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.909+0000 7f3cb5800640 1 Processor -- start 2024-04-25T14:11:37.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.909+0000 7f3cb5800640 1 -- start start 2024-04-25T14:11:37.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.909+0000 7f3cb5800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3cb015e860 0x7f3cb00ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:37.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.909+0000 7f3cb5800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3cb015ec70 0x7f3cb006d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:37.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.909+0000 7f3cb5800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cb00cf8f0 con 0x7f3cb00cf3e0 2024-04-25T14:11:37.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.909+0000 7f3cb5800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3cb006cfa0 con 0x7f3cb015e860 2024-04-25T14:11:37.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.909+0000 7f3cb5800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3cb006c670 con 0x7f3cb015ec70 2024-04-25T14:11:37.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.909+0000 7f3caea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3cb015e860 0x7f3cb00ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:37.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.909+0000 7f3ca6000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3cb00cf3e0 0x7f3cb00d0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60480/0 (socket says 172.21.15.86:60480) 2024-04-25T14:11:37.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.909+0000 7f3ca6000640 1 -- 172.21.15.86:0/431152074 learned_addr learned my addr 172.21.15.86:0/431152074 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:37.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.909+0000 7f3caf400640 1 --2- 172.21.15.86:0/431152074 >> v2:172.21.15.155:3300/0 conn(0x7f3cb015ec70 0x7f3cb006d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:37.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.909+0000 7f3caea00640 1 -- 172.21.15.86:0/431152074 >> v2:172.21.15.155:3300/0 conn(0x7f3cb015ec70 msgr2=0x7f3cb006d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:37.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.909+0000 7f3caea00640 1 --2- 172.21.15.86:0/431152074 >> v2:172.21.15.155:3300/0 conn(0x7f3cb015ec70 0x7f3cb006d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:37.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.909+0000 7f3caea00640 1 -- 172.21.15.86:0/431152074 >> v1:172.21.15.86:6789/0 conn(0x7f3cb00cf3e0 legacy=0x7f3cb00d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:37.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.909+0000 7f3caea00640 1 -- 172.21.15.86:0/431152074 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb006c950 con 0x7f3cb015e860 2024-04-25T14:11:37.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.913+0000 7f3caea00640 1 --2- 172.21.15.86:0/431152074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cb015e860 0x7f3cb00ceea0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0071290 tx=0x7f3cb006e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9b595a19a298a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:37.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.913+0000 7f3cae000640 1 -- 172.21.15.86:0/431152074 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca00b1070 con 0x7f3cb015e860 2024-04-25T14:11:37.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.913+0000 7f3cae000640 1 -- 172.21.15.86:0/431152074 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ca00a8050 con 0x7f3cb015e860 2024-04-25T14:11:37.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.913+0000 7f3cae000640 1 -- 172.21.15.86:0/431152074 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca00ac860 con 0x7f3cb015e860 2024-04-25T14:11:37.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.913+0000 7f3cb5800640 1 -- 172.21.15.86:0/431152074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cb015e860 msgr2=0x7f3cb00ceea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:37.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.913+0000 7f3cb5800640 1 --2- 172.21.15.86:0/431152074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cb015e860 0x7f3cb00ceea0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0071290 tx=0x7f3cb006e180 comp rx=0 tx=0).stop 2024-04-25T14:11:37.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.913+0000 7f3cb5800640 1 -- 172.21.15.86:0/431152074 shutdown_connections 2024-04-25T14:11:37.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.913+0000 7f3cb5800640 1 --2- 172.21.15.86:0/431152074 >> v2:172.21.15.155:3300/0 conn(0x7f3cb015ec70 0x7f3cb006d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:37.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.913+0000 7f3cb5800640 1 --2- 172.21.15.86:0/431152074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cb015e860 0x7f3cb00ceea0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:37.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.913+0000 7f3cb5800640 1 -- 172.21.15.86:0/431152074 >> 172.21.15.86:0/431152074 conn(0x7f3cb00d2230 msgr2=0x7f3cb00d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:37.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.913+0000 7f3cb5800640 1 -- 172.21.15.86:0/431152074 shutdown_connections 2024-04-25T14:11:37.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.913+0000 7f3cb5800640 1 -- 172.21.15.86:0/431152074 wait complete. 2024-04-25T14:11:37.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.917+0000 7f3cb5800640 1 Processor -- start 2024-04-25T14:11:37.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.917+0000 7f3cb5800640 1 -- start start 2024-04-25T14:11:37.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.917+0000 7f3cb5800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cb00cf3e0 0x7f3cb01529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:37.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.917+0000 7f3cb5800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cb015e860 0x7f3cb0155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:37.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.917+0000 7f3cb5800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cb015ec70 0x7f3cb0155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:37.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.917+0000 7f3cb5800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb016aab0 con 0x7f3cb015e860 2024-04-25T14:11:37.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.917+0000 7f3cb5800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb016a930 con 0x7f3cb015ec70 2024-04-25T14:11:37.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.917+0000 7f3cb5800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3cb016ac30 con 0x7f3cb00cf3e0 2024-04-25T14:11:37.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.917+0000 7f3ca6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cb015e860 0x7f3cb0155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:37.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.917+0000 7f3ca6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cb015e860 0x7f3cb0155150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40994/0 (socket says 172.21.15.86:40994) 2024-04-25T14:11:37.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.917+0000 7f3ca6000640 1 -- 172.21.15.86:0/598323662 learned_addr learned my addr 172.21.15.86:0/598323662 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:37.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.921+0000 7f3caea00640 1 --2- 172.21.15.86:0/598323662 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cb00cf3e0 0x7f3cb01529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:37.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.921+0000 7f3ca6000640 1 -- 172.21.15.86:0/598323662 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cb00cf3e0 msgr2=0x7f3cb01529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:37.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.921+0000 7f3caf400640 1 --2- 172.21.15.86:0/598323662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cb015ec70 0x7f3cb0155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:37.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.921+0000 7f3ca6000640 1 --2- 172.21.15.86:0/598323662 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cb00cf3e0 0x7f3cb01529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:37.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.921+0000 7f3ca6000640 1 -- 172.21.15.86:0/598323662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cb015ec70 msgr2=0x7f3cb0155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:37.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.921+0000 7f3ca6000640 1 --2- 172.21.15.86:0/598323662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cb015ec70 0x7f3cb0155690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:37.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.921+0000 7f3ca6000640 1 -- 172.21.15.86:0/598323662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb0155ed0 con 0x7f3cb015e860 2024-04-25T14:11:37.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.921+0000 7f3caea00640 1 --2- 172.21.15.86:0/598323662 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cb00cf3e0 0x7f3cb01529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:37.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.921+0000 7f3caf400640 1 --2- 172.21.15.86:0/598323662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cb015ec70 0x7f3cb0155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:37.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.921+0000 7f3ca6000640 1 --2- 172.21.15.86:0/598323662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cb015e860 0x7f3cb0155150 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800cad0 tx=0x7f3c9800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:37.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.921+0000 7f3ca7400640 1 -- 172.21.15.86:0/598323662 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c98017070 con 0x7f3cb015e860 2024-04-25T14:11:37.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.921+0000 7f3ca7400640 1 -- 172.21.15.86:0/598323662 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c9800ac30 con 0x7f3cb015e860 2024-04-25T14:11:37.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.921+0000 7f3ca7400640 1 -- 172.21.15.86:0/598323662 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c98012400 con 0x7f3cb015e860 2024-04-25T14:11:37.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.921+0000 7f3cb5800640 1 -- 172.21.15.86:0/598323662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cb01627b0 con 0x7f3cb015e860 2024-04-25T14:11:37.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.921+0000 7f3cb5800640 1 -- 172.21.15.86:0/598323662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cb0162d70 con 0x7f3cb015e860 2024-04-25T14:11:37.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.921+0000 7f3cb5800640 1 -- 172.21.15.86:0/598323662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cb0074cf0 con 0x7f3cb015e860 2024-04-25T14:11:37.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.921+0000 7f3ca7400640 1 -- 172.21.15.86:0/598323662 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7f3c9801a070 con 0x7f3cb015e860 2024-04-25T14:11:37.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.925+0000 7f3ca7400640 1 --2- 172.21.15.86:0/598323662 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f3c8c06f550 0x7f3c8c071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:37.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.925+0000 7f3ca7400640 1 -- 172.21.15.86:0/598323662 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f3c9808f730 con 0x7f3cb015e860 2024-04-25T14:11:37.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.925+0000 7f3caea00640 1 --2- 172.21.15.86:0/598323662 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f3c8c06f550 0x7f3c8c071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:37.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.925+0000 7f3caea00640 1 --2- 172.21.15.86:0/598323662 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f3c8c06f550 0x7f3c8c071a60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3ca00ae000 tx=0x7f3ca0008040 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:37.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:37.925+0000 7f3ca7400640 1 -- 172.21.15.86:0/598323662 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3c98021020 con 0x7f3cb015e860 2024-04-25T14:11:38.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.081+0000 7f3cb5800640 1 -- 172.21.15.86:0/598323662 --> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3cb01561b0 con 0x7f3c8c06f550 2024-04-25T14:11:38.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.089+0000 7f3ca7400640 1 -- 172.21.15.86:0/598323662 <== mgr.9282 v2:172.21.15.86:6832/264890479 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247100 (secure 0 0 0) 0x7f3ca00ac710 con 0x7f3c8c06f550 2024-04-25T14:11:38.086 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:38.088 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:11:38.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.089+0000 7f3cb5800640 1 -- 172.21.15.86:0/598323662 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f3c8c06f550 msgr2=0x7f3c8c071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:38.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.089+0000 7f3cb5800640 1 --2- 172.21.15.86:0/598323662 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f3c8c06f550 0x7f3c8c071a60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3ca00ae000 tx=0x7f3ca0008040 comp rx=0 tx=0).stop 2024-04-25T14:11:38.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.089+0000 7f3cb5800640 1 -- 172.21.15.86:0/598323662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cb015e860 msgr2=0x7f3cb0155150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:38.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.089+0000 7f3cb5800640 1 --2- 172.21.15.86:0/598323662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cb015e860 0x7f3cb0155150 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800cad0 tx=0x7f3c9800cf90 comp rx=0 tx=0).stop 2024-04-25T14:11:38.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.089+0000 7f3cb5800640 1 -- 172.21.15.86:0/598323662 shutdown_connections 2024-04-25T14:11:38.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.089+0000 7f3cb5800640 1 --2- 172.21.15.86:0/598323662 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f3c8c06f550 0x7f3c8c071a60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:38.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.089+0000 7f3cb5800640 1 --2- 172.21.15.86:0/598323662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cb015ec70 0x7f3cb0155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:38.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.089+0000 7f3cb5800640 1 --2- 172.21.15.86:0/598323662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cb015e860 0x7f3cb0155150 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:38.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.089+0000 7f3cb5800640 1 --2- 172.21.15.86:0/598323662 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cb00cf3e0 0x7f3cb01529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:38.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.089+0000 7f3cb5800640 1 -- 172.21.15.86:0/598323662 >> 172.21.15.86:0/598323662 conn(0x7f3cb00d2230 msgr2=0x7f3cb00b5a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:38.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.089+0000 7f3cb5800640 1 -- 172.21.15.86:0/598323662 shutdown_connections 2024-04-25T14:11:38.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.089+0000 7f3cb5800640 1 -- 172.21.15.86:0/598323662 wait complete. 2024-04-25T14:11:38.107 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T14:11:36.734296+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":717,"num_read_kb":1927,"num_write":223,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3277,"ondisk_log_size":3277,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":255768,"kb_used_data":14636,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754718952,"statfs":{"total":773094113280,"available":772832206848,"internally_reserved":0,"allocated":14987264,"data_stored":8077777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.166599"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.583286+0000","last_change":"2024-04-25T14:11:32.583286+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.583286+0000","last_undegraded":"2024-04-25T14:11:32.583286+0000","last_fullsized":"2024-04-25T14:11:32.583286+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.593406+0000","last_change":"2024-04-25T14:11:32.593406+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.593406+0000","last_undegraded":"2024-04-25T14:11:32.593406+0000","last_fullsized":"2024-04-25T14:11:32.593406+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.597461+0000","last_change":"2024-04-25T14:11:32.597461+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.597461+0000","last_undegraded":"2024-04-25T14:11:32.597461+0000","last_fullsized":"2024-04-25T14:11:32.597461+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.595145+0000","last_change":"2024-04-25T14:11:32.595145+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.595145+0000","last_undegraded":"2024-04-25T14:11:32.595145+0000","last_fullsized":"2024-04-25T14:11:32.595145+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.590150+0000","last_change":"2024-04-25T14:11:32.590150+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.590150+0000","last_undegraded":"2024-04-25T14:11:32.590150+0000","last_fullsized":"2024-04-25T14:11:32.590150+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.588606+0000","last_change":"2024-04-25T14:11:32.588606+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.588606+0000","last_undegraded":"2024-04-25T14:11:32.588606+0000","last_fullsized":"2024-04-25T14:11:32.588606+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.582342+0000","last_change":"2024-04-25T14:11:32.582342+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.582342+0000","last_undegraded":"2024-04-25T14:11:32.582342+0000","last_fullsized":"2024-04-25T14:11:32.582342+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.583423+0000","last_change":"2024-04-25T14:11:32.583423+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.583423+0000","last_undegraded":"2024-04-25T14:11:32.583423+0000","last_fullsized":"2024-04-25T14:11:32.583423+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.588120+0000","last_change":"2024-04-25T14:11:32.588120+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.588120+0000","last_undegraded":"2024-04-25T14:11:32.588120+0000","last_fullsized":"2024-04-25T14:11:32.588120+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.585963+0000","last_change":"2024-04-25T14:11:32.585963+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.585963+0000","last_undegraded":"2024-04-25T14:11:32.585963+0000","last_fullsized":"2024-04-25T14:11:32.585963+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.588205+0000","last_change":"2024-04-25T14:11:32.588205+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.588205+0000","last_undegraded":"2024-04-25T14:11:32.588205+0000","last_fullsized":"2024-04-25T14:11:32.588205+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.593307+0000","last_change":"2024-04-25T14:11:32.593307+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.593307+0000","last_undegraded":"2024-04-25T14:11:32.593307+0000","last_fullsized":"2024-04-25T14:11:32.593307+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.589718+0000","last_change":"2024-04-25T14:11:32.589718+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.589718+0000","last_undegraded":"2024-04-25T14:11:32.589718+0000","last_fullsized":"2024-04-25T14:11:32.589718+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.596485+0000","last_change":"2024-04-25T14:11:32.596485+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.596485+0000","last_undegraded":"2024-04-25T14:11:32.596485+0000","last_fullsized":"2024-04-25T14:11:32.596485+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.600818+0000","last_change":"2024-04-25T14:11:32.600818+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.600818+0000","last_undegraded":"2024-04-25T14:11:32.600818+0000","last_fullsized":"2024-04-25T14:11:32.600818+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.586259+0000","last_change":"2024-04-25T14:11:32.586259+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.586259+0000","last_undegraded":"2024-04-25T14:11:32.586259+0000","last_fullsized":"2024-04-25T14:11:32.586259+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.594205+0000","last_change":"2024-04-25T14:11:32.594205+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.594205+0000","last_undegraded":"2024-04-25T14:11:32.594205+0000","last_fullsized":"2024-04-25T14:11:32.594205+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.588827+0000","last_change":"2024-04-25T14:11:32.588827+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.588827+0000","last_undegraded":"2024-04-25T14:11:32.588827+0000","last_fullsized":"2024-04-25T14:11:32.588827+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.588488+0000","last_change":"2024-04-25T14:11:32.588488+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.588488+0000","last_undegraded":"2024-04-25T14:11:32.588488+0000","last_fullsized":"2024-04-25T14:11:32.588488+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.585021+0000","last_change":"2024-04-25T14:11:32.585021+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.585021+0000","last_undegraded":"2024-04-25T14:11:32.585021+0000","last_fullsized":"2024-04-25T14:11:32.585021+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.589265+0000","last_change":"2024-04-25T14:11:32.589265+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.589265+0000","last_undegraded":"2024-04-25T14:11:32.589265+0000","last_fullsized":"2024-04-25T14:11:32.589265+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.581795+0000","last_change":"2024-04-25T14:11:32.581795+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.581795+0000","last_undegraded":"2024-04-25T14:11:32.581795+0000","last_fullsized":"2024-04-25T14:11:32.581795+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.583620+0000","last_change":"2024-04-25T14:11:32.583620+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.583620+0000","last_undegraded":"2024-04-25T14:11:32.583620+0000","last_fullsized":"2024-04-25T14:11:32.583620+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.586990+0000","last_change":"2024-04-25T14:11:32.586990+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.586990+0000","last_undegraded":"2024-04-25T14:11:32.586990+0000","last_fullsized":"2024-04-25T14:11:32.586990+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.580738+0000","last_change":"2024-04-25T14:11:32.580738+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.580738+0000","last_undegraded":"2024-04-25T14:11:32.580738+0000","last_fullsized":"2024-04-25T14:11:32.580738+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.581641+0000","last_change":"2024-04-25T14:11:32.581641+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.581641+0000","last_undegraded":"2024-04-25T14:11:32.581641+0000","last_fullsized":"2024-04-25T14:11:32.581641+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.586189+0000","last_change":"2024-04-25T14:11:32.586189+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.586189+0000","last_undegraded":"2024-04-25T14:11:32.586189+0000","last_fullsized":"2024-04-25T14:11:32.586189+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.588712+0000","last_change":"2024-04-25T14:11:32.588712+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.588712+0000","last_undegraded":"2024-04-25T14:11:32.588712+0000","last_fullsized":"2024-04-25T14:11:32.588712+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.582773+0000","last_change":"2024-04-25T14:11:32.582773+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.582773+0000","last_undegraded":"2024-04-25T14:11:32.582773+0000","last_fullsized":"2024-04-25T14:11:32.582773+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.587206+0000","last_change":"2024-04-25T14:11:32.587206+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.587206+0000","last_undegraded":"2024-04-25T14:11:32.587206+0000","last_fullsized":"2024-04-25T14:11:32.587206+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.584884+0000","last_change":"2024-04-25T14:11:32.584884+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.584884+0000","last_undegraded":"2024-04-25T14:11:32.584884+0000","last_fullsized":"2024-04-25T14:11:32.584884+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.582726+0000","last_change":"2024-04-25T14:11:32.582726+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.582726+0000","last_undegraded":"2024-04-25T14:11:32.582726+0000","last_fullsized":"2024-04-25T14:11:32.582726+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.596501+0000","last_change":"2024-04-25T14:11:32.596501+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.596501+0000","last_undegraded":"2024-04-25T14:11:32.596501+0000","last_fullsized":"2024-04-25T14:11:32.596501+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.594681+0000","last_change":"2024-04-25T14:11:32.591452+0000","last_active":"2024-04-25T14:11:32.594681+0000","last_peered":"2024-04-25T14:11:32.594681+0000","last_clean":"2024-04-25T14:11:32.594681+0000","last_became_active":"2024-04-25T14:11:32.591159+0000","last_became_peered":"2024-04-25T14:11:32.591159+0000","last_unstale":"2024-04-25T14:11:32.594681+0000","last_undegraded":"2024-04-25T14:11:32.594681+0000","last_fullsized":"2024-04-25T14:11:32.594681+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:16.063389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.596966+0000","last_change":"2024-04-25T14:11:32.596966+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.596966+0000","last_undegraded":"2024-04-25T14:11:32.596966+0000","last_fullsized":"2024-04-25T14:11:32.596966+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.606856+0000","last_change":"2024-04-25T14:11:32.603965+0000","last_active":"2024-04-25T14:11:32.606856+0000","last_peered":"2024-04-25T14:11:32.606856+0000","last_clean":"2024-04-25T14:11:32.606856+0000","last_became_active":"2024-04-25T14:11:32.602019+0000","last_became_peered":"2024-04-25T14:11:32.602019+0000","last_unstale":"2024-04-25T14:11:32.606856+0000","last_undegraded":"2024-04-25T14:11:32.606856+0000","last_fullsized":"2024-04-25T14:11:32.606856+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:44.182089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.582818+0000","last_change":"2024-04-25T14:11:32.582818+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.582818+0000","last_undegraded":"2024-04-25T14:11:32.582818+0000","last_fullsized":"2024-04-25T14:11:32.582818+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.606798+0000","last_change":"2024-04-25T14:11:32.601614+0000","last_active":"2024-04-25T14:11:32.606798+0000","last_peered":"2024-04-25T14:11:32.606798+0000","last_clean":"2024-04-25T14:11:32.606798+0000","last_became_active":"2024-04-25T14:11:32.601092+0000","last_became_peered":"2024-04-25T14:11:32.601092+0000","last_unstale":"2024-04-25T14:11:32.606798+0000","last_undegraded":"2024-04-25T14:11:32.606798+0000","last_fullsized":"2024-04-25T14:11:32.606798+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:16.063389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.587090+0000","last_change":"2024-04-25T14:11:32.587090+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.587090+0000","last_undegraded":"2024-04-25T14:11:32.587090+0000","last_fullsized":"2024-04-25T14:11:32.587090+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.605321+0000","last_change":"2024-04-25T14:11:32.602711+0000","last_active":"2024-04-25T14:11:32.605321+0000","last_peered":"2024-04-25T14:11:32.605321+0000","last_clean":"2024-04-25T14:11:32.605321+0000","last_became_active":"2024-04-25T14:11:32.602368+0000","last_became_peered":"2024-04-25T14:11:32.602368+0000","last_unstale":"2024-04-25T14:11:32.605321+0000","last_undegraded":"2024-04-25T14:11:32.605321+0000","last_fullsized":"2024-04-25T14:11:32.605321+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:12.830306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.584817+0000","last_change":"2024-04-25T14:11:32.584817+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.584817+0000","last_undegraded":"2024-04-25T14:11:32.584817+0000","last_fullsized":"2024-04-25T14:11:32.584817+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.601151+0000","last_change":"2024-04-25T14:11:32.598778+0000","last_active":"2024-04-25T14:11:32.601151+0000","last_peered":"2024-04-25T14:11:32.601151+0000","last_clean":"2024-04-25T14:11:32.601151+0000","last_became_active":"2024-04-25T14:11:32.598190+0000","last_became_peered":"2024-04-25T14:11:32.598190+0000","last_unstale":"2024-04-25T14:11:32.601151+0000","last_undegraded":"2024-04-25T14:11:32.601151+0000","last_fullsized":"2024-04-25T14:11:32.601151+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:55.757498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.592800+0000","last_change":"2024-04-25T14:11:32.592800+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.592800+0000","last_undegraded":"2024-04-25T14:11:32.592800+0000","last_fullsized":"2024-04-25T14:11:32.592800+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.605858+0000","last_change":"2024-04-25T14:11:32.605097+0000","last_active":"2024-04-25T14:11:32.605858+0000","last_peered":"2024-04-25T14:11:32.605858+0000","last_clean":"2024-04-25T14:11:32.605858+0000","last_became_active":"2024-04-25T14:11:32.604770+0000","last_became_peered":"2024-04-25T14:11:32.604770+0000","last_unstale":"2024-04-25T14:11:32.605858+0000","last_undegraded":"2024-04-25T14:11:32.605858+0000","last_fullsized":"2024-04-25T14:11:32.605858+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:44.182089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.600464+0000","last_change":"2024-04-25T14:11:32.600464+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.600464+0000","last_undegraded":"2024-04-25T14:11:32.600464+0000","last_fullsized":"2024-04-25T14:11:32.600464+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.606597+0000","last_change":"2024-04-25T14:11:32.605101+0000","last_active":"2024-04-25T14:11:32.606597+0000","last_peered":"2024-04-25T14:11:32.606597+0000","last_clean":"2024-04-25T14:11:32.606597+0000","last_became_active":"2024-04-25T14:11:32.604386+0000","last_became_peered":"2024-04-25T14:11:32.604386+0000","last_unstale":"2024-04-25T14:11:32.606597+0000","last_undegraded":"2024-04-25T14:11:32.606597+0000","last_fullsized":"2024-04-25T14:11:32.606597+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:00.011427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.583992+0000","last_change":"2024-04-25T14:11:32.583992+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.583992+0000","last_undegraded":"2024-04-25T14:11:32.583992+0000","last_fullsized":"2024-04-25T14:11:32.583992+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.594681+0000","last_change":"2024-04-25T14:11:32.591107+0000","last_active":"2024-04-25T14:11:32.594681+0000","last_peered":"2024-04-25T14:11:32.594681+0000","last_clean":"2024-04-25T14:11:32.594681+0000","last_became_active":"2024-04-25T14:11:32.590762+0000","last_became_peered":"2024-04-25T14:11:32.590762+0000","last_unstale":"2024-04-25T14:11:32.594681+0000","last_undegraded":"2024-04-25T14:11:32.594681+0000","last_fullsized":"2024-04-25T14:11:32.594681+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:03.664590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.581752+0000","last_change":"2024-04-25T14:11:32.581752+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.581752+0000","last_undegraded":"2024-04-25T14:11:32.581752+0000","last_fullsized":"2024-04-25T14:11:32.581752+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.606645+0000","last_change":"2024-04-25T14:11:32.602319+0000","last_active":"2024-04-25T14:11:32.606645+0000","last_peered":"2024-04-25T14:11:32.606645+0000","last_clean":"2024-04-25T14:11:32.606645+0000","last_became_active":"2024-04-25T14:11:32.601994+0000","last_became_peered":"2024-04-25T14:11:32.601994+0000","last_unstale":"2024-04-25T14:11:32.606645+0000","last_undegraded":"2024-04-25T14:11:32.606645+0000","last_fullsized":"2024-04-25T14:11:32.606645+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:31.082017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.587696+0000","last_change":"2024-04-25T14:11:32.587696+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.587696+0000","last_undegraded":"2024-04-25T14:11:32.587696+0000","last_fullsized":"2024-04-25T14:11:32.587696+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.605694+0000","last_change":"2024-04-25T14:11:32.599420+0000","last_active":"2024-04-25T14:11:32.605694+0000","last_peered":"2024-04-25T14:11:32.605694+0000","last_clean":"2024-04-25T14:11:32.605694+0000","last_became_active":"2024-04-25T14:11:32.599133+0000","last_became_peered":"2024-04-25T14:11:32.599133+0000","last_unstale":"2024-04-25T14:11:32.605694+0000","last_undegraded":"2024-04-25T14:11:32.605694+0000","last_fullsized":"2024-04-25T14:11:32.605694+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:57.130911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.588360+0000","last_change":"2024-04-25T14:11:32.588360+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.588360+0000","last_undegraded":"2024-04-25T14:11:32.588360+0000","last_fullsized":"2024-04-25T14:11:32.588360+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.597686+0000","last_change":"2024-04-25T14:11:32.596904+0000","last_active":"2024-04-25T14:11:32.597686+0000","last_peered":"2024-04-25T14:11:32.597686+0000","last_clean":"2024-04-25T14:11:32.597686+0000","last_became_active":"2024-04-25T14:11:32.595555+0000","last_became_peered":"2024-04-25T14:11:32.595555+0000","last_unstale":"2024-04-25T14:11:32.597686+0000","last_undegraded":"2024-04-25T14:11:32.597686+0000","last_fullsized":"2024-04-25T14:11:32.597686+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:26:39.588788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.582115+0000","last_change":"2024-04-25T14:11:32.582115+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.582115+0000","last_undegraded":"2024-04-25T14:11:32.582115+0000","last_fullsized":"2024-04-25T14:11:32.582115+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.594949+0000","last_change":"2024-04-25T14:11:32.593779+0000","last_active":"2024-04-25T14:11:32.594949+0000","last_peered":"2024-04-25T14:11:32.594949+0000","last_clean":"2024-04-25T14:11:32.594949+0000","last_became_active":"2024-04-25T14:11:32.593185+0000","last_became_peered":"2024-04-25T14:11:32.593185+0000","last_unstale":"2024-04-25T14:11:32.594949+0000","last_undegraded":"2024-04-25T14:11:32.594949+0000","last_fullsized":"2024-04-25T14:11:32.594949+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:31.082017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.600313+0000","last_change":"2024-04-25T14:11:32.600313+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.600313+0000","last_undegraded":"2024-04-25T14:11:32.600313+0000","last_fullsized":"2024-04-25T14:11:32.600313+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.606874+0000","last_change":"2024-04-25T14:11:32.602817+0000","last_active":"2024-04-25T14:11:32.606874+0000","last_peered":"2024-04-25T14:11:32.606874+0000","last_clean":"2024-04-25T14:11:32.606874+0000","last_became_active":"2024-04-25T14:11:32.602507+0000","last_became_peered":"2024-04-25T14:11:32.602507+0000","last_unstale":"2024-04-25T14:11:32.606874+0000","last_undegraded":"2024-04-25T14:11:32.606874+0000","last_fullsized":"2024-04-25T14:11:32.606874+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:57.130911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.585777+0000","last_change":"2024-04-25T14:11:32.585777+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.585777+0000","last_undegraded":"2024-04-25T14:11:32.585777+0000","last_fullsized":"2024-04-25T14:11:32.585777+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.607043+0000","last_change":"2024-04-25T14:11:32.604165+0000","last_active":"2024-04-25T14:11:32.607043+0000","last_peered":"2024-04-25T14:11:32.607043+0000","last_clean":"2024-04-25T14:11:32.607043+0000","last_became_active":"2024-04-25T14:11:32.603846+0000","last_became_peered":"2024-04-25T14:11:32.603846+0000","last_unstale":"2024-04-25T14:11:32.607043+0000","last_undegraded":"2024-04-25T14:11:32.607043+0000","last_fullsized":"2024-04-25T14:11:32.607043+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:51.825452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.596371+0000","last_change":"2024-04-25T14:11:32.596371+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.596371+0000","last_undegraded":"2024-04-25T14:11:32.596371+0000","last_fullsized":"2024-04-25T14:11:32.596371+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.605403+0000","last_change":"2024-04-25T14:11:32.598249+0000","last_active":"2024-04-25T14:11:32.605403+0000","last_peered":"2024-04-25T14:11:32.605403+0000","last_clean":"2024-04-25T14:11:32.605403+0000","last_became_active":"2024-04-25T14:11:32.596737+0000","last_became_peered":"2024-04-25T14:11:32.596737+0000","last_unstale":"2024-04-25T14:11:32.605403+0000","last_undegraded":"2024-04-25T14:11:32.605403+0000","last_fullsized":"2024-04-25T14:11:32.605403+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:03.787508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.586319+0000","last_change":"2024-04-25T14:11:32.586319+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.586319+0000","last_undegraded":"2024-04-25T14:11:32.586319+0000","last_fullsized":"2024-04-25T14:11:32.586319+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.600618+0000","last_change":"2024-04-25T14:11:32.598361+0000","last_active":"2024-04-25T14:11:32.600618+0000","last_peered":"2024-04-25T14:11:32.600618+0000","last_clean":"2024-04-25T14:11:32.600618+0000","last_became_active":"2024-04-25T14:11:32.597623+0000","last_became_peered":"2024-04-25T14:11:32.597623+0000","last_unstale":"2024-04-25T14:11:32.600618+0000","last_undegraded":"2024-04-25T14:11:32.600618+0000","last_fullsized":"2024-04-25T14:11:32.600618+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:47:20.782498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.586726+0000","last_change":"2024-04-25T14:11:32.586726+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.586726+0000","last_undegraded":"2024-04-25T14:11:32.586726+0000","last_fullsized":"2024-04-25T14:11:32.586726+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.606881+0000","last_change":"2024-04-25T14:11:32.605007+0000","last_active":"2024-04-25T14:11:32.606881+0000","last_peered":"2024-04-25T14:11:32.606881+0000","last_clean":"2024-04-25T14:11:32.606881+0000","last_became_active":"2024-04-25T14:11:32.604678+0000","last_became_peered":"2024-04-25T14:11:32.604678+0000","last_unstale":"2024-04-25T14:11:32.606881+0000","last_undegraded":"2024-04-25T14:11:32.606881+0000","last_fullsized":"2024-04-25T14:11:32.606881+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:09.032042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.581119+0000","last_change":"2024-04-25T14:11:32.581119+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.581119+0000","last_undegraded":"2024-04-25T14:11:32.581119+0000","last_fullsized":"2024-04-25T14:11:32.581119+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.606097+0000","last_change":"2024-04-25T14:11:32.603829+0000","last_active":"2024-04-25T14:11:32.606097+0000","last_peered":"2024-04-25T14:11:32.606097+0000","last_clean":"2024-04-25T14:11:32.606097+0000","last_became_active":"2024-04-25T14:11:32.603424+0000","last_became_peered":"2024-04-25T14:11:32.603424+0000","last_unstale":"2024-04-25T14:11:32.606097+0000","last_undegraded":"2024-04-25T14:11:32.606097+0000","last_fullsized":"2024-04-25T14:11:32.606097+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:03.664590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.579652+0000","last_change":"2024-04-25T14:11:32.579652+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.579652+0000","last_undegraded":"2024-04-25T14:11:32.579652+0000","last_fullsized":"2024-04-25T14:11:32.579652+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.607006+0000","last_change":"2024-04-25T14:11:32.603572+0000","last_active":"2024-04-25T14:11:32.607006+0000","last_peered":"2024-04-25T14:11:32.607006+0000","last_clean":"2024-04-25T14:11:32.607006+0000","last_became_active":"2024-04-25T14:11:32.603270+0000","last_became_peered":"2024-04-25T14:11:32.603270+0000","last_unstale":"2024-04-25T14:11:32.607006+0000","last_undegraded":"2024-04-25T14:11:32.607006+0000","last_fullsized":"2024-04-25T14:11:32.607006+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:14.883395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.589786+0000","last_change":"2024-04-25T14:11:32.589786+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.589786+0000","last_undegraded":"2024-04-25T14:11:32.589786+0000","last_fullsized":"2024-04-25T14:11:32.589786+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.598593+0000","last_change":"2024-04-25T14:11:32.593507+0000","last_active":"2024-04-25T14:11:32.598593+0000","last_peered":"2024-04-25T14:11:32.598593+0000","last_clean":"2024-04-25T14:11:32.598593+0000","last_became_active":"2024-04-25T14:11:32.593173+0000","last_became_peered":"2024-04-25T14:11:32.593173+0000","last_unstale":"2024-04-25T14:11:32.598593+0000","last_undegraded":"2024-04-25T14:11:32.598593+0000","last_fullsized":"2024-04-25T14:11:32.598593+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:30.080066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.599854+0000","last_change":"2024-04-25T14:11:32.599854+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.599854+0000","last_undegraded":"2024-04-25T14:11:32.599854+0000","last_fullsized":"2024-04-25T14:11:32.599854+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.607359+0000","last_change":"2024-04-25T14:11:32.606046+0000","last_active":"2024-04-25T14:11:32.607359+0000","last_peered":"2024-04-25T14:11:32.607359+0000","last_clean":"2024-04-25T14:11:32.607359+0000","last_became_active":"2024-04-25T14:11:32.605755+0000","last_became_peered":"2024-04-25T14:11:32.605755+0000","last_unstale":"2024-04-25T14:11:32.607359+0000","last_undegraded":"2024-04-25T14:11:32.607359+0000","last_fullsized":"2024-04-25T14:11:32.607359+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:07.907191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.585978+0000","last_change":"2024-04-25T14:11:32.585978+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.585978+0000","last_undegraded":"2024-04-25T14:11:32.585978+0000","last_fullsized":"2024-04-25T14:11:32.585978+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.597673+0000","last_change":"2024-04-25T14:11:32.591966+0000","last_active":"2024-04-25T14:11:32.597673+0000","last_peered":"2024-04-25T14:11:32.597673+0000","last_clean":"2024-04-25T14:11:32.597673+0000","last_became_active":"2024-04-25T14:11:32.591633+0000","last_became_peered":"2024-04-25T14:11:32.591633+0000","last_unstale":"2024-04-25T14:11:32.597673+0000","last_undegraded":"2024-04-25T14:11:32.597673+0000","last_fullsized":"2024-04-25T14:11:32.597673+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:28.560866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.595836+0000","last_change":"2024-04-25T14:11:32.595836+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.595836+0000","last_undegraded":"2024-04-25T14:11:32.595836+0000","last_fullsized":"2024-04-25T14:11:32.595836+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.596752+0000","last_change":"2024-04-25T14:11:32.590551+0000","last_active":"2024-04-25T14:11:32.596752+0000","last_peered":"2024-04-25T14:11:32.596752+0000","last_clean":"2024-04-25T14:11:32.596752+0000","last_became_active":"2024-04-25T14:11:32.590233+0000","last_became_peered":"2024-04-25T14:11:32.590233+0000","last_unstale":"2024-04-25T14:11:32.596752+0000","last_undegraded":"2024-04-25T14:11:32.596752+0000","last_fullsized":"2024-04-25T14:11:32.596752+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:01.692524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.585051+0000","last_change":"2024-04-25T14:11:32.585051+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.585051+0000","last_undegraded":"2024-04-25T14:11:32.585051+0000","last_fullsized":"2024-04-25T14:11:32.585051+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.606893+0000","last_change":"2024-04-25T14:11:32.602907+0000","last_active":"2024-04-25T14:11:32.606893+0000","last_peered":"2024-04-25T14:11:32.606893+0000","last_clean":"2024-04-25T14:11:32.606893+0000","last_became_active":"2024-04-25T14:11:32.602600+0000","last_became_peered":"2024-04-25T14:11:32.602600+0000","last_unstale":"2024-04-25T14:11:32.606893+0000","last_undegraded":"2024-04-25T14:11:32.606893+0000","last_fullsized":"2024-04-25T14:11:32.606893+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:00.011427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.591752+0000","last_change":"2024-04-25T14:11:32.591752+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.591752+0000","last_undegraded":"2024-04-25T14:11:32.591752+0000","last_fullsized":"2024-04-25T14:11:32.591752+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.599321+0000","last_change":"2024-04-25T14:11:32.595961+0000","last_active":"2024-04-25T14:11:32.599321+0000","last_peered":"2024-04-25T14:11:32.599321+0000","last_clean":"2024-04-25T14:11:32.599321+0000","last_became_active":"2024-04-25T14:11:32.595641+0000","last_became_peered":"2024-04-25T14:11:32.595641+0000","last_unstale":"2024-04-25T14:11:32.599321+0000","last_undegraded":"2024-04-25T14:11:32.599321+0000","last_fullsized":"2024-04-25T14:11:32.599321+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:07.907191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.580561+0000","last_change":"2024-04-25T14:11:32.580561+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.580561+0000","last_undegraded":"2024-04-25T14:11:32.580561+0000","last_fullsized":"2024-04-25T14:11:32.580561+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.606072+0000","last_change":"2024-04-25T14:11:32.603997+0000","last_active":"2024-04-25T14:11:32.606072+0000","last_peered":"2024-04-25T14:11:32.606072+0000","last_clean":"2024-04-25T14:11:32.606072+0000","last_became_active":"2024-04-25T14:11:32.603630+0000","last_became_peered":"2024-04-25T14:11:32.603630+0000","last_unstale":"2024-04-25T14:11:32.606072+0000","last_undegraded":"2024-04-25T14:11:32.606072+0000","last_fullsized":"2024-04-25T14:11:32.606072+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:57.877724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.585864+0000","last_change":"2024-04-25T14:11:32.585864+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.585864+0000","last_undegraded":"2024-04-25T14:11:32.585864+0000","last_fullsized":"2024-04-25T14:11:32.585864+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.594581+0000","last_change":"2024-04-25T14:11:32.590839+0000","last_active":"2024-04-25T14:11:32.594581+0000","last_peered":"2024-04-25T14:11:32.594581+0000","last_clean":"2024-04-25T14:11:32.594581+0000","last_became_active":"2024-04-25T14:11:32.590349+0000","last_became_peered":"2024-04-25T14:11:32.590349+0000","last_unstale":"2024-04-25T14:11:32.594581+0000","last_undegraded":"2024-04-25T14:11:32.594581+0000","last_fullsized":"2024-04-25T14:11:32.594581+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:57.877724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.580907+0000","last_change":"2024-04-25T14:11:32.580907+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.580907+0000","last_undegraded":"2024-04-25T14:11:32.580907+0000","last_fullsized":"2024-04-25T14:11:32.580907+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.595070+0000","last_change":"2024-04-25T14:11:32.592951+0000","last_active":"2024-04-25T14:11:32.595070+0000","last_peered":"2024-04-25T14:11:32.595070+0000","last_clean":"2024-04-25T14:11:32.595070+0000","last_became_active":"2024-04-25T14:11:32.592568+0000","last_became_peered":"2024-04-25T14:11:32.592568+0000","last_unstale":"2024-04-25T14:11:32.595070+0000","last_undegraded":"2024-04-25T14:11:32.595070+0000","last_fullsized":"2024-04-25T14:11:32.595070+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:55.757498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.590620+0000","last_change":"2024-04-25T14:11:32.590620+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.590620+0000","last_undegraded":"2024-04-25T14:11:32.590620+0000","last_fullsized":"2024-04-25T14:11:32.590620+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.600664+0000","last_change":"2024-04-25T14:11:32.598702+0000","last_active":"2024-04-25T14:11:32.600664+0000","last_peered":"2024-04-25T14:11:32.600664+0000","last_clean":"2024-04-25T14:11:32.600664+0000","last_became_active":"2024-04-25T14:11:32.598372+0000","last_became_peered":"2024-04-25T14:11:32.598372+0000","last_unstale":"2024-04-25T14:11:32.600664+0000","last_undegraded":"2024-04-25T14:11:32.600664+0000","last_fullsized":"2024-04-25T14:11:32.600664+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:10.168462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.600010+0000","last_change":"2024-04-25T14:11:32.600010+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.600010+0000","last_undegraded":"2024-04-25T14:11:32.600010+0000","last_fullsized":"2024-04-25T14:11:32.600010+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.601509+0000","last_change":"2024-04-25T14:11:32.596091+0000","last_active":"2024-04-25T14:11:32.601509+0000","last_peered":"2024-04-25T14:11:32.601509+0000","last_clean":"2024-04-25T14:11:32.601509+0000","last_became_active":"2024-04-25T14:11:32.595711+0000","last_became_peered":"2024-04-25T14:11:32.595711+0000","last_unstale":"2024-04-25T14:11:32.601509+0000","last_undegraded":"2024-04-25T14:11:32.601509+0000","last_fullsized":"2024-04-25T14:11:32.601509+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:21.401742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.582257+0000","last_change":"2024-04-25T14:11:32.582257+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.582257+0000","last_undegraded":"2024-04-25T14:11:32.582257+0000","last_fullsized":"2024-04-25T14:11:32.582257+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.602463+0000","last_change":"2024-04-25T14:11:32.599842+0000","last_active":"2024-04-25T14:11:32.602463+0000","last_peered":"2024-04-25T14:11:32.602463+0000","last_clean":"2024-04-25T14:11:32.602463+0000","last_became_active":"2024-04-25T14:11:32.599226+0000","last_became_peered":"2024-04-25T14:11:32.599226+0000","last_unstale":"2024-04-25T14:11:32.602463+0000","last_undegraded":"2024-04-25T14:11:32.602463+0000","last_fullsized":"2024-04-25T14:11:32.602463+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:03.664590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.583391+0000","last_change":"2024-04-25T14:11:32.583391+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.583391+0000","last_undegraded":"2024-04-25T14:11:32.583391+0000","last_fullsized":"2024-04-25T14:11:32.583391+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.598499+0000","last_change":"2024-04-25T14:11:32.593191+0000","last_active":"2024-04-25T14:11:32.598499+0000","last_peered":"2024-04-25T14:11:32.598499+0000","last_clean":"2024-04-25T14:11:32.598499+0000","last_became_active":"2024-04-25T14:11:32.592628+0000","last_became_peered":"2024-04-25T14:11:32.592628+0000","last_unstale":"2024-04-25T14:11:32.598499+0000","last_undegraded":"2024-04-25T14:11:32.598499+0000","last_fullsized":"2024-04-25T14:11:32.598499+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:20.285653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"129'3277","reported_seq":3750,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.591126+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:11:32.591126+0000","last_peered":"2024-04-25T14:11:32.591126+0000","last_clean":"2024-04-25T14:11:32.591126+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:11:32.591126+0000","last_undegraded":"2024-04-25T14:11:32.591126+0000","last_fullsized":"2024-04-25T14:11:32.591126+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":3277,"log_dups_size":0,"ondisk_log_size":3277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":717,"num_read_kb":1927,"num_write":223,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":717,"num_read_kb":1927,"num_write":223,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3277,"ondisk_log_size":3277,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574939,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29296,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342544,"statfs":{"total":96636764160,"available":96606765056,"internally_reserved":0,"allocated":2326528,"data_stored":1456665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574940,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1732608,"data_stored":866297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32860,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338980,"statfs":{"total":96636764160,"available":96603115520,"internally_reserved":0,"allocated":1720320,"data_stored":859630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574940,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32860,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338980,"statfs":{"total":96636764160,"available":96603115520,"internally_reserved":0,"allocated":1720320,"data_stored":859630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32860,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338980,"statfs":{"total":96636764160,"available":96603115520,"internally_reserved":0,"allocated":1720320,"data_stored":859630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574939,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32860,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338980,"statfs":{"total":96636764160,"available":96603115520,"internally_reserved":0,"allocated":1720320,"data_stored":859630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1732608,"data_stored":866297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29280,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342560,"statfs":{"total":96636764160,"available":96606781440,"internally_reserved":0,"allocated":2314240,"data_stored":1449998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:11:38.113 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:11:38.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.297+0000 7fcb08a00640 1 Processor -- start 2024-04-25T14:11:38.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.297+0000 7fcb08a00640 1 -- start start 2024-04-25T14:11:38.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.297+0000 7fcb08a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcb04071530 0x7fcb041638a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:38.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.297+0000 7fcb08a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb0406dc30 con 0x7fcb0415fe00 2024-04-25T14:11:38.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.297+0000 7fcb08a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb041641f0 con 0x7fcb04163de0 2024-04-25T14:11:38.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.297+0000 7fcb08a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fcb04160210 con 0x7fcb04071530 2024-04-25T14:11:38.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.297+0000 7fcb01e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fcb0415fe00 0x7fcb04070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60490/0 (socket says 172.21.15.86:60490) 2024-04-25T14:11:38.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.297+0000 7fcb01e00640 1 -- 172.21.15.86:0/2749941812 learned_addr learned my addr 172.21.15.86:0/2749941812 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:38.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.297+0000 7fcb01400640 1 --2- 172.21.15.86:0/2749941812 >> v2:172.21.15.155:3300/0 conn(0x7fcb04071530 0x7fcb041638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:38.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.297+0000 7fcb00a00640 1 -- 172.21.15.86:0/2749941812 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 760544646 0 0) 0x7fcb0406dc30 con 0x7fcb0415fe00 2024-04-25T14:11:38.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.297+0000 7fcb00a00640 1 -- 172.21.15.86:0/2749941812 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcae8003530 con 0x7fcb0415fe00 2024-04-25T14:11:38.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.301+0000 7fcb00a00640 1 -- 172.21.15.86:0/2749941812 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3260893814 0 0) 0x7fcae8003530 con 0x7fcb0415fe00 2024-04-25T14:11:38.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.301+0000 7fcb00a00640 1 -- 172.21.15.86:0/2749941812 >> v2:172.21.15.155:3300/0 conn(0x7fcb04071530 msgr2=0x7fcb041638a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:38.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.301+0000 7fcb00a00640 1 --2- 172.21.15.86:0/2749941812 >> v2:172.21.15.155:3300/0 conn(0x7fcb04071530 0x7fcb041638a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:38.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.301+0000 7fcb00a00640 1 -- 172.21.15.86:0/2749941812 >> v1:172.21.15.155:6789/0 conn(0x7fcb04163de0 legacy=0x7fcb040cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:38.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.301+0000 7fcb00a00640 1 -- 172.21.15.86:0/2749941812 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb04164480 con 0x7fcb0415fe00 2024-04-25T14:11:38.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.301+0000 7fcb00a00640 1 -- 172.21.15.86:0/2749941812 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fcaf4001680 con 0x7fcb0415fe00 2024-04-25T14:11:38.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.301+0000 7fcb01400640 1 --2- 172.21.15.86:0/2749941812 >> v2:172.21.15.155:3300/0 conn(0x7fcb04071530 0x7fcb041638a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:11:38.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.301+0000 7fcb00a00640 1 -- 172.21.15.86:0/2749941812 >> v1:172.21.15.86:6789/0 conn(0x7fcb0415fe00 legacy=0x7fcb04070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:38.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.301+0000 7fcb00a00640 1 --2- 172.21.15.86:0/2749941812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcae8003a30 0x7fcae8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:38.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.301+0000 7fcb00a00640 1 --2- 172.21.15.86:0/2749941812 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcae8004810 0x7fcae8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:38.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.301+0000 7fcb00a00640 1 --2- 172.21.15.86:0/2749941812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcae8005230 0x7fcae800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:38.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.301+0000 7fcb00a00640 1 -- 172.21.15.86:0/2749941812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcae800c190 con 0x7fcae8003a30 2024-04-25T14:11:38.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.301+0000 7fcb00a00640 1 -- 172.21.15.86:0/2749941812 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcae800c340 con 0x7fcae8005230 2024-04-25T14:11:38.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.301+0000 7fcb00a00640 1 -- 172.21.15.86:0/2749941812 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcae800c540 con 0x7fcae8004810 2024-04-25T14:11:38.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.301+0000 7fcb01e00640 1 --2- 172.21.15.86:0/2749941812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcae8003a30 0x7fcae8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:38.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.301+0000 7fcb01400640 1 --2- 172.21.15.86:0/2749941812 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcae8004810 0x7fcae8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:38.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.301+0000 7fcb02800640 1 --2- 172.21.15.86:0/2749941812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcae8005230 0x7fcae800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:38.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.301+0000 7fcb01e00640 1 -- 172.21.15.86:0/2749941812 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcae8004810 msgr2=0x7fcae8004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:38.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.301+0000 7fcb01e00640 1 --2- 172.21.15.86:0/2749941812 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcae8004810 0x7fcae8004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:38.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.301+0000 7fcb01e00640 1 -- 172.21.15.86:0/2749941812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcae8005230 msgr2=0x7fcae800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:38.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.301+0000 7fcb01e00640 1 --2- 172.21.15.86:0/2749941812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcae8005230 0x7fcae800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:38.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.301+0000 7fcb01e00640 1 -- 172.21.15.86:0/2749941812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae8003530 con 0x7fcae8003a30 2024-04-25T14:11:38.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.301+0000 7fcb01400640 1 --2- 172.21.15.86:0/2749941812 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcae8004810 0x7fcae8004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:38.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.301+0000 7fcb02800640 1 --2- 172.21.15.86:0/2749941812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcae8005230 0x7fcae800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:38.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.301+0000 7fcb02800640 1 -- 172.21.15.86:0/2749941812 reap_dead start 2024-04-25T14:11:38.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.301+0000 7fcb01e00640 1 --2- 172.21.15.86:0/2749941812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcae8003a30 0x7fcae8003ea0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fcb04163800 tx=0x7fcaf401cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd782953e1fb5f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:38.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.301+0000 7fcb00a00640 1 -- 172.21.15.86:0/2749941812 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4007d00 con 0x7fcae8003a30 2024-04-25T14:11:38.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.305+0000 7fcb00a00640 1 -- 172.21.15.86:0/2749941812 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaf40a9020 con 0x7fcae8003a30 2024-04-25T14:11:38.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.305+0000 7fcb00a00640 1 -- 172.21.15.86:0/2749941812 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4005ae0 con 0x7fcae8003a30 2024-04-25T14:11:38.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.305+0000 7fcb08a00640 1 -- 172.21.15.86:0/2749941812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcae8003a30 msgr2=0x7fcae8003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:38.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.305+0000 7fcb08a00640 1 --2- 172.21.15.86:0/2749941812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcae8003a30 0x7fcae8003ea0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fcb04163800 tx=0x7fcaf401cc00 comp rx=0 tx=0).stop 2024-04-25T14:11:38.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.305+0000 7fcb08a00640 1 -- 172.21.15.86:0/2749941812 shutdown_connections 2024-04-25T14:11:38.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.305+0000 7fcb08a00640 1 --2- 172.21.15.86:0/2749941812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcae8003a30 0x7fcae8003ea0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:38.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.305+0000 7fcb08a00640 1 -- 172.21.15.86:0/2749941812 >> 172.21.15.86:0/2749941812 conn(0x7fcb040cffc0 msgr2=0x7fcb040d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:38.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.305+0000 7fcb08a00640 1 -- 172.21.15.86:0/2749941812 shutdown_connections 2024-04-25T14:11:38.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.305+0000 7fcb08a00640 1 -- 172.21.15.86:0/2749941812 wait complete. 2024-04-25T14:11:38.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.305+0000 7fcb08a00640 1 Processor -- start 2024-04-25T14:11:38.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.305+0000 7fcb08a00640 1 -- start start 2024-04-25T14:11:38.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.309+0000 7fcb08a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcae8003a30 0x7fcb040b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:38.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.309+0000 7fcb08a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb04150d10 0x7fcb04162590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:38.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.309+0000 7fcb08a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb04162ad0 0x7fcb04151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:38.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.309+0000 7fcb08a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcb04072470 con 0x7fcb04150d10 2024-04-25T14:11:38.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.309+0000 7fcb08a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcb040722f0 con 0x7fcae8003a30 2024-04-25T14:11:38.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.309+0000 7fcb08a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcb040725f0 con 0x7fcb04162ad0 2024-04-25T14:11:38.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.309+0000 7fcb01400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb04150d10 0x7fcb04162590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:38.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.309+0000 7fcb01400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb04150d10 0x7fcb04162590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41006/0 (socket says 172.21.15.86:41006) 2024-04-25T14:11:38.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.309+0000 7fcb01400640 1 -- 172.21.15.86:0/1303603161 learned_addr learned my addr 172.21.15.86:0/1303603161 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:38.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.309+0000 7fcb01e00640 1 --2- 172.21.15.86:0/1303603161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcae8003a30 0x7fcb040b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:38.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.309+0000 7fcb02800640 1 --2- 172.21.15.86:0/1303603161 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb04162ad0 0x7fcb04151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:38.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.309+0000 7fcb01400640 1 -- 172.21.15.86:0/1303603161 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb04162ad0 msgr2=0x7fcb04151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:38.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.309+0000 7fcb01400640 1 --2- 172.21.15.86:0/1303603161 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb04162ad0 0x7fcb04151490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:38.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.309+0000 7fcb01400640 1 -- 172.21.15.86:0/1303603161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcae8003a30 msgr2=0x7fcb040b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:38.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.309+0000 7fcb01400640 1 --2- 172.21.15.86:0/1303603161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcae8003a30 0x7fcb040b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:38.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.309+0000 7fcb01400640 1 -- 172.21.15.86:0/1303603161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb04162ee0 con 0x7fcb04150d10 2024-04-25T14:11:38.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.309+0000 7fcb01e00640 1 --2- 172.21.15.86:0/1303603161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcae8003a30 0x7fcb040b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:38.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.309+0000 7fcb01400640 1 --2- 172.21.15.86:0/1303603161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb04150d10 0x7fcb04162590 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fcaec011860 tx=0x7fcaec011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:38.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.309+0000 7fcafa000640 1 -- 172.21.15.86:0/1303603161 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec00fbd0 con 0x7fcb04150d10 2024-04-25T14:11:38.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.309+0000 7fcafa000640 1 -- 172.21.15.86:0/1303603161 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaec00fd70 con 0x7fcb04150d10 2024-04-25T14:11:38.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.309+0000 7fcb08a00640 1 -- 172.21.15.86:0/1303603161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb040b6200 con 0x7fcb04150d10 2024-04-25T14:11:38.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.309+0000 7fcafa000640 1 -- 172.21.15.86:0/1303603161 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec015680 con 0x7fcb04150d10 2024-04-25T14:11:38.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.309+0000 7fcb08a00640 1 -- 172.21.15.86:0/1303603161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb040b67c0 con 0x7fcb04150d10 2024-04-25T14:11:38.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.309+0000 7fcb08a00640 1 -- 172.21.15.86:0/1303603161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb040b8000 con 0x7fcb04150d10 2024-04-25T14:11:38.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.313+0000 7fcafa000640 1 -- 172.21.15.86:0/1303603161 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7fcaec015820 con 0x7fcb04150d10 2024-04-25T14:11:38.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.313+0000 7fcafa000640 1 --2- 172.21.15.86:0/1303603161 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fcae006f520 0x7fcae0071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:38.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.313+0000 7fcafa000640 1 -- 172.21.15.86:0/1303603161 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7fcaec0901a0 con 0x7fcb04150d10 2024-04-25T14:11:38.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.313+0000 7fcb01e00640 1 --2- 172.21.15.86:0/1303603161 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fcae006f520 0x7fcae0071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:38.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.313+0000 7fcb01e00640 1 --2- 172.21.15.86:0/1303603161 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fcae006f520 0x7fcae0071a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcaf40aa000 tx=0x7fcb040d2d70 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:38.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.317+0000 7fcafa000640 1 -- 172.21.15.86:0/1303603161 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcaec019030 con 0x7fcb04150d10 2024-04-25T14:11:38.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.469+0000 7fcb08a00640 1 -- 172.21.15.86:0/1303603161 --> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcb040720e0 con 0x7fcae006f520 2024-04-25T14:11:38.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.473+0000 7fcafa000640 1 -- 172.21.15.86:0/1303603161 <== mgr.9282 v2:172.21.15.86:6832/264890479 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247100 (secure 0 0 0) 0x7fcb040720e0 con 0x7fcae006f520 2024-04-25T14:11:38.474 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:38.475 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:11:38.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.477+0000 7fcb08a00640 1 -- 172.21.15.86:0/1303603161 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fcae006f520 msgr2=0x7fcae0071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:38.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.477+0000 7fcb08a00640 1 --2- 172.21.15.86:0/1303603161 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fcae006f520 0x7fcae0071a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcaf40aa000 tx=0x7fcb040d2d70 comp rx=0 tx=0).stop 2024-04-25T14:11:38.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.477+0000 7fcb08a00640 1 -- 172.21.15.86:0/1303603161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb04150d10 msgr2=0x7fcb04162590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:38.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.477+0000 7fcb08a00640 1 --2- 172.21.15.86:0/1303603161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb04150d10 0x7fcb04162590 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fcaec011860 tx=0x7fcaec011d20 comp rx=0 tx=0).stop 2024-04-25T14:11:38.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.477+0000 7fcb08a00640 1 -- 172.21.15.86:0/1303603161 shutdown_connections 2024-04-25T14:11:38.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.477+0000 7fcb08a00640 1 --2- 172.21.15.86:0/1303603161 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fcae006f520 0x7fcae0071a30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:38.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.477+0000 7fcb08a00640 1 --2- 172.21.15.86:0/1303603161 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb04162ad0 0x7fcb04151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:38.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.477+0000 7fcb08a00640 1 --2- 172.21.15.86:0/1303603161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb04150d10 0x7fcb04162590 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:38.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.477+0000 7fcb08a00640 1 --2- 172.21.15.86:0/1303603161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcae8003a30 0x7fcb040b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:38.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.477+0000 7fcb08a00640 1 -- 172.21.15.86:0/1303603161 >> 172.21.15.86:0/1303603161 conn(0x7fcb040cffc0 msgr2=0x7fcb0406f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:38.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.477+0000 7fcb08a00640 1 -- 172.21.15.86:0/1303603161 shutdown_connections 2024-04-25T14:11:38.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.477+0000 7fcb08a00640 1 -- 172.21.15.86:0/1303603161 wait complete. 2024-04-25T14:11:38.495 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T14:11:36.734296+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":717,"num_read_kb":1927,"num_write":223,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3277,"ondisk_log_size":3277,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":255768,"kb_used_data":14636,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754718952,"statfs":{"total":773094113280,"available":772832206848,"internally_reserved":0,"allocated":14987264,"data_stored":8077777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.166599"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.583286+0000","last_change":"2024-04-25T14:11:32.583286+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.583286+0000","last_undegraded":"2024-04-25T14:11:32.583286+0000","last_fullsized":"2024-04-25T14:11:32.583286+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.593406+0000","last_change":"2024-04-25T14:11:32.593406+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.593406+0000","last_undegraded":"2024-04-25T14:11:32.593406+0000","last_fullsized":"2024-04-25T14:11:32.593406+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.597461+0000","last_change":"2024-04-25T14:11:32.597461+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.597461+0000","last_undegraded":"2024-04-25T14:11:32.597461+0000","last_fullsized":"2024-04-25T14:11:32.597461+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.595145+0000","last_change":"2024-04-25T14:11:32.595145+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.595145+0000","last_undegraded":"2024-04-25T14:11:32.595145+0000","last_fullsized":"2024-04-25T14:11:32.595145+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.590150+0000","last_change":"2024-04-25T14:11:32.590150+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.590150+0000","last_undegraded":"2024-04-25T14:11:32.590150+0000","last_fullsized":"2024-04-25T14:11:32.590150+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.588606+0000","last_change":"2024-04-25T14:11:32.588606+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.588606+0000","last_undegraded":"2024-04-25T14:11:32.588606+0000","last_fullsized":"2024-04-25T14:11:32.588606+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.582342+0000","last_change":"2024-04-25T14:11:32.582342+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.582342+0000","last_undegraded":"2024-04-25T14:11:32.582342+0000","last_fullsized":"2024-04-25T14:11:32.582342+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.583423+0000","last_change":"2024-04-25T14:11:32.583423+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.583423+0000","last_undegraded":"2024-04-25T14:11:32.583423+0000","last_fullsized":"2024-04-25T14:11:32.583423+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.588120+0000","last_change":"2024-04-25T14:11:32.588120+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.588120+0000","last_undegraded":"2024-04-25T14:11:32.588120+0000","last_fullsized":"2024-04-25T14:11:32.588120+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.585963+0000","last_change":"2024-04-25T14:11:32.585963+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.585963+0000","last_undegraded":"2024-04-25T14:11:32.585963+0000","last_fullsized":"2024-04-25T14:11:32.585963+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.588205+0000","last_change":"2024-04-25T14:11:32.588205+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.588205+0000","last_undegraded":"2024-04-25T14:11:32.588205+0000","last_fullsized":"2024-04-25T14:11:32.588205+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.593307+0000","last_change":"2024-04-25T14:11:32.593307+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.593307+0000","last_undegraded":"2024-04-25T14:11:32.593307+0000","last_fullsized":"2024-04-25T14:11:32.593307+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.589718+0000","last_change":"2024-04-25T14:11:32.589718+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.589718+0000","last_undegraded":"2024-04-25T14:11:32.589718+0000","last_fullsized":"2024-04-25T14:11:32.589718+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.596485+0000","last_change":"2024-04-25T14:11:32.596485+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.596485+0000","last_undegraded":"2024-04-25T14:11:32.596485+0000","last_fullsized":"2024-04-25T14:11:32.596485+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.600818+0000","last_change":"2024-04-25T14:11:32.600818+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.600818+0000","last_undegraded":"2024-04-25T14:11:32.600818+0000","last_fullsized":"2024-04-25T14:11:32.600818+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.586259+0000","last_change":"2024-04-25T14:11:32.586259+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.586259+0000","last_undegraded":"2024-04-25T14:11:32.586259+0000","last_fullsized":"2024-04-25T14:11:32.586259+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.594205+0000","last_change":"2024-04-25T14:11:32.594205+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.594205+0000","last_undegraded":"2024-04-25T14:11:32.594205+0000","last_fullsized":"2024-04-25T14:11:32.594205+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.588827+0000","last_change":"2024-04-25T14:11:32.588827+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.588827+0000","last_undegraded":"2024-04-25T14:11:32.588827+0000","last_fullsized":"2024-04-25T14:11:32.588827+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.588488+0000","last_change":"2024-04-25T14:11:32.588488+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.588488+0000","last_undegraded":"2024-04-25T14:11:32.588488+0000","last_fullsized":"2024-04-25T14:11:32.588488+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.585021+0000","last_change":"2024-04-25T14:11:32.585021+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.585021+0000","last_undegraded":"2024-04-25T14:11:32.585021+0000","last_fullsized":"2024-04-25T14:11:32.585021+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.589265+0000","last_change":"2024-04-25T14:11:32.589265+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.589265+0000","last_undegraded":"2024-04-25T14:11:32.589265+0000","last_fullsized":"2024-04-25T14:11:32.589265+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.581795+0000","last_change":"2024-04-25T14:11:32.581795+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.581795+0000","last_undegraded":"2024-04-25T14:11:32.581795+0000","last_fullsized":"2024-04-25T14:11:32.581795+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.583620+0000","last_change":"2024-04-25T14:11:32.583620+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.583620+0000","last_undegraded":"2024-04-25T14:11:32.583620+0000","last_fullsized":"2024-04-25T14:11:32.583620+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.586990+0000","last_change":"2024-04-25T14:11:32.586990+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.586990+0000","last_undegraded":"2024-04-25T14:11:32.586990+0000","last_fullsized":"2024-04-25T14:11:32.586990+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.580738+0000","last_change":"2024-04-25T14:11:32.580738+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.580738+0000","last_undegraded":"2024-04-25T14:11:32.580738+0000","last_fullsized":"2024-04-25T14:11:32.580738+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.581641+0000","last_change":"2024-04-25T14:11:32.581641+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.581641+0000","last_undegraded":"2024-04-25T14:11:32.581641+0000","last_fullsized":"2024-04-25T14:11:32.581641+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.586189+0000","last_change":"2024-04-25T14:11:32.586189+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.586189+0000","last_undegraded":"2024-04-25T14:11:32.586189+0000","last_fullsized":"2024-04-25T14:11:32.586189+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.588712+0000","last_change":"2024-04-25T14:11:32.588712+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.588712+0000","last_undegraded":"2024-04-25T14:11:32.588712+0000","last_fullsized":"2024-04-25T14:11:32.588712+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.582773+0000","last_change":"2024-04-25T14:11:32.582773+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.582773+0000","last_undegraded":"2024-04-25T14:11:32.582773+0000","last_fullsized":"2024-04-25T14:11:32.582773+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.587206+0000","last_change":"2024-04-25T14:11:32.587206+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.587206+0000","last_undegraded":"2024-04-25T14:11:32.587206+0000","last_fullsized":"2024-04-25T14:11:32.587206+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.584884+0000","last_change":"2024-04-25T14:11:32.584884+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.584884+0000","last_undegraded":"2024-04-25T14:11:32.584884+0000","last_fullsized":"2024-04-25T14:11:32.584884+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.582726+0000","last_change":"2024-04-25T14:11:32.582726+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.582726+0000","last_undegraded":"2024-04-25T14:11:32.582726+0000","last_fullsized":"2024-04-25T14:11:32.582726+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.596501+0000","last_change":"2024-04-25T14:11:32.596501+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.596501+0000","last_undegraded":"2024-04-25T14:11:32.596501+0000","last_fullsized":"2024-04-25T14:11:32.596501+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.594681+0000","last_change":"2024-04-25T14:11:32.591452+0000","last_active":"2024-04-25T14:11:32.594681+0000","last_peered":"2024-04-25T14:11:32.594681+0000","last_clean":"2024-04-25T14:11:32.594681+0000","last_became_active":"2024-04-25T14:11:32.591159+0000","last_became_peered":"2024-04-25T14:11:32.591159+0000","last_unstale":"2024-04-25T14:11:32.594681+0000","last_undegraded":"2024-04-25T14:11:32.594681+0000","last_fullsized":"2024-04-25T14:11:32.594681+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:16.063389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.596966+0000","last_change":"2024-04-25T14:11:32.596966+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.596966+0000","last_undegraded":"2024-04-25T14:11:32.596966+0000","last_fullsized":"2024-04-25T14:11:32.596966+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.606856+0000","last_change":"2024-04-25T14:11:32.603965+0000","last_active":"2024-04-25T14:11:32.606856+0000","last_peered":"2024-04-25T14:11:32.606856+0000","last_clean":"2024-04-25T14:11:32.606856+0000","last_became_active":"2024-04-25T14:11:32.602019+0000","last_became_peered":"2024-04-25T14:11:32.602019+0000","last_unstale":"2024-04-25T14:11:32.606856+0000","last_undegraded":"2024-04-25T14:11:32.606856+0000","last_fullsized":"2024-04-25T14:11:32.606856+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:44.182089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.582818+0000","last_change":"2024-04-25T14:11:32.582818+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.582818+0000","last_undegraded":"2024-04-25T14:11:32.582818+0000","last_fullsized":"2024-04-25T14:11:32.582818+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.606798+0000","last_change":"2024-04-25T14:11:32.601614+0000","last_active":"2024-04-25T14:11:32.606798+0000","last_peered":"2024-04-25T14:11:32.606798+0000","last_clean":"2024-04-25T14:11:32.606798+0000","last_became_active":"2024-04-25T14:11:32.601092+0000","last_became_peered":"2024-04-25T14:11:32.601092+0000","last_unstale":"2024-04-25T14:11:32.606798+0000","last_undegraded":"2024-04-25T14:11:32.606798+0000","last_fullsized":"2024-04-25T14:11:32.606798+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:16.063389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.587090+0000","last_change":"2024-04-25T14:11:32.587090+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.587090+0000","last_undegraded":"2024-04-25T14:11:32.587090+0000","last_fullsized":"2024-04-25T14:11:32.587090+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.605321+0000","last_change":"2024-04-25T14:11:32.602711+0000","last_active":"2024-04-25T14:11:32.605321+0000","last_peered":"2024-04-25T14:11:32.605321+0000","last_clean":"2024-04-25T14:11:32.605321+0000","last_became_active":"2024-04-25T14:11:32.602368+0000","last_became_peered":"2024-04-25T14:11:32.602368+0000","last_unstale":"2024-04-25T14:11:32.605321+0000","last_undegraded":"2024-04-25T14:11:32.605321+0000","last_fullsized":"2024-04-25T14:11:32.605321+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:12.830306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.584817+0000","last_change":"2024-04-25T14:11:32.584817+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.584817+0000","last_undegraded":"2024-04-25T14:11:32.584817+0000","last_fullsized":"2024-04-25T14:11:32.584817+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.601151+0000","last_change":"2024-04-25T14:11:32.598778+0000","last_active":"2024-04-25T14:11:32.601151+0000","last_peered":"2024-04-25T14:11:32.601151+0000","last_clean":"2024-04-25T14:11:32.601151+0000","last_became_active":"2024-04-25T14:11:32.598190+0000","last_became_peered":"2024-04-25T14:11:32.598190+0000","last_unstale":"2024-04-25T14:11:32.601151+0000","last_undegraded":"2024-04-25T14:11:32.601151+0000","last_fullsized":"2024-04-25T14:11:32.601151+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:55.757498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.592800+0000","last_change":"2024-04-25T14:11:32.592800+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.592800+0000","last_undegraded":"2024-04-25T14:11:32.592800+0000","last_fullsized":"2024-04-25T14:11:32.592800+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.605858+0000","last_change":"2024-04-25T14:11:32.605097+0000","last_active":"2024-04-25T14:11:32.605858+0000","last_peered":"2024-04-25T14:11:32.605858+0000","last_clean":"2024-04-25T14:11:32.605858+0000","last_became_active":"2024-04-25T14:11:32.604770+0000","last_became_peered":"2024-04-25T14:11:32.604770+0000","last_unstale":"2024-04-25T14:11:32.605858+0000","last_undegraded":"2024-04-25T14:11:32.605858+0000","last_fullsized":"2024-04-25T14:11:32.605858+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:44.182089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.600464+0000","last_change":"2024-04-25T14:11:32.600464+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.600464+0000","last_undegraded":"2024-04-25T14:11:32.600464+0000","last_fullsized":"2024-04-25T14:11:32.600464+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.606597+0000","last_change":"2024-04-25T14:11:32.605101+0000","last_active":"2024-04-25T14:11:32.606597+0000","last_peered":"2024-04-25T14:11:32.606597+0000","last_clean":"2024-04-25T14:11:32.606597+0000","last_became_active":"2024-04-25T14:11:32.604386+0000","last_became_peered":"2024-04-25T14:11:32.604386+0000","last_unstale":"2024-04-25T14:11:32.606597+0000","last_undegraded":"2024-04-25T14:11:32.606597+0000","last_fullsized":"2024-04-25T14:11:32.606597+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:00.011427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.583992+0000","last_change":"2024-04-25T14:11:32.583992+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.583992+0000","last_undegraded":"2024-04-25T14:11:32.583992+0000","last_fullsized":"2024-04-25T14:11:32.583992+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.594681+0000","last_change":"2024-04-25T14:11:32.591107+0000","last_active":"2024-04-25T14:11:32.594681+0000","last_peered":"2024-04-25T14:11:32.594681+0000","last_clean":"2024-04-25T14:11:32.594681+0000","last_became_active":"2024-04-25T14:11:32.590762+0000","last_became_peered":"2024-04-25T14:11:32.590762+0000","last_unstale":"2024-04-25T14:11:32.594681+0000","last_undegraded":"2024-04-25T14:11:32.594681+0000","last_fullsized":"2024-04-25T14:11:32.594681+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:03.664590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.581752+0000","last_change":"2024-04-25T14:11:32.581752+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.581752+0000","last_undegraded":"2024-04-25T14:11:32.581752+0000","last_fullsized":"2024-04-25T14:11:32.581752+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.606645+0000","last_change":"2024-04-25T14:11:32.602319+0000","last_active":"2024-04-25T14:11:32.606645+0000","last_peered":"2024-04-25T14:11:32.606645+0000","last_clean":"2024-04-25T14:11:32.606645+0000","last_became_active":"2024-04-25T14:11:32.601994+0000","last_became_peered":"2024-04-25T14:11:32.601994+0000","last_unstale":"2024-04-25T14:11:32.606645+0000","last_undegraded":"2024-04-25T14:11:32.606645+0000","last_fullsized":"2024-04-25T14:11:32.606645+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:31.082017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.587696+0000","last_change":"2024-04-25T14:11:32.587696+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.587696+0000","last_undegraded":"2024-04-25T14:11:32.587696+0000","last_fullsized":"2024-04-25T14:11:32.587696+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.605694+0000","last_change":"2024-04-25T14:11:32.599420+0000","last_active":"2024-04-25T14:11:32.605694+0000","last_peered":"2024-04-25T14:11:32.605694+0000","last_clean":"2024-04-25T14:11:32.605694+0000","last_became_active":"2024-04-25T14:11:32.599133+0000","last_became_peered":"2024-04-25T14:11:32.599133+0000","last_unstale":"2024-04-25T14:11:32.605694+0000","last_undegraded":"2024-04-25T14:11:32.605694+0000","last_fullsized":"2024-04-25T14:11:32.605694+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:57.130911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.588360+0000","last_change":"2024-04-25T14:11:32.588360+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.588360+0000","last_undegraded":"2024-04-25T14:11:32.588360+0000","last_fullsized":"2024-04-25T14:11:32.588360+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.597686+0000","last_change":"2024-04-25T14:11:32.596904+0000","last_active":"2024-04-25T14:11:32.597686+0000","last_peered":"2024-04-25T14:11:32.597686+0000","last_clean":"2024-04-25T14:11:32.597686+0000","last_became_active":"2024-04-25T14:11:32.595555+0000","last_became_peered":"2024-04-25T14:11:32.595555+0000","last_unstale":"2024-04-25T14:11:32.597686+0000","last_undegraded":"2024-04-25T14:11:32.597686+0000","last_fullsized":"2024-04-25T14:11:32.597686+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:26:39.588788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.582115+0000","last_change":"2024-04-25T14:11:32.582115+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.582115+0000","last_undegraded":"2024-04-25T14:11:32.582115+0000","last_fullsized":"2024-04-25T14:11:32.582115+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.594949+0000","last_change":"2024-04-25T14:11:32.593779+0000","last_active":"2024-04-25T14:11:32.594949+0000","last_peered":"2024-04-25T14:11:32.594949+0000","last_clean":"2024-04-25T14:11:32.594949+0000","last_became_active":"2024-04-25T14:11:32.593185+0000","last_became_peered":"2024-04-25T14:11:32.593185+0000","last_unstale":"2024-04-25T14:11:32.594949+0000","last_undegraded":"2024-04-25T14:11:32.594949+0000","last_fullsized":"2024-04-25T14:11:32.594949+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:31.082017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.600313+0000","last_change":"2024-04-25T14:11:32.600313+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.600313+0000","last_undegraded":"2024-04-25T14:11:32.600313+0000","last_fullsized":"2024-04-25T14:11:32.600313+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.606874+0000","last_change":"2024-04-25T14:11:32.602817+0000","last_active":"2024-04-25T14:11:32.606874+0000","last_peered":"2024-04-25T14:11:32.606874+0000","last_clean":"2024-04-25T14:11:32.606874+0000","last_became_active":"2024-04-25T14:11:32.602507+0000","last_became_peered":"2024-04-25T14:11:32.602507+0000","last_unstale":"2024-04-25T14:11:32.606874+0000","last_undegraded":"2024-04-25T14:11:32.606874+0000","last_fullsized":"2024-04-25T14:11:32.606874+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:57.130911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.585777+0000","last_change":"2024-04-25T14:11:32.585777+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.585777+0000","last_undegraded":"2024-04-25T14:11:32.585777+0000","last_fullsized":"2024-04-25T14:11:32.585777+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.607043+0000","last_change":"2024-04-25T14:11:32.604165+0000","last_active":"2024-04-25T14:11:32.607043+0000","last_peered":"2024-04-25T14:11:32.607043+0000","last_clean":"2024-04-25T14:11:32.607043+0000","last_became_active":"2024-04-25T14:11:32.603846+0000","last_became_peered":"2024-04-25T14:11:32.603846+0000","last_unstale":"2024-04-25T14:11:32.607043+0000","last_undegraded":"2024-04-25T14:11:32.607043+0000","last_fullsized":"2024-04-25T14:11:32.607043+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:51.825452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.596371+0000","last_change":"2024-04-25T14:11:32.596371+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.596371+0000","last_undegraded":"2024-04-25T14:11:32.596371+0000","last_fullsized":"2024-04-25T14:11:32.596371+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.605403+0000","last_change":"2024-04-25T14:11:32.598249+0000","last_active":"2024-04-25T14:11:32.605403+0000","last_peered":"2024-04-25T14:11:32.605403+0000","last_clean":"2024-04-25T14:11:32.605403+0000","last_became_active":"2024-04-25T14:11:32.596737+0000","last_became_peered":"2024-04-25T14:11:32.596737+0000","last_unstale":"2024-04-25T14:11:32.605403+0000","last_undegraded":"2024-04-25T14:11:32.605403+0000","last_fullsized":"2024-04-25T14:11:32.605403+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:03.787508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.586319+0000","last_change":"2024-04-25T14:11:32.586319+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.586319+0000","last_undegraded":"2024-04-25T14:11:32.586319+0000","last_fullsized":"2024-04-25T14:11:32.586319+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.600618+0000","last_change":"2024-04-25T14:11:32.598361+0000","last_active":"2024-04-25T14:11:32.600618+0000","last_peered":"2024-04-25T14:11:32.600618+0000","last_clean":"2024-04-25T14:11:32.600618+0000","last_became_active":"2024-04-25T14:11:32.597623+0000","last_became_peered":"2024-04-25T14:11:32.597623+0000","last_unstale":"2024-04-25T14:11:32.600618+0000","last_undegraded":"2024-04-25T14:11:32.600618+0000","last_fullsized":"2024-04-25T14:11:32.600618+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:47:20.782498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.586726+0000","last_change":"2024-04-25T14:11:32.586726+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.586726+0000","last_undegraded":"2024-04-25T14:11:32.586726+0000","last_fullsized":"2024-04-25T14:11:32.586726+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.606881+0000","last_change":"2024-04-25T14:11:32.605007+0000","last_active":"2024-04-25T14:11:32.606881+0000","last_peered":"2024-04-25T14:11:32.606881+0000","last_clean":"2024-04-25T14:11:32.606881+0000","last_became_active":"2024-04-25T14:11:32.604678+0000","last_became_peered":"2024-04-25T14:11:32.604678+0000","last_unstale":"2024-04-25T14:11:32.606881+0000","last_undegraded":"2024-04-25T14:11:32.606881+0000","last_fullsized":"2024-04-25T14:11:32.606881+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:09.032042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.581119+0000","last_change":"2024-04-25T14:11:32.581119+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.581119+0000","last_undegraded":"2024-04-25T14:11:32.581119+0000","last_fullsized":"2024-04-25T14:11:32.581119+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.606097+0000","last_change":"2024-04-25T14:11:32.603829+0000","last_active":"2024-04-25T14:11:32.606097+0000","last_peered":"2024-04-25T14:11:32.606097+0000","last_clean":"2024-04-25T14:11:32.606097+0000","last_became_active":"2024-04-25T14:11:32.603424+0000","last_became_peered":"2024-04-25T14:11:32.603424+0000","last_unstale":"2024-04-25T14:11:32.606097+0000","last_undegraded":"2024-04-25T14:11:32.606097+0000","last_fullsized":"2024-04-25T14:11:32.606097+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:03.664590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.579652+0000","last_change":"2024-04-25T14:11:32.579652+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.579652+0000","last_undegraded":"2024-04-25T14:11:32.579652+0000","last_fullsized":"2024-04-25T14:11:32.579652+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.607006+0000","last_change":"2024-04-25T14:11:32.603572+0000","last_active":"2024-04-25T14:11:32.607006+0000","last_peered":"2024-04-25T14:11:32.607006+0000","last_clean":"2024-04-25T14:11:32.607006+0000","last_became_active":"2024-04-25T14:11:32.603270+0000","last_became_peered":"2024-04-25T14:11:32.603270+0000","last_unstale":"2024-04-25T14:11:32.607006+0000","last_undegraded":"2024-04-25T14:11:32.607006+0000","last_fullsized":"2024-04-25T14:11:32.607006+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:14.883395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.589786+0000","last_change":"2024-04-25T14:11:32.589786+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.589786+0000","last_undegraded":"2024-04-25T14:11:32.589786+0000","last_fullsized":"2024-04-25T14:11:32.589786+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.598593+0000","last_change":"2024-04-25T14:11:32.593507+0000","last_active":"2024-04-25T14:11:32.598593+0000","last_peered":"2024-04-25T14:11:32.598593+0000","last_clean":"2024-04-25T14:11:32.598593+0000","last_became_active":"2024-04-25T14:11:32.593173+0000","last_became_peered":"2024-04-25T14:11:32.593173+0000","last_unstale":"2024-04-25T14:11:32.598593+0000","last_undegraded":"2024-04-25T14:11:32.598593+0000","last_fullsized":"2024-04-25T14:11:32.598593+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:30.080066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.599854+0000","last_change":"2024-04-25T14:11:32.599854+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.599854+0000","last_undegraded":"2024-04-25T14:11:32.599854+0000","last_fullsized":"2024-04-25T14:11:32.599854+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.607359+0000","last_change":"2024-04-25T14:11:32.606046+0000","last_active":"2024-04-25T14:11:32.607359+0000","last_peered":"2024-04-25T14:11:32.607359+0000","last_clean":"2024-04-25T14:11:32.607359+0000","last_became_active":"2024-04-25T14:11:32.605755+0000","last_became_peered":"2024-04-25T14:11:32.605755+0000","last_unstale":"2024-04-25T14:11:32.607359+0000","last_undegraded":"2024-04-25T14:11:32.607359+0000","last_fullsized":"2024-04-25T14:11:32.607359+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:07.907191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.585978+0000","last_change":"2024-04-25T14:11:32.585978+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.585978+0000","last_undegraded":"2024-04-25T14:11:32.585978+0000","last_fullsized":"2024-04-25T14:11:32.585978+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.597673+0000","last_change":"2024-04-25T14:11:32.591966+0000","last_active":"2024-04-25T14:11:32.597673+0000","last_peered":"2024-04-25T14:11:32.597673+0000","last_clean":"2024-04-25T14:11:32.597673+0000","last_became_active":"2024-04-25T14:11:32.591633+0000","last_became_peered":"2024-04-25T14:11:32.591633+0000","last_unstale":"2024-04-25T14:11:32.597673+0000","last_undegraded":"2024-04-25T14:11:32.597673+0000","last_fullsized":"2024-04-25T14:11:32.597673+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:28.560866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.595836+0000","last_change":"2024-04-25T14:11:32.595836+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.595836+0000","last_undegraded":"2024-04-25T14:11:32.595836+0000","last_fullsized":"2024-04-25T14:11:32.595836+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.596752+0000","last_change":"2024-04-25T14:11:32.590551+0000","last_active":"2024-04-25T14:11:32.596752+0000","last_peered":"2024-04-25T14:11:32.596752+0000","last_clean":"2024-04-25T14:11:32.596752+0000","last_became_active":"2024-04-25T14:11:32.590233+0000","last_became_peered":"2024-04-25T14:11:32.590233+0000","last_unstale":"2024-04-25T14:11:32.596752+0000","last_undegraded":"2024-04-25T14:11:32.596752+0000","last_fullsized":"2024-04-25T14:11:32.596752+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:01.692524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.585051+0000","last_change":"2024-04-25T14:11:32.585051+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.585051+0000","last_undegraded":"2024-04-25T14:11:32.585051+0000","last_fullsized":"2024-04-25T14:11:32.585051+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.606893+0000","last_change":"2024-04-25T14:11:32.602907+0000","last_active":"2024-04-25T14:11:32.606893+0000","last_peered":"2024-04-25T14:11:32.606893+0000","last_clean":"2024-04-25T14:11:32.606893+0000","last_became_active":"2024-04-25T14:11:32.602600+0000","last_became_peered":"2024-04-25T14:11:32.602600+0000","last_unstale":"2024-04-25T14:11:32.606893+0000","last_undegraded":"2024-04-25T14:11:32.606893+0000","last_fullsized":"2024-04-25T14:11:32.606893+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:00.011427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.591752+0000","last_change":"2024-04-25T14:11:32.591752+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.591752+0000","last_undegraded":"2024-04-25T14:11:32.591752+0000","last_fullsized":"2024-04-25T14:11:32.591752+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.599321+0000","last_change":"2024-04-25T14:11:32.595961+0000","last_active":"2024-04-25T14:11:32.599321+0000","last_peered":"2024-04-25T14:11:32.599321+0000","last_clean":"2024-04-25T14:11:32.599321+0000","last_became_active":"2024-04-25T14:11:32.595641+0000","last_became_peered":"2024-04-25T14:11:32.595641+0000","last_unstale":"2024-04-25T14:11:32.599321+0000","last_undegraded":"2024-04-25T14:11:32.599321+0000","last_fullsized":"2024-04-25T14:11:32.599321+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:07.907191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.580561+0000","last_change":"2024-04-25T14:11:32.580561+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.580561+0000","last_undegraded":"2024-04-25T14:11:32.580561+0000","last_fullsized":"2024-04-25T14:11:32.580561+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.606072+0000","last_change":"2024-04-25T14:11:32.603997+0000","last_active":"2024-04-25T14:11:32.606072+0000","last_peered":"2024-04-25T14:11:32.606072+0000","last_clean":"2024-04-25T14:11:32.606072+0000","last_became_active":"2024-04-25T14:11:32.603630+0000","last_became_peered":"2024-04-25T14:11:32.603630+0000","last_unstale":"2024-04-25T14:11:32.606072+0000","last_undegraded":"2024-04-25T14:11:32.606072+0000","last_fullsized":"2024-04-25T14:11:32.606072+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:57.877724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.585864+0000","last_change":"2024-04-25T14:11:32.585864+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.585864+0000","last_undegraded":"2024-04-25T14:11:32.585864+0000","last_fullsized":"2024-04-25T14:11:32.585864+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.594581+0000","last_change":"2024-04-25T14:11:32.590839+0000","last_active":"2024-04-25T14:11:32.594581+0000","last_peered":"2024-04-25T14:11:32.594581+0000","last_clean":"2024-04-25T14:11:32.594581+0000","last_became_active":"2024-04-25T14:11:32.590349+0000","last_became_peered":"2024-04-25T14:11:32.590349+0000","last_unstale":"2024-04-25T14:11:32.594581+0000","last_undegraded":"2024-04-25T14:11:32.594581+0000","last_fullsized":"2024-04-25T14:11:32.594581+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:57.877724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.580907+0000","last_change":"2024-04-25T14:11:32.580907+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.580907+0000","last_undegraded":"2024-04-25T14:11:32.580907+0000","last_fullsized":"2024-04-25T14:11:32.580907+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.595070+0000","last_change":"2024-04-25T14:11:32.592951+0000","last_active":"2024-04-25T14:11:32.595070+0000","last_peered":"2024-04-25T14:11:32.595070+0000","last_clean":"2024-04-25T14:11:32.595070+0000","last_became_active":"2024-04-25T14:11:32.592568+0000","last_became_peered":"2024-04-25T14:11:32.592568+0000","last_unstale":"2024-04-25T14:11:32.595070+0000","last_undegraded":"2024-04-25T14:11:32.595070+0000","last_fullsized":"2024-04-25T14:11:32.595070+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:55.757498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.590620+0000","last_change":"2024-04-25T14:11:32.590620+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.590620+0000","last_undegraded":"2024-04-25T14:11:32.590620+0000","last_fullsized":"2024-04-25T14:11:32.590620+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.600664+0000","last_change":"2024-04-25T14:11:32.598702+0000","last_active":"2024-04-25T14:11:32.600664+0000","last_peered":"2024-04-25T14:11:32.600664+0000","last_clean":"2024-04-25T14:11:32.600664+0000","last_became_active":"2024-04-25T14:11:32.598372+0000","last_became_peered":"2024-04-25T14:11:32.598372+0000","last_unstale":"2024-04-25T14:11:32.600664+0000","last_undegraded":"2024-04-25T14:11:32.600664+0000","last_fullsized":"2024-04-25T14:11:32.600664+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:10.168462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.600010+0000","last_change":"2024-04-25T14:11:32.600010+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.600010+0000","last_undegraded":"2024-04-25T14:11:32.600010+0000","last_fullsized":"2024-04-25T14:11:32.600010+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.601509+0000","last_change":"2024-04-25T14:11:32.596091+0000","last_active":"2024-04-25T14:11:32.601509+0000","last_peered":"2024-04-25T14:11:32.601509+0000","last_clean":"2024-04-25T14:11:32.601509+0000","last_became_active":"2024-04-25T14:11:32.595711+0000","last_became_peered":"2024-04-25T14:11:32.595711+0000","last_unstale":"2024-04-25T14:11:32.601509+0000","last_undegraded":"2024-04-25T14:11:32.601509+0000","last_fullsized":"2024-04-25T14:11:32.601509+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:21.401742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.582257+0000","last_change":"2024-04-25T14:11:32.582257+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.582257+0000","last_undegraded":"2024-04-25T14:11:32.582257+0000","last_fullsized":"2024-04-25T14:11:32.582257+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.602463+0000","last_change":"2024-04-25T14:11:32.599842+0000","last_active":"2024-04-25T14:11:32.602463+0000","last_peered":"2024-04-25T14:11:32.602463+0000","last_clean":"2024-04-25T14:11:32.602463+0000","last_became_active":"2024-04-25T14:11:32.599226+0000","last_became_peered":"2024-04-25T14:11:32.599226+0000","last_unstale":"2024-04-25T14:11:32.602463+0000","last_undegraded":"2024-04-25T14:11:32.602463+0000","last_fullsized":"2024-04-25T14:11:32.602463+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:03.664590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2024-04-25T14:11:32.583391+0000","last_change":"2024-04-25T14:11:32.583391+0000","last_active":"2024-04-25T14:11:32.567346+0000","last_peered":"2024-04-25T14:11:32.567346+0000","last_clean":"2024-04-25T14:11:32.567346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:11:32.583391+0000","last_undegraded":"2024-04-25T14:11:32.583391+0000","last_fullsized":"2024-04-25T14:11:32.583391+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.598499+0000","last_change":"2024-04-25T14:11:32.593191+0000","last_active":"2024-04-25T14:11:32.598499+0000","last_peered":"2024-04-25T14:11:32.598499+0000","last_clean":"2024-04-25T14:11:32.598499+0000","last_became_active":"2024-04-25T14:11:32.592628+0000","last_became_peered":"2024-04-25T14:11:32.592628+0000","last_unstale":"2024-04-25T14:11:32.598499+0000","last_undegraded":"2024-04-25T14:11:32.598499+0000","last_fullsized":"2024-04-25T14:11:32.598499+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:20.285653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"129'3277","reported_seq":3750,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-04-25T14:11:32.591126+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:11:32.591126+0000","last_peered":"2024-04-25T14:11:32.591126+0000","last_clean":"2024-04-25T14:11:32.591126+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:11:32.591126+0000","last_undegraded":"2024-04-25T14:11:32.591126+0000","last_fullsized":"2024-04-25T14:11:32.591126+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":3277,"log_dups_size":0,"ondisk_log_size":3277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":717,"num_read_kb":1927,"num_write":223,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":717,"num_read_kb":1927,"num_write":223,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3277,"ondisk_log_size":3277,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574939,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29296,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342544,"statfs":{"total":96636764160,"available":96606765056,"internally_reserved":0,"allocated":2326528,"data_stored":1456665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574940,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1732608,"data_stored":866297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32860,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338980,"statfs":{"total":96636764160,"available":96603115520,"internally_reserved":0,"allocated":1720320,"data_stored":859630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574940,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32860,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338980,"statfs":{"total":96636764160,"available":96603115520,"internally_reserved":0,"allocated":1720320,"data_stored":859630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32860,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338980,"statfs":{"total":96636764160,"available":96603115520,"internally_reserved":0,"allocated":1720320,"data_stored":859630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574939,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32860,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338980,"statfs":{"total":96636764160,"available":96603115520,"internally_reserved":0,"allocated":1720320,"data_stored":859630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1732608,"data_stored":866297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29280,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342560,"statfs":{"total":96636764160,"available":96606781440,"internally_reserved":0,"allocated":2314240,"data_stored":1449998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:11:38.498 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3f is not active+clean 2024-04-25T14:11:38.498 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.583286+0000', 'last_change': '2024-04-25T14:11:32.583286+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.583286+0000', 'last_undegraded': '2024-04-25T14:11:32.583286+0000', 'last_fullsized': '2024-04-25T14:11:32.583286+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:11:38.498 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3e is not active+clean 2024-04-25T14:11:38.498 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.593406+0000', 'last_change': '2024-04-25T14:11:32.593406+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.593406+0000', 'last_undegraded': '2024-04-25T14:11:32.593406+0000', 'last_fullsized': '2024-04-25T14:11:32.593406+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:11:38.498 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3d is not active+clean 2024-04-25T14:11:38.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.597461+0000', 'last_change': '2024-04-25T14:11:32.597461+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.597461+0000', 'last_undegraded': '2024-04-25T14:11:32.597461+0000', 'last_fullsized': '2024-04-25T14:11:32.597461+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:11:38.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3c is not active+clean 2024-04-25T14:11:38.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.595145+0000', 'last_change': '2024-04-25T14:11:32.595145+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.595145+0000', 'last_undegraded': '2024-04-25T14:11:32.595145+0000', 'last_fullsized': '2024-04-25T14:11:32.595145+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:11:38.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3b is not active+clean 2024-04-25T14:11:38.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.590150+0000', 'last_change': '2024-04-25T14:11:32.590150+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.590150+0000', 'last_undegraded': '2024-04-25T14:11:32.590150+0000', 'last_fullsized': '2024-04-25T14:11:32.590150+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:11:38.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3a is not active+clean 2024-04-25T14:11:38.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.588606+0000', 'last_change': '2024-04-25T14:11:32.588606+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.588606+0000', 'last_undegraded': '2024-04-25T14:11:32.588606+0000', 'last_fullsized': '2024-04-25T14:11:32.588606+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:11:38.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.39 is not active+clean 2024-04-25T14:11:38.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.582342+0000', 'last_change': '2024-04-25T14:11:32.582342+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.582342+0000', 'last_undegraded': '2024-04-25T14:11:32.582342+0000', 'last_fullsized': '2024-04-25T14:11:32.582342+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:11:38.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.38 is not active+clean 2024-04-25T14:11:38.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.583423+0000', 'last_change': '2024-04-25T14:11:32.583423+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.583423+0000', 'last_undegraded': '2024-04-25T14:11:32.583423+0000', 'last_fullsized': '2024-04-25T14:11:32.583423+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:11:38.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.37 is not active+clean 2024-04-25T14:11:38.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.588120+0000', 'last_change': '2024-04-25T14:11:32.588120+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.588120+0000', 'last_undegraded': '2024-04-25T14:11:32.588120+0000', 'last_fullsized': '2024-04-25T14:11:32.588120+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:11:38.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.36 is not active+clean 2024-04-25T14:11:38.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.585963+0000', 'last_change': '2024-04-25T14:11:32.585963+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.585963+0000', 'last_undegraded': '2024-04-25T14:11:32.585963+0000', 'last_fullsized': '2024-04-25T14:11:32.585963+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:11:38.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.35 is not active+clean 2024-04-25T14:11:38.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.588205+0000', 'last_change': '2024-04-25T14:11:32.588205+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.588205+0000', 'last_undegraded': '2024-04-25T14:11:32.588205+0000', 'last_fullsized': '2024-04-25T14:11:32.588205+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:11:38.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.34 is not active+clean 2024-04-25T14:11:38.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.593307+0000', 'last_change': '2024-04-25T14:11:32.593307+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.593307+0000', 'last_undegraded': '2024-04-25T14:11:32.593307+0000', 'last_fullsized': '2024-04-25T14:11:32.593307+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:11:38.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.33 is not active+clean 2024-04-25T14:11:38.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.589718+0000', 'last_change': '2024-04-25T14:11:32.589718+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.589718+0000', 'last_undegraded': '2024-04-25T14:11:32.589718+0000', 'last_fullsized': '2024-04-25T14:11:32.589718+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:11:38.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.32 is not active+clean 2024-04-25T14:11:38.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.596485+0000', 'last_change': '2024-04-25T14:11:32.596485+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.596485+0000', 'last_undegraded': '2024-04-25T14:11:32.596485+0000', 'last_fullsized': '2024-04-25T14:11:32.596485+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:11:38.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.31 is not active+clean 2024-04-25T14:11:38.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.600818+0000', 'last_change': '2024-04-25T14:11:32.600818+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.600818+0000', 'last_undegraded': '2024-04-25T14:11:32.600818+0000', 'last_fullsized': '2024-04-25T14:11:32.600818+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:11:38.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.30 is not active+clean 2024-04-25T14:11:38.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.586259+0000', 'last_change': '2024-04-25T14:11:32.586259+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.586259+0000', 'last_undegraded': '2024-04-25T14:11:32.586259+0000', 'last_fullsized': '2024-04-25T14:11:32.586259+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:11:38.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2f is not active+clean 2024-04-25T14:11:38.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.594205+0000', 'last_change': '2024-04-25T14:11:32.594205+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.594205+0000', 'last_undegraded': '2024-04-25T14:11:32.594205+0000', 'last_fullsized': '2024-04-25T14:11:32.594205+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:11:38.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2e is not active+clean 2024-04-25T14:11:38.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.588827+0000', 'last_change': '2024-04-25T14:11:32.588827+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.588827+0000', 'last_undegraded': '2024-04-25T14:11:32.588827+0000', 'last_fullsized': '2024-04-25T14:11:32.588827+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:11:38.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2d is not active+clean 2024-04-25T14:11:38.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.588488+0000', 'last_change': '2024-04-25T14:11:32.588488+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.588488+0000', 'last_undegraded': '2024-04-25T14:11:32.588488+0000', 'last_fullsized': '2024-04-25T14:11:32.588488+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:11:38.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2c is not active+clean 2024-04-25T14:11:38.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.585021+0000', 'last_change': '2024-04-25T14:11:32.585021+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.585021+0000', 'last_undegraded': '2024-04-25T14:11:32.585021+0000', 'last_fullsized': '2024-04-25T14:11:32.585021+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:11:38.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2b is not active+clean 2024-04-25T14:11:38.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.589265+0000', 'last_change': '2024-04-25T14:11:32.589265+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.589265+0000', 'last_undegraded': '2024-04-25T14:11:32.589265+0000', 'last_fullsized': '2024-04-25T14:11:32.589265+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:11:38.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2a is not active+clean 2024-04-25T14:11:38.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.581795+0000', 'last_change': '2024-04-25T14:11:32.581795+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.581795+0000', 'last_undegraded': '2024-04-25T14:11:32.581795+0000', 'last_fullsized': '2024-04-25T14:11:32.581795+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:11:38.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.29 is not active+clean 2024-04-25T14:11:38.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.583620+0000', 'last_change': '2024-04-25T14:11:32.583620+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.583620+0000', 'last_undegraded': '2024-04-25T14:11:32.583620+0000', 'last_fullsized': '2024-04-25T14:11:32.583620+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:11:38.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.28 is not active+clean 2024-04-25T14:11:38.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.586990+0000', 'last_change': '2024-04-25T14:11:32.586990+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.586990+0000', 'last_undegraded': '2024-04-25T14:11:32.586990+0000', 'last_fullsized': '2024-04-25T14:11:32.586990+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:11:38.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.27 is not active+clean 2024-04-25T14:11:38.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.580738+0000', 'last_change': '2024-04-25T14:11:32.580738+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.580738+0000', 'last_undegraded': '2024-04-25T14:11:32.580738+0000', 'last_fullsized': '2024-04-25T14:11:32.580738+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:11:38.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.26 is not active+clean 2024-04-25T14:11:38.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.581641+0000', 'last_change': '2024-04-25T14:11:32.581641+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.581641+0000', 'last_undegraded': '2024-04-25T14:11:32.581641+0000', 'last_fullsized': '2024-04-25T14:11:32.581641+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:11:38.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.25 is not active+clean 2024-04-25T14:11:38.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.586189+0000', 'last_change': '2024-04-25T14:11:32.586189+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.586189+0000', 'last_undegraded': '2024-04-25T14:11:32.586189+0000', 'last_fullsized': '2024-04-25T14:11:32.586189+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:11:38.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.24 is not active+clean 2024-04-25T14:11:38.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.588712+0000', 'last_change': '2024-04-25T14:11:32.588712+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.588712+0000', 'last_undegraded': '2024-04-25T14:11:32.588712+0000', 'last_fullsized': '2024-04-25T14:11:32.588712+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:11:38.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.23 is not active+clean 2024-04-25T14:11:38.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.582773+0000', 'last_change': '2024-04-25T14:11:32.582773+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.582773+0000', 'last_undegraded': '2024-04-25T14:11:32.582773+0000', 'last_fullsized': '2024-04-25T14:11:32.582773+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:11:38.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.22 is not active+clean 2024-04-25T14:11:38.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.587206+0000', 'last_change': '2024-04-25T14:11:32.587206+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.587206+0000', 'last_undegraded': '2024-04-25T14:11:32.587206+0000', 'last_fullsized': '2024-04-25T14:11:32.587206+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:11:38.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.21 is not active+clean 2024-04-25T14:11:38.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.584884+0000', 'last_change': '2024-04-25T14:11:32.584884+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.584884+0000', 'last_undegraded': '2024-04-25T14:11:32.584884+0000', 'last_fullsized': '2024-04-25T14:11:32.584884+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:11:38.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.20 is not active+clean 2024-04-25T14:11:38.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.582726+0000', 'last_change': '2024-04-25T14:11:32.582726+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.582726+0000', 'last_undegraded': '2024-04-25T14:11:32.582726+0000', 'last_fullsized': '2024-04-25T14:11:32.582726+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:11:38.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1c is not active+clean 2024-04-25T14:11:38.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.596501+0000', 'last_change': '2024-04-25T14:11:32.596501+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.596501+0000', 'last_undegraded': '2024-04-25T14:11:32.596501+0000', 'last_fullsized': '2024-04-25T14:11:32.596501+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:11:38.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1d is not active+clean 2024-04-25T14:11:38.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.596966+0000', 'last_change': '2024-04-25T14:11:32.596966+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.596966+0000', 'last_undegraded': '2024-04-25T14:11:32.596966+0000', 'last_fullsized': '2024-04-25T14:11:32.596966+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:11:38.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1e is not active+clean 2024-04-25T14:11:38.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.582818+0000', 'last_change': '2024-04-25T14:11:32.582818+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.582818+0000', 'last_undegraded': '2024-04-25T14:11:32.582818+0000', 'last_fullsized': '2024-04-25T14:11:32.582818+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:11:38.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1f is not active+clean 2024-04-25T14:11:38.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.587090+0000', 'last_change': '2024-04-25T14:11:32.587090+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.587090+0000', 'last_undegraded': '2024-04-25T14:11:32.587090+0000', 'last_fullsized': '2024-04-25T14:11:32.587090+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:11:38.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.18 is not active+clean 2024-04-25T14:11:38.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.584817+0000', 'last_change': '2024-04-25T14:11:32.584817+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.584817+0000', 'last_undegraded': '2024-04-25T14:11:32.584817+0000', 'last_fullsized': '2024-04-25T14:11:32.584817+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:11:38.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.19 is not active+clean 2024-04-25T14:11:38.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.592800+0000', 'last_change': '2024-04-25T14:11:32.592800+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.592800+0000', 'last_undegraded': '2024-04-25T14:11:32.592800+0000', 'last_fullsized': '2024-04-25T14:11:32.592800+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:11:38.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1a is not active+clean 2024-04-25T14:11:38.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.600464+0000', 'last_change': '2024-04-25T14:11:32.600464+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.600464+0000', 'last_undegraded': '2024-04-25T14:11:32.600464+0000', 'last_fullsized': '2024-04-25T14:11:32.600464+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:11:38.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1b is not active+clean 2024-04-25T14:11:38.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.583992+0000', 'last_change': '2024-04-25T14:11:32.583992+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.583992+0000', 'last_undegraded': '2024-04-25T14:11:32.583992+0000', 'last_fullsized': '2024-04-25T14:11:32.583992+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:11:38.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.14 is not active+clean 2024-04-25T14:11:38.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.581752+0000', 'last_change': '2024-04-25T14:11:32.581752+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.581752+0000', 'last_undegraded': '2024-04-25T14:11:32.581752+0000', 'last_fullsized': '2024-04-25T14:11:32.581752+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:11:38.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.15 is not active+clean 2024-04-25T14:11:38.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.587696+0000', 'last_change': '2024-04-25T14:11:32.587696+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.587696+0000', 'last_undegraded': '2024-04-25T14:11:32.587696+0000', 'last_fullsized': '2024-04-25T14:11:32.587696+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:11:38.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.16 is not active+clean 2024-04-25T14:11:38.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.588360+0000', 'last_change': '2024-04-25T14:11:32.588360+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.588360+0000', 'last_undegraded': '2024-04-25T14:11:32.588360+0000', 'last_fullsized': '2024-04-25T14:11:32.588360+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:11:38.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.10 is not active+clean 2024-04-25T14:11:38.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.582115+0000', 'last_change': '2024-04-25T14:11:32.582115+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.582115+0000', 'last_undegraded': '2024-04-25T14:11:32.582115+0000', 'last_fullsized': '2024-04-25T14:11:32.582115+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:11:38.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.11 is not active+clean 2024-04-25T14:11:38.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.600313+0000', 'last_change': '2024-04-25T14:11:32.600313+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.600313+0000', 'last_undegraded': '2024-04-25T14:11:32.600313+0000', 'last_fullsized': '2024-04-25T14:11:32.600313+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:11:38.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.12 is not active+clean 2024-04-25T14:11:38.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.585777+0000', 'last_change': '2024-04-25T14:11:32.585777+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.585777+0000', 'last_undegraded': '2024-04-25T14:11:32.585777+0000', 'last_fullsized': '2024-04-25T14:11:32.585777+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:11:38.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.13 is not active+clean 2024-04-25T14:11:38.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.596371+0000', 'last_change': '2024-04-25T14:11:32.596371+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.596371+0000', 'last_undegraded': '2024-04-25T14:11:32.596371+0000', 'last_fullsized': '2024-04-25T14:11:32.596371+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:11:38.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.c is not active+clean 2024-04-25T14:11:38.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.586319+0000', 'last_change': '2024-04-25T14:11:32.586319+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.586319+0000', 'last_undegraded': '2024-04-25T14:11:32.586319+0000', 'last_fullsized': '2024-04-25T14:11:32.586319+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:11:38.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.d is not active+clean 2024-04-25T14:11:38.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.586726+0000', 'last_change': '2024-04-25T14:11:32.586726+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.586726+0000', 'last_undegraded': '2024-04-25T14:11:32.586726+0000', 'last_fullsized': '2024-04-25T14:11:32.586726+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:11:38.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.e is not active+clean 2024-04-25T14:11:38.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.581119+0000', 'last_change': '2024-04-25T14:11:32.581119+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.581119+0000', 'last_undegraded': '2024-04-25T14:11:32.581119+0000', 'last_fullsized': '2024-04-25T14:11:32.581119+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:11:38.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.f is not active+clean 2024-04-25T14:11:38.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.579652+0000', 'last_change': '2024-04-25T14:11:32.579652+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.579652+0000', 'last_undegraded': '2024-04-25T14:11:32.579652+0000', 'last_fullsized': '2024-04-25T14:11:32.579652+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:11:38.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.8 is not active+clean 2024-04-25T14:11:38.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.589786+0000', 'last_change': '2024-04-25T14:11:32.589786+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.589786+0000', 'last_undegraded': '2024-04-25T14:11:32.589786+0000', 'last_fullsized': '2024-04-25T14:11:32.589786+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:11:38.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.9 is not active+clean 2024-04-25T14:11:38.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.599854+0000', 'last_change': '2024-04-25T14:11:32.599854+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.599854+0000', 'last_undegraded': '2024-04-25T14:11:32.599854+0000', 'last_fullsized': '2024-04-25T14:11:32.599854+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:11:38.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.a is not active+clean 2024-04-25T14:11:38.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.585978+0000', 'last_change': '2024-04-25T14:11:32.585978+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.585978+0000', 'last_undegraded': '2024-04-25T14:11:32.585978+0000', 'last_fullsized': '2024-04-25T14:11:32.585978+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:11:38.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.b is not active+clean 2024-04-25T14:11:38.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.595836+0000', 'last_change': '2024-04-25T14:11:32.595836+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.595836+0000', 'last_undegraded': '2024-04-25T14:11:32.595836+0000', 'last_fullsized': '2024-04-25T14:11:32.595836+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:11:38.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.4 is not active+clean 2024-04-25T14:11:38.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.585051+0000', 'last_change': '2024-04-25T14:11:32.585051+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.585051+0000', 'last_undegraded': '2024-04-25T14:11:32.585051+0000', 'last_fullsized': '2024-04-25T14:11:32.585051+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:11:38.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.5 is not active+clean 2024-04-25T14:11:38.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.591752+0000', 'last_change': '2024-04-25T14:11:32.591752+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.591752+0000', 'last_undegraded': '2024-04-25T14:11:32.591752+0000', 'last_fullsized': '2024-04-25T14:11:32.591752+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:11:38.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.6 is not active+clean 2024-04-25T14:11:38.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.580561+0000', 'last_change': '2024-04-25T14:11:32.580561+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.580561+0000', 'last_undegraded': '2024-04-25T14:11:32.580561+0000', 'last_fullsized': '2024-04-25T14:11:32.580561+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:11:38.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.7 is not active+clean 2024-04-25T14:11:38.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.585864+0000', 'last_change': '2024-04-25T14:11:32.585864+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.585864+0000', 'last_undegraded': '2024-04-25T14:11:32.585864+0000', 'last_fullsized': '2024-04-25T14:11:32.585864+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:11:38.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.0 is not active+clean 2024-04-25T14:11:38.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.580907+0000', 'last_change': '2024-04-25T14:11:32.580907+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.580907+0000', 'last_undegraded': '2024-04-25T14:11:32.580907+0000', 'last_fullsized': '2024-04-25T14:11:32.580907+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:11:38.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1 is not active+clean 2024-04-25T14:11:38.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.590620+0000', 'last_change': '2024-04-25T14:11:32.590620+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.590620+0000', 'last_undegraded': '2024-04-25T14:11:32.590620+0000', 'last_fullsized': '2024-04-25T14:11:32.590620+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:11:38.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2 is not active+clean 2024-04-25T14:11:38.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.600010+0000', 'last_change': '2024-04-25T14:11:32.600010+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.600010+0000', 'last_undegraded': '2024-04-25T14:11:32.600010+0000', 'last_fullsized': '2024-04-25T14:11:32.600010+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:11:38.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3 is not active+clean 2024-04-25T14:11:38.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.582257+0000', 'last_change': '2024-04-25T14:11:32.582257+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.582257+0000', 'last_undegraded': '2024-04-25T14:11:32.582257+0000', 'last_fullsized': '2024-04-25T14:11:32.582257+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:11:38.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.17 is not active+clean 2024-04-25T14:11:38.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 138, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:11:32.583391+0000', 'last_change': '2024-04-25T14:11:32.583391+0000', 'last_active': '2024-04-25T14:11:32.567346+0000', 'last_peered': '2024-04-25T14:11:32.567346+0000', 'last_clean': '2024-04-25T14:11:32.567346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:11:32.583391+0000', 'last_undegraded': '2024-04-25T14:11:32.583391+0000', 'last_fullsized': '2024-04-25T14:11:32.583391+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'last_clean_scrub_stamp': '2024-04-25T14:11:32.567346+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:11:38.509 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T14:11:38.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.689+0000 7f5abb000640 1 Processor -- start 2024-04-25T14:11:38.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.689+0000 7f5abb000640 1 -- start start 2024-04-25T14:11:38.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.693+0000 7f5abb000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5ab406ef40 0x7f5ab415c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:38.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.693+0000 7f5abb000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ab406d120 con 0x7f5ab40ae130 2024-04-25T14:11:38.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.693+0000 7f5abb000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ab406d400 con 0x7f5ab40d08c0 2024-04-25T14:11:38.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.693+0000 7f5abb000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f5ab406d6e0 con 0x7f5ab406ef40 2024-04-25T14:11:38.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.693+0000 7f5aba600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5ab40ae130 0x7f5ab40ae540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60496/0 (socket says 172.21.15.86:60496) 2024-04-25T14:11:38.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.693+0000 7f5aba600640 1 -- 172.21.15.86:0/3693600747 learned_addr learned my addr 172.21.15.86:0/3693600747 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:38.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.693+0000 7f5ab9200640 1 --2- 172.21.15.86:0/3693600747 >> v2:172.21.15.155:3300/0 conn(0x7f5ab406ef40 0x7f5ab415c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:38.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.693+0000 7f5ab3e00640 1 -- 172.21.15.86:0/3693600747 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3697170959 0 0) 0x7f5ab406d120 con 0x7f5ab40ae130 2024-04-25T14:11:38.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.693+0000 7f5ab3e00640 1 -- 172.21.15.86:0/3693600747 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a9c005740 con 0x7f5ab40ae130 2024-04-25T14:11:38.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.693+0000 7f5ab3e00640 1 -- 172.21.15.86:0/3693600747 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1105471794 0 0) 0x7f5ab406d400 con 0x7f5ab40d08c0 2024-04-25T14:11:38.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.693+0000 7f5ab3e00640 1 -- 172.21.15.86:0/3693600747 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ab406d120 con 0x7f5ab40d08c0 2024-04-25T14:11:38.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.693+0000 7f5ab9200640 1 -- 172.21.15.86:0/3693600747 >> v1:172.21.15.155:6789/0 conn(0x7f5ab40d08c0 legacy=0x7f5ab406e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:38.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.693+0000 7f5ab9200640 1 -- 172.21.15.86:0/3693600747 >> v1:172.21.15.86:6789/0 conn(0x7f5ab40ae130 legacy=0x7f5ab40ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:38.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.693+0000 7f5ab9200640 1 -- 172.21.15.86:0/3693600747 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab406d9c0 con 0x7f5ab406ef40 2024-04-25T14:11:38.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.697+0000 7f5ab9200640 1 --2- 172.21.15.86:0/3693600747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ab406ef40 0x7f5ab415c990 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00097c0 tx=0x7f5aa001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=819d112dd7339025 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:38.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.697+0000 7f5ab3e00640 1 -- 172.21.15.86:0/3693600747 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa00b0070 con 0x7f5ab406ef40 2024-04-25T14:11:38.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.697+0000 7f5ab3e00640 1 -- 172.21.15.86:0/3693600747 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aa00ab600 con 0x7f5ab406ef40 2024-04-25T14:11:38.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.697+0000 7f5ab3e00640 1 -- 172.21.15.86:0/3693600747 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa00ab9d0 con 0x7f5ab406ef40 2024-04-25T14:11:38.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.697+0000 7f5abb000640 1 -- 172.21.15.86:0/3693600747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ab406ef40 msgr2=0x7f5ab415c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:38.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.697+0000 7f5abb000640 1 --2- 172.21.15.86:0/3693600747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ab406ef40 0x7f5ab415c990 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00097c0 tx=0x7f5aa001c4a0 comp rx=0 tx=0).stop 2024-04-25T14:11:38.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.697+0000 7f5abb000640 1 -- 172.21.15.86:0/3693600747 shutdown_connections 2024-04-25T14:11:38.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.697+0000 7f5abb000640 1 --2- 172.21.15.86:0/3693600747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ab406ef40 0x7f5ab415c990 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:38.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.697+0000 7f5abb000640 1 -- 172.21.15.86:0/3693600747 >> 172.21.15.86:0/3693600747 conn(0x7f5ab40b4850 msgr2=0x7f5ab40b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:38.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.697+0000 7f5abb000640 1 -- 172.21.15.86:0/3693600747 shutdown_connections 2024-04-25T14:11:38.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.697+0000 7f5abb000640 1 -- 172.21.15.86:0/3693600747 wait complete. 2024-04-25T14:11:38.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.697+0000 7f5abb000640 1 Processor -- start 2024-04-25T14:11:38.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.697+0000 7f5abb000640 1 -- start start 2024-04-25T14:11:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.701+0000 7f5abb000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ab40ae130 0x7f5ab409aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.701+0000 7f5abb000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ab40d08c0 0x7f5ab41f97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.701+0000 7f5abb000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ab414fa00 0x7f5ab41547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.701+0000 7f5abb000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab40af1e0 con 0x7f5ab40d08c0 2024-04-25T14:11:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.701+0000 7f5abb000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab40af060 con 0x7f5ab414fa00 2024-04-25T14:11:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.701+0000 7f5abb000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5ab40ce470 con 0x7f5ab40ae130 2024-04-25T14:11:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.701+0000 7f5ab9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ab40d08c0 0x7f5ab41f97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.701+0000 7f5ab9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ab40d08c0 0x7f5ab41f97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41012/0 (socket says 172.21.15.86:41012) 2024-04-25T14:11:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.701+0000 7f5ab9200640 1 -- 172.21.15.86:0/2641264107 learned_addr learned my addr 172.21.15.86:0/2641264107 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.701+0000 7f5ab9c00640 1 --2- 172.21.15.86:0/2641264107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ab40ae130 0x7f5ab409aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.701+0000 7f5aba600640 1 --2- 172.21.15.86:0/2641264107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ab414fa00 0x7f5ab41547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.701+0000 7f5ab9200640 1 -- 172.21.15.86:0/2641264107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ab40ae130 msgr2=0x7f5ab409aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.701+0000 7f5ab9200640 1 --2- 172.21.15.86:0/2641264107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ab40ae130 0x7f5ab409aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.701+0000 7f5ab9200640 1 -- 172.21.15.86:0/2641264107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ab414fa00 msgr2=0x7f5ab41547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.701+0000 7f5ab9200640 1 --2- 172.21.15.86:0/2641264107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ab414fa00 0x7f5ab41547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.701+0000 7f5ab9200640 1 -- 172.21.15.86:0/2641264107 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab4158c90 con 0x7f5ab40d08c0 2024-04-25T14:11:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.701+0000 7f5ab9200640 1 --2- 172.21.15.86:0/2641264107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ab40d08c0 0x7f5ab41f97a0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00190b0 tx=0x7f5aa0019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.701+0000 7f5ab1600640 1 -- 172.21.15.86:0/2641264107 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa00b0040 con 0x7f5ab40d08c0 2024-04-25T14:11:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.701+0000 7f5ab1600640 1 -- 172.21.15.86:0/2641264107 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aa00b9070 con 0x7f5ab40d08c0 2024-04-25T14:11:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.701+0000 7f5ab1600640 1 -- 172.21.15.86:0/2641264107 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa00b4650 con 0x7f5ab40d08c0 2024-04-25T14:11:38.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.701+0000 7f5abb000640 1 -- 172.21.15.86:0/2641264107 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab415d3f0 con 0x7f5ab40d08c0 2024-04-25T14:11:38.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.701+0000 7f5abb000640 1 -- 172.21.15.86:0/2641264107 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab41589b0 con 0x7f5ab40d08c0 2024-04-25T14:11:38.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.701+0000 7f5abb000640 1 -- 172.21.15.86:0/2641264107 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab4076b80 con 0x7f5ab40d08c0 2024-04-25T14:11:38.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.705+0000 7f5ab1600640 1 -- 172.21.15.86:0/2641264107 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7f5aa00b47f0 con 0x7f5ab40d08c0 2024-04-25T14:11:38.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.705+0000 7f5ab1600640 1 --2- 172.21.15.86:0/2641264107 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f5a9406f520 0x7f5a94071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:38.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.705+0000 7f5ab1600640 1 -- 172.21.15.86:0/2641264107 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f5aa0129220 con 0x7f5ab40d08c0 2024-04-25T14:11:38.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.705+0000 7f5ab9c00640 1 --2- 172.21.15.86:0/2641264107 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f5a9406f520 0x7f5a94071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:38.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.705+0000 7f5ab9c00640 1 --2- 172.21.15.86:0/2641264107 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f5a9406f520 0x7f5a94071a30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8007fd0 tx=0x7f5aa8009040 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:38.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.709+0000 7f5ab1600640 1 -- 172.21.15.86:0/2641264107 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5aa00c3020 con 0x7f5ab40d08c0 2024-04-25T14:11:38.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.925+0000 7f5abb000640 1 -- 172.21.15.86:0/2641264107 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f5ab415d180 con 0x7f5ab40d08c0 2024-04-25T14:11:38.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.925+0000 7f5ab1600640 1 -- 172.21.15.86:0/2641264107 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1038 (secure 0 0 0) 0x7f5aa0016090 con 0x7f5ab40d08c0 2024-04-25T14:11:38.923 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:38.924 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":493,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":140,"num_osds":8,"num_up_osds":8,"osd_up_since":1714053818,"num_in_osds":8,"osd_in_since":1714053808,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":261906432,"bytes_avail":772832206848,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":103,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"a","status":"up:active","gid":9546}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-04-25T14:08:15.095679+0000","services":{}},"progress_events":{}} 2024-04-25T14:11:38.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.929+0000 7f5abb000640 1 -- 172.21.15.86:0/2641264107 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f5a9406f520 msgr2=0x7f5a94071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:38.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.929+0000 7f5abb000640 1 --2- 172.21.15.86:0/2641264107 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f5a9406f520 0x7f5a94071a30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8007fd0 tx=0x7f5aa8009040 comp rx=0 tx=0).stop 2024-04-25T14:11:38.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.929+0000 7f5abb000640 1 -- 172.21.15.86:0/2641264107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ab40d08c0 msgr2=0x7f5ab41f97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:38.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.929+0000 7f5abb000640 1 --2- 172.21.15.86:0/2641264107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ab40d08c0 0x7f5ab41f97a0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00190b0 tx=0x7f5aa0019240 comp rx=0 tx=0).stop 2024-04-25T14:11:38.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.929+0000 7f5abb000640 1 -- 172.21.15.86:0/2641264107 shutdown_connections 2024-04-25T14:11:38.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.929+0000 7f5abb000640 1 --2- 172.21.15.86:0/2641264107 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f5a9406f520 0x7f5a94071a30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:38.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.929+0000 7f5abb000640 1 --2- 172.21.15.86:0/2641264107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ab414fa00 0x7f5ab41547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:38.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.929+0000 7f5abb000640 1 --2- 172.21.15.86:0/2641264107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ab40d08c0 0x7f5ab41f97a0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:38.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.929+0000 7f5abb000640 1 --2- 172.21.15.86:0/2641264107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ab40ae130 0x7f5ab409aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:38.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.929+0000 7f5abb000640 1 -- 172.21.15.86:0/2641264107 >> 172.21.15.86:0/2641264107 conn(0x7f5ab40b4850 msgr2=0x7f5ab40cfa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:38.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.929+0000 7f5abb000640 1 -- 172.21.15.86:0/2641264107 shutdown_connections 2024-04-25T14:11:38.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:38.929+0000 7f5abb000640 1 -- 172.21.15.86:0/2641264107 wait complete. 2024-04-25T14:11:38.945 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T14:11:38.945 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:11:38.993 INFO:tasks.ceph.mds.a.smithi086.stderr:2024-04-25T14:11:38.997+0000 7f4c1b600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T14:11:39.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.093+0000 7fa6afe00640 1 Processor -- start 2024-04-25T14:11:39.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.097+0000 7fa6afe00640 1 -- start start 2024-04-25T14:11:39.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.097+0000 7fa6afe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa6a80cfca0 0x7fa6a806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:39.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.097+0000 7fa6afe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa6a80d16e0 0x7fa6a80d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:39.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.097+0000 7fa6afe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a806c670 0x7fa6a806ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:39.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.097+0000 7fa6afe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa6a8070470 con 0x7fa6a80d16e0 2024-04-25T14:11:39.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.097+0000 7fa6afe00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa6a80d2030 con 0x7fa6a80cfca0 2024-04-25T14:11:39.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.097+0000 7fa6afe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa6a8072bd0 con 0x7fa6a806c670 2024-04-25T14:11:39.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.097+0000 7fa6aca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa6a80d16e0 0x7fa6a80d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:39.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.097+0000 7fa6aca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa6a80d16e0 0x7fa6a80d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52906/0 (socket says 172.21.15.86:52906) 2024-04-25T14:11:39.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.097+0000 7fa6aca00640 1 -- 172.21.15.86:0/4207930313 learned_addr learned my addr 172.21.15.86:0/4207930313 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:39.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.097+0000 7fa6ad400640 1 --2- 172.21.15.86:0/4207930313 >> v2:172.21.15.155:3300/0 conn(0x7fa6a80cfca0 0x7fa6a806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:39.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.097+0000 7fa6ade00640 1 --2- 172.21.15.86:0/4207930313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a806c670 0x7fa6a806ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:39.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.097+0000 7fa6aca00640 1 -- 172.21.15.86:0/4207930313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a806c670 msgr2=0x7fa6a806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:39.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.097+0000 7fa6aca00640 1 --2- 172.21.15.86:0/4207930313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a806c670 0x7fa6a806ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:39.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.097+0000 7fa6aca00640 1 -- 172.21.15.86:0/4207930313 >> v2:172.21.15.155:3300/0 conn(0x7fa6a80cfca0 msgr2=0x7fa6a806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:39.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.097+0000 7fa6aca00640 1 --2- 172.21.15.86:0/4207930313 >> v2:172.21.15.155:3300/0 conn(0x7fa6a80cfca0 0x7fa6a806f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:39.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.097+0000 7fa6aca00640 1 -- 172.21.15.86:0/4207930313 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a806f930 con 0x7fa6a80d16e0 2024-04-25T14:11:39.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.097+0000 7fa6ad400640 1 --2- 172.21.15.86:0/4207930313 >> v2:172.21.15.155:3300/0 conn(0x7fa6a80cfca0 0x7fa6a806f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:39.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.101+0000 7fa6aca00640 1 --2- 172.21.15.86:0/4207930313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a80d16e0 0x7fa6a80d1af0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fa6a80d22a0 tx=0x7fa69401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd54931ec1bed77b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:39.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.101+0000 7fa6a7e00640 1 -- 172.21.15.86:0/4207930313 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6940ab5f0 con 0x7fa6a80d16e0 2024-04-25T14:11:39.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.101+0000 7fa6a7e00640 1 -- 172.21.15.86:0/4207930313 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6940b0070 con 0x7fa6a80d16e0 2024-04-25T14:11:39.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.101+0000 7fa6a7e00640 1 -- 172.21.15.86:0/4207930313 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6940ab9b0 con 0x7fa6a80d16e0 2024-04-25T14:11:39.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.101+0000 7fa6afe00640 1 -- 172.21.15.86:0/4207930313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a80d16e0 msgr2=0x7fa6a80d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:39.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.101+0000 7fa6afe00640 1 --2- 172.21.15.86:0/4207930313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a80d16e0 0x7fa6a80d1af0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fa6a80d22a0 tx=0x7fa69401c520 comp rx=0 tx=0).stop 2024-04-25T14:11:39.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.101+0000 7fa6afe00640 1 -- 172.21.15.86:0/4207930313 shutdown_connections 2024-04-25T14:11:39.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.101+0000 7fa6afe00640 1 --2- 172.21.15.86:0/4207930313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a806c670 0x7fa6a806ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:39.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.101+0000 7fa6afe00640 1 --2- 172.21.15.86:0/4207930313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a80d16e0 0x7fa6a80d1af0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:39.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.101+0000 7fa6afe00640 1 --2- 172.21.15.86:0/4207930313 >> v2:172.21.15.155:3300/0 conn(0x7fa6a80cfca0 0x7fa6a806f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:39.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.101+0000 7fa6afe00640 1 -- 172.21.15.86:0/4207930313 >> 172.21.15.86:0/4207930313 conn(0x7fa6a80b4650 msgr2=0x7fa6a80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:39.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.101+0000 7fa6afe00640 1 -- 172.21.15.86:0/4207930313 shutdown_connections 2024-04-25T14:11:39.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.101+0000 7fa6afe00640 1 -- 172.21.15.86:0/4207930313 wait complete. 2024-04-25T14:11:39.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.105+0000 7fa6afe00640 1 Processor -- start 2024-04-25T14:11:39.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.105+0000 7fa6afe00640 1 -- start start 2024-04-25T14:11:39.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.105+0000 7fa6afe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6a806c670 0x7fa6a80af240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:39.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.105+0000 7fa6afe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a80cfca0 0x7fa6a809a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:39.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.105+0000 7fa6afe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a80d16e0 0x7fa6a80c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:39.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.105+0000 7fa6afe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a80738c0 con 0x7fa6a80cfca0 2024-04-25T14:11:39.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.105+0000 7fa6afe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a806d3d0 con 0x7fa6a806c670 2024-04-25T14:11:39.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.105+0000 7fa6afe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa6a8073a40 con 0x7fa6a80d16e0 2024-04-25T14:11:39.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.105+0000 7fa6aca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a80cfca0 0x7fa6a809a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:39.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.105+0000 7fa6aca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a80cfca0 0x7fa6a809a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52916/0 (socket says 172.21.15.86:52916) 2024-04-25T14:11:39.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.105+0000 7fa6aca00640 1 -- 172.21.15.86:0/1897880416 learned_addr learned my addr 172.21.15.86:0/1897880416 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:39.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.105+0000 7fa6ad400640 1 --2- 172.21.15.86:0/1897880416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6a806c670 0x7fa6a80af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:39.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.105+0000 7fa6ade00640 1 --2- 172.21.15.86:0/1897880416 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a80d16e0 0x7fa6a80c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:39.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.105+0000 7fa6aca00640 1 -- 172.21.15.86:0/1897880416 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a80d16e0 msgr2=0x7fa6a80c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:39.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.105+0000 7fa6aca00640 1 --2- 172.21.15.86:0/1897880416 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a80d16e0 0x7fa6a80c9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:39.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.105+0000 7fa6aca00640 1 -- 172.21.15.86:0/1897880416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6a806c670 msgr2=0x7fa6a80af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:39.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.105+0000 7fa6aca00640 1 --2- 172.21.15.86:0/1897880416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6a806c670 0x7fa6a80af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:39.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.105+0000 7fa6aca00640 1 -- 172.21.15.86:0/1897880416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a81fdcd0 con 0x7fa6a80cfca0 2024-04-25T14:11:39.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.105+0000 7fa6aca00640 1 --2- 172.21.15.86:0/1897880416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a80cfca0 0x7fa6a809a870 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fa69401ca00 tx=0x7fa69401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:39.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.105+0000 7fa6a5600640 1 -- 172.21.15.86:0/1897880416 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6940be3f0 con 0x7fa6a80cfca0 2024-04-25T14:11:39.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.105+0000 7fa6a5600640 1 -- 172.21.15.86:0/1897880416 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6940be5b0 con 0x7fa6a80cfca0 2024-04-25T14:11:39.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.105+0000 7fa6a5600640 1 -- 172.21.15.86:0/1897880416 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6940be9a0 con 0x7fa6a80cfca0 2024-04-25T14:11:39.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.105+0000 7fa6afe00640 1 -- 172.21.15.86:0/1897880416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a809d3f0 con 0x7fa6a80cfca0 2024-04-25T14:11:39.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.105+0000 7fa6afe00640 1 -- 172.21.15.86:0/1897880416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a80af780 con 0x7fa6a80cfca0 2024-04-25T14:11:39.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.109+0000 7fa6afe00640 1 -- 172.21.15.86:0/1897880416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6a809f4b0 con 0x7fa6a80cfca0 2024-04-25T14:11:39.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.109+0000 7fa6a5600640 1 -- 172.21.15.86:0/1897880416 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7fa6940beb40 con 0x7fa6a80cfca0 2024-04-25T14:11:39.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.109+0000 7fa6a5600640 1 --2- 172.21.15.86:0/1897880416 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa68806f550 0x7fa688071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:39.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.109+0000 7fa6a5600640 1 -- 172.21.15.86:0/1897880416 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7fa694129650 con 0x7fa6a80cfca0 2024-04-25T14:11:39.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.109+0000 7fa6ad400640 1 --2- 172.21.15.86:0/1897880416 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa68806f550 0x7fa688071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:39.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.109+0000 7fa6ad400640 1 --2- 172.21.15.86:0/1897880416 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa68806f550 0x7fa688071a60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8070710 tx=0x7fa69c008040 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:39.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.113+0000 7fa6a5600640 1 -- 172.21.15.86:0/1897880416 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa6940c4060 con 0x7fa6a80cfca0 2024-04-25T14:11:39.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.265+0000 7fa6afe00640 1 -- 172.21.15.86:0/1897880416 --> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa6a8069d00 con 0x7fa68806f550 2024-04-25T14:11:39.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.273+0000 7fa6a5600640 1 -- 172.21.15.86:0/1897880416 <== mgr.9282 v2:172.21.15.86:6832/264890479 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252525 (secure 0 0 0) 0x7fa6a8069d00 con 0x7fa68806f550 2024-04-25T14:11:39.272 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:39.274 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:11:39.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.277+0000 7fa6afe00640 1 -- 172.21.15.86:0/1897880416 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa68806f550 msgr2=0x7fa688071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:39.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.277+0000 7fa6afe00640 1 --2- 172.21.15.86:0/1897880416 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa68806f550 0x7fa688071a60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8070710 tx=0x7fa69c008040 comp rx=0 tx=0).stop 2024-04-25T14:11:39.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.277+0000 7fa6afe00640 1 -- 172.21.15.86:0/1897880416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a80cfca0 msgr2=0x7fa6a809a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:39.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.277+0000 7fa6afe00640 1 --2- 172.21.15.86:0/1897880416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a80cfca0 0x7fa6a809a870 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fa69401ca00 tx=0x7fa69401ad60 comp rx=0 tx=0).stop 2024-04-25T14:11:39.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.277+0000 7fa6afe00640 1 -- 172.21.15.86:0/1897880416 shutdown_connections 2024-04-25T14:11:39.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.277+0000 7fa6afe00640 1 --2- 172.21.15.86:0/1897880416 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa68806f550 0x7fa688071a60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:39.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.277+0000 7fa6afe00640 1 --2- 172.21.15.86:0/1897880416 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a80d16e0 0x7fa6a80c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:39.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.277+0000 7fa6afe00640 1 --2- 172.21.15.86:0/1897880416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a80cfca0 0x7fa6a809a870 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:39.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.277+0000 7fa6afe00640 1 --2- 172.21.15.86:0/1897880416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6a806c670 0x7fa6a80af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:39.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.277+0000 7fa6afe00640 1 -- 172.21.15.86:0/1897880416 >> 172.21.15.86:0/1897880416 conn(0x7fa6a80b4650 msgr2=0x7fa6a80ceeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:39.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.277+0000 7fa6afe00640 1 -- 172.21.15.86:0/1897880416 shutdown_connections 2024-04-25T14:11:39.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:39.277+0000 7fa6afe00640 1 -- 172.21.15.86:0/1897880416 wait complete. 2024-04-25T14:11:39.293 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-04-25T14:11:38.734731+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":1927,"num_write":268,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3316,"ondisk_log_size":3316,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256076,"kb_used_data":14952,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754718644,"statfs":{"total":773094113280,"available":772831891456,"internally_reserved":0,"allocated":15310848,"data_stored":8226804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,1,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.155560"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.603137+0000","last_change":"2024-04-25T14:11:33.600545+0000","last_active":"2024-04-25T14:11:34.603137+0000","last_peered":"2024-04-25T14:11:34.603137+0000","last_clean":"2024-04-25T14:11:34.603137+0000","last_became_active":"2024-04-25T14:11:33.599990+0000","last_became_peered":"2024-04-25T14:11:33.599990+0000","last_unstale":"2024-04-25T14:11:34.603137+0000","last_undegraded":"2024-04-25T14:11:34.603137+0000","last_fullsized":"2024-04-25T14:11:34.603137+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:57.034214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602667+0000","last_change":"2024-04-25T14:11:33.614602+0000","last_active":"2024-04-25T14:11:34.602667+0000","last_peered":"2024-04-25T14:11:34.602667+0000","last_clean":"2024-04-25T14:11:34.602667+0000","last_became_active":"2024-04-25T14:11:33.614279+0000","last_became_peered":"2024-04-25T14:11:33.614279+0000","last_unstale":"2024-04-25T14:11:34.602667+0000","last_undegraded":"2024-04-25T14:11:34.602667+0000","last_fullsized":"2024-04-25T14:11:34.602667+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:05.122162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602490+0000","last_change":"2024-04-25T14:11:33.612130+0000","last_active":"2024-04-25T14:11:34.602490+0000","last_peered":"2024-04-25T14:11:34.602490+0000","last_clean":"2024-04-25T14:11:34.602490+0000","last_became_active":"2024-04-25T14:11:33.611772+0000","last_became_peered":"2024-04-25T14:11:33.611772+0000","last_unstale":"2024-04-25T14:11:34.602490+0000","last_undegraded":"2024-04-25T14:11:34.602490+0000","last_fullsized":"2024-04-25T14:11:34.602490+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:17.778893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.660010+0000","last_change":"2024-04-25T14:11:33.620093+0000","last_active":"2024-04-25T14:11:34.660010+0000","last_peered":"2024-04-25T14:11:34.660010+0000","last_clean":"2024-04-25T14:11:34.660010+0000","last_became_active":"2024-04-25T14:11:33.619776+0000","last_became_peered":"2024-04-25T14:11:33.619776+0000","last_unstale":"2024-04-25T14:11:34.660010+0000","last_undegraded":"2024-04-25T14:11:34.660010+0000","last_fullsized":"2024-04-25T14:11:34.660010+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:10.040630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600597+0000","last_change":"2024-04-25T14:11:33.602823+0000","last_active":"2024-04-25T14:11:34.600597+0000","last_peered":"2024-04-25T14:11:34.600597+0000","last_clean":"2024-04-25T14:11:34.600597+0000","last_became_active":"2024-04-25T14:11:33.593014+0000","last_became_peered":"2024-04-25T14:11:33.593014+0000","last_unstale":"2024-04-25T14:11:34.600597+0000","last_undegraded":"2024-04-25T14:11:34.600597+0000","last_fullsized":"2024-04-25T14:11:34.600597+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:21.294240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600359+0000","last_change":"2024-04-25T14:11:33.593267+0000","last_active":"2024-04-25T14:11:34.600359+0000","last_peered":"2024-04-25T14:11:34.600359+0000","last_clean":"2024-04-25T14:11:34.600359+0000","last_became_active":"2024-04-25T14:11:33.592870+0000","last_became_peered":"2024-04-25T14:11:33.592870+0000","last_unstale":"2024-04-25T14:11:34.600359+0000","last_undegraded":"2024-04-25T14:11:34.600359+0000","last_fullsized":"2024-04-25T14:11:34.600359+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:05.246429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602729+0000","last_change":"2024-04-25T14:11:33.598268+0000","last_active":"2024-04-25T14:11:34.602729+0000","last_peered":"2024-04-25T14:11:34.602729+0000","last_clean":"2024-04-25T14:11:34.602729+0000","last_became_active":"2024-04-25T14:11:33.597942+0000","last_became_peered":"2024-04-25T14:11:33.597942+0000","last_unstale":"2024-04-25T14:11:34.602729+0000","last_undegraded":"2024-04-25T14:11:34.602729+0000","last_fullsized":"2024-04-25T14:11:34.602729+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:04.160373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659001+0000","last_change":"2024-04-25T14:11:33.616240+0000","last_active":"2024-04-25T14:11:34.659001+0000","last_peered":"2024-04-25T14:11:34.659001+0000","last_clean":"2024-04-25T14:11:34.659001+0000","last_became_active":"2024-04-25T14:11:33.615903+0000","last_became_peered":"2024-04-25T14:11:33.615903+0000","last_unstale":"2024-04-25T14:11:34.659001+0000","last_undegraded":"2024-04-25T14:11:34.659001+0000","last_fullsized":"2024-04-25T14:11:34.659001+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:04.673178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600405+0000","last_change":"2024-04-25T14:11:33.603215+0000","last_active":"2024-04-25T14:11:34.600405+0000","last_peered":"2024-04-25T14:11:34.600405+0000","last_clean":"2024-04-25T14:11:34.600405+0000","last_became_active":"2024-04-25T14:11:33.602886+0000","last_became_peered":"2024-04-25T14:11:33.602886+0000","last_unstale":"2024-04-25T14:11:34.600405+0000","last_undegraded":"2024-04-25T14:11:34.600405+0000","last_fullsized":"2024-04-25T14:11:34.600405+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:57.463642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600352+0000","last_change":"2024-04-25T14:11:33.602442+0000","last_active":"2024-04-25T14:11:34.600352+0000","last_peered":"2024-04-25T14:11:34.600352+0000","last_clean":"2024-04-25T14:11:34.600352+0000","last_became_active":"2024-04-25T14:11:33.602108+0000","last_became_peered":"2024-04-25T14:11:33.602108+0000","last_unstale":"2024-04-25T14:11:34.600352+0000","last_undegraded":"2024-04-25T14:11:34.600352+0000","last_fullsized":"2024-04-25T14:11:34.600352+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:24:55.518082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659208+0000","last_change":"2024-04-25T14:11:33.627206+0000","last_active":"2024-04-25T14:11:34.659208+0000","last_peered":"2024-04-25T14:11:34.659208+0000","last_clean":"2024-04-25T14:11:34.659208+0000","last_became_active":"2024-04-25T14:11:33.626877+0000","last_became_peered":"2024-04-25T14:11:33.626877+0000","last_unstale":"2024-04-25T14:11:34.659208+0000","last_undegraded":"2024-04-25T14:11:34.659208+0000","last_fullsized":"2024-04-25T14:11:34.659208+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:13.084865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.660179+0000","last_change":"2024-04-25T14:11:33.596026+0000","last_active":"2024-04-25T14:11:34.660179+0000","last_peered":"2024-04-25T14:11:34.660179+0000","last_clean":"2024-04-25T14:11:34.660179+0000","last_became_active":"2024-04-25T14:11:33.595721+0000","last_became_peered":"2024-04-25T14:11:33.595721+0000","last_unstale":"2024-04-25T14:11:34.660179+0000","last_undegraded":"2024-04-25T14:11:34.660179+0000","last_fullsized":"2024-04-25T14:11:34.660179+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:04.160373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600451+0000","last_change":"2024-04-25T14:11:33.602904+0000","last_active":"2024-04-25T14:11:34.600451+0000","last_peered":"2024-04-25T14:11:34.600451+0000","last_clean":"2024-04-25T14:11:34.600451+0000","last_became_active":"2024-04-25T14:11:33.599935+0000","last_became_peered":"2024-04-25T14:11:33.599935+0000","last_unstale":"2024-04-25T14:11:34.600451+0000","last_undegraded":"2024-04-25T14:11:34.600451+0000","last_fullsized":"2024-04-25T14:11:34.600451+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:49:36.789630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.660405+0000","last_change":"2024-04-25T14:11:33.597740+0000","last_active":"2024-04-25T14:11:34.660405+0000","last_peered":"2024-04-25T14:11:34.660405+0000","last_clean":"2024-04-25T14:11:34.660405+0000","last_became_active":"2024-04-25T14:11:33.597430+0000","last_became_peered":"2024-04-25T14:11:33.597430+0000","last_unstale":"2024-04-25T14:11:34.660405+0000","last_undegraded":"2024-04-25T14:11:34.660405+0000","last_fullsized":"2024-04-25T14:11:34.660405+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:17.186121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602355+0000","last_change":"2024-04-25T14:11:33.615341+0000","last_active":"2024-04-25T14:11:34.602355+0000","last_peered":"2024-04-25T14:11:34.602355+0000","last_clean":"2024-04-25T14:11:34.602355+0000","last_became_active":"2024-04-25T14:11:33.614985+0000","last_became_peered":"2024-04-25T14:11:33.614985+0000","last_unstale":"2024-04-25T14:11:34.602355+0000","last_undegraded":"2024-04-25T14:11:34.602355+0000","last_fullsized":"2024-04-25T14:11:34.602355+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:49.326225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600068+0000","last_change":"2024-04-25T14:11:33.608973+0000","last_active":"2024-04-25T14:11:34.600068+0000","last_peered":"2024-04-25T14:11:34.600068+0000","last_clean":"2024-04-25T14:11:34.600068+0000","last_became_active":"2024-04-25T14:11:33.608576+0000","last_became_peered":"2024-04-25T14:11:33.608576+0000","last_unstale":"2024-04-25T14:11:34.600068+0000","last_undegraded":"2024-04-25T14:11:34.600068+0000","last_fullsized":"2024-04-25T14:11:34.600068+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:58:03.795240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.660667+0000","last_change":"2024-04-25T14:11:33.598496+0000","last_active":"2024-04-25T14:11:34.660667+0000","last_peered":"2024-04-25T14:11:34.660667+0000","last_clean":"2024-04-25T14:11:34.660667+0000","last_became_active":"2024-04-25T14:11:33.598197+0000","last_became_peered":"2024-04-25T14:11:33.598197+0000","last_unstale":"2024-04-25T14:11:34.660667+0000","last_undegraded":"2024-04-25T14:11:34.660667+0000","last_fullsized":"2024-04-25T14:11:34.660667+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:57.034214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659300+0000","last_change":"2024-04-25T14:11:33.617124+0000","last_active":"2024-04-25T14:11:34.659300+0000","last_peered":"2024-04-25T14:11:34.659300+0000","last_clean":"2024-04-25T14:11:34.659300+0000","last_became_active":"2024-04-25T14:11:33.616830+0000","last_became_peered":"2024-04-25T14:11:33.616830+0000","last_unstale":"2024-04-25T14:11:34.659300+0000","last_undegraded":"2024-04-25T14:11:34.659300+0000","last_fullsized":"2024-04-25T14:11:34.659300+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:04.637134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659498+0000","last_change":"2024-04-25T14:11:33.610363+0000","last_active":"2024-04-25T14:11:34.659498+0000","last_peered":"2024-04-25T14:11:34.659498+0000","last_clean":"2024-04-25T14:11:34.659498+0000","last_became_active":"2024-04-25T14:11:33.609578+0000","last_became_peered":"2024-04-25T14:11:33.609578+0000","last_unstale":"2024-04-25T14:11:34.659498+0000","last_undegraded":"2024-04-25T14:11:34.659498+0000","last_fullsized":"2024-04-25T14:11:34.659498+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:17.778893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.658728+0000","last_change":"2024-04-25T14:11:33.622426+0000","last_active":"2024-04-25T14:11:34.658728+0000","last_peered":"2024-04-25T14:11:34.658728+0000","last_clean":"2024-04-25T14:11:34.658728+0000","last_became_active":"2024-04-25T14:11:33.621583+0000","last_became_peered":"2024-04-25T14:11:33.621583+0000","last_unstale":"2024-04-25T14:11:34.658728+0000","last_undegraded":"2024-04-25T14:11:34.658728+0000","last_fullsized":"2024-04-25T14:11:34.658728+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:32.466772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600561+0000","last_change":"2024-04-25T14:11:33.603619+0000","last_active":"2024-04-25T14:11:34.600561+0000","last_peered":"2024-04-25T14:11:34.600561+0000","last_clean":"2024-04-25T14:11:34.600561+0000","last_became_active":"2024-04-25T14:11:33.603206+0000","last_became_peered":"2024-04-25T14:11:33.603206+0000","last_unstale":"2024-04-25T14:11:34.600561+0000","last_undegraded":"2024-04-25T14:11:34.600561+0000","last_fullsized":"2024-04-25T14:11:34.600561+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:31.565395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602769+0000","last_change":"2024-04-25T14:11:33.601209+0000","last_active":"2024-04-25T14:11:34.602769+0000","last_peered":"2024-04-25T14:11:34.602769+0000","last_clean":"2024-04-25T14:11:34.602769+0000","last_became_active":"2024-04-25T14:11:33.600904+0000","last_became_peered":"2024-04-25T14:11:33.600904+0000","last_unstale":"2024-04-25T14:11:34.602769+0000","last_undegraded":"2024-04-25T14:11:34.602769+0000","last_fullsized":"2024-04-25T14:11:34.602769+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.223787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659180+0000","last_change":"2024-04-25T14:11:33.595593+0000","last_active":"2024-04-25T14:11:34.659180+0000","last_peered":"2024-04-25T14:11:34.659180+0000","last_clean":"2024-04-25T14:11:34.659180+0000","last_became_active":"2024-04-25T14:11:33.595294+0000","last_became_peered":"2024-04-25T14:11:33.595294+0000","last_unstale":"2024-04-25T14:11:34.659180+0000","last_undegraded":"2024-04-25T14:11:34.659180+0000","last_fullsized":"2024-04-25T14:11:34.659180+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:32.090605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.658624+0000","last_change":"2024-04-25T14:11:33.616089+0000","last_active":"2024-04-25T14:11:34.658624+0000","last_peered":"2024-04-25T14:11:34.658624+0000","last_clean":"2024-04-25T14:11:34.658624+0000","last_became_active":"2024-04-25T14:11:33.615770+0000","last_became_peered":"2024-04-25T14:11:33.615770+0000","last_unstale":"2024-04-25T14:11:34.658624+0000","last_undegraded":"2024-04-25T14:11:34.658624+0000","last_fullsized":"2024-04-25T14:11:34.658624+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:23.567530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.603242+0000","last_change":"2024-04-25T14:11:33.600201+0000","last_active":"2024-04-25T14:11:34.603242+0000","last_peered":"2024-04-25T14:11:34.603242+0000","last_clean":"2024-04-25T14:11:34.603242+0000","last_became_active":"2024-04-25T14:11:33.599319+0000","last_became_peered":"2024-04-25T14:11:33.599319+0000","last_unstale":"2024-04-25T14:11:34.603242+0000","last_undegraded":"2024-04-25T14:11:34.603242+0000","last_fullsized":"2024-04-25T14:11:34.603242+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:29.349474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.603196+0000","last_change":"2024-04-25T14:11:33.617962+0000","last_active":"2024-04-25T14:11:34.603196+0000","last_peered":"2024-04-25T14:11:34.603196+0000","last_clean":"2024-04-25T14:11:34.603196+0000","last_became_active":"2024-04-25T14:11:33.617613+0000","last_became_peered":"2024-04-25T14:11:33.617613+0000","last_unstale":"2024-04-25T14:11:34.603196+0000","last_undegraded":"2024-04-25T14:11:34.603196+0000","last_fullsized":"2024-04-25T14:11:34.603196+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:46:19.184891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600234+0000","last_change":"2024-04-25T14:11:33.603938+0000","last_active":"2024-04-25T14:11:34.600234+0000","last_peered":"2024-04-25T14:11:34.600234+0000","last_clean":"2024-04-25T14:11:34.600234+0000","last_became_active":"2024-04-25T14:11:33.603587+0000","last_became_peered":"2024-04-25T14:11:33.603587+0000","last_unstale":"2024-04-25T14:11:34.600234+0000","last_undegraded":"2024-04-25T14:11:34.600234+0000","last_fullsized":"2024-04-25T14:11:34.600234+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:16.358014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659150+0000","last_change":"2024-04-25T14:11:33.623158+0000","last_active":"2024-04-25T14:11:34.659150+0000","last_peered":"2024-04-25T14:11:34.659150+0000","last_clean":"2024-04-25T14:11:34.659150+0000","last_became_active":"2024-04-25T14:11:33.622821+0000","last_became_peered":"2024-04-25T14:11:33.622821+0000","last_unstale":"2024-04-25T14:11:34.659150+0000","last_undegraded":"2024-04-25T14:11:34.659150+0000","last_fullsized":"2024-04-25T14:11:34.659150+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:05.771659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659754+0000","last_change":"2024-04-25T14:11:33.609928+0000","last_active":"2024-04-25T14:11:34.659754+0000","last_peered":"2024-04-25T14:11:34.659754+0000","last_clean":"2024-04-25T14:11:34.659754+0000","last_became_active":"2024-04-25T14:11:33.609569+0000","last_became_peered":"2024-04-25T14:11:33.609569+0000","last_unstale":"2024-04-25T14:11:34.659754+0000","last_undegraded":"2024-04-25T14:11:34.659754+0000","last_fullsized":"2024-04-25T14:11:34.659754+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:17.186121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.658814+0000","last_change":"2024-04-25T14:11:33.620076+0000","last_active":"2024-04-25T14:11:34.658814+0000","last_peered":"2024-04-25T14:11:34.658814+0000","last_clean":"2024-04-25T14:11:34.658814+0000","last_became_active":"2024-04-25T14:11:33.619774+0000","last_became_peered":"2024-04-25T14:11:33.619774+0000","last_unstale":"2024-04-25T14:11:34.658814+0000","last_undegraded":"2024-04-25T14:11:34.658814+0000","last_fullsized":"2024-04-25T14:11:34.658814+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:35.800429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600226+0000","last_change":"2024-04-25T14:11:33.597817+0000","last_active":"2024-04-25T14:11:34.600226+0000","last_peered":"2024-04-25T14:11:34.600226+0000","last_clean":"2024-04-25T14:11:34.600226+0000","last_became_active":"2024-04-25T14:11:33.597513+0000","last_became_peered":"2024-04-25T14:11:33.597513+0000","last_unstale":"2024-04-25T14:11:34.600226+0000","last_undegraded":"2024-04-25T14:11:34.600226+0000","last_fullsized":"2024-04-25T14:11:34.600226+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:58.403145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659174+0000","last_change":"2024-04-25T14:11:33.612165+0000","last_active":"2024-04-25T14:11:34.659174+0000","last_peered":"2024-04-25T14:11:34.659174+0000","last_clean":"2024-04-25T14:11:34.659174+0000","last_became_active":"2024-04-25T14:11:33.611831+0000","last_became_peered":"2024-04-25T14:11:33.611831+0000","last_unstale":"2024-04-25T14:11:34.659174+0000","last_undegraded":"2024-04-25T14:11:34.659174+0000","last_fullsized":"2024-04-25T14:11:34.659174+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:17.071976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602855+0000","last_change":"2024-04-25T14:11:33.618207+0000","last_active":"2024-04-25T14:11:34.602855+0000","last_peered":"2024-04-25T14:11:34.602855+0000","last_clean":"2024-04-25T14:11:34.602855+0000","last_became_active":"2024-04-25T14:11:33.617893+0000","last_became_peered":"2024-04-25T14:11:33.617893+0000","last_unstale":"2024-04-25T14:11:34.602855+0000","last_undegraded":"2024-04-25T14:11:34.602855+0000","last_fullsized":"2024-04-25T14:11:34.602855+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:16.358014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"139'4","reported_seq":18,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600663+0000","last_change":"2024-04-25T14:11:32.591452+0000","last_active":"2024-04-25T14:11:34.600663+0000","last_peered":"2024-04-25T14:11:34.600663+0000","last_clean":"2024-04-25T14:11:34.600663+0000","last_became_active":"2024-04-25T14:11:32.591159+0000","last_became_peered":"2024-04-25T14:11:32.591159+0000","last_unstale":"2024-04-25T14:11:34.600663+0000","last_undegraded":"2024-04-25T14:11:34.600663+0000","last_fullsized":"2024-04-25T14:11:34.600663+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:16.177533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602677+0000","last_change":"2024-04-25T14:11:33.607419+0000","last_active":"2024-04-25T14:11:34.602677+0000","last_peered":"2024-04-25T14:11:34.602677+0000","last_clean":"2024-04-25T14:11:34.602677+0000","last_became_active":"2024-04-25T14:11:33.607086+0000","last_became_peered":"2024-04-25T14:11:33.607086+0000","last_unstale":"2024-04-25T14:11:34.602677+0000","last_undegraded":"2024-04-25T14:11:34.602677+0000","last_fullsized":"2024-04-25T14:11:34.602677+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:58.403145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659400+0000","last_change":"2024-04-25T14:11:32.603965+0000","last_active":"2024-04-25T14:11:34.659400+0000","last_peered":"2024-04-25T14:11:34.659400+0000","last_clean":"2024-04-25T14:11:34.659400+0000","last_became_active":"2024-04-25T14:11:32.602019+0000","last_became_peered":"2024-04-25T14:11:32.602019+0000","last_unstale":"2024-04-25T14:11:34.659400+0000","last_undegraded":"2024-04-25T14:11:34.659400+0000","last_fullsized":"2024-04-25T14:11:34.659400+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:35.215199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.660329+0000","last_change":"2024-04-25T14:11:33.600332+0000","last_active":"2024-04-25T14:11:34.660329+0000","last_peered":"2024-04-25T14:11:34.660329+0000","last_clean":"2024-04-25T14:11:34.660329+0000","last_became_active":"2024-04-25T14:11:33.599668+0000","last_became_peered":"2024-04-25T14:11:33.599668+0000","last_unstale":"2024-04-25T14:11:34.660329+0000","last_undegraded":"2024-04-25T14:11:34.660329+0000","last_fullsized":"2024-04-25T14:11:34.660329+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:57.034214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659785+0000","last_change":"2024-04-25T14:11:32.601614+0000","last_active":"2024-04-25T14:11:34.659785+0000","last_peered":"2024-04-25T14:11:34.659785+0000","last_clean":"2024-04-25T14:11:34.659785+0000","last_became_active":"2024-04-25T14:11:32.601092+0000","last_became_peered":"2024-04-25T14:11:32.601092+0000","last_unstale":"2024-04-25T14:11:34.659785+0000","last_undegraded":"2024-04-25T14:11:34.659785+0000","last_fullsized":"2024-04-25T14:11:34.659785+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:46:18.176303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659197+0000","last_change":"2024-04-25T14:11:33.615565+0000","last_active":"2024-04-25T14:11:34.659197+0000","last_peered":"2024-04-25T14:11:34.659197+0000","last_clean":"2024-04-25T14:11:34.659197+0000","last_became_active":"2024-04-25T14:11:33.611366+0000","last_became_peered":"2024-04-25T14:11:33.611366+0000","last_unstale":"2024-04-25T14:11:34.659197+0000","last_undegraded":"2024-04-25T14:11:34.659197+0000","last_fullsized":"2024-04-25T14:11:34.659197+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:56:42.715756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602949+0000","last_change":"2024-04-25T14:11:32.602711+0000","last_active":"2024-04-25T14:11:34.602949+0000","last_peered":"2024-04-25T14:11:34.602949+0000","last_clean":"2024-04-25T14:11:34.602949+0000","last_became_active":"2024-04-25T14:11:32.602368+0000","last_became_peered":"2024-04-25T14:11:32.602368+0000","last_unstale":"2024-04-25T14:11:34.602949+0000","last_undegraded":"2024-04-25T14:11:34.602949+0000","last_fullsized":"2024-04-25T14:11:34.602949+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:16.063389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.658730+0000","last_change":"2024-04-25T14:11:33.627653+0000","last_active":"2024-04-25T14:11:34.658730+0000","last_peered":"2024-04-25T14:11:34.658730+0000","last_clean":"2024-04-25T14:11:34.658730+0000","last_became_active":"2024-04-25T14:11:33.626207+0000","last_became_peered":"2024-04-25T14:11:33.626207+0000","last_unstale":"2024-04-25T14:11:34.658730+0000","last_undegraded":"2024-04-25T14:11:34.658730+0000","last_fullsized":"2024-04-25T14:11:34.658730+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:40.168428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659866+0000","last_change":"2024-04-25T14:11:32.598778+0000","last_active":"2024-04-25T14:11:34.659866+0000","last_peered":"2024-04-25T14:11:34.659866+0000","last_clean":"2024-04-25T14:11:34.659866+0000","last_became_active":"2024-04-25T14:11:32.598190+0000","last_became_peered":"2024-04-25T14:11:32.598190+0000","last_unstale":"2024-04-25T14:11:34.659866+0000","last_undegraded":"2024-04-25T14:11:34.659866+0000","last_fullsized":"2024-04-25T14:11:34.659866+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:14.883395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.660668+0000","last_change":"2024-04-25T14:11:33.619924+0000","last_active":"2024-04-25T14:11:34.660668+0000","last_peered":"2024-04-25T14:11:34.660668+0000","last_clean":"2024-04-25T14:11:34.660668+0000","last_became_active":"2024-04-25T14:11:33.619291+0000","last_became_peered":"2024-04-25T14:11:33.619291+0000","last_unstale":"2024-04-25T14:11:34.660668+0000","last_undegraded":"2024-04-25T14:11:34.660668+0000","last_fullsized":"2024-04-25T14:11:34.660668+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:46:19.184891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600184+0000","last_change":"2024-04-25T14:11:32.605097+0000","last_active":"2024-04-25T14:11:34.600184+0000","last_peered":"2024-04-25T14:11:34.600184+0000","last_clean":"2024-04-25T14:11:34.600184+0000","last_became_active":"2024-04-25T14:11:32.604770+0000","last_became_peered":"2024-04-25T14:11:32.604770+0000","last_unstale":"2024-04-25T14:11:34.600184+0000","last_undegraded":"2024-04-25T14:11:34.600184+0000","last_fullsized":"2024-04-25T14:11:34.600184+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:09.032042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602397+0000","last_change":"2024-04-25T14:11:33.614281+0000","last_active":"2024-04-25T14:11:34.602397+0000","last_peered":"2024-04-25T14:11:34.602397+0000","last_clean":"2024-04-25T14:11:34.602397+0000","last_became_active":"2024-04-25T14:11:33.613639+0000","last_became_peered":"2024-04-25T14:11:33.613639+0000","last_unstale":"2024-04-25T14:11:34.602397+0000","last_undegraded":"2024-04-25T14:11:34.602397+0000","last_fullsized":"2024-04-25T14:11:34.602397+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:07.671897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602742+0000","last_change":"2024-04-25T14:11:32.605101+0000","last_active":"2024-04-25T14:11:34.602742+0000","last_peered":"2024-04-25T14:11:34.602742+0000","last_clean":"2024-04-25T14:11:34.602742+0000","last_became_active":"2024-04-25T14:11:32.604386+0000","last_became_peered":"2024-04-25T14:11:32.604386+0000","last_unstale":"2024-04-25T14:11:34.602742+0000","last_undegraded":"2024-04-25T14:11:34.602742+0000","last_fullsized":"2024-04-25T14:11:34.602742+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:51.825452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.603015+0000","last_change":"2024-04-25T14:11:33.604679+0000","last_active":"2024-04-25T14:11:34.603015+0000","last_peered":"2024-04-25T14:11:34.603015+0000","last_clean":"2024-04-25T14:11:34.603015+0000","last_became_active":"2024-04-25T14:11:33.604366+0000","last_became_peered":"2024-04-25T14:11:33.604366+0000","last_unstale":"2024-04-25T14:11:34.603015+0000","last_undegraded":"2024-04-25T14:11:34.603015+0000","last_fullsized":"2024-04-25T14:11:34.603015+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:17.186121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.599957+0000","last_change":"2024-04-25T14:11:32.591107+0000","last_active":"2024-04-25T14:11:34.599957+0000","last_peered":"2024-04-25T14:11:34.599957+0000","last_clean":"2024-04-25T14:11:34.599957+0000","last_became_active":"2024-04-25T14:11:32.590762+0000","last_became_peered":"2024-04-25T14:11:32.590762+0000","last_unstale":"2024-04-25T14:11:34.599957+0000","last_undegraded":"2024-04-25T14:11:34.599957+0000","last_fullsized":"2024-04-25T14:11:34.599957+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:35.215199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659728+0000","last_change":"2024-04-25T14:11:33.609347+0000","last_active":"2024-04-25T14:11:34.659728+0000","last_peered":"2024-04-25T14:11:34.659728+0000","last_clean":"2024-04-25T14:11:34.659728+0000","last_became_active":"2024-04-25T14:11:33.609020+0000","last_became_peered":"2024-04-25T14:11:33.609020+0000","last_unstale":"2024-04-25T14:11:34.659728+0000","last_undegraded":"2024-04-25T14:11:34.659728+0000","last_fullsized":"2024-04-25T14:11:34.659728+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.223787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659164+0000","last_change":"2024-04-25T14:11:32.602319+0000","last_active":"2024-04-25T14:11:34.659164+0000","last_peered":"2024-04-25T14:11:34.659164+0000","last_clean":"2024-04-25T14:11:34.659164+0000","last_became_active":"2024-04-25T14:11:32.601994+0000","last_became_peered":"2024-04-25T14:11:32.601994+0000","last_unstale":"2024-04-25T14:11:34.659164+0000","last_undegraded":"2024-04-25T14:11:34.659164+0000","last_fullsized":"2024-04-25T14:11:34.659164+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:56.025626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659223+0000","last_change":"2024-04-25T14:11:33.624243+0000","last_active":"2024-04-25T14:11:34.659223+0000","last_peered":"2024-04-25T14:11:34.659223+0000","last_clean":"2024-04-25T14:11:34.659223+0000","last_became_active":"2024-04-25T14:11:33.623178+0000","last_became_peered":"2024-04-25T14:11:33.623178+0000","last_unstale":"2024-04-25T14:11:34.659223+0000","last_undegraded":"2024-04-25T14:11:34.659223+0000","last_fullsized":"2024-04-25T14:11:34.659223+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:25.536000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.603393+0000","last_change":"2024-04-25T14:11:32.599420+0000","last_active":"2024-04-25T14:11:34.603393+0000","last_peered":"2024-04-25T14:11:34.603393+0000","last_clean":"2024-04-25T14:11:34.603393+0000","last_became_active":"2024-04-25T14:11:32.599133+0000","last_became_peered":"2024-04-25T14:11:32.599133+0000","last_unstale":"2024-04-25T14:11:34.603393+0000","last_undegraded":"2024-04-25T14:11:34.603393+0000","last_fullsized":"2024-04-25T14:11:34.603393+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:26.979452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659406+0000","last_change":"2024-04-25T14:11:33.616568+0000","last_active":"2024-04-25T14:11:34.659406+0000","last_peered":"2024-04-25T14:11:34.659406+0000","last_clean":"2024-04-25T14:11:34.659406+0000","last_became_active":"2024-04-25T14:11:33.616173+0000","last_became_peered":"2024-04-25T14:11:33.616173+0000","last_unstale":"2024-04-25T14:11:34.659406+0000","last_undegraded":"2024-04-25T14:11:34.659406+0000","last_fullsized":"2024-04-25T14:11:34.659406+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:23.600455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"139'3","reported_seq":17,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659324+0000","last_change":"2024-04-25T14:11:32.596904+0000","last_active":"2024-04-25T14:11:34.659324+0000","last_peered":"2024-04-25T14:11:34.659324+0000","last_clean":"2024-04-25T14:11:34.659324+0000","last_became_active":"2024-04-25T14:11:32.595555+0000","last_became_peered":"2024-04-25T14:11:32.595555+0000","last_unstale":"2024-04-25T14:11:34.659324+0000","last_undegraded":"2024-04-25T14:11:34.659324+0000","last_fullsized":"2024-04-25T14:11:34.659324+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:26.253319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659347+0000","last_change":"2024-04-25T14:11:33.607908+0000","last_active":"2024-04-25T14:11:34.659347+0000","last_peered":"2024-04-25T14:11:34.659347+0000","last_clean":"2024-04-25T14:11:34.659347+0000","last_became_active":"2024-04-25T14:11:33.607595+0000","last_became_peered":"2024-04-25T14:11:33.607595+0000","last_unstale":"2024-04-25T14:11:34.659347+0000","last_undegraded":"2024-04-25T14:11:34.659347+0000","last_fullsized":"2024-04-25T14:11:34.659347+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:10.040630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600654+0000","last_change":"2024-04-25T14:11:32.593779+0000","last_active":"2024-04-25T14:11:34.600654+0000","last_peered":"2024-04-25T14:11:34.600654+0000","last_clean":"2024-04-25T14:11:34.600654+0000","last_became_active":"2024-04-25T14:11:32.593185+0000","last_became_peered":"2024-04-25T14:11:32.593185+0000","last_unstale":"2024-04-25T14:11:34.600654+0000","last_undegraded":"2024-04-25T14:11:34.600654+0000","last_fullsized":"2024-04-25T14:11:34.600654+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:03.151785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602557+0000","last_change":"2024-04-25T14:11:33.623899+0000","last_active":"2024-04-25T14:11:34.602557+0000","last_peered":"2024-04-25T14:11:34.602557+0000","last_clean":"2024-04-25T14:11:34.602557+0000","last_became_active":"2024-04-25T14:11:33.621421+0000","last_became_peered":"2024-04-25T14:11:33.621421+0000","last_unstale":"2024-04-25T14:11:34.602557+0000","last_undegraded":"2024-04-25T14:11:34.602557+0000","last_fullsized":"2024-04-25T14:11:34.602557+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:50.996554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.660123+0000","last_change":"2024-04-25T14:11:32.602817+0000","last_active":"2024-04-25T14:11:34.660123+0000","last_peered":"2024-04-25T14:11:34.660123+0000","last_clean":"2024-04-25T14:11:34.660123+0000","last_became_active":"2024-04-25T14:11:32.602507+0000","last_became_peered":"2024-04-25T14:11:32.602507+0000","last_unstale":"2024-04-25T14:11:34.660123+0000","last_undegraded":"2024-04-25T14:11:34.660123+0000","last_fullsized":"2024-04-25T14:11:34.660123+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:45.684366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600077+0000","last_change":"2024-04-25T14:11:33.627596+0000","last_active":"2024-04-25T14:11:34.600077+0000","last_peered":"2024-04-25T14:11:34.600077+0000","last_clean":"2024-04-25T14:11:34.600077+0000","last_became_active":"2024-04-25T14:11:33.627282+0000","last_became_peered":"2024-04-25T14:11:33.627282+0000","last_unstale":"2024-04-25T14:11:34.600077+0000","last_undegraded":"2024-04-25T14:11:34.600077+0000","last_fullsized":"2024-04-25T14:11:34.600077+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:05.122162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.660786+0000","last_change":"2024-04-25T14:11:32.604165+0000","last_active":"2024-04-25T14:11:34.660786+0000","last_peered":"2024-04-25T14:11:34.660786+0000","last_clean":"2024-04-25T14:11:34.660786+0000","last_became_active":"2024-04-25T14:11:32.603846+0000","last_became_peered":"2024-04-25T14:11:32.603846+0000","last_unstale":"2024-04-25T14:11:34.660786+0000","last_undegraded":"2024-04-25T14:11:34.660786+0000","last_fullsized":"2024-04-25T14:11:34.660786+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:26.979452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602443+0000","last_change":"2024-04-25T14:11:33.624837+0000","last_active":"2024-04-25T14:11:34.602443+0000","last_peered":"2024-04-25T14:11:34.602443+0000","last_clean":"2024-04-25T14:11:34.602443+0000","last_became_active":"2024-04-25T14:11:33.624568+0000","last_became_peered":"2024-04-25T14:11:33.624568+0000","last_unstale":"2024-04-25T14:11:34.602443+0000","last_undegraded":"2024-04-25T14:11:34.602443+0000","last_fullsized":"2024-04-25T14:11:34.602443+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:56:42.715756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"139'8","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602537+0000","last_change":"2024-04-25T14:11:32.598249+0000","last_active":"2024-04-25T14:11:34.602537+0000","last_peered":"2024-04-25T14:11:34.602537+0000","last_clean":"2024-04-25T14:11:34.602537+0000","last_became_active":"2024-04-25T14:11:32.596737+0000","last_became_peered":"2024-04-25T14:11:32.596737+0000","last_unstale":"2024-04-25T14:11:34.602537+0000","last_undegraded":"2024-04-25T14:11:34.602537+0000","last_fullsized":"2024-04-25T14:11:34.602537+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:14.883395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659934+0000","last_change":"2024-04-25T14:11:33.615242+0000","last_active":"2024-04-25T14:11:34.659934+0000","last_peered":"2024-04-25T14:11:34.659934+0000","last_clean":"2024-04-25T14:11:34.659934+0000","last_became_active":"2024-04-25T14:11:33.611339+0000","last_became_peered":"2024-04-25T14:11:33.611339+0000","last_unstale":"2024-04-25T14:11:34.659934+0000","last_undegraded":"2024-04-25T14:11:34.659934+0000","last_fullsized":"2024-04-25T14:11:34.659934+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:02.982431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659386+0000","last_change":"2024-04-25T14:11:32.598361+0000","last_active":"2024-04-25T14:11:34.659386+0000","last_peered":"2024-04-25T14:11:34.659386+0000","last_clean":"2024-04-25T14:11:34.659386+0000","last_became_active":"2024-04-25T14:11:32.597623+0000","last_became_peered":"2024-04-25T14:11:32.597623+0000","last_unstale":"2024-04-25T14:11:34.659386+0000","last_undegraded":"2024-04-25T14:11:34.659386+0000","last_fullsized":"2024-04-25T14:11:34.659386+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:42.654645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659679+0000","last_change":"2024-04-25T14:11:33.610673+0000","last_active":"2024-04-25T14:11:34.659679+0000","last_peered":"2024-04-25T14:11:34.659679+0000","last_clean":"2024-04-25T14:11:34.659679+0000","last_became_active":"2024-04-25T14:11:33.610052+0000","last_became_peered":"2024-04-25T14:11:33.610052+0000","last_unstale":"2024-04-25T14:11:34.659679+0000","last_undegraded":"2024-04-25T14:11:34.659679+0000","last_fullsized":"2024-04-25T14:11:34.659679+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:50.996554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659525+0000","last_change":"2024-04-25T14:11:32.605007+0000","last_active":"2024-04-25T14:11:34.659525+0000","last_peered":"2024-04-25T14:11:34.659525+0000","last_clean":"2024-04-25T14:11:34.659525+0000","last_became_active":"2024-04-25T14:11:32.604678+0000","last_became_peered":"2024-04-25T14:11:32.604678+0000","last_unstale":"2024-04-25T14:11:34.659525+0000","last_undegraded":"2024-04-25T14:11:34.659525+0000","last_fullsized":"2024-04-25T14:11:34.659525+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:16.177533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.603334+0000","last_change":"2024-04-25T14:11:33.595266+0000","last_active":"2024-04-25T14:11:34.603334+0000","last_peered":"2024-04-25T14:11:34.603334+0000","last_clean":"2024-04-25T14:11:34.603334+0000","last_became_active":"2024-04-25T14:11:33.594843+0000","last_became_peered":"2024-04-25T14:11:33.594843+0000","last_unstale":"2024-04-25T14:11:34.603334+0000","last_undegraded":"2024-04-25T14:11:34.603334+0000","last_fullsized":"2024-04-25T14:11:34.603334+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:59.483240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602771+0000","last_change":"2024-04-25T14:11:32.603829+0000","last_active":"2024-04-25T14:11:34.602771+0000","last_peered":"2024-04-25T14:11:34.602771+0000","last_clean":"2024-04-25T14:11:34.602771+0000","last_became_active":"2024-04-25T14:11:32.603424+0000","last_became_peered":"2024-04-25T14:11:32.603424+0000","last_unstale":"2024-04-25T14:11:34.602771+0000","last_undegraded":"2024-04-25T14:11:34.602771+0000","last_fullsized":"2024-04-25T14:11:34.602771+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:31.082017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.603309+0000","last_change":"2024-04-25T14:11:33.588115+0000","last_active":"2024-04-25T14:11:34.603309+0000","last_peered":"2024-04-25T14:11:34.603309+0000","last_clean":"2024-04-25T14:11:34.603309+0000","last_became_active":"2024-04-25T14:11:33.587768+0000","last_became_peered":"2024-04-25T14:11:33.587768+0000","last_unstale":"2024-04-25T14:11:34.603309+0000","last_undegraded":"2024-04-25T14:11:34.603309+0000","last_fullsized":"2024-04-25T14:11:34.603309+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:40.795170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.660281+0000","last_change":"2024-04-25T14:11:32.603572+0000","last_active":"2024-04-25T14:11:34.660281+0000","last_peered":"2024-04-25T14:11:34.660281+0000","last_clean":"2024-04-25T14:11:34.660281+0000","last_became_active":"2024-04-25T14:11:32.603270+0000","last_became_peered":"2024-04-25T14:11:32.603270+0000","last_unstale":"2024-04-25T14:11:34.660281+0000","last_undegraded":"2024-04-25T14:11:34.660281+0000","last_fullsized":"2024-04-25T14:11:34.660281+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:50:44.121532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.658815+0000","last_change":"2024-04-25T14:11:33.603405+0000","last_active":"2024-04-25T14:11:34.658815+0000","last_peered":"2024-04-25T14:11:34.658815+0000","last_clean":"2024-04-25T14:11:34.658815+0000","last_became_active":"2024-04-25T14:11:33.602948+0000","last_became_peered":"2024-04-25T14:11:33.602948+0000","last_unstale":"2024-04-25T14:11:34.658815+0000","last_undegraded":"2024-04-25T14:11:34.658815+0000","last_fullsized":"2024-04-25T14:11:34.658815+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:49.326225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.658698+0000","last_change":"2024-04-25T14:11:32.593507+0000","last_active":"2024-04-25T14:11:34.658698+0000","last_peered":"2024-04-25T14:11:34.658698+0000","last_clean":"2024-04-25T14:11:34.658698+0000","last_became_active":"2024-04-25T14:11:32.593173+0000","last_became_peered":"2024-04-25T14:11:32.593173+0000","last_unstale":"2024-04-25T14:11:34.658698+0000","last_undegraded":"2024-04-25T14:11:34.658698+0000","last_fullsized":"2024-04-25T14:11:34.658698+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:01.673093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602661+0000","last_change":"2024-04-25T14:11:33.612339+0000","last_active":"2024-04-25T14:11:34.602661+0000","last_peered":"2024-04-25T14:11:34.602661+0000","last_clean":"2024-04-25T14:11:34.602661+0000","last_became_active":"2024-04-25T14:11:33.611897+0000","last_became_peered":"2024-04-25T14:11:33.611897+0000","last_unstale":"2024-04-25T14:11:34.602661+0000","last_undegraded":"2024-04-25T14:11:34.602661+0000","last_fullsized":"2024-04-25T14:11:34.602661+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:58:03.795240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659563+0000","last_change":"2024-04-25T14:11:32.606046+0000","last_active":"2024-04-25T14:11:34.659563+0000","last_peered":"2024-04-25T14:11:34.659563+0000","last_clean":"2024-04-25T14:11:34.659563+0000","last_became_active":"2024-04-25T14:11:32.605755+0000","last_became_peered":"2024-04-25T14:11:32.605755+0000","last_unstale":"2024-04-25T14:11:34.659563+0000","last_undegraded":"2024-04-25T14:11:34.659563+0000","last_fullsized":"2024-04-25T14:11:34.659563+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:00.011427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.658997+0000","last_change":"2024-04-25T14:11:33.625795+0000","last_active":"2024-04-25T14:11:34.658997+0000","last_peered":"2024-04-25T14:11:34.658997+0000","last_clean":"2024-04-25T14:11:34.658997+0000","last_became_active":"2024-04-25T14:11:33.625497+0000","last_became_peered":"2024-04-25T14:11:33.625497+0000","last_unstale":"2024-04-25T14:11:34.658997+0000","last_undegraded":"2024-04-25T14:11:34.658997+0000","last_fullsized":"2024-04-25T14:11:34.658997+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:07.671897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.658486+0000","last_change":"2024-04-25T14:11:32.591966+0000","last_active":"2024-04-25T14:11:34.658486+0000","last_peered":"2024-04-25T14:11:34.658486+0000","last_clean":"2024-04-25T14:11:34.658486+0000","last_became_active":"2024-04-25T14:11:32.591633+0000","last_became_peered":"2024-04-25T14:11:32.591633+0000","last_unstale":"2024-04-25T14:11:34.658486+0000","last_undegraded":"2024-04-25T14:11:34.658486+0000","last_fullsized":"2024-04-25T14:11:34.658486+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:57.758937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602912+0000","last_change":"2024-04-25T14:11:33.611220+0000","last_active":"2024-04-25T14:11:34.602912+0000","last_peered":"2024-04-25T14:11:34.602912+0000","last_clean":"2024-04-25T14:11:34.602912+0000","last_became_active":"2024-04-25T14:11:33.610910+0000","last_became_peered":"2024-04-25T14:11:33.610910+0000","last_unstale":"2024-04-25T14:11:34.602912+0000","last_undegraded":"2024-04-25T14:11:34.602912+0000","last_fullsized":"2024-04-25T14:11:34.602912+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:24:55.518082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659655+0000","last_change":"2024-04-25T14:11:32.590551+0000","last_active":"2024-04-25T14:11:34.659655+0000","last_peered":"2024-04-25T14:11:34.659655+0000","last_clean":"2024-04-25T14:11:34.659655+0000","last_became_active":"2024-04-25T14:11:32.590233+0000","last_became_peered":"2024-04-25T14:11:32.590233+0000","last_unstale":"2024-04-25T14:11:34.659655+0000","last_undegraded":"2024-04-25T14:11:34.659655+0000","last_fullsized":"2024-04-25T14:11:34.659655+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:57.130911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600731+0000","last_change":"2024-04-25T14:11:33.596960+0000","last_active":"2024-04-25T14:11:34.600731+0000","last_peered":"2024-04-25T14:11:34.600731+0000","last_clean":"2024-04-25T14:11:34.600731+0000","last_became_active":"2024-04-25T14:11:33.596123+0000","last_became_peered":"2024-04-25T14:11:33.596123+0000","last_unstale":"2024-04-25T14:11:34.600731+0000","last_undegraded":"2024-04-25T14:11:34.600731+0000","last_fullsized":"2024-04-25T14:11:34.600731+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:31.088654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.660670+0000","last_change":"2024-04-25T14:11:32.602907+0000","last_active":"2024-04-25T14:11:34.660670+0000","last_peered":"2024-04-25T14:11:34.660670+0000","last_clean":"2024-04-25T14:11:34.660670+0000","last_became_active":"2024-04-25T14:11:32.602600+0000","last_became_peered":"2024-04-25T14:11:32.602600+0000","last_unstale":"2024-04-25T14:11:34.660670+0000","last_undegraded":"2024-04-25T14:11:34.660670+0000","last_fullsized":"2024-04-25T14:11:34.660670+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:21.954678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.660303+0000","last_change":"2024-04-25T14:11:33.598878+0000","last_active":"2024-04-25T14:11:34.660303+0000","last_peered":"2024-04-25T14:11:34.660303+0000","last_clean":"2024-04-25T14:11:34.660303+0000","last_became_active":"2024-04-25T14:11:33.596559+0000","last_became_peered":"2024-04-25T14:11:33.596559+0000","last_unstale":"2024-04-25T14:11:34.660303+0000","last_undegraded":"2024-04-25T14:11:34.660303+0000","last_fullsized":"2024-04-25T14:11:34.660303+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.223787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"140'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.603498+0000","last_change":"2024-04-25T14:11:32.595961+0000","last_active":"2024-04-25T14:11:34.603498+0000","last_peered":"2024-04-25T14:11:34.603498+0000","last_clean":"2024-04-25T14:11:34.603498+0000","last_became_active":"2024-04-25T14:11:32.595641+0000","last_became_peered":"2024-04-25T14:11:32.595641+0000","last_unstale":"2024-04-25T14:11:34.603498+0000","last_undegraded":"2024-04-25T14:11:34.603498+0000","last_fullsized":"2024-04-25T14:11:34.603498+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:50:44.121532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.603310+0000","last_change":"2024-04-25T14:11:33.603158+0000","last_active":"2024-04-25T14:11:34.603310+0000","last_peered":"2024-04-25T14:11:34.603310+0000","last_clean":"2024-04-25T14:11:34.603310+0000","last_became_active":"2024-04-25T14:11:33.602866+0000","last_became_peered":"2024-04-25T14:11:33.602866+0000","last_unstale":"2024-04-25T14:11:34.603310+0000","last_undegraded":"2024-04-25T14:11:34.603310+0000","last_fullsized":"2024-04-25T14:11:34.603310+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:11.244407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602809+0000","last_change":"2024-04-25T14:11:32.603997+0000","last_active":"2024-04-25T14:11:34.602809+0000","last_peered":"2024-04-25T14:11:34.602809+0000","last_clean":"2024-04-25T14:11:34.602809+0000","last_became_active":"2024-04-25T14:11:32.603630+0000","last_became_peered":"2024-04-25T14:11:32.603630+0000","last_unstale":"2024-04-25T14:11:34.602809+0000","last_undegraded":"2024-04-25T14:11:34.602809+0000","last_fullsized":"2024-04-25T14:11:34.602809+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:44.182089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600567+0000","last_change":"2024-04-25T14:11:33.608436+0000","last_active":"2024-04-25T14:11:34.600567+0000","last_peered":"2024-04-25T14:11:34.600567+0000","last_clean":"2024-04-25T14:11:34.600567+0000","last_became_active":"2024-04-25T14:11:33.608046+0000","last_became_peered":"2024-04-25T14:11:33.608046+0000","last_unstale":"2024-04-25T14:11:34.600567+0000","last_undegraded":"2024-04-25T14:11:34.600567+0000","last_fullsized":"2024-04-25T14:11:34.600567+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:29.569454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600146+0000","last_change":"2024-04-25T14:11:32.590839+0000","last_active":"2024-04-25T14:11:34.600146+0000","last_peered":"2024-04-25T14:11:34.600146+0000","last_clean":"2024-04-25T14:11:34.600146+0000","last_became_active":"2024-04-25T14:11:32.590349+0000","last_became_peered":"2024-04-25T14:11:32.590349+0000","last_unstale":"2024-04-25T14:11:34.600146+0000","last_undegraded":"2024-04-25T14:11:34.600146+0000","last_fullsized":"2024-04-25T14:11:34.600146+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:56.025626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659543+0000","last_change":"2024-04-25T14:11:33.602353+0000","last_active":"2024-04-25T14:11:34.659543+0000","last_peered":"2024-04-25T14:11:34.659543+0000","last_clean":"2024-04-25T14:11:34.659543+0000","last_became_active":"2024-04-25T14:11:33.601990+0000","last_became_peered":"2024-04-25T14:11:33.601990+0000","last_unstale":"2024-04-25T14:11:34.659543+0000","last_undegraded":"2024-04-25T14:11:34.659543+0000","last_fullsized":"2024-04-25T14:11:34.659543+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:04.160373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.603084+0000","last_change":"2024-04-25T14:11:32.592951+0000","last_active":"2024-04-25T14:11:34.603084+0000","last_peered":"2024-04-25T14:11:34.603084+0000","last_clean":"2024-04-25T14:11:34.603084+0000","last_became_active":"2024-04-25T14:11:32.592568+0000","last_became_peered":"2024-04-25T14:11:32.592568+0000","last_unstale":"2024-04-25T14:11:34.603084+0000","last_undegraded":"2024-04-25T14:11:34.603084+0000","last_fullsized":"2024-04-25T14:11:34.603084+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:45.684366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.660753+0000","last_change":"2024-04-25T14:11:33.595107+0000","last_active":"2024-04-25T14:11:34.660753+0000","last_peered":"2024-04-25T14:11:34.660753+0000","last_clean":"2024-04-25T14:11:34.660753+0000","last_became_active":"2024-04-25T14:11:33.594810+0000","last_became_peered":"2024-04-25T14:11:33.594810+0000","last_unstale":"2024-04-25T14:11:34.660753+0000","last_undegraded":"2024-04-25T14:11:34.660753+0000","last_fullsized":"2024-04-25T14:11:34.660753+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:29.349474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.658920+0000","last_change":"2024-04-25T14:11:32.598702+0000","last_active":"2024-04-25T14:11:34.658920+0000","last_peered":"2024-04-25T14:11:34.658920+0000","last_clean":"2024-04-25T14:11:34.658920+0000","last_became_active":"2024-04-25T14:11:32.598372+0000","last_became_peered":"2024-04-25T14:11:32.598372+0000","last_unstale":"2024-04-25T14:11:34.658920+0000","last_undegraded":"2024-04-25T14:11:34.658920+0000","last_fullsized":"2024-04-25T14:11:34.658920+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:58.105185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602593+0000","last_change":"2024-04-25T14:11:33.619891+0000","last_active":"2024-04-25T14:11:34.602593+0000","last_peered":"2024-04-25T14:11:34.602593+0000","last_clean":"2024-04-25T14:11:34.602593+0000","last_became_active":"2024-04-25T14:11:33.619416+0000","last_became_peered":"2024-04-25T14:11:33.619416+0000","last_unstale":"2024-04-25T14:11:34.602593+0000","last_undegraded":"2024-04-25T14:11:34.602593+0000","last_fullsized":"2024-04-25T14:11:34.602593+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:02.982431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602910+0000","last_change":"2024-04-25T14:11:32.596091+0000","last_active":"2024-04-25T14:11:34.602910+0000","last_peered":"2024-04-25T14:11:34.602910+0000","last_clean":"2024-04-25T14:11:34.602910+0000","last_became_active":"2024-04-25T14:11:32.595711+0000","last_became_peered":"2024-04-25T14:11:32.595711+0000","last_unstale":"2024-04-25T14:11:34.602910+0000","last_undegraded":"2024-04-25T14:11:34.602910+0000","last_fullsized":"2024-04-25T14:11:34.602910+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:17.575105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659319+0000","last_change":"2024-04-25T14:11:33.609197+0000","last_active":"2024-04-25T14:11:34.659319+0000","last_peered":"2024-04-25T14:11:34.659319+0000","last_clean":"2024-04-25T14:11:34.659319+0000","last_became_active":"2024-04-25T14:11:33.608865+0000","last_became_peered":"2024-04-25T14:11:33.608865+0000","last_unstale":"2024-04-25T14:11:34.659319+0000","last_undegraded":"2024-04-25T14:11:34.659319+0000","last_fullsized":"2024-04-25T14:11:34.659319+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:45.190677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.658970+0000","last_change":"2024-04-25T14:11:32.599842+0000","last_active":"2024-04-25T14:11:34.658970+0000","last_peered":"2024-04-25T14:11:34.658970+0000","last_clean":"2024-04-25T14:11:34.658970+0000","last_became_active":"2024-04-25T14:11:32.599226+0000","last_became_peered":"2024-04-25T14:11:32.599226+0000","last_unstale":"2024-04-25T14:11:34.658970+0000","last_undegraded":"2024-04-25T14:11:34.658970+0000","last_fullsized":"2024-04-25T14:11:34.658970+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:03.151785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.660452+0000","last_change":"2024-04-25T14:11:33.619555+0000","last_active":"2024-04-25T14:11:34.660452+0000","last_peered":"2024-04-25T14:11:34.660452+0000","last_clean":"2024-04-25T14:11:34.660452+0000","last_became_active":"2024-04-25T14:11:33.618630+0000","last_became_peered":"2024-04-25T14:11:33.618630+0000","last_unstale":"2024-04-25T14:11:34.660452+0000","last_undegraded":"2024-04-25T14:11:34.660452+0000","last_fullsized":"2024-04-25T14:11:34.660452+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:46:19.184891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"139'6","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.658859+0000","last_change":"2024-04-25T14:11:32.593191+0000","last_active":"2024-04-25T14:11:34.658859+0000","last_peered":"2024-04-25T14:11:34.658859+0000","last_clean":"2024-04-25T14:11:34.658859+0000","last_became_active":"2024-04-25T14:11:32.592628+0000","last_became_peered":"2024-04-25T14:11:32.592628+0000","last_unstale":"2024-04-25T14:11:34.658859+0000","last_undegraded":"2024-04-25T14:11:34.658859+0000","last_fullsized":"2024-04-25T14:11:34.658859+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:41.184218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"129'3277","reported_seq":3754,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659103+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:11:34.659103+0000","last_peered":"2024-04-25T14:11:34.659103+0000","last_clean":"2024-04-25T14:11:34.659103+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:11:34.659103+0000","last_undegraded":"2024-04-25T14:11:34.659103+0000","last_fullsized":"2024-04-25T14:11:34.659103+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":3277,"log_dups_size":0,"ondisk_log_size":3277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":717,"num_read_kb":1927,"num_write":223,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":717,"num_read_kb":1927,"num_write":223,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3277,"ondisk_log_size":3277,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574940,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29320,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342520,"statfs":{"total":96636764160,"available":96606740480,"internally_reserved":0,"allocated":2355200,"data_stored":1470546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574941,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32900,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338940,"statfs":{"total":96636764160,"available":96603074560,"internally_reserved":0,"allocated":1761280,"data_stored":880226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574942,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32896,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338944,"statfs":{"total":96636764160,"available":96603078656,"internally_reserved":0,"allocated":1757184,"data_stored":880156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574941,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32908,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338932,"statfs":{"total":96636764160,"available":96603066368,"internally_reserved":0,"allocated":1769472,"data_stored":881768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574941,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1765376,"data_stored":880772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574940,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1765376,"data_stored":880268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32900,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338940,"statfs":{"total":96636764160,"available":96603074560,"internally_reserved":0,"allocated":1761280,"data_stored":880226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574941,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29344,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342496,"statfs":{"total":96636764160,"available":96606715904,"internally_reserved":0,"allocated":2375680,"data_stored":1472842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,1,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:11:42.298 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:11:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.461+0000 7f105fc00640 1 Processor -- start 2024-04-25T14:11:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.461+0000 7f105fc00640 1 -- start start 2024-04-25T14:11:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.461+0000 7f105fc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f10580cde90 0x7f10580ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.461+0000 7f105fc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f105815e5c0 con 0x7f10580d1df0 2024-04-25T14:11:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.461+0000 7f105fc00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f105815e8a0 con 0x7f1058070330 2024-04-25T14:11:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.461+0000 7f105fc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f105806fd40 con 0x7f10580cde90 2024-04-25T14:11:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.461+0000 7f105dc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f10580cde90 0x7f10580ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.461+0000 7f1057e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f10580d1df0 0x7f10580cd780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44020/0 (socket says 172.21.15.86:44020) 2024-04-25T14:11:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.461+0000 7f1057e00640 1 -- 172.21.15.86:0/2721155020 learned_addr learned my addr 172.21.15.86:0/2721155020 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.461+0000 7f105dc00640 1 -- 172.21.15.86:0/2721155020 >> v1:172.21.15.155:6789/0 conn(0x7f1058070330 legacy=0x7f10580d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.461+0000 7f105dc00640 1 -- 172.21.15.86:0/2721155020 >> v1:172.21.15.86:6789/0 conn(0x7f10580d1df0 legacy=0x7f10580cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.461+0000 7f105dc00640 1 -- 172.21.15.86:0/2721155020 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f105815eb80 con 0x7f10580cde90 2024-04-25T14:11:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.461+0000 7f1057400640 1 -- 172.21.15.86:0/2721155020 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 840556275 0 0) 0x7f105815e5c0 con 0x7f10580d1df0 2024-04-25T14:11:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.465+0000 7f105dc00640 1 --2- 172.21.15.86:0/2721155020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10580cde90 0x7f10580ce2a0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f10500097f0 tx=0x7f105001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8707e94394a94e10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.465+0000 7f1057400640 1 -- 172.21.15.86:0/2721155020 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10500b0070 con 0x7f10580cde90 2024-04-25T14:11:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.465+0000 7f1057400640 1 -- 172.21.15.86:0/2721155020 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10500ab600 con 0x7f10580cde90 2024-04-25T14:11:42.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.465+0000 7f1057400640 1 -- 172.21.15.86:0/2721155020 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10500ab9d0 con 0x7f10580cde90 2024-04-25T14:11:42.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.465+0000 7f105fc00640 1 -- 172.21.15.86:0/2721155020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10580cde90 msgr2=0x7f10580ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:42.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.465+0000 7f105fc00640 1 --2- 172.21.15.86:0/2721155020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10580cde90 0x7f10580ce2a0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f10500097f0 tx=0x7f105001c3e0 comp rx=0 tx=0).stop 2024-04-25T14:11:42.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.465+0000 7f105fc00640 1 -- 172.21.15.86:0/2721155020 shutdown_connections 2024-04-25T14:11:42.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.465+0000 7f105fc00640 1 --2- 172.21.15.86:0/2721155020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10580cde90 0x7f10580ce2a0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:42.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.465+0000 7f105fc00640 1 -- 172.21.15.86:0/2721155020 >> 172.21.15.86:0/2721155020 conn(0x7f10580b4650 msgr2=0x7f10580b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:42.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.465+0000 7f105fc00640 1 -- 172.21.15.86:0/2721155020 shutdown_connections 2024-04-25T14:11:42.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.465+0000 7f105fc00640 1 -- 172.21.15.86:0/2721155020 wait complete. 2024-04-25T14:11:42.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.469+0000 7f105fc00640 1 Processor -- start 2024-04-25T14:11:42.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.469+0000 7f105fc00640 1 -- start start 2024-04-25T14:11:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.469+0000 7f105fc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1058070330 0x7f10580cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.469+0000 7f105fc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10580cde90 0x7f105815ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.469+0000 7f105fc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f105815b220 0x7f10580c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.469+0000 7f105fc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f10580708c0 con 0x7f1058070330 2024-04-25T14:11:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.469+0000 7f105fc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1058070740 con 0x7f10580cde90 2024-04-25T14:11:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.469+0000 7f105fc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f105816bfd0 con 0x7f105815b220 2024-04-25T14:11:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.469+0000 7f105d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1058070330 0x7f10580cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.469+0000 7f105d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1058070330 0x7f10580cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52936/0 (socket says 172.21.15.86:52936) 2024-04-25T14:11:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.469+0000 7f105d200640 1 -- 172.21.15.86:0/254522865 learned_addr learned my addr 172.21.15.86:0/254522865 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.469+0000 7f1057e00640 1 --2- 172.21.15.86:0/254522865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10580cde90 0x7f105815ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.469+0000 7f105dc00640 1 --2- 172.21.15.86:0/254522865 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f105815b220 0x7f10580c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.469+0000 7f105d200640 1 -- 172.21.15.86:0/254522865 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f105815b220 msgr2=0x7f10580c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.469+0000 7f105d200640 1 --2- 172.21.15.86:0/254522865 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f105815b220 0x7f10580c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.469+0000 7f105d200640 1 -- 172.21.15.86:0/254522865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10580cde90 msgr2=0x7f105815ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.469+0000 7f105d200640 1 --2- 172.21.15.86:0/254522865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10580cde90 0x7f105815ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.469+0000 7f105d200640 1 -- 172.21.15.86:0/254522865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1058159f20 con 0x7f1058070330 2024-04-25T14:11:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.469+0000 7f105dc00640 1 --2- 172.21.15.86:0/254522865 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f105815b220 0x7f10580c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.469+0000 7f1057e00640 1 --2- 172.21.15.86:0/254522865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10580cde90 0x7f105815ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:42.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.469+0000 7f105d200640 1 --2- 172.21.15.86:0/254522865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1058070330 0x7f10580cfd70 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f104800c760 tx=0x7f104800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:42.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.469+0000 7f1054c00640 1 -- 172.21.15.86:0/254522865 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1048003b80 con 0x7f1058070330 2024-04-25T14:11:42.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.469+0000 7f1054c00640 1 -- 172.21.15.86:0/254522865 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1048003d20 con 0x7f1058070330 2024-04-25T14:11:42.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.469+0000 7f105fc00640 1 -- 172.21.15.86:0/254522865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f105815a200 con 0x7f1058070330 2024-04-25T14:11:42.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.469+0000 7f1054c00640 1 -- 172.21.15.86:0/254522865 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104800aa90 con 0x7f1058070330 2024-04-25T14:11:42.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.469+0000 7f105fc00640 1 -- 172.21.15.86:0/254522865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f105815a7c0 con 0x7f1058070330 2024-04-25T14:11:42.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.473+0000 7f105fc00640 1 -- 172.21.15.86:0/254522865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1058069d00 con 0x7f1058070330 2024-04-25T14:11:42.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.473+0000 7f1054c00640 1 -- 172.21.15.86:0/254522865 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7f104800ac30 con 0x7f1058070330 2024-04-25T14:11:42.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.473+0000 7f1054c00640 1 --2- 172.21.15.86:0/254522865 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f103806f520 0x7f1038071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:42.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.473+0000 7f1054c00640 1 -- 172.21.15.86:0/254522865 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f104808f500 con 0x7f1058070330 2024-04-25T14:11:42.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.473+0000 7f1057e00640 1 --2- 172.21.15.86:0/254522865 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f103806f520 0x7f1038071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:42.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.473+0000 7f1057e00640 1 --2- 172.21.15.86:0/254522865 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f103806f520 0x7f1038071a30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1058159ca0 tx=0x7f1044007df0 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:42.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.477+0000 7f1054c00640 1 -- 172.21.15.86:0/254522865 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f104805c150 con 0x7f1058070330 2024-04-25T14:11:42.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.633+0000 7f105fc00640 1 -- 172.21.15.86:0/254522865 --> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1058073ea0 con 0x7f103806f520 2024-04-25T14:11:42.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.641+0000 7f1054c00640 1 -- 172.21.15.86:0/254522865 <== mgr.9282 v2:172.21.15.86:6832/264890479 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252525 (secure 0 0 0) 0x7f1058073ea0 con 0x7f103806f520 2024-04-25T14:11:42.639 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:42.640 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:11:42.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.645+0000 7f105fc00640 1 -- 172.21.15.86:0/254522865 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f103806f520 msgr2=0x7f1038071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:42.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.645+0000 7f105fc00640 1 --2- 172.21.15.86:0/254522865 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f103806f520 0x7f1038071a30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1058159ca0 tx=0x7f1044007df0 comp rx=0 tx=0).stop 2024-04-25T14:11:42.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.645+0000 7f105fc00640 1 -- 172.21.15.86:0/254522865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1058070330 msgr2=0x7f10580cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:42.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.645+0000 7f105fc00640 1 --2- 172.21.15.86:0/254522865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1058070330 0x7f10580cfd70 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f104800c760 tx=0x7f104800cc20 comp rx=0 tx=0).stop 2024-04-25T14:11:42.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.645+0000 7f105fc00640 1 -- 172.21.15.86:0/254522865 shutdown_connections 2024-04-25T14:11:42.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.645+0000 7f105fc00640 1 --2- 172.21.15.86:0/254522865 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f103806f520 0x7f1038071a30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:42.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.645+0000 7f105fc00640 1 --2- 172.21.15.86:0/254522865 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f105815b220 0x7f10580c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:42.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.645+0000 7f105fc00640 1 --2- 172.21.15.86:0/254522865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10580cde90 0x7f105815ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:42.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.645+0000 7f105fc00640 1 --2- 172.21.15.86:0/254522865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1058070330 0x7f10580cfd70 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:42.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.645+0000 7f105fc00640 1 -- 172.21.15.86:0/254522865 >> 172.21.15.86:0/254522865 conn(0x7f10580b4650 msgr2=0x7f105815bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:42.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.645+0000 7f105fc00640 1 -- 172.21.15.86:0/254522865 shutdown_connections 2024-04-25T14:11:42.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.645+0000 7f105fc00640 1 -- 172.21.15.86:0/254522865 wait complete. 2024-04-25T14:11:42.660 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-04-25T14:11:40.735256+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":717,"num_read_kb":1927,"num_write":268,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3316,"ondisk_log_size":3316,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256076,"kb_used_data":14952,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754718644,"statfs":{"total":773094113280,"available":772831891456,"internally_reserved":0,"allocated":15310848,"data_stored":8226804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,1,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001716"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.603137+0000","last_change":"2024-04-25T14:11:33.600545+0000","last_active":"2024-04-25T14:11:34.603137+0000","last_peered":"2024-04-25T14:11:34.603137+0000","last_clean":"2024-04-25T14:11:34.603137+0000","last_became_active":"2024-04-25T14:11:33.599990+0000","last_became_peered":"2024-04-25T14:11:33.599990+0000","last_unstale":"2024-04-25T14:11:34.603137+0000","last_undegraded":"2024-04-25T14:11:34.603137+0000","last_fullsized":"2024-04-25T14:11:34.603137+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:57.034214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602667+0000","last_change":"2024-04-25T14:11:33.614602+0000","last_active":"2024-04-25T14:11:34.602667+0000","last_peered":"2024-04-25T14:11:34.602667+0000","last_clean":"2024-04-25T14:11:34.602667+0000","last_became_active":"2024-04-25T14:11:33.614279+0000","last_became_peered":"2024-04-25T14:11:33.614279+0000","last_unstale":"2024-04-25T14:11:34.602667+0000","last_undegraded":"2024-04-25T14:11:34.602667+0000","last_fullsized":"2024-04-25T14:11:34.602667+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:05.122162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602490+0000","last_change":"2024-04-25T14:11:33.612130+0000","last_active":"2024-04-25T14:11:34.602490+0000","last_peered":"2024-04-25T14:11:34.602490+0000","last_clean":"2024-04-25T14:11:34.602490+0000","last_became_active":"2024-04-25T14:11:33.611772+0000","last_became_peered":"2024-04-25T14:11:33.611772+0000","last_unstale":"2024-04-25T14:11:34.602490+0000","last_undegraded":"2024-04-25T14:11:34.602490+0000","last_fullsized":"2024-04-25T14:11:34.602490+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:17.778893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.660010+0000","last_change":"2024-04-25T14:11:33.620093+0000","last_active":"2024-04-25T14:11:34.660010+0000","last_peered":"2024-04-25T14:11:34.660010+0000","last_clean":"2024-04-25T14:11:34.660010+0000","last_became_active":"2024-04-25T14:11:33.619776+0000","last_became_peered":"2024-04-25T14:11:33.619776+0000","last_unstale":"2024-04-25T14:11:34.660010+0000","last_undegraded":"2024-04-25T14:11:34.660010+0000","last_fullsized":"2024-04-25T14:11:34.660010+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:10.040630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600597+0000","last_change":"2024-04-25T14:11:33.602823+0000","last_active":"2024-04-25T14:11:34.600597+0000","last_peered":"2024-04-25T14:11:34.600597+0000","last_clean":"2024-04-25T14:11:34.600597+0000","last_became_active":"2024-04-25T14:11:33.593014+0000","last_became_peered":"2024-04-25T14:11:33.593014+0000","last_unstale":"2024-04-25T14:11:34.600597+0000","last_undegraded":"2024-04-25T14:11:34.600597+0000","last_fullsized":"2024-04-25T14:11:34.600597+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:21.294240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600359+0000","last_change":"2024-04-25T14:11:33.593267+0000","last_active":"2024-04-25T14:11:34.600359+0000","last_peered":"2024-04-25T14:11:34.600359+0000","last_clean":"2024-04-25T14:11:34.600359+0000","last_became_active":"2024-04-25T14:11:33.592870+0000","last_became_peered":"2024-04-25T14:11:33.592870+0000","last_unstale":"2024-04-25T14:11:34.600359+0000","last_undegraded":"2024-04-25T14:11:34.600359+0000","last_fullsized":"2024-04-25T14:11:34.600359+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:05.246429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602729+0000","last_change":"2024-04-25T14:11:33.598268+0000","last_active":"2024-04-25T14:11:34.602729+0000","last_peered":"2024-04-25T14:11:34.602729+0000","last_clean":"2024-04-25T14:11:34.602729+0000","last_became_active":"2024-04-25T14:11:33.597942+0000","last_became_peered":"2024-04-25T14:11:33.597942+0000","last_unstale":"2024-04-25T14:11:34.602729+0000","last_undegraded":"2024-04-25T14:11:34.602729+0000","last_fullsized":"2024-04-25T14:11:34.602729+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:04.160373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659001+0000","last_change":"2024-04-25T14:11:33.616240+0000","last_active":"2024-04-25T14:11:34.659001+0000","last_peered":"2024-04-25T14:11:34.659001+0000","last_clean":"2024-04-25T14:11:34.659001+0000","last_became_active":"2024-04-25T14:11:33.615903+0000","last_became_peered":"2024-04-25T14:11:33.615903+0000","last_unstale":"2024-04-25T14:11:34.659001+0000","last_undegraded":"2024-04-25T14:11:34.659001+0000","last_fullsized":"2024-04-25T14:11:34.659001+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:04.673178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600405+0000","last_change":"2024-04-25T14:11:33.603215+0000","last_active":"2024-04-25T14:11:34.600405+0000","last_peered":"2024-04-25T14:11:34.600405+0000","last_clean":"2024-04-25T14:11:34.600405+0000","last_became_active":"2024-04-25T14:11:33.602886+0000","last_became_peered":"2024-04-25T14:11:33.602886+0000","last_unstale":"2024-04-25T14:11:34.600405+0000","last_undegraded":"2024-04-25T14:11:34.600405+0000","last_fullsized":"2024-04-25T14:11:34.600405+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:57.463642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600352+0000","last_change":"2024-04-25T14:11:33.602442+0000","last_active":"2024-04-25T14:11:34.600352+0000","last_peered":"2024-04-25T14:11:34.600352+0000","last_clean":"2024-04-25T14:11:34.600352+0000","last_became_active":"2024-04-25T14:11:33.602108+0000","last_became_peered":"2024-04-25T14:11:33.602108+0000","last_unstale":"2024-04-25T14:11:34.600352+0000","last_undegraded":"2024-04-25T14:11:34.600352+0000","last_fullsized":"2024-04-25T14:11:34.600352+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:24:55.518082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659208+0000","last_change":"2024-04-25T14:11:33.627206+0000","last_active":"2024-04-25T14:11:34.659208+0000","last_peered":"2024-04-25T14:11:34.659208+0000","last_clean":"2024-04-25T14:11:34.659208+0000","last_became_active":"2024-04-25T14:11:33.626877+0000","last_became_peered":"2024-04-25T14:11:33.626877+0000","last_unstale":"2024-04-25T14:11:34.659208+0000","last_undegraded":"2024-04-25T14:11:34.659208+0000","last_fullsized":"2024-04-25T14:11:34.659208+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:13.084865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.660179+0000","last_change":"2024-04-25T14:11:33.596026+0000","last_active":"2024-04-25T14:11:34.660179+0000","last_peered":"2024-04-25T14:11:34.660179+0000","last_clean":"2024-04-25T14:11:34.660179+0000","last_became_active":"2024-04-25T14:11:33.595721+0000","last_became_peered":"2024-04-25T14:11:33.595721+0000","last_unstale":"2024-04-25T14:11:34.660179+0000","last_undegraded":"2024-04-25T14:11:34.660179+0000","last_fullsized":"2024-04-25T14:11:34.660179+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:04.160373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600451+0000","last_change":"2024-04-25T14:11:33.602904+0000","last_active":"2024-04-25T14:11:34.600451+0000","last_peered":"2024-04-25T14:11:34.600451+0000","last_clean":"2024-04-25T14:11:34.600451+0000","last_became_active":"2024-04-25T14:11:33.599935+0000","last_became_peered":"2024-04-25T14:11:33.599935+0000","last_unstale":"2024-04-25T14:11:34.600451+0000","last_undegraded":"2024-04-25T14:11:34.600451+0000","last_fullsized":"2024-04-25T14:11:34.600451+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:49:36.789630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.660405+0000","last_change":"2024-04-25T14:11:33.597740+0000","last_active":"2024-04-25T14:11:34.660405+0000","last_peered":"2024-04-25T14:11:34.660405+0000","last_clean":"2024-04-25T14:11:34.660405+0000","last_became_active":"2024-04-25T14:11:33.597430+0000","last_became_peered":"2024-04-25T14:11:33.597430+0000","last_unstale":"2024-04-25T14:11:34.660405+0000","last_undegraded":"2024-04-25T14:11:34.660405+0000","last_fullsized":"2024-04-25T14:11:34.660405+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:17.186121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602355+0000","last_change":"2024-04-25T14:11:33.615341+0000","last_active":"2024-04-25T14:11:34.602355+0000","last_peered":"2024-04-25T14:11:34.602355+0000","last_clean":"2024-04-25T14:11:34.602355+0000","last_became_active":"2024-04-25T14:11:33.614985+0000","last_became_peered":"2024-04-25T14:11:33.614985+0000","last_unstale":"2024-04-25T14:11:34.602355+0000","last_undegraded":"2024-04-25T14:11:34.602355+0000","last_fullsized":"2024-04-25T14:11:34.602355+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:49.326225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600068+0000","last_change":"2024-04-25T14:11:33.608973+0000","last_active":"2024-04-25T14:11:34.600068+0000","last_peered":"2024-04-25T14:11:34.600068+0000","last_clean":"2024-04-25T14:11:34.600068+0000","last_became_active":"2024-04-25T14:11:33.608576+0000","last_became_peered":"2024-04-25T14:11:33.608576+0000","last_unstale":"2024-04-25T14:11:34.600068+0000","last_undegraded":"2024-04-25T14:11:34.600068+0000","last_fullsized":"2024-04-25T14:11:34.600068+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:58:03.795240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.660667+0000","last_change":"2024-04-25T14:11:33.598496+0000","last_active":"2024-04-25T14:11:34.660667+0000","last_peered":"2024-04-25T14:11:34.660667+0000","last_clean":"2024-04-25T14:11:34.660667+0000","last_became_active":"2024-04-25T14:11:33.598197+0000","last_became_peered":"2024-04-25T14:11:33.598197+0000","last_unstale":"2024-04-25T14:11:34.660667+0000","last_undegraded":"2024-04-25T14:11:34.660667+0000","last_fullsized":"2024-04-25T14:11:34.660667+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:57.034214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659300+0000","last_change":"2024-04-25T14:11:33.617124+0000","last_active":"2024-04-25T14:11:34.659300+0000","last_peered":"2024-04-25T14:11:34.659300+0000","last_clean":"2024-04-25T14:11:34.659300+0000","last_became_active":"2024-04-25T14:11:33.616830+0000","last_became_peered":"2024-04-25T14:11:33.616830+0000","last_unstale":"2024-04-25T14:11:34.659300+0000","last_undegraded":"2024-04-25T14:11:34.659300+0000","last_fullsized":"2024-04-25T14:11:34.659300+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:04.637134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659498+0000","last_change":"2024-04-25T14:11:33.610363+0000","last_active":"2024-04-25T14:11:34.659498+0000","last_peered":"2024-04-25T14:11:34.659498+0000","last_clean":"2024-04-25T14:11:34.659498+0000","last_became_active":"2024-04-25T14:11:33.609578+0000","last_became_peered":"2024-04-25T14:11:33.609578+0000","last_unstale":"2024-04-25T14:11:34.659498+0000","last_undegraded":"2024-04-25T14:11:34.659498+0000","last_fullsized":"2024-04-25T14:11:34.659498+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:17.778893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.658728+0000","last_change":"2024-04-25T14:11:33.622426+0000","last_active":"2024-04-25T14:11:34.658728+0000","last_peered":"2024-04-25T14:11:34.658728+0000","last_clean":"2024-04-25T14:11:34.658728+0000","last_became_active":"2024-04-25T14:11:33.621583+0000","last_became_peered":"2024-04-25T14:11:33.621583+0000","last_unstale":"2024-04-25T14:11:34.658728+0000","last_undegraded":"2024-04-25T14:11:34.658728+0000","last_fullsized":"2024-04-25T14:11:34.658728+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:32.466772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600561+0000","last_change":"2024-04-25T14:11:33.603619+0000","last_active":"2024-04-25T14:11:34.600561+0000","last_peered":"2024-04-25T14:11:34.600561+0000","last_clean":"2024-04-25T14:11:34.600561+0000","last_became_active":"2024-04-25T14:11:33.603206+0000","last_became_peered":"2024-04-25T14:11:33.603206+0000","last_unstale":"2024-04-25T14:11:34.600561+0000","last_undegraded":"2024-04-25T14:11:34.600561+0000","last_fullsized":"2024-04-25T14:11:34.600561+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:31.565395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602769+0000","last_change":"2024-04-25T14:11:33.601209+0000","last_active":"2024-04-25T14:11:34.602769+0000","last_peered":"2024-04-25T14:11:34.602769+0000","last_clean":"2024-04-25T14:11:34.602769+0000","last_became_active":"2024-04-25T14:11:33.600904+0000","last_became_peered":"2024-04-25T14:11:33.600904+0000","last_unstale":"2024-04-25T14:11:34.602769+0000","last_undegraded":"2024-04-25T14:11:34.602769+0000","last_fullsized":"2024-04-25T14:11:34.602769+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.223787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659180+0000","last_change":"2024-04-25T14:11:33.595593+0000","last_active":"2024-04-25T14:11:34.659180+0000","last_peered":"2024-04-25T14:11:34.659180+0000","last_clean":"2024-04-25T14:11:34.659180+0000","last_became_active":"2024-04-25T14:11:33.595294+0000","last_became_peered":"2024-04-25T14:11:33.595294+0000","last_unstale":"2024-04-25T14:11:34.659180+0000","last_undegraded":"2024-04-25T14:11:34.659180+0000","last_fullsized":"2024-04-25T14:11:34.659180+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:32.090605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.658624+0000","last_change":"2024-04-25T14:11:33.616089+0000","last_active":"2024-04-25T14:11:34.658624+0000","last_peered":"2024-04-25T14:11:34.658624+0000","last_clean":"2024-04-25T14:11:34.658624+0000","last_became_active":"2024-04-25T14:11:33.615770+0000","last_became_peered":"2024-04-25T14:11:33.615770+0000","last_unstale":"2024-04-25T14:11:34.658624+0000","last_undegraded":"2024-04-25T14:11:34.658624+0000","last_fullsized":"2024-04-25T14:11:34.658624+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:23.567530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.603242+0000","last_change":"2024-04-25T14:11:33.600201+0000","last_active":"2024-04-25T14:11:34.603242+0000","last_peered":"2024-04-25T14:11:34.603242+0000","last_clean":"2024-04-25T14:11:34.603242+0000","last_became_active":"2024-04-25T14:11:33.599319+0000","last_became_peered":"2024-04-25T14:11:33.599319+0000","last_unstale":"2024-04-25T14:11:34.603242+0000","last_undegraded":"2024-04-25T14:11:34.603242+0000","last_fullsized":"2024-04-25T14:11:34.603242+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:29.349474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.603196+0000","last_change":"2024-04-25T14:11:33.617962+0000","last_active":"2024-04-25T14:11:34.603196+0000","last_peered":"2024-04-25T14:11:34.603196+0000","last_clean":"2024-04-25T14:11:34.603196+0000","last_became_active":"2024-04-25T14:11:33.617613+0000","last_became_peered":"2024-04-25T14:11:33.617613+0000","last_unstale":"2024-04-25T14:11:34.603196+0000","last_undegraded":"2024-04-25T14:11:34.603196+0000","last_fullsized":"2024-04-25T14:11:34.603196+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:46:19.184891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600234+0000","last_change":"2024-04-25T14:11:33.603938+0000","last_active":"2024-04-25T14:11:34.600234+0000","last_peered":"2024-04-25T14:11:34.600234+0000","last_clean":"2024-04-25T14:11:34.600234+0000","last_became_active":"2024-04-25T14:11:33.603587+0000","last_became_peered":"2024-04-25T14:11:33.603587+0000","last_unstale":"2024-04-25T14:11:34.600234+0000","last_undegraded":"2024-04-25T14:11:34.600234+0000","last_fullsized":"2024-04-25T14:11:34.600234+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:16.358014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659150+0000","last_change":"2024-04-25T14:11:33.623158+0000","last_active":"2024-04-25T14:11:34.659150+0000","last_peered":"2024-04-25T14:11:34.659150+0000","last_clean":"2024-04-25T14:11:34.659150+0000","last_became_active":"2024-04-25T14:11:33.622821+0000","last_became_peered":"2024-04-25T14:11:33.622821+0000","last_unstale":"2024-04-25T14:11:34.659150+0000","last_undegraded":"2024-04-25T14:11:34.659150+0000","last_fullsized":"2024-04-25T14:11:34.659150+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:05.771659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659754+0000","last_change":"2024-04-25T14:11:33.609928+0000","last_active":"2024-04-25T14:11:34.659754+0000","last_peered":"2024-04-25T14:11:34.659754+0000","last_clean":"2024-04-25T14:11:34.659754+0000","last_became_active":"2024-04-25T14:11:33.609569+0000","last_became_peered":"2024-04-25T14:11:33.609569+0000","last_unstale":"2024-04-25T14:11:34.659754+0000","last_undegraded":"2024-04-25T14:11:34.659754+0000","last_fullsized":"2024-04-25T14:11:34.659754+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:17.186121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.658814+0000","last_change":"2024-04-25T14:11:33.620076+0000","last_active":"2024-04-25T14:11:34.658814+0000","last_peered":"2024-04-25T14:11:34.658814+0000","last_clean":"2024-04-25T14:11:34.658814+0000","last_became_active":"2024-04-25T14:11:33.619774+0000","last_became_peered":"2024-04-25T14:11:33.619774+0000","last_unstale":"2024-04-25T14:11:34.658814+0000","last_undegraded":"2024-04-25T14:11:34.658814+0000","last_fullsized":"2024-04-25T14:11:34.658814+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:35.800429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600226+0000","last_change":"2024-04-25T14:11:33.597817+0000","last_active":"2024-04-25T14:11:34.600226+0000","last_peered":"2024-04-25T14:11:34.600226+0000","last_clean":"2024-04-25T14:11:34.600226+0000","last_became_active":"2024-04-25T14:11:33.597513+0000","last_became_peered":"2024-04-25T14:11:33.597513+0000","last_unstale":"2024-04-25T14:11:34.600226+0000","last_undegraded":"2024-04-25T14:11:34.600226+0000","last_fullsized":"2024-04-25T14:11:34.600226+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:58.403145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659174+0000","last_change":"2024-04-25T14:11:33.612165+0000","last_active":"2024-04-25T14:11:34.659174+0000","last_peered":"2024-04-25T14:11:34.659174+0000","last_clean":"2024-04-25T14:11:34.659174+0000","last_became_active":"2024-04-25T14:11:33.611831+0000","last_became_peered":"2024-04-25T14:11:33.611831+0000","last_unstale":"2024-04-25T14:11:34.659174+0000","last_undegraded":"2024-04-25T14:11:34.659174+0000","last_fullsized":"2024-04-25T14:11:34.659174+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:17.071976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602855+0000","last_change":"2024-04-25T14:11:33.618207+0000","last_active":"2024-04-25T14:11:34.602855+0000","last_peered":"2024-04-25T14:11:34.602855+0000","last_clean":"2024-04-25T14:11:34.602855+0000","last_became_active":"2024-04-25T14:11:33.617893+0000","last_became_peered":"2024-04-25T14:11:33.617893+0000","last_unstale":"2024-04-25T14:11:34.602855+0000","last_undegraded":"2024-04-25T14:11:34.602855+0000","last_fullsized":"2024-04-25T14:11:34.602855+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:16.358014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"139'4","reported_seq":18,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600663+0000","last_change":"2024-04-25T14:11:32.591452+0000","last_active":"2024-04-25T14:11:34.600663+0000","last_peered":"2024-04-25T14:11:34.600663+0000","last_clean":"2024-04-25T14:11:34.600663+0000","last_became_active":"2024-04-25T14:11:32.591159+0000","last_became_peered":"2024-04-25T14:11:32.591159+0000","last_unstale":"2024-04-25T14:11:34.600663+0000","last_undegraded":"2024-04-25T14:11:34.600663+0000","last_fullsized":"2024-04-25T14:11:34.600663+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:16.177533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602677+0000","last_change":"2024-04-25T14:11:33.607419+0000","last_active":"2024-04-25T14:11:34.602677+0000","last_peered":"2024-04-25T14:11:34.602677+0000","last_clean":"2024-04-25T14:11:34.602677+0000","last_became_active":"2024-04-25T14:11:33.607086+0000","last_became_peered":"2024-04-25T14:11:33.607086+0000","last_unstale":"2024-04-25T14:11:34.602677+0000","last_undegraded":"2024-04-25T14:11:34.602677+0000","last_fullsized":"2024-04-25T14:11:34.602677+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:58.403145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659400+0000","last_change":"2024-04-25T14:11:32.603965+0000","last_active":"2024-04-25T14:11:34.659400+0000","last_peered":"2024-04-25T14:11:34.659400+0000","last_clean":"2024-04-25T14:11:34.659400+0000","last_became_active":"2024-04-25T14:11:32.602019+0000","last_became_peered":"2024-04-25T14:11:32.602019+0000","last_unstale":"2024-04-25T14:11:34.659400+0000","last_undegraded":"2024-04-25T14:11:34.659400+0000","last_fullsized":"2024-04-25T14:11:34.659400+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:35.215199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.660329+0000","last_change":"2024-04-25T14:11:33.600332+0000","last_active":"2024-04-25T14:11:34.660329+0000","last_peered":"2024-04-25T14:11:34.660329+0000","last_clean":"2024-04-25T14:11:34.660329+0000","last_became_active":"2024-04-25T14:11:33.599668+0000","last_became_peered":"2024-04-25T14:11:33.599668+0000","last_unstale":"2024-04-25T14:11:34.660329+0000","last_undegraded":"2024-04-25T14:11:34.660329+0000","last_fullsized":"2024-04-25T14:11:34.660329+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:57.034214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659785+0000","last_change":"2024-04-25T14:11:32.601614+0000","last_active":"2024-04-25T14:11:34.659785+0000","last_peered":"2024-04-25T14:11:34.659785+0000","last_clean":"2024-04-25T14:11:34.659785+0000","last_became_active":"2024-04-25T14:11:32.601092+0000","last_became_peered":"2024-04-25T14:11:32.601092+0000","last_unstale":"2024-04-25T14:11:34.659785+0000","last_undegraded":"2024-04-25T14:11:34.659785+0000","last_fullsized":"2024-04-25T14:11:34.659785+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:46:18.176303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659197+0000","last_change":"2024-04-25T14:11:33.615565+0000","last_active":"2024-04-25T14:11:34.659197+0000","last_peered":"2024-04-25T14:11:34.659197+0000","last_clean":"2024-04-25T14:11:34.659197+0000","last_became_active":"2024-04-25T14:11:33.611366+0000","last_became_peered":"2024-04-25T14:11:33.611366+0000","last_unstale":"2024-04-25T14:11:34.659197+0000","last_undegraded":"2024-04-25T14:11:34.659197+0000","last_fullsized":"2024-04-25T14:11:34.659197+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:56:42.715756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602949+0000","last_change":"2024-04-25T14:11:32.602711+0000","last_active":"2024-04-25T14:11:34.602949+0000","last_peered":"2024-04-25T14:11:34.602949+0000","last_clean":"2024-04-25T14:11:34.602949+0000","last_became_active":"2024-04-25T14:11:32.602368+0000","last_became_peered":"2024-04-25T14:11:32.602368+0000","last_unstale":"2024-04-25T14:11:34.602949+0000","last_undegraded":"2024-04-25T14:11:34.602949+0000","last_fullsized":"2024-04-25T14:11:34.602949+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:16.063389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.658730+0000","last_change":"2024-04-25T14:11:33.627653+0000","last_active":"2024-04-25T14:11:34.658730+0000","last_peered":"2024-04-25T14:11:34.658730+0000","last_clean":"2024-04-25T14:11:34.658730+0000","last_became_active":"2024-04-25T14:11:33.626207+0000","last_became_peered":"2024-04-25T14:11:33.626207+0000","last_unstale":"2024-04-25T14:11:34.658730+0000","last_undegraded":"2024-04-25T14:11:34.658730+0000","last_fullsized":"2024-04-25T14:11:34.658730+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:40.168428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659866+0000","last_change":"2024-04-25T14:11:32.598778+0000","last_active":"2024-04-25T14:11:34.659866+0000","last_peered":"2024-04-25T14:11:34.659866+0000","last_clean":"2024-04-25T14:11:34.659866+0000","last_became_active":"2024-04-25T14:11:32.598190+0000","last_became_peered":"2024-04-25T14:11:32.598190+0000","last_unstale":"2024-04-25T14:11:34.659866+0000","last_undegraded":"2024-04-25T14:11:34.659866+0000","last_fullsized":"2024-04-25T14:11:34.659866+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:14.883395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.660668+0000","last_change":"2024-04-25T14:11:33.619924+0000","last_active":"2024-04-25T14:11:34.660668+0000","last_peered":"2024-04-25T14:11:34.660668+0000","last_clean":"2024-04-25T14:11:34.660668+0000","last_became_active":"2024-04-25T14:11:33.619291+0000","last_became_peered":"2024-04-25T14:11:33.619291+0000","last_unstale":"2024-04-25T14:11:34.660668+0000","last_undegraded":"2024-04-25T14:11:34.660668+0000","last_fullsized":"2024-04-25T14:11:34.660668+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:46:19.184891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600184+0000","last_change":"2024-04-25T14:11:32.605097+0000","last_active":"2024-04-25T14:11:34.600184+0000","last_peered":"2024-04-25T14:11:34.600184+0000","last_clean":"2024-04-25T14:11:34.600184+0000","last_became_active":"2024-04-25T14:11:32.604770+0000","last_became_peered":"2024-04-25T14:11:32.604770+0000","last_unstale":"2024-04-25T14:11:34.600184+0000","last_undegraded":"2024-04-25T14:11:34.600184+0000","last_fullsized":"2024-04-25T14:11:34.600184+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:09.032042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602397+0000","last_change":"2024-04-25T14:11:33.614281+0000","last_active":"2024-04-25T14:11:34.602397+0000","last_peered":"2024-04-25T14:11:34.602397+0000","last_clean":"2024-04-25T14:11:34.602397+0000","last_became_active":"2024-04-25T14:11:33.613639+0000","last_became_peered":"2024-04-25T14:11:33.613639+0000","last_unstale":"2024-04-25T14:11:34.602397+0000","last_undegraded":"2024-04-25T14:11:34.602397+0000","last_fullsized":"2024-04-25T14:11:34.602397+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:07.671897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602742+0000","last_change":"2024-04-25T14:11:32.605101+0000","last_active":"2024-04-25T14:11:34.602742+0000","last_peered":"2024-04-25T14:11:34.602742+0000","last_clean":"2024-04-25T14:11:34.602742+0000","last_became_active":"2024-04-25T14:11:32.604386+0000","last_became_peered":"2024-04-25T14:11:32.604386+0000","last_unstale":"2024-04-25T14:11:34.602742+0000","last_undegraded":"2024-04-25T14:11:34.602742+0000","last_fullsized":"2024-04-25T14:11:34.602742+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:51.825452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.603015+0000","last_change":"2024-04-25T14:11:33.604679+0000","last_active":"2024-04-25T14:11:34.603015+0000","last_peered":"2024-04-25T14:11:34.603015+0000","last_clean":"2024-04-25T14:11:34.603015+0000","last_became_active":"2024-04-25T14:11:33.604366+0000","last_became_peered":"2024-04-25T14:11:33.604366+0000","last_unstale":"2024-04-25T14:11:34.603015+0000","last_undegraded":"2024-04-25T14:11:34.603015+0000","last_fullsized":"2024-04-25T14:11:34.603015+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:17.186121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.599957+0000","last_change":"2024-04-25T14:11:32.591107+0000","last_active":"2024-04-25T14:11:34.599957+0000","last_peered":"2024-04-25T14:11:34.599957+0000","last_clean":"2024-04-25T14:11:34.599957+0000","last_became_active":"2024-04-25T14:11:32.590762+0000","last_became_peered":"2024-04-25T14:11:32.590762+0000","last_unstale":"2024-04-25T14:11:34.599957+0000","last_undegraded":"2024-04-25T14:11:34.599957+0000","last_fullsized":"2024-04-25T14:11:34.599957+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:35.215199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659728+0000","last_change":"2024-04-25T14:11:33.609347+0000","last_active":"2024-04-25T14:11:34.659728+0000","last_peered":"2024-04-25T14:11:34.659728+0000","last_clean":"2024-04-25T14:11:34.659728+0000","last_became_active":"2024-04-25T14:11:33.609020+0000","last_became_peered":"2024-04-25T14:11:33.609020+0000","last_unstale":"2024-04-25T14:11:34.659728+0000","last_undegraded":"2024-04-25T14:11:34.659728+0000","last_fullsized":"2024-04-25T14:11:34.659728+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.223787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659164+0000","last_change":"2024-04-25T14:11:32.602319+0000","last_active":"2024-04-25T14:11:34.659164+0000","last_peered":"2024-04-25T14:11:34.659164+0000","last_clean":"2024-04-25T14:11:34.659164+0000","last_became_active":"2024-04-25T14:11:32.601994+0000","last_became_peered":"2024-04-25T14:11:32.601994+0000","last_unstale":"2024-04-25T14:11:34.659164+0000","last_undegraded":"2024-04-25T14:11:34.659164+0000","last_fullsized":"2024-04-25T14:11:34.659164+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:56.025626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659223+0000","last_change":"2024-04-25T14:11:33.624243+0000","last_active":"2024-04-25T14:11:34.659223+0000","last_peered":"2024-04-25T14:11:34.659223+0000","last_clean":"2024-04-25T14:11:34.659223+0000","last_became_active":"2024-04-25T14:11:33.623178+0000","last_became_peered":"2024-04-25T14:11:33.623178+0000","last_unstale":"2024-04-25T14:11:34.659223+0000","last_undegraded":"2024-04-25T14:11:34.659223+0000","last_fullsized":"2024-04-25T14:11:34.659223+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:25.536000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.603393+0000","last_change":"2024-04-25T14:11:32.599420+0000","last_active":"2024-04-25T14:11:34.603393+0000","last_peered":"2024-04-25T14:11:34.603393+0000","last_clean":"2024-04-25T14:11:34.603393+0000","last_became_active":"2024-04-25T14:11:32.599133+0000","last_became_peered":"2024-04-25T14:11:32.599133+0000","last_unstale":"2024-04-25T14:11:34.603393+0000","last_undegraded":"2024-04-25T14:11:34.603393+0000","last_fullsized":"2024-04-25T14:11:34.603393+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:26.979452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659406+0000","last_change":"2024-04-25T14:11:33.616568+0000","last_active":"2024-04-25T14:11:34.659406+0000","last_peered":"2024-04-25T14:11:34.659406+0000","last_clean":"2024-04-25T14:11:34.659406+0000","last_became_active":"2024-04-25T14:11:33.616173+0000","last_became_peered":"2024-04-25T14:11:33.616173+0000","last_unstale":"2024-04-25T14:11:34.659406+0000","last_undegraded":"2024-04-25T14:11:34.659406+0000","last_fullsized":"2024-04-25T14:11:34.659406+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:23.600455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"139'3","reported_seq":17,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659324+0000","last_change":"2024-04-25T14:11:32.596904+0000","last_active":"2024-04-25T14:11:34.659324+0000","last_peered":"2024-04-25T14:11:34.659324+0000","last_clean":"2024-04-25T14:11:34.659324+0000","last_became_active":"2024-04-25T14:11:32.595555+0000","last_became_peered":"2024-04-25T14:11:32.595555+0000","last_unstale":"2024-04-25T14:11:34.659324+0000","last_undegraded":"2024-04-25T14:11:34.659324+0000","last_fullsized":"2024-04-25T14:11:34.659324+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:26.253319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659347+0000","last_change":"2024-04-25T14:11:33.607908+0000","last_active":"2024-04-25T14:11:34.659347+0000","last_peered":"2024-04-25T14:11:34.659347+0000","last_clean":"2024-04-25T14:11:34.659347+0000","last_became_active":"2024-04-25T14:11:33.607595+0000","last_became_peered":"2024-04-25T14:11:33.607595+0000","last_unstale":"2024-04-25T14:11:34.659347+0000","last_undegraded":"2024-04-25T14:11:34.659347+0000","last_fullsized":"2024-04-25T14:11:34.659347+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:10.040630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600654+0000","last_change":"2024-04-25T14:11:32.593779+0000","last_active":"2024-04-25T14:11:34.600654+0000","last_peered":"2024-04-25T14:11:34.600654+0000","last_clean":"2024-04-25T14:11:34.600654+0000","last_became_active":"2024-04-25T14:11:32.593185+0000","last_became_peered":"2024-04-25T14:11:32.593185+0000","last_unstale":"2024-04-25T14:11:34.600654+0000","last_undegraded":"2024-04-25T14:11:34.600654+0000","last_fullsized":"2024-04-25T14:11:34.600654+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:03.151785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602557+0000","last_change":"2024-04-25T14:11:33.623899+0000","last_active":"2024-04-25T14:11:34.602557+0000","last_peered":"2024-04-25T14:11:34.602557+0000","last_clean":"2024-04-25T14:11:34.602557+0000","last_became_active":"2024-04-25T14:11:33.621421+0000","last_became_peered":"2024-04-25T14:11:33.621421+0000","last_unstale":"2024-04-25T14:11:34.602557+0000","last_undegraded":"2024-04-25T14:11:34.602557+0000","last_fullsized":"2024-04-25T14:11:34.602557+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:50.996554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.660123+0000","last_change":"2024-04-25T14:11:32.602817+0000","last_active":"2024-04-25T14:11:34.660123+0000","last_peered":"2024-04-25T14:11:34.660123+0000","last_clean":"2024-04-25T14:11:34.660123+0000","last_became_active":"2024-04-25T14:11:32.602507+0000","last_became_peered":"2024-04-25T14:11:32.602507+0000","last_unstale":"2024-04-25T14:11:34.660123+0000","last_undegraded":"2024-04-25T14:11:34.660123+0000","last_fullsized":"2024-04-25T14:11:34.660123+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:45.684366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600077+0000","last_change":"2024-04-25T14:11:33.627596+0000","last_active":"2024-04-25T14:11:34.600077+0000","last_peered":"2024-04-25T14:11:34.600077+0000","last_clean":"2024-04-25T14:11:34.600077+0000","last_became_active":"2024-04-25T14:11:33.627282+0000","last_became_peered":"2024-04-25T14:11:33.627282+0000","last_unstale":"2024-04-25T14:11:34.600077+0000","last_undegraded":"2024-04-25T14:11:34.600077+0000","last_fullsized":"2024-04-25T14:11:34.600077+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:05.122162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.660786+0000","last_change":"2024-04-25T14:11:32.604165+0000","last_active":"2024-04-25T14:11:34.660786+0000","last_peered":"2024-04-25T14:11:34.660786+0000","last_clean":"2024-04-25T14:11:34.660786+0000","last_became_active":"2024-04-25T14:11:32.603846+0000","last_became_peered":"2024-04-25T14:11:32.603846+0000","last_unstale":"2024-04-25T14:11:34.660786+0000","last_undegraded":"2024-04-25T14:11:34.660786+0000","last_fullsized":"2024-04-25T14:11:34.660786+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:26.979452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602443+0000","last_change":"2024-04-25T14:11:33.624837+0000","last_active":"2024-04-25T14:11:34.602443+0000","last_peered":"2024-04-25T14:11:34.602443+0000","last_clean":"2024-04-25T14:11:34.602443+0000","last_became_active":"2024-04-25T14:11:33.624568+0000","last_became_peered":"2024-04-25T14:11:33.624568+0000","last_unstale":"2024-04-25T14:11:34.602443+0000","last_undegraded":"2024-04-25T14:11:34.602443+0000","last_fullsized":"2024-04-25T14:11:34.602443+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:56:42.715756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"139'8","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602537+0000","last_change":"2024-04-25T14:11:32.598249+0000","last_active":"2024-04-25T14:11:34.602537+0000","last_peered":"2024-04-25T14:11:34.602537+0000","last_clean":"2024-04-25T14:11:34.602537+0000","last_became_active":"2024-04-25T14:11:32.596737+0000","last_became_peered":"2024-04-25T14:11:32.596737+0000","last_unstale":"2024-04-25T14:11:34.602537+0000","last_undegraded":"2024-04-25T14:11:34.602537+0000","last_fullsized":"2024-04-25T14:11:34.602537+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:14.883395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659934+0000","last_change":"2024-04-25T14:11:33.615242+0000","last_active":"2024-04-25T14:11:34.659934+0000","last_peered":"2024-04-25T14:11:34.659934+0000","last_clean":"2024-04-25T14:11:34.659934+0000","last_became_active":"2024-04-25T14:11:33.611339+0000","last_became_peered":"2024-04-25T14:11:33.611339+0000","last_unstale":"2024-04-25T14:11:34.659934+0000","last_undegraded":"2024-04-25T14:11:34.659934+0000","last_fullsized":"2024-04-25T14:11:34.659934+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:02.982431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659386+0000","last_change":"2024-04-25T14:11:32.598361+0000","last_active":"2024-04-25T14:11:34.659386+0000","last_peered":"2024-04-25T14:11:34.659386+0000","last_clean":"2024-04-25T14:11:34.659386+0000","last_became_active":"2024-04-25T14:11:32.597623+0000","last_became_peered":"2024-04-25T14:11:32.597623+0000","last_unstale":"2024-04-25T14:11:34.659386+0000","last_undegraded":"2024-04-25T14:11:34.659386+0000","last_fullsized":"2024-04-25T14:11:34.659386+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:42.654645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659679+0000","last_change":"2024-04-25T14:11:33.610673+0000","last_active":"2024-04-25T14:11:34.659679+0000","last_peered":"2024-04-25T14:11:34.659679+0000","last_clean":"2024-04-25T14:11:34.659679+0000","last_became_active":"2024-04-25T14:11:33.610052+0000","last_became_peered":"2024-04-25T14:11:33.610052+0000","last_unstale":"2024-04-25T14:11:34.659679+0000","last_undegraded":"2024-04-25T14:11:34.659679+0000","last_fullsized":"2024-04-25T14:11:34.659679+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:50.996554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659525+0000","last_change":"2024-04-25T14:11:32.605007+0000","last_active":"2024-04-25T14:11:34.659525+0000","last_peered":"2024-04-25T14:11:34.659525+0000","last_clean":"2024-04-25T14:11:34.659525+0000","last_became_active":"2024-04-25T14:11:32.604678+0000","last_became_peered":"2024-04-25T14:11:32.604678+0000","last_unstale":"2024-04-25T14:11:34.659525+0000","last_undegraded":"2024-04-25T14:11:34.659525+0000","last_fullsized":"2024-04-25T14:11:34.659525+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:16.177533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.603334+0000","last_change":"2024-04-25T14:11:33.595266+0000","last_active":"2024-04-25T14:11:34.603334+0000","last_peered":"2024-04-25T14:11:34.603334+0000","last_clean":"2024-04-25T14:11:34.603334+0000","last_became_active":"2024-04-25T14:11:33.594843+0000","last_became_peered":"2024-04-25T14:11:33.594843+0000","last_unstale":"2024-04-25T14:11:34.603334+0000","last_undegraded":"2024-04-25T14:11:34.603334+0000","last_fullsized":"2024-04-25T14:11:34.603334+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:59.483240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602771+0000","last_change":"2024-04-25T14:11:32.603829+0000","last_active":"2024-04-25T14:11:34.602771+0000","last_peered":"2024-04-25T14:11:34.602771+0000","last_clean":"2024-04-25T14:11:34.602771+0000","last_became_active":"2024-04-25T14:11:32.603424+0000","last_became_peered":"2024-04-25T14:11:32.603424+0000","last_unstale":"2024-04-25T14:11:34.602771+0000","last_undegraded":"2024-04-25T14:11:34.602771+0000","last_fullsized":"2024-04-25T14:11:34.602771+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:31.082017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.603309+0000","last_change":"2024-04-25T14:11:33.588115+0000","last_active":"2024-04-25T14:11:34.603309+0000","last_peered":"2024-04-25T14:11:34.603309+0000","last_clean":"2024-04-25T14:11:34.603309+0000","last_became_active":"2024-04-25T14:11:33.587768+0000","last_became_peered":"2024-04-25T14:11:33.587768+0000","last_unstale":"2024-04-25T14:11:34.603309+0000","last_undegraded":"2024-04-25T14:11:34.603309+0000","last_fullsized":"2024-04-25T14:11:34.603309+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:40.795170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.660281+0000","last_change":"2024-04-25T14:11:32.603572+0000","last_active":"2024-04-25T14:11:34.660281+0000","last_peered":"2024-04-25T14:11:34.660281+0000","last_clean":"2024-04-25T14:11:34.660281+0000","last_became_active":"2024-04-25T14:11:32.603270+0000","last_became_peered":"2024-04-25T14:11:32.603270+0000","last_unstale":"2024-04-25T14:11:34.660281+0000","last_undegraded":"2024-04-25T14:11:34.660281+0000","last_fullsized":"2024-04-25T14:11:34.660281+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:50:44.121532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.658815+0000","last_change":"2024-04-25T14:11:33.603405+0000","last_active":"2024-04-25T14:11:34.658815+0000","last_peered":"2024-04-25T14:11:34.658815+0000","last_clean":"2024-04-25T14:11:34.658815+0000","last_became_active":"2024-04-25T14:11:33.602948+0000","last_became_peered":"2024-04-25T14:11:33.602948+0000","last_unstale":"2024-04-25T14:11:34.658815+0000","last_undegraded":"2024-04-25T14:11:34.658815+0000","last_fullsized":"2024-04-25T14:11:34.658815+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:49.326225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.658698+0000","last_change":"2024-04-25T14:11:32.593507+0000","last_active":"2024-04-25T14:11:34.658698+0000","last_peered":"2024-04-25T14:11:34.658698+0000","last_clean":"2024-04-25T14:11:34.658698+0000","last_became_active":"2024-04-25T14:11:32.593173+0000","last_became_peered":"2024-04-25T14:11:32.593173+0000","last_unstale":"2024-04-25T14:11:34.658698+0000","last_undegraded":"2024-04-25T14:11:34.658698+0000","last_fullsized":"2024-04-25T14:11:34.658698+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:01.673093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602661+0000","last_change":"2024-04-25T14:11:33.612339+0000","last_active":"2024-04-25T14:11:34.602661+0000","last_peered":"2024-04-25T14:11:34.602661+0000","last_clean":"2024-04-25T14:11:34.602661+0000","last_became_active":"2024-04-25T14:11:33.611897+0000","last_became_peered":"2024-04-25T14:11:33.611897+0000","last_unstale":"2024-04-25T14:11:34.602661+0000","last_undegraded":"2024-04-25T14:11:34.602661+0000","last_fullsized":"2024-04-25T14:11:34.602661+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:58:03.795240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659563+0000","last_change":"2024-04-25T14:11:32.606046+0000","last_active":"2024-04-25T14:11:34.659563+0000","last_peered":"2024-04-25T14:11:34.659563+0000","last_clean":"2024-04-25T14:11:34.659563+0000","last_became_active":"2024-04-25T14:11:32.605755+0000","last_became_peered":"2024-04-25T14:11:32.605755+0000","last_unstale":"2024-04-25T14:11:34.659563+0000","last_undegraded":"2024-04-25T14:11:34.659563+0000","last_fullsized":"2024-04-25T14:11:34.659563+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:00.011427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.658997+0000","last_change":"2024-04-25T14:11:33.625795+0000","last_active":"2024-04-25T14:11:34.658997+0000","last_peered":"2024-04-25T14:11:34.658997+0000","last_clean":"2024-04-25T14:11:34.658997+0000","last_became_active":"2024-04-25T14:11:33.625497+0000","last_became_peered":"2024-04-25T14:11:33.625497+0000","last_unstale":"2024-04-25T14:11:34.658997+0000","last_undegraded":"2024-04-25T14:11:34.658997+0000","last_fullsized":"2024-04-25T14:11:34.658997+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:07.671897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.658486+0000","last_change":"2024-04-25T14:11:32.591966+0000","last_active":"2024-04-25T14:11:34.658486+0000","last_peered":"2024-04-25T14:11:34.658486+0000","last_clean":"2024-04-25T14:11:34.658486+0000","last_became_active":"2024-04-25T14:11:32.591633+0000","last_became_peered":"2024-04-25T14:11:32.591633+0000","last_unstale":"2024-04-25T14:11:34.658486+0000","last_undegraded":"2024-04-25T14:11:34.658486+0000","last_fullsized":"2024-04-25T14:11:34.658486+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:57.758937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602912+0000","last_change":"2024-04-25T14:11:33.611220+0000","last_active":"2024-04-25T14:11:34.602912+0000","last_peered":"2024-04-25T14:11:34.602912+0000","last_clean":"2024-04-25T14:11:34.602912+0000","last_became_active":"2024-04-25T14:11:33.610910+0000","last_became_peered":"2024-04-25T14:11:33.610910+0000","last_unstale":"2024-04-25T14:11:34.602912+0000","last_undegraded":"2024-04-25T14:11:34.602912+0000","last_fullsized":"2024-04-25T14:11:34.602912+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:24:55.518082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659655+0000","last_change":"2024-04-25T14:11:32.590551+0000","last_active":"2024-04-25T14:11:34.659655+0000","last_peered":"2024-04-25T14:11:34.659655+0000","last_clean":"2024-04-25T14:11:34.659655+0000","last_became_active":"2024-04-25T14:11:32.590233+0000","last_became_peered":"2024-04-25T14:11:32.590233+0000","last_unstale":"2024-04-25T14:11:34.659655+0000","last_undegraded":"2024-04-25T14:11:34.659655+0000","last_fullsized":"2024-04-25T14:11:34.659655+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:57.130911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600731+0000","last_change":"2024-04-25T14:11:33.596960+0000","last_active":"2024-04-25T14:11:34.600731+0000","last_peered":"2024-04-25T14:11:34.600731+0000","last_clean":"2024-04-25T14:11:34.600731+0000","last_became_active":"2024-04-25T14:11:33.596123+0000","last_became_peered":"2024-04-25T14:11:33.596123+0000","last_unstale":"2024-04-25T14:11:34.600731+0000","last_undegraded":"2024-04-25T14:11:34.600731+0000","last_fullsized":"2024-04-25T14:11:34.600731+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:31.088654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.660670+0000","last_change":"2024-04-25T14:11:32.602907+0000","last_active":"2024-04-25T14:11:34.660670+0000","last_peered":"2024-04-25T14:11:34.660670+0000","last_clean":"2024-04-25T14:11:34.660670+0000","last_became_active":"2024-04-25T14:11:32.602600+0000","last_became_peered":"2024-04-25T14:11:32.602600+0000","last_unstale":"2024-04-25T14:11:34.660670+0000","last_undegraded":"2024-04-25T14:11:34.660670+0000","last_fullsized":"2024-04-25T14:11:34.660670+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:21.954678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.660303+0000","last_change":"2024-04-25T14:11:33.598878+0000","last_active":"2024-04-25T14:11:34.660303+0000","last_peered":"2024-04-25T14:11:34.660303+0000","last_clean":"2024-04-25T14:11:34.660303+0000","last_became_active":"2024-04-25T14:11:33.596559+0000","last_became_peered":"2024-04-25T14:11:33.596559+0000","last_unstale":"2024-04-25T14:11:34.660303+0000","last_undegraded":"2024-04-25T14:11:34.660303+0000","last_fullsized":"2024-04-25T14:11:34.660303+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:36.223787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"140'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.603498+0000","last_change":"2024-04-25T14:11:32.595961+0000","last_active":"2024-04-25T14:11:34.603498+0000","last_peered":"2024-04-25T14:11:34.603498+0000","last_clean":"2024-04-25T14:11:34.603498+0000","last_became_active":"2024-04-25T14:11:32.595641+0000","last_became_peered":"2024-04-25T14:11:32.595641+0000","last_unstale":"2024-04-25T14:11:34.603498+0000","last_undegraded":"2024-04-25T14:11:34.603498+0000","last_fullsized":"2024-04-25T14:11:34.603498+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:50:44.121532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.603310+0000","last_change":"2024-04-25T14:11:33.603158+0000","last_active":"2024-04-25T14:11:34.603310+0000","last_peered":"2024-04-25T14:11:34.603310+0000","last_clean":"2024-04-25T14:11:34.603310+0000","last_became_active":"2024-04-25T14:11:33.602866+0000","last_became_peered":"2024-04-25T14:11:33.602866+0000","last_unstale":"2024-04-25T14:11:34.603310+0000","last_undegraded":"2024-04-25T14:11:34.603310+0000","last_fullsized":"2024-04-25T14:11:34.603310+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:11.244407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602809+0000","last_change":"2024-04-25T14:11:32.603997+0000","last_active":"2024-04-25T14:11:34.602809+0000","last_peered":"2024-04-25T14:11:34.602809+0000","last_clean":"2024-04-25T14:11:34.602809+0000","last_became_active":"2024-04-25T14:11:32.603630+0000","last_became_peered":"2024-04-25T14:11:32.603630+0000","last_unstale":"2024-04-25T14:11:34.602809+0000","last_undegraded":"2024-04-25T14:11:34.602809+0000","last_fullsized":"2024-04-25T14:11:34.602809+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:44.182089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600567+0000","last_change":"2024-04-25T14:11:33.608436+0000","last_active":"2024-04-25T14:11:34.600567+0000","last_peered":"2024-04-25T14:11:34.600567+0000","last_clean":"2024-04-25T14:11:34.600567+0000","last_became_active":"2024-04-25T14:11:33.608046+0000","last_became_peered":"2024-04-25T14:11:33.608046+0000","last_unstale":"2024-04-25T14:11:34.600567+0000","last_undegraded":"2024-04-25T14:11:34.600567+0000","last_fullsized":"2024-04-25T14:11:34.600567+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:29.569454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.600146+0000","last_change":"2024-04-25T14:11:32.590839+0000","last_active":"2024-04-25T14:11:34.600146+0000","last_peered":"2024-04-25T14:11:34.600146+0000","last_clean":"2024-04-25T14:11:34.600146+0000","last_became_active":"2024-04-25T14:11:32.590349+0000","last_became_peered":"2024-04-25T14:11:32.590349+0000","last_unstale":"2024-04-25T14:11:34.600146+0000","last_undegraded":"2024-04-25T14:11:34.600146+0000","last_fullsized":"2024-04-25T14:11:34.600146+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:56.025626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659543+0000","last_change":"2024-04-25T14:11:33.602353+0000","last_active":"2024-04-25T14:11:34.659543+0000","last_peered":"2024-04-25T14:11:34.659543+0000","last_clean":"2024-04-25T14:11:34.659543+0000","last_became_active":"2024-04-25T14:11:33.601990+0000","last_became_peered":"2024-04-25T14:11:33.601990+0000","last_unstale":"2024-04-25T14:11:34.659543+0000","last_undegraded":"2024-04-25T14:11:34.659543+0000","last_fullsized":"2024-04-25T14:11:34.659543+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:04.160373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.603084+0000","last_change":"2024-04-25T14:11:32.592951+0000","last_active":"2024-04-25T14:11:34.603084+0000","last_peered":"2024-04-25T14:11:34.603084+0000","last_clean":"2024-04-25T14:11:34.603084+0000","last_became_active":"2024-04-25T14:11:32.592568+0000","last_became_peered":"2024-04-25T14:11:32.592568+0000","last_unstale":"2024-04-25T14:11:34.603084+0000","last_undegraded":"2024-04-25T14:11:34.603084+0000","last_fullsized":"2024-04-25T14:11:34.603084+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:45.684366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.660753+0000","last_change":"2024-04-25T14:11:33.595107+0000","last_active":"2024-04-25T14:11:34.660753+0000","last_peered":"2024-04-25T14:11:34.660753+0000","last_clean":"2024-04-25T14:11:34.660753+0000","last_became_active":"2024-04-25T14:11:33.594810+0000","last_became_peered":"2024-04-25T14:11:33.594810+0000","last_unstale":"2024-04-25T14:11:34.660753+0000","last_undegraded":"2024-04-25T14:11:34.660753+0000","last_fullsized":"2024-04-25T14:11:34.660753+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:29.349474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.658920+0000","last_change":"2024-04-25T14:11:32.598702+0000","last_active":"2024-04-25T14:11:34.658920+0000","last_peered":"2024-04-25T14:11:34.658920+0000","last_clean":"2024-04-25T14:11:34.658920+0000","last_became_active":"2024-04-25T14:11:32.598372+0000","last_became_peered":"2024-04-25T14:11:32.598372+0000","last_unstale":"2024-04-25T14:11:34.658920+0000","last_undegraded":"2024-04-25T14:11:34.658920+0000","last_fullsized":"2024-04-25T14:11:34.658920+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:58.105185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602593+0000","last_change":"2024-04-25T14:11:33.619891+0000","last_active":"2024-04-25T14:11:34.602593+0000","last_peered":"2024-04-25T14:11:34.602593+0000","last_clean":"2024-04-25T14:11:34.602593+0000","last_became_active":"2024-04-25T14:11:33.619416+0000","last_became_peered":"2024-04-25T14:11:33.619416+0000","last_unstale":"2024-04-25T14:11:34.602593+0000","last_undegraded":"2024-04-25T14:11:34.602593+0000","last_fullsized":"2024-04-25T14:11:34.602593+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:02.982431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.602910+0000","last_change":"2024-04-25T14:11:32.596091+0000","last_active":"2024-04-25T14:11:34.602910+0000","last_peered":"2024-04-25T14:11:34.602910+0000","last_clean":"2024-04-25T14:11:34.602910+0000","last_became_active":"2024-04-25T14:11:32.595711+0000","last_became_peered":"2024-04-25T14:11:32.595711+0000","last_unstale":"2024-04-25T14:11:34.602910+0000","last_undegraded":"2024-04-25T14:11:34.602910+0000","last_fullsized":"2024-04-25T14:11:34.602910+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:17.575105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659319+0000","last_change":"2024-04-25T14:11:33.609197+0000","last_active":"2024-04-25T14:11:34.659319+0000","last_peered":"2024-04-25T14:11:34.659319+0000","last_clean":"2024-04-25T14:11:34.659319+0000","last_became_active":"2024-04-25T14:11:33.608865+0000","last_became_peered":"2024-04-25T14:11:33.608865+0000","last_unstale":"2024-04-25T14:11:34.659319+0000","last_undegraded":"2024-04-25T14:11:34.659319+0000","last_fullsized":"2024-04-25T14:11:34.659319+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:45.190677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.658970+0000","last_change":"2024-04-25T14:11:32.599842+0000","last_active":"2024-04-25T14:11:34.658970+0000","last_peered":"2024-04-25T14:11:34.658970+0000","last_clean":"2024-04-25T14:11:34.658970+0000","last_became_active":"2024-04-25T14:11:32.599226+0000","last_became_peered":"2024-04-25T14:11:32.599226+0000","last_unstale":"2024-04-25T14:11:34.658970+0000","last_undegraded":"2024-04-25T14:11:34.658970+0000","last_fullsized":"2024-04-25T14:11:34.658970+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:03.151785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.660452+0000","last_change":"2024-04-25T14:11:33.619555+0000","last_active":"2024-04-25T14:11:34.660452+0000","last_peered":"2024-04-25T14:11:34.660452+0000","last_clean":"2024-04-25T14:11:34.660452+0000","last_became_active":"2024-04-25T14:11:33.618630+0000","last_became_peered":"2024-04-25T14:11:33.618630+0000","last_unstale":"2024-04-25T14:11:34.660452+0000","last_undegraded":"2024-04-25T14:11:34.660452+0000","last_fullsized":"2024-04-25T14:11:34.660452+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:32.567346+0000","last_clean_scrub_stamp":"2024-04-25T14:11:32.567346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:46:19.184891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"139'6","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.658859+0000","last_change":"2024-04-25T14:11:32.593191+0000","last_active":"2024-04-25T14:11:34.658859+0000","last_peered":"2024-04-25T14:11:34.658859+0000","last_clean":"2024-04-25T14:11:34.658859+0000","last_became_active":"2024-04-25T14:11:32.592628+0000","last_became_peered":"2024-04-25T14:11:32.592628+0000","last_unstale":"2024-04-25T14:11:34.658859+0000","last_undegraded":"2024-04-25T14:11:34.658859+0000","last_fullsized":"2024-04-25T14:11:34.658859+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:11:31.558758+0000","last_clean_scrub_stamp":"2024-04-25T14:11:31.558758+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:41.184218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"129'3277","reported_seq":3754,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T14:11:34.659103+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:11:34.659103+0000","last_peered":"2024-04-25T14:11:34.659103+0000","last_clean":"2024-04-25T14:11:34.659103+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:11:34.659103+0000","last_undegraded":"2024-04-25T14:11:34.659103+0000","last_fullsized":"2024-04-25T14:11:34.659103+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":3277,"log_dups_size":0,"ondisk_log_size":3277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":717,"num_read_kb":1927,"num_write":223,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":717,"num_read_kb":1927,"num_write":223,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3277,"ondisk_log_size":3277,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574940,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29320,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342520,"statfs":{"total":96636764160,"available":96606740480,"internally_reserved":0,"allocated":2355200,"data_stored":1470546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574941,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32900,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338940,"statfs":{"total":96636764160,"available":96603074560,"internally_reserved":0,"allocated":1761280,"data_stored":880226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574942,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32896,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338944,"statfs":{"total":96636764160,"available":96603078656,"internally_reserved":0,"allocated":1757184,"data_stored":880156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574941,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32908,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338932,"statfs":{"total":96636764160,"available":96603066368,"internally_reserved":0,"allocated":1769472,"data_stored":881768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574941,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1765376,"data_stored":880772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574940,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1765376,"data_stored":880268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32900,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338940,"statfs":{"total":96636764160,"available":96603074560,"internally_reserved":0,"allocated":1761280,"data_stored":880226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574941,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29344,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342496,"statfs":{"total":96636764160,"available":96606715904,"internally_reserved":0,"allocated":2375680,"data_stored":1472842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,1,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:11:42.664 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T14:11:42.664 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T14:11:42.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.861+0000 7f39bf200640 1 Processor -- start 2024-04-25T14:11:42.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.861+0000 7f39bf200640 1 -- start start 2024-04-25T14:11:42.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.865+0000 7f39bf200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39b80d2c80 0x7f39b815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:42.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.865+0000 7f39bf200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f39b81616e0 0x7f39b8161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:42.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.865+0000 7f39bf200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39b80cd8d0 con 0x7f39b8070e20 2024-04-25T14:11:42.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.865+0000 7f39bf200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f39b80ce720 con 0x7f39b81616e0 2024-04-25T14:11:42.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.865+0000 7f39bf200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f39b80ce970 con 0x7f39b80d2c80 2024-04-25T14:11:42.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.865+0000 7f39bd200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f39b81616e0 0x7f39b8161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:42.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.865+0000 7f39bd200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f39b81616e0 0x7f39b8161af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52948/0 (socket says 172.21.15.86:52948) 2024-04-25T14:11:42.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.865+0000 7f39bd200640 1 -- 172.21.15.86:0/1334164441 learned_addr learned my addr 172.21.15.86:0/1334164441 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:42.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.865+0000 7f39b7e00640 1 --2- 172.21.15.86:0/1334164441 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39b80d2c80 0x7f39b815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:42.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.865+0000 7f39bd200640 1 -- 172.21.15.86:0/1334164441 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39b80d2c80 msgr2=0x7f39b815dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:42.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.865+0000 7f39bd200640 1 --2- 172.21.15.86:0/1334164441 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39b80d2c80 0x7f39b815dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:42.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.865+0000 7f39bd200640 1 -- 172.21.15.86:0/1334164441 >> v1:172.21.15.155:6789/0 conn(0x7f39b8070e20 legacy=0x7f39b8071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:42.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.865+0000 7f39bd200640 1 -- 172.21.15.86:0/1334164441 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b80d1be0 con 0x7f39b81616e0 2024-04-25T14:11:42.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.869+0000 7f39bd200640 1 --2- 172.21.15.86:0/1334164441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39b81616e0 0x7f39b8161af0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f39b0009870 tx=0x7f39b001ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c55f556ad7ffda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:42.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.869+0000 7f39b6a00640 1 -- 172.21.15.86:0/1334164441 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b00b5070 con 0x7f39b81616e0 2024-04-25T14:11:42.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.869+0000 7f39b6a00640 1 -- 172.21.15.86:0/1334164441 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39b00a8a50 con 0x7f39b81616e0 2024-04-25T14:11:42.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.869+0000 7f39b6a00640 1 -- 172.21.15.86:0/1334164441 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b00a8e20 con 0x7f39b81616e0 2024-04-25T14:11:42.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.869+0000 7f39bf200640 1 -- 172.21.15.86:0/1334164441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39b81616e0 msgr2=0x7f39b8161af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:42.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.869+0000 7f39bf200640 1 --2- 172.21.15.86:0/1334164441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39b81616e0 0x7f39b8161af0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f39b0009870 tx=0x7f39b001ed50 comp rx=0 tx=0).stop 2024-04-25T14:11:42.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.869+0000 7f39bf200640 1 -- 172.21.15.86:0/1334164441 shutdown_connections 2024-04-25T14:11:42.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.869+0000 7f39bf200640 1 --2- 172.21.15.86:0/1334164441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39b81616e0 0x7f39b8161af0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:42.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.869+0000 7f39bf200640 1 --2- 172.21.15.86:0/1334164441 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39b80d2c80 0x7f39b815dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:42.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.869+0000 7f39bf200640 1 -- 172.21.15.86:0/1334164441 >> 172.21.15.86:0/1334164441 conn(0x7f39b80cffc0 msgr2=0x7f39b80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:42.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.869+0000 7f39bf200640 1 -- 172.21.15.86:0/1334164441 shutdown_connections 2024-04-25T14:11:42.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.869+0000 7f39bf200640 1 -- 172.21.15.86:0/1334164441 wait complete. 2024-04-25T14:11:42.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.869+0000 7f39bf200640 1 Processor -- start 2024-04-25T14:11:42.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.869+0000 7f39bf200640 1 -- start start 2024-04-25T14:11:42.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.873+0000 7f39bf200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39b8070e20 0x7f39b815aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:42.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.873+0000 7f39bf200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39b80d2c80 0x7f39b8156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:42.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.873+0000 7f39bf200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39b81616e0 0x7f39b8160400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:42.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.873+0000 7f39bf200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39b80d2480 con 0x7f39b80d2c80 2024-04-25T14:11:42.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.873+0000 7f39bf200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f39b80d21a0 con 0x7f39b81616e0 2024-04-25T14:11:42.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.873+0000 7f39bf200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f39b80d29a0 con 0x7f39b8070e20 2024-04-25T14:11:42.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.873+0000 7f39b7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39b80d2c80 0x7f39b8156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:42.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.873+0000 7f39b7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39b80d2c80 0x7f39b8156900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52950/0 (socket says 172.21.15.86:52950) 2024-04-25T14:11:42.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.873+0000 7f39b7400640 1 -- 172.21.15.86:0/1652122449 learned_addr learned my addr 172.21.15.86:0/1652122449 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:42.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.873+0000 7f39b7400640 1 -- 172.21.15.86:0/1652122449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39b8070e20 msgr2=0x7f39b815aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:42.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.873+0000 7f39b7400640 1 --2- 172.21.15.86:0/1652122449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39b8070e20 0x7f39b815aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:42.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.873+0000 7f39b7400640 1 -- 172.21.15.86:0/1652122449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39b81616e0 msgr2=0x7f39b8160400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:42.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.873+0000 7f39b7400640 1 --2- 172.21.15.86:0/1652122449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39b81616e0 0x7f39b8160400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:42.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.873+0000 7f39b7400640 1 -- 172.21.15.86:0/1652122449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b806d8b0 con 0x7f39b80d2c80 2024-04-25T14:11:42.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.873+0000 7f39b7400640 1 --2- 172.21.15.86:0/1652122449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39b80d2c80 0x7f39b8156900 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f39a400c3f0 tx=0x7f39a400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:42.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.873+0000 7f39afe00640 1 -- 172.21.15.86:0/1652122449 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a40052d0 con 0x7f39b80d2c80 2024-04-25T14:11:42.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.873+0000 7f39afe00640 1 -- 172.21.15.86:0/1652122449 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39a40054b0 con 0x7f39b80d2c80 2024-04-25T14:11:42.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.873+0000 7f39bf200640 1 -- 172.21.15.86:0/1652122449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39b806db90 con 0x7f39b80d2c80 2024-04-25T14:11:42.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.873+0000 7f39afe00640 1 -- 172.21.15.86:0/1652122449 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a4006a70 con 0x7f39b80d2c80 2024-04-25T14:11:42.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.873+0000 7f39bf200640 1 -- 172.21.15.86:0/1652122449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39b806e150 con 0x7f39b80d2c80 2024-04-25T14:11:42.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.873+0000 7f39bf200640 1 -- 172.21.15.86:0/1652122449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39b80720e0 con 0x7f39b80d2c80 2024-04-25T14:11:42.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.877+0000 7f39afe00640 1 -- 172.21.15.86:0/1652122449 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7f39a4006c10 con 0x7f39b80d2c80 2024-04-25T14:11:42.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.877+0000 7f39afe00640 1 --2- 172.21.15.86:0/1652122449 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f399806f520 0x7f3998071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:42.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.877+0000 7f39afe00640 1 -- 172.21.15.86:0/1652122449 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f39a408eab0 con 0x7f39b80d2c80 2024-04-25T14:11:42.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.877+0000 7f39b7e00640 1 --2- 172.21.15.86:0/1652122449 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f399806f520 0x7f3998071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:42.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.877+0000 7f39b7e00640 1 --2- 172.21.15.86:0/1652122449 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f399806f520 0x7f3998071a30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f39a80070b0 tx=0x7f39a8007040 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:42.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:42.881+0000 7f39afe00640 1 -- 172.21.15.86:0/1652122449 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f39a405b660 con 0x7f39b80d2c80 2024-04-25T14:11:43.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.093+0000 7f39bf200640 1 -- 172.21.15.86:0/1652122449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f39b8076bf0 con 0x7f39b80d2c80 2024-04-25T14:11:43.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.097+0000 7f39afe00640 1 -- 172.21.15.86:0/1652122449 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v30) ==== 190+0+147 (secure 0 0 0) 0x7f39a405af60 con 0x7f39b80d2c80 2024-04-25T14:11:43.092 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-04-25T14:11:43.093 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCLYipm5u4xHRAACDFe3Eb4KLrsdLj4eFe7mA== 2024-04-25T14:11:43.093 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-04-25T14:11:43.093 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-04-25T14:11:43.093 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T14:11:43.093 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.0 2024-04-25T14:11:43.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.097+0000 7f39bf200640 1 -- 172.21.15.86:0/1652122449 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f399806f520 msgr2=0x7f3998071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:43.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.097+0000 7f39bf200640 1 --2- 172.21.15.86:0/1652122449 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f399806f520 0x7f3998071a30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f39a80070b0 tx=0x7f39a8007040 comp rx=0 tx=0).stop 2024-04-25T14:11:43.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.097+0000 7f39bf200640 1 -- 172.21.15.86:0/1652122449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39b80d2c80 msgr2=0x7f39b8156900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:43.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.097+0000 7f39bf200640 1 --2- 172.21.15.86:0/1652122449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39b80d2c80 0x7f39b8156900 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f39a400c3f0 tx=0x7f39a400c7f0 comp rx=0 tx=0).stop 2024-04-25T14:11:43.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.097+0000 7f39bf200640 1 -- 172.21.15.86:0/1652122449 shutdown_connections 2024-04-25T14:11:43.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.097+0000 7f39bf200640 1 --2- 172.21.15.86:0/1652122449 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f399806f520 0x7f3998071a30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:43.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.097+0000 7f39bf200640 1 --2- 172.21.15.86:0/1652122449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39b81616e0 0x7f39b8160400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:43.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.097+0000 7f39bf200640 1 --2- 172.21.15.86:0/1652122449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39b80d2c80 0x7f39b8156900 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:43.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.097+0000 7f39bf200640 1 --2- 172.21.15.86:0/1652122449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39b8070e20 0x7f39b815aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:43.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.097+0000 7f39bf200640 1 -- 172.21.15.86:0/1652122449 >> 172.21.15.86:0/1652122449 conn(0x7f39b80cffc0 msgr2=0x7f39b80cdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:43.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.097+0000 7f39bf200640 1 -- 172.21.15.86:0/1652122449 shutdown_connections 2024-04-25T14:11:43.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.097+0000 7f39bf200640 1 -- 172.21.15.86:0/1652122449 wait complete. 2024-04-25T14:11:43.115 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:11:43.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.265+0000 7f0fe6e00640 1 Processor -- start 2024-04-25T14:11:43.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.265+0000 7f0fe6e00640 1 -- start start 2024-04-25T14:11:43.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.265+0000 7f0fe6e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0fe0071530 0x7f0fe015dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:43.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.265+0000 7f0fe6e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fe00cd8d0 con 0x7f0fe00d2c80 2024-04-25T14:11:43.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.265+0000 7f0fe6e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fe00ce970 con 0x7f0fe01616e0 2024-04-25T14:11:43.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.265+0000 7f0fe6e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0fe00ce720 con 0x7f0fe0071530 2024-04-25T14:11:43.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.265+0000 7f0fdfe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0fe00d2c80 0x7f0fe0070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44034/0 (socket says 172.21.15.86:44034) 2024-04-25T14:11:43.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.265+0000 7f0fdfe00640 1 -- 172.21.15.86:0/3140401829 learned_addr learned my addr 172.21.15.86:0/3140401829 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:43.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.265+0000 7f0fd7400640 1 --2- 172.21.15.86:0/3140401829 >> v2:172.21.15.155:3300/0 conn(0x7f0fe0071530 0x7f0fe015dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:43.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.265+0000 7f0fdf400640 1 -- 172.21.15.86:0/3140401829 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2781637421 0 0) 0x7f0fe00cd8d0 con 0x7f0fe00d2c80 2024-04-25T14:11:43.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.265+0000 7f0fdf400640 1 -- 172.21.15.86:0/3140401829 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0fc4003530 con 0x7f0fe00d2c80 2024-04-25T14:11:43.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.265+0000 7f0fdf400640 1 -- 172.21.15.86:0/3140401829 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3574739913 0 0) 0x7f0fc4003530 con 0x7f0fe00d2c80 2024-04-25T14:11:43.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.269+0000 7f0fdf400640 1 -- 172.21.15.86:0/3140401829 >> v2:172.21.15.155:3300/0 conn(0x7f0fe0071530 msgr2=0x7f0fe015dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:43.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.269+0000 7f0fdf400640 1 --2- 172.21.15.86:0/3140401829 >> v2:172.21.15.155:3300/0 conn(0x7f0fe0071530 0x7f0fe015dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:43.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.269+0000 7f0fdf400640 1 -- 172.21.15.86:0/3140401829 >> v1:172.21.15.155:6789/0 conn(0x7f0fe01616e0 legacy=0x7f0fe0161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:43.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.269+0000 7f0fdf400640 1 -- 172.21.15.86:0/3140401829 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe00d1be0 con 0x7f0fe00d2c80 2024-04-25T14:11:43.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.269+0000 7f0fdf400640 1 -- 172.21.15.86:0/3140401829 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f0fd0001680 con 0x7f0fe00d2c80 2024-04-25T14:11:43.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.269+0000 7f0fdf400640 1 -- 172.21.15.86:0/3140401829 >> v1:172.21.15.86:6789/0 conn(0x7f0fe00d2c80 legacy=0x7f0fe0070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:43.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.269+0000 7f0fdf400640 1 --2- 172.21.15.86:0/3140401829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fc4003a30 0x7f0fc4003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:43.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.269+0000 7f0fdf400640 1 --2- 172.21.15.86:0/3140401829 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fc4004810 0x7f0fc4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:43.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.269+0000 7f0fdf400640 1 --2- 172.21.15.86:0/3140401829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc4005230 0x7f0fc400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:43.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.269+0000 7f0fdf400640 1 -- 172.21.15.86:0/3140401829 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe00ce720 con 0x7f0fc4005230 2024-04-25T14:11:43.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.269+0000 7f0fdf400640 1 -- 172.21.15.86:0/3140401829 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc400c1b0 con 0x7f0fc4003a30 2024-04-25T14:11:43.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.269+0000 7f0fdf400640 1 -- 172.21.15.86:0/3140401829 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0fc400c3b0 con 0x7f0fc4004810 2024-04-25T14:11:43.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.269+0000 7f0fe4e00640 1 --2- 172.21.15.86:0/3140401829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc4005230 0x7f0fc400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:43.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.269+0000 7f0fdfe00640 1 --2- 172.21.15.86:0/3140401829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fc4003a30 0x7f0fc4003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:43.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.269+0000 7f0fd7400640 1 --2- 172.21.15.86:0/3140401829 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fc4004810 0x7f0fc4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:43.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.269+0000 7f0fe4e00640 1 -- 172.21.15.86:0/3140401829 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fc4004810 msgr2=0x7f0fc4004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:43.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.269+0000 7f0fe4e00640 1 --2- 172.21.15.86:0/3140401829 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fc4004810 0x7f0fc4004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:43.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.269+0000 7f0fe4e00640 1 -- 172.21.15.86:0/3140401829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fc4003a30 msgr2=0x7f0fc4003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:43.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.269+0000 7f0fe4e00640 1 --2- 172.21.15.86:0/3140401829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fc4003a30 0x7f0fc4003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:43.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.269+0000 7f0fe4e00640 1 -- 172.21.15.86:0/3140401829 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc4003530 con 0x7f0fc4005230 2024-04-25T14:11:43.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.269+0000 7f0fe4e00640 1 -- 172.21.15.86:0/3140401829 reap_dead start 2024-04-25T14:11:43.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.269+0000 7f0fe4e00640 1 --2- 172.21.15.86:0/3140401829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc4005230 0x7f0fc400b9d0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8001e40 tx=0x7f0fd801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb66419b7f456b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:43.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.273+0000 7f0fdf400640 1 -- 172.21.15.86:0/3140401829 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8001a90 con 0x7f0fc4005230 2024-04-25T14:11:43.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.273+0000 7f0fdf400640 1 -- 172.21.15.86:0/3140401829 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fd8003b80 con 0x7f0fc4005230 2024-04-25T14:11:43.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.273+0000 7f0fdf400640 1 -- 172.21.15.86:0/3140401829 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8005020 con 0x7f0fc4005230 2024-04-25T14:11:43.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.273+0000 7f0fe6e00640 1 -- 172.21.15.86:0/3140401829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc4005230 msgr2=0x7f0fc400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:43.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.273+0000 7f0fe6e00640 1 --2- 172.21.15.86:0/3140401829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc4005230 0x7f0fc400b9d0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8001e40 tx=0x7f0fd801c760 comp rx=0 tx=0).stop 2024-04-25T14:11:43.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.273+0000 7f0fe6e00640 1 -- 172.21.15.86:0/3140401829 shutdown_connections 2024-04-25T14:11:43.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.273+0000 7f0fe6e00640 1 --2- 172.21.15.86:0/3140401829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc4005230 0x7f0fc400b9d0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:43.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.273+0000 7f0fe6e00640 1 -- 172.21.15.86:0/3140401829 >> 172.21.15.86:0/3140401829 conn(0x7f0fe00cffc0 msgr2=0x7f0fe00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:43.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.273+0000 7f0fe6e00640 1 -- 172.21.15.86:0/3140401829 shutdown_connections 2024-04-25T14:11:43.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.273+0000 7f0fe6e00640 1 -- 172.21.15.86:0/3140401829 wait complete. 2024-04-25T14:11:43.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.273+0000 7f0fe6e00640 1 Processor -- start 2024-04-25T14:11:43.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.273+0000 7f0fe6e00640 1 -- start start 2024-04-25T14:11:43.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.277+0000 7f0fe6e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc4005230 0x7f0fe0153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:43.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.277+0000 7f0fe6e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fe00b4690 0x7f0fe00b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:43.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.277+0000 7f0fe6e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe00b5400 0x7f0fe0154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:43.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.277+0000 7f0fe6e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe00d2480 con 0x7f0fc4005230 2024-04-25T14:11:43.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.277+0000 7f0fe6e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe00d21a0 con 0x7f0fe00b4690 2024-04-25T14:11:43.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.277+0000 7f0fe6e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0fe00d29a0 con 0x7f0fe00b5400 2024-04-25T14:11:43.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.277+0000 7f0fdfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc4005230 0x7f0fe0153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:43.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.277+0000 7f0fdfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc4005230 0x7f0fe0153cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52962/0 (socket says 172.21.15.86:52962) 2024-04-25T14:11:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.277+0000 7f0fdfe00640 1 -- 172.21.15.86:0/733821198 learned_addr learned my addr 172.21.15.86:0/733821198 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.277+0000 7f0fd7400640 1 --2- 172.21.15.86:0/733821198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fe00b4690 0x7f0fe00b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.277+0000 7f0fe4e00640 1 --2- 172.21.15.86:0/733821198 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe00b5400 0x7f0fe0154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.277+0000 7f0fdfe00640 1 -- 172.21.15.86:0/733821198 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe00b5400 msgr2=0x7f0fe0154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.277+0000 7f0fdfe00640 1 --2- 172.21.15.86:0/733821198 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe00b5400 0x7f0fe0154550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.277+0000 7f0fdfe00640 1 -- 172.21.15.86:0/733821198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fe00b4690 msgr2=0x7f0fe00b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.277+0000 7f0fdfe00640 1 --2- 172.21.15.86:0/733821198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fe00b4690 0x7f0fe00b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.277+0000 7f0fdfe00640 1 -- 172.21.15.86:0/733821198 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe00b5810 con 0x7f0fc4005230 2024-04-25T14:11:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.277+0000 7f0fd7400640 1 --2- 172.21.15.86:0/733821198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fe00b4690 0x7f0fe00b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.277+0000 7f0fe4e00640 1 --2- 172.21.15.86:0/733821198 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe00b5400 0x7f0fe0154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.277+0000 7f0fdfe00640 1 --2- 172.21.15.86:0/733821198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc4005230 0x7f0fe0153cd0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0004f00 tx=0x7f0fd0007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.277+0000 7f0fdcc00640 1 -- 172.21.15.86:0/733821198 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd000f750 con 0x7f0fc4005230 2024-04-25T14:11:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.277+0000 7f0fdcc00640 1 -- 172.21.15.86:0/733821198 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fd0018070 con 0x7f0fc4005230 2024-04-25T14:11:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.277+0000 7f0fdcc00640 1 -- 172.21.15.86:0/733821198 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd000fb50 con 0x7f0fc4005230 2024-04-25T14:11:43.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.277+0000 7f0fe6e00640 1 -- 172.21.15.86:0/733821198 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fe00b35f0 con 0x7f0fc4005230 2024-04-25T14:11:43.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.277+0000 7f0fe6e00640 1 -- 172.21.15.86:0/733821198 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fe00b3bb0 con 0x7f0fc4005230 2024-04-25T14:11:43.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.277+0000 7f0fe6e00640 1 -- 172.21.15.86:0/733821198 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fe00720e0 con 0x7f0fc4005230 2024-04-25T14:11:43.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.281+0000 7f0fdcc00640 1 -- 172.21.15.86:0/733821198 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7f0fd000fcf0 con 0x7f0fc4005230 2024-04-25T14:11:43.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.281+0000 7f0fdcc00640 1 --2- 172.21.15.86:0/733821198 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f0fbc06f550 0x7f0fbc071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:43.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.281+0000 7f0fdcc00640 1 -- 172.21.15.86:0/733821198 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f0fd008f4e0 con 0x7f0fc4005230 2024-04-25T14:11:43.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.281+0000 7f0fd7400640 1 --2- 172.21.15.86:0/733821198 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f0fbc06f550 0x7f0fbc071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:43.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.281+0000 7f0fd7400640 1 --2- 172.21.15.86:0/733821198 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f0fbc06f550 0x7f0fbc071a60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0fe006a480 tx=0x7f0fc8008040 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.285+0000 7f0fdcc00640 1 -- 172.21.15.86:0/733821198 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0fe00b5810 con 0x7f0fc4005230 2024-04-25T14:11:43.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.473+0000 7f0fe6e00640 1 -- 172.21.15.86:0/733821198 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0fe0076bf0 con 0x7f0fc4005230 2024-04-25T14:11:43.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.473+0000 7f0fdcc00640 1 -- 172.21.15.86:0/733821198 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5187 (secure 0 0 0) 0x7f0fd005c120 con 0x7f0fc4005230 2024-04-25T14:11:43.470 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:43.470 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":103,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/1828317917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1828317917},{"type":"v1","addr":"172.21.15.155:6834","nonce":1828317917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":9124,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1389714390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1389714390},{"type":"v1","addr":"172.21.15.86:6837","nonce":1389714390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":9136,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/91793677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":91793677},{"type":"v1","addr":"172.21.15.155:6835","nonce":91793677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:11:32.981538+0000","modified":"2024-04-25T14:11:35.996552+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9546},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9546":{"gid":9546,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/2571655765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2571655765},{"type":"v1","addr":"172.21.15.86:6835","nonce":2571655765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9546,"qdb_cluster":[9546]},"id":10}]} 2024-04-25T14:11:43.470 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 103 2024-04-25T14:11:43.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.473+0000 7f0fe6e00640 1 -- 172.21.15.86:0/733821198 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f0fbc06f550 msgr2=0x7f0fbc071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:43.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.477+0000 7f0fe6e00640 1 --2- 172.21.15.86:0/733821198 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f0fbc06f550 0x7f0fbc071a60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0fe006a480 tx=0x7f0fc8008040 comp rx=0 tx=0).stop 2024-04-25T14:11:43.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.477+0000 7f0fe6e00640 1 -- 172.21.15.86:0/733821198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc4005230 msgr2=0x7f0fe0153cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:43.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.477+0000 7f0fe6e00640 1 --2- 172.21.15.86:0/733821198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc4005230 0x7f0fe0153cd0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0004f00 tx=0x7f0fd0007430 comp rx=0 tx=0).stop 2024-04-25T14:11:43.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.477+0000 7f0fe6e00640 1 -- 172.21.15.86:0/733821198 shutdown_connections 2024-04-25T14:11:43.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.477+0000 7f0fe6e00640 1 --2- 172.21.15.86:0/733821198 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f0fbc06f550 0x7f0fbc071a60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:43.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.477+0000 7f0fe6e00640 1 --2- 172.21.15.86:0/733821198 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe00b5400 0x7f0fe0154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:43.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.477+0000 7f0fe6e00640 1 --2- 172.21.15.86:0/733821198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fe00b4690 0x7f0fe00b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:43.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.477+0000 7f0fe6e00640 1 --2- 172.21.15.86:0/733821198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc4005230 0x7f0fe0153cd0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:43.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.477+0000 7f0fe6e00640 1 -- 172.21.15.86:0/733821198 >> 172.21.15.86:0/733821198 conn(0x7f0fe00cffc0 msgr2=0x7f0fe00cb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:43.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.477+0000 7f0fe6e00640 1 -- 172.21.15.86:0/733821198 shutdown_connections 2024-04-25T14:11:43.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:43.477+0000 7f0fe6e00640 1 -- 172.21.15.86:0/733821198 wait complete. 2024-04-25T14:11:43.894 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 103, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:11:32.981538+0000', 'modified': '2024-04-25T14:11:35.996552+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9546}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9546': {'gid': 9546, 'name': 'a', 'rank': 0, 'incarnation': 99, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6835/2571655765', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6834', 'nonce': 2571655765}, {'type': 'v1', 'addr': '172.21.15.86:6835', 'nonce': 2571655765}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9546, 'qdb_cluster': [9546]} 2024-04-25T14:11:43.894 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T14:11:43.894 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T14:11:44.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.045+0000 7fa6c0600640 1 Processor -- start 2024-04-25T14:11:44.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.045+0000 7fa6c0600640 1 -- start start 2024-04-25T14:11:44.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.049+0000 7fa6c0600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa6b815c2a0 0x7fa6b80ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:44.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.049+0000 7fa6c0600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6b80b0a60 con 0x7fa6b815feb0 2024-04-25T14:11:44.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.049+0000 7fa6c0600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6b806ef90 con 0x7fa6b806fd50 2024-04-25T14:11:44.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.049+0000 7fa6c0600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa6b806f270 con 0x7fa6b815c2a0 2024-04-25T14:11:44.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.049+0000 7fa6be600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa6b815feb0 0x7fa6b81602c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44038/0 (socket says 172.21.15.86:44038) 2024-04-25T14:11:44.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.049+0000 7fa6be600640 1 -- 172.21.15.86:0/3445797647 learned_addr learned my addr 172.21.15.86:0/3445797647 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:44.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.049+0000 7fa6bdc00640 1 --2- 172.21.15.86:0/3445797647 >> v2:172.21.15.155:3300/0 conn(0x7fa6b815c2a0 0x7fa6b80ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:44.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.049+0000 7fa6b7e00640 1 -- 172.21.15.86:0/3445797647 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2476027717 0 0) 0x7fa6b80b0a60 con 0x7fa6b815feb0 2024-04-25T14:11:44.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.049+0000 7fa6b7e00640 1 -- 172.21.15.86:0/3445797647 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6a0003450 con 0x7fa6b815feb0 2024-04-25T14:11:44.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.049+0000 7fa6b7e00640 1 -- 172.21.15.86:0/3445797647 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3087715533 0 0) 0x7fa6b806ef90 con 0x7fa6b806fd50 2024-04-25T14:11:44.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.049+0000 7fa6b7e00640 1 -- 172.21.15.86:0/3445797647 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6b80b0a60 con 0x7fa6b806fd50 2024-04-25T14:11:44.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.049+0000 7fa6bdc00640 1 -- 172.21.15.86:0/3445797647 >> v1:172.21.15.155:6789/0 conn(0x7fa6b806fd50 legacy=0x7fa6b8070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:44.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.049+0000 7fa6bdc00640 1 -- 172.21.15.86:0/3445797647 >> v1:172.21.15.86:6789/0 conn(0x7fa6b815feb0 legacy=0x7fa6b81602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:44.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.049+0000 7fa6bdc00640 1 -- 172.21.15.86:0/3445797647 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b806f550 con 0x7fa6b815c2a0 2024-04-25T14:11:44.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.053+0000 7fa6bdc00640 1 --2- 172.21.15.86:0/3445797647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6b815c2a0 0x7fa6b80ae130 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009d90 tx=0x7fa6ac01c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=951ceeeae8a1b56d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:44.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.053+0000 7fa6b7e00640 1 -- 172.21.15.86:0/3445797647 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac0b2070 con 0x7fa6b815c2a0 2024-04-25T14:11:44.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.053+0000 7fa6b7e00640 1 -- 172.21.15.86:0/3445797647 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6ac0a5800 con 0x7fa6b815c2a0 2024-04-25T14:11:44.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.053+0000 7fa6b7e00640 1 -- 172.21.15.86:0/3445797647 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac0a5bb0 con 0x7fa6b815c2a0 2024-04-25T14:11:44.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.053+0000 7fa6c0600640 1 -- 172.21.15.86:0/3445797647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6b815c2a0 msgr2=0x7fa6b80ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:44.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.053+0000 7fa6c0600640 1 --2- 172.21.15.86:0/3445797647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6b815c2a0 0x7fa6b80ae130 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009d90 tx=0x7fa6ac01c8b0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.053+0000 7fa6c0600640 1 -- 172.21.15.86:0/3445797647 shutdown_connections 2024-04-25T14:11:44.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.053+0000 7fa6c0600640 1 --2- 172.21.15.86:0/3445797647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6b815c2a0 0x7fa6b80ae130 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.053+0000 7fa6c0600640 1 -- 172.21.15.86:0/3445797647 >> 172.21.15.86:0/3445797647 conn(0x7fa6b80b4650 msgr2=0x7fa6b80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:44.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.053+0000 7fa6c0600640 1 -- 172.21.15.86:0/3445797647 shutdown_connections 2024-04-25T14:11:44.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.053+0000 7fa6c0600640 1 -- 172.21.15.86:0/3445797647 wait complete. 2024-04-25T14:11:44.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.057+0000 7fa6c0600640 1 Processor -- start 2024-04-25T14:11:44.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.057+0000 7fa6c0600640 1 -- start start 2024-04-25T14:11:44.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.057+0000 7fa6c0600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6b806fd50 0x7fa6b80913d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:44.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.057+0000 7fa6c0600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6b815c2a0 0x7fa6b8091910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:44.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.057+0000 7fa6c0600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6b815feb0 0x7fa6b8099500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:44.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.057+0000 7fa6c0600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b80ae7f0 con 0x7fa6b806fd50 2024-04-25T14:11:44.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.057+0000 7fa6c0600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b80ae670 con 0x7fa6b815c2a0 2024-04-25T14:11:44.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.057+0000 7fa6c0600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa6b816a200 con 0x7fa6b815feb0 2024-04-25T14:11:44.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.057+0000 7fa6bdc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6b806fd50 0x7fa6b80913d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:44.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.057+0000 7fa6bdc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6b806fd50 0x7fa6b80913d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52976/0 (socket says 172.21.15.86:52976) 2024-04-25T14:11:44.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.057+0000 7fa6bdc00640 1 -- 172.21.15.86:0/29004781 learned_addr learned my addr 172.21.15.86:0/29004781 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:44.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.057+0000 7fa6be600640 1 --2- 172.21.15.86:0/29004781 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6b815feb0 0x7fa6b8099500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:44.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.057+0000 7fa6bd200640 1 --2- 172.21.15.86:0/29004781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6b815c2a0 0x7fa6b8091910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:44.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.057+0000 7fa6bdc00640 1 -- 172.21.15.86:0/29004781 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6b815feb0 msgr2=0x7fa6b8099500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:44.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.057+0000 7fa6bdc00640 1 --2- 172.21.15.86:0/29004781 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6b815feb0 0x7fa6b8099500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.057+0000 7fa6bdc00640 1 -- 172.21.15.86:0/29004781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6b815c2a0 msgr2=0x7fa6b8091910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:44.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.057+0000 7fa6bdc00640 1 --2- 172.21.15.86:0/29004781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6b815c2a0 0x7fa6b8091910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.057+0000 7fa6bdc00640 1 -- 172.21.15.86:0/29004781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b8098d50 con 0x7fa6b806fd50 2024-04-25T14:11:44.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.057+0000 7fa6be600640 1 --2- 172.21.15.86:0/29004781 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6b815feb0 0x7fa6b8099500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:44.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.057+0000 7fa6bd200640 1 --2- 172.21.15.86:0/29004781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6b815c2a0 0x7fa6b8091910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:44.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.057+0000 7fa6bdc00640 1 --2- 172.21.15.86:0/29004781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6b806fd50 0x7fa6b80913d0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac01a4e0 tx=0x7fa6ac019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:44.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.057+0000 7fa6b5600640 1 -- 172.21.15.86:0/29004781 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac0bd020 con 0x7fa6b806fd50 2024-04-25T14:11:44.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.057+0000 7fa6b5600640 1 -- 172.21.15.86:0/29004781 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6ac0039a0 con 0x7fa6b806fd50 2024-04-25T14:11:44.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.057+0000 7fa6b5600640 1 -- 172.21.15.86:0/29004781 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac0b2040 con 0x7fa6b806fd50 2024-04-25T14:11:44.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.057+0000 7fa6c0600640 1 -- 172.21.15.86:0/29004781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b815dd40 con 0x7fa6b806fd50 2024-04-25T14:11:44.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.061+0000 7fa6c0600640 1 -- 172.21.15.86:0/29004781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b809a620 con 0x7fa6b806fd50 2024-04-25T14:11:44.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.061+0000 7fa6b5600640 1 -- 172.21.15.86:0/29004781 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7fa6ac0c0020 con 0x7fa6b806fd50 2024-04-25T14:11:44.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.061+0000 7fa6b5600640 1 --2- 172.21.15.86:0/29004781 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa69806f520 0x7fa698071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:44.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.061+0000 7fa6b5600640 1 -- 172.21.15.86:0/29004781 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7fa6ac12a340 con 0x7fa6b806fd50 2024-04-25T14:11:44.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.061+0000 7fa6bd200640 1 --2- 172.21.15.86:0/29004781 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa69806f520 0x7fa698071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:44.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.061+0000 7fa6c7ba7000 1 Processor -- start 2024-04-25T14:11:44.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.061+0000 7fa6c7ba7000 1 -- start start 2024-04-25T14:11:44.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6bd200640 1 --2- 172.21.15.86:0/29004781 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa69806f520 0x7fa698071a30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8097b60 tx=0x7fa6a4009040 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:44.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6c7ba7000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566e7875c20 0x5566e79d3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:44.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6c7ba7000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566e7861ea0 0x5566e79d4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:44.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6c7ba7000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5566e745a280 0x5566e79d4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:44.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6c7ba7000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5566e7843160 con 0x5566e7861ea0 2024-04-25T14:11:44.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6c7ba7000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5566e77d8e20 con 0x5566e7875c20 2024-04-25T14:11:44.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6c7ba7000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5566e7791a20 con 0x5566e745a280 2024-04-25T14:11:44.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6bd200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566e7861ea0 0x5566e79d4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:44.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6bd200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566e7861ea0 0x5566e79d4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52992/0 (socket says 172.21.15.86:52992) 2024-04-25T14:11:44.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6bd200640 1 -- 172.21.15.86:0/2304565356 learned_addr learned my addr 172.21.15.86:0/2304565356 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:44.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6be600640 1 --2- 172.21.15.86:0/2304565356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5566e745a280 0x5566e79d4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:44.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6bdc00640 1 --2- 172.21.15.86:0/2304565356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566e7875c20 0x5566e79d3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:44.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6bd200640 1 -- 172.21.15.86:0/2304565356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5566e745a280 msgr2=0x5566e79d4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:44.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6bd200640 1 --2- 172.21.15.86:0/2304565356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5566e745a280 0x5566e79d4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6bd200640 1 -- 172.21.15.86:0/2304565356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566e7875c20 msgr2=0x5566e79d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:44.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6bd200640 1 --2- 172.21.15.86:0/2304565356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566e7875c20 0x5566e79d3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6bd200640 1 -- 172.21.15.86:0/2304565356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566e7796100 con 0x5566e7861ea0 2024-04-25T14:11:44.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6bd200640 1 --2- 172.21.15.86:0/2304565356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566e7861ea0 0x5566e79d4420 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400d8d0 tx=0x7fa6a4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410e48b79d5834df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:44.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6aa000640 1 -- 172.21.15.86:0/2304565356 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4024070 con 0x5566e7861ea0 2024-04-25T14:11:44.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6aa000640 1 -- 172.21.15.86:0/2304565356 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6a4014db0 con 0x5566e7861ea0 2024-04-25T14:11:44.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6aa000640 1 -- 172.21.15.86:0/2304565356 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a401f600 con 0x5566e7861ea0 2024-04-25T14:11:44.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6c7ba7000 1 -- 172.21.15.86:0/2304565356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566e7861ea0 msgr2=0x5566e79d4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:44.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6c7ba7000 1 --2- 172.21.15.86:0/2304565356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566e7861ea0 0x5566e79d4420 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400d8d0 tx=0x7fa6a4010000 comp rx=0 tx=0).stop 2024-04-25T14:11:44.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6c7ba7000 1 -- 172.21.15.86:0/2304565356 shutdown_connections 2024-04-25T14:11:44.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6c7ba7000 1 --2- 172.21.15.86:0/2304565356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5566e745a280 0x5566e79d4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6c7ba7000 1 --2- 172.21.15.86:0/2304565356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566e7861ea0 0x5566e79d4420 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6c7ba7000 1 --2- 172.21.15.86:0/2304565356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566e7875c20 0x5566e79d3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6c7ba7000 1 -- 172.21.15.86:0/2304565356 >> 172.21.15.86:0/2304565356 conn(0x5566e7499ff0 msgr2=0x5566e79894c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:44.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6c7ba7000 1 -- 172.21.15.86:0/2304565356 shutdown_connections 2024-04-25T14:11:44.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.065+0000 7fa6c7ba7000 1 -- 172.21.15.86:0/2304565356 wait complete. 2024-04-25T14:11:44.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa6c7ba7000 1 -- start start 2024-04-25T14:11:44.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa6c7ba7000 1 Processor -- start 2024-04-25T14:11:44.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa6c7ba7000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5566e745a280 0x5566e79db070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:44.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa6c7ba7000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566e7861ea0 0x5566e79db5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:44.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa6c7ba7000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566e7875c20 0x5566e79960c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:44.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa6c7ba7000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5566e77e6220 con 0x5566e7861ea0 2024-04-25T14:11:44.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa6c7ba7000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5566e777cea0 con 0x5566e7875c20 2024-04-25T14:11:44.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa6c7ba7000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5566e77d3b00 con 0x5566e745a280 2024-04-25T14:11:44.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa6c7ba7000 10 client.0 _ll_register_callbacks cb 0x5566e77e5dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:11:44.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa6bd200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566e7861ea0 0x5566e79db5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:44.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa6bd200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566e7861ea0 0x5566e79db5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53002/0 (socket says 172.21.15.86:53002) 2024-04-25T14:11:44.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa6bd200640 1 -- 172.21.15.86:0/156928413 learned_addr learned my addr 172.21.15.86:0/156928413 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:44.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa6bdc00640 1 --2- 172.21.15.86:0/156928413 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5566e745a280 0x5566e79db070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:44.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa6be600640 1 --2- 172.21.15.86:0/156928413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566e7875c20 0x5566e79960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:44.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa6bd200640 1 -- 172.21.15.86:0/156928413 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5566e745a280 msgr2=0x5566e79db070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:44.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa6bd200640 1 --2- 172.21.15.86:0/156928413 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5566e745a280 0x5566e79db070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa6bd200640 1 -- 172.21.15.86:0/156928413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566e7875c20 msgr2=0x5566e79960c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:44.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa6bd200640 1 --2- 172.21.15.86:0/156928413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566e7875c20 0x5566e79960c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa6bd200640 1 -- 172.21.15.86:0/156928413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566e7778500 con 0x5566e7861ea0 2024-04-25T14:11:44.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa6bdc00640 1 --2- 172.21.15.86:0/156928413 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5566e745a280 0x5566e79db070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:44.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa6be600640 1 --2- 172.21.15.86:0/156928413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566e7875c20 0x5566e79960c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:44.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa6bd200640 1 --2- 172.21.15.86:0/156928413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566e7861ea0 0x5566e79db5b0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4001ec0 tx=0x7fa6a4000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ae63af896cf168 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:44.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa69ea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:11:44.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa69ea00640 1 -- 172.21.15.86:0/156928413 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a402c070 con 0x5566e7861ea0 2024-04-25T14:11:44.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa69ea00640 1 -- 172.21.15.86:0/156928413 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7fa6a4020840 con 0x5566e7861ea0 2024-04-25T14:11:44.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa6c7ba7000 1 -- 172.21.15.86:0/156928413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6a4014db0 con 0x5566e7861ea0 2024-04-25T14:11:44.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa69ea00640 1 -- 172.21.15.86:0/156928413 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) ==== 24+0+0 (secure 0 0 0) 0x7fa6a4037050 con 0x5566e7861ea0 2024-04-25T14:11:44.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa6c7ba7000 10 client.9714 fetch_fsmap learned FSMap version 103 2024-04-25T14:11:44.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa6c7ba7000 1 -- 172.21.15.86:0/156928413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) -- 0x5566e775eea0 con 0x5566e7861ea0 2024-04-25T14:11:44.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.069+0000 7fa69ea00640 1 -- 172.21.15.86:0/156928413 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6a4014db0 con 0x5566e7861ea0 2024-04-25T14:11:44.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.073+0000 7fa69ea00640 1 -- 172.21.15.86:0/156928413 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 103) ==== 3494+0+0 (secure 0 0 0) 0x7fa6a403a020 con 0x5566e7861ea0 2024-04-25T14:11:44.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.073+0000 7fa6c7ba7000 10 client.9714 fetch_fsmap finished waiting for FSMap version 103 2024-04-25T14:11:44.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.073+0000 7fa6c7ba7000 10 client.9714 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T14:11:44.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.073+0000 7fa6c7ba7000 20 client.9714 populate_metadata read hostname 'smithi086' 2024-04-25T14:11:44.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.073+0000 7fa6c7ba7000 1 --2- 172.21.15.86:0/156928413 >> [v2:172.21.15.86:6834/2571655765,v1:172.21.15.86:6835/2571655765] conn(0x5566e747e7d0 0x5566e7996e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:44.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.073+0000 7fa6c7ba7000 4 client.9714 mds_command: new command op to 9546 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:11:44.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.073+0000 7fa6c7ba7000 1 -- 172.21.15.86:0/156928413 --> [v2:172.21.15.86:6834/2571655765,v1:172.21.15.86:6835/2571655765] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566e7771be0 con 0x5566e747e7d0 2024-04-25T14:11:44.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.073+0000 7fa6bdc00640 1 --2- 172.21.15.86:0/156928413 >> [v2:172.21.15.86:6834/2571655765,v1:172.21.15.86:6835/2571655765] conn(0x5566e747e7d0 0x5566e7996e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:44.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.073+0000 7fa69ea00640 10 client.9714 ms_handle_connect on v2:172.21.15.86:6834/2571655765 2024-04-25T14:11:44.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.073+0000 7fa6bdc00640 1 --2- 172.21.15.86:0/156928413 >> [v2:172.21.15.86:6834/2571655765,v1:172.21.15.86:6835/2571655765] conn(0x5566e747e7d0 0x5566e7996e00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f39dfcf2442422a server_cookie=79245e6620d381a3 in_seq=0 out_seq=0 2024-04-25T14:11:44.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.073+0000 7fa69ea00640 1 -- 172.21.15.86:0/156928413 <== mds.0 v2:172.21.15.86:6834/2571655765 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5566e7771be0 con 0x5566e747e7d0 2024-04-25T14:11:44.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.073+0000 7fa69ea00640 10 client.9714 handle_command_reply: tid=0 2024-04-25T14:11:44.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.073+0000 7fa6c7ba7000 1 client.9714 shutdown 2024-04-25T14:11:44.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.073+0000 7fa6c7ba7000 1 -- 172.21.15.86:0/156928413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566e7861ea0 msgr2=0x5566e79db5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:44.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.073+0000 7fa6c7ba7000 1 --2- 172.21.15.86:0/156928413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566e7861ea0 0x5566e79db5b0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4001ec0 tx=0x7fa6a4000ee0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.073+0000 7fa6c7ba7000 1 -- 172.21.15.86:0/156928413 shutdown_connections 2024-04-25T14:11:44.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.073+0000 7fa6c7ba7000 1 --2- 172.21.15.86:0/156928413 >> [v2:172.21.15.86:6834/2571655765,v1:172.21.15.86:6835/2571655765] conn(0x5566e747e7d0 0x5566e7996e00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.073+0000 7fa6c7ba7000 1 --2- 172.21.15.86:0/156928413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566e7875c20 0x5566e79960c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.073+0000 7fa6c7ba7000 1 --2- 172.21.15.86:0/156928413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566e7861ea0 0x5566e79db5b0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.073+0000 7fa69ea00640 0 client.9714 ms_handle_reset on v2:172.21.15.86:6834/2571655765 2024-04-25T14:11:44.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.073+0000 7fa6c7ba7000 1 --2- 172.21.15.86:0/156928413 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5566e745a280 0x5566e79db070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.073+0000 7fa6c7ba7000 1 -- 172.21.15.86:0/156928413 >> 172.21.15.86:0/156928413 conn(0x5566e7499ff0 msgr2=0x5566e7989c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:44.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.073+0000 7fa6c7ba7000 1 -- 172.21.15.86:0/156928413 shutdown_connections 2024-04-25T14:11:44.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.073+0000 7fa6c7ba7000 1 -- 172.21.15.86:0/156928413 wait complete. 2024-04-25T14:11:44.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.073+0000 7fa6c7ba7000 20 client.9714 trim_cache size 0 max 16384 2024-04-25T14:11:44.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.093+0000 7fa6c7ba7000 1 Processor -- start 2024-04-25T14:11:44.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.093+0000 7fa6c7ba7000 1 -- start start 2024-04-25T14:11:44.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.093+0000 7fa6c7ba7000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566e745a280 0x5566e79eb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:44.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.093+0000 7fa6c7ba7000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566e7861ea0 0x5566e79eb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:44.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.093+0000 7fa6c7ba7000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5566e7875c20 0x5566e79d9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:44.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.093+0000 7fa6c7ba7000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b82055f0 con 0x5566e745a280 2024-04-25T14:11:44.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.093+0000 7fa6c7ba7000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b8097520 con 0x5566e7861ea0 2024-04-25T14:11:44.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.093+0000 7fa6c7ba7000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa6b8099fa0 con 0x5566e7875c20 2024-04-25T14:11:44.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.093+0000 7fa6bdc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566e745a280 0x5566e79eb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:44.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.093+0000 7fa6bdc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566e745a280 0x5566e79eb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53006/0 (socket says 172.21.15.86:53006) 2024-04-25T14:11:44.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.093+0000 7fa6bdc00640 1 -- 172.21.15.86:0/3291637829 learned_addr learned my addr 172.21.15.86:0/3291637829 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:44.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.093+0000 7fa6bdc00640 1 -- 172.21.15.86:0/3291637829 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5566e7875c20 msgr2=0x5566e79d9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:44.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.093+0000 7fa6bdc00640 1 --2- 172.21.15.86:0/3291637829 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5566e7875c20 0x5566e79d9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.093+0000 7fa6bdc00640 1 -- 172.21.15.86:0/3291637829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566e7861ea0 msgr2=0x5566e79eb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:44.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.093+0000 7fa6bdc00640 1 --2- 172.21.15.86:0/3291637829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566e7861ea0 0x5566e79eb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.093+0000 7fa6bdc00640 1 -- 172.21.15.86:0/3291637829 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566e7996430 con 0x5566e745a280 2024-04-25T14:11:44.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.093+0000 7fa6bdc00640 1 --2- 172.21.15.86:0/3291637829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566e745a280 0x5566e79eb210 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac0a7040 tx=0x7fa6ac009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3844f8618f35410 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:44.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.093+0000 7fa6a9600640 1 -- 172.21.15.86:0/3291637829 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac0f6420 con 0x5566e745a280 2024-04-25T14:11:44.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.093+0000 7fa6a9600640 1 -- 172.21.15.86:0/3291637829 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6ac10f070 con 0x5566e745a280 2024-04-25T14:11:44.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.093+0000 7fa6a9600640 1 -- 172.21.15.86:0/3291637829 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac0f7010 con 0x5566e745a280 2024-04-25T14:11:44.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.093+0000 7fa6c7ba7000 1 -- 172.21.15.86:0/3291637829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566e745a280 msgr2=0x5566e79eb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:44.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.093+0000 7fa6c7ba7000 1 --2- 172.21.15.86:0/3291637829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566e745a280 0x5566e79eb210 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac0a7040 tx=0x7fa6ac009750 comp rx=0 tx=0).stop 2024-04-25T14:11:44.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.093+0000 7fa6c7ba7000 1 -- 172.21.15.86:0/3291637829 shutdown_connections 2024-04-25T14:11:44.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.093+0000 7fa6c7ba7000 1 --2- 172.21.15.86:0/3291637829 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5566e7875c20 0x5566e79d9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.093+0000 7fa6c7ba7000 1 --2- 172.21.15.86:0/3291637829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566e7861ea0 0x5566e79eb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.093+0000 7fa6c7ba7000 1 --2- 172.21.15.86:0/3291637829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566e745a280 0x5566e79eb210 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.093+0000 7fa6c7ba7000 1 -- 172.21.15.86:0/3291637829 >> 172.21.15.86:0/3291637829 conn(0x5566e7499ff0 msgr2=0x5566e7991040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:44.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.093+0000 7fa6c7ba7000 1 -- 172.21.15.86:0/3291637829 shutdown_connections 2024-04-25T14:11:44.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.097+0000 7fa6c7ba7000 1 -- 172.21.15.86:0/3291637829 wait complete. 2024-04-25T14:11:44.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.097+0000 7fa6c7ba7000 1 -- start start 2024-04-25T14:11:44.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.097+0000 7fa6c7ba7000 1 Processor -- start 2024-04-25T14:11:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.097+0000 7fa6c7ba7000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5566e745a280 0x5566e7a46590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.097+0000 7fa6c7ba7000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566e7861ea0 0x5566e7a46ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.097+0000 7fa6c7ba7000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566e7875c20 0x5566e79deb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.097+0000 7fa6c7ba7000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5566e77d8130 con 0x5566e7861ea0 2024-04-25T14:11:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.097+0000 7fa6c7ba7000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5566e776e7d0 con 0x5566e7875c20 2024-04-25T14:11:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.097+0000 7fa6c7ba7000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5566e77dd450 con 0x5566e745a280 2024-04-25T14:11:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.097+0000 7fa6c7ba7000 10 client.0 _ll_register_callbacks cb 0x7fa6a4022070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:11:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.097+0000 7fa6bd200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566e7861ea0 0x5566e7a46ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.097+0000 7fa6bd200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566e7861ea0 0x5566e7a46ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53012/0 (socket says 172.21.15.86:53012) 2024-04-25T14:11:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.097+0000 7fa6bd200640 1 -- 172.21.15.86:0/662463062 learned_addr learned my addr 172.21.15.86:0/662463062 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.097+0000 7fa6bdc00640 1 --2- 172.21.15.86:0/662463062 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5566e745a280 0x5566e7a46590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:44.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.097+0000 7fa6be600640 1 --2- 172.21.15.86:0/662463062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566e7875c20 0x5566e79deb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:44.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.097+0000 7fa6bd200640 1 -- 172.21.15.86:0/662463062 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5566e745a280 msgr2=0x5566e7a46590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:44.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.097+0000 7fa6bd200640 1 --2- 172.21.15.86:0/662463062 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5566e745a280 0x5566e7a46590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.097+0000 7fa6bd200640 1 -- 172.21.15.86:0/662463062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566e7875c20 msgr2=0x5566e79deb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:44.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.097+0000 7fa6bd200640 1 --2- 172.21.15.86:0/662463062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566e7875c20 0x5566e79deb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.097+0000 7fa6bd200640 1 -- 172.21.15.86:0/662463062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566e7791c70 con 0x5566e7861ea0 2024-04-25T14:11:44.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.097+0000 7fa6bd200640 1 --2- 172.21.15.86:0/662463062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566e7861ea0 0x5566e7a46ad0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400d8d0 tx=0x7fa6a4001160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d54aa558da40112 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:44.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.097+0000 7fa69ea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:11:44.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.097+0000 7fa69ea00640 1 -- 172.21.15.86:0/662463062 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a402c040 con 0x5566e7861ea0 2024-04-25T14:11:44.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.097+0000 7fa69ea00640 1 -- 172.21.15.86:0/662463062 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7fa6a4031850 con 0x5566e7861ea0 2024-04-25T14:11:44.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.097+0000 7fa6c7ba7000 1 -- 172.21.15.86:0/662463062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6ac10f070 con 0x5566e7861ea0 2024-04-25T14:11:44.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa69ea00640 1 -- 172.21.15.86:0/662463062 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) ==== 24+0+0 (secure 0 0 0) 0x7fa6a4048050 con 0x5566e7861ea0 2024-04-25T14:11:44.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c7ba7000 10 client.9720 fetch_fsmap learned FSMap version 103 2024-04-25T14:11:44.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c7ba7000 1 -- 172.21.15.86:0/662463062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) -- 0x5566e785b180 con 0x5566e7861ea0 2024-04-25T14:11:44.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa69ea00640 1 -- 172.21.15.86:0/662463062 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6ac10f070 con 0x5566e7861ea0 2024-04-25T14:11:44.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa69ea00640 1 -- 172.21.15.86:0/662463062 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 103) ==== 3494+0+0 (secure 0 0 0) 0x7fa6a404b020 con 0x5566e7861ea0 2024-04-25T14:11:44.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c7ba7000 10 client.9720 fetch_fsmap finished waiting for FSMap version 103 2024-04-25T14:11:44.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c7ba7000 10 client.9720 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T14:11:44.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c7ba7000 20 client.9720 populate_metadata read hostname 'smithi086' 2024-04-25T14:11:44.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c7ba7000 1 --2- 172.21.15.86:0/662463062 >> [v2:172.21.15.86:6834/2571655765,v1:172.21.15.86:6835/2571655765] conn(0x5566e746c830 0x5566e79df4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:44.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c7ba7000 4 client.9720 mds_command: new command op to 9546 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:11:44.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c7ba7000 1 -- 172.21.15.86:0/662463062 --> [v2:172.21.15.86:6834/2571655765,v1:172.21.15.86:6835/2571655765] -- command(tid 0: {"prefix": "status"}) -- 0x5566e77e6a30 con 0x5566e746c830 2024-04-25T14:11:44.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6bdc00640 1 --2- 172.21.15.86:0/662463062 >> [v2:172.21.15.86:6834/2571655765,v1:172.21.15.86:6835/2571655765] conn(0x5566e746c830 0x5566e79df4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:44.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa69ea00640 10 client.9720 ms_handle_connect on v2:172.21.15.86:6834/2571655765 2024-04-25T14:11:44.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6bdc00640 1 --2- 172.21.15.86:0/662463062 >> [v2:172.21.15.86:6834/2571655765,v1:172.21.15.86:6835/2571655765] conn(0x5566e746c830 0x5566e79df4c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3e2f92f2522e0e3 server_cookie=57daaf62ccf7bc55 in_seq=0 out_seq=0 2024-04-25T14:11:44.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa69ea00640 1 -- 172.21.15.86:0/662463062 <== mds.0 v2:172.21.15.86:6834/2571655765 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5566e77e6a30 con 0x5566e746c830 2024-04-25T14:11:44.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa69ea00640 10 client.9720 handle_command_reply: tid=0 2024-04-25T14:11:44.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c7ba7000 1 client.9720 shutdown 2024-04-25T14:11:44.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c7ba7000 1 -- 172.21.15.86:0/662463062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566e7861ea0 msgr2=0x5566e7a46ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:44.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c7ba7000 1 --2- 172.21.15.86:0/662463062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566e7861ea0 0x5566e7a46ad0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400d8d0 tx=0x7fa6a4001160 comp rx=0 tx=0).stop 2024-04-25T14:11:44.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c7ba7000 1 -- 172.21.15.86:0/662463062 shutdown_connections 2024-04-25T14:11:44.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c7ba7000 1 --2- 172.21.15.86:0/662463062 >> [v2:172.21.15.86:6834/2571655765,v1:172.21.15.86:6835/2571655765] conn(0x5566e746c830 0x5566e79df4c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c7ba7000 1 --2- 172.21.15.86:0/662463062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5566e7875c20 0x5566e79deb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c7ba7000 1 --2- 172.21.15.86:0/662463062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566e7861ea0 0x5566e7a46ad0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa69ea00640 0 client.9720 ms_handle_reset on v2:172.21.15.86:6834/2571655765 2024-04-25T14:11:44.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c7ba7000 1 --2- 172.21.15.86:0/662463062 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5566e745a280 0x5566e7a46590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c7ba7000 1 -- 172.21.15.86:0/662463062 >> 172.21.15.86:0/662463062 conn(0x5566e7499ff0 msgr2=0x5566e79ea200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:44.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c7ba7000 1 -- 172.21.15.86:0/662463062 shutdown_connections 2024-04-25T14:11:44.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c7ba7000 1 -- 172.21.15.86:0/662463062 wait complete. 2024-04-25T14:11:44.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c7ba7000 20 client.9720 trim_cache size 0 max 16384 2024-04-25T14:11:44.102 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:11:44.102 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb", 2024-04-25T14:11:44.102 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-25T14:11:44.102 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 9546, 2024-04-25T14:11:44.102 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T14:11:44.102 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T14:11:44.102 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T14:11:44.102 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 10.52415693, 2024-04-25T14:11:44.102 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 103, 2024-04-25T14:11:44.102 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 140, 2024-04-25T14:11:44.102 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 140, 2024-04-25T14:11:44.102 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 15.853289137999999 2024-04-25T14:11:44.103 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:11:44.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c0600640 1 -- 172.21.15.86:0/29004781 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa69806f520 msgr2=0x7fa698071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:44.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c0600640 1 --2- 172.21.15.86:0/29004781 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa69806f520 0x7fa698071a30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8097b60 tx=0x7fa6a4009040 comp rx=0 tx=0).stop 2024-04-25T14:11:44.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c0600640 1 -- 172.21.15.86:0/29004781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6b806fd50 msgr2=0x7fa6b80913d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:44.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c0600640 1 --2- 172.21.15.86:0/29004781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6b806fd50 0x7fa6b80913d0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac01a4e0 tx=0x7fa6ac019650 comp rx=0 tx=0).stop 2024-04-25T14:11:44.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c0600640 1 -- 172.21.15.86:0/29004781 shutdown_connections 2024-04-25T14:11:44.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c0600640 1 --2- 172.21.15.86:0/29004781 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7fa69806f520 0x7fa698071a30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c0600640 1 --2- 172.21.15.86:0/29004781 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6b815feb0 0x7fa6b8099500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c0600640 1 --2- 172.21.15.86:0/29004781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6b815c2a0 0x7fa6b8091910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c0600640 1 --2- 172.21.15.86:0/29004781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6b806fd50 0x7fa6b80913d0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c0600640 1 -- 172.21.15.86:0/29004781 >> 172.21.15.86:0/29004781 conn(0x7fa6b80b4650 msgr2=0x7fa6b815a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:44.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c0600640 1 -- 172.21.15.86:0/29004781 shutdown_connections 2024-04-25T14:11:44.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.101+0000 7fa6c0600640 1 -- 172.21.15.86:0/29004781 wait complete. 2024-04-25T14:11:44.118 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T14:11:44.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.269+0000 7f9cede00640 1 Processor -- start 2024-04-25T14:11:44.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.269+0000 7f9cede00640 1 -- start start 2024-04-25T14:11:44.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.269+0000 7f9cede00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ce8071530 0x7f9ce80ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:44.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.269+0000 7f9cede00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9ce806e700 0x7f9ce806eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:44.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.269+0000 7f9cede00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ce806f050 con 0x7f9ce80cdd70 2024-04-25T14:11:44.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.269+0000 7f9cede00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9ce806d940 con 0x7f9ce806e700 2024-04-25T14:11:44.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.269+0000 7f9cede00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9ce806dc20 con 0x7f9ce8071530 2024-04-25T14:11:44.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.269+0000 7f9ce7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9ce806e700 0x7f9ce806eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:44.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.269+0000 7f9ce6a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9ce80cdd70 0x7f9ce8070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44040/0 (socket says 172.21.15.86:44040) 2024-04-25T14:11:44.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.269+0000 7f9ce6a00640 1 -- 172.21.15.86:0/1452354219 learned_addr learned my addr 172.21.15.86:0/1452354219 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:44.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.269+0000 7f9cde000640 1 --2- 172.21.15.86:0/1452354219 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ce8071530 0x7f9ce80ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:44.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.269+0000 7f9ce7400640 1 -- 172.21.15.86:0/1452354219 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ce8071530 msgr2=0x7f9ce80ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:44.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.269+0000 7f9ce7400640 1 --2- 172.21.15.86:0/1452354219 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ce8071530 0x7f9ce80ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.269+0000 7f9ce7400640 1 -- 172.21.15.86:0/1452354219 >> v1:172.21.15.86:6789/0 conn(0x7f9ce80cdd70 legacy=0x7f9ce8070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:44.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.269+0000 7f9ce7400640 1 -- 172.21.15.86:0/1452354219 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce806df00 con 0x7f9ce806e700 2024-04-25T14:11:44.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.269+0000 7f9cde000640 1 --2- 172.21.15.86:0/1452354219 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ce8071530 0x7f9ce80ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:44.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.269+0000 7f9ce6000640 1 -- 172.21.15.86:0/1452354219 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4249512771 0 0) 0x7f9ce806f050 con 0x7f9ce80cdd70 2024-04-25T14:11:44.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.273+0000 7f9ce7400640 1 --2- 172.21.15.86:0/1452354219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ce806e700 0x7f9ce806eb10 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0009d90 tx=0x7f9ce001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb3c30c1cc99e83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:44.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.273+0000 7f9ce6000640 1 -- 172.21.15.86:0/1452354219 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce00b2070 con 0x7f9ce806e700 2024-04-25T14:11:44.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.273+0000 7f9ce6000640 1 -- 172.21.15.86:0/1452354219 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ce00a57f0 con 0x7f9ce806e700 2024-04-25T14:11:44.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.273+0000 7f9ce6000640 1 -- 172.21.15.86:0/1452354219 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce00a5bc0 con 0x7f9ce806e700 2024-04-25T14:11:44.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.273+0000 7f9cede00640 1 -- 172.21.15.86:0/1452354219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ce806e700 msgr2=0x7f9ce806eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:44.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.273+0000 7f9cede00640 1 --2- 172.21.15.86:0/1452354219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ce806e700 0x7f9ce806eb10 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0009d90 tx=0x7f9ce001c8e0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.273+0000 7f9cede00640 1 -- 172.21.15.86:0/1452354219 shutdown_connections 2024-04-25T14:11:44.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.273+0000 7f9cede00640 1 --2- 172.21.15.86:0/1452354219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ce806e700 0x7f9ce806eb10 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.273+0000 7f9cede00640 1 --2- 172.21.15.86:0/1452354219 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ce8071530 0x7f9ce80ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.273+0000 7f9cede00640 1 -- 172.21.15.86:0/1452354219 >> 172.21.15.86:0/1452354219 conn(0x7f9ce80cffc0 msgr2=0x7f9ce80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:44.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.273+0000 7f9cede00640 1 -- 172.21.15.86:0/1452354219 shutdown_connections 2024-04-25T14:11:44.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.273+0000 7f9cede00640 1 -- 172.21.15.86:0/1452354219 wait complete. 2024-04-25T14:11:44.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.277+0000 7f9cede00640 1 Processor -- start 2024-04-25T14:11:44.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.277+0000 7f9cede00640 1 -- start start 2024-04-25T14:11:44.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.277+0000 7f9cede00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ce806e700 0x7f9ce814e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:44.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.277+0000 7f9cede00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ce80cdd70 0x7f9ce8153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:44.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.277+0000 7f9cede00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ce815ce10 0x7f9ce8154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:44.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.277+0000 7f9cede00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce816cc00 con 0x7f9ce815ce10 2024-04-25T14:11:44.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.277+0000 7f9cede00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce8164380 con 0x7f9ce806e700 2024-04-25T14:11:44.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.277+0000 7f9cede00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9ce816cd80 con 0x7f9ce80cdd70 2024-04-25T14:11:44.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.277+0000 7f9ce7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ce815ce10 0x7f9ce8154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:44.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.277+0000 7f9ce7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ce815ce10 0x7f9ce8154500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53022/0 (socket says 172.21.15.86:53022) 2024-04-25T14:11:44.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.277+0000 7f9ce7400640 1 -- 172.21.15.86:0/22197118 learned_addr learned my addr 172.21.15.86:0/22197118 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:44.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.277+0000 7f9ce6a00640 1 --2- 172.21.15.86:0/22197118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ce806e700 0x7f9ce814e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:44.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.277+0000 7f9cde000640 1 --2- 172.21.15.86:0/22197118 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ce80cdd70 0x7f9ce8153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:44.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.281+0000 7f9ce7400640 1 -- 172.21.15.86:0/22197118 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ce80cdd70 msgr2=0x7f9ce8153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:44.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.281+0000 7f9ce7400640 1 --2- 172.21.15.86:0/22197118 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ce80cdd70 0x7f9ce8153fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.281+0000 7f9ce7400640 1 -- 172.21.15.86:0/22197118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ce806e700 msgr2=0x7f9ce814e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:44.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.281+0000 7f9ce7400640 1 --2- 172.21.15.86:0/22197118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ce806e700 0x7f9ce814e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.281+0000 7f9ce7400640 1 -- 172.21.15.86:0/22197118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce80b81b0 con 0x7f9ce815ce10 2024-04-25T14:11:44.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.281+0000 7f9ce7400640 1 --2- 172.21.15.86:0/22197118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ce815ce10 0x7f9ce8154500 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f9ce001af90 tx=0x7f9ce0019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:44.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.281+0000 7f9cdfe00640 1 -- 172.21.15.86:0/22197118 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce00b2050 con 0x7f9ce815ce10 2024-04-25T14:11:44.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.281+0000 7f9cdfe00640 1 -- 172.21.15.86:0/22197118 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ce00be070 con 0x7f9ce815ce10 2024-04-25T14:11:44.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.281+0000 7f9cede00640 1 -- 172.21.15.86:0/22197118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce8160380 con 0x7f9ce815ce10 2024-04-25T14:11:44.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.281+0000 7f9cdfe00640 1 -- 172.21.15.86:0/22197118 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce00ad850 con 0x7f9ce815ce10 2024-04-25T14:11:44.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.281+0000 7f9cede00640 1 -- 172.21.15.86:0/22197118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce80b8730 con 0x7f9ce815ce10 2024-04-25T14:11:44.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.281+0000 7f9cede00640 1 -- 172.21.15.86:0/22197118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ce80720e0 con 0x7f9ce815ce10 2024-04-25T14:11:44.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.281+0000 7f9cdfe00640 1 -- 172.21.15.86:0/22197118 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7f9ce00ad9f0 con 0x7f9ce815ce10 2024-04-25T14:11:44.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.285+0000 7f9cdfe00640 1 --2- 172.21.15.86:0/22197118 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f9cc406f550 0x7f9cc4071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:44.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.285+0000 7f9cdfe00640 1 -- 172.21.15.86:0/22197118 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f9ce0129e80 con 0x7f9ce815ce10 2024-04-25T14:11:44.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.285+0000 7f9ce6a00640 1 --2- 172.21.15.86:0/22197118 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f9cc406f550 0x7f9cc4071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:44.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.285+0000 7f9ce6a00640 1 --2- 172.21.15.86:0/22197118 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f9cc406f550 0x7f9cc4071a60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8000f70 tx=0x7f9cd8007df0 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:44.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.289+0000 7f9cdfe00640 1 -- 172.21.15.86:0/22197118 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9ce00c7020 con 0x7f9ce815ce10 2024-04-25T14:11:44.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.481+0000 7f9cede00640 1 -- 172.21.15.86:0/22197118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9ce8076bf0 con 0x7f9ce815ce10 2024-04-25T14:11:44.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.481+0000 7f9cdfe00640 1 -- 172.21.15.86:0/22197118 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v103) ==== 53+0+73 (secure 0 0 0) 0x7f9ce0016560 con 0x7f9ce815ce10 2024-04-25T14:11:44.479 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T14:11:44.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.485+0000 7f9cede00640 1 -- 172.21.15.86:0/22197118 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f9cc406f550 msgr2=0x7f9cc4071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:44.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.485+0000 7f9cede00640 1 --2- 172.21.15.86:0/22197118 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f9cc406f550 0x7f9cc4071a60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8000f70 tx=0x7f9cd8007df0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.485+0000 7f9cede00640 1 -- 172.21.15.86:0/22197118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ce815ce10 msgr2=0x7f9ce8154500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:44.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.485+0000 7f9cede00640 1 --2- 172.21.15.86:0/22197118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ce815ce10 0x7f9ce8154500 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f9ce001af90 tx=0x7f9ce0019680 comp rx=0 tx=0).stop 2024-04-25T14:11:44.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.485+0000 7f9cede00640 1 -- 172.21.15.86:0/22197118 shutdown_connections 2024-04-25T14:11:44.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.485+0000 7f9cede00640 1 --2- 172.21.15.86:0/22197118 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f9cc406f550 0x7f9cc4071a60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.485+0000 7f9cede00640 1 --2- 172.21.15.86:0/22197118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ce815ce10 0x7f9ce8154500 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.485+0000 7f9cede00640 1 --2- 172.21.15.86:0/22197118 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ce80cdd70 0x7f9ce8153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.485+0000 7f9cede00640 1 --2- 172.21.15.86:0/22197118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ce806e700 0x7f9ce814e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:44.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.485+0000 7f9cede00640 1 -- 172.21.15.86:0/22197118 >> 172.21.15.86:0/22197118 conn(0x7f9ce80cffc0 msgr2=0x7f9ce80cf220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:44.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.485+0000 7f9cede00640 1 -- 172.21.15.86:0/22197118 shutdown_connections 2024-04-25T14:11:44.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44.485+0000 7f9cede00640 1 -- 172.21.15.86:0/22197118 wait complete. 2024-04-25T14:11:44.500 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T14:11:44.500 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T14:11:44.500 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-25T14:11:44.500 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T14:11:44.500 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T14:11:44.500 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T14:11:44.500 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T14:11:44.500 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T14:11:44.500 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:11:44.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:11:44.501 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-25T14:11:44.510 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:11:44.510 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:11:44.510 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:11:44.510 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:11:44.510 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-25T14:11:44.510 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:11:44.510 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:11:44.510 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:11:44.511 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-25T14:11:44.511 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:11:44.511 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:11:44.511 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-25T14:11:44.511 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:11:44.511 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:11:44.511 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T14:11:44.511 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:11:44.511 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T14:11:44.511 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 41960sec preferred_lft 41960sec 2024-04-25T14:11:44.511 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-25T14:11:44.511 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:11:44.511 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T14:11:44.512 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:dc:87:9c:96 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:11:44.512 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T14:11:44.512 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:11:44.512 INFO:teuthology.orchestra.run.smithi086.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T14:11:44.512 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T14:11:44.512 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T14:11:44.512 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:11:44.512 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T14:11:44.512 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:11:44.512 INFO:teuthology.orchestra.run.smithi086.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:11:44.512 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T14:11:44.512 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T14:11:44.512 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:11:44.512 INFO:teuthology.orchestra.run.smithi086.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:11:44.513 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T14:11:44.513 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T14:11:44.513 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:11:44.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:11:44.514 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T14:11:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-25T14:11:44.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:11:44.580 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-25T14:11:44.639 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-25T14:11:44.639 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T14:11:44.639 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T14:11:44.639 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T14:11:44.639 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T14:11:44.639 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T14:11:44.640 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T14:11:44.640 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:11:44.640 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:11:44.640 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:11:44.640 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T14:11:44.640 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:11:44.640 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T14:11:44.640 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T14:11:44.640 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T14:11:44.640 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T14:11:44.640 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T14:11:44.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:11:44.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:11:44.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:11:44.773 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T14:11:44.782 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T14:11:44.782 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T14:11:44.783 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:11:44.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:11:44.783 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-25T14:11:44.847 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:11:44.848 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T14:11:44.899 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:11:44.899 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:11:44.899 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T14:11:44.947 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T14:11:44.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:11:44.947 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:11:45.047 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:11:45.047 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T14:11:45.047 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:11:45.065 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:11:45.065 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T14:11:45.065 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:11:45.065 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:11:45.065 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:11:45.075 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:11:45.075 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T14:11:45.075 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:11:45.075 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:11:45.075 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:11:45.075 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790" 2024-04-25T14:11:45.075 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790:/ 2024-04-25T14:11:45.075 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb" will pass to kernel 2024-04-25T14:11:45.079 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T14:11:45.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:11:45.079 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T14:11:45.098 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:11:45.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:11:45.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:11:45.201 INFO:teuthology.orchestra.run.smithi086.stdout:client9732 2024-04-25T14:11:45.201 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:11:45.201 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:11:45.210 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T14:11:45.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.405+0000 7f1403200640 1 Processor -- start 2024-04-25T14:11:45.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.405+0000 7f1403200640 1 -- start start 2024-04-25T14:11:45.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.405+0000 7f1403200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f14040d2d70 0x7f1404163780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:45.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.405+0000 7f1403200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1404071580 0x7f14040d1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:45.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.405+0000 7f1403200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14040cf9e0 con 0x7f14040d16b0 2024-04-25T14:11:45.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.405+0000 7f1403200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1404163e80 con 0x7f14040d2d70 2024-04-25T14:11:45.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.405+0000 7f1403200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1404164160 con 0x7f1404071580 2024-04-25T14:11:45.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.409+0000 7f1401e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f14040d2d70 0x7f1404163780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:45.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.409+0000 7f1401e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f14040d2d70 0x7f1404163780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53026/0 (socket says 172.21.15.86:53026) 2024-04-25T14:11:45.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.409+0000 7f1401e00640 1 -- 172.21.15.86:0/3496040808 learned_addr learned my addr 172.21.15.86:0/3496040808 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:45.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.409+0000 7f1402800640 1 --2- 172.21.15.86:0/3496040808 >> v2:172.21.15.155:3300/0 conn(0x7f1404071580 0x7f14040d1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:45.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.409+0000 7f1401e00640 1 -- 172.21.15.86:0/3496040808 >> v2:172.21.15.155:3300/0 conn(0x7f1404071580 msgr2=0x7f14040d1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:45.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.409+0000 7f1401e00640 1 --2- 172.21.15.86:0/3496040808 >> v2:172.21.15.155:3300/0 conn(0x7f1404071580 0x7f14040d1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:45.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.409+0000 7f1401e00640 1 -- 172.21.15.86:0/3496040808 >> v1:172.21.15.155:6789/0 conn(0x7f14040d16b0 legacy=0x7f1404070e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:45.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.409+0000 7f1401e00640 1 -- 172.21.15.86:0/3496040808 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1404164440 con 0x7f14040d2d70 2024-04-25T14:11:45.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.409+0000 7f1402800640 1 --2- 172.21.15.86:0/3496040808 >> v2:172.21.15.155:3300/0 conn(0x7f1404071580 0x7f14040d1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:45.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.409+0000 7f1401e00640 1 --2- 172.21.15.86:0/3496040808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14040d2d70 0x7f1404163780 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f14040ca430 tx=0x7f13f401c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8cd5bc3e1559397 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:45.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.409+0000 7f1401400640 1 -- 172.21.15.86:0/3496040808 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f40b0070 con 0x7f14040d2d70 2024-04-25T14:11:45.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.409+0000 7f1401400640 1 -- 172.21.15.86:0/3496040808 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13f40ab600 con 0x7f14040d2d70 2024-04-25T14:11:45.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.409+0000 7f1401400640 1 -- 172.21.15.86:0/3496040808 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f40ab9d0 con 0x7f14040d2d70 2024-04-25T14:11:45.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.413+0000 7f1403200640 1 -- 172.21.15.86:0/3496040808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14040d2d70 msgr2=0x7f1404163780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:45.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.413+0000 7f1403200640 1 --2- 172.21.15.86:0/3496040808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14040d2d70 0x7f1404163780 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f14040ca430 tx=0x7f13f401c270 comp rx=0 tx=0).stop 2024-04-25T14:11:45.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.413+0000 7f1403200640 1 -- 172.21.15.86:0/3496040808 shutdown_connections 2024-04-25T14:11:45.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.413+0000 7f1403200640 1 --2- 172.21.15.86:0/3496040808 >> v2:172.21.15.155:3300/0 conn(0x7f1404071580 0x7f14040d1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:45.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.413+0000 7f1403200640 1 --2- 172.21.15.86:0/3496040808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14040d2d70 0x7f1404163780 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:45.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.413+0000 7f1403200640 1 -- 172.21.15.86:0/3496040808 >> 172.21.15.86:0/3496040808 conn(0x7f14040cfea0 msgr2=0x7f14040d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:45.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.413+0000 7f1403200640 1 -- 172.21.15.86:0/3496040808 shutdown_connections 2024-04-25T14:11:45.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.413+0000 7f1403200640 1 -- 172.21.15.86:0/3496040808 wait complete. 2024-04-25T14:11:45.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.413+0000 7f1403200640 1 Processor -- start 2024-04-25T14:11:45.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.413+0000 7f1403200640 1 -- start start 2024-04-25T14:11:45.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.417+0000 7f1403200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14040d16b0 0x7f1404159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:45.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.417+0000 7f1403200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14040d2d70 0x7f140414c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:45.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.417+0000 7f1403200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1404154eb0 0x7f14041552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:45.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.417+0000 7f1403200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f140416d660 con 0x7f14040d2d70 2024-04-25T14:11:45.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.417+0000 7f1403200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f140416d4e0 con 0x7f1404154eb0 2024-04-25T14:11:45.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.417+0000 7f1403200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f140416d7e0 con 0x7f14040d16b0 2024-04-25T14:11:45.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.417+0000 7f13f9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14040d2d70 0x7f140414c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:45.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.417+0000 7f13f9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14040d2d70 0x7f140414c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53030/0 (socket says 172.21.15.86:53030) 2024-04-25T14:11:45.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.417+0000 7f13f9400640 1 -- 172.21.15.86:0/1769882488 learned_addr learned my addr 172.21.15.86:0/1769882488 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:45.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.417+0000 7f13f9400640 1 -- 172.21.15.86:0/1769882488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14040d16b0 msgr2=0x7f1404159560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:45.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.417+0000 7f13f9400640 1 --2- 172.21.15.86:0/1769882488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14040d16b0 0x7f1404159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:45.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.417+0000 7f13f9400640 1 -- 172.21.15.86:0/1769882488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1404154eb0 msgr2=0x7f14041552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:45.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.417+0000 7f1402800640 1 --2- 172.21.15.86:0/1769882488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1404154eb0 0x7f14041552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:45.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.417+0000 7f13f9400640 1 --2- 172.21.15.86:0/1769882488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1404154eb0 0x7f14041552c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:45.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.417+0000 7f13f9400640 1 -- 172.21.15.86:0/1769882488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1404155800 con 0x7f14040d2d70 2024-04-25T14:11:45.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.417+0000 7f1402800640 1 --2- 172.21.15.86:0/1769882488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1404154eb0 0x7f14041552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:45.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.417+0000 7f13f9400640 1 --2- 172.21.15.86:0/1769882488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14040d2d70 0x7f140414c610 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f13ec00b3f0 tx=0x7f13ec00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:45.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.417+0000 7f13faa00640 1 -- 172.21.15.86:0/1769882488 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec015020 con 0x7f14040d2d70 2024-04-25T14:11:45.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.417+0000 7f13faa00640 1 -- 172.21.15.86:0/1769882488 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13ec011850 con 0x7f14040d2d70 2024-04-25T14:11:45.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.417+0000 7f1403200640 1 -- 172.21.15.86:0/1769882488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f140415a8c0 con 0x7f14040d2d70 2024-04-25T14:11:45.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.417+0000 7f13faa00640 1 -- 172.21.15.86:0/1769882488 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec011c40 con 0x7f14040d2d70 2024-04-25T14:11:45.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.417+0000 7f1403200640 1 -- 172.21.15.86:0/1769882488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f140415a300 con 0x7f14040d2d70 2024-04-25T14:11:45.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.417+0000 7f1403200640 1 -- 172.21.15.86:0/1769882488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1404159d90 con 0x7f14040d2d70 2024-04-25T14:11:45.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.421+0000 7f13faa00640 1 -- 172.21.15.86:0/1769882488 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7f13ec018020 con 0x7f14040d2d70 2024-04-25T14:11:45.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.421+0000 7f13faa00640 1 --2- 172.21.15.86:0/1769882488 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f13e006f550 0x7f13e0071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:45.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.421+0000 7f13faa00640 1 -- 172.21.15.86:0/1769882488 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f13ec08f220 con 0x7f14040d2d70 2024-04-25T14:11:45.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.421+0000 7f1401e00640 1 --2- 172.21.15.86:0/1769882488 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f13e006f550 0x7f13e0071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:45.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.421+0000 7f1401e00640 1 --2- 172.21.15.86:0/1769882488 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f13e006f550 0x7f13e0071a60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f13f40ad000 tx=0x7f13f4004740 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:45.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.425+0000 7f13faa00640 1 -- 172.21.15.86:0/1769882488 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1404155800 con 0x7f14040d2d70 2024-04-25T14:11:45.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.617+0000 7f1403200640 1 -- 172.21.15.86:0/1769882488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1404076b80 con 0x7f14040d2d70 2024-04-25T14:11:45.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.617+0000 7f13faa00640 1 -- 172.21.15.86:0/1769882488 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v103) ==== 53+0+73 (secure 0 0 0) 0x7f13ec05be50 con 0x7f14040d2d70 2024-04-25T14:11:45.615 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T14:11:45.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.621+0000 7f1403200640 1 -- 172.21.15.86:0/1769882488 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f13e006f550 msgr2=0x7f13e0071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:45.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.621+0000 7f1403200640 1 --2- 172.21.15.86:0/1769882488 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f13e006f550 0x7f13e0071a60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f13f40ad000 tx=0x7f13f4004740 comp rx=0 tx=0).stop 2024-04-25T14:11:45.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.621+0000 7f1403200640 1 -- 172.21.15.86:0/1769882488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14040d2d70 msgr2=0x7f140414c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:45.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.621+0000 7f1403200640 1 --2- 172.21.15.86:0/1769882488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14040d2d70 0x7f140414c610 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f13ec00b3f0 tx=0x7f13ec00b8b0 comp rx=0 tx=0).stop 2024-04-25T14:11:45.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.621+0000 7f1403200640 1 -- 172.21.15.86:0/1769882488 shutdown_connections 2024-04-25T14:11:45.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.621+0000 7f1403200640 1 --2- 172.21.15.86:0/1769882488 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f13e006f550 0x7f13e0071a60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:45.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.621+0000 7f1403200640 1 --2- 172.21.15.86:0/1769882488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1404154eb0 0x7f14041552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:45.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.621+0000 7f1403200640 1 --2- 172.21.15.86:0/1769882488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14040d2d70 0x7f140414c610 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:45.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.621+0000 7f1403200640 1 --2- 172.21.15.86:0/1769882488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14040d16b0 0x7f1404159560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:45.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.621+0000 7f1403200640 1 -- 172.21.15.86:0/1769882488 >> 172.21.15.86:0/1769882488 conn(0x7f14040cfea0 msgr2=0x7f1404070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:45.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.621+0000 7f1403200640 1 -- 172.21.15.86:0/1769882488 shutdown_connections 2024-04-25T14:11:45.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:45.621+0000 7f1403200640 1 -- 172.21.15.86:0/1769882488 wait complete. 2024-04-25T14:11:45.636 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T14:11:45.636 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T14:11:45.637 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2024-04-25T14:11:45.637 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T14:11:45.637 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T14:11:45.637 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T14:11:45.637 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T14:11:45.637 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T14:11:45.637 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:11:45.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:11:45.637 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2024-04-25T14:11:45.648 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:11:45.648 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:11:45.648 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:11:45.648 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:11:45.648 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2024-04-25T14:11:45.648 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:11:45.648 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:11:45.648 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2024-04-25T14:11:45.648 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f0 2024-04-25T14:11:45.648 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:11:45.648 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2024-04-25T14:11:45.649 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f1 2024-04-25T14:11:45.649 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:11:45.649 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2024-04-25T14:11:45.649 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T14:11:45.649 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2024-04-25T14:11:45.649 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T14:11:45.649 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 41952sec preferred_lft 41952sec 2024-04-25T14:11:45.649 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::ec4:7aff:fe88:6ded/64 scope link 2024-04-25T14:11:45.649 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:11:45.649 INFO:teuthology.orchestra.run.smithi155.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T14:11:45.649 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 02:42:b1:ff:60:50 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:11:45.649 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T14:11:45.649 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:11:45.649 INFO:teuthology.orchestra.run.smithi155.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T14:11:45.650 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T14:11:45.650 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T14:11:45.650 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:11:45.650 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T14:11:45.650 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:11:45.650 INFO:teuthology.orchestra.run.smithi155.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:11:45.650 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T14:11:45.650 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T14:11:45.650 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:11:45.650 INFO:teuthology.orchestra.run.smithi155.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:11:45.650 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T14:11:45.650 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T14:11:45.650 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:11:45.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:11:45.651 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T14:11:45.715 INFO:teuthology.orchestra.run.smithi155.stdout:1 2024-04-25T14:11:45.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:11:45.718 DEBUG:teuthology.orchestra.run.smithi155:> route 2024-04-25T14:11:45.772 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2024-04-25T14:11:45.773 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T14:11:45.773 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T14:11:45.773 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T14:11:45.773 INFO:teuthology.orchestra.run.smithi155.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T14:11:45.773 INFO:teuthology.orchestra.run.smithi155.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T14:11:45.773 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T14:11:45.773 INFO:teuthology.orchestra.run.smithi155.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:11:45.773 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:11:45.773 INFO:teuthology.orchestra.run.smithi155.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:11:45.773 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T14:11:45.774 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:11:45.774 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-04-25T14:11:45.774 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T14:11:45.774 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T14:11:45.774 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T14:11:45.774 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-04-25T14:11:45.869 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:11:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:11:45.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:11:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:11:45.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:11:45.918 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2024-04-25T14:11:45.927 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T14:11:45.927 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T14:11:45.928 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:11:45.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:11:45.928 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.0 up 2024-04-25T14:11:45.995 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:11:45.995 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T14:11:46.047 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:11:46.047 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:11:46.047 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T14:11:46.099 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T14:11:46.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:11:46.099 DEBUG:teuthology.orchestra.run.smithi155:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:11:46.199 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:11:46.199 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T14:11:46.199 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:11:46.217 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:11:46.217 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T14:11:46.217 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:11:46.217 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:11:46.217 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:11:46.227 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:11:46.228 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T14:11:46.228 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:11:46.228 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:11:46.228 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:11:46.228 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790" 2024-04-25T14:11:46.228 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790:/ 2024-04-25T14:11:46.228 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb" will pass to kernel 2024-04-25T14:11:46.232 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T14:11:46.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:11:46.232 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T14:11:46.251 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:11:46.338 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:11:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:11:46.341 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:11:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:11:46.347 INFO:teuthology.orchestra.run.smithi155.stdout:client9292 2024-04-25T14:11:46.347 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:11:46.348 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:11:46.357 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:11:46.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.509+0000 7f9f39400640 1 Processor -- start 2024-04-25T14:11:46.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.509+0000 7f9f39400640 1 -- start start 2024-04-25T14:11:46.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.509+0000 7f9f39400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9f3415fc70 0x7f9f34070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:46.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.509+0000 7f9f39400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f34071360 0x7f9f34163810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:46.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.509+0000 7f9f39400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f3406f0a0 con 0x7f9f34163d50 2024-04-25T14:11:46.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.509+0000 7f9f39400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9f3406e7f0 con 0x7f9f3415fc70 2024-04-25T14:11:46.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.509+0000 7f9f39400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9f34071770 con 0x7f9f34071360 2024-04-25T14:11:46.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.509+0000 7f9f2be00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f34071360 0x7f9f34163810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:46.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.509+0000 7f9f32800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9f3415fc70 0x7f9f34070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:46.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.509+0000 7f9f2be00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f34071360 0x7f9f34163810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:35908/0 (socket says 172.21.15.86:35908) 2024-04-25T14:11:46.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.509+0000 7f9f2be00640 1 -- 172.21.15.86:0/2625730540 learned_addr learned my addr 172.21.15.86:0/2625730540 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:46.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.509+0000 7f9f33200640 1 --1- 172.21.15.86:0/2625730540 >> v1:172.21.15.155:6789/0 conn(0x7f9f34163d50 0x7f9f340722f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:33936/0 (socket says 172.21.15.86:33936) 2024-04-25T14:11:46.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.513+0000 7f9f2be00640 1 -- 172.21.15.86:0/2625730540 >> v2:172.21.15.155:3300/0 conn(0x7f9f3415fc70 msgr2=0x7f9f34070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:46.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.513+0000 7f9f2be00640 1 --2- 172.21.15.86:0/2625730540 >> v2:172.21.15.155:3300/0 conn(0x7f9f3415fc70 0x7f9f34070e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.513+0000 7f9f2be00640 1 -- 172.21.15.86:0/2625730540 >> v1:172.21.15.155:6789/0 conn(0x7f9f34163d50 legacy=0x7f9f340722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:46.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.513+0000 7f9f2be00640 1 -- 172.21.15.86:0/2625730540 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f34164160 con 0x7f9f34071360 2024-04-25T14:11:46.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.513+0000 7f9f2be00640 1 --2- 172.21.15.86:0/2625730540 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f34071360 0x7f9f34163810 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009870 tx=0x7f9f1c01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdd1376d6dd9de0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:46.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.513+0000 7f9f31e00640 1 -- 172.21.15.86:0/2625730540 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c0ab680 con 0x7f9f34071360 2024-04-25T14:11:46.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.513+0000 7f9f31e00640 1 -- 172.21.15.86:0/2625730540 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f1c0b0070 con 0x7f9f34071360 2024-04-25T14:11:46.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.513+0000 7f9f31e00640 1 -- 172.21.15.86:0/2625730540 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c0aba40 con 0x7f9f34071360 2024-04-25T14:11:46.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.517+0000 7f9f39400640 1 -- 172.21.15.86:0/2625730540 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f34071360 msgr2=0x7f9f34163810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:46.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.517+0000 7f9f39400640 1 --2- 172.21.15.86:0/2625730540 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f34071360 0x7f9f34163810 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009870 tx=0x7f9f1c01c560 comp rx=0 tx=0).stop 2024-04-25T14:11:46.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.517+0000 7f9f39400640 1 -- 172.21.15.86:0/2625730540 shutdown_connections 2024-04-25T14:11:46.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.517+0000 7f9f39400640 1 --2- 172.21.15.86:0/2625730540 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f34071360 0x7f9f34163810 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.517+0000 7f9f39400640 1 --2- 172.21.15.86:0/2625730540 >> v2:172.21.15.155:3300/0 conn(0x7f9f3415fc70 0x7f9f34070e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.517+0000 7f9f39400640 1 -- 172.21.15.86:0/2625730540 >> 172.21.15.86:0/2625730540 conn(0x7f9f340cffc0 msgr2=0x7f9f340d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:46.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.517+0000 7f9f39400640 1 -- 172.21.15.86:0/2625730540 shutdown_connections 2024-04-25T14:11:46.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.517+0000 7f9f39400640 1 -- 172.21.15.86:0/2625730540 wait complete. 2024-04-25T14:11:46.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.517+0000 7f9f39400640 1 Processor -- start 2024-04-25T14:11:46.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.517+0000 7f9f39400640 1 -- start start 2024-04-25T14:11:46.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.521+0000 7f9f39400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f34071360 0x7f9f34152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:46.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.521+0000 7f9f39400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f3415fc70 0x7f9f340b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:46.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.521+0000 7f9f39400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f34163d50 0x7f9f340b6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:46.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.521+0000 7f9f39400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9f34160080 con 0x7f9f34071360 2024-04-25T14:11:46.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.521+0000 7f9f39400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3406e480 con 0x7f9f3415fc70 2024-04-25T14:11:46.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.521+0000 7f9f39400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9f3416cb70 con 0x7f9f34163d50 2024-04-25T14:11:46.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.521+0000 7f9f32800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f34071360 0x7f9f34152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:46.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.521+0000 7f9f32800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f34071360 0x7f9f34152bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53036/0 (socket says 172.21.15.86:53036) 2024-04-25T14:11:46.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.521+0000 7f9f32800640 1 -- 172.21.15.86:0/3021987567 learned_addr learned my addr 172.21.15.86:0/3021987567 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:46.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.521+0000 7f9f2be00640 1 --2- 172.21.15.86:0/3021987567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f3415fc70 0x7f9f340b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:46.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.521+0000 7f9f33200640 1 --2- 172.21.15.86:0/3021987567 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f34163d50 0x7f9f340b6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:46.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.521+0000 7f9f32800640 1 -- 172.21.15.86:0/3021987567 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f34163d50 msgr2=0x7f9f340b6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:46.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.521+0000 7f9f32800640 1 --2- 172.21.15.86:0/3021987567 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f34163d50 0x7f9f340b6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.521+0000 7f9f32800640 1 -- 172.21.15.86:0/3021987567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f3415fc70 msgr2=0x7f9f340b6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:46.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.521+0000 7f9f32800640 1 --2- 172.21.15.86:0/3021987567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f3415fc70 0x7f9f340b6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.521+0000 7f9f32800640 1 -- 172.21.15.86:0/3021987567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f34153700 con 0x7f9f34071360 2024-04-25T14:11:46.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.521+0000 7f9f2be00640 1 --2- 172.21.15.86:0/3021987567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f3415fc70 0x7f9f340b6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:46.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.521+0000 7f9f32800640 1 --2- 172.21.15.86:0/3021987567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f34071360 0x7f9f34152bc0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400e870 tx=0x7f9f2400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:46.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.521+0000 7f9f2aa00640 1 -- 172.21.15.86:0/3021987567 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24017070 con 0x7f9f34071360 2024-04-25T14:11:46.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.521+0000 7f9f2aa00640 1 -- 172.21.15.86:0/3021987567 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f2400c940 con 0x7f9f34071360 2024-04-25T14:11:46.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.521+0000 7f9f2aa00640 1 -- 172.21.15.86:0/3021987567 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2400ccf0 con 0x7f9f34071360 2024-04-25T14:11:46.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.521+0000 7f9f39400640 1 -- 172.21.15.86:0/3021987567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f340b85c0 con 0x7f9f34071360 2024-04-25T14:11:46.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.521+0000 7f9f39400640 1 -- 172.21.15.86:0/3021987567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f340b8b80 con 0x7f9f34071360 2024-04-25T14:11:46.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.521+0000 7f9f39400640 1 -- 172.21.15.86:0/3021987567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f340720e0 con 0x7f9f34071360 2024-04-25T14:11:46.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.525+0000 7f9f2aa00640 1 -- 172.21.15.86:0/3021987567 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7f9f2401a070 con 0x7f9f34071360 2024-04-25T14:11:46.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.525+0000 7f9f2aa00640 1 --2- 172.21.15.86:0/3021987567 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f9f1006f520 0x7f9f10071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:46.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.525+0000 7f9f2aa00640 1 -- 172.21.15.86:0/3021987567 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f9f2408f840 con 0x7f9f34071360 2024-04-25T14:11:46.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.525+0000 7f9f2be00640 1 --2- 172.21.15.86:0/3021987567 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f9f1006f520 0x7f9f10071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:46.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.525+0000 7f9f2be00640 1 --2- 172.21.15.86:0/3021987567 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f9f1006f520 0x7f9f10071a30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c0af000 tx=0x7f9f1c0a7000 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:46.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.529+0000 7f9f2aa00640 1 -- 172.21.15.86:0/3021987567 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9f2405c480 con 0x7f9f34071360 2024-04-25T14:11:46.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.717+0000 7f9f39400640 1 -- 172.21.15.86:0/3021987567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f34076bf0 con 0x7f9f34071360 2024-04-25T14:11:46.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.717+0000 7f9f2aa00640 1 -- 172.21.15.86:0/3021987567 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5187 (secure 0 0 0) 0x7f9f2405bd80 con 0x7f9f34071360 2024-04-25T14:11:46.716 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:46.716 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":103,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/1828317917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1828317917},{"type":"v1","addr":"172.21.15.155:6834","nonce":1828317917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":9124,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1389714390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1389714390},{"type":"v1","addr":"172.21.15.86:6837","nonce":1389714390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":9136,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/91793677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":91793677},{"type":"v1","addr":"172.21.15.155:6835","nonce":91793677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:11:32.981538+0000","modified":"2024-04-25T14:11:35.996552+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9546},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9546":{"gid":9546,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/2571655765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2571655765},{"type":"v1","addr":"172.21.15.86:6835","nonce":2571655765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9546,"qdb_cluster":[9546]},"id":10}]} 2024-04-25T14:11:46.716 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 103 2024-04-25T14:11:46.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.721+0000 7f9f39400640 1 -- 172.21.15.86:0/3021987567 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f9f1006f520 msgr2=0x7f9f10071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:46.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.721+0000 7f9f39400640 1 --2- 172.21.15.86:0/3021987567 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f9f1006f520 0x7f9f10071a30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c0af000 tx=0x7f9f1c0a7000 comp rx=0 tx=0).stop 2024-04-25T14:11:46.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.721+0000 7f9f39400640 1 -- 172.21.15.86:0/3021987567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f34071360 msgr2=0x7f9f34152bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:46.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.721+0000 7f9f39400640 1 --2- 172.21.15.86:0/3021987567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f34071360 0x7f9f34152bc0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400e870 tx=0x7f9f2400ec70 comp rx=0 tx=0).stop 2024-04-25T14:11:46.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.721+0000 7f9f39400640 1 -- 172.21.15.86:0/3021987567 shutdown_connections 2024-04-25T14:11:46.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.721+0000 7f9f39400640 1 --2- 172.21.15.86:0/3021987567 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f9f1006f520 0x7f9f10071a30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.721+0000 7f9f39400640 1 --2- 172.21.15.86:0/3021987567 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f34163d50 0x7f9f340b6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.721+0000 7f9f39400640 1 --2- 172.21.15.86:0/3021987567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f3415fc70 0x7f9f340b6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.721+0000 7f9f39400640 1 --2- 172.21.15.86:0/3021987567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f34071360 0x7f9f34152bc0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.721+0000 7f9f39400640 1 -- 172.21.15.86:0/3021987567 >> 172.21.15.86:0/3021987567 conn(0x7f9f340cffc0 msgr2=0x7f9f34161830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:46.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.721+0000 7f9f39400640 1 -- 172.21.15.86:0/3021987567 shutdown_connections 2024-04-25T14:11:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.721+0000 7f9f39400640 1 -- 172.21.15.86:0/3021987567 wait complete. 2024-04-25T14:11:46.737 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable stats 2024-04-25T14:11:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.897+0000 7f4f12800640 1 Processor -- start 2024-04-25T14:11:46.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.897+0000 7f4f12800640 1 -- start start 2024-04-25T14:11:46.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.897+0000 7f4f12800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f0c06fd50 0x7f4f0c070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:46.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.897+0000 7f4f12800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4f0c15feb0 0x7f4f0c1602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:46.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.897+0000 7f4f12800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f0c0b0a60 con 0x7f4f0c15c2a0 2024-04-25T14:11:46.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.897+0000 7f4f12800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4f0c0ae840 con 0x7f4f0c15feb0 2024-04-25T14:11:46.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.897+0000 7f4f12800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4f0c06ef90 con 0x7f4f0c06fd50 2024-04-25T14:11:46.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.897+0000 7f4f0be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4f0c15feb0 0x7f4f0c1602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:46.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.897+0000 7f4f0b400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f4f0c15c2a0 0x7f4f0c0ae130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44052/0 (socket says 172.21.15.86:44052) 2024-04-25T14:11:46.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.897+0000 7f4f0b400640 1 -- 172.21.15.86:0/4196516436 learned_addr learned my addr 172.21.15.86:0/4196516436 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:46.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.897+0000 7f4f02a00640 1 --2- 172.21.15.86:0/4196516436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f0c06fd50 0x7f4f0c070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:46.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.897+0000 7f4f0be00640 1 -- 172.21.15.86:0/4196516436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f0c06fd50 msgr2=0x7f4f0c070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:46.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.897+0000 7f4f0be00640 1 --2- 172.21.15.86:0/4196516436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f0c06fd50 0x7f4f0c070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.897+0000 7f4f0be00640 1 -- 172.21.15.86:0/4196516436 >> v1:172.21.15.86:6789/0 conn(0x7f4f0c15c2a0 legacy=0x7f4f0c0ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:46.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.897+0000 7f4f0be00640 1 -- 172.21.15.86:0/4196516436 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0c06f270 con 0x7f4f0c15feb0 2024-04-25T14:11:46.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.897+0000 7f4f0aa00640 1 -- 172.21.15.86:0/4196516436 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3871746484 0 0) 0x7f4f0c0b0a60 con 0x7f4f0c15c2a0 2024-04-25T14:11:46.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.901+0000 7f4f0be00640 1 --2- 172.21.15.86:0/4196516436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c15feb0 0x7f4f0c1602c0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400aea0 tx=0x7f4f0401d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ee022ccfec18a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:46.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.901+0000 7f4f0aa00640 1 -- 172.21.15.86:0/4196516436 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f040a67f0 con 0x7f4f0c15feb0 2024-04-25T14:11:46.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.901+0000 7f4f0aa00640 1 -- 172.21.15.86:0/4196516436 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f040a6990 con 0x7f4f0c15feb0 2024-04-25T14:11:46.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.901+0000 7f4f0aa00640 1 -- 172.21.15.86:0/4196516436 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f040a6d60 con 0x7f4f0c15feb0 2024-04-25T14:11:46.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.901+0000 7f4f12800640 1 -- 172.21.15.86:0/4196516436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c15feb0 msgr2=0x7f4f0c1602c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:46.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.901+0000 7f4f12800640 1 --2- 172.21.15.86:0/4196516436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c15feb0 0x7f4f0c1602c0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400aea0 tx=0x7f4f0401d9b0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.901+0000 7f4f12800640 1 -- 172.21.15.86:0/4196516436 shutdown_connections 2024-04-25T14:11:46.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.901+0000 7f4f12800640 1 --2- 172.21.15.86:0/4196516436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c15feb0 0x7f4f0c1602c0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.901+0000 7f4f12800640 1 --2- 172.21.15.86:0/4196516436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f0c06fd50 0x7f4f0c070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.901+0000 7f4f12800640 1 -- 172.21.15.86:0/4196516436 >> 172.21.15.86:0/4196516436 conn(0x7f4f0c0b4650 msgr2=0x7f4f0c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:46.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.901+0000 7f4f12800640 1 -- 172.21.15.86:0/4196516436 shutdown_connections 2024-04-25T14:11:46.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.901+0000 7f4f12800640 1 -- 172.21.15.86:0/4196516436 wait complete. 2024-04-25T14:11:46.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.905+0000 7f4f12800640 1 Processor -- start 2024-04-25T14:11:46.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.905+0000 7f4f12800640 1 -- start start 2024-04-25T14:11:46.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.905+0000 7f4f12800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f0c06fd50 0x7f4f0c158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:46.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.905+0000 7f4f12800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c15c2a0 0x7f4f0c0b3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:46.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.905+0000 7f4f12800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f0c15feb0 0x7f4f0c0b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:46.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.905+0000 7f4f12800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c15c6b0 con 0x7f4f0c15c2a0 2024-04-25T14:11:46.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.905+0000 7f4f12800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c0d0fc0 con 0x7f4f0c15feb0 2024-04-25T14:11:46.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.905+0000 7f4f12800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4f0c15c830 con 0x7f4f0c06fd50 2024-04-25T14:11:46.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.905+0000 7f4f02a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c15c2a0 0x7f4f0c0b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:46.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.905+0000 7f4f02a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c15c2a0 0x7f4f0c0b3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53060/0 (socket says 172.21.15.86:53060) 2024-04-25T14:11:46.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.905+0000 7f4f02a00640 1 -- 172.21.15.86:0/1499182496 learned_addr learned my addr 172.21.15.86:0/1499182496 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:46.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.905+0000 7f4f0b400640 1 --2- 172.21.15.86:0/1499182496 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f0c06fd50 0x7f4f0c158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:46.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.905+0000 7f4f02a00640 1 -- 172.21.15.86:0/1499182496 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f0c06fd50 msgr2=0x7f4f0c158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:46.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.905+0000 7f4f0be00640 1 --2- 172.21.15.86:0/1499182496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f0c15feb0 0x7f4f0c0b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:46.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.905+0000 7f4f02a00640 1 --2- 172.21.15.86:0/1499182496 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f0c06fd50 0x7f4f0c158a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.905+0000 7f4f02a00640 1 -- 172.21.15.86:0/1499182496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f0c15feb0 msgr2=0x7f4f0c0b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:46.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.905+0000 7f4f02a00640 1 --2- 172.21.15.86:0/1499182496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f0c15feb0 0x7f4f0c0b0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.905+0000 7f4f02a00640 1 -- 172.21.15.86:0/1499182496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0c15f810 con 0x7f4f0c15c2a0 2024-04-25T14:11:46.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.905+0000 7f4f0be00640 1 --2- 172.21.15.86:0/1499182496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f0c15feb0 0x7f4f0c0b0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:46.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.905+0000 7f4f0b400640 1 --2- 172.21.15.86:0/1499182496 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f0c06fd50 0x7f4f0c158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:46.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.905+0000 7f4f02a00640 1 --2- 172.21.15.86:0/1499182496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c15c2a0 0x7f4f0c0b3310 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00e870 tx=0x7f4efc00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:46.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.905+0000 7f4f03e00640 1 -- 172.21.15.86:0/1499182496 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4efc017070 con 0x7f4f0c15c2a0 2024-04-25T14:11:46.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.905+0000 7f4f03e00640 1 -- 172.21.15.86:0/1499182496 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4efc00c940 con 0x7f4f0c15c2a0 2024-04-25T14:11:46.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.905+0000 7f4f03e00640 1 -- 172.21.15.86:0/1499182496 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4efc00ccf0 con 0x7f4f0c15c2a0 2024-04-25T14:11:46.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.905+0000 7f4f12800640 1 -- 172.21.15.86:0/1499182496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f0c0c8920 con 0x7f4f0c15c2a0 2024-04-25T14:11:46.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.905+0000 7f4f12800640 1 -- 172.21.15.86:0/1499182496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f0c0c8ee0 con 0x7f4f0c15c2a0 2024-04-25T14:11:46.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.909+0000 7f4f12800640 1 -- 172.21.15.86:0/1499182496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f0c069d00 con 0x7f4f0c15c2a0 2024-04-25T14:11:46.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.909+0000 7f4f03e00640 1 -- 172.21.15.86:0/1499182496 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 66) ==== 93309+0+0 (secure 0 0 0) 0x7f4efc01a070 con 0x7f4f0c15c2a0 2024-04-25T14:11:46.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.909+0000 7f4f03e00640 1 --2- 172.21.15.86:0/1499182496 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f4ee806f520 0x7f4ee8071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:46.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.909+0000 7f4f03e00640 1 -- 172.21.15.86:0/1499182496 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f4efc08f560 con 0x7f4f0c15c2a0 2024-04-25T14:11:46.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.909+0000 7f4f0b400640 1 --2- 172.21.15.86:0/1499182496 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f4ee806f520 0x7f4ee8071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:46.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.909+0000 7f4f0b400640 1 --2- 172.21.15.86:0/1499182496 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f4ee806f520 0x7f4ee8071a30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4007fd0 tx=0x7f4ef4009040 comp rx=0 tx=0).ready entity=mgr.9282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:46.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:46.913+0000 7f4f03e00640 1 -- 172.21.15.86:0/1499182496 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4efc05c110 con 0x7f4f0c15c2a0 2024-04-25T14:11:47.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:47.105+0000 7f4f12800640 1 -- 172.21.15.86:0/1499182496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "stats"} v 0) -- 0x7f4f0c076bf0 con 0x7f4f0c15c2a0 2024-04-25T14:11:47.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:47.813+0000 7f4f03e00640 1 -- 172.21.15.86:0/1499182496 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "stats"}]=0 v67) ==== 84+0+0 (secure 0 0 0) 0x7f4efc05ba10 con 0x7f4f0c15c2a0 2024-04-25T14:11:47.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:47.813+0000 7f4f12800640 1 -- 172.21.15.86:0/1499182496 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f4ee806f520 msgr2=0x7f4ee8071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:47.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:47.813+0000 7f4f12800640 1 --2- 172.21.15.86:0/1499182496 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f4ee806f520 0x7f4ee8071a30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4007fd0 tx=0x7f4ef4009040 comp rx=0 tx=0).stop 2024-04-25T14:11:47.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:47.813+0000 7f4f12800640 1 -- 172.21.15.86:0/1499182496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c15c2a0 msgr2=0x7f4f0c0b3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:47.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:47.813+0000 7f4f12800640 1 --2- 172.21.15.86:0/1499182496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c15c2a0 0x7f4f0c0b3310 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00e870 tx=0x7f4efc00ec70 comp rx=0 tx=0).stop 2024-04-25T14:11:47.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:47.813+0000 7f4f12800640 1 -- 172.21.15.86:0/1499182496 shutdown_connections 2024-04-25T14:11:47.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:47.813+0000 7f4f12800640 1 --2- 172.21.15.86:0/1499182496 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f4ee806f520 0x7f4ee8071a30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:47.813+0000 7f4f12800640 1 --2- 172.21.15.86:0/1499182496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f0c15feb0 0x7f4f0c0b0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:47.813+0000 7f4f12800640 1 --2- 172.21.15.86:0/1499182496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c15c2a0 0x7f4f0c0b3310 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:47.813+0000 7f4f12800640 1 --2- 172.21.15.86:0/1499182496 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f0c06fd50 0x7f4f0c158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:47.813+0000 7f4f12800640 1 -- 172.21.15.86:0/1499182496 >> 172.21.15.86:0/1499182496 conn(0x7f4f0c0b4650 msgr2=0x7f4f0c0ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:47.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:47.817+0000 7f4f12800640 1 -- 172.21.15.86:0/1499182496 shutdown_connections 2024-04-25T14:11:47.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:47.817+0000 7f4f12800640 1 -- 172.21.15.86:0/1499182496 wait complete. 2024-04-25T14:11:47.843 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:11:47.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.001+0000 7f7194000640 1 Processor -- start 2024-04-25T14:11:47.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.001+0000 7f7194000640 1 -- start start 2024-04-25T14:11:48.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.005+0000 7f7194000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f718c163980 0x7f718c163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:48.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.005+0000 7f7194000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f718c06e340 0x7f718c06e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:48.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.005+0000 7f7194000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f718c1648e0 con 0x7f718c072630 2024-04-25T14:11:48.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.005+0000 7f7194000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f718c0d1520 con 0x7f718c163980 2024-04-25T14:11:48.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.005+0000 7f7194000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f718c0d23e0 con 0x7f718c06e340 2024-04-25T14:11:48.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.005+0000 7f7191600640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f718c072630 0x7f718c072a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:33946/0 (socket says 172.21.15.86:33946) 2024-04-25T14:11:48.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.005+0000 7f7192000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f718c06e340 0x7f718c06e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:48.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.005+0000 7f7191600640 1 -- 172.21.15.86:0/4214135945 learned_addr learned my addr 172.21.15.86:0/4214135945 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:48.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.005+0000 7f7190c00640 1 --2- 172.21.15.86:0/4214135945 >> v2:172.21.15.155:3300/0 conn(0x7f718c163980 0x7f718c163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:48.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.005+0000 7f7192000640 1 -- 172.21.15.86:0/4214135945 >> v2:172.21.15.155:3300/0 conn(0x7f718c163980 msgr2=0x7f718c163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:48.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.005+0000 7f7192000640 1 --2- 172.21.15.86:0/4214135945 >> v2:172.21.15.155:3300/0 conn(0x7f718c163980 0x7f718c163d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.005+0000 7f7192000640 1 -- 172.21.15.86:0/4214135945 >> v1:172.21.15.155:6789/0 conn(0x7f718c072630 legacy=0x7f718c072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:48.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.005+0000 7f7192000640 1 -- 172.21.15.86:0/4214135945 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718c164600 con 0x7f718c06e340 2024-04-25T14:11:48.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.005+0000 7f718be00640 1 -- 172.21.15.86:0/4214135945 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4035472657 0 0) 0x7f718c1648e0 con 0x7f718c072630 2024-04-25T14:11:48.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.009+0000 7f7192000640 1 --2- 172.21.15.86:0/4214135945 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f718c06e340 0x7f718c06e750 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7184009d90 tx=0x7f718401c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ed7eae3279d1406 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:48.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.009+0000 7f718be00640 1 -- 172.21.15.86:0/4214135945 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71840b2070 con 0x7f718c06e340 2024-04-25T14:11:48.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.009+0000 7f718be00640 1 -- 172.21.15.86:0/4214135945 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71840a5710 con 0x7f718c06e340 2024-04-25T14:11:48.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.009+0000 7f718be00640 1 -- 172.21.15.86:0/4214135945 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71840a5ae0 con 0x7f718c06e340 2024-04-25T14:11:48.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.009+0000 7f7194000640 1 -- 172.21.15.86:0/4214135945 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f718c06e340 msgr2=0x7f718c06e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:48.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.009+0000 7f7194000640 1 --2- 172.21.15.86:0/4214135945 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f718c06e340 0x7f718c06e750 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7184009d90 tx=0x7f718401c850 comp rx=0 tx=0).stop 2024-04-25T14:11:48.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.009+0000 7f7194000640 1 -- 172.21.15.86:0/4214135945 shutdown_connections 2024-04-25T14:11:48.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.009+0000 7f7194000640 1 --2- 172.21.15.86:0/4214135945 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f718c06e340 0x7f718c06e750 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.009+0000 7f7194000640 1 --2- 172.21.15.86:0/4214135945 >> v2:172.21.15.155:3300/0 conn(0x7f718c163980 0x7f718c163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.009+0000 7f7194000640 1 -- 172.21.15.86:0/4214135945 >> 172.21.15.86:0/4214135945 conn(0x7f718c0d3580 msgr2=0x7f718c0d3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:48.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.009+0000 7f7194000640 1 -- 172.21.15.86:0/4214135945 shutdown_connections 2024-04-25T14:11:48.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.009+0000 7f7194000640 1 -- 172.21.15.86:0/4214135945 wait complete. 2024-04-25T14:11:48.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.009+0000 7f7194000640 1 Processor -- start 2024-04-25T14:11:48.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.013+0000 7f7194000640 1 -- start start 2024-04-25T14:11:48.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.013+0000 7f7194000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f718c06e340 0x7f718c14f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:48.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.013+0000 7f7194000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f718c072630 0x7f718c150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:48.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.013+0000 7f7194000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f718c163980 0x7f718c15e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:48.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.013+0000 7f7194000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f718c16d530 con 0x7f718c072630 2024-04-25T14:11:48.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.013+0000 7f7194000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f718c06f0a0 con 0x7f718c06e340 2024-04-25T14:11:48.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.013+0000 7f7194000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f718c16d6b0 con 0x7f718c163980 2024-04-25T14:11:48.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.013+0000 7f7190c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f718c072630 0x7f718c150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:48.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.013+0000 7f7190c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f718c072630 0x7f718c150b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53070/0 (socket says 172.21.15.86:53070) 2024-04-25T14:11:48.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.013+0000 7f7190c00640 1 -- 172.21.15.86:0/3163188731 learned_addr learned my addr 172.21.15.86:0/3163188731 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:48.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.013+0000 7f7192000640 1 --2- 172.21.15.86:0/3163188731 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f718c163980 0x7f718c15e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:48.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.013+0000 7f7191600640 1 --2- 172.21.15.86:0/3163188731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f718c06e340 0x7f718c14f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:48.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.013+0000 7f7190c00640 1 -- 172.21.15.86:0/3163188731 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f718c163980 msgr2=0x7f718c15e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:48.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.013+0000 7f7190c00640 1 --2- 172.21.15.86:0/3163188731 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f718c163980 0x7f718c15e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.013+0000 7f7190c00640 1 -- 172.21.15.86:0/3163188731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f718c06e340 msgr2=0x7f718c14f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:48.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.013+0000 7f7190c00640 1 --2- 172.21.15.86:0/3163188731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f718c06e340 0x7f718c14f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.013+0000 7f7190c00640 1 -- 172.21.15.86:0/3163188731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718c1f9a60 con 0x7f718c072630 2024-04-25T14:11:48.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.013+0000 7f7190c00640 1 --2- 172.21.15.86:0/3163188731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f718c072630 0x7f718c150b10 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f717800e8a0 tx=0x7f717800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:48.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.013+0000 7f7189600640 1 -- 172.21.15.86:0/3163188731 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7178017070 con 0x7f718c072630 2024-04-25T14:11:48.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.013+0000 7f7189600640 1 -- 172.21.15.86:0/3163188731 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f717800c8e0 con 0x7f718c072630 2024-04-25T14:11:48.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.013+0000 7f7189600640 1 -- 172.21.15.86:0/3163188731 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717800cc90 con 0x7f718c072630 2024-04-25T14:11:48.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.013+0000 7f7194000640 1 -- 172.21.15.86:0/3163188731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f718c1f9d40 con 0x7f718c072630 2024-04-25T14:11:48.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.013+0000 7f7194000640 1 -- 172.21.15.86:0/3163188731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f718c1fa300 con 0x7f718c072630 2024-04-25T14:11:48.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.013+0000 7f7194000640 1 -- 172.21.15.86:0/3163188731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f718c072170 con 0x7f718c072630 2024-04-25T14:11:48.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.017+0000 7f7189600640 1 -- 172.21.15.86:0/3163188731 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 67) ==== 93318+0+0 (secure 0 0 0) 0x7f717801a070 con 0x7f718c072630 2024-04-25T14:11:48.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.017+0000 7f7189600640 1 --2- 172.21.15.86:0/3163188731 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f716c06f570 0x7f716c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:48.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.017+0000 7f7189600640 1 -- 172.21.15.86:0/3163188731 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6101+0+0 (secure 0 0 0) 0x7f717808f850 con 0x7f718c072630 2024-04-25T14:11:48.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.017+0000 7f7191600640 1 -- 172.21.15.86:0/3163188731 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f716c06f570 msgr2=0x7f716c071a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/264890479 2024-04-25T14:11:48.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.017+0000 7f7191600640 1 --2- 172.21.15.86:0/3163188731 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f716c06f570 0x7f716c071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:11:48.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.021+0000 7f7189600640 1 -- 172.21.15.86:0/3163188731 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f717805bdb0 con 0x7f718c072630 2024-04-25T14:11:48.066 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:48.066+0000 7f87a7f7e180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:11:48.087 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:48.089+0000 7fc5db136180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:11:48.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.205+0000 7f7194000640 1 -- 172.21.15.86:0/3163188731 --> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f718c068eb0 con 0x7f716c06f570 2024-04-25T14:11:48.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.221+0000 7f7191600640 1 -- 172.21.15.86:0/3163188731 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f716c06f570 msgr2=0x7f716c071a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/264890479 2024-04-25T14:11:48.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.221+0000 7f7191600640 1 --2- 172.21.15.86:0/3163188731 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f716c06f570 0x7f716c071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:11:48.482 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:48.482+0000 7f87a7f7e180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:11:48.510 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:48.513+0000 7fc5db136180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:11:48.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.625+0000 7f7191600640 1 -- 172.21.15.86:0/3163188731 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f716c06f570 msgr2=0x7f716c071a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/264890479 2024-04-25T14:11:48.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:48.625+0000 7f7191600640 1 --2- 172.21.15.86:0/3163188731 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f716c06f570 0x7f716c071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T14:11:48.635 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:48.634+0000 7f87a7f7e180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:11:48.662 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:48.665+0000 7fc5db136180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:11:49.105 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:49.106+0000 7f87a7f7e180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:11:49.134 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:49.137+0000 7fc5db136180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:11:49.163 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:49.162+0000 7f87a7f7e180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:11:49.193 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:49.197+0000 7fc5db136180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:11:49.218 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:49.218+0000 7f87a7f7e180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:11:49.255 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:49.257+0000 7fc5db136180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:11:49.304 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:49.306+0000 7f87a7f7e180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:11:49.350 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:49.353+0000 7fc5db136180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:11:49.360 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:49.362+0000 7f87a7f7e180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:11:49.407 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:49.409+0000 7fc5db136180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:11:49.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:49.429+0000 7f7191600640 1 -- 172.21.15.86:0/3163188731 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f716c06f570 msgr2=0x7f716c071a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/264890479 2024-04-25T14:11:49.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:49.429+0000 7f7191600640 1 --2- 172.21.15.86:0/3163188731 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f716c06f570 0x7f716c071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T14:11:49.444 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:49.446+0000 7f87a7f7e180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:11:49.492 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:49.497+0000 7fc5db136180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:11:49.503 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:49.502+0000 7f87a7f7e180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:11:49.552 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:49.557+0000 7fc5db136180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:11:49.660 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:49.662+0000 7f87a7f7e180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:11:49.710 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:49.713+0000 7fc5db136180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:11:49.719 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:49.718+0000 7f87a7f7e180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:11:49.769 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:49.773+0000 7fc5db136180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:11:49.933 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:11:49.944 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:11:49.960 INFO:tasks.ceph.mgr.x.smithi155.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:11:49.960 INFO:tasks.ceph.mgr.x.smithi155.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:11:49.960 INFO:tasks.ceph.mgr.x.smithi155.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:11:49.969 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:49.970+0000 7f87a7f7e180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:11:50.010 INFO:tasks.ceph.mgr.y.smithi086.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:11:50.011 INFO:tasks.ceph.mgr.y.smithi086.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:11:50.011 INFO:tasks.ceph.mgr.y.smithi086.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:11:50.019 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:50.021+0000 7fc5db136180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:11:50.118 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:50.118+0000 7f87a7f7e180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:11:50.169 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:50.173+0000 7fc5db136180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:11:50.243 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:50.242+0000 7f87a7f7e180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:11:50.295 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:50.297+0000 7fc5db136180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:11:50.314 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:50.314+0000 7f87a7f7e180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:11:50.366 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:50.369+0000 7fc5db136180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:11:50.461 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:50.462+0000 7f87a7f7e180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:11:50.514 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:50.517+0000 7fc5db136180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:11:51.002 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:51.002+0000 7f87a7f7e180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:11:51.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:51.033+0000 7f7191600640 1 -- 172.21.15.86:0/3163188731 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f716c06f570 msgr2=0x7f716c071a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/264890479 2024-04-25T14:11:51.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:51.033+0000 7f7191600640 1 --2- 172.21.15.86:0/3163188731 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f716c06f570 0x7f716c071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T14:11:51.059 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:51.061+0000 7fc5db136180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:11:51.087 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:51.086+0000 7f87a7f7e180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:11:51.144 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:51.149+0000 7fc5db136180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:11:51.246 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:51.246+0000 7f87a7f7e180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:11:51.304 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:51.309+0000 7fc5db136180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:11:51.329 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:51.330+0000 7f87a7f7e180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:11:51.388 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:51.393+0000 7fc5db136180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:11:51.647 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:51.646+0000 7f87a7f7e180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:11:51.706 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:51.709+0000 7fc5db136180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:11:51.708 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:11:51.710+0000 7f87a7f7e180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:11:51.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:51.769+0000 7f7189600640 1 -- 172.21.15.86:0/3163188731 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 68) ==== 93318+0+0 (secure 0 0 0) 0x7f7178058040 con 0x7f718c072630 2024-04-25T14:11:51.771 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:51.773+0000 7fc5db136180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:11:51.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:51.785+0000 7f7189600640 1 -- 172.21.15.86:0/3163188731 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mgrmap(e 69) ==== 92927+0+0 (secure 0 0 0) 0x7f71780754f0 con 0x7f718c072630 2024-04-25T14:11:51.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:51.785+0000 7f7189600640 1 -- 172.21.15.86:0/3163188731 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f716c06f570 msgr2=0x7f716c071a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:11:51.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:51.785+0000 7f7189600640 1 --2- 172.21.15.86:0/3163188731 >> [v2:172.21.15.86:6832/264890479,v1:172.21.15.86:6833/264890479] conn(0x7f716c06f570 0x7f716c071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:51.856 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:51.857+0000 7fc55ae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:51.856 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:51.857+0000 7fc55ae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:51.856 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:51.857+0000 7fc55ae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:51.856 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:51.857+0000 7fc55ae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:51.856 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:51.857+0000 7fc55ae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:51.857 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:51.861+0000 7fc557200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:51.857 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:51.861+0000 7fc557200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:51.857 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:51.861+0000 7fc557200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:51.857 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:51.861+0000 7fc557200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:51.857 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:11:51.861+0000 7fc557200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:11:52.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:52.788+0000 7f7189600640 1 -- 172.21.15.86:0/3163188731 <== mon.0 v2:172.21.15.86:3300/0 9 ==== mgrmap(e 70) ==== 93054+0+0 (secure 0 0 0) 0x7f7178058910 con 0x7f718c072630 2024-04-25T14:11:52.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:52.792+0000 7f7189600640 1 --2- 172.21.15.86:0/3163188731 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f716c037950 0x7f716c079ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:52.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:52.792+0000 7f7189600640 1 -- 172.21.15.86:0/3163188731 --> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f7178008eb0 con 0x7f716c037950 2024-04-25T14:11:52.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:52.796+0000 7f7191600640 1 --2- 172.21.15.86:0/3163188731 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f716c037950 0x7f716c079ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:52.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:52.796+0000 7f7191600640 1 --2- 172.21.15.86:0/3163188731 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f716c037950 0x7f716c079ad0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f71800011a0 tx=0x7f7180001130 comp rx=0 tx=0).ready entity=mgr.9313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:52.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:52.804+0000 7f7189600640 1 -- 172.21.15.86:0/3163188731 <== mgr.9313 v2:172.21.15.86:6832/1173654665 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f7178008eb0 con 0x7f716c037950 2024-04-25T14:11:52.801 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {}, "global_metrics": {}, "metrics": {"delayed_ranks": []}} 2024-04-25T14:11:52.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:52.808+0000 7f7194000640 1 -- 172.21.15.86:0/3163188731 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f716c037950 msgr2=0x7f716c079ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:52.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:52.808+0000 7f7194000640 1 --2- 172.21.15.86:0/3163188731 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f716c037950 0x7f716c079ad0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f71800011a0 tx=0x7f7180001130 comp rx=0 tx=0).stop 2024-04-25T14:11:52.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:52.808+0000 7f7194000640 1 -- 172.21.15.86:0/3163188731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f718c072630 msgr2=0x7f718c150b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:52.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:52.808+0000 7f7194000640 1 --2- 172.21.15.86:0/3163188731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f718c072630 0x7f718c150b10 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f717800e8a0 tx=0x7f717800eca0 comp rx=0 tx=0).stop 2024-04-25T14:11:52.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:52.808+0000 7f7192000640 1 -- 172.21.15.86:0/3163188731 reap_dead start 2024-04-25T14:11:52.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:52.808+0000 7f7194000640 1 -- 172.21.15.86:0/3163188731 shutdown_connections 2024-04-25T14:11:52.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:52.808+0000 7f7194000640 1 -- 172.21.15.86:0/3163188731 >> 172.21.15.86:0/3163188731 conn(0x7f718c0d3580 msgr2=0x7f718c162350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:52.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:52.808+0000 7f7194000640 1 -- 172.21.15.86:0/3163188731 shutdown_connections 2024-04-25T14:11:52.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:52.808+0000 7f7194000640 1 -- 172.21.15.86:0/3163188731 wait complete. 2024-04-25T14:11:53.829 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:11:53.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.988+0000 7fd263200640 1 Processor -- start 2024-04-25T14:11:53.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.988+0000 7fd263200640 1 -- start start 2024-04-25T14:11:53.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.988+0000 7fd263200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd264157f80 0x7fd2640aea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:53.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.988+0000 7fd263200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd264161b20 0x7fd26406c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:53.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.988+0000 7fd263200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2640b0710 con 0x7fd26406f6d0 2024-04-25T14:11:53.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.988+0000 7fd263200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd2640aef70 con 0x7fd264157f80 2024-04-25T14:11:53.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.988+0000 7fd263200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd2640ae1c0 con 0x7fd264161b20 2024-04-25T14:11:53.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.988+0000 7fd261e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd264157f80 0x7fd2640aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:53.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.988+0000 7fd261e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd264157f80 0x7fd2640aea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36060/0 (socket says 172.21.15.86:36060) 2024-04-25T14:11:53.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.988+0000 7fd261400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd26406f6d0 0x7fd264161410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:38858/0 (socket says 172.21.15.86:38858) 2024-04-25T14:11:53.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.988+0000 7fd261e00640 1 -- 172.21.15.86:0/2701179663 learned_addr learned my addr 172.21.15.86:0/2701179663 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:53.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.988+0000 7fd262800640 1 --2- 172.21.15.86:0/2701179663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd264161b20 0x7fd26406c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:53.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.988+0000 7fd261e00640 1 -- 172.21.15.86:0/2701179663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd264161b20 msgr2=0x7fd26406c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:53.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.988+0000 7fd261e00640 1 --2- 172.21.15.86:0/2701179663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd264161b20 0x7fd26406c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:53.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.988+0000 7fd261e00640 1 -- 172.21.15.86:0/2701179663 >> v1:172.21.15.86:6789/0 conn(0x7fd26406f6d0 legacy=0x7fd264161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:53.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.988+0000 7fd261e00640 1 -- 172.21.15.86:0/2701179663 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2640ae4a0 con 0x7fd264157f80 2024-04-25T14:11:53.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.988+0000 7fd262800640 1 --2- 172.21.15.86:0/2701179663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd264161b20 0x7fd26406c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:53.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.988+0000 7fd260a00640 1 -- 172.21.15.86:0/2701179663 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1777248508 0 0) 0x7fd2640b0710 con 0x7fd26406f6d0 2024-04-25T14:11:53.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.992+0000 7fd261e00640 1 --2- 172.21.15.86:0/2701179663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd264157f80 0x7fd2640aea30 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fd254009870 tx=0x7fd25401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e595ec1c176d0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:53.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.992+0000 7fd260a00640 1 -- 172.21.15.86:0/2701179663 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2540b0070 con 0x7fd264157f80 2024-04-25T14:11:53.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.992+0000 7fd260a00640 1 -- 172.21.15.86:0/2701179663 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2540ab600 con 0x7fd264157f80 2024-04-25T14:11:53.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.992+0000 7fd260a00640 1 -- 172.21.15.86:0/2701179663 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2540ab9d0 con 0x7fd264157f80 2024-04-25T14:11:53.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.992+0000 7fd263200640 1 -- 172.21.15.86:0/2701179663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd264157f80 msgr2=0x7fd2640aea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:53.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.992+0000 7fd263200640 1 --2- 172.21.15.86:0/2701179663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd264157f80 0x7fd2640aea30 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fd254009870 tx=0x7fd25401c4d0 comp rx=0 tx=0).stop 2024-04-25T14:11:53.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.992+0000 7fd263200640 1 -- 172.21.15.86:0/2701179663 shutdown_connections 2024-04-25T14:11:53.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.992+0000 7fd263200640 1 --2- 172.21.15.86:0/2701179663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd264161b20 0x7fd26406c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:53.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.992+0000 7fd263200640 1 --2- 172.21.15.86:0/2701179663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd264157f80 0x7fd2640aea30 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:53.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.992+0000 7fd263200640 1 -- 172.21.15.86:0/2701179663 >> 172.21.15.86:0/2701179663 conn(0x7fd2640b4740 msgr2=0x7fd2640b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:53.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.992+0000 7fd263200640 1 -- 172.21.15.86:0/2701179663 shutdown_connections 2024-04-25T14:11:53.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.992+0000 7fd263200640 1 -- 172.21.15.86:0/2701179663 wait complete. 2024-04-25T14:11:53.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.996+0000 7fd263200640 1 Processor -- start 2024-04-25T14:11:53.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.996+0000 7fd263200640 1 -- start start 2024-04-25T14:11:53.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.996+0000 7fd263200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd26406f6d0 0x7fd26414a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:53.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.996+0000 7fd263200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd264157f80 0x7fd264145200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:53.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.996+0000 7fd263200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd26414d990 0x7fd264145740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:53.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.996+0000 7fd263200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd26416b100 con 0x7fd26406f6d0 2024-04-25T14:11:53.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.996+0000 7fd263200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd26416af80 con 0x7fd26414d990 2024-04-25T14:11:53.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.996+0000 7fd263200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd26416b280 con 0x7fd264157f80 2024-04-25T14:11:53.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.996+0000 7fd261e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd26406f6d0 0x7fd26414a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:53.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.996+0000 7fd261e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd26406f6d0 0x7fd26414a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36076/0 (socket says 172.21.15.86:36076) 2024-04-25T14:11:53.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:53.996+0000 7fd261e00640 1 -- 172.21.15.86:0/121495248 learned_addr learned my addr 172.21.15.86:0/121495248 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:53.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.000+0000 7fd261e00640 1 -- 172.21.15.86:0/121495248 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd264157f80 msgr2=0x7fd264145200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:53.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.000+0000 7fd261e00640 1 --2- 172.21.15.86:0/121495248 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd264157f80 0x7fd264145200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:53.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.000+0000 7fd261e00640 1 -- 172.21.15.86:0/121495248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd26414d990 msgr2=0x7fd264145740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:53.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.000+0000 7fd261e00640 1 --2- 172.21.15.86:0/121495248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd26414d990 0x7fd264145740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:53.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.000+0000 7fd261e00640 1 -- 172.21.15.86:0/121495248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd26409a930 con 0x7fd26406f6d0 2024-04-25T14:11:53.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.000+0000 7fd261e00640 1 --2- 172.21.15.86:0/121495248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd26406f6d0 0x7fd26414a6d0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fd2540ae040 tx=0x7fd25401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:53.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.000+0000 7fd25a000640 1 -- 172.21.15.86:0/121495248 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2540b0040 con 0x7fd26406f6d0 2024-04-25T14:11:53.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.000+0000 7fd25a000640 1 -- 172.21.15.86:0/121495248 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2540bc070 con 0x7fd26406f6d0 2024-04-25T14:11:53.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.000+0000 7fd25a000640 1 -- 172.21.15.86:0/121495248 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2540aa9f0 con 0x7fd26406f6d0 2024-04-25T14:11:53.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.000+0000 7fd263200640 1 -- 172.21.15.86:0/121495248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd264148f90 con 0x7fd26406f6d0 2024-04-25T14:11:53.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.000+0000 7fd263200640 1 -- 172.21.15.86:0/121495248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd26414b410 con 0x7fd26406f6d0 2024-04-25T14:11:53.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.000+0000 7fd263200640 1 -- 172.21.15.86:0/121495248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd264076b80 con 0x7fd26406f6d0 2024-04-25T14:11:53.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.000+0000 7fd25a000640 1 -- 172.21.15.86:0/121495248 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 71) ==== 93262+0+0 (secure 0 0 0) 0x7fd2540aab90 con 0x7fd26406f6d0 2024-04-25T14:11:54.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.004+0000 7fd25a000640 1 --2- 172.21.15.86:0/121495248 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fd24406f4e0 0x7fd2440719f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:54.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.004+0000 7fd25a000640 1 -- 172.21.15.86:0/121495248 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 6445+0+0 (secure 0 0 0) 0x7fd254129cc0 con 0x7fd26406f6d0 2024-04-25T14:11:54.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.004+0000 7fd261400640 1 --2- 172.21.15.86:0/121495248 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fd24406f4e0 0x7fd2440719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:54.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.004+0000 7fd261400640 1 --2- 172.21.15.86:0/121495248 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fd24406f4e0 0x7fd2440719f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd264068f30 tx=0x7fd250007970 comp rx=0 tx=0).ready entity=mgr.9313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:54.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.008+0000 7fd25a000640 1 -- 172.21.15.86:0/121495248 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd254130080 con 0x7fd26406f6d0 2024-04-25T14:11:54.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.188+0000 7fd263200640 1 -- 172.21.15.86:0/121495248 --> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7fd2641507f0 con 0x7fd24406f4e0 2024-04-25T14:11:54.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.188+0000 7fd25a000640 1 -- 172.21.15.86:0/121495248 <== mgr.9313 v2:172.21.15.86:6832/1173654665 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fd2641507f0 con 0x7fd24406f4e0 2024-04-25T14:11:54.185 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {}, "global_metrics": {}, "metrics": {"delayed_ranks": []}} 2024-04-25T14:11:54.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.188+0000 7fd263200640 1 -- 172.21.15.86:0/121495248 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fd24406f4e0 msgr2=0x7fd2440719f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:54.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.188+0000 7fd263200640 1 --2- 172.21.15.86:0/121495248 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fd24406f4e0 0x7fd2440719f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd264068f30 tx=0x7fd250007970 comp rx=0 tx=0).stop 2024-04-25T14:11:54.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.188+0000 7fd263200640 1 -- 172.21.15.86:0/121495248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd26406f6d0 msgr2=0x7fd26414a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:54.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.188+0000 7fd263200640 1 --2- 172.21.15.86:0/121495248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd26406f6d0 0x7fd26414a6d0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fd2540ae040 tx=0x7fd25401ad10 comp rx=0 tx=0).stop 2024-04-25T14:11:54.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.188+0000 7fd263200640 1 -- 172.21.15.86:0/121495248 shutdown_connections 2024-04-25T14:11:54.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.188+0000 7fd263200640 1 --2- 172.21.15.86:0/121495248 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fd24406f4e0 0x7fd2440719f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:54.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.188+0000 7fd263200640 1 --2- 172.21.15.86:0/121495248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd26414d990 0x7fd264145740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:54.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.188+0000 7fd263200640 1 --2- 172.21.15.86:0/121495248 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd264157f80 0x7fd264145200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:54.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.188+0000 7fd263200640 1 --2- 172.21.15.86:0/121495248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd26406f6d0 0x7fd26414a6d0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:54.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.188+0000 7fd263200640 1 -- 172.21.15.86:0/121495248 >> 172.21.15.86:0/121495248 conn(0x7fd2640b4740 msgr2=0x7fd2641617e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:54.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.188+0000 7fd263200640 1 -- 172.21.15.86:0/121495248 shutdown_connections 2024-04-25T14:11:54.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:54.188+0000 7fd263200640 1 -- 172.21.15.86:0/121495248 wait complete. 2024-04-25T14:11:55.209 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:11:55.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.368+0000 7f80d4200640 1 Processor -- start 2024-04-25T14:11:55.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.368+0000 7f80d4200640 1 -- start start 2024-04-25T14:11:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.372+0000 7f80d4200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80cc070330 0x7f80cc0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.372+0000 7f80d4200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f80cc0d1c20 0x7f80cc06ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.372+0000 7f80d4200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80cc0d2030 con 0x7f80cc06f270 2024-04-25T14:11:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.372+0000 7f80d4200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f80cc15e5c0 con 0x7f80cc0d1c20 2024-04-25T14:11:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.372+0000 7f80d4200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f80cc15e8a0 con 0x7f80cc070330 2024-04-25T14:11:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.372+0000 7f80d0e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f80cc0d1c20 0x7f80cc06ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.372+0000 7f80d0e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f80cc0d1c20 0x7f80cc06ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36086/0 (socket says 172.21.15.86:36086) 2024-04-25T14:11:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.372+0000 7f80d0e00640 1 -- 172.21.15.86:0/250706735 learned_addr learned my addr 172.21.15.86:0/250706735 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.372+0000 7f80d0e00640 1 -- 172.21.15.86:0/250706735 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80cc070330 msgr2=0x7f80cc0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.372+0000 7f80d1800640 1 --2- 172.21.15.86:0/250706735 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80cc070330 0x7f80cc0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.372+0000 7f80d0e00640 1 --2- 172.21.15.86:0/250706735 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80cc070330 0x7f80cc0d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.372+0000 7f80d0e00640 1 -- 172.21.15.86:0/250706735 >> v1:172.21.15.155:6789/0 conn(0x7f80cc06f270 legacy=0x7f80cc06f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.372+0000 7f80d0e00640 1 -- 172.21.15.86:0/250706735 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80cc15eb80 con 0x7f80cc0d1c20 2024-04-25T14:11:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.372+0000 7f80d1800640 1 --2- 172.21.15.86:0/250706735 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80cc070330 0x7f80cc0d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:55.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.376+0000 7f80d0e00640 1 --2- 172.21.15.86:0/250706735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80cc0d1c20 0x7f80cc06ed30 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f80b80098a0 tx=0x7f80b801c290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5c05f773a9bc96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:55.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.376+0000 7f80cbe00640 1 -- 172.21.15.86:0/250706735 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b80b0070 con 0x7f80cc0d1c20 2024-04-25T14:11:55.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.376+0000 7f80cbe00640 1 -- 172.21.15.86:0/250706735 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80b80ab600 con 0x7f80cc0d1c20 2024-04-25T14:11:55.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.376+0000 7f80cbe00640 1 -- 172.21.15.86:0/250706735 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b80ab9d0 con 0x7f80cc0d1c20 2024-04-25T14:11:55.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.376+0000 7f80d4200640 1 -- 172.21.15.86:0/250706735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80cc0d1c20 msgr2=0x7f80cc06ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:55.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.376+0000 7f80d4200640 1 --2- 172.21.15.86:0/250706735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80cc0d1c20 0x7f80cc06ed30 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f80b80098a0 tx=0x7f80b801c290 comp rx=0 tx=0).stop 2024-04-25T14:11:55.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.376+0000 7f80d4200640 1 -- 172.21.15.86:0/250706735 shutdown_connections 2024-04-25T14:11:55.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.376+0000 7f80d4200640 1 --2- 172.21.15.86:0/250706735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80cc0d1c20 0x7f80cc06ed30 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:55.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.376+0000 7f80d4200640 1 --2- 172.21.15.86:0/250706735 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80cc070330 0x7f80cc0d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:55.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.376+0000 7f80d4200640 1 -- 172.21.15.86:0/250706735 >> 172.21.15.86:0/250706735 conn(0x7f80cc0b4650 msgr2=0x7f80cc0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:55.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.376+0000 7f80d4200640 1 -- 172.21.15.86:0/250706735 shutdown_connections 2024-04-25T14:11:55.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.376+0000 7f80d4200640 1 -- 172.21.15.86:0/250706735 wait complete. 2024-04-25T14:11:55.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.380+0000 7f80d4200640 1 Processor -- start 2024-04-25T14:11:55.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.380+0000 7f80d4200640 1 -- start start 2024-04-25T14:11:55.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.380+0000 7f80d4200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80cc06f270 0x7f80cc159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:55.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.380+0000 7f80d4200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80cc070330 0x7f80cc0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:55.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.380+0000 7f80d4200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80cc0d1c20 0x7f80cc073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:55.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.380+0000 7f80d4200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc0708c0 con 0x7f80cc06f270 2024-04-25T14:11:55.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.380+0000 7f80d4200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc070740 con 0x7f80cc0d1c20 2024-04-25T14:11:55.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.380+0000 7f80d4200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f80cc16afc0 con 0x7f80cc070330 2024-04-25T14:11:55.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.380+0000 7f80d1800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80cc06f270 0x7f80cc159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:55.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.380+0000 7f80d1800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80cc06f270 0x7f80cc159fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36094/0 (socket says 172.21.15.86:36094) 2024-04-25T14:11:55.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.380+0000 7f80d1800640 1 -- 172.21.15.86:0/3645708483 learned_addr learned my addr 172.21.15.86:0/3645708483 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:55.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.380+0000 7f80d0e00640 1 --2- 172.21.15.86:0/3645708483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80cc070330 0x7f80cc0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:55.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.380+0000 7f80d2200640 1 --2- 172.21.15.86:0/3645708483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80cc0d1c20 0x7f80cc073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:55.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.380+0000 7f80d1800640 1 -- 172.21.15.86:0/3645708483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80cc070330 msgr2=0x7f80cc0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:55.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.380+0000 7f80d1800640 1 --2- 172.21.15.86:0/3645708483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80cc070330 0x7f80cc0738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:55.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.380+0000 7f80d1800640 1 -- 172.21.15.86:0/3645708483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80cc0d1c20 msgr2=0x7f80cc073e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:55.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.380+0000 7f80d1800640 1 --2- 172.21.15.86:0/3645708483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80cc0d1c20 0x7f80cc073e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:55.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.380+0000 7f80d1800640 1 -- 172.21.15.86:0/3645708483 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80cc0af3a0 con 0x7f80cc06f270 2024-04-25T14:11:55.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.380+0000 7f80d2200640 1 --2- 172.21.15.86:0/3645708483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80cc0d1c20 0x7f80cc073e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:55.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.380+0000 7f80d1800640 1 --2- 172.21.15.86:0/3645708483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80cc06f270 0x7f80cc159fc0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f80c000c820 tx=0x7f80c000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:55.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.380+0000 7f80c9600640 1 -- 172.21.15.86:0/3645708483 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c0015070 con 0x7f80cc06f270 2024-04-25T14:11:55.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.380+0000 7f80c9600640 1 -- 172.21.15.86:0/3645708483 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c0007a30 con 0x7f80cc06f270 2024-04-25T14:11:55.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.380+0000 7f80c9600640 1 -- 172.21.15.86:0/3645708483 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c0007de0 con 0x7f80cc06f270 2024-04-25T14:11:55.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.380+0000 7f80d4200640 1 -- 172.21.15.86:0/3645708483 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80cc15a500 con 0x7f80cc06f270 2024-04-25T14:11:55.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.380+0000 7f80d4200640 1 -- 172.21.15.86:0/3645708483 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80cc0be310 con 0x7f80cc06f270 2024-04-25T14:11:55.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.384+0000 7f80d4200640 1 -- 172.21.15.86:0/3645708483 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80cc069d00 con 0x7f80cc06f270 2024-04-25T14:11:55.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.384+0000 7f80c9600640 1 -- 172.21.15.86:0/3645708483 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 71) ==== 93262+0+0 (secure 0 0 0) 0x7f80c0018070 con 0x7f80cc06f270 2024-04-25T14:11:55.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.384+0000 7f80c9600640 1 --2- 172.21.15.86:0/3645708483 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f80ac06f510 0x7f80ac071a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:55.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.384+0000 7f80c9600640 1 -- 172.21.15.86:0/3645708483 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 6445+0+0 (secure 0 0 0) 0x7f80c008f630 con 0x7f80cc06f270 2024-04-25T14:11:55.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.384+0000 7f80d0e00640 1 --2- 172.21.15.86:0/3645708483 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f80ac06f510 0x7f80ac071a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:55.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.384+0000 7f80d0e00640 1 --2- 172.21.15.86:0/3645708483 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f80ac06f510 0x7f80ac071a20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f80b80ad000 tx=0x7f80b80a6000 comp rx=0 tx=0).ready entity=mgr.9313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:55.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.388+0000 7f80c9600640 1 -- 172.21.15.86:0/3645708483 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f80cc15a500 con 0x7f80cc06f270 2024-04-25T14:11:55.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.568+0000 7f80d4200640 1 -- 172.21.15.86:0/3645708483 --> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f80cc076bf0 con 0x7f80ac06f510 2024-04-25T14:11:55.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.568+0000 7f80c9600640 1 -- 172.21.15.86:0/3645708483 <== mgr.9313 v2:172.21.15.86:6832/1173654665 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f80cc076bf0 con 0x7f80ac06f510 2024-04-25T14:11:55.567 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {}, "global_metrics": {}, "metrics": {"delayed_ranks": []}} 2024-04-25T14:11:55.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.572+0000 7f80d4200640 1 -- 172.21.15.86:0/3645708483 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f80ac06f510 msgr2=0x7f80ac071a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:55.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.572+0000 7f80d4200640 1 --2- 172.21.15.86:0/3645708483 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f80ac06f510 0x7f80ac071a20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f80b80ad000 tx=0x7f80b80a6000 comp rx=0 tx=0).stop 2024-04-25T14:11:55.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.572+0000 7f80d4200640 1 -- 172.21.15.86:0/3645708483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80cc06f270 msgr2=0x7f80cc159fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:55.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.572+0000 7f80d4200640 1 --2- 172.21.15.86:0/3645708483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80cc06f270 0x7f80cc159fc0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f80c000c820 tx=0x7f80c000cce0 comp rx=0 tx=0).stop 2024-04-25T14:11:55.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.572+0000 7f80d4200640 1 -- 172.21.15.86:0/3645708483 shutdown_connections 2024-04-25T14:11:55.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.572+0000 7f80d4200640 1 --2- 172.21.15.86:0/3645708483 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f80ac06f510 0x7f80ac071a20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:55.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.572+0000 7f80d4200640 1 --2- 172.21.15.86:0/3645708483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80cc0d1c20 0x7f80cc073e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:55.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.572+0000 7f80d4200640 1 --2- 172.21.15.86:0/3645708483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80cc070330 0x7f80cc0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:55.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.572+0000 7f80d4200640 1 --2- 172.21.15.86:0/3645708483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80cc06f270 0x7f80cc159fc0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:55.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.572+0000 7f80d4200640 1 -- 172.21.15.86:0/3645708483 >> 172.21.15.86:0/3645708483 conn(0x7f80cc0b4650 msgr2=0x7f80cc15d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:55.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.572+0000 7f80d4200640 1 -- 172.21.15.86:0/3645708483 shutdown_connections 2024-04-25T14:11:55.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:55.572+0000 7f80d4200640 1 -- 172.21.15.86:0/3645708483 wait complete. 2024-04-25T14:11:56.589 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:11:56.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.748+0000 7f5b57c00640 1 Processor -- start 2024-04-25T14:11:56.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.748+0000 7f5b57c00640 1 -- start start 2024-04-25T14:11:56.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.752+0000 7f5b57c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b500b59e0 0x7f5b500b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:56.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.752+0000 7f5b57c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5b500aeea0 0x7f5b500af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:56.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.752+0000 7f5b57c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b500b6330 con 0x7f5b5015faa0 2024-04-25T14:11:56.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.752+0000 7f5b57c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5b5006d560 con 0x7f5b500aeea0 2024-04-25T14:11:56.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.752+0000 7f5b57c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5b5006c670 con 0x7f5b500b59e0 2024-04-25T14:11:56.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.752+0000 7f5b55c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5b500aeea0 0x7f5b500af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:56.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.752+0000 7f5b55200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5b5015faa0 0x7f5b5006d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:38864/0 (socket says 172.21.15.86:38864) 2024-04-25T14:11:56.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.752+0000 7f5b55c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5b500aeea0 0x7f5b500af2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36102/0 (socket says 172.21.15.86:36102) 2024-04-25T14:11:56.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.752+0000 7f5b55200640 1 -- 172.21.15.86:0/485336223 learned_addr learned my addr 172.21.15.86:0/485336223 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:56.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.752+0000 7f5b4fe00640 1 --2- 172.21.15.86:0/485336223 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b500b59e0 0x7f5b500b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:56.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.752+0000 7f5b55c00640 1 -- 172.21.15.86:0/485336223 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b500b59e0 msgr2=0x7f5b500b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:56.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.752+0000 7f5b55c00640 1 --2- 172.21.15.86:0/485336223 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b500b59e0 0x7f5b500b5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:56.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.752+0000 7f5b55c00640 1 -- 172.21.15.86:0/485336223 >> v1:172.21.15.86:6789/0 conn(0x7f5b5015faa0 legacy=0x7f5b5006d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:56.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.752+0000 7f5b4f400640 1 -- 172.21.15.86:0/485336223 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1855813032 0 0) 0x7f5b400014a0 con 0x7f5b5015faa0 2024-04-25T14:11:56.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.752+0000 7f5b55c00640 1 -- 172.21.15.86:0/485336223 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5006c950 con 0x7f5b500aeea0 2024-04-25T14:11:56.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.752+0000 7f5b4fe00640 1 --2- 172.21.15.86:0/485336223 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b500b59e0 0x7f5b500b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:56.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.752+0000 7f5b55c00640 1 --2- 172.21.15.86:0/485336223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b500aeea0 0x7f5b500af2b0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f5b480097c0 tx=0x7f5b4801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ccb6e664c746b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:56.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.756+0000 7f5b4f400640 1 -- 172.21.15.86:0/485336223 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b480aa620 con 0x7f5b500aeea0 2024-04-25T14:11:56.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.756+0000 7f5b4f400640 1 -- 172.21.15.86:0/485336223 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b480af070 con 0x7f5b500aeea0 2024-04-25T14:11:56.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.756+0000 7f5b4f400640 1 -- 172.21.15.86:0/485336223 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b480aa9e0 con 0x7f5b500aeea0 2024-04-25T14:11:56.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.756+0000 7f5b57c00640 1 -- 172.21.15.86:0/485336223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b500aeea0 msgr2=0x7f5b500af2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:56.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.756+0000 7f5b57c00640 1 --2- 172.21.15.86:0/485336223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b500aeea0 0x7f5b500af2b0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f5b480097c0 tx=0x7f5b4801c4a0 comp rx=0 tx=0).stop 2024-04-25T14:11:56.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.756+0000 7f5b57c00640 1 -- 172.21.15.86:0/485336223 shutdown_connections 2024-04-25T14:11:56.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.756+0000 7f5b57c00640 1 --2- 172.21.15.86:0/485336223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b500aeea0 0x7f5b500af2b0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:56.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.756+0000 7f5b57c00640 1 --2- 172.21.15.86:0/485336223 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b500b59e0 0x7f5b500b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:56.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.756+0000 7f5b57c00640 1 -- 172.21.15.86:0/485336223 >> 172.21.15.86:0/485336223 conn(0x7f5b500d2230 msgr2=0x7f5b500d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:56.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.756+0000 7f5b57c00640 1 -- 172.21.15.86:0/485336223 shutdown_connections 2024-04-25T14:11:56.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.756+0000 7f5b57c00640 1 -- 172.21.15.86:0/485336223 wait complete. 2024-04-25T14:11:56.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.756+0000 7f5b57c00640 1 Processor -- start 2024-04-25T14:11:56.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.756+0000 7f5b57c00640 1 -- start start 2024-04-25T14:11:56.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.760+0000 7f5b57c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b500aeea0 0x7f5b50164da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:56.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.760+0000 7f5b57c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b500b59e0 0x7f5b501626d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:56.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.760+0000 7f5b57c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b5015faa0 0x7f5b5015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:56.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.760+0000 7f5b57c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b50160030 con 0x7f5b500b59e0 2024-04-25T14:11:56.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.760+0000 7f5b57c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5015feb0 con 0x7f5b5015faa0 2024-04-25T14:11:56.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.760+0000 7f5b57c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5b50070f30 con 0x7f5b500aeea0 2024-04-25T14:11:56.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.760+0000 7f5b4fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b500b59e0 0x7f5b501626d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:56.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.760+0000 7f5b4fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b500b59e0 0x7f5b501626d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36116/0 (socket says 172.21.15.86:36116) 2024-04-25T14:11:56.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.760+0000 7f5b4fe00640 1 -- 172.21.15.86:0/708597255 learned_addr learned my addr 172.21.15.86:0/708597255 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:56.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.760+0000 7f5b55200640 1 --2- 172.21.15.86:0/708597255 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b500aeea0 0x7f5b50164da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:56.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.760+0000 7f5b55c00640 1 --2- 172.21.15.86:0/708597255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b5015faa0 0x7f5b5015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:56.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.760+0000 7f5b4fe00640 1 -- 172.21.15.86:0/708597255 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b500aeea0 msgr2=0x7f5b50164da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:56.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.760+0000 7f5b4fe00640 1 --2- 172.21.15.86:0/708597255 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b500aeea0 0x7f5b50164da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:56.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.760+0000 7f5b4fe00640 1 -- 172.21.15.86:0/708597255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b5015faa0 msgr2=0x7f5b5015b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:56.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.760+0000 7f5b4fe00640 1 --2- 172.21.15.86:0/708597255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b5015faa0 0x7f5b5015b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:56.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.760+0000 7f5b4fe00640 1 -- 172.21.15.86:0/708597255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5015a900 con 0x7f5b500b59e0 2024-04-25T14:11:56.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.760+0000 7f5b55200640 1 --2- 172.21.15.86:0/708597255 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b500aeea0 0x7f5b50164da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:56.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.760+0000 7f5b55c00640 1 --2- 172.21.15.86:0/708597255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b5015faa0 0x7f5b5015b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:56.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.760+0000 7f5b4fe00640 1 --2- 172.21.15.86:0/708597255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b500b59e0 0x7f5b501626d0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00d830 tx=0x7f5b3c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:56.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.760+0000 7f5b4cc00640 1 -- 172.21.15.86:0/708597255 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c00b6c0 con 0x7f5b500b59e0 2024-04-25T14:11:56.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.760+0000 7f5b4cc00640 1 -- 172.21.15.86:0/708597255 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b3c019070 con 0x7f5b500b59e0 2024-04-25T14:11:56.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.760+0000 7f5b4cc00640 1 -- 172.21.15.86:0/708597255 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c00baa0 con 0x7f5b500b59e0 2024-04-25T14:11:56.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.760+0000 7f5b57c00640 1 -- 172.21.15.86:0/708597255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b5015abe0 con 0x7f5b500b59e0 2024-04-25T14:11:56.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.760+0000 7f5b57c00640 1 -- 172.21.15.86:0/708597255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b5015b1a0 con 0x7f5b500b59e0 2024-04-25T14:11:56.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.760+0000 7f5b57c00640 1 -- 172.21.15.86:0/708597255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b5015d7d0 con 0x7f5b500b59e0 2024-04-25T14:11:56.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.764+0000 7f5b4cc00640 1 -- 172.21.15.86:0/708597255 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 72) ==== 93318+0+0 (secure 0 0 0) 0x7f5b3c01e460 con 0x7f5b500b59e0 2024-04-25T14:11:56.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.764+0000 7f5b4cc00640 1 --2- 172.21.15.86:0/708597255 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f5b3006f570 0x7f5b30071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:56.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.764+0000 7f5b4cc00640 1 -- 172.21.15.86:0/708597255 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 6445+0+0 (secure 0 0 0) 0x7f5b3c08fe20 con 0x7f5b500b59e0 2024-04-25T14:11:56.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.764+0000 7f5b55200640 1 --2- 172.21.15.86:0/708597255 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f5b3006f570 0x7f5b30071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:56.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.764+0000 7f5b55200640 1 --2- 172.21.15.86:0/708597255 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f5b3006f570 0x7f5b30071a80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5b40007fd0 tx=0x7f5b40005b80 comp rx=0 tx=0).ready entity=mgr.9313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:56.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.768+0000 7f5b4cc00640 1 -- 172.21.15.86:0/708597255 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5b3c05c920 con 0x7f5b500b59e0 2024-04-25T14:11:56.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.948+0000 7f5b57c00640 1 -- 172.21.15.86:0/708597255 --> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f5b50074cf0 con 0x7f5b3006f570 2024-04-25T14:11:56.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.948+0000 7f5b4cc00640 1 -- 172.21.15.86:0/708597255 <== mgr.9313 v2:172.21.15.86:6832/1173654665 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f5b50074cf0 con 0x7f5b3006f570 2024-04-25T14:11:56.947 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {}, "global_metrics": {}, "metrics": {"delayed_ranks": []}} 2024-04-25T14:11:56.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.952+0000 7f5b57c00640 1 -- 172.21.15.86:0/708597255 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f5b3006f570 msgr2=0x7f5b30071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:56.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.952+0000 7f5b57c00640 1 --2- 172.21.15.86:0/708597255 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f5b3006f570 0x7f5b30071a80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5b40007fd0 tx=0x7f5b40005b80 comp rx=0 tx=0).stop 2024-04-25T14:11:56.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.952+0000 7f5b57c00640 1 -- 172.21.15.86:0/708597255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b500b59e0 msgr2=0x7f5b501626d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:56.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.952+0000 7f5b57c00640 1 --2- 172.21.15.86:0/708597255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b500b59e0 0x7f5b501626d0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00d830 tx=0x7f5b3c00dcf0 comp rx=0 tx=0).stop 2024-04-25T14:11:56.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.952+0000 7f5b57c00640 1 -- 172.21.15.86:0/708597255 shutdown_connections 2024-04-25T14:11:56.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.952+0000 7f5b57c00640 1 --2- 172.21.15.86:0/708597255 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f5b3006f570 0x7f5b30071a80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:56.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.952+0000 7f5b57c00640 1 --2- 172.21.15.86:0/708597255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b5015faa0 0x7f5b5015b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:56.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.952+0000 7f5b57c00640 1 --2- 172.21.15.86:0/708597255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b500b59e0 0x7f5b501626d0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:56.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.952+0000 7f5b57c00640 1 --2- 172.21.15.86:0/708597255 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b500aeea0 0x7f5b50164da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:56.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.952+0000 7f5b57c00640 1 -- 172.21.15.86:0/708597255 >> 172.21.15.86:0/708597255 conn(0x7f5b500d2230 msgr2=0x7f5b50070710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:56.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.952+0000 7f5b57c00640 1 -- 172.21.15.86:0/708597255 shutdown_connections 2024-04-25T14:11:56.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:56.952+0000 7f5b57c00640 1 -- 172.21.15.86:0/708597255 wait complete. 2024-04-25T14:11:57.970 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:11:58.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.124+0000 7f3e37200640 1 Processor -- start 2024-04-25T14:11:58.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.124+0000 7f3e37200640 1 -- start start 2024-04-25T14:11:58.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.124+0000 7f3e37200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e3015faa0 0x7f3e3006d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:58.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.124+0000 7f3e37200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e3006c670 con 0x7f3e300aeea0 2024-04-25T14:11:58.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.124+0000 7f3e37200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e3006c950 con 0x7f3e300b59e0 2024-04-25T14:11:58.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.124+0000 7f3e37200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3e3006d560 con 0x7f3e3015faa0 2024-04-25T14:11:58.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.124+0000 7f3e35200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3e300aeea0 0x7f3e300af2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:38878/0 (socket says 172.21.15.86:38878) 2024-04-25T14:11:58.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.124+0000 7f3e35200640 1 -- 172.21.15.86:0/2429870805 learned_addr learned my addr 172.21.15.86:0/2429870805 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:58.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.124+0000 7f3e2fe00640 1 --2- 172.21.15.86:0/2429870805 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e3015faa0 0x7f3e3006d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:58.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.124+0000 7f3e2f400640 1 -- 172.21.15.86:0/2429870805 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3944788433 0 0) 0x7f3e3006c670 con 0x7f3e300aeea0 2024-04-25T14:11:58.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.124+0000 7f3e2f400640 1 -- 172.21.15.86:0/2429870805 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e14003530 con 0x7f3e300aeea0 2024-04-25T14:11:58.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.124+0000 7f3e2f400640 1 -- 172.21.15.86:0/2429870805 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2517443485 0 0) 0x7f3e14003530 con 0x7f3e300aeea0 2024-04-25T14:11:58.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.124+0000 7f3e2f400640 1 -- 172.21.15.86:0/2429870805 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e3015faa0 msgr2=0x7f3e3006d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:58.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.124+0000 7f3e2f400640 1 --2- 172.21.15.86:0/2429870805 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e3015faa0 0x7f3e3006d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:58.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.124+0000 7f3e2f400640 1 -- 172.21.15.86:0/2429870805 >> v1:172.21.15.155:6789/0 conn(0x7f3e300b59e0 legacy=0x7f3e300b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:58.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.124+0000 7f3e2f400640 1 -- 172.21.15.86:0/2429870805 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3006cc30 con 0x7f3e300aeea0 2024-04-25T14:11:58.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.124+0000 7f3e2f400640 1 -- 172.21.15.86:0/2429870805 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f3e28001a00 con 0x7f3e300aeea0 2024-04-25T14:11:58.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.124+0000 7f3e2f400640 1 -- 172.21.15.86:0/2429870805 >> v1:172.21.15.86:6789/0 conn(0x7f3e300aeea0 legacy=0x7f3e300af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:58.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.124+0000 7f3e2f400640 1 --2- 172.21.15.86:0/2429870805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e14003a60 0x7f3e14003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:58.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.124+0000 7f3e2f400640 1 --2- 172.21.15.86:0/2429870805 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e3015faa0 0x7f3e14004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:58.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.124+0000 7f3e2f400640 1 --2- 172.21.15.86:0/2429870805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e14004d50 0x7f3e14005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:58.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.124+0000 7f3e2f400640 1 -- 172.21.15.86:0/2429870805 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3006d560 con 0x7f3e14004d50 2024-04-25T14:11:58.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.124+0000 7f3e2f400640 1 -- 172.21.15.86:0/2429870805 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1400bad0 con 0x7f3e14003a60 2024-04-25T14:11:58.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.124+0000 7f3e2f400640 1 -- 172.21.15.86:0/2429870805 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3e1400bcd0 con 0x7f3e3015faa0 2024-04-25T14:11:58.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.124+0000 7f3e35200640 1 --2- 172.21.15.86:0/2429870805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e14004d50 0x7f3e14005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:58.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.124+0000 7f3e27400640 1 --2- 172.21.15.86:0/2429870805 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e3015faa0 0x7f3e14004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:58.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.124+0000 7f3e2fe00640 1 --2- 172.21.15.86:0/2429870805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e14003a60 0x7f3e14003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:58.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.128+0000 7f3e35200640 1 -- 172.21.15.86:0/2429870805 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e3015faa0 msgr2=0x7f3e14004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:58.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.128+0000 7f3e35200640 1 --2- 172.21.15.86:0/2429870805 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e3015faa0 0x7f3e14004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:58.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.128+0000 7f3e35200640 1 -- 172.21.15.86:0/2429870805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e14003a60 msgr2=0x7f3e14003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:58.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.128+0000 7f3e35200640 1 --2- 172.21.15.86:0/2429870805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e14003a60 0x7f3e14003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:58.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.128+0000 7f3e35200640 1 -- 172.21.15.86:0/2429870805 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e14003530 con 0x7f3e14004d50 2024-04-25T14:11:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.128+0000 7f3e35200640 1 --2- 172.21.15.86:0/2429870805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e14004d50 0x7f3e14005160 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800a860 tx=0x7f3e2801fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0874d29639b9ed3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.128+0000 7f3e2f400640 1 -- 172.21.15.86:0/2429870805 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2800a480 con 0x7f3e14004d50 2024-04-25T14:11:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.128+0000 7f3e2f400640 1 -- 172.21.15.86:0/2429870805 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e280aa6f0 con 0x7f3e14004d50 2024-04-25T14:11:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.128+0000 7f3e2f400640 1 -- 172.21.15.86:0/2429870805 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e280aab00 con 0x7f3e14004d50 2024-04-25T14:11:58.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.128+0000 7f3e37200640 1 -- 172.21.15.86:0/2429870805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e14004d50 msgr2=0x7f3e14005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:58.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.128+0000 7f3e37200640 1 --2- 172.21.15.86:0/2429870805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e14004d50 0x7f3e14005160 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800a860 tx=0x7f3e2801fac0 comp rx=0 tx=0).stop 2024-04-25T14:11:58.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.128+0000 7f3e35200640 1 -- 172.21.15.86:0/2429870805 reap_dead start 2024-04-25T14:11:58.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.132+0000 7f3e37200640 1 -- 172.21.15.86:0/2429870805 shutdown_connections 2024-04-25T14:11:58.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.132+0000 7f3e37200640 1 -- 172.21.15.86:0/2429870805 >> 172.21.15.86:0/2429870805 conn(0x7f3e300d2230 msgr2=0x7f3e300d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:58.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.132+0000 7f3e37200640 1 -- 172.21.15.86:0/2429870805 shutdown_connections 2024-04-25T14:11:58.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.132+0000 7f3e37200640 1 -- 172.21.15.86:0/2429870805 wait complete. 2024-04-25T14:11:58.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.132+0000 7f3e37200640 1 Processor -- start 2024-04-25T14:11:58.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.132+0000 7f3e37200640 1 -- start start 2024-04-25T14:11:58.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.132+0000 7f3e37200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e30154680 0x7f3e301f69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:58.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.132+0000 7f3e37200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e30153d20 0x7f3e30157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:58.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.132+0000 7f3e37200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e30154130 0x7f3e3015df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:58.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.132+0000 7f3e37200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e300af9c0 con 0x7f3e30154680 2024-04-25T14:11:58.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.132+0000 7f3e37200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3006d1f0 con 0x7f3e30153d20 2024-04-25T14:11:58.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.132+0000 7f3e37200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3e3015feb0 con 0x7f3e30154130 2024-04-25T14:11:58.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.136+0000 7f3e2fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e30154680 0x7f3e301f69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:58.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.136+0000 7f3e2fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e30154680 0x7f3e301f69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36148/0 (socket says 172.21.15.86:36148) 2024-04-25T14:11:58.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.136+0000 7f3e2fe00640 1 -- 172.21.15.86:0/2132707055 learned_addr learned my addr 172.21.15.86:0/2132707055 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:58.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.136+0000 7f3e35200640 1 --2- 172.21.15.86:0/2132707055 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e30154130 0x7f3e3015df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:58.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.136+0000 7f3e27400640 1 --2- 172.21.15.86:0/2132707055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e30153d20 0x7f3e30157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:58.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.136+0000 7f3e2fe00640 1 -- 172.21.15.86:0/2132707055 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e30154130 msgr2=0x7f3e3015df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:58.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.136+0000 7f3e2fe00640 1 --2- 172.21.15.86:0/2132707055 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e30154130 0x7f3e3015df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:58.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.136+0000 7f3e2fe00640 1 -- 172.21.15.86:0/2132707055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e30153d20 msgr2=0x7f3e30157a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:58.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.136+0000 7f3e2fe00640 1 --2- 172.21.15.86:0/2132707055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e30153d20 0x7f3e30157a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:58.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.136+0000 7f3e2fe00640 1 -- 172.21.15.86:0/2132707055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e301f6f20 con 0x7f3e30154680 2024-04-25T14:11:58.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.136+0000 7f3e2fe00640 1 --2- 172.21.15.86:0/2132707055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e30154680 0x7f3e301f69e0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f3e200119d0 tx=0x7f3e20011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:58.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.136+0000 7f3e2cc00640 1 -- 172.21.15.86:0/2132707055 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2000fb50 con 0x7f3e30154680 2024-04-25T14:11:58.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.136+0000 7f3e2cc00640 1 -- 172.21.15.86:0/2132707055 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e2000fcf0 con 0x7f3e30154680 2024-04-25T14:11:58.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.136+0000 7f3e37200640 1 -- 172.21.15.86:0/2132707055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e30161660 con 0x7f3e30154680 2024-04-25T14:11:58.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.136+0000 7f3e2cc00640 1 -- 172.21.15.86:0/2132707055 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2000fea0 con 0x7f3e30154680 2024-04-25T14:11:58.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.136+0000 7f3e37200640 1 -- 172.21.15.86:0/2132707055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e30161c20 con 0x7f3e30154680 2024-04-25T14:11:58.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.136+0000 7f3e37200640 1 -- 172.21.15.86:0/2132707055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e30074cf0 con 0x7f3e30154680 2024-04-25T14:11:58.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.136+0000 7f3e2cc00640 1 -- 172.21.15.86:0/2132707055 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 72) ==== 93318+0+0 (secure 0 0 0) 0x7f3e20012040 con 0x7f3e30154680 2024-04-25T14:11:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.140+0000 7f3e2cc00640 1 --2- 172.21.15.86:0/2132707055 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f3e0c06f570 0x7f3e0c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.140+0000 7f3e2cc00640 1 -- 172.21.15.86:0/2132707055 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 6445+0+0 (secure 0 0 0) 0x7f3e2008fc90 con 0x7f3e30154680 2024-04-25T14:11:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.140+0000 7f3e27400640 1 --2- 172.21.15.86:0/2132707055 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f3e0c06f570 0x7f3e0c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.140+0000 7f3e27400640 1 --2- 172.21.15.86:0/2132707055 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f3e0c06f570 0x7f3e0c071a80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3e18000c10 tx=0x7f3e18009040 comp rx=0 tx=0).ready entity=mgr.9313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:58.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.144+0000 7f3e2cc00640 1 -- 172.21.15.86:0/2132707055 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3e20019030 con 0x7f3e30154680 2024-04-25T14:11:58.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.320+0000 7f3e37200640 1 -- 172.21.15.86:0/2132707055 --> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f3e300b55b0 con 0x7f3e0c06f570 2024-04-25T14:11:58.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.324+0000 7f3e2cc00640 1 -- 172.21.15.86:0/2132707055 <== mgr.9313 v2:172.21.15.86:6832/1173654665 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+2957 (secure 0 0 0) 0x7f3e300b55b0 con 0x7f3e0c06f570 2024-04-25T14:11:58.320 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {"cephfs": {"client.9292": {"hostname": "smithi155", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:192.168.0.1"}, "client.9732": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:192.168.0.1"}, "client.9792": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}, "client.9795": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}}}, "global_metrics": {"cephfs": {"client.9292": [[22, 0], [0, 0], [0, 0], [0, 5877839], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 1469461], [12322642740577, 4]], "client.9732": [[3, 19], [0, 0], [0, 0], [0, 10916606], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 454864], [12082656693350, 24]], "client.9792": [[2, 0], [0, 0], [0, 0], [0, 579907], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 579907], [0, 1]], "client.9795": [[2, 0], [0, 0], [0, 0], [0, 802945], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 802945], [0, 1]]}}, "metrics": {"delayed_ranks": [], "mds.0": {"client.9292": [], "client.9732": [], "client.9792": [], "client.9795": []}}} 2024-04-25T14:11:58.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.324+0000 7f3e37200640 1 -- 172.21.15.86:0/2132707055 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f3e0c06f570 msgr2=0x7f3e0c071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:58.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.324+0000 7f3e37200640 1 --2- 172.21.15.86:0/2132707055 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f3e0c06f570 0x7f3e0c071a80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3e18000c10 tx=0x7f3e18009040 comp rx=0 tx=0).stop 2024-04-25T14:11:58.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.324+0000 7f3e37200640 1 -- 172.21.15.86:0/2132707055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e30154680 msgr2=0x7f3e301f69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:58.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.324+0000 7f3e37200640 1 --2- 172.21.15.86:0/2132707055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e30154680 0x7f3e301f69e0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f3e200119d0 tx=0x7f3e20011e90 comp rx=0 tx=0).stop 2024-04-25T14:11:58.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.324+0000 7f3e37200640 1 -- 172.21.15.86:0/2132707055 shutdown_connections 2024-04-25T14:11:58.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.324+0000 7f3e37200640 1 --2- 172.21.15.86:0/2132707055 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f3e0c06f570 0x7f3e0c071a80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:58.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.324+0000 7f3e37200640 1 --2- 172.21.15.86:0/2132707055 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e30154130 0x7f3e3015df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:58.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.324+0000 7f3e37200640 1 --2- 172.21.15.86:0/2132707055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e30153d20 0x7f3e30157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:58.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.324+0000 7f3e37200640 1 --2- 172.21.15.86:0/2132707055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e30154680 0x7f3e301f69e0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:58.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.324+0000 7f3e37200640 1 -- 172.21.15.86:0/2132707055 >> 172.21.15.86:0/2132707055 conn(0x7f3e300d2230 msgr2=0x7f3e300b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:58.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.324+0000 7f3e37200640 1 -- 172.21.15.86:0/2132707055 shutdown_connections 2024-04-25T14:11:58.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.324+0000 7f3e37200640 1 -- 172.21.15.86:0/2132707055 wait complete. 2024-04-25T14:11:58.342 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:11:58.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.492+0000 7fdc42600640 1 Processor -- start 2024-04-25T14:11:58.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.492+0000 7fdc42600640 1 -- start start 2024-04-25T14:11:58.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.492+0000 7fdc42600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc3c0cfca0 0x7fdc3c06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:58.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.492+0000 7fdc42600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdc3c0d16e0 0x7fdc3c0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:58.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.492+0000 7fdc42600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc3c0d2030 con 0x7fdc3c06c670 2024-04-25T14:11:58.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.492+0000 7fdc42600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdc3c070470 con 0x7fdc3c0d16e0 2024-04-25T14:11:58.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.492+0000 7fdc42600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdc3c072bd0 con 0x7fdc3c0cfca0 2024-04-25T14:11:58.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.492+0000 7fdc3aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdc3c0d16e0 0x7fdc3c0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:58.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.492+0000 7fdc3be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fdc3c06c670 0x7fdc3c06ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:38892/0 (socket says 172.21.15.86:38892) 2024-04-25T14:11:58.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.492+0000 7fdc3be00640 1 -- 172.21.15.86:0/4079997183 learned_addr learned my addr 172.21.15.86:0/4079997183 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:58.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.492+0000 7fdc3b400640 1 --2- 172.21.15.86:0/4079997183 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc3c0cfca0 0x7fdc3c06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:58.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.492+0000 7fdc3aa00640 1 -- 172.21.15.86:0/4079997183 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc3c0cfca0 msgr2=0x7fdc3c06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:58.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.492+0000 7fdc3aa00640 1 --2- 172.21.15.86:0/4079997183 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc3c0cfca0 0x7fdc3c06f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:58.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.492+0000 7fdc3aa00640 1 -- 172.21.15.86:0/4079997183 >> v1:172.21.15.86:6789/0 conn(0x7fdc3c06c670 legacy=0x7fdc3c06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:58.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.492+0000 7fdc3aa00640 1 -- 172.21.15.86:0/4079997183 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc3c06f930 con 0x7fdc3c0d16e0 2024-04-25T14:11:58.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.492+0000 7fdc3a000640 1 -- 172.21.15.86:0/4079997183 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 562496607 0 0) 0x7fdc340014a0 con 0x7fdc3c06c670 2024-04-25T14:11:58.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.496+0000 7fdc3aa00640 1 --2- 172.21.15.86:0/4079997183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc3c0d16e0 0x7fdc3c0d1af0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fdc280097c0 tx=0x7fdc2801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4faf3fa9b6772ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:58.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.496+0000 7fdc3a000640 1 -- 172.21.15.86:0/4079997183 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc280b1070 con 0x7fdc3c0d16e0 2024-04-25T14:11:58.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.496+0000 7fdc3a000640 1 -- 172.21.15.86:0/4079997183 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc280ac410 con 0x7fdc3c0d16e0 2024-04-25T14:11:58.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.496+0000 7fdc3a000640 1 -- 172.21.15.86:0/4079997183 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc280ac7e0 con 0x7fdc3c0d16e0 2024-04-25T14:11:58.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.496+0000 7fdc42600640 1 -- 172.21.15.86:0/4079997183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc3c0d16e0 msgr2=0x7fdc3c0d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:58.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.496+0000 7fdc42600640 1 --2- 172.21.15.86:0/4079997183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc3c0d16e0 0x7fdc3c0d1af0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fdc280097c0 tx=0x7fdc2801c4a0 comp rx=0 tx=0).stop 2024-04-25T14:11:58.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.496+0000 7fdc42600640 1 -- 172.21.15.86:0/4079997183 shutdown_connections 2024-04-25T14:11:58.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.496+0000 7fdc42600640 1 --2- 172.21.15.86:0/4079997183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc3c0d16e0 0x7fdc3c0d1af0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:58.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.496+0000 7fdc42600640 1 --2- 172.21.15.86:0/4079997183 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc3c0cfca0 0x7fdc3c06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:58.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.496+0000 7fdc42600640 1 -- 172.21.15.86:0/4079997183 >> 172.21.15.86:0/4079997183 conn(0x7fdc3c0b4650 msgr2=0x7fdc3c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:58.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.496+0000 7fdc42600640 1 -- 172.21.15.86:0/4079997183 shutdown_connections 2024-04-25T14:11:58.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.496+0000 7fdc42600640 1 -- 172.21.15.86:0/4079997183 wait complete. 2024-04-25T14:11:58.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.500+0000 7fdc42600640 1 Processor -- start 2024-04-25T14:11:58.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.500+0000 7fdc42600640 1 -- start start 2024-04-25T14:11:58.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.500+0000 7fdc42600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc3c06c670 0x7fdc3c09a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:58.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.500+0000 7fdc42600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc3c0cfca0 0x7fdc3c09e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:58.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.500+0000 7fdc42600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc3c0d16e0 0x7fdc3c068590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:58.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.500+0000 7fdc42600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3c16a070 con 0x7fdc3c0cfca0 2024-04-25T14:11:58.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.500+0000 7fdc42600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3c169ef0 con 0x7fdc3c06c670 2024-04-25T14:11:58.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.500+0000 7fdc42600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdc3c16a1f0 con 0x7fdc3c0d16e0 2024-04-25T14:11:58.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.500+0000 7fdc3aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc3c0cfca0 0x7fdc3c09e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:58.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.500+0000 7fdc3aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc3c0cfca0 0x7fdc3c09e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36156/0 (socket says 172.21.15.86:36156) 2024-04-25T14:11:58.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.500+0000 7fdc3aa00640 1 -- 172.21.15.86:0/3755362593 learned_addr learned my addr 172.21.15.86:0/3755362593 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:58.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.500+0000 7fdc3b400640 1 --2- 172.21.15.86:0/3755362593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc3c06c670 0x7fdc3c09a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:58.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.500+0000 7fdc3be00640 1 --2- 172.21.15.86:0/3755362593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc3c0d16e0 0x7fdc3c068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:58.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.500+0000 7fdc3aa00640 1 -- 172.21.15.86:0/3755362593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc3c0d16e0 msgr2=0x7fdc3c068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:58.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.500+0000 7fdc3aa00640 1 --2- 172.21.15.86:0/3755362593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc3c0d16e0 0x7fdc3c068590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:58.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.500+0000 7fdc3aa00640 1 -- 172.21.15.86:0/3755362593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc3c06c670 msgr2=0x7fdc3c09a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:58.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.500+0000 7fdc3aa00640 1 --2- 172.21.15.86:0/3755362593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc3c06c670 0x7fdc3c09a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:58.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.500+0000 7fdc3aa00640 1 -- 172.21.15.86:0/3755362593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc3c15b800 con 0x7fdc3c0cfca0 2024-04-25T14:11:58.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.500+0000 7fdc3aa00640 1 --2- 172.21.15.86:0/3755362593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc3c0cfca0 0x7fdc3c09e100 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fdc280af040 tx=0x7fdc28019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:58.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.500+0000 7fdc33400640 1 -- 172.21.15.86:0/3755362593 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc280b1040 con 0x7fdc3c0cfca0 2024-04-25T14:11:58.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.500+0000 7fdc33400640 1 -- 172.21.15.86:0/3755362593 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc280ba070 con 0x7fdc3c0cfca0 2024-04-25T14:11:58.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.500+0000 7fdc33400640 1 -- 172.21.15.86:0/3755362593 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc280b5650 con 0x7fdc3c0cfca0 2024-04-25T14:11:58.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.500+0000 7fdc42600640 1 -- 172.21.15.86:0/3755362593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc3c0c59b0 con 0x7fdc3c0cfca0 2024-04-25T14:11:58.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.504+0000 7fdc42600640 1 -- 172.21.15.86:0/3755362593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc3c0c5f70 con 0x7fdc3c0cfca0 2024-04-25T14:11:58.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.504+0000 7fdc42600640 1 -- 172.21.15.86:0/3755362593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc3c09f4b0 con 0x7fdc3c0cfca0 2024-04-25T14:11:58.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.504+0000 7fdc33400640 1 -- 172.21.15.86:0/3755362593 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 72) ==== 93318+0+0 (secure 0 0 0) 0x7fdc280b57f0 con 0x7fdc3c0cfca0 2024-04-25T14:11:58.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.504+0000 7fdc33400640 1 --2- 172.21.15.86:0/3755362593 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fdc1c06f5a0 0x7fdc1c071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:58.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.504+0000 7fdc33400640 1 -- 172.21.15.86:0/3755362593 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 6445+0+0 (secure 0 0 0) 0x7fdc28129540 con 0x7fdc3c0cfca0 2024-04-25T14:11:58.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.504+0000 7fdc3b400640 1 --2- 172.21.15.86:0/3755362593 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fdc1c06f5a0 0x7fdc1c071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:58.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.508+0000 7fdc3b400640 1 --2- 172.21.15.86:0/3755362593 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fdc1c06f5a0 0x7fdc1c071ab0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c070710 tx=0x7fdc2c008040 comp rx=0 tx=0).ready entity=mgr.9313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:58.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.512+0000 7fdc33400640 1 -- 172.21.15.86:0/3755362593 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdc28016090 con 0x7fdc3c0cfca0 2024-04-25T14:11:58.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.700+0000 7fdc42600640 1 -- 172.21.15.86:0/3755362593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc3c069d00 con 0x7fdc3c0cfca0 2024-04-25T14:11:58.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.700+0000 7fdc33400640 1 -- 172.21.15.86:0/3755362593 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5187 (secure 0 0 0) 0x7fdc280f5e90 con 0x7fdc3c0cfca0 2024-04-25T14:11:58.698 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:58.698 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":103,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/1828317917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1828317917},{"type":"v1","addr":"172.21.15.155:6834","nonce":1828317917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":9124,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1389714390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1389714390},{"type":"v1","addr":"172.21.15.86:6837","nonce":1389714390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":9136,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/91793677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":91793677},{"type":"v1","addr":"172.21.15.155:6835","nonce":91793677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:11:32.981538+0000","modified":"2024-04-25T14:11:35.996552+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9546},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9546":{"gid":9546,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/2571655765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2571655765},{"type":"v1","addr":"172.21.15.86:6835","nonce":2571655765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9546,"qdb_cluster":[9546]},"id":10}]} 2024-04-25T14:11:58.698 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 103 2024-04-25T14:11:58.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.704+0000 7fdc42600640 1 -- 172.21.15.86:0/3755362593 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fdc1c06f5a0 msgr2=0x7fdc1c071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:58.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.704+0000 7fdc42600640 1 --2- 172.21.15.86:0/3755362593 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fdc1c06f5a0 0x7fdc1c071ab0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c070710 tx=0x7fdc2c008040 comp rx=0 tx=0).stop 2024-04-25T14:11:58.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.704+0000 7fdc42600640 1 -- 172.21.15.86:0/3755362593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc3c0cfca0 msgr2=0x7fdc3c09e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:58.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.704+0000 7fdc42600640 1 --2- 172.21.15.86:0/3755362593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc3c0cfca0 0x7fdc3c09e100 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fdc280af040 tx=0x7fdc28019240 comp rx=0 tx=0).stop 2024-04-25T14:11:58.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.704+0000 7fdc42600640 1 -- 172.21.15.86:0/3755362593 shutdown_connections 2024-04-25T14:11:58.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.704+0000 7fdc42600640 1 --2- 172.21.15.86:0/3755362593 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fdc1c06f5a0 0x7fdc1c071ab0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:58.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.704+0000 7fdc42600640 1 --2- 172.21.15.86:0/3755362593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc3c0d16e0 0x7fdc3c068590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:58.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.704+0000 7fdc42600640 1 --2- 172.21.15.86:0/3755362593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc3c0cfca0 0x7fdc3c09e100 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:58.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.704+0000 7fdc42600640 1 --2- 172.21.15.86:0/3755362593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc3c06c670 0x7fdc3c09a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:58.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.704+0000 7fdc42600640 1 -- 172.21.15.86:0/3755362593 >> 172.21.15.86:0/3755362593 conn(0x7fdc3c0b4650 msgr2=0x7fdc3c0b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:58.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.704+0000 7fdc42600640 1 -- 172.21.15.86:0/3755362593 shutdown_connections 2024-04-25T14:11:58.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.704+0000 7fdc42600640 1 -- 172.21.15.86:0/3755362593 wait complete. 2024-04-25T14:11:58.720 DEBUG:tasks.cephfs.test_mds_metrics:metrics={'version': 2, 'global_counters': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'counters': [], 'client_metadata': {'cephfs': {'client.9292': {'hostname': 'smithi155', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'kernel_version': '6.9.0-rc3-g806ec36b2112', 'IP': 'v1:192.168.0.1'}, 'client.9732': {'hostname': 'smithi086', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'kernel_version': '6.9.0-rc3-g806ec36b2112', 'IP': 'v1:192.168.0.1'}, 'client.9792': {'hostname': 'smithi086', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'IP': '172.21.15.86'}, 'client.9795': {'hostname': 'smithi086', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'IP': '172.21.15.86'}}}, 'global_metrics': {'cephfs': {'client.9292': [[22, 0], [0, 0], [0, 0], [0, 5877839], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 1469461], [12322642740577, 4]], 'client.9732': [[3, 19], [0, 0], [0, 0], [0, 10916606], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 454864], [12082656693350, 24]], 'client.9792': [[2, 0], [0, 0], [0, 0], [0, 579907], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 579907], [0, 1]], 'client.9795': [[2, 0], [0, 0], [0, 0], [0, 802945], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 802945], [0, 1]]}}, 'metrics': {'delayed_ranks': [], 'mds.0': {'client.9292': [], 'client.9732': [], 'client.9792': [], 'client.9795': []}}} 2024-04-25T14:11:58.720 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:11:58.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.868+0000 7f265a000640 1 Processor -- start 2024-04-25T14:11:58.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.868+0000 7f265a000640 1 -- start start 2024-04-25T14:11:58.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.872+0000 7f265a000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26540cfca0 0x7f265406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:58.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.872+0000 7f265a000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f26540d16e0 0x7f26540d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:58.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.872+0000 7f265a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f265406c670 0x7f265406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:58.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.872+0000 7f265a000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2654070470 con 0x7f265406c670 2024-04-25T14:11:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.872+0000 7f265a000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f26540d2030 con 0x7f26540d16e0 2024-04-25T14:11:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.872+0000 7f265a000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2654072bd0 con 0x7f26540cfca0 2024-04-25T14:11:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.872+0000 7f2653e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f265406c670 0x7f265406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.872+0000 7f2653e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f265406c670 0x7f265406ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58336/0 (socket says 172.21.15.86:58336) 2024-04-25T14:11:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.872+0000 7f2653e00640 1 -- 172.21.15.86:0/2789974842 learned_addr learned my addr 172.21.15.86:0/2789974842 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.872+0000 7f264aa00640 1 --2- 172.21.15.86:0/2789974842 >> v2:172.21.15.155:3300/0 conn(0x7f26540d16e0 0x7f26540d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.872+0000 7f2653400640 1 --2- 172.21.15.86:0/2789974842 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26540cfca0 0x7f265406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:58.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.872+0000 7f2653e00640 1 -- 172.21.15.86:0/2789974842 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26540cfca0 msgr2=0x7f265406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:58.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.872+0000 7f2653e00640 1 --2- 172.21.15.86:0/2789974842 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26540cfca0 0x7f265406f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:58.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.872+0000 7f2653e00640 1 -- 172.21.15.86:0/2789974842 >> v2:172.21.15.155:3300/0 conn(0x7f26540d16e0 msgr2=0x7f26540d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:58.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.872+0000 7f2653e00640 1 --2- 172.21.15.86:0/2789974842 >> v2:172.21.15.155:3300/0 conn(0x7f26540d16e0 0x7f26540d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:58.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.872+0000 7f2653e00640 1 -- 172.21.15.86:0/2789974842 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265406f930 con 0x7f265406c670 2024-04-25T14:11:58.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.876+0000 7f2653e00640 1 --2- 172.21.15.86:0/2789974842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f265406c670 0x7f265406ca80 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f2654070f40 tx=0x7f264c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182c6b1bb7a6a48b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:58.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.876+0000 7f2652a00640 1 -- 172.21.15.86:0/2789974842 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c0a5780 con 0x7f265406c670 2024-04-25T14:11:58.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.876+0000 7f2652a00640 1 -- 172.21.15.86:0/2789974842 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f264c0a5920 con 0x7f265406c670 2024-04-25T14:11:58.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.876+0000 7f2652a00640 1 -- 172.21.15.86:0/2789974842 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c0a5cf0 con 0x7f265406c670 2024-04-25T14:11:58.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.876+0000 7f265a000640 1 -- 172.21.15.86:0/2789974842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f265406c670 msgr2=0x7f265406ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:58.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.876+0000 7f265a000640 1 --2- 172.21.15.86:0/2789974842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f265406c670 0x7f265406ca80 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f2654070f40 tx=0x7f264c01c9f0 comp rx=0 tx=0).stop 2024-04-25T14:11:58.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.876+0000 7f265a000640 1 -- 172.21.15.86:0/2789974842 shutdown_connections 2024-04-25T14:11:58.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.876+0000 7f265a000640 1 --2- 172.21.15.86:0/2789974842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f265406c670 0x7f265406ca80 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:58.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.876+0000 7f265a000640 1 --2- 172.21.15.86:0/2789974842 >> v2:172.21.15.155:3300/0 conn(0x7f26540d16e0 0x7f26540d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:58.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.876+0000 7f265a000640 1 --2- 172.21.15.86:0/2789974842 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26540cfca0 0x7f265406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:58.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.876+0000 7f265a000640 1 -- 172.21.15.86:0/2789974842 >> 172.21.15.86:0/2789974842 conn(0x7f26540b4650 msgr2=0x7f26540b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:58.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.876+0000 7f265a000640 1 -- 172.21.15.86:0/2789974842 shutdown_connections 2024-04-25T14:11:58.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.876+0000 7f265a000640 1 -- 172.21.15.86:0/2789974842 wait complete. 2024-04-25T14:11:58.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.880+0000 7f265a000640 1 Processor -- start 2024-04-25T14:11:58.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.880+0000 7f265a000640 1 -- start start 2024-04-25T14:11:58.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.880+0000 7f265a000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f265406c670 0x7f265409aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:58.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.880+0000 7f265a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26540cfca0 0x7f265415ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:58.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.880+0000 7f265a000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26540d16e0 0x7f265415df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:58.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.880+0000 7f265a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f26540738c0 con 0x7f26540cfca0 2024-04-25T14:11:58.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.880+0000 7f265a000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f265406d3d0 con 0x7f26540d16e0 2024-04-25T14:11:58.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.880+0000 7f265a000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2654073a40 con 0x7f265406c670 2024-04-25T14:11:58.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.880+0000 7f264aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26540cfca0 0x7f265415ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:58.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.880+0000 7f264aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26540cfca0 0x7f265415ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58338/0 (socket says 172.21.15.86:58338) 2024-04-25T14:11:58.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.880+0000 7f264aa00640 1 -- 172.21.15.86:0/2012724132 learned_addr learned my addr 172.21.15.86:0/2012724132 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:58.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.880+0000 7f2653400640 1 --2- 172.21.15.86:0/2012724132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f265406c670 0x7f265409aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:58.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.880+0000 7f2653e00640 1 --2- 172.21.15.86:0/2012724132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26540d16e0 0x7f265415df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:58.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.880+0000 7f264aa00640 1 -- 172.21.15.86:0/2012724132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f265406c670 msgr2=0x7f265409aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:58.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.880+0000 7f264aa00640 1 --2- 172.21.15.86:0/2012724132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f265406c670 0x7f265409aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:58.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.880+0000 7f264aa00640 1 -- 172.21.15.86:0/2012724132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26540d16e0 msgr2=0x7f265415df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:58.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.880+0000 7f264aa00640 1 --2- 172.21.15.86:0/2012724132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26540d16e0 0x7f265415df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:58.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.880+0000 7f264aa00640 1 -- 172.21.15.86:0/2012724132 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265415ebf0 con 0x7f26540cfca0 2024-04-25T14:11:58.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.880+0000 7f264aa00640 1 --2- 172.21.15.86:0/2012724132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26540cfca0 0x7f265415ff10 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f263c00e870 tx=0x7f263c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:58.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.880+0000 7f264be00640 1 -- 172.21.15.86:0/2012724132 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c017070 con 0x7f26540cfca0 2024-04-25T14:11:58.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.880+0000 7f264be00640 1 -- 172.21.15.86:0/2012724132 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f263c00c940 con 0x7f26540cfca0 2024-04-25T14:11:58.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.880+0000 7f265a000640 1 -- 172.21.15.86:0/2012724132 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265415ee90 con 0x7f26540cfca0 2024-04-25T14:11:58.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.880+0000 7f264be00640 1 -- 172.21.15.86:0/2012724132 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c00ccf0 con 0x7f26540cfca0 2024-04-25T14:11:58.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.880+0000 7f265a000640 1 -- 172.21.15.86:0/2012724132 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f265415d590 con 0x7f26540cfca0 2024-04-25T14:11:58.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.884+0000 7f265a000640 1 -- 172.21.15.86:0/2012724132 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f265415da00 con 0x7f26540cfca0 2024-04-25T14:11:58.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.884+0000 7f264be00640 1 -- 172.21.15.86:0/2012724132 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 72) ==== 93318+0+0 (secure 0 0 0) 0x7f263c01a070 con 0x7f26540cfca0 2024-04-25T14:11:58.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.884+0000 7f264be00640 1 --2- 172.21.15.86:0/2012724132 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f263006f570 0x7f2630071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:58.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.884+0000 7f264be00640 1 -- 172.21.15.86:0/2012724132 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 6445+0+0 (secure 0 0 0) 0x7f263c08f530 con 0x7f26540cfca0 2024-04-25T14:11:58.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.884+0000 7f2653400640 1 --2- 172.21.15.86:0/2012724132 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f263006f570 0x7f2630071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:58.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.888+0000 7f2653400640 1 --2- 172.21.15.86:0/2012724132 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f263006f570 0x7f2630071a80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2654070710 tx=0x7f2644009190 comp rx=0 tx=0).ready entity=mgr.9313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:58.888+0000 7f264be00640 1 -- 172.21.15.86:0/2012724132 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f263c05b9d0 con 0x7f26540cfca0 2024-04-25T14:11:59.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.080+0000 7f265a000640 1 -- 172.21.15.86:0/2012724132 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2654160450 con 0x7f26540cfca0 2024-04-25T14:11:59.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.080+0000 7f264be00640 1 -- 172.21.15.86:0/2012724132 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5187 (secure 0 0 0) 0x7f263c05fcd0 con 0x7f26540cfca0 2024-04-25T14:11:59.078 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:11:59.078 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":103,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/1828317917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1828317917},{"type":"v1","addr":"172.21.15.155:6834","nonce":1828317917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":9124,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1389714390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1389714390},{"type":"v1","addr":"172.21.15.86:6837","nonce":1389714390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":9136,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/91793677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":91793677},{"type":"v1","addr":"172.21.15.155:6835","nonce":91793677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:11:32.981538+0000","modified":"2024-04-25T14:11:35.996552+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9546},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9546":{"gid":9546,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/2571655765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2571655765},{"type":"v1","addr":"172.21.15.86:6835","nonce":2571655765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9546,"qdb_cluster":[9546]},"id":10}]} 2024-04-25T14:11:59.078 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 103 2024-04-25T14:11:59.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.084+0000 7f265a000640 1 -- 172.21.15.86:0/2012724132 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f263006f570 msgr2=0x7f2630071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:59.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.084+0000 7f265a000640 1 --2- 172.21.15.86:0/2012724132 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f263006f570 0x7f2630071a80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2654070710 tx=0x7f2644009190 comp rx=0 tx=0).stop 2024-04-25T14:11:59.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.084+0000 7f265a000640 1 -- 172.21.15.86:0/2012724132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26540cfca0 msgr2=0x7f265415ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:59.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.084+0000 7f265a000640 1 --2- 172.21.15.86:0/2012724132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26540cfca0 0x7f265415ff10 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f263c00e870 tx=0x7f263c00ec70 comp rx=0 tx=0).stop 2024-04-25T14:11:59.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.084+0000 7f265a000640 1 -- 172.21.15.86:0/2012724132 shutdown_connections 2024-04-25T14:11:59.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.084+0000 7f265a000640 1 --2- 172.21.15.86:0/2012724132 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f263006f570 0x7f2630071a80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:59.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.084+0000 7f265a000640 1 --2- 172.21.15.86:0/2012724132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26540d16e0 0x7f265415df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:59.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.084+0000 7f265a000640 1 --2- 172.21.15.86:0/2012724132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26540cfca0 0x7f265415ff10 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:59.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.084+0000 7f265a000640 1 --2- 172.21.15.86:0/2012724132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f265406c670 0x7f265409aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:59.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.084+0000 7f265a000640 1 -- 172.21.15.86:0/2012724132 >> 172.21.15.86:0/2012724132 conn(0x7f26540b4650 msgr2=0x7f26540ae380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:59.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.084+0000 7f265a000640 1 -- 172.21.15.86:0/2012724132 shutdown_connections 2024-04-25T14:11:59.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.084+0000 7f265a000640 1 -- 172.21.15.86:0/2012724132 wait complete. 2024-04-25T14:11:59.100 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2024-04-25T14:11:59.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.252+0000 7ff28e400640 1 Processor -- start 2024-04-25T14:11:59.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.252+0000 7ff28e400640 1 -- start start 2024-04-25T14:11:59.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.256+0000 7ff28e400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff2880cf970 0x7ff288070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:59.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.256+0000 7ff28e400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff288071360 0x7ff28806e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:59.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.256+0000 7ff28e400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff288071770 con 0x7ff28806e7a0 2024-04-25T14:11:59.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.256+0000 7ff28e400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff2880cedf0 con 0x7ff2880cf970 2024-04-25T14:11:59.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.256+0000 7ff28e400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff288163830 con 0x7ff288071360 2024-04-25T14:11:59.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.256+0000 7ff287400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff2880cf970 0x7ff288070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:59.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.256+0000 7ff287400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff2880cf970 0x7ff288070e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58350/0 (socket says 172.21.15.86:58350) 2024-04-25T14:11:59.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.256+0000 7ff287400640 1 -- 172.21.15.86:0/3357759696 learned_addr learned my addr 172.21.15.86:0/3357759696 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:59.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.256+0000 7ff27ea00640 1 --2- 172.21.15.86:0/3357759696 >> v2:172.21.15.155:3300/0 conn(0x7ff288071360 0x7ff28806e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:59.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.256+0000 7ff287400640 1 -- 172.21.15.86:0/3357759696 >> v2:172.21.15.155:3300/0 conn(0x7ff288071360 msgr2=0x7ff28806e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:59.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.256+0000 7ff287400640 1 --2- 172.21.15.86:0/3357759696 >> v2:172.21.15.155:3300/0 conn(0x7ff288071360 0x7ff28806e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:59.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.256+0000 7ff287400640 1 -- 172.21.15.86:0/3357759696 >> v1:172.21.15.155:6789/0 conn(0x7ff28806e7a0 legacy=0x7ff2880722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:59.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.256+0000 7ff287400640 1 -- 172.21.15.86:0/3357759696 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28806ef90 con 0x7ff2880cf970 2024-04-25T14:11:59.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.256+0000 7ff27ea00640 1 --2- 172.21.15.86:0/3357759696 >> v2:172.21.15.155:3300/0 conn(0x7ff288071360 0x7ff28806e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.256+0000 7ff287400640 1 --2- 172.21.15.86:0/3357759696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2880cf970 0x7ff288070e20 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7ff28806d8a0 tx=0x7ff27801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9297af6dfea3b73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:59.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.256+0000 7ff286a00640 1 -- 172.21.15.86:0/3357759696 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2780a57a0 con 0x7ff2880cf970 2024-04-25T14:11:59.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.260+0000 7ff286a00640 1 -- 172.21.15.86:0/3357759696 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2780a5940 con 0x7ff2880cf970 2024-04-25T14:11:59.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.260+0000 7ff286a00640 1 -- 172.21.15.86:0/3357759696 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2780a5d10 con 0x7ff2880cf970 2024-04-25T14:11:59.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.260+0000 7ff28e400640 1 -- 172.21.15.86:0/3357759696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2880cf970 msgr2=0x7ff288070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:59.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.260+0000 7ff28e400640 1 --2- 172.21.15.86:0/3357759696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2880cf970 0x7ff288070e20 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7ff28806d8a0 tx=0x7ff27801c9c0 comp rx=0 tx=0).stop 2024-04-25T14:11:59.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.260+0000 7ff28e400640 1 -- 172.21.15.86:0/3357759696 shutdown_connections 2024-04-25T14:11:59.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.260+0000 7ff28e400640 1 --2- 172.21.15.86:0/3357759696 >> v2:172.21.15.155:3300/0 conn(0x7ff288071360 0x7ff28806e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:59.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.260+0000 7ff28e400640 1 --2- 172.21.15.86:0/3357759696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2880cf970 0x7ff288070e20 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:59.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.260+0000 7ff28e400640 1 -- 172.21.15.86:0/3357759696 >> 172.21.15.86:0/3357759696 conn(0x7ff2880cffc0 msgr2=0x7ff2880d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:59.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.260+0000 7ff28e400640 1 -- 172.21.15.86:0/3357759696 shutdown_connections 2024-04-25T14:11:59.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.260+0000 7ff28e400640 1 -- 172.21.15.86:0/3357759696 wait complete. 2024-04-25T14:11:59.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.260+0000 7ff28e400640 1 Processor -- start 2024-04-25T14:11:59.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.260+0000 7ff28e400640 1 -- start start 2024-04-25T14:11:59.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.264+0000 7ff28e400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff28806e7a0 0x7ff2880c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:59.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.264+0000 7ff28e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff288071360 0x7ff2880b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:59.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.264+0000 7ff28e400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2880cf970 0x7ff2880b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:59.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.264+0000 7ff28e400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff28806ed30 con 0x7ff288071360 2024-04-25T14:11:59.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.264+0000 7ff28e400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff28806ebb0 con 0x7ff2880cf970 2024-04-25T14:11:59.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.264+0000 7ff28e400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff28816cbd0 con 0x7ff28806e7a0 2024-04-25T14:11:59.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.264+0000 7ff27ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff288071360 0x7ff2880b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:59.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.264+0000 7ff27ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff288071360 0x7ff2880b6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58354/0 (socket says 172.21.15.86:58354) 2024-04-25T14:11:59.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.264+0000 7ff27ea00640 1 -- 172.21.15.86:0/301370542 learned_addr learned my addr 172.21.15.86:0/301370542 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:11:59.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.264+0000 7ff287400640 1 --2- 172.21.15.86:0/301370542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff28806e7a0 0x7ff2880c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:59.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.264+0000 7ff287e00640 1 --2- 172.21.15.86:0/301370542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2880cf970 0x7ff2880b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.264+0000 7ff27ea00640 1 -- 172.21.15.86:0/301370542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff28806e7a0 msgr2=0x7ff2880c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.264+0000 7ff27ea00640 1 --2- 172.21.15.86:0/301370542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff28806e7a0 0x7ff2880c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.264+0000 7ff27ea00640 1 -- 172.21.15.86:0/301370542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2880cf970 msgr2=0x7ff2880b66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.264+0000 7ff27ea00640 1 --2- 172.21.15.86:0/301370542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2880cf970 0x7ff2880b66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.264+0000 7ff27ea00640 1 -- 172.21.15.86:0/301370542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2880af200 con 0x7ff288071360 2024-04-25T14:11:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.264+0000 7ff287400640 1 --2- 172.21.15.86:0/301370542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff28806e7a0 0x7ff2880c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.264+0000 7ff27ea00640 1 --2- 172.21.15.86:0/301370542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff288071360 0x7ff2880b6160 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7ff27000d800 tx=0x7ff27000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.264+0000 7ff27fe00640 1 -- 172.21.15.86:0/301370542 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff270018020 con 0x7ff288071360 2024-04-25T14:11:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.264+0000 7ff27fe00640 1 -- 172.21.15.86:0/301370542 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2700059a0 con 0x7ff288071360 2024-04-25T14:11:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.264+0000 7ff27fe00640 1 -- 172.21.15.86:0/301370542 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff270005d70 con 0x7ff288071360 2024-04-25T14:11:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.264+0000 7ff28e400640 1 -- 172.21.15.86:0/301370542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2880af4e0 con 0x7ff288071360 2024-04-25T14:11:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.264+0000 7ff28e400640 1 -- 172.21.15.86:0/301370542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2880afaa0 con 0x7ff288071360 2024-04-25T14:11:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.264+0000 7ff28e400640 1 -- 172.21.15.86:0/301370542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2880720e0 con 0x7ff288071360 2024-04-25T14:11:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.268+0000 7ff27fe00640 1 -- 172.21.15.86:0/301370542 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 72) ==== 93318+0+0 (secure 0 0 0) 0x7ff270004030 con 0x7ff288071360 2024-04-25T14:11:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.268+0000 7ff27fe00640 1 --2- 172.21.15.86:0/301370542 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7ff26406f570 0x7ff264071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.268+0000 7ff27fe00640 1 -- 172.21.15.86:0/301370542 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 6445+0+0 (secure 0 0 0) 0x7ff27008f7c0 con 0x7ff288071360 2024-04-25T14:11:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.268+0000 7ff287400640 1 --2- 172.21.15.86:0/301370542 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7ff26406f570 0x7ff264071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:59.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.268+0000 7ff287400640 1 --2- 172.21.15.86:0/301370542 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7ff26406f570 0x7ff264071a80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff2780ae000 tx=0x7ff2780a7070 comp rx=0 tx=0).ready entity=mgr.9313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:59.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.272+0000 7ff27fe00640 1 -- 172.21.15.86:0/301370542 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff270022020 con 0x7ff288071360 2024-04-25T14:11:59.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.464+0000 7ff28e400640 1 -- 172.21.15.86:0/301370542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) -- 0x7ff2880aef50 con 0x7ff288071360 2024-04-25T14:11:59.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.824+0000 7ff27fe00640 1 -- 172.21.15.86:0/301370542 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v104) ==== 105+0+0 (secure 0 0 0) 0x7ff27005c340 con 0x7ff288071360 2024-04-25T14:11:59.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.828+0000 7ff28e400640 1 -- 172.21.15.86:0/301370542 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7ff26406f570 msgr2=0x7ff264071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:59.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.828+0000 7ff28e400640 1 --2- 172.21.15.86:0/301370542 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7ff26406f570 0x7ff264071a80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff2780ae000 tx=0x7ff2780a7070 comp rx=0 tx=0).stop 2024-04-25T14:11:59.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.828+0000 7ff28e400640 1 -- 172.21.15.86:0/301370542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff288071360 msgr2=0x7ff2880b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:59.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.828+0000 7ff28e400640 1 --2- 172.21.15.86:0/301370542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff288071360 0x7ff2880b6160 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7ff27000d800 tx=0x7ff27000dcc0 comp rx=0 tx=0).stop 2024-04-25T14:11:59.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.828+0000 7ff28e400640 1 -- 172.21.15.86:0/301370542 shutdown_connections 2024-04-25T14:11:59.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.828+0000 7ff28e400640 1 --2- 172.21.15.86:0/301370542 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7ff26406f570 0x7ff264071a80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:59.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.828+0000 7ff28e400640 1 --2- 172.21.15.86:0/301370542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2880cf970 0x7ff2880b66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:59.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.828+0000 7ff28e400640 1 --2- 172.21.15.86:0/301370542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff288071360 0x7ff2880b6160 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:59.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.828+0000 7ff28e400640 1 --2- 172.21.15.86:0/301370542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff28806e7a0 0x7ff2880c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:59.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.828+0000 7ff28e400640 1 -- 172.21.15.86:0/301370542 >> 172.21.15.86:0/301370542 conn(0x7ff2880cffc0 msgr2=0x7ff2880b5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:59.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.828+0000 7ff28e400640 1 -- 172.21.15.86:0/301370542 shutdown_connections 2024-04-25T14:11:59.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:11:59.828+0000 7ff28e400640 1 -- 172.21.15.86:0/301370542 wait complete. 2024-04-25T14:11:59.849 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:11:59.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.000+0000 7f7b76800640 1 Processor -- start 2024-04-25T14:11:59.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.000+0000 7f7b76800640 1 -- start start 2024-04-25T14:11:59.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.000+0000 7f7b76800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7b701642a0 0x7f7b700d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:59.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.000+0000 7f7b76800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b700b0860 0x7f7b700d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:59.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.000+0000 7f7b76800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b700d2030 con 0x7f7b700d1c20 2024-04-25T14:11:59.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.000+0000 7f7b76800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f7b700cea30 con 0x7f7b701642a0 2024-04-25T14:11:59.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.000+0000 7f7b76800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7b700cde20 con 0x7f7b700b0860 2024-04-25T14:11:59.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.000+0000 7f7b6f400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7b701642a0 0x7f7b700d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:59.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.000+0000 7f7b6fe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b700b0860 0x7f7b700d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:59.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.000+0000 7f7b67e00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f7b700d1c20 0x7f7b700b0150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:37750/0 (socket says 172.21.15.86:37750) 2024-04-25T14:11:59.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.000+0000 7f7b67e00640 1 -- 172.21.15.86:0/3989677979 learned_addr learned my addr 172.21.15.86:0/3989677979 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:11:59.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.000+0000 7f7b6fe00640 1 -- 172.21.15.86:0/3989677979 >> v2:172.21.15.155:3300/0 conn(0x7f7b701642a0 msgr2=0x7f7b700d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:59.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.000+0000 7f7b6fe00640 1 --2- 172.21.15.86:0/3989677979 >> v2:172.21.15.155:3300/0 conn(0x7f7b701642a0 0x7f7b700d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:59.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.000+0000 7f7b6fe00640 1 -- 172.21.15.86:0/3989677979 >> v1:172.21.15.155:6789/0 conn(0x7f7b700d1c20 legacy=0x7f7b700b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:59.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.000+0000 7f7b6fe00640 1 -- 172.21.15.86:0/3989677979 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b700ce100 con 0x7f7b700b0860 2024-04-25T14:11:59.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.000+0000 7f7b6ea00640 1 -- 172.21.15.86:0/3989677979 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2004937569 0 0) 0x7f7b58001120 con 0x7f7b700d1c20 2024-04-25T14:11:59.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.000+0000 7f7b6f400640 1 --2- 172.21.15.86:0/3989677979 >> v2:172.21.15.155:3300/0 conn(0x7f7b701642a0 0x7f7b700d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:12:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.004+0000 7f7b6fe00640 1 --2- 172.21.15.86:0/3989677979 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b700b0860 0x7f7b700d0640 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7b68009d90 tx=0x7f7b6801c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0c5b566a4fc953b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:00.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.004+0000 7f7b6ea00640 1 -- 172.21.15.86:0/3989677979 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b680b2070 con 0x7f7b700b0860 2024-04-25T14:12:00.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.004+0000 7f7b6ea00640 1 -- 172.21.15.86:0/3989677979 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b680a57a0 con 0x7f7b700b0860 2024-04-25T14:12:00.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.004+0000 7f7b6ea00640 1 -- 172.21.15.86:0/3989677979 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b680a5b70 con 0x7f7b700b0860 2024-04-25T14:12:00.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.004+0000 7f7b76800640 1 -- 172.21.15.86:0/3989677979 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b700b0860 msgr2=0x7f7b700d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:00.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.004+0000 7f7b76800640 1 --2- 172.21.15.86:0/3989677979 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b700b0860 0x7f7b700d0640 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7b68009d90 tx=0x7f7b6801c8e0 comp rx=0 tx=0).stop 2024-04-25T14:12:00.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.004+0000 7f7b76800640 1 -- 172.21.15.86:0/3989677979 shutdown_connections 2024-04-25T14:12:00.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.004+0000 7f7b76800640 1 --2- 172.21.15.86:0/3989677979 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b700b0860 0x7f7b700d0640 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:00.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.004+0000 7f7b76800640 1 --2- 172.21.15.86:0/3989677979 >> v2:172.21.15.155:3300/0 conn(0x7f7b701642a0 0x7f7b700d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:00.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.004+0000 7f7b76800640 1 -- 172.21.15.86:0/3989677979 >> 172.21.15.86:0/3989677979 conn(0x7f7b700b4650 msgr2=0x7f7b700b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:00.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.004+0000 7f7b76800640 1 -- 172.21.15.86:0/3989677979 shutdown_connections 2024-04-25T14:12:00.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.004+0000 7f7b76800640 1 -- 172.21.15.86:0/3989677979 wait complete. 2024-04-25T14:12:00.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.008+0000 7f7b76800640 1 Processor -- start 2024-04-25T14:12:00.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.008+0000 7f7b76800640 1 -- start start 2024-04-25T14:12:00.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.008+0000 7f7b76800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b700b0860 0x7f7b701fd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:00.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.008+0000 7f7b76800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b700d1c20 0x7f7b7015d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:00.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.008+0000 7f7b76800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b701642a0 0x7f7b700ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:00.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.008+0000 7f7b76800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7b70164830 con 0x7f7b700d1c20 2024-04-25T14:12:00.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.008+0000 7f7b76800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7b701646b0 con 0x7f7b700b0860 2024-04-25T14:12:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.008+0000 7f7b76800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7b70160da0 con 0x7f7b701642a0 2024-04-25T14:12:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.008+0000 7f7b67e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b700d1c20 0x7f7b7015d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.008+0000 7f7b67e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b700d1c20 0x7f7b7015d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58362/0 (socket says 172.21.15.86:58362) 2024-04-25T14:12:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.008+0000 7f7b67e00640 1 -- 172.21.15.86:0/3719187068 learned_addr learned my addr 172.21.15.86:0/3719187068 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:12:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.008+0000 7f7b6f400640 1 --2- 172.21.15.86:0/3719187068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b700b0860 0x7f7b701fd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.008+0000 7f7b6fe00640 1 --2- 172.21.15.86:0/3719187068 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b701642a0 0x7f7b700ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.008+0000 7f7b67e00640 1 -- 172.21.15.86:0/3719187068 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b701642a0 msgr2=0x7f7b700ae200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.008+0000 7f7b67e00640 1 --2- 172.21.15.86:0/3719187068 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b701642a0 0x7f7b700ae200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.008+0000 7f7b67e00640 1 -- 172.21.15.86:0/3719187068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b700b0860 msgr2=0x7f7b701fd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.008+0000 7f7b67e00640 1 --2- 172.21.15.86:0/3719187068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b700b0860 0x7f7b701fd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.008+0000 7f7b67e00640 1 -- 172.21.15.86:0/3719187068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b701fd6c0 con 0x7f7b700d1c20 2024-04-25T14:12:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.008+0000 7f7b6f400640 1 --2- 172.21.15.86:0/3719187068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b700b0860 0x7f7b701fd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:12:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.008+0000 7f7b6fe00640 1 --2- 172.21.15.86:0/3719187068 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b701642a0 0x7f7b700ae200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:12:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.012+0000 7f7b67e00640 1 --2- 172.21.15.86:0/3719187068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b700d1c20 0x7f7b7015d030 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f7b5800c480 tx=0x7f7b5800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:00.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.012+0000 7f7b67400640 1 -- 172.21.15.86:0/3719187068 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b58018070 con 0x7f7b700d1c20 2024-04-25T14:12:00.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.012+0000 7f7b67400640 1 -- 172.21.15.86:0/3719187068 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b580074e0 con 0x7f7b700d1c20 2024-04-25T14:12:00.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.012+0000 7f7b67400640 1 -- 172.21.15.86:0/3719187068 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b580049f0 con 0x7f7b700d1c20 2024-04-25T14:12:00.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.012+0000 7f7b76800640 1 -- 172.21.15.86:0/3719187068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b7015c690 con 0x7f7b700d1c20 2024-04-25T14:12:00.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.012+0000 7f7b76800640 1 -- 172.21.15.86:0/3719187068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b7015e110 con 0x7f7b700d1c20 2024-04-25T14:12:00.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.012+0000 7f7b76800640 1 -- 172.21.15.86:0/3719187068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b7015cb90 con 0x7f7b700d1c20 2024-04-25T14:12:00.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.012+0000 7f7b67400640 1 -- 172.21.15.86:0/3719187068 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 72) ==== 93318+0+0 (secure 0 0 0) 0x7f7b58004b90 con 0x7f7b700d1c20 2024-04-25T14:12:00.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.012+0000 7f7b67400640 1 --2- 172.21.15.86:0/3719187068 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f7b4c06f570 0x7f7b4c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:00.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.012+0000 7f7b67400640 1 -- 172.21.15.86:0/3719187068 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 6445+0+0 (secure 0 0 0) 0x7f7b5808ead0 con 0x7f7b700d1c20 2024-04-25T14:12:00.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.012+0000 7f7b6f400640 1 --2- 172.21.15.86:0/3719187068 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f7b4c06f570 0x7f7b4c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:00.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.016+0000 7f7b6f400640 1 --2- 172.21.15.86:0/3719187068 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f7b4c06f570 0x7f7b4c071a80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7b70070ac0 tx=0x7f7b60008040 comp rx=0 tx=0).ready entity=mgr.9313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:00.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.016+0000 7f7b67400640 1 -- 172.21.15.86:0/3719187068 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7b700d2030 con 0x7f7b700d1c20 2024-04-25T14:12:00.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.208+0000 7f7b76800640 1 -- 172.21.15.86:0/3719187068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b70069d00 con 0x7f7b700d1c20 2024-04-25T14:12:00.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.208+0000 7f7b67400640 1 -- 172.21.15.86:0/3719187068 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5204 (secure 0 0 0) 0x7f7b5805b6f0 con 0x7f7b700d1c20 2024-04-25T14:12:00.207 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:12:00.208 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":105,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/1828317917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1828317917},{"type":"v1","addr":"172.21.15.155:6834","nonce":1828317917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":9124,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1389714390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1389714390},{"type":"v1","addr":"172.21.15.86:6837","nonce":1389714390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:11:32.981538+0000","modified":"2024-04-25T14:11:59.832275+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":9546,"mds_1":9136},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9136":{"gid":9136,"name":"d","rank":1,"incarnation":105,"state":"up:creating","state_seq":1,"addr":"172.21.15.155:6835/91793677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":91793677},{"type":"v1","addr":"172.21.15.155:6835","nonce":91793677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9546":{"gid":9546,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/2571655765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2571655765},{"type":"v1","addr":"172.21.15.86:6835","nonce":2571655765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9546,"qdb_cluster":[9546]},"id":10}]} 2024-04-25T14:12:00.208 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 105 2024-04-25T14:12:00.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.212+0000 7f7b76800640 1 -- 172.21.15.86:0/3719187068 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f7b4c06f570 msgr2=0x7f7b4c071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:00.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.212+0000 7f7b76800640 1 --2- 172.21.15.86:0/3719187068 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f7b4c06f570 0x7f7b4c071a80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7b70070ac0 tx=0x7f7b60008040 comp rx=0 tx=0).stop 2024-04-25T14:12:00.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.212+0000 7f7b76800640 1 -- 172.21.15.86:0/3719187068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b700d1c20 msgr2=0x7f7b7015d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:00.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.212+0000 7f7b76800640 1 --2- 172.21.15.86:0/3719187068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b700d1c20 0x7f7b7015d030 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f7b5800c480 tx=0x7f7b5800c940 comp rx=0 tx=0).stop 2024-04-25T14:12:00.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.212+0000 7f7b76800640 1 -- 172.21.15.86:0/3719187068 shutdown_connections 2024-04-25T14:12:00.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.212+0000 7f7b76800640 1 --2- 172.21.15.86:0/3719187068 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f7b4c06f570 0x7f7b4c071a80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:00.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.212+0000 7f7b76800640 1 --2- 172.21.15.86:0/3719187068 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b701642a0 0x7f7b700ae200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:00.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.212+0000 7f7b76800640 1 --2- 172.21.15.86:0/3719187068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b700d1c20 0x7f7b7015d030 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:00.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.212+0000 7f7b76800640 1 --2- 172.21.15.86:0/3719187068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b700b0860 0x7f7b701fd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:00.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.212+0000 7f7b76800640 1 -- 172.21.15.86:0/3719187068 >> 172.21.15.86:0/3719187068 conn(0x7f7b700b4650 msgr2=0x7f7b70161dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:00.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.212+0000 7f7b76800640 1 -- 172.21.15.86:0/3719187068 shutdown_connections 2024-04-25T14:12:00.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:00.212+0000 7f7b76800640 1 -- 172.21.15.86:0/3719187068 wait complete. 2024-04-25T14:12:00.632 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:11:32.981538+0000', 'modified': '2024-04-25T14:11:59.832275+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 9546, 'mds_1': 9136}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9136': {'gid': 9136, 'name': 'd', 'rank': 1, 'incarnation': 105, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.155:6835/91793677', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6833', 'nonce': 91793677}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 91793677}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_9546': {'gid': 9546, 'name': 'a', 'rank': 0, 'incarnation': 99, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6835/2571655765', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6834', 'nonce': 2571655765}, {'type': 'v1', 'addr': '172.21.15.86:6835', 'nonce': 2571655765}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9546, 'qdb_cluster': [9546]} 2024-04-25T14:12:00.632 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9136:up:creating 2024-04-25T14:12:01.633 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:12:01.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.796+0000 7fddf4e00640 1 Processor -- start 2024-04-25T14:12:01.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.796+0000 7fddf4e00640 1 -- start start 2024-04-25T14:12:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.796+0000 7fddf4e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddf00ccab0 0x7fddf00d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.796+0000 7fddf4e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddf0162610 con 0x7fddf00cf9d0 2024-04-25T14:12:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.796+0000 7fddf4e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddf00b3bd0 con 0x7fddf00b79c0 2024-04-25T14:12:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.796+0000 7fddf4e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fddf00cf190 con 0x7fddf00ccab0 2024-04-25T14:12:01.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.796+0000 7fddee800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fddf00cf9d0 0x7fddf0161f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41616/0 (socket says 172.21.15.86:41616) 2024-04-25T14:12:01.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.796+0000 7fddee800640 1 -- 172.21.15.86:0/1532115658 learned_addr learned my addr 172.21.15.86:0/1532115658 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:12:01.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.800+0000 7fddede00640 1 --2- 172.21.15.86:0/1532115658 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddf00ccab0 0x7fddf00d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:01.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.800+0000 7fdded400640 1 -- 172.21.15.86:0/1532115658 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1837589581 0 0) 0x7fddf0162610 con 0x7fddf00cf9d0 2024-04-25T14:12:01.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.800+0000 7fdded400640 1 -- 172.21.15.86:0/1532115658 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddd4003530 con 0x7fddf00cf9d0 2024-04-25T14:12:01.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.800+0000 7fdded400640 1 -- 172.21.15.86:0/1532115658 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 632947739 0 0) 0x7fddf00b3bd0 con 0x7fddf00b79c0 2024-04-25T14:12:01.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.800+0000 7fdded400640 1 -- 172.21.15.86:0/1532115658 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddf0162610 con 0x7fddf00b79c0 2024-04-25T14:12:01.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.800+0000 7fdded400640 1 -- 172.21.15.86:0/1532115658 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3921116766 0 0) 0x7fddd4003530 con 0x7fddf00cf9d0 2024-04-25T14:12:01.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.800+0000 7fdded400640 1 -- 172.21.15.86:0/1532115658 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddf00ccab0 msgr2=0x7fddf00d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:12:01.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.800+0000 7fdded400640 1 --2- 172.21.15.86:0/1532115658 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddf00ccab0 0x7fddf00d23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:01.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.800+0000 7fdded400640 1 -- 172.21.15.86:0/1532115658 >> v1:172.21.15.155:6789/0 conn(0x7fddf00b79c0 legacy=0x7fddf01617f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:01.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.800+0000 7fdded400640 1 -- 172.21.15.86:0/1532115658 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf00b1cf0 con 0x7fddf00cf9d0 2024-04-25T14:12:01.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.800+0000 7fdded400640 1 -- 172.21.15.86:0/1532115658 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fdde8001680 con 0x7fddf00cf9d0 2024-04-25T14:12:01.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.800+0000 7fdded400640 1 -- 172.21.15.86:0/1532115658 >> v1:172.21.15.86:6789/0 conn(0x7fddf00cf9d0 legacy=0x7fddf0161f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:01.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.800+0000 7fdded400640 1 --2- 172.21.15.86:0/1532115658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddd4003bf0 0x7fddd4004040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:01.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.800+0000 7fdded400640 1 --2- 172.21.15.86:0/1532115658 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddf00ccab0 0x7fddd4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:01.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.800+0000 7fdded400640 1 --2- 172.21.15.86:0/1532115658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddd4004d50 0x7fddd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:01.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.800+0000 7fdded400640 1 -- 172.21.15.86:0/1532115658 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fddf00cf190 con 0x7fddd4003bf0 2024-04-25T14:12:01.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.800+0000 7fdded400640 1 -- 172.21.15.86:0/1532115658 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fddd400bb60 con 0x7fddd4004d50 2024-04-25T14:12:01.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.800+0000 7fdded400640 1 -- 172.21.15.86:0/1532115658 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fddd400bd60 con 0x7fddf00ccab0 2024-04-25T14:12:01.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.800+0000 7fddede00640 1 --2- 172.21.15.86:0/1532115658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddd4003bf0 0x7fddd4004040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:01.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.800+0000 7fdde5400640 1 --2- 172.21.15.86:0/1532115658 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddf00ccab0 0x7fddd4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:01.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.800+0000 7fddee800640 1 --2- 172.21.15.86:0/1532115658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddd4004d50 0x7fddd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:01.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.800+0000 7fddede00640 1 -- 172.21.15.86:0/1532115658 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddf00ccab0 msgr2=0x7fddd4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:12:01.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.800+0000 7fddede00640 1 --2- 172.21.15.86:0/1532115658 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddf00ccab0 0x7fddd4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:01.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.800+0000 7fddede00640 1 -- 172.21.15.86:0/1532115658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddd4004d50 msgr2=0x7fddd40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:12:01.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.800+0000 7fddede00640 1 --2- 172.21.15.86:0/1532115658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddd4004d50 0x7fddd40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:01.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.800+0000 7fddede00640 1 -- 172.21.15.86:0/1532115658 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd4003530 con 0x7fddd4003bf0 2024-04-25T14:12:01.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.804+0000 7fddede00640 1 --2- 172.21.15.86:0/1532115658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddd4003bf0 0x7fddd4004040 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fdde000e170 tx=0x7fdde001cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79ad520cdd44810 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:01.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.804+0000 7fdded400640 1 -- 172.21.15.86:0/1532115658 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde00b2070 con 0x7fddd4003bf0 2024-04-25T14:12:01.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.804+0000 7fdded400640 1 -- 172.21.15.86:0/1532115658 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdde00a5be0 con 0x7fddd4003bf0 2024-04-25T14:12:01.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.804+0000 7fdded400640 1 -- 172.21.15.86:0/1532115658 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde0009c60 con 0x7fddd4003bf0 2024-04-25T14:12:01.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.804+0000 7fddf4e00640 1 -- 172.21.15.86:0/1532115658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddd4003bf0 msgr2=0x7fddd4004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:01.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.804+0000 7fddf4e00640 1 --2- 172.21.15.86:0/1532115658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddd4003bf0 0x7fddd4004040 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fdde000e170 tx=0x7fdde001cce0 comp rx=0 tx=0).stop 2024-04-25T14:12:01.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.804+0000 7fddee800640 1 -- 172.21.15.86:0/1532115658 reap_dead start 2024-04-25T14:12:01.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.804+0000 7fddf4e00640 1 -- 172.21.15.86:0/1532115658 shutdown_connections 2024-04-25T14:12:01.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.804+0000 7fddf4e00640 1 -- 172.21.15.86:0/1532115658 >> 172.21.15.86:0/1532115658 conn(0x7fddf009ba30 msgr2=0x7fddf00b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:01.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.804+0000 7fddf4e00640 1 -- 172.21.15.86:0/1532115658 shutdown_connections 2024-04-25T14:12:01.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.804+0000 7fddf4e00640 1 -- 172.21.15.86:0/1532115658 wait complete. 2024-04-25T14:12:01.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.804+0000 7fddf4e00640 1 Processor -- start 2024-04-25T14:12:01.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.804+0000 7fddf4e00640 1 -- start start 2024-04-25T14:12:01.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.808+0000 7fddf4e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddd4003bf0 0x7fddf0070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:01.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.808+0000 7fddf4e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddf00d1930 0x7fddf00711a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:01.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.808+0000 7fddf4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddf00d0430 0x7fddf00d0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:01.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.808+0000 7fddf4e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fddf0169980 con 0x7fddf00d0430 2024-04-25T14:12:01.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.808+0000 7fddf4e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fddf00b2620 con 0x7fddd4003bf0 2024-04-25T14:12:01.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.808+0000 7fddf4e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fddf0169b00 con 0x7fddf00d1930 2024-04-25T14:12:01.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.808+0000 7fddee800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddf00d0430 0x7fddf00d0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:01.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.808+0000 7fddee800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddf00d0430 0x7fddf00d0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58380/0 (socket says 172.21.15.86:58380) 2024-04-25T14:12:01.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.808+0000 7fddee800640 1 -- 172.21.15.86:0/3441766552 learned_addr learned my addr 172.21.15.86:0/3441766552 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:12:01.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.808+0000 7fddede00640 1 --2- 172.21.15.86:0/3441766552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddd4003bf0 0x7fddf0070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:01.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.808+0000 7fdde5400640 1 --2- 172.21.15.86:0/3441766552 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddf00d1930 0x7fddf00711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:01.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.808+0000 7fddee800640 1 -- 172.21.15.86:0/3441766552 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddf00d1930 msgr2=0x7fddf00711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:01.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.808+0000 7fddee800640 1 --2- 172.21.15.86:0/3441766552 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddf00d1930 0x7fddf00711a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:01.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.808+0000 7fddee800640 1 -- 172.21.15.86:0/3441766552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddd4003bf0 msgr2=0x7fddf0070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:01.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.808+0000 7fddee800640 1 --2- 172.21.15.86:0/3441766552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddd4003bf0 0x7fddf0070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:01.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.808+0000 7fddee800640 1 -- 172.21.15.86:0/3441766552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf014e9e0 con 0x7fddf00d0430 2024-04-25T14:12:01.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.808+0000 7fddee800640 1 --2- 172.21.15.86:0/3441766552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddf00d0430 0x7fddf00d0840 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fddf00d2240 tx=0x7fdde800e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:01.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.808+0000 7fdde6a00640 1 -- 172.21.15.86:0/3441766552 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde8019070 con 0x7fddf00d0430 2024-04-25T14:12:01.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.808+0000 7fdde6a00640 1 -- 172.21.15.86:0/3441766552 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdde8010510 con 0x7fddf00d0430 2024-04-25T14:12:01.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.808+0000 7fddf4e00640 1 -- 172.21.15.86:0/3441766552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf014b620 con 0x7fddf00d0430 2024-04-25T14:12:01.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.808+0000 7fdde6a00640 1 -- 172.21.15.86:0/3441766552 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde8010900 con 0x7fddf00d0430 2024-04-25T14:12:01.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.808+0000 7fddf4e00640 1 -- 172.21.15.86:0/3441766552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf014bbe0 con 0x7fddf00d0430 2024-04-25T14:12:01.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.808+0000 7fddf4e00640 1 -- 172.21.15.86:0/3441766552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddf014ebb0 con 0x7fddf00d0430 2024-04-25T14:12:01.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.812+0000 7fdde6a00640 1 -- 172.21.15.86:0/3441766552 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 72) ==== 93318+0+0 (secure 0 0 0) 0x7fdde801c070 con 0x7fddf00d0430 2024-04-25T14:12:01.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.812+0000 7fdde6a00640 1 --2- 172.21.15.86:0/3441766552 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fddcc06f570 0x7fddcc071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:01.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.812+0000 7fdde6a00640 1 -- 172.21.15.86:0/3441766552 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 6445+0+0 (secure 0 0 0) 0x7fdde808eb90 con 0x7fddf00d0430 2024-04-25T14:12:01.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.812+0000 7fddede00640 1 --2- 172.21.15.86:0/3441766552 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fddcc06f570 0x7fddcc071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:01.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.812+0000 7fddede00640 1 --2- 172.21.15.86:0/3441766552 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fddcc06f570 0x7fddcc071a80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdde000e170 tx=0x7fdde0004ac0 comp rx=0 tx=0).ready entity=mgr.9313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:01.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:01.816+0000 7fdde6a00640 1 -- 172.21.15.86:0/3441766552 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdde805b030 con 0x7fddf00d0430 2024-04-25T14:12:02.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.012+0000 7fddf4e00640 1 -- 172.21.15.86:0/3441766552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddf00d2f90 con 0x7fddf00d0430 2024-04-25T14:12:02.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.012+0000 7fdde6a00640 1 -- 172.21.15.86:0/3441766552 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) ==== 95+0+5207 (secure 0 0 0) 0x7fdde805f330 con 0x7fddf00d0430 2024-04-25T14:12:02.010 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:12:02.010 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":106,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/1828317917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1828317917},{"type":"v1","addr":"172.21.15.155:6834","nonce":1828317917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":9124,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1389714390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1389714390},{"type":"v1","addr":"172.21.15.86:6837","nonce":1389714390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:11:32.981538+0000","modified":"2024-04-25T14:12:00.834239+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":9546,"mds_1":9136},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9136":{"gid":9136,"name":"d","rank":1,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/91793677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":91793677},{"type":"v1","addr":"172.21.15.155:6835","nonce":91793677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9546":{"gid":9546,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/2571655765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2571655765},{"type":"v1","addr":"172.21.15.86:6835","nonce":2571655765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9546,"qdb_cluster":[9136,9546]},"id":10}]} 2024-04-25T14:12:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 106 2024-04-25T14:12:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.016+0000 7fddf4e00640 1 -- 172.21.15.86:0/3441766552 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fddcc06f570 msgr2=0x7fddcc071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.016+0000 7fddf4e00640 1 --2- 172.21.15.86:0/3441766552 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fddcc06f570 0x7fddcc071a80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdde000e170 tx=0x7fdde0004ac0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.016+0000 7fddf4e00640 1 -- 172.21.15.86:0/3441766552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddf00d0430 msgr2=0x7fddf00d0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.016+0000 7fddf4e00640 1 --2- 172.21.15.86:0/3441766552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddf00d0430 0x7fddf00d0840 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fddf00d2240 tx=0x7fdde800e910 comp rx=0 tx=0).stop 2024-04-25T14:12:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.016+0000 7fddf4e00640 1 -- 172.21.15.86:0/3441766552 shutdown_connections 2024-04-25T14:12:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.016+0000 7fddf4e00640 1 --2- 172.21.15.86:0/3441766552 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fddcc06f570 0x7fddcc071a80 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.016+0000 7fddf4e00640 1 --2- 172.21.15.86:0/3441766552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddf00d0430 0x7fddf00d0840 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.016+0000 7fddf4e00640 1 --2- 172.21.15.86:0/3441766552 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddf00d1930 0x7fddf00711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.016+0000 7fddf4e00640 1 --2- 172.21.15.86:0/3441766552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddd4003bf0 0x7fddf0070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.016+0000 7fddf4e00640 1 -- 172.21.15.86:0/3441766552 >> 172.21.15.86:0/3441766552 conn(0x7fddf009ba30 msgr2=0x7fddf015d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.016+0000 7fddf4e00640 1 -- 172.21.15.86:0/3441766552 shutdown_connections 2024-04-25T14:12:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.016+0000 7fddf4e00640 1 -- 172.21.15.86:0/3441766552 wait complete. 2024-04-25T14:12:02.435 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 106, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:11:32.981538+0000', 'modified': '2024-04-25T14:12:00.834239+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 9546, 'mds_1': 9136}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9136': {'gid': 9136, 'name': 'd', 'rank': 1, 'incarnation': 105, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.155:6835/91793677', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6833', 'nonce': 91793677}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 91793677}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_9546': {'gid': 9546, 'name': 'a', 'rank': 0, 'incarnation': 99, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6835/2571655765', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6834', 'nonce': 2571655765}, {'type': 'v1', 'addr': '172.21.15.86:6835', 'nonce': 2571655765}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9546, 'qdb_cluster': [9136, 9546]} 2024-04-25T14:12:02.436 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2024-04-25T14:12:02.436 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T14:12:02.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.584+0000 7f20dd200640 1 Processor -- start 2024-04-25T14:12:02.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.584+0000 7f20dd200640 1 -- start start 2024-04-25T14:12:02.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20dd200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20d80ae130 0x7f20d806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20dd200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20d80b0a60 con 0x7f20d815c2a0 2024-04-25T14:12:02.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20dd200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20d806ef90 con 0x7f20d80ae540 2024-04-25T14:12:02.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20dd200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f20d80d1df0 con 0x7f20d80ae130 2024-04-25T14:12:02.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20d5e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f20d815c2a0 0x7f20d80d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41626/0 (socket says 172.21.15.86:41626) 2024-04-25T14:12:02.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20d5e00640 1 -- 172.21.15.86:0/3240751912 learned_addr learned my addr 172.21.15.86:0/3240751912 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:12:02.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20d5400640 1 --2- 172.21.15.86:0/3240751912 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20d80ae130 0x7f20d806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20d4a00640 1 -- 172.21.15.86:0/3240751912 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2268960918 0 0) 0x7f20d80b0a60 con 0x7f20d815c2a0 2024-04-25T14:12:02.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20d4a00640 1 -- 172.21.15.86:0/3240751912 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20bc005740 con 0x7f20d815c2a0 2024-04-25T14:12:02.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20d4a00640 1 -- 172.21.15.86:0/3240751912 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 701284323 0 0) 0x7f20bc005740 con 0x7f20d815c2a0 2024-04-25T14:12:02.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20d4a00640 1 -- 172.21.15.86:0/3240751912 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20d80ae130 msgr2=0x7f20d806fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:12:02.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20d4a00640 1 --2- 172.21.15.86:0/3240751912 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20d80ae130 0x7f20d806fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20d4a00640 1 -- 172.21.15.86:0/3240751912 >> v1:172.21.15.155:6789/0 conn(0x7f20d80ae540 legacy=0x7f20d815feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20d4a00640 1 -- 172.21.15.86:0/3240751912 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d806f270 con 0x7f20d815c2a0 2024-04-25T14:12:02.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20d4a00640 1 -- 172.21.15.86:0/3240751912 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f20c8001680 con 0x7f20d815c2a0 2024-04-25T14:12:02.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20d4a00640 1 -- 172.21.15.86:0/3240751912 >> v1:172.21.15.86:6789/0 conn(0x7f20d815c2a0 legacy=0x7f20d80d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20d4a00640 1 --2- 172.21.15.86:0/3240751912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20bc005c70 0x7f20bc0060a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20d4a00640 1 --2- 172.21.15.86:0/3240751912 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20d80ae130 0x7f20bc006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20d4a00640 1 --2- 172.21.15.86:0/3240751912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20bc006f60 0x7f20bc007370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20d4a00640 1 -- 172.21.15.86:0/3240751912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20d80d1df0 con 0x7f20bc006f60 2024-04-25T14:12:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20d4a00640 1 -- 172.21.15.86:0/3240751912 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f20bc00dda0 con 0x7f20bc005c70 2024-04-25T14:12:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20d4a00640 1 -- 172.21.15.86:0/3240751912 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f20bc00dfa0 con 0x7f20d80ae130 2024-04-25T14:12:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20d6800640 1 --2- 172.21.15.86:0/3240751912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20bc006f60 0x7f20bc007370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20d5400640 1 --2- 172.21.15.86:0/3240751912 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20d80ae130 0x7f20bc006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20d5e00640 1 --2- 172.21.15.86:0/3240751912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20bc005c70 0x7f20bc0060a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20d6800640 1 -- 172.21.15.86:0/3240751912 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20d80ae130 msgr2=0x7f20bc006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:12:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20d6800640 1 --2- 172.21.15.86:0/3240751912 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20d80ae130 0x7f20bc006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20d6800640 1 -- 172.21.15.86:0/3240751912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20bc005c70 msgr2=0x7f20bc0060a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:12:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20d6800640 1 --2- 172.21.15.86:0/3240751912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20bc005c70 0x7f20bc0060a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.588+0000 7f20d6800640 1 -- 172.21.15.86:0/3240751912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20bc005740 con 0x7f20bc006f60 2024-04-25T14:12:02.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.592+0000 7f20d6800640 1 --2- 172.21.15.86:0/3240751912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20bc006f60 0x7f20bc007370 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f20d0001f60 tx=0x7f20d001f010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1af93c535614965 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:02.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.592+0000 7f20d4a00640 1 -- 172.21.15.86:0/3240751912 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d00b5070 con 0x7f20bc006f60 2024-04-25T14:12:02.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.592+0000 7f20d4a00640 1 -- 172.21.15.86:0/3240751912 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20d00a8c30 con 0x7f20bc006f60 2024-04-25T14:12:02.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.592+0000 7f20d4a00640 1 -- 172.21.15.86:0/3240751912 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d0001b60 con 0x7f20bc006f60 2024-04-25T14:12:02.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.592+0000 7f20dd200640 1 -- 172.21.15.86:0/3240751912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20bc006f60 msgr2=0x7f20bc007370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.592+0000 7f20dd200640 1 --2- 172.21.15.86:0/3240751912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20bc006f60 0x7f20bc007370 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f20d0001f60 tx=0x7f20d001f010 comp rx=0 tx=0).stop 2024-04-25T14:12:02.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.592+0000 7f20d6800640 1 -- 172.21.15.86:0/3240751912 reap_dead start 2024-04-25T14:12:02.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.592+0000 7f20dd200640 1 -- 172.21.15.86:0/3240751912 shutdown_connections 2024-04-25T14:12:02.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.592+0000 7f20dd200640 1 -- 172.21.15.86:0/3240751912 >> 172.21.15.86:0/3240751912 conn(0x7f20d80b4650 msgr2=0x7f20d80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:02.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.592+0000 7f20dd200640 1 -- 172.21.15.86:0/3240751912 shutdown_connections 2024-04-25T14:12:02.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.592+0000 7f20dd200640 1 -- 172.21.15.86:0/3240751912 wait complete. 2024-04-25T14:12:02.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.596+0000 7f20dd200640 1 Processor -- start 2024-04-25T14:12:02.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.596+0000 7f20dd200640 1 -- start start 2024-04-25T14:12:02.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.596+0000 7f20dd200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20bc006f60 0x7f20d81f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.596+0000 7f20dd200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20d80cdc10 0x7f20d809b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.596+0000 7f20dd200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20d809cf00 0x7f20d809d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.596+0000 7f20dd200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20d80d0fc0 con 0x7f20d80cdc10 2024-04-25T14:12:02.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.596+0000 7f20dd200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f20d806f830 con 0x7f20d809cf00 2024-04-25T14:12:02.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.596+0000 7f20dd200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f20d815c6b0 con 0x7f20bc006f60 2024-04-25T14:12:02.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.596+0000 7f20d5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20d80cdc10 0x7f20d809b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.596+0000 7f20d5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20d80cdc10 0x7f20d809b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58388/0 (socket says 172.21.15.86:58388) 2024-04-25T14:12:02.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.596+0000 7f20d5400640 1 -- 172.21.15.86:0/3336472553 learned_addr learned my addr 172.21.15.86:0/3336472553 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:12:02.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.596+0000 7f20d6800640 1 --2- 172.21.15.86:0/3336472553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20d809cf00 0x7f20d809d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.596+0000 7f20d5e00640 1 --2- 172.21.15.86:0/3336472553 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20bc006f60 0x7f20d81f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.596+0000 7f20d5400640 1 -- 172.21.15.86:0/3336472553 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20bc006f60 msgr2=0x7f20d81f1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.596+0000 7f20d5400640 1 --2- 172.21.15.86:0/3336472553 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20bc006f60 0x7f20d81f1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.596+0000 7f20d5400640 1 -- 172.21.15.86:0/3336472553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20d809cf00 msgr2=0x7f20d809d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.596+0000 7f20d5400640 1 --2- 172.21.15.86:0/3336472553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20d809cf00 0x7f20d809d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.596+0000 7f20d5400640 1 -- 172.21.15.86:0/3336472553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d809be60 con 0x7f20d80cdc10 2024-04-25T14:12:02.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.596+0000 7f20d6800640 1 --2- 172.21.15.86:0/3336472553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20d809cf00 0x7f20d809d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:12:02.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.596+0000 7f20d5400640 1 --2- 172.21.15.86:0/3336472553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20d80cdc10 0x7f20d809b2f0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f20c40119d0 tx=0x7f20c4011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:02.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.600+0000 7f20ce000640 1 -- 172.21.15.86:0/3336472553 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c400fb50 con 0x7f20d80cdc10 2024-04-25T14:12:02.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.600+0000 7f20ce000640 1 -- 172.21.15.86:0/3336472553 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c400fcf0 con 0x7f20d80cdc10 2024-04-25T14:12:02.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.600+0000 7f20ce000640 1 -- 172.21.15.86:0/3336472553 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c400fea0 con 0x7f20d80cdc10 2024-04-25T14:12:02.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.600+0000 7f20dd200640 1 -- 172.21.15.86:0/3336472553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d809c140 con 0x7f20d80cdc10 2024-04-25T14:12:02.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.600+0000 7f20dd200640 1 -- 172.21.15.86:0/3336472553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d809c700 con 0x7f20d80cdc10 2024-04-25T14:12:02.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.600+0000 7f20ce000640 1 -- 172.21.15.86:0/3336472553 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 72) ==== 93318+0+0 (secure 0 0 0) 0x7f20c4012040 con 0x7f20d80cdc10 2024-04-25T14:12:02.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.604+0000 7f20ce000640 1 --2- 172.21.15.86:0/3336472553 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f20b806f5a0 0x7f20b8071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.604+0000 7f20ce000640 1 -- 172.21.15.86:0/3336472553 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 6445+0+0 (secure 0 0 0) 0x7f20c408fe30 con 0x7f20d80cdc10 2024-04-25T14:12:02.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.604+0000 7f20d5e00640 1 --2- 172.21.15.86:0/3336472553 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f20b806f5a0 0x7f20b8071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.604+0000 7f20e4913000 1 Processor -- start 2024-04-25T14:12:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.604+0000 7f20e4913000 1 -- start start 2024-04-25T14:12:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.604+0000 7f20d5e00640 1 --2- 172.21.15.86:0/3336472553 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f20b806f5a0 0x7f20b8071ab0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f20d80b4200 tx=0x7f20c8009040 comp rx=0 tx=0).ready entity=mgr.9313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:02.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.604+0000 7f20e4913000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d93302cc20 0x55d93318aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.604+0000 7f20e4913000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d933018ea0 0x55d93318b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.604+0000 7f20e4913000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d932c11280 0x55d93318b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.604+0000 7f20e4913000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d932ffa160 con 0x55d933018ea0 2024-04-25T14:12:02.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.604+0000 7f20e4913000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d932f8fe20 con 0x55d93302cc20 2024-04-25T14:12:02.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.604+0000 7f20e4913000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55d932f48a20 con 0x55d932c11280 2024-04-25T14:12:02.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.604+0000 7f20d5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d933018ea0 0x55d93318b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.604+0000 7f20d5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d933018ea0 0x55d93318b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58398/0 (socket says 172.21.15.86:58398) 2024-04-25T14:12:02.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.604+0000 7f20d5400640 1 -- 172.21.15.86:0/4200505763 learned_addr learned my addr 172.21.15.86:0/4200505763 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:12:02.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.604+0000 7f20d5e00640 1 --2- 172.21.15.86:0/4200505763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d93302cc20 0x55d93318aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.604+0000 7f20d6800640 1 --2- 172.21.15.86:0/4200505763 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d932c11280 0x55d93318b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.604+0000 7f20d5400640 1 -- 172.21.15.86:0/4200505763 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d932c11280 msgr2=0x55d93318b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:12:02.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.604+0000 7f20d5400640 1 --2- 172.21.15.86:0/4200505763 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d932c11280 0x55d93318b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.604+0000 7f20d5400640 1 -- 172.21.15.86:0/4200505763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d93302cc20 msgr2=0x55d93318aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:12:02.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.604+0000 7f20d5400640 1 --2- 172.21.15.86:0/4200505763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d93302cc20 0x55d93318aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.604+0000 7f20d5400640 1 -- 172.21.15.86:0/4200505763 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d932f4d100 con 0x55d933018ea0 2024-04-25T14:12:02.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.604+0000 7f20d5400640 1 --2- 172.21.15.86:0/4200505763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d933018ea0 0x55d93318b420 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f20c4058980 tx=0x7f20c405c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a05cf0a1ea1abe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:02.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.604+0000 7f20c2a00640 1 -- 172.21.15.86:0/4200505763 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c408d070 con 0x55d933018ea0 2024-04-25T14:12:02.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.604+0000 7f20c2a00640 1 -- 172.21.15.86:0/4200505763 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c405c5f0 con 0x55d933018ea0 2024-04-25T14:12:02.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.604+0000 7f20c2a00640 1 -- 172.21.15.86:0/4200505763 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4022040 con 0x55d933018ea0 2024-04-25T14:12:02.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20e4913000 1 -- 172.21.15.86:0/4200505763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d933018ea0 msgr2=0x55d93318b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20e4913000 1 --2- 172.21.15.86:0/4200505763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d933018ea0 0x55d93318b420 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f20c4058980 tx=0x7f20c405c390 comp rx=0 tx=0).stop 2024-04-25T14:12:02.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20e4913000 1 -- 172.21.15.86:0/4200505763 shutdown_connections 2024-04-25T14:12:02.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20e4913000 1 --2- 172.21.15.86:0/4200505763 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d932c11280 0x55d93318b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20e4913000 1 --2- 172.21.15.86:0/4200505763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d933018ea0 0x55d93318b420 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20e4913000 1 --2- 172.21.15.86:0/4200505763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d93302cc20 0x55d93318aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20e4913000 1 -- 172.21.15.86:0/4200505763 >> 172.21.15.86:0/4200505763 conn(0x55d932c50ff0 msgr2=0x55d9331404c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:02.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20e4913000 1 -- 172.21.15.86:0/4200505763 shutdown_connections 2024-04-25T14:12:02.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20e4913000 1 -- 172.21.15.86:0/4200505763 wait complete. 2024-04-25T14:12:02.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20e4913000 1 -- start start 2024-04-25T14:12:02.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20e4913000 1 Processor -- start 2024-04-25T14:12:02.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20e4913000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d932c11280 0x55d933191e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20e4913000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d933018ea0 0x55d9331923a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20e4913000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d93302cc20 0x55d93314d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20e4913000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d932f9d220 con 0x55d932c11280 2024-04-25T14:12:02.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20e4913000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d932f33ea0 con 0x55d93302cc20 2024-04-25T14:12:02.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20e4913000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55d932f8ab00 con 0x55d933018ea0 2024-04-25T14:12:02.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20e4913000 10 client.0 _ll_register_callbacks cb 0x55d932f9cdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:12:02.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20d5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d932c11280 0x55d933191e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20d5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d932c11280 0x55d933191e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58406/0 (socket says 172.21.15.86:58406) 2024-04-25T14:12:02.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20d5e00640 1 -- 172.21.15.86:0/2886268974 learned_addr learned my addr 172.21.15.86:0/2886268974 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:12:02.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20d6800640 1 --2- 172.21.15.86:0/2886268974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d93302cc20 0x55d93314d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20d5400640 1 --2- 172.21.15.86:0/2886268974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d933018ea0 0x55d9331923a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20d5e00640 1 -- 172.21.15.86:0/2886268974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d933018ea0 msgr2=0x55d9331923a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:12:02.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20d5e00640 1 --2- 172.21.15.86:0/2886268974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d933018ea0 0x55d9331923a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20d5e00640 1 -- 172.21.15.86:0/2886268974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d93302cc20 msgr2=0x55d93314d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:12:02.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20d5e00640 1 --2- 172.21.15.86:0/2886268974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d93302cc20 0x55d93314d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20d5e00640 1 -- 172.21.15.86:0/2886268974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d932f2f500 con 0x55d932c11280 2024-04-25T14:12:02.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20d5e00640 1 --2- 172.21.15.86:0/2886268974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d932c11280 0x55d933191e60 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f20c8015730 tx=0x7f20c8007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b017b6b7097031c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:02.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.608+0000 7f20b7400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:12:02.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20b7400640 1 -- 172.21.15.86:0/2886268974 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c8027030 con 0x55d932c11280 2024-04-25T14:12:02.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20b7400640 1 -- 172.21.15.86:0/2886268974 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(141..141 src has 1..141) ==== 6445+0+0 (secure 0 0 0) 0x7f20c8010040 con 0x55d932c11280 2024-04-25T14:12:02.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20e4913000 1 -- 172.21.15.86:0/2886268974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20c405c5f0 con 0x55d932c11280 2024-04-25T14:12:02.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20b7400640 1 -- 172.21.15.86:0/2886268974 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) ==== 24+0+0 (secure 0 0 0) 0x7f20c80147d0 con 0x55d932c11280 2024-04-25T14:12:02.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20e4913000 10 client.9873 fetch_fsmap learned FSMap version 106 2024-04-25T14:12:02.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20e4913000 1 -- 172.21.15.86:0/2886268974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) -- 0x55d932f15ea0 con 0x55d932c11280 2024-04-25T14:12:02.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20b7400640 1 -- 172.21.15.86:0/2886268974 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c405c5f0 con 0x55d932c11280 2024-04-25T14:12:02.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20b7400640 1 -- 172.21.15.86:0/2886268974 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 106) ==== 3514+0+0 (secure 0 0 0) 0x7f20c8014970 con 0x55d932c11280 2024-04-25T14:12:02.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20e4913000 10 client.9873 fetch_fsmap finished waiting for FSMap version 106 2024-04-25T14:12:02.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20e4913000 10 client.9873 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T14:12:02.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20e4913000 20 client.9873 populate_metadata read hostname 'smithi086' 2024-04-25T14:12:02.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20e4913000 1 --2- 172.21.15.86:0/2886268974 >> [v2:172.21.15.155:6833/91793677,v1:172.21.15.155:6835/91793677] conn(0x55d932c357d0 0x55d93314de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20e4913000 4 client.9873 mds_command: new command op to 9136 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:12:02.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20e4913000 1 -- 172.21.15.86:0/2886268974 --> [v2:172.21.15.155:6833/91793677,v1:172.21.15.155:6835/91793677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d932f28be0 con 0x55d932c357d0 2024-04-25T14:12:02.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20d5400640 1 --2- 172.21.15.86:0/2886268974 >> [v2:172.21.15.155:6833/91793677,v1:172.21.15.155:6835/91793677] conn(0x55d932c357d0 0x55d93314de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20b7400640 10 client.9873 ms_handle_connect on v2:172.21.15.155:6833/91793677 2024-04-25T14:12:02.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20d5400640 1 --2- 172.21.15.86:0/2886268974 >> [v2:172.21.15.155:6833/91793677,v1:172.21.15.155:6835/91793677] conn(0x55d932c357d0 0x55d93314de60 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d65c598538849d76 server_cookie=ed26e445741151fc in_seq=0 out_seq=0 2024-04-25T14:12:02.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20b7400640 1 -- 172.21.15.86:0/2886268974 <== mds.1 v2:172.21.15.155:6833/91793677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d932f28be0 con 0x55d932c357d0 2024-04-25T14:12:02.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20b7400640 10 client.9873 handle_command_reply: tid=0 2024-04-25T14:12:02.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20e4913000 1 client.9873 shutdown 2024-04-25T14:12:02.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20e4913000 1 -- 172.21.15.86:0/2886268974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d932c11280 msgr2=0x55d933191e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20e4913000 1 --2- 172.21.15.86:0/2886268974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d932c11280 0x55d933191e60 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f20c8015730 tx=0x7f20c8007900 comp rx=0 tx=0).stop 2024-04-25T14:12:02.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20e4913000 1 -- 172.21.15.86:0/2886268974 shutdown_connections 2024-04-25T14:12:02.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20e4913000 1 --2- 172.21.15.86:0/2886268974 >> [v2:172.21.15.155:6833/91793677,v1:172.21.15.155:6835/91793677] conn(0x55d932c357d0 0x55d93314de60 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20e4913000 1 --2- 172.21.15.86:0/2886268974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d93302cc20 0x55d93314d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20e4913000 1 --2- 172.21.15.86:0/2886268974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d933018ea0 0x55d9331923a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20e4913000 1 --2- 172.21.15.86:0/2886268974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d932c11280 0x55d933191e60 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20b7400640 0 client.9873 ms_handle_reset on v2:172.21.15.155:6833/91793677 2024-04-25T14:12:02.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20e4913000 1 -- 172.21.15.86:0/2886268974 >> 172.21.15.86:0/2886268974 conn(0x55d932c50ff0 msgr2=0x55d933140c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:02.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20e4913000 1 -- 172.21.15.86:0/2886268974 shutdown_connections 2024-04-25T14:12:02.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20e4913000 1 -- 172.21.15.86:0/2886268974 wait complete. 2024-04-25T14:12:02.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.612+0000 7f20e4913000 20 client.9873 trim_cache size 0 max 16384 2024-04-25T14:12:02.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.632+0000 7f20e4913000 1 Processor -- start 2024-04-25T14:12:02.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.632+0000 7f20e4913000 1 -- start start 2024-04-25T14:12:02.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.632+0000 7f20e4913000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d932c11280 0x55d9331970b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.632+0000 7f20e4913000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d933018ea0 0x55d9331a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.632+0000 7f20e4913000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d93302cc20 0x55d9331a1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.632+0000 7f20e4913000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20d81ff6a0 con 0x55d93302cc20 2024-04-25T14:12:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.632+0000 7f20e4913000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f20d80b34f0 con 0x55d933018ea0 2024-04-25T14:12:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.632+0000 7f20e4913000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f20d81fdfe0 con 0x55d932c11280 2024-04-25T14:12:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.632+0000 7f20d6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d93302cc20 0x55d9331a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.632+0000 7f20d6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d93302cc20 0x55d9331a1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58414/0 (socket says 172.21.15.86:58414) 2024-04-25T14:12:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.632+0000 7f20d6800640 1 -- 172.21.15.86:0/3284763798 learned_addr learned my addr 172.21.15.86:0/3284763798 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:12:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.632+0000 7f20d5400640 1 --2- 172.21.15.86:0/3284763798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d933018ea0 0x55d9331a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.632+0000 7f20d5e00640 1 --2- 172.21.15.86:0/3284763798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d932c11280 0x55d9331970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.632+0000 7f20d6800640 1 -- 172.21.15.86:0/3284763798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d932c11280 msgr2=0x55d9331970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:12:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.632+0000 7f20d6800640 1 --2- 172.21.15.86:0/3284763798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d932c11280 0x55d9331970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.632+0000 7f20d6800640 1 -- 172.21.15.86:0/3284763798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d933018ea0 msgr2=0x55d9331a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:12:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.632+0000 7f20d6800640 1 --2- 172.21.15.86:0/3284763798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d933018ea0 0x55d9331a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.632+0000 7f20d6800640 1 -- 172.21.15.86:0/3284763798 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d93314d430 con 0x55d93302cc20 2024-04-25T14:12:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.632+0000 7f20d5400640 1 --2- 172.21.15.86:0/3284763798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d933018ea0 0x55d9331a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:12:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.632+0000 7f20d6800640 1 --2- 172.21.15.86:0/3284763798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d93302cc20 0x55d9331a1530 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f20d001f050 tx=0x7f20d00015c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2148d999e69d270 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.636+0000 7f20c2000640 1 -- 172.21.15.86:0/3284763798 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d00b5040 con 0x55d93302cc20 2024-04-25T14:12:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.636+0000 7f20c2000640 1 -- 172.21.15.86:0/3284763798 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20d00be070 con 0x55d93302cc20 2024-04-25T14:12:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.636+0000 7f20c2000640 1 -- 172.21.15.86:0/3284763798 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d00a8a30 con 0x55d93302cc20 2024-04-25T14:12:02.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.636+0000 7f20e4913000 1 -- 172.21.15.86:0/3284763798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d93302cc20 msgr2=0x55d9331a1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.636+0000 7f20e4913000 1 --2- 172.21.15.86:0/3284763798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d93302cc20 0x55d9331a1530 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f20d001f050 tx=0x7f20d00015c0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.636+0000 7f20e4913000 1 -- 172.21.15.86:0/3284763798 shutdown_connections 2024-04-25T14:12:02.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.636+0000 7f20e4913000 1 --2- 172.21.15.86:0/3284763798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d93302cc20 0x55d9331a1530 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.636+0000 7f20e4913000 1 --2- 172.21.15.86:0/3284763798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d933018ea0 0x55d9331a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.636+0000 7f20e4913000 1 --2- 172.21.15.86:0/3284763798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d932c11280 0x55d9331970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.636+0000 7f20e4913000 1 -- 172.21.15.86:0/3284763798 >> 172.21.15.86:0/3284763798 conn(0x55d932c50ff0 msgr2=0x55d933148000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:02.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.636+0000 7f20e4913000 1 -- 172.21.15.86:0/3284763798 shutdown_connections 2024-04-25T14:12:02.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.636+0000 7f20e4913000 1 -- 172.21.15.86:0/3284763798 wait complete. 2024-04-25T14:12:02.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.636+0000 7f20e4913000 1 -- start start 2024-04-25T14:12:02.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.636+0000 7f20e4913000 1 Processor -- start 2024-04-25T14:12:02.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.636+0000 7f20e4913000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d932c11280 0x55d9331fd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.636+0000 7f20e4913000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d933018ea0 0x55d9331fdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.636+0000 7f20e4913000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d93302cc20 0x55d9331959b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.636+0000 7f20e4913000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d932f8f130 con 0x55d93302cc20 2024-04-25T14:12:02.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.636+0000 7f20e4913000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d932f257d0 con 0x55d932c11280 2024-04-25T14:12:02.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.636+0000 7f20e4913000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55d932f94450 con 0x55d933018ea0 2024-04-25T14:12:02.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.636+0000 7f20e4913000 10 client.0 _ll_register_callbacks cb 0x7f20c80015c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:12:02.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.636+0000 7f20d6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d93302cc20 0x55d9331959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.636+0000 7f20d6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d93302cc20 0x55d9331959b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58430/0 (socket says 172.21.15.86:58430) 2024-04-25T14:12:02.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.636+0000 7f20d6800640 1 -- 172.21.15.86:0/1359644454 learned_addr learned my addr 172.21.15.86:0/1359644454 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:12:02.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.636+0000 7f20d5400640 1 --2- 172.21.15.86:0/1359644454 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d933018ea0 0x55d9331fdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.636+0000 7f20d5e00640 1 --2- 172.21.15.86:0/1359644454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d932c11280 0x55d9331fd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20d6800640 1 -- 172.21.15.86:0/1359644454 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d933018ea0 msgr2=0x55d9331fdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:12:02.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20d6800640 1 --2- 172.21.15.86:0/1359644454 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d933018ea0 0x55d9331fdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20d6800640 1 -- 172.21.15.86:0/1359644454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d932c11280 msgr2=0x55d9331fd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:12:02.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20d6800640 1 --2- 172.21.15.86:0/1359644454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d932c11280 0x55d9331fd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20d6800640 1 -- 172.21.15.86:0/1359644454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d932f48c70 con 0x55d93302cc20 2024-04-25T14:12:02.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20d6800640 1 --2- 172.21.15.86:0/1359644454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d93302cc20 0x55d9331959b0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f20d001ee40 tx=0x7f20d001bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2dd807a8cc89792 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:02.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20b7400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:12:02.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20b7400640 1 -- 172.21.15.86:0/1359644454 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d00ca060 con 0x55d93302cc20 2024-04-25T14:12:02.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20b7400640 1 -- 172.21.15.86:0/1359644454 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(141..141 src has 1..141) ==== 6445+0+0 (secure 0 0 0) 0x7f20d00a8cf0 con 0x55d93302cc20 2024-04-25T14:12:02.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20e4913000 1 -- 172.21.15.86:0/1359644454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20d00be070 con 0x55d93302cc20 2024-04-25T14:12:02.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20b7400640 1 -- 172.21.15.86:0/1359644454 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) ==== 24+0+0 (secure 0 0 0) 0x7f20d00d0050 con 0x55d93302cc20 2024-04-25T14:12:02.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20e4913000 10 client.9879 fetch_fsmap learned FSMap version 106 2024-04-25T14:12:02.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20e4913000 1 -- 172.21.15.86:0/1359644454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) -- 0x55d933012180 con 0x55d93302cc20 2024-04-25T14:12:02.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20b7400640 1 -- 172.21.15.86:0/1359644454 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20d00be070 con 0x55d93302cc20 2024-04-25T14:12:02.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20b7400640 1 -- 172.21.15.86:0/1359644454 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 106) ==== 3514+0+0 (secure 0 0 0) 0x7f20d00d3020 con 0x55d93302cc20 2024-04-25T14:12:02.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20e4913000 10 client.9879 fetch_fsmap finished waiting for FSMap version 106 2024-04-25T14:12:02.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20e4913000 10 client.9879 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T14:12:02.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20e4913000 20 client.9879 populate_metadata read hostname 'smithi086' 2024-04-25T14:12:02.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20e4913000 1 --2- 172.21.15.86:0/1359644454 >> [v2:172.21.15.155:6833/91793677,v1:172.21.15.155:6835/91793677] conn(0x55d932c23830 0x55d933196340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20e4913000 4 client.9879 mds_command: new command op to 9136 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:12:02.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20e4913000 1 -- 172.21.15.86:0/1359644454 --> [v2:172.21.15.155:6833/91793677,v1:172.21.15.155:6835/91793677] -- command(tid 0: {"prefix": "status"}) -- 0x55d932f9da30 con 0x55d932c23830 2024-04-25T14:12:02.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20d5e00640 1 --2- 172.21.15.86:0/1359644454 >> [v2:172.21.15.155:6833/91793677,v1:172.21.15.155:6835/91793677] conn(0x55d932c23830 0x55d933196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20b7400640 10 client.9879 ms_handle_connect on v2:172.21.15.155:6833/91793677 2024-04-25T14:12:02.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20d5e00640 1 --2- 172.21.15.86:0/1359644454 >> [v2:172.21.15.155:6833/91793677,v1:172.21.15.155:6835/91793677] conn(0x55d932c23830 0x55d933196340 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b4103d105bf5a9c server_cookie=57a5513dee5d966c in_seq=0 out_seq=0 2024-04-25T14:12:02.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20b7400640 1 -- 172.21.15.86:0/1359644454 <== mds.1 v2:172.21.15.155:6833/91793677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55d932f9da30 con 0x55d932c23830 2024-04-25T14:12:02.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20b7400640 10 client.9879 handle_command_reply: tid=0 2024-04-25T14:12:02.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20e4913000 1 client.9879 shutdown 2024-04-25T14:12:02.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20e4913000 1 -- 172.21.15.86:0/1359644454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d93302cc20 msgr2=0x55d9331959b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20e4913000 1 --2- 172.21.15.86:0/1359644454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d93302cc20 0x55d9331959b0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f20d001ee40 tx=0x7f20d001bdb0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20e4913000 1 -- 172.21.15.86:0/1359644454 shutdown_connections 2024-04-25T14:12:02.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20e4913000 1 --2- 172.21.15.86:0/1359644454 >> [v2:172.21.15.155:6833/91793677,v1:172.21.15.155:6835/91793677] conn(0x55d932c23830 0x55d933196340 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20e4913000 1 --2- 172.21.15.86:0/1359644454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d93302cc20 0x55d9331959b0 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20e4913000 1 --2- 172.21.15.86:0/1359644454 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d933018ea0 0x55d9331fdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20e4913000 1 --2- 172.21.15.86:0/1359644454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d932c11280 0x55d9331fd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20b7400640 0 client.9879 ms_handle_reset on v2:172.21.15.155:6833/91793677 2024-04-25T14:12:02.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20e4913000 1 -- 172.21.15.86:0/1359644454 >> 172.21.15.86:0/1359644454 conn(0x55d932c50ff0 msgr2=0x55d933190070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:02.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20e4913000 1 -- 172.21.15.86:0/1359644454 shutdown_connections 2024-04-25T14:12:02.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20e4913000 1 -- 172.21.15.86:0/1359644454 wait complete. 2024-04-25T14:12:02.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.640+0000 7f20e4913000 20 client.9879 trim_cache size 0 max 16384 2024-04-25T14:12:02.642 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:12:02.642 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb", 2024-04-25T14:12:02.642 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 1, 2024-04-25T14:12:02.642 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 9136, 2024-04-25T14:12:02.642 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T14:12:02.643 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T14:12:02.643 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T14:12:02.643 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 2.8103899550000002, 2024-04-25T14:12:02.643 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 106, 2024-04-25T14:12:02.643 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 141, 2024-04-25T14:12:02.643 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 141, 2024-04-25T14:12:02.643 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 34.374794821999998 2024-04-25T14:12:02.643 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:12:02.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.644+0000 7f20dd200640 1 -- 172.21.15.86:0/3336472553 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f20b806f5a0 msgr2=0x7f20b8071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.644+0000 7f20dd200640 1 --2- 172.21.15.86:0/3336472553 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f20b806f5a0 0x7f20b8071ab0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f20d80b4200 tx=0x7f20c8009040 comp rx=0 tx=0).stop 2024-04-25T14:12:02.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.644+0000 7f20dd200640 1 -- 172.21.15.86:0/3336472553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20d80cdc10 msgr2=0x7f20d809b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.644+0000 7f20dd200640 1 --2- 172.21.15.86:0/3336472553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20d80cdc10 0x7f20d809b2f0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f20c40119d0 tx=0x7f20c4011e90 comp rx=0 tx=0).stop 2024-04-25T14:12:02.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.644+0000 7f20dd200640 1 -- 172.21.15.86:0/3336472553 shutdown_connections 2024-04-25T14:12:02.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.644+0000 7f20dd200640 1 --2- 172.21.15.86:0/3336472553 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f20b806f5a0 0x7f20b8071ab0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.644+0000 7f20dd200640 1 --2- 172.21.15.86:0/3336472553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20d809cf00 0x7f20d809d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.644+0000 7f20dd200640 1 --2- 172.21.15.86:0/3336472553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20d80cdc10 0x7f20d809b2f0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.644+0000 7f20dd200640 1 --2- 172.21.15.86:0/3336472553 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20bc006f60 0x7f20d81f1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.644+0000 7f20dd200640 1 -- 172.21.15.86:0/3336472553 >> 172.21.15.86:0/3336472553 conn(0x7f20d80b4650 msgr2=0x7f20d8074450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:02.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.644+0000 7f20dd200640 1 -- 172.21.15.86:0/3336472553 shutdown_connections 2024-04-25T14:12:02.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.644+0000 7f20dd200640 1 -- 172.21.15.86:0/3336472553 wait complete. 2024-04-25T14:12:02.659 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T14:12:02.794 INFO:tasks.ceph.mds.a.smithi086.stderr:2024-04-25T14:12:02.796+0000 7f4c1b600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-25T14:12:02.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.812+0000 7f0b73000640 1 Processor -- start 2024-04-25T14:12:02.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.812+0000 7f0b73000640 1 -- start start 2024-04-25T14:12:02.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.812+0000 7f0b73000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b6c0b3a90 0x7f0b6c15cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.812+0000 7f0b73000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0b6c0cf500 0x7f0b6c0d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.812+0000 7f0b73000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b6c0cf910 con 0x7f0b6c0ce430 2024-04-25T14:12:02.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.812+0000 7f0b73000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0b6c0cdfa0 con 0x7f0b6c0cf500 2024-04-25T14:12:02.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.812+0000 7f0b73000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0b6c0b3ea0 con 0x7f0b6c0b3a90 2024-04-25T14:12:02.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.812+0000 7f0b6b400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0b6c0ce430 0x7f0b6c0cedf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41636/0 (socket says 172.21.15.86:41636) 2024-04-25T14:12:02.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.812+0000 7f0b6b400640 1 -- 172.21.15.86:0/2839697316 learned_addr learned my addr 172.21.15.86:0/2839697316 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:12:02.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.812+0000 7f0b71000640 1 --2- 172.21.15.86:0/2839697316 >> v2:172.21.15.155:3300/0 conn(0x7f0b6c0cf500 0x7f0b6c0d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.812+0000 7f0b6be00640 1 --2- 172.21.15.86:0/2839697316 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b6c0b3a90 0x7f0b6c15cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.812+0000 7f0b6aa00640 1 -- 172.21.15.86:0/2839697316 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1141234343 0 0) 0x7f0b6c0cf910 con 0x7f0b6c0ce430 2024-04-25T14:12:02.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.812+0000 7f0b6aa00640 1 -- 172.21.15.86:0/2839697316 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b50003530 con 0x7f0b6c0ce430 2024-04-25T14:12:02.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.812+0000 7f0b71000640 1 -- 172.21.15.86:0/2839697316 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b6c0b3a90 msgr2=0x7f0b6c15cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:12:02.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.812+0000 7f0b71000640 1 --2- 172.21.15.86:0/2839697316 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b6c0b3a90 0x7f0b6c15cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.812+0000 7f0b71000640 1 -- 172.21.15.86:0/2839697316 >> v1:172.21.15.86:6789/0 conn(0x7f0b6c0ce430 legacy=0x7f0b6c0cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.812+0000 7f0b71000640 1 -- 172.21.15.86:0/2839697316 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6c0cfbc0 con 0x7f0b6c0cf500 2024-04-25T14:12:02.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.816+0000 7f0b71000640 1 --2- 172.21.15.86:0/2839697316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b6c0cf500 0x7f0b6c0d0140 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f0b64009d90 tx=0x7f0b6401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fea382ff475073b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:02.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.816+0000 7f0b6aa00640 1 -- 172.21.15.86:0/2839697316 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b640b2070 con 0x7f0b6c0cf500 2024-04-25T14:12:02.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.816+0000 7f0b6aa00640 1 -- 172.21.15.86:0/2839697316 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b640a5920 con 0x7f0b6c0cf500 2024-04-25T14:12:02.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.816+0000 7f0b6aa00640 1 -- 172.21.15.86:0/2839697316 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b640a5cf0 con 0x7f0b6c0cf500 2024-04-25T14:12:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.816+0000 7f0b73000640 1 -- 172.21.15.86:0/2839697316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b6c0cf500 msgr2=0x7f0b6c0d0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.816+0000 7f0b73000640 1 --2- 172.21.15.86:0/2839697316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b6c0cf500 0x7f0b6c0d0140 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f0b64009d90 tx=0x7f0b6401c970 comp rx=0 tx=0).stop 2024-04-25T14:12:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.816+0000 7f0b73000640 1 -- 172.21.15.86:0/2839697316 shutdown_connections 2024-04-25T14:12:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.816+0000 7f0b73000640 1 --2- 172.21.15.86:0/2839697316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b6c0cf500 0x7f0b6c0d0140 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.816+0000 7f0b73000640 1 --2- 172.21.15.86:0/2839697316 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b6c0b3a90 0x7f0b6c15cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.816+0000 7f0b73000640 1 -- 172.21.15.86:0/2839697316 >> 172.21.15.86:0/2839697316 conn(0x7f0b6c09ba30 msgr2=0x7f0b6c0b73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.816+0000 7f0b73000640 1 -- 172.21.15.86:0/2839697316 shutdown_connections 2024-04-25T14:12:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.816+0000 7f0b73000640 1 -- 172.21.15.86:0/2839697316 wait complete. 2024-04-25T14:12:02.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.820+0000 7f0b73000640 1 Processor -- start 2024-04-25T14:12:02.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.820+0000 7f0b73000640 1 -- start start 2024-04-25T14:12:02.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.820+0000 7f0b73000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b6c0b3a90 0x7f0b6c158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.820+0000 7f0b73000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b6c0ce430 0x7f0b6c0b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.820+0000 7f0b73000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b6c0cf500 0x7f0b6c0b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.820+0000 7f0b73000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c0ce9c0 con 0x7f0b6c0b3a90 2024-04-25T14:12:02.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.820+0000 7f0b73000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c0ce840 con 0x7f0b6c0cf500 2024-04-25T14:12:02.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.820+0000 7f0b73000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0b6c0d0a90 con 0x7f0b6c0ce430 2024-04-25T14:12:02.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.820+0000 7f0b6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b6c0b3a90 0x7f0b6c158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.820+0000 7f0b6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b6c0b3a90 0x7f0b6c158490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58434/0 (socket says 172.21.15.86:58434) 2024-04-25T14:12:02.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.820+0000 7f0b6be00640 1 -- 172.21.15.86:0/173715420 learned_addr learned my addr 172.21.15.86:0/173715420 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:12:02.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.820+0000 7f0b71000640 1 --2- 172.21.15.86:0/173715420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b6c0cf500 0x7f0b6c0b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.820+0000 7f0b6b400640 1 --2- 172.21.15.86:0/173715420 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b6c0ce430 0x7f0b6c0b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.820+0000 7f0b6be00640 1 -- 172.21.15.86:0/173715420 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b6c0ce430 msgr2=0x7f0b6c0b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.820+0000 7f0b6be00640 1 --2- 172.21.15.86:0/173715420 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b6c0ce430 0x7f0b6c0b07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.820+0000 7f0b6be00640 1 -- 172.21.15.86:0/173715420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b6c0cf500 msgr2=0x7f0b6c0b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.820+0000 7f0b6be00640 1 --2- 172.21.15.86:0/173715420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b6c0cf500 0x7f0b6c0b0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.820+0000 7f0b6be00640 1 -- 172.21.15.86:0/173715420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6c0b1550 con 0x7f0b6c0b3a90 2024-04-25T14:12:02.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.820+0000 7f0b71000640 1 --2- 172.21.15.86:0/173715420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b6c0cf500 0x7f0b6c0b0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:12:02.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.820+0000 7f0b6be00640 1 --2- 172.21.15.86:0/173715420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b6c0b3a90 0x7f0b6c158490 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00e870 tx=0x7f0b5c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:02.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.820+0000 7f0b63e00640 1 -- 172.21.15.86:0/173715420 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c017070 con 0x7f0b6c0b3a90 2024-04-25T14:12:02.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.820+0000 7f0b63e00640 1 -- 172.21.15.86:0/173715420 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b5c00c940 con 0x7f0b6c0b3a90 2024-04-25T14:12:02.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.820+0000 7f0b73000640 1 -- 172.21.15.86:0/173715420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b6c0af450 con 0x7f0b6c0b3a90 2024-04-25T14:12:02.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.820+0000 7f0b63e00640 1 -- 172.21.15.86:0/173715420 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c00ccf0 con 0x7f0b6c0b3a90 2024-04-25T14:12:02.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.824+0000 7f0b73000640 1 -- 172.21.15.86:0/173715420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b6c0afa10 con 0x7f0b6c0b3a90 2024-04-25T14:12:02.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.824+0000 7f0b63e00640 1 -- 172.21.15.86:0/173715420 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 72) ==== 93318+0+0 (secure 0 0 0) 0x7f0b5c01a070 con 0x7f0b6c0b3a90 2024-04-25T14:12:02.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.824+0000 7f0b63e00640 1 --2- 172.21.15.86:0/173715420 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f0b4c06f570 0x7f0b4c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.824+0000 7f0b63e00640 1 -- 172.21.15.86:0/173715420 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 6445+0+0 (secure 0 0 0) 0x7f0b5c08fa30 con 0x7f0b6c0b3a90 2024-04-25T14:12:02.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.824+0000 7f0b6b400640 1 --2- 172.21.15.86:0/173715420 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f0b4c06f570 0x7f0b4c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.824+0000 7f0b7a74e000 1 Processor -- start 2024-04-25T14:12:02.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b7a74e000 1 -- start start 2024-04-25T14:12:02.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b6b400640 1 --2- 172.21.15.86:0/173715420 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f0b4c06f570 0x7f0b4c071a80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c1565a0 tx=0x7f0b58009040 comp rx=0 tx=0).ready entity=mgr.9313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:02.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b7a74e000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7a4eac20 0x558e7a648ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b7a74e000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558e7a4d6ea0 0x558e7a649420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b7a74e000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e7a0cf280 0x558e7a649960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b7a74e000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558e7a4b8160 con 0x558e7a0cf280 2024-04-25T14:12:02.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b7a74e000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558e7a44de20 con 0x558e7a4eac20 2024-04-25T14:12:02.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b7a74e000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x558e7a406a20 con 0x558e7a4d6ea0 2024-04-25T14:12:02.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b71000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e7a0cf280 0x558e7a649960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b71000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e7a0cf280 0x558e7a649960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58442/0 (socket says 172.21.15.86:58442) 2024-04-25T14:12:02.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b71000640 1 -- 172.21.15.86:0/3828689787 learned_addr learned my addr 172.21.15.86:0/3828689787 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:12:02.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b6b400640 1 --2- 172.21.15.86:0/3828689787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558e7a4d6ea0 0x558e7a649420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b6be00640 1 --2- 172.21.15.86:0/3828689787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7a4eac20 0x558e7a648ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b71000640 1 -- 172.21.15.86:0/3828689787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558e7a4d6ea0 msgr2=0x558e7a649420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:12:02.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b71000640 1 --2- 172.21.15.86:0/3828689787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558e7a4d6ea0 0x558e7a649420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b71000640 1 -- 172.21.15.86:0/3828689787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7a4eac20 msgr2=0x558e7a648ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:12:02.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b71000640 1 --2- 172.21.15.86:0/3828689787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7a4eac20 0x558e7a648ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b71000640 1 -- 172.21.15.86:0/3828689787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e7a40b100 con 0x558e7a0cf280 2024-04-25T14:12:02.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b6b400640 1 --2- 172.21.15.86:0/3828689787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558e7a4d6ea0 0x558e7a649420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:12:02.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b6be00640 1 --2- 172.21.15.86:0/3828689787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7a4eac20 0x558e7a648ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:12:02.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b71000640 1 --2- 172.21.15.86:0/3828689787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e7a0cf280 0x558e7a649960 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f0b6401b020 tx=0x7f0b64004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342dac66746d01f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:02.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b60c00640 1 -- 172.21.15.86:0/3828689787 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b640b2040 con 0x558e7a0cf280 2024-04-25T14:12:02.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b60c00640 1 -- 172.21.15.86:0/3828689787 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b640a5920 con 0x558e7a0cf280 2024-04-25T14:12:02.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b60c00640 1 -- 172.21.15.86:0/3828689787 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b640a5cd0 con 0x558e7a0cf280 2024-04-25T14:12:02.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b7a74e000 1 -- 172.21.15.86:0/3828689787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e7a0cf280 msgr2=0x558e7a649960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b7a74e000 1 --2- 172.21.15.86:0/3828689787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e7a0cf280 0x558e7a649960 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f0b6401b020 tx=0x7f0b64004ac0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b7a74e000 1 -- 172.21.15.86:0/3828689787 shutdown_connections 2024-04-25T14:12:02.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b7a74e000 1 --2- 172.21.15.86:0/3828689787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e7a0cf280 0x558e7a649960 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b7a74e000 1 --2- 172.21.15.86:0/3828689787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558e7a4d6ea0 0x558e7a649420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b7a74e000 1 --2- 172.21.15.86:0/3828689787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7a4eac20 0x558e7a648ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b7a74e000 1 -- 172.21.15.86:0/3828689787 >> 172.21.15.86:0/3828689787 conn(0x558e7a10eff0 msgr2=0x558e7a5fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:02.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b7a74e000 1 -- 172.21.15.86:0/3828689787 shutdown_connections 2024-04-25T14:12:02.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.828+0000 7f0b7a74e000 1 -- 172.21.15.86:0/3828689787 wait complete. 2024-04-25T14:12:02.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.832+0000 7f0b7a74e000 1 -- start start 2024-04-25T14:12:02.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.832+0000 7f0b7a74e000 1 Processor -- start 2024-04-25T14:12:02.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.832+0000 7f0b7a74e000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558e7a0cf280 0x558e7a64fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.832+0000 7f0b7a74e000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7a4d6ea0 0x558e7a6503a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.832+0000 7f0b7a74e000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e7a4eac20 0x558e7a60b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.832+0000 7f0b7a74e000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558e7a45b220 con 0x558e7a4eac20 2024-04-25T14:12:02.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.832+0000 7f0b7a74e000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558e7a3f1ea0 con 0x558e7a4d6ea0 2024-04-25T14:12:02.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.832+0000 7f0b7a74e000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x558e7a448b00 con 0x558e7a0cf280 2024-04-25T14:12:02.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.832+0000 7f0b7a74e000 10 client.0 _ll_register_callbacks cb 0x558e7a45adc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:12:02.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.832+0000 7f0b71000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e7a4eac20 0x558e7a60b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.832+0000 7f0b6b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7a4d6ea0 0x558e7a6503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.832+0000 7f0b6be00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558e7a0cf280 0x558e7a64fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.832+0000 7f0b71000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e7a4eac20 0x558e7a60b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58446/0 (socket says 172.21.15.86:58446) 2024-04-25T14:12:02.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.832+0000 7f0b71000640 1 -- 172.21.15.86:0/1769853316 learned_addr learned my addr 172.21.15.86:0/1769853316 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:12:02.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.832+0000 7f0b6be00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558e7a0cf280 0x558e7a64fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:37732/0 (socket says 172.21.15.86:37732) 2024-04-25T14:12:02.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.832+0000 7f0b71000640 1 -- 172.21.15.86:0/1769853316 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558e7a0cf280 msgr2=0x558e7a64fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:12:02.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.832+0000 7f0b71000640 1 --2- 172.21.15.86:0/1769853316 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558e7a0cf280 0x558e7a64fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.832+0000 7f0b71000640 1 -- 172.21.15.86:0/1769853316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7a4d6ea0 msgr2=0x558e7a6503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:12:02.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.832+0000 7f0b71000640 1 --2- 172.21.15.86:0/1769853316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7a4d6ea0 0x558e7a6503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.832+0000 7f0b71000640 1 -- 172.21.15.86:0/1769853316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e7a3ed500 con 0x558e7a4eac20 2024-04-25T14:12:02.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.832+0000 7f0b71000640 1 --2- 172.21.15.86:0/1769853316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e7a4eac20 0x558e7a60b0c0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f0b6401b020 tx=0x7f0b6401b1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1623fc9ad0de399 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:02.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.832+0000 7f0b55600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:12:02.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b55600640 1 -- 172.21.15.86:0/1769853316 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b640c5070 con 0x558e7a4eac20 2024-04-25T14:12:02.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b55600640 1 -- 172.21.15.86:0/1769853316 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(141..141 src has 1..141) ==== 6445+0+0 (secure 0 0 0) 0x7f0b64003ba0 con 0x558e7a4eac20 2024-04-25T14:12:02.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b7a74e000 1 -- 172.21.15.86:0/1769853316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b640a5920 con 0x558e7a4eac20 2024-04-25T14:12:02.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b55600640 1 -- 172.21.15.86:0/1769853316 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) ==== 24+0+0 (secure 0 0 0) 0x7f0b640d2050 con 0x558e7a4eac20 2024-04-25T14:12:02.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b7a74e000 10 client.9891 fetch_fsmap learned FSMap version 106 2024-04-25T14:12:02.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b7a74e000 1 -- 172.21.15.86:0/1769853316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) -- 0x558e7a3d3ea0 con 0x558e7a4eac20 2024-04-25T14:12:02.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b55600640 1 -- 172.21.15.86:0/1769853316 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b640a5920 con 0x558e7a4eac20 2024-04-25T14:12:02.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b55600640 1 -- 172.21.15.86:0/1769853316 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 106) ==== 3514+0+0 (secure 0 0 0) 0x7f0b640d5020 con 0x558e7a4eac20 2024-04-25T14:12:02.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b7a74e000 10 client.9891 fetch_fsmap finished waiting for FSMap version 106 2024-04-25T14:12:02.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b7a74e000 10 client.9891 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T14:12:02.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b7a74e000 20 client.9891 populate_metadata read hostname 'smithi086' 2024-04-25T14:12:02.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b7a74e000 1 --2- 172.21.15.86:0/1769853316 >> [v2:172.21.15.86:6834/2571655765,v1:172.21.15.86:6835/2571655765] conn(0x558e7a0f37d0 0x558e7a60be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b7a74e000 4 client.9891 mds_command: new command op to 9546 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:12:02.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b7a74e000 1 -- 172.21.15.86:0/1769853316 --> [v2:172.21.15.86:6834/2571655765,v1:172.21.15.86:6835/2571655765] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e7a3e6be0 con 0x558e7a0f37d0 2024-04-25T14:12:02.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b6be00640 1 --2- 172.21.15.86:0/1769853316 >> [v2:172.21.15.86:6834/2571655765,v1:172.21.15.86:6835/2571655765] conn(0x558e7a0f37d0 0x558e7a60be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b55600640 10 client.9891 ms_handle_connect on v2:172.21.15.86:6834/2571655765 2024-04-25T14:12:02.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b6be00640 1 --2- 172.21.15.86:0/1769853316 >> [v2:172.21.15.86:6834/2571655765,v1:172.21.15.86:6835/2571655765] conn(0x558e7a0f37d0 0x558e7a60be30 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a9525e8018309fb server_cookie=add32b434e30cb81 in_seq=0 out_seq=0 2024-04-25T14:12:02.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b55600640 1 -- 172.21.15.86:0/1769853316 <== mds.0 v2:172.21.15.86:6834/2571655765 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558e7a3e6be0 con 0x558e7a0f37d0 2024-04-25T14:12:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b55600640 10 client.9891 handle_command_reply: tid=0 2024-04-25T14:12:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b7a74e000 1 client.9891 shutdown 2024-04-25T14:12:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b7a74e000 1 -- 172.21.15.86:0/1769853316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e7a4eac20 msgr2=0x558e7a60b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b7a74e000 1 --2- 172.21.15.86:0/1769853316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e7a4eac20 0x558e7a60b0c0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f0b6401b020 tx=0x7f0b6401b1b0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b7a74e000 1 -- 172.21.15.86:0/1769853316 shutdown_connections 2024-04-25T14:12:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b7a74e000 1 --2- 172.21.15.86:0/1769853316 >> [v2:172.21.15.86:6834/2571655765,v1:172.21.15.86:6835/2571655765] conn(0x558e7a0f37d0 0x558e7a60be30 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b7a74e000 1 --2- 172.21.15.86:0/1769853316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e7a4eac20 0x558e7a60b0c0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b7a74e000 1 --2- 172.21.15.86:0/1769853316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7a4d6ea0 0x558e7a6503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b55600640 0 client.9891 ms_handle_reset on v2:172.21.15.86:6834/2571655765 2024-04-25T14:12:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b7a74e000 1 --2- 172.21.15.86:0/1769853316 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558e7a0cf280 0x558e7a64fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b7a74e000 1 -- 172.21.15.86:0/1769853316 >> 172.21.15.86:0/1769853316 conn(0x558e7a10eff0 msgr2=0x558e7a5fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b7a74e000 1 -- 172.21.15.86:0/1769853316 shutdown_connections 2024-04-25T14:12:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b7a74e000 1 -- 172.21.15.86:0/1769853316 wait complete. 2024-04-25T14:12:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.836+0000 7f0b7a74e000 20 client.9891 trim_cache size 0 max 16384 2024-04-25T14:12:02.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.856+0000 7f0b7a74e000 1 Processor -- start 2024-04-25T14:12:02.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.856+0000 7f0b7a74e000 1 -- start start 2024-04-25T14:12:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.856+0000 7f0b7a74e000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7a0cf280 0x558e7a6552c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.856+0000 7f0b7a74e000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e7a4d6ea0 0x558e7a65eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.856+0000 7f0b7a74e000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558e7a4eac20 0x558e7a65f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.856+0000 7f0b7a74e000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c206830 con 0x558e7a4d6ea0 2024-04-25T14:12:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.856+0000 7f0b7a74e000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c0cc0a0 con 0x558e7a0cf280 2024-04-25T14:12:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.856+0000 7f0b7a74e000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0b6c205210 con 0x558e7a4eac20 2024-04-25T14:12:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.856+0000 7f0b6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e7a4d6ea0 0x558e7a65eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.856+0000 7f0b6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e7a4d6ea0 0x558e7a65eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58450/0 (socket says 172.21.15.86:58450) 2024-04-25T14:12:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.856+0000 7f0b6b400640 1 -- 172.21.15.86:0/1733689302 learned_addr learned my addr 172.21.15.86:0/1733689302 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:12:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.856+0000 7f0b6be00640 1 --2- 172.21.15.86:0/1733689302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7a0cf280 0x558e7a6552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.856+0000 7f0b71000640 1 --2- 172.21.15.86:0/1733689302 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558e7a4eac20 0x558e7a65f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.860+0000 7f0b6b400640 1 -- 172.21.15.86:0/1733689302 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558e7a4eac20 msgr2=0x558e7a65f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:12:02.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.860+0000 7f0b6b400640 1 --2- 172.21.15.86:0/1733689302 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558e7a4eac20 0x558e7a65f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.860+0000 7f0b6b400640 1 -- 172.21.15.86:0/1733689302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7a0cf280 msgr2=0x558e7a6552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:12:02.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.860+0000 7f0b6b400640 1 --2- 172.21.15.86:0/1733689302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7a0cf280 0x558e7a6552c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.860+0000 7f0b6b400640 1 -- 172.21.15.86:0/1733689302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e7a602840 con 0x558e7a4d6ea0 2024-04-25T14:12:02.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.860+0000 7f0b6be00640 1 --2- 172.21.15.86:0/1733689302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7a0cf280 0x558e7a6552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:12:02.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.860+0000 7f0b6b400640 1 --2- 172.21.15.86:0/1733689302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e7a4d6ea0 0x558e7a65eff0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0734e0 tx=0x7f0b58011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686c38008ff22500 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:02.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.860+0000 7f0b57e00640 1 -- 172.21.15.86:0/1733689302 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b58028070 con 0x558e7a4d6ea0 2024-04-25T14:12:02.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.860+0000 7f0b57e00640 1 -- 172.21.15.86:0/1733689302 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b5800e7b0 con 0x558e7a4d6ea0 2024-04-25T14:12:02.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.860+0000 7f0b57e00640 1 -- 172.21.15.86:0/1733689302 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5800eb60 con 0x558e7a4d6ea0 2024-04-25T14:12:02.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.860+0000 7f0b7a74e000 1 -- 172.21.15.86:0/1733689302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e7a4d6ea0 msgr2=0x558e7a65eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.860+0000 7f0b7a74e000 1 --2- 172.21.15.86:0/1733689302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e7a4d6ea0 0x558e7a65eff0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0734e0 tx=0x7f0b58011000 comp rx=0 tx=0).stop 2024-04-25T14:12:02.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.860+0000 7f0b7a74e000 1 -- 172.21.15.86:0/1733689302 shutdown_connections 2024-04-25T14:12:02.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.860+0000 7f0b7a74e000 1 --2- 172.21.15.86:0/1733689302 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558e7a4eac20 0x558e7a65f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.860+0000 7f0b7a74e000 1 --2- 172.21.15.86:0/1733689302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e7a4d6ea0 0x558e7a65eff0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.860+0000 7f0b7a74e000 1 --2- 172.21.15.86:0/1733689302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7a0cf280 0x558e7a6552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.860+0000 7f0b7a74e000 1 -- 172.21.15.86:0/1733689302 >> 172.21.15.86:0/1733689302 conn(0x558e7a10eff0 msgr2=0x558e7a64ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:02.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.860+0000 7f0b7a74e000 1 -- 172.21.15.86:0/1733689302 shutdown_connections 2024-04-25T14:12:02.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.860+0000 7f0b7a74e000 1 -- 172.21.15.86:0/1733689302 wait complete. 2024-04-25T14:12:02.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.860+0000 7f0b7a74e000 1 -- start start 2024-04-25T14:12:02.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.860+0000 7f0b7a74e000 1 Processor -- start 2024-04-25T14:12:02.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b7a74e000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558e7a0cf280 0x558e7a6bb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b7a74e000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e7a4d6ea0 0x558e7a6bbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b7a74e000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7a4eac20 0x558e7a653920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b7a74e000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558e7a44d130 con 0x558e7a4d6ea0 2024-04-25T14:12:02.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b7a74e000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558e7a3e37d0 con 0x558e7a4eac20 2024-04-25T14:12:02.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b7a74e000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x558e7a452450 con 0x558e7a0cf280 2024-04-25T14:12:02.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b7a74e000 10 client.0 _ll_register_callbacks cb 0x7f0b50003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:12:02.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e7a4d6ea0 0x558e7a6bbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e7a4d6ea0 0x558e7a6bbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58452/0 (socket says 172.21.15.86:58452) 2024-04-25T14:12:02.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b6b400640 1 -- 172.21.15.86:0/2865935067 learned_addr learned my addr 172.21.15.86:0/2865935067 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:12:02.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b6be00640 1 --2- 172.21.15.86:0/2865935067 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558e7a0cf280 0x558e7a6bb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b71000640 1 --2- 172.21.15.86:0/2865935067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7a4eac20 0x558e7a653920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b6b400640 1 -- 172.21.15.86:0/2865935067 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558e7a0cf280 msgr2=0x558e7a6bb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:12:02.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b6b400640 1 --2- 172.21.15.86:0/2865935067 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558e7a0cf280 0x558e7a6bb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b6b400640 1 -- 172.21.15.86:0/2865935067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7a4eac20 msgr2=0x558e7a653920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:12:02.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b6b400640 1 --2- 172.21.15.86:0/2865935067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7a4eac20 0x558e7a653920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b6b400640 1 -- 172.21.15.86:0/2865935067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e7a60b430 con 0x558e7a4d6ea0 2024-04-25T14:12:02.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b71000640 1 --2- 172.21.15.86:0/2865935067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7a4eac20 0x558e7a653920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:12:02.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b6b400640 1 --2- 172.21.15.86:0/2865935067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e7a4d6ea0 0x558e7a6bbad0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f0b58000f70 tx=0x7f0b580010b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72d942145fcbf45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:02.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b55600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:12:02.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b55600640 1 -- 172.21.15.86:0/2865935067 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b58028040 con 0x558e7a4d6ea0 2024-04-25T14:12:02.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b55600640 1 -- 172.21.15.86:0/2865935067 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(141..141 src has 1..141) ==== 6445+0+0 (secure 0 0 0) 0x7f0b5800ed90 con 0x558e7a4d6ea0 2024-04-25T14:12:02.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b7a74e000 1 -- 172.21.15.86:0/2865935067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b5800e7b0 con 0x558e7a4d6ea0 2024-04-25T14:12:02.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b55600640 1 -- 172.21.15.86:0/2865935067 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) ==== 24+0+0 (secure 0 0 0) 0x7f0b580047f0 con 0x558e7a4d6ea0 2024-04-25T14:12:02.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b7a74e000 10 client.9897 fetch_fsmap learned FSMap version 106 2024-04-25T14:12:02.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b7a74e000 1 -- 172.21.15.86:0/2865935067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) -- 0x558e7a406c70 con 0x558e7a4d6ea0 2024-04-25T14:12:02.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b55600640 1 -- 172.21.15.86:0/2865935067 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b5800e7b0 con 0x558e7a4d6ea0 2024-04-25T14:12:02.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b55600640 1 -- 172.21.15.86:0/2865935067 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 106) ==== 3514+0+0 (secure 0 0 0) 0x7f0b5802b070 con 0x558e7a4d6ea0 2024-04-25T14:12:02.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b7a74e000 10 client.9897 fetch_fsmap finished waiting for FSMap version 106 2024-04-25T14:12:02.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b7a74e000 10 client.9897 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T14:12:02.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b7a74e000 20 client.9897 populate_metadata read hostname 'smithi086' 2024-04-25T14:12:02.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b7a74e000 1 --2- 172.21.15.86:0/2865935067 >> [v2:172.21.15.86:6834/2571655765,v1:172.21.15.86:6835/2571655765] conn(0x558e7a0e1830 0x558e7a6542e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:02.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b7a74e000 4 client.9897 mds_command: new command op to 9546 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:12:02.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b7a74e000 1 -- 172.21.15.86:0/2865935067 --> [v2:172.21.15.86:6834/2571655765,v1:172.21.15.86:6835/2571655765] -- command(tid 0: {"prefix": "status"}) -- 0x558e7a45ba30 con 0x558e7a0e1830 2024-04-25T14:12:02.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b6be00640 1 --2- 172.21.15.86:0/2865935067 >> [v2:172.21.15.86:6834/2571655765,v1:172.21.15.86:6835/2571655765] conn(0x558e7a0e1830 0x558e7a6542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:02.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b6be00640 1 --2- 172.21.15.86:0/2865935067 >> [v2:172.21.15.86:6834/2571655765,v1:172.21.15.86:6835/2571655765] conn(0x558e7a0e1830 0x558e7a6542e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de12a7553d08937f server_cookie=1c7749d7c5453f4a in_seq=0 out_seq=0 2024-04-25T14:12:02.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b55600640 10 client.9897 ms_handle_connect on v2:172.21.15.86:6834/2571655765 2024-04-25T14:12:02.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b55600640 1 -- 172.21.15.86:0/2865935067 <== mds.0 v2:172.21.15.86:6834/2571655765 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x558e7a45ba30 con 0x558e7a0e1830 2024-04-25T14:12:02.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b55600640 10 client.9897 handle_command_reply: tid=0 2024-04-25T14:12:02.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b7a74e000 1 client.9897 shutdown 2024-04-25T14:12:02.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b7a74e000 1 -- 172.21.15.86:0/2865935067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e7a4d6ea0 msgr2=0x558e7a6bbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b7a74e000 1 --2- 172.21.15.86:0/2865935067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e7a4d6ea0 0x558e7a6bbad0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f0b58000f70 tx=0x7f0b580010b0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b7a74e000 1 -- 172.21.15.86:0/2865935067 shutdown_connections 2024-04-25T14:12:02.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b7a74e000 1 --2- 172.21.15.86:0/2865935067 >> [v2:172.21.15.86:6834/2571655765,v1:172.21.15.86:6835/2571655765] conn(0x558e7a0e1830 0x558e7a6542e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b7a74e000 1 --2- 172.21.15.86:0/2865935067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7a4eac20 0x558e7a653920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b7a74e000 1 --2- 172.21.15.86:0/2865935067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e7a4d6ea0 0x558e7a6bbad0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b7a74e000 1 --2- 172.21.15.86:0/2865935067 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558e7a0cf280 0x558e7a6bb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b55600640 0 client.9897 ms_handle_reset on v2:172.21.15.86:6834/2571655765 2024-04-25T14:12:02.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b7a74e000 1 -- 172.21.15.86:0/2865935067 >> 172.21.15.86:0/2865935067 conn(0x558e7a10eff0 msgr2=0x558e7a64f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:02.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b7a74e000 1 -- 172.21.15.86:0/2865935067 shutdown_connections 2024-04-25T14:12:02.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b7a74e000 1 -- 172.21.15.86:0/2865935067 wait complete. 2024-04-25T14:12:02.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.864+0000 7f0b7a74e000 20 client.9897 trim_cache size 0 max 16384 2024-04-25T14:12:02.867 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:12:02.867 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb", 2024-04-25T14:12:02.867 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-25T14:12:02.867 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 9546, 2024-04-25T14:12:02.867 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T14:12:02.867 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T14:12:02.867 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T14:12:02.867 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 29.290001873000001, 2024-04-25T14:12:02.867 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 106, 2024-04-25T14:12:02.867 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 141, 2024-04-25T14:12:02.867 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 141, 2024-04-25T14:12:02.867 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 34.619136263000001 2024-04-25T14:12:02.867 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:12:02.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.868+0000 7f0b73000640 1 -- 172.21.15.86:0/173715420 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f0b4c06f570 msgr2=0x7f0b4c071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.868+0000 7f0b73000640 1 --2- 172.21.15.86:0/173715420 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f0b4c06f570 0x7f0b4c071a80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c1565a0 tx=0x7f0b58009040 comp rx=0 tx=0).stop 2024-04-25T14:12:02.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.868+0000 7f0b73000640 1 -- 172.21.15.86:0/173715420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b6c0b3a90 msgr2=0x7f0b6c158490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:02.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.868+0000 7f0b73000640 1 --2- 172.21.15.86:0/173715420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b6c0b3a90 0x7f0b6c158490 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00e870 tx=0x7f0b5c00ec70 comp rx=0 tx=0).stop 2024-04-25T14:12:02.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.868+0000 7f0b73000640 1 -- 172.21.15.86:0/173715420 shutdown_connections 2024-04-25T14:12:02.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.868+0000 7f0b73000640 1 --2- 172.21.15.86:0/173715420 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f0b4c06f570 0x7f0b4c071a80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.868+0000 7f0b73000640 1 --2- 172.21.15.86:0/173715420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b6c0cf500 0x7f0b6c0b0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.868+0000 7f0b73000640 1 --2- 172.21.15.86:0/173715420 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b6c0ce430 0x7f0b6c0b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.868+0000 7f0b73000640 1 --2- 172.21.15.86:0/173715420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b6c0b3a90 0x7f0b6c158490 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:02.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.868+0000 7f0b73000640 1 -- 172.21.15.86:0/173715420 >> 172.21.15.86:0/173715420 conn(0x7f0b6c09ba30 msgr2=0x7f0b6c156690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:02.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.868+0000 7f0b73000640 1 -- 172.21.15.86:0/173715420 shutdown_connections 2024-04-25T14:12:02.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:02.868+0000 7f0b73000640 1 -- 172.21.15.86:0/173715420 wait complete. 2024-04-25T14:12:02.884 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:12:03.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.032+0000 7fa55a000640 1 Processor -- start 2024-04-25T14:12:03.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.032+0000 7fa55a000640 1 -- start start 2024-04-25T14:12:03.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.036+0000 7fa55a000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa55406fd50 0x7fa554070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:03.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.036+0000 7fa55a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa55415feb0 0x7fa5541602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:03.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.036+0000 7fa55a000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5540b0a60 con 0x7fa55415c2a0 2024-04-25T14:12:03.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.036+0000 7fa55a000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa5540ae840 con 0x7fa55415feb0 2024-04-25T14:12:03.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.036+0000 7fa55a000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa55406ef90 con 0x7fa55406fd50 2024-04-25T14:12:03.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.036+0000 7fa553e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa55415feb0 0x7fa5541602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:03.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.036+0000 7fa553e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa55415feb0 0x7fa5541602c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58466/0 (socket says 172.21.15.86:58466) 2024-04-25T14:12:03.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.036+0000 7fa553e00640 1 -- 172.21.15.86:0/3202290546 learned_addr learned my addr 172.21.15.86:0/3202290546 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:12:03.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.036+0000 7fa552a00640 1 --2- 172.21.15.86:0/3202290546 >> v2:172.21.15.155:3300/0 conn(0x7fa55406fd50 0x7fa554070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:03.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.036+0000 7fa553e00640 1 -- 172.21.15.86:0/3202290546 >> v2:172.21.15.155:3300/0 conn(0x7fa55406fd50 msgr2=0x7fa554070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:12:03.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.036+0000 7fa553e00640 1 --2- 172.21.15.86:0/3202290546 >> v2:172.21.15.155:3300/0 conn(0x7fa55406fd50 0x7fa554070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:03.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.036+0000 7fa553e00640 1 -- 172.21.15.86:0/3202290546 >> v1:172.21.15.155:6789/0 conn(0x7fa55415c2a0 legacy=0x7fa5540ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:03.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.036+0000 7fa553e00640 1 -- 172.21.15.86:0/3202290546 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55406f270 con 0x7fa55415feb0 2024-04-25T14:12:03.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.040+0000 7fa553e00640 1 --2- 172.21.15.86:0/3202290546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa55415feb0 0x7fa5541602c0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fa554074f40 tx=0x7fa54c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63bfd28c7e3270ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:03.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.040+0000 7fa552000640 1 -- 172.21.15.86:0/3202290546 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c0a5660 con 0x7fa55415feb0 2024-04-25T14:12:03.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.040+0000 7fa552000640 1 -- 172.21.15.86:0/3202290546 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa54c0a5800 con 0x7fa55415feb0 2024-04-25T14:12:03.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.040+0000 7fa552000640 1 -- 172.21.15.86:0/3202290546 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c0a5bd0 con 0x7fa55415feb0 2024-04-25T14:12:03.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.040+0000 7fa55a000640 1 -- 172.21.15.86:0/3202290546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa55415feb0 msgr2=0x7fa5541602c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:03.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.040+0000 7fa55a000640 1 --2- 172.21.15.86:0/3202290546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa55415feb0 0x7fa5541602c0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fa554074f40 tx=0x7fa54c01c930 comp rx=0 tx=0).stop 2024-04-25T14:12:03.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.040+0000 7fa55a000640 1 -- 172.21.15.86:0/3202290546 shutdown_connections 2024-04-25T14:12:03.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.040+0000 7fa55a000640 1 --2- 172.21.15.86:0/3202290546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa55415feb0 0x7fa5541602c0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:03.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.040+0000 7fa55a000640 1 --2- 172.21.15.86:0/3202290546 >> v2:172.21.15.155:3300/0 conn(0x7fa55406fd50 0x7fa554070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:03.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.040+0000 7fa55a000640 1 -- 172.21.15.86:0/3202290546 >> 172.21.15.86:0/3202290546 conn(0x7fa5540b4650 msgr2=0x7fa5540b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:03.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.040+0000 7fa55a000640 1 -- 172.21.15.86:0/3202290546 shutdown_connections 2024-04-25T14:12:03.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.040+0000 7fa55a000640 1 -- 172.21.15.86:0/3202290546 wait complete. 2024-04-25T14:12:03.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.044+0000 7fa55a000640 1 Processor -- start 2024-04-25T14:12:03.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.044+0000 7fa55a000640 1 -- start start 2024-04-25T14:12:03.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.044+0000 7fa55a000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa55406fd50 0x7fa5540af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:03.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.044+0000 7fa55a000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa55415c2a0 0x7fa5541f8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:03.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.044+0000 7fa55a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa55415feb0 0x7fa55415f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:03.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.044+0000 7fa55a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa55415c6b0 con 0x7fa55415feb0 2024-04-25T14:12:03.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.044+0000 7fa55a000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa5540d0fc0 con 0x7fa55415c2a0 2024-04-25T14:12:03.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.044+0000 7fa55a000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa55415c830 con 0x7fa55406fd50 2024-04-25T14:12:03.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.044+0000 7fa553e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa55415feb0 0x7fa55415f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:03.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.044+0000 7fa553e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa55415feb0 0x7fa55415f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58476/0 (socket says 172.21.15.86:58476) 2024-04-25T14:12:03.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.044+0000 7fa553e00640 1 -- 172.21.15.86:0/2564018943 learned_addr learned my addr 172.21.15.86:0/2564018943 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:12:03.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.044+0000 7fa553400640 1 --2- 172.21.15.86:0/2564018943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa55406fd50 0x7fa5540af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:03.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.044+0000 7fa552a00640 1 --2- 172.21.15.86:0/2564018943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa55415c2a0 0x7fa5541f8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:03.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.044+0000 7fa553e00640 1 -- 172.21.15.86:0/2564018943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa55406fd50 msgr2=0x7fa5540af0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:03.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.044+0000 7fa553e00640 1 --2- 172.21.15.86:0/2564018943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa55406fd50 0x7fa5540af0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:03.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.044+0000 7fa553e00640 1 -- 172.21.15.86:0/2564018943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa55415c2a0 msgr2=0x7fa5541f8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:03.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.044+0000 7fa553e00640 1 --2- 172.21.15.86:0/2564018943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa55415c2a0 0x7fa5541f8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:03.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.044+0000 7fa553e00640 1 -- 172.21.15.86:0/2564018943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5540af620 con 0x7fa55415feb0 2024-04-25T14:12:03.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.044+0000 7fa553e00640 1 --2- 172.21.15.86:0/2564018943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa55415feb0 0x7fa55415f0c0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fa54c019540 tx=0x7fa54c0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:03.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.044+0000 7fa54b400640 1 -- 172.21.15.86:0/2564018943 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c009510 con 0x7fa55415feb0 2024-04-25T14:12:03.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.044+0000 7fa54b400640 1 -- 172.21.15.86:0/2564018943 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa54c0be070 con 0x7fa55415feb0 2024-04-25T14:12:03.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.044+0000 7fa54b400640 1 -- 172.21.15.86:0/2564018943 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c0b88b0 con 0x7fa55415feb0 2024-04-25T14:12:03.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.044+0000 7fa55a000640 1 -- 172.21.15.86:0/2564018943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5540b3ad0 con 0x7fa55415feb0 2024-04-25T14:12:03.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.044+0000 7fa55a000640 1 -- 172.21.15.86:0/2564018943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa55415e8c0 con 0x7fa55415feb0 2024-04-25T14:12:03.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.048+0000 7fa55a000640 1 -- 172.21.15.86:0/2564018943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa55415e3d0 con 0x7fa55415feb0 2024-04-25T14:12:03.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.048+0000 7fa54b400640 1 -- 172.21.15.86:0/2564018943 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 72) ==== 93318+0+0 (secure 0 0 0) 0x7fa54c0b8a50 con 0x7fa55415feb0 2024-04-25T14:12:03.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.048+0000 7fa54b400640 1 --2- 172.21.15.86:0/2564018943 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fa53006f5a0 0x7fa530071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:03.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.048+0000 7fa54b400640 1 -- 172.21.15.86:0/2564018943 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 6445+0+0 (secure 0 0 0) 0x7fa54c129fc0 con 0x7fa55415feb0 2024-04-25T14:12:03.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.048+0000 7fa553400640 1 --2- 172.21.15.86:0/2564018943 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fa53006f5a0 0x7fa530071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:03.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.048+0000 7fa553400640 1 --2- 172.21.15.86:0/2564018943 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fa53006f5a0 0x7fa530071ab0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa5540b4200 tx=0x7fa544009040 comp rx=0 tx=0).ready entity=mgr.9313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:03.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.052+0000 7fa54b400640 1 -- 172.21.15.86:0/2564018943 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa54c0c7020 con 0x7fa55415feb0 2024-04-25T14:12:03.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.240+0000 7fa55a000640 1 -- 172.21.15.86:0/2564018943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5540b4af0 con 0x7fa55415feb0 2024-04-25T14:12:03.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.240+0000 7fa54b400640 1 -- 172.21.15.86:0/2564018943 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) ==== 95+0+5207 (secure 0 0 0) 0x7fa54c0165e0 con 0x7fa55415feb0 2024-04-25T14:12:03.239 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:12:03.239 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":106,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/1828317917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1828317917},{"type":"v1","addr":"172.21.15.155:6834","nonce":1828317917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":9124,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1389714390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1389714390},{"type":"v1","addr":"172.21.15.86:6837","nonce":1389714390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:11:32.981538+0000","modified":"2024-04-25T14:12:00.834239+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":9546,"mds_1":9136},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9136":{"gid":9136,"name":"d","rank":1,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/91793677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":91793677},{"type":"v1","addr":"172.21.15.155:6835","nonce":91793677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9546":{"gid":9546,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/2571655765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2571655765},{"type":"v1","addr":"172.21.15.86:6835","nonce":2571655765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9546,"qdb_cluster":[9136,9546]},"id":10}]} 2024-04-25T14:12:03.239 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 106 2024-04-25T14:12:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.244+0000 7fa55a000640 1 -- 172.21.15.86:0/2564018943 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fa53006f5a0 msgr2=0x7fa530071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.244+0000 7fa55a000640 1 --2- 172.21.15.86:0/2564018943 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fa53006f5a0 0x7fa530071ab0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa5540b4200 tx=0x7fa544009040 comp rx=0 tx=0).stop 2024-04-25T14:12:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.244+0000 7fa55a000640 1 -- 172.21.15.86:0/2564018943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa55415feb0 msgr2=0x7fa55415f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.244+0000 7fa55a000640 1 --2- 172.21.15.86:0/2564018943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa55415feb0 0x7fa55415f0c0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fa54c019540 tx=0x7fa54c0196d0 comp rx=0 tx=0).stop 2024-04-25T14:12:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.244+0000 7fa55a000640 1 -- 172.21.15.86:0/2564018943 shutdown_connections 2024-04-25T14:12:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.244+0000 7fa55a000640 1 --2- 172.21.15.86:0/2564018943 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fa53006f5a0 0x7fa530071ab0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.244+0000 7fa55a000640 1 --2- 172.21.15.86:0/2564018943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa55415feb0 0x7fa55415f0c0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.244+0000 7fa55a000640 1 --2- 172.21.15.86:0/2564018943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa55415c2a0 0x7fa5541f8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.244+0000 7fa55a000640 1 --2- 172.21.15.86:0/2564018943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa55406fd50 0x7fa5540af0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.244+0000 7fa55a000640 1 -- 172.21.15.86:0/2564018943 >> 172.21.15.86:0/2564018943 conn(0x7fa5540b4650 msgr2=0x7fa55409a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.244+0000 7fa55a000640 1 -- 172.21.15.86:0/2564018943 shutdown_connections 2024-04-25T14:12:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:03.244+0000 7fa55a000640 1 -- 172.21.15.86:0/2564018943 wait complete. 2024-04-25T14:12:03.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:03.261 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test_mds_metrics_0) 2024-04-25T14:12:03.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:03.276 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 test_mds_metrics_0) 2024-04-25T14:12:03.334 INFO:tasks.cephfs.test_mds_metrics:pinning directory test_mds_metrics_0 to rank 0 2024-04-25T14:12:03.334 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:03.334 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:03.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:03.387 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.0 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:03.467 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:03.467 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:03.467 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226925 s, 46.2 MB/s 2024-04-25T14:12:03.468 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:03.468 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:03.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:03.520 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.1 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:03.596 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:03.596 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:03.596 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243994 s, 43.0 MB/s 2024-04-25T14:12:03.597 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:03.597 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:03.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:03.647 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.2 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:03.724 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:03.724 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:03.724 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250077 s, 41.9 MB/s 2024-04-25T14:12:03.725 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:03.725 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:03.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:03.780 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.3 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:03.849 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:03.849 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:03.849 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171781 s, 61.0 MB/s 2024-04-25T14:12:03.850 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:03.850 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:03.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:03.899 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.4 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:03.966 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:03.966 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:03.966 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140993 s, 74.4 MB/s 2024-04-25T14:12:03.967 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:03.967 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:04.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:04.019 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.5 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:04.098 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:04.098 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:04.098 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240174 s, 43.7 MB/s 2024-04-25T14:12:04.099 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:04.099 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:04.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:04.147 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.6 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:04.215 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:04.215 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:04.215 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194192 s, 54.0 MB/s 2024-04-25T14:12:04.216 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:04.216 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:04.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:04.268 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.7 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:04.336 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:04.336 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:04.337 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173426 s, 60.5 MB/s 2024-04-25T14:12:04.338 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:04.338 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:04.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:04.388 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.8 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:04.463 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:04.464 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:04.464 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023483 s, 44.7 MB/s 2024-04-25T14:12:04.465 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:04.465 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:04.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:04.516 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.9 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:04.584 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:04.584 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:04.584 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165675 s, 63.3 MB/s 2024-04-25T14:12:04.585 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:04.585 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:04.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:04.636 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.10 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:04.710 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:04.710 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:04.710 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221414 s, 47.4 MB/s 2024-04-25T14:12:04.711 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:04.711 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:04.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:04.759 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.11 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:04.829 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:04.829 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:04.829 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175082 s, 59.9 MB/s 2024-04-25T14:12:04.830 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:04.830 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:04.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:04.884 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.12 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:04.964 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:04.964 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:04.964 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245337 s, 42.7 MB/s 2024-04-25T14:12:04.965 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:04.965 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:05.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:05.016 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.13 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:05.086 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:05.086 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:05.086 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181238 s, 57.9 MB/s 2024-04-25T14:12:05.087 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:05.087 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:05.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:05.136 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.14 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:05.214 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:05.215 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:05.215 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232433 s, 45.1 MB/s 2024-04-25T14:12:05.216 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:05.216 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:05.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:05.268 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.15 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:05.338 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:05.338 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:05.338 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018479 s, 56.7 MB/s 2024-04-25T14:12:05.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:05.339 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test_mds_metrics_1) 2024-04-25T14:12:05.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:05.395 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 test_mds_metrics_1) 2024-04-25T14:12:05.455 INFO:tasks.cephfs.test_mds_metrics:pinning directory test_mds_metrics_1 to rank 1 2024-04-25T14:12:05.455 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:05.455 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:05.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:05.508 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.0 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:05.578 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:05.578 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:05.579 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184122 s, 57.0 MB/s 2024-04-25T14:12:05.580 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:05.580 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:05.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:05.632 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.1 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:05.705 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:05.705 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:05.705 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212266 s, 49.4 MB/s 2024-04-25T14:12:05.706 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:05.706 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:05.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:05.756 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.2 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:05.822 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:05.822 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:05.822 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141541 s, 74.1 MB/s 2024-04-25T14:12:05.823 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:05.823 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:05.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:05.875 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.3 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:05.952 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:05.953 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:05.953 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251742 s, 41.7 MB/s 2024-04-25T14:12:05.954 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:05.954 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:06.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:06.004 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.4 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:06.074 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:06.074 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:06.074 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189042 s, 55.5 MB/s 2024-04-25T14:12:06.075 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:06.075 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:06.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:06.127 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.5 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:06.197 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:06.197 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:06.197 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017366 s, 60.4 MB/s 2024-04-25T14:12:06.198 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:06.198 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:06.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:06.251 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.6 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:06.325 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:06.325 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:06.325 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021703 s, 48.3 MB/s 2024-04-25T14:12:06.326 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:06.326 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:06.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:06.375 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.7 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:06.446 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:06.446 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:06.446 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185751 s, 56.5 MB/s 2024-04-25T14:12:06.447 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:06.447 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:06.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:06.496 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.8 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:06.563 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:06.563 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:06.563 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154306 s, 68.0 MB/s 2024-04-25T14:12:06.564 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:06.564 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:06.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:06.615 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.9 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:06.692 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:06.692 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:06.692 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207101 s, 50.6 MB/s 2024-04-25T14:12:06.693 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:06.693 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:06.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:06.743 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.10 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:06.810 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:06.810 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:06.810 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145123 s, 72.3 MB/s 2024-04-25T14:12:06.811 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:06.811 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:06.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:06.859 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.11 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:06.928 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161967 s, 64.7 MB/s 2024-04-25T14:12:06.929 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:06.929 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:06.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:06.979 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.12 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:07.056 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:07.056 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:07.056 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206045 s, 50.9 MB/s 2024-04-25T14:12:07.057 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:07.057 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:07.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:07.108 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.13 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:07.177 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:07.177 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:07.177 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174939 s, 59.9 MB/s 2024-04-25T14:12:07.178 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:07.178 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:07.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:07.227 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.14 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:07.302 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:07.302 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:07.302 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186272 s, 56.3 MB/s 2024-04-25T14:12:07.303 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:12:07.303 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:12:07.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:07.355 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.15 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:12:07.426 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:12:07.426 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:12:07.426 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183282 s, 57.2 MB/s 2024-04-25T14:12:07.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:07.427 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest/mnt.1 && exec find .) 2024-04-25T14:12:07.454 INFO:teuthology.orchestra.run.smithi155.stdout:. 2024-04-25T14:12:07.454 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0 2024-04-25T14:12:07.454 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.5 2024-04-25T14:12:07.454 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.7 2024-04-25T14:12:07.454 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.6 2024-04-25T14:12:07.454 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.10 2024-04-25T14:12:07.454 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.4 2024-04-25T14:12:07.455 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.12 2024-04-25T14:12:07.455 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.1 2024-04-25T14:12:07.455 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.9 2024-04-25T14:12:07.455 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.13 2024-04-25T14:12:07.455 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.0 2024-04-25T14:12:07.455 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.11 2024-04-25T14:12:07.455 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.8 2024-04-25T14:12:07.455 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.3 2024-04-25T14:12:07.455 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.2 2024-04-25T14:12:07.455 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.14 2024-04-25T14:12:07.455 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.15 2024-04-25T14:12:07.455 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1 2024-04-25T14:12:07.455 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.5 2024-04-25T14:12:07.455 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.7 2024-04-25T14:12:07.456 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.6 2024-04-25T14:12:07.456 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.10 2024-04-25T14:12:07.456 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.4 2024-04-25T14:12:07.456 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.12 2024-04-25T14:12:07.456 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.1 2024-04-25T14:12:07.456 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.9 2024-04-25T14:12:07.456 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.13 2024-04-25T14:12:07.456 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.0 2024-04-25T14:12:07.456 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.11 2024-04-25T14:12:07.456 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.8 2024-04-25T14:12:07.456 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.3 2024-04-25T14:12:07.456 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.2 2024-04-25T14:12:07.456 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.14 2024-04-25T14:12:07.457 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.15 2024-04-25T14:12:12.458 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:12:12.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.620+0000 7fb34b200640 1 Processor -- start 2024-04-25T14:12:12.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.620+0000 7fb34b200640 1 -- start start 2024-04-25T14:12:12.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.620+0000 7fb34b200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb34c06f990 0x7fb34c06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:12.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.620+0000 7fb34b200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb34c0b6530 0x7fb34c0b6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:12.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.620+0000 7fb34b200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb34c0702e0 con 0x7fb34c1615c0 2024-04-25T14:12:12.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.620+0000 7fb34b200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb34c1622f0 con 0x7fb34c06f990 2024-04-25T14:12:12.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.620+0000 7fb34b200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb34c0d1730 con 0x7fb34c0b6530 2024-04-25T14:12:12.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.620+0000 7fb349e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb34c06f990 0x7fb34c06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:12.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.620+0000 7fb341400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb34c1615c0 0x7fb34c1619d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:55236/0 (socket says 172.21.15.86:55236) 2024-04-25T14:12:12.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.620+0000 7fb341400640 1 -- 172.21.15.86:0/572391613 learned_addr learned my addr 172.21.15.86:0/572391613 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:12:12.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.624+0000 7fb34a800640 1 --2- 172.21.15.86:0/572391613 >> v2:172.21.15.155:3300/0 conn(0x7fb34c0b6530 0x7fb34c0b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:12.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.624+0000 7fb349e00640 1 -- 172.21.15.86:0/572391613 >> v2:172.21.15.155:3300/0 conn(0x7fb34c0b6530 msgr2=0x7fb34c0b6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:12:12.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.624+0000 7fb349e00640 1 --2- 172.21.15.86:0/572391613 >> v2:172.21.15.155:3300/0 conn(0x7fb34c0b6530 0x7fb34c0b6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:12.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.624+0000 7fb349e00640 1 -- 172.21.15.86:0/572391613 >> v1:172.21.15.86:6789/0 conn(0x7fb34c1615c0 legacy=0x7fb34c1619d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:12.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.624+0000 7fb349e00640 1 -- 172.21.15.86:0/572391613 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34c0d1a10 con 0x7fb34c06f990 2024-04-25T14:12:12.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.624+0000 7fb349400640 1 -- 172.21.15.86:0/572391613 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4038085195 0 0) 0x7fb334001120 con 0x7fb34c1615c0 2024-04-25T14:12:12.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.624+0000 7fb349e00640 1 --2- 172.21.15.86:0/572391613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb34c06f990 0x7fb34c06fda0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0705e0 tx=0x7fb33c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83fbb22c72f0994a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:12.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.624+0000 7fb349400640 1 -- 172.21.15.86:0/572391613 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c0b0070 con 0x7fb34c06f990 2024-04-25T14:12:12.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.624+0000 7fb349400640 1 -- 172.21.15.86:0/572391613 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb33c0ab600 con 0x7fb34c06f990 2024-04-25T14:12:12.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.624+0000 7fb349400640 1 -- 172.21.15.86:0/572391613 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c0ab9d0 con 0x7fb34c06f990 2024-04-25T14:12:12.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.628+0000 7fb34b200640 1 -- 172.21.15.86:0/572391613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb34c06f990 msgr2=0x7fb34c06fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:12.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.628+0000 7fb34b200640 1 --2- 172.21.15.86:0/572391613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb34c06f990 0x7fb34c06fda0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0705e0 tx=0x7fb33c01c3f0 comp rx=0 tx=0).stop 2024-04-25T14:12:12.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.628+0000 7fb34b200640 1 -- 172.21.15.86:0/572391613 shutdown_connections 2024-04-25T14:12:12.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.628+0000 7fb34b200640 1 --2- 172.21.15.86:0/572391613 >> v2:172.21.15.155:3300/0 conn(0x7fb34c0b6530 0x7fb34c0b6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:12.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.628+0000 7fb34b200640 1 --2- 172.21.15.86:0/572391613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb34c06f990 0x7fb34c06fda0 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:12.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.628+0000 7fb34b200640 1 -- 172.21.15.86:0/572391613 >> 172.21.15.86:0/572391613 conn(0x7fb34c0cfea0 msgr2=0x7fb34c0d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:12.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.628+0000 7fb34b200640 1 -- 172.21.15.86:0/572391613 shutdown_connections 2024-04-25T14:12:12.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.628+0000 7fb34b200640 1 -- 172.21.15.86:0/572391613 wait complete. 2024-04-25T14:12:12.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.628+0000 7fb34b200640 1 Processor -- start 2024-04-25T14:12:12.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.628+0000 7fb34b200640 1 -- start start 2024-04-25T14:12:12.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.632+0000 7fb34b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb34c06f990 0x7fb34c0c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:12.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.632+0000 7fb34b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb34c0b6530 0x7fb34c1f9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:12.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.632+0000 7fb34b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb34c1615c0 0x7fb34c15ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:12.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.632+0000 7fb34b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb34c0d2880 con 0x7fb34c06f990 2024-04-25T14:12:12.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.632+0000 7fb34b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb34c15df20 con 0x7fb34c1615c0 2024-04-25T14:12:12.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.632+0000 7fb34b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb34c16b170 con 0x7fb34c0b6530 2024-04-25T14:12:12.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.632+0000 7fb349e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb34c06f990 0x7fb34c0c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:12.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.632+0000 7fb349e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb34c06f990 0x7fb34c0c9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57636/0 (socket says 172.21.15.86:57636) 2024-04-25T14:12:12.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.632+0000 7fb349e00640 1 -- 172.21.15.86:0/825864578 learned_addr learned my addr 172.21.15.86:0/825864578 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:12:12.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.632+0000 7fb349e00640 1 -- 172.21.15.86:0/825864578 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb34c0b6530 msgr2=0x7fb34c1f9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:12.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.632+0000 7fb349e00640 1 --2- 172.21.15.86:0/825864578 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb34c0b6530 0x7fb34c1f9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:12.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.632+0000 7fb34a800640 1 --2- 172.21.15.86:0/825864578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb34c1615c0 0x7fb34c15ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:12.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.632+0000 7fb349e00640 1 -- 172.21.15.86:0/825864578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb34c1615c0 msgr2=0x7fb34c15ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:12.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.632+0000 7fb349e00640 1 --2- 172.21.15.86:0/825864578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb34c1615c0 0x7fb34c15ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:12.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.632+0000 7fb349e00640 1 -- 172.21.15.86:0/825864578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34c0cc340 con 0x7fb34c06f990 2024-04-25T14:12:12.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.632+0000 7fb349e00640 1 --2- 172.21.15.86:0/825864578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb34c06f990 0x7fb34c0c9620 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fb33c0ae040 tx=0x7fb33c019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:12.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.632+0000 7fb342a00640 1 -- 172.21.15.86:0/825864578 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c0b0040 con 0x7fb34c06f990 2024-04-25T14:12:12.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.632+0000 7fb342a00640 1 -- 172.21.15.86:0/825864578 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb33c0ba070 con 0x7fb34c06f990 2024-04-25T14:12:12.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.632+0000 7fb34b200640 1 -- 172.21.15.86:0/825864578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb34c0cc5b0 con 0x7fb34c06f990 2024-04-25T14:12:12.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.632+0000 7fb342a00640 1 -- 172.21.15.86:0/825864578 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c0b4870 con 0x7fb34c06f990 2024-04-25T14:12:12.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.632+0000 7fb34b200640 1 -- 172.21.15.86:0/825864578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb34c0ccb70 con 0x7fb34c06f990 2024-04-25T14:12:12.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.632+0000 7fb34b200640 1 -- 172.21.15.86:0/825864578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb34c076b80 con 0x7fb34c06f990 2024-04-25T14:12:12.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.632+0000 7fb342a00640 1 -- 172.21.15.86:0/825864578 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 72) ==== 93318+0+0 (secure 0 0 0) 0x7fb33c0ab4c0 con 0x7fb34c06f990 2024-04-25T14:12:12.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.636+0000 7fb342a00640 1 --2- 172.21.15.86:0/825864578 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fb32806f570 0x7fb328071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:12.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.636+0000 7fb342a00640 1 -- 172.21.15.86:0/825864578 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 6445+0+0 (secure 0 0 0) 0x7fb33c129650 con 0x7fb34c06f990 2024-04-25T14:12:12.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.636+0000 7fb341400640 1 --2- 172.21.15.86:0/825864578 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fb32806f570 0x7fb328071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:12.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.636+0000 7fb341400640 1 --2- 172.21.15.86:0/825864578 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fb32806f570 0x7fb328071a80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb34c15c6a0 tx=0x7fb334007970 comp rx=0 tx=0).ready entity=mgr.9313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:12.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.640+0000 7fb342a00640 1 -- 172.21.15.86:0/825864578 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb33c016070 con 0x7fb34c06f990 2024-04-25T14:12:12.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.820+0000 7fb34b200640 1 -- 172.21.15.86:0/825864578 --> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7fb34c069d00 con 0x7fb32806f570 2024-04-25T14:12:12.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.824+0000 7fb342a00640 1 -- 172.21.15.86:0/825864578 <== mgr.9313 v2:172.21.15.86:6832/1173654665 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3053 (secure 0 0 0) 0x7fb34c069d00 con 0x7fb32806f570 2024-04-25T14:12:12.820 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {"cephfs": {"client.9292": {"hostname": "smithi155", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:192.168.0.1"}, "client.9732": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:192.168.0.1"}, "client.9792": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}, "client.9795": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}}}, "global_metrics": {"cephfs": {"client.9292": [[48, 2], [0, 0], [0, 0], [0, 22428528], [6, 0], [0, 3], [3, 3], [0, 3], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 2492059], [102755950540132, 9]], "client.9732": [[415, 22], [0, 0], [0, 409284244], [0, 57582621], [34, 0], [0, 35], [35, 35], [0, 35], [0, 0], [32, 33554432], [0, 0], [0, 0], [0, 12790136], [325622615982392, 32], [0, 899725], [76566333209702, 64]], "client.9792": [[2, 0], [0, 0], [0, 0], [0, 579907], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 579907], [0, 1]], "client.9795": [[2, 0], [0, 0], [0, 0], [0, 802945], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 802945], [0, 1]]}}, "metrics": {"delayed_ranks": [], "mds.0": {"client.9292": [], "client.9732": [], "client.9792": [], "client.9795": []}, "mds.1": {"client.9292": [], "client.9732": []}}} 2024-04-25T14:12:12.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.824+0000 7fb34b200640 1 -- 172.21.15.86:0/825864578 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fb32806f570 msgr2=0x7fb328071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:12.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.824+0000 7fb34b200640 1 --2- 172.21.15.86:0/825864578 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fb32806f570 0x7fb328071a80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb34c15c6a0 tx=0x7fb334007970 comp rx=0 tx=0).stop 2024-04-25T14:12:12.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.824+0000 7fb34b200640 1 -- 172.21.15.86:0/825864578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb34c06f990 msgr2=0x7fb34c0c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:12.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.824+0000 7fb34b200640 1 --2- 172.21.15.86:0/825864578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb34c06f990 0x7fb34c0c9620 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fb33c0ae040 tx=0x7fb33c019190 comp rx=0 tx=0).stop 2024-04-25T14:12:12.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.824+0000 7fb34b200640 1 -- 172.21.15.86:0/825864578 shutdown_connections 2024-04-25T14:12:12.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.824+0000 7fb34b200640 1 --2- 172.21.15.86:0/825864578 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fb32806f570 0x7fb328071a80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:12.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.824+0000 7fb34b200640 1 --2- 172.21.15.86:0/825864578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb34c1615c0 0x7fb34c15ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:12.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.824+0000 7fb34b200640 1 --2- 172.21.15.86:0/825864578 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb34c0b6530 0x7fb34c1f9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:12.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.824+0000 7fb34b200640 1 --2- 172.21.15.86:0/825864578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb34c06f990 0x7fb34c0c9620 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:12.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.824+0000 7fb34b200640 1 -- 172.21.15.86:0/825864578 >> 172.21.15.86:0/825864578 conn(0x7fb34c0cfea0 msgr2=0x7fb34c0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:12.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.824+0000 7fb34b200640 1 -- 172.21.15.86:0/825864578 shutdown_connections 2024-04-25T14:12:12.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.824+0000 7fb34b200640 1 -- 172.21.15.86:0/825864578 wait complete. 2024-04-25T14:12:12.841 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:12:12.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.992+0000 7f5ad8e00640 1 Processor -- start 2024-04-25T14:12:13.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.992+0000 7f5ad8e00640 1 -- start start 2024-04-25T14:12:13.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.992+0000 7f5ad8e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ad4071530 0x7f5ad415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:13.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.992+0000 7f5ad8e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5ad41616e0 0x7f5ad4161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:13.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.992+0000 7f5ad8e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ad40cd8d0 con 0x7f5ad40d2c80 2024-04-25T14:12:13.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.992+0000 7f5ad8e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f5ad40ce720 con 0x7f5ad41616e0 2024-04-25T14:12:13.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.992+0000 7f5ad8e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5ad40ce970 con 0x7f5ad4071530 2024-04-25T14:12:13.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.992+0000 7f5ac9400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ad4071530 0x7f5ad415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:13.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.992+0000 7f5ad2800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5ad41616e0 0x7f5ad4161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:13.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.992+0000 7f5ac9400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ad4071530 0x7f5ad415dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:42680/0 (socket says 172.21.15.86:42680) 2024-04-25T14:12:13.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.992+0000 7f5ad2800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5ad41616e0 0x7f5ad4161af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:35434/0 (socket says 172.21.15.86:35434) 2024-04-25T14:12:13.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.992+0000 7f5ac9400640 1 -- 172.21.15.86:0/454515257 learned_addr learned my addr 172.21.15.86:0/454515257 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:12:13.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.992+0000 7f5ad1400640 1 -- 172.21.15.86:0/454515257 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3843083779 0 0) 0x7f5ad40cd8d0 con 0x7f5ad40d2c80 2024-04-25T14:12:13.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.996+0000 7f5ad1400640 1 -- 172.21.15.86:0/454515257 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ab8003530 con 0x7f5ad40d2c80 2024-04-25T14:12:13.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.996+0000 7f5ad2800640 1 -- 172.21.15.86:0/454515257 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ad4071530 msgr2=0x7f5ad415dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:12:13.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.996+0000 7f5ad2800640 1 --2- 172.21.15.86:0/454515257 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ad4071530 0x7f5ad415dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:13.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.996+0000 7f5ad2800640 1 -- 172.21.15.86:0/454515257 >> v1:172.21.15.155:6789/0 conn(0x7f5ad40d2c80 legacy=0x7f5ad4070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:13.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.996+0000 7f5ad2800640 1 -- 172.21.15.86:0/454515257 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad40d1be0 con 0x7f5ad41616e0 2024-04-25T14:12:13.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.996+0000 7f5ac9400640 1 --2- 172.21.15.86:0/454515257 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ad4071530 0x7f5ad415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:12:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.996+0000 7f5ad2800640 1 --2- 172.21.15.86:0/454515257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ad41616e0 0x7f5ad4161af0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5acc0097c0 tx=0x7f5acc01ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af8a22d7edc55bb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.996+0000 7f5ad1400640 1 -- 172.21.15.86:0/454515257 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc0b5070 con 0x7f5ad41616e0 2024-04-25T14:12:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.996+0000 7f5ad1400640 1 -- 172.21.15.86:0/454515257 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5acc0a8990 con 0x7f5ad41616e0 2024-04-25T14:12:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.996+0000 7f5ad1400640 1 -- 172.21.15.86:0/454515257 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc0a8d60 con 0x7f5ad41616e0 2024-04-25T14:12:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.996+0000 7f5ad8e00640 1 -- 172.21.15.86:0/454515257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ad41616e0 msgr2=0x7f5ad4161af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.996+0000 7f5ad8e00640 1 --2- 172.21.15.86:0/454515257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ad41616e0 0x7f5ad4161af0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5acc0097c0 tx=0x7f5acc01ec90 comp rx=0 tx=0).stop 2024-04-25T14:12:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.996+0000 7f5ad8e00640 1 -- 172.21.15.86:0/454515257 shutdown_connections 2024-04-25T14:12:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.996+0000 7f5ad8e00640 1 --2- 172.21.15.86:0/454515257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ad41616e0 0x7f5ad4161af0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.996+0000 7f5ad8e00640 1 --2- 172.21.15.86:0/454515257 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ad4071530 0x7f5ad415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:12.996+0000 7f5ad8e00640 1 -- 172.21.15.86:0/454515257 >> 172.21.15.86:0/454515257 conn(0x7f5ad40cffc0 msgr2=0x7f5ad40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.000+0000 7f5ad8e00640 1 -- 172.21.15.86:0/454515257 shutdown_connections 2024-04-25T14:12:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.000+0000 7f5ad8e00640 1 -- 172.21.15.86:0/454515257 wait complete. 2024-04-25T14:12:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.000+0000 7f5ad8e00640 1 Processor -- start 2024-04-25T14:12:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.000+0000 7f5ad8e00640 1 -- start start 2024-04-25T14:12:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.000+0000 7f5ad8e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ad40d2c80 0x7f5ad415e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.000+0000 7f5ad8e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ad41616e0 0x7f5ad415fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.000+0000 7f5ad8e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ad4160420 0x7f5ad4160830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.000+0000 7f5ad8e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad40d2480 con 0x7f5ad4160420 2024-04-25T14:12:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.000+0000 7f5ad8e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad40d21a0 con 0x7f5ad40d2c80 2024-04-25T14:12:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.000+0000 7f5ad8e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5ad40d29a0 con 0x7f5ad41616e0 2024-04-25T14:12:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.000+0000 7f5ad2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ad4160420 0x7f5ad4160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.004+0000 7f5ad2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ad4160420 0x7f5ad4160830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57652/0 (socket says 172.21.15.86:57652) 2024-04-25T14:12:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.004+0000 7f5ad1e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ad40d2c80 0x7f5ad415e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.004+0000 7f5ad2800640 1 -- 172.21.15.86:0/875304539 learned_addr learned my addr 172.21.15.86:0/875304539 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:12:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.004+0000 7f5ac9400640 1 --2- 172.21.15.86:0/875304539 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ad41616e0 0x7f5ad415fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.004+0000 7f5ad2800640 1 -- 172.21.15.86:0/875304539 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ad41616e0 msgr2=0x7f5ad415fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.004+0000 7f5ad2800640 1 --2- 172.21.15.86:0/875304539 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ad41616e0 0x7f5ad415fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.004+0000 7f5ad2800640 1 -- 172.21.15.86:0/875304539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ad40d2c80 msgr2=0x7f5ad415e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.004+0000 7f5ad2800640 1 --2- 172.21.15.86:0/875304539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ad40d2c80 0x7f5ad415e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.004+0000 7f5ad2800640 1 -- 172.21.15.86:0/875304539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad40b2420 con 0x7f5ad4160420 2024-04-25T14:12:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.004+0000 7f5ad2800640 1 --2- 172.21.15.86:0/875304539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ad4160420 0x7f5ad4160830 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f5acc0097c0 tx=0x7f5acc01ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.004+0000 7f5acaa00640 1 -- 172.21.15.86:0/875304539 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc0c1070 con 0x7f5ad4160420 2024-04-25T14:12:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.004+0000 7f5acaa00640 1 -- 172.21.15.86:0/875304539 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5acc0b0640 con 0x7f5ad4160420 2024-04-25T14:12:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.004+0000 7f5acaa00640 1 -- 172.21.15.86:0/875304539 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc0b5040 con 0x7f5ad4160420 2024-04-25T14:12:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.004+0000 7f5ad8e00640 1 -- 172.21.15.86:0/875304539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad406e330 con 0x7f5ad4160420 2024-04-25T14:12:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.004+0000 7f5ad8e00640 1 -- 172.21.15.86:0/875304539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad406e8f0 con 0x7f5ad4160420 2024-04-25T14:12:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.004+0000 7f5ad8e00640 1 -- 172.21.15.86:0/875304539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ad4076bf0 con 0x7f5ad4160420 2024-04-25T14:12:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.004+0000 7f5acaa00640 1 -- 172.21.15.86:0/875304539 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 72) ==== 93318+0+0 (secure 0 0 0) 0x7f5acc0c4070 con 0x7f5ad4160420 2024-04-25T14:12:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.008+0000 7f5acaa00640 1 --2- 172.21.15.86:0/875304539 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f5ab006f5a0 0x7f5ab0071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:12:13.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.008+0000 7f5acaa00640 1 -- 172.21.15.86:0/875304539 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 6445+0+0 (secure 0 0 0) 0x7f5acc12c470 con 0x7f5ad4160420 2024-04-25T14:12:13.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.008+0000 7f5ad1e00640 1 --2- 172.21.15.86:0/875304539 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f5ab006f5a0 0x7f5ab0071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:12:13.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.008+0000 7f5ad1e00640 1 --2- 172.21.15.86:0/875304539 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f5ab006f5a0 0x7f5ab0071ab0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4007fd0 tx=0x7f5ac4009040 comp rx=0 tx=0).ready entity=mgr.9313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:12:13.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.012+0000 7f5acaa00640 1 -- 172.21.15.86:0/875304539 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5acc0187f0 con 0x7f5ad4160420 2024-04-25T14:12:13.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.200+0000 7f5ad8e00640 1 -- 172.21.15.86:0/875304539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ad40d1470 con 0x7f5ad4160420 2024-04-25T14:12:13.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.200+0000 7f5acaa00640 1 -- 172.21.15.86:0/875304539 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+5208 (secure 0 0 0) 0x7f5acc0f8910 con 0x7f5ad4160420 2024-04-25T14:12:13.200 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:12:13.200 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":107,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/1828317917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1828317917},{"type":"v1","addr":"172.21.15.155:6834","nonce":1828317917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":9124,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1389714390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1389714390},{"type":"v1","addr":"172.21.15.86:6837","nonce":1389714390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:11:32.981538+0000","modified":"2024-04-25T14:12:08.255286+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":9546,"mds_1":9136},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9136":{"gid":9136,"name":"d","rank":1,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/91793677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":91793677},{"type":"v1","addr":"172.21.15.155:6835","nonce":91793677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9546":{"gid":9546,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/2571655765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2571655765},{"type":"v1","addr":"172.21.15.86:6835","nonce":2571655765}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9546,"qdb_cluster":[9546,9136]},"id":10}]} 2024-04-25T14:12:13.200 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 107 2024-04-25T14:12:13.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.204+0000 7f5ad8e00640 1 -- 172.21.15.86:0/875304539 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f5ab006f5a0 msgr2=0x7f5ab0071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:13.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.204+0000 7f5ad8e00640 1 --2- 172.21.15.86:0/875304539 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f5ab006f5a0 0x7f5ab0071ab0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4007fd0 tx=0x7f5ac4009040 comp rx=0 tx=0).stop 2024-04-25T14:12:13.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.204+0000 7f5ad8e00640 1 -- 172.21.15.86:0/875304539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ad4160420 msgr2=0x7f5ad4160830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:12:13.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.204+0000 7f5ad8e00640 1 --2- 172.21.15.86:0/875304539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ad4160420 0x7f5ad4160830 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f5acc0097c0 tx=0x7f5acc01ba30 comp rx=0 tx=0).stop 2024-04-25T14:12:13.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.204+0000 7f5ad8e00640 1 -- 172.21.15.86:0/875304539 shutdown_connections 2024-04-25T14:12:13.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.204+0000 7f5ad8e00640 1 --2- 172.21.15.86:0/875304539 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f5ab006f5a0 0x7f5ab0071ab0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:13.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.204+0000 7f5ad8e00640 1 --2- 172.21.15.86:0/875304539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ad4160420 0x7f5ad4160830 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:13.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.204+0000 7f5ad8e00640 1 --2- 172.21.15.86:0/875304539 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ad41616e0 0x7f5ad415fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:13.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.204+0000 7f5ad8e00640 1 --2- 172.21.15.86:0/875304539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ad40d2c80 0x7f5ad415e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:12:13.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.204+0000 7f5ad8e00640 1 -- 172.21.15.86:0/875304539 >> 172.21.15.86:0/875304539 conn(0x7f5ad40cffc0 msgr2=0x7f5ad40cbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:12:13.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.204+0000 7f5ad8e00640 1 -- 172.21.15.86:0/875304539 shutdown_connections 2024-04-25T14:12:13.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:12:13.204+0000 7f5ad8e00640 1 -- 172.21.15.86:0/875304539 wait complete. 2024-04-25T14:12:13.221 DEBUG:tasks.cephfs.test_mds_metrics:metrics={'version': 2, 'global_counters': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'counters': [], 'client_metadata': {'cephfs': {'client.9292': {'hostname': 'smithi155', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'kernel_version': '6.9.0-rc3-g806ec36b2112', 'IP': 'v1:192.168.0.1'}, 'client.9732': {'hostname': 'smithi086', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'kernel_version': '6.9.0-rc3-g806ec36b2112', 'IP': 'v1:192.168.0.1'}, 'client.9792': {'hostname': 'smithi086', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'IP': '172.21.15.86'}, 'client.9795': {'hostname': 'smithi086', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'IP': '172.21.15.86'}}}, 'global_metrics': {'cephfs': {'client.9292': [[48, 2], [0, 0], [0, 0], [0, 22428528], [6, 0], [0, 3], [3, 3], [0, 3], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 2492059], [102755950540132, 9]], 'client.9732': [[415, 22], [0, 0], [0, 409284244], [0, 57582621], [34, 0], [0, 35], [35, 35], [0, 35], [0, 0], [32, 33554432], [0, 0], [0, 0], [0, 12790136], [325622615982392, 32], [0, 899725], [76566333209702, 64]], 'client.9792': [[2, 0], [0, 0], [0, 0], [0, 579907], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 579907], [0, 1]], 'client.9795': [[2, 0], [0, 0], [0, 0], [0, 802945], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 802945], [0, 1]]}}, 'metrics': {'delayed_ranks': [], 'mds.0': {'client.9292': [], 'client.9732': [], 'client.9792': [], 'client.9795': []}, 'mds.1': {'client.9292': [], 'client.9732': []}}} 2024-04-25T14:12:13.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:13.221 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls) 2024-04-25T14:12:13.231 INFO:teuthology.orchestra.run.smithi086.stdout:test_mds_metrics_0 2024-04-25T14:12:13.231 INFO:teuthology.orchestra.run.smithi086.stdout:test_mds_metrics_1 2024-04-25T14:12:13.232 INFO:tasks.cephfs.test_mds_metrics:cleaning directory test_mds_metrics_0 2024-04-25T14:12:13.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:13.232 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -rf test_mds_metrics_0) 2024-04-25T14:12:13.334 INFO:tasks.cephfs.test_mds_metrics:cleaning directory test_mds_metrics_1 2024-04-25T14:12:13.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:12:13.335 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -rf test_mds_metrics_1) 2024-04-25T14:12:20.017 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:12:20.021 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:12:50.083 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:12:50.088 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:13:13.355 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable stats 2024-04-25T14:13:13.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9bb4600640 1 Processor -- start 2024-04-25T14:13:13.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9bb4600640 1 -- start start 2024-04-25T14:13:13.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9bb4600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bac1651d0 0x7f9bac1675f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:13.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9bb4600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bac0b0a60 con 0x7f9bac06fd50 2024-04-25T14:13:13.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9bb4600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bac0ae840 con 0x7f9bac15c2a0 2024-04-25T14:13:13.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9bb4600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9bac06ef90 con 0x7f9bac1651d0 2024-04-25T14:13:13.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9bb1200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9bac06fd50 0x7f9bac070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45466/0 (socket says 172.21.15.86:45466) 2024-04-25T14:13:13.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9bb1200640 1 -- 172.21.15.86:0/4046513334 learned_addr learned my addr 172.21.15.86:0/4046513334 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:13.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9bb2600640 1 --2- 172.21.15.86:0/4046513334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bac1651d0 0x7f9bac1675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:13.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9babe00640 1 -- 172.21.15.86:0/4046513334 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3391005261 0 0) 0x7f9bac0b0a60 con 0x7f9bac06fd50 2024-04-25T14:13:13.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9babe00640 1 -- 172.21.15.86:0/4046513334 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b94003530 con 0x7f9bac06fd50 2024-04-25T14:13:13.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9babe00640 1 -- 172.21.15.86:0/4046513334 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 792519264 0 0) 0x7f9b94003530 con 0x7f9bac06fd50 2024-04-25T14:13:13.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9babe00640 1 -- 172.21.15.86:0/4046513334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bac1651d0 msgr2=0x7f9bac1675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:13.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9babe00640 1 --2- 172.21.15.86:0/4046513334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bac1651d0 0x7f9bac1675f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:13.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9babe00640 1 -- 172.21.15.86:0/4046513334 >> v1:172.21.15.155:6789/0 conn(0x7f9bac15c2a0 legacy=0x7f9bac0ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:13.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9babe00640 1 -- 172.21.15.86:0/4046513334 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bac06f270 con 0x7f9bac06fd50 2024-04-25T14:13:13.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9babe00640 1 -- 172.21.15.86:0/4046513334 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f9b98001680 con 0x7f9bac06fd50 2024-04-25T14:13:13.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9babe00640 1 -- 172.21.15.86:0/4046513334 >> v1:172.21.15.86:6789/0 conn(0x7f9bac06fd50 legacy=0x7f9bac070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:13.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9babe00640 1 --2- 172.21.15.86:0/4046513334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b94003a60 0x7f9b94003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:13.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9babe00640 1 --2- 172.21.15.86:0/4046513334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b94004810 0x7f9b94004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:13.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9babe00640 1 --2- 172.21.15.86:0/4046513334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bac1651d0 0x7f9b9400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:13.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9babe00640 1 -- 172.21.15.86:0/4046513334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac06ef90 con 0x7f9b94004810 2024-04-25T14:13:13.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9babe00640 1 -- 172.21.15.86:0/4046513334 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9b94005510 con 0x7f9b94003a60 2024-04-25T14:13:13.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9babe00640 1 -- 172.21.15.86:0/4046513334 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9b9400bf10 con 0x7f9bac1651d0 2024-04-25T14:13:13.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9bb1200640 1 --2- 172.21.15.86:0/4046513334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b94004810 0x7f9b94004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:13.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9bb2600640 1 --2- 172.21.15.86:0/4046513334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bac1651d0 0x7f9b9400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:13.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9bb1c00640 1 --2- 172.21.15.86:0/4046513334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b94003a60 0x7f9b94003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:13.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9bb1200640 1 -- 172.21.15.86:0/4046513334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bac1651d0 msgr2=0x7f9b9400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:13.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9bb1200640 1 --2- 172.21.15.86:0/4046513334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bac1651d0 0x7f9b9400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:13.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9bb1200640 1 -- 172.21.15.86:0/4046513334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b94003a60 msgr2=0x7f9b94003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:13.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9bb1200640 1 --2- 172.21.15.86:0/4046513334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b94003a60 0x7f9b94003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:13.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9bb1200640 1 -- 172.21.15.86:0/4046513334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b94003530 con 0x7f9b94004810 2024-04-25T14:13:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.515+0000 7f9bb2600640 1 --2- 172.21.15.86:0/4046513334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bac1651d0 0x7f9b9400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.519+0000 7f9bb1200640 1 --2- 172.21.15.86:0/4046513334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b94004810 0x7f9b94004c20 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f9b9800c210 tx=0x7f9b9801cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fac48d35112b05f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.519+0000 7f9babe00640 1 -- 172.21.15.86:0/4046513334 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b98007d20 con 0x7f9b94004810 2024-04-25T14:13:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.519+0000 7f9babe00640 1 -- 172.21.15.86:0/4046513334 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b980a9070 con 0x7f9b94004810 2024-04-25T14:13:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.519+0000 7f9babe00640 1 -- 172.21.15.86:0/4046513334 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b98005d40 con 0x7f9b94004810 2024-04-25T14:13:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.519+0000 7f9bb4600640 1 -- 172.21.15.86:0/4046513334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b94004810 msgr2=0x7f9b94004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.519+0000 7f9bb4600640 1 --2- 172.21.15.86:0/4046513334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b94004810 0x7f9b94004c20 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f9b9800c210 tx=0x7f9b9801cc30 comp rx=0 tx=0).stop 2024-04-25T14:13:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.519+0000 7f9bb2600640 1 -- 172.21.15.86:0/4046513334 reap_dead start 2024-04-25T14:13:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.519+0000 7f9bb4600640 1 -- 172.21.15.86:0/4046513334 shutdown_connections 2024-04-25T14:13:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.519+0000 7f9bb4600640 1 -- 172.21.15.86:0/4046513334 >> 172.21.15.86:0/4046513334 conn(0x7f9bac0b4650 msgr2=0x7f9bac0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.519+0000 7f9bb4600640 1 -- 172.21.15.86:0/4046513334 shutdown_connections 2024-04-25T14:13:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.519+0000 7f9bb4600640 1 -- 172.21.15.86:0/4046513334 wait complete. 2024-04-25T14:13:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.523+0000 7f9bb4600640 1 Processor -- start 2024-04-25T14:13:13.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.523+0000 7f9bb4600640 1 -- start start 2024-04-25T14:13:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.523+0000 7f9bb4600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b94004810 0x7f9bac1f91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.523+0000 7f9bb4600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bac0b2940 0x7f9bac1554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.523+0000 7f9bb4600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bac155a00 0x7f9bac0b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.523+0000 7f9bb4600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac15c830 con 0x7f9bac0b2940 2024-04-25T14:13:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.523+0000 7f9bb4600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac15c6b0 con 0x7f9b94004810 2024-04-25T14:13:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.523+0000 7f9bb4600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9bac1675f0 con 0x7f9bac155a00 2024-04-25T14:13:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.523+0000 7f9bb1200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bac0b2940 0x7f9bac1554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.523+0000 7f9bb1200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bac0b2940 0x7f9bac1554c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46092/0 (socket says 172.21.15.86:46092) 2024-04-25T14:13:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.523+0000 7f9bb1200640 1 -- 172.21.15.86:0/3239231464 learned_addr learned my addr 172.21.15.86:0/3239231464 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.527+0000 7f9bb1c00640 1 --2- 172.21.15.86:0/3239231464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b94004810 0x7f9bac1f91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.527+0000 7f9bb2600640 1 --2- 172.21.15.86:0/3239231464 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bac155a00 0x7f9bac0b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.527+0000 7f9bb1200640 1 -- 172.21.15.86:0/3239231464 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bac155a00 msgr2=0x7f9bac0b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:13.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.527+0000 7f9bb1200640 1 --2- 172.21.15.86:0/3239231464 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bac155a00 0x7f9bac0b0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:13.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.527+0000 7f9bb1200640 1 -- 172.21.15.86:0/3239231464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b94004810 msgr2=0x7f9bac1f91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:13.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.527+0000 7f9bb1200640 1 --2- 172.21.15.86:0/3239231464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b94004810 0x7f9bac1f91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:13.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.527+0000 7f9bb1200640 1 -- 172.21.15.86:0/3239231464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bac155e10 con 0x7f9bac0b2940 2024-04-25T14:13:13.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.527+0000 7f9bb1c00640 1 --2- 172.21.15.86:0/3239231464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b94004810 0x7f9bac1f91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:13.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.527+0000 7f9bb1200640 1 --2- 172.21.15.86:0/3239231464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bac0b2940 0x7f9bac1554c0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f9b9800c210 tx=0x7f9b94005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:13.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.527+0000 7f9ba9600640 1 -- 172.21.15.86:0/3239231464 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b980a5890 con 0x7f9bac0b2940 2024-04-25T14:13:13.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.527+0000 7f9ba9600640 1 -- 172.21.15.86:0/3239231464 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b980a5a30 con 0x7f9bac0b2940 2024-04-25T14:13:13.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.527+0000 7f9bb4600640 1 -- 172.21.15.86:0/3239231464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bac0b3a60 con 0x7f9bac0b2940 2024-04-25T14:13:13.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.527+0000 7f9ba9600640 1 -- 172.21.15.86:0/3239231464 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b980a5de0 con 0x7f9bac0b2940 2024-04-25T14:13:13.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.527+0000 7f9bb4600640 1 -- 172.21.15.86:0/3239231464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bac154700 con 0x7f9bac0b2940 2024-04-25T14:13:13.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.527+0000 7f9bb4600640 1 -- 172.21.15.86:0/3239231464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bac069d00 con 0x7f9bac0b2940 2024-04-25T14:13:13.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.527+0000 7f9ba9600640 1 -- 172.21.15.86:0/3239231464 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 72) ==== 93318+0+0 (secure 0 0 0) 0x7f9b980047f0 con 0x7f9bac0b2940 2024-04-25T14:13:13.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.531+0000 7f9ba9600640 1 --2- 172.21.15.86:0/3239231464 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f9b8c06f570 0x7f9b8c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:13.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.531+0000 7f9ba9600640 1 -- 172.21.15.86:0/3239231464 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6532+0+0 (secure 0 0 0) 0x7f9b981291a0 con 0x7f9bac0b2940 2024-04-25T14:13:13.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.531+0000 7f9bb1c00640 1 --2- 172.21.15.86:0/3239231464 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f9b8c06f570 0x7f9b8c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:13.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.531+0000 7f9bb1c00640 1 --2- 172.21.15.86:0/3239231464 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f9b8c06f570 0x7f9b8c071a80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9bac0b4200 tx=0x7f9ba0009040 comp rx=0 tx=0).ready entity=mgr.9313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:13.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.535+0000 7f9ba9600640 1 -- 172.21.15.86:0/3239231464 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9b980bf060 con 0x7f9bac0b2940 2024-04-25T14:13:13.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.731+0000 7f9bb4600640 1 -- 172.21.15.86:0/3239231464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "stats"} v 0) -- 0x7f9bac076bf0 con 0x7f9bac0b2940 2024-04-25T14:13:13.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.903+0000 7f9ba9600640 1 -- 172.21.15.86:0/3239231464 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "stats"}]=0 v73) ==== 85+0+0 (secure 0 0 0) 0x7f9b98016970 con 0x7f9bac0b2940 2024-04-25T14:13:13.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.907+0000 7f9bb4600640 1 -- 172.21.15.86:0/3239231464 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f9b8c06f570 msgr2=0x7f9b8c071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:13.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.907+0000 7f9bb4600640 1 --2- 172.21.15.86:0/3239231464 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f9b8c06f570 0x7f9b8c071a80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9bac0b4200 tx=0x7f9ba0009040 comp rx=0 tx=0).stop 2024-04-25T14:13:13.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.907+0000 7f9bb4600640 1 -- 172.21.15.86:0/3239231464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bac0b2940 msgr2=0x7f9bac1554c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:13.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.907+0000 7f9bb4600640 1 --2- 172.21.15.86:0/3239231464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bac0b2940 0x7f9bac1554c0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f9b9800c210 tx=0x7f9b94005840 comp rx=0 tx=0).stop 2024-04-25T14:13:13.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.907+0000 7f9bb4600640 1 -- 172.21.15.86:0/3239231464 shutdown_connections 2024-04-25T14:13:13.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.907+0000 7f9bb4600640 1 --2- 172.21.15.86:0/3239231464 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f9b8c06f570 0x7f9b8c071a80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:13.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.907+0000 7f9bb4600640 1 --2- 172.21.15.86:0/3239231464 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9bac155a00 0x7f9bac0b0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:13.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.907+0000 7f9bb4600640 1 --2- 172.21.15.86:0/3239231464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bac0b2940 0x7f9bac1554c0 secure :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f9b9800c210 tx=0x7f9b94005840 comp rx=0 tx=0).stop 2024-04-25T14:13:13.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.907+0000 7f9bb4600640 1 --2- 172.21.15.86:0/3239231464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b94004810 0x7f9bac1f91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:13.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.907+0000 7f9bb4600640 1 -- 172.21.15.86:0/3239231464 >> 172.21.15.86:0/3239231464 conn(0x7f9bac0b4650 msgr2=0x7f9bac159070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:13.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.907+0000 7f9bb4600640 1 -- 172.21.15.86:0/3239231464 shutdown_connections 2024-04-25T14:13:13.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:13.907+0000 7f9bb4600640 1 -- 172.21.15.86:0/3239231464 wait complete. 2024-04-25T14:13:13.928 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T14:13:13.929 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:13:14.026 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:13:14.076 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T14:13:14.076 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:13:14.076 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:13:14.076 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:13:14.086 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:13:14.086 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:13:14.138 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:13:14.161 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:14.164+0000 7faa93941180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:13:14.166 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:14.167+0000 7f7e35034180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:13:14.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.315+0000 7fe96ce00640 1 Processor -- start 2024-04-25T14:13:14.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.315+0000 7fe96ce00640 1 -- start start 2024-04-25T14:13:14.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.319+0000 7fe96ce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe9680b79c0 0x7fe9680ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:14.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.319+0000 7fe96ce00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9680b3bd0 con 0x7fe9680d2ae0 2024-04-25T14:13:14.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.319+0000 7fe96ce00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9680b1cf0 con 0x7fe9680ccab0 2024-04-25T14:13:14.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.319+0000 7fe96ce00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe9680b2620 con 0x7fe9680b79c0 2024-04-25T14:13:14.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.319+0000 7fe965400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe9680d2ae0 0x7fe9680ce180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45480/0 (socket says 172.21.15.86:45480) 2024-04-25T14:13:14.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.319+0000 7fe965400640 1 -- 172.21.15.86:0/622379405 learned_addr learned my addr 172.21.15.86:0/622379405 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:14.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.319+0000 7fe966800640 1 --2- 172.21.15.86:0/622379405 >> v2:172.21.15.86:3300/0 conn(0x7fe9680b79c0 0x7fe9680ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:14.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.319+0000 7fe966800640 1 -- 172.21.15.86:0/622379405 >> v1:172.21.15.155:6789/0 conn(0x7fe9680ccab0 legacy=0x7fe9680d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:14.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.319+0000 7fe966800640 1 -- 172.21.15.86:0/622379405 >> v1:172.21.15.86:6789/0 conn(0x7fe9680d2ae0 legacy=0x7fe9680ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:14.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.319+0000 7fe964a00640 1 -- 172.21.15.86:0/622379405 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 244267887 0 0) 0x7fe9680b1cf0 con 0x7fe9680ccab0 2024-04-25T14:13:14.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.319+0000 7fe966800640 1 -- 172.21.15.86:0/622379405 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9680b1fd0 con 0x7fe9680b79c0 2024-04-25T14:13:14.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.323+0000 7fe966800640 1 --2- 172.21.15.86:0/622379405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9680b79c0 0x7fe9680ce890 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fe960009d90 tx=0x7fe96001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759bcbb4b8484234 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:14.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.323+0000 7fe964a00640 1 -- 172.21.15.86:0/622379405 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9600b2070 con 0x7fe9680b79c0 2024-04-25T14:13:14.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.323+0000 7fe964a00640 1 -- 172.21.15.86:0/622379405 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9600a5940 con 0x7fe9680b79c0 2024-04-25T14:13:14.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.323+0000 7fe964a00640 1 -- 172.21.15.86:0/622379405 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9600a5d10 con 0x7fe9680b79c0 2024-04-25T14:13:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.323+0000 7fe96ce00640 1 -- 172.21.15.86:0/622379405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9680b79c0 msgr2=0x7fe9680ce890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.323+0000 7fe96ce00640 1 --2- 172.21.15.86:0/622379405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9680b79c0 0x7fe9680ce890 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fe960009d90 tx=0x7fe96001c970 comp rx=0 tx=0).stop 2024-04-25T14:13:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.323+0000 7fe96ce00640 1 -- 172.21.15.86:0/622379405 shutdown_connections 2024-04-25T14:13:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.323+0000 7fe96ce00640 1 --2- 172.21.15.86:0/622379405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9680b79c0 0x7fe9680ce890 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.323+0000 7fe96ce00640 1 -- 172.21.15.86:0/622379405 >> 172.21.15.86:0/622379405 conn(0x7fe96809ba30 msgr2=0x7fe9680b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.323+0000 7fe96ce00640 1 -- 172.21.15.86:0/622379405 shutdown_connections 2024-04-25T14:13:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.323+0000 7fe96ce00640 1 -- 172.21.15.86:0/622379405 wait complete. 2024-04-25T14:13:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.327+0000 7fe96ce00640 1 Processor -- start 2024-04-25T14:13:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.327+0000 7fe96ce00640 1 -- start start 2024-04-25T14:13:14.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.327+0000 7fe96ce00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9680b79c0 0x7fe96815d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:14.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.327+0000 7fe96ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9680ccab0 0x7fe9680c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:14.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.327+0000 7fe96ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9680cbf50 0x7fe9680c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:14.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.327+0000 7fe96ce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe96816c530 con 0x7fe9680ccab0 2024-04-25T14:13:14.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.327+0000 7fe96ce00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe96816c3b0 con 0x7fe9680cbf50 2024-04-25T14:13:14.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.327+0000 7fe96ce00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe96816c6b0 con 0x7fe9680b79c0 2024-04-25T14:13:14.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.327+0000 7fe965400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9680ccab0 0x7fe9680c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:14.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.327+0000 7fe965400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9680ccab0 0x7fe9680c7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46138/0 (socket says 172.21.15.86:46138) 2024-04-25T14:13:14.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.327+0000 7fe965400640 1 -- 172.21.15.86:0/663456790 learned_addr learned my addr 172.21.15.86:0/663456790 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:14.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.327+0000 7fe966800640 1 --2- 172.21.15.86:0/663456790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9680cbf50 0x7fe9680c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:14.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.327+0000 7fe965e00640 1 --2- 172.21.15.86:0/663456790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9680b79c0 0x7fe96815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:14.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.327+0000 7fe965400640 1 -- 172.21.15.86:0/663456790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9680b79c0 msgr2=0x7fe96815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:14.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.327+0000 7fe965400640 1 --2- 172.21.15.86:0/663456790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9680b79c0 0x7fe96815d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:14.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.327+0000 7fe965400640 1 -- 172.21.15.86:0/663456790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9680cbf50 msgr2=0x7fe9680c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:14.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.327+0000 7fe965400640 1 --2- 172.21.15.86:0/663456790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9680cbf50 0x7fe9680c77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:14.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.327+0000 7fe965400640 1 -- 172.21.15.86:0/663456790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9680ca640 con 0x7fe9680ccab0 2024-04-25T14:13:14.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.327+0000 7fe965400640 1 --2- 172.21.15.86:0/663456790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9680ccab0 0x7fe9680c7280 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fe95400c3a0 tx=0x7fe95400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:14.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.327+0000 7fe95e000640 1 -- 172.21.15.86:0/663456790 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe954017070 con 0x7fe9680ccab0 2024-04-25T14:13:14.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.327+0000 7fe95e000640 1 -- 172.21.15.86:0/663456790 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe954005cf0 con 0x7fe9680ccab0 2024-04-25T14:13:14.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.327+0000 7fe95e000640 1 -- 172.21.15.86:0/663456790 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9540072f0 con 0x7fe9680ccab0 2024-04-25T14:13:14.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.327+0000 7fe96ce00640 1 -- 172.21.15.86:0/663456790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe96815f8d0 con 0x7fe9680ccab0 2024-04-25T14:13:14.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.331+0000 7fe96ce00640 1 -- 172.21.15.86:0/663456790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe96815fe90 con 0x7fe9680ccab0 2024-04-25T14:13:14.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.331+0000 7fe96ce00640 1 -- 172.21.15.86:0/663456790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe968169760 con 0x7fe9680ccab0 2024-04-25T14:13:14.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.331+0000 7fe95e000640 1 -- 172.21.15.86:0/663456790 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 73) ==== 93309+0+0 (secure 0 0 0) 0x7fe95401a070 con 0x7fe9680ccab0 2024-04-25T14:13:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.331+0000 7fe95e000640 1 --2- 172.21.15.86:0/663456790 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fe94806f520 0x7fe948071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.331+0000 7fe95e000640 1 -- 172.21.15.86:0/663456790 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6532+0+0 (secure 0 0 0) 0x7fe95408ea70 con 0x7fe9680ccab0 2024-04-25T14:13:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.331+0000 7fe965e00640 1 -- 172.21.15.86:0/663456790 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fe94806f520 msgr2=0x7fe948071a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/1173654665 2024-04-25T14:13:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.331+0000 7fe965e00640 1 --2- 172.21.15.86:0/663456790 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fe94806f520 0x7fe948071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:13:14.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.335+0000 7fe95e000640 1 -- 172.21.15.86:0/663456790 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe95405aeb0 con 0x7fe9680ccab0 2024-04-25T14:13:14.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.495+0000 7fe96ce00640 1 -- 172.21.15.86:0/663456790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9680ca390 con 0x7fe9680ccab0 2024-04-25T14:13:14.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.499+0000 7fe95e000640 1 -- 172.21.15.86:0/663456790 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) ==== 74+0+17534 (secure 0 0 0) 0x7fe95405f1b0 con 0x7fe9680ccab0 2024-04-25T14:13:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:13:14.497 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":143,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:12:52.973064+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:11:30.810321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T14:11:31.944888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.e","mappings":[{"from":3,"to":1}]},{"pgid":"22.32","mappings":[{"from":3,"to":0}]},{"pgid":"22.34","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/4083287014":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/746190999":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/1426015871":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/3962343281":"2024-04-26T14:11:51.781677+0000","172.21.15.86:6833/264890479":"2024-04-26T14:11:51.781677+0000","172.21.15.86:6832/264890479":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/2724786983":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/285240843":"2024-04-26T14:11:51.781677+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:13:14.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.499+0000 7fe96ce00640 1 -- 172.21.15.86:0/663456790 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fe94806f520 msgr2=0x7fe948071a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:13:14.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.499+0000 7fe96ce00640 1 --2- 172.21.15.86:0/663456790 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fe94806f520 0x7fe948071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:14.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.499+0000 7fe96ce00640 1 -- 172.21.15.86:0/663456790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9680ccab0 msgr2=0x7fe9680c7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:14.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.499+0000 7fe96ce00640 1 --2- 172.21.15.86:0/663456790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9680ccab0 0x7fe9680c7280 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fe95400c3a0 tx=0x7fe95400c860 comp rx=0 tx=0).stop 2024-04-25T14:13:14.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.499+0000 7fe96ce00640 1 -- 172.21.15.86:0/663456790 shutdown_connections 2024-04-25T14:13:14.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.499+0000 7fe96ce00640 1 --2- 172.21.15.86:0/663456790 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7fe94806f520 0x7fe948071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:14.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.499+0000 7fe96ce00640 1 --2- 172.21.15.86:0/663456790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9680cbf50 0x7fe9680c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:14.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.499+0000 7fe96ce00640 1 --2- 172.21.15.86:0/663456790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9680ccab0 0x7fe9680c7280 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:14.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.499+0000 7fe96ce00640 1 --2- 172.21.15.86:0/663456790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9680b79c0 0x7fe96815d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:14.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.499+0000 7fe96ce00640 1 -- 172.21.15.86:0/663456790 >> 172.21.15.86:0/663456790 conn(0x7fe96809ba30 msgr2=0x7fe9680d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:14.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.499+0000 7fe96ce00640 1 -- 172.21.15.86:0/663456790 shutdown_connections 2024-04-25T14:13:14.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.499+0000 7fe96ce00640 1 -- 172.21.15.86:0/663456790 wait complete. 2024-04-25T14:13:14.520 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:11:30.810321+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:11:31.944888+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:13:14.521 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:13:14.580 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:14.584+0000 7faa93941180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:13:14.601 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:14.603+0000 7f7e35034180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:13:14.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.675+0000 7f54d3200640 1 Processor -- start 2024-04-25T14:13:14.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.675+0000 7f54d3200640 1 -- start start 2024-04-25T14:13:14.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.679+0000 7f54d3200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f54cc0cf970 0x7f54cc070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:14.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.679+0000 7f54d3200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54cc071360 0x7f54cc06e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:14.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.679+0000 7f54d3200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54cc071770 con 0x7f54cc06e7a0 2024-04-25T14:13:14.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.679+0000 7f54d3200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f54cc0cedf0 con 0x7f54cc0cf970 2024-04-25T14:13:14.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.679+0000 7f54d3200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f54cc163830 con 0x7f54cc071360 2024-04-25T14:13:14.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.679+0000 7f54cbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f54cc0cf970 0x7f54cc070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:14.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.679+0000 7f54cbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f54cc0cf970 0x7f54cc070e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46140/0 (socket says 172.21.15.86:46140) 2024-04-25T14:13:14.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.679+0000 7f54cbe00640 1 -- 172.21.15.86:0/2802805586 learned_addr learned my addr 172.21.15.86:0/2802805586 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:14.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.679+0000 7f54d1200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f54cc06e7a0 0x7f54cc0722f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45484/0 (socket says 172.21.15.86:45484) 2024-04-25T14:13:14.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.679+0000 7f54cb400640 1 --2- 172.21.15.86:0/2802805586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54cc071360 0x7f54cc06e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:14.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.679+0000 7f54cbe00640 1 -- 172.21.15.86:0/2802805586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54cc071360 msgr2=0x7f54cc06e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:14.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.679+0000 7f54cbe00640 1 --2- 172.21.15.86:0/2802805586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54cc071360 0x7f54cc06e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:14.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.679+0000 7f54cbe00640 1 -- 172.21.15.86:0/2802805586 >> v1:172.21.15.86:6789/0 conn(0x7f54cc06e7a0 legacy=0x7f54cc0722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:14.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.679+0000 7f54cbe00640 1 -- 172.21.15.86:0/2802805586 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54cc06ef90 con 0x7f54cc0cf970 2024-04-25T14:13:14.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.683+0000 7f54cbe00640 1 --2- 172.21.15.86:0/2802805586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54cc0cf970 0x7f54cc070e20 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f54bc009870 tx=0x7f54bc01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f91b9e11242087 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:14.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.683+0000 7f54caa00640 1 -- 172.21.15.86:0/2802805586 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc0ab650 con 0x7f54cc0cf970 2024-04-25T14:13:14.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.683+0000 7f54caa00640 1 -- 172.21.15.86:0/2802805586 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54bc0b0070 con 0x7f54cc0cf970 2024-04-25T14:13:14.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.683+0000 7f54caa00640 1 -- 172.21.15.86:0/2802805586 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc0aba10 con 0x7f54cc0cf970 2024-04-25T14:13:14.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.683+0000 7f54d3200640 1 -- 172.21.15.86:0/2802805586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54cc0cf970 msgr2=0x7f54cc070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:14.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.683+0000 7f54d3200640 1 --2- 172.21.15.86:0/2802805586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54cc0cf970 0x7f54cc070e20 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f54bc009870 tx=0x7f54bc01c530 comp rx=0 tx=0).stop 2024-04-25T14:13:14.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.683+0000 7f54d3200640 1 -- 172.21.15.86:0/2802805586 shutdown_connections 2024-04-25T14:13:14.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.683+0000 7f54d3200640 1 --2- 172.21.15.86:0/2802805586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54cc071360 0x7f54cc06e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:14.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.683+0000 7f54d3200640 1 --2- 172.21.15.86:0/2802805586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54cc0cf970 0x7f54cc070e20 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:14.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.683+0000 7f54d3200640 1 -- 172.21.15.86:0/2802805586 >> 172.21.15.86:0/2802805586 conn(0x7f54cc0cffc0 msgr2=0x7f54cc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:14.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.683+0000 7f54d3200640 1 -- 172.21.15.86:0/2802805586 shutdown_connections 2024-04-25T14:13:14.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.683+0000 7f54d3200640 1 -- 172.21.15.86:0/2802805586 wait complete. 2024-04-25T14:13:14.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.687+0000 7f54d3200640 1 Processor -- start 2024-04-25T14:13:14.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.687+0000 7f54d3200640 1 -- start start 2024-04-25T14:13:14.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.687+0000 7f54d3200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54cc06e7a0 0x7f54cc0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:14.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.687+0000 7f54d3200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54cc071360 0x7f54cc0b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:14.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.687+0000 7f54d3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54cc0cf970 0x7f54cc15bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:14.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.687+0000 7f54d3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f54cc06ed30 con 0x7f54cc0cf970 2024-04-25T14:13:14.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.687+0000 7f54d3200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f54cc06ebb0 con 0x7f54cc071360 2024-04-25T14:13:14.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.687+0000 7f54d3200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f54cc16cbd0 con 0x7f54cc06e7a0 2024-04-25T14:13:14.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.691+0000 7f54d1200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54cc0cf970 0x7f54cc15bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:14.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.691+0000 7f54cbe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54cc06e7a0 0x7f54cc0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:14.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.691+0000 7f54d1200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54cc0cf970 0x7f54cc15bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46156/0 (socket says 172.21.15.86:46156) 2024-04-25T14:13:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.691+0000 7f54d1200640 1 -- 172.21.15.86:0/1272927260 learned_addr learned my addr 172.21.15.86:0/1272927260 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.691+0000 7f54cb400640 1 --2- 172.21.15.86:0/1272927260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54cc071360 0x7f54cc0b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.691+0000 7f54cbe00640 1 -- 172.21.15.86:0/1272927260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54cc071360 msgr2=0x7f54cc0b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.691+0000 7f54cbe00640 1 --2- 172.21.15.86:0/1272927260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54cc071360 0x7f54cc0b36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.691+0000 7f54cbe00640 1 -- 172.21.15.86:0/1272927260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54cc0cf970 msgr2=0x7f54cc15bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.691+0000 7f54cbe00640 1 --2- 172.21.15.86:0/1272927260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54cc0cf970 0x7f54cc15bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.691+0000 7f54cbe00640 1 -- 172.21.15.86:0/1272927260 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54cc15b050 con 0x7f54cc06e7a0 2024-04-25T14:13:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.691+0000 7f54cbe00640 1 --2- 172.21.15.86:0/1272927260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54cc06e7a0 0x7f54cc0cca70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f54cc0d1030 tx=0x7f54bc01ad70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.691+0000 7f54c3e00640 1 -- 172.21.15.86:0/1272927260 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc0be640 con 0x7f54cc06e7a0 2024-04-25T14:13:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.691+0000 7f54c3e00640 1 -- 172.21.15.86:0/1272927260 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54bc0be7e0 con 0x7f54cc06e7a0 2024-04-25T14:13:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.691+0000 7f54c3e00640 1 -- 172.21.15.86:0/1272927260 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc0bebd0 con 0x7f54cc06e7a0 2024-04-25T14:13:14.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.691+0000 7f54d3200640 1 -- 172.21.15.86:0/1272927260 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54cc15b330 con 0x7f54cc06e7a0 2024-04-25T14:13:14.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.691+0000 7f54d3200640 1 -- 172.21.15.86:0/1272927260 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f54cc149260 con 0x7f54cc06e7a0 2024-04-25T14:13:14.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.691+0000 7f54d3200640 1 -- 172.21.15.86:0/1272927260 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54cc0720e0 con 0x7f54cc06e7a0 2024-04-25T14:13:14.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.691+0000 7f54c3e00640 1 -- 172.21.15.86:0/1272927260 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 73) ==== 93309+0+0 (secure 0 0 0) 0x7f54bc0b43f0 con 0x7f54cc06e7a0 2024-04-25T14:13:14.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.695+0000 7f54c3e00640 1 --2- 172.21.15.86:0/1272927260 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f54ac06f520 0x7f54ac071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:14.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.695+0000 7f54cb400640 1 -- 172.21.15.86:0/1272927260 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f54ac06f520 msgr2=0x7f54ac071a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/1173654665 2024-04-25T14:13:14.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.695+0000 7f54c3e00640 1 -- 172.21.15.86:0/1272927260 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(143..143 src has 1..143) ==== 6532+0+0 (secure 0 0 0) 0x7f54bc12a240 con 0x7f54cc06e7a0 2024-04-25T14:13:14.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.695+0000 7f54cb400640 1 --2- 172.21.15.86:0/1272927260 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f54ac06f520 0x7f54ac071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:13:14.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.699+0000 7f54c3e00640 1 -- 172.21.15.86:0/1272927260 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f54bc0abe20 con 0x7f54cc06e7a0 2024-04-25T14:13:14.731 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:14.732+0000 7faa93941180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:13:14.761 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:14.763+0000 7f7e35034180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:13:14.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.859+0000 7f54d3200640 1 -- 172.21.15.86:0/1272927260 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f54cc0b94c0 con 0x7f54cc06e7a0 2024-04-25T14:13:14.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.859+0000 7f54c3e00640 1 -- 172.21.15.86:0/1272927260 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v143) ==== 93+0+10 (secure 0 0 0) 0x7f54bc016150 con 0x7f54cc06e7a0 2024-04-25T14:13:14.859 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:13:14.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.863+0000 7f54d3200640 1 -- 172.21.15.86:0/1272927260 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f54ac06f520 msgr2=0x7f54ac071a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:13:14.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.863+0000 7f54d3200640 1 --2- 172.21.15.86:0/1272927260 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f54ac06f520 0x7f54ac071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:14.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.863+0000 7f54d3200640 1 -- 172.21.15.86:0/1272927260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54cc06e7a0 msgr2=0x7f54cc0cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:14.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.863+0000 7f54d3200640 1 --2- 172.21.15.86:0/1272927260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54cc06e7a0 0x7f54cc0cca70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f54cc0d1030 tx=0x7f54bc01ad70 comp rx=0 tx=0).stop 2024-04-25T14:13:14.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.863+0000 7f54d3200640 1 -- 172.21.15.86:0/1272927260 shutdown_connections 2024-04-25T14:13:14.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.863+0000 7f54d3200640 1 --2- 172.21.15.86:0/1272927260 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f54ac06f520 0x7f54ac071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:14.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.863+0000 7f54d3200640 1 --2- 172.21.15.86:0/1272927260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54cc0cf970 0x7f54cc15bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:14.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.863+0000 7f54d3200640 1 --2- 172.21.15.86:0/1272927260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54cc071360 0x7f54cc0b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:14.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.863+0000 7f54d3200640 1 --2- 172.21.15.86:0/1272927260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54cc06e7a0 0x7f54cc0cca70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:14.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.863+0000 7f54d3200640 1 -- 172.21.15.86:0/1272927260 >> 172.21.15.86:0/1272927260 conn(0x7f54cc0cffc0 msgr2=0x7f54cc0722f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:14.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.863+0000 7f54d3200640 1 -- 172.21.15.86:0/1272927260 shutdown_connections 2024-04-25T14:13:14.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:14.863+0000 7f54d3200640 1 -- 172.21.15.86:0/1272927260 wait complete. 2024-04-25T14:13:14.881 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:13:15.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.035+0000 7f90a6000640 1 Processor -- start 2024-04-25T14:13:15.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.035+0000 7f90a6000640 1 -- start start 2024-04-25T14:13:15.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.039+0000 7f90a6000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90a0071530 0x7f90a015dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:15.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.039+0000 7f90a6000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f90a01616e0 0x7f90a0161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:15.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.039+0000 7f90a6000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90a00cd8d0 con 0x7f90a00d2c80 2024-04-25T14:13:15.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.039+0000 7f90a6000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f90a00ce720 con 0x7f90a01616e0 2024-04-25T14:13:15.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.039+0000 7f90a6000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f90a00ce970 con 0x7f90a0071530 2024-04-25T14:13:15.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.039+0000 7f909f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f90a00d2c80 0x7f90a0070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45492/0 (socket says 172.21.15.86:45492) 2024-04-25T14:13:15.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.039+0000 7f909f400640 1 -- 172.21.15.86:0/3828670937 learned_addr learned my addr 172.21.15.86:0/3828670937 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:15.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.039+0000 7f909fe00640 1 --2- 172.21.15.86:0/3828670937 >> v2:172.21.15.86:3300/0 conn(0x7f90a01616e0 0x7f90a0161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:15.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.039+0000 7f909ea00640 1 --2- 172.21.15.86:0/3828670937 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90a0071530 0x7f90a015dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:15.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.039+0000 7f909fe00640 1 -- 172.21.15.86:0/3828670937 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90a0071530 msgr2=0x7f90a015dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:15.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.039+0000 7f909fe00640 1 --2- 172.21.15.86:0/3828670937 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90a0071530 0x7f90a015dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.039+0000 7f909fe00640 1 -- 172.21.15.86:0/3828670937 >> v1:172.21.15.86:6789/0 conn(0x7f90a00d2c80 legacy=0x7f90a0070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:15.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.039+0000 7f909fe00640 1 -- 172.21.15.86:0/3828670937 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a00d1be0 con 0x7f90a01616e0 2024-04-25T14:13:15.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.043+0000 7f909fe00640 1 --2- 172.21.15.86:0/3828670937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90a01616e0 0x7f90a0161af0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f9098009d90 tx=0x7f909801f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec96472f49e4379 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:15.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.043+0000 7f909e000640 1 -- 172.21.15.86:0/3828670937 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90980b4070 con 0x7f90a01616e0 2024-04-25T14:13:15.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.043+0000 7f909e000640 1 -- 172.21.15.86:0/3828670937 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90980af600 con 0x7f90a01616e0 2024-04-25T14:13:15.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.043+0000 7f909e000640 1 -- 172.21.15.86:0/3828670937 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90980af9d0 con 0x7f90a01616e0 2024-04-25T14:13:15.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.043+0000 7f90a6000640 1 -- 172.21.15.86:0/3828670937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90a01616e0 msgr2=0x7f90a0161af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:15.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.043+0000 7f90a6000640 1 --2- 172.21.15.86:0/3828670937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90a01616e0 0x7f90a0161af0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f9098009d90 tx=0x7f909801f150 comp rx=0 tx=0).stop 2024-04-25T14:13:15.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.043+0000 7f90a6000640 1 -- 172.21.15.86:0/3828670937 shutdown_connections 2024-04-25T14:13:15.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.043+0000 7f90a6000640 1 --2- 172.21.15.86:0/3828670937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90a01616e0 0x7f90a0161af0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.043+0000 7f90a6000640 1 --2- 172.21.15.86:0/3828670937 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90a0071530 0x7f90a015dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.043+0000 7f90a6000640 1 -- 172.21.15.86:0/3828670937 >> 172.21.15.86:0/3828670937 conn(0x7f90a00cffc0 msgr2=0x7f90a00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:15.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.043+0000 7f90a6000640 1 -- 172.21.15.86:0/3828670937 shutdown_connections 2024-04-25T14:13:15.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.043+0000 7f90a6000640 1 -- 172.21.15.86:0/3828670937 wait complete. 2024-04-25T14:13:15.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.047+0000 7f90a6000640 1 Processor -- start 2024-04-25T14:13:15.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.047+0000 7f90a6000640 1 -- start start 2024-04-25T14:13:15.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.047+0000 7f90a6000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90a00d2c80 0x7f90a015ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:15.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.047+0000 7f90a6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90a01616e0 0x7f90a0150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:15.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.047+0000 7f90a6000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90a0154a50 0x7f90a0154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:15.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.047+0000 7f90a6000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f90a00d2480 con 0x7f90a01616e0 2024-04-25T14:13:15.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.047+0000 7f90a6000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f90a00d21a0 con 0x7f90a0154a50 2024-04-25T14:13:15.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.047+0000 7f90a6000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f90a00d29a0 con 0x7f90a00d2c80 2024-04-25T14:13:15.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.047+0000 7f909ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90a01616e0 0x7f90a0150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:15.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.047+0000 7f909ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90a01616e0 0x7f90a0150a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46186/0 (socket says 172.21.15.86:46186) 2024-04-25T14:13:15.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.047+0000 7f909ea00640 1 -- 172.21.15.86:0/1425561490 learned_addr learned my addr 172.21.15.86:0/1425561490 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:15.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.047+0000 7f909f400640 1 --2- 172.21.15.86:0/1425561490 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90a00d2c80 0x7f90a015ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:15.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.047+0000 7f909fe00640 1 --2- 172.21.15.86:0/1425561490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90a0154a50 0x7f90a0154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:15.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.047+0000 7f909ea00640 1 -- 172.21.15.86:0/1425561490 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90a00d2c80 msgr2=0x7f90a015ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:15.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.047+0000 7f909ea00640 1 --2- 172.21.15.86:0/1425561490 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90a00d2c80 0x7f90a015ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.047+0000 7f909ea00640 1 -- 172.21.15.86:0/1425561490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90a0154a50 msgr2=0x7f90a0154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:15.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.047+0000 7f909ea00640 1 --2- 172.21.15.86:0/1425561490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90a0154a50 0x7f90a0154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.047+0000 7f909ea00640 1 -- 172.21.15.86:0/1425561490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a015dd30 con 0x7f90a01616e0 2024-04-25T14:13:15.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.047+0000 7f909ea00640 1 --2- 172.21.15.86:0/1425561490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90a01616e0 0x7f90a0150a30 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f908c00e8a0 tx=0x7f908c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:15.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.051+0000 7f9097400640 1 -- 172.21.15.86:0/1425561490 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c017070 con 0x7f90a01616e0 2024-04-25T14:13:15.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.051+0000 7f9097400640 1 -- 172.21.15.86:0/1425561490 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f908c00c940 con 0x7f90a01616e0 2024-04-25T14:13:15.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.051+0000 7f9097400640 1 -- 172.21.15.86:0/1425561490 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c00ccf0 con 0x7f90a01616e0 2024-04-25T14:13:15.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.051+0000 7f90a6000640 1 -- 172.21.15.86:0/1425561490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a015dfa0 con 0x7f90a01616e0 2024-04-25T14:13:15.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.051+0000 7f90a6000640 1 -- 172.21.15.86:0/1425561490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a0155520 con 0x7f90a01616e0 2024-04-25T14:13:15.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.051+0000 7f90a6000640 1 -- 172.21.15.86:0/1425561490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90a00c9ca0 con 0x7f90a01616e0 2024-04-25T14:13:15.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.051+0000 7f9097400640 1 -- 172.21.15.86:0/1425561490 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 73) ==== 93309+0+0 (secure 0 0 0) 0x7f908c01a070 con 0x7f90a01616e0 2024-04-25T14:13:15.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.051+0000 7f9097400640 1 --2- 172.21.15.86:0/1425561490 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f908006f520 0x7f9080071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:15.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.051+0000 7f909f400640 1 -- 172.21.15.86:0/1425561490 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f908006f520 msgr2=0x7f9080071a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/1173654665 2024-04-25T14:13:15.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.051+0000 7f9097400640 1 -- 172.21.15.86:0/1425561490 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6532+0+0 (secure 0 0 0) 0x7f908c08fa30 con 0x7f90a01616e0 2024-04-25T14:13:15.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.051+0000 7f909f400640 1 --2- 172.21.15.86:0/1425561490 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f908006f520 0x7f9080071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:13:15.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.055+0000 7f9097400640 1 -- 172.21.15.86:0/1425561490 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f908c05be70 con 0x7f90a01616e0 2024-04-25T14:13:15.207 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:15.208+0000 7faa93941180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:13:15.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.215+0000 7f90a6000640 1 -- 172.21.15.86:0/1425561490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f90a0076bf0 con 0x7f90a01616e0 2024-04-25T14:13:15.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.215+0000 7f9097400640 1 -- 172.21.15.86:0/1425561490 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v143) ==== 104+0+11 (secure 0 0 0) 0x7f908c060170 con 0x7f90a01616e0 2024-04-25T14:13:15.214 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:13:15.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.219+0000 7f90a6000640 1 -- 172.21.15.86:0/1425561490 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f908006f520 msgr2=0x7f9080071a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:13:15.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.219+0000 7f90a6000640 1 --2- 172.21.15.86:0/1425561490 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f908006f520 0x7f9080071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.219+0000 7f90a6000640 1 -- 172.21.15.86:0/1425561490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90a01616e0 msgr2=0x7f90a0150a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:15.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.219+0000 7f90a6000640 1 --2- 172.21.15.86:0/1425561490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90a01616e0 0x7f90a0150a30 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f908c00e8a0 tx=0x7f908c00ed60 comp rx=0 tx=0).stop 2024-04-25T14:13:15.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.219+0000 7f90a6000640 1 -- 172.21.15.86:0/1425561490 shutdown_connections 2024-04-25T14:13:15.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.219+0000 7f90a6000640 1 --2- 172.21.15.86:0/1425561490 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f908006f520 0x7f9080071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.219+0000 7f90a6000640 1 --2- 172.21.15.86:0/1425561490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90a0154a50 0x7f90a0154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.219+0000 7f90a6000640 1 --2- 172.21.15.86:0/1425561490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90a01616e0 0x7f90a0150a30 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.219+0000 7f90a6000640 1 --2- 172.21.15.86:0/1425561490 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90a00d2c80 0x7f90a015ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.219+0000 7f90a6000640 1 -- 172.21.15.86:0/1425561490 >> 172.21.15.86:0/1425561490 conn(0x7f90a00cffc0 msgr2=0x7f90a0070e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:15.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.219+0000 7f90a6000640 1 -- 172.21.15.86:0/1425561490 shutdown_connections 2024-04-25T14:13:15.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.219+0000 7f90a6000640 1 -- 172.21.15.86:0/1425561490 wait complete. 2024-04-25T14:13:15.237 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:13:15.254 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:15.255+0000 7f7e35034180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:13:15.266 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:15.268+0000 7faa93941180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:13:15.315 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:15.315+0000 7f7e35034180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:13:15.322 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:15.324+0000 7faa93941180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:13:15.374 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:15.375+0000 7f7e35034180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:13:15.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.391+0000 7f2924200640 1 Processor -- start 2024-04-25T14:13:15.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.391+0000 7f2924200640 1 -- start start 2024-04-25T14:13:15.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.395+0000 7f2924200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f291c1610d0 0x7f291c0b7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:15.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.395+0000 7f2924200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f291c0d15e0 0x7f291c0d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:15.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.395+0000 7f2924200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f291c0b3750 con 0x7f291c0b2250 2024-04-25T14:13:15.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.395+0000 7f2924200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f291c0b7f70 con 0x7f291c0d15e0 2024-04-25T14:13:15.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.395+0000 7f2924200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f291c161620 con 0x7f291c1610d0 2024-04-25T14:13:15.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.395+0000 7f2922200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f291c0d15e0 0x7f291c0d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:15.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.395+0000 7f2921800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f291c1610d0 0x7f291c0b7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:15.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.395+0000 7f2920e00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f291c0b2250 0x7f291c0b2660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:36432/0 (socket says 172.21.15.86:36432) 2024-04-25T14:13:15.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.395+0000 7f2920e00640 1 -- 172.21.15.86:0/3922135228 learned_addr learned my addr 172.21.15.86:0/3922135228 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:15.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.395+0000 7f2922200640 1 -- 172.21.15.86:0/3922135228 >> v2:172.21.15.155:3300/0 conn(0x7f291c1610d0 msgr2=0x7f291c0b7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:15.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.395+0000 7f2922200640 1 --2- 172.21.15.86:0/3922135228 >> v2:172.21.15.155:3300/0 conn(0x7f291c1610d0 0x7f291c0b7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.395+0000 7f2922200640 1 -- 172.21.15.86:0/3922135228 >> v1:172.21.15.155:6789/0 conn(0x7f291c0b2250 legacy=0x7f291c0b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:15.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.395+0000 7f291be00640 1 -- 172.21.15.86:0/3922135228 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3777108811 0 0) 0x7f291c0b3750 con 0x7f291c0b2250 2024-04-25T14:13:15.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.395+0000 7f2922200640 1 -- 172.21.15.86:0/3922135228 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291c0ce7b0 con 0x7f291c0d15e0 2024-04-25T14:13:15.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.395+0000 7f2921800640 1 --2- 172.21.15.86:0/3922135228 >> v2:172.21.15.155:3300/0 conn(0x7f291c1610d0 0x7f291c0b7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:13:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.399+0000 7f2922200640 1 --2- 172.21.15.86:0/3922135228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f291c0d15e0 0x7f291c0d19f0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f291c0735f0 tx=0x7f291401c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db4ebfe0b1087ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.399+0000 7f291be00640 1 -- 172.21.15.86:0/3922135228 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29140b2070 con 0x7f291c0d15e0 2024-04-25T14:13:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.399+0000 7f291be00640 1 -- 172.21.15.86:0/3922135228 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29140a5730 con 0x7f291c0d15e0 2024-04-25T14:13:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.399+0000 7f291be00640 1 -- 172.21.15.86:0/3922135228 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29140a5b00 con 0x7f291c0d15e0 2024-04-25T14:13:15.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.403+0000 7f2924200640 1 -- 172.21.15.86:0/3922135228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f291c0d15e0 msgr2=0x7f291c0d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:15.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.403+0000 7f2924200640 1 --2- 172.21.15.86:0/3922135228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f291c0d15e0 0x7f291c0d19f0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f291c0735f0 tx=0x7f291401c870 comp rx=0 tx=0).stop 2024-04-25T14:13:15.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.403+0000 7f2924200640 1 -- 172.21.15.86:0/3922135228 shutdown_connections 2024-04-25T14:13:15.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.403+0000 7f2924200640 1 --2- 172.21.15.86:0/3922135228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f291c0d15e0 0x7f291c0d19f0 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.403+0000 7f2924200640 1 --2- 172.21.15.86:0/3922135228 >> v2:172.21.15.155:3300/0 conn(0x7f291c1610d0 0x7f291c0b7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.403+0000 7f2924200640 1 -- 172.21.15.86:0/3922135228 >> 172.21.15.86:0/3922135228 conn(0x7f291c09ba30 msgr2=0x7f291c0b7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:15.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.403+0000 7f2924200640 1 -- 172.21.15.86:0/3922135228 shutdown_connections 2024-04-25T14:13:15.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.403+0000 7f2924200640 1 -- 172.21.15.86:0/3922135228 wait complete. 2024-04-25T14:13:15.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.403+0000 7f2924200640 1 Processor -- start 2024-04-25T14:13:15.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.403+0000 7f2924200640 1 -- start start 2024-04-25T14:13:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.407+0000 7f2924200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291c0b2250 0x7f291c0c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.407+0000 7f2924200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f291c0d15e0 0x7f291c1501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.407+0000 7f2924200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f291c1610d0 0x7f291c070c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.407+0000 7f2924200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f291c0b2ef0 con 0x7f291c1610d0 2024-04-25T14:13:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.407+0000 7f2924200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f291c0b2d70 con 0x7f291c0b2250 2024-04-25T14:13:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.407+0000 7f2924200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f291c0cf610 con 0x7f291c0d15e0 2024-04-25T14:13:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.407+0000 7f2922200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f291c1610d0 0x7f291c070c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.407+0000 7f2921800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291c0b2250 0x7f291c0c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:15.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.407+0000 7f2922200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f291c1610d0 0x7f291c070c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46194/0 (socket says 172.21.15.86:46194) 2024-04-25T14:13:15.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.407+0000 7f2922200640 1 -- 172.21.15.86:0/3389086201 learned_addr learned my addr 172.21.15.86:0/3389086201 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:15.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.407+0000 7f2920e00640 1 --2- 172.21.15.86:0/3389086201 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f291c0d15e0 0x7f291c1501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:15.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.407+0000 7f2922200640 1 -- 172.21.15.86:0/3389086201 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f291c0d15e0 msgr2=0x7f291c1501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:15.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.407+0000 7f2922200640 1 --2- 172.21.15.86:0/3389086201 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f291c0d15e0 0x7f291c1501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.407+0000 7f2922200640 1 -- 172.21.15.86:0/3389086201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291c0b2250 msgr2=0x7f291c0c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:15.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.407+0000 7f2922200640 1 --2- 172.21.15.86:0/3389086201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291c0b2250 0x7f291c0c6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.407+0000 7f2922200640 1 -- 172.21.15.86:0/3389086201 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291c0c7800 con 0x7f291c1610d0 2024-04-25T14:13:15.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.407+0000 7f2921800640 1 --2- 172.21.15.86:0/3389086201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291c0b2250 0x7f291c0c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:13:15.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.407+0000 7f2922200640 1 --2- 172.21.15.86:0/3389086201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f291c1610d0 0x7f291c070c30 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f29140b0040 tx=0x7f2914019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:15.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.407+0000 7f2919600640 1 -- 172.21.15.86:0/3389086201 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29140b2050 con 0x7f291c1610d0 2024-04-25T14:13:15.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.407+0000 7f2919600640 1 -- 172.21.15.86:0/3389086201 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29140b3040 con 0x7f291c1610d0 2024-04-25T14:13:15.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.407+0000 7f2919600640 1 -- 172.21.15.86:0/3389086201 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29140ad650 con 0x7f291c1610d0 2024-04-25T14:13:15.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.407+0000 7f2924200640 1 -- 172.21.15.86:0/3389086201 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f291c0c7ae0 con 0x7f291c1610d0 2024-04-25T14:13:15.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.407+0000 7f2924200640 1 -- 172.21.15.86:0/3389086201 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f291c0c80a0 con 0x7f291c1610d0 2024-04-25T14:13:15.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.407+0000 7f2924200640 1 -- 172.21.15.86:0/3389086201 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f291c0c7570 con 0x7f291c1610d0 2024-04-25T14:13:15.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.411+0000 7f2919600640 1 -- 172.21.15.86:0/3389086201 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 73) ==== 93309+0+0 (secure 0 0 0) 0x7f29140c0020 con 0x7f291c1610d0 2024-04-25T14:13:15.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.411+0000 7f2919600640 1 --2- 172.21.15.86:0/3389086201 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f28fc06f520 0x7f28fc071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:15.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.411+0000 7f2921800640 1 -- 172.21.15.86:0/3389086201 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f28fc06f520 msgr2=0x7f28fc071a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/1173654665 2024-04-25T14:13:15.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.411+0000 7f2919600640 1 -- 172.21.15.86:0/3389086201 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6532+0+0 (secure 0 0 0) 0x7f2914129e30 con 0x7f291c1610d0 2024-04-25T14:13:15.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.411+0000 7f2921800640 1 --2- 172.21.15.86:0/3389086201 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f28fc06f520 0x7f28fc071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:13:15.411 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:15.412+0000 7faa93941180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:13:15.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.415+0000 7f2919600640 1 -- 172.21.15.86:0/3389086201 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2914016520 con 0x7f291c1610d0 2024-04-25T14:13:15.468 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:15.472+0000 7faa93941180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:13:15.469 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:15.471+0000 7f7e35034180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:13:15.526 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:15.527+0000 7f7e35034180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:13:15.552 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:15.556+0000 7faa93941180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:13:15.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.575+0000 7f2924200640 1 -- 172.21.15.86:0/3389086201 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f291c0af220 con 0x7f291c1610d0 2024-04-25T14:13:15.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.575+0000 7f2919600640 1 -- 172.21.15.86:0/3389086201 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v143) ==== 100+0+11 (secure 0 0 0) 0x7f29140f6280 con 0x7f291c1610d0 2024-04-25T14:13:15.575 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:13:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.579+0000 7f2924200640 1 -- 172.21.15.86:0/3389086201 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f28fc06f520 msgr2=0x7f28fc071a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:13:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.579+0000 7f2924200640 1 --2- 172.21.15.86:0/3389086201 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f28fc06f520 0x7f28fc071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.579+0000 7f2924200640 1 -- 172.21.15.86:0/3389086201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f291c1610d0 msgr2=0x7f291c070c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.579+0000 7f2924200640 1 --2- 172.21.15.86:0/3389086201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f291c1610d0 0x7f291c070c30 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f29140b0040 tx=0x7f2914019610 comp rx=0 tx=0).stop 2024-04-25T14:13:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.579+0000 7f2924200640 1 -- 172.21.15.86:0/3389086201 shutdown_connections 2024-04-25T14:13:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.579+0000 7f2924200640 1 --2- 172.21.15.86:0/3389086201 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f28fc06f520 0x7f28fc071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.579+0000 7f2924200640 1 --2- 172.21.15.86:0/3389086201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f291c1610d0 0x7f291c070c30 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.579+0000 7f2924200640 1 --2- 172.21.15.86:0/3389086201 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f291c0d15e0 0x7f291c1501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.579+0000 7f2924200640 1 --2- 172.21.15.86:0/3389086201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291c0b2250 0x7f291c0c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.579+0000 7f2924200640 1 -- 172.21.15.86:0/3389086201 >> 172.21.15.86:0/3389086201 conn(0x7f291c09ba30 msgr2=0x7f291c06cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.579+0000 7f2924200640 1 -- 172.21.15.86:0/3389086201 shutdown_connections 2024-04-25T14:13:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.579+0000 7f2924200640 1 -- 172.21.15.86:0/3389086201 wait complete. 2024-04-25T14:13:15.598 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T14:13:15.611 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:15.612+0000 7faa93941180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:13:15.613 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:15.615+0000 7f7e35034180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:13:15.675 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:15.675+0000 7f7e35034180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:13:15.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.763+0000 7f30d3600640 1 Processor -- start 2024-04-25T14:13:15.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.763+0000 7f30d3600640 1 -- start start 2024-04-25T14:13:15.764 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:15.768+0000 7faa93941180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:13:15.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.767+0000 7f30d3600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30cc0d2420 0x7f30cc0d2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:15.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.767+0000 7f30d3600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f30cc0b6490 0x7f30cc0b68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:15.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.767+0000 7f30d3600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30cc162480 con 0x7f30cc070e20 2024-04-25T14:13:15.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.767+0000 7f30d3600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f30cc0d16f0 con 0x7f30cc0b6490 2024-04-25T14:13:15.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.767+0000 7f30d3600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f30cc0d1940 con 0x7f30cc0d2420 2024-04-25T14:13:15.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.767+0000 7f30d1600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f30cc0b6490 0x7f30cc0b68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:15.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.767+0000 7f30d1600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f30cc0b6490 0x7f30cc0b68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46200/0 (socket says 172.21.15.86:46200) 2024-04-25T14:13:15.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.767+0000 7f30d1600640 1 -- 172.21.15.86:0/2549042132 learned_addr learned my addr 172.21.15.86:0/2549042132 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:15.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.767+0000 7f30d0c00640 1 --2- 172.21.15.86:0/2549042132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30cc0d2420 0x7f30cc0d2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:15.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.767+0000 7f30d1600640 1 -- 172.21.15.86:0/2549042132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30cc0d2420 msgr2=0x7f30cc0d2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:15.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.767+0000 7f30d1600640 1 --2- 172.21.15.86:0/2549042132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30cc0d2420 0x7f30cc0d2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.767+0000 7f30d1600640 1 -- 172.21.15.86:0/2549042132 >> v1:172.21.15.155:6789/0 conn(0x7f30cc070e20 legacy=0x7f30cc071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:15.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.767+0000 7f30d1600640 1 -- 172.21.15.86:0/2549042132 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30cc0d1c20 con 0x7f30cc0b6490 2024-04-25T14:13:15.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.767+0000 7f30d0c00640 1 --2- 172.21.15.86:0/2549042132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30cc0d2420 0x7f30cc0d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:15.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.771+0000 7f30d1600640 1 --2- 172.21.15.86:0/2549042132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc0b6490 0x7f30cc0b68a0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009e50 tx=0x7f30c401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318d693f2fbc5f56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:15.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.771+0000 7f30cbe00640 1 -- 172.21.15.86:0/2549042132 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c40a57b0 con 0x7f30cc0b6490 2024-04-25T14:13:15.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.771+0000 7f30cbe00640 1 -- 172.21.15.86:0/2549042132 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30c40a5950 con 0x7f30cc0b6490 2024-04-25T14:13:15.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.771+0000 7f30cbe00640 1 -- 172.21.15.86:0/2549042132 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c40a5d20 con 0x7f30cc0b6490 2024-04-25T14:13:15.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.771+0000 7f30d3600640 1 -- 172.21.15.86:0/2549042132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc0b6490 msgr2=0x7f30cc0b68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:15.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.771+0000 7f30d3600640 1 --2- 172.21.15.86:0/2549042132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc0b6490 0x7f30cc0b68a0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009e50 tx=0x7f30c401ca30 comp rx=0 tx=0).stop 2024-04-25T14:13:15.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.771+0000 7f30d3600640 1 -- 172.21.15.86:0/2549042132 shutdown_connections 2024-04-25T14:13:15.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.771+0000 7f30d3600640 1 --2- 172.21.15.86:0/2549042132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc0b6490 0x7f30cc0b68a0 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.771+0000 7f30d3600640 1 --2- 172.21.15.86:0/2549042132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30cc0d2420 0x7f30cc0d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.771+0000 7f30d3600640 1 -- 172.21.15.86:0/2549042132 >> 172.21.15.86:0/2549042132 conn(0x7f30cc0cffc0 msgr2=0x7f30cc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:15.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.771+0000 7f30d3600640 1 -- 172.21.15.86:0/2549042132 shutdown_connections 2024-04-25T14:13:15.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.771+0000 7f30d3600640 1 -- 172.21.15.86:0/2549042132 wait complete. 2024-04-25T14:13:15.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.775+0000 7f30d3600640 1 Processor -- start 2024-04-25T14:13:15.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.775+0000 7f30d3600640 1 -- start start 2024-04-25T14:13:15.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.775+0000 7f30d3600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30cc070e20 0x7f30cc14f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:15.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.775+0000 7f30d3600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30cc0b6490 0x7f30cc06ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:15.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.775+0000 7f30d3600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc0d2420 0x7f30cc06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:15.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.775+0000 7f30d3600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc161310 con 0x7f30cc0d2420 2024-04-25T14:13:15.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.775+0000 7f30d3600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc0cdf30 con 0x7f30cc070e20 2024-04-25T14:13:15.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.775+0000 7f30d3600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f30cc158ee0 con 0x7f30cc0b6490 2024-04-25T14:13:15.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.775+0000 7f30d1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc0d2420 0x7f30cc06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:15.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.775+0000 7f30d1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc0d2420 0x7f30cc06efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46216/0 (socket says 172.21.15.86:46216) 2024-04-25T14:13:15.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.775+0000 7f30d1600640 1 -- 172.21.15.86:0/320699123 learned_addr learned my addr 172.21.15.86:0/320699123 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:15.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.775+0000 7f30d0c00640 1 --2- 172.21.15.86:0/320699123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30cc070e20 0x7f30cc14f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:15.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.775+0000 7f30c3e00640 1 --2- 172.21.15.86:0/320699123 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30cc0b6490 0x7f30cc06ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:15.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.775+0000 7f30d1600640 1 -- 172.21.15.86:0/320699123 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30cc0b6490 msgr2=0x7f30cc06ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:15.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.775+0000 7f30d1600640 1 --2- 172.21.15.86:0/320699123 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30cc0b6490 0x7f30cc06ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.775+0000 7f30d1600640 1 -- 172.21.15.86:0/320699123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30cc070e20 msgr2=0x7f30cc14f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:15.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.775+0000 7f30d1600640 1 --2- 172.21.15.86:0/320699123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30cc070e20 0x7f30cc14f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.775+0000 7f30d1600640 1 -- 172.21.15.86:0/320699123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30cc06e500 con 0x7f30cc0d2420 2024-04-25T14:13:15.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.775+0000 7f30c3e00640 1 --2- 172.21.15.86:0/320699123 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30cc0b6490 0x7f30cc06ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:15.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.775+0000 7f30d1600640 1 --2- 172.21.15.86:0/320699123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc0d2420 0x7f30cc06efb0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f30c40af040 tx=0x7f30c40197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:15.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.775+0000 7f30c9600640 1 -- 172.21.15.86:0/320699123 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c40b8670 con 0x7f30cc0d2420 2024-04-25T14:13:15.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.775+0000 7f30c9600640 1 -- 172.21.15.86:0/320699123 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30c40be070 con 0x7f30cc0d2420 2024-04-25T14:13:15.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.775+0000 7f30c9600640 1 -- 172.21.15.86:0/320699123 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c40b8a30 con 0x7f30cc0d2420 2024-04-25T14:13:15.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.775+0000 7f30d3600640 1 -- 172.21.15.86:0/320699123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30cc1570c0 con 0x7f30cc0d2420 2024-04-25T14:13:15.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.775+0000 7f30d3600640 1 -- 172.21.15.86:0/320699123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30cc157680 con 0x7f30cc0d2420 2024-04-25T14:13:15.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.779+0000 7f30d3600640 1 -- 172.21.15.86:0/320699123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30cc06e270 con 0x7f30cc0d2420 2024-04-25T14:13:15.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.779+0000 7f30c9600640 1 -- 172.21.15.86:0/320699123 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 73) ==== 93309+0+0 (secure 0 0 0) 0x7f30c40b8bd0 con 0x7f30cc0d2420 2024-04-25T14:13:15.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.779+0000 7f30c9600640 1 --2- 172.21.15.86:0/320699123 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f30a806f550 0x7f30a8071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:15.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.779+0000 7f30d0c00640 1 -- 172.21.15.86:0/320699123 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f30a806f550 msgr2=0x7f30a8071a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/1173654665 2024-04-25T14:13:15.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.779+0000 7f30c9600640 1 -- 172.21.15.86:0/320699123 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6532+0+0 (secure 0 0 0) 0x7f30c412a700 con 0x7f30cc0d2420 2024-04-25T14:13:15.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.779+0000 7f30d0c00640 1 --2- 172.21.15.86:0/320699123 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f30a806f550 0x7f30a8071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:13:15.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.787+0000 7f30c9600640 1 -- 172.21.15.86:0/320699123 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f30c4016620 con 0x7f30cc0d2420 2024-04-25T14:13:15.825 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:15.828+0000 7faa93941180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:13:15.838 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:15.839+0000 7f7e35034180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:13:15.898 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:15.899+0000 7f7e35034180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:13:15.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.955+0000 7f30d3600640 1 -- 172.21.15.86:0/320699123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f30cc076bf0 con 0x7f30cc0d2420 2024-04-25T14:13:15.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.959+0000 7f30c9600640 1 -- 172.21.15.86:0/320699123 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v143) ==== 80+0+466 (secure 0 0 0) 0x7f30c40f7230 con 0x7f30cc0d2420 2024-04-25T14:13:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/4083287014 2024-04-26T14:11:51.781677+0000 2024-04-25T14:13:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/746190999 2024-04-26T14:11:51.781677+0000 2024-04-25T14:13:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1426015871 2024-04-26T14:11:51.781677+0000 2024-04-25T14:13:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3962343281 2024-04-26T14:11:51.781677+0000 2024-04-25T14:13:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6833/264890479 2024-04-26T14:11:51.781677+0000 2024-04-25T14:13:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6832/264890479 2024-04-26T14:11:51.781677+0000 2024-04-25T14:13:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2724786983 2024-04-26T14:11:51.781677+0000 2024-04-25T14:13:15.957 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/285240843 2024-04-26T14:11:51.781677+0000 2024-04-25T14:13:15.957 INFO:teuthology.orchestra.run.smithi086.stderr:listed 8 entries 2024-04-25T14:13:15.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.959+0000 7f30d3600640 1 -- 172.21.15.86:0/320699123 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f30a806f550 msgr2=0x7f30a8071a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:13:15.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.959+0000 7f30d3600640 1 --2- 172.21.15.86:0/320699123 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f30a806f550 0x7f30a8071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.959+0000 7f30d3600640 1 -- 172.21.15.86:0/320699123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc0d2420 msgr2=0x7f30cc06efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:15.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.959+0000 7f30d3600640 1 --2- 172.21.15.86:0/320699123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc0d2420 0x7f30cc06efb0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f30c40af040 tx=0x7f30c40197d0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.959+0000 7f30d3600640 1 -- 172.21.15.86:0/320699123 shutdown_connections 2024-04-25T14:13:15.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.959+0000 7f30d3600640 1 --2- 172.21.15.86:0/320699123 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f30a806f550 0x7f30a8071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.959+0000 7f30d3600640 1 --2- 172.21.15.86:0/320699123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc0d2420 0x7f30cc06efb0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.959+0000 7f30d3600640 1 --2- 172.21.15.86:0/320699123 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30cc0b6490 0x7f30cc06ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.959+0000 7f30d3600640 1 --2- 172.21.15.86:0/320699123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30cc070e20 0x7f30cc14f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:15.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.959+0000 7f30d3600640 1 -- 172.21.15.86:0/320699123 >> 172.21.15.86:0/320699123 conn(0x7f30cc0cffc0 msgr2=0x7f30cc0687a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:15.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.959+0000 7f30d3600640 1 -- 172.21.15.86:0/320699123 shutdown_connections 2024-04-25T14:13:15.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:15.959+0000 7f30d3600640 1 -- 172.21.15.86:0/320699123 wait complete. 2024-04-25T14:13:15.982 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T14:13:15.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:15.982 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T14:13:16.036 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T14:13:16.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:16.036 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T14:13:16.060 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:13:16.061 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:13:16.061 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:13:16.077 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:13:16.077 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:13:16.098 INFO:tasks.ceph.mgr.x.smithi155.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:13:16.099 INFO:tasks.ceph.mgr.x.smithi155.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:13:16.099 INFO:tasks.ceph.mgr.x.smithi155.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:13:16.107 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:13:16.108 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:16.108+0000 7faa93941180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:13:16.190 INFO:tasks.ceph.mgr.y.smithi086.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:13:16.190 INFO:tasks.ceph.mgr.y.smithi086.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:13:16.190 INFO:tasks.ceph.mgr.y.smithi086.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:13:16.197 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:16.199+0000 7f7e35034180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:13:16.256 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:16.260+0000 7faa93941180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:13:16.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.295+0000 7f0fc4400640 1 Processor -- start 2024-04-25T14:13:16.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.295+0000 7f0fc4400640 1 -- start start 2024-04-25T14:13:16.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.299+0000 7f0fc4400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0fbc071530 0x7f0fbc15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:16.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.299+0000 7f0fc4400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fbc0cd8d0 con 0x7f0fbc1616e0 2024-04-25T14:13:16.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.299+0000 7f0fc4400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fbc0ce970 con 0x7f0fbc0d2c80 2024-04-25T14:13:16.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.299+0000 7f0fc4400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0fbc0ce720 con 0x7f0fbc071530 2024-04-25T14:13:16.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.299+0000 7f0fc1000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0fbc071530 0x7f0fbc15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:16.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.299+0000 7f0fc2400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0fbc1616e0 0x7f0fbc161af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45502/0 (socket says 172.21.15.86:45502) 2024-04-25T14:13:16.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.299+0000 7f0fc2400640 1 -- 172.21.15.86:0/2444217979 learned_addr learned my addr 172.21.15.86:0/2444217979 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:16.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.299+0000 7f0fc1000640 1 -- 172.21.15.86:0/2444217979 >> v1:172.21.15.155:6789/0 conn(0x7f0fbc0d2c80 legacy=0x7f0fbc070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:16.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.299+0000 7f0fc1000640 1 -- 172.21.15.86:0/2444217979 >> v1:172.21.15.86:6789/0 conn(0x7f0fbc1616e0 legacy=0x7f0fbc161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:16.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.299+0000 7f0fc1000640 1 -- 172.21.15.86:0/2444217979 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fbc0d1be0 con 0x7f0fbc071530 2024-04-25T14:13:16.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.299+0000 7f0fbbe00640 1 -- 172.21.15.86:0/2444217979 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3899569023 0 0) 0x7f0fbc0cd8d0 con 0x7f0fbc1616e0 2024-04-25T14:13:16.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.303+0000 7f0fc1000640 1 --2- 172.21.15.86:0/2444217979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fbc071530 0x7f0fbc15dc90 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f0fa80097a0 tx=0x7f0fa801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740240fc2fe97c15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:16.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.303+0000 7f0fbbe00640 1 -- 172.21.15.86:0/2444217979 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa80b5070 con 0x7f0fbc071530 2024-04-25T14:13:16.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.303+0000 7f0fbbe00640 1 -- 172.21.15.86:0/2444217979 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fa80a8970 con 0x7f0fbc071530 2024-04-25T14:13:16.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.303+0000 7f0fbbe00640 1 -- 172.21.15.86:0/2444217979 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa80a8d40 con 0x7f0fbc071530 2024-04-25T14:13:16.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.303+0000 7f0fc4400640 1 -- 172.21.15.86:0/2444217979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fbc071530 msgr2=0x7f0fbc15dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:16.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.303+0000 7f0fc4400640 1 --2- 172.21.15.86:0/2444217979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fbc071530 0x7f0fbc15dc90 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f0fa80097a0 tx=0x7f0fa801ec70 comp rx=0 tx=0).stop 2024-04-25T14:13:16.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.303+0000 7f0fc4400640 1 -- 172.21.15.86:0/2444217979 shutdown_connections 2024-04-25T14:13:16.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.303+0000 7f0fc4400640 1 --2- 172.21.15.86:0/2444217979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fbc071530 0x7f0fbc15dc90 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:16.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.303+0000 7f0fc4400640 1 -- 172.21.15.86:0/2444217979 >> 172.21.15.86:0/2444217979 conn(0x7f0fbc0cffc0 msgr2=0x7f0fbc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:16.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.303+0000 7f0fc4400640 1 -- 172.21.15.86:0/2444217979 shutdown_connections 2024-04-25T14:13:16.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.303+0000 7f0fc4400640 1 -- 172.21.15.86:0/2444217979 wait complete. 2024-04-25T14:13:16.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.307+0000 7f0fc4400640 1 Processor -- start 2024-04-25T14:13:16.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.307+0000 7f0fc4400640 1 -- start start 2024-04-25T14:13:16.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.307+0000 7f0fc4400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fbc0d2c80 0x7f0fbc0af290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:16.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.307+0000 7f0fc4400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fbc1616e0 0x7f0fbc151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:16.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.307+0000 7f0fc4400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fbc0b4240 0x7f0fbc1f6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:16.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.307+0000 7f0fc4400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc0d2480 con 0x7f0fbc1616e0 2024-04-25T14:13:16.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.307+0000 7f0fc4400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc0d21a0 con 0x7f0fbc0d2c80 2024-04-25T14:13:16.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.307+0000 7f0fc4400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0fbc0d29a0 con 0x7f0fbc0b4240 2024-04-25T14:13:16.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.307+0000 7f0fc1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fbc1616e0 0x7f0fbc151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:16.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.307+0000 7f0fc1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fbc1616e0 0x7f0fbc151e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46228/0 (socket says 172.21.15.86:46228) 2024-04-25T14:13:16.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.307+0000 7f0fc1000640 1 -- 172.21.15.86:0/3204787312 learned_addr learned my addr 172.21.15.86:0/3204787312 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:16.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.307+0000 7f0fc1a00640 1 --2- 172.21.15.86:0/3204787312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fbc0d2c80 0x7f0fbc0af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:16.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.307+0000 7f0fc2400640 1 --2- 172.21.15.86:0/3204787312 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fbc0b4240 0x7f0fbc1f6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:16.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.307+0000 7f0fc1000640 1 -- 172.21.15.86:0/3204787312 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fbc0b4240 msgr2=0x7f0fbc1f6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:16.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.307+0000 7f0fc1000640 1 --2- 172.21.15.86:0/3204787312 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fbc0b4240 0x7f0fbc1f6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:16.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.307+0000 7f0fc1000640 1 -- 172.21.15.86:0/3204787312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fbc0d2c80 msgr2=0x7f0fbc0af290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:16.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.307+0000 7f0fc1000640 1 --2- 172.21.15.86:0/3204787312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fbc0d2c80 0x7f0fbc0af290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:16.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.307+0000 7f0fc1000640 1 -- 172.21.15.86:0/3204787312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fbc152390 con 0x7f0fbc1616e0 2024-04-25T14:13:16.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.307+0000 7f0fc1000640 1 --2- 172.21.15.86:0/3204787312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fbc1616e0 0x7f0fbc151e50 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f0fa80b3040 tx=0x7f0fa801ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:16.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.307+0000 7f0fb9600640 1 -- 172.21.15.86:0/3204787312 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa80c0070 con 0x7f0fbc1616e0 2024-04-25T14:13:16.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.307+0000 7f0fb9600640 1 -- 172.21.15.86:0/3204787312 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fa8005ba0 con 0x7f0fbc1616e0 2024-04-25T14:13:16.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.307+0000 7f0fb9600640 1 -- 172.21.15.86:0/3204787312 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa80b5040 con 0x7f0fbc1616e0 2024-04-25T14:13:16.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.307+0000 7f0fc4400640 1 -- 172.21.15.86:0/3204787312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fbc0ae9e0 con 0x7f0fbc1616e0 2024-04-25T14:13:16.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.307+0000 7f0fc4400640 1 -- 172.21.15.86:0/3204787312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fbc0b0f20 con 0x7f0fbc1616e0 2024-04-25T14:13:16.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.311+0000 7f0fc4400640 1 -- 172.21.15.86:0/3204787312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fbc0720e0 con 0x7f0fbc1616e0 2024-04-25T14:13:16.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.311+0000 7f0fb9600640 1 -- 172.21.15.86:0/3204787312 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 73) ==== 93309+0+0 (secure 0 0 0) 0x7f0fa8006030 con 0x7f0fbc1616e0 2024-04-25T14:13:16.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.311+0000 7f0fb9600640 1 --2- 172.21.15.86:0/3204787312 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f0f9c06f520 0x7f0f9c071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:16.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.311+0000 7f0fc1a00640 1 -- 172.21.15.86:0/3204787312 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f0f9c06f520 msgr2=0x7f0f9c071a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/1173654665 2024-04-25T14:13:16.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.311+0000 7f0fb9600640 1 -- 172.21.15.86:0/3204787312 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6532+0+0 (secure 0 0 0) 0x7f0fa812c0d0 con 0x7f0fbc1616e0 2024-04-25T14:13:16.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.311+0000 7f0fc1a00640 1 --2- 172.21.15.86:0/3204787312 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f0f9c06f520 0x7f0f9c071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:13:16.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.319+0000 7f0fb9600640 1 -- 172.21.15.86:0/3204787312 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0fa80187d0 con 0x7f0fbc1616e0 2024-04-25T14:13:16.360 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:16.363+0000 7f7e35034180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:13:16.384 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:16.388+0000 7faa93941180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:13:16.455 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:16.456+0000 7faa93941180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:13:16.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.483+0000 7f0fc4400640 1 -- 172.21.15.86:0/3204787312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0fbc076bf0 con 0x7f0fbc1616e0 2024-04-25T14:13:16.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.487+0000 7f0fb9600640 1 -- 172.21.15.86:0/3204787312 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) ==== 74+0+17534 (secure 0 0 0) 0x7f0fa80a8940 con 0x7f0fbc1616e0 2024-04-25T14:13:16.486 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:13:16.486 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":143,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:12:52.973064+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:11:30.810321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T14:11:31.944888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.e","mappings":[{"from":3,"to":1}]},{"pgid":"22.32","mappings":[{"from":3,"to":0}]},{"pgid":"22.34","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/4083287014":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/746190999":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/1426015871":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/3962343281":"2024-04-26T14:11:51.781677+0000","172.21.15.86:6833/264890479":"2024-04-26T14:11:51.781677+0000","172.21.15.86:6832/264890479":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/2724786983":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/285240843":"2024-04-26T14:11:51.781677+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:13:16.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.491+0000 7f0fc4400640 1 -- 172.21.15.86:0/3204787312 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f0f9c06f520 msgr2=0x7f0f9c071a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:13:16.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.491+0000 7f0fc4400640 1 --2- 172.21.15.86:0/3204787312 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f0f9c06f520 0x7f0f9c071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:16.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.491+0000 7f0fc4400640 1 -- 172.21.15.86:0/3204787312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fbc1616e0 msgr2=0x7f0fbc151e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:16.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.491+0000 7f0fc4400640 1 --2- 172.21.15.86:0/3204787312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fbc1616e0 0x7f0fbc151e50 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f0fa80b3040 tx=0x7f0fa801ba10 comp rx=0 tx=0).stop 2024-04-25T14:13:16.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.491+0000 7f0fc4400640 1 -- 172.21.15.86:0/3204787312 shutdown_connections 2024-04-25T14:13:16.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.491+0000 7f0fc4400640 1 --2- 172.21.15.86:0/3204787312 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f0f9c06f520 0x7f0f9c071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:16.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.491+0000 7f0fc4400640 1 --2- 172.21.15.86:0/3204787312 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fbc0b4240 0x7f0fbc1f6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:16.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.491+0000 7f0fc4400640 1 --2- 172.21.15.86:0/3204787312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fbc1616e0 0x7f0fbc151e50 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:16.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.491+0000 7f0fc4400640 1 --2- 172.21.15.86:0/3204787312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fbc0d2c80 0x7f0fbc0af290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:16.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.491+0000 7f0fc4400640 1 -- 172.21.15.86:0/3204787312 >> 172.21.15.86:0/3204787312 conn(0x7f0fbc0cffc0 msgr2=0x7f0fbc0722f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:16.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.491+0000 7f0fc4400640 1 -- 172.21.15.86:0/3204787312 shutdown_connections 2024-04-25T14:13:16.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.491+0000 7f0fc4400640 1 -- 172.21.15.86:0/3204787312 wait complete. 2024-04-25T14:13:16.489 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:16.491+0000 7f7e35034180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:13:16.510 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:11:30.810321+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:11:31.944888+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:13:16.510 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:13:16.564 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:16.567+0000 7f7e35034180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:13:16.607 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:16.608+0000 7faa93941180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:13:16.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.667+0000 7f9793a00640 1 Processor -- start 2024-04-25T14:13:16.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.667+0000 7f9793a00640 1 -- start start 2024-04-25T14:13:16.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.667+0000 7f9793a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f978c0d1df0 0x7f978c0cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:16.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.667+0000 7f9793a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f978c0cdcc0 0x7f978c0ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:16.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.667+0000 7f9793a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f978c15e5c0 con 0x7f978c070330 2024-04-25T14:13:16.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.667+0000 7f9793a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f978c06fd40 con 0x7f978c0cdcc0 2024-04-25T14:13:16.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.667+0000 7f9793a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f978c15e8a0 con 0x7f978c0d1df0 2024-04-25T14:13:16.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.667+0000 7f9791a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f978c0cdcc0 0x7f978c0ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:16.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.667+0000 7f9791a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f978c0cdcc0 0x7f978c0ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46230/0 (socket says 172.21.15.86:46230) 2024-04-25T14:13:16.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.667+0000 7f9791a00640 1 -- 172.21.15.86:0/3770582800 learned_addr learned my addr 172.21.15.86:0/3770582800 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:16.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.667+0000 7f978be00640 1 --2- 172.21.15.86:0/3770582800 >> v2:172.21.15.155:3300/0 conn(0x7f978c0d1df0 0x7f978c0cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:16.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.667+0000 7f9791a00640 1 -- 172.21.15.86:0/3770582800 >> v2:172.21.15.155:3300/0 conn(0x7f978c0d1df0 msgr2=0x7f978c0cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:16.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.667+0000 7f9791a00640 1 --2- 172.21.15.86:0/3770582800 >> v2:172.21.15.155:3300/0 conn(0x7f978c0d1df0 0x7f978c0cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:16.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.667+0000 7f9791a00640 1 -- 172.21.15.86:0/3770582800 >> v1:172.21.15.155:6789/0 conn(0x7f978c070330 legacy=0x7f978c0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:16.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.667+0000 7f9791a00640 1 -- 172.21.15.86:0/3770582800 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978c15eb80 con 0x7f978c0cdcc0 2024-04-25T14:13:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.667+0000 7f978b400640 1 -- 172.21.15.86:0/3770582800 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2826200827 0 0) 0x7f978c15e5c0 con 0x7f978c070330 2024-04-25T14:13:16.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.671+0000 7f9791a00640 1 --2- 172.21.15.86:0/3770582800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f978c0cdcc0 0x7f978c0ce0d0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f9784009e50 tx=0x7f978401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf10fc98ec2e2067 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:16.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.671+0000 7f978b400640 1 -- 172.21.15.86:0/3770582800 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97840a57d0 con 0x7f978c0cdcc0 2024-04-25T14:13:16.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.671+0000 7f978b400640 1 -- 172.21.15.86:0/3770582800 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97840a5970 con 0x7f978c0cdcc0 2024-04-25T14:13:16.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.671+0000 7f978b400640 1 -- 172.21.15.86:0/3770582800 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97840a5d40 con 0x7f978c0cdcc0 2024-04-25T14:13:16.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.671+0000 7f9793a00640 1 -- 172.21.15.86:0/3770582800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f978c0cdcc0 msgr2=0x7f978c0ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:16.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.671+0000 7f9793a00640 1 --2- 172.21.15.86:0/3770582800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f978c0cdcc0 0x7f978c0ce0d0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f9784009e50 tx=0x7f978401c930 comp rx=0 tx=0).stop 2024-04-25T14:13:16.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.671+0000 7f9793a00640 1 -- 172.21.15.86:0/3770582800 shutdown_connections 2024-04-25T14:13:16.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.671+0000 7f9793a00640 1 --2- 172.21.15.86:0/3770582800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f978c0cdcc0 0x7f978c0ce0d0 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:16.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.671+0000 7f9793a00640 1 --2- 172.21.15.86:0/3770582800 >> v2:172.21.15.155:3300/0 conn(0x7f978c0d1df0 0x7f978c0cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:16.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.671+0000 7f9793a00640 1 -- 172.21.15.86:0/3770582800 >> 172.21.15.86:0/3770582800 conn(0x7f978c0b4650 msgr2=0x7f978c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:16.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.671+0000 7f9793a00640 1 -- 172.21.15.86:0/3770582800 shutdown_connections 2024-04-25T14:13:16.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.671+0000 7f9793a00640 1 -- 172.21.15.86:0/3770582800 wait complete. 2024-04-25T14:13:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.675+0000 7f9793a00640 1 Processor -- start 2024-04-25T14:13:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.675+0000 7f9793a00640 1 -- start start 2024-04-25T14:13:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.675+0000 7f9793a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f978c070330 0x7f978c0ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.675+0000 7f9793a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f978c0cdcc0 0x7f978c09b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.675+0000 7f9793a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f978c1fa070 0x7f978c0ae130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.675+0000 7f9793a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f978c15f4b0 con 0x7f978c0cdcc0 2024-04-25T14:13:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.675+0000 7f9793a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f978c15f330 con 0x7f978c1fa070 2024-04-25T14:13:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.675+0000 7f9793a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f978c070740 con 0x7f978c070330 2024-04-25T14:13:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.675+0000 7f978be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f978c0cdcc0 0x7f978c09b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.675+0000 7f978be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f978c0cdcc0 0x7f978c09b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46236/0 (socket says 172.21.15.86:46236) 2024-04-25T14:13:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.675+0000 7f978be00640 1 -- 172.21.15.86:0/2507859917 learned_addr learned my addr 172.21.15.86:0/2507859917 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.675+0000 7f9791000640 1 --2- 172.21.15.86:0/2507859917 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f978c070330 0x7f978c0ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.675+0000 7f9791a00640 1 --2- 172.21.15.86:0/2507859917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f978c1fa070 0x7f978c0ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.675+0000 7f978be00640 1 -- 172.21.15.86:0/2507859917 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f978c070330 msgr2=0x7f978c0ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.675+0000 7f978be00640 1 --2- 172.21.15.86:0/2507859917 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f978c070330 0x7f978c0ca180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.675+0000 7f978be00640 1 -- 172.21.15.86:0/2507859917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f978c1fa070 msgr2=0x7f978c0ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.675+0000 7f978be00640 1 --2- 172.21.15.86:0/2507859917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f978c1fa070 0x7f978c0ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.675+0000 7f978be00640 1 -- 172.21.15.86:0/2507859917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978c06d840 con 0x7f978c0cdcc0 2024-04-25T14:13:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.675+0000 7f978be00640 1 --2- 172.21.15.86:0/2507859917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f978c0cdcc0 0x7f978c09b6f0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f977800e8a0 tx=0x7f977800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.679+0000 7f9788c00640 1 -- 172.21.15.86:0/2507859917 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9778017070 con 0x7f978c0cdcc0 2024-04-25T14:13:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.679+0000 7f9788c00640 1 -- 172.21.15.86:0/2507859917 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f977800ee90 con 0x7f978c0cdcc0 2024-04-25T14:13:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.679+0000 7f9793a00640 1 -- 172.21.15.86:0/2507859917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f978c09c220 con 0x7f978c0cdcc0 2024-04-25T14:13:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.679+0000 7f9788c00640 1 -- 172.21.15.86:0/2507859917 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977800cb40 con 0x7f978c0cdcc0 2024-04-25T14:13:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.679+0000 7f9793a00640 1 -- 172.21.15.86:0/2507859917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f978c09c7e0 con 0x7f978c0cdcc0 2024-04-25T14:13:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.679+0000 7f9793a00640 1 -- 172.21.15.86:0/2507859917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f978c069d00 con 0x7f978c0cdcc0 2024-04-25T14:13:16.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.679+0000 7f9788c00640 1 -- 172.21.15.86:0/2507859917 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 73) ==== 93309+0+0 (secure 0 0 0) 0x7f977801a070 con 0x7f978c0cdcc0 2024-04-25T14:13:16.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.679+0000 7f9788c00640 1 --2- 172.21.15.86:0/2507859917 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f976c06f550 0x7f976c071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:16.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.679+0000 7f9788c00640 1 -- 172.21.15.86:0/2507859917 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6532+0+0 (secure 0 0 0) 0x7f977808f580 con 0x7f978c0cdcc0 2024-04-25T14:13:16.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.679+0000 7f9791000640 1 -- 172.21.15.86:0/2507859917 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f976c06f550 msgr2=0x7f976c071a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/1173654665 2024-04-25T14:13:16.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.679+0000 7f9791000640 1 --2- 172.21.15.86:0/2507859917 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f976c06f550 0x7f976c071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:13:16.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.683+0000 7f9788c00640 1 -- 172.21.15.86:0/2507859917 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f977805b930 con 0x7f978c0cdcc0 2024-04-25T14:13:16.723 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:16.723+0000 7f7e35034180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:13:16.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.843+0000 7f9793a00640 1 -- 172.21.15.86:0/2507859917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f978c076bf0 con 0x7f978c0cdcc0 2024-04-25T14:13:16.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.843+0000 7f9788c00640 1 -- 172.21.15.86:0/2507859917 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v143) ==== 93+0+10 (secure 0 0 0) 0x7f977805fc30 con 0x7f978c0cdcc0 2024-04-25T14:13:16.844 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:13:16.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.847+0000 7f9793a00640 1 -- 172.21.15.86:0/2507859917 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f976c06f550 msgr2=0x7f976c071a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:13:16.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.847+0000 7f9793a00640 1 --2- 172.21.15.86:0/2507859917 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f976c06f550 0x7f976c071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:16.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.847+0000 7f9793a00640 1 -- 172.21.15.86:0/2507859917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f978c0cdcc0 msgr2=0x7f978c09b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:16.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.847+0000 7f9793a00640 1 --2- 172.21.15.86:0/2507859917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f978c0cdcc0 0x7f978c09b6f0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f977800e8a0 tx=0x7f977800ed60 comp rx=0 tx=0).stop 2024-04-25T14:13:16.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.847+0000 7f9793a00640 1 -- 172.21.15.86:0/2507859917 shutdown_connections 2024-04-25T14:13:16.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.847+0000 7f9793a00640 1 --2- 172.21.15.86:0/2507859917 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f976c06f550 0x7f976c071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:16.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.847+0000 7f9793a00640 1 --2- 172.21.15.86:0/2507859917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f978c1fa070 0x7f978c0ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:16.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.847+0000 7f9793a00640 1 --2- 172.21.15.86:0/2507859917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f978c0cdcc0 0x7f978c09b6f0 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:16.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.847+0000 7f9793a00640 1 --2- 172.21.15.86:0/2507859917 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f978c070330 0x7f978c0ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:16.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.847+0000 7f9793a00640 1 -- 172.21.15.86:0/2507859917 >> 172.21.15.86:0/2507859917 conn(0x7f978c0b4650 msgr2=0x7f978c15c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:16.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.847+0000 7f9793a00640 1 -- 172.21.15.86:0/2507859917 shutdown_connections 2024-04-25T14:13:16.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:16.847+0000 7f9793a00640 1 -- 172.21.15.86:0/2507859917 wait complete. 2024-04-25T14:13:16.868 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:13:17.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.023+0000 7f14f6a00640 1 Processor -- start 2024-04-25T14:13:17.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.023+0000 7f14f6a00640 1 -- start start 2024-04-25T14:13:17.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.027+0000 7f14f6a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f14f00d2c80 0x7f14f015dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:17.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.027+0000 7f14f6a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f14f0070e20 0x7f14f0071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:17.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.027+0000 7f14f6a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14f00722f0 0x7f14f0072700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:17.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.027+0000 7f14f6a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f14f00ce720 con 0x7f14f00d2c80 2024-04-25T14:13:17.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.027+0000 7f14f6a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f14f0071770 con 0x7f14f0070e20 2024-04-25T14:13:17.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.027+0000 7f14f6a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f14f00cd8d0 con 0x7f14f00722f0 2024-04-25T14:13:17.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.027+0000 7f14efe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f14f00d2c80 0x7f14f015dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:17.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.027+0000 7f14efe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f14f00d2c80 0x7f14f015dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46238/0 (socket says 172.21.15.86:46238) 2024-04-25T14:13:17.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.027+0000 7f14efe00640 1 -- 172.21.15.86:0/2693617053 learned_addr learned my addr 172.21.15.86:0/2693617053 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:17.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.027+0000 7f14e7e00640 1 --2- 172.21.15.86:0/2693617053 >> v2:172.21.15.155:3300/0 conn(0x7f14f0070e20 0x7f14f0071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:17.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.027+0000 7f14f4a00640 1 --2- 172.21.15.86:0/2693617053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14f00722f0 0x7f14f0072700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:17.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.027+0000 7f14efe00640 1 -- 172.21.15.86:0/2693617053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14f00722f0 msgr2=0x7f14f0072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:17.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.027+0000 7f14efe00640 1 --2- 172.21.15.86:0/2693617053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14f00722f0 0x7f14f0072700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.027+0000 7f14efe00640 1 -- 172.21.15.86:0/2693617053 >> v2:172.21.15.155:3300/0 conn(0x7f14f0070e20 msgr2=0x7f14f0071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:17.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.027+0000 7f14efe00640 1 --2- 172.21.15.86:0/2693617053 >> v2:172.21.15.155:3300/0 conn(0x7f14f0070e20 0x7f14f0071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.027+0000 7f14efe00640 1 -- 172.21.15.86:0/2693617053 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f00ce970 con 0x7f14f00d2c80 2024-04-25T14:13:17.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.031+0000 7f14efe00640 1 --2- 172.21.15.86:0/2693617053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14f00d2c80 0x7f14f015dc90 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f14f00b6070 tx=0x7f14e001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59b2aba45bb456b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:17.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.031+0000 7f14ef400640 1 -- 172.21.15.86:0/2693617053 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e00b5070 con 0x7f14f00d2c80 2024-04-25T14:13:17.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.031+0000 7f14ef400640 1 -- 172.21.15.86:0/2693617053 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e00a89d0 con 0x7f14f00d2c80 2024-04-25T14:13:17.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.031+0000 7f14ef400640 1 -- 172.21.15.86:0/2693617053 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e00a8da0 con 0x7f14f00d2c80 2024-04-25T14:13:17.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.031+0000 7f14f6a00640 1 -- 172.21.15.86:0/2693617053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14f00d2c80 msgr2=0x7f14f015dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:17.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.031+0000 7f14f6a00640 1 --2- 172.21.15.86:0/2693617053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14f00d2c80 0x7f14f015dc90 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f14f00b6070 tx=0x7f14e001ed10 comp rx=0 tx=0).stop 2024-04-25T14:13:17.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.031+0000 7f14f6a00640 1 -- 172.21.15.86:0/2693617053 shutdown_connections 2024-04-25T14:13:17.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.031+0000 7f14f6a00640 1 --2- 172.21.15.86:0/2693617053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14f00722f0 0x7f14f0072700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.031+0000 7f14f6a00640 1 --2- 172.21.15.86:0/2693617053 >> v2:172.21.15.155:3300/0 conn(0x7f14f0070e20 0x7f14f0071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.031+0000 7f14f6a00640 1 --2- 172.21.15.86:0/2693617053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14f00d2c80 0x7f14f015dc90 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.031+0000 7f14f6a00640 1 -- 172.21.15.86:0/2693617053 >> 172.21.15.86:0/2693617053 conn(0x7f14f00cffc0 msgr2=0x7f14f00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:17.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.031+0000 7f14f6a00640 1 -- 172.21.15.86:0/2693617053 shutdown_connections 2024-04-25T14:13:17.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.031+0000 7f14f6a00640 1 -- 172.21.15.86:0/2693617053 wait complete. 2024-04-25T14:13:17.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.031+0000 7f14f6a00640 1 Processor -- start 2024-04-25T14:13:17.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.031+0000 7f14f6a00640 1 -- start start 2024-04-25T14:13:17.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.035+0000 7f14f6a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14f0070e20 0x7f14f015f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:17.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.035+0000 7f14f6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14f00722f0 0x7f14f015f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:17.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.035+0000 7f14f6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14f00d2c80 0x7f14f0152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:17.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.035+0000 7f14f6a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14f00d21a0 con 0x7f14f00d2c80 2024-04-25T14:13:17.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.035+0000 7f14f6a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f14f00d1ec0 con 0x7f14f00722f0 2024-04-25T14:13:17.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.035+0000 7f14f6a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f14f00d2480 con 0x7f14f0070e20 2024-04-25T14:13:17.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.035+0000 7f14f4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14f00d2c80 0x7f14f0152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:17.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.035+0000 7f14f4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14f00d2c80 0x7f14f0152ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46250/0 (socket says 172.21.15.86:46250) 2024-04-25T14:13:17.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.035+0000 7f14f4a00640 1 -- 172.21.15.86:0/4107843388 learned_addr learned my addr 172.21.15.86:0/4107843388 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:17.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.035+0000 7f14efe00640 1 --2- 172.21.15.86:0/4107843388 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14f0070e20 0x7f14f015f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:17.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.035+0000 7f14e7e00640 1 --2- 172.21.15.86:0/4107843388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14f00722f0 0x7f14f015f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:17.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.035+0000 7f14efe00640 1 -- 172.21.15.86:0/4107843388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14f00722f0 msgr2=0x7f14f015f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:17.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.035+0000 7f14efe00640 1 --2- 172.21.15.86:0/4107843388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14f00722f0 0x7f14f015f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.035+0000 7f14efe00640 1 -- 172.21.15.86:0/4107843388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14f00d2c80 msgr2=0x7f14f0152ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:17.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.035+0000 7f14efe00640 1 --2- 172.21.15.86:0/4107843388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14f00d2c80 0x7f14f0152ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.035+0000 7f14efe00640 1 -- 172.21.15.86:0/4107843388 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f01e6f50 con 0x7f14f0070e20 2024-04-25T14:13:17.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.035+0000 7f14f4a00640 1 --2- 172.21.15.86:0/4107843388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14f00d2c80 0x7f14f0152ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:13:17.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.035+0000 7f14efe00640 1 --2- 172.21.15.86:0/4107843388 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14f0070e20 0x7f14f015f8f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f14e00af000 tx=0x7f14e001bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:17.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.035+0000 7f14ecc00640 1 -- 172.21.15.86:0/4107843388 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e00b5070 con 0x7f14f0070e20 2024-04-25T14:13:17.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.035+0000 7f14ecc00640 1 -- 172.21.15.86:0/4107843388 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e00b0950 con 0x7f14f0070e20 2024-04-25T14:13:17.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.035+0000 7f14f6a00640 1 -- 172.21.15.86:0/4107843388 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f01e71f0 con 0x7f14f0070e20 2024-04-25T14:13:17.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.035+0000 7f14ecc00640 1 -- 172.21.15.86:0/4107843388 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e00b0d40 con 0x7f14f0070e20 2024-04-25T14:13:17.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.035+0000 7f14f6a00640 1 -- 172.21.15.86:0/4107843388 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f015edd0 con 0x7f14f0070e20 2024-04-25T14:13:17.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.035+0000 7f14f6a00640 1 -- 172.21.15.86:0/4107843388 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14f00720e0 con 0x7f14f0070e20 2024-04-25T14:13:17.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.039+0000 7f14ecc00640 1 -- 172.21.15.86:0/4107843388 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 73) ==== 93309+0+0 (secure 0 0 0) 0x7f14e00c2060 con 0x7f14f0070e20 2024-04-25T14:13:17.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.039+0000 7f14ecc00640 1 --2- 172.21.15.86:0/4107843388 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f14cc06f550 0x7f14cc071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:17.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.039+0000 7f14ecc00640 1 -- 172.21.15.86:0/4107843388 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(143..143 src has 1..143) ==== 6532+0+0 (secure 0 0 0) 0x7f14e012c630 con 0x7f14f0070e20 2024-04-25T14:13:17.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.039+0000 7f14e7e00640 1 -- 172.21.15.86:0/4107843388 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f14cc06f550 msgr2=0x7f14cc071a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/1173654665 2024-04-25T14:13:17.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.039+0000 7f14e7e00640 1 --2- 172.21.15.86:0/4107843388 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f14cc06f550 0x7f14cc071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:13:17.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.043+0000 7f14ecc00640 1 -- 172.21.15.86:0/4107843388 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f14e0018870 con 0x7f14f0070e20 2024-04-25T14:13:17.163 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:17.164+0000 7faa93941180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:13:17.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.203+0000 7f14f6a00640 1 -- 172.21.15.86:0/4107843388 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f14f0076bf0 con 0x7f14f0070e20 2024-04-25T14:13:17.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.203+0000 7f14ecc00640 1 -- 172.21.15.86:0/4107843388 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v143) ==== 104+0+11 (secure 0 0 0) 0x7f14e00f8a70 con 0x7f14f0070e20 2024-04-25T14:13:17.204 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:13:17.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.207+0000 7f14f6a00640 1 -- 172.21.15.86:0/4107843388 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f14cc06f550 msgr2=0x7f14cc071a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:13:17.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.207+0000 7f14f6a00640 1 --2- 172.21.15.86:0/4107843388 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f14cc06f550 0x7f14cc071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.207+0000 7f14f6a00640 1 -- 172.21.15.86:0/4107843388 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14f0070e20 msgr2=0x7f14f015f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:17.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.207+0000 7f14f6a00640 1 --2- 172.21.15.86:0/4107843388 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14f0070e20 0x7f14f015f8f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f14e00af000 tx=0x7f14e001bab0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.207+0000 7f14f6a00640 1 -- 172.21.15.86:0/4107843388 shutdown_connections 2024-04-25T14:13:17.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.207+0000 7f14f6a00640 1 --2- 172.21.15.86:0/4107843388 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f14cc06f550 0x7f14cc071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.207+0000 7f14f6a00640 1 --2- 172.21.15.86:0/4107843388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14f00d2c80 0x7f14f0152ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.207+0000 7f14f6a00640 1 --2- 172.21.15.86:0/4107843388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14f00722f0 0x7f14f015f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.207+0000 7f14f6a00640 1 --2- 172.21.15.86:0/4107843388 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14f0070e20 0x7f14f015f8f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.207+0000 7f14f6a00640 1 -- 172.21.15.86:0/4107843388 >> 172.21.15.86:0/4107843388 conn(0x7f14f00cffc0 msgr2=0x7f14f0159530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:17.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.207+0000 7f14f6a00640 1 -- 172.21.15.86:0/4107843388 shutdown_connections 2024-04-25T14:13:17.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.207+0000 7f14f6a00640 1 -- 172.21.15.86:0/4107843388 wait complete. 2024-04-25T14:13:17.226 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:13:17.247 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:17.248+0000 7faa93941180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:13:17.292 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:17.295+0000 7f7e35034180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:13:17.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.379+0000 7f747be00640 1 Processor -- start 2024-04-25T14:13:17.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.379+0000 7f747be00640 1 -- start start 2024-04-25T14:13:17.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.379+0000 7f747be00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74741643d0 0x7f74740d15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:17.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.379+0000 7f747be00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74740d1eb0 con 0x7f74740cdde0 2024-04-25T14:13:17.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.379+0000 7f747be00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f747406e290 con 0x7f7474071530 2024-04-25T14:13:17.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.379+0000 7f747be00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f747406e570 con 0x7f74741643d0 2024-04-25T14:13:17.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.379+0000 7f7478a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f74740cdde0 0x7f7474070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45506/0 (socket says 172.21.15.86:45506) 2024-04-25T14:13:17.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.379+0000 7f7478a00640 1 -- 172.21.15.86:0/3199521313 learned_addr learned my addr 172.21.15.86:0/3199521313 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:17.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.383+0000 7f7479400640 1 --2- 172.21.15.86:0/3199521313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74741643d0 0x7f74740d15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:17.381 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:17.383+0000 7f7e35034180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:13:17.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.383+0000 7f7473e00640 1 -- 172.21.15.86:0/3199521313 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 851733553 0 0) 0x7f74740d1eb0 con 0x7f74740cdde0 2024-04-25T14:13:17.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.383+0000 7f7473e00640 1 -- 172.21.15.86:0/3199521313 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f745c003530 con 0x7f74740cdde0 2024-04-25T14:13:17.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.383+0000 7f7473e00640 1 -- 172.21.15.86:0/3199521313 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3782721058 0 0) 0x7f747406e290 con 0x7f7474071530 2024-04-25T14:13:17.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.383+0000 7f7473e00640 1 -- 172.21.15.86:0/3199521313 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74740d1eb0 con 0x7f7474071530 2024-04-25T14:13:17.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.383+0000 7f7479400640 1 -- 172.21.15.86:0/3199521313 >> v1:172.21.15.155:6789/0 conn(0x7f7474071530 legacy=0x7f747406ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:17.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.383+0000 7f7479400640 1 -- 172.21.15.86:0/3199521313 >> v1:172.21.15.86:6789/0 conn(0x7f74740cdde0 legacy=0x7f7474070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:17.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.383+0000 7f7479400640 1 -- 172.21.15.86:0/3199521313 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f747406e850 con 0x7f74741643d0 2024-04-25T14:13:17.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.383+0000 7f7479400640 1 --2- 172.21.15.86:0/3199521313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74741643d0 0x7f74740d15b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f74680097f0 tx=0x7f746801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6db2beb22d43b9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:17.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.383+0000 7f7473e00640 1 -- 172.21.15.86:0/3199521313 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74680ab5f0 con 0x7f74741643d0 2024-04-25T14:13:17.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.383+0000 7f7473e00640 1 -- 172.21.15.86:0/3199521313 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74680b0070 con 0x7f74741643d0 2024-04-25T14:13:17.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.387+0000 7f7473e00640 1 -- 172.21.15.86:0/3199521313 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74680ab9b0 con 0x7f74741643d0 2024-04-25T14:13:17.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.387+0000 7f747be00640 1 -- 172.21.15.86:0/3199521313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74741643d0 msgr2=0x7f74740d15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:17.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.387+0000 7f747be00640 1 --2- 172.21.15.86:0/3199521313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74741643d0 0x7f74740d15b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f74680097f0 tx=0x7f746801c4e0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.387+0000 7f747be00640 1 -- 172.21.15.86:0/3199521313 shutdown_connections 2024-04-25T14:13:17.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.387+0000 7f747be00640 1 --2- 172.21.15.86:0/3199521313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74741643d0 0x7f74740d15b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.387+0000 7f747be00640 1 -- 172.21.15.86:0/3199521313 >> 172.21.15.86:0/3199521313 conn(0x7f74740cffc0 msgr2=0x7f74740d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:17.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.387+0000 7f747be00640 1 -- 172.21.15.86:0/3199521313 shutdown_connections 2024-04-25T14:13:17.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.387+0000 7f747be00640 1 -- 172.21.15.86:0/3199521313 wait complete. 2024-04-25T14:13:17.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.387+0000 7f747be00640 1 Processor -- start 2024-04-25T14:13:17.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.387+0000 7f747be00640 1 -- start start 2024-04-25T14:13:17.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.391+0000 7f747be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74740cdde0 0x7f74740ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:17.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.391+0000 7f747be00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74741643d0 0x7f747415b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:17.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.391+0000 7f747be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74741481f0 0x7f74740b8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:17.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.391+0000 7f747be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7474172ab0 con 0x7f74740cdde0 2024-04-25T14:13:17.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.391+0000 7f747be00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f74740ce600 con 0x7f74741481f0 2024-04-25T14:13:17.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.391+0000 7f747be00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7474172c30 con 0x7f74741643d0 2024-04-25T14:13:17.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.391+0000 7f7479400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74740cdde0 0x7f74740ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:17.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.391+0000 7f7479400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74740cdde0 0x7f74740ca580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46256/0 (socket says 172.21.15.86:46256) 2024-04-25T14:13:17.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.391+0000 7f7479400640 1 -- 172.21.15.86:0/526479069 learned_addr learned my addr 172.21.15.86:0/526479069 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:17.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.391+0000 7f7478a00640 1 --2- 172.21.15.86:0/526479069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74741643d0 0x7f747415b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:17.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.391+0000 7f7479e00640 1 --2- 172.21.15.86:0/526479069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74741481f0 0x7f74740b8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:17.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.391+0000 7f7479400640 1 -- 172.21.15.86:0/526479069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74741643d0 msgr2=0x7f747415b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:17.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.391+0000 7f7479400640 1 --2- 172.21.15.86:0/526479069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74741643d0 0x7f747415b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.391+0000 7f7479400640 1 -- 172.21.15.86:0/526479069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74741481f0 msgr2=0x7f74740b8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:17.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.391+0000 7f7479400640 1 --2- 172.21.15.86:0/526479069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74741481f0 0x7f74740b8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.391+0000 7f7479400640 1 -- 172.21.15.86:0/526479069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74740cabf0 con 0x7f74740cdde0 2024-04-25T14:13:17.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.391+0000 7f7479400640 1 --2- 172.21.15.86:0/526479069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74740cdde0 0x7f74740ca580 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f74680190f0 tx=0x7f746801ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:17.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.391+0000 7f7471600640 1 -- 172.21.15.86:0/526479069 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74680bf410 con 0x7f74740cdde0 2024-04-25T14:13:17.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.391+0000 7f7471600640 1 -- 172.21.15.86:0/526479069 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74680bf5d0 con 0x7f74740cdde0 2024-04-25T14:13:17.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.391+0000 7f747be00640 1 -- 172.21.15.86:0/526479069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f747415de80 con 0x7f74740cdde0 2024-04-25T14:13:17.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.391+0000 7f7471600640 1 -- 172.21.15.86:0/526479069 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74680bf9c0 con 0x7f74740cdde0 2024-04-25T14:13:17.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.391+0000 7f747be00640 1 -- 172.21.15.86:0/526479069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f747415e440 con 0x7f74740cdde0 2024-04-25T14:13:17.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.391+0000 7f747be00640 1 -- 172.21.15.86:0/526479069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74740720e0 con 0x7f74740cdde0 2024-04-25T14:13:17.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.395+0000 7f7471600640 1 -- 172.21.15.86:0/526479069 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 73) ==== 93309+0+0 (secure 0 0 0) 0x7f74680bd020 con 0x7f74740cdde0 2024-04-25T14:13:17.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.395+0000 7f7471600640 1 --2- 172.21.15.86:0/526479069 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f745406f550 0x7f7454071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:17.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.395+0000 7f7471600640 1 -- 172.21.15.86:0/526479069 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6532+0+0 (secure 0 0 0) 0x7f7468129810 con 0x7f74740cdde0 2024-04-25T14:13:17.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.395+0000 7f7478a00640 1 -- 172.21.15.86:0/526479069 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f745406f550 msgr2=0x7f7454071a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/1173654665 2024-04-25T14:13:17.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.395+0000 7f7478a00640 1 --2- 172.21.15.86:0/526479069 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f745406f550 0x7f7454071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:13:17.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.399+0000 7f7471600640 1 -- 172.21.15.86:0/526479069 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f74680c6030 con 0x7f74740cdde0 2024-04-25T14:13:17.409 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:17.412+0000 7faa93941180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:13:17.493 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:17.496+0000 7faa93941180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:13:17.545 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:17.547+0000 7f7e35034180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:13:17.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.559+0000 7f747be00640 1 -- 172.21.15.86:0/526479069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f74740b94c0 con 0x7f74740cdde0 2024-04-25T14:13:17.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.559+0000 7f7471600640 1 -- 172.21.15.86:0/526479069 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v143) ==== 100+0+11 (secure 0 0 0) 0x7f74680160d0 con 0x7f74740cdde0 2024-04-25T14:13:17.557 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:13:17.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.559+0000 7f747be00640 1 -- 172.21.15.86:0/526479069 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f745406f550 msgr2=0x7f7454071a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:13:17.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.559+0000 7f747be00640 1 --2- 172.21.15.86:0/526479069 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f745406f550 0x7f7454071a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.559+0000 7f747be00640 1 -- 172.21.15.86:0/526479069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74740cdde0 msgr2=0x7f74740ca580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:17.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.559+0000 7f747be00640 1 --2- 172.21.15.86:0/526479069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74740cdde0 0x7f74740ca580 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f74680190f0 tx=0x7f746801ad20 comp rx=0 tx=0).stop 2024-04-25T14:13:17.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.559+0000 7f747be00640 1 -- 172.21.15.86:0/526479069 shutdown_connections 2024-04-25T14:13:17.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.559+0000 7f747be00640 1 --2- 172.21.15.86:0/526479069 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f745406f550 0x7f7454071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.559+0000 7f747be00640 1 --2- 172.21.15.86:0/526479069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74741481f0 0x7f74740b8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.559+0000 7f747be00640 1 --2- 172.21.15.86:0/526479069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74741643d0 0x7f747415b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.559+0000 7f747be00640 1 --2- 172.21.15.86:0/526479069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74740cdde0 0x7f74740ca580 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.559+0000 7f747be00640 1 -- 172.21.15.86:0/526479069 >> 172.21.15.86:0/526479069 conn(0x7f74740cffc0 msgr2=0x7f7474070e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:17.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.559+0000 7f747be00640 1 -- 172.21.15.86:0/526479069 shutdown_connections 2024-04-25T14:13:17.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.559+0000 7f747be00640 1 -- 172.21.15.86:0/526479069 wait complete. 2024-04-25T14:13:17.579 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T14:13:17.637 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:17.639+0000 7f7e35034180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:13:17.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.735+0000 7f321f200640 1 Processor -- start 2024-04-25T14:13:17.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.735+0000 7f321f200640 1 -- start start 2024-04-25T14:13:17.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.735+0000 7f321f200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3218071530 0x7f32180ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:17.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.735+0000 7f321f200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f321806d940 con 0x7f32180cdd70 2024-04-25T14:13:17.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.735+0000 7f321f200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f321806dc20 con 0x7f321806e700 2024-04-25T14:13:17.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.735+0000 7f321f200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f321806df00 con 0x7f3218071530 2024-04-25T14:13:17.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.735+0000 7f320fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3218071530 0x7f32180ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:17.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.735+0000 7f3217e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f32180cdd70 0x7f3218070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45514/0 (socket says 172.21.15.86:45514) 2024-04-25T14:13:17.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.735+0000 7f3217e00640 1 -- 172.21.15.86:0/3659652381 learned_addr learned my addr 172.21.15.86:0/3659652381 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:17.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.735+0000 7f320fe00640 1 -- 172.21.15.86:0/3659652381 >> v1:172.21.15.155:6789/0 conn(0x7f321806e700 legacy=0x7f321806eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:17.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.735+0000 7f320fe00640 1 -- 172.21.15.86:0/3659652381 >> v1:172.21.15.86:6789/0 conn(0x7f32180cdd70 legacy=0x7f3218070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:17.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.735+0000 7f320fe00640 1 -- 172.21.15.86:0/3659652381 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321806e1e0 con 0x7f3218071530 2024-04-25T14:13:17.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.735+0000 7f3217400640 1 -- 172.21.15.86:0/3659652381 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1509513310 0 0) 0x7f321806d940 con 0x7f32180cdd70 2024-04-25T14:13:17.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.739+0000 7f320fe00640 1 --2- 172.21.15.86:0/3659652381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3218071530 0x7f32180ce180 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f32000097a0 tx=0x7f320001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2dc27543bfdb44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:17.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.739+0000 7f3217400640 1 -- 172.21.15.86:0/3659652381 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32000b0070 con 0x7f3218071530 2024-04-25T14:13:17.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.739+0000 7f3217400640 1 -- 172.21.15.86:0/3659652381 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32000ab600 con 0x7f3218071530 2024-04-25T14:13:17.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.739+0000 7f3217400640 1 -- 172.21.15.86:0/3659652381 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32000ab9d0 con 0x7f3218071530 2024-04-25T14:13:17.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.739+0000 7f321f200640 1 -- 172.21.15.86:0/3659652381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3218071530 msgr2=0x7f32180ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:17.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.739+0000 7f321f200640 1 --2- 172.21.15.86:0/3659652381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3218071530 0x7f32180ce180 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f32000097a0 tx=0x7f320001c3f0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.739+0000 7f321f200640 1 -- 172.21.15.86:0/3659652381 shutdown_connections 2024-04-25T14:13:17.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.739+0000 7f321f200640 1 --2- 172.21.15.86:0/3659652381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3218071530 0x7f32180ce180 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.739+0000 7f321f200640 1 -- 172.21.15.86:0/3659652381 >> 172.21.15.86:0/3659652381 conn(0x7f32180cffc0 msgr2=0x7f32180d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:17.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.739+0000 7f321f200640 1 -- 172.21.15.86:0/3659652381 shutdown_connections 2024-04-25T14:13:17.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.743+0000 7f321f200640 1 -- 172.21.15.86:0/3659652381 wait complete. 2024-04-25T14:13:17.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.743+0000 7f321f200640 1 Processor -- start 2024-04-25T14:13:17.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.743+0000 7f321f200640 1 -- start start 2024-04-25T14:13:17.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.743+0000 7f321f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f321806e700 0x7f321815c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:17.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.743+0000 7f321f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32180cdd70 0x7f32181560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:17.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.743+0000 7f321f200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32181565e0 0x7f32180c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:17.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.743+0000 7f321f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f321816dc10 con 0x7f321806e700 2024-04-25T14:13:17.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.743+0000 7f321f200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3218164380 con 0x7f32180cdd70 2024-04-25T14:13:17.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.743+0000 7f321f200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f321816dd90 con 0x7f32181565e0 2024-04-25T14:13:17.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.747+0000 7f3217e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f321806e700 0x7f321815c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:17.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.747+0000 7f3217e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f321806e700 0x7f321815c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46280/0 (socket says 172.21.15.86:46280) 2024-04-25T14:13:17.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.747+0000 7f3217e00640 1 -- 172.21.15.86:0/334916291 learned_addr learned my addr 172.21.15.86:0/334916291 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:17.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.747+0000 7f320fe00640 1 --2- 172.21.15.86:0/334916291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32180cdd70 0x7f32181560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:17.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.747+0000 7f321d200640 1 --2- 172.21.15.86:0/334916291 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32181565e0 0x7f32180c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:17.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.747+0000 7f3217e00640 1 -- 172.21.15.86:0/334916291 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32181565e0 msgr2=0x7f32180c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:17.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.747+0000 7f3217e00640 1 --2- 172.21.15.86:0/334916291 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32181565e0 0x7f32180c8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.747+0000 7f3217e00640 1 -- 172.21.15.86:0/334916291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32180cdd70 msgr2=0x7f32181560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:17.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.747+0000 7f3217e00640 1 --2- 172.21.15.86:0/334916291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32180cdd70 0x7f32181560a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.747+0000 7f3217e00640 1 -- 172.21.15.86:0/334916291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32181569f0 con 0x7f321806e700 2024-04-25T14:13:17.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.747+0000 7f3217e00640 1 --2- 172.21.15.86:0/334916291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f321806e700 0x7f321815c950 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f320800c410 tx=0x7f320800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:17.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.747+0000 7f3214c00640 1 -- 172.21.15.86:0/334916291 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32080074e0 con 0x7f321806e700 2024-04-25T14:13:17.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.747+0000 7f3214c00640 1 -- 172.21.15.86:0/334916291 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32080047f0 con 0x7f321806e700 2024-04-25T14:13:17.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.747+0000 7f3214c00640 1 -- 172.21.15.86:0/334916291 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208007680 con 0x7f321806e700 2024-04-25T14:13:17.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.747+0000 7f321f200640 1 -- 172.21.15.86:0/334916291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3218162d30 con 0x7f321806e700 2024-04-25T14:13:17.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.747+0000 7f321f200640 1 -- 172.21.15.86:0/334916291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32180c9490 con 0x7f321806e700 2024-04-25T14:13:17.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.747+0000 7f321f200640 1 -- 172.21.15.86:0/334916291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32180720e0 con 0x7f321806e700 2024-04-25T14:13:17.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.747+0000 7f3214c00640 1 -- 172.21.15.86:0/334916291 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 73) ==== 93309+0+0 (secure 0 0 0) 0x7f3208011070 con 0x7f321806e700 2024-04-25T14:13:17.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.751+0000 7f3214c00640 1 --2- 172.21.15.86:0/334916291 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f31f406f520 0x7f31f4071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:17.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.751+0000 7f3214c00640 1 -- 172.21.15.86:0/334916291 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6532+0+0 (secure 0 0 0) 0x7f320808f7f0 con 0x7f321806e700 2024-04-25T14:13:17.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.751+0000 7f320fe00640 1 -- 172.21.15.86:0/334916291 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f31f406f520 msgr2=0x7f31f4071a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/1173654665 2024-04-25T14:13:17.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.751+0000 7f320fe00640 1 --2- 172.21.15.86:0/334916291 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f31f406f520 0x7f31f4071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:13:17.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.751+0000 7f3214c00640 1 -- 172.21.15.86:0/334916291 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f32181569f0 con 0x7f321806e700 2024-04-25T14:13:17.822 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:17.824+0000 7faa93941180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:13:17.884 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:17.888+0000 7faa93941180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:13:17.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.919+0000 7f3214c00640 1 -- 172.21.15.86:0/334916291 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 74) ==== 93309+0+0 (secure 0 0 0) 0x7f3208057f40 con 0x7f321806e700 2024-04-25T14:13:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.927+0000 7f321f200640 1 -- 172.21.15.86:0/334916291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3218076bf0 con 0x7f321806e700 2024-04-25T14:13:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.927+0000 7f3214c00640 1 -- 172.21.15.86:0/334916291 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v143) ==== 80+0+466 (secure 0 0 0) 0x7f3208080420 con 0x7f321806e700 2024-04-25T14:13:17.927 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/4083287014 2024-04-26T14:11:51.781677+0000 2024-04-25T14:13:17.927 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/746190999 2024-04-26T14:11:51.781677+0000 2024-04-25T14:13:17.927 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1426015871 2024-04-26T14:11:51.781677+0000 2024-04-25T14:13:17.927 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3962343281 2024-04-26T14:11:51.781677+0000 2024-04-25T14:13:17.927 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6833/264890479 2024-04-26T14:11:51.781677+0000 2024-04-25T14:13:17.927 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6832/264890479 2024-04-26T14:11:51.781677+0000 2024-04-25T14:13:17.928 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2724786983 2024-04-26T14:11:51.781677+0000 2024-04-25T14:13:17.928 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/285240843 2024-04-26T14:11:51.781677+0000 2024-04-25T14:13:17.929 INFO:teuthology.orchestra.run.smithi086.stderr:listed 8 entries 2024-04-25T14:13:17.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.931+0000 7f321f200640 1 -- 172.21.15.86:0/334916291 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f31f406f520 msgr2=0x7f31f4071a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:13:17.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.931+0000 7f321f200640 1 --2- 172.21.15.86:0/334916291 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f31f406f520 0x7f31f4071a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.931+0000 7f321f200640 1 -- 172.21.15.86:0/334916291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f321806e700 msgr2=0x7f321815c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:17.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.931+0000 7f321f200640 1 --2- 172.21.15.86:0/334916291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f321806e700 0x7f321815c950 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f320800c410 tx=0x7f320800c8d0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.931+0000 7f321f200640 1 -- 172.21.15.86:0/334916291 shutdown_connections 2024-04-25T14:13:17.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.931+0000 7f321f200640 1 --2- 172.21.15.86:0/334916291 >> [v2:172.21.15.86:6832/1173654665,v1:172.21.15.86:6833/1173654665] conn(0x7f31f406f520 0x7f31f4071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.931+0000 7f321f200640 1 --2- 172.21.15.86:0/334916291 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32181565e0 0x7f32180c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.931+0000 7f321f200640 1 --2- 172.21.15.86:0/334916291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32180cdd70 0x7f32181560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.931+0000 7f321f200640 1 --2- 172.21.15.86:0/334916291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f321806e700 0x7f321815c950 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:17.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.931+0000 7f321f200640 1 -- 172.21.15.86:0/334916291 >> 172.21.15.86:0/334916291 conn(0x7f32180cffc0 msgr2=0x7f3218154390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:17.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.931+0000 7f321f200640 1 -- 172.21.15.86:0/334916291 shutdown_connections 2024-04-25T14:13:17.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:17.931+0000 7f321f200640 1 -- 172.21.15.86:0/334916291 wait complete. 2024-04-25T14:13:17.949 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T14:13:17.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:17.949 DEBUG:teuthology.orchestra.run.smithi155:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T14:13:17.976 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T14:13:17.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:17.976 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T14:13:17.977 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:17.979+0000 7f7e35034180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:13:18.032 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:13:18.032 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:13:18.032 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:13:18.040 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:18.043+0000 7f7e35034180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:13:18.042 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2893 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T14:13:18.042 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:13:18.042 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:13:18.042 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:13:18.087 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4517 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T14:13:18.088 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:13:18.124 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:18.127+0000 7f7db6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:13:18.124 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:18.127+0000 7f7db6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:13:18.124 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:18.127+0000 7f7db6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:13:18.124 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:18.127+0000 7f7db6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:13:18.124 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:18.127+0000 7f7db6600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:13:18.125 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:18.127+0000 7f7db2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:13:18.125 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:18.127+0000 7f7db2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:13:18.125 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:18.127+0000 7f7db2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:13:18.125 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:18.127+0000 7f7db2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:13:18.125 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:18.127+0000 7f7db2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:13:18.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.247+0000 7f4ca9800640 1 Processor -- start 2024-04-25T14:13:18.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.247+0000 7f4ca9800640 1 -- start start 2024-04-25T14:13:18.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.247+0000 7f4ca9800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4ca40722a0 0x7f4ca40726b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:18.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.247+0000 7f4ca9800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4ca41614c0 0x7f4ca41618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:18.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.247+0000 7f4ca9800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ca4072bf0 con 0x7f4ca40b6ee0 2024-04-25T14:13:18.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.247+0000 7f4ca9800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4ca415df00 con 0x7f4ca41614c0 2024-04-25T14:13:18.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.247+0000 7f4ca9800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f4ca415e1e0 con 0x7f4ca40722a0 2024-04-25T14:13:18.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.247+0000 7f4ca3e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f4ca40b6ee0 0x7f4ca40d06c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45518/0 (socket says 172.21.15.86:45518) 2024-04-25T14:13:18.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.247+0000 7f4ca3e00640 1 -- 172.21.15.86:0/2215519795 learned_addr learned my addr 172.21.15.86:0/2215519795 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:18.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.247+0000 7f4ca8e00640 1 --2- 172.21.15.86:0/2215519795 >> v2:172.21.15.86:3300/0 conn(0x7f4ca41614c0 0x7f4ca41618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:18.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.247+0000 7f4ca3400640 1 --2- 172.21.15.86:0/2215519795 >> v2:172.21.15.155:3300/0 conn(0x7f4ca40722a0 0x7f4ca40726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:18.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.251+0000 7f4ca2a00640 1 -- 172.21.15.86:0/2215519795 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3584713331 0 0) 0x7f4c94001120 con 0x7f4ca40b6ee0 2024-04-25T14:13:18.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.251+0000 7f4ca2a00640 1 -- 172.21.15.86:0/2215519795 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c88003530 con 0x7f4ca40b6ee0 2024-04-25T14:13:18.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.251+0000 7f4ca8e00640 1 -- 172.21.15.86:0/2215519795 >> v2:172.21.15.155:3300/0 conn(0x7f4ca40722a0 msgr2=0x7f4ca40726b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:18.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.251+0000 7f4ca8e00640 1 --2- 172.21.15.86:0/2215519795 >> v2:172.21.15.155:3300/0 conn(0x7f4ca40722a0 0x7f4ca40726b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:18.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.251+0000 7f4ca8e00640 1 -- 172.21.15.86:0/2215519795 >> v1:172.21.15.86:6789/0 conn(0x7f4ca40b6ee0 legacy=0x7f4ca40d06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:18.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.251+0000 7f4ca8e00640 1 -- 172.21.15.86:0/2215519795 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca415e4c0 con 0x7f4ca41614c0 2024-04-25T14:13:18.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.251+0000 7f4ca8e00640 1 --2- 172.21.15.86:0/2215519795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ca41614c0 0x7f4ca41618d0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c009d90 tx=0x7f4c9c01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5df3e2b0f618ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:18.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.251+0000 7f4ca2a00640 1 -- 172.21.15.86:0/2215519795 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c0b2070 con 0x7f4ca41614c0 2024-04-25T14:13:18.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.251+0000 7f4ca2a00640 1 -- 172.21.15.86:0/2215519795 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c9c0a5760 con 0x7f4ca41614c0 2024-04-25T14:13:18.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.251+0000 7f4ca2a00640 1 -- 172.21.15.86:0/2215519795 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c0a5b10 con 0x7f4ca41614c0 2024-04-25T14:13:18.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.255+0000 7f4ca9800640 1 -- 172.21.15.86:0/2215519795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ca41614c0 msgr2=0x7f4ca41618d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:18.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.255+0000 7f4ca9800640 1 --2- 172.21.15.86:0/2215519795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ca41614c0 0x7f4ca41618d0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c009d90 tx=0x7f4c9c01c8a0 comp rx=0 tx=0).stop 2024-04-25T14:13:18.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.255+0000 7f4ca9800640 1 -- 172.21.15.86:0/2215519795 shutdown_connections 2024-04-25T14:13:18.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.255+0000 7f4ca9800640 1 --2- 172.21.15.86:0/2215519795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ca41614c0 0x7f4ca41618d0 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:18.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.255+0000 7f4ca9800640 1 --2- 172.21.15.86:0/2215519795 >> v2:172.21.15.155:3300/0 conn(0x7f4ca40722a0 0x7f4ca40726b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:18.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.255+0000 7f4ca9800640 1 -- 172.21.15.86:0/2215519795 >> 172.21.15.86:0/2215519795 conn(0x7f4ca40b4f80 msgr2=0x7f4ca40b5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:18.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.255+0000 7f4ca9800640 1 -- 172.21.15.86:0/2215519795 shutdown_connections 2024-04-25T14:13:18.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.255+0000 7f4ca9800640 1 -- 172.21.15.86:0/2215519795 wait complete. 2024-04-25T14:13:18.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.255+0000 7f4ca9800640 1 Processor -- start 2024-04-25T14:13:18.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.255+0000 7f4ca9800640 1 -- start start 2024-04-25T14:13:18.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.259+0000 7f4ca9800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ca40722a0 0x7f4ca415fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:18.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.259+0000 7f4ca9800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ca40b6ee0 0x7f4ca41409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:18.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.259+0000 7f4ca9800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ca41614c0 0x7f4ca415c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:18.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.259+0000 7f4ca9800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca40cfd40 con 0x7f4ca41614c0 2024-04-25T14:13:18.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.259+0000 7f4ca9800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca4162220 con 0x7f4ca40b6ee0 2024-04-25T14:13:18.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.259+0000 7f4ca9800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca40cfec0 con 0x7f4ca40722a0 2024-04-25T14:13:18.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.259+0000 7f4ca8e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ca41614c0 0x7f4ca415c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:18.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.259+0000 7f4ca8e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ca41614c0 0x7f4ca415c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46402/0 (socket says 172.21.15.86:46402) 2024-04-25T14:13:18.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.259+0000 7f4ca8e00640 1 -- 172.21.15.86:0/2092498939 learned_addr learned my addr 172.21.15.86:0/2092498939 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:18.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.259+0000 7f4ca3e00640 1 --2- 172.21.15.86:0/2092498939 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ca40722a0 0x7f4ca415fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:18.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.259+0000 7f4ca3400640 1 --2- 172.21.15.86:0/2092498939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ca40b6ee0 0x7f4ca41409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:18.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.259+0000 7f4ca8e00640 1 -- 172.21.15.86:0/2092498939 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ca40722a0 msgr2=0x7f4ca415fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:18.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.259+0000 7f4ca8e00640 1 --2- 172.21.15.86:0/2092498939 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ca40722a0 0x7f4ca415fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:18.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.259+0000 7f4ca8e00640 1 -- 172.21.15.86:0/2092498939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ca40b6ee0 msgr2=0x7f4ca41409f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:18.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.259+0000 7f4ca8e00640 1 --2- 172.21.15.86:0/2092498939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ca40b6ee0 0x7f4ca41409f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:18.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.259+0000 7f4ca8e00640 1 -- 172.21.15.86:0/2092498939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca41605e0 con 0x7f4ca41614c0 2024-04-25T14:13:18.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.259+0000 7f4ca8e00640 1 --2- 172.21.15.86:0/2092498939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ca41614c0 0x7f4ca415c040 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c01a4d0 tx=0x7f4c9c019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:18.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.259+0000 7f4c9be00640 1 -- 172.21.15.86:0/2092498939 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c0b2040 con 0x7f4ca41614c0 2024-04-25T14:13:18.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.259+0000 7f4c9be00640 1 -- 172.21.15.86:0/2092498939 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c9c0bf070 con 0x7f4ca41614c0 2024-04-25T14:13:18.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.259+0000 7f4c9be00640 1 -- 172.21.15.86:0/2092498939 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c009450 con 0x7f4ca41614c0 2024-04-25T14:13:18.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.259+0000 7f4ca9800640 1 -- 172.21.15.86:0/2092498939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca41608c0 con 0x7f4ca41614c0 2024-04-25T14:13:18.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.259+0000 7f4ca9800640 1 -- 172.21.15.86:0/2092498939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca4142e20 con 0x7f4ca41614c0 2024-04-25T14:13:18.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.259+0000 7f4ca9800640 1 -- 172.21.15.86:0/2092498939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ca4076b80 con 0x7f4ca41614c0 2024-04-25T14:13:18.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.259+0000 7f4c9be00640 1 -- 172.21.15.86:0/2092498939 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 75) ==== 92918+0+0 (secure 0 0 0) 0x7f4c9c0b8690 con 0x7f4ca41614c0 2024-04-25T14:13:18.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.263+0000 7f4c9be00640 1 -- 172.21.15.86:0/2092498939 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6876+0+0 (secure 0 0 0) 0x7f4c9c129ff0 con 0x7f4ca41614c0 2024-04-25T14:13:18.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.267+0000 7f4c9be00640 1 -- 172.21.15.86:0/2092498939 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4c9c016520 con 0x7f4ca41614c0 2024-04-25T14:13:18.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.455+0000 7f4ca9800640 1 -- 172.21.15.86:0/2092498939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ca4160de0 con 0x7f4ca41614c0 2024-04-25T14:13:18.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.455+0000 7f4c9be00640 1 -- 172.21.15.86:0/2092498939 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+5208 (secure 0 0 0) 0x7f4c9c0f6880 con 0x7f4ca41614c0 2024-04-25T14:13:18.453 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:13:18.453 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":107,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/1828317917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1828317917},{"type":"v1","addr":"172.21.15.155:6834","nonce":1828317917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":9124,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1389714390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1389714390},{"type":"v1","addr":"172.21.15.86:6837","nonce":1389714390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:11:32.981538+0000","modified":"2024-04-25T14:12:08.255286+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":9546,"mds_1":9136},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9136":{"gid":9136,"name":"d","rank":1,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/91793677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":91793677},{"type":"v1","addr":"172.21.15.155:6835","nonce":91793677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9546":{"gid":9546,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/2571655765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2571655765},{"type":"v1","addr":"172.21.15.86:6835","nonce":2571655765}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9546,"qdb_cluster":[9546,9136]},"id":10}]} 2024-04-25T14:13:18.453 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 107 2024-04-25T14:13:18.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.455+0000 7f4ca9800640 1 -- 172.21.15.86:0/2092498939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ca41614c0 msgr2=0x7f4ca415c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:18.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.455+0000 7f4ca9800640 1 --2- 172.21.15.86:0/2092498939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ca41614c0 0x7f4ca415c040 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c01a4d0 tx=0x7f4c9c019640 comp rx=0 tx=0).stop 2024-04-25T14:13:18.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.455+0000 7f4ca9800640 1 -- 172.21.15.86:0/2092498939 shutdown_connections 2024-04-25T14:13:18.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.455+0000 7f4ca9800640 1 --2- 172.21.15.86:0/2092498939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ca41614c0 0x7f4ca415c040 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:18.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.455+0000 7f4ca9800640 1 --2- 172.21.15.86:0/2092498939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ca40b6ee0 0x7f4ca41409f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:18.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.455+0000 7f4ca9800640 1 --2- 172.21.15.86:0/2092498939 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ca40722a0 0x7f4ca415fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:18.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.455+0000 7f4ca9800640 1 -- 172.21.15.86:0/2092498939 >> 172.21.15.86:0/2092498939 conn(0x7f4ca40b4f80 msgr2=0x7f4ca40b8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:18.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.455+0000 7f4ca9800640 1 -- 172.21.15.86:0/2092498939 shutdown_connections 2024-04-25T14:13:18.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.455+0000 7f4ca9800640 1 -- 172.21.15.86:0/2092498939 wait complete. 2024-04-25T14:13:18.474 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:13:18.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.667+0000 7f5bbae00640 1 Processor -- start 2024-04-25T14:13:18.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.667+0000 7f5bbae00640 1 -- start start 2024-04-25T14:13:18.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.667+0000 7f5bbae00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5bb4071530 0x7f5bb406e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:18.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.667+0000 7f5bbae00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bb406ef90 con 0x7f5bb40cf970 2024-04-25T14:13:18.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.667+0000 7f5bbae00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bb4163a80 con 0x7f5bb406e7a0 2024-04-25T14:13:18.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.667+0000 7f5bbae00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5bb4163830 con 0x7f5bb4071530 2024-04-25T14:13:18.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.667+0000 7f5bb3e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5bb40cf970 0x7f5bb4070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45520/0 (socket says 172.21.15.86:45520) 2024-04-25T14:13:18.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.667+0000 7f5bb3e00640 1 -- 172.21.15.86:0/1102209491 learned_addr learned my addr 172.21.15.86:0/1102209491 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:18.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.667+0000 7f5bab400640 1 --2- 172.21.15.86:0/1102209491 >> v2:172.21.15.86:3300/0 conn(0x7f5bb4071530 0x7f5bb406e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:18.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.667+0000 7f5bab400640 1 -- 172.21.15.86:0/1102209491 >> v1:172.21.15.155:6789/0 conn(0x7f5bb406e7a0 legacy=0x7f5bb40cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:18.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.667+0000 7f5bab400640 1 -- 172.21.15.86:0/1102209491 >> v1:172.21.15.86:6789/0 conn(0x7f5bb40cf970 legacy=0x7f5bb4070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:18.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.667+0000 7f5bab400640 1 -- 172.21.15.86:0/1102209491 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb4163d60 con 0x7f5bb4071530 2024-04-25T14:13:18.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.667+0000 7f5bb3400640 1 -- 172.21.15.86:0/1102209491 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 686737641 0 0) 0x7f5bb406ef90 con 0x7f5bb40cf970 2024-04-25T14:13:18.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.671+0000 7f5bab400640 1 --2- 172.21.15.86:0/1102209491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bb4071530 0x7f5bb406e260 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40097a0 tx=0x7f5ba401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c315b05c7ea61b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:18.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.671+0000 7f5bb3400640 1 -- 172.21.15.86:0/1102209491 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba40a7070 con 0x7f5bb4071530 2024-04-25T14:13:18.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.671+0000 7f5bb3400640 1 -- 172.21.15.86:0/1102209491 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ba40b0070 con 0x7f5bb4071530 2024-04-25T14:13:18.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.671+0000 7f5bb3400640 1 -- 172.21.15.86:0/1102209491 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba40ab890 con 0x7f5bb4071530 2024-04-25T14:13:18.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.671+0000 7f5bbae00640 1 -- 172.21.15.86:0/1102209491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bb4071530 msgr2=0x7f5bb406e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:18.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.671+0000 7f5bbae00640 1 --2- 172.21.15.86:0/1102209491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bb4071530 0x7f5bb406e260 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40097a0 tx=0x7f5ba401c450 comp rx=0 tx=0).stop 2024-04-25T14:13:18.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.671+0000 7f5bbae00640 1 -- 172.21.15.86:0/1102209491 shutdown_connections 2024-04-25T14:13:18.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.671+0000 7f5bbae00640 1 --2- 172.21.15.86:0/1102209491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bb4071530 0x7f5bb406e260 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:18.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.671+0000 7f5bbae00640 1 -- 172.21.15.86:0/1102209491 >> 172.21.15.86:0/1102209491 conn(0x7f5bb40cffc0 msgr2=0x7f5bb40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:18.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.671+0000 7f5bbae00640 1 -- 172.21.15.86:0/1102209491 shutdown_connections 2024-04-25T14:13:18.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.671+0000 7f5bbae00640 1 -- 172.21.15.86:0/1102209491 wait complete. 2024-04-25T14:13:18.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.675+0000 7f5bbae00640 1 Processor -- start 2024-04-25T14:13:18.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.675+0000 7f5bbae00640 1 -- start start 2024-04-25T14:13:18.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.675+0000 7f5bbae00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bb406e7a0 0x7f5bb40b8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:18.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.675+0000 7f5bbae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bb40cf970 0x7f5bb415be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:18.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.675+0000 7f5bbae00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bb4154ce0 0x7f5bb4155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:18.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.675+0000 7f5bbae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb416df70 con 0x7f5bb40cf970 2024-04-25T14:13:18.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.675+0000 7f5bbae00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb416ddf0 con 0x7f5bb4154ce0 2024-04-25T14:13:18.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.675+0000 7f5bbae00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5bb416e0f0 con 0x7f5bb406e7a0 2024-04-25T14:13:18.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.675+0000 7f5bab400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bb40cf970 0x7f5bb415be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:18.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.675+0000 7f5bab400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bb40cf970 0x7f5bb415be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46422/0 (socket says 172.21.15.86:46422) 2024-04-25T14:13:18.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.679+0000 7f5bab400640 1 -- 172.21.15.86:0/2085725209 learned_addr learned my addr 172.21.15.86:0/2085725209 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:18.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.679+0000 7f5bb3e00640 1 --2- 172.21.15.86:0/2085725209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bb406e7a0 0x7f5bb40b8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:18.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.679+0000 7f5bb8e00640 1 --2- 172.21.15.86:0/2085725209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bb4154ce0 0x7f5bb4155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:18.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.679+0000 7f5bab400640 1 -- 172.21.15.86:0/2085725209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bb406e7a0 msgr2=0x7f5bb40b8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:18.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.679+0000 7f5bab400640 1 --2- 172.21.15.86:0/2085725209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bb406e7a0 0x7f5bb40b8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:18.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.679+0000 7f5bab400640 1 -- 172.21.15.86:0/2085725209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bb4154ce0 msgr2=0x7f5bb4155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:18.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.679+0000 7f5bab400640 1 --2- 172.21.15.86:0/2085725209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bb4154ce0 0x7f5bb4155580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.679+0000 7f5bab400640 1 -- 172.21.15.86:0/2085725209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb415adc0 con 0x7f5bb40cf970 2024-04-25T14:13:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.679+0000 7f5bab400640 1 --2- 172.21.15.86:0/2085725209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bb40cf970 0x7f5bb415be60 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40ad040 tx=0x7f5ba401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.679+0000 7f5bb0c00640 1 -- 172.21.15.86:0/2085725209 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4005ba0 con 0x7f5bb40cf970 2024-04-25T14:13:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.679+0000 7f5bb0c00640 1 -- 172.21.15.86:0/2085725209 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ba4005d40 con 0x7f5bb40cf970 2024-04-25T14:13:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.679+0000 7f5bbae00640 1 -- 172.21.15.86:0/2085725209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb415b0a0 con 0x7f5bb40cf970 2024-04-25T14:13:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.679+0000 7f5bb0c00640 1 -- 172.21.15.86:0/2085725209 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4006260 con 0x7f5bb40cf970 2024-04-25T14:13:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.679+0000 7f5bbae00640 1 -- 172.21.15.86:0/2085725209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb415b660 con 0x7f5bb40cf970 2024-04-25T14:13:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.679+0000 7f5bbae00640 1 -- 172.21.15.86:0/2085725209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bb415ab10 con 0x7f5bb40cf970 2024-04-25T14:13:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.679+0000 7f5bb0c00640 1 -- 172.21.15.86:0/2085725209 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 75) ==== 92918+0+0 (secure 0 0 0) 0x7f5ba40bd030 con 0x7f5bb40cf970 2024-04-25T14:13:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.683+0000 7f5bb0c00640 1 -- 172.21.15.86:0/2085725209 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6876+0+0 (secure 0 0 0) 0x7f5ba4129370 con 0x7f5bb40cf970 2024-04-25T14:13:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.687+0000 7f5bb0c00640 1 -- 172.21.15.86:0/2085725209 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5ba4016070 con 0x7f5bb40cf970 2024-04-25T14:13:18.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.843+0000 7f5bbae00640 1 -- 172.21.15.86:0/2085725209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5bb40720e0 con 0x7f5bb40cf970 2024-04-25T14:13:18.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.843+0000 7f5bb0c00640 1 -- 172.21.15.86:0/2085725209 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+18036 (secure 0 0 0) 0x7f5ba40fe1b0 con 0x7f5bb40cf970 2024-04-25T14:13:18.843 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:13:18.843 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":144,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:13:18.050429+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:11:30.810321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T14:11:31.944888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.e","mappings":[{"from":3,"to":1}]},{"pgid":"22.32","mappings":[{"from":3,"to":0}]},{"pgid":"22.34","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/3190429862":"2024-04-26T14:13:18.050389+0000","172.21.15.86:0/3060165022":"2024-04-26T14:13:18.050389+0000","172.21.15.86:6832/1173654665":"2024-04-26T14:13:18.050389+0000","172.21.15.86:0/2751178284":"2024-04-26T14:13:18.050389+0000","172.21.15.86:6833/1173654665":"2024-04-26T14:13:18.050389+0000","172.21.15.86:0/4083287014":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/1048620009":"2024-04-26T14:13:18.050389+0000","172.21.15.86:0/746190999":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/1426015871":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/3750847077":"2024-04-26T14:13:18.050389+0000","172.21.15.86:0/3962343281":"2024-04-26T14:11:51.781677+0000","172.21.15.86:6833/264890479":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/3349072784":"2024-04-26T14:13:18.050389+0000","172.21.15.86:6832/264890479":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/2724786983":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/285240843":"2024-04-26T14:11:51.781677+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:13:18.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.847+0000 7f5bbae00640 1 -- 172.21.15.86:0/2085725209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bb40cf970 msgr2=0x7f5bb415be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:18.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.847+0000 7f5bbae00640 1 --2- 172.21.15.86:0/2085725209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bb40cf970 0x7f5bb415be60 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40ad040 tx=0x7f5ba401ac90 comp rx=0 tx=0).stop 2024-04-25T14:13:18.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.847+0000 7f5bbae00640 1 -- 172.21.15.86:0/2085725209 shutdown_connections 2024-04-25T14:13:18.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.847+0000 7f5bbae00640 1 --2- 172.21.15.86:0/2085725209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bb4154ce0 0x7f5bb4155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:18.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.847+0000 7f5bbae00640 1 --2- 172.21.15.86:0/2085725209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bb40cf970 0x7f5bb415be60 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:18.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.847+0000 7f5bbae00640 1 --2- 172.21.15.86:0/2085725209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bb406e7a0 0x7f5bb40b8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:18.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.847+0000 7f5bbae00640 1 -- 172.21.15.86:0/2085725209 >> 172.21.15.86:0/2085725209 conn(0x7f5bb40cffc0 msgr2=0x7f5bb40b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:18.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.847+0000 7f5bbae00640 1 -- 172.21.15.86:0/2085725209 shutdown_connections 2024-04-25T14:13:18.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:18.847+0000 7f5bbae00640 1 -- 172.21.15.86:0/2085725209 wait complete. 2024-04-25T14:13:18.865 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:11:30.810321+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:11:31.944888+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:13:18.865 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:13:19.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.015+0000 7fdbfda00640 1 Processor -- start 2024-04-25T14:13:19.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.015+0000 7fdbfda00640 1 -- start start 2024-04-25T14:13:19.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.019+0000 7fdbfda00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbf80d16e0 0x7fdbf80d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:19.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.019+0000 7fdbfda00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdbf80730b0 0x7fdbf80734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:19.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.019+0000 7fdbfda00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbf80d2030 con 0x7fdbf80cfca0 2024-04-25T14:13:19.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.019+0000 7fdbfda00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdbf8070470 con 0x7fdbf80730b0 2024-04-25T14:13:19.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.019+0000 7fdbfda00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdbf8072bd0 con 0x7fdbf80d16e0 2024-04-25T14:13:19.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.019+0000 7fdbf7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdbf80730b0 0x7fdbf80734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:19.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.019+0000 7fdbf7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdbf80730b0 0x7fdbf80734c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41210/0 (socket says 172.21.15.86:41210) 2024-04-25T14:13:19.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.019+0000 7fdbf7400640 1 -- 172.21.15.86:0/1170756365 learned_addr learned my addr 172.21.15.86:0/1170756365 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:19.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.019+0000 7fdbee000640 1 --2- 172.21.15.86:0/1170756365 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbf80d16e0 0x7fdbf80d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:19.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.019+0000 7fdbf7400640 1 -- 172.21.15.86:0/1170756365 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbf80d16e0 msgr2=0x7fdbf80d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:19.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.019+0000 7fdbf7400640 1 --2- 172.21.15.86:0/1170756365 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbf80d16e0 0x7fdbf80d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.019+0000 7fdbf7400640 1 -- 172.21.15.86:0/1170756365 >> v1:172.21.15.155:6789/0 conn(0x7fdbf80cfca0 legacy=0x7fdbf806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:19.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.019+0000 7fdbf7400640 1 -- 172.21.15.86:0/1170756365 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf806fbd0 con 0x7fdbf80730b0 2024-04-25T14:13:19.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.023+0000 7fdbf7400640 1 --2- 172.21.15.86:0/1170756365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80730b0 0x7fdbf80734c0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0009e50 tx=0x7fdbf001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec56669c877f834c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:19.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.023+0000 7fdbf6000640 1 -- 172.21.15.86:0/1170756365 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf00a5810 con 0x7fdbf80730b0 2024-04-25T14:13:19.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.023+0000 7fdbf6000640 1 -- 172.21.15.86:0/1170756365 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbf00a59b0 con 0x7fdbf80730b0 2024-04-25T14:13:19.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.023+0000 7fdbf6000640 1 -- 172.21.15.86:0/1170756365 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf00a5d80 con 0x7fdbf80730b0 2024-04-25T14:13:19.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.023+0000 7fdbfda00640 1 -- 172.21.15.86:0/1170756365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80730b0 msgr2=0x7fdbf80734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:19.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.023+0000 7fdbfda00640 1 --2- 172.21.15.86:0/1170756365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80730b0 0x7fdbf80734c0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0009e50 tx=0x7fdbf001ca30 comp rx=0 tx=0).stop 2024-04-25T14:13:19.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.023+0000 7fdbfda00640 1 -- 172.21.15.86:0/1170756365 shutdown_connections 2024-04-25T14:13:19.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.023+0000 7fdbfda00640 1 --2- 172.21.15.86:0/1170756365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80730b0 0x7fdbf80734c0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.023+0000 7fdbfda00640 1 --2- 172.21.15.86:0/1170756365 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbf80d16e0 0x7fdbf80d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.023+0000 7fdbfda00640 1 -- 172.21.15.86:0/1170756365 >> 172.21.15.86:0/1170756365 conn(0x7fdbf80b4650 msgr2=0x7fdbf80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:19.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.023+0000 7fdbfda00640 1 -- 172.21.15.86:0/1170756365 shutdown_connections 2024-04-25T14:13:19.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.023+0000 7fdbfda00640 1 -- 172.21.15.86:0/1170756365 wait complete. 2024-04-25T14:13:19.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.023+0000 7fdbfda00640 1 Processor -- start 2024-04-25T14:13:19.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.023+0000 7fdbfda00640 1 -- start start 2024-04-25T14:13:19.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.027+0000 7fdbfda00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbf80730b0 0x7fdbf809a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.027+0000 7fdbfda00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80cfca0 0x7fdbf80cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.027+0000 7fdbfda00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbf80d16e0 0x7fdbf80cc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.027+0000 7fdbfda00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf8169a70 con 0x7fdbf80cfca0 2024-04-25T14:13:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.027+0000 7fdbfda00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf81698f0 con 0x7fdbf80d16e0 2024-04-25T14:13:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.027+0000 7fdbfda00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdbf8169bf0 con 0x7fdbf80730b0 2024-04-25T14:13:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.027+0000 7fdbee000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80cfca0 0x7fdbf80cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.027+0000 7fdbee000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80cfca0 0x7fdbf80cbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41224/0 (socket says 172.21.15.86:41224) 2024-04-25T14:13:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.027+0000 7fdbee000640 1 -- 172.21.15.86:0/163321252 learned_addr learned my addr 172.21.15.86:0/163321252 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.027+0000 7fdbf7400640 1 --2- 172.21.15.86:0/163321252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbf80d16e0 0x7fdbf80cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.027+0000 7fdbf6a00640 1 --2- 172.21.15.86:0/163321252 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbf80730b0 0x7fdbf809a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.027+0000 7fdbee000640 1 -- 172.21.15.86:0/163321252 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbf80730b0 msgr2=0x7fdbf809a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.027+0000 7fdbee000640 1 --2- 172.21.15.86:0/163321252 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbf80730b0 0x7fdbf809a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.027+0000 7fdbee000640 1 -- 172.21.15.86:0/163321252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbf80d16e0 msgr2=0x7fdbf80cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:19.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.027+0000 7fdbee000640 1 --2- 172.21.15.86:0/163321252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbf80d16e0 0x7fdbf80cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.027+0000 7fdbee000640 1 -- 172.21.15.86:0/163321252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf80ccd00 con 0x7fdbf80cfca0 2024-04-25T14:13:19.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.027+0000 7fdbf7400640 1 --2- 172.21.15.86:0/163321252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbf80d16e0 0x7fdbf80cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:19.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.027+0000 7fdbee000640 1 --2- 172.21.15.86:0/163321252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80cfca0 0x7fdbf80cbf40 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000d620 tx=0x7fdbe000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:19.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.027+0000 7fdbef400640 1 -- 172.21.15.86:0/163321252 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe00125f0 con 0x7fdbf80cfca0 2024-04-25T14:13:19.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.027+0000 7fdbef400640 1 -- 172.21.15.86:0/163321252 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbe0019070 con 0x7fdbf80cfca0 2024-04-25T14:13:19.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.027+0000 7fdbfda00640 1 -- 172.21.15.86:0/163321252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf81574a0 con 0x7fdbf80cfca0 2024-04-25T14:13:19.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.027+0000 7fdbef400640 1 -- 172.21.15.86:0/163321252 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe00129b0 con 0x7fdbf80cfca0 2024-04-25T14:13:19.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.027+0000 7fdbfda00640 1 -- 172.21.15.86:0/163321252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf8157a60 con 0x7fdbf80cfca0 2024-04-25T14:13:19.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.027+0000 7fdbfda00640 1 -- 172.21.15.86:0/163321252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf809f4b0 con 0x7fdbf80cfca0 2024-04-25T14:13:19.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.031+0000 7fdbef400640 1 -- 172.21.15.86:0/163321252 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 75) ==== 92918+0+0 (secure 0 0 0) 0x7fdbe0012b50 con 0x7fdbf80cfca0 2024-04-25T14:13:19.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.031+0000 7fdbef400640 1 -- 172.21.15.86:0/163321252 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6876+0+0 (secure 0 0 0) 0x7fdbe008f6e0 con 0x7fdbf80cfca0 2024-04-25T14:13:19.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.035+0000 7fdbef400640 1 -- 172.21.15.86:0/163321252 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdbf80ccd00 con 0x7fdbf80cfca0 2024-04-25T14:13:19.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.059+0000 7fdbef400640 1 -- 172.21.15.86:0/163321252 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 76) ==== 93045+0+0 (secure 0 0 0) 0x7fdbe0057710 con 0x7fdbf80cfca0 2024-04-25T14:13:19.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.059+0000 7fdbef400640 1 --2- 172.21.15.86:0/163321252 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fdbd4075370 0x7fdbd4077790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:19.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.059+0000 7fdbf6a00640 1 --2- 172.21.15.86:0/163321252 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fdbd4075370 0x7fdbd4077790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:19.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.063+0000 7fdbf6a00640 1 --2- 172.21.15.86:0/163321252 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fdbd4075370 0x7fdbd4077790 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8070710 tx=0x7fdbe8009040 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:19.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.203+0000 7fdbfda00640 1 -- 172.21.15.86:0/163321252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdbf8069d00 con 0x7fdbf80cfca0 2024-04-25T14:13:19.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.203+0000 7fdbef400640 1 -- 172.21.15.86:0/163321252 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v144) ==== 93+0+10 (secure 0 0 0) 0x7fdbe005c4a0 con 0x7fdbf80cfca0 2024-04-25T14:13:19.203 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:13:19.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.207+0000 7fdbfda00640 1 -- 172.21.15.86:0/163321252 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fdbd4075370 msgr2=0x7fdbd4077790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:19.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.207+0000 7fdbfda00640 1 --2- 172.21.15.86:0/163321252 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fdbd4075370 0x7fdbd4077790 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8070710 tx=0x7fdbe8009040 comp rx=0 tx=0).stop 2024-04-25T14:13:19.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.207+0000 7fdbfda00640 1 -- 172.21.15.86:0/163321252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80cfca0 msgr2=0x7fdbf80cbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:19.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.207+0000 7fdbfda00640 1 --2- 172.21.15.86:0/163321252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80cfca0 0x7fdbf80cbf40 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000d620 tx=0x7fdbe000dae0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.207+0000 7fdbfda00640 1 -- 172.21.15.86:0/163321252 shutdown_connections 2024-04-25T14:13:19.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.207+0000 7fdbfda00640 1 --2- 172.21.15.86:0/163321252 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fdbd4075370 0x7fdbd4077790 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.207+0000 7fdbfda00640 1 --2- 172.21.15.86:0/163321252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbf80d16e0 0x7fdbf80cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.207+0000 7fdbfda00640 1 --2- 172.21.15.86:0/163321252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80cfca0 0x7fdbf80cbf40 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.207+0000 7fdbfda00640 1 --2- 172.21.15.86:0/163321252 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbf80730b0 0x7fdbf809a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.207+0000 7fdbfda00640 1 -- 172.21.15.86:0/163321252 >> 172.21.15.86:0/163321252 conn(0x7fdbf80b4650 msgr2=0x7fdbf81564b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:19.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.207+0000 7fdbfda00640 1 -- 172.21.15.86:0/163321252 shutdown_connections 2024-04-25T14:13:19.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.207+0000 7fdbfda00640 1 -- 172.21.15.86:0/163321252 wait complete. 2024-04-25T14:13:19.224 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:13:19.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.375+0000 7fca1ce00640 1 Processor -- start 2024-04-25T14:13:19.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.375+0000 7fca1ce00640 1 -- start start 2024-04-25T14:13:19.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.375+0000 7fca1ce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fca18070330 0x7fca180d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:19.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.375+0000 7fca1ce00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fca180ce790 0x7fca1806f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:19.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.375+0000 7fca1ce00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca180d2030 con 0x7fca180d1c20 2024-04-25T14:13:19.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.375+0000 7fca1ce00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fca1815e5c0 con 0x7fca18070330 2024-04-25T14:13:19.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.375+0000 7fca1ce00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fca1815e8a0 con 0x7fca180ce790 2024-04-25T14:13:19.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.379+0000 7fca15e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fca18070330 0x7fca180d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:19.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.379+0000 7fca15e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fca18070330 0x7fca180d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41230/0 (socket says 172.21.15.86:41230) 2024-04-25T14:13:19.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.379+0000 7fca15e00640 1 -- 172.21.15.86:0/4063605133 learned_addr learned my addr 172.21.15.86:0/4063605133 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:19.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.379+0000 7fca16800640 1 --2- 172.21.15.86:0/4063605133 >> v2:172.21.15.155:3300/0 conn(0x7fca180ce790 0x7fca1806f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:19.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.379+0000 7fca15e00640 1 -- 172.21.15.86:0/4063605133 >> v2:172.21.15.155:3300/0 conn(0x7fca180ce790 msgr2=0x7fca1806f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:19.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.379+0000 7fca15e00640 1 --2- 172.21.15.86:0/4063605133 >> v2:172.21.15.155:3300/0 conn(0x7fca180ce790 0x7fca1806f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.379+0000 7fca15e00640 1 -- 172.21.15.86:0/4063605133 >> v1:172.21.15.155:6789/0 conn(0x7fca180d1c20 legacy=0x7fca1806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:19.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.379+0000 7fca15e00640 1 -- 172.21.15.86:0/4063605133 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca1815eb80 con 0x7fca18070330 2024-04-25T14:13:19.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.379+0000 7fca15e00640 1 --2- 172.21.15.86:0/4063605133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca18070330 0x7fca180d16e0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fca08009e50 tx=0x7fca0801c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487028f5556286b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:19.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.379+0000 7fca14a00640 1 -- 172.21.15.86:0/4063605133 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca080ac020 con 0x7fca18070330 2024-04-25T14:13:19.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.379+0000 7fca14a00640 1 -- 172.21.15.86:0/4063605133 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca080a6410 con 0x7fca18070330 2024-04-25T14:13:19.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.379+0000 7fca14a00640 1 -- 172.21.15.86:0/4063605133 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca080a67e0 con 0x7fca18070330 2024-04-25T14:13:19.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.383+0000 7fca1ce00640 1 -- 172.21.15.86:0/4063605133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca18070330 msgr2=0x7fca180d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:19.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.383+0000 7fca1ce00640 1 --2- 172.21.15.86:0/4063605133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca18070330 0x7fca180d16e0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fca08009e50 tx=0x7fca0801c740 comp rx=0 tx=0).stop 2024-04-25T14:13:19.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.383+0000 7fca1ce00640 1 -- 172.21.15.86:0/4063605133 shutdown_connections 2024-04-25T14:13:19.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.383+0000 7fca1ce00640 1 --2- 172.21.15.86:0/4063605133 >> v2:172.21.15.155:3300/0 conn(0x7fca180ce790 0x7fca1806f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.383+0000 7fca1ce00640 1 --2- 172.21.15.86:0/4063605133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca18070330 0x7fca180d16e0 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.383+0000 7fca1ce00640 1 -- 172.21.15.86:0/4063605133 >> 172.21.15.86:0/4063605133 conn(0x7fca180b4650 msgr2=0x7fca180b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:19.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.383+0000 7fca1ce00640 1 -- 172.21.15.86:0/4063605133 shutdown_connections 2024-04-25T14:13:19.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.383+0000 7fca1ce00640 1 -- 172.21.15.86:0/4063605133 wait complete. 2024-04-25T14:13:19.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.383+0000 7fca1ce00640 1 Processor -- start 2024-04-25T14:13:19.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.383+0000 7fca1ce00640 1 -- start start 2024-04-25T14:13:19.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.387+0000 7fca1ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca18070330 0x7fca1809a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:19.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.387+0000 7fca1ce00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca180ce790 0x7fca180996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:19.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.387+0000 7fca1ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca180d1c20 0x7fca181556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:19.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.387+0000 7fca1ce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca180708c0 con 0x7fca180d1c20 2024-04-25T14:13:19.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.387+0000 7fca1ce00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fca18070740 con 0x7fca18070330 2024-04-25T14:13:19.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.387+0000 7fca1ce00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fca1816afc0 con 0x7fca180ce790 2024-04-25T14:13:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.387+0000 7fca16800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca180d1c20 0x7fca181556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.387+0000 7fca16800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca180d1c20 0x7fca181556b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41232/0 (socket says 172.21.15.86:41232) 2024-04-25T14:13:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.387+0000 7fca16800640 1 -- 172.21.15.86:0/3279293183 learned_addr learned my addr 172.21.15.86:0/3279293183 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.387+0000 7fca15e00640 1 --2- 172.21.15.86:0/3279293183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca18070330 0x7fca1809a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.387+0000 7fca15400640 1 --2- 172.21.15.86:0/3279293183 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca180ce790 0x7fca180996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.387+0000 7fca16800640 1 -- 172.21.15.86:0/3279293183 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca180ce790 msgr2=0x7fca180996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.387+0000 7fca16800640 1 --2- 172.21.15.86:0/3279293183 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca180ce790 0x7fca180996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.387+0000 7fca16800640 1 -- 172.21.15.86:0/3279293183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca18070330 msgr2=0x7fca1809a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.387+0000 7fca16800640 1 --2- 172.21.15.86:0/3279293183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca18070330 0x7fca1809a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.387+0000 7fca16800640 1 -- 172.21.15.86:0/3279293183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca181548f0 con 0x7fca180d1c20 2024-04-25T14:13:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.387+0000 7fca15400640 1 --2- 172.21.15.86:0/3279293183 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca180ce790 0x7fca180996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.387+0000 7fca16800640 1 --2- 172.21.15.86:0/3279293183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca180d1c20 0x7fca181556b0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fca10006e90 tx=0x7fca1000d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.387+0000 7fca0e000640 1 -- 172.21.15.86:0/3279293183 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca100125f0 con 0x7fca180d1c20 2024-04-25T14:13:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.387+0000 7fca0e000640 1 -- 172.21.15.86:0/3279293183 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca10019070 con 0x7fca180d1c20 2024-04-25T14:13:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.387+0000 7fca1ce00640 1 -- 172.21.15.86:0/3279293183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca18154bd0 con 0x7fca180d1c20 2024-04-25T14:13:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.387+0000 7fca0e000640 1 -- 172.21.15.86:0/3279293183 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca100129d0 con 0x7fca180d1c20 2024-04-25T14:13:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.387+0000 7fca1ce00640 1 -- 172.21.15.86:0/3279293183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca18155190 con 0x7fca180d1c20 2024-04-25T14:13:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.387+0000 7fca1ce00640 1 -- 172.21.15.86:0/3279293183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca18154650 con 0x7fca180d1c20 2024-04-25T14:13:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.387+0000 7fca0e000640 1 -- 172.21.15.86:0/3279293183 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 76) ==== 93045+0+0 (secure 0 0 0) 0x7fca10012b70 con 0x7fca180d1c20 2024-04-25T14:13:19.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.391+0000 7fca0e000640 1 --2- 172.21.15.86:0/3279293183 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fc9f806f280 0x7fc9f8071790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:19.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.391+0000 7fca0e000640 1 -- 172.21.15.86:0/3279293183 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6876+0+0 (secure 0 0 0) 0x7fca1008f700 con 0x7fca180d1c20 2024-04-25T14:13:19.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.391+0000 7fca15e00640 1 --2- 172.21.15.86:0/3279293183 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fc9f806f280 0x7fc9f8071790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:19.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.391+0000 7fca15e00640 1 --2- 172.21.15.86:0/3279293183 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fc9f806f280 0x7fc9f8071790 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fca080ad000 tx=0x7fca080b3040 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:19.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.395+0000 7fca0e000640 1 -- 172.21.15.86:0/3279293183 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fca1005c460 con 0x7fca180d1c20 2024-04-25T14:13:19.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.551+0000 7fca1ce00640 1 -- 172.21.15.86:0/3279293183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fca18069d00 con 0x7fca180d1c20 2024-04-25T14:13:19.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.551+0000 7fca0e000640 1 -- 172.21.15.86:0/3279293183 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v144) ==== 104+0+11 (secure 0 0 0) 0x7fca1005bd60 con 0x7fca180d1c20 2024-04-25T14:13:19.551 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:13:19.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.555+0000 7fca1ce00640 1 -- 172.21.15.86:0/3279293183 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fc9f806f280 msgr2=0x7fc9f8071790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:19.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.555+0000 7fca1ce00640 1 --2- 172.21.15.86:0/3279293183 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fc9f806f280 0x7fc9f8071790 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fca080ad000 tx=0x7fca080b3040 comp rx=0 tx=0).stop 2024-04-25T14:13:19.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.555+0000 7fca1ce00640 1 -- 172.21.15.86:0/3279293183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca180d1c20 msgr2=0x7fca181556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:19.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.555+0000 7fca1ce00640 1 --2- 172.21.15.86:0/3279293183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca180d1c20 0x7fca181556b0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fca10006e90 tx=0x7fca1000d9e0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.555+0000 7fca1ce00640 1 -- 172.21.15.86:0/3279293183 shutdown_connections 2024-04-25T14:13:19.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.555+0000 7fca1ce00640 1 --2- 172.21.15.86:0/3279293183 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fc9f806f280 0x7fc9f8071790 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.555+0000 7fca1ce00640 1 --2- 172.21.15.86:0/3279293183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca180d1c20 0x7fca181556b0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.555+0000 7fca1ce00640 1 --2- 172.21.15.86:0/3279293183 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca180ce790 0x7fca180996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.555+0000 7fca1ce00640 1 --2- 172.21.15.86:0/3279293183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca18070330 0x7fca1809a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.555+0000 7fca1ce00640 1 -- 172.21.15.86:0/3279293183 >> 172.21.15.86:0/3279293183 conn(0x7fca180b4650 msgr2=0x7fca1815a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:19.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.555+0000 7fca1ce00640 1 -- 172.21.15.86:0/3279293183 shutdown_connections 2024-04-25T14:13:19.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.555+0000 7fca1ce00640 1 -- 172.21.15.86:0/3279293183 wait complete. 2024-04-25T14:13:19.572 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:13:19.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.723+0000 7f03a4200640 1 Processor -- start 2024-04-25T14:13:19.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.723+0000 7f03a4200640 1 -- start start 2024-04-25T14:13:19.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.723+0000 7f03a4200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f039c15c2a0 0x7f039c0ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:19.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.723+0000 7f03a4200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f039c15feb0 0x7f039c1602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:19.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.727+0000 7f03a4200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f039c0b0a60 con 0x7f039c06fd50 2024-04-25T14:13:19.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.727+0000 7f03a4200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f039c06ef90 con 0x7f039c15feb0 2024-04-25T14:13:19.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.727+0000 7f03a4200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f039c06f270 con 0x7f039c15c2a0 2024-04-25T14:13:19.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.727+0000 7f03a0e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f039c06fd50 0x7f039c070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47350/0 (socket says 172.21.15.86:47350) 2024-04-25T14:13:19.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.727+0000 7f03a0e00640 1 -- 172.21.15.86:0/118057091 learned_addr learned my addr 172.21.15.86:0/118057091 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:19.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.727+0000 7f03a1800640 1 --2- 172.21.15.86:0/118057091 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f039c15c2a0 0x7f039c0ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:19.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.727+0000 7f03a2200640 1 --2- 172.21.15.86:0/118057091 >> v2:172.21.15.155:3300/0 conn(0x7f039c15feb0 0x7f039c1602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:19.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.727+0000 7f039be00640 1 -- 172.21.15.86:0/118057091 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3432137672 0 0) 0x7f039c0b0a60 con 0x7f039c06fd50 2024-04-25T14:13:19.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.727+0000 7f039be00640 1 -- 172.21.15.86:0/118057091 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0384003530 con 0x7f039c06fd50 2024-04-25T14:13:19.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.727+0000 7f03a2200640 1 -- 172.21.15.86:0/118057091 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f039c15c2a0 msgr2=0x7f039c0ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:19.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.727+0000 7f03a2200640 1 --2- 172.21.15.86:0/118057091 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f039c15c2a0 0x7f039c0ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.727+0000 7f03a2200640 1 -- 172.21.15.86:0/118057091 >> v1:172.21.15.86:6789/0 conn(0x7f039c06fd50 legacy=0x7f039c070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:19.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.727+0000 7f03a2200640 1 -- 172.21.15.86:0/118057091 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039c06f550 con 0x7f039c15feb0 2024-04-25T14:13:19.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.727+0000 7f03a1800640 1 --2- 172.21.15.86:0/118057091 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f039c15c2a0 0x7f039c0ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:13:19.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.731+0000 7f03a2200640 1 --2- 172.21.15.86:0/118057091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f039c15feb0 0x7f039c1602c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f039400aea0 tx=0x7f039401e410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c4dbb139534c579 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:19.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.731+0000 7f039be00640 1 -- 172.21.15.86:0/118057091 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03940a6830 con 0x7f039c15feb0 2024-04-25T14:13:19.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.731+0000 7f039be00640 1 -- 172.21.15.86:0/118057091 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03940a69d0 con 0x7f039c15feb0 2024-04-25T14:13:19.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.731+0000 7f039be00640 1 -- 172.21.15.86:0/118057091 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03940a6da0 con 0x7f039c15feb0 2024-04-25T14:13:19.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.731+0000 7f03a4200640 1 -- 172.21.15.86:0/118057091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f039c15feb0 msgr2=0x7f039c1602c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:19.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.731+0000 7f03a4200640 1 --2- 172.21.15.86:0/118057091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f039c15feb0 0x7f039c1602c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f039400aea0 tx=0x7f039401e410 comp rx=0 tx=0).stop 2024-04-25T14:13:19.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.731+0000 7f03a4200640 1 -- 172.21.15.86:0/118057091 shutdown_connections 2024-04-25T14:13:19.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.731+0000 7f03a4200640 1 --2- 172.21.15.86:0/118057091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f039c15feb0 0x7f039c1602c0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.731+0000 7f03a4200640 1 --2- 172.21.15.86:0/118057091 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f039c15c2a0 0x7f039c0ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.731+0000 7f03a4200640 1 -- 172.21.15.86:0/118057091 >> 172.21.15.86:0/118057091 conn(0x7f039c0b4650 msgr2=0x7f039c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:19.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.731+0000 7f03a4200640 1 -- 172.21.15.86:0/118057091 shutdown_connections 2024-04-25T14:13:19.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.731+0000 7f03a4200640 1 -- 172.21.15.86:0/118057091 wait complete. 2024-04-25T14:13:19.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.731+0000 7f03a4200640 1 Processor -- start 2024-04-25T14:13:19.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.731+0000 7f03a4200640 1 -- start start 2024-04-25T14:13:19.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.735+0000 7f03a4200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f039c06fd50 0x7f039c09ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:19.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.735+0000 7f03a4200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f039c15c2a0 0x7f039c09ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:19.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.735+0000 7f03a4200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f039c15feb0 0x7f039c0b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:19.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.735+0000 7f03a4200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f039c160d90 con 0x7f039c15c2a0 2024-04-25T14:13:19.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.735+0000 7f03a4200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f039c160c10 con 0x7f039c06fd50 2024-04-25T14:13:19.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.735+0000 7f03a4200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f039c0ae670 con 0x7f039c15feb0 2024-04-25T14:13:19.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.735+0000 7f03a0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f039c15c2a0 0x7f039c09ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:19.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.735+0000 7f03a0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f039c15c2a0 0x7f039c09ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41236/0 (socket says 172.21.15.86:41236) 2024-04-25T14:13:19.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.735+0000 7f03a0e00640 1 -- 172.21.15.86:0/3803736239 learned_addr learned my addr 172.21.15.86:0/3803736239 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:19.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.735+0000 7f03a1800640 1 --2- 172.21.15.86:0/3803736239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f039c06fd50 0x7f039c09ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:19.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.735+0000 7f03a2200640 1 --2- 172.21.15.86:0/3803736239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f039c15feb0 0x7f039c0b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:19.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.735+0000 7f03a0e00640 1 -- 172.21.15.86:0/3803736239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f039c15feb0 msgr2=0x7f039c0b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:19.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.735+0000 7f03a0e00640 1 --2- 172.21.15.86:0/3803736239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f039c15feb0 0x7f039c0b0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.735+0000 7f03a0e00640 1 -- 172.21.15.86:0/3803736239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f039c06fd50 msgr2=0x7f039c09ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:19.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.735+0000 7f03a0e00640 1 --2- 172.21.15.86:0/3803736239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f039c06fd50 0x7f039c09ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.735+0000 7f03a0e00640 1 -- 172.21.15.86:0/3803736239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039c0b2ef0 con 0x7f039c15c2a0 2024-04-25T14:13:19.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.735+0000 7f03a0e00640 1 --2- 172.21.15.86:0/3803736239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f039c15c2a0 0x7f039c09ecc0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f038800c3c0 tx=0x7f038800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:19.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.735+0000 7f0399600640 1 -- 172.21.15.86:0/3803736239 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0388017070 con 0x7f039c15c2a0 2024-04-25T14:13:19.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.735+0000 7f0399600640 1 -- 172.21.15.86:0/3803736239 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0388005cf0 con 0x7f039c15c2a0 2024-04-25T14:13:19.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.735+0000 7f03a4200640 1 -- 172.21.15.86:0/3803736239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039c0b31d0 con 0x7f039c15c2a0 2024-04-25T14:13:19.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.735+0000 7f0399600640 1 -- 172.21.15.86:0/3803736239 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03880072f0 con 0x7f039c15c2a0 2024-04-25T14:13:19.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.735+0000 7f03a4200640 1 -- 172.21.15.86:0/3803736239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f039c0b3790 con 0x7f039c15c2a0 2024-04-25T14:13:19.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.735+0000 7f03a4200640 1 -- 172.21.15.86:0/3803736239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f039c069d00 con 0x7f039c15c2a0 2024-04-25T14:13:19.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.739+0000 7f0399600640 1 -- 172.21.15.86:0/3803736239 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 76) ==== 93045+0+0 (secure 0 0 0) 0x7f038801a070 con 0x7f039c15c2a0 2024-04-25T14:13:19.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.739+0000 7f0399600640 1 --2- 172.21.15.86:0/3803736239 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f037c06f250 0x7f037c071760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:19.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.739+0000 7f0399600640 1 -- 172.21.15.86:0/3803736239 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6876+0+0 (secure 0 0 0) 0x7f038808e7d0 con 0x7f039c15c2a0 2024-04-25T14:13:19.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.739+0000 7f03a1800640 1 --2- 172.21.15.86:0/3803736239 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f037c06f250 0x7f037c071760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:19.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.739+0000 7f03a1800640 1 --2- 172.21.15.86:0/3803736239 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f037c06f250 0x7f037c071760 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0390009580 tx=0x7f0390009190 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:19.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.743+0000 7f0399600640 1 -- 172.21.15.86:0/3803736239 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f038805ae50 con 0x7f039c15c2a0 2024-04-25T14:13:19.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.903+0000 7f03a4200640 1 -- 172.21.15.86:0/3803736239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f039c076bf0 con 0x7f039c15c2a0 2024-04-25T14:13:19.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.903+0000 7f0399600640 1 -- 172.21.15.86:0/3803736239 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v144) ==== 100+0+11 (secure 0 0 0) 0x7f038805f150 con 0x7f039c15c2a0 2024-04-25T14:13:19.901 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:13:19.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.903+0000 7f03a4200640 1 -- 172.21.15.86:0/3803736239 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f037c06f250 msgr2=0x7f037c071760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:19.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.903+0000 7f03a4200640 1 --2- 172.21.15.86:0/3803736239 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f037c06f250 0x7f037c071760 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0390009580 tx=0x7f0390009190 comp rx=0 tx=0).stop 2024-04-25T14:13:19.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.903+0000 7f03a4200640 1 -- 172.21.15.86:0/3803736239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f039c15c2a0 msgr2=0x7f039c09ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:19.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.903+0000 7f03a4200640 1 --2- 172.21.15.86:0/3803736239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f039c15c2a0 0x7f039c09ecc0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f038800c3c0 tx=0x7f038800c880 comp rx=0 tx=0).stop 2024-04-25T14:13:19.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.903+0000 7f03a4200640 1 -- 172.21.15.86:0/3803736239 shutdown_connections 2024-04-25T14:13:19.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.903+0000 7f03a4200640 1 --2- 172.21.15.86:0/3803736239 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f037c06f250 0x7f037c071760 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.903+0000 7f03a4200640 1 --2- 172.21.15.86:0/3803736239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f039c15feb0 0x7f039c0b0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.903+0000 7f03a4200640 1 --2- 172.21.15.86:0/3803736239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f039c15c2a0 0x7f039c09ecc0 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.903+0000 7f03a4200640 1 --2- 172.21.15.86:0/3803736239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f039c06fd50 0x7f039c09ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:19.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.903+0000 7f03a4200640 1 -- 172.21.15.86:0/3803736239 >> 172.21.15.86:0/3803736239 conn(0x7f039c0b4650 msgr2=0x7f039c15cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:19.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.903+0000 7f03a4200640 1 -- 172.21.15.86:0/3803736239 shutdown_connections 2024-04-25T14:13:19.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:19.903+0000 7f03a4200640 1 -- 172.21.15.86:0/3803736239 wait complete. 2024-04-25T14:13:19.923 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:13:20.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.071+0000 7f985f200640 1 Processor -- start 2024-04-25T14:13:20.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.071+0000 7f985f200640 1 -- start start 2024-04-25T14:13:20.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.075+0000 7f985f200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f98580d2c80 0x7f985815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:20.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.075+0000 7f985f200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98581616e0 0x7f9858161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:20.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.075+0000 7f985f200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98580cd8d0 con 0x7f9858070e20 2024-04-25T14:13:20.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.075+0000 7f985f200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f98580ce720 con 0x7f98580d2c80 2024-04-25T14:13:20.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.075+0000 7f985f200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f98580ce970 con 0x7f98581616e0 2024-04-25T14:13:20.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.075+0000 7f984f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9858070e20 0x7f9858071230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47354/0 (socket says 172.21.15.86:47354) 2024-04-25T14:13:20.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.075+0000 7f984f400640 1 -- 172.21.15.86:0/473541754 learned_addr learned my addr 172.21.15.86:0/473541754 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:20.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.075+0000 7f985d200640 1 --2- 172.21.15.86:0/473541754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98581616e0 0x7f9858161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:20.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.075+0000 7f9857e00640 1 --2- 172.21.15.86:0/473541754 >> v2:172.21.15.155:3300/0 conn(0x7f98580d2c80 0x7f985815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:20.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.075+0000 7f985d200640 1 -- 172.21.15.86:0/473541754 >> v2:172.21.15.155:3300/0 conn(0x7f98580d2c80 msgr2=0x7f985815dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:20.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.075+0000 7f985d200640 1 --2- 172.21.15.86:0/473541754 >> v2:172.21.15.155:3300/0 conn(0x7f98580d2c80 0x7f985815dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:20.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.075+0000 7f985d200640 1 -- 172.21.15.86:0/473541754 >> v1:172.21.15.86:6789/0 conn(0x7f9858070e20 legacy=0x7f9858071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:20.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.075+0000 7f985d200640 1 -- 172.21.15.86:0/473541754 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98580d1be0 con 0x7f98581616e0 2024-04-25T14:13:20.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.075+0000 7f9857e00640 1 --2- 172.21.15.86:0/473541754 >> v2:172.21.15.155:3300/0 conn(0x7f98580d2c80 0x7f985815dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:13:20.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.079+0000 7f985d200640 1 --2- 172.21.15.86:0/473541754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98581616e0 0x7f9858161af0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9850009d90 tx=0x7f985001f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55c885cc686fb134 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:20.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.083+0000 7f9857400640 1 -- 172.21.15.86:0/473541754 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98500b4070 con 0x7f98581616e0 2024-04-25T14:13:20.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.083+0000 7f9857400640 1 -- 172.21.15.86:0/473541754 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98500af600 con 0x7f98581616e0 2024-04-25T14:13:20.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.083+0000 7f9857400640 1 -- 172.21.15.86:0/473541754 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98500af9d0 con 0x7f98581616e0 2024-04-25T14:13:20.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.083+0000 7f985f200640 1 -- 172.21.15.86:0/473541754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98581616e0 msgr2=0x7f9858161af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:20.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.083+0000 7f985f200640 1 --2- 172.21.15.86:0/473541754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98581616e0 0x7f9858161af0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9850009d90 tx=0x7f985001f150 comp rx=0 tx=0).stop 2024-04-25T14:13:20.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.083+0000 7f985f200640 1 -- 172.21.15.86:0/473541754 shutdown_connections 2024-04-25T14:13:20.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.083+0000 7f985f200640 1 --2- 172.21.15.86:0/473541754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98581616e0 0x7f9858161af0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:20.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.083+0000 7f985f200640 1 --2- 172.21.15.86:0/473541754 >> v2:172.21.15.155:3300/0 conn(0x7f98580d2c80 0x7f985815dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:20.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.083+0000 7f985f200640 1 -- 172.21.15.86:0/473541754 >> 172.21.15.86:0/473541754 conn(0x7f98580cffc0 msgr2=0x7f98580d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:20.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.083+0000 7f985f200640 1 -- 172.21.15.86:0/473541754 shutdown_connections 2024-04-25T14:13:20.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.083+0000 7f985f200640 1 -- 172.21.15.86:0/473541754 wait complete. 2024-04-25T14:13:20.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.087+0000 7f985f200640 1 Processor -- start 2024-04-25T14:13:20.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.087+0000 7f985f200640 1 -- start start 2024-04-25T14:13:20.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.087+0000 7f985f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9858070e20 0x7f98581526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:20.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.087+0000 7f985f200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98580d2c80 0x7f985814cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:20.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.087+0000 7f985f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98581616e0 0x7f985815c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:20.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.087+0000 7f985f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98580d2480 con 0x7f98581616e0 2024-04-25T14:13:20.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.087+0000 7f985f200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f98580d21a0 con 0x7f9858070e20 2024-04-25T14:13:20.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.087+0000 7f985f200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f98580d29a0 con 0x7f98580d2c80 2024-04-25T14:13:20.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.087+0000 7f985d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98581616e0 0x7f985815c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:20.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.087+0000 7f985d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98581616e0 0x7f985815c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41248/0 (socket says 172.21.15.86:41248) 2024-04-25T14:13:20.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.087+0000 7f985d200640 1 -- 172.21.15.86:0/3921186072 learned_addr learned my addr 172.21.15.86:0/3921186072 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:20.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.087+0000 7f9857e00640 1 --2- 172.21.15.86:0/3921186072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9858070e20 0x7f98581526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:20.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.087+0000 7f984f400640 1 --2- 172.21.15.86:0/3921186072 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98580d2c80 0x7f985814cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:20.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.087+0000 7f985d200640 1 -- 172.21.15.86:0/3921186072 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98580d2c80 msgr2=0x7f985814cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:20.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.087+0000 7f985d200640 1 --2- 172.21.15.86:0/3921186072 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98580d2c80 0x7f985814cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:20.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.087+0000 7f985d200640 1 -- 172.21.15.86:0/3921186072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9858070e20 msgr2=0x7f98581526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:20.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.087+0000 7f985d200640 1 --2- 172.21.15.86:0/3921186072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9858070e20 0x7f98581526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:20.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.087+0000 7f985d200640 1 -- 172.21.15.86:0/3921186072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9858151350 con 0x7f98581616e0 2024-04-25T14:13:20.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.087+0000 7f985d200640 1 --2- 172.21.15.86:0/3921186072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98581616e0 0x7f985815c790 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f98500b2040 tx=0x7f985001bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:20.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.087+0000 7f9854c00640 1 -- 172.21.15.86:0/3921186072 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98500b4040 con 0x7f98581616e0 2024-04-25T14:13:20.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.087+0000 7f9854c00640 1 -- 172.21.15.86:0/3921186072 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98500c1070 con 0x7f98581616e0 2024-04-25T14:13:20.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.087+0000 7f985f200640 1 -- 172.21.15.86:0/3921186072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9858151630 con 0x7f98581616e0 2024-04-25T14:13:20.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.087+0000 7f9854c00640 1 -- 172.21.15.86:0/3921186072 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9850009450 con 0x7f98581616e0 2024-04-25T14:13:20.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.091+0000 7f985f200640 1 -- 172.21.15.86:0/3921186072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9858151bf0 con 0x7f98581616e0 2024-04-25T14:13:20.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.091+0000 7f985f200640 1 -- 172.21.15.86:0/3921186072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98580720e0 con 0x7f98581616e0 2024-04-25T14:13:20.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.091+0000 7f9854c00640 1 -- 172.21.15.86:0/3921186072 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 77) ==== 93253+0+0 (secure 0 0 0) 0x7f98500ba6b0 con 0x7f98581616e0 2024-04-25T14:13:20.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.091+0000 7f9854c00640 1 --2- 172.21.15.86:0/3921186072 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f983406f490 0x7f98340719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:20.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.091+0000 7f9854c00640 1 -- 172.21.15.86:0/3921186072 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6876+0+0 (secure 0 0 0) 0x7f985012ed00 con 0x7f98581616e0 2024-04-25T14:13:20.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.095+0000 7f9857e00640 1 --2- 172.21.15.86:0/3921186072 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f983406f490 0x7f98340719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:20.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.095+0000 7f9857e00640 1 --2- 172.21.15.86:0/3921186072 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f983406f490 0x7f98340719a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9848009580 tx=0x7f9848009190 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:20.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.099+0000 7f9854c00640 1 -- 172.21.15.86:0/3921186072 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9850018cb0 con 0x7f98581616e0 2024-04-25T14:13:20.142 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:13:20.145 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:13:20.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.291+0000 7f985f200640 1 -- 172.21.15.86:0/3921186072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98581510e0 con 0x7f98581616e0 2024-04-25T14:13:20.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.295+0000 7f9854c00640 1 -- 172.21.15.86:0/3921186072 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+5209 (secure 0 0 0) 0x7f98500faf10 con 0x7f98581616e0 2024-04-25T14:13:20.292 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:13:20.292 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":108,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/1828317917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1828317917},{"type":"v1","addr":"172.21.15.155:6834","nonce":1828317917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":9124,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1389714390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1389714390},{"type":"v1","addr":"172.21.15.86:6837","nonce":1389714390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:11:32.981538+0000","modified":"2024-04-25T14:13:18.277105+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":9546,"mds_1":9136},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9136":{"gid":9136,"name":"d","rank":1,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/91793677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":91793677},{"type":"v1","addr":"172.21.15.155:6835","nonce":91793677}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9546":{"gid":9546,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/2571655765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2571655765},{"type":"v1","addr":"172.21.15.86:6835","nonce":2571655765}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9546,"qdb_cluster":[9136,9546]},"id":10}]} 2024-04-25T14:13:20.293 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 108 2024-04-25T14:13:20.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.295+0000 7f985f200640 1 -- 172.21.15.86:0/3921186072 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f983406f490 msgr2=0x7f98340719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:20.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.295+0000 7f985f200640 1 --2- 172.21.15.86:0/3921186072 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f983406f490 0x7f98340719a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9848009580 tx=0x7f9848009190 comp rx=0 tx=0).stop 2024-04-25T14:13:20.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.295+0000 7f985f200640 1 -- 172.21.15.86:0/3921186072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98581616e0 msgr2=0x7f985815c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:20.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.295+0000 7f985f200640 1 --2- 172.21.15.86:0/3921186072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98581616e0 0x7f985815c790 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f98500b2040 tx=0x7f985001bef0 comp rx=0 tx=0).stop 2024-04-25T14:13:20.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.295+0000 7f985f200640 1 -- 172.21.15.86:0/3921186072 shutdown_connections 2024-04-25T14:13:20.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.295+0000 7f985f200640 1 --2- 172.21.15.86:0/3921186072 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f983406f490 0x7f98340719a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:20.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.295+0000 7f985f200640 1 --2- 172.21.15.86:0/3921186072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98581616e0 0x7f985815c790 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:20.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.295+0000 7f985f200640 1 --2- 172.21.15.86:0/3921186072 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98580d2c80 0x7f985814cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:20.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.295+0000 7f985f200640 1 --2- 172.21.15.86:0/3921186072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9858070e20 0x7f98581526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:20.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.295+0000 7f985f200640 1 -- 172.21.15.86:0/3921186072 >> 172.21.15.86:0/3921186072 conn(0x7f98580cffc0 msgr2=0x7f9858150240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:20.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.295+0000 7f985f200640 1 -- 172.21.15.86:0/3921186072 shutdown_connections 2024-04-25T14:13:20.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.295+0000 7f985f200640 1 -- 172.21.15.86:0/3921186072 wait complete. 2024-04-25T14:13:20.313 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:13:20.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.459+0000 7fb5b2000640 1 Processor -- start 2024-04-25T14:13:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.459+0000 7fb5b2000640 1 -- start start 2024-04-25T14:13:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.463+0000 7fb5b2000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb5ac0cfca0 0x7fb5ac06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.463+0000 7fb5b2000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb5ac0d16e0 0x7fb5ac0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.463+0000 7fb5b2000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5ac0d2030 con 0x7fb5ac06c670 2024-04-25T14:13:20.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.463+0000 7fb5b2000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb5ac070470 con 0x7fb5ac0d16e0 2024-04-25T14:13:20.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.463+0000 7fb5b2000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb5ac072bd0 con 0x7fb5ac0cfca0 2024-04-25T14:13:20.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.463+0000 7fb5abe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb5ac06c670 0x7fb5ac06ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47358/0 (socket says 172.21.15.86:47358) 2024-04-25T14:13:20.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.463+0000 7fb5abe00640 1 -- 172.21.15.86:0/1962914918 learned_addr learned my addr 172.21.15.86:0/1962914918 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:20.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.463+0000 7fb5a2a00640 1 --2- 172.21.15.86:0/1962914918 >> v2:172.21.15.155:3300/0 conn(0x7fb5ac0d16e0 0x7fb5ac0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:20.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.463+0000 7fb5ab400640 1 --2- 172.21.15.86:0/1962914918 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb5ac0cfca0 0x7fb5ac06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:20.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.463+0000 7fb5aaa00640 1 -- 172.21.15.86:0/1962914918 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1113012594 0 0) 0x7fb5a4001120 con 0x7fb5ac06c670 2024-04-25T14:13:20.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.463+0000 7fb5aaa00640 1 -- 172.21.15.86:0/1962914918 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb590003530 con 0x7fb5ac06c670 2024-04-25T14:13:20.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.463+0000 7fb5a2a00640 1 -- 172.21.15.86:0/1962914918 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb5ac0cfca0 msgr2=0x7fb5ac06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:20.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.463+0000 7fb5a2a00640 1 --2- 172.21.15.86:0/1962914918 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb5ac0cfca0 0x7fb5ac06f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:20.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.463+0000 7fb5a2a00640 1 -- 172.21.15.86:0/1962914918 >> v1:172.21.15.86:6789/0 conn(0x7fb5ac06c670 legacy=0x7fb5ac06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:20.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.463+0000 7fb5a2a00640 1 -- 172.21.15.86:0/1962914918 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5ac06f930 con 0x7fb5ac0d16e0 2024-04-25T14:13:20.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.467+0000 7fb5a2a00640 1 --2- 172.21.15.86:0/1962914918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5ac0d16e0 0x7fb5ac0d1af0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb5940097c0 tx=0x7fb59401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44e5b903781592c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:20.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.467+0000 7fb5aaa00640 1 -- 172.21.15.86:0/1962914918 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5940b1070 con 0x7fb5ac0d16e0 2024-04-25T14:13:20.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.467+0000 7fb5aaa00640 1 -- 172.21.15.86:0/1962914918 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5940ac410 con 0x7fb5ac0d16e0 2024-04-25T14:13:20.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.467+0000 7fb5aaa00640 1 -- 172.21.15.86:0/1962914918 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5940ac7e0 con 0x7fb5ac0d16e0 2024-04-25T14:13:20.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.467+0000 7fb5b2000640 1 -- 172.21.15.86:0/1962914918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5ac0d16e0 msgr2=0x7fb5ac0d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:20.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.467+0000 7fb5b2000640 1 --2- 172.21.15.86:0/1962914918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5ac0d16e0 0x7fb5ac0d1af0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb5940097c0 tx=0x7fb59401c4a0 comp rx=0 tx=0).stop 2024-04-25T14:13:20.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.467+0000 7fb5b2000640 1 -- 172.21.15.86:0/1962914918 shutdown_connections 2024-04-25T14:13:20.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.467+0000 7fb5b2000640 1 --2- 172.21.15.86:0/1962914918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5ac0d16e0 0x7fb5ac0d1af0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:20.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.467+0000 7fb5b2000640 1 --2- 172.21.15.86:0/1962914918 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb5ac0cfca0 0x7fb5ac06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:20.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.467+0000 7fb5b2000640 1 -- 172.21.15.86:0/1962914918 >> 172.21.15.86:0/1962914918 conn(0x7fb5ac0b4650 msgr2=0x7fb5ac0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:20.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.467+0000 7fb5b2000640 1 -- 172.21.15.86:0/1962914918 shutdown_connections 2024-04-25T14:13:20.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.467+0000 7fb5b2000640 1 -- 172.21.15.86:0/1962914918 wait complete. 2024-04-25T14:13:20.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.471+0000 7fb5b2000640 1 Processor -- start 2024-04-25T14:13:20.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.471+0000 7fb5b2000640 1 -- start start 2024-04-25T14:13:20.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.471+0000 7fb5b2000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb5ac06c670 0x7fb5ac148f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:20.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.471+0000 7fb5b2000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5ac0cfca0 0x7fb5ac0c1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:20.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.471+0000 7fb5b2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5ac0d16e0 0x7fb5ac0c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:20.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.471+0000 7fb5b2000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ac16a070 con 0x7fb5ac0d16e0 2024-04-25T14:13:20.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.471+0000 7fb5b2000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ac169ef0 con 0x7fb5ac0cfca0 2024-04-25T14:13:20.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.471+0000 7fb5b2000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb5ac16a1f0 con 0x7fb5ac06c670 2024-04-25T14:13:20.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.471+0000 7fb5abe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5ac0d16e0 0x7fb5ac0c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:20.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.471+0000 7fb5abe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5ac0d16e0 0x7fb5ac0c21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41274/0 (socket says 172.21.15.86:41274) 2024-04-25T14:13:20.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.471+0000 7fb5abe00640 1 -- 172.21.15.86:0/3271037000 learned_addr learned my addr 172.21.15.86:0/3271037000 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:20.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.471+0000 7fb5a2a00640 1 --2- 172.21.15.86:0/3271037000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5ac0cfca0 0x7fb5ac0c1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:20.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.471+0000 7fb5ab400640 1 --2- 172.21.15.86:0/3271037000 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb5ac06c670 0x7fb5ac148f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:20.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.471+0000 7fb5abe00640 1 -- 172.21.15.86:0/3271037000 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb5ac06c670 msgr2=0x7fb5ac148f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:20.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.471+0000 7fb5abe00640 1 --2- 172.21.15.86:0/3271037000 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb5ac06c670 0x7fb5ac148f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:20.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.471+0000 7fb5abe00640 1 -- 172.21.15.86:0/3271037000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5ac0cfca0 msgr2=0x7fb5ac0c1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:20.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.471+0000 7fb5abe00640 1 --2- 172.21.15.86:0/3271037000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5ac0cfca0 0x7fb5ac0c1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:20.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.471+0000 7fb5abe00640 1 -- 172.21.15.86:0/3271037000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5ac158360 con 0x7fb5ac0d16e0 2024-04-25T14:13:20.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.471+0000 7fb5a2a00640 1 --2- 172.21.15.86:0/3271037000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5ac0cfca0 0x7fb5ac0c1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:20.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.471+0000 7fb5ab400640 1 --2- 172.21.15.86:0/3271037000 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb5ac06c670 0x7fb5ac148f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:20.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.471+0000 7fb5abe00640 1 --2- 172.21.15.86:0/3271037000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5ac0d16e0 0x7fb5ac0c21b0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400c3f0 tx=0x7fb5a400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:20.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.471+0000 7fb5a3e00640 1 -- 172.21.15.86:0/3271037000 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4017070 con 0x7fb5ac0d16e0 2024-04-25T14:13:20.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.471+0000 7fb5a3e00640 1 -- 172.21.15.86:0/3271037000 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5a4005cf0 con 0x7fb5ac0d16e0 2024-04-25T14:13:20.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.471+0000 7fb5b2000640 1 -- 172.21.15.86:0/3271037000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5ac0c2940 con 0x7fb5ac0d16e0 2024-04-25T14:13:20.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.471+0000 7fb5a3e00640 1 -- 172.21.15.86:0/3271037000 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a40072d0 con 0x7fb5ac0d16e0 2024-04-25T14:13:20.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.471+0000 7fb5b2000640 1 -- 172.21.15.86:0/3271037000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5ac149440 con 0x7fb5ac0d16e0 2024-04-25T14:13:20.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.475+0000 7fb5b2000640 1 -- 172.21.15.86:0/3271037000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5ac147a40 con 0x7fb5ac0d16e0 2024-04-25T14:13:20.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.475+0000 7fb5a3e00640 1 -- 172.21.15.86:0/3271037000 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 77) ==== 93253+0+0 (secure 0 0 0) 0x7fb5a401a070 con 0x7fb5ac0d16e0 2024-04-25T14:13:20.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.475+0000 7fb5a3e00640 1 --2- 172.21.15.86:0/3271037000 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fb58806f4c0 0x7fb5880719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:20.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.475+0000 7fb5a3e00640 1 -- 172.21.15.86:0/3271037000 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6876+0+0 (secure 0 0 0) 0x7fb5a408eb30 con 0x7fb5ac0d16e0 2024-04-25T14:13:20.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.475+0000 7fb5ab400640 1 --2- 172.21.15.86:0/3271037000 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fb58806f4c0 0x7fb5880719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:20.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.475+0000 7fb5ab400640 1 --2- 172.21.15.86:0/3271037000 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fb58806f4c0 0x7fb5880719d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac070710 tx=0x7fb59c00b040 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:20.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.479+0000 7fb5a3e00640 1 -- 172.21.15.86:0/3271037000 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb5ac147a40 con 0x7fb5ac0d16e0 2024-04-25T14:13:20.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.639+0000 7fb5b2000640 1 -- 172.21.15.86:0/3271037000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5ac09f4b0 con 0x7fb5ac0d16e0 2024-04-25T14:13:20.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.643+0000 7fb5a3e00640 1 -- 172.21.15.86:0/3271037000 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+18036 (secure 0 0 0) 0x7fb5ac0d2030 con 0x7fb5ac0d16e0 2024-04-25T14:13:20.641 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:13:20.641 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":144,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:13:18.050429+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:11:30.810321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T14:11:31.944888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.e","mappings":[{"from":3,"to":1}]},{"pgid":"22.32","mappings":[{"from":3,"to":0}]},{"pgid":"22.34","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/3190429862":"2024-04-26T14:13:18.050389+0000","172.21.15.86:0/3060165022":"2024-04-26T14:13:18.050389+0000","172.21.15.86:6832/1173654665":"2024-04-26T14:13:18.050389+0000","172.21.15.86:0/2751178284":"2024-04-26T14:13:18.050389+0000","172.21.15.86:6833/1173654665":"2024-04-26T14:13:18.050389+0000","172.21.15.86:0/4083287014":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/1048620009":"2024-04-26T14:13:18.050389+0000","172.21.15.86:0/746190999":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/1426015871":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/3750847077":"2024-04-26T14:13:18.050389+0000","172.21.15.86:0/3962343281":"2024-04-26T14:11:51.781677+0000","172.21.15.86:6833/264890479":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/3349072784":"2024-04-26T14:13:18.050389+0000","172.21.15.86:6832/264890479":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/2724786983":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/285240843":"2024-04-26T14:11:51.781677+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:13:20.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.643+0000 7fb5b2000640 1 -- 172.21.15.86:0/3271037000 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fb58806f4c0 msgr2=0x7fb5880719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:20.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.643+0000 7fb5b2000640 1 --2- 172.21.15.86:0/3271037000 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fb58806f4c0 0x7fb5880719d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac070710 tx=0x7fb59c00b040 comp rx=0 tx=0).stop 2024-04-25T14:13:20.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.643+0000 7fb5b2000640 1 -- 172.21.15.86:0/3271037000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5ac0d16e0 msgr2=0x7fb5ac0c21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:20.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.643+0000 7fb5b2000640 1 --2- 172.21.15.86:0/3271037000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5ac0d16e0 0x7fb5ac0c21b0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400c3f0 tx=0x7fb5a400c8b0 comp rx=0 tx=0).stop 2024-04-25T14:13:20.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.643+0000 7fb5b2000640 1 -- 172.21.15.86:0/3271037000 shutdown_connections 2024-04-25T14:13:20.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.643+0000 7fb5b2000640 1 --2- 172.21.15.86:0/3271037000 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fb58806f4c0 0x7fb5880719d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:20.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.643+0000 7fb5b2000640 1 --2- 172.21.15.86:0/3271037000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5ac0d16e0 0x7fb5ac0c21b0 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:20.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.643+0000 7fb5b2000640 1 --2- 172.21.15.86:0/3271037000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5ac0cfca0 0x7fb5ac0c1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:20.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.643+0000 7fb5b2000640 1 --2- 172.21.15.86:0/3271037000 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb5ac06c670 0x7fb5ac148f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:20.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.643+0000 7fb5b2000640 1 -- 172.21.15.86:0/3271037000 >> 172.21.15.86:0/3271037000 conn(0x7fb5ac0b4650 msgr2=0x7fb5ac09a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:20.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.643+0000 7fb5b2000640 1 -- 172.21.15.86:0/3271037000 shutdown_connections 2024-04-25T14:13:20.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.643+0000 7fb5b2000640 1 -- 172.21.15.86:0/3271037000 wait complete. 2024-04-25T14:13:20.663 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T14:13:20.663 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:13:20.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.811+0000 7f9b93200640 1 Processor -- start 2024-04-25T14:13:20.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.811+0000 7f9b93200640 1 -- start start 2024-04-25T14:13:20.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b93200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9b94161450 0x7f9b94161860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:20.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b93200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b94072440 con 0x7f9b94070d60 2024-04-25T14:13:20.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b93200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b94162140 con 0x7f9b9415da10 2024-04-25T14:13:20.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b93200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9b94162420 con 0x7f9b94161450 2024-04-25T14:13:20.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b91e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9b94070d60 0x7f9b94071170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47364/0 (socket says 172.21.15.86:47364) 2024-04-25T14:13:20.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b91e00640 1 -- 172.21.15.86:0/3270244757 learned_addr learned my addr 172.21.15.86:0/3270244757 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:20.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b89400640 1 --2- 172.21.15.86:0/3270244757 >> v2:172.21.15.155:3300/0 conn(0x7f9b94161450 0x7f9b94161860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:20.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b91400640 1 -- 172.21.15.86:0/3270244757 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4258825141 0 0) 0x7f9b94072440 con 0x7f9b94070d60 2024-04-25T14:13:20.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b91400640 1 -- 172.21.15.86:0/3270244757 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b78003530 con 0x7f9b94070d60 2024-04-25T14:13:20.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b91400640 1 -- 172.21.15.86:0/3270244757 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4156898259 0 0) 0x7f9b78003530 con 0x7f9b94070d60 2024-04-25T14:13:20.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b91400640 1 -- 172.21.15.86:0/3270244757 >> v2:172.21.15.155:3300/0 conn(0x7f9b94161450 msgr2=0x7f9b94161860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:20.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b91400640 1 --2- 172.21.15.86:0/3270244757 >> v2:172.21.15.155:3300/0 conn(0x7f9b94161450 0x7f9b94161860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:20.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b91400640 1 -- 172.21.15.86:0/3270244757 >> v1:172.21.15.155:6789/0 conn(0x7f9b9415da10 legacy=0x7f9b9415de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:20.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b91400640 1 -- 172.21.15.86:0/3270244757 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b940726e0 con 0x7f9b94070d60 2024-04-25T14:13:20.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b91400640 1 -- 172.21.15.86:0/3270244757 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f9b84001680 con 0x7f9b94070d60 2024-04-25T14:13:20.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b91400640 1 -- 172.21.15.86:0/3270244757 >> v1:172.21.15.86:6789/0 conn(0x7f9b94070d60 legacy=0x7f9b94071170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:20.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b91400640 1 --2- 172.21.15.86:0/3270244757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b780039c0 0x7f9b78003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:20.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b91400640 1 --2- 172.21.15.86:0/3270244757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b78004810 0x7f9b78004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:20.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b91400640 1 --2- 172.21.15.86:0/3270244757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b78005360 0x7f9b7800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:20.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b91400640 1 -- 172.21.15.86:0/3270244757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b94162420 con 0x7f9b78004810 2024-04-25T14:13:20.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b91400640 1 -- 172.21.15.86:0/3270244757 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7800c290 con 0x7f9b78005360 2024-04-25T14:13:20.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b91400640 1 -- 172.21.15.86:0/3270244757 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9b7800c490 con 0x7f9b780039c0 2024-04-25T14:13:20.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b89400640 1 --2- 172.21.15.86:0/3270244757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b78004810 0x7f9b78004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:20.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b91e00640 1 --2- 172.21.15.86:0/3270244757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b780039c0 0x7f9b78003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:20.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b92800640 1 --2- 172.21.15.86:0/3270244757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b78005360 0x7f9b7800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:20.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b89400640 1 -- 172.21.15.86:0/3270244757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b780039c0 msgr2=0x7f9b78003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:20.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b89400640 1 --2- 172.21.15.86:0/3270244757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b780039c0 0x7f9b78003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:20.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b89400640 1 -- 172.21.15.86:0/3270244757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b78005360 msgr2=0x7f9b7800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:20.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b89400640 1 --2- 172.21.15.86:0/3270244757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b78005360 0x7f9b7800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:20.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b92800640 1 -- 172.21.15.86:0/3270244757 reap_dead start 2024-04-25T14:13:20.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.815+0000 7f9b89400640 1 -- 172.21.15.86:0/3270244757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b78003530 con 0x7f9b78004810 2024-04-25T14:13:20.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.819+0000 7f9b89400640 1 --2- 172.21.15.86:0/3270244757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b78004810 0x7f9b78004c20 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c009ae0 tx=0x7f9b7c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da7a2f9fadb414d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:20.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.819+0000 7f9b91400640 1 -- 172.21.15.86:0/3270244757 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c0b2070 con 0x7f9b78004810 2024-04-25T14:13:20.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.819+0000 7f9b91400640 1 -- 172.21.15.86:0/3270244757 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b7c0a5520 con 0x7f9b78004810 2024-04-25T14:13:20.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.819+0000 7f9b91400640 1 -- 172.21.15.86:0/3270244757 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c0096b0 con 0x7f9b78004810 2024-04-25T14:13:20.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.819+0000 7f9b93200640 1 -- 172.21.15.86:0/3270244757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b78004810 msgr2=0x7f9b78004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:20.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.819+0000 7f9b93200640 1 --2- 172.21.15.86:0/3270244757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b78004810 0x7f9b78004c20 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c009ae0 tx=0x7f9b7c01ca70 comp rx=0 tx=0).stop 2024-04-25T14:13:20.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.819+0000 7f9b93200640 1 -- 172.21.15.86:0/3270244757 shutdown_connections 2024-04-25T14:13:20.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.819+0000 7f9b93200640 1 --2- 172.21.15.86:0/3270244757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b78004810 0x7f9b78004c20 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:20.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.819+0000 7f9b93200640 1 -- 172.21.15.86:0/3270244757 >> 172.21.15.86:0/3270244757 conn(0x7f9b940d3330 msgr2=0x7f9b940d3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:20.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.819+0000 7f9b93200640 1 -- 172.21.15.86:0/3270244757 shutdown_connections 2024-04-25T14:13:20.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.819+0000 7f9b93200640 1 -- 172.21.15.86:0/3270244757 wait complete. 2024-04-25T14:13:20.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.823+0000 7f9b93200640 1 Processor -- start 2024-04-25T14:13:20.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.823+0000 7f9b93200640 1 -- start start 2024-04-25T14:13:20.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.823+0000 7f9b93200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b78004810 0x7f9b9415be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:20.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.823+0000 7f9b93200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b940cb510 0x7f9b9414c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:20.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.823+0000 7f9b93200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b941fcb40 0x7f9b941fcf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:20.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.823+0000 7f9b93200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b940cde10 con 0x7f9b940cb510 2024-04-25T14:13:20.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.823+0000 7f9b93200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9b940cdc90 con 0x7f9b941fcb40 2024-04-25T14:13:20.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.823+0000 7f9b93200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9b940cdf90 con 0x7f9b78004810 2024-04-25T14:13:20.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.823+0000 7f9b89400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b940cb510 0x7f9b9414c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:20.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.823+0000 7f9b89400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b940cb510 0x7f9b9414c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41296/0 (socket says 172.21.15.86:41296) 2024-04-25T14:13:20.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.823+0000 7f9b89400640 1 -- 172.21.15.86:0/1484187449 learned_addr learned my addr 172.21.15.86:0/1484187449 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:20.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.823+0000 7f9b91e00640 1 --2- 172.21.15.86:0/1484187449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b78004810 0x7f9b9415be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:20.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.823+0000 7f9b92800640 1 --2- 172.21.15.86:0/1484187449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b941fcb40 0x7f9b941fcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:20.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.823+0000 7f9b89400640 1 -- 172.21.15.86:0/1484187449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b78004810 msgr2=0x7f9b9415be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:20.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.823+0000 7f9b89400640 1 --2- 172.21.15.86:0/1484187449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b78004810 0x7f9b9415be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:20.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.823+0000 7f9b89400640 1 -- 172.21.15.86:0/1484187449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b941fcb40 msgr2=0x7f9b941fcf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:20.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.823+0000 7f9b89400640 1 --2- 172.21.15.86:0/1484187449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b941fcb40 0x7f9b941fcf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:20.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.823+0000 7f9b89400640 1 -- 172.21.15.86:0/1484187449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b941fd490 con 0x7f9b940cb510 2024-04-25T14:13:20.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.827+0000 7f9b89400640 1 --2- 172.21.15.86:0/1484187449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b940cb510 0x7f9b9414c900 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c01c8a0 tx=0x7f9b7c019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:20.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.827+0000 7f9b8aa00640 1 -- 172.21.15.86:0/1484187449 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c0b2040 con 0x7f9b940cb510 2024-04-25T14:13:20.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.827+0000 7f9b8aa00640 1 -- 172.21.15.86:0/1484187449 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b7c0a5df0 con 0x7f9b940cb510 2024-04-25T14:13:20.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.827+0000 7f9b93200640 1 -- 172.21.15.86:0/1484187449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b9406d9d0 con 0x7f9b940cb510 2024-04-25T14:13:20.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.827+0000 7f9b8aa00640 1 -- 172.21.15.86:0/1484187449 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c004020 con 0x7f9b940cb510 2024-04-25T14:13:20.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.827+0000 7f9b93200640 1 -- 172.21.15.86:0/1484187449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b9406df90 con 0x7f9b940cb510 2024-04-25T14:13:20.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.827+0000 7f9b93200640 1 -- 172.21.15.86:0/1484187449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b9415c3d0 con 0x7f9b940cb510 2024-04-25T14:13:20.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.827+0000 7f9b8aa00640 1 -- 172.21.15.86:0/1484187449 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 77) ==== 93253+0+0 (secure 0 0 0) 0x7f9b7c0bf030 con 0x7f9b940cb510 2024-04-25T14:13:20.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.827+0000 7f9b8aa00640 1 --2- 172.21.15.86:0/1484187449 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f9b7006f490 0x7f9b700719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:20.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.827+0000 7f9b8aa00640 1 -- 172.21.15.86:0/1484187449 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6876+0+0 (secure 0 0 0) 0x7f9b7c129300 con 0x7f9b940cb510 2024-04-25T14:13:20.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.827+0000 7f9b91e00640 1 --2- 172.21.15.86:0/1484187449 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f9b7006f490 0x7f9b700719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:20.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.831+0000 7f9b91e00640 1 --2- 172.21.15.86:0/1484187449 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f9b7006f490 0x7f9b700719a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9b84000c10 tx=0x7f9b84007040 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:20.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:20.835+0000 7f9b8aa00640 1 -- 172.21.15.86:0/1484187449 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9b7c0c6060 con 0x7f9b940cb510 2024-04-25T14:13:21.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.027+0000 7f9b93200640 1 -- 172.21.15.86:0/1484187449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b9406d760 con 0x7f9b940cb510 2024-04-25T14:13:21.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.027+0000 7f9b8aa00640 1 -- 172.21.15.86:0/1484187449 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+5209 (secure 0 0 0) 0x7f9b941fd490 con 0x7f9b940cb510 2024-04-25T14:13:21.025 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:13:21.025 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":108,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/1828317917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1828317917},{"type":"v1","addr":"172.21.15.155:6834","nonce":1828317917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":9124,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1389714390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1389714390},{"type":"v1","addr":"172.21.15.86:6837","nonce":1389714390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:11:32.981538+0000","modified":"2024-04-25T14:13:18.277105+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":9546,"mds_1":9136},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9136":{"gid":9136,"name":"d","rank":1,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/91793677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":91793677},{"type":"v1","addr":"172.21.15.155:6835","nonce":91793677}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9546":{"gid":9546,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/2571655765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2571655765},{"type":"v1","addr":"172.21.15.86:6835","nonce":2571655765}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9546,"qdb_cluster":[9136,9546]},"id":10}]} 2024-04-25T14:13:21.025 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 108 2024-04-25T14:13:21.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.027+0000 7f9b93200640 1 -- 172.21.15.86:0/1484187449 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f9b7006f490 msgr2=0x7f9b700719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:21.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.027+0000 7f9b93200640 1 --2- 172.21.15.86:0/1484187449 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f9b7006f490 0x7f9b700719a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9b84000c10 tx=0x7f9b84007040 comp rx=0 tx=0).stop 2024-04-25T14:13:21.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.027+0000 7f9b93200640 1 -- 172.21.15.86:0/1484187449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b940cb510 msgr2=0x7f9b9414c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:21.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.027+0000 7f9b93200640 1 --2- 172.21.15.86:0/1484187449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b940cb510 0x7f9b9414c900 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c01c8a0 tx=0x7f9b7c019810 comp rx=0 tx=0).stop 2024-04-25T14:13:21.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.027+0000 7f9b93200640 1 -- 172.21.15.86:0/1484187449 shutdown_connections 2024-04-25T14:13:21.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.027+0000 7f9b93200640 1 --2- 172.21.15.86:0/1484187449 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f9b7006f490 0x7f9b700719a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.027+0000 7f9b93200640 1 --2- 172.21.15.86:0/1484187449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b941fcb40 0x7f9b941fcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.027+0000 7f9b93200640 1 --2- 172.21.15.86:0/1484187449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b940cb510 0x7f9b9414c900 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.027+0000 7f9b93200640 1 --2- 172.21.15.86:0/1484187449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b78004810 0x7f9b9415be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.027+0000 7f9b93200640 1 -- 172.21.15.86:0/1484187449 >> 172.21.15.86:0/1484187449 conn(0x7f9b940d3330 msgr2=0x7f9b94153ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:21.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.027+0000 7f9b93200640 1 -- 172.21.15.86:0/1484187449 shutdown_connections 2024-04-25T14:13:21.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.027+0000 7f9b93200640 1 -- 172.21.15.86:0/1484187449 wait complete. 2024-04-25T14:13:21.047 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:13:21.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.199+0000 7f302e200640 1 Processor -- start 2024-04-25T14:13:21.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.199+0000 7f302e200640 1 -- start start 2024-04-25T14:13:21.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.199+0000 7f302e200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f30280d2910 0x7f302815dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:21.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.199+0000 7f302e200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30280cdc70 con 0x7f30280b7b50 2024-04-25T14:13:21.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.199+0000 7f302e200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30280cdf10 con 0x7f30280b1cc0 2024-04-25T14:13:21.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.199+0000 7f302e200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f302806ece0 con 0x7f30280d2910 2024-04-25T14:13:21.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.199+0000 7f302ce00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f30280b7b50 0x7f30280d2200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47372/0 (socket says 172.21.15.86:47372) 2024-04-25T14:13:21.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.199+0000 7f302ce00640 1 -- 172.21.15.86:0/2050037073 learned_addr learned my addr 172.21.15.86:0/2050037073 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:21.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.199+0000 7f3027e00640 1 --2- 172.21.15.86:0/2050037073 >> v2:172.21.15.86:3300/0 conn(0x7f30280d2910 0x7f302815dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:21.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.199+0000 7f3027400640 1 -- 172.21.15.86:0/2050037073 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3089784715 0 0) 0x7f30280cdc70 con 0x7f30280b7b50 2024-04-25T14:13:21.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.199+0000 7f3027400640 1 -- 172.21.15.86:0/2050037073 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f300c003530 con 0x7f30280b7b50 2024-04-25T14:13:21.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.199+0000 7f3027e00640 1 -- 172.21.15.86:0/2050037073 >> v1:172.21.15.155:6789/0 conn(0x7f30280b1cc0 legacy=0x7f30280b20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:21.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.199+0000 7f3027e00640 1 -- 172.21.15.86:0/2050037073 >> v1:172.21.15.86:6789/0 conn(0x7f30280b7b50 legacy=0x7f30280d2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:21.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.199+0000 7f3027e00640 1 -- 172.21.15.86:0/2050037073 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30280ce1f0 con 0x7f30280d2910 2024-04-25T14:13:21.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.203+0000 7f3027e00640 1 --2- 172.21.15.86:0/2050037073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30280d2910 0x7f302815dbe0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f30140097c0 tx=0x7f301401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d13e8fe57fc5704 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:21.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.203+0000 7f3027400640 1 -- 172.21.15.86:0/2050037073 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30140b0070 con 0x7f30280d2910 2024-04-25T14:13:21.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.203+0000 7f3027400640 1 -- 172.21.15.86:0/2050037073 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30140ab600 con 0x7f30280d2910 2024-04-25T14:13:21.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.203+0000 7f3027400640 1 -- 172.21.15.86:0/2050037073 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30280b2ff0 con 0x7f30280d2910 2024-04-25T14:13:21.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.203+0000 7f302e200640 1 -- 172.21.15.86:0/2050037073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30280d2910 msgr2=0x7f302815dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:21.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.203+0000 7f302e200640 1 --2- 172.21.15.86:0/2050037073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30280d2910 0x7f302815dbe0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f30140097c0 tx=0x7f301401c4a0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.203+0000 7f302e200640 1 -- 172.21.15.86:0/2050037073 shutdown_connections 2024-04-25T14:13:21.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.203+0000 7f302e200640 1 --2- 172.21.15.86:0/2050037073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30280d2910 0x7f302815dbe0 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.203+0000 7f302e200640 1 -- 172.21.15.86:0/2050037073 >> 172.21.15.86:0/2050037073 conn(0x7f302809ba30 msgr2=0x7f30280b70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:21.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.203+0000 7f302e200640 1 -- 172.21.15.86:0/2050037073 shutdown_connections 2024-04-25T14:13:21.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.203+0000 7f302e200640 1 -- 172.21.15.86:0/2050037073 wait complete. 2024-04-25T14:13:21.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.207+0000 7f302e200640 1 Processor -- start 2024-04-25T14:13:21.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.207+0000 7f302e200640 1 -- start start 2024-04-25T14:13:21.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.207+0000 7f302e200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30280b1cc0 0x7f3028151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:21.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.207+0000 7f302e200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30280b7b50 0x7f3028156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:21.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.207+0000 7f302e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3028151720 0x7f3028157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:21.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.207+0000 7f302e200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f302816bb00 con 0x7f3028151720 2024-04-25T14:13:21.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.207+0000 7f302e200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f302816b980 con 0x7f30280b1cc0 2024-04-25T14:13:21.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.207+0000 7f302e200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f302816bc80 con 0x7f30280b7b50 2024-04-25T14:13:21.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.207+0000 7f302d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3028151720 0x7f3028157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:21.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.207+0000 7f302d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3028151720 0x7f3028157280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41310/0 (socket says 172.21.15.86:41310) 2024-04-25T14:13:21.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.207+0000 7f302d800640 1 -- 172.21.15.86:0/3697640472 learned_addr learned my addr 172.21.15.86:0/3697640472 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:21.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.207+0000 7f3027e00640 1 --2- 172.21.15.86:0/3697640472 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30280b7b50 0x7f3028156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:21.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.207+0000 7f302ce00640 1 --2- 172.21.15.86:0/3697640472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30280b1cc0 0x7f3028151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:21.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.207+0000 7f302d800640 1 -- 172.21.15.86:0/3697640472 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30280b7b50 msgr2=0x7f3028156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:21.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.207+0000 7f302d800640 1 --2- 172.21.15.86:0/3697640472 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30280b7b50 0x7f3028156d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.207+0000 7f302d800640 1 -- 172.21.15.86:0/3697640472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30280b1cc0 msgr2=0x7f3028151060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:21.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.207+0000 7f302d800640 1 --2- 172.21.15.86:0/3697640472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30280b1cc0 0x7f3028151060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.207+0000 7f302d800640 1 -- 172.21.15.86:0/3697640472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30280c92b0 con 0x7f3028151720 2024-04-25T14:13:21.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.207+0000 7f3027e00640 1 --2- 172.21.15.86:0/3697640472 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30280b7b50 0x7f3028156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:21.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.207+0000 7f302ce00640 1 --2- 172.21.15.86:0/3697640472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30280b1cc0 0x7f3028151060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:21.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.211+0000 7f302d800640 1 --2- 172.21.15.86:0/3697640472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3028151720 0x7f3028157280 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f302000c340 tx=0x7f302000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:21.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.211+0000 7f3024c00640 1 -- 172.21.15.86:0/3697640472 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3020017070 con 0x7f3028151720 2024-04-25T14:13:21.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.211+0000 7f3024c00640 1 -- 172.21.15.86:0/3697640472 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30200052d0 con 0x7f3028151720 2024-04-25T14:13:21.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.211+0000 7f3024c00640 1 -- 172.21.15.86:0/3697640472 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3020003800 con 0x7f3028151720 2024-04-25T14:13:21.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.211+0000 7f302e200640 1 -- 172.21.15.86:0/3697640472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30280c97b0 con 0x7f3028151720 2024-04-25T14:13:21.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.211+0000 7f302e200640 1 -- 172.21.15.86:0/3697640472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30280c9d70 con 0x7f3028151720 2024-04-25T14:13:21.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.211+0000 7f302e200640 1 -- 172.21.15.86:0/3697640472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30280c9520 con 0x7f3028151720 2024-04-25T14:13:21.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.211+0000 7f3024c00640 1 -- 172.21.15.86:0/3697640472 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 77) ==== 93253+0+0 (secure 0 0 0) 0x7f302001a070 con 0x7f3028151720 2024-04-25T14:13:21.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.215+0000 7f3024c00640 1 --2- 172.21.15.86:0/3697640472 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f300806f4c0 0x7f30080719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:21.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.215+0000 7f3024c00640 1 -- 172.21.15.86:0/3697640472 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6876+0+0 (secure 0 0 0) 0x7f302008e8c0 con 0x7f3028151720 2024-04-25T14:13:21.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.215+0000 7f302ce00640 1 --2- 172.21.15.86:0/3697640472 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f300806f4c0 0x7f30080719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:21.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.215+0000 7f302ce00640 1 --2- 172.21.15.86:0/3697640472 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f300806f4c0 0x7f30080719d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3018005c10 tx=0x7f3018009040 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:21.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.219+0000 7f3024c00640 1 -- 172.21.15.86:0/3697640472 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3020005480 con 0x7f3028151720 2024-04-25T14:13:21.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.407+0000 7f302e200640 1 -- 172.21.15.86:0/3697640472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3028076b80 con 0x7f3028151720 2024-04-25T14:13:21.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.407+0000 7f3024c00640 1 -- 172.21.15.86:0/3697640472 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+5209 (secure 0 0 0) 0x7f3020063340 con 0x7f3028151720 2024-04-25T14:13:21.406 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:13:21.406 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":108,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/1828317917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1828317917},{"type":"v1","addr":"172.21.15.155:6834","nonce":1828317917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":9124,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1389714390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1389714390},{"type":"v1","addr":"172.21.15.86:6837","nonce":1389714390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:11:32.981538+0000","modified":"2024-04-25T14:13:18.277105+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":9546,"mds_1":9136},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9136":{"gid":9136,"name":"d","rank":1,"incarnation":105,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/91793677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":91793677},{"type":"v1","addr":"172.21.15.155:6835","nonce":91793677}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9546":{"gid":9546,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/2571655765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2571655765},{"type":"v1","addr":"172.21.15.86:6835","nonce":2571655765}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9546,"qdb_cluster":[9136,9546]},"id":10}]} 2024-04-25T14:13:21.407 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 108 2024-04-25T14:13:21.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.411+0000 7f302e200640 1 -- 172.21.15.86:0/3697640472 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f300806f4c0 msgr2=0x7f30080719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:21.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.411+0000 7f302e200640 1 --2- 172.21.15.86:0/3697640472 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f300806f4c0 0x7f30080719d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3018005c10 tx=0x7f3018009040 comp rx=0 tx=0).stop 2024-04-25T14:13:21.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.411+0000 7f302e200640 1 -- 172.21.15.86:0/3697640472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3028151720 msgr2=0x7f3028157280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:21.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.411+0000 7f302e200640 1 --2- 172.21.15.86:0/3697640472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3028151720 0x7f3028157280 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f302000c340 tx=0x7f302000c800 comp rx=0 tx=0).stop 2024-04-25T14:13:21.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.411+0000 7f302e200640 1 -- 172.21.15.86:0/3697640472 shutdown_connections 2024-04-25T14:13:21.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.411+0000 7f302e200640 1 --2- 172.21.15.86:0/3697640472 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f300806f4c0 0x7f30080719d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.411+0000 7f302e200640 1 --2- 172.21.15.86:0/3697640472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3028151720 0x7f3028157280 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.411+0000 7f302e200640 1 --2- 172.21.15.86:0/3697640472 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30280b7b50 0x7f3028156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.411+0000 7f302e200640 1 --2- 172.21.15.86:0/3697640472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30280b1cc0 0x7f3028151060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.411+0000 7f302e200640 1 -- 172.21.15.86:0/3697640472 >> 172.21.15.86:0/3697640472 conn(0x7f302809ba30 msgr2=0x7f30280d0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:21.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.411+0000 7f302e200640 1 -- 172.21.15.86:0/3697640472 shutdown_connections 2024-04-25T14:13:21.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.411+0000 7f302e200640 1 -- 172.21.15.86:0/3697640472 wait complete. 2024-04-25T14:13:21.429 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:13:21.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.575+0000 7f8eff400640 1 Processor -- start 2024-04-25T14:13:21.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.575+0000 7f8eff400640 1 -- start start 2024-04-25T14:13:21.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.579+0000 7f8eff400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8ef8070330 0x7f8ef80d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:21.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.579+0000 7f8eff400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ef80ce790 0x7f8ef806f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:21.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.579+0000 7f8eff400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ef80d2030 con 0x7f8ef80d1c20 2024-04-25T14:13:21.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.579+0000 7f8eff400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8ef815e5c0 con 0x7f8ef8070330 2024-04-25T14:13:21.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.579+0000 7f8eff400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8ef815e8a0 con 0x7f8ef80ce790 2024-04-25T14:13:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.579+0000 7f8efca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8ef8070330 0x7f8ef80d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.579+0000 7f8efca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8ef8070330 0x7f8ef80d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41320/0 (socket says 172.21.15.86:41320) 2024-04-25T14:13:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.579+0000 7f8efca00640 1 -- 172.21.15.86:0/1292477638 learned_addr learned my addr 172.21.15.86:0/1292477638 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.579+0000 7f8efd400640 1 --2- 172.21.15.86:0/1292477638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ef80ce790 0x7f8ef806f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.579+0000 7f8efca00640 1 -- 172.21.15.86:0/1292477638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ef80ce790 msgr2=0x7f8ef806f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.579+0000 7f8efca00640 1 --2- 172.21.15.86:0/1292477638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ef80ce790 0x7f8ef806f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.579+0000 7f8efca00640 1 -- 172.21.15.86:0/1292477638 >> v1:172.21.15.155:6789/0 conn(0x7f8ef80d1c20 legacy=0x7f8ef806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.579+0000 7f8efca00640 1 -- 172.21.15.86:0/1292477638 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef815eb80 con 0x7f8ef8070330 2024-04-25T14:13:21.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.579+0000 7f8efd400640 1 --2- 172.21.15.86:0/1292477638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ef80ce790 0x7f8ef806f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:21.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.583+0000 7f8efca00640 1 --2- 172.21.15.86:0/1292477638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ef8070330 0x7f8ef80d16e0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009870 tx=0x7f8eec01c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139d4f9eab377516 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:21.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.583+0000 7f8ef7400640 1 -- 172.21.15.86:0/1292477638 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec0b0070 con 0x7f8ef8070330 2024-04-25T14:13:21.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.583+0000 7f8ef7400640 1 -- 172.21.15.86:0/1292477638 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eec0ab600 con 0x7f8ef8070330 2024-04-25T14:13:21.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.583+0000 7f8ef7400640 1 -- 172.21.15.86:0/1292477638 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec0ab9d0 con 0x7f8ef8070330 2024-04-25T14:13:21.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.583+0000 7f8eff400640 1 -- 172.21.15.86:0/1292477638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ef8070330 msgr2=0x7f8ef80d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:21.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.583+0000 7f8eff400640 1 --2- 172.21.15.86:0/1292477638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ef8070330 0x7f8ef80d16e0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009870 tx=0x7f8eec01c2b0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.583+0000 7f8eff400640 1 -- 172.21.15.86:0/1292477638 shutdown_connections 2024-04-25T14:13:21.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.583+0000 7f8eff400640 1 --2- 172.21.15.86:0/1292477638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ef80ce790 0x7f8ef806f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.583+0000 7f8eff400640 1 --2- 172.21.15.86:0/1292477638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ef8070330 0x7f8ef80d16e0 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.583+0000 7f8eff400640 1 -- 172.21.15.86:0/1292477638 >> 172.21.15.86:0/1292477638 conn(0x7f8ef80b4650 msgr2=0x7f8ef80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:21.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.583+0000 7f8eff400640 1 -- 172.21.15.86:0/1292477638 shutdown_connections 2024-04-25T14:13:21.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.583+0000 7f8eff400640 1 -- 172.21.15.86:0/1292477638 wait complete. 2024-04-25T14:13:21.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.587+0000 7f8eff400640 1 Processor -- start 2024-04-25T14:13:21.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.587+0000 7f8eff400640 1 -- start start 2024-04-25T14:13:21.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.587+0000 7f8eff400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ef8070330 0x7f8ef80c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:21.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.587+0000 7f8eff400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ef80ce790 0x7f8ef81f95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:21.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.587+0000 7f8eff400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ef80d1c20 0x7f8ef81595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:21.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.587+0000 7f8eff400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef80708c0 con 0x7f8ef80d1c20 2024-04-25T14:13:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.587+0000 7f8eff400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef8070740 con 0x7f8ef8070330 2024-04-25T14:13:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.587+0000 7f8eff400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8ef816afc0 con 0x7f8ef80ce790 2024-04-25T14:13:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.587+0000 7f8efd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ef80d1c20 0x7f8ef81595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.587+0000 7f8efd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ef80d1c20 0x7f8ef81595e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41336/0 (socket says 172.21.15.86:41336) 2024-04-25T14:13:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.587+0000 7f8efd400640 1 -- 172.21.15.86:0/1625184903 learned_addr learned my addr 172.21.15.86:0/1625184903 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.587+0000 7f8efca00640 1 --2- 172.21.15.86:0/1625184903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ef8070330 0x7f8ef80c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.587+0000 7f8ef7e00640 1 --2- 172.21.15.86:0/1625184903 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ef80ce790 0x7f8ef81f95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.587+0000 7f8efd400640 1 -- 172.21.15.86:0/1625184903 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ef80ce790 msgr2=0x7f8ef81f95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.587+0000 7f8efd400640 1 --2- 172.21.15.86:0/1625184903 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ef80ce790 0x7f8ef81f95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.587+0000 7f8efd400640 1 -- 172.21.15.86:0/1625184903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ef8070330 msgr2=0x7f8ef80c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.587+0000 7f8efd400640 1 --2- 172.21.15.86:0/1625184903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ef8070330 0x7f8ef80c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.587+0000 7f8efd400640 1 -- 172.21.15.86:0/1625184903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef81f9b30 con 0x7f8ef80d1c20 2024-04-25T14:13:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.587+0000 7f8ef7e00640 1 --2- 172.21.15.86:0/1625184903 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ef80ce790 0x7f8ef81f95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:21.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.587+0000 7f8efd400640 1 --2- 172.21.15.86:0/1625184903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ef80d1c20 0x7f8ef81595e0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f8ef000cd80 tx=0x7f8ef000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:21.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.587+0000 7f8ef4c00640 1 -- 172.21.15.86:0/1625184903 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef000f680 con 0x7f8ef80d1c20 2024-04-25T14:13:21.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.587+0000 7f8ef4c00640 1 -- 172.21.15.86:0/1625184903 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ef000f840 con 0x7f8ef80d1c20 2024-04-25T14:13:21.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.587+0000 7f8eff400640 1 -- 172.21.15.86:0/1625184903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef809b450 con 0x7f8ef80d1c20 2024-04-25T14:13:21.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.587+0000 7f8ef4c00640 1 -- 172.21.15.86:0/1625184903 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef000fc30 con 0x7f8ef80d1c20 2024-04-25T14:13:21.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.587+0000 7f8eff400640 1 -- 172.21.15.86:0/1625184903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef809b9d0 con 0x7f8ef80d1c20 2024-04-25T14:13:21.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.591+0000 7f8eff400640 1 -- 172.21.15.86:0/1625184903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ef8069d00 con 0x7f8ef80d1c20 2024-04-25T14:13:21.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.591+0000 7f8ef4c00640 1 -- 172.21.15.86:0/1625184903 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 77) ==== 93253+0+0 (secure 0 0 0) 0x7f8ef00213f0 con 0x7f8ef80d1c20 2024-04-25T14:13:21.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.591+0000 7f8ef4c00640 1 --2- 172.21.15.86:0/1625184903 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f8ed806f4c0 0x7f8ed80719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:21.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.591+0000 7f8ef4c00640 1 -- 172.21.15.86:0/1625184903 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6876+0+0 (secure 0 0 0) 0x7f8ef0090680 con 0x7f8ef80d1c20 2024-04-25T14:13:21.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.591+0000 7f8efca00640 1 --2- 172.21.15.86:0/1625184903 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f8ed806f4c0 0x7f8ed80719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:21.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.591+0000 7f8efca00640 1 --2- 172.21.15.86:0/1625184903 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f8ed806f4c0 0x7f8ed80719d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0ad000 tx=0x7f8eec0a6000 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:21.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.595+0000 7f8ef4c00640 1 -- 172.21.15.86:0/1625184903 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8ef0014030 con 0x7f8ef80d1c20 2024-04-25T14:13:21.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.755+0000 7f8eff400640 1 -- 172.21.15.86:0/1625184903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ef806a1b0 con 0x7f8ef80d1c20 2024-04-25T14:13:21.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.755+0000 7f8ef4c00640 1 -- 172.21.15.86:0/1625184903 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+18036 (secure 0 0 0) 0x7f8ef005cee0 con 0x7f8ef80d1c20 2024-04-25T14:13:21.754 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:13:21.754 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":144,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:13:18.050429+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:11:30.810321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T14:11:31.944888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.e","mappings":[{"from":3,"to":1}]},{"pgid":"22.32","mappings":[{"from":3,"to":0}]},{"pgid":"22.34","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/3190429862":"2024-04-26T14:13:18.050389+0000","172.21.15.86:0/3060165022":"2024-04-26T14:13:18.050389+0000","172.21.15.86:6832/1173654665":"2024-04-26T14:13:18.050389+0000","172.21.15.86:0/2751178284":"2024-04-26T14:13:18.050389+0000","172.21.15.86:6833/1173654665":"2024-04-26T14:13:18.050389+0000","172.21.15.86:0/4083287014":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/1048620009":"2024-04-26T14:13:18.050389+0000","172.21.15.86:0/746190999":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/1426015871":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/3750847077":"2024-04-26T14:13:18.050389+0000","172.21.15.86:0/3962343281":"2024-04-26T14:11:51.781677+0000","172.21.15.86:6833/264890479":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/3349072784":"2024-04-26T14:13:18.050389+0000","172.21.15.86:6832/264890479":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/2724786983":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/285240843":"2024-04-26T14:11:51.781677+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:13:21.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.759+0000 7f8eff400640 1 -- 172.21.15.86:0/1625184903 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f8ed806f4c0 msgr2=0x7f8ed80719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:21.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.759+0000 7f8eff400640 1 --2- 172.21.15.86:0/1625184903 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f8ed806f4c0 0x7f8ed80719d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0ad000 tx=0x7f8eec0a6000 comp rx=0 tx=0).stop 2024-04-25T14:13:21.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.759+0000 7f8eff400640 1 -- 172.21.15.86:0/1625184903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ef80d1c20 msgr2=0x7f8ef81595e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:21.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.759+0000 7f8eff400640 1 --2- 172.21.15.86:0/1625184903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ef80d1c20 0x7f8ef81595e0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f8ef000cd80 tx=0x7f8ef000a650 comp rx=0 tx=0).stop 2024-04-25T14:13:21.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.759+0000 7f8eff400640 1 -- 172.21.15.86:0/1625184903 shutdown_connections 2024-04-25T14:13:21.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.759+0000 7f8eff400640 1 --2- 172.21.15.86:0/1625184903 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f8ed806f4c0 0x7f8ed80719d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.759+0000 7f8eff400640 1 --2- 172.21.15.86:0/1625184903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ef80d1c20 0x7f8ef81595e0 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.759+0000 7f8eff400640 1 --2- 172.21.15.86:0/1625184903 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ef80ce790 0x7f8ef81f95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.759+0000 7f8eff400640 1 --2- 172.21.15.86:0/1625184903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ef8070330 0x7f8ef80c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.759+0000 7f8eff400640 1 -- 172.21.15.86:0/1625184903 >> 172.21.15.86:0/1625184903 conn(0x7f8ef80b4650 msgr2=0x7f8ef815bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:21.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.759+0000 7f8eff400640 1 -- 172.21.15.86:0/1625184903 shutdown_connections 2024-04-25T14:13:21.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.759+0000 7f8eff400640 1 -- 172.21.15.86:0/1625184903 wait complete. 2024-04-25T14:13:21.777 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T14:13:21.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.927+0000 7f2eaf000640 1 Processor -- start 2024-04-25T14:13:21.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.927+0000 7f2eaf000640 1 -- start start 2024-04-25T14:13:21.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.927+0000 7f2eaf000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2ea80cfca0 0x7f2ea806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:21.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.927+0000 7f2eaf000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ea8072bd0 con 0x7f2ea80d16e0 2024-04-25T14:13:21.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.927+0000 7f2eaf000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ea806f930 con 0x7f2ea80730b0 2024-04-25T14:13:21.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.927+0000 7f2eaf000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f2ea8070470 con 0x7f2ea80cfca0 2024-04-25T14:13:21.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.927+0000 7f2ea7400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2ea80d16e0 0x7f2ea80d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47378/0 (socket says 172.21.15.86:47378) 2024-04-25T14:13:21.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.927+0000 7f2ea7400640 1 -- 172.21.15.86:0/392710510 learned_addr learned my addr 172.21.15.86:0/392710510 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:21.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.927+0000 7f2ea7e00640 1 --2- 172.21.15.86:0/392710510 >> v2:172.21.15.155:3300/0 conn(0x7f2ea80cfca0 0x7f2ea806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:21.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.927+0000 7f2ea6a00640 1 -- 172.21.15.86:0/392710510 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 869968050 0 0) 0x7f2ea8072bd0 con 0x7f2ea80d16e0 2024-04-25T14:13:21.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.927+0000 7f2ea6a00640 1 -- 172.21.15.86:0/392710510 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e8c005740 con 0x7f2ea80d16e0 2024-04-25T14:13:21.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.927+0000 7f2ea6a00640 1 -- 172.21.15.86:0/392710510 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2496054203 0 0) 0x7f2e8c005740 con 0x7f2ea80d16e0 2024-04-25T14:13:21.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.927+0000 7f2ea6a00640 1 -- 172.21.15.86:0/392710510 >> v2:172.21.15.155:3300/0 conn(0x7f2ea80cfca0 msgr2=0x7f2ea806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:21.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.927+0000 7f2ea6a00640 1 --2- 172.21.15.86:0/392710510 >> v2:172.21.15.155:3300/0 conn(0x7f2ea80cfca0 0x7f2ea806f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.927+0000 7f2ea6a00640 1 -- 172.21.15.86:0/392710510 >> v1:172.21.15.155:6789/0 conn(0x7f2ea80730b0 legacy=0x7f2ea80734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:21.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.927+0000 7f2ea6a00640 1 -- 172.21.15.86:0/392710510 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea806fbd0 con 0x7f2ea80d16e0 2024-04-25T14:13:21.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.927+0000 7f2ea6a00640 1 -- 172.21.15.86:0/392710510 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f2e94001680 con 0x7f2ea80d16e0 2024-04-25T14:13:21.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.927+0000 7f2ea7e00640 1 --2- 172.21.15.86:0/392710510 >> v2:172.21.15.155:3300/0 conn(0x7f2ea80cfca0 0x7f2ea806f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:13:21.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.927+0000 7f2ea6a00640 1 -- 172.21.15.86:0/392710510 >> v1:172.21.15.86:6789/0 conn(0x7f2ea80d16e0 legacy=0x7f2ea80d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:21.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.927+0000 7f2ea6a00640 1 --2- 172.21.15.86:0/392710510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e8c005c40 0x7f2e8c0060b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:21.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.927+0000 7f2ea6a00640 1 --2- 172.21.15.86:0/392710510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e8c006a20 0x7f2e8c006e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:21.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.931+0000 7f2ea6a00640 1 --2- 172.21.15.86:0/392710510 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e8c007440 0x7f2e8c00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:21.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.931+0000 7f2ea6a00640 1 -- 172.21.15.86:0/392710510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea8070470 con 0x7f2e8c006a20 2024-04-25T14:13:21.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.931+0000 7f2ea6a00640 1 -- 172.21.15.86:0/392710510 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c00e3d0 con 0x7f2e8c005c40 2024-04-25T14:13:21.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.931+0000 7f2ea6a00640 1 -- 172.21.15.86:0/392710510 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2e8c00e5d0 con 0x7f2e8c007440 2024-04-25T14:13:21.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.931+0000 7f2ea7400640 1 --2- 172.21.15.86:0/392710510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e8c006a20 0x7f2e8c006e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:21.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.931+0000 7f2ea7e00640 1 --2- 172.21.15.86:0/392710510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e8c005c40 0x7f2e8c0060b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:21.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.931+0000 7f2ead000640 1 --2- 172.21.15.86:0/392710510 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e8c007440 0x7f2e8c00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:21.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.931+0000 7f2ea7400640 1 -- 172.21.15.86:0/392710510 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e8c007440 msgr2=0x7f2e8c00dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:21.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.931+0000 7f2ea7400640 1 --2- 172.21.15.86:0/392710510 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e8c007440 0x7f2e8c00dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.931+0000 7f2ea7400640 1 -- 172.21.15.86:0/392710510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e8c005c40 msgr2=0x7f2e8c0060b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:21.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.931+0000 7f2ea7400640 1 --2- 172.21.15.86:0/392710510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e8c005c40 0x7f2e8c0060b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.931+0000 7f2ea7400640 1 -- 172.21.15.86:0/392710510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8c005740 con 0x7f2e8c006a20 2024-04-25T14:13:21.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.931+0000 7f2ead000640 1 -- 172.21.15.86:0/392710510 reap_dead start 2024-04-25T14:13:21.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.931+0000 7f2ea7400640 1 --2- 172.21.15.86:0/392710510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e8c006a20 0x7f2e8c006e30 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f2e9400c210 tx=0x7f2e9401f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d3a0988e5dd30b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:21.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.931+0000 7f2ea6a00640 1 -- 172.21.15.86:0/392710510 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e94007d20 con 0x7f2e8c006a20 2024-04-25T14:13:21.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.931+0000 7f2ea6a00640 1 -- 172.21.15.86:0/392710510 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e94005ae0 con 0x7f2e8c006a20 2024-04-25T14:13:21.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.931+0000 7f2ea6a00640 1 -- 172.21.15.86:0/392710510 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e94007070 con 0x7f2e8c006a20 2024-04-25T14:13:21.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.935+0000 7f2eaf000640 1 -- 172.21.15.86:0/392710510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e8c006a20 msgr2=0x7f2e8c006e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:21.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.935+0000 7f2eaf000640 1 --2- 172.21.15.86:0/392710510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e8c006a20 0x7f2e8c006e30 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f2e9400c210 tx=0x7f2e9401f3b0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.935+0000 7f2eaf000640 1 -- 172.21.15.86:0/392710510 shutdown_connections 2024-04-25T14:13:21.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.935+0000 7f2eaf000640 1 --2- 172.21.15.86:0/392710510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e8c006a20 0x7f2e8c006e30 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.935+0000 7f2eaf000640 1 -- 172.21.15.86:0/392710510 >> 172.21.15.86:0/392710510 conn(0x7f2ea80b4650 msgr2=0x7f2ea80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:21.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.935+0000 7f2eaf000640 1 -- 172.21.15.86:0/392710510 shutdown_connections 2024-04-25T14:13:21.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.935+0000 7f2eaf000640 1 -- 172.21.15.86:0/392710510 wait complete. 2024-04-25T14:13:21.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.935+0000 7f2eaf000640 1 Processor -- start 2024-04-25T14:13:21.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.935+0000 7f2eaf000640 1 -- start start 2024-04-25T14:13:21.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.939+0000 7f2eaf000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e8c006a20 0x7f2ea80cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:21.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.939+0000 7f2eaf000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ea815f920 0x7f2ea80af500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:21.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.939+0000 7f2eaf000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ea80afa40 0x7f2ea8159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:21.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.939+0000 7f2eaf000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea806f3f0 con 0x7f2ea80afa40 2024-04-25T14:13:21.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.939+0000 7f2eaf000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea8070190 con 0x7f2e8c006a20 2024-04-25T14:13:21.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.939+0000 7f2eaf000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2ea806f570 con 0x7f2ea815f920 2024-04-25T14:13:21.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.939+0000 7f2ead000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ea80afa40 0x7f2ea8159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:21.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.939+0000 7f2ead000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ea80afa40 0x7f2ea8159f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41340/0 (socket says 172.21.15.86:41340) 2024-04-25T14:13:21.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.939+0000 7f2ead000640 1 -- 172.21.15.86:0/1941956344 learned_addr learned my addr 172.21.15.86:0/1941956344 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:21.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.939+0000 7f2ea7e00640 1 --2- 172.21.15.86:0/1941956344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e8c006a20 0x7f2ea80cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:21.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.939+0000 7f2ea7400640 1 --2- 172.21.15.86:0/1941956344 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ea815f920 0x7f2ea80af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:21.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.939+0000 7f2ead000640 1 -- 172.21.15.86:0/1941956344 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ea815f920 msgr2=0x7f2ea80af500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:21.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.939+0000 7f2ead000640 1 --2- 172.21.15.86:0/1941956344 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ea815f920 0x7f2ea80af500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.939+0000 7f2ead000640 1 -- 172.21.15.86:0/1941956344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e8c006a20 msgr2=0x7f2ea80cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:21.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.939+0000 7f2ead000640 1 --2- 172.21.15.86:0/1941956344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e8c006a20 0x7f2ea80cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:21.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.939+0000 7f2ead000640 1 -- 172.21.15.86:0/1941956344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea8160350 con 0x7f2ea80afa40 2024-04-25T14:13:21.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.939+0000 7f2ea7400640 1 --2- 172.21.15.86:0/1941956344 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ea815f920 0x7f2ea80af500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:21.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.939+0000 7f2ead000640 1 --2- 172.21.15.86:0/1941956344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ea80afa40 0x7f2ea8159f50 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f2ea806dd30 tx=0x7f2ea000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:21.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.939+0000 7f2e9fe00640 1 -- 172.21.15.86:0/1941956344 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea0003980 con 0x7f2ea80afa40 2024-04-25T14:13:21.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.939+0000 7f2e9fe00640 1 -- 172.21.15.86:0/1941956344 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ea0018070 con 0x7f2ea80afa40 2024-04-25T14:13:21.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.939+0000 7f2e9fe00640 1 -- 172.21.15.86:0/1941956344 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea0003da0 con 0x7f2ea80afa40 2024-04-25T14:13:21.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.939+0000 7f2eaf000640 1 -- 172.21.15.86:0/1941956344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ea80afe50 con 0x7f2ea80afa40 2024-04-25T14:13:21.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.939+0000 7f2eaf000640 1 -- 172.21.15.86:0/1941956344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ea80cc790 con 0x7f2ea80afa40 2024-04-25T14:13:21.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.939+0000 7f2eaf000640 1 -- 172.21.15.86:0/1941956344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ea809f4b0 con 0x7f2ea80afa40 2024-04-25T14:13:21.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.939+0000 7f2e9fe00640 1 -- 172.21.15.86:0/1941956344 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 77) ==== 93253+0+0 (secure 0 0 0) 0x7f2ea00050d0 con 0x7f2ea80afa40 2024-04-25T14:13:21.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.943+0000 7f2e9fe00640 1 --2- 172.21.15.86:0/1941956344 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f2e8806f4c0 0x7f2e880719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:21.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.943+0000 7f2e9fe00640 1 -- 172.21.15.86:0/1941956344 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6876+0+0 (secure 0 0 0) 0x7f2ea008f380 con 0x7f2ea80afa40 2024-04-25T14:13:21.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.943+0000 7f2ea7e00640 1 --2- 172.21.15.86:0/1941956344 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f2e8806f4c0 0x7f2e880719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:21.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.943+0000 7f2ea7e00640 1 --2- 172.21.15.86:0/1941956344 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f2e8806f4c0 0x7f2e880719d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2ea806b910 tx=0x7f2e98008040 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:21.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:21.947+0000 7f2e9fe00640 1 -- 172.21.15.86:0/1941956344 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2ea005bf10 con 0x7f2ea80afa40 2024-04-25T14:13:22.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.091+0000 7f2e9fe00640 1 -- 172.21.15.86:0/1941956344 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7f2ea0057aa0 con 0x7f2ea80afa40 2024-04-25T14:13:22.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.143+0000 7f2eaf000640 1 -- 172.21.15.86:0/1941956344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2ea8069d00 con 0x7f2ea80afa40 2024-04-25T14:13:22.141 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-25T14:13:22.143+0000 7fea29c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:13:22.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.143+0000 7f2e9fe00640 1 -- 172.21.15.86:0/1941956344 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v109) ==== 161+0+0 (secure 0 0 0) 0x7f2ea0003980 con 0x7f2ea80afa40 2024-04-25T14:13:22.147 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T14:13:22.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.147+0000 7f2eaf000640 1 -- 172.21.15.86:0/1941956344 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f2e8806f4c0 msgr2=0x7f2e880719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:22.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.147+0000 7f2eaf000640 1 --2- 172.21.15.86:0/1941956344 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f2e8806f4c0 0x7f2e880719d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2ea806b910 tx=0x7f2e98008040 comp rx=0 tx=0).stop 2024-04-25T14:13:22.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.147+0000 7f2eaf000640 1 -- 172.21.15.86:0/1941956344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ea80afa40 msgr2=0x7f2ea8159f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.147+0000 7f2eaf000640 1 --2- 172.21.15.86:0/1941956344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ea80afa40 0x7f2ea8159f50 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f2ea806dd30 tx=0x7f2ea000ec10 comp rx=0 tx=0).stop 2024-04-25T14:13:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.147+0000 7f2eaf000640 1 -- 172.21.15.86:0/1941956344 shutdown_connections 2024-04-25T14:13:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.147+0000 7f2eaf000640 1 --2- 172.21.15.86:0/1941956344 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f2e8806f4c0 0x7f2e880719d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.147+0000 7f2eaf000640 1 --2- 172.21.15.86:0/1941956344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ea80afa40 0x7f2ea8159f50 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.147+0000 7f2eaf000640 1 --2- 172.21.15.86:0/1941956344 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ea815f920 0x7f2ea80af500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.147+0000 7f2eaf000640 1 --2- 172.21.15.86:0/1941956344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e8c006a20 0x7f2ea80cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.147+0000 7f2eaf000640 1 -- 172.21.15.86:0/1941956344 >> 172.21.15.86:0/1941956344 conn(0x7f2ea80b4650 msgr2=0x7f2ea80734c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.147+0000 7f2eaf000640 1 -- 172.21.15.86:0/1941956344 shutdown_connections 2024-04-25T14:13:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.147+0000 7f2eaf000640 1 -- 172.21.15.86:0/1941956344 wait complete. 2024-04-25T14:13:22.168 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T14:13:22.279 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-25T14:13:22.287 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2024-04-25T14:13:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.323+0000 7f309b200640 1 Processor -- start 2024-04-25T14:13:22.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.323+0000 7f309b200640 1 -- start start 2024-04-25T14:13:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.327+0000 7f309b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30940cf970 0x7f3094070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.327+0000 7f309b200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f309406e970 0x7f30940cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.327+0000 7f309b200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3094071770 con 0x7f3094071360 2024-04-25T14:13:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.327+0000 7f309b200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3094163830 con 0x7f309406e970 2024-04-25T14:13:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.327+0000 7f309b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f309406ef90 con 0x7f30940cf970 2024-04-25T14:13:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.327+0000 7f3099200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f309406e970 0x7f30940cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.327+0000 7f3099200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f309406e970 0x7f30940cdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41354/0 (socket says 172.21.15.86:41354) 2024-04-25T14:13:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.327+0000 7f3099200640 1 -- 172.21.15.86:0/475727955 learned_addr learned my addr 172.21.15.86:0/475727955 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.327+0000 7f3093e00640 1 --2- 172.21.15.86:0/475727955 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30940cf970 0x7f3094070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:22.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.327+0000 7f3099200640 1 -- 172.21.15.86:0/475727955 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30940cf970 msgr2=0x7f3094070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:22.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.327+0000 7f3099200640 1 --2- 172.21.15.86:0/475727955 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30940cf970 0x7f3094070e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:22.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.327+0000 7f3099200640 1 -- 172.21.15.86:0/475727955 >> v1:172.21.15.155:6789/0 conn(0x7f3094071360 legacy=0x7f309406e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:22.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.327+0000 7f3099200640 1 -- 172.21.15.86:0/475727955 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3094163a80 con 0x7f309406e970 2024-04-25T14:13:22.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.327+0000 7f3093400640 1 -- 172.21.15.86:0/475727955 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4226981185 0 0) 0x7f307c001170 con 0x7f3094071360 2024-04-25T14:13:22.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.331+0000 7f3099200640 1 --2- 172.21.15.86:0/475727955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f309406e970 0x7f30940cdde0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f30941639b0 tx=0x7f308c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c0e0d4bccfbd9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:22.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.331+0000 7f3093400640 1 -- 172.21.15.86:0/475727955 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c0a57f0 con 0x7f309406e970 2024-04-25T14:13:22.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.331+0000 7f3093400640 1 -- 172.21.15.86:0/475727955 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f308c0a5990 con 0x7f309406e970 2024-04-25T14:13:22.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.331+0000 7f3093400640 1 -- 172.21.15.86:0/475727955 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c0a5d60 con 0x7f309406e970 2024-04-25T14:13:22.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.331+0000 7f309b200640 1 -- 172.21.15.86:0/475727955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f309406e970 msgr2=0x7f30940cdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:22.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.331+0000 7f309b200640 1 --2- 172.21.15.86:0/475727955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f309406e970 0x7f30940cdde0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f30941639b0 tx=0x7f308c01ca30 comp rx=0 tx=0).stop 2024-04-25T14:13:22.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.331+0000 7f309b200640 1 -- 172.21.15.86:0/475727955 shutdown_connections 2024-04-25T14:13:22.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.331+0000 7f309b200640 1 --2- 172.21.15.86:0/475727955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f309406e970 0x7f30940cdde0 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:22.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.331+0000 7f309b200640 1 --2- 172.21.15.86:0/475727955 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30940cf970 0x7f3094070e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:22.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.331+0000 7f309b200640 1 -- 172.21.15.86:0/475727955 >> 172.21.15.86:0/475727955 conn(0x7f30940cffc0 msgr2=0x7f30940d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:22.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.331+0000 7f309b200640 1 -- 172.21.15.86:0/475727955 shutdown_connections 2024-04-25T14:13:22.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.331+0000 7f309b200640 1 -- 172.21.15.86:0/475727955 wait complete. 2024-04-25T14:13:22.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.331+0000 7f309b200640 1 Processor -- start 2024-04-25T14:13:22.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.331+0000 7f309b200640 1 -- start start 2024-04-25T14:13:22.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.335+0000 7f309b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f309406e970 0x7f30941576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:22.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.335+0000 7f309b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3094071360 0x7f3094159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:22.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.335+0000 7f309b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30940cf970 0x7f309415a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:22.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.335+0000 7f309b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f309416cbd0 con 0x7f30940cf970 2024-04-25T14:13:22.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.335+0000 7f309b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f30940cedc0 con 0x7f3094071360 2024-04-25T14:13:22.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.335+0000 7f309b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f309416cd50 con 0x7f309406e970 2024-04-25T14:13:22.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.335+0000 7f3099200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30940cf970 0x7f309415a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:22.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.335+0000 7f3093e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f309406e970 0x7f30941576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:22.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.335+0000 7f3099200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30940cf970 0x7f309415a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41358/0 (socket says 172.21.15.86:41358) 2024-04-25T14:13:22.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.335+0000 7f3099200640 1 -- 172.21.15.86:0/2536881137 learned_addr learned my addr 172.21.15.86:0/2536881137 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:22.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.335+0000 7f308b400640 1 --2- 172.21.15.86:0/2536881137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3094071360 0x7f3094159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:22.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.335+0000 7f3099200640 1 -- 172.21.15.86:0/2536881137 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f309406e970 msgr2=0x7f30941576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:22.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.335+0000 7f3099200640 1 --2- 172.21.15.86:0/2536881137 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f309406e970 0x7f30941576a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:22.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.335+0000 7f3099200640 1 -- 172.21.15.86:0/2536881137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3094071360 msgr2=0x7f3094159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:22.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.335+0000 7f3099200640 1 --2- 172.21.15.86:0/2536881137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3094071360 0x7f3094159d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:22.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.335+0000 7f3099200640 1 -- 172.21.15.86:0/2536881137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30940cba00 con 0x7f30940cf970 2024-04-25T14:13:22.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.335+0000 7f3099200640 1 --2- 172.21.15.86:0/2536881137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30940cf970 0x7f309415a2b0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f308c0af040 tx=0x7f308c0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:22.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.335+0000 7f3090c00640 1 -- 172.21.15.86:0/2536881137 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c0b8670 con 0x7f30940cf970 2024-04-25T14:13:22.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.335+0000 7f3090c00640 1 -- 172.21.15.86:0/2536881137 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f308c0be070 con 0x7f30940cf970 2024-04-25T14:13:22.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.335+0000 7f309b200640 1 -- 172.21.15.86:0/2536881137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30940c9e30 con 0x7f30940cf970 2024-04-25T14:13:22.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.335+0000 7f3090c00640 1 -- 172.21.15.86:0/2536881137 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c0b8a90 con 0x7f30940cf970 2024-04-25T14:13:22.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.335+0000 7f309b200640 1 -- 172.21.15.86:0/2536881137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30940ca0d0 con 0x7f30940cf970 2024-04-25T14:13:22.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.335+0000 7f309b200640 1 -- 172.21.15.86:0/2536881137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30940cbbd0 con 0x7f30940cf970 2024-04-25T14:13:22.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.339+0000 7f3090c00640 1 -- 172.21.15.86:0/2536881137 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7f308c0b8c30 con 0x7f30940cf970 2024-04-25T14:13:22.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.339+0000 7f3090c00640 1 --2- 172.21.15.86:0/2536881137 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f307006f520 0x7f3070071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:22.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.339+0000 7f3090c00640 1 -- 172.21.15.86:0/2536881137 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 7048+0+0 (secure 0 0 0) 0x7f308c12a820 con 0x7f30940cf970 2024-04-25T14:13:22.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.339+0000 7f3093e00640 1 --2- 172.21.15.86:0/2536881137 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f307006f520 0x7f3070071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:22.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.339+0000 7f3093e00640 1 --2- 172.21.15.86:0/2536881137 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f307006f520 0x7f3070071a30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3084009ac0 tx=0x7f3084009190 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:22.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.343+0000 7f3090c00640 1 -- 172.21.15.86:0/2536881137 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f308c016620 con 0x7f30940cf970 2024-04-25T14:13:22.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:22.535+0000 7f309b200640 1 -- 172.21.15.86:0/2536881137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f30940720e0 con 0x7f30940cf970 2024-04-25T14:13:23.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.147+0000 7f3090c00640 1 -- 172.21.15.86:0/2536881137 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v110) ==== 104+0+0 (secure 0 0 0) 0x7f308c0a5960 con 0x7f30940cf970 2024-04-25T14:13:23.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.151+0000 7f309b200640 1 -- 172.21.15.86:0/2536881137 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f307006f520 msgr2=0x7f3070071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:23.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.151+0000 7f309b200640 1 --2- 172.21.15.86:0/2536881137 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f307006f520 0x7f3070071a30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3084009ac0 tx=0x7f3084009190 comp rx=0 tx=0).stop 2024-04-25T14:13:23.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.151+0000 7f309b200640 1 -- 172.21.15.86:0/2536881137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30940cf970 msgr2=0x7f309415a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:23.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.151+0000 7f309b200640 1 --2- 172.21.15.86:0/2536881137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30940cf970 0x7f309415a2b0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f308c0af040 tx=0x7f308c0197d0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.151+0000 7f309b200640 1 -- 172.21.15.86:0/2536881137 shutdown_connections 2024-04-25T14:13:23.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.151+0000 7f309b200640 1 --2- 172.21.15.86:0/2536881137 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f307006f520 0x7f3070071a30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.151+0000 7f309b200640 1 --2- 172.21.15.86:0/2536881137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30940cf970 0x7f309415a2b0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.151+0000 7f309b200640 1 --2- 172.21.15.86:0/2536881137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3094071360 0x7f3094159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.151+0000 7f309b200640 1 --2- 172.21.15.86:0/2536881137 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f309406e970 0x7f30941576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.151+0000 7f309b200640 1 -- 172.21.15.86:0/2536881137 >> 172.21.15.86:0/2536881137 conn(0x7f30940cffc0 msgr2=0x7f309415fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.151+0000 7f309b200640 1 -- 172.21.15.86:0/2536881137 shutdown_connections 2024-04-25T14:13:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.151+0000 7f309b200640 1 -- 172.21.15.86:0/2536881137 wait complete. 2024-04-25T14:13:23.173 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T14:13:23.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.327+0000 7fb2b6a00640 1 Processor -- start 2024-04-25T14:13:23.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.327+0000 7fb2b6a00640 1 -- start start 2024-04-25T14:13:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.327+0000 7fb2b6a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2b0072e10 0x7fb2b015d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.327+0000 7fb2b6a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb2b015d960 0x7fb2b006f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.327+0000 7fb2b6a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2b00af2e0 con 0x7fb2b015d550 2024-04-25T14:13:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.327+0000 7fb2b6a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb2b0073220 con 0x7fb2b015d960 2024-04-25T14:13:23.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.327+0000 7fb2b6a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb2b0073470 con 0x7fb2b0072e10 2024-04-25T14:13:23.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.327+0000 7fb2b6000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb2b015d960 0x7fb2b006f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:23.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.327+0000 7fb2b5600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2b0072e10 0x7fb2b015d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.327+0000 7fb2b5600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2b0072e10 0x7fb2b015d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:55788/0 (socket says 172.21.15.86:55788) 2024-04-25T14:13:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.327+0000 7fb2b5600640 1 -- 172.21.15.86:0/2757431031 learned_addr learned my addr 172.21.15.86:0/2757431031 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.327+0000 7fb2b4c00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fb2b015d550 0x7fb2b006eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:51138/0 (socket says 172.21.15.86:51138) 2024-04-25T14:13:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.327+0000 7fb2b6000640 1 -- 172.21.15.86:0/2757431031 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2b0072e10 msgr2=0x7fb2b015d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.327+0000 7fb2b6000640 1 --2- 172.21.15.86:0/2757431031 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2b0072e10 0x7fb2b015d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.331+0000 7fb2b6000640 1 -- 172.21.15.86:0/2757431031 >> v1:172.21.15.155:6789/0 conn(0x7fb2b015d550 legacy=0x7fb2b006eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.331+0000 7fb2b6000640 1 -- 172.21.15.86:0/2757431031 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b0073750 con 0x7fb2b015d960 2024-04-25T14:13:23.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.331+0000 7fb2b5600640 1 --2- 172.21.15.86:0/2757431031 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2b0072e10 0x7fb2b015d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:13:23.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.331+0000 7fb2b6000640 1 --2- 172.21.15.86:0/2757431031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2b015d960 0x7fb2b006f5b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb2a800bfa0 tx=0x7fb2a801eb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7b0ac36e1346473 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:23.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.331+0000 7fb2afe00640 1 -- 172.21.15.86:0/2757431031 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a800b2d0 con 0x7fb2b015d960 2024-04-25T14:13:23.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.331+0000 7fb2afe00640 1 -- 172.21.15.86:0/2757431031 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2a80a7830 con 0x7fb2b015d960 2024-04-25T14:13:23.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.331+0000 7fb2afe00640 1 -- 172.21.15.86:0/2757431031 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a80a7c00 con 0x7fb2b015d960 2024-04-25T14:13:23.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.335+0000 7fb2b6a00640 1 -- 172.21.15.86:0/2757431031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2b015d960 msgr2=0x7fb2b006f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:23.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.335+0000 7fb2b6a00640 1 --2- 172.21.15.86:0/2757431031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2b015d960 0x7fb2b006f5b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb2a800bfa0 tx=0x7fb2a801eb70 comp rx=0 tx=0).stop 2024-04-25T14:13:23.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.335+0000 7fb2b6a00640 1 -- 172.21.15.86:0/2757431031 shutdown_connections 2024-04-25T14:13:23.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.335+0000 7fb2b6a00640 1 --2- 172.21.15.86:0/2757431031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2b015d960 0x7fb2b006f5b0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.335+0000 7fb2b6a00640 1 --2- 172.21.15.86:0/2757431031 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2b0072e10 0x7fb2b015d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.335+0000 7fb2b6a00640 1 -- 172.21.15.86:0/2757431031 >> 172.21.15.86:0/2757431031 conn(0x7fb2b00b2fc0 msgr2=0x7fb2b00b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:23.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.335+0000 7fb2b6a00640 1 -- 172.21.15.86:0/2757431031 shutdown_connections 2024-04-25T14:13:23.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.335+0000 7fb2b6a00640 1 -- 172.21.15.86:0/2757431031 wait complete. 2024-04-25T14:13:23.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.335+0000 7fb2b6a00640 1 Processor -- start 2024-04-25T14:13:23.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.335+0000 7fb2b6a00640 1 -- start start 2024-04-25T14:13:23.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.335+0000 7fb2b6a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2b0072e10 0x7fb2b01493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:23.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.339+0000 7fb2b6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2b015d550 0x7fb2b01578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:23.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.339+0000 7fb2b6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b015d960 0x7fb2b0155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:23.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.339+0000 7fb2b6a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b0070080 con 0x7fb2b015d960 2024-04-25T14:13:23.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.339+0000 7fb2b6a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b006ff00 con 0x7fb2b015d550 2024-04-25T14:13:23.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.339+0000 7fb2b6a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb2b006d710 con 0x7fb2b0072e10 2024-04-25T14:13:23.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.339+0000 7fb2b5600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2b0072e10 0x7fb2b01493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:23.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.339+0000 7fb2b5600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2b0072e10 0x7fb2b01493a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:55790/0 (socket says 172.21.15.86:55790) 2024-04-25T14:13:23.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.339+0000 7fb2b5600640 1 -- 172.21.15.86:0/2171296858 learned_addr learned my addr 172.21.15.86:0/2171296858 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:23.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.339+0000 7fb2b6000640 1 --2- 172.21.15.86:0/2171296858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b015d960 0x7fb2b0155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:23.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.339+0000 7fb2b4c00640 1 --2- 172.21.15.86:0/2171296858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2b015d550 0x7fb2b01578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:23.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.339+0000 7fb2b5600640 1 -- 172.21.15.86:0/2171296858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2b015d550 msgr2=0x7fb2b01578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:23.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.339+0000 7fb2b5600640 1 --2- 172.21.15.86:0/2171296858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2b015d550 0x7fb2b01578b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.339+0000 7fb2b5600640 1 -- 172.21.15.86:0/2171296858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b015d960 msgr2=0x7fb2b0155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:23.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.339+0000 7fb2b5600640 1 --2- 172.21.15.86:0/2171296858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b015d960 0x7fb2b0155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:23.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.339+0000 7fb2b5600640 1 -- 172.21.15.86:0/2171296858 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b0165250 con 0x7fb2b0072e10 2024-04-25T14:13:23.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.339+0000 7fb2b6000640 1 --2- 172.21.15.86:0/2171296858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b015d960 0x7fb2b0155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:13:23.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.339+0000 7fb2b5600640 1 --2- 172.21.15.86:0/2171296858 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2b0072e10 0x7fb2b01493a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb2a400ea10 tx=0x7fb2a400eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:23.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.339+0000 7fb2ad600640 1 -- 172.21.15.86:0/2171296858 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4017070 con 0x7fb2b0072e10 2024-04-25T14:13:23.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.339+0000 7fb2ad600640 1 -- 172.21.15.86:0/2171296858 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2a400ca00 con 0x7fb2b0072e10 2024-04-25T14:13:23.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.339+0000 7fb2ad600640 1 -- 172.21.15.86:0/2171296858 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a400cdf0 con 0x7fb2b0072e10 2024-04-25T14:13:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.339+0000 7fb2b6a00640 1 -- 172.21.15.86:0/2171296858 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2b0165af0 con 0x7fb2b0072e10 2024-04-25T14:13:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.339+0000 7fb2b6a00640 1 -- 172.21.15.86:0/2171296858 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b0165810 con 0x7fb2b0072e10 2024-04-25T14:13:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.339+0000 7fb2b6a00640 1 -- 172.21.15.86:0/2171296858 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2b0076b80 con 0x7fb2b0072e10 2024-04-25T14:13:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.339+0000 7fb2ad600640 1 -- 172.21.15.86:0/2171296858 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7fb2a401a070 con 0x7fb2b0072e10 2024-04-25T14:13:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.343+0000 7fb2ad600640 1 --2- 172.21.15.86:0/2171296858 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fb29006f550 0x7fb290071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.343+0000 7fb2ad600640 1 -- 172.21.15.86:0/2171296858 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(145..145 src has 1..145) ==== 7048+0+0 (secure 0 0 0) 0x7fb2a408f710 con 0x7fb2b0072e10 2024-04-25T14:13:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.343+0000 7fb2b4c00640 1 --2- 172.21.15.86:0/2171296858 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fb29006f550 0x7fb290071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:23.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.343+0000 7fb2b4c00640 1 --2- 172.21.15.86:0/2171296858 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fb29006f550 0x7fb290071a60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0164a80 tx=0x7fb29c007e90 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:23.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.347+0000 7fb2ad600640 1 -- 172.21.15.86:0/2171296858 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb2a405b9f0 con 0x7fb2b0072e10 2024-04-25T14:13:23.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:23.503+0000 7fb2b6a00640 1 -- 172.21.15.86:0/2171296858 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb2b015fb80 con 0x7fb2b0072e10 2024-04-25T14:13:24.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.163+0000 7fb2ad600640 1 -- 172.21.15.86:0/2171296858 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v146) ==== 181+0+0 (secure 0 0 0) 0x7fb2a405fcf0 con 0x7fb2b0072e10 2024-04-25T14:13:24.161 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-04-25T14:13:24.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.163+0000 7fb2b6a00640 1 -- 172.21.15.86:0/2171296858 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fb29006f550 msgr2=0x7fb290071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:24.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.163+0000 7fb2b6a00640 1 --2- 172.21.15.86:0/2171296858 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fb29006f550 0x7fb290071a60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0164a80 tx=0x7fb29c007e90 comp rx=0 tx=0).stop 2024-04-25T14:13:24.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.163+0000 7fb2b6a00640 1 -- 172.21.15.86:0/2171296858 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2b0072e10 msgr2=0x7fb2b01493a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:24.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.163+0000 7fb2b6a00640 1 --2- 172.21.15.86:0/2171296858 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2b0072e10 0x7fb2b01493a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb2a400ea10 tx=0x7fb2a400eed0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.167+0000 7fb2b6a00640 1 -- 172.21.15.86:0/2171296858 shutdown_connections 2024-04-25T14:13:24.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.167+0000 7fb2b6a00640 1 --2- 172.21.15.86:0/2171296858 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fb29006f550 0x7fb290071a60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.167+0000 7fb2b6a00640 1 --2- 172.21.15.86:0/2171296858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2b015d960 0x7fb2b0155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.167+0000 7fb2b6a00640 1 --2- 172.21.15.86:0/2171296858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2b015d550 0x7fb2b01578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.167+0000 7fb2b6a00640 1 --2- 172.21.15.86:0/2171296858 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2b0072e10 0x7fb2b01493a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.167+0000 7fb2b6a00640 1 -- 172.21.15.86:0/2171296858 >> 172.21.15.86:0/2171296858 conn(0x7fb2b00b2fc0 msgr2=0x7fb2b015f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:24.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.167+0000 7fb2b6a00640 1 -- 172.21.15.86:0/2171296858 shutdown_connections 2024-04-25T14:13:24.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.167+0000 7fb2b6a00640 1 -- 172.21.15.86:0/2171296858 wait complete. 2024-04-25T14:13:24.185 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T14:13:24.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.339+0000 7feea1400640 1 Processor -- start 2024-04-25T14:13:24.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.339+0000 7feea1400640 1 -- start start 2024-04-25T14:13:24.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.339+0000 7feea1400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee9c160d00 0x7fee9c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:24.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.339+0000 7feea1400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee9c15ce60 con 0x7fee9c0743c0 2024-04-25T14:13:24.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.339+0000 7feea1400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee9c072b90 con 0x7fee9c15ca50 2024-04-25T14:13:24.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.339+0000 7feea1400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fee9c072e70 con 0x7fee9c160d00 2024-04-25T14:13:24.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.339+0000 7fee99e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fee9c0743c0 0x7fee9c1605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47388/0 (socket says 172.21.15.86:47388) 2024-04-25T14:13:24.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.339+0000 7fee99e00640 1 -- 172.21.15.86:0/178251814 learned_addr learned my addr 172.21.15.86:0/178251814 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:24.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.339+0000 7fee99400640 1 -- 172.21.15.86:0/178251814 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1376800139 0 0) 0x7fee9c15ce60 con 0x7fee9c0743c0 2024-04-25T14:13:24.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.339+0000 7fee9b200640 1 --2- 172.21.15.86:0/178251814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee9c160d00 0x7fee9c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:24.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.339+0000 7fee99400640 1 -- 172.21.15.86:0/178251814 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee80003530 con 0x7fee9c0743c0 2024-04-25T14:13:24.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.339+0000 7fee99400640 1 -- 172.21.15.86:0/178251814 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 906299900 0 0) 0x7fee80003530 con 0x7fee9c0743c0 2024-04-25T14:13:24.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.339+0000 7fee99400640 1 -- 172.21.15.86:0/178251814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee9c160d00 msgr2=0x7fee9c06e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:24.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.339+0000 7fee99400640 1 --2- 172.21.15.86:0/178251814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee9c160d00 0x7fee9c06e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.339+0000 7fee99400640 1 -- 172.21.15.86:0/178251814 >> v1:172.21.15.155:6789/0 conn(0x7fee9c15ca50 legacy=0x7fee9c073950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:24.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.339+0000 7fee99400640 1 -- 172.21.15.86:0/178251814 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9c073150 con 0x7fee9c0743c0 2024-04-25T14:13:24.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.339+0000 7fee99400640 1 -- 172.21.15.86:0/178251814 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fee88001680 con 0x7fee9c0743c0 2024-04-25T14:13:24.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.339+0000 7fee99400640 1 -- 172.21.15.86:0/178251814 >> v1:172.21.15.86:6789/0 conn(0x7fee9c0743c0 legacy=0x7fee9c1605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:24.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.339+0000 7fee99400640 1 --2- 172.21.15.86:0/178251814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee800039f0 0x7fee80003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:24.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.339+0000 7fee99400640 1 --2- 172.21.15.86:0/178251814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee80004360 0x7fee80004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:24.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.339+0000 7fee99400640 1 --2- 172.21.15.86:0/178251814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee80005040 0x7fee8000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:24.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.339+0000 7fee99400640 1 -- 172.21.15.86:0/178251814 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee9c072e70 con 0x7fee80004360 2024-04-25T14:13:24.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.339+0000 7fee99400640 1 -- 172.21.15.86:0/178251814 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fee8000bf10 con 0x7fee80005040 2024-04-25T14:13:24.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.339+0000 7fee99400640 1 -- 172.21.15.86:0/178251814 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fee8000c110 con 0x7fee800039f0 2024-04-25T14:13:24.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.339+0000 7fee99e00640 1 --2- 172.21.15.86:0/178251814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee80004360 0x7fee80004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:24.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.339+0000 7fee9a800640 1 --2- 172.21.15.86:0/178251814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee800039f0 0x7fee80003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:24.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.339+0000 7fee9b200640 1 --2- 172.21.15.86:0/178251814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee80005040 0x7fee8000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:24.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.343+0000 7fee99e00640 1 -- 172.21.15.86:0/178251814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee800039f0 msgr2=0x7fee80003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:24.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.343+0000 7fee99e00640 1 --2- 172.21.15.86:0/178251814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee800039f0 0x7fee80003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.343+0000 7fee99e00640 1 -- 172.21.15.86:0/178251814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee80005040 msgr2=0x7fee8000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:24.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.343+0000 7fee99e00640 1 --2- 172.21.15.86:0/178251814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee80005040 0x7fee8000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.343+0000 7fee99e00640 1 -- 172.21.15.86:0/178251814 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee80003530 con 0x7fee80004360 2024-04-25T14:13:24.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.343+0000 7fee9a800640 1 --2- 172.21.15.86:0/178251814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee800039f0 0x7fee80003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:24.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.343+0000 7fee99e00640 1 --2- 172.21.15.86:0/178251814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee80004360 0x7fee80004890 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fee8800c160 tx=0x7fee8801cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e789eb290137b47d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:24.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.343+0000 7fee99400640 1 -- 172.21.15.86:0/178251814 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88007cb0 con 0x7fee80004360 2024-04-25T14:13:24.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.343+0000 7fee99400640 1 -- 172.21.15.86:0/178251814 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee880a9020 con 0x7fee80004360 2024-04-25T14:13:24.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.343+0000 7fee99400640 1 -- 172.21.15.86:0/178251814 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88005ae0 con 0x7fee80004360 2024-04-25T14:13:24.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.343+0000 7feea1400640 1 -- 172.21.15.86:0/178251814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee80004360 msgr2=0x7fee80004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:24.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.343+0000 7feea1400640 1 --2- 172.21.15.86:0/178251814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee80004360 0x7fee80004890 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fee8800c160 tx=0x7fee8801cbe0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.343+0000 7fee9b200640 1 -- 172.21.15.86:0/178251814 reap_dead start 2024-04-25T14:13:24.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.343+0000 7feea1400640 1 -- 172.21.15.86:0/178251814 shutdown_connections 2024-04-25T14:13:24.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.343+0000 7feea1400640 1 -- 172.21.15.86:0/178251814 >> 172.21.15.86:0/178251814 conn(0x7fee9c0b4a40 msgr2=0x7fee9c0b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:24.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.343+0000 7feea1400640 1 -- 172.21.15.86:0/178251814 shutdown_connections 2024-04-25T14:13:24.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.343+0000 7feea1400640 1 -- 172.21.15.86:0/178251814 wait complete. 2024-04-25T14:13:24.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.347+0000 7feea1400640 1 Processor -- start 2024-04-25T14:13:24.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.347+0000 7feea1400640 1 -- start start 2024-04-25T14:13:24.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.347+0000 7feea1400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee80004360 0x7fee9c15f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:24.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.347+0000 7feea1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9c09b490 0x7fee9c158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:24.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.347+0000 7feea1400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee9c09b8a0 0x7fee9c1586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:24.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.347+0000 7feea1400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee9c0747d0 con 0x7fee9c09b490 2024-04-25T14:13:24.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.347+0000 7feea1400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fee9c160f50 con 0x7fee9c09b8a0 2024-04-25T14:13:24.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.347+0000 7feea1400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fee9c074950 con 0x7fee80004360 2024-04-25T14:13:24.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.347+0000 7fee99e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9c09b490 0x7fee9c158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:24.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.347+0000 7fee99e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9c09b490 0x7fee9c158190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41378/0 (socket says 172.21.15.86:41378) 2024-04-25T14:13:24.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.347+0000 7fee99e00640 1 -- 172.21.15.86:0/1130542661 learned_addr learned my addr 172.21.15.86:0/1130542661 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:24.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.351+0000 7fee9b200640 1 --2- 172.21.15.86:0/1130542661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee9c09b8a0 0x7fee9c1586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:24.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.351+0000 7fee9a800640 1 --2- 172.21.15.86:0/1130542661 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee80004360 0x7fee9c15f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:24.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.351+0000 7fee99e00640 1 -- 172.21.15.86:0/1130542661 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee80004360 msgr2=0x7fee9c15f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:24.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.351+0000 7fee99e00640 1 --2- 172.21.15.86:0/1130542661 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee80004360 0x7fee9c15f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.351+0000 7fee99e00640 1 -- 172.21.15.86:0/1130542661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee9c09b8a0 msgr2=0x7fee9c1586d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:24.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.351+0000 7fee99e00640 1 --2- 172.21.15.86:0/1130542661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee9c09b8a0 0x7fee9c1586d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:24.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.351+0000 7fee99e00640 1 -- 172.21.15.86:0/1130542661 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9c15fe70 con 0x7fee9c09b490 2024-04-25T14:13:24.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.351+0000 7fee99e00640 1 --2- 172.21.15.86:0/1130542661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9c09b490 0x7fee9c158190 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fee8801ca10 tx=0x7fee80005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:24.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.351+0000 7fee92a00640 1 -- 172.21.15.86:0/1130542661 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee880a5cf0 con 0x7fee9c09b490 2024-04-25T14:13:24.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.351+0000 7fee92a00640 1 -- 172.21.15.86:0/1130542661 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee880047f0 con 0x7fee9c09b490 2024-04-25T14:13:24.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.351+0000 7fee92a00640 1 -- 172.21.15.86:0/1130542661 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88004ba0 con 0x7fee9c09b490 2024-04-25T14:13:24.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.351+0000 7feea1400640 1 -- 172.21.15.86:0/1130542661 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee9c0b1f20 con 0x7fee9c09b490 2024-04-25T14:13:24.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.351+0000 7feea1400640 1 -- 172.21.15.86:0/1130542661 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee9c15d540 con 0x7fee9c09b490 2024-04-25T14:13:24.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.351+0000 7feea1400640 1 -- 172.21.15.86:0/1130542661 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee9c15d050 con 0x7fee9c09b490 2024-04-25T14:13:24.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.351+0000 7fee92a00640 1 -- 172.21.15.86:0/1130542661 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7fee88007070 con 0x7fee9c09b490 2024-04-25T14:13:24.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.355+0000 7fee92a00640 1 --2- 172.21.15.86:0/1130542661 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fee7c06f520 0x7fee7c071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:24.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.355+0000 7fee92a00640 1 -- 172.21.15.86:0/1130542661 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6610+0+0 (secure 0 0 0) 0x7fee881297e0 con 0x7fee9c09b490 2024-04-25T14:13:24.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.355+0000 7fee9a800640 1 --2- 172.21.15.86:0/1130542661 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fee7c06f520 0x7fee7c071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:24.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.355+0000 7fee9a800640 1 --2- 172.21.15.86:0/1130542661 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fee7c06f520 0x7fee7c071a30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fee8c000c10 tx=0x7fee8c00a9c0 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:24.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.359+0000 7fee92a00640 1 -- 172.21.15.86:0/1130542661 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fee880168f0 con 0x7fee9c09b490 2024-04-25T14:13:24.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:24.519+0000 7feea1400640 1 -- 172.21.15.86:0/1130542661 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fee9c050750 con 0x7fee9c09b490 2024-04-25T14:13:25.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.159+0000 7fee92a00640 1 -- 172.21.15.86:0/1130542661 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v147) ==== 169+0+0 (secure 0 0 0) 0x7fee880fe210 con 0x7fee9c09b490 2024-04-25T14:13:25.157 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-04-25T14:13:25.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.163+0000 7feea1400640 1 -- 172.21.15.86:0/1130542661 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fee7c06f520 msgr2=0x7fee7c071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:25.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.163+0000 7feea1400640 1 --2- 172.21.15.86:0/1130542661 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fee7c06f520 0x7fee7c071a30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fee8c000c10 tx=0x7fee8c00a9c0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.163+0000 7feea1400640 1 -- 172.21.15.86:0/1130542661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9c09b490 msgr2=0x7fee9c158190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:25.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.163+0000 7feea1400640 1 --2- 172.21.15.86:0/1130542661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9c09b490 0x7fee9c158190 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fee8801ca10 tx=0x7fee80005840 comp rx=0 tx=0).stop 2024-04-25T14:13:25.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.167+0000 7feea1400640 1 -- 172.21.15.86:0/1130542661 shutdown_connections 2024-04-25T14:13:25.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.167+0000 7feea1400640 1 --2- 172.21.15.86:0/1130542661 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fee7c06f520 0x7fee7c071a30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.167+0000 7feea1400640 1 --2- 172.21.15.86:0/1130542661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee9c09b8a0 0x7fee9c1586d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.167+0000 7feea1400640 1 --2- 172.21.15.86:0/1130542661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9c09b490 0x7fee9c158190 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.167+0000 7feea1400640 1 --2- 172.21.15.86:0/1130542661 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee80004360 0x7fee9c15f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.167+0000 7feea1400640 1 -- 172.21.15.86:0/1130542661 >> 172.21.15.86:0/1130542661 conn(0x7fee9c0b4a40 msgr2=0x7fee9c06e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:25.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.167+0000 7feea1400640 1 -- 172.21.15.86:0/1130542661 shutdown_connections 2024-04-25T14:13:25.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.167+0000 7feea1400640 1 -- 172.21.15.86:0/1130542661 wait complete. 2024-04-25T14:13:25.191 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T14:13:25.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.343+0000 7f9f92600640 1 Processor -- start 2024-04-25T14:13:25.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.343+0000 7f9f92600640 1 -- start start 2024-04-25T14:13:25.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.343+0000 7f9f92600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f8c0d1df0 0x7f9f8c0cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:25.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.343+0000 7f9f92600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9f8c0cdcc0 0x7f9f8c0ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:25.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.343+0000 7f9f92600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f8c15e5c0 con 0x7f9f8c070330 2024-04-25T14:13:25.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.343+0000 7f9f92600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9f8c06fd40 con 0x7f9f8c0cdcc0 2024-04-25T14:13:25.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.343+0000 7f9f92600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9f8c15e8a0 con 0x7f9f8c0d1df0 2024-04-25T14:13:25.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.343+0000 7f9f83e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f8c0d1df0 0x7f9f8c0cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:25.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.343+0000 7f9f83e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f8c0d1df0 0x7f9f8c0cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:55816/0 (socket says 172.21.15.86:55816) 2024-04-25T14:13:25.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.343+0000 7f9f83e00640 1 -- 172.21.15.86:0/2391120030 learned_addr learned my addr 172.21.15.86:0/2391120030 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:25.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.343+0000 7f9f8be00640 1 --2- 172.21.15.86:0/2391120030 >> v2:172.21.15.155:3300/0 conn(0x7f9f8c0cdcc0 0x7f9f8c0ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:25.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.343+0000 7f9f83e00640 1 -- 172.21.15.86:0/2391120030 >> v2:172.21.15.155:3300/0 conn(0x7f9f8c0cdcc0 msgr2=0x7f9f8c0ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:25.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.343+0000 7f9f83e00640 1 --2- 172.21.15.86:0/2391120030 >> v2:172.21.15.155:3300/0 conn(0x7f9f8c0cdcc0 0x7f9f8c0ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.343+0000 7f9f83e00640 1 -- 172.21.15.86:0/2391120030 >> v1:172.21.15.155:6789/0 conn(0x7f9f8c070330 legacy=0x7f9f8c0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:25.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.343+0000 7f9f83e00640 1 -- 172.21.15.86:0/2391120030 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8c15eb80 con 0x7f9f8c0d1df0 2024-04-25T14:13:25.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.343+0000 7f9f8be00640 1 --2- 172.21.15.86:0/2391120030 >> v2:172.21.15.155:3300/0 conn(0x7f9f8c0cdcc0 0x7f9f8c0ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:13:25.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.347+0000 7f9f83e00640 1 --2- 172.21.15.86:0/2391120030 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f8c0d1df0 0x7f9f8c0cd780 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9f74009870 tx=0x7f9f7401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ec49814dea80c77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:25.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.347+0000 7f9f8aa00640 1 -- 172.21.15.86:0/2391120030 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f740ab680 con 0x7f9f8c0d1df0 2024-04-25T14:13:25.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.347+0000 7f9f8aa00640 1 -- 172.21.15.86:0/2391120030 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f740b0070 con 0x7f9f8c0d1df0 2024-04-25T14:13:25.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.347+0000 7f9f8aa00640 1 -- 172.21.15.86:0/2391120030 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f740aba40 con 0x7f9f8c0d1df0 2024-04-25T14:13:25.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.347+0000 7f9f92600640 1 -- 172.21.15.86:0/2391120030 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f8c0d1df0 msgr2=0x7f9f8c0cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:25.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.347+0000 7f9f92600640 1 --2- 172.21.15.86:0/2391120030 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f8c0d1df0 0x7f9f8c0cd780 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9f74009870 tx=0x7f9f7401c4a0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.347+0000 7f9f92600640 1 -- 172.21.15.86:0/2391120030 shutdown_connections 2024-04-25T14:13:25.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.347+0000 7f9f92600640 1 --2- 172.21.15.86:0/2391120030 >> v2:172.21.15.155:3300/0 conn(0x7f9f8c0cdcc0 0x7f9f8c0ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.347+0000 7f9f92600640 1 --2- 172.21.15.86:0/2391120030 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f8c0d1df0 0x7f9f8c0cd780 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.347+0000 7f9f92600640 1 -- 172.21.15.86:0/2391120030 >> 172.21.15.86:0/2391120030 conn(0x7f9f8c0b4650 msgr2=0x7f9f8c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:25.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.347+0000 7f9f92600640 1 -- 172.21.15.86:0/2391120030 shutdown_connections 2024-04-25T14:13:25.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.347+0000 7f9f92600640 1 -- 172.21.15.86:0/2391120030 wait complete. 2024-04-25T14:13:25.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.351+0000 7f9f92600640 1 Processor -- start 2024-04-25T14:13:25.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.351+0000 7f9f92600640 1 -- start start 2024-04-25T14:13:25.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.351+0000 7f9f92600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f8c070330 0x7f9f8c0aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:25.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.351+0000 7f9f92600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f8c0cdcc0 0x7f9f8c15b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:25.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.351+0000 7f9f92600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f8c09e690 0x7f9f8c09da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:25.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.351+0000 7f9f92600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c15f4b0 con 0x7f9f8c0cdcc0 2024-04-25T14:13:25.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.351+0000 7f9f92600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c15f330 con 0x7f9f8c070330 2024-04-25T14:13:25.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.351+0000 7f9f92600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9f8c070740 con 0x7f9f8c09e690 2024-04-25T14:13:25.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.351+0000 7f9f83e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f8c0cdcc0 0x7f9f8c15b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:25.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.351+0000 7f9f83e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f8c0cdcc0 0x7f9f8c15b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41386/0 (socket says 172.21.15.86:41386) 2024-04-25T14:13:25.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.351+0000 7f9f83e00640 1 -- 172.21.15.86:0/2451386054 learned_addr learned my addr 172.21.15.86:0/2451386054 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:25.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.351+0000 7f9f8b400640 1 --2- 172.21.15.86:0/2451386054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f8c070330 0x7f9f8c0aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:25.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.351+0000 7f9f8be00640 1 --2- 172.21.15.86:0/2451386054 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f8c09e690 0x7f9f8c09da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:25.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.351+0000 7f9f83e00640 1 -- 172.21.15.86:0/2451386054 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f8c09e690 msgr2=0x7f9f8c09da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:25.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.351+0000 7f9f83e00640 1 --2- 172.21.15.86:0/2451386054 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f8c09e690 0x7f9f8c09da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.351+0000 7f9f83e00640 1 -- 172.21.15.86:0/2451386054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f8c070330 msgr2=0x7f9f8c0aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:25.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.351+0000 7f9f83e00640 1 --2- 172.21.15.86:0/2451386054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f8c070330 0x7f9f8c0aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.351+0000 7f9f83e00640 1 -- 172.21.15.86:0/2451386054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8c0af440 con 0x7f9f8c0cdcc0 2024-04-25T14:13:25.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.355+0000 7f9f83e00640 1 --2- 172.21.15.86:0/2451386054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f8c0cdcc0 0x7f9f8c15b390 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f9f7401ab50 tx=0x7f9f74019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:25.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.355+0000 7f9f83400640 1 -- 172.21.15.86:0/2451386054 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f740bf610 con 0x7f9f8c0cdcc0 2024-04-25T14:13:25.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.355+0000 7f9f83400640 1 -- 172.21.15.86:0/2451386054 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f740a7070 con 0x7f9f8c0cdcc0 2024-04-25T14:13:25.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.355+0000 7f9f83400640 1 -- 172.21.15.86:0/2451386054 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f740bf9f0 con 0x7f9f8c0cdcc0 2024-04-25T14:13:25.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.355+0000 7f9f92600640 1 -- 172.21.15.86:0/2451386054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f8c0af720 con 0x7f9f8c0cdcc0 2024-04-25T14:13:25.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.355+0000 7f9f92600640 1 -- 172.21.15.86:0/2451386054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f8c0afce0 con 0x7f9f8c0cdcc0 2024-04-25T14:13:25.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.355+0000 7f9f92600640 1 -- 172.21.15.86:0/2451386054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f8c069d00 con 0x7f9f8c0cdcc0 2024-04-25T14:13:25.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.355+0000 7f9f83400640 1 -- 172.21.15.86:0/2451386054 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7f9f740bd020 con 0x7f9f8c0cdcc0 2024-04-25T14:13:25.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.359+0000 7f9f83400640 1 --2- 172.21.15.86:0/2451386054 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f9f6806f520 0x7f9f68071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:25.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.359+0000 7f9f83400640 1 -- 172.21.15.86:0/2451386054 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6109+0+0 (secure 0 0 0) 0x7f9f74129850 con 0x7f9f8c0cdcc0 2024-04-25T14:13:25.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.359+0000 7f9f8b400640 1 --2- 172.21.15.86:0/2451386054 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f9f6806f520 0x7f9f68071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:25.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.359+0000 7f9f8b400640 1 --2- 172.21.15.86:0/2451386054 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f9f6806f520 0x7f9f68071a30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c072200 tx=0x7f9f7c009040 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:25.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.363+0000 7f9f83400640 1 -- 172.21.15.86:0/2451386054 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9f740c6030 con 0x7f9f8c0cdcc0 2024-04-25T14:13:25.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.519+0000 7f9f92600640 1 -- 172.21.15.86:0/2451386054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f9f8c076bf0 con 0x7f9f8c0cdcc0 2024-04-25T14:13:25.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.523+0000 7f9f83400640 1 -- 172.21.15.86:0/2451386054 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7f9f74016150 con 0x7f9f8c0cdcc0 2024-04-25T14:13:25.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.523+0000 7f9f92600640 1 -- 172.21.15.86:0/2451386054 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f9f6806f520 msgr2=0x7f9f68071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:25.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.523+0000 7f9f92600640 1 --2- 172.21.15.86:0/2451386054 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f9f6806f520 0x7f9f68071a30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c072200 tx=0x7f9f7c009040 comp rx=0 tx=0).stop 2024-04-25T14:13:25.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.523+0000 7f9f92600640 1 -- 172.21.15.86:0/2451386054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f8c0cdcc0 msgr2=0x7f9f8c15b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:25.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.523+0000 7f9f92600640 1 --2- 172.21.15.86:0/2451386054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f8c0cdcc0 0x7f9f8c15b390 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f9f7401ab50 tx=0x7f9f74019240 comp rx=0 tx=0).stop 2024-04-25T14:13:25.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.527+0000 7f9f92600640 1 -- 172.21.15.86:0/2451386054 shutdown_connections 2024-04-25T14:13:25.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.527+0000 7f9f92600640 1 --2- 172.21.15.86:0/2451386054 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f9f6806f520 0x7f9f68071a30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.527+0000 7f9f92600640 1 --2- 172.21.15.86:0/2451386054 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f8c09e690 0x7f9f8c09da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.527+0000 7f9f92600640 1 --2- 172.21.15.86:0/2451386054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f8c0cdcc0 0x7f9f8c15b390 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.527+0000 7f9f92600640 1 --2- 172.21.15.86:0/2451386054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f8c070330 0x7f9f8c0aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.527+0000 7f9f92600640 1 -- 172.21.15.86:0/2451386054 >> 172.21.15.86:0/2451386054 conn(0x7f9f8c0b4650 msgr2=0x7f9f8c073920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:25.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.527+0000 7f9f92600640 1 -- 172.21.15.86:0/2451386054 shutdown_connections 2024-04-25T14:13:25.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.527+0000 7f9f92600640 1 -- 172.21.15.86:0/2451386054 wait complete. 2024-04-25T14:13:25.543 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_metrics_mds_grow' 2024-04-25T14:13:25.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.695+0000 7f2a5aa00640 1 Processor -- start 2024-04-25T14:13:25.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.695+0000 7f2a5aa00640 1 -- start start 2024-04-25T14:13:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.695+0000 7f2a5aa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2a5406e7a0 0x7f2a541602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.695+0000 7f2a5aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2a541607e0 0x7f2a54070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.695+0000 7f2a5aa00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2a5406ebb0 0x7f2a54071360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.695+0000 7f2a5aa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2a540d1c10 con 0x7f2a541607e0 2024-04-25T14:13:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.695+0000 7f2a5aa00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f2a540d1ef0 con 0x7f2a5406ebb0 2024-04-25T14:13:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.695+0000 7f2a5aa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2a540d17d0 con 0x7f2a5406e7a0 2024-04-25T14:13:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.695+0000 7f2a53400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2a541607e0 0x7f2a54070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.695+0000 7f2a53400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2a541607e0 0x7f2a54070e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41392/0 (socket says 172.21.15.86:41392) 2024-04-25T14:13:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.695+0000 7f2a53400640 1 -- 172.21.15.86:0/1664145671 learned_addr learned my addr 172.21.15.86:0/1664145671 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.695+0000 7f2a53e00640 1 --2- 172.21.15.86:0/1664145671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2a5406e7a0 0x7f2a541602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:25.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.695+0000 7f2a58a00640 1 --2- 172.21.15.86:0/1664145671 >> v2:172.21.15.155:3300/0 conn(0x7f2a5406ebb0 0x7f2a54071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:25.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.695+0000 7f2a53400640 1 -- 172.21.15.86:0/1664145671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2a5406e7a0 msgr2=0x7f2a541602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:25.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.695+0000 7f2a53400640 1 --2- 172.21.15.86:0/1664145671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2a5406e7a0 0x7f2a541602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.695+0000 7f2a53400640 1 -- 172.21.15.86:0/1664145671 >> v2:172.21.15.155:3300/0 conn(0x7f2a5406ebb0 msgr2=0x7f2a54071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:25.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.695+0000 7f2a53400640 1 --2- 172.21.15.86:0/1664145671 >> v2:172.21.15.155:3300/0 conn(0x7f2a5406ebb0 0x7f2a54071360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.695+0000 7f2a53400640 1 -- 172.21.15.86:0/1664145671 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a540d21d0 con 0x7f2a541607e0 2024-04-25T14:13:25.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.695+0000 7f2a58a00640 1 --2- 172.21.15.86:0/1664145671 >> v2:172.21.15.155:3300/0 conn(0x7f2a5406ebb0 0x7f2a54071360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:25.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.699+0000 7f2a53400640 1 --2- 172.21.15.86:0/1664145671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a541607e0 0x7f2a54070e20 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f2a540ca530 tx=0x7f2a4001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63915dfdab1f9227 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:25.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.699+0000 7f2a52a00640 1 -- 172.21.15.86:0/1664145671 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a400b0070 con 0x7f2a541607e0 2024-04-25T14:13:25.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.699+0000 7f2a52a00640 1 -- 172.21.15.86:0/1664145671 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a400ab600 con 0x7f2a541607e0 2024-04-25T14:13:25.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.699+0000 7f2a52a00640 1 -- 172.21.15.86:0/1664145671 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a400ab9d0 con 0x7f2a541607e0 2024-04-25T14:13:25.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.699+0000 7f2a5aa00640 1 -- 172.21.15.86:0/1664145671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a541607e0 msgr2=0x7f2a54070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:25.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.699+0000 7f2a5aa00640 1 --2- 172.21.15.86:0/1664145671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a541607e0 0x7f2a54070e20 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f2a540ca530 tx=0x7f2a4001c490 comp rx=0 tx=0).stop 2024-04-25T14:13:25.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.699+0000 7f2a5aa00640 1 -- 172.21.15.86:0/1664145671 shutdown_connections 2024-04-25T14:13:25.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.699+0000 7f2a5aa00640 1 --2- 172.21.15.86:0/1664145671 >> v2:172.21.15.155:3300/0 conn(0x7f2a5406ebb0 0x7f2a54071360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.699+0000 7f2a5aa00640 1 --2- 172.21.15.86:0/1664145671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a541607e0 0x7f2a54070e20 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.699+0000 7f2a5aa00640 1 --2- 172.21.15.86:0/1664145671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2a5406e7a0 0x7f2a541602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.699+0000 7f2a5aa00640 1 -- 172.21.15.86:0/1664145671 >> 172.21.15.86:0/1664145671 conn(0x7f2a540cffc0 msgr2=0x7f2a540d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:25.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.699+0000 7f2a5aa00640 1 -- 172.21.15.86:0/1664145671 shutdown_connections 2024-04-25T14:13:25.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.699+0000 7f2a5aa00640 1 -- 172.21.15.86:0/1664145671 wait complete. 2024-04-25T14:13:25.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.703+0000 7f2a5aa00640 1 Processor -- start 2024-04-25T14:13:25.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.703+0000 7f2a5aa00640 1 -- start start 2024-04-25T14:13:25.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.703+0000 7f2a5aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a5406e7a0 0x7f2a5414d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:25.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.703+0000 7f2a5aa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2a5406ebb0 0x7f2a5415e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:25.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.703+0000 7f2a5aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a541607e0 0x7f2a5415ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:25.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.703+0000 7f2a5aa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2a54072e90 con 0x7f2a5406e7a0 2024-04-25T14:13:25.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.703+0000 7f2a5aa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2a54072d10 con 0x7f2a541607e0 2024-04-25T14:13:25.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.703+0000 7f2a5aa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2a54073010 con 0x7f2a5406ebb0 2024-04-25T14:13:25.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.703+0000 7f2a53e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a5406e7a0 0x7f2a5414d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:25.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.703+0000 7f2a53e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a5406e7a0 0x7f2a5414d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41408/0 (socket says 172.21.15.86:41408) 2024-04-25T14:13:25.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.703+0000 7f2a53e00640 1 -- 172.21.15.86:0/771744509 learned_addr learned my addr 172.21.15.86:0/771744509 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:25.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.703+0000 7f2a58a00640 1 --2- 172.21.15.86:0/771744509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a541607e0 0x7f2a5415ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:25.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.703+0000 7f2a53400640 1 --2- 172.21.15.86:0/771744509 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2a5406ebb0 0x7f2a5415e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:25.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.703+0000 7f2a53e00640 1 -- 172.21.15.86:0/771744509 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2a5406ebb0 msgr2=0x7f2a5415e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:25.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.703+0000 7f2a53e00640 1 --2- 172.21.15.86:0/771744509 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2a5406ebb0 0x7f2a5415e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.703+0000 7f2a53e00640 1 -- 172.21.15.86:0/771744509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a541607e0 msgr2=0x7f2a5415ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:25.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.703+0000 7f2a53e00640 1 --2- 172.21.15.86:0/771744509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a541607e0 0x7f2a5415ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:25.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.703+0000 7f2a53e00640 1 -- 172.21.15.86:0/771744509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a54155730 con 0x7f2a5406e7a0 2024-04-25T14:13:25.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.703+0000 7f2a58a00640 1 --2- 172.21.15.86:0/771744509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a541607e0 0x7f2a5415ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:25.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.703+0000 7f2a53e00640 1 --2- 172.21.15.86:0/771744509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a5406e7a0 0x7f2a5414d260 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400e870 tx=0x7f2a4400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:25.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.703+0000 7f2a4be00640 1 -- 172.21.15.86:0/771744509 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a44017070 con 0x7f2a5406e7a0 2024-04-25T14:13:25.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.703+0000 7f2a4be00640 1 -- 172.21.15.86:0/771744509 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a4400c8e0 con 0x7f2a5406e7a0 2024-04-25T14:13:25.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.707+0000 7f2a5aa00640 1 -- 172.21.15.86:0/771744509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a54155a10 con 0x7f2a5406e7a0 2024-04-25T14:13:25.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.707+0000 7f2a4be00640 1 -- 172.21.15.86:0/771744509 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4400cc90 con 0x7f2a5406e7a0 2024-04-25T14:13:25.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.707+0000 7f2a5aa00640 1 -- 172.21.15.86:0/771744509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a5415b5a0 con 0x7f2a5406e7a0 2024-04-25T14:13:25.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.707+0000 7f2a5aa00640 1 -- 172.21.15.86:0/771744509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a540720e0 con 0x7f2a5406e7a0 2024-04-25T14:13:25.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.707+0000 7f2a4be00640 1 -- 172.21.15.86:0/771744509 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7f2a4401a070 con 0x7f2a5406e7a0 2024-04-25T14:13:25.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.707+0000 7f2a4be00640 1 --2- 172.21.15.86:0/771744509 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f2a3406f520 0x7f2a34071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:25.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.707+0000 7f2a4be00640 1 -- 172.21.15.86:0/771744509 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6109+0+0 (secure 0 0 0) 0x7f2a4408f580 con 0x7f2a5406e7a0 2024-04-25T14:13:25.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.707+0000 7f2a53400640 1 --2- 172.21.15.86:0/771744509 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f2a3406f520 0x7f2a34071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:25.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.711+0000 7f2a53400640 1 --2- 172.21.15.86:0/771744509 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f2a3406f520 0x7f2a34071a30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2a400ad000 tx=0x7f2a40008040 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:25.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.711+0000 7f2a4be00640 1 -- 172.21.15.86:0/771744509 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2a4405c130 con 0x7f2a5406e7a0 2024-04-25T14:13:25.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:25.927+0000 7f2a5aa00640 1 -- 172.21.15.86:0/771744509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_metrics_mds_grow"]} v 0) -- 0x7f2a540b94c0 con 0x7f2a5406e7a0 2024-04-25T14:13:26.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.167+0000 7f2a4be00640 1 -- 172.21.15.86:0/771744509 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_metrics_mds_grow"]}]=0 v517) ==== 145+0+0 (secure 0 0 0) 0x7f2a4405ba30 con 0x7f2a5406e7a0 2024-04-25T14:13:26.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.171+0000 7f2a5aa00640 1 -- 172.21.15.86:0/771744509 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f2a3406f520 msgr2=0x7f2a34071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.171+0000 7f2a5aa00640 1 --2- 172.21.15.86:0/771744509 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f2a3406f520 0x7f2a34071a30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2a400ad000 tx=0x7f2a40008040 comp rx=0 tx=0).stop 2024-04-25T14:13:26.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.171+0000 7f2a5aa00640 1 -- 172.21.15.86:0/771744509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a5406e7a0 msgr2=0x7f2a5414d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.171+0000 7f2a5aa00640 1 --2- 172.21.15.86:0/771744509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a5406e7a0 0x7f2a5414d260 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400e870 tx=0x7f2a4400ec70 comp rx=0 tx=0).stop 2024-04-25T14:13:26.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.171+0000 7f2a5aa00640 1 -- 172.21.15.86:0/771744509 shutdown_connections 2024-04-25T14:13:26.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.171+0000 7f2a5aa00640 1 --2- 172.21.15.86:0/771744509 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f2a3406f520 0x7f2a34071a30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.171+0000 7f2a5aa00640 1 --2- 172.21.15.86:0/771744509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a541607e0 0x7f2a5415ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.171+0000 7f2a5aa00640 1 --2- 172.21.15.86:0/771744509 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2a5406ebb0 0x7f2a5415e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.171+0000 7f2a5aa00640 1 --2- 172.21.15.86:0/771744509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a5406e7a0 0x7f2a5414d260 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.171+0000 7f2a5aa00640 1 -- 172.21.15.86:0/771744509 >> 172.21.15.86:0/771744509 conn(0x7f2a540cffc0 msgr2=0x7f2a5415a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.171+0000 7f2a5aa00640 1 -- 172.21.15.86:0/771744509 shutdown_connections 2024-04-25T14:13:26.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.171+0000 7f2a5aa00640 1 -- 172.21.15.86:0/771744509 wait complete. 2024-04-25T14:13:26.190 INFO:tasks.cephfs_test_runner:test_metrics_mds_grow (tasks.cephfs.test_mds_metrics.TestMDSMetrics) ... ok 2024-04-25T14:13:26.190 INFO:tasks.cephfs_test_runner:Starting test: test_metrics_mds_grow_and_shrink (tasks.cephfs.test_mds_metrics.TestMDSMetrics) 2024-04-25T14:13:26.191 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:13:26.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.339+0000 7f1b2a000640 1 Processor -- start 2024-04-25T14:13:26.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.339+0000 7f1b2a000640 1 -- start start 2024-04-25T14:13:26.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.339+0000 7f1b2a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1b240cde90 0x7f1b240ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.339+0000 7f1b2a000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b2415e5c0 con 0x7f1b240d1df0 2024-04-25T14:13:26.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.339+0000 7f1b2a000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b2415e8a0 con 0x7f1b24070330 2024-04-25T14:13:26.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.339+0000 7f1b2a000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1b2406fd40 con 0x7f1b240cde90 2024-04-25T14:13:26.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.339+0000 7f1b23e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1b240cde90 0x7f1b240ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.343+0000 7f1b23e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1b240cde90 0x7f1b240ce2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41420/0 (socket says 172.21.15.86:41420) 2024-04-25T14:13:26.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.343+0000 7f1b23e00640 1 -- 172.21.15.86:0/3031096729 learned_addr learned my addr 172.21.15.86:0/3031096729 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:26.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.343+0000 7f1b23e00640 1 -- 172.21.15.86:0/3031096729 >> v1:172.21.15.155:6789/0 conn(0x7f1b24070330 legacy=0x7f1b240d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.343+0000 7f1b23e00640 1 -- 172.21.15.86:0/3031096729 >> v1:172.21.15.86:6789/0 conn(0x7f1b240d1df0 legacy=0x7f1b240cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.343+0000 7f1b23e00640 1 -- 172.21.15.86:0/3031096729 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2415eb80 con 0x7f1b240cde90 2024-04-25T14:13:26.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.343+0000 7f1b23e00640 1 --2- 172.21.15.86:0/3031096729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b240cde90 0x7f1b240ce2a0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c0098a0 tx=0x7f1b1c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a31e40a9a1eef4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.343+0000 7f1b22a00640 1 -- 172.21.15.86:0/3031096729 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c0b0070 con 0x7f1b240cde90 2024-04-25T14:13:26.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.343+0000 7f1b22a00640 1 -- 172.21.15.86:0/3031096729 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b1c004230 con 0x7f1b240cde90 2024-04-25T14:13:26.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.343+0000 7f1b22a00640 1 -- 172.21.15.86:0/3031096729 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c003620 con 0x7f1b240cde90 2024-04-25T14:13:26.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.347+0000 7f1b2a000640 1 -- 172.21.15.86:0/3031096729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b240cde90 msgr2=0x7f1b240ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.347+0000 7f1b2a000640 1 --2- 172.21.15.86:0/3031096729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b240cde90 0x7f1b240ce2a0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c0098a0 tx=0x7f1b1c01c440 comp rx=0 tx=0).stop 2024-04-25T14:13:26.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.347+0000 7f1b2a000640 1 -- 172.21.15.86:0/3031096729 shutdown_connections 2024-04-25T14:13:26.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.347+0000 7f1b2a000640 1 --2- 172.21.15.86:0/3031096729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b240cde90 0x7f1b240ce2a0 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.347+0000 7f1b2a000640 1 -- 172.21.15.86:0/3031096729 >> 172.21.15.86:0/3031096729 conn(0x7f1b240b4650 msgr2=0x7f1b240b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.347+0000 7f1b2a000640 1 -- 172.21.15.86:0/3031096729 shutdown_connections 2024-04-25T14:13:26.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.347+0000 7f1b2a000640 1 -- 172.21.15.86:0/3031096729 wait complete. 2024-04-25T14:13:26.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.347+0000 7f1b2a000640 1 Processor -- start 2024-04-25T14:13:26.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.347+0000 7f1b2a000640 1 -- start start 2024-04-25T14:13:26.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.351+0000 7f1b2a000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b24070330 0x7f1b24159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.351+0000 7f1b2a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b240cde90 0x7f1b24098430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.351+0000 7f1b2a000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b2409ef00 0x7f1b24098ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.351+0000 7f1b2a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b240708c0 con 0x7f1b240cde90 2024-04-25T14:13:26.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.351+0000 7f1b2a000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1b24070740 con 0x7f1b2409ef00 2024-04-25T14:13:26.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.351+0000 7f1b2a000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1b2416bfd0 con 0x7f1b24070330 2024-04-25T14:13:26.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.351+0000 7f1b1aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b240cde90 0x7f1b24098430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.351+0000 7f1b1aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b240cde90 0x7f1b24098430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41426/0 (socket says 172.21.15.86:41426) 2024-04-25T14:13:26.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.351+0000 7f1b1aa00640 1 -- 172.21.15.86:0/2595658699 learned_addr learned my addr 172.21.15.86:0/2595658699 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:26.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.351+0000 7f1b23e00640 1 --2- 172.21.15.86:0/2595658699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b2409ef00 0x7f1b24098ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.351+0000 7f1b23400640 1 --2- 172.21.15.86:0/2595658699 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b24070330 0x7f1b24159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.351+0000 7f1b1aa00640 1 -- 172.21.15.86:0/2595658699 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b24070330 msgr2=0x7f1b24159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.351+0000 7f1b1aa00640 1 --2- 172.21.15.86:0/2595658699 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b24070330 0x7f1b24159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.351+0000 7f1b1aa00640 1 -- 172.21.15.86:0/2595658699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b2409ef00 msgr2=0x7f1b24098ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.351+0000 7f1b1aa00640 1 --2- 172.21.15.86:0/2595658699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b2409ef00 0x7f1b24098ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.351+0000 7f1b1aa00640 1 -- 172.21.15.86:0/2595658699 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b24098970 con 0x7f1b240cde90 2024-04-25T14:13:26.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.351+0000 7f1b23400640 1 --2- 172.21.15.86:0/2595658699 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b24070330 0x7f1b24159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:26.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.351+0000 7f1b1aa00640 1 --2- 172.21.15.86:0/2595658699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b240cde90 0x7f1b24098430 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00c310 tx=0x7f1b0c00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.351+0000 7f1b1be00640 1 -- 172.21.15.86:0/2595658699 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c0052d0 con 0x7f1b240cde90 2024-04-25T14:13:26.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.351+0000 7f1b1be00640 1 -- 172.21.15.86:0/2595658699 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b0c005470 con 0x7f1b240cde90 2024-04-25T14:13:26.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.351+0000 7f1b2a000640 1 -- 172.21.15.86:0/2595658699 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b24158dc0 con 0x7f1b240cde90 2024-04-25T14:13:26.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.351+0000 7f1b1be00640 1 -- 172.21.15.86:0/2595658699 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c007260 con 0x7f1b240cde90 2024-04-25T14:13:26.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.351+0000 7f1b2a000640 1 -- 172.21.15.86:0/2595658699 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b24159380 con 0x7f1b240cde90 2024-04-25T14:13:26.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.351+0000 7f1b2a000640 1 -- 172.21.15.86:0/2595658699 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b24069d00 con 0x7f1b240cde90 2024-04-25T14:13:26.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.351+0000 7f1b1be00640 1 -- 172.21.15.86:0/2595658699 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7f1b0c007400 con 0x7f1b240cde90 2024-04-25T14:13:26.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.355+0000 7f1b1be00640 1 --2- 172.21.15.86:0/2595658699 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f1b0006f520 0x7f1b00071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.355+0000 7f1b1be00640 1 -- 172.21.15.86:0/2595658699 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6109+0+0 (secure 0 0 0) 0x7f1b0c08e9e0 con 0x7f1b240cde90 2024-04-25T14:13:26.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.355+0000 7f1b23400640 1 --2- 172.21.15.86:0/2595658699 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f1b0006f520 0x7f1b00071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.355+0000 7f1b23400640 1 --2- 172.21.15.86:0/2595658699 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f1b0006f520 0x7f1b00071a30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1b24072200 tx=0x7f1b14009550 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.359+0000 7f1b1be00640 1 -- 172.21.15.86:0/2595658699 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1b0c05af40 con 0x7f1b240cde90 2024-04-25T14:13:26.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.515+0000 7f1b2a000640 1 -- 172.21.15.86:0/2595658699 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b2406db60 con 0x7f1b240cde90 2024-04-25T14:13:26.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.515+0000 7f1b1be00640 1 -- 172.21.15.86:0/2595658699 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) ==== 74+0+14431 (secure 0 0 0) 0x7f1b0c05f240 con 0x7f1b240cde90 2024-04-25T14:13:26.515 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:13:26.515 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":147,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:13:25.161096+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.155:6835/91793677":"2024-04-26T14:13:22.147843+0000","172.21.15.155:6833/91793677":"2024-04-26T14:13:22.147843+0000","172.21.15.86:0/3190429862":"2024-04-26T14:13:18.050389+0000","172.21.15.86:0/3060165022":"2024-04-26T14:13:18.050389+0000","172.21.15.86:6832/1173654665":"2024-04-26T14:13:18.050389+0000","172.21.15.86:0/2751178284":"2024-04-26T14:13:18.050389+0000","172.21.15.86:6833/1173654665":"2024-04-26T14:13:18.050389+0000","172.21.15.86:0/4083287014":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/1048620009":"2024-04-26T14:13:18.050389+0000","172.21.15.86:0/746190999":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/1426015871":"2024-04-26T14:11:51.781677+0000","172.21.15.86:6835/2571655765":"2024-04-26T14:13:22.147863+0000","172.21.15.86:0/3750847077":"2024-04-26T14:13:18.050389+0000","172.21.15.86:0/3962343281":"2024-04-26T14:11:51.781677+0000","172.21.15.86:6834/2571655765":"2024-04-26T14:13:22.147863+0000","172.21.15.86:6833/264890479":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/3349072784":"2024-04-26T14:13:18.050389+0000","172.21.15.86:6832/264890479":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/2724786983":"2024-04-26T14:11:51.781677+0000","172.21.15.86:0/285240843":"2024-04-26T14:11:51.781677+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:13:26.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.519+0000 7f1b2a000640 1 -- 172.21.15.86:0/2595658699 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f1b0006f520 msgr2=0x7f1b00071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.519+0000 7f1b2a000640 1 --2- 172.21.15.86:0/2595658699 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f1b0006f520 0x7f1b00071a30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1b24072200 tx=0x7f1b14009550 comp rx=0 tx=0).stop 2024-04-25T14:13:26.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.519+0000 7f1b2a000640 1 -- 172.21.15.86:0/2595658699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b240cde90 msgr2=0x7f1b24098430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.519+0000 7f1b2a000640 1 --2- 172.21.15.86:0/2595658699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b240cde90 0x7f1b24098430 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00c310 tx=0x7f1b0c00c7d0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.519+0000 7f1b2a000640 1 -- 172.21.15.86:0/2595658699 shutdown_connections 2024-04-25T14:13:26.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.519+0000 7f1b2a000640 1 --2- 172.21.15.86:0/2595658699 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f1b0006f520 0x7f1b00071a30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.519+0000 7f1b2a000640 1 --2- 172.21.15.86:0/2595658699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b2409ef00 0x7f1b24098ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.519+0000 7f1b2a000640 1 --2- 172.21.15.86:0/2595658699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b240cde90 0x7f1b24098430 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.519+0000 7f1b2a000640 1 --2- 172.21.15.86:0/2595658699 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b24070330 0x7f1b24159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.519+0000 7f1b2a000640 1 -- 172.21.15.86:0/2595658699 >> 172.21.15.86:0/2595658699 conn(0x7f1b240b4650 msgr2=0x7f1b240b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.519+0000 7f1b2a000640 1 -- 172.21.15.86:0/2595658699 shutdown_connections 2024-04-25T14:13:26.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.519+0000 7f1b2a000640 1 -- 172.21.15.86:0/2595658699 wait complete. 2024-04-25T14:13:26.538 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:13:26.538 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:13:26.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.691+0000 7f6d61800640 1 Processor -- start 2024-04-25T14:13:26.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.691+0000 7f6d61800640 1 -- start start 2024-04-25T14:13:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.691+0000 7f6d61800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6d5c0d1c70 0x7f6d5c0d2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.691+0000 7f6d61800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6d5c0b7480 0x7f6d5c0b7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.691+0000 7f6d61800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d5c160ec0 con 0x7f6d5c070e70 2024-04-25T14:13:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.691+0000 7f6d61800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6d5c1611a0 con 0x7f6d5c0d1c70 2024-04-25T14:13:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.691+0000 7f6d61800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f6d5c0d1190 con 0x7f6d5c0b7480 2024-04-25T14:13:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.691+0000 7f6d5be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6d5c0d1c70 0x7f6d5c0d2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.691+0000 7f6d5be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6d5c0d1c70 0x7f6d5c0d2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41436/0 (socket says 172.21.15.86:41436) 2024-04-25T14:13:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.691+0000 7f6d5be00640 1 -- 172.21.15.86:0/709031056 learned_addr learned my addr 172.21.15.86:0/709031056 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.691+0000 7f6d53400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6d5c070e70 0x7f6d5c071280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47402/0 (socket says 172.21.15.86:47402) 2024-04-25T14:13:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.695+0000 7f6d60e00640 1 --2- 172.21.15.86:0/709031056 >> v2:172.21.15.155:3300/0 conn(0x7f6d5c0b7480 0x7f6d5c0b7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.695+0000 7f6d5be00640 1 -- 172.21.15.86:0/709031056 >> v2:172.21.15.155:3300/0 conn(0x7f6d5c0b7480 msgr2=0x7f6d5c0b7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.695+0000 7f6d5be00640 1 --2- 172.21.15.86:0/709031056 >> v2:172.21.15.155:3300/0 conn(0x7f6d5c0b7480 0x7f6d5c0b7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.695+0000 7f6d5be00640 1 -- 172.21.15.86:0/709031056 >> v1:172.21.15.86:6789/0 conn(0x7f6d5c070e70 legacy=0x7f6d5c071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.695+0000 7f6d5be00640 1 -- 172.21.15.86:0/709031056 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5c0d1470 con 0x7f6d5c0d1c70 2024-04-25T14:13:26.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.695+0000 7f6d5be00640 1 --2- 172.21.15.86:0/709031056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d5c0d1c70 0x7f6d5c0d2080 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c0cfe40 tx=0x7f6d4c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9788decb5794f04e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.695+0000 7f6d5b400640 1 -- 172.21.15.86:0/709031056 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c0a58a0 con 0x7f6d5c0d1c70 2024-04-25T14:13:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.695+0000 7f6d5b400640 1 -- 172.21.15.86:0/709031056 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d4c0a5a40 con 0x7f6d5c0d1c70 2024-04-25T14:13:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.695+0000 7f6d5b400640 1 -- 172.21.15.86:0/709031056 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c0a5e10 con 0x7f6d5c0d1c70 2024-04-25T14:13:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.699+0000 7f6d61800640 1 -- 172.21.15.86:0/709031056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d5c0d1c70 msgr2=0x7f6d5c0d2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.699+0000 7f6d61800640 1 --2- 172.21.15.86:0/709031056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d5c0d1c70 0x7f6d5c0d2080 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c0cfe40 tx=0x7f6d4c01c9c0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.699+0000 7f6d61800640 1 -- 172.21.15.86:0/709031056 shutdown_connections 2024-04-25T14:13:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.699+0000 7f6d61800640 1 --2- 172.21.15.86:0/709031056 >> v2:172.21.15.155:3300/0 conn(0x7f6d5c0b7480 0x7f6d5c0b7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.699+0000 7f6d61800640 1 --2- 172.21.15.86:0/709031056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d5c0d1c70 0x7f6d5c0d2080 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.699+0000 7f6d61800640 1 -- 172.21.15.86:0/709031056 >> 172.21.15.86:0/709031056 conn(0x7f6d5c0cfea0 msgr2=0x7f6d5c0d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.699+0000 7f6d61800640 1 -- 172.21.15.86:0/709031056 shutdown_connections 2024-04-25T14:13:26.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.699+0000 7f6d61800640 1 -- 172.21.15.86:0/709031056 wait complete. 2024-04-25T14:13:26.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.699+0000 7f6d61800640 1 Processor -- start 2024-04-25T14:13:26.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.699+0000 7f6d61800640 1 -- start start 2024-04-25T14:13:26.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.703+0000 7f6d61800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d5c070e70 0x7f6d5c0cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.703+0000 7f6d61800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d5c0b7480 0x7f6d5c152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.703+0000 7f6d61800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6d5c0d1c70 0x7f6d5c1545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.703+0000 7f6d61800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c16c7d0 con 0x7f6d5c0b7480 2024-04-25T14:13:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.703+0000 7f6d61800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c160bb0 con 0x7f6d5c070e70 2024-04-25T14:13:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.703+0000 7f6d61800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6d5c16c950 con 0x7f6d5c0d1c70 2024-04-25T14:13:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.703+0000 7f6d53400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d5c0b7480 0x7f6d5c152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.703+0000 7f6d53400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d5c0b7480 0x7f6d5c152dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41442/0 (socket says 172.21.15.86:41442) 2024-04-25T14:13:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.703+0000 7f6d53400640 1 -- 172.21.15.86:0/1486693185 learned_addr learned my addr 172.21.15.86:0/1486693185 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.703+0000 7f6d5be00640 1 --2- 172.21.15.86:0/1486693185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d5c070e70 0x7f6d5c0cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.703+0000 7f6d60e00640 1 --2- 172.21.15.86:0/1486693185 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6d5c0d1c70 0x7f6d5c1545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.703+0000 7f6d53400640 1 -- 172.21.15.86:0/1486693185 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6d5c0d1c70 msgr2=0x7f6d5c1545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.703+0000 7f6d53400640 1 --2- 172.21.15.86:0/1486693185 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6d5c0d1c70 0x7f6d5c1545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.703+0000 7f6d53400640 1 -- 172.21.15.86:0/1486693185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d5c070e70 msgr2=0x7f6d5c0cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.703+0000 7f6d53400640 1 --2- 172.21.15.86:0/1486693185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d5c070e70 0x7f6d5c0cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.703+0000 7f6d53400640 1 -- 172.21.15.86:0/1486693185 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5c06d820 con 0x7f6d5c0b7480 2024-04-25T14:13:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.703+0000 7f6d53400640 1 --2- 172.21.15.86:0/1486693185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d5c0b7480 0x7f6d5c152dd0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400c3a0 tx=0x7f6d4400c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.703+0000 7f6d58c00640 1 -- 172.21.15.86:0/1486693185 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d440074e0 con 0x7f6d5c0b7480 2024-04-25T14:13:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.703+0000 7f6d58c00640 1 -- 172.21.15.86:0/1486693185 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d44007680 con 0x7f6d5c0b7480 2024-04-25T14:13:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.703+0000 7f6d58c00640 1 -- 172.21.15.86:0/1486693185 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44004a30 con 0x7f6d5c0b7480 2024-04-25T14:13:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.703+0000 7f6d61800640 1 -- 172.21.15.86:0/1486693185 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d5c06db00 con 0x7f6d5c0b7480 2024-04-25T14:13:26.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.703+0000 7f6d61800640 1 -- 172.21.15.86:0/1486693185 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d5c06e0c0 con 0x7f6d5c0b7480 2024-04-25T14:13:26.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.703+0000 7f6d61800640 1 -- 172.21.15.86:0/1486693185 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d5c076b80 con 0x7f6d5c0b7480 2024-04-25T14:13:26.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.703+0000 7f6d58c00640 1 -- 172.21.15.86:0/1486693185 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7f6d4401a020 con 0x7f6d5c0b7480 2024-04-25T14:13:26.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.707+0000 7f6d58c00640 1 --2- 172.21.15.86:0/1486693185 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f6d3806f520 0x7f6d38071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:26.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.707+0000 7f6d58c00640 1 -- 172.21.15.86:0/1486693185 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6109+0+0 (secure 0 0 0) 0x7f6d4408ec70 con 0x7f6d5c0b7480 2024-04-25T14:13:26.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.707+0000 7f6d5be00640 1 --2- 172.21.15.86:0/1486693185 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f6d3806f520 0x7f6d38071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:26.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.707+0000 7f6d5be00640 1 --2- 172.21.15.86:0/1486693185 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f6d3806f520 0x7f6d38071a30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0ae000 tx=0x7f6d4c0a7070 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:26.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.711+0000 7f6d58c00640 1 -- 172.21.15.86:0/1486693185 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6d4405b8b0 con 0x7f6d5c0b7480 2024-04-25T14:13:26.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.867+0000 7f6d61800640 1 -- 172.21.15.86:0/1486693185 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6d5c069d00 con 0x7f6d5c0b7480 2024-04-25T14:13:26.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.871+0000 7f6d58c00640 1 -- 172.21.15.86:0/1486693185 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v147) ==== 93+0+10 (secure 0 0 0) 0x7f6d4405b1b0 con 0x7f6d5c0b7480 2024-04-25T14:13:26.868 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:13:26.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.871+0000 7f6d61800640 1 -- 172.21.15.86:0/1486693185 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f6d3806f520 msgr2=0x7f6d38071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.871+0000 7f6d61800640 1 --2- 172.21.15.86:0/1486693185 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f6d3806f520 0x7f6d38071a30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0ae000 tx=0x7f6d4c0a7070 comp rx=0 tx=0).stop 2024-04-25T14:13:26.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.871+0000 7f6d61800640 1 -- 172.21.15.86:0/1486693185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d5c0b7480 msgr2=0x7f6d5c152dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:26.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.871+0000 7f6d61800640 1 --2- 172.21.15.86:0/1486693185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d5c0b7480 0x7f6d5c152dd0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400c3a0 tx=0x7f6d4400c7a0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.871+0000 7f6d61800640 1 -- 172.21.15.86:0/1486693185 shutdown_connections 2024-04-25T14:13:26.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.871+0000 7f6d61800640 1 --2- 172.21.15.86:0/1486693185 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f6d3806f520 0x7f6d38071a30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.871+0000 7f6d61800640 1 --2- 172.21.15.86:0/1486693185 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6d5c0d1c70 0x7f6d5c1545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.871+0000 7f6d61800640 1 --2- 172.21.15.86:0/1486693185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d5c0b7480 0x7f6d5c152dd0 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.871+0000 7f6d61800640 1 --2- 172.21.15.86:0/1486693185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d5c070e70 0x7f6d5c0cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:26.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.871+0000 7f6d61800640 1 -- 172.21.15.86:0/1486693185 >> 172.21.15.86:0/1486693185 conn(0x7f6d5c0cfea0 msgr2=0x7f6d5c15ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:26.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.871+0000 7f6d61800640 1 -- 172.21.15.86:0/1486693185 shutdown_connections 2024-04-25T14:13:26.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:26.871+0000 7f6d61800640 1 -- 172.21.15.86:0/1486693185 wait complete. 2024-04-25T14:13:26.890 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_metrics_mds_grow_and_shrink' 2024-04-25T14:13:27.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.039+0000 7f5c8b000640 1 Processor -- start 2024-04-25T14:13:27.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.039+0000 7f5c8b000640 1 -- start start 2024-04-25T14:13:27.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.039+0000 7f5c8b000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5c84070330 0x7f5c840d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:27.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.039+0000 7f5c8b000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c840d1c20 0x7f5c8406ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:27.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.039+0000 7f5c8b000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c840d2030 con 0x7f5c8406f270 2024-04-25T14:13:27.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.039+0000 7f5c8b000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f5c8415e5c0 con 0x7f5c84070330 2024-04-25T14:13:27.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.039+0000 7f5c8b000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5c8415e8a0 con 0x7f5c840d1c20 2024-04-25T14:13:27.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.039+0000 7f5c89000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5c8406f270 0x7f5c8406f680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47410/0 (socket says 172.21.15.86:47410) 2024-04-25T14:13:27.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.039+0000 7f5c89000640 1 -- 172.21.15.86:0/1895983314 learned_addr learned my addr 172.21.15.86:0/1895983314 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:27.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.039+0000 7f5c83400640 1 --2- 172.21.15.86:0/1895983314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c840d1c20 0x7f5c8406ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:27.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.039+0000 7f5c83e00640 1 --2- 172.21.15.86:0/1895983314 >> v2:172.21.15.155:3300/0 conn(0x7f5c84070330 0x7f5c840d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:27.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.039+0000 7f5c82a00640 1 -- 172.21.15.86:0/1895983314 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 715574631 0 0) 0x7f5c7c0014a0 con 0x7f5c8406f270 2024-04-25T14:13:27.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.039+0000 7f5c82a00640 1 -- 172.21.15.86:0/1895983314 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c68003530 con 0x7f5c8406f270 2024-04-25T14:13:27.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.039+0000 7f5c82a00640 1 -- 172.21.15.86:0/1895983314 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1088074939 0 0) 0x7f5c68003530 con 0x7f5c8406f270 2024-04-25T14:13:27.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.039+0000 7f5c83e00640 1 -- 172.21.15.86:0/1895983314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c840d1c20 msgr2=0x7f5c8406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:27.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.039+0000 7f5c83e00640 1 --2- 172.21.15.86:0/1895983314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c840d1c20 0x7f5c8406ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:27.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.039+0000 7f5c83e00640 1 -- 172.21.15.86:0/1895983314 >> v1:172.21.15.86:6789/0 conn(0x7f5c8406f270 legacy=0x7f5c8406f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:27.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.039+0000 7f5c83e00640 1 -- 172.21.15.86:0/1895983314 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c8415eb80 con 0x7f5c84070330 2024-04-25T14:13:27.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.039+0000 7f5c83400640 1 --2- 172.21.15.86:0/1895983314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c840d1c20 0x7f5c8406ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:13:27.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.043+0000 7f5c83e00640 1 --2- 172.21.15.86:0/1895983314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c84070330 0x7f5c840d16e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5c740097c0 tx=0x7f5c7401c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55a40c14339b59df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:27.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.043+0000 7f5c82a00640 1 -- 172.21.15.86:0/1895983314 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c740b0070 con 0x7f5c84070330 2024-04-25T14:13:27.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.043+0000 7f5c82a00640 1 -- 172.21.15.86:0/1895983314 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c740ab400 con 0x7f5c84070330 2024-04-25T14:13:27.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.043+0000 7f5c82a00640 1 -- 172.21.15.86:0/1895983314 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c001bd0 con 0x7f5c84070330 2024-04-25T14:13:27.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.043+0000 7f5c8b000640 1 -- 172.21.15.86:0/1895983314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c84070330 msgr2=0x7f5c840d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:27.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.043+0000 7f5c8b000640 1 --2- 172.21.15.86:0/1895983314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c84070330 0x7f5c840d16e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5c740097c0 tx=0x7f5c7401c330 comp rx=0 tx=0).stop 2024-04-25T14:13:27.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.043+0000 7f5c8b000640 1 -- 172.21.15.86:0/1895983314 shutdown_connections 2024-04-25T14:13:27.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.043+0000 7f5c8b000640 1 --2- 172.21.15.86:0/1895983314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c840d1c20 0x7f5c8406ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:27.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.043+0000 7f5c8b000640 1 --2- 172.21.15.86:0/1895983314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c84070330 0x7f5c840d16e0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:27.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.043+0000 7f5c8b000640 1 -- 172.21.15.86:0/1895983314 >> 172.21.15.86:0/1895983314 conn(0x7f5c840b4650 msgr2=0x7f5c840b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:27.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.043+0000 7f5c8b000640 1 -- 172.21.15.86:0/1895983314 shutdown_connections 2024-04-25T14:13:27.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.043+0000 7f5c8b000640 1 -- 172.21.15.86:0/1895983314 wait complete. 2024-04-25T14:13:27.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.047+0000 7f5c8b000640 1 Processor -- start 2024-04-25T14:13:27.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.047+0000 7f5c8b000640 1 -- start start 2024-04-25T14:13:27.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.047+0000 7f5c8b000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c8406f270 0x7f5c84156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:27.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.047+0000 7f5c8b000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c84070330 0x7f5c84099a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:27.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.047+0000 7f5c8b000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c840d1c20 0x7f5c84152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:27.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.047+0000 7f5c8b000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c840708c0 con 0x7f5c8406f270 2024-04-25T14:13:27.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.047+0000 7f5c8b000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5c84070740 con 0x7f5c840d1c20 2024-04-25T14:13:27.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.047+0000 7f5c8b000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5c8416afc0 con 0x7f5c84070330 2024-04-25T14:13:27.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.047+0000 7f5c83e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c8406f270 0x7f5c84156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:27.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.047+0000 7f5c83e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c8406f270 0x7f5c84156060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41458/0 (socket says 172.21.15.86:41458) 2024-04-25T14:13:27.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.047+0000 7f5c83e00640 1 -- 172.21.15.86:0/3402818605 learned_addr learned my addr 172.21.15.86:0/3402818605 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:27.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.047+0000 7f5c83400640 1 --2- 172.21.15.86:0/3402818605 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c84070330 0x7f5c84099a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:27.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.047+0000 7f5c89000640 1 --2- 172.21.15.86:0/3402818605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c840d1c20 0x7f5c84152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:27.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.047+0000 7f5c83e00640 1 -- 172.21.15.86:0/3402818605 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c84070330 msgr2=0x7f5c84099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:27.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.047+0000 7f5c83e00640 1 --2- 172.21.15.86:0/3402818605 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c84070330 0x7f5c84099a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:27.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.047+0000 7f5c83e00640 1 -- 172.21.15.86:0/3402818605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c840d1c20 msgr2=0x7f5c84152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:27.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.047+0000 7f5c83e00640 1 --2- 172.21.15.86:0/3402818605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c840d1c20 0x7f5c84152eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:27.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.047+0000 7f5c83e00640 1 -- 172.21.15.86:0/3402818605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c84155400 con 0x7f5c8406f270 2024-04-25T14:13:27.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.047+0000 7f5c89000640 1 --2- 172.21.15.86:0/3402818605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c840d1c20 0x7f5c84152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:27.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.051+0000 7f5c83e00640 1 --2- 172.21.15.86:0/3402818605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c8406f270 0x7f5c84156060 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f5c74000c10 tx=0x7f5c740190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:27.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.051+0000 7f5c7be00640 1 -- 172.21.15.86:0/3402818605 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c740b9020 con 0x7f5c8406f270 2024-04-25T14:13:27.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.051+0000 7f5c7be00640 1 -- 172.21.15.86:0/3402818605 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c74003820 con 0x7f5c8406f270 2024-04-25T14:13:27.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.051+0000 7f5c7be00640 1 -- 172.21.15.86:0/3402818605 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c740b0040 con 0x7f5c8406f270 2024-04-25T14:13:27.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.051+0000 7f5c8b000640 1 -- 172.21.15.86:0/3402818605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c841556a0 con 0x7f5c8406f270 2024-04-25T14:13:27.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.051+0000 7f5c8b000640 1 -- 172.21.15.86:0/3402818605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c841523a0 con 0x7f5c8406f270 2024-04-25T14:13:27.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.051+0000 7f5c8b000640 1 -- 172.21.15.86:0/3402818605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c84069d00 con 0x7f5c8406f270 2024-04-25T14:13:27.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.051+0000 7f5c7be00640 1 -- 172.21.15.86:0/3402818605 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7f5c740bc020 con 0x7f5c8406f270 2024-04-25T14:13:27.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.051+0000 7f5c7be00640 1 --2- 172.21.15.86:0/3402818605 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f5c6406f520 0x7f5c64071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:27.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.051+0000 7f5c7be00640 1 -- 172.21.15.86:0/3402818605 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6109+0+0 (secure 0 0 0) 0x7f5c74129600 con 0x7f5c8406f270 2024-04-25T14:13:27.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.051+0000 7f5c83400640 1 --2- 172.21.15.86:0/3402818605 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f5c6406f520 0x7f5c64071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:27.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.055+0000 7f5c83400640 1 --2- 172.21.15.86:0/3402818605 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f5c6406f520 0x7f5c64071a30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5c8409bd60 tx=0x7f5c70008040 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:27.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.059+0000 7f5c7be00640 1 -- 172.21.15.86:0/3402818605 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5c740c3030 con 0x7f5c8406f270 2024-04-25T14:13:27.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:27.267+0000 7f5c8b000640 1 -- 172.21.15.86:0/3402818605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_metrics_mds_grow_and_shrink"]} v 0) -- 0x7f5c84076bf0 con 0x7f5c8406f270 2024-04-25T14:13:28.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.175+0000 7f5c7be00640 1 -- 172.21.15.86:0/3402818605 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_metrics_mds_grow_and_shrink"]}]=0 v519) ==== 159+0+0 (secure 0 0 0) 0x7f5c74015fb0 con 0x7f5c8406f270 2024-04-25T14:13:28.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.179+0000 7f5c8b000640 1 -- 172.21.15.86:0/3402818605 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f5c6406f520 msgr2=0x7f5c64071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:28.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.179+0000 7f5c8b000640 1 --2- 172.21.15.86:0/3402818605 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f5c6406f520 0x7f5c64071a30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5c8409bd60 tx=0x7f5c70008040 comp rx=0 tx=0).stop 2024-04-25T14:13:28.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.179+0000 7f5c8b000640 1 -- 172.21.15.86:0/3402818605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c8406f270 msgr2=0x7f5c84156060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:28.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.179+0000 7f5c8b000640 1 --2- 172.21.15.86:0/3402818605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c8406f270 0x7f5c84156060 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f5c74000c10 tx=0x7f5c740190d0 comp rx=0 tx=0).stop 2024-04-25T14:13:28.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.179+0000 7f5c8b000640 1 -- 172.21.15.86:0/3402818605 shutdown_connections 2024-04-25T14:13:28.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.179+0000 7f5c8b000640 1 --2- 172.21.15.86:0/3402818605 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f5c6406f520 0x7f5c64071a30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:28.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.179+0000 7f5c8b000640 1 --2- 172.21.15.86:0/3402818605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c840d1c20 0x7f5c84152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:28.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.179+0000 7f5c8b000640 1 --2- 172.21.15.86:0/3402818605 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c84070330 0x7f5c84099a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:28.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.179+0000 7f5c8b000640 1 --2- 172.21.15.86:0/3402818605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c8406f270 0x7f5c84156060 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:28.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.179+0000 7f5c8b000640 1 -- 172.21.15.86:0/3402818605 >> 172.21.15.86:0/3402818605 conn(0x7f5c840b4650 msgr2=0x7f5c84073ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:28.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.179+0000 7f5c8b000640 1 -- 172.21.15.86:0/3402818605 shutdown_connections 2024-04-25T14:13:28.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.179+0000 7f5c8b000640 1 -- 172.21.15.86:0/3402818605 wait complete. 2024-04-25T14:13:28.202 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T14:13:28.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.359+0000 7fdbfd600640 1 Processor -- start 2024-04-25T14:13:28.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.359+0000 7fdbfd600640 1 -- start start 2024-04-25T14:13:28.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.359+0000 7fdbfd600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdbf8070330 0x7fdbf80d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:28.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.359+0000 7fdbfd600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbf80d2030 con 0x7fdbf80ce790 2024-04-25T14:13:28.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.359+0000 7fdbfd600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbf815e5c0 con 0x7fdbf80d1c20 2024-04-25T14:13:28.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.359+0000 7fdbfd600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fdbf815e8a0 con 0x7fdbf8070330 2024-04-25T14:13:28.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.359+0000 7fdbf7400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fdbf80ce790 0x7fdbf806f440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47424/0 (socket says 172.21.15.86:47424) 2024-04-25T14:13:28.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.359+0000 7fdbf7400640 1 -- 172.21.15.86:0/1458837123 learned_addr learned my addr 172.21.15.86:0/1458837123 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:28.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.359+0000 7fdbf6a00640 1 --2- 172.21.15.86:0/1458837123 >> v2:172.21.15.155:3300/0 conn(0x7fdbf8070330 0x7fdbf80d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:28.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.359+0000 7fdbf6000640 1 -- 172.21.15.86:0/1458837123 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3154898135 0 0) 0x7fdbf00014a0 con 0x7fdbf80ce790 2024-04-25T14:13:28.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.359+0000 7fdbf6000640 1 -- 172.21.15.86:0/1458837123 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdbdc003530 con 0x7fdbf80ce790 2024-04-25T14:13:28.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.359+0000 7fdbf6000640 1 -- 172.21.15.86:0/1458837123 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1163946836 0 0) 0x7fdbdc003530 con 0x7fdbf80ce790 2024-04-25T14:13:28.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.359+0000 7fdbf6000640 1 -- 172.21.15.86:0/1458837123 >> v2:172.21.15.155:3300/0 conn(0x7fdbf8070330 msgr2=0x7fdbf80d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:28.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.359+0000 7fdbf6000640 1 --2- 172.21.15.86:0/1458837123 >> v2:172.21.15.155:3300/0 conn(0x7fdbf8070330 0x7fdbf80d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:28.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.359+0000 7fdbf6000640 1 -- 172.21.15.86:0/1458837123 >> v1:172.21.15.155:6789/0 conn(0x7fdbf80d1c20 legacy=0x7fdbf806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:28.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.359+0000 7fdbf6000640 1 -- 172.21.15.86:0/1458837123 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf815eb80 con 0x7fdbf80ce790 2024-04-25T14:13:28.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.359+0000 7fdbf6000640 1 -- 172.21.15.86:0/1458837123 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fdbf0001bd0 con 0x7fdbf80ce790 2024-04-25T14:13:28.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.359+0000 7fdbf6000640 1 -- 172.21.15.86:0/1458837123 >> v1:172.21.15.86:6789/0 conn(0x7fdbf80ce790 legacy=0x7fdbf806f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:28.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.359+0000 7fdbf6000640 1 --2- 172.21.15.86:0/1458837123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbdc003a30 0x7fdbdc003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:28.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.363+0000 7fdbf6000640 1 --2- 172.21.15.86:0/1458837123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbdc004810 0x7fdbdc004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:28.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.363+0000 7fdbf6000640 1 --2- 172.21.15.86:0/1458837123 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbdc005360 0x7fdbdc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:28.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.363+0000 7fdbf6000640 1 -- 172.21.15.86:0/1458837123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf815e8a0 con 0x7fdbdc004810 2024-04-25T14:13:28.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.363+0000 7fdbf6000640 1 -- 172.21.15.86:0/1458837123 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc00c260 con 0x7fdbdc003a30 2024-04-25T14:13:28.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.363+0000 7fdbf6000640 1 -- 172.21.15.86:0/1458837123 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdbdc00c460 con 0x7fdbdc005360 2024-04-25T14:13:28.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.363+0000 7fdbee000640 1 --2- 172.21.15.86:0/1458837123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbdc004810 0x7fdbdc004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:28.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.363+0000 7fdbf6a00640 1 --2- 172.21.15.86:0/1458837123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbdc003a30 0x7fdbdc003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:28.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.363+0000 7fdbf7400640 1 --2- 172.21.15.86:0/1458837123 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbdc005360 0x7fdbdc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:28.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.363+0000 7fdbee000640 1 -- 172.21.15.86:0/1458837123 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbdc005360 msgr2=0x7fdbdc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:28.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.363+0000 7fdbee000640 1 --2- 172.21.15.86:0/1458837123 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbdc005360 0x7fdbdc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:28.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.363+0000 7fdbee000640 1 -- 172.21.15.86:0/1458837123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbdc003a30 msgr2=0x7fdbdc003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:28.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.363+0000 7fdbee000640 1 --2- 172.21.15.86:0/1458837123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbdc003a30 0x7fdbdc003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:28.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.363+0000 7fdbee000640 1 -- 172.21.15.86:0/1458837123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbdc003530 con 0x7fdbdc004810 2024-04-25T14:13:28.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.363+0000 7fdbf7400640 1 -- 172.21.15.86:0/1458837123 reap_dead start 2024-04-25T14:13:28.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.363+0000 7fdbee000640 1 --2- 172.21.15.86:0/1458837123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbdc004810 0x7fdbdc004c20 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8074ed0 tx=0x7fdbe001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3499aa9156b47bd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:28.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.363+0000 7fdbf6000640 1 -- 172.21.15.86:0/1458837123 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe00013a0 con 0x7fdbdc004810 2024-04-25T14:13:28.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.363+0000 7fdbf6000640 1 -- 172.21.15.86:0/1458837123 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbe00052f0 con 0x7fdbdc004810 2024-04-25T14:13:28.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.363+0000 7fdbf6000640 1 -- 172.21.15.86:0/1458837123 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0003840 con 0x7fdbdc004810 2024-04-25T14:13:28.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.367+0000 7fdbfd600640 1 -- 172.21.15.86:0/1458837123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbdc004810 msgr2=0x7fdbdc004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:28.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.367+0000 7fdbfd600640 1 --2- 172.21.15.86:0/1458837123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbdc004810 0x7fdbdc004c20 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8074ed0 tx=0x7fdbe001c3c0 comp rx=0 tx=0).stop 2024-04-25T14:13:28.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.367+0000 7fdbfd600640 1 -- 172.21.15.86:0/1458837123 shutdown_connections 2024-04-25T14:13:28.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.367+0000 7fdbfd600640 1 --2- 172.21.15.86:0/1458837123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbdc004810 0x7fdbdc004c20 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:28.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.367+0000 7fdbfd600640 1 -- 172.21.15.86:0/1458837123 >> 172.21.15.86:0/1458837123 conn(0x7fdbf80b4650 msgr2=0x7fdbf80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:28.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.367+0000 7fdbfd600640 1 -- 172.21.15.86:0/1458837123 shutdown_connections 2024-04-25T14:13:28.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.367+0000 7fdbfd600640 1 -- 172.21.15.86:0/1458837123 wait complete. 2024-04-25T14:13:28.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.367+0000 7fdbfd600640 1 Processor -- start 2024-04-25T14:13:28.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.367+0000 7fdbfd600640 1 -- start start 2024-04-25T14:13:28.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.371+0000 7fdbfd600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbdc004810 0x7fdbf8158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:28.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.371+0000 7fdbfd600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbf815fdc0 0x7fdbf81601d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:28.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.371+0000 7fdbfd600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80cfba0 0x7fdbf80cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:28.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.371+0000 7fdbfd600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf80708c0 con 0x7fdbf80cfba0 2024-04-25T14:13:28.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.371+0000 7fdbfd600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf8070740 con 0x7fdbf815fdc0 2024-04-25T14:13:28.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.371+0000 7fdbfd600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdbf80cdf90 con 0x7fdbdc004810 2024-04-25T14:13:28.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.371+0000 7fdbf7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80cfba0 0x7fdbf80cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:28.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.371+0000 7fdbf7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80cfba0 0x7fdbf80cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41482/0 (socket says 172.21.15.86:41482) 2024-04-25T14:13:28.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.371+0000 7fdbf7400640 1 -- 172.21.15.86:0/2191314845 learned_addr learned my addr 172.21.15.86:0/2191314845 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:28.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.371+0000 7fdbf6a00640 1 --2- 172.21.15.86:0/2191314845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbdc004810 0x7fdbf8158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:28.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.371+0000 7fdbee000640 1 --2- 172.21.15.86:0/2191314845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbf815fdc0 0x7fdbf81601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:28.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.371+0000 7fdbf7400640 1 -- 172.21.15.86:0/2191314845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbdc004810 msgr2=0x7fdbf8158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:28.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.371+0000 7fdbf7400640 1 --2- 172.21.15.86:0/2191314845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbdc004810 0x7fdbf8158720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:28.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.371+0000 7fdbf7400640 1 -- 172.21.15.86:0/2191314845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbf815fdc0 msgr2=0x7fdbf81601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:28.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.371+0000 7fdbf7400640 1 --2- 172.21.15.86:0/2191314845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbf815fdc0 0x7fdbf81601d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:28.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.371+0000 7fdbf7400640 1 -- 172.21.15.86:0/2191314845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf8160710 con 0x7fdbf80cfba0 2024-04-25T14:13:28.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.371+0000 7fdbf7400640 1 --2- 172.21.15.86:0/2191314845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80cfba0 0x7fdbf80cffb0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fdbf815bd10 tx=0x7fdbf000af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:28.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.371+0000 7fdbef400640 1 -- 172.21.15.86:0/2191314845 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf0007840 con 0x7fdbf80cfba0 2024-04-25T14:13:28.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.371+0000 7fdbef400640 1 -- 172.21.15.86:0/2191314845 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbf0019070 con 0x7fdbf80cfba0 2024-04-25T14:13:28.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.371+0000 7fdbfd600640 1 -- 172.21.15.86:0/2191314845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf81f9ab0 con 0x7fdbf80cfba0 2024-04-25T14:13:28.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.371+0000 7fdbef400640 1 -- 172.21.15.86:0/2191314845 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf00133f0 con 0x7fdbf80cfba0 2024-04-25T14:13:28.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.371+0000 7fdbfd600640 1 -- 172.21.15.86:0/2191314845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf81620a0 con 0x7fdbf80cfba0 2024-04-25T14:13:28.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.371+0000 7fdbfd600640 1 -- 172.21.15.86:0/2191314845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf8069d00 con 0x7fdbf80cfba0 2024-04-25T14:13:28.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.375+0000 7fdbef400640 1 -- 172.21.15.86:0/2191314845 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7fdbf0013590 con 0x7fdbf80cfba0 2024-04-25T14:13:28.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.375+0000 7fdbef400640 1 --2- 172.21.15.86:0/2191314845 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fdbd406f550 0x7fdbd4071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.375+0000 7fdbef400640 1 -- 172.21.15.86:0/2191314845 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6109+0+0 (secure 0 0 0) 0x7fdbf008faf0 con 0x7fdbf80cfba0 2024-04-25T14:13:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.375+0000 7fdbf6a00640 1 --2- 172.21.15.86:0/2191314845 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fdbd406f550 0x7fdbd4071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.375+0000 7fdbf6a00640 1 --2- 172.21.15.86:0/2191314845 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fdbd406f550 0x7fdbd4071a60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8006b20 tx=0x7fdbe8009190 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:28.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.379+0000 7fdbef400640 1 -- 172.21.15.86:0/2191314845 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdbf8160710 con 0x7fdbf80cfba0 2024-04-25T14:13:28.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.535+0000 7fdbfd600640 1 -- 172.21.15.86:0/2191314845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fdbf80cb020 con 0x7fdbf80cfba0 2024-04-25T14:13:28.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.539+0000 7fdbef400640 1 -- 172.21.15.86:0/2191314845 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7fdbf80d2030 con 0x7fdbf80cfba0 2024-04-25T14:13:28.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.539+0000 7fdbfd600640 1 -- 172.21.15.86:0/2191314845 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fdbd406f550 msgr2=0x7fdbd4071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:28.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.539+0000 7fdbfd600640 1 --2- 172.21.15.86:0/2191314845 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fdbd406f550 0x7fdbd4071a60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8006b20 tx=0x7fdbe8009190 comp rx=0 tx=0).stop 2024-04-25T14:13:28.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.539+0000 7fdbfd600640 1 -- 172.21.15.86:0/2191314845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80cfba0 msgr2=0x7fdbf80cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:28.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.539+0000 7fdbfd600640 1 --2- 172.21.15.86:0/2191314845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80cfba0 0x7fdbf80cffb0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fdbf815bd10 tx=0x7fdbf000af40 comp rx=0 tx=0).stop 2024-04-25T14:13:28.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.539+0000 7fdbfd600640 1 -- 172.21.15.86:0/2191314845 shutdown_connections 2024-04-25T14:13:28.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.539+0000 7fdbfd600640 1 --2- 172.21.15.86:0/2191314845 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fdbd406f550 0x7fdbd4071a60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:28.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.539+0000 7fdbfd600640 1 --2- 172.21.15.86:0/2191314845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf80cfba0 0x7fdbf80cffb0 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:28.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.539+0000 7fdbfd600640 1 --2- 172.21.15.86:0/2191314845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbf815fdc0 0x7fdbf81601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:28.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.539+0000 7fdbfd600640 1 --2- 172.21.15.86:0/2191314845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbdc004810 0x7fdbf8158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:28.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.539+0000 7fdbfd600640 1 -- 172.21.15.86:0/2191314845 >> 172.21.15.86:0/2191314845 conn(0x7fdbf80b4650 msgr2=0x7fdbf8068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:28.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.543+0000 7fdbfd600640 1 -- 172.21.15.86:0/2191314845 shutdown_connections 2024-04-25T14:13:28.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.543+0000 7fdbfd600640 1 -- 172.21.15.86:0/2191314845 wait complete. 2024-04-25T14:13:28.559 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T14:13:28.559 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:13:28.611 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:13:28.694 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T14:13:28.695 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:13:28.695 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:13:28.703 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2893 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T14:13:28.703 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:13:28.703 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:13:28.713 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4517 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T14:13:28.713 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:13:28.713 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:13:28.755 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2893 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T14:13:28.755 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:13:28.755 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:13:28.765 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4517 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T14:13:28.765 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T14:13:28.765 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:13:28.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.947+0000 7facfd400640 1 Processor -- start 2024-04-25T14:13:28.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.947+0000 7facfd400640 1 -- start start 2024-04-25T14:13:28.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.947+0000 7facfd400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facf81651d0 0x7facf81675f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:28.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.947+0000 7facfd400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facf80b0a60 con 0x7facf815c2a0 2024-04-25T14:13:28.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.947+0000 7facfd400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facf80ae840 con 0x7facf806fd50 2024-04-25T14:13:28.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.947+0000 7facfd400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7facf806ef90 con 0x7facf81651d0 2024-04-25T14:13:28.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.947+0000 7facf6800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7facf815c2a0 0x7facf80ae130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40158/0 (socket says 172.21.15.86:40158) 2024-04-25T14:13:28.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.947+0000 7facf6800640 1 -- 172.21.15.86:0/2746820590 learned_addr learned my addr 172.21.15.86:0/2746820590 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:28.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.951+0000 7facf5400640 1 -- 172.21.15.86:0/2746820590 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1862398402 0 0) 0x7facf80b0a60 con 0x7facf815c2a0 2024-04-25T14:13:28.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.951+0000 7facf5400640 1 -- 172.21.15.86:0/2746820590 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7facdc003530 con 0x7facf815c2a0 2024-04-25T14:13:28.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.951+0000 7facf7200640 1 --2- 172.21.15.86:0/2746820590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facf81651d0 0x7facf81675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:28.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.951+0000 7facf5400640 1 -- 172.21.15.86:0/2746820590 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3630085366 0 0) 0x7facdc003530 con 0x7facf815c2a0 2024-04-25T14:13:28.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.951+0000 7facf5400640 1 -- 172.21.15.86:0/2746820590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facf81651d0 msgr2=0x7facf81675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:28.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.951+0000 7facf5400640 1 --2- 172.21.15.86:0/2746820590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facf81651d0 0x7facf81675f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:28.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.951+0000 7facf5400640 1 -- 172.21.15.86:0/2746820590 >> v1:172.21.15.155:6789/0 conn(0x7facf806fd50 legacy=0x7facf8070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:28.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.951+0000 7facf5400640 1 -- 172.21.15.86:0/2746820590 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf806f270 con 0x7facf815c2a0 2024-04-25T14:13:28.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.951+0000 7facf5400640 1 -- 172.21.15.86:0/2746820590 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7face8001680 con 0x7facf815c2a0 2024-04-25T14:13:28.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.951+0000 7facf7200640 1 --2- 172.21.15.86:0/2746820590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facf81651d0 0x7facf81675f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:28.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.951+0000 7facf5400640 1 -- 172.21.15.86:0/2746820590 >> v1:172.21.15.86:6789/0 conn(0x7facf815c2a0 legacy=0x7facf80ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:28.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.951+0000 7facf5400640 1 --2- 172.21.15.86:0/2746820590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facdc003a60 0x7facdc003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:28.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.951+0000 7facf5400640 1 --2- 172.21.15.86:0/2746820590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facf81651d0 0x7facdc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:28.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.951+0000 7facf5400640 1 --2- 172.21.15.86:0/2746820590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facdc004da0 0x7facdc005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:28.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.951+0000 7facf5400640 1 -- 172.21.15.86:0/2746820590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7facf806ef90 con 0x7facdc004da0 2024-04-25T14:13:28.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.951+0000 7facf5400640 1 -- 172.21.15.86:0/2746820590 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7facdc00bc70 con 0x7facdc003a60 2024-04-25T14:13:28.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.951+0000 7facf5400640 1 -- 172.21.15.86:0/2746820590 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7facdc00be70 con 0x7facf81651d0 2024-04-25T14:13:28.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.951+0000 7facf7200640 1 --2- 172.21.15.86:0/2746820590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facdc004da0 0x7facdc005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:28.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.951+0000 7facf6800640 1 --2- 172.21.15.86:0/2746820590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facdc003a60 0x7facdc003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:28.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.951+0000 7facf5e00640 1 --2- 172.21.15.86:0/2746820590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facf81651d0 0x7facdc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:28.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.951+0000 7facf7200640 1 -- 172.21.15.86:0/2746820590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facf81651d0 msgr2=0x7facdc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:28.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.951+0000 7facf7200640 1 --2- 172.21.15.86:0/2746820590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facf81651d0 0x7facdc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:28.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.951+0000 7facf7200640 1 -- 172.21.15.86:0/2746820590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facdc003a60 msgr2=0x7facdc003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:28.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.951+0000 7facf7200640 1 --2- 172.21.15.86:0/2746820590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facdc003a60 0x7facdc003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:28.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.951+0000 7facf7200640 1 -- 172.21.15.86:0/2746820590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facdc003530 con 0x7facdc004da0 2024-04-25T14:13:28.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.951+0000 7facf6800640 1 --2- 172.21.15.86:0/2746820590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facdc003a60 0x7facdc003ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:28.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.951+0000 7facf5e00640 1 --2- 172.21.15.86:0/2746820590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facf81651d0 0x7facdc004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:28.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.955+0000 7facf7200640 1 --2- 172.21.15.86:0/2746820590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facdc004da0 0x7facdc005270 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7facf80708e0 tx=0x7facf001dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4ee6f18cff1752 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:28.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.955+0000 7facf5400640 1 -- 172.21.15.86:0/2746820590 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf00ac070 con 0x7facdc004da0 2024-04-25T14:13:28.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.955+0000 7facf5400640 1 -- 172.21.15.86:0/2746820590 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facf00aa070 con 0x7facdc004da0 2024-04-25T14:13:28.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.955+0000 7facf5400640 1 -- 172.21.15.86:0/2746820590 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf000a9a0 con 0x7facdc004da0 2024-04-25T14:13:28.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.955+0000 7facfd400640 1 -- 172.21.15.86:0/2746820590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facdc004da0 msgr2=0x7facdc005270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:28.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.955+0000 7facfd400640 1 --2- 172.21.15.86:0/2746820590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facdc004da0 0x7facdc005270 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7facf80708e0 tx=0x7facf001dc70 comp rx=0 tx=0).stop 2024-04-25T14:13:28.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.955+0000 7facf7200640 1 -- 172.21.15.86:0/2746820590 reap_dead start 2024-04-25T14:13:28.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.955+0000 7facfd400640 1 -- 172.21.15.86:0/2746820590 shutdown_connections 2024-04-25T14:13:28.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.955+0000 7facfd400640 1 -- 172.21.15.86:0/2746820590 >> 172.21.15.86:0/2746820590 conn(0x7facf80b4650 msgr2=0x7facf80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:28.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.955+0000 7facfd400640 1 -- 172.21.15.86:0/2746820590 shutdown_connections 2024-04-25T14:13:28.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.955+0000 7facfd400640 1 -- 172.21.15.86:0/2746820590 wait complete. 2024-04-25T14:13:28.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.955+0000 7facfd400640 1 Processor -- start 2024-04-25T14:13:28.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.955+0000 7facfd400640 1 -- start start 2024-04-25T14:13:28.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.959+0000 7facfd400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facf809ef40 0x7facf809a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:28.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.959+0000 7facfd400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facf8148140 0x7facf8164000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:28.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.959+0000 7facfd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facf8148550 0x7facf8164540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:28.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.959+0000 7facfd400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7facf80d0fc0 con 0x7facf8148550 2024-04-25T14:13:28.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.959+0000 7facfd400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7facf806f830 con 0x7facf8148140 2024-04-25T14:13:28.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.959+0000 7facfd400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7facf815c6b0 con 0x7facf809ef40 2024-04-25T14:13:28.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.959+0000 7facf7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facf8148550 0x7facf8164540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:28.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.959+0000 7facf7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facf8148550 0x7facf8164540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55132/0 (socket says 172.21.15.86:55132) 2024-04-25T14:13:28.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.959+0000 7facf7200640 1 -- 172.21.15.86:0/731060897 learned_addr learned my addr 172.21.15.86:0/731060897 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:28.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.959+0000 7facf6800640 1 --2- 172.21.15.86:0/731060897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facf809ef40 0x7facf809a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:28.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.959+0000 7facf5e00640 1 --2- 172.21.15.86:0/731060897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facf8148140 0x7facf8164000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:28.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.959+0000 7facf7200640 1 -- 172.21.15.86:0/731060897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facf809ef40 msgr2=0x7facf809a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:28.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.959+0000 7facf7200640 1 --2- 172.21.15.86:0/731060897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facf809ef40 0x7facf809a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:28.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.959+0000 7facf7200640 1 -- 172.21.15.86:0/731060897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facf8148140 msgr2=0x7facf8164000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:28.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.959+0000 7facf7200640 1 --2- 172.21.15.86:0/731060897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facf8148140 0x7facf8164000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:28.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.959+0000 7facf7200640 1 -- 172.21.15.86:0/731060897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf809ae90 con 0x7facf8148550 2024-04-25T14:13:28.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.959+0000 7facf7200640 1 --2- 172.21.15.86:0/731060897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facf8148550 0x7facf8164540 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7facf001f350 tx=0x7facf001aa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:28.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.959+0000 7faceea00640 1 -- 172.21.15.86:0/731060897 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf00ac040 con 0x7facf8148550 2024-04-25T14:13:28.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.959+0000 7faceea00640 1 -- 172.21.15.86:0/731060897 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facf00b5030 con 0x7facf8148550 2024-04-25T14:13:28.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.959+0000 7facfd400640 1 -- 172.21.15.86:0/731060897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf8148a50 con 0x7facf8148550 2024-04-25T14:13:28.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.959+0000 7faceea00640 1 -- 172.21.15.86:0/731060897 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf0003da0 con 0x7facf8148550 2024-04-25T14:13:28.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.959+0000 7facfd400640 1 -- 172.21.15.86:0/731060897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf8149010 con 0x7facf8148550 2024-04-25T14:13:28.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.959+0000 7facfd400640 1 -- 172.21.15.86:0/731060897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facf8069d00 con 0x7facf8148550 2024-04-25T14:13:28.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.963+0000 7faceea00640 1 -- 172.21.15.86:0/731060897 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7facf00b1670 con 0x7facf8148550 2024-04-25T14:13:28.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.963+0000 7faceea00640 1 --2- 172.21.15.86:0/731060897 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7facd806f550 0x7facd8071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:28.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.963+0000 7faceea00640 1 -- 172.21.15.86:0/731060897 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6109+0+0 (secure 0 0 0) 0x7facf01292c0 con 0x7facf8148550 2024-04-25T14:13:28.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.963+0000 7facf6800640 1 --2- 172.21.15.86:0/731060897 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7facd806f550 0x7facd8071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:28.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.963+0000 7facf6800640 1 --2- 172.21.15.86:0/731060897 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7facd806f550 0x7facd8071a60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7facf80ce760 tx=0x7face8009040 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:28.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:28.967+0000 7faceea00640 1 -- 172.21.15.86:0/731060897 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7facf0017980 con 0x7facf8148550 2024-04-25T14:13:29.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.159+0000 7facfd400640 1 -- 172.21.15.86:0/731060897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7facf8076bf0 con 0x7facf8148550 2024-04-25T14:13:29.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.159+0000 7faceea00640 1 -- 172.21.15.86:0/731060897 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+3876 (secure 0 0 0) 0x7facf00f6020 con 0x7facf8148550 2024-04-25T14:13:29.157 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:13:29.157 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":110,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/1828317917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1828317917},{"type":"v1","addr":"172.21.15.155:6834","nonce":1828317917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":8321,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2470039722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2470039722},{"type":"v1","addr":"172.21.15.155:6835","nonce":2470039722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110},{"gid":9124,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1389714390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1389714390},{"type":"v1","addr":"172.21.15.86:6837","nonce":1389714390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":10119,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/768741462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":768741462},{"type":"v1","addr":"172.21.15.86:6835","nonce":768741462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110}],"filesystems":[]} 2024-04-25T14:13:29.157 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 110 2024-04-25T14:13:29.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.159+0000 7facfd400640 1 -- 172.21.15.86:0/731060897 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7facd806f550 msgr2=0x7facd8071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:29.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.159+0000 7facfd400640 1 --2- 172.21.15.86:0/731060897 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7facd806f550 0x7facd8071a60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7facf80ce760 tx=0x7face8009040 comp rx=0 tx=0).stop 2024-04-25T14:13:29.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.159+0000 7facfd400640 1 -- 172.21.15.86:0/731060897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facf8148550 msgr2=0x7facf8164540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:29.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.159+0000 7facfd400640 1 --2- 172.21.15.86:0/731060897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facf8148550 0x7facf8164540 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7facf001f350 tx=0x7facf001aa10 comp rx=0 tx=0).stop 2024-04-25T14:13:29.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.159+0000 7facfd400640 1 -- 172.21.15.86:0/731060897 shutdown_connections 2024-04-25T14:13:29.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.159+0000 7facfd400640 1 --2- 172.21.15.86:0/731060897 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7facd806f550 0x7facd8071a60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:29.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.159+0000 7facfd400640 1 --2- 172.21.15.86:0/731060897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facf8148550 0x7facf8164540 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:29.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.159+0000 7facfd400640 1 --2- 172.21.15.86:0/731060897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facf8148140 0x7facf8164000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:29.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.159+0000 7facfd400640 1 --2- 172.21.15.86:0/731060897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facf809ef40 0x7facf809a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:29.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.159+0000 7facfd400640 1 -- 172.21.15.86:0/731060897 >> 172.21.15.86:0/731060897 conn(0x7facf80b4650 msgr2=0x7facf815cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:29.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.159+0000 7facfd400640 1 -- 172.21.15.86:0/731060897 shutdown_connections 2024-04-25T14:13:29.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.159+0000 7facfd400640 1 -- 172.21.15.86:0/731060897 wait complete. 2024-04-25T14:13:29.178 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T14:13:29.178 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T14:13:29.179 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T14:13:29.179 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:13:29.179 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T14:13:29.179 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T14:13:29.179 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T14:13:29.179 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:13:29.179 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T14:13:29.179 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T14:13:29.180 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T14:13:29.180 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:13:29.180 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T14:13:29.180 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T14:13:29.180 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T14:13:29.180 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:13:29.192 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T14:13:29.192 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T14:13:29.195 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T14:13:29.195 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T14:13:29.196 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T14:13:29.196 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T14:13:29.201 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T14:13:29.201 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T14:13:29.245 INFO:tasks.ceph.mds.c:Started 2024-04-25T14:13:29.246 DEBUG:teuthology.parallel:result is None 2024-04-25T14:13:29.248 INFO:tasks.ceph.mds.a:Started 2024-04-25T14:13:29.248 DEBUG:teuthology.parallel:result is None 2024-04-25T14:13:29.249 INFO:tasks.ceph.mds.b:Started 2024-04-25T14:13:29.250 DEBUG:teuthology.parallel:result is None 2024-04-25T14:13:29.251 INFO:tasks.ceph.mds.d:Started 2024-04-25T14:13:29.252 DEBUG:teuthology.parallel:result is None 2024-04-25T14:13:29.252 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T14:13:29.344 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-04-25T14:13:29.347 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2024-04-25T14:13:29.348 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-25T14:13:29.350 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2024-04-25T14:13:29.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.411+0000 7f1711800640 1 Processor -- start 2024-04-25T14:13:29.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.411+0000 7f1711800640 1 -- start start 2024-04-25T14:13:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.411+0000 7f1711800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f170c0d16e0 0x7f170c0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.411+0000 7f1711800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f170c0d2030 con 0x7f170c0cfca0 2024-04-25T14:13:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.411+0000 7f1711800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f170c072bd0 con 0x7f170c0730b0 2024-04-25T14:13:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.411+0000 7f1711800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f170c070470 con 0x7f170c0d16e0 2024-04-25T14:13:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.411+0000 7f170aa00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f170c0cfca0 0x7f170c06f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40174/0 (socket says 172.21.15.86:40174) 2024-04-25T14:13:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.411+0000 7f170aa00640 1 -- 172.21.15.86:0/2855996451 learned_addr learned my addr 172.21.15.86:0/2855996451 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.411+0000 7f1703e00640 1 --2- 172.21.15.86:0/2855996451 >> v2:172.21.15.155:3300/0 conn(0x7f170c0d16e0 0x7f170c0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.411+0000 7f170a000640 1 -- 172.21.15.86:0/2855996451 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1410217700 0 0) 0x7f16fc001120 con 0x7f170c0cfca0 2024-04-25T14:13:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.411+0000 7f170a000640 1 -- 172.21.15.86:0/2855996451 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16f0003530 con 0x7f170c0cfca0 2024-04-25T14:13:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.411+0000 7f170a000640 1 -- 172.21.15.86:0/2855996451 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1061968836 0 0) 0x7f170c072bd0 con 0x7f170c0730b0 2024-04-25T14:13:29.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.411+0000 7f170a000640 1 -- 172.21.15.86:0/2855996451 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16fc001120 con 0x7f170c0730b0 2024-04-25T14:13:29.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.411+0000 7f1703e00640 1 -- 172.21.15.86:0/2855996451 >> v1:172.21.15.155:6789/0 conn(0x7f170c0730b0 legacy=0x7f170c0734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:29.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.411+0000 7f1703e00640 1 -- 172.21.15.86:0/2855996451 >> v1:172.21.15.86:6789/0 conn(0x7f170c0cfca0 legacy=0x7f170c06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:29.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.411+0000 7f1703e00640 1 -- 172.21.15.86:0/2855996451 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170c06fbd0 con 0x7f170c0d16e0 2024-04-25T14:13:29.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.415+0000 7f1703e00640 1 --2- 172.21.15.86:0/2855996451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f170c0d16e0 0x7f170c0d1af0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f16f40097c0 tx=0x7f16f401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c60970ab13656e0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:29.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.415+0000 7f170a000640 1 -- 172.21.15.86:0/2855996451 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f40b1070 con 0x7f170c0d16e0 2024-04-25T14:13:29.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.415+0000 7f170a000640 1 -- 172.21.15.86:0/2855996451 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16f40ac410 con 0x7f170c0d16e0 2024-04-25T14:13:29.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.415+0000 7f170a000640 1 -- 172.21.15.86:0/2855996451 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c0743e0 con 0x7f170c0d16e0 2024-04-25T14:13:29.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.415+0000 7f1711800640 1 -- 172.21.15.86:0/2855996451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f170c0d16e0 msgr2=0x7f170c0d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:29.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.415+0000 7f1711800640 1 --2- 172.21.15.86:0/2855996451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f170c0d16e0 0x7f170c0d1af0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f16f40097c0 tx=0x7f16f401c4a0 comp rx=0 tx=0).stop 2024-04-25T14:13:29.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.415+0000 7f1711800640 1 -- 172.21.15.86:0/2855996451 shutdown_connections 2024-04-25T14:13:29.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.415+0000 7f1711800640 1 --2- 172.21.15.86:0/2855996451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f170c0d16e0 0x7f170c0d1af0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:29.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.415+0000 7f1711800640 1 -- 172.21.15.86:0/2855996451 >> 172.21.15.86:0/2855996451 conn(0x7f170c0b4650 msgr2=0x7f170c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:29.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.415+0000 7f1711800640 1 -- 172.21.15.86:0/2855996451 shutdown_connections 2024-04-25T14:13:29.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.415+0000 7f1711800640 1 -- 172.21.15.86:0/2855996451 wait complete. 2024-04-25T14:13:29.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.419+0000 7f1711800640 1 Processor -- start 2024-04-25T14:13:29.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.419+0000 7f1711800640 1 -- start start 2024-04-25T14:13:29.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.419+0000 7f1711800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f170c0730b0 0x7f170c1fa190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:29.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.419+0000 7f1711800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f170c0cfca0 0x7f170c158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:29.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.419+0000 7f1711800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f170c0d16e0 0x7f170c159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:29.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.419+0000 7f1711800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f170c16aa80 con 0x7f170c0cfca0 2024-04-25T14:13:29.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.419+0000 7f1711800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f170c16a900 con 0x7f170c0d16e0 2024-04-25T14:13:29.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.419+0000 7f1711800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f170c16ac00 con 0x7f170c0730b0 2024-04-25T14:13:29.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.419+0000 7f1703e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f170c0cfca0 0x7f170c158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:29.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.419+0000 7f1703e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f170c0cfca0 0x7f170c158f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55176/0 (socket says 172.21.15.86:55176) 2024-04-25T14:13:29.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.419+0000 7f1703e00640 1 -- 172.21.15.86:0/1135716188 learned_addr learned my addr 172.21.15.86:0/1135716188 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:29.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.419+0000 7f170aa00640 1 --2- 172.21.15.86:0/1135716188 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f170c0730b0 0x7f170c1fa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:29.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.419+0000 7f170b400640 1 --2- 172.21.15.86:0/1135716188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f170c0d16e0 0x7f170c159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:29.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.423+0000 7f1703e00640 1 -- 172.21.15.86:0/1135716188 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f170c0730b0 msgr2=0x7f170c1fa190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:29.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.423+0000 7f1703e00640 1 --2- 172.21.15.86:0/1135716188 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f170c0730b0 0x7f170c1fa190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:29.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.423+0000 7f1703e00640 1 -- 172.21.15.86:0/1135716188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f170c0d16e0 msgr2=0x7f170c159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:29.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.423+0000 7f1703e00640 1 --2- 172.21.15.86:0/1135716188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f170c0d16e0 0x7f170c159450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:29.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.423+0000 7f1703e00640 1 -- 172.21.15.86:0/1135716188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170c0ce5e0 con 0x7f170c0cfca0 2024-04-25T14:13:29.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.423+0000 7f1703e00640 1 --2- 172.21.15.86:0/1135716188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f170c0cfca0 0x7f170c158f10 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f16f40190b0 tx=0x7f16f4019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:29.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.423+0000 7f1702a00640 1 -- 172.21.15.86:0/1135716188 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f40b1040 con 0x7f170c0cfca0 2024-04-25T14:13:29.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.423+0000 7f1702a00640 1 -- 172.21.15.86:0/1135716188 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16f40ba070 con 0x7f170c0cfca0 2024-04-25T14:13:29.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.423+0000 7f1702a00640 1 -- 172.21.15.86:0/1135716188 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f40b5650 con 0x7f170c0cfca0 2024-04-25T14:13:29.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.423+0000 7f1711800640 1 -- 172.21.15.86:0/1135716188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f170c15f2d0 con 0x7f170c0cfca0 2024-04-25T14:13:29.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.423+0000 7f1711800640 1 -- 172.21.15.86:0/1135716188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f170c1f8350 con 0x7f170c0cfca0 2024-04-25T14:13:29.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.423+0000 7f1711800640 1 -- 172.21.15.86:0/1135716188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f170c09f4b0 con 0x7f170c0cfca0 2024-04-25T14:13:29.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.423+0000 7f1702a00640 1 -- 172.21.15.86:0/1135716188 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7f16f40b57f0 con 0x7f170c0cfca0 2024-04-25T14:13:29.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.423+0000 7f1702a00640 1 --2- 172.21.15.86:0/1135716188 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f16e806f550 0x7f16e8071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:29.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.423+0000 7f1702a00640 1 -- 172.21.15.86:0/1135716188 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6109+0+0 (secure 0 0 0) 0x7f16f4129010 con 0x7f170c0cfca0 2024-04-25T14:13:29.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.427+0000 7f170aa00640 1 --2- 172.21.15.86:0/1135716188 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f16e806f550 0x7f16e8071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:29.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.427+0000 7f170aa00640 1 --2- 172.21.15.86:0/1135716188 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f16e806f550 0x7f16e8071a60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f16fc007fd0 tx=0x7f16fc009040 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:29.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.431+0000 7f1702a00640 1 -- 172.21.15.86:0/1135716188 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f16f4016090 con 0x7f170c0cfca0 2024-04-25T14:13:29.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:29.599+0000 7f1711800640 1 -- 172.21.15.86:0/1135716188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f170c0cbfa0 con 0x7f170c0cfca0 2024-04-25T14:13:30.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.191+0000 7f1702a00640 1 -- 172.21.15.86:0/1135716188 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v149) ==== 97+0+0 (secure 0 0 0) 0x7f16f40f5620 con 0x7f170c0cfca0 2024-04-25T14:13:30.191 INFO:teuthology.orchestra.run.smithi086.stderr: removed all blocklist entries 2024-04-25T14:13:30.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.195+0000 7f1711800640 1 -- 172.21.15.86:0/1135716188 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f16e806f550 msgr2=0x7f16e8071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:30.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.195+0000 7f1711800640 1 --2- 172.21.15.86:0/1135716188 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f16e806f550 0x7f16e8071a60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f16fc007fd0 tx=0x7f16fc009040 comp rx=0 tx=0).stop 2024-04-25T14:13:30.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.195+0000 7f1711800640 1 -- 172.21.15.86:0/1135716188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f170c0cfca0 msgr2=0x7f170c158f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:30.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.195+0000 7f1711800640 1 --2- 172.21.15.86:0/1135716188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f170c0cfca0 0x7f170c158f10 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f16f40190b0 tx=0x7f16f4019240 comp rx=0 tx=0).stop 2024-04-25T14:13:30.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.195+0000 7f1711800640 1 -- 172.21.15.86:0/1135716188 shutdown_connections 2024-04-25T14:13:30.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.195+0000 7f1711800640 1 --2- 172.21.15.86:0/1135716188 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f16e806f550 0x7f16e8071a60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:30.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.195+0000 7f1711800640 1 --2- 172.21.15.86:0/1135716188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f170c0d16e0 0x7f170c159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:30.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.195+0000 7f1711800640 1 --2- 172.21.15.86:0/1135716188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f170c0cfca0 0x7f170c158f10 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:30.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.195+0000 7f1711800640 1 --2- 172.21.15.86:0/1135716188 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f170c0730b0 0x7f170c1fa190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:30.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.195+0000 7f1711800640 1 -- 172.21.15.86:0/1135716188 >> 172.21.15.86:0/1135716188 conn(0x7f170c0b4650 msgr2=0x7f170c0cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:30.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.195+0000 7f1711800640 1 -- 172.21.15.86:0/1135716188 shutdown_connections 2024-04-25T14:13:30.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.195+0000 7f1711800640 1 -- 172.21.15.86:0/1135716188 wait complete. 2024-04-25T14:13:30.218 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T14:13:30.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.367+0000 7f1cdf800640 1 Processor -- start 2024-04-25T14:13:30.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.367+0000 7f1cdf800640 1 -- start start 2024-04-25T14:13:30.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.371+0000 7f1cdf800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1cd80c68d0 0x7f1cd8167df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:30.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.371+0000 7f1cdf800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1cd8072d40 0x7f1cd806e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:30.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.371+0000 7f1cdf800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cd8168490 con 0x7f1cd80cdf20 2024-04-25T14:13:30.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.371+0000 7f1cdf800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1cd8168d30 con 0x7f1cd80c68d0 2024-04-25T14:13:30.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.371+0000 7f1cdf800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd806e500 con 0x7f1cd8072d40 2024-04-25T14:13:30.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.371+0000 7f1cdd800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1cd8072d40 0x7f1cd806e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:30.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.371+0000 7f1cdce00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1cd80c68d0 0x7f1cd8167df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:30.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.371+0000 7f1cd7e00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f1cd80cdf20 0x7f1cd8072630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:46640/0 (socket says 172.21.15.86:46640) 2024-04-25T14:13:30.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.371+0000 7f1cd7e00640 1 -- 172.21.15.86:0/1133221472 learned_addr learned my addr 172.21.15.86:0/1133221472 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:30.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.371+0000 7f1cdce00640 1 -- 172.21.15.86:0/1133221472 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1cd8072d40 msgr2=0x7f1cd806e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:30.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.371+0000 7f1cdce00640 1 --2- 172.21.15.86:0/1133221472 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1cd8072d40 0x7f1cd806e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:30.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.371+0000 7f1cdce00640 1 -- 172.21.15.86:0/1133221472 >> v1:172.21.15.155:6789/0 conn(0x7f1cd80cdf20 legacy=0x7f1cd8072630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:30.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.371+0000 7f1cdce00640 1 -- 172.21.15.86:0/1133221472 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd8168770 con 0x7f1cd80c68d0 2024-04-25T14:13:30.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.371+0000 7f1cd7400640 1 -- 172.21.15.86:0/1133221472 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3210870613 0 0) 0x7f1cd8168490 con 0x7f1cd80cdf20 2024-04-25T14:13:30.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.371+0000 7f1cdd800640 1 --2- 172.21.15.86:0/1133221472 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1cd8072d40 0x7f1cd806e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:13:30.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.371+0000 7f1cdce00640 1 --2- 172.21.15.86:0/1133221472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cd80c68d0 0x7f1cd8167df0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80097c0 tx=0x7f1cc801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a51c3a142be3bc55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:30.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.375+0000 7f1cd7400640 1 -- 172.21.15.86:0/1133221472 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc80b1070 con 0x7f1cd80c68d0 2024-04-25T14:13:30.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.375+0000 7f1cd7400640 1 -- 172.21.15.86:0/1133221472 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc80ac620 con 0x7f1cd80c68d0 2024-04-25T14:13:30.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.375+0000 7f1cd7400640 1 -- 172.21.15.86:0/1133221472 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc80ac9f0 con 0x7f1cd80c68d0 2024-04-25T14:13:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.375+0000 7f1cdf800640 1 -- 172.21.15.86:0/1133221472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cd80c68d0 msgr2=0x7f1cd8167df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.375+0000 7f1cdf800640 1 --2- 172.21.15.86:0/1133221472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cd80c68d0 0x7f1cd8167df0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80097c0 tx=0x7f1cc801c3e0 comp rx=0 tx=0).stop 2024-04-25T14:13:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.375+0000 7f1cdf800640 1 -- 172.21.15.86:0/1133221472 shutdown_connections 2024-04-25T14:13:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.375+0000 7f1cdf800640 1 --2- 172.21.15.86:0/1133221472 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1cd8072d40 0x7f1cd806e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.375+0000 7f1cdf800640 1 --2- 172.21.15.86:0/1133221472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cd80c68d0 0x7f1cd8167df0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.375+0000 7f1cdf800640 1 -- 172.21.15.86:0/1133221472 >> 172.21.15.86:0/1133221472 conn(0x7f1cd80d3580 msgr2=0x7f1cd80d3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.375+0000 7f1cdf800640 1 -- 172.21.15.86:0/1133221472 shutdown_connections 2024-04-25T14:13:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.375+0000 7f1cdf800640 1 -- 172.21.15.86:0/1133221472 wait complete. 2024-04-25T14:13:30.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.375+0000 7f1cdf800640 1 Processor -- start 2024-04-25T14:13:30.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.375+0000 7f1cdf800640 1 -- start start 2024-04-25T14:13:30.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.379+0000 7f1cdf800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1cd80c68d0 0x7f1cd8155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:30.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.379+0000 7f1cdf800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cd80cdf20 0x7f1cd81637a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:30.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.379+0000 7f1cdf800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cd8163ce0 0x7f1cd815be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:30.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.379+0000 7f1cdf800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd81712b0 con 0x7f1cd80cdf20 2024-04-25T14:13:30.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.379+0000 7f1cdf800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd8171130 con 0x7f1cd8163ce0 2024-04-25T14:13:30.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.379+0000 7f1cdf800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd8171430 con 0x7f1cd80c68d0 2024-04-25T14:13:30.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.379+0000 7f1cd7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cd80cdf20 0x7f1cd81637a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:30.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.379+0000 7f1cd7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cd80cdf20 0x7f1cd81637a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55190/0 (socket says 172.21.15.86:55190) 2024-04-25T14:13:30.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.379+0000 7f1cd7e00640 1 -- 172.21.15.86:0/3534802415 learned_addr learned my addr 172.21.15.86:0/3534802415 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:30.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.379+0000 7f1cdce00640 1 --2- 172.21.15.86:0/3534802415 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1cd80c68d0 0x7f1cd8155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:30.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.379+0000 7f1cdd800640 1 --2- 172.21.15.86:0/3534802415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cd8163ce0 0x7f1cd815be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:30.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.379+0000 7f1cd7e00640 1 -- 172.21.15.86:0/3534802415 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1cd80c68d0 msgr2=0x7f1cd8155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:30.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.379+0000 7f1cd7e00640 1 --2- 172.21.15.86:0/3534802415 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1cd80c68d0 0x7f1cd8155550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:30.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.379+0000 7f1cd7e00640 1 -- 172.21.15.86:0/3534802415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cd8163ce0 msgr2=0x7f1cd815be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:30.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.379+0000 7f1cd7e00640 1 --2- 172.21.15.86:0/3534802415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cd8163ce0 0x7f1cd815be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:30.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.379+0000 7f1cd7e00640 1 -- 172.21.15.86:0/3534802415 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd8155a90 con 0x7f1cd80cdf20 2024-04-25T14:13:30.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.379+0000 7f1cd7e00640 1 --2- 172.21.15.86:0/3534802415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cd80cdf20 0x7f1cd81637a0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4005890 tx=0x7f1cc400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:30.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.379+0000 7f1cd4c00640 1 -- 172.21.15.86:0/3534802415 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4017070 con 0x7f1cd80cdf20 2024-04-25T14:13:30.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.379+0000 7f1cd4c00640 1 -- 172.21.15.86:0/3534802415 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc4005d10 con 0x7f1cd80cdf20 2024-04-25T14:13:30.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.379+0000 7f1cdf800640 1 -- 172.21.15.86:0/3534802415 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd8162390 con 0x7f1cd80cdf20 2024-04-25T14:13:30.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.379+0000 7f1cd4c00640 1 -- 172.21.15.86:0/3534802415 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4007310 con 0x7f1cd80cdf20 2024-04-25T14:13:30.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.379+0000 7f1cdf800640 1 -- 172.21.15.86:0/3534802415 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd81561b0 con 0x7f1cd80cdf20 2024-04-25T14:13:30.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.379+0000 7f1cdf800640 1 -- 172.21.15.86:0/3534802415 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cd8072170 con 0x7f1cd80cdf20 2024-04-25T14:13:30.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.383+0000 7f1cd4c00640 1 -- 172.21.15.86:0/3534802415 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7f1cc401a070 con 0x7f1cd80cdf20 2024-04-25T14:13:30.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.383+0000 7f1cd4c00640 1 --2- 172.21.15.86:0/3534802415 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f1cb806f550 0x7f1cb8071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:30.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.383+0000 7f1cd4c00640 1 -- 172.21.15.86:0/3534802415 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5249+0+0 (secure 0 0 0) 0x7f1cc408e520 con 0x7f1cd80cdf20 2024-04-25T14:13:30.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.383+0000 7f1cdce00640 1 --2- 172.21.15.86:0/3534802415 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f1cb806f550 0x7f1cb8071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:30.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.383+0000 7f1cdce00640 1 --2- 172.21.15.86:0/3534802415 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f1cb806f550 0x7f1cb8071a60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80ae000 tx=0x7f1cc80a7000 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:30.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.387+0000 7f1cd4c00640 1 -- 172.21.15.86:0/3534802415 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1cc405ae60 con 0x7f1cd80cdf20 2024-04-25T14:13:30.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.599+0000 7f1cdf800640 1 -- 172.21.15.86:0/3534802415 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1cd8076bf0 con 0x7f1cd80cdf20 2024-04-25T14:13:30.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.599+0000 7f1cd4c00640 1 -- 172.21.15.86:0/3534802415 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v30) ==== 80+0+5238 (secure 0 0 0) 0x7f1cc405f160 con 0x7f1cd80cdf20 2024-04-25T14:13:30.597 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:13:30.597 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:13:30.597 INFO:teuthology.orchestra.run.smithi086.stdout: "auth_dump": [ 2024-04-25T14:13:30.597 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:13:30.598 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.a", 2024-04-25T14:13:30.598 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmWk5fAxAAKEBKhWewpl23MUcvMi8qDg==", 2024-04-25T14:13:30.598 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:13:30.598 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:13:30.598 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:13:30.598 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:13:30.598 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:13:30.598 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:13:30.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:13:30.598 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:13:30.598 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.b", 2024-04-25T14:13:30.598 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmDhzOEBAABNys/NAEhucQ7A+HzQlzQw==", 2024-04-25T14:13:30.598 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:13:30.599 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:13:30.599 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:13:30.599 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:13:30.599 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:13:30.599 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:13:30.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:13:30.599 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:13:30.599 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.c", 2024-04-25T14:13:30.599 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmA/hNCxAAclDGbIjLfjqyXn0MwX/IkA==", 2024-04-25T14:13:30.599 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:13:30.599 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:13:30.599 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:13:30.599 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:13:30.599 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:13:30.600 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:13:30.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:13:30.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:13:30.600 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.d", 2024-04-25T14:13:30.600 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmemCcGBAAGrCx6C850qRJSWYjebxOBA==", 2024-04-25T14:13:30.600 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:13:30.600 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:13:30.600 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:13:30.600 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:13:30.600 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:13:30.600 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:13:30.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:13:30.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:13:30.600 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.0", 2024-04-25T14:13:30.600 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCOYipm9etgGxAAemg9CqXN0ShduDtDuKF2Zw==", 2024-04-25T14:13:30.601 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:13:30.601 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:13:30.601 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:13:30.601 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:13:30.601 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:13:30.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:13:30.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:13:30.601 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.1", 2024-04-25T14:13:30.601 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCRYipmBCiNJhAABAWHk8NEGVvqieL2klGhmA==", 2024-04-25T14:13:30.601 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:13:30.601 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:13:30.601 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:13:30.601 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:13:30.601 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:13:30.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:13:30.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:13:30.602 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.2", 2024-04-25T14:13:30.602 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCUYipm/XGYLxAAPXsw1svKseFzFiuDUjAsJQ==", 2024-04-25T14:13:30.602 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:13:30.602 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:13:30.602 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:13:30.602 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:13:30.602 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:13:30.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:13:30.602 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:13:30.602 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.3", 2024-04-25T14:13:30.602 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCXYipmlSRaORAAwpKyzZDdVYHWl2pAsQjmpw==", 2024-04-25T14:13:30.602 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:13:30.602 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:13:30.603 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:13:30.603 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:13:30.603 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:13:30.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:13:30.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:13:30.603 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.4", 2024-04-25T14:13:30.603 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCdYipmBI3FMBAAV2NdggfRSFnQvrcOC2YqiA==", 2024-04-25T14:13:30.603 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:13:30.603 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:13:30.603 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:13:30.603 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:13:30.603 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:13:30.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:13:30.603 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:13:30.604 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.5", 2024-04-25T14:13:30.604 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQChYipmctYxABAABwmOMgyYuOfRWD9KdHILng==", 2024-04-25T14:13:30.604 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:13:30.604 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:13:30.604 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:13:30.604 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:13:30.604 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:13:30.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:13:30.604 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:13:30.604 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.6", 2024-04-25T14:13:30.604 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCkYipmSL7TCRAAhFMWg1AQm54bB+x1GDcusA==", 2024-04-25T14:13:30.604 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:13:30.604 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:13:30.604 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:13:30.605 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:13:30.605 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:13:30.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:13:30.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:13:30.605 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.7", 2024-04-25T14:13:30.605 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCnYipmZ4F+FRAAEGaZN+byv/qJUFZb7WxTFg==", 2024-04-25T14:13:30.605 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:13:30.605 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-25T14:13:30.605 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-25T14:13:30.605 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:13:30.605 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:13:30.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:13:30.605 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:13:30.605 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.0", 2024-04-25T14:13:30.605 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipm5u4xHRAACDFe3Eb4KLrsdLj4eFe7mA==", 2024-04-25T14:13:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:13:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:13:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-25T14:13:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T14:13:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:13:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:13:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:13:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.1", 2024-04-25T14:13:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmDp1hJRAAQkBc5PdQoMTT7W4HahkTyg==", 2024-04-25T14:13:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:13:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:13:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-25T14:13:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw pool=fs2_data" 2024-04-25T14:13:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:13:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:13:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:13:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.2", 2024-04-25T14:13:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmhbhGIRAAxao/J0WY53IITLETFgwIVw==", 2024-04-25T14:13:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:13:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:13:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-25T14:13:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw pool=fs2_data" 2024-04-25T14:13:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:13:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:13:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:13:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.3", 2024-04-25T14:13:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCLYipmJsCBKRAAzXn2qDtblIpqt7U/TUUlyA==", 2024-04-25T14:13:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:13:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-25T14:13:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-25T14:13:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw pool=fs2_data" 2024-04-25T14:13:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:13:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:13:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:13:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.admin", 2024-04-25T14:13:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCKYipmTqbVIBAA1VpOtj24f2ibVEIT4EJwmw==", 2024-04-25T14:13:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:13:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-25T14:13:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-25T14:13:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-25T14:13:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:13:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:13:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:13:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:13:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.x", 2024-04-25T14:13:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCKYipmKMzAORAAtRyTNJ5HWi2XwuamoqwB0Q==", 2024-04-25T14:13:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:13:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-25T14:13:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-25T14:13:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:13:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:13:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-25T14:13:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:13:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.y", 2024-04-25T14:13:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQCKYipmC7A7NRAAEwVNA0yxt3fwdHXrsV0dvg==", 2024-04-25T14:13:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-25T14:13:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-25T14:13:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-25T14:13:30.610 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-25T14:13:30.610 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:13:30.610 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:13:30.610 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-25T14:13:30.610 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:13:30.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.599+0000 7f1cdf800640 1 -- 172.21.15.86:0/3534802415 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f1cb806f550 msgr2=0x7f1cb8071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:30.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.599+0000 7f1cdf800640 1 --2- 172.21.15.86:0/3534802415 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f1cb806f550 0x7f1cb8071a60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80ae000 tx=0x7f1cc80a7000 comp rx=0 tx=0).stop 2024-04-25T14:13:30.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.599+0000 7f1cdf800640 1 -- 172.21.15.86:0/3534802415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cd80cdf20 msgr2=0x7f1cd81637a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:30.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.599+0000 7f1cdf800640 1 --2- 172.21.15.86:0/3534802415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cd80cdf20 0x7f1cd81637a0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4005890 tx=0x7f1cc400c820 comp rx=0 tx=0).stop 2024-04-25T14:13:30.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.599+0000 7f1cdf800640 1 -- 172.21.15.86:0/3534802415 shutdown_connections 2024-04-25T14:13:30.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.599+0000 7f1cdf800640 1 --2- 172.21.15.86:0/3534802415 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f1cb806f550 0x7f1cb8071a60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:30.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.599+0000 7f1cdf800640 1 --2- 172.21.15.86:0/3534802415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cd8163ce0 0x7f1cd815be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:30.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.599+0000 7f1cdf800640 1 --2- 172.21.15.86:0/3534802415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cd80cdf20 0x7f1cd81637a0 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:30.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.599+0000 7f1cdf800640 1 --2- 172.21.15.86:0/3534802415 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1cd80c68d0 0x7f1cd8155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:30.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.599+0000 7f1cdf800640 1 -- 172.21.15.86:0/3534802415 >> 172.21.15.86:0/3534802415 conn(0x7f1cd80d3580 msgr2=0x7f1cd80b48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:30.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.599+0000 7f1cdf800640 1 -- 172.21.15.86:0/3534802415 shutdown_connections 2024-04-25T14:13:30.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.599+0000 7f1cdf800640 1 -- 172.21.15.86:0/3534802415 wait complete. 2024-04-25T14:13:30.619 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:13:30.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.767+0000 7f0b6be00640 1 Processor -- start 2024-04-25T14:13:30.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.767+0000 7f0b6be00640 1 -- start start 2024-04-25T14:13:30.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.767+0000 7f0b6be00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b640b3a90 0x7f0b6415cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:30.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.767+0000 7f0b6be00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0b640cf500 0x7f0b640d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:30.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.767+0000 7f0b6be00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b640cf910 con 0x7f0b640ce430 2024-04-25T14:13:30.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.767+0000 7f0b6be00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0b640cdfa0 con 0x7f0b640cf500 2024-04-25T14:13:30.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.767+0000 7f0b6be00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0b640b3ea0 con 0x7f0b640b3a90 2024-04-25T14:13:30.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.767+0000 7f0b68a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0b640ce430 0x7f0b640cedf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40176/0 (socket says 172.21.15.86:40176) 2024-04-25T14:13:30.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.767+0000 7f0b68a00640 1 -- 172.21.15.86:0/919363037 learned_addr learned my addr 172.21.15.86:0/919363037 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:30.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.767+0000 7f0b69400640 1 --2- 172.21.15.86:0/919363037 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b640b3a90 0x7f0b6415cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:30.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.767+0000 7f0b69e00640 1 --2- 172.21.15.86:0/919363037 >> v2:172.21.15.155:3300/0 conn(0x7f0b640cf500 0x7f0b640d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:30.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.767+0000 7f0b63e00640 1 -- 172.21.15.86:0/919363037 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 536707379 0 0) 0x7f0b640cf910 con 0x7f0b640ce430 2024-04-25T14:13:30.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.767+0000 7f0b63e00640 1 -- 172.21.15.86:0/919363037 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b4c003530 con 0x7f0b640ce430 2024-04-25T14:13:30.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.767+0000 7f0b69400640 1 -- 172.21.15.86:0/919363037 >> v2:172.21.15.155:3300/0 conn(0x7f0b640cf500 msgr2=0x7f0b640d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:30.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.767+0000 7f0b69400640 1 --2- 172.21.15.86:0/919363037 >> v2:172.21.15.155:3300/0 conn(0x7f0b640cf500 0x7f0b640d0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:30.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.767+0000 7f0b69400640 1 -- 172.21.15.86:0/919363037 >> v1:172.21.15.86:6789/0 conn(0x7f0b640ce430 legacy=0x7f0b640cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:30.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.767+0000 7f0b69400640 1 -- 172.21.15.86:0/919363037 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b640cfbc0 con 0x7f0b640b3a90 2024-04-25T14:13:30.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.767+0000 7f0b69e00640 1 --2- 172.21.15.86:0/919363037 >> v2:172.21.15.155:3300/0 conn(0x7f0b640cf500 0x7f0b640d0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:13:30.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.771+0000 7f0b69400640 1 --2- 172.21.15.86:0/919363037 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b640b3a90 0x7f0b6415cd30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0b580097c0 tx=0x7f0b5801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=799680a32957a56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:30.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.771+0000 7f0b63e00640 1 -- 172.21.15.86:0/919363037 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b580b2070 con 0x7f0b640b3a90 2024-04-25T14:13:30.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.771+0000 7f0b63e00640 1 -- 172.21.15.86:0/919363037 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b580ad640 con 0x7f0b640b3a90 2024-04-25T14:13:30.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.771+0000 7f0b63e00640 1 -- 172.21.15.86:0/919363037 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b580ada10 con 0x7f0b640b3a90 2024-04-25T14:13:30.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.771+0000 7f0b6be00640 1 -- 172.21.15.86:0/919363037 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b640b3a90 msgr2=0x7f0b6415cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:30.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.771+0000 7f0b6be00640 1 --2- 172.21.15.86:0/919363037 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b640b3a90 0x7f0b6415cd30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0b580097c0 tx=0x7f0b5801c4a0 comp rx=0 tx=0).stop 2024-04-25T14:13:30.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.771+0000 7f0b6be00640 1 -- 172.21.15.86:0/919363037 shutdown_connections 2024-04-25T14:13:30.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.771+0000 7f0b6be00640 1 --2- 172.21.15.86:0/919363037 >> v2:172.21.15.155:3300/0 conn(0x7f0b640cf500 0x7f0b640d0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:30.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.771+0000 7f0b6be00640 1 --2- 172.21.15.86:0/919363037 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b640b3a90 0x7f0b6415cd30 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:30.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.771+0000 7f0b6be00640 1 -- 172.21.15.86:0/919363037 >> 172.21.15.86:0/919363037 conn(0x7f0b6409ba30 msgr2=0x7f0b640b73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:30.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.771+0000 7f0b6be00640 1 -- 172.21.15.86:0/919363037 shutdown_connections 2024-04-25T14:13:30.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.771+0000 7f0b6be00640 1 -- 172.21.15.86:0/919363037 wait complete. 2024-04-25T14:13:30.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.775+0000 7f0b6be00640 1 Processor -- start 2024-04-25T14:13:30.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.775+0000 7f0b6be00640 1 -- start start 2024-04-25T14:13:30.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.775+0000 7f0b6be00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b640b3a90 0x7f0b6415d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:30.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.775+0000 7f0b6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b640ce430 0x7f0b64159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:30.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.775+0000 7f0b6be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b640cf500 0x7f0b641596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:30.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.775+0000 7f0b6be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0b640ce9c0 con 0x7f0b640ce430 2024-04-25T14:13:30.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.775+0000 7f0b6be00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0b640ce840 con 0x7f0b640cf500 2024-04-25T14:13:30.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.775+0000 7f0b6be00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0b640d0a90 con 0x7f0b640b3a90 2024-04-25T14:13:30.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.775+0000 7f0b68a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b640ce430 0x7f0b64159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:30.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.775+0000 7f0b68a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b640ce430 0x7f0b64159170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55204/0 (socket says 172.21.15.86:55204) 2024-04-25T14:13:30.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.775+0000 7f0b68a00640 1 -- 172.21.15.86:0/3263688297 learned_addr learned my addr 172.21.15.86:0/3263688297 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:30.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.775+0000 7f0b69e00640 1 --2- 172.21.15.86:0/3263688297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b640cf500 0x7f0b641596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:30.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.775+0000 7f0b69400640 1 --2- 172.21.15.86:0/3263688297 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b640b3a90 0x7f0b6415d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:30.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.779+0000 7f0b68a00640 1 -- 172.21.15.86:0/3263688297 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b640b3a90 msgr2=0x7f0b6415d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:30.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.779+0000 7f0b68a00640 1 --2- 172.21.15.86:0/3263688297 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b640b3a90 0x7f0b6415d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:30.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.779+0000 7f0b68a00640 1 -- 172.21.15.86:0/3263688297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b640cf500 msgr2=0x7f0b641596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:30.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.779+0000 7f0b68a00640 1 --2- 172.21.15.86:0/3263688297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b640cf500 0x7f0b641596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:30.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.779+0000 7f0b68a00640 1 -- 172.21.15.86:0/3263688297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6414da00 con 0x7f0b640ce430 2024-04-25T14:13:30.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.779+0000 7f0b69400640 1 --2- 172.21.15.86:0/3263688297 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b640b3a90 0x7f0b6415d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:30.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.779+0000 7f0b69e00640 1 --2- 172.21.15.86:0/3263688297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b640cf500 0x7f0b641596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:30.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.779+0000 7f0b68a00640 1 --2- 172.21.15.86:0/3263688297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b640ce430 0x7f0b64159170 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f0b5000c390 tx=0x7f0b50001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:30.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.779+0000 7f0b61600640 1 -- 172.21.15.86:0/3263688297 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b50017070 con 0x7f0b640ce430 2024-04-25T14:13:30.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.779+0000 7f0b61600640 1 -- 172.21.15.86:0/3263688297 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b50005cf0 con 0x7f0b640ce430 2024-04-25T14:13:30.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.779+0000 7f0b61600640 1 -- 172.21.15.86:0/3263688297 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b500072f0 con 0x7f0b640ce430 2024-04-25T14:13:30.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.779+0000 7f0b6be00640 1 -- 172.21.15.86:0/3263688297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b6414dce0 con 0x7f0b640ce430 2024-04-25T14:13:30.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.779+0000 7f0b6be00640 1 -- 172.21.15.86:0/3263688297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b6414e2a0 con 0x7f0b640ce430 2024-04-25T14:13:30.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.779+0000 7f0b6be00640 1 -- 172.21.15.86:0/3263688297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b640a0630 con 0x7f0b640ce430 2024-04-25T14:13:30.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.779+0000 7f0b61600640 1 -- 172.21.15.86:0/3263688297 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7f0b5001a070 con 0x7f0b640ce430 2024-04-25T14:13:30.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.779+0000 7f0b61600640 1 --2- 172.21.15.86:0/3263688297 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f0b4406f550 0x7f0b44071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:30.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.779+0000 7f0b61600640 1 -- 172.21.15.86:0/3263688297 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5249+0+0 (secure 0 0 0) 0x7f0b5008e580 con 0x7f0b640ce430 2024-04-25T14:13:30.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.783+0000 7f0b69400640 1 --2- 172.21.15.86:0/3263688297 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f0b4406f550 0x7f0b44071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:30.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.783+0000 7f0b69400640 1 --2- 172.21.15.86:0/3263688297 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f0b4406f550 0x7f0b44071a60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0b580af000 tx=0x7f0b58008040 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:30.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.787+0000 7f0b61600640 1 -- 172.21.15.86:0/3263688297 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0b5005b500 con 0x7f0b640ce430 2024-04-25T14:13:30.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.943+0000 7f0b6be00640 1 -- 172.21.15.86:0/3263688297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b6415de00 con 0x7f0b640ce430 2024-04-25T14:13:30.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.943+0000 7f0b61600640 1 -- 172.21.15.86:0/3263688297 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) ==== 74+0+13174 (secure 0 0 0) 0x7f0b5005ae00 con 0x7f0b640ce430 2024-04-25T14:13:30.944 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:13:30.944 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":151,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:13:30.211687+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:13:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.947+0000 7f0b6be00640 1 -- 172.21.15.86:0/3263688297 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f0b4406f550 msgr2=0x7f0b44071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.947+0000 7f0b6be00640 1 --2- 172.21.15.86:0/3263688297 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f0b4406f550 0x7f0b44071a60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0b580af000 tx=0x7f0b58008040 comp rx=0 tx=0).stop 2024-04-25T14:13:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.947+0000 7f0b6be00640 1 -- 172.21.15.86:0/3263688297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b640ce430 msgr2=0x7f0b64159170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.947+0000 7f0b6be00640 1 --2- 172.21.15.86:0/3263688297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b640ce430 0x7f0b64159170 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f0b5000c390 tx=0x7f0b50001e80 comp rx=0 tx=0).stop 2024-04-25T14:13:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.947+0000 7f0b6be00640 1 -- 172.21.15.86:0/3263688297 shutdown_connections 2024-04-25T14:13:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.947+0000 7f0b6be00640 1 --2- 172.21.15.86:0/3263688297 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f0b4406f550 0x7f0b44071a60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.947+0000 7f0b6be00640 1 --2- 172.21.15.86:0/3263688297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b640cf500 0x7f0b641596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.947+0000 7f0b6be00640 1 --2- 172.21.15.86:0/3263688297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b640ce430 0x7f0b64159170 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.947+0000 7f0b6be00640 1 --2- 172.21.15.86:0/3263688297 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b640b3a90 0x7f0b6415d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:30.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.947+0000 7f0b6be00640 1 -- 172.21.15.86:0/3263688297 >> 172.21.15.86:0/3263688297 conn(0x7f0b6409ba30 msgr2=0x7f0b641544e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:30.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.947+0000 7f0b6be00640 1 -- 172.21.15.86:0/3263688297 shutdown_connections 2024-04-25T14:13:30.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:30.947+0000 7f0b6be00640 1 -- 172.21.15.86:0/3263688297 wait complete. 2024-04-25T14:13:30.966 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:13:30.967 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:13:31.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.115+0000 7f2a4d600640 1 Processor -- start 2024-04-25T14:13:31.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.115+0000 7f2a4d600640 1 -- start start 2024-04-25T14:13:31.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.119+0000 7f2a4d600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2a4815e860 0x7f2a480ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:31.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.119+0000 7f2a4d600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a480cf8f0 con 0x7f2a4815ec70 2024-04-25T14:13:31.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.119+0000 7f2a4d600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a4806c670 con 0x7f2a480cf3e0 2024-04-25T14:13:31.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.119+0000 7f2a4d600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2a4806cfa0 con 0x7f2a4815e860 2024-04-25T14:13:31.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.119+0000 7f2a46a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2a4815e860 0x7f2a480ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:31.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.119+0000 7f2a47400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2a4815ec70 0x7f2a4806d430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40190/0 (socket says 172.21.15.86:40190) 2024-04-25T14:13:31.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.119+0000 7f2a47400640 1 -- 172.21.15.86:0/3798635845 learned_addr learned my addr 172.21.15.86:0/3798635845 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:31.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.119+0000 7f2a46a00640 1 -- 172.21.15.86:0/3798635845 >> v1:172.21.15.155:6789/0 conn(0x7f2a480cf3e0 legacy=0x7f2a480d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:31.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.119+0000 7f2a46a00640 1 -- 172.21.15.86:0/3798635845 >> v1:172.21.15.86:6789/0 conn(0x7f2a4815ec70 legacy=0x7f2a4806d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:31.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.119+0000 7f2a46a00640 1 -- 172.21.15.86:0/3798635845 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a4806c950 con 0x7f2a4815e860 2024-04-25T14:13:31.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.119+0000 7f2a46000640 1 -- 172.21.15.86:0/3798635845 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 271580985 0 0) 0x7f2a480cf8f0 con 0x7f2a4815ec70 2024-04-25T14:13:31.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.123+0000 7f2a46a00640 1 --2- 172.21.15.86:0/3798635845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a4815e860 0x7f2a480ceea0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f2a380097c0 tx=0x7f2a3801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97eda01f03a28d58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:31.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.123+0000 7f2a46000640 1 -- 172.21.15.86:0/3798635845 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a380ab600 con 0x7f2a4815e860 2024-04-25T14:13:31.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.123+0000 7f2a46000640 1 -- 172.21.15.86:0/3798635845 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a380b0070 con 0x7f2a4815e860 2024-04-25T14:13:31.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.123+0000 7f2a46000640 1 -- 172.21.15.86:0/3798635845 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a380ab9c0 con 0x7f2a4815e860 2024-04-25T14:13:31.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.123+0000 7f2a4d600640 1 -- 172.21.15.86:0/3798635845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a4815e860 msgr2=0x7f2a480ceea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:31.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.123+0000 7f2a4d600640 1 --2- 172.21.15.86:0/3798635845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a4815e860 0x7f2a480ceea0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f2a380097c0 tx=0x7f2a3801c3e0 comp rx=0 tx=0).stop 2024-04-25T14:13:31.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.123+0000 7f2a4d600640 1 -- 172.21.15.86:0/3798635845 shutdown_connections 2024-04-25T14:13:31.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.123+0000 7f2a4d600640 1 --2- 172.21.15.86:0/3798635845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a4815e860 0x7f2a480ceea0 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:31.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.123+0000 7f2a4d600640 1 -- 172.21.15.86:0/3798635845 >> 172.21.15.86:0/3798635845 conn(0x7f2a480d2230 msgr2=0x7f2a480d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:31.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.123+0000 7f2a4d600640 1 -- 172.21.15.86:0/3798635845 shutdown_connections 2024-04-25T14:13:31.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.123+0000 7f2a4d600640 1 -- 172.21.15.86:0/3798635845 wait complete. 2024-04-25T14:13:31.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.127+0000 7f2a4d600640 1 Processor -- start 2024-04-25T14:13:31.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.127+0000 7f2a4d600640 1 -- start start 2024-04-25T14:13:31.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.127+0000 7f2a4d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a480cf3e0 0x7f2a480b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:31.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.127+0000 7f2a4d600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a4815e860 0x7f2a4815c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.127+0000 7f2a4d600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2a4815ec70 0x7f2a4815c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.127+0000 7f2a4d600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4816c0c0 con 0x7f2a480cf3e0 2024-04-25T14:13:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.127+0000 7f2a4d600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4816bf40 con 0x7f2a4815e860 2024-04-25T14:13:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.127+0000 7f2a4d600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2a4816c240 con 0x7f2a4815ec70 2024-04-25T14:13:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.127+0000 7f2a46a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a480cf3e0 0x7f2a480b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.127+0000 7f2a46a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a480cf3e0 0x7f2a480b0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55220/0 (socket says 172.21.15.86:55220) 2024-04-25T14:13:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.127+0000 7f2a46a00640 1 -- 172.21.15.86:0/14547218 learned_addr learned my addr 172.21.15.86:0/14547218 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.127+0000 7f2a3e000640 1 --2- 172.21.15.86:0/14547218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a4815e860 0x7f2a4815c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.127+0000 7f2a47400640 1 --2- 172.21.15.86:0/14547218 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2a4815ec70 0x7f2a4815c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.127+0000 7f2a46a00640 1 -- 172.21.15.86:0/14547218 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2a4815ec70 msgr2=0x7f2a4815c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.127+0000 7f2a46a00640 1 --2- 172.21.15.86:0/14547218 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2a4815ec70 0x7f2a4815c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.127+0000 7f2a46a00640 1 -- 172.21.15.86:0/14547218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a4815e860 msgr2=0x7f2a4815c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.127+0000 7f2a46a00640 1 --2- 172.21.15.86:0/14547218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a4815e860 0x7f2a4815c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:31.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.127+0000 7f2a46a00640 1 -- 172.21.15.86:0/14547218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a481f7880 con 0x7f2a480cf3e0 2024-04-25T14:13:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.127+0000 7f2a46a00640 1 --2- 172.21.15.86:0/14547218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a480cf3e0 0x7f2a480b0040 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f2a38018ff0 tx=0x7f2a3801ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.127+0000 7f2a3f400640 1 -- 172.21.15.86:0/14547218 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a380bf610 con 0x7f2a480cf3e0 2024-04-25T14:13:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.127+0000 7f2a3f400640 1 -- 172.21.15.86:0/14547218 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a380bf7d0 con 0x7f2a480cf3e0 2024-04-25T14:13:31.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.127+0000 7f2a4d600640 1 -- 172.21.15.86:0/14547218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a4815cff0 con 0x7f2a480cf3e0 2024-04-25T14:13:31.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.127+0000 7f2a3f400640 1 -- 172.21.15.86:0/14547218 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a380bfbc0 con 0x7f2a480cf3e0 2024-04-25T14:13:31.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.127+0000 7f2a4d600640 1 -- 172.21.15.86:0/14547218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a4815d5b0 con 0x7f2a480cf3e0 2024-04-25T14:13:31.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.131+0000 7f2a4d600640 1 -- 172.21.15.86:0/14547218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a48074cf0 con 0x7f2a480cf3e0 2024-04-25T14:13:31.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.131+0000 7f2a3f400640 1 -- 172.21.15.86:0/14547218 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7f2a380bd020 con 0x7f2a480cf3e0 2024-04-25T14:13:31.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.131+0000 7f2a3f400640 1 --2- 172.21.15.86:0/14547218 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f2a2406f550 0x7f2a24071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:31.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.131+0000 7f2a3f400640 1 -- 172.21.15.86:0/14547218 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5249+0+0 (secure 0 0 0) 0x7f2a38129aa0 con 0x7f2a480cf3e0 2024-04-25T14:13:31.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.131+0000 7f2a3e000640 1 --2- 172.21.15.86:0/14547218 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f2a2406f550 0x7f2a24071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.131+0000 7f2a3e000640 1 --2- 172.21.15.86:0/14547218 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f2a2406f550 0x7f2a24071a60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2a481625d0 tx=0x7f2a30009040 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:31.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.135+0000 7f2a3f400640 1 -- 172.21.15.86:0/14547218 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2a3812be20 con 0x7f2a480cf3e0 2024-04-25T14:13:31.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.299+0000 7f2a4d600640 1 -- 172.21.15.86:0/14547218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2a480cfd90 con 0x7f2a480cf3e0 2024-04-25T14:13:31.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.299+0000 7f2a3f400640 1 -- 172.21.15.86:0/14547218 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v151) ==== 93+0+10 (secure 0 0 0) 0x7f2a38016090 con 0x7f2a480cf3e0 2024-04-25T14:13:31.296 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:13:31.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.299+0000 7f2a4d600640 1 -- 172.21.15.86:0/14547218 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f2a2406f550 msgr2=0x7f2a24071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:31.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.299+0000 7f2a4d600640 1 --2- 172.21.15.86:0/14547218 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f2a2406f550 0x7f2a24071a60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2a481625d0 tx=0x7f2a30009040 comp rx=0 tx=0).stop 2024-04-25T14:13:31.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.299+0000 7f2a4d600640 1 -- 172.21.15.86:0/14547218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a480cf3e0 msgr2=0x7f2a480b0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:31.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.299+0000 7f2a4d600640 1 --2- 172.21.15.86:0/14547218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a480cf3e0 0x7f2a480b0040 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f2a38018ff0 tx=0x7f2a3801ac20 comp rx=0 tx=0).stop 2024-04-25T14:13:31.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.299+0000 7f2a4d600640 1 -- 172.21.15.86:0/14547218 shutdown_connections 2024-04-25T14:13:31.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.299+0000 7f2a4d600640 1 --2- 172.21.15.86:0/14547218 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f2a2406f550 0x7f2a24071a60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:31.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.299+0000 7f2a4d600640 1 --2- 172.21.15.86:0/14547218 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2a4815ec70 0x7f2a4815c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:31.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.299+0000 7f2a4d600640 1 --2- 172.21.15.86:0/14547218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a4815e860 0x7f2a4815c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:31.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.299+0000 7f2a4d600640 1 --2- 172.21.15.86:0/14547218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a480cf3e0 0x7f2a480b0040 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:31.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.299+0000 7f2a4d600640 1 -- 172.21.15.86:0/14547218 >> 172.21.15.86:0/14547218 conn(0x7f2a480d2230 msgr2=0x7f2a480d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:31.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.299+0000 7f2a4d600640 1 -- 172.21.15.86:0/14547218 shutdown_connections 2024-04-25T14:13:31.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.299+0000 7f2a4d600640 1 -- 172.21.15.86:0/14547218 wait complete. 2024-04-25T14:13:31.318 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T14:13:31.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.467+0000 7f240d800640 1 Processor -- start 2024-04-25T14:13:31.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.467+0000 7f240d800640 1 -- start start 2024-04-25T14:13:31.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.467+0000 7f240d800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2408070330 0x7f24080d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:31.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.467+0000 7f240d800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f24080ce790 0x7f240806f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:31.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.467+0000 7f240d800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24080d2030 con 0x7f24080d1c20 2024-04-25T14:13:31.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.467+0000 7f240d800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f240815e5c0 con 0x7f2408070330 2024-04-25T14:13:31.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.467+0000 7f240d800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f240815e8a0 con 0x7f24080ce790 2024-04-25T14:13:31.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.467+0000 7f2406a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2408070330 0x7f24080d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:31.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.467+0000 7f2406a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2408070330 0x7f24080d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55226/0 (socket says 172.21.15.86:55226) 2024-04-25T14:13:31.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.467+0000 7f2406a00640 1 -- 172.21.15.86:0/1268061532 learned_addr learned my addr 172.21.15.86:0/1268061532 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:31.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.467+0000 7f2407400640 1 --2- 172.21.15.86:0/1268061532 >> v2:172.21.15.155:3300/0 conn(0x7f24080ce790 0x7f240806f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:31.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.467+0000 7f2406a00640 1 -- 172.21.15.86:0/1268061532 >> v2:172.21.15.155:3300/0 conn(0x7f24080ce790 msgr2=0x7f240806f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:31.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.467+0000 7f2406a00640 1 --2- 172.21.15.86:0/1268061532 >> v2:172.21.15.155:3300/0 conn(0x7f24080ce790 0x7f240806f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:31.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.467+0000 7f2406a00640 1 -- 172.21.15.86:0/1268061532 >> v1:172.21.15.155:6789/0 conn(0x7f24080d1c20 legacy=0x7f240806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:31.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.467+0000 7f2406a00640 1 -- 172.21.15.86:0/1268061532 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240815eb80 con 0x7f2408070330 2024-04-25T14:13:31.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.471+0000 7f2406a00640 1 --2- 172.21.15.86:0/1268061532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2408070330 0x7f24080d16e0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f23f8009870 tx=0x7f23f801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371fb462287947d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:31.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.471+0000 7f2406000640 1 -- 172.21.15.86:0/1268061532 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f80b0070 con 0x7f2408070330 2024-04-25T14:13:31.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.471+0000 7f2406000640 1 -- 172.21.15.86:0/1268061532 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23f80ab600 con 0x7f2408070330 2024-04-25T14:13:31.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.471+0000 7f2406000640 1 -- 172.21.15.86:0/1268061532 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f80ab9d0 con 0x7f2408070330 2024-04-25T14:13:31.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.471+0000 7f240d800640 1 -- 172.21.15.86:0/1268061532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2408070330 msgr2=0x7f24080d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:31.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.471+0000 7f240d800640 1 --2- 172.21.15.86:0/1268061532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2408070330 0x7f24080d16e0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f23f8009870 tx=0x7f23f801c490 comp rx=0 tx=0).stop 2024-04-25T14:13:31.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.471+0000 7f240d800640 1 -- 172.21.15.86:0/1268061532 shutdown_connections 2024-04-25T14:13:31.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.471+0000 7f240d800640 1 --2- 172.21.15.86:0/1268061532 >> v2:172.21.15.155:3300/0 conn(0x7f24080ce790 0x7f240806f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:31.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.471+0000 7f240d800640 1 --2- 172.21.15.86:0/1268061532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2408070330 0x7f24080d16e0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:31.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.471+0000 7f240d800640 1 -- 172.21.15.86:0/1268061532 >> 172.21.15.86:0/1268061532 conn(0x7f24080b4650 msgr2=0x7f24080b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:31.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.471+0000 7f240d800640 1 -- 172.21.15.86:0/1268061532 shutdown_connections 2024-04-25T14:13:31.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.471+0000 7f240d800640 1 -- 172.21.15.86:0/1268061532 wait complete. 2024-04-25T14:13:31.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.475+0000 7f240d800640 1 Processor -- start 2024-04-25T14:13:31.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.475+0000 7f240d800640 1 -- start start 2024-04-25T14:13:31.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.475+0000 7f240d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2408070330 0x7f24081f9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:31.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.475+0000 7f240d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24080ce790 0x7f240815a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:31.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.475+0000 7f240d800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f24080d1c20 0x7f240815a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:31.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.475+0000 7f240d800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24080708c0 con 0x7f24080ce790 2024-04-25T14:13:31.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.475+0000 7f240d800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2408070740 con 0x7f2408070330 2024-04-25T14:13:31.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.475+0000 7f240d800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f240816afc0 con 0x7f24080d1c20 2024-04-25T14:13:31.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.475+0000 7f23fe000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24080ce790 0x7f240815a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:31.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.475+0000 7f23fe000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24080ce790 0x7f240815a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55240/0 (socket says 172.21.15.86:55240) 2024-04-25T14:13:31.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.475+0000 7f23fe000640 1 -- 172.21.15.86:0/443989120 learned_addr learned my addr 172.21.15.86:0/443989120 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:31.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.475+0000 7f2406a00640 1 --2- 172.21.15.86:0/443989120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2408070330 0x7f24081f9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:31.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.475+0000 7f2407400640 1 --2- 172.21.15.86:0/443989120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f24080d1c20 0x7f240815a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:31.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.475+0000 7f23fe000640 1 -- 172.21.15.86:0/443989120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f24080d1c20 msgr2=0x7f240815a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:31.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.475+0000 7f23fe000640 1 --2- 172.21.15.86:0/443989120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f24080d1c20 0x7f240815a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:31.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.475+0000 7f23fe000640 1 -- 172.21.15.86:0/443989120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2408070330 msgr2=0x7f24081f9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:31.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.475+0000 7f23fe000640 1 --2- 172.21.15.86:0/443989120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2408070330 0x7f24081f9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:31.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.475+0000 7f23fe000640 1 -- 172.21.15.86:0/443989120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2408072ad0 con 0x7f24080ce790 2024-04-25T14:13:31.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.475+0000 7f2406a00640 1 --2- 172.21.15.86:0/443989120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2408070330 0x7f24081f9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:31.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.475+0000 7f23fe000640 1 --2- 172.21.15.86:0/443989120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24080ce790 0x7f240815a2b0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f23f000c310 tx=0x7f23f000c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:31.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.475+0000 7f23ff400640 1 -- 172.21.15.86:0/443989120 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f00052d0 con 0x7f24080ce790 2024-04-25T14:13:31.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.475+0000 7f23ff400640 1 -- 172.21.15.86:0/443989120 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23f0005470 con 0x7f24080ce790 2024-04-25T14:13:31.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.475+0000 7f240d800640 1 -- 172.21.15.86:0/443989120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2408072db0 con 0x7f24080ce790 2024-04-25T14:13:31.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.475+0000 7f23ff400640 1 -- 172.21.15.86:0/443989120 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f0007260 con 0x7f24080ce790 2024-04-25T14:13:31.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.475+0000 7f240d800640 1 -- 172.21.15.86:0/443989120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2408073370 con 0x7f24080ce790 2024-04-25T14:13:31.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.479+0000 7f240d800640 1 -- 172.21.15.86:0/443989120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2408069d00 con 0x7f24080ce790 2024-04-25T14:13:31.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.479+0000 7f23ff400640 1 -- 172.21.15.86:0/443989120 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7f23f0007400 con 0x7f24080ce790 2024-04-25T14:13:31.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.479+0000 7f23ff400640 1 --2- 172.21.15.86:0/443989120 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f23e406f520 0x7f23e4071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:31.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.479+0000 7f23ff400640 1 -- 172.21.15.86:0/443989120 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5249+0+0 (secure 0 0 0) 0x7f23f008e770 con 0x7f24080ce790 2024-04-25T14:13:31.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.479+0000 7f2406a00640 1 --2- 172.21.15.86:0/443989120 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f23e406f520 0x7f23e4071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:31.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.479+0000 7f2406a00640 1 --2- 172.21.15.86:0/443989120 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f23e406f520 0x7f23e4071a30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f23f80ad000 tx=0x7f23f80a6000 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:31.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.483+0000 7f23ff400640 1 -- 172.21.15.86:0/443989120 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f24080d2030 con 0x7f24080ce790 2024-04-25T14:13:31.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.643+0000 7f240d800640 1 -- 172.21.15.86:0/443989120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2408076bf0 con 0x7f24080ce790 2024-04-25T14:13:31.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.643+0000 7f23ff400640 1 -- 172.21.15.86:0/443989120 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v151) ==== 84+0+65 (secure 0 0 0) 0x7f23f005b670 con 0x7f24080ce790 2024-04-25T14:13:31.641 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:13:31.641 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-04-25T14:13:31.641 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-25T14:13:31.642 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-04-25T14:13:31.642 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-04-25T14:13:31.642 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-25T14:13:31.642 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-04-25T14:13:31.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.643+0000 7f240d800640 1 -- 172.21.15.86:0/443989120 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f23e406f520 msgr2=0x7f23e4071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:31.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.643+0000 7f240d800640 1 --2- 172.21.15.86:0/443989120 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f23e406f520 0x7f23e4071a30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f23f80ad000 tx=0x7f23f80a6000 comp rx=0 tx=0).stop 2024-04-25T14:13:31.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.643+0000 7f240d800640 1 -- 172.21.15.86:0/443989120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24080ce790 msgr2=0x7f240815a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:31.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.643+0000 7f240d800640 1 --2- 172.21.15.86:0/443989120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24080ce790 0x7f240815a2b0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f23f000c310 tx=0x7f23f000c7d0 comp rx=0 tx=0).stop 2024-04-25T14:13:31.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.643+0000 7f240d800640 1 -- 172.21.15.86:0/443989120 shutdown_connections 2024-04-25T14:13:31.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.643+0000 7f240d800640 1 --2- 172.21.15.86:0/443989120 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f23e406f520 0x7f23e4071a30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:31.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.643+0000 7f240d800640 1 --2- 172.21.15.86:0/443989120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f24080d1c20 0x7f240815a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:31.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.643+0000 7f240d800640 1 --2- 172.21.15.86:0/443989120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24080ce790 0x7f240815a2b0 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:31.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.643+0000 7f240d800640 1 --2- 172.21.15.86:0/443989120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2408070330 0x7f24081f9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:31.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.643+0000 7f240d800640 1 -- 172.21.15.86:0/443989120 >> 172.21.15.86:0/443989120 conn(0x7f24080b4650 msgr2=0x7f240815bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:31.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.643+0000 7f240d800640 1 -- 172.21.15.86:0/443989120 shutdown_connections 2024-04-25T14:13:31.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.643+0000 7f240d800640 1 -- 172.21.15.86:0/443989120 wait complete. 2024-04-25T14:13:31.662 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T14:13:31.662 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T14:13:31.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.815+0000 7f1b4ce00640 1 Processor -- start 2024-04-25T14:13:31.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.815+0000 7f1b4ce00640 1 -- start start 2024-04-25T14:13:31.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.815+0000 7f1b4ce00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b4806ec00 0x7f1b4806f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:31.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.815+0000 7f1b4ce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1b4806fa20 0x7f1b480ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:31.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.815+0000 7f1b4ce00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b4806d120 con 0x7f1b4806fe30 2024-04-25T14:13:31.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.815+0000 7f1b4ce00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1b481600c0 con 0x7f1b4806fa20 2024-04-25T14:13:31.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.815+0000 7f1b4ce00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1b4806d3c0 con 0x7f1b4806ec00 2024-04-25T14:13:31.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.815+0000 7f1b3d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1b4806fa20 0x7f1b480ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:31.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.815+0000 7f1b46800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1b4806fe30 0x7f1b480ae7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40198/0 (socket says 172.21.15.86:40198) 2024-04-25T14:13:31.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.815+0000 7f1b46800640 1 -- 172.21.15.86:0/1279308382 learned_addr learned my addr 172.21.15.86:0/1279308382 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:31.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.815+0000 7f1b45e00640 1 --2- 172.21.15.86:0/1279308382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b4806ec00 0x7f1b4806f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:31.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.815+0000 7f1b3d400640 1 -- 172.21.15.86:0/1279308382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b4806ec00 msgr2=0x7f1b4806f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:31.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.815+0000 7f1b3d400640 1 --2- 172.21.15.86:0/1279308382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b4806ec00 0x7f1b4806f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:31.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.815+0000 7f1b3d400640 1 -- 172.21.15.86:0/1279308382 >> v1:172.21.15.86:6789/0 conn(0x7f1b4806fe30 legacy=0x7f1b480ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:31.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.815+0000 7f1b3d400640 1 -- 172.21.15.86:0/1279308382 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4806d6a0 con 0x7f1b4806fa20 2024-04-25T14:13:31.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.815+0000 7f1b45e00640 1 --2- 172.21.15.86:0/1279308382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b4806ec00 0x7f1b4806f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:31.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.815+0000 7f1b45400640 1 -- 172.21.15.86:0/1279308382 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3725502819 0 0) 0x7f1b4806d120 con 0x7f1b4806fe30 2024-04-25T14:13:31.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.819+0000 7f1b3d400640 1 --2- 172.21.15.86:0/1279308382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b4806fa20 0x7f1b480ae2a0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f1b48160240 tx=0x7f1b3001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73c7254c5fe0940 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:31.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.819+0000 7f1b45400640 1 -- 172.21.15.86:0/1279308382 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b300ab430 con 0x7f1b4806fa20 2024-04-25T14:13:31.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.819+0000 7f1b45400640 1 -- 172.21.15.86:0/1279308382 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b300b0070 con 0x7f1b4806fa20 2024-04-25T14:13:31.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.819+0000 7f1b45400640 1 -- 172.21.15.86:0/1279308382 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b300ab7f0 con 0x7f1b4806fa20 2024-04-25T14:13:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.819+0000 7f1b4ce00640 1 -- 172.21.15.86:0/1279308382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b4806fa20 msgr2=0x7f1b480ae2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.819+0000 7f1b4ce00640 1 --2- 172.21.15.86:0/1279308382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b4806fa20 0x7f1b480ae2a0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f1b48160240 tx=0x7f1b3001c470 comp rx=0 tx=0).stop 2024-04-25T14:13:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.819+0000 7f1b4ce00640 1 -- 172.21.15.86:0/1279308382 shutdown_connections 2024-04-25T14:13:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.819+0000 7f1b4ce00640 1 --2- 172.21.15.86:0/1279308382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b4806fa20 0x7f1b480ae2a0 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.819+0000 7f1b4ce00640 1 --2- 172.21.15.86:0/1279308382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b4806ec00 0x7f1b4806f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.819+0000 7f1b4ce00640 1 -- 172.21.15.86:0/1279308382 >> 172.21.15.86:0/1279308382 conn(0x7f1b480b4a40 msgr2=0x7f1b480b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.819+0000 7f1b4ce00640 1 -- 172.21.15.86:0/1279308382 shutdown_connections 2024-04-25T14:13:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.819+0000 7f1b4ce00640 1 -- 172.21.15.86:0/1279308382 wait complete. 2024-04-25T14:13:31.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.823+0000 7f1b4ce00640 1 Processor -- start 2024-04-25T14:13:31.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.823+0000 7f1b4ce00640 1 -- start start 2024-04-25T14:13:31.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.823+0000 7f1b4ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b4806ec00 0x7f1b4815f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:31.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.823+0000 7f1b4ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b4806fa20 0x7f1b48149ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:31.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.823+0000 7f1b4ce00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b4806fe30 0x7f1b48155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:31.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.823+0000 7f1b4ce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4806dea0 con 0x7f1b4806fa20 2024-04-25T14:13:31.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.823+0000 7f1b4ce00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1b480af300 con 0x7f1b4806ec00 2024-04-25T14:13:31.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.823+0000 7f1b4ce00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1b48150850 con 0x7f1b4806fe30 2024-04-25T14:13:31.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.823+0000 7f1b3d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b4806fa20 0x7f1b48149ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:31.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.823+0000 7f1b3d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b4806fa20 0x7f1b48149ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55266/0 (socket says 172.21.15.86:55266) 2024-04-25T14:13:31.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.823+0000 7f1b3d400640 1 -- 172.21.15.86:0/1886949633 learned_addr learned my addr 172.21.15.86:0/1886949633 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:31.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.827+0000 7f1b45e00640 1 --2- 172.21.15.86:0/1886949633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b4806ec00 0x7f1b4815f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:31.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.827+0000 7f1b46800640 1 --2- 172.21.15.86:0/1886949633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b4806fe30 0x7f1b48155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:31.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.827+0000 7f1b3d400640 1 -- 172.21.15.86:0/1886949633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b4806fe30 msgr2=0x7f1b48155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:31.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.827+0000 7f1b3d400640 1 --2- 172.21.15.86:0/1886949633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b4806fe30 0x7f1b48155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:31.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.827+0000 7f1b3d400640 1 -- 172.21.15.86:0/1886949633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b4806ec00 msgr2=0x7f1b4815f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:31.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.827+0000 7f1b3d400640 1 --2- 172.21.15.86:0/1886949633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b4806ec00 0x7f1b4815f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:31.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.827+0000 7f1b3d400640 1 -- 172.21.15.86:0/1886949633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4815a520 con 0x7f1b4806fa20 2024-04-25T14:13:31.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.827+0000 7f1b46800640 1 --2- 172.21.15.86:0/1886949633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b4806fe30 0x7f1b48155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:31.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.827+0000 7f1b3d400640 1 --2- 172.21.15.86:0/1886949633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b4806fa20 0x7f1b48149ab0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f1b300ad040 tx=0x7f1b30019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:31.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.827+0000 7f1b3ea00640 1 -- 172.21.15.86:0/1886949633 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b300b43f0 con 0x7f1b4806fa20 2024-04-25T14:13:31.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.827+0000 7f1b3ea00640 1 -- 172.21.15.86:0/1886949633 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b300bc070 con 0x7f1b4806fa20 2024-04-25T14:13:31.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.827+0000 7f1b4ce00640 1 -- 172.21.15.86:0/1886949633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b4815a800 con 0x7f1b4806fa20 2024-04-25T14:13:31.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.827+0000 7f1b3ea00640 1 -- 172.21.15.86:0/1886949633 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b300b47f0 con 0x7f1b4806fa20 2024-04-25T14:13:31.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.827+0000 7f1b4ce00640 1 -- 172.21.15.86:0/1886949633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b4815adc0 con 0x7f1b4806fa20 2024-04-25T14:13:31.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.827+0000 7f1b4ce00640 1 -- 172.21.15.86:0/1886949633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b48050750 con 0x7f1b4806fa20 2024-04-25T14:13:31.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.827+0000 7f1b3ea00640 1 -- 172.21.15.86:0/1886949633 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7f1b300b4990 con 0x7f1b4806fa20 2024-04-25T14:13:31.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.827+0000 7f1b3ea00640 1 --2- 172.21.15.86:0/1886949633 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f1b2406f520 0x7f1b24071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:31.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.831+0000 7f1b3ea00640 1 -- 172.21.15.86:0/1886949633 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5249+0+0 (secure 0 0 0) 0x7f1b301290e0 con 0x7f1b4806fa20 2024-04-25T14:13:31.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.831+0000 7f1b45e00640 1 --2- 172.21.15.86:0/1886949633 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f1b2406f520 0x7f1b24071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:31.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.831+0000 7f1b45e00640 1 --2- 172.21.15.86:0/1886949633 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f1b2406f520 0x7f1b24071a30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800ba10 tx=0x7f1b38008040 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:31.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.835+0000 7f1b3ea00640 1 -- 172.21.15.86:0/1886949633 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1b30016090 con 0x7f1b4806fa20 2024-04-25T14:13:31.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:31.991+0000 7f1b4ce00640 1 -- 172.21.15.86:0/1886949633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f1b4815c5d0 con 0x7f1b4806fa20 2024-04-25T14:13:32.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.199+0000 7f1b3ea00640 1 -- 172.21.15.86:0/1886949633 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v152) ==== 138+0+0 (secure 0 0 0) 0x7f1b300f5ed0 con 0x7f1b4806fa20 2024-04-25T14:13:32.198 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-04-25T14:13:32.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.199+0000 7f1b4ce00640 1 -- 172.21.15.86:0/1886949633 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f1b2406f520 msgr2=0x7f1b24071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:32.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.199+0000 7f1b4ce00640 1 --2- 172.21.15.86:0/1886949633 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f1b2406f520 0x7f1b24071a30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800ba10 tx=0x7f1b38008040 comp rx=0 tx=0).stop 2024-04-25T14:13:32.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.199+0000 7f1b4ce00640 1 -- 172.21.15.86:0/1886949633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b4806fa20 msgr2=0x7f1b48149ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:32.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.199+0000 7f1b4ce00640 1 --2- 172.21.15.86:0/1886949633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b4806fa20 0x7f1b48149ab0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f1b300ad040 tx=0x7f1b30019210 comp rx=0 tx=0).stop 2024-04-25T14:13:32.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.203+0000 7f1b4ce00640 1 -- 172.21.15.86:0/1886949633 shutdown_connections 2024-04-25T14:13:32.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.203+0000 7f1b4ce00640 1 --2- 172.21.15.86:0/1886949633 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f1b2406f520 0x7f1b24071a30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:32.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.203+0000 7f1b4ce00640 1 --2- 172.21.15.86:0/1886949633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b4806fe30 0x7f1b48155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:32.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.203+0000 7f1b4ce00640 1 --2- 172.21.15.86:0/1886949633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b4806fa20 0x7f1b48149ab0 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:32.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.203+0000 7f1b4ce00640 1 --2- 172.21.15.86:0/1886949633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b4806ec00 0x7f1b4815f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:32.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.203+0000 7f1b4ce00640 1 -- 172.21.15.86:0/1886949633 >> 172.21.15.86:0/1886949633 conn(0x7f1b480b4a40 msgr2=0x7f1b480b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:32.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.203+0000 7f1b4ce00640 1 -- 172.21.15.86:0/1886949633 shutdown_connections 2024-04-25T14:13:32.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.203+0000 7f1b4ce00640 1 -- 172.21.15.86:0/1886949633 wait complete. 2024-04-25T14:13:32.227 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T14:13:32.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.383+0000 7f632ae00640 1 Processor -- start 2024-04-25T14:13:32.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.383+0000 7f632ae00640 1 -- start start 2024-04-25T14:13:32.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.383+0000 7f632ae00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6324070330 0x7f63240d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:32.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.383+0000 7f632ae00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63240d2030 con 0x7f63240ce790 2024-04-25T14:13:32.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.383+0000 7f632ae00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f632415e5c0 con 0x7f63240d1c20 2024-04-25T14:13:32.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.383+0000 7f632ae00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f632415e8a0 con 0x7f6324070330 2024-04-25T14:13:32.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.383+0000 7f6323e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6324070330 0x7f63240d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:32.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.383+0000 7f6328e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f63240ce790 0x7f632406f440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40214/0 (socket says 172.21.15.86:40214) 2024-04-25T14:13:32.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.383+0000 7f6328e00640 1 -- 172.21.15.86:0/1349422390 learned_addr learned my addr 172.21.15.86:0/1349422390 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:32.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.383+0000 7f6323e00640 1 -- 172.21.15.86:0/1349422390 >> v1:172.21.15.155:6789/0 conn(0x7f63240d1c20 legacy=0x7f632406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:32.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.383+0000 7f6323e00640 1 -- 172.21.15.86:0/1349422390 >> v1:172.21.15.86:6789/0 conn(0x7f63240ce790 legacy=0x7f632406f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:32.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.383+0000 7f6323400640 1 -- 172.21.15.86:0/1349422390 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1526212539 0 0) 0x7f631c0014a0 con 0x7f63240ce790 2024-04-25T14:13:32.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.383+0000 7f6323e00640 1 -- 172.21.15.86:0/1349422390 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f632415eb80 con 0x7f6324070330 2024-04-25T14:13:32.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.387+0000 7f6323e00640 1 --2- 172.21.15.86:0/1349422390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6324070330 0x7f63240d16e0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f63140097f0 tx=0x7f631401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4b64dc766abd36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:32.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.387+0000 7f6323400640 1 -- 172.21.15.86:0/1349422390 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63140b0070 con 0x7f6324070330 2024-04-25T14:13:32.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.387+0000 7f6323400640 1 -- 172.21.15.86:0/1349422390 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63140ab600 con 0x7f6324070330 2024-04-25T14:13:32.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.387+0000 7f6323400640 1 -- 172.21.15.86:0/1349422390 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63140ab9d0 con 0x7f6324070330 2024-04-25T14:13:32.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.387+0000 7f632ae00640 1 -- 172.21.15.86:0/1349422390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6324070330 msgr2=0x7f63240d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:32.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.387+0000 7f632ae00640 1 --2- 172.21.15.86:0/1349422390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6324070330 0x7f63240d16e0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f63140097f0 tx=0x7f631401c410 comp rx=0 tx=0).stop 2024-04-25T14:13:32.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.387+0000 7f632ae00640 1 -- 172.21.15.86:0/1349422390 shutdown_connections 2024-04-25T14:13:32.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.387+0000 7f632ae00640 1 --2- 172.21.15.86:0/1349422390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6324070330 0x7f63240d16e0 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:32.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.387+0000 7f632ae00640 1 -- 172.21.15.86:0/1349422390 >> 172.21.15.86:0/1349422390 conn(0x7f63240b4650 msgr2=0x7f63240b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:32.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.387+0000 7f632ae00640 1 -- 172.21.15.86:0/1349422390 shutdown_connections 2024-04-25T14:13:32.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.387+0000 7f632ae00640 1 -- 172.21.15.86:0/1349422390 wait complete. 2024-04-25T14:13:32.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.391+0000 7f632ae00640 1 Processor -- start 2024-04-25T14:13:32.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.391+0000 7f632ae00640 1 -- start start 2024-04-25T14:13:32.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.391+0000 7f632ae00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6324070330 0x7f6324090440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:32.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.391+0000 7f632ae00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63240ce790 0x7f63240c4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:32.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.391+0000 7f632ae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63240d1c20 0x7f63241580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:32.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.391+0000 7f632ae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f63240708c0 con 0x7f63240d1c20 2024-04-25T14:13:32.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.391+0000 7f632ae00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6324070740 con 0x7f6324070330 2024-04-25T14:13:32.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.391+0000 7f632ae00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f632406fb50 con 0x7f63240ce790 2024-04-25T14:13:32.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.391+0000 7f6328e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63240d1c20 0x7f63241580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:32.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.391+0000 7f6328e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63240d1c20 0x7f63241580d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55282/0 (socket says 172.21.15.86:55282) 2024-04-25T14:13:32.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.391+0000 7f6328e00640 1 -- 172.21.15.86:0/4284477211 learned_addr learned my addr 172.21.15.86:0/4284477211 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:32.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.391+0000 7f6323e00640 1 --2- 172.21.15.86:0/4284477211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6324070330 0x7f6324090440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:32.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.391+0000 7f631b400640 1 --2- 172.21.15.86:0/4284477211 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63240ce790 0x7f63240c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:32.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.391+0000 7f6328e00640 1 -- 172.21.15.86:0/4284477211 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63240ce790 msgr2=0x7f63240c4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:32.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.391+0000 7f6328e00640 1 --2- 172.21.15.86:0/4284477211 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63240ce790 0x7f63240c4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:32.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.391+0000 7f6328e00640 1 -- 172.21.15.86:0/4284477211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6324070330 msgr2=0x7f6324090440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:32.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.391+0000 7f6328e00640 1 --2- 172.21.15.86:0/4284477211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6324070330 0x7f6324090440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:32.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.395+0000 7f6328e00640 1 -- 172.21.15.86:0/4284477211 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6324159fc0 con 0x7f63240d1c20 2024-04-25T14:13:32.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.395+0000 7f631b400640 1 --2- 172.21.15.86:0/4284477211 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63240ce790 0x7f63240c4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:32.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.395+0000 7f6328e00640 1 --2- 172.21.15.86:0/4284477211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63240d1c20 0x7f63241580d0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f631c00c730 tx=0x7f631c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:32.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.395+0000 7f6320c00640 1 -- 172.21.15.86:0/4284477211 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c018070 con 0x7f63240d1c20 2024-04-25T14:13:32.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.395+0000 7f6320c00640 1 -- 172.21.15.86:0/4284477211 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f631c00ce40 con 0x7f63240d1c20 2024-04-25T14:13:32.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.395+0000 7f6320c00640 1 -- 172.21.15.86:0/4284477211 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c00a880 con 0x7f63240d1c20 2024-04-25T14:13:32.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.395+0000 7f632ae00640 1 -- 172.21.15.86:0/4284477211 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f632415a2a0 con 0x7f63240d1c20 2024-04-25T14:13:32.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.395+0000 7f632ae00640 1 -- 172.21.15.86:0/4284477211 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f632415a860 con 0x7f63240d1c20 2024-04-25T14:13:32.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.395+0000 7f632ae00640 1 -- 172.21.15.86:0/4284477211 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6324069d00 con 0x7f63240d1c20 2024-04-25T14:13:32.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.395+0000 7f6320c00640 1 -- 172.21.15.86:0/4284477211 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7f631c01b070 con 0x7f63240d1c20 2024-04-25T14:13:32.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.399+0000 7f6320c00640 1 --2- 172.21.15.86:0/4284477211 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f630006f550 0x7f6300071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:32.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.399+0000 7f6320c00640 1 -- 172.21.15.86:0/4284477211 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5619+0+0 (secure 0 0 0) 0x7f631c08ec10 con 0x7f63240d1c20 2024-04-25T14:13:32.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.399+0000 7f6323e00640 1 --2- 172.21.15.86:0/4284477211 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f630006f550 0x7f6300071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:32.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.399+0000 7f6323e00640 1 --2- 172.21.15.86:0/4284477211 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f630006f550 0x7f6300071a60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f63140ad000 tx=0x7f63140a6000 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:32.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.399+0000 7f6320c00640 1 -- 172.21.15.86:0/4284477211 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f63240d2030 con 0x7f63240d1c20 2024-04-25T14:13:32.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:32.559+0000 7f632ae00640 1 -- 172.21.15.86:0/4284477211 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f6324076bf0 con 0x7f63240d1c20 2024-04-25T14:13:33.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.207+0000 7f6320c00640 1 -- 172.21.15.86:0/4284477211 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v153) ==== 170+0+0 (secure 0 0 0) 0x7f631c05b3d0 con 0x7f63240d1c20 2024-04-25T14:13:33.206 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-04-25T14:13:33.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.211+0000 7f632ae00640 1 -- 172.21.15.86:0/4284477211 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f630006f550 msgr2=0x7f6300071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:33.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.211+0000 7f632ae00640 1 --2- 172.21.15.86:0/4284477211 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f630006f550 0x7f6300071a60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f63140ad000 tx=0x7f63140a6000 comp rx=0 tx=0).stop 2024-04-25T14:13:33.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.211+0000 7f632ae00640 1 -- 172.21.15.86:0/4284477211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63240d1c20 msgr2=0x7f63241580d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:33.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.211+0000 7f632ae00640 1 --2- 172.21.15.86:0/4284477211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63240d1c20 0x7f63241580d0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f631c00c730 tx=0x7f631c00cbf0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.211+0000 7f632ae00640 1 -- 172.21.15.86:0/4284477211 shutdown_connections 2024-04-25T14:13:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.211+0000 7f632ae00640 1 --2- 172.21.15.86:0/4284477211 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f630006f550 0x7f6300071a60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.211+0000 7f632ae00640 1 --2- 172.21.15.86:0/4284477211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63240d1c20 0x7f63241580d0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.211+0000 7f632ae00640 1 --2- 172.21.15.86:0/4284477211 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63240ce790 0x7f63240c4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.211+0000 7f632ae00640 1 --2- 172.21.15.86:0/4284477211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6324070330 0x7f6324090440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.211+0000 7f632ae00640 1 -- 172.21.15.86:0/4284477211 >> 172.21.15.86:0/4284477211 conn(0x7f63240b4650 msgr2=0x7f632415c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.211+0000 7f632ae00640 1 -- 172.21.15.86:0/4284477211 shutdown_connections 2024-04-25T14:13:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.211+0000 7f632ae00640 1 -- 172.21.15.86:0/4284477211 wait complete. 2024-04-25T14:13:33.246 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T14:13:33.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.399+0000 7f7610a00640 1 Processor -- start 2024-04-25T14:13:33.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.399+0000 7f7610a00640 1 -- start start 2024-04-25T14:13:33.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.399+0000 7f7610a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f760c15c2a0 0x7f760c0ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:33.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.399+0000 7f7610a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f760c06fd50 0x7f760c070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:33.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.399+0000 7f7610a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f760c0b0a60 con 0x7f760c06c670 2024-04-25T14:13:33.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.399+0000 7f7610a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f760c06ef90 con 0x7f760c15c2a0 2024-04-25T14:13:33.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.399+0000 7f7610a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f760c06f270 con 0x7f760c06fd50 2024-04-25T14:13:33.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.399+0000 7f7609e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f760c15c2a0 0x7f760c0ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:33.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.399+0000 7f760a800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f760c06c670 0x7f760c06ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40222/0 (socket says 172.21.15.86:40222) 2024-04-25T14:13:33.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.399+0000 7f760a800640 1 -- 172.21.15.86:0/1943178326 learned_addr learned my addr 172.21.15.86:0/1943178326 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:33.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.399+0000 7f7609400640 1 --2- 172.21.15.86:0/1943178326 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f760c06fd50 0x7f760c070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:33.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.399+0000 7f7609e00640 1 -- 172.21.15.86:0/1943178326 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f760c06fd50 msgr2=0x7f760c070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:33.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.399+0000 7f7609e00640 1 --2- 172.21.15.86:0/1943178326 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f760c06fd50 0x7f760c070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.399+0000 7f7609e00640 1 -- 172.21.15.86:0/1943178326 >> v1:172.21.15.86:6789/0 conn(0x7f760c06c670 legacy=0x7f760c06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:33.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.399+0000 7f7608a00640 1 -- 172.21.15.86:0/1943178326 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 396575975 0 0) 0x7f760c0b0a60 con 0x7f760c06c670 2024-04-25T14:13:33.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.399+0000 7f7609e00640 1 -- 172.21.15.86:0/1943178326 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f760c06f550 con 0x7f760c15c2a0 2024-04-25T14:13:33.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.403+0000 7f7609e00640 1 --2- 172.21.15.86:0/1943178326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f760c15c2a0 0x7f760c0ae130 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f75fc0097c0 tx=0x7f75fc01d610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8662848dc2510eca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:33.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.403+0000 7f7608a00640 1 -- 172.21.15.86:0/1943178326 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75fc0ad070 con 0x7f760c15c2a0 2024-04-25T14:13:33.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.403+0000 7f7608a00640 1 -- 172.21.15.86:0/1943178326 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75fc0a7630 con 0x7f760c15c2a0 2024-04-25T14:13:33.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.403+0000 7f7608a00640 1 -- 172.21.15.86:0/1943178326 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75fc0a7a00 con 0x7f760c15c2a0 2024-04-25T14:13:33.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.403+0000 7f7610a00640 1 -- 172.21.15.86:0/1943178326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f760c15c2a0 msgr2=0x7f760c0ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:33.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.403+0000 7f7610a00640 1 --2- 172.21.15.86:0/1943178326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f760c15c2a0 0x7f760c0ae130 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f75fc0097c0 tx=0x7f75fc01d610 comp rx=0 tx=0).stop 2024-04-25T14:13:33.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.403+0000 7f7610a00640 1 -- 172.21.15.86:0/1943178326 shutdown_connections 2024-04-25T14:13:33.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.403+0000 7f7610a00640 1 --2- 172.21.15.86:0/1943178326 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f760c06fd50 0x7f760c070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.403+0000 7f7610a00640 1 --2- 172.21.15.86:0/1943178326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f760c15c2a0 0x7f760c0ae130 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.403+0000 7f7610a00640 1 -- 172.21.15.86:0/1943178326 >> 172.21.15.86:0/1943178326 conn(0x7f760c0b4650 msgr2=0x7f760c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:33.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.403+0000 7f7610a00640 1 -- 172.21.15.86:0/1943178326 shutdown_connections 2024-04-25T14:13:33.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.403+0000 7f7610a00640 1 -- 172.21.15.86:0/1943178326 wait complete. 2024-04-25T14:13:33.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.407+0000 7f7610a00640 1 Processor -- start 2024-04-25T14:13:33.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.407+0000 7f7610a00640 1 -- start start 2024-04-25T14:13:33.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.407+0000 7f7610a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f760c06c670 0x7f760c15d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:33.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.407+0000 7f7610a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f760c06fd50 0x7f760c09ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:33.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.407+0000 7f7610a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f760c15c2a0 0x7f760c09b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:33.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.407+0000 7f7610a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f760c15c830 con 0x7f760c15c2a0 2024-04-25T14:13:33.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.407+0000 7f7610a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f760c15c6b0 con 0x7f760c06c670 2024-04-25T14:13:33.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.407+0000 7f7610a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f760c0ae670 con 0x7f760c06fd50 2024-04-25T14:13:33.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.407+0000 7f760a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f760c15c2a0 0x7f760c09b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:33.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.407+0000 7f760a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f760c15c2a0 0x7f760c09b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55306/0 (socket says 172.21.15.86:55306) 2024-04-25T14:13:33.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.407+0000 7f760a800640 1 -- 172.21.15.86:0/3791501534 learned_addr learned my addr 172.21.15.86:0/3791501534 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:33.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.407+0000 7f7609400640 1 --2- 172.21.15.86:0/3791501534 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f760c06fd50 0x7f760c09ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:33.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.407+0000 7f7609e00640 1 --2- 172.21.15.86:0/3791501534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f760c06c670 0x7f760c15d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:33.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.407+0000 7f760a800640 1 -- 172.21.15.86:0/3791501534 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f760c06fd50 msgr2=0x7f760c09ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:33.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.407+0000 7f760a800640 1 --2- 172.21.15.86:0/3791501534 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f760c06fd50 0x7f760c09ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.407+0000 7f760a800640 1 -- 172.21.15.86:0/3791501534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f760c06c670 msgr2=0x7f760c15d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:33.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.407+0000 7f760a800640 1 --2- 172.21.15.86:0/3791501534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f760c06c670 0x7f760c15d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:33.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.407+0000 7f760a800640 1 -- 172.21.15.86:0/3791501534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f760c154e60 con 0x7f760c15c2a0 2024-04-25T14:13:33.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.407+0000 7f760a800640 1 --2- 172.21.15.86:0/3791501534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f760c15c2a0 0x7f760c09b2b0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f760400c730 tx=0x7f760400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:33.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.411+0000 7f7602000640 1 -- 172.21.15.86:0/3791501534 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7604018070 con 0x7f760c15c2a0 2024-04-25T14:13:33.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.411+0000 7f7602000640 1 -- 172.21.15.86:0/3791501534 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f760400ce40 con 0x7f760c15c2a0 2024-04-25T14:13:33.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.411+0000 7f7602000640 1 -- 172.21.15.86:0/3791501534 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760400a880 con 0x7f760c15c2a0 2024-04-25T14:13:33.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.411+0000 7f7610a00640 1 -- 172.21.15.86:0/3791501534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f760c155120 con 0x7f760c15c2a0 2024-04-25T14:13:33.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.411+0000 7f7610a00640 1 -- 172.21.15.86:0/3791501534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f760c09a610 con 0x7f760c15c2a0 2024-04-25T14:13:33.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.411+0000 7f7610a00640 1 -- 172.21.15.86:0/3791501534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f760c166f30 con 0x7f760c15c2a0 2024-04-25T14:13:33.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.411+0000 7f7602000640 1 -- 172.21.15.86:0/3791501534 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7f760400aa20 con 0x7f760c15c2a0 2024-04-25T14:13:33.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.415+0000 7f7602000640 1 --2- 172.21.15.86:0/3791501534 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f75ec06f550 0x7f75ec071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:33.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.415+0000 7f7602000640 1 -- 172.21.15.86:0/3791501534 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 6001+0+0 (secure 0 0 0) 0x7f760408efb0 con 0x7f760c15c2a0 2024-04-25T14:13:33.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.415+0000 7f7609e00640 1 --2- 172.21.15.86:0/3791501534 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f75ec06f550 0x7f75ec071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:33.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.415+0000 7f7609e00640 1 --2- 172.21.15.86:0/3791501534 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f75ec06f550 0x7f75ec071a60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f75fc0af000 tx=0x7f75fc0a8000 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:33.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.419+0000 7f7602000640 1 -- 172.21.15.86:0/3791501534 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7604021020 con 0x7f760c15c2a0 2024-04-25T14:13:33.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:33.607+0000 7f7610a00640 1 -- 172.21.15.86:0/3791501534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f760c0c4750 con 0x7f760c15c2a0 2024-04-25T14:13:34.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.223+0000 7f7602000640 1 -- 172.21.15.86:0/3791501534 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 23 and data pool 24 v113) ==== 174+0+0 (secure 0 0 0) 0x7f760405bc40 con 0x7f760c15c2a0 2024-04-25T14:13:34.220 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 23 and data pool 24 2024-04-25T14:13:34.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.227+0000 7f7610a00640 1 -- 172.21.15.86:0/3791501534 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f75ec06f550 msgr2=0x7f75ec071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.231+0000 7f7610a00640 1 --2- 172.21.15.86:0/3791501534 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f75ec06f550 0x7f75ec071a60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f75fc0af000 tx=0x7f75fc0a8000 comp rx=0 tx=0).stop 2024-04-25T14:13:34.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.231+0000 7f7610a00640 1 -- 172.21.15.86:0/3791501534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f760c15c2a0 msgr2=0x7f760c09b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.231+0000 7f7610a00640 1 --2- 172.21.15.86:0/3791501534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f760c15c2a0 0x7f760c09b2b0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f760400c730 tx=0x7f760400cbf0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.231+0000 7f7610a00640 1 -- 172.21.15.86:0/3791501534 shutdown_connections 2024-04-25T14:13:34.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.231+0000 7f7610a00640 1 --2- 172.21.15.86:0/3791501534 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f75ec06f550 0x7f75ec071a60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.231+0000 7f7610a00640 1 --2- 172.21.15.86:0/3791501534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f760c15c2a0 0x7f760c09b2b0 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.231+0000 7f7610a00640 1 --2- 172.21.15.86:0/3791501534 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f760c06fd50 0x7f760c09ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.231+0000 7f7610a00640 1 --2- 172.21.15.86:0/3791501534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f760c06c670 0x7f760c15d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.231+0000 7f7610a00640 1 -- 172.21.15.86:0/3791501534 >> 172.21.15.86:0/3791501534 conn(0x7f760c0b4650 msgr2=0x7f760c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:34.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.231+0000 7f7610a00640 1 -- 172.21.15.86:0/3791501534 shutdown_connections 2024-04-25T14:13:34.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.235+0000 7f7610a00640 1 -- 172.21.15.86:0/3791501534 wait complete. 2024-04-25T14:13:34.272 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:13:34.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.419+0000 7f43df400640 1 Processor -- start 2024-04-25T14:13:34.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.419+0000 7f43df400640 1 -- start start 2024-04-25T14:13:34.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.423+0000 7f43df400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f43d806fd50 0x7f43d8070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:34.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.423+0000 7f43df400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f43d815feb0 0x7f43d81602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:34.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.423+0000 7f43df400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43d80b0a60 con 0x7f43d815c2a0 2024-04-25T14:13:34.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.423+0000 7f43df400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f43d80ae840 con 0x7f43d806fd50 2024-04-25T14:13:34.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.423+0000 7f43df400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f43d806ef90 con 0x7f43d815feb0 2024-04-25T14:13:34.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.423+0000 7f43d7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f43d806fd50 0x7f43d8070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:34.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.423+0000 7f43dca00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f43d815c2a0 0x7f43d80ae130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40228/0 (socket says 172.21.15.86:40228) 2024-04-25T14:13:34.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.423+0000 7f43dca00640 1 -- 172.21.15.86:0/4160508636 learned_addr learned my addr 172.21.15.86:0/4160508636 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:34.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.423+0000 7f43dd400640 1 --2- 172.21.15.86:0/4160508636 >> v2:172.21.15.155:3300/0 conn(0x7f43d815feb0 0x7f43d81602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:34.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.423+0000 7f43d7e00640 1 -- 172.21.15.86:0/4160508636 >> v2:172.21.15.155:3300/0 conn(0x7f43d815feb0 msgr2=0x7f43d81602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:34.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.423+0000 7f43d7e00640 1 --2- 172.21.15.86:0/4160508636 >> v2:172.21.15.155:3300/0 conn(0x7f43d815feb0 0x7f43d81602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.423+0000 7f43d7e00640 1 -- 172.21.15.86:0/4160508636 >> v1:172.21.15.86:6789/0 conn(0x7f43d815c2a0 legacy=0x7f43d80ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.423+0000 7f43d7e00640 1 -- 172.21.15.86:0/4160508636 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d806f270 con 0x7f43d806fd50 2024-04-25T14:13:34.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.423+0000 7f43dd400640 1 --2- 172.21.15.86:0/4160508636 >> v2:172.21.15.155:3300/0 conn(0x7f43d815feb0 0x7f43d81602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:34.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.427+0000 7f43d7e00640 1 --2- 172.21.15.86:0/4160508636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d806fd50 0x7f43d8070160 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f43d80b0940 tx=0x7f43c401d790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea51dbbc9f1ebc7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:34.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.427+0000 7f43d7400640 1 -- 172.21.15.86:0/4160508636 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c40b3070 con 0x7f43d806fd50 2024-04-25T14:13:34.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.427+0000 7f43d7400640 1 -- 172.21.15.86:0/4160508636 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43c40a67c0 con 0x7f43d806fd50 2024-04-25T14:13:34.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.427+0000 7f43d7400640 1 -- 172.21.15.86:0/4160508636 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c400b040 con 0x7f43d806fd50 2024-04-25T14:13:34.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.427+0000 7f43df400640 1 -- 172.21.15.86:0/4160508636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d806fd50 msgr2=0x7f43d8070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.427+0000 7f43df400640 1 --2- 172.21.15.86:0/4160508636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d806fd50 0x7f43d8070160 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f43d80b0940 tx=0x7f43c401d790 comp rx=0 tx=0).stop 2024-04-25T14:13:34.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.427+0000 7f43df400640 1 -- 172.21.15.86:0/4160508636 shutdown_connections 2024-04-25T14:13:34.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.427+0000 7f43df400640 1 --2- 172.21.15.86:0/4160508636 >> v2:172.21.15.155:3300/0 conn(0x7f43d815feb0 0x7f43d81602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.427+0000 7f43df400640 1 --2- 172.21.15.86:0/4160508636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d806fd50 0x7f43d8070160 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.427+0000 7f43df400640 1 -- 172.21.15.86:0/4160508636 >> 172.21.15.86:0/4160508636 conn(0x7f43d80b4650 msgr2=0x7f43d80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:34.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.427+0000 7f43df400640 1 -- 172.21.15.86:0/4160508636 shutdown_connections 2024-04-25T14:13:34.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.427+0000 7f43df400640 1 -- 172.21.15.86:0/4160508636 wait complete. 2024-04-25T14:13:34.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.431+0000 7f43df400640 1 Processor -- start 2024-04-25T14:13:34.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.431+0000 7f43df400640 1 -- start start 2024-04-25T14:13:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.431+0000 7f43df400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43d815c2a0 0x7f43d8158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.431+0000 7f43df400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d815feb0 0x7f43d81fcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.431+0000 7f43df400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43d80af1e0 0x7f43d80af5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.431+0000 7f43df400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43d815c6b0 con 0x7f43d815feb0 2024-04-25T14:13:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.431+0000 7f43df400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f43d80d0fc0 con 0x7f43d80af1e0 2024-04-25T14:13:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.431+0000 7f43df400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f43d815c830 con 0x7f43d815c2a0 2024-04-25T14:13:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.431+0000 7f43d7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d815feb0 0x7f43d81fcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.431+0000 7f43d7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d815feb0 0x7f43d81fcfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55330/0 (socket says 172.21.15.86:55330) 2024-04-25T14:13:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.431+0000 7f43d7e00640 1 -- 172.21.15.86:0/2862025241 learned_addr learned my addr 172.21.15.86:0/2862025241 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.431+0000 7f43dd400640 1 --2- 172.21.15.86:0/2862025241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43d80af1e0 0x7f43d80af5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.431+0000 7f43dca00640 1 --2- 172.21.15.86:0/2862025241 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43d815c2a0 0x7f43d8158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.431+0000 7f43d7e00640 1 -- 172.21.15.86:0/2862025241 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43d815c2a0 msgr2=0x7f43d8158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.431+0000 7f43d7e00640 1 --2- 172.21.15.86:0/2862025241 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43d815c2a0 0x7f43d8158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.431+0000 7f43d7e00640 1 -- 172.21.15.86:0/2862025241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43d80af1e0 msgr2=0x7f43d80af5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.431+0000 7f43d7e00640 1 --2- 172.21.15.86:0/2862025241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43d80af1e0 0x7f43d80af5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.431+0000 7f43d7e00640 1 -- 172.21.15.86:0/2862025241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d80afb30 con 0x7f43d815feb0 2024-04-25T14:13:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.431+0000 7f43d7e00640 1 --2- 172.21.15.86:0/2862025241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d815feb0 0x7f43d81fcfa0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f43c40b1040 tx=0x7f43c401a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:34.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.431+0000 7f43d4c00640 1 -- 172.21.15.86:0/2862025241 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c40b3070 con 0x7f43d815feb0 2024-04-25T14:13:34.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.431+0000 7f43d4c00640 1 -- 172.21.15.86:0/2862025241 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43c4004030 con 0x7f43d815feb0 2024-04-25T14:13:34.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.431+0000 7f43df400640 1 -- 172.21.15.86:0/2862025241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d81fd4e0 con 0x7f43d815feb0 2024-04-25T14:13:34.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.431+0000 7f43d4c00640 1 -- 172.21.15.86:0/2862025241 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4004420 con 0x7f43d815feb0 2024-04-25T14:13:34.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.431+0000 7f43df400640 1 -- 172.21.15.86:0/2862025241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d81fdc80 con 0x7f43d815feb0 2024-04-25T14:13:34.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.435+0000 7f43df400640 1 -- 172.21.15.86:0/2862025241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43d8159220 con 0x7f43d815feb0 2024-04-25T14:13:34.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.435+0000 7f43d4c00640 1 -- 172.21.15.86:0/2862025241 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7f43c40b7070 con 0x7f43d815feb0 2024-04-25T14:13:34.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.435+0000 7f43d4c00640 1 --2- 172.21.15.86:0/2862025241 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f43b806f550 0x7f43b8071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:34.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.435+0000 7f43d4c00640 1 -- 172.21.15.86:0/2862025241 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 6101+0+0 (secure 0 0 0) 0x7f43c41291d0 con 0x7f43d815feb0 2024-04-25T14:13:34.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.435+0000 7f43dca00640 1 --2- 172.21.15.86:0/2862025241 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f43b806f550 0x7f43b8071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:34.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.435+0000 7f43dca00640 1 --2- 172.21.15.86:0/2862025241 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f43b806f550 0x7f43b8071a60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f43cc000f70 tx=0x7f43cc009040 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:34.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.439+0000 7f43d4c00640 1 -- 172.21.15.86:0/2862025241 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f43d8159220 con 0x7f43d815feb0 2024-04-25T14:13:34.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.595+0000 7f43df400640 1 -- 172.21.15.86:0/2862025241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f43d8069d00 con 0x7f43d815feb0 2024-04-25T14:13:34.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.599+0000 7f43d4c00640 1 -- 172.21.15.86:0/2862025241 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) ==== 74+0+16897 (secure 0 0 0) 0x7f43c400a020 con 0x7f43d815feb0 2024-04-25T14:13:34.597 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:13:34.597 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":154,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:13:33.612504+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:13:32.006877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T14:13:32.574088+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:13:34.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.599+0000 7f43df400640 1 -- 172.21.15.86:0/2862025241 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f43b806f550 msgr2=0x7f43b8071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.599+0000 7f43df400640 1 --2- 172.21.15.86:0/2862025241 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f43b806f550 0x7f43b8071a60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f43cc000f70 tx=0x7f43cc009040 comp rx=0 tx=0).stop 2024-04-25T14:13:34.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.599+0000 7f43df400640 1 -- 172.21.15.86:0/2862025241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d815feb0 msgr2=0x7f43d81fcfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.599+0000 7f43df400640 1 --2- 172.21.15.86:0/2862025241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d815feb0 0x7f43d81fcfa0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f43c40b1040 tx=0x7f43c401a530 comp rx=0 tx=0).stop 2024-04-25T14:13:34.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.599+0000 7f43df400640 1 -- 172.21.15.86:0/2862025241 shutdown_connections 2024-04-25T14:13:34.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.599+0000 7f43df400640 1 --2- 172.21.15.86:0/2862025241 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f43b806f550 0x7f43b8071a60 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.599+0000 7f43df400640 1 --2- 172.21.15.86:0/2862025241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43d80af1e0 0x7f43d80af5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.599+0000 7f43df400640 1 --2- 172.21.15.86:0/2862025241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43d815feb0 0x7f43d81fcfa0 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.599+0000 7f43df400640 1 --2- 172.21.15.86:0/2862025241 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43d815c2a0 0x7f43d8158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.599+0000 7f43df400640 1 -- 172.21.15.86:0/2862025241 >> 172.21.15.86:0/2862025241 conn(0x7f43d80b4650 msgr2=0x7f43d809a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:34.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.599+0000 7f43df400640 1 -- 172.21.15.86:0/2862025241 shutdown_connections 2024-04-25T14:13:34.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.599+0000 7f43df400640 1 -- 172.21.15.86:0/2862025241 wait complete. 2024-04-25T14:13:34.619 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:13:34.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.767+0000 7fa443a00640 1 Processor -- start 2024-04-25T14:13:34.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.767+0000 7fa443a00640 1 -- start start 2024-04-25T14:13:34.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.771+0000 7fa443a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa43c0740c0 0x7fa43c1676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:34.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.771+0000 7fa443a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa43c072bd0 con 0x7fa43c0d16e0 2024-04-25T14:13:34.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.771+0000 7fa443a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa43c06fbd0 con 0x7fa43c0cfca0 2024-04-25T14:13:34.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.771+0000 7fa443a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa43c070470 con 0x7fa43c0740c0 2024-04-25T14:13:34.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.771+0000 7fa43be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa43c0d16e0 0x7fa43c0d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40236/0 (socket says 172.21.15.86:40236) 2024-04-25T14:13:34.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.771+0000 7fa43be00640 1 -- 172.21.15.86:0/1251988932 learned_addr learned my addr 172.21.15.86:0/1251988932 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:34.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.771+0000 7fa441a00640 1 --2- 172.21.15.86:0/1251988932 >> v2:172.21.15.155:3300/0 conn(0x7fa43c0740c0 0x7fa43c1676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:34.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.771+0000 7fa43b400640 1 -- 172.21.15.86:0/1251988932 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 652194121 0 0) 0x7fa43c072bd0 con 0x7fa43c0d16e0 2024-04-25T14:13:34.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.771+0000 7fa43b400640 1 -- 172.21.15.86:0/1251988932 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa420003530 con 0x7fa43c0d16e0 2024-04-25T14:13:34.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.771+0000 7fa441a00640 1 -- 172.21.15.86:0/1251988932 >> v1:172.21.15.155:6789/0 conn(0x7fa43c0cfca0 legacy=0x7fa43c06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.771+0000 7fa441a00640 1 -- 172.21.15.86:0/1251988932 >> v1:172.21.15.86:6789/0 conn(0x7fa43c0d16e0 legacy=0x7fa43c0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.771+0000 7fa441a00640 1 -- 172.21.15.86:0/1251988932 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43c06feb0 con 0x7fa43c0740c0 2024-04-25T14:13:34.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.775+0000 7fa441a00640 1 --2- 172.21.15.86:0/1251988932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa43c0740c0 0x7fa43c1676d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa434009d90 tx=0x7fa43401eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c711bac19dba431 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:34.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.775+0000 7fa43b400640 1 -- 172.21.15.86:0/1251988932 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4340b4070 con 0x7fa43c0740c0 2024-04-25T14:13:34.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.775+0000 7fa43b400640 1 -- 172.21.15.86:0/1251988932 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4340a7a60 con 0x7fa43c0740c0 2024-04-25T14:13:34.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.775+0000 7fa43b400640 1 -- 172.21.15.86:0/1251988932 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4340a7e10 con 0x7fa43c0740c0 2024-04-25T14:13:34.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.775+0000 7fa443a00640 1 -- 172.21.15.86:0/1251988932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa43c0740c0 msgr2=0x7fa43c1676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.775+0000 7fa443a00640 1 --2- 172.21.15.86:0/1251988932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa43c0740c0 0x7fa43c1676d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa434009d90 tx=0x7fa43401eb80 comp rx=0 tx=0).stop 2024-04-25T14:13:34.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.775+0000 7fa443a00640 1 -- 172.21.15.86:0/1251988932 shutdown_connections 2024-04-25T14:13:34.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.775+0000 7fa443a00640 1 --2- 172.21.15.86:0/1251988932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa43c0740c0 0x7fa43c1676d0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.775+0000 7fa443a00640 1 -- 172.21.15.86:0/1251988932 >> 172.21.15.86:0/1251988932 conn(0x7fa43c0b4650 msgr2=0x7fa43c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:34.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.775+0000 7fa443a00640 1 -- 172.21.15.86:0/1251988932 shutdown_connections 2024-04-25T14:13:34.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.775+0000 7fa443a00640 1 -- 172.21.15.86:0/1251988932 wait complete. 2024-04-25T14:13:34.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.775+0000 7fa443a00640 1 Processor -- start 2024-04-25T14:13:34.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.775+0000 7fa443a00640 1 -- start start 2024-04-25T14:13:34.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.779+0000 7fa443a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa43c0740c0 0x7fa43c09a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:34.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.779+0000 7fa443a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa43c0cfca0 0x7fa43c15dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:34.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.779+0000 7fa443a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa43c0d16e0 0x7fa43c15bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:34.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.779+0000 7fa443a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c0ce900 con 0x7fa43c0d16e0 2024-04-25T14:13:34.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.779+0000 7fa443a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c0ce780 con 0x7fa43c0740c0 2024-04-25T14:13:34.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.779+0000 7fa443a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa43c0cea80 con 0x7fa43c0cfca0 2024-04-25T14:13:34.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.779+0000 7fa441a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa43c0d16e0 0x7fa43c15bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:34.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.779+0000 7fa441a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa43c0d16e0 0x7fa43c15bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55338/0 (socket says 172.21.15.86:55338) 2024-04-25T14:13:34.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.779+0000 7fa441a00640 1 -- 172.21.15.86:0/3420662769 learned_addr learned my addr 172.21.15.86:0/3420662769 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:34.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.779+0000 7fa441000640 1 --2- 172.21.15.86:0/3420662769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa43c0740c0 0x7fa43c09a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:34.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.779+0000 7fa43be00640 1 --2- 172.21.15.86:0/3420662769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa43c0cfca0 0x7fa43c15dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:34.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.779+0000 7fa441a00640 1 -- 172.21.15.86:0/3420662769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa43c0cfca0 msgr2=0x7fa43c15dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.779+0000 7fa441a00640 1 --2- 172.21.15.86:0/3420662769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa43c0cfca0 0x7fa43c15dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.779+0000 7fa441a00640 1 -- 172.21.15.86:0/3420662769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa43c0740c0 msgr2=0x7fa43c09a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.779+0000 7fa441a00640 1 --2- 172.21.15.86:0/3420662769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa43c0740c0 0x7fa43c09a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.779+0000 7fa441a00640 1 -- 172.21.15.86:0/3420662769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43c160350 con 0x7fa43c0d16e0 2024-04-25T14:13:34.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.779+0000 7fa441000640 1 --2- 172.21.15.86:0/3420662769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa43c0740c0 0x7fa43c09a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:34.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.779+0000 7fa441a00640 1 --2- 172.21.15.86:0/3420662769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa43c0d16e0 0x7fa43c15bb30 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fa434009d90 tx=0x7fa43401b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:34.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.779+0000 7fa438c00640 1 -- 172.21.15.86:0/3420662769 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4340b4040 con 0x7fa43c0d16e0 2024-04-25T14:13:34.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.779+0000 7fa438c00640 1 -- 172.21.15.86:0/3420662769 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4340a7860 con 0x7fa43c0d16e0 2024-04-25T14:13:34.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.779+0000 7fa443a00640 1 -- 172.21.15.86:0/3420662769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa43c15fd90 con 0x7fa43c0d16e0 2024-04-25T14:13:34.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.779+0000 7fa438c00640 1 -- 172.21.15.86:0/3420662769 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa434009450 con 0x7fa43c0d16e0 2024-04-25T14:13:34.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.779+0000 7fa443a00640 1 -- 172.21.15.86:0/3420662769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa43c158570 con 0x7fa43c0d16e0 2024-04-25T14:13:34.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.779+0000 7fa443a00640 1 -- 172.21.15.86:0/3420662769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa43c15fb00 con 0x7fa43c0d16e0 2024-04-25T14:13:34.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.783+0000 7fa438c00640 1 -- 172.21.15.86:0/3420662769 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7fa4340bf020 con 0x7fa43c0d16e0 2024-04-25T14:13:34.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.783+0000 7fa438c00640 1 --2- 172.21.15.86:0/3420662769 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fa41806f550 0x7fa418071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:34.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.783+0000 7fa438c00640 1 -- 172.21.15.86:0/3420662769 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 6101+0+0 (secure 0 0 0) 0x7fa434129b10 con 0x7fa43c0d16e0 2024-04-25T14:13:34.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.783+0000 7fa441000640 1 --2- 172.21.15.86:0/3420662769 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fa41806f550 0x7fa418071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:34.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.783+0000 7fa441000640 1 --2- 172.21.15.86:0/3420662769 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fa41806f550 0x7fa418071a60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa43c070710 tx=0x7fa424009040 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:34.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.787+0000 7fa438c00640 1 -- 172.21.15.86:0/3420662769 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa434018770 con 0x7fa43c0d16e0 2024-04-25T14:13:34.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.947+0000 7fa443a00640 1 -- 172.21.15.86:0/3420662769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa43c09f4b0 con 0x7fa43c0d16e0 2024-04-25T14:13:34.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.951+0000 7fa438c00640 1 -- 172.21.15.86:0/3420662769 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) ==== 74+0+16897 (secure 0 0 0) 0x7fa4340f5ff0 con 0x7fa43c0d16e0 2024-04-25T14:13:34.948 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:13:34.949 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":154,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:13:33.612504+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:13:32.006877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T14:13:32.574088+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:13:34.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.951+0000 7fa443a00640 1 -- 172.21.15.86:0/3420662769 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fa41806f550 msgr2=0x7fa418071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.951+0000 7fa443a00640 1 --2- 172.21.15.86:0/3420662769 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fa41806f550 0x7fa418071a60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa43c070710 tx=0x7fa424009040 comp rx=0 tx=0).stop 2024-04-25T14:13:34.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.951+0000 7fa443a00640 1 -- 172.21.15.86:0/3420662769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa43c0d16e0 msgr2=0x7fa43c15bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:34.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.951+0000 7fa443a00640 1 --2- 172.21.15.86:0/3420662769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa43c0d16e0 0x7fa43c15bb30 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fa434009d90 tx=0x7fa43401b920 comp rx=0 tx=0).stop 2024-04-25T14:13:34.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.951+0000 7fa443a00640 1 -- 172.21.15.86:0/3420662769 shutdown_connections 2024-04-25T14:13:34.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.951+0000 7fa443a00640 1 --2- 172.21.15.86:0/3420662769 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fa41806f550 0x7fa418071a60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.951+0000 7fa443a00640 1 --2- 172.21.15.86:0/3420662769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa43c0d16e0 0x7fa43c15bb30 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.951+0000 7fa443a00640 1 --2- 172.21.15.86:0/3420662769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa43c0cfca0 0x7fa43c15dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.951+0000 7fa443a00640 1 --2- 172.21.15.86:0/3420662769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa43c0740c0 0x7fa43c09a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:34.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.951+0000 7fa443a00640 1 -- 172.21.15.86:0/3420662769 >> 172.21.15.86:0/3420662769 conn(0x7fa43c0b4650 msgr2=0x7fa43c0b2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:34.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.951+0000 7fa443a00640 1 -- 172.21.15.86:0/3420662769 shutdown_connections 2024-04-25T14:13:34.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:34.951+0000 7fa443a00640 1 -- 172.21.15.86:0/3420662769 wait complete. 2024-04-25T14:13:34.971 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T14:13:35.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.123+0000 7fb768a00640 1 Processor -- start 2024-04-25T14:13:35.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.123+0000 7fb768a00640 1 -- start start 2024-04-25T14:13:35.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.123+0000 7fb768a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb76406fd50 0x7fb764070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:35.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.123+0000 7fb768a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb76415feb0 0x7fb7641602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:35.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.123+0000 7fb768a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7640b0a60 con 0x7fb76415c2a0 2024-04-25T14:13:35.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.123+0000 7fb768a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb7640ae840 con 0x7fb76406fd50 2024-04-25T14:13:35.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.123+0000 7fb768a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb76406ef90 con 0x7fb76415feb0 2024-04-25T14:13:35.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.123+0000 7fb759400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb76406fd50 0x7fb764070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:35.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.123+0000 7fb759400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb76406fd50 0x7fb764070160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55354/0 (socket says 172.21.15.86:55354) 2024-04-25T14:13:35.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.123+0000 7fb759400640 1 -- 172.21.15.86:0/1454393532 learned_addr learned my addr 172.21.15.86:0/1454393532 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:35.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.123+0000 7fb762800640 1 --2- 172.21.15.86:0/1454393532 >> v2:172.21.15.155:3300/0 conn(0x7fb76415feb0 0x7fb7641602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:35.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.123+0000 7fb759400640 1 -- 172.21.15.86:0/1454393532 >> v2:172.21.15.155:3300/0 conn(0x7fb76415feb0 msgr2=0x7fb7641602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:35.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.123+0000 7fb759400640 1 --2- 172.21.15.86:0/1454393532 >> v2:172.21.15.155:3300/0 conn(0x7fb76415feb0 0x7fb7641602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.123+0000 7fb759400640 1 -- 172.21.15.86:0/1454393532 >> v1:172.21.15.155:6789/0 conn(0x7fb76415c2a0 legacy=0x7fb7640ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:35.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.123+0000 7fb759400640 1 -- 172.21.15.86:0/1454393532 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76406f270 con 0x7fb76406fd50 2024-04-25T14:13:35.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.123+0000 7fb762800640 1 --2- 172.21.15.86:0/1454393532 >> v2:172.21.15.155:3300/0 conn(0x7fb76415feb0 0x7fb7641602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:35.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.127+0000 7fb759400640 1 --2- 172.21.15.86:0/1454393532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb76406fd50 0x7fb764070160 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fb7640b0940 tx=0x7fb74c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf95f8151f59057 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:35.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.127+0000 7fb761400640 1 -- 172.21.15.86:0/1454393532 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c0b0070 con 0x7fb76406fd50 2024-04-25T14:13:35.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.127+0000 7fb761400640 1 -- 172.21.15.86:0/1454393532 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb74c0ab600 con 0x7fb76406fd50 2024-04-25T14:13:35.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.127+0000 7fb761400640 1 -- 172.21.15.86:0/1454393532 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c0ab9d0 con 0x7fb76406fd50 2024-04-25T14:13:35.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.127+0000 7fb768a00640 1 -- 172.21.15.86:0/1454393532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb76406fd50 msgr2=0x7fb764070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:35.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.127+0000 7fb768a00640 1 --2- 172.21.15.86:0/1454393532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb76406fd50 0x7fb764070160 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fb7640b0940 tx=0x7fb74c01c420 comp rx=0 tx=0).stop 2024-04-25T14:13:35.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.127+0000 7fb768a00640 1 -- 172.21.15.86:0/1454393532 shutdown_connections 2024-04-25T14:13:35.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.127+0000 7fb768a00640 1 --2- 172.21.15.86:0/1454393532 >> v2:172.21.15.155:3300/0 conn(0x7fb76415feb0 0x7fb7641602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.127+0000 7fb768a00640 1 --2- 172.21.15.86:0/1454393532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb76406fd50 0x7fb764070160 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.127+0000 7fb768a00640 1 -- 172.21.15.86:0/1454393532 >> 172.21.15.86:0/1454393532 conn(0x7fb7640b4650 msgr2=0x7fb7640b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:35.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.127+0000 7fb768a00640 1 -- 172.21.15.86:0/1454393532 shutdown_connections 2024-04-25T14:13:35.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.127+0000 7fb768a00640 1 -- 172.21.15.86:0/1454393532 wait complete. 2024-04-25T14:13:35.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.131+0000 7fb768a00640 1 Processor -- start 2024-04-25T14:13:35.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.131+0000 7fb768a00640 1 -- start start 2024-04-25T14:13:35.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.131+0000 7fb768a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb76406fd50 0x7fb76409a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:35.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.131+0000 7fb768a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb76415c2a0 0x7fb7640729e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:35.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.131+0000 7fb768a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb76415feb0 0x7fb76415ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:35.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.131+0000 7fb768a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb76415c6b0 con 0x7fb76415c2a0 2024-04-25T14:13:35.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.131+0000 7fb768a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb7640d0fc0 con 0x7fb76415feb0 2024-04-25T14:13:35.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.131+0000 7fb768a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb76415c830 con 0x7fb76406fd50 2024-04-25T14:13:35.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.131+0000 7fb759400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb76415c2a0 0x7fb7640729e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:35.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.131+0000 7fb759400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb76415c2a0 0x7fb7640729e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55364/0 (socket says 172.21.15.86:55364) 2024-04-25T14:13:35.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.131+0000 7fb759400640 1 -- 172.21.15.86:0/3938531675 learned_addr learned my addr 172.21.15.86:0/3938531675 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:35.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.135+0000 7fb762800640 1 --2- 172.21.15.86:0/3938531675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb76415feb0 0x7fb76415ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:35.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.135+0000 7fb761e00640 1 --2- 172.21.15.86:0/3938531675 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb76406fd50 0x7fb76409a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:35.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.135+0000 7fb759400640 1 -- 172.21.15.86:0/3938531675 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb76406fd50 msgr2=0x7fb76409a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:35.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.135+0000 7fb759400640 1 --2- 172.21.15.86:0/3938531675 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb76406fd50 0x7fb76409a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.135+0000 7fb759400640 1 -- 172.21.15.86:0/3938531675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb76415feb0 msgr2=0x7fb76415ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:35.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.135+0000 7fb759400640 1 --2- 172.21.15.86:0/3938531675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb76415feb0 0x7fb76415ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:35.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.135+0000 7fb759400640 1 -- 172.21.15.86:0/3938531675 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7640af5f0 con 0x7fb76415c2a0 2024-04-25T14:13:35.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.135+0000 7fb762800640 1 --2- 172.21.15.86:0/3938531675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb76415feb0 0x7fb76415ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:35.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.135+0000 7fb761e00640 1 --2- 172.21.15.86:0/3938531675 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb76406fd50 0x7fb76409a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:35.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.135+0000 7fb759400640 1 --2- 172.21.15.86:0/3938531675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb76415c2a0 0x7fb7640729e0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fb74c0ae040 tx=0x7fb74c01ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:35.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.135+0000 7fb75aa00640 1 -- 172.21.15.86:0/3938531675 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c0ba020 con 0x7fb76415c2a0 2024-04-25T14:13:35.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.135+0000 7fb75aa00640 1 -- 172.21.15.86:0/3938531675 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb74c0b6640 con 0x7fb76415c2a0 2024-04-25T14:13:35.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.135+0000 7fb75aa00640 1 -- 172.21.15.86:0/3938531675 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c0b0040 con 0x7fb76415c2a0 2024-04-25T14:13:35.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.135+0000 7fb768a00640 1 -- 172.21.15.86:0/3938531675 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7640734a0 con 0x7fb76415c2a0 2024-04-25T14:13:35.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.135+0000 7fb768a00640 1 -- 172.21.15.86:0/3938531675 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb764073a60 con 0x7fb76415c2a0 2024-04-25T14:13:35.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.135+0000 7fb768a00640 1 -- 172.21.15.86:0/3938531675 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb764073210 con 0x7fb76415c2a0 2024-04-25T14:13:35.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.135+0000 7fb75aa00640 1 -- 172.21.15.86:0/3938531675 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7fb74c0bd020 con 0x7fb76415c2a0 2024-04-25T14:13:35.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.139+0000 7fb75aa00640 1 --2- 172.21.15.86:0/3938531675 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fb74006f550 0x7fb740071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:35.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.139+0000 7fb75aa00640 1 -- 172.21.15.86:0/3938531675 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 6101+0+0 (secure 0 0 0) 0x7fb74c12a180 con 0x7fb76415c2a0 2024-04-25T14:13:35.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.139+0000 7fb761e00640 1 --2- 172.21.15.86:0/3938531675 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fb74006f550 0x7fb740071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:35.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.139+0000 7fb761e00640 1 --2- 172.21.15.86:0/3938531675 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fb74006f550 0x7fb740071a60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb7640b4200 tx=0x7fb754008040 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:35.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.139+0000 7fb75aa00640 1 -- 172.21.15.86:0/3938531675 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb74c0c4030 con 0x7fb76415c2a0 2024-04-25T14:13:35.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:35.343+0000 7fb768a00640 1 -- 172.21.15.86:0/3938531675 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb764069d00 con 0x7fb76415c2a0 2024-04-25T14:13:36.196 INFO:tasks.ceph.mds.a.smithi086.stderr:2024-04-25T14:13:36.199+0000 7f39f3600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T14:13:36.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.239+0000 7fb75aa00640 1 -- 172.21.15.86:0/3938531675 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v116) ==== 118+0+0 (secure 0 0 0) 0x7fb74c0160d0 con 0x7fb76415c2a0 2024-04-25T14:13:36.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.243+0000 7fb768a00640 1 -- 172.21.15.86:0/3938531675 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fb74006f550 msgr2=0x7fb740071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:36.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.243+0000 7fb768a00640 1 --2- 172.21.15.86:0/3938531675 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fb74006f550 0x7fb740071a60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb7640b4200 tx=0x7fb754008040 comp rx=0 tx=0).stop 2024-04-25T14:13:36.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.243+0000 7fb768a00640 1 -- 172.21.15.86:0/3938531675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb76415c2a0 msgr2=0x7fb7640729e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:36.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.243+0000 7fb768a00640 1 --2- 172.21.15.86:0/3938531675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb76415c2a0 0x7fb7640729e0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fb74c0ae040 tx=0x7fb74c01ac60 comp rx=0 tx=0).stop 2024-04-25T14:13:36.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.243+0000 7fb768a00640 1 -- 172.21.15.86:0/3938531675 shutdown_connections 2024-04-25T14:13:36.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.243+0000 7fb768a00640 1 --2- 172.21.15.86:0/3938531675 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fb74006f550 0x7fb740071a60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.243+0000 7fb768a00640 1 --2- 172.21.15.86:0/3938531675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb76415feb0 0x7fb76415ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.243+0000 7fb768a00640 1 --2- 172.21.15.86:0/3938531675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb76415c2a0 0x7fb7640729e0 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.243+0000 7fb768a00640 1 --2- 172.21.15.86:0/3938531675 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb76406fd50 0x7fb76409a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.243+0000 7fb768a00640 1 -- 172.21.15.86:0/3938531675 >> 172.21.15.86:0/3938531675 conn(0x7fb7640b4650 msgr2=0x7fb76416ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:36.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.243+0000 7fb768a00640 1 -- 172.21.15.86:0/3938531675 shutdown_connections 2024-04-25T14:13:36.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.243+0000 7fb768a00640 1 -- 172.21.15.86:0/3938531675 wait complete. 2024-04-25T14:13:36.265 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T14:13:36.266 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T14:13:36.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.423+0000 7fcc85800640 1 Processor -- start 2024-04-25T14:13:36.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.423+0000 7fcc85800640 1 -- start start 2024-04-25T14:13:36.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.423+0000 7fcc85800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcc800cfca0 0x7fcc8006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:36.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.423+0000 7fcc85800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcc800d16e0 0x7fcc800d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:36.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.423+0000 7fcc85800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc800d2030 con 0x7fcc8006c670 2024-04-25T14:13:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.423+0000 7fcc85800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcc80070470 con 0x7fcc800cfca0 2024-04-25T14:13:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.423+0000 7fcc85800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fcc80072bd0 con 0x7fcc800d16e0 2024-04-25T14:13:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.423+0000 7fcc7ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcc800cfca0 0x7fcc8006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.423+0000 7fcc7f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fcc8006c670 0x7fcc8006ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40246/0 (socket says 172.21.15.86:40246) 2024-04-25T14:13:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.423+0000 7fcc7f400640 1 -- 172.21.15.86:0/3007503392 learned_addr learned my addr 172.21.15.86:0/3007503392 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.423+0000 7fcc7e000640 1 --2- 172.21.15.86:0/3007503392 >> v2:172.21.15.155:3300/0 conn(0x7fcc800d16e0 0x7fcc800d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.423+0000 7fcc7ea00640 1 -- 172.21.15.86:0/3007503392 >> v2:172.21.15.155:3300/0 conn(0x7fcc800d16e0 msgr2=0x7fcc800d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.423+0000 7fcc7ea00640 1 --2- 172.21.15.86:0/3007503392 >> v2:172.21.15.155:3300/0 conn(0x7fcc800d16e0 0x7fcc800d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.423+0000 7fcc7ea00640 1 -- 172.21.15.86:0/3007503392 >> v1:172.21.15.86:6789/0 conn(0x7fcc8006c670 legacy=0x7fcc8006ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.423+0000 7fcc7ea00640 1 -- 172.21.15.86:0/3007503392 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8006f930 con 0x7fcc800cfca0 2024-04-25T14:13:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.423+0000 7fcc7d600640 1 -- 172.21.15.86:0/3007503392 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3575870692 0 0) 0x7fcc780014a0 con 0x7fcc8006c670 2024-04-25T14:13:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.423+0000 7fcc7e000640 1 --2- 172.21.15.86:0/3007503392 >> v2:172.21.15.155:3300/0 conn(0x7fcc800d16e0 0x7fcc800d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:36.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.427+0000 7fcc7ea00640 1 --2- 172.21.15.86:0/3007503392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc800cfca0 0x7fcc8006f3f0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fcc800729c0 tx=0x7fcc7001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b133e01ec4bb279 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:36.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.427+0000 7fcc7d600640 1 -- 172.21.15.86:0/3007503392 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc700b1070 con 0x7fcc800cfca0 2024-04-25T14:13:36.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.427+0000 7fcc7d600640 1 -- 172.21.15.86:0/3007503392 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc700ac5f0 con 0x7fcc800cfca0 2024-04-25T14:13:36.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.427+0000 7fcc7d600640 1 -- 172.21.15.86:0/3007503392 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc700ac9c0 con 0x7fcc800cfca0 2024-04-25T14:13:36.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.427+0000 7fcc85800640 1 -- 172.21.15.86:0/3007503392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc800cfca0 msgr2=0x7fcc8006f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:36.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.427+0000 7fcc85800640 1 --2- 172.21.15.86:0/3007503392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc800cfca0 0x7fcc8006f3f0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fcc800729c0 tx=0x7fcc7001c480 comp rx=0 tx=0).stop 2024-04-25T14:13:36.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.427+0000 7fcc85800640 1 -- 172.21.15.86:0/3007503392 shutdown_connections 2024-04-25T14:13:36.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.427+0000 7fcc85800640 1 --2- 172.21.15.86:0/3007503392 >> v2:172.21.15.155:3300/0 conn(0x7fcc800d16e0 0x7fcc800d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.427+0000 7fcc85800640 1 --2- 172.21.15.86:0/3007503392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc800cfca0 0x7fcc8006f3f0 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.427+0000 7fcc85800640 1 -- 172.21.15.86:0/3007503392 >> 172.21.15.86:0/3007503392 conn(0x7fcc800b4650 msgr2=0x7fcc800b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:36.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.427+0000 7fcc85800640 1 -- 172.21.15.86:0/3007503392 shutdown_connections 2024-04-25T14:13:36.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.427+0000 7fcc85800640 1 -- 172.21.15.86:0/3007503392 wait complete. 2024-04-25T14:13:36.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.431+0000 7fcc85800640 1 Processor -- start 2024-04-25T14:13:36.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.431+0000 7fcc85800640 1 -- start start 2024-04-25T14:13:36.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.431+0000 7fcc85800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc8006c670 0x7fcc8009a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:36.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.431+0000 7fcc85800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc800cfca0 0x7fcc8015e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:36.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.431+0000 7fcc85800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc800d16e0 0x7fcc800af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:36.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.431+0000 7fcc85800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc800cdee0 con 0x7fcc800cfca0 2024-04-25T14:13:36.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.431+0000 7fcc85800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcc800cdd60 con 0x7fcc800d16e0 2024-04-25T14:13:36.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.431+0000 7fcc85800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcc800ce060 con 0x7fcc8006c670 2024-04-25T14:13:36.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.431+0000 7fcc7e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc800cfca0 0x7fcc8015e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:36.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.431+0000 7fcc7e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc800cfca0 0x7fcc8015e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55396/0 (socket says 172.21.15.86:55396) 2024-04-25T14:13:36.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.431+0000 7fcc7e000640 1 -- 172.21.15.86:0/2680058209 learned_addr learned my addr 172.21.15.86:0/2680058209 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:36.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.431+0000 7fcc7f400640 1 --2- 172.21.15.86:0/2680058209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc800d16e0 0x7fcc800af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:36.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.431+0000 7fcc7ea00640 1 --2- 172.21.15.86:0/2680058209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc8006c670 0x7fcc8009a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:36.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.431+0000 7fcc7e000640 1 -- 172.21.15.86:0/2680058209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc8006c670 msgr2=0x7fcc8009a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:36.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.431+0000 7fcc7e000640 1 --2- 172.21.15.86:0/2680058209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc8006c670 0x7fcc8009a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.431+0000 7fcc7e000640 1 -- 172.21.15.86:0/2680058209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc800d16e0 msgr2=0x7fcc800af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:36.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.431+0000 7fcc7e000640 1 --2- 172.21.15.86:0/2680058209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc800d16e0 0x7fcc800af690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:36.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.431+0000 7fcc7e000640 1 -- 172.21.15.86:0/2680058209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8009aeb0 con 0x7fcc800cfca0 2024-04-25T14:13:36.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.431+0000 7fcc7f400640 1 --2- 172.21.15.86:0/2680058209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc800d16e0 0x7fcc800af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:36.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.435+0000 7fcc7e000640 1 --2- 172.21.15.86:0/2680058209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc800cfca0 0x7fcc8015e660 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00c7f0 tx=0x7fcc6c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:36.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.435+0000 7fcc76a00640 1 -- 172.21.15.86:0/2680058209 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c015070 con 0x7fcc800cfca0 2024-04-25T14:13:36.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.435+0000 7fcc76a00640 1 -- 172.21.15.86:0/2680058209 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc6c004220 con 0x7fcc800cfca0 2024-04-25T14:13:36.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.435+0000 7fcc76a00640 1 -- 172.21.15.86:0/2680058209 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c003620 con 0x7fcc800cfca0 2024-04-25T14:13:36.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.435+0000 7fcc85800640 1 -- 172.21.15.86:0/2680058209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc801594a0 con 0x7fcc800cfca0 2024-04-25T14:13:36.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.435+0000 7fcc85800640 1 -- 172.21.15.86:0/2680058209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc80159a40 con 0x7fcc800cfca0 2024-04-25T14:13:36.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.435+0000 7fcc85800640 1 -- 172.21.15.86:0/2680058209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc8009f4b0 con 0x7fcc800cfca0 2024-04-25T14:13:36.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.435+0000 7fcc76a00640 1 -- 172.21.15.86:0/2680058209 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7fcc6c0039d0 con 0x7fcc800cfca0 2024-04-25T14:13:36.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.435+0000 7fcc76a00640 1 --2- 172.21.15.86:0/2680058209 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fcc6006f520 0x7fcc60071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:36.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.435+0000 7fcc76a00640 1 -- 172.21.15.86:0/2680058209 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6101+0+0 (secure 0 0 0) 0x7fcc6c08f7e0 con 0x7fcc800cfca0 2024-04-25T14:13:36.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.435+0000 7fcc7ea00640 1 --2- 172.21.15.86:0/2680058209 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fcc6006f520 0x7fcc60071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:36.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.439+0000 7fcc7ea00640 1 --2- 172.21.15.86:0/2680058209 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fcc6006f520 0x7fcc60071a30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcc700ae000 tx=0x7fcc70008040 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:36.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.443+0000 7fcc76a00640 1 -- 172.21.15.86:0/2680058209 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcc6c05c310 con 0x7fcc800cfca0 2024-04-25T14:13:36.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:36.631+0000 7fcc85800640 1 -- 172.21.15.86:0/2680058209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fcc80069d00 con 0x7fcc800cfca0 2024-04-25T14:13:37.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.243+0000 7fcc76a00640 1 -- 172.21.15.86:0/2680058209 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v117) ==== 122+0+0 (secure 0 0 0) 0x7fcc6c05bc10 con 0x7fcc800cfca0 2024-04-25T14:13:37.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.243+0000 7fcc85800640 1 -- 172.21.15.86:0/2680058209 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fcc6006f520 msgr2=0x7fcc60071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.243+0000 7fcc85800640 1 --2- 172.21.15.86:0/2680058209 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fcc6006f520 0x7fcc60071a30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcc700ae000 tx=0x7fcc70008040 comp rx=0 tx=0).stop 2024-04-25T14:13:37.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.243+0000 7fcc85800640 1 -- 172.21.15.86:0/2680058209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc800cfca0 msgr2=0x7fcc8015e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.243+0000 7fcc85800640 1 --2- 172.21.15.86:0/2680058209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc800cfca0 0x7fcc8015e660 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00c7f0 tx=0x7fcc6c00ccb0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.243+0000 7fcc85800640 1 -- 172.21.15.86:0/2680058209 shutdown_connections 2024-04-25T14:13:37.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.243+0000 7fcc85800640 1 --2- 172.21.15.86:0/2680058209 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fcc6006f520 0x7fcc60071a30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.243+0000 7fcc85800640 1 --2- 172.21.15.86:0/2680058209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc800d16e0 0x7fcc800af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.243+0000 7fcc85800640 1 --2- 172.21.15.86:0/2680058209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc800cfca0 0x7fcc8015e660 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.243+0000 7fcc85800640 1 --2- 172.21.15.86:0/2680058209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc8006c670 0x7fcc8009a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.243+0000 7fcc85800640 1 -- 172.21.15.86:0/2680058209 >> 172.21.15.86:0/2680058209 conn(0x7fcc800b4650 msgr2=0x7fcc800cbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:37.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.243+0000 7fcc85800640 1 -- 172.21.15.86:0/2680058209 shutdown_connections 2024-04-25T14:13:37.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.243+0000 7fcc85800640 1 -- 172.21.15.86:0/2680058209 wait complete. 2024-04-25T14:13:37.265 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T14:13:37.266 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:13:37.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.415+0000 7fce21400640 1 Processor -- start 2024-04-25T14:13:37.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.415+0000 7fce21400640 1 -- start start 2024-04-25T14:13:37.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.415+0000 7fce21400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fce1c0d2c80 0x7fce1c15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:37.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.415+0000 7fce21400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fce1c070e20 0x7fce1c071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:37.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.415+0000 7fce21400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce1c071770 con 0x7fce1c0722f0 2024-04-25T14:13:37.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.415+0000 7fce21400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fce1c0ce720 con 0x7fce1c070e20 2024-04-25T14:13:37.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.415+0000 7fce21400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fce1c0cd8d0 con 0x7fce1c0d2c80 2024-04-25T14:13:37.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.419+0000 7fce1b200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fce1c0722f0 0x7fce1c072700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40256/0 (socket says 172.21.15.86:40256) 2024-04-25T14:13:37.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.419+0000 7fce11e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fce1c070e20 0x7fce1c071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:37.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.419+0000 7fce1b200640 1 -- 172.21.15.86:0/3488897425 learned_addr learned my addr 172.21.15.86:0/3488897425 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:37.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.419+0000 7fce1a800640 1 --2- 172.21.15.86:0/3488897425 >> v2:172.21.15.155:3300/0 conn(0x7fce1c0d2c80 0x7fce1c15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:37.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.419+0000 7fce11e00640 1 -- 172.21.15.86:0/3488897425 >> v2:172.21.15.155:3300/0 conn(0x7fce1c0d2c80 msgr2=0x7fce1c15dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:37.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.419+0000 7fce11e00640 1 --2- 172.21.15.86:0/3488897425 >> v2:172.21.15.155:3300/0 conn(0x7fce1c0d2c80 0x7fce1c15dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.419+0000 7fce11e00640 1 -- 172.21.15.86:0/3488897425 >> v1:172.21.15.86:6789/0 conn(0x7fce1c0722f0 legacy=0x7fce1c072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.419+0000 7fce19e00640 1 -- 172.21.15.86:0/3488897425 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1765397037 0 0) 0x7fce14001120 con 0x7fce1c0722f0 2024-04-25T14:13:37.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.419+0000 7fce11e00640 1 -- 172.21.15.86:0/3488897425 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce1c0ce970 con 0x7fce1c070e20 2024-04-25T14:13:37.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.419+0000 7fce11e00640 1 --2- 172.21.15.86:0/3488897425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce1c070e20 0x7fce1c071230 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0d11a0 tx=0x7fce0401ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688925aa65ecd642 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:37.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.419+0000 7fce19e00640 1 -- 172.21.15.86:0/3488897425 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce040b5070 con 0x7fce1c070e20 2024-04-25T14:13:37.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.419+0000 7fce19e00640 1 -- 172.21.15.86:0/3488897425 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce040a8950 con 0x7fce1c070e20 2024-04-25T14:13:37.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.419+0000 7fce19e00640 1 -- 172.21.15.86:0/3488897425 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce040a8d20 con 0x7fce1c070e20 2024-04-25T14:13:37.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.423+0000 7fce21400640 1 -- 172.21.15.86:0/3488897425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce1c070e20 msgr2=0x7fce1c071230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.423+0000 7fce21400640 1 --2- 172.21.15.86:0/3488897425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce1c070e20 0x7fce1c071230 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0d11a0 tx=0x7fce0401ec50 comp rx=0 tx=0).stop 2024-04-25T14:13:37.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.423+0000 7fce21400640 1 -- 172.21.15.86:0/3488897425 shutdown_connections 2024-04-25T14:13:37.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.423+0000 7fce21400640 1 --2- 172.21.15.86:0/3488897425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce1c070e20 0x7fce1c071230 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.423+0000 7fce21400640 1 --2- 172.21.15.86:0/3488897425 >> v2:172.21.15.155:3300/0 conn(0x7fce1c0d2c80 0x7fce1c15dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.423+0000 7fce21400640 1 -- 172.21.15.86:0/3488897425 >> 172.21.15.86:0/3488897425 conn(0x7fce1c0cffc0 msgr2=0x7fce1c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:37.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.423+0000 7fce21400640 1 -- 172.21.15.86:0/3488897425 shutdown_connections 2024-04-25T14:13:37.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.423+0000 7fce21400640 1 -- 172.21.15.86:0/3488897425 wait complete. 2024-04-25T14:13:37.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.423+0000 7fce21400640 1 Processor -- start 2024-04-25T14:13:37.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.423+0000 7fce21400640 1 -- start start 2024-04-25T14:13:37.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.427+0000 7fce21400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce1c070e20 0x7fce1c0af240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:37.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.427+0000 7fce21400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce1c0722f0 0x7fce1c14fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:37.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.427+0000 7fce21400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce1c0d2c80 0x7fce1c0ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:37.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.427+0000 7fce21400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce1c0d21a0 con 0x7fce1c0722f0 2024-04-25T14:13:37.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.427+0000 7fce21400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fce1c0d1ec0 con 0x7fce1c070e20 2024-04-25T14:13:37.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.427+0000 7fce21400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fce1c0d2480 con 0x7fce1c0d2c80 2024-04-25T14:13:37.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.427+0000 7fce11e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce1c0722f0 0x7fce1c14fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:37.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.427+0000 7fce11e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce1c0722f0 0x7fce1c14fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55414/0 (socket says 172.21.15.86:55414) 2024-04-25T14:13:37.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.427+0000 7fce11e00640 1 -- 172.21.15.86:0/1933004771 learned_addr learned my addr 172.21.15.86:0/1933004771 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:37.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.427+0000 7fce11e00640 1 -- 172.21.15.86:0/1933004771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce1c0d2c80 msgr2=0x7fce1c0ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.427+0000 7fce1b200640 1 --2- 172.21.15.86:0/1933004771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce1c0d2c80 0x7fce1c0ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:37.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.427+0000 7fce11e00640 1 --2- 172.21.15.86:0/1933004771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce1c0d2c80 0x7fce1c0ac8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.427+0000 7fce11e00640 1 -- 172.21.15.86:0/1933004771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce1c070e20 msgr2=0x7fce1c0af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.427+0000 7fce1a800640 1 --2- 172.21.15.86:0/1933004771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce1c070e20 0x7fce1c0af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:37.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.427+0000 7fce11e00640 1 --2- 172.21.15.86:0/1933004771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce1c070e20 0x7fce1c0af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.427+0000 7fce11e00640 1 -- 172.21.15.86:0/1933004771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce1c0ab800 con 0x7fce1c0722f0 2024-04-25T14:13:37.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.427+0000 7fce1b200640 1 --2- 172.21.15.86:0/1933004771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce1c0d2c80 0x7fce1c0ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:37.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.427+0000 7fce1a800640 1 --2- 172.21.15.86:0/1933004771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce1c070e20 0x7fce1c0af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:37.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.427+0000 7fce11e00640 1 --2- 172.21.15.86:0/1933004771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce1c0722f0 0x7fce1c14fb20 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fce0401c880 tx=0x7fce0401b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:37.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.427+0000 7fce13400640 1 -- 172.21.15.86:0/1933004771 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce040b5040 con 0x7fce1c0722f0 2024-04-25T14:13:37.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.427+0000 7fce13400640 1 -- 172.21.15.86:0/1933004771 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce04005ba0 con 0x7fce1c0722f0 2024-04-25T14:13:37.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.427+0000 7fce13400640 1 -- 172.21.15.86:0/1933004771 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce04006020 con 0x7fce1c0722f0 2024-04-25T14:13:37.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.427+0000 7fce21400640 1 -- 172.21.15.86:0/1933004771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce1c0abae0 con 0x7fce1c0722f0 2024-04-25T14:13:37.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.427+0000 7fce21400640 1 -- 172.21.15.86:0/1933004771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce1c0ac0a0 con 0x7fce1c0722f0 2024-04-25T14:13:37.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.427+0000 7fce21400640 1 -- 172.21.15.86:0/1933004771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce1c0720e0 con 0x7fce1c0722f0 2024-04-25T14:13:37.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.431+0000 7fce13400640 1 -- 172.21.15.86:0/1933004771 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7fce040061c0 con 0x7fce1c0722f0 2024-04-25T14:13:37.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.431+0000 7fce13400640 1 --2- 172.21.15.86:0/1933004771 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fcdf806f550 0x7fcdf8071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:37.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.431+0000 7fce13400640 1 -- 172.21.15.86:0/1933004771 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6101+0+0 (secure 0 0 0) 0x7fce0412c330 con 0x7fce1c0722f0 2024-04-25T14:13:37.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.431+0000 7fce1a800640 1 --2- 172.21.15.86:0/1933004771 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fcdf806f550 0x7fcdf8071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:37.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.431+0000 7fce1a800640 1 --2- 172.21.15.86:0/1933004771 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fcdf806f550 0x7fcdf8071a60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fce0c009f50 tx=0x7fce0c00b040 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:37.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.435+0000 7fce13400640 1 -- 172.21.15.86:0/1933004771 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fce040187b0 con 0x7fce1c0722f0 2024-04-25T14:13:37.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.623+0000 7fce21400640 1 -- 172.21.15.86:0/1933004771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce1c0c9fa0 con 0x7fce1c0722f0 2024-04-25T14:13:37.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.623+0000 7fce13400640 1 -- 172.21.15.86:0/1933004771 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) ==== 95+0+5202 (secure 0 0 0) 0x7fce040f8f80 con 0x7fce1c0722f0 2024-04-25T14:13:37.623 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:13:37.623 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":117,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112},{"gid":8387,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:13:36.638299+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10209},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:13:37.623 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 117 2024-04-25T14:13:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.627+0000 7fce21400640 1 -- 172.21.15.86:0/1933004771 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fcdf806f550 msgr2=0x7fcdf8071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.627+0000 7fce21400640 1 --2- 172.21.15.86:0/1933004771 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fcdf806f550 0x7fcdf8071a60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fce0c009f50 tx=0x7fce0c00b040 comp rx=0 tx=0).stop 2024-04-25T14:13:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.627+0000 7fce21400640 1 -- 172.21.15.86:0/1933004771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce1c0722f0 msgr2=0x7fce1c14fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.627+0000 7fce21400640 1 --2- 172.21.15.86:0/1933004771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce1c0722f0 0x7fce1c14fb20 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fce0401c880 tx=0x7fce0401b9f0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.627+0000 7fce21400640 1 -- 172.21.15.86:0/1933004771 shutdown_connections 2024-04-25T14:13:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.627+0000 7fce21400640 1 --2- 172.21.15.86:0/1933004771 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fcdf806f550 0x7fcdf8071a60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.627+0000 7fce21400640 1 --2- 172.21.15.86:0/1933004771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce1c0d2c80 0x7fce1c0ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.627+0000 7fce21400640 1 --2- 172.21.15.86:0/1933004771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce1c0722f0 0x7fce1c14fb20 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.627+0000 7fce21400640 1 --2- 172.21.15.86:0/1933004771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce1c070e20 0x7fce1c0af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.627+0000 7fce21400640 1 -- 172.21.15.86:0/1933004771 >> 172.21.15.86:0/1933004771 conn(0x7fce1c0cffc0 msgr2=0x7fce1c0b4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:37.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.627+0000 7fce21400640 1 -- 172.21.15.86:0/1933004771 shutdown_connections 2024-04-25T14:13:37.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.627+0000 7fce21400640 1 -- 172.21.15.86:0/1933004771 wait complete. 2024-04-25T14:13:37.645 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:13:37.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.791+0000 7f54ab000640 1 Processor -- start 2024-04-25T14:13:37.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.791+0000 7f54ab000640 1 -- start start 2024-04-25T14:13:37.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54ab000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f54a4070330 0x7f54a40d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:37.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54ab000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54a40d2030 con 0x7f54a40d1c20 2024-04-25T14:13:37.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54ab000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54a415e5c0 con 0x7f54a40ce790 2024-04-25T14:13:37.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54ab000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f54a415e8a0 con 0x7f54a4070330 2024-04-25T14:13:37.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f549b400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f54a40d1c20 0x7f54a406ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40272/0 (socket says 172.21.15.86:40272) 2024-04-25T14:13:37.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f549b400640 1 -- 172.21.15.86:0/4240875679 learned_addr learned my addr 172.21.15.86:0/4240875679 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:37.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54a3e00640 1 --2- 172.21.15.86:0/4240875679 >> v2:172.21.15.155:3300/0 conn(0x7f54a4070330 0x7f54a40d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:37.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54a3400640 1 -- 172.21.15.86:0/4240875679 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3005490044 0 0) 0x7f548c001120 con 0x7f54a40d1c20 2024-04-25T14:13:37.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54a3400640 1 -- 172.21.15.86:0/4240875679 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5488003530 con 0x7f54a40d1c20 2024-04-25T14:13:37.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54a3400640 1 -- 172.21.15.86:0/4240875679 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 288034421 0 0) 0x7f54a415e5c0 con 0x7f54a40ce790 2024-04-25T14:13:37.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54a3400640 1 -- 172.21.15.86:0/4240875679 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f548c001120 con 0x7f54a40ce790 2024-04-25T14:13:37.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54a3400640 1 -- 172.21.15.86:0/4240875679 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3355191978 0 0) 0x7f5488003530 con 0x7f54a40d1c20 2024-04-25T14:13:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54a3400640 1 -- 172.21.15.86:0/4240875679 >> v2:172.21.15.155:3300/0 conn(0x7f54a4070330 msgr2=0x7f54a40d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54a3400640 1 --2- 172.21.15.86:0/4240875679 >> v2:172.21.15.155:3300/0 conn(0x7f54a4070330 0x7f54a40d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54a3400640 1 -- 172.21.15.86:0/4240875679 >> v1:172.21.15.155:6789/0 conn(0x7f54a40ce790 legacy=0x7f54a406f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54a3400640 1 -- 172.21.15.86:0/4240875679 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a415eb80 con 0x7f54a40d1c20 2024-04-25T14:13:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54a3400640 1 -- 172.21.15.86:0/4240875679 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f548c001850 con 0x7f54a40d1c20 2024-04-25T14:13:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54a3400640 1 -- 172.21.15.86:0/4240875679 >> v1:172.21.15.86:6789/0 conn(0x7f54a40d1c20 legacy=0x7f54a406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54a3400640 1 --2- 172.21.15.86:0/4240875679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5488003bc0 0x7f5488004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54a3400640 1 --2- 172.21.15.86:0/4240875679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5488004810 0x7f5488004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54a3400640 1 --2- 172.21.15.86:0/4240875679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5488005330 0x7f548800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54a3400640 1 -- 172.21.15.86:0/4240875679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f54a415e8a0 con 0x7f5488005330 2024-04-25T14:13:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54a3400640 1 -- 172.21.15.86:0/4240875679 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f548800c260 con 0x7f5488004810 2024-04-25T14:13:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54a3400640 1 -- 172.21.15.86:0/4240875679 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f548800c460 con 0x7f5488003bc0 2024-04-25T14:13:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54a9000640 1 --2- 172.21.15.86:0/4240875679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5488005330 0x7f548800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f549b400640 1 --2- 172.21.15.86:0/4240875679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5488004810 0x7f5488004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:37.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54a3e00640 1 --2- 172.21.15.86:0/4240875679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5488003bc0 0x7f5488004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:37.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54a9000640 1 -- 172.21.15.86:0/4240875679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5488003bc0 msgr2=0x7f5488004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:37.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54a9000640 1 --2- 172.21.15.86:0/4240875679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5488003bc0 0x7f5488004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54a9000640 1 -- 172.21.15.86:0/4240875679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5488004810 msgr2=0x7f5488004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:37.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54a9000640 1 --2- 172.21.15.86:0/4240875679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5488004810 0x7f5488004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54a9000640 1 -- 172.21.15.86:0/4240875679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5488003530 con 0x7f5488005330 2024-04-25T14:13:37.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f549b400640 1 --2- 172.21.15.86:0/4240875679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5488004810 0x7f5488004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:37.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.795+0000 7f54a3e00640 1 --2- 172.21.15.86:0/4240875679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5488003bc0 0x7f5488004010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:37.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.799+0000 7f54a9000640 1 -- 172.21.15.86:0/4240875679 reap_dead start 2024-04-25T14:13:37.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.799+0000 7f54a9000640 1 --2- 172.21.15.86:0/4240875679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5488005330 0x7f548800b9d0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f54a40cb9a0 tx=0x7f549c01f050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96fe99abee65126 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.799+0000 7f54a3400640 1 -- 172.21.15.86:0/4240875679 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c001c20 con 0x7f5488005330 2024-04-25T14:13:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.799+0000 7f54a3400640 1 -- 172.21.15.86:0/4240875679 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f549c0a8c70 con 0x7f5488005330 2024-04-25T14:13:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.799+0000 7f54a3400640 1 -- 172.21.15.86:0/4240875679 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c003bc0 con 0x7f5488005330 2024-04-25T14:13:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.799+0000 7f54ab000640 1 -- 172.21.15.86:0/4240875679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5488005330 msgr2=0x7f548800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.799+0000 7f54ab000640 1 --2- 172.21.15.86:0/4240875679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5488005330 0x7f548800b9d0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f54a40cb9a0 tx=0x7f549c01f050 comp rx=0 tx=0).stop 2024-04-25T14:13:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.799+0000 7f54ab000640 1 -- 172.21.15.86:0/4240875679 shutdown_connections 2024-04-25T14:13:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.799+0000 7f54ab000640 1 --2- 172.21.15.86:0/4240875679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5488005330 0x7f548800b9d0 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.799+0000 7f54ab000640 1 -- 172.21.15.86:0/4240875679 >> 172.21.15.86:0/4240875679 conn(0x7f54a40b4650 msgr2=0x7f54a40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.799+0000 7f54ab000640 1 -- 172.21.15.86:0/4240875679 shutdown_connections 2024-04-25T14:13:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.799+0000 7f54ab000640 1 -- 172.21.15.86:0/4240875679 wait complete. 2024-04-25T14:13:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.803+0000 7f54ab000640 1 Processor -- start 2024-04-25T14:13:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.803+0000 7f54ab000640 1 -- start start 2024-04-25T14:13:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.803+0000 7f54ab000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5488005330 0x7f54a4156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.803+0000 7f54ab000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54a4159af0 0x7f54a4159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.803+0000 7f54ab000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54a406ec30 0x7f54a406f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.803+0000 7f54ab000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f54a415f330 con 0x7f54a406ec30 2024-04-25T14:13:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.803+0000 7f54ab000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f54a406d170 con 0x7f5488005330 2024-04-25T14:13:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.803+0000 7f54ab000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f54a415f4b0 con 0x7f54a4159af0 2024-04-25T14:13:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.803+0000 7f54a9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54a406ec30 0x7f54a406f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.803+0000 7f54a9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54a406ec30 0x7f54a406f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55440/0 (socket says 172.21.15.86:55440) 2024-04-25T14:13:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.803+0000 7f54a9000640 1 -- 172.21.15.86:0/1091387196 learned_addr learned my addr 172.21.15.86:0/1091387196 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.803+0000 7f54a3e00640 1 --2- 172.21.15.86:0/1091387196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5488005330 0x7f54a4156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.803+0000 7f549b400640 1 --2- 172.21.15.86:0/1091387196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54a4159af0 0x7f54a4159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.803+0000 7f54a9000640 1 -- 172.21.15.86:0/1091387196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54a4159af0 msgr2=0x7f54a4159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.803+0000 7f54a9000640 1 --2- 172.21.15.86:0/1091387196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54a4159af0 0x7f54a4159f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.803+0000 7f54a9000640 1 -- 172.21.15.86:0/1091387196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5488005330 msgr2=0x7f54a4156e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.803+0000 7f54a9000640 1 --2- 172.21.15.86:0/1091387196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5488005330 0x7f54a4156e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.803+0000 7f54a9000640 1 -- 172.21.15.86:0/1091387196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a406f580 con 0x7f54a406ec30 2024-04-25T14:13:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.803+0000 7f54a9000640 1 --2- 172.21.15.86:0/1091387196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54a406ec30 0x7f54a406f040 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f549c01ee80 tx=0x7f5488005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.803+0000 7f54a0c00640 1 -- 172.21.15.86:0/1091387196 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c0a8a70 con 0x7f54a406ec30 2024-04-25T14:13:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.807+0000 7f54a0c00640 1 -- 172.21.15.86:0/1091387196 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f549c003980 con 0x7f54a406ec30 2024-04-25T14:13:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.807+0000 7f54a0c00640 1 -- 172.21.15.86:0/1091387196 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c003d30 con 0x7f54a406ec30 2024-04-25T14:13:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.807+0000 7f54ab000640 1 -- 172.21.15.86:0/1091387196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a4158290 con 0x7f54a406ec30 2024-04-25T14:13:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.807+0000 7f54ab000640 1 -- 172.21.15.86:0/1091387196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a4158850 con 0x7f54a406ec30 2024-04-25T14:13:37.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.807+0000 7f54ab000640 1 -- 172.21.15.86:0/1091387196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54a4069d00 con 0x7f54a406ec30 2024-04-25T14:13:37.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.807+0000 7f54a0c00640 1 -- 172.21.15.86:0/1091387196 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7f549c0050d0 con 0x7f54a406ec30 2024-04-25T14:13:37.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.807+0000 7f54a0c00640 1 --2- 172.21.15.86:0/1091387196 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f548006f4c0 0x7f54800719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:37.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.807+0000 7f54a0c00640 1 -- 172.21.15.86:0/1091387196 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6101+0+0 (secure 0 0 0) 0x7f549c12c150 con 0x7f54a406ec30 2024-04-25T14:13:37.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.807+0000 7f54a3e00640 1 --2- 172.21.15.86:0/1091387196 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f548006f4c0 0x7f54800719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:37.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.811+0000 7f54a3e00640 1 --2- 172.21.15.86:0/1091387196 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f548006f4c0 0x7f54800719d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5494006f90 tx=0x7f5494008040 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:37.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.811+0000 7f54a0c00640 1 -- 172.21.15.86:0/1091387196 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f549c0c1030 con 0x7f54a406ec30 2024-04-25T14:13:37.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.971+0000 7f54ab000640 1 -- 172.21.15.86:0/1091387196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f54a40d0ed0 con 0x7f54a406ec30 2024-04-25T14:13:37.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.971+0000 7f54a0c00640 1 -- 172.21.15.86:0/1091387196 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) ==== 74+0+16884 (secure 0 0 0) 0x7f54a406f580 con 0x7f54a406ec30 2024-04-25T14:13:37.971 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:13:37.971 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":155,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:13:35.224415+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:13:32.006877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T14:13:32.574088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:13:37.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.975+0000 7f54ab000640 1 -- 172.21.15.86:0/1091387196 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f548006f4c0 msgr2=0x7f54800719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.975+0000 7f54ab000640 1 --2- 172.21.15.86:0/1091387196 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f548006f4c0 0x7f54800719d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5494006f90 tx=0x7f5494008040 comp rx=0 tx=0).stop 2024-04-25T14:13:37.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.975+0000 7f54ab000640 1 -- 172.21.15.86:0/1091387196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54a406ec30 msgr2=0x7f54a406f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:37.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.975+0000 7f54ab000640 1 --2- 172.21.15.86:0/1091387196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54a406ec30 0x7f54a406f040 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f549c01ee80 tx=0x7f5488005840 comp rx=0 tx=0).stop 2024-04-25T14:13:37.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.975+0000 7f54ab000640 1 -- 172.21.15.86:0/1091387196 shutdown_connections 2024-04-25T14:13:37.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.975+0000 7f54ab000640 1 --2- 172.21.15.86:0/1091387196 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f548006f4c0 0x7f54800719d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.975+0000 7f54ab000640 1 --2- 172.21.15.86:0/1091387196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54a406ec30 0x7f54a406f040 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.975+0000 7f54ab000640 1 --2- 172.21.15.86:0/1091387196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54a4159af0 0x7f54a4159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.975+0000 7f54ab000640 1 --2- 172.21.15.86:0/1091387196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5488005330 0x7f54a4156e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:37.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.975+0000 7f54ab000640 1 -- 172.21.15.86:0/1091387196 >> 172.21.15.86:0/1091387196 conn(0x7f54a40b4650 msgr2=0x7f54a40b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:37.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.975+0000 7f54ab000640 1 -- 172.21.15.86:0/1091387196 shutdown_connections 2024-04-25T14:13:37.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:37.975+0000 7f54ab000640 1 -- 172.21.15.86:0/1091387196 wait complete. 2024-04-25T14:13:37.994 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T14:13:37.994 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:13:38.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.143+0000 7f92e6c00640 1 Processor -- start 2024-04-25T14:13:38.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.143+0000 7f92e6c00640 1 -- start start 2024-04-25T14:13:38.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.143+0000 7f92e6c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f92e00d2ae0 0x7f92e00ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.143+0000 7f92e6c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92e00b3bd0 con 0x7f92e00ccab0 2024-04-25T14:13:38.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.143+0000 7f92e6c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92e00b1cf0 con 0x7f92e00b79c0 2024-04-25T14:13:38.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.143+0000 7f92e6c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f92e00b2620 con 0x7f92e00d2ae0 2024-04-25T14:13:38.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.143+0000 7f92dfe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f92e00ccab0 0x7f92e00d23d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40282/0 (socket says 172.21.15.86:40282) 2024-04-25T14:13:38.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.143+0000 7f92dfe00640 1 -- 172.21.15.86:0/2653163488 learned_addr learned my addr 172.21.15.86:0/2653163488 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:38.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.143+0000 7f92df400640 1 --2- 172.21.15.86:0/2653163488 >> v2:172.21.15.155:3300/0 conn(0x7f92e00d2ae0 0x7f92e00ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.147+0000 7f92dea00640 1 -- 172.21.15.86:0/2653163488 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2405664398 0 0) 0x7f92e00b3bd0 con 0x7f92e00ccab0 2024-04-25T14:13:38.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.147+0000 7f92dea00640 1 -- 172.21.15.86:0/2653163488 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92c4003530 con 0x7f92e00ccab0 2024-04-25T14:13:38.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.147+0000 7f92df400640 1 -- 172.21.15.86:0/2653163488 >> v1:172.21.15.155:6789/0 conn(0x7f92e00b79c0 legacy=0x7f92e00ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.147+0000 7f92df400640 1 -- 172.21.15.86:0/2653163488 >> v1:172.21.15.86:6789/0 conn(0x7f92e00ccab0 legacy=0x7f92e00d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.147+0000 7f92dea00640 1 -- 172.21.15.86:0/2653163488 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4007736078 0 0) 0x7f92c4003530 con 0x7f92e00ccab0 2024-04-25T14:13:38.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.147+0000 7f92df400640 1 -- 172.21.15.86:0/2653163488 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e00b1fd0 con 0x7f92e00d2ae0 2024-04-25T14:13:38.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.147+0000 7f92df400640 1 --2- 172.21.15.86:0/2653163488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92e00d2ae0 0x7f92e00ce180 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f92cc0097c0 tx=0x7f92cc01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39c24d609fc3bf96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:38.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.147+0000 7f92dea00640 1 -- 172.21.15.86:0/2653163488 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc0b0070 con 0x7f92e00d2ae0 2024-04-25T14:13:38.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.147+0000 7f92dea00640 1 -- 172.21.15.86:0/2653163488 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92cc0ab400 con 0x7f92e00d2ae0 2024-04-25T14:13:38.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.147+0000 7f92dea00640 1 -- 172.21.15.86:0/2653163488 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d0001800 con 0x7f92e00d2ae0 2024-04-25T14:13:38.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.151+0000 7f92e6c00640 1 -- 172.21.15.86:0/2653163488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92e00d2ae0 msgr2=0x7f92e00ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.151+0000 7f92e6c00640 1 --2- 172.21.15.86:0/2653163488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92e00d2ae0 0x7f92e00ce180 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f92cc0097c0 tx=0x7f92cc01c3c0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.151+0000 7f92e6c00640 1 -- 172.21.15.86:0/2653163488 shutdown_connections 2024-04-25T14:13:38.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.151+0000 7f92e6c00640 1 --2- 172.21.15.86:0/2653163488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92e00d2ae0 0x7f92e00ce180 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.151+0000 7f92e6c00640 1 -- 172.21.15.86:0/2653163488 >> 172.21.15.86:0/2653163488 conn(0x7f92e009ba30 msgr2=0x7f92e00b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:38.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.151+0000 7f92e6c00640 1 -- 172.21.15.86:0/2653163488 shutdown_connections 2024-04-25T14:13:38.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.151+0000 7f92e6c00640 1 -- 172.21.15.86:0/2653163488 wait complete. 2024-04-25T14:13:38.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.151+0000 7f92e6c00640 1 Processor -- start 2024-04-25T14:13:38.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.151+0000 7f92e6c00640 1 -- start start 2024-04-25T14:13:38.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.155+0000 7f92e6c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92e00b79c0 0x7f92e0157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.155+0000 7f92e6c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92e00ccab0 0x7f92e0156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.155+0000 7f92e6c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92e0158140 0x7f92e01565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.155+0000 7f92e6c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92e016c530 con 0x7f92e0158140 2024-04-25T14:13:38.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.155+0000 7f92e6c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f92e016c3b0 con 0x7f92e00ccab0 2024-04-25T14:13:38.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.155+0000 7f92e6c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f92e016c6b0 con 0x7f92e00b79c0 2024-04-25T14:13:38.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.155+0000 7f92e4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92e0158140 0x7f92e01565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.155+0000 7f92e4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92e0158140 0x7f92e01565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55444/0 (socket says 172.21.15.86:55444) 2024-04-25T14:13:38.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.155+0000 7f92e4c00640 1 -- 172.21.15.86:0/1353526636 learned_addr learned my addr 172.21.15.86:0/1353526636 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:38.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.155+0000 7f92dfe00640 1 --2- 172.21.15.86:0/1353526636 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92e00b79c0 0x7f92e0157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.155+0000 7f92df400640 1 --2- 172.21.15.86:0/1353526636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92e00ccab0 0x7f92e0156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.155+0000 7f92e4c00640 1 -- 172.21.15.86:0/1353526636 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92e00b79c0 msgr2=0x7f92e0157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.155+0000 7f92e4c00640 1 --2- 172.21.15.86:0/1353526636 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92e00b79c0 0x7f92e0157c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.155+0000 7f92e4c00640 1 -- 172.21.15.86:0/1353526636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92e00ccab0 msgr2=0x7f92e0156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.155+0000 7f92e4c00640 1 --2- 172.21.15.86:0/1353526636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92e00ccab0 0x7f92e0156080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.155+0000 7f92e4c00640 1 -- 172.21.15.86:0/1353526636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e01544b0 con 0x7f92e0158140 2024-04-25T14:13:38.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.155+0000 7f92e4c00640 1 --2- 172.21.15.86:0/1353526636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92e0158140 0x7f92e01565c0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f92d800c760 tx=0x7f92d800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:38.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.155+0000 7f92d7e00640 1 -- 172.21.15.86:0/1353526636 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8003b80 con 0x7f92e0158140 2024-04-25T14:13:38.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.155+0000 7f92d7e00640 1 -- 172.21.15.86:0/1353526636 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d8003d20 con 0x7f92e0158140 2024-04-25T14:13:38.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.155+0000 7f92e6c00640 1 -- 172.21.15.86:0/1353526636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92e0154790 con 0x7f92e0158140 2024-04-25T14:13:38.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.155+0000 7f92d7e00640 1 -- 172.21.15.86:0/1353526636 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d800aa30 con 0x7f92e0158140 2024-04-25T14:13:38.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.155+0000 7f92e6c00640 1 -- 172.21.15.86:0/1353526636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92e0154d50 con 0x7f92e0158140 2024-04-25T14:13:38.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.155+0000 7f92e6c00640 1 -- 172.21.15.86:0/1353526636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92e0169360 con 0x7f92e0158140 2024-04-25T14:13:38.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.159+0000 7f92d7e00640 1 -- 172.21.15.86:0/1353526636 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7f92d800abd0 con 0x7f92e0158140 2024-04-25T14:13:38.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.159+0000 7f92d7e00640 1 --2- 172.21.15.86:0/1353526636 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f92c006f520 0x7f92c0071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.159+0000 7f92d7e00640 1 -- 172.21.15.86:0/1353526636 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6101+0+0 (secure 0 0 0) 0x7f92d808f440 con 0x7f92e0158140 2024-04-25T14:13:38.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.159+0000 7f92dfe00640 1 --2- 172.21.15.86:0/1353526636 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f92c006f520 0x7f92c0071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.159+0000 7f92dfe00640 1 --2- 172.21.15.86:0/1353526636 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f92c006f520 0x7f92c0071a30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f92d0001970 tx=0x7f92d00017e0 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.163+0000 7f92d7e00640 1 -- 172.21.15.86:0/1353526636 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f92d805b9a0 con 0x7f92e0158140 2024-04-25T14:13:38.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.319+0000 7f92e6c00640 1 -- 172.21.15.86:0/1353526636 --> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f92e00d2f90 con 0x7f92c006f520 2024-04-25T14:13:38.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.323+0000 7f92d7e00640 1 -- 172.21.15.86:0/1353526636 <== mgr.8198 v2:172.21.15.86:6832/505730993 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247100 (secure 0 0 0) 0x7f92e00d2f90 con 0x7f92c006f520 2024-04-25T14:13:38.327 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:13:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:13:38.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.327+0000 7f92e6c00640 1 -- 172.21.15.86:0/1353526636 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f92c006f520 msgr2=0x7f92c0071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.327+0000 7f92e6c00640 1 --2- 172.21.15.86:0/1353526636 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f92c006f520 0x7f92c0071a30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f92d0001970 tx=0x7f92d00017e0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.327+0000 7f92e6c00640 1 -- 172.21.15.86:0/1353526636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92e0158140 msgr2=0x7f92e01565c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.327+0000 7f92e6c00640 1 --2- 172.21.15.86:0/1353526636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92e0158140 0x7f92e01565c0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f92d800c760 tx=0x7f92d800cc20 comp rx=0 tx=0).stop 2024-04-25T14:13:38.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.327+0000 7f92e6c00640 1 -- 172.21.15.86:0/1353526636 shutdown_connections 2024-04-25T14:13:38.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.327+0000 7f92e6c00640 1 --2- 172.21.15.86:0/1353526636 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f92c006f520 0x7f92c0071a30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.327+0000 7f92e6c00640 1 --2- 172.21.15.86:0/1353526636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92e0158140 0x7f92e01565c0 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.327+0000 7f92e6c00640 1 --2- 172.21.15.86:0/1353526636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92e00ccab0 0x7f92e0156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.327+0000 7f92e6c00640 1 --2- 172.21.15.86:0/1353526636 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92e00b79c0 0x7f92e0157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.327+0000 7f92e6c00640 1 -- 172.21.15.86:0/1353526636 >> 172.21.15.86:0/1353526636 conn(0x7f92e009ba30 msgr2=0x7f92e0068620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:38.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.327+0000 7f92e6c00640 1 -- 172.21.15.86:0/1353526636 shutdown_connections 2024-04-25T14:13:38.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.327+0000 7f92e6c00640 1 -- 172.21.15.86:0/1353526636 wait complete. 2024-04-25T14:13:38.347 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T14:13:38.077493+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":837,"num_read_kb":2279,"num_write":247,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3291,"ondisk_log_size":3291,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257340,"kb_used_data":16188,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754717380,"statfs":{"total":773094113280,"available":772830597120,"internally_reserved":0,"allocated":16576512,"data_stored":8900589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.870692"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.217839+0000","last_change":"2024-04-25T14:13:33.217839+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.217839+0000","last_undegraded":"2024-04-25T14:13:33.217839+0000","last_fullsized":"2024-04-25T14:13:33.217839+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.220477+0000","last_change":"2024-04-25T14:13:33.220477+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.220477+0000","last_undegraded":"2024-04-25T14:13:33.220477+0000","last_fullsized":"2024-04-25T14:13:33.220477+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.216321+0000","last_change":"2024-04-25T14:13:33.216321+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.216321+0000","last_undegraded":"2024-04-25T14:13:33.216321+0000","last_fullsized":"2024-04-25T14:13:33.216321+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.216741+0000","last_change":"2024-04-25T14:13:33.216741+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.216741+0000","last_undegraded":"2024-04-25T14:13:33.216741+0000","last_fullsized":"2024-04-25T14:13:33.216741+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219621+0000","last_change":"2024-04-25T14:13:33.219621+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219621+0000","last_undegraded":"2024-04-25T14:13:33.219621+0000","last_fullsized":"2024-04-25T14:13:33.219621+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.221767+0000","last_change":"2024-04-25T14:13:33.221767+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.221767+0000","last_undegraded":"2024-04-25T14:13:33.221767+0000","last_fullsized":"2024-04-25T14:13:33.221767+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.223617+0000","last_change":"2024-04-25T14:13:33.223617+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.223617+0000","last_undegraded":"2024-04-25T14:13:33.223617+0000","last_fullsized":"2024-04-25T14:13:33.223617+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.228099+0000","last_change":"2024-04-25T14:13:33.228099+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.228099+0000","last_undegraded":"2024-04-25T14:13:33.228099+0000","last_fullsized":"2024-04-25T14:13:33.228099+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.222924+0000","last_change":"2024-04-25T14:13:33.222924+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.222924+0000","last_undegraded":"2024-04-25T14:13:33.222924+0000","last_fullsized":"2024-04-25T14:13:33.222924+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.229249+0000","last_change":"2024-04-25T14:13:33.229249+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.229249+0000","last_undegraded":"2024-04-25T14:13:33.229249+0000","last_fullsized":"2024-04-25T14:13:33.229249+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.231347+0000","last_change":"2024-04-25T14:13:33.231347+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.231347+0000","last_undegraded":"2024-04-25T14:13:33.231347+0000","last_fullsized":"2024-04-25T14:13:33.231347+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.222546+0000","last_change":"2024-04-25T14:13:33.222546+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.222546+0000","last_undegraded":"2024-04-25T14:13:33.222546+0000","last_fullsized":"2024-04-25T14:13:33.222546+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.224550+0000","last_change":"2024-04-25T14:13:33.224550+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.224550+0000","last_undegraded":"2024-04-25T14:13:33.224550+0000","last_fullsized":"2024-04-25T14:13:33.224550+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219360+0000","last_change":"2024-04-25T14:13:33.219360+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219360+0000","last_undegraded":"2024-04-25T14:13:33.219360+0000","last_fullsized":"2024-04-25T14:13:33.219360+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.229250+0000","last_change":"2024-04-25T14:13:33.229250+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.229250+0000","last_undegraded":"2024-04-25T14:13:33.229250+0000","last_fullsized":"2024-04-25T14:13:33.229250+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.216778+0000","last_change":"2024-04-25T14:13:33.216778+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.216778+0000","last_undegraded":"2024-04-25T14:13:33.216778+0000","last_fullsized":"2024-04-25T14:13:33.216778+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.222484+0000","last_change":"2024-04-25T14:13:33.222484+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.222484+0000","last_undegraded":"2024-04-25T14:13:33.222484+0000","last_fullsized":"2024-04-25T14:13:33.222484+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.225152+0000","last_change":"2024-04-25T14:13:33.225152+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.225152+0000","last_undegraded":"2024-04-25T14:13:33.225152+0000","last_fullsized":"2024-04-25T14:13:33.225152+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.228898+0000","last_change":"2024-04-25T14:13:33.228898+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.228898+0000","last_undegraded":"2024-04-25T14:13:33.228898+0000","last_fullsized":"2024-04-25T14:13:33.228898+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.229071+0000","last_change":"2024-04-25T14:13:33.229071+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.229071+0000","last_undegraded":"2024-04-25T14:13:33.229071+0000","last_fullsized":"2024-04-25T14:13:33.229071+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.224104+0000","last_change":"2024-04-25T14:13:33.224104+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.224104+0000","last_undegraded":"2024-04-25T14:13:33.224104+0000","last_fullsized":"2024-04-25T14:13:33.224104+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.221177+0000","last_change":"2024-04-25T14:13:33.221177+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.221177+0000","last_undegraded":"2024-04-25T14:13:33.221177+0000","last_fullsized":"2024-04-25T14:13:33.221177+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.218713+0000","last_change":"2024-04-25T14:13:33.218713+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.218713+0000","last_undegraded":"2024-04-25T14:13:33.218713+0000","last_fullsized":"2024-04-25T14:13:33.218713+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.215974+0000","last_change":"2024-04-25T14:13:33.215974+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.215974+0000","last_undegraded":"2024-04-25T14:13:33.215974+0000","last_fullsized":"2024-04-25T14:13:33.215974+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.222118+0000","last_change":"2024-04-25T14:13:33.222118+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.222118+0000","last_undegraded":"2024-04-25T14:13:33.222118+0000","last_fullsized":"2024-04-25T14:13:33.222118+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.228839+0000","last_change":"2024-04-25T14:13:33.228839+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.228839+0000","last_undegraded":"2024-04-25T14:13:33.228839+0000","last_fullsized":"2024-04-25T14:13:33.228839+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.221814+0000","last_change":"2024-04-25T14:13:33.221814+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.221814+0000","last_undegraded":"2024-04-25T14:13:33.221814+0000","last_fullsized":"2024-04-25T14:13:33.221814+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.223958+0000","last_change":"2024-04-25T14:13:33.223958+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.223958+0000","last_undegraded":"2024-04-25T14:13:33.223958+0000","last_fullsized":"2024-04-25T14:13:33.223958+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219450+0000","last_change":"2024-04-25T14:13:33.219450+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219450+0000","last_undegraded":"2024-04-25T14:13:33.219450+0000","last_fullsized":"2024-04-25T14:13:33.219450+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.224193+0000","last_change":"2024-04-25T14:13:33.224193+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.224193+0000","last_undegraded":"2024-04-25T14:13:33.224193+0000","last_fullsized":"2024-04-25T14:13:33.224193+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.226906+0000","last_change":"2024-04-25T14:13:33.226906+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.226906+0000","last_undegraded":"2024-04-25T14:13:33.226906+0000","last_fullsized":"2024-04-25T14:13:33.226906+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.223332+0000","last_change":"2024-04-25T14:13:33.223332+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.223332+0000","last_undegraded":"2024-04-25T14:13:33.223332+0000","last_fullsized":"2024-04-25T14:13:33.223332+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.217581+0000","last_change":"2024-04-25T14:13:33.217581+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.217581+0000","last_undegraded":"2024-04-25T14:13:33.217581+0000","last_fullsized":"2024-04-25T14:13:33.217581+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.238496+0000","last_change":"2024-04-25T14:13:33.233760+0000","last_active":"2024-04-25T14:13:33.238496+0000","last_peered":"2024-04-25T14:13:33.238496+0000","last_clean":"2024-04-25T14:13:33.238496+0000","last_became_active":"2024-04-25T14:13:33.233378+0000","last_became_peered":"2024-04-25T14:13:33.233378+0000","last_unstale":"2024-04-25T14:13:33.238496+0000","last_undegraded":"2024-04-25T14:13:33.238496+0000","last_fullsized":"2024-04-25T14:13:33.238496+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:24.899595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219833+0000","last_change":"2024-04-25T14:13:33.219833+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219833+0000","last_undegraded":"2024-04-25T14:13:33.219833+0000","last_fullsized":"2024-04-25T14:13:33.219833+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.243213+0000","last_change":"2024-04-25T14:13:33.239975+0000","last_active":"2024-04-25T14:13:33.243213+0000","last_peered":"2024-04-25T14:13:33.243213+0000","last_clean":"2024-04-25T14:13:33.243213+0000","last_became_active":"2024-04-25T14:13:33.239676+0000","last_became_peered":"2024-04-25T14:13:33.239676+0000","last_unstale":"2024-04-25T14:13:33.243213+0000","last_undegraded":"2024-04-25T14:13:33.243213+0000","last_fullsized":"2024-04-25T14:13:33.243213+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:44.934234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.218811+0000","last_change":"2024-04-25T14:13:33.218811+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.218811+0000","last_undegraded":"2024-04-25T14:13:33.218811+0000","last_fullsized":"2024-04-25T14:13:33.218811+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.242321+0000","last_change":"2024-04-25T14:13:33.241587+0000","last_active":"2024-04-25T14:13:33.242321+0000","last_peered":"2024-04-25T14:13:33.242321+0000","last_clean":"2024-04-25T14:13:33.242321+0000","last_became_active":"2024-04-25T14:13:33.240960+0000","last_became_peered":"2024-04-25T14:13:33.240960+0000","last_unstale":"2024-04-25T14:13:33.242321+0000","last_undegraded":"2024-04-25T14:13:33.242321+0000","last_fullsized":"2024-04-25T14:13:33.242321+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:34.418730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.217794+0000","last_change":"2024-04-25T14:13:33.217794+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.217794+0000","last_undegraded":"2024-04-25T14:13:33.217794+0000","last_fullsized":"2024-04-25T14:13:33.217794+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.242412+0000","last_change":"2024-04-25T14:13:33.239782+0000","last_active":"2024-04-25T14:13:33.242412+0000","last_peered":"2024-04-25T14:13:33.242412+0000","last_clean":"2024-04-25T14:13:33.242412+0000","last_became_active":"2024-04-25T14:13:33.239128+0000","last_became_peered":"2024-04-25T14:13:33.239128+0000","last_unstale":"2024-04-25T14:13:33.242412+0000","last_undegraded":"2024-04-25T14:13:33.242412+0000","last_fullsized":"2024-04-25T14:13:33.242412+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:40.423932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.221899+0000","last_change":"2024-04-25T14:13:33.221899+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.221899+0000","last_undegraded":"2024-04-25T14:13:33.221899+0000","last_fullsized":"2024-04-25T14:13:33.221899+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.244471+0000","last_change":"2024-04-25T14:13:33.242875+0000","last_active":"2024-04-25T14:13:33.244471+0000","last_peered":"2024-04-25T14:13:33.244471+0000","last_clean":"2024-04-25T14:13:33.244471+0000","last_became_active":"2024-04-25T14:13:33.242337+0000","last_became_peered":"2024-04-25T14:13:33.242337+0000","last_unstale":"2024-04-25T14:13:33.244471+0000","last_undegraded":"2024-04-25T14:13:33.244471+0000","last_fullsized":"2024-04-25T14:13:33.244471+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:16.518784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.226798+0000","last_change":"2024-04-25T14:13:33.226798+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.226798+0000","last_undegraded":"2024-04-25T14:13:33.226798+0000","last_fullsized":"2024-04-25T14:13:33.226798+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.241274+0000","last_change":"2024-04-25T14:13:33.233642+0000","last_active":"2024-04-25T14:13:33.241274+0000","last_peered":"2024-04-25T14:13:33.241274+0000","last_clean":"2024-04-25T14:13:33.241274+0000","last_became_active":"2024-04-25T14:13:33.233355+0000","last_became_peered":"2024-04-25T14:13:33.233355+0000","last_unstale":"2024-04-25T14:13:33.241274+0000","last_undegraded":"2024-04-25T14:13:33.241274+0000","last_fullsized":"2024-04-25T14:13:33.241274+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:37.218976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.228411+0000","last_change":"2024-04-25T14:13:33.228411+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.228411+0000","last_undegraded":"2024-04-25T14:13:33.228411+0000","last_fullsized":"2024-04-25T14:13:33.228411+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.241759+0000","last_change":"2024-04-25T14:13:33.239682+0000","last_active":"2024-04-25T14:13:33.241759+0000","last_peered":"2024-04-25T14:13:33.241759+0000","last_clean":"2024-04-25T14:13:33.241759+0000","last_became_active":"2024-04-25T14:13:33.239378+0000","last_became_peered":"2024-04-25T14:13:33.239378+0000","last_unstale":"2024-04-25T14:13:33.241759+0000","last_undegraded":"2024-04-25T14:13:33.241759+0000","last_fullsized":"2024-04-25T14:13:33.241759+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:10.873169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.217318+0000","last_change":"2024-04-25T14:13:33.217318+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.217318+0000","last_undegraded":"2024-04-25T14:13:33.217318+0000","last_fullsized":"2024-04-25T14:13:33.217318+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.238019+0000","last_change":"2024-04-25T14:13:33.232394+0000","last_active":"2024-04-25T14:13:33.238019+0000","last_peered":"2024-04-25T14:13:33.238019+0000","last_clean":"2024-04-25T14:13:33.238019+0000","last_became_active":"2024-04-25T14:13:33.231789+0000","last_became_peered":"2024-04-25T14:13:33.231789+0000","last_unstale":"2024-04-25T14:13:33.238019+0000","last_undegraded":"2024-04-25T14:13:33.238019+0000","last_fullsized":"2024-04-25T14:13:33.238019+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:45.600418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.218265+0000","last_change":"2024-04-25T14:13:33.218265+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.218265+0000","last_undegraded":"2024-04-25T14:13:33.218265+0000","last_fullsized":"2024-04-25T14:13:33.218265+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.243068+0000","last_change":"2024-04-25T14:13:33.235450+0000","last_active":"2024-04-25T14:13:33.243068+0000","last_peered":"2024-04-25T14:13:33.243068+0000","last_clean":"2024-04-25T14:13:33.243068+0000","last_became_active":"2024-04-25T14:13:33.235152+0000","last_became_peered":"2024-04-25T14:13:33.235152+0000","last_unstale":"2024-04-25T14:13:33.243068+0000","last_undegraded":"2024-04-25T14:13:33.243068+0000","last_fullsized":"2024-04-25T14:13:33.243068+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:35.585117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219518+0000","last_change":"2024-04-25T14:13:33.219518+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219518+0000","last_undegraded":"2024-04-25T14:13:33.219518+0000","last_fullsized":"2024-04-25T14:13:33.219518+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.238678+0000","last_change":"2024-04-25T14:13:33.235678+0000","last_active":"2024-04-25T14:13:33.238678+0000","last_peered":"2024-04-25T14:13:33.238678+0000","last_clean":"2024-04-25T14:13:33.238678+0000","last_became_active":"2024-04-25T14:13:33.235038+0000","last_became_peered":"2024-04-25T14:13:33.235038+0000","last_unstale":"2024-04-25T14:13:33.238678+0000","last_undegraded":"2024-04-25T14:13:33.238678+0000","last_fullsized":"2024-04-25T14:13:33.238678+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:23.471943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.225982+0000","last_change":"2024-04-25T14:13:33.225982+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.225982+0000","last_undegraded":"2024-04-25T14:13:33.225982+0000","last_fullsized":"2024-04-25T14:13:33.225982+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.243523+0000","last_change":"2024-04-25T14:13:33.235754+0000","last_active":"2024-04-25T14:13:33.243523+0000","last_peered":"2024-04-25T14:13:33.243523+0000","last_clean":"2024-04-25T14:13:33.243523+0000","last_became_active":"2024-04-25T14:13:33.235119+0000","last_became_peered":"2024-04-25T14:13:33.235119+0000","last_unstale":"2024-04-25T14:13:33.243523+0000","last_undegraded":"2024-04-25T14:13:33.243523+0000","last_fullsized":"2024-04-25T14:13:33.243523+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:09.237922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.222837+0000","last_change":"2024-04-25T14:13:33.222837+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.222837+0000","last_undegraded":"2024-04-25T14:13:33.222837+0000","last_fullsized":"2024-04-25T14:13:33.222837+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.240723+0000","last_change":"2024-04-25T14:13:33.237307+0000","last_active":"2024-04-25T14:13:33.240723+0000","last_peered":"2024-04-25T14:13:33.240723+0000","last_clean":"2024-04-25T14:13:33.240723+0000","last_became_active":"2024-04-25T14:13:33.237012+0000","last_became_peered":"2024-04-25T14:13:33.237012+0000","last_unstale":"2024-04-25T14:13:33.240723+0000","last_undegraded":"2024-04-25T14:13:33.240723+0000","last_fullsized":"2024-04-25T14:13:33.240723+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:45.356743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.222740+0000","last_change":"2024-04-25T14:13:33.222740+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.222740+0000","last_undegraded":"2024-04-25T14:13:33.222740+0000","last_fullsized":"2024-04-25T14:13:33.222740+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.241914+0000","last_change":"2024-04-25T14:13:33.234217+0000","last_active":"2024-04-25T14:13:33.241914+0000","last_peered":"2024-04-25T14:13:33.241914+0000","last_clean":"2024-04-25T14:13:33.241914+0000","last_became_active":"2024-04-25T14:13:33.233902+0000","last_became_peered":"2024-04-25T14:13:33.233902+0000","last_unstale":"2024-04-25T14:13:33.241914+0000","last_undegraded":"2024-04-25T14:13:33.241914+0000","last_fullsized":"2024-04-25T14:13:33.241914+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:09.455778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.227060+0000","last_change":"2024-04-25T14:13:33.227060+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.227060+0000","last_undegraded":"2024-04-25T14:13:33.227060+0000","last_fullsized":"2024-04-25T14:13:33.227060+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.240405+0000","last_change":"2024-04-25T14:13:33.239716+0000","last_active":"2024-04-25T14:13:33.240405+0000","last_peered":"2024-04-25T14:13:33.240405+0000","last_clean":"2024-04-25T14:13:33.240405+0000","last_became_active":"2024-04-25T14:13:33.239375+0000","last_became_peered":"2024-04-25T14:13:33.239375+0000","last_unstale":"2024-04-25T14:13:33.240405+0000","last_undegraded":"2024-04-25T14:13:33.240405+0000","last_fullsized":"2024-04-25T14:13:33.240405+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:05.463091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.226077+0000","last_change":"2024-04-25T14:13:33.226077+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.226077+0000","last_undegraded":"2024-04-25T14:13:33.226077+0000","last_fullsized":"2024-04-25T14:13:33.226077+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.239375+0000","last_change":"2024-04-25T14:13:33.238315+0000","last_active":"2024-04-25T14:13:33.239375+0000","last_peered":"2024-04-25T14:13:33.239375+0000","last_clean":"2024-04-25T14:13:33.239375+0000","last_became_active":"2024-04-25T14:13:33.237955+0000","last_became_peered":"2024-04-25T14:13:33.237955+0000","last_unstale":"2024-04-25T14:13:33.239375+0000","last_undegraded":"2024-04-25T14:13:33.239375+0000","last_fullsized":"2024-04-25T14:13:33.239375+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:50.318605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.223560+0000","last_change":"2024-04-25T14:13:33.223560+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.223560+0000","last_undegraded":"2024-04-25T14:13:33.223560+0000","last_fullsized":"2024-04-25T14:13:33.223560+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.237567+0000","last_change":"2024-04-25T14:13:33.226581+0000","last_active":"2024-04-25T14:13:33.237567+0000","last_peered":"2024-04-25T14:13:33.237567+0000","last_clean":"2024-04-25T14:13:33.237567+0000","last_became_active":"2024-04-25T14:13:33.225988+0000","last_became_peered":"2024-04-25T14:13:33.225988+0000","last_unstale":"2024-04-25T14:13:33.237567+0000","last_undegraded":"2024-04-25T14:13:33.237567+0000","last_fullsized":"2024-04-25T14:13:33.237567+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:08.407184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.224808+0000","last_change":"2024-04-25T14:13:33.224808+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.224808+0000","last_undegraded":"2024-04-25T14:13:33.224808+0000","last_fullsized":"2024-04-25T14:13:33.224808+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.244892+0000","last_change":"2024-04-25T14:13:33.239707+0000","last_active":"2024-04-25T14:13:33.244892+0000","last_peered":"2024-04-25T14:13:33.244892+0000","last_clean":"2024-04-25T14:13:33.244892+0000","last_became_active":"2024-04-25T14:13:33.239408+0000","last_became_peered":"2024-04-25T14:13:33.239408+0000","last_unstale":"2024-04-25T14:13:33.244892+0000","last_undegraded":"2024-04-25T14:13:33.244892+0000","last_fullsized":"2024-04-25T14:13:33.244892+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:51:01.497289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.226373+0000","last_change":"2024-04-25T14:13:33.226373+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.226373+0000","last_undegraded":"2024-04-25T14:13:33.226373+0000","last_fullsized":"2024-04-25T14:13:33.226373+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.239011+0000","last_change":"2024-04-25T14:13:33.232965+0000","last_active":"2024-04-25T14:13:33.239011+0000","last_peered":"2024-04-25T14:13:33.239011+0000","last_clean":"2024-04-25T14:13:33.239011+0000","last_became_active":"2024-04-25T14:13:33.232647+0000","last_became_peered":"2024-04-25T14:13:33.232647+0000","last_unstale":"2024-04-25T14:13:33.239011+0000","last_undegraded":"2024-04-25T14:13:33.239011+0000","last_fullsized":"2024-04-25T14:13:33.239011+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:12.089780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.229146+0000","last_change":"2024-04-25T14:13:33.229146+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.229146+0000","last_undegraded":"2024-04-25T14:13:33.229146+0000","last_fullsized":"2024-04-25T14:13:33.229146+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.244360+0000","last_change":"2024-04-25T14:13:33.240376+0000","last_active":"2024-04-25T14:13:33.244360+0000","last_peered":"2024-04-25T14:13:33.244360+0000","last_clean":"2024-04-25T14:13:33.244360+0000","last_became_active":"2024-04-25T14:13:33.240103+0000","last_became_peered":"2024-04-25T14:13:33.240103+0000","last_unstale":"2024-04-25T14:13:33.244360+0000","last_undegraded":"2024-04-25T14:13:33.244360+0000","last_fullsized":"2024-04-25T14:13:33.244360+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:55.225878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219516+0000","last_change":"2024-04-25T14:13:33.219516+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219516+0000","last_undegraded":"2024-04-25T14:13:33.219516+0000","last_fullsized":"2024-04-25T14:13:33.219516+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.240700+0000","last_change":"2024-04-25T14:13:33.231549+0000","last_active":"2024-04-25T14:13:33.240700+0000","last_peered":"2024-04-25T14:13:33.240700+0000","last_clean":"2024-04-25T14:13:33.240700+0000","last_became_active":"2024-04-25T14:13:33.231239+0000","last_became_peered":"2024-04-25T14:13:33.231239+0000","last_unstale":"2024-04-25T14:13:33.240700+0000","last_undegraded":"2024-04-25T14:13:33.240700+0000","last_fullsized":"2024-04-25T14:13:33.240700+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:46.823734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219647+0000","last_change":"2024-04-25T14:13:33.219647+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219647+0000","last_undegraded":"2024-04-25T14:13:33.219647+0000","last_fullsized":"2024-04-25T14:13:33.219647+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.239404+0000","last_change":"2024-04-25T14:13:33.238316+0000","last_active":"2024-04-25T14:13:33.239404+0000","last_peered":"2024-04-25T14:13:33.239404+0000","last_clean":"2024-04-25T14:13:33.239404+0000","last_became_active":"2024-04-25T14:13:33.237984+0000","last_became_peered":"2024-04-25T14:13:33.237984+0000","last_unstale":"2024-04-25T14:13:33.239404+0000","last_undegraded":"2024-04-25T14:13:33.239404+0000","last_fullsized":"2024-04-25T14:13:33.239404+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:01.863933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.220719+0000","last_change":"2024-04-25T14:13:33.220719+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.220719+0000","last_undegraded":"2024-04-25T14:13:33.220719+0000","last_fullsized":"2024-04-25T14:13:33.220719+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.240694+0000","last_change":"2024-04-25T14:13:33.233345+0000","last_active":"2024-04-25T14:13:33.240694+0000","last_peered":"2024-04-25T14:13:33.240694+0000","last_clean":"2024-04-25T14:13:33.240694+0000","last_became_active":"2024-04-25T14:13:33.233045+0000","last_became_peered":"2024-04-25T14:13:33.233045+0000","last_unstale":"2024-04-25T14:13:33.240694+0000","last_undegraded":"2024-04-25T14:13:33.240694+0000","last_fullsized":"2024-04-25T14:13:33.240694+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:28.978236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.218329+0000","last_change":"2024-04-25T14:13:33.218329+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.218329+0000","last_undegraded":"2024-04-25T14:13:33.218329+0000","last_fullsized":"2024-04-25T14:13:33.218329+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.235430+0000","last_change":"2024-04-25T14:13:33.233786+0000","last_active":"2024-04-25T14:13:33.235430+0000","last_peered":"2024-04-25T14:13:33.235430+0000","last_clean":"2024-04-25T14:13:33.235430+0000","last_became_active":"2024-04-25T14:13:33.233479+0000","last_became_peered":"2024-04-25T14:13:33.233479+0000","last_unstale":"2024-04-25T14:13:33.235430+0000","last_undegraded":"2024-04-25T14:13:33.235430+0000","last_fullsized":"2024-04-25T14:13:33.235430+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:57.369446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.225358+0000","last_change":"2024-04-25T14:13:33.225358+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.225358+0000","last_undegraded":"2024-04-25T14:13:33.225358+0000","last_fullsized":"2024-04-25T14:13:33.225358+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.239073+0000","last_change":"2024-04-25T14:13:33.235217+0000","last_active":"2024-04-25T14:13:33.239073+0000","last_peered":"2024-04-25T14:13:33.239073+0000","last_clean":"2024-04-25T14:13:33.239073+0000","last_became_active":"2024-04-25T14:13:33.234923+0000","last_became_peered":"2024-04-25T14:13:33.234923+0000","last_unstale":"2024-04-25T14:13:33.239073+0000","last_undegraded":"2024-04-25T14:13:33.239073+0000","last_fullsized":"2024-04-25T14:13:33.239073+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:16.491560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.223188+0000","last_change":"2024-04-25T14:13:33.223188+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.223188+0000","last_undegraded":"2024-04-25T14:13:33.223188+0000","last_fullsized":"2024-04-25T14:13:33.223188+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.237237+0000","last_change":"2024-04-25T14:13:33.233600+0000","last_active":"2024-04-25T14:13:33.237237+0000","last_peered":"2024-04-25T14:13:33.237237+0000","last_clean":"2024-04-25T14:13:33.237237+0000","last_became_active":"2024-04-25T14:13:33.233290+0000","last_became_peered":"2024-04-25T14:13:33.233290+0000","last_unstale":"2024-04-25T14:13:33.237237+0000","last_undegraded":"2024-04-25T14:13:33.237237+0000","last_fullsized":"2024-04-25T14:13:33.237237+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:16.534415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.226375+0000","last_change":"2024-04-25T14:13:33.226375+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.226375+0000","last_undegraded":"2024-04-25T14:13:33.226375+0000","last_fullsized":"2024-04-25T14:13:33.226375+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.236234+0000","last_change":"2024-04-25T14:13:33.229751+0000","last_active":"2024-04-25T14:13:33.236234+0000","last_peered":"2024-04-25T14:13:33.236234+0000","last_clean":"2024-04-25T14:13:33.236234+0000","last_became_active":"2024-04-25T14:13:33.229451+0000","last_became_peered":"2024-04-25T14:13:33.229451+0000","last_unstale":"2024-04-25T14:13:33.236234+0000","last_undegraded":"2024-04-25T14:13:33.236234+0000","last_fullsized":"2024-04-25T14:13:33.236234+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:34.938781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.220859+0000","last_change":"2024-04-25T14:13:33.220859+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.220859+0000","last_undegraded":"2024-04-25T14:13:33.220859+0000","last_fullsized":"2024-04-25T14:13:33.220859+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.246616+0000","last_change":"2024-04-25T14:13:33.245950+0000","last_active":"2024-04-25T14:13:33.246616+0000","last_peered":"2024-04-25T14:13:33.246616+0000","last_clean":"2024-04-25T14:13:33.246616+0000","last_became_active":"2024-04-25T14:13:33.245627+0000","last_became_peered":"2024-04-25T14:13:33.245627+0000","last_unstale":"2024-04-25T14:13:33.246616+0000","last_undegraded":"2024-04-25T14:13:33.246616+0000","last_fullsized":"2024-04-25T14:13:33.246616+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:59.112002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.216195+0000","last_change":"2024-04-25T14:13:33.216195+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.216195+0000","last_undegraded":"2024-04-25T14:13:33.216195+0000","last_fullsized":"2024-04-25T14:13:33.216195+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.242071+0000","last_change":"2024-04-25T14:13:33.241343+0000","last_active":"2024-04-25T14:13:33.242071+0000","last_peered":"2024-04-25T14:13:33.242071+0000","last_clean":"2024-04-25T14:13:33.242071+0000","last_became_active":"2024-04-25T14:13:33.240975+0000","last_became_peered":"2024-04-25T14:13:33.240975+0000","last_unstale":"2024-04-25T14:13:33.242071+0000","last_undegraded":"2024-04-25T14:13:33.242071+0000","last_fullsized":"2024-04-25T14:13:33.242071+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:43.434745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.217868+0000","last_change":"2024-04-25T14:13:33.217868+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.217868+0000","last_undegraded":"2024-04-25T14:13:33.217868+0000","last_fullsized":"2024-04-25T14:13:33.217868+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.236031+0000","last_change":"2024-04-25T14:13:33.230789+0000","last_active":"2024-04-25T14:13:33.236031+0000","last_peered":"2024-04-25T14:13:33.236031+0000","last_clean":"2024-04-25T14:13:33.236031+0000","last_became_active":"2024-04-25T14:13:33.230422+0000","last_became_peered":"2024-04-25T14:13:33.230422+0000","last_unstale":"2024-04-25T14:13:33.236031+0000","last_undegraded":"2024-04-25T14:13:33.236031+0000","last_fullsized":"2024-04-25T14:13:33.236031+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:05:59.451071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219902+0000","last_change":"2024-04-25T14:13:33.219902+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219902+0000","last_undegraded":"2024-04-25T14:13:33.219902+0000","last_fullsized":"2024-04-25T14:13:33.219902+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.242301+0000","last_change":"2024-04-25T14:13:33.241318+0000","last_active":"2024-04-25T14:13:33.242301+0000","last_peered":"2024-04-25T14:13:33.242301+0000","last_clean":"2024-04-25T14:13:33.242301+0000","last_became_active":"2024-04-25T14:13:33.240931+0000","last_became_peered":"2024-04-25T14:13:33.240931+0000","last_unstale":"2024-04-25T14:13:33.242301+0000","last_undegraded":"2024-04-25T14:13:33.242301+0000","last_fullsized":"2024-04-25T14:13:33.242301+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:39.911385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.225571+0000","last_change":"2024-04-25T14:13:33.225571+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.225571+0000","last_undegraded":"2024-04-25T14:13:33.225571+0000","last_fullsized":"2024-04-25T14:13:33.225571+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.238829+0000","last_change":"2024-04-25T14:13:33.229866+0000","last_active":"2024-04-25T14:13:33.238829+0000","last_peered":"2024-04-25T14:13:33.238829+0000","last_clean":"2024-04-25T14:13:33.238829+0000","last_became_active":"2024-04-25T14:13:33.229562+0000","last_became_peered":"2024-04-25T14:13:33.229562+0000","last_unstale":"2024-04-25T14:13:33.238829+0000","last_undegraded":"2024-04-25T14:13:33.238829+0000","last_fullsized":"2024-04-25T14:13:33.238829+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:00.731593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.218315+0000","last_change":"2024-04-25T14:13:33.218315+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.218315+0000","last_undegraded":"2024-04-25T14:13:33.218315+0000","last_fullsized":"2024-04-25T14:13:33.218315+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.244337+0000","last_change":"2024-04-25T14:13:33.242678+0000","last_active":"2024-04-25T14:13:33.244337+0000","last_peered":"2024-04-25T14:13:33.244337+0000","last_clean":"2024-04-25T14:13:33.244337+0000","last_became_active":"2024-04-25T14:13:33.241884+0000","last_became_peered":"2024-04-25T14:13:33.241884+0000","last_unstale":"2024-04-25T14:13:33.244337+0000","last_undegraded":"2024-04-25T14:13:33.244337+0000","last_fullsized":"2024-04-25T14:13:33.244337+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:00.731593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"144'3291","reported_seq":3828,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.226260+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:13:33.226260+0000","last_peered":"2024-04-25T14:13:33.226260+0000","last_clean":"2024-04-25T14:13:33.226260+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:13:33.226260+0000","last_undegraded":"2024-04-25T14:13:33.226260+0000","last_fullsized":"2024-04-25T14:13:33.226260+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":3291,"log_dups_size":0,"ondisk_log_size":3291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":837,"num_read_kb":2279,"num_write":247,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":837,"num_read_kb":2279,"num_write":247,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3291,"ondisk_log_size":3291,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574963,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29484,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342356,"statfs":{"total":96636764160,"available":96606572544,"internally_reserved":0,"allocated":2519040,"data_stored":1556182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574964,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33064,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1925120,"data_stored":965814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574965,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33064,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1925120,"data_stored":965814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574964,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33064,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1925120,"data_stored":965814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574964,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33068,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1925120,"data_stored":965814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33048,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338792,"statfs":{"total":96636764160,"available":96602923008,"internally_reserved":0,"allocated":1912832,"data_stored":959155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33064,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1925120,"data_stored":965814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574964,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29484,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342356,"statfs":{"total":96636764160,"available":96606572544,"internally_reserved":0,"allocated":2519040,"data_stored":1556182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:13:38.352 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:13:38.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.539+0000 7f62f1e00640 1 Processor -- start 2024-04-25T14:13:38.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.539+0000 7f62f1e00640 1 -- start start 2024-04-25T14:13:38.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.539+0000 7f62f1e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62ec0cdd70 0x7f62ec0ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.539+0000 7f62f1e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f62ec070e20 0x7f62ec071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.539+0000 7f62f1e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f62ec06e700 0x7f62ec06eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.539+0000 7f62f1e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f62ec071770 con 0x7f62ec070e20 2024-04-25T14:13:38.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.539+0000 7f62f1e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f62ec06f050 con 0x7f62ec06e700 2024-04-25T14:13:38.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.539+0000 7f62f1e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f62ec06d940 con 0x7f62ec0cdd70 2024-04-25T14:13:38.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.539+0000 7f62e2000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f62ec070e20 0x7f62ec071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.539+0000 7f62e2000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f62ec070e20 0x7f62ec071230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55456/0 (socket says 172.21.15.86:55456) 2024-04-25T14:13:38.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.539+0000 7f62e2000640 1 -- 172.21.15.86:0/2307838751 learned_addr learned my addr 172.21.15.86:0/2307838751 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:38.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.539+0000 7f62e2000640 1 -- 172.21.15.86:0/2307838751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62ec0cdd70 msgr2=0x7f62ec0ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:38.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.539+0000 7f62eaa00640 1 --2- 172.21.15.86:0/2307838751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62ec0cdd70 0x7f62ec0ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.539+0000 7f62eb400640 1 --2- 172.21.15.86:0/2307838751 >> v2:172.21.15.155:3300/0 conn(0x7f62ec06e700 0x7f62ec06eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.539+0000 7f62e2000640 1 --2- 172.21.15.86:0/2307838751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62ec0cdd70 0x7f62ec0ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.539+0000 7f62e2000640 1 -- 172.21.15.86:0/2307838751 >> v2:172.21.15.155:3300/0 conn(0x7f62ec06e700 msgr2=0x7f62ec06eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:38.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.539+0000 7f62e2000640 1 --2- 172.21.15.86:0/2307838751 >> v2:172.21.15.155:3300/0 conn(0x7f62ec06e700 0x7f62ec06eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.543+0000 7f62e2000640 1 -- 172.21.15.86:0/2307838751 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ec06dc20 con 0x7f62ec070e20 2024-04-25T14:13:38.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.543+0000 7f62eaa00640 1 --2- 172.21.15.86:0/2307838751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62ec0cdd70 0x7f62ec0ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:38.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.543+0000 7f62eb400640 1 --2- 172.21.15.86:0/2307838751 >> v2:172.21.15.155:3300/0 conn(0x7f62ec06e700 0x7f62ec06eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:38.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.543+0000 7f62e2000640 1 --2- 172.21.15.86:0/2307838751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62ec070e20 0x7f62ec071230 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f62ec071b00 tx=0x7f62d401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1eb48fe3460cd4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:38.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.543+0000 7f62ea000640 1 -- 172.21.15.86:0/2307838751 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d40b0070 con 0x7f62ec070e20 2024-04-25T14:13:38.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.543+0000 7f62ea000640 1 -- 172.21.15.86:0/2307838751 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62d40ab600 con 0x7f62ec070e20 2024-04-25T14:13:38.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.543+0000 7f62ea000640 1 -- 172.21.15.86:0/2307838751 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d40ab9d0 con 0x7f62ec070e20 2024-04-25T14:13:38.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.543+0000 7f62f1e00640 1 -- 172.21.15.86:0/2307838751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62ec070e20 msgr2=0x7f62ec071230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.543+0000 7f62f1e00640 1 --2- 172.21.15.86:0/2307838751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62ec070e20 0x7f62ec071230 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f62ec071b00 tx=0x7f62d401c490 comp rx=0 tx=0).stop 2024-04-25T14:13:38.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.543+0000 7f62f1e00640 1 -- 172.21.15.86:0/2307838751 shutdown_connections 2024-04-25T14:13:38.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.543+0000 7f62f1e00640 1 --2- 172.21.15.86:0/2307838751 >> v2:172.21.15.155:3300/0 conn(0x7f62ec06e700 0x7f62ec06eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.543+0000 7f62f1e00640 1 --2- 172.21.15.86:0/2307838751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62ec070e20 0x7f62ec071230 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.543+0000 7f62f1e00640 1 --2- 172.21.15.86:0/2307838751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62ec0cdd70 0x7f62ec0ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.543+0000 7f62f1e00640 1 -- 172.21.15.86:0/2307838751 >> 172.21.15.86:0/2307838751 conn(0x7f62ec0cffc0 msgr2=0x7f62ec0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:38.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.547+0000 7f62f1e00640 1 -- 172.21.15.86:0/2307838751 shutdown_connections 2024-04-25T14:13:38.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.547+0000 7f62f1e00640 1 -- 172.21.15.86:0/2307838751 wait complete. 2024-04-25T14:13:38.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.547+0000 7f62f1e00640 1 Processor -- start 2024-04-25T14:13:38.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.547+0000 7f62f1e00640 1 -- start start 2024-04-25T14:13:38.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.547+0000 7f62f1e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62ec06e700 0x7f62ec0cb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.547+0000 7f62f1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62ec070e20 0x7f62ec156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.547+0000 7f62f1e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62ec0cdd70 0x7f62ec158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.547+0000 7f62f1e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec1642f0 con 0x7f62ec070e20 2024-04-25T14:13:38.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.547+0000 7f62f1e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec164170 con 0x7f62ec0cdd70 2024-04-25T14:13:38.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.547+0000 7f62f1e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f62ec164470 con 0x7f62ec06e700 2024-04-25T14:13:38.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.547+0000 7f62e2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62ec070e20 0x7f62ec156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.547+0000 7f62e2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62ec070e20 0x7f62ec156970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55464/0 (socket says 172.21.15.86:55464) 2024-04-25T14:13:38.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.547+0000 7f62e2000640 1 -- 172.21.15.86:0/3804294182 learned_addr learned my addr 172.21.15.86:0/3804294182 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:38.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.551+0000 7f62eaa00640 1 --2- 172.21.15.86:0/3804294182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62ec06e700 0x7f62ec0cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.551+0000 7f62eb400640 1 --2- 172.21.15.86:0/3804294182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62ec0cdd70 0x7f62ec158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.551+0000 7f62e2000640 1 -- 172.21.15.86:0/3804294182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62ec06e700 msgr2=0x7f62ec0cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.551+0000 7f62e2000640 1 --2- 172.21.15.86:0/3804294182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62ec06e700 0x7f62ec0cb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.551+0000 7f62e2000640 1 -- 172.21.15.86:0/3804294182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62ec0cdd70 msgr2=0x7f62ec158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.551+0000 7f62e2000640 1 --2- 172.21.15.86:0/3804294182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62ec0cdd70 0x7f62ec158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.551+0000 7f62e2000640 1 -- 172.21.15.86:0/3804294182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ec0b3c60 con 0x7f62ec070e20 2024-04-25T14:13:38.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.551+0000 7f62e2000640 1 --2- 172.21.15.86:0/3804294182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62ec070e20 0x7f62ec156970 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f62d401cd30 tx=0x7f62d4019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:38.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.551+0000 7f62e3e00640 1 -- 172.21.15.86:0/3804294182 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d40b9070 con 0x7f62ec070e20 2024-04-25T14:13:38.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.551+0000 7f62e3e00640 1 -- 172.21.15.86:0/3804294182 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62d40ab9b0 con 0x7f62ec070e20 2024-04-25T14:13:38.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.551+0000 7f62f1e00640 1 -- 172.21.15.86:0/3804294182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62ec161eb0 con 0x7f62ec070e20 2024-04-25T14:13:38.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.551+0000 7f62e3e00640 1 -- 172.21.15.86:0/3804294182 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d40b0040 con 0x7f62ec070e20 2024-04-25T14:13:38.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.551+0000 7f62f1e00640 1 -- 172.21.15.86:0/3804294182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62ec0b7860 con 0x7f62ec070e20 2024-04-25T14:13:38.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.551+0000 7f62f1e00640 1 -- 172.21.15.86:0/3804294182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62ec076bf0 con 0x7f62ec070e20 2024-04-25T14:13:38.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.551+0000 7f62e3e00640 1 -- 172.21.15.86:0/3804294182 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7f62d40bc070 con 0x7f62ec070e20 2024-04-25T14:13:38.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.555+0000 7f62e3e00640 1 --2- 172.21.15.86:0/3804294182 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f62c806f520 0x7f62c8071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.555+0000 7f62e3e00640 1 -- 172.21.15.86:0/3804294182 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6101+0+0 (secure 0 0 0) 0x7f62d4128d90 con 0x7f62ec070e20 2024-04-25T14:13:38.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.555+0000 7f62eaa00640 1 --2- 172.21.15.86:0/3804294182 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f62c806f520 0x7f62c8071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.555+0000 7f62eaa00640 1 --2- 172.21.15.86:0/3804294182 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f62c806f520 0x7f62c8071a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f62dc006fd0 tx=0x7f62dc008040 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:38.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.555+0000 7f62e3e00640 1 -- 172.21.15.86:0/3804294182 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f62d4016110 con 0x7f62ec070e20 2024-04-25T14:13:38.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.707+0000 7f62f1e00640 1 -- 172.21.15.86:0/3804294182 --> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f62ec156eb0 con 0x7f62c806f520 2024-04-25T14:13:38.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.715+0000 7f62e3e00640 1 -- 172.21.15.86:0/3804294182 <== mgr.8198 v2:172.21.15.86:6832/505730993 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247100 (secure 0 0 0) 0x7f62ec156eb0 con 0x7f62c806f520 2024-04-25T14:13:38.716 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:13:38.717 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:13:38.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.719+0000 7f62f1e00640 1 -- 172.21.15.86:0/3804294182 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f62c806f520 msgr2=0x7f62c8071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.719+0000 7f62f1e00640 1 --2- 172.21.15.86:0/3804294182 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f62c806f520 0x7f62c8071a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f62dc006fd0 tx=0x7f62dc008040 comp rx=0 tx=0).stop 2024-04-25T14:13:38.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.719+0000 7f62f1e00640 1 -- 172.21.15.86:0/3804294182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62ec070e20 msgr2=0x7f62ec156970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.719+0000 7f62f1e00640 1 --2- 172.21.15.86:0/3804294182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62ec070e20 0x7f62ec156970 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f62d401cd30 tx=0x7f62d4019230 comp rx=0 tx=0).stop 2024-04-25T14:13:38.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.719+0000 7f62f1e00640 1 -- 172.21.15.86:0/3804294182 shutdown_connections 2024-04-25T14:13:38.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.719+0000 7f62f1e00640 1 --2- 172.21.15.86:0/3804294182 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f62c806f520 0x7f62c8071a30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.719+0000 7f62f1e00640 1 --2- 172.21.15.86:0/3804294182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62ec0cdd70 0x7f62ec158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.719+0000 7f62f1e00640 1 --2- 172.21.15.86:0/3804294182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62ec070e20 0x7f62ec156970 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.719+0000 7f62f1e00640 1 --2- 172.21.15.86:0/3804294182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62ec06e700 0x7f62ec0cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.719+0000 7f62f1e00640 1 -- 172.21.15.86:0/3804294182 >> 172.21.15.86:0/3804294182 conn(0x7f62ec0cffc0 msgr2=0x7f62ec15c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:38.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.719+0000 7f62f1e00640 1 -- 172.21.15.86:0/3804294182 shutdown_connections 2024-04-25T14:13:38.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.719+0000 7f62f1e00640 1 -- 172.21.15.86:0/3804294182 wait complete. 2024-04-25T14:13:38.737 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T14:13:38.077493+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":837,"num_read_kb":2279,"num_write":247,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3291,"ondisk_log_size":3291,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257340,"kb_used_data":16188,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754717380,"statfs":{"total":773094113280,"available":772830597120,"internally_reserved":0,"allocated":16576512,"data_stored":8900589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.870692"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.217839+0000","last_change":"2024-04-25T14:13:33.217839+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.217839+0000","last_undegraded":"2024-04-25T14:13:33.217839+0000","last_fullsized":"2024-04-25T14:13:33.217839+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.220477+0000","last_change":"2024-04-25T14:13:33.220477+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.220477+0000","last_undegraded":"2024-04-25T14:13:33.220477+0000","last_fullsized":"2024-04-25T14:13:33.220477+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.216321+0000","last_change":"2024-04-25T14:13:33.216321+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.216321+0000","last_undegraded":"2024-04-25T14:13:33.216321+0000","last_fullsized":"2024-04-25T14:13:33.216321+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.216741+0000","last_change":"2024-04-25T14:13:33.216741+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.216741+0000","last_undegraded":"2024-04-25T14:13:33.216741+0000","last_fullsized":"2024-04-25T14:13:33.216741+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219621+0000","last_change":"2024-04-25T14:13:33.219621+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219621+0000","last_undegraded":"2024-04-25T14:13:33.219621+0000","last_fullsized":"2024-04-25T14:13:33.219621+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.221767+0000","last_change":"2024-04-25T14:13:33.221767+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.221767+0000","last_undegraded":"2024-04-25T14:13:33.221767+0000","last_fullsized":"2024-04-25T14:13:33.221767+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.223617+0000","last_change":"2024-04-25T14:13:33.223617+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.223617+0000","last_undegraded":"2024-04-25T14:13:33.223617+0000","last_fullsized":"2024-04-25T14:13:33.223617+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.228099+0000","last_change":"2024-04-25T14:13:33.228099+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.228099+0000","last_undegraded":"2024-04-25T14:13:33.228099+0000","last_fullsized":"2024-04-25T14:13:33.228099+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.222924+0000","last_change":"2024-04-25T14:13:33.222924+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.222924+0000","last_undegraded":"2024-04-25T14:13:33.222924+0000","last_fullsized":"2024-04-25T14:13:33.222924+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.229249+0000","last_change":"2024-04-25T14:13:33.229249+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.229249+0000","last_undegraded":"2024-04-25T14:13:33.229249+0000","last_fullsized":"2024-04-25T14:13:33.229249+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.231347+0000","last_change":"2024-04-25T14:13:33.231347+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.231347+0000","last_undegraded":"2024-04-25T14:13:33.231347+0000","last_fullsized":"2024-04-25T14:13:33.231347+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.222546+0000","last_change":"2024-04-25T14:13:33.222546+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.222546+0000","last_undegraded":"2024-04-25T14:13:33.222546+0000","last_fullsized":"2024-04-25T14:13:33.222546+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.224550+0000","last_change":"2024-04-25T14:13:33.224550+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.224550+0000","last_undegraded":"2024-04-25T14:13:33.224550+0000","last_fullsized":"2024-04-25T14:13:33.224550+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219360+0000","last_change":"2024-04-25T14:13:33.219360+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219360+0000","last_undegraded":"2024-04-25T14:13:33.219360+0000","last_fullsized":"2024-04-25T14:13:33.219360+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.229250+0000","last_change":"2024-04-25T14:13:33.229250+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.229250+0000","last_undegraded":"2024-04-25T14:13:33.229250+0000","last_fullsized":"2024-04-25T14:13:33.229250+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.216778+0000","last_change":"2024-04-25T14:13:33.216778+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.216778+0000","last_undegraded":"2024-04-25T14:13:33.216778+0000","last_fullsized":"2024-04-25T14:13:33.216778+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.222484+0000","last_change":"2024-04-25T14:13:33.222484+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.222484+0000","last_undegraded":"2024-04-25T14:13:33.222484+0000","last_fullsized":"2024-04-25T14:13:33.222484+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.225152+0000","last_change":"2024-04-25T14:13:33.225152+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.225152+0000","last_undegraded":"2024-04-25T14:13:33.225152+0000","last_fullsized":"2024-04-25T14:13:33.225152+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.228898+0000","last_change":"2024-04-25T14:13:33.228898+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.228898+0000","last_undegraded":"2024-04-25T14:13:33.228898+0000","last_fullsized":"2024-04-25T14:13:33.228898+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.229071+0000","last_change":"2024-04-25T14:13:33.229071+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.229071+0000","last_undegraded":"2024-04-25T14:13:33.229071+0000","last_fullsized":"2024-04-25T14:13:33.229071+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.224104+0000","last_change":"2024-04-25T14:13:33.224104+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.224104+0000","last_undegraded":"2024-04-25T14:13:33.224104+0000","last_fullsized":"2024-04-25T14:13:33.224104+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.221177+0000","last_change":"2024-04-25T14:13:33.221177+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.221177+0000","last_undegraded":"2024-04-25T14:13:33.221177+0000","last_fullsized":"2024-04-25T14:13:33.221177+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.218713+0000","last_change":"2024-04-25T14:13:33.218713+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.218713+0000","last_undegraded":"2024-04-25T14:13:33.218713+0000","last_fullsized":"2024-04-25T14:13:33.218713+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.215974+0000","last_change":"2024-04-25T14:13:33.215974+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.215974+0000","last_undegraded":"2024-04-25T14:13:33.215974+0000","last_fullsized":"2024-04-25T14:13:33.215974+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.222118+0000","last_change":"2024-04-25T14:13:33.222118+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.222118+0000","last_undegraded":"2024-04-25T14:13:33.222118+0000","last_fullsized":"2024-04-25T14:13:33.222118+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.228839+0000","last_change":"2024-04-25T14:13:33.228839+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.228839+0000","last_undegraded":"2024-04-25T14:13:33.228839+0000","last_fullsized":"2024-04-25T14:13:33.228839+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.221814+0000","last_change":"2024-04-25T14:13:33.221814+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.221814+0000","last_undegraded":"2024-04-25T14:13:33.221814+0000","last_fullsized":"2024-04-25T14:13:33.221814+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.223958+0000","last_change":"2024-04-25T14:13:33.223958+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.223958+0000","last_undegraded":"2024-04-25T14:13:33.223958+0000","last_fullsized":"2024-04-25T14:13:33.223958+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219450+0000","last_change":"2024-04-25T14:13:33.219450+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219450+0000","last_undegraded":"2024-04-25T14:13:33.219450+0000","last_fullsized":"2024-04-25T14:13:33.219450+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.224193+0000","last_change":"2024-04-25T14:13:33.224193+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.224193+0000","last_undegraded":"2024-04-25T14:13:33.224193+0000","last_fullsized":"2024-04-25T14:13:33.224193+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.226906+0000","last_change":"2024-04-25T14:13:33.226906+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.226906+0000","last_undegraded":"2024-04-25T14:13:33.226906+0000","last_fullsized":"2024-04-25T14:13:33.226906+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.223332+0000","last_change":"2024-04-25T14:13:33.223332+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.223332+0000","last_undegraded":"2024-04-25T14:13:33.223332+0000","last_fullsized":"2024-04-25T14:13:33.223332+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.217581+0000","last_change":"2024-04-25T14:13:33.217581+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.217581+0000","last_undegraded":"2024-04-25T14:13:33.217581+0000","last_fullsized":"2024-04-25T14:13:33.217581+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.238496+0000","last_change":"2024-04-25T14:13:33.233760+0000","last_active":"2024-04-25T14:13:33.238496+0000","last_peered":"2024-04-25T14:13:33.238496+0000","last_clean":"2024-04-25T14:13:33.238496+0000","last_became_active":"2024-04-25T14:13:33.233378+0000","last_became_peered":"2024-04-25T14:13:33.233378+0000","last_unstale":"2024-04-25T14:13:33.238496+0000","last_undegraded":"2024-04-25T14:13:33.238496+0000","last_fullsized":"2024-04-25T14:13:33.238496+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:24.899595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219833+0000","last_change":"2024-04-25T14:13:33.219833+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219833+0000","last_undegraded":"2024-04-25T14:13:33.219833+0000","last_fullsized":"2024-04-25T14:13:33.219833+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.243213+0000","last_change":"2024-04-25T14:13:33.239975+0000","last_active":"2024-04-25T14:13:33.243213+0000","last_peered":"2024-04-25T14:13:33.243213+0000","last_clean":"2024-04-25T14:13:33.243213+0000","last_became_active":"2024-04-25T14:13:33.239676+0000","last_became_peered":"2024-04-25T14:13:33.239676+0000","last_unstale":"2024-04-25T14:13:33.243213+0000","last_undegraded":"2024-04-25T14:13:33.243213+0000","last_fullsized":"2024-04-25T14:13:33.243213+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:44.934234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.218811+0000","last_change":"2024-04-25T14:13:33.218811+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.218811+0000","last_undegraded":"2024-04-25T14:13:33.218811+0000","last_fullsized":"2024-04-25T14:13:33.218811+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.242321+0000","last_change":"2024-04-25T14:13:33.241587+0000","last_active":"2024-04-25T14:13:33.242321+0000","last_peered":"2024-04-25T14:13:33.242321+0000","last_clean":"2024-04-25T14:13:33.242321+0000","last_became_active":"2024-04-25T14:13:33.240960+0000","last_became_peered":"2024-04-25T14:13:33.240960+0000","last_unstale":"2024-04-25T14:13:33.242321+0000","last_undegraded":"2024-04-25T14:13:33.242321+0000","last_fullsized":"2024-04-25T14:13:33.242321+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:34.418730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.217794+0000","last_change":"2024-04-25T14:13:33.217794+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.217794+0000","last_undegraded":"2024-04-25T14:13:33.217794+0000","last_fullsized":"2024-04-25T14:13:33.217794+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.242412+0000","last_change":"2024-04-25T14:13:33.239782+0000","last_active":"2024-04-25T14:13:33.242412+0000","last_peered":"2024-04-25T14:13:33.242412+0000","last_clean":"2024-04-25T14:13:33.242412+0000","last_became_active":"2024-04-25T14:13:33.239128+0000","last_became_peered":"2024-04-25T14:13:33.239128+0000","last_unstale":"2024-04-25T14:13:33.242412+0000","last_undegraded":"2024-04-25T14:13:33.242412+0000","last_fullsized":"2024-04-25T14:13:33.242412+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:40.423932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.221899+0000","last_change":"2024-04-25T14:13:33.221899+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.221899+0000","last_undegraded":"2024-04-25T14:13:33.221899+0000","last_fullsized":"2024-04-25T14:13:33.221899+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.244471+0000","last_change":"2024-04-25T14:13:33.242875+0000","last_active":"2024-04-25T14:13:33.244471+0000","last_peered":"2024-04-25T14:13:33.244471+0000","last_clean":"2024-04-25T14:13:33.244471+0000","last_became_active":"2024-04-25T14:13:33.242337+0000","last_became_peered":"2024-04-25T14:13:33.242337+0000","last_unstale":"2024-04-25T14:13:33.244471+0000","last_undegraded":"2024-04-25T14:13:33.244471+0000","last_fullsized":"2024-04-25T14:13:33.244471+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:16.518784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.226798+0000","last_change":"2024-04-25T14:13:33.226798+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.226798+0000","last_undegraded":"2024-04-25T14:13:33.226798+0000","last_fullsized":"2024-04-25T14:13:33.226798+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.241274+0000","last_change":"2024-04-25T14:13:33.233642+0000","last_active":"2024-04-25T14:13:33.241274+0000","last_peered":"2024-04-25T14:13:33.241274+0000","last_clean":"2024-04-25T14:13:33.241274+0000","last_became_active":"2024-04-25T14:13:33.233355+0000","last_became_peered":"2024-04-25T14:13:33.233355+0000","last_unstale":"2024-04-25T14:13:33.241274+0000","last_undegraded":"2024-04-25T14:13:33.241274+0000","last_fullsized":"2024-04-25T14:13:33.241274+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:37.218976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.228411+0000","last_change":"2024-04-25T14:13:33.228411+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.228411+0000","last_undegraded":"2024-04-25T14:13:33.228411+0000","last_fullsized":"2024-04-25T14:13:33.228411+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.241759+0000","last_change":"2024-04-25T14:13:33.239682+0000","last_active":"2024-04-25T14:13:33.241759+0000","last_peered":"2024-04-25T14:13:33.241759+0000","last_clean":"2024-04-25T14:13:33.241759+0000","last_became_active":"2024-04-25T14:13:33.239378+0000","last_became_peered":"2024-04-25T14:13:33.239378+0000","last_unstale":"2024-04-25T14:13:33.241759+0000","last_undegraded":"2024-04-25T14:13:33.241759+0000","last_fullsized":"2024-04-25T14:13:33.241759+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:10.873169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.217318+0000","last_change":"2024-04-25T14:13:33.217318+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.217318+0000","last_undegraded":"2024-04-25T14:13:33.217318+0000","last_fullsized":"2024-04-25T14:13:33.217318+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.238019+0000","last_change":"2024-04-25T14:13:33.232394+0000","last_active":"2024-04-25T14:13:33.238019+0000","last_peered":"2024-04-25T14:13:33.238019+0000","last_clean":"2024-04-25T14:13:33.238019+0000","last_became_active":"2024-04-25T14:13:33.231789+0000","last_became_peered":"2024-04-25T14:13:33.231789+0000","last_unstale":"2024-04-25T14:13:33.238019+0000","last_undegraded":"2024-04-25T14:13:33.238019+0000","last_fullsized":"2024-04-25T14:13:33.238019+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:45.600418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.218265+0000","last_change":"2024-04-25T14:13:33.218265+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.218265+0000","last_undegraded":"2024-04-25T14:13:33.218265+0000","last_fullsized":"2024-04-25T14:13:33.218265+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.243068+0000","last_change":"2024-04-25T14:13:33.235450+0000","last_active":"2024-04-25T14:13:33.243068+0000","last_peered":"2024-04-25T14:13:33.243068+0000","last_clean":"2024-04-25T14:13:33.243068+0000","last_became_active":"2024-04-25T14:13:33.235152+0000","last_became_peered":"2024-04-25T14:13:33.235152+0000","last_unstale":"2024-04-25T14:13:33.243068+0000","last_undegraded":"2024-04-25T14:13:33.243068+0000","last_fullsized":"2024-04-25T14:13:33.243068+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:35.585117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219518+0000","last_change":"2024-04-25T14:13:33.219518+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219518+0000","last_undegraded":"2024-04-25T14:13:33.219518+0000","last_fullsized":"2024-04-25T14:13:33.219518+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.238678+0000","last_change":"2024-04-25T14:13:33.235678+0000","last_active":"2024-04-25T14:13:33.238678+0000","last_peered":"2024-04-25T14:13:33.238678+0000","last_clean":"2024-04-25T14:13:33.238678+0000","last_became_active":"2024-04-25T14:13:33.235038+0000","last_became_peered":"2024-04-25T14:13:33.235038+0000","last_unstale":"2024-04-25T14:13:33.238678+0000","last_undegraded":"2024-04-25T14:13:33.238678+0000","last_fullsized":"2024-04-25T14:13:33.238678+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:23.471943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.225982+0000","last_change":"2024-04-25T14:13:33.225982+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.225982+0000","last_undegraded":"2024-04-25T14:13:33.225982+0000","last_fullsized":"2024-04-25T14:13:33.225982+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.243523+0000","last_change":"2024-04-25T14:13:33.235754+0000","last_active":"2024-04-25T14:13:33.243523+0000","last_peered":"2024-04-25T14:13:33.243523+0000","last_clean":"2024-04-25T14:13:33.243523+0000","last_became_active":"2024-04-25T14:13:33.235119+0000","last_became_peered":"2024-04-25T14:13:33.235119+0000","last_unstale":"2024-04-25T14:13:33.243523+0000","last_undegraded":"2024-04-25T14:13:33.243523+0000","last_fullsized":"2024-04-25T14:13:33.243523+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:09.237922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.222837+0000","last_change":"2024-04-25T14:13:33.222837+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.222837+0000","last_undegraded":"2024-04-25T14:13:33.222837+0000","last_fullsized":"2024-04-25T14:13:33.222837+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.240723+0000","last_change":"2024-04-25T14:13:33.237307+0000","last_active":"2024-04-25T14:13:33.240723+0000","last_peered":"2024-04-25T14:13:33.240723+0000","last_clean":"2024-04-25T14:13:33.240723+0000","last_became_active":"2024-04-25T14:13:33.237012+0000","last_became_peered":"2024-04-25T14:13:33.237012+0000","last_unstale":"2024-04-25T14:13:33.240723+0000","last_undegraded":"2024-04-25T14:13:33.240723+0000","last_fullsized":"2024-04-25T14:13:33.240723+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:45.356743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.222740+0000","last_change":"2024-04-25T14:13:33.222740+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.222740+0000","last_undegraded":"2024-04-25T14:13:33.222740+0000","last_fullsized":"2024-04-25T14:13:33.222740+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.241914+0000","last_change":"2024-04-25T14:13:33.234217+0000","last_active":"2024-04-25T14:13:33.241914+0000","last_peered":"2024-04-25T14:13:33.241914+0000","last_clean":"2024-04-25T14:13:33.241914+0000","last_became_active":"2024-04-25T14:13:33.233902+0000","last_became_peered":"2024-04-25T14:13:33.233902+0000","last_unstale":"2024-04-25T14:13:33.241914+0000","last_undegraded":"2024-04-25T14:13:33.241914+0000","last_fullsized":"2024-04-25T14:13:33.241914+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:09.455778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.227060+0000","last_change":"2024-04-25T14:13:33.227060+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.227060+0000","last_undegraded":"2024-04-25T14:13:33.227060+0000","last_fullsized":"2024-04-25T14:13:33.227060+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.240405+0000","last_change":"2024-04-25T14:13:33.239716+0000","last_active":"2024-04-25T14:13:33.240405+0000","last_peered":"2024-04-25T14:13:33.240405+0000","last_clean":"2024-04-25T14:13:33.240405+0000","last_became_active":"2024-04-25T14:13:33.239375+0000","last_became_peered":"2024-04-25T14:13:33.239375+0000","last_unstale":"2024-04-25T14:13:33.240405+0000","last_undegraded":"2024-04-25T14:13:33.240405+0000","last_fullsized":"2024-04-25T14:13:33.240405+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:05.463091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.226077+0000","last_change":"2024-04-25T14:13:33.226077+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.226077+0000","last_undegraded":"2024-04-25T14:13:33.226077+0000","last_fullsized":"2024-04-25T14:13:33.226077+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.239375+0000","last_change":"2024-04-25T14:13:33.238315+0000","last_active":"2024-04-25T14:13:33.239375+0000","last_peered":"2024-04-25T14:13:33.239375+0000","last_clean":"2024-04-25T14:13:33.239375+0000","last_became_active":"2024-04-25T14:13:33.237955+0000","last_became_peered":"2024-04-25T14:13:33.237955+0000","last_unstale":"2024-04-25T14:13:33.239375+0000","last_undegraded":"2024-04-25T14:13:33.239375+0000","last_fullsized":"2024-04-25T14:13:33.239375+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:50.318605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.223560+0000","last_change":"2024-04-25T14:13:33.223560+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.223560+0000","last_undegraded":"2024-04-25T14:13:33.223560+0000","last_fullsized":"2024-04-25T14:13:33.223560+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.237567+0000","last_change":"2024-04-25T14:13:33.226581+0000","last_active":"2024-04-25T14:13:33.237567+0000","last_peered":"2024-04-25T14:13:33.237567+0000","last_clean":"2024-04-25T14:13:33.237567+0000","last_became_active":"2024-04-25T14:13:33.225988+0000","last_became_peered":"2024-04-25T14:13:33.225988+0000","last_unstale":"2024-04-25T14:13:33.237567+0000","last_undegraded":"2024-04-25T14:13:33.237567+0000","last_fullsized":"2024-04-25T14:13:33.237567+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:08.407184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.224808+0000","last_change":"2024-04-25T14:13:33.224808+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.224808+0000","last_undegraded":"2024-04-25T14:13:33.224808+0000","last_fullsized":"2024-04-25T14:13:33.224808+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.244892+0000","last_change":"2024-04-25T14:13:33.239707+0000","last_active":"2024-04-25T14:13:33.244892+0000","last_peered":"2024-04-25T14:13:33.244892+0000","last_clean":"2024-04-25T14:13:33.244892+0000","last_became_active":"2024-04-25T14:13:33.239408+0000","last_became_peered":"2024-04-25T14:13:33.239408+0000","last_unstale":"2024-04-25T14:13:33.244892+0000","last_undegraded":"2024-04-25T14:13:33.244892+0000","last_fullsized":"2024-04-25T14:13:33.244892+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:51:01.497289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.226373+0000","last_change":"2024-04-25T14:13:33.226373+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.226373+0000","last_undegraded":"2024-04-25T14:13:33.226373+0000","last_fullsized":"2024-04-25T14:13:33.226373+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.239011+0000","last_change":"2024-04-25T14:13:33.232965+0000","last_active":"2024-04-25T14:13:33.239011+0000","last_peered":"2024-04-25T14:13:33.239011+0000","last_clean":"2024-04-25T14:13:33.239011+0000","last_became_active":"2024-04-25T14:13:33.232647+0000","last_became_peered":"2024-04-25T14:13:33.232647+0000","last_unstale":"2024-04-25T14:13:33.239011+0000","last_undegraded":"2024-04-25T14:13:33.239011+0000","last_fullsized":"2024-04-25T14:13:33.239011+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:12.089780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.229146+0000","last_change":"2024-04-25T14:13:33.229146+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.229146+0000","last_undegraded":"2024-04-25T14:13:33.229146+0000","last_fullsized":"2024-04-25T14:13:33.229146+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.244360+0000","last_change":"2024-04-25T14:13:33.240376+0000","last_active":"2024-04-25T14:13:33.244360+0000","last_peered":"2024-04-25T14:13:33.244360+0000","last_clean":"2024-04-25T14:13:33.244360+0000","last_became_active":"2024-04-25T14:13:33.240103+0000","last_became_peered":"2024-04-25T14:13:33.240103+0000","last_unstale":"2024-04-25T14:13:33.244360+0000","last_undegraded":"2024-04-25T14:13:33.244360+0000","last_fullsized":"2024-04-25T14:13:33.244360+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:55.225878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219516+0000","last_change":"2024-04-25T14:13:33.219516+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219516+0000","last_undegraded":"2024-04-25T14:13:33.219516+0000","last_fullsized":"2024-04-25T14:13:33.219516+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.240700+0000","last_change":"2024-04-25T14:13:33.231549+0000","last_active":"2024-04-25T14:13:33.240700+0000","last_peered":"2024-04-25T14:13:33.240700+0000","last_clean":"2024-04-25T14:13:33.240700+0000","last_became_active":"2024-04-25T14:13:33.231239+0000","last_became_peered":"2024-04-25T14:13:33.231239+0000","last_unstale":"2024-04-25T14:13:33.240700+0000","last_undegraded":"2024-04-25T14:13:33.240700+0000","last_fullsized":"2024-04-25T14:13:33.240700+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:46.823734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219647+0000","last_change":"2024-04-25T14:13:33.219647+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219647+0000","last_undegraded":"2024-04-25T14:13:33.219647+0000","last_fullsized":"2024-04-25T14:13:33.219647+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.239404+0000","last_change":"2024-04-25T14:13:33.238316+0000","last_active":"2024-04-25T14:13:33.239404+0000","last_peered":"2024-04-25T14:13:33.239404+0000","last_clean":"2024-04-25T14:13:33.239404+0000","last_became_active":"2024-04-25T14:13:33.237984+0000","last_became_peered":"2024-04-25T14:13:33.237984+0000","last_unstale":"2024-04-25T14:13:33.239404+0000","last_undegraded":"2024-04-25T14:13:33.239404+0000","last_fullsized":"2024-04-25T14:13:33.239404+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:01.863933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.220719+0000","last_change":"2024-04-25T14:13:33.220719+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.220719+0000","last_undegraded":"2024-04-25T14:13:33.220719+0000","last_fullsized":"2024-04-25T14:13:33.220719+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.240694+0000","last_change":"2024-04-25T14:13:33.233345+0000","last_active":"2024-04-25T14:13:33.240694+0000","last_peered":"2024-04-25T14:13:33.240694+0000","last_clean":"2024-04-25T14:13:33.240694+0000","last_became_active":"2024-04-25T14:13:33.233045+0000","last_became_peered":"2024-04-25T14:13:33.233045+0000","last_unstale":"2024-04-25T14:13:33.240694+0000","last_undegraded":"2024-04-25T14:13:33.240694+0000","last_fullsized":"2024-04-25T14:13:33.240694+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:28.978236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.218329+0000","last_change":"2024-04-25T14:13:33.218329+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.218329+0000","last_undegraded":"2024-04-25T14:13:33.218329+0000","last_fullsized":"2024-04-25T14:13:33.218329+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.235430+0000","last_change":"2024-04-25T14:13:33.233786+0000","last_active":"2024-04-25T14:13:33.235430+0000","last_peered":"2024-04-25T14:13:33.235430+0000","last_clean":"2024-04-25T14:13:33.235430+0000","last_became_active":"2024-04-25T14:13:33.233479+0000","last_became_peered":"2024-04-25T14:13:33.233479+0000","last_unstale":"2024-04-25T14:13:33.235430+0000","last_undegraded":"2024-04-25T14:13:33.235430+0000","last_fullsized":"2024-04-25T14:13:33.235430+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:57.369446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.225358+0000","last_change":"2024-04-25T14:13:33.225358+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.225358+0000","last_undegraded":"2024-04-25T14:13:33.225358+0000","last_fullsized":"2024-04-25T14:13:33.225358+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.239073+0000","last_change":"2024-04-25T14:13:33.235217+0000","last_active":"2024-04-25T14:13:33.239073+0000","last_peered":"2024-04-25T14:13:33.239073+0000","last_clean":"2024-04-25T14:13:33.239073+0000","last_became_active":"2024-04-25T14:13:33.234923+0000","last_became_peered":"2024-04-25T14:13:33.234923+0000","last_unstale":"2024-04-25T14:13:33.239073+0000","last_undegraded":"2024-04-25T14:13:33.239073+0000","last_fullsized":"2024-04-25T14:13:33.239073+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:16.491560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.223188+0000","last_change":"2024-04-25T14:13:33.223188+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.223188+0000","last_undegraded":"2024-04-25T14:13:33.223188+0000","last_fullsized":"2024-04-25T14:13:33.223188+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.237237+0000","last_change":"2024-04-25T14:13:33.233600+0000","last_active":"2024-04-25T14:13:33.237237+0000","last_peered":"2024-04-25T14:13:33.237237+0000","last_clean":"2024-04-25T14:13:33.237237+0000","last_became_active":"2024-04-25T14:13:33.233290+0000","last_became_peered":"2024-04-25T14:13:33.233290+0000","last_unstale":"2024-04-25T14:13:33.237237+0000","last_undegraded":"2024-04-25T14:13:33.237237+0000","last_fullsized":"2024-04-25T14:13:33.237237+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:16.534415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.226375+0000","last_change":"2024-04-25T14:13:33.226375+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.226375+0000","last_undegraded":"2024-04-25T14:13:33.226375+0000","last_fullsized":"2024-04-25T14:13:33.226375+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.236234+0000","last_change":"2024-04-25T14:13:33.229751+0000","last_active":"2024-04-25T14:13:33.236234+0000","last_peered":"2024-04-25T14:13:33.236234+0000","last_clean":"2024-04-25T14:13:33.236234+0000","last_became_active":"2024-04-25T14:13:33.229451+0000","last_became_peered":"2024-04-25T14:13:33.229451+0000","last_unstale":"2024-04-25T14:13:33.236234+0000","last_undegraded":"2024-04-25T14:13:33.236234+0000","last_fullsized":"2024-04-25T14:13:33.236234+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:34.938781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.220859+0000","last_change":"2024-04-25T14:13:33.220859+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.220859+0000","last_undegraded":"2024-04-25T14:13:33.220859+0000","last_fullsized":"2024-04-25T14:13:33.220859+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.246616+0000","last_change":"2024-04-25T14:13:33.245950+0000","last_active":"2024-04-25T14:13:33.246616+0000","last_peered":"2024-04-25T14:13:33.246616+0000","last_clean":"2024-04-25T14:13:33.246616+0000","last_became_active":"2024-04-25T14:13:33.245627+0000","last_became_peered":"2024-04-25T14:13:33.245627+0000","last_unstale":"2024-04-25T14:13:33.246616+0000","last_undegraded":"2024-04-25T14:13:33.246616+0000","last_fullsized":"2024-04-25T14:13:33.246616+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:59.112002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.216195+0000","last_change":"2024-04-25T14:13:33.216195+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.216195+0000","last_undegraded":"2024-04-25T14:13:33.216195+0000","last_fullsized":"2024-04-25T14:13:33.216195+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.242071+0000","last_change":"2024-04-25T14:13:33.241343+0000","last_active":"2024-04-25T14:13:33.242071+0000","last_peered":"2024-04-25T14:13:33.242071+0000","last_clean":"2024-04-25T14:13:33.242071+0000","last_became_active":"2024-04-25T14:13:33.240975+0000","last_became_peered":"2024-04-25T14:13:33.240975+0000","last_unstale":"2024-04-25T14:13:33.242071+0000","last_undegraded":"2024-04-25T14:13:33.242071+0000","last_fullsized":"2024-04-25T14:13:33.242071+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:43.434745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.217868+0000","last_change":"2024-04-25T14:13:33.217868+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.217868+0000","last_undegraded":"2024-04-25T14:13:33.217868+0000","last_fullsized":"2024-04-25T14:13:33.217868+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.236031+0000","last_change":"2024-04-25T14:13:33.230789+0000","last_active":"2024-04-25T14:13:33.236031+0000","last_peered":"2024-04-25T14:13:33.236031+0000","last_clean":"2024-04-25T14:13:33.236031+0000","last_became_active":"2024-04-25T14:13:33.230422+0000","last_became_peered":"2024-04-25T14:13:33.230422+0000","last_unstale":"2024-04-25T14:13:33.236031+0000","last_undegraded":"2024-04-25T14:13:33.236031+0000","last_fullsized":"2024-04-25T14:13:33.236031+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:05:59.451071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219902+0000","last_change":"2024-04-25T14:13:33.219902+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219902+0000","last_undegraded":"2024-04-25T14:13:33.219902+0000","last_fullsized":"2024-04-25T14:13:33.219902+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.242301+0000","last_change":"2024-04-25T14:13:33.241318+0000","last_active":"2024-04-25T14:13:33.242301+0000","last_peered":"2024-04-25T14:13:33.242301+0000","last_clean":"2024-04-25T14:13:33.242301+0000","last_became_active":"2024-04-25T14:13:33.240931+0000","last_became_peered":"2024-04-25T14:13:33.240931+0000","last_unstale":"2024-04-25T14:13:33.242301+0000","last_undegraded":"2024-04-25T14:13:33.242301+0000","last_fullsized":"2024-04-25T14:13:33.242301+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:39.911385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.225571+0000","last_change":"2024-04-25T14:13:33.225571+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.225571+0000","last_undegraded":"2024-04-25T14:13:33.225571+0000","last_fullsized":"2024-04-25T14:13:33.225571+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.238829+0000","last_change":"2024-04-25T14:13:33.229866+0000","last_active":"2024-04-25T14:13:33.238829+0000","last_peered":"2024-04-25T14:13:33.238829+0000","last_clean":"2024-04-25T14:13:33.238829+0000","last_became_active":"2024-04-25T14:13:33.229562+0000","last_became_peered":"2024-04-25T14:13:33.229562+0000","last_unstale":"2024-04-25T14:13:33.238829+0000","last_undegraded":"2024-04-25T14:13:33.238829+0000","last_fullsized":"2024-04-25T14:13:33.238829+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:00.731593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.218315+0000","last_change":"2024-04-25T14:13:33.218315+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.218315+0000","last_undegraded":"2024-04-25T14:13:33.218315+0000","last_fullsized":"2024-04-25T14:13:33.218315+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.244337+0000","last_change":"2024-04-25T14:13:33.242678+0000","last_active":"2024-04-25T14:13:33.244337+0000","last_peered":"2024-04-25T14:13:33.244337+0000","last_clean":"2024-04-25T14:13:33.244337+0000","last_became_active":"2024-04-25T14:13:33.241884+0000","last_became_peered":"2024-04-25T14:13:33.241884+0000","last_unstale":"2024-04-25T14:13:33.244337+0000","last_undegraded":"2024-04-25T14:13:33.244337+0000","last_fullsized":"2024-04-25T14:13:33.244337+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:00.731593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"144'3291","reported_seq":3828,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.226260+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:13:33.226260+0000","last_peered":"2024-04-25T14:13:33.226260+0000","last_clean":"2024-04-25T14:13:33.226260+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:13:33.226260+0000","last_undegraded":"2024-04-25T14:13:33.226260+0000","last_fullsized":"2024-04-25T14:13:33.226260+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":3291,"log_dups_size":0,"ondisk_log_size":3291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":837,"num_read_kb":2279,"num_write":247,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":837,"num_read_kb":2279,"num_write":247,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3291,"ondisk_log_size":3291,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574963,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29484,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342356,"statfs":{"total":96636764160,"available":96606572544,"internally_reserved":0,"allocated":2519040,"data_stored":1556182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574964,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33064,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1925120,"data_stored":965814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574965,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33064,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1925120,"data_stored":965814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574964,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33064,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1925120,"data_stored":965814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574964,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33068,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1925120,"data_stored":965814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33048,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338792,"statfs":{"total":96636764160,"available":96602923008,"internally_reserved":0,"allocated":1912832,"data_stored":959155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33064,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1925120,"data_stored":965814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574964,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29484,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342356,"statfs":{"total":96636764160,"available":96606572544,"internally_reserved":0,"allocated":2519040,"data_stored":1556182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:13:38.741 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:13:38.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.927+0000 7fb39ba00640 1 Processor -- start 2024-04-25T14:13:38.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.927+0000 7fb39ba00640 1 -- start start 2024-04-25T14:13:38.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.927+0000 7fb39ba00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb3940cf970 0x7fb394070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.927+0000 7fb39ba00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb394071360 0x7fb39406e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.927+0000 7fb39ba00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb394071770 con 0x7fb39406e7a0 2024-04-25T14:13:38.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.927+0000 7fb39ba00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb3940cedf0 con 0x7fb394071360 2024-04-25T14:13:38.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.927+0000 7fb39ba00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb394163830 con 0x7fb3940cf970 2024-04-25T14:13:38.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.927+0000 7fb38be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb394071360 0x7fb39406e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.927+0000 7fb399a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb39406e7a0 0x7fb3940722f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45200/0 (socket says 172.21.15.86:45200) 2024-04-25T14:13:38.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.927+0000 7fb399a00640 1 -- 172.21.15.86:0/3359016349 learned_addr learned my addr 172.21.15.86:0/3359016349 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:38.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.927+0000 7fb399000640 1 --2- 172.21.15.86:0/3359016349 >> v2:172.21.15.155:3300/0 conn(0x7fb3940cf970 0x7fb394070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.931+0000 7fb393e00640 1 -- 172.21.15.86:0/3359016349 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 80984438 0 0) 0x7fb38c0014a0 con 0x7fb39406e7a0 2024-04-25T14:13:38.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.931+0000 7fb393e00640 1 -- 172.21.15.86:0/3359016349 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb378003530 con 0x7fb39406e7a0 2024-04-25T14:13:38.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.931+0000 7fb38be00640 1 -- 172.21.15.86:0/3359016349 >> v2:172.21.15.155:3300/0 conn(0x7fb3940cf970 msgr2=0x7fb394070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:38.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.931+0000 7fb38be00640 1 --2- 172.21.15.86:0/3359016349 >> v2:172.21.15.155:3300/0 conn(0x7fb3940cf970 0x7fb394070e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.931+0000 7fb38be00640 1 -- 172.21.15.86:0/3359016349 >> v1:172.21.15.86:6789/0 conn(0x7fb39406e7a0 legacy=0x7fb3940722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.931+0000 7fb38be00640 1 -- 172.21.15.86:0/3359016349 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb39406ef90 con 0x7fb394071360 2024-04-25T14:13:38.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.931+0000 7fb38be00640 1 --2- 172.21.15.86:0/3359016349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb394071360 0x7fb39406e260 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7fb394073c10 tx=0x7fb37c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840e8130984f7c85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:38.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.931+0000 7fb393e00640 1 -- 172.21.15.86:0/3359016349 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37c0b0070 con 0x7fb394071360 2024-04-25T14:13:38.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.931+0000 7fb393e00640 1 -- 172.21.15.86:0/3359016349 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb37c004230 con 0x7fb394071360 2024-04-25T14:13:38.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.931+0000 7fb393e00640 1 -- 172.21.15.86:0/3359016349 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37c003620 con 0x7fb394071360 2024-04-25T14:13:38.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.931+0000 7fb39ba00640 1 -- 172.21.15.86:0/3359016349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb394071360 msgr2=0x7fb39406e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.931+0000 7fb39ba00640 1 --2- 172.21.15.86:0/3359016349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb394071360 0x7fb39406e260 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7fb394073c10 tx=0x7fb37c01c430 comp rx=0 tx=0).stop 2024-04-25T14:13:38.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.935+0000 7fb39ba00640 1 -- 172.21.15.86:0/3359016349 shutdown_connections 2024-04-25T14:13:38.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.935+0000 7fb39ba00640 1 --2- 172.21.15.86:0/3359016349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb394071360 0x7fb39406e260 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.935+0000 7fb39ba00640 1 --2- 172.21.15.86:0/3359016349 >> v2:172.21.15.155:3300/0 conn(0x7fb3940cf970 0x7fb394070e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.935+0000 7fb39ba00640 1 -- 172.21.15.86:0/3359016349 >> 172.21.15.86:0/3359016349 conn(0x7fb3940cffc0 msgr2=0x7fb3940d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:38.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.935+0000 7fb39ba00640 1 -- 172.21.15.86:0/3359016349 shutdown_connections 2024-04-25T14:13:38.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.935+0000 7fb39ba00640 1 -- 172.21.15.86:0/3359016349 wait complete. 2024-04-25T14:13:38.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.935+0000 7fb39ba00640 1 Processor -- start 2024-04-25T14:13:38.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.935+0000 7fb39ba00640 1 -- start start 2024-04-25T14:13:38.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.935+0000 7fb39ba00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb39406e7a0 0x7fb3941518e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.935+0000 7fb39ba00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb394071360 0x7fb39415e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.935+0000 7fb39ba00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3940cf970 0x7fb39415ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.935+0000 7fb39ba00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb39406ed30 con 0x7fb3940cf970 2024-04-25T14:13:38.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.935+0000 7fb39ba00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb39406ebb0 con 0x7fb39406e7a0 2024-04-25T14:13:38.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.935+0000 7fb39ba00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb39416cbd0 con 0x7fb394071360 2024-04-25T14:13:38.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.939+0000 7fb399a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3940cf970 0x7fb39415ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.939+0000 7fb399a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3940cf970 0x7fb39415ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41604/0 (socket says 172.21.15.86:41604) 2024-04-25T14:13:38.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.939+0000 7fb399000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb39406e7a0 0x7fb3941518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.939+0000 7fb399a00640 1 -- 172.21.15.86:0/2433783021 learned_addr learned my addr 172.21.15.86:0/2433783021 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:38.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.939+0000 7fb38be00640 1 --2- 172.21.15.86:0/2433783021 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb394071360 0x7fb39415e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.939+0000 7fb399a00640 1 -- 172.21.15.86:0/2433783021 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb394071360 msgr2=0x7fb39415e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.939+0000 7fb399a00640 1 --2- 172.21.15.86:0/2433783021 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb394071360 0x7fb39415e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.939+0000 7fb399a00640 1 -- 172.21.15.86:0/2433783021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb39406e7a0 msgr2=0x7fb3941518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:38.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.939+0000 7fb399a00640 1 --2- 172.21.15.86:0/2433783021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb39406e7a0 0x7fb3941518e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:38.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.939+0000 7fb399a00640 1 -- 172.21.15.86:0/2433783021 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb394153a90 con 0x7fb3940cf970 2024-04-25T14:13:38.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.939+0000 7fb399a00640 1 --2- 172.21.15.86:0/2433783021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3940cf970 0x7fb39415ea20 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00cb40 tx=0x7fb38c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:38.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.939+0000 7fb391600640 1 -- 172.21.15.86:0/2433783021 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c017070 con 0x7fb3940cf970 2024-04-25T14:13:38.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.939+0000 7fb391600640 1 -- 172.21.15.86:0/2433783021 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb38c00ac40 con 0x7fb3940cf970 2024-04-25T14:13:38.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.939+0000 7fb39ba00640 1 -- 172.21.15.86:0/2433783021 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3941585b0 con 0x7fb3940cf970 2024-04-25T14:13:38.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.939+0000 7fb391600640 1 -- 172.21.15.86:0/2433783021 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c012400 con 0x7fb3940cf970 2024-04-25T14:13:38.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.939+0000 7fb39ba00640 1 -- 172.21.15.86:0/2433783021 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb394158b70 con 0x7fb3940cf970 2024-04-25T14:13:38.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.939+0000 7fb39ba00640 1 -- 172.21.15.86:0/2433783021 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3940720e0 con 0x7fb3940cf970 2024-04-25T14:13:38.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.939+0000 7fb391600640 1 -- 172.21.15.86:0/2433783021 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7fb38c01a070 con 0x7fb3940cf970 2024-04-25T14:13:38.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.943+0000 7fb391600640 1 --2- 172.21.15.86:0/2433783021 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fb37006f550 0x7fb370071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:38.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.943+0000 7fb391600640 1 -- 172.21.15.86:0/2433783021 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6101+0+0 (secure 0 0 0) 0x7fb38c08f310 con 0x7fb3940cf970 2024-04-25T14:13:38.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.943+0000 7fb399000640 1 --2- 172.21.15.86:0/2433783021 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fb37006f550 0x7fb370071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:38.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.943+0000 7fb399000640 1 --2- 172.21.15.86:0/2433783021 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fb37006f550 0x7fb370071a60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb384009ac0 tx=0x7fb384009190 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:38.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:38.947+0000 7fb391600640 1 -- 172.21.15.86:0/2433783021 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb394071770 con 0x7fb3940cf970 2024-04-25T14:13:39.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.099+0000 7fb39ba00640 1 -- 172.21.15.86:0/2433783021 --> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb394069d60 con 0x7fb37006f550 2024-04-25T14:13:39.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.107+0000 7fb391600640 1 -- 172.21.15.86:0/2433783021 <== mgr.8198 v2:172.21.15.86:6832/505730993 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247100 (secure 0 0 0) 0x7fb3840119e0 con 0x7fb37006f550 2024-04-25T14:13:39.108 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:13:39.109 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:13:39.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.111+0000 7fb39ba00640 1 -- 172.21.15.86:0/2433783021 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fb37006f550 msgr2=0x7fb370071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.111+0000 7fb39ba00640 1 --2- 172.21.15.86:0/2433783021 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fb37006f550 0x7fb370071a60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb384009ac0 tx=0x7fb384009190 comp rx=0 tx=0).stop 2024-04-25T14:13:39.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.111+0000 7fb39ba00640 1 -- 172.21.15.86:0/2433783021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3940cf970 msgr2=0x7fb39415ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.111+0000 7fb39ba00640 1 --2- 172.21.15.86:0/2433783021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3940cf970 0x7fb39415ea20 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00cb40 tx=0x7fb38c00a060 comp rx=0 tx=0).stop 2024-04-25T14:13:39.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.111+0000 7fb39ba00640 1 -- 172.21.15.86:0/2433783021 shutdown_connections 2024-04-25T14:13:39.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.111+0000 7fb39ba00640 1 --2- 172.21.15.86:0/2433783021 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fb37006f550 0x7fb370071a60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.111+0000 7fb39ba00640 1 --2- 172.21.15.86:0/2433783021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3940cf970 0x7fb39415ea20 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.111+0000 7fb39ba00640 1 --2- 172.21.15.86:0/2433783021 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb394071360 0x7fb39415e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.111+0000 7fb39ba00640 1 --2- 172.21.15.86:0/2433783021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb39406e7a0 0x7fb3941518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.111+0000 7fb39ba00640 1 -- 172.21.15.86:0/2433783021 >> 172.21.15.86:0/2433783021 conn(0x7fb3940cffc0 msgr2=0x7fb394156700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.111+0000 7fb39ba00640 1 -- 172.21.15.86:0/2433783021 shutdown_connections 2024-04-25T14:13:39.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.111+0000 7fb39ba00640 1 -- 172.21.15.86:0/2433783021 wait complete. 2024-04-25T14:13:39.129 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T14:13:38.077493+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":837,"num_read_kb":2279,"num_write":247,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3291,"ondisk_log_size":3291,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257340,"kb_used_data":16188,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754717380,"statfs":{"total":773094113280,"available":772830597120,"internally_reserved":0,"allocated":16576512,"data_stored":8900589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.870692"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.217839+0000","last_change":"2024-04-25T14:13:33.217839+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.217839+0000","last_undegraded":"2024-04-25T14:13:33.217839+0000","last_fullsized":"2024-04-25T14:13:33.217839+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.220477+0000","last_change":"2024-04-25T14:13:33.220477+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.220477+0000","last_undegraded":"2024-04-25T14:13:33.220477+0000","last_fullsized":"2024-04-25T14:13:33.220477+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.216321+0000","last_change":"2024-04-25T14:13:33.216321+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.216321+0000","last_undegraded":"2024-04-25T14:13:33.216321+0000","last_fullsized":"2024-04-25T14:13:33.216321+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.216741+0000","last_change":"2024-04-25T14:13:33.216741+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.216741+0000","last_undegraded":"2024-04-25T14:13:33.216741+0000","last_fullsized":"2024-04-25T14:13:33.216741+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219621+0000","last_change":"2024-04-25T14:13:33.219621+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219621+0000","last_undegraded":"2024-04-25T14:13:33.219621+0000","last_fullsized":"2024-04-25T14:13:33.219621+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.221767+0000","last_change":"2024-04-25T14:13:33.221767+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.221767+0000","last_undegraded":"2024-04-25T14:13:33.221767+0000","last_fullsized":"2024-04-25T14:13:33.221767+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.223617+0000","last_change":"2024-04-25T14:13:33.223617+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.223617+0000","last_undegraded":"2024-04-25T14:13:33.223617+0000","last_fullsized":"2024-04-25T14:13:33.223617+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.228099+0000","last_change":"2024-04-25T14:13:33.228099+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.228099+0000","last_undegraded":"2024-04-25T14:13:33.228099+0000","last_fullsized":"2024-04-25T14:13:33.228099+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.222924+0000","last_change":"2024-04-25T14:13:33.222924+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.222924+0000","last_undegraded":"2024-04-25T14:13:33.222924+0000","last_fullsized":"2024-04-25T14:13:33.222924+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.229249+0000","last_change":"2024-04-25T14:13:33.229249+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.229249+0000","last_undegraded":"2024-04-25T14:13:33.229249+0000","last_fullsized":"2024-04-25T14:13:33.229249+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.231347+0000","last_change":"2024-04-25T14:13:33.231347+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.231347+0000","last_undegraded":"2024-04-25T14:13:33.231347+0000","last_fullsized":"2024-04-25T14:13:33.231347+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.222546+0000","last_change":"2024-04-25T14:13:33.222546+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.222546+0000","last_undegraded":"2024-04-25T14:13:33.222546+0000","last_fullsized":"2024-04-25T14:13:33.222546+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.224550+0000","last_change":"2024-04-25T14:13:33.224550+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.224550+0000","last_undegraded":"2024-04-25T14:13:33.224550+0000","last_fullsized":"2024-04-25T14:13:33.224550+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219360+0000","last_change":"2024-04-25T14:13:33.219360+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219360+0000","last_undegraded":"2024-04-25T14:13:33.219360+0000","last_fullsized":"2024-04-25T14:13:33.219360+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.229250+0000","last_change":"2024-04-25T14:13:33.229250+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.229250+0000","last_undegraded":"2024-04-25T14:13:33.229250+0000","last_fullsized":"2024-04-25T14:13:33.229250+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.216778+0000","last_change":"2024-04-25T14:13:33.216778+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.216778+0000","last_undegraded":"2024-04-25T14:13:33.216778+0000","last_fullsized":"2024-04-25T14:13:33.216778+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.222484+0000","last_change":"2024-04-25T14:13:33.222484+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.222484+0000","last_undegraded":"2024-04-25T14:13:33.222484+0000","last_fullsized":"2024-04-25T14:13:33.222484+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.225152+0000","last_change":"2024-04-25T14:13:33.225152+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.225152+0000","last_undegraded":"2024-04-25T14:13:33.225152+0000","last_fullsized":"2024-04-25T14:13:33.225152+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.228898+0000","last_change":"2024-04-25T14:13:33.228898+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.228898+0000","last_undegraded":"2024-04-25T14:13:33.228898+0000","last_fullsized":"2024-04-25T14:13:33.228898+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.229071+0000","last_change":"2024-04-25T14:13:33.229071+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.229071+0000","last_undegraded":"2024-04-25T14:13:33.229071+0000","last_fullsized":"2024-04-25T14:13:33.229071+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.224104+0000","last_change":"2024-04-25T14:13:33.224104+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.224104+0000","last_undegraded":"2024-04-25T14:13:33.224104+0000","last_fullsized":"2024-04-25T14:13:33.224104+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.221177+0000","last_change":"2024-04-25T14:13:33.221177+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.221177+0000","last_undegraded":"2024-04-25T14:13:33.221177+0000","last_fullsized":"2024-04-25T14:13:33.221177+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.218713+0000","last_change":"2024-04-25T14:13:33.218713+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.218713+0000","last_undegraded":"2024-04-25T14:13:33.218713+0000","last_fullsized":"2024-04-25T14:13:33.218713+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.215974+0000","last_change":"2024-04-25T14:13:33.215974+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.215974+0000","last_undegraded":"2024-04-25T14:13:33.215974+0000","last_fullsized":"2024-04-25T14:13:33.215974+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.222118+0000","last_change":"2024-04-25T14:13:33.222118+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.222118+0000","last_undegraded":"2024-04-25T14:13:33.222118+0000","last_fullsized":"2024-04-25T14:13:33.222118+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.228839+0000","last_change":"2024-04-25T14:13:33.228839+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.228839+0000","last_undegraded":"2024-04-25T14:13:33.228839+0000","last_fullsized":"2024-04-25T14:13:33.228839+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.221814+0000","last_change":"2024-04-25T14:13:33.221814+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.221814+0000","last_undegraded":"2024-04-25T14:13:33.221814+0000","last_fullsized":"2024-04-25T14:13:33.221814+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.223958+0000","last_change":"2024-04-25T14:13:33.223958+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.223958+0000","last_undegraded":"2024-04-25T14:13:33.223958+0000","last_fullsized":"2024-04-25T14:13:33.223958+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219450+0000","last_change":"2024-04-25T14:13:33.219450+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219450+0000","last_undegraded":"2024-04-25T14:13:33.219450+0000","last_fullsized":"2024-04-25T14:13:33.219450+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.224193+0000","last_change":"2024-04-25T14:13:33.224193+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.224193+0000","last_undegraded":"2024-04-25T14:13:33.224193+0000","last_fullsized":"2024-04-25T14:13:33.224193+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.226906+0000","last_change":"2024-04-25T14:13:33.226906+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.226906+0000","last_undegraded":"2024-04-25T14:13:33.226906+0000","last_fullsized":"2024-04-25T14:13:33.226906+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.223332+0000","last_change":"2024-04-25T14:13:33.223332+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.223332+0000","last_undegraded":"2024-04-25T14:13:33.223332+0000","last_fullsized":"2024-04-25T14:13:33.223332+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.217581+0000","last_change":"2024-04-25T14:13:33.217581+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.217581+0000","last_undegraded":"2024-04-25T14:13:33.217581+0000","last_fullsized":"2024-04-25T14:13:33.217581+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.238496+0000","last_change":"2024-04-25T14:13:33.233760+0000","last_active":"2024-04-25T14:13:33.238496+0000","last_peered":"2024-04-25T14:13:33.238496+0000","last_clean":"2024-04-25T14:13:33.238496+0000","last_became_active":"2024-04-25T14:13:33.233378+0000","last_became_peered":"2024-04-25T14:13:33.233378+0000","last_unstale":"2024-04-25T14:13:33.238496+0000","last_undegraded":"2024-04-25T14:13:33.238496+0000","last_fullsized":"2024-04-25T14:13:33.238496+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:24.899595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219833+0000","last_change":"2024-04-25T14:13:33.219833+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219833+0000","last_undegraded":"2024-04-25T14:13:33.219833+0000","last_fullsized":"2024-04-25T14:13:33.219833+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.243213+0000","last_change":"2024-04-25T14:13:33.239975+0000","last_active":"2024-04-25T14:13:33.243213+0000","last_peered":"2024-04-25T14:13:33.243213+0000","last_clean":"2024-04-25T14:13:33.243213+0000","last_became_active":"2024-04-25T14:13:33.239676+0000","last_became_peered":"2024-04-25T14:13:33.239676+0000","last_unstale":"2024-04-25T14:13:33.243213+0000","last_undegraded":"2024-04-25T14:13:33.243213+0000","last_fullsized":"2024-04-25T14:13:33.243213+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:44.934234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.218811+0000","last_change":"2024-04-25T14:13:33.218811+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.218811+0000","last_undegraded":"2024-04-25T14:13:33.218811+0000","last_fullsized":"2024-04-25T14:13:33.218811+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.242321+0000","last_change":"2024-04-25T14:13:33.241587+0000","last_active":"2024-04-25T14:13:33.242321+0000","last_peered":"2024-04-25T14:13:33.242321+0000","last_clean":"2024-04-25T14:13:33.242321+0000","last_became_active":"2024-04-25T14:13:33.240960+0000","last_became_peered":"2024-04-25T14:13:33.240960+0000","last_unstale":"2024-04-25T14:13:33.242321+0000","last_undegraded":"2024-04-25T14:13:33.242321+0000","last_fullsized":"2024-04-25T14:13:33.242321+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:34.418730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.217794+0000","last_change":"2024-04-25T14:13:33.217794+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.217794+0000","last_undegraded":"2024-04-25T14:13:33.217794+0000","last_fullsized":"2024-04-25T14:13:33.217794+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.242412+0000","last_change":"2024-04-25T14:13:33.239782+0000","last_active":"2024-04-25T14:13:33.242412+0000","last_peered":"2024-04-25T14:13:33.242412+0000","last_clean":"2024-04-25T14:13:33.242412+0000","last_became_active":"2024-04-25T14:13:33.239128+0000","last_became_peered":"2024-04-25T14:13:33.239128+0000","last_unstale":"2024-04-25T14:13:33.242412+0000","last_undegraded":"2024-04-25T14:13:33.242412+0000","last_fullsized":"2024-04-25T14:13:33.242412+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:40.423932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.221899+0000","last_change":"2024-04-25T14:13:33.221899+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.221899+0000","last_undegraded":"2024-04-25T14:13:33.221899+0000","last_fullsized":"2024-04-25T14:13:33.221899+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.244471+0000","last_change":"2024-04-25T14:13:33.242875+0000","last_active":"2024-04-25T14:13:33.244471+0000","last_peered":"2024-04-25T14:13:33.244471+0000","last_clean":"2024-04-25T14:13:33.244471+0000","last_became_active":"2024-04-25T14:13:33.242337+0000","last_became_peered":"2024-04-25T14:13:33.242337+0000","last_unstale":"2024-04-25T14:13:33.244471+0000","last_undegraded":"2024-04-25T14:13:33.244471+0000","last_fullsized":"2024-04-25T14:13:33.244471+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:16.518784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.226798+0000","last_change":"2024-04-25T14:13:33.226798+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.226798+0000","last_undegraded":"2024-04-25T14:13:33.226798+0000","last_fullsized":"2024-04-25T14:13:33.226798+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.241274+0000","last_change":"2024-04-25T14:13:33.233642+0000","last_active":"2024-04-25T14:13:33.241274+0000","last_peered":"2024-04-25T14:13:33.241274+0000","last_clean":"2024-04-25T14:13:33.241274+0000","last_became_active":"2024-04-25T14:13:33.233355+0000","last_became_peered":"2024-04-25T14:13:33.233355+0000","last_unstale":"2024-04-25T14:13:33.241274+0000","last_undegraded":"2024-04-25T14:13:33.241274+0000","last_fullsized":"2024-04-25T14:13:33.241274+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:37.218976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.228411+0000","last_change":"2024-04-25T14:13:33.228411+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.228411+0000","last_undegraded":"2024-04-25T14:13:33.228411+0000","last_fullsized":"2024-04-25T14:13:33.228411+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.241759+0000","last_change":"2024-04-25T14:13:33.239682+0000","last_active":"2024-04-25T14:13:33.241759+0000","last_peered":"2024-04-25T14:13:33.241759+0000","last_clean":"2024-04-25T14:13:33.241759+0000","last_became_active":"2024-04-25T14:13:33.239378+0000","last_became_peered":"2024-04-25T14:13:33.239378+0000","last_unstale":"2024-04-25T14:13:33.241759+0000","last_undegraded":"2024-04-25T14:13:33.241759+0000","last_fullsized":"2024-04-25T14:13:33.241759+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:10.873169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.217318+0000","last_change":"2024-04-25T14:13:33.217318+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.217318+0000","last_undegraded":"2024-04-25T14:13:33.217318+0000","last_fullsized":"2024-04-25T14:13:33.217318+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.238019+0000","last_change":"2024-04-25T14:13:33.232394+0000","last_active":"2024-04-25T14:13:33.238019+0000","last_peered":"2024-04-25T14:13:33.238019+0000","last_clean":"2024-04-25T14:13:33.238019+0000","last_became_active":"2024-04-25T14:13:33.231789+0000","last_became_peered":"2024-04-25T14:13:33.231789+0000","last_unstale":"2024-04-25T14:13:33.238019+0000","last_undegraded":"2024-04-25T14:13:33.238019+0000","last_fullsized":"2024-04-25T14:13:33.238019+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:45.600418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.218265+0000","last_change":"2024-04-25T14:13:33.218265+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.218265+0000","last_undegraded":"2024-04-25T14:13:33.218265+0000","last_fullsized":"2024-04-25T14:13:33.218265+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.243068+0000","last_change":"2024-04-25T14:13:33.235450+0000","last_active":"2024-04-25T14:13:33.243068+0000","last_peered":"2024-04-25T14:13:33.243068+0000","last_clean":"2024-04-25T14:13:33.243068+0000","last_became_active":"2024-04-25T14:13:33.235152+0000","last_became_peered":"2024-04-25T14:13:33.235152+0000","last_unstale":"2024-04-25T14:13:33.243068+0000","last_undegraded":"2024-04-25T14:13:33.243068+0000","last_fullsized":"2024-04-25T14:13:33.243068+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:35.585117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219518+0000","last_change":"2024-04-25T14:13:33.219518+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219518+0000","last_undegraded":"2024-04-25T14:13:33.219518+0000","last_fullsized":"2024-04-25T14:13:33.219518+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.238678+0000","last_change":"2024-04-25T14:13:33.235678+0000","last_active":"2024-04-25T14:13:33.238678+0000","last_peered":"2024-04-25T14:13:33.238678+0000","last_clean":"2024-04-25T14:13:33.238678+0000","last_became_active":"2024-04-25T14:13:33.235038+0000","last_became_peered":"2024-04-25T14:13:33.235038+0000","last_unstale":"2024-04-25T14:13:33.238678+0000","last_undegraded":"2024-04-25T14:13:33.238678+0000","last_fullsized":"2024-04-25T14:13:33.238678+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:23.471943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.225982+0000","last_change":"2024-04-25T14:13:33.225982+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.225982+0000","last_undegraded":"2024-04-25T14:13:33.225982+0000","last_fullsized":"2024-04-25T14:13:33.225982+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.243523+0000","last_change":"2024-04-25T14:13:33.235754+0000","last_active":"2024-04-25T14:13:33.243523+0000","last_peered":"2024-04-25T14:13:33.243523+0000","last_clean":"2024-04-25T14:13:33.243523+0000","last_became_active":"2024-04-25T14:13:33.235119+0000","last_became_peered":"2024-04-25T14:13:33.235119+0000","last_unstale":"2024-04-25T14:13:33.243523+0000","last_undegraded":"2024-04-25T14:13:33.243523+0000","last_fullsized":"2024-04-25T14:13:33.243523+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:09.237922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.222837+0000","last_change":"2024-04-25T14:13:33.222837+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.222837+0000","last_undegraded":"2024-04-25T14:13:33.222837+0000","last_fullsized":"2024-04-25T14:13:33.222837+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.240723+0000","last_change":"2024-04-25T14:13:33.237307+0000","last_active":"2024-04-25T14:13:33.240723+0000","last_peered":"2024-04-25T14:13:33.240723+0000","last_clean":"2024-04-25T14:13:33.240723+0000","last_became_active":"2024-04-25T14:13:33.237012+0000","last_became_peered":"2024-04-25T14:13:33.237012+0000","last_unstale":"2024-04-25T14:13:33.240723+0000","last_undegraded":"2024-04-25T14:13:33.240723+0000","last_fullsized":"2024-04-25T14:13:33.240723+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:45.356743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.222740+0000","last_change":"2024-04-25T14:13:33.222740+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.222740+0000","last_undegraded":"2024-04-25T14:13:33.222740+0000","last_fullsized":"2024-04-25T14:13:33.222740+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.241914+0000","last_change":"2024-04-25T14:13:33.234217+0000","last_active":"2024-04-25T14:13:33.241914+0000","last_peered":"2024-04-25T14:13:33.241914+0000","last_clean":"2024-04-25T14:13:33.241914+0000","last_became_active":"2024-04-25T14:13:33.233902+0000","last_became_peered":"2024-04-25T14:13:33.233902+0000","last_unstale":"2024-04-25T14:13:33.241914+0000","last_undegraded":"2024-04-25T14:13:33.241914+0000","last_fullsized":"2024-04-25T14:13:33.241914+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:09.455778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.227060+0000","last_change":"2024-04-25T14:13:33.227060+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.227060+0000","last_undegraded":"2024-04-25T14:13:33.227060+0000","last_fullsized":"2024-04-25T14:13:33.227060+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.240405+0000","last_change":"2024-04-25T14:13:33.239716+0000","last_active":"2024-04-25T14:13:33.240405+0000","last_peered":"2024-04-25T14:13:33.240405+0000","last_clean":"2024-04-25T14:13:33.240405+0000","last_became_active":"2024-04-25T14:13:33.239375+0000","last_became_peered":"2024-04-25T14:13:33.239375+0000","last_unstale":"2024-04-25T14:13:33.240405+0000","last_undegraded":"2024-04-25T14:13:33.240405+0000","last_fullsized":"2024-04-25T14:13:33.240405+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:05.463091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.226077+0000","last_change":"2024-04-25T14:13:33.226077+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.226077+0000","last_undegraded":"2024-04-25T14:13:33.226077+0000","last_fullsized":"2024-04-25T14:13:33.226077+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.239375+0000","last_change":"2024-04-25T14:13:33.238315+0000","last_active":"2024-04-25T14:13:33.239375+0000","last_peered":"2024-04-25T14:13:33.239375+0000","last_clean":"2024-04-25T14:13:33.239375+0000","last_became_active":"2024-04-25T14:13:33.237955+0000","last_became_peered":"2024-04-25T14:13:33.237955+0000","last_unstale":"2024-04-25T14:13:33.239375+0000","last_undegraded":"2024-04-25T14:13:33.239375+0000","last_fullsized":"2024-04-25T14:13:33.239375+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:50.318605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.223560+0000","last_change":"2024-04-25T14:13:33.223560+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.223560+0000","last_undegraded":"2024-04-25T14:13:33.223560+0000","last_fullsized":"2024-04-25T14:13:33.223560+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.237567+0000","last_change":"2024-04-25T14:13:33.226581+0000","last_active":"2024-04-25T14:13:33.237567+0000","last_peered":"2024-04-25T14:13:33.237567+0000","last_clean":"2024-04-25T14:13:33.237567+0000","last_became_active":"2024-04-25T14:13:33.225988+0000","last_became_peered":"2024-04-25T14:13:33.225988+0000","last_unstale":"2024-04-25T14:13:33.237567+0000","last_undegraded":"2024-04-25T14:13:33.237567+0000","last_fullsized":"2024-04-25T14:13:33.237567+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:08.407184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.224808+0000","last_change":"2024-04-25T14:13:33.224808+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.224808+0000","last_undegraded":"2024-04-25T14:13:33.224808+0000","last_fullsized":"2024-04-25T14:13:33.224808+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.244892+0000","last_change":"2024-04-25T14:13:33.239707+0000","last_active":"2024-04-25T14:13:33.244892+0000","last_peered":"2024-04-25T14:13:33.244892+0000","last_clean":"2024-04-25T14:13:33.244892+0000","last_became_active":"2024-04-25T14:13:33.239408+0000","last_became_peered":"2024-04-25T14:13:33.239408+0000","last_unstale":"2024-04-25T14:13:33.244892+0000","last_undegraded":"2024-04-25T14:13:33.244892+0000","last_fullsized":"2024-04-25T14:13:33.244892+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:51:01.497289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.226373+0000","last_change":"2024-04-25T14:13:33.226373+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.226373+0000","last_undegraded":"2024-04-25T14:13:33.226373+0000","last_fullsized":"2024-04-25T14:13:33.226373+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.239011+0000","last_change":"2024-04-25T14:13:33.232965+0000","last_active":"2024-04-25T14:13:33.239011+0000","last_peered":"2024-04-25T14:13:33.239011+0000","last_clean":"2024-04-25T14:13:33.239011+0000","last_became_active":"2024-04-25T14:13:33.232647+0000","last_became_peered":"2024-04-25T14:13:33.232647+0000","last_unstale":"2024-04-25T14:13:33.239011+0000","last_undegraded":"2024-04-25T14:13:33.239011+0000","last_fullsized":"2024-04-25T14:13:33.239011+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:12.089780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.229146+0000","last_change":"2024-04-25T14:13:33.229146+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.229146+0000","last_undegraded":"2024-04-25T14:13:33.229146+0000","last_fullsized":"2024-04-25T14:13:33.229146+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.244360+0000","last_change":"2024-04-25T14:13:33.240376+0000","last_active":"2024-04-25T14:13:33.244360+0000","last_peered":"2024-04-25T14:13:33.244360+0000","last_clean":"2024-04-25T14:13:33.244360+0000","last_became_active":"2024-04-25T14:13:33.240103+0000","last_became_peered":"2024-04-25T14:13:33.240103+0000","last_unstale":"2024-04-25T14:13:33.244360+0000","last_undegraded":"2024-04-25T14:13:33.244360+0000","last_fullsized":"2024-04-25T14:13:33.244360+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:55.225878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219516+0000","last_change":"2024-04-25T14:13:33.219516+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219516+0000","last_undegraded":"2024-04-25T14:13:33.219516+0000","last_fullsized":"2024-04-25T14:13:33.219516+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.240700+0000","last_change":"2024-04-25T14:13:33.231549+0000","last_active":"2024-04-25T14:13:33.240700+0000","last_peered":"2024-04-25T14:13:33.240700+0000","last_clean":"2024-04-25T14:13:33.240700+0000","last_became_active":"2024-04-25T14:13:33.231239+0000","last_became_peered":"2024-04-25T14:13:33.231239+0000","last_unstale":"2024-04-25T14:13:33.240700+0000","last_undegraded":"2024-04-25T14:13:33.240700+0000","last_fullsized":"2024-04-25T14:13:33.240700+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:46.823734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219647+0000","last_change":"2024-04-25T14:13:33.219647+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219647+0000","last_undegraded":"2024-04-25T14:13:33.219647+0000","last_fullsized":"2024-04-25T14:13:33.219647+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.239404+0000","last_change":"2024-04-25T14:13:33.238316+0000","last_active":"2024-04-25T14:13:33.239404+0000","last_peered":"2024-04-25T14:13:33.239404+0000","last_clean":"2024-04-25T14:13:33.239404+0000","last_became_active":"2024-04-25T14:13:33.237984+0000","last_became_peered":"2024-04-25T14:13:33.237984+0000","last_unstale":"2024-04-25T14:13:33.239404+0000","last_undegraded":"2024-04-25T14:13:33.239404+0000","last_fullsized":"2024-04-25T14:13:33.239404+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:01.863933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.220719+0000","last_change":"2024-04-25T14:13:33.220719+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.220719+0000","last_undegraded":"2024-04-25T14:13:33.220719+0000","last_fullsized":"2024-04-25T14:13:33.220719+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.240694+0000","last_change":"2024-04-25T14:13:33.233345+0000","last_active":"2024-04-25T14:13:33.240694+0000","last_peered":"2024-04-25T14:13:33.240694+0000","last_clean":"2024-04-25T14:13:33.240694+0000","last_became_active":"2024-04-25T14:13:33.233045+0000","last_became_peered":"2024-04-25T14:13:33.233045+0000","last_unstale":"2024-04-25T14:13:33.240694+0000","last_undegraded":"2024-04-25T14:13:33.240694+0000","last_fullsized":"2024-04-25T14:13:33.240694+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:28.978236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.218329+0000","last_change":"2024-04-25T14:13:33.218329+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.218329+0000","last_undegraded":"2024-04-25T14:13:33.218329+0000","last_fullsized":"2024-04-25T14:13:33.218329+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.235430+0000","last_change":"2024-04-25T14:13:33.233786+0000","last_active":"2024-04-25T14:13:33.235430+0000","last_peered":"2024-04-25T14:13:33.235430+0000","last_clean":"2024-04-25T14:13:33.235430+0000","last_became_active":"2024-04-25T14:13:33.233479+0000","last_became_peered":"2024-04-25T14:13:33.233479+0000","last_unstale":"2024-04-25T14:13:33.235430+0000","last_undegraded":"2024-04-25T14:13:33.235430+0000","last_fullsized":"2024-04-25T14:13:33.235430+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:57.369446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.225358+0000","last_change":"2024-04-25T14:13:33.225358+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.225358+0000","last_undegraded":"2024-04-25T14:13:33.225358+0000","last_fullsized":"2024-04-25T14:13:33.225358+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.239073+0000","last_change":"2024-04-25T14:13:33.235217+0000","last_active":"2024-04-25T14:13:33.239073+0000","last_peered":"2024-04-25T14:13:33.239073+0000","last_clean":"2024-04-25T14:13:33.239073+0000","last_became_active":"2024-04-25T14:13:33.234923+0000","last_became_peered":"2024-04-25T14:13:33.234923+0000","last_unstale":"2024-04-25T14:13:33.239073+0000","last_undegraded":"2024-04-25T14:13:33.239073+0000","last_fullsized":"2024-04-25T14:13:33.239073+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:16.491560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.223188+0000","last_change":"2024-04-25T14:13:33.223188+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.223188+0000","last_undegraded":"2024-04-25T14:13:33.223188+0000","last_fullsized":"2024-04-25T14:13:33.223188+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.237237+0000","last_change":"2024-04-25T14:13:33.233600+0000","last_active":"2024-04-25T14:13:33.237237+0000","last_peered":"2024-04-25T14:13:33.237237+0000","last_clean":"2024-04-25T14:13:33.237237+0000","last_became_active":"2024-04-25T14:13:33.233290+0000","last_became_peered":"2024-04-25T14:13:33.233290+0000","last_unstale":"2024-04-25T14:13:33.237237+0000","last_undegraded":"2024-04-25T14:13:33.237237+0000","last_fullsized":"2024-04-25T14:13:33.237237+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:16.534415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.226375+0000","last_change":"2024-04-25T14:13:33.226375+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.226375+0000","last_undegraded":"2024-04-25T14:13:33.226375+0000","last_fullsized":"2024-04-25T14:13:33.226375+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.236234+0000","last_change":"2024-04-25T14:13:33.229751+0000","last_active":"2024-04-25T14:13:33.236234+0000","last_peered":"2024-04-25T14:13:33.236234+0000","last_clean":"2024-04-25T14:13:33.236234+0000","last_became_active":"2024-04-25T14:13:33.229451+0000","last_became_peered":"2024-04-25T14:13:33.229451+0000","last_unstale":"2024-04-25T14:13:33.236234+0000","last_undegraded":"2024-04-25T14:13:33.236234+0000","last_fullsized":"2024-04-25T14:13:33.236234+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:34.938781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.220859+0000","last_change":"2024-04-25T14:13:33.220859+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.220859+0000","last_undegraded":"2024-04-25T14:13:33.220859+0000","last_fullsized":"2024-04-25T14:13:33.220859+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.246616+0000","last_change":"2024-04-25T14:13:33.245950+0000","last_active":"2024-04-25T14:13:33.246616+0000","last_peered":"2024-04-25T14:13:33.246616+0000","last_clean":"2024-04-25T14:13:33.246616+0000","last_became_active":"2024-04-25T14:13:33.245627+0000","last_became_peered":"2024-04-25T14:13:33.245627+0000","last_unstale":"2024-04-25T14:13:33.246616+0000","last_undegraded":"2024-04-25T14:13:33.246616+0000","last_fullsized":"2024-04-25T14:13:33.246616+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:59.112002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.216195+0000","last_change":"2024-04-25T14:13:33.216195+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.216195+0000","last_undegraded":"2024-04-25T14:13:33.216195+0000","last_fullsized":"2024-04-25T14:13:33.216195+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.242071+0000","last_change":"2024-04-25T14:13:33.241343+0000","last_active":"2024-04-25T14:13:33.242071+0000","last_peered":"2024-04-25T14:13:33.242071+0000","last_clean":"2024-04-25T14:13:33.242071+0000","last_became_active":"2024-04-25T14:13:33.240975+0000","last_became_peered":"2024-04-25T14:13:33.240975+0000","last_unstale":"2024-04-25T14:13:33.242071+0000","last_undegraded":"2024-04-25T14:13:33.242071+0000","last_fullsized":"2024-04-25T14:13:33.242071+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:43.434745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.217868+0000","last_change":"2024-04-25T14:13:33.217868+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.217868+0000","last_undegraded":"2024-04-25T14:13:33.217868+0000","last_fullsized":"2024-04-25T14:13:33.217868+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.236031+0000","last_change":"2024-04-25T14:13:33.230789+0000","last_active":"2024-04-25T14:13:33.236031+0000","last_peered":"2024-04-25T14:13:33.236031+0000","last_clean":"2024-04-25T14:13:33.236031+0000","last_became_active":"2024-04-25T14:13:33.230422+0000","last_became_peered":"2024-04-25T14:13:33.230422+0000","last_unstale":"2024-04-25T14:13:33.236031+0000","last_undegraded":"2024-04-25T14:13:33.236031+0000","last_fullsized":"2024-04-25T14:13:33.236031+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:05:59.451071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219902+0000","last_change":"2024-04-25T14:13:33.219902+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219902+0000","last_undegraded":"2024-04-25T14:13:33.219902+0000","last_fullsized":"2024-04-25T14:13:33.219902+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.242301+0000","last_change":"2024-04-25T14:13:33.241318+0000","last_active":"2024-04-25T14:13:33.242301+0000","last_peered":"2024-04-25T14:13:33.242301+0000","last_clean":"2024-04-25T14:13:33.242301+0000","last_became_active":"2024-04-25T14:13:33.240931+0000","last_became_peered":"2024-04-25T14:13:33.240931+0000","last_unstale":"2024-04-25T14:13:33.242301+0000","last_undegraded":"2024-04-25T14:13:33.242301+0000","last_fullsized":"2024-04-25T14:13:33.242301+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:39.911385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.225571+0000","last_change":"2024-04-25T14:13:33.225571+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.225571+0000","last_undegraded":"2024-04-25T14:13:33.225571+0000","last_fullsized":"2024-04-25T14:13:33.225571+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.238829+0000","last_change":"2024-04-25T14:13:33.229866+0000","last_active":"2024-04-25T14:13:33.238829+0000","last_peered":"2024-04-25T14:13:33.238829+0000","last_clean":"2024-04-25T14:13:33.238829+0000","last_became_active":"2024-04-25T14:13:33.229562+0000","last_became_peered":"2024-04-25T14:13:33.229562+0000","last_unstale":"2024-04-25T14:13:33.238829+0000","last_undegraded":"2024-04-25T14:13:33.238829+0000","last_fullsized":"2024-04-25T14:13:33.238829+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:00.731593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.218315+0000","last_change":"2024-04-25T14:13:33.218315+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.218315+0000","last_undegraded":"2024-04-25T14:13:33.218315+0000","last_fullsized":"2024-04-25T14:13:33.218315+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.244337+0000","last_change":"2024-04-25T14:13:33.242678+0000","last_active":"2024-04-25T14:13:33.244337+0000","last_peered":"2024-04-25T14:13:33.244337+0000","last_clean":"2024-04-25T14:13:33.244337+0000","last_became_active":"2024-04-25T14:13:33.241884+0000","last_became_peered":"2024-04-25T14:13:33.241884+0000","last_unstale":"2024-04-25T14:13:33.244337+0000","last_undegraded":"2024-04-25T14:13:33.244337+0000","last_fullsized":"2024-04-25T14:13:33.244337+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:00.731593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"144'3291","reported_seq":3828,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.226260+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:13:33.226260+0000","last_peered":"2024-04-25T14:13:33.226260+0000","last_clean":"2024-04-25T14:13:33.226260+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:13:33.226260+0000","last_undegraded":"2024-04-25T14:13:33.226260+0000","last_fullsized":"2024-04-25T14:13:33.226260+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":3291,"log_dups_size":0,"ondisk_log_size":3291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":837,"num_read_kb":2279,"num_write":247,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":837,"num_read_kb":2279,"num_write":247,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3291,"ondisk_log_size":3291,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574963,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29484,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342356,"statfs":{"total":96636764160,"available":96606572544,"internally_reserved":0,"allocated":2519040,"data_stored":1556182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574964,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33064,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1925120,"data_stored":965814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574965,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33064,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1925120,"data_stored":965814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574964,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33064,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1925120,"data_stored":965814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574964,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33068,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1925120,"data_stored":965814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33048,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338792,"statfs":{"total":96636764160,"available":96602923008,"internally_reserved":0,"allocated":1912832,"data_stored":959155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33064,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1925120,"data_stored":965814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574964,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29484,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342356,"statfs":{"total":96636764160,"available":96606572544,"internally_reserved":0,"allocated":2519040,"data_stored":1556182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:13:39.133 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3f is not active+clean 2024-04-25T14:13:39.134 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.217839+0000', 'last_change': '2024-04-25T14:13:33.217839+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.217839+0000', 'last_undegraded': '2024-04-25T14:13:33.217839+0000', 'last_fullsized': '2024-04-25T14:13:33.217839+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:13:39.134 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3e is not active+clean 2024-04-25T14:13:39.134 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.220477+0000', 'last_change': '2024-04-25T14:13:33.220477+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.220477+0000', 'last_undegraded': '2024-04-25T14:13:33.220477+0000', 'last_fullsized': '2024-04-25T14:13:33.220477+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:13:39.134 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3d is not active+clean 2024-04-25T14:13:39.134 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.216321+0000', 'last_change': '2024-04-25T14:13:33.216321+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.216321+0000', 'last_undegraded': '2024-04-25T14:13:33.216321+0000', 'last_fullsized': '2024-04-25T14:13:33.216321+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:13:39.134 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3c is not active+clean 2024-04-25T14:13:39.134 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.216741+0000', 'last_change': '2024-04-25T14:13:33.216741+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.216741+0000', 'last_undegraded': '2024-04-25T14:13:33.216741+0000', 'last_fullsized': '2024-04-25T14:13:33.216741+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:13:39.134 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3b is not active+clean 2024-04-25T14:13:39.134 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.219621+0000', 'last_change': '2024-04-25T14:13:33.219621+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.219621+0000', 'last_undegraded': '2024-04-25T14:13:33.219621+0000', 'last_fullsized': '2024-04-25T14:13:33.219621+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:13:39.134 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3a is not active+clean 2024-04-25T14:13:39.135 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.221767+0000', 'last_change': '2024-04-25T14:13:33.221767+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.221767+0000', 'last_undegraded': '2024-04-25T14:13:33.221767+0000', 'last_fullsized': '2024-04-25T14:13:33.221767+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:13:39.135 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.39 is not active+clean 2024-04-25T14:13:39.135 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.223617+0000', 'last_change': '2024-04-25T14:13:33.223617+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.223617+0000', 'last_undegraded': '2024-04-25T14:13:33.223617+0000', 'last_fullsized': '2024-04-25T14:13:33.223617+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:13:39.135 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.38 is not active+clean 2024-04-25T14:13:39.135 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.228099+0000', 'last_change': '2024-04-25T14:13:33.228099+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.228099+0000', 'last_undegraded': '2024-04-25T14:13:33.228099+0000', 'last_fullsized': '2024-04-25T14:13:33.228099+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:13:39.135 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.37 is not active+clean 2024-04-25T14:13:39.135 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.222924+0000', 'last_change': '2024-04-25T14:13:33.222924+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.222924+0000', 'last_undegraded': '2024-04-25T14:13:33.222924+0000', 'last_fullsized': '2024-04-25T14:13:33.222924+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:13:39.135 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.36 is not active+clean 2024-04-25T14:13:39.135 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.229249+0000', 'last_change': '2024-04-25T14:13:33.229249+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.229249+0000', 'last_undegraded': '2024-04-25T14:13:33.229249+0000', 'last_fullsized': '2024-04-25T14:13:33.229249+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:13:39.136 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.35 is not active+clean 2024-04-25T14:13:39.136 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.231347+0000', 'last_change': '2024-04-25T14:13:33.231347+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.231347+0000', 'last_undegraded': '2024-04-25T14:13:33.231347+0000', 'last_fullsized': '2024-04-25T14:13:33.231347+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:13:39.136 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.34 is not active+clean 2024-04-25T14:13:39.136 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.222546+0000', 'last_change': '2024-04-25T14:13:33.222546+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.222546+0000', 'last_undegraded': '2024-04-25T14:13:33.222546+0000', 'last_fullsized': '2024-04-25T14:13:33.222546+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:13:39.136 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.33 is not active+clean 2024-04-25T14:13:39.136 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.224550+0000', 'last_change': '2024-04-25T14:13:33.224550+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.224550+0000', 'last_undegraded': '2024-04-25T14:13:33.224550+0000', 'last_fullsized': '2024-04-25T14:13:33.224550+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:13:39.136 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.32 is not active+clean 2024-04-25T14:13:39.136 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.219360+0000', 'last_change': '2024-04-25T14:13:33.219360+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.219360+0000', 'last_undegraded': '2024-04-25T14:13:33.219360+0000', 'last_fullsized': '2024-04-25T14:13:33.219360+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:13:39.137 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.31 is not active+clean 2024-04-25T14:13:39.137 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.229250+0000', 'last_change': '2024-04-25T14:13:33.229250+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.229250+0000', 'last_undegraded': '2024-04-25T14:13:33.229250+0000', 'last_fullsized': '2024-04-25T14:13:33.229250+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:13:39.137 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.30 is not active+clean 2024-04-25T14:13:39.137 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.216778+0000', 'last_change': '2024-04-25T14:13:33.216778+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.216778+0000', 'last_undegraded': '2024-04-25T14:13:33.216778+0000', 'last_fullsized': '2024-04-25T14:13:33.216778+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:13:39.137 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2f is not active+clean 2024-04-25T14:13:39.137 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.222484+0000', 'last_change': '2024-04-25T14:13:33.222484+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.222484+0000', 'last_undegraded': '2024-04-25T14:13:33.222484+0000', 'last_fullsized': '2024-04-25T14:13:33.222484+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:13:39.137 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2e is not active+clean 2024-04-25T14:13:39.137 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.225152+0000', 'last_change': '2024-04-25T14:13:33.225152+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.225152+0000', 'last_undegraded': '2024-04-25T14:13:33.225152+0000', 'last_fullsized': '2024-04-25T14:13:33.225152+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:13:39.137 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2d is not active+clean 2024-04-25T14:13:39.138 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.228898+0000', 'last_change': '2024-04-25T14:13:33.228898+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.228898+0000', 'last_undegraded': '2024-04-25T14:13:33.228898+0000', 'last_fullsized': '2024-04-25T14:13:33.228898+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:13:39.138 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2c is not active+clean 2024-04-25T14:13:39.138 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.229071+0000', 'last_change': '2024-04-25T14:13:33.229071+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.229071+0000', 'last_undegraded': '2024-04-25T14:13:33.229071+0000', 'last_fullsized': '2024-04-25T14:13:33.229071+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:13:39.138 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2b is not active+clean 2024-04-25T14:13:39.138 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.224104+0000', 'last_change': '2024-04-25T14:13:33.224104+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.224104+0000', 'last_undegraded': '2024-04-25T14:13:33.224104+0000', 'last_fullsized': '2024-04-25T14:13:33.224104+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:13:39.138 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2a is not active+clean 2024-04-25T14:13:39.138 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.221177+0000', 'last_change': '2024-04-25T14:13:33.221177+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.221177+0000', 'last_undegraded': '2024-04-25T14:13:33.221177+0000', 'last_fullsized': '2024-04-25T14:13:33.221177+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:13:39.138 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.29 is not active+clean 2024-04-25T14:13:39.138 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.218713+0000', 'last_change': '2024-04-25T14:13:33.218713+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.218713+0000', 'last_undegraded': '2024-04-25T14:13:33.218713+0000', 'last_fullsized': '2024-04-25T14:13:33.218713+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:13:39.138 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.28 is not active+clean 2024-04-25T14:13:39.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.215974+0000', 'last_change': '2024-04-25T14:13:33.215974+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.215974+0000', 'last_undegraded': '2024-04-25T14:13:33.215974+0000', 'last_fullsized': '2024-04-25T14:13:33.215974+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:13:39.139 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.27 is not active+clean 2024-04-25T14:13:39.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.222118+0000', 'last_change': '2024-04-25T14:13:33.222118+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.222118+0000', 'last_undegraded': '2024-04-25T14:13:33.222118+0000', 'last_fullsized': '2024-04-25T14:13:33.222118+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:13:39.139 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.26 is not active+clean 2024-04-25T14:13:39.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.228839+0000', 'last_change': '2024-04-25T14:13:33.228839+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.228839+0000', 'last_undegraded': '2024-04-25T14:13:33.228839+0000', 'last_fullsized': '2024-04-25T14:13:33.228839+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:13:39.139 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.25 is not active+clean 2024-04-25T14:13:39.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.221814+0000', 'last_change': '2024-04-25T14:13:33.221814+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.221814+0000', 'last_undegraded': '2024-04-25T14:13:33.221814+0000', 'last_fullsized': '2024-04-25T14:13:33.221814+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:13:39.139 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.24 is not active+clean 2024-04-25T14:13:39.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.223958+0000', 'last_change': '2024-04-25T14:13:33.223958+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.223958+0000', 'last_undegraded': '2024-04-25T14:13:33.223958+0000', 'last_fullsized': '2024-04-25T14:13:33.223958+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:13:39.139 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.23 is not active+clean 2024-04-25T14:13:39.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.219450+0000', 'last_change': '2024-04-25T14:13:33.219450+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.219450+0000', 'last_undegraded': '2024-04-25T14:13:33.219450+0000', 'last_fullsized': '2024-04-25T14:13:33.219450+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:13:39.140 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.22 is not active+clean 2024-04-25T14:13:39.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.224193+0000', 'last_change': '2024-04-25T14:13:33.224193+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.224193+0000', 'last_undegraded': '2024-04-25T14:13:33.224193+0000', 'last_fullsized': '2024-04-25T14:13:33.224193+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:13:39.140 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.21 is not active+clean 2024-04-25T14:13:39.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.226906+0000', 'last_change': '2024-04-25T14:13:33.226906+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.226906+0000', 'last_undegraded': '2024-04-25T14:13:33.226906+0000', 'last_fullsized': '2024-04-25T14:13:33.226906+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:13:39.140 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.20 is not active+clean 2024-04-25T14:13:39.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.223332+0000', 'last_change': '2024-04-25T14:13:33.223332+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.223332+0000', 'last_undegraded': '2024-04-25T14:13:33.223332+0000', 'last_fullsized': '2024-04-25T14:13:33.223332+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:13:39.140 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.10 is not active+clean 2024-04-25T14:13:39.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.217581+0000', 'last_change': '2024-04-25T14:13:33.217581+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.217581+0000', 'last_undegraded': '2024-04-25T14:13:33.217581+0000', 'last_fullsized': '2024-04-25T14:13:33.217581+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:13:39.140 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.11 is not active+clean 2024-04-25T14:13:39.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.219833+0000', 'last_change': '2024-04-25T14:13:33.219833+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.219833+0000', 'last_undegraded': '2024-04-25T14:13:33.219833+0000', 'last_fullsized': '2024-04-25T14:13:33.219833+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:13:39.140 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.12 is not active+clean 2024-04-25T14:13:39.141 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.218811+0000', 'last_change': '2024-04-25T14:13:33.218811+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.218811+0000', 'last_undegraded': '2024-04-25T14:13:33.218811+0000', 'last_fullsized': '2024-04-25T14:13:33.218811+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:13:39.141 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.13 is not active+clean 2024-04-25T14:13:39.141 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.217794+0000', 'last_change': '2024-04-25T14:13:33.217794+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.217794+0000', 'last_undegraded': '2024-04-25T14:13:33.217794+0000', 'last_fullsized': '2024-04-25T14:13:33.217794+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:13:39.141 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.14 is not active+clean 2024-04-25T14:13:39.141 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.221899+0000', 'last_change': '2024-04-25T14:13:33.221899+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.221899+0000', 'last_undegraded': '2024-04-25T14:13:33.221899+0000', 'last_fullsized': '2024-04-25T14:13:33.221899+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:13:39.141 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.15 is not active+clean 2024-04-25T14:13:39.141 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.226798+0000', 'last_change': '2024-04-25T14:13:33.226798+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.226798+0000', 'last_undegraded': '2024-04-25T14:13:33.226798+0000', 'last_fullsized': '2024-04-25T14:13:33.226798+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:13:39.141 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.16 is not active+clean 2024-04-25T14:13:39.141 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.228411+0000', 'last_change': '2024-04-25T14:13:33.228411+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.228411+0000', 'last_undegraded': '2024-04-25T14:13:33.228411+0000', 'last_fullsized': '2024-04-25T14:13:33.228411+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:13:39.141 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.17 is not active+clean 2024-04-25T14:13:39.141 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.217318+0000', 'last_change': '2024-04-25T14:13:33.217318+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.217318+0000', 'last_undegraded': '2024-04-25T14:13:33.217318+0000', 'last_fullsized': '2024-04-25T14:13:33.217318+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:13:39.142 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.18 is not active+clean 2024-04-25T14:13:39.142 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.218265+0000', 'last_change': '2024-04-25T14:13:33.218265+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.218265+0000', 'last_undegraded': '2024-04-25T14:13:33.218265+0000', 'last_fullsized': '2024-04-25T14:13:33.218265+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:13:39.142 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1a is not active+clean 2024-04-25T14:13:39.142 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.219518+0000', 'last_change': '2024-04-25T14:13:33.219518+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.219518+0000', 'last_undegraded': '2024-04-25T14:13:33.219518+0000', 'last_fullsized': '2024-04-25T14:13:33.219518+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:13:39.142 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1b is not active+clean 2024-04-25T14:13:39.142 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.225982+0000', 'last_change': '2024-04-25T14:13:33.225982+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.225982+0000', 'last_undegraded': '2024-04-25T14:13:33.225982+0000', 'last_fullsized': '2024-04-25T14:13:33.225982+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:13:39.142 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1c is not active+clean 2024-04-25T14:13:39.142 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.222837+0000', 'last_change': '2024-04-25T14:13:33.222837+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.222837+0000', 'last_undegraded': '2024-04-25T14:13:33.222837+0000', 'last_fullsized': '2024-04-25T14:13:33.222837+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:13:39.142 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1d is not active+clean 2024-04-25T14:13:39.142 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.222740+0000', 'last_change': '2024-04-25T14:13:33.222740+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.222740+0000', 'last_undegraded': '2024-04-25T14:13:33.222740+0000', 'last_fullsized': '2024-04-25T14:13:33.222740+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:13:39.143 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1e is not active+clean 2024-04-25T14:13:39.143 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.227060+0000', 'last_change': '2024-04-25T14:13:33.227060+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.227060+0000', 'last_undegraded': '2024-04-25T14:13:33.227060+0000', 'last_fullsized': '2024-04-25T14:13:33.227060+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:13:39.143 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1f is not active+clean 2024-04-25T14:13:39.143 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.226077+0000', 'last_change': '2024-04-25T14:13:33.226077+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.226077+0000', 'last_undegraded': '2024-04-25T14:13:33.226077+0000', 'last_fullsized': '2024-04-25T14:13:33.226077+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:13:39.143 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.0 is not active+clean 2024-04-25T14:13:39.143 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.223560+0000', 'last_change': '2024-04-25T14:13:33.223560+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.223560+0000', 'last_undegraded': '2024-04-25T14:13:33.223560+0000', 'last_fullsized': '2024-04-25T14:13:33.223560+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:13:39.143 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1 is not active+clean 2024-04-25T14:13:39.143 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.224808+0000', 'last_change': '2024-04-25T14:13:33.224808+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.224808+0000', 'last_undegraded': '2024-04-25T14:13:33.224808+0000', 'last_fullsized': '2024-04-25T14:13:33.224808+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T14:13:39.143 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2 is not active+clean 2024-04-25T14:13:39.143 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.226373+0000', 'last_change': '2024-04-25T14:13:33.226373+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.226373+0000', 'last_undegraded': '2024-04-25T14:13:33.226373+0000', 'last_fullsized': '2024-04-25T14:13:33.226373+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:13:39.144 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3 is not active+clean 2024-04-25T14:13:39.144 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.229146+0000', 'last_change': '2024-04-25T14:13:33.229146+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.229146+0000', 'last_undegraded': '2024-04-25T14:13:33.229146+0000', 'last_fullsized': '2024-04-25T14:13:33.229146+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:13:39.144 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.4 is not active+clean 2024-04-25T14:13:39.144 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.219516+0000', 'last_change': '2024-04-25T14:13:33.219516+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.219516+0000', 'last_undegraded': '2024-04-25T14:13:33.219516+0000', 'last_fullsized': '2024-04-25T14:13:33.219516+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:13:39.144 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.5 is not active+clean 2024-04-25T14:13:39.144 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.219647+0000', 'last_change': '2024-04-25T14:13:33.219647+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.219647+0000', 'last_undegraded': '2024-04-25T14:13:33.219647+0000', 'last_fullsized': '2024-04-25T14:13:33.219647+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:13:39.144 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.6 is not active+clean 2024-04-25T14:13:39.144 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.220719+0000', 'last_change': '2024-04-25T14:13:33.220719+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.220719+0000', 'last_undegraded': '2024-04-25T14:13:33.220719+0000', 'last_fullsized': '2024-04-25T14:13:33.220719+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T14:13:39.144 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.7 is not active+clean 2024-04-25T14:13:39.144 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.218329+0000', 'last_change': '2024-04-25T14:13:33.218329+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.218329+0000', 'last_undegraded': '2024-04-25T14:13:33.218329+0000', 'last_fullsized': '2024-04-25T14:13:33.218329+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:13:39.145 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.8 is not active+clean 2024-04-25T14:13:39.145 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.225358+0000', 'last_change': '2024-04-25T14:13:33.225358+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.225358+0000', 'last_undegraded': '2024-04-25T14:13:33.225358+0000', 'last_fullsized': '2024-04-25T14:13:33.225358+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T14:13:39.145 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.9 is not active+clean 2024-04-25T14:13:39.145 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.223188+0000', 'last_change': '2024-04-25T14:13:33.223188+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.223188+0000', 'last_undegraded': '2024-04-25T14:13:33.223188+0000', 'last_fullsized': '2024-04-25T14:13:33.223188+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:13:39.145 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.a is not active+clean 2024-04-25T14:13:39.145 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.226375+0000', 'last_change': '2024-04-25T14:13:33.226375+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.226375+0000', 'last_undegraded': '2024-04-25T14:13:33.226375+0000', 'last_fullsized': '2024-04-25T14:13:33.226375+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:13:39.145 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.b is not active+clean 2024-04-25T14:13:39.145 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.220859+0000', 'last_change': '2024-04-25T14:13:33.220859+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.220859+0000', 'last_undegraded': '2024-04-25T14:13:33.220859+0000', 'last_fullsized': '2024-04-25T14:13:33.220859+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T14:13:39.145 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.c is not active+clean 2024-04-25T14:13:39.145 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.216195+0000', 'last_change': '2024-04-25T14:13:33.216195+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.216195+0000', 'last_undegraded': '2024-04-25T14:13:33.216195+0000', 'last_fullsized': '2024-04-25T14:13:33.216195+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T14:13:39.145 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.d is not active+clean 2024-04-25T14:13:39.145 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.217868+0000', 'last_change': '2024-04-25T14:13:33.217868+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.217868+0000', 'last_undegraded': '2024-04-25T14:13:33.217868+0000', 'last_fullsized': '2024-04-25T14:13:33.217868+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:13:39.146 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.e is not active+clean 2024-04-25T14:13:39.146 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.219902+0000', 'last_change': '2024-04-25T14:13:33.219902+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.219902+0000', 'last_undegraded': '2024-04-25T14:13:33.219902+0000', 'last_fullsized': '2024-04-25T14:13:33.219902+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T14:13:39.146 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.f is not active+clean 2024-04-25T14:13:39.146 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.225571+0000', 'last_change': '2024-04-25T14:13:33.225571+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.225571+0000', 'last_undegraded': '2024-04-25T14:13:33.225571+0000', 'last_fullsized': '2024-04-25T14:13:33.225571+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T14:13:39.146 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.19 is not active+clean 2024-04-25T14:13:39.146 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2024-04-25T14:13:33.218315+0000', 'last_change': '2024-04-25T14:13:33.218315+0000', 'last_active': '2024-04-25T14:13:33.204667+0000', 'last_peered': '2024-04-25T14:13:33.204667+0000', 'last_clean': '2024-04-25T14:13:33.204667+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T14:13:33.218315+0000', 'last_undegraded': '2024-04-25T14:13:33.218315+0000', 'last_fullsized': '2024-04-25T14:13:33.218315+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'last_clean_scrub_stamp': '2024-04-25T14:13:33.204667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T14:13:39.146 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T14:13:39.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.319+0000 7f8c21c00640 1 Processor -- start 2024-04-25T14:13:39.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.319+0000 7f8c21c00640 1 -- start start 2024-04-25T14:13:39.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.319+0000 7f8c21c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c1c15dc90 0x7f8c1c161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.319+0000 7f8c21c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c1c0cd8d0 con 0x7f8c1c071530 2024-04-25T14:13:39.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.319+0000 7f8c21c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c1c0ce970 con 0x7f8c1c0d2c80 2024-04-25T14:13:39.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.319+0000 7f8c21c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8c1c0ce720 con 0x7f8c1c15dc90 2024-04-25T14:13:39.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.319+0000 7f8c12000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8c1c071530 0x7f8c1c1616e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45204/0 (socket says 172.21.15.86:45204) 2024-04-25T14:13:39.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.319+0000 7f8c12000640 1 -- 172.21.15.86:0/4260448062 learned_addr learned my addr 172.21.15.86:0/4260448062 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:39.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.323+0000 7f8c1a000640 1 -- 172.21.15.86:0/4260448062 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3827301876 0 0) 0x7f8c1c0cd8d0 con 0x7f8c1c071530 2024-04-25T14:13:39.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.323+0000 7f8c1b400640 1 --2- 172.21.15.86:0/4260448062 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c1c15dc90 0x7f8c1c161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.323+0000 7f8c1a000640 1 -- 172.21.15.86:0/4260448062 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c00003530 con 0x7f8c1c071530 2024-04-25T14:13:39.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.323+0000 7f8c1a000640 1 -- 172.21.15.86:0/4260448062 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 203174113 0 0) 0x7f8c00003530 con 0x7f8c1c071530 2024-04-25T14:13:39.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.323+0000 7f8c1a000640 1 -- 172.21.15.86:0/4260448062 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c1c15dc90 msgr2=0x7f8c1c161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.323+0000 7f8c1a000640 1 --2- 172.21.15.86:0/4260448062 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c1c15dc90 0x7f8c1c161df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.323+0000 7f8c1a000640 1 -- 172.21.15.86:0/4260448062 >> v1:172.21.15.155:6789/0 conn(0x7f8c1c0d2c80 legacy=0x7f8c1c070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.323+0000 7f8c1a000640 1 -- 172.21.15.86:0/4260448062 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1c0d1be0 con 0x7f8c1c071530 2024-04-25T14:13:39.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.323+0000 7f8c1a000640 1 -- 172.21.15.86:0/4260448062 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f8c04001800 con 0x7f8c1c071530 2024-04-25T14:13:39.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.323+0000 7f8c1a000640 1 -- 172.21.15.86:0/4260448062 >> v1:172.21.15.86:6789/0 conn(0x7f8c1c071530 legacy=0x7f8c1c1616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.323+0000 7f8c1a000640 1 --2- 172.21.15.86:0/4260448062 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c1c15dc90 0x7f8c00003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.323+0000 7f8c1a000640 1 --2- 172.21.15.86:0/4260448062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c00003fc0 0x7f8c00004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.323+0000 7f8c1a000640 1 --2- 172.21.15.86:0/4260448062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c00004d50 0x7f8c00005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.323+0000 7f8c1a000640 1 -- 172.21.15.86:0/4260448062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c0ce720 con 0x7f8c00003fc0 2024-04-25T14:13:39.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.323+0000 7f8c1a000640 1 -- 172.21.15.86:0/4260448062 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0000bb60 con 0x7f8c00004d50 2024-04-25T14:13:39.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.323+0000 7f8c1a000640 1 -- 172.21.15.86:0/4260448062 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8c0000bd60 con 0x7f8c1c15dc90 2024-04-25T14:13:39.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.323+0000 7f8c12000640 1 --2- 172.21.15.86:0/4260448062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c00003fc0 0x7f8c00004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.323+0000 7f8c1aa00640 1 --2- 172.21.15.86:0/4260448062 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c1c15dc90 0x7f8c00003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.323+0000 7f8c1b400640 1 --2- 172.21.15.86:0/4260448062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c00004d50 0x7f8c00005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.323+0000 7f8c12000640 1 -- 172.21.15.86:0/4260448062 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c1c15dc90 msgr2=0x7f8c00003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.323+0000 7f8c12000640 1 --2- 172.21.15.86:0/4260448062 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c1c15dc90 0x7f8c00003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.323+0000 7f8c12000640 1 -- 172.21.15.86:0/4260448062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c00004d50 msgr2=0x7f8c00005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.323+0000 7f8c12000640 1 --2- 172.21.15.86:0/4260448062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c00004d50 0x7f8c00005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.323+0000 7f8c12000640 1 -- 172.21.15.86:0/4260448062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c00003530 con 0x7f8c00003fc0 2024-04-25T14:13:39.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.327+0000 7f8c12000640 1 --2- 172.21.15.86:0/4260448062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c00003fc0 0x7f8c00004810 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f8c0400c210 tx=0x7f8c0401f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5efbfb2b665f451a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.327+0000 7f8c1a000640 1 -- 172.21.15.86:0/4260448062 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c04007d50 con 0x7f8c00003fc0 2024-04-25T14:13:39.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.327+0000 7f8c1a000640 1 -- 172.21.15.86:0/4260448062 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c04007270 con 0x7f8c00003fc0 2024-04-25T14:13:39.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.327+0000 7f8c1a000640 1 -- 172.21.15.86:0/4260448062 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c040057b0 con 0x7f8c00003fc0 2024-04-25T14:13:39.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.327+0000 7f8c21c00640 1 -- 172.21.15.86:0/4260448062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c00003fc0 msgr2=0x7f8c00004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.327+0000 7f8c21c00640 1 --2- 172.21.15.86:0/4260448062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c00003fc0 0x7f8c00004810 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f8c0400c210 tx=0x7f8c0401f3f0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.327+0000 7f8c1b400640 1 -- 172.21.15.86:0/4260448062 reap_dead start 2024-04-25T14:13:39.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.327+0000 7f8c21c00640 1 -- 172.21.15.86:0/4260448062 shutdown_connections 2024-04-25T14:13:39.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.327+0000 7f8c21c00640 1 -- 172.21.15.86:0/4260448062 >> 172.21.15.86:0/4260448062 conn(0x7f8c1c0cffc0 msgr2=0x7f8c1c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.327+0000 7f8c21c00640 1 -- 172.21.15.86:0/4260448062 shutdown_connections 2024-04-25T14:13:39.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.327+0000 7f8c21c00640 1 -- 172.21.15.86:0/4260448062 wait complete. 2024-04-25T14:13:39.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.327+0000 7f8c21c00640 1 Processor -- start 2024-04-25T14:13:39.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.327+0000 7f8c21c00640 1 -- start start 2024-04-25T14:13:39.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.331+0000 7f8c21c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c00003fc0 0x7f8c1c15f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.331+0000 7f8c21c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c1c1563b0 0x7f8c1c155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.331+0000 7f8c21c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c1c1f47d0 0x7f8c1c1f4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.331+0000 7f8c21c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c0d2480 con 0x7f8c1c1f47d0 2024-04-25T14:13:39.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.331+0000 7f8c21c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c0d21a0 con 0x7f8c00003fc0 2024-04-25T14:13:39.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.331+0000 7f8c21c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8c1c0d29a0 con 0x7f8c1c1563b0 2024-04-25T14:13:39.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.331+0000 7f8c1b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c1c1f47d0 0x7f8c1c1f4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.331+0000 7f8c1b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c1c1f47d0 0x7f8c1c1f4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41614/0 (socket says 172.21.15.86:41614) 2024-04-25T14:13:39.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.331+0000 7f8c1b400640 1 -- 172.21.15.86:0/3254406317 learned_addr learned my addr 172.21.15.86:0/3254406317 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:39.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.331+0000 7f8c1aa00640 1 --2- 172.21.15.86:0/3254406317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c00003fc0 0x7f8c1c15f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.331+0000 7f8c12000640 1 --2- 172.21.15.86:0/3254406317 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c1c1563b0 0x7f8c1c155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.331+0000 7f8c1b400640 1 -- 172.21.15.86:0/3254406317 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c1c1563b0 msgr2=0x7f8c1c155af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.331+0000 7f8c1b400640 1 --2- 172.21.15.86:0/3254406317 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c1c1563b0 0x7f8c1c155af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.331+0000 7f8c1b400640 1 -- 172.21.15.86:0/3254406317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c00003fc0 msgr2=0x7f8c1c15f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.331+0000 7f8c1b400640 1 --2- 172.21.15.86:0/3254406317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c00003fc0 0x7f8c1c15f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.331+0000 7f8c1b400640 1 -- 172.21.15.86:0/3254406317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1c15fb10 con 0x7f8c1c1f47d0 2024-04-25T14:13:39.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.331+0000 7f8c1b400640 1 --2- 172.21.15.86:0/3254406317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c1c1f47d0 0x7f8c1c1f4be0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c161640 tx=0x7f8c1400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.331+0000 7f8c13400640 1 -- 172.21.15.86:0/3254406317 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1400fe10 con 0x7f8c1c1f47d0 2024-04-25T14:13:39.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.331+0000 7f8c13400640 1 -- 172.21.15.86:0/3254406317 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c1401d070 con 0x7f8c1c1f47d0 2024-04-25T14:13:39.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.331+0000 7f8c21c00640 1 -- 172.21.15.86:0/3254406317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c1c15e530 con 0x7f8c1c1f47d0 2024-04-25T14:13:39.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.331+0000 7f8c13400640 1 -- 172.21.15.86:0/3254406317 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c14015670 con 0x7f8c1c1f47d0 2024-04-25T14:13:39.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.331+0000 7f8c21c00640 1 -- 172.21.15.86:0/3254406317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c1c15eaf0 con 0x7f8c1c1f47d0 2024-04-25T14:13:39.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.335+0000 7f8c21c00640 1 -- 172.21.15.86:0/3254406317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c1c15b570 con 0x7f8c1c1f47d0 2024-04-25T14:13:39.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.335+0000 7f8c13400640 1 -- 172.21.15.86:0/3254406317 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7f8c14015810 con 0x7f8c1c1f47d0 2024-04-25T14:13:39.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.335+0000 7f8c13400640 1 --2- 172.21.15.86:0/3254406317 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f8bf806f520 0x7f8bf8071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.335+0000 7f8c13400640 1 -- 172.21.15.86:0/3254406317 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6101+0+0 (secure 0 0 0) 0x7f8c1408fef0 con 0x7f8c1c1f47d0 2024-04-25T14:13:39.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.335+0000 7f8c1aa00640 1 --2- 172.21.15.86:0/3254406317 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f8bf806f520 0x7f8bf8071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.335+0000 7f8c1aa00640 1 --2- 172.21.15.86:0/3254406317 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f8bf806f520 0x7f8bf8071a30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c000c10 tx=0x7f8c0c009040 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.339+0000 7f8c13400640 1 -- 172.21.15.86:0/3254406317 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8c14010040 con 0x7f8c1c1f47d0 2024-04-25T14:13:39.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.555+0000 7f8c21c00640 1 -- 172.21.15.86:0/3254406317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f8c1c076bf0 con 0x7f8c1c1f47d0 2024-04-25T14:13:39.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.555+0000 7f8c13400640 1 -- 172.21.15.86:0/3254406317 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1040 (secure 0 0 0) 0x7f8c1405c3e0 con 0x7f8c1c1f47d0 2024-04-25T14:13:39.553 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:13:39.553 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":613,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":155,"num_osds":8,"num_up_osds":8,"osd_up_since":1714053818,"num_in_osds":8,"osd_in_since":1714053808,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":263516160,"bytes_avail":772830597120,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":117,"id":11,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":11,"rank":0,"name":"a","status":"up:active","gid":10209}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-04-25T14:13:03.822604+0000","services":{}},"progress_events":{}} 2024-04-25T14:13:39.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.555+0000 7f8c21c00640 1 -- 172.21.15.86:0/3254406317 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f8bf806f520 msgr2=0x7f8bf8071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.555+0000 7f8c21c00640 1 --2- 172.21.15.86:0/3254406317 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f8bf806f520 0x7f8bf8071a30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c000c10 tx=0x7f8c0c009040 comp rx=0 tx=0).stop 2024-04-25T14:13:39.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.555+0000 7f8c21c00640 1 -- 172.21.15.86:0/3254406317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c1c1f47d0 msgr2=0x7f8c1c1f4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.555+0000 7f8c21c00640 1 --2- 172.21.15.86:0/3254406317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c1c1f47d0 0x7f8c1c1f4be0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c161640 tx=0x7f8c1400f580 comp rx=0 tx=0).stop 2024-04-25T14:13:39.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.555+0000 7f8c21c00640 1 -- 172.21.15.86:0/3254406317 shutdown_connections 2024-04-25T14:13:39.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.555+0000 7f8c21c00640 1 --2- 172.21.15.86:0/3254406317 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f8bf806f520 0x7f8bf8071a30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.555+0000 7f8c21c00640 1 --2- 172.21.15.86:0/3254406317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c1c1f47d0 0x7f8c1c1f4be0 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.555+0000 7f8c21c00640 1 --2- 172.21.15.86:0/3254406317 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c1c1563b0 0x7f8c1c155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.555+0000 7f8c21c00640 1 --2- 172.21.15.86:0/3254406317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c00003fc0 0x7f8c1c15f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.555+0000 7f8c21c00640 1 -- 172.21.15.86:0/3254406317 >> 172.21.15.86:0/3254406317 conn(0x7f8c1c0cffc0 msgr2=0x7f8c1c15bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.555+0000 7f8c21c00640 1 -- 172.21.15.86:0/3254406317 shutdown_connections 2024-04-25T14:13:39.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.555+0000 7f8c21c00640 1 -- 172.21.15.86:0/3254406317 wait complete. 2024-04-25T14:13:39.574 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T14:13:39.575 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:13:39.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.722+0000 7f3b56000640 1 Processor -- start 2024-04-25T14:13:39.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.722+0000 7f3b56000640 1 -- start start 2024-04-25T14:13:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.722+0000 7f3b56000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b500b6ee0 0x7f3b500d06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.722+0000 7f3b56000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3b500d0c00 0x7f3b500722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.722+0000 7f3b56000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b50072bf0 con 0x7f3b500727e0 2024-04-25T14:13:39.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.722+0000 7f3b56000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3b5015df00 con 0x7f3b500d0c00 2024-04-25T14:13:39.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.722+0000 7f3b56000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3b5015e1e0 con 0x7f3b500b6ee0 2024-04-25T14:13:39.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.726+0000 7f3b54c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b500b6ee0 0x7f3b500d06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.726+0000 7f3b47e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3b500d0c00 0x7f3b500722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.726+0000 7f3b47e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3b500d0c00 0x7f3b500722a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:48632/0 (socket says 172.21.15.86:48632) 2024-04-25T14:13:39.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.726+0000 7f3b47e00640 1 -- 172.21.15.86:0/3239738402 learned_addr learned my addr 172.21.15.86:0/3239738402 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.726+0000 7f3b4fe00640 1 -- 172.21.15.86:0/3239738402 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3028001678 0 0) 0x7f3b480014a0 con 0x7f3b500727e0 2024-04-25T14:13:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.726+0000 7f3b4fe00640 1 -- 172.21.15.86:0/3239738402 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b34003530 con 0x7f3b500727e0 2024-04-25T14:13:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.726+0000 7f3b54c00640 1 -- 172.21.15.86:0/3239738402 >> v2:172.21.15.155:3300/0 conn(0x7f3b500d0c00 msgr2=0x7f3b500722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.726+0000 7f3b54c00640 1 --2- 172.21.15.86:0/3239738402 >> v2:172.21.15.155:3300/0 conn(0x7f3b500d0c00 0x7f3b500722a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.726+0000 7f3b54c00640 1 -- 172.21.15.86:0/3239738402 >> v1:172.21.15.155:6789/0 conn(0x7f3b500727e0 legacy=0x7f3b501614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.726+0000 7f3b54c00640 1 -- 172.21.15.86:0/3239738402 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b5015e4c0 con 0x7f3b500b6ee0 2024-04-25T14:13:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.726+0000 7f3b47e00640 1 --2- 172.21.15.86:0/3239738402 >> v2:172.21.15.155:3300/0 conn(0x7f3b500d0c00 0x7f3b500722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:13:39.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.726+0000 7f3b54c00640 1 --2- 172.21.15.86:0/3239738402 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b500b6ee0 0x7f3b500d06c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3b400097c0 tx=0x7f3b4001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e8e866c23fe7cfd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.730+0000 7f3b4fe00640 1 -- 172.21.15.86:0/3239738402 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b400b0070 con 0x7f3b500b6ee0 2024-04-25T14:13:39.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.730+0000 7f3b4fe00640 1 -- 172.21.15.86:0/3239738402 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b400ab600 con 0x7f3b500b6ee0 2024-04-25T14:13:39.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.730+0000 7f3b4fe00640 1 -- 172.21.15.86:0/3239738402 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b400ab9d0 con 0x7f3b500b6ee0 2024-04-25T14:13:39.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.730+0000 7f3b56000640 1 -- 172.21.15.86:0/3239738402 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b500b6ee0 msgr2=0x7f3b500d06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.730+0000 7f3b56000640 1 --2- 172.21.15.86:0/3239738402 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b500b6ee0 0x7f3b500d06c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3b400097c0 tx=0x7f3b4001c410 comp rx=0 tx=0).stop 2024-04-25T14:13:39.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.730+0000 7f3b56000640 1 -- 172.21.15.86:0/3239738402 shutdown_connections 2024-04-25T14:13:39.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.730+0000 7f3b56000640 1 --2- 172.21.15.86:0/3239738402 >> v2:172.21.15.155:3300/0 conn(0x7f3b500d0c00 0x7f3b500722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.730+0000 7f3b56000640 1 --2- 172.21.15.86:0/3239738402 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b500b6ee0 0x7f3b500d06c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.730+0000 7f3b56000640 1 -- 172.21.15.86:0/3239738402 >> 172.21.15.86:0/3239738402 conn(0x7f3b500b4f80 msgr2=0x7f3b500b5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.730+0000 7f3b56000640 1 -- 172.21.15.86:0/3239738402 shutdown_connections 2024-04-25T14:13:39.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.730+0000 7f3b56000640 1 -- 172.21.15.86:0/3239738402 wait complete. 2024-04-25T14:13:39.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.730+0000 7f3b56000640 1 Processor -- start 2024-04-25T14:13:39.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.730+0000 7f3b56000640 1 -- start start 2024-04-25T14:13:39.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.734+0000 7f3b56000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b500727e0 0x7f3b50140e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.734+0000 7f3b56000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b500b6ee0 0x7f3b501f74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.734+0000 7f3b56000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b500d0c00 0x7f3b50162980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.734+0000 7f3b56000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b50161fe0 con 0x7f3b500d0c00 2024-04-25T14:13:39.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.734+0000 7f3b56000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3b500cff50 con 0x7f3b500b6ee0 2024-04-25T14:13:39.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.734+0000 7f3b56000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3b50162160 con 0x7f3b500727e0 2024-04-25T14:13:39.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.734+0000 7f3b55600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b500d0c00 0x7f3b50162980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.734+0000 7f3b55600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b500d0c00 0x7f3b50162980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41628/0 (socket says 172.21.15.86:41628) 2024-04-25T14:13:39.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.734+0000 7f3b55600640 1 -- 172.21.15.86:0/103961396 learned_addr learned my addr 172.21.15.86:0/103961396 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:39.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.734+0000 7f3b47e00640 1 --2- 172.21.15.86:0/103961396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b500b6ee0 0x7f3b501f74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.734+0000 7f3b54c00640 1 --2- 172.21.15.86:0/103961396 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b500727e0 0x7f3b50140e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.734+0000 7f3b55600640 1 -- 172.21.15.86:0/103961396 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b500727e0 msgr2=0x7f3b50140e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.734+0000 7f3b55600640 1 --2- 172.21.15.86:0/103961396 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b500727e0 0x7f3b50140e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.734+0000 7f3b55600640 1 -- 172.21.15.86:0/103961396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b500b6ee0 msgr2=0x7f3b501f74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.734+0000 7f3b55600640 1 --2- 172.21.15.86:0/103961396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b500b6ee0 0x7f3b501f74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.734+0000 7f3b55600640 1 -- 172.21.15.86:0/103961396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b501f7fb0 con 0x7f3b500d0c00 2024-04-25T14:13:39.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.734+0000 7f3b55600640 1 --2- 172.21.15.86:0/103961396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b500d0c00 0x7f3b50162980 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f3b4800c720 tx=0x7f3b4800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.734+0000 7f3b4d600640 1 -- 172.21.15.86:0/103961396 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b48016020 con 0x7f3b500d0c00 2024-04-25T14:13:39.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.734+0000 7f3b4d600640 1 -- 172.21.15.86:0/103961396 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b48006d60 con 0x7f3b500d0c00 2024-04-25T14:13:39.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.734+0000 7f3b4d600640 1 -- 172.21.15.86:0/103961396 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b48003bc0 con 0x7f3b500d0c00 2024-04-25T14:13:39.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.734+0000 7f3b56000640 1 -- 172.21.15.86:0/103961396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b501f8290 con 0x7f3b500d0c00 2024-04-25T14:13:39.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.734+0000 7f3b56000640 1 -- 172.21.15.86:0/103961396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b501fa0b0 con 0x7f3b500d0c00 2024-04-25T14:13:39.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.734+0000 7f3b56000640 1 -- 172.21.15.86:0/103961396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b501fa310 con 0x7f3b500d0c00 2024-04-25T14:13:39.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.738+0000 7f3b4d600640 1 -- 172.21.15.86:0/103961396 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7f3b4800a7f0 con 0x7f3b500d0c00 2024-04-25T14:13:39.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.738+0000 7f3b4d600640 1 --2- 172.21.15.86:0/103961396 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f3b2c06f550 0x7f3b2c071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:39.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.738+0000 7f3b4d600640 1 -- 172.21.15.86:0/103961396 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6101+0+0 (secure 0 0 0) 0x7f3b4808fe90 con 0x7f3b500d0c00 2024-04-25T14:13:39.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.738+0000 7f3b54c00640 1 --2- 172.21.15.86:0/103961396 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f3b2c06f550 0x7f3b2c071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:39.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.738+0000 7f3b54c00640 1 --2- 172.21.15.86:0/103961396 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f3b2c06f550 0x7f3b2c071a60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3b400ad000 tx=0x7f3b400a6000 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:39.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.742+0000 7f3b4d600640 1 -- 172.21.15.86:0/103961396 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3b48020020 con 0x7f3b500d0c00 2024-04-25T14:13:39.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.898+0000 7f3b56000640 1 -- 172.21.15.86:0/103961396 --> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3b50076b80 con 0x7f3b2c06f550 2024-04-25T14:13:39.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.902+0000 7f3b4d600640 1 -- 172.21.15.86:0/103961396 <== mgr.8198 v2:172.21.15.86:6832/505730993 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247100 (secure 0 0 0) 0x7f3b50076b80 con 0x7f3b2c06f550 2024-04-25T14:13:39.906 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:13:39.907 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:13:39.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.906+0000 7f3b56000640 1 -- 172.21.15.86:0/103961396 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f3b2c06f550 msgr2=0x7f3b2c071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.906+0000 7f3b56000640 1 --2- 172.21.15.86:0/103961396 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f3b2c06f550 0x7f3b2c071a60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3b400ad000 tx=0x7f3b400a6000 comp rx=0 tx=0).stop 2024-04-25T14:13:39.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.906+0000 7f3b56000640 1 -- 172.21.15.86:0/103961396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b500d0c00 msgr2=0x7f3b50162980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:39.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.906+0000 7f3b56000640 1 --2- 172.21.15.86:0/103961396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b500d0c00 0x7f3b50162980 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f3b4800c720 tx=0x7f3b4800cbe0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.906+0000 7f3b56000640 1 -- 172.21.15.86:0/103961396 shutdown_connections 2024-04-25T14:13:39.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.906+0000 7f3b56000640 1 --2- 172.21.15.86:0/103961396 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f3b2c06f550 0x7f3b2c071a60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.906+0000 7f3b56000640 1 --2- 172.21.15.86:0/103961396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b500d0c00 0x7f3b50162980 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.906+0000 7f3b56000640 1 --2- 172.21.15.86:0/103961396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b500b6ee0 0x7f3b501f74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.906+0000 7f3b56000640 1 --2- 172.21.15.86:0/103961396 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b500727e0 0x7f3b50140e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:39.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.906+0000 7f3b56000640 1 -- 172.21.15.86:0/103961396 >> 172.21.15.86:0/103961396 conn(0x7f3b500b4f80 msgr2=0x7f3b500b7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:39.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.906+0000 7f3b56000640 1 -- 172.21.15.86:0/103961396 shutdown_connections 2024-04-25T14:13:39.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:39.906+0000 7f3b56000640 1 -- 172.21.15.86:0/103961396 wait complete. 2024-04-25T14:13:39.927 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T14:13:38.077493+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":837,"num_read_kb":2279,"num_write":247,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3291,"ondisk_log_size":3291,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257340,"kb_used_data":16188,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754717380,"statfs":{"total":773094113280,"available":772830597120,"internally_reserved":0,"allocated":16576512,"data_stored":8900589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.870692"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.217839+0000","last_change":"2024-04-25T14:13:33.217839+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.217839+0000","last_undegraded":"2024-04-25T14:13:33.217839+0000","last_fullsized":"2024-04-25T14:13:33.217839+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.220477+0000","last_change":"2024-04-25T14:13:33.220477+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.220477+0000","last_undegraded":"2024-04-25T14:13:33.220477+0000","last_fullsized":"2024-04-25T14:13:33.220477+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.216321+0000","last_change":"2024-04-25T14:13:33.216321+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.216321+0000","last_undegraded":"2024-04-25T14:13:33.216321+0000","last_fullsized":"2024-04-25T14:13:33.216321+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.216741+0000","last_change":"2024-04-25T14:13:33.216741+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.216741+0000","last_undegraded":"2024-04-25T14:13:33.216741+0000","last_fullsized":"2024-04-25T14:13:33.216741+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219621+0000","last_change":"2024-04-25T14:13:33.219621+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219621+0000","last_undegraded":"2024-04-25T14:13:33.219621+0000","last_fullsized":"2024-04-25T14:13:33.219621+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.221767+0000","last_change":"2024-04-25T14:13:33.221767+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.221767+0000","last_undegraded":"2024-04-25T14:13:33.221767+0000","last_fullsized":"2024-04-25T14:13:33.221767+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.223617+0000","last_change":"2024-04-25T14:13:33.223617+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.223617+0000","last_undegraded":"2024-04-25T14:13:33.223617+0000","last_fullsized":"2024-04-25T14:13:33.223617+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.228099+0000","last_change":"2024-04-25T14:13:33.228099+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.228099+0000","last_undegraded":"2024-04-25T14:13:33.228099+0000","last_fullsized":"2024-04-25T14:13:33.228099+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.222924+0000","last_change":"2024-04-25T14:13:33.222924+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.222924+0000","last_undegraded":"2024-04-25T14:13:33.222924+0000","last_fullsized":"2024-04-25T14:13:33.222924+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.229249+0000","last_change":"2024-04-25T14:13:33.229249+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.229249+0000","last_undegraded":"2024-04-25T14:13:33.229249+0000","last_fullsized":"2024-04-25T14:13:33.229249+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.231347+0000","last_change":"2024-04-25T14:13:33.231347+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.231347+0000","last_undegraded":"2024-04-25T14:13:33.231347+0000","last_fullsized":"2024-04-25T14:13:33.231347+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.222546+0000","last_change":"2024-04-25T14:13:33.222546+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.222546+0000","last_undegraded":"2024-04-25T14:13:33.222546+0000","last_fullsized":"2024-04-25T14:13:33.222546+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.224550+0000","last_change":"2024-04-25T14:13:33.224550+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.224550+0000","last_undegraded":"2024-04-25T14:13:33.224550+0000","last_fullsized":"2024-04-25T14:13:33.224550+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219360+0000","last_change":"2024-04-25T14:13:33.219360+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219360+0000","last_undegraded":"2024-04-25T14:13:33.219360+0000","last_fullsized":"2024-04-25T14:13:33.219360+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.229250+0000","last_change":"2024-04-25T14:13:33.229250+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.229250+0000","last_undegraded":"2024-04-25T14:13:33.229250+0000","last_fullsized":"2024-04-25T14:13:33.229250+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.216778+0000","last_change":"2024-04-25T14:13:33.216778+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.216778+0000","last_undegraded":"2024-04-25T14:13:33.216778+0000","last_fullsized":"2024-04-25T14:13:33.216778+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.222484+0000","last_change":"2024-04-25T14:13:33.222484+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.222484+0000","last_undegraded":"2024-04-25T14:13:33.222484+0000","last_fullsized":"2024-04-25T14:13:33.222484+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.225152+0000","last_change":"2024-04-25T14:13:33.225152+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.225152+0000","last_undegraded":"2024-04-25T14:13:33.225152+0000","last_fullsized":"2024-04-25T14:13:33.225152+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.228898+0000","last_change":"2024-04-25T14:13:33.228898+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.228898+0000","last_undegraded":"2024-04-25T14:13:33.228898+0000","last_fullsized":"2024-04-25T14:13:33.228898+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.229071+0000","last_change":"2024-04-25T14:13:33.229071+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.229071+0000","last_undegraded":"2024-04-25T14:13:33.229071+0000","last_fullsized":"2024-04-25T14:13:33.229071+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.224104+0000","last_change":"2024-04-25T14:13:33.224104+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.224104+0000","last_undegraded":"2024-04-25T14:13:33.224104+0000","last_fullsized":"2024-04-25T14:13:33.224104+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.221177+0000","last_change":"2024-04-25T14:13:33.221177+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.221177+0000","last_undegraded":"2024-04-25T14:13:33.221177+0000","last_fullsized":"2024-04-25T14:13:33.221177+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.218713+0000","last_change":"2024-04-25T14:13:33.218713+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.218713+0000","last_undegraded":"2024-04-25T14:13:33.218713+0000","last_fullsized":"2024-04-25T14:13:33.218713+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.215974+0000","last_change":"2024-04-25T14:13:33.215974+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.215974+0000","last_undegraded":"2024-04-25T14:13:33.215974+0000","last_fullsized":"2024-04-25T14:13:33.215974+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.222118+0000","last_change":"2024-04-25T14:13:33.222118+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.222118+0000","last_undegraded":"2024-04-25T14:13:33.222118+0000","last_fullsized":"2024-04-25T14:13:33.222118+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.228839+0000","last_change":"2024-04-25T14:13:33.228839+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.228839+0000","last_undegraded":"2024-04-25T14:13:33.228839+0000","last_fullsized":"2024-04-25T14:13:33.228839+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.221814+0000","last_change":"2024-04-25T14:13:33.221814+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.221814+0000","last_undegraded":"2024-04-25T14:13:33.221814+0000","last_fullsized":"2024-04-25T14:13:33.221814+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.223958+0000","last_change":"2024-04-25T14:13:33.223958+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.223958+0000","last_undegraded":"2024-04-25T14:13:33.223958+0000","last_fullsized":"2024-04-25T14:13:33.223958+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219450+0000","last_change":"2024-04-25T14:13:33.219450+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219450+0000","last_undegraded":"2024-04-25T14:13:33.219450+0000","last_fullsized":"2024-04-25T14:13:33.219450+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.224193+0000","last_change":"2024-04-25T14:13:33.224193+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.224193+0000","last_undegraded":"2024-04-25T14:13:33.224193+0000","last_fullsized":"2024-04-25T14:13:33.224193+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.226906+0000","last_change":"2024-04-25T14:13:33.226906+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.226906+0000","last_undegraded":"2024-04-25T14:13:33.226906+0000","last_fullsized":"2024-04-25T14:13:33.226906+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.223332+0000","last_change":"2024-04-25T14:13:33.223332+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.223332+0000","last_undegraded":"2024-04-25T14:13:33.223332+0000","last_fullsized":"2024-04-25T14:13:33.223332+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.217581+0000","last_change":"2024-04-25T14:13:33.217581+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.217581+0000","last_undegraded":"2024-04-25T14:13:33.217581+0000","last_fullsized":"2024-04-25T14:13:33.217581+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.238496+0000","last_change":"2024-04-25T14:13:33.233760+0000","last_active":"2024-04-25T14:13:33.238496+0000","last_peered":"2024-04-25T14:13:33.238496+0000","last_clean":"2024-04-25T14:13:33.238496+0000","last_became_active":"2024-04-25T14:13:33.233378+0000","last_became_peered":"2024-04-25T14:13:33.233378+0000","last_unstale":"2024-04-25T14:13:33.238496+0000","last_undegraded":"2024-04-25T14:13:33.238496+0000","last_fullsized":"2024-04-25T14:13:33.238496+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:24.899595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219833+0000","last_change":"2024-04-25T14:13:33.219833+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219833+0000","last_undegraded":"2024-04-25T14:13:33.219833+0000","last_fullsized":"2024-04-25T14:13:33.219833+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.243213+0000","last_change":"2024-04-25T14:13:33.239975+0000","last_active":"2024-04-25T14:13:33.243213+0000","last_peered":"2024-04-25T14:13:33.243213+0000","last_clean":"2024-04-25T14:13:33.243213+0000","last_became_active":"2024-04-25T14:13:33.239676+0000","last_became_peered":"2024-04-25T14:13:33.239676+0000","last_unstale":"2024-04-25T14:13:33.243213+0000","last_undegraded":"2024-04-25T14:13:33.243213+0000","last_fullsized":"2024-04-25T14:13:33.243213+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:44.934234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.218811+0000","last_change":"2024-04-25T14:13:33.218811+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.218811+0000","last_undegraded":"2024-04-25T14:13:33.218811+0000","last_fullsized":"2024-04-25T14:13:33.218811+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.242321+0000","last_change":"2024-04-25T14:13:33.241587+0000","last_active":"2024-04-25T14:13:33.242321+0000","last_peered":"2024-04-25T14:13:33.242321+0000","last_clean":"2024-04-25T14:13:33.242321+0000","last_became_active":"2024-04-25T14:13:33.240960+0000","last_became_peered":"2024-04-25T14:13:33.240960+0000","last_unstale":"2024-04-25T14:13:33.242321+0000","last_undegraded":"2024-04-25T14:13:33.242321+0000","last_fullsized":"2024-04-25T14:13:33.242321+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:34.418730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.217794+0000","last_change":"2024-04-25T14:13:33.217794+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.217794+0000","last_undegraded":"2024-04-25T14:13:33.217794+0000","last_fullsized":"2024-04-25T14:13:33.217794+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.242412+0000","last_change":"2024-04-25T14:13:33.239782+0000","last_active":"2024-04-25T14:13:33.242412+0000","last_peered":"2024-04-25T14:13:33.242412+0000","last_clean":"2024-04-25T14:13:33.242412+0000","last_became_active":"2024-04-25T14:13:33.239128+0000","last_became_peered":"2024-04-25T14:13:33.239128+0000","last_unstale":"2024-04-25T14:13:33.242412+0000","last_undegraded":"2024-04-25T14:13:33.242412+0000","last_fullsized":"2024-04-25T14:13:33.242412+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:40.423932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.221899+0000","last_change":"2024-04-25T14:13:33.221899+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.221899+0000","last_undegraded":"2024-04-25T14:13:33.221899+0000","last_fullsized":"2024-04-25T14:13:33.221899+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.244471+0000","last_change":"2024-04-25T14:13:33.242875+0000","last_active":"2024-04-25T14:13:33.244471+0000","last_peered":"2024-04-25T14:13:33.244471+0000","last_clean":"2024-04-25T14:13:33.244471+0000","last_became_active":"2024-04-25T14:13:33.242337+0000","last_became_peered":"2024-04-25T14:13:33.242337+0000","last_unstale":"2024-04-25T14:13:33.244471+0000","last_undegraded":"2024-04-25T14:13:33.244471+0000","last_fullsized":"2024-04-25T14:13:33.244471+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:16.518784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.226798+0000","last_change":"2024-04-25T14:13:33.226798+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.226798+0000","last_undegraded":"2024-04-25T14:13:33.226798+0000","last_fullsized":"2024-04-25T14:13:33.226798+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.241274+0000","last_change":"2024-04-25T14:13:33.233642+0000","last_active":"2024-04-25T14:13:33.241274+0000","last_peered":"2024-04-25T14:13:33.241274+0000","last_clean":"2024-04-25T14:13:33.241274+0000","last_became_active":"2024-04-25T14:13:33.233355+0000","last_became_peered":"2024-04-25T14:13:33.233355+0000","last_unstale":"2024-04-25T14:13:33.241274+0000","last_undegraded":"2024-04-25T14:13:33.241274+0000","last_fullsized":"2024-04-25T14:13:33.241274+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:37.218976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.228411+0000","last_change":"2024-04-25T14:13:33.228411+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.228411+0000","last_undegraded":"2024-04-25T14:13:33.228411+0000","last_fullsized":"2024-04-25T14:13:33.228411+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.241759+0000","last_change":"2024-04-25T14:13:33.239682+0000","last_active":"2024-04-25T14:13:33.241759+0000","last_peered":"2024-04-25T14:13:33.241759+0000","last_clean":"2024-04-25T14:13:33.241759+0000","last_became_active":"2024-04-25T14:13:33.239378+0000","last_became_peered":"2024-04-25T14:13:33.239378+0000","last_unstale":"2024-04-25T14:13:33.241759+0000","last_undegraded":"2024-04-25T14:13:33.241759+0000","last_fullsized":"2024-04-25T14:13:33.241759+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:10.873169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.217318+0000","last_change":"2024-04-25T14:13:33.217318+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.217318+0000","last_undegraded":"2024-04-25T14:13:33.217318+0000","last_fullsized":"2024-04-25T14:13:33.217318+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.238019+0000","last_change":"2024-04-25T14:13:33.232394+0000","last_active":"2024-04-25T14:13:33.238019+0000","last_peered":"2024-04-25T14:13:33.238019+0000","last_clean":"2024-04-25T14:13:33.238019+0000","last_became_active":"2024-04-25T14:13:33.231789+0000","last_became_peered":"2024-04-25T14:13:33.231789+0000","last_unstale":"2024-04-25T14:13:33.238019+0000","last_undegraded":"2024-04-25T14:13:33.238019+0000","last_fullsized":"2024-04-25T14:13:33.238019+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:45.600418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.218265+0000","last_change":"2024-04-25T14:13:33.218265+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.218265+0000","last_undegraded":"2024-04-25T14:13:33.218265+0000","last_fullsized":"2024-04-25T14:13:33.218265+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.243068+0000","last_change":"2024-04-25T14:13:33.235450+0000","last_active":"2024-04-25T14:13:33.243068+0000","last_peered":"2024-04-25T14:13:33.243068+0000","last_clean":"2024-04-25T14:13:33.243068+0000","last_became_active":"2024-04-25T14:13:33.235152+0000","last_became_peered":"2024-04-25T14:13:33.235152+0000","last_unstale":"2024-04-25T14:13:33.243068+0000","last_undegraded":"2024-04-25T14:13:33.243068+0000","last_fullsized":"2024-04-25T14:13:33.243068+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:35.585117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219518+0000","last_change":"2024-04-25T14:13:33.219518+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219518+0000","last_undegraded":"2024-04-25T14:13:33.219518+0000","last_fullsized":"2024-04-25T14:13:33.219518+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.238678+0000","last_change":"2024-04-25T14:13:33.235678+0000","last_active":"2024-04-25T14:13:33.238678+0000","last_peered":"2024-04-25T14:13:33.238678+0000","last_clean":"2024-04-25T14:13:33.238678+0000","last_became_active":"2024-04-25T14:13:33.235038+0000","last_became_peered":"2024-04-25T14:13:33.235038+0000","last_unstale":"2024-04-25T14:13:33.238678+0000","last_undegraded":"2024-04-25T14:13:33.238678+0000","last_fullsized":"2024-04-25T14:13:33.238678+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:23.471943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.225982+0000","last_change":"2024-04-25T14:13:33.225982+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.225982+0000","last_undegraded":"2024-04-25T14:13:33.225982+0000","last_fullsized":"2024-04-25T14:13:33.225982+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.243523+0000","last_change":"2024-04-25T14:13:33.235754+0000","last_active":"2024-04-25T14:13:33.243523+0000","last_peered":"2024-04-25T14:13:33.243523+0000","last_clean":"2024-04-25T14:13:33.243523+0000","last_became_active":"2024-04-25T14:13:33.235119+0000","last_became_peered":"2024-04-25T14:13:33.235119+0000","last_unstale":"2024-04-25T14:13:33.243523+0000","last_undegraded":"2024-04-25T14:13:33.243523+0000","last_fullsized":"2024-04-25T14:13:33.243523+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:09.237922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.222837+0000","last_change":"2024-04-25T14:13:33.222837+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.222837+0000","last_undegraded":"2024-04-25T14:13:33.222837+0000","last_fullsized":"2024-04-25T14:13:33.222837+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.240723+0000","last_change":"2024-04-25T14:13:33.237307+0000","last_active":"2024-04-25T14:13:33.240723+0000","last_peered":"2024-04-25T14:13:33.240723+0000","last_clean":"2024-04-25T14:13:33.240723+0000","last_became_active":"2024-04-25T14:13:33.237012+0000","last_became_peered":"2024-04-25T14:13:33.237012+0000","last_unstale":"2024-04-25T14:13:33.240723+0000","last_undegraded":"2024-04-25T14:13:33.240723+0000","last_fullsized":"2024-04-25T14:13:33.240723+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:45.356743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.222740+0000","last_change":"2024-04-25T14:13:33.222740+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.222740+0000","last_undegraded":"2024-04-25T14:13:33.222740+0000","last_fullsized":"2024-04-25T14:13:33.222740+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.241914+0000","last_change":"2024-04-25T14:13:33.234217+0000","last_active":"2024-04-25T14:13:33.241914+0000","last_peered":"2024-04-25T14:13:33.241914+0000","last_clean":"2024-04-25T14:13:33.241914+0000","last_became_active":"2024-04-25T14:13:33.233902+0000","last_became_peered":"2024-04-25T14:13:33.233902+0000","last_unstale":"2024-04-25T14:13:33.241914+0000","last_undegraded":"2024-04-25T14:13:33.241914+0000","last_fullsized":"2024-04-25T14:13:33.241914+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:09.455778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.227060+0000","last_change":"2024-04-25T14:13:33.227060+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.227060+0000","last_undegraded":"2024-04-25T14:13:33.227060+0000","last_fullsized":"2024-04-25T14:13:33.227060+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.240405+0000","last_change":"2024-04-25T14:13:33.239716+0000","last_active":"2024-04-25T14:13:33.240405+0000","last_peered":"2024-04-25T14:13:33.240405+0000","last_clean":"2024-04-25T14:13:33.240405+0000","last_became_active":"2024-04-25T14:13:33.239375+0000","last_became_peered":"2024-04-25T14:13:33.239375+0000","last_unstale":"2024-04-25T14:13:33.240405+0000","last_undegraded":"2024-04-25T14:13:33.240405+0000","last_fullsized":"2024-04-25T14:13:33.240405+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:05.463091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.226077+0000","last_change":"2024-04-25T14:13:33.226077+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.226077+0000","last_undegraded":"2024-04-25T14:13:33.226077+0000","last_fullsized":"2024-04-25T14:13:33.226077+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.239375+0000","last_change":"2024-04-25T14:13:33.238315+0000","last_active":"2024-04-25T14:13:33.239375+0000","last_peered":"2024-04-25T14:13:33.239375+0000","last_clean":"2024-04-25T14:13:33.239375+0000","last_became_active":"2024-04-25T14:13:33.237955+0000","last_became_peered":"2024-04-25T14:13:33.237955+0000","last_unstale":"2024-04-25T14:13:33.239375+0000","last_undegraded":"2024-04-25T14:13:33.239375+0000","last_fullsized":"2024-04-25T14:13:33.239375+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:50.318605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.223560+0000","last_change":"2024-04-25T14:13:33.223560+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.223560+0000","last_undegraded":"2024-04-25T14:13:33.223560+0000","last_fullsized":"2024-04-25T14:13:33.223560+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.237567+0000","last_change":"2024-04-25T14:13:33.226581+0000","last_active":"2024-04-25T14:13:33.237567+0000","last_peered":"2024-04-25T14:13:33.237567+0000","last_clean":"2024-04-25T14:13:33.237567+0000","last_became_active":"2024-04-25T14:13:33.225988+0000","last_became_peered":"2024-04-25T14:13:33.225988+0000","last_unstale":"2024-04-25T14:13:33.237567+0000","last_undegraded":"2024-04-25T14:13:33.237567+0000","last_fullsized":"2024-04-25T14:13:33.237567+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:08.407184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.224808+0000","last_change":"2024-04-25T14:13:33.224808+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.224808+0000","last_undegraded":"2024-04-25T14:13:33.224808+0000","last_fullsized":"2024-04-25T14:13:33.224808+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.244892+0000","last_change":"2024-04-25T14:13:33.239707+0000","last_active":"2024-04-25T14:13:33.244892+0000","last_peered":"2024-04-25T14:13:33.244892+0000","last_clean":"2024-04-25T14:13:33.244892+0000","last_became_active":"2024-04-25T14:13:33.239408+0000","last_became_peered":"2024-04-25T14:13:33.239408+0000","last_unstale":"2024-04-25T14:13:33.244892+0000","last_undegraded":"2024-04-25T14:13:33.244892+0000","last_fullsized":"2024-04-25T14:13:33.244892+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:51:01.497289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.226373+0000","last_change":"2024-04-25T14:13:33.226373+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.226373+0000","last_undegraded":"2024-04-25T14:13:33.226373+0000","last_fullsized":"2024-04-25T14:13:33.226373+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.239011+0000","last_change":"2024-04-25T14:13:33.232965+0000","last_active":"2024-04-25T14:13:33.239011+0000","last_peered":"2024-04-25T14:13:33.239011+0000","last_clean":"2024-04-25T14:13:33.239011+0000","last_became_active":"2024-04-25T14:13:33.232647+0000","last_became_peered":"2024-04-25T14:13:33.232647+0000","last_unstale":"2024-04-25T14:13:33.239011+0000","last_undegraded":"2024-04-25T14:13:33.239011+0000","last_fullsized":"2024-04-25T14:13:33.239011+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:12.089780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.229146+0000","last_change":"2024-04-25T14:13:33.229146+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.229146+0000","last_undegraded":"2024-04-25T14:13:33.229146+0000","last_fullsized":"2024-04-25T14:13:33.229146+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.244360+0000","last_change":"2024-04-25T14:13:33.240376+0000","last_active":"2024-04-25T14:13:33.244360+0000","last_peered":"2024-04-25T14:13:33.244360+0000","last_clean":"2024-04-25T14:13:33.244360+0000","last_became_active":"2024-04-25T14:13:33.240103+0000","last_became_peered":"2024-04-25T14:13:33.240103+0000","last_unstale":"2024-04-25T14:13:33.244360+0000","last_undegraded":"2024-04-25T14:13:33.244360+0000","last_fullsized":"2024-04-25T14:13:33.244360+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:55.225878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219516+0000","last_change":"2024-04-25T14:13:33.219516+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219516+0000","last_undegraded":"2024-04-25T14:13:33.219516+0000","last_fullsized":"2024-04-25T14:13:33.219516+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.240700+0000","last_change":"2024-04-25T14:13:33.231549+0000","last_active":"2024-04-25T14:13:33.240700+0000","last_peered":"2024-04-25T14:13:33.240700+0000","last_clean":"2024-04-25T14:13:33.240700+0000","last_became_active":"2024-04-25T14:13:33.231239+0000","last_became_peered":"2024-04-25T14:13:33.231239+0000","last_unstale":"2024-04-25T14:13:33.240700+0000","last_undegraded":"2024-04-25T14:13:33.240700+0000","last_fullsized":"2024-04-25T14:13:33.240700+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:46.823734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219647+0000","last_change":"2024-04-25T14:13:33.219647+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219647+0000","last_undegraded":"2024-04-25T14:13:33.219647+0000","last_fullsized":"2024-04-25T14:13:33.219647+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.239404+0000","last_change":"2024-04-25T14:13:33.238316+0000","last_active":"2024-04-25T14:13:33.239404+0000","last_peered":"2024-04-25T14:13:33.239404+0000","last_clean":"2024-04-25T14:13:33.239404+0000","last_became_active":"2024-04-25T14:13:33.237984+0000","last_became_peered":"2024-04-25T14:13:33.237984+0000","last_unstale":"2024-04-25T14:13:33.239404+0000","last_undegraded":"2024-04-25T14:13:33.239404+0000","last_fullsized":"2024-04-25T14:13:33.239404+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:01.863933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.220719+0000","last_change":"2024-04-25T14:13:33.220719+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.220719+0000","last_undegraded":"2024-04-25T14:13:33.220719+0000","last_fullsized":"2024-04-25T14:13:33.220719+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.240694+0000","last_change":"2024-04-25T14:13:33.233345+0000","last_active":"2024-04-25T14:13:33.240694+0000","last_peered":"2024-04-25T14:13:33.240694+0000","last_clean":"2024-04-25T14:13:33.240694+0000","last_became_active":"2024-04-25T14:13:33.233045+0000","last_became_peered":"2024-04-25T14:13:33.233045+0000","last_unstale":"2024-04-25T14:13:33.240694+0000","last_undegraded":"2024-04-25T14:13:33.240694+0000","last_fullsized":"2024-04-25T14:13:33.240694+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:28.978236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.218329+0000","last_change":"2024-04-25T14:13:33.218329+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.218329+0000","last_undegraded":"2024-04-25T14:13:33.218329+0000","last_fullsized":"2024-04-25T14:13:33.218329+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.235430+0000","last_change":"2024-04-25T14:13:33.233786+0000","last_active":"2024-04-25T14:13:33.235430+0000","last_peered":"2024-04-25T14:13:33.235430+0000","last_clean":"2024-04-25T14:13:33.235430+0000","last_became_active":"2024-04-25T14:13:33.233479+0000","last_became_peered":"2024-04-25T14:13:33.233479+0000","last_unstale":"2024-04-25T14:13:33.235430+0000","last_undegraded":"2024-04-25T14:13:33.235430+0000","last_fullsized":"2024-04-25T14:13:33.235430+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:57.369446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.225358+0000","last_change":"2024-04-25T14:13:33.225358+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.225358+0000","last_undegraded":"2024-04-25T14:13:33.225358+0000","last_fullsized":"2024-04-25T14:13:33.225358+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.239073+0000","last_change":"2024-04-25T14:13:33.235217+0000","last_active":"2024-04-25T14:13:33.239073+0000","last_peered":"2024-04-25T14:13:33.239073+0000","last_clean":"2024-04-25T14:13:33.239073+0000","last_became_active":"2024-04-25T14:13:33.234923+0000","last_became_peered":"2024-04-25T14:13:33.234923+0000","last_unstale":"2024-04-25T14:13:33.239073+0000","last_undegraded":"2024-04-25T14:13:33.239073+0000","last_fullsized":"2024-04-25T14:13:33.239073+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:16.491560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.223188+0000","last_change":"2024-04-25T14:13:33.223188+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.223188+0000","last_undegraded":"2024-04-25T14:13:33.223188+0000","last_fullsized":"2024-04-25T14:13:33.223188+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.237237+0000","last_change":"2024-04-25T14:13:33.233600+0000","last_active":"2024-04-25T14:13:33.237237+0000","last_peered":"2024-04-25T14:13:33.237237+0000","last_clean":"2024-04-25T14:13:33.237237+0000","last_became_active":"2024-04-25T14:13:33.233290+0000","last_became_peered":"2024-04-25T14:13:33.233290+0000","last_unstale":"2024-04-25T14:13:33.237237+0000","last_undegraded":"2024-04-25T14:13:33.237237+0000","last_fullsized":"2024-04-25T14:13:33.237237+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:16.534415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.226375+0000","last_change":"2024-04-25T14:13:33.226375+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.226375+0000","last_undegraded":"2024-04-25T14:13:33.226375+0000","last_fullsized":"2024-04-25T14:13:33.226375+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.236234+0000","last_change":"2024-04-25T14:13:33.229751+0000","last_active":"2024-04-25T14:13:33.236234+0000","last_peered":"2024-04-25T14:13:33.236234+0000","last_clean":"2024-04-25T14:13:33.236234+0000","last_became_active":"2024-04-25T14:13:33.229451+0000","last_became_peered":"2024-04-25T14:13:33.229451+0000","last_unstale":"2024-04-25T14:13:33.236234+0000","last_undegraded":"2024-04-25T14:13:33.236234+0000","last_fullsized":"2024-04-25T14:13:33.236234+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:34.938781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.220859+0000","last_change":"2024-04-25T14:13:33.220859+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.220859+0000","last_undegraded":"2024-04-25T14:13:33.220859+0000","last_fullsized":"2024-04-25T14:13:33.220859+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.246616+0000","last_change":"2024-04-25T14:13:33.245950+0000","last_active":"2024-04-25T14:13:33.246616+0000","last_peered":"2024-04-25T14:13:33.246616+0000","last_clean":"2024-04-25T14:13:33.246616+0000","last_became_active":"2024-04-25T14:13:33.245627+0000","last_became_peered":"2024-04-25T14:13:33.245627+0000","last_unstale":"2024-04-25T14:13:33.246616+0000","last_undegraded":"2024-04-25T14:13:33.246616+0000","last_fullsized":"2024-04-25T14:13:33.246616+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:59.112002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.216195+0000","last_change":"2024-04-25T14:13:33.216195+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.216195+0000","last_undegraded":"2024-04-25T14:13:33.216195+0000","last_fullsized":"2024-04-25T14:13:33.216195+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.242071+0000","last_change":"2024-04-25T14:13:33.241343+0000","last_active":"2024-04-25T14:13:33.242071+0000","last_peered":"2024-04-25T14:13:33.242071+0000","last_clean":"2024-04-25T14:13:33.242071+0000","last_became_active":"2024-04-25T14:13:33.240975+0000","last_became_peered":"2024-04-25T14:13:33.240975+0000","last_unstale":"2024-04-25T14:13:33.242071+0000","last_undegraded":"2024-04-25T14:13:33.242071+0000","last_fullsized":"2024-04-25T14:13:33.242071+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:43.434745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.217868+0000","last_change":"2024-04-25T14:13:33.217868+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.217868+0000","last_undegraded":"2024-04-25T14:13:33.217868+0000","last_fullsized":"2024-04-25T14:13:33.217868+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.236031+0000","last_change":"2024-04-25T14:13:33.230789+0000","last_active":"2024-04-25T14:13:33.236031+0000","last_peered":"2024-04-25T14:13:33.236031+0000","last_clean":"2024-04-25T14:13:33.236031+0000","last_became_active":"2024-04-25T14:13:33.230422+0000","last_became_peered":"2024-04-25T14:13:33.230422+0000","last_unstale":"2024-04-25T14:13:33.236031+0000","last_undegraded":"2024-04-25T14:13:33.236031+0000","last_fullsized":"2024-04-25T14:13:33.236031+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:05:59.451071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.219902+0000","last_change":"2024-04-25T14:13:33.219902+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.219902+0000","last_undegraded":"2024-04-25T14:13:33.219902+0000","last_fullsized":"2024-04-25T14:13:33.219902+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.242301+0000","last_change":"2024-04-25T14:13:33.241318+0000","last_active":"2024-04-25T14:13:33.242301+0000","last_peered":"2024-04-25T14:13:33.242301+0000","last_clean":"2024-04-25T14:13:33.242301+0000","last_became_active":"2024-04-25T14:13:33.240931+0000","last_became_peered":"2024-04-25T14:13:33.240931+0000","last_unstale":"2024-04-25T14:13:33.242301+0000","last_undegraded":"2024-04-25T14:13:33.242301+0000","last_fullsized":"2024-04-25T14:13:33.242301+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:39.911385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.225571+0000","last_change":"2024-04-25T14:13:33.225571+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.225571+0000","last_undegraded":"2024-04-25T14:13:33.225571+0000","last_fullsized":"2024-04-25T14:13:33.225571+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.238829+0000","last_change":"2024-04-25T14:13:33.229866+0000","last_active":"2024-04-25T14:13:33.238829+0000","last_peered":"2024-04-25T14:13:33.238829+0000","last_clean":"2024-04-25T14:13:33.238829+0000","last_became_active":"2024-04-25T14:13:33.229562+0000","last_became_peered":"2024-04-25T14:13:33.229562+0000","last_unstale":"2024-04-25T14:13:33.238829+0000","last_undegraded":"2024-04-25T14:13:33.238829+0000","last_fullsized":"2024-04-25T14:13:33.238829+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:00.731593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2024-04-25T14:13:33.218315+0000","last_change":"2024-04-25T14:13:33.218315+0000","last_active":"2024-04-25T14:13:33.204667+0000","last_peered":"2024-04-25T14:13:33.204667+0000","last_clean":"2024-04-25T14:13:33.204667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T14:13:33.218315+0000","last_undegraded":"2024-04-25T14:13:33.218315+0000","last_fullsized":"2024-04-25T14:13:33.218315+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.244337+0000","last_change":"2024-04-25T14:13:33.242678+0000","last_active":"2024-04-25T14:13:33.244337+0000","last_peered":"2024-04-25T14:13:33.244337+0000","last_clean":"2024-04-25T14:13:33.244337+0000","last_became_active":"2024-04-25T14:13:33.241884+0000","last_became_peered":"2024-04-25T14:13:33.241884+0000","last_unstale":"2024-04-25T14:13:33.244337+0000","last_undegraded":"2024-04-25T14:13:33.244337+0000","last_fullsized":"2024-04-25T14:13:33.244337+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:00.731593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"144'3291","reported_seq":3828,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T14:13:33.226260+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:13:33.226260+0000","last_peered":"2024-04-25T14:13:33.226260+0000","last_clean":"2024-04-25T14:13:33.226260+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:13:33.226260+0000","last_undegraded":"2024-04-25T14:13:33.226260+0000","last_fullsized":"2024-04-25T14:13:33.226260+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":3291,"log_dups_size":0,"ondisk_log_size":3291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":837,"num_read_kb":2279,"num_write":247,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":837,"num_read_kb":2279,"num_write":247,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3291,"ondisk_log_size":3291,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574963,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29484,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342356,"statfs":{"total":96636764160,"available":96606572544,"internally_reserved":0,"allocated":2519040,"data_stored":1556182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574964,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33064,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1925120,"data_stored":965814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574965,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33064,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1925120,"data_stored":965814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574964,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33064,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1925120,"data_stored":965814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574964,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33068,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1925120,"data_stored":965814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33048,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338792,"statfs":{"total":96636764160,"available":96602923008,"internally_reserved":0,"allocated":1912832,"data_stored":959155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33064,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1925120,"data_stored":965814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574964,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29484,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342356,"statfs":{"total":96636764160,"available":96606572544,"internally_reserved":0,"allocated":2519040,"data_stored":1556182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:13:42.932 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:13:43.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.090+0000 7f6ba8c00640 1 Processor -- start 2024-04-25T14:13:43.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.090+0000 7f6ba8c00640 1 -- start start 2024-04-25T14:13:43.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.094+0000 7f6ba8c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6ba40cde90 0x7f6ba40ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:43.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.094+0000 7f6ba8c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ba415e5c0 con 0x7f6ba40d1df0 2024-04-25T14:13:43.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.094+0000 7f6ba8c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ba415e8a0 con 0x7f6ba4070330 2024-04-25T14:13:43.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.094+0000 7f6ba8c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6ba406fd40 con 0x7f6ba40cde90 2024-04-25T14:13:43.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.094+0000 7f6ba1400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6ba40d1df0 0x7f6ba40cd780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45212/0 (socket says 172.21.15.86:45212) 2024-04-25T14:13:43.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.094+0000 7f6ba1400640 1 -- 172.21.15.86:0/3438833767 learned_addr learned my addr 172.21.15.86:0/3438833767 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:43.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.094+0000 7f6ba2800640 1 --2- 172.21.15.86:0/3438833767 >> v2:172.21.15.86:3300/0 conn(0x7f6ba40cde90 0x7f6ba40ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:43.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.094+0000 7f6ba0a00640 1 -- 172.21.15.86:0/3438833767 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 737004467 0 0) 0x7f6ba415e5c0 con 0x7f6ba40d1df0 2024-04-25T14:13:43.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.094+0000 7f6ba0a00640 1 -- 172.21.15.86:0/3438833767 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b88003530 con 0x7f6ba40d1df0 2024-04-25T14:13:43.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.094+0000 7f6ba2800640 1 -- 172.21.15.86:0/3438833767 >> v1:172.21.15.155:6789/0 conn(0x7f6ba4070330 legacy=0x7f6ba40d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:43.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.094+0000 7f6ba2800640 1 -- 172.21.15.86:0/3438833767 >> v1:172.21.15.86:6789/0 conn(0x7f6ba40d1df0 legacy=0x7f6ba40cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:43.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.094+0000 7f6ba2800640 1 -- 172.21.15.86:0/3438833767 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba415eb80 con 0x7f6ba40cde90 2024-04-25T14:13:43.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.098+0000 7f6ba2800640 1 --2- 172.21.15.86:0/3438833767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ba40cde90 0x7f6ba40ce2a0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009dc0 tx=0x7f6b9c01c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474274e01b05b871 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:43.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.098+0000 7f6ba0a00640 1 -- 172.21.15.86:0/3438833767 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c0b2070 con 0x7f6ba40cde90 2024-04-25T14:13:43.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.098+0000 7f6ba0a00640 1 -- 172.21.15.86:0/3438833767 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b9c0a5860 con 0x7f6ba40cde90 2024-04-25T14:13:43.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.098+0000 7f6ba0a00640 1 -- 172.21.15.86:0/3438833767 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c0a5c30 con 0x7f6ba40cde90 2024-04-25T14:13:43.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.098+0000 7f6ba8c00640 1 -- 172.21.15.86:0/3438833767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ba40cde90 msgr2=0x7f6ba40ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:43.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.098+0000 7f6ba8c00640 1 --2- 172.21.15.86:0/3438833767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ba40cde90 0x7f6ba40ce2a0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009dc0 tx=0x7f6b9c01c890 comp rx=0 tx=0).stop 2024-04-25T14:13:43.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.098+0000 7f6ba8c00640 1 -- 172.21.15.86:0/3438833767 shutdown_connections 2024-04-25T14:13:43.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.098+0000 7f6ba8c00640 1 --2- 172.21.15.86:0/3438833767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ba40cde90 0x7f6ba40ce2a0 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:43.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.098+0000 7f6ba8c00640 1 -- 172.21.15.86:0/3438833767 >> 172.21.15.86:0/3438833767 conn(0x7f6ba40b4650 msgr2=0x7f6ba40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:43.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.098+0000 7f6ba8c00640 1 -- 172.21.15.86:0/3438833767 shutdown_connections 2024-04-25T14:13:43.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.098+0000 7f6ba8c00640 1 -- 172.21.15.86:0/3438833767 wait complete. 2024-04-25T14:13:43.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.098+0000 7f6ba8c00640 1 Processor -- start 2024-04-25T14:13:43.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.098+0000 7f6ba8c00640 1 -- start start 2024-04-25T14:13:43.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.102+0000 7f6ba8c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ba4070330 0x7f6ba4154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:43.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.102+0000 7f6ba8c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ba40cde90 0x7f6ba409a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:43.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.102+0000 7f6ba8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ba41577c0 0x7f6ba4156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:43.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.102+0000 7f6ba8c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba40708c0 con 0x7f6ba41577c0 2024-04-25T14:13:43.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.102+0000 7f6ba8c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba4070740 con 0x7f6ba40cde90 2024-04-25T14:13:43.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.102+0000 7f6ba8c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba416bfd0 con 0x7f6ba4070330 2024-04-25T14:13:43.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.102+0000 7f6ba2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ba41577c0 0x7f6ba4156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:43.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.102+0000 7f6ba2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ba41577c0 0x7f6ba4156b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41644/0 (socket says 172.21.15.86:41644) 2024-04-25T14:13:43.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.102+0000 7f6ba2800640 1 -- 172.21.15.86:0/2576889264 learned_addr learned my addr 172.21.15.86:0/2576889264 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:43.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.102+0000 7f6ba2800640 1 -- 172.21.15.86:0/2576889264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ba4070330 msgr2=0x7f6ba4154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:43.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.102+0000 7f6ba1e00640 1 --2- 172.21.15.86:0/2576889264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ba4070330 0x7f6ba4154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:43.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.102+0000 7f6ba1400640 1 --2- 172.21.15.86:0/2576889264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ba40cde90 0x7f6ba409a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:43.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.102+0000 7f6ba2800640 1 --2- 172.21.15.86:0/2576889264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ba4070330 0x7f6ba4154880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:43.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.102+0000 7f6ba2800640 1 -- 172.21.15.86:0/2576889264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ba40cde90 msgr2=0x7f6ba409a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:43.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.102+0000 7f6ba2800640 1 --2- 172.21.15.86:0/2576889264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ba40cde90 0x7f6ba409a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:43.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.102+0000 7f6ba2800640 1 -- 172.21.15.86:0/2576889264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba4157ea0 con 0x7f6ba41577c0 2024-04-25T14:13:43.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.102+0000 7f6ba1e00640 1 --2- 172.21.15.86:0/2576889264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ba4070330 0x7f6ba4154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:43.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.102+0000 7f6ba1400640 1 --2- 172.21.15.86:0/2576889264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ba40cde90 0x7f6ba409a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:43.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.102+0000 7f6ba2800640 1 --2- 172.21.15.86:0/2576889264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ba41577c0 0x7f6ba4156b20 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0194a0 tx=0x7f6b9c019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:43.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.102+0000 7f6b9a000640 1 -- 172.21.15.86:0/2576889264 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c0b2040 con 0x7f6ba41577c0 2024-04-25T14:13:43.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.102+0000 7f6b9a000640 1 -- 172.21.15.86:0/2576889264 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b9c0bf070 con 0x7f6ba41577c0 2024-04-25T14:13:43.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.102+0000 7f6ba8c00640 1 -- 172.21.15.86:0/2576889264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba4158180 con 0x7f6ba41577c0 2024-04-25T14:13:43.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.102+0000 7f6b9a000640 1 -- 172.21.15.86:0/2576889264 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c0b8690 con 0x7f6ba41577c0 2024-04-25T14:13:43.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.102+0000 7f6ba8c00640 1 -- 172.21.15.86:0/2576889264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba4156320 con 0x7f6ba41577c0 2024-04-25T14:13:43.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.102+0000 7f6ba8c00640 1 -- 172.21.15.86:0/2576889264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba4069d00 con 0x7f6ba41577c0 2024-04-25T14:13:43.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.106+0000 7f6b9a000640 1 -- 172.21.15.86:0/2576889264 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7f6b9c0b8830 con 0x7f6ba41577c0 2024-04-25T14:13:43.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.106+0000 7f6b9a000640 1 --2- 172.21.15.86:0/2576889264 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f6b8406f520 0x7f6b84071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:43.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.106+0000 7f6b9a000640 1 -- 172.21.15.86:0/2576889264 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6101+0+0 (secure 0 0 0) 0x7f6b9c12a860 con 0x7f6ba41577c0 2024-04-25T14:13:43.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.106+0000 7f6ba1e00640 1 --2- 172.21.15.86:0/2576889264 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f6b8406f520 0x7f6b84071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:43.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.106+0000 7f6ba1e00640 1 --2- 172.21.15.86:0/2576889264 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f6b8406f520 0x7f6b84071a30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4072200 tx=0x7f6b94009040 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.110+0000 7f6b9a000640 1 -- 172.21.15.86:0/2576889264 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6b9c016540 con 0x7f6ba41577c0 2024-04-25T14:13:43.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.262+0000 7f6ba8c00640 1 -- 172.21.15.86:0/2576889264 --> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6ba4157c30 con 0x7f6b8406f520 2024-04-25T14:13:43.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.270+0000 7f6b9a000640 1 -- 172.21.15.86:0/2576889264 <== mgr.8198 v2:172.21.15.86:6832/505730993 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252491 (secure 0 0 0) 0x7f6ba4157c30 con 0x7f6b8406f520 2024-04-25T14:13:43.272 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:13:43.274 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:13:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.274+0000 7f6ba8c00640 1 -- 172.21.15.86:0/2576889264 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f6b8406f520 msgr2=0x7f6b84071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.274+0000 7f6ba8c00640 1 --2- 172.21.15.86:0/2576889264 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f6b8406f520 0x7f6b84071a30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4072200 tx=0x7f6b94009040 comp rx=0 tx=0).stop 2024-04-25T14:13:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.274+0000 7f6ba8c00640 1 -- 172.21.15.86:0/2576889264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ba41577c0 msgr2=0x7f6ba4156b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.274+0000 7f6ba8c00640 1 --2- 172.21.15.86:0/2576889264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ba41577c0 0x7f6ba4156b20 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0194a0 tx=0x7f6b9c019630 comp rx=0 tx=0).stop 2024-04-25T14:13:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.274+0000 7f6ba8c00640 1 -- 172.21.15.86:0/2576889264 shutdown_connections 2024-04-25T14:13:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.274+0000 7f6ba8c00640 1 --2- 172.21.15.86:0/2576889264 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f6b8406f520 0x7f6b84071a30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.274+0000 7f6ba8c00640 1 --2- 172.21.15.86:0/2576889264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ba41577c0 0x7f6ba4156b20 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.274+0000 7f6ba8c00640 1 --2- 172.21.15.86:0/2576889264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ba40cde90 0x7f6ba409a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.274+0000 7f6ba8c00640 1 --2- 172.21.15.86:0/2576889264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ba4070330 0x7f6ba4154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:43.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.274+0000 7f6ba8c00640 1 -- 172.21.15.86:0/2576889264 >> 172.21.15.86:0/2576889264 conn(0x7f6ba40b4650 msgr2=0x7f6ba40ae130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:43.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.274+0000 7f6ba8c00640 1 -- 172.21.15.86:0/2576889264 shutdown_connections 2024-04-25T14:13:43.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.274+0000 7f6ba8c00640 1 -- 172.21.15.86:0/2576889264 wait complete. 2024-04-25T14:13:43.294 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-04-25T14:13:42.078461+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":837,"num_read_kb":2279,"num_write":292,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3330,"ondisk_log_size":3330,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257580,"kb_used_data":16456,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754717140,"statfs":{"total":773094113280,"available":772830351360,"internally_reserved":0,"allocated":16850944,"data_stored":9022940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001959"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.231400+0000","last_change":"2024-04-25T14:13:34.235951+0000","last_active":"2024-04-25T14:13:35.231400+0000","last_peered":"2024-04-25T14:13:35.231400+0000","last_clean":"2024-04-25T14:13:35.231400+0000","last_became_active":"2024-04-25T14:13:34.235563+0000","last_became_peered":"2024-04-25T14:13:34.235563+0000","last_unstale":"2024-04-25T14:13:35.231400+0000","last_undegraded":"2024-04-25T14:13:35.231400+0000","last_fullsized":"2024-04-25T14:13:35.231400+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:46.608977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.351526+0000","last_change":"2024-04-25T14:13:34.240499+0000","last_active":"2024-04-25T14:13:35.351526+0000","last_peered":"2024-04-25T14:13:35.351526+0000","last_clean":"2024-04-25T14:13:35.351526+0000","last_became_active":"2024-04-25T14:13:34.240111+0000","last_became_peered":"2024-04-25T14:13:34.240111+0000","last_unstale":"2024-04-25T14:13:35.351526+0000","last_undegraded":"2024-04-25T14:13:35.351526+0000","last_fullsized":"2024-04-25T14:13:35.351526+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:38.227535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.231465+0000","last_change":"2024-04-25T14:13:34.234398+0000","last_active":"2024-04-25T14:13:35.231465+0000","last_peered":"2024-04-25T14:13:35.231465+0000","last_clean":"2024-04-25T14:13:35.231465+0000","last_became_active":"2024-04-25T14:13:34.234036+0000","last_became_peered":"2024-04-25T14:13:34.234036+0000","last_unstale":"2024-04-25T14:13:35.231465+0000","last_undegraded":"2024-04-25T14:13:35.231465+0000","last_fullsized":"2024-04-25T14:13:35.231465+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:51:02.505848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.231368+0000","last_change":"2024-04-25T14:13:34.244307+0000","last_active":"2024-04-25T14:13:35.231368+0000","last_peered":"2024-04-25T14:13:35.231368+0000","last_clean":"2024-04-25T14:13:35.231368+0000","last_became_active":"2024-04-25T14:13:34.242654+0000","last_became_peered":"2024-04-25T14:13:34.242654+0000","last_unstale":"2024-04-25T14:13:35.231368+0000","last_undegraded":"2024-04-25T14:13:35.231368+0000","last_fullsized":"2024-04-25T14:13:35.231368+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:03.984540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.338292+0000","last_change":"2024-04-25T14:13:34.225591+0000","last_active":"2024-04-25T14:13:35.338292+0000","last_peered":"2024-04-25T14:13:35.338292+0000","last_clean":"2024-04-25T14:13:35.338292+0000","last_became_active":"2024-04-25T14:13:34.225271+0000","last_became_peered":"2024-04-25T14:13:34.225271+0000","last_unstale":"2024-04-25T14:13:35.338292+0000","last_undegraded":"2024-04-25T14:13:35.338292+0000","last_fullsized":"2024-04-25T14:13:35.338292+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:37.998063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.350912+0000","last_change":"2024-04-25T14:13:34.247124+0000","last_active":"2024-04-25T14:13:35.350912+0000","last_peered":"2024-04-25T14:13:35.350912+0000","last_clean":"2024-04-25T14:13:35.350912+0000","last_became_active":"2024-04-25T14:13:34.246694+0000","last_became_peered":"2024-04-25T14:13:34.246694+0000","last_unstale":"2024-04-25T14:13:35.350912+0000","last_undegraded":"2024-04-25T14:13:35.350912+0000","last_fullsized":"2024-04-25T14:13:35.350912+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:09.415743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.242998+0000","last_change":"2024-04-25T14:13:34.238705+0000","last_active":"2024-04-25T14:13:35.242998+0000","last_peered":"2024-04-25T14:13:35.242998+0000","last_clean":"2024-04-25T14:13:35.242998+0000","last_became_active":"2024-04-25T14:13:34.238385+0000","last_became_peered":"2024-04-25T14:13:34.238385+0000","last_unstale":"2024-04-25T14:13:35.242998+0000","last_undegraded":"2024-04-25T14:13:35.242998+0000","last_fullsized":"2024-04-25T14:13:35.242998+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:45.942793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.242984+0000","last_change":"2024-04-25T14:13:34.244352+0000","last_active":"2024-04-25T14:13:35.242984+0000","last_peered":"2024-04-25T14:13:35.242984+0000","last_clean":"2024-04-25T14:13:35.242984+0000","last_became_active":"2024-04-25T14:13:34.244019+0000","last_became_peered":"2024-04-25T14:13:34.244019+0000","last_unstale":"2024-04-25T14:13:35.242984+0000","last_undegraded":"2024-04-25T14:13:35.242984+0000","last_fullsized":"2024-04-25T14:13:35.242984+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:46.365302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.243645+0000","last_change":"2024-04-25T14:13:34.243723+0000","last_active":"2024-04-25T14:13:35.243645+0000","last_peered":"2024-04-25T14:13:35.243645+0000","last_clean":"2024-04-25T14:13:35.243645+0000","last_became_active":"2024-04-25T14:13:34.243419+0000","last_became_peered":"2024-04-25T14:13:34.243419+0000","last_unstale":"2024-04-25T14:13:35.243645+0000","last_undegraded":"2024-04-25T14:13:35.243645+0000","last_fullsized":"2024-04-25T14:13:35.243645+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:14.291661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.243489+0000","last_change":"2024-04-25T14:13:34.246116+0000","last_active":"2024-04-25T14:13:35.243489+0000","last_peered":"2024-04-25T14:13:35.243489+0000","last_clean":"2024-04-25T14:13:35.243489+0000","last_became_active":"2024-04-25T14:13:34.245829+0000","last_became_peered":"2024-04-25T14:13:34.245829+0000","last_unstale":"2024-04-25T14:13:35.243489+0000","last_undegraded":"2024-04-25T14:13:35.243489+0000","last_fullsized":"2024-04-25T14:13:35.243489+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:17.249909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.230388+0000","last_change":"2024-04-25T14:13:34.224665+0000","last_active":"2024-04-25T14:13:35.230388+0000","last_peered":"2024-04-25T14:13:35.230388+0000","last_clean":"2024-04-25T14:13:35.230388+0000","last_became_active":"2024-04-25T14:13:34.224339+0000","last_became_peered":"2024-04-25T14:13:34.224339+0000","last_unstale":"2024-04-25T14:13:35.230388+0000","last_undegraded":"2024-04-25T14:13:35.230388+0000","last_fullsized":"2024-04-25T14:13:35.230388+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:27.201346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.351311+0000","last_change":"2024-04-25T14:13:34.245417+0000","last_active":"2024-04-25T14:13:35.351311+0000","last_peered":"2024-04-25T14:13:35.351311+0000","last_clean":"2024-04-25T14:13:35.351311+0000","last_became_active":"2024-04-25T14:13:34.245125+0000","last_became_peered":"2024-04-25T14:13:34.245125+0000","last_unstale":"2024-04-25T14:13:35.351311+0000","last_undegraded":"2024-04-25T14:13:35.351311+0000","last_fullsized":"2024-04-25T14:13:35.351311+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:10.246481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.229264+0000","last_change":"2024-04-25T14:13:34.252059+0000","last_active":"2024-04-25T14:13:35.229264+0000","last_peered":"2024-04-25T14:13:35.229264+0000","last_clean":"2024-04-25T14:13:35.229264+0000","last_became_active":"2024-04-25T14:13:34.251753+0000","last_became_peered":"2024-04-25T14:13:34.251753+0000","last_unstale":"2024-04-25T14:13:35.229264+0000","last_undegraded":"2024-04-25T14:13:35.229264+0000","last_fullsized":"2024-04-25T14:13:35.229264+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:16.746440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.337850+0000","last_change":"2024-04-25T14:13:34.240594+0000","last_active":"2024-04-25T14:13:35.337850+0000","last_peered":"2024-04-25T14:13:35.337850+0000","last_clean":"2024-04-25T14:13:35.337850+0000","last_became_active":"2024-04-25T14:13:34.235963+0000","last_became_peered":"2024-04-25T14:13:34.235963+0000","last_unstale":"2024-04-25T14:13:35.337850+0000","last_undegraded":"2024-04-25T14:13:35.337850+0000","last_fullsized":"2024-04-25T14:13:35.337850+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:11.878423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.230243+0000","last_change":"2024-04-25T14:13:34.237092+0000","last_active":"2024-04-25T14:13:35.230243+0000","last_peered":"2024-04-25T14:13:35.230243+0000","last_clean":"2024-04-25T14:13:35.230243+0000","last_became_active":"2024-04-25T14:13:34.236783+0000","last_became_peered":"2024-04-25T14:13:34.236783+0000","last_unstale":"2024-04-25T14:13:35.230243+0000","last_undegraded":"2024-04-25T14:13:35.230243+0000","last_fullsized":"2024-04-25T14:13:35.230243+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:09.516317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.230030+0000","last_change":"2024-04-25T14:13:34.250675+0000","last_active":"2024-04-25T14:13:35.230030+0000","last_peered":"2024-04-25T14:13:35.230030+0000","last_clean":"2024-04-25T14:13:35.230030+0000","last_became_active":"2024-04-25T14:13:34.250312+0000","last_became_peered":"2024-04-25T14:13:34.250312+0000","last_unstale":"2024-04-25T14:13:35.230030+0000","last_undegraded":"2024-04-25T14:13:35.230030+0000","last_fullsized":"2024-04-25T14:13:35.230030+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:49.606889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.243609+0000","last_change":"2024-04-25T14:13:34.240621+0000","last_active":"2024-04-25T14:13:35.243609+0000","last_peered":"2024-04-25T14:13:35.243609+0000","last_clean":"2024-04-25T14:13:35.243609+0000","last_became_active":"2024-04-25T14:13:34.240288+0000","last_became_peered":"2024-04-25T14:13:34.240288+0000","last_unstale":"2024-04-25T14:13:35.243609+0000","last_undegraded":"2024-04-25T14:13:35.243609+0000","last_fullsized":"2024-04-25T14:13:35.243609+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:53.850900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.350599+0000","last_change":"2024-04-25T14:13:34.243576+0000","last_active":"2024-04-25T14:13:35.350599+0000","last_peered":"2024-04-25T14:13:35.350599+0000","last_clean":"2024-04-25T14:13:35.350599+0000","last_became_active":"2024-04-25T14:13:34.243122+0000","last_became_peered":"2024-04-25T14:13:34.243122+0000","last_unstale":"2024-04-25T14:13:35.350599+0000","last_undegraded":"2024-04-25T14:13:35.350599+0000","last_fullsized":"2024-04-25T14:13:35.350599+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:04.986512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.230625+0000","last_change":"2024-04-25T14:13:34.252024+0000","last_active":"2024-04-25T14:13:35.230625+0000","last_peered":"2024-04-25T14:13:35.230625+0000","last_clean":"2024-04-25T14:13:35.230625+0000","last_became_active":"2024-04-25T14:13:34.251706+0000","last_became_peered":"2024-04-25T14:13:34.251706+0000","last_unstale":"2024-04-25T14:13:35.230625+0000","last_undegraded":"2024-04-25T14:13:35.230625+0000","last_fullsized":"2024-04-25T14:13:35.230625+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:35.947339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.349614+0000","last_change":"2024-04-25T14:13:34.233701+0000","last_active":"2024-04-25T14:13:35.349614+0000","last_peered":"2024-04-25T14:13:35.349614+0000","last_clean":"2024-04-25T14:13:35.349614+0000","last_became_active":"2024-04-25T14:13:34.233171+0000","last_became_peered":"2024-04-25T14:13:34.233171+0000","last_unstale":"2024-04-25T14:13:35.349614+0000","last_undegraded":"2024-04-25T14:13:35.349614+0000","last_fullsized":"2024-04-25T14:13:35.349614+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:05.883749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.229430+0000","last_change":"2024-04-25T14:13:34.250231+0000","last_active":"2024-04-25T14:13:35.229430+0000","last_peered":"2024-04-25T14:13:35.229430+0000","last_clean":"2024-04-25T14:13:35.229430+0000","last_became_active":"2024-04-25T14:13:34.248513+0000","last_became_peered":"2024-04-25T14:13:34.248513+0000","last_unstale":"2024-04-25T14:13:35.229430+0000","last_undegraded":"2024-04-25T14:13:35.229430+0000","last_fullsized":"2024-04-25T14:13:35.229430+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:11.878423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.350947+0000","last_change":"2024-04-25T14:13:34.248870+0000","last_active":"2024-04-25T14:13:35.350947+0000","last_peered":"2024-04-25T14:13:35.350947+0000","last_clean":"2024-04-25T14:13:35.350947+0000","last_became_active":"2024-04-25T14:13:34.248515+0000","last_became_peered":"2024-04-25T14:13:34.248515+0000","last_unstale":"2024-04-25T14:13:35.350947+0000","last_undegraded":"2024-04-25T14:13:35.350947+0000","last_fullsized":"2024-04-25T14:13:35.350947+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:17.542973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.337762+0000","last_change":"2024-04-25T14:13:34.236844+0000","last_active":"2024-04-25T14:13:35.337762+0000","last_peered":"2024-04-25T14:13:35.337762+0000","last_clean":"2024-04-25T14:13:35.337762+0000","last_became_active":"2024-04-25T14:13:34.236547+0000","last_became_peered":"2024-04-25T14:13:34.236547+0000","last_unstale":"2024-04-25T14:13:35.337762+0000","last_undegraded":"2024-04-25T14:13:35.337762+0000","last_fullsized":"2024-04-25T14:13:35.337762+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:06.596651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.230593+0000","last_change":"2024-04-25T14:13:34.250670+0000","last_active":"2024-04-25T14:13:35.230593+0000","last_peered":"2024-04-25T14:13:35.230593+0000","last_clean":"2024-04-25T14:13:35.230593+0000","last_became_active":"2024-04-25T14:13:34.250287+0000","last_became_peered":"2024-04-25T14:13:34.250287+0000","last_unstale":"2024-04-25T14:13:35.230593+0000","last_undegraded":"2024-04-25T14:13:35.230593+0000","last_fullsized":"2024-04-25T14:13:35.230593+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:02.600269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.229732+0000","last_change":"2024-04-25T14:13:34.250669+0000","last_active":"2024-04-25T14:13:35.229732+0000","last_peered":"2024-04-25T14:13:35.229732+0000","last_clean":"2024-04-25T14:13:35.229732+0000","last_became_active":"2024-04-25T14:13:34.245287+0000","last_became_peered":"2024-04-25T14:13:34.245287+0000","last_unstale":"2024-04-25T14:13:35.229732+0000","last_undegraded":"2024-04-25T14:13:35.229732+0000","last_fullsized":"2024-04-25T14:13:35.229732+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:42.807728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.349897+0000","last_change":"2024-04-25T14:13:34.234049+0000","last_active":"2024-04-25T14:13:35.349897+0000","last_peered":"2024-04-25T14:13:35.349897+0000","last_clean":"2024-04-25T14:13:35.349897+0000","last_became_active":"2024-04-25T14:13:34.233708+0000","last_became_peered":"2024-04-25T14:13:34.233708+0000","last_unstale":"2024-04-25T14:13:35.349897+0000","last_undegraded":"2024-04-25T14:13:35.349897+0000","last_fullsized":"2024-04-25T14:13:35.349897+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:30.206775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.351439+0000","last_change":"2024-04-25T14:13:34.251397+0000","last_active":"2024-04-25T14:13:35.351439+0000","last_peered":"2024-04-25T14:13:35.351439+0000","last_clean":"2024-04-25T14:13:35.351439+0000","last_became_active":"2024-04-25T14:13:34.250975+0000","last_became_peered":"2024-04-25T14:13:34.250975+0000","last_unstale":"2024-04-25T14:13:35.351439+0000","last_undegraded":"2024-04-25T14:13:35.351439+0000","last_fullsized":"2024-04-25T14:13:35.351439+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:47.832293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.229257+0000","last_change":"2024-04-25T14:13:34.242764+0000","last_active":"2024-04-25T14:13:35.229257+0000","last_peered":"2024-04-25T14:13:35.229257+0000","last_clean":"2024-04-25T14:13:35.229257+0000","last_became_active":"2024-04-25T14:13:34.242408+0000","last_became_peered":"2024-04-25T14:13:34.242408+0000","last_unstale":"2024-04-25T14:13:35.229257+0000","last_undegraded":"2024-04-25T14:13:35.229257+0000","last_fullsized":"2024-04-25T14:13:35.229257+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:16.736813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.351180+0000","last_change":"2024-04-25T14:13:34.238413+0000","last_active":"2024-04-25T14:13:35.351180+0000","last_peered":"2024-04-25T14:13:35.351180+0000","last_clean":"2024-04-25T14:13:35.351180+0000","last_became_active":"2024-04-25T14:13:34.238078+0000","last_became_peered":"2024-04-25T14:13:34.238078+0000","last_unstale":"2024-04-25T14:13:35.351180+0000","last_undegraded":"2024-04-25T14:13:35.351180+0000","last_fullsized":"2024-04-25T14:13:35.351180+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:36.593676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.350269+0000","last_change":"2024-04-25T14:13:34.234942+0000","last_active":"2024-04-25T14:13:35.350269+0000","last_peered":"2024-04-25T14:13:35.350269+0000","last_clean":"2024-04-25T14:13:35.350269+0000","last_became_active":"2024-04-25T14:13:34.234617+0000","last_became_peered":"2024-04-25T14:13:34.234617+0000","last_unstale":"2024-04-25T14:13:35.350269+0000","last_undegraded":"2024-04-25T14:13:35.350269+0000","last_fullsized":"2024-04-25T14:13:35.350269+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:32.307429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.230372+0000","last_change":"2024-04-25T14:13:34.240140+0000","last_active":"2024-04-25T14:13:35.230372+0000","last_peered":"2024-04-25T14:13:35.230372+0000","last_clean":"2024-04-25T14:13:35.230372+0000","last_became_active":"2024-04-25T14:13:34.239296+0000","last_became_peered":"2024-04-25T14:13:34.239296+0000","last_unstale":"2024-04-25T14:13:35.230372+0000","last_undegraded":"2024-04-25T14:13:35.230372+0000","last_fullsized":"2024-04-25T14:13:35.230372+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:06:00.459630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.350130+0000","last_change":"2024-04-25T14:13:34.244386+0000","last_active":"2024-04-25T14:13:35.350130+0000","last_peered":"2024-04-25T14:13:35.350130+0000","last_clean":"2024-04-25T14:13:35.350130+0000","last_became_active":"2024-04-25T14:13:34.242988+0000","last_became_peered":"2024-04-25T14:13:34.242988+0000","last_unstale":"2024-04-25T14:13:35.350130+0000","last_undegraded":"2024-04-25T14:13:35.350130+0000","last_fullsized":"2024-04-25T14:13:35.350130+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:35.427289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.351017+0000","last_change":"2024-04-25T14:13:34.262715+0000","last_active":"2024-04-25T14:13:35.351017+0000","last_peered":"2024-04-25T14:13:35.351017+0000","last_clean":"2024-04-25T14:13:35.351017+0000","last_became_active":"2024-04-25T14:13:34.262438+0000","last_became_peered":"2024-04-25T14:13:34.262438+0000","last_unstale":"2024-04-25T14:13:35.351017+0000","last_undegraded":"2024-04-25T14:13:35.351017+0000","last_fullsized":"2024-04-25T14:13:35.351017+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:10.464336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"154'4","reported_seq":18,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.231622+0000","last_change":"2024-04-25T14:13:33.233760+0000","last_active":"2024-04-25T14:13:35.231622+0000","last_peered":"2024-04-25T14:13:35.231622+0000","last_clean":"2024-04-25T14:13:35.231622+0000","last_became_active":"2024-04-25T14:13:33.233378+0000","last_became_peered":"2024-04-25T14:13:33.233378+0000","last_unstale":"2024-04-25T14:13:35.231622+0000","last_undegraded":"2024-04-25T14:13:35.231622+0000","last_fullsized":"2024-04-25T14:13:35.231622+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:01.863933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.229267+0000","last_change":"2024-04-25T14:13:34.246364+0000","last_active":"2024-04-25T14:13:35.229267+0000","last_peered":"2024-04-25T14:13:35.229267+0000","last_clean":"2024-04-25T14:13:35.229267+0000","last_became_active":"2024-04-25T14:13:34.246089+0000","last_became_peered":"2024-04-25T14:13:34.246089+0000","last_unstale":"2024-04-25T14:13:35.229267+0000","last_undegraded":"2024-04-25T14:13:35.229267+0000","last_fullsized":"2024-04-25T14:13:35.229267+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:06.596651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"154'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.338060+0000","last_change":"2024-04-25T14:13:33.239975+0000","last_active":"2024-04-25T14:13:35.338060+0000","last_peered":"2024-04-25T14:13:35.338060+0000","last_clean":"2024-04-25T14:13:35.338060+0000","last_became_active":"2024-04-25T14:13:33.239676+0000","last_became_peered":"2024-04-25T14:13:33.239676+0000","last_unstale":"2024-04-25T14:13:35.338060+0000","last_undegraded":"2024-04-25T14:13:35.338060+0000","last_fullsized":"2024-04-25T14:13:35.338060+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:16.241350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.338185+0000","last_change":"2024-04-25T14:13:34.237994+0000","last_active":"2024-04-25T14:13:35.338185+0000","last_peered":"2024-04-25T14:13:35.338185+0000","last_clean":"2024-04-25T14:13:35.338185+0000","last_became_active":"2024-04-25T14:13:34.235540+0000","last_became_peered":"2024-04-25T14:13:34.235540+0000","last_unstale":"2024-04-25T14:13:35.338185+0000","last_undegraded":"2024-04-25T14:13:35.338185+0000","last_fullsized":"2024-04-25T14:13:35.338185+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:14.621116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.230833+0000","last_change":"2024-04-25T14:13:33.241587+0000","last_active":"2024-04-25T14:13:35.230833+0000","last_peered":"2024-04-25T14:13:35.230833+0000","last_clean":"2024-04-25T14:13:35.230833+0000","last_became_active":"2024-04-25T14:13:33.240960+0000","last_became_peered":"2024-04-25T14:13:33.240960+0000","last_unstale":"2024-04-25T14:13:35.230833+0000","last_undegraded":"2024-04-25T14:13:35.230833+0000","last_fullsized":"2024-04-25T14:13:35.230833+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:57.369446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.351452+0000","last_change":"2024-04-25T14:13:34.263442+0000","last_active":"2024-04-25T14:13:35.351452+0000","last_peered":"2024-04-25T14:13:35.351452+0000","last_clean":"2024-04-25T14:13:35.351452+0000","last_became_active":"2024-04-25T14:13:34.263086+0000","last_became_peered":"2024-04-25T14:13:34.263086+0000","last_unstale":"2024-04-25T14:13:35.351452+0000","last_undegraded":"2024-04-25T14:13:35.351452+0000","last_fullsized":"2024-04-25T14:13:35.351452+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:13.098339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.349679+0000","last_change":"2024-04-25T14:13:33.239782+0000","last_active":"2024-04-25T14:13:35.349679+0000","last_peered":"2024-04-25T14:13:35.349679+0000","last_clean":"2024-04-25T14:13:35.349679+0000","last_became_active":"2024-04-25T14:13:33.239128+0000","last_became_peered":"2024-04-25T14:13:33.239128+0000","last_unstale":"2024-04-25T14:13:35.349679+0000","last_undegraded":"2024-04-25T14:13:35.349679+0000","last_fullsized":"2024-04-25T14:13:35.349679+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:02.611193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.243349+0000","last_change":"2024-04-25T14:13:34.244388+0000","last_active":"2024-04-25T14:13:35.243349+0000","last_peered":"2024-04-25T14:13:35.243349+0000","last_clean":"2024-04-25T14:13:35.243349+0000","last_became_active":"2024-04-25T14:13:34.244093+0000","last_became_peered":"2024-04-25T14:13:34.244093+0000","last_unstale":"2024-04-25T14:13:35.243349+0000","last_undegraded":"2024-04-25T14:13:35.243349+0000","last_fullsized":"2024-04-25T14:13:35.243349+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:37.448698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"154'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.243141+0000","last_change":"2024-04-25T14:13:33.242875+0000","last_active":"2024-04-25T14:13:35.243141+0000","last_peered":"2024-04-25T14:13:35.243141+0000","last_clean":"2024-04-25T14:13:35.243141+0000","last_became_active":"2024-04-25T14:13:33.242337+0000","last_became_peered":"2024-04-25T14:13:33.242337+0000","last_unstale":"2024-04-25T14:13:35.243141+0000","last_undegraded":"2024-04-25T14:13:35.243141+0000","last_fullsized":"2024-04-25T14:13:35.243141+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:05:59.451071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.349875+0000","last_change":"2024-04-25T14:13:34.247909+0000","last_active":"2024-04-25T14:13:35.349875+0000","last_peered":"2024-04-25T14:13:35.349875+0000","last_clean":"2024-04-25T14:13:35.349875+0000","last_became_active":"2024-04-25T14:13:34.245685+0000","last_became_peered":"2024-04-25T14:13:34.245685+0000","last_unstale":"2024-04-25T14:13:35.349875+0000","last_undegraded":"2024-04-25T14:13:35.349875+0000","last_fullsized":"2024-04-25T14:13:35.349875+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:32.202716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"154'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.337975+0000","last_change":"2024-04-25T14:13:33.233642+0000","last_active":"2024-04-25T14:13:35.337975+0000","last_peered":"2024-04-25T14:13:35.337975+0000","last_clean":"2024-04-25T14:13:35.337975+0000","last_became_active":"2024-04-25T14:13:33.233355+0000","last_became_peered":"2024-04-25T14:13:33.233355+0000","last_unstale":"2024-04-25T14:13:35.337975+0000","last_undegraded":"2024-04-25T14:13:35.337975+0000","last_fullsized":"2024-04-25T14:13:35.337975+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:52.842341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.349946+0000","last_change":"2024-04-25T14:13:34.237020+0000","last_active":"2024-04-25T14:13:35.349946+0000","last_peered":"2024-04-25T14:13:35.349946+0000","last_clean":"2024-04-25T14:13:35.349946+0000","last_became_active":"2024-04-25T14:13:34.236696+0000","last_became_peered":"2024-04-25T14:13:34.236696+0000","last_unstale":"2024-04-25T14:13:35.349946+0000","last_undegraded":"2024-04-25T14:13:35.349946+0000","last_fullsized":"2024-04-25T14:13:35.349946+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:21.931561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.349462+0000","last_change":"2024-04-25T14:13:33.239682+0000","last_active":"2024-04-25T14:13:35.349462+0000","last_peered":"2024-04-25T14:13:35.349462+0000","last_clean":"2024-04-25T14:13:35.349462+0000","last_became_active":"2024-04-25T14:13:33.239378+0000","last_became_peered":"2024-04-25T14:13:33.239378+0000","last_unstale":"2024-04-25T14:13:35.349462+0000","last_undegraded":"2024-04-25T14:13:35.349462+0000","last_fullsized":"2024-04-25T14:13:35.349462+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:48.954987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.231478+0000","last_change":"2024-04-25T14:13:34.230987+0000","last_active":"2024-04-25T14:13:35.231478+0000","last_peered":"2024-04-25T14:13:35.231478+0000","last_clean":"2024-04-25T14:13:35.231478+0000","last_became_active":"2024-04-25T14:13:34.230669+0000","last_became_peered":"2024-04-25T14:13:34.230669+0000","last_unstale":"2024-04-25T14:13:35.231478+0000","last_undegraded":"2024-04-25T14:13:35.231478+0000","last_fullsized":"2024-04-25T14:13:35.231478+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:56.234437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.243127+0000","last_change":"2024-04-25T14:13:33.232394+0000","last_active":"2024-04-25T14:13:35.243127+0000","last_peered":"2024-04-25T14:13:35.243127+0000","last_clean":"2024-04-25T14:13:35.243127+0000","last_became_active":"2024-04-25T14:13:33.231789+0000","last_became_peered":"2024-04-25T14:13:33.231789+0000","last_unstale":"2024-04-25T14:13:35.243127+0000","last_undegraded":"2024-04-25T14:13:35.243127+0000","last_fullsized":"2024-04-25T14:13:35.243127+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:16.534415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.350930+0000","last_change":"2024-04-25T14:13:34.262566+0000","last_active":"2024-04-25T14:13:35.350930+0000","last_peered":"2024-04-25T14:13:35.350930+0000","last_clean":"2024-04-25T14:13:35.350930+0000","last_became_active":"2024-04-25T14:13:34.262294+0000","last_became_peered":"2024-04-25T14:13:34.262294+0000","last_unstale":"2024-04-25T14:13:35.350930+0000","last_undegraded":"2024-04-25T14:13:35.350930+0000","last_fullsized":"2024-04-25T14:13:35.350930+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:06:00.459630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.229560+0000","last_change":"2024-04-25T14:13:33.235450+0000","last_active":"2024-04-25T14:13:35.229560+0000","last_peered":"2024-04-25T14:13:35.229560+0000","last_clean":"2024-04-25T14:13:35.229560+0000","last_became_active":"2024-04-25T14:13:33.235152+0000","last_became_peered":"2024-04-25T14:13:33.235152+0000","last_unstale":"2024-04-25T14:13:35.229560+0000","last_undegraded":"2024-04-25T14:13:35.229560+0000","last_fullsized":"2024-04-25T14:13:35.229560+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:12.089780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.229075+0000","last_change":"2024-04-25T14:13:34.248795+0000","last_active":"2024-04-25T14:13:35.229075+0000","last_peered":"2024-04-25T14:13:35.229075+0000","last_clean":"2024-04-25T14:13:35.229075+0000","last_became_active":"2024-04-25T14:13:34.248460+0000","last_became_peered":"2024-04-25T14:13:34.248460+0000","last_unstale":"2024-04-25T14:13:35.229075+0000","last_undegraded":"2024-04-25T14:13:35.229075+0000","last_fullsized":"2024-04-25T14:13:35.229075+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:17.330944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"154'3","reported_seq":17,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.231639+0000","last_change":"2024-04-25T14:13:33.235678+0000","last_active":"2024-04-25T14:13:35.231639+0000","last_peered":"2024-04-25T14:13:35.231639+0000","last_clean":"2024-04-25T14:13:35.231639+0000","last_became_active":"2024-04-25T14:13:33.235038+0000","last_became_peered":"2024-04-25T14:13:33.235038+0000","last_unstale":"2024-04-25T14:13:35.231639+0000","last_undegraded":"2024-04-25T14:13:35.231639+0000","last_fullsized":"2024-04-25T14:13:35.231639+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:08.683471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.349612+0000","last_change":"2024-04-25T14:13:34.234595+0000","last_active":"2024-04-25T14:13:35.349612+0000","last_peered":"2024-04-25T14:13:35.349612+0000","last_clean":"2024-04-25T14:13:35.349612+0000","last_became_active":"2024-04-25T14:13:34.234181+0000","last_became_peered":"2024-04-25T14:13:34.234181+0000","last_unstale":"2024-04-25T14:13:35.349612+0000","last_undegraded":"2024-04-25T14:13:35.349612+0000","last_fullsized":"2024-04-25T14:13:35.349612+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:58.100963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"154'6","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.229559+0000","last_change":"2024-04-25T14:13:33.235754+0000","last_active":"2024-04-25T14:13:35.229559+0000","last_peered":"2024-04-25T14:13:35.229559+0000","last_clean":"2024-04-25T14:13:35.229559+0000","last_became_active":"2024-04-25T14:13:33.235119+0000","last_became_peered":"2024-04-25T14:13:33.235119+0000","last_unstale":"2024-04-25T14:13:35.229559+0000","last_undegraded":"2024-04-25T14:13:35.229559+0000","last_fullsized":"2024-04-25T14:13:35.229559+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:44.459505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.350386+0000","last_change":"2024-04-25T14:13:34.240628+0000","last_active":"2024-04-25T14:13:35.350386+0000","last_peered":"2024-04-25T14:13:35.350386+0000","last_clean":"2024-04-25T14:13:35.350386+0000","last_became_active":"2024-04-25T14:13:34.240287+0000","last_became_peered":"2024-04-25T14:13:34.240287+0000","last_unstale":"2024-04-25T14:13:35.350386+0000","last_undegraded":"2024-04-25T14:13:35.350386+0000","last_fullsized":"2024-04-25T14:13:35.350386+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:24.710592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.338267+0000","last_change":"2024-04-25T14:13:33.237307+0000","last_active":"2024-04-25T14:13:35.338267+0000","last_peered":"2024-04-25T14:13:35.338267+0000","last_clean":"2024-04-25T14:13:35.338267+0000","last_became_active":"2024-04-25T14:13:33.237012+0000","last_became_peered":"2024-04-25T14:13:33.237012+0000","last_unstale":"2024-04-25T14:13:35.338267+0000","last_undegraded":"2024-04-25T14:13:35.338267+0000","last_fullsized":"2024-04-25T14:13:35.338267+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:13.283102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.243457+0000","last_change":"2024-04-25T14:13:34.256173+0000","last_active":"2024-04-25T14:13:35.243457+0000","last_peered":"2024-04-25T14:13:35.243457+0000","last_clean":"2024-04-25T14:13:35.243457+0000","last_became_active":"2024-04-25T14:13:34.255852+0000","last_became_peered":"2024-04-25T14:13:34.255852+0000","last_unstale":"2024-04-25T14:13:35.243457+0000","last_undegraded":"2024-04-25T14:13:35.243457+0000","last_fullsized":"2024-04-25T14:13:35.243457+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:44.896058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.229065+0000","last_change":"2024-04-25T14:13:33.234217+0000","last_active":"2024-04-25T14:13:35.229065+0000","last_peered":"2024-04-25T14:13:35.229065+0000","last_clean":"2024-04-25T14:13:35.229065+0000","last_became_active":"2024-04-25T14:13:33.233902+0000","last_became_peered":"2024-04-25T14:13:33.233902+0000","last_unstale":"2024-04-25T14:13:35.229065+0000","last_undegraded":"2024-04-25T14:13:35.229065+0000","last_fullsized":"2024-04-25T14:13:35.229065+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:16.491560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.231738+0000","last_change":"2024-04-25T14:13:34.239880+0000","last_active":"2024-04-25T14:13:35.231738+0000","last_peered":"2024-04-25T14:13:35.231738+0000","last_clean":"2024-04-25T14:13:35.231738+0000","last_became_active":"2024-04-25T14:13:34.239513+0000","last_became_peered":"2024-04-25T14:13:34.239513+0000","last_unstale":"2024-04-25T14:13:35.231738+0000","last_undegraded":"2024-04-25T14:13:35.231738+0000","last_fullsized":"2024-04-25T14:13:35.231738+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:17.542973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.230396+0000","last_change":"2024-04-25T14:13:33.239716+0000","last_active":"2024-04-25T14:13:35.230396+0000","last_peered":"2024-04-25T14:13:35.230396+0000","last_clean":"2024-04-25T14:13:35.230396+0000","last_became_active":"2024-04-25T14:13:33.239375+0000","last_became_peered":"2024-04-25T14:13:33.239375+0000","last_unstale":"2024-04-25T14:13:35.230396+0000","last_undegraded":"2024-04-25T14:13:35.230396+0000","last_fullsized":"2024-04-25T14:13:35.230396+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:31.298870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.349995+0000","last_change":"2024-04-25T14:13:34.235892+0000","last_active":"2024-04-25T14:13:35.349995+0000","last_peered":"2024-04-25T14:13:35.349995+0000","last_clean":"2024-04-25T14:13:35.349995+0000","last_became_active":"2024-04-25T14:13:34.233611+0000","last_became_peered":"2024-04-25T14:13:34.233611+0000","last_unstale":"2024-04-25T14:13:35.349995+0000","last_undegraded":"2024-04-25T14:13:35.349995+0000","last_fullsized":"2024-04-25T14:13:35.349995+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:31.725974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"154'8","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.243136+0000","last_change":"2024-04-25T14:13:33.238315+0000","last_active":"2024-04-25T14:13:35.243136+0000","last_peered":"2024-04-25T14:13:35.243136+0000","last_clean":"2024-04-25T14:13:35.243136+0000","last_became_active":"2024-04-25T14:13:33.237955+0000","last_became_peered":"2024-04-25T14:13:33.237955+0000","last_unstale":"2024-04-25T14:13:35.243136+0000","last_undegraded":"2024-04-25T14:13:35.243136+0000","last_fullsized":"2024-04-25T14:13:35.243136+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:09.455778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.242975+0000","last_change":"2024-04-25T14:13:34.234057+0000","last_active":"2024-04-25T14:13:35.242975+0000","last_peered":"2024-04-25T14:13:35.242975+0000","last_clean":"2024-04-25T14:13:35.242975+0000","last_became_active":"2024-04-25T14:13:34.233738+0000","last_became_peered":"2024-04-25T14:13:34.233738+0000","last_unstale":"2024-04-25T14:13:35.242975+0000","last_undegraded":"2024-04-25T14:13:35.242975+0000","last_fullsized":"2024-04-25T14:13:35.242975+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:17.500119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"154'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.229765+0000","last_change":"2024-04-25T14:13:33.226581+0000","last_active":"2024-04-25T14:13:35.229765+0000","last_peered":"2024-04-25T14:13:35.229765+0000","last_clean":"2024-04-25T14:13:35.229765+0000","last_became_active":"2024-04-25T14:13:33.225988+0000","last_became_peered":"2024-04-25T14:13:33.225988+0000","last_unstale":"2024-04-25T14:13:35.229765+0000","last_undegraded":"2024-04-25T14:13:35.229765+0000","last_fullsized":"2024-04-25T14:13:35.229765+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:37.218976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.350544+0000","last_change":"2024-04-25T14:13:34.238105+0000","last_active":"2024-04-25T14:13:35.350544+0000","last_peered":"2024-04-25T14:13:35.350544+0000","last_clean":"2024-04-25T14:13:35.350544+0000","last_became_active":"2024-04-25T14:13:34.237783+0000","last_became_peered":"2024-04-25T14:13:34.237783+0000","last_unstale":"2024-04-25T14:13:35.350544+0000","last_undegraded":"2024-04-25T14:13:35.350544+0000","last_fullsized":"2024-04-25T14:13:35.350544+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:58.378005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.243593+0000","last_change":"2024-04-25T14:13:33.239707+0000","last_active":"2024-04-25T14:13:35.243593+0000","last_peered":"2024-04-25T14:13:35.243593+0000","last_clean":"2024-04-25T14:13:35.243593+0000","last_became_active":"2024-04-25T14:13:33.239408+0000","last_became_peered":"2024-04-25T14:13:33.239408+0000","last_unstale":"2024-04-25T14:13:35.243593+0000","last_undegraded":"2024-04-25T14:13:35.243593+0000","last_fullsized":"2024-04-25T14:13:35.243593+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:08.407184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.230293+0000","last_change":"2024-04-25T14:13:34.227025+0000","last_active":"2024-04-25T14:13:35.230293+0000","last_peered":"2024-04-25T14:13:35.230293+0000","last_clean":"2024-04-25T14:13:35.230293+0000","last_became_active":"2024-04-25T14:13:34.226730+0000","last_became_peered":"2024-04-25T14:13:34.226730+0000","last_unstale":"2024-04-25T14:13:35.230293+0000","last_undegraded":"2024-04-25T14:13:35.230293+0000","last_fullsized":"2024-04-25T14:13:35.230293+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:28.346277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"154'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.338045+0000","last_change":"2024-04-25T14:13:33.232965+0000","last_active":"2024-04-25T14:13:35.338045+0000","last_peered":"2024-04-25T14:13:35.338045+0000","last_clean":"2024-04-25T14:13:35.338045+0000","last_became_active":"2024-04-25T14:13:33.232647+0000","last_became_peered":"2024-04-25T14:13:33.232647+0000","last_unstale":"2024-04-25T14:13:35.338045+0000","last_undegraded":"2024-04-25T14:13:35.338045+0000","last_fullsized":"2024-04-25T14:13:35.338045+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:44.459505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.230486+0000","last_change":"2024-04-25T14:13:34.230968+0000","last_active":"2024-04-25T14:13:35.230486+0000","last_peered":"2024-04-25T14:13:35.230486+0000","last_clean":"2024-04-25T14:13:35.230486+0000","last_became_active":"2024-04-25T14:13:34.230639+0000","last_became_peered":"2024-04-25T14:13:34.230639+0000","last_unstale":"2024-04-25T14:13:35.230486+0000","last_undegraded":"2024-04-25T14:13:35.230486+0000","last_fullsized":"2024-04-25T14:13:35.230486+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:09.415743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.243360+0000","last_change":"2024-04-25T14:13:33.240376+0000","last_active":"2024-04-25T14:13:35.243360+0000","last_peered":"2024-04-25T14:13:35.243360+0000","last_clean":"2024-04-25T14:13:35.243360+0000","last_became_active":"2024-04-25T14:13:33.240103+0000","last_became_peered":"2024-04-25T14:13:33.240103+0000","last_unstale":"2024-04-25T14:13:35.243360+0000","last_undegraded":"2024-04-25T14:13:35.243360+0000","last_fullsized":"2024-04-25T14:13:35.243360+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:34.938781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.229429+0000","last_change":"2024-04-25T14:13:34.244351+0000","last_active":"2024-04-25T14:13:35.229429+0000","last_peered":"2024-04-25T14:13:35.229429+0000","last_clean":"2024-04-25T14:13:35.229429+0000","last_became_active":"2024-04-25T14:13:34.242206+0000","last_became_peered":"2024-04-25T14:13:34.242206+0000","last_unstale":"2024-04-25T14:13:35.229429+0000","last_undegraded":"2024-04-25T14:13:35.229429+0000","last_fullsized":"2024-04-25T14:13:35.229429+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:10.743120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.338384+0000","last_change":"2024-04-25T14:13:33.231549+0000","last_active":"2024-04-25T14:13:35.338384+0000","last_peered":"2024-04-25T14:13:35.338384+0000","last_clean":"2024-04-25T14:13:35.338384+0000","last_became_active":"2024-04-25T14:13:33.231239+0000","last_became_peered":"2024-04-25T14:13:33.231239+0000","last_unstale":"2024-04-25T14:13:35.338384+0000","last_undegraded":"2024-04-25T14:13:35.338384+0000","last_fullsized":"2024-04-25T14:13:35.338384+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:43.887500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.338076+0000","last_change":"2024-04-25T14:13:34.253741+0000","last_active":"2024-04-25T14:13:35.338076+0000","last_peered":"2024-04-25T14:13:35.338076+0000","last_clean":"2024-04-25T14:13:35.338076+0000","last_became_active":"2024-04-25T14:13:34.253475+0000","last_became_peered":"2024-04-25T14:13:34.253475+0000","last_unstale":"2024-04-25T14:13:35.338076+0000","last_undegraded":"2024-04-25T14:13:35.338076+0000","last_fullsized":"2024-04-25T14:13:35.338076+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:10.743120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.230551+0000","last_change":"2024-04-25T14:13:33.238316+0000","last_active":"2024-04-25T14:13:35.230551+0000","last_peered":"2024-04-25T14:13:35.230551+0000","last_clean":"2024-04-25T14:13:35.230551+0000","last_became_active":"2024-04-25T14:13:33.237984+0000","last_became_peered":"2024-04-25T14:13:33.237984+0000","last_unstale":"2024-04-25T14:13:35.230551+0000","last_undegraded":"2024-04-25T14:13:35.230551+0000","last_fullsized":"2024-04-25T14:13:35.230551+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:23.702033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.338083+0000","last_change":"2024-04-25T14:13:34.241560+0000","last_active":"2024-04-25T14:13:35.338083+0000","last_peered":"2024-04-25T14:13:35.338083+0000","last_clean":"2024-04-25T14:13:35.338083+0000","last_became_active":"2024-04-25T14:13:34.241259+0000","last_became_peered":"2024-04-25T14:13:34.241259+0000","last_unstale":"2024-04-25T14:13:35.338083+0000","last_undegraded":"2024-04-25T14:13:35.338083+0000","last_fullsized":"2024-04-25T14:13:35.338083+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:42.807728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.349588+0000","last_change":"2024-04-25T14:13:33.233345+0000","last_active":"2024-04-25T14:13:35.349588+0000","last_peered":"2024-04-25T14:13:35.349588+0000","last_clean":"2024-04-25T14:13:35.349588+0000","last_became_active":"2024-04-25T14:13:33.233045+0000","last_became_peered":"2024-04-25T14:13:33.233045+0000","last_unstale":"2024-04-25T14:13:35.349588+0000","last_undegraded":"2024-04-25T14:13:35.349588+0000","last_fullsized":"2024-04-25T14:13:35.349588+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:07.300659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.351561+0000","last_change":"2024-04-25T14:13:34.244624+0000","last_active":"2024-04-25T14:13:35.351561+0000","last_peered":"2024-04-25T14:13:35.351561+0000","last_clean":"2024-04-25T14:13:35.351561+0000","last_became_active":"2024-04-25T14:13:34.243797+0000","last_became_peered":"2024-04-25T14:13:34.243797+0000","last_unstale":"2024-04-25T14:13:35.351561+0000","last_undegraded":"2024-04-25T14:13:35.351561+0000","last_fullsized":"2024-04-25T14:13:35.351561+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:17.500119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.351283+0000","last_change":"2024-04-25T14:13:33.233786+0000","last_active":"2024-04-25T14:13:35.351283+0000","last_peered":"2024-04-25T14:13:35.351283+0000","last_clean":"2024-04-25T14:13:35.351283+0000","last_became_active":"2024-04-25T14:13:33.233479+0000","last_became_peered":"2024-04-25T14:13:33.233479+0000","last_unstale":"2024-04-25T14:13:35.351283+0000","last_undegraded":"2024-04-25T14:13:35.351283+0000","last_fullsized":"2024-04-25T14:13:35.351283+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:03.977953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.230213+0000","last_change":"2024-04-25T14:13:34.235218+0000","last_active":"2024-04-25T14:13:35.230213+0000","last_peered":"2024-04-25T14:13:35.230213+0000","last_clean":"2024-04-25T14:13:35.230213+0000","last_became_active":"2024-04-25T14:13:34.234898+0000","last_became_peered":"2024-04-25T14:13:34.234898+0000","last_unstale":"2024-04-25T14:13:35.230213+0000","last_undegraded":"2024-04-25T14:13:35.230213+0000","last_fullsized":"2024-04-25T14:13:35.230213+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:47:32.620163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.338020+0000","last_change":"2024-04-25T14:13:33.235217+0000","last_active":"2024-04-25T14:13:35.338020+0000","last_peered":"2024-04-25T14:13:35.338020+0000","last_clean":"2024-04-25T14:13:35.338020+0000","last_became_active":"2024-04-25T14:13:33.234923+0000","last_became_peered":"2024-04-25T14:13:33.234923+0000","last_unstale":"2024-04-25T14:13:35.338020+0000","last_undegraded":"2024-04-25T14:13:35.338020+0000","last_fullsized":"2024-04-25T14:13:35.338020+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:36.440139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.243079+0000","last_change":"2024-04-25T14:13:34.236039+0000","last_active":"2024-04-25T14:13:35.243079+0000","last_peered":"2024-04-25T14:13:35.243079+0000","last_clean":"2024-04-25T14:13:35.243079+0000","last_became_active":"2024-04-25T14:13:34.235638+0000","last_became_peered":"2024-04-25T14:13:34.235638+0000","last_unstale":"2024-04-25T14:13:35.243079+0000","last_undegraded":"2024-04-25T14:13:35.243079+0000","last_fullsized":"2024-04-25T14:13:35.243079+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:19.722740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"155'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.229503+0000","last_change":"2024-04-25T14:13:33.233600+0000","last_active":"2024-04-25T14:13:35.229503+0000","last_peered":"2024-04-25T14:13:35.229503+0000","last_clean":"2024-04-25T14:13:35.229503+0000","last_became_active":"2024-04-25T14:13:33.233290+0000","last_became_peered":"2024-04-25T14:13:33.233290+0000","last_unstale":"2024-04-25T14:13:35.229503+0000","last_undegraded":"2024-04-25T14:13:35.229503+0000","last_fullsized":"2024-04-25T14:13:35.229503+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:18.714182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.349662+0000","last_change":"2024-04-25T14:13:34.235768+0000","last_active":"2024-04-25T14:13:35.349662+0000","last_peered":"2024-04-25T14:13:35.349662+0000","last_clean":"2024-04-25T14:13:35.349662+0000","last_became_active":"2024-04-25T14:13:34.235479+0000","last_became_peered":"2024-04-25T14:13:34.235479+0000","last_unstale":"2024-04-25T14:13:35.349662+0000","last_undegraded":"2024-04-25T14:13:35.349662+0000","last_fullsized":"2024-04-25T14:13:35.349662+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:37.426951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.229731+0000","last_change":"2024-04-25T14:13:33.229751+0000","last_active":"2024-04-25T14:13:35.229731+0000","last_peered":"2024-04-25T14:13:35.229731+0000","last_clean":"2024-04-25T14:13:35.229731+0000","last_became_active":"2024-04-25T14:13:33.229451+0000","last_became_peered":"2024-04-25T14:13:33.229451+0000","last_unstale":"2024-04-25T14:13:35.229731+0000","last_undegraded":"2024-04-25T14:13:35.229731+0000","last_fullsized":"2024-04-25T14:13:35.229731+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:46.823734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.243105+0000","last_change":"2024-04-25T14:13:34.245089+0000","last_active":"2024-04-25T14:13:35.243105+0000","last_peered":"2024-04-25T14:13:35.243105+0000","last_clean":"2024-04-25T14:13:35.243105+0000","last_became_active":"2024-04-25T14:13:34.244763+0000","last_became_peered":"2024-04-25T14:13:34.244763+0000","last_unstale":"2024-04-25T14:13:35.243105+0000","last_undegraded":"2024-04-25T14:13:35.243105+0000","last_fullsized":"2024-04-25T14:13:35.243105+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:45.468064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.349746+0000","last_change":"2024-04-25T14:13:33.245950+0000","last_active":"2024-04-25T14:13:35.349746+0000","last_peered":"2024-04-25T14:13:35.349746+0000","last_clean":"2024-04-25T14:13:35.349746+0000","last_became_active":"2024-04-25T14:13:33.245627+0000","last_became_peered":"2024-04-25T14:13:33.245627+0000","last_unstale":"2024-04-25T14:13:35.349746+0000","last_undegraded":"2024-04-25T14:13:35.349746+0000","last_fullsized":"2024-04-25T14:13:35.349746+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:03.424002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.231414+0000","last_change":"2024-04-25T14:13:34.225946+0000","last_active":"2024-04-25T14:13:35.231414+0000","last_peered":"2024-04-25T14:13:35.231414+0000","last_clean":"2024-04-25T14:13:35.231414+0000","last_became_active":"2024-04-25T14:13:34.225458+0000","last_became_peered":"2024-04-25T14:13:34.225458+0000","last_unstale":"2024-04-25T14:13:35.231414+0000","last_undegraded":"2024-04-25T14:13:35.231414+0000","last_fullsized":"2024-04-25T14:13:35.231414+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:51.327163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.231101+0000","last_change":"2024-04-25T14:13:33.241343+0000","last_active":"2024-04-25T14:13:35.231101+0000","last_peered":"2024-04-25T14:13:35.231101+0000","last_clean":"2024-04-25T14:13:35.231101+0000","last_became_active":"2024-04-25T14:13:33.240975+0000","last_became_peered":"2024-04-25T14:13:33.240975+0000","last_unstale":"2024-04-25T14:13:35.231101+0000","last_undegraded":"2024-04-25T14:13:35.231101+0000","last_fullsized":"2024-04-25T14:13:35.231101+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:35.047986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.351436+0000","last_change":"2024-04-25T14:13:34.263451+0000","last_active":"2024-04-25T14:13:35.351436+0000","last_peered":"2024-04-25T14:13:35.351436+0000","last_clean":"2024-04-25T14:13:35.351436+0000","last_became_active":"2024-04-25T14:13:34.263094+0000","last_became_peered":"2024-04-25T14:13:34.263094+0000","last_unstale":"2024-04-25T14:13:35.351436+0000","last_undegraded":"2024-04-25T14:13:35.351436+0000","last_fullsized":"2024-04-25T14:13:35.351436+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:01.740152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.229187+0000","last_change":"2024-04-25T14:13:33.230789+0000","last_active":"2024-04-25T14:13:35.229187+0000","last_peered":"2024-04-25T14:13:35.229187+0000","last_clean":"2024-04-25T14:13:35.229187+0000","last_became_active":"2024-04-25T14:13:33.230422+0000","last_became_peered":"2024-04-25T14:13:33.230422+0000","last_unstale":"2024-04-25T14:13:35.229187+0000","last_undegraded":"2024-04-25T14:13:35.229187+0000","last_fullsized":"2024-04-25T14:13:35.229187+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:44.934234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.229175+0000","last_change":"2024-04-25T14:13:34.243715+0000","last_active":"2024-04-25T14:13:35.229175+0000","last_peered":"2024-04-25T14:13:35.229175+0000","last_clean":"2024-04-25T14:13:35.229175+0000","last_became_active":"2024-04-25T14:13:34.243366+0000","last_became_peered":"2024-04-25T14:13:34.243366+0000","last_unstale":"2024-04-25T14:13:35.229175+0000","last_undegraded":"2024-04-25T14:13:35.229175+0000","last_fullsized":"2024-04-25T14:13:35.229175+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:30.285001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.230551+0000","last_change":"2024-04-25T14:13:33.241318+0000","last_active":"2024-04-25T14:13:35.230551+0000","last_peered":"2024-04-25T14:13:35.230551+0000","last_clean":"2024-04-25T14:13:35.230551+0000","last_became_active":"2024-04-25T14:13:33.240931+0000","last_became_peered":"2024-04-25T14:13:33.240931+0000","last_unstale":"2024-04-25T14:13:35.230551+0000","last_undegraded":"2024-04-25T14:13:35.230551+0000","last_fullsized":"2024-04-25T14:13:35.230551+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:03.977953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.350052+0000","last_change":"2024-04-25T14:13:34.251825+0000","last_active":"2024-04-25T14:13:35.350052+0000","last_peered":"2024-04-25T14:13:35.350052+0000","last_clean":"2024-04-25T14:13:35.350052+0000","last_became_active":"2024-04-25T14:13:34.251529+0000","last_became_peered":"2024-04-25T14:13:34.251529+0000","last_unstale":"2024-04-25T14:13:35.350052+0000","last_undegraded":"2024-04-25T14:13:35.350052+0000","last_fullsized":"2024-04-25T14:13:35.350052+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:42.830126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.337617+0000","last_change":"2024-04-25T14:13:33.229866+0000","last_active":"2024-04-25T14:13:35.337617+0000","last_peered":"2024-04-25T14:13:35.337617+0000","last_clean":"2024-04-25T14:13:35.337617+0000","last_became_active":"2024-04-25T14:13:33.229562+0000","last_became_peered":"2024-04-25T14:13:33.229562+0000","last_unstale":"2024-04-25T14:13:35.337617+0000","last_undegraded":"2024-04-25T14:13:35.337617+0000","last_fullsized":"2024-04-25T14:13:35.337617+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:18.714182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.350912+0000","last_change":"2024-04-25T14:13:34.244696+0000","last_active":"2024-04-25T14:13:35.350912+0000","last_peered":"2024-04-25T14:13:35.350912+0000","last_clean":"2024-04-25T14:13:35.350912+0000","last_became_active":"2024-04-25T14:13:34.244386+0000","last_became_peered":"2024-04-25T14:13:34.244386+0000","last_unstale":"2024-04-25T14:13:35.350912+0000","last_undegraded":"2024-04-25T14:13:35.350912+0000","last_fullsized":"2024-04-25T14:13:35.350912+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:33.204667+0000","last_clean_scrub_stamp":"2024-04-25T14:13:33.204667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:35.947339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.229317+0000","last_change":"2024-04-25T14:13:33.242678+0000","last_active":"2024-04-25T14:13:35.229317+0000","last_peered":"2024-04-25T14:13:35.229317+0000","last_clean":"2024-04-25T14:13:35.229317+0000","last_became_active":"2024-04-25T14:13:33.241884+0000","last_became_peered":"2024-04-25T14:13:33.241884+0000","last_unstale":"2024-04-25T14:13:35.229317+0000","last_undegraded":"2024-04-25T14:13:35.229317+0000","last_fullsized":"2024-04-25T14:13:35.229317+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:13:32.196108+0000","last_clean_scrub_stamp":"2024-04-25T14:13:32.196108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:45.356743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"144'3291","reported_seq":3832,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T14:13:35.351190+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:13:35.351190+0000","last_peered":"2024-04-25T14:13:35.351190+0000","last_clean":"2024-04-25T14:13:35.351190+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:13:35.351190+0000","last_undegraded":"2024-04-25T14:13:35.351190+0000","last_fullsized":"2024-04-25T14:13:35.351190+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":3291,"log_dups_size":0,"ondisk_log_size":3291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":837,"num_read_kb":2279,"num_write":247,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":837,"num_read_kb":2279,"num_write":247,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3291,"ondisk_log_size":3291,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574964,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29504,"kb_used_data":2484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342336,"statfs":{"total":96636764160,"available":96606552064,"internally_reserved":0,"allocated":2543616,"data_stored":1570041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574965,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33084,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338756,"statfs":{"total":96636764160,"available":96602886144,"internally_reserved":0,"allocated":1949696,"data_stored":979673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574966,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1957888,"data_stored":979797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574965,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33100,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338740,"statfs":{"total":96636764160,"available":96602869760,"internally_reserved":0,"allocated":1966080,"data_stored":980437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574965,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33104,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338736,"statfs":{"total":96636764160,"available":96602865664,"internally_reserved":0,"allocated":1970176,"data_stored":981889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574964,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33084,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338756,"statfs":{"total":96636764160,"available":96602886144,"internally_reserved":0,"allocated":1949696,"data_stored":979673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574966,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33096,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338744,"statfs":{"total":96636764160,"available":96602873856,"internally_reserved":0,"allocated":1961984,"data_stored":981285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574965,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29516,"kb_used_data":2492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342324,"statfs":{"total":96636764160,"available":96606539776,"internally_reserved":0,"allocated":2551808,"data_stored":1570145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:13:43.301 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T14:13:43.301 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T14:13:43.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.486+0000 7f2ac9800640 1 Processor -- start 2024-04-25T14:13:43.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.486+0000 7f2ac9800640 1 -- start start 2024-04-25T14:13:43.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.490+0000 7f2ac9800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2ac40740c0 0x7f2ac41676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:43.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.490+0000 7f2ac9800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ac4072bd0 con 0x7f2ac40cfca0 2024-04-25T14:13:43.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.490+0000 7f2ac9800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ac406fbd0 con 0x7f2ac40d16e0 2024-04-25T14:13:43.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.490+0000 7f2ac9800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2ac4070470 con 0x7f2ac40740c0 2024-04-25T14:13:43.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.490+0000 7f2ac2a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2ac40cfca0 0x7f2ac406f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45226/0 (socket says 172.21.15.86:45226) 2024-04-25T14:13:43.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.490+0000 7f2ac2a00640 1 -- 172.21.15.86:0/2136668946 learned_addr learned my addr 172.21.15.86:0/2136668946 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:43.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.490+0000 7f2ac3400640 1 --2- 172.21.15.86:0/2136668946 >> v2:172.21.15.86:3300/0 conn(0x7f2ac40740c0 0x7f2ac41676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:43.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.490+0000 7f2ac1600640 1 -- 172.21.15.86:0/2136668946 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3732437608 0 0) 0x7f2ac4072bd0 con 0x7f2ac40cfca0 2024-04-25T14:13:43.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.490+0000 7f2ac1600640 1 -- 172.21.15.86:0/2136668946 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2aa8003530 con 0x7f2ac40cfca0 2024-04-25T14:13:43.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.490+0000 7f2ac3400640 1 -- 172.21.15.86:0/2136668946 >> v1:172.21.15.155:6789/0 conn(0x7f2ac40d16e0 legacy=0x7f2ac40d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:43.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.490+0000 7f2ac3400640 1 -- 172.21.15.86:0/2136668946 >> v1:172.21.15.86:6789/0 conn(0x7f2ac40cfca0 legacy=0x7f2ac406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:43.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.490+0000 7f2ac3400640 1 -- 172.21.15.86:0/2136668946 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac406feb0 con 0x7f2ac40740c0 2024-04-25T14:13:43.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.494+0000 7f2ac3400640 1 --2- 172.21.15.86:0/2136668946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac40740c0 0x7f2ac41676d0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0097c0 tx=0x7f2abc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0e2759783cfd89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:43.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.494+0000 7f2ac1600640 1 -- 172.21.15.86:0/2136668946 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc0b1070 con 0x7f2ac40740c0 2024-04-25T14:13:43.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.494+0000 7f2ac1600640 1 -- 172.21.15.86:0/2136668946 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2abc0ac410 con 0x7f2ac40740c0 2024-04-25T14:13:43.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.494+0000 7f2ac1600640 1 -- 172.21.15.86:0/2136668946 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc0ac7e0 con 0x7f2ac40740c0 2024-04-25T14:13:43.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.494+0000 7f2ac9800640 1 -- 172.21.15.86:0/2136668946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac40740c0 msgr2=0x7f2ac41676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:43.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.494+0000 7f2ac9800640 1 --2- 172.21.15.86:0/2136668946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac40740c0 0x7f2ac41676d0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0097c0 tx=0x7f2abc01c4a0 comp rx=0 tx=0).stop 2024-04-25T14:13:43.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.494+0000 7f2ac9800640 1 -- 172.21.15.86:0/2136668946 shutdown_connections 2024-04-25T14:13:43.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.494+0000 7f2ac9800640 1 --2- 172.21.15.86:0/2136668946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac40740c0 0x7f2ac41676d0 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:43.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.494+0000 7f2ac9800640 1 -- 172.21.15.86:0/2136668946 >> 172.21.15.86:0/2136668946 conn(0x7f2ac40b4650 msgr2=0x7f2ac40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:43.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.494+0000 7f2ac9800640 1 -- 172.21.15.86:0/2136668946 shutdown_connections 2024-04-25T14:13:43.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.494+0000 7f2ac9800640 1 -- 172.21.15.86:0/2136668946 wait complete. 2024-04-25T14:13:43.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.494+0000 7f2ac9800640 1 Processor -- start 2024-04-25T14:13:43.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.498+0000 7f2ac9800640 1 -- start start 2024-04-25T14:13:43.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.498+0000 7f2ac9800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ac40740c0 0x7f2ac415dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:43.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.498+0000 7f2ac9800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ac40cfca0 0x7f2ac4099f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:43.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.498+0000 7f2ac9800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac40d16e0 0x7f2ac415ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:43.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.498+0000 7f2ac9800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac416ade0 con 0x7f2ac40d16e0 2024-04-25T14:13:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.498+0000 7f2ac9800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac416ac60 con 0x7f2ac40cfca0 2024-04-25T14:13:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.498+0000 7f2ac9800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2ac416af60 con 0x7f2ac40740c0 2024-04-25T14:13:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.498+0000 7f2ac3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac40d16e0 0x7f2ac415ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.498+0000 7f2ac3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac40d16e0 0x7f2ac415ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41662/0 (socket says 172.21.15.86:41662) 2024-04-25T14:13:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.498+0000 7f2ac3400640 1 -- 172.21.15.86:0/1740318241 learned_addr learned my addr 172.21.15.86:0/1740318241 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.498+0000 7f2ac2a00640 1 --2- 172.21.15.86:0/1740318241 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ac40740c0 0x7f2ac415dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.498+0000 7f2ac2000640 1 --2- 172.21.15.86:0/1740318241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ac40cfca0 0x7f2ac4099f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.498+0000 7f2ac3400640 1 -- 172.21.15.86:0/1740318241 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ac40740c0 msgr2=0x7f2ac415dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.498+0000 7f2ac3400640 1 --2- 172.21.15.86:0/1740318241 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ac40740c0 0x7f2ac415dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.498+0000 7f2ac3400640 1 -- 172.21.15.86:0/1740318241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ac40cfca0 msgr2=0x7f2ac4099f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.498+0000 7f2ac3400640 1 --2- 172.21.15.86:0/1740318241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ac40cfca0 0x7f2ac4099f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.498+0000 7f2ac3400640 1 -- 172.21.15.86:0/1740318241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac409ab50 con 0x7f2ac40d16e0 2024-04-25T14:13:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.498+0000 7f2ac3400640 1 --2- 172.21.15.86:0/1740318241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac40d16e0 0x7f2ac415ea70 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0af040 tx=0x7f2abc019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:43.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.498+0000 7f2abaa00640 1 -- 172.21.15.86:0/1740318241 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc0b1040 con 0x7f2ac40d16e0 2024-04-25T14:13:43.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.498+0000 7f2abaa00640 1 -- 172.21.15.86:0/1740318241 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2abc0bb070 con 0x7f2ac40d16e0 2024-04-25T14:13:43.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.498+0000 7f2abaa00640 1 -- 172.21.15.86:0/1740318241 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc0b5410 con 0x7f2ac40d16e0 2024-04-25T14:13:43.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.498+0000 7f2ac9800640 1 -- 172.21.15.86:0/1740318241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac415f650 con 0x7f2ac40d16e0 2024-04-25T14:13:43.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.498+0000 7f2ac9800640 1 -- 172.21.15.86:0/1740318241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac415d550 con 0x7f2ac40d16e0 2024-04-25T14:13:43.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.498+0000 7f2ac9800640 1 -- 172.21.15.86:0/1740318241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ac409f4b0 con 0x7f2ac40d16e0 2024-04-25T14:13:43.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.502+0000 7f2abaa00640 1 -- 172.21.15.86:0/1740318241 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7f2abc0b55b0 con 0x7f2ac40d16e0 2024-04-25T14:13:43.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.502+0000 7f2abaa00640 1 --2- 172.21.15.86:0/1740318241 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f2aa406f520 0x7f2aa4071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:43.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.502+0000 7f2abaa00640 1 -- 172.21.15.86:0/1740318241 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6101+0+0 (secure 0 0 0) 0x7f2abc129070 con 0x7f2ac40d16e0 2024-04-25T14:13:43.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.502+0000 7f2ac2a00640 1 --2- 172.21.15.86:0/1740318241 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f2aa406f520 0x7f2aa4071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:43.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.502+0000 7f2ac2a00640 1 --2- 172.21.15.86:0/1740318241 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f2aa406f520 0x7f2aa4071a30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4005c10 tx=0x7f2ab4009040 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:43.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.506+0000 7f2abaa00640 1 -- 172.21.15.86:0/1740318241 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2ac409ab50 con 0x7f2ac40d16e0 2024-04-25T14:13:43.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.718+0000 7f2ac9800640 1 -- 172.21.15.86:0/1740318241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f2ac415f3e0 con 0x7f2ac40d16e0 2024-04-25T14:13:43.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.722+0000 7f2abaa00640 1 -- 172.21.15.86:0/1740318241 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v31) ==== 190+0+147 (secure 0 0 0) 0x7f2abc016090 con 0x7f2ac40d16e0 2024-04-25T14:13:43.721 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-04-25T14:13:43.721 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCLYipm5u4xHRAACDFe3Eb4KLrsdLj4eFe7mA== 2024-04-25T14:13:43.721 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-04-25T14:13:43.721 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-04-25T14:13:43.721 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T14:13:43.721 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.0 2024-04-25T14:13:43.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.722+0000 7f2ac9800640 1 -- 172.21.15.86:0/1740318241 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f2aa406f520 msgr2=0x7f2aa4071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:43.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.722+0000 7f2ac9800640 1 --2- 172.21.15.86:0/1740318241 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f2aa406f520 0x7f2aa4071a30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4005c10 tx=0x7f2ab4009040 comp rx=0 tx=0).stop 2024-04-25T14:13:43.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.722+0000 7f2ac9800640 1 -- 172.21.15.86:0/1740318241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac40d16e0 msgr2=0x7f2ac415ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:43.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.722+0000 7f2ac9800640 1 --2- 172.21.15.86:0/1740318241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac40d16e0 0x7f2ac415ea70 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0af040 tx=0x7f2abc019240 comp rx=0 tx=0).stop 2024-04-25T14:13:43.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.722+0000 7f2ac9800640 1 -- 172.21.15.86:0/1740318241 shutdown_connections 2024-04-25T14:13:43.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.722+0000 7f2ac9800640 1 --2- 172.21.15.86:0/1740318241 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f2aa406f520 0x7f2aa4071a30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:43.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.722+0000 7f2ac9800640 1 --2- 172.21.15.86:0/1740318241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ac40d16e0 0x7f2ac415ea70 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:43.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.722+0000 7f2ac9800640 1 --2- 172.21.15.86:0/1740318241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ac40cfca0 0x7f2ac4099f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:43.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.722+0000 7f2ac9800640 1 --2- 172.21.15.86:0/1740318241 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ac40740c0 0x7f2ac415dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:43.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.722+0000 7f2ac9800640 1 -- 172.21.15.86:0/1740318241 >> 172.21.15.86:0/1740318241 conn(0x7f2ac40b4650 msgr2=0x7f2ac40b1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:43.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.722+0000 7f2ac9800640 1 -- 172.21.15.86:0/1740318241 shutdown_connections 2024-04-25T14:13:43.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.722+0000 7f2ac9800640 1 -- 172.21.15.86:0/1740318241 wait complete. 2024-04-25T14:13:43.743 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:13:43.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.890+0000 7fc711000640 1 Processor -- start 2024-04-25T14:13:43.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.890+0000 7fc711000640 1 -- start start 2024-04-25T14:13:43.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.890+0000 7fc711000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc70c0d05e0 0x7fc70c0b6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:43.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.890+0000 7fc711000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc70c0d2420 con 0x7fc70c0ce530 2024-04-25T14:13:43.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.890+0000 7fc711000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc70c0d2700 con 0x7fc70c0b6dc0 2024-04-25T14:13:43.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.890+0000 7fc711000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc70c0d29e0 con 0x7fc70c0d05e0 2024-04-25T14:13:43.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.890+0000 7fc709e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc70c0ce530 0x7fc70c0cfed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45232/0 (socket says 172.21.15.86:45232) 2024-04-25T14:13:43.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.890+0000 7fc709e00640 1 -- 172.21.15.86:0/2460403082 learned_addr learned my addr 172.21.15.86:0/2460403082 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:43.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.890+0000 7fc709400640 1 --2- 172.21.15.86:0/2460403082 >> v2:172.21.15.155:3300/0 conn(0x7fc70c0d05e0 0x7fc70c0b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:43.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.894+0000 7fc708a00640 1 -- 172.21.15.86:0/2460403082 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3725409139 0 0) 0x7fc70c0d2420 con 0x7fc70c0ce530 2024-04-25T14:13:43.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.894+0000 7fc708a00640 1 -- 172.21.15.86:0/2460403082 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc6f0003530 con 0x7fc70c0ce530 2024-04-25T14:13:43.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.894+0000 7fc708a00640 1 -- 172.21.15.86:0/2460403082 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1600784013 0 0) 0x7fc70c0d2700 con 0x7fc70c0b6dc0 2024-04-25T14:13:43.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.894+0000 7fc708a00640 1 -- 172.21.15.86:0/2460403082 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc70c0d2420 con 0x7fc70c0b6dc0 2024-04-25T14:13:43.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.894+0000 7fc708a00640 1 -- 172.21.15.86:0/2460403082 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1687645837 0 0) 0x7fc6f0003530 con 0x7fc70c0ce530 2024-04-25T14:13:43.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.894+0000 7fc708a00640 1 -- 172.21.15.86:0/2460403082 >> v2:172.21.15.155:3300/0 conn(0x7fc70c0d05e0 msgr2=0x7fc70c0b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:43.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.894+0000 7fc708a00640 1 --2- 172.21.15.86:0/2460403082 >> v2:172.21.15.155:3300/0 conn(0x7fc70c0d05e0 0x7fc70c0b6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:43.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.894+0000 7fc708a00640 1 -- 172.21.15.86:0/2460403082 >> v1:172.21.15.155:6789/0 conn(0x7fc70c0b6dc0 legacy=0x7fc70c0b71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:43.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.894+0000 7fc708a00640 1 -- 172.21.15.86:0/2460403082 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70c0d2cc0 con 0x7fc70c0ce530 2024-04-25T14:13:43.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.894+0000 7fc708a00640 1 -- 172.21.15.86:0/2460403082 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fc6fc001a00 con 0x7fc70c0ce530 2024-04-25T14:13:43.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.894+0000 7fc709400640 1 --2- 172.21.15.86:0/2460403082 >> v2:172.21.15.155:3300/0 conn(0x7fc70c0d05e0 0x7fc70c0b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:13:43.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.894+0000 7fc708a00640 1 -- 172.21.15.86:0/2460403082 >> v1:172.21.15.86:6789/0 conn(0x7fc70c0ce530 legacy=0x7fc70c0cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:43.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.894+0000 7fc708a00640 1 --2- 172.21.15.86:0/2460403082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6f0003bc0 0x7fc6f0004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:43.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.894+0000 7fc708a00640 1 --2- 172.21.15.86:0/2460403082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6f0004810 0x7fc6f0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:43.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.894+0000 7fc708a00640 1 --2- 172.21.15.86:0/2460403082 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6f0005220 0x7fc6f000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:43.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.894+0000 7fc708a00640 1 -- 172.21.15.86:0/2460403082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c0d29e0 con 0x7fc6f0003bc0 2024-04-25T14:13:43.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.894+0000 7fc708a00640 1 -- 172.21.15.86:0/2460403082 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f000c120 con 0x7fc6f0004810 2024-04-25T14:13:43.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.894+0000 7fc708a00640 1 -- 172.21.15.86:0/2460403082 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc6f000c320 con 0x7fc6f0005220 2024-04-25T14:13:43.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.894+0000 7fc709e00640 1 --2- 172.21.15.86:0/2460403082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6f0003bc0 0x7fc6f0004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:43.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.894+0000 7fc709400640 1 --2- 172.21.15.86:0/2460403082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6f0004810 0x7fc6f0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:43.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.894+0000 7fc70a800640 1 --2- 172.21.15.86:0/2460403082 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6f0005220 0x7fc6f000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:43.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.894+0000 7fc709e00640 1 -- 172.21.15.86:0/2460403082 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6f0005220 msgr2=0x7fc6f000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:43.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.894+0000 7fc709e00640 1 --2- 172.21.15.86:0/2460403082 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6f0005220 0x7fc6f000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:43.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.894+0000 7fc709e00640 1 -- 172.21.15.86:0/2460403082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6f0004810 msgr2=0x7fc6f0004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:43.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.894+0000 7fc709e00640 1 --2- 172.21.15.86:0/2460403082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6f0004810 0x7fc6f0004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:43.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.894+0000 7fc709e00640 1 -- 172.21.15.86:0/2460403082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f0003530 con 0x7fc6f0003bc0 2024-04-25T14:13:43.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.894+0000 7fc70a800640 1 --2- 172.21.15.86:0/2460403082 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6f0005220 0x7fc6f000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:43.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.894+0000 7fc70a800640 1 -- 172.21.15.86:0/2460403082 reap_dead start 2024-04-25T14:13:43.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.898+0000 7fc709e00640 1 --2- 172.21.15.86:0/2460403082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6f0003bc0 0x7fc6f0004010 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00a810 tx=0x7fc6fc01d320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ef17ef665ede0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:43.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.898+0000 7fc708a00640 1 -- 172.21.15.86:0/2460403082 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc00a430 con 0x7fc6f0003bc0 2024-04-25T14:13:43.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.898+0000 7fc708a00640 1 -- 172.21.15.86:0/2460403082 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6fc0ad650 con 0x7fc6f0003bc0 2024-04-25T14:13:43.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.898+0000 7fc708a00640 1 -- 172.21.15.86:0/2460403082 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc0ada80 con 0x7fc6f0003bc0 2024-04-25T14:13:43.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.898+0000 7fc711000640 1 -- 172.21.15.86:0/2460403082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6f0003bc0 msgr2=0x7fc6f0004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:43.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.898+0000 7fc711000640 1 --2- 172.21.15.86:0/2460403082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6f0003bc0 0x7fc6f0004010 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00a810 tx=0x7fc6fc01d320 comp rx=0 tx=0).stop 2024-04-25T14:13:43.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.898+0000 7fc711000640 1 -- 172.21.15.86:0/2460403082 shutdown_connections 2024-04-25T14:13:43.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.898+0000 7fc711000640 1 --2- 172.21.15.86:0/2460403082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6f0003bc0 0x7fc6f0004010 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:43.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.898+0000 7fc711000640 1 -- 172.21.15.86:0/2460403082 >> 172.21.15.86:0/2460403082 conn(0x7fc70c0b5020 msgr2=0x7fc70c0b5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:43.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.898+0000 7fc711000640 1 -- 172.21.15.86:0/2460403082 shutdown_connections 2024-04-25T14:13:43.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.898+0000 7fc711000640 1 -- 172.21.15.86:0/2460403082 wait complete. 2024-04-25T14:13:43.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.898+0000 7fc711000640 1 Processor -- start 2024-04-25T14:13:43.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.902+0000 7fc711000640 1 -- start start 2024-04-25T14:13:43.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.902+0000 7fc711000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6f0003bc0 0x7fc70c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:43.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.902+0000 7fc711000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc70c1e7b80 0x7fc70c1e91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:43.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.902+0000 7fc711000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc70c1f7710 0x7fc70c1f7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:43.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.902+0000 7fc711000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c0ce940 con 0x7fc70c1e7b80 2024-04-25T14:13:43.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.902+0000 7fc711000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c15df70 con 0x7fc70c1f7710 2024-04-25T14:13:43.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.902+0000 7fc711000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc70c0ceac0 con 0x7fc6f0003bc0 2024-04-25T14:13:43.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.902+0000 7fc709400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc70c1e7b80 0x7fc70c1e91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:43.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.902+0000 7fc709400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc70c1e7b80 0x7fc70c1e91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41694/0 (socket says 172.21.15.86:41694) 2024-04-25T14:13:43.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.902+0000 7fc709400640 1 -- 172.21.15.86:0/2833014431 learned_addr learned my addr 172.21.15.86:0/2833014431 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:43.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.902+0000 7fc709e00640 1 --2- 172.21.15.86:0/2833014431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6f0003bc0 0x7fc70c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:43.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.902+0000 7fc70a800640 1 --2- 172.21.15.86:0/2833014431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc70c1f7710 0x7fc70c1f7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:43.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.902+0000 7fc709400640 1 -- 172.21.15.86:0/2833014431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6f0003bc0 msgr2=0x7fc70c0b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:43.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.902+0000 7fc709400640 1 --2- 172.21.15.86:0/2833014431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6f0003bc0 0x7fc70c0b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:43.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.902+0000 7fc709400640 1 -- 172.21.15.86:0/2833014431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc70c1f7710 msgr2=0x7fc70c1f7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:43.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.902+0000 7fc709400640 1 --2- 172.21.15.86:0/2833014431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc70c1f7710 0x7fc70c1f7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:43.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.902+0000 7fc709400640 1 -- 172.21.15.86:0/2833014431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70c0b2530 con 0x7fc70c1e7b80 2024-04-25T14:13:43.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.902+0000 7fc709400640 1 --2- 172.21.15.86:0/2833014431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc70c1e7b80 0x7fc70c1e91e0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8011d00 tx=0x7fc6f800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:43.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.902+0000 7fc702000640 1 -- 172.21.15.86:0/2833014431 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f800fcc0 con 0x7fc70c1e7b80 2024-04-25T14:13:43.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.902+0000 7fc702000640 1 -- 172.21.15.86:0/2833014431 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6f800fe60 con 0x7fc70c1e7b80 2024-04-25T14:13:43.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.902+0000 7fc711000640 1 -- 172.21.15.86:0/2833014431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc70c1f8060 con 0x7fc70c1e7b80 2024-04-25T14:13:43.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.902+0000 7fc702000640 1 -- 172.21.15.86:0/2833014431 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8004270 con 0x7fc70c1e7b80 2024-04-25T14:13:43.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.902+0000 7fc711000640 1 -- 172.21.15.86:0/2833014431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc70c1e6dc0 con 0x7fc70c1e7b80 2024-04-25T14:13:43.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.906+0000 7fc711000640 1 -- 172.21.15.86:0/2833014431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc70c072170 con 0x7fc70c1e7b80 2024-04-25T14:13:43.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.906+0000 7fc702000640 1 -- 172.21.15.86:0/2833014431 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7fc6f8012040 con 0x7fc70c1e7b80 2024-04-25T14:13:43.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.906+0000 7fc702000640 1 --2- 172.21.15.86:0/2833014431 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fc6ec06f550 0x7fc6ec071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:43.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.906+0000 7fc702000640 1 -- 172.21.15.86:0/2833014431 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6101+0+0 (secure 0 0 0) 0x7fc6f808fbe0 con 0x7fc70c1e7b80 2024-04-25T14:13:43.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.906+0000 7fc709e00640 1 --2- 172.21.15.86:0/2833014431 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fc6ec06f550 0x7fc6ec071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:43.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.906+0000 7fc709e00640 1 --2- 172.21.15.86:0/2833014431 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fc6ec06f550 0x7fc6ec071a60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc0ad000 tx=0x7fc6fc001d50 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:43.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:43.910+0000 7fc702000640 1 -- 172.21.15.86:0/2833014431 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc70c1f8060 con 0x7fc70c1e7b80 2024-04-25T14:13:44.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.102+0000 7fc711000640 1 -- 172.21.15.86:0/2833014431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc70c0ce2f0 con 0x7fc70c1e7b80 2024-04-25T14:13:44.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.102+0000 7fc702000640 1 -- 172.21.15.86:0/2833014431 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) ==== 95+0+5202 (secure 0 0 0) 0x7fc6f8019070 con 0x7fc70c1e7b80 2024-04-25T14:13:44.100 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:13:44.101 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":117,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112},{"gid":8387,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:13:36.638299+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10209},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:13:44.101 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 117 2024-04-25T14:13:44.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.102+0000 7fc711000640 1 -- 172.21.15.86:0/2833014431 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fc6ec06f550 msgr2=0x7fc6ec071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:44.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.102+0000 7fc711000640 1 --2- 172.21.15.86:0/2833014431 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fc6ec06f550 0x7fc6ec071a60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc0ad000 tx=0x7fc6fc001d50 comp rx=0 tx=0).stop 2024-04-25T14:13:44.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.102+0000 7fc711000640 1 -- 172.21.15.86:0/2833014431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc70c1e7b80 msgr2=0x7fc70c1e91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:44.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.102+0000 7fc711000640 1 --2- 172.21.15.86:0/2833014431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc70c1e7b80 0x7fc70c1e91e0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8011d00 tx=0x7fc6f800f610 comp rx=0 tx=0).stop 2024-04-25T14:13:44.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.102+0000 7fc711000640 1 -- 172.21.15.86:0/2833014431 shutdown_connections 2024-04-25T14:13:44.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.102+0000 7fc711000640 1 --2- 172.21.15.86:0/2833014431 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fc6ec06f550 0x7fc6ec071a60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.102+0000 7fc711000640 1 --2- 172.21.15.86:0/2833014431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc70c1f7710 0x7fc70c1f7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.102+0000 7fc711000640 1 --2- 172.21.15.86:0/2833014431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc70c1e7b80 0x7fc70c1e91e0 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.102+0000 7fc711000640 1 --2- 172.21.15.86:0/2833014431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6f0003bc0 0x7fc70c0b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.102+0000 7fc711000640 1 -- 172.21.15.86:0/2833014431 >> 172.21.15.86:0/2833014431 conn(0x7fc70c0b5020 msgr2=0x7fc70c0b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:44.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.102+0000 7fc711000640 1 -- 172.21.15.86:0/2833014431 shutdown_connections 2024-04-25T14:13:44.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.102+0000 7fc711000640 1 -- 172.21.15.86:0/2833014431 wait complete. 2024-04-25T14:13:44.525 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 117, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:13:36.638299+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10209}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:13:44.525 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T14:13:44.525 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T14:13:44.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.674+0000 7f0cbe600640 1 Processor -- start 2024-04-25T14:13:44.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.674+0000 7f0cbe600640 1 -- start start 2024-04-25T14:13:44.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.678+0000 7f0cbe600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0cb80cfca0 0x7f0cb806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:44.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.678+0000 7f0cbe600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0cb80d16e0 0x7f0cb80d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:44.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.678+0000 7f0cbe600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cb806c670 0x7f0cb806ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:44.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.678+0000 7f0cbe600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0cb8070470 con 0x7f0cb80d16e0 2024-04-25T14:13:44.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.678+0000 7f0cbe600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0cb80d2030 con 0x7f0cb80cfca0 2024-04-25T14:13:44.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.678+0000 7f0cbe600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0cb8072bd0 con 0x7f0cb806c670 2024-04-25T14:13:44.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.678+0000 7f0caea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0cb80d16e0 0x7f0cb80d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:44.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.678+0000 7f0caea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0cb80d16e0 0x7f0cb80d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41698/0 (socket says 172.21.15.86:41698) 2024-04-25T14:13:44.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.678+0000 7f0caea00640 1 -- 172.21.15.86:0/4092051276 learned_addr learned my addr 172.21.15.86:0/4092051276 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:44.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.678+0000 7f0cb7e00640 1 --2- 172.21.15.86:0/4092051276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cb806c670 0x7f0cb806ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:44.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.678+0000 7f0cb7400640 1 --2- 172.21.15.86:0/4092051276 >> v2:172.21.15.155:3300/0 conn(0x7f0cb80cfca0 0x7f0cb806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:44.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.678+0000 7f0caea00640 1 -- 172.21.15.86:0/4092051276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cb806c670 msgr2=0x7f0cb806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:44.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.678+0000 7f0caea00640 1 --2- 172.21.15.86:0/4092051276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cb806c670 0x7f0cb806ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.678+0000 7f0caea00640 1 -- 172.21.15.86:0/4092051276 >> v2:172.21.15.155:3300/0 conn(0x7f0cb80cfca0 msgr2=0x7f0cb806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:44.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.678+0000 7f0caea00640 1 --2- 172.21.15.86:0/4092051276 >> v2:172.21.15.155:3300/0 conn(0x7f0cb80cfca0 0x7f0cb806f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.678+0000 7f0caea00640 1 -- 172.21.15.86:0/4092051276 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb806f930 con 0x7f0cb80d16e0 2024-04-25T14:13:44.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.678+0000 7f0cb7400640 1 --2- 172.21.15.86:0/4092051276 >> v2:172.21.15.155:3300/0 conn(0x7f0cb80cfca0 0x7f0cb806f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:44.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.682+0000 7f0caea00640 1 --2- 172.21.15.86:0/4092051276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cb80d16e0 0x7f0cb80d1af0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f0cb80d22a0 tx=0x7f0ca001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca3f5f1ae4360c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:44.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.682+0000 7f0cb6a00640 1 -- 172.21.15.86:0/4092051276 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca00ab5f0 con 0x7f0cb80d16e0 2024-04-25T14:13:44.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.682+0000 7f0cb6a00640 1 -- 172.21.15.86:0/4092051276 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ca00b0070 con 0x7f0cb80d16e0 2024-04-25T14:13:44.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.682+0000 7f0cb6a00640 1 -- 172.21.15.86:0/4092051276 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca00ab9b0 con 0x7f0cb80d16e0 2024-04-25T14:13:44.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.682+0000 7f0cbe600640 1 -- 172.21.15.86:0/4092051276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cb80d16e0 msgr2=0x7f0cb80d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:44.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.682+0000 7f0cbe600640 1 --2- 172.21.15.86:0/4092051276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cb80d16e0 0x7f0cb80d1af0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f0cb80d22a0 tx=0x7f0ca001c520 comp rx=0 tx=0).stop 2024-04-25T14:13:44.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.682+0000 7f0cbe600640 1 -- 172.21.15.86:0/4092051276 shutdown_connections 2024-04-25T14:13:44.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.682+0000 7f0cbe600640 1 --2- 172.21.15.86:0/4092051276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cb806c670 0x7f0cb806ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.682+0000 7f0cbe600640 1 --2- 172.21.15.86:0/4092051276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cb80d16e0 0x7f0cb80d1af0 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.682+0000 7f0cbe600640 1 --2- 172.21.15.86:0/4092051276 >> v2:172.21.15.155:3300/0 conn(0x7f0cb80cfca0 0x7f0cb806f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.682+0000 7f0cbe600640 1 -- 172.21.15.86:0/4092051276 >> 172.21.15.86:0/4092051276 conn(0x7f0cb80b4650 msgr2=0x7f0cb80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:44.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.682+0000 7f0cbe600640 1 -- 172.21.15.86:0/4092051276 shutdown_connections 2024-04-25T14:13:44.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.682+0000 7f0cbe600640 1 -- 172.21.15.86:0/4092051276 wait complete. 2024-04-25T14:13:44.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.686+0000 7f0cbe600640 1 Processor -- start 2024-04-25T14:13:44.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.686+0000 7f0cbe600640 1 -- start start 2024-04-25T14:13:44.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.686+0000 7f0cbe600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cb806c670 0x7f0cb8155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:44.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.686+0000 7f0cbe600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cb80cfca0 0x7f0cb815efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:44.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.686+0000 7f0cbe600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cb80d16e0 0x7f0cb815f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:44.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.686+0000 7f0cbe600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb80738c0 con 0x7f0cb80d16e0 2024-04-25T14:13:44.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.686+0000 7f0cbe600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb806d3d0 con 0x7f0cb80cfca0 2024-04-25T14:13:44.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.686+0000 7f0cbe600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0cb8073a40 con 0x7f0cb806c670 2024-04-25T14:13:44.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.686+0000 7f0cb7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cb80d16e0 0x7f0cb815f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:44.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.686+0000 7f0cb7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cb80d16e0 0x7f0cb815f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41706/0 (socket says 172.21.15.86:41706) 2024-04-25T14:13:44.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.686+0000 7f0cb7e00640 1 -- 172.21.15.86:0/3331476397 learned_addr learned my addr 172.21.15.86:0/3331476397 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:44.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.686+0000 7f0cb7400640 1 --2- 172.21.15.86:0/3331476397 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cb806c670 0x7f0cb8155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:44.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.686+0000 7f0caea00640 1 --2- 172.21.15.86:0/3331476397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cb80cfca0 0x7f0cb815efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:44.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.686+0000 7f0cb7e00640 1 -- 172.21.15.86:0/3331476397 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cb806c670 msgr2=0x7f0cb8155a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:44.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.686+0000 7f0cb7e00640 1 --2- 172.21.15.86:0/3331476397 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cb806c670 0x7f0cb8155a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.686+0000 7f0cb7e00640 1 -- 172.21.15.86:0/3331476397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cb80cfca0 msgr2=0x7f0cb815efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:44.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.686+0000 7f0cb7e00640 1 --2- 172.21.15.86:0/3331476397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cb80cfca0 0x7f0cb815efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.686+0000 7f0cb7e00640 1 -- 172.21.15.86:0/3331476397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb8155f70 con 0x7f0cb80d16e0 2024-04-25T14:13:44.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.686+0000 7f0cb7e00640 1 --2- 172.21.15.86:0/3331476397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cb80d16e0 0x7f0cb815f510 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0009fd0 tx=0x7f0cb000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:44.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.686+0000 7f0cafe00640 1 -- 172.21.15.86:0/3331476397 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0014420 con 0x7f0cb80d16e0 2024-04-25T14:13:44.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.686+0000 7f0cafe00640 1 -- 172.21.15.86:0/3331476397 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cb0019070 con 0x7f0cb80d16e0 2024-04-25T14:13:44.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.686+0000 7f0cbe600640 1 -- 172.21.15.86:0/3331476397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb815fde0 con 0x7f0cb80d16e0 2024-04-25T14:13:44.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.686+0000 7f0cafe00640 1 -- 172.21.15.86:0/3331476397 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0014800 con 0x7f0cb80d16e0 2024-04-25T14:13:44.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.686+0000 7f0cbe600640 1 -- 172.21.15.86:0/3331476397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb8158e00 con 0x7f0cb80d16e0 2024-04-25T14:13:44.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.690+0000 7f0cafe00640 1 -- 172.21.15.86:0/3331476397 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7f0cb00149a0 con 0x7f0cb80d16e0 2024-04-25T14:13:44.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.690+0000 7f0cafe00640 1 --2- 172.21.15.86:0/3331476397 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f0c9406f550 0x7f0c94071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:44.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.690+0000 7f0cc5ce0000 1 Processor -- start 2024-04-25T14:13:44.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.690+0000 7f0cafe00640 1 -- 172.21.15.86:0/3331476397 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6101+0+0 (secure 0 0 0) 0x7f0cb008fc90 con 0x7f0cb80d16e0 2024-04-25T14:13:44.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.690+0000 7f0cb7400640 1 --2- 172.21.15.86:0/3331476397 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f0c9406f550 0x7f0c94071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:44.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.690+0000 7f0cc5ce0000 1 -- start start 2024-04-25T14:13:44.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.690+0000 7f0cb7400640 1 --2- 172.21.15.86:0/3331476397 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f0c9406f550 0x7f0c94071a60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8070710 tx=0x7f0ca8008040 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:44.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0cc5ce0000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562ac23f3c20 0x562ac2551ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:44.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0cc5ce0000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ac23dfea0 0x562ac2552420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:44.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0cc5ce0000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac1fd8280 0x562ac2552960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:44.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0cc5ce0000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562ac23c1160 con 0x562ac23dfea0 2024-04-25T14:13:44.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0cc5ce0000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562ac2356e20 con 0x562ac1fd8280 2024-04-25T14:13:44.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0cc5ce0000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x562ac230fa20 con 0x562ac23f3c20 2024-04-25T14:13:44.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0caea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ac23dfea0 0x562ac2552420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:44.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0caea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ac23dfea0 0x562ac2552420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41712/0 (socket says 172.21.15.86:41712) 2024-04-25T14:13:44.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0caea00640 1 -- 172.21.15.86:0/3145067197 learned_addr learned my addr 172.21.15.86:0/3145067197 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:44.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0cb7400640 1 --2- 172.21.15.86:0/3145067197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562ac23f3c20 0x562ac2551ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:44.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0cb7e00640 1 --2- 172.21.15.86:0/3145067197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac1fd8280 0x562ac2552960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:44.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0caea00640 1 -- 172.21.15.86:0/3145067197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562ac23f3c20 msgr2=0x562ac2551ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:44.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0caea00640 1 --2- 172.21.15.86:0/3145067197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562ac23f3c20 0x562ac2551ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0caea00640 1 -- 172.21.15.86:0/3145067197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac1fd8280 msgr2=0x562ac2552960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:44.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0caea00640 1 --2- 172.21.15.86:0/3145067197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac1fd8280 0x562ac2552960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0caea00640 1 -- 172.21.15.86:0/3145067197 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ac2314100 con 0x562ac23dfea0 2024-04-25T14:13:44.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0caea00640 1 --2- 172.21.15.86:0/3145067197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ac23dfea0 0x562ac2552420 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00ad040 tx=0x7f0ca0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226812a3b39fb45a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:44.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0ca7e00640 1 -- 172.21.15.86:0/3145067197 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca00b4960 con 0x562ac23dfea0 2024-04-25T14:13:44.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0ca7e00640 1 -- 172.21.15.86:0/3145067197 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ca00b0040 con 0x562ac23dfea0 2024-04-25T14:13:44.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0ca7e00640 1 -- 172.21.15.86:0/3145067197 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca00b4d20 con 0x562ac23dfea0 2024-04-25T14:13:44.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0cc5ce0000 1 -- 172.21.15.86:0/3145067197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ac23dfea0 msgr2=0x562ac2552420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:44.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0cc5ce0000 1 --2- 172.21.15.86:0/3145067197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ac23dfea0 0x562ac2552420 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00ad040 tx=0x7f0ca0004740 comp rx=0 tx=0).stop 2024-04-25T14:13:44.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0cc5ce0000 1 -- 172.21.15.86:0/3145067197 shutdown_connections 2024-04-25T14:13:44.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0cc5ce0000 1 --2- 172.21.15.86:0/3145067197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac1fd8280 0x562ac2552960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0cc5ce0000 1 --2- 172.21.15.86:0/3145067197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ac23dfea0 0x562ac2552420 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0cc5ce0000 1 --2- 172.21.15.86:0/3145067197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562ac23f3c20 0x562ac2551ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0cc5ce0000 1 -- 172.21.15.86:0/3145067197 >> 172.21.15.86:0/3145067197 conn(0x562ac2017ff0 msgr2=0x562ac25074c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:44.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0cc5ce0000 1 -- 172.21.15.86:0/3145067197 shutdown_connections 2024-04-25T14:13:44.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.694+0000 7f0cc5ce0000 1 -- 172.21.15.86:0/3145067197 wait complete. 2024-04-25T14:13:44.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0cc5ce0000 1 -- start start 2024-04-25T14:13:44.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0cc5ce0000 1 Processor -- start 2024-04-25T14:13:44.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0cc5ce0000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac1fd8280 0x562ac2558e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:44.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0cc5ce0000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ac23dfea0 0x562ac25593a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:44.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0cc5ce0000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562ac23f3c20 0x562ac25140c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:44.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0cc5ce0000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562ac2364220 con 0x562ac23dfea0 2024-04-25T14:13:44.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0cc5ce0000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562ac22faea0 con 0x562ac1fd8280 2024-04-25T14:13:44.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0cc5ce0000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x562ac2351b00 con 0x562ac23f3c20 2024-04-25T14:13:44.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0cc5ce0000 10 client.0 _ll_register_callbacks cb 0x562ac2363dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:44.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0caea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ac23dfea0 0x562ac25593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:44.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0caea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ac23dfea0 0x562ac25593a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41724/0 (socket says 172.21.15.86:41724) 2024-04-25T14:13:44.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0caea00640 1 -- 172.21.15.86:0/912632844 learned_addr learned my addr 172.21.15.86:0/912632844 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:44.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0cb7400640 1 --2- 172.21.15.86:0/912632844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac1fd8280 0x562ac2558e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:44.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0cb7e00640 1 --2- 172.21.15.86:0/912632844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562ac23f3c20 0x562ac25140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:44.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0caea00640 1 -- 172.21.15.86:0/912632844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562ac23f3c20 msgr2=0x562ac25140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:44.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0caea00640 1 --2- 172.21.15.86:0/912632844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562ac23f3c20 0x562ac25140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0caea00640 1 -- 172.21.15.86:0/912632844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac1fd8280 msgr2=0x562ac2558e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:44.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0caea00640 1 --2- 172.21.15.86:0/912632844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac1fd8280 0x562ac2558e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0caea00640 1 -- 172.21.15.86:0/912632844 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ac22f6500 con 0x562ac23dfea0 2024-04-25T14:13:44.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0caea00640 1 --2- 172.21.15.86:0/912632844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ac23dfea0 0x562ac25593a0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f0ca001ca00 tx=0x7f0ca001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a305492ed73a3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:44.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0ca4c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:13:44.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0ca4c00640 1 -- 172.21.15.86:0/912632844 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca00bf450 con 0x562ac23dfea0 2024-04-25T14:13:44.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0ca4c00640 1 -- 172.21.15.86:0/912632844 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(155..155 src has 1..155) ==== 6101+0+0 (secure 0 0 0) 0x7f0ca00bf5f0 con 0x562ac23dfea0 2024-04-25T14:13:44.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0cc5ce0000 1 -- 172.21.15.86:0/912632844 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ca00b0040 con 0x562ac23dfea0 2024-04-25T14:13:44.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0ca4c00640 1 -- 172.21.15.86:0/912632844 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=117) ==== 24+0+0 (secure 0 0 0) 0x7f0ca00cf050 con 0x562ac23dfea0 2024-04-25T14:13:44.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0cc5ce0000 10 client.10380 fetch_fsmap learned FSMap version 117 2024-04-25T14:13:44.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0cc5ce0000 1 -- 172.21.15.86:0/912632844 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=117,monmap=2+}) -- 0x562ac22dcea0 con 0x562ac23dfea0 2024-04-25T14:13:44.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0ca4c00640 1 -- 172.21.15.86:0/912632844 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ca00b0040 con 0x562ac23dfea0 2024-04-25T14:13:44.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0ca4c00640 1 -- 172.21.15.86:0/912632844 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 117) ==== 3494+0+0 (secure 0 0 0) 0x7f0ca00d2020 con 0x562ac23dfea0 2024-04-25T14:13:44.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0cc5ce0000 10 client.10380 fetch_fsmap finished waiting for FSMap version 117 2024-04-25T14:13:44.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0cc5ce0000 10 client.10380 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T14:13:44.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0cc5ce0000 20 client.10380 populate_metadata read hostname 'smithi086' 2024-04-25T14:13:44.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0cc5ce0000 1 --2- 172.21.15.86:0/912632844 >> [v2:172.21.15.86:6836/3141015835,v1:172.21.15.86:6837/3141015835] conn(0x562ac1ffc7d0 0x562ac2514e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:44.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0cc5ce0000 4 client.10380 mds_command: new command op to 10209 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:13:44.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0cc5ce0000 1 -- 172.21.15.86:0/912632844 --> [v2:172.21.15.86:6836/3141015835,v1:172.21.15.86:6837/3141015835] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ac22efbe0 con 0x562ac1ffc7d0 2024-04-25T14:13:44.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.698+0000 7f0cb7400640 1 --2- 172.21.15.86:0/912632844 >> [v2:172.21.15.86:6836/3141015835,v1:172.21.15.86:6837/3141015835] conn(0x562ac1ffc7d0 0x562ac2514e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:44.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.702+0000 7f0ca4c00640 10 client.10380 ms_handle_connect on v2:172.21.15.86:6836/3141015835 2024-04-25T14:13:44.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.702+0000 7f0cb7400640 1 --2- 172.21.15.86:0/912632844 >> [v2:172.21.15.86:6836/3141015835,v1:172.21.15.86:6837/3141015835] conn(0x562ac1ffc7d0 0x562ac2514e30 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a952a83b3dfd8ced server_cookie=eb01c1ff54cb958d in_seq=0 out_seq=0 2024-04-25T14:13:44.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.702+0000 7f0ca4c00640 1 -- 172.21.15.86:0/912632844 <== mds.0 v2:172.21.15.86:6836/3141015835 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562ac22efbe0 con 0x562ac1ffc7d0 2024-04-25T14:13:44.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.702+0000 7f0ca4c00640 10 client.10380 handle_command_reply: tid=0 2024-04-25T14:13:44.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.702+0000 7f0cc5ce0000 1 client.10380 shutdown 2024-04-25T14:13:44.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.702+0000 7f0cc5ce0000 1 -- 172.21.15.86:0/912632844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ac23dfea0 msgr2=0x562ac25593a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:44.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.702+0000 7f0cc5ce0000 1 --2- 172.21.15.86:0/912632844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ac23dfea0 0x562ac25593a0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f0ca001ca00 tx=0x7f0ca001ad60 comp rx=0 tx=0).stop 2024-04-25T14:13:44.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.702+0000 7f0cc5ce0000 1 -- 172.21.15.86:0/912632844 shutdown_connections 2024-04-25T14:13:44.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.702+0000 7f0cc5ce0000 1 --2- 172.21.15.86:0/912632844 >> [v2:172.21.15.86:6836/3141015835,v1:172.21.15.86:6837/3141015835] conn(0x562ac1ffc7d0 0x562ac2514e30 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.702+0000 7f0cc5ce0000 1 --2- 172.21.15.86:0/912632844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562ac23f3c20 0x562ac25140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.702+0000 7f0cc5ce0000 1 --2- 172.21.15.86:0/912632844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ac23dfea0 0x562ac25593a0 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.702+0000 7f0ca4c00640 0 client.10380 ms_handle_reset on v2:172.21.15.86:6836/3141015835 2024-04-25T14:13:44.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.702+0000 7f0cc5ce0000 1 --2- 172.21.15.86:0/912632844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac1fd8280 0x562ac2558e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.702+0000 7f0cc5ce0000 1 -- 172.21.15.86:0/912632844 >> 172.21.15.86:0/912632844 conn(0x562ac2017ff0 msgr2=0x562ac2507c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:44.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.702+0000 7f0cc5ce0000 1 -- 172.21.15.86:0/912632844 shutdown_connections 2024-04-25T14:13:44.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.702+0000 7f0cc5ce0000 1 -- 172.21.15.86:0/912632844 wait complete. 2024-04-25T14:13:44.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.702+0000 7f0cc5ce0000 20 client.10380 trim_cache size 0 max 16384 2024-04-25T14:13:44.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0cc5ce0000 1 Processor -- start 2024-04-25T14:13:44.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0cc5ce0000 1 -- start start 2024-04-25T14:13:44.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0cc5ce0000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ac1fd8280 0x562ac255e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:44.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0cc5ce0000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562ac23dfea0 0x562ac2567ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:44.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0cc5ce0000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac23f3c20 0x562ac2568530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:44.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0cc5ce0000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb82054a0 con 0x562ac1fd8280 2024-04-25T14:13:44.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0cc5ce0000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb80aebf0 con 0x562ac23f3c20 2024-04-25T14:13:44.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0cc5ce0000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0cb80af240 con 0x562ac23dfea0 2024-04-25T14:13:44.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0cb7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ac1fd8280 0x562ac255e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:44.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0cb7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ac1fd8280 0x562ac255e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41728/0 (socket says 172.21.15.86:41728) 2024-04-25T14:13:44.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0cb7400640 1 -- 172.21.15.86:0/4014878587 learned_addr learned my addr 172.21.15.86:0/4014878587 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:44.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0cb7400640 1 -- 172.21.15.86:0/4014878587 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562ac23dfea0 msgr2=0x562ac2567ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:44.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0cb7400640 1 --2- 172.21.15.86:0/4014878587 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562ac23dfea0 0x562ac2567ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0cb7e00640 1 --2- 172.21.15.86:0/4014878587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac23f3c20 0x562ac2568530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:44.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0cb7400640 1 -- 172.21.15.86:0/4014878587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac23f3c20 msgr2=0x562ac2568530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:44.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0cb7400640 1 --2- 172.21.15.86:0/4014878587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac23f3c20 0x562ac2568530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0cb7400640 1 -- 172.21.15.86:0/4014878587 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ac250b840 con 0x562ac1fd8280 2024-04-25T14:13:44.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0cb7e00640 1 --2- 172.21.15.86:0/4014878587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac23f3c20 0x562ac2568530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:44.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0cb7400640 1 --2- 172.21.15.86:0/4014878587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ac1fd8280 0x562ac255e0b0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8004120 tx=0x7f0ca8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6663d49eaed4d44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:44.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0ca7400640 1 -- 172.21.15.86:0/4014878587 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca802e070 con 0x562ac1fd8280 2024-04-25T14:13:44.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0ca7400640 1 -- 172.21.15.86:0/4014878587 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ca8019040 con 0x562ac1fd8280 2024-04-25T14:13:44.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0ca7400640 1 -- 172.21.15.86:0/4014878587 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8003d60 con 0x562ac1fd8280 2024-04-25T14:13:44.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0cc5ce0000 1 -- 172.21.15.86:0/4014878587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ac1fd8280 msgr2=0x562ac255e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:44.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0cc5ce0000 1 --2- 172.21.15.86:0/4014878587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ac1fd8280 0x562ac255e0b0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8004120 tx=0x7f0ca8016000 comp rx=0 tx=0).stop 2024-04-25T14:13:44.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0cc5ce0000 1 -- 172.21.15.86:0/4014878587 shutdown_connections 2024-04-25T14:13:44.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0cc5ce0000 1 --2- 172.21.15.86:0/4014878587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac23f3c20 0x562ac2568530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0cc5ce0000 1 --2- 172.21.15.86:0/4014878587 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562ac23dfea0 0x562ac2567ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0cc5ce0000 1 --2- 172.21.15.86:0/4014878587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ac1fd8280 0x562ac255e0b0 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0cc5ce0000 1 -- 172.21.15.86:0/4014878587 >> 172.21.15.86:0/4014878587 conn(0x562ac2017ff0 msgr2=0x562ac250f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:44.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0cc5ce0000 1 -- 172.21.15.86:0/4014878587 shutdown_connections 2024-04-25T14:13:44.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.722+0000 7f0cc5ce0000 1 -- 172.21.15.86:0/4014878587 wait complete. 2024-04-25T14:13:44.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.726+0000 7f0cc5ce0000 1 -- start start 2024-04-25T14:13:44.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.726+0000 7f0cc5ce0000 1 Processor -- start 2024-04-25T14:13:44.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.726+0000 7f0cc5ce0000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ac1fd8280 0x562ac25c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:44.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.726+0000 7f0cc5ce0000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac23dfea0 0x562ac25c48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:44.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.726+0000 7f0cc5ce0000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562ac23f3c20 0x562ac255c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:44.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.726+0000 7f0cc5ce0000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562ac2356130 con 0x562ac1fd8280 2024-04-25T14:13:44.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.726+0000 7f0cc5ce0000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562ac22ec7d0 con 0x562ac23dfea0 2024-04-25T14:13:44.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.726+0000 7f0cc5ce0000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x562ac235b450 con 0x562ac23f3c20 2024-04-25T14:13:44.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.726+0000 7f0cc5ce0000 10 client.0 _ll_register_callbacks cb 0x7f0ca00b43f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:13:44.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.726+0000 7f0cb7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ac1fd8280 0x562ac25c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:44.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.726+0000 7f0cb7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ac1fd8280 0x562ac25c4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41738/0 (socket says 172.21.15.86:41738) 2024-04-25T14:13:44.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.726+0000 7f0cb7400640 1 -- 172.21.15.86:0/2752329837 learned_addr learned my addr 172.21.15.86:0/2752329837 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:44.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.726+0000 7f0caea00640 1 --2- 172.21.15.86:0/2752329837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac23dfea0 0x562ac25c48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:44.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.726+0000 7f0cb7e00640 1 --2- 172.21.15.86:0/2752329837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562ac23f3c20 0x562ac255c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:44.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.726+0000 7f0cb7400640 1 -- 172.21.15.86:0/2752329837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562ac23f3c20 msgr2=0x562ac255c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:44.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.726+0000 7f0cb7400640 1 --2- 172.21.15.86:0/2752329837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562ac23f3c20 0x562ac255c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.726+0000 7f0cb7400640 1 -- 172.21.15.86:0/2752329837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac23dfea0 msgr2=0x562ac25c48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:44.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.726+0000 7f0cb7400640 1 --2- 172.21.15.86:0/2752329837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac23dfea0 0x562ac25c48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.726+0000 7f0cb7400640 1 -- 172.21.15.86:0/2752329837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ac2514430 con 0x562ac1fd8280 2024-04-25T14:13:44.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.726+0000 7f0caea00640 1 --2- 172.21.15.86:0/2752329837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac23dfea0 0x562ac25c48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:44.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.726+0000 7f0cb7400640 1 --2- 172.21.15.86:0/2752329837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ac1fd8280 0x562ac25c4380 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8004120 tx=0x7f0ca8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac57978923dfa692 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:44.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.726+0000 7f0ca4c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:13:44.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.726+0000 7f0ca4c00640 1 -- 172.21.15.86:0/2752329837 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8037060 con 0x562ac1fd8280 2024-04-25T14:13:44.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.726+0000 7f0ca4c00640 1 -- 172.21.15.86:0/2752329837 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(155..155 src has 1..155) ==== 6101+0+0 (secure 0 0 0) 0x7f0ca8027040 con 0x562ac1fd8280 2024-04-25T14:13:44.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.726+0000 7f0cc5ce0000 1 -- 172.21.15.86:0/2752329837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ca8019040 con 0x562ac1fd8280 2024-04-25T14:13:44.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.726+0000 7f0ca4c00640 1 -- 172.21.15.86:0/2752329837 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=117) ==== 24+0+0 (secure 0 0 0) 0x7f0ca803d050 con 0x562ac1fd8280 2024-04-25T14:13:44.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.726+0000 7f0cc5ce0000 10 client.10386 fetch_fsmap learned FSMap version 117 2024-04-25T14:13:44.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.726+0000 7f0cc5ce0000 1 -- 172.21.15.86:0/2752329837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=117,monmap=2+}) -- 0x562ac230fc70 con 0x562ac1fd8280 2024-04-25T14:13:44.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0ca4c00640 1 -- 172.21.15.86:0/2752329837 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ca8019040 con 0x562ac1fd8280 2024-04-25T14:13:44.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0ca4c00640 1 -- 172.21.15.86:0/2752329837 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 117) ==== 3494+0+0 (secure 0 0 0) 0x7f0ca8040020 con 0x562ac1fd8280 2024-04-25T14:13:44.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cc5ce0000 10 client.10386 fetch_fsmap finished waiting for FSMap version 117 2024-04-25T14:13:44.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cc5ce0000 10 client.10386 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T14:13:44.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cc5ce0000 20 client.10386 populate_metadata read hostname 'smithi086' 2024-04-25T14:13:44.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cc5ce0000 1 --2- 172.21.15.86:0/2752329837 >> [v2:172.21.15.86:6836/3141015835,v1:172.21.15.86:6837/3141015835] conn(0x562ac1fea830 0x562ac25cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:44.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cc5ce0000 4 client.10386 mds_command: new command op to 10209 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:13:44.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cc5ce0000 1 -- 172.21.15.86:0/2752329837 --> [v2:172.21.15.86:6836/3141015835,v1:172.21.15.86:6837/3141015835] -- command(tid 0: {"prefix": "status"}) -- 0x562ac2364a30 con 0x562ac1fea830 2024-04-25T14:13:44.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0caea00640 1 --2- 172.21.15.86:0/2752329837 >> [v2:172.21.15.86:6836/3141015835,v1:172.21.15.86:6837/3141015835] conn(0x562ac1fea830 0x562ac25cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:44.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0caea00640 1 --2- 172.21.15.86:0/2752329837 >> [v2:172.21.15.86:6836/3141015835,v1:172.21.15.86:6837/3141015835] conn(0x562ac1fea830 0x562ac25cd1f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=619376fd2a858877 server_cookie=25fe5d9d256f10a7 in_seq=0 out_seq=0 2024-04-25T14:13:44.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0ca4c00640 10 client.10386 ms_handle_connect on v2:172.21.15.86:6836/3141015835 2024-04-25T14:13:44.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0ca4c00640 1 -- 172.21.15.86:0/2752329837 <== mds.0 v2:172.21.15.86:6836/3141015835 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x562ac2364a30 con 0x562ac1fea830 2024-04-25T14:13:44.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0ca4c00640 10 client.10386 handle_command_reply: tid=0 2024-04-25T14:13:44.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cc5ce0000 1 client.10386 shutdown 2024-04-25T14:13:44.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cc5ce0000 1 -- 172.21.15.86:0/2752329837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ac1fd8280 msgr2=0x562ac25c4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:44.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cc5ce0000 1 --2- 172.21.15.86:0/2752329837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ac1fd8280 0x562ac25c4380 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8004120 tx=0x7f0ca8004740 comp rx=0 tx=0).stop 2024-04-25T14:13:44.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cc5ce0000 1 -- 172.21.15.86:0/2752329837 shutdown_connections 2024-04-25T14:13:44.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cc5ce0000 1 --2- 172.21.15.86:0/2752329837 >> [v2:172.21.15.86:6836/3141015835,v1:172.21.15.86:6837/3141015835] conn(0x562ac1fea830 0x562ac25cd1f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cc5ce0000 1 --2- 172.21.15.86:0/2752329837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562ac23f3c20 0x562ac255c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cc5ce0000 1 --2- 172.21.15.86:0/2752329837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac23dfea0 0x562ac25c48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cc5ce0000 1 --2- 172.21.15.86:0/2752329837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ac1fd8280 0x562ac25c4380 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0ca4c00640 0 client.10386 ms_handle_reset on v2:172.21.15.86:6836/3141015835 2024-04-25T14:13:44.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cc5ce0000 1 -- 172.21.15.86:0/2752329837 >> 172.21.15.86:0/2752329837 conn(0x562ac2017ff0 msgr2=0x562ac2557040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:44.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cc5ce0000 1 -- 172.21.15.86:0/2752329837 shutdown_connections 2024-04-25T14:13:44.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cc5ce0000 1 -- 172.21.15.86:0/2752329837 wait complete. 2024-04-25T14:13:44.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cc5ce0000 20 client.10386 trim_cache size 0 max 16384 2024-04-25T14:13:44.734 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:13:44.734 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb", 2024-04-25T14:13:44.735 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-25T14:13:44.735 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 10209, 2024-04-25T14:13:44.735 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T14:13:44.735 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T14:13:44.735 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T14:13:44.735 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 10.506611347, 2024-04-25T14:13:44.735 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 117, 2024-04-25T14:13:44.736 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 155, 2024-04-25T14:13:44.736 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 155, 2024-04-25T14:13:44.736 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 15.376278669 2024-04-25T14:13:44.736 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:13:44.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cbe600640 1 -- 172.21.15.86:0/3331476397 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f0c9406f550 msgr2=0x7f0c94071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:44.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cbe600640 1 --2- 172.21.15.86:0/3331476397 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f0c9406f550 0x7f0c94071a60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8070710 tx=0x7f0ca8008040 comp rx=0 tx=0).stop 2024-04-25T14:13:44.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cbe600640 1 -- 172.21.15.86:0/3331476397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cb80d16e0 msgr2=0x7f0cb815f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:44.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cbe600640 1 --2- 172.21.15.86:0/3331476397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cb80d16e0 0x7f0cb815f510 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0009fd0 tx=0x7f0cb000e4b0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cbe600640 1 -- 172.21.15.86:0/3331476397 shutdown_connections 2024-04-25T14:13:44.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cbe600640 1 --2- 172.21.15.86:0/3331476397 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f0c9406f550 0x7f0c94071a60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cbe600640 1 --2- 172.21.15.86:0/3331476397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cb80d16e0 0x7f0cb815f510 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cbe600640 1 --2- 172.21.15.86:0/3331476397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cb80cfca0 0x7f0cb815efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cbe600640 1 --2- 172.21.15.86:0/3331476397 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cb806c670 0x7f0cb8155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cbe600640 1 -- 172.21.15.86:0/3331476397 >> 172.21.15.86:0/3331476397 conn(0x7f0cb80b4650 msgr2=0x7f0cb815a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:44.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cbe600640 1 -- 172.21.15.86:0/3331476397 shutdown_connections 2024-04-25T14:13:44.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.730+0000 7f0cbe600640 1 -- 172.21.15.86:0/3331476397 wait complete. 2024-04-25T14:13:44.750 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T14:13:44.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.898+0000 7ff1e2200640 1 Processor -- start 2024-04-25T14:13:44.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.898+0000 7ff1e2200640 1 -- start start 2024-04-25T14:13:44.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.898+0000 7ff1e2200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1dc070330 0x7ff1dc0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:44.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.898+0000 7ff1e2200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff1dc0ce790 0x7ff1dc06f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:44.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.898+0000 7ff1e2200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1dc0d2030 con 0x7ff1dc0d1c20 2024-04-25T14:13:44.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.898+0000 7ff1e2200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff1dc15e5c0 con 0x7ff1dc0ce790 2024-04-25T14:13:44.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.898+0000 7ff1e2200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff1dc15e8a0 con 0x7ff1dc070330 2024-04-25T14:13:44.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.902+0000 7ff1dbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff1dc0ce790 0x7ff1dc06f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:44.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.902+0000 7ff1dbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff1dc0ce790 0x7ff1dc06f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41744/0 (socket says 172.21.15.86:41744) 2024-04-25T14:13:44.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.902+0000 7ff1db400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1dc070330 0x7ff1dc0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:44.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.902+0000 7ff1dbe00640 1 -- 172.21.15.86:0/3594297709 learned_addr learned my addr 172.21.15.86:0/3594297709 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:44.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.902+0000 7ff1dbe00640 1 -- 172.21.15.86:0/3594297709 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1dc070330 msgr2=0x7ff1dc0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:44.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.902+0000 7ff1dbe00640 1 --2- 172.21.15.86:0/3594297709 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1dc070330 0x7ff1dc0d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.902+0000 7ff1dbe00640 1 -- 172.21.15.86:0/3594297709 >> v1:172.21.15.155:6789/0 conn(0x7ff1dc0d1c20 legacy=0x7ff1dc06ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:44.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.902+0000 7ff1da000640 1 -- 172.21.15.86:0/3594297709 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3604174341 0 0) 0x7ff1c8001120 con 0x7ff1dc0d1c20 2024-04-25T14:13:44.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.902+0000 7ff1dbe00640 1 -- 172.21.15.86:0/3594297709 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1dc15eb80 con 0x7ff1dc0ce790 2024-04-25T14:13:44.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.902+0000 7ff1dbe00640 1 --2- 172.21.15.86:0/3594297709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1dc0ce790 0x7ff1dc06f440 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40098a0 tx=0x7ff1d401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6abeaa246c5c30a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:44.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.902+0000 7ff1da000640 1 -- 172.21.15.86:0/3594297709 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d40b0070 con 0x7ff1dc0ce790 2024-04-25T14:13:44.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.902+0000 7ff1da000640 1 -- 172.21.15.86:0/3594297709 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1d40ab600 con 0x7ff1dc0ce790 2024-04-25T14:13:44.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.902+0000 7ff1da000640 1 -- 172.21.15.86:0/3594297709 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d40ab9d0 con 0x7ff1dc0ce790 2024-04-25T14:13:44.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.906+0000 7ff1e2200640 1 -- 172.21.15.86:0/3594297709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1dc0ce790 msgr2=0x7ff1dc06f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:44.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.906+0000 7ff1e2200640 1 --2- 172.21.15.86:0/3594297709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1dc0ce790 0x7ff1dc06f440 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40098a0 tx=0x7ff1d401c4b0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.906+0000 7ff1e2200640 1 -- 172.21.15.86:0/3594297709 shutdown_connections 2024-04-25T14:13:44.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.906+0000 7ff1e2200640 1 --2- 172.21.15.86:0/3594297709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1dc0ce790 0x7ff1dc06f440 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.906+0000 7ff1e2200640 1 --2- 172.21.15.86:0/3594297709 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1dc070330 0x7ff1dc0d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.906+0000 7ff1e2200640 1 -- 172.21.15.86:0/3594297709 >> 172.21.15.86:0/3594297709 conn(0x7ff1dc0b4650 msgr2=0x7ff1dc0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:44.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.906+0000 7ff1e2200640 1 -- 172.21.15.86:0/3594297709 shutdown_connections 2024-04-25T14:13:44.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.906+0000 7ff1e2200640 1 -- 172.21.15.86:0/3594297709 wait complete. 2024-04-25T14:13:44.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.906+0000 7ff1e2200640 1 Processor -- start 2024-04-25T14:13:44.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.906+0000 7ff1e2200640 1 -- start start 2024-04-25T14:13:44.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.910+0000 7ff1e2200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1dc070330 0x7ff1dc099a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:44.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.910+0000 7ff1e2200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1dc0ce790 0x7ff1dc0c36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:44.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.910+0000 7ff1e2200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1dc0d1c20 0x7ff1dc0c3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:44.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.910+0000 7ff1e2200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc0708c0 con 0x7ff1dc0ce790 2024-04-25T14:13:44.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.910+0000 7ff1e2200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc070740 con 0x7ff1dc070330 2024-04-25T14:13:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.910+0000 7ff1e2200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff1dc16afc0 con 0x7ff1dc0d1c20 2024-04-25T14:13:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.910+0000 7ff1daa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1dc0ce790 0x7ff1dc0c36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.910+0000 7ff1daa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1dc0ce790 0x7ff1dc0c36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41758/0 (socket says 172.21.15.86:41758) 2024-04-25T14:13:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.910+0000 7ff1daa00640 1 -- 172.21.15.86:0/300078515 learned_addr learned my addr 172.21.15.86:0/300078515 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.910+0000 7ff1db400640 1 --2- 172.21.15.86:0/300078515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1dc070330 0x7ff1dc099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.910+0000 7ff1dbe00640 1 --2- 172.21.15.86:0/300078515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1dc0d1c20 0x7ff1dc0c3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.910+0000 7ff1daa00640 1 -- 172.21.15.86:0/300078515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1dc0d1c20 msgr2=0x7ff1dc0c3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.910+0000 7ff1daa00640 1 --2- 172.21.15.86:0/300078515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1dc0d1c20 0x7ff1dc0c3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.910+0000 7ff1daa00640 1 -- 172.21.15.86:0/300078515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1dc070330 msgr2=0x7ff1dc099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.910+0000 7ff1daa00640 1 --2- 172.21.15.86:0/300078515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1dc070330 0x7ff1dc099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.910+0000 7ff1daa00640 1 -- 172.21.15.86:0/300078515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1dc073850 con 0x7ff1dc0ce790 2024-04-25T14:13:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.910+0000 7ff1daa00640 1 --2- 172.21.15.86:0/300078515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1dc0ce790 0x7ff1dc0c36f0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800c450 tx=0x7ff1c800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.910+0000 7ff1d3400640 1 -- 172.21.15.86:0/300078515 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8017070 con 0x7ff1dc0ce790 2024-04-25T14:13:44.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.910+0000 7ff1d3400640 1 -- 172.21.15.86:0/300078515 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c8005480 con 0x7ff1dc0ce790 2024-04-25T14:13:44.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.910+0000 7ff1e2200640 1 -- 172.21.15.86:0/300078515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1dc073570 con 0x7ff1dc0ce790 2024-04-25T14:13:44.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.910+0000 7ff1d3400640 1 -- 172.21.15.86:0/300078515 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8003840 con 0x7ff1dc0ce790 2024-04-25T14:13:44.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.910+0000 7ff1e2200640 1 -- 172.21.15.86:0/300078515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1dc074500 con 0x7ff1dc0ce790 2024-04-25T14:13:44.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.910+0000 7ff1e2200640 1 -- 172.21.15.86:0/300078515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1dc069d00 con 0x7ff1dc0ce790 2024-04-25T14:13:44.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.910+0000 7ff1d3400640 1 -- 172.21.15.86:0/300078515 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7ff1c801a070 con 0x7ff1dc0ce790 2024-04-25T14:13:44.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.914+0000 7ff1d3400640 1 --2- 172.21.15.86:0/300078515 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7ff1bc06f520 0x7ff1bc071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:44.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.914+0000 7ff1d3400640 1 -- 172.21.15.86:0/300078515 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6101+0+0 (secure 0 0 0) 0x7ff1c808e830 con 0x7ff1dc0ce790 2024-04-25T14:13:44.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.914+0000 7ff1db400640 1 --2- 172.21.15.86:0/300078515 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7ff1bc06f520 0x7ff1bc071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:44.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.914+0000 7ff1db400640 1 --2- 172.21.15.86:0/300078515 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7ff1bc06f520 0x7ff1bc071a30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc072200 tx=0x7ff1cc00b040 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:44.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:44.918+0000 7ff1d3400640 1 -- 172.21.15.86:0/300078515 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff1dc0d2030 con 0x7ff1dc0ce790 2024-04-25T14:13:45.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:45.110+0000 7ff1e2200640 1 -- 172.21.15.86:0/300078515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff1dc076bf0 con 0x7ff1dc0ce790 2024-04-25T14:13:45.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:45.110+0000 7ff1d3400640 1 -- 172.21.15.86:0/300078515 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v117) ==== 53+0+73 (secure 0 0 0) 0x7ff1c805b500 con 0x7ff1dc0ce790 2024-04-25T14:13:45.111 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T14:13:45.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:45.114+0000 7ff1e2200640 1 -- 172.21.15.86:0/300078515 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7ff1bc06f520 msgr2=0x7ff1bc071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:45.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:45.114+0000 7ff1e2200640 1 --2- 172.21.15.86:0/300078515 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7ff1bc06f520 0x7ff1bc071a30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc072200 tx=0x7ff1cc00b040 comp rx=0 tx=0).stop 2024-04-25T14:13:45.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:45.114+0000 7ff1e2200640 1 -- 172.21.15.86:0/300078515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1dc0ce790 msgr2=0x7ff1dc0c36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:45.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:45.114+0000 7ff1e2200640 1 --2- 172.21.15.86:0/300078515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1dc0ce790 0x7ff1dc0c36f0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800c450 tx=0x7ff1c800c910 comp rx=0 tx=0).stop 2024-04-25T14:13:45.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:45.114+0000 7ff1e2200640 1 -- 172.21.15.86:0/300078515 shutdown_connections 2024-04-25T14:13:45.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:45.114+0000 7ff1e2200640 1 --2- 172.21.15.86:0/300078515 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7ff1bc06f520 0x7ff1bc071a30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:45.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:45.114+0000 7ff1e2200640 1 --2- 172.21.15.86:0/300078515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1dc0d1c20 0x7ff1dc0c3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:45.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:45.114+0000 7ff1e2200640 1 --2- 172.21.15.86:0/300078515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1dc0ce790 0x7ff1dc0c36f0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:45.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:45.114+0000 7ff1e2200640 1 --2- 172.21.15.86:0/300078515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1dc070330 0x7ff1dc099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:45.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:45.114+0000 7ff1e2200640 1 -- 172.21.15.86:0/300078515 >> 172.21.15.86:0/300078515 conn(0x7ff1dc0b4650 msgr2=0x7ff1dc0b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:45.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:45.114+0000 7ff1e2200640 1 -- 172.21.15.86:0/300078515 shutdown_connections 2024-04-25T14:13:45.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:45.114+0000 7ff1e2200640 1 -- 172.21.15.86:0/300078515 wait complete. 2024-04-25T14:13:45.133 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T14:13:45.133 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T14:13:45.133 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-25T14:13:45.133 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T14:13:45.133 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T14:13:45.133 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T14:13:45.133 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T14:13:45.133 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T14:13:45.133 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:13:45.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:45.134 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-25T14:13:45.143 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:13:45.143 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:13:45.143 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:13:45.143 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:13:45.143 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-25T14:13:45.143 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:13:45.143 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:13:45.143 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:13:45.143 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-25T14:13:45.143 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:13:45.143 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:13:45.143 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-25T14:13:45.144 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:13:45.144 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:13:45.144 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T14:13:45.144 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:13:45.144 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T14:13:45.144 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 41839sec preferred_lft 41839sec 2024-04-25T14:13:45.144 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-25T14:13:45.144 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:13:45.144 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T14:13:45.144 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:dc:87:9c:96 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:13:45.144 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T14:13:45.144 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:13:45.144 INFO:teuthology.orchestra.run.smithi086.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T14:13:45.144 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T14:13:45.144 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T14:13:45.145 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:13:45.145 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T14:13:45.145 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:13:45.145 INFO:teuthology.orchestra.run.smithi086.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:13:45.145 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T14:13:45.145 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T14:13:45.145 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:13:45.145 INFO:teuthology.orchestra.run.smithi086.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:13:45.145 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T14:13:45.145 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T14:13:45.145 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:13:45.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:45.146 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T14:13:45.201 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-25T14:13:45.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:45.203 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-25T14:13:45.260 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-25T14:13:45.260 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T14:13:45.261 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T14:13:45.261 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T14:13:45.261 INFO:teuthology.orchestra.run.smithi086.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T14:13:45.261 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T14:13:45.262 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T14:13:45.262 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:13:45.262 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:13:45.262 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:13:45.262 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T14:13:45.263 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:13:45.263 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-25T14:13:45.263 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T14:13:45.264 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T14:13:45.264 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T14:13:45.264 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-25T14:13:45.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:13:45.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:13:45.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:45.397 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T14:13:45.406 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T14:13:45.406 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T14:13:45.406 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:13:45.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:45.406 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-25T14:13:45.467 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:13:45.467 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T14:13:45.515 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T14:13:45.515 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:13:45.515 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T14:13:45.570 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T14:13:45.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:45.571 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:13:45.674 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:13:45.674 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T14:13:45.674 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:13:45.692 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:13:45.692 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T14:13:45.692 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:13:45.693 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:13:45.693 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:13:45.702 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T14:13:45.702 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T14:13:45.702 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-25T14:13:45.702 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:13:45.703 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:13:45.703 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790" 2024-04-25T14:13:45.703 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790:/ 2024-04-25T14:13:45.703 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb" will pass to kernel 2024-04-25T14:13:45.706 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T14:13:45.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:45.706 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T14:13:45.731 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:13:45.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:13:45.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:13:45.837 INFO:teuthology.orchestra.run.smithi086.stdout:client9880 2024-04-25T14:13:45.838 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:13:45.838 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:13:45.847 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T14:13:46.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.038+0000 7f1f6f600640 1 Processor -- start 2024-04-25T14:13:46.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.038+0000 7f1f6f600640 1 -- start start 2024-04-25T14:13:46.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.038+0000 7f1f6f600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f680cdd70 0x7f1f680ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:46.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.038+0000 7f1f6f600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1f6806e700 0x7f1f6806eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:46.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.038+0000 7f1f6f600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f6806f050 con 0x7f1f68070e20 2024-04-25T14:13:46.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.038+0000 7f1f6f600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1f6806d940 con 0x7f1f6806e700 2024-04-25T14:13:46.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.038+0000 7f1f6f600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1f6806dc20 con 0x7f1f680cdd70 2024-04-25T14:13:46.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.038+0000 7f1f6d600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1f6806e700 0x7f1f6806eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:46.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.038+0000 7f1f6d600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1f6806e700 0x7f1f6806eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41764/0 (socket says 172.21.15.86:41764) 2024-04-25T14:13:46.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.038+0000 7f1f6d600640 1 -- 172.21.15.86:0/270730626 learned_addr learned my addr 172.21.15.86:0/270730626 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:46.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.038+0000 7f1f6cc00640 1 --2- 172.21.15.86:0/270730626 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f680cdd70 0x7f1f680ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.038+0000 7f1f6d600640 1 -- 172.21.15.86:0/270730626 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f680cdd70 msgr2=0x7f1f680ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.038+0000 7f1f6d600640 1 --2- 172.21.15.86:0/270730626 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f680cdd70 0x7f1f680ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.038+0000 7f1f6d600640 1 -- 172.21.15.86:0/270730626 >> v1:172.21.15.155:6789/0 conn(0x7f1f68070e20 legacy=0x7f1f68071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.038+0000 7f1f6d600640 1 -- 172.21.15.86:0/270730626 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6806df00 con 0x7f1f6806e700 2024-04-25T14:13:46.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.042+0000 7f1f6d600640 1 --2- 172.21.15.86:0/270730626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f6806e700 0x7f1f6806eb10 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009e50 tx=0x7f1f6001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f94d51baa4d75f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:46.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.042+0000 7f1f67400640 1 -- 172.21.15.86:0/270730626 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f600a56b0 con 0x7f1f6806e700 2024-04-25T14:13:46.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.042+0000 7f1f67400640 1 -- 172.21.15.86:0/270730626 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f600a5850 con 0x7f1f6806e700 2024-04-25T14:13:46.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.042+0000 7f1f67400640 1 -- 172.21.15.86:0/270730626 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f600a5c20 con 0x7f1f6806e700 2024-04-25T14:13:46.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.042+0000 7f1f6f600640 1 -- 172.21.15.86:0/270730626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f6806e700 msgr2=0x7f1f6806eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:46.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.042+0000 7f1f6f600640 1 --2- 172.21.15.86:0/270730626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f6806e700 0x7f1f6806eb10 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009e50 tx=0x7f1f6001c9a0 comp rx=0 tx=0).stop 2024-04-25T14:13:46.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.042+0000 7f1f6f600640 1 -- 172.21.15.86:0/270730626 shutdown_connections 2024-04-25T14:13:46.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.042+0000 7f1f6f600640 1 --2- 172.21.15.86:0/270730626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f6806e700 0x7f1f6806eb10 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:46.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.042+0000 7f1f6f600640 1 --2- 172.21.15.86:0/270730626 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f680cdd70 0x7f1f680ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:46.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.042+0000 7f1f6f600640 1 -- 172.21.15.86:0/270730626 >> 172.21.15.86:0/270730626 conn(0x7f1f680cffc0 msgr2=0x7f1f680d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:46.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.042+0000 7f1f6f600640 1 -- 172.21.15.86:0/270730626 shutdown_connections 2024-04-25T14:13:46.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.042+0000 7f1f6f600640 1 -- 172.21.15.86:0/270730626 wait complete. 2024-04-25T14:13:46.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.046+0000 7f1f6f600640 1 Processor -- start 2024-04-25T14:13:46.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.046+0000 7f1f6f600640 1 -- start start 2024-04-25T14:13:46.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.046+0000 7f1f6f600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f6806e700 0x7f1f681526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:46.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.046+0000 7f1f6f600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f68070e20 0x7f1f68159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:46.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.046+0000 7f1f6f600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f680cdd70 0x7f1f6815a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:46.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.046+0000 7f1f6f600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6816cc00 con 0x7f1f680cdd70 2024-04-25T14:13:46.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.046+0000 7f1f6f600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1f68164400 con 0x7f1f6806e700 2024-04-25T14:13:46.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.046+0000 7f1f6f600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1f6816cd80 con 0x7f1f68070e20 2024-04-25T14:13:46.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.046+0000 7f1f6d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f680cdd70 0x7f1f6815a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:46.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.046+0000 7f1f6d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f680cdd70 0x7f1f6815a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41770/0 (socket says 172.21.15.86:41770) 2024-04-25T14:13:46.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.046+0000 7f1f6d600640 1 -- 172.21.15.86:0/511414374 learned_addr learned my addr 172.21.15.86:0/511414374 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:46.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.050+0000 7f1f67e00640 1 --2- 172.21.15.86:0/511414374 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f68070e20 0x7f1f68159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:46.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.050+0000 7f1f6cc00640 1 --2- 172.21.15.86:0/511414374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f6806e700 0x7f1f681526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:46.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.050+0000 7f1f6d600640 1 -- 172.21.15.86:0/511414374 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f68070e20 msgr2=0x7f1f68159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:46.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.050+0000 7f1f6d600640 1 --2- 172.21.15.86:0/511414374 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f68070e20 0x7f1f68159ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:46.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.050+0000 7f1f6d600640 1 -- 172.21.15.86:0/511414374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f6806e700 msgr2=0x7f1f681526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:46.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.050+0000 7f1f6d600640 1 --2- 172.21.15.86:0/511414374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f6806e700 0x7f1f681526f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:46.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.050+0000 7f1f6d600640 1 -- 172.21.15.86:0/511414374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f681605a0 con 0x7f1f680cdd70 2024-04-25T14:13:46.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.050+0000 7f1f67e00640 1 --2- 172.21.15.86:0/511414374 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f68070e20 0x7f1f68159ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:46.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.050+0000 7f1f6cc00640 1 --2- 172.21.15.86:0/511414374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f6806e700 0x7f1f681526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:46.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.050+0000 7f1f6d600640 1 --2- 172.21.15.86:0/511414374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f680cdd70 0x7f1f6815a3e0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f1f6001a5d0 tx=0x7f1f60019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:46.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.050+0000 7f1f64c00640 1 -- 172.21.15.86:0/511414374 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f60009510 con 0x7f1f680cdd70 2024-04-25T14:13:46.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.050+0000 7f1f64c00640 1 -- 172.21.15.86:0/511414374 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f600be070 con 0x7f1f680cdd70 2024-04-25T14:13:46.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.050+0000 7f1f6f600640 1 -- 172.21.15.86:0/511414374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f68160880 con 0x7f1f680cdd70 2024-04-25T14:13:46.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.050+0000 7f1f64c00640 1 -- 172.21.15.86:0/511414374 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f600b88b0 con 0x7f1f680cdd70 2024-04-25T14:13:46.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.050+0000 7f1f6f600640 1 -- 172.21.15.86:0/511414374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f680b7460 con 0x7f1f680cdd70 2024-04-25T14:13:46.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.050+0000 7f1f6f600640 1 -- 172.21.15.86:0/511414374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f680720e0 con 0x7f1f680cdd70 2024-04-25T14:13:46.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.050+0000 7f1f64c00640 1 -- 172.21.15.86:0/511414374 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7f1f600b8a50 con 0x7f1f680cdd70 2024-04-25T14:13:46.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.054+0000 7f1f64c00640 1 --2- 172.21.15.86:0/511414374 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f1f4806f550 0x7f1f48071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:46.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.054+0000 7f1f64c00640 1 -- 172.21.15.86:0/511414374 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6101+0+0 (secure 0 0 0) 0x7f1f60129df0 con 0x7f1f680cdd70 2024-04-25T14:13:46.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.054+0000 7f1f6cc00640 1 --2- 172.21.15.86:0/511414374 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f1f4806f550 0x7f1f48071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:46.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.054+0000 7f1f6cc00640 1 --2- 172.21.15.86:0/511414374 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f1f4806f550 0x7f1f48071a60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00b9f0 tx=0x7f1f5c008040 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.058+0000 7f1f64c00640 1 -- 172.21.15.86:0/511414374 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1f60016620 con 0x7f1f680cdd70 2024-04-25T14:13:46.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.250+0000 7f1f6f600640 1 -- 172.21.15.86:0/511414374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1f68153160 con 0x7f1f680cdd70 2024-04-25T14:13:46.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.250+0000 7f1f64c00640 1 -- 172.21.15.86:0/511414374 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v117) ==== 53+0+73 (secure 0 0 0) 0x7f1f600f6a40 con 0x7f1f680cdd70 2024-04-25T14:13:46.250 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T14:13:46.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.254+0000 7f1f6f600640 1 -- 172.21.15.86:0/511414374 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f1f4806f550 msgr2=0x7f1f48071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:46.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.254+0000 7f1f6f600640 1 --2- 172.21.15.86:0/511414374 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f1f4806f550 0x7f1f48071a60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00b9f0 tx=0x7f1f5c008040 comp rx=0 tx=0).stop 2024-04-25T14:13:46.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.254+0000 7f1f6f600640 1 -- 172.21.15.86:0/511414374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f680cdd70 msgr2=0x7f1f6815a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:46.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.254+0000 7f1f6f600640 1 --2- 172.21.15.86:0/511414374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f680cdd70 0x7f1f6815a3e0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f1f6001a5d0 tx=0x7f1f60019740 comp rx=0 tx=0).stop 2024-04-25T14:13:46.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.254+0000 7f1f6f600640 1 -- 172.21.15.86:0/511414374 shutdown_connections 2024-04-25T14:13:46.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.254+0000 7f1f6f600640 1 --2- 172.21.15.86:0/511414374 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f1f4806f550 0x7f1f48071a60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:46.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.254+0000 7f1f6f600640 1 --2- 172.21.15.86:0/511414374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f680cdd70 0x7f1f6815a3e0 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:46.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.254+0000 7f1f6f600640 1 --2- 172.21.15.86:0/511414374 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f68070e20 0x7f1f68159ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:46.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.254+0000 7f1f6f600640 1 --2- 172.21.15.86:0/511414374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f6806e700 0x7f1f681526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:46.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.254+0000 7f1f6f600640 1 -- 172.21.15.86:0/511414374 >> 172.21.15.86:0/511414374 conn(0x7f1f680cffc0 msgr2=0x7f1f68156150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:46.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.254+0000 7f1f6f600640 1 -- 172.21.15.86:0/511414374 shutdown_connections 2024-04-25T14:13:46.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:46.254+0000 7f1f6f600640 1 -- 172.21.15.86:0/511414374 wait complete. 2024-04-25T14:13:46.272 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T14:13:46.273 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T14:13:46.273 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2024-04-25T14:13:46.273 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T14:13:46.273 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T14:13:46.273 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T14:13:46.273 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T14:13:46.273 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T14:13:46.273 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:13:46.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:46.274 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2024-04-25T14:13:46.284 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T14:13:46.284 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T14:13:46.285 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T14:13:46.285 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:13:46.285 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2024-04-25T14:13:46.285 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:13:46.285 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:13:46.285 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2024-04-25T14:13:46.285 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f0 2024-04-25T14:13:46.285 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:13:46.285 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2024-04-25T14:13:46.285 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f1 2024-04-25T14:13:46.285 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T14:13:46.285 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2024-04-25T14:13:46.285 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T14:13:46.286 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2024-04-25T14:13:46.286 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T14:13:46.286 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 41831sec preferred_lft 41831sec 2024-04-25T14:13:46.286 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::ec4:7aff:fe88:6ded/64 scope link 2024-04-25T14:13:46.286 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:13:46.286 INFO:teuthology.orchestra.run.smithi155.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T14:13:46.286 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 02:42:b1:ff:60:50 brd ff:ff:ff:ff:ff:ff 2024-04-25T14:13:46.286 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T14:13:46.286 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:13:46.286 INFO:teuthology.orchestra.run.smithi155.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T14:13:46.286 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T14:13:46.286 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T14:13:46.287 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:13:46.287 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T14:13:46.287 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:13:46.287 INFO:teuthology.orchestra.run.smithi155.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:13:46.287 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T14:13:46.287 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T14:13:46.287 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:13:46.287 INFO:teuthology.orchestra.run.smithi155.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T14:13:46.287 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T14:13:46.287 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T14:13:46.287 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-25T14:13:46.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:46.288 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T14:13:46.351 INFO:teuthology.orchestra.run.smithi155.stdout:1 2024-04-25T14:13:46.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:46.353 DEBUG:teuthology.orchestra.run.smithi155:> route 2024-04-25T14:13:46.413 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2024-04-25T14:13:46.413 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T14:13:46.413 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T14:13:46.413 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T14:13:46.413 INFO:teuthology.orchestra.run.smithi155.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T14:13:46.414 INFO:teuthology.orchestra.run.smithi155.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T14:13:46.414 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T14:13:46.414 INFO:teuthology.orchestra.run.smithi155.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:13:46.414 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:13:46.414 INFO:teuthology.orchestra.run.smithi155.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T14:13:46.414 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T14:13:46.415 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T14:13:46.415 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-04-25T14:13:46.415 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T14:13:46.415 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T14:13:46.415 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T14:13:46.415 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-04-25T14:13:46.507 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:13:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T14:13:46.551 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:13:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:13:46.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:46.558 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2024-04-25T14:13:46.567 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T14:13:46.567 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T14:13:46.567 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:13:46.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:46.568 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.0 up 2024-04-25T14:13:46.631 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:13:46.631 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T14:13:46.683 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T14:13:46.683 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T14:13:46.683 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T14:13:46.735 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T14:13:46.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:46.735 DEBUG:teuthology.orchestra.run.smithi155:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:13:46.843 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:13:46.843 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T14:13:46.843 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:13:46.861 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:13:46.861 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T14:13:46.861 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:13:46.861 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:13:46.861 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:13:46.872 INFO:teuthology.orchestra.run.smithi155.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T14:13:46.872 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T14:13:46.872 INFO:teuthology.orchestra.run.smithi155.stdout:invalid new device string format 2024-04-25T14:13:46.872 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.155:3300,172.21.15.155:3301" 2024-04-25T14:13:46.872 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T14:13:46.873 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: resolved to: "172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790" 2024-04-25T14:13:46.873 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:6789,172.21.15.155:6789,172.21.15.155:6790:/ 2024-04-25T14:13:46.873 INFO:teuthology.orchestra.run.smithi155.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb" will pass to kernel 2024-04-25T14:13:46.876 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T14:13:46.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:13:46.876 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T14:13:46.896 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T14:13:46.986 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:13:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T14:13:46.989 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-25T14:13:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T14:13:46.995 INFO:teuthology.orchestra.run.smithi155.stdout:client8492 2024-04-25T14:13:46.995 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:13:46.996 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:13:47.004 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:13:47.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.158+0000 7f925fe00640 1 Processor -- start 2024-04-25T14:13:47.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.158+0000 7f925fe00640 1 -- start start 2024-04-25T14:13:47.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.162+0000 7f925fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f92580cfca0 0x7f925806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:47.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.162+0000 7f925fe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f92580d16e0 0x7f92580d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:47.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.162+0000 7f925fe00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92580d2030 con 0x7f925806c670 2024-04-25T14:13:47.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.162+0000 7f925fe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9258070470 con 0x7f92580cfca0 2024-04-25T14:13:47.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.162+0000 7f925fe00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9258072bd0 con 0x7f92580d16e0 2024-04-25T14:13:47.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.162+0000 7f925d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f92580cfca0 0x7f925806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:47.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.162+0000 7f925d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f92580cfca0 0x7f925806f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41778/0 (socket says 172.21.15.86:41778) 2024-04-25T14:13:47.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.162+0000 7f925d400640 1 -- 172.21.15.86:0/259257896 learned_addr learned my addr 172.21.15.86:0/259257896 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:47.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.162+0000 7f925ca00640 1 --2- 172.21.15.86:0/259257896 >> v2:172.21.15.155:3300/0 conn(0x7f92580d16e0 0x7f92580d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:47.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.162+0000 7f925d400640 1 -- 172.21.15.86:0/259257896 >> v2:172.21.15.155:3300/0 conn(0x7f92580d16e0 msgr2=0x7f92580d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:47.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.162+0000 7f925d400640 1 --2- 172.21.15.86:0/259257896 >> v2:172.21.15.155:3300/0 conn(0x7f92580d16e0 0x7f92580d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:47.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.162+0000 7f925d400640 1 -- 172.21.15.86:0/259257896 >> v1:172.21.15.155:6789/0 conn(0x7f925806c670 legacy=0x7f925806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:47.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.162+0000 7f925d400640 1 -- 172.21.15.86:0/259257896 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925806f930 con 0x7f92580cfca0 2024-04-25T14:13:47.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.162+0000 7f925d400640 1 --2- 172.21.15.86:0/259257896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92580cfca0 0x7f925806f3f0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f92580729c0 tx=0x7f924c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe740def70b16cdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:47.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.162+0000 7f9257e00640 1 -- 172.21.15.86:0/259257896 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c0a57d0 con 0x7f92580cfca0 2024-04-25T14:13:47.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.166+0000 7f9257e00640 1 -- 172.21.15.86:0/259257896 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f924c0a5970 con 0x7f92580cfca0 2024-04-25T14:13:47.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.166+0000 7f9257e00640 1 -- 172.21.15.86:0/259257896 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c0a5d40 con 0x7f92580cfca0 2024-04-25T14:13:47.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.166+0000 7f925fe00640 1 -- 172.21.15.86:0/259257896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92580cfca0 msgr2=0x7f925806f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:47.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.166+0000 7f925fe00640 1 --2- 172.21.15.86:0/259257896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92580cfca0 0x7f925806f3f0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f92580729c0 tx=0x7f924c01c9f0 comp rx=0 tx=0).stop 2024-04-25T14:13:47.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.166+0000 7f925fe00640 1 -- 172.21.15.86:0/259257896 shutdown_connections 2024-04-25T14:13:47.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.166+0000 7f925fe00640 1 --2- 172.21.15.86:0/259257896 >> v2:172.21.15.155:3300/0 conn(0x7f92580d16e0 0x7f92580d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:47.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.166+0000 7f925fe00640 1 --2- 172.21.15.86:0/259257896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92580cfca0 0x7f925806f3f0 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:47.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.166+0000 7f925fe00640 1 -- 172.21.15.86:0/259257896 >> 172.21.15.86:0/259257896 conn(0x7f92580b4650 msgr2=0x7f92580b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:47.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.166+0000 7f925fe00640 1 -- 172.21.15.86:0/259257896 shutdown_connections 2024-04-25T14:13:47.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.166+0000 7f925fe00640 1 -- 172.21.15.86:0/259257896 wait complete. 2024-04-25T14:13:47.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.166+0000 7f925fe00640 1 Processor -- start 2024-04-25T14:13:47.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.166+0000 7f925fe00640 1 -- start start 2024-04-25T14:13:47.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.170+0000 7f925fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f925806c670 0x7f9258099af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:47.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.170+0000 7f925fe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92580cfca0 0x7f92581fce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:47.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.170+0000 7f925fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92580d16e0 0x7f92581fd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:47.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.170+0000 7f925fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f925816a070 con 0x7f92580d16e0 2024-04-25T14:13:47.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.170+0000 7f925fe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9258169ef0 con 0x7f925806c670 2024-04-25T14:13:47.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.170+0000 7f925fe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f925816a1f0 con 0x7f92580cfca0 2024-04-25T14:13:47.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.170+0000 7f925d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f925806c670 0x7f9258099af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:47.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.170+0000 7f925de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92580d16e0 0x7f92581fd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:47.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.170+0000 7f925ca00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92580cfca0 0x7f92581fce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:47.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.170+0000 7f925d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f925806c670 0x7f9258099af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:48778/0 (socket says 172.21.15.86:48778) 2024-04-25T14:13:47.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.170+0000 7f925d400640 1 -- 172.21.15.86:0/3232079893 learned_addr learned my addr 172.21.15.86:0/3232079893 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:47.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.170+0000 7f925de00640 1 -- 172.21.15.86:0/3232079893 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92580cfca0 msgr2=0x7f92581fce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:47.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.170+0000 7f925de00640 1 --2- 172.21.15.86:0/3232079893 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92580cfca0 0x7f92581fce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:47.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.170+0000 7f925de00640 1 -- 172.21.15.86:0/3232079893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f925806c670 msgr2=0x7f9258099af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:47.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.170+0000 7f925de00640 1 --2- 172.21.15.86:0/3232079893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f925806c670 0x7f9258099af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:47.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.170+0000 7f925de00640 1 -- 172.21.15.86:0/3232079893 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92581fdd10 con 0x7f92580d16e0 2024-04-25T14:13:47.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.170+0000 7f925d400640 1 --2- 172.21.15.86:0/3232079893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f925806c670 0x7f9258099af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:13:47.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.170+0000 7f925de00640 1 --2- 172.21.15.86:0/3232079893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92580d16e0 0x7f92581fd3a0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f925806dd30 tx=0x7f925000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:47.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.170+0000 7f9255600640 1 -- 172.21.15.86:0/3232079893 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9250017070 con 0x7f92580d16e0 2024-04-25T14:13:47.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.170+0000 7f9255600640 1 -- 172.21.15.86:0/3232079893 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92500052d0 con 0x7f92580d16e0 2024-04-25T14:13:47.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.170+0000 7f9255600640 1 -- 172.21.15.86:0/3232079893 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9250003840 con 0x7f92580d16e0 2024-04-25T14:13:47.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.170+0000 7f925fe00640 1 -- 172.21.15.86:0/3232079893 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92581fda30 con 0x7f92580d16e0 2024-04-25T14:13:47.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.170+0000 7f925fe00640 1 -- 172.21.15.86:0/3232079893 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92580af840 con 0x7f92580d16e0 2024-04-25T14:13:47.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.170+0000 7f925fe00640 1 -- 172.21.15.86:0/3232079893 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f925809f4b0 con 0x7f92580d16e0 2024-04-25T14:13:47.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.174+0000 7f9255600640 1 -- 172.21.15.86:0/3232079893 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7f925001a070 con 0x7f92580d16e0 2024-04-25T14:13:47.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.174+0000 7f9255600640 1 --2- 172.21.15.86:0/3232079893 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f923806f550 0x7f9238071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:47.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.174+0000 7f9255600640 1 -- 172.21.15.86:0/3232079893 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6101+0+0 (secure 0 0 0) 0x7f925008e660 con 0x7f92580d16e0 2024-04-25T14:13:47.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.174+0000 7f925d400640 1 --2- 172.21.15.86:0/3232079893 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f923806f550 0x7f9238071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:47.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.174+0000 7f925d400640 1 --2- 172.21.15.86:0/3232079893 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f923806f550 0x7f9238071a60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f924c000f90 tx=0x7f924c0a7070 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:47.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.178+0000 7f9255600640 1 -- 172.21.15.86:0/3232079893 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f92580d2030 con 0x7f92580d16e0 2024-04-25T14:13:47.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.366+0000 7f925fe00640 1 -- 172.21.15.86:0/3232079893 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9258076bf0 con 0x7f92580d16e0 2024-04-25T14:13:47.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.366+0000 7f9255600640 1 -- 172.21.15.86:0/3232079893 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) ==== 95+0+5202 (secure 0 0 0) 0x7f925008ce20 con 0x7f92580d16e0 2024-04-25T14:13:47.368 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:13:47.368 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":117,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112},{"gid":8387,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:13:36.638299+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10209},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:13:47.368 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 117 2024-04-25T14:13:47.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.370+0000 7f925fe00640 1 -- 172.21.15.86:0/3232079893 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f923806f550 msgr2=0x7f9238071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:47.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.370+0000 7f925fe00640 1 --2- 172.21.15.86:0/3232079893 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f923806f550 0x7f9238071a60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f924c000f90 tx=0x7f924c0a7070 comp rx=0 tx=0).stop 2024-04-25T14:13:47.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.370+0000 7f925fe00640 1 -- 172.21.15.86:0/3232079893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92580d16e0 msgr2=0x7f92581fd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:47.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.370+0000 7f925fe00640 1 --2- 172.21.15.86:0/3232079893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92580d16e0 0x7f92581fd3a0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f925806dd30 tx=0x7f925000c820 comp rx=0 tx=0).stop 2024-04-25T14:13:47.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.370+0000 7f925fe00640 1 -- 172.21.15.86:0/3232079893 shutdown_connections 2024-04-25T14:13:47.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.370+0000 7f925fe00640 1 --2- 172.21.15.86:0/3232079893 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f923806f550 0x7f9238071a60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:47.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.370+0000 7f925fe00640 1 --2- 172.21.15.86:0/3232079893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92580d16e0 0x7f92581fd3a0 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:47.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.370+0000 7f925fe00640 1 --2- 172.21.15.86:0/3232079893 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92580cfca0 0x7f92581fce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:47.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.370+0000 7f925fe00640 1 --2- 172.21.15.86:0/3232079893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f925806c670 0x7f9258099af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:47.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.370+0000 7f925fe00640 1 -- 172.21.15.86:0/3232079893 >> 172.21.15.86:0/3232079893 conn(0x7f92580b4650 msgr2=0x7f9258068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:47.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.370+0000 7f925fe00640 1 -- 172.21.15.86:0/3232079893 shutdown_connections 2024-04-25T14:13:47.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.370+0000 7f925fe00640 1 -- 172.21.15.86:0/3232079893 wait complete. 2024-04-25T14:13:47.390 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable stats 2024-04-25T14:13:47.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.538+0000 7f585d800640 1 Processor -- start 2024-04-25T14:13:47.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.538+0000 7f585d800640 1 -- start start 2024-04-25T14:13:47.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.538+0000 7f585d800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5858070330 0x7f58580d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:47.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.538+0000 7f585d800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f58580ce790 0x7f585806f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:47.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.538+0000 7f585d800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58580d2030 con 0x7f58580d1c20 2024-04-25T14:13:47.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.538+0000 7f585d800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f585815e5c0 con 0x7f58580ce790 2024-04-25T14:13:47.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.538+0000 7f585d800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f585815e8a0 con 0x7f5858070330 2024-04-25T14:13:47.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.542+0000 7f5857400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f58580ce790 0x7f585806f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:47.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.542+0000 7f5857400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f58580ce790 0x7f585806f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41792/0 (socket says 172.21.15.86:41792) 2024-04-25T14:13:47.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.542+0000 7f5857400640 1 -- 172.21.15.86:0/3529744125 learned_addr learned my addr 172.21.15.86:0/3529744125 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:47.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.542+0000 7f5856a00640 1 --2- 172.21.15.86:0/3529744125 >> v2:172.21.15.155:3300/0 conn(0x7f5858070330 0x7f58580d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:47.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.542+0000 7f5857400640 1 -- 172.21.15.86:0/3529744125 >> v2:172.21.15.155:3300/0 conn(0x7f5858070330 msgr2=0x7f58580d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:47.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.542+0000 7f5857400640 1 --2- 172.21.15.86:0/3529744125 >> v2:172.21.15.155:3300/0 conn(0x7f5858070330 0x7f58580d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:47.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.542+0000 7f5857400640 1 -- 172.21.15.86:0/3529744125 >> v1:172.21.15.155:6789/0 conn(0x7f58580d1c20 legacy=0x7f585806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:47.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.542+0000 7f5857400640 1 -- 172.21.15.86:0/3529744125 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585815eb80 con 0x7f58580ce790 2024-04-25T14:13:47.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.542+0000 7f5856a00640 1 --2- 172.21.15.86:0/3529744125 >> v2:172.21.15.155:3300/0 conn(0x7f5858070330 0x7f58580d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:47.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.542+0000 7f5857400640 1 --2- 172.21.15.86:0/3529744125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58580ce790 0x7f585806f440 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f5850009e50 tx=0x7f585001c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c77e6b795f57b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:47.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.542+0000 7f5856000640 1 -- 172.21.15.86:0/3529744125 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58500ac020 con 0x7f58580ce790 2024-04-25T14:13:47.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.542+0000 7f5856000640 1 -- 172.21.15.86:0/3529744125 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58500a6410 con 0x7f58580ce790 2024-04-25T14:13:47.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.542+0000 7f5856000640 1 -- 172.21.15.86:0/3529744125 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58500a67e0 con 0x7f58580ce790 2024-04-25T14:13:47.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.546+0000 7f585d800640 1 -- 172.21.15.86:0/3529744125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58580ce790 msgr2=0x7f585806f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:47.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.546+0000 7f585d800640 1 --2- 172.21.15.86:0/3529744125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58580ce790 0x7f585806f440 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f5850009e50 tx=0x7f585001c740 comp rx=0 tx=0).stop 2024-04-25T14:13:47.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.546+0000 7f585d800640 1 -- 172.21.15.86:0/3529744125 shutdown_connections 2024-04-25T14:13:47.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.546+0000 7f585d800640 1 --2- 172.21.15.86:0/3529744125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58580ce790 0x7f585806f440 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:47.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.546+0000 7f585d800640 1 --2- 172.21.15.86:0/3529744125 >> v2:172.21.15.155:3300/0 conn(0x7f5858070330 0x7f58580d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:47.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.546+0000 7f585d800640 1 -- 172.21.15.86:0/3529744125 >> 172.21.15.86:0/3529744125 conn(0x7f58580b4650 msgr2=0x7f58580b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:47.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.546+0000 7f585d800640 1 -- 172.21.15.86:0/3529744125 shutdown_connections 2024-04-25T14:13:47.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.546+0000 7f585d800640 1 -- 172.21.15.86:0/3529744125 wait complete. 2024-04-25T14:13:47.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.546+0000 7f585d800640 1 Processor -- start 2024-04-25T14:13:47.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.546+0000 7f585d800640 1 -- start start 2024-04-25T14:13:47.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.550+0000 7f585d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5858070330 0x7f58580c3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:47.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.550+0000 7f585d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58580ce790 0x7f5858158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:47.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.550+0000 7f585d800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58580d1c20 0x7f58580c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:47.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.550+0000 7f585d800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f58580708c0 con 0x7f5858070330 2024-04-25T14:13:47.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.550+0000 7f585d800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5858070740 con 0x7f58580ce790 2024-04-25T14:13:47.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.550+0000 7f585d800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f585816afc0 con 0x7f58580d1c20 2024-04-25T14:13:47.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.550+0000 7f5856a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5858070330 0x7f58580c3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:47.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.550+0000 7f5856a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5858070330 0x7f58580c3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41804/0 (socket says 172.21.15.86:41804) 2024-04-25T14:13:47.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.550+0000 7f5856a00640 1 -- 172.21.15.86:0/698877074 learned_addr learned my addr 172.21.15.86:0/698877074 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:47.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.550+0000 7f584e000640 1 --2- 172.21.15.86:0/698877074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58580ce790 0x7f5858158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:47.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.550+0000 7f5857400640 1 --2- 172.21.15.86:0/698877074 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58580d1c20 0x7f58580c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:47.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.550+0000 7f5856a00640 1 -- 172.21.15.86:0/698877074 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58580d1c20 msgr2=0x7f58580c83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:47.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.550+0000 7f5856a00640 1 --2- 172.21.15.86:0/698877074 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58580d1c20 0x7f58580c83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:47.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.550+0000 7f5856a00640 1 -- 172.21.15.86:0/698877074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58580ce790 msgr2=0x7f5858158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:47.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.550+0000 7f5856a00640 1 --2- 172.21.15.86:0/698877074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58580ce790 0x7f5858158940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:47.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.550+0000 7f5856a00640 1 -- 172.21.15.86:0/698877074 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58580c97a0 con 0x7f5858070330 2024-04-25T14:13:47.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.550+0000 7f5857400640 1 --2- 172.21.15.86:0/698877074 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58580d1c20 0x7f58580c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:47.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.550+0000 7f584e000640 1 --2- 172.21.15.86:0/698877074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58580ce790 0x7f5858158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:47.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.550+0000 7f5856a00640 1 --2- 172.21.15.86:0/698877074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5858070330 0x7f58580c3140 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f584800c820 tx=0x7f584800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:47.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.550+0000 7f584f400640 1 -- 172.21.15.86:0/698877074 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5848015070 con 0x7f5858070330 2024-04-25T14:13:47.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.550+0000 7f584f400640 1 -- 172.21.15.86:0/698877074 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5848007a30 con 0x7f5858070330 2024-04-25T14:13:47.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.550+0000 7f584f400640 1 -- 172.21.15.86:0/698877074 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5848007de0 con 0x7f5858070330 2024-04-25T14:13:47.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.550+0000 7f585d800640 1 -- 172.21.15.86:0/698877074 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58580c9a80 con 0x7f5858070330 2024-04-25T14:13:47.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.550+0000 7f585d800640 1 -- 172.21.15.86:0/698877074 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58580ca040 con 0x7f5858070330 2024-04-25T14:13:47.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.550+0000 7f585d800640 1 -- 172.21.15.86:0/698877074 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5858069d00 con 0x7f5858070330 2024-04-25T14:13:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.554+0000 7f584f400640 1 -- 172.21.15.86:0/698877074 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 78) ==== 93309+0+0 (secure 0 0 0) 0x7f5848018070 con 0x7f5858070330 2024-04-25T14:13:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.554+0000 7f584f400640 1 --2- 172.21.15.86:0/698877074 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f583406f550 0x7f5834071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.554+0000 7f584f400640 1 -- 172.21.15.86:0/698877074 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6101+0+0 (secure 0 0 0) 0x7f584808f690 con 0x7f5858070330 2024-04-25T14:13:47.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.554+0000 7f584e000640 1 --2- 172.21.15.86:0/698877074 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f583406f550 0x7f5834071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:47.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.554+0000 7f584e000640 1 --2- 172.21.15.86:0/698877074 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f583406f550 0x7f5834071a60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f58580c49d0 tx=0x7f5840008040 comp rx=0 tx=0).ready entity=mgr.8198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:47.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.558+0000 7f584f400640 1 -- 172.21.15.86:0/698877074 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f584805c130 con 0x7f5858070330 2024-04-25T14:13:47.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.750+0000 7f585d800640 1 -- 172.21.15.86:0/698877074 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "stats"} v 0) -- 0x7f5858076bf0 con 0x7f5858070330 2024-04-25T14:13:47.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.790+0000 7f584f400640 1 -- 172.21.15.86:0/698877074 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "stats"}]=0 v79) ==== 84+0+0 (secure 0 0 0) 0x7f584805ba30 con 0x7f5858070330 2024-04-25T14:13:47.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.794+0000 7f585d800640 1 -- 172.21.15.86:0/698877074 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f583406f550 msgr2=0x7f5834071a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:47.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.794+0000 7f585d800640 1 --2- 172.21.15.86:0/698877074 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f583406f550 0x7f5834071a60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f58580c49d0 tx=0x7f5840008040 comp rx=0 tx=0).stop 2024-04-25T14:13:47.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.794+0000 7f585d800640 1 -- 172.21.15.86:0/698877074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5858070330 msgr2=0x7f58580c3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:47.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.794+0000 7f585d800640 1 --2- 172.21.15.86:0/698877074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5858070330 0x7f58580c3140 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f584800c820 tx=0x7f584800cce0 comp rx=0 tx=0).stop 2024-04-25T14:13:47.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.794+0000 7f585d800640 1 -- 172.21.15.86:0/698877074 shutdown_connections 2024-04-25T14:13:47.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.794+0000 7f585d800640 1 --2- 172.21.15.86:0/698877074 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7f583406f550 0x7f5834071a60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:47.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.794+0000 7f585d800640 1 --2- 172.21.15.86:0/698877074 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58580d1c20 0x7f58580c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:47.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.794+0000 7f585d800640 1 --2- 172.21.15.86:0/698877074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58580ce790 0x7f5858158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:47.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.794+0000 7f585d800640 1 --2- 172.21.15.86:0/698877074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5858070330 0x7f58580c3140 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:47.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.794+0000 7f585d800640 1 -- 172.21.15.86:0/698877074 >> 172.21.15.86:0/698877074 conn(0x7f58580b4650 msgr2=0x7f58580b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:47.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.794+0000 7f585d800640 1 -- 172.21.15.86:0/698877074 shutdown_connections 2024-04-25T14:13:47.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.794+0000 7f585d800640 1 -- 172.21.15.86:0/698877074 wait complete. 2024-04-25T14:13:47.817 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:13:47.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.970+0000 7fc51e000640 1 Processor -- start 2024-04-25T14:13:47.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.970+0000 7fc51e000640 1 -- start start 2024-04-25T14:13:47.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.974+0000 7fc51e000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc5180cec60 0x7fc5180cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:47.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.974+0000 7fc51e000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc5180cf5b0 0x7fc5180ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:47.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.974+0000 7fc51e000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc5180cf9c0 0x7fc5180cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:47.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.974+0000 7fc51e000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc518163940 con 0x7fc5180cec60 2024-04-25T14:13:47.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.974+0000 7fc51e000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc5180d2f40 con 0x7fc5180cf5b0 2024-04-25T14:13:47.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.974+0000 7fc51e000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc5180b3720 con 0x7fc5180cf9c0 2024-04-25T14:13:47.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.974+0000 7fc517400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc5180cec60 0x7fc5180cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:47.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.974+0000 7fc517400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc5180cec60 0x7fc5180cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41816/0 (socket says 172.21.15.86:41816) 2024-04-25T14:13:47.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.974+0000 7fc517400640 1 -- 172.21.15.86:0/1357700425 learned_addr learned my addr 172.21.15.86:0/1357700425 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:47.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.974+0000 7fc516a00640 1 --2- 172.21.15.86:0/1357700425 >> v2:172.21.15.155:3300/0 conn(0x7fc5180cf5b0 0x7fc5180ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:47.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.974+0000 7fc517e00640 1 --2- 172.21.15.86:0/1357700425 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc5180cf9c0 0x7fc5180cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:47.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.974+0000 7fc517400640 1 -- 172.21.15.86:0/1357700425 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc5180cf9c0 msgr2=0x7fc5180cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:47.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.974+0000 7fc517400640 1 --2- 172.21.15.86:0/1357700425 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc5180cf9c0 0x7fc5180cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:47.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.974+0000 7fc517400640 1 -- 172.21.15.86:0/1357700425 >> v2:172.21.15.155:3300/0 conn(0x7fc5180cf5b0 msgr2=0x7fc5180ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:47.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.974+0000 7fc517400640 1 --2- 172.21.15.86:0/1357700425 >> v2:172.21.15.155:3300/0 conn(0x7fc5180cf5b0 0x7fc5180ccd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:47.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.974+0000 7fc517400640 1 -- 172.21.15.86:0/1357700425 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5180b3a00 con 0x7fc5180cec60 2024-04-25T14:13:47.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.974+0000 7fc516a00640 1 --2- 172.21.15.86:0/1357700425 >> v2:172.21.15.155:3300/0 conn(0x7fc5180cf5b0 0x7fc5180ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:13:47.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.978+0000 7fc517400640 1 --2- 172.21.15.86:0/1357700425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5180cec60 0x7fc5180cf070 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7fc5180724b0 tx=0x7fc50801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f22aed91f8a6a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:47.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.978+0000 7fc516000640 1 -- 172.21.15.86:0/1357700425 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5080ab5f0 con 0x7fc5180cec60 2024-04-25T14:13:47.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.978+0000 7fc516000640 1 -- 172.21.15.86:0/1357700425 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5080b0070 con 0x7fc5180cec60 2024-04-25T14:13:47.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.978+0000 7fc516000640 1 -- 172.21.15.86:0/1357700425 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5080ab9b0 con 0x7fc5180cec60 2024-04-25T14:13:47.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.978+0000 7fc51e000640 1 -- 172.21.15.86:0/1357700425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5180cec60 msgr2=0x7fc5180cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:47.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.978+0000 7fc51e000640 1 --2- 172.21.15.86:0/1357700425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5180cec60 0x7fc5180cf070 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7fc5180724b0 tx=0x7fc50801c520 comp rx=0 tx=0).stop 2024-04-25T14:13:47.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.978+0000 7fc51e000640 1 -- 172.21.15.86:0/1357700425 shutdown_connections 2024-04-25T14:13:47.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.978+0000 7fc51e000640 1 --2- 172.21.15.86:0/1357700425 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc5180cf9c0 0x7fc5180cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:47.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.978+0000 7fc51e000640 1 --2- 172.21.15.86:0/1357700425 >> v2:172.21.15.155:3300/0 conn(0x7fc5180cf5b0 0x7fc5180ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:47.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.978+0000 7fc51e000640 1 --2- 172.21.15.86:0/1357700425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5180cec60 0x7fc5180cf070 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:47.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.978+0000 7fc51e000640 1 -- 172.21.15.86:0/1357700425 >> 172.21.15.86:0/1357700425 conn(0x7fc51809ba30 msgr2=0x7fc5180b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:47.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.978+0000 7fc51e000640 1 -- 172.21.15.86:0/1357700425 shutdown_connections 2024-04-25T14:13:47.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.978+0000 7fc51e000640 1 -- 172.21.15.86:0/1357700425 wait complete. 2024-04-25T14:13:47.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.982+0000 7fc51e000640 1 Processor -- start 2024-04-25T14:13:47.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.982+0000 7fc51e000640 1 -- start start 2024-04-25T14:13:47.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.982+0000 7fc51e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5180cec60 0x7fc518154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:47.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.982+0000 7fc51e000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc5180cf5b0 0x7fc5181f9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:47.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.982+0000 7fc51e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5180cf9c0 0x7fc5180b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:47.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.982+0000 7fc51e000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc5180d2480 con 0x7fc5180cf9c0 2024-04-25T14:13:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.982+0000 7fc51e000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc5180d2300 con 0x7fc5180cec60 2024-04-25T14:13:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.982+0000 7fc51e000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc5180d2600 con 0x7fc5180cf5b0 2024-04-25T14:13:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.982+0000 7fc517e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5180cf9c0 0x7fc5180b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.982+0000 7fc517e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5180cf9c0 0x7fc5180b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41826/0 (socket says 172.21.15.86:41826) 2024-04-25T14:13:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.982+0000 7fc517e00640 1 -- 172.21.15.86:0/3886177859 learned_addr learned my addr 172.21.15.86:0/3886177859 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.982+0000 7fc516a00640 1 --2- 172.21.15.86:0/3886177859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc5180cf5b0 0x7fc5181f9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.982+0000 7fc517400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5180cec60 0x7fc518154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.982+0000 7fc517e00640 1 -- 172.21.15.86:0/3886177859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc5180cf5b0 msgr2=0x7fc5181f9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.982+0000 7fc517e00640 1 --2- 172.21.15.86:0/3886177859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc5180cf5b0 0x7fc5181f9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.982+0000 7fc517e00640 1 -- 172.21.15.86:0/3886177859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5180cec60 msgr2=0x7fc518154ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.982+0000 7fc517e00640 1 --2- 172.21.15.86:0/3886177859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5180cec60 0x7fc518154ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.982+0000 7fc517e00640 1 -- 172.21.15.86:0/3886177859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5180b0b70 con 0x7fc5180cf9c0 2024-04-25T14:13:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.982+0000 7fc517400640 1 --2- 172.21.15.86:0/3886177859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5180cec60 0x7fc518154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:13:47.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.982+0000 7fc516a00640 1 --2- 172.21.15.86:0/3886177859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc5180cf5b0 0x7fc5181f9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:13:47.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.986+0000 7fc517e00640 1 --2- 172.21.15.86:0/3886177859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5180cf9c0 0x7fc5180b1930 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7fc51000efc0 tx=0x7fc51000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:47.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.986+0000 7fc50f400640 1 -- 172.21.15.86:0/3886177859 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510018020 con 0x7fc5180cf9c0 2024-04-25T14:13:47.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.986+0000 7fc50f400640 1 -- 172.21.15.86:0/3886177859 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc510014480 con 0x7fc5180cf9c0 2024-04-25T14:13:47.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.986+0000 7fc50f400640 1 -- 172.21.15.86:0/3886177859 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510009e60 con 0x7fc5180cf9c0 2024-04-25T14:13:47.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.986+0000 7fc51e000640 1 -- 172.21.15.86:0/3886177859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5180b0e50 con 0x7fc5180cf9c0 2024-04-25T14:13:47.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.986+0000 7fc51e000640 1 -- 172.21.15.86:0/3886177859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5181fa210 con 0x7fc5180cf9c0 2024-04-25T14:13:47.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.986+0000 7fc51e000640 1 -- 172.21.15.86:0/3886177859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5180cc710 con 0x7fc5180cf9c0 2024-04-25T14:13:47.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.986+0000 7fc50f400640 1 -- 172.21.15.86:0/3886177859 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 79) ==== 93318+0+0 (secure 0 0 0) 0x7fc51001b020 con 0x7fc5180cf9c0 2024-04-25T14:13:47.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.990+0000 7fc50f400640 1 --2- 172.21.15.86:0/3886177859 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fc4f806f570 0x7fc4f8071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:47.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.990+0000 7fc517400640 1 -- 172.21.15.86:0/3886177859 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fc4f806f570 msgr2=0x7fc4f8071a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/505730993 2024-04-25T14:13:47.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.990+0000 7fc50f400640 1 -- 172.21.15.86:0/3886177859 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6101+0+0 (secure 0 0 0) 0x7fc510090330 con 0x7fc5180cf9c0 2024-04-25T14:13:47.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.990+0000 7fc517400640 1 --2- 172.21.15.86:0/3886177859 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fc4f806f570 0x7fc4f8071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:13:47.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:47.994+0000 7fc50f400640 1 -- 172.21.15.86:0/3886177859 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc510022030 con 0x7fc5180cf9c0 2024-04-25T14:13:48.046 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:48.048+0000 7f0052b3a180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:13:48.060 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:48.062+0000 7fee49a57180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:13:48.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:48.178+0000 7fc51e000640 1 -- 172.21.15.86:0/3886177859 --> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7fc518155410 con 0x7fc4f806f570 2024-04-25T14:13:48.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:48.194+0000 7fc517400640 1 -- 172.21.15.86:0/3886177859 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fc4f806f570 msgr2=0x7fc4f8071a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/505730993 2024-04-25T14:13:48.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:48.194+0000 7fc517400640 1 --2- 172.21.15.86:0/3886177859 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fc4f806f570 0x7fc4f8071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:13:48.465 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:48.468+0000 7f0052b3a180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:13:48.480 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:48.482+0000 7fee49a57180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:13:48.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:48.598+0000 7fc517400640 1 -- 172.21.15.86:0/3886177859 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fc4f806f570 msgr2=0x7fc4f8071a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/505730993 2024-04-25T14:13:48.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:48.598+0000 7fc517400640 1 --2- 172.21.15.86:0/3886177859 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fc4f806f570 0x7fc4f8071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T14:13:48.615 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:48.616+0000 7f0052b3a180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:13:48.627 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:48.626+0000 7fee49a57180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:13:49.086 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:49.088+0000 7f0052b3a180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:13:49.094 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:49.094+0000 7fee49a57180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:13:49.145 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:49.148+0000 7f0052b3a180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:13:49.153 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:49.154+0000 7fee49a57180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:13:49.201 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:49.204+0000 7f0052b3a180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:13:49.209 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:49.210+0000 7fee49a57180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:13:49.290 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:49.292+0000 7f0052b3a180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:13:49.300 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:49.302+0000 7fee49a57180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:13:49.346 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:49.348+0000 7f0052b3a180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:13:49.358 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:49.358+0000 7fee49a57180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:13:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:49.402+0000 7fc517400640 1 -- 172.21.15.86:0/3886177859 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fc4f806f570 msgr2=0x7fc4f8071a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/505730993 2024-04-25T14:13:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:49.402+0000 7fc517400640 1 --2- 172.21.15.86:0/3886177859 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fc4f806f570 0x7fc4f8071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T14:13:49.431 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:49.436+0000 7f0052b3a180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:13:49.442 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:49.442+0000 7fee49a57180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:13:49.491 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:49.492+0000 7f0052b3a180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:13:49.501 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:49.502+0000 7fee49a57180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:13:49.643 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:49.644+0000 7f0052b3a180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:13:49.654 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:49.654+0000 7fee49a57180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:13:49.702 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:49.704+0000 7f0052b3a180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:13:49.713 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:49.714+0000 7fee49a57180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:13:49.943 INFO:tasks.ceph.mgr.x.smithi155.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:13:49.943 INFO:tasks.ceph.mgr.x.smithi155.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:13:49.943 INFO:tasks.ceph.mgr.x.smithi155.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:13:49.951 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:49.956+0000 7f0052b3a180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:13:49.954 INFO:tasks.ceph.mgr.y.smithi086.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:13:49.954 INFO:tasks.ceph.mgr.y.smithi086.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:13:49.954 INFO:tasks.ceph.mgr.y.smithi086.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:13:49.963 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:49.962+0000 7fee49a57180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:13:50.112 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:50.110+0000 7fee49a57180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:13:50.168 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:50.172+0000 7f0052b3a180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:13:50.203 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:13:50.206 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:13:50.241 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:50.242+0000 7fee49a57180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:13:50.298 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:50.300+0000 7f0052b3a180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:13:50.314 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:50.314+0000 7fee49a57180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:13:50.369 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:50.372+0000 7f0052b3a180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:13:50.462 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:50.462+0000 7fee49a57180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:13:50.518 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:50.520+0000 7f0052b3a180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:13:51.002 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:51.002+0000 7fee49a57180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:13:51.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:51.006+0000 7fc517400640 1 -- 172.21.15.86:0/3886177859 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fc4f806f570 msgr2=0x7fc4f8071a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/505730993 2024-04-25T14:13:51.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:51.006+0000 7fc517400640 1 --2- 172.21.15.86:0/3886177859 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fc4f806f570 0x7fc4f8071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T14:13:51.063 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:51.068+0000 7f0052b3a180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:13:51.086 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:51.086+0000 7fee49a57180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:13:51.148 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:51.152+0000 7f0052b3a180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:13:51.245 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:51.246+0000 7fee49a57180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:13:51.308 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:51.312+0000 7f0052b3a180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:13:51.329 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:51.330+0000 7fee49a57180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:13:51.392 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:51.396+0000 7f0052b3a180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:13:51.646 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:51.646+0000 7fee49a57180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:13:51.708 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:51.710+0000 7fee49a57180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:13:51.710 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:51.712+0000 7f0052b3a180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:13:51.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:51.718+0000 7fc50f400640 1 -- 172.21.15.86:0/3886177859 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 80) ==== 92927+0+0 (secure 0 0 0) 0x7fc510012050 con 0x7fc5180cf9c0 2024-04-25T14:13:51.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:51.718+0000 7fc50f400640 1 -- 172.21.15.86:0/3886177859 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fc4f806f570 msgr2=0x7fc4f8071a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:13:51.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:51.718+0000 7fc50f400640 1 --2- 172.21.15.86:0/3886177859 >> [v2:172.21.15.86:6832/505730993,v1:172.21.15.86:6833/505730993] conn(0x7fc4f806f570 0x7fc4f8071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:51.782 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:13:51.784+0000 7f0052b3a180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:13:51.786 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:51.786+0000 7fedc9600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:13:51.786 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:51.786+0000 7fedc9600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:13:51.786 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:51.786+0000 7fedc9600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:13:51.786 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:51.786+0000 7fedc9600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:13:51.786 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:51.786+0000 7fedc9600640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:13:51.787 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:51.786+0000 7fedc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:13:51.787 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:51.786+0000 7fedc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:13:51.787 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:51.786+0000 7fedc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:13:51.787 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:51.786+0000 7fedc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:13:51.788 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:13:51.786+0000 7fedc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:13:52.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:52.726+0000 7fc50f400640 1 -- 172.21.15.86:0/3886177859 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mgrmap(e 81) ==== 93054+0+0 (secure 0 0 0) 0x7fc51008e6c0 con 0x7fc5180cf9c0 2024-04-25T14:13:52.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:52.726+0000 7fc50f400640 1 --2- 172.21.15.86:0/3886177859 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc4f8077ac0 0x7fc4f8079ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:52.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:52.726+0000 7fc50f400640 1 -- 172.21.15.86:0/3886177859 --> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7fc510063100 con 0x7fc4f8077ac0 2024-04-25T14:13:52.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:52.726+0000 7fc517400640 1 --2- 172.21.15.86:0/3886177859 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc4f8077ac0 0x7fc4f8079ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:52.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:52.730+0000 7fc517400640 1 --2- 172.21.15.86:0/3886177859 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc4f8077ac0 0x7fc4f8079ee0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc5080af000 tx=0x7fc5080a7000 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:52.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:52.734+0000 7fc50f400640 1 -- 172.21.15.86:0/3886177859 <== mgr.9913 v2:172.21.15.86:6832/2318291413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fc510063100 con 0x7fc4f8077ac0 2024-04-25T14:13:52.735 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {}, "global_metrics": {}, "metrics": {"delayed_ranks": []}} 2024-04-25T14:13:52.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:52.738+0000 7fc51e000640 1 -- 172.21.15.86:0/3886177859 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc4f8077ac0 msgr2=0x7fc4f8079ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:52.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:52.738+0000 7fc51e000640 1 --2- 172.21.15.86:0/3886177859 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc4f8077ac0 0x7fc4f8079ee0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc5080af000 tx=0x7fc5080a7000 comp rx=0 tx=0).stop 2024-04-25T14:13:52.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:52.738+0000 7fc51e000640 1 -- 172.21.15.86:0/3886177859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5180cf9c0 msgr2=0x7fc5180b1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:52.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:52.738+0000 7fc51e000640 1 --2- 172.21.15.86:0/3886177859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5180cf9c0 0x7fc5180b1930 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7fc51000efc0 tx=0x7fc51000c490 comp rx=0 tx=0).stop 2024-04-25T14:13:52.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:52.738+0000 7fc517e00640 1 -- 172.21.15.86:0/3886177859 reap_dead start 2024-04-25T14:13:52.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:52.738+0000 7fc51e000640 1 -- 172.21.15.86:0/3886177859 shutdown_connections 2024-04-25T14:13:52.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:52.738+0000 7fc51e000640 1 -- 172.21.15.86:0/3886177859 >> 172.21.15.86:0/3886177859 conn(0x7fc51809ba30 msgr2=0x7fc518068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:52.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:52.738+0000 7fc51e000640 1 -- 172.21.15.86:0/3886177859 shutdown_connections 2024-04-25T14:13:52.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:52.738+0000 7fc51e000640 1 -- 172.21.15.86:0/3886177859 wait complete. 2024-04-25T14:13:53.759 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:13:53.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.926+0000 7f8c12c00640 1 Processor -- start 2024-04-25T14:13:53.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.926+0000 7f8c12c00640 1 -- start start 2024-04-25T14:13:53.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.930+0000 7f8c12c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c0c070330 0x7f8c0c0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:53.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.930+0000 7f8c12c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8c0c0ce790 0x7f8c0c06f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:53.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.930+0000 7f8c12c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c0c0d2030 con 0x7f8c0c0d1c20 2024-04-25T14:13:53.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.930+0000 7f8c12c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8c0c15e5c0 con 0x7f8c0c0ce790 2024-04-25T14:13:53.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.930+0000 7f8c12c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8c0c15e8a0 con 0x7f8c0c070330 2024-04-25T14:13:53.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.930+0000 7f8c10c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8c0c0ce790 0x7f8c0c06f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:53.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.930+0000 7f8c10c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8c0c0ce790 0x7f8c0c06f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38168/0 (socket says 172.21.15.86:38168) 2024-04-25T14:13:53.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.930+0000 7f8c10c00640 1 -- 172.21.15.86:0/3765698833 learned_addr learned my addr 172.21.15.86:0/3765698833 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:53.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.930+0000 7f8c0be00640 1 --2- 172.21.15.86:0/3765698833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c0c070330 0x7f8c0c0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:53.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.930+0000 7f8c10c00640 1 -- 172.21.15.86:0/3765698833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c0c070330 msgr2=0x7f8c0c0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:53.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.930+0000 7f8c10c00640 1 --2- 172.21.15.86:0/3765698833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c0c070330 0x7f8c0c0d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:53.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.930+0000 7f8c10c00640 1 -- 172.21.15.86:0/3765698833 >> v1:172.21.15.155:6789/0 conn(0x7f8c0c0d1c20 legacy=0x7f8c0c06ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:53.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.930+0000 7f8c10c00640 1 -- 172.21.15.86:0/3765698833 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0c15eb80 con 0x7f8c0c0ce790 2024-04-25T14:13:53.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.930+0000 7f8c0be00640 1 --2- 172.21.15.86:0/3765698833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c0c070330 0x7f8c0c0d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:13:53.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.934+0000 7f8c10c00640 1 --2- 172.21.15.86:0/3765698833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c0c0ce790 0x7f8c0c06f440 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f8c040098a0 tx=0x7f8c0401c290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d9f9cf7bda1bcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:53.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.934+0000 7f8c0b400640 1 -- 172.21.15.86:0/3765698833 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c040b0070 con 0x7f8c0c0ce790 2024-04-25T14:13:53.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.934+0000 7f8c0b400640 1 -- 172.21.15.86:0/3765698833 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c040ab600 con 0x7f8c0c0ce790 2024-04-25T14:13:53.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.934+0000 7f8c0b400640 1 -- 172.21.15.86:0/3765698833 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c040ab9d0 con 0x7f8c0c0ce790 2024-04-25T14:13:53.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.934+0000 7f8c12c00640 1 -- 172.21.15.86:0/3765698833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c0c0ce790 msgr2=0x7f8c0c06f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:53.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.934+0000 7f8c12c00640 1 --2- 172.21.15.86:0/3765698833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c0c0ce790 0x7f8c0c06f440 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f8c040098a0 tx=0x7f8c0401c290 comp rx=0 tx=0).stop 2024-04-25T14:13:53.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.934+0000 7f8c12c00640 1 -- 172.21.15.86:0/3765698833 shutdown_connections 2024-04-25T14:13:53.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.934+0000 7f8c12c00640 1 --2- 172.21.15.86:0/3765698833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c0c0ce790 0x7f8c0c06f440 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:53.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.934+0000 7f8c12c00640 1 --2- 172.21.15.86:0/3765698833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c0c070330 0x7f8c0c0d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:53.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.934+0000 7f8c12c00640 1 -- 172.21.15.86:0/3765698833 >> 172.21.15.86:0/3765698833 conn(0x7f8c0c0b4650 msgr2=0x7f8c0c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:53.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.934+0000 7f8c12c00640 1 -- 172.21.15.86:0/3765698833 shutdown_connections 2024-04-25T14:13:53.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.934+0000 7f8c12c00640 1 -- 172.21.15.86:0/3765698833 wait complete. 2024-04-25T14:13:53.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.938+0000 7f8c12c00640 1 Processor -- start 2024-04-25T14:13:53.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.938+0000 7f8c12c00640 1 -- start start 2024-04-25T14:13:53.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.938+0000 7f8c12c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c0c070330 0x7f8c0c0d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:53.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.938+0000 7f8c12c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c0c0ce790 0x7f8c0c0ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:53.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.938+0000 7f8c12c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c0c0d1c20 0x7f8c0c0b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:53.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.938+0000 7f8c12c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c0708c0 con 0x7f8c0c0d1c20 2024-04-25T14:13:53.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.938+0000 7f8c12c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c070740 con 0x7f8c0c0ce790 2024-04-25T14:13:53.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.938+0000 7f8c12c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8c0c16afc0 con 0x7f8c0c070330 2024-04-25T14:13:53.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.938+0000 7f8c10c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c0c0d1c20 0x7f8c0c0b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:53.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.938+0000 7f8c10c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c0c0d1c20 0x7f8c0c0b1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38184/0 (socket says 172.21.15.86:38184) 2024-04-25T14:13:53.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.938+0000 7f8c10c00640 1 -- 172.21.15.86:0/4147138294 learned_addr learned my addr 172.21.15.86:0/4147138294 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:53.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.938+0000 7f8c0be00640 1 --2- 172.21.15.86:0/4147138294 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c0c070330 0x7f8c0c0d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:53.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.938+0000 7f8c03400640 1 --2- 172.21.15.86:0/4147138294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c0c0ce790 0x7f8c0c0ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:53.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.938+0000 7f8c10c00640 1 -- 172.21.15.86:0/4147138294 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c0c070330 msgr2=0x7f8c0c0d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:53.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.938+0000 7f8c10c00640 1 --2- 172.21.15.86:0/4147138294 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c0c070330 0x7f8c0c0d09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:53.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.938+0000 7f8c10c00640 1 -- 172.21.15.86:0/4147138294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c0c0ce790 msgr2=0x7f8c0c0ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:53.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.938+0000 7f8c10c00640 1 --2- 172.21.15.86:0/4147138294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c0c0ce790 0x7f8c0c0ca780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:53.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.938+0000 7f8c10c00640 1 -- 172.21.15.86:0/4147138294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0c1fcd10 con 0x7f8c0c0d1c20 2024-04-25T14:13:53.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.938+0000 7f8c0be00640 1 --2- 172.21.15.86:0/4147138294 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c0c070330 0x7f8c0c0d09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:53.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.938+0000 7f8c03400640 1 --2- 172.21.15.86:0/4147138294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c0c0ce790 0x7f8c0c0ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:53.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.938+0000 7f8c10c00640 1 --2- 172.21.15.86:0/4147138294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c0c0d1c20 0x7f8c0c0b1f40 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f8c0401a940 tx=0x7f8c04019030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:53.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.938+0000 7f8c08c00640 1 -- 172.21.15.86:0/4147138294 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c040b0040 con 0x7f8c0c0d1c20 2024-04-25T14:13:53.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.938+0000 7f8c08c00640 1 -- 172.21.15.86:0/4147138294 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c040b4620 con 0x7f8c0c0d1c20 2024-04-25T14:13:53.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.938+0000 7f8c08c00640 1 -- 172.21.15.86:0/4147138294 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c040b49d0 con 0x7f8c0c0d1c20 2024-04-25T14:13:53.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.938+0000 7f8c12c00640 1 -- 172.21.15.86:0/4147138294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c0c1fcfd0 con 0x7f8c0c0d1c20 2024-04-25T14:13:53.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.938+0000 7f8c12c00640 1 -- 172.21.15.86:0/4147138294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c0c1fd590 con 0x7f8c0c0d1c20 2024-04-25T14:13:53.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.942+0000 7f8c12c00640 1 -- 172.21.15.86:0/4147138294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c0c069d00 con 0x7f8c0c0d1c20 2024-04-25T14:13:53.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.942+0000 7f8c08c00640 1 -- 172.21.15.86:0/4147138294 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 82) ==== 93262+0+0 (secure 0 0 0) 0x7f8c040ab4c0 con 0x7f8c0c0d1c20 2024-04-25T14:13:53.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.942+0000 7f8c08c00640 1 --2- 172.21.15.86:0/4147138294 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8be806f510 0x7f8be8071a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:53.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.942+0000 7f8c08c00640 1 -- 172.21.15.86:0/4147138294 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f8c041293d0 con 0x7f8c0c0d1c20 2024-04-25T14:13:53.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.942+0000 7f8c0be00640 1 --2- 172.21.15.86:0/4147138294 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8be806f510 0x7f8be8071a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:53.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.942+0000 7f8c0be00640 1 --2- 172.21.15.86:0/4147138294 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8be806f510 0x7f8be8071a20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c072200 tx=0x7f8bfc007a30 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:53.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:53.946+0000 7f8c08c00640 1 -- 172.21.15.86:0/4147138294 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8c0c0d2030 con 0x7f8c0c0d1c20 2024-04-25T14:13:54.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:54.126+0000 7f8c12c00640 1 -- 172.21.15.86:0/4147138294 --> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f8c0c0d0f10 con 0x7f8be806f510 2024-04-25T14:13:54.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:54.126+0000 7f8c08c00640 1 -- 172.21.15.86:0/4147138294 <== mgr.9913 v2:172.21.15.86:6832/2318291413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f8c0c0d0f10 con 0x7f8be806f510 2024-04-25T14:13:54.126 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {}, "global_metrics": {}, "metrics": {"delayed_ranks": []}} 2024-04-25T14:13:54.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:54.130+0000 7f8c12c00640 1 -- 172.21.15.86:0/4147138294 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8be806f510 msgr2=0x7f8be8071a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:54.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:54.130+0000 7f8c12c00640 1 --2- 172.21.15.86:0/4147138294 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8be806f510 0x7f8be8071a20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c072200 tx=0x7f8bfc007a30 comp rx=0 tx=0).stop 2024-04-25T14:13:54.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:54.130+0000 7f8c12c00640 1 -- 172.21.15.86:0/4147138294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c0c0d1c20 msgr2=0x7f8c0c0b1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:54.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:54.130+0000 7f8c12c00640 1 --2- 172.21.15.86:0/4147138294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c0c0d1c20 0x7f8c0c0b1f40 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f8c0401a940 tx=0x7f8c04019030 comp rx=0 tx=0).stop 2024-04-25T14:13:54.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:54.130+0000 7f8c12c00640 1 -- 172.21.15.86:0/4147138294 shutdown_connections 2024-04-25T14:13:54.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:54.130+0000 7f8c12c00640 1 --2- 172.21.15.86:0/4147138294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c0c0d1c20 0x7f8c0c0b1f40 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:54.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:54.130+0000 7f8c12c00640 1 --2- 172.21.15.86:0/4147138294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c0c0ce790 0x7f8c0c0ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:54.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:54.130+0000 7f8c12c00640 1 --2- 172.21.15.86:0/4147138294 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8be806f510 0x7f8be8071a20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:54.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:54.130+0000 7f8c12c00640 1 --2- 172.21.15.86:0/4147138294 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c0c070330 0x7f8c0c0d09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:54.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:54.130+0000 7f8c12c00640 1 -- 172.21.15.86:0/4147138294 >> 172.21.15.86:0/4147138294 conn(0x7f8c0c0b4650 msgr2=0x7f8c0c0c5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:54.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:54.130+0000 7f8c12c00640 1 -- 172.21.15.86:0/4147138294 shutdown_connections 2024-04-25T14:13:54.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:54.130+0000 7f8c12c00640 1 -- 172.21.15.86:0/4147138294 wait complete. 2024-04-25T14:13:55.148 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:13:55.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.310+0000 7f997da00640 1 Processor -- start 2024-04-25T14:13:55.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.310+0000 7f997da00640 1 -- start start 2024-04-25T14:13:55.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.310+0000 7f997da00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f99780d1df0 0x7f99780cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:55.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.310+0000 7f997da00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99780cdcc0 0x7f99780ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:55.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.310+0000 7f997da00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f997815e5c0 con 0x7f9978070330 2024-04-25T14:13:55.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.310+0000 7f997da00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f997806fd40 con 0x7f99780d1df0 2024-04-25T14:13:55.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.310+0000 7f997da00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f997815e8a0 con 0x7f99780cdcc0 2024-04-25T14:13:55.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.310+0000 7f9976a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9978070330 0x7f99780d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:56430/0 (socket says 172.21.15.86:56430) 2024-04-25T14:13:55.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.310+0000 7f9976a00640 1 -- 172.21.15.86:0/1163976371 learned_addr learned my addr 172.21.15.86:0/1163976371 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:55.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.310+0000 7f996fe00640 1 --2- 172.21.15.86:0/1163976371 >> v2:172.21.15.155:3300/0 conn(0x7f99780d1df0 0x7f99780cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:55.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.310+0000 7f9977400640 1 --2- 172.21.15.86:0/1163976371 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99780cdcc0 0x7f99780ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:55.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.310+0000 7f9976000640 1 -- 172.21.15.86:0/1163976371 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3814221906 0 0) 0x7f997815e5c0 con 0x7f9978070330 2024-04-25T14:13:55.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.310+0000 7f9976000640 1 -- 172.21.15.86:0/1163976371 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f995c003530 con 0x7f9978070330 2024-04-25T14:13:55.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.314+0000 7f9976000640 1 -- 172.21.15.86:0/1163976371 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 555299359 0 0) 0x7f995c003530 con 0x7f9978070330 2024-04-25T14:13:55.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.314+0000 7f9976000640 1 -- 172.21.15.86:0/1163976371 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99780cdcc0 msgr2=0x7f99780ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:55.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.314+0000 7f9976000640 1 --2- 172.21.15.86:0/1163976371 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99780cdcc0 0x7f99780ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:55.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.314+0000 7f9976000640 1 -- 172.21.15.86:0/1163976371 >> v2:172.21.15.155:3300/0 conn(0x7f99780d1df0 msgr2=0x7f99780cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:55.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.314+0000 7f9976000640 1 --2- 172.21.15.86:0/1163976371 >> v2:172.21.15.155:3300/0 conn(0x7f99780d1df0 0x7f99780cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:55.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.314+0000 7f9976000640 1 -- 172.21.15.86:0/1163976371 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997815eb80 con 0x7f9978070330 2024-04-25T14:13:55.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.314+0000 7f9976000640 1 -- 172.21.15.86:0/1163976371 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f9968001680 con 0x7f9978070330 2024-04-25T14:13:55.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.314+0000 7f9976000640 1 -- 172.21.15.86:0/1163976371 >> v1:172.21.15.86:6789/0 conn(0x7f9978070330 legacy=0x7f99780d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:55.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.314+0000 7f9976000640 1 --2- 172.21.15.86:0/1163976371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f995c003a30 0x7f995c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:55.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.314+0000 7f9976000640 1 --2- 172.21.15.86:0/1163976371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f995c004810 0x7f995c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:55.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.314+0000 7f9976000640 1 --2- 172.21.15.86:0/1163976371 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99780cdcc0 0x7f995c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:55.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.314+0000 7f9976000640 1 -- 172.21.15.86:0/1163976371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f997815e8a0 con 0x7f995c004810 2024-04-25T14:13:55.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.314+0000 7f9976000640 1 -- 172.21.15.86:0/1163976371 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f995c00ba80 con 0x7f995c003a30 2024-04-25T14:13:55.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.314+0000 7f9976000640 1 -- 172.21.15.86:0/1163976371 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f995c00bc50 con 0x7f99780cdcc0 2024-04-25T14:13:55.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.314+0000 7f996fe00640 1 --2- 172.21.15.86:0/1163976371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f995c004810 0x7f995c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:55.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.314+0000 7f9976a00640 1 --2- 172.21.15.86:0/1163976371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f995c003a30 0x7f995c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:55.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.314+0000 7f9977400640 1 --2- 172.21.15.86:0/1163976371 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99780cdcc0 0x7f995c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:55.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.314+0000 7f996fe00640 1 -- 172.21.15.86:0/1163976371 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99780cdcc0 msgr2=0x7f995c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:55.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.314+0000 7f996fe00640 1 --2- 172.21.15.86:0/1163976371 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99780cdcc0 0x7f995c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:55.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.314+0000 7f996fe00640 1 -- 172.21.15.86:0/1163976371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f995c003a30 msgr2=0x7f995c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:55.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.314+0000 7f996fe00640 1 --2- 172.21.15.86:0/1163976371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f995c003a30 0x7f995c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:55.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.314+0000 7f996fe00640 1 -- 172.21.15.86:0/1163976371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f995c003530 con 0x7f995c004810 2024-04-25T14:13:55.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.314+0000 7f9976a00640 1 --2- 172.21.15.86:0/1163976371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f995c003a30 0x7f995c003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:55.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.314+0000 7f9977400640 1 --2- 172.21.15.86:0/1163976371 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99780cdcc0 0x7f995c005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:55.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.314+0000 7f996fe00640 1 --2- 172.21.15.86:0/1163976371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f995c004810 0x7f995c004c20 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f9978074ed0 tx=0x7f996001ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d253e40b39f5360f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:55.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.318+0000 7f9976000640 1 -- 172.21.15.86:0/1163976371 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99600b2070 con 0x7f995c004810 2024-04-25T14:13:55.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.318+0000 7f9976000640 1 -- 172.21.15.86:0/1163976371 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99600a55d0 con 0x7f995c004810 2024-04-25T14:13:55.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.318+0000 7f9976000640 1 -- 172.21.15.86:0/1163976371 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99600096b0 con 0x7f995c004810 2024-04-25T14:13:55.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.318+0000 7f997da00640 1 -- 172.21.15.86:0/1163976371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f995c004810 msgr2=0x7f995c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:55.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.318+0000 7f997da00640 1 --2- 172.21.15.86:0/1163976371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f995c004810 0x7f995c004c20 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f9978074ed0 tx=0x7f996001ca20 comp rx=0 tx=0).stop 2024-04-25T14:13:55.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.318+0000 7f9977400640 1 -- 172.21.15.86:0/1163976371 reap_dead start 2024-04-25T14:13:55.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.318+0000 7f997da00640 1 -- 172.21.15.86:0/1163976371 shutdown_connections 2024-04-25T14:13:55.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.318+0000 7f997da00640 1 -- 172.21.15.86:0/1163976371 >> 172.21.15.86:0/1163976371 conn(0x7f99780b4650 msgr2=0x7f99780b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:55.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.318+0000 7f997da00640 1 -- 172.21.15.86:0/1163976371 shutdown_connections 2024-04-25T14:13:55.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.318+0000 7f997da00640 1 -- 172.21.15.86:0/1163976371 wait complete. 2024-04-25T14:13:55.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.318+0000 7f997da00640 1 Processor -- start 2024-04-25T14:13:55.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.318+0000 7f997da00640 1 -- start start 2024-04-25T14:13:55.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.322+0000 7f997da00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f995c004810 0x7f997809a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:55.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.322+0000 7f997da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9978159840 0x7f99781ee500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:55.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.322+0000 7f997da00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f997809a9a0 0x7f99781eea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:55.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.322+0000 7f997da00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f99780708c0 con 0x7f9978159840 2024-04-25T14:13:55.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.322+0000 7f997da00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9978070740 con 0x7f997809a9a0 2024-04-25T14:13:55.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.322+0000 7f997da00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f99780ce8e0 con 0x7f995c004810 2024-04-25T14:13:55.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.322+0000 7f996fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9978159840 0x7f99781ee500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:55.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.322+0000 7f996fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9978159840 0x7f99781ee500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38192/0 (socket says 172.21.15.86:38192) 2024-04-25T14:13:55.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.322+0000 7f996fe00640 1 -- 172.21.15.86:0/2846811466 learned_addr learned my addr 172.21.15.86:0/2846811466 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:55.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.322+0000 7f9976a00640 1 --2- 172.21.15.86:0/2846811466 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f995c004810 0x7f997809a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:55.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.322+0000 7f9977400640 1 --2- 172.21.15.86:0/2846811466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f997809a9a0 0x7f99781eea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:55.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.322+0000 7f996fe00640 1 -- 172.21.15.86:0/2846811466 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f995c004810 msgr2=0x7f997809a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:55.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.322+0000 7f996fe00640 1 --2- 172.21.15.86:0/2846811466 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f995c004810 0x7f997809a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:55.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.322+0000 7f996fe00640 1 -- 172.21.15.86:0/2846811466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f997809a9a0 msgr2=0x7f99781eea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:55.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.322+0000 7f996fe00640 1 --2- 172.21.15.86:0/2846811466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f997809a9a0 0x7f99781eea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:55.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.322+0000 7f996fe00640 1 -- 172.21.15.86:0/2846811466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99780b1900 con 0x7f9978159840 2024-04-25T14:13:55.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.322+0000 7f996fe00640 1 --2- 172.21.15.86:0/2846811466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9978159840 0x7f99781ee500 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f99600b0040 tx=0x7f995c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:55.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.322+0000 7f996ea00640 1 -- 172.21.15.86:0/2846811466 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99600be070 con 0x7f9978159840 2024-04-25T14:13:55.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.322+0000 7f996ea00640 1 -- 172.21.15.86:0/2846811466 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f996000f000 con 0x7f9978159840 2024-04-25T14:13:55.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.322+0000 7f997da00640 1 -- 172.21.15.86:0/2846811466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99780b1be0 con 0x7f9978159840 2024-04-25T14:13:55.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.322+0000 7f996ea00640 1 -- 172.21.15.86:0/2846811466 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99600b2040 con 0x7f9978159840 2024-04-25T14:13:55.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.322+0000 7f997da00640 1 -- 172.21.15.86:0/2846811466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99780b21a0 con 0x7f9978159840 2024-04-25T14:13:55.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.322+0000 7f997da00640 1 -- 172.21.15.86:0/2846811466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9978069d00 con 0x7f9978159840 2024-04-25T14:13:55.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.326+0000 7f996ea00640 1 -- 172.21.15.86:0/2846811466 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 82) ==== 93262+0+0 (secure 0 0 0) 0x7f99600c1070 con 0x7f9978159840 2024-04-25T14:13:55.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.326+0000 7f996ea00640 1 --2- 172.21.15.86:0/2846811466 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f995406f4e0 0x7f99540719f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:55.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.326+0000 7f996ea00640 1 -- 172.21.15.86:0/2846811466 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f9960129220 con 0x7f9978159840 2024-04-25T14:13:55.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.326+0000 7f9976a00640 1 --2- 172.21.15.86:0/2846811466 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f995406f4e0 0x7f99540719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:55.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.326+0000 7f9976a00640 1 --2- 172.21.15.86:0/2846811466 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f995406f4e0 0x7f99540719f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9978072200 tx=0x7f9968009040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:55.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.330+0000 7f996ea00640 1 -- 172.21.15.86:0/2846811466 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9960016760 con 0x7f9978159840 2024-04-25T14:13:55.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.506+0000 7f997da00640 1 -- 172.21.15.86:0/2846811466 --> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f9978076bf0 con 0x7f995406f4e0 2024-04-25T14:13:55.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.506+0000 7f996ea00640 1 -- 172.21.15.86:0/2846811466 <== mgr.9913 v2:172.21.15.86:6832/2318291413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f9978076bf0 con 0x7f995406f4e0 2024-04-25T14:13:55.507 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {}, "global_metrics": {}, "metrics": {"delayed_ranks": []}} 2024-04-25T14:13:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.510+0000 7f997da00640 1 -- 172.21.15.86:0/2846811466 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f995406f4e0 msgr2=0x7f99540719f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.510+0000 7f997da00640 1 --2- 172.21.15.86:0/2846811466 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f995406f4e0 0x7f99540719f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9978072200 tx=0x7f9968009040 comp rx=0 tx=0).stop 2024-04-25T14:13:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.510+0000 7f997da00640 1 -- 172.21.15.86:0/2846811466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9978159840 msgr2=0x7f99781ee500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.510+0000 7f997da00640 1 --2- 172.21.15.86:0/2846811466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9978159840 0x7f99781ee500 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f99600b0040 tx=0x7f995c005840 comp rx=0 tx=0).stop 2024-04-25T14:13:55.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.510+0000 7f997da00640 1 -- 172.21.15.86:0/2846811466 shutdown_connections 2024-04-25T14:13:55.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.510+0000 7f997da00640 1 --2- 172.21.15.86:0/2846811466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f997809a9a0 0x7f99781eea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:55.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.510+0000 7f997da00640 1 --2- 172.21.15.86:0/2846811466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9978159840 0x7f99781ee500 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:55.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.510+0000 7f997da00640 1 --2- 172.21.15.86:0/2846811466 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f995406f4e0 0x7f99540719f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:55.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.510+0000 7f997da00640 1 --2- 172.21.15.86:0/2846811466 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f995c004810 0x7f997809a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:55.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.510+0000 7f997da00640 1 -- 172.21.15.86:0/2846811466 >> 172.21.15.86:0/2846811466 conn(0x7f99780b4650 msgr2=0x7f99780ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:55.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.510+0000 7f997da00640 1 -- 172.21.15.86:0/2846811466 shutdown_connections 2024-04-25T14:13:55.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:55.510+0000 7f997da00640 1 -- 172.21.15.86:0/2846811466 wait complete. 2024-04-25T14:13:56.530 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:13:56.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.690+0000 7ff7cb200640 1 Processor -- start 2024-04-25T14:13:56.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.690+0000 7ff7cb200640 1 -- start start 2024-04-25T14:13:56.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.690+0000 7ff7cb200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff7c415c2a0 0x7ff7c40ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:56.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.690+0000 7ff7cb200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7c406fd50 0x7ff7c4070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:56.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.690+0000 7ff7cb200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7c40b0a60 con 0x7ff7c406c670 2024-04-25T14:13:56.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.690+0000 7ff7cb200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff7c406ef90 con 0x7ff7c415c2a0 2024-04-25T14:13:56.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.690+0000 7ff7cb200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff7c406f270 con 0x7ff7c406fd50 2024-04-25T14:13:56.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.694+0000 7ff7c3400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7c406fd50 0x7ff7c4070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:56.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.694+0000 7ff7c3e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff7c415c2a0 0x7ff7c40ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:56.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.694+0000 7ff7c3e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff7c415c2a0 0x7ff7c40ae130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:41974/0 (socket says 172.21.15.86:41974) 2024-04-25T14:13:56.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.694+0000 7ff7c3e00640 1 -- 172.21.15.86:0/844292812 learned_addr learned my addr 172.21.15.86:0/844292812 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:56.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.694+0000 7ff7c3400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7c406fd50 0x7ff7c4070160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:43790/0 (socket says 172.21.15.86:43790) 2024-04-25T14:13:56.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.694+0000 7ff7c9200640 1 --1- 172.21.15.86:0/844292812 >> v1:172.21.15.155:6789/0 conn(0x7ff7c406c670 0x7ff7c406ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:50144/0 (socket says 172.21.15.86:50144) 2024-04-25T14:13:56.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.694+0000 7ff7c3e00640 1 -- 172.21.15.86:0/844292812 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7c406fd50 msgr2=0x7ff7c4070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:56.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.694+0000 7ff7c3e00640 1 --2- 172.21.15.86:0/844292812 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7c406fd50 0x7ff7c4070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:56.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.694+0000 7ff7c3e00640 1 -- 172.21.15.86:0/844292812 >> v1:172.21.15.155:6789/0 conn(0x7ff7c406c670 legacy=0x7ff7c406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:56.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.694+0000 7ff7c3e00640 1 -- 172.21.15.86:0/844292812 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c406f550 con 0x7ff7c415c2a0 2024-04-25T14:13:56.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.694+0000 7ff7c2a00640 1 -- 172.21.15.86:0/844292812 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2237311449 0 0) 0x7ff7c40b0a60 con 0x7ff7c406c670 2024-04-25T14:13:56.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.694+0000 7ff7c3e00640 1 --2- 172.21.15.86:0/844292812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7c415c2a0 0x7ff7c40ae130 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009e50 tx=0x7ff7b401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e450efc66a1b5d90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:56.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.698+0000 7ff7c2a00640 1 -- 172.21.15.86:0/844292812 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b40a5770 con 0x7ff7c415c2a0 2024-04-25T14:13:56.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.698+0000 7ff7c2a00640 1 -- 172.21.15.86:0/844292812 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7b40a5910 con 0x7ff7c415c2a0 2024-04-25T14:13:56.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.698+0000 7ff7c2a00640 1 -- 172.21.15.86:0/844292812 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b40a5ce0 con 0x7ff7c415c2a0 2024-04-25T14:13:56.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.698+0000 7ff7cb200640 1 -- 172.21.15.86:0/844292812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7c415c2a0 msgr2=0x7ff7c40ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:56.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.698+0000 7ff7cb200640 1 --2- 172.21.15.86:0/844292812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7c415c2a0 0x7ff7c40ae130 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009e50 tx=0x7ff7b401c970 comp rx=0 tx=0).stop 2024-04-25T14:13:56.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.698+0000 7ff7cb200640 1 -- 172.21.15.86:0/844292812 shutdown_connections 2024-04-25T14:13:56.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.698+0000 7ff7cb200640 1 --2- 172.21.15.86:0/844292812 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7c406fd50 0x7ff7c4070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:56.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.698+0000 7ff7cb200640 1 --2- 172.21.15.86:0/844292812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7c415c2a0 0x7ff7c40ae130 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:56.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.698+0000 7ff7cb200640 1 -- 172.21.15.86:0/844292812 >> 172.21.15.86:0/844292812 conn(0x7ff7c40b4650 msgr2=0x7ff7c40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:56.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.698+0000 7ff7cb200640 1 -- 172.21.15.86:0/844292812 shutdown_connections 2024-04-25T14:13:56.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.698+0000 7ff7cb200640 1 -- 172.21.15.86:0/844292812 wait complete. 2024-04-25T14:13:56.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.698+0000 7ff7cb200640 1 Processor -- start 2024-04-25T14:13:56.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.698+0000 7ff7cb200640 1 -- start start 2024-04-25T14:13:56.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.702+0000 7ff7cb200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7c406c670 0x7ff7c409aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:56.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.702+0000 7ff7cb200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7c406fd50 0x7ff7c409e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:56.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.702+0000 7ff7cb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7c415c2a0 0x7ff7c409eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:56.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.702+0000 7ff7cb200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c415c830 con 0x7ff7c415c2a0 2024-04-25T14:13:56.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.702+0000 7ff7cb200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c415c6b0 con 0x7ff7c406c670 2024-04-25T14:13:56.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.702+0000 7ff7cb200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff7c40ae670 con 0x7ff7c406fd50 2024-04-25T14:13:56.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.702+0000 7ff7c9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7c415c2a0 0x7ff7c409eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:56.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.702+0000 7ff7c9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7c415c2a0 0x7ff7c409eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38198/0 (socket says 172.21.15.86:38198) 2024-04-25T14:13:56.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.702+0000 7ff7c3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7c406c670 0x7ff7c409aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:56.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.702+0000 7ff7c9200640 1 -- 172.21.15.86:0/988237632 learned_addr learned my addr 172.21.15.86:0/988237632 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:56.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.702+0000 7ff7c3400640 1 --2- 172.21.15.86:0/988237632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7c406fd50 0x7ff7c409e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:56.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.702+0000 7ff7c9200640 1 -- 172.21.15.86:0/988237632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7c406fd50 msgr2=0x7ff7c409e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:56.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.702+0000 7ff7c9200640 1 --2- 172.21.15.86:0/988237632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7c406fd50 0x7ff7c409e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:56.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.702+0000 7ff7c9200640 1 -- 172.21.15.86:0/988237632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7c406c670 msgr2=0x7ff7c409aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:56.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.702+0000 7ff7c9200640 1 --2- 172.21.15.86:0/988237632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7c406c670 0x7ff7c409aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:56.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.702+0000 7ff7c9200640 1 -- 172.21.15.86:0/988237632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c4068f60 con 0x7ff7c415c2a0 2024-04-25T14:13:56.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.702+0000 7ff7c9200640 1 --2- 172.21.15.86:0/988237632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7c415c2a0 0x7ff7c409eba0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc00c410 tx=0x7ff7bc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:56.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.702+0000 7ff7bbe00640 1 -- 172.21.15.86:0/988237632 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc0074e0 con 0x7ff7c415c2a0 2024-04-25T14:13:56.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.702+0000 7ff7bbe00640 1 -- 172.21.15.86:0/988237632 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7bc0047f0 con 0x7ff7c415c2a0 2024-04-25T14:13:56.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.702+0000 7ff7bbe00640 1 -- 172.21.15.86:0/988237632 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc007680 con 0x7ff7c415c2a0 2024-04-25T14:13:56.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.702+0000 7ff7cb200640 1 -- 172.21.15.86:0/988237632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c4155490 con 0x7ff7c415c2a0 2024-04-25T14:13:56.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.702+0000 7ff7cb200640 1 -- 172.21.15.86:0/988237632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c409c040 con 0x7ff7c415c2a0 2024-04-25T14:13:56.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.702+0000 7ff7cb200640 1 -- 172.21.15.86:0/988237632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7c4069d00 con 0x7ff7c415c2a0 2024-04-25T14:13:56.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.706+0000 7ff7bbe00640 1 -- 172.21.15.86:0/988237632 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7ff7bc011070 con 0x7ff7c415c2a0 2024-04-25T14:13:56.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.706+0000 7ff7bbe00640 1 --2- 172.21.15.86:0/988237632 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ff7a406f5a0 0x7ff7a4071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:56.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.706+0000 7ff7bbe00640 1 -- 172.21.15.86:0/988237632 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7ff7bc08f0f0 con 0x7ff7c415c2a0 2024-04-25T14:13:56.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.706+0000 7ff7c3e00640 1 --2- 172.21.15.86:0/988237632 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ff7a406f5a0 0x7ff7a4071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:56.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.706+0000 7ff7c3e00640 1 --2- 172.21.15.86:0/988237632 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ff7a406f5a0 0x7ff7a4071ab0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40ae000 tx=0x7ff7b40a7070 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:56.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.710+0000 7ff7bbe00640 1 -- 172.21.15.86:0/988237632 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff7bc05b630 con 0x7ff7c415c2a0 2024-04-25T14:13:56.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.890+0000 7ff7cb200640 1 -- 172.21.15.86:0/988237632 --> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7ff7c4076bf0 con 0x7ff7a406f5a0 2024-04-25T14:13:56.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.890+0000 7ff7bbe00640 1 -- 172.21.15.86:0/988237632 <== mgr.9913 v2:172.21.15.86:6832/2318291413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7ff7c4076bf0 con 0x7ff7a406f5a0 2024-04-25T14:13:56.891 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {}, "global_metrics": {}, "metrics": {"delayed_ranks": []}} 2024-04-25T14:13:56.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.894+0000 7ff7cb200640 1 -- 172.21.15.86:0/988237632 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ff7a406f5a0 msgr2=0x7ff7a4071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:56.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.894+0000 7ff7cb200640 1 --2- 172.21.15.86:0/988237632 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ff7a406f5a0 0x7ff7a4071ab0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40ae000 tx=0x7ff7b40a7070 comp rx=0 tx=0).stop 2024-04-25T14:13:56.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.894+0000 7ff7cb200640 1 -- 172.21.15.86:0/988237632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7c415c2a0 msgr2=0x7ff7c409eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:56.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.894+0000 7ff7cb200640 1 --2- 172.21.15.86:0/988237632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7c415c2a0 0x7ff7c409eba0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc00c410 tx=0x7ff7bc00c8d0 comp rx=0 tx=0).stop 2024-04-25T14:13:56.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.894+0000 7ff7cb200640 1 -- 172.21.15.86:0/988237632 shutdown_connections 2024-04-25T14:13:56.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.894+0000 7ff7cb200640 1 --2- 172.21.15.86:0/988237632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7c415c2a0 0x7ff7c409eba0 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:56.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.894+0000 7ff7cb200640 1 --2- 172.21.15.86:0/988237632 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ff7a406f5a0 0x7ff7a4071ab0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:56.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.894+0000 7ff7cb200640 1 --2- 172.21.15.86:0/988237632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7c406fd50 0x7ff7c409e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:56.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.894+0000 7ff7cb200640 1 --2- 172.21.15.86:0/988237632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7c406c670 0x7ff7c409aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:56.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.894+0000 7ff7cb200640 1 -- 172.21.15.86:0/988237632 >> 172.21.15.86:0/988237632 conn(0x7ff7c40b4650 msgr2=0x7ff7c40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:56.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.894+0000 7ff7cb200640 1 -- 172.21.15.86:0/988237632 shutdown_connections 2024-04-25T14:13:56.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:56.894+0000 7ff7cb200640 1 -- 172.21.15.86:0/988237632 wait complete. 2024-04-25T14:13:57.913 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:13:58.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.074+0000 7fdb12200640 1 Processor -- start 2024-04-25T14:13:58.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.074+0000 7fdb12200640 1 -- start start 2024-04-25T14:13:58.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.074+0000 7fdb12200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdb0c0cfca0 0x7fdb0c06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:58.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.074+0000 7fdb12200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb0c0730b0 0x7fdb0c0734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.074+0000 7fdb12200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb0c072bd0 con 0x7fdb0c0d16e0 2024-04-25T14:13:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.074+0000 7fdb12200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fdb0c070470 con 0x7fdb0c0cfca0 2024-04-25T14:13:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.074+0000 7fdb12200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdb0c06f930 con 0x7fdb0c0730b0 2024-04-25T14:13:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.074+0000 7fdb0aa00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fdb0c0d16e0 0x7fdb0c0d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:56434/0 (socket says 172.21.15.86:56434) 2024-04-25T14:13:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.074+0000 7fdb0aa00640 1 -- 172.21.15.86:0/2057663298 learned_addr learned my addr 172.21.15.86:0/2057663298 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.074+0000 7fdb0b400640 1 --2- 172.21.15.86:0/2057663298 >> v2:172.21.15.155:3300/0 conn(0x7fdb0c0cfca0 0x7fdb0c06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.074+0000 7fdb0be00640 1 --2- 172.21.15.86:0/2057663298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb0c0730b0 0x7fdb0c0734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.074+0000 7fdb0a000640 1 -- 172.21.15.86:0/2057663298 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2613380773 0 0) 0x7fdb0c072bd0 con 0x7fdb0c0d16e0 2024-04-25T14:13:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.074+0000 7fdb0a000640 1 -- 172.21.15.86:0/2057663298 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdaf0003530 con 0x7fdb0c0d16e0 2024-04-25T14:13:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.074+0000 7fdb0a000640 1 -- 172.21.15.86:0/2057663298 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3906242785 0 0) 0x7fdaf0003530 con 0x7fdb0c0d16e0 2024-04-25T14:13:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.074+0000 7fdb0a000640 1 -- 172.21.15.86:0/2057663298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb0c0730b0 msgr2=0x7fdb0c0734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.074+0000 7fdb0a000640 1 --2- 172.21.15.86:0/2057663298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb0c0730b0 0x7fdb0c0734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.074+0000 7fdb0a000640 1 -- 172.21.15.86:0/2057663298 >> v2:172.21.15.155:3300/0 conn(0x7fdb0c0cfca0 msgr2=0x7fdb0c06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.074+0000 7fdb0a000640 1 --2- 172.21.15.86:0/2057663298 >> v2:172.21.15.155:3300/0 conn(0x7fdb0c0cfca0 0x7fdb0c06f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.074+0000 7fdb0a000640 1 -- 172.21.15.86:0/2057663298 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0c06fbd0 con 0x7fdb0c0d16e0 2024-04-25T14:13:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.074+0000 7fdb0a000640 1 -- 172.21.15.86:0/2057663298 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fdaf80016a0 con 0x7fdb0c0d16e0 2024-04-25T14:13:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.074+0000 7fdb0b400640 1 --2- 172.21.15.86:0/2057663298 >> v2:172.21.15.155:3300/0 conn(0x7fdb0c0cfca0 0x7fdb0c06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:13:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.078+0000 7fdb0a000640 1 -- 172.21.15.86:0/2057663298 >> v1:172.21.15.86:6789/0 conn(0x7fdb0c0d16e0 legacy=0x7fdb0c0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.078+0000 7fdb0a000640 1 --2- 172.21.15.86:0/2057663298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdaf0003a30 0x7fdaf0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.078+0000 7fdb0a000640 1 --2- 172.21.15.86:0/2057663298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf0004810 0x7fdaf0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.078+0000 7fdb0a000640 1 --2- 172.21.15.86:0/2057663298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb0c0730b0 0x7fdaf0005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.078+0000 7fdb0a000640 1 -- 172.21.15.86:0/2057663298 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c070470 con 0x7fdaf0004810 2024-04-25T14:13:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.078+0000 7fdb0a000640 1 -- 172.21.15.86:0/2057663298 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c06f930 con 0x7fdaf0003a30 2024-04-25T14:13:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.078+0000 7fdb0a000640 1 -- 172.21.15.86:0/2057663298 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf000ba80 con 0x7fdb0c0730b0 2024-04-25T14:13:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.078+0000 7fdb0aa00640 1 --2- 172.21.15.86:0/2057663298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf0004810 0x7fdaf0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.078+0000 7fdb0b400640 1 --2- 172.21.15.86:0/2057663298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdaf0003a30 0x7fdaf0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.078+0000 7fdb0be00640 1 --2- 172.21.15.86:0/2057663298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb0c0730b0 0x7fdaf0005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.078+0000 7fdb0b400640 1 -- 172.21.15.86:0/2057663298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb0c0730b0 msgr2=0x7fdaf0005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.078+0000 7fdb0b400640 1 --2- 172.21.15.86:0/2057663298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb0c0730b0 0x7fdaf0005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.078+0000 7fdb0b400640 1 -- 172.21.15.86:0/2057663298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf0004810 msgr2=0x7fdaf0004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.078+0000 7fdb0b400640 1 --2- 172.21.15.86:0/2057663298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf0004810 0x7fdaf0004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.078+0000 7fdb0b400640 1 -- 172.21.15.86:0/2057663298 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf0003530 con 0x7fdaf0003a30 2024-04-25T14:13:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.078+0000 7fdb0aa00640 1 --2- 172.21.15.86:0/2057663298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf0004810 0x7fdaf0004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:13:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.078+0000 7fdb0b400640 1 --2- 172.21.15.86:0/2057663298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdaf0003a30 0x7fdaf0003ea0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fdafc009cb0 tx=0x7fdafc01cc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a652a6dfdf4b52e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.082+0000 7fdb0a000640 1 -- 172.21.15.86:0/2057663298 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc009880 con 0x7fdaf0003a30 2024-04-25T14:13:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.082+0000 7fdb0a000640 1 -- 172.21.15.86:0/2057663298 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdafc004020 con 0x7fdaf0003a30 2024-04-25T14:13:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.082+0000 7fdb0a000640 1 -- 172.21.15.86:0/2057663298 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0043f0 con 0x7fdaf0003a30 2024-04-25T14:13:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.082+0000 7fdb12200640 1 -- 172.21.15.86:0/2057663298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdaf0003a30 msgr2=0x7fdaf0003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.082+0000 7fdb12200640 1 --2- 172.21.15.86:0/2057663298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdaf0003a30 0x7fdaf0003ea0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fdafc009cb0 tx=0x7fdafc01cc50 comp rx=0 tx=0).stop 2024-04-25T14:13:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.082+0000 7fdb0be00640 1 -- 172.21.15.86:0/2057663298 reap_dead start 2024-04-25T14:13:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.082+0000 7fdb12200640 1 -- 172.21.15.86:0/2057663298 shutdown_connections 2024-04-25T14:13:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.082+0000 7fdb12200640 1 -- 172.21.15.86:0/2057663298 >> 172.21.15.86:0/2057663298 conn(0x7fdb0c0b4650 msgr2=0x7fdb0c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.082+0000 7fdb12200640 1 -- 172.21.15.86:0/2057663298 shutdown_connections 2024-04-25T14:13:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.082+0000 7fdb12200640 1 -- 172.21.15.86:0/2057663298 wait complete. 2024-04-25T14:13:58.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.082+0000 7fdb12200640 1 Processor -- start 2024-04-25T14:13:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.082+0000 7fdb12200640 1 -- start start 2024-04-25T14:13:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.086+0000 7fdb12200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdaf0003a30 0x7fdb0c0ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.086+0000 7fdb12200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb0c0cacf0 0x7fdb0c15a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.086+0000 7fdb12200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb0c15ac90 0x7fdb0c15b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.086+0000 7fdb12200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c0734c0 con 0x7fdb0c15ac90 2024-04-25T14:13:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.086+0000 7fdb12200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c070190 con 0x7fdaf0003a30 2024-04-25T14:13:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.086+0000 7fdb12200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdb0c073640 con 0x7fdb0c0cacf0 2024-04-25T14:13:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.086+0000 7fdb0be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb0c15ac90 0x7fdb0c15b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.086+0000 7fdb0be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb0c15ac90 0x7fdb0c15b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38208/0 (socket says 172.21.15.86:38208) 2024-04-25T14:13:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.086+0000 7fdb0be00640 1 -- 172.21.15.86:0/1014043360 learned_addr learned my addr 172.21.15.86:0/1014043360 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.086+0000 7fdb0b400640 1 --2- 172.21.15.86:0/1014043360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdaf0003a30 0x7fdb0c0ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.086+0000 7fdb0aa00640 1 --2- 172.21.15.86:0/1014043360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb0c0cacf0 0x7fdb0c15a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.086+0000 7fdb0be00640 1 -- 172.21.15.86:0/1014043360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb0c0cacf0 msgr2=0x7fdb0c15a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.086+0000 7fdb0be00640 1 --2- 172.21.15.86:0/1014043360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb0c0cacf0 0x7fdb0c15a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.086+0000 7fdb0be00640 1 -- 172.21.15.86:0/1014043360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdaf0003a30 msgr2=0x7fdb0c0ca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.086+0000 7fdb0be00640 1 --2- 172.21.15.86:0/1014043360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdaf0003a30 0x7fdb0c0ca4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.086+0000 7fdb0be00640 1 -- 172.21.15.86:0/1014043360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0c0d0300 con 0x7fdb0c15ac90 2024-04-25T14:13:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.086+0000 7fdb0be00640 1 --2- 172.21.15.86:0/1014043360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb0c15ac90 0x7fdb0c15b0a0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0b3f10 tx=0x7fdb0400f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.086+0000 7fdb03400640 1 -- 172.21.15.86:0/1014043360 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8007d70 con 0x7fdb0c15ac90 2024-04-25T14:13:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.086+0000 7fdb03400640 1 -- 172.21.15.86:0/1014043360 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb0401d070 con 0x7fdb0c15ac90 2024-04-25T14:13:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.086+0000 7fdb12200640 1 -- 172.21.15.86:0/1014043360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb0c1596b0 con 0x7fdb0c15ac90 2024-04-25T14:13:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.086+0000 7fdb03400640 1 -- 172.21.15.86:0/1014043360 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04015430 con 0x7fdb0c15ac90 2024-04-25T14:13:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.086+0000 7fdb12200640 1 -- 172.21.15.86:0/1014043360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb0c159c70 con 0x7fdb0c15ac90 2024-04-25T14:13:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.086+0000 7fdb12200640 1 -- 172.21.15.86:0/1014043360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb0c15bd30 con 0x7fdb0c15ac90 2024-04-25T14:13:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.090+0000 7fdb03400640 1 -- 172.21.15.86:0/1014043360 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7fdb040155d0 con 0x7fdb0c15ac90 2024-04-25T14:13:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.090+0000 7fdb03400640 1 --2- 172.21.15.86:0/1014043360 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fdaec06f570 0x7fdaec071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.090+0000 7fdb03400640 1 -- 172.21.15.86:0/1014043360 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fdb0408ff00 con 0x7fdb0c15ac90 2024-04-25T14:13:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.090+0000 7fdb0b400640 1 --2- 172.21.15.86:0/1014043360 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fdaec06f570 0x7fdaec071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.090+0000 7fdb0b400640 1 --2- 172.21.15.86:0/1014043360 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fdaec06f570 0x7fdaec071a80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0ae000 tx=0x7fdafc0046d0 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:58.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.094+0000 7fdb03400640 1 -- 172.21.15.86:0/1014043360 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdb04010040 con 0x7fdb0c15ac90 2024-04-25T14:13:58.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.278+0000 7fdb12200640 1 -- 172.21.15.86:0/1014043360 --> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7fdb0c076bf0 con 0x7fdaec06f570 2024-04-25T14:13:58.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.278+0000 7fdb03400640 1 -- 172.21.15.86:0/1014043360 <== mgr.9913 v2:172.21.15.86:6832/2318291413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+2956 (secure 0 0 0) 0x7fdb0c076bf0 con 0x7fdaec06f570 2024-04-25T14:13:58.280 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {"cephfs": {"client.8492": {"hostname": "smithi155", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:192.168.0.1"}, "client.9880": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:172.21.15.86"}, "client.9946": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}, "client.9949": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}}}, "global_metrics": {"cephfs": {"client.8492": [[22, 0], [0, 0], [0, 0], [0, 5266510], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 1316628], [7806089317574, 4]], "client.9880": [[3, 19], [0, 0], [0, 0], [0, 10509008], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 437880], [7665411007838, 24]], "client.9946": [[2, 0], [0, 0], [0, 0], [0, 890756], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 890756], [0, 1]], "client.9949": [[2, 0], [0, 0], [0, 0], [0, 695530], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 695530], [0, 1]]}}, "metrics": {"delayed_ranks": [], "mds.0": {"client.8492": [], "client.9880": [], "client.9946": [], "client.9949": []}}} 2024-04-25T14:13:58.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.282+0000 7fdb12200640 1 -- 172.21.15.86:0/1014043360 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fdaec06f570 msgr2=0x7fdaec071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:58.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.282+0000 7fdb12200640 1 --2- 172.21.15.86:0/1014043360 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fdaec06f570 0x7fdaec071a80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0ae000 tx=0x7fdafc0046d0 comp rx=0 tx=0).stop 2024-04-25T14:13:58.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.282+0000 7fdb12200640 1 -- 172.21.15.86:0/1014043360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb0c15ac90 msgr2=0x7fdb0c15b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:58.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.282+0000 7fdb12200640 1 --2- 172.21.15.86:0/1014043360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb0c15ac90 0x7fdb0c15b0a0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0b3f10 tx=0x7fdb0400f1b0 comp rx=0 tx=0).stop 2024-04-25T14:13:58.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.282+0000 7fdb12200640 1 -- 172.21.15.86:0/1014043360 shutdown_connections 2024-04-25T14:13:58.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.282+0000 7fdb12200640 1 --2- 172.21.15.86:0/1014043360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb0c15ac90 0x7fdb0c15b0a0 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:58.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.282+0000 7fdb12200640 1 --2- 172.21.15.86:0/1014043360 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fdaec06f570 0x7fdaec071a80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:58.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.282+0000 7fdb12200640 1 --2- 172.21.15.86:0/1014043360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb0c0cacf0 0x7fdb0c15a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:58.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.282+0000 7fdb12200640 1 --2- 172.21.15.86:0/1014043360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdaf0003a30 0x7fdb0c0ca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:58.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.282+0000 7fdb12200640 1 -- 172.21.15.86:0/1014043360 >> 172.21.15.86:0/1014043360 conn(0x7fdb0c0b4650 msgr2=0x7fdb0c0cdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:58.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.282+0000 7fdb12200640 1 -- 172.21.15.86:0/1014043360 shutdown_connections 2024-04-25T14:13:58.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.282+0000 7fdb12200640 1 -- 172.21.15.86:0/1014043360 wait complete. 2024-04-25T14:13:58.302 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:13:58.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.450+0000 7f8e11800640 1 Processor -- start 2024-04-25T14:13:58.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.450+0000 7f8e11800640 1 -- start start 2024-04-25T14:13:58.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.450+0000 7f8e11800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8e0c0d16e0 0x7f8e0c0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:58.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.450+0000 7f8e11800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e0c0730b0 0x7f8e0c0734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:58.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.450+0000 7f8e11800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e0c0d2030 con 0x7f8e0c0cfca0 2024-04-25T14:13:58.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.450+0000 7f8e11800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f8e0c070470 con 0x7f8e0c0d16e0 2024-04-25T14:13:58.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.450+0000 7f8e11800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8e0c072bd0 con 0x7f8e0c0730b0 2024-04-25T14:13:58.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.450+0000 7f8e0b400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e0c0730b0 0x7f8e0c0734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:58.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.450+0000 7f8e03e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8e0c0d16e0 0x7f8e0c0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:58.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.450+0000 7f8e0aa00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f8e0c0cfca0 0x7f8e0c06f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:50150/0 (socket says 172.21.15.86:50150) 2024-04-25T14:13:58.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.450+0000 7f8e0b400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e0c0730b0 0x7f8e0c0734c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:43836/0 (socket says 172.21.15.86:43836) 2024-04-25T14:13:58.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.450+0000 7f8e0aa00640 1 -- 172.21.15.86:0/3615894873 learned_addr learned my addr 172.21.15.86:0/3615894873 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:58.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.450+0000 7f8e0b400640 1 -- 172.21.15.86:0/3615894873 >> v2:172.21.15.155:3300/0 conn(0x7f8e0c0d16e0 msgr2=0x7f8e0c0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:58.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.450+0000 7f8e0a000640 1 -- 172.21.15.86:0/3615894873 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1764282236 0 0) 0x7f8dfc001120 con 0x7f8e0c0cfca0 2024-04-25T14:13:58.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.450+0000 7f8e0b400640 1 --2- 172.21.15.86:0/3615894873 >> v2:172.21.15.155:3300/0 conn(0x7f8e0c0d16e0 0x7f8e0c0d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:58.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.450+0000 7f8e0b400640 1 -- 172.21.15.86:0/3615894873 >> v1:172.21.15.155:6789/0 conn(0x7f8e0c0cfca0 legacy=0x7f8e0c06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:58.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.450+0000 7f8e0b400640 1 -- 172.21.15.86:0/3615894873 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0c06fbd0 con 0x7f8e0c0730b0 2024-04-25T14:13:58.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.454+0000 7f8e0b400640 1 --2- 172.21.15.86:0/3615894873 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e0c0730b0 0x7f8e0c0734c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8e04009d90 tx=0x7f8e0401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=733c5ff6b490e0d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:58.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.454+0000 7f8e0a000640 1 -- 172.21.15.86:0/3615894873 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e040b2070 con 0x7f8e0c0730b0 2024-04-25T14:13:58.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.454+0000 7f8e0a000640 1 -- 172.21.15.86:0/3615894873 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e040a5940 con 0x7f8e0c0730b0 2024-04-25T14:13:58.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.454+0000 7f8e0a000640 1 -- 172.21.15.86:0/3615894873 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e040a5d10 con 0x7f8e0c0730b0 2024-04-25T14:13:58.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.454+0000 7f8e11800640 1 -- 172.21.15.86:0/3615894873 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e0c0730b0 msgr2=0x7f8e0c0734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:58.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.454+0000 7f8e11800640 1 --2- 172.21.15.86:0/3615894873 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e0c0730b0 0x7f8e0c0734c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8e04009d90 tx=0x7f8e0401c970 comp rx=0 tx=0).stop 2024-04-25T14:13:58.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.454+0000 7f8e11800640 1 -- 172.21.15.86:0/3615894873 shutdown_connections 2024-04-25T14:13:58.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.454+0000 7f8e11800640 1 --2- 172.21.15.86:0/3615894873 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e0c0730b0 0x7f8e0c0734c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:58.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.454+0000 7f8e11800640 1 --2- 172.21.15.86:0/3615894873 >> v2:172.21.15.155:3300/0 conn(0x7f8e0c0d16e0 0x7f8e0c0d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:58.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.454+0000 7f8e11800640 1 -- 172.21.15.86:0/3615894873 >> 172.21.15.86:0/3615894873 conn(0x7f8e0c0b4650 msgr2=0x7f8e0c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:58.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.454+0000 7f8e11800640 1 -- 172.21.15.86:0/3615894873 shutdown_connections 2024-04-25T14:13:58.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.454+0000 7f8e11800640 1 -- 172.21.15.86:0/3615894873 wait complete. 2024-04-25T14:13:58.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.458+0000 7f8e11800640 1 Processor -- start 2024-04-25T14:13:58.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.458+0000 7f8e11800640 1 -- start start 2024-04-25T14:13:58.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.458+0000 7f8e11800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e0c0730b0 0x7f8e0c0af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:58.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.458+0000 7f8e11800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e0c0cfca0 0x7f8e0c0afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:58.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.458+0000 7f8e11800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e0c0d16e0 0x7f8e0c1552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:58.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.458+0000 7f8e11800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c0ce130 con 0x7f8e0c0730b0 2024-04-25T14:13:58.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.458+0000 7f8e11800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c0cdfb0 con 0x7f8e0c0cfca0 2024-04-25T14:13:58.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.458+0000 7f8e11800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8e0c0ce2b0 con 0x7f8e0c0d16e0 2024-04-25T14:13:58.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.458+0000 7f8e0aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e0c0730b0 0x7f8e0c0af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:58.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.458+0000 7f8e0aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e0c0730b0 0x7f8e0c0af690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38218/0 (socket says 172.21.15.86:38218) 2024-04-25T14:13:58.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.458+0000 7f8e0aa00640 1 -- 172.21.15.86:0/1141743648 learned_addr learned my addr 172.21.15.86:0/1141743648 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:58.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.458+0000 7f8e0b400640 1 --2- 172.21.15.86:0/1141743648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e0c0d16e0 0x7f8e0c1552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:58.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.458+0000 7f8e03e00640 1 --2- 172.21.15.86:0/1141743648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e0c0cfca0 0x7f8e0c0afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:58.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.458+0000 7f8e0aa00640 1 -- 172.21.15.86:0/1141743648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e0c0d16e0 msgr2=0x7f8e0c1552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:58.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.458+0000 7f8e0aa00640 1 --2- 172.21.15.86:0/1141743648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e0c0d16e0 0x7f8e0c1552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:58.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.458+0000 7f8e0aa00640 1 -- 172.21.15.86:0/1141743648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e0c0cfca0 msgr2=0x7f8e0c0afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:58.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.458+0000 7f8e0aa00640 1 --2- 172.21.15.86:0/1141743648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e0c0cfca0 0x7f8e0c0afbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:58.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.458+0000 7f8e0aa00640 1 -- 172.21.15.86:0/1141743648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0c154dd0 con 0x7f8e0c0730b0 2024-04-25T14:13:58.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.458+0000 7f8e0b400640 1 --2- 172.21.15.86:0/1141743648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e0c0d16e0 0x7f8e0c1552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:58.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.458+0000 7f8e0aa00640 1 --2- 172.21.15.86:0/1141743648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e0c0730b0 0x7f8e0c0af690 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc00c480 tx=0x7f8dfc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:58.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.462+0000 7f8e02a00640 1 -- 172.21.15.86:0/1141743648 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc018070 con 0x7f8e0c0730b0 2024-04-25T14:13:58.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.462+0000 7f8e02a00640 1 -- 172.21.15.86:0/1141743648 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dfc0074e0 con 0x7f8e0c0730b0 2024-04-25T14:13:58.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.462+0000 7f8e02a00640 1 -- 172.21.15.86:0/1141743648 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc0049f0 con 0x7f8e0c0730b0 2024-04-25T14:13:58.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.462+0000 7f8e11800640 1 -- 172.21.15.86:0/1141743648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e0c158ff0 con 0x7f8e0c0730b0 2024-04-25T14:13:58.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.462+0000 7f8e11800640 1 -- 172.21.15.86:0/1141743648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e0c1595b0 con 0x7f8e0c0730b0 2024-04-25T14:13:58.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.462+0000 7f8e11800640 1 -- 172.21.15.86:0/1141743648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e0c09f4b0 con 0x7f8e0c0730b0 2024-04-25T14:13:58.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.462+0000 7f8e02a00640 1 -- 172.21.15.86:0/1141743648 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f8dfc004b90 con 0x7f8e0c0730b0 2024-04-25T14:13:58.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.462+0000 7f8e02a00640 1 --2- 172.21.15.86:0/1141743648 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8de806f5a0 0x7f8de8071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:58.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.462+0000 7f8e02a00640 1 -- 172.21.15.86:0/1141743648 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f8dfc08ea50 con 0x7f8e0c0730b0 2024-04-25T14:13:58.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.462+0000 7f8e03e00640 1 --2- 172.21.15.86:0/1141743648 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8de806f5a0 0x7f8de8071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:58.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.466+0000 7f8e03e00640 1 --2- 172.21.15.86:0/1141743648 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8de806f5a0 0x7f8de8071ab0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c1fd780 tx=0x7f8df4008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:58.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.466+0000 7f8e02a00640 1 -- 172.21.15.86:0/1141743648 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8e0c0d2030 con 0x7f8e0c0730b0 2024-04-25T14:13:58.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.658+0000 7f8e11800640 1 -- 172.21.15.86:0/1141743648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e0c069d00 con 0x7f8e0c0730b0 2024-04-25T14:13:58.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.658+0000 7f8e02a00640 1 -- 172.21.15.86:0/1141743648 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) ==== 95+0+5202 (secure 0 0 0) 0x7f8dfc05b5e0 con 0x7f8e0c0730b0 2024-04-25T14:13:58.657 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:13:58.658 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":117,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112},{"gid":8387,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:13:36.638299+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10209},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:13:58.658 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 117 2024-04-25T14:13:58.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.658+0000 7f8e11800640 1 -- 172.21.15.86:0/1141743648 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8de806f5a0 msgr2=0x7f8de8071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:58.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.658+0000 7f8e11800640 1 --2- 172.21.15.86:0/1141743648 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8de806f5a0 0x7f8de8071ab0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c1fd780 tx=0x7f8df4008040 comp rx=0 tx=0).stop 2024-04-25T14:13:58.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.658+0000 7f8e11800640 1 -- 172.21.15.86:0/1141743648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e0c0730b0 msgr2=0x7f8e0c0af690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:58.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.658+0000 7f8e11800640 1 --2- 172.21.15.86:0/1141743648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e0c0730b0 0x7f8e0c0af690 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc00c480 tx=0x7f8dfc00c940 comp rx=0 tx=0).stop 2024-04-25T14:13:58.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.658+0000 7f8e11800640 1 -- 172.21.15.86:0/1141743648 shutdown_connections 2024-04-25T14:13:58.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.658+0000 7f8e11800640 1 --2- 172.21.15.86:0/1141743648 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8de806f5a0 0x7f8de8071ab0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:58.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.658+0000 7f8e11800640 1 --2- 172.21.15.86:0/1141743648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e0c0d16e0 0x7f8e0c1552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:58.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.658+0000 7f8e11800640 1 --2- 172.21.15.86:0/1141743648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e0c0cfca0 0x7f8e0c0afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:58.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.658+0000 7f8e11800640 1 --2- 172.21.15.86:0/1141743648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e0c0730b0 0x7f8e0c0af690 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:58.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.658+0000 7f8e11800640 1 -- 172.21.15.86:0/1141743648 >> 172.21.15.86:0/1141743648 conn(0x7f8e0c0b4650 msgr2=0x7f8e0c15bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:58.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.658+0000 7f8e11800640 1 -- 172.21.15.86:0/1141743648 shutdown_connections 2024-04-25T14:13:58.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.662+0000 7f8e11800640 1 -- 172.21.15.86:0/1141743648 wait complete. 2024-04-25T14:13:58.679 DEBUG:tasks.cephfs.test_mds_metrics:metrics={'version': 2, 'global_counters': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'counters': [], 'client_metadata': {'cephfs': {'client.8492': {'hostname': 'smithi155', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'kernel_version': '6.9.0-rc3-g806ec36b2112', 'IP': 'v1:192.168.0.1'}, 'client.9880': {'hostname': 'smithi086', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'kernel_version': '6.9.0-rc3-g806ec36b2112', 'IP': 'v1:172.21.15.86'}, 'client.9946': {'hostname': 'smithi086', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'IP': '172.21.15.86'}, 'client.9949': {'hostname': 'smithi086', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'IP': '172.21.15.86'}}}, 'global_metrics': {'cephfs': {'client.8492': [[22, 0], [0, 0], [0, 0], [0, 5266510], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 1316628], [7806089317574, 4]], 'client.9880': [[3, 19], [0, 0], [0, 0], [0, 10509008], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 437880], [7665411007838, 24]], 'client.9946': [[2, 0], [0, 0], [0, 0], [0, 890756], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 890756], [0, 1]], 'client.9949': [[2, 0], [0, 0], [0, 0], [0, 695530], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 695530], [0, 1]]}}, 'metrics': {'delayed_ranks': [], 'mds.0': {'client.8492': [], 'client.9880': [], 'client.9946': [], 'client.9949': []}}} 2024-04-25T14:13:58.680 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:13:58.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.830+0000 7ffa96600640 1 Processor -- start 2024-04-25T14:13:58.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.830+0000 7ffa96600640 1 -- start start 2024-04-25T14:13:58.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.830+0000 7ffa96600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffa900d1df0 0x7ffa900cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:58.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.830+0000 7ffa96600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ffa900cdcc0 0x7ffa900ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:58.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.830+0000 7ffa96600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa9015e5c0 con 0x7ffa90070330 2024-04-25T14:13:58.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.830+0000 7ffa96600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ffa9006fd40 con 0x7ffa900d1df0 2024-04-25T14:13:58.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.830+0000 7ffa96600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ffa9015e8a0 con 0x7ffa900cdcc0 2024-04-25T14:13:58.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.830+0000 7ffa87e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffa900d1df0 0x7ffa900cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:58.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.830+0000 7ffa87e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffa900d1df0 0x7ffa900cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35520/0 (socket says 172.21.15.86:35520) 2024-04-25T14:13:58.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.830+0000 7ffa87e00640 1 -- 172.21.15.86:0/3287817831 learned_addr learned my addr 172.21.15.86:0/3287817831 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:58.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.830+0000 7ffa8fe00640 1 --2- 172.21.15.86:0/3287817831 >> v2:172.21.15.155:3300/0 conn(0x7ffa900cdcc0 0x7ffa900ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:58.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.830+0000 7ffa87e00640 1 -- 172.21.15.86:0/3287817831 >> v2:172.21.15.155:3300/0 conn(0x7ffa900cdcc0 msgr2=0x7ffa900ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:58.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.830+0000 7ffa87e00640 1 --2- 172.21.15.86:0/3287817831 >> v2:172.21.15.155:3300/0 conn(0x7ffa900cdcc0 0x7ffa900ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:58.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.830+0000 7ffa87e00640 1 -- 172.21.15.86:0/3287817831 >> v1:172.21.15.86:6789/0 conn(0x7ffa90070330 legacy=0x7ffa900d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:58.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.830+0000 7ffa87e00640 1 -- 172.21.15.86:0/3287817831 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa9015eb80 con 0x7ffa900d1df0 2024-04-25T14:13:58.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.830+0000 7ffa8fe00640 1 --2- 172.21.15.86:0/3287817831 >> v2:172.21.15.155:3300/0 conn(0x7ffa900cdcc0 0x7ffa900ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:58.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.834+0000 7ffa87e00640 1 --2- 172.21.15.86:0/3287817831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa900d1df0 0x7ffa900cd780 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7ffa78009870 tx=0x7ffa7801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd85efa9d25e14bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:58.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.834+0000 7ffa8ea00640 1 -- 172.21.15.86:0/3287817831 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa780ab640 con 0x7ffa900d1df0 2024-04-25T14:13:58.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.834+0000 7ffa8ea00640 1 -- 172.21.15.86:0/3287817831 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa780b0070 con 0x7ffa900d1df0 2024-04-25T14:13:58.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.834+0000 7ffa8ea00640 1 -- 172.21.15.86:0/3287817831 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa780aba00 con 0x7ffa900d1df0 2024-04-25T14:13:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.834+0000 7ffa96600640 1 -- 172.21.15.86:0/3287817831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa900d1df0 msgr2=0x7ffa900cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.834+0000 7ffa96600640 1 --2- 172.21.15.86:0/3287817831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa900d1df0 0x7ffa900cd780 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7ffa78009870 tx=0x7ffa7801c460 comp rx=0 tx=0).stop 2024-04-25T14:13:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.834+0000 7ffa96600640 1 -- 172.21.15.86:0/3287817831 shutdown_connections 2024-04-25T14:13:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.834+0000 7ffa96600640 1 --2- 172.21.15.86:0/3287817831 >> v2:172.21.15.155:3300/0 conn(0x7ffa900cdcc0 0x7ffa900ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.834+0000 7ffa96600640 1 --2- 172.21.15.86:0/3287817831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa900d1df0 0x7ffa900cd780 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.834+0000 7ffa96600640 1 -- 172.21.15.86:0/3287817831 >> 172.21.15.86:0/3287817831 conn(0x7ffa900b4650 msgr2=0x7ffa900b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.834+0000 7ffa96600640 1 -- 172.21.15.86:0/3287817831 shutdown_connections 2024-04-25T14:13:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.834+0000 7ffa96600640 1 -- 172.21.15.86:0/3287817831 wait complete. 2024-04-25T14:13:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.838+0000 7ffa96600640 1 Processor -- start 2024-04-25T14:13:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.838+0000 7ffa96600640 1 -- start start 2024-04-25T14:13:58.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.838+0000 7ffa96600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa90070330 0x7ffa9015aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:58.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.838+0000 7ffa96600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa900cdcc0 0x7ffa90159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:58.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.838+0000 7ffa96600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa9015b0e0 0x7ffa9015a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:58.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.838+0000 7ffa96600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9015f4b0 con 0x7ffa900cdcc0 2024-04-25T14:13:58.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.838+0000 7ffa96600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9015f330 con 0x7ffa9015b0e0 2024-04-25T14:13:58.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.838+0000 7ffa96600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ffa90070740 con 0x7ffa90070330 2024-04-25T14:13:58.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.838+0000 7ffa87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa900cdcc0 0x7ffa90159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:58.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.838+0000 7ffa87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa900cdcc0 0x7ffa90159df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35532/0 (socket says 172.21.15.86:35532) 2024-04-25T14:13:58.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.838+0000 7ffa87e00640 1 -- 172.21.15.86:0/428791849 learned_addr learned my addr 172.21.15.86:0/428791849 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:58.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.838+0000 7ffa8f400640 1 --2- 172.21.15.86:0/428791849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa90070330 0x7ffa9015aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:58.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.838+0000 7ffa8fe00640 1 --2- 172.21.15.86:0/428791849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa9015b0e0 0x7ffa9015a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:58.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.838+0000 7ffa87e00640 1 -- 172.21.15.86:0/428791849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa90070330 msgr2=0x7ffa9015aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:58.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.838+0000 7ffa87e00640 1 --2- 172.21.15.86:0/428791849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa90070330 0x7ffa9015aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:58.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.838+0000 7ffa87e00640 1 -- 172.21.15.86:0/428791849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa9015b0e0 msgr2=0x7ffa9015a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:58.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.838+0000 7ffa87e00640 1 --2- 172.21.15.86:0/428791849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa9015b0e0 0x7ffa9015a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:58.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.838+0000 7ffa87e00640 1 -- 172.21.15.86:0/428791849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa900af340 con 0x7ffa900cdcc0 2024-04-25T14:13:58.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.838+0000 7ffa87e00640 1 --2- 172.21.15.86:0/428791849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa900cdcc0 0x7ffa90159df0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7ffa780ad040 tx=0x7ffa78019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:58.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.838+0000 7ffa87400640 1 -- 172.21.15.86:0/428791849 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa780bf410 con 0x7ffa900cdcc0 2024-04-25T14:13:58.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.838+0000 7ffa87400640 1 -- 172.21.15.86:0/428791849 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa780bf5d0 con 0x7ffa900cdcc0 2024-04-25T14:13:58.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.838+0000 7ffa96600640 1 -- 172.21.15.86:0/428791849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa900af620 con 0x7ffa900cdcc0 2024-04-25T14:13:58.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.838+0000 7ffa87400640 1 -- 172.21.15.86:0/428791849 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa780bf9c0 con 0x7ffa900cdcc0 2024-04-25T14:13:58.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.842+0000 7ffa96600640 1 -- 172.21.15.86:0/428791849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa900c2870 con 0x7ffa900cdcc0 2024-04-25T14:13:58.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.842+0000 7ffa96600640 1 -- 172.21.15.86:0/428791849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa900c2660 con 0x7ffa900cdcc0 2024-04-25T14:13:58.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.842+0000 7ffa87400640 1 -- 172.21.15.86:0/428791849 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7ffa780bd020 con 0x7ffa900cdcc0 2024-04-25T14:13:58.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.842+0000 7ffa87400640 1 --2- 172.21.15.86:0/428791849 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ffa6c06f570 0x7ffa6c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:58.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.842+0000 7ffa87400640 1 -- 172.21.15.86:0/428791849 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7ffa78129d10 con 0x7ffa900cdcc0 2024-04-25T14:13:58.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.842+0000 7ffa8f400640 1 --2- 172.21.15.86:0/428791849 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ffa6c06f570 0x7ffa6c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:58.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.846+0000 7ffa8f400640 1 --2- 172.21.15.86:0/428791849 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ffa6c06f570 0x7ffa6c071a80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffa90072200 tx=0x7ffa80009040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:58.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:58.850+0000 7ffa87400640 1 -- 172.21.15.86:0/428791849 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffa78016110 con 0x7ffa900cdcc0 2024-04-25T14:13:59.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.038+0000 7ffa96600640 1 -- 172.21.15.86:0/428791849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa900af0d0 con 0x7ffa900cdcc0 2024-04-25T14:13:59.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.038+0000 7ffa87400640 1 -- 172.21.15.86:0/428791849 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) ==== 95+0+5202 (secure 0 0 0) 0x7ffa780f60b0 con 0x7ffa900cdcc0 2024-04-25T14:13:59.039 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:13:59.039 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":117,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112},{"gid":8387,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:13:36.638299+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10209},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:13:59.039 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 117 2024-04-25T14:13:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.042+0000 7ffa96600640 1 -- 172.21.15.86:0/428791849 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ffa6c06f570 msgr2=0x7ffa6c071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.042+0000 7ffa96600640 1 --2- 172.21.15.86:0/428791849 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ffa6c06f570 0x7ffa6c071a80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffa90072200 tx=0x7ffa80009040 comp rx=0 tx=0).stop 2024-04-25T14:13:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.042+0000 7ffa96600640 1 -- 172.21.15.86:0/428791849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa900cdcc0 msgr2=0x7ffa90159df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.042+0000 7ffa96600640 1 --2- 172.21.15.86:0/428791849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa900cdcc0 0x7ffa90159df0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7ffa780ad040 tx=0x7ffa78019200 comp rx=0 tx=0).stop 2024-04-25T14:13:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.042+0000 7ffa96600640 1 -- 172.21.15.86:0/428791849 shutdown_connections 2024-04-25T14:13:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.042+0000 7ffa96600640 1 --2- 172.21.15.86:0/428791849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa9015b0e0 0x7ffa9015a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.042+0000 7ffa96600640 1 --2- 172.21.15.86:0/428791849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa900cdcc0 0x7ffa90159df0 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.042+0000 7ffa96600640 1 --2- 172.21.15.86:0/428791849 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ffa6c06f570 0x7ffa6c071a80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:59.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.042+0000 7ffa96600640 1 --2- 172.21.15.86:0/428791849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa90070330 0x7ffa9015aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:59.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.042+0000 7ffa96600640 1 -- 172.21.15.86:0/428791849 >> 172.21.15.86:0/428791849 conn(0x7ffa900b4650 msgr2=0x7ffa900c4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:59.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.042+0000 7ffa96600640 1 -- 172.21.15.86:0/428791849 shutdown_connections 2024-04-25T14:13:59.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.042+0000 7ffa96600640 1 -- 172.21.15.86:0/428791849 wait complete. 2024-04-25T14:13:59.061 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2024-04-25T14:13:59.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.210+0000 7f721a400640 1 Processor -- start 2024-04-25T14:13:59.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.210+0000 7f721a400640 1 -- start start 2024-04-25T14:13:59.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.210+0000 7f721a400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721415fc30 0x7f72140d2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:59.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.214+0000 7f721a400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f72140c8550 0x7f7214070d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:59.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.214+0000 7f721a400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f721406e0a0 con 0x7f72140c8960 2024-04-25T14:13:59.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.214+0000 7f721a400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f721406e380 con 0x7f72140c8550 2024-04-25T14:13:59.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.214+0000 7f721a400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f72140d1380 con 0x7f721415fc30 2024-04-25T14:13:59.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.214+0000 7f7219000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721415fc30 0x7f72140d2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:59.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.214+0000 7f720be00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f72140c8550 0x7f7214070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:59.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.214+0000 7f7219000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721415fc30 0x7f72140d2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:59324/0 (socket says 172.21.15.86:59324) 2024-04-25T14:13:59.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.214+0000 7f7219000640 1 -- 172.21.15.86:0/3105172163 learned_addr learned my addr 172.21.15.86:0/3105172163 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:59.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.214+0000 7f7219000640 1 -- 172.21.15.86:0/3105172163 >> v2:172.21.15.155:3300/0 conn(0x7f72140c8550 msgr2=0x7f7214070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:59.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.214+0000 7f7219000640 1 --2- 172.21.15.86:0/3105172163 >> v2:172.21.15.155:3300/0 conn(0x7f72140c8550 0x7f7214070d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:59.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.214+0000 7f7219000640 1 -- 172.21.15.86:0/3105172163 >> v1:172.21.15.155:6789/0 conn(0x7f72140c8960 legacy=0x7f72140d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:59.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.214+0000 7f7219000640 1 -- 172.21.15.86:0/3105172163 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72140d1660 con 0x7f721415fc30 2024-04-25T14:13:59.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.218+0000 7f7219000640 1 --2- 172.21.15.86:0/3105172163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721415fc30 0x7f72140d2a30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7204009870 tx=0x7f720401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ead40ec064e2afc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:59.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.218+0000 7f7213e00640 1 -- 172.21.15.86:0/3105172163 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72040a7070 con 0x7f721415fc30 2024-04-25T14:13:59.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.218+0000 7f7213e00640 1 -- 172.21.15.86:0/3105172163 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72040b0070 con 0x7f721415fc30 2024-04-25T14:13:59.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.218+0000 7f7213e00640 1 -- 172.21.15.86:0/3105172163 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72040ab890 con 0x7f721415fc30 2024-04-25T14:13:59.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.218+0000 7f721a400640 1 -- 172.21.15.86:0/3105172163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721415fc30 msgr2=0x7f72140d2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:59.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.218+0000 7f721a400640 1 --2- 172.21.15.86:0/3105172163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721415fc30 0x7f72140d2a30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7204009870 tx=0x7f720401c560 comp rx=0 tx=0).stop 2024-04-25T14:13:59.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.218+0000 7f721a400640 1 -- 172.21.15.86:0/3105172163 shutdown_connections 2024-04-25T14:13:59.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.218+0000 7f721a400640 1 --2- 172.21.15.86:0/3105172163 >> v2:172.21.15.155:3300/0 conn(0x7f72140c8550 0x7f7214070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:59.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.218+0000 7f721a400640 1 --2- 172.21.15.86:0/3105172163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721415fc30 0x7f72140d2a30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:59.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.218+0000 7f721a400640 1 -- 172.21.15.86:0/3105172163 >> 172.21.15.86:0/3105172163 conn(0x7f72140d3330 msgr2=0x7f72140d3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:59.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.218+0000 7f721a400640 1 -- 172.21.15.86:0/3105172163 shutdown_connections 2024-04-25T14:13:59.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.218+0000 7f721a400640 1 -- 172.21.15.86:0/3105172163 wait complete. 2024-04-25T14:13:59.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.218+0000 7f721a400640 1 Processor -- start 2024-04-25T14:13:59.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.218+0000 7f721a400640 1 -- start start 2024-04-25T14:13:59.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.222+0000 7f721a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72140c8550 0x7f721415a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:59.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.222+0000 7f721a400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72140c8960 0x7f7214156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:59.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.222+0000 7f721a400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721415fc30 0x7f7214157ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:59.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.222+0000 7f721a400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f721416d3e0 con 0x7f72140c8550 2024-04-25T14:13:59.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.222+0000 7f721a400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f72140716b0 con 0x7f72140c8960 2024-04-25T14:13:59.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.222+0000 7f721a400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f721416d560 con 0x7f721415fc30 2024-04-25T14:13:59.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.222+0000 7f7219000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72140c8550 0x7f721415a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.222+0000 7f7219000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72140c8550 0x7f721415a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35544/0 (socket says 172.21.15.86:35544) 2024-04-25T14:13:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.222+0000 7f7219000640 1 -- 172.21.15.86:0/822496323 learned_addr learned my addr 172.21.15.86:0/822496323 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:13:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.222+0000 7f720be00640 1 --2- 172.21.15.86:0/822496323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72140c8960 0x7f7214156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.222+0000 7f7219a00640 1 --2- 172.21.15.86:0/822496323 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721415fc30 0x7f7214157ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.222+0000 7f7219000640 1 -- 172.21.15.86:0/822496323 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721415fc30 msgr2=0x7f7214157ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.222+0000 7f7219000640 1 --2- 172.21.15.86:0/822496323 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721415fc30 0x7f7214157ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.222+0000 7f7219000640 1 -- 172.21.15.86:0/822496323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72140c8960 msgr2=0x7f7214156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.222+0000 7f7219000640 1 --2- 172.21.15.86:0/822496323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72140c8960 0x7f7214156d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.222+0000 7f7219000640 1 -- 172.21.15.86:0/822496323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721414d740 con 0x7f72140c8550 2024-04-25T14:13:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.222+0000 7f7219a00640 1 --2- 172.21.15.86:0/822496323 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721415fc30 0x7f7214157ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.222+0000 7f7219000640 1 --2- 172.21.15.86:0/822496323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72140c8550 0x7f721415a9f0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f72040ad040 tx=0x7f72140d3180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.222+0000 7f7211600640 1 -- 172.21.15.86:0/822496323 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72040bf610 con 0x7f72140c8550 2024-04-25T14:13:59.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.222+0000 7f7211600640 1 -- 172.21.15.86:0/822496323 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72040bf7d0 con 0x7f72140c8550 2024-04-25T14:13:59.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.222+0000 7f7211600640 1 -- 172.21.15.86:0/822496323 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72040bfbc0 con 0x7f72140c8550 2024-04-25T14:13:59.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.222+0000 7f721a400640 1 -- 172.21.15.86:0/822496323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f721414da20 con 0x7f72140c8550 2024-04-25T14:13:59.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.222+0000 7f721a400640 1 -- 172.21.15.86:0/822496323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f721414dfe0 con 0x7f72140c8550 2024-04-25T14:13:59.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.222+0000 7f721a400640 1 -- 172.21.15.86:0/822496323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7214076b80 con 0x7f72140c8550 2024-04-25T14:13:59.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.226+0000 7f7211600640 1 -- 172.21.15.86:0/822496323 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f72040bd020 con 0x7f72140c8550 2024-04-25T14:13:59.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.226+0000 7f7211600640 1 --2- 172.21.15.86:0/822496323 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f71f006f570 0x7f71f0071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:59.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.226+0000 7f7211600640 1 -- 172.21.15.86:0/822496323 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f720412a260 con 0x7f72140c8550 2024-04-25T14:13:59.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.226+0000 7f720be00640 1 --2- 172.21.15.86:0/822496323 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f71f006f570 0x7f71f0071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:59.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.226+0000 7f720be00640 1 --2- 172.21.15.86:0/822496323 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f71f006f570 0x7f71f0071a80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f72140d3e00 tx=0x7f71fc009190 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:59.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.230+0000 7f7211600640 1 -- 172.21.15.86:0/822496323 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7204016150 con 0x7f72140c8550 2024-04-25T14:13:59.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.422+0000 7f721a400640 1 -- 172.21.15.86:0/822496323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) -- 0x7f72140ced10 con 0x7f72140c8550 2024-04-25T14:13:59.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.810+0000 7f7211600640 1 -- 172.21.15.86:0/822496323 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v118) ==== 105+0+0 (secure 0 0 0) 0x7f72040f6600 con 0x7f72140c8550 2024-04-25T14:13:59.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.810+0000 7f721a400640 1 -- 172.21.15.86:0/822496323 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f71f006f570 msgr2=0x7f71f0071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:59.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.810+0000 7f721a400640 1 --2- 172.21.15.86:0/822496323 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f71f006f570 0x7f71f0071a80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f72140d3e00 tx=0x7f71fc009190 comp rx=0 tx=0).stop 2024-04-25T14:13:59.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.810+0000 7f721a400640 1 -- 172.21.15.86:0/822496323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72140c8550 msgr2=0x7f721415a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:59.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.810+0000 7f721a400640 1 --2- 172.21.15.86:0/822496323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72140c8550 0x7f721415a9f0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f72040ad040 tx=0x7f72140d3180 comp rx=0 tx=0).stop 2024-04-25T14:13:59.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.810+0000 7f721a400640 1 -- 172.21.15.86:0/822496323 shutdown_connections 2024-04-25T14:13:59.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.810+0000 7f721a400640 1 --2- 172.21.15.86:0/822496323 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f71f006f570 0x7f71f0071a80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:59.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.810+0000 7f721a400640 1 --2- 172.21.15.86:0/822496323 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721415fc30 0x7f7214157ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:59.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.810+0000 7f721a400640 1 --2- 172.21.15.86:0/822496323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72140c8960 0x7f7214156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:59.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.810+0000 7f721a400640 1 --2- 172.21.15.86:0/822496323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72140c8550 0x7f721415a9f0 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:59.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.810+0000 7f721a400640 1 -- 172.21.15.86:0/822496323 >> 172.21.15.86:0/822496323 conn(0x7f72140d3330 msgr2=0x7f72140cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:59.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.810+0000 7f721a400640 1 -- 172.21.15.86:0/822496323 shutdown_connections 2024-04-25T14:13:59.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.810+0000 7f721a400640 1 -- 172.21.15.86:0/822496323 wait complete. 2024-04-25T14:13:59.840 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:13:59.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.986+0000 7fa2dbe00640 1 Processor -- start 2024-04-25T14:13:59.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.986+0000 7fa2dbe00640 1 -- start start 2024-04-25T14:13:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2dbe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2d40cfca0 0x7fa2d406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2dbe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa2d40730b0 0x7fa2d40734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2dbe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2d4072bd0 con 0x7fa2d40d16e0 2024-04-25T14:13:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2dbe00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa2d4070470 con 0x7fa2d40730b0 2024-04-25T14:13:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2dbe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa2d406f930 con 0x7fa2d40cfca0 2024-04-25T14:13:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d8a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa2d40d16e0 0x7fa2d40d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:55542/0 (socket says 172.21.15.86:55542) 2024-04-25T14:13:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d8a00640 1 -- 172.21.15.86:0/2801015596 learned_addr learned my addr 172.21.15.86:0/2801015596 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:13:59.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d3e00640 1 -- 172.21.15.86:0/2801015596 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 569608016 0 0) 0x7fa2d4072bd0 con 0x7fa2d40d16e0 2024-04-25T14:13:59.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d3e00640 1 -- 172.21.15.86:0/2801015596 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2bc003530 con 0x7fa2d40d16e0 2024-04-25T14:13:59.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d9400640 1 --2- 172.21.15.86:0/2801015596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2d40cfca0 0x7fa2d406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:59.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d9e00640 1 --2- 172.21.15.86:0/2801015596 >> v2:172.21.15.155:3300/0 conn(0x7fa2d40730b0 0x7fa2d40734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:59.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d3e00640 1 -- 172.21.15.86:0/2801015596 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1598433322 0 0) 0x7fa2bc003530 con 0x7fa2d40d16e0 2024-04-25T14:13:59.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d3e00640 1 -- 172.21.15.86:0/2801015596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2d40cfca0 msgr2=0x7fa2d406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d3e00640 1 --2- 172.21.15.86:0/2801015596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2d40cfca0 0x7fa2d406f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d3e00640 1 -- 172.21.15.86:0/2801015596 >> v2:172.21.15.155:3300/0 conn(0x7fa2d40730b0 msgr2=0x7fa2d40734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d3e00640 1 --2- 172.21.15.86:0/2801015596 >> v2:172.21.15.155:3300/0 conn(0x7fa2d40730b0 0x7fa2d40734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d3e00640 1 -- 172.21.15.86:0/2801015596 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d406fbd0 con 0x7fa2d40d16e0 2024-04-25T14:13:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d3e00640 1 -- 172.21.15.86:0/2801015596 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fa2c0001680 con 0x7fa2d40d16e0 2024-04-25T14:13:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d9e00640 1 --2- 172.21.15.86:0/2801015596 >> v2:172.21.15.155:3300/0 conn(0x7fa2d40730b0 0x7fa2d40734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:13:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d3e00640 1 -- 172.21.15.86:0/2801015596 >> v1:172.21.15.86:6789/0 conn(0x7fa2d40d16e0 legacy=0x7fa2d40d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d3e00640 1 --2- 172.21.15.86:0/2801015596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2bc003a30 0x7fa2bc003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d3e00640 1 --2- 172.21.15.86:0/2801015596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2d40cfca0 0x7fa2bc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d3e00640 1 --2- 172.21.15.86:0/2801015596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2bc004d50 0x7fa2bc005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:13:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d3e00640 1 -- 172.21.15.86:0/2801015596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d4070470 con 0x7fa2bc004d50 2024-04-25T14:13:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d3e00640 1 -- 172.21.15.86:0/2801015596 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d406f930 con 0x7fa2bc003a30 2024-04-25T14:13:59.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d3e00640 1 -- 172.21.15.86:0/2801015596 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa2bc00ff10 con 0x7fa2d40cfca0 2024-04-25T14:13:59.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d9e00640 1 --2- 172.21.15.86:0/2801015596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2bc004d50 0x7fa2bc005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:59.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d8a00640 1 --2- 172.21.15.86:0/2801015596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2d40cfca0 0x7fa2bc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:59.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d9400640 1 --2- 172.21.15.86:0/2801015596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2bc003a30 0x7fa2bc003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:13:59.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d8a00640 1 -- 172.21.15.86:0/2801015596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2bc003a30 msgr2=0x7fa2bc003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:59.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d8a00640 1 --2- 172.21.15.86:0/2801015596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2bc003a30 0x7fa2bc003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:59.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d8a00640 1 -- 172.21.15.86:0/2801015596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2bc004d50 msgr2=0x7fa2bc005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:13:59.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d8a00640 1 --2- 172.21.15.86:0/2801015596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2bc004d50 0x7fa2bc005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:13:59.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d8a00640 1 -- 172.21.15.86:0/2801015596 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2bc003530 con 0x7fa2d40cfca0 2024-04-25T14:13:59.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.990+0000 7fa2d9e00640 1 --2- 172.21.15.86:0/2801015596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2bc004d50 0x7fa2bc005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:13:59.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.994+0000 7fa2d8a00640 1 --2- 172.21.15.86:0/2801015596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2d40cfca0 0x7fa2bc004810 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa2c000c290 tx=0x7fa2c001cd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a19f72684fa4fd09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:13:59.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.994+0000 7fa2d3e00640 1 -- 172.21.15.86:0/2801015596 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0007d60 con 0x7fa2d40cfca0 2024-04-25T14:13:59.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.994+0000 7fa2d3e00640 1 -- 172.21.15.86:0/2801015596 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c00a9020 con 0x7fa2d40cfca0 2024-04-25T14:13:59.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.994+0000 7fa2d3e00640 1 -- 172.21.15.86:0/2801015596 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c00a5b90 con 0x7fa2d40cfca0 2024-04-25T14:13:59.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.994+0000 7fa2dbe00640 1 -- 172.21.15.86:0/2801015596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2d40cfca0 msgr2=0x7fa2bc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:13:59.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.994+0000 7fa2dbe00640 1 --2- 172.21.15.86:0/2801015596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2d40cfca0 0x7fa2bc004810 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa2c000c290 tx=0x7fa2c001cd20 comp rx=0 tx=0).stop 2024-04-25T14:13:59.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.994+0000 7fa2d9e00640 1 -- 172.21.15.86:0/2801015596 reap_dead start 2024-04-25T14:13:59.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.994+0000 7fa2dbe00640 1 -- 172.21.15.86:0/2801015596 shutdown_connections 2024-04-25T14:13:59.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.994+0000 7fa2dbe00640 1 -- 172.21.15.86:0/2801015596 >> 172.21.15.86:0/2801015596 conn(0x7fa2d40b4650 msgr2=0x7fa2d40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:13:59.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.994+0000 7fa2dbe00640 1 -- 172.21.15.86:0/2801015596 shutdown_connections 2024-04-25T14:13:59.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.994+0000 7fa2dbe00640 1 -- 172.21.15.86:0/2801015596 wait complete. 2024-04-25T14:13:59.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.998+0000 7fa2dbe00640 1 Processor -- start 2024-04-25T14:13:59.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.998+0000 7fa2dbe00640 1 -- start start 2024-04-25T14:14:00.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.998+0000 7fa2dbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2d40cfca0 0x7fa2d415a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:00.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.998+0000 7fa2dbe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2d40c6430 0x7fa2d40c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:00.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.998+0000 7fa2dbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2d40c58d0 0x7fa2d40c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:00.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.998+0000 7fa2dbe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d406f3f0 con 0x7fa2d40c58d0 2024-04-25T14:14:00.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.998+0000 7fa2dbe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d4070190 con 0x7fa2d40cfca0 2024-04-25T14:14:00.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.998+0000 7fa2dbe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa2d406f570 con 0x7fa2d40c6430 2024-04-25T14:14:00.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.998+0000 7fa2d9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2d40c58d0 0x7fa2d40c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:00.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.998+0000 7fa2d9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2d40c58d0 0x7fa2d40c5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35556/0 (socket says 172.21.15.86:35556) 2024-04-25T14:14:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.998+0000 7fa2d9e00640 1 -- 172.21.15.86:0/3483383964 learned_addr learned my addr 172.21.15.86:0/3483383964 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.998+0000 7fa2d9400640 1 --2- 172.21.15.86:0/3483383964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2d40cfca0 0x7fa2d415a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.998+0000 7fa2d8a00640 1 --2- 172.21.15.86:0/3483383964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2d40c6430 0x7fa2d40c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.998+0000 7fa2d9e00640 1 -- 172.21.15.86:0/3483383964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2d40c6430 msgr2=0x7fa2d40c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.998+0000 7fa2d9e00640 1 --2- 172.21.15.86:0/3483383964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2d40c6430 0x7fa2d40c5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.998+0000 7fa2d9e00640 1 -- 172.21.15.86:0/3483383964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2d40cfca0 msgr2=0x7fa2d415a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:13:59.998+0000 7fa2d9e00640 1 --2- 172.21.15.86:0/3483383964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2d40cfca0 0x7fa2d415a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:00.002+0000 7fa2d9e00640 1 -- 172.21.15.86:0/3483383964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d4158a40 con 0x7fa2d40c58d0 2024-04-25T14:14:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:00.002+0000 7fa2d9e00640 1 --2- 172.21.15.86:0/3483383964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2d40c58d0 0x7fa2d40c5ce0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7fa2d406dd30 tx=0x7fa2cc007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:00.002+0000 7fa2d1600640 1 -- 172.21.15.86:0/3483383964 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc009c80 con 0x7fa2d40c58d0 2024-04-25T14:14:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:00.002+0000 7fa2d1600640 1 -- 172.21.15.86:0/3483383964 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2cc01a070 con 0x7fa2d40c58d0 2024-04-25T14:14:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:00.002+0000 7fa2d1600640 1 -- 172.21.15.86:0/3483383964 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc015660 con 0x7fa2d40c58d0 2024-04-25T14:14:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:00.002+0000 7fa2dbe00640 1 -- 172.21.15.86:0/3483383964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d415af70 con 0x7fa2d40c58d0 2024-04-25T14:14:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:00.002+0000 7fa2dbe00640 1 -- 172.21.15.86:0/3483383964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d415b530 con 0x7fa2d40c58d0 2024-04-25T14:14:00.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:00.002+0000 7fa2dbe00640 1 -- 172.21.15.86:0/3483383964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2d409f4b0 con 0x7fa2d40c58d0 2024-04-25T14:14:00.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:00.002+0000 7fa2d1600640 1 -- 172.21.15.86:0/3483383964 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7fa2cc015800 con 0x7fa2d40c58d0 2024-04-25T14:14:00.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:00.006+0000 7fa2d1600640 1 --2- 172.21.15.86:0/3483383964 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fa2b406f570 0x7fa2b4071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:00.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:00.006+0000 7fa2d1600640 1 -- 172.21.15.86:0/3483383964 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fa2cc08fe90 con 0x7fa2d40c58d0 2024-04-25T14:14:00.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:00.006+0000 7fa2d9400640 1 --2- 172.21.15.86:0/3483383964 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fa2b406f570 0x7fa2b4071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:00.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:00.006+0000 7fa2d9400640 1 --2- 172.21.15.86:0/3483383964 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fa2b406f570 0x7fa2b4071a80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4070710 tx=0x7fa2c8008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:00.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:00.010+0000 7fa2d1600640 1 -- 172.21.15.86:0/3483383964 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa2cc05c990 con 0x7fa2d40c58d0 2024-04-25T14:14:00.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:00.198+0000 7fa2dbe00640 1 -- 172.21.15.86:0/3483383964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2d4069d00 con 0x7fa2d40c58d0 2024-04-25T14:14:00.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:00.198+0000 7fa2d1600640 1 -- 172.21.15.86:0/3483383964 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) ==== 95+0+5218 (secure 0 0 0) 0x7fa2cc05c290 con 0x7fa2d40c58d0 2024-04-25T14:14:00.199 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:00.199 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":119,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:13:59.820422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:creating","state_seq":1,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:14:00.199 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 119 2024-04-25T14:14:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:00.202+0000 7fa2dbe00640 1 -- 172.21.15.86:0/3483383964 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fa2b406f570 msgr2=0x7fa2b4071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:00.202+0000 7fa2dbe00640 1 --2- 172.21.15.86:0/3483383964 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fa2b406f570 0x7fa2b4071a80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4070710 tx=0x7fa2c8008040 comp rx=0 tx=0).stop 2024-04-25T14:14:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:00.202+0000 7fa2dbe00640 1 -- 172.21.15.86:0/3483383964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2d40c58d0 msgr2=0x7fa2d40c5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:00.202+0000 7fa2dbe00640 1 --2- 172.21.15.86:0/3483383964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2d40c58d0 0x7fa2d40c5ce0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7fa2d406dd30 tx=0x7fa2cc007700 comp rx=0 tx=0).stop 2024-04-25T14:14:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:00.202+0000 7fa2dbe00640 1 -- 172.21.15.86:0/3483383964 shutdown_connections 2024-04-25T14:14:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:00.202+0000 7fa2dbe00640 1 --2- 172.21.15.86:0/3483383964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2d40c58d0 0x7fa2d40c5ce0 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:00.202+0000 7fa2dbe00640 1 --2- 172.21.15.86:0/3483383964 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fa2b406f570 0x7fa2b4071a80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:00.202+0000 7fa2dbe00640 1 --2- 172.21.15.86:0/3483383964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2d40c6430 0x7fa2d40c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:00.202+0000 7fa2dbe00640 1 --2- 172.21.15.86:0/3483383964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2d40cfca0 0x7fa2d415a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:00.202+0000 7fa2dbe00640 1 -- 172.21.15.86:0/3483383964 >> 172.21.15.86:0/3483383964 conn(0x7fa2d40b4650 msgr2=0x7fa2d40734c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:00.202+0000 7fa2dbe00640 1 -- 172.21.15.86:0/3483383964 shutdown_connections 2024-04-25T14:14:00.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:00.202+0000 7fa2dbe00640 1 -- 172.21.15.86:0/3483383964 wait complete. 2024-04-25T14:14:00.624 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 119, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:13:59.820422+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:14:00.624 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:creating 2024-04-25T14:14:01.625 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:01.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.782+0000 7fb866400640 1 Processor -- start 2024-04-25T14:14:01.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.782+0000 7fb866400640 1 -- start start 2024-04-25T14:14:01.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.786+0000 7fb866400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb86009ba30 0x7fb86015da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:01.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.786+0000 7fb866400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb86006e370 0x7fb86006e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:01.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.786+0000 7fb866400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb86006cf10 con 0x7fb8601616a0 2024-04-25T14:14:01.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.786+0000 7fb866400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb8600d27b0 con 0x7fb86006e370 2024-04-25T14:14:01.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.786+0000 7fb866400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb8600b23d0 con 0x7fb86009ba30 2024-04-25T14:14:01.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.786+0000 7fb85fe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb86006e370 0x7fb86006e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:01.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.786+0000 7fb865a00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fb8601616a0 0x7fb860161ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:56516/0 (socket says 172.21.15.86:56516) 2024-04-25T14:14:01.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.786+0000 7fb85fe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb86006e370 0x7fb86006e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:49354/0 (socket says 172.21.15.86:49354) 2024-04-25T14:14:01.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.786+0000 7fb865a00640 1 -- 172.21.15.86:0/3330554384 learned_addr learned my addr 172.21.15.86:0/3330554384 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:14:01.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.786+0000 7fb865000640 1 --2- 172.21.15.86:0/3330554384 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb86009ba30 0x7fb86015da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:01.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.786+0000 7fb85fe00640 1 -- 172.21.15.86:0/3330554384 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb86009ba30 msgr2=0x7fb86015da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:01.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.786+0000 7fb85fe00640 1 --2- 172.21.15.86:0/3330554384 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb86009ba30 0x7fb86015da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:01.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.786+0000 7fb85fe00640 1 -- 172.21.15.86:0/3330554384 >> v1:172.21.15.155:6789/0 conn(0x7fb8601616a0 legacy=0x7fb860161ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:01.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.786+0000 7fb85fe00640 1 -- 172.21.15.86:0/3330554384 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86006c670 con 0x7fb86006e370 2024-04-25T14:14:01.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.790+0000 7fb85fe00640 1 --2- 172.21.15.86:0/3330554384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb86006e370 0x7fb86006e780 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0097c0 tx=0x7fb84c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=442f56126ebb63bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:01.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.790+0000 7fb85f400640 1 -- 172.21.15.86:0/3330554384 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c0b0070 con 0x7fb86006e370 2024-04-25T14:14:01.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.790+0000 7fb85f400640 1 -- 172.21.15.86:0/3330554384 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb84c0ab600 con 0x7fb86006e370 2024-04-25T14:14:01.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.790+0000 7fb85f400640 1 -- 172.21.15.86:0/3330554384 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c0ab9d0 con 0x7fb86006e370 2024-04-25T14:14:01.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.790+0000 7fb866400640 1 -- 172.21.15.86:0/3330554384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb86006e370 msgr2=0x7fb86006e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:01.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.790+0000 7fb866400640 1 --2- 172.21.15.86:0/3330554384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb86006e370 0x7fb86006e780 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0097c0 tx=0x7fb84c01c410 comp rx=0 tx=0).stop 2024-04-25T14:14:01.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.790+0000 7fb866400640 1 -- 172.21.15.86:0/3330554384 shutdown_connections 2024-04-25T14:14:01.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.790+0000 7fb866400640 1 --2- 172.21.15.86:0/3330554384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb86006e370 0x7fb86006e780 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:01.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.790+0000 7fb866400640 1 --2- 172.21.15.86:0/3330554384 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb86009ba30 0x7fb86015da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:01.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.790+0000 7fb866400640 1 -- 172.21.15.86:0/3330554384 >> 172.21.15.86:0/3330554384 conn(0x7fb860073ba0 msgr2=0x7fb8600b76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:01.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.790+0000 7fb866400640 1 -- 172.21.15.86:0/3330554384 shutdown_connections 2024-04-25T14:14:01.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.790+0000 7fb866400640 1 -- 172.21.15.86:0/3330554384 wait complete. 2024-04-25T14:14:01.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.794+0000 7fb866400640 1 Processor -- start 2024-04-25T14:14:01.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.794+0000 7fb866400640 1 -- start start 2024-04-25T14:14:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.794+0000 7fb866400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb86006e370 0x7fb86013d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.794+0000 7fb866400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb86009ba30 0x7fb86013a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.794+0000 7fb866400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8601616a0 0x7fb86014fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.794+0000 7fb866400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb86006ecc0 con 0x7fb86006e370 2024-04-25T14:14:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.794+0000 7fb866400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb86009be40 con 0x7fb8601616a0 2024-04-25T14:14:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.794+0000 7fb866400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb8600d2c40 con 0x7fb86009ba30 2024-04-25T14:14:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.794+0000 7fb865000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb86006e370 0x7fb86013d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.794+0000 7fb865000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb86006e370 0x7fb86013d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35560/0 (socket says 172.21.15.86:35560) 2024-04-25T14:14:01.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.794+0000 7fb865000640 1 -- 172.21.15.86:0/1882845998 learned_addr learned my addr 172.21.15.86:0/1882845998 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:01.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.794+0000 7fb865a00640 1 --2- 172.21.15.86:0/1882845998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8601616a0 0x7fb86014fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:01.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.794+0000 7fb85fe00640 1 --2- 172.21.15.86:0/1882845998 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb86009ba30 0x7fb86013a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:01.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.794+0000 7fb865000640 1 -- 172.21.15.86:0/1882845998 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb86009ba30 msgr2=0x7fb86013a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:01.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.794+0000 7fb865000640 1 --2- 172.21.15.86:0/1882845998 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb86009ba30 0x7fb86013a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:01.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.794+0000 7fb865000640 1 -- 172.21.15.86:0/1882845998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8601616a0 msgr2=0x7fb86014fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:01.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.794+0000 7fb865000640 1 --2- 172.21.15.86:0/1882845998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8601616a0 0x7fb86014fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:01.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.794+0000 7fb865000640 1 -- 172.21.15.86:0/1882845998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86013ddc0 con 0x7fb86006e370 2024-04-25T14:14:01.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.794+0000 7fb865000640 1 --2- 172.21.15.86:0/1882845998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb86006e370 0x7fb86013d880 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fb85400e870 tx=0x7fb85400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:01.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.794+0000 7fb85cc00640 1 -- 172.21.15.86:0/1882845998 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb854017070 con 0x7fb86006e370 2024-04-25T14:14:01.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.794+0000 7fb85cc00640 1 -- 172.21.15.86:0/1882845998 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb85400c940 con 0x7fb86006e370 2024-04-25T14:14:01.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.794+0000 7fb85cc00640 1 -- 172.21.15.86:0/1882845998 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85400cd10 con 0x7fb86006e370 2024-04-25T14:14:01.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.794+0000 7fb866400640 1 -- 172.21.15.86:0/1882845998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8601f9ee0 con 0x7fb86006e370 2024-04-25T14:14:01.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.794+0000 7fb866400640 1 -- 172.21.15.86:0/1882845998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8601fa4a0 con 0x7fb86006e370 2024-04-25T14:14:01.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.798+0000 7fb866400640 1 -- 172.21.15.86:0/1882845998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8601fa6e0 con 0x7fb86006e370 2024-04-25T14:14:01.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.798+0000 7fb85cc00640 1 -- 172.21.15.86:0/1882845998 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7fb85401a070 con 0x7fb86006e370 2024-04-25T14:14:01.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.798+0000 7fb85cc00640 1 --2- 172.21.15.86:0/1882845998 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fb84006f5a0 0x7fb840071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:01.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.798+0000 7fb85cc00640 1 -- 172.21.15.86:0/1882845998 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fb85408f530 con 0x7fb86006e370 2024-04-25T14:14:01.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.798+0000 7fb85fe00640 1 --2- 172.21.15.86:0/1882845998 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fb84006f5a0 0x7fb840071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:01.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.798+0000 7fb85fe00640 1 --2- 172.21.15.86:0/1882845998 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fb84006f5a0 0x7fb840071ab0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0ad000 tx=0x7fb84c008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:01.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.802+0000 7fb85cc00640 1 -- 172.21.15.86:0/1882845998 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb85405b9d0 con 0x7fb86006e370 2024-04-25T14:14:01.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.998+0000 7fb866400640 1 -- 172.21.15.86:0/1882845998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb860076b80 con 0x7fb86006e370 2024-04-25T14:14:01.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.998+0000 7fb85cc00640 1 -- 172.21.15.86:0/1882845998 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) ==== 95+0+5221 (secure 0 0 0) 0x7fb85405fcd0 con 0x7fb86006e370 2024-04-25T14:14:01.997 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:01.997 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":120,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:00.821918+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[8387,10209]},"id":11}]} 2024-04-25T14:14:01.998 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 120 2024-04-25T14:14:01.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.998+0000 7fb866400640 1 -- 172.21.15.86:0/1882845998 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fb84006f5a0 msgr2=0x7fb840071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:01.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.998+0000 7fb866400640 1 --2- 172.21.15.86:0/1882845998 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fb84006f5a0 0x7fb840071ab0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0ad000 tx=0x7fb84c008040 comp rx=0 tx=0).stop 2024-04-25T14:14:01.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.998+0000 7fb866400640 1 -- 172.21.15.86:0/1882845998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb86006e370 msgr2=0x7fb86013d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:01.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.998+0000 7fb866400640 1 --2- 172.21.15.86:0/1882845998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb86006e370 0x7fb86013d880 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fb85400e870 tx=0x7fb85400ec70 comp rx=0 tx=0).stop 2024-04-25T14:14:01.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.998+0000 7fb866400640 1 -- 172.21.15.86:0/1882845998 shutdown_connections 2024-04-25T14:14:01.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.998+0000 7fb866400640 1 --2- 172.21.15.86:0/1882845998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8601616a0 0x7fb86014fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:01.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.998+0000 7fb866400640 1 --2- 172.21.15.86:0/1882845998 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fb84006f5a0 0x7fb840071ab0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:01.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.998+0000 7fb866400640 1 --2- 172.21.15.86:0/1882845998 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb86009ba30 0x7fb86013a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.998+0000 7fb866400640 1 --2- 172.21.15.86:0/1882845998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb86006e370 0x7fb86013d880 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.998+0000 7fb866400640 1 -- 172.21.15.86:0/1882845998 >> 172.21.15.86:0/1882845998 conn(0x7fb860073ba0 msgr2=0x7fb860152b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:02.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.998+0000 7fb866400640 1 -- 172.21.15.86:0/1882845998 shutdown_connections 2024-04-25T14:14:02.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:01.998+0000 7fb866400640 1 -- 172.21.15.86:0/1882845998 wait complete. 2024-04-25T14:14:02.421 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 120, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:00.821918+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [8387, 10209]} 2024-04-25T14:14:02.422 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2024-04-25T14:14:02.422 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T14:14:02.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.574+0000 7f2dc5000640 1 Processor -- start 2024-04-25T14:14:02.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.574+0000 7f2dc5000640 1 -- start start 2024-04-25T14:14:02.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.574+0000 7f2dc5000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2dc0070330 0x7f2dc00d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.574+0000 7f2dc5000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2dc00ce790 0x7f2dc006f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.574+0000 7f2dc5000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dc00d2030 con 0x7f2dc00d1c20 2024-04-25T14:14:02.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.574+0000 7f2dc5000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2dc015e5c0 con 0x7f2dc00ce790 2024-04-25T14:14:02.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.574+0000 7f2dc5000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2dc015e8a0 con 0x7f2dc0070330 2024-04-25T14:14:02.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.574+0000 7f2dbe800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2dc00ce790 0x7f2dc006f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.574+0000 7f2dbe800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2dc00ce790 0x7f2dc006f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35572/0 (socket says 172.21.15.86:35572) 2024-04-25T14:14:02.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.574+0000 7f2dbe800640 1 -- 172.21.15.86:0/2308693511 learned_addr learned my addr 172.21.15.86:0/2308693511 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:02.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.574+0000 7f2dbde00640 1 --2- 172.21.15.86:0/2308693511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2dc0070330 0x7f2dc00d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.574+0000 7f2dbe800640 1 -- 172.21.15.86:0/2308693511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2dc0070330 msgr2=0x7f2dc00d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:02.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.574+0000 7f2dbe800640 1 --2- 172.21.15.86:0/2308693511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2dc0070330 0x7f2dc00d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.574+0000 7f2dbe800640 1 -- 172.21.15.86:0/2308693511 >> v1:172.21.15.155:6789/0 conn(0x7f2dc00d1c20 legacy=0x7f2dc006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:02.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.574+0000 7f2dbe800640 1 -- 172.21.15.86:0/2308693511 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc015eb80 con 0x7f2dc00ce790 2024-04-25T14:14:02.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.578+0000 7f2dbe800640 1 --2- 172.21.15.86:0/2308693511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dc00ce790 0x7f2dc006f440 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f2db80098a0 tx=0x7f2db801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2a75add2817243 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:02.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.578+0000 7f2dbd400640 1 -- 172.21.15.86:0/2308693511 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db80b0070 con 0x7f2dc00ce790 2024-04-25T14:14:02.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.578+0000 7f2dbd400640 1 -- 172.21.15.86:0/2308693511 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2db80ab600 con 0x7f2dc00ce790 2024-04-25T14:14:02.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.578+0000 7f2dbd400640 1 -- 172.21.15.86:0/2308693511 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db80ab9d0 con 0x7f2dc00ce790 2024-04-25T14:14:02.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.578+0000 7f2dc5000640 1 -- 172.21.15.86:0/2308693511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dc00ce790 msgr2=0x7f2dc006f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:02.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.578+0000 7f2dc5000640 1 --2- 172.21.15.86:0/2308693511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dc00ce790 0x7f2dc006f440 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f2db80098a0 tx=0x7f2db801c4b0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.578+0000 7f2dc5000640 1 -- 172.21.15.86:0/2308693511 shutdown_connections 2024-04-25T14:14:02.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.578+0000 7f2dc5000640 1 --2- 172.21.15.86:0/2308693511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dc00ce790 0x7f2dc006f440 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.578+0000 7f2dc5000640 1 --2- 172.21.15.86:0/2308693511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2dc0070330 0x7f2dc00d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.578+0000 7f2dc5000640 1 -- 172.21.15.86:0/2308693511 >> 172.21.15.86:0/2308693511 conn(0x7f2dc00b4650 msgr2=0x7f2dc00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:02.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.578+0000 7f2dc5000640 1 -- 172.21.15.86:0/2308693511 shutdown_connections 2024-04-25T14:14:02.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.578+0000 7f2dc5000640 1 -- 172.21.15.86:0/2308693511 wait complete. 2024-04-25T14:14:02.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.582+0000 7f2dc5000640 1 Processor -- start 2024-04-25T14:14:02.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.582+0000 7f2dc5000640 1 -- start start 2024-04-25T14:14:02.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.582+0000 7f2dc5000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2dc0070330 0x7f2dc0156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.582+0000 7f2dc5000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2dc00ce790 0x7f2dc015c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.582+0000 7f2dc5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dc00d1c20 0x7f2dc015c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.582+0000 7f2dc5000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc00708c0 con 0x7f2dc00d1c20 2024-04-25T14:14:02.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.582+0000 7f2dc5000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc0070740 con 0x7f2dc0070330 2024-04-25T14:14:02.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.582+0000 7f2dc5000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2dc016afc0 con 0x7f2dc00ce790 2024-04-25T14:14:02.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.582+0000 7f2dbe800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dc00d1c20 0x7f2dc015c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.582+0000 7f2dbe800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dc00d1c20 0x7f2dc015c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35574/0 (socket says 172.21.15.86:35574) 2024-04-25T14:14:02.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.582+0000 7f2dbe800640 1 -- 172.21.15.86:0/2446096272 learned_addr learned my addr 172.21.15.86:0/2446096272 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:02.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.586+0000 7f2db5400640 1 --2- 172.21.15.86:0/2446096272 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2dc00ce790 0x7f2dc015c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.586+0000 7f2dbde00640 1 --2- 172.21.15.86:0/2446096272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2dc0070330 0x7f2dc0156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.586+0000 7f2dbe800640 1 -- 172.21.15.86:0/2446096272 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2dc00ce790 msgr2=0x7f2dc015c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:02.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.586+0000 7f2dbe800640 1 --2- 172.21.15.86:0/2446096272 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2dc00ce790 0x7f2dc015c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.586+0000 7f2dbe800640 1 -- 172.21.15.86:0/2446096272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2dc0070330 msgr2=0x7f2dc0156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:02.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.586+0000 7f2dbe800640 1 --2- 172.21.15.86:0/2446096272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2dc0070330 0x7f2dc0156020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.586+0000 7f2dbe800640 1 -- 172.21.15.86:0/2446096272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc0073db0 con 0x7f2dc00d1c20 2024-04-25T14:14:02.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.586+0000 7f2db5400640 1 --2- 172.21.15.86:0/2446096272 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2dc00ce790 0x7f2dc015c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:02.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.586+0000 7f2dbe800640 1 --2- 172.21.15.86:0/2446096272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dc00d1c20 0x7f2dc015c770 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f2db801a0e0 tx=0x7f2db8019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:02.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.586+0000 7f2db6a00640 1 -- 172.21.15.86:0/2446096272 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db80b0040 con 0x7f2dc00d1c20 2024-04-25T14:14:02.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.586+0000 7f2db6a00640 1 -- 172.21.15.86:0/2446096272 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2db80b9070 con 0x7f2dc00d1c20 2024-04-25T14:14:02.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.586+0000 7f2db6a00640 1 -- 172.21.15.86:0/2446096272 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db80b4640 con 0x7f2dc00d1c20 2024-04-25T14:14:02.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.586+0000 7f2dc5000640 1 -- 172.21.15.86:0/2446096272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dc00c8f00 con 0x7f2dc00d1c20 2024-04-25T14:14:02.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.586+0000 7f2dc5000640 1 -- 172.21.15.86:0/2446096272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dc0074020 con 0x7f2dc00d1c20 2024-04-25T14:14:02.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.586+0000 7f2db6a00640 1 -- 172.21.15.86:0/2446096272 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f2db80ab4c0 con 0x7f2dc00d1c20 2024-04-25T14:14:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.590+0000 7f2db6a00640 1 --2- 172.21.15.86:0/2446096272 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f2d9c06f570 0x7f2d9c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.590+0000 7f2db6a00640 1 -- 172.21.15.86:0/2446096272 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f2db8129490 con 0x7f2dc00d1c20 2024-04-25T14:14:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.590+0000 7f2dcc78a000 1 Processor -- start 2024-04-25T14:14:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.590+0000 7f2dbde00640 1 --2- 172.21.15.86:0/2446096272 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f2d9c06f570 0x7f2d9c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.590+0000 7f2dcc78a000 1 -- start start 2024-04-25T14:14:02.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.590+0000 7f2dbde00640 1 --2- 172.21.15.86:0/2446096272 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f2d9c06f570 0x7f2d9c071a80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0072200 tx=0x7f2db000b040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:02.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.590+0000 7f2dcc78a000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a2d8d1bc20 0x55a2d8e7e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.590+0000 7f2dcc78a000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2d8d07ea0 0x55a2d8e7e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.590+0000 7f2dcc78a000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2d8900280 0x55a2d8e7edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.590+0000 7f2dcc78a000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a2d8ce9160 con 0x55a2d8d07ea0 2024-04-25T14:14:02.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.590+0000 7f2dcc78a000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a2d8c7ee20 con 0x55a2d8900280 2024-04-25T14:14:02.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.590+0000 7f2dcc78a000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55a2d8c37a20 con 0x55a2d8d1bc20 2024-04-25T14:14:02.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.590+0000 7f2db5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2d8d07ea0 0x55a2d8e7e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.590+0000 7f2dbde00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a2d8d1bc20 0x55a2d8e7e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.590+0000 7f2db5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2d8d07ea0 0x55a2d8e7e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35584/0 (socket says 172.21.15.86:35584) 2024-04-25T14:14:02.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.590+0000 7f2db5400640 1 -- 172.21.15.86:0/2194367879 learned_addr learned my addr 172.21.15.86:0/2194367879 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:02.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.590+0000 7f2dbe800640 1 --2- 172.21.15.86:0/2194367879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2d8900280 0x55a2d8e7edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.590+0000 7f2dbde00640 1 -- 172.21.15.86:0/2194367879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2d8900280 msgr2=0x55a2d8e7edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:02.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.590+0000 7f2dbde00640 1 --2- 172.21.15.86:0/2194367879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2d8900280 0x55a2d8e7edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.590+0000 7f2dbde00640 1 -- 172.21.15.86:0/2194367879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2d8d07ea0 msgr2=0x55a2d8e7e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:02.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.590+0000 7f2dbde00640 1 --2- 172.21.15.86:0/2194367879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2d8d07ea0 0x55a2d8e7e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.590+0000 7f2dbde00640 1 -- 172.21.15.86:0/2194367879 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2d8c3c100 con 0x55a2d8d1bc20 2024-04-25T14:14:02.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.590+0000 7f2db5400640 1 --2- 172.21.15.86:0/2194367879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2d8d07ea0 0x55a2d8e7e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:02.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.590+0000 7f2dbde00640 1 --2- 172.21.15.86:0/2194367879 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a2d8d1bc20 0x55a2d8e7e340 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2db0008010 tx=0x7f2db0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1404c25c38d7646 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:02.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.590+0000 7f2daea00640 1 -- 172.21.15.86:0/2194367879 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db0017830 con 0x55a2d8d1bc20 2024-04-25T14:14:02.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.590+0000 7f2daea00640 1 -- 172.21.15.86:0/2194367879 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2db0026070 con 0x55a2d8d1bc20 2024-04-25T14:14:02.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.590+0000 7f2daea00640 1 -- 172.21.15.86:0/2194367879 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db0017c10 con 0x55a2d8d1bc20 2024-04-25T14:14:02.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2dcc78a000 1 -- 172.21.15.86:0/2194367879 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a2d8d1bc20 msgr2=0x55a2d8e7e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:02.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2dcc78a000 1 --2- 172.21.15.86:0/2194367879 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a2d8d1bc20 0x55a2d8e7e340 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2db0008010 tx=0x7f2db0014000 comp rx=0 tx=0).stop 2024-04-25T14:14:02.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2dcc78a000 1 -- 172.21.15.86:0/2194367879 shutdown_connections 2024-04-25T14:14:02.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2dcc78a000 1 --2- 172.21.15.86:0/2194367879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2d8900280 0x55a2d8e7edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2dcc78a000 1 --2- 172.21.15.86:0/2194367879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2d8d07ea0 0x55a2d8e7e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2dcc78a000 1 --2- 172.21.15.86:0/2194367879 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a2d8d1bc20 0x55a2d8e7e340 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2dcc78a000 1 -- 172.21.15.86:0/2194367879 >> 172.21.15.86:0/2194367879 conn(0x55a2d893fff0 msgr2=0x55a2d8e2f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:02.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2dcc78a000 1 -- 172.21.15.86:0/2194367879 shutdown_connections 2024-04-25T14:14:02.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2dcc78a000 1 -- 172.21.15.86:0/2194367879 wait complete. 2024-04-25T14:14:02.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2dcc78a000 1 -- start start 2024-04-25T14:14:02.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2dcc78a000 1 Processor -- start 2024-04-25T14:14:02.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2dcc78a000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2d8900280 0x55a2d8e83810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2dcc78a000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2d8d07ea0 0x55a2d8e83d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2dcc78a000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a2d8d1bc20 0x55a2d8e3e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2dcc78a000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a2d8c8c220 con 0x55a2d8900280 2024-04-25T14:14:02.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2dcc78a000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a2d8c22ea0 con 0x55a2d8d07ea0 2024-04-25T14:14:02.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2dcc78a000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55a2d8c79b00 con 0x55a2d8d1bc20 2024-04-25T14:14:02.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2dcc78a000 10 client.0 _ll_register_callbacks cb 0x55a2d8c8bdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:02.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2dbde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2d8900280 0x55a2d8e83810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2dbde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2d8900280 0x55a2d8e83810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35586/0 (socket says 172.21.15.86:35586) 2024-04-25T14:14:02.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2dbde00640 1 -- 172.21.15.86:0/3538520868 learned_addr learned my addr 172.21.15.86:0/3538520868 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:02.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2dbe800640 1 --2- 172.21.15.86:0/3538520868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a2d8d1bc20 0x55a2d8e3e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2db5400640 1 --2- 172.21.15.86:0/3538520868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2d8d07ea0 0x55a2d8e83d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2dbde00640 1 -- 172.21.15.86:0/3538520868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a2d8d1bc20 msgr2=0x55a2d8e3e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:02.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2dbde00640 1 --2- 172.21.15.86:0/3538520868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a2d8d1bc20 0x55a2d8e3e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2dbde00640 1 -- 172.21.15.86:0/3538520868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2d8d07ea0 msgr2=0x55a2d8e83d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:02.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2dbde00640 1 --2- 172.21.15.86:0/3538520868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2d8d07ea0 0x55a2d8e83d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2dbde00640 1 -- 172.21.15.86:0/3538520868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2d8c1e500 con 0x55a2d8900280 2024-04-25T14:14:02.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2db5400640 1 --2- 172.21.15.86:0/3538520868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2d8d07ea0 0x55a2d8e83d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:14:02.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.594+0000 7f2dbe800640 1 --2- 172.21.15.86:0/3538520868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a2d8d1bc20 0x55a2d8e3e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:14:02.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2dbde00640 1 --2- 172.21.15.86:0/3538520868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2d8900280 0x55a2d8e83810 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f2db0016fd0 tx=0x7f2db0004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3dfb7e019cdd1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:02.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2da3400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:14:02.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2da3400640 1 -- 172.21.15.86:0/3538520868 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db002a600 con 0x55a2d8900280 2024-04-25T14:14:02.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2da3400640 1 -- 172.21.15.86:0/3538520868 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f2db0035d80 con 0x55a2d8900280 2024-04-25T14:14:02.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2dcc78a000 1 -- 172.21.15.86:0/3538520868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2db0026070 con 0x55a2d8900280 2024-04-25T14:14:02.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2da3400640 1 -- 172.21.15.86:0/3538520868 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) ==== 24+0+0 (secure 0 0 0) 0x7f2db003c050 con 0x55a2d8900280 2024-04-25T14:14:02.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2dcc78a000 10 client.10515 fetch_fsmap learned FSMap version 120 2024-04-25T14:14:02.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2dcc78a000 1 -- 172.21.15.86:0/3538520868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=2+}) -- 0x55a2d8c04ea0 con 0x55a2d8900280 2024-04-25T14:14:02.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2da3400640 1 -- 172.21.15.86:0/3538520868 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2db0026070 con 0x55a2d8900280 2024-04-25T14:14:02.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2da3400640 1 -- 172.21.15.86:0/3538520868 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 120) ==== 3514+0+0 (secure 0 0 0) 0x7f2db003f020 con 0x55a2d8900280 2024-04-25T14:14:02.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2dcc78a000 10 client.10515 fetch_fsmap finished waiting for FSMap version 120 2024-04-25T14:14:02.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2dcc78a000 10 client.10515 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T14:14:02.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2dcc78a000 20 client.10515 populate_metadata read hostname 'smithi086' 2024-04-25T14:14:02.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2dcc78a000 1 --2- 172.21.15.86:0/3538520868 >> [v2:172.21.15.155:6834/4231378267,v1:172.21.15.155:6835/4231378267] conn(0x55a2d89247d0 0x55a2d8e3f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2dcc78a000 4 client.10515 mds_command: new command op to 8387 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:14:02.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2dcc78a000 1 -- 172.21.15.86:0/3538520868 --> [v2:172.21.15.155:6834/4231378267,v1:172.21.15.155:6835/4231378267] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2d8c17be0 con 0x55a2d89247d0 2024-04-25T14:14:02.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2db5400640 1 --2- 172.21.15.86:0/3538520868 >> [v2:172.21.15.155:6834/4231378267,v1:172.21.15.155:6835/4231378267] conn(0x55a2d89247d0 0x55a2d8e3f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2db5400640 1 --2- 172.21.15.86:0/3538520868 >> [v2:172.21.15.155:6834/4231378267,v1:172.21.15.155:6835/4231378267] conn(0x55a2d89247d0 0x55a2d8e3f1b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=28a33e8cc832a718 server_cookie=42b4edab5dec7f73 in_seq=0 out_seq=0 2024-04-25T14:14:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2da3400640 10 client.10515 ms_handle_connect on v2:172.21.15.155:6834/4231378267 2024-04-25T14:14:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2da3400640 1 -- 172.21.15.86:0/3538520868 <== mds.1 v2:172.21.15.155:6834/4231378267 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a2d8c17be0 con 0x55a2d89247d0 2024-04-25T14:14:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2da3400640 10 client.10515 handle_command_reply: tid=0 2024-04-25T14:14:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2dcc78a000 1 client.10515 shutdown 2024-04-25T14:14:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2dcc78a000 1 -- 172.21.15.86:0/3538520868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2d8900280 msgr2=0x55a2d8e83810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2dcc78a000 1 --2- 172.21.15.86:0/3538520868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2d8900280 0x55a2d8e83810 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f2db0016fd0 tx=0x7f2db0004a50 comp rx=0 tx=0).stop 2024-04-25T14:14:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2dcc78a000 1 -- 172.21.15.86:0/3538520868 shutdown_connections 2024-04-25T14:14:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2dcc78a000 1 --2- 172.21.15.86:0/3538520868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a2d8d1bc20 0x55a2d8e3e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2dcc78a000 1 --2- 172.21.15.86:0/3538520868 >> [v2:172.21.15.155:6834/4231378267,v1:172.21.15.155:6835/4231378267] conn(0x55a2d89247d0 0x55a2d8e3f1b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2dcc78a000 1 --2- 172.21.15.86:0/3538520868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2d8d07ea0 0x55a2d8e83d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2dcc78a000 1 --2- 172.21.15.86:0/3538520868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2d8900280 0x55a2d8e83810 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2da3400640 0 client.10515 ms_handle_reset on v2:172.21.15.155:6834/4231378267 2024-04-25T14:14:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2dcc78a000 1 -- 172.21.15.86:0/3538520868 >> 172.21.15.86:0/3538520868 conn(0x55a2d893fff0 msgr2=0x55a2d8e361b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:02.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2dcc78a000 1 -- 172.21.15.86:0/3538520868 shutdown_connections 2024-04-25T14:14:02.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.598+0000 7f2dcc78a000 1 -- 172.21.15.86:0/3538520868 wait complete. 2024-04-25T14:14:02.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.602+0000 7f2dcc78a000 20 client.10515 trim_cache size 0 max 16384 2024-04-25T14:14:02.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dcc78a000 1 Processor -- start 2024-04-25T14:14:02.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dcc78a000 1 -- start start 2024-04-25T14:14:02.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dcc78a000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2d8900280 0x55a2d8e939b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dcc78a000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a2d8d07ea0 0x55a2d8e93ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dcc78a000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2d8d1bc20 0x55a2d8e85820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dcc78a000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc0202610 con 0x55a2d8900280 2024-04-25T14:14:02.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dcc78a000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc00b14f0 con 0x55a2d8d1bc20 2024-04-25T14:14:02.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dcc78a000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2dc00b1b40 con 0x55a2d8d07ea0 2024-04-25T14:14:02.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dbde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2d8900280 0x55a2d8e939b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dbde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2d8900280 0x55a2d8e939b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35596/0 (socket says 172.21.15.86:35596) 2024-04-25T14:14:02.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dbde00640 1 -- 172.21.15.86:0/2630471639 learned_addr learned my addr 172.21.15.86:0/2630471639 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:02.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dbe800640 1 --2- 172.21.15.86:0/2630471639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2d8d1bc20 0x55a2d8e85820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2db5400640 1 --2- 172.21.15.86:0/2630471639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a2d8d07ea0 0x55a2d8e93ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dbde00640 1 -- 172.21.15.86:0/2630471639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a2d8d07ea0 msgr2=0x55a2d8e93ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:02.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dbde00640 1 --2- 172.21.15.86:0/2630471639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a2d8d07ea0 0x55a2d8e93ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dbde00640 1 -- 172.21.15.86:0/2630471639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2d8d1bc20 msgr2=0x55a2d8e85820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:02.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dbde00640 1 --2- 172.21.15.86:0/2630471639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2d8d1bc20 0x55a2d8e85820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dbde00640 1 -- 172.21.15.86:0/2630471639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2d8c37c70 con 0x55a2d8900280 2024-04-25T14:14:02.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dbde00640 1 --2- 172.21.15.86:0/2630471639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2d8900280 0x55a2d8e939b0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f2db0007dc0 tx=0x7f2db002b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93face4f1e14732b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:02.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dae000640 1 -- 172.21.15.86:0/2630471639 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db0045070 con 0x55a2d8900280 2024-04-25T14:14:02.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dae000640 1 -- 172.21.15.86:0/2630471639 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2db0026040 con 0x55a2d8900280 2024-04-25T14:14:02.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dae000640 1 -- 172.21.15.86:0/2630471639 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db002ac80 con 0x55a2d8900280 2024-04-25T14:14:02.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dcc78a000 1 -- 172.21.15.86:0/2630471639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2d8900280 msgr2=0x55a2d8e939b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:02.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dcc78a000 1 --2- 172.21.15.86:0/2630471639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2d8900280 0x55a2d8e939b0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f2db0007dc0 tx=0x7f2db002b000 comp rx=0 tx=0).stop 2024-04-25T14:14:02.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dcc78a000 1 -- 172.21.15.86:0/2630471639 shutdown_connections 2024-04-25T14:14:02.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dcc78a000 1 --2- 172.21.15.86:0/2630471639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2d8d1bc20 0x55a2d8e85820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dcc78a000 1 --2- 172.21.15.86:0/2630471639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a2d8d07ea0 0x55a2d8e93ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dcc78a000 1 --2- 172.21.15.86:0/2630471639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2d8900280 0x55a2d8e939b0 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dcc78a000 1 -- 172.21.15.86:0/2630471639 >> 172.21.15.86:0/2630471639 conn(0x55a2d893fff0 msgr2=0x55a2d8e2fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:02.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dcc78a000 1 -- 172.21.15.86:0/2630471639 shutdown_connections 2024-04-25T14:14:02.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dcc78a000 1 -- 172.21.15.86:0/2630471639 wait complete. 2024-04-25T14:14:02.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dcc78a000 1 -- start start 2024-04-25T14:14:02.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dcc78a000 1 Processor -- start 2024-04-25T14:14:02.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dcc78a000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2d8900280 0x55a2d8e892b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dcc78a000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2d8d07ea0 0x55a2d8eac310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dcc78a000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a2d8d1bc20 0x55a2d8eac850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dcc78a000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a2d8c7e130 con 0x55a2d8900280 2024-04-25T14:14:02.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dcc78a000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a2d8c147d0 con 0x55a2d8d07ea0 2024-04-25T14:14:02.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dcc78a000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55a2d8c83450 con 0x55a2d8d1bc20 2024-04-25T14:14:02.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.622+0000 7f2dcc78a000 10 client.0 _ll_register_callbacks cb 0x7f2da8001090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:02.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dbde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2d8900280 0x55a2d8e892b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dbde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2d8900280 0x55a2d8e892b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35602/0 (socket says 172.21.15.86:35602) 2024-04-25T14:14:02.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dbde00640 1 -- 172.21.15.86:0/1670442064 learned_addr learned my addr 172.21.15.86:0/1670442064 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:02.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dbe800640 1 --2- 172.21.15.86:0/1670442064 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a2d8d1bc20 0x55a2d8eac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2db5400640 1 --2- 172.21.15.86:0/1670442064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2d8d07ea0 0x55a2d8eac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dbde00640 1 -- 172.21.15.86:0/1670442064 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a2d8d1bc20 msgr2=0x55a2d8eac850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:02.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dbde00640 1 --2- 172.21.15.86:0/1670442064 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a2d8d1bc20 0x55a2d8eac850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dbde00640 1 -- 172.21.15.86:0/1670442064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2d8d07ea0 msgr2=0x55a2d8eac310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:02.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dbde00640 1 --2- 172.21.15.86:0/1670442064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2d8d07ea0 0x55a2d8eac310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dbde00640 1 -- 172.21.15.86:0/1670442064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2d8d01180 con 0x55a2d8900280 2024-04-25T14:14:02.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2db5400640 1 --2- 172.21.15.86:0/1670442064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2d8d07ea0 0x55a2d8eac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:02.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dbde00640 1 --2- 172.21.15.86:0/1670442064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2d8900280 0x55a2d8e892b0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f2db0007e40 tx=0x7f2db0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf93b413f602161 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:02.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2da3400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:14:02.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2da3400640 1 -- 172.21.15.86:0/1670442064 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db0045030 con 0x55a2d8900280 2024-04-25T14:14:02.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2da3400640 1 -- 172.21.15.86:0/1670442064 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f2db0046070 con 0x55a2d8900280 2024-04-25T14:14:02.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dcc78a000 1 -- 172.21.15.86:0/1670442064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2db0026040 con 0x55a2d8900280 2024-04-25T14:14:02.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2da3400640 1 -- 172.21.15.86:0/1670442064 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) ==== 24+0+0 (secure 0 0 0) 0x7f2db0053050 con 0x55a2d8900280 2024-04-25T14:14:02.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dcc78a000 10 client.10521 fetch_fsmap learned FSMap version 120 2024-04-25T14:14:02.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dcc78a000 1 -- 172.21.15.86:0/1670442064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=2+}) -- 0x55a2d89380c0 con 0x55a2d8900280 2024-04-25T14:14:02.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2da3400640 1 -- 172.21.15.86:0/1670442064 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2db0026040 con 0x55a2d8900280 2024-04-25T14:14:02.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2da3400640 1 -- 172.21.15.86:0/1670442064 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 120) ==== 3514+0+0 (secure 0 0 0) 0x7f2db0056020 con 0x55a2d8900280 2024-04-25T14:14:02.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dcc78a000 10 client.10521 fetch_fsmap finished waiting for FSMap version 120 2024-04-25T14:14:02.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dcc78a000 10 client.10521 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T14:14:02.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dcc78a000 20 client.10521 populate_metadata read hostname 'smithi086' 2024-04-25T14:14:02.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dcc78a000 1 --2- 172.21.15.86:0/1670442064 >> [v2:172.21.15.155:6834/4231378267,v1:172.21.15.155:6835/4231378267] conn(0x55a2d8912830 0x55a2d8eef740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dcc78a000 4 client.10521 mds_command: new command op to 8387 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:14:02.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dcc78a000 1 -- 172.21.15.86:0/1670442064 --> [v2:172.21.15.155:6834/4231378267,v1:172.21.15.155:6835/4231378267] -- command(tid 0: {"prefix": "status"}) -- 0x55a2d8c8ca30 con 0x55a2d8912830 2024-04-25T14:14:02.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2db5400640 1 --2- 172.21.15.86:0/1670442064 >> [v2:172.21.15.155:6834/4231378267,v1:172.21.15.155:6835/4231378267] conn(0x55a2d8912830 0x55a2d8eef740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2db5400640 1 --2- 172.21.15.86:0/1670442064 >> [v2:172.21.15.155:6834/4231378267,v1:172.21.15.155:6835/4231378267] conn(0x55a2d8912830 0x55a2d8eef740 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8642571b41c25410 server_cookie=54201ea440f865f1 in_seq=0 out_seq=0 2024-04-25T14:14:02.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2da3400640 10 client.10521 ms_handle_connect on v2:172.21.15.155:6834/4231378267 2024-04-25T14:14:02.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2da3400640 1 -- 172.21.15.86:0/1670442064 <== mds.1 v2:172.21.15.155:6834/4231378267 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55a2d8c8ca30 con 0x55a2d8912830 2024-04-25T14:14:02.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2da3400640 10 client.10521 handle_command_reply: tid=0 2024-04-25T14:14:02.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dcc78a000 1 client.10521 shutdown 2024-04-25T14:14:02.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dcc78a000 1 -- 172.21.15.86:0/1670442064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2d8900280 msgr2=0x55a2d8e892b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:02.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dcc78a000 1 --2- 172.21.15.86:0/1670442064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2d8900280 0x55a2d8e892b0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f2db0007e40 tx=0x7f2db0009f90 comp rx=0 tx=0).stop 2024-04-25T14:14:02.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dcc78a000 1 -- 172.21.15.86:0/1670442064 shutdown_connections 2024-04-25T14:14:02.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dcc78a000 1 --2- 172.21.15.86:0/1670442064 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a2d8d1bc20 0x55a2d8eac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dcc78a000 1 --2- 172.21.15.86:0/1670442064 >> [v2:172.21.15.155:6834/4231378267,v1:172.21.15.155:6835/4231378267] conn(0x55a2d8912830 0x55a2d8eef740 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dcc78a000 1 --2- 172.21.15.86:0/1670442064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2d8d07ea0 0x55a2d8eac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dcc78a000 1 --2- 172.21.15.86:0/1670442064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2d8900280 0x55a2d8e892b0 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2da3400640 0 client.10521 ms_handle_reset on v2:172.21.15.155:6834/4231378267 2024-04-25T14:14:02.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dcc78a000 1 -- 172.21.15.86:0/1670442064 >> 172.21.15.86:0/1670442064 conn(0x55a2d893fff0 msgr2=0x55a2d8e929a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:02.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dcc78a000 1 -- 172.21.15.86:0/1670442064 shutdown_connections 2024-04-25T14:14:02.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.626+0000 7f2dcc78a000 1 -- 172.21.15.86:0/1670442064 wait complete. 2024-04-25T14:14:02.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.630+0000 7f2dcc78a000 20 client.10521 trim_cache size 0 max 16384 2024-04-25T14:14:02.631 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:14:02.631 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb", 2024-04-25T14:14:02.631 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 1, 2024-04-25T14:14:02.631 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 8387, 2024-04-25T14:14:02.631 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T14:14:02.631 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T14:14:02.631 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T14:14:02.631 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 2.8089715580000001, 2024-04-25T14:14:02.631 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 120, 2024-04-25T14:14:02.631 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 156, 2024-04-25T14:14:02.632 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 156, 2024-04-25T14:14:02.632 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 33.273222464 2024-04-25T14:14:02.632 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:14:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.630+0000 7f2dc5000640 1 -- 172.21.15.86:0/2446096272 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f2d9c06f570 msgr2=0x7f2d9c071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:02.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.630+0000 7f2dc5000640 1 --2- 172.21.15.86:0/2446096272 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f2d9c06f570 0x7f2d9c071a80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0072200 tx=0x7f2db000b040 comp rx=0 tx=0).stop 2024-04-25T14:14:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.630+0000 7f2dc5000640 1 -- 172.21.15.86:0/2446096272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dc00d1c20 msgr2=0x7f2dc015c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.630+0000 7f2dc5000640 1 --2- 172.21.15.86:0/2446096272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dc00d1c20 0x7f2dc015c770 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f2db801a0e0 tx=0x7f2db8019250 comp rx=0 tx=0).stop 2024-04-25T14:14:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.630+0000 7f2dc5000640 1 -- 172.21.15.86:0/2446096272 shutdown_connections 2024-04-25T14:14:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.630+0000 7f2dc5000640 1 --2- 172.21.15.86:0/2446096272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dc00d1c20 0x7f2dc015c770 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.630+0000 7f2dc5000640 1 --2- 172.21.15.86:0/2446096272 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f2d9c06f570 0x7f2d9c071a80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.630+0000 7f2dc5000640 1 --2- 172.21.15.86:0/2446096272 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2dc00ce790 0x7f2dc015c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.630+0000 7f2dc5000640 1 --2- 172.21.15.86:0/2446096272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2dc0070330 0x7f2dc0156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.630+0000 7f2dc5000640 1 -- 172.21.15.86:0/2446096272 >> 172.21.15.86:0/2446096272 conn(0x7f2dc00b4650 msgr2=0x7f2dc00b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.630+0000 7f2dc5000640 1 -- 172.21.15.86:0/2446096272 shutdown_connections 2024-04-25T14:14:02.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.630+0000 7f2dc5000640 1 -- 172.21.15.86:0/2446096272 wait complete. 2024-04-25T14:14:02.647 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T14:14:02.732 INFO:tasks.ceph.mds.a.smithi086.stderr:2024-04-25T14:14:02.738+0000 7f39f3600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-25T14:14:02.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.798+0000 7fc780200640 1 Processor -- start 2024-04-25T14:14:02.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.798+0000 7fc780200640 1 -- start start 2024-04-25T14:14:02.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.798+0000 7fc780200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7780cfca0 0x7fc77806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.798+0000 7fc780200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc7780730b0 0x7fc7780734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.798+0000 7fc780200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc778072bd0 con 0x7fc7780d16e0 2024-04-25T14:14:02.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.798+0000 7fc780200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc778070470 con 0x7fc7780730b0 2024-04-25T14:14:02.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.798+0000 7fc780200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc77806f930 con 0x7fc7780cfca0 2024-04-25T14:14:02.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.798+0000 7fc77ce00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc7780d16e0 0x7fc7780d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:55548/0 (socket says 172.21.15.86:55548) 2024-04-25T14:14:02.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.798+0000 7fc77ce00640 1 -- 172.21.15.86:0/670327584 learned_addr learned my addr 172.21.15.86:0/670327584 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:14:02.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.798+0000 7fc77e200640 1 --2- 172.21.15.86:0/670327584 >> v2:172.21.15.86:3300/0 conn(0x7fc7780730b0 0x7fc7780734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.798+0000 7fc77d800640 1 --2- 172.21.15.86:0/670327584 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7780cfca0 0x7fc77806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.798+0000 7fc77e200640 1 -- 172.21.15.86:0/670327584 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7780cfca0 msgr2=0x7fc77806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:02.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.798+0000 7fc77e200640 1 --2- 172.21.15.86:0/670327584 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7780cfca0 0x7fc77806f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.798+0000 7fc77e200640 1 -- 172.21.15.86:0/670327584 >> v1:172.21.15.86:6789/0 conn(0x7fc7780d16e0 legacy=0x7fc7780d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:02.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.798+0000 7fc77e200640 1 -- 172.21.15.86:0/670327584 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77806fbd0 con 0x7fc7780730b0 2024-04-25T14:14:02.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.798+0000 7fc77d800640 1 --2- 172.21.15.86:0/670327584 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7780cfca0 0x7fc77806f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:02.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.802+0000 7fc77e200640 1 --2- 172.21.15.86:0/670327584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7780730b0 0x7fc7780734c0 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7fc770009d90 tx=0x7fc77001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7357ca6ec3f90fa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:02.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.802+0000 7fc777e00640 1 -- 172.21.15.86:0/670327584 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7700b2070 con 0x7fc7780730b0 2024-04-25T14:14:02.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.802+0000 7fc777e00640 1 -- 172.21.15.86:0/670327584 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7700a5940 con 0x7fc7780730b0 2024-04-25T14:14:02.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.802+0000 7fc777e00640 1 -- 172.21.15.86:0/670327584 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7700a5d10 con 0x7fc7780730b0 2024-04-25T14:14:02.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.802+0000 7fc780200640 1 -- 172.21.15.86:0/670327584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7780730b0 msgr2=0x7fc7780734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:02.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.802+0000 7fc780200640 1 --2- 172.21.15.86:0/670327584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7780730b0 0x7fc7780734c0 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7fc770009d90 tx=0x7fc77001c970 comp rx=0 tx=0).stop 2024-04-25T14:14:02.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.802+0000 7fc780200640 1 -- 172.21.15.86:0/670327584 shutdown_connections 2024-04-25T14:14:02.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.802+0000 7fc780200640 1 --2- 172.21.15.86:0/670327584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7780730b0 0x7fc7780734c0 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.802+0000 7fc780200640 1 --2- 172.21.15.86:0/670327584 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7780cfca0 0x7fc77806f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.802+0000 7fc780200640 1 -- 172.21.15.86:0/670327584 >> 172.21.15.86:0/670327584 conn(0x7fc7780b4650 msgr2=0x7fc7780b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:02.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.802+0000 7fc780200640 1 -- 172.21.15.86:0/670327584 shutdown_connections 2024-04-25T14:14:02.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.802+0000 7fc780200640 1 -- 172.21.15.86:0/670327584 wait complete. 2024-04-25T14:14:02.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.806+0000 7fc780200640 1 Processor -- start 2024-04-25T14:14:02.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.806+0000 7fc780200640 1 -- start start 2024-04-25T14:14:02.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.806+0000 7fc780200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7780730b0 0x7fc7781596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.806+0000 7fc780200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7780cfca0 0x7fc7781fcd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.806+0000 7fc780200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7780d16e0 0x7fc77815d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.806+0000 7fc780200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc778169a70 con 0x7fc7780730b0 2024-04-25T14:14:02.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.806+0000 7fc780200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc7781698f0 con 0x7fc7780cfca0 2024-04-25T14:14:02.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.806+0000 7fc780200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc778169bf0 con 0x7fc7780d16e0 2024-04-25T14:14:02.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.806+0000 7fc77d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7780730b0 0x7fc7781596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.806+0000 7fc77d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7780730b0 0x7fc7781596b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35620/0 (socket says 172.21.15.86:35620) 2024-04-25T14:14:02.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.806+0000 7fc77d800640 1 -- 172.21.15.86:0/2805712287 learned_addr learned my addr 172.21.15.86:0/2805712287 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:02.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.806+0000 7fc77d800640 1 -- 172.21.15.86:0/2805712287 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7780d16e0 msgr2=0x7fc77815d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:02.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.806+0000 7fc77d800640 1 --2- 172.21.15.86:0/2805712287 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7780d16e0 0x7fc77815d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.806+0000 7fc77d800640 1 -- 172.21.15.86:0/2805712287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7780cfca0 msgr2=0x7fc7781fcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:02.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.806+0000 7fc77ce00640 1 --2- 172.21.15.86:0/2805712287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7780cfca0 0x7fc7781fcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.806+0000 7fc77d800640 1 --2- 172.21.15.86:0/2805712287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7780cfca0 0x7fc7781fcd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.806+0000 7fc77d800640 1 -- 172.21.15.86:0/2805712287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7780ce4b0 con 0x7fc7780730b0 2024-04-25T14:14:02.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.806+0000 7fc77ce00640 1 --2- 172.21.15.86:0/2805712287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7780cfca0 0x7fc7781fcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:14:02.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.806+0000 7fc77d800640 1 --2- 172.21.15.86:0/2805712287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7780730b0 0x7fc7781596b0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00d800 tx=0x7fc76c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:02.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.806+0000 7fc775600640 1 -- 172.21.15.86:0/2805712287 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c00b6c0 con 0x7fc7780730b0 2024-04-25T14:14:02.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.806+0000 7fc775600640 1 -- 172.21.15.86:0/2805712287 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc76c019070 con 0x7fc7780730b0 2024-04-25T14:14:02.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.806+0000 7fc780200640 1 -- 172.21.15.86:0/2805712287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7780ce790 con 0x7fc7780730b0 2024-04-25T14:14:02.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.806+0000 7fc775600640 1 -- 172.21.15.86:0/2805712287 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c00baa0 con 0x7fc7780730b0 2024-04-25T14:14:02.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.806+0000 7fc780200640 1 -- 172.21.15.86:0/2805712287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc778158930 con 0x7fc7780730b0 2024-04-25T14:14:02.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.810+0000 7fc775600640 1 -- 172.21.15.86:0/2805712287 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7fc76c00bc40 con 0x7fc7780730b0 2024-04-25T14:14:02.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.810+0000 7fc775600640 1 --2- 172.21.15.86:0/2805712287 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc75806f5a0 0x7fc758071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.810+0000 7fc78797e000 1 Processor -- start 2024-04-25T14:14:02.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.810+0000 7fc775600640 1 -- 172.21.15.86:0/2805712287 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fc76c08fad0 con 0x7fc7780730b0 2024-04-25T14:14:02.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.810+0000 7fc77ce00640 1 --2- 172.21.15.86:0/2805712287 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc75806f5a0 0x7fc758071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.810+0000 7fc78797e000 1 -- start start 2024-04-25T14:14:02.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc77ce00640 1 --2- 172.21.15.86:0/2805712287 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc75806f5a0 0x7fc758071ab0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc77815bcb0 tx=0x7fc764009040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:02.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc78797e000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559805dcbc20 0x559805f29ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc78797e000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559805db7ea0 0x559805f2a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc78797e000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5598059b0280 0x559805f2a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc78797e000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559805d99160 con 0x559805dcbc20 2024-04-25T14:14:02.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc78797e000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559805d2ee20 con 0x559805db7ea0 2024-04-25T14:14:02.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc78797e000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x559805ce7a20 con 0x5598059b0280 2024-04-25T14:14:02.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc77d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559805dcbc20 0x559805f29ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc77d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559805dcbc20 0x559805f29ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35624/0 (socket says 172.21.15.86:35624) 2024-04-25T14:14:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc77d800640 1 -- 172.21.15.86:0/2194671994 learned_addr learned my addr 172.21.15.86:0/2194671994 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc77e200640 1 --2- 172.21.15.86:0/2194671994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5598059b0280 0x559805f2a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc77ce00640 1 --2- 172.21.15.86:0/2194671994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559805db7ea0 0x559805f2a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc77d800640 1 -- 172.21.15.86:0/2194671994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5598059b0280 msgr2=0x559805f2a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc77d800640 1 --2- 172.21.15.86:0/2194671994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5598059b0280 0x559805f2a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc77d800640 1 -- 172.21.15.86:0/2194671994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559805db7ea0 msgr2=0x559805f2a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc77d800640 1 --2- 172.21.15.86:0/2194671994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559805db7ea0 0x559805f2a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc77d800640 1 -- 172.21.15.86:0/2194671994 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559805cec100 con 0x559805dcbc20 2024-04-25T14:14:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc77d800640 1 --2- 172.21.15.86:0/2194671994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559805dcbc20 0x559805f29ee0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7fc76c01d000 tx=0x7fc76c05b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926bd4da8119b865 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc76a000640 1 -- 172.21.15.86:0/2194671994 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c05bf60 con 0x559805dcbc20 2024-04-25T14:14:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc76a000640 1 -- 172.21.15.86:0/2194671994 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc76c060310 con 0x559805dcbc20 2024-04-25T14:14:02.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc76a000640 1 -- 172.21.15.86:0/2194671994 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c011e10 con 0x559805dcbc20 2024-04-25T14:14:02.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc78797e000 1 -- 172.21.15.86:0/2194671994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559805dcbc20 msgr2=0x559805f29ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:02.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc78797e000 1 --2- 172.21.15.86:0/2194671994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559805dcbc20 0x559805f29ee0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7fc76c01d000 tx=0x7fc76c05b6d0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc78797e000 1 -- 172.21.15.86:0/2194671994 shutdown_connections 2024-04-25T14:14:02.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc78797e000 1 --2- 172.21.15.86:0/2194671994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5598059b0280 0x559805f2a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc78797e000 1 --2- 172.21.15.86:0/2194671994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559805db7ea0 0x559805f2a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc78797e000 1 --2- 172.21.15.86:0/2194671994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559805dcbc20 0x559805f29ee0 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc78797e000 1 -- 172.21.15.86:0/2194671994 >> 172.21.15.86:0/2194671994 conn(0x5598059efff0 msgr2=0x559805edf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:02.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc78797e000 1 -- 172.21.15.86:0/2194671994 shutdown_connections 2024-04-25T14:14:02.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.814+0000 7fc78797e000 1 -- 172.21.15.86:0/2194671994 wait complete. 2024-04-25T14:14:02.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.818+0000 7fc78797e000 1 -- start start 2024-04-25T14:14:02.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.818+0000 7fc78797e000 1 Processor -- start 2024-04-25T14:14:02.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.818+0000 7fc78797e000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5598059b0280 0x559805f30e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.818+0000 7fc78797e000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559805db7ea0 0x559805f313a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.818+0000 7fc78797e000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559805dcbc20 0x559805eec0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.818+0000 7fc78797e000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559805d3c220 con 0x559805db7ea0 2024-04-25T14:14:02.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.818+0000 7fc78797e000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559805cd2ea0 con 0x559805dcbc20 2024-04-25T14:14:02.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.818+0000 7fc78797e000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x559805d29b00 con 0x5598059b0280 2024-04-25T14:14:02.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.818+0000 7fc78797e000 10 client.0 _ll_register_callbacks cb 0x559805d3bdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:02.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.818+0000 7fc77ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559805db7ea0 0x559805f313a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.818+0000 7fc77ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559805db7ea0 0x559805f313a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35640/0 (socket says 172.21.15.86:35640) 2024-04-25T14:14:02.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.818+0000 7fc77ce00640 1 -- 172.21.15.86:0/3025368985 learned_addr learned my addr 172.21.15.86:0/3025368985 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:02.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.818+0000 7fc77d800640 1 --2- 172.21.15.86:0/3025368985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5598059b0280 0x559805f30e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.818+0000 7fc77e200640 1 --2- 172.21.15.86:0/3025368985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559805dcbc20 0x559805eec0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.818+0000 7fc77ce00640 1 -- 172.21.15.86:0/3025368985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5598059b0280 msgr2=0x559805f30e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:02.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.818+0000 7fc77ce00640 1 --2- 172.21.15.86:0/3025368985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5598059b0280 0x559805f30e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.818+0000 7fc77ce00640 1 -- 172.21.15.86:0/3025368985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559805dcbc20 msgr2=0x559805eec0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:02.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.818+0000 7fc77ce00640 1 --2- 172.21.15.86:0/3025368985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559805dcbc20 0x559805eec0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.818+0000 7fc77ce00640 1 -- 172.21.15.86:0/3025368985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559805cce500 con 0x559805db7ea0 2024-04-25T14:14:02.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.818+0000 7fc77ce00640 1 --2- 172.21.15.86:0/3025368985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559805db7ea0 0x559805f313a0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7fc764007450 tx=0x7fc764000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59bd2003b83bd1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:02.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.818+0000 7fc75ea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:14:02.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.818+0000 7fc75ea00640 1 -- 172.21.15.86:0/3025368985 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc764026060 con 0x559805db7ea0 2024-04-25T14:14:02.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.818+0000 7fc75ea00640 1 -- 172.21.15.86:0/3025368985 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fc76400de00 con 0x559805db7ea0 2024-04-25T14:14:02.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.818+0000 7fc78797e000 1 -- 172.21.15.86:0/3025368985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc76c060310 con 0x559805db7ea0 2024-04-25T14:14:02.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.818+0000 7fc75ea00640 1 -- 172.21.15.86:0/3025368985 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) ==== 24+0+0 (secure 0 0 0) 0x7fc7640047f0 con 0x559805db7ea0 2024-04-25T14:14:02.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.818+0000 7fc78797e000 10 client.10536 fetch_fsmap learned FSMap version 120 2024-04-25T14:14:02.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.818+0000 7fc78797e000 1 -- 172.21.15.86:0/3025368985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=2+}) -- 0x559805cb4ea0 con 0x559805db7ea0 2024-04-25T14:14:02.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.818+0000 7fc75ea00640 1 -- 172.21.15.86:0/3025368985 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc76c060310 con 0x559805db7ea0 2024-04-25T14:14:02.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.822+0000 7fc75ea00640 1 -- 172.21.15.86:0/3025368985 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 120) ==== 3514+0+0 (secure 0 0 0) 0x7fc764004990 con 0x559805db7ea0 2024-04-25T14:14:02.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.822+0000 7fc78797e000 10 client.10536 fetch_fsmap finished waiting for FSMap version 120 2024-04-25T14:14:02.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.822+0000 7fc78797e000 10 client.10536 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T14:14:02.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.822+0000 7fc78797e000 20 client.10536 populate_metadata read hostname 'smithi086' 2024-04-25T14:14:02.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.822+0000 7fc78797e000 1 --2- 172.21.15.86:0/3025368985 >> [v2:172.21.15.86:6836/3141015835,v1:172.21.15.86:6837/3141015835] conn(0x5598059d47d0 0x559805eece00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.822+0000 7fc78797e000 4 client.10536 mds_command: new command op to 10209 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:14:02.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.822+0000 7fc78797e000 1 -- 172.21.15.86:0/3025368985 --> [v2:172.21.15.86:6836/3141015835,v1:172.21.15.86:6837/3141015835] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559805cc7be0 con 0x5598059d47d0 2024-04-25T14:14:02.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.822+0000 7fc77d800640 1 --2- 172.21.15.86:0/3025368985 >> [v2:172.21.15.86:6836/3141015835,v1:172.21.15.86:6837/3141015835] conn(0x5598059d47d0 0x559805eece00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.822+0000 7fc75ea00640 10 client.10536 ms_handle_connect on v2:172.21.15.86:6836/3141015835 2024-04-25T14:14:02.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.822+0000 7fc77d800640 1 --2- 172.21.15.86:0/3025368985 >> [v2:172.21.15.86:6836/3141015835,v1:172.21.15.86:6837/3141015835] conn(0x5598059d47d0 0x559805eece00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92d65dbd23f131cb server_cookie=242ace1bdbd84ed8 in_seq=0 out_seq=0 2024-04-25T14:14:02.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.822+0000 7fc75ea00640 1 -- 172.21.15.86:0/3025368985 <== mds.0 v2:172.21.15.86:6836/3141015835 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559805cc7be0 con 0x5598059d47d0 2024-04-25T14:14:02.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.822+0000 7fc75ea00640 10 client.10536 handle_command_reply: tid=0 2024-04-25T14:14:02.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.822+0000 7fc78797e000 1 client.10536 shutdown 2024-04-25T14:14:02.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.822+0000 7fc78797e000 1 -- 172.21.15.86:0/3025368985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559805db7ea0 msgr2=0x559805f313a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:02.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.822+0000 7fc78797e000 1 --2- 172.21.15.86:0/3025368985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559805db7ea0 0x559805f313a0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7fc764007450 tx=0x7fc764000bd0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.822+0000 7fc78797e000 1 -- 172.21.15.86:0/3025368985 shutdown_connections 2024-04-25T14:14:02.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.822+0000 7fc78797e000 1 --2- 172.21.15.86:0/3025368985 >> [v2:172.21.15.86:6836/3141015835,v1:172.21.15.86:6837/3141015835] conn(0x5598059d47d0 0x559805eece00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.822+0000 7fc78797e000 1 --2- 172.21.15.86:0/3025368985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559805dcbc20 0x559805eec0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.822+0000 7fc78797e000 1 --2- 172.21.15.86:0/3025368985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559805db7ea0 0x559805f313a0 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.822+0000 7fc75ea00640 0 client.10536 ms_handle_reset on v2:172.21.15.86:6836/3141015835 2024-04-25T14:14:02.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.822+0000 7fc78797e000 1 --2- 172.21.15.86:0/3025368985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5598059b0280 0x559805f30e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.822+0000 7fc78797e000 1 -- 172.21.15.86:0/3025368985 >> 172.21.15.86:0/3025368985 conn(0x5598059efff0 msgr2=0x559805edfc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:02.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.822+0000 7fc78797e000 1 -- 172.21.15.86:0/3025368985 shutdown_connections 2024-04-25T14:14:02.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.822+0000 7fc78797e000 1 -- 172.21.15.86:0/3025368985 wait complete. 2024-04-25T14:14:02.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.822+0000 7fc78797e000 20 client.10536 trim_cache size 0 max 16384 2024-04-25T14:14:02.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.842+0000 7fc78797e000 1 Processor -- start 2024-04-25T14:14:02.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.842+0000 7fc78797e000 1 -- start start 2024-04-25T14:14:02.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.842+0000 7fc78797e000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598059b0280 0x559805f35ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.842+0000 7fc78797e000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559805db7ea0 0x559805f363e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.842+0000 7fc78797e000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559805dcbc20 0x559805f3fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.842+0000 7fc78797e000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc778206b30 con 0x5598059b0280 2024-04-25T14:14:02.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.842+0000 7fc78797e000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc7781575d0 con 0x559805dcbc20 2024-04-25T14:14:02.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.842+0000 7fc78797e000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc778205600 con 0x559805db7ea0 2024-04-25T14:14:02.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.842+0000 7fc77d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598059b0280 0x559805f35ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.842+0000 7fc77d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598059b0280 0x559805f35ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35656/0 (socket says 172.21.15.86:35656) 2024-04-25T14:14:02.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.842+0000 7fc77d800640 1 -- 172.21.15.86:0/769072106 learned_addr learned my addr 172.21.15.86:0/769072106 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:02.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.842+0000 7fc77d800640 1 -- 172.21.15.86:0/769072106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559805db7ea0 msgr2=0x559805f363e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:02.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.842+0000 7fc77d800640 1 --2- 172.21.15.86:0/769072106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559805db7ea0 0x559805f363e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.842+0000 7fc77d800640 1 -- 172.21.15.86:0/769072106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559805dcbc20 msgr2=0x559805f3fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:02.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.842+0000 7fc77d800640 1 --2- 172.21.15.86:0/769072106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559805dcbc20 0x559805f3fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.842+0000 7fc77d800640 1 -- 172.21.15.86:0/769072106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559805eec430 con 0x5598059b0280 2024-04-25T14:14:02.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.842+0000 7fc77d800640 1 --2- 172.21.15.86:0/769072106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598059b0280 0x559805f35ea0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7fc76c05bc60 tx=0x7fc76c072000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f8ece64731e876 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:02.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.842+0000 7fc769600640 1 -- 172.21.15.86:0/769072106 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c060310 con 0x5598059b0280 2024-04-25T14:14:02.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.842+0000 7fc769600640 1 -- 172.21.15.86:0/769072106 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc76c079030 con 0x5598059b0280 2024-04-25T14:14:02.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.842+0000 7fc769600640 1 -- 172.21.15.86:0/769072106 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c064d30 con 0x5598059b0280 2024-04-25T14:14:02.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.846+0000 7fc78797e000 1 -- 172.21.15.86:0/769072106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598059b0280 msgr2=0x559805f35ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:02.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.846+0000 7fc78797e000 1 --2- 172.21.15.86:0/769072106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598059b0280 0x559805f35ea0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7fc76c05bc60 tx=0x7fc76c072000 comp rx=0 tx=0).stop 2024-04-25T14:14:02.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.846+0000 7fc78797e000 1 -- 172.21.15.86:0/769072106 shutdown_connections 2024-04-25T14:14:02.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.846+0000 7fc78797e000 1 --2- 172.21.15.86:0/769072106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559805dcbc20 0x559805f3fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.846+0000 7fc78797e000 1 --2- 172.21.15.86:0/769072106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559805db7ea0 0x559805f363e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.846+0000 7fc78797e000 1 --2- 172.21.15.86:0/769072106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598059b0280 0x559805f35ea0 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.846+0000 7fc78797e000 1 -- 172.21.15.86:0/769072106 >> 172.21.15.86:0/769072106 conn(0x5598059efff0 msgr2=0x559805ee7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:02.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.846+0000 7fc78797e000 1 -- 172.21.15.86:0/769072106 shutdown_connections 2024-04-25T14:14:02.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.846+0000 7fc78797e000 1 -- 172.21.15.86:0/769072106 wait complete. 2024-04-25T14:14:02.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.846+0000 7fc78797e000 1 -- start start 2024-04-25T14:14:02.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.846+0000 7fc78797e000 1 Processor -- start 2024-04-25T14:14:02.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.846+0000 7fc78797e000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598059b0280 0x559805f9c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.846+0000 7fc78797e000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559805db7ea0 0x559805f9cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.846+0000 7fc78797e000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559805dcbc20 0x559805f349b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.846+0000 7fc78797e000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559805d2e130 con 0x5598059b0280 2024-04-25T14:14:02.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.846+0000 7fc78797e000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559805cc47d0 con 0x559805db7ea0 2024-04-25T14:14:02.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.846+0000 7fc78797e000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x559805d33450 con 0x559805dcbc20 2024-04-25T14:14:02.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.846+0000 7fc78797e000 10 client.0 _ll_register_callbacks cb 0x7fc7640013b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:14:02.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.846+0000 7fc77d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598059b0280 0x559805f9c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.846+0000 7fc77d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598059b0280 0x559805f9c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35668/0 (socket says 172.21.15.86:35668) 2024-04-25T14:14:02.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.846+0000 7fc77d800640 1 -- 172.21.15.86:0/4057218363 learned_addr learned my addr 172.21.15.86:0/4057218363 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:02.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.846+0000 7fc77ce00640 1 --2- 172.21.15.86:0/4057218363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559805db7ea0 0x559805f9cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.846+0000 7fc77e200640 1 --2- 172.21.15.86:0/4057218363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559805dcbc20 0x559805f349b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.846+0000 7fc77d800640 1 -- 172.21.15.86:0/4057218363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559805dcbc20 msgr2=0x559805f349b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:02.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.846+0000 7fc77d800640 1 --2- 172.21.15.86:0/4057218363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559805dcbc20 0x559805f349b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.846+0000 7fc77d800640 1 -- 172.21.15.86:0/4057218363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559805db7ea0 msgr2=0x559805f9cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:02.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.846+0000 7fc77d800640 1 --2- 172.21.15.86:0/4057218363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559805db7ea0 0x559805f9cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.846+0000 7fc77d800640 1 -- 172.21.15.86:0/4057218363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559805ce7c70 con 0x5598059b0280 2024-04-25T14:14:02.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc77d800640 1 --2- 172.21.15.86:0/4057218363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598059b0280 0x559805f9c590 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7fc76c05b840 tx=0x7fc76c05e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a94c2064e6015c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:02.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc75ea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-25T14:14:02.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc75ea00640 1 -- 172.21.15.86:0/4057218363 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c05bf60 con 0x5598059b0280 2024-04-25T14:14:02.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc75ea00640 1 -- 172.21.15.86:0/4057218363 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fc76c0581f0 con 0x5598059b0280 2024-04-25T14:14:02.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc78797e000 1 -- 172.21.15.86:0/4057218363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc76c079030 con 0x5598059b0280 2024-04-25T14:14:02.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc75ea00640 1 -- 172.21.15.86:0/4057218363 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) ==== 24+0+0 (secure 0 0 0) 0x7fc76c064d30 con 0x5598059b0280 2024-04-25T14:14:02.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc78797e000 10 client.10542 fetch_fsmap learned FSMap version 120 2024-04-25T14:14:02.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc78797e000 1 -- 172.21.15.86:0/4057218363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=2+}) -- 0x559805db1180 con 0x5598059b0280 2024-04-25T14:14:02.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc75ea00640 1 -- 172.21.15.86:0/4057218363 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc76c079030 con 0x5598059b0280 2024-04-25T14:14:02.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc75ea00640 1 -- 172.21.15.86:0/4057218363 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 120) ==== 3514+0+0 (secure 0 0 0) 0x7fc76c013070 con 0x5598059b0280 2024-04-25T14:14:02.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc78797e000 10 client.10542 fetch_fsmap finished waiting for FSMap version 120 2024-04-25T14:14:02.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc78797e000 10 client.10542 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T14:14:02.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc78797e000 20 client.10542 populate_metadata read hostname 'smithi086' 2024-04-25T14:14:02.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc78797e000 1 --2- 172.21.15.86:0/4057218363 >> [v2:172.21.15.86:6836/3141015835,v1:172.21.15.86:6837/3141015835] conn(0x5598059c2830 0x559805f352e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:02.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc78797e000 4 client.10542 mds_command: new command op to 10209 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T14:14:02.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc78797e000 1 -- 172.21.15.86:0/4057218363 --> [v2:172.21.15.86:6836/3141015835,v1:172.21.15.86:6837/3141015835] -- command(tid 0: {"prefix": "status"}) -- 0x559805d3ca30 con 0x5598059c2830 2024-04-25T14:14:02.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc77ce00640 1 --2- 172.21.15.86:0/4057218363 >> [v2:172.21.15.86:6836/3141015835,v1:172.21.15.86:6837/3141015835] conn(0x5598059c2830 0x559805f352e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:02.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc77ce00640 1 --2- 172.21.15.86:0/4057218363 >> [v2:172.21.15.86:6836/3141015835,v1:172.21.15.86:6837/3141015835] conn(0x5598059c2830 0x559805f352e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1dfaf58c166b2e8 server_cookie=4c775cc52efaf4fa in_seq=0 out_seq=0 2024-04-25T14:14:02.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc75ea00640 10 client.10542 ms_handle_connect on v2:172.21.15.86:6836/3141015835 2024-04-25T14:14:02.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc75ea00640 1 -- 172.21.15.86:0/4057218363 <== mds.0 v2:172.21.15.86:6836/3141015835 1 ==== command_reply(tid 0: 0 ) ==== 8+0+329 (crc 0 0 0) 0x559805d3ca30 con 0x5598059c2830 2024-04-25T14:14:02.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc75ea00640 10 client.10542 handle_command_reply: tid=0 2024-04-25T14:14:02.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc78797e000 1 client.10542 shutdown 2024-04-25T14:14:02.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc78797e000 1 -- 172.21.15.86:0/4057218363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598059b0280 msgr2=0x559805f9c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:02.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc78797e000 1 --2- 172.21.15.86:0/4057218363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598059b0280 0x559805f9c590 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7fc76c05b840 tx=0x7fc76c05e000 comp rx=0 tx=0).stop 2024-04-25T14:14:02.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc78797e000 1 -- 172.21.15.86:0/4057218363 shutdown_connections 2024-04-25T14:14:02.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc78797e000 1 --2- 172.21.15.86:0/4057218363 >> [v2:172.21.15.86:6836/3141015835,v1:172.21.15.86:6837/3141015835] conn(0x5598059c2830 0x559805f352e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc78797e000 1 --2- 172.21.15.86:0/4057218363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559805dcbc20 0x559805f349b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc78797e000 1 --2- 172.21.15.86:0/4057218363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559805db7ea0 0x559805f9cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc75ea00640 0 client.10542 ms_handle_reset on v2:172.21.15.86:6836/3141015835 2024-04-25T14:14:02.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc78797e000 1 --2- 172.21.15.86:0/4057218363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598059b0280 0x559805f9c590 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc78797e000 1 -- 172.21.15.86:0/4057218363 >> 172.21.15.86:0/4057218363 conn(0x5598059efff0 msgr2=0x559805f2f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:02.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc78797e000 1 -- 172.21.15.86:0/4057218363 shutdown_connections 2024-04-25T14:14:02.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc78797e000 1 -- 172.21.15.86:0/4057218363 wait complete. 2024-04-25T14:14:02.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc78797e000 20 client.10542 trim_cache size 0 max 16384 2024-04-25T14:14:02.854 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:14:02.854 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb", 2024-04-25T14:14:02.854 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-25T14:14:02.854 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 10209, 2024-04-25T14:14:02.854 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-25T14:14:02.854 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-25T14:14:02.854 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-25T14:14:02.854 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 28.627299379, 2024-04-25T14:14:02.854 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 120, 2024-04-25T14:14:02.854 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 156, 2024-04-25T14:14:02.855 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 156, 2024-04-25T14:14:02.855 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 33.496966262000001 2024-04-25T14:14:02.855 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:14:02.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc780200640 1 -- 172.21.15.86:0/2805712287 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc75806f5a0 msgr2=0x7fc758071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc780200640 1 --2- 172.21.15.86:0/2805712287 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc75806f5a0 0x7fc758071ab0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc77815bcb0 tx=0x7fc764009040 comp rx=0 tx=0).stop 2024-04-25T14:14:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc780200640 1 -- 172.21.15.86:0/2805712287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7780730b0 msgr2=0x7fc7781596b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc780200640 1 --2- 172.21.15.86:0/2805712287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7780730b0 0x7fc7781596b0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00d800 tx=0x7fc76c00dcc0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc780200640 1 -- 172.21.15.86:0/2805712287 shutdown_connections 2024-04-25T14:14:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc780200640 1 --2- 172.21.15.86:0/2805712287 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc75806f5a0 0x7fc758071ab0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc780200640 1 --2- 172.21.15.86:0/2805712287 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7780d16e0 0x7fc77815d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc780200640 1 --2- 172.21.15.86:0/2805712287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7780cfca0 0x7fc7781fcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc780200640 1 --2- 172.21.15.86:0/2805712287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7780730b0 0x7fc7781596b0 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc780200640 1 -- 172.21.15.86:0/2805712287 >> 172.21.15.86:0/2805712287 conn(0x7fc7780b4650 msgr2=0x7fc77815bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc780200640 1 -- 172.21.15.86:0/2805712287 shutdown_connections 2024-04-25T14:14:02.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:02.850+0000 7fc780200640 1 -- 172.21.15.86:0/2805712287 wait complete. 2024-04-25T14:14:02.870 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:03.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.022+0000 7fd8da600640 1 Processor -- start 2024-04-25T14:14:03.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.022+0000 7fd8da600640 1 -- start start 2024-04-25T14:14:03.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.022+0000 7fd8da600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd8d415c2a0 0x7fd8d40ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:03.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.022+0000 7fd8da600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8d40b0a60 con 0x7fd8d406fd50 2024-04-25T14:14:03.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.022+0000 7fd8da600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8d406ef90 con 0x7fd8d415feb0 2024-04-25T14:14:03.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.022+0000 7fd8da600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd8d406f270 con 0x7fd8d415c2a0 2024-04-25T14:14:03.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.022+0000 7fd8d3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd8d415c2a0 0x7fd8d40ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:03.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.022+0000 7fd8d3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd8d415c2a0 0x7fd8d40ae130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35684/0 (socket says 172.21.15.86:35684) 2024-04-25T14:14:03.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.022+0000 7fd8d3400640 1 -- 172.21.15.86:0/2223821654 learned_addr learned my addr 172.21.15.86:0/2223821654 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:03.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.022+0000 7fd8d3400640 1 -- 172.21.15.86:0/2223821654 >> v1:172.21.15.155:6789/0 conn(0x7fd8d415feb0 legacy=0x7fd8d41602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:03.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.022+0000 7fd8d3400640 1 -- 172.21.15.86:0/2223821654 >> v1:172.21.15.86:6789/0 conn(0x7fd8d406fd50 legacy=0x7fd8d4070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:03.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.022+0000 7fd8d3400640 1 -- 172.21.15.86:0/2223821654 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d406f550 con 0x7fd8d415c2a0 2024-04-25T14:14:03.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.022+0000 7fd8d3400640 1 --2- 172.21.15.86:0/2223821654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8d415c2a0 0x7fd8d40ae130 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4006fd0 tx=0x7fd8c401e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2261015693ba4d11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:03.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.022+0000 7fd8d2a00640 1 -- 172.21.15.86:0/2223821654 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c40ad070 con 0x7fd8d415c2a0 2024-04-25T14:14:03.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.022+0000 7fd8d2a00640 1 -- 172.21.15.86:0/2223821654 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8c4004230 con 0x7fd8d415c2a0 2024-04-25T14:14:03.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.022+0000 7fd8d2a00640 1 -- 172.21.15.86:0/2223821654 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c40a7690 con 0x7fd8d415c2a0 2024-04-25T14:14:03.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.022+0000 7fd8da600640 1 -- 172.21.15.86:0/2223821654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8d415c2a0 msgr2=0x7fd8d40ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:03.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.022+0000 7fd8da600640 1 --2- 172.21.15.86:0/2223821654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8d415c2a0 0x7fd8d40ae130 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4006fd0 tx=0x7fd8c401e930 comp rx=0 tx=0).stop 2024-04-25T14:14:03.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.022+0000 7fd8da600640 1 -- 172.21.15.86:0/2223821654 shutdown_connections 2024-04-25T14:14:03.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.022+0000 7fd8da600640 1 --2- 172.21.15.86:0/2223821654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8d415c2a0 0x7fd8d40ae130 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:03.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.022+0000 7fd8da600640 1 -- 172.21.15.86:0/2223821654 >> 172.21.15.86:0/2223821654 conn(0x7fd8d40b4650 msgr2=0x7fd8d40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:03.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.022+0000 7fd8da600640 1 -- 172.21.15.86:0/2223821654 shutdown_connections 2024-04-25T14:14:03.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.022+0000 7fd8da600640 1 -- 172.21.15.86:0/2223821654 wait complete. 2024-04-25T14:14:03.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.026+0000 7fd8da600640 1 Processor -- start 2024-04-25T14:14:03.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.026+0000 7fd8da600640 1 -- start start 2024-04-25T14:14:03.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.026+0000 7fd8da600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8d406fd50 0x7fd8d41f9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:03.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.026+0000 7fd8da600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8d415c2a0 0x7fd8d41569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:03.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.026+0000 7fd8da600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8d415feb0 0x7fd8d40741a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:03.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.026+0000 7fd8da600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d40ae7f0 con 0x7fd8d415feb0 2024-04-25T14:14:03.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.026+0000 7fd8da600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d40ae670 con 0x7fd8d406fd50 2024-04-25T14:14:03.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.026+0000 7fd8da600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd8d416a200 con 0x7fd8d415c2a0 2024-04-25T14:14:03.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.026+0000 7fd8d3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8d415feb0 0x7fd8d40741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:03.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.026+0000 7fd8d3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8d415feb0 0x7fd8d40741a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35696/0 (socket says 172.21.15.86:35696) 2024-04-25T14:14:03.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.026+0000 7fd8d3e00640 1 -- 172.21.15.86:0/1806741934 learned_addr learned my addr 172.21.15.86:0/1806741934 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:03.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.026+0000 7fd8d3400640 1 --2- 172.21.15.86:0/1806741934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8d406fd50 0x7fd8d41f9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:03.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.026+0000 7fd8caa00640 1 --2- 172.21.15.86:0/1806741934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8d415c2a0 0x7fd8d41569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:03.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.026+0000 7fd8d3e00640 1 -- 172.21.15.86:0/1806741934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8d415c2a0 msgr2=0x7fd8d41569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:03.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.026+0000 7fd8d3e00640 1 --2- 172.21.15.86:0/1806741934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8d415c2a0 0x7fd8d41569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:03.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.026+0000 7fd8d3e00640 1 -- 172.21.15.86:0/1806741934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8d406fd50 msgr2=0x7fd8d41f9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:03.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.026+0000 7fd8d3e00640 1 --2- 172.21.15.86:0/1806741934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8d406fd50 0x7fd8d41f9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:03.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.026+0000 7fd8d3e00640 1 -- 172.21.15.86:0/1806741934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d415e5e0 con 0x7fd8d415feb0 2024-04-25T14:14:03.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.026+0000 7fd8d3e00640 1 --2- 172.21.15.86:0/1806741934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8d415feb0 0x7fd8d40741a0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc00c760 tx=0x7fd8cc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:03.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.026+0000 7fd8cbe00640 1 -- 172.21.15.86:0/1806741934 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc003b80 con 0x7fd8d415feb0 2024-04-25T14:14:03.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.026+0000 7fd8cbe00640 1 -- 172.21.15.86:0/1806741934 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8cc003d20 con 0x7fd8d415feb0 2024-04-25T14:14:03.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.026+0000 7fd8cbe00640 1 -- 172.21.15.86:0/1806741934 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc00aa30 con 0x7fd8d415feb0 2024-04-25T14:14:03.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.026+0000 7fd8da600640 1 -- 172.21.15.86:0/1806741934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8d415d540 con 0x7fd8d415feb0 2024-04-25T14:14:03.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.030+0000 7fd8da600640 1 -- 172.21.15.86:0/1806741934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8d415db00 con 0x7fd8d415feb0 2024-04-25T14:14:03.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.030+0000 7fd8da600640 1 -- 172.21.15.86:0/1806741934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8d4069d00 con 0x7fd8d415feb0 2024-04-25T14:14:03.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.030+0000 7fd8cbe00640 1 -- 172.21.15.86:0/1806741934 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7fd8cc00abd0 con 0x7fd8d415feb0 2024-04-25T14:14:03.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.030+0000 7fd8cbe00640 1 --2- 172.21.15.86:0/1806741934 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd8b006f5a0 0x7fd8b0071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:03.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.030+0000 7fd8cbe00640 1 -- 172.21.15.86:0/1806741934 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fd8cc08f590 con 0x7fd8d415feb0 2024-04-25T14:14:03.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.030+0000 7fd8d3400640 1 --2- 172.21.15.86:0/1806741934 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd8b006f5a0 0x7fd8b0071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:03.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.030+0000 7fd8d3400640 1 --2- 172.21.15.86:0/1806741934 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd8b006f5a0 0x7fd8b0071ab0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd8c40b1000 tx=0x7fd8c40a9070 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:03.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.034+0000 7fd8cbe00640 1 -- 172.21.15.86:0/1806741934 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd8cc05b9a0 con 0x7fd8d415feb0 2024-04-25T14:14:03.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.226+0000 7fd8da600640 1 -- 172.21.15.86:0/1806741934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8d406db40 con 0x7fd8d415feb0 2024-04-25T14:14:03.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.226+0000 7fd8cbe00640 1 -- 172.21.15.86:0/1806741934 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) ==== 95+0+5221 (secure 0 0 0) 0x7fd8cc064070 con 0x7fd8d415feb0 2024-04-25T14:14:03.225 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:03.225 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":120,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:00.821918+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[8387,10209]},"id":11}]} 2024-04-25T14:14:03.225 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 120 2024-04-25T14:14:03.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.226+0000 7fd8da600640 1 -- 172.21.15.86:0/1806741934 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd8b006f5a0 msgr2=0x7fd8b0071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:03.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.226+0000 7fd8da600640 1 --2- 172.21.15.86:0/1806741934 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd8b006f5a0 0x7fd8b0071ab0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd8c40b1000 tx=0x7fd8c40a9070 comp rx=0 tx=0).stop 2024-04-25T14:14:03.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.226+0000 7fd8da600640 1 -- 172.21.15.86:0/1806741934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8d415feb0 msgr2=0x7fd8d40741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:03.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.226+0000 7fd8da600640 1 --2- 172.21.15.86:0/1806741934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8d415feb0 0x7fd8d40741a0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc00c760 tx=0x7fd8cc00cc20 comp rx=0 tx=0).stop 2024-04-25T14:14:03.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.226+0000 7fd8da600640 1 -- 172.21.15.86:0/1806741934 shutdown_connections 2024-04-25T14:14:03.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.226+0000 7fd8da600640 1 --2- 172.21.15.86:0/1806741934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8d415feb0 0x7fd8d40741a0 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:03.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.226+0000 7fd8da600640 1 --2- 172.21.15.86:0/1806741934 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd8b006f5a0 0x7fd8b0071ab0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:03.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.226+0000 7fd8da600640 1 --2- 172.21.15.86:0/1806741934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8d415c2a0 0x7fd8d41569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:03.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.226+0000 7fd8da600640 1 --2- 172.21.15.86:0/1806741934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8d406fd50 0x7fd8d41f9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:03.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.226+0000 7fd8da600640 1 -- 172.21.15.86:0/1806741934 >> 172.21.15.86:0/1806741934 conn(0x7fd8d40b4650 msgr2=0x7fd8d41ef090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:03.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.226+0000 7fd8da600640 1 -- 172.21.15.86:0/1806741934 shutdown_connections 2024-04-25T14:14:03.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:03.226+0000 7fd8da600640 1 -- 172.21.15.86:0/1806741934 wait complete. 2024-04-25T14:14:03.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:03.250 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test_mds_metrics_0) 2024-04-25T14:14:03.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:03.265 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 test_mds_metrics_0) 2024-04-25T14:14:03.322 INFO:tasks.cephfs.test_mds_metrics:pinning directory test_mds_metrics_0 to rank 0 2024-04-25T14:14:03.322 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:03.322 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:03.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:03.372 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.0 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:03.444 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:03.444 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:03.444 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201469 s, 52.0 MB/s 2024-04-25T14:14:03.445 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:03.445 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:03.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:03.500 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.1 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:03.577 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:03.577 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:03.577 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212278 s, 49.4 MB/s 2024-04-25T14:14:03.578 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:03.578 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:03.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:03.632 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.2 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:03.704 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:03.704 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:03.705 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207513 s, 50.5 MB/s 2024-04-25T14:14:03.705 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:03.706 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:03.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:03.760 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.3 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:03.836 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:03.837 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:03.837 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210963 s, 49.7 MB/s 2024-04-25T14:14:03.838 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:03.838 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:03.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:03.888 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.4 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:03.970 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:03.971 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:03.971 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248604 s, 42.2 MB/s 2024-04-25T14:14:03.971 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:03.971 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:04.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:04.019 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.5 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:04.091 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:04.091 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:04.091 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191286 s, 54.8 MB/s 2024-04-25T14:14:04.092 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:04.092 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:04.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:04.143 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.6 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:04.218 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:04.218 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:04.218 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229134 s, 45.8 MB/s 2024-04-25T14:14:04.219 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:04.219 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:04.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:04.272 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.7 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:04.342 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:04.342 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:04.342 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179231 s, 58.5 MB/s 2024-04-25T14:14:04.343 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:04.343 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:04.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:04.392 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.8 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:04.465 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:04.465 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:04.465 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200898 s, 52.2 MB/s 2024-04-25T14:14:04.466 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:04.466 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:04.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:04.516 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.9 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:04.588 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:04.589 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:04.589 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209547 s, 50.0 MB/s 2024-04-25T14:14:04.590 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:04.590 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:04.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:04.639 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.10 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:04.712 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:04.712 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:04.712 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198111 s, 52.9 MB/s 2024-04-25T14:14:04.713 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:04.713 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:04.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:04.764 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.11 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:04.839 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:04.840 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:04.840 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024356 s, 43.1 MB/s 2024-04-25T14:14:04.841 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:04.841 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:04.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:04.892 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.12 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:04.962 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:04.962 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:04.963 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184915 s, 56.7 MB/s 2024-04-25T14:14:04.964 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:04.964 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:05.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:05.016 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.13 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:05.093 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:05.093 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:05.093 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217064 s, 48.3 MB/s 2024-04-25T14:14:05.094 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:05.095 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:05.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:05.143 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.14 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:05.215 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:05.215 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:05.215 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194578 s, 53.9 MB/s 2024-04-25T14:14:05.216 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:05.216 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:05.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:05.267 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_0/test.15 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:05.339 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:05.340 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:05.340 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193522 s, 54.2 MB/s 2024-04-25T14:14:05.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:05.341 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test_mds_metrics_1) 2024-04-25T14:14:05.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:05.395 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 test_mds_metrics_1) 2024-04-25T14:14:05.451 INFO:tasks.cephfs.test_mds_metrics:pinning directory test_mds_metrics_1 to rank 1 2024-04-25T14:14:05.451 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:05.451 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:05.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:05.505 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.0 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:05.573 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:05.573 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:05.573 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174034 s, 60.3 MB/s 2024-04-25T14:14:05.574 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:05.574 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:05.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:05.624 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.1 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:05.694 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:05.694 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:05.694 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183937 s, 57.0 MB/s 2024-04-25T14:14:05.695 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:05.695 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:05.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:05.752 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.2 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:05.826 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:05.826 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:05.826 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211107 s, 49.7 MB/s 2024-04-25T14:14:05.827 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:05.827 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:05.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:05.880 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.3 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:05.958 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:05.958 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:05.958 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255838 s, 41.0 MB/s 2024-04-25T14:14:05.959 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:05.959 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:06.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:06.016 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.4 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:06.086 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:06.086 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:06.087 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183899 s, 57.0 MB/s 2024-04-25T14:14:06.088 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:06.088 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:06.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:06.140 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.5 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:06.210 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:06.210 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:06.210 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178327 s, 58.8 MB/s 2024-04-25T14:14:06.211 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:06.211 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:06.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:06.264 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.6 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:06.336 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:06.336 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:06.336 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207044 s, 50.6 MB/s 2024-04-25T14:14:06.337 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:06.338 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:06.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:06.388 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.7 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:06.467 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:06.467 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:06.468 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027376 s, 38.3 MB/s 2024-04-25T14:14:06.468 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:06.469 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:06.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:06.520 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.8 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:06.593 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:06.593 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:06.593 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211592 s, 49.6 MB/s 2024-04-25T14:14:06.594 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:06.594 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:06.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:06.644 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.9 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:06.714 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:06.714 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:06.714 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182017 s, 57.6 MB/s 2024-04-25T14:14:06.715 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:06.715 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:06.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:06.768 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.10 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:06.846 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:06.846 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:06.846 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021951 s, 47.8 MB/s 2024-04-25T14:14:06.847 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:06.847 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:06.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:06.904 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.11 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:06.974 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:06.974 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:06.975 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188756 s, 55.6 MB/s 2024-04-25T14:14:06.976 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:06.976 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:07.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:07.027 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.12 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:07.101 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:07.102 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:07.102 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180378 s, 58.1 MB/s 2024-04-25T14:14:07.103 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:07.103 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:07.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:07.156 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.13 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:07.229 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:07.229 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:07.229 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210802 s, 49.7 MB/s 2024-04-25T14:14:07.230 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:07.230 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:07.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:07.284 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.14 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:07.361 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:07.361 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:07.361 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018933 s, 55.4 MB/s 2024-04-25T14:14:07.363 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:14:07.363 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:14:07.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:07.416 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=test_mds_metrics_1/test.15 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T14:14:07.482 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-25T14:14:07.482 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-25T14:14:07.482 INFO:teuthology.orchestra.run.smithi086.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138112 s, 75.9 MB/s 2024-04-25T14:14:07.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:14:07.483 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest/mnt.1 && exec find .) 2024-04-25T14:14:07.505 INFO:teuthology.orchestra.run.smithi155.stdout:. 2024-04-25T14:14:07.505 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0 2024-04-25T14:14:07.505 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.5 2024-04-25T14:14:07.505 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.7 2024-04-25T14:14:07.505 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.6 2024-04-25T14:14:07.505 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.10 2024-04-25T14:14:07.505 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.4 2024-04-25T14:14:07.506 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.12 2024-04-25T14:14:07.506 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.1 2024-04-25T14:14:07.506 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.9 2024-04-25T14:14:07.506 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.13 2024-04-25T14:14:07.506 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.0 2024-04-25T14:14:07.506 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.11 2024-04-25T14:14:07.506 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.8 2024-04-25T14:14:07.506 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.3 2024-04-25T14:14:07.506 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.2 2024-04-25T14:14:07.506 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.14 2024-04-25T14:14:07.506 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_0/test.15 2024-04-25T14:14:07.506 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1 2024-04-25T14:14:07.506 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.5 2024-04-25T14:14:07.506 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.7 2024-04-25T14:14:07.507 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.6 2024-04-25T14:14:07.507 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.10 2024-04-25T14:14:07.507 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.4 2024-04-25T14:14:07.507 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.12 2024-04-25T14:14:07.507 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.1 2024-04-25T14:14:07.507 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.9 2024-04-25T14:14:07.507 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.13 2024-04-25T14:14:07.507 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.0 2024-04-25T14:14:07.507 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.11 2024-04-25T14:14:07.507 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.8 2024-04-25T14:14:07.507 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.3 2024-04-25T14:14:07.507 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.2 2024-04-25T14:14:07.507 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.14 2024-04-25T14:14:07.508 INFO:teuthology.orchestra.run.smithi155.stdout:./test_mds_metrics_1/test.15 2024-04-25T14:14:12.509 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs perf stats 2024-04-25T14:14:12.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.674+0000 7f60ade00640 1 Processor -- start 2024-04-25T14:14:12.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.674+0000 7f60ade00640 1 -- start start 2024-04-25T14:14:12.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.678+0000 7f60ade00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f60a80cfca0 0x7f60a806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.678+0000 7f60ade00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f60a80d16e0 0x7f60a80d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.678+0000 7f60ade00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60a80d2030 con 0x7f60a806c670 2024-04-25T14:14:12.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.678+0000 7f60ade00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f60a8070470 con 0x7f60a80d16e0 2024-04-25T14:14:12.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.678+0000 7f60ade00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f60a8072bd0 con 0x7f60a80cfca0 2024-04-25T14:14:12.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.678+0000 7f609e000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f60a80d16e0 0x7f60a80d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.678+0000 7f609e000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f60a80d16e0 0x7f60a80d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51114/0 (socket says 172.21.15.86:51114) 2024-04-25T14:14:12.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.678+0000 7f609e000640 1 -- 172.21.15.86:0/1151612958 learned_addr learned my addr 172.21.15.86:0/1151612958 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:12.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.678+0000 7f60a7400640 1 --1- 172.21.15.86:0/1151612958 >> v1:172.21.15.86:6789/0 conn(0x7f60a806c670 0x7f60a806ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50758/0 (socket says 172.21.15.86:50758) 2024-04-25T14:14:12.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.678+0000 7f60a6a00640 1 --2- 172.21.15.86:0/1151612958 >> v2:172.21.15.155:3300/0 conn(0x7f60a80cfca0 0x7f60a806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.678+0000 7f609e000640 1 -- 172.21.15.86:0/1151612958 >> v2:172.21.15.155:3300/0 conn(0x7f60a80cfca0 msgr2=0x7f60a806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:12.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.678+0000 7f609e000640 1 --2- 172.21.15.86:0/1151612958 >> v2:172.21.15.155:3300/0 conn(0x7f60a80cfca0 0x7f60a806f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.678+0000 7f609e000640 1 -- 172.21.15.86:0/1151612958 >> v1:172.21.15.86:6789/0 conn(0x7f60a806c670 legacy=0x7f60a806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.678+0000 7f609e000640 1 -- 172.21.15.86:0/1151612958 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a806f930 con 0x7f60a80d16e0 2024-04-25T14:14:12.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.682+0000 7f609e000640 1 --2- 172.21.15.86:0/1151612958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60a80d16e0 0x7f60a80d1af0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f60a80d22a0 tx=0x7f609801ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86bb1d9a323eb242 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.682+0000 7f60a6000640 1 -- 172.21.15.86:0/1151612958 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60980b4070 con 0x7f60a80d16e0 2024-04-25T14:14:12.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.682+0000 7f60a6000640 1 -- 172.21.15.86:0/1151612958 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60980a7ad0 con 0x7f60a80d16e0 2024-04-25T14:14:12.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.682+0000 7f60a6000640 1 -- 172.21.15.86:0/1151612958 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60980af440 con 0x7f60a80d16e0 2024-04-25T14:14:12.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.682+0000 7f60ade00640 1 -- 172.21.15.86:0/1151612958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60a80d16e0 msgr2=0x7f60a80d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.682+0000 7f60ade00640 1 --2- 172.21.15.86:0/1151612958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60a80d16e0 0x7f60a80d1af0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f60a80d22a0 tx=0x7f609801ebf0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.682+0000 7f60ade00640 1 -- 172.21.15.86:0/1151612958 shutdown_connections 2024-04-25T14:14:12.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.682+0000 7f60ade00640 1 --2- 172.21.15.86:0/1151612958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60a80d16e0 0x7f60a80d1af0 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.682+0000 7f60ade00640 1 --2- 172.21.15.86:0/1151612958 >> v2:172.21.15.155:3300/0 conn(0x7f60a80cfca0 0x7f60a806f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.682+0000 7f60ade00640 1 -- 172.21.15.86:0/1151612958 >> 172.21.15.86:0/1151612958 conn(0x7f60a80b4650 msgr2=0x7f60a80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.682+0000 7f60ade00640 1 -- 172.21.15.86:0/1151612958 shutdown_connections 2024-04-25T14:14:12.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.682+0000 7f60ade00640 1 -- 172.21.15.86:0/1151612958 wait complete. 2024-04-25T14:14:12.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.686+0000 7f60ade00640 1 Processor -- start 2024-04-25T14:14:12.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.686+0000 7f60ade00640 1 -- start start 2024-04-25T14:14:12.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.686+0000 7f60ade00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60a806c670 0x7f60a81f9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.686+0000 7f60ade00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60a80cfca0 0x7f60a80ae130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.686+0000 7f60ade00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60a80d16e0 0x7f60a80c3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.686+0000 7f60ade00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60a80ce6f0 con 0x7f60a80d16e0 2024-04-25T14:14:12.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.686+0000 7f60ade00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f60a80ce570 con 0x7f60a80cfca0 2024-04-25T14:14:12.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.686+0000 7f60ade00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f60a80ce870 con 0x7f60a806c670 2024-04-25T14:14:12.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.686+0000 7f60a7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60a80d16e0 0x7f60a80c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.686+0000 7f60a7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60a80d16e0 0x7f60a80c3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51130/0 (socket says 172.21.15.86:51130) 2024-04-25T14:14:12.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.686+0000 7f60a7400640 1 -- 172.21.15.86:0/862249163 learned_addr learned my addr 172.21.15.86:0/862249163 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:12.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.686+0000 7f60a6a00640 1 --2- 172.21.15.86:0/862249163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60a806c670 0x7f60a81f9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.686+0000 7f609e000640 1 --2- 172.21.15.86:0/862249163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60a80cfca0 0x7f60a80ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.686+0000 7f60a7400640 1 -- 172.21.15.86:0/862249163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60a806c670 msgr2=0x7f60a81f9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.686+0000 7f60a7400640 1 --2- 172.21.15.86:0/862249163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60a806c670 0x7f60a81f9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.686+0000 7f60a7400640 1 -- 172.21.15.86:0/862249163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60a80cfca0 msgr2=0x7f60a80ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.686+0000 7f60a7400640 1 --2- 172.21.15.86:0/862249163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60a80cfca0 0x7f60a80ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.686+0000 7f60a7400640 1 -- 172.21.15.86:0/862249163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a80cdb90 con 0x7f60a80d16e0 2024-04-25T14:14:12.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.686+0000 7f60a7400640 1 --2- 172.21.15.86:0/862249163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60a80d16e0 0x7f60a80c3660 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f60a000c6c0 tx=0x7f60a000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.686+0000 7f609fe00640 1 -- 172.21.15.86:0/862249163 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a0018070 con 0x7f60a80d16e0 2024-04-25T14:14:12.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.686+0000 7f609fe00640 1 -- 172.21.15.86:0/862249163 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60a0004d70 con 0x7f60a80d16e0 2024-04-25T14:14:12.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.686+0000 7f609fe00640 1 -- 172.21.15.86:0/862249163 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a000a840 con 0x7f60a80d16e0 2024-04-25T14:14:12.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.686+0000 7f60ade00640 1 -- 172.21.15.86:0/862249163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a80c25c0 con 0x7f60a80d16e0 2024-04-25T14:14:12.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.686+0000 7f60ade00640 1 -- 172.21.15.86:0/862249163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a80c2b80 con 0x7f60a80d16e0 2024-04-25T14:14:12.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.690+0000 7f60ade00640 1 -- 172.21.15.86:0/862249163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60a809f4b0 con 0x7f60a80d16e0 2024-04-25T14:14:12.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.690+0000 7f609fe00640 1 -- 172.21.15.86:0/862249163 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f60a000a9e0 con 0x7f60a80d16e0 2024-04-25T14:14:12.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.690+0000 7f609fe00640 1 --2- 172.21.15.86:0/862249163 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f608406f570 0x7f6084071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:12.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.690+0000 7f609fe00640 1 -- 172.21.15.86:0/862249163 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f60a008ef80 con 0x7f60a80d16e0 2024-04-25T14:14:12.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.690+0000 7f60a6a00640 1 --2- 172.21.15.86:0/862249163 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f608406f570 0x7f6084071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:12.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.690+0000 7f60a6a00640 1 --2- 172.21.15.86:0/862249163 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f608406f570 0x7f6084071a80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f60a8070710 tx=0x7f6090008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:12.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.694+0000 7f609fe00640 1 -- 172.21.15.86:0/862249163 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f60a80d2030 con 0x7f60a80d16e0 2024-04-25T14:14:12.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.874+0000 7f60ade00640 1 -- 172.21.15.86:0/862249163 --> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] -- mgr_command(tid 0: {"prefix": "fs perf stats", "target": ["mon-mgr", ""]}) -- 0x7f60a8069d00 con 0x7f608406f570 2024-04-25T14:14:12.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.878+0000 7f609fe00640 1 -- 172.21.15.86:0/862249163 <== mgr.9913 v2:172.21.15.86:6832/2318291413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3053 (secure 0 0 0) 0x7f60a8069d00 con 0x7f608406f570 2024-04-25T14:14:12.872 INFO:teuthology.orchestra.run.smithi086.stdout:{"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {"cephfs": {"client.8492": {"hostname": "smithi155", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:192.168.0.1"}, "client.9880": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "kernel_version": "6.9.0-rc3-g806ec36b2112", "IP": "v1:172.21.15.86"}, "client.9946": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}, "client.9949": {"hostname": "smithi086", "root": "/", "mount_point": "N/A", "valid_metrics": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", "opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metadata_latency", "stdev_metadata_latency"], "IP": "172.21.15.86"}}}, "global_metrics": {"cephfs": {"client.8492": [[48, 2], [0, 0], [0, 0], [0, 17450075], [6, 0], [0, 3], [3, 3], [0, 3], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 1938897], [39594750870950, 9]], "client.9880": [[415, 22], [0, 0], [0, 434554230], [0, 60063119], [34, 0], [0, 35], [35, 35], [0, 35], [0, 0], [32, 33554432], [0, 0], [0, 0], [0, 13579821], [221586358188848, 32], [0, 938476], [61486942346492, 64]], "client.9946": [[2, 0], [0, 0], [0, 0], [0, 890756], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 890756], [0, 1]], "client.9949": [[2, 0], [0, 0], [0, 0], [0, 695530], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 695530], [0, 1]]}}, "metrics": {"delayed_ranks": [], "mds.0": {"client.8492": [], "client.9880": [], "client.9946": [], "client.9949": []}, "mds.1": {"client.8492": [], "client.9880": []}}} 2024-04-25T14:14:12.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.878+0000 7f60ade00640 1 -- 172.21.15.86:0/862249163 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f608406f570 msgr2=0x7f6084071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.878+0000 7f60ade00640 1 --2- 172.21.15.86:0/862249163 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f608406f570 0x7f6084071a80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f60a8070710 tx=0x7f6090008040 comp rx=0 tx=0).stop 2024-04-25T14:14:12.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.878+0000 7f60ade00640 1 -- 172.21.15.86:0/862249163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60a80d16e0 msgr2=0x7f60a80c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:12.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.878+0000 7f60ade00640 1 --2- 172.21.15.86:0/862249163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60a80d16e0 0x7f60a80c3660 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f60a000c6c0 tx=0x7f60a000cb80 comp rx=0 tx=0).stop 2024-04-25T14:14:12.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.878+0000 7f60ade00640 1 -- 172.21.15.86:0/862249163 shutdown_connections 2024-04-25T14:14:12.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.878+0000 7f60ade00640 1 --2- 172.21.15.86:0/862249163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60a80d16e0 0x7f60a80c3660 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.878+0000 7f60ade00640 1 --2- 172.21.15.86:0/862249163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60a80cfca0 0x7f60a80ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.878+0000 7f60ade00640 1 --2- 172.21.15.86:0/862249163 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f608406f570 0x7f6084071a80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.878+0000 7f60ade00640 1 --2- 172.21.15.86:0/862249163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60a806c670 0x7f60a81f9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.878+0000 7f60ade00640 1 -- 172.21.15.86:0/862249163 >> 172.21.15.86:0/862249163 conn(0x7f60a80b4650 msgr2=0x7f60a80cc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.878+0000 7f60ade00640 1 -- 172.21.15.86:0/862249163 shutdown_connections 2024-04-25T14:14:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:12.878+0000 7f60ade00640 1 -- 172.21.15.86:0/862249163 wait complete. 2024-04-25T14:14:12.894 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:13.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.046+0000 7f5eaa000640 1 Processor -- start 2024-04-25T14:14:13.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.046+0000 7f5eaa000640 1 -- start start 2024-04-25T14:14:13.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.046+0000 7f5eaa000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5ea40d1b40 0x7f5ea40b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.046+0000 7f5eaa000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5ea415dcd0 0x7f5ea415e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.046+0000 7f5eaa000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ea4161b30 con 0x7f5ea40ced30 2024-04-25T14:14:13.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.046+0000 7f5eaa000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5ea41618e0 con 0x7f5ea40d1b40 2024-04-25T14:14:13.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.046+0000 7f5eaa000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f5ea4161e10 con 0x7f5ea415dcd0 2024-04-25T14:14:13.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.046+0000 7f5ea8c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5ea40ced30 0x7f5ea40d1430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50772/0 (socket says 172.21.15.86:50772) 2024-04-25T14:14:13.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.046+0000 7f5ea8c00640 1 -- 172.21.15.86:0/3942285458 learned_addr learned my addr 172.21.15.86:0/3942285458 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:14:13.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.046+0000 7f5e9be00640 1 --2- 172.21.15.86:0/3942285458 >> v2:172.21.15.86:3300/0 conn(0x7f5ea40d1b40 0x7f5ea40b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.050+0000 7f5ea9600640 1 --2- 172.21.15.86:0/3942285458 >> v2:172.21.15.155:3300/0 conn(0x7f5ea415dcd0 0x7f5ea415e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.050+0000 7f5e9be00640 1 -- 172.21.15.86:0/3942285458 >> v2:172.21.15.155:3300/0 conn(0x7f5ea415dcd0 msgr2=0x7f5ea415e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:13.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.050+0000 7f5e9be00640 1 --2- 172.21.15.86:0/3942285458 >> v2:172.21.15.155:3300/0 conn(0x7f5ea415dcd0 0x7f5ea415e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.050+0000 7f5e9be00640 1 -- 172.21.15.86:0/3942285458 >> v1:172.21.15.86:6789/0 conn(0x7f5ea40ced30 legacy=0x7f5ea40d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:13.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.050+0000 7f5e9be00640 1 -- 172.21.15.86:0/3942285458 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea41620f0 con 0x7f5ea40d1b40 2024-04-25T14:14:13.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.050+0000 7f5e9be00640 1 --2- 172.21.15.86:0/3942285458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ea40d1b40 0x7f5ea40b1e20 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4073940 tx=0x7f5e8c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9505a27b75c88e9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:13.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.050+0000 7f5ea3e00640 1 -- 172.21.15.86:0/3942285458 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c0b1070 con 0x7f5ea40d1b40 2024-04-25T14:14:13.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.050+0000 7f5ea3e00640 1 -- 172.21.15.86:0/3942285458 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e8c0ac3f0 con 0x7f5ea40d1b40 2024-04-25T14:14:13.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.050+0000 7f5ea3e00640 1 -- 172.21.15.86:0/3942285458 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c0ac7c0 con 0x7f5ea40d1b40 2024-04-25T14:14:13.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.054+0000 7f5eaa000640 1 -- 172.21.15.86:0/3942285458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ea40d1b40 msgr2=0x7f5ea40b1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:13.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.054+0000 7f5eaa000640 1 --2- 172.21.15.86:0/3942285458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ea40d1b40 0x7f5ea40b1e20 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4073940 tx=0x7f5e8c01c480 comp rx=0 tx=0).stop 2024-04-25T14:14:13.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.054+0000 7f5eaa000640 1 -- 172.21.15.86:0/3942285458 shutdown_connections 2024-04-25T14:14:13.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.054+0000 7f5eaa000640 1 --2- 172.21.15.86:0/3942285458 >> v2:172.21.15.155:3300/0 conn(0x7f5ea415dcd0 0x7f5ea415e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.054+0000 7f5eaa000640 1 --2- 172.21.15.86:0/3942285458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ea40d1b40 0x7f5ea40b1e20 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.054+0000 7f5eaa000640 1 -- 172.21.15.86:0/3942285458 >> 172.21.15.86:0/3942285458 conn(0x7f5ea409ba30 msgr2=0x7f5ea40b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:13.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.054+0000 7f5eaa000640 1 -- 172.21.15.86:0/3942285458 shutdown_connections 2024-04-25T14:14:13.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.054+0000 7f5eaa000640 1 -- 172.21.15.86:0/3942285458 wait complete. 2024-04-25T14:14:13.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.054+0000 7f5eaa000640 1 Processor -- start 2024-04-25T14:14:13.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.054+0000 7f5eaa000640 1 -- start start 2024-04-25T14:14:13.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.054+0000 7f5eaa000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ea40ced30 0x7f5ea4151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.054+0000 7f5eaa000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ea40d1b40 0x7f5ea40b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.058+0000 7f5eaa000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ea415dcd0 0x7f5ea413f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.058+0000 7f5eaa000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea416abf0 con 0x7f5ea40ced30 2024-04-25T14:14:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.058+0000 7f5eaa000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea416aa70 con 0x7f5ea40d1b40 2024-04-25T14:14:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.058+0000 7f5eaa000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5ea416ad70 con 0x7f5ea415dcd0 2024-04-25T14:14:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.058+0000 7f5ea8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ea40ced30 0x7f5ea4151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.058+0000 7f5ea8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ea40ced30 0x7f5ea4151880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51150/0 (socket says 172.21.15.86:51150) 2024-04-25T14:14:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.058+0000 7f5ea8c00640 1 -- 172.21.15.86:0/3552931906 learned_addr learned my addr 172.21.15.86:0/3552931906 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.058+0000 7f5ea8c00640 1 -- 172.21.15.86:0/3552931906 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ea415dcd0 msgr2=0x7f5ea413f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.058+0000 7f5ea8c00640 1 --2- 172.21.15.86:0/3552931906 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ea415dcd0 0x7f5ea413f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.058+0000 7f5ea8c00640 1 -- 172.21.15.86:0/3552931906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ea40d1b40 msgr2=0x7f5ea40b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.058+0000 7f5ea8c00640 1 --2- 172.21.15.86:0/3552931906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ea40d1b40 0x7f5ea40b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.058+0000 7f5ea8c00640 1 -- 172.21.15.86:0/3552931906 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea41597e0 con 0x7f5ea40ced30 2024-04-25T14:14:13.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.058+0000 7f5ea8c00640 1 --2- 172.21.15.86:0/3552931906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ea40ced30 0x7f5ea4151880 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f5e9400c340 tx=0x7f5e9400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:13.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.058+0000 7f5ea1600640 1 -- 172.21.15.86:0/3552931906 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e94017070 con 0x7f5ea40ced30 2024-04-25T14:14:13.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.058+0000 7f5ea1600640 1 -- 172.21.15.86:0/3552931906 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e94005cf0 con 0x7f5ea40ced30 2024-04-25T14:14:13.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.058+0000 7f5eaa000640 1 -- 172.21.15.86:0/3552931906 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea4159aa0 con 0x7f5ea40ced30 2024-04-25T14:14:13.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.058+0000 7f5ea1600640 1 -- 172.21.15.86:0/3552931906 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e940072f0 con 0x7f5ea40ced30 2024-04-25T14:14:13.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.058+0000 7f5eaa000640 1 -- 172.21.15.86:0/3552931906 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea415a060 con 0x7f5ea40ced30 2024-04-25T14:14:13.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.058+0000 7f5eaa000640 1 -- 172.21.15.86:0/3552931906 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ea4076b80 con 0x7f5ea40ced30 2024-04-25T14:14:13.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.058+0000 7f5ea1600640 1 -- 172.21.15.86:0/3552931906 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f5e9401a070 con 0x7f5ea40ced30 2024-04-25T14:14:13.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.062+0000 7f5ea1600640 1 --2- 172.21.15.86:0/3552931906 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f5e8006f570 0x7f5e80071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.062+0000 7f5ea1600640 1 -- 172.21.15.86:0/3552931906 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f5e9408ea50 con 0x7f5ea40ced30 2024-04-25T14:14:13.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.062+0000 7f5e9be00640 1 --2- 172.21.15.86:0/3552931906 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f5e8006f570 0x7f5e80071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.062+0000 7f5e9be00640 1 --2- 172.21.15.86:0/3552931906 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f5e8006f570 0x7f5e80071a80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c0ae000 tx=0x7f5e8c004740 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:13.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.066+0000 7f5ea1600640 1 -- 172.21.15.86:0/3552931906 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5e9405b5e0 con 0x7f5ea40ced30 2024-04-25T14:14:13.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.258+0000 7f5eaa000640 1 -- 172.21.15.86:0/3552931906 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ea40b79c0 con 0x7f5ea40ced30 2024-04-25T14:14:13.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.262+0000 7f5ea1600640 1 -- 172.21.15.86:0/3552931906 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) ==== 95+0+5222 (secure 0 0 0) 0x7f5e9405aee0 con 0x7f5ea40ced30 2024-04-25T14:14:13.256 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:13.256 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":121,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:09.363511+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209,8387]},"id":11}]} 2024-04-25T14:14:13.256 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 121 2024-04-25T14:14:13.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.262+0000 7f5eaa000640 1 -- 172.21.15.86:0/3552931906 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f5e8006f570 msgr2=0x7f5e80071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:13.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.262+0000 7f5eaa000640 1 --2- 172.21.15.86:0/3552931906 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f5e8006f570 0x7f5e80071a80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c0ae000 tx=0x7f5e8c004740 comp rx=0 tx=0).stop 2024-04-25T14:14:13.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.262+0000 7f5eaa000640 1 -- 172.21.15.86:0/3552931906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ea40ced30 msgr2=0x7f5ea4151880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:13.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.262+0000 7f5eaa000640 1 --2- 172.21.15.86:0/3552931906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ea40ced30 0x7f5ea4151880 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f5e9400c340 tx=0x7f5e9400c800 comp rx=0 tx=0).stop 2024-04-25T14:14:13.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.262+0000 7f5eaa000640 1 -- 172.21.15.86:0/3552931906 shutdown_connections 2024-04-25T14:14:13.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.262+0000 7f5eaa000640 1 --2- 172.21.15.86:0/3552931906 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f5e8006f570 0x7f5e80071a80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.262+0000 7f5eaa000640 1 --2- 172.21.15.86:0/3552931906 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ea415dcd0 0x7f5ea413f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.262+0000 7f5eaa000640 1 --2- 172.21.15.86:0/3552931906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ea40d1b40 0x7f5ea40b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.262+0000 7f5eaa000640 1 --2- 172.21.15.86:0/3552931906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ea40ced30 0x7f5ea4151880 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.262+0000 7f5eaa000640 1 -- 172.21.15.86:0/3552931906 >> 172.21.15.86:0/3552931906 conn(0x7f5ea409ba30 msgr2=0x7f5ea415b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:13.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.262+0000 7f5eaa000640 1 -- 172.21.15.86:0/3552931906 shutdown_connections 2024-04-25T14:14:13.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.262+0000 7f5eaa000640 1 -- 172.21.15.86:0/3552931906 wait complete. 2024-04-25T14:14:13.278 DEBUG:tasks.cephfs.test_mds_metrics:metrics={'version': 2, 'global_counters': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'counters': [], 'client_metadata': {'cephfs': {'client.8492': {'hostname': 'smithi155', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'kernel_version': '6.9.0-rc3-g806ec36b2112', 'IP': 'v1:192.168.0.1'}, 'client.9880': {'hostname': 'smithi086', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'kernel_version': '6.9.0-rc3-g806ec36b2112', 'IP': 'v1:172.21.15.86'}, 'client.9946': {'hostname': 'smithi086', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'IP': '172.21.15.86'}, 'client.9949': {'hostname': 'smithi086', 'root': '/', 'mount_point': 'N/A', 'valid_metrics': ['cap_hit', 'read_latency', 'write_latency', 'metadata_latency', 'dentry_lease', 'opened_files', 'pinned_icaps', 'opened_inodes', 'read_io_sizes', 'write_io_sizes', 'avg_read_latency', 'stdev_read_latency', 'avg_write_latency', 'stdev_write_latency', 'avg_metadata_latency', 'stdev_metadata_latency'], 'IP': '172.21.15.86'}}}, 'global_metrics': {'cephfs': {'client.8492': [[48, 2], [0, 0], [0, 0], [0, 17450075], [6, 0], [0, 3], [3, 3], [0, 3], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 1938897], [39594750870950, 9]], 'client.9880': [[415, 22], [0, 0], [0, 434554230], [0, 60063119], [34, 0], [0, 35], [35, 35], [0, 35], [0, 0], [32, 33554432], [0, 0], [0, 0], [0, 13579821], [221586358188848, 32], [0, 938476], [61486942346492, 64]], 'client.9946': [[2, 0], [0, 0], [0, 0], [0, 890756], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 890756], [0, 1]], 'client.9949': [[2, 0], [0, 0], [0, 0], [0, 695530], [0, 0], [0, 1], [1, 1], [0, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 695530], [0, 1]]}}, 'metrics': {'delayed_ranks': [], 'mds.0': {'client.8492': [], 'client.9880': [], 'client.9946': [], 'client.9949': []}, 'mds.1': {'client.8492': [], 'client.9880': []}}} 2024-04-25T14:14:13.278 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:13.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.430+0000 7fdada600640 1 Processor -- start 2024-04-25T14:14:13.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.430+0000 7fdada600640 1 -- start start 2024-04-25T14:14:13.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.430+0000 7fdada600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdad40d2c80 0x7fdad415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.430+0000 7fdada600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdad4070e20 0x7fdad4071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.430+0000 7fdada600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdad40722f0 0x7fdad4072700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.430+0000 7fdada600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdad40ce720 con 0x7fdad40d2c80 2024-04-25T14:14:13.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.430+0000 7fdada600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fdad4071770 con 0x7fdad4070e20 2024-04-25T14:14:13.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.430+0000 7fdada600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdad40cd8d0 con 0x7fdad40722f0 2024-04-25T14:14:13.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.430+0000 7fdad3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdad40d2c80 0x7fdad415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.430+0000 7fdad3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdad40d2c80 0x7fdad415dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51160/0 (socket says 172.21.15.86:51160) 2024-04-25T14:14:13.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.430+0000 7fdad3400640 1 -- 172.21.15.86:0/3966732896 learned_addr learned my addr 172.21.15.86:0/3966732896 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:13.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.430+0000 7fdad2a00640 1 --2- 172.21.15.86:0/3966732896 >> v2:172.21.15.155:3300/0 conn(0x7fdad4070e20 0x7fdad4071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.430+0000 7fdad3e00640 1 --2- 172.21.15.86:0/3966732896 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdad40722f0 0x7fdad4072700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.430+0000 7fdad3400640 1 -- 172.21.15.86:0/3966732896 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdad40722f0 msgr2=0x7fdad4072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:13.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.430+0000 7fdad3400640 1 --2- 172.21.15.86:0/3966732896 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdad40722f0 0x7fdad4072700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.430+0000 7fdad3400640 1 -- 172.21.15.86:0/3966732896 >> v2:172.21.15.155:3300/0 conn(0x7fdad4070e20 msgr2=0x7fdad4071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:13.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.430+0000 7fdad3400640 1 --2- 172.21.15.86:0/3966732896 >> v2:172.21.15.155:3300/0 conn(0x7fdad4070e20 0x7fdad4071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.430+0000 7fdad3400640 1 -- 172.21.15.86:0/3966732896 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad40ce970 con 0x7fdad40d2c80 2024-04-25T14:14:13.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.430+0000 7fdad2a00640 1 --2- 172.21.15.86:0/3966732896 >> v2:172.21.15.155:3300/0 conn(0x7fdad4070e20 0x7fdad4071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:14:13.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.430+0000 7fdad3e00640 1 --2- 172.21.15.86:0/3966732896 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdad40722f0 0x7fdad4072700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:14:13.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.434+0000 7fdad3400640 1 --2- 172.21.15.86:0/3966732896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdad40d2c80 0x7fdad415dc90 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7fdad40b6070 tx=0x7fdac401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13779f51d783b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:13.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.434+0000 7fdad2000640 1 -- 172.21.15.86:0/3966732896 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac40b5070 con 0x7fdad40d2c80 2024-04-25T14:14:13.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.434+0000 7fdad2000640 1 -- 172.21.15.86:0/3966732896 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdac40a89d0 con 0x7fdad40d2c80 2024-04-25T14:14:13.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.434+0000 7fdad2000640 1 -- 172.21.15.86:0/3966732896 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac40a8da0 con 0x7fdad40d2c80 2024-04-25T14:14:13.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.434+0000 7fdada600640 1 -- 172.21.15.86:0/3966732896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdad40d2c80 msgr2=0x7fdad415dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:13.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.434+0000 7fdada600640 1 --2- 172.21.15.86:0/3966732896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdad40d2c80 0x7fdad415dc90 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7fdad40b6070 tx=0x7fdac401ed10 comp rx=0 tx=0).stop 2024-04-25T14:14:13.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.434+0000 7fdada600640 1 -- 172.21.15.86:0/3966732896 shutdown_connections 2024-04-25T14:14:13.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.434+0000 7fdada600640 1 --2- 172.21.15.86:0/3966732896 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdad40722f0 0x7fdad4072700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.434+0000 7fdada600640 1 --2- 172.21.15.86:0/3966732896 >> v2:172.21.15.155:3300/0 conn(0x7fdad4070e20 0x7fdad4071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.434+0000 7fdada600640 1 --2- 172.21.15.86:0/3966732896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdad40d2c80 0x7fdad415dc90 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.434+0000 7fdada600640 1 -- 172.21.15.86:0/3966732896 >> 172.21.15.86:0/3966732896 conn(0x7fdad40cffc0 msgr2=0x7fdad40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:13.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.434+0000 7fdada600640 1 -- 172.21.15.86:0/3966732896 shutdown_connections 2024-04-25T14:14:13.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.434+0000 7fdada600640 1 -- 172.21.15.86:0/3966732896 wait complete. 2024-04-25T14:14:13.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.438+0000 7fdada600640 1 Processor -- start 2024-04-25T14:14:13.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.438+0000 7fdada600640 1 -- start start 2024-04-25T14:14:13.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.438+0000 7fdada600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdad4070e20 0x7fdad4150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.438+0000 7fdada600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdad40722f0 0x7fdad415ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.438+0000 7fdada600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdad40d2c80 0x7fdad4154980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.438+0000 7fdada600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdad40d21a0 con 0x7fdad4070e20 2024-04-25T14:14:13.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.438+0000 7fdada600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdad40d1ec0 con 0x7fdad40d2c80 2024-04-25T14:14:13.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.438+0000 7fdada600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdad40d2480 con 0x7fdad40722f0 2024-04-25T14:14:13.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.438+0000 7fdad3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdad4070e20 0x7fdad4150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.438+0000 7fdad3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdad4070e20 0x7fdad4150910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51164/0 (socket says 172.21.15.86:51164) 2024-04-25T14:14:13.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.438+0000 7fdad3400640 1 -- 172.21.15.86:0/1309186346 learned_addr learned my addr 172.21.15.86:0/1309186346 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:13.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.438+0000 7fdad2a00640 1 --2- 172.21.15.86:0/1309186346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdad40722f0 0x7fdad415ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.438+0000 7fdad3e00640 1 --2- 172.21.15.86:0/1309186346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdad40d2c80 0x7fdad4154980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.438+0000 7fdad3400640 1 -- 172.21.15.86:0/1309186346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdad40722f0 msgr2=0x7fdad415ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:13.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.438+0000 7fdad3400640 1 --2- 172.21.15.86:0/1309186346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdad40722f0 0x7fdad415ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.438+0000 7fdad3400640 1 -- 172.21.15.86:0/1309186346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdad40d2c80 msgr2=0x7fdad4154980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:13.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.438+0000 7fdad3400640 1 --2- 172.21.15.86:0/1309186346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdad40d2c80 0x7fdad4154980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.438+0000 7fdad3400640 1 -- 172.21.15.86:0/1309186346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad4160780 con 0x7fdad4070e20 2024-04-25T14:14:13.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.438+0000 7fdad3e00640 1 --2- 172.21.15.86:0/1309186346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdad40d2c80 0x7fdad4154980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:13.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.438+0000 7fdad3400640 1 --2- 172.21.15.86:0/1309186346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdad4070e20 0x7fdad4150910 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7fdac401d3c0 tx=0x7fdac401bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:13.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.438+0000 7fdacb400640 1 -- 172.21.15.86:0/1309186346 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac40b5070 con 0x7fdad4070e20 2024-04-25T14:14:13.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.442+0000 7fdacb400640 1 -- 172.21.15.86:0/1309186346 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdac40b0c20 con 0x7fdad4070e20 2024-04-25T14:14:13.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.442+0000 7fdacb400640 1 -- 172.21.15.86:0/1309186346 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac40bb410 con 0x7fdad4070e20 2024-04-25T14:14:13.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.442+0000 7fdada600640 1 -- 172.21.15.86:0/1309186346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad415f6e0 con 0x7fdad4070e20 2024-04-25T14:14:13.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.442+0000 7fdada600640 1 -- 172.21.15.86:0/1309186346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad415fca0 con 0x7fdad4070e20 2024-04-25T14:14:13.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.442+0000 7fdada600640 1 -- 172.21.15.86:0/1309186346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdad40720e0 con 0x7fdad4070e20 2024-04-25T14:14:13.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.442+0000 7fdacb400640 1 -- 172.21.15.86:0/1309186346 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7fdac40c2060 con 0x7fdad4070e20 2024-04-25T14:14:13.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.442+0000 7fdacb400640 1 --2- 172.21.15.86:0/1309186346 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fdab406f570 0x7fdab4071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.442+0000 7fdacb400640 1 -- 172.21.15.86:0/1309186346 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fdac412c800 con 0x7fdad4070e20 2024-04-25T14:14:13.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.442+0000 7fdad2a00640 1 --2- 172.21.15.86:0/1309186346 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fdab406f570 0x7fdab4071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.446+0000 7fdad2a00640 1 --2- 172.21.15.86:0/1309186346 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fdab406f570 0x7fdab4071a80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdac0006fd0 tx=0x7fdac0008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:13.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.446+0000 7fdacb400640 1 -- 172.21.15.86:0/1309186346 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdac4018870 con 0x7fdad4070e20 2024-04-25T14:14:13.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.634+0000 7fdada600640 1 -- 172.21.15.86:0/1309186346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdad4076bf0 con 0x7fdad4070e20 2024-04-25T14:14:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.634+0000 7fdacb400640 1 -- 172.21.15.86:0/1309186346 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) ==== 95+0+5222 (secure 0 0 0) 0x7fdac40f9420 con 0x7fdad4070e20 2024-04-25T14:14:13.631 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:13.631 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":121,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:09.363511+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:active","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209,8387]},"id":11}]} 2024-04-25T14:14:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 121 2024-04-25T14:14:13.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.638+0000 7fdada600640 1 -- 172.21.15.86:0/1309186346 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fdab406f570 msgr2=0x7fdab4071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:13.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.638+0000 7fdada600640 1 --2- 172.21.15.86:0/1309186346 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fdab406f570 0x7fdab4071a80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdac0006fd0 tx=0x7fdac0008040 comp rx=0 tx=0).stop 2024-04-25T14:14:13.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.638+0000 7fdada600640 1 -- 172.21.15.86:0/1309186346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdad4070e20 msgr2=0x7fdad4150910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:13.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.638+0000 7fdada600640 1 --2- 172.21.15.86:0/1309186346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdad4070e20 0x7fdad4150910 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7fdac401d3c0 tx=0x7fdac401bab0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.638+0000 7fdada600640 1 -- 172.21.15.86:0/1309186346 shutdown_connections 2024-04-25T14:14:13.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.638+0000 7fdada600640 1 --2- 172.21.15.86:0/1309186346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdad40d2c80 0x7fdad4154980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.638+0000 7fdada600640 1 --2- 172.21.15.86:0/1309186346 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fdab406f570 0x7fdab4071a80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.638+0000 7fdada600640 1 --2- 172.21.15.86:0/1309186346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdad40722f0 0x7fdad415ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.638+0000 7fdada600640 1 --2- 172.21.15.86:0/1309186346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdad4070e20 0x7fdad4150910 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.638+0000 7fdada600640 1 -- 172.21.15.86:0/1309186346 >> 172.21.15.86:0/1309186346 conn(0x7fdad40cffc0 msgr2=0x7fdad40ca580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:13.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.638+0000 7fdada600640 1 -- 172.21.15.86:0/1309186346 shutdown_connections 2024-04-25T14:14:13.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.638+0000 7fdada600640 1 -- 172.21.15.86:0/1309186346 wait complete. 2024-04-25T14:14:13.653 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 1 2024-04-25T14:14:13.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.806+0000 7f1b6e800640 1 Processor -- start 2024-04-25T14:14:13.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.806+0000 7f1b6e800640 1 -- start start 2024-04-25T14:14:13.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.810+0000 7f1b6e800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1b68070330 0x7f1b680d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.810+0000 7f1b6e800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b680d2030 con 0x7f1b680d1c20 2024-04-25T14:14:13.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.810+0000 7f1b6e800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b6815e5c0 con 0x7f1b680ce790 2024-04-25T14:14:13.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.810+0000 7f1b6e800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1b6815e8a0 con 0x7f1b68070330 2024-04-25T14:14:13.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.810+0000 7f1b67400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1b68070330 0x7f1b680d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.810+0000 7f1b67400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1b68070330 0x7f1b680d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51178/0 (socket says 172.21.15.86:51178) 2024-04-25T14:14:13.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.810+0000 7f1b67400640 1 -- 172.21.15.86:0/1446129947 learned_addr learned my addr 172.21.15.86:0/1446129947 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:13.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.810+0000 7f1b67400640 1 -- 172.21.15.86:0/1446129947 >> v1:172.21.15.155:6789/0 conn(0x7f1b680ce790 legacy=0x7f1b6806f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:13.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.810+0000 7f1b67400640 1 -- 172.21.15.86:0/1446129947 >> v1:172.21.15.86:6789/0 conn(0x7f1b680d1c20 legacy=0x7f1b6806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:13.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.810+0000 7f1b67400640 1 -- 172.21.15.86:0/1446129947 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6815eb80 con 0x7f1b68070330 2024-04-25T14:14:13.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.814+0000 7f1b67400640 1 --2- 172.21.15.86:0/1446129947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b68070330 0x7f1b680d16e0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f1b580098a0 tx=0x7f1b5801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5a782ab3a72854 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:13.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.814+0000 7f1b66000640 1 -- 172.21.15.86:0/1446129947 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b580b2070 con 0x7f1b68070330 2024-04-25T14:14:13.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.814+0000 7f1b66000640 1 -- 172.21.15.86:0/1446129947 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b580ad680 con 0x7f1b68070330 2024-04-25T14:14:13.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.814+0000 7f1b66000640 1 -- 172.21.15.86:0/1446129947 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b580ada50 con 0x7f1b68070330 2024-04-25T14:14:13.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.814+0000 7f1b6e800640 1 -- 172.21.15.86:0/1446129947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b68070330 msgr2=0x7f1b680d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:13.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.814+0000 7f1b6e800640 1 --2- 172.21.15.86:0/1446129947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b68070330 0x7f1b680d16e0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f1b580098a0 tx=0x7f1b5801c4b0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.814+0000 7f1b6e800640 1 -- 172.21.15.86:0/1446129947 shutdown_connections 2024-04-25T14:14:13.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.814+0000 7f1b6e800640 1 --2- 172.21.15.86:0/1446129947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b68070330 0x7f1b680d16e0 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.814+0000 7f1b6e800640 1 -- 172.21.15.86:0/1446129947 >> 172.21.15.86:0/1446129947 conn(0x7f1b680b4650 msgr2=0x7f1b680b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:13.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.814+0000 7f1b6e800640 1 -- 172.21.15.86:0/1446129947 shutdown_connections 2024-04-25T14:14:13.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.814+0000 7f1b6e800640 1 -- 172.21.15.86:0/1446129947 wait complete. 2024-04-25T14:14:13.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.814+0000 7f1b6e800640 1 Processor -- start 2024-04-25T14:14:13.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.814+0000 7f1b6e800640 1 -- start start 2024-04-25T14:14:13.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.818+0000 7f1b6e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b68070330 0x7f1b68098950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.818+0000 7f1b6e800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b680ce790 0x7f1b681555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.818+0000 7f1b6e800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b680d1c20 0x7f1b68155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.818+0000 7f1b6e800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b680708c0 con 0x7f1b68070330 2024-04-25T14:14:13.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.818+0000 7f1b6e800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1b68070740 con 0x7f1b680ce790 2024-04-25T14:14:13.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.818+0000 7f1b6e800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1b6806fb50 con 0x7f1b680d1c20 2024-04-25T14:14:13.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.818+0000 7f1b67400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b68070330 0x7f1b68098950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.818+0000 7f1b67400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b68070330 0x7f1b68098950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51188/0 (socket says 172.21.15.86:51188) 2024-04-25T14:14:13.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.818+0000 7f1b67400640 1 -- 172.21.15.86:0/1633794215 learned_addr learned my addr 172.21.15.86:0/1633794215 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:13.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.818+0000 7f1b67e00640 1 --2- 172.21.15.86:0/1633794215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b680d1c20 0x7f1b68155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.818+0000 7f1b66a00640 1 --2- 172.21.15.86:0/1633794215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b680ce790 0x7f1b681555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.818+0000 7f1b67400640 1 -- 172.21.15.86:0/1633794215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b680d1c20 msgr2=0x7f1b68155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:13.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.818+0000 7f1b67400640 1 --2- 172.21.15.86:0/1633794215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b680d1c20 0x7f1b68155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.818+0000 7f1b67400640 1 -- 172.21.15.86:0/1633794215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b680ce790 msgr2=0x7f1b681555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:13.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.818+0000 7f1b67400640 1 --2- 172.21.15.86:0/1633794215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b680ce790 0x7f1b681555e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:13.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.818+0000 7f1b67400640 1 -- 172.21.15.86:0/1633794215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b68155b20 con 0x7f1b68070330 2024-04-25T14:14:13.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.818+0000 7f1b67e00640 1 --2- 172.21.15.86:0/1633794215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b680d1c20 0x7f1b68155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:14:13.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.818+0000 7f1b66a00640 1 --2- 172.21.15.86:0/1633794215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b680ce790 0x7f1b681555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:14:13.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.818+0000 7f1b67400640 1 --2- 172.21.15.86:0/1633794215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b68070330 0x7f1b68098950 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f1b580190c0 tx=0x7f1b5801acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:13.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.818+0000 7f1b5f400640 1 -- 172.21.15.86:0/1633794215 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b580b2040 con 0x7f1b68070330 2024-04-25T14:14:13.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.818+0000 7f1b5f400640 1 -- 172.21.15.86:0/1633794215 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b580be070 con 0x7f1b68070330 2024-04-25T14:14:13.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.818+0000 7f1b5f400640 1 -- 172.21.15.86:0/1633794215 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b580b8660 con 0x7f1b68070330 2024-04-25T14:14:13.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.818+0000 7f1b6e800640 1 -- 172.21.15.86:0/1633794215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b680ae710 con 0x7f1b68070330 2024-04-25T14:14:13.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.818+0000 7f1b6e800640 1 -- 172.21.15.86:0/1633794215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b680aecd0 con 0x7f1b68070330 2024-04-25T14:14:13.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.818+0000 7f1b6e800640 1 -- 172.21.15.86:0/1633794215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b68069d00 con 0x7f1b68070330 2024-04-25T14:14:13.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.822+0000 7f1b5f400640 1 -- 172.21.15.86:0/1633794215 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f1b580b8800 con 0x7f1b68070330 2024-04-25T14:14:13.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.822+0000 7f1b5f400640 1 --2- 172.21.15.86:0/1633794215 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1b4806f5a0 0x7f1b48071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:13.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.822+0000 7f1b5f400640 1 -- 172.21.15.86:0/1633794215 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f1b58129bb0 con 0x7f1b68070330 2024-04-25T14:14:13.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.822+0000 7f1b66a00640 1 --2- 172.21.15.86:0/1633794215 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1b4806f5a0 0x7f1b48071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:13.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.822+0000 7f1b66a00640 1 --2- 172.21.15.86:0/1633794215 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1b4806f5a0 0x7f1b48071ab0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1b6815d640 tx=0x7f1b54009040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:13.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:13.826+0000 7f1b5f400640 1 -- 172.21.15.86:0/1633794215 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1b68155b20 con 0x7f1b68070330 2024-04-25T14:14:14.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:14.018+0000 7f1b6e800640 1 -- 172.21.15.86:0/1633794215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) -- 0x7f1b680ccde0 con 0x7f1b68070330 2024-04-25T14:14:14.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:14.870+0000 7f1b5f400640 1 -- 172.21.15.86:0/1633794215 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v122) ==== 105+0+0 (secure 0 0 0) 0x7f1b58016130 con 0x7f1b68070330 2024-04-25T14:14:14.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:14.874+0000 7f1b6e800640 1 -- 172.21.15.86:0/1633794215 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1b4806f5a0 msgr2=0x7f1b48071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:14.874+0000 7f1b6e800640 1 --2- 172.21.15.86:0/1633794215 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1b4806f5a0 0x7f1b48071ab0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1b6815d640 tx=0x7f1b54009040 comp rx=0 tx=0).stop 2024-04-25T14:14:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:14.874+0000 7f1b6e800640 1 -- 172.21.15.86:0/1633794215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b68070330 msgr2=0x7f1b68098950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:14.874+0000 7f1b6e800640 1 --2- 172.21.15.86:0/1633794215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b68070330 0x7f1b68098950 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f1b580190c0 tx=0x7f1b5801acf0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:14.874+0000 7f1b6e800640 1 -- 172.21.15.86:0/1633794215 shutdown_connections 2024-04-25T14:14:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:14.874+0000 7f1b6e800640 1 --2- 172.21.15.86:0/1633794215 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1b4806f5a0 0x7f1b48071ab0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:14.874+0000 7f1b6e800640 1 --2- 172.21.15.86:0/1633794215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b680d1c20 0x7f1b68155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:14.874+0000 7f1b6e800640 1 --2- 172.21.15.86:0/1633794215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b680ce790 0x7f1b681555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:14.874+0000 7f1b6e800640 1 --2- 172.21.15.86:0/1633794215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b68070330 0x7f1b68098950 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:14.874+0000 7f1b6e800640 1 -- 172.21.15.86:0/1633794215 >> 172.21.15.86:0/1633794215 conn(0x7f1b680b4650 msgr2=0x7f1b6815c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:14.874+0000 7f1b6e800640 1 -- 172.21.15.86:0/1633794215 shutdown_connections 2024-04-25T14:14:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:14.874+0000 7f1b6e800640 1 -- 172.21.15.86:0/1633794215 wait complete. 2024-04-25T14:14:14.895 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:15.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.050+0000 7fab6b000640 1 Processor -- start 2024-04-25T14:14:15.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.050+0000 7fab6b000640 1 -- start start 2024-04-25T14:14:15.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.054+0000 7fab6b000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fab640cfca0 0x7fab6406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:15.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.054+0000 7fab6b000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab640d16e0 0x7fab640d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:15.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.054+0000 7fab6b000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fab6406c670 0x7fab6406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:15.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.054+0000 7fab6b000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fab64070470 con 0x7fab640cfca0 2024-04-25T14:14:15.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.054+0000 7fab6b000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fab640d2030 con 0x7fab6406c670 2024-04-25T14:14:15.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.054+0000 7fab6b000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fab64072bd0 con 0x7fab640d16e0 2024-04-25T14:14:15.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.054+0000 7fab63e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fab640cfca0 0x7fab6406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:15.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.054+0000 7fab63e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fab640cfca0 0x7fab6406f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51204/0 (socket says 172.21.15.86:51204) 2024-04-25T14:14:15.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.054+0000 7fab63e00640 1 -- 172.21.15.86:0/698143516 learned_addr learned my addr 172.21.15.86:0/698143516 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:15.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.054+0000 7fab5b400640 1 --2- 172.21.15.86:0/698143516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab640d16e0 0x7fab640d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:15.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.054+0000 7fab69000640 1 --2- 172.21.15.86:0/698143516 >> v2:172.21.15.155:3300/0 conn(0x7fab6406c670 0x7fab6406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:15.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.054+0000 7fab63e00640 1 -- 172.21.15.86:0/698143516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab640d16e0 msgr2=0x7fab640d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:15.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.054+0000 7fab63e00640 1 --2- 172.21.15.86:0/698143516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab640d16e0 0x7fab640d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:15.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.054+0000 7fab63e00640 1 -- 172.21.15.86:0/698143516 >> v2:172.21.15.155:3300/0 conn(0x7fab6406c670 msgr2=0x7fab6406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:15.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.054+0000 7fab63e00640 1 --2- 172.21.15.86:0/698143516 >> v2:172.21.15.155:3300/0 conn(0x7fab6406c670 0x7fab6406ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:15.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.054+0000 7fab63e00640 1 -- 172.21.15.86:0/698143516 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab6406f930 con 0x7fab640cfca0 2024-04-25T14:14:15.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.054+0000 7fab5b400640 1 --2- 172.21.15.86:0/698143516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab640d16e0 0x7fab640d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:14:15.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.058+0000 7fab63e00640 1 --2- 172.21.15.86:0/698143516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab640cfca0 0x7fab6406f3f0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fab6406e060 tx=0x7fab5401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740a43354b5d8d3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:15.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.058+0000 7fab63400640 1 -- 172.21.15.86:0/698143516 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab540ab5f0 con 0x7fab640cfca0 2024-04-25T14:14:15.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.058+0000 7fab63400640 1 -- 172.21.15.86:0/698143516 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab540b0070 con 0x7fab640cfca0 2024-04-25T14:14:15.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.058+0000 7fab63400640 1 -- 172.21.15.86:0/698143516 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab540ab9b0 con 0x7fab640cfca0 2024-04-25T14:14:15.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.058+0000 7fab6b000640 1 -- 172.21.15.86:0/698143516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab640cfca0 msgr2=0x7fab6406f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:15.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.058+0000 7fab6b000640 1 --2- 172.21.15.86:0/698143516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab640cfca0 0x7fab6406f3f0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fab6406e060 tx=0x7fab5401c520 comp rx=0 tx=0).stop 2024-04-25T14:14:15.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.058+0000 7fab6b000640 1 -- 172.21.15.86:0/698143516 shutdown_connections 2024-04-25T14:14:15.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.058+0000 7fab6b000640 1 --2- 172.21.15.86:0/698143516 >> v2:172.21.15.155:3300/0 conn(0x7fab6406c670 0x7fab6406ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:15.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.058+0000 7fab6b000640 1 --2- 172.21.15.86:0/698143516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab640d16e0 0x7fab640d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:15.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.058+0000 7fab6b000640 1 --2- 172.21.15.86:0/698143516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab640cfca0 0x7fab6406f3f0 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:15.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.058+0000 7fab6b000640 1 -- 172.21.15.86:0/698143516 >> 172.21.15.86:0/698143516 conn(0x7fab640b4650 msgr2=0x7fab640b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:15.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.058+0000 7fab6b000640 1 -- 172.21.15.86:0/698143516 shutdown_connections 2024-04-25T14:14:15.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.058+0000 7fab6b000640 1 -- 172.21.15.86:0/698143516 wait complete. 2024-04-25T14:14:15.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.062+0000 7fab6b000640 1 Processor -- start 2024-04-25T14:14:15.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.062+0000 7fab6b000640 1 -- start start 2024-04-25T14:14:15.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.062+0000 7fab6b000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab6406c670 0x7fab64158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:15.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.062+0000 7fab6b000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab640cfca0 0x7fab6409a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:15.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.062+0000 7fab6b000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab640d16e0 0x7fab640cde60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:15.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.062+0000 7fab6b000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fab640738c0 con 0x7fab640d16e0 2024-04-25T14:14:15.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.062+0000 7fab6b000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fab6406d3d0 con 0x7fab640cfca0 2024-04-25T14:14:15.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.062+0000 7fab6b000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fab64073a40 con 0x7fab6406c670 2024-04-25T14:14:15.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.062+0000 7fab69000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab640d16e0 0x7fab640cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:15.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.062+0000 7fab69000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab640d16e0 0x7fab640cde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51208/0 (socket says 172.21.15.86:51208) 2024-04-25T14:14:15.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.062+0000 7fab69000640 1 -- 172.21.15.86:0/1647289233 learned_addr learned my addr 172.21.15.86:0/1647289233 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:15.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.062+0000 7fab63e00640 1 --2- 172.21.15.86:0/1647289233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab6406c670 0x7fab64158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:15.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.062+0000 7fab5b400640 1 --2- 172.21.15.86:0/1647289233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab640cfca0 0x7fab6409a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:15.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.062+0000 7fab69000640 1 -- 172.21.15.86:0/1647289233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab6406c670 msgr2=0x7fab64158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:15.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.062+0000 7fab69000640 1 --2- 172.21.15.86:0/1647289233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab6406c670 0x7fab64158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:15.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.062+0000 7fab69000640 1 -- 172.21.15.86:0/1647289233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab640cfca0 msgr2=0x7fab6409a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:15.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.062+0000 7fab69000640 1 --2- 172.21.15.86:0/1647289233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab640cfca0 0x7fab6409a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:15.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.062+0000 7fab69000640 1 -- 172.21.15.86:0/1647289233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab6409b350 con 0x7fab640d16e0 2024-04-25T14:14:15.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.062+0000 7fab69000640 1 --2- 172.21.15.86:0/1647289233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab640d16e0 0x7fab640cde60 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7fab5c007e90 tx=0x7fab5c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:15.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.062+0000 7fab60c00640 1 -- 172.21.15.86:0/1647289233 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c014420 con 0x7fab640d16e0 2024-04-25T14:14:15.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.062+0000 7fab60c00640 1 -- 172.21.15.86:0/1647289233 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab5c019070 con 0x7fab640d16e0 2024-04-25T14:14:15.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.062+0000 7fab60c00640 1 -- 172.21.15.86:0/1647289233 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c014800 con 0x7fab640d16e0 2024-04-25T14:14:15.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.062+0000 7fab6b000640 1 -- 172.21.15.86:0/1647289233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab6409b630 con 0x7fab640d16e0 2024-04-25T14:14:15.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.062+0000 7fab6b000640 1 -- 172.21.15.86:0/1647289233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab6409bbf0 con 0x7fab640d16e0 2024-04-25T14:14:15.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.066+0000 7fab6b000640 1 -- 172.21.15.86:0/1647289233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab6409f4b0 con 0x7fab640d16e0 2024-04-25T14:14:15.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.066+0000 7fab60c00640 1 -- 172.21.15.86:0/1647289233 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7fab5c0149a0 con 0x7fab640d16e0 2024-04-25T14:14:15.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.066+0000 7fab60c00640 1 --2- 172.21.15.86:0/1647289233 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fab4006f5a0 0x7fab40071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:15.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.066+0000 7fab60c00640 1 -- 172.21.15.86:0/1647289233 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fab5c08fed0 con 0x7fab640d16e0 2024-04-25T14:14:15.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.066+0000 7fab63e00640 1 --2- 172.21.15.86:0/1647289233 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fab4006f5a0 0x7fab40071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:15.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.066+0000 7fab63e00640 1 --2- 172.21.15.86:0/1647289233 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fab4006f5a0 0x7fab40071ab0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fab540af000 tx=0x7fab540a7000 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:15.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.070+0000 7fab60c00640 1 -- 172.21.15.86:0/1647289233 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fab5c05c410 con 0x7fab640d16e0 2024-04-25T14:14:15.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.262+0000 7fab6b000640 1 -- 172.21.15.86:0/1647289233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab64069d00 con 0x7fab640d16e0 2024-04-25T14:14:15.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.262+0000 7fab60c00640 1 -- 172.21.15.86:0/1647289233 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) ==== 95+0+5219 (secure 0 0 0) 0x7fab5c060710 con 0x7fab640d16e0 2024-04-25T14:14:15.257 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:15.258 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":123,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:14.876442+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:14:15.258 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 123 2024-04-25T14:14:15.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.262+0000 7fab6b000640 1 -- 172.21.15.86:0/1647289233 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fab4006f5a0 msgr2=0x7fab40071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:15.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.262+0000 7fab6b000640 1 --2- 172.21.15.86:0/1647289233 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fab4006f5a0 0x7fab40071ab0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fab540af000 tx=0x7fab540a7000 comp rx=0 tx=0).stop 2024-04-25T14:14:15.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.262+0000 7fab6b000640 1 -- 172.21.15.86:0/1647289233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab640d16e0 msgr2=0x7fab640cde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:15.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.262+0000 7fab6b000640 1 --2- 172.21.15.86:0/1647289233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab640d16e0 0x7fab640cde60 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7fab5c007e90 tx=0x7fab5c00e4b0 comp rx=0 tx=0).stop 2024-04-25T14:14:15.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.262+0000 7fab6b000640 1 -- 172.21.15.86:0/1647289233 shutdown_connections 2024-04-25T14:14:15.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.262+0000 7fab6b000640 1 --2- 172.21.15.86:0/1647289233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab640d16e0 0x7fab640cde60 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:15.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.262+0000 7fab6b000640 1 --2- 172.21.15.86:0/1647289233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab640cfca0 0x7fab6409a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:15.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.262+0000 7fab6b000640 1 --2- 172.21.15.86:0/1647289233 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fab4006f5a0 0x7fab40071ab0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:15.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.262+0000 7fab6b000640 1 --2- 172.21.15.86:0/1647289233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab6406c670 0x7fab64158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:15.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.262+0000 7fab6b000640 1 -- 172.21.15.86:0/1647289233 >> 172.21.15.86:0/1647289233 conn(0x7fab640b4650 msgr2=0x7fab6415a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:15.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.262+0000 7fab6b000640 1 -- 172.21.15.86:0/1647289233 shutdown_connections 2024-04-25T14:14:15.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:15.262+0000 7fab6b000640 1 -- 172.21.15.86:0/1647289233 wait complete. 2024-04-25T14:14:15.683 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 123, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:14.876442+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:14:15.684 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:14:16.685 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:16.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.850+0000 7fb87b400640 1 Processor -- start 2024-04-25T14:14:16.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.850+0000 7fb87b400640 1 -- start start 2024-04-25T14:14:16.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.850+0000 7fb87b400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb874071530 0x7fb87415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:16.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.850+0000 7fb87b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb8741616e0 0x7fb874161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:16.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.850+0000 7fb87b400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8740cd8d0 con 0x7fb8740d2c80 2024-04-25T14:14:16.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.850+0000 7fb87b400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb8740ce720 con 0x7fb8741616e0 2024-04-25T14:14:16.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.850+0000 7fb87b400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb8740ce970 con 0x7fb874071530 2024-04-25T14:14:16.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.850+0000 7fb878a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb8740d2c80 0x7fb874070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50788/0 (socket says 172.21.15.86:50788) 2024-04-25T14:14:16.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.850+0000 7fb878a00640 1 -- 172.21.15.86:0/3419675468 learned_addr learned my addr 172.21.15.86:0/3419675468 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:14:16.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.850+0000 7fb879400640 1 --2- 172.21.15.86:0/3419675468 >> v2:172.21.15.86:3300/0 conn(0x7fb8741616e0 0x7fb874161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:16.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.850+0000 7fb879400640 1 -- 172.21.15.86:0/3419675468 >> v2:172.21.15.155:3300/0 conn(0x7fb874071530 msgr2=0x7fb87415dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:16.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.850+0000 7fb86be00640 1 --2- 172.21.15.86:0/3419675468 >> v2:172.21.15.155:3300/0 conn(0x7fb874071530 0x7fb87415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:16.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.850+0000 7fb879400640 1 --2- 172.21.15.86:0/3419675468 >> v2:172.21.15.155:3300/0 conn(0x7fb874071530 0x7fb87415dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:16.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.850+0000 7fb879400640 1 -- 172.21.15.86:0/3419675468 >> v1:172.21.15.86:6789/0 conn(0x7fb8740d2c80 legacy=0x7fb874070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:16.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.850+0000 7fb879400640 1 -- 172.21.15.86:0/3419675468 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8740d1be0 con 0x7fb8741616e0 2024-04-25T14:14:16.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.850+0000 7fb873e00640 1 -- 172.21.15.86:0/3419675468 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2091970750 0 0) 0x7fb8740cd8d0 con 0x7fb8740d2c80 2024-04-25T14:14:16.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.850+0000 7fb86be00640 1 --2- 172.21.15.86:0/3419675468 >> v2:172.21.15.155:3300/0 conn(0x7fb874071530 0x7fb87415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:14:16.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.854+0000 7fb879400640 1 --2- 172.21.15.86:0/3419675468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8741616e0 0x7fb874161af0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7fb8740b6070 tx=0x7fb86c01f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f6d466f20b7a93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:16.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.854+0000 7fb873e00640 1 -- 172.21.15.86:0/3419675468 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c0b4070 con 0x7fb8741616e0 2024-04-25T14:14:16.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.854+0000 7fb873e00640 1 -- 172.21.15.86:0/3419675468 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb86c0af600 con 0x7fb8741616e0 2024-04-25T14:14:16.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.854+0000 7fb873e00640 1 -- 172.21.15.86:0/3419675468 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c0af9d0 con 0x7fb8741616e0 2024-04-25T14:14:16.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.854+0000 7fb87b400640 1 -- 172.21.15.86:0/3419675468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8741616e0 msgr2=0x7fb874161af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:16.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.854+0000 7fb87b400640 1 --2- 172.21.15.86:0/3419675468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8741616e0 0x7fb874161af0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7fb8740b6070 tx=0x7fb86c01f110 comp rx=0 tx=0).stop 2024-04-25T14:14:16.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.854+0000 7fb87b400640 1 -- 172.21.15.86:0/3419675468 shutdown_connections 2024-04-25T14:14:16.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.854+0000 7fb87b400640 1 --2- 172.21.15.86:0/3419675468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8741616e0 0x7fb874161af0 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:16.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.854+0000 7fb87b400640 1 --2- 172.21.15.86:0/3419675468 >> v2:172.21.15.155:3300/0 conn(0x7fb874071530 0x7fb87415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:16.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.854+0000 7fb87b400640 1 -- 172.21.15.86:0/3419675468 >> 172.21.15.86:0/3419675468 conn(0x7fb8740cffc0 msgr2=0x7fb8740d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:16.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.854+0000 7fb87b400640 1 -- 172.21.15.86:0/3419675468 shutdown_connections 2024-04-25T14:14:16.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.854+0000 7fb87b400640 1 -- 172.21.15.86:0/3419675468 wait complete. 2024-04-25T14:14:16.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.858+0000 7fb87b400640 1 Processor -- start 2024-04-25T14:14:16.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.858+0000 7fb87b400640 1 -- start start 2024-04-25T14:14:16.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.858+0000 7fb87b400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8740d2c80 0x7fb8740ac010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:16.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.858+0000 7fb87b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8741616e0 0x7fb87415ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:16.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.858+0000 7fb87b400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8741f7af0 0x7fb8741f7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:16.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.858+0000 7fb87b400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8740d2480 con 0x7fb8741616e0 2024-04-25T14:14:16.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.858+0000 7fb87b400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb8740d21a0 con 0x7fb8740d2c80 2024-04-25T14:14:16.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.858+0000 7fb87b400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb8740d29a0 con 0x7fb8741f7af0 2024-04-25T14:14:16.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.858+0000 7fb86be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8741616e0 0x7fb87415ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:16.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.858+0000 7fb86be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8741616e0 0x7fb87415ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51222/0 (socket says 172.21.15.86:51222) 2024-04-25T14:14:16.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.858+0000 7fb86be00640 1 -- 172.21.15.86:0/3245607550 learned_addr learned my addr 172.21.15.86:0/3245607550 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:16.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.858+0000 7fb878a00640 1 --2- 172.21.15.86:0/3245607550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8740d2c80 0x7fb8740ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:16.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.858+0000 7fb879400640 1 --2- 172.21.15.86:0/3245607550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8741f7af0 0x7fb8741f7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:16.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.862+0000 7fb86be00640 1 -- 172.21.15.86:0/3245607550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8741f7af0 msgr2=0x7fb8741f7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:16.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.862+0000 7fb86be00640 1 --2- 172.21.15.86:0/3245607550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8741f7af0 0x7fb8741f7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:16.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.862+0000 7fb86be00640 1 -- 172.21.15.86:0/3245607550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8740d2c80 msgr2=0x7fb8740ac010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:16.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.862+0000 7fb86be00640 1 --2- 172.21.15.86:0/3245607550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8740d2c80 0x7fb8740ac010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:16.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.862+0000 7fb86be00640 1 -- 172.21.15.86:0/3245607550 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8741f8950 con 0x7fb8741616e0 2024-04-25T14:14:16.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.862+0000 7fb86be00640 1 --2- 172.21.15.86:0/3245607550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8741616e0 0x7fb87415ffd0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00c7f0 tx=0x7fb85c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:16.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.862+0000 7fb871600640 1 -- 172.21.15.86:0/3245607550 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c005ba0 con 0x7fb8741616e0 2024-04-25T14:14:16.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.862+0000 7fb871600640 1 -- 172.21.15.86:0/3245607550 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb85c005d40 con 0x7fb8741616e0 2024-04-25T14:14:16.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.862+0000 7fb871600640 1 -- 172.21.15.86:0/3245607550 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c006250 con 0x7fb8741616e0 2024-04-25T14:14:16.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.862+0000 7fb87b400640 1 -- 172.21.15.86:0/3245607550 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8741f8440 con 0x7fb8741616e0 2024-04-25T14:14:16.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.862+0000 7fb87b400640 1 -- 172.21.15.86:0/3245607550 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb87415ebd0 con 0x7fb8741616e0 2024-04-25T14:14:16.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.862+0000 7fb87b400640 1 -- 172.21.15.86:0/3245607550 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8741572c0 con 0x7fb8741616e0 2024-04-25T14:14:16.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.862+0000 7fb871600640 1 -- 172.21.15.86:0/3245607550 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7fb85c006600 con 0x7fb8741616e0 2024-04-25T14:14:16.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.862+0000 7fb871600640 1 --2- 172.21.15.86:0/3245607550 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fb85006f5a0 0x7fb850071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:16.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.866+0000 7fb871600640 1 -- 172.21.15.86:0/3245607550 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fb85c08f9c0 con 0x7fb8741616e0 2024-04-25T14:14:16.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.866+0000 7fb878a00640 1 --2- 172.21.15.86:0/3245607550 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fb85006f5a0 0x7fb850071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:16.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.866+0000 7fb878a00640 1 --2- 172.21.15.86:0/3245607550 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fb85006f5a0 0x7fb850071ab0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb864000f70 tx=0x7fb864007df0 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:16.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:16.870+0000 7fb871600640 1 -- 172.21.15.86:0/3245607550 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb8741572c0 con 0x7fb8741616e0 2024-04-25T14:14:17.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:17.066+0000 7fb87b400640 1 -- 172.21.15.86:0/3245607550 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb87415e670 con 0x7fb8741616e0 2024-04-25T14:14:17.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:17.066+0000 7fb871600640 1 -- 172.21.15.86:0/3245607550 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) ==== 95+0+5219 (secure 0 0 0) 0x7fb87415e670 con 0x7fb8741616e0 2024-04-25T14:14:17.062 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:17.062 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":123,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:14.876442+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:14:17.062 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 123 2024-04-25T14:14:17.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:17.066+0000 7fb87b400640 1 -- 172.21.15.86:0/3245607550 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fb85006f5a0 msgr2=0x7fb850071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:17.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:17.066+0000 7fb87b400640 1 --2- 172.21.15.86:0/3245607550 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fb85006f5a0 0x7fb850071ab0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb864000f70 tx=0x7fb864007df0 comp rx=0 tx=0).stop 2024-04-25T14:14:17.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:17.066+0000 7fb87b400640 1 -- 172.21.15.86:0/3245607550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8741616e0 msgr2=0x7fb87415ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:17.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:17.066+0000 7fb87b400640 1 --2- 172.21.15.86:0/3245607550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8741616e0 0x7fb87415ffd0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00c7f0 tx=0x7fb85c00ccb0 comp rx=0 tx=0).stop 2024-04-25T14:14:17.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:17.066+0000 7fb87b400640 1 -- 172.21.15.86:0/3245607550 shutdown_connections 2024-04-25T14:14:17.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:17.066+0000 7fb87b400640 1 --2- 172.21.15.86:0/3245607550 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fb85006f5a0 0x7fb850071ab0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:17.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:17.066+0000 7fb87b400640 1 --2- 172.21.15.86:0/3245607550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8741f7af0 0x7fb8741f7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:17.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:17.066+0000 7fb87b400640 1 --2- 172.21.15.86:0/3245607550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8741616e0 0x7fb87415ffd0 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:17.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:17.066+0000 7fb87b400640 1 --2- 172.21.15.86:0/3245607550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8740d2c80 0x7fb8740ac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:17.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:17.066+0000 7fb87b400640 1 -- 172.21.15.86:0/3245607550 >> 172.21.15.86:0/3245607550 conn(0x7fb8740cffc0 msgr2=0x7fb8740b4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:17.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:17.066+0000 7fb87b400640 1 -- 172.21.15.86:0/3245607550 shutdown_connections 2024-04-25T14:14:17.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:17.070+0000 7fb87b400640 1 -- 172.21.15.86:0/3245607550 wait complete. 2024-04-25T14:14:17.486 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 123, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:14.876442+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:14:17.486 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:14:18.487 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:18.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.650+0000 7faa86c00640 1 Processor -- start 2024-04-25T14:14:18.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.650+0000 7faa86c00640 1 -- start start 2024-04-25T14:14:18.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.650+0000 7faa86c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7faa800cfca0 0x7faa8006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:18.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.650+0000 7faa86c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa800730b0 0x7faa800734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:18.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.650+0000 7faa86c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa80072bd0 con 0x7faa800d16e0 2024-04-25T14:14:18.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.650+0000 7faa86c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7faa80070470 con 0x7faa800cfca0 2024-04-25T14:14:18.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.650+0000 7faa86c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7faa8006f930 con 0x7faa800730b0 2024-04-25T14:14:18.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.650+0000 7faa77400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7faa800d16e0 0x7faa800d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50794/0 (socket says 172.21.15.86:50794) 2024-04-25T14:14:18.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.650+0000 7faa77400640 1 -- 172.21.15.86:0/1045629212 learned_addr learned my addr 172.21.15.86:0/1045629212 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:14:18.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.650+0000 7faa7fe00640 1 --2- 172.21.15.86:0/1045629212 >> v2:172.21.15.155:3300/0 conn(0x7faa800cfca0 0x7faa8006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:18.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.650+0000 7faa84c00640 1 --2- 172.21.15.86:0/1045629212 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa800730b0 0x7faa800734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:18.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.650+0000 7faa7f400640 1 -- 172.21.15.86:0/1045629212 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3489283247 0 0) 0x7faa80072bd0 con 0x7faa800d16e0 2024-04-25T14:14:18.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.650+0000 7faa7f400640 1 -- 172.21.15.86:0/1045629212 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa64003530 con 0x7faa800d16e0 2024-04-25T14:14:18.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.650+0000 7faa7f400640 1 -- 172.21.15.86:0/1045629212 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3903947381 0 0) 0x7faa64003530 con 0x7faa800d16e0 2024-04-25T14:14:18.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.650+0000 7faa7f400640 1 -- 172.21.15.86:0/1045629212 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa800730b0 msgr2=0x7faa800734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:18.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.650+0000 7faa7f400640 1 --2- 172.21.15.86:0/1045629212 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa800730b0 0x7faa800734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.650+0000 7faa7f400640 1 -- 172.21.15.86:0/1045629212 >> v2:172.21.15.155:3300/0 conn(0x7faa800cfca0 msgr2=0x7faa8006f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:18.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.650+0000 7faa7f400640 1 --2- 172.21.15.86:0/1045629212 >> v2:172.21.15.155:3300/0 conn(0x7faa800cfca0 0x7faa8006f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.650+0000 7faa7f400640 1 -- 172.21.15.86:0/1045629212 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa8006fbd0 con 0x7faa800d16e0 2024-04-25T14:14:18.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.650+0000 7faa7f400640 1 -- 172.21.15.86:0/1045629212 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7faa680016a0 con 0x7faa800d16e0 2024-04-25T14:14:18.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.654+0000 7faa7f400640 1 -- 172.21.15.86:0/1045629212 >> v1:172.21.15.86:6789/0 conn(0x7faa800d16e0 legacy=0x7faa800d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:18.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.654+0000 7faa7f400640 1 --2- 172.21.15.86:0/1045629212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa64003a30 0x7faa64003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:18.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.654+0000 7faa7f400640 1 --2- 172.21.15.86:0/1045629212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa64004810 0x7faa64004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:18.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.654+0000 7faa7f400640 1 --2- 172.21.15.86:0/1045629212 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa800730b0 0x7faa64005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:18.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.654+0000 7faa7f400640 1 -- 172.21.15.86:0/1045629212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa80070470 con 0x7faa64003a30 2024-04-25T14:14:18.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.654+0000 7faa7f400640 1 -- 172.21.15.86:0/1045629212 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faa8006f930 con 0x7faa64004810 2024-04-25T14:14:18.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.654+0000 7faa7f400640 1 -- 172.21.15.86:0/1045629212 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7faa6400dc90 con 0x7faa800730b0 2024-04-25T14:14:18.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.654+0000 7faa7fe00640 1 --2- 172.21.15.86:0/1045629212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa64003a30 0x7faa64003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:18.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.654+0000 7faa77400640 1 --2- 172.21.15.86:0/1045629212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa64004810 0x7faa64004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:18.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.654+0000 7faa84c00640 1 --2- 172.21.15.86:0/1045629212 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa800730b0 0x7faa64005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:18.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.654+0000 7faa7fe00640 1 -- 172.21.15.86:0/1045629212 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa800730b0 msgr2=0x7faa64005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:18.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.654+0000 7faa7fe00640 1 --2- 172.21.15.86:0/1045629212 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa800730b0 0x7faa64005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.654+0000 7faa7fe00640 1 -- 172.21.15.86:0/1045629212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa64004810 msgr2=0x7faa64004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:18.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.654+0000 7faa7fe00640 1 --2- 172.21.15.86:0/1045629212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa64004810 0x7faa64004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.654+0000 7faa7fe00640 1 -- 172.21.15.86:0/1045629212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa64003530 con 0x7faa64003a30 2024-04-25T14:14:18.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.654+0000 7faa7fe00640 1 --2- 172.21.15.86:0/1045629212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa64003a30 0x7faa64003ea0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7faa70009ab0 tx=0x7faa7001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7402af1768616289 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:18.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.654+0000 7faa7f400640 1 -- 172.21.15.86:0/1045629212 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa700b2070 con 0x7faa64003a30 2024-04-25T14:14:18.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.654+0000 7faa7f400640 1 -- 172.21.15.86:0/1045629212 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa700a5570 con 0x7faa64003a30 2024-04-25T14:14:18.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.654+0000 7faa7f400640 1 -- 172.21.15.86:0/1045629212 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa700096b0 con 0x7faa64003a30 2024-04-25T14:14:18.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.658+0000 7faa86c00640 1 -- 172.21.15.86:0/1045629212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa64003a30 msgr2=0x7faa64003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:18.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.658+0000 7faa86c00640 1 --2- 172.21.15.86:0/1045629212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa64003a30 0x7faa64003ea0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7faa70009ab0 tx=0x7faa7001ca90 comp rx=0 tx=0).stop 2024-04-25T14:14:18.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.658+0000 7faa84c00640 1 -- 172.21.15.86:0/1045629212 reap_dead start 2024-04-25T14:14:18.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.658+0000 7faa86c00640 1 -- 172.21.15.86:0/1045629212 shutdown_connections 2024-04-25T14:14:18.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.658+0000 7faa86c00640 1 -- 172.21.15.86:0/1045629212 >> 172.21.15.86:0/1045629212 conn(0x7faa800b4650 msgr2=0x7faa800b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:18.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.658+0000 7faa86c00640 1 -- 172.21.15.86:0/1045629212 shutdown_connections 2024-04-25T14:14:18.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.658+0000 7faa86c00640 1 -- 172.21.15.86:0/1045629212 wait complete. 2024-04-25T14:14:18.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.658+0000 7faa86c00640 1 Processor -- start 2024-04-25T14:14:18.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.658+0000 7faa86c00640 1 -- start start 2024-04-25T14:14:18.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.662+0000 7faa86c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa8009a6f0 0x7faa801586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:18.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.662+0000 7faa86c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa80091620 0x7faa801f9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:18.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.662+0000 7faa86c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa80091a30 0x7faa800ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:18.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.662+0000 7faa86c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa80074060 con 0x7faa8009a6f0 2024-04-25T14:14:18.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.662+0000 7faa86c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faa80073ee0 con 0x7faa80091a30 2024-04-25T14:14:18.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.662+0000 7faa86c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7faa800741e0 con 0x7faa80091620 2024-04-25T14:14:18.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.662+0000 7faa7fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa8009a6f0 0x7faa801586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:18.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.662+0000 7faa7fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa8009a6f0 0x7faa801586d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51232/0 (socket says 172.21.15.86:51232) 2024-04-25T14:14:18.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.662+0000 7faa7fe00640 1 -- 172.21.15.86:0/206516568 learned_addr learned my addr 172.21.15.86:0/206516568 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:18.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.662+0000 7faa84c00640 1 --2- 172.21.15.86:0/206516568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa80091a30 0x7faa800ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:18.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.662+0000 7faa77400640 1 --2- 172.21.15.86:0/206516568 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa80091620 0x7faa801f9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:18.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.662+0000 7faa7fe00640 1 -- 172.21.15.86:0/206516568 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa80091620 msgr2=0x7faa801f9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:18.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.662+0000 7faa7fe00640 1 --2- 172.21.15.86:0/206516568 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa80091620 0x7faa801f9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.662+0000 7faa7fe00640 1 -- 172.21.15.86:0/206516568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa80091a30 msgr2=0x7faa800ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:18.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.662+0000 7faa7fe00640 1 --2- 172.21.15.86:0/206516568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa80091a30 0x7faa800ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.662+0000 7faa7fe00640 1 -- 172.21.15.86:0/206516568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa80155a90 con 0x7faa8009a6f0 2024-04-25T14:14:18.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.662+0000 7faa77400640 1 --2- 172.21.15.86:0/206516568 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa80091620 0x7faa801f9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:18.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.662+0000 7faa7fe00640 1 --2- 172.21.15.86:0/206516568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa8009a6f0 0x7faa801586d0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7faa7001d330 tx=0x7faa70019830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:18.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.662+0000 7faa7cc00640 1 -- 172.21.15.86:0/206516568 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa700bd070 con 0x7faa8009a6f0 2024-04-25T14:14:18.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.662+0000 7faa7cc00640 1 -- 172.21.15.86:0/206516568 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa700a5d80 con 0x7faa8009a6f0 2024-04-25T14:14:18.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.662+0000 7faa7cc00640 1 -- 172.21.15.86:0/206516568 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa700b2040 con 0x7faa8009a6f0 2024-04-25T14:14:18.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.662+0000 7faa86c00640 1 -- 172.21.15.86:0/206516568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa80155d70 con 0x7faa8009a6f0 2024-04-25T14:14:18.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.662+0000 7faa86c00640 1 -- 172.21.15.86:0/206516568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa800cdcd0 con 0x7faa8009a6f0 2024-04-25T14:14:18.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.662+0000 7faa86c00640 1 -- 172.21.15.86:0/206516568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa8009f4b0 con 0x7faa8009a6f0 2024-04-25T14:14:18.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.666+0000 7faa7cc00640 1 -- 172.21.15.86:0/206516568 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7faa700ad660 con 0x7faa8009a6f0 2024-04-25T14:14:18.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.666+0000 7faa7cc00640 1 --2- 172.21.15.86:0/206516568 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7faa5c06f570 0x7faa5c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:18.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.666+0000 7faa7cc00640 1 -- 172.21.15.86:0/206516568 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7faa70128ff0 con 0x7faa8009a6f0 2024-04-25T14:14:18.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.666+0000 7faa77400640 1 --2- 172.21.15.86:0/206516568 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7faa5c06f570 0x7faa5c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:18.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.666+0000 7faa77400640 1 --2- 172.21.15.86:0/206516568 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7faa5c06f570 0x7faa5c071a80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7faa68000c10 tx=0x7faa68009040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:18.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.670+0000 7faa7cc00640 1 -- 172.21.15.86:0/206516568 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faa700167a0 con 0x7faa8009a6f0 2024-04-25T14:14:18.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.858+0000 7faa86c00640 1 -- 172.21.15.86:0/206516568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa80069d00 con 0x7faa8009a6f0 2024-04-25T14:14:18.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.862+0000 7faa7cc00640 1 -- 172.21.15.86:0/206516568 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) ==== 95+0+5219 (secure 0 0 0) 0x7faa700f5580 con 0x7faa8009a6f0 2024-04-25T14:14:18.856 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:18.856 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":123,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:14.876442+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:14:18.856 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 123 2024-04-25T14:14:18.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.862+0000 7faa86c00640 1 -- 172.21.15.86:0/206516568 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7faa5c06f570 msgr2=0x7faa5c071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:18.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.862+0000 7faa86c00640 1 --2- 172.21.15.86:0/206516568 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7faa5c06f570 0x7faa5c071a80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7faa68000c10 tx=0x7faa68009040 comp rx=0 tx=0).stop 2024-04-25T14:14:18.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.862+0000 7faa86c00640 1 -- 172.21.15.86:0/206516568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa8009a6f0 msgr2=0x7faa801586d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:18.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.862+0000 7faa86c00640 1 --2- 172.21.15.86:0/206516568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa8009a6f0 0x7faa801586d0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7faa7001d330 tx=0x7faa70019830 comp rx=0 tx=0).stop 2024-04-25T14:14:18.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.862+0000 7faa86c00640 1 -- 172.21.15.86:0/206516568 shutdown_connections 2024-04-25T14:14:18.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.862+0000 7faa86c00640 1 --2- 172.21.15.86:0/206516568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa80091a30 0x7faa800ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.862+0000 7faa86c00640 1 --2- 172.21.15.86:0/206516568 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7faa5c06f570 0x7faa5c071a80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.862+0000 7faa86c00640 1 --2- 172.21.15.86:0/206516568 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa80091620 0x7faa801f9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.862+0000 7faa86c00640 1 --2- 172.21.15.86:0/206516568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa8009a6f0 0x7faa801586d0 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:18.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.862+0000 7faa86c00640 1 -- 172.21.15.86:0/206516568 >> 172.21.15.86:0/206516568 conn(0x7faa800b4650 msgr2=0x7faa801551f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:18.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.862+0000 7faa86c00640 1 -- 172.21.15.86:0/206516568 shutdown_connections 2024-04-25T14:14:18.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:18.862+0000 7faa86c00640 1 -- 172.21.15.86:0/206516568 wait complete. 2024-04-25T14:14:19.281 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 123, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:14.876442+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:14:19.281 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:14:20.263 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:14:20.267 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:14:20.281 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:20.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.470+0000 7f2d42600640 1 Processor -- start 2024-04-25T14:14:20.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.470+0000 7f2d42600640 1 -- start start 2024-04-25T14:14:20.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d42600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d3c0cde90 0x7f2d3c0ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:20.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d42600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d3c15e5c0 con 0x7f2d3c0d1df0 2024-04-25T14:14:20.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d42600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d3c15e8a0 con 0x7f2d3c070330 2024-04-25T14:14:20.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d42600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2d3c06fd40 con 0x7f2d3c0cde90 2024-04-25T14:14:20.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d32a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2d3c0d1df0 0x7f2d3c0cd780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35430/0 (socket says 172.21.15.86:35430) 2024-04-25T14:14:20.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d32a00640 1 -- 172.21.15.86:0/3355283557 learned_addr learned my addr 172.21.15.86:0/3355283557 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:14:20.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d3be00640 1 --2- 172.21.15.86:0/3355283557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d3c0cde90 0x7f2d3c0ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:20.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d3aa00640 1 -- 172.21.15.86:0/3355283557 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2524829751 0 0) 0x7f2d3c15e5c0 con 0x7f2d3c0d1df0 2024-04-25T14:14:20.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d3aa00640 1 -- 172.21.15.86:0/3355283557 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d20003530 con 0x7f2d3c0d1df0 2024-04-25T14:14:20.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d3aa00640 1 -- 172.21.15.86:0/3355283557 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3585356229 0 0) 0x7f2d20003530 con 0x7f2d3c0d1df0 2024-04-25T14:14:20.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d3aa00640 1 -- 172.21.15.86:0/3355283557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d3c0cde90 msgr2=0x7f2d3c0ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:20.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d3aa00640 1 --2- 172.21.15.86:0/3355283557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d3c0cde90 0x7f2d3c0ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:20.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d3aa00640 1 -- 172.21.15.86:0/3355283557 >> v1:172.21.15.155:6789/0 conn(0x7f2d3c070330 legacy=0x7f2d3c0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:20.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d3aa00640 1 -- 172.21.15.86:0/3355283557 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d3c15eb80 con 0x7f2d3c0d1df0 2024-04-25T14:14:20.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d3aa00640 1 -- 172.21.15.86:0/3355283557 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f2d24001680 con 0x7f2d3c0d1df0 2024-04-25T14:14:20.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d3aa00640 1 -- 172.21.15.86:0/3355283557 >> v1:172.21.15.86:6789/0 conn(0x7f2d3c0d1df0 legacy=0x7f2d3c0cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:20.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d3aa00640 1 --2- 172.21.15.86:0/3355283557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d3c0cde90 0x7f2d20003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:20.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d3aa00640 1 --2- 172.21.15.86:0/3355283557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d20003fc0 0x7f2d20004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:20.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d3aa00640 1 --2- 172.21.15.86:0/3355283557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d20004d50 0x7f2d20005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:20.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d3aa00640 1 -- 172.21.15.86:0/3355283557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2000bbe0 con 0x7f2d20003fc0 2024-04-25T14:14:20.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d3aa00640 1 -- 172.21.15.86:0/3355283557 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2000bd90 con 0x7f2d20004d50 2024-04-25T14:14:20.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d3aa00640 1 -- 172.21.15.86:0/3355283557 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2d2000bf90 con 0x7f2d3c0cde90 2024-04-25T14:14:20.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d32a00640 1 --2- 172.21.15.86:0/3355283557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d20003fc0 0x7f2d20004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:20.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d3b400640 1 --2- 172.21.15.86:0/3355283557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d3c0cde90 0x7f2d20003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:20.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d3be00640 1 --2- 172.21.15.86:0/3355283557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d20004d50 0x7f2d20005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:20.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d32a00640 1 -- 172.21.15.86:0/3355283557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d3c0cde90 msgr2=0x7f2d20003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:20.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d32a00640 1 --2- 172.21.15.86:0/3355283557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d3c0cde90 0x7f2d20003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:20.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d32a00640 1 -- 172.21.15.86:0/3355283557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d20004d50 msgr2=0x7f2d20005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:20.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d32a00640 1 --2- 172.21.15.86:0/3355283557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d20004d50 0x7f2d20005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:20.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d32a00640 1 -- 172.21.15.86:0/3355283557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d20003530 con 0x7f2d20003fc0 2024-04-25T14:14:20.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d3b400640 1 --2- 172.21.15.86:0/3355283557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d3c0cde90 0x7f2d20003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:20.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.474+0000 7f2d3be00640 1 --2- 172.21.15.86:0/3355283557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d20004d50 0x7f2d20005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:20.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.478+0000 7f2d32a00640 1 --2- 172.21.15.86:0/3355283557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d20003fc0 0x7f2d20004810 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c074ed0 tx=0x7f2d2401cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd03f7946ada019f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:20.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.478+0000 7f2d3aa00640 1 -- 172.21.15.86:0/3355283557 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d24007d40 con 0x7f2d20003fc0 2024-04-25T14:14:20.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.478+0000 7f2d3aa00640 1 -- 172.21.15.86:0/3355283557 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d240a9070 con 0x7f2d20003fc0 2024-04-25T14:14:20.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.478+0000 7f2d3aa00640 1 -- 172.21.15.86:0/3355283557 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d24005d80 con 0x7f2d20003fc0 2024-04-25T14:14:20.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.478+0000 7f2d42600640 1 -- 172.21.15.86:0/3355283557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d20003fc0 msgr2=0x7f2d20004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:20.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.478+0000 7f2d42600640 1 --2- 172.21.15.86:0/3355283557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d20003fc0 0x7f2d20004810 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c074ed0 tx=0x7f2d2401cbf0 comp rx=0 tx=0).stop 2024-04-25T14:14:20.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.478+0000 7f2d3be00640 1 -- 172.21.15.86:0/3355283557 reap_dead start 2024-04-25T14:14:20.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.478+0000 7f2d42600640 1 -- 172.21.15.86:0/3355283557 shutdown_connections 2024-04-25T14:14:20.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.478+0000 7f2d42600640 1 -- 172.21.15.86:0/3355283557 >> 172.21.15.86:0/3355283557 conn(0x7f2d3c0b4650 msgr2=0x7f2d3c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:20.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.478+0000 7f2d42600640 1 -- 172.21.15.86:0/3355283557 shutdown_connections 2024-04-25T14:14:20.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.478+0000 7f2d42600640 1 -- 172.21.15.86:0/3355283557 wait complete. 2024-04-25T14:14:20.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.482+0000 7f2d42600640 1 Processor -- start 2024-04-25T14:14:20.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.482+0000 7f2d42600640 1 -- start start 2024-04-25T14:14:20.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.482+0000 7f2d42600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d20003fc0 0x7f2d3c09b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:20.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.482+0000 7f2d42600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d3c0d0310 0x7f2d3c0b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:20.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.482+0000 7f2d42600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d3c0d0720 0x7f2d3c156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:20.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.482+0000 7f2d42600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c0708c0 con 0x7f2d20003fc0 2024-04-25T14:14:20.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.482+0000 7f2d42600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c070740 con 0x7f2d3c0d0310 2024-04-25T14:14:20.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.482+0000 7f2d42600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2d3c1697b0 con 0x7f2d3c0d0720 2024-04-25T14:14:20.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.482+0000 7f2d3b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d20003fc0 0x7f2d3c09b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:20.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.482+0000 7f2d3b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d20003fc0 0x7f2d3c09b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54686/0 (socket says 172.21.15.86:54686) 2024-04-25T14:14:20.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.482+0000 7f2d3b400640 1 -- 172.21.15.86:0/3503558356 learned_addr learned my addr 172.21.15.86:0/3503558356 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:20.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.482+0000 7f2d32a00640 1 --2- 172.21.15.86:0/3503558356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d3c0d0310 0x7f2d3c0b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:20.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.482+0000 7f2d3be00640 1 --2- 172.21.15.86:0/3503558356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d3c0d0720 0x7f2d3c156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:20.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.482+0000 7f2d3b400640 1 -- 172.21.15.86:0/3503558356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d3c0d0720 msgr2=0x7f2d3c156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:20.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.482+0000 7f2d3b400640 1 --2- 172.21.15.86:0/3503558356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d3c0d0720 0x7f2d3c156ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:20.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.482+0000 7f2d3b400640 1 -- 172.21.15.86:0/3503558356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d3c0d0310 msgr2=0x7f2d3c0b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:20.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.482+0000 7f2d3b400640 1 --2- 172.21.15.86:0/3503558356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d3c0d0310 0x7f2d3c0b0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:20.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.482+0000 7f2d3b400640 1 -- 172.21.15.86:0/3503558356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d3c0cf860 con 0x7f2d20003fc0 2024-04-25T14:14:20.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.482+0000 7f2d3be00640 1 --2- 172.21.15.86:0/3503558356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d3c0d0720 0x7f2d3c156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:14:20.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.482+0000 7f2d3b400640 1 --2- 172.21.15.86:0/3503558356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d20003fc0 0x7f2d3c09b2b0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c00e7d0 tx=0x7f2d2c00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:20.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.482+0000 7f2d33e00640 1 -- 172.21.15.86:0/3503558356 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c0050d0 con 0x7f2d20003fc0 2024-04-25T14:14:20.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.482+0000 7f2d33e00640 1 -- 172.21.15.86:0/3503558356 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d2c018070 con 0x7f2d20003fc0 2024-04-25T14:14:20.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.482+0000 7f2d33e00640 1 -- 172.21.15.86:0/3503558356 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c00c6c0 con 0x7f2d20003fc0 2024-04-25T14:14:20.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.482+0000 7f2d42600640 1 -- 172.21.15.86:0/3503558356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d3c1591b0 con 0x7f2d20003fc0 2024-04-25T14:14:20.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.482+0000 7f2d42600640 1 -- 172.21.15.86:0/3503558356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d3c0cfb10 con 0x7f2d20003fc0 2024-04-25T14:14:20.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.486+0000 7f2d42600640 1 -- 172.21.15.86:0/3503558356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d3c069d00 con 0x7f2d20003fc0 2024-04-25T14:14:20.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.486+0000 7f2d33e00640 1 -- 172.21.15.86:0/3503558356 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f2d2c00c860 con 0x7f2d20003fc0 2024-04-25T14:14:20.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.486+0000 7f2d33e00640 1 --2- 172.21.15.86:0/3503558356 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f2d1806f5a0 0x7f2d18071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:20.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.486+0000 7f2d33e00640 1 -- 172.21.15.86:0/3503558356 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f2d2c08f5b0 con 0x7f2d20003fc0 2024-04-25T14:14:20.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.486+0000 7f2d32a00640 1 --2- 172.21.15.86:0/3503558356 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f2d1806f5a0 0x7f2d18071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:20.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.486+0000 7f2d32a00640 1 --2- 172.21.15.86:0/3503558356 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f2d1806f5a0 0x7f2d18071ab0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2d240ab000 tx=0x7f2d240b1040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:20.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.490+0000 7f2d33e00640 1 -- 172.21.15.86:0/3503558356 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2d2c05c0b0 con 0x7f2d20003fc0 2024-04-25T14:14:20.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.682+0000 7f2d42600640 1 -- 172.21.15.86:0/3503558356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d3c076bf0 con 0x7f2d20003fc0 2024-04-25T14:14:20.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.682+0000 7f2d33e00640 1 -- 172.21.15.86:0/3503558356 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) ==== 95+0+5219 (secure 0 0 0) 0x7f2d2c05b9b0 con 0x7f2d20003fc0 2024-04-25T14:14:20.677 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:20.677 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":123,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:14.876442+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:14:20.677 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 123 2024-04-25T14:14:20.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.682+0000 7f2d42600640 1 -- 172.21.15.86:0/3503558356 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f2d1806f5a0 msgr2=0x7f2d18071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:20.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.682+0000 7f2d42600640 1 --2- 172.21.15.86:0/3503558356 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f2d1806f5a0 0x7f2d18071ab0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2d240ab000 tx=0x7f2d240b1040 comp rx=0 tx=0).stop 2024-04-25T14:14:20.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.682+0000 7f2d42600640 1 -- 172.21.15.86:0/3503558356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d20003fc0 msgr2=0x7f2d3c09b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:20.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.682+0000 7f2d42600640 1 --2- 172.21.15.86:0/3503558356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d20003fc0 0x7f2d3c09b2b0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c00e7d0 tx=0x7f2d2c00ec90 comp rx=0 tx=0).stop 2024-04-25T14:14:20.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.682+0000 7f2d42600640 1 -- 172.21.15.86:0/3503558356 shutdown_connections 2024-04-25T14:14:20.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.682+0000 7f2d42600640 1 --2- 172.21.15.86:0/3503558356 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f2d1806f5a0 0x7f2d18071ab0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:20.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.682+0000 7f2d42600640 1 --2- 172.21.15.86:0/3503558356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d3c0d0720 0x7f2d3c156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:20.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.682+0000 7f2d42600640 1 --2- 172.21.15.86:0/3503558356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d3c0d0310 0x7f2d3c0b0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:20.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.682+0000 7f2d42600640 1 --2- 172.21.15.86:0/3503558356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d20003fc0 0x7f2d3c09b2b0 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:20.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.682+0000 7f2d42600640 1 -- 172.21.15.86:0/3503558356 >> 172.21.15.86:0/3503558356 conn(0x7f2d3c0b4650 msgr2=0x7f2d3c0ae1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:20.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.682+0000 7f2d42600640 1 -- 172.21.15.86:0/3503558356 shutdown_connections 2024-04-25T14:14:20.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:20.682+0000 7f2d42600640 1 -- 172.21.15.86:0/3503558356 wait complete. 2024-04-25T14:14:21.102 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 123, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:14.876442+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:14:21.102 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:14:22.103 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:22.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.266+0000 7f7049400640 1 Processor -- start 2024-04-25T14:14:22.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.266+0000 7f7049400640 1 -- start start 2024-04-25T14:14:22.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.266+0000 7f7049400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70440722a0 0x7f70440726b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:22.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.266+0000 7f7049400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7044072bf0 con 0x7f70441614c0 2024-04-25T14:14:22.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.266+0000 7f7049400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f704415df00 con 0x7f70440b6ee0 2024-04-25T14:14:22.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.266+0000 7f7049400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f704415e1e0 con 0x7f70440722a0 2024-04-25T14:14:22.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.270+0000 7f7048a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f70441614c0 0x7f70441618d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35444/0 (socket says 172.21.15.86:35444) 2024-04-25T14:14:22.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.270+0000 7f7048a00640 1 -- 172.21.15.86:0/1025054598 learned_addr learned my addr 172.21.15.86:0/1025054598 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:14:22.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.270+0000 7f7043400640 1 --2- 172.21.15.86:0/1025054598 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70440722a0 0x7f70440726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:22.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.270+0000 7f7042a00640 1 -- 172.21.15.86:0/1025054598 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 930888702 0 0) 0x7f703c001120 con 0x7f70441614c0 2024-04-25T14:14:22.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.270+0000 7f7042a00640 1 -- 172.21.15.86:0/1025054598 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7028003530 con 0x7f70441614c0 2024-04-25T14:14:22.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.270+0000 7f7042a00640 1 -- 172.21.15.86:0/1025054598 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2851611869 0 0) 0x7f704415df00 con 0x7f70440b6ee0 2024-04-25T14:14:22.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.270+0000 7f7042a00640 1 -- 172.21.15.86:0/1025054598 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f703c001120 con 0x7f70440b6ee0 2024-04-25T14:14:22.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.270+0000 7f7043400640 1 -- 172.21.15.86:0/1025054598 >> v1:172.21.15.155:6789/0 conn(0x7f70440b6ee0 legacy=0x7f70440d06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:22.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.270+0000 7f7043400640 1 -- 172.21.15.86:0/1025054598 >> v1:172.21.15.86:6789/0 conn(0x7f70441614c0 legacy=0x7f70441618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:22.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.270+0000 7f7043400640 1 -- 172.21.15.86:0/1025054598 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704415e4c0 con 0x7f70440722a0 2024-04-25T14:14:22.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.274+0000 7f7043400640 1 --2- 172.21.15.86:0/1025054598 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70440722a0 0x7f70440726b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f70300097f0 tx=0x7f703001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc244d88f62f595f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:22.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.274+0000 7f7042a00640 1 -- 172.21.15.86:0/1025054598 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70300b0070 con 0x7f70440722a0 2024-04-25T14:14:22.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.274+0000 7f7042a00640 1 -- 172.21.15.86:0/1025054598 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70300ab600 con 0x7f70440722a0 2024-04-25T14:14:22.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.274+0000 7f7042a00640 1 -- 172.21.15.86:0/1025054598 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70300ab9b0 con 0x7f70440722a0 2024-04-25T14:14:22.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.274+0000 7f7049400640 1 -- 172.21.15.86:0/1025054598 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70440722a0 msgr2=0x7f70440726b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:22.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.274+0000 7f7049400640 1 --2- 172.21.15.86:0/1025054598 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70440722a0 0x7f70440726b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f70300097f0 tx=0x7f703001c450 comp rx=0 tx=0).stop 2024-04-25T14:14:22.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.274+0000 7f7049400640 1 -- 172.21.15.86:0/1025054598 shutdown_connections 2024-04-25T14:14:22.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.274+0000 7f7049400640 1 --2- 172.21.15.86:0/1025054598 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70440722a0 0x7f70440726b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:22.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.274+0000 7f7049400640 1 -- 172.21.15.86:0/1025054598 >> 172.21.15.86:0/1025054598 conn(0x7f70440b4f80 msgr2=0x7f70440b5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:22.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.274+0000 7f7049400640 1 -- 172.21.15.86:0/1025054598 shutdown_connections 2024-04-25T14:14:22.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.274+0000 7f7049400640 1 -- 172.21.15.86:0/1025054598 wait complete. 2024-04-25T14:14:22.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.274+0000 7f7049400640 1 Processor -- start 2024-04-25T14:14:22.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.274+0000 7f7049400640 1 -- start start 2024-04-25T14:14:22.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.278+0000 7f7049400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70440722a0 0x7f7044154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:22.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.278+0000 7f7049400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70440b6ee0 0x7f70440b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:22.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.278+0000 7f7049400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70441614c0 0x7f70440b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:22.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.278+0000 7f7049400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f70440cfec0 con 0x7f70440b6ee0 2024-04-25T14:14:22.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.278+0000 7f7049400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f70440cfd40 con 0x7f70441614c0 2024-04-25T14:14:22.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.278+0000 7f7049400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7044161fe0 con 0x7f70440722a0 2024-04-25T14:14:22.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.278+0000 7f7043400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70440b6ee0 0x7f70440b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:22.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.278+0000 7f7043400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70440b6ee0 0x7f70440b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54690/0 (socket says 172.21.15.86:54690) 2024-04-25T14:14:22.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.278+0000 7f7043400640 1 -- 172.21.15.86:0/2648356380 learned_addr learned my addr 172.21.15.86:0/2648356380 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:22.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.278+0000 7f7043e00640 1 --2- 172.21.15.86:0/2648356380 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70440722a0 0x7f7044154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:22.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.278+0000 7f7048a00640 1 --2- 172.21.15.86:0/2648356380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70441614c0 0x7f70440b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:22.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.278+0000 7f7043400640 1 -- 172.21.15.86:0/2648356380 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70440722a0 msgr2=0x7f7044154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:22.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.278+0000 7f7043400640 1 --2- 172.21.15.86:0/2648356380 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70440722a0 0x7f7044154d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:22.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.278+0000 7f7043400640 1 -- 172.21.15.86:0/2648356380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70441614c0 msgr2=0x7f70440b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:22.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.278+0000 7f7043400640 1 --2- 172.21.15.86:0/2648356380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70441614c0 0x7f70440b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:22.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.278+0000 7f7043400640 1 -- 172.21.15.86:0/2648356380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7044168c60 con 0x7f70440b6ee0 2024-04-25T14:14:22.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.278+0000 7f7048a00640 1 --2- 172.21.15.86:0/2648356380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70441614c0 0x7f70440b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:22.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.278+0000 7f7043400640 1 --2- 172.21.15.86:0/2648356380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70440b6ee0 0x7f70440b3de0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f70300097f0 tx=0x7f70300191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:22.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.278+0000 7f703be00640 1 -- 172.21.15.86:0/2648356380 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70300b9020 con 0x7f70440b6ee0 2024-04-25T14:14:22.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.278+0000 7f703be00640 1 -- 172.21.15.86:0/2648356380 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70300ab7b0 con 0x7f70440b6ee0 2024-04-25T14:14:22.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.278+0000 7f703be00640 1 -- 172.21.15.86:0/2648356380 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70300b0040 con 0x7f70440b6ee0 2024-04-25T14:14:22.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.278+0000 7f7049400640 1 -- 172.21.15.86:0/2648356380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7044168f40 con 0x7f70440b6ee0 2024-04-25T14:14:22.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.278+0000 7f7049400640 1 -- 172.21.15.86:0/2648356380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70441403d0 con 0x7f70440b6ee0 2024-04-25T14:14:22.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.278+0000 7f7049400640 1 -- 172.21.15.86:0/2648356380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7044140f90 con 0x7f70440b6ee0 2024-04-25T14:14:22.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.282+0000 7f703be00640 1 -- 172.21.15.86:0/2648356380 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f70300bc020 con 0x7f70440b6ee0 2024-04-25T14:14:22.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.282+0000 7f703be00640 1 --2- 172.21.15.86:0/2648356380 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f702406f570 0x7f7024071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:22.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.282+0000 7f703be00640 1 -- 172.21.15.86:0/2648356380 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f70301294b0 con 0x7f70440b6ee0 2024-04-25T14:14:22.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.282+0000 7f7043e00640 1 --2- 172.21.15.86:0/2648356380 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f702406f570 0x7f7024071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:22.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.282+0000 7f7043e00640 1 --2- 172.21.15.86:0/2648356380 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f702406f570 0x7f7024071a80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7034007fd0 tx=0x7f7034009040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:22.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.286+0000 7f703be00640 1 -- 172.21.15.86:0/2648356380 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f70300c3030 con 0x7f70440b6ee0 2024-04-25T14:14:22.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.474+0000 7f7049400640 1 -- 172.21.15.86:0/2648356380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7044076b80 con 0x7f70440b6ee0 2024-04-25T14:14:22.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.474+0000 7f703be00640 1 -- 172.21.15.86:0/2648356380 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) ==== 95+0+5219 (secure 0 0 0) 0x7f70300160d0 con 0x7f70440b6ee0 2024-04-25T14:14:22.470 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:22.470 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":123,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:14.876442+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:14:22.470 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 123 2024-04-25T14:14:22.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.478+0000 7f7049400640 1 -- 172.21.15.86:0/2648356380 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f702406f570 msgr2=0x7f7024071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:22.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.478+0000 7f7049400640 1 --2- 172.21.15.86:0/2648356380 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f702406f570 0x7f7024071a80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7034007fd0 tx=0x7f7034009040 comp rx=0 tx=0).stop 2024-04-25T14:14:22.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.478+0000 7f7049400640 1 -- 172.21.15.86:0/2648356380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70440b6ee0 msgr2=0x7f70440b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:22.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.478+0000 7f7049400640 1 --2- 172.21.15.86:0/2648356380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70440b6ee0 0x7f70440b3de0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f70300097f0 tx=0x7f70300191f0 comp rx=0 tx=0).stop 2024-04-25T14:14:22.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.478+0000 7f7049400640 1 -- 172.21.15.86:0/2648356380 shutdown_connections 2024-04-25T14:14:22.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.478+0000 7f7049400640 1 --2- 172.21.15.86:0/2648356380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70441614c0 0x7f70440b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:22.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.478+0000 7f7049400640 1 --2- 172.21.15.86:0/2648356380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70440b6ee0 0x7f70440b3de0 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:22.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.478+0000 7f7049400640 1 --2- 172.21.15.86:0/2648356380 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f702406f570 0x7f7024071a80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:22.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.478+0000 7f7049400640 1 --2- 172.21.15.86:0/2648356380 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70440722a0 0x7f7044154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:22.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.478+0000 7f7049400640 1 -- 172.21.15.86:0/2648356380 >> 172.21.15.86:0/2648356380 conn(0x7f70440b4f80 msgr2=0x7f70440b5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:22.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.478+0000 7f7049400640 1 -- 172.21.15.86:0/2648356380 shutdown_connections 2024-04-25T14:14:22.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:22.478+0000 7f7049400640 1 -- 172.21.15.86:0/2648356380 wait complete. 2024-04-25T14:14:22.895 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 123, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:14.876442+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:14:22.895 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:14:23.896 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.058+0000 7fd9a4c00640 1 Processor -- start 2024-04-25T14:14:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.058+0000 7fd9a4c00640 1 -- start start 2024-04-25T14:14:24.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.062+0000 7fd9a4c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd9a015c2a0 0x7fd9a00ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:24.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.062+0000 7fd9a4c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9a00b0a60 con 0x7fd9a015feb0 2024-04-25T14:14:24.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.062+0000 7fd9a4c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9a006ef90 con 0x7fd9a006fd50 2024-04-25T14:14:24.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.062+0000 7fd9a4c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd9a006f270 con 0x7fd9a015c2a0 2024-04-25T14:14:24.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.062+0000 7fd99e800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd9a015feb0 0x7fd9a01602c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35460/0 (socket says 172.21.15.86:35460) 2024-04-25T14:14:24.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.062+0000 7fd99e800640 1 -- 172.21.15.86:0/1254263734 learned_addr learned my addr 172.21.15.86:0/1254263734 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:14:24.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.062+0000 7fd99de00640 1 --2- 172.21.15.86:0/1254263734 >> v2:172.21.15.155:3300/0 conn(0x7fd9a015c2a0 0x7fd9a00ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:24.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.062+0000 7fd99d400640 1 -- 172.21.15.86:0/1254263734 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3391353885 0 0) 0x7fd9a00b0a60 con 0x7fd9a015feb0 2024-04-25T14:14:24.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.062+0000 7fd99d400640 1 -- 172.21.15.86:0/1254263734 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd984004640 con 0x7fd9a015feb0 2024-04-25T14:14:24.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.062+0000 7fd99d400640 1 -- 172.21.15.86:0/1254263734 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3996496723 0 0) 0x7fd9a006ef90 con 0x7fd9a006fd50 2024-04-25T14:14:24.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.062+0000 7fd99d400640 1 -- 172.21.15.86:0/1254263734 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9a00b0a60 con 0x7fd9a006fd50 2024-04-25T14:14:24.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.062+0000 7fd99de00640 1 -- 172.21.15.86:0/1254263734 >> v1:172.21.15.155:6789/0 conn(0x7fd9a006fd50 legacy=0x7fd9a0070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:24.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.062+0000 7fd99de00640 1 -- 172.21.15.86:0/1254263734 >> v1:172.21.15.86:6789/0 conn(0x7fd9a015feb0 legacy=0x7fd9a01602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:24.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.062+0000 7fd99de00640 1 -- 172.21.15.86:0/1254263734 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a006f550 con 0x7fd9a015c2a0 2024-04-25T14:14:24.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.066+0000 7fd99de00640 1 --2- 172.21.15.86:0/1254263734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9a015c2a0 0x7fd9a00ae130 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fd9900097c0 tx=0x7fd99001c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a908664228daa0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:24.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.066+0000 7fd99d400640 1 -- 172.21.15.86:0/1254263734 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9900b0070 con 0x7fd9a015c2a0 2024-04-25T14:14:24.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.066+0000 7fd99d400640 1 -- 172.21.15.86:0/1254263734 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9900ab600 con 0x7fd9a015c2a0 2024-04-25T14:14:24.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.066+0000 7fd99d400640 1 -- 172.21.15.86:0/1254263734 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9900ab9d0 con 0x7fd9a015c2a0 2024-04-25T14:14:24.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.066+0000 7fd9a4c00640 1 -- 172.21.15.86:0/1254263734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9a015c2a0 msgr2=0x7fd9a00ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:24.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.066+0000 7fd9a4c00640 1 --2- 172.21.15.86:0/1254263734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9a015c2a0 0x7fd9a00ae130 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fd9900097c0 tx=0x7fd99001c3e0 comp rx=0 tx=0).stop 2024-04-25T14:14:24.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.066+0000 7fd9a4c00640 1 -- 172.21.15.86:0/1254263734 shutdown_connections 2024-04-25T14:14:24.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.066+0000 7fd9a4c00640 1 --2- 172.21.15.86:0/1254263734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9a015c2a0 0x7fd9a00ae130 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:24.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.066+0000 7fd9a4c00640 1 -- 172.21.15.86:0/1254263734 >> 172.21.15.86:0/1254263734 conn(0x7fd9a00b4650 msgr2=0x7fd9a00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:24.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.066+0000 7fd9a4c00640 1 -- 172.21.15.86:0/1254263734 shutdown_connections 2024-04-25T14:14:24.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.066+0000 7fd9a4c00640 1 -- 172.21.15.86:0/1254263734 wait complete. 2024-04-25T14:14:24.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.070+0000 7fd9a4c00640 1 Processor -- start 2024-04-25T14:14:24.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.070+0000 7fd9a4c00640 1 -- start start 2024-04-25T14:14:24.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.070+0000 7fd9a4c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9a006fd50 0x7fd9a00c0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:24.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.070+0000 7fd9a4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a015c2a0 0x7fd9a00b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:24.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.070+0000 7fd9a4c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9a015feb0 0x7fd9a015ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:24.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.070+0000 7fd9a4c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a00ae7f0 con 0x7fd9a015c2a0 2024-04-25T14:14:24.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.070+0000 7fd9a4c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a00ae670 con 0x7fd9a006fd50 2024-04-25T14:14:24.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.070+0000 7fd9a4c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a016a200 con 0x7fd9a015feb0 2024-04-25T14:14:24.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.070+0000 7fd995400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a015c2a0 0x7fd9a00b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:24.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.070+0000 7fd995400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a015c2a0 0x7fd9a00b0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54706/0 (socket says 172.21.15.86:54706) 2024-04-25T14:14:24.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.070+0000 7fd995400640 1 -- 172.21.15.86:0/2819610639 learned_addr learned my addr 172.21.15.86:0/2819610639 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:24.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.070+0000 7fd99de00640 1 --2- 172.21.15.86:0/2819610639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9a006fd50 0x7fd9a00c0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:24.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.070+0000 7fd99e800640 1 --2- 172.21.15.86:0/2819610639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9a015feb0 0x7fd9a015ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:24.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.070+0000 7fd995400640 1 -- 172.21.15.86:0/2819610639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9a015feb0 msgr2=0x7fd9a015ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:24.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.070+0000 7fd995400640 1 --2- 172.21.15.86:0/2819610639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9a015feb0 0x7fd9a015ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:24.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.070+0000 7fd995400640 1 -- 172.21.15.86:0/2819610639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9a006fd50 msgr2=0x7fd9a00c0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:24.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.070+0000 7fd995400640 1 --2- 172.21.15.86:0/2819610639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9a006fd50 0x7fd9a00c0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:24.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.070+0000 7fd995400640 1 -- 172.21.15.86:0/2819610639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a0099dc0 con 0x7fd9a015c2a0 2024-04-25T14:14:24.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.070+0000 7fd995400640 1 --2- 172.21.15.86:0/2819610639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a015c2a0 0x7fd9a00b0d50 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7fd98800c3c0 tx=0x7fd98800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:24.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.070+0000 7fd996a00640 1 -- 172.21.15.86:0/2819610639 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd988017070 con 0x7fd9a015c2a0 2024-04-25T14:14:24.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.070+0000 7fd996a00640 1 -- 172.21.15.86:0/2819610639 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9880052d0 con 0x7fd9a015c2a0 2024-04-25T14:14:24.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.070+0000 7fd9a4c00640 1 -- 172.21.15.86:0/2819610639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a00b1580 con 0x7fd9a015c2a0 2024-04-25T14:14:24.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.070+0000 7fd996a00640 1 -- 172.21.15.86:0/2819610639 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd988003840 con 0x7fd9a015c2a0 2024-04-25T14:14:24.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.074+0000 7fd9a4c00640 1 -- 172.21.15.86:0/2819610639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a00b1b40 con 0x7fd9a015c2a0 2024-04-25T14:14:24.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.074+0000 7fd9a4c00640 1 -- 172.21.15.86:0/2819610639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9a0069d00 con 0x7fd9a015c2a0 2024-04-25T14:14:24.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.074+0000 7fd996a00640 1 -- 172.21.15.86:0/2819610639 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7fd98801a070 con 0x7fd9a015c2a0 2024-04-25T14:14:24.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.074+0000 7fd996a00640 1 --2- 172.21.15.86:0/2819610639 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd97c06f5a0 0x7fd97c071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:24.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.074+0000 7fd996a00640 1 -- 172.21.15.86:0/2819610639 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fd98808ea30 con 0x7fd9a015c2a0 2024-04-25T14:14:24.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.074+0000 7fd99de00640 1 --2- 172.21.15.86:0/2819610639 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd97c06f5a0 0x7fd97c071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:24.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.078+0000 7fd99de00640 1 --2- 172.21.15.86:0/2819610639 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd97c06f5a0 0x7fd97c071ab0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd9900ad000 tx=0x7fd9900a6000 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:24.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.078+0000 7fd996a00640 1 -- 172.21.15.86:0/2819610639 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd988005480 con 0x7fd9a015c2a0 2024-04-25T14:14:24.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.266+0000 7fd9a4c00640 1 -- 172.21.15.86:0/2819610639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9a00b1310 con 0x7fd9a015c2a0 2024-04-25T14:14:24.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.270+0000 7fd996a00640 1 -- 172.21.15.86:0/2819610639 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) ==== 95+0+5219 (secure 0 0 0) 0x7fd98805aee0 con 0x7fd9a015c2a0 2024-04-25T14:14:24.264 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:24.264 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":123,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:14.876442+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:14:24.264 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 123 2024-04-25T14:14:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.270+0000 7fd9a4c00640 1 -- 172.21.15.86:0/2819610639 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd97c06f5a0 msgr2=0x7fd97c071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.270+0000 7fd9a4c00640 1 --2- 172.21.15.86:0/2819610639 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd97c06f5a0 0x7fd97c071ab0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd9900ad000 tx=0x7fd9900a6000 comp rx=0 tx=0).stop 2024-04-25T14:14:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.270+0000 7fd9a4c00640 1 -- 172.21.15.86:0/2819610639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a015c2a0 msgr2=0x7fd9a00b0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.270+0000 7fd9a4c00640 1 --2- 172.21.15.86:0/2819610639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a015c2a0 0x7fd9a00b0d50 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7fd98800c3c0 tx=0x7fd98800c880 comp rx=0 tx=0).stop 2024-04-25T14:14:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.270+0000 7fd9a4c00640 1 -- 172.21.15.86:0/2819610639 shutdown_connections 2024-04-25T14:14:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.270+0000 7fd9a4c00640 1 --2- 172.21.15.86:0/2819610639 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd97c06f5a0 0x7fd97c071ab0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.270+0000 7fd9a4c00640 1 --2- 172.21.15.86:0/2819610639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9a015feb0 0x7fd9a015ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.270+0000 7fd9a4c00640 1 --2- 172.21.15.86:0/2819610639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a015c2a0 0x7fd9a00b0d50 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.270+0000 7fd9a4c00640 1 --2- 172.21.15.86:0/2819610639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9a006fd50 0x7fd9a00c0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:24.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.270+0000 7fd9a4c00640 1 -- 172.21.15.86:0/2819610639 >> 172.21.15.86:0/2819610639 conn(0x7fd9a00b4650 msgr2=0x7fd9a0159930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:24.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.270+0000 7fd9a4c00640 1 -- 172.21.15.86:0/2819610639 shutdown_connections 2024-04-25T14:14:24.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:24.270+0000 7fd9a4c00640 1 -- 172.21.15.86:0/2819610639 wait complete. 2024-04-25T14:14:24.688 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 123, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:14.876442+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:14:24.688 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:14:25.689 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:25.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.854+0000 7fc3eee00640 1 Processor -- start 2024-04-25T14:14:25.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.854+0000 7fc3eee00640 1 -- start start 2024-04-25T14:14:25.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.854+0000 7fc3eee00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e80b69a0 0x7fc3e80b6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:25.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.854+0000 7fc3eee00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc3e806ef30 0x7fc3e806f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:25.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.854+0000 7fc3eee00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3e80b7460 con 0x7fc3e8161510 2024-04-25T14:14:25.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.854+0000 7fc3eee00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc3e80b7740 con 0x7fc3e806ef30 2024-04-25T14:14:25.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.854+0000 7fc3eee00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc3e806e730 con 0x7fc3e80b69a0 2024-04-25T14:14:25.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.854+0000 7fc3e7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc3e806ef30 0x7fc3e806f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:25.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.854+0000 7fc3ece00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc3e8161510 0x7fc3e8072410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35472/0 (socket says 172.21.15.86:35472) 2024-04-25T14:14:25.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.854+0000 7fc3e7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc3e806ef30 0x7fc3e806f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54712/0 (socket says 172.21.15.86:54712) 2024-04-25T14:14:25.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.854+0000 7fc3ece00640 1 -- 172.21.15.86:0/614611774 learned_addr learned my addr 172.21.15.86:0/614611774 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:14:25.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.854+0000 7fc3e7e00640 1 --2- 172.21.15.86:0/614611774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e80b69a0 0x7fc3e80b6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:25.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.854+0000 7fc3e7400640 1 -- 172.21.15.86:0/614611774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e80b69a0 msgr2=0x7fc3e80b6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:25.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.854+0000 7fc3e7400640 1 --2- 172.21.15.86:0/614611774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e80b69a0 0x7fc3e80b6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:25.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.858+0000 7fc3e7400640 1 -- 172.21.15.86:0/614611774 >> v1:172.21.15.86:6789/0 conn(0x7fc3e8161510 legacy=0x7fc3e8072410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:25.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.858+0000 7fc3e7400640 1 -- 172.21.15.86:0/614611774 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e806ea10 con 0x7fc3e806ef30 2024-04-25T14:14:25.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.858+0000 7fc3e7400640 1 --2- 172.21.15.86:0/614611774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3e806ef30 0x7fc3e806f340 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7fc3d40097c0 tx=0x7fc3d401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbecc5573b7043df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:25.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.858+0000 7fc3e6a00640 1 -- 172.21.15.86:0/614611774 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d40b0070 con 0x7fc3e806ef30 2024-04-25T14:14:25.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.858+0000 7fc3e6a00640 1 -- 172.21.15.86:0/614611774 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d40ab600 con 0x7fc3e806ef30 2024-04-25T14:14:25.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.858+0000 7fc3e6a00640 1 -- 172.21.15.86:0/614611774 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d40ab9d0 con 0x7fc3e806ef30 2024-04-25T14:14:25.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.858+0000 7fc3eee00640 1 -- 172.21.15.86:0/614611774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3e806ef30 msgr2=0x7fc3e806f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:25.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.858+0000 7fc3eee00640 1 --2- 172.21.15.86:0/614611774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3e806ef30 0x7fc3e806f340 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7fc3d40097c0 tx=0x7fc3d401c4a0 comp rx=0 tx=0).stop 2024-04-25T14:14:25.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.858+0000 7fc3eee00640 1 -- 172.21.15.86:0/614611774 shutdown_connections 2024-04-25T14:14:25.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.858+0000 7fc3eee00640 1 --2- 172.21.15.86:0/614611774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3e806ef30 0x7fc3e806f340 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:25.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.858+0000 7fc3eee00640 1 --2- 172.21.15.86:0/614611774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e80b69a0 0x7fc3e80b6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:25.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.858+0000 7fc3eee00640 1 -- 172.21.15.86:0/614611774 >> 172.21.15.86:0/614611774 conn(0x7fc3e80b5100 msgr2=0x7fc3e80b5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:25.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.858+0000 7fc3eee00640 1 -- 172.21.15.86:0/614611774 shutdown_connections 2024-04-25T14:14:25.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.862+0000 7fc3eee00640 1 -- 172.21.15.86:0/614611774 wait complete. 2024-04-25T14:14:25.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.862+0000 7fc3eee00640 1 Processor -- start 2024-04-25T14:14:25.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.862+0000 7fc3eee00640 1 -- start start 2024-04-25T14:14:25.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.862+0000 7fc3eee00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e806ef30 0x7fc3e8141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:25.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.862+0000 7fc3eee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3e80b69a0 0x7fc3e8144d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:25.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.862+0000 7fc3eee00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3e8161510 0x7fc3e815afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:25.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.862+0000 7fc3eee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e8072ca0 con 0x7fc3e80b69a0 2024-04-25T14:14:25.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.862+0000 7fc3eee00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e8072b20 con 0x7fc3e8161510 2024-04-25T14:14:25.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.862+0000 7fc3eee00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc3e8161920 con 0x7fc3e806ef30 2024-04-25T14:14:25.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.862+0000 7fc3e7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3e80b69a0 0x7fc3e8144d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:25.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.862+0000 7fc3e7e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e806ef30 0x7fc3e8141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:25.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.862+0000 7fc3e7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3e80b69a0 0x7fc3e8144d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54726/0 (socket says 172.21.15.86:54726) 2024-04-25T14:14:25.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.862+0000 7fc3e7400640 1 -- 172.21.15.86:0/2066046181 learned_addr learned my addr 172.21.15.86:0/2066046181 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:25.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.866+0000 7fc3ece00640 1 --2- 172.21.15.86:0/2066046181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3e8161510 0x7fc3e815afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:25.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.866+0000 7fc3e7400640 1 -- 172.21.15.86:0/2066046181 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e806ef30 msgr2=0x7fc3e8141ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:25.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.866+0000 7fc3e7400640 1 --2- 172.21.15.86:0/2066046181 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e806ef30 0x7fc3e8141ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:25.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.866+0000 7fc3e7400640 1 -- 172.21.15.86:0/2066046181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3e8161510 msgr2=0x7fc3e815afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:25.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.866+0000 7fc3e7400640 1 --2- 172.21.15.86:0/2066046181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3e8161510 0x7fc3e815afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:25.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.866+0000 7fc3e7400640 1 -- 172.21.15.86:0/2066046181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e8141ff0 con 0x7fc3e80b69a0 2024-04-25T14:14:25.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.866+0000 7fc3ece00640 1 --2- 172.21.15.86:0/2066046181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3e8161510 0x7fc3e815afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:25.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.866+0000 7fc3e7400640 1 --2- 172.21.15.86:0/2066046181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3e80b69a0 0x7fc3e8144d50 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fc3d40ae040 tx=0x7fc3d4019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:25.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.866+0000 7fc3dfe00640 1 -- 172.21.15.86:0/2066046181 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d40b0040 con 0x7fc3e80b69a0 2024-04-25T14:14:25.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.866+0000 7fc3dfe00640 1 -- 172.21.15.86:0/2066046181 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d40ab400 con 0x7fc3e80b69a0 2024-04-25T14:14:25.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.866+0000 7fc3eee00640 1 -- 172.21.15.86:0/2066046181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e8145290 con 0x7fc3e80b69a0 2024-04-25T14:14:25.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.866+0000 7fc3dfe00640 1 -- 172.21.15.86:0/2066046181 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d40b4650 con 0x7fc3e80b69a0 2024-04-25T14:14:25.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.866+0000 7fc3eee00640 1 -- 172.21.15.86:0/2066046181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e8156c90 con 0x7fc3e80b69a0 2024-04-25T14:14:25.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.866+0000 7fc3eee00640 1 -- 172.21.15.86:0/2066046181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e8072200 con 0x7fc3e80b69a0 2024-04-25T14:14:25.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.866+0000 7fc3dfe00640 1 -- 172.21.15.86:0/2066046181 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7fc3d40bc020 con 0x7fc3e80b69a0 2024-04-25T14:14:25.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.866+0000 7fc3dfe00640 1 --2- 172.21.15.86:0/2066046181 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc3c806f570 0x7fc3c8071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:25.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.870+0000 7fc3dfe00640 1 -- 172.21.15.86:0/2066046181 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fc3d4128950 con 0x7fc3e80b69a0 2024-04-25T14:14:25.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.870+0000 7fc3e7e00640 1 --2- 172.21.15.86:0/2066046181 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc3c806f570 0x7fc3c8071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:25.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.870+0000 7fc3e7e00640 1 --2- 172.21.15.86:0/2066046181 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc3c806f570 0x7fc3c8071a80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc3e80b8910 tx=0x7fc3d8009830 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:25.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:25.874+0000 7fc3dfe00640 1 -- 172.21.15.86:0/2066046181 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc3d4016090 con 0x7fc3e80b69a0 2024-04-25T14:14:26.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:26.066+0000 7fc3eee00640 1 -- 172.21.15.86:0/2066046181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3e816aab0 con 0x7fc3e80b69a0 2024-04-25T14:14:26.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:26.066+0000 7fc3dfe00640 1 -- 172.21.15.86:0/2066046181 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7fc3d40f5340 con 0x7fc3e80b69a0 2024-04-25T14:14:26.061 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:26.061 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:14:26.061 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:14:26.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:26.066+0000 7fc3eee00640 1 -- 172.21.15.86:0/2066046181 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc3c806f570 msgr2=0x7fc3c8071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:26.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:26.066+0000 7fc3eee00640 1 --2- 172.21.15.86:0/2066046181 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc3c806f570 0x7fc3c8071a80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc3e80b8910 tx=0x7fc3d8009830 comp rx=0 tx=0).stop 2024-04-25T14:14:26.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:26.066+0000 7fc3eee00640 1 -- 172.21.15.86:0/2066046181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3e80b69a0 msgr2=0x7fc3e8144d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:26.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:26.066+0000 7fc3eee00640 1 --2- 172.21.15.86:0/2066046181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3e80b69a0 0x7fc3e8144d50 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fc3d40ae040 tx=0x7fc3d4019240 comp rx=0 tx=0).stop 2024-04-25T14:14:26.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:26.066+0000 7fc3eee00640 1 -- 172.21.15.86:0/2066046181 shutdown_connections 2024-04-25T14:14:26.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:26.066+0000 7fc3eee00640 1 --2- 172.21.15.86:0/2066046181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3e8161510 0x7fc3e815afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:26.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:26.066+0000 7fc3eee00640 1 --2- 172.21.15.86:0/2066046181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3e80b69a0 0x7fc3e8144d50 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:26.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:26.066+0000 7fc3eee00640 1 --2- 172.21.15.86:0/2066046181 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc3c806f570 0x7fc3c8071a80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:26.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:26.066+0000 7fc3eee00640 1 --2- 172.21.15.86:0/2066046181 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e806ef30 0x7fc3e8141ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:26.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:26.066+0000 7fc3eee00640 1 -- 172.21.15.86:0/2066046181 >> 172.21.15.86:0/2066046181 conn(0x7fc3e80b5100 msgr2=0x7fc3e81638b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:26.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:26.066+0000 7fc3eee00640 1 -- 172.21.15.86:0/2066046181 shutdown_connections 2024-04-25T14:14:26.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:26.066+0000 7fc3eee00640 1 -- 172.21.15.86:0/2066046181 wait complete. 2024-04-25T14:14:26.485 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:14:26.485 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:14:27.486 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:27.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.650+0000 7f3b7da00640 1 Processor -- start 2024-04-25T14:14:27.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.650+0000 7f3b7da00640 1 -- start start 2024-04-25T14:14:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.650+0000 7f3b7da00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3b78070330 0x7f3b780d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:27.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.650+0000 7f3b7da00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b780d1c20 0x7f3b7806ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:27.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.650+0000 7f3b7da00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3b7806f270 0x7f3b7806f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:27.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.650+0000 7f3b7da00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3b780d2030 con 0x7f3b7806f270 2024-04-25T14:14:27.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.650+0000 7f3b7da00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3b7815e5c0 con 0x7f3b78070330 2024-04-25T14:14:27.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.650+0000 7f3b7da00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3b7815e8a0 con 0x7f3b780d1c20 2024-04-25T14:14:27.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.650+0000 7f3b77400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3b7806f270 0x7f3b7806f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:27.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.650+0000 7f3b77400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3b7806f270 0x7f3b7806f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54740/0 (socket says 172.21.15.86:54740) 2024-04-25T14:14:27.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.650+0000 7f3b77400640 1 -- 172.21.15.86:0/3926134311 learned_addr learned my addr 172.21.15.86:0/3926134311 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:27.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.650+0000 7f3b76000640 1 --2- 172.21.15.86:0/3926134311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b780d1c20 0x7f3b7806ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:27.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.650+0000 7f3b76a00640 1 --2- 172.21.15.86:0/3926134311 >> v2:172.21.15.155:3300/0 conn(0x7f3b78070330 0x7f3b780d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:27.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.650+0000 7f3b77400640 1 -- 172.21.15.86:0/3926134311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b780d1c20 msgr2=0x7f3b7806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:27.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.650+0000 7f3b77400640 1 --2- 172.21.15.86:0/3926134311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b780d1c20 0x7f3b7806ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:27.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.650+0000 7f3b77400640 1 -- 172.21.15.86:0/3926134311 >> v2:172.21.15.155:3300/0 conn(0x7f3b78070330 msgr2=0x7f3b780d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:27.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.650+0000 7f3b77400640 1 --2- 172.21.15.86:0/3926134311 >> v2:172.21.15.155:3300/0 conn(0x7f3b78070330 0x7f3b780d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:27.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.650+0000 7f3b77400640 1 -- 172.21.15.86:0/3926134311 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b7815eb80 con 0x7f3b7806f270 2024-04-25T14:14:27.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.650+0000 7f3b76000640 1 --2- 172.21.15.86:0/3926134311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b780d1c20 0x7f3b7806ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:27.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.650+0000 7f3b76a00640 1 --2- 172.21.15.86:0/3926134311 >> v2:172.21.15.155:3300/0 conn(0x7f3b78070330 0x7f3b780d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:27.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.654+0000 7f3b77400640 1 --2- 172.21.15.86:0/3926134311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b7806f270 0x7f3b7806f680 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f3b780d23c0 tx=0x7f3b7001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b6aacee83567367 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:27.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.654+0000 7f3b75600640 1 -- 172.21.15.86:0/3926134311 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b700ab600 con 0x7f3b7806f270 2024-04-25T14:14:27.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.654+0000 7f3b75600640 1 -- 172.21.15.86:0/3926134311 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b700b0070 con 0x7f3b7806f270 2024-04-25T14:14:27.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.654+0000 7f3b75600640 1 -- 172.21.15.86:0/3926134311 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b700ab9c0 con 0x7f3b7806f270 2024-04-25T14:14:27.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.654+0000 7f3b7da00640 1 -- 172.21.15.86:0/3926134311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b7806f270 msgr2=0x7f3b7806f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:27.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.654+0000 7f3b7da00640 1 --2- 172.21.15.86:0/3926134311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b7806f270 0x7f3b7806f680 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f3b780d23c0 tx=0x7f3b7001c460 comp rx=0 tx=0).stop 2024-04-25T14:14:27.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.654+0000 7f3b7da00640 1 -- 172.21.15.86:0/3926134311 shutdown_connections 2024-04-25T14:14:27.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.654+0000 7f3b7da00640 1 --2- 172.21.15.86:0/3926134311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b7806f270 0x7f3b7806f680 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:27.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.654+0000 7f3b7da00640 1 --2- 172.21.15.86:0/3926134311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b780d1c20 0x7f3b7806ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:27.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.654+0000 7f3b7da00640 1 --2- 172.21.15.86:0/3926134311 >> v2:172.21.15.155:3300/0 conn(0x7f3b78070330 0x7f3b780d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:27.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.654+0000 7f3b7da00640 1 -- 172.21.15.86:0/3926134311 >> 172.21.15.86:0/3926134311 conn(0x7f3b780b4650 msgr2=0x7f3b780b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:27.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.654+0000 7f3b7da00640 1 -- 172.21.15.86:0/3926134311 shutdown_connections 2024-04-25T14:14:27.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.654+0000 7f3b7da00640 1 -- 172.21.15.86:0/3926134311 wait complete. 2024-04-25T14:14:27.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.658+0000 7f3b7da00640 1 Processor -- start 2024-04-25T14:14:27.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.658+0000 7f3b7da00640 1 -- start start 2024-04-25T14:14:27.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.658+0000 7f3b7da00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b7806f270 0x7f3b7809a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:27.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.658+0000 7f3b7da00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b78070330 0x7f3b780b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:27.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.658+0000 7f3b7da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b780d1c20 0x7f3b780b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:27.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.658+0000 7f3b7da00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7815f4b0 con 0x7f3b780d1c20 2024-04-25T14:14:27.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.658+0000 7f3b7da00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7815f330 con 0x7f3b78070330 2024-04-25T14:14:27.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.658+0000 7f3b7da00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3b78070740 con 0x7f3b7806f270 2024-04-25T14:14:27.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.658+0000 7f3b77400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b780d1c20 0x7f3b780b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:27.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.658+0000 7f3b77400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b780d1c20 0x7f3b780b1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54754/0 (socket says 172.21.15.86:54754) 2024-04-25T14:14:27.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.658+0000 7f3b77400640 1 -- 172.21.15.86:0/965361497 learned_addr learned my addr 172.21.15.86:0/965361497 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:27.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.658+0000 7f3b76a00640 1 --2- 172.21.15.86:0/965361497 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b7806f270 0x7f3b7809a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:27.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.658+0000 7f3b76000640 1 --2- 172.21.15.86:0/965361497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b78070330 0x7f3b780b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:27.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.658+0000 7f3b77400640 1 -- 172.21.15.86:0/965361497 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b7806f270 msgr2=0x7f3b7809a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:27.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.658+0000 7f3b77400640 1 --2- 172.21.15.86:0/965361497 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b7806f270 0x7f3b7809a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:27.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.658+0000 7f3b77400640 1 -- 172.21.15.86:0/965361497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b78070330 msgr2=0x7f3b780b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:27.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.658+0000 7f3b77400640 1 --2- 172.21.15.86:0/965361497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b78070330 0x7f3b780b0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:27.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.658+0000 7f3b77400640 1 -- 172.21.15.86:0/965361497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b7809aeb0 con 0x7f3b780d1c20 2024-04-25T14:14:27.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.658+0000 7f3b77400640 1 --2- 172.21.15.86:0/965361497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b780d1c20 0x7f3b780b1270 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f3b7001cd00 tx=0x7f3b7001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:27.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.658+0000 7f3b6ea00640 1 -- 172.21.15.86:0/965361497 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b700be5f0 con 0x7f3b780d1c20 2024-04-25T14:14:27.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.658+0000 7f3b6ea00640 1 -- 172.21.15.86:0/965361497 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b700be790 con 0x7f3b780d1c20 2024-04-25T14:14:27.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.658+0000 7f3b7da00640 1 -- 172.21.15.86:0/965361497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b780c3e90 con 0x7f3b780d1c20 2024-04-25T14:14:27.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.658+0000 7f3b6ea00640 1 -- 172.21.15.86:0/965361497 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b700beb80 con 0x7f3b780d1c20 2024-04-25T14:14:27.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.658+0000 7f3b7da00640 1 -- 172.21.15.86:0/965361497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b780c4450 con 0x7f3b780d1c20 2024-04-25T14:14:27.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.662+0000 7f3b7da00640 1 -- 172.21.15.86:0/965361497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b78069d00 con 0x7f3b780d1c20 2024-04-25T14:14:27.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.662+0000 7f3b6ea00640 1 -- 172.21.15.86:0/965361497 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f3b700aa070 con 0x7f3b780d1c20 2024-04-25T14:14:27.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.662+0000 7f3b6ea00640 1 --2- 172.21.15.86:0/965361497 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f3b5806f570 0x7f3b58071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:27.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.662+0000 7f3b6ea00640 1 -- 172.21.15.86:0/965361497 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f3b70129b30 con 0x7f3b780d1c20 2024-04-25T14:14:27.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.662+0000 7f3b76a00640 1 --2- 172.21.15.86:0/965361497 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f3b5806f570 0x7f3b58071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:27.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.662+0000 7f3b76a00640 1 --2- 172.21.15.86:0/965361497 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f3b5806f570 0x7f3b58071a80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3b78072200 tx=0x7f3b680097f0 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:27.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.666+0000 7f3b6ea00640 1 -- 172.21.15.86:0/965361497 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3b700c4060 con 0x7f3b780d1c20 2024-04-25T14:14:27.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.862+0000 7f3b7da00640 1 -- 172.21.15.86:0/965361497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b78069d00 con 0x7f3b780d1c20 2024-04-25T14:14:27.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.862+0000 7f3b6ea00640 1 -- 172.21.15.86:0/965361497 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f3b70016110 con 0x7f3b780d1c20 2024-04-25T14:14:27.857 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:27.857 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:14:27.858 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:14:27.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.862+0000 7f3b7da00640 1 -- 172.21.15.86:0/965361497 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f3b5806f570 msgr2=0x7f3b58071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:27.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.862+0000 7f3b7da00640 1 --2- 172.21.15.86:0/965361497 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f3b5806f570 0x7f3b58071a80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3b78072200 tx=0x7f3b680097f0 comp rx=0 tx=0).stop 2024-04-25T14:14:27.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.862+0000 7f3b7da00640 1 -- 172.21.15.86:0/965361497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b780d1c20 msgr2=0x7f3b780b1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:27.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.862+0000 7f3b7da00640 1 --2- 172.21.15.86:0/965361497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b780d1c20 0x7f3b780b1270 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f3b7001cd00 tx=0x7f3b7001aca0 comp rx=0 tx=0).stop 2024-04-25T14:14:27.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.862+0000 7f3b7da00640 1 -- 172.21.15.86:0/965361497 shutdown_connections 2024-04-25T14:14:27.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.862+0000 7f3b7da00640 1 --2- 172.21.15.86:0/965361497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b780d1c20 0x7f3b780b1270 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:27.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.862+0000 7f3b7da00640 1 --2- 172.21.15.86:0/965361497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b78070330 0x7f3b780b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:27.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.862+0000 7f3b7da00640 1 --2- 172.21.15.86:0/965361497 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f3b5806f570 0x7f3b58071a80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:27.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.862+0000 7f3b7da00640 1 --2- 172.21.15.86:0/965361497 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b7806f270 0x7f3b7809a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:27.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.862+0000 7f3b7da00640 1 -- 172.21.15.86:0/965361497 >> 172.21.15.86:0/965361497 conn(0x7f3b780b4650 msgr2=0x7f3b7815ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:27.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.862+0000 7f3b7da00640 1 -- 172.21.15.86:0/965361497 shutdown_connections 2024-04-25T14:14:27.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:27.862+0000 7f3b7da00640 1 -- 172.21.15.86:0/965361497 wait complete. 2024-04-25T14:14:28.282 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:14:28.282 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:14:29.283 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:29.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.445+0000 7f7d47000640 1 Processor -- start 2024-04-25T14:14:29.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.445+0000 7f7d47000640 1 -- start start 2024-04-25T14:14:29.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.445+0000 7f7d47000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d4015fe00 0x7f7d40070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:29.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.445+0000 7f7d47000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7d40071360 0x7f7d401638a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:29.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.445+0000 7f7d47000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d40071770 con 0x7f7d40163de0 2024-04-25T14:14:29.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.445+0000 7f7d47000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f7d40160210 con 0x7f7d40071360 2024-04-25T14:14:29.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.445+0000 7f7d47000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7d4006dc30 con 0x7f7d4015fe00 2024-04-25T14:14:29.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.445+0000 7f7d45000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7d40163de0 0x7f7d400722f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57944/0 (socket says 172.21.15.86:57944) 2024-04-25T14:14:29.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.445+0000 7f7d45000640 1 -- 172.21.15.86:0/2124848669 learned_addr learned my addr 172.21.15.86:0/2124848669 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:14:29.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.445+0000 7f7d3fe00640 1 --2- 172.21.15.86:0/2124848669 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d4015fe00 0x7f7d40070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:29.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.445+0000 7f7d37400640 1 --2- 172.21.15.86:0/2124848669 >> v2:172.21.15.155:3300/0 conn(0x7f7d40071360 0x7f7d401638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:29.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.445+0000 7f7d3f400640 1 -- 172.21.15.86:0/2124848669 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1631812556 0 0) 0x7f7d380014a0 con 0x7f7d40163de0 2024-04-25T14:14:29.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.445+0000 7f7d3f400640 1 -- 172.21.15.86:0/2124848669 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d24003530 con 0x7f7d40163de0 2024-04-25T14:14:29.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.449+0000 7f7d37400640 1 -- 172.21.15.86:0/2124848669 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d4015fe00 msgr2=0x7f7d40070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:29.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.449+0000 7f7d37400640 1 --2- 172.21.15.86:0/2124848669 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d4015fe00 0x7f7d40070e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:29.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.449+0000 7f7d37400640 1 -- 172.21.15.86:0/2124848669 >> v1:172.21.15.86:6789/0 conn(0x7f7d40163de0 legacy=0x7f7d400722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:29.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.449+0000 7f7d37400640 1 -- 172.21.15.86:0/2124848669 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d401641f0 con 0x7f7d40071360 2024-04-25T14:14:29.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.449+0000 7f7d3fe00640 1 --2- 172.21.15.86:0/2124848669 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d4015fe00 0x7f7d40070e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:14:29.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.449+0000 7f7d37400640 1 --2- 172.21.15.86:0/2124848669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d40071360 0x7f7d401638a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f7d280097c0 tx=0x7f7d2801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b0375e17454d1d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:29.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.449+0000 7f7d3f400640 1 -- 172.21.15.86:0/2124848669 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d280b0070 con 0x7f7d40071360 2024-04-25T14:14:29.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.449+0000 7f7d3f400640 1 -- 172.21.15.86:0/2124848669 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d280a4e50 con 0x7f7d40071360 2024-04-25T14:14:29.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.449+0000 7f7d3f400640 1 -- 172.21.15.86:0/2124848669 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d280ab800 con 0x7f7d40071360 2024-04-25T14:14:29.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.453+0000 7f7d47000640 1 -- 172.21.15.86:0/2124848669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d40071360 msgr2=0x7f7d401638a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:29.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.453+0000 7f7d47000640 1 --2- 172.21.15.86:0/2124848669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d40071360 0x7f7d401638a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f7d280097c0 tx=0x7f7d2801c4a0 comp rx=0 tx=0).stop 2024-04-25T14:14:29.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.453+0000 7f7d47000640 1 -- 172.21.15.86:0/2124848669 shutdown_connections 2024-04-25T14:14:29.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.453+0000 7f7d47000640 1 --2- 172.21.15.86:0/2124848669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d40071360 0x7f7d401638a0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:29.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.453+0000 7f7d47000640 1 --2- 172.21.15.86:0/2124848669 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d4015fe00 0x7f7d40070e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:29.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.453+0000 7f7d47000640 1 -- 172.21.15.86:0/2124848669 >> 172.21.15.86:0/2124848669 conn(0x7f7d400cffc0 msgr2=0x7f7d400d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:29.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.453+0000 7f7d47000640 1 -- 172.21.15.86:0/2124848669 shutdown_connections 2024-04-25T14:14:29.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.453+0000 7f7d47000640 1 -- 172.21.15.86:0/2124848669 wait complete. 2024-04-25T14:14:29.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.453+0000 7f7d47000640 1 Processor -- start 2024-04-25T14:14:29.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.453+0000 7f7d47000640 1 -- start start 2024-04-25T14:14:29.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.457+0000 7f7d47000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d40071360 0x7f7d400cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:29.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.457+0000 7f7d47000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d4015fe00 0x7f7d400cc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:29.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.457+0000 7f7d47000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d40163de0 0x7f7d400b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:29.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.457+0000 7f7d47000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4016cc20 con 0x7f7d4015fe00 2024-04-25T14:14:29.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.457+0000 7f7d47000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4006e910 con 0x7f7d40071360 2024-04-25T14:14:29.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.457+0000 7f7d47000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7d4016cda0 con 0x7f7d40163de0 2024-04-25T14:14:29.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.457+0000 7f7d37400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d4015fe00 0x7f7d400cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:29.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.457+0000 7f7d37400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d4015fe00 0x7f7d400cc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54478/0 (socket says 172.21.15.86:54478) 2024-04-25T14:14:29.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.457+0000 7f7d37400640 1 -- 172.21.15.86:0/1763841447 learned_addr learned my addr 172.21.15.86:0/1763841447 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:29.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.457+0000 7f7d3fe00640 1 --2- 172.21.15.86:0/1763841447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d40071360 0x7f7d400cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:29.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.457+0000 7f7d45000640 1 --2- 172.21.15.86:0/1763841447 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d40163de0 0x7f7d400b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:29.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.457+0000 7f7d37400640 1 -- 172.21.15.86:0/1763841447 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d40163de0 msgr2=0x7f7d400b6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:29.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.457+0000 7f7d37400640 1 --2- 172.21.15.86:0/1763841447 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d40163de0 0x7f7d400b6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:29.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.457+0000 7f7d37400640 1 -- 172.21.15.86:0/1763841447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d40071360 msgr2=0x7f7d400cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:29.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.457+0000 7f7d37400640 1 --2- 172.21.15.86:0/1763841447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d40071360 0x7f7d400cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:29.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.457+0000 7f7d37400640 1 -- 172.21.15.86:0/1763841447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d40151400 con 0x7f7d4015fe00 2024-04-25T14:14:29.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.457+0000 7f7d37400640 1 --2- 172.21.15.86:0/1763841447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d4015fe00 0x7f7d400cc170 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f7d2801a0d0 tx=0x7f7d28019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:29.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.457+0000 7f7d3cc00640 1 -- 172.21.15.86:0/1763841447 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d280ba070 con 0x7f7d4015fe00 2024-04-25T14:14:29.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.457+0000 7f7d3cc00640 1 -- 172.21.15.86:0/1763841447 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d28003820 con 0x7f7d4015fe00 2024-04-25T14:14:29.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.457+0000 7f7d3cc00640 1 -- 172.21.15.86:0/1763841447 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d280b0040 con 0x7f7d4015fe00 2024-04-25T14:14:29.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.457+0000 7f7d47000640 1 -- 172.21.15.86:0/1763841447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d400b8fc0 con 0x7f7d4015fe00 2024-04-25T14:14:29.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.457+0000 7f7d47000640 1 -- 172.21.15.86:0/1763841447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d4014e7c0 con 0x7f7d4015fe00 2024-04-25T14:14:29.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.457+0000 7f7d47000640 1 -- 172.21.15.86:0/1763841447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d401515d0 con 0x7f7d4015fe00 2024-04-25T14:14:29.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.457+0000 7f7d3cc00640 1 -- 172.21.15.86:0/1763841447 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f7d28003bd0 con 0x7f7d4015fe00 2024-04-25T14:14:29.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.461+0000 7f7d3cc00640 1 --2- 172.21.15.86:0/1763841447 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7d1c06f5a0 0x7f7d1c071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:29.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.461+0000 7f7d3cc00640 1 -- 172.21.15.86:0/1763841447 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f7d28128700 con 0x7f7d4015fe00 2024-04-25T14:14:29.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.461+0000 7f7d3fe00640 1 --2- 172.21.15.86:0/1763841447 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7d1c06f5a0 0x7f7d1c071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:29.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.461+0000 7f7d3fe00640 1 --2- 172.21.15.86:0/1763841447 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7d1c06f5a0 0x7f7d1c071ab0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7d30009580 tx=0x7f7d30009190 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:29.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.465+0000 7f7d3cc00640 1 -- 172.21.15.86:0/1763841447 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7d401515d0 con 0x7f7d4015fe00 2024-04-25T14:14:29.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.653+0000 7f7d47000640 1 -- 172.21.15.86:0/1763841447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d400720e0 con 0x7f7d4015fe00 2024-04-25T14:14:29.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.653+0000 7f7d3cc00640 1 -- 172.21.15.86:0/1763841447 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f7d28016090 con 0x7f7d4015fe00 2024-04-25T14:14:29.650 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:29.650 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:14:29.650 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:14:29.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.657+0000 7f7d47000640 1 -- 172.21.15.86:0/1763841447 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7d1c06f5a0 msgr2=0x7f7d1c071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:29.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.657+0000 7f7d47000640 1 --2- 172.21.15.86:0/1763841447 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7d1c06f5a0 0x7f7d1c071ab0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7d30009580 tx=0x7f7d30009190 comp rx=0 tx=0).stop 2024-04-25T14:14:29.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.657+0000 7f7d47000640 1 -- 172.21.15.86:0/1763841447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d4015fe00 msgr2=0x7f7d400cc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:29.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.657+0000 7f7d47000640 1 --2- 172.21.15.86:0/1763841447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d4015fe00 0x7f7d400cc170 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f7d2801a0d0 tx=0x7f7d28019240 comp rx=0 tx=0).stop 2024-04-25T14:14:29.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.657+0000 7f7d47000640 1 -- 172.21.15.86:0/1763841447 shutdown_connections 2024-04-25T14:14:29.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.657+0000 7f7d47000640 1 --2- 172.21.15.86:0/1763841447 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7d1c06f5a0 0x7f7d1c071ab0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:29.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.657+0000 7f7d47000640 1 --2- 172.21.15.86:0/1763841447 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d40163de0 0x7f7d400b6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:29.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.657+0000 7f7d47000640 1 --2- 172.21.15.86:0/1763841447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d4015fe00 0x7f7d400cc170 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:29.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.657+0000 7f7d47000640 1 --2- 172.21.15.86:0/1763841447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d40071360 0x7f7d400cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:29.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.657+0000 7f7d47000640 1 -- 172.21.15.86:0/1763841447 >> 172.21.15.86:0/1763841447 conn(0x7f7d400cffc0 msgr2=0x7f7d40152a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:29.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.657+0000 7f7d47000640 1 -- 172.21.15.86:0/1763841447 shutdown_connections 2024-04-25T14:14:29.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:29.657+0000 7f7d47000640 1 -- 172.21.15.86:0/1763841447 wait complete. 2024-04-25T14:14:30.075 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:14:30.075 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:14:31.076 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:31.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.237+0000 7fece9e00640 1 Processor -- start 2024-04-25T14:14:31.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.237+0000 7fece9e00640 1 -- start start 2024-04-25T14:14:31.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.241+0000 7fece9e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fece40d1df0 0x7fece40cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:31.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.241+0000 7fece9e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fece40cdcc0 0x7fece40ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:31.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.241+0000 7fece9e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fece415e5c0 con 0x7fece4070330 2024-04-25T14:14:31.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.241+0000 7fece9e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fece406fd40 con 0x7fece40cdcc0 2024-04-25T14:14:31.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.241+0000 7fece9e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fece415e8a0 con 0x7fece40d1df0 2024-04-25T14:14:31.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.241+0000 7fece3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fece40cdcc0 0x7fece40ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:31.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.241+0000 7fece3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fece40cdcc0 0x7fece40ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54480/0 (socket says 172.21.15.86:54480) 2024-04-25T14:14:31.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.241+0000 7fece3400640 1 -- 172.21.15.86:0/1413614021 learned_addr learned my addr 172.21.15.86:0/1413614021 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:31.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.241+0000 7fece2a00640 1 --1- 172.21.15.86:0/1413614021 >> v1:172.21.15.86:6789/0 conn(0x7fece4070330 0x7fece40d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57950/0 (socket says 172.21.15.86:57950) 2024-04-25T14:14:31.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.241+0000 7fecda000640 1 --2- 172.21.15.86:0/1413614021 >> v2:172.21.15.155:3300/0 conn(0x7fece40d1df0 0x7fece40cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:31.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.241+0000 7fece3400640 1 -- 172.21.15.86:0/1413614021 >> v2:172.21.15.155:3300/0 conn(0x7fece40d1df0 msgr2=0x7fece40cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:31.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.241+0000 7fece3400640 1 --2- 172.21.15.86:0/1413614021 >> v2:172.21.15.155:3300/0 conn(0x7fece40d1df0 0x7fece40cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.241+0000 7fece3400640 1 -- 172.21.15.86:0/1413614021 >> v1:172.21.15.86:6789/0 conn(0x7fece4070330 legacy=0x7fece40d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:31.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.241+0000 7fece3400640 1 -- 172.21.15.86:0/1413614021 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece415eb80 con 0x7fece40cdcc0 2024-04-25T14:14:31.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.241+0000 7fece2000640 1 -- 172.21.15.86:0/1413614021 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 817609968 0 0) 0x7fece415e5c0 con 0x7fece4070330 2024-04-25T14:14:31.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.245+0000 7fece3400640 1 --2- 172.21.15.86:0/1413614021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece40cdcc0 0x7fece40ce0d0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fecdc009870 tx=0x7fecdc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7a82774c2ef709 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:31.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.245+0000 7fece2000640 1 -- 172.21.15.86:0/1413614021 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc0ab640 con 0x7fece40cdcc0 2024-04-25T14:14:31.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.245+0000 7fece2000640 1 -- 172.21.15.86:0/1413614021 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecdc0b0070 con 0x7fece40cdcc0 2024-04-25T14:14:31.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.245+0000 7fece2000640 1 -- 172.21.15.86:0/1413614021 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc0aba00 con 0x7fece40cdcc0 2024-04-25T14:14:31.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.245+0000 7fece9e00640 1 -- 172.21.15.86:0/1413614021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece40cdcc0 msgr2=0x7fece40ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:31.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.245+0000 7fece9e00640 1 --2- 172.21.15.86:0/1413614021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece40cdcc0 0x7fece40ce0d0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fecdc009870 tx=0x7fecdc01c460 comp rx=0 tx=0).stop 2024-04-25T14:14:31.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.245+0000 7fece9e00640 1 -- 172.21.15.86:0/1413614021 shutdown_connections 2024-04-25T14:14:31.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.245+0000 7fece9e00640 1 --2- 172.21.15.86:0/1413614021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece40cdcc0 0x7fece40ce0d0 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.245+0000 7fece9e00640 1 --2- 172.21.15.86:0/1413614021 >> v2:172.21.15.155:3300/0 conn(0x7fece40d1df0 0x7fece40cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.245+0000 7fece9e00640 1 -- 172.21.15.86:0/1413614021 >> 172.21.15.86:0/1413614021 conn(0x7fece40b4650 msgr2=0x7fece40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:31.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.245+0000 7fece9e00640 1 -- 172.21.15.86:0/1413614021 shutdown_connections 2024-04-25T14:14:31.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.245+0000 7fece9e00640 1 -- 172.21.15.86:0/1413614021 wait complete. 2024-04-25T14:14:31.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.249+0000 7fece9e00640 1 Processor -- start 2024-04-25T14:14:31.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.249+0000 7fece9e00640 1 -- start start 2024-04-25T14:14:31.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.249+0000 7fece9e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fece4070330 0x7fece40ae580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:31.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.249+0000 7fece9e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fece40cdcc0 0x7fece41fcb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:31.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.249+0000 7fece9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece41fd070 0x7fece40c6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:31.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.249+0000 7fece9e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fece415f4b0 con 0x7fece41fd070 2024-04-25T14:14:31.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.249+0000 7fece9e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fece415f330 con 0x7fece40cdcc0 2024-04-25T14:14:31.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.249+0000 7fece9e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fece4070740 con 0x7fece4070330 2024-04-25T14:14:31.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.249+0000 7fece3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece41fd070 0x7fece40c6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:31.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.249+0000 7fece3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece41fd070 0x7fece40c6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54496/0 (socket says 172.21.15.86:54496) 2024-04-25T14:14:31.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.249+0000 7fece3400640 1 -- 172.21.15.86:0/517690120 learned_addr learned my addr 172.21.15.86:0/517690120 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:31.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.249+0000 7fecda000640 1 --2- 172.21.15.86:0/517690120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fece40cdcc0 0x7fece41fcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:31.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.249+0000 7fece2a00640 1 --2- 172.21.15.86:0/517690120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fece4070330 0x7fece40ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:31.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.249+0000 7fece3400640 1 -- 172.21.15.86:0/517690120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fece4070330 msgr2=0x7fece40ae580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:31.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.249+0000 7fece3400640 1 --2- 172.21.15.86:0/517690120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fece4070330 0x7fece40ae580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.249+0000 7fece3400640 1 -- 172.21.15.86:0/517690120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fece40cdcc0 msgr2=0x7fece41fcb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:31.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.249+0000 7fece3400640 1 --2- 172.21.15.86:0/517690120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fece40cdcc0 0x7fece41fcb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.249+0000 7fece3400640 1 -- 172.21.15.86:0/517690120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece41fd480 con 0x7fece41fd070 2024-04-25T14:14:31.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.249+0000 7fece2a00640 1 --2- 172.21.15.86:0/517690120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fece4070330 0x7fece40ae580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:31.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.249+0000 7fece3400640 1 --2- 172.21.15.86:0/517690120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece41fd070 0x7fece40c6570 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7fecdc019070 tx=0x7fecdc01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:31.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.249+0000 7fecdbe00640 1 -- 172.21.15.86:0/517690120 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc0abe20 con 0x7fece41fd070 2024-04-25T14:14:31.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.249+0000 7fecdbe00640 1 -- 172.21.15.86:0/517690120 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecdc0be3f0 con 0x7fece41fd070 2024-04-25T14:14:31.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.249+0000 7fece9e00640 1 -- 172.21.15.86:0/517690120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fece415a0b0 con 0x7fece41fd070 2024-04-25T14:14:31.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.249+0000 7fecdbe00640 1 -- 172.21.15.86:0/517690120 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc0be7e0 con 0x7fece41fd070 2024-04-25T14:14:31.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.249+0000 7fece9e00640 1 -- 172.21.15.86:0/517690120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fece415a670 con 0x7fece41fd070 2024-04-25T14:14:31.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.253+0000 7fece9e00640 1 -- 172.21.15.86:0/517690120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fece4069d00 con 0x7fece41fd070 2024-04-25T14:14:31.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.253+0000 7fecdbe00640 1 -- 172.21.15.86:0/517690120 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7fecdc0b43f0 con 0x7fece41fd070 2024-04-25T14:14:31.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.253+0000 7fecdbe00640 1 --2- 172.21.15.86:0/517690120 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fecc006f570 0x7fecc0071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:31.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.253+0000 7fecdbe00640 1 -- 172.21.15.86:0/517690120 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fecdc129780 con 0x7fece41fd070 2024-04-25T14:14:31.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.253+0000 7fece2a00640 1 --2- 172.21.15.86:0/517690120 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fecc006f570 0x7fecc0071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:31.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.253+0000 7fece2a00640 1 --2- 172.21.15.86:0/517690120 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fecc006f570 0x7fecc0071a80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fecd4007fd0 tx=0x7fecd4005b80 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:31.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.257+0000 7fecdbe00640 1 -- 172.21.15.86:0/517690120 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fecdc0c4060 con 0x7fece41fd070 2024-04-25T14:14:31.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.445+0000 7fece9e00640 1 -- 172.21.15.86:0/517690120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fece415b7a0 con 0x7fece41fd070 2024-04-25T14:14:31.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.449+0000 7fecdbe00640 1 -- 172.21.15.86:0/517690120 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7fece41fd480 con 0x7fece41fd070 2024-04-25T14:14:31.444 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:31.444 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:14:31.444 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:14:31.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.449+0000 7fece9e00640 1 -- 172.21.15.86:0/517690120 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fecc006f570 msgr2=0x7fecc0071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:31.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.449+0000 7fece9e00640 1 --2- 172.21.15.86:0/517690120 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fecc006f570 0x7fecc0071a80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fecd4007fd0 tx=0x7fecd4005b80 comp rx=0 tx=0).stop 2024-04-25T14:14:31.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.449+0000 7fece9e00640 1 -- 172.21.15.86:0/517690120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece41fd070 msgr2=0x7fece40c6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:31.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.449+0000 7fece9e00640 1 --2- 172.21.15.86:0/517690120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece41fd070 0x7fece40c6570 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7fecdc019070 tx=0x7fecdc01aca0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.449+0000 7fece9e00640 1 -- 172.21.15.86:0/517690120 shutdown_connections 2024-04-25T14:14:31.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.449+0000 7fece9e00640 1 --2- 172.21.15.86:0/517690120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece41fd070 0x7fece40c6570 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.449+0000 7fece9e00640 1 --2- 172.21.15.86:0/517690120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fece40cdcc0 0x7fece41fcb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.449+0000 7fece9e00640 1 --2- 172.21.15.86:0/517690120 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fecc006f570 0x7fecc0071a80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.449+0000 7fece9e00640 1 --2- 172.21.15.86:0/517690120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fece4070330 0x7fece40ae580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:31.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.449+0000 7fece9e00640 1 -- 172.21.15.86:0/517690120 >> 172.21.15.86:0/517690120 conn(0x7fece40b4650 msgr2=0x7fece40b17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:31.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.449+0000 7fece9e00640 1 -- 172.21.15.86:0/517690120 shutdown_connections 2024-04-25T14:14:31.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:31.449+0000 7fece9e00640 1 -- 172.21.15.86:0/517690120 wait complete. 2024-04-25T14:14:31.868 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:14:31.868 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:14:32.870 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:33.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.029+0000 7fd19d000640 1 Processor -- start 2024-04-25T14:14:33.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.029+0000 7fd19d000640 1 -- start start 2024-04-25T14:14:33.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.029+0000 7fd19d000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd198070330 0x7fd1980d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:33.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.029+0000 7fd19d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd1980ce790 0x7fd19806f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:33.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.029+0000 7fd19d000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1980d2030 con 0x7fd1980d1c20 2024-04-25T14:14:33.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.029+0000 7fd19d000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd19815e5c0 con 0x7fd1980ce790 2024-04-25T14:14:33.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.029+0000 7fd19d000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd19815e8a0 con 0x7fd198070330 2024-04-25T14:14:33.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.029+0000 7fd196800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd1980ce790 0x7fd19806f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:33.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.029+0000 7fd196800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd1980ce790 0x7fd19806f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54504/0 (socket says 172.21.15.86:54504) 2024-04-25T14:14:33.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.029+0000 7fd196800640 1 -- 172.21.15.86:0/2222774882 learned_addr learned my addr 172.21.15.86:0/2222774882 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:33.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.029+0000 7fd195e00640 1 --2- 172.21.15.86:0/2222774882 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd198070330 0x7fd1980d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:33.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.029+0000 7fd196800640 1 -- 172.21.15.86:0/2222774882 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd198070330 msgr2=0x7fd1980d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:33.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.029+0000 7fd196800640 1 --2- 172.21.15.86:0/2222774882 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd198070330 0x7fd1980d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:33.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.029+0000 7fd196800640 1 -- 172.21.15.86:0/2222774882 >> v1:172.21.15.155:6789/0 conn(0x7fd1980d1c20 legacy=0x7fd19806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:33.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.029+0000 7fd196800640 1 -- 172.21.15.86:0/2222774882 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19815eb80 con 0x7fd1980ce790 2024-04-25T14:14:33.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.033+0000 7fd196800640 1 --2- 172.21.15.86:0/2222774882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1980ce790 0x7fd19806f440 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7fd190009e80 tx=0x7fd19001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e186c6c528b8f72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:33.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.033+0000 7fd195400640 1 -- 172.21.15.86:0/2222774882 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1900a5640 con 0x7fd1980ce790 2024-04-25T14:14:33.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.033+0000 7fd195400640 1 -- 172.21.15.86:0/2222774882 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1900a57e0 con 0x7fd1980ce790 2024-04-25T14:14:33.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.033+0000 7fd195400640 1 -- 172.21.15.86:0/2222774882 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1900a5bb0 con 0x7fd1980ce790 2024-04-25T14:14:33.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.033+0000 7fd19d000640 1 -- 172.21.15.86:0/2222774882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1980ce790 msgr2=0x7fd19806f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:33.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.033+0000 7fd19d000640 1 --2- 172.21.15.86:0/2222774882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1980ce790 0x7fd19806f440 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7fd190009e80 tx=0x7fd19001c980 comp rx=0 tx=0).stop 2024-04-25T14:14:33.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.033+0000 7fd19d000640 1 -- 172.21.15.86:0/2222774882 shutdown_connections 2024-04-25T14:14:33.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.033+0000 7fd19d000640 1 --2- 172.21.15.86:0/2222774882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1980ce790 0x7fd19806f440 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:33.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.033+0000 7fd19d000640 1 --2- 172.21.15.86:0/2222774882 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd198070330 0x7fd1980d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:33.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.033+0000 7fd19d000640 1 -- 172.21.15.86:0/2222774882 >> 172.21.15.86:0/2222774882 conn(0x7fd1980b4650 msgr2=0x7fd1980b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:33.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.033+0000 7fd19d000640 1 -- 172.21.15.86:0/2222774882 shutdown_connections 2024-04-25T14:14:33.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.033+0000 7fd19d000640 1 -- 172.21.15.86:0/2222774882 wait complete. 2024-04-25T14:14:33.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.037+0000 7fd19d000640 1 Processor -- start 2024-04-25T14:14:33.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.037+0000 7fd19d000640 1 -- start start 2024-04-25T14:14:33.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.037+0000 7fd19d000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd198070330 0x7fd19809a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:33.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.037+0000 7fd19d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1980ce790 0x7fd19815c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:33.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.037+0000 7fd19d000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1980d1c20 0x7fd19815caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:33.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.037+0000 7fd19d000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1980708c0 con 0x7fd1980ce790 2024-04-25T14:14:33.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.037+0000 7fd19d000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd198070740 con 0x7fd1980d1c20 2024-04-25T14:14:33.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.037+0000 7fd19d000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd19816afc0 con 0x7fd198070330 2024-04-25T14:14:33.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.037+0000 7fd18d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1980ce790 0x7fd19815c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:33.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.037+0000 7fd18d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1980ce790 0x7fd19815c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54514/0 (socket says 172.21.15.86:54514) 2024-04-25T14:14:33.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.037+0000 7fd18d400640 1 -- 172.21.15.86:0/3894588523 learned_addr learned my addr 172.21.15.86:0/3894588523 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:33.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.041+0000 7fd196800640 1 --2- 172.21.15.86:0/3894588523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1980d1c20 0x7fd19815caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:33.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.041+0000 7fd195e00640 1 --2- 172.21.15.86:0/3894588523 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd198070330 0x7fd19809a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:33.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.041+0000 7fd18d400640 1 -- 172.21.15.86:0/3894588523 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd198070330 msgr2=0x7fd19809a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:33.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.041+0000 7fd18d400640 1 --2- 172.21.15.86:0/3894588523 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd198070330 0x7fd19809a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:33.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.041+0000 7fd18d400640 1 -- 172.21.15.86:0/3894588523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1980d1c20 msgr2=0x7fd19815caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:33.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.041+0000 7fd18d400640 1 --2- 172.21.15.86:0/3894588523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1980d1c20 0x7fd19815caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:33.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.041+0000 7fd18d400640 1 -- 172.21.15.86:0/3894588523 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19809ae90 con 0x7fd1980ce790 2024-04-25T14:14:33.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.041+0000 7fd18d400640 1 --2- 172.21.15.86:0/3894588523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1980ce790 0x7fd19815c560 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7fd18000c3a0 tx=0x7fd18000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:33.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.041+0000 7fd18ea00640 1 -- 172.21.15.86:0/3894588523 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1800052d0 con 0x7fd1980ce790 2024-04-25T14:14:33.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.041+0000 7fd18ea00640 1 -- 172.21.15.86:0/3894588523 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd180005470 con 0x7fd1980ce790 2024-04-25T14:14:33.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.041+0000 7fd19d000640 1 -- 172.21.15.86:0/3894588523 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd19815d2b0 con 0x7fd1980ce790 2024-04-25T14:14:33.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.041+0000 7fd18ea00640 1 -- 172.21.15.86:0/3894588523 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd180007260 con 0x7fd1980ce790 2024-04-25T14:14:33.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.041+0000 7fd19d000640 1 -- 172.21.15.86:0/3894588523 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd198072c00 con 0x7fd1980ce790 2024-04-25T14:14:33.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.041+0000 7fd19d000640 1 -- 172.21.15.86:0/3894588523 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd198069d00 con 0x7fd1980ce790 2024-04-25T14:14:33.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.041+0000 7fd18ea00640 1 -- 172.21.15.86:0/3894588523 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7fd180007400 con 0x7fd1980ce790 2024-04-25T14:14:33.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.045+0000 7fd18ea00640 1 --2- 172.21.15.86:0/3894588523 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd17406f570 0x7fd174071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:33.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.045+0000 7fd18ea00640 1 -- 172.21.15.86:0/3894588523 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fd18008eb90 con 0x7fd1980ce790 2024-04-25T14:14:33.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.045+0000 7fd195e00640 1 --2- 172.21.15.86:0/3894588523 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd17406f570 0x7fd174071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:33.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.045+0000 7fd195e00640 1 --2- 172.21.15.86:0/3894588523 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd17406f570 0x7fd174071a80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd198072200 tx=0x7fd188008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:33.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.049+0000 7fd18ea00640 1 -- 172.21.15.86:0/3894588523 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd1980d2030 con 0x7fd1980ce790 2024-04-25T14:14:33.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.237+0000 7fd19d000640 1 -- 172.21.15.86:0/3894588523 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd19815d040 con 0x7fd1980ce790 2024-04-25T14:14:33.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.237+0000 7fd18ea00640 1 -- 172.21.15.86:0/3894588523 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7fd18005b720 con 0x7fd1980ce790 2024-04-25T14:14:33.234 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:33.234 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:14:33.234 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:14:33.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.237+0000 7fd19d000640 1 -- 172.21.15.86:0/3894588523 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd17406f570 msgr2=0x7fd174071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:33.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.237+0000 7fd19d000640 1 --2- 172.21.15.86:0/3894588523 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd17406f570 0x7fd174071a80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd198072200 tx=0x7fd188008040 comp rx=0 tx=0).stop 2024-04-25T14:14:33.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.237+0000 7fd19d000640 1 -- 172.21.15.86:0/3894588523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1980ce790 msgr2=0x7fd19815c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:33.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.237+0000 7fd19d000640 1 --2- 172.21.15.86:0/3894588523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1980ce790 0x7fd19815c560 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7fd18000c3a0 tx=0x7fd18000c860 comp rx=0 tx=0).stop 2024-04-25T14:14:33.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.237+0000 7fd19d000640 1 -- 172.21.15.86:0/3894588523 shutdown_connections 2024-04-25T14:14:33.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.237+0000 7fd19d000640 1 --2- 172.21.15.86:0/3894588523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1980d1c20 0x7fd19815caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:33.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.237+0000 7fd19d000640 1 --2- 172.21.15.86:0/3894588523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1980ce790 0x7fd19815c560 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:33.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.237+0000 7fd19d000640 1 --2- 172.21.15.86:0/3894588523 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd17406f570 0x7fd174071a80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:33.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.237+0000 7fd19d000640 1 --2- 172.21.15.86:0/3894588523 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd198070330 0x7fd19809a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:33.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.237+0000 7fd19d000640 1 -- 172.21.15.86:0/3894588523 >> 172.21.15.86:0/3894588523 conn(0x7fd1980b4650 msgr2=0x7fd19815af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:33.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.241+0000 7fd19d000640 1 -- 172.21.15.86:0/3894588523 shutdown_connections 2024-04-25T14:14:33.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:33.241+0000 7fd19d000640 1 -- 172.21.15.86:0/3894588523 wait complete. 2024-04-25T14:14:33.659 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:14:33.659 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:14:34.659 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:34.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.821+0000 7f3a5f800640 1 Processor -- start 2024-04-25T14:14:34.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.821+0000 7f3a5f800640 1 -- start start 2024-04-25T14:14:34.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.825+0000 7f3a5f800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3a5806fd50 0x7f3a58070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:34.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.825+0000 7f3a5f800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3a5815feb0 0x7f3a581602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:34.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.825+0000 7f3a5f800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a580b0a60 con 0x7f3a5815c2a0 2024-04-25T14:14:34.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.825+0000 7f3a5f800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3a580ae840 con 0x7f3a5806fd50 2024-04-25T14:14:34.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.825+0000 7f3a5f800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3a5806ef90 con 0x7f3a5815feb0 2024-04-25T14:14:34.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.825+0000 7f3a4fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3a5806fd50 0x7f3a58070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:34.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.825+0000 7f3a4fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3a5806fd50 0x7f3a58070160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54520/0 (socket says 172.21.15.86:54520) 2024-04-25T14:14:34.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.825+0000 7f3a4fe00640 1 -- 172.21.15.86:0/3301221963 learned_addr learned my addr 172.21.15.86:0/3301221963 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:34.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.825+0000 7f3a4fe00640 1 -- 172.21.15.86:0/3301221963 >> v2:172.21.15.155:3300/0 conn(0x7f3a5815feb0 msgr2=0x7f3a581602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:34.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.825+0000 7f3a5d800640 1 --2- 172.21.15.86:0/3301221963 >> v2:172.21.15.155:3300/0 conn(0x7f3a5815feb0 0x7f3a581602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:34.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.825+0000 7f3a4fe00640 1 --2- 172.21.15.86:0/3301221963 >> v2:172.21.15.155:3300/0 conn(0x7f3a5815feb0 0x7f3a581602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:34.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.825+0000 7f3a4fe00640 1 -- 172.21.15.86:0/3301221963 >> v1:172.21.15.86:6789/0 conn(0x7f3a5815c2a0 legacy=0x7f3a580ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:34.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.825+0000 7f3a4fe00640 1 -- 172.21.15.86:0/3301221963 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5806f270 con 0x7f3a5806fd50 2024-04-25T14:14:34.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.825+0000 7f3a5d800640 1 --2- 172.21.15.86:0/3301221963 >> v2:172.21.15.155:3300/0 conn(0x7f3a5815feb0 0x7f3a581602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:14:34.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.825+0000 7f3a4fe00640 1 --2- 172.21.15.86:0/3301221963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a5806fd50 0x7f3a58070160 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f3a580b0940 tx=0x7f3a4001d570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8805ca5c487288d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:34.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.829+0000 7f3a57e00640 1 -- 172.21.15.86:0/3301221963 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a400a7040 con 0x7f3a5806fd50 2024-04-25T14:14:34.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.829+0000 7f3a57e00640 1 -- 172.21.15.86:0/3301221963 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a400b0070 con 0x7f3a5806fd50 2024-04-25T14:14:34.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.829+0000 7f3a57e00640 1 -- 172.21.15.86:0/3301221963 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a400ab860 con 0x7f3a5806fd50 2024-04-25T14:14:34.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.829+0000 7f3a5f800640 1 -- 172.21.15.86:0/3301221963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a5806fd50 msgr2=0x7f3a58070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:34.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.829+0000 7f3a5f800640 1 --2- 172.21.15.86:0/3301221963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a5806fd50 0x7f3a58070160 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f3a580b0940 tx=0x7f3a4001d570 comp rx=0 tx=0).stop 2024-04-25T14:14:34.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.829+0000 7f3a5f800640 1 -- 172.21.15.86:0/3301221963 shutdown_connections 2024-04-25T14:14:34.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.829+0000 7f3a5f800640 1 --2- 172.21.15.86:0/3301221963 >> v2:172.21.15.155:3300/0 conn(0x7f3a5815feb0 0x7f3a581602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:34.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.829+0000 7f3a5f800640 1 --2- 172.21.15.86:0/3301221963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a5806fd50 0x7f3a58070160 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:34.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.829+0000 7f3a5f800640 1 -- 172.21.15.86:0/3301221963 >> 172.21.15.86:0/3301221963 conn(0x7f3a580b4650 msgr2=0x7f3a580b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:34.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.829+0000 7f3a5f800640 1 -- 172.21.15.86:0/3301221963 shutdown_connections 2024-04-25T14:14:34.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.829+0000 7f3a5f800640 1 -- 172.21.15.86:0/3301221963 wait complete. 2024-04-25T14:14:34.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.829+0000 7f3a5f800640 1 Processor -- start 2024-04-25T14:14:34.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.829+0000 7f3a5f800640 1 -- start start 2024-04-25T14:14:34.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.833+0000 7f3a5f800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a5806fd50 0x7f3a58157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:34.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.833+0000 7f3a5f800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a5815c2a0 0x7f3a580af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:34.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.833+0000 7f3a5f800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a5815feb0 0x7f3a580af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:34.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.833+0000 7f3a5f800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5815c6b0 con 0x7f3a5815feb0 2024-04-25T14:14:34.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.833+0000 7f3a5f800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3a580d0fc0 con 0x7f3a5815c2a0 2024-04-25T14:14:34.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.833+0000 7f3a5f800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3a5815c830 con 0x7f3a5806fd50 2024-04-25T14:14:34.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.833+0000 7f3a5d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a5815feb0 0x7f3a580af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:34.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.833+0000 7f3a5d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a5815feb0 0x7f3a580af720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54532/0 (socket says 172.21.15.86:54532) 2024-04-25T14:14:34.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.833+0000 7f3a5d800640 1 -- 172.21.15.86:0/2254321857 learned_addr learned my addr 172.21.15.86:0/2254321857 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:34.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.833+0000 7f3a4fe00640 1 --2- 172.21.15.86:0/2254321857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a5815c2a0 0x7f3a580af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:34.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.833+0000 7f3a5ce00640 1 --2- 172.21.15.86:0/2254321857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a5806fd50 0x7f3a58157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:34.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.833+0000 7f3a5d800640 1 -- 172.21.15.86:0/2254321857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a5806fd50 msgr2=0x7f3a58157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:34.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.833+0000 7f3a5d800640 1 --2- 172.21.15.86:0/2254321857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a5806fd50 0x7f3a58157030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:34.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.833+0000 7f3a5d800640 1 -- 172.21.15.86:0/2254321857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a5815c2a0 msgr2=0x7f3a580af1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:34.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.833+0000 7f3a5d800640 1 --2- 172.21.15.86:0/2254321857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a5815c2a0 0x7f3a580af1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:34.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.833+0000 7f3a5d800640 1 -- 172.21.15.86:0/2254321857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5815cf90 con 0x7f3a5815feb0 2024-04-25T14:14:34.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.833+0000 7f3a5d800640 1 --2- 172.21.15.86:0/2254321857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a5815feb0 0x7f3a580af720 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000cd50 tx=0x7f3a5000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:34.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.833+0000 7f3a55600640 1 -- 172.21.15.86:0/2254321857 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5000f680 con 0x7f3a5815feb0 2024-04-25T14:14:34.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.833+0000 7f3a55600640 1 -- 172.21.15.86:0/2254321857 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a5000f840 con 0x7f3a5815feb0 2024-04-25T14:14:34.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.833+0000 7f3a55600640 1 -- 172.21.15.86:0/2254321857 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5000fc30 con 0x7f3a5815feb0 2024-04-25T14:14:34.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.833+0000 7f3a5f800640 1 -- 172.21.15.86:0/2254321857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a5815d270 con 0x7f3a5815feb0 2024-04-25T14:14:34.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.833+0000 7f3a5f800640 1 -- 172.21.15.86:0/2254321857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a5815d830 con 0x7f3a5815feb0 2024-04-25T14:14:34.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.833+0000 7f3a5f800640 1 -- 172.21.15.86:0/2254321857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a5815cd00 con 0x7f3a5815feb0 2024-04-25T14:14:34.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.837+0000 7f3a55600640 1 -- 172.21.15.86:0/2254321857 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f3a5000e3e0 con 0x7f3a5815feb0 2024-04-25T14:14:34.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.837+0000 7f3a55600640 1 --2- 172.21.15.86:0/2254321857 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f3a3406f5a0 0x7f3a34071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:34.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.837+0000 7f3a55600640 1 -- 172.21.15.86:0/2254321857 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f3a500905b0 con 0x7f3a5815feb0 2024-04-25T14:14:34.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.837+0000 7f3a5ce00640 1 --2- 172.21.15.86:0/2254321857 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f3a3406f5a0 0x7f3a34071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:34.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.837+0000 7f3a5ce00640 1 --2- 172.21.15.86:0/2254321857 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f3a3406f5a0 0x7f3a34071ab0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3a580b4200 tx=0x7f3a48009040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:34.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:34.841+0000 7f3a55600640 1 -- 172.21.15.86:0/2254321857 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3a50014030 con 0x7f3a5815feb0 2024-04-25T14:14:35.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:35.033+0000 7f3a5f800640 1 -- 172.21.15.86:0/2254321857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a5806db40 con 0x7f3a5815feb0 2024-04-25T14:14:35.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:35.033+0000 7f3a55600640 1 -- 172.21.15.86:0/2254321857 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f3a5005cfa0 con 0x7f3a5815feb0 2024-04-25T14:14:35.030 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:35.030 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:14:35.030 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:14:35.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:35.037+0000 7f3a5f800640 1 -- 172.21.15.86:0/2254321857 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f3a3406f5a0 msgr2=0x7f3a34071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:35.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:35.037+0000 7f3a5f800640 1 --2- 172.21.15.86:0/2254321857 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f3a3406f5a0 0x7f3a34071ab0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3a580b4200 tx=0x7f3a48009040 comp rx=0 tx=0).stop 2024-04-25T14:14:35.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:35.037+0000 7f3a5f800640 1 -- 172.21.15.86:0/2254321857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a5815feb0 msgr2=0x7f3a580af720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:35.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:35.037+0000 7f3a5f800640 1 --2- 172.21.15.86:0/2254321857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a5815feb0 0x7f3a580af720 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000cd50 tx=0x7f3a5000a650 comp rx=0 tx=0).stop 2024-04-25T14:14:35.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:35.037+0000 7f3a5f800640 1 -- 172.21.15.86:0/2254321857 shutdown_connections 2024-04-25T14:14:35.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:35.037+0000 7f3a5f800640 1 --2- 172.21.15.86:0/2254321857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a5815feb0 0x7f3a580af720 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:35.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:35.037+0000 7f3a5f800640 1 --2- 172.21.15.86:0/2254321857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a5815c2a0 0x7f3a580af1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:35.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:35.037+0000 7f3a5f800640 1 --2- 172.21.15.86:0/2254321857 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f3a3406f5a0 0x7f3a34071ab0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:35.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:35.037+0000 7f3a5f800640 1 --2- 172.21.15.86:0/2254321857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a5806fd50 0x7f3a58157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:35.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:35.037+0000 7f3a5f800640 1 -- 172.21.15.86:0/2254321857 >> 172.21.15.86:0/2254321857 conn(0x7f3a580b4650 msgr2=0x7f3a58165ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:35.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:35.037+0000 7f3a5f800640 1 -- 172.21.15.86:0/2254321857 shutdown_connections 2024-04-25T14:14:35.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:35.037+0000 7f3a5f800640 1 -- 172.21.15.86:0/2254321857 wait complete. 2024-04-25T14:14:35.455 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:14:35.455 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:14:36.456 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:36.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.613+0000 7fe893c00640 1 Processor -- start 2024-04-25T14:14:36.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.613+0000 7fe893c00640 1 -- start start 2024-04-25T14:14:36.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.613+0000 7fe893c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe88c0b4be0 0x7fe88c0b7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:36.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.613+0000 7fe893c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe88c0b76c0 0x7fe88c0d4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:36.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.613+0000 7fe893c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe88c0d45c0 0x7fe88c0d49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:36.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.613+0000 7fe893c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe88c0b7ad0 con 0x7fe88c0b76c0 2024-04-25T14:14:36.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.613+0000 7fe893c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe88c0d06b0 con 0x7fe88c0b4be0 2024-04-25T14:14:36.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.613+0000 7fe893c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe88c160e10 con 0x7fe88c0d45c0 2024-04-25T14:14:36.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.613+0000 7fe88be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe88c0b76c0 0x7fe88c0d4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:36.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.613+0000 7fe88be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe88c0b76c0 0x7fe88c0d4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54534/0 (socket says 172.21.15.86:54534) 2024-04-25T14:14:36.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.613+0000 7fe88be00640 1 -- 172.21.15.86:0/3183759856 learned_addr learned my addr 172.21.15.86:0/3183759856 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:36.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.613+0000 7fe88be00640 1 -- 172.21.15.86:0/3183759856 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe88c0d45c0 msgr2=0x7fe88c0d49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:36.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.613+0000 7fe891200640 1 --2- 172.21.15.86:0/3183759856 >> v2:172.21.15.155:3300/0 conn(0x7fe88c0b4be0 0x7fe88c0b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:36.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.613+0000 7fe891c00640 1 --2- 172.21.15.86:0/3183759856 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe88c0d45c0 0x7fe88c0d49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:36.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.617+0000 7fe88be00640 1 --2- 172.21.15.86:0/3183759856 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe88c0d45c0 0x7fe88c0d49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:36.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.617+0000 7fe88be00640 1 -- 172.21.15.86:0/3183759856 >> v2:172.21.15.155:3300/0 conn(0x7fe88c0b4be0 msgr2=0x7fe88c0b7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:36.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.617+0000 7fe88be00640 1 --2- 172.21.15.86:0/3183759856 >> v2:172.21.15.155:3300/0 conn(0x7fe88c0b4be0 0x7fe88c0b7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:36.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.617+0000 7fe88be00640 1 -- 172.21.15.86:0/3183759856 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88c1638a0 con 0x7fe88c0b76c0 2024-04-25T14:14:36.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.617+0000 7fe891200640 1 --2- 172.21.15.86:0/3183759856 >> v2:172.21.15.155:3300/0 conn(0x7fe88c0b4be0 0x7fe88c0b7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:14:36.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.617+0000 7fe891c00640 1 --2- 172.21.15.86:0/3183759856 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe88c0d45c0 0x7fe88c0d49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:14:36.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.617+0000 7fe88be00640 1 --2- 172.21.15.86:0/3183759856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe88c0b76c0 0x7fe88c0d4080 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fe88c08e100 tx=0x7fe87801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e3775ed9d99ac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:36.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.617+0000 7fe88b400640 1 -- 172.21.15.86:0/3183759856 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8780ab5f0 con 0x7fe88c0b76c0 2024-04-25T14:14:36.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.617+0000 7fe88b400640 1 -- 172.21.15.86:0/3183759856 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8780b0070 con 0x7fe88c0b76c0 2024-04-25T14:14:36.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.617+0000 7fe88b400640 1 -- 172.21.15.86:0/3183759856 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8780ab9b0 con 0x7fe88c0b76c0 2024-04-25T14:14:36.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.617+0000 7fe893c00640 1 -- 172.21.15.86:0/3183759856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe88c0b76c0 msgr2=0x7fe88c0d4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:36.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.617+0000 7fe893c00640 1 --2- 172.21.15.86:0/3183759856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe88c0b76c0 0x7fe88c0d4080 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fe88c08e100 tx=0x7fe87801c520 comp rx=0 tx=0).stop 2024-04-25T14:14:36.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.621+0000 7fe893c00640 1 -- 172.21.15.86:0/3183759856 shutdown_connections 2024-04-25T14:14:36.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.621+0000 7fe893c00640 1 --2- 172.21.15.86:0/3183759856 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe88c0d45c0 0x7fe88c0d49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:36.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.621+0000 7fe893c00640 1 --2- 172.21.15.86:0/3183759856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe88c0b76c0 0x7fe88c0d4080 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:36.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.621+0000 7fe893c00640 1 --2- 172.21.15.86:0/3183759856 >> v2:172.21.15.155:3300/0 conn(0x7fe88c0b4be0 0x7fe88c0b7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:36.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.621+0000 7fe893c00640 1 -- 172.21.15.86:0/3183759856 >> 172.21.15.86:0/3183759856 conn(0x7fe88c0d2180 msgr2=0x7fe88c0d2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:36.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.621+0000 7fe893c00640 1 -- 172.21.15.86:0/3183759856 shutdown_connections 2024-04-25T14:14:36.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.621+0000 7fe893c00640 1 -- 172.21.15.86:0/3183759856 wait complete. 2024-04-25T14:14:36.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.621+0000 7fe893c00640 1 Processor -- start 2024-04-25T14:14:36.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.621+0000 7fe893c00640 1 -- start start 2024-04-25T14:14:36.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.621+0000 7fe893c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe88c0b4be0 0x7fe88c165960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:36.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.621+0000 7fe893c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe88c0b76c0 0x7fe88c0cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:36.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.621+0000 7fe893c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe88c0d45c0 0x7fe88c148850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:36.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.621+0000 7fe893c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c06e520 con 0x7fe88c0d45c0 2024-04-25T14:14:36.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.621+0000 7fe893c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c06e3a0 con 0x7fe88c0b76c0 2024-04-25T14:14:36.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.621+0000 7fe893c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe88c06e6a0 con 0x7fe88c0b4be0 2024-04-25T14:14:36.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.625+0000 7fe891c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe88c0d45c0 0x7fe88c148850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:36.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.625+0000 7fe891c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe88c0d45c0 0x7fe88c148850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54546/0 (socket says 172.21.15.86:54546) 2024-04-25T14:14:36.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.625+0000 7fe891c00640 1 -- 172.21.15.86:0/1801606446 learned_addr learned my addr 172.21.15.86:0/1801606446 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:36.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.625+0000 7fe88be00640 1 --2- 172.21.15.86:0/1801606446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe88c0b76c0 0x7fe88c0cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:36.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.625+0000 7fe891200640 1 --2- 172.21.15.86:0/1801606446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe88c0b4be0 0x7fe88c165960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:36.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.625+0000 7fe891c00640 1 -- 172.21.15.86:0/1801606446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe88c0b4be0 msgr2=0x7fe88c165960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:36.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.625+0000 7fe891c00640 1 --2- 172.21.15.86:0/1801606446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe88c0b4be0 0x7fe88c165960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:36.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.625+0000 7fe891c00640 1 -- 172.21.15.86:0/1801606446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe88c0b76c0 msgr2=0x7fe88c0cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:36.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.625+0000 7fe891c00640 1 --2- 172.21.15.86:0/1801606446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe88c0b76c0 0x7fe88c0cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:36.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.625+0000 7fe891c00640 1 -- 172.21.15.86:0/1801606446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88c1661b0 con 0x7fe88c0d45c0 2024-04-25T14:14:36.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.625+0000 7fe891c00640 1 --2- 172.21.15.86:0/1801606446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe88c0d45c0 0x7fe88c148850 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7fe88400c5f0 tx=0x7fe88400cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:36.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.625+0000 7fe888c00640 1 -- 172.21.15.86:0/1801606446 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe884015070 con 0x7fe88c0d45c0 2024-04-25T14:14:36.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.625+0000 7fe888c00640 1 -- 172.21.15.86:0/1801606446 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe884011820 con 0x7fe88c0d45c0 2024-04-25T14:14:36.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.625+0000 7fe888c00640 1 -- 172.21.15.86:0/1801606446 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe884011c10 con 0x7fe88c0d45c0 2024-04-25T14:14:36.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.625+0000 7fe893c00640 1 -- 172.21.15.86:0/1801606446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88c166490 con 0x7fe88c0d45c0 2024-04-25T14:14:36.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.625+0000 7fe893c00640 1 -- 172.21.15.86:0/1801606446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe88c166a50 con 0x7fe88c0d45c0 2024-04-25T14:14:36.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.625+0000 7fe893c00640 1 -- 172.21.15.86:0/1801606446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe88c06ee30 con 0x7fe88c0d45c0 2024-04-25T14:14:36.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.625+0000 7fe888c00640 1 -- 172.21.15.86:0/1801606446 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7fe884018070 con 0x7fe88c0d45c0 2024-04-25T14:14:36.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.629+0000 7fe888c00640 1 --2- 172.21.15.86:0/1801606446 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe86c06f570 0x7fe86c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:36.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.629+0000 7fe888c00640 1 -- 172.21.15.86:0/1801606446 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fe88408f690 con 0x7fe88c0d45c0 2024-04-25T14:14:36.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.629+0000 7fe891200640 1 --2- 172.21.15.86:0/1801606446 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe86c06f570 0x7fe86c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:36.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.629+0000 7fe891200640 1 --2- 172.21.15.86:0/1801606446 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe86c06f570 0x7fe86c071a80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0c90f0 tx=0x7fe87c007740 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:36.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.633+0000 7fe888c00640 1 -- 172.21.15.86:0/1801606446 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe88405ba30 con 0x7fe88c0d45c0 2024-04-25T14:14:36.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.825+0000 7fe893c00640 1 -- 172.21.15.86:0/1801606446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe88c0b80a0 con 0x7fe88c0d45c0 2024-04-25T14:14:36.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.829+0000 7fe888c00640 1 -- 172.21.15.86:0/1801606446 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7fe88405fd30 con 0x7fe88c0d45c0 2024-04-25T14:14:36.824 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:36.825 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:14:36.825 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:14:36.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.829+0000 7fe893c00640 1 -- 172.21.15.86:0/1801606446 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe86c06f570 msgr2=0x7fe86c071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:36.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.829+0000 7fe893c00640 1 --2- 172.21.15.86:0/1801606446 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe86c06f570 0x7fe86c071a80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0c90f0 tx=0x7fe87c007740 comp rx=0 tx=0).stop 2024-04-25T14:14:36.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.829+0000 7fe893c00640 1 -- 172.21.15.86:0/1801606446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe88c0d45c0 msgr2=0x7fe88c148850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:36.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.829+0000 7fe893c00640 1 --2- 172.21.15.86:0/1801606446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe88c0d45c0 0x7fe88c148850 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7fe88400c5f0 tx=0x7fe88400cab0 comp rx=0 tx=0).stop 2024-04-25T14:14:36.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.829+0000 7fe893c00640 1 -- 172.21.15.86:0/1801606446 shutdown_connections 2024-04-25T14:14:36.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.829+0000 7fe893c00640 1 --2- 172.21.15.86:0/1801606446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe88c0d45c0 0x7fe88c148850 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:36.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.829+0000 7fe893c00640 1 --2- 172.21.15.86:0/1801606446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe88c0b76c0 0x7fe88c0cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:36.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.829+0000 7fe893c00640 1 --2- 172.21.15.86:0/1801606446 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe86c06f570 0x7fe86c071a80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:36.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.829+0000 7fe893c00640 1 --2- 172.21.15.86:0/1801606446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe88c0b4be0 0x7fe88c165960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:36.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.829+0000 7fe893c00640 1 -- 172.21.15.86:0/1801606446 >> 172.21.15.86:0/1801606446 conn(0x7fe88c0d2180 msgr2=0x7fe88c15e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:36.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.829+0000 7fe893c00640 1 -- 172.21.15.86:0/1801606446 shutdown_connections 2024-04-25T14:14:36.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:36.829+0000 7fe893c00640 1 -- 172.21.15.86:0/1801606446 wait complete. 2024-04-25T14:14:37.250 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:14:37.250 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:14:38.252 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:38.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.413+0000 7f13c7200640 1 Processor -- start 2024-04-25T14:14:38.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.413+0000 7f13c7200640 1 -- start start 2024-04-25T14:14:38.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.413+0000 7f13c7200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13c00d1df0 0x7f13c00cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:38.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.413+0000 7f13c7200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13c015e5c0 con 0x7f13c00cdcc0 2024-04-25T14:14:38.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.413+0000 7f13c7200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13c015e8a0 con 0x7f13c0070330 2024-04-25T14:14:38.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.413+0000 7f13c7200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f13c006fd40 con 0x7f13c00d1df0 2024-04-25T14:14:38.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13c5200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f13c00cdcc0 0x7f13c00ce0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57974/0 (socket says 172.21.15.86:57974) 2024-04-25T14:14:38.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13c5200640 1 -- 172.21.15.86:0/3018173639 learned_addr learned my addr 172.21.15.86:0/3018173639 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:14:38.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13b7400640 1 --2- 172.21.15.86:0/3018173639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13c00d1df0 0x7f13c00cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:38.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13bf400640 1 -- 172.21.15.86:0/3018173639 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3242255887 0 0) 0x7f13c015e5c0 con 0x7f13c00cdcc0 2024-04-25T14:14:38.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13bf400640 1 -- 172.21.15.86:0/3018173639 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13a4003530 con 0x7f13c00cdcc0 2024-04-25T14:14:38.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13bf400640 1 -- 172.21.15.86:0/3018173639 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1072839024 0 0) 0x7f13c015e8a0 con 0x7f13c0070330 2024-04-25T14:14:38.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13bf400640 1 -- 172.21.15.86:0/3018173639 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13c015e5c0 con 0x7f13c0070330 2024-04-25T14:14:38.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13bf400640 1 -- 172.21.15.86:0/3018173639 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1243701792 0 0) 0x7f13a4003530 con 0x7f13c00cdcc0 2024-04-25T14:14:38.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13bf400640 1 -- 172.21.15.86:0/3018173639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13c00d1df0 msgr2=0x7f13c00cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:38.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13bf400640 1 --2- 172.21.15.86:0/3018173639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13c00d1df0 0x7f13c00cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:38.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13bf400640 1 -- 172.21.15.86:0/3018173639 >> v1:172.21.15.155:6789/0 conn(0x7f13c0070330 legacy=0x7f13c00d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:38.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13bf400640 1 -- 172.21.15.86:0/3018173639 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c015eb80 con 0x7f13c00cdcc0 2024-04-25T14:14:38.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13bf400640 1 -- 172.21.15.86:0/3018173639 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f13b8001a00 con 0x7f13c00cdcc0 2024-04-25T14:14:38.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13bf400640 1 -- 172.21.15.86:0/3018173639 >> v1:172.21.15.86:6789/0 conn(0x7f13c00cdcc0 legacy=0x7f13c00ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:38.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13bf400640 1 --2- 172.21.15.86:0/3018173639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13a4003bf0 0x7f13a4004040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:38.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13bf400640 1 --2- 172.21.15.86:0/3018173639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13a4004810 0x7f13a4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:38.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13bf400640 1 --2- 172.21.15.86:0/3018173639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13a4005160 0x7f13a400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:38.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13bf400640 1 -- 172.21.15.86:0/3018173639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f13a400c010 con 0x7f13a4003bf0 2024-04-25T14:14:38.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13bf400640 1 -- 172.21.15.86:0/3018173639 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f13a400c1c0 con 0x7f13a4005160 2024-04-25T14:14:38.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13bf400640 1 -- 172.21.15.86:0/3018173639 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f13a400c3c0 con 0x7f13a4004810 2024-04-25T14:14:38.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13bfe00640 1 --2- 172.21.15.86:0/3018173639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13a4003bf0 0x7f13a4004040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:38.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13b7400640 1 --2- 172.21.15.86:0/3018173639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13a4004810 0x7f13a4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:38.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13c5200640 1 --2- 172.21.15.86:0/3018173639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13a4005160 0x7f13a400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:38.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13bfe00640 1 -- 172.21.15.86:0/3018173639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13a4004810 msgr2=0x7f13a4004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:38.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13bfe00640 1 --2- 172.21.15.86:0/3018173639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13a4004810 0x7f13a4004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:38.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13bfe00640 1 -- 172.21.15.86:0/3018173639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13a4005160 msgr2=0x7f13a400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:38.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13bfe00640 1 --2- 172.21.15.86:0/3018173639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13a4005160 0x7f13a400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:38.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13bfe00640 1 -- 172.21.15.86:0/3018173639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a4003530 con 0x7f13a4003bf0 2024-04-25T14:14:38.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.417+0000 7f13c5200640 1 --2- 172.21.15.86:0/3018173639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13a4005160 0x7f13a400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:38.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.421+0000 7f13bfe00640 1 --2- 172.21.15.86:0/3018173639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13a4003bf0 0x7f13a4004040 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f13c0072200 tx=0x7f13b001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f5a4606022b5e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:38.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.421+0000 7f13bf400640 1 -- 172.21.15.86:0/3018173639 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0001520 con 0x7f13a4003bf0 2024-04-25T14:14:38.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.421+0000 7f13bf400640 1 -- 172.21.15.86:0/3018173639 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b00052d0 con 0x7f13a4003bf0 2024-04-25T14:14:38.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.421+0000 7f13bf400640 1 -- 172.21.15.86:0/3018173639 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0003840 con 0x7f13a4003bf0 2024-04-25T14:14:38.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.421+0000 7f13c7200640 1 -- 172.21.15.86:0/3018173639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13a4003bf0 msgr2=0x7f13a4004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:38.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.421+0000 7f13c7200640 1 --2- 172.21.15.86:0/3018173639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13a4003bf0 0x7f13a4004040 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f13c0072200 tx=0x7f13b001c530 comp rx=0 tx=0).stop 2024-04-25T14:14:38.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.421+0000 7f13c5200640 1 -- 172.21.15.86:0/3018173639 reap_dead start 2024-04-25T14:14:38.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.421+0000 7f13c7200640 1 -- 172.21.15.86:0/3018173639 shutdown_connections 2024-04-25T14:14:38.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.421+0000 7f13c7200640 1 -- 172.21.15.86:0/3018173639 >> 172.21.15.86:0/3018173639 conn(0x7f13c00b4650 msgr2=0x7f13c00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:38.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.421+0000 7f13c7200640 1 -- 172.21.15.86:0/3018173639 shutdown_connections 2024-04-25T14:14:38.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.421+0000 7f13c7200640 1 -- 172.21.15.86:0/3018173639 wait complete. 2024-04-25T14:14:38.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.421+0000 7f13c7200640 1 Processor -- start 2024-04-25T14:14:38.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.421+0000 7f13c7200640 1 -- start start 2024-04-25T14:14:38.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.425+0000 7f13c7200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13a4003bf0 0x7f13c01f94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:38.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.425+0000 7f13c7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c01581d0 0x7f13c015a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:38.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.425+0000 7f13c7200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13c015ab70 0x7f13c00b1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:38.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.425+0000 7f13c7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f13c00708c0 con 0x7f13c01581d0 2024-04-25T14:14:38.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.425+0000 7f13c7200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f13c0070740 con 0x7f13a4003bf0 2024-04-25T14:14:38.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.425+0000 7f13c7200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f13c00d1df0 con 0x7f13c015ab70 2024-04-25T14:14:38.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.425+0000 7f13b7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c01581d0 0x7f13c015a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:38.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.425+0000 7f13b7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c01581d0 0x7f13c015a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54558/0 (socket says 172.21.15.86:54558) 2024-04-25T14:14:38.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.425+0000 7f13b7400640 1 -- 172.21.15.86:0/4189510942 learned_addr learned my addr 172.21.15.86:0/4189510942 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:38.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.425+0000 7f13bfe00640 1 --2- 172.21.15.86:0/4189510942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13a4003bf0 0x7f13c01f94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:38.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.425+0000 7f13c5200640 1 --2- 172.21.15.86:0/4189510942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13c015ab70 0x7f13c00b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:38.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.425+0000 7f13b7400640 1 -- 172.21.15.86:0/4189510942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13c015ab70 msgr2=0x7f13c00b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:38.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.425+0000 7f13b7400640 1 --2- 172.21.15.86:0/4189510942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13c015ab70 0x7f13c00b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:38.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.425+0000 7f13b7400640 1 -- 172.21.15.86:0/4189510942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13a4003bf0 msgr2=0x7f13c01f94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:38.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.425+0000 7f13b7400640 1 --2- 172.21.15.86:0/4189510942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13a4003bf0 0x7f13c01f94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:38.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.425+0000 7f13b7400640 1 -- 172.21.15.86:0/4189510942 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c00b3d40 con 0x7f13c01581d0 2024-04-25T14:14:38.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.425+0000 7f13b7400640 1 --2- 172.21.15.86:0/4189510942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c01581d0 0x7f13c015a630 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f13a8011a00 tx=0x7f13a8011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:38.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.425+0000 7f13bcc00640 1 -- 172.21.15.86:0/4189510942 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a800fb50 con 0x7f13c01581d0 2024-04-25T14:14:38.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.425+0000 7f13bcc00640 1 -- 172.21.15.86:0/4189510942 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13a800fcf0 con 0x7f13c01581d0 2024-04-25T14:14:38.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.425+0000 7f13c7200640 1 -- 172.21.15.86:0/4189510942 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c0159870 con 0x7f13c01581d0 2024-04-25T14:14:38.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.425+0000 7f13bcc00640 1 -- 172.21.15.86:0/4189510942 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a800fea0 con 0x7f13c01581d0 2024-04-25T14:14:38.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.425+0000 7f13c7200640 1 -- 172.21.15.86:0/4189510942 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c0159e30 con 0x7f13c01581d0 2024-04-25T14:14:38.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.425+0000 7f13c7200640 1 -- 172.21.15.86:0/4189510942 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13c00b3ab0 con 0x7f13c01581d0 2024-04-25T14:14:38.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.429+0000 7f13bcc00640 1 -- 172.21.15.86:0/4189510942 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f13a8012040 con 0x7f13c01581d0 2024-04-25T14:14:38.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.429+0000 7f13bcc00640 1 --2- 172.21.15.86:0/4189510942 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f139c06f5a0 0x7f139c071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:38.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.429+0000 7f13bcc00640 1 -- 172.21.15.86:0/4189510942 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f13a808fea0 con 0x7f13c01581d0 2024-04-25T14:14:38.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.429+0000 7f13bfe00640 1 --2- 172.21.15.86:0/4189510942 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f139c06f5a0 0x7f139c071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:38.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.429+0000 7f13bfe00640 1 --2- 172.21.15.86:0/4189510942 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f139c06f5a0 0x7f139c071ab0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f13b00ac000 tx=0x7f13b00a6000 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:38.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.433+0000 7f13bcc00640 1 -- 172.21.15.86:0/4189510942 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f13a8019030 con 0x7f13c01581d0 2024-04-25T14:14:38.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.625+0000 7f13c7200640 1 -- 172.21.15.86:0/4189510942 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13c0069d00 con 0x7f13c01581d0 2024-04-25T14:14:38.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.625+0000 7f13bcc00640 1 -- 172.21.15.86:0/4189510942 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f13a805ca50 con 0x7f13c01581d0 2024-04-25T14:14:38.621 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:38.621 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:14:38.621 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:14:38.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.625+0000 7f13c7200640 1 -- 172.21.15.86:0/4189510942 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f139c06f5a0 msgr2=0x7f139c071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:38.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.625+0000 7f13c7200640 1 --2- 172.21.15.86:0/4189510942 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f139c06f5a0 0x7f139c071ab0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f13b00ac000 tx=0x7f13b00a6000 comp rx=0 tx=0).stop 2024-04-25T14:14:38.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.625+0000 7f13c7200640 1 -- 172.21.15.86:0/4189510942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c01581d0 msgr2=0x7f13c015a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:38.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.625+0000 7f13c7200640 1 --2- 172.21.15.86:0/4189510942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c01581d0 0x7f13c015a630 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f13a8011a00 tx=0x7f13a8011ec0 comp rx=0 tx=0).stop 2024-04-25T14:14:38.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.625+0000 7f13c7200640 1 -- 172.21.15.86:0/4189510942 shutdown_connections 2024-04-25T14:14:38.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.625+0000 7f13c7200640 1 --2- 172.21.15.86:0/4189510942 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f139c06f5a0 0x7f139c071ab0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:38.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.625+0000 7f13c7200640 1 --2- 172.21.15.86:0/4189510942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13c015ab70 0x7f13c00b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:38.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.625+0000 7f13c7200640 1 --2- 172.21.15.86:0/4189510942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c01581d0 0x7f13c015a630 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:38.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.625+0000 7f13c7200640 1 --2- 172.21.15.86:0/4189510942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13a4003bf0 0x7f13c01f94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:38.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.625+0000 7f13c7200640 1 -- 172.21.15.86:0/4189510942 >> 172.21.15.86:0/4189510942 conn(0x7f13c00b4650 msgr2=0x7f13c00ae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:38.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.625+0000 7f13c7200640 1 -- 172.21.15.86:0/4189510942 shutdown_connections 2024-04-25T14:14:38.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:38.625+0000 7f13c7200640 1 -- 172.21.15.86:0/4189510942 wait complete. 2024-04-25T14:14:39.045 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:14:39.045 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:14:40.046 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:40.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.209+0000 7f1d6f200640 1 Processor -- start 2024-04-25T14:14:40.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.209+0000 7f1d6f200640 1 -- start start 2024-04-25T14:14:40.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.209+0000 7f1d6f200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d680cf800 0x7f1d680b2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:40.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.209+0000 7f1d6f200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d680b37d0 con 0x7f1d680d1a00 2024-04-25T14:14:40.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.209+0000 7f1d6f200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d680cfce0 con 0x7f1d680b7ad0 2024-04-25T14:14:40.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.209+0000 7f1d6f200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1d680b3a70 con 0x7f1d680cf800 2024-04-25T14:14:40.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.209+0000 7f1d6d200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1d680d1a00 0x7f1d680d1e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60414/0 (socket says 172.21.15.86:60414) 2024-04-25T14:14:40.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.209+0000 7f1d6d200640 1 -- 172.21.15.86:0/4006786599 learned_addr learned my addr 172.21.15.86:0/4006786599 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:14:40.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.209+0000 7f1d67400640 1 --2- 172.21.15.86:0/4006786599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d680cf800 0x7f1d680b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:40.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.209+0000 7f1d66a00640 1 -- 172.21.15.86:0/4006786599 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3181935114 0 0) 0x7f1d680b37d0 con 0x7f1d680d1a00 2024-04-25T14:14:40.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.209+0000 7f1d66a00640 1 -- 172.21.15.86:0/4006786599 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d4c003530 con 0x7f1d680d1a00 2024-04-25T14:14:40.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.209+0000 7f1d66a00640 1 -- 172.21.15.86:0/4006786599 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 654566220 0 0) 0x7f1d4c003530 con 0x7f1d680d1a00 2024-04-25T14:14:40.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.209+0000 7f1d66a00640 1 -- 172.21.15.86:0/4006786599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d680cf800 msgr2=0x7f1d680b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:40.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.209+0000 7f1d66a00640 1 --2- 172.21.15.86:0/4006786599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d680cf800 0x7f1d680b2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:40.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.209+0000 7f1d66a00640 1 -- 172.21.15.86:0/4006786599 >> v1:172.21.15.155:6789/0 conn(0x7f1d680b7ad0 legacy=0x7f1d680b1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:40.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.209+0000 7f1d66a00640 1 -- 172.21.15.86:0/4006786599 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d680b3d50 con 0x7f1d680d1a00 2024-04-25T14:14:40.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.209+0000 7f1d66a00640 1 -- 172.21.15.86:0/4006786599 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f1d60001680 con 0x7f1d680d1a00 2024-04-25T14:14:40.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.209+0000 7f1d66a00640 1 -- 172.21.15.86:0/4006786599 >> v1:172.21.15.86:6789/0 conn(0x7f1d680d1a00 legacy=0x7f1d680d1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:40.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.213+0000 7f1d66a00640 1 --2- 172.21.15.86:0/4006786599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d680cf800 0x7f1d4c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:40.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.213+0000 7f1d66a00640 1 --2- 172.21.15.86:0/4006786599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c003fc0 0x7f1d4c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:40.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.213+0000 7f1d66a00640 1 --2- 172.21.15.86:0/4006786599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d4c004d50 0x7f1d4c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:40.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.213+0000 7f1d66a00640 1 -- 172.21.15.86:0/4006786599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1d680b3a70 con 0x7f1d4c003fc0 2024-04-25T14:14:40.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.213+0000 7f1d66a00640 1 -- 172.21.15.86:0/4006786599 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c00bc10 con 0x7f1d4c004d50 2024-04-25T14:14:40.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.213+0000 7f1d66a00640 1 -- 172.21.15.86:0/4006786599 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1d4c00be10 con 0x7f1d680cf800 2024-04-25T14:14:40.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.213+0000 7f1d67400640 1 --2- 172.21.15.86:0/4006786599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c003fc0 0x7f1d4c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:40.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.213+0000 7f1d6d200640 1 --2- 172.21.15.86:0/4006786599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d4c004d50 0x7f1d4c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:40.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.213+0000 7f1d67e00640 1 --2- 172.21.15.86:0/4006786599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d680cf800 0x7f1d4c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:40.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.213+0000 7f1d67400640 1 -- 172.21.15.86:0/4006786599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d680cf800 msgr2=0x7f1d4c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:40.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.213+0000 7f1d67400640 1 --2- 172.21.15.86:0/4006786599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d680cf800 0x7f1d4c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:40.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.213+0000 7f1d67400640 1 -- 172.21.15.86:0/4006786599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d4c004d50 msgr2=0x7f1d4c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:40.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.213+0000 7f1d67400640 1 --2- 172.21.15.86:0/4006786599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d4c004d50 0x7f1d4c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:40.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.213+0000 7f1d67400640 1 -- 172.21.15.86:0/4006786599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4c003530 con 0x7f1d4c003fc0 2024-04-25T14:14:40.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.213+0000 7f1d67400640 1 --2- 172.21.15.86:0/4006786599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c003fc0 0x7f1d4c004810 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009ae0 tx=0x7f1d5401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d9e24e4dcd1587 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:40.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.213+0000 7f1d66a00640 1 -- 172.21.15.86:0/4006786599 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d540b2070 con 0x7f1d4c003fc0 2024-04-25T14:14:40.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.213+0000 7f1d66a00640 1 -- 172.21.15.86:0/4006786599 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d540a5680 con 0x7f1d4c003fc0 2024-04-25T14:14:40.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.213+0000 7f1d66a00640 1 -- 172.21.15.86:0/4006786599 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d540096b0 con 0x7f1d4c003fc0 2024-04-25T14:14:40.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.217+0000 7f1d6f200640 1 -- 172.21.15.86:0/4006786599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c003fc0 msgr2=0x7f1d4c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:40.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.217+0000 7f1d6f200640 1 --2- 172.21.15.86:0/4006786599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c003fc0 0x7f1d4c004810 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009ae0 tx=0x7f1d5401ca90 comp rx=0 tx=0).stop 2024-04-25T14:14:40.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.217+0000 7f1d6d200640 1 -- 172.21.15.86:0/4006786599 reap_dead start 2024-04-25T14:14:40.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.217+0000 7f1d6f200640 1 -- 172.21.15.86:0/4006786599 shutdown_connections 2024-04-25T14:14:40.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.217+0000 7f1d6f200640 1 -- 172.21.15.86:0/4006786599 >> 172.21.15.86:0/4006786599 conn(0x7f1d6809ba30 msgr2=0x7f1d680b73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:40.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.217+0000 7f1d6f200640 1 -- 172.21.15.86:0/4006786599 shutdown_connections 2024-04-25T14:14:40.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.217+0000 7f1d6f200640 1 -- 172.21.15.86:0/4006786599 wait complete. 2024-04-25T14:14:40.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.217+0000 7f1d6f200640 1 Processor -- start 2024-04-25T14:14:40.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.217+0000 7f1d6f200640 1 -- start start 2024-04-25T14:14:40.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.221+0000 7f1d6f200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d4c003fc0 0x7f1d680cad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:40.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.221+0000 7f1d6f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d680c86f0 0x7f1d6815ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:40.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.221+0000 7f1d6f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d6809d060 0x7f1d6809d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:40.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.221+0000 7f1d6f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1d680b2410 con 0x7f1d680c86f0 2024-04-25T14:14:40.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.221+0000 7f1d6f200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1d680b80f0 con 0x7f1d6809d060 2024-04-25T14:14:40.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.221+0000 7f1d6f200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1d680b2590 con 0x7f1d4c003fc0 2024-04-25T14:14:40.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.221+0000 7f1d67400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d680c86f0 0x7f1d6815ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:40.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.221+0000 7f1d67400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d680c86f0 0x7f1d6815ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53498/0 (socket says 172.21.15.86:53498) 2024-04-25T14:14:40.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.221+0000 7f1d67400640 1 -- 172.21.15.86:0/2322965513 learned_addr learned my addr 172.21.15.86:0/2322965513 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:40.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.221+0000 7f1d67e00640 1 --2- 172.21.15.86:0/2322965513 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d4c003fc0 0x7f1d680cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:40.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.221+0000 7f1d6d200640 1 --2- 172.21.15.86:0/2322965513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d6809d060 0x7f1d6809d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:40.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.221+0000 7f1d67400640 1 -- 172.21.15.86:0/2322965513 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d4c003fc0 msgr2=0x7f1d680cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:40.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.221+0000 7f1d67400640 1 --2- 172.21.15.86:0/2322965513 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d4c003fc0 0x7f1d680cad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:40.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.221+0000 7f1d67400640 1 -- 172.21.15.86:0/2322965513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d6809d060 msgr2=0x7f1d6809d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:40.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.221+0000 7f1d67400640 1 --2- 172.21.15.86:0/2322965513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d6809d060 0x7f1d6809d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:40.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.221+0000 7f1d67400640 1 -- 172.21.15.86:0/2322965513 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d680cb290 con 0x7f1d680c86f0 2024-04-25T14:14:40.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.221+0000 7f1d67e00640 1 --2- 172.21.15.86:0/2322965513 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d4c003fc0 0x7f1d680cad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:40.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.221+0000 7f1d67400640 1 --2- 172.21.15.86:0/2322965513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d680c86f0 0x7f1d6815ea30 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f1d5401c8c0 tx=0x7f1d4c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:40.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.221+0000 7f1d5fe00640 1 -- 172.21.15.86:0/2322965513 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d540b2040 con 0x7f1d680c86f0 2024-04-25T14:14:40.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.221+0000 7f1d5fe00640 1 -- 172.21.15.86:0/2322965513 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d5400f000 con 0x7f1d680c86f0 2024-04-25T14:14:40.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.221+0000 7f1d6f200640 1 -- 172.21.15.86:0/2322965513 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d680a0420 con 0x7f1d680c86f0 2024-04-25T14:14:40.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.221+0000 7f1d5fe00640 1 -- 172.21.15.86:0/2322965513 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54004260 con 0x7f1d680c86f0 2024-04-25T14:14:40.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.221+0000 7f1d6f200640 1 -- 172.21.15.86:0/2322965513 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d680c9f90 con 0x7f1d680c86f0 2024-04-25T14:14:40.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.221+0000 7f1d6f200640 1 -- 172.21.15.86:0/2322965513 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d680c9a20 con 0x7f1d680c86f0 2024-04-25T14:14:40.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.225+0000 7f1d5fe00640 1 -- 172.21.15.86:0/2322965513 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f1d540c0030 con 0x7f1d680c86f0 2024-04-25T14:14:40.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.225+0000 7f1d5fe00640 1 --2- 172.21.15.86:0/2322965513 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1d4806f5a0 0x7f1d48071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:40.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.225+0000 7f1d5fe00640 1 -- 172.21.15.86:0/2322965513 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f1d541296e0 con 0x7f1d680c86f0 2024-04-25T14:14:40.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.225+0000 7f1d67e00640 1 --2- 172.21.15.86:0/2322965513 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1d4806f5a0 0x7f1d48071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:40.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.225+0000 7f1d67e00640 1 --2- 172.21.15.86:0/2322965513 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1d4806f5a0 0x7f1d48071ab0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800afd0 tx=0x7f1d5800aba0 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:40.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.229+0000 7f1d5fe00640 1 -- 172.21.15.86:0/2322965513 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1d540c7060 con 0x7f1d680c86f0 2024-04-25T14:14:40.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.417+0000 7f1d6f200640 1 -- 172.21.15.86:0/2322965513 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d68076bf0 con 0x7f1d680c86f0 2024-04-25T14:14:40.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.421+0000 7f1d5fe00640 1 -- 172.21.15.86:0/2322965513 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f1d540167a0 con 0x7f1d680c86f0 2024-04-25T14:14:40.416 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:40.416 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:14:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:14:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.421+0000 7f1d6f200640 1 -- 172.21.15.86:0/2322965513 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1d4806f5a0 msgr2=0x7f1d48071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.421+0000 7f1d6f200640 1 --2- 172.21.15.86:0/2322965513 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1d4806f5a0 0x7f1d48071ab0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800afd0 tx=0x7f1d5800aba0 comp rx=0 tx=0).stop 2024-04-25T14:14:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.421+0000 7f1d6f200640 1 -- 172.21.15.86:0/2322965513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d680c86f0 msgr2=0x7f1d6815ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.421+0000 7f1d6f200640 1 --2- 172.21.15.86:0/2322965513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d680c86f0 0x7f1d6815ea30 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f1d5401c8c0 tx=0x7f1d4c005840 comp rx=0 tx=0).stop 2024-04-25T14:14:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.421+0000 7f1d6f200640 1 -- 172.21.15.86:0/2322965513 shutdown_connections 2024-04-25T14:14:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.421+0000 7f1d6f200640 1 --2- 172.21.15.86:0/2322965513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d6809d060 0x7f1d6809d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.421+0000 7f1d6f200640 1 --2- 172.21.15.86:0/2322965513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d680c86f0 0x7f1d6815ea30 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.421+0000 7f1d6f200640 1 --2- 172.21.15.86:0/2322965513 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1d4806f5a0 0x7f1d48071ab0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.421+0000 7f1d6f200640 1 --2- 172.21.15.86:0/2322965513 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d4c003fc0 0x7f1d680cad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.421+0000 7f1d6f200640 1 -- 172.21.15.86:0/2322965513 >> 172.21.15.86:0/2322965513 conn(0x7f1d6809ba30 msgr2=0x7f1d680b48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.421+0000 7f1d6f200640 1 -- 172.21.15.86:0/2322965513 shutdown_connections 2024-04-25T14:14:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:40.421+0000 7f1d6f200640 1 -- 172.21.15.86:0/2322965513 wait complete. 2024-04-25T14:14:40.842 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:14:40.842 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:14:41.843 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:42.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.009+0000 7f0089000640 1 Processor -- start 2024-04-25T14:14:42.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.009+0000 7f0089000640 1 -- start start 2024-04-25T14:14:42.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.009+0000 7f0089000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f00840722a0 0x7f00840726b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:42.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.009+0000 7f0089000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f00841614c0 0x7f00841618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:42.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.009+0000 7f0089000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0084072bf0 con 0x7f00840b6ee0 2024-04-25T14:14:42.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.009+0000 7f0089000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f008415df00 con 0x7f00840722a0 2024-04-25T14:14:42.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.009+0000 7f0089000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f008415e1e0 con 0x7f00841614c0 2024-04-25T14:14:42.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.009+0000 7f0082a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f00840722a0 0x7f00840726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:42.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.009+0000 7f0083400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f00840b6ee0 0x7f00840d06c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60426/0 (socket says 172.21.15.86:60426) 2024-04-25T14:14:42.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.009+0000 7f0083400640 1 -- 172.21.15.86:0/2701348463 learned_addr learned my addr 172.21.15.86:0/2701348463 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:14:42.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.009+0000 7f0083e00640 1 --2- 172.21.15.86:0/2701348463 >> v2:172.21.15.155:3300/0 conn(0x7f00841614c0 0x7f00841618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:42.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.013+0000 7f0082a00640 1 -- 172.21.15.86:0/2701348463 >> v2:172.21.15.155:3300/0 conn(0x7f00841614c0 msgr2=0x7f00841618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:42.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.013+0000 7f0082a00640 1 --2- 172.21.15.86:0/2701348463 >> v2:172.21.15.155:3300/0 conn(0x7f00841614c0 0x7f00841618d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:42.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.013+0000 7f0082a00640 1 -- 172.21.15.86:0/2701348463 >> v1:172.21.15.86:6789/0 conn(0x7f00840b6ee0 legacy=0x7f00840d06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:42.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.013+0000 7f0082a00640 1 -- 172.21.15.86:0/2701348463 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f008415e4c0 con 0x7f00840722a0 2024-04-25T14:14:42.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.013+0000 7f0082000640 1 -- 172.21.15.86:0/2701348463 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3796323336 0 0) 0x7f0074001120 con 0x7f00840b6ee0 2024-04-25T14:14:42.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.013+0000 7f0082a00640 1 --2- 172.21.15.86:0/2701348463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00840722a0 0x7f00840726b0 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f00700097c0 tx=0x7f007001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445ccbbe3f166ba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:42.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.013+0000 7f0082000640 1 -- 172.21.15.86:0/2701348463 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00700b0070 con 0x7f00840722a0 2024-04-25T14:14:42.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.013+0000 7f0082000640 1 -- 172.21.15.86:0/2701348463 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00700ab600 con 0x7f00840722a0 2024-04-25T14:14:42.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.013+0000 7f0082000640 1 -- 172.21.15.86:0/2701348463 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00700a4e60 con 0x7f00840722a0 2024-04-25T14:14:42.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.013+0000 7f0089000640 1 -- 172.21.15.86:0/2701348463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00840722a0 msgr2=0x7f00840726b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:42.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.017+0000 7f0089000640 1 --2- 172.21.15.86:0/2701348463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00840722a0 0x7f00840726b0 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f00700097c0 tx=0x7f007001c3d0 comp rx=0 tx=0).stop 2024-04-25T14:14:42.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.017+0000 7f0089000640 1 -- 172.21.15.86:0/2701348463 shutdown_connections 2024-04-25T14:14:42.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.017+0000 7f0089000640 1 --2- 172.21.15.86:0/2701348463 >> v2:172.21.15.155:3300/0 conn(0x7f00841614c0 0x7f00841618d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:42.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.017+0000 7f0089000640 1 --2- 172.21.15.86:0/2701348463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00840722a0 0x7f00840726b0 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:42.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.017+0000 7f0089000640 1 -- 172.21.15.86:0/2701348463 >> 172.21.15.86:0/2701348463 conn(0x7f00840b4f80 msgr2=0x7f00840b5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:42.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.017+0000 7f0089000640 1 -- 172.21.15.86:0/2701348463 shutdown_connections 2024-04-25T14:14:42.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.017+0000 7f0089000640 1 -- 172.21.15.86:0/2701348463 wait complete. 2024-04-25T14:14:42.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.017+0000 7f0089000640 1 Processor -- start 2024-04-25T14:14:42.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.017+0000 7f0089000640 1 -- start start 2024-04-25T14:14:42.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.017+0000 7f0089000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00840722a0 0x7f0084159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:42.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.017+0000 7f0089000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00840b6ee0 0x7f00840b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:42.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.017+0000 7f0089000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00841614c0 0x7f00840b4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:42.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.017+0000 7f0089000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f00840cfd40 con 0x7f00840722a0 2024-04-25T14:14:42.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.017+0000 7f0089000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0084162220 con 0x7f00841614c0 2024-04-25T14:14:42.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.017+0000 7f0089000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f00840cfec0 con 0x7f00840b6ee0 2024-04-25T14:14:42.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.021+0000 7f0083400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00840722a0 0x7f0084159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:42.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.021+0000 7f0083400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00840722a0 0x7f0084159e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53508/0 (socket says 172.21.15.86:53508) 2024-04-25T14:14:42.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.021+0000 7f0083400640 1 -- 172.21.15.86:0/3032446555 learned_addr learned my addr 172.21.15.86:0/3032446555 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:42.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.021+0000 7f0083e00640 1 --2- 172.21.15.86:0/3032446555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00841614c0 0x7f00840b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:42.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.021+0000 7f0082a00640 1 --2- 172.21.15.86:0/3032446555 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00840b6ee0 0x7f00840b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:42.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.021+0000 7f0083400640 1 -- 172.21.15.86:0/3032446555 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00840b6ee0 msgr2=0x7f00840b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:42.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.021+0000 7f0083400640 1 --2- 172.21.15.86:0/3032446555 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00840b6ee0 0x7f00840b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:42.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.021+0000 7f0083400640 1 -- 172.21.15.86:0/3032446555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00841614c0 msgr2=0x7f00840b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:42.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.021+0000 7f0083400640 1 --2- 172.21.15.86:0/3032446555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00841614c0 0x7f00840b4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:42.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.021+0000 7f0083400640 1 -- 172.21.15.86:0/3032446555 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0084141680 con 0x7f00840722a0 2024-04-25T14:14:42.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.021+0000 7f0083400640 1 --2- 172.21.15.86:0/3032446555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00840722a0 0x7f0084159e90 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f007400c490 tx=0x7f007400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:42.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.021+0000 7f007b400640 1 -- 172.21.15.86:0/3032446555 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0074017070 con 0x7f00840722a0 2024-04-25T14:14:42.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.021+0000 7f007b400640 1 -- 172.21.15.86:0/3032446555 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0074005cf0 con 0x7f00840722a0 2024-04-25T14:14:42.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.021+0000 7f0089000640 1 -- 172.21.15.86:0/3032446555 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0084141960 con 0x7f00840722a0 2024-04-25T14:14:42.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.021+0000 7f007b400640 1 -- 172.21.15.86:0/3032446555 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00740072f0 con 0x7f00840722a0 2024-04-25T14:14:42.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.021+0000 7f0089000640 1 -- 172.21.15.86:0/3032446555 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0084142440 con 0x7f00840722a0 2024-04-25T14:14:42.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.021+0000 7f0089000640 1 -- 172.21.15.86:0/3032446555 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0084076b80 con 0x7f00840722a0 2024-04-25T14:14:42.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.021+0000 7f007b400640 1 -- 172.21.15.86:0/3032446555 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f007401a070 con 0x7f00840722a0 2024-04-25T14:14:42.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.025+0000 7f007b400640 1 --2- 172.21.15.86:0/3032446555 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f006406f570 0x7f0064071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:42.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.025+0000 7f007b400640 1 -- 172.21.15.86:0/3032446555 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f007408eaa0 con 0x7f00840722a0 2024-04-25T14:14:42.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.025+0000 7f0082a00640 1 --2- 172.21.15.86:0/3032446555 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f006406f570 0x7f0064071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:42.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.025+0000 7f0082a00640 1 --2- 172.21.15.86:0/3032446555 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f006406f570 0x7f0064071a80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f00700ad000 tx=0x7f00700a6000 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:42.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.029+0000 7f007b400640 1 -- 172.21.15.86:0/3032446555 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0084072bf0 con 0x7f00840722a0 2024-04-25T14:14:42.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.217+0000 7f0089000640 1 -- 172.21.15.86:0/3032446555 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0084073330 con 0x7f00840722a0 2024-04-25T14:14:42.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.217+0000 7f007b400640 1 -- 172.21.15.86:0/3032446555 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f007405af50 con 0x7f00840722a0 2024-04-25T14:14:42.215 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:42.216 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:14:42.216 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:14:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.221+0000 7f0089000640 1 -- 172.21.15.86:0/3032446555 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f006406f570 msgr2=0x7f0064071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.221+0000 7f0089000640 1 --2- 172.21.15.86:0/3032446555 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f006406f570 0x7f0064071a80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f00700ad000 tx=0x7f00700a6000 comp rx=0 tx=0).stop 2024-04-25T14:14:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.221+0000 7f0089000640 1 -- 172.21.15.86:0/3032446555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00840722a0 msgr2=0x7f0084159e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.221+0000 7f0089000640 1 --2- 172.21.15.86:0/3032446555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00840722a0 0x7f0084159e90 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f007400c490 tx=0x7f007400c950 comp rx=0 tx=0).stop 2024-04-25T14:14:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.221+0000 7f0089000640 1 -- 172.21.15.86:0/3032446555 shutdown_connections 2024-04-25T14:14:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.221+0000 7f0089000640 1 --2- 172.21.15.86:0/3032446555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00841614c0 0x7f00840b4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.221+0000 7f0089000640 1 --2- 172.21.15.86:0/3032446555 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f006406f570 0x7f0064071a80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.221+0000 7f0089000640 1 --2- 172.21.15.86:0/3032446555 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00840b6ee0 0x7f00840b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.221+0000 7f0089000640 1 --2- 172.21.15.86:0/3032446555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00840722a0 0x7f0084159e90 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.221+0000 7f0089000640 1 -- 172.21.15.86:0/3032446555 >> 172.21.15.86:0/3032446555 conn(0x7f00840b4f80 msgr2=0x7f008415d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.221+0000 7f0089000640 1 -- 172.21.15.86:0/3032446555 shutdown_connections 2024-04-25T14:14:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:42.221+0000 7f0089000640 1 -- 172.21.15.86:0/3032446555 wait complete. 2024-04-25T14:14:42.642 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:14:42.642 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:14:43.643 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:43.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.805+0000 7fdda3c00640 1 Processor -- start 2024-04-25T14:14:43.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.805+0000 7fdda3c00640 1 -- start start 2024-04-25T14:14:43.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.809+0000 7fdda3c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdd9c070330 0x7fdd9c0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:43.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.809+0000 7fdda3c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdd9c0d1c20 0x7fdd9c06ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:43.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.809+0000 7fdda3c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd9c06f270 0x7fdd9c06f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:43.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.809+0000 7fdda3c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdd9c0d2030 con 0x7fdd9c0d1c20 2024-04-25T14:14:43.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.809+0000 7fdda3c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fdd9c15e5c0 con 0x7fdd9c070330 2024-04-25T14:14:43.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.809+0000 7fdda3c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdd9c15e8a0 con 0x7fdd9c06f270 2024-04-25T14:14:43.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.809+0000 7fdd9be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdd9c0d1c20 0x7fdd9c06ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:43.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.809+0000 7fdd9be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdd9c0d1c20 0x7fdd9c06ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53522/0 (socket says 172.21.15.86:53522) 2024-04-25T14:14:43.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.809+0000 7fdd9be00640 1 -- 172.21.15.86:0/2235141849 learned_addr learned my addr 172.21.15.86:0/2235141849 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:43.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.809+0000 7fdda1200640 1 --2- 172.21.15.86:0/2235141849 >> v2:172.21.15.155:3300/0 conn(0x7fdd9c070330 0x7fdd9c0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:43.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.809+0000 7fdda1c00640 1 --2- 172.21.15.86:0/2235141849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd9c06f270 0x7fdd9c06f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:43.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.809+0000 7fdd9be00640 1 -- 172.21.15.86:0/2235141849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd9c06f270 msgr2=0x7fdd9c06f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:43.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.809+0000 7fdd9be00640 1 --2- 172.21.15.86:0/2235141849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd9c06f270 0x7fdd9c06f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:43.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.809+0000 7fdd9be00640 1 -- 172.21.15.86:0/2235141849 >> v2:172.21.15.155:3300/0 conn(0x7fdd9c070330 msgr2=0x7fdd9c0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:43.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.809+0000 7fdd9be00640 1 --2- 172.21.15.86:0/2235141849 >> v2:172.21.15.155:3300/0 conn(0x7fdd9c070330 0x7fdd9c0d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:43.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.809+0000 7fdd9be00640 1 -- 172.21.15.86:0/2235141849 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd9c15eb80 con 0x7fdd9c0d1c20 2024-04-25T14:14:43.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.809+0000 7fdda1200640 1 --2- 172.21.15.86:0/2235141849 >> v2:172.21.15.155:3300/0 conn(0x7fdd9c070330 0x7fdd9c0d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:43.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.809+0000 7fdd9be00640 1 --2- 172.21.15.86:0/2235141849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd9c0d1c20 0x7fdd9c06ed30 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c0d23c0 tx=0x7fdd8801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83961eb5db1e137b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:43.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.813+0000 7fdd9b400640 1 -- 172.21.15.86:0/2235141849 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd880ab600 con 0x7fdd9c0d1c20 2024-04-25T14:14:43.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.813+0000 7fdd9b400640 1 -- 172.21.15.86:0/2235141849 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd880b0070 con 0x7fdd9c0d1c20 2024-04-25T14:14:43.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.813+0000 7fdd9b400640 1 -- 172.21.15.86:0/2235141849 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd880ab9c0 con 0x7fdd9c0d1c20 2024-04-25T14:14:43.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.813+0000 7fdda3c00640 1 -- 172.21.15.86:0/2235141849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd9c0d1c20 msgr2=0x7fdd9c06ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:43.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.813+0000 7fdda3c00640 1 --2- 172.21.15.86:0/2235141849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd9c0d1c20 0x7fdd9c06ed30 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c0d23c0 tx=0x7fdd8801c460 comp rx=0 tx=0).stop 2024-04-25T14:14:43.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.813+0000 7fdda3c00640 1 -- 172.21.15.86:0/2235141849 shutdown_connections 2024-04-25T14:14:43.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.813+0000 7fdda3c00640 1 --2- 172.21.15.86:0/2235141849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd9c06f270 0x7fdd9c06f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:43.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.813+0000 7fdda3c00640 1 --2- 172.21.15.86:0/2235141849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd9c0d1c20 0x7fdd9c06ed30 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:43.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.813+0000 7fdda3c00640 1 --2- 172.21.15.86:0/2235141849 >> v2:172.21.15.155:3300/0 conn(0x7fdd9c070330 0x7fdd9c0d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:43.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.813+0000 7fdda3c00640 1 -- 172.21.15.86:0/2235141849 >> 172.21.15.86:0/2235141849 conn(0x7fdd9c0b4650 msgr2=0x7fdd9c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:43.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.813+0000 7fdda3c00640 1 -- 172.21.15.86:0/2235141849 shutdown_connections 2024-04-25T14:14:43.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.813+0000 7fdda3c00640 1 -- 172.21.15.86:0/2235141849 wait complete. 2024-04-25T14:14:43.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.813+0000 7fdda3c00640 1 Processor -- start 2024-04-25T14:14:43.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.813+0000 7fdda3c00640 1 -- start start 2024-04-25T14:14:43.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.817+0000 7fdda3c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd9c06f270 0x7fdd9c156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:43.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.817+0000 7fdda3c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd9c070330 0x7fdd9c1573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:43.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.817+0000 7fdda3c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd9c0d1c20 0x7fdd9c072f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:43.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.817+0000 7fdda3c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9c15f4b0 con 0x7fdd9c070330 2024-04-25T14:14:43.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.817+0000 7fdda3c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9c15f330 con 0x7fdd9c0d1c20 2024-04-25T14:14:43.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.817+0000 7fdda3c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdd9c070740 con 0x7fdd9c06f270 2024-04-25T14:14:43.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.817+0000 7fdd9be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd9c070330 0x7fdd9c1573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:43.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.817+0000 7fdd9be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd9c070330 0x7fdd9c1573c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53538/0 (socket says 172.21.15.86:53538) 2024-04-25T14:14:43.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.817+0000 7fdd9be00640 1 -- 172.21.15.86:0/1451720538 learned_addr learned my addr 172.21.15.86:0/1451720538 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:43.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.817+0000 7fdda1200640 1 --2- 172.21.15.86:0/1451720538 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd9c06f270 0x7fdd9c156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:43.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.817+0000 7fdda1c00640 1 --2- 172.21.15.86:0/1451720538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd9c0d1c20 0x7fdd9c072f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:43.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.817+0000 7fdd9be00640 1 -- 172.21.15.86:0/1451720538 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd9c06f270 msgr2=0x7fdd9c156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:43.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.817+0000 7fdd9be00640 1 --2- 172.21.15.86:0/1451720538 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd9c06f270 0x7fdd9c156e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:43.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.817+0000 7fdd9be00640 1 -- 172.21.15.86:0/1451720538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd9c0d1c20 msgr2=0x7fdd9c072f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:43.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.817+0000 7fdd9be00640 1 --2- 172.21.15.86:0/1451720538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd9c0d1c20 0x7fdd9c072f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:43.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.817+0000 7fdd9be00640 1 -- 172.21.15.86:0/1451720538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd9c159040 con 0x7fdd9c070330 2024-04-25T14:14:43.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.817+0000 7fdda1c00640 1 --2- 172.21.15.86:0/1451720538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd9c0d1c20 0x7fdd9c072f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:43.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.817+0000 7fdd9be00640 1 --2- 172.21.15.86:0/1451720538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd9c070330 0x7fdd9c1573c0 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7fdd8801cd00 tx=0x7fdd88019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:43.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.817+0000 7fdd98c00640 1 -- 172.21.15.86:0/1451720538 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd880ba030 con 0x7fdd9c070330 2024-04-25T14:14:43.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.817+0000 7fdd98c00640 1 -- 172.21.15.86:0/1451720538 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd880bf410 con 0x7fdd9c070330 2024-04-25T14:14:43.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.817+0000 7fdd98c00640 1 -- 172.21.15.86:0/1451720538 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd880bf7e0 con 0x7fdd9c070330 2024-04-25T14:14:43.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.817+0000 7fdda3c00640 1 -- 172.21.15.86:0/1451720538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd9c159320 con 0x7fdd9c070330 2024-04-25T14:14:43.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.817+0000 7fdda3c00640 1 -- 172.21.15.86:0/1451720538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd9c155b20 con 0x7fdd9c070330 2024-04-25T14:14:43.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.817+0000 7fdda3c00640 1 -- 172.21.15.86:0/1451720538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd9c069d00 con 0x7fdd9c070330 2024-04-25T14:14:43.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.821+0000 7fdd98c00640 1 -- 172.21.15.86:0/1451720538 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7fdd880bd030 con 0x7fdd9c070330 2024-04-25T14:14:43.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.821+0000 7fdd98c00640 1 --2- 172.21.15.86:0/1451720538 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fdd7c06f5a0 0x7fdd7c071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:43.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.821+0000 7fdd98c00640 1 -- 172.21.15.86:0/1451720538 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fdd881297a0 con 0x7fdd9c070330 2024-04-25T14:14:43.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.821+0000 7fdda1200640 1 --2- 172.21.15.86:0/1451720538 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fdd7c06f5a0 0x7fdd7c071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:43.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.821+0000 7fdda1200640 1 --2- 172.21.15.86:0/1451720538 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fdd7c06f5a0 0x7fdd7c071ab0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c072200 tx=0x7fdd8c009870 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:43.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:43.825+0000 7fdd98c00640 1 -- 172.21.15.86:0/1451720538 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdd88016110 con 0x7fdd9c070330 2024-04-25T14:14:44.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:44.017+0000 7fdda3c00640 1 -- 172.21.15.86:0/1451720538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd9c076bf0 con 0x7fdd9c070330 2024-04-25T14:14:44.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:44.017+0000 7fdd98c00640 1 -- 172.21.15.86:0/1451720538 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7fdd880fe170 con 0x7fdd9c070330 2024-04-25T14:14:44.013 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:44.013 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:14:44.013 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:14:44.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:44.017+0000 7fdda3c00640 1 -- 172.21.15.86:0/1451720538 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fdd7c06f5a0 msgr2=0x7fdd7c071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:44.017+0000 7fdda3c00640 1 --2- 172.21.15.86:0/1451720538 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fdd7c06f5a0 0x7fdd7c071ab0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c072200 tx=0x7fdd8c009870 comp rx=0 tx=0).stop 2024-04-25T14:14:44.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:44.017+0000 7fdda3c00640 1 -- 172.21.15.86:0/1451720538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd9c070330 msgr2=0x7fdd9c1573c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:44.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:44.017+0000 7fdda3c00640 1 --2- 172.21.15.86:0/1451720538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd9c070330 0x7fdd9c1573c0 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7fdd8801cd00 tx=0x7fdd88019200 comp rx=0 tx=0).stop 2024-04-25T14:14:44.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:44.017+0000 7fdda3c00640 1 -- 172.21.15.86:0/1451720538 shutdown_connections 2024-04-25T14:14:44.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:44.017+0000 7fdda3c00640 1 --2- 172.21.15.86:0/1451720538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd9c0d1c20 0x7fdd9c072f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:44.017+0000 7fdda3c00640 1 --2- 172.21.15.86:0/1451720538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd9c070330 0x7fdd9c1573c0 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:44.017+0000 7fdda3c00640 1 --2- 172.21.15.86:0/1451720538 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fdd7c06f5a0 0x7fdd7c071ab0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:44.017+0000 7fdda3c00640 1 --2- 172.21.15.86:0/1451720538 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd9c06f270 0x7fdd9c156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:44.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:44.017+0000 7fdda3c00640 1 -- 172.21.15.86:0/1451720538 >> 172.21.15.86:0/1451720538 conn(0x7fdd9c0b4650 msgr2=0x7fdd9c0c4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:44.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:44.017+0000 7fdda3c00640 1 -- 172.21.15.86:0/1451720538 shutdown_connections 2024-04-25T14:14:44.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:44.017+0000 7fdda3c00640 1 -- 172.21.15.86:0/1451720538 wait complete. 2024-04-25T14:14:44.437 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:14:44.438 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:14:45.439 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:45.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.601+0000 7fe7fc800640 1 Processor -- start 2024-04-25T14:14:45.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.601+0000 7fe7fc800640 1 -- start start 2024-04-25T14:14:45.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.601+0000 7fe7fc800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe7f406fd50 0x7fe7f4070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.601+0000 7fe7fc800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7f415feb0 0x7fe7f41602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.601+0000 7fe7fc800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7f40b0a60 con 0x7fe7f415c2a0 2024-04-25T14:14:45.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.601+0000 7fe7fc800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe7f40ae840 con 0x7fe7f406fd50 2024-04-25T14:14:45.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.601+0000 7fe7fc800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f406ef90 con 0x7fe7f415feb0 2024-04-25T14:14:45.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.601+0000 7fe7f9e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe7f415c2a0 0x7fe7f40ae130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60440/0 (socket says 172.21.15.86:60440) 2024-04-25T14:14:45.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.601+0000 7fe7f9e00640 1 -- 172.21.15.86:0/408102841 learned_addr learned my addr 172.21.15.86:0/408102841 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:14:45.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.605+0000 7fe7f9400640 1 --2- 172.21.15.86:0/408102841 >> v2:172.21.15.86:3300/0 conn(0x7fe7f406fd50 0x7fe7f4070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.605+0000 7fe7fa800640 1 --2- 172.21.15.86:0/408102841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7f415feb0 0x7fe7f41602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.605+0000 7fe7f9400640 1 -- 172.21.15.86:0/408102841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7f415feb0 msgr2=0x7fe7f41602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:45.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.605+0000 7fe7f9400640 1 --2- 172.21.15.86:0/408102841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7f415feb0 0x7fe7f41602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.605+0000 7fe7f9400640 1 -- 172.21.15.86:0/408102841 >> v1:172.21.15.86:6789/0 conn(0x7fe7f415c2a0 legacy=0x7fe7f40ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.605+0000 7fe7f9400640 1 -- 172.21.15.86:0/408102841 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f406f270 con 0x7fe7f406fd50 2024-04-25T14:14:45.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.605+0000 7fe7f9400640 1 --2- 172.21.15.86:0/408102841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7f406fd50 0x7fe7f4070160 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7fe7e40097c0 tx=0x7fe7e401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e42b91ce425d56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:45.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.605+0000 7fe7f8a00640 1 -- 172.21.15.86:0/408102841 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e40b2070 con 0x7fe7f406fd50 2024-04-25T14:14:45.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.605+0000 7fe7f8a00640 1 -- 172.21.15.86:0/408102841 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e40a9070 con 0x7fe7f406fd50 2024-04-25T14:14:45.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.605+0000 7fe7f8a00640 1 -- 172.21.15.86:0/408102841 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e40ad880 con 0x7fe7f406fd50 2024-04-25T14:14:45.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.609+0000 7fe7fc800640 1 -- 172.21.15.86:0/408102841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7f406fd50 msgr2=0x7fe7f4070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.609+0000 7fe7fc800640 1 --2- 172.21.15.86:0/408102841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7f406fd50 0x7fe7f4070160 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7fe7e40097c0 tx=0x7fe7e401c3e0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.609+0000 7fe7fc800640 1 -- 172.21.15.86:0/408102841 shutdown_connections 2024-04-25T14:14:45.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.609+0000 7fe7fc800640 1 --2- 172.21.15.86:0/408102841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7f415feb0 0x7fe7f41602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.609+0000 7fe7fc800640 1 --2- 172.21.15.86:0/408102841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7f406fd50 0x7fe7f4070160 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.609+0000 7fe7fc800640 1 -- 172.21.15.86:0/408102841 >> 172.21.15.86:0/408102841 conn(0x7fe7f40b4650 msgr2=0x7fe7f40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:45.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.609+0000 7fe7fc800640 1 -- 172.21.15.86:0/408102841 shutdown_connections 2024-04-25T14:14:45.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.609+0000 7fe7fc800640 1 -- 172.21.15.86:0/408102841 wait complete. 2024-04-25T14:14:45.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.609+0000 7fe7fc800640 1 Processor -- start 2024-04-25T14:14:45.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.609+0000 7fe7fc800640 1 -- start start 2024-04-25T14:14:45.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.613+0000 7fe7fc800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7f406fd50 0x7fe7f40c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.613+0000 7fe7fc800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7f415c2a0 0x7fe7f40c34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.613+0000 7fe7fc800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7f415feb0 0x7fe7f4157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.613+0000 7fe7fc800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f415c6b0 con 0x7fe7f406fd50 2024-04-25T14:14:45.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.613+0000 7fe7fc800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f40d0fc0 con 0x7fe7f415feb0 2024-04-25T14:14:45.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.613+0000 7fe7fc800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f415c830 con 0x7fe7f415c2a0 2024-04-25T14:14:45.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.613+0000 7fe7f9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7f406fd50 0x7fe7f40c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.613+0000 7fe7f9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7f406fd50 0x7fe7f40c3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53552/0 (socket says 172.21.15.86:53552) 2024-04-25T14:14:45.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.613+0000 7fe7f9e00640 1 -- 172.21.15.86:0/2472403881 learned_addr learned my addr 172.21.15.86:0/2472403881 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:45.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.613+0000 7fe7f9e00640 1 -- 172.21.15.86:0/2472403881 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7f415c2a0 msgr2=0x7fe7f40c34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.613+0000 7fe7f9400640 1 --2- 172.21.15.86:0/2472403881 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7f415c2a0 0x7fe7f40c34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.613+0000 7fe7fa800640 1 --2- 172.21.15.86:0/2472403881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7f415feb0 0x7fe7f4157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.613+0000 7fe7f9e00640 1 --2- 172.21.15.86:0/2472403881 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7f415c2a0 0x7fe7f40c34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.613+0000 7fe7f9e00640 1 -- 172.21.15.86:0/2472403881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7f415feb0 msgr2=0x7fe7f4157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.613+0000 7fe7f9e00640 1 --2- 172.21.15.86:0/2472403881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7f415feb0 0x7fe7f4157280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.613+0000 7fe7f9e00640 1 -- 172.21.15.86:0/2472403881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f4158520 con 0x7fe7f406fd50 2024-04-25T14:14:45.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.613+0000 7fe7fa800640 1 --2- 172.21.15.86:0/2472403881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7f415feb0 0x7fe7f4157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:14:45.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.613+0000 7fe7f9e00640 1 --2- 172.21.15.86:0/2472403881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7f406fd50 0x7fe7f40c3eb0 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800c340 tx=0x7fe7e800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:45.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.613+0000 7fe7ee000640 1 -- 172.21.15.86:0/2472403881 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8017070 con 0x7fe7f406fd50 2024-04-25T14:14:45.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.613+0000 7fe7ee000640 1 -- 172.21.15.86:0/2472403881 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e8005cf0 con 0x7fe7f406fd50 2024-04-25T14:14:45.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.613+0000 7fe7ee000640 1 -- 172.21.15.86:0/2472403881 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8007310 con 0x7fe7f406fd50 2024-04-25T14:14:45.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.613+0000 7fe7fc800640 1 -- 172.21.15.86:0/2472403881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f4159270 con 0x7fe7f406fd50 2024-04-25T14:14:45.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.613+0000 7fe7fc800640 1 -- 172.21.15.86:0/2472403881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f415b190 con 0x7fe7f406fd50 2024-04-25T14:14:45.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.613+0000 7fe7fc800640 1 -- 172.21.15.86:0/2472403881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f415b3e0 con 0x7fe7f406fd50 2024-04-25T14:14:45.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.613+0000 7fe7ee000640 1 -- 172.21.15.86:0/2472403881 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7fe7e801a070 con 0x7fe7f406fd50 2024-04-25T14:14:45.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.617+0000 7fe7ee000640 1 --2- 172.21.15.86:0/2472403881 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe7d806f570 0x7fe7d8071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:45.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.617+0000 7fe7ee000640 1 -- 172.21.15.86:0/2472403881 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fe7e808ea20 con 0x7fe7f406fd50 2024-04-25T14:14:45.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.617+0000 7fe7f9400640 1 --2- 172.21.15.86:0/2472403881 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe7d806f570 0x7fe7d8071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:45.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.617+0000 7fe7f9400640 1 --2- 172.21.15.86:0/2472403881 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe7d806f570 0x7fe7d8071a80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe7e40af000 tx=0x7fe7e4008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:45.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.621+0000 7fe7ee000640 1 -- 172.21.15.86:0/2472403881 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe7e805b5b0 con 0x7fe7f406fd50 2024-04-25T14:14:45.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.809+0000 7fe7fc800640 1 -- 172.21.15.86:0/2472403881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7f4069d00 con 0x7fe7f406fd50 2024-04-25T14:14:45.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.809+0000 7fe7ee000640 1 -- 172.21.15.86:0/2472403881 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7fe7e805aeb0 con 0x7fe7f406fd50 2024-04-25T14:14:45.807 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:45.807 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:14:45.808 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:14:45.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.813+0000 7fe7fc800640 1 -- 172.21.15.86:0/2472403881 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe7d806f570 msgr2=0x7fe7d8071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.813+0000 7fe7fc800640 1 --2- 172.21.15.86:0/2472403881 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe7d806f570 0x7fe7d8071a80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe7e40af000 tx=0x7fe7e4008040 comp rx=0 tx=0).stop 2024-04-25T14:14:45.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.813+0000 7fe7fc800640 1 -- 172.21.15.86:0/2472403881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7f406fd50 msgr2=0x7fe7f40c3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:45.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.813+0000 7fe7fc800640 1 --2- 172.21.15.86:0/2472403881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7f406fd50 0x7fe7f40c3eb0 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800c340 tx=0x7fe7e800c800 comp rx=0 tx=0).stop 2024-04-25T14:14:45.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.813+0000 7fe7fc800640 1 -- 172.21.15.86:0/2472403881 shutdown_connections 2024-04-25T14:14:45.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.813+0000 7fe7fc800640 1 --2- 172.21.15.86:0/2472403881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7f415feb0 0x7fe7f4157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.813+0000 7fe7fc800640 1 --2- 172.21.15.86:0/2472403881 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe7d806f570 0x7fe7d8071a80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.813+0000 7fe7fc800640 1 --2- 172.21.15.86:0/2472403881 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7f415c2a0 0x7fe7f40c34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.813+0000 7fe7fc800640 1 --2- 172.21.15.86:0/2472403881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7f406fd50 0x7fe7f40c3eb0 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:45.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.813+0000 7fe7fc800640 1 -- 172.21.15.86:0/2472403881 >> 172.21.15.86:0/2472403881 conn(0x7fe7f40b4650 msgr2=0x7fe7f40aea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:45.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.813+0000 7fe7fc800640 1 -- 172.21.15.86:0/2472403881 shutdown_connections 2024-04-25T14:14:45.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:45.813+0000 7fe7fc800640 1 -- 172.21.15.86:0/2472403881 wait complete. 2024-04-25T14:14:46.231 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:14:46.232 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:14:47.233 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:47.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.397+0000 7f60a4400640 1 Processor -- start 2024-04-25T14:14:47.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.397+0000 7f60a4400640 1 -- start start 2024-04-25T14:14:47.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.397+0000 7f60a4400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f609c15fc70 0x7f609c070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:47.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.397+0000 7f60a4400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f609c071360 0x7f609c163810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:47.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.397+0000 7f60a4400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f609c163d50 0x7f609c0722f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:47.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.397+0000 7f60a4400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f609c06e7f0 con 0x7f609c163d50 2024-04-25T14:14:47.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.397+0000 7f60a4400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f609c06f0a0 con 0x7f609c15fc70 2024-04-25T14:14:47.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.397+0000 7f60a4400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f609c071770 con 0x7f609c071360 2024-04-25T14:14:47.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.397+0000 7f60a2400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f609c163d50 0x7f609c0722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:47.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.397+0000 7f60a2400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f609c163d50 0x7f609c0722f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53558/0 (socket says 172.21.15.86:53558) 2024-04-25T14:14:47.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.397+0000 7f60a2400640 1 -- 172.21.15.86:0/3187983997 learned_addr learned my addr 172.21.15.86:0/3187983997 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:47.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.397+0000 7f60a1000640 1 --2- 172.21.15.86:0/3187983997 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f609c071360 0x7f609c163810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:47.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.397+0000 7f60a1a00640 1 --2- 172.21.15.86:0/3187983997 >> v2:172.21.15.155:3300/0 conn(0x7f609c15fc70 0x7f609c070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:47.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.397+0000 7f60a2400640 1 -- 172.21.15.86:0/3187983997 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f609c071360 msgr2=0x7f609c163810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:47.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.397+0000 7f60a2400640 1 --2- 172.21.15.86:0/3187983997 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f609c071360 0x7f609c163810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:47.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.397+0000 7f60a2400640 1 -- 172.21.15.86:0/3187983997 >> v2:172.21.15.155:3300/0 conn(0x7f609c15fc70 msgr2=0x7f609c070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:47.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.397+0000 7f60a2400640 1 --2- 172.21.15.86:0/3187983997 >> v2:172.21.15.155:3300/0 conn(0x7f609c15fc70 0x7f609c070e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:47.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.397+0000 7f60a2400640 1 -- 172.21.15.86:0/3187983997 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f609c164160 con 0x7f609c163d50 2024-04-25T14:14:47.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.401+0000 7f60a2400640 1 --2- 172.21.15.86:0/3187983997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f609c163d50 0x7f609c0722f0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f609c074330 tx=0x7f609401c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4684daf9de76478d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:47.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.401+0000 7f609be00640 1 -- 172.21.15.86:0/3187983997 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60940a55b0 con 0x7f609c163d50 2024-04-25T14:14:47.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.401+0000 7f609be00640 1 -- 172.21.15.86:0/3187983997 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60940a5750 con 0x7f609c163d50 2024-04-25T14:14:47.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.401+0000 7f609be00640 1 -- 172.21.15.86:0/3187983997 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60940a5b20 con 0x7f609c163d50 2024-04-25T14:14:47.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.401+0000 7f60a4400640 1 -- 172.21.15.86:0/3187983997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f609c163d50 msgr2=0x7f609c0722f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:47.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.401+0000 7f60a4400640 1 --2- 172.21.15.86:0/3187983997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f609c163d50 0x7f609c0722f0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f609c074330 tx=0x7f609401c730 comp rx=0 tx=0).stop 2024-04-25T14:14:47.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.401+0000 7f60a4400640 1 -- 172.21.15.86:0/3187983997 shutdown_connections 2024-04-25T14:14:47.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.401+0000 7f60a4400640 1 --2- 172.21.15.86:0/3187983997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f609c163d50 0x7f609c0722f0 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:47.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.401+0000 7f60a4400640 1 --2- 172.21.15.86:0/3187983997 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f609c071360 0x7f609c163810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:47.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.401+0000 7f60a4400640 1 --2- 172.21.15.86:0/3187983997 >> v2:172.21.15.155:3300/0 conn(0x7f609c15fc70 0x7f609c070e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:47.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.401+0000 7f60a4400640 1 -- 172.21.15.86:0/3187983997 >> 172.21.15.86:0/3187983997 conn(0x7f609c0cffc0 msgr2=0x7f609c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:47.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.401+0000 7f60a4400640 1 -- 172.21.15.86:0/3187983997 shutdown_connections 2024-04-25T14:14:47.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.401+0000 7f60a4400640 1 -- 172.21.15.86:0/3187983997 wait complete. 2024-04-25T14:14:47.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.405+0000 7f60a4400640 1 Processor -- start 2024-04-25T14:14:47.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.405+0000 7f60a4400640 1 -- start start 2024-04-25T14:14:47.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.405+0000 7f60a4400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f609c071360 0x7f609c1628e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:47.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.405+0000 7f60a4400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f609c15fc70 0x7f609c14e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:47.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.405+0000 7f60a4400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f609c163d50 0x7f609c14ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:47.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.405+0000 7f60a4400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f609c160080 con 0x7f609c071360 2024-04-25T14:14:47.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.405+0000 7f60a4400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f609c06e480 con 0x7f609c163d50 2024-04-25T14:14:47.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.405+0000 7f60a4400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f609c0cdde0 con 0x7f609c15fc70 2024-04-25T14:14:47.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.405+0000 7f60a1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f609c071360 0x7f609c1628e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:47.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.405+0000 7f60a1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f609c071360 0x7f609c1628e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53562/0 (socket says 172.21.15.86:53562) 2024-04-25T14:14:47.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.405+0000 7f60a1a00640 1 -- 172.21.15.86:0/2541275367 learned_addr learned my addr 172.21.15.86:0/2541275367 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:47.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.405+0000 7f60a1000640 1 --2- 172.21.15.86:0/2541275367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f609c15fc70 0x7f609c14e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:47.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.405+0000 7f60a2400640 1 --2- 172.21.15.86:0/2541275367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f609c163d50 0x7f609c14ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:47.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.405+0000 7f60a1a00640 1 -- 172.21.15.86:0/2541275367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f609c15fc70 msgr2=0x7f609c14e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:47.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.405+0000 7f60a1a00640 1 --2- 172.21.15.86:0/2541275367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f609c15fc70 0x7f609c14e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:47.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.405+0000 7f60a1a00640 1 -- 172.21.15.86:0/2541275367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f609c163d50 msgr2=0x7f609c14ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:47.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.405+0000 7f60a1a00640 1 --2- 172.21.15.86:0/2541275367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f609c163d50 0x7f609c14ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:47.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.405+0000 7f60a1a00640 1 -- 172.21.15.86:0/2541275367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f609c0cb4c0 con 0x7f609c071360 2024-04-25T14:14:47.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.409+0000 7f60a1000640 1 --2- 172.21.15.86:0/2541275367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f609c15fc70 0x7f609c14e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:47.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.409+0000 7f60a2400640 1 --2- 172.21.15.86:0/2541275367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f609c163d50 0x7f609c14ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:47.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.409+0000 7f60a1a00640 1 --2- 172.21.15.86:0/2541275367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f609c071360 0x7f609c1628e0 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f609000d5f0 tx=0x7f609000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:47.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.409+0000 7f6099600640 1 -- 172.21.15.86:0/2541275367 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60900125f0 con 0x7f609c071360 2024-04-25T14:14:47.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.409+0000 7f6099600640 1 -- 172.21.15.86:0/2541275367 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6090019070 con 0x7f609c071360 2024-04-25T14:14:47.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.409+0000 7f6099600640 1 -- 172.21.15.86:0/2541275367 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60900129b0 con 0x7f609c071360 2024-04-25T14:14:47.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.409+0000 7f60a4400640 1 -- 172.21.15.86:0/2541275367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f609c0cb7a0 con 0x7f609c071360 2024-04-25T14:14:47.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.409+0000 7f60a4400640 1 -- 172.21.15.86:0/2541275367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f609c14fa50 con 0x7f609c071360 2024-04-25T14:14:47.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.409+0000 7f60a4400640 1 -- 172.21.15.86:0/2541275367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f609c0720e0 con 0x7f609c071360 2024-04-25T14:14:47.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.409+0000 7f6099600640 1 -- 172.21.15.86:0/2541275367 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f6090012b50 con 0x7f609c071360 2024-04-25T14:14:47.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.409+0000 7f6099600640 1 --2- 172.21.15.86:0/2541275367 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f607c06f570 0x7f607c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:47.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.409+0000 7f6099600640 1 -- 172.21.15.86:0/2541275367 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f609008fba0 con 0x7f609c071360 2024-04-25T14:14:47.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.409+0000 7f60a1000640 1 --2- 172.21.15.86:0/2541275367 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f607c06f570 0x7f607c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:47.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.413+0000 7f60a1000640 1 --2- 172.21.15.86:0/2541275367 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f607c06f570 0x7f607c071a80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f609c1632e0 tx=0x7f6088008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.413+0000 7f6099600640 1 -- 172.21.15.86:0/2541275367 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f609005c690 con 0x7f609c071360 2024-04-25T14:14:47.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.605+0000 7f60a4400640 1 -- 172.21.15.86:0/2541275367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f609c076bf0 con 0x7f609c071360 2024-04-25T14:14:47.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.605+0000 7f6099600640 1 -- 172.21.15.86:0/2541275367 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f609008f020 con 0x7f609c071360 2024-04-25T14:14:47.602 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:47.602 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:14:47.602 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:14:47.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.609+0000 7f60a4400640 1 -- 172.21.15.86:0/2541275367 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f607c06f570 msgr2=0x7f607c071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:47.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.609+0000 7f60a4400640 1 --2- 172.21.15.86:0/2541275367 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f607c06f570 0x7f607c071a80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f609c1632e0 tx=0x7f6088008040 comp rx=0 tx=0).stop 2024-04-25T14:14:47.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.609+0000 7f60a4400640 1 -- 172.21.15.86:0/2541275367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f609c071360 msgr2=0x7f609c1628e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:47.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.609+0000 7f60a4400640 1 --2- 172.21.15.86:0/2541275367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f609c071360 0x7f609c1628e0 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f609000d5f0 tx=0x7f609000dab0 comp rx=0 tx=0).stop 2024-04-25T14:14:47.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.609+0000 7f60a4400640 1 -- 172.21.15.86:0/2541275367 shutdown_connections 2024-04-25T14:14:47.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.609+0000 7f60a4400640 1 --2- 172.21.15.86:0/2541275367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f609c163d50 0x7f609c14ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:47.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.609+0000 7f60a4400640 1 --2- 172.21.15.86:0/2541275367 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f607c06f570 0x7f607c071a80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:47.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.609+0000 7f60a4400640 1 --2- 172.21.15.86:0/2541275367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f609c15fc70 0x7f609c14e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:47.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.609+0000 7f60a4400640 1 --2- 172.21.15.86:0/2541275367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f609c071360 0x7f609c1628e0 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:47.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.609+0000 7f60a4400640 1 -- 172.21.15.86:0/2541275367 >> 172.21.15.86:0/2541275367 conn(0x7f609c0cffc0 msgr2=0x7f609c0b3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:47.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.609+0000 7f60a4400640 1 -- 172.21.15.86:0/2541275367 shutdown_connections 2024-04-25T14:14:47.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:47.609+0000 7f60a4400640 1 -- 172.21.15.86:0/2541275367 wait complete. 2024-04-25T14:14:48.026 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:14:48.027 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:14:49.028 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.189+0000 7fb6e9000640 1 Processor -- start 2024-04-25T14:14:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e9000640 1 -- start start 2024-04-25T14:14:49.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e9000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb6e4070330 0x7fb6e40d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:49.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e9000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6e40d1c20 0x7fb6e406ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:49.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e9000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6e40d2030 con 0x7fb6e406f270 2024-04-25T14:14:49.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e9000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb6e415e5c0 con 0x7fb6e4070330 2024-04-25T14:14:49.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e9000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb6e415e8a0 con 0x7fb6e40d1c20 2024-04-25T14:14:49.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e2800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb6e406f270 0x7fb6e406f680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36238/0 (socket says 172.21.15.86:36238) 2024-04-25T14:14:49.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e2800640 1 -- 172.21.15.86:0/1577243665 learned_addr learned my addr 172.21.15.86:0/1577243665 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:14:49.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6d9400640 1 --2- 172.21.15.86:0/1577243665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6e40d1c20 0x7fb6e406ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:49.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e1e00640 1 --2- 172.21.15.86:0/1577243665 >> v2:172.21.15.155:3300/0 conn(0x7fb6e4070330 0x7fb6e40d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:49.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e1400640 1 -- 172.21.15.86:0/1577243665 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3772180091 0 0) 0x7fb6dc0014a0 con 0x7fb6e406f270 2024-04-25T14:14:49.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e1400640 1 -- 172.21.15.86:0/1577243665 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6c8003530 con 0x7fb6e406f270 2024-04-25T14:14:49.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e1400640 1 -- 172.21.15.86:0/1577243665 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1818804475 0 0) 0x7fb6c8003530 con 0x7fb6e406f270 2024-04-25T14:14:49.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e1400640 1 -- 172.21.15.86:0/1577243665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6e40d1c20 msgr2=0x7fb6e406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:49.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e1400640 1 --2- 172.21.15.86:0/1577243665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6e40d1c20 0x7fb6e406ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:49.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e1400640 1 -- 172.21.15.86:0/1577243665 >> v2:172.21.15.155:3300/0 conn(0x7fb6e4070330 msgr2=0x7fb6e40d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:49.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e1400640 1 --2- 172.21.15.86:0/1577243665 >> v2:172.21.15.155:3300/0 conn(0x7fb6e4070330 0x7fb6e40d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:49.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e1400640 1 -- 172.21.15.86:0/1577243665 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e415eb80 con 0x7fb6e406f270 2024-04-25T14:14:49.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e1400640 1 -- 172.21.15.86:0/1577243665 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fb6dc001bd0 con 0x7fb6e406f270 2024-04-25T14:14:49.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e1400640 1 -- 172.21.15.86:0/1577243665 >> v1:172.21.15.86:6789/0 conn(0x7fb6e406f270 legacy=0x7fb6e406f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:49.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e1400640 1 --2- 172.21.15.86:0/1577243665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6e40d1c20 0x7fb6c8003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:49.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e1400640 1 --2- 172.21.15.86:0/1577243665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6c8003fb0 0x7fb6c8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:49.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e1400640 1 --2- 172.21.15.86:0/1577243665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6c8004d50 0x7fb6c8005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:49.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e1400640 1 -- 172.21.15.86:0/1577243665 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e415e5c0 con 0x7fb6c8004d50 2024-04-25T14:14:49.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e1400640 1 -- 172.21.15.86:0/1577243665 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e415e8a0 con 0x7fb6c8003fb0 2024-04-25T14:14:49.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e1400640 1 -- 172.21.15.86:0/1577243665 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb6c800bb30 con 0x7fb6e40d1c20 2024-04-25T14:14:49.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e2800640 1 --2- 172.21.15.86:0/1577243665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6c8004d50 0x7fb6c8005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:49.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e1e00640 1 --2- 172.21.15.86:0/1577243665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6e40d1c20 0x7fb6c8003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:49.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6d9400640 1 --2- 172.21.15.86:0/1577243665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6c8003fb0 0x7fb6c8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:49.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e2800640 1 -- 172.21.15.86:0/1577243665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6e40d1c20 msgr2=0x7fb6c8003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:49.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e2800640 1 --2- 172.21.15.86:0/1577243665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6e40d1c20 0x7fb6c8003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:49.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e2800640 1 -- 172.21.15.86:0/1577243665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6c8003fb0 msgr2=0x7fb6c8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:49.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e2800640 1 --2- 172.21.15.86:0/1577243665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6c8003fb0 0x7fb6c8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:49.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.193+0000 7fb6e2800640 1 -- 172.21.15.86:0/1577243665 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c8003530 con 0x7fb6c8004d50 2024-04-25T14:14:49.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.197+0000 7fb6e2800640 1 --2- 172.21.15.86:0/1577243665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6c8004d50 0x7fb6c8005160 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00aa40 tx=0x7fb6dc01fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6cd21ddafa671f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:49.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.197+0000 7fb6e1400640 1 -- 172.21.15.86:0/1577243665 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc00a690 con 0x7fb6c8004d50 2024-04-25T14:14:49.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.197+0000 7fb6e1400640 1 -- 172.21.15.86:0/1577243665 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6dc0aa730 con 0x7fb6c8004d50 2024-04-25T14:14:49.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.197+0000 7fb6e1400640 1 -- 172.21.15.86:0/1577243665 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc0aab60 con 0x7fb6c8004d50 2024-04-25T14:14:49.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.197+0000 7fb6e9000640 1 -- 172.21.15.86:0/1577243665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6c8004d50 msgr2=0x7fb6c8005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:49.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.197+0000 7fb6e9000640 1 --2- 172.21.15.86:0/1577243665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6c8004d50 0x7fb6c8005160 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00aa40 tx=0x7fb6dc01fae0 comp rx=0 tx=0).stop 2024-04-25T14:14:49.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.197+0000 7fb6e2800640 1 -- 172.21.15.86:0/1577243665 reap_dead start 2024-04-25T14:14:49.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.197+0000 7fb6e9000640 1 -- 172.21.15.86:0/1577243665 shutdown_connections 2024-04-25T14:14:49.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.197+0000 7fb6e9000640 1 -- 172.21.15.86:0/1577243665 >> 172.21.15.86:0/1577243665 conn(0x7fb6e40b4650 msgr2=0x7fb6e40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:49.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.197+0000 7fb6e9000640 1 -- 172.21.15.86:0/1577243665 shutdown_connections 2024-04-25T14:14:49.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.197+0000 7fb6e9000640 1 -- 172.21.15.86:0/1577243665 wait complete. 2024-04-25T14:14:49.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.201+0000 7fb6e9000640 1 Processor -- start 2024-04-25T14:14:49.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.201+0000 7fb6e9000640 1 -- start start 2024-04-25T14:14:49.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.201+0000 7fb6e9000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6c8004d50 0x7fb6e41f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:49.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.201+0000 7fb6e9000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6e4152a70 0x7fb6e40c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:49.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.201+0000 7fb6e9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e4152420 0x7fb6e40c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:49.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.201+0000 7fb6e9000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e415f330 con 0x7fb6e4152420 2024-04-25T14:14:49.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.201+0000 7fb6e9000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e406d170 con 0x7fb6e4152a70 2024-04-25T14:14:49.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.201+0000 7fb6e9000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb6e415f4b0 con 0x7fb6c8004d50 2024-04-25T14:14:49.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.201+0000 7fb6e2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e4152420 0x7fb6e40c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:49.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.201+0000 7fb6e2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e4152420 0x7fb6e40c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58270/0 (socket says 172.21.15.86:58270) 2024-04-25T14:14:49.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.201+0000 7fb6e2800640 1 -- 172.21.15.86:0/3018468849 learned_addr learned my addr 172.21.15.86:0/3018468849 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:49.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.201+0000 7fb6e1e00640 1 --2- 172.21.15.86:0/3018468849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6c8004d50 0x7fb6e41f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:49.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.205+0000 7fb6d9400640 1 --2- 172.21.15.86:0/3018468849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6e4152a70 0x7fb6e40c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:49.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.205+0000 7fb6e2800640 1 -- 172.21.15.86:0/3018468849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6c8004d50 msgr2=0x7fb6e41f5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:49.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.205+0000 7fb6e2800640 1 --2- 172.21.15.86:0/3018468849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6c8004d50 0x7fb6e41f5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:49.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.205+0000 7fb6e2800640 1 -- 172.21.15.86:0/3018468849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6e4152a70 msgr2=0x7fb6e40c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:49.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.205+0000 7fb6e2800640 1 --2- 172.21.15.86:0/3018468849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6e4152a70 0x7fb6e40c9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:49.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.205+0000 7fb6e2800640 1 -- 172.21.15.86:0/3018468849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e41e5da0 con 0x7fb6e4152420 2024-04-25T14:14:49.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.205+0000 7fb6e1e00640 1 --2- 172.21.15.86:0/3018468849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6c8004d50 0x7fb6e41f5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:49.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.205+0000 7fb6e2800640 1 --2- 172.21.15.86:0/3018468849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e4152420 0x7fb6e40c9760 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc007040 tx=0x7fb6c8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:49.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.205+0000 7fb6daa00640 1 -- 172.21.15.86:0/3018468849 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc00a690 con 0x7fb6e4152420 2024-04-25T14:14:49.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.205+0000 7fb6daa00640 1 -- 172.21.15.86:0/3018468849 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6dc0014a0 con 0x7fb6e4152420 2024-04-25T14:14:49.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.205+0000 7fb6daa00640 1 -- 172.21.15.86:0/3018468849 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc0aaae0 con 0x7fb6e4152420 2024-04-25T14:14:49.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.205+0000 7fb6e9000640 1 -- 172.21.15.86:0/3018468849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6e41e62a0 con 0x7fb6e4152420 2024-04-25T14:14:49.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.205+0000 7fb6e9000640 1 -- 172.21.15.86:0/3018468849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6e41e6860 con 0x7fb6e4152420 2024-04-25T14:14:49.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.205+0000 7fb6e9000640 1 -- 172.21.15.86:0/3018468849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6e41e6010 con 0x7fb6e4152420 2024-04-25T14:14:49.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.205+0000 7fb6daa00640 1 -- 172.21.15.86:0/3018468849 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7fb6dc0aac80 con 0x7fb6e4152420 2024-04-25T14:14:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.209+0000 7fb6daa00640 1 --2- 172.21.15.86:0/3018468849 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fb6c006f570 0x7fb6c0071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.209+0000 7fb6daa00640 1 -- 172.21.15.86:0/3018468849 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fb6dc12cb00 con 0x7fb6e4152420 2024-04-25T14:14:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.209+0000 7fb6e1e00640 1 --2- 172.21.15.86:0/3018468849 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fb6c006f570 0x7fb6c0071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.209+0000 7fb6e1e00640 1 --2- 172.21.15.86:0/3018468849 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fb6c006f570 0x7fb6c0071a80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4072200 tx=0x7fb6d4008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:49.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.213+0000 7fb6daa00640 1 -- 172.21.15.86:0/3018468849 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb6dc0c1030 con 0x7fb6e4152420 2024-04-25T14:14:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.401+0000 7fb6e9000640 1 -- 172.21.15.86:0/3018468849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6e4076bf0 con 0x7fb6e4152420 2024-04-25T14:14:49.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.405+0000 7fb6daa00640 1 -- 172.21.15.86:0/3018468849 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7fb6dc0ae030 con 0x7fb6e4152420 2024-04-25T14:14:49.400 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:49.400 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:14:49.400 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:14:49.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.405+0000 7fb6e9000640 1 -- 172.21.15.86:0/3018468849 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fb6c006f570 msgr2=0x7fb6c0071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.405+0000 7fb6e9000640 1 --2- 172.21.15.86:0/3018468849 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fb6c006f570 0x7fb6c0071a80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4072200 tx=0x7fb6d4008040 comp rx=0 tx=0).stop 2024-04-25T14:14:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.405+0000 7fb6e9000640 1 -- 172.21.15.86:0/3018468849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e4152420 msgr2=0x7fb6e40c9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.405+0000 7fb6e9000640 1 --2- 172.21.15.86:0/3018468849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e4152420 0x7fb6e40c9760 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc007040 tx=0x7fb6c8005840 comp rx=0 tx=0).stop 2024-04-25T14:14:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.405+0000 7fb6e9000640 1 -- 172.21.15.86:0/3018468849 shutdown_connections 2024-04-25T14:14:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.405+0000 7fb6e9000640 1 --2- 172.21.15.86:0/3018468849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6e4152420 0x7fb6e40c9760 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.405+0000 7fb6e9000640 1 --2- 172.21.15.86:0/3018468849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6e4152a70 0x7fb6e40c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.405+0000 7fb6e9000640 1 --2- 172.21.15.86:0/3018468849 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fb6c006f570 0x7fb6c0071a80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.405+0000 7fb6e9000640 1 --2- 172.21.15.86:0/3018468849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6c8004d50 0x7fb6e41f5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.405+0000 7fb6e9000640 1 -- 172.21.15.86:0/3018468849 >> 172.21.15.86:0/3018468849 conn(0x7fb6e40b4650 msgr2=0x7fb6e40cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.405+0000 7fb6e9000640 1 -- 172.21.15.86:0/3018468849 shutdown_connections 2024-04-25T14:14:49.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:49.405+0000 7fb6e9000640 1 -- 172.21.15.86:0/3018468849 wait complete. 2024-04-25T14:14:49.826 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:14:49.826 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:14:50.330 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:14:50.333 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:14:50.827 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:50.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.977+0000 7f52c7400640 1 Processor -- start 2024-04-25T14:14:50.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.977+0000 7f52c7400640 1 -- start start 2024-04-25T14:14:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.981+0000 7f52c7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f52c00d1df0 0x7f52c00cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.981+0000 7f52c7400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f52c00cdcc0 0x7f52c00ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.981+0000 7f52c7400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52c015e5c0 con 0x7f52c0070330 2024-04-25T14:14:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.981+0000 7f52c7400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f52c006fd40 con 0x7f52c00d1df0 2024-04-25T14:14:50.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.981+0000 7f52c7400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f52c015e8a0 con 0x7f52c00cdcc0 2024-04-25T14:14:50.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.981+0000 7f52b7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f52c00d1df0 0x7f52c00cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:50.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.981+0000 7f52b7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f52c00d1df0 0x7f52c00cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58276/0 (socket says 172.21.15.86:58276) 2024-04-25T14:14:50.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.981+0000 7f52b7e00640 1 -- 172.21.15.86:0/525179253 learned_addr learned my addr 172.21.15.86:0/525179253 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:50.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.981+0000 7f52c5400640 1 --2- 172.21.15.86:0/525179253 >> v2:172.21.15.155:3300/0 conn(0x7f52c00cdcc0 0x7f52c00ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:50.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.981+0000 7f52b7e00640 1 -- 172.21.15.86:0/525179253 >> v2:172.21.15.155:3300/0 conn(0x7f52c00cdcc0 msgr2=0x7f52c00ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:50.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.981+0000 7f52b7e00640 1 --2- 172.21.15.86:0/525179253 >> v2:172.21.15.155:3300/0 conn(0x7f52c00cdcc0 0x7f52c00ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:50.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.981+0000 7f52b7e00640 1 -- 172.21.15.86:0/525179253 >> v1:172.21.15.86:6789/0 conn(0x7f52c0070330 legacy=0x7f52c00d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:50.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.981+0000 7f52b7e00640 1 -- 172.21.15.86:0/525179253 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c015eb80 con 0x7f52c00d1df0 2024-04-25T14:14:50.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.985+0000 7f52b7e00640 1 --2- 172.21.15.86:0/525179253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52c00d1df0 0x7f52c00cd780 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009870 tx=0x7f52a801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864726c6396adc83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:50.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.985+0000 7f52bfe00640 1 -- 172.21.15.86:0/525179253 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a80ab640 con 0x7f52c00d1df0 2024-04-25T14:14:50.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.985+0000 7f52bfe00640 1 -- 172.21.15.86:0/525179253 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52a80b0070 con 0x7f52c00d1df0 2024-04-25T14:14:50.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.985+0000 7f52bfe00640 1 -- 172.21.15.86:0/525179253 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a80aba00 con 0x7f52c00d1df0 2024-04-25T14:14:50.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.985+0000 7f52c7400640 1 -- 172.21.15.86:0/525179253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52c00d1df0 msgr2=0x7f52c00cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:50.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.985+0000 7f52c7400640 1 --2- 172.21.15.86:0/525179253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52c00d1df0 0x7f52c00cd780 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009870 tx=0x7f52a801c460 comp rx=0 tx=0).stop 2024-04-25T14:14:50.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.985+0000 7f52c7400640 1 -- 172.21.15.86:0/525179253 shutdown_connections 2024-04-25T14:14:50.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.985+0000 7f52c7400640 1 --2- 172.21.15.86:0/525179253 >> v2:172.21.15.155:3300/0 conn(0x7f52c00cdcc0 0x7f52c00ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:50.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.985+0000 7f52c7400640 1 --2- 172.21.15.86:0/525179253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52c00d1df0 0x7f52c00cd780 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:50.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.985+0000 7f52c7400640 1 -- 172.21.15.86:0/525179253 >> 172.21.15.86:0/525179253 conn(0x7f52c00b4650 msgr2=0x7f52c00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:50.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.985+0000 7f52c7400640 1 -- 172.21.15.86:0/525179253 shutdown_connections 2024-04-25T14:14:50.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.985+0000 7f52c7400640 1 -- 172.21.15.86:0/525179253 wait complete. 2024-04-25T14:14:50.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.985+0000 7f52c7400640 1 Processor -- start 2024-04-25T14:14:50.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.985+0000 7f52c7400640 1 -- start start 2024-04-25T14:14:50.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.989+0000 7f52c7400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52c0070330 0x7f52c00b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:50.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.989+0000 7f52c7400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52c00cdcc0 0x7f52c0158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:50.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.989+0000 7f52c7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52c015f660 0x7f52c015ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:50.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.989+0000 7f52c7400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f52c015f4b0 con 0x7f52c015f660 2024-04-25T14:14:50.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.989+0000 7f52c7400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f52c015f330 con 0x7f52c00cdcc0 2024-04-25T14:14:50.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.989+0000 7f52c7400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f52c0070740 con 0x7f52c0070330 2024-04-25T14:14:50.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.989+0000 7f52c5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52c015f660 0x7f52c015ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:50.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.989+0000 7f52c5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52c015f660 0x7f52c015ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58290/0 (socket says 172.21.15.86:58290) 2024-04-25T14:14:50.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.989+0000 7f52c5400640 1 -- 172.21.15.86:0/347894997 learned_addr learned my addr 172.21.15.86:0/347894997 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:50.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.989+0000 7f52c4a00640 1 --2- 172.21.15.86:0/347894997 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52c0070330 0x7f52c00b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:50.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.989+0000 7f52b7e00640 1 --2- 172.21.15.86:0/347894997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52c00cdcc0 0x7f52c0158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:50.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.989+0000 7f52c5400640 1 -- 172.21.15.86:0/347894997 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52c0070330 msgr2=0x7f52c00b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:50.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.989+0000 7f52c5400640 1 --2- 172.21.15.86:0/347894997 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52c0070330 0x7f52c00b0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:50.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.989+0000 7f52c5400640 1 -- 172.21.15.86:0/347894997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52c00cdcc0 msgr2=0x7f52c0158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:50.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.989+0000 7f52c5400640 1 --2- 172.21.15.86:0/347894997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52c00cdcc0 0x7f52c0158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:50.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.989+0000 7f52c5400640 1 -- 172.21.15.86:0/347894997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c009db20 con 0x7f52c015f660 2024-04-25T14:14:50.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.989+0000 7f52b7e00640 1 --2- 172.21.15.86:0/347894997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52c00cdcc0 0x7f52c0158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:14:50.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.989+0000 7f52c5400640 1 --2- 172.21.15.86:0/347894997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52c015f660 0x7f52c015ab40 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f52b8007d80 tx=0x7f52b800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:50.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.989+0000 7f52bd600640 1 -- 172.21.15.86:0/347894997 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b8014420 con 0x7f52c015f660 2024-04-25T14:14:50.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.989+0000 7f52bd600640 1 -- 172.21.15.86:0/347894997 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b8019070 con 0x7f52c015f660 2024-04-25T14:14:50.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.989+0000 7f52c7400640 1 -- 172.21.15.86:0/347894997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c009b3e0 con 0x7f52c015f660 2024-04-25T14:14:50.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.989+0000 7f52bd600640 1 -- 172.21.15.86:0/347894997 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b8014800 con 0x7f52c015f660 2024-04-25T14:14:50.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.989+0000 7f52c7400640 1 -- 172.21.15.86:0/347894997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c009b9a0 con 0x7f52c015f660 2024-04-25T14:14:50.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.989+0000 7f52c7400640 1 -- 172.21.15.86:0/347894997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52c0069d00 con 0x7f52c015f660 2024-04-25T14:14:50.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.993+0000 7f52bd600640 1 -- 172.21.15.86:0/347894997 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f52b80149a0 con 0x7f52c015f660 2024-04-25T14:14:50.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.993+0000 7f52bd600640 1 --2- 172.21.15.86:0/347894997 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f529c06f570 0x7f529c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:50.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.993+0000 7f52bd600640 1 -- 172.21.15.86:0/347894997 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f52b80900a0 con 0x7f52c015f660 2024-04-25T14:14:50.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.993+0000 7f52c4a00640 1 --2- 172.21.15.86:0/347894997 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f529c06f570 0x7f529c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:50.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.993+0000 7f52c4a00640 1 --2- 172.21.15.86:0/347894997 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f529c06f570 0x7f529c071a80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f52c0072200 tx=0x7f52b0009040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:50.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:50.997+0000 7f52bd600640 1 -- 172.21.15.86:0/347894997 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f52b805cb10 con 0x7f52c015f660 2024-04-25T14:14:51.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:51.185+0000 7f52c7400640 1 -- 172.21.15.86:0/347894997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52c009d8b0 con 0x7f52c015f660 2024-04-25T14:14:51.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:51.185+0000 7f52bd600640 1 -- 172.21.15.86:0/347894997 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f52b805c410 con 0x7f52c015f660 2024-04-25T14:14:51.182 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:51.182 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:14:51.182 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:14:51.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:51.189+0000 7f52c7400640 1 -- 172.21.15.86:0/347894997 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f529c06f570 msgr2=0x7f529c071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:51.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:51.189+0000 7f52c7400640 1 --2- 172.21.15.86:0/347894997 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f529c06f570 0x7f529c071a80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f52c0072200 tx=0x7f52b0009040 comp rx=0 tx=0).stop 2024-04-25T14:14:51.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:51.189+0000 7f52c7400640 1 -- 172.21.15.86:0/347894997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52c015f660 msgr2=0x7f52c015ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:51.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:51.189+0000 7f52c7400640 1 --2- 172.21.15.86:0/347894997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52c015f660 0x7f52c015ab40 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f52b8007d80 tx=0x7f52b800e4b0 comp rx=0 tx=0).stop 2024-04-25T14:14:51.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:51.189+0000 7f52c7400640 1 -- 172.21.15.86:0/347894997 shutdown_connections 2024-04-25T14:14:51.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:51.189+0000 7f52c7400640 1 --2- 172.21.15.86:0/347894997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52c015f660 0x7f52c015ab40 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:51.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:51.189+0000 7f52c7400640 1 --2- 172.21.15.86:0/347894997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52c00cdcc0 0x7f52c0158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:51.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:51.189+0000 7f52c7400640 1 --2- 172.21.15.86:0/347894997 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f529c06f570 0x7f529c071a80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:51.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:51.189+0000 7f52c7400640 1 --2- 172.21.15.86:0/347894997 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52c0070330 0x7f52c00b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:51.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:51.189+0000 7f52c7400640 1 -- 172.21.15.86:0/347894997 >> 172.21.15.86:0/347894997 conn(0x7f52c00b4650 msgr2=0x7f52c00b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:51.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:51.189+0000 7f52c7400640 1 -- 172.21.15.86:0/347894997 shutdown_connections 2024-04-25T14:14:51.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:51.189+0000 7f52c7400640 1 -- 172.21.15.86:0/347894997 wait complete. 2024-04-25T14:14:51.606 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:14:51.607 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:14:52.608 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:52.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.773+0000 7f16ac800640 1 Processor -- start 2024-04-25T14:14:52.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.773+0000 7f16ac800640 1 -- start start 2024-04-25T14:14:52.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.773+0000 7f16ac800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f16a406fd50 0x7f16a4070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:52.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.773+0000 7f16ac800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16a40b0a60 con 0x7f16a415feb0 2024-04-25T14:14:52.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.773+0000 7f16ac800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16a40ae840 con 0x7f16a415c2a0 2024-04-25T14:14:52.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.773+0000 7f16ac800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f16a406ef90 con 0x7f16a406fd50 2024-04-25T14:14:52.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.773+0000 7f16a9400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f16a406fd50 0x7f16a4070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:52.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.773+0000 7f16a9400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f16a406fd50 0x7f16a4070160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58300/0 (socket says 172.21.15.86:58300) 2024-04-25T14:14:52.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.773+0000 7f16a9400640 1 -- 172.21.15.86:0/1421031143 learned_addr learned my addr 172.21.15.86:0/1421031143 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:52.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.777+0000 7f16a9400640 1 -- 172.21.15.86:0/1421031143 >> v1:172.21.15.155:6789/0 conn(0x7f16a415c2a0 legacy=0x7f16a40ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:52.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.777+0000 7f16a9400640 1 -- 172.21.15.86:0/1421031143 >> v1:172.21.15.86:6789/0 conn(0x7f16a415feb0 legacy=0x7f16a41602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:52.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.777+0000 7f16a9400640 1 -- 172.21.15.86:0/1421031143 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a406f270 con 0x7f16a406fd50 2024-04-25T14:14:52.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.777+0000 7f16a9400640 1 --2- 172.21.15.86:0/1421031143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16a406fd50 0x7f16a4070160 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f1694009870 tx=0x7f169401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ab05a85b553623 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:52.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.777+0000 7f16a8a00640 1 -- 172.21.15.86:0/1421031143 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16940b0070 con 0x7f16a406fd50 2024-04-25T14:14:52.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.777+0000 7f16a8a00640 1 -- 172.21.15.86:0/1421031143 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16940ab600 con 0x7f16a406fd50 2024-04-25T14:14:52.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.777+0000 7f16a8a00640 1 -- 172.21.15.86:0/1421031143 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16940ab9d0 con 0x7f16a406fd50 2024-04-25T14:14:52.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.781+0000 7f16ac800640 1 -- 172.21.15.86:0/1421031143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16a406fd50 msgr2=0x7f16a4070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:52.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.781+0000 7f16ac800640 1 --2- 172.21.15.86:0/1421031143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16a406fd50 0x7f16a4070160 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f1694009870 tx=0x7f169401c460 comp rx=0 tx=0).stop 2024-04-25T14:14:52.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.781+0000 7f16ac800640 1 -- 172.21.15.86:0/1421031143 shutdown_connections 2024-04-25T14:14:52.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.781+0000 7f16ac800640 1 --2- 172.21.15.86:0/1421031143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16a406fd50 0x7f16a4070160 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:52.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.781+0000 7f16ac800640 1 -- 172.21.15.86:0/1421031143 >> 172.21.15.86:0/1421031143 conn(0x7f16a40b4650 msgr2=0x7f16a40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:52.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.781+0000 7f16ac800640 1 -- 172.21.15.86:0/1421031143 shutdown_connections 2024-04-25T14:14:52.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.781+0000 7f16ac800640 1 -- 172.21.15.86:0/1421031143 wait complete. 2024-04-25T14:14:52.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.781+0000 7f16ac800640 1 Processor -- start 2024-04-25T14:14:52.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.781+0000 7f16ac800640 1 -- start start 2024-04-25T14:14:52.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.781+0000 7f16ac800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16a406fd50 0x7f16a4155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:52.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.781+0000 7f16ac800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16a415c2a0 0x7f16a41fb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:52.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.781+0000 7f16ac800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f16a415feb0 0x7f16a41fb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:52.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.781+0000 7f16ac800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f16a415c6b0 con 0x7f16a406fd50 2024-04-25T14:14:52.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.781+0000 7f16ac800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f16a40d0fc0 con 0x7f16a415c2a0 2024-04-25T14:14:52.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.781+0000 7f16ac800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f16a415c830 con 0x7f16a415feb0 2024-04-25T14:14:52.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.785+0000 7f16a9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16a406fd50 0x7f16a4155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:52.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.785+0000 7f16a9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16a406fd50 0x7f16a4155780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58310/0 (socket says 172.21.15.86:58310) 2024-04-25T14:14:52.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.785+0000 7f16a9e00640 1 -- 172.21.15.86:0/2880330982 learned_addr learned my addr 172.21.15.86:0/2880330982 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:52.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.785+0000 7f16a9e00640 1 -- 172.21.15.86:0/2880330982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f16a415feb0 msgr2=0x7f16a41fb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:52.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.785+0000 7f16a9e00640 1 --2- 172.21.15.86:0/2880330982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f16a415feb0 0x7f16a41fb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:52.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.785+0000 7f16a9e00640 1 -- 172.21.15.86:0/2880330982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16a415c2a0 msgr2=0x7f16a41fb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:52.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.785+0000 7f16a9400640 1 --2- 172.21.15.86:0/2880330982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16a415c2a0 0x7f16a41fb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:52.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.785+0000 7f16a9e00640 1 --2- 172.21.15.86:0/2880330982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16a415c2a0 0x7f16a41fb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:52.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.785+0000 7f16a9e00640 1 -- 172.21.15.86:0/2880330982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a40cecc0 con 0x7f16a406fd50 2024-04-25T14:14:52.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.785+0000 7f16a9400640 1 --2- 172.21.15.86:0/2880330982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16a415c2a0 0x7f16a41fb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:14:52.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.785+0000 7f16a9e00640 1 --2- 172.21.15.86:0/2880330982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16a406fd50 0x7f16a4155780 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f169800b3f0 tx=0x7f169800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:52.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.785+0000 7f169e000640 1 -- 172.21.15.86:0/2880330982 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1698015020 con 0x7f16a406fd50 2024-04-25T14:14:52.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.785+0000 7f169e000640 1 -- 172.21.15.86:0/2880330982 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1698011830 con 0x7f16a406fd50 2024-04-25T14:14:52.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.785+0000 7f169e000640 1 -- 172.21.15.86:0/2880330982 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1698011c00 con 0x7f16a406fd50 2024-04-25T14:14:52.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.785+0000 7f16ac800640 1 -- 172.21.15.86:0/2880330982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a4155cc0 con 0x7f16a406fd50 2024-04-25T14:14:52.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.785+0000 7f16ac800640 1 -- 172.21.15.86:0/2880330982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a4155f70 con 0x7f16a406fd50 2024-04-25T14:14:52.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.785+0000 7f16ac800640 1 -- 172.21.15.86:0/2880330982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16a4069d00 con 0x7f16a406fd50 2024-04-25T14:14:52.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.785+0000 7f169e000640 1 -- 172.21.15.86:0/2880330982 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f1698018020 con 0x7f16a406fd50 2024-04-25T14:14:52.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.789+0000 7f169e000640 1 --2- 172.21.15.86:0/2880330982 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f168806f5a0 0x7f1688071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:52.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.789+0000 7f169e000640 1 -- 172.21.15.86:0/2880330982 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f169808f270 con 0x7f16a406fd50 2024-04-25T14:14:52.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.789+0000 7f16a9400640 1 --2- 172.21.15.86:0/2880330982 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f168806f5a0 0x7f1688071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:52.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.789+0000 7f16a9400640 1 --2- 172.21.15.86:0/2880330982 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f168806f5a0 0x7f1688071ab0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f16940ad000 tx=0x7f1694008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:52.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.793+0000 7f169e000640 1 -- 172.21.15.86:0/2880330982 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f169801f030 con 0x7f16a406fd50 2024-04-25T14:14:52.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.981+0000 7f16ac800640 1 -- 172.21.15.86:0/2880330982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16a4076bf0 con 0x7f16a406fd50 2024-04-25T14:14:52.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.985+0000 7f169e000640 1 -- 172.21.15.86:0/2880330982 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f169805bde0 con 0x7f16a406fd50 2024-04-25T14:14:52.980 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:52.980 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:14:52.980 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:14:52.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.985+0000 7f16ac800640 1 -- 172.21.15.86:0/2880330982 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f168806f5a0 msgr2=0x7f1688071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:52.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.985+0000 7f16ac800640 1 --2- 172.21.15.86:0/2880330982 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f168806f5a0 0x7f1688071ab0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f16940ad000 tx=0x7f1694008040 comp rx=0 tx=0).stop 2024-04-25T14:14:52.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.985+0000 7f16ac800640 1 -- 172.21.15.86:0/2880330982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16a406fd50 msgr2=0x7f16a4155780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:52.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.985+0000 7f16ac800640 1 --2- 172.21.15.86:0/2880330982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16a406fd50 0x7f16a4155780 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f169800b3f0 tx=0x7f169800b8b0 comp rx=0 tx=0).stop 2024-04-25T14:14:52.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.985+0000 7f16ac800640 1 -- 172.21.15.86:0/2880330982 shutdown_connections 2024-04-25T14:14:52.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.985+0000 7f16ac800640 1 --2- 172.21.15.86:0/2880330982 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f168806f5a0 0x7f1688071ab0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:52.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.985+0000 7f16ac800640 1 --2- 172.21.15.86:0/2880330982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f16a415feb0 0x7f16a41fb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:52.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.985+0000 7f16ac800640 1 --2- 172.21.15.86:0/2880330982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16a415c2a0 0x7f16a41fb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:52.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.985+0000 7f16ac800640 1 --2- 172.21.15.86:0/2880330982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16a406fd50 0x7f16a4155780 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:52.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.985+0000 7f16ac800640 1 -- 172.21.15.86:0/2880330982 >> 172.21.15.86:0/2880330982 conn(0x7f16a40b4650 msgr2=0x7f16a415c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:52.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.985+0000 7f16ac800640 1 -- 172.21.15.86:0/2880330982 shutdown_connections 2024-04-25T14:14:52.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:52.985+0000 7f16ac800640 1 -- 172.21.15.86:0/2880330982 wait complete. 2024-04-25T14:14:53.404 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:14:53.404 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:14:54.405 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:54.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.565+0000 7f7213a00640 1 Processor -- start 2024-04-25T14:14:54.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.565+0000 7f7213a00640 1 -- start start 2024-04-25T14:14:54.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.569+0000 7f7213a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f720c0cfca0 0x7f720c06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:54.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.569+0000 7f7213a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f720c0d16e0 0x7f720c0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:54.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.569+0000 7f7213a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f720c06c670 0x7f720c06ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:54.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.569+0000 7f7213a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f720c070470 con 0x7f720c0d16e0 2024-04-25T14:14:54.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.569+0000 7f7213a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f720c0d2030 con 0x7f720c0cfca0 2024-04-25T14:14:54.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.569+0000 7f7213a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f720c072bd0 con 0x7f720c06c670 2024-04-25T14:14:54.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.569+0000 7f720be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f720c0d16e0 0x7f720c0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:54.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.569+0000 7f720be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f720c0d16e0 0x7f720c0d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58312/0 (socket says 172.21.15.86:58312) 2024-04-25T14:14:54.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.569+0000 7f720be00640 1 -- 172.21.15.86:0/2087073892 learned_addr learned my addr 172.21.15.86:0/2087073892 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:54.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.569+0000 7f7211a00640 1 --2- 172.21.15.86:0/2087073892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f720c06c670 0x7f720c06ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:54.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.569+0000 7f7211000640 1 --2- 172.21.15.86:0/2087073892 >> v2:172.21.15.155:3300/0 conn(0x7f720c0cfca0 0x7f720c06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:54.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.569+0000 7f720be00640 1 -- 172.21.15.86:0/2087073892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f720c06c670 msgr2=0x7f720c06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:54.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.569+0000 7f720be00640 1 --2- 172.21.15.86:0/2087073892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f720c06c670 0x7f720c06ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:54.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.569+0000 7f720be00640 1 -- 172.21.15.86:0/2087073892 >> v2:172.21.15.155:3300/0 conn(0x7f720c0cfca0 msgr2=0x7f720c06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:54.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.569+0000 7f720be00640 1 --2- 172.21.15.86:0/2087073892 >> v2:172.21.15.155:3300/0 conn(0x7f720c0cfca0 0x7f720c06f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:54.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.569+0000 7f720be00640 1 -- 172.21.15.86:0/2087073892 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720c06f930 con 0x7f720c0d16e0 2024-04-25T14:14:54.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.573+0000 7f720be00640 1 --2- 172.21.15.86:0/2087073892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f720c0d16e0 0x7f720c0d1af0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f720c0d22a0 tx=0x7f71f801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806ed21c558f5f60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:54.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.573+0000 7f720b400640 1 -- 172.21.15.86:0/2087073892 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f80b0070 con 0x7f720c0d16e0 2024-04-25T14:14:54.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.573+0000 7f720b400640 1 -- 172.21.15.86:0/2087073892 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71f80ab600 con 0x7f720c0d16e0 2024-04-25T14:14:54.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.573+0000 7f720b400640 1 -- 172.21.15.86:0/2087073892 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f80ab9d0 con 0x7f720c0d16e0 2024-04-25T14:14:54.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.573+0000 7f7213a00640 1 -- 172.21.15.86:0/2087073892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f720c0d16e0 msgr2=0x7f720c0d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:54.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.573+0000 7f7213a00640 1 --2- 172.21.15.86:0/2087073892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f720c0d16e0 0x7f720c0d1af0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f720c0d22a0 tx=0x7f71f801c520 comp rx=0 tx=0).stop 2024-04-25T14:14:54.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.573+0000 7f7213a00640 1 -- 172.21.15.86:0/2087073892 shutdown_connections 2024-04-25T14:14:54.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.573+0000 7f7213a00640 1 --2- 172.21.15.86:0/2087073892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f720c06c670 0x7f720c06ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:54.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.573+0000 7f7213a00640 1 --2- 172.21.15.86:0/2087073892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f720c0d16e0 0x7f720c0d1af0 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:54.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.573+0000 7f7213a00640 1 --2- 172.21.15.86:0/2087073892 >> v2:172.21.15.155:3300/0 conn(0x7f720c0cfca0 0x7f720c06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:54.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.573+0000 7f7213a00640 1 -- 172.21.15.86:0/2087073892 >> 172.21.15.86:0/2087073892 conn(0x7f720c0b4650 msgr2=0x7f720c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:54.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.573+0000 7f7213a00640 1 -- 172.21.15.86:0/2087073892 shutdown_connections 2024-04-25T14:14:54.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.573+0000 7f7213a00640 1 -- 172.21.15.86:0/2087073892 wait complete. 2024-04-25T14:14:54.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.573+0000 7f7213a00640 1 Processor -- start 2024-04-25T14:14:54.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.573+0000 7f7213a00640 1 -- start start 2024-04-25T14:14:54.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.577+0000 7f7213a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f720c06c670 0x7f720c09b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:54.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.577+0000 7f7213a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f720c0cfca0 0x7f720c0ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:54.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.577+0000 7f7213a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f720c0d16e0 0x7f720c099990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:54.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.577+0000 7f7213a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f720c0ce570 con 0x7f720c06c670 2024-04-25T14:14:54.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.577+0000 7f7213a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f720c06d3d0 con 0x7f720c0cfca0 2024-04-25T14:14:54.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.577+0000 7f7213a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f720c0ce6f0 con 0x7f720c0d16e0 2024-04-25T14:14:54.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.577+0000 7f7211000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f720c06c670 0x7f720c09b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:54.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.577+0000 7f7211000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f720c06c670 0x7f720c09b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58322/0 (socket says 172.21.15.86:58322) 2024-04-25T14:14:54.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.577+0000 7f7211000640 1 -- 172.21.15.86:0/2738948216 learned_addr learned my addr 172.21.15.86:0/2738948216 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:54.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.577+0000 7f720be00640 1 --2- 172.21.15.86:0/2738948216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f720c0cfca0 0x7f720c0ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:54.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.577+0000 7f7211a00640 1 --2- 172.21.15.86:0/2738948216 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f720c0d16e0 0x7f720c099990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:54.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.577+0000 7f7211000640 1 -- 172.21.15.86:0/2738948216 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f720c0d16e0 msgr2=0x7f720c099990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:54.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.577+0000 7f7211000640 1 --2- 172.21.15.86:0/2738948216 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f720c0d16e0 0x7f720c099990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:54.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.577+0000 7f7211000640 1 -- 172.21.15.86:0/2738948216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f720c0cfca0 msgr2=0x7f720c0ca6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:54.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.577+0000 7f7211000640 1 --2- 172.21.15.86:0/2738948216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f720c0cfca0 0x7f720c0ca6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:54.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.577+0000 7f7211000640 1 -- 172.21.15.86:0/2738948216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720c09b960 con 0x7f720c06c670 2024-04-25T14:14:54.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.577+0000 7f7211a00640 1 --2- 172.21.15.86:0/2738948216 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f720c0d16e0 0x7f720c099990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:54.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.577+0000 7f7211000640 1 --2- 172.21.15.86:0/2738948216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f720c06c670 0x7f720c09b2b0 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f720000d600 tx=0x7f720000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:54.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.577+0000 7f7208c00640 1 -- 172.21.15.86:0/2738948216 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7200004240 con 0x7f720c06c670 2024-04-25T14:14:54.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.577+0000 7f7208c00640 1 -- 172.21.15.86:0/2738948216 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72000043e0 con 0x7f720c06c670 2024-04-25T14:14:54.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.577+0000 7f7213a00640 1 -- 172.21.15.86:0/2738948216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f720c09cfe0 con 0x7f720c06c670 2024-04-25T14:14:54.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.577+0000 7f7208c00640 1 -- 172.21.15.86:0/2738948216 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7200003840 con 0x7f720c06c670 2024-04-25T14:14:54.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.577+0000 7f7213a00640 1 -- 172.21.15.86:0/2738948216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f720c09d5a0 con 0x7f720c06c670 2024-04-25T14:14:54.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.577+0000 7f7213a00640 1 -- 172.21.15.86:0/2738948216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f720c09f4b0 con 0x7f720c06c670 2024-04-25T14:14:54.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.581+0000 7f7208c00640 1 -- 172.21.15.86:0/2738948216 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f72000039e0 con 0x7f720c06c670 2024-04-25T14:14:54.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.581+0000 7f7208c00640 1 --2- 172.21.15.86:0/2738948216 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f71ec06f5a0 0x7f71ec071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:54.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.581+0000 7f7208c00640 1 -- 172.21.15.86:0/2738948216 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f720008f890 con 0x7f720c06c670 2024-04-25T14:14:54.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.581+0000 7f720be00640 1 --2- 172.21.15.86:0/2738948216 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f71ec06f5a0 0x7f71ec071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:54.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.581+0000 7f720be00640 1 --2- 172.21.15.86:0/2738948216 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f71ec06f5a0 0x7f71ec071ab0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f71f80ad000 tx=0x7f71f8008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:54.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.585+0000 7f7208c00640 1 -- 172.21.15.86:0/2738948216 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f720005c410 con 0x7f720c06c670 2024-04-25T14:14:54.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.777+0000 7f7213a00640 1 -- 172.21.15.86:0/2738948216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f720c069d00 con 0x7f720c06c670 2024-04-25T14:14:54.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.777+0000 7f7208c00640 1 -- 172.21.15.86:0/2738948216 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f720005bd10 con 0x7f720c06c670 2024-04-25T14:14:54.773 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:54.773 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:14:54.773 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:14:54.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.777+0000 7f7213a00640 1 -- 172.21.15.86:0/2738948216 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f71ec06f5a0 msgr2=0x7f71ec071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:54.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.777+0000 7f7213a00640 1 --2- 172.21.15.86:0/2738948216 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f71ec06f5a0 0x7f71ec071ab0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f71f80ad000 tx=0x7f71f8008040 comp rx=0 tx=0).stop 2024-04-25T14:14:54.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.777+0000 7f7213a00640 1 -- 172.21.15.86:0/2738948216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f720c06c670 msgr2=0x7f720c09b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:54.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.777+0000 7f7213a00640 1 --2- 172.21.15.86:0/2738948216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f720c06c670 0x7f720c09b2b0 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f720000d600 tx=0x7f720000dac0 comp rx=0 tx=0).stop 2024-04-25T14:14:54.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.777+0000 7f7213a00640 1 -- 172.21.15.86:0/2738948216 shutdown_connections 2024-04-25T14:14:54.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.777+0000 7f7213a00640 1 --2- 172.21.15.86:0/2738948216 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f71ec06f5a0 0x7f71ec071ab0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:54.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.777+0000 7f7213a00640 1 --2- 172.21.15.86:0/2738948216 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f720c0d16e0 0x7f720c099990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:54.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.777+0000 7f7213a00640 1 --2- 172.21.15.86:0/2738948216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f720c0cfca0 0x7f720c0ca6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:54.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.777+0000 7f7213a00640 1 --2- 172.21.15.86:0/2738948216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f720c06c670 0x7f720c09b2b0 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:54.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.777+0000 7f7213a00640 1 -- 172.21.15.86:0/2738948216 >> 172.21.15.86:0/2738948216 conn(0x7f720c0b4650 msgr2=0x7f720c0cd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:54.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.777+0000 7f7213a00640 1 -- 172.21.15.86:0/2738948216 shutdown_connections 2024-04-25T14:14:54.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:54.777+0000 7f7213a00640 1 -- 172.21.15.86:0/2738948216 wait complete. 2024-04-25T14:14:55.197 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:14:55.197 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:14:56.199 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:56.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.361+0000 7f6a2b800640 1 Processor -- start 2024-04-25T14:14:56.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.361+0000 7f6a2b800640 1 -- start start 2024-04-25T14:14:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.365+0000 7f6a2b800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a240d16e0 0x7f6a240d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.365+0000 7f6a2b800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6a240730b0 0x7f6a240734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.365+0000 7f6a2b800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a240d2030 con 0x7f6a240cfca0 2024-04-25T14:14:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.365+0000 7f6a2b800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f6a24070470 con 0x7f6a240730b0 2024-04-25T14:14:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.365+0000 7f6a2b800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6a24072bd0 con 0x7f6a240d16e0 2024-04-25T14:14:56.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.365+0000 7f6a28e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6a240cfca0 0x7f6a2406f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36270/0 (socket says 172.21.15.86:36270) 2024-04-25T14:14:56.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.365+0000 7f6a28e00640 1 -- 172.21.15.86:0/3218474675 learned_addr learned my addr 172.21.15.86:0/3218474675 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:14:56.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.365+0000 7f6a1be00640 1 --2- 172.21.15.86:0/3218474675 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a240d16e0 0x7f6a240d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:56.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.365+0000 7f6a29800640 1 --2- 172.21.15.86:0/3218474675 >> v2:172.21.15.155:3300/0 conn(0x7f6a240730b0 0x7f6a240734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:56.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.365+0000 7f6a23e00640 1 -- 172.21.15.86:0/3218474675 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 330452864 0 0) 0x7f6a14001120 con 0x7f6a240cfca0 2024-04-25T14:14:56.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.365+0000 7f6a23e00640 1 -- 172.21.15.86:0/3218474675 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a08005740 con 0x7f6a240cfca0 2024-04-25T14:14:56.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.365+0000 7f6a1be00640 1 -- 172.21.15.86:0/3218474675 >> v2:172.21.15.155:3300/0 conn(0x7f6a240730b0 msgr2=0x7f6a240734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:56.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.365+0000 7f6a1be00640 1 --2- 172.21.15.86:0/3218474675 >> v2:172.21.15.155:3300/0 conn(0x7f6a240730b0 0x7f6a240734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:56.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.365+0000 7f6a1be00640 1 -- 172.21.15.86:0/3218474675 >> v1:172.21.15.86:6789/0 conn(0x7f6a240cfca0 legacy=0x7f6a2406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:56.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.365+0000 7f6a1be00640 1 -- 172.21.15.86:0/3218474675 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2406fbd0 con 0x7f6a240d16e0 2024-04-25T14:14:56.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.369+0000 7f6a1be00640 1 --2- 172.21.15.86:0/3218474675 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a240d16e0 0x7f6a240d1af0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c0097c0 tx=0x7f6a0c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=899c996be9c8f5eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:56.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.369+0000 7f6a23e00640 1 -- 172.21.15.86:0/3218474675 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c0b1070 con 0x7f6a240d16e0 2024-04-25T14:14:56.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.369+0000 7f6a23e00640 1 -- 172.21.15.86:0/3218474675 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a0c0ac410 con 0x7f6a240d16e0 2024-04-25T14:14:56.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.369+0000 7f6a23e00640 1 -- 172.21.15.86:0/3218474675 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c0ac7c0 con 0x7f6a240d16e0 2024-04-25T14:14:56.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.369+0000 7f6a2b800640 1 -- 172.21.15.86:0/3218474675 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a240d16e0 msgr2=0x7f6a240d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:56.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.369+0000 7f6a2b800640 1 --2- 172.21.15.86:0/3218474675 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a240d16e0 0x7f6a240d1af0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c0097c0 tx=0x7f6a0c01c4a0 comp rx=0 tx=0).stop 2024-04-25T14:14:56.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.369+0000 7f6a2b800640 1 -- 172.21.15.86:0/3218474675 shutdown_connections 2024-04-25T14:14:56.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.369+0000 7f6a2b800640 1 --2- 172.21.15.86:0/3218474675 >> v2:172.21.15.155:3300/0 conn(0x7f6a240730b0 0x7f6a240734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:56.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.369+0000 7f6a2b800640 1 --2- 172.21.15.86:0/3218474675 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a240d16e0 0x7f6a240d1af0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:56.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.369+0000 7f6a2b800640 1 -- 172.21.15.86:0/3218474675 >> 172.21.15.86:0/3218474675 conn(0x7f6a240b4650 msgr2=0x7f6a240b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:56.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.369+0000 7f6a2b800640 1 -- 172.21.15.86:0/3218474675 shutdown_connections 2024-04-25T14:14:56.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.369+0000 7f6a2b800640 1 -- 172.21.15.86:0/3218474675 wait complete. 2024-04-25T14:14:56.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.373+0000 7f6a2b800640 1 Processor -- start 2024-04-25T14:14:56.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.373+0000 7f6a2b800640 1 -- start start 2024-04-25T14:14:56.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.373+0000 7f6a2b800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a240730b0 0x7f6a240af880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:56.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.373+0000 7f6a2b800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a240cfca0 0x7f6a2409bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:56.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.373+0000 7f6a2b800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a240d16e0 0x7f6a2409c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:56.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.373+0000 7f6a2b800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6a240ce170 con 0x7f6a240730b0 2024-04-25T14:14:56.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.373+0000 7f6a2b800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6a240cdff0 con 0x7f6a240d16e0 2024-04-25T14:14:56.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.373+0000 7f6a2b800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6a240ce2f0 con 0x7f6a240cfca0 2024-04-25T14:14:56.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.373+0000 7f6a28e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a240730b0 0x7f6a240af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:56.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.373+0000 7f6a28e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a240730b0 0x7f6a240af880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58324/0 (socket says 172.21.15.86:58324) 2024-04-25T14:14:56.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.373+0000 7f6a28e00640 1 -- 172.21.15.86:0/1005604982 learned_addr learned my addr 172.21.15.86:0/1005604982 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:56.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.373+0000 7f6a1be00640 1 --2- 172.21.15.86:0/1005604982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a240cfca0 0x7f6a2409bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:56.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.373+0000 7f6a29800640 1 --2- 172.21.15.86:0/1005604982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a240d16e0 0x7f6a2409c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:56.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.373+0000 7f6a28e00640 1 -- 172.21.15.86:0/1005604982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a240cfca0 msgr2=0x7f6a2409bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:56.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.373+0000 7f6a28e00640 1 --2- 172.21.15.86:0/1005604982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a240cfca0 0x7f6a2409bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:56.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.373+0000 7f6a28e00640 1 -- 172.21.15.86:0/1005604982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a240d16e0 msgr2=0x7f6a2409c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:56.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.373+0000 7f6a28e00640 1 --2- 172.21.15.86:0/1005604982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a240d16e0 0x7f6a2409c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:56.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.373+0000 7f6a28e00640 1 -- 172.21.15.86:0/1005604982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a24159070 con 0x7f6a240730b0 2024-04-25T14:14:56.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.373+0000 7f6a28e00640 1 --2- 172.21.15.86:0/1005604982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a240730b0 0x7f6a240af880 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400c420 tx=0x7f6a1400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:56.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.373+0000 7f6a21600640 1 -- 172.21.15.86:0/1005604982 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a14017070 con 0x7f6a240730b0 2024-04-25T14:14:56.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.373+0000 7f6a21600640 1 -- 172.21.15.86:0/1005604982 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a14005cf0 con 0x7f6a240730b0 2024-04-25T14:14:56.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.373+0000 7f6a21600640 1 -- 172.21.15.86:0/1005604982 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a140072f0 con 0x7f6a240730b0 2024-04-25T14:14:56.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.373+0000 7f6a2b800640 1 -- 172.21.15.86:0/1005604982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a2415e140 con 0x7f6a240730b0 2024-04-25T14:14:56.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.373+0000 7f6a2b800640 1 -- 172.21.15.86:0/1005604982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a2409b450 con 0x7f6a240730b0 2024-04-25T14:14:56.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.377+0000 7f6a2b800640 1 -- 172.21.15.86:0/1005604982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a2415e310 con 0x7f6a240730b0 2024-04-25T14:14:56.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.377+0000 7f6a21600640 1 -- 172.21.15.86:0/1005604982 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f6a1401a070 con 0x7f6a240730b0 2024-04-25T14:14:56.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.377+0000 7f6a21600640 1 --2- 172.21.15.86:0/1005604982 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f6a0006f5a0 0x7f6a00071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:56.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.377+0000 7f6a21600640 1 -- 172.21.15.86:0/1005604982 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f6a1408eb90 con 0x7f6a240730b0 2024-04-25T14:14:56.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.377+0000 7f6a1be00640 1 --2- 172.21.15.86:0/1005604982 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f6a0006f5a0 0x7f6a00071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:56.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.377+0000 7f6a1be00640 1 --2- 172.21.15.86:0/1005604982 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f6a0006f5a0 0x7f6a00071ab0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c0ae000 tx=0x7f6a0c008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:56.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.381+0000 7f6a21600640 1 -- 172.21.15.86:0/1005604982 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6a24159070 con 0x7f6a240730b0 2024-04-25T14:14:56.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.573+0000 7f6a2b800640 1 -- 172.21.15.86:0/1005604982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a2409f4b0 con 0x7f6a240730b0 2024-04-25T14:14:56.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.573+0000 7f6a21600640 1 -- 172.21.15.86:0/1005604982 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f6a240d2030 con 0x7f6a240730b0 2024-04-25T14:14:56.569 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:56.569 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:14:56.569 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:14:56.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.573+0000 7f6a2b800640 1 -- 172.21.15.86:0/1005604982 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f6a0006f5a0 msgr2=0x7f6a00071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:56.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.573+0000 7f6a2b800640 1 --2- 172.21.15.86:0/1005604982 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f6a0006f5a0 0x7f6a00071ab0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c0ae000 tx=0x7f6a0c008040 comp rx=0 tx=0).stop 2024-04-25T14:14:56.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.573+0000 7f6a2b800640 1 -- 172.21.15.86:0/1005604982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a240730b0 msgr2=0x7f6a240af880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:56.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.573+0000 7f6a2b800640 1 --2- 172.21.15.86:0/1005604982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a240730b0 0x7f6a240af880 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400c420 tx=0x7f6a1400c8e0 comp rx=0 tx=0).stop 2024-04-25T14:14:56.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.573+0000 7f6a2b800640 1 -- 172.21.15.86:0/1005604982 shutdown_connections 2024-04-25T14:14:56.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.573+0000 7f6a2b800640 1 --2- 172.21.15.86:0/1005604982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a240d16e0 0x7f6a2409c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:56.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.573+0000 7f6a2b800640 1 --2- 172.21.15.86:0/1005604982 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f6a0006f5a0 0x7f6a00071ab0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:56.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.573+0000 7f6a2b800640 1 --2- 172.21.15.86:0/1005604982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a240cfca0 0x7f6a2409bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:56.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.573+0000 7f6a2b800640 1 --2- 172.21.15.86:0/1005604982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a240730b0 0x7f6a240af880 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:56.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.573+0000 7f6a2b800640 1 -- 172.21.15.86:0/1005604982 >> 172.21.15.86:0/1005604982 conn(0x7f6a240b4650 msgr2=0x7f6a240cc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:56.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.573+0000 7f6a2b800640 1 -- 172.21.15.86:0/1005604982 shutdown_connections 2024-04-25T14:14:56.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:56.573+0000 7f6a2b800640 1 -- 172.21.15.86:0/1005604982 wait complete. 2024-04-25T14:14:56.995 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:14:56.995 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:14:57.996 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:58.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.157+0000 7fc93c000640 1 Processor -- start 2024-04-25T14:14:58.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.157+0000 7fc93c000640 1 -- start start 2024-04-25T14:14:58.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc93c000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc93415dc90 0x7fc934161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:58.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc93c000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9340cd8d0 con 0x7fc934071530 2024-04-25T14:14:58.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc93c000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9340ce970 con 0x7fc9340d2c80 2024-04-25T14:14:58.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc93c000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc9340ce720 con 0x7fc93415dc90 2024-04-25T14:14:58.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc938c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc934071530 0x7fc9341616e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36286/0 (socket says 172.21.15.86:36286) 2024-04-25T14:14:58.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc938c00640 1 -- 172.21.15.86:0/266340076 learned_addr learned my addr 172.21.15.86:0/266340076 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:14:58.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc93a000640 1 --2- 172.21.15.86:0/266340076 >> v2:172.21.15.155:3300/0 conn(0x7fc93415dc90 0x7fc934161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:58.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc933e00640 1 -- 172.21.15.86:0/266340076 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3731172172 0 0) 0x7fc9340cd8d0 con 0x7fc934071530 2024-04-25T14:14:58.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc933e00640 1 -- 172.21.15.86:0/266340076 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc91c003530 con 0x7fc934071530 2024-04-25T14:14:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc933e00640 1 -- 172.21.15.86:0/266340076 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2962755919 0 0) 0x7fc9340ce970 con 0x7fc9340d2c80 2024-04-25T14:14:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc933e00640 1 -- 172.21.15.86:0/266340076 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9340cd8d0 con 0x7fc9340d2c80 2024-04-25T14:14:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc933e00640 1 -- 172.21.15.86:0/266340076 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3310649497 0 0) 0x7fc91c003530 con 0x7fc934071530 2024-04-25T14:14:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc933e00640 1 -- 172.21.15.86:0/266340076 >> v2:172.21.15.155:3300/0 conn(0x7fc93415dc90 msgr2=0x7fc934161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc933e00640 1 --2- 172.21.15.86:0/266340076 >> v2:172.21.15.155:3300/0 conn(0x7fc93415dc90 0x7fc934161df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc933e00640 1 -- 172.21.15.86:0/266340076 >> v1:172.21.15.155:6789/0 conn(0x7fc9340d2c80 legacy=0x7fc934070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc933e00640 1 -- 172.21.15.86:0/266340076 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9340d1be0 con 0x7fc934071530 2024-04-25T14:14:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc933e00640 1 -- 172.21.15.86:0/266340076 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fc920001800 con 0x7fc934071530 2024-04-25T14:14:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc933e00640 1 -- 172.21.15.86:0/266340076 >> v1:172.21.15.86:6789/0 conn(0x7fc934071530 legacy=0x7fc9341616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc933e00640 1 --2- 172.21.15.86:0/266340076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc91c003bc0 0x7fc91c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc933e00640 1 --2- 172.21.15.86:0/266340076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc91c004810 0x7fc91c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc933e00640 1 --2- 172.21.15.86:0/266340076 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc91c005220 0x7fc91c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc933e00640 1 -- 172.21.15.86:0/266340076 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc9340ce720 con 0x7fc91c003bc0 2024-04-25T14:14:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc933e00640 1 -- 172.21.15.86:0/266340076 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc91c00c120 con 0x7fc91c004810 2024-04-25T14:14:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc933e00640 1 -- 172.21.15.86:0/266340076 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc91c00c320 con 0x7fc91c005220 2024-04-25T14:14:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc939600640 1 --2- 172.21.15.86:0/266340076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc91c003bc0 0x7fc91c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc93a000640 1 --2- 172.21.15.86:0/266340076 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc91c005220 0x7fc91c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc938c00640 1 --2- 172.21.15.86:0/266340076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc91c004810 0x7fc91c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc939600640 1 -- 172.21.15.86:0/266340076 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc91c005220 msgr2=0x7fc91c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc939600640 1 --2- 172.21.15.86:0/266340076 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc91c005220 0x7fc91c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc939600640 1 -- 172.21.15.86:0/266340076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc91c004810 msgr2=0x7fc91c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc939600640 1 --2- 172.21.15.86:0/266340076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc91c004810 0x7fc91c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc939600640 1 -- 172.21.15.86:0/266340076 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc91c003530 con 0x7fc91c003bc0 2024-04-25T14:14:58.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.161+0000 7fc93a000640 1 -- 172.21.15.86:0/266340076 reap_dead start 2024-04-25T14:14:58.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.165+0000 7fc939600640 1 --2- 172.21.15.86:0/266340076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc91c003bc0 0x7fc91c004010 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fc9280018d0 tx=0x7fc92801ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43faa8a96ac4882d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:58.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.165+0000 7fc933e00640 1 -- 172.21.15.86:0/266340076 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc928001520 con 0x7fc91c003bc0 2024-04-25T14:14:58.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.165+0000 7fc933e00640 1 -- 172.21.15.86:0/266340076 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9280052f0 con 0x7fc91c003bc0 2024-04-25T14:14:58.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.165+0000 7fc933e00640 1 -- 172.21.15.86:0/266340076 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9280a88f0 con 0x7fc91c003bc0 2024-04-25T14:14:58.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.165+0000 7fc93c000640 1 -- 172.21.15.86:0/266340076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc91c003bc0 msgr2=0x7fc91c004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:58.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.165+0000 7fc93c000640 1 --2- 172.21.15.86:0/266340076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc91c003bc0 0x7fc91c004010 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fc9280018d0 tx=0x7fc92801ec90 comp rx=0 tx=0).stop 2024-04-25T14:14:58.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.165+0000 7fc93c000640 1 -- 172.21.15.86:0/266340076 shutdown_connections 2024-04-25T14:14:58.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.165+0000 7fc93c000640 1 --2- 172.21.15.86:0/266340076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc91c003bc0 0x7fc91c004010 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:58.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.165+0000 7fc93c000640 1 -- 172.21.15.86:0/266340076 >> 172.21.15.86:0/266340076 conn(0x7fc9340cffc0 msgr2=0x7fc9340d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:58.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.165+0000 7fc93c000640 1 -- 172.21.15.86:0/266340076 shutdown_connections 2024-04-25T14:14:58.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.165+0000 7fc93c000640 1 -- 172.21.15.86:0/266340076 wait complete. 2024-04-25T14:14:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.169+0000 7fc93c000640 1 Processor -- start 2024-04-25T14:14:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.169+0000 7fc93c000640 1 -- start start 2024-04-25T14:14:58.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.169+0000 7fc93c000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc91c003bc0 0x7fc93414bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:58.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.169+0000 7fc93c000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9340c9320 0x7fc9340c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:58.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.169+0000 7fc93c000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc934152020 0x7fc934152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:58.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.169+0000 7fc93c000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc9340d2480 con 0x7fc934152020 2024-04-25T14:14:58.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.169+0000 7fc93c000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc9340d21a0 con 0x7fc9340c9320 2024-04-25T14:14:58.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.169+0000 7fc93c000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc9340d29a0 con 0x7fc91c003bc0 2024-04-25T14:14:58.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.169+0000 7fc93a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc934152020 0x7fc934152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:58.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.169+0000 7fc93a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc934152020 0x7fc934152430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58342/0 (socket says 172.21.15.86:58342) 2024-04-25T14:14:58.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.169+0000 7fc93a000640 1 -- 172.21.15.86:0/1519907444 learned_addr learned my addr 172.21.15.86:0/1519907444 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:58.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.169+0000 7fc939600640 1 --2- 172.21.15.86:0/1519907444 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc91c003bc0 0x7fc93414bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:58.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.169+0000 7fc938c00640 1 --2- 172.21.15.86:0/1519907444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9340c9320 0x7fc9340c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:58.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.169+0000 7fc93a000640 1 -- 172.21.15.86:0/1519907444 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc91c003bc0 msgr2=0x7fc93414bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:58.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.169+0000 7fc93a000640 1 --2- 172.21.15.86:0/1519907444 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc91c003bc0 0x7fc93414bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:58.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.169+0000 7fc93a000640 1 -- 172.21.15.86:0/1519907444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9340c9320 msgr2=0x7fc9340c9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:58.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.169+0000 7fc93a000640 1 --2- 172.21.15.86:0/1519907444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9340c9320 0x7fc9340c9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:58.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.169+0000 7fc93a000640 1 -- 172.21.15.86:0/1519907444 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc934152970 con 0x7fc934152020 2024-04-25T14:14:58.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.169+0000 7fc938c00640 1 --2- 172.21.15.86:0/1519907444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9340c9320 0x7fc9340c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:58.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.169+0000 7fc93a000640 1 --2- 172.21.15.86:0/1519907444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc934152020 0x7fc934152430 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7fc934073a90 tx=0x7fc92c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:58.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.173+0000 7fc931600640 1 -- 172.21.15.86:0/1519907444 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c011db0 con 0x7fc934152020 2024-04-25T14:14:58.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.173+0000 7fc931600640 1 -- 172.21.15.86:0/1519907444 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc92c01b070 con 0x7fc934152020 2024-04-25T14:14:58.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.173+0000 7fc93c000640 1 -- 172.21.15.86:0/1519907444 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc934161100 con 0x7fc934152020 2024-04-25T14:14:58.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.173+0000 7fc931600640 1 -- 172.21.15.86:0/1519907444 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c0039a0 con 0x7fc934152020 2024-04-25T14:14:58.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.173+0000 7fc93c000640 1 -- 172.21.15.86:0/1519907444 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9341fa010 con 0x7fc934152020 2024-04-25T14:14:58.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.173+0000 7fc93c000640 1 -- 172.21.15.86:0/1519907444 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9340720e0 con 0x7fc934152020 2024-04-25T14:14:58.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.173+0000 7fc931600640 1 -- 172.21.15.86:0/1519907444 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7fc92c003b40 con 0x7fc934152020 2024-04-25T14:14:58.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.173+0000 7fc931600640 1 --2- 172.21.15.86:0/1519907444 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc91406f570 0x7fc914071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:58.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.173+0000 7fc931600640 1 -- 172.21.15.86:0/1519907444 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fc92c0901f0 con 0x7fc934152020 2024-04-25T14:14:58.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.173+0000 7fc939600640 1 --2- 172.21.15.86:0/1519907444 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc91406f570 0x7fc914071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:58.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.177+0000 7fc939600640 1 --2- 172.21.15.86:0/1519907444 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc91406f570 0x7fc914071a80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc9280b0000 tx=0x7fc9280aa070 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:58.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.181+0000 7fc931600640 1 -- 172.21.15.86:0/1519907444 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc934152970 con 0x7fc934152020 2024-04-25T14:14:58.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.365+0000 7fc93c000640 1 -- 172.21.15.86:0/1519907444 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc934076bf0 con 0x7fc934152020 2024-04-25T14:14:58.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.369+0000 7fc931600640 1 -- 172.21.15.86:0/1519907444 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7fc92c05cb40 con 0x7fc934152020 2024-04-25T14:14:58.364 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:14:58.364 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:14:58.364 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:14:58.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.369+0000 7fc93c000640 1 -- 172.21.15.86:0/1519907444 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc91406f570 msgr2=0x7fc914071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:58.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.369+0000 7fc93c000640 1 --2- 172.21.15.86:0/1519907444 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc91406f570 0x7fc914071a80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc9280b0000 tx=0x7fc9280aa070 comp rx=0 tx=0).stop 2024-04-25T14:14:58.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.369+0000 7fc93c000640 1 -- 172.21.15.86:0/1519907444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc934152020 msgr2=0x7fc934152430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:58.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.369+0000 7fc93c000640 1 --2- 172.21.15.86:0/1519907444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc934152020 0x7fc934152430 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7fc934073a90 tx=0x7fc92c0115a0 comp rx=0 tx=0).stop 2024-04-25T14:14:58.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.369+0000 7fc93c000640 1 -- 172.21.15.86:0/1519907444 shutdown_connections 2024-04-25T14:14:58.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.369+0000 7fc93c000640 1 --2- 172.21.15.86:0/1519907444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc934152020 0x7fc934152430 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:58.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.369+0000 7fc93c000640 1 --2- 172.21.15.86:0/1519907444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9340c9320 0x7fc9340c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:58.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.369+0000 7fc93c000640 1 --2- 172.21.15.86:0/1519907444 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc91406f570 0x7fc914071a80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:58.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.369+0000 7fc93c000640 1 --2- 172.21.15.86:0/1519907444 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc91c003bc0 0x7fc93414bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:58.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.369+0000 7fc93c000640 1 -- 172.21.15.86:0/1519907444 >> 172.21.15.86:0/1519907444 conn(0x7fc9340cffc0 msgr2=0x7fc9340cb830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:58.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.369+0000 7fc93c000640 1 -- 172.21.15.86:0/1519907444 shutdown_connections 2024-04-25T14:14:58.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:58.369+0000 7fc93c000640 1 -- 172.21.15.86:0/1519907444 wait complete. 2024-04-25T14:14:58.788 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:14:58.788 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:14:59.789 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:14:59.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.953+0000 7f372b200640 1 Processor -- start 2024-04-25T14:14:59.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.953+0000 7f372b200640 1 -- start start 2024-04-25T14:14:59.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.957+0000 7f372b200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f37240cfca0 0x7f372406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:59.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.957+0000 7f372b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37240d16e0 0x7f37240d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:59.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.957+0000 7f372b200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f372406c670 0x7f372406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:59.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.957+0000 7f372b200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3724070470 con 0x7f372406c670 2024-04-25T14:14:59.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.957+0000 7f372b200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f37240d2030 con 0x7f37240cfca0 2024-04-25T14:14:59.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.957+0000 7f372b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3724072bd0 con 0x7f37240d16e0 2024-04-25T14:14:59.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.957+0000 7f3729200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f372406c670 0x7f372406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:59.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.957+0000 7f3729200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f372406c670 0x7f372406ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41296/0 (socket says 172.21.15.86:41296) 2024-04-25T14:14:59.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.957+0000 7f3729200640 1 -- 172.21.15.86:0/3493034883 learned_addr learned my addr 172.21.15.86:0/3493034883 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:59.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.957+0000 7f3723400640 1 --2- 172.21.15.86:0/3493034883 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37240d16e0 0x7f37240d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:59.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.957+0000 7f3723e00640 1 --2- 172.21.15.86:0/3493034883 >> v2:172.21.15.155:3300/0 conn(0x7f37240cfca0 0x7f372406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:59.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.957+0000 7f3729200640 1 -- 172.21.15.86:0/3493034883 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37240d16e0 msgr2=0x7f37240d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:59.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.957+0000 7f3729200640 1 --2- 172.21.15.86:0/3493034883 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37240d16e0 0x7f37240d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:59.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.957+0000 7f3729200640 1 -- 172.21.15.86:0/3493034883 >> v2:172.21.15.155:3300/0 conn(0x7f37240cfca0 msgr2=0x7f372406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:14:59.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.957+0000 7f3729200640 1 --2- 172.21.15.86:0/3493034883 >> v2:172.21.15.155:3300/0 conn(0x7f37240cfca0 0x7f372406f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:59.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.957+0000 7f3729200640 1 -- 172.21.15.86:0/3493034883 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372406f930 con 0x7f372406c670 2024-04-25T14:14:59.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.957+0000 7f3723e00640 1 --2- 172.21.15.86:0/3493034883 >> v2:172.21.15.155:3300/0 conn(0x7f37240cfca0 0x7f372406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:14:59.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.961+0000 7f3729200640 1 --2- 172.21.15.86:0/3493034883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f372406c670 0x7f372406ca80 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f37240d22a0 tx=0x7f371c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a89a2a29a2c541 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:59.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.961+0000 7f3722a00640 1 -- 172.21.15.86:0/3493034883 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c0ab5f0 con 0x7f372406c670 2024-04-25T14:14:59.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.961+0000 7f3722a00640 1 -- 172.21.15.86:0/3493034883 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f371c0b0070 con 0x7f372406c670 2024-04-25T14:14:59.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.961+0000 7f3722a00640 1 -- 172.21.15.86:0/3493034883 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c0ab9b0 con 0x7f372406c670 2024-04-25T14:14:59.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.961+0000 7f372b200640 1 -- 172.21.15.86:0/3493034883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f372406c670 msgr2=0x7f372406ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:59.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.961+0000 7f372b200640 1 --2- 172.21.15.86:0/3493034883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f372406c670 0x7f372406ca80 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f37240d22a0 tx=0x7f371c01c520 comp rx=0 tx=0).stop 2024-04-25T14:14:59.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.961+0000 7f372b200640 1 -- 172.21.15.86:0/3493034883 shutdown_connections 2024-04-25T14:14:59.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.961+0000 7f372b200640 1 --2- 172.21.15.86:0/3493034883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f372406c670 0x7f372406ca80 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:59.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.961+0000 7f372b200640 1 --2- 172.21.15.86:0/3493034883 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37240d16e0 0x7f37240d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:59.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.961+0000 7f372b200640 1 --2- 172.21.15.86:0/3493034883 >> v2:172.21.15.155:3300/0 conn(0x7f37240cfca0 0x7f372406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:59.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.961+0000 7f372b200640 1 -- 172.21.15.86:0/3493034883 >> 172.21.15.86:0/3493034883 conn(0x7f37240b4650 msgr2=0x7f37240b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:14:59.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.961+0000 7f372b200640 1 -- 172.21.15.86:0/3493034883 shutdown_connections 2024-04-25T14:14:59.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.961+0000 7f372b200640 1 -- 172.21.15.86:0/3493034883 wait complete. 2024-04-25T14:14:59.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.961+0000 7f372b200640 1 Processor -- start 2024-04-25T14:14:59.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.965+0000 7f372b200640 1 -- start start 2024-04-25T14:14:59.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.965+0000 7f372b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f372406c670 0x7f3724155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:59.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.965+0000 7f372b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37240cfca0 0x7f372409e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:59.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.965+0000 7f372b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37240d16e0 0x7f372409e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:59.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.965+0000 7f372b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37240738c0 con 0x7f372406c670 2024-04-25T14:14:59.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.965+0000 7f372b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f372406d3d0 con 0x7f37240cfca0 2024-04-25T14:14:59.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.965+0000 7f372b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3724073a40 con 0x7f37240d16e0 2024-04-25T14:14:59.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.965+0000 7f3723e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f372406c670 0x7f3724155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:59.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.965+0000 7f3723e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f372406c670 0x7f3724155cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41300/0 (socket says 172.21.15.86:41300) 2024-04-25T14:14:59.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.965+0000 7f3723e00640 1 -- 172.21.15.86:0/4023579823 learned_addr learned my addr 172.21.15.86:0/4023579823 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:14:59.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.965+0000 7f3723400640 1 --2- 172.21.15.86:0/4023579823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37240cfca0 0x7f372409e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:59.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.965+0000 7f3729200640 1 --2- 172.21.15.86:0/4023579823 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37240d16e0 0x7f372409e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:59.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.965+0000 7f3723e00640 1 -- 172.21.15.86:0/4023579823 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37240d16e0 msgr2=0x7f372409e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:59.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.965+0000 7f3723e00640 1 --2- 172.21.15.86:0/4023579823 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37240d16e0 0x7f372409e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:59.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.965+0000 7f3723e00640 1 -- 172.21.15.86:0/4023579823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37240cfca0 msgr2=0x7f372409e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:14:59.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.965+0000 7f3723e00640 1 --2- 172.21.15.86:0/4023579823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37240cfca0 0x7f372409e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:14:59.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.965+0000 7f3723e00640 1 -- 172.21.15.86:0/4023579823 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37241f81b0 con 0x7f372406c670 2024-04-25T14:14:59.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.965+0000 7f3723e00640 1 --2- 172.21.15.86:0/4023579823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f372406c670 0x7f3724155cf0 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f3714007e70 tx=0x7f371400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:59.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.965+0000 7f371be00640 1 -- 172.21.15.86:0/4023579823 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3714014620 con 0x7f372406c670 2024-04-25T14:14:59.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.965+0000 7f371be00640 1 -- 172.21.15.86:0/4023579823 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3714019070 con 0x7f372406c670 2024-04-25T14:14:59.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.965+0000 7f372b200640 1 -- 172.21.15.86:0/4023579823 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37241f8450 con 0x7f372406c670 2024-04-25T14:14:59.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.965+0000 7f371be00640 1 -- 172.21.15.86:0/4023579823 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3714014a00 con 0x7f372406c670 2024-04-25T14:14:59.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.965+0000 7f372b200640 1 -- 172.21.15.86:0/4023579823 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37241f8a10 con 0x7f372406c670 2024-04-25T14:14:59.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.965+0000 7f372b200640 1 -- 172.21.15.86:0/4023579823 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f372409f4b0 con 0x7f372406c670 2024-04-25T14:14:59.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.969+0000 7f371be00640 1 -- 172.21.15.86:0/4023579823 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f3714014ba0 con 0x7f372406c670 2024-04-25T14:14:59.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.969+0000 7f371be00640 1 --2- 172.21.15.86:0/4023579823 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f370406f570 0x7f3704071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:14:59.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.969+0000 7f371be00640 1 -- 172.21.15.86:0/4023579823 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f371408fe00 con 0x7f372406c670 2024-04-25T14:14:59.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.969+0000 7f3723400640 1 --2- 172.21.15.86:0/4023579823 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f370406f570 0x7f3704071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:14:59.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.969+0000 7f3723400640 1 --2- 172.21.15.86:0/4023579823 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f370406f570 0x7f3704071a80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f37240cb3a0 tx=0x7f3710008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:14:59.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:14:59.973+0000 7f371be00640 1 -- 172.21.15.86:0/4023579823 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f371405c340 con 0x7f372406c670 2024-04-25T14:15:00.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:00.169+0000 7f372b200640 1 -- 172.21.15.86:0/4023579823 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3724069d00 con 0x7f372406c670 2024-04-25T14:15:00.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:00.169+0000 7f371be00640 1 -- 172.21.15.86:0/4023579823 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f3714060640 con 0x7f372406c670 2024-04-25T14:15:00.167 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:00.167 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:00.167 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:00.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:00.173+0000 7f372b200640 1 -- 172.21.15.86:0/4023579823 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f370406f570 msgr2=0x7f3704071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:00.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:00.173+0000 7f372b200640 1 --2- 172.21.15.86:0/4023579823 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f370406f570 0x7f3704071a80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f37240cb3a0 tx=0x7f3710008040 comp rx=0 tx=0).stop 2024-04-25T14:15:00.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:00.173+0000 7f372b200640 1 -- 172.21.15.86:0/4023579823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f372406c670 msgr2=0x7f3724155cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:00.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:00.173+0000 7f372b200640 1 --2- 172.21.15.86:0/4023579823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f372406c670 0x7f3724155cf0 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f3714007e70 tx=0x7f371400e4b0 comp rx=0 tx=0).stop 2024-04-25T14:15:00.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:00.173+0000 7f372b200640 1 -- 172.21.15.86:0/4023579823 shutdown_connections 2024-04-25T14:15:00.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:00.173+0000 7f372b200640 1 --2- 172.21.15.86:0/4023579823 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f370406f570 0x7f3704071a80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:00.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:00.173+0000 7f372b200640 1 --2- 172.21.15.86:0/4023579823 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37240d16e0 0x7f372409e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:00.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:00.173+0000 7f372b200640 1 --2- 172.21.15.86:0/4023579823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37240cfca0 0x7f372409e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:00.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:00.173+0000 7f372b200640 1 --2- 172.21.15.86:0/4023579823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f372406c670 0x7f3724155cf0 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:00.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:00.173+0000 7f372b200640 1 -- 172.21.15.86:0/4023579823 >> 172.21.15.86:0/4023579823 conn(0x7f37240b4650 msgr2=0x7f372406eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:00.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:00.173+0000 7f372b200640 1 -- 172.21.15.86:0/4023579823 shutdown_connections 2024-04-25T14:15:00.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:00.173+0000 7f372b200640 1 -- 172.21.15.86:0/4023579823 wait complete. 2024-04-25T14:15:00.592 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:00.592 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:01.592 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:01.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.757+0000 7f5330000640 1 Processor -- start 2024-04-25T14:15:01.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.757+0000 7f5330000640 1 -- start start 2024-04-25T14:15:01.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.757+0000 7f5330000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53280d2420 0x7f53280d2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:01.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.757+0000 7f5330000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f53280b6490 0x7f53280b68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:01.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.757+0000 7f5330000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5328162480 con 0x7f5328070e20 2024-04-25T14:15:01.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.757+0000 7f5330000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f53280d16f0 con 0x7f53280b6490 2024-04-25T14:15:01.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.757+0000 7f5330000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f53280d1940 con 0x7f53280d2420 2024-04-25T14:15:01.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.757+0000 7f532d600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53280d2420 0x7f53280d2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:01.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.757+0000 7f532e000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f53280b6490 0x7f53280b68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:01.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.757+0000 7f532d600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53280d2420 0x7f53280d2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:51106/0 (socket says 172.21.15.86:51106) 2024-04-25T14:15:01.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.757+0000 7f532d600640 1 -- 172.21.15.86:0/1440471423 learned_addr learned my addr 172.21.15.86:0/1440471423 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:01.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.757+0000 7f532e000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f53280b6490 0x7f53280b68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:44350/0 (socket says 172.21.15.86:44350) 2024-04-25T14:15:01.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.757+0000 7f532e000640 1 -- 172.21.15.86:0/1440471423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53280d2420 msgr2=0x7f53280d2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:01.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.757+0000 7f532e000640 1 --2- 172.21.15.86:0/1440471423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53280d2420 0x7f53280d2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:01.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.757+0000 7f532e000640 1 -- 172.21.15.86:0/1440471423 >> v1:172.21.15.155:6789/0 conn(0x7f5328070e20 legacy=0x7f5328071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:01.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.757+0000 7f532e000640 1 -- 172.21.15.86:0/1440471423 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53280d1c20 con 0x7f53280b6490 2024-04-25T14:15:01.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.757+0000 7f532d600640 1 --2- 172.21.15.86:0/1440471423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53280d2420 0x7f53280d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:01.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.757+0000 7f5327e00640 1 -- 172.21.15.86:0/1440471423 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1009179956 0 0) 0x7f5328162480 con 0x7f5328070e20 2024-04-25T14:15:01.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.761+0000 7f532e000640 1 --2- 172.21.15.86:0/1440471423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53280b6490 0x7f53280b68a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f5320009d90 tx=0x7f532001c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46aebfad5a9a3e1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:01.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.761+0000 7f5327e00640 1 -- 172.21.15.86:0/1440471423 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53200b2070 con 0x7f53280b6490 2024-04-25T14:15:01.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.761+0000 7f5327e00640 1 -- 172.21.15.86:0/1440471423 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53200a58e0 con 0x7f53280b6490 2024-04-25T14:15:01.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.761+0000 7f5327e00640 1 -- 172.21.15.86:0/1440471423 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53200a5cb0 con 0x7f53280b6490 2024-04-25T14:15:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.761+0000 7f5330000640 1 -- 172.21.15.86:0/1440471423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53280b6490 msgr2=0x7f53280b68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.761+0000 7f5330000640 1 --2- 172.21.15.86:0/1440471423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53280b6490 0x7f53280b68a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f5320009d90 tx=0x7f532001c970 comp rx=0 tx=0).stop 2024-04-25T14:15:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.761+0000 7f5330000640 1 -- 172.21.15.86:0/1440471423 shutdown_connections 2024-04-25T14:15:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.761+0000 7f5330000640 1 --2- 172.21.15.86:0/1440471423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53280b6490 0x7f53280b68a0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.761+0000 7f5330000640 1 --2- 172.21.15.86:0/1440471423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53280d2420 0x7f53280d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.761+0000 7f5330000640 1 -- 172.21.15.86:0/1440471423 >> 172.21.15.86:0/1440471423 conn(0x7f53280cffc0 msgr2=0x7f53280d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.761+0000 7f5330000640 1 -- 172.21.15.86:0/1440471423 shutdown_connections 2024-04-25T14:15:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.761+0000 7f5330000640 1 -- 172.21.15.86:0/1440471423 wait complete. 2024-04-25T14:15:01.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.765+0000 7f5330000640 1 Processor -- start 2024-04-25T14:15:01.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.765+0000 7f5330000640 1 -- start start 2024-04-25T14:15:01.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.765+0000 7f5330000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5328070e20 0x7f5328160650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:01.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.765+0000 7f5330000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53280b6490 0x7f5328166bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:01.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.765+0000 7f5330000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53280d2420 0x7f5328167130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:01.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.765+0000 7f5330000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5328161310 con 0x7f53280b6490 2024-04-25T14:15:01.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.765+0000 7f5330000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f53280cdf30 con 0x7f5328070e20 2024-04-25T14:15:01.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.765+0000 7f5330000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5328167be0 con 0x7f53280d2420 2024-04-25T14:15:01.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.765+0000 7f532cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53280b6490 0x7f5328166bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:01.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.765+0000 7f532cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53280b6490 0x7f5328166bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41312/0 (socket says 172.21.15.86:41312) 2024-04-25T14:15:01.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.765+0000 7f532cc00640 1 -- 172.21.15.86:0/332905199 learned_addr learned my addr 172.21.15.86:0/332905199 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:01.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.765+0000 7f532e000640 1 --2- 172.21.15.86:0/332905199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53280d2420 0x7f5328167130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:01.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.765+0000 7f532d600640 1 --2- 172.21.15.86:0/332905199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5328070e20 0x7f5328160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:01.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.765+0000 7f532cc00640 1 -- 172.21.15.86:0/332905199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53280d2420 msgr2=0x7f5328167130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:01.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.765+0000 7f532cc00640 1 --2- 172.21.15.86:0/332905199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53280d2420 0x7f5328167130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:01.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.765+0000 7f532cc00640 1 -- 172.21.15.86:0/332905199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5328070e20 msgr2=0x7f5328160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:01.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.765+0000 7f532cc00640 1 --2- 172.21.15.86:0/332905199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5328070e20 0x7f5328160650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:01.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.765+0000 7f532cc00640 1 -- 172.21.15.86:0/332905199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5328160b90 con 0x7f53280b6490 2024-04-25T14:15:01.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.769+0000 7f532cc00640 1 --2- 172.21.15.86:0/332905199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53280b6490 0x7f5328166bf0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f531400c440 tx=0x7f531400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:01.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.769+0000 7f5325600640 1 -- 172.21.15.86:0/332905199 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5314017070 con 0x7f53280b6490 2024-04-25T14:15:01.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.769+0000 7f5325600640 1 -- 172.21.15.86:0/332905199 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53140054e0 con 0x7f53280b6490 2024-04-25T14:15:01.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.769+0000 7f5325600640 1 -- 172.21.15.86:0/332905199 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5314003890 con 0x7f53280b6490 2024-04-25T14:15:01.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.769+0000 7f5330000640 1 -- 172.21.15.86:0/332905199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f532806e5a0 con 0x7f53280b6490 2024-04-25T14:15:01.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.769+0000 7f5330000640 1 -- 172.21.15.86:0/332905199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f532806eb60 con 0x7f53280b6490 2024-04-25T14:15:01.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.769+0000 7f5330000640 1 -- 172.21.15.86:0/332905199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53280b2b00 con 0x7f53280b6490 2024-04-25T14:15:01.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.769+0000 7f5325600640 1 -- 172.21.15.86:0/332905199 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f531401a070 con 0x7f53280b6490 2024-04-25T14:15:01.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.769+0000 7f5325600640 1 --2- 172.21.15.86:0/332905199 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f530806f5a0 0x7f5308071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:01.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.769+0000 7f5325600640 1 -- 172.21.15.86:0/332905199 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f531408ea70 con 0x7f53280b6490 2024-04-25T14:15:01.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.773+0000 7f532d600640 1 --2- 172.21.15.86:0/332905199 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f530806f5a0 0x7f5308071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:01.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.773+0000 7f532d600640 1 --2- 172.21.15.86:0/332905199 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f530806f5a0 0x7f5308071ab0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f531c006cb0 tx=0x7f531c009240 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:01.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.777+0000 7f5325600640 1 -- 172.21.15.86:0/332905199 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f531405ae80 con 0x7f53280b6490 2024-04-25T14:15:01.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.969+0000 7f5330000640 1 -- 172.21.15.86:0/332905199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f532806e330 con 0x7f53280b6490 2024-04-25T14:15:01.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.973+0000 7f5325600640 1 -- 172.21.15.86:0/332905199 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f531405f180 con 0x7f53280b6490 2024-04-25T14:15:01.968 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:01.968 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:01.968 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:01.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.973+0000 7f5330000640 1 -- 172.21.15.86:0/332905199 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f530806f5a0 msgr2=0x7f5308071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:01.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.973+0000 7f5330000640 1 --2- 172.21.15.86:0/332905199 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f530806f5a0 0x7f5308071ab0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f531c006cb0 tx=0x7f531c009240 comp rx=0 tx=0).stop 2024-04-25T14:15:01.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.973+0000 7f5330000640 1 -- 172.21.15.86:0/332905199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53280b6490 msgr2=0x7f5328166bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:01.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.973+0000 7f5330000640 1 --2- 172.21.15.86:0/332905199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53280b6490 0x7f5328166bf0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f531400c440 tx=0x7f531400c840 comp rx=0 tx=0).stop 2024-04-25T14:15:01.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.973+0000 7f5330000640 1 -- 172.21.15.86:0/332905199 shutdown_connections 2024-04-25T14:15:01.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.973+0000 7f5330000640 1 --2- 172.21.15.86:0/332905199 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f530806f5a0 0x7f5308071ab0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:01.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.973+0000 7f5330000640 1 --2- 172.21.15.86:0/332905199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53280d2420 0x7f5328167130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:01.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.973+0000 7f5330000640 1 --2- 172.21.15.86:0/332905199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53280b6490 0x7f5328166bf0 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:01.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.973+0000 7f5330000640 1 --2- 172.21.15.86:0/332905199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5328070e20 0x7f5328160650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:01.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.973+0000 7f5330000640 1 -- 172.21.15.86:0/332905199 >> 172.21.15.86:0/332905199 conn(0x7f53280cffc0 msgr2=0x7f53280c9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:01.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.973+0000 7f5330000640 1 -- 172.21.15.86:0/332905199 shutdown_connections 2024-04-25T14:15:01.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:01.973+0000 7f5330000640 1 -- 172.21.15.86:0/332905199 wait complete. 2024-04-25T14:15:02.392 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:02.392 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:03.393 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:03.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.557+0000 7f727de00640 1 Processor -- start 2024-04-25T14:15:03.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.557+0000 7f727de00640 1 -- start start 2024-04-25T14:15:03.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.557+0000 7f727de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7278070330 0x7f72780d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:03.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.557+0000 7f727de00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72780d1c20 0x7f727806ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:03.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.557+0000 7f727de00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72780d2030 con 0x7f727806f270 2024-04-25T14:15:03.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.557+0000 7f727de00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f727815e5c0 con 0x7f7278070330 2024-04-25T14:15:03.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.557+0000 7f727de00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f727815e8a0 con 0x7f72780d1c20 2024-04-25T14:15:03.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.557+0000 7f7276a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7278070330 0x7f72780d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:03.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.557+0000 7f7276a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7278070330 0x7f72780d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41326/0 (socket says 172.21.15.86:41326) 2024-04-25T14:15:03.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.557+0000 7f7276a00640 1 -- 172.21.15.86:0/459020684 learned_addr learned my addr 172.21.15.86:0/459020684 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:03.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.557+0000 7f7276000640 1 --2- 172.21.15.86:0/459020684 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72780d1c20 0x7f727806ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:03.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.557+0000 7f7276a00640 1 -- 172.21.15.86:0/459020684 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72780d1c20 msgr2=0x7f727806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:03.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.557+0000 7f7276a00640 1 --2- 172.21.15.86:0/459020684 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72780d1c20 0x7f727806ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:03.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.557+0000 7f7276a00640 1 -- 172.21.15.86:0/459020684 >> v1:172.21.15.86:6789/0 conn(0x7f727806f270 legacy=0x7f727806f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:03.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.557+0000 7f7276a00640 1 -- 172.21.15.86:0/459020684 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f727815eb80 con 0x7f7278070330 2024-04-25T14:15:03.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.557+0000 7f7276000640 1 --2- 172.21.15.86:0/459020684 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72780d1c20 0x7f727806ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:15:03.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.561+0000 7f7276a00640 1 --2- 172.21.15.86:0/459020684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7278070330 0x7f72780d16e0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f7268009870 tx=0x7f726801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e9cef81c836cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:03.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.561+0000 7f7275600640 1 -- 172.21.15.86:0/459020684 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72680b0070 con 0x7f7278070330 2024-04-25T14:15:03.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.561+0000 7f7275600640 1 -- 172.21.15.86:0/459020684 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72680ab600 con 0x7f7278070330 2024-04-25T14:15:03.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.561+0000 7f7275600640 1 -- 172.21.15.86:0/459020684 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72680ab9d0 con 0x7f7278070330 2024-04-25T14:15:03.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.561+0000 7f727de00640 1 -- 172.21.15.86:0/459020684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7278070330 msgr2=0x7f72780d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:03.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.561+0000 7f727de00640 1 --2- 172.21.15.86:0/459020684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7278070330 0x7f72780d16e0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f7268009870 tx=0x7f726801c4d0 comp rx=0 tx=0).stop 2024-04-25T14:15:03.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.561+0000 7f727de00640 1 -- 172.21.15.86:0/459020684 shutdown_connections 2024-04-25T14:15:03.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.561+0000 7f727de00640 1 --2- 172.21.15.86:0/459020684 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72780d1c20 0x7f727806ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:03.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.561+0000 7f727de00640 1 --2- 172.21.15.86:0/459020684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7278070330 0x7f72780d16e0 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:03.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.561+0000 7f727de00640 1 -- 172.21.15.86:0/459020684 >> 172.21.15.86:0/459020684 conn(0x7f72780b4650 msgr2=0x7f72780b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:03.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.561+0000 7f727de00640 1 -- 172.21.15.86:0/459020684 shutdown_connections 2024-04-25T14:15:03.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.561+0000 7f727de00640 1 -- 172.21.15.86:0/459020684 wait complete. 2024-04-25T14:15:03.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.565+0000 7f727de00640 1 Processor -- start 2024-04-25T14:15:03.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.565+0000 7f727de00640 1 -- start start 2024-04-25T14:15:03.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.565+0000 7f727de00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f727806f270 0x7f7278073ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:03.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.565+0000 7f727de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7278070330 0x7f72780b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:03.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.565+0000 7f727de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72780d1c20 0x7f72780b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:03.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.565+0000 7f727de00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72780708c0 con 0x7f7278070330 2024-04-25T14:15:03.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.565+0000 7f727de00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7278070740 con 0x7f72780d1c20 2024-04-25T14:15:03.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.565+0000 7f727de00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f727816afc0 con 0x7f727806f270 2024-04-25T14:15:03.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.565+0000 7f7276000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7278070330 0x7f72780b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:03.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.565+0000 7f7276000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7278070330 0x7f72780b16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41342/0 (socket says 172.21.15.86:41342) 2024-04-25T14:15:03.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.565+0000 7f7276000640 1 -- 172.21.15.86:0/2206661077 learned_addr learned my addr 172.21.15.86:0/2206661077 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:03.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.565+0000 7f7276a00640 1 --2- 172.21.15.86:0/2206661077 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f727806f270 0x7f7278073ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:03.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.565+0000 7f7277400640 1 --2- 172.21.15.86:0/2206661077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72780d1c20 0x7f72780b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:03.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.565+0000 7f7276000640 1 -- 172.21.15.86:0/2206661077 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f727806f270 msgr2=0x7f7278073ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:03.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.565+0000 7f7276000640 1 --2- 172.21.15.86:0/2206661077 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f727806f270 0x7f7278073ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:03.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.565+0000 7f7276000640 1 -- 172.21.15.86:0/2206661077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72780d1c20 msgr2=0x7f72780b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:03.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.565+0000 7f7276000640 1 --2- 172.21.15.86:0/2206661077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72780d1c20 0x7f72780b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:03.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.565+0000 7f7276000640 1 -- 172.21.15.86:0/2206661077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7278074220 con 0x7f7278070330 2024-04-25T14:15:03.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.565+0000 7f7277400640 1 --2- 172.21.15.86:0/2206661077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72780d1c20 0x7f72780b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:03.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.565+0000 7f7276000640 1 --2- 172.21.15.86:0/2206661077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7278070330 0x7f72780b16c0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f726400c7f0 tx=0x7f726400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:03.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.565+0000 7f726f400640 1 -- 172.21.15.86:0/2206661077 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7264015070 con 0x7f7278070330 2024-04-25T14:15:03.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.565+0000 7f726f400640 1 -- 172.21.15.86:0/2206661077 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7264007820 con 0x7f7278070330 2024-04-25T14:15:03.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.565+0000 7f727de00640 1 -- 172.21.15.86:0/2206661077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7278072c40 con 0x7f7278070330 2024-04-25T14:15:03.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.565+0000 7f726f400640 1 -- 172.21.15.86:0/2206661077 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7264007c10 con 0x7f7278070330 2024-04-25T14:15:03.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.569+0000 7f727de00640 1 -- 172.21.15.86:0/2206661077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7278073200 con 0x7f7278070330 2024-04-25T14:15:03.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.569+0000 7f727de00640 1 -- 172.21.15.86:0/2206661077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7278069d00 con 0x7f7278070330 2024-04-25T14:15:03.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.569+0000 7f726f400640 1 -- 172.21.15.86:0/2206661077 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f7264018070 con 0x7f7278070330 2024-04-25T14:15:03.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.569+0000 7f726f400640 1 --2- 172.21.15.86:0/2206661077 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f725806f570 0x7f7258071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:03.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.569+0000 7f726f400640 1 -- 172.21.15.86:0/2206661077 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f726408f990 con 0x7f7278070330 2024-04-25T14:15:03.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.569+0000 7f7276a00640 1 --2- 172.21.15.86:0/2206661077 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f725806f570 0x7f7258071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:03.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.573+0000 7f7276a00640 1 --2- 172.21.15.86:0/2206661077 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f725806f570 0x7f7258071a80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f72680ad000 tx=0x7f72680a6000 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:03.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.573+0000 7f726f400640 1 -- 172.21.15.86:0/2206661077 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f726405bc90 con 0x7f7278070330 2024-04-25T14:15:03.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.765+0000 7f727de00640 1 -- 172.21.15.86:0/2206661077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7278076bf0 con 0x7f7278070330 2024-04-25T14:15:03.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.765+0000 7f726f400640 1 -- 172.21.15.86:0/2206661077 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f726405ff90 con 0x7f7278070330 2024-04-25T14:15:03.761 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:03.761 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:03.761 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:03.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.765+0000 7f727de00640 1 -- 172.21.15.86:0/2206661077 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f725806f570 msgr2=0x7f7258071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:03.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.765+0000 7f727de00640 1 --2- 172.21.15.86:0/2206661077 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f725806f570 0x7f7258071a80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f72680ad000 tx=0x7f72680a6000 comp rx=0 tx=0).stop 2024-04-25T14:15:03.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.765+0000 7f727de00640 1 -- 172.21.15.86:0/2206661077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7278070330 msgr2=0x7f72780b16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:03.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.765+0000 7f727de00640 1 --2- 172.21.15.86:0/2206661077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7278070330 0x7f72780b16c0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f726400c7f0 tx=0x7f726400ccb0 comp rx=0 tx=0).stop 2024-04-25T14:15:03.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.765+0000 7f727de00640 1 -- 172.21.15.86:0/2206661077 shutdown_connections 2024-04-25T14:15:03.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.765+0000 7f727de00640 1 --2- 172.21.15.86:0/2206661077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72780d1c20 0x7f72780b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:03.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.765+0000 7f727de00640 1 --2- 172.21.15.86:0/2206661077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7278070330 0x7f72780b16c0 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:03.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.765+0000 7f727de00640 1 --2- 172.21.15.86:0/2206661077 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f725806f570 0x7f7258071a80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:03.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.765+0000 7f727de00640 1 --2- 172.21.15.86:0/2206661077 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f727806f270 0x7f7278073ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:03.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.765+0000 7f727de00640 1 -- 172.21.15.86:0/2206661077 >> 172.21.15.86:0/2206661077 conn(0x7f72780b4650 msgr2=0x7f72780af7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:03.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.765+0000 7f727de00640 1 -- 172.21.15.86:0/2206661077 shutdown_connections 2024-04-25T14:15:03.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:03.765+0000 7f727de00640 1 -- 172.21.15.86:0/2206661077 wait complete. 2024-04-25T14:15:04.186 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:04.186 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:05.187 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:05.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.349+0000 7f47b9200640 1 Processor -- start 2024-04-25T14:15:05.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.349+0000 7f47b9200640 1 -- start start 2024-04-25T14:15:05.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.353+0000 7f47b9200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f47b40cfca0 0x7f47b406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:05.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.353+0000 7f47b9200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47b40730b0 0x7f47b40734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:05.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.353+0000 7f47b9200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47b4072bd0 con 0x7f47b40d16e0 2024-04-25T14:15:05.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.353+0000 7f47b9200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f47b4070470 con 0x7f47b40cfca0 2024-04-25T14:15:05.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.353+0000 7f47b9200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f47b406f930 con 0x7f47b40730b0 2024-04-25T14:15:05.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.353+0000 7f47b1e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f47b40cfca0 0x7f47b406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:05.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.353+0000 7f47b2800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47b40730b0 0x7f47b40734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:05.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.353+0000 7f47abe00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f47b40d16e0 0x7f47b40d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:57144/0 (socket says 172.21.15.86:57144) 2024-04-25T14:15:05.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.353+0000 7f47abe00640 1 -- 172.21.15.86:0/2594299083 learned_addr learned my addr 172.21.15.86:0/2594299083 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:15:05.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.353+0000 7f47b1400640 1 -- 172.21.15.86:0/2594299083 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1403894687 0 0) 0x7f47b4072bd0 con 0x7f47b40d16e0 2024-04-25T14:15:05.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.353+0000 7f47b1400640 1 -- 172.21.15.86:0/2594299083 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4798003530 con 0x7f47b40d16e0 2024-04-25T14:15:05.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.353+0000 7f47b2800640 1 -- 172.21.15.86:0/2594299083 >> v2:172.21.15.155:3300/0 conn(0x7f47b40cfca0 msgr2=0x7f47b406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:05.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.353+0000 7f47b2800640 1 --2- 172.21.15.86:0/2594299083 >> v2:172.21.15.155:3300/0 conn(0x7f47b40cfca0 0x7f47b406f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:05.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.353+0000 7f47b2800640 1 -- 172.21.15.86:0/2594299083 >> v1:172.21.15.155:6789/0 conn(0x7f47b40d16e0 legacy=0x7f47b40d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:05.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.353+0000 7f47b2800640 1 -- 172.21.15.86:0/2594299083 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b406fbd0 con 0x7f47b40730b0 2024-04-25T14:15:05.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.353+0000 7f47b1e00640 1 --2- 172.21.15.86:0/2594299083 >> v2:172.21.15.155:3300/0 conn(0x7f47b40cfca0 0x7f47b406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:05.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.357+0000 7f47b2800640 1 --2- 172.21.15.86:0/2594299083 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47b40730b0 0x7f47b40734c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009d90 tx=0x7f47ac01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33caafc0b0870791 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:05.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.357+0000 7f47b1400640 1 -- 172.21.15.86:0/2594299083 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac0b2070 con 0x7f47b40730b0 2024-04-25T14:15:05.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.357+0000 7f47b1400640 1 -- 172.21.15.86:0/2594299083 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47ac0a5940 con 0x7f47b40730b0 2024-04-25T14:15:05.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.357+0000 7f47b1400640 1 -- 172.21.15.86:0/2594299083 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac0a5d10 con 0x7f47b40730b0 2024-04-25T14:15:05.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.357+0000 7f47b9200640 1 -- 172.21.15.86:0/2594299083 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47b40730b0 msgr2=0x7f47b40734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:05.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.357+0000 7f47b9200640 1 --2- 172.21.15.86:0/2594299083 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47b40730b0 0x7f47b40734c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009d90 tx=0x7f47ac01c970 comp rx=0 tx=0).stop 2024-04-25T14:15:05.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.357+0000 7f47b9200640 1 -- 172.21.15.86:0/2594299083 shutdown_connections 2024-04-25T14:15:05.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.357+0000 7f47b9200640 1 --2- 172.21.15.86:0/2594299083 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47b40730b0 0x7f47b40734c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:05.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.357+0000 7f47b9200640 1 --2- 172.21.15.86:0/2594299083 >> v2:172.21.15.155:3300/0 conn(0x7f47b40cfca0 0x7f47b406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:05.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.357+0000 7f47b9200640 1 -- 172.21.15.86:0/2594299083 >> 172.21.15.86:0/2594299083 conn(0x7f47b40b4650 msgr2=0x7f47b40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:05.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.357+0000 7f47b9200640 1 -- 172.21.15.86:0/2594299083 shutdown_connections 2024-04-25T14:15:05.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.357+0000 7f47b9200640 1 -- 172.21.15.86:0/2594299083 wait complete. 2024-04-25T14:15:05.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.361+0000 7f47b9200640 1 Processor -- start 2024-04-25T14:15:05.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.361+0000 7f47b9200640 1 -- start start 2024-04-25T14:15:05.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.361+0000 7f47b9200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47b40730b0 0x7f47b40c14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:05.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.361+0000 7f47b9200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47b40cfca0 0x7f47b4155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:05.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.361+0000 7f47b9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47b40d16e0 0x7f47b409a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:05.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.361+0000 7f47b9200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47b4169a70 con 0x7f47b40d16e0 2024-04-25T14:15:05.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.361+0000 7f47b9200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41698f0 con 0x7f47b40730b0 2024-04-25T14:15:05.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.361+0000 7f47b9200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f47b4169bf0 con 0x7f47b40cfca0 2024-04-25T14:15:05.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.361+0000 7f47b2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47b40d16e0 0x7f47b409a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:05.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.361+0000 7f47b2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47b40d16e0 0x7f47b409a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41356/0 (socket says 172.21.15.86:41356) 2024-04-25T14:15:05.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.361+0000 7f47b2800640 1 -- 172.21.15.86:0/887405493 learned_addr learned my addr 172.21.15.86:0/887405493 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:05.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.361+0000 7f47b1e00640 1 --2- 172.21.15.86:0/887405493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47b40730b0 0x7f47b40c14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:05.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.361+0000 7f47abe00640 1 --2- 172.21.15.86:0/887405493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47b40cfca0 0x7f47b4155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:05.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.361+0000 7f47b2800640 1 -- 172.21.15.86:0/887405493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47b40cfca0 msgr2=0x7f47b4155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:05.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.361+0000 7f47b2800640 1 --2- 172.21.15.86:0/887405493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47b40cfca0 0x7f47b4155a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:05.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.361+0000 7f47b2800640 1 -- 172.21.15.86:0/887405493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47b40730b0 msgr2=0x7f47b40c14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:05.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.361+0000 7f47b2800640 1 --2- 172.21.15.86:0/887405493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47b40730b0 0x7f47b40c14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:05.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.361+0000 7f47b2800640 1 -- 172.21.15.86:0/887405493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b406dd20 con 0x7f47b40d16e0 2024-04-25T14:15:05.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.361+0000 7f47b1e00640 1 --2- 172.21.15.86:0/887405493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47b40730b0 0x7f47b40c14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:05.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.361+0000 7f47b2800640 1 --2- 172.21.15.86:0/887405493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47b40d16e0 0x7f47b409a7b0 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0b0040 tx=0x7f47ac019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:05.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.361+0000 7f47aa000640 1 -- 172.21.15.86:0/887405493 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac0b2050 con 0x7f47b40d16e0 2024-04-25T14:15:05.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.361+0000 7f47aa000640 1 -- 172.21.15.86:0/887405493 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47ac0be070 con 0x7f47b40d16e0 2024-04-25T14:15:05.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.361+0000 7f47b9200640 1 -- 172.21.15.86:0/887405493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47b40c19e0 con 0x7f47b40d16e0 2024-04-25T14:15:05.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.361+0000 7f47aa000640 1 -- 172.21.15.86:0/887405493 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac0ad850 con 0x7f47b40d16e0 2024-04-25T14:15:05.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.365+0000 7f47b9200640 1 -- 172.21.15.86:0/887405493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47b4154530 con 0x7f47b40d16e0 2024-04-25T14:15:05.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.365+0000 7f47b9200640 1 -- 172.21.15.86:0/887405493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47b4154700 con 0x7f47b40d16e0 2024-04-25T14:15:05.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.365+0000 7f47aa000640 1 -- 172.21.15.86:0/887405493 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f47ac0ad9f0 con 0x7f47b40d16e0 2024-04-25T14:15:05.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.365+0000 7f47aa000640 1 --2- 172.21.15.86:0/887405493 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f479006f570 0x7f4790071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:05.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.365+0000 7f47aa000640 1 -- 172.21.15.86:0/887405493 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f47ac12a6c0 con 0x7f47b40d16e0 2024-04-25T14:15:05.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.365+0000 7f47b1e00640 1 --2- 172.21.15.86:0/887405493 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f479006f570 0x7f4790071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:05.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.365+0000 7f47b1e00640 1 --2- 172.21.15.86:0/887405493 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f479006f570 0x7f4790071a80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f47b4070710 tx=0x7f47a4008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:05.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.369+0000 7f47aa000640 1 -- 172.21.15.86:0/887405493 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f47b4154700 con 0x7f47b40d16e0 2024-04-25T14:15:05.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.557+0000 7f47b9200640 1 -- 172.21.15.86:0/887405493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47b409f4b0 con 0x7f47b40d16e0 2024-04-25T14:15:05.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.561+0000 7f47aa000640 1 -- 172.21.15.86:0/887405493 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f47b40c19e0 con 0x7f47b40d16e0 2024-04-25T14:15:05.556 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:05.556 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:05.556 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:05.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.561+0000 7f47b9200640 1 -- 172.21.15.86:0/887405493 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f479006f570 msgr2=0x7f4790071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:05.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.561+0000 7f47b9200640 1 --2- 172.21.15.86:0/887405493 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f479006f570 0x7f4790071a80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f47b4070710 tx=0x7f47a4008040 comp rx=0 tx=0).stop 2024-04-25T14:15:05.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.561+0000 7f47b9200640 1 -- 172.21.15.86:0/887405493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47b40d16e0 msgr2=0x7f47b409a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:05.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.561+0000 7f47b9200640 1 --2- 172.21.15.86:0/887405493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47b40d16e0 0x7f47b409a7b0 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0b0040 tx=0x7f47ac019710 comp rx=0 tx=0).stop 2024-04-25T14:15:05.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.561+0000 7f47b9200640 1 -- 172.21.15.86:0/887405493 shutdown_connections 2024-04-25T14:15:05.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.561+0000 7f47b9200640 1 --2- 172.21.15.86:0/887405493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47b40d16e0 0x7f47b409a7b0 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:05.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.561+0000 7f47b9200640 1 --2- 172.21.15.86:0/887405493 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f479006f570 0x7f4790071a80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:05.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.561+0000 7f47b9200640 1 --2- 172.21.15.86:0/887405493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47b40cfca0 0x7f47b4155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:05.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.561+0000 7f47b9200640 1 --2- 172.21.15.86:0/887405493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47b40730b0 0x7f47b40c14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:05.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.561+0000 7f47b9200640 1 -- 172.21.15.86:0/887405493 >> 172.21.15.86:0/887405493 conn(0x7f47b40b4650 msgr2=0x7f47b40b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:05.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.561+0000 7f47b9200640 1 -- 172.21.15.86:0/887405493 shutdown_connections 2024-04-25T14:15:05.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:05.561+0000 7f47b9200640 1 -- 172.21.15.86:0/887405493 wait complete. 2024-04-25T14:15:05.980 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:05.980 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:06.981 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:07.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.145+0000 7f360fa00640 1 Processor -- start 2024-04-25T14:15:07.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.145+0000 7f360fa00640 1 -- start start 2024-04-25T14:15:07.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.145+0000 7f360fa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3608070330 0x7f36080d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:07.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.145+0000 7f360fa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36080d1c20 0x7f360806ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:07.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.145+0000 7f360fa00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36080d2030 con 0x7f360806f270 2024-04-25T14:15:07.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.145+0000 7f360fa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f360815e5c0 con 0x7f3608070330 2024-04-25T14:15:07.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.145+0000 7f360fa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f360815e8a0 con 0x7f36080d1c20 2024-04-25T14:15:07.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.145+0000 7f360d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3608070330 0x7f36080d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:07.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.145+0000 7f360d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3608070330 0x7f36080d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41368/0 (socket says 172.21.15.86:41368) 2024-04-25T14:15:07.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.145+0000 7f360d000640 1 -- 172.21.15.86:0/538714824 learned_addr learned my addr 172.21.15.86:0/538714824 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:07.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.145+0000 7f35ffe00640 1 --2- 172.21.15.86:0/538714824 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36080d1c20 0x7f360806ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:07.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.145+0000 7f360d000640 1 -- 172.21.15.86:0/538714824 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36080d1c20 msgr2=0x7f360806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:07.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.145+0000 7f360d000640 1 --2- 172.21.15.86:0/538714824 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36080d1c20 0x7f360806ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:07.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.145+0000 7f360d000640 1 -- 172.21.15.86:0/538714824 >> v1:172.21.15.86:6789/0 conn(0x7f360806f270 legacy=0x7f360806f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:07.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.145+0000 7f360d000640 1 -- 172.21.15.86:0/538714824 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f360815eb80 con 0x7f3608070330 2024-04-25T14:15:07.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.149+0000 7f360d000640 1 --2- 172.21.15.86:0/538714824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3608070330 0x7f36080d16e0 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f35f8009870 tx=0x7f35f801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a53853d799136a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:07.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.149+0000 7f3607e00640 1 -- 172.21.15.86:0/538714824 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f80b0070 con 0x7f3608070330 2024-04-25T14:15:07.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.149+0000 7f3607e00640 1 -- 172.21.15.86:0/538714824 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35f80ab600 con 0x7f3608070330 2024-04-25T14:15:07.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.149+0000 7f3607e00640 1 -- 172.21.15.86:0/538714824 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f80ab9d0 con 0x7f3608070330 2024-04-25T14:15:07.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.149+0000 7f360fa00640 1 -- 172.21.15.86:0/538714824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3608070330 msgr2=0x7f36080d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:07.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.149+0000 7f360fa00640 1 --2- 172.21.15.86:0/538714824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3608070330 0x7f36080d16e0 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f35f8009870 tx=0x7f35f801c4d0 comp rx=0 tx=0).stop 2024-04-25T14:15:07.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.149+0000 7f360fa00640 1 -- 172.21.15.86:0/538714824 shutdown_connections 2024-04-25T14:15:07.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.149+0000 7f360fa00640 1 --2- 172.21.15.86:0/538714824 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36080d1c20 0x7f360806ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:07.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.149+0000 7f360fa00640 1 --2- 172.21.15.86:0/538714824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3608070330 0x7f36080d16e0 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:07.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.149+0000 7f360fa00640 1 -- 172.21.15.86:0/538714824 >> 172.21.15.86:0/538714824 conn(0x7f36080b4650 msgr2=0x7f36080b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:07.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.149+0000 7f360fa00640 1 -- 172.21.15.86:0/538714824 shutdown_connections 2024-04-25T14:15:07.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.149+0000 7f360fa00640 1 -- 172.21.15.86:0/538714824 wait complete. 2024-04-25T14:15:07.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.153+0000 7f360fa00640 1 Processor -- start 2024-04-25T14:15:07.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.153+0000 7f360fa00640 1 -- start start 2024-04-25T14:15:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.153+0000 7f360fa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f360806f270 0x7f36080b1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.153+0000 7f360fa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3608070330 0x7f36080c39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.153+0000 7f360fa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36080d1c20 0x7f36080c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.153+0000 7f360fa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f36080708c0 con 0x7f360806f270 2024-04-25T14:15:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.153+0000 7f360fa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3608070740 con 0x7f3608070330 2024-04-25T14:15:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.153+0000 7f360fa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f360816afc0 con 0x7f36080d1c20 2024-04-25T14:15:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.153+0000 7f360d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f360806f270 0x7f36080b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.153+0000 7f360d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f360806f270 0x7f36080b1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41378/0 (socket says 172.21.15.86:41378) 2024-04-25T14:15:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.153+0000 7f360d000640 1 -- 172.21.15.86:0/1344601126 learned_addr learned my addr 172.21.15.86:0/1344601126 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.153+0000 7f360d000640 1 -- 172.21.15.86:0/1344601126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36080d1c20 msgr2=0x7f36080c3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.153+0000 7f360d000640 1 --2- 172.21.15.86:0/1344601126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36080d1c20 0x7f36080c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.153+0000 7f360d000640 1 -- 172.21.15.86:0/1344601126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3608070330 msgr2=0x7f36080c39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.153+0000 7f360d000640 1 --2- 172.21.15.86:0/1344601126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3608070330 0x7f36080c39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.153+0000 7f360d000640 1 -- 172.21.15.86:0/1344601126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36080cf2a0 con 0x7f360806f270 2024-04-25T14:15:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.153+0000 7f360d000640 1 --2- 172.21.15.86:0/1344601126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f360806f270 0x7f36080b1650 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f35f801a100 tx=0x7f35f801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.153+0000 7f3605600640 1 -- 172.21.15.86:0/1344601126 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f80b0040 con 0x7f360806f270 2024-04-25T14:15:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.153+0000 7f3605600640 1 -- 172.21.15.86:0/1344601126 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35f80bc070 con 0x7f360806f270 2024-04-25T14:15:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.153+0000 7f360fa00640 1 -- 172.21.15.86:0/1344601126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36080cf580 con 0x7f360806f270 2024-04-25T14:15:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.153+0000 7f3605600640 1 -- 172.21.15.86:0/1344601126 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f80b6640 con 0x7f360806f270 2024-04-25T14:15:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.153+0000 7f360fa00640 1 -- 172.21.15.86:0/1344601126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36080cfb40 con 0x7f360806f270 2024-04-25T14:15:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.157+0000 7f360fa00640 1 -- 172.21.15.86:0/1344601126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3608069d00 con 0x7f360806f270 2024-04-25T14:15:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.157+0000 7f3605600640 1 -- 172.21.15.86:0/1344601126 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f35f80b67e0 con 0x7f360806f270 2024-04-25T14:15:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.157+0000 7f3605600640 1 --2- 172.21.15.86:0/1344601126 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f35e406f5a0 0x7f35e4071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.157+0000 7f3605600640 1 -- 172.21.15.86:0/1344601126 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f35f8129bd0 con 0x7f360806f270 2024-04-25T14:15:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.157+0000 7f35ffe00640 1 --2- 172.21.15.86:0/1344601126 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f35e406f5a0 0x7f35e4071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:07.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.157+0000 7f35ffe00640 1 --2- 172.21.15.86:0/1344601126 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f35e406f5a0 0x7f35e4071ab0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f36080c4aa0 tx=0x7f35f0008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:07.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.161+0000 7f3605600640 1 -- 172.21.15.86:0/1344601126 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f35f8016150 con 0x7f360806f270 2024-04-25T14:15:07.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.353+0000 7f360fa00640 1 -- 172.21.15.86:0/1344601126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f360806d8d0 con 0x7f360806f270 2024-04-25T14:15:07.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.353+0000 7f3605600640 1 -- 172.21.15.86:0/1344601126 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f35f80f65c0 con 0x7f360806f270 2024-04-25T14:15:07.349 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:07.349 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:07.349 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:07.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.353+0000 7f360fa00640 1 -- 172.21.15.86:0/1344601126 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f35e406f5a0 msgr2=0x7f35e4071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:07.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.353+0000 7f360fa00640 1 --2- 172.21.15.86:0/1344601126 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f35e406f5a0 0x7f35e4071ab0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f36080c4aa0 tx=0x7f35f0008040 comp rx=0 tx=0).stop 2024-04-25T14:15:07.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.353+0000 7f360fa00640 1 -- 172.21.15.86:0/1344601126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f360806f270 msgr2=0x7f36080b1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:07.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.353+0000 7f360fa00640 1 --2- 172.21.15.86:0/1344601126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f360806f270 0x7f36080b1650 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f35f801a100 tx=0x7f35f801ad10 comp rx=0 tx=0).stop 2024-04-25T14:15:07.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.353+0000 7f360fa00640 1 -- 172.21.15.86:0/1344601126 shutdown_connections 2024-04-25T14:15:07.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.353+0000 7f360fa00640 1 --2- 172.21.15.86:0/1344601126 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f35e406f5a0 0x7f35e4071ab0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:07.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.353+0000 7f360fa00640 1 --2- 172.21.15.86:0/1344601126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36080d1c20 0x7f36080c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:07.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.353+0000 7f360fa00640 1 --2- 172.21.15.86:0/1344601126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3608070330 0x7f36080c39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:07.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.353+0000 7f360fa00640 1 --2- 172.21.15.86:0/1344601126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f360806f270 0x7f36080b1650 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:07.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.353+0000 7f360fa00640 1 -- 172.21.15.86:0/1344601126 >> 172.21.15.86:0/1344601126 conn(0x7f36080b4650 msgr2=0x7f36080ae130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:07.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.353+0000 7f360fa00640 1 -- 172.21.15.86:0/1344601126 shutdown_connections 2024-04-25T14:15:07.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:07.353+0000 7f360fa00640 1 -- 172.21.15.86:0/1344601126 wait complete. 2024-04-25T14:15:07.773 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:07.773 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:08.774 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.937+0000 7f2556400640 1 Processor -- start 2024-04-25T14:15:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.937+0000 7f2556400640 1 -- start start 2024-04-25T14:15:08.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f2556400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f255006fd50 0x7f2550070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:08.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f2556400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25500b0a60 con 0x7f255015c2a0 2024-04-25T14:15:08.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f2556400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25500ae840 con 0x7f255015feb0 2024-04-25T14:15:08.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f2556400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f255006ef90 con 0x7f255006fd50 2024-04-25T14:15:08.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f254f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f255015c2a0 0x7f25500ae130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:38578/0 (socket says 172.21.15.86:38578) 2024-04-25T14:15:08.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f254f400640 1 -- 172.21.15.86:0/461476132 learned_addr learned my addr 172.21.15.86:0/461476132 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:15:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f2546a00640 1 --2- 172.21.15.86:0/461476132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f255006fd50 0x7f2550070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f254ea00640 1 -- 172.21.15.86:0/461476132 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3298462585 0 0) 0x7f25500b0a60 con 0x7f255015c2a0 2024-04-25T14:15:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f254ea00640 1 -- 172.21.15.86:0/461476132 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2534003530 con 0x7f255015c2a0 2024-04-25T14:15:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f254ea00640 1 -- 172.21.15.86:0/461476132 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 125901184 0 0) 0x7f2534003530 con 0x7f255015c2a0 2024-04-25T14:15:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f254ea00640 1 -- 172.21.15.86:0/461476132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f255006fd50 msgr2=0x7f2550070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f254ea00640 1 --2- 172.21.15.86:0/461476132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f255006fd50 0x7f2550070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f254ea00640 1 -- 172.21.15.86:0/461476132 >> v1:172.21.15.155:6789/0 conn(0x7f255015feb0 legacy=0x7f25501602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f254ea00640 1 -- 172.21.15.86:0/461476132 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f255006f270 con 0x7f255015c2a0 2024-04-25T14:15:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f254ea00640 1 -- 172.21.15.86:0/461476132 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f2540001a00 con 0x7f255015c2a0 2024-04-25T14:15:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f254ea00640 1 -- 172.21.15.86:0/461476132 >> v1:172.21.15.86:6789/0 conn(0x7f255015c2a0 legacy=0x7f25500ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f254ea00640 1 --2- 172.21.15.86:0/461476132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2534003a60 0x7f2534003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f254ea00640 1 --2- 172.21.15.86:0/461476132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2534004810 0x7f2534004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f254ea00640 1 --2- 172.21.15.86:0/461476132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f255006fd50 0x7f25340052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f254ea00640 1 -- 172.21.15.86:0/461476132 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f255006ef90 con 0x7f2534003a60 2024-04-25T14:15:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f254ea00640 1 -- 172.21.15.86:0/461476132 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f253400dda0 con 0x7f2534004810 2024-04-25T14:15:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f254ea00640 1 -- 172.21.15.86:0/461476132 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f253400dfa0 con 0x7f255006fd50 2024-04-25T14:15:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f254f400640 1 --2- 172.21.15.86:0/461476132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2534003a60 0x7f2534003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f254fe00640 1 --2- 172.21.15.86:0/461476132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f255006fd50 0x7f25340052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f2546a00640 1 --2- 172.21.15.86:0/461476132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2534004810 0x7f2534004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f254f400640 1 -- 172.21.15.86:0/461476132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f255006fd50 msgr2=0x7f25340052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f254f400640 1 --2- 172.21.15.86:0/461476132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f255006fd50 0x7f25340052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f254f400640 1 -- 172.21.15.86:0/461476132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2534004810 msgr2=0x7f2534004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f254f400640 1 --2- 172.21.15.86:0/461476132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2534004810 0x7f2534004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.941+0000 7f254f400640 1 -- 172.21.15.86:0/461476132 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2534003530 con 0x7f2534003a60 2024-04-25T14:15:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.945+0000 7f254f400640 1 --2- 172.21.15.86:0/461476132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2534003a60 0x7f2534003e90 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f2550070bb0 tx=0x7f254001d310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae02fd69548cfcae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.945+0000 7f254ea00640 1 -- 172.21.15.86:0/461476132 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254000a4a0 con 0x7f2534003a60 2024-04-25T14:15:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.945+0000 7f254ea00640 1 -- 172.21.15.86:0/461476132 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25400ad650 con 0x7f2534003a60 2024-04-25T14:15:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.945+0000 7f254ea00640 1 -- 172.21.15.86:0/461476132 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25400ada40 con 0x7f2534003a60 2024-04-25T14:15:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.945+0000 7f2556400640 1 -- 172.21.15.86:0/461476132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2534003a60 msgr2=0x7f2534003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.945+0000 7f2556400640 1 --2- 172.21.15.86:0/461476132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2534003a60 0x7f2534003e90 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f2550070bb0 tx=0x7f254001d310 comp rx=0 tx=0).stop 2024-04-25T14:15:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.945+0000 7f254fe00640 1 -- 172.21.15.86:0/461476132 reap_dead start 2024-04-25T14:15:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.945+0000 7f2556400640 1 -- 172.21.15.86:0/461476132 shutdown_connections 2024-04-25T14:15:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.945+0000 7f2556400640 1 -- 172.21.15.86:0/461476132 >> 172.21.15.86:0/461476132 conn(0x7f25500b4650 msgr2=0x7f25500b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.945+0000 7f2556400640 1 -- 172.21.15.86:0/461476132 shutdown_connections 2024-04-25T14:15:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.945+0000 7f2556400640 1 -- 172.21.15.86:0/461476132 wait complete. 2024-04-25T14:15:08.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.949+0000 7f2556400640 1 Processor -- start 2024-04-25T14:15:08.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.949+0000 7f2556400640 1 -- start start 2024-04-25T14:15:08.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.949+0000 7f2556400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2534003a60 0x7f2550155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:08.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.949+0000 7f2556400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2550073c60 0x7f2550155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:08.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.949+0000 7f2556400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2550157970 0x7f25500b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:08.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.949+0000 7f2556400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f255015c830 con 0x7f2550073c60 2024-04-25T14:15:08.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.949+0000 7f2556400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f255015c6b0 con 0x7f2550157970 2024-04-25T14:15:08.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.949+0000 7f2556400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2550070570 con 0x7f2534003a60 2024-04-25T14:15:08.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.949+0000 7f2546a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2550073c60 0x7f2550155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:08.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.949+0000 7f2546a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2550073c60 0x7f2550155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48238/0 (socket says 172.21.15.86:48238) 2024-04-25T14:15:08.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.949+0000 7f2546a00640 1 -- 172.21.15.86:0/1775331546 learned_addr learned my addr 172.21.15.86:0/1775331546 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:08.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.949+0000 7f254f400640 1 --2- 172.21.15.86:0/1775331546 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2534003a60 0x7f2550155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:08.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.949+0000 7f254fe00640 1 --2- 172.21.15.86:0/1775331546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2550157970 0x7f25500b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:08.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.949+0000 7f2546a00640 1 -- 172.21.15.86:0/1775331546 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2534003a60 msgr2=0x7f2550155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:08.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.949+0000 7f2546a00640 1 --2- 172.21.15.86:0/1775331546 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2534003a60 0x7f2550155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:08.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.949+0000 7f2546a00640 1 -- 172.21.15.86:0/1775331546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2550157970 msgr2=0x7f25500b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:08.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.949+0000 7f2546a00640 1 --2- 172.21.15.86:0/1775331546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2550157970 0x7f25500b0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:08.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.949+0000 7f2546a00640 1 -- 172.21.15.86:0/1775331546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25500b2ae0 con 0x7f2550073c60 2024-04-25T14:15:08.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.949+0000 7f2546a00640 1 --2- 172.21.15.86:0/1775331546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2550073c60 0x7f2550155d60 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f2538011610 tx=0x7f2538011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:08.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.953+0000 7f2547e00640 1 -- 172.21.15.86:0/1775331546 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253800f8f0 con 0x7f2550073c60 2024-04-25T14:15:08.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.953+0000 7f2547e00640 1 -- 172.21.15.86:0/1775331546 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f253800fa90 con 0x7f2550073c60 2024-04-25T14:15:08.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.953+0000 7f2556400640 1 -- 172.21.15.86:0/1775331546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25500b2dc0 con 0x7f2550073c60 2024-04-25T14:15:08.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.953+0000 7f2547e00640 1 -- 172.21.15.86:0/1775331546 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253800fe40 con 0x7f2550073c60 2024-04-25T14:15:08.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.953+0000 7f2556400640 1 -- 172.21.15.86:0/1775331546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25500b3380 con 0x7f2550073c60 2024-04-25T14:15:08.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.953+0000 7f2556400640 1 -- 172.21.15.86:0/1775331546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25500b2850 con 0x7f2550073c60 2024-04-25T14:15:08.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.953+0000 7f2547e00640 1 -- 172.21.15.86:0/1775331546 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f2538015460 con 0x7f2550073c60 2024-04-25T14:15:08.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.953+0000 7f2547e00640 1 --2- 172.21.15.86:0/1775331546 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f252c06f570 0x7f252c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:08.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.953+0000 7f2547e00640 1 -- 172.21.15.86:0/1775331546 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f253808ff30 con 0x7f2550073c60 2024-04-25T14:15:08.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.953+0000 7f254f400640 1 --2- 172.21.15.86:0/1775331546 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f252c06f570 0x7f252c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:08.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.957+0000 7f254f400640 1 --2- 172.21.15.86:0/1775331546 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f252c06f570 0x7f252c071a80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f25400ad000 tx=0x7f25400a7000 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:08.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:08.957+0000 7f2547e00640 1 -- 172.21.15.86:0/1775331546 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2538019030 con 0x7f2550073c60 2024-04-25T14:15:09.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:09.149+0000 7f2556400640 1 -- 172.21.15.86:0/1775331546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2550069d00 con 0x7f2550073c60 2024-04-25T14:15:09.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:09.149+0000 7f2547e00640 1 -- 172.21.15.86:0/1775331546 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f253805c3d0 con 0x7f2550073c60 2024-04-25T14:15:09.146 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:09.146 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:09.146 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:09.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:09.153+0000 7f2556400640 1 -- 172.21.15.86:0/1775331546 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f252c06f570 msgr2=0x7f252c071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:09.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:09.153+0000 7f2556400640 1 --2- 172.21.15.86:0/1775331546 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f252c06f570 0x7f252c071a80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f25400ad000 tx=0x7f25400a7000 comp rx=0 tx=0).stop 2024-04-25T14:15:09.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:09.153+0000 7f2556400640 1 -- 172.21.15.86:0/1775331546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2550073c60 msgr2=0x7f2550155d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:09.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:09.153+0000 7f2556400640 1 --2- 172.21.15.86:0/1775331546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2550073c60 0x7f2550155d60 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f2538011610 tx=0x7f2538011ad0 comp rx=0 tx=0).stop 2024-04-25T14:15:09.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:09.153+0000 7f2556400640 1 -- 172.21.15.86:0/1775331546 shutdown_connections 2024-04-25T14:15:09.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:09.153+0000 7f2556400640 1 --2- 172.21.15.86:0/1775331546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2550157970 0x7f25500b0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:09.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:09.153+0000 7f2556400640 1 --2- 172.21.15.86:0/1775331546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2550073c60 0x7f2550155d60 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:09.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:09.153+0000 7f2556400640 1 --2- 172.21.15.86:0/1775331546 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f252c06f570 0x7f252c071a80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:09.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:09.153+0000 7f2556400640 1 --2- 172.21.15.86:0/1775331546 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2534003a60 0x7f2550155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:09.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:09.153+0000 7f2556400640 1 -- 172.21.15.86:0/1775331546 >> 172.21.15.86:0/1775331546 conn(0x7f25500b4650 msgr2=0x7f2550073550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:09.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:09.153+0000 7f2556400640 1 -- 172.21.15.86:0/1775331546 shutdown_connections 2024-04-25T14:15:09.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:09.153+0000 7f2556400640 1 -- 172.21.15.86:0/1775331546 wait complete. 2024-04-25T14:15:09.571 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:09.571 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:10.572 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:10.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.733+0000 7fd31e200640 1 Processor -- start 2024-04-25T14:15:10.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.733+0000 7fd31e200640 1 -- start start 2024-04-25T14:15:10.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.737+0000 7fd31e200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd3180d16e0 0x7fd3180d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:10.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.737+0000 7fd31e200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3180d2030 con 0x7fd3180730b0 2024-04-25T14:15:10.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.737+0000 7fd31e200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd318072bd0 con 0x7fd3180cfca0 2024-04-25T14:15:10.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.737+0000 7fd31e200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd318070470 con 0x7fd3180d16e0 2024-04-25T14:15:10.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.737+0000 7fd30fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd3180d16e0 0x7fd3180d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:10.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.737+0000 7fd30fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd3180d16e0 0x7fd3180d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48248/0 (socket says 172.21.15.86:48248) 2024-04-25T14:15:10.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.737+0000 7fd30fe00640 1 -- 172.21.15.86:0/915123789 learned_addr learned my addr 172.21.15.86:0/915123789 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:10.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.737+0000 7fd30fe00640 1 -- 172.21.15.86:0/915123789 >> v1:172.21.15.155:6789/0 conn(0x7fd3180cfca0 legacy=0x7fd31806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:10.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.737+0000 7fd30fe00640 1 -- 172.21.15.86:0/915123789 >> v1:172.21.15.86:6789/0 conn(0x7fd3180730b0 legacy=0x7fd3180734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:10.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.737+0000 7fd30fe00640 1 -- 172.21.15.86:0/915123789 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31806fbd0 con 0x7fd3180d16e0 2024-04-25T14:15:10.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.741+0000 7fd30fe00640 1 --2- 172.21.15.86:0/915123789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3180d16e0 0x7fd3180d1af0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7fd30000bb10 tx=0x7fd30001ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9151a1eec77b4ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:10.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.741+0000 7fd316a00640 1 -- 172.21.15.86:0/915123789 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3000b4070 con 0x7fd3180d16e0 2024-04-25T14:15:10.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.741+0000 7fd316a00640 1 -- 172.21.15.86:0/915123789 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3000a7bd0 con 0x7fd3180d16e0 2024-04-25T14:15:10.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.741+0000 7fd316a00640 1 -- 172.21.15.86:0/915123789 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3180743e0 con 0x7fd3180d16e0 2024-04-25T14:15:10.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.741+0000 7fd31e200640 1 -- 172.21.15.86:0/915123789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3180d16e0 msgr2=0x7fd3180d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:10.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.741+0000 7fd31e200640 1 --2- 172.21.15.86:0/915123789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3180d16e0 0x7fd3180d1af0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7fd30000bb10 tx=0x7fd30001ebf0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.741+0000 7fd31e200640 1 -- 172.21.15.86:0/915123789 shutdown_connections 2024-04-25T14:15:10.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.741+0000 7fd31e200640 1 --2- 172.21.15.86:0/915123789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3180d16e0 0x7fd3180d1af0 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.741+0000 7fd31e200640 1 -- 172.21.15.86:0/915123789 >> 172.21.15.86:0/915123789 conn(0x7fd3180b4650 msgr2=0x7fd3180b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:10.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.741+0000 7fd31e200640 1 -- 172.21.15.86:0/915123789 shutdown_connections 2024-04-25T14:15:10.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.741+0000 7fd31e200640 1 -- 172.21.15.86:0/915123789 wait complete. 2024-04-25T14:15:10.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.741+0000 7fd31e200640 1 Processor -- start 2024-04-25T14:15:10.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.745+0000 7fd31e200640 1 -- start start 2024-04-25T14:15:10.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.745+0000 7fd31e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3180730b0 0x7fd31815b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:10.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.745+0000 7fd31e200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3180cfca0 0x7fd31809a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:10.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.745+0000 7fd31e200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3180d16e0 0x7fd318156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:10.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.745+0000 7fd31e200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3180ce1b0 con 0x7fd3180730b0 2024-04-25T14:15:10.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.745+0000 7fd31e200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd3180ce030 con 0x7fd3180cfca0 2024-04-25T14:15:10.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.745+0000 7fd31e200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd3180ce330 con 0x7fd3180d16e0 2024-04-25T14:15:10.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.745+0000 7fd317400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3180730b0 0x7fd31815b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:10.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.745+0000 7fd317400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3180730b0 0x7fd31815b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48262/0 (socket says 172.21.15.86:48262) 2024-04-25T14:15:10.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.745+0000 7fd317400640 1 -- 172.21.15.86:0/3511644083 learned_addr learned my addr 172.21.15.86:0/3511644083 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:10.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.745+0000 7fd30fe00640 1 --2- 172.21.15.86:0/3511644083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3180cfca0 0x7fd31809a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:10.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.745+0000 7fd317e00640 1 --2- 172.21.15.86:0/3511644083 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3180d16e0 0x7fd318156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:10.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.745+0000 7fd317400640 1 -- 172.21.15.86:0/3511644083 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3180d16e0 msgr2=0x7fd318156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:10.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.745+0000 7fd317400640 1 --2- 172.21.15.86:0/3511644083 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3180d16e0 0x7fd318156180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.745+0000 7fd317400640 1 -- 172.21.15.86:0/3511644083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3180cfca0 msgr2=0x7fd31809a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:10.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.745+0000 7fd317400640 1 --2- 172.21.15.86:0/3511644083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3180cfca0 0x7fd31809a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.745+0000 7fd317400640 1 -- 172.21.15.86:0/3511644083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31809a9d0 con 0x7fd3180730b0 2024-04-25T14:15:10.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.745+0000 7fd30fe00640 1 --2- 172.21.15.86:0/3511644083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3180cfca0 0x7fd31809a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:10.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.745+0000 7fd317400640 1 --2- 172.21.15.86:0/3511644083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3180730b0 0x7fd31815b6f0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7fd30800b3e0 tx=0x7fd30800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:10.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.745+0000 7fd30f400640 1 -- 172.21.15.86:0/3511644083 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd308015020 con 0x7fd3180730b0 2024-04-25T14:15:10.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.745+0000 7fd30f400640 1 -- 172.21.15.86:0/3511644083 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd308011830 con 0x7fd3180730b0 2024-04-25T14:15:10.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.745+0000 7fd30f400640 1 -- 172.21.15.86:0/3511644083 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd308011c60 con 0x7fd3180730b0 2024-04-25T14:15:10.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.745+0000 7fd31e200640 1 -- 172.21.15.86:0/3511644083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd318155c60 con 0x7fd3180730b0 2024-04-25T14:15:10.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.745+0000 7fd31e200640 1 -- 172.21.15.86:0/3511644083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd318159a60 con 0x7fd3180730b0 2024-04-25T14:15:10.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.745+0000 7fd31e200640 1 -- 172.21.15.86:0/3511644083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3180b50e0 con 0x7fd3180730b0 2024-04-25T14:15:10.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.749+0000 7fd30f400640 1 -- 172.21.15.86:0/3511644083 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7fd308018020 con 0x7fd3180730b0 2024-04-25T14:15:10.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.749+0000 7fd30f400640 1 --2- 172.21.15.86:0/3511644083 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd2f406f5a0 0x7fd2f4071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:10.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.749+0000 7fd30f400640 1 -- 172.21.15.86:0/3511644083 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fd30808f630 con 0x7fd3180730b0 2024-04-25T14:15:10.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.749+0000 7fd30fe00640 1 --2- 172.21.15.86:0/3511644083 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd2f406f5a0 0x7fd2f4071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:10.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.749+0000 7fd30fe00640 1 --2- 172.21.15.86:0/3511644083 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd2f406f5a0 0x7fd2f4071ab0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd3000b1000 tx=0x7fd3000a9070 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:10.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.753+0000 7fd30f400640 1 -- 172.21.15.86:0/3511644083 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd30801f020 con 0x7fd3180730b0 2024-04-25T14:15:10.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.945+0000 7fd31e200640 1 -- 172.21.15.86:0/3511644083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd31809f4b0 con 0x7fd3180730b0 2024-04-25T14:15:10.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.945+0000 7fd30f400640 1 -- 172.21.15.86:0/3511644083 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7fd318155c60 con 0x7fd3180730b0 2024-04-25T14:15:10.941 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:10.941 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:10.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.945+0000 7fd31e200640 1 -- 172.21.15.86:0/3511644083 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd2f406f5a0 msgr2=0x7fd2f4071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:10.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.945+0000 7fd31e200640 1 --2- 172.21.15.86:0/3511644083 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd2f406f5a0 0x7fd2f4071ab0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd3000b1000 tx=0x7fd3000a9070 comp rx=0 tx=0).stop 2024-04-25T14:15:10.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.945+0000 7fd31e200640 1 -- 172.21.15.86:0/3511644083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3180730b0 msgr2=0x7fd31815b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:10.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.945+0000 7fd31e200640 1 --2- 172.21.15.86:0/3511644083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3180730b0 0x7fd31815b6f0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7fd30800b3e0 tx=0x7fd30800b8a0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.945+0000 7fd31e200640 1 -- 172.21.15.86:0/3511644083 shutdown_connections 2024-04-25T14:15:10.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.945+0000 7fd31e200640 1 --2- 172.21.15.86:0/3511644083 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd2f406f5a0 0x7fd2f4071ab0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.945+0000 7fd31e200640 1 --2- 172.21.15.86:0/3511644083 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3180d16e0 0x7fd318156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.945+0000 7fd31e200640 1 --2- 172.21.15.86:0/3511644083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3180cfca0 0x7fd31809a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.945+0000 7fd31e200640 1 --2- 172.21.15.86:0/3511644083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3180730b0 0x7fd31815b6f0 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:10.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.945+0000 7fd31e200640 1 -- 172.21.15.86:0/3511644083 >> 172.21.15.86:0/3511644083 conn(0x7fd3180b4650 msgr2=0x7fd3180b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:10.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.945+0000 7fd31e200640 1 -- 172.21.15.86:0/3511644083 shutdown_connections 2024-04-25T14:15:10.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:10.945+0000 7fd31e200640 1 -- 172.21.15.86:0/3511644083 wait complete. 2024-04-25T14:15:11.365 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:11.365 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:12.367 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:12.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.529+0000 7fe0df200640 1 Processor -- start 2024-04-25T14:15:12.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.529+0000 7fe0df200640 1 -- start start 2024-04-25T14:15:12.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.529+0000 7fe0df200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe0d80d1df0 0x7fe0d80cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:12.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.529+0000 7fe0df200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0d815e5c0 con 0x7fe0d80cdcc0 2024-04-25T14:15:12.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.529+0000 7fe0df200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0d815e8a0 con 0x7fe0d8070330 2024-04-25T14:15:12.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.529+0000 7fe0df200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe0d806fd40 con 0x7fe0d80d1df0 2024-04-25T14:15:12.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.529+0000 7fe0dd200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe0d80cdcc0 0x7fe0d80ce0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:38604/0 (socket says 172.21.15.86:38604) 2024-04-25T14:15:12.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.533+0000 7fe0dd200640 1 -- 172.21.15.86:0/2902890076 learned_addr learned my addr 172.21.15.86:0/2902890076 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:15:12.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.533+0000 7fe0cf400640 1 --2- 172.21.15.86:0/2902890076 >> v2:172.21.15.155:3300/0 conn(0x7fe0d80d1df0 0x7fe0d80cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:12.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.533+0000 7fe0d7400640 1 -- 172.21.15.86:0/2902890076 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 972272694 0 0) 0x7fe0d815e5c0 con 0x7fe0d80cdcc0 2024-04-25T14:15:12.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.533+0000 7fe0d7400640 1 -- 172.21.15.86:0/2902890076 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0bc003530 con 0x7fe0d80cdcc0 2024-04-25T14:15:12.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.533+0000 7fe0d7400640 1 -- 172.21.15.86:0/2902890076 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 143004584 0 0) 0x7fe0bc003530 con 0x7fe0d80cdcc0 2024-04-25T14:15:12.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.533+0000 7fe0d7400640 1 -- 172.21.15.86:0/2902890076 >> v2:172.21.15.155:3300/0 conn(0x7fe0d80d1df0 msgr2=0x7fe0d80cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:12.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.533+0000 7fe0d7400640 1 --2- 172.21.15.86:0/2902890076 >> v2:172.21.15.155:3300/0 conn(0x7fe0d80d1df0 0x7fe0d80cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:12.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.533+0000 7fe0d7400640 1 -- 172.21.15.86:0/2902890076 >> v1:172.21.15.155:6789/0 conn(0x7fe0d8070330 legacy=0x7fe0d80d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:12.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.533+0000 7fe0d7400640 1 -- 172.21.15.86:0/2902890076 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d815eb80 con 0x7fe0d80cdcc0 2024-04-25T14:15:12.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.533+0000 7fe0d7400640 1 -- 172.21.15.86:0/2902890076 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fe0d0001a00 con 0x7fe0d80cdcc0 2024-04-25T14:15:12.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.533+0000 7fe0d7400640 1 -- 172.21.15.86:0/2902890076 >> v1:172.21.15.86:6789/0 conn(0x7fe0d80cdcc0 legacy=0x7fe0d80ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:12.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.533+0000 7fe0d7400640 1 --2- 172.21.15.86:0/2902890076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0bc003a30 0x7fe0bc003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:12.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.533+0000 7fe0d7400640 1 --2- 172.21.15.86:0/2902890076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0bc004810 0x7fe0bc004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:12.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.533+0000 7fe0d7400640 1 --2- 172.21.15.86:0/2902890076 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0bc005360 0x7fe0bc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:12.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.533+0000 7fe0d7400640 1 -- 172.21.15.86:0/2902890076 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc00c230 con 0x7fe0bc004810 2024-04-25T14:15:12.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.533+0000 7fe0d7400640 1 -- 172.21.15.86:0/2902890076 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc00c3e0 con 0x7fe0bc003a30 2024-04-25T14:15:12.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.533+0000 7fe0d7400640 1 -- 172.21.15.86:0/2902890076 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe0bc00c5e0 con 0x7fe0bc005360 2024-04-25T14:15:12.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.533+0000 7fe0cf400640 1 --2- 172.21.15.86:0/2902890076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0bc004810 0x7fe0bc004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:12.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.533+0000 7fe0d7e00640 1 --2- 172.21.15.86:0/2902890076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0bc003a30 0x7fe0bc003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:12.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.533+0000 7fe0dd200640 1 --2- 172.21.15.86:0/2902890076 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0bc005360 0x7fe0bc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:12.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.533+0000 7fe0cf400640 1 -- 172.21.15.86:0/2902890076 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0bc005360 msgr2=0x7fe0bc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:12.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.533+0000 7fe0cf400640 1 --2- 172.21.15.86:0/2902890076 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0bc005360 0x7fe0bc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:12.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.533+0000 7fe0cf400640 1 -- 172.21.15.86:0/2902890076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0bc003a30 msgr2=0x7fe0bc003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:12.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.533+0000 7fe0cf400640 1 --2- 172.21.15.86:0/2902890076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0bc003a30 0x7fe0bc003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:12.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.533+0000 7fe0cf400640 1 -- 172.21.15.86:0/2902890076 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0bc003530 con 0x7fe0bc004810 2024-04-25T14:15:12.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.533+0000 7fe0dd200640 1 -- 172.21.15.86:0/2902890076 reap_dead start 2024-04-25T14:15:12.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.537+0000 7fe0cf400640 1 --2- 172.21.15.86:0/2902890076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0bc004810 0x7fe0bc004c20 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8074ed0 tx=0x7fe0c001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ade4a46246a70d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:12.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.537+0000 7fe0d7400640 1 -- 172.21.15.86:0/2902890076 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c00096b0 con 0x7fe0bc004810 2024-04-25T14:15:12.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.537+0000 7fe0d7400640 1 -- 172.21.15.86:0/2902890076 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0c00a64f0 con 0x7fe0bc004810 2024-04-25T14:15:12.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.537+0000 7fe0d7400640 1 -- 172.21.15.86:0/2902890076 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c00a68e0 con 0x7fe0bc004810 2024-04-25T14:15:12.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.537+0000 7fe0df200640 1 -- 172.21.15.86:0/2902890076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0bc004810 msgr2=0x7fe0bc004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:12.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.537+0000 7fe0df200640 1 --2- 172.21.15.86:0/2902890076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0bc004810 0x7fe0bc004c20 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8074ed0 tx=0x7fe0c001ca00 comp rx=0 tx=0).stop 2024-04-25T14:15:12.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.537+0000 7fe0df200640 1 -- 172.21.15.86:0/2902890076 shutdown_connections 2024-04-25T14:15:12.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.537+0000 7fe0df200640 1 --2- 172.21.15.86:0/2902890076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0bc004810 0x7fe0bc004c20 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:12.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.537+0000 7fe0df200640 1 -- 172.21.15.86:0/2902890076 >> 172.21.15.86:0/2902890076 conn(0x7fe0d80b4650 msgr2=0x7fe0d80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:12.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.537+0000 7fe0df200640 1 -- 172.21.15.86:0/2902890076 shutdown_connections 2024-04-25T14:15:12.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.537+0000 7fe0df200640 1 -- 172.21.15.86:0/2902890076 wait complete. 2024-04-25T14:15:12.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.537+0000 7fe0df200640 1 Processor -- start 2024-04-25T14:15:12.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.541+0000 7fe0df200640 1 -- start start 2024-04-25T14:15:12.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.541+0000 7fe0df200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0bc004810 0x7fe0d815b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:12.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.541+0000 7fe0df200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0d815f5d0 0x7fe0d815f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:12.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.541+0000 7fe0df200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0d80ae130 0x7fe0d80ae540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:12.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.541+0000 7fe0df200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d80708c0 con 0x7fe0bc004810 2024-04-25T14:15:12.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.541+0000 7fe0df200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d8070740 con 0x7fe0d80ae130 2024-04-25T14:15:12.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.541+0000 7fe0df200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe0d80ce0d0 con 0x7fe0d815f5d0 2024-04-25T14:15:12.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.541+0000 7fe0d7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0bc004810 0x7fe0d815b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:12.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.541+0000 7fe0d7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0bc004810 0x7fe0d815b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48280/0 (socket says 172.21.15.86:48280) 2024-04-25T14:15:12.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.541+0000 7fe0d7e00640 1 -- 172.21.15.86:0/2654692399 learned_addr learned my addr 172.21.15.86:0/2654692399 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:12.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.541+0000 7fe0dd200640 1 --2- 172.21.15.86:0/2654692399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0d80ae130 0x7fe0d80ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:12.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.541+0000 7fe0cf400640 1 --2- 172.21.15.86:0/2654692399 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0d815f5d0 0x7fe0d815f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:12.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.541+0000 7fe0d7e00640 1 -- 172.21.15.86:0/2654692399 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0d815f5d0 msgr2=0x7fe0d815f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:12.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.541+0000 7fe0d7e00640 1 --2- 172.21.15.86:0/2654692399 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0d815f5d0 0x7fe0d815f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:12.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.541+0000 7fe0d7e00640 1 -- 172.21.15.86:0/2654692399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0d80ae130 msgr2=0x7fe0d80ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:12.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.541+0000 7fe0d7e00640 1 --2- 172.21.15.86:0/2654692399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0d80ae130 0x7fe0d80ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:12.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.541+0000 7fe0d7e00640 1 -- 172.21.15.86:0/2654692399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d815b760 con 0x7fe0bc004810 2024-04-25T14:15:12.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.541+0000 7fe0cf400640 1 --2- 172.21.15.86:0/2654692399 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0d815f5d0 0x7fe0d815f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:12.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.541+0000 7fe0d7e00640 1 --2- 172.21.15.86:0/2654692399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0bc004810 0x7fe0d815b220 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800e490 tx=0x7fe0c800e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:12.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.541+0000 7fe0d4c00640 1 -- 172.21.15.86:0/2654692399 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c80052e0 con 0x7fe0bc004810 2024-04-25T14:15:12.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.541+0000 7fe0d4c00640 1 -- 172.21.15.86:0/2654692399 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0c8018070 con 0x7fe0bc004810 2024-04-25T14:15:12.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.541+0000 7fe0df200640 1 -- 172.21.15.86:0/2654692399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0d80aea80 con 0x7fe0bc004810 2024-04-25T14:15:12.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.541+0000 7fe0d4c00640 1 -- 172.21.15.86:0/2654692399 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8005480 con 0x7fe0bc004810 2024-04-25T14:15:12.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.541+0000 7fe0df200640 1 -- 172.21.15.86:0/2654692399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0d8160630 con 0x7fe0bc004810 2024-04-25T14:15:12.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.541+0000 7fe0df200640 1 -- 172.21.15.86:0/2654692399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0d8069d00 con 0x7fe0bc004810 2024-04-25T14:15:12.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.545+0000 7fe0d4c00640 1 -- 172.21.15.86:0/2654692399 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7fe0c8003820 con 0x7fe0bc004810 2024-04-25T14:15:12.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.545+0000 7fe0d4c00640 1 --2- 172.21.15.86:0/2654692399 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe0b406f570 0x7fe0b4071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:12.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.545+0000 7fe0d4c00640 1 -- 172.21.15.86:0/2654692399 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fe0c808fec0 con 0x7fe0bc004810 2024-04-25T14:15:12.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.545+0000 7fe0cf400640 1 --2- 172.21.15.86:0/2654692399 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe0b406f570 0x7fe0b4071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:12.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.545+0000 7fe0cf400640 1 --2- 172.21.15.86:0/2654692399 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe0b406f570 0x7fe0b4071a80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe0c00af000 tx=0x7fe0c00046d0 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:12.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.549+0000 7fe0d4c00640 1 -- 172.21.15.86:0/2654692399 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe0d80aea80 con 0x7fe0bc004810 2024-04-25T14:15:12.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.737+0000 7fe0df200640 1 -- 172.21.15.86:0/2654692399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0d8076bf0 con 0x7fe0bc004810 2024-04-25T14:15:12.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.737+0000 7fe0d4c00640 1 -- 172.21.15.86:0/2654692399 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7fe0c805c9a0 con 0x7fe0bc004810 2024-04-25T14:15:12.735 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:12.735 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:12.735 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:12.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.741+0000 7fe0df200640 1 -- 172.21.15.86:0/2654692399 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe0b406f570 msgr2=0x7fe0b4071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:12.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.741+0000 7fe0df200640 1 --2- 172.21.15.86:0/2654692399 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe0b406f570 0x7fe0b4071a80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe0c00af000 tx=0x7fe0c00046d0 comp rx=0 tx=0).stop 2024-04-25T14:15:12.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.741+0000 7fe0df200640 1 -- 172.21.15.86:0/2654692399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0bc004810 msgr2=0x7fe0d815b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:12.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.741+0000 7fe0df200640 1 --2- 172.21.15.86:0/2654692399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0bc004810 0x7fe0d815b220 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800e490 tx=0x7fe0c800e950 comp rx=0 tx=0).stop 2024-04-25T14:15:12.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.741+0000 7fe0df200640 1 -- 172.21.15.86:0/2654692399 shutdown_connections 2024-04-25T14:15:12.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.741+0000 7fe0df200640 1 --2- 172.21.15.86:0/2654692399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0d80ae130 0x7fe0d80ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:12.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.741+0000 7fe0df200640 1 --2- 172.21.15.86:0/2654692399 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe0b406f570 0x7fe0b4071a80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:12.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.741+0000 7fe0df200640 1 --2- 172.21.15.86:0/2654692399 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0d815f5d0 0x7fe0d815f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:12.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.741+0000 7fe0df200640 1 --2- 172.21.15.86:0/2654692399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0bc004810 0x7fe0d815b220 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:12.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.741+0000 7fe0df200640 1 -- 172.21.15.86:0/2654692399 >> 172.21.15.86:0/2654692399 conn(0x7fe0d80b4650 msgr2=0x7fe0d80b1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:12.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.741+0000 7fe0df200640 1 -- 172.21.15.86:0/2654692399 shutdown_connections 2024-04-25T14:15:12.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:12.741+0000 7fe0df200640 1 -- 172.21.15.86:0/2654692399 wait complete. 2024-04-25T14:15:13.161 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:13.161 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:14.162 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:14.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.325+0000 7f2d99e00640 1 Processor -- start 2024-04-25T14:15:14.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.325+0000 7f2d99e00640 1 -- start start 2024-04-25T14:15:14.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.325+0000 7f2d99e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2d940ced30 0x7f2d940b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:14.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.325+0000 7f2d99e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2d9415dcd0 0x7f2d940d1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.325+0000 7f2d99e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d94161b30 con 0x7f2d940d1970 2024-04-25T14:15:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.325+0000 7f2d99e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2d941618e0 con 0x7f2d940ced30 2024-04-25T14:15:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.325+0000 7f2d99e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f2d94161e10 con 0x7f2d9415dcd0 2024-04-25T14:15:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.325+0000 7f2d98a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2d940ced30 0x7f2d940b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.325+0000 7f2d98a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2d940ced30 0x7f2d940b1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48292/0 (socket says 172.21.15.86:48292) 2024-04-25T14:15:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.325+0000 7f2d98a00640 1 -- 172.21.15.86:0/2867426046 learned_addr learned my addr 172.21.15.86:0/2867426046 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:14.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.325+0000 7f2d98a00640 1 -- 172.21.15.86:0/2867426046 >> v2:172.21.15.155:3300/0 conn(0x7f2d9415dcd0 msgr2=0x7f2d940d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.325+0000 7f2d98a00640 1 --2- 172.21.15.86:0/2867426046 >> v2:172.21.15.155:3300/0 conn(0x7f2d9415dcd0 0x7f2d940d1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.325+0000 7f2d98a00640 1 -- 172.21.15.86:0/2867426046 >> v1:172.21.15.155:6789/0 conn(0x7f2d940d1970 legacy=0x7f2d9415e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.325+0000 7f2d98a00640 1 -- 172.21.15.86:0/2867426046 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d941620f0 con 0x7f2d940ced30 2024-04-25T14:15:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.329+0000 7f2d98a00640 1 --2- 172.21.15.86:0/2867426046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d940ced30 0x7f2d940b1e20 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f2d94073180 tx=0x7f2d8401c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1082adb7a4ab7021 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:14.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.329+0000 7f2d93400640 1 -- 172.21.15.86:0/2867426046 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d840a5820 con 0x7f2d940ced30 2024-04-25T14:15:14.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.329+0000 7f2d93400640 1 -- 172.21.15.86:0/2867426046 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d840a59c0 con 0x7f2d940ced30 2024-04-25T14:15:14.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.329+0000 7f2d93400640 1 -- 172.21.15.86:0/2867426046 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d840a5d90 con 0x7f2d940ced30 2024-04-25T14:15:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.329+0000 7f2d99e00640 1 -- 172.21.15.86:0/2867426046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d940ced30 msgr2=0x7f2d940b1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.329+0000 7f2d99e00640 1 --2- 172.21.15.86:0/2867426046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d940ced30 0x7f2d940b1e20 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f2d94073180 tx=0x7f2d8401c980 comp rx=0 tx=0).stop 2024-04-25T14:15:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.329+0000 7f2d99e00640 1 -- 172.21.15.86:0/2867426046 shutdown_connections 2024-04-25T14:15:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.329+0000 7f2d99e00640 1 --2- 172.21.15.86:0/2867426046 >> v2:172.21.15.155:3300/0 conn(0x7f2d9415dcd0 0x7f2d940d1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.329+0000 7f2d99e00640 1 --2- 172.21.15.86:0/2867426046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d940ced30 0x7f2d940b1e20 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.329+0000 7f2d99e00640 1 -- 172.21.15.86:0/2867426046 >> 172.21.15.86:0/2867426046 conn(0x7f2d9409ba30 msgr2=0x7f2d940b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.329+0000 7f2d99e00640 1 -- 172.21.15.86:0/2867426046 shutdown_connections 2024-04-25T14:15:14.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.329+0000 7f2d99e00640 1 -- 172.21.15.86:0/2867426046 wait complete. 2024-04-25T14:15:14.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.333+0000 7f2d99e00640 1 Processor -- start 2024-04-25T14:15:14.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.333+0000 7f2d99e00640 1 -- start start 2024-04-25T14:15:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.333+0000 7f2d99e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d940ced30 0x7f2d94157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.333+0000 7f2d99e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d940d1970 0x7f2d940cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:14.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.333+0000 7f2d99e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d9415dcd0 0x7f2d940cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:14.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.333+0000 7f2d99e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9416aa70 con 0x7f2d940ced30 2024-04-25T14:15:14.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.333+0000 7f2d99e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2d940d1d80 con 0x7f2d940d1970 2024-04-25T14:15:14.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.333+0000 7f2d99e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2d9416abf0 con 0x7f2d9415dcd0 2024-04-25T14:15:14.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.333+0000 7f2d98a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d940ced30 0x7f2d94157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:14.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.333+0000 7f2d98a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d940ced30 0x7f2d94157d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48304/0 (socket says 172.21.15.86:48304) 2024-04-25T14:15:14.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.333+0000 7f2d98a00640 1 -- 172.21.15.86:0/2343858588 learned_addr learned my addr 172.21.15.86:0/2343858588 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:14.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.333+0000 7f2d93e00640 1 --2- 172.21.15.86:0/2343858588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d940d1970 0x7f2d940cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:14.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.333+0000 7f2d98a00640 1 -- 172.21.15.86:0/2343858588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d9415dcd0 msgr2=0x7f2d940cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:14.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.333+0000 7f2d98a00640 1 --2- 172.21.15.86:0/2343858588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d9415dcd0 0x7f2d940cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:14.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.333+0000 7f2d98a00640 1 -- 172.21.15.86:0/2343858588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d940d1970 msgr2=0x7f2d940cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:14.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.333+0000 7f2d98a00640 1 --2- 172.21.15.86:0/2343858588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d940d1970 0x7f2d940cb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:14.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.333+0000 7f2d98a00640 1 -- 172.21.15.86:0/2343858588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d94155fa0 con 0x7f2d940ced30 2024-04-25T14:15:14.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.333+0000 7f2d93e00640 1 --2- 172.21.15.86:0/2343858588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d940d1970 0x7f2d940cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:15:14.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.337+0000 7f2d98a00640 1 --2- 172.21.15.86:0/2343858588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d940ced30 0x7f2d94157d80 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f2d8401b030 tx=0x7f2d84019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:14.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.337+0000 7f2d90c00640 1 -- 172.21.15.86:0/2343858588 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d840b8880 con 0x7f2d940ced30 2024-04-25T14:15:14.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.337+0000 7f2d90c00640 1 -- 172.21.15.86:0/2343858588 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d840be070 con 0x7f2d940ced30 2024-04-25T14:15:14.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.337+0000 7f2d90c00640 1 -- 172.21.15.86:0/2343858588 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d840b8ca0 con 0x7f2d940ced30 2024-04-25T14:15:14.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.337+0000 7f2d99e00640 1 -- 172.21.15.86:0/2343858588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d940cd480 con 0x7f2d940ced30 2024-04-25T14:15:14.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.337+0000 7f2d99e00640 1 -- 172.21.15.86:0/2343858588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d940cda40 con 0x7f2d940ced30 2024-04-25T14:15:14.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.337+0000 7f2d99e00640 1 -- 172.21.15.86:0/2343858588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d94076b80 con 0x7f2d940ced30 2024-04-25T14:15:14.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.337+0000 7f2d90c00640 1 -- 172.21.15.86:0/2343858588 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f2d840c2410 con 0x7f2d940ced30 2024-04-25T14:15:14.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.337+0000 7f2d90c00640 1 --2- 172.21.15.86:0/2343858588 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f2d7406f5a0 0x7f2d74071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:14.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.337+0000 7f2d90c00640 1 -- 172.21.15.86:0/2343858588 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f2d8412a390 con 0x7f2d940ced30 2024-04-25T14:15:14.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.341+0000 7f2d93e00640 1 --2- 172.21.15.86:0/2343858588 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f2d7406f5a0 0x7f2d74071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:14.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.341+0000 7f2d93e00640 1 --2- 172.21.15.86:0/2343858588 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f2d7406f5a0 0x7f2d74071ab0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2d94161820 tx=0x7f2d80008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:14.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.345+0000 7f2d90c00640 1 -- 172.21.15.86:0/2343858588 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2d840c7020 con 0x7f2d940ced30 2024-04-25T14:15:14.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.533+0000 7f2d99e00640 1 -- 172.21.15.86:0/2343858588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d940d13f0 con 0x7f2d940ced30 2024-04-25T14:15:14.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.533+0000 7f2d90c00640 1 -- 172.21.15.86:0/2343858588 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f2d840fde20 con 0x7f2d940ced30 2024-04-25T14:15:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:14.531 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:14.532 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:14.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.537+0000 7f2d99e00640 1 -- 172.21.15.86:0/2343858588 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f2d7406f5a0 msgr2=0x7f2d74071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:14.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.537+0000 7f2d99e00640 1 --2- 172.21.15.86:0/2343858588 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f2d7406f5a0 0x7f2d74071ab0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2d94161820 tx=0x7f2d80008040 comp rx=0 tx=0).stop 2024-04-25T14:15:14.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.537+0000 7f2d99e00640 1 -- 172.21.15.86:0/2343858588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d940ced30 msgr2=0x7f2d94157d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:14.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.537+0000 7f2d99e00640 1 --2- 172.21.15.86:0/2343858588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d940ced30 0x7f2d94157d80 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f2d8401b030 tx=0x7f2d84019720 comp rx=0 tx=0).stop 2024-04-25T14:15:14.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.537+0000 7f2d99e00640 1 -- 172.21.15.86:0/2343858588 shutdown_connections 2024-04-25T14:15:14.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.537+0000 7f2d99e00640 1 --2- 172.21.15.86:0/2343858588 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f2d7406f5a0 0x7f2d74071ab0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:14.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.537+0000 7f2d99e00640 1 --2- 172.21.15.86:0/2343858588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d9415dcd0 0x7f2d940cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:14.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.537+0000 7f2d99e00640 1 --2- 172.21.15.86:0/2343858588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d940d1970 0x7f2d940cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:14.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.537+0000 7f2d99e00640 1 --2- 172.21.15.86:0/2343858588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d940ced30 0x7f2d94157d80 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:14.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.537+0000 7f2d99e00640 1 -- 172.21.15.86:0/2343858588 >> 172.21.15.86:0/2343858588 conn(0x7f2d9409ba30 msgr2=0x7f2d9415bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:14.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.537+0000 7f2d99e00640 1 -- 172.21.15.86:0/2343858588 shutdown_connections 2024-04-25T14:15:14.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:14.537+0000 7f2d99e00640 1 -- 172.21.15.86:0/2343858588 wait complete. 2024-04-25T14:15:14.956 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:14.956 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:15.957 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:16.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.121+0000 7f41c4e00640 1 Processor -- start 2024-04-25T14:15:16.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.121+0000 7f41c4e00640 1 -- start start 2024-04-25T14:15:16.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.121+0000 7f41c4e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f41c00cfca0 0x7f41c006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:16.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.121+0000 7f41c4e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f41c00730b0 0x7f41c00734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:16.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.121+0000 7f41c4e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41c0072bd0 con 0x7f41c00d16e0 2024-04-25T14:15:16.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.121+0000 7f41c4e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f41c0070470 con 0x7f41c00cfca0 2024-04-25T14:15:16.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.121+0000 7f41c4e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f41c006f930 con 0x7f41c00730b0 2024-04-25T14:15:16.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.121+0000 7f41bde00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f41c00cfca0 0x7f41c006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:16.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.121+0000 7f41bde00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f41c00cfca0 0x7f41c006f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48320/0 (socket says 172.21.15.86:48320) 2024-04-25T14:15:16.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.121+0000 7f41bde00640 1 -- 172.21.15.86:0/2542706706 learned_addr learned my addr 172.21.15.86:0/2542706706 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:16.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.121+0000 7f41be800640 1 --2- 172.21.15.86:0/2542706706 >> v2:172.21.15.155:3300/0 conn(0x7f41c00730b0 0x7f41c00734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:16.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.121+0000 7f41bde00640 1 -- 172.21.15.86:0/2542706706 >> v2:172.21.15.155:3300/0 conn(0x7f41c00730b0 msgr2=0x7f41c00734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:16.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.121+0000 7f41bde00640 1 --2- 172.21.15.86:0/2542706706 >> v2:172.21.15.155:3300/0 conn(0x7f41c00730b0 0x7f41c00734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:16.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.121+0000 7f41bde00640 1 -- 172.21.15.86:0/2542706706 >> v1:172.21.15.155:6789/0 conn(0x7f41c00d16e0 legacy=0x7f41c00d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:16.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.121+0000 7f41bde00640 1 -- 172.21.15.86:0/2542706706 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c006fbd0 con 0x7f41c00cfca0 2024-04-25T14:15:16.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.121+0000 7f41be800640 1 --2- 172.21.15.86:0/2542706706 >> v2:172.21.15.155:3300/0 conn(0x7f41c00730b0 0x7f41c00734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:16.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.125+0000 7f41bde00640 1 --2- 172.21.15.86:0/2542706706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41c00cfca0 0x7f41c006f3f0 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f41c00cb8c0 tx=0x7f41b001ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f06fcd6cb68e5b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:16.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.125+0000 7f41bca00640 1 -- 172.21.15.86:0/2542706706 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b000b2d0 con 0x7f41c00cfca0 2024-04-25T14:15:16.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.125+0000 7f41bca00640 1 -- 172.21.15.86:0/2542706706 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41b000b470 con 0x7f41c00cfca0 2024-04-25T14:15:16.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.125+0000 7f41bca00640 1 -- 172.21.15.86:0/2542706706 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b00a7bf0 con 0x7f41c00cfca0 2024-04-25T14:15:16.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.125+0000 7f41c4e00640 1 -- 172.21.15.86:0/2542706706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41c00cfca0 msgr2=0x7f41c006f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:16.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.125+0000 7f41c4e00640 1 --2- 172.21.15.86:0/2542706706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41c00cfca0 0x7f41c006f3f0 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f41c00cb8c0 tx=0x7f41b001ebf0 comp rx=0 tx=0).stop 2024-04-25T14:15:16.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.125+0000 7f41c4e00640 1 -- 172.21.15.86:0/2542706706 shutdown_connections 2024-04-25T14:15:16.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.125+0000 7f41c4e00640 1 --2- 172.21.15.86:0/2542706706 >> v2:172.21.15.155:3300/0 conn(0x7f41c00730b0 0x7f41c00734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:16.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.125+0000 7f41c4e00640 1 --2- 172.21.15.86:0/2542706706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41c00cfca0 0x7f41c006f3f0 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:16.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.125+0000 7f41c4e00640 1 -- 172.21.15.86:0/2542706706 >> 172.21.15.86:0/2542706706 conn(0x7f41c00b4650 msgr2=0x7f41c00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:16.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.125+0000 7f41c4e00640 1 -- 172.21.15.86:0/2542706706 shutdown_connections 2024-04-25T14:15:16.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.125+0000 7f41c4e00640 1 -- 172.21.15.86:0/2542706706 wait complete. 2024-04-25T14:15:16.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.129+0000 7f41c4e00640 1 Processor -- start 2024-04-25T14:15:16.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.129+0000 7f41c4e00640 1 -- start start 2024-04-25T14:15:16.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.129+0000 7f41c4e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41c00730b0 0x7f41c00cb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:16.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.129+0000 7f41c4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41c00cfca0 0x7f41c00ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:16.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.129+0000 7f41c4e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41c00d16e0 0x7f41c00c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:16.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.129+0000 7f41c4e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f41c00ce0f0 con 0x7f41c00cfca0 2024-04-25T14:15:16.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.129+0000 7f41c4e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f41c00cdf70 con 0x7f41c00730b0 2024-04-25T14:15:16.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.129+0000 7f41c4e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f41c00ce270 con 0x7f41c00d16e0 2024-04-25T14:15:16.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.129+0000 7f41bd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41c00cfca0 0x7f41c00ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:16.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.129+0000 7f41bd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41c00cfca0 0x7f41c00ccc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48322/0 (socket says 172.21.15.86:48322) 2024-04-25T14:15:16.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.129+0000 7f41bd400640 1 -- 172.21.15.86:0/2793811152 learned_addr learned my addr 172.21.15.86:0/2793811152 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:16.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.129+0000 7f41bde00640 1 --2- 172.21.15.86:0/2793811152 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41c00730b0 0x7f41c00cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:16.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.129+0000 7f41be800640 1 --2- 172.21.15.86:0/2793811152 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41c00d16e0 0x7f41c00c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:16.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.133+0000 7f41bd400640 1 -- 172.21.15.86:0/2793811152 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41c00d16e0 msgr2=0x7f41c00c3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:16.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.133+0000 7f41bd400640 1 --2- 172.21.15.86:0/2793811152 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41c00d16e0 0x7f41c00c3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:16.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.133+0000 7f41bd400640 1 -- 172.21.15.86:0/2793811152 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41c00730b0 msgr2=0x7f41c00cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:16.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.133+0000 7f41bd400640 1 --2- 172.21.15.86:0/2793811152 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41c00730b0 0x7f41c00cb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:16.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.133+0000 7f41bd400640 1 -- 172.21.15.86:0/2793811152 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c00ca0c0 con 0x7f41c00cfca0 2024-04-25T14:15:16.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.133+0000 7f41bd400640 1 --2- 172.21.15.86:0/2793811152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41c00cfca0 0x7f41c00ccc20 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f41ac00c340 tx=0x7f41ac00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:16.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.133+0000 7f41b6000640 1 -- 172.21.15.86:0/2793811152 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac0052d0 con 0x7f41c00cfca0 2024-04-25T14:15:16.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.133+0000 7f41b6000640 1 -- 172.21.15.86:0/2793811152 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41ac005470 con 0x7f41c00cfca0 2024-04-25T14:15:16.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.133+0000 7f41c4e00640 1 -- 172.21.15.86:0/2793811152 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c00ca3a0 con 0x7f41c00cfca0 2024-04-25T14:15:16.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.133+0000 7f41b6000640 1 -- 172.21.15.86:0/2793811152 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac007260 con 0x7f41c00cfca0 2024-04-25T14:15:16.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.133+0000 7f41c4e00640 1 -- 172.21.15.86:0/2793811152 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c00ca960 con 0x7f41c00cfca0 2024-04-25T14:15:16.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.133+0000 7f41c4e00640 1 -- 172.21.15.86:0/2793811152 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c00c9e30 con 0x7f41c00cfca0 2024-04-25T14:15:16.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.133+0000 7f41b6000640 1 -- 172.21.15.86:0/2793811152 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f41ac007400 con 0x7f41c00cfca0 2024-04-25T14:15:16.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.133+0000 7f41b6000640 1 --2- 172.21.15.86:0/2793811152 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f41a006f570 0x7f41a0071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:16.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.133+0000 7f41b6000640 1 -- 172.21.15.86:0/2793811152 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f41ac08eb40 con 0x7f41c00cfca0 2024-04-25T14:15:16.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.137+0000 7f41bde00640 1 --2- 172.21.15.86:0/2793811152 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f41a006f570 0x7f41a0071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:16.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.137+0000 7f41bde00640 1 --2- 172.21.15.86:0/2793811152 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f41a006f570 0x7f41a0071a80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f41b00b0000 tx=0x7f41b00a9070 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:16.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.141+0000 7f41b6000640 1 -- 172.21.15.86:0/2793811152 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f41ac05afe0 con 0x7f41c00cfca0 2024-04-25T14:15:16.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.329+0000 7f41c4e00640 1 -- 172.21.15.86:0/2793811152 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41c009f4b0 con 0x7f41c00cfca0 2024-04-25T14:15:16.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.329+0000 7f41b6000640 1 -- 172.21.15.86:0/2793811152 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f41ac05f2e0 con 0x7f41c00cfca0 2024-04-25T14:15:16.326 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:16.326 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:16.326 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:16.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.329+0000 7f41c4e00640 1 -- 172.21.15.86:0/2793811152 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f41a006f570 msgr2=0x7f41a0071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:16.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.329+0000 7f41c4e00640 1 --2- 172.21.15.86:0/2793811152 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f41a006f570 0x7f41a0071a80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f41b00b0000 tx=0x7f41b00a9070 comp rx=0 tx=0).stop 2024-04-25T14:15:16.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.329+0000 7f41c4e00640 1 -- 172.21.15.86:0/2793811152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41c00cfca0 msgr2=0x7f41c00ccc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:16.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.329+0000 7f41c4e00640 1 --2- 172.21.15.86:0/2793811152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41c00cfca0 0x7f41c00ccc20 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f41ac00c340 tx=0x7f41ac00c800 comp rx=0 tx=0).stop 2024-04-25T14:15:16.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.329+0000 7f41c4e00640 1 -- 172.21.15.86:0/2793811152 shutdown_connections 2024-04-25T14:15:16.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.329+0000 7f41c4e00640 1 --2- 172.21.15.86:0/2793811152 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f41a006f570 0x7f41a0071a80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:16.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.329+0000 7f41c4e00640 1 --2- 172.21.15.86:0/2793811152 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41c00d16e0 0x7f41c00c3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:16.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.329+0000 7f41c4e00640 1 --2- 172.21.15.86:0/2793811152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41c00cfca0 0x7f41c00ccc20 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:16.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.329+0000 7f41c4e00640 1 --2- 172.21.15.86:0/2793811152 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41c00730b0 0x7f41c00cb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:16.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.333+0000 7f41c4e00640 1 -- 172.21.15.86:0/2793811152 >> 172.21.15.86:0/2793811152 conn(0x7f41c00b4650 msgr2=0x7f41c00b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:16.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.333+0000 7f41c4e00640 1 -- 172.21.15.86:0/2793811152 shutdown_connections 2024-04-25T14:15:16.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:16.333+0000 7f41c4e00640 1 -- 172.21.15.86:0/2793811152 wait complete. 2024-04-25T14:15:16.754 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:16.755 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:17.756 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:17.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.917+0000 7f5ef1e00640 1 Processor -- start 2024-04-25T14:15:17.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.917+0000 7f5ef1e00640 1 -- start start 2024-04-25T14:15:17.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.921+0000 7f5ef1e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5eec0cde90 0x7f5eec0ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.921+0000 7f5ef1e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5eec15e5c0 con 0x7f5eec0d1df0 2024-04-25T14:15:17.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.921+0000 7f5ef1e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5eec15e8a0 con 0x7f5eec070330 2024-04-25T14:15:17.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.921+0000 7f5ef1e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5eec06fd40 con 0x7f5eec0cde90 2024-04-25T14:15:17.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.921+0000 7f5eeb400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5eec0cde90 0x7f5eec0ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.921+0000 7f5eeb400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5eec0cde90 0x7f5eec0ce2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48330/0 (socket says 172.21.15.86:48330) 2024-04-25T14:15:17.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.921+0000 7f5eeb400640 1 -- 172.21.15.86:0/3937711084 learned_addr learned my addr 172.21.15.86:0/3937711084 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:17.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.921+0000 7f5eeb400640 1 -- 172.21.15.86:0/3937711084 >> v1:172.21.15.155:6789/0 conn(0x7f5eec070330 legacy=0x7f5eec0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.921+0000 7f5eeb400640 1 -- 172.21.15.86:0/3937711084 >> v1:172.21.15.86:6789/0 conn(0x7f5eec0d1df0 legacy=0x7f5eec0cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.921+0000 7f5eeb400640 1 -- 172.21.15.86:0/3937711084 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eec15eb80 con 0x7f5eec0cde90 2024-04-25T14:15:17.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.925+0000 7f5eeb400640 1 --2- 172.21.15.86:0/3937711084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eec0cde90 0x7f5eec0ce2a0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4009e80 tx=0x7f5ee401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696516ba31db741e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.925+0000 7f5ee9600640 1 -- 172.21.15.86:0/3937711084 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee40a5720 con 0x7f5eec0cde90 2024-04-25T14:15:17.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.925+0000 7f5ee9600640 1 -- 172.21.15.86:0/3937711084 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ee40a58c0 con 0x7f5eec0cde90 2024-04-25T14:15:17.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.925+0000 7f5ee9600640 1 -- 172.21.15.86:0/3937711084 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee40a5c90 con 0x7f5eec0cde90 2024-04-25T14:15:17.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.925+0000 7f5ef1e00640 1 -- 172.21.15.86:0/3937711084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eec0cde90 msgr2=0x7f5eec0ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.925+0000 7f5ef1e00640 1 --2- 172.21.15.86:0/3937711084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eec0cde90 0x7f5eec0ce2a0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4009e80 tx=0x7f5ee401c950 comp rx=0 tx=0).stop 2024-04-25T14:15:17.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.925+0000 7f5ef1e00640 1 -- 172.21.15.86:0/3937711084 shutdown_connections 2024-04-25T14:15:17.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.925+0000 7f5ef1e00640 1 --2- 172.21.15.86:0/3937711084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eec0cde90 0x7f5eec0ce2a0 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.925+0000 7f5ef1e00640 1 -- 172.21.15.86:0/3937711084 >> 172.21.15.86:0/3937711084 conn(0x7f5eec0b4650 msgr2=0x7f5eec0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:17.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.925+0000 7f5ef1e00640 1 -- 172.21.15.86:0/3937711084 shutdown_connections 2024-04-25T14:15:17.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.925+0000 7f5ef1e00640 1 -- 172.21.15.86:0/3937711084 wait complete. 2024-04-25T14:15:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.929+0000 7f5ef1e00640 1 Processor -- start 2024-04-25T14:15:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.929+0000 7f5ef1e00640 1 -- start start 2024-04-25T14:15:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.929+0000 7f5ef1e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5eec070330 0x7f5eec157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.929+0000 7f5ef1e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5eec0cde90 0x7f5eec09a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.929+0000 7f5ef1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eec158a40 0x7f5eec158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.929+0000 7f5ef1e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5eec0708c0 con 0x7f5eec158a40 2024-04-25T14:15:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.929+0000 7f5ef1e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5eec070740 con 0x7f5eec070330 2024-04-25T14:15:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.929+0000 7f5ef1e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5eec16bfd0 con 0x7f5eec0cde90 2024-04-25T14:15:17.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.929+0000 7f5eeb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eec158a40 0x7f5eec158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.929+0000 7f5eeb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eec158a40 0x7f5eec158e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48336/0 (socket says 172.21.15.86:48336) 2024-04-25T14:15:17.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.929+0000 7f5eeb400640 1 -- 172.21.15.86:0/2153268363 learned_addr learned my addr 172.21.15.86:0/2153268363 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:17.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.929+0000 7f5eeaa00640 1 --2- 172.21.15.86:0/2153268363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5eec070330 0x7f5eec157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.929+0000 7f5eea000640 1 --2- 172.21.15.86:0/2153268363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5eec0cde90 0x7f5eec09a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.929+0000 7f5eeb400640 1 -- 172.21.15.86:0/2153268363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5eec0cde90 msgr2=0x7f5eec09a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.929+0000 7f5eeb400640 1 --2- 172.21.15.86:0/2153268363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5eec0cde90 0x7f5eec09a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.929+0000 7f5eeb400640 1 -- 172.21.15.86:0/2153268363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5eec070330 msgr2=0x7f5eec157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:17.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.929+0000 7f5eeb400640 1 --2- 172.21.15.86:0/2153268363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5eec070330 0x7f5eec157ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:17.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.929+0000 7f5eeb400640 1 -- 172.21.15.86:0/2153268363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eec0d1ea0 con 0x7f5eec158a40 2024-04-25T14:15:17.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.929+0000 7f5eeb400640 1 --2- 172.21.15.86:0/2153268363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eec158a40 0x7f5eec158e50 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4019560 tx=0x7f5ee40196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.929+0000 7f5ee3400640 1 -- 172.21.15.86:0/2153268363 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee40bc020 con 0x7f5eec158a40 2024-04-25T14:15:17.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.929+0000 7f5ee3400640 1 -- 172.21.15.86:0/2153268363 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ee40b86b0 con 0x7f5eec158a40 2024-04-25T14:15:17.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.929+0000 7f5ef1e00640 1 -- 172.21.15.86:0/2153268363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eec15a9e0 con 0x7f5eec158a40 2024-04-25T14:15:17.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.929+0000 7f5ee3400640 1 -- 172.21.15.86:0/2153268363 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee40b8aa0 con 0x7f5eec158a40 2024-04-25T14:15:17.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.929+0000 7f5ef1e00640 1 -- 172.21.15.86:0/2153268363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eec154c10 con 0x7f5eec158a40 2024-04-25T14:15:17.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.933+0000 7f5ef1e00640 1 -- 172.21.15.86:0/2153268363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eec0c3150 con 0x7f5eec158a40 2024-04-25T14:15:17.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.933+0000 7f5ee3400640 1 -- 172.21.15.86:0/2153268363 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f5ee40bf020 con 0x7f5eec158a40 2024-04-25T14:15:17.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.933+0000 7f5ee3400640 1 --2- 172.21.15.86:0/2153268363 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f5ecc06f5a0 0x7f5ecc071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:17.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.933+0000 7f5ee3400640 1 -- 172.21.15.86:0/2153268363 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f5ee412ad80 con 0x7f5eec158a40 2024-04-25T14:15:17.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.937+0000 7f5eeaa00640 1 --2- 172.21.15.86:0/2153268363 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f5ecc06f5a0 0x7f5ecc071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:17.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.937+0000 7f5eeaa00640 1 --2- 172.21.15.86:0/2153268363 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f5ecc06f5a0 0x7f5ecc071ab0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5edc000f70 tx=0x7f5edc009040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:17.937+0000 7f5ee3400640 1 -- 172.21.15.86:0/2153268363 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5ee40c6020 con 0x7f5eec158a40 2024-04-25T14:15:18.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:18.125+0000 7f5ef1e00640 1 -- 172.21.15.86:0/2153268363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5eec069d00 con 0x7f5eec158a40 2024-04-25T14:15:18.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:18.125+0000 7f5ee3400640 1 -- 172.21.15.86:0/2153268363 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f5eec15a9e0 con 0x7f5eec158a40 2024-04-25T14:15:18.123 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:18.123 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:18.123 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:18.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:18.129+0000 7f5ef1e00640 1 -- 172.21.15.86:0/2153268363 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f5ecc06f5a0 msgr2=0x7f5ecc071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:18.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:18.129+0000 7f5ef1e00640 1 --2- 172.21.15.86:0/2153268363 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f5ecc06f5a0 0x7f5ecc071ab0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5edc000f70 tx=0x7f5edc009040 comp rx=0 tx=0).stop 2024-04-25T14:15:18.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:18.129+0000 7f5ef1e00640 1 -- 172.21.15.86:0/2153268363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eec158a40 msgr2=0x7f5eec158e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:18.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:18.129+0000 7f5ef1e00640 1 --2- 172.21.15.86:0/2153268363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eec158a40 0x7f5eec158e50 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4019560 tx=0x7f5ee40196f0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:18.129+0000 7f5ef1e00640 1 -- 172.21.15.86:0/2153268363 shutdown_connections 2024-04-25T14:15:18.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:18.129+0000 7f5ef1e00640 1 --2- 172.21.15.86:0/2153268363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eec158a40 0x7f5eec158e50 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:18.129+0000 7f5ef1e00640 1 --2- 172.21.15.86:0/2153268363 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f5ecc06f5a0 0x7f5ecc071ab0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:18.129+0000 7f5ef1e00640 1 --2- 172.21.15.86:0/2153268363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5eec0cde90 0x7f5eec09a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:18.129+0000 7f5ef1e00640 1 --2- 172.21.15.86:0/2153268363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5eec070330 0x7f5eec157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:18.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:18.129+0000 7f5ef1e00640 1 -- 172.21.15.86:0/2153268363 >> 172.21.15.86:0/2153268363 conn(0x7f5eec0b4650 msgr2=0x7f5eec0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:18.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:18.129+0000 7f5ef1e00640 1 -- 172.21.15.86:0/2153268363 shutdown_connections 2024-04-25T14:15:18.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:18.129+0000 7f5ef1e00640 1 -- 172.21.15.86:0/2153268363 wait complete. 2024-04-25T14:15:18.547 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:18.547 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:19.548 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:19.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.709+0000 7f325a800640 1 Processor -- start 2024-04-25T14:15:19.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.709+0000 7f325a800640 1 -- start start 2024-04-25T14:15:19.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f325a800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32540cfca0 0x7f325406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f325a800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f32540730b0 0x7f32540734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f325a800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3254072bd0 con 0x7f32540d16e0 2024-04-25T14:15:19.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f325a800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3254070470 con 0x7f32540730b0 2024-04-25T14:15:19.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f325a800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f325406f930 con 0x7f32540cfca0 2024-04-25T14:15:19.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f324aa00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f32540d16e0 0x7f32540d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44024/0 (socket says 172.21.15.86:44024) 2024-04-25T14:15:19.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f324aa00640 1 -- 172.21.15.86:0/602388033 learned_addr learned my addr 172.21.15.86:0/602388033 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:15:19.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f3253e00640 1 --2- 172.21.15.86:0/602388033 >> v2:172.21.15.155:3300/0 conn(0x7f32540730b0 0x7f32540734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f3253400640 1 --2- 172.21.15.86:0/602388033 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32540cfca0 0x7f325406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f3252a00640 1 -- 172.21.15.86:0/602388033 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3848579834 0 0) 0x7f3254072bd0 con 0x7f32540d16e0 2024-04-25T14:15:19.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f3252a00640 1 -- 172.21.15.86:0/602388033 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3238003530 con 0x7f32540d16e0 2024-04-25T14:15:19.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f3252a00640 1 -- 172.21.15.86:0/602388033 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1406584507 0 0) 0x7f3238003530 con 0x7f32540d16e0 2024-04-25T14:15:19.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f3252a00640 1 -- 172.21.15.86:0/602388033 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32540cfca0 msgr2=0x7f325406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:19.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f3252a00640 1 --2- 172.21.15.86:0/602388033 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32540cfca0 0x7f325406f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f3252a00640 1 -- 172.21.15.86:0/602388033 >> v2:172.21.15.155:3300/0 conn(0x7f32540730b0 msgr2=0x7f32540734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:19.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f3252a00640 1 --2- 172.21.15.86:0/602388033 >> v2:172.21.15.155:3300/0 conn(0x7f32540730b0 0x7f32540734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f3252a00640 1 -- 172.21.15.86:0/602388033 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f325406fbd0 con 0x7f32540d16e0 2024-04-25T14:15:19.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f3252a00640 1 -- 172.21.15.86:0/602388033 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f323c001680 con 0x7f32540d16e0 2024-04-25T14:15:19.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f3252a00640 1 -- 172.21.15.86:0/602388033 >> v1:172.21.15.86:6789/0 conn(0x7f32540d16e0 legacy=0x7f32540d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:19.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f3252a00640 1 --2- 172.21.15.86:0/602388033 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32540cfca0 0x7f3238003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f3252a00640 1 --2- 172.21.15.86:0/602388033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3238003fb0 0x7f3238004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f3252a00640 1 --2- 172.21.15.86:0/602388033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3238004d50 0x7f3238005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f3252a00640 1 -- 172.21.15.86:0/602388033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3254070470 con 0x7f3238003fb0 2024-04-25T14:15:19.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f3252a00640 1 -- 172.21.15.86:0/602388033 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f325406f930 con 0x7f3238004d50 2024-04-25T14:15:19.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f3252a00640 1 -- 172.21.15.86:0/602388033 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f323800bab0 con 0x7f32540cfca0 2024-04-25T14:15:19.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f324aa00640 1 --2- 172.21.15.86:0/602388033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3238003fb0 0x7f3238004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f3253400640 1 --2- 172.21.15.86:0/602388033 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32540cfca0 0x7f3238003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f3253e00640 1 --2- 172.21.15.86:0/602388033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3238004d50 0x7f3238005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f3253400640 1 -- 172.21.15.86:0/602388033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3238004d50 msgr2=0x7f3238005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:19.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f3253400640 1 --2- 172.21.15.86:0/602388033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3238004d50 0x7f3238005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f3253400640 1 -- 172.21.15.86:0/602388033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3238003fb0 msgr2=0x7f3238004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:19.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f3253400640 1 --2- 172.21.15.86:0/602388033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3238003fb0 0x7f3238004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f3253400640 1 -- 172.21.15.86:0/602388033 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3238003530 con 0x7f32540cfca0 2024-04-25T14:15:19.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.713+0000 7f324aa00640 1 --2- 172.21.15.86:0/602388033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3238003fb0 0x7f3238004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:19.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.717+0000 7f3253400640 1 --2- 172.21.15.86:0/602388033 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32540cfca0 0x7f3238003a30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3244009f90 tx=0x7f324401cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d93856f99ece8192 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:19.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.717+0000 7f3252a00640 1 -- 172.21.15.86:0/602388033 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32440b2070 con 0x7f32540cfca0 2024-04-25T14:15:19.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.717+0000 7f3252a00640 1 -- 172.21.15.86:0/602388033 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32440a5c90 con 0x7f32540cfca0 2024-04-25T14:15:19.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.717+0000 7f3252a00640 1 -- 172.21.15.86:0/602388033 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3244009c10 con 0x7f32540cfca0 2024-04-25T14:15:19.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.717+0000 7f325a800640 1 -- 172.21.15.86:0/602388033 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32540cfca0 msgr2=0x7f3238003a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:19.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.717+0000 7f325a800640 1 --2- 172.21.15.86:0/602388033 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32540cfca0 0x7f3238003a30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3244009f90 tx=0x7f324401cc80 comp rx=0 tx=0).stop 2024-04-25T14:15:19.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.717+0000 7f3253e00640 1 -- 172.21.15.86:0/602388033 reap_dead start 2024-04-25T14:15:19.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.717+0000 7f325a800640 1 -- 172.21.15.86:0/602388033 shutdown_connections 2024-04-25T14:15:19.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.717+0000 7f325a800640 1 -- 172.21.15.86:0/602388033 >> 172.21.15.86:0/602388033 conn(0x7f32540b4650 msgr2=0x7f32540b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:19.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.717+0000 7f325a800640 1 -- 172.21.15.86:0/602388033 shutdown_connections 2024-04-25T14:15:19.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.717+0000 7f325a800640 1 -- 172.21.15.86:0/602388033 wait complete. 2024-04-25T14:15:19.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.721+0000 7f325a800640 1 Processor -- start 2024-04-25T14:15:19.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.721+0000 7f325a800640 1 -- start start 2024-04-25T14:15:19.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.721+0000 7f325a800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32540cfca0 0x7f32541f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.721+0000 7f325a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f325415bae0 0x7f325409e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.721+0000 7f325a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f325409ed00 0x7f32541f9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.721+0000 7f325a800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f325406f3f0 con 0x7f325409ed00 2024-04-25T14:15:19.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.721+0000 7f325a800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3254070190 con 0x7f325415bae0 2024-04-25T14:15:19.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.721+0000 7f325a800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f325406f570 con 0x7f32540cfca0 2024-04-25T14:15:19.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.721+0000 7f3253e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f325409ed00 0x7f32541f9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.721+0000 7f3253e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f325409ed00 0x7f32541f9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39724/0 (socket says 172.21.15.86:39724) 2024-04-25T14:15:19.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.721+0000 7f3253e00640 1 -- 172.21.15.86:0/2418122754 learned_addr learned my addr 172.21.15.86:0/2418122754 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:19.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.721+0000 7f3253400640 1 --2- 172.21.15.86:0/2418122754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32540cfca0 0x7f32541f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.721+0000 7f324aa00640 1 --2- 172.21.15.86:0/2418122754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f325415bae0 0x7f325409e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.725+0000 7f3253e00640 1 -- 172.21.15.86:0/2418122754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32540cfca0 msgr2=0x7f32541f5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:19.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.725+0000 7f3253e00640 1 --2- 172.21.15.86:0/2418122754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32540cfca0 0x7f32541f5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.725+0000 7f3253e00640 1 -- 172.21.15.86:0/2418122754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f325415bae0 msgr2=0x7f325409e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:19.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.725+0000 7f3253e00640 1 --2- 172.21.15.86:0/2418122754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f325415bae0 0x7f325409e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.725+0000 7f3253e00640 1 -- 172.21.15.86:0/2418122754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f325409f110 con 0x7f325409ed00 2024-04-25T14:15:19.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.725+0000 7f3253e00640 1 --2- 172.21.15.86:0/2418122754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f325409ed00 0x7f32541f9890 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f32540b4220 tx=0x7f324c011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:19.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.725+0000 7f324be00640 1 -- 172.21.15.86:0/2418122754 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c007d60 con 0x7f325409ed00 2024-04-25T14:15:19.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.725+0000 7f324be00640 1 -- 172.21.15.86:0/2418122754 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f324c01b070 con 0x7f325409ed00 2024-04-25T14:15:19.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.725+0000 7f324be00640 1 -- 172.21.15.86:0/2418122754 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c004460 con 0x7f325409ed00 2024-04-25T14:15:19.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.725+0000 7f325a800640 1 -- 172.21.15.86:0/2418122754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f325409d2f0 con 0x7f325409ed00 2024-04-25T14:15:19.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.725+0000 7f325a800640 1 -- 172.21.15.86:0/2418122754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f325409d8b0 con 0x7f325409ed00 2024-04-25T14:15:19.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.725+0000 7f325a800640 1 -- 172.21.15.86:0/2418122754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f325409d060 con 0x7f325409ed00 2024-04-25T14:15:19.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.725+0000 7f324be00640 1 -- 172.21.15.86:0/2418122754 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f324c012040 con 0x7f325409ed00 2024-04-25T14:15:19.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.725+0000 7f324be00640 1 --2- 172.21.15.86:0/2418122754 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f323006f570 0x7f3230071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:19.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.729+0000 7f324be00640 1 -- 172.21.15.86:0/2418122754 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f324c08f230 con 0x7f325409ed00 2024-04-25T14:15:19.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.729+0000 7f3253400640 1 --2- 172.21.15.86:0/2418122754 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f323006f570 0x7f3230071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:19.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.729+0000 7f3253400640 1 --2- 172.21.15.86:0/2418122754 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f323006f570 0x7f3230071a80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f32440af000 tx=0x7f32440a7040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:19.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.733+0000 7f324be00640 1 -- 172.21.15.86:0/2418122754 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f325409f110 con 0x7f325409ed00 2024-04-25T14:15:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.921+0000 7f325a800640 1 -- 172.21.15.86:0/2418122754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f325409f4b0 con 0x7f325409ed00 2024-04-25T14:15:19.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.921+0000 7f324be00640 1 -- 172.21.15.86:0/2418122754 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f324c05bda0 con 0x7f325409ed00 2024-04-25T14:15:19.918 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:19.919 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:19.919 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:19.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.925+0000 7f325a800640 1 -- 172.21.15.86:0/2418122754 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f323006f570 msgr2=0x7f3230071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:19.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.925+0000 7f325a800640 1 --2- 172.21.15.86:0/2418122754 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f323006f570 0x7f3230071a80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f32440af000 tx=0x7f32440a7040 comp rx=0 tx=0).stop 2024-04-25T14:15:19.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.925+0000 7f325a800640 1 -- 172.21.15.86:0/2418122754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f325409ed00 msgr2=0x7f32541f9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:19.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.925+0000 7f325a800640 1 --2- 172.21.15.86:0/2418122754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f325409ed00 0x7f32541f9890 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f32540b4220 tx=0x7f324c011c30 comp rx=0 tx=0).stop 2024-04-25T14:15:19.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.925+0000 7f325a800640 1 -- 172.21.15.86:0/2418122754 shutdown_connections 2024-04-25T14:15:19.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.925+0000 7f325a800640 1 --2- 172.21.15.86:0/2418122754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f325409ed00 0x7f32541f9890 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.925+0000 7f325a800640 1 --2- 172.21.15.86:0/2418122754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f325415bae0 0x7f325409e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.925+0000 7f325a800640 1 --2- 172.21.15.86:0/2418122754 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f323006f570 0x7f3230071a80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.925+0000 7f325a800640 1 --2- 172.21.15.86:0/2418122754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32540cfca0 0x7f32541f5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:19.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.925+0000 7f325a800640 1 -- 172.21.15.86:0/2418122754 >> 172.21.15.86:0/2418122754 conn(0x7f32540b4650 msgr2=0x7f32540b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:19.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.925+0000 7f325a800640 1 -- 172.21.15.86:0/2418122754 shutdown_connections 2024-04-25T14:15:19.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:19.925+0000 7f325a800640 1 -- 172.21.15.86:0/2418122754 wait complete. 2024-04-25T14:15:20.343 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:20.343 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:20.401 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:15:20.405 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:15:21.344 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:21.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.505+0000 7f6082400640 1 Processor -- start 2024-04-25T14:15:21.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.505+0000 7f6082400640 1 -- start start 2024-04-25T14:15:21.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.509+0000 7f6082400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f607c0cfca0 0x7f607c06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:21.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.509+0000 7f6082400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f607c0d16e0 0x7f607c0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:21.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.509+0000 7f6082400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f607c0d2030 con 0x7f607c06c670 2024-04-25T14:15:21.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.509+0000 7f6082400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f607c070470 con 0x7f607c0cfca0 2024-04-25T14:15:21.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.509+0000 7f6082400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f607c072bd0 con 0x7f607c0d16e0 2024-04-25T14:15:21.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.509+0000 7f607b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f607c0cfca0 0x7f607c06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:21.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.509+0000 7f607b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f607c0cfca0 0x7f607c06f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39736/0 (socket says 172.21.15.86:39736) 2024-04-25T14:15:21.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.509+0000 7f607b400640 1 -- 172.21.15.86:0/3735562660 learned_addr learned my addr 172.21.15.86:0/3735562660 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:21.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.509+0000 7f6072a00640 1 --2- 172.21.15.86:0/3735562660 >> v2:172.21.15.155:3300/0 conn(0x7f607c0d16e0 0x7f607c0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:21.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.509+0000 7f607b400640 1 -- 172.21.15.86:0/3735562660 >> v2:172.21.15.155:3300/0 conn(0x7f607c0d16e0 msgr2=0x7f607c0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:21.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.509+0000 7f607b400640 1 --2- 172.21.15.86:0/3735562660 >> v2:172.21.15.155:3300/0 conn(0x7f607c0d16e0 0x7f607c0d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:21.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.509+0000 7f607b400640 1 -- 172.21.15.86:0/3735562660 >> v1:172.21.15.86:6789/0 conn(0x7f607c06c670 legacy=0x7f607c06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:21.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.509+0000 7f607b400640 1 -- 172.21.15.86:0/3735562660 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f607c06f930 con 0x7f607c0cfca0 2024-04-25T14:15:21.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.509+0000 7f6072a00640 1 --2- 172.21.15.86:0/3735562660 >> v2:172.21.15.155:3300/0 conn(0x7f607c0d16e0 0x7f607c0d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:21.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.513+0000 7f607b400640 1 --2- 172.21.15.86:0/3735562660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f607c0cfca0 0x7f607c06f3f0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f607c0729c0 tx=0x7f606c01ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9fdf25b56a7631 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:21.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.513+0000 7f607aa00640 1 -- 172.21.15.86:0/3735562660 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c00b2d0 con 0x7f607c0cfca0 2024-04-25T14:15:21.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.513+0000 7f607aa00640 1 -- 172.21.15.86:0/3735562660 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f606c00b470 con 0x7f607c0cfca0 2024-04-25T14:15:21.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.513+0000 7f607aa00640 1 -- 172.21.15.86:0/3735562660 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c0a7af0 con 0x7f607c0cfca0 2024-04-25T14:15:21.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.513+0000 7f6082400640 1 -- 172.21.15.86:0/3735562660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f607c0cfca0 msgr2=0x7f607c06f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:21.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.513+0000 7f6082400640 1 --2- 172.21.15.86:0/3735562660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f607c0cfca0 0x7f607c06f3f0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f607c0729c0 tx=0x7f606c01ebf0 comp rx=0 tx=0).stop 2024-04-25T14:15:21.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.513+0000 7f6082400640 1 -- 172.21.15.86:0/3735562660 shutdown_connections 2024-04-25T14:15:21.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.513+0000 7f6082400640 1 --2- 172.21.15.86:0/3735562660 >> v2:172.21.15.155:3300/0 conn(0x7f607c0d16e0 0x7f607c0d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:21.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.513+0000 7f6082400640 1 --2- 172.21.15.86:0/3735562660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f607c0cfca0 0x7f607c06f3f0 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:21.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.513+0000 7f6082400640 1 -- 172.21.15.86:0/3735562660 >> 172.21.15.86:0/3735562660 conn(0x7f607c0b4650 msgr2=0x7f607c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:21.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.513+0000 7f6082400640 1 -- 172.21.15.86:0/3735562660 shutdown_connections 2024-04-25T14:15:21.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.513+0000 7f6082400640 1 -- 172.21.15.86:0/3735562660 wait complete. 2024-04-25T14:15:21.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.517+0000 7f6082400640 1 Processor -- start 2024-04-25T14:15:21.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.517+0000 7f6082400640 1 -- start start 2024-04-25T14:15:21.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.517+0000 7f6082400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f607c06c670 0x7f607c15a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:21.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.517+0000 7f6082400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f607c0cfca0 0x7f607c15bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:21.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.517+0000 7f6082400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f607c0d16e0 0x7f607c15c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:21.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.517+0000 7f6082400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f607c0ce6f0 con 0x7f607c0d16e0 2024-04-25T14:15:21.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.517+0000 7f6082400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f607c0ce570 con 0x7f607c0cfca0 2024-04-25T14:15:21.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.517+0000 7f6082400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f607c0ce870 con 0x7f607c06c670 2024-04-25T14:15:21.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.517+0000 7f607be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f607c0d16e0 0x7f607c15c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:21.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.517+0000 7f607be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f607c0d16e0 0x7f607c15c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39742/0 (socket says 172.21.15.86:39742) 2024-04-25T14:15:21.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.517+0000 7f607be00640 1 -- 172.21.15.86:0/1195602526 learned_addr learned my addr 172.21.15.86:0/1195602526 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:21.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.517+0000 7f607b400640 1 --2- 172.21.15.86:0/1195602526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f607c06c670 0x7f607c15a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:21.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.517+0000 7f607be00640 1 -- 172.21.15.86:0/1195602526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f607c06c670 msgr2=0x7f607c15a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:21.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.517+0000 7f607be00640 1 --2- 172.21.15.86:0/1195602526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f607c06c670 0x7f607c15a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:21.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.517+0000 7f607be00640 1 -- 172.21.15.86:0/1195602526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f607c0cfca0 msgr2=0x7f607c15bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:21.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.517+0000 7f607be00640 1 --2- 172.21.15.86:0/1195602526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f607c0cfca0 0x7f607c15bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:21.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.517+0000 7f607be00640 1 -- 172.21.15.86:0/1195602526 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f607c09aea0 con 0x7f607c0d16e0 2024-04-25T14:15:21.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.517+0000 7f607be00640 1 --2- 172.21.15.86:0/1195602526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f607c0d16e0 0x7f607c15c310 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f607400c310 tx=0x7f607400c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:21.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.517+0000 7f6073e00640 1 -- 172.21.15.86:0/1195602526 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6074017070 con 0x7f607c0d16e0 2024-04-25T14:15:21.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.517+0000 7f6073e00640 1 -- 172.21.15.86:0/1195602526 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60740052d0 con 0x7f607c0d16e0 2024-04-25T14:15:21.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.517+0000 7f6082400640 1 -- 172.21.15.86:0/1195602526 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f607c0caad0 con 0x7f607c0d16e0 2024-04-25T14:15:21.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.517+0000 7f6073e00640 1 -- 172.21.15.86:0/1195602526 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6074003800 con 0x7f607c0d16e0 2024-04-25T14:15:21.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.517+0000 7f6082400640 1 -- 172.21.15.86:0/1195602526 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f607c1584e0 con 0x7f607c0d16e0 2024-04-25T14:15:21.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.521+0000 7f6082400640 1 -- 172.21.15.86:0/1195602526 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f607c09f4b0 con 0x7f607c0d16e0 2024-04-25T14:15:21.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.521+0000 7f6073e00640 1 -- 172.21.15.86:0/1195602526 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f607401a070 con 0x7f607c0d16e0 2024-04-25T14:15:21.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.521+0000 7f6073e00640 1 --2- 172.21.15.86:0/1195602526 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f605806f5a0 0x7f6058071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:21.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.521+0000 7f6073e00640 1 -- 172.21.15.86:0/1195602526 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f607408e890 con 0x7f607c0d16e0 2024-04-25T14:15:21.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.521+0000 7f607b400640 1 --2- 172.21.15.86:0/1195602526 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f605806f5a0 0x7f6058071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:21.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.521+0000 7f607b400640 1 --2- 172.21.15.86:0/1195602526 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f605806f5a0 0x7f6058071ab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f606c0b0000 tx=0x7f606c0a9070 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:21.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.525+0000 7f6073e00640 1 -- 172.21.15.86:0/1195602526 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f607c0d2030 con 0x7f607c0d16e0 2024-04-25T14:15:21.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.717+0000 7f6082400640 1 -- 172.21.15.86:0/1195602526 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f607c069d00 con 0x7f607c0d16e0 2024-04-25T14:15:21.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.717+0000 7f6073e00640 1 -- 172.21.15.86:0/1195602526 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f6074005480 con 0x7f607c0d16e0 2024-04-25T14:15:21.712 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:21.712 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:21.713 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:21.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.717+0000 7f6082400640 1 -- 172.21.15.86:0/1195602526 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f605806f5a0 msgr2=0x7f6058071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:21.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.717+0000 7f6082400640 1 --2- 172.21.15.86:0/1195602526 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f605806f5a0 0x7f6058071ab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f606c0b0000 tx=0x7f606c0a9070 comp rx=0 tx=0).stop 2024-04-25T14:15:21.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.717+0000 7f6082400640 1 -- 172.21.15.86:0/1195602526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f607c0d16e0 msgr2=0x7f607c15c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:21.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.717+0000 7f6082400640 1 --2- 172.21.15.86:0/1195602526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f607c0d16e0 0x7f607c15c310 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f607400c310 tx=0x7f607400c7d0 comp rx=0 tx=0).stop 2024-04-25T14:15:21.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.717+0000 7f6082400640 1 -- 172.21.15.86:0/1195602526 shutdown_connections 2024-04-25T14:15:21.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.717+0000 7f6082400640 1 --2- 172.21.15.86:0/1195602526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f607c0d16e0 0x7f607c15c310 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:21.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.717+0000 7f6082400640 1 --2- 172.21.15.86:0/1195602526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f607c0cfca0 0x7f607c15bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:21.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.717+0000 7f6082400640 1 --2- 172.21.15.86:0/1195602526 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f605806f5a0 0x7f6058071ab0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:21.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.717+0000 7f6082400640 1 --2- 172.21.15.86:0/1195602526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f607c06c670 0x7f607c15a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:21.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.717+0000 7f6082400640 1 -- 172.21.15.86:0/1195602526 >> 172.21.15.86:0/1195602526 conn(0x7f607c0b4650 msgr2=0x7f607c158e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:21.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.717+0000 7f6082400640 1 -- 172.21.15.86:0/1195602526 shutdown_connections 2024-04-25T14:15:21.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:21.717+0000 7f6082400640 1 -- 172.21.15.86:0/1195602526 wait complete. 2024-04-25T14:15:22.137 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:22.137 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:23.137 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:23.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.296+0000 7f0234400640 1 Processor -- start 2024-04-25T14:15:23.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.300+0000 7f0234400640 1 -- start start 2024-04-25T14:15:23.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.300+0000 7f0234400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f022c0cfca0 0x7f022c06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:23.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.300+0000 7f0234400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f022c0d16e0 0x7f022c0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:23.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.300+0000 7f0234400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f022c0d2030 con 0x7f022c06c670 2024-04-25T14:15:23.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.300+0000 7f0234400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f022c070470 con 0x7f022c0cfca0 2024-04-25T14:15:23.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.300+0000 7f0234400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f022c072bd0 con 0x7f022c0d16e0 2024-04-25T14:15:23.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.300+0000 7f0231000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f022c0d16e0 0x7f022c0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:23.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.300+0000 7f0231a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f022c0cfca0 0x7f022c06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:23.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.300+0000 7f0231a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f022c0cfca0 0x7f022c06f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:46946/0 (socket says 172.21.15.86:46946) 2024-04-25T14:15:23.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.300+0000 7f0231a00640 1 -- 172.21.15.86:0/1047499429 learned_addr learned my addr 172.21.15.86:0/1047499429 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:23.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.300+0000 7f0231000640 1 -- 172.21.15.86:0/1047499429 >> v2:172.21.15.155:3300/0 conn(0x7f022c0cfca0 msgr2=0x7f022c06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:23.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.300+0000 7f0231000640 1 --2- 172.21.15.86:0/1047499429 >> v2:172.21.15.155:3300/0 conn(0x7f022c0cfca0 0x7f022c06f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.300+0000 7f0231000640 1 -- 172.21.15.86:0/1047499429 >> v1:172.21.15.155:6789/0 conn(0x7f022c06c670 legacy=0x7f022c06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:23.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.300+0000 7f0231000640 1 -- 172.21.15.86:0/1047499429 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f022c06f930 con 0x7f022c0d16e0 2024-04-25T14:15:23.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.300+0000 7f0231a00640 1 --2- 172.21.15.86:0/1047499429 >> v2:172.21.15.155:3300/0 conn(0x7f022c0cfca0 0x7f022c06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:23.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.304+0000 7f0231000640 1 --2- 172.21.15.86:0/1047499429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f022c0d16e0 0x7f022c0d1af0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f02180097c0 tx=0x7f021801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=357cca8451b511f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:23.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.304+0000 7f022be00640 1 -- 172.21.15.86:0/1047499429 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02180b1070 con 0x7f022c0d16e0 2024-04-25T14:15:23.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.304+0000 7f022be00640 1 -- 172.21.15.86:0/1047499429 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02180ac410 con 0x7f022c0d16e0 2024-04-25T14:15:23.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.304+0000 7f022be00640 1 -- 172.21.15.86:0/1047499429 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02180ac7e0 con 0x7f022c0d16e0 2024-04-25T14:15:23.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.304+0000 7f0234400640 1 -- 172.21.15.86:0/1047499429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f022c0d16e0 msgr2=0x7f022c0d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:23.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.304+0000 7f0234400640 1 --2- 172.21.15.86:0/1047499429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f022c0d16e0 0x7f022c0d1af0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f02180097c0 tx=0x7f021801c4a0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.304+0000 7f0234400640 1 -- 172.21.15.86:0/1047499429 shutdown_connections 2024-04-25T14:15:23.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.304+0000 7f0234400640 1 --2- 172.21.15.86:0/1047499429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f022c0d16e0 0x7f022c0d1af0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.304+0000 7f0234400640 1 --2- 172.21.15.86:0/1047499429 >> v2:172.21.15.155:3300/0 conn(0x7f022c0cfca0 0x7f022c06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.304+0000 7f0234400640 1 -- 172.21.15.86:0/1047499429 >> 172.21.15.86:0/1047499429 conn(0x7f022c0b4650 msgr2=0x7f022c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:23.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.304+0000 7f0234400640 1 -- 172.21.15.86:0/1047499429 shutdown_connections 2024-04-25T14:15:23.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.304+0000 7f0234400640 1 -- 172.21.15.86:0/1047499429 wait complete. 2024-04-25T14:15:23.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.308+0000 7f0234400640 1 Processor -- start 2024-04-25T14:15:23.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.308+0000 7f0234400640 1 -- start start 2024-04-25T14:15:23.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.308+0000 7f0234400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f022c06c670 0x7f022c0b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:23.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.308+0000 7f0234400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f022c0cfca0 0x7f022c169710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:23.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.308+0000 7f0234400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f022c0d16e0 0x7f022c169c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:23.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.308+0000 7f0234400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f022c168060 con 0x7f022c06c670 2024-04-25T14:15:23.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.308+0000 7f0234400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f022c167ee0 con 0x7f022c0d16e0 2024-04-25T14:15:23.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.308+0000 7f0234400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f022c1681e0 con 0x7f022c0cfca0 2024-04-25T14:15:23.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.308+0000 7f0231a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f022c06c670 0x7f022c0b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:23.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.308+0000 7f0231a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f022c06c670 0x7f022c0b36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39750/0 (socket says 172.21.15.86:39750) 2024-04-25T14:15:23.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.308+0000 7f0231a00640 1 -- 172.21.15.86:0/1558620332 learned_addr learned my addr 172.21.15.86:0/1558620332 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:23.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.308+0000 7f0231000640 1 --2- 172.21.15.86:0/1558620332 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f022c0cfca0 0x7f022c169710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:23.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.308+0000 7f0232400640 1 --2- 172.21.15.86:0/1558620332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f022c0d16e0 0x7f022c169c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:23.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.308+0000 7f0231a00640 1 -- 172.21.15.86:0/1558620332 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f022c0cfca0 msgr2=0x7f022c169710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:23.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.308+0000 7f0231a00640 1 --2- 172.21.15.86:0/1558620332 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f022c0cfca0 0x7f022c169710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.308+0000 7f0231a00640 1 -- 172.21.15.86:0/1558620332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f022c0d16e0 msgr2=0x7f022c169c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:23.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.308+0000 7f0231a00640 1 --2- 172.21.15.86:0/1558620332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f022c0d16e0 0x7f022c169c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.308+0000 7f0231a00640 1 -- 172.21.15.86:0/1558620332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f022c16a520 con 0x7f022c06c670 2024-04-25T14:15:23.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.308+0000 7f0231a00640 1 --2- 172.21.15.86:0/1558620332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f022c06c670 0x7f022c0b36b0 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f022000e9b0 tx=0x7f022000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:23.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.312+0000 7f0229600640 1 -- 172.21.15.86:0/1558620332 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220017070 con 0x7f022c06c670 2024-04-25T14:15:23.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.312+0000 7f0229600640 1 -- 172.21.15.86:0/1558620332 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f022000c9a0 con 0x7f022c06c670 2024-04-25T14:15:23.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.312+0000 7f0234400640 1 -- 172.21.15.86:0/1558620332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f022c0ce8d0 con 0x7f022c06c670 2024-04-25T14:15:23.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.312+0000 7f0229600640 1 -- 172.21.15.86:0/1558620332 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022000cd90 con 0x7f022c06c670 2024-04-25T14:15:23.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.312+0000 7f0234400640 1 -- 172.21.15.86:0/1558620332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f022c0b2340 con 0x7f022c06c670 2024-04-25T14:15:23.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.312+0000 7f0234400640 1 -- 172.21.15.86:0/1558620332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f022c09f4b0 con 0x7f022c06c670 2024-04-25T14:15:23.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.312+0000 7f0229600640 1 -- 172.21.15.86:0/1558620332 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 83) ==== 93318+0+0 (secure 0 0 0) 0x7f022001a070 con 0x7f022c06c670 2024-04-25T14:15:23.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.312+0000 7f0229600640 1 --2- 172.21.15.86:0/1558620332 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f020c06f5a0 0x7f020c071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:23.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.312+0000 7f0229600640 1 -- 172.21.15.86:0/1558620332 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f022008f5f0 con 0x7f022c06c670 2024-04-25T14:15:23.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.312+0000 7f0231000640 1 --2- 172.21.15.86:0/1558620332 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f020c06f5a0 0x7f020c071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:23.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.316+0000 7f0231000640 1 --2- 172.21.15.86:0/1558620332 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f020c06f5a0 0x7f020c071ab0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f02180ae000 tx=0x7f0218008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:23.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.316+0000 7f0229600640 1 -- 172.21.15.86:0/1558620332 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f022005c0f0 con 0x7f022c06c670 2024-04-25T14:15:23.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.504+0000 7f0234400640 1 -- 172.21.15.86:0/1558620332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f022c069d00 con 0x7f022c06c670 2024-04-25T14:15:23.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.504+0000 7f0229600640 1 -- 172.21.15.86:0/1558620332 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f022005b9f0 con 0x7f022c06c670 2024-04-25T14:15:23.503 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:23.503 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:23.503 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:23.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.508+0000 7f0234400640 1 -- 172.21.15.86:0/1558620332 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f020c06f5a0 msgr2=0x7f020c071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:23.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.508+0000 7f0234400640 1 --2- 172.21.15.86:0/1558620332 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f020c06f5a0 0x7f020c071ab0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f02180ae000 tx=0x7f0218008040 comp rx=0 tx=0).stop 2024-04-25T14:15:23.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.508+0000 7f0234400640 1 -- 172.21.15.86:0/1558620332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f022c06c670 msgr2=0x7f022c0b36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:23.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.508+0000 7f0234400640 1 --2- 172.21.15.86:0/1558620332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f022c06c670 0x7f022c0b36b0 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f022000e9b0 tx=0x7f022000ee70 comp rx=0 tx=0).stop 2024-04-25T14:15:23.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.508+0000 7f0234400640 1 -- 172.21.15.86:0/1558620332 shutdown_connections 2024-04-25T14:15:23.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.508+0000 7f0234400640 1 --2- 172.21.15.86:0/1558620332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f022c0d16e0 0x7f022c169c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.508+0000 7f0234400640 1 --2- 172.21.15.86:0/1558620332 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f020c06f5a0 0x7f020c071ab0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.508+0000 7f0234400640 1 --2- 172.21.15.86:0/1558620332 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f022c0cfca0 0x7f022c169710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.508+0000 7f0234400640 1 --2- 172.21.15.86:0/1558620332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f022c06c670 0x7f022c0b36b0 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:23.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.508+0000 7f0234400640 1 -- 172.21.15.86:0/1558620332 >> 172.21.15.86:0/1558620332 conn(0x7f022c0b4650 msgr2=0x7f022c0ae130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:23.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.508+0000 7f0234400640 1 -- 172.21.15.86:0/1558620332 shutdown_connections 2024-04-25T14:15:23.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:23.508+0000 7f0234400640 1 -- 172.21.15.86:0/1558620332 wait complete. 2024-04-25T14:15:23.929 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:23.929 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:24.930 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:25.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.096+0000 7f8acb200640 1 Processor -- start 2024-04-25T14:15:25.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.096+0000 7f8acb200640 1 -- start start 2024-04-25T14:15:25.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.096+0000 7f8acb200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ac4161450 0x7f8ac4161860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:25.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.096+0000 7f8acb200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ac4072440 con 0x7f8ac4070d60 2024-04-25T14:15:25.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.096+0000 7f8acb200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ac4162140 con 0x7f8ac415da10 2024-04-25T14:15:25.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.096+0000 7f8acb200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8ac4162420 con 0x7f8ac4161450 2024-04-25T14:15:25.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.096+0000 7f8ac9e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8ac4070d60 0x7f8ac4071170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44036/0 (socket says 172.21.15.86:44036) 2024-04-25T14:15:25.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.096+0000 7f8ac9e00640 1 -- 172.21.15.86:0/587131033 learned_addr learned my addr 172.21.15.86:0/587131033 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:15:25.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.096+0000 7f8ac9400640 1 --2- 172.21.15.86:0/587131033 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ac4161450 0x7f8ac4161860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:25.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.096+0000 7f8ac8a00640 1 -- 172.21.15.86:0/587131033 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 533848034 0 0) 0x7f8ac4072440 con 0x7f8ac4070d60 2024-04-25T14:15:25.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.096+0000 7f8ac8a00640 1 -- 172.21.15.86:0/587131033 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8aac003530 con 0x7f8ac4070d60 2024-04-25T14:15:25.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.096+0000 7f8ac8a00640 1 -- 172.21.15.86:0/587131033 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2934517156 0 0) 0x7f8ac4162140 con 0x7f8ac415da10 2024-04-25T14:15:25.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.096+0000 7f8ac8a00640 1 -- 172.21.15.86:0/587131033 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ac4072440 con 0x7f8ac415da10 2024-04-25T14:15:25.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.096+0000 7f8ac8a00640 1 -- 172.21.15.86:0/587131033 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2063755036 0 0) 0x7f8aac003530 con 0x7f8ac4070d60 2024-04-25T14:15:25.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.096+0000 7f8ac8a00640 1 -- 172.21.15.86:0/587131033 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ac4161450 msgr2=0x7f8ac4161860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:25.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.096+0000 7f8ac8a00640 1 --2- 172.21.15.86:0/587131033 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ac4161450 0x7f8ac4161860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:25.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.096+0000 7f8ac8a00640 1 -- 172.21.15.86:0/587131033 >> v1:172.21.15.155:6789/0 conn(0x7f8ac415da10 legacy=0x7f8ac415de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:25.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.096+0000 7f8ac8a00640 1 -- 172.21.15.86:0/587131033 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac40726e0 con 0x7f8ac4070d60 2024-04-25T14:15:25.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.096+0000 7f8ac8a00640 1 -- 172.21.15.86:0/587131033 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f8ab8001680 con 0x7f8ac4070d60 2024-04-25T14:15:25.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.096+0000 7f8ac9400640 1 --2- 172.21.15.86:0/587131033 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ac4161450 0x7f8ac4161860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:25.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.096+0000 7f8ac8a00640 1 -- 172.21.15.86:0/587131033 >> v1:172.21.15.86:6789/0 conn(0x7f8ac4070d60 legacy=0x7f8ac4071170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:25.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.096+0000 7f8ac8a00640 1 --2- 172.21.15.86:0/587131033 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ac4161450 0x7f8aac003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:25.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.096+0000 7f8ac8a00640 1 --2- 172.21.15.86:0/587131033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aac0040e0 0x7f8aac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:25.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.096+0000 7f8ac8a00640 1 --2- 172.21.15.86:0/587131033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8aac004d50 0x7f8aac005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:25.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.100+0000 7f8ac8a00640 1 -- 172.21.15.86:0/587131033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac4162420 con 0x7f8aac0040e0 2024-04-25T14:15:25.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.100+0000 7f8ac8a00640 1 -- 172.21.15.86:0/587131033 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8aac00bbe0 con 0x7f8aac004d50 2024-04-25T14:15:25.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.100+0000 7f8ac8a00640 1 -- 172.21.15.86:0/587131033 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8aac00bde0 con 0x7f8ac4161450 2024-04-25T14:15:25.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.100+0000 7f8ac9400640 1 --2- 172.21.15.86:0/587131033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aac0040e0 0x7f8aac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:25.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.100+0000 7f8aca800640 1 --2- 172.21.15.86:0/587131033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8aac004d50 0x7f8aac005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:25.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.100+0000 7f8ac9e00640 1 --2- 172.21.15.86:0/587131033 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ac4161450 0x7f8aac003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:25.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.100+0000 7f8ac9400640 1 -- 172.21.15.86:0/587131033 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ac4161450 msgr2=0x7f8aac003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:25.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.100+0000 7f8ac9400640 1 --2- 172.21.15.86:0/587131033 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ac4161450 0x7f8aac003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:25.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.100+0000 7f8ac9400640 1 -- 172.21.15.86:0/587131033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8aac004d50 msgr2=0x7f8aac005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:25.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.100+0000 7f8ac9400640 1 --2- 172.21.15.86:0/587131033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8aac004d50 0x7f8aac005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:25.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.100+0000 7f8ac9400640 1 -- 172.21.15.86:0/587131033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aac003530 con 0x7f8aac0040e0 2024-04-25T14:15:25.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.100+0000 7f8aca800640 1 --2- 172.21.15.86:0/587131033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8aac004d50 0x7f8aac005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:25.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.100+0000 7f8ac9400640 1 --2- 172.21.15.86:0/587131033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aac0040e0 0x7f8aac004810 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009c90 tx=0x7f8ab401cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3cdaa9217beb96a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:25.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.100+0000 7f8ac8a00640 1 -- 172.21.15.86:0/587131033 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4009860 con 0x7f8aac0040e0 2024-04-25T14:15:25.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.100+0000 7f8ac8a00640 1 -- 172.21.15.86:0/587131033 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ab4004030 con 0x7f8aac0040e0 2024-04-25T14:15:25.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.100+0000 7f8ac8a00640 1 -- 172.21.15.86:0/587131033 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4004460 con 0x7f8aac0040e0 2024-04-25T14:15:25.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.104+0000 7f8acb200640 1 -- 172.21.15.86:0/587131033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aac0040e0 msgr2=0x7f8aac004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:25.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.104+0000 7f8acb200640 1 --2- 172.21.15.86:0/587131033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aac0040e0 0x7f8aac004810 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009c90 tx=0x7f8ab401cc00 comp rx=0 tx=0).stop 2024-04-25T14:15:25.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.104+0000 7f8aca800640 1 -- 172.21.15.86:0/587131033 reap_dead start 2024-04-25T14:15:25.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.104+0000 7f8acb200640 1 -- 172.21.15.86:0/587131033 shutdown_connections 2024-04-25T14:15:25.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.104+0000 7f8acb200640 1 -- 172.21.15.86:0/587131033 >> 172.21.15.86:0/587131033 conn(0x7f8ac40d3330 msgr2=0x7f8ac40d3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:25.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.104+0000 7f8acb200640 1 -- 172.21.15.86:0/587131033 shutdown_connections 2024-04-25T14:15:25.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.104+0000 7f8acb200640 1 -- 172.21.15.86:0/587131033 wait complete. 2024-04-25T14:15:25.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.104+0000 7f8acb200640 1 Processor -- start 2024-04-25T14:15:25.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.104+0000 7f8acb200640 1 -- start start 2024-04-25T14:15:25.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.108+0000 7f8acb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ac415af80 0x7f8ac41593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:25.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.108+0000 7f8acb200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ac40b69b0 0x7f8ac40b6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:25.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.108+0000 7f8acb200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ac40ca3c0 0x7f8ac40ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:25.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.108+0000 7f8acb200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac4169a00 con 0x7f8ac415af80 2024-04-25T14:15:25.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.108+0000 7f8acb200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac415ed40 con 0x7f8ac40b69b0 2024-04-25T14:15:25.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.108+0000 7f8acb200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8ac4169b80 con 0x7f8ac40ca3c0 2024-04-25T14:15:25.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.108+0000 7f8ac9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ac415af80 0x7f8ac41593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:25.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.108+0000 7f8ac9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ac415af80 0x7f8ac41593f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39770/0 (socket says 172.21.15.86:39770) 2024-04-25T14:15:25.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.108+0000 7f8ac9e00640 1 -- 172.21.15.86:0/1407585399 learned_addr learned my addr 172.21.15.86:0/1407585399 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:25.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.108+0000 7f8ac9400640 1 --2- 172.21.15.86:0/1407585399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ac40b69b0 0x7f8ac40b6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:25.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.108+0000 7f8aca800640 1 --2- 172.21.15.86:0/1407585399 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ac40ca3c0 0x7f8ac40ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:25.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.108+0000 7f8ac9e00640 1 -- 172.21.15.86:0/1407585399 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ac40ca3c0 msgr2=0x7f8ac40ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:25.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.108+0000 7f8ac9e00640 1 --2- 172.21.15.86:0/1407585399 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ac40ca3c0 0x7f8ac40ca7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:25.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.108+0000 7f8ac9e00640 1 -- 172.21.15.86:0/1407585399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ac40b69b0 msgr2=0x7f8ac40b6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:25.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.108+0000 7f8ac9e00640 1 --2- 172.21.15.86:0/1407585399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ac40b69b0 0x7f8ac40b6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:25.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.108+0000 7f8ac9e00640 1 -- 172.21.15.86:0/1407585399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac415b440 con 0x7f8ac415af80 2024-04-25T14:15:25.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.108+0000 7f8ac9e00640 1 --2- 172.21.15.86:0/1407585399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ac415af80 0x7f8ac41593f0 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8004ee0 tx=0x7f8ab8007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:25.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.108+0000 7f8abe000640 1 -- 172.21.15.86:0/1407585399 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab800f780 con 0x7f8ac415af80 2024-04-25T14:15:25.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.108+0000 7f8abe000640 1 -- 172.21.15.86:0/1407585399 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ab8018070 con 0x7f8ac415af80 2024-04-25T14:15:25.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.108+0000 7f8abe000640 1 -- 172.21.15.86:0/1407585399 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab800fb80 con 0x7f8ac415af80 2024-04-25T14:15:25.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.108+0000 7f8acb200640 1 -- 172.21.15.86:0/1407585399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ac415b720 con 0x7f8ac415af80 2024-04-25T14:15:25.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.108+0000 7f8acb200640 1 -- 172.21.15.86:0/1407585399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ac415bce0 con 0x7f8ac415af80 2024-04-25T14:15:25.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.108+0000 7f8acb200640 1 -- 172.21.15.86:0/1407585399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ac4076b80 con 0x7f8ac415af80 2024-04-25T14:15:25.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.108+0000 7f8abe000640 1 -- 172.21.15.86:0/1407585399 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f8ab800fd20 con 0x7f8ac415af80 2024-04-25T14:15:25.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.112+0000 7f8abe000640 1 --2- 172.21.15.86:0/1407585399 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8aa806f480 0x7f8aa8071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:25.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.112+0000 7f8abe000640 1 -- 172.21.15.86:0/1407585399 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f8ab808f480 con 0x7f8ac415af80 2024-04-25T14:15:25.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.112+0000 7f8ac9400640 1 --2- 172.21.15.86:0/1407585399 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8aa806f480 0x7f8aa8071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:25.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.112+0000 7f8ac9400640 1 --2- 172.21.15.86:0/1407585399 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8aa806f480 0x7f8aa8071990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40ae000 tx=0x7f8ab40046d0 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:25.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.116+0000 7f8abe000640 1 -- 172.21.15.86:0/1407585399 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8ab805c070 con 0x7f8ac415af80 2024-04-25T14:15:25.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.312+0000 7f8acb200640 1 -- 172.21.15.86:0/1407585399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ac40c83a0 con 0x7f8ac415af80 2024-04-25T14:15:25.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.312+0000 7f8abe000640 1 -- 172.21.15.86:0/1407585399 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f8ab805b970 con 0x7f8ac415af80 2024-04-25T14:15:25.310 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:25.310 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:25.311 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:25.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.316+0000 7f8acb200640 1 -- 172.21.15.86:0/1407585399 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8aa806f480 msgr2=0x7f8aa8071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:25.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.316+0000 7f8acb200640 1 --2- 172.21.15.86:0/1407585399 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8aa806f480 0x7f8aa8071990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40ae000 tx=0x7f8ab40046d0 comp rx=0 tx=0).stop 2024-04-25T14:15:25.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.316+0000 7f8acb200640 1 -- 172.21.15.86:0/1407585399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ac415af80 msgr2=0x7f8ac41593f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:25.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.316+0000 7f8acb200640 1 --2- 172.21.15.86:0/1407585399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ac415af80 0x7f8ac41593f0 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8004ee0 tx=0x7f8ab8007430 comp rx=0 tx=0).stop 2024-04-25T14:15:25.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.316+0000 7f8acb200640 1 -- 172.21.15.86:0/1407585399 shutdown_connections 2024-04-25T14:15:25.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.316+0000 7f8acb200640 1 --2- 172.21.15.86:0/1407585399 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8aa806f480 0x7f8aa8071990 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:25.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.316+0000 7f8acb200640 1 --2- 172.21.15.86:0/1407585399 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ac40ca3c0 0x7f8ac40ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:25.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.316+0000 7f8acb200640 1 --2- 172.21.15.86:0/1407585399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ac40b69b0 0x7f8ac40b6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:25.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.316+0000 7f8acb200640 1 --2- 172.21.15.86:0/1407585399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ac415af80 0x7f8ac41593f0 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:25.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.316+0000 7f8acb200640 1 -- 172.21.15.86:0/1407585399 >> 172.21.15.86:0/1407585399 conn(0x7f8ac40d3330 msgr2=0x7f8ac40d3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:25.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.316+0000 7f8acb200640 1 -- 172.21.15.86:0/1407585399 shutdown_connections 2024-04-25T14:15:25.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:25.316+0000 7f8acb200640 1 -- 172.21.15.86:0/1407585399 wait complete. 2024-04-25T14:15:25.735 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:25.735 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:26.736 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:26.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.900+0000 7f59a4a00640 1 Processor -- start 2024-04-25T14:15:26.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.900+0000 7f59a4a00640 1 -- start start 2024-04-25T14:15:26.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.900+0000 7f59a4a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f59a00b77b0 0x7f59a015c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:26.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.900+0000 7f59a4a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59a00b3b00 con 0x7f59a00b36f0 2024-04-25T14:15:26.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.900+0000 7f59a4a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59a00b1a90 con 0x7f59a006de90 2024-04-25T14:15:26.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.900+0000 7f59a4a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f59a00d1c40 con 0x7f59a00b77b0 2024-04-25T14:15:26.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.900+0000 7f599e800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f59a00b36f0 0x7f59a006d780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44046/0 (socket says 172.21.15.86:44046) 2024-04-25T14:15:26.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.900+0000 7f599e800640 1 -- 172.21.15.86:0/3940942656 learned_addr learned my addr 172.21.15.86:0/3940942656 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:15:26.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.900+0000 7f599f200640 1 --2- 172.21.15.86:0/3940942656 >> v2:172.21.15.155:3300/0 conn(0x7f59a00b77b0 0x7f59a015c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:26.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.900+0000 7f599f200640 1 -- 172.21.15.86:0/3940942656 >> v1:172.21.15.155:6789/0 conn(0x7f59a006de90 legacy=0x7f59a015bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:26.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.900+0000 7f599f200640 1 -- 172.21.15.86:0/3940942656 >> v1:172.21.15.86:6789/0 conn(0x7f59a00b36f0 legacy=0x7f59a006d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:26.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.900+0000 7f599f200640 1 -- 172.21.15.86:0/3940942656 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a00b2250 con 0x7f59a00b77b0 2024-04-25T14:15:26.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.904+0000 7f599f200640 1 --2- 172.21.15.86:0/3940942656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59a00b77b0 0x7f59a015c5d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5998009d90 tx=0x7f599801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fd717c6e5596f7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:26.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.904+0000 7f599d400640 1 -- 172.21.15.86:0/3940942656 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59980b2070 con 0x7f59a00b77b0 2024-04-25T14:15:26.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.904+0000 7f599d400640 1 -- 172.21.15.86:0/3940942656 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59980a5920 con 0x7f59a00b77b0 2024-04-25T14:15:26.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.904+0000 7f599d400640 1 -- 172.21.15.86:0/3940942656 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59980a5cf0 con 0x7f59a00b77b0 2024-04-25T14:15:26.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.904+0000 7f59a4a00640 1 -- 172.21.15.86:0/3940942656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59a00b77b0 msgr2=0x7f59a015c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:26.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.908+0000 7f59a4a00640 1 --2- 172.21.15.86:0/3940942656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59a00b77b0 0x7f59a015c5d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5998009d90 tx=0x7f599801c970 comp rx=0 tx=0).stop 2024-04-25T14:15:26.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.908+0000 7f59a4a00640 1 -- 172.21.15.86:0/3940942656 shutdown_connections 2024-04-25T14:15:26.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.908+0000 7f59a4a00640 1 --2- 172.21.15.86:0/3940942656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59a00b77b0 0x7f59a015c5d0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:26.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.908+0000 7f59a4a00640 1 -- 172.21.15.86:0/3940942656 >> 172.21.15.86:0/3940942656 conn(0x7f59a009ba30 msgr2=0x7f59a00b70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:26.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.908+0000 7f59a4a00640 1 -- 172.21.15.86:0/3940942656 shutdown_connections 2024-04-25T14:15:26.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.908+0000 7f59a4a00640 1 -- 172.21.15.86:0/3940942656 wait complete. 2024-04-25T14:15:26.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.908+0000 7f59a4a00640 1 Processor -- start 2024-04-25T14:15:26.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.908+0000 7f59a4a00640 1 -- start start 2024-04-25T14:15:26.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.908+0000 7f59a4a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59a00b36f0 0x7f59a0070a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:26.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.908+0000 7f59a4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59a00b77b0 0x7f59a015c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:26.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.912+0000 7f59a4a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59a0151a60 0x7f59a009cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:26.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.912+0000 7f59a4a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f59a00d0e50 con 0x7f59a00b77b0 2024-04-25T14:15:26.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.912+0000 7f59a4a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f59a00b7bc0 con 0x7f59a00b36f0 2024-04-25T14:15:26.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.912+0000 7f59a4a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f59a00d0fd0 con 0x7f59a0151a60 2024-04-25T14:15:26.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.912+0000 7f599de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59a00b77b0 0x7f59a015c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:26.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.912+0000 7f599de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59a00b77b0 0x7f59a015c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39782/0 (socket says 172.21.15.86:39782) 2024-04-25T14:15:26.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.912+0000 7f599de00640 1 -- 172.21.15.86:0/2779246328 learned_addr learned my addr 172.21.15.86:0/2779246328 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:26.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.912+0000 7f599e800640 1 --2- 172.21.15.86:0/2779246328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59a00b36f0 0x7f59a0070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:26.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.912+0000 7f599f200640 1 --2- 172.21.15.86:0/2779246328 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59a0151a60 0x7f59a009cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:26.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.912+0000 7f599de00640 1 -- 172.21.15.86:0/2779246328 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59a0151a60 msgr2=0x7f59a009cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:26.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.912+0000 7f599de00640 1 --2- 172.21.15.86:0/2779246328 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59a0151a60 0x7f59a009cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:26.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.912+0000 7f599de00640 1 -- 172.21.15.86:0/2779246328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59a00b36f0 msgr2=0x7f59a0070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:26.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.912+0000 7f599de00640 1 --2- 172.21.15.86:0/2779246328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59a00b36f0 0x7f59a0070a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:26.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.912+0000 7f599de00640 1 -- 172.21.15.86:0/2779246328 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a015c850 con 0x7f59a00b77b0 2024-04-25T14:15:26.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.912+0000 7f599f200640 1 --2- 172.21.15.86:0/2779246328 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59a0151a60 0x7f59a009cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:15:26.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.912+0000 7f599de00640 1 --2- 172.21.15.86:0/2779246328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59a00b77b0 0x7f59a015c310 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f598c00c370 tx=0x7f598c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:26.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.912+0000 7f5996a00640 1 -- 172.21.15.86:0/2779246328 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c0052d0 con 0x7f59a00b77b0 2024-04-25T14:15:26.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.912+0000 7f5996a00640 1 -- 172.21.15.86:0/2779246328 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f598c005470 con 0x7f59a00b77b0 2024-04-25T14:15:26.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.912+0000 7f59a4a00640 1 -- 172.21.15.86:0/2779246328 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a009ed00 con 0x7f59a00b77b0 2024-04-25T14:15:26.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.912+0000 7f5996a00640 1 -- 172.21.15.86:0/2779246328 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c007260 con 0x7f59a00b77b0 2024-04-25T14:15:26.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.912+0000 7f59a4a00640 1 -- 172.21.15.86:0/2779246328 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a009c5a0 con 0x7f59a00b77b0 2024-04-25T14:15:26.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.912+0000 7f59a4a00640 1 -- 172.21.15.86:0/2779246328 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59a009c0b0 con 0x7f59a00b77b0 2024-04-25T14:15:26.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.912+0000 7f5996a00640 1 -- 172.21.15.86:0/2779246328 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f598c007400 con 0x7f59a00b77b0 2024-04-25T14:15:26.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.916+0000 7f5996a00640 1 --2- 172.21.15.86:0/2779246328 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f598006f480 0x7f5980071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:26.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.916+0000 7f5996a00640 1 -- 172.21.15.86:0/2779246328 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f598c08eb10 con 0x7f59a00b77b0 2024-04-25T14:15:26.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.916+0000 7f599e800640 1 --2- 172.21.15.86:0/2779246328 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f598006f480 0x7f5980071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:26.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.916+0000 7f599e800640 1 --2- 172.21.15.86:0/2779246328 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f598006f480 0x7f5980071990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5990005890 tx=0x7f5990009040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:26.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:26.920+0000 7f5996a00640 1 -- 172.21.15.86:0/2779246328 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f598c05b680 con 0x7f59a00b77b0 2024-04-25T14:15:27.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:27.108+0000 7f59a4a00640 1 -- 172.21.15.86:0/2779246328 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59a0076b80 con 0x7f59a00b77b0 2024-04-25T14:15:27.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:27.108+0000 7f5996a00640 1 -- 172.21.15.86:0/2779246328 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f598c05af80 con 0x7f59a00b77b0 2024-04-25T14:15:27.105 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:27.105 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:27.106 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:27.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:27.108+0000 7f59a4a00640 1 -- 172.21.15.86:0/2779246328 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f598006f480 msgr2=0x7f5980071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:27.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:27.108+0000 7f59a4a00640 1 --2- 172.21.15.86:0/2779246328 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f598006f480 0x7f5980071990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5990005890 tx=0x7f5990009040 comp rx=0 tx=0).stop 2024-04-25T14:15:27.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:27.108+0000 7f59a4a00640 1 -- 172.21.15.86:0/2779246328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59a00b77b0 msgr2=0x7f59a015c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:27.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:27.108+0000 7f59a4a00640 1 --2- 172.21.15.86:0/2779246328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59a00b77b0 0x7f59a015c310 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f598c00c370 tx=0x7f598c00c830 comp rx=0 tx=0).stop 2024-04-25T14:15:27.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:27.108+0000 7f59a4a00640 1 -- 172.21.15.86:0/2779246328 shutdown_connections 2024-04-25T14:15:27.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:27.108+0000 7f59a4a00640 1 --2- 172.21.15.86:0/2779246328 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f598006f480 0x7f5980071990 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:27.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:27.108+0000 7f59a4a00640 1 --2- 172.21.15.86:0/2779246328 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59a0151a60 0x7f59a009cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:27.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:27.108+0000 7f59a4a00640 1 --2- 172.21.15.86:0/2779246328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59a00b77b0 0x7f59a015c310 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:27.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:27.108+0000 7f59a4a00640 1 --2- 172.21.15.86:0/2779246328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59a00b36f0 0x7f59a0070a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:27.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:27.108+0000 7f59a4a00640 1 -- 172.21.15.86:0/2779246328 >> 172.21.15.86:0/2779246328 conn(0x7f59a009ba30 msgr2=0x7f59a00b0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:27.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:27.108+0000 7f59a4a00640 1 -- 172.21.15.86:0/2779246328 shutdown_connections 2024-04-25T14:15:27.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:27.108+0000 7f59a4a00640 1 -- 172.21.15.86:0/2779246328 wait complete. 2024-04-25T14:15:27.530 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:27.530 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:28.531 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:28.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.692+0000 7f81bc600640 1 Processor -- start 2024-04-25T14:15:28.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.692+0000 7f81bc600640 1 -- start start 2024-04-25T14:15:28.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.692+0000 7f81bc600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81b40d16e0 0x7f81b40d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:28.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.696+0000 7f81bc600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81b40d2030 con 0x7f81b40730b0 2024-04-25T14:15:28.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.696+0000 7f81bc600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81b4072bd0 con 0x7f81b40cfca0 2024-04-25T14:15:28.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.696+0000 7f81bc600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f81b4070470 con 0x7f81b40d16e0 2024-04-25T14:15:28.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.696+0000 7f81ba600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f81b40730b0 0x7f81b40734c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44062/0 (socket says 172.21.15.86:44062) 2024-04-25T14:15:28.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.696+0000 7f81ba600640 1 -- 172.21.15.86:0/2503542417 learned_addr learned my addr 172.21.15.86:0/2503542417 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:15:28.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.696+0000 7f81b9200640 1 --2- 172.21.15.86:0/2503542417 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81b40d16e0 0x7f81b40d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:28.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.696+0000 7f81b3e00640 1 -- 172.21.15.86:0/2503542417 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 560467426 0 0) 0x7f81ac0014a0 con 0x7f81b40730b0 2024-04-25T14:15:28.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.696+0000 7f81b3e00640 1 -- 172.21.15.86:0/2503542417 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f819c003530 con 0x7f81b40730b0 2024-04-25T14:15:28.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.696+0000 7f81b3e00640 1 -- 172.21.15.86:0/2503542417 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2497244744 0 0) 0x7f81b4072bd0 con 0x7f81b40cfca0 2024-04-25T14:15:28.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.696+0000 7f81b3e00640 1 -- 172.21.15.86:0/2503542417 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f81ac0014a0 con 0x7f81b40cfca0 2024-04-25T14:15:28.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.696+0000 7f81b9200640 1 -- 172.21.15.86:0/2503542417 >> v1:172.21.15.155:6789/0 conn(0x7f81b40cfca0 legacy=0x7f81b406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:28.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.696+0000 7f81b9200640 1 -- 172.21.15.86:0/2503542417 >> v1:172.21.15.86:6789/0 conn(0x7f81b40730b0 legacy=0x7f81b40734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:28.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.696+0000 7f81b3e00640 1 -- 172.21.15.86:0/2503542417 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4076627176 0 0) 0x7f819c003530 con 0x7f81b40730b0 2024-04-25T14:15:28.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.696+0000 7f81b9200640 1 -- 172.21.15.86:0/2503542417 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b406fbd0 con 0x7f81b40d16e0 2024-04-25T14:15:28.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.700+0000 7f81b9200640 1 --2- 172.21.15.86:0/2503542417 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81b40d16e0 0x7f81b40d1af0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f81b4072da0 tx=0x7f81a001c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bf08c032b2fa86b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:28.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.700+0000 7f81b3e00640 1 -- 172.21.15.86:0/2503542417 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a00b0070 con 0x7f81b40d16e0 2024-04-25T14:15:28.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.700+0000 7f81b3e00640 1 -- 172.21.15.86:0/2503542417 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81a00ab400 con 0x7f81b40d16e0 2024-04-25T14:15:28.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.700+0000 7f81b3e00640 1 -- 172.21.15.86:0/2503542417 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b40743e0 con 0x7f81b40d16e0 2024-04-25T14:15:28.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.700+0000 7f81bc600640 1 -- 172.21.15.86:0/2503542417 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81b40d16e0 msgr2=0x7f81b40d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:28.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.700+0000 7f81bc600640 1 --2- 172.21.15.86:0/2503542417 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81b40d16e0 0x7f81b40d1af0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f81b4072da0 tx=0x7f81a001c3e0 comp rx=0 tx=0).stop 2024-04-25T14:15:28.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.700+0000 7f81bc600640 1 -- 172.21.15.86:0/2503542417 shutdown_connections 2024-04-25T14:15:28.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.700+0000 7f81bc600640 1 --2- 172.21.15.86:0/2503542417 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81b40d16e0 0x7f81b40d1af0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:28.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.700+0000 7f81bc600640 1 -- 172.21.15.86:0/2503542417 >> 172.21.15.86:0/2503542417 conn(0x7f81b40b4650 msgr2=0x7f81b40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:28.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.700+0000 7f81bc600640 1 -- 172.21.15.86:0/2503542417 shutdown_connections 2024-04-25T14:15:28.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.700+0000 7f81bc600640 1 -- 172.21.15.86:0/2503542417 wait complete. 2024-04-25T14:15:28.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.700+0000 7f81bc600640 1 Processor -- start 2024-04-25T14:15:28.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.700+0000 7f81bc600640 1 -- start start 2024-04-25T14:15:28.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.704+0000 7f81bc600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81b40730b0 0x7f81b409eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:28.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.704+0000 7f81bc600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81b40cfca0 0x7f81b4157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:28.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.704+0000 7f81bc600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81b40d16e0 0x7f81b415d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:28.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.704+0000 7f81bc600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f81b40ce1b0 con 0x7f81b40730b0 2024-04-25T14:15:28.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.704+0000 7f81bc600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f81b40ce030 con 0x7f81b40cfca0 2024-04-25T14:15:28.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.704+0000 7f81bc600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f81b40ce330 con 0x7f81b40d16e0 2024-04-25T14:15:28.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.704+0000 7f81b9c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81b40730b0 0x7f81b409eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:28.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.704+0000 7f81b9c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81b40730b0 0x7f81b409eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39792/0 (socket says 172.21.15.86:39792) 2024-04-25T14:15:28.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.704+0000 7f81b9c00640 1 -- 172.21.15.86:0/3545438053 learned_addr learned my addr 172.21.15.86:0/3545438053 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:28.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.704+0000 7f81b9200640 1 --2- 172.21.15.86:0/3545438053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81b40cfca0 0x7f81b4157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:28.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.704+0000 7f81ba600640 1 --2- 172.21.15.86:0/3545438053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81b40d16e0 0x7f81b415d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:28.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.704+0000 7f81b9c00640 1 -- 172.21.15.86:0/3545438053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81b40d16e0 msgr2=0x7f81b415d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:28.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.704+0000 7f81b9c00640 1 --2- 172.21.15.86:0/3545438053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81b40d16e0 0x7f81b415d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:28.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.704+0000 7f81b9c00640 1 -- 172.21.15.86:0/3545438053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81b40cfca0 msgr2=0x7f81b4157e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:28.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.704+0000 7f81b9c00640 1 --2- 172.21.15.86:0/3545438053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81b40cfca0 0x7f81b4157e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:28.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.704+0000 7f81b9c00640 1 -- 172.21.15.86:0/3545438053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b41586d0 con 0x7f81b40730b0 2024-04-25T14:15:28.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.704+0000 7f81b9200640 1 --2- 172.21.15.86:0/3545438053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81b40cfca0 0x7f81b4157e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:15:28.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.704+0000 7f81ba600640 1 --2- 172.21.15.86:0/3545438053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81b40d16e0 0x7f81b415d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:15:28.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.704+0000 7f81b9c00640 1 --2- 172.21.15.86:0/3545438053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81b40730b0 0x7f81b409eec0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f81a800c430 tx=0x7f81a800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:28.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.704+0000 7f81b1600640 1 -- 172.21.15.86:0/3545438053 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8017070 con 0x7f81b40730b0 2024-04-25T14:15:28.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.704+0000 7f81b1600640 1 -- 172.21.15.86:0/3545438053 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81a80052f0 con 0x7f81b40730b0 2024-04-25T14:15:28.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.704+0000 7f81bc600640 1 -- 172.21.15.86:0/3545438053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b409e700 con 0x7f81b40730b0 2024-04-25T14:15:28.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.704+0000 7f81b1600640 1 -- 172.21.15.86:0/3545438053 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8003840 con 0x7f81b40730b0 2024-04-25T14:15:28.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.704+0000 7f81bc600640 1 -- 172.21.15.86:0/3545438053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b415dbc0 con 0x7f81b40730b0 2024-04-25T14:15:28.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.704+0000 7f81bc600640 1 -- 172.21.15.86:0/3545438053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81b409f4b0 con 0x7f81b40730b0 2024-04-25T14:15:28.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.708+0000 7f81b1600640 1 -- 172.21.15.86:0/3545438053 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f81a801a070 con 0x7f81b40730b0 2024-04-25T14:15:28.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.708+0000 7f81b1600640 1 --2- 172.21.15.86:0/3545438053 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f819406f480 0x7f8194071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:28.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.708+0000 7f81b1600640 1 -- 172.21.15.86:0/3545438053 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f81a808e9d0 con 0x7f81b40730b0 2024-04-25T14:15:28.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.708+0000 7f81b9200640 1 --2- 172.21.15.86:0/3545438053 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f819406f480 0x7f8194071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:28.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.708+0000 7f81b9200640 1 --2- 172.21.15.86:0/3545438053 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f819406f480 0x7f8194071990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f81a00ad000 tx=0x7f81a0008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:28.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.712+0000 7f81b1600640 1 -- 172.21.15.86:0/3545438053 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f81a805b560 con 0x7f81b40730b0 2024-04-25T14:15:28.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.904+0000 7f81bc600640 1 -- 172.21.15.86:0/3545438053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81b4069d00 con 0x7f81b40730b0 2024-04-25T14:15:28.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.904+0000 7f81b1600640 1 -- 172.21.15.86:0/3545438053 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f81a805ae60 con 0x7f81b40730b0 2024-04-25T14:15:28.901 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:28.901 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:28.901 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:28.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.904+0000 7f81bc600640 1 -- 172.21.15.86:0/3545438053 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f819406f480 msgr2=0x7f8194071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:28.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.904+0000 7f81bc600640 1 --2- 172.21.15.86:0/3545438053 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f819406f480 0x7f8194071990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f81a00ad000 tx=0x7f81a0008040 comp rx=0 tx=0).stop 2024-04-25T14:15:28.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.904+0000 7f81bc600640 1 -- 172.21.15.86:0/3545438053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81b40730b0 msgr2=0x7f81b409eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:28.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.904+0000 7f81bc600640 1 --2- 172.21.15.86:0/3545438053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81b40730b0 0x7f81b409eec0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f81a800c430 tx=0x7f81a800c8f0 comp rx=0 tx=0).stop 2024-04-25T14:15:28.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.904+0000 7f81bc600640 1 -- 172.21.15.86:0/3545438053 shutdown_connections 2024-04-25T14:15:28.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.904+0000 7f81bc600640 1 --2- 172.21.15.86:0/3545438053 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f819406f480 0x7f8194071990 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:28.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.904+0000 7f81bc600640 1 --2- 172.21.15.86:0/3545438053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81b40d16e0 0x7f81b415d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:28.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.904+0000 7f81bc600640 1 --2- 172.21.15.86:0/3545438053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81b40cfca0 0x7f81b4157e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:28.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.904+0000 7f81bc600640 1 --2- 172.21.15.86:0/3545438053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81b40730b0 0x7f81b409eec0 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:28.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.904+0000 7f81bc600640 1 -- 172.21.15.86:0/3545438053 >> 172.21.15.86:0/3545438053 conn(0x7f81b40b4650 msgr2=0x7f81b40b0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:28.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.904+0000 7f81bc600640 1 -- 172.21.15.86:0/3545438053 shutdown_connections 2024-04-25T14:15:28.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:28.904+0000 7f81bc600640 1 -- 172.21.15.86:0/3545438053 wait complete. 2024-04-25T14:15:29.327 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:29.327 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:30.328 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:30.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.488+0000 7fba31e00640 1 Processor -- start 2024-04-25T14:15:30.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.488+0000 7fba31e00640 1 -- start start 2024-04-25T14:15:30.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.492+0000 7fba31e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fba2c0cfca0 0x7fba2c06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:30.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.492+0000 7fba31e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba2c0d16e0 0x7fba2c0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:30.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.492+0000 7fba31e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba2c0d2030 con 0x7fba2c06c670 2024-04-25T14:15:30.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.492+0000 7fba31e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fba2c070470 con 0x7fba2c0cfca0 2024-04-25T14:15:30.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.492+0000 7fba31e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fba2c072bd0 con 0x7fba2c0d16e0 2024-04-25T14:15:30.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.492+0000 7fba23e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba2c0d16e0 0x7fba2c0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:30.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.492+0000 7fba2aa00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fba2c0cfca0 0x7fba2c06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:30.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.492+0000 7fba2b400640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fba2c06c670 0x7fba2c06ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:50042/0 (socket says 172.21.15.86:50042) 2024-04-25T14:15:30.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.492+0000 7fba2b400640 1 -- 172.21.15.86:0/1028281703 learned_addr learned my addr 172.21.15.86:0/1028281703 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:15:30.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.492+0000 7fba23e00640 1 -- 172.21.15.86:0/1028281703 >> v2:172.21.15.155:3300/0 conn(0x7fba2c0cfca0 msgr2=0x7fba2c06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:30.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.492+0000 7fba23e00640 1 --2- 172.21.15.86:0/1028281703 >> v2:172.21.15.155:3300/0 conn(0x7fba2c0cfca0 0x7fba2c06f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:30.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.492+0000 7fba23e00640 1 -- 172.21.15.86:0/1028281703 >> v1:172.21.15.155:6789/0 conn(0x7fba2c06c670 legacy=0x7fba2c06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:30.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.492+0000 7fba23e00640 1 -- 172.21.15.86:0/1028281703 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2c06f930 con 0x7fba2c0d16e0 2024-04-25T14:15:30.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.492+0000 7fba2aa00640 1 --2- 172.21.15.86:0/1028281703 >> v2:172.21.15.155:3300/0 conn(0x7fba2c0cfca0 0x7fba2c06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:30.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.496+0000 7fba23e00640 1 --2- 172.21.15.86:0/1028281703 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba2c0d16e0 0x7fba2c0d1af0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fba140097c0 tx=0x7fba1401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c20b72a772fdf0cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:30.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.496+0000 7fba2a000640 1 -- 172.21.15.86:0/1028281703 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba140b1070 con 0x7fba2c0d16e0 2024-04-25T14:15:30.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.496+0000 7fba2a000640 1 -- 172.21.15.86:0/1028281703 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba140ac410 con 0x7fba2c0d16e0 2024-04-25T14:15:30.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.496+0000 7fba2a000640 1 -- 172.21.15.86:0/1028281703 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba140ac7e0 con 0x7fba2c0d16e0 2024-04-25T14:15:30.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.496+0000 7fba31e00640 1 -- 172.21.15.86:0/1028281703 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba2c0d16e0 msgr2=0x7fba2c0d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:30.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.496+0000 7fba31e00640 1 --2- 172.21.15.86:0/1028281703 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba2c0d16e0 0x7fba2c0d1af0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fba140097c0 tx=0x7fba1401c4a0 comp rx=0 tx=0).stop 2024-04-25T14:15:30.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.496+0000 7fba31e00640 1 -- 172.21.15.86:0/1028281703 shutdown_connections 2024-04-25T14:15:30.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.496+0000 7fba31e00640 1 --2- 172.21.15.86:0/1028281703 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba2c0d16e0 0x7fba2c0d1af0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:30.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.496+0000 7fba31e00640 1 --2- 172.21.15.86:0/1028281703 >> v2:172.21.15.155:3300/0 conn(0x7fba2c0cfca0 0x7fba2c06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:30.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.496+0000 7fba31e00640 1 -- 172.21.15.86:0/1028281703 >> 172.21.15.86:0/1028281703 conn(0x7fba2c0b4650 msgr2=0x7fba2c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:30.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.496+0000 7fba31e00640 1 -- 172.21.15.86:0/1028281703 shutdown_connections 2024-04-25T14:15:30.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.496+0000 7fba31e00640 1 -- 172.21.15.86:0/1028281703 wait complete. 2024-04-25T14:15:30.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.500+0000 7fba31e00640 1 Processor -- start 2024-04-25T14:15:30.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.500+0000 7fba31e00640 1 -- start start 2024-04-25T14:15:30.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.500+0000 7fba31e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba2c06c670 0x7fba2c09a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:30.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.500+0000 7fba31e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba2c0cfca0 0x7fba2c15f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:30.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.500+0000 7fba31e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba2c0d16e0 0x7fba2c15fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:30.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.500+0000 7fba31e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c0cdee0 con 0x7fba2c06c670 2024-04-25T14:15:30.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.500+0000 7fba31e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c0cdd60 con 0x7fba2c0cfca0 2024-04-25T14:15:30.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.500+0000 7fba31e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fba2c0ce060 con 0x7fba2c0d16e0 2024-04-25T14:15:30.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.500+0000 7fba2aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba2c06c670 0x7fba2c09a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:30.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.500+0000 7fba2aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba2c06c670 0x7fba2c09a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47008/0 (socket says 172.21.15.86:47008) 2024-04-25T14:15:30.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.500+0000 7fba2aa00640 1 -- 172.21.15.86:0/4180118250 learned_addr learned my addr 172.21.15.86:0/4180118250 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:30.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.500+0000 7fba23e00640 1 --2- 172.21.15.86:0/4180118250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba2c0cfca0 0x7fba2c15f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:30.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.500+0000 7fba2b400640 1 --2- 172.21.15.86:0/4180118250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba2c0d16e0 0x7fba2c15fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:30.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.500+0000 7fba2aa00640 1 -- 172.21.15.86:0/4180118250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba2c0d16e0 msgr2=0x7fba2c15fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:30.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.500+0000 7fba2aa00640 1 --2- 172.21.15.86:0/4180118250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba2c0d16e0 0x7fba2c15fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:30.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.500+0000 7fba2aa00640 1 -- 172.21.15.86:0/4180118250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba2c0cfca0 msgr2=0x7fba2c15f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:30.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.500+0000 7fba2aa00640 1 --2- 172.21.15.86:0/4180118250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba2c0cfca0 0x7fba2c15f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:30.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.500+0000 7fba2aa00640 1 -- 172.21.15.86:0/4180118250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2c15de80 con 0x7fba2c06c670 2024-04-25T14:15:30.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.500+0000 7fba2aa00640 1 --2- 172.21.15.86:0/4180118250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba2c06c670 0x7fba2c09a0b0 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00e960 tx=0x7fba1c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:30.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.500+0000 7fba23400640 1 -- 172.21.15.86:0/4180118250 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c017070 con 0x7fba2c06c670 2024-04-25T14:15:30.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.500+0000 7fba23400640 1 -- 172.21.15.86:0/4180118250 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba1c00ca00 con 0x7fba2c06c670 2024-04-25T14:15:30.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.500+0000 7fba31e00640 1 -- 172.21.15.86:0/4180118250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba2c0b0e40 con 0x7fba2c06c670 2024-04-25T14:15:30.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.500+0000 7fba23400640 1 -- 172.21.15.86:0/4180118250 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c00cdd0 con 0x7fba2c06c670 2024-04-25T14:15:30.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.500+0000 7fba31e00640 1 -- 172.21.15.86:0/4180118250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba2c0b1400 con 0x7fba2c06c670 2024-04-25T14:15:30.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.504+0000 7fba31e00640 1 -- 172.21.15.86:0/4180118250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba2c09f4b0 con 0x7fba2c06c670 2024-04-25T14:15:30.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.504+0000 7fba23400640 1 -- 172.21.15.86:0/4180118250 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7fba1c01a070 con 0x7fba2c06c670 2024-04-25T14:15:30.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.504+0000 7fba23400640 1 --2- 172.21.15.86:0/4180118250 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fba0806f450 0x7fba08071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:30.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.504+0000 7fba23400640 1 -- 172.21.15.86:0/4180118250 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fba1c08f4a0 con 0x7fba2c06c670 2024-04-25T14:15:30.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.504+0000 7fba23e00640 1 --2- 172.21.15.86:0/4180118250 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fba0806f450 0x7fba08071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:30.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.504+0000 7fba23e00640 1 --2- 172.21.15.86:0/4180118250 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fba0806f450 0x7fba08071960 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fba140ae000 tx=0x7fba14008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:30.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.508+0000 7fba23400640 1 -- 172.21.15.86:0/4180118250 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fba1c05c0a0 con 0x7fba2c06c670 2024-04-25T14:15:30.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.700+0000 7fba31e00640 1 -- 172.21.15.86:0/4180118250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba2c069d00 con 0x7fba2c06c670 2024-04-25T14:15:30.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.700+0000 7fba23400640 1 -- 172.21.15.86:0/4180118250 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7fba1c05b9a0 con 0x7fba2c06c670 2024-04-25T14:15:30.697 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:30.697 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:30.697 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:30.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.700+0000 7fba31e00640 1 -- 172.21.15.86:0/4180118250 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fba0806f450 msgr2=0x7fba08071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:30.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.700+0000 7fba31e00640 1 --2- 172.21.15.86:0/4180118250 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fba0806f450 0x7fba08071960 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fba140ae000 tx=0x7fba14008040 comp rx=0 tx=0).stop 2024-04-25T14:15:30.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.700+0000 7fba31e00640 1 -- 172.21.15.86:0/4180118250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba2c06c670 msgr2=0x7fba2c09a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:30.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.700+0000 7fba31e00640 1 --2- 172.21.15.86:0/4180118250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba2c06c670 0x7fba2c09a0b0 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00e960 tx=0x7fba1c00ee20 comp rx=0 tx=0).stop 2024-04-25T14:15:30.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.700+0000 7fba31e00640 1 -- 172.21.15.86:0/4180118250 shutdown_connections 2024-04-25T14:15:30.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.700+0000 7fba31e00640 1 --2- 172.21.15.86:0/4180118250 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fba0806f450 0x7fba08071960 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:30.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.700+0000 7fba31e00640 1 --2- 172.21.15.86:0/4180118250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba2c0d16e0 0x7fba2c15fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:30.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.700+0000 7fba31e00640 1 --2- 172.21.15.86:0/4180118250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba2c0cfca0 0x7fba2c15f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:30.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.700+0000 7fba31e00640 1 --2- 172.21.15.86:0/4180118250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba2c06c670 0x7fba2c09a0b0 unknown :-1 s=CLOSED pgs=2217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:30.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.700+0000 7fba31e00640 1 -- 172.21.15.86:0/4180118250 >> 172.21.15.86:0/4180118250 conn(0x7fba2c0b4650 msgr2=0x7fba2c0cd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:30.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.700+0000 7fba31e00640 1 -- 172.21.15.86:0/4180118250 shutdown_connections 2024-04-25T14:15:30.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:30.700+0000 7fba31e00640 1 -- 172.21.15.86:0/4180118250 wait complete. 2024-04-25T14:15:31.121 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:31.121 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:32.123 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:32.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.284+0000 7f1318a00640 1 Processor -- start 2024-04-25T14:15:32.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.284+0000 7f1318a00640 1 -- start start 2024-04-25T14:15:32.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.288+0000 7f1318a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f13140cfca0 0x7f131406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:32.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.288+0000 7f1318a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f13140d16e0 0x7f13140d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:32.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.288+0000 7f1318a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13140d2030 con 0x7f131406c670 2024-04-25T14:15:32.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.288+0000 7f1318a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1314070470 con 0x7f13140cfca0 2024-04-25T14:15:32.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.288+0000 7f1318a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1314072bd0 con 0x7f13140d16e0 2024-04-25T14:15:32.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.288+0000 7f1311e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f13140cfca0 0x7f131406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:32.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.288+0000 7f1311e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f13140cfca0 0x7f131406f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47014/0 (socket says 172.21.15.86:47014) 2024-04-25T14:15:32.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.288+0000 7f1311e00640 1 -- 172.21.15.86:0/4129624942 learned_addr learned my addr 172.21.15.86:0/4129624942 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:32.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.288+0000 7f1309400640 1 --2- 172.21.15.86:0/4129624942 >> v2:172.21.15.155:3300/0 conn(0x7f13140d16e0 0x7f13140d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:32.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.288+0000 7f1311e00640 1 -- 172.21.15.86:0/4129624942 >> v2:172.21.15.155:3300/0 conn(0x7f13140d16e0 msgr2=0x7f13140d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:32.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.288+0000 7f1311e00640 1 --2- 172.21.15.86:0/4129624942 >> v2:172.21.15.155:3300/0 conn(0x7f13140d16e0 0x7f13140d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:32.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.288+0000 7f1311e00640 1 -- 172.21.15.86:0/4129624942 >> v1:172.21.15.155:6789/0 conn(0x7f131406c670 legacy=0x7f131406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:32.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.288+0000 7f1311e00640 1 -- 172.21.15.86:0/4129624942 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f131406f930 con 0x7f13140cfca0 2024-04-25T14:15:32.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.292+0000 7f1311e00640 1 --2- 172.21.15.86:0/4129624942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13140cfca0 0x7f131406f3f0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f13140729c0 tx=0x7f130401e9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d885bd0486655f63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:32.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.292+0000 7f1311400640 1 -- 172.21.15.86:0/4129624942 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13040b4070 con 0x7f13140cfca0 2024-04-25T14:15:32.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.292+0000 7f1311400640 1 -- 172.21.15.86:0/4129624942 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13040a78b0 con 0x7f13140cfca0 2024-04-25T14:15:32.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.292+0000 7f1311400640 1 -- 172.21.15.86:0/4129624942 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13040a7c80 con 0x7f13140cfca0 2024-04-25T14:15:32.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.292+0000 7f1318a00640 1 -- 172.21.15.86:0/4129624942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13140cfca0 msgr2=0x7f131406f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:32.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.292+0000 7f1318a00640 1 --2- 172.21.15.86:0/4129624942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13140cfca0 0x7f131406f3f0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f13140729c0 tx=0x7f130401e9d0 comp rx=0 tx=0).stop 2024-04-25T14:15:32.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.292+0000 7f1318a00640 1 -- 172.21.15.86:0/4129624942 shutdown_connections 2024-04-25T14:15:32.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.292+0000 7f1318a00640 1 --2- 172.21.15.86:0/4129624942 >> v2:172.21.15.155:3300/0 conn(0x7f13140d16e0 0x7f13140d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:32.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.292+0000 7f1318a00640 1 --2- 172.21.15.86:0/4129624942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13140cfca0 0x7f131406f3f0 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:32.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.292+0000 7f1318a00640 1 -- 172.21.15.86:0/4129624942 >> 172.21.15.86:0/4129624942 conn(0x7f13140b4650 msgr2=0x7f13140b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:32.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.292+0000 7f1318a00640 1 -- 172.21.15.86:0/4129624942 shutdown_connections 2024-04-25T14:15:32.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.292+0000 7f1318a00640 1 -- 172.21.15.86:0/4129624942 wait complete. 2024-04-25T14:15:32.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.292+0000 7f1318a00640 1 Processor -- start 2024-04-25T14:15:32.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.292+0000 7f1318a00640 1 -- start start 2024-04-25T14:15:32.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.296+0000 7f1318a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f131406c670 0x7f13141fcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:32.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.296+0000 7f1318a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13140cfca0 0x7f1314157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:32.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.296+0000 7f1318a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13140d16e0 0x7f1314158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:32.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.296+0000 7f1318a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f13140ce6f0 con 0x7f13140d16e0 2024-04-25T14:15:32.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.296+0000 7f1318a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f13140ce570 con 0x7f131406c670 2024-04-25T14:15:32.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.296+0000 7f1318a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f13140ce870 con 0x7f13140cfca0 2024-04-25T14:15:32.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.296+0000 7f1312800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13140d16e0 0x7f1314158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:32.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.296+0000 7f1312800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13140d16e0 0x7f1314158210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47020/0 (socket says 172.21.15.86:47020) 2024-04-25T14:15:32.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.296+0000 7f1312800640 1 -- 172.21.15.86:0/3340020296 learned_addr learned my addr 172.21.15.86:0/3340020296 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.296+0000 7f1311e00640 1 --2- 172.21.15.86:0/3340020296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f131406c670 0x7f13141fcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.296+0000 7f1309400640 1 --2- 172.21.15.86:0/3340020296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13140cfca0 0x7f1314157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.296+0000 7f1312800640 1 -- 172.21.15.86:0/3340020296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13140cfca0 msgr2=0x7f1314157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.296+0000 7f1312800640 1 --2- 172.21.15.86:0/3340020296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13140cfca0 0x7f1314157cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.296+0000 7f1312800640 1 -- 172.21.15.86:0/3340020296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f131406c670 msgr2=0x7f13141fcfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.296+0000 7f1312800640 1 --2- 172.21.15.86:0/3340020296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f131406c670 0x7f13141fcfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.296+0000 7f1312800640 1 -- 172.21.15.86:0/3340020296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f131409ae90 con 0x7f13140d16e0 2024-04-25T14:15:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.296+0000 7f1312800640 1 --2- 172.21.15.86:0/3340020296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13140d16e0 0x7f1314158210 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f130c00b4c0 tx=0x7f130c00b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.296+0000 7f130aa00640 1 -- 172.21.15.86:0/3340020296 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c015020 con 0x7f13140d16e0 2024-04-25T14:15:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.296+0000 7f130aa00640 1 -- 172.21.15.86:0/3340020296 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f130c011830 con 0x7f13140d16e0 2024-04-25T14:15:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.296+0000 7f130aa00640 1 -- 172.21.15.86:0/3340020296 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c011c20 con 0x7f13140d16e0 2024-04-25T14:15:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.296+0000 7f1318a00640 1 -- 172.21.15.86:0/3340020296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f131409abb0 con 0x7f13140d16e0 2024-04-25T14:15:32.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.296+0000 7f1318a00640 1 -- 172.21.15.86:0/3340020296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13141561a0 con 0x7f13140d16e0 2024-04-25T14:15:32.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.296+0000 7f1318a00640 1 -- 172.21.15.86:0/3340020296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f131409f4b0 con 0x7f13140d16e0 2024-04-25T14:15:32.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.300+0000 7f130aa00640 1 -- 172.21.15.86:0/3340020296 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f130c018020 con 0x7f13140d16e0 2024-04-25T14:15:32.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.300+0000 7f130aa00640 1 --2- 172.21.15.86:0/3340020296 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f12f006f480 0x7f12f0071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:32.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.300+0000 7f130aa00640 1 -- 172.21.15.86:0/3340020296 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f130c08f0f0 con 0x7f13140d16e0 2024-04-25T14:15:32.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.300+0000 7f1311e00640 1 --2- 172.21.15.86:0/3340020296 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f12f006f480 0x7f12f0071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:32.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.300+0000 7f1311e00640 1 --2- 172.21.15.86:0/3340020296 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f12f006f480 0x7f12f0071990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f13040b1000 tx=0x7f13040a9070 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:32.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.304+0000 7f130aa00640 1 -- 172.21.15.86:0/3340020296 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f130c01f030 con 0x7f13140d16e0 2024-04-25T14:15:32.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.496+0000 7f1318a00640 1 -- 172.21.15.86:0/3340020296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1314069d00 con 0x7f13140d16e0 2024-04-25T14:15:32.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.496+0000 7f130aa00640 1 -- 172.21.15.86:0/3340020296 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f130c05b610 con 0x7f13140d16e0 2024-04-25T14:15:32.493 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:32.493 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:32.493 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:32.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.496+0000 7f1318a00640 1 -- 172.21.15.86:0/3340020296 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f12f006f480 msgr2=0x7f12f0071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:32.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.496+0000 7f1318a00640 1 --2- 172.21.15.86:0/3340020296 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f12f006f480 0x7f12f0071990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f13040b1000 tx=0x7f13040a9070 comp rx=0 tx=0).stop 2024-04-25T14:15:32.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.496+0000 7f1318a00640 1 -- 172.21.15.86:0/3340020296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13140d16e0 msgr2=0x7f1314158210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:32.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.496+0000 7f1318a00640 1 --2- 172.21.15.86:0/3340020296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13140d16e0 0x7f1314158210 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f130c00b4c0 tx=0x7f130c00b980 comp rx=0 tx=0).stop 2024-04-25T14:15:32.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.496+0000 7f1318a00640 1 -- 172.21.15.86:0/3340020296 shutdown_connections 2024-04-25T14:15:32.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.496+0000 7f1318a00640 1 --2- 172.21.15.86:0/3340020296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13140d16e0 0x7f1314158210 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:32.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.496+0000 7f1318a00640 1 --2- 172.21.15.86:0/3340020296 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f12f006f480 0x7f12f0071990 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:32.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.496+0000 7f1318a00640 1 --2- 172.21.15.86:0/3340020296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13140cfca0 0x7f1314157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:32.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.496+0000 7f1318a00640 1 --2- 172.21.15.86:0/3340020296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f131406c670 0x7f13141fcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:32.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.496+0000 7f1318a00640 1 -- 172.21.15.86:0/3340020296 >> 172.21.15.86:0/3340020296 conn(0x7f13140b4650 msgr2=0x7f1314068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:32.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.496+0000 7f1318a00640 1 -- 172.21.15.86:0/3340020296 shutdown_connections 2024-04-25T14:15:32.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:32.496+0000 7f1318a00640 1 -- 172.21.15.86:0/3340020296 wait complete. 2024-04-25T14:15:32.917 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:32.917 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:33.918 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:34.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.076+0000 7fad17800640 1 Processor -- start 2024-04-25T14:15:34.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.076+0000 7fad17800640 1 -- start start 2024-04-25T14:15:34.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.080+0000 7fad17800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fad10070330 0x7fad100d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:34.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.080+0000 7fad17800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad100ce790 0x7fad1006f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:34.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.080+0000 7fad17800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad100d2030 con 0x7fad100d1c20 2024-04-25T14:15:34.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.080+0000 7fad17800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fad1015e5c0 con 0x7fad10070330 2024-04-25T14:15:34.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.080+0000 7fad17800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fad1015e8a0 con 0x7fad100ce790 2024-04-25T14:15:34.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.080+0000 7fad14e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fad10070330 0x7fad100d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:34.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.080+0000 7fad14e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fad10070330 0x7fad100d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47036/0 (socket says 172.21.15.86:47036) 2024-04-25T14:15:34.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.080+0000 7fad14e00640 1 -- 172.21.15.86:0/313821115 learned_addr learned my addr 172.21.15.86:0/313821115 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:34.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.080+0000 7fad15800640 1 --2- 172.21.15.86:0/313821115 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad100ce790 0x7fad1006f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:34.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.080+0000 7fad14e00640 1 -- 172.21.15.86:0/313821115 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad100ce790 msgr2=0x7fad1006f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:34.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.080+0000 7fad14e00640 1 --2- 172.21.15.86:0/313821115 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad100ce790 0x7fad1006f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:34.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.080+0000 7fad14e00640 1 -- 172.21.15.86:0/313821115 >> v1:172.21.15.155:6789/0 conn(0x7fad100d1c20 legacy=0x7fad1006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:34.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.080+0000 7fad14e00640 1 -- 172.21.15.86:0/313821115 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1015eb80 con 0x7fad10070330 2024-04-25T14:15:34.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.084+0000 7fad14e00640 1 --2- 172.21.15.86:0/313821115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad10070330 0x7fad100d16e0 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7fad00009870 tx=0x7fad0001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d371087cf607d482 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:34.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.084+0000 7fad0fe00640 1 -- 172.21.15.86:0/313821115 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad000b0070 con 0x7fad10070330 2024-04-25T14:15:34.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.084+0000 7fad0fe00640 1 -- 172.21.15.86:0/313821115 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad000ab600 con 0x7fad10070330 2024-04-25T14:15:34.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.084+0000 7fad0fe00640 1 -- 172.21.15.86:0/313821115 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad000ab9d0 con 0x7fad10070330 2024-04-25T14:15:34.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.084+0000 7fad17800640 1 -- 172.21.15.86:0/313821115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad10070330 msgr2=0x7fad100d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:34.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.084+0000 7fad17800640 1 --2- 172.21.15.86:0/313821115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad10070330 0x7fad100d16e0 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7fad00009870 tx=0x7fad0001c4d0 comp rx=0 tx=0).stop 2024-04-25T14:15:34.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.084+0000 7fad17800640 1 -- 172.21.15.86:0/313821115 shutdown_connections 2024-04-25T14:15:34.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.084+0000 7fad17800640 1 --2- 172.21.15.86:0/313821115 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad100ce790 0x7fad1006f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:34.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.084+0000 7fad17800640 1 --2- 172.21.15.86:0/313821115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad10070330 0x7fad100d16e0 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:34.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.084+0000 7fad17800640 1 -- 172.21.15.86:0/313821115 >> 172.21.15.86:0/313821115 conn(0x7fad100b4650 msgr2=0x7fad100b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:34.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.084+0000 7fad17800640 1 -- 172.21.15.86:0/313821115 shutdown_connections 2024-04-25T14:15:34.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.084+0000 7fad17800640 1 -- 172.21.15.86:0/313821115 wait complete. 2024-04-25T14:15:34.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.088+0000 7fad17800640 1 Processor -- start 2024-04-25T14:15:34.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.088+0000 7fad17800640 1 -- start start 2024-04-25T14:15:34.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.088+0000 7fad17800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad10070330 0x7fad10157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:34.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.088+0000 7fad17800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad100ce790 0x7fad100c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:34.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.088+0000 7fad17800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad100d1c20 0x7fad100c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:34.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.088+0000 7fad17800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fad100708c0 con 0x7fad100ce790 2024-04-25T14:15:34.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.088+0000 7fad17800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fad10070740 con 0x7fad10070330 2024-04-25T14:15:34.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.088+0000 7fad17800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fad1016afc0 con 0x7fad100d1c20 2024-04-25T14:15:34.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.088+0000 7fad07e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad100ce790 0x7fad100c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:34.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.088+0000 7fad07e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad100ce790 0x7fad100c60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47048/0 (socket says 172.21.15.86:47048) 2024-04-25T14:15:34.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.088+0000 7fad07e00640 1 -- 172.21.15.86:0/2052080091 learned_addr learned my addr 172.21.15.86:0/2052080091 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:34.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.088+0000 7fad14e00640 1 --2- 172.21.15.86:0/2052080091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad10070330 0x7fad10157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:34.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.088+0000 7fad15800640 1 --2- 172.21.15.86:0/2052080091 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad100d1c20 0x7fad100c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:34.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.088+0000 7fad07e00640 1 -- 172.21.15.86:0/2052080091 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad100d1c20 msgr2=0x7fad100c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:34.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.088+0000 7fad07e00640 1 --2- 172.21.15.86:0/2052080091 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad100d1c20 0x7fad100c6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:34.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.088+0000 7fad07e00640 1 -- 172.21.15.86:0/2052080091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad10070330 msgr2=0x7fad10157c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:34.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.088+0000 7fad07e00640 1 --2- 172.21.15.86:0/2052080091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad10070330 0x7fad10157c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:34.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.088+0000 7fad07e00640 1 -- 172.21.15.86:0/2052080091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad100d0200 con 0x7fad100ce790 2024-04-25T14:15:34.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.088+0000 7fad15800640 1 --2- 172.21.15.86:0/2052080091 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad100d1c20 0x7fad100c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:34.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.088+0000 7fad07e00640 1 --2- 172.21.15.86:0/2052080091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad100ce790 0x7fad100c60d0 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7facf800c310 tx=0x7facf800c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:34.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.088+0000 7fad0d600640 1 -- 172.21.15.86:0/2052080091 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf80052d0 con 0x7fad100ce790 2024-04-25T14:15:34.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.088+0000 7fad0d600640 1 -- 172.21.15.86:0/2052080091 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facf8005470 con 0x7fad100ce790 2024-04-25T14:15:34.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.088+0000 7fad17800640 1 -- 172.21.15.86:0/2052080091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad100d04e0 con 0x7fad100ce790 2024-04-25T14:15:34.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.088+0000 7fad0d600640 1 -- 172.21.15.86:0/2052080091 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8007260 con 0x7fad100ce790 2024-04-25T14:15:34.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.088+0000 7fad17800640 1 -- 172.21.15.86:0/2052080091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad1015aaa0 con 0x7fad100ce790 2024-04-25T14:15:34.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.092+0000 7fad17800640 1 -- 172.21.15.86:0/2052080091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad1015ad80 con 0x7fad100ce790 2024-04-25T14:15:34.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.092+0000 7fad0d600640 1 -- 172.21.15.86:0/2052080091 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7facf8007400 con 0x7fad100ce790 2024-04-25T14:15:34.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.092+0000 7fad0d600640 1 --2- 172.21.15.86:0/2052080091 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7facec06f450 0x7facec071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:34.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.092+0000 7fad0d600640 1 -- 172.21.15.86:0/2052080091 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7facf808e9d0 con 0x7fad100ce790 2024-04-25T14:15:34.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.092+0000 7fad14e00640 1 --2- 172.21.15.86:0/2052080091 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7facec06f450 0x7facec071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:34.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.092+0000 7fad14e00640 1 --2- 172.21.15.86:0/2052080091 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7facec06f450 0x7facec071960 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fad000ad000 tx=0x7fad000a6000 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:34.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.096+0000 7fad0d600640 1 -- 172.21.15.86:0/2052080091 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fad100d2030 con 0x7fad100ce790 2024-04-25T14:15:34.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.292+0000 7fad17800640 1 -- 172.21.15.86:0/2052080091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad10069d00 con 0x7fad100ce790 2024-04-25T14:15:34.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.292+0000 7fad0d600640 1 -- 172.21.15.86:0/2052080091 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7facf805af80 con 0x7fad100ce790 2024-04-25T14:15:34.290 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:34.290 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:34.291 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:34.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.296+0000 7fad17800640 1 -- 172.21.15.86:0/2052080091 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7facec06f450 msgr2=0x7facec071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:34.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.296+0000 7fad17800640 1 --2- 172.21.15.86:0/2052080091 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7facec06f450 0x7facec071960 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fad000ad000 tx=0x7fad000a6000 comp rx=0 tx=0).stop 2024-04-25T14:15:34.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.296+0000 7fad17800640 1 -- 172.21.15.86:0/2052080091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad100ce790 msgr2=0x7fad100c60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:34.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.296+0000 7fad17800640 1 --2- 172.21.15.86:0/2052080091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad100ce790 0x7fad100c60d0 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7facf800c310 tx=0x7facf800c7d0 comp rx=0 tx=0).stop 2024-04-25T14:15:34.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.296+0000 7fad17800640 1 -- 172.21.15.86:0/2052080091 shutdown_connections 2024-04-25T14:15:34.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.296+0000 7fad17800640 1 --2- 172.21.15.86:0/2052080091 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7facec06f450 0x7facec071960 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:34.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.296+0000 7fad17800640 1 --2- 172.21.15.86:0/2052080091 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad100d1c20 0x7fad100c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:34.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.296+0000 7fad17800640 1 --2- 172.21.15.86:0/2052080091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad100ce790 0x7fad100c60d0 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:34.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.296+0000 7fad17800640 1 --2- 172.21.15.86:0/2052080091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad10070330 0x7fad10157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:34.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.296+0000 7fad17800640 1 -- 172.21.15.86:0/2052080091 >> 172.21.15.86:0/2052080091 conn(0x7fad100b4650 msgr2=0x7fad1015b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:34.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.296+0000 7fad17800640 1 -- 172.21.15.86:0/2052080091 shutdown_connections 2024-04-25T14:15:34.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:34.296+0000 7fad17800640 1 -- 172.21.15.86:0/2052080091 wait complete. 2024-04-25T14:15:34.715 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:34.715 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:35.716 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:35.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.880+0000 7fc0e9c00640 1 Processor -- start 2024-04-25T14:15:35.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.880+0000 7fc0e9c00640 1 -- start start 2024-04-25T14:15:35.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.880+0000 7fc0e9c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc0e4070330 0x7fc0e40d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:35.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.880+0000 7fc0e9c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc0e40d1c20 0x7fc0e406ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:35.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.880+0000 7fc0e9c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0e406f270 0x7fc0e406f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.880+0000 7fc0e9c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc0e40d2030 con 0x7fc0e40d1c20 2024-04-25T14:15:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.880+0000 7fc0e9c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc0e415e5c0 con 0x7fc0e4070330 2024-04-25T14:15:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.880+0000 7fc0e9c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc0e415e8a0 con 0x7fc0e406f270 2024-04-25T14:15:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.880+0000 7fc0e2000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc0e40d1c20 0x7fc0e406ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.880+0000 7fc0e2000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc0e40d1c20 0x7fc0e406ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47054/0 (socket says 172.21.15.86:47054) 2024-04-25T14:15:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.880+0000 7fc0e2000640 1 -- 172.21.15.86:0/4191860612 learned_addr learned my addr 172.21.15.86:0/4191860612 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.880+0000 7fc0e2a00640 1 --2- 172.21.15.86:0/4191860612 >> v2:172.21.15.155:3300/0 conn(0x7fc0e4070330 0x7fc0e40d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.880+0000 7fc0e3400640 1 --2- 172.21.15.86:0/4191860612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0e406f270 0x7fc0e406f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.880+0000 7fc0e2000640 1 -- 172.21.15.86:0/4191860612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0e406f270 msgr2=0x7fc0e406f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.880+0000 7fc0e2000640 1 --2- 172.21.15.86:0/4191860612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0e406f270 0x7fc0e406f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.880+0000 7fc0e2000640 1 -- 172.21.15.86:0/4191860612 >> v2:172.21.15.155:3300/0 conn(0x7fc0e4070330 msgr2=0x7fc0e40d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.880+0000 7fc0e2000640 1 --2- 172.21.15.86:0/4191860612 >> v2:172.21.15.155:3300/0 conn(0x7fc0e4070330 0x7fc0e40d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.880+0000 7fc0e2000640 1 -- 172.21.15.86:0/4191860612 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e415eb80 con 0x7fc0e40d1c20 2024-04-25T14:15:35.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.880+0000 7fc0e3400640 1 --2- 172.21.15.86:0/4191860612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0e406f270 0x7fc0e406f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:35.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.880+0000 7fc0e2a00640 1 --2- 172.21.15.86:0/4191860612 >> v2:172.21.15.155:3300/0 conn(0x7fc0e4070330 0x7fc0e40d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:35.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.884+0000 7fc0e2000640 1 --2- 172.21.15.86:0/4191860612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0e40d1c20 0x7fc0e406ed30 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7fc0e40d23c0 tx=0x7fc0d001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892bb14f025f0615 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:35.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.884+0000 7fc0e1600640 1 -- 172.21.15.86:0/4191860612 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d00ab600 con 0x7fc0e40d1c20 2024-04-25T14:15:35.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.884+0000 7fc0e1600640 1 -- 172.21.15.86:0/4191860612 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0d00b0070 con 0x7fc0e40d1c20 2024-04-25T14:15:35.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.884+0000 7fc0e1600640 1 -- 172.21.15.86:0/4191860612 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d00ab9c0 con 0x7fc0e40d1c20 2024-04-25T14:15:35.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.884+0000 7fc0e9c00640 1 -- 172.21.15.86:0/4191860612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0e40d1c20 msgr2=0x7fc0e406ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:35.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.884+0000 7fc0e9c00640 1 --2- 172.21.15.86:0/4191860612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0e40d1c20 0x7fc0e406ed30 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7fc0e40d23c0 tx=0x7fc0d001c460 comp rx=0 tx=0).stop 2024-04-25T14:15:35.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.884+0000 7fc0e9c00640 1 -- 172.21.15.86:0/4191860612 shutdown_connections 2024-04-25T14:15:35.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.884+0000 7fc0e9c00640 1 --2- 172.21.15.86:0/4191860612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0e406f270 0x7fc0e406f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:35.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.884+0000 7fc0e9c00640 1 --2- 172.21.15.86:0/4191860612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0e40d1c20 0x7fc0e406ed30 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:35.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.884+0000 7fc0e9c00640 1 --2- 172.21.15.86:0/4191860612 >> v2:172.21.15.155:3300/0 conn(0x7fc0e4070330 0x7fc0e40d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:35.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.884+0000 7fc0e9c00640 1 -- 172.21.15.86:0/4191860612 >> 172.21.15.86:0/4191860612 conn(0x7fc0e40b4650 msgr2=0x7fc0e40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:35.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.884+0000 7fc0e9c00640 1 -- 172.21.15.86:0/4191860612 shutdown_connections 2024-04-25T14:15:35.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.884+0000 7fc0e9c00640 1 -- 172.21.15.86:0/4191860612 wait complete. 2024-04-25T14:15:35.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.888+0000 7fc0e9c00640 1 Processor -- start 2024-04-25T14:15:35.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.888+0000 7fc0e9c00640 1 -- start start 2024-04-25T14:15:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.888+0000 7fc0e9c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0e406f270 0x7fc0e40c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.888+0000 7fc0e9c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0e4070330 0x7fc0e40ae810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.888+0000 7fc0e9c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0e40d1c20 0x7fc0e40aed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.888+0000 7fc0e9c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e415f4b0 con 0x7fc0e40d1c20 2024-04-25T14:15:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.888+0000 7fc0e9c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e415f330 con 0x7fc0e406f270 2024-04-25T14:15:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.888+0000 7fc0e9c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc0e4070740 con 0x7fc0e4070330 2024-04-25T14:15:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.888+0000 7fc0e3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0e40d1c20 0x7fc0e40aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.888+0000 7fc0e3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0e40d1c20 0x7fc0e40aed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47070/0 (socket says 172.21.15.86:47070) 2024-04-25T14:15:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.888+0000 7fc0e3400640 1 -- 172.21.15.86:0/557626997 learned_addr learned my addr 172.21.15.86:0/557626997 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.888+0000 7fc0e2a00640 1 --2- 172.21.15.86:0/557626997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0e406f270 0x7fc0e40c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.892+0000 7fc0e2000640 1 --2- 172.21.15.86:0/557626997 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0e4070330 0x7fc0e40ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.892+0000 7fc0e3400640 1 -- 172.21.15.86:0/557626997 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0e4070330 msgr2=0x7fc0e40ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.892+0000 7fc0e3400640 1 --2- 172.21.15.86:0/557626997 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0e4070330 0x7fc0e40ae810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.892+0000 7fc0e3400640 1 -- 172.21.15.86:0/557626997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0e406f270 msgr2=0x7fc0e40c77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.892+0000 7fc0e3400640 1 --2- 172.21.15.86:0/557626997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0e406f270 0x7fc0e40c77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.892+0000 7fc0e3400640 1 -- 172.21.15.86:0/557626997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e4158f80 con 0x7fc0e40d1c20 2024-04-25T14:15:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.892+0000 7fc0e2000640 1 --2- 172.21.15.86:0/557626997 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0e4070330 0x7fc0e40ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.892+0000 7fc0e2a00640 1 --2- 172.21.15.86:0/557626997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0e406f270 0x7fc0e40c77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.892+0000 7fc0e3400640 1 --2- 172.21.15.86:0/557626997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0e40d1c20 0x7fc0e40aed50 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc007db0 tx=0x7fc0dc00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.892+0000 7fc0daa00640 1 -- 172.21.15.86:0/557626997 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc01a070 con 0x7fc0e40d1c20 2024-04-25T14:15:35.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.892+0000 7fc0daa00640 1 -- 172.21.15.86:0/557626997 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0dc0039a0 con 0x7fc0e40d1c20 2024-04-25T14:15:35.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.892+0000 7fc0e9c00640 1 -- 172.21.15.86:0/557626997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0e4159260 con 0x7fc0e40d1c20 2024-04-25T14:15:35.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.892+0000 7fc0daa00640 1 -- 172.21.15.86:0/557626997 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc003d70 con 0x7fc0e40d1c20 2024-04-25T14:15:35.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.892+0000 7fc0e9c00640 1 -- 172.21.15.86:0/557626997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0e4159820 con 0x7fc0e40d1c20 2024-04-25T14:15:35.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.892+0000 7fc0e9c00640 1 -- 172.21.15.86:0/557626997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0e4069d00 con 0x7fc0e40d1c20 2024-04-25T14:15:35.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.892+0000 7fc0daa00640 1 -- 172.21.15.86:0/557626997 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7fc0dc01d070 con 0x7fc0e40d1c20 2024-04-25T14:15:35.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.892+0000 7fc0daa00640 1 --2- 172.21.15.86:0/557626997 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc0c406f450 0x7fc0c4071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:35.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.892+0000 7fc0daa00640 1 -- 172.21.15.86:0/557626997 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fc0dc08f7f0 con 0x7fc0e40d1c20 2024-04-25T14:15:35.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.896+0000 7fc0e2a00640 1 --2- 172.21.15.86:0/557626997 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc0c406f450 0x7fc0c4071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:35.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.896+0000 7fc0e2a00640 1 --2- 172.21.15.86:0/557626997 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc0c406f450 0x7fc0c4071960 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4072200 tx=0x7fc0d4008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:35.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:35.900+0000 7fc0daa00640 1 -- 172.21.15.86:0/557626997 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc0dc010070 con 0x7fc0e40d1c20 2024-04-25T14:15:36.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:36.088+0000 7fc0e9c00640 1 -- 172.21.15.86:0/557626997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0e40c7f10 con 0x7fc0e40d1c20 2024-04-25T14:15:36.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:36.088+0000 7fc0daa00640 1 -- 172.21.15.86:0/557626997 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7fc0dc05c400 con 0x7fc0e40d1c20 2024-04-25T14:15:36.087 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:36.087 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:36.088 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:36.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:36.092+0000 7fc0e9c00640 1 -- 172.21.15.86:0/557626997 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc0c406f450 msgr2=0x7fc0c4071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:36.092+0000 7fc0e9c00640 1 --2- 172.21.15.86:0/557626997 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc0c406f450 0x7fc0c4071960 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4072200 tx=0x7fc0d4008040 comp rx=0 tx=0).stop 2024-04-25T14:15:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:36.092+0000 7fc0e9c00640 1 -- 172.21.15.86:0/557626997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0e40d1c20 msgr2=0x7fc0e40aed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:36.092+0000 7fc0e9c00640 1 --2- 172.21.15.86:0/557626997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0e40d1c20 0x7fc0e40aed50 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc007db0 tx=0x7fc0dc00d4a0 comp rx=0 tx=0).stop 2024-04-25T14:15:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:36.092+0000 7fc0e9c00640 1 -- 172.21.15.86:0/557626997 shutdown_connections 2024-04-25T14:15:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:36.092+0000 7fc0e9c00640 1 --2- 172.21.15.86:0/557626997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0e40d1c20 0x7fc0e40aed50 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:36.092+0000 7fc0e9c00640 1 --2- 172.21.15.86:0/557626997 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc0c406f450 0x7fc0c4071960 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:36.092+0000 7fc0e9c00640 1 --2- 172.21.15.86:0/557626997 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0e4070330 0x7fc0e40ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:36.092+0000 7fc0e9c00640 1 --2- 172.21.15.86:0/557626997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0e406f270 0x7fc0e40c77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:36.092+0000 7fc0e9c00640 1 -- 172.21.15.86:0/557626997 >> 172.21.15.86:0/557626997 conn(0x7fc0e40b4650 msgr2=0x7fc0e415bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:36.092+0000 7fc0e9c00640 1 -- 172.21.15.86:0/557626997 shutdown_connections 2024-04-25T14:15:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:36.092+0000 7fc0e9c00640 1 -- 172.21.15.86:0/557626997 wait complete. 2024-04-25T14:15:36.513 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:36.513 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:37.514 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:37.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.676+0000 7fe21d800640 1 Processor -- start 2024-04-25T14:15:37.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.676+0000 7fe21d800640 1 -- start start 2024-04-25T14:15:37.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.676+0000 7fe21d800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2181635a0 0x7fe21806eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:37.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.676+0000 7fe21d800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe218163d80 con 0x7fe2180d08c0 2024-04-25T14:15:37.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.676+0000 7fe21d800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe218164060 con 0x7fe21806f060 2024-04-25T14:15:37.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.676+0000 7fe21d800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe21806f470 con 0x7fe2181635a0 2024-04-25T14:15:37.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.676+0000 7fe21ce00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe2180d08c0 0x7fe2180722a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53292/0 (socket says 172.21.15.86:53292) 2024-04-25T14:15:37.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.676+0000 7fe21ce00640 1 -- 172.21.15.86:0/1098605981 learned_addr learned my addr 172.21.15.86:0/1098605981 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:15:37.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.676+0000 7fe217e00640 1 --2- 172.21.15.86:0/1098605981 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2181635a0 0x7fe21806eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:37.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.676+0000 7fe216a00640 1 -- 172.21.15.86:0/1098605981 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1815090605 0 0) 0x7fe218163d80 con 0x7fe2180d08c0 2024-04-25T14:15:37.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.676+0000 7fe216a00640 1 -- 172.21.15.86:0/1098605981 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1fc003530 con 0x7fe2180d08c0 2024-04-25T14:15:37.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.676+0000 7fe216a00640 1 -- 172.21.15.86:0/1098605981 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1779156311 0 0) 0x7fe218164060 con 0x7fe21806f060 2024-04-25T14:15:37.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.676+0000 7fe216a00640 1 -- 172.21.15.86:0/1098605981 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe218163d80 con 0x7fe21806f060 2024-04-25T14:15:37.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.676+0000 7fe217e00640 1 -- 172.21.15.86:0/1098605981 >> v1:172.21.15.155:6789/0 conn(0x7fe21806f060 legacy=0x7fe2180d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:37.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.676+0000 7fe217e00640 1 -- 172.21.15.86:0/1098605981 >> v1:172.21.15.86:6789/0 conn(0x7fe2180d08c0 legacy=0x7fe2180722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:37.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.676+0000 7fe217e00640 1 -- 172.21.15.86:0/1098605981 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe218164340 con 0x7fe2181635a0 2024-04-25T14:15:37.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.680+0000 7fe217e00640 1 --2- 172.21.15.86:0/1098605981 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2181635a0 0x7fe21806eb20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe2080097f0 tx=0x7fe20801c500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=258016a6629e27bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:37.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.680+0000 7fe216a00640 1 -- 172.21.15.86:0/1098605981 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2080b0070 con 0x7fe2181635a0 2024-04-25T14:15:37.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.680+0000 7fe216a00640 1 -- 172.21.15.86:0/1098605981 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2080ab600 con 0x7fe2181635a0 2024-04-25T14:15:37.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.680+0000 7fe216a00640 1 -- 172.21.15.86:0/1098605981 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2080ab9d0 con 0x7fe2181635a0 2024-04-25T14:15:37.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.680+0000 7fe21d800640 1 -- 172.21.15.86:0/1098605981 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2181635a0 msgr2=0x7fe21806eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:37.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.680+0000 7fe21d800640 1 --2- 172.21.15.86:0/1098605981 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2181635a0 0x7fe21806eb20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe2080097f0 tx=0x7fe20801c500 comp rx=0 tx=0).stop 2024-04-25T14:15:37.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.680+0000 7fe21d800640 1 -- 172.21.15.86:0/1098605981 shutdown_connections 2024-04-25T14:15:37.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.680+0000 7fe21d800640 1 --2- 172.21.15.86:0/1098605981 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2181635a0 0x7fe21806eb20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:37.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.680+0000 7fe21d800640 1 -- 172.21.15.86:0/1098605981 >> 172.21.15.86:0/1098605981 conn(0x7fe2180b4f80 msgr2=0x7fe2180b5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:37.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.680+0000 7fe21d800640 1 -- 172.21.15.86:0/1098605981 shutdown_connections 2024-04-25T14:15:37.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.680+0000 7fe21d800640 1 -- 172.21.15.86:0/1098605981 wait complete. 2024-04-25T14:15:37.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.684+0000 7fe21d800640 1 Processor -- start 2024-04-25T14:15:37.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.684+0000 7fe21d800640 1 -- start start 2024-04-25T14:15:37.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.684+0000 7fe21d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe21806f060 0x7fe218150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:37.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.684+0000 7fe21d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2180d08c0 0x7fe218150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:37.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.684+0000 7fe21d800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2181635a0 0x7fe21813c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:37.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.684+0000 7fe21d800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe218072b30 con 0x7fe2180d08c0 2024-04-25T14:15:37.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.684+0000 7fe21d800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe2180729b0 con 0x7fe21806f060 2024-04-25T14:15:37.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.684+0000 7fe21d800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe21816df50 con 0x7fe2181635a0 2024-04-25T14:15:37.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.684+0000 7fe217400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2180d08c0 0x7fe218150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:37.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.684+0000 7fe217400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2180d08c0 0x7fe218150550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47076/0 (socket says 172.21.15.86:47076) 2024-04-25T14:15:37.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.684+0000 7fe217400640 1 -- 172.21.15.86:0/941559176 learned_addr learned my addr 172.21.15.86:0/941559176 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:37.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.684+0000 7fe217e00640 1 --2- 172.21.15.86:0/941559176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe21806f060 0x7fe218150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:37.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.684+0000 7fe21ce00640 1 --2- 172.21.15.86:0/941559176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2181635a0 0x7fe21813c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:37.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.684+0000 7fe217400640 1 -- 172.21.15.86:0/941559176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2181635a0 msgr2=0x7fe21813c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:37.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.684+0000 7fe217400640 1 --2- 172.21.15.86:0/941559176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2181635a0 0x7fe21813c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:37.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.684+0000 7fe217400640 1 -- 172.21.15.86:0/941559176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe21806f060 msgr2=0x7fe218150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:37.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.684+0000 7fe217400640 1 --2- 172.21.15.86:0/941559176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe21806f060 0x7fe218150f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:37.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.684+0000 7fe217400640 1 -- 172.21.15.86:0/941559176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe218151440 con 0x7fe2180d08c0 2024-04-25T14:15:37.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.688+0000 7fe217400640 1 --2- 172.21.15.86:0/941559176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2180d08c0 0x7fe218150550 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7fe218073c00 tx=0x7fe20400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:37.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.688+0000 7fe20fe00640 1 -- 172.21.15.86:0/941559176 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe204017070 con 0x7fe2180d08c0 2024-04-25T14:15:37.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.688+0000 7fe20fe00640 1 -- 172.21.15.86:0/941559176 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2040052d0 con 0x7fe2180d08c0 2024-04-25T14:15:37.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.688+0000 7fe20fe00640 1 -- 172.21.15.86:0/941559176 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2040054c0 con 0x7fe2180d08c0 2024-04-25T14:15:37.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.688+0000 7fe21d800640 1 -- 172.21.15.86:0/941559176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe218150a90 con 0x7fe2180d08c0 2024-04-25T14:15:37.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.688+0000 7fe21d800640 1 -- 172.21.15.86:0/941559176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2181520c0 con 0x7fe2180d08c0 2024-04-25T14:15:37.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.688+0000 7fe21d800640 1 -- 172.21.15.86:0/941559176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe218151b50 con 0x7fe2180d08c0 2024-04-25T14:15:37.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.688+0000 7fe20fe00640 1 -- 172.21.15.86:0/941559176 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7fe20401a070 con 0x7fe2180d08c0 2024-04-25T14:15:37.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.688+0000 7fe20fe00640 1 --2- 172.21.15.86:0/941559176 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe1f806f480 0x7fe1f8071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:37.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.688+0000 7fe20fe00640 1 -- 172.21.15.86:0/941559176 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fe20408e850 con 0x7fe2180d08c0 2024-04-25T14:15:37.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.688+0000 7fe217e00640 1 --2- 172.21.15.86:0/941559176 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe1f806f480 0x7fe1f8071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:37.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.692+0000 7fe217e00640 1 --2- 172.21.15.86:0/941559176 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe1f806f480 0x7fe1f8071990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe2080ad000 tx=0x7fe2080a4f90 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:37.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.692+0000 7fe20fe00640 1 -- 172.21.15.86:0/941559176 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe218150a90 con 0x7fe2180d08c0 2024-04-25T14:15:37.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.920+0000 7fe21d800640 1 -- 172.21.15.86:0/941559176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe218076b80 con 0x7fe2180d08c0 2024-04-25T14:15:37.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.920+0000 7fe20fe00640 1 -- 172.21.15.86:0/941559176 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7fe20408ce20 con 0x7fe2180d08c0 2024-04-25T14:15:37.916 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:37.917 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:37.917 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:37.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.920+0000 7fe21d800640 1 -- 172.21.15.86:0/941559176 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe1f806f480 msgr2=0x7fe1f8071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:37.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.920+0000 7fe21d800640 1 --2- 172.21.15.86:0/941559176 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe1f806f480 0x7fe1f8071990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe2080ad000 tx=0x7fe2080a4f90 comp rx=0 tx=0).stop 2024-04-25T14:15:37.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.920+0000 7fe21d800640 1 -- 172.21.15.86:0/941559176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2180d08c0 msgr2=0x7fe218150550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:37.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.920+0000 7fe21d800640 1 --2- 172.21.15.86:0/941559176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2180d08c0 0x7fe218150550 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7fe218073c00 tx=0x7fe20400c820 comp rx=0 tx=0).stop 2024-04-25T14:15:37.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.920+0000 7fe21d800640 1 -- 172.21.15.86:0/941559176 shutdown_connections 2024-04-25T14:15:37.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.920+0000 7fe21d800640 1 --2- 172.21.15.86:0/941559176 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe1f806f480 0x7fe1f8071990 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:37.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.920+0000 7fe21d800640 1 --2- 172.21.15.86:0/941559176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2181635a0 0x7fe21813c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:37.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.920+0000 7fe21d800640 1 --2- 172.21.15.86:0/941559176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2180d08c0 0x7fe218150550 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:37.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.920+0000 7fe21d800640 1 --2- 172.21.15.86:0/941559176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe21806f060 0x7fe218150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:37.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.920+0000 7fe21d800640 1 -- 172.21.15.86:0/941559176 >> 172.21.15.86:0/941559176 conn(0x7fe2180b4f80 msgr2=0x7fe2180d3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:37.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.920+0000 7fe21d800640 1 -- 172.21.15.86:0/941559176 shutdown_connections 2024-04-25T14:15:37.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:37.920+0000 7fe21d800640 1 -- 172.21.15.86:0/941559176 wait complete. 2024-04-25T14:15:38.354 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:38.354 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:39.355 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:39.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.516+0000 7effc1a00640 1 Processor -- start 2024-04-25T14:15:39.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.516+0000 7effc1a00640 1 -- start start 2024-04-25T14:15:39.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.520+0000 7effc1a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effbc0ce360 0x7effbc162090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:39.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.520+0000 7effc1a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effbc1625d0 con 0x7effbc0d2380 2024-04-25T14:15:39.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.520+0000 7effc1a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effbc0d15c0 con 0x7effbc0700a0 2024-04-25T14:15:39.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.520+0000 7effc1a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7effbc0d18a0 con 0x7effbc0ce360 2024-04-25T14:15:39.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.520+0000 7effb3400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7effbc0d2380 0x7effbc06f990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40772/0 (socket says 172.21.15.86:40772) 2024-04-25T14:15:39.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.520+0000 7effb3400640 1 -- 172.21.15.86:0/3973460678 learned_addr learned my addr 172.21.15.86:0/3973460678 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:15:39.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.520+0000 7effbbe00640 1 --2- 172.21.15.86:0/3973460678 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effbc0ce360 0x7effbc162090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:39.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.520+0000 7effbb400640 1 -- 172.21.15.86:0/3973460678 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 828506387 0 0) 0x7effbc1625d0 con 0x7effbc0d2380 2024-04-25T14:15:39.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.520+0000 7effbb400640 1 -- 172.21.15.86:0/3973460678 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7effa0003530 con 0x7effbc0d2380 2024-04-25T14:15:39.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.520+0000 7effbb400640 1 -- 172.21.15.86:0/3973460678 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4187289453 0 0) 0x7effbc0d15c0 con 0x7effbc0700a0 2024-04-25T14:15:39.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.520+0000 7effbb400640 1 -- 172.21.15.86:0/3973460678 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7effbc1625d0 con 0x7effbc0700a0 2024-04-25T14:15:39.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.520+0000 7effbbe00640 1 -- 172.21.15.86:0/3973460678 >> v1:172.21.15.155:6789/0 conn(0x7effbc0700a0 legacy=0x7effbc0b63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:39.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.520+0000 7effbbe00640 1 -- 172.21.15.86:0/3973460678 >> v1:172.21.15.86:6789/0 conn(0x7effbc0d2380 legacy=0x7effbc06f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:39.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.520+0000 7effbbe00640 1 -- 172.21.15.86:0/3973460678 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effbc0d1b80 con 0x7effbc0ce360 2024-04-25T14:15:39.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.524+0000 7effbbe00640 1 --2- 172.21.15.86:0/3973460678 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effbc0ce360 0x7effbc162090 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7effac009dc0 tx=0x7effac01f170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80b9ef7bdb4dbbc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:39.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.524+0000 7effbb400640 1 -- 172.21.15.86:0/3973460678 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac0b4070 con 0x7effbc0ce360 2024-04-25T14:15:39.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.524+0000 7effbb400640 1 -- 172.21.15.86:0/3973460678 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effac0af600 con 0x7effbc0ce360 2024-04-25T14:15:39.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.524+0000 7effbb400640 1 -- 172.21.15.86:0/3973460678 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac0af9d0 con 0x7effbc0ce360 2024-04-25T14:15:39.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.524+0000 7effc1a00640 1 -- 172.21.15.86:0/3973460678 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effbc0ce360 msgr2=0x7effbc162090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:39.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.524+0000 7effc1a00640 1 --2- 172.21.15.86:0/3973460678 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effbc0ce360 0x7effbc162090 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7effac009dc0 tx=0x7effac01f170 comp rx=0 tx=0).stop 2024-04-25T14:15:39.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.524+0000 7effc1a00640 1 -- 172.21.15.86:0/3973460678 shutdown_connections 2024-04-25T14:15:39.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.524+0000 7effc1a00640 1 --2- 172.21.15.86:0/3973460678 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effbc0ce360 0x7effbc162090 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:39.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.524+0000 7effc1a00640 1 -- 172.21.15.86:0/3973460678 >> 172.21.15.86:0/3973460678 conn(0x7effbc0cfea0 msgr2=0x7effbc0d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:39.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.524+0000 7effc1a00640 1 -- 172.21.15.86:0/3973460678 shutdown_connections 2024-04-25T14:15:39.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.524+0000 7effc1a00640 1 -- 172.21.15.86:0/3973460678 wait complete. 2024-04-25T14:15:39.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.528+0000 7effc1a00640 1 Processor -- start 2024-04-25T14:15:39.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.528+0000 7effc1a00640 1 -- start start 2024-04-25T14:15:39.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.528+0000 7effc1a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effbc0ce360 0x7effbc0cb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:39.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.528+0000 7effc1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effbc0d2380 0x7effbc0b2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:39.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.528+0000 7effc1a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effbc0b66d0 0x7effbc160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:39.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.528+0000 7effc1a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7effbc0d2790 con 0x7effbc0d2380 2024-04-25T14:15:39.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.528+0000 7effc1a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7effbc0ce960 con 0x7effbc0ce360 2024-04-25T14:15:39.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.528+0000 7effc1a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7effbc0d2910 con 0x7effbc0b66d0 2024-04-25T14:15:39.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.528+0000 7effb3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effbc0d2380 0x7effbc0b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:39.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.528+0000 7effb3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effbc0d2380 0x7effbc0b2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35892/0 (socket says 172.21.15.86:35892) 2024-04-25T14:15:39.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.528+0000 7effb3400640 1 -- 172.21.15.86:0/3553171072 learned_addr learned my addr 172.21.15.86:0/3553171072 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:39.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.528+0000 7effbbe00640 1 --2- 172.21.15.86:0/3553171072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effbc0ce360 0x7effbc0cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:39.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.528+0000 7effc1000640 1 --2- 172.21.15.86:0/3553171072 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effbc0b66d0 0x7effbc160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:39.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.528+0000 7effb3400640 1 -- 172.21.15.86:0/3553171072 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effbc0b66d0 msgr2=0x7effbc160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:39.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.528+0000 7effb3400640 1 --2- 172.21.15.86:0/3553171072 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effbc0b66d0 0x7effbc160280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:39.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.528+0000 7effb3400640 1 -- 172.21.15.86:0/3553171072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effbc0ce360 msgr2=0x7effbc0cb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:39.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.528+0000 7effb3400640 1 --2- 172.21.15.86:0/3553171072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effbc0ce360 0x7effbc0cb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:39.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.528+0000 7effb3400640 1 -- 172.21.15.86:0/3553171072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effbc15f200 con 0x7effbc0d2380 2024-04-25T14:15:39.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.528+0000 7effc1000640 1 --2- 172.21.15.86:0/3553171072 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effbc0b66d0 0x7effbc160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:15:39.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.528+0000 7effb3400640 1 --2- 172.21.15.86:0/3553171072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effbc0d2380 0x7effbc0b2030 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7effa4001ec0 tx=0x7effa400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:39.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.528+0000 7effb8c00640 1 -- 172.21.15.86:0/3553171072 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa4017070 con 0x7effbc0d2380 2024-04-25T14:15:39.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.528+0000 7effb8c00640 1 -- 172.21.15.86:0/3553171072 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effa40074e0 con 0x7effbc0d2380 2024-04-25T14:15:39.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.528+0000 7effc1a00640 1 -- 172.21.15.86:0/3553171072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effbc15f4c0 con 0x7effbc0d2380 2024-04-25T14:15:39.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.528+0000 7effb8c00640 1 -- 172.21.15.86:0/3553171072 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa40059b0 con 0x7effbc0d2380 2024-04-25T14:15:39.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.528+0000 7effc1a00640 1 -- 172.21.15.86:0/3553171072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effbc15fa80 con 0x7effbc0d2380 2024-04-25T14:15:39.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.532+0000 7effc1a00640 1 -- 172.21.15.86:0/3553171072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effbc0b6f40 con 0x7effbc0d2380 2024-04-25T14:15:39.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.532+0000 7effb8c00640 1 -- 172.21.15.86:0/3553171072 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7effa401a070 con 0x7effbc0d2380 2024-04-25T14:15:39.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.532+0000 7effb8c00640 1 --2- 172.21.15.86:0/3553171072 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7eff9806f450 0x7eff98071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:39.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.532+0000 7effb8c00640 1 -- 172.21.15.86:0/3553171072 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7effa408e800 con 0x7effbc0d2380 2024-04-25T14:15:39.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.532+0000 7effbbe00640 1 --2- 172.21.15.86:0/3553171072 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7eff9806f450 0x7eff98071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:39.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.532+0000 7effbbe00640 1 --2- 172.21.15.86:0/3553171072 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7eff9806f450 0x7eff98071960 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7effac0b1000 tx=0x7effac0aa000 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:39.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.536+0000 7effb8c00640 1 -- 172.21.15.86:0/3553171072 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7effa405ae40 con 0x7effbc0d2380 2024-04-25T14:15:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.724+0000 7effc1a00640 1 -- 172.21.15.86:0/3553171072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effbc076b80 con 0x7effbc0d2380 2024-04-25T14:15:39.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.728+0000 7effb8c00640 1 -- 172.21.15.86:0/3553171072 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7effa405f140 con 0x7effbc0d2380 2024-04-25T14:15:39.724 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:39.724 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:39.724 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.728+0000 7effc1a00640 1 -- 172.21.15.86:0/3553171072 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7eff9806f450 msgr2=0x7eff98071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.728+0000 7effc1a00640 1 --2- 172.21.15.86:0/3553171072 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7eff9806f450 0x7eff98071960 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7effac0b1000 tx=0x7effac0aa000 comp rx=0 tx=0).stop 2024-04-25T14:15:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.728+0000 7effc1a00640 1 -- 172.21.15.86:0/3553171072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effbc0d2380 msgr2=0x7effbc0b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.728+0000 7effc1a00640 1 --2- 172.21.15.86:0/3553171072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effbc0d2380 0x7effbc0b2030 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7effa4001ec0 tx=0x7effa400c8e0 comp rx=0 tx=0).stop 2024-04-25T14:15:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.728+0000 7effc1a00640 1 -- 172.21.15.86:0/3553171072 shutdown_connections 2024-04-25T14:15:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.728+0000 7effc1a00640 1 --2- 172.21.15.86:0/3553171072 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7eff9806f450 0x7eff98071960 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.728+0000 7effc1a00640 1 --2- 172.21.15.86:0/3553171072 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effbc0b66d0 0x7effbc160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.728+0000 7effc1a00640 1 --2- 172.21.15.86:0/3553171072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effbc0d2380 0x7effbc0b2030 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.728+0000 7effc1a00640 1 --2- 172.21.15.86:0/3553171072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effbc0ce360 0x7effbc0cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.728+0000 7effc1a00640 1 -- 172.21.15.86:0/3553171072 >> 172.21.15.86:0/3553171072 conn(0x7effbc0cfea0 msgr2=0x7effbc0d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:39.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.728+0000 7effc1a00640 1 -- 172.21.15.86:0/3553171072 shutdown_connections 2024-04-25T14:15:39.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:39.728+0000 7effc1a00640 1 -- 172.21.15.86:0/3553171072 wait complete. 2024-04-25T14:15:40.148 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:40.148 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:41.150 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:41.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.312+0000 7f587da00640 1 Processor -- start 2024-04-25T14:15:41.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.312+0000 7f587da00640 1 -- start start 2024-04-25T14:15:41.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.312+0000 7f587da00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5878070330 0x7f58780d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:41.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.312+0000 7f587da00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f58780ce790 0x7f587806f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:41.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.312+0000 7f587da00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58780d2030 con 0x7f58780d1c20 2024-04-25T14:15:41.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.312+0000 7f587da00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f587815e5c0 con 0x7f58780ce790 2024-04-25T14:15:41.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.312+0000 7f587da00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f587815e8a0 con 0x7f5878070330 2024-04-25T14:15:41.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.312+0000 7f5877400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f58780ce790 0x7f587806f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:41.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.312+0000 7f5877400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f58780ce790 0x7f587806f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35896/0 (socket says 172.21.15.86:35896) 2024-04-25T14:15:41.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.312+0000 7f5877400640 1 -- 172.21.15.86:0/3661503493 learned_addr learned my addr 172.21.15.86:0/3661503493 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:41.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.316+0000 7f5876a00640 1 --2- 172.21.15.86:0/3661503493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5878070330 0x7f58780d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:41.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.316+0000 7f5877400640 1 -- 172.21.15.86:0/3661503493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5878070330 msgr2=0x7f58780d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:41.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.316+0000 7f5877400640 1 --2- 172.21.15.86:0/3661503493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5878070330 0x7f58780d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:41.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.316+0000 7f5877400640 1 -- 172.21.15.86:0/3661503493 >> v1:172.21.15.86:6789/0 conn(0x7f58780d1c20 legacy=0x7f587806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:41.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.316+0000 7f5877400640 1 -- 172.21.15.86:0/3661503493 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587815eb80 con 0x7f58780ce790 2024-04-25T14:15:41.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.316+0000 7f5876a00640 1 --2- 172.21.15.86:0/3661503493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5878070330 0x7f58780d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:15:41.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.316+0000 7f5877400640 1 --2- 172.21.15.86:0/3661503493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58780ce790 0x7f587806f440 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f5870009e80 tx=0x7f587001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9cf4578c67ad25b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:41.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.316+0000 7f5876000640 1 -- 172.21.15.86:0/3661503493 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58700a5640 con 0x7f58780ce790 2024-04-25T14:15:41.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.316+0000 7f5876000640 1 -- 172.21.15.86:0/3661503493 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58700a57e0 con 0x7f58780ce790 2024-04-25T14:15:41.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.316+0000 7f5876000640 1 -- 172.21.15.86:0/3661503493 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58700a5bb0 con 0x7f58780ce790 2024-04-25T14:15:41.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.316+0000 7f587da00640 1 -- 172.21.15.86:0/3661503493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58780ce790 msgr2=0x7f587806f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:41.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.320+0000 7f587da00640 1 --2- 172.21.15.86:0/3661503493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58780ce790 0x7f587806f440 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f5870009e80 tx=0x7f587001c980 comp rx=0 tx=0).stop 2024-04-25T14:15:41.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.320+0000 7f587da00640 1 -- 172.21.15.86:0/3661503493 shutdown_connections 2024-04-25T14:15:41.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.320+0000 7f587da00640 1 --2- 172.21.15.86:0/3661503493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58780ce790 0x7f587806f440 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:41.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.320+0000 7f587da00640 1 --2- 172.21.15.86:0/3661503493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5878070330 0x7f58780d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:41.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.320+0000 7f587da00640 1 -- 172.21.15.86:0/3661503493 >> 172.21.15.86:0/3661503493 conn(0x7f58780b4650 msgr2=0x7f58780b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:41.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.320+0000 7f587da00640 1 -- 172.21.15.86:0/3661503493 shutdown_connections 2024-04-25T14:15:41.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.320+0000 7f587da00640 1 -- 172.21.15.86:0/3661503493 wait complete. 2024-04-25T14:15:41.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.320+0000 7f587da00640 1 Processor -- start 2024-04-25T14:15:41.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.320+0000 7f587da00640 1 -- start start 2024-04-25T14:15:41.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.320+0000 7f587da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5878070330 0x7f58780cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:41.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.324+0000 7f587da00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58780ce790 0x7f58780d0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:41.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.324+0000 7f587da00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58780d1c20 0x7f58780c4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:41.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.324+0000 7f587da00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f58780708c0 con 0x7f5878070330 2024-04-25T14:15:41.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.324+0000 7f587da00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5878070740 con 0x7f58780d1c20 2024-04-25T14:15:41.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.324+0000 7f587da00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f587816afc0 con 0x7f58780ce790 2024-04-25T14:15:41.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.324+0000 7f5876a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5878070330 0x7f58780cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:41.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.324+0000 7f5876a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5878070330 0x7f58780cfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35906/0 (socket says 172.21.15.86:35906) 2024-04-25T14:15:41.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.324+0000 7f5876a00640 1 -- 172.21.15.86:0/552477397 learned_addr learned my addr 172.21.15.86:0/552477397 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:41.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.324+0000 7f586e000640 1 --2- 172.21.15.86:0/552477397 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58780ce790 0x7f58780d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:41.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.324+0000 7f5877400640 1 --2- 172.21.15.86:0/552477397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58780d1c20 0x7f58780c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:41.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.324+0000 7f5876a00640 1 -- 172.21.15.86:0/552477397 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58780ce790 msgr2=0x7f58780d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:41.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.324+0000 7f5876a00640 1 --2- 172.21.15.86:0/552477397 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58780ce790 0x7f58780d0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:41.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.324+0000 7f5876a00640 1 -- 172.21.15.86:0/552477397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58780d1c20 msgr2=0x7f58780c4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:41.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.324+0000 7f5876a00640 1 --2- 172.21.15.86:0/552477397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58780d1c20 0x7f58780c4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:41.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.324+0000 7f5876a00640 1 -- 172.21.15.86:0/552477397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5878072920 con 0x7f5878070330 2024-04-25T14:15:41.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.324+0000 7f5876a00640 1 --2- 172.21.15.86:0/552477397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5878070330 0x7f58780cfb00 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f586800c820 tx=0x7f586800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:41.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.324+0000 7f586f400640 1 -- 172.21.15.86:0/552477397 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868015070 con 0x7f5878070330 2024-04-25T14:15:41.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.324+0000 7f586f400640 1 -- 172.21.15.86:0/552477397 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5868007a30 con 0x7f5878070330 2024-04-25T14:15:41.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.324+0000 7f586f400640 1 -- 172.21.15.86:0/552477397 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868007de0 con 0x7f5878070330 2024-04-25T14:15:41.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.324+0000 7f587da00640 1 -- 172.21.15.86:0/552477397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5878072c00 con 0x7f5878070330 2024-04-25T14:15:41.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.324+0000 7f587da00640 1 -- 172.21.15.86:0/552477397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58780c6920 con 0x7f5878070330 2024-04-25T14:15:41.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.324+0000 7f587da00640 1 -- 172.21.15.86:0/552477397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5878069d00 con 0x7f5878070330 2024-04-25T14:15:41.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.324+0000 7f586f400640 1 -- 172.21.15.86:0/552477397 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f5868018070 con 0x7f5878070330 2024-04-25T14:15:41.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.328+0000 7f586f400640 1 --2- 172.21.15.86:0/552477397 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f585406f480 0x7f5854071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:41.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.328+0000 7f586f400640 1 -- 172.21.15.86:0/552477397 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f586808f460 con 0x7f5878070330 2024-04-25T14:15:41.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.328+0000 7f586e000640 1 --2- 172.21.15.86:0/552477397 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f585406f480 0x7f5854071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:41.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.328+0000 7f586e000640 1 --2- 172.21.15.86:0/552477397 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f585406f480 0x7f5854071990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f58780b14f0 tx=0x7f5860009040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:41.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.332+0000 7f586f400640 1 -- 172.21.15.86:0/552477397 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f586805c0f0 con 0x7f5878070330 2024-04-25T14:15:41.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.524+0000 7f587da00640 1 -- 172.21.15.86:0/552477397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5878068f70 con 0x7f5878070330 2024-04-25T14:15:41.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.524+0000 7f586f400640 1 -- 172.21.15.86:0/552477397 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f586805b9f0 con 0x7f5878070330 2024-04-25T14:15:41.523 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:41.523 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:41.524 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:41.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.528+0000 7f587da00640 1 -- 172.21.15.86:0/552477397 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f585406f480 msgr2=0x7f5854071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.528+0000 7f587da00640 1 --2- 172.21.15.86:0/552477397 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f585406f480 0x7f5854071990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f58780b14f0 tx=0x7f5860009040 comp rx=0 tx=0).stop 2024-04-25T14:15:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.528+0000 7f587da00640 1 -- 172.21.15.86:0/552477397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5878070330 msgr2=0x7f58780cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.528+0000 7f587da00640 1 --2- 172.21.15.86:0/552477397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5878070330 0x7f58780cfb00 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f586800c820 tx=0x7f586800cce0 comp rx=0 tx=0).stop 2024-04-25T14:15:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.528+0000 7f587da00640 1 -- 172.21.15.86:0/552477397 shutdown_connections 2024-04-25T14:15:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.528+0000 7f587da00640 1 --2- 172.21.15.86:0/552477397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58780d1c20 0x7f58780c4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.528+0000 7f587da00640 1 --2- 172.21.15.86:0/552477397 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f585406f480 0x7f5854071990 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.528+0000 7f587da00640 1 --2- 172.21.15.86:0/552477397 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58780ce790 0x7f58780d0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.528+0000 7f587da00640 1 --2- 172.21.15.86:0/552477397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5878070330 0x7f58780cfb00 unknown :-1 s=CLOSED pgs=2230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.528+0000 7f587da00640 1 -- 172.21.15.86:0/552477397 >> 172.21.15.86:0/552477397 conn(0x7f58780b4650 msgr2=0x7f587815b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.528+0000 7f587da00640 1 -- 172.21.15.86:0/552477397 shutdown_connections 2024-04-25T14:15:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:41.528+0000 7f587da00640 1 -- 172.21.15.86:0/552477397 wait complete. 2024-04-25T14:15:41.948 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:41.948 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:42.949 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:43.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.100+0000 7f1a96800640 1 Processor -- start 2024-04-25T14:15:43.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.100+0000 7f1a96800640 1 -- start start 2024-04-25T14:15:43.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.104+0000 7f1a96800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a90070330 0x7f1a900d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:43.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.104+0000 7f1a96800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1a900d1c20 0x7f1a9006ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:43.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.104+0000 7f1a96800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1a9006f270 0x7f1a9006f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:43.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.104+0000 7f1a96800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1a900d2030 con 0x7f1a900d1c20 2024-04-25T14:15:43.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.104+0000 7f1a96800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1a9015e5c0 con 0x7f1a9006f270 2024-04-25T14:15:43.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.104+0000 7f1a96800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1a9015e8a0 con 0x7f1a90070330 2024-04-25T14:15:43.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.104+0000 7f1a86a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1a900d1c20 0x7f1a9006ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:43.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.104+0000 7f1a86a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1a900d1c20 0x7f1a9006ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35914/0 (socket says 172.21.15.86:35914) 2024-04-25T14:15:43.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.104+0000 7f1a86a00640 1 -- 172.21.15.86:0/3228475007 learned_addr learned my addr 172.21.15.86:0/3228475007 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:43.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.104+0000 7f1a8f400640 1 --2- 172.21.15.86:0/3228475007 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a90070330 0x7f1a900d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:43.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.104+0000 7f1a8fe00640 1 --2- 172.21.15.86:0/3228475007 >> v2:172.21.15.155:3300/0 conn(0x7f1a9006f270 0x7f1a9006f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:43.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.104+0000 7f1a86a00640 1 -- 172.21.15.86:0/3228475007 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a90070330 msgr2=0x7f1a900d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:43.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.104+0000 7f1a86a00640 1 --2- 172.21.15.86:0/3228475007 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a90070330 0x7f1a900d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:43.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.104+0000 7f1a86a00640 1 -- 172.21.15.86:0/3228475007 >> v2:172.21.15.155:3300/0 conn(0x7f1a9006f270 msgr2=0x7f1a9006f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:43.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.104+0000 7f1a86a00640 1 --2- 172.21.15.86:0/3228475007 >> v2:172.21.15.155:3300/0 conn(0x7f1a9006f270 0x7f1a9006f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:43.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.104+0000 7f1a86a00640 1 -- 172.21.15.86:0/3228475007 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a9015eb80 con 0x7f1a900d1c20 2024-04-25T14:15:43.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.104+0000 7f1a8fe00640 1 --2- 172.21.15.86:0/3228475007 >> v2:172.21.15.155:3300/0 conn(0x7f1a9006f270 0x7f1a9006f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:43.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.104+0000 7f1a8f400640 1 --2- 172.21.15.86:0/3228475007 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a90070330 0x7f1a900d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:43.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.108+0000 7f1a86a00640 1 --2- 172.21.15.86:0/3228475007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a900d1c20 0x7f1a9006ed30 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f1a80009870 tx=0x7f1a8001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba69f87a1cece75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:43.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.108+0000 7f1a8ea00640 1 -- 172.21.15.86:0/3228475007 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a800ab600 con 0x7f1a900d1c20 2024-04-25T14:15:43.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.108+0000 7f1a8ea00640 1 -- 172.21.15.86:0/3228475007 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a800b0070 con 0x7f1a900d1c20 2024-04-25T14:15:43.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.108+0000 7f1a8ea00640 1 -- 172.21.15.86:0/3228475007 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a800ab9c0 con 0x7f1a900d1c20 2024-04-25T14:15:43.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.108+0000 7f1a96800640 1 -- 172.21.15.86:0/3228475007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a900d1c20 msgr2=0x7f1a9006ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:43.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.108+0000 7f1a96800640 1 --2- 172.21.15.86:0/3228475007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a900d1c20 0x7f1a9006ed30 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f1a80009870 tx=0x7f1a8001c460 comp rx=0 tx=0).stop 2024-04-25T14:15:43.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.108+0000 7f1a96800640 1 -- 172.21.15.86:0/3228475007 shutdown_connections 2024-04-25T14:15:43.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.108+0000 7f1a96800640 1 --2- 172.21.15.86:0/3228475007 >> v2:172.21.15.155:3300/0 conn(0x7f1a9006f270 0x7f1a9006f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:43.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.108+0000 7f1a96800640 1 --2- 172.21.15.86:0/3228475007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a900d1c20 0x7f1a9006ed30 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:43.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.108+0000 7f1a96800640 1 --2- 172.21.15.86:0/3228475007 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a90070330 0x7f1a900d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:43.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.108+0000 7f1a96800640 1 -- 172.21.15.86:0/3228475007 >> 172.21.15.86:0/3228475007 conn(0x7f1a900b4650 msgr2=0x7f1a900b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:43.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.108+0000 7f1a96800640 1 -- 172.21.15.86:0/3228475007 shutdown_connections 2024-04-25T14:15:43.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.108+0000 7f1a96800640 1 -- 172.21.15.86:0/3228475007 wait complete. 2024-04-25T14:15:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.108+0000 7f1a96800640 1 Processor -- start 2024-04-25T14:15:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.112+0000 7f1a96800640 1 -- start start 2024-04-25T14:15:43.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.112+0000 7f1a96800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a9006f270 0x7f1a90158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:43.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.112+0000 7f1a96800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a90070330 0x7f1a900c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:43.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.112+0000 7f1a96800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a900d1c20 0x7f1a900c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:43.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.112+0000 7f1a96800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9015f4b0 con 0x7f1a90070330 2024-04-25T14:15:43.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.112+0000 7f1a96800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9015f330 con 0x7f1a900d1c20 2024-04-25T14:15:43.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.112+0000 7f1a96800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1a90070740 con 0x7f1a9006f270 2024-04-25T14:15:43.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.112+0000 7f1a86a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a90070330 0x7f1a900c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:43.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.112+0000 7f1a86a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a90070330 0x7f1a900c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35916/0 (socket says 172.21.15.86:35916) 2024-04-25T14:15:43.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.112+0000 7f1a86a00640 1 -- 172.21.15.86:0/1135519184 learned_addr learned my addr 172.21.15.86:0/1135519184 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:43.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.112+0000 7f1a8f400640 1 --2- 172.21.15.86:0/1135519184 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a9006f270 0x7f1a90158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:43.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.112+0000 7f1a8fe00640 1 --2- 172.21.15.86:0/1135519184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a900d1c20 0x7f1a900c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:43.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.112+0000 7f1a86a00640 1 -- 172.21.15.86:0/1135519184 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a9006f270 msgr2=0x7f1a90158b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:43.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.112+0000 7f1a86a00640 1 --2- 172.21.15.86:0/1135519184 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a9006f270 0x7f1a90158b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:43.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.112+0000 7f1a86a00640 1 -- 172.21.15.86:0/1135519184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a900d1c20 msgr2=0x7f1a900c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:43.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.112+0000 7f1a86a00640 1 --2- 172.21.15.86:0/1135519184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a900d1c20 0x7f1a900c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.112+0000 7f1a86a00640 1 -- 172.21.15.86:0/1135519184 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a900c5f40 con 0x7f1a90070330 2024-04-25T14:15:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.112+0000 7f1a8fe00640 1 --2- 172.21.15.86:0/1135519184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a900d1c20 0x7f1a900c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.112+0000 7f1a86a00640 1 --2- 172.21.15.86:0/1135519184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a90070330 0x7f1a900c8180 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f1a8001ab10 tx=0x7f1a80019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.112+0000 7f1a87e00640 1 -- 172.21.15.86:0/1135519184 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a800bf610 con 0x7f1a90070330 2024-04-25T14:15:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.112+0000 7f1a87e00640 1 -- 172.21.15.86:0/1135519184 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a800bf7d0 con 0x7f1a90070330 2024-04-25T14:15:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.112+0000 7f1a87e00640 1 -- 172.21.15.86:0/1135519184 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a800bfbc0 con 0x7f1a90070330 2024-04-25T14:15:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.112+0000 7f1a96800640 1 -- 172.21.15.86:0/1135519184 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a90072920 con 0x7f1a90070330 2024-04-25T14:15:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.112+0000 7f1a96800640 1 -- 172.21.15.86:0/1135519184 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a90072ee0 con 0x7f1a90070330 2024-04-25T14:15:43.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.112+0000 7f1a96800640 1 -- 172.21.15.86:0/1135519184 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a90069d00 con 0x7f1a90070330 2024-04-25T14:15:43.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.116+0000 7f1a87e00640 1 -- 172.21.15.86:0/1135519184 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f1a800bd020 con 0x7f1a90070330 2024-04-25T14:15:43.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.116+0000 7f1a87e00640 1 --2- 172.21.15.86:0/1135519184 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1a6c06f480 0x7f1a6c071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:43.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.116+0000 7f1a87e00640 1 -- 172.21.15.86:0/1135519184 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f1a80129360 con 0x7f1a90070330 2024-04-25T14:15:43.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.116+0000 7f1a8f400640 1 --2- 172.21.15.86:0/1135519184 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1a6c06f480 0x7f1a6c071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:43.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.116+0000 7f1a8f400640 1 --2- 172.21.15.86:0/1135519184 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1a6c06f480 0x7f1a6c071990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1a90072200 tx=0x7f1a78008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:43.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.120+0000 7f1a87e00640 1 -- 172.21.15.86:0/1135519184 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1a800c6020 con 0x7f1a90070330 2024-04-25T14:15:43.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.312+0000 7f1a96800640 1 -- 172.21.15.86:0/1135519184 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a90076bf0 con 0x7f1a90070330 2024-04-25T14:15:43.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.312+0000 7f1a87e00640 1 -- 172.21.15.86:0/1135519184 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f1a80016110 con 0x7f1a90070330 2024-04-25T14:15:43.308 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:43.309 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:43.309 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:43.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.312+0000 7f1a96800640 1 -- 172.21.15.86:0/1135519184 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1a6c06f480 msgr2=0x7f1a6c071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:43.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.312+0000 7f1a96800640 1 --2- 172.21.15.86:0/1135519184 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1a6c06f480 0x7f1a6c071990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1a90072200 tx=0x7f1a78008040 comp rx=0 tx=0).stop 2024-04-25T14:15:43.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.312+0000 7f1a96800640 1 -- 172.21.15.86:0/1135519184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a90070330 msgr2=0x7f1a900c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:43.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.312+0000 7f1a96800640 1 --2- 172.21.15.86:0/1135519184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a90070330 0x7f1a900c8180 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f1a8001ab10 tx=0x7f1a80019200 comp rx=0 tx=0).stop 2024-04-25T14:15:43.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.312+0000 7f1a96800640 1 -- 172.21.15.86:0/1135519184 shutdown_connections 2024-04-25T14:15:43.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.312+0000 7f1a96800640 1 --2- 172.21.15.86:0/1135519184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a900d1c20 0x7f1a900c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:43.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.312+0000 7f1a96800640 1 --2- 172.21.15.86:0/1135519184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a90070330 0x7f1a900c8180 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:43.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.312+0000 7f1a96800640 1 --2- 172.21.15.86:0/1135519184 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1a6c06f480 0x7f1a6c071990 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:43.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.312+0000 7f1a96800640 1 --2- 172.21.15.86:0/1135519184 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a9006f270 0x7f1a90158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:43.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.312+0000 7f1a96800640 1 -- 172.21.15.86:0/1135519184 >> 172.21.15.86:0/1135519184 conn(0x7f1a900b4650 msgr2=0x7f1a900cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:43.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.312+0000 7f1a96800640 1 -- 172.21.15.86:0/1135519184 shutdown_connections 2024-04-25T14:15:43.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:43.312+0000 7f1a96800640 1 -- 172.21.15.86:0/1135519184 wait complete. 2024-04-25T14:15:43.734 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:43.734 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:44.736 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:44.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.900+0000 7f6aaf400640 1 Processor -- start 2024-04-25T14:15:44.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.900+0000 7f6aaf400640 1 -- start start 2024-04-25T14:15:44.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.900+0000 7f6aaf400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6aa80740c0 0x7f6aa81676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:44.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.900+0000 7f6aaf400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6aa8072bd0 con 0x7f6aa80cfca0 2024-04-25T14:15:44.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.900+0000 7f6aaf400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6aa806fbd0 con 0x7f6aa80d16e0 2024-04-25T14:15:44.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.900+0000 7f6aaf400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6aa8070470 con 0x7f6aa80740c0 2024-04-25T14:15:44.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.900+0000 7f6aad400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6aa80740c0 0x7f6aa81676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:44.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.900+0000 7f6aad400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6aa80740c0 0x7f6aa81676d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35928/0 (socket says 172.21.15.86:35928) 2024-04-25T14:15:44.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.900+0000 7f6aad400640 1 -- 172.21.15.86:0/3939974608 learned_addr learned my addr 172.21.15.86:0/3939974608 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:44.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.900+0000 7f6aad400640 1 -- 172.21.15.86:0/3939974608 >> v1:172.21.15.155:6789/0 conn(0x7f6aa80d16e0 legacy=0x7f6aa80d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:44.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.900+0000 7f6aad400640 1 -- 172.21.15.86:0/3939974608 >> v1:172.21.15.86:6789/0 conn(0x7f6aa80cfca0 legacy=0x7f6aa806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:44.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.900+0000 7f6aad400640 1 -- 172.21.15.86:0/3939974608 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa806feb0 con 0x7f6aa80740c0 2024-04-25T14:15:44.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.904+0000 7f6aad400640 1 --2- 172.21.15.86:0/3939974608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6aa80740c0 0x7f6aa81676d0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009e50 tx=0x7f6aa001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4bf61df8e6db4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:44.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.904+0000 7f6aa7e00640 1 -- 172.21.15.86:0/3939974608 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa00a5810 con 0x7f6aa80740c0 2024-04-25T14:15:44.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.904+0000 7f6aa7e00640 1 -- 172.21.15.86:0/3939974608 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aa00a59b0 con 0x7f6aa80740c0 2024-04-25T14:15:44.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.904+0000 7f6aa7e00640 1 -- 172.21.15.86:0/3939974608 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa00a5d80 con 0x7f6aa80740c0 2024-04-25T14:15:44.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.904+0000 7f6aaf400640 1 -- 172.21.15.86:0/3939974608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6aa80740c0 msgr2=0x7f6aa81676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:44.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.904+0000 7f6aaf400640 1 --2- 172.21.15.86:0/3939974608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6aa80740c0 0x7f6aa81676d0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009e50 tx=0x7f6aa001ca30 comp rx=0 tx=0).stop 2024-04-25T14:15:44.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.904+0000 7f6aaf400640 1 -- 172.21.15.86:0/3939974608 shutdown_connections 2024-04-25T14:15:44.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.904+0000 7f6aaf400640 1 --2- 172.21.15.86:0/3939974608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6aa80740c0 0x7f6aa81676d0 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:44.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.904+0000 7f6aaf400640 1 -- 172.21.15.86:0/3939974608 >> 172.21.15.86:0/3939974608 conn(0x7f6aa80b4650 msgr2=0x7f6aa80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:44.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.904+0000 7f6aaf400640 1 -- 172.21.15.86:0/3939974608 shutdown_connections 2024-04-25T14:15:44.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.904+0000 7f6aaf400640 1 -- 172.21.15.86:0/3939974608 wait complete. 2024-04-25T14:15:44.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.908+0000 7f6aaf400640 1 Processor -- start 2024-04-25T14:15:44.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.908+0000 7f6aaf400640 1 -- start start 2024-04-25T14:15:44.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.908+0000 7f6aaf400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6aa80740c0 0x7f6aa809a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:44.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.908+0000 7f6aaf400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6aa80cfca0 0x7f6aa815e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:44.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.908+0000 7f6aaf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6aa80d16e0 0x7f6aa815e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:44.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.908+0000 7f6aaf400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa816ade0 con 0x7f6aa80d16e0 2024-04-25T14:15:44.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.908+0000 7f6aaf400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa816ac60 con 0x7f6aa80cfca0 2024-04-25T14:15:44.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.908+0000 7f6aaf400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6aa816af60 con 0x7f6aa80740c0 2024-04-25T14:15:44.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.908+0000 7f6aad400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6aa80d16e0 0x7f6aa815e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:44.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.908+0000 7f6aad400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6aa80d16e0 0x7f6aa815e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35942/0 (socket says 172.21.15.86:35942) 2024-04-25T14:15:44.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.908+0000 7f6aad400640 1 -- 172.21.15.86:0/2304780743 learned_addr learned my addr 172.21.15.86:0/2304780743 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:44.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.908+0000 7f6a9fe00640 1 --2- 172.21.15.86:0/2304780743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6aa80cfca0 0x7f6aa815e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:44.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.908+0000 7f6aaca00640 1 --2- 172.21.15.86:0/2304780743 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6aa80740c0 0x7f6aa809a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:44.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.908+0000 7f6aad400640 1 -- 172.21.15.86:0/2304780743 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6aa80740c0 msgr2=0x7f6aa809a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:44.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.908+0000 7f6aad400640 1 --2- 172.21.15.86:0/2304780743 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6aa80740c0 0x7f6aa809a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:44.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.908+0000 7f6aad400640 1 -- 172.21.15.86:0/2304780743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6aa80cfca0 msgr2=0x7f6aa815e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:44.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.908+0000 7f6aad400640 1 --2- 172.21.15.86:0/2304780743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6aa80cfca0 0x7f6aa815e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:44.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.908+0000 7f6aad400640 1 -- 172.21.15.86:0/2304780743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa80cca90 con 0x7f6aa80d16e0 2024-04-25T14:15:44.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.908+0000 7f6aaca00640 1 --2- 172.21.15.86:0/2304780743 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6aa80740c0 0x7f6aa809a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:44.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.908+0000 7f6aad400640 1 --2- 172.21.15.86:0/2304780743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6aa80d16e0 0x7f6aa815e6b0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009e50 tx=0x7f6aa00197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:44.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.908+0000 7f6aa5600640 1 -- 172.21.15.86:0/2304780743 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa00b8670 con 0x7f6aa80d16e0 2024-04-25T14:15:44.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.908+0000 7f6aa5600640 1 -- 172.21.15.86:0/2304780743 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aa00be070 con 0x7f6aa80d16e0 2024-04-25T14:15:44.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.908+0000 7f6aa5600640 1 -- 172.21.15.86:0/2304780743 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa00b8a90 con 0x7f6aa80d16e0 2024-04-25T14:15:44.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.912+0000 7f6aaf400640 1 -- 172.21.15.86:0/2304780743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa80ccd70 con 0x7f6aa80d16e0 2024-04-25T14:15:44.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.912+0000 7f6aaf400640 1 -- 172.21.15.86:0/2304780743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa80cd330 con 0x7f6aa80d16e0 2024-04-25T14:15:44.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.912+0000 7f6aaf400640 1 -- 172.21.15.86:0/2304780743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aa809f4b0 con 0x7f6aa80d16e0 2024-04-25T14:15:44.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.912+0000 7f6aa5600640 1 -- 172.21.15.86:0/2304780743 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f6aa00b8c30 con 0x7f6aa80d16e0 2024-04-25T14:15:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.912+0000 7f6aa5600640 1 --2- 172.21.15.86:0/2304780743 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f6a8406f450 0x7f6a84071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.912+0000 7f6aa5600640 1 -- 172.21.15.86:0/2304780743 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f6aa012a6b0 con 0x7f6aa80d16e0 2024-04-25T14:15:44.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.912+0000 7f6aaca00640 1 --2- 172.21.15.86:0/2304780743 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f6a8406f450 0x7f6a84071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:44.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.916+0000 7f6aaca00640 1 --2- 172.21.15.86:0/2304780743 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f6a8406f450 0x7f6a84071960 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6a98000f70 tx=0x7f6a98009040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:44.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:44.916+0000 7f6aa5600640 1 -- 172.21.15.86:0/2304780743 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6aa0016620 con 0x7f6aa80d16e0 2024-04-25T14:15:45.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:45.112+0000 7f6aaf400640 1 -- 172.21.15.86:0/2304780743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6aa8069d00 con 0x7f6aa80d16e0 2024-04-25T14:15:45.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:45.112+0000 7f6aa5600640 1 -- 172.21.15.86:0/2304780743 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f6aa00f72c0 con 0x7f6aa80d16e0 2024-04-25T14:15:45.109 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:45.109 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:45.109 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:45.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:45.112+0000 7f6aaf400640 1 -- 172.21.15.86:0/2304780743 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f6a8406f450 msgr2=0x7f6a84071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:45.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:45.112+0000 7f6aaf400640 1 --2- 172.21.15.86:0/2304780743 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f6a8406f450 0x7f6a84071960 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6a98000f70 tx=0x7f6a98009040 comp rx=0 tx=0).stop 2024-04-25T14:15:45.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:45.112+0000 7f6aaf400640 1 -- 172.21.15.86:0/2304780743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6aa80d16e0 msgr2=0x7f6aa815e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:45.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:45.112+0000 7f6aaf400640 1 --2- 172.21.15.86:0/2304780743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6aa80d16e0 0x7f6aa815e6b0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009e50 tx=0x7f6aa00197d0 comp rx=0 tx=0).stop 2024-04-25T14:15:45.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:45.112+0000 7f6aaf400640 1 -- 172.21.15.86:0/2304780743 shutdown_connections 2024-04-25T14:15:45.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:45.112+0000 7f6aaf400640 1 --2- 172.21.15.86:0/2304780743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6aa80d16e0 0x7f6aa815e6b0 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:45.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:45.112+0000 7f6aaf400640 1 --2- 172.21.15.86:0/2304780743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6aa80cfca0 0x7f6aa815e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:45.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:45.112+0000 7f6aaf400640 1 --2- 172.21.15.86:0/2304780743 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f6a8406f450 0x7f6a84071960 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:45.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:45.112+0000 7f6aaf400640 1 --2- 172.21.15.86:0/2304780743 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6aa80740c0 0x7f6aa809a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:45.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:45.112+0000 7f6aaf400640 1 -- 172.21.15.86:0/2304780743 >> 172.21.15.86:0/2304780743 conn(0x7f6aa80b4650 msgr2=0x7f6aa80b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:45.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:45.112+0000 7f6aaf400640 1 -- 172.21.15.86:0/2304780743 shutdown_connections 2024-04-25T14:15:45.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:45.112+0000 7f6aaf400640 1 -- 172.21.15.86:0/2304780743 wait complete. 2024-04-25T14:15:45.534 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:45.534 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:46.535 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:46.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.696+0000 7fd64cc00640 1 Processor -- start 2024-04-25T14:15:46.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.696+0000 7fd64cc00640 1 -- start start 2024-04-25T14:15:46.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.696+0000 7fd64cc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd64815c2a0 0x7fd6480ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:46.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.696+0000 7fd64cc00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd64806fd50 0x7fd648070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:46.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.696+0000 7fd64cc00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6480b0a60 con 0x7fd64806c670 2024-04-25T14:15:46.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.696+0000 7fd64cc00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd64806ef90 con 0x7fd64806fd50 2024-04-25T14:15:46.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.696+0000 7fd64cc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd64806f270 con 0x7fd64815c2a0 2024-04-25T14:15:46.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.700+0000 7fd63d400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd64806fd50 0x7fd648070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:46.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.700+0000 7fd645e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd64815c2a0 0x7fd6480ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:46.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.700+0000 7fd646800640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fd64806c670 0x7fd64806ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:52808/0 (socket says 172.21.15.86:52808) 2024-04-25T14:15:46.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.700+0000 7fd646800640 1 -- 172.21.15.86:0/3199248382 learned_addr learned my addr 172.21.15.86:0/3199248382 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:15:46.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.700+0000 7fd645400640 1 -- 172.21.15.86:0/3199248382 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4043665961 0 0) 0x7fd6480b0a60 con 0x7fd64806c670 2024-04-25T14:15:46.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.700+0000 7fd645400640 1 -- 172.21.15.86:0/3199248382 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd62c005740 con 0x7fd64806c670 2024-04-25T14:15:46.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.700+0000 7fd63d400640 1 -- 172.21.15.86:0/3199248382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd64815c2a0 msgr2=0x7fd6480ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:46.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.700+0000 7fd63d400640 1 --2- 172.21.15.86:0/3199248382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd64815c2a0 0x7fd6480ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:46.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.700+0000 7fd63d400640 1 -- 172.21.15.86:0/3199248382 >> v1:172.21.15.155:6789/0 conn(0x7fd64806c670 legacy=0x7fd64806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:46.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.700+0000 7fd63d400640 1 -- 172.21.15.86:0/3199248382 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd64806f550 con 0x7fd64806fd50 2024-04-25T14:15:46.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.700+0000 7fd645e00640 1 --2- 172.21.15.86:0/3199248382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd64815c2a0 0x7fd6480ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:46.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.704+0000 7fd63d400640 1 --2- 172.21.15.86:0/3199248382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd64806fd50 0x7fd648070160 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fd63000a960 tx=0x7fd63001c6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29ec8c16aa10b04e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:46.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.704+0000 7fd645400640 1 -- 172.21.15.86:0/3199248382 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63000b040 con 0x7fd64806fd50 2024-04-25T14:15:46.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.704+0000 7fd645400640 1 -- 172.21.15.86:0/3199248382 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6300a66c0 con 0x7fd64806fd50 2024-04-25T14:15:46.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.704+0000 7fd645400640 1 -- 172.21.15.86:0/3199248382 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6300a6a70 con 0x7fd64806fd50 2024-04-25T14:15:46.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.704+0000 7fd64cc00640 1 -- 172.21.15.86:0/3199248382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd64806fd50 msgr2=0x7fd648070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:46.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.704+0000 7fd64cc00640 1 --2- 172.21.15.86:0/3199248382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd64806fd50 0x7fd648070160 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fd63000a960 tx=0x7fd63001c6b0 comp rx=0 tx=0).stop 2024-04-25T14:15:46.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.704+0000 7fd64cc00640 1 -- 172.21.15.86:0/3199248382 shutdown_connections 2024-04-25T14:15:46.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.704+0000 7fd64cc00640 1 --2- 172.21.15.86:0/3199248382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd64806fd50 0x7fd648070160 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:46.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.704+0000 7fd64cc00640 1 --2- 172.21.15.86:0/3199248382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd64815c2a0 0x7fd6480ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:46.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.704+0000 7fd64cc00640 1 -- 172.21.15.86:0/3199248382 >> 172.21.15.86:0/3199248382 conn(0x7fd6480b4650 msgr2=0x7fd6480b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:46.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.704+0000 7fd64cc00640 1 -- 172.21.15.86:0/3199248382 shutdown_connections 2024-04-25T14:15:46.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.704+0000 7fd64cc00640 1 -- 172.21.15.86:0/3199248382 wait complete. 2024-04-25T14:15:46.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.704+0000 7fd64cc00640 1 Processor -- start 2024-04-25T14:15:46.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.704+0000 7fd64cc00640 1 -- start start 2024-04-25T14:15:46.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.708+0000 7fd64cc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd64806c670 0x7fd6481649a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:46.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.708+0000 7fd64cc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd64806fd50 0x7fd6481fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:46.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.708+0000 7fd64cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd64815c2a0 0x7fd64815ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:46.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.708+0000 7fd64cc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd64815c830 con 0x7fd64815c2a0 2024-04-25T14:15:46.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.708+0000 7fd64cc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd64815c6b0 con 0x7fd64806fd50 2024-04-25T14:15:46.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.708+0000 7fd64cc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd6480ae670 con 0x7fd64806c670 2024-04-25T14:15:46.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.708+0000 7fd646800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd64815c2a0 0x7fd64815ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:46.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.708+0000 7fd646800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd64815c2a0 0x7fd64815ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35944/0 (socket says 172.21.15.86:35944) 2024-04-25T14:15:46.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.708+0000 7fd646800640 1 -- 172.21.15.86:0/1995907402 learned_addr learned my addr 172.21.15.86:0/1995907402 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:46.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.708+0000 7fd645e00640 1 --2- 172.21.15.86:0/1995907402 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd64806c670 0x7fd6481649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:46.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.708+0000 7fd63d400640 1 --2- 172.21.15.86:0/1995907402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd64806fd50 0x7fd6481fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:46.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.708+0000 7fd646800640 1 -- 172.21.15.86:0/1995907402 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd64806c670 msgr2=0x7fd6481649a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:46.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.708+0000 7fd646800640 1 --2- 172.21.15.86:0/1995907402 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd64806c670 0x7fd6481649a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:46.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.708+0000 7fd646800640 1 -- 172.21.15.86:0/1995907402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd64806fd50 msgr2=0x7fd6481fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:46.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.708+0000 7fd646800640 1 --2- 172.21.15.86:0/1995907402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd64806fd50 0x7fd6481fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:46.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.708+0000 7fd646800640 1 -- 172.21.15.86:0/1995907402 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd64815de80 con 0x7fd64815c2a0 2024-04-25T14:15:46.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.708+0000 7fd63d400640 1 --2- 172.21.15.86:0/1995907402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd64806fd50 0x7fd6481fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:46.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.708+0000 7fd646800640 1 --2- 172.21.15.86:0/1995907402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd64815c2a0 0x7fd64815ef20 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fd64000c3c0 tx=0x7fd64000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:46.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.708+0000 7fd63ea00640 1 -- 172.21.15.86:0/1995907402 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd640017070 con 0x7fd64815c2a0 2024-04-25T14:15:46.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.708+0000 7fd63ea00640 1 -- 172.21.15.86:0/1995907402 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd640005cf0 con 0x7fd64815c2a0 2024-04-25T14:15:46.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.708+0000 7fd64cc00640 1 -- 172.21.15.86:0/1995907402 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd64815e160 con 0x7fd64815c2a0 2024-04-25T14:15:46.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.708+0000 7fd63ea00640 1 -- 172.21.15.86:0/1995907402 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd640007310 con 0x7fd64815c2a0 2024-04-25T14:15:46.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.708+0000 7fd64cc00640 1 -- 172.21.15.86:0/1995907402 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd64815e720 con 0x7fd64815c2a0 2024-04-25T14:15:46.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.708+0000 7fd64cc00640 1 -- 172.21.15.86:0/1995907402 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd648069d00 con 0x7fd64815c2a0 2024-04-25T14:15:46.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.712+0000 7fd63ea00640 1 -- 172.21.15.86:0/1995907402 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7fd64001a070 con 0x7fd64815c2a0 2024-04-25T14:15:46.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.712+0000 7fd63ea00640 1 --2- 172.21.15.86:0/1995907402 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd62406f450 0x7fd624071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:46.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.712+0000 7fd63ea00640 1 -- 172.21.15.86:0/1995907402 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fd64008e880 con 0x7fd64815c2a0 2024-04-25T14:15:46.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.712+0000 7fd645e00640 1 --2- 172.21.15.86:0/1995907402 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd62406f450 0x7fd624071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:46.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.712+0000 7fd645e00640 1 --2- 172.21.15.86:0/1995907402 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd62406f450 0x7fd624071960 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd638009bd0 tx=0x7fd63800b040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:46.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.716+0000 7fd63ea00640 1 -- 172.21.15.86:0/1995907402 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd64005ae20 con 0x7fd64815c2a0 2024-04-25T14:15:46.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.908+0000 7fd64cc00640 1 -- 172.21.15.86:0/1995907402 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd648076bf0 con 0x7fd64815c2a0 2024-04-25T14:15:46.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.908+0000 7fd63ea00640 1 -- 172.21.15.86:0/1995907402 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7fd64005f120 con 0x7fd64815c2a0 2024-04-25T14:15:46.906 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:46.906 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:46.906 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:46.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.912+0000 7fd64cc00640 1 -- 172.21.15.86:0/1995907402 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd62406f450 msgr2=0x7fd624071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:46.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.912+0000 7fd64cc00640 1 --2- 172.21.15.86:0/1995907402 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd62406f450 0x7fd624071960 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd638009bd0 tx=0x7fd63800b040 comp rx=0 tx=0).stop 2024-04-25T14:15:46.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.912+0000 7fd64cc00640 1 -- 172.21.15.86:0/1995907402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd64815c2a0 msgr2=0x7fd64815ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:46.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.912+0000 7fd64cc00640 1 --2- 172.21.15.86:0/1995907402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd64815c2a0 0x7fd64815ef20 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fd64000c3c0 tx=0x7fd64000c880 comp rx=0 tx=0).stop 2024-04-25T14:15:46.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.912+0000 7fd64cc00640 1 -- 172.21.15.86:0/1995907402 shutdown_connections 2024-04-25T14:15:46.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.912+0000 7fd64cc00640 1 --2- 172.21.15.86:0/1995907402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd64815c2a0 0x7fd64815ef20 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:46.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.912+0000 7fd64cc00640 1 --2- 172.21.15.86:0/1995907402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd64806fd50 0x7fd6481fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:46.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.912+0000 7fd64cc00640 1 --2- 172.21.15.86:0/1995907402 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd62406f450 0x7fd624071960 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:46.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.912+0000 7fd64cc00640 1 --2- 172.21.15.86:0/1995907402 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd64806c670 0x7fd6481649a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:46.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.912+0000 7fd64cc00640 1 -- 172.21.15.86:0/1995907402 >> 172.21.15.86:0/1995907402 conn(0x7fd6480b4650 msgr2=0x7fd648156b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:46.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.912+0000 7fd64cc00640 1 -- 172.21.15.86:0/1995907402 shutdown_connections 2024-04-25T14:15:46.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:46.912+0000 7fd64cc00640 1 -- 172.21.15.86:0/1995907402 wait complete. 2024-04-25T14:15:47.330 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:47.330 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:48.331 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:48.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.492+0000 7f0ed7600640 1 Processor -- start 2024-04-25T14:15:48.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.492+0000 7f0ed7600640 1 -- start start 2024-04-25T14:15:48.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.492+0000 7f0ed7600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ed00d2c80 0x7f0ed015dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:48.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.492+0000 7f0ed7600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0ed0070e20 0x7f0ed0071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:48.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.492+0000 7f0ed7600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ed0071770 con 0x7f0ed00722f0 2024-04-25T14:15:48.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.492+0000 7f0ed7600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0ed00ce720 con 0x7f0ed0070e20 2024-04-25T14:15:48.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.492+0000 7f0ed7600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0ed00cd8d0 con 0x7f0ed00d2c80 2024-04-25T14:15:48.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.496+0000 7f0ed4c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ed00d2c80 0x7f0ed015dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:48.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.496+0000 7f0ecfe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0ed0070e20 0x7f0ed0071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:48.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.496+0000 7f0ed5600640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f0ed00722f0 0x7f0ed0072700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:52824/0 (socket says 172.21.15.86:52824) 2024-04-25T14:15:48.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.496+0000 7f0ed5600640 1 -- 172.21.15.86:0/1765409345 learned_addr learned my addr 172.21.15.86:0/1765409345 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:15:48.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.496+0000 7f0ecf400640 1 -- 172.21.15.86:0/1765409345 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3683881581 0 0) 0x7f0ec80014a0 con 0x7f0ed00722f0 2024-04-25T14:15:48.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.496+0000 7f0ecf400640 1 -- 172.21.15.86:0/1765409345 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0eb4003530 con 0x7f0ed00722f0 2024-04-25T14:15:48.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.496+0000 7f0ecfe00640 1 -- 172.21.15.86:0/1765409345 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ed00d2c80 msgr2=0x7f0ed015dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:48.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.496+0000 7f0ecfe00640 1 --2- 172.21.15.86:0/1765409345 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ed00d2c80 0x7f0ed015dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:48.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.496+0000 7f0ecfe00640 1 -- 172.21.15.86:0/1765409345 >> v1:172.21.15.155:6789/0 conn(0x7f0ed00722f0 legacy=0x7f0ed0072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:48.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.496+0000 7f0ecfe00640 1 -- 172.21.15.86:0/1765409345 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed00ce970 con 0x7f0ed0070e20 2024-04-25T14:15:48.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.496+0000 7f0ed4c00640 1 --2- 172.21.15.86:0/1765409345 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ed00d2c80 0x7f0ed015dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:48.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.496+0000 7f0ecfe00640 1 --2- 172.21.15.86:0/1765409345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ed0070e20 0x7f0ed0071230 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0097a0 tx=0x7f0ebc01ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d6854ddc3983d1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:48.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.500+0000 7f0ecf400640 1 -- 172.21.15.86:0/1765409345 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc0b5070 con 0x7f0ed0070e20 2024-04-25T14:15:48.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.500+0000 7f0ecf400640 1 -- 172.21.15.86:0/1765409345 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ebc0a8970 con 0x7f0ed0070e20 2024-04-25T14:15:48.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.500+0000 7f0ecf400640 1 -- 172.21.15.86:0/1765409345 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc0a8d40 con 0x7f0ed0070e20 2024-04-25T14:15:48.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.500+0000 7f0ed7600640 1 -- 172.21.15.86:0/1765409345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ed0070e20 msgr2=0x7f0ed0071230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:48.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.500+0000 7f0ed7600640 1 --2- 172.21.15.86:0/1765409345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ed0070e20 0x7f0ed0071230 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0097a0 tx=0x7f0ebc01ec70 comp rx=0 tx=0).stop 2024-04-25T14:15:48.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.500+0000 7f0ed7600640 1 -- 172.21.15.86:0/1765409345 shutdown_connections 2024-04-25T14:15:48.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.500+0000 7f0ed7600640 1 --2- 172.21.15.86:0/1765409345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ed0070e20 0x7f0ed0071230 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:48.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.500+0000 7f0ed7600640 1 --2- 172.21.15.86:0/1765409345 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ed00d2c80 0x7f0ed015dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:48.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.500+0000 7f0ed7600640 1 -- 172.21.15.86:0/1765409345 >> 172.21.15.86:0/1765409345 conn(0x7f0ed00cffc0 msgr2=0x7f0ed00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:48.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.500+0000 7f0ed7600640 1 -- 172.21.15.86:0/1765409345 shutdown_connections 2024-04-25T14:15:48.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.500+0000 7f0ed7600640 1 -- 172.21.15.86:0/1765409345 wait complete. 2024-04-25T14:15:48.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.500+0000 7f0ed7600640 1 Processor -- start 2024-04-25T14:15:48.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.500+0000 7f0ed7600640 1 -- start start 2024-04-25T14:15:48.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.504+0000 7f0ed7600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ed0070e20 0x7f0ed01534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:48.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.504+0000 7f0ed7600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ed00722f0 0x7f0ed00b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:48.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.504+0000 7f0ed7600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ed00d2c80 0x7f0ed00af020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:48.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.504+0000 7f0ed7600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed00d21a0 con 0x7f0ed00d2c80 2024-04-25T14:15:48.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.504+0000 7f0ed7600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed00d1ec0 con 0x7f0ed0070e20 2024-04-25T14:15:48.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.504+0000 7f0ed7600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0ed00d2480 con 0x7f0ed00722f0 2024-04-25T14:15:48.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.504+0000 7f0ed5600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ed00d2c80 0x7f0ed00af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:48.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.504+0000 7f0ed5600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ed00d2c80 0x7f0ed00af020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35960/0 (socket says 172.21.15.86:35960) 2024-04-25T14:15:48.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.504+0000 7f0ed5600640 1 -- 172.21.15.86:0/3009087684 learned_addr learned my addr 172.21.15.86:0/3009087684 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:48.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.504+0000 7f0ecfe00640 1 --2- 172.21.15.86:0/3009087684 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ed00722f0 0x7f0ed00b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:48.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.504+0000 7f0ed4c00640 1 --2- 172.21.15.86:0/3009087684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ed0070e20 0x7f0ed01534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:48.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.504+0000 7f0ed5600640 1 -- 172.21.15.86:0/3009087684 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ed00722f0 msgr2=0x7f0ed00b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:48.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.504+0000 7f0ed5600640 1 --2- 172.21.15.86:0/3009087684 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ed00722f0 0x7f0ed00b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:48.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.504+0000 7f0ed5600640 1 -- 172.21.15.86:0/3009087684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ed0070e20 msgr2=0x7f0ed01534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:48.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.504+0000 7f0ed5600640 1 --2- 172.21.15.86:0/3009087684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ed0070e20 0x7f0ed01534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:48.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.504+0000 7f0ed5600640 1 -- 172.21.15.86:0/3009087684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed00b4a60 con 0x7f0ed00d2c80 2024-04-25T14:15:48.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.504+0000 7f0ed5600640 1 --2- 172.21.15.86:0/3009087684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ed00d2c80 0x7f0ed00af020 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800cad0 tx=0x7f0ec800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:48.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.504+0000 7f0eccc00640 1 -- 172.21.15.86:0/3009087684 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8017070 con 0x7f0ed00d2c80 2024-04-25T14:15:48.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.504+0000 7f0eccc00640 1 -- 172.21.15.86:0/3009087684 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ec800ac30 con 0x7f0ed00d2c80 2024-04-25T14:15:48.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.504+0000 7f0ed7600640 1 -- 172.21.15.86:0/3009087684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed00b3480 con 0x7f0ed00d2c80 2024-04-25T14:15:48.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.504+0000 7f0eccc00640 1 -- 172.21.15.86:0/3009087684 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8012400 con 0x7f0ed00d2c80 2024-04-25T14:15:48.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.504+0000 7f0ed7600640 1 -- 172.21.15.86:0/3009087684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed00b3a40 con 0x7f0ed00d2c80 2024-04-25T14:15:48.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.504+0000 7f0ed7600640 1 -- 172.21.15.86:0/3009087684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ed00720e0 con 0x7f0ed00d2c80 2024-04-25T14:15:48.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.508+0000 7f0eccc00640 1 -- 172.21.15.86:0/3009087684 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f0ec801a070 con 0x7f0ed00d2c80 2024-04-25T14:15:48.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.508+0000 7f0eccc00640 1 --2- 172.21.15.86:0/3009087684 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f0eb006f480 0x7f0eb0071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:48.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.508+0000 7f0eccc00640 1 -- 172.21.15.86:0/3009087684 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f0ec808f110 con 0x7f0ed00d2c80 2024-04-25T14:15:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.508+0000 7f0ed4c00640 1 --2- 172.21.15.86:0/3009087684 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f0eb006f480 0x7f0eb0071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.508+0000 7f0ed4c00640 1 --2- 172.21.15.86:0/3009087684 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f0eb006f480 0x7f0eb0071990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0009580 tx=0x7f0ec0009190 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:48.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.512+0000 7f0eccc00640 1 -- 172.21.15.86:0/3009087684 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0ed0071770 con 0x7f0ed00d2c80 2024-04-25T14:15:48.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.700+0000 7f0ed7600640 1 -- 172.21.15.86:0/3009087684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ed0161420 con 0x7f0ed00d2c80 2024-04-25T14:15:48.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.704+0000 7f0eccc00640 1 -- 172.21.15.86:0/3009087684 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f0ec805b6d0 con 0x7f0ed00d2c80 2024-04-25T14:15:48.701 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:48.701 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:48.703 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:48.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.704+0000 7f0ed7600640 1 -- 172.21.15.86:0/3009087684 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f0eb006f480 msgr2=0x7f0eb0071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:48.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.704+0000 7f0ed7600640 1 --2- 172.21.15.86:0/3009087684 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f0eb006f480 0x7f0eb0071990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0009580 tx=0x7f0ec0009190 comp rx=0 tx=0).stop 2024-04-25T14:15:48.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.704+0000 7f0ed7600640 1 -- 172.21.15.86:0/3009087684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ed00d2c80 msgr2=0x7f0ed00af020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:48.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.704+0000 7f0ed7600640 1 --2- 172.21.15.86:0/3009087684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ed00d2c80 0x7f0ed00af020 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800cad0 tx=0x7f0ec800cf90 comp rx=0 tx=0).stop 2024-04-25T14:15:48.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.704+0000 7f0ed7600640 1 -- 172.21.15.86:0/3009087684 shutdown_connections 2024-04-25T14:15:48.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.704+0000 7f0ed7600640 1 --2- 172.21.15.86:0/3009087684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ed00d2c80 0x7f0ed00af020 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:48.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.704+0000 7f0ed7600640 1 --2- 172.21.15.86:0/3009087684 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f0eb006f480 0x7f0eb0071990 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:48.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.704+0000 7f0ed7600640 1 --2- 172.21.15.86:0/3009087684 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ed00722f0 0x7f0ed00b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:48.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.704+0000 7f0ed7600640 1 --2- 172.21.15.86:0/3009087684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ed0070e20 0x7f0ed01534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:48.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.704+0000 7f0ed7600640 1 -- 172.21.15.86:0/3009087684 >> 172.21.15.86:0/3009087684 conn(0x7f0ed00cffc0 msgr2=0x7f0ed00cdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:48.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.704+0000 7f0ed7600640 1 -- 172.21.15.86:0/3009087684 shutdown_connections 2024-04-25T14:15:48.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:48.704+0000 7f0ed7600640 1 -- 172.21.15.86:0/3009087684 wait complete. 2024-04-25T14:15:49.124 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:49.125 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:50.125 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:50.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.280+0000 7f578f600640 1 Processor -- start 2024-04-25T14:15:50.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.280+0000 7f578f600640 1 -- start start 2024-04-25T14:15:50.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.284+0000 7f578f600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f57880d16e0 0x7f57880d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.284+0000 7f578f600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57880730b0 0x7f57880734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.284+0000 7f578f600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57880d2030 con 0x7f57880cfca0 2024-04-25T14:15:50.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.284+0000 7f578f600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5788070470 con 0x7f57880d16e0 2024-04-25T14:15:50.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.284+0000 7f578f600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5788072bd0 con 0x7f57880730b0 2024-04-25T14:15:50.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.284+0000 7f578cc00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f57880cfca0 0x7f578806f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:43206/0 (socket says 172.21.15.86:43206) 2024-04-25T14:15:50.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.284+0000 7f578cc00640 1 -- 172.21.15.86:0/1356300018 learned_addr learned my addr 172.21.15.86:0/1356300018 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:15:50.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.284+0000 7f5787e00640 1 --2- 172.21.15.86:0/1356300018 >> v2:172.21.15.86:3300/0 conn(0x7f57880d16e0 0x7f57880d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.284+0000 7f578d600640 1 --2- 172.21.15.86:0/1356300018 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57880730b0 0x7f57880734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.284+0000 7f5787e00640 1 -- 172.21.15.86:0/1356300018 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57880730b0 msgr2=0x7f57880734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:50.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.284+0000 7f5787e00640 1 --2- 172.21.15.86:0/1356300018 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57880730b0 0x7f57880734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.284+0000 7f5787e00640 1 -- 172.21.15.86:0/1356300018 >> v1:172.21.15.86:6789/0 conn(0x7f57880cfca0 legacy=0x7f578806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.284+0000 7f5787e00640 1 -- 172.21.15.86:0/1356300018 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578806fbd0 con 0x7f57880d16e0 2024-04-25T14:15:50.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.284+0000 7f5787400640 1 -- 172.21.15.86:0/1356300018 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3616795525 0 0) 0x7f577c001120 con 0x7f57880cfca0 2024-04-25T14:15:50.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.288+0000 7f5787e00640 1 --2- 172.21.15.86:0/1356300018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57880d16e0 0x7f57880d1af0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f57740097c0 tx=0x7f577401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7d96aa3e68e9de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.288+0000 7f5787400640 1 -- 172.21.15.86:0/1356300018 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57740b1070 con 0x7f57880d16e0 2024-04-25T14:15:50.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.288+0000 7f5787400640 1 -- 172.21.15.86:0/1356300018 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57740ac410 con 0x7f57880d16e0 2024-04-25T14:15:50.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.288+0000 7f5787400640 1 -- 172.21.15.86:0/1356300018 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57740ac7e0 con 0x7f57880d16e0 2024-04-25T14:15:50.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.288+0000 7f578f600640 1 -- 172.21.15.86:0/1356300018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57880d16e0 msgr2=0x7f57880d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.288+0000 7f578f600640 1 --2- 172.21.15.86:0/1356300018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57880d16e0 0x7f57880d1af0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f57740097c0 tx=0x7f577401c4a0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.288+0000 7f578f600640 1 -- 172.21.15.86:0/1356300018 shutdown_connections 2024-04-25T14:15:50.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.288+0000 7f578f600640 1 --2- 172.21.15.86:0/1356300018 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57880730b0 0x7f57880734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.288+0000 7f578f600640 1 --2- 172.21.15.86:0/1356300018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57880d16e0 0x7f57880d1af0 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.288+0000 7f578f600640 1 -- 172.21.15.86:0/1356300018 >> 172.21.15.86:0/1356300018 conn(0x7f57880b4650 msgr2=0x7f57880b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.288+0000 7f578f600640 1 -- 172.21.15.86:0/1356300018 shutdown_connections 2024-04-25T14:15:50.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.288+0000 7f578f600640 1 -- 172.21.15.86:0/1356300018 wait complete. 2024-04-25T14:15:50.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.292+0000 7f578f600640 1 Processor -- start 2024-04-25T14:15:50.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.292+0000 7f578f600640 1 -- start start 2024-04-25T14:15:50.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.292+0000 7f578f600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57880730b0 0x7f5788158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.292+0000 7f578f600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57880cfca0 0x7f5788158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.292+0000 7f578f600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57880d16e0 0x7f578815b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.292+0000 7f578f600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5788169a70 con 0x7f57880730b0 2024-04-25T14:15:50.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.292+0000 7f578f600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f57881698f0 con 0x7f57880d16e0 2024-04-25T14:15:50.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.292+0000 7f578f600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5788169bf0 con 0x7f57880cfca0 2024-04-25T14:15:50.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.292+0000 7f578cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57880730b0 0x7f5788158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.292+0000 7f578cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57880730b0 0x7f5788158580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45076/0 (socket says 172.21.15.86:45076) 2024-04-25T14:15:50.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.292+0000 7f578cc00640 1 -- 172.21.15.86:0/3399055857 learned_addr learned my addr 172.21.15.86:0/3399055857 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:50.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.292+0000 7f5787e00640 1 --2- 172.21.15.86:0/3399055857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57880cfca0 0x7f5788158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.292+0000 7f578d600640 1 --2- 172.21.15.86:0/3399055857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57880d16e0 0x7f578815b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.292+0000 7f578cc00640 1 -- 172.21.15.86:0/3399055857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57880cfca0 msgr2=0x7f5788158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.292+0000 7f578cc00640 1 --2- 172.21.15.86:0/3399055857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57880cfca0 0x7f5788158ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.292+0000 7f578cc00640 1 -- 172.21.15.86:0/3399055857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57880d16e0 msgr2=0x7f578815b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.292+0000 7f578cc00640 1 --2- 172.21.15.86:0/3399055857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57880d16e0 0x7f578815b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.292+0000 7f578cc00640 1 -- 172.21.15.86:0/3399055857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578815a700 con 0x7f57880730b0 2024-04-25T14:15:50.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.292+0000 7f578d600640 1 --2- 172.21.15.86:0/3399055857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57880d16e0 0x7f578815b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:50.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.292+0000 7f578cc00640 1 --2- 172.21.15.86:0/3399055857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57880730b0 0x7f5788158580 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f577c00c4b0 tx=0x7f577c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.292+0000 7f5784c00640 1 -- 172.21.15.86:0/3399055857 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c018070 con 0x7f57880730b0 2024-04-25T14:15:50.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.292+0000 7f5784c00640 1 -- 172.21.15.86:0/3399055857 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f577c0074e0 con 0x7f57880730b0 2024-04-25T14:15:50.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.292+0000 7f5784c00640 1 -- 172.21.15.86:0/3399055857 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c0049f0 con 0x7f57880730b0 2024-04-25T14:15:50.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.292+0000 7f578f600640 1 -- 172.21.15.86:0/3399055857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f578815a9e0 con 0x7f57880730b0 2024-04-25T14:15:50.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.292+0000 7f578f600640 1 -- 172.21.15.86:0/3399055857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f578815afa0 con 0x7f57880730b0 2024-04-25T14:15:50.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.296+0000 7f578f600640 1 -- 172.21.15.86:0/3399055857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f578809f4b0 con 0x7f57880730b0 2024-04-25T14:15:50.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.296+0000 7f5784c00640 1 -- 172.21.15.86:0/3399055857 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f577c004b90 con 0x7f57880730b0 2024-04-25T14:15:50.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.296+0000 7f5784c00640 1 --2- 172.21.15.86:0/3399055857 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f576806f480 0x7f5768071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:50.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.296+0000 7f5784c00640 1 -- 172.21.15.86:0/3399055857 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f577c08e9a0 con 0x7f57880730b0 2024-04-25T14:15:50.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.296+0000 7f5787e00640 1 --2- 172.21.15.86:0/3399055857 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f576806f480 0x7f5768071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:50.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.296+0000 7f5787e00640 1 --2- 172.21.15.86:0/3399055857 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f576806f480 0x7f5768071990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f57740ae000 tx=0x7f5774008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:50.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.300+0000 7f5784c00640 1 -- 172.21.15.86:0/3399055857 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f577c021080 con 0x7f57880730b0 2024-04-25T14:15:50.465 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:15:50.468 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:15:50.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.488+0000 7f578f600640 1 -- 172.21.15.86:0/3399055857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5788069d00 con 0x7f57880730b0 2024-04-25T14:15:50.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.488+0000 7f5784c00640 1 -- 172.21.15.86:0/3399055857 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f57880d2030 con 0x7f57880730b0 2024-04-25T14:15:50.488 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:50.488 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:50.489 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:50.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.492+0000 7f578f600640 1 -- 172.21.15.86:0/3399055857 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f576806f480 msgr2=0x7f5768071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.492+0000 7f578f600640 1 --2- 172.21.15.86:0/3399055857 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f576806f480 0x7f5768071990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f57740ae000 tx=0x7f5774008040 comp rx=0 tx=0).stop 2024-04-25T14:15:50.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.492+0000 7f578f600640 1 -- 172.21.15.86:0/3399055857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57880730b0 msgr2=0x7f5788158580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:50.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.492+0000 7f578f600640 1 --2- 172.21.15.86:0/3399055857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57880730b0 0x7f5788158580 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f577c00c4b0 tx=0x7f577c00c970 comp rx=0 tx=0).stop 2024-04-25T14:15:50.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.492+0000 7f578f600640 1 -- 172.21.15.86:0/3399055857 shutdown_connections 2024-04-25T14:15:50.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.492+0000 7f578f600640 1 --2- 172.21.15.86:0/3399055857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57880d16e0 0x7f578815b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.492+0000 7f578f600640 1 --2- 172.21.15.86:0/3399055857 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f576806f480 0x7f5768071990 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.492+0000 7f578f600640 1 --2- 172.21.15.86:0/3399055857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57880cfca0 0x7f5788158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.492+0000 7f578f600640 1 --2- 172.21.15.86:0/3399055857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57880730b0 0x7f5788158580 unknown :-1 s=CLOSED pgs=2240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:50.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.492+0000 7f578f600640 1 -- 172.21.15.86:0/3399055857 >> 172.21.15.86:0/3399055857 conn(0x7f57880b4650 msgr2=0x7f5788156620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:50.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.492+0000 7f578f600640 1 -- 172.21.15.86:0/3399055857 shutdown_connections 2024-04-25T14:15:50.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:50.492+0000 7f578f600640 1 -- 172.21.15.86:0/3399055857 wait complete. 2024-04-25T14:15:50.913 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:50.913 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:51.915 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:52.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.076+0000 7f1027e00640 1 Processor -- start 2024-04-25T14:15:52.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.076+0000 7f1027e00640 1 -- start start 2024-04-25T14:15:52.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.076+0000 7f1027e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10280d01b0 0x7f10280d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:52.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.076+0000 7f1027e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f10280722a0 0x7f10280726b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:52.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.076+0000 7f1027e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10280d0b00 con 0x7f10281635a0 2024-04-25T14:15:52.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.076+0000 7f1027e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1028072bf0 con 0x7f10280722a0 2024-04-25T14:15:52.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.076+0000 7f1027e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1028163d80 con 0x7f10280d01b0 2024-04-25T14:15:52.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.076+0000 7f1027400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f10280722a0 0x7f10280726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:52.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.076+0000 7f1027400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f10280722a0 0x7f10280726b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45080/0 (socket says 172.21.15.86:45080) 2024-04-25T14:15:52.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.076+0000 7f1027400640 1 -- 172.21.15.86:0/2037243542 learned_addr learned my addr 172.21.15.86:0/2037243542 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:52.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.076+0000 7f1026000640 1 --2- 172.21.15.86:0/2037243542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10280d01b0 0x7f10280d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:52.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.076+0000 7f1027400640 1 -- 172.21.15.86:0/2037243542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10280d01b0 msgr2=0x7f10280d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:52.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.076+0000 7f1027400640 1 --2- 172.21.15.86:0/2037243542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10280d01b0 0x7f10280d05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.076+0000 7f1027400640 1 -- 172.21.15.86:0/2037243542 >> v1:172.21.15.155:6789/0 conn(0x7f10281635a0 legacy=0x7f102806eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:52.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.076+0000 7f1027400640 1 -- 172.21.15.86:0/2037243542 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1028164060 con 0x7f10280722a0 2024-04-25T14:15:52.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.076+0000 7f1026000640 1 --2- 172.21.15.86:0/2037243542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10280d01b0 0x7f10280d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:52.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.080+0000 7f1027400640 1 --2- 172.21.15.86:0/2037243542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10280722a0 0x7f10280726b0 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f1020009e50 tx=0x7f102001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8454508765cbc49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:52.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.080+0000 7f1025600640 1 -- 172.21.15.86:0/2037243542 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10200a5700 con 0x7f10280722a0 2024-04-25T14:15:52.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.080+0000 7f1025600640 1 -- 172.21.15.86:0/2037243542 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10200a58a0 con 0x7f10280722a0 2024-04-25T14:15:52.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.080+0000 7f1025600640 1 -- 172.21.15.86:0/2037243542 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10200a5c70 con 0x7f10280722a0 2024-04-25T14:15:52.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.080+0000 7f1027e00640 1 -- 172.21.15.86:0/2037243542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10280722a0 msgr2=0x7f10280726b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:52.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.080+0000 7f1027e00640 1 --2- 172.21.15.86:0/2037243542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10280722a0 0x7f10280726b0 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f1020009e50 tx=0x7f102001ca30 comp rx=0 tx=0).stop 2024-04-25T14:15:52.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.080+0000 7f1027e00640 1 -- 172.21.15.86:0/2037243542 shutdown_connections 2024-04-25T14:15:52.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.080+0000 7f1027e00640 1 --2- 172.21.15.86:0/2037243542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10280722a0 0x7f10280726b0 unknown :-1 s=CLOSED pgs=2241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.080+0000 7f1027e00640 1 --2- 172.21.15.86:0/2037243542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10280d01b0 0x7f10280d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.080+0000 7f1027e00640 1 -- 172.21.15.86:0/2037243542 >> 172.21.15.86:0/2037243542 conn(0x7f10280b4f80 msgr2=0x7f10280b5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:52.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.080+0000 7f1027e00640 1 -- 172.21.15.86:0/2037243542 shutdown_connections 2024-04-25T14:15:52.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.080+0000 7f1027e00640 1 -- 172.21.15.86:0/2037243542 wait complete. 2024-04-25T14:15:52.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.084+0000 7f1027e00640 1 Processor -- start 2024-04-25T14:15:52.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.084+0000 7f1027e00640 1 -- start start 2024-04-25T14:15:52.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.084+0000 7f1027e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10280722a0 0x7f102814d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:52.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.084+0000 7f1027e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10280d01b0 0x7f102815b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:52.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.084+0000 7f1027e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10281635a0 0x7f102815be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:52.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.084+0000 7f1027e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1028163b30 con 0x7f10280d01b0 2024-04-25T14:15:52.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.084+0000 7f1027e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f10281639b0 con 0x7f10281635a0 2024-04-25T14:15:52.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.084+0000 7f1027e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f102806f230 con 0x7f10280722a0 2024-04-25T14:15:52.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.084+0000 7f1026000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10280d01b0 0x7f102815b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:52.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.084+0000 7f1026000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10280d01b0 0x7f102815b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45088/0 (socket says 172.21.15.86:45088) 2024-04-25T14:15:52.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.084+0000 7f1026000640 1 -- 172.21.15.86:0/3025866253 learned_addr learned my addr 172.21.15.86:0/3025866253 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:52.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.084+0000 7f1026a00640 1 --2- 172.21.15.86:0/3025866253 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10280722a0 0x7f102814d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:52.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.084+0000 7f1027400640 1 --2- 172.21.15.86:0/3025866253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10281635a0 0x7f102815be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:52.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.084+0000 7f1026000640 1 -- 172.21.15.86:0/3025866253 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10280722a0 msgr2=0x7f102814d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:52.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.084+0000 7f1026000640 1 --2- 172.21.15.86:0/3025866253 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10280722a0 0x7f102814d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.084+0000 7f1026000640 1 -- 172.21.15.86:0/3025866253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10281635a0 msgr2=0x7f102815be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:52.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.084+0000 7f1026000640 1 --2- 172.21.15.86:0/3025866253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10281635a0 0x7f102815be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.084+0000 7f1026000640 1 -- 172.21.15.86:0/3025866253 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10281fd0f0 con 0x7f10280d01b0 2024-04-25T14:15:52.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.084+0000 7f1027400640 1 --2- 172.21.15.86:0/3025866253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10281635a0 0x7f102815be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:52.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.088+0000 7f1026000640 1 --2- 172.21.15.86:0/3025866253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10280d01b0 0x7f102815b910 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f101400d800 tx=0x7f101400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:52.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.088+0000 7f101ea00640 1 -- 172.21.15.86:0/3025866253 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101400b6c0 con 0x7f10280d01b0 2024-04-25T14:15:52.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.088+0000 7f101ea00640 1 -- 172.21.15.86:0/3025866253 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1014019070 con 0x7f10280d01b0 2024-04-25T14:15:52.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.088+0000 7f101ea00640 1 -- 172.21.15.86:0/3025866253 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101400baa0 con 0x7f10280d01b0 2024-04-25T14:15:52.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.088+0000 7f1027e00640 1 -- 172.21.15.86:0/3025866253 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10281fd3d0 con 0x7f10280d01b0 2024-04-25T14:15:52.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.088+0000 7f1027e00640 1 -- 172.21.15.86:0/3025866253 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f102814e3d0 con 0x7f10280d01b0 2024-04-25T14:15:52.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.088+0000 7f1027e00640 1 -- 172.21.15.86:0/3025866253 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1028076b80 con 0x7f10280d01b0 2024-04-25T14:15:52.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.088+0000 7f101ea00640 1 -- 172.21.15.86:0/3025866253 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f101400bc40 con 0x7f10280d01b0 2024-04-25T14:15:52.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.092+0000 7f101ea00640 1 --2- 172.21.15.86:0/3025866253 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f100806f450 0x7f1008071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:52.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.092+0000 7f101ea00640 1 -- 172.21.15.86:0/3025866253 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f101408f9b0 con 0x7f10280d01b0 2024-04-25T14:15:52.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.092+0000 7f1026a00640 1 --2- 172.21.15.86:0/3025866253 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f100806f450 0x7f1008071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:52.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.092+0000 7f1026a00640 1 --2- 172.21.15.86:0/3025866253 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f100806f450 0x7f1008071960 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f10280cfc00 tx=0x7f1018009040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:52.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.096+0000 7f101ea00640 1 -- 172.21.15.86:0/3025866253 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f101405c5b0 con 0x7f10280d01b0 2024-04-25T14:15:52.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.284+0000 7f1027e00640 1 -- 172.21.15.86:0/3025866253 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f102815ca10 con 0x7f10280d01b0 2024-04-25T14:15:52.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.284+0000 7f101ea00640 1 -- 172.21.15.86:0/3025866253 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f101405beb0 con 0x7f10280d01b0 2024-04-25T14:15:52.283 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:52.283 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:52.283 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:52.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.288+0000 7f1027e00640 1 -- 172.21.15.86:0/3025866253 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f100806f450 msgr2=0x7f1008071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:52.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.288+0000 7f1027e00640 1 --2- 172.21.15.86:0/3025866253 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f100806f450 0x7f1008071960 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f10280cfc00 tx=0x7f1018009040 comp rx=0 tx=0).stop 2024-04-25T14:15:52.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.288+0000 7f1027e00640 1 -- 172.21.15.86:0/3025866253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10280d01b0 msgr2=0x7f102815b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:52.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.288+0000 7f1027e00640 1 --2- 172.21.15.86:0/3025866253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10280d01b0 0x7f102815b910 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f101400d800 tx=0x7f101400dcc0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.288+0000 7f1027e00640 1 -- 172.21.15.86:0/3025866253 shutdown_connections 2024-04-25T14:15:52.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.288+0000 7f1027e00640 1 --2- 172.21.15.86:0/3025866253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10281635a0 0x7f102815be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.288+0000 7f1027e00640 1 --2- 172.21.15.86:0/3025866253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10280d01b0 0x7f102815b910 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.288+0000 7f1027e00640 1 --2- 172.21.15.86:0/3025866253 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f100806f450 0x7f1008071960 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.288+0000 7f1027e00640 1 --2- 172.21.15.86:0/3025866253 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10280722a0 0x7f102814d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:52.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.288+0000 7f1027e00640 1 -- 172.21.15.86:0/3025866253 >> 172.21.15.86:0/3025866253 conn(0x7f10280b4f80 msgr2=0x7f10280b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:52.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.288+0000 7f1027e00640 1 -- 172.21.15.86:0/3025866253 shutdown_connections 2024-04-25T14:15:52.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:52.288+0000 7f1027e00640 1 -- 172.21.15.86:0/3025866253 wait complete. 2024-04-25T14:15:52.707 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:52.707 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:53.708 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:53.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.872+0000 7f1e16a00640 1 Processor -- start 2024-04-25T14:15:53.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.872+0000 7f1e16a00640 1 -- start start 2024-04-25T14:15:53.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.872+0000 7f1e16a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e10070d60 0x7f1e10071170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:53.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.872+0000 7f1e16a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1e100d1b00 0x7f1e100d1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:53.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.872+0000 7f1e16a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e100716b0 con 0x7f1e100c8b80 2024-04-25T14:15:53.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.872+0000 7f1e16a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1e1006de70 con 0x7f1e100d1b00 2024-04-25T14:15:53.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.872+0000 7f1e16a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1e1006e0c0 con 0x7f1e10070d60 2024-04-25T14:15:53.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.872+0000 7f1e16000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1e100d1b00 0x7f1e100d1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:53.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.872+0000 7f1e14c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e10070d60 0x7f1e10071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:53.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.872+0000 7f1e15600640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f1e100c8b80 0x7f1e10072cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:47118/0 (socket says 172.21.15.86:47118) 2024-04-25T14:15:53.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.872+0000 7f1e15600640 1 -- 172.21.15.86:0/1380923303 learned_addr learned my addr 172.21.15.86:0/1380923303 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:15:53.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.872+0000 7f1e0fe00640 1 -- 172.21.15.86:0/1380923303 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1803027805 0 0) 0x7f1e04001120 con 0x7f1e100c8b80 2024-04-25T14:15:53.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.872+0000 7f1e0fe00640 1 -- 172.21.15.86:0/1380923303 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1df8003530 con 0x7f1e100c8b80 2024-04-25T14:15:53.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.872+0000 7f1e16000640 1 -- 172.21.15.86:0/1380923303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e10070d60 msgr2=0x7f1e10071170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:53.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.872+0000 7f1e16000640 1 --2- 172.21.15.86:0/1380923303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e10070d60 0x7f1e10071170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:53.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.872+0000 7f1e16000640 1 -- 172.21.15.86:0/1380923303 >> v1:172.21.15.155:6789/0 conn(0x7f1e100c8b80 legacy=0x7f1e10072cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:53.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.872+0000 7f1e16000640 1 -- 172.21.15.86:0/1380923303 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e1006e3a0 con 0x7f1e100d1b00 2024-04-25T14:15:53.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.872+0000 7f1e14c00640 1 --2- 172.21.15.86:0/1380923303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e10070d60 0x7f1e10071170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:15:53.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.876+0000 7f1e16000640 1 --2- 172.21.15.86:0/1380923303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e100d1b00 0x7f1e100d1f10 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1e08009d90 tx=0x7f1e0801c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f6b6248c8568874 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:53.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.876+0000 7f1e0fe00640 1 -- 172.21.15.86:0/1380923303 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e080b2070 con 0x7f1e100d1b00 2024-04-25T14:15:53.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.876+0000 7f1e0fe00640 1 -- 172.21.15.86:0/1380923303 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e080a5970 con 0x7f1e100d1b00 2024-04-25T14:15:53.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.876+0000 7f1e0fe00640 1 -- 172.21.15.86:0/1380923303 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e080a5d40 con 0x7f1e100d1b00 2024-04-25T14:15:53.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.876+0000 7f1e16a00640 1 -- 172.21.15.86:0/1380923303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e100d1b00 msgr2=0x7f1e100d1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:53.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.876+0000 7f1e16a00640 1 --2- 172.21.15.86:0/1380923303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e100d1b00 0x7f1e100d1f10 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1e08009d90 tx=0x7f1e0801c8b0 comp rx=0 tx=0).stop 2024-04-25T14:15:53.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.876+0000 7f1e16a00640 1 -- 172.21.15.86:0/1380923303 shutdown_connections 2024-04-25T14:15:53.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.876+0000 7f1e16a00640 1 --2- 172.21.15.86:0/1380923303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e100d1b00 0x7f1e100d1f10 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:53.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.876+0000 7f1e16a00640 1 --2- 172.21.15.86:0/1380923303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e10070d60 0x7f1e10071170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:53.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.876+0000 7f1e16a00640 1 -- 172.21.15.86:0/1380923303 >> 172.21.15.86:0/1380923303 conn(0x7f1e100d3330 msgr2=0x7f1e100d3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:53.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.876+0000 7f1e16a00640 1 -- 172.21.15.86:0/1380923303 shutdown_connections 2024-04-25T14:15:53.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.876+0000 7f1e16a00640 1 -- 172.21.15.86:0/1380923303 wait complete. 2024-04-25T14:15:53.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.880+0000 7f1e16a00640 1 Processor -- start 2024-04-25T14:15:53.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.880+0000 7f1e16a00640 1 -- start start 2024-04-25T14:15:53.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.880+0000 7f1e16a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e10070d60 0x7f1e1013e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:53.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.880+0000 7f1e16a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e100c8b80 0x7f1e1015e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:53.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.880+0000 7f1e16a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e100d1b00 0x7f1e100cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:53.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.880+0000 7f1e16a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e100c8f90 con 0x7f1e10070d60 2024-04-25T14:15:53.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.880+0000 7f1e16a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1015fd50 con 0x7f1e100d1b00 2024-04-25T14:15:53.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.880+0000 7f1e16a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1e100c9110 con 0x7f1e100c8b80 2024-04-25T14:15:53.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.880+0000 7f1e15600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e10070d60 0x7f1e1013e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:53.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.880+0000 7f1e15600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e10070d60 0x7f1e1013e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45102/0 (socket says 172.21.15.86:45102) 2024-04-25T14:15:53.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.880+0000 7f1e15600640 1 -- 172.21.15.86:0/4248336743 learned_addr learned my addr 172.21.15.86:0/4248336743 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:53.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.880+0000 7f1e14c00640 1 --2- 172.21.15.86:0/4248336743 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e100c8b80 0x7f1e1015e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:53.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.880+0000 7f1e16000640 1 --2- 172.21.15.86:0/4248336743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e100d1b00 0x7f1e100cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:53.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.884+0000 7f1e15600640 1 -- 172.21.15.86:0/4248336743 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e100c8b80 msgr2=0x7f1e1015e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:53.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.884+0000 7f1e15600640 1 --2- 172.21.15.86:0/4248336743 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e100c8b80 0x7f1e1015e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:53.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.884+0000 7f1e15600640 1 -- 172.21.15.86:0/4248336743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e100d1b00 msgr2=0x7f1e100cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:53.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.884+0000 7f1e15600640 1 --2- 172.21.15.86:0/4248336743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e100d1b00 0x7f1e100cfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:53.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.884+0000 7f1e15600640 1 -- 172.21.15.86:0/4248336743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e1013ed60 con 0x7f1e10070d60 2024-04-25T14:15:53.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.884+0000 7f1e15600640 1 --2- 172.21.15.86:0/4248336743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e10070d60 0x7f1e1013e820 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f1e0400c3f0 tx=0x7f1e0400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:53.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.884+0000 7f1e0d600640 1 -- 172.21.15.86:0/4248336743 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e04017070 con 0x7f1e10070d60 2024-04-25T14:15:53.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.884+0000 7f1e0d600640 1 -- 172.21.15.86:0/4248336743 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e04005cf0 con 0x7f1e10070d60 2024-04-25T14:15:53.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.884+0000 7f1e16a00640 1 -- 172.21.15.86:0/4248336743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e10141350 con 0x7f1e10070d60 2024-04-25T14:15:53.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.884+0000 7f1e0d600640 1 -- 172.21.15.86:0/4248336743 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e040072f0 con 0x7f1e10070d60 2024-04-25T14:15:53.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.884+0000 7f1e16a00640 1 -- 172.21.15.86:0/4248336743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e10141910 con 0x7f1e10070d60 2024-04-25T14:15:53.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.884+0000 7f1e16a00640 1 -- 172.21.15.86:0/4248336743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e10076b80 con 0x7f1e10070d60 2024-04-25T14:15:53.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.884+0000 7f1e0d600640 1 -- 172.21.15.86:0/4248336743 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f1e0401a070 con 0x7f1e10070d60 2024-04-25T14:15:53.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.888+0000 7f1e0d600640 1 --2- 172.21.15.86:0/4248336743 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1df006f450 0x7f1df0071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:53.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.888+0000 7f1e0d600640 1 -- 172.21.15.86:0/4248336743 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f1e0408e8c0 con 0x7f1e10070d60 2024-04-25T14:15:53.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.888+0000 7f1e14c00640 1 --2- 172.21.15.86:0/4248336743 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1df006f450 0x7f1df0071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:53.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.888+0000 7f1e14c00640 1 --2- 172.21.15.86:0/4248336743 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1df006f450 0x7f1df0071960 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1e100b4eb0 tx=0x7f1dfc009190 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:53.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:53.892+0000 7f1e0d600640 1 -- 172.21.15.86:0/4248336743 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1e100716b0 con 0x7f1e10070d60 2024-04-25T14:15:54.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:54.084+0000 7f1e16a00640 1 -- 172.21.15.86:0/4248336743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e100c8410 con 0x7f1e10070d60 2024-04-25T14:15:54.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:54.084+0000 7f1e0d600640 1 -- 172.21.15.86:0/4248336743 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f1e0405ae70 con 0x7f1e10070d60 2024-04-25T14:15:54.080 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:54.080 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:54.081 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:54.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:54.084+0000 7f1e16a00640 1 -- 172.21.15.86:0/4248336743 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1df006f450 msgr2=0x7f1df0071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:54.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:54.084+0000 7f1e16a00640 1 --2- 172.21.15.86:0/4248336743 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1df006f450 0x7f1df0071960 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1e100b4eb0 tx=0x7f1dfc009190 comp rx=0 tx=0).stop 2024-04-25T14:15:54.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:54.084+0000 7f1e16a00640 1 -- 172.21.15.86:0/4248336743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e10070d60 msgr2=0x7f1e1013e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:54.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:54.084+0000 7f1e16a00640 1 --2- 172.21.15.86:0/4248336743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e10070d60 0x7f1e1013e820 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f1e0400c3f0 tx=0x7f1e0400c8b0 comp rx=0 tx=0).stop 2024-04-25T14:15:54.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:54.084+0000 7f1e16a00640 1 -- 172.21.15.86:0/4248336743 shutdown_connections 2024-04-25T14:15:54.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:54.084+0000 7f1e16a00640 1 --2- 172.21.15.86:0/4248336743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e100d1b00 0x7f1e100cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:54.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:54.084+0000 7f1e16a00640 1 --2- 172.21.15.86:0/4248336743 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1df006f450 0x7f1df0071960 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:54.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:54.084+0000 7f1e16a00640 1 --2- 172.21.15.86:0/4248336743 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e100c8b80 0x7f1e1015e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:54.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:54.084+0000 7f1e16a00640 1 --2- 172.21.15.86:0/4248336743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e10070d60 0x7f1e1013e820 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:54.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:54.084+0000 7f1e16a00640 1 -- 172.21.15.86:0/4248336743 >> 172.21.15.86:0/4248336743 conn(0x7f1e100d3330 msgr2=0x7f1e100cf150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:54.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:54.084+0000 7f1e16a00640 1 -- 172.21.15.86:0/4248336743 shutdown_connections 2024-04-25T14:15:54.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:54.084+0000 7f1e16a00640 1 -- 172.21.15.86:0/4248336743 wait complete. 2024-04-25T14:15:54.504 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:54.504 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:55.506 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:55.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.664+0000 7f9399800640 1 Processor -- start 2024-04-25T14:15:55.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.664+0000 7f9399800640 1 -- start start 2024-04-25T14:15:55.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.664+0000 7f9399800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f93941651d0 0x7f93941675f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:55.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.664+0000 7f9399800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93940b0a60 con 0x7f939406fd50 2024-04-25T14:15:55.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.664+0000 7f9399800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93940ae840 con 0x7f939415c2a0 2024-04-25T14:15:55.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.664+0000 7f9399800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f939406ef90 con 0x7f93941651d0 2024-04-25T14:15:55.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.664+0000 7f9393400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f93941651d0 0x7f93941675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:55.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.664+0000 7f9393400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f93941651d0 0x7f93941675f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45118/0 (socket says 172.21.15.86:45118) 2024-04-25T14:15:55.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.664+0000 7f9393400640 1 -- 172.21.15.86:0/1584190214 learned_addr learned my addr 172.21.15.86:0/1584190214 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:55.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.664+0000 7f9393400640 1 -- 172.21.15.86:0/1584190214 >> v1:172.21.15.155:6789/0 conn(0x7f939415c2a0 legacy=0x7f93940ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:55.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.664+0000 7f9393400640 1 -- 172.21.15.86:0/1584190214 >> v1:172.21.15.86:6789/0 conn(0x7f939406fd50 legacy=0x7f9394070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:55.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.664+0000 7f9393400640 1 -- 172.21.15.86:0/1584190214 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939406f270 con 0x7f93941651d0 2024-04-25T14:15:55.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.668+0000 7f9393400640 1 --2- 172.21.15.86:0/1584190214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93941651d0 0x7f93941675f0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f938c009e50 tx=0x7f938c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d0dca29147a5e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:55.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.668+0000 7f9392000640 1 -- 172.21.15.86:0/1584190214 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c0a5770 con 0x7f93941651d0 2024-04-25T14:15:55.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.668+0000 7f9392000640 1 -- 172.21.15.86:0/1584190214 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f938c0a5910 con 0x7f93941651d0 2024-04-25T14:15:55.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.668+0000 7f9392000640 1 -- 172.21.15.86:0/1584190214 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c0a5ce0 con 0x7f93941651d0 2024-04-25T14:15:55.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.668+0000 7f9399800640 1 -- 172.21.15.86:0/1584190214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93941651d0 msgr2=0x7f93941675f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:55.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.668+0000 7f9399800640 1 --2- 172.21.15.86:0/1584190214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93941651d0 0x7f93941675f0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f938c009e50 tx=0x7f938c01c930 comp rx=0 tx=0).stop 2024-04-25T14:15:55.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.668+0000 7f9399800640 1 -- 172.21.15.86:0/1584190214 shutdown_connections 2024-04-25T14:15:55.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.668+0000 7f9399800640 1 --2- 172.21.15.86:0/1584190214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93941651d0 0x7f93941675f0 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:55.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.668+0000 7f9399800640 1 -- 172.21.15.86:0/1584190214 >> 172.21.15.86:0/1584190214 conn(0x7f93940b4650 msgr2=0x7f93940b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:55.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.668+0000 7f9399800640 1 -- 172.21.15.86:0/1584190214 shutdown_connections 2024-04-25T14:15:55.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.668+0000 7f9399800640 1 -- 172.21.15.86:0/1584190214 wait complete. 2024-04-25T14:15:55.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.672+0000 7f9399800640 1 Processor -- start 2024-04-25T14:15:55.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.672+0000 7f9399800640 1 -- start start 2024-04-25T14:15:55.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.672+0000 7f9399800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f939406fd50 0x7f93941fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:55.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.672+0000 7f9399800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f939415c2a0 0x7f9394158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:55.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.672+0000 7f9399800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93941651d0 0x7f9394158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:55.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.672+0000 7f9399800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f939415c830 con 0x7f93941651d0 2024-04-25T14:15:55.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.672+0000 7f9399800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f939415c6b0 con 0x7f939406fd50 2024-04-25T14:15:55.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.672+0000 7f9399800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f939416ac60 con 0x7f939415c2a0 2024-04-25T14:15:55.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.672+0000 7f9393400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93941651d0 0x7f9394158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:55.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.672+0000 7f9393400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93941651d0 0x7f9394158a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45124/0 (socket says 172.21.15.86:45124) 2024-04-25T14:15:55.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.672+0000 7f9393400640 1 -- 172.21.15.86:0/1195027406 learned_addr learned my addr 172.21.15.86:0/1195027406 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:55.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.672+0000 7f9392a00640 1 --2- 172.21.15.86:0/1195027406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f939406fd50 0x7f93941fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:55.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.672+0000 7f938be00640 1 --2- 172.21.15.86:0/1195027406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f939415c2a0 0x7f9394158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:55.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.672+0000 7f9393400640 1 -- 172.21.15.86:0/1195027406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f939415c2a0 msgr2=0x7f9394158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:55.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.672+0000 7f9393400640 1 --2- 172.21.15.86:0/1195027406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f939415c2a0 0x7f9394158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:55.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.672+0000 7f9393400640 1 -- 172.21.15.86:0/1195027406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f939406fd50 msgr2=0x7f93941fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:55.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.672+0000 7f9393400640 1 --2- 172.21.15.86:0/1195027406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f939406fd50 0x7f93941fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:55.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.672+0000 7f9393400640 1 -- 172.21.15.86:0/1195027406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93940732e0 con 0x7f93941651d0 2024-04-25T14:15:55.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.672+0000 7f9392a00640 1 --2- 172.21.15.86:0/1195027406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f939406fd50 0x7f93941fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:55.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.672+0000 7f9393400640 1 --2- 172.21.15.86:0/1195027406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93941651d0 0x7f9394158a90 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f938c0af040 tx=0x7f938c0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:55.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.672+0000 7f938aa00640 1 -- 172.21.15.86:0/1195027406 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c0bc020 con 0x7f93941651d0 2024-04-25T14:15:55.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.672+0000 7f938aa00640 1 -- 172.21.15.86:0/1195027406 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f938c0b8690 con 0x7f93941651d0 2024-04-25T14:15:55.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.672+0000 7f938aa00640 1 -- 172.21.15.86:0/1195027406 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c0b8a80 con 0x7f93941651d0 2024-04-25T14:15:55.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.672+0000 7f9399800640 1 -- 172.21.15.86:0/1195027406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9394159320 con 0x7f93941651d0 2024-04-25T14:15:55.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.672+0000 7f9399800640 1 -- 172.21.15.86:0/1195027406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9394159930 con 0x7f93941651d0 2024-04-25T14:15:55.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.676+0000 7f9399800640 1 -- 172.21.15.86:0/1195027406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9394167470 con 0x7f93941651d0 2024-04-25T14:15:55.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.676+0000 7f938aa00640 1 -- 172.21.15.86:0/1195027406 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f938c0bf020 con 0x7f93941651d0 2024-04-25T14:15:55.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.676+0000 7f938aa00640 1 --2- 172.21.15.86:0/1195027406 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f937006f480 0x7f9370071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:55.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.676+0000 7f938aa00640 1 -- 172.21.15.86:0/1195027406 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f938c12a680 con 0x7f93941651d0 2024-04-25T14:15:55.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.676+0000 7f9392a00640 1 --2- 172.21.15.86:0/1195027406 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f937006f480 0x7f9370071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:55.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.676+0000 7f9392a00640 1 --2- 172.21.15.86:0/1195027406 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f937006f480 0x7f9370071990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f93940b4200 tx=0x7f9384008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:55.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.680+0000 7f938aa00640 1 -- 172.21.15.86:0/1195027406 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f938c0c6070 con 0x7f93941651d0 2024-04-25T14:15:55.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.872+0000 7f9399800640 1 -- 172.21.15.86:0/1195027406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9394069d00 con 0x7f93941651d0 2024-04-25T14:15:55.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.872+0000 7f938aa00640 1 -- 172.21.15.86:0/1195027406 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f93940732e0 con 0x7f93941651d0 2024-04-25T14:15:55.871 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:55.871 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:55.872 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:55.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.876+0000 7f9399800640 1 -- 172.21.15.86:0/1195027406 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f937006f480 msgr2=0x7f9370071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:55.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.876+0000 7f9399800640 1 --2- 172.21.15.86:0/1195027406 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f937006f480 0x7f9370071990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f93940b4200 tx=0x7f9384008040 comp rx=0 tx=0).stop 2024-04-25T14:15:55.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.876+0000 7f9399800640 1 -- 172.21.15.86:0/1195027406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93941651d0 msgr2=0x7f9394158a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:55.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.876+0000 7f9399800640 1 --2- 172.21.15.86:0/1195027406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93941651d0 0x7f9394158a90 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f938c0af040 tx=0x7f938c0196d0 comp rx=0 tx=0).stop 2024-04-25T14:15:55.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.876+0000 7f9399800640 1 -- 172.21.15.86:0/1195027406 shutdown_connections 2024-04-25T14:15:55.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.876+0000 7f9399800640 1 --2- 172.21.15.86:0/1195027406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93941651d0 0x7f9394158a90 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:55.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.876+0000 7f9399800640 1 --2- 172.21.15.86:0/1195027406 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f937006f480 0x7f9370071990 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:55.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.876+0000 7f9399800640 1 --2- 172.21.15.86:0/1195027406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f939415c2a0 0x7f9394158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:55.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.876+0000 7f9399800640 1 --2- 172.21.15.86:0/1195027406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f939406fd50 0x7f93941fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:55.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.876+0000 7f9399800640 1 -- 172.21.15.86:0/1195027406 >> 172.21.15.86:0/1195027406 conn(0x7f93940b4650 msgr2=0x7f93940b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:55.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.876+0000 7f9399800640 1 -- 172.21.15.86:0/1195027406 shutdown_connections 2024-04-25T14:15:55.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:55.876+0000 7f9399800640 1 -- 172.21.15.86:0/1195027406 wait complete. 2024-04-25T14:15:56.296 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:56.296 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:57.296 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:57.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.456+0000 7fd187200640 1 Processor -- start 2024-04-25T14:15:57.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.456+0000 7fd187200640 1 -- start start 2024-04-25T14:15:57.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.460+0000 7fd187200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd1800cfca0 0x7fd18006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:57.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.460+0000 7fd187200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd1800d16e0 0x7fd1800d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:57.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.460+0000 7fd187200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1800d2030 con 0x7fd18006c670 2024-04-25T14:15:57.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.460+0000 7fd187200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd180070470 con 0x7fd1800d16e0 2024-04-25T14:15:57.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.460+0000 7fd187200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd180072bd0 con 0x7fd1800cfca0 2024-04-25T14:15:57.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.460+0000 7fd177400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd1800d16e0 0x7fd1800d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:57.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.460+0000 7fd177400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd1800d16e0 0x7fd1800d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45138/0 (socket says 172.21.15.86:45138) 2024-04-25T14:15:57.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.460+0000 7fd177400640 1 -- 172.21.15.86:0/2975463322 learned_addr learned my addr 172.21.15.86:0/2975463322 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:57.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.460+0000 7fd177400640 1 -- 172.21.15.86:0/2975463322 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd1800cfca0 msgr2=0x7fd18006f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:57.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.460+0000 7fd17fe00640 1 --2- 172.21.15.86:0/2975463322 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd1800cfca0 0x7fd18006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:57.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.460+0000 7fd177400640 1 --2- 172.21.15.86:0/2975463322 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd1800cfca0 0x7fd18006f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:57.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.460+0000 7fd177400640 1 -- 172.21.15.86:0/2975463322 >> v1:172.21.15.86:6789/0 conn(0x7fd18006c670 legacy=0x7fd18006ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:57.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.460+0000 7fd177400640 1 -- 172.21.15.86:0/2975463322 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd18006f930 con 0x7fd1800d16e0 2024-04-25T14:15:57.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.460+0000 7fd17fe00640 1 --2- 172.21.15.86:0/2975463322 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd1800cfca0 0x7fd18006f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:15:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.464+0000 7fd177400640 1 --2- 172.21.15.86:0/2975463322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1800d16e0 0x7fd1800d1af0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7fd168009870 tx=0x7fd16801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8800e781c80f8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.464+0000 7fd17f400640 1 -- 172.21.15.86:0/2975463322 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1680a7070 con 0x7fd1800d16e0 2024-04-25T14:15:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.464+0000 7fd17f400640 1 -- 172.21.15.86:0/2975463322 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1680b0070 con 0x7fd1800d16e0 2024-04-25T14:15:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.464+0000 7fd17f400640 1 -- 172.21.15.86:0/2975463322 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1680ab890 con 0x7fd1800d16e0 2024-04-25T14:15:57.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.464+0000 7fd187200640 1 -- 172.21.15.86:0/2975463322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1800d16e0 msgr2=0x7fd1800d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:57.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.464+0000 7fd187200640 1 --2- 172.21.15.86:0/2975463322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1800d16e0 0x7fd1800d1af0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7fd168009870 tx=0x7fd16801c560 comp rx=0 tx=0).stop 2024-04-25T14:15:57.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.464+0000 7fd187200640 1 -- 172.21.15.86:0/2975463322 shutdown_connections 2024-04-25T14:15:57.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.464+0000 7fd187200640 1 --2- 172.21.15.86:0/2975463322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1800d16e0 0x7fd1800d1af0 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:57.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.464+0000 7fd187200640 1 --2- 172.21.15.86:0/2975463322 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd1800cfca0 0x7fd18006f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:57.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.464+0000 7fd187200640 1 -- 172.21.15.86:0/2975463322 >> 172.21.15.86:0/2975463322 conn(0x7fd1800b4650 msgr2=0x7fd1800b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:57.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.464+0000 7fd187200640 1 -- 172.21.15.86:0/2975463322 shutdown_connections 2024-04-25T14:15:57.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.464+0000 7fd187200640 1 -- 172.21.15.86:0/2975463322 wait complete. 2024-04-25T14:15:57.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.468+0000 7fd187200640 1 Processor -- start 2024-04-25T14:15:57.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.468+0000 7fd187200640 1 -- start start 2024-04-25T14:15:57.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.468+0000 7fd187200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd18006c670 0x7fd18009a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:57.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.468+0000 7fd187200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd1800cfca0 0x7fd18009b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:57.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.468+0000 7fd187200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1800d16e0 0x7fd18009b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:57.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.468+0000 7fd187200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1800cdee0 con 0x7fd1800d16e0 2024-04-25T14:15:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.468+0000 7fd187200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd1800cdd60 con 0x7fd18006c670 2024-04-25T14:15:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.468+0000 7fd187200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd1800ce060 con 0x7fd1800cfca0 2024-04-25T14:15:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.468+0000 7fd185200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1800d16e0 0x7fd18009b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.468+0000 7fd185200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1800d16e0 0x7fd18009b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45152/0 (socket says 172.21.15.86:45152) 2024-04-25T14:15:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.468+0000 7fd185200640 1 -- 172.21.15.86:0/759790570 learned_addr learned my addr 172.21.15.86:0/759790570 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.468+0000 7fd177400640 1 --2- 172.21.15.86:0/759790570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd1800cfca0 0x7fd18009b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.468+0000 7fd17fe00640 1 --2- 172.21.15.86:0/759790570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd18006c670 0x7fd18009a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.468+0000 7fd185200640 1 -- 172.21.15.86:0/759790570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd1800cfca0 msgr2=0x7fd18009b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.468+0000 7fd185200640 1 --2- 172.21.15.86:0/759790570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd1800cfca0 0x7fd18009b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.468+0000 7fd185200640 1 -- 172.21.15.86:0/759790570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd18006c670 msgr2=0x7fd18009a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.468+0000 7fd185200640 1 --2- 172.21.15.86:0/759790570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd18006c670 0x7fd18009a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.468+0000 7fd185200640 1 -- 172.21.15.86:0/759790570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd18009ae70 con 0x7fd1800d16e0 2024-04-25T14:15:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.468+0000 7fd17fe00640 1 --2- 172.21.15.86:0/759790570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd18006c670 0x7fd18009a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:15:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.468+0000 7fd185200640 1 --2- 172.21.15.86:0/759790570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1800d16e0 0x7fd18009b7f0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7fd17800c760 tx=0x7fd17800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:57.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.468+0000 7fd17cc00640 1 -- 172.21.15.86:0/759790570 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd178003b80 con 0x7fd1800d16e0 2024-04-25T14:15:57.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.468+0000 7fd17cc00640 1 -- 172.21.15.86:0/759790570 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd178003d20 con 0x7fd1800d16e0 2024-04-25T14:15:57.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.468+0000 7fd187200640 1 -- 172.21.15.86:0/759790570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd18015d730 con 0x7fd1800d16e0 2024-04-25T14:15:57.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.468+0000 7fd17cc00640 1 -- 172.21.15.86:0/759790570 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17800aa90 con 0x7fd1800d16e0 2024-04-25T14:15:57.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.468+0000 7fd187200640 1 -- 172.21.15.86:0/759790570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd18015dcf0 con 0x7fd1800d16e0 2024-04-25T14:15:57.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.472+0000 7fd187200640 1 -- 172.21.15.86:0/759790570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd18009f4b0 con 0x7fd1800d16e0 2024-04-25T14:15:57.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.472+0000 7fd17cc00640 1 -- 172.21.15.86:0/759790570 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7fd17800ac30 con 0x7fd1800d16e0 2024-04-25T14:15:57.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.472+0000 7fd17cc00640 1 --2- 172.21.15.86:0/759790570 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd15c06f450 0x7fd15c071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:57.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.472+0000 7fd17cc00640 1 -- 172.21.15.86:0/759790570 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fd17808f480 con 0x7fd1800d16e0 2024-04-25T14:15:57.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.472+0000 7fd17fe00640 1 --2- 172.21.15.86:0/759790570 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd15c06f450 0x7fd15c071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:57.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.472+0000 7fd17fe00640 1 --2- 172.21.15.86:0/759790570 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd15c06f450 0x7fd15c071960 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd180070710 tx=0x7fd170008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:57.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.476+0000 7fd17cc00640 1 -- 172.21.15.86:0/759790570 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd1800d2030 con 0x7fd1800d16e0 2024-04-25T14:15:57.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.664+0000 7fd187200640 1 -- 172.21.15.86:0/759790570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd180076bf0 con 0x7fd1800d16e0 2024-04-25T14:15:57.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.664+0000 7fd17cc00640 1 -- 172.21.15.86:0/759790570 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7fd17805c080 con 0x7fd1800d16e0 2024-04-25T14:15:57.663 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:57.663 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:57.664 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:57.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.668+0000 7fd187200640 1 -- 172.21.15.86:0/759790570 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd15c06f450 msgr2=0x7fd15c071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:57.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.668+0000 7fd187200640 1 --2- 172.21.15.86:0/759790570 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd15c06f450 0x7fd15c071960 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd180070710 tx=0x7fd170008040 comp rx=0 tx=0).stop 2024-04-25T14:15:57.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.668+0000 7fd187200640 1 -- 172.21.15.86:0/759790570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1800d16e0 msgr2=0x7fd18009b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:57.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.668+0000 7fd187200640 1 --2- 172.21.15.86:0/759790570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1800d16e0 0x7fd18009b7f0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7fd17800c760 tx=0x7fd17800cc20 comp rx=0 tx=0).stop 2024-04-25T14:15:57.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.668+0000 7fd187200640 1 -- 172.21.15.86:0/759790570 shutdown_connections 2024-04-25T14:15:57.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.668+0000 7fd187200640 1 --2- 172.21.15.86:0/759790570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1800d16e0 0x7fd18009b7f0 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:57.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.668+0000 7fd187200640 1 --2- 172.21.15.86:0/759790570 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd15c06f450 0x7fd15c071960 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:57.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.668+0000 7fd187200640 1 --2- 172.21.15.86:0/759790570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd1800cfca0 0x7fd18009b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:57.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.668+0000 7fd187200640 1 --2- 172.21.15.86:0/759790570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd18006c670 0x7fd18009a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:57.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.668+0000 7fd187200640 1 -- 172.21.15.86:0/759790570 >> 172.21.15.86:0/759790570 conn(0x7fd1800b4650 msgr2=0x7fd1800ce440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:57.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.668+0000 7fd187200640 1 -- 172.21.15.86:0/759790570 shutdown_connections 2024-04-25T14:15:57.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:57.668+0000 7fd187200640 1 -- 172.21.15.86:0/759790570 wait complete. 2024-04-25T14:15:58.088 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:58.088 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:15:59.089 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:15:59.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.244+0000 7f7b90400640 1 Processor -- start 2024-04-25T14:15:59.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.244+0000 7f7b90400640 1 -- start start 2024-04-25T14:15:59.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.248+0000 7f7b90400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7b88070330 0x7f7b880d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:59.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.248+0000 7f7b90400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b880d1c20 0x7f7b8806ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:59.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.248+0000 7f7b90400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7b8806f270 0x7f7b8806f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:59.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.248+0000 7f7b90400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7b880d2030 con 0x7f7b88070330 2024-04-25T14:15:59.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.248+0000 7f7b90400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f7b8815e5c0 con 0x7f7b8806f270 2024-04-25T14:15:59.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.248+0000 7f7b90400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7b8815e8a0 con 0x7f7b880d1c20 2024-04-25T14:15:59.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.248+0000 7f7b8da00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7b88070330 0x7f7b880d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:59.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.248+0000 7f7b8da00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7b88070330 0x7f7b880d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52670/0 (socket says 172.21.15.86:52670) 2024-04-25T14:15:59.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.248+0000 7f7b8da00640 1 -- 172.21.15.86:0/2569949845 learned_addr learned my addr 172.21.15.86:0/2569949845 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:59.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.248+0000 7f7b8e400640 1 --2- 172.21.15.86:0/2569949845 >> v2:172.21.15.155:3300/0 conn(0x7f7b8806f270 0x7f7b8806f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:59.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.248+0000 7f7b8d000640 1 --2- 172.21.15.86:0/2569949845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b880d1c20 0x7f7b8806ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:59.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.248+0000 7f7b8da00640 1 -- 172.21.15.86:0/2569949845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b880d1c20 msgr2=0x7f7b8806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:59.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.248+0000 7f7b8da00640 1 --2- 172.21.15.86:0/2569949845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b880d1c20 0x7f7b8806ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:59.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.248+0000 7f7b8da00640 1 -- 172.21.15.86:0/2569949845 >> v2:172.21.15.155:3300/0 conn(0x7f7b8806f270 msgr2=0x7f7b8806f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:15:59.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.248+0000 7f7b8da00640 1 --2- 172.21.15.86:0/2569949845 >> v2:172.21.15.155:3300/0 conn(0x7f7b8806f270 0x7f7b8806f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:59.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.248+0000 7f7b8da00640 1 -- 172.21.15.86:0/2569949845 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b8815eb80 con 0x7f7b88070330 2024-04-25T14:15:59.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.248+0000 7f7b8d000640 1 --2- 172.21.15.86:0/2569949845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b880d1c20 0x7f7b8806ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:15:59.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.248+0000 7f7b8da00640 1 --2- 172.21.15.86:0/2569949845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b88070330 0x7f7b880d16e0 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f7b8806e9e0 tx=0x7f7b7c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a9696295e7395c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:59.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.252+0000 7f7b87e00640 1 -- 172.21.15.86:0/2569949845 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c0ab600 con 0x7f7b88070330 2024-04-25T14:15:59.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.252+0000 7f7b87e00640 1 -- 172.21.15.86:0/2569949845 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b7c0b0070 con 0x7f7b88070330 2024-04-25T14:15:59.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.252+0000 7f7b87e00640 1 -- 172.21.15.86:0/2569949845 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c0ab9c0 con 0x7f7b88070330 2024-04-25T14:15:59.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.252+0000 7f7b90400640 1 -- 172.21.15.86:0/2569949845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b88070330 msgr2=0x7f7b880d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:59.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.252+0000 7f7b90400640 1 --2- 172.21.15.86:0/2569949845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b88070330 0x7f7b880d16e0 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f7b8806e9e0 tx=0x7f7b7c01c460 comp rx=0 tx=0).stop 2024-04-25T14:15:59.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.252+0000 7f7b90400640 1 -- 172.21.15.86:0/2569949845 shutdown_connections 2024-04-25T14:15:59.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.252+0000 7f7b90400640 1 --2- 172.21.15.86:0/2569949845 >> v2:172.21.15.155:3300/0 conn(0x7f7b8806f270 0x7f7b8806f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:59.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.252+0000 7f7b90400640 1 --2- 172.21.15.86:0/2569949845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b880d1c20 0x7f7b8806ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:59.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.252+0000 7f7b90400640 1 --2- 172.21.15.86:0/2569949845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b88070330 0x7f7b880d16e0 unknown :-1 s=CLOSED pgs=2250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:59.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.252+0000 7f7b90400640 1 -- 172.21.15.86:0/2569949845 >> 172.21.15.86:0/2569949845 conn(0x7f7b880b4650 msgr2=0x7f7b880b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:59.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.252+0000 7f7b90400640 1 -- 172.21.15.86:0/2569949845 shutdown_connections 2024-04-25T14:15:59.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.252+0000 7f7b90400640 1 -- 172.21.15.86:0/2569949845 wait complete. 2024-04-25T14:15:59.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.252+0000 7f7b90400640 1 Processor -- start 2024-04-25T14:15:59.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.252+0000 7f7b90400640 1 -- start start 2024-04-25T14:15:59.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.256+0000 7f7b90400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b8806f270 0x7f7b8815ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:59.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.256+0000 7f7b90400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b88070330 0x7f7b880b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:59.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.256+0000 7f7b90400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b880d1c20 0x7f7b880c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:59.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.256+0000 7f7b90400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8815f4b0 con 0x7f7b88070330 2024-04-25T14:15:59.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.256+0000 7f7b90400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8815f330 con 0x7f7b8806f270 2024-04-25T14:15:59.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.256+0000 7f7b90400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7b88070740 con 0x7f7b880d1c20 2024-04-25T14:15:59.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.256+0000 7f7b8d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b88070330 0x7f7b880b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:59.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.256+0000 7f7b8d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b88070330 0x7f7b880b05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52678/0 (socket says 172.21.15.86:52678) 2024-04-25T14:15:59.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.256+0000 7f7b8d000640 1 -- 172.21.15.86:0/2935746403 learned_addr learned my addr 172.21.15.86:0/2935746403 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:15:59.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.256+0000 7f7b8da00640 1 --2- 172.21.15.86:0/2935746403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b8806f270 0x7f7b8815ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:59.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.256+0000 7f7b8e400640 1 --2- 172.21.15.86:0/2935746403 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b880d1c20 0x7f7b880c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:59.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.256+0000 7f7b8d000640 1 -- 172.21.15.86:0/2935746403 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b880d1c20 msgr2=0x7f7b880c89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:59.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.256+0000 7f7b8d000640 1 --2- 172.21.15.86:0/2935746403 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b880d1c20 0x7f7b880c89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:59.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.256+0000 7f7b8d000640 1 -- 172.21.15.86:0/2935746403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b8806f270 msgr2=0x7f7b8815ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.256+0000 7f7b8d000640 1 --2- 172.21.15.86:0/2935746403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b8806f270 0x7f7b8815ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.256+0000 7f7b8d000640 1 -- 172.21.15.86:0/2935746403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b880b2340 con 0x7f7b88070330 2024-04-25T14:15:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.256+0000 7f7b8d000640 1 --2- 172.21.15.86:0/2935746403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b88070330 0x7f7b880b05b0 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400d830 tx=0x7f7b7400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.256+0000 7f7b85600640 1 -- 172.21.15.86:0/2935746403 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b74018020 con 0x7f7b88070330 2024-04-25T14:15:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.256+0000 7f7b85600640 1 -- 172.21.15.86:0/2935746403 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b740096c0 con 0x7f7b88070330 2024-04-25T14:15:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.256+0000 7f7b90400640 1 -- 172.21.15.86:0/2935746403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b880b12a0 con 0x7f7b88070330 2024-04-25T14:15:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.256+0000 7f7b85600640 1 -- 172.21.15.86:0/2935746403 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b74009ab0 con 0x7f7b88070330 2024-04-25T14:15:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.256+0000 7f7b90400640 1 -- 172.21.15.86:0/2935746403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b880b1860 con 0x7f7b88070330 2024-04-25T14:15:59.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.256+0000 7f7b90400640 1 -- 172.21.15.86:0/2935746403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b880b1010 con 0x7f7b88070330 2024-04-25T14:15:59.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.260+0000 7f7b85600640 1 -- 172.21.15.86:0/2935746403 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f7b7401b020 con 0x7f7b88070330 2024-04-25T14:15:59.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.260+0000 7f7b85600640 1 --2- 172.21.15.86:0/2935746403 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7b6806f450 0x7f7b68071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:15:59.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.260+0000 7f7b85600640 1 -- 172.21.15.86:0/2935746403 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f7b7408fd40 con 0x7f7b88070330 2024-04-25T14:15:59.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.260+0000 7f7b8da00640 1 --2- 172.21.15.86:0/2935746403 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7b6806f450 0x7f7b68071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:15:59.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.260+0000 7f7b8da00640 1 --2- 172.21.15.86:0/2935746403 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7b6806f450 0x7f7b68071960 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c0af000 tx=0x7f7b7c0a7000 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:15:59.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.264+0000 7f7b85600640 1 -- 172.21.15.86:0/2935746403 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7b74022020 con 0x7f7b88070330 2024-04-25T14:15:59.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.452+0000 7f7b90400640 1 -- 172.21.15.86:0/2935746403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b88069d00 con 0x7f7b88070330 2024-04-25T14:15:59.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.456+0000 7f7b85600640 1 -- 172.21.15.86:0/2935746403 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f7b7405c940 con 0x7f7b88070330 2024-04-25T14:15:59.452 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:15:59.452 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:15:59.452 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:15:59.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.456+0000 7f7b90400640 1 -- 172.21.15.86:0/2935746403 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7b6806f450 msgr2=0x7f7b68071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:59.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.456+0000 7f7b90400640 1 --2- 172.21.15.86:0/2935746403 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7b6806f450 0x7f7b68071960 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c0af000 tx=0x7f7b7c0a7000 comp rx=0 tx=0).stop 2024-04-25T14:15:59.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.456+0000 7f7b90400640 1 -- 172.21.15.86:0/2935746403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b88070330 msgr2=0x7f7b880b05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:15:59.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.456+0000 7f7b90400640 1 --2- 172.21.15.86:0/2935746403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b88070330 0x7f7b880b05b0 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400d830 tx=0x7f7b7400dcf0 comp rx=0 tx=0).stop 2024-04-25T14:15:59.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.456+0000 7f7b90400640 1 -- 172.21.15.86:0/2935746403 shutdown_connections 2024-04-25T14:15:59.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.456+0000 7f7b90400640 1 --2- 172.21.15.86:0/2935746403 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7b6806f450 0x7f7b68071960 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:59.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.456+0000 7f7b90400640 1 --2- 172.21.15.86:0/2935746403 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b880d1c20 0x7f7b880c89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:59.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.456+0000 7f7b90400640 1 --2- 172.21.15.86:0/2935746403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b88070330 0x7f7b880b05b0 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:59.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.456+0000 7f7b90400640 1 --2- 172.21.15.86:0/2935746403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b8806f270 0x7f7b8815ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:15:59.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.456+0000 7f7b90400640 1 -- 172.21.15.86:0/2935746403 >> 172.21.15.86:0/2935746403 conn(0x7f7b880b4650 msgr2=0x7f7b88073a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:15:59.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.456+0000 7f7b90400640 1 -- 172.21.15.86:0/2935746403 shutdown_connections 2024-04-25T14:15:59.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:15:59.456+0000 7f7b90400640 1 -- 172.21.15.86:0/2935746403 wait complete. 2024-04-25T14:15:59.876 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:15:59.876 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:00.877 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:01.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.036+0000 7f4085e00640 1 Processor -- start 2024-04-25T14:16:01.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.036+0000 7f4085e00640 1 -- start start 2024-04-25T14:16:01.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.036+0000 7f4085e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40800cdd70 0x7f40800ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:01.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.036+0000 7f4085e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4080070e20 0x7f4080071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:01.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.036+0000 7f4085e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4080071770 con 0x7f408006e700 2024-04-25T14:16:01.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.036+0000 7f4085e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f408006d940 con 0x7f4080070e20 2024-04-25T14:16:01.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.036+0000 7f4085e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f408006dc20 con 0x7f40800cdd70 2024-04-25T14:16:01.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.036+0000 7f407ea00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40800cdd70 0x7f40800ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:01.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.036+0000 7f4076000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4080070e20 0x7f4080071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:01.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.036+0000 7f407ea00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40800cdd70 0x7f40800ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:39448/0 (socket says 172.21.15.86:39448) 2024-04-25T14:16:01.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.036+0000 7f407ea00640 1 -- 172.21.15.86:0/1134999892 learned_addr learned my addr 172.21.15.86:0/1134999892 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:01.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.036+0000 7f4076000640 1 -- 172.21.15.86:0/1134999892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40800cdd70 msgr2=0x7f40800ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:01.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.036+0000 7f4076000640 1 --2- 172.21.15.86:0/1134999892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40800cdd70 0x7f40800ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:01.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.036+0000 7f4076000640 1 -- 172.21.15.86:0/1134999892 >> v1:172.21.15.155:6789/0 conn(0x7f408006e700 legacy=0x7f408006eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:01.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.036+0000 7f4076000640 1 -- 172.21.15.86:0/1134999892 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408006df00 con 0x7f4080070e20 2024-04-25T14:16:01.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.036+0000 7f407ea00640 1 --2- 172.21.15.86:0/1134999892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40800cdd70 0x7f40800ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:16:01.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.040+0000 7f4076000640 1 --2- 172.21.15.86:0/1134999892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4080070e20 0x7f4080071230 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f40680097a0 tx=0x7f406801c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8bf0cc30ac53a3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:01.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.040+0000 7f407e000640 1 -- 172.21.15.86:0/1134999892 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40680b0070 con 0x7f4080070e20 2024-04-25T14:16:01.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.040+0000 7f407e000640 1 -- 172.21.15.86:0/1134999892 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40680ab600 con 0x7f4080070e20 2024-04-25T14:16:01.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.040+0000 7f407e000640 1 -- 172.21.15.86:0/1134999892 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40680ab9d0 con 0x7f4080070e20 2024-04-25T14:16:01.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.040+0000 7f4085e00640 1 -- 172.21.15.86:0/1134999892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4080070e20 msgr2=0x7f4080071230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:01.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.040+0000 7f4085e00640 1 --2- 172.21.15.86:0/1134999892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4080070e20 0x7f4080071230 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f40680097a0 tx=0x7f406801c3f0 comp rx=0 tx=0).stop 2024-04-25T14:16:01.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.040+0000 7f4085e00640 1 -- 172.21.15.86:0/1134999892 shutdown_connections 2024-04-25T14:16:01.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.040+0000 7f4085e00640 1 --2- 172.21.15.86:0/1134999892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4080070e20 0x7f4080071230 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:01.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.040+0000 7f4085e00640 1 --2- 172.21.15.86:0/1134999892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40800cdd70 0x7f40800ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:01.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.040+0000 7f4085e00640 1 -- 172.21.15.86:0/1134999892 >> 172.21.15.86:0/1134999892 conn(0x7f40800cffc0 msgr2=0x7f40800d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:01.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.040+0000 7f4085e00640 1 -- 172.21.15.86:0/1134999892 shutdown_connections 2024-04-25T14:16:01.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.040+0000 7f4085e00640 1 -- 172.21.15.86:0/1134999892 wait complete. 2024-04-25T14:16:01.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.044+0000 7f4085e00640 1 Processor -- start 2024-04-25T14:16:01.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.044+0000 7f4085e00640 1 -- start start 2024-04-25T14:16:01.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.044+0000 7f4085e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f408006e700 0x7f40801534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:01.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.044+0000 7f4085e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4080070e20 0x7f408015c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:01.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.044+0000 7f4085e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40800cdd70 0x7f40801f9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:01.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.044+0000 7f4085e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f408016cc00 con 0x7f4080070e20 2024-04-25T14:16:01.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.044+0000 7f4085e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4080164400 con 0x7f408006e700 2024-04-25T14:16:01.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.044+0000 7f4085e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f408016cd80 con 0x7f40800cdd70 2024-04-25T14:16:01.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.044+0000 7f4076000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4080070e20 0x7f408015c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:01.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.044+0000 7f4076000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4080070e20 0x7f408015c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52686/0 (socket says 172.21.15.86:52686) 2024-04-25T14:16:01.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.044+0000 7f4076000640 1 -- 172.21.15.86:0/613192516 learned_addr learned my addr 172.21.15.86:0/613192516 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:01.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.048+0000 7f407f400640 1 --2- 172.21.15.86:0/613192516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40800cdd70 0x7f40801f9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:01.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.048+0000 7f407ea00640 1 --2- 172.21.15.86:0/613192516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f408006e700 0x7f40801534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:01.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.048+0000 7f4076000640 1 -- 172.21.15.86:0/613192516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40800cdd70 msgr2=0x7f40801f9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:01.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.048+0000 7f4076000640 1 --2- 172.21.15.86:0/613192516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40800cdd70 0x7f40801f9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:01.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.048+0000 7f4076000640 1 -- 172.21.15.86:0/613192516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f408006e700 msgr2=0x7f40801534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:01.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.048+0000 7f4076000640 1 --2- 172.21.15.86:0/613192516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f408006e700 0x7f40801534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:01.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.048+0000 7f4076000640 1 -- 172.21.15.86:0/613192516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4080153a30 con 0x7f4080070e20 2024-04-25T14:16:01.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.048+0000 7f4076000640 1 --2- 172.21.15.86:0/613192516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4080070e20 0x7f408015c4f0 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f40680ae040 tx=0x7f4068019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:01.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.048+0000 7f4077e00640 1 -- 172.21.15.86:0/613192516 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40680b0040 con 0x7f4080070e20 2024-04-25T14:16:01.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.048+0000 7f4077e00640 1 -- 172.21.15.86:0/613192516 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40680ab400 con 0x7f4080070e20 2024-04-25T14:16:01.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.048+0000 7f4085e00640 1 -- 172.21.15.86:0/613192516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4080153c20 con 0x7f4080070e20 2024-04-25T14:16:01.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.048+0000 7f4077e00640 1 -- 172.21.15.86:0/613192516 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4068006dc0 con 0x7f4080070e20 2024-04-25T14:16:01.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.048+0000 7f4085e00640 1 -- 172.21.15.86:0/613192516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4080154400 con 0x7f4080070e20 2024-04-25T14:16:01.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.048+0000 7f4085e00640 1 -- 172.21.15.86:0/613192516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4080153e90 con 0x7f4080070e20 2024-04-25T14:16:01.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.048+0000 7f4077e00640 1 -- 172.21.15.86:0/613192516 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f40680bb020 con 0x7f4080070e20 2024-04-25T14:16:01.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.052+0000 7f4077e00640 1 --2- 172.21.15.86:0/613192516 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f405c06f450 0x7f405c071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:01.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.052+0000 7f4077e00640 1 -- 172.21.15.86:0/613192516 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f4068128840 con 0x7f4080070e20 2024-04-25T14:16:01.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.052+0000 7f407ea00640 1 --2- 172.21.15.86:0/613192516 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f405c06f450 0x7f405c071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:01.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.052+0000 7f407ea00640 1 --2- 172.21.15.86:0/613192516 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f405c06f450 0x7f405c071960 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4070006cb0 tx=0x7f4070009240 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:01.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.056+0000 7f4077e00640 1 -- 172.21.15.86:0/613192516 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f40680c2020 con 0x7f4080070e20 2024-04-25T14:16:01.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.244+0000 7f4085e00640 1 -- 172.21.15.86:0/613192516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40800720e0 con 0x7f4080070e20 2024-04-25T14:16:01.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.244+0000 7f4077e00640 1 -- 172.21.15.86:0/613192516 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f4068016070 con 0x7f4080070e20 2024-04-25T14:16:01.243 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:01.243 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:01.243 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:16:01.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.248+0000 7f4085e00640 1 -- 172.21.15.86:0/613192516 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f405c06f450 msgr2=0x7f405c071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:01.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.248+0000 7f4085e00640 1 --2- 172.21.15.86:0/613192516 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f405c06f450 0x7f405c071960 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4070006cb0 tx=0x7f4070009240 comp rx=0 tx=0).stop 2024-04-25T14:16:01.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.248+0000 7f4085e00640 1 -- 172.21.15.86:0/613192516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4080070e20 msgr2=0x7f408015c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:01.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.248+0000 7f4085e00640 1 --2- 172.21.15.86:0/613192516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4080070e20 0x7f408015c4f0 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f40680ae040 tx=0x7f4068019190 comp rx=0 tx=0).stop 2024-04-25T14:16:01.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.248+0000 7f4085e00640 1 -- 172.21.15.86:0/613192516 shutdown_connections 2024-04-25T14:16:01.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.248+0000 7f4085e00640 1 --2- 172.21.15.86:0/613192516 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f405c06f450 0x7f405c071960 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:01.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.248+0000 7f4085e00640 1 --2- 172.21.15.86:0/613192516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40800cdd70 0x7f40801f9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:01.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.248+0000 7f4085e00640 1 --2- 172.21.15.86:0/613192516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4080070e20 0x7f408015c4f0 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:01.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.248+0000 7f4085e00640 1 --2- 172.21.15.86:0/613192516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f408006e700 0x7f40801534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:01.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.248+0000 7f4085e00640 1 -- 172.21.15.86:0/613192516 >> 172.21.15.86:0/613192516 conn(0x7f40800cffc0 msgr2=0x7f40800b4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:01.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.248+0000 7f4085e00640 1 -- 172.21.15.86:0/613192516 shutdown_connections 2024-04-25T14:16:01.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:01.248+0000 7f4085e00640 1 -- 172.21.15.86:0/613192516 wait complete. 2024-04-25T14:16:01.671 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:01.672 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:02.673 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:02.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.828+0000 7f5522c00640 1 Processor -- start 2024-04-25T14:16:02.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.828+0000 7f5522c00640 1 -- start start 2024-04-25T14:16:02.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.832+0000 7f5522c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f551c070330 0x7f551c0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:02.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.832+0000 7f5522c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f551c0d1c20 0x7f551c06ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:02.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.832+0000 7f5522c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f551c06f270 0x7f551c06f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:02.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.832+0000 7f5522c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f551c0d2030 con 0x7f551c070330 2024-04-25T14:16:02.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.832+0000 7f5522c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f551c15e5c0 con 0x7f551c06f270 2024-04-25T14:16:02.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.832+0000 7f5522c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f551c15e8a0 con 0x7f551c0d1c20 2024-04-25T14:16:02.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.832+0000 7f551be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f551c070330 0x7f551c0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:02.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.832+0000 7f551be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f551c070330 0x7f551c0d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52698/0 (socket says 172.21.15.86:52698) 2024-04-25T14:16:02.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.832+0000 7f551be00640 1 -- 172.21.15.86:0/3881556799 learned_addr learned my addr 172.21.15.86:0/3881556799 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:02.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.832+0000 7f551be00640 1 -- 172.21.15.86:0/3881556799 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f551c0d1c20 msgr2=0x7f551c06ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:02.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.832+0000 7f5520c00640 1 --2- 172.21.15.86:0/3881556799 >> v2:172.21.15.155:3300/0 conn(0x7f551c06f270 0x7f551c06f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:02.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.832+0000 7f5513400640 1 --2- 172.21.15.86:0/3881556799 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f551c0d1c20 0x7f551c06ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:02.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.832+0000 7f551be00640 1 --2- 172.21.15.86:0/3881556799 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f551c0d1c20 0x7f551c06ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:02.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.832+0000 7f551be00640 1 -- 172.21.15.86:0/3881556799 >> v2:172.21.15.155:3300/0 conn(0x7f551c06f270 msgr2=0x7f551c06f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:02.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.832+0000 7f551be00640 1 --2- 172.21.15.86:0/3881556799 >> v2:172.21.15.155:3300/0 conn(0x7f551c06f270 0x7f551c06f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:02.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.832+0000 7f551be00640 1 -- 172.21.15.86:0/3881556799 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551c15eb80 con 0x7f551c070330 2024-04-25T14:16:02.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.832+0000 7f5513400640 1 --2- 172.21.15.86:0/3881556799 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f551c0d1c20 0x7f551c06ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:16:02.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.832+0000 7f5520c00640 1 --2- 172.21.15.86:0/3881556799 >> v2:172.21.15.155:3300/0 conn(0x7f551c06f270 0x7f551c06f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:16:02.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.836+0000 7f551be00640 1 --2- 172.21.15.86:0/3881556799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f551c070330 0x7f551c0d16e0 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f551c06e9e0 tx=0x7f550c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c7e276b4acd0fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:02.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.836+0000 7f551b400640 1 -- 172.21.15.86:0/3881556799 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c0a5790 con 0x7f551c070330 2024-04-25T14:16:02.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.836+0000 7f551b400640 1 -- 172.21.15.86:0/3881556799 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f550c0a5930 con 0x7f551c070330 2024-04-25T14:16:02.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.836+0000 7f551b400640 1 -- 172.21.15.86:0/3881556799 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c0a5d00 con 0x7f551c070330 2024-04-25T14:16:02.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.836+0000 7f5522c00640 1 -- 172.21.15.86:0/3881556799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f551c070330 msgr2=0x7f551c0d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:02.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.836+0000 7f5522c00640 1 --2- 172.21.15.86:0/3881556799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f551c070330 0x7f551c0d16e0 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f551c06e9e0 tx=0x7f550c01c930 comp rx=0 tx=0).stop 2024-04-25T14:16:02.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.836+0000 7f5522c00640 1 -- 172.21.15.86:0/3881556799 shutdown_connections 2024-04-25T14:16:02.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.836+0000 7f5522c00640 1 --2- 172.21.15.86:0/3881556799 >> v2:172.21.15.155:3300/0 conn(0x7f551c06f270 0x7f551c06f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:02.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.836+0000 7f5522c00640 1 --2- 172.21.15.86:0/3881556799 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f551c0d1c20 0x7f551c06ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:02.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.836+0000 7f5522c00640 1 --2- 172.21.15.86:0/3881556799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f551c070330 0x7f551c0d16e0 unknown :-1 s=CLOSED pgs=2253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:02.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.836+0000 7f5522c00640 1 -- 172.21.15.86:0/3881556799 >> 172.21.15.86:0/3881556799 conn(0x7f551c0b4650 msgr2=0x7f551c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:02.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.836+0000 7f5522c00640 1 -- 172.21.15.86:0/3881556799 shutdown_connections 2024-04-25T14:16:02.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.836+0000 7f5522c00640 1 -- 172.21.15.86:0/3881556799 wait complete. 2024-04-25T14:16:02.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.836+0000 7f5522c00640 1 Processor -- start 2024-04-25T14:16:02.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.836+0000 7f5522c00640 1 -- start start 2024-04-25T14:16:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.840+0000 7f5522c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f551c06f270 0x7f551c15cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.840+0000 7f5522c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f551c070330 0x7f551c167eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.840+0000 7f5522c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f551c0d1c20 0x7f551c0c3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.840+0000 7f5522c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f551c15f4b0 con 0x7f551c0d1c20 2024-04-25T14:16:02.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.840+0000 7f5522c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f551c15f330 con 0x7f551c070330 2024-04-25T14:16:02.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.840+0000 7f5522c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f551c070740 con 0x7f551c06f270 2024-04-25T14:16:02.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.840+0000 7f5520c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f551c0d1c20 0x7f551c0c3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:02.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.840+0000 7f5520c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f551c0d1c20 0x7f551c0c3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52704/0 (socket says 172.21.15.86:52704) 2024-04-25T14:16:02.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.840+0000 7f5520c00640 1 -- 172.21.15.86:0/2990665186 learned_addr learned my addr 172.21.15.86:0/2990665186 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:02.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.840+0000 7f551be00640 1 --2- 172.21.15.86:0/2990665186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f551c06f270 0x7f551c15cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:02.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.840+0000 7f5513400640 1 --2- 172.21.15.86:0/2990665186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f551c070330 0x7f551c167eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:02.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.840+0000 7f5520c00640 1 -- 172.21.15.86:0/2990665186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f551c06f270 msgr2=0x7f551c15cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:02.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.840+0000 7f5520c00640 1 --2- 172.21.15.86:0/2990665186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f551c06f270 0x7f551c15cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:02.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.840+0000 7f5520c00640 1 -- 172.21.15.86:0/2990665186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f551c070330 msgr2=0x7f551c167eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:02.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.840+0000 7f5520c00640 1 --2- 172.21.15.86:0/2990665186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f551c070330 0x7f551c167eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:02.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.840+0000 7f5520c00640 1 -- 172.21.15.86:0/2990665186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551c0c9320 con 0x7f551c0d1c20 2024-04-25T14:16:02.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.840+0000 7f551be00640 1 --2- 172.21.15.86:0/2990665186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f551c06f270 0x7f551c15cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:16:02.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.840+0000 7f5520c00640 1 --2- 172.21.15.86:0/2990665186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f551c0d1c20 0x7f551c0c3b90 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f551400c540 tx=0x7f551400ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:02.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.840+0000 7f5518c00640 1 -- 172.21.15.86:0/2990665186 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5514015020 con 0x7f551c0d1c20 2024-04-25T14:16:02.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.840+0000 7f5518c00640 1 -- 172.21.15.86:0/2990665186 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5514004440 con 0x7f551c0d1c20 2024-04-25T14:16:02.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.840+0000 7f5522c00640 1 -- 172.21.15.86:0/2990665186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f551c15d1c0 con 0x7f551c0d1c20 2024-04-25T14:16:02.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.840+0000 7f5518c00640 1 -- 172.21.15.86:0/2990665186 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5514003840 con 0x7f551c0d1c20 2024-04-25T14:16:02.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.840+0000 7f5522c00640 1 -- 172.21.15.86:0/2990665186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f551c0cf3a0 con 0x7f551c0d1c20 2024-04-25T14:16:02.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.840+0000 7f5522c00640 1 -- 172.21.15.86:0/2990665186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f551c069d00 con 0x7f551c0d1c20 2024-04-25T14:16:02.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.844+0000 7f5518c00640 1 -- 172.21.15.86:0/2990665186 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f5514018020 con 0x7f551c0d1c20 2024-04-25T14:16:02.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.844+0000 7f5518c00640 1 --2- 172.21.15.86:0/2990665186 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f54f806f480 0x7f54f8071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:02.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.844+0000 7f5518c00640 1 -- 172.21.15.86:0/2990665186 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f551408f4a0 con 0x7f551c0d1c20 2024-04-25T14:16:02.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.844+0000 7f551be00640 1 --2- 172.21.15.86:0/2990665186 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f54f806f480 0x7f54f8071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:02.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.844+0000 7f551be00640 1 --2- 172.21.15.86:0/2990665186 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f54f806f480 0x7f54f8071990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f550c0ae000 tx=0x7f550c0a7070 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:02.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:02.848+0000 7f5518c00640 1 -- 172.21.15.86:0/2990665186 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f551401f020 con 0x7f551c0d1c20 2024-04-25T14:16:03.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:03.036+0000 7f5522c00640 1 -- 172.21.15.86:0/2990665186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f551c076bf0 con 0x7f551c0d1c20 2024-04-25T14:16:03.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:03.036+0000 7f5518c00640 1 -- 172.21.15.86:0/2990665186 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f551405c0a0 con 0x7f551c0d1c20 2024-04-25T14:16:03.034 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:03.035 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:03.035 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:16:03.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:03.040+0000 7f5522c00640 1 -- 172.21.15.86:0/2990665186 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f54f806f480 msgr2=0x7f54f8071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:03.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:03.040+0000 7f5522c00640 1 --2- 172.21.15.86:0/2990665186 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f54f806f480 0x7f54f8071990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f550c0ae000 tx=0x7f550c0a7070 comp rx=0 tx=0).stop 2024-04-25T14:16:03.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:03.040+0000 7f5522c00640 1 -- 172.21.15.86:0/2990665186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f551c0d1c20 msgr2=0x7f551c0c3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:03.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:03.040+0000 7f5522c00640 1 --2- 172.21.15.86:0/2990665186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f551c0d1c20 0x7f551c0c3b90 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f551400c540 tx=0x7f551400ca00 comp rx=0 tx=0).stop 2024-04-25T14:16:03.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:03.040+0000 7f5522c00640 1 -- 172.21.15.86:0/2990665186 shutdown_connections 2024-04-25T14:16:03.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:03.040+0000 7f5522c00640 1 --2- 172.21.15.86:0/2990665186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f551c0d1c20 0x7f551c0c3b90 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:03.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:03.040+0000 7f5522c00640 1 --2- 172.21.15.86:0/2990665186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f551c070330 0x7f551c167eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:03.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:03.040+0000 7f5522c00640 1 --2- 172.21.15.86:0/2990665186 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f54f806f480 0x7f54f8071990 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:03.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:03.040+0000 7f5522c00640 1 --2- 172.21.15.86:0/2990665186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f551c06f270 0x7f551c15cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:03.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:03.040+0000 7f5522c00640 1 -- 172.21.15.86:0/2990665186 >> 172.21.15.86:0/2990665186 conn(0x7f551c0b4650 msgr2=0x7f551c15af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:03.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:03.040+0000 7f5522c00640 1 -- 172.21.15.86:0/2990665186 shutdown_connections 2024-04-25T14:16:03.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:03.040+0000 7f5522c00640 1 -- 172.21.15.86:0/2990665186 wait complete. 2024-04-25T14:16:03.459 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:03.499 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:04.500 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:04.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.664+0000 7f8f98a00640 1 Processor -- start 2024-04-25T14:16:04.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.664+0000 7f8f98a00640 1 -- start start 2024-04-25T14:16:04.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.664+0000 7f8f98a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8f940b3a90 0x7f8f94167ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:04.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.664+0000 7f8f98a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f9415d180 con 0x7f8f940d2960 2024-04-25T14:16:04.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.664+0000 7f8f98a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f9415d440 con 0x7f8f940b3680 2024-04-25T14:16:04.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.664+0000 7f8f98a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f8f9415d720 con 0x7f8f940b3a90 2024-04-25T14:16:04.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.664+0000 7f8f91e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8f940d2960 0x7f8f94164790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33500/0 (socket says 172.21.15.86:33500) 2024-04-25T14:16:04.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.664+0000 7f8f91e00640 1 -- 172.21.15.86:0/2937643929 learned_addr learned my addr 172.21.15.86:0/2937643929 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:16:04.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.664+0000 7f8f93200640 1 --2- 172.21.15.86:0/2937643929 >> v2:172.21.15.155:3300/0 conn(0x7f8f940b3a90 0x7f8f94167ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:04.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.664+0000 7f8f91400640 1 -- 172.21.15.86:0/2937643929 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2842143871 0 0) 0x7f8f9415d180 con 0x7f8f940d2960 2024-04-25T14:16:04.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.664+0000 7f8f91400640 1 -- 172.21.15.86:0/2937643929 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f78005740 con 0x7f8f940d2960 2024-04-25T14:16:04.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.664+0000 7f8f91400640 1 -- 172.21.15.86:0/2937643929 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2112733541 0 0) 0x7f8f78005740 con 0x7f8f940d2960 2024-04-25T14:16:04.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.664+0000 7f8f91400640 1 -- 172.21.15.86:0/2937643929 >> v2:172.21.15.155:3300/0 conn(0x7f8f940b3a90 msgr2=0x7f8f94167ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:04.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.664+0000 7f8f91400640 1 --2- 172.21.15.86:0/2937643929 >> v2:172.21.15.155:3300/0 conn(0x7f8f940b3a90 0x7f8f94167ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:04.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.664+0000 7f8f91400640 1 -- 172.21.15.86:0/2937643929 >> v1:172.21.15.155:6789/0 conn(0x7f8f940b3680 legacy=0x7f8f940d2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:04.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.664+0000 7f8f91400640 1 -- 172.21.15.86:0/2937643929 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f9415da00 con 0x7f8f940d2960 2024-04-25T14:16:04.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.664+0000 7f8f91400640 1 -- 172.21.15.86:0/2937643929 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f8f80001680 con 0x7f8f940d2960 2024-04-25T14:16:04.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.664+0000 7f8f93200640 1 --2- 172.21.15.86:0/2937643929 >> v2:172.21.15.155:3300/0 conn(0x7f8f940b3a90 0x7f8f94167ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:16:04.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.664+0000 7f8f91400640 1 -- 172.21.15.86:0/2937643929 >> v1:172.21.15.86:6789/0 conn(0x7f8f940d2960 legacy=0x7f8f94164790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:04.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.664+0000 7f8f91400640 1 --2- 172.21.15.86:0/2937643929 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f78005bd0 0x7f8f78006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:04.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.664+0000 7f8f91400640 1 --2- 172.21.15.86:0/2937643929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f78006a20 0x7f8f78006e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:04.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.668+0000 7f8f91400640 1 --2- 172.21.15.86:0/2937643929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f78007470 0x7f8f7800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:04.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.668+0000 7f8f91400640 1 -- 172.21.15.86:0/2937643929 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9415d720 con 0x7f8f78007470 2024-04-25T14:16:04.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.668+0000 7f8f91400640 1 -- 172.21.15.86:0/2937643929 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7800e400 con 0x7f8f78006a20 2024-04-25T14:16:04.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.668+0000 7f8f91400640 1 -- 172.21.15.86:0/2937643929 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8f7800e600 con 0x7f8f78005bd0 2024-04-25T14:16:04.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.668+0000 7f8f93200640 1 --2- 172.21.15.86:0/2937643929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f78007470 0x7f8f7800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:04.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.668+0000 7f8f91e00640 1 --2- 172.21.15.86:0/2937643929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f78006a20 0x7f8f78006e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:04.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.668+0000 7f8f92800640 1 --2- 172.21.15.86:0/2937643929 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f78005bd0 0x7f8f78006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:04.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.668+0000 7f8f93200640 1 -- 172.21.15.86:0/2937643929 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f78005bd0 msgr2=0x7f8f78006040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:04.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.668+0000 7f8f93200640 1 --2- 172.21.15.86:0/2937643929 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f78005bd0 0x7f8f78006040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:04.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.668+0000 7f8f93200640 1 -- 172.21.15.86:0/2937643929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f78006a20 msgr2=0x7f8f78006e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:04.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.668+0000 7f8f93200640 1 --2- 172.21.15.86:0/2937643929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f78006a20 0x7f8f78006e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:04.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.668+0000 7f8f93200640 1 -- 172.21.15.86:0/2937643929 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f78005740 con 0x7f8f78007470 2024-04-25T14:16:04.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.668+0000 7f8f91e00640 1 --2- 172.21.15.86:0/2937643929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f78006a20 0x7f8f78006e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:16:04.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.668+0000 7f8f92800640 1 --2- 172.21.15.86:0/2937643929 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f78005bd0 0x7f8f78006040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:16:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.668+0000 7f8f93200640 1 -- 172.21.15.86:0/2937643929 reap_dead start 2024-04-25T14:16:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.668+0000 7f8f93200640 1 --2- 172.21.15.86:0/2937643929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f78007470 0x7f8f7800dbe0 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f8f940b7da0 tx=0x7f8f8c01ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e13566c36d0f1c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.668+0000 7f8f91400640 1 -- 172.21.15.86:0/2937643929 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c009de0 con 0x7f8f78007470 2024-04-25T14:16:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.668+0000 7f8f91400640 1 -- 172.21.15.86:0/2937643929 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f8c0a5e30 con 0x7f8f78007470 2024-04-25T14:16:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.668+0000 7f8f91400640 1 -- 172.21.15.86:0/2937643929 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c003bc0 con 0x7f8f78007470 2024-04-25T14:16:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.672+0000 7f8f98a00640 1 -- 172.21.15.86:0/2937643929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f78007470 msgr2=0x7f8f7800dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.672+0000 7f8f98a00640 1 --2- 172.21.15.86:0/2937643929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f78007470 0x7f8f7800dbe0 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f8f940b7da0 tx=0x7f8f8c01ce20 comp rx=0 tx=0).stop 2024-04-25T14:16:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.672+0000 7f8f98a00640 1 -- 172.21.15.86:0/2937643929 shutdown_connections 2024-04-25T14:16:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.672+0000 7f8f98a00640 1 --2- 172.21.15.86:0/2937643929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f78007470 0x7f8f7800dbe0 unknown :-1 s=CLOSED pgs=2256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.672+0000 7f8f98a00640 1 -- 172.21.15.86:0/2937643929 >> 172.21.15.86:0/2937643929 conn(0x7f8f9409ba30 msgr2=0x7f8f940b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.672+0000 7f8f98a00640 1 -- 172.21.15.86:0/2937643929 shutdown_connections 2024-04-25T14:16:04.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.672+0000 7f8f98a00640 1 -- 172.21.15.86:0/2937643929 wait complete. 2024-04-25T14:16:04.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.672+0000 7f8f98a00640 1 Processor -- start 2024-04-25T14:16:04.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.672+0000 7f8f98a00640 1 -- start start 2024-04-25T14:16:04.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.676+0000 7f8f98a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f78007470 0x7f8f940cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:04.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.676+0000 7f8f98a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f940af240 0x7f8f94149fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:04.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.676+0000 7f8f98a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f9415f350 0x7f8f9415f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:04.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.676+0000 7f8f98a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f940d3080 con 0x7f8f940af240 2024-04-25T14:16:04.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.676+0000 7f8f98a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8f940d2f00 con 0x7f8f78007470 2024-04-25T14:16:04.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.676+0000 7f8f98a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8f9415a9f0 con 0x7f8f9415f350 2024-04-25T14:16:04.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.676+0000 7f8f91e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f940af240 0x7f8f94149fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:04.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.676+0000 7f8f91e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f940af240 0x7f8f94149fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52722/0 (socket says 172.21.15.86:52722) 2024-04-25T14:16:04.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.676+0000 7f8f91e00640 1 -- 172.21.15.86:0/2479634451 learned_addr learned my addr 172.21.15.86:0/2479634451 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:04.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.676+0000 7f8f93200640 1 --2- 172.21.15.86:0/2479634451 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f9415f350 0x7f8f9415f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:04.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.676+0000 7f8f91e00640 1 -- 172.21.15.86:0/2479634451 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f9415f350 msgr2=0x7f8f9415f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:04.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.676+0000 7f8f91e00640 1 --2- 172.21.15.86:0/2479634451 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f9415f350 0x7f8f9415f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:04.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.676+0000 7f8f91e00640 1 -- 172.21.15.86:0/2479634451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f78007470 msgr2=0x7f8f940cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:04.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.676+0000 7f8f92800640 1 --2- 172.21.15.86:0/2479634451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f78007470 0x7f8f940cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:04.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.676+0000 7f8f91e00640 1 --2- 172.21.15.86:0/2479634451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f78007470 0x7f8f940cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:04.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.676+0000 7f8f91e00640 1 -- 172.21.15.86:0/2479634451 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f9415fca0 con 0x7f8f940af240 2024-04-25T14:16:04.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.676+0000 7f8f93200640 1 --2- 172.21.15.86:0/2479634451 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f9415f350 0x7f8f9415f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:16:04.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.676+0000 7f8f92800640 1 --2- 172.21.15.86:0/2479634451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f78007470 0x7f8f940cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:16:04.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.676+0000 7f8f91e00640 1 --2- 172.21.15.86:0/2479634451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f940af240 0x7f8f94149fe0 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f8f80011870 tx=0x7f8f80011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:04.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.676+0000 7f8f8aa00640 1 -- 172.21.15.86:0/2479634451 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80010040 con 0x7f8f940af240 2024-04-25T14:16:04.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.676+0000 7f8f8aa00640 1 -- 172.21.15.86:0/2479634451 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f8000fb00 con 0x7f8f940af240 2024-04-25T14:16:04.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.676+0000 7f8f8aa00640 1 -- 172.21.15.86:0/2479634451 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80011e60 con 0x7f8f940af240 2024-04-25T14:16:04.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.676+0000 7f8f98a00640 1 -- 172.21.15.86:0/2479634451 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f940ce350 con 0x7f8f940af240 2024-04-25T14:16:04.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.676+0000 7f8f98a00640 1 -- 172.21.15.86:0/2479634451 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f940af9b0 con 0x7f8f940af240 2024-04-25T14:16:04.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.676+0000 7f8f98a00640 1 -- 172.21.15.86:0/2479634451 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f94076b80 con 0x7f8f940af240 2024-04-25T14:16:04.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.680+0000 7f8f8aa00640 1 -- 172.21.15.86:0/2479634451 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f8f80020470 con 0x7f8f940af240 2024-04-25T14:16:04.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.680+0000 7f8f8aa00640 1 --2- 172.21.15.86:0/2479634451 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8f7406f480 0x7f8f74071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:04.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.680+0000 7f8f8aa00640 1 -- 172.21.15.86:0/2479634451 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f8f800928f0 con 0x7f8f940af240 2024-04-25T14:16:04.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.680+0000 7f8f92800640 1 --2- 172.21.15.86:0/2479634451 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8f7406f480 0x7f8f74071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:04.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.680+0000 7f8f92800640 1 --2- 172.21.15.86:0/2479634451 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8f7406f480 0x7f8f74071990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8f84007fd0 tx=0x7f8f84009040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:04.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.684+0000 7f8f8aa00640 1 -- 172.21.15.86:0/2479634451 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8f9415fca0 con 0x7f8f940af240 2024-04-25T14:16:04.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.876+0000 7f8f98a00640 1 -- 172.21.15.86:0/2479634451 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f9406ed50 con 0x7f8f940af240 2024-04-25T14:16:04.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.876+0000 7f8f8aa00640 1 -- 172.21.15.86:0/2479634451 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f8f8005f460 con 0x7f8f940af240 2024-04-25T14:16:04.874 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:04.874 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:04.874 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:16:04.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.880+0000 7f8f98a00640 1 -- 172.21.15.86:0/2479634451 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8f7406f480 msgr2=0x7f8f74071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:04.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.880+0000 7f8f98a00640 1 --2- 172.21.15.86:0/2479634451 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8f7406f480 0x7f8f74071990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8f84007fd0 tx=0x7f8f84009040 comp rx=0 tx=0).stop 2024-04-25T14:16:04.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.880+0000 7f8f98a00640 1 -- 172.21.15.86:0/2479634451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f940af240 msgr2=0x7f8f94149fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:04.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.880+0000 7f8f98a00640 1 --2- 172.21.15.86:0/2479634451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f940af240 0x7f8f94149fe0 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f8f80011870 tx=0x7f8f80011d30 comp rx=0 tx=0).stop 2024-04-25T14:16:04.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.880+0000 7f8f98a00640 1 -- 172.21.15.86:0/2479634451 shutdown_connections 2024-04-25T14:16:04.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.880+0000 7f8f98a00640 1 --2- 172.21.15.86:0/2479634451 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8f7406f480 0x7f8f74071990 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:04.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.880+0000 7f8f98a00640 1 --2- 172.21.15.86:0/2479634451 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f9415f350 0x7f8f9415f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:04.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.880+0000 7f8f98a00640 1 --2- 172.21.15.86:0/2479634451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f940af240 0x7f8f94149fe0 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:04.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.880+0000 7f8f98a00640 1 --2- 172.21.15.86:0/2479634451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f78007470 0x7f8f940cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:04.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.880+0000 7f8f98a00640 1 -- 172.21.15.86:0/2479634451 >> 172.21.15.86:0/2479634451 conn(0x7f8f9409ba30 msgr2=0x7f8f940d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:04.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.880+0000 7f8f98a00640 1 -- 172.21.15.86:0/2479634451 shutdown_connections 2024-04-25T14:16:04.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:04.880+0000 7f8f98a00640 1 -- 172.21.15.86:0/2479634451 wait complete. 2024-04-25T14:16:05.298 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:05.298 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:06.300 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:06.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.452+0000 7efcf0a00640 1 Processor -- start 2024-04-25T14:16:06.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.452+0000 7efcf0a00640 1 -- start start 2024-04-25T14:16:06.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.452+0000 7efcf0a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efcec1610d0 0x7efcec0b7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:06.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.452+0000 7efcf0a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7efcec0b2250 0x7efcec0b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:06.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.452+0000 7efcf0a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efcec0b2ba0 0x7efcec0d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:06.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.452+0000 7efcf0a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7efcec0b7f70 con 0x7efcec1610d0 2024-04-25T14:16:06.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.452+0000 7efcf0a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7efcec161620 con 0x7efcec0b2250 2024-04-25T14:16:06.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.452+0000 7efcf0a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efcec0b3750 con 0x7efcec0b2ba0 2024-04-25T14:16:06.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.452+0000 7efce9e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efcec1610d0 0x7efcec0b7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:06.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.452+0000 7efce9e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efcec1610d0 0x7efcec0b7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52728/0 (socket says 172.21.15.86:52728) 2024-04-25T14:16:06.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.452+0000 7efce9e00640 1 -- 172.21.15.86:0/2224365525 learned_addr learned my addr 172.21.15.86:0/2224365525 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:06.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.456+0000 7efcea800640 1 --2- 172.21.15.86:0/2224365525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efcec0b2ba0 0x7efcec0d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:06.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.456+0000 7efce9e00640 1 -- 172.21.15.86:0/2224365525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efcec0b2ba0 msgr2=0x7efcec0d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:06.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.456+0000 7efce1400640 1 --2- 172.21.15.86:0/2224365525 >> v2:172.21.15.155:3300/0 conn(0x7efcec0b2250 0x7efcec0b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:06.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.456+0000 7efce9e00640 1 --2- 172.21.15.86:0/2224365525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efcec0b2ba0 0x7efcec0d15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:06.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.456+0000 7efce9e00640 1 -- 172.21.15.86:0/2224365525 >> v2:172.21.15.155:3300/0 conn(0x7efcec0b2250 msgr2=0x7efcec0b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:06.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.456+0000 7efce9e00640 1 --2- 172.21.15.86:0/2224365525 >> v2:172.21.15.155:3300/0 conn(0x7efcec0b2250 0x7efcec0b2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:06.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.456+0000 7efce9e00640 1 -- 172.21.15.86:0/2224365525 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcec0ce7b0 con 0x7efcec1610d0 2024-04-25T14:16:06.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.456+0000 7efcea800640 1 --2- 172.21.15.86:0/2224365525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efcec0b2ba0 0x7efcec0d15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:16:06.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.456+0000 7efce1400640 1 --2- 172.21.15.86:0/2224365525 >> v2:172.21.15.155:3300/0 conn(0x7efcec0b2250 0x7efcec0b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:16:06.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.456+0000 7efce9e00640 1 --2- 172.21.15.86:0/2224365525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcec1610d0 0x7efcec0b7a30 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7efcec0b52f0 tx=0x7efcdc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a789bdc5e9f400b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:06.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.456+0000 7efce9400640 1 -- 172.21.15.86:0/2224365525 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc0b0070 con 0x7efcec1610d0 2024-04-25T14:16:06.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.456+0000 7efce9400640 1 -- 172.21.15.86:0/2224365525 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcdc0ab600 con 0x7efcec1610d0 2024-04-25T14:16:06.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.456+0000 7efce9400640 1 -- 172.21.15.86:0/2224365525 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc0ab9d0 con 0x7efcec1610d0 2024-04-25T14:16:06.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.460+0000 7efcf0a00640 1 -- 172.21.15.86:0/2224365525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcec1610d0 msgr2=0x7efcec0b7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:06.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.460+0000 7efcf0a00640 1 --2- 172.21.15.86:0/2224365525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcec1610d0 0x7efcec0b7a30 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7efcec0b52f0 tx=0x7efcdc01c520 comp rx=0 tx=0).stop 2024-04-25T14:16:06.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.460+0000 7efcf0a00640 1 -- 172.21.15.86:0/2224365525 shutdown_connections 2024-04-25T14:16:06.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.460+0000 7efcf0a00640 1 --2- 172.21.15.86:0/2224365525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efcec0b2ba0 0x7efcec0d15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:06.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.460+0000 7efcf0a00640 1 --2- 172.21.15.86:0/2224365525 >> v2:172.21.15.155:3300/0 conn(0x7efcec0b2250 0x7efcec0b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:06.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.460+0000 7efcf0a00640 1 --2- 172.21.15.86:0/2224365525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcec1610d0 0x7efcec0b7a30 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:06.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.460+0000 7efcf0a00640 1 -- 172.21.15.86:0/2224365525 >> 172.21.15.86:0/2224365525 conn(0x7efcec09ba30 msgr2=0x7efcec0b7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:06.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.460+0000 7efcf0a00640 1 -- 172.21.15.86:0/2224365525 shutdown_connections 2024-04-25T14:16:06.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.460+0000 7efcf0a00640 1 -- 172.21.15.86:0/2224365525 wait complete. 2024-04-25T14:16:06.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.460+0000 7efcf0a00640 1 Processor -- start 2024-04-25T14:16:06.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.460+0000 7efcf0a00640 1 -- start start 2024-04-25T14:16:06.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.460+0000 7efcf0a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efcec0b2250 0x7efcec15cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:06.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.464+0000 7efcf0a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efcec0b2ba0 0x7efcec157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:06.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.464+0000 7efcf0a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcec1610d0 0x7efcec158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:06.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.464+0000 7efcf0a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efcec163d40 con 0x7efcec1610d0 2024-04-25T14:16:06.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.464+0000 7efcf0a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efcec163bc0 con 0x7efcec0b2250 2024-04-25T14:16:06.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.464+0000 7efcf0a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efcec163ec0 con 0x7efcec0b2ba0 2024-04-25T14:16:06.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.464+0000 7efcea800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcec1610d0 0x7efcec158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:06.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.464+0000 7efcea800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcec1610d0 0x7efcec158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52740/0 (socket says 172.21.15.86:52740) 2024-04-25T14:16:06.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.464+0000 7efcea800640 1 -- 172.21.15.86:0/42410162 learned_addr learned my addr 172.21.15.86:0/42410162 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:06.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.464+0000 7efce1400640 1 --2- 172.21.15.86:0/42410162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efcec0b2ba0 0x7efcec157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:06.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.464+0000 7efce9e00640 1 --2- 172.21.15.86:0/42410162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efcec0b2250 0x7efcec15cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:06.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.464+0000 7efcea800640 1 -- 172.21.15.86:0/42410162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efcec0b2ba0 msgr2=0x7efcec157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:06.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.464+0000 7efcea800640 1 --2- 172.21.15.86:0/42410162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efcec0b2ba0 0x7efcec157bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:06.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.464+0000 7efcea800640 1 -- 172.21.15.86:0/42410162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efcec0b2250 msgr2=0x7efcec15cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:06.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.464+0000 7efcea800640 1 --2- 172.21.15.86:0/42410162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efcec0b2250 0x7efcec15cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:06.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.464+0000 7efcea800640 1 -- 172.21.15.86:0/42410162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcec15c4e0 con 0x7efcec1610d0 2024-04-25T14:16:06.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.464+0000 7efcea800640 1 --2- 172.21.15.86:0/42410162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcec1610d0 0x7efcec158110 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7efce400cdb0 tx=0x7efce400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:06.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.464+0000 7efce2a00640 1 -- 172.21.15.86:0/42410162 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce400f680 con 0x7efcec1610d0 2024-04-25T14:16:06.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.464+0000 7efce2a00640 1 -- 172.21.15.86:0/42410162 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efce400f840 con 0x7efcec1610d0 2024-04-25T14:16:06.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.464+0000 7efce2a00640 1 -- 172.21.15.86:0/42410162 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce400fc30 con 0x7efcec1610d0 2024-04-25T14:16:06.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.464+0000 7efcf0a00640 1 -- 172.21.15.86:0/42410162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcec1630d0 con 0x7efcec1610d0 2024-04-25T14:16:06.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.464+0000 7efcf0a00640 1 -- 172.21.15.86:0/42410162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcec163690 con 0x7efcec1610d0 2024-04-25T14:16:06.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.464+0000 7efcf0a00640 1 -- 172.21.15.86:0/42410162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcec076bf0 con 0x7efcec1610d0 2024-04-25T14:16:06.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.468+0000 7efce2a00640 1 -- 172.21.15.86:0/42410162 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7efce40213f0 con 0x7efcec1610d0 2024-04-25T14:16:06.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.468+0000 7efce2a00640 1 --2- 172.21.15.86:0/42410162 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7efcc806f450 0x7efcc8071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:06.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.468+0000 7efce2a00640 1 -- 172.21.15.86:0/42410162 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7efce4090630 con 0x7efcec1610d0 2024-04-25T14:16:06.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.468+0000 7efce9e00640 1 --2- 172.21.15.86:0/42410162 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7efcc806f450 0x7efcc8071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:06.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.468+0000 7efce9e00640 1 --2- 172.21.15.86:0/42410162 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7efcc806f450 0x7efcc8071960 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0ad000 tx=0x7efcdc008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:06.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.472+0000 7efce2a00640 1 -- 172.21.15.86:0/42410162 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efce4014030 con 0x7efcec1610d0 2024-04-25T14:16:06.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.660+0000 7efcf0a00640 1 -- 172.21.15.86:0/42410162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efcec074750 con 0x7efcec1610d0 2024-04-25T14:16:06.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.664+0000 7efce2a00640 1 -- 172.21.15.86:0/42410162 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7efce405d2f0 con 0x7efcec1610d0 2024-04-25T14:16:06.660 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:06.660 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:06.660 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:16:06.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.664+0000 7efcf0a00640 1 -- 172.21.15.86:0/42410162 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7efcc806f450 msgr2=0x7efcc8071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:06.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.664+0000 7efcf0a00640 1 --2- 172.21.15.86:0/42410162 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7efcc806f450 0x7efcc8071960 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0ad000 tx=0x7efcdc008040 comp rx=0 tx=0).stop 2024-04-25T14:16:06.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.664+0000 7efcf0a00640 1 -- 172.21.15.86:0/42410162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcec1610d0 msgr2=0x7efcec158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:06.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.664+0000 7efcf0a00640 1 --2- 172.21.15.86:0/42410162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcec1610d0 0x7efcec158110 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7efce400cdb0 tx=0x7efce400a650 comp rx=0 tx=0).stop 2024-04-25T14:16:06.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.664+0000 7efcf0a00640 1 -- 172.21.15.86:0/42410162 shutdown_connections 2024-04-25T14:16:06.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.664+0000 7efcf0a00640 1 --2- 172.21.15.86:0/42410162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcec1610d0 0x7efcec158110 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:06.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.664+0000 7efcf0a00640 1 --2- 172.21.15.86:0/42410162 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7efcc806f450 0x7efcc8071960 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:06.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.664+0000 7efcf0a00640 1 --2- 172.21.15.86:0/42410162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efcec0b2ba0 0x7efcec157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:06.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.664+0000 7efcf0a00640 1 --2- 172.21.15.86:0/42410162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efcec0b2250 0x7efcec15cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:06.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.664+0000 7efcf0a00640 1 -- 172.21.15.86:0/42410162 >> 172.21.15.86:0/42410162 conn(0x7efcec09ba30 msgr2=0x7efcec0b4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:06.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.664+0000 7efcf0a00640 1 -- 172.21.15.86:0/42410162 shutdown_connections 2024-04-25T14:16:06.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:06.664+0000 7efcf0a00640 1 -- 172.21.15.86:0/42410162 wait complete. 2024-04-25T14:16:07.084 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:07.084 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:08.085 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:08.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.244+0000 7fbabf600640 1 Processor -- start 2024-04-25T14:16:08.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.244+0000 7fbabf600640 1 -- start start 2024-04-25T14:16:08.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.248+0000 7fbabf600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbab80d1df0 0x7fbab80cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:08.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.248+0000 7fbabf600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbab80cdcc0 0x7fbab80ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:08.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.248+0000 7fbabf600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbab815e5c0 con 0x7fbab8070330 2024-04-25T14:16:08.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.248+0000 7fbabf600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbab806fd40 con 0x7fbab80cdcc0 2024-04-25T14:16:08.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.248+0000 7fbabf600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fbab815e8a0 con 0x7fbab80d1df0 2024-04-25T14:16:08.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.248+0000 7fbabd600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbab80cdcc0 0x7fbab80ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:08.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.248+0000 7fbabd600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbab80cdcc0 0x7fbab80ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52752/0 (socket says 172.21.15.86:52752) 2024-04-25T14:16:08.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.248+0000 7fbabd600640 1 -- 172.21.15.86:0/72287071 learned_addr learned my addr 172.21.15.86:0/72287071 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:08.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.248+0000 7fbaafe00640 1 --2- 172.21.15.86:0/72287071 >> v2:172.21.15.155:3300/0 conn(0x7fbab80d1df0 0x7fbab80cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:08.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.248+0000 7fbabd600640 1 -- 172.21.15.86:0/72287071 >> v2:172.21.15.155:3300/0 conn(0x7fbab80d1df0 msgr2=0x7fbab80cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:08.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.248+0000 7fbabd600640 1 --2- 172.21.15.86:0/72287071 >> v2:172.21.15.155:3300/0 conn(0x7fbab80d1df0 0x7fbab80cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:08.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.248+0000 7fbabd600640 1 -- 172.21.15.86:0/72287071 >> v1:172.21.15.155:6789/0 conn(0x7fbab8070330 legacy=0x7fbab80d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:08.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.248+0000 7fbabd600640 1 -- 172.21.15.86:0/72287071 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab815eb80 con 0x7fbab80cdcc0 2024-04-25T14:16:08.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.252+0000 7fbabd600640 1 --2- 172.21.15.86:0/72287071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbab80cdcc0 0x7fbab80ce0d0 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7fbab0009e50 tx=0x7fbab001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6baae3df9202f3ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:08.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.252+0000 7fbab7e00640 1 -- 172.21.15.86:0/72287071 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab00a57d0 con 0x7fbab80cdcc0 2024-04-25T14:16:08.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.252+0000 7fbab7e00640 1 -- 172.21.15.86:0/72287071 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbab00a5970 con 0x7fbab80cdcc0 2024-04-25T14:16:08.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.252+0000 7fbab7e00640 1 -- 172.21.15.86:0/72287071 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab00a5d40 con 0x7fbab80cdcc0 2024-04-25T14:16:08.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.252+0000 7fbabf600640 1 -- 172.21.15.86:0/72287071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbab80cdcc0 msgr2=0x7fbab80ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:08.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.252+0000 7fbabf600640 1 --2- 172.21.15.86:0/72287071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbab80cdcc0 0x7fbab80ce0d0 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7fbab0009e50 tx=0x7fbab001c930 comp rx=0 tx=0).stop 2024-04-25T14:16:08.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.252+0000 7fbabf600640 1 -- 172.21.15.86:0/72287071 shutdown_connections 2024-04-25T14:16:08.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.252+0000 7fbabf600640 1 --2- 172.21.15.86:0/72287071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbab80cdcc0 0x7fbab80ce0d0 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:08.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.252+0000 7fbabf600640 1 --2- 172.21.15.86:0/72287071 >> v2:172.21.15.155:3300/0 conn(0x7fbab80d1df0 0x7fbab80cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:08.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.252+0000 7fbabf600640 1 -- 172.21.15.86:0/72287071 >> 172.21.15.86:0/72287071 conn(0x7fbab80b4650 msgr2=0x7fbab80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:08.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.252+0000 7fbabf600640 1 -- 172.21.15.86:0/72287071 shutdown_connections 2024-04-25T14:16:08.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.252+0000 7fbabf600640 1 -- 172.21.15.86:0/72287071 wait complete. 2024-04-25T14:16:08.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.256+0000 7fbabf600640 1 Processor -- start 2024-04-25T14:16:08.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.256+0000 7fbabf600640 1 -- start start 2024-04-25T14:16:08.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.256+0000 7fbabf600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbab8070330 0x7fbab80c41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:08.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.256+0000 7fbabf600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbab80cdcc0 0x7fbab80cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:08.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.256+0000 7fbabf600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbab80cb110 0x7fbab80d0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:08.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.256+0000 7fbabf600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbab815f4b0 con 0x7fbab80cdcc0 2024-04-25T14:16:08.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.256+0000 7fbabf600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbab815f330 con 0x7fbab8070330 2024-04-25T14:16:08.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.256+0000 7fbabf600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbab8070740 con 0x7fbab80cb110 2024-04-25T14:16:08.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.256+0000 7fbaafe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbab80cdcc0 0x7fbab80cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:08.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.256+0000 7fbaafe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbab80cdcc0 0x7fbab80cabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52766/0 (socket says 172.21.15.86:52766) 2024-04-25T14:16:08.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.256+0000 7fbaafe00640 1 -- 172.21.15.86:0/37445718 learned_addr learned my addr 172.21.15.86:0/37445718 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:08.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.256+0000 7fbabd600640 1 --2- 172.21.15.86:0/37445718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbab80cb110 0x7fbab80d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:08.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.256+0000 7fbabcc00640 1 --2- 172.21.15.86:0/37445718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbab8070330 0x7fbab80c41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:08.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.256+0000 7fbaafe00640 1 -- 172.21.15.86:0/37445718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbab80cb110 msgr2=0x7fbab80d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:08.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.256+0000 7fbaafe00640 1 --2- 172.21.15.86:0/37445718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbab80cb110 0x7fbab80d0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:08.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.256+0000 7fbaafe00640 1 -- 172.21.15.86:0/37445718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbab8070330 msgr2=0x7fbab80c41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:08.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.256+0000 7fbaafe00640 1 --2- 172.21.15.86:0/37445718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbab8070330 0x7fbab80c41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:08.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.256+0000 7fbaafe00640 1 -- 172.21.15.86:0/37445718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab80cb520 con 0x7fbab80cdcc0 2024-04-25T14:16:08.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.256+0000 7fbaafe00640 1 --2- 172.21.15.86:0/37445718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbab80cdcc0 0x7fbab80cabd0 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7fbaa000d5c0 tx=0x7fbaa000da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:08.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.256+0000 7fbab5600640 1 -- 172.21.15.86:0/37445718 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0004240 con 0x7fbab80cdcc0 2024-04-25T14:16:08.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.256+0000 7fbab5600640 1 -- 172.21.15.86:0/37445718 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaa00043e0 con 0x7fbab80cdcc0 2024-04-25T14:16:08.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.256+0000 7fbab5600640 1 -- 172.21.15.86:0/37445718 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0003840 con 0x7fbab80cdcc0 2024-04-25T14:16:08.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.256+0000 7fbabf600640 1 -- 172.21.15.86:0/37445718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab80af490 con 0x7fbab80cdcc0 2024-04-25T14:16:08.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.256+0000 7fbabf600640 1 -- 172.21.15.86:0/37445718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab80afa50 con 0x7fbab80cdcc0 2024-04-25T14:16:08.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.260+0000 7fbabf600640 1 -- 172.21.15.86:0/37445718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbab80af200 con 0x7fbab80cdcc0 2024-04-25T14:16:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.260+0000 7fbab5600640 1 -- 172.21.15.86:0/37445718 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7fbaa00039e0 con 0x7fbab80cdcc0 2024-04-25T14:16:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.260+0000 7fbab5600640 1 --2- 172.21.15.86:0/37445718 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fba9406f480 0x7fba94071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.260+0000 7fbab5600640 1 -- 172.21.15.86:0/37445718 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fbaa008f700 con 0x7fbab80cdcc0 2024-04-25T14:16:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.260+0000 7fbabcc00640 1 --2- 172.21.15.86:0/37445718 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fba9406f480 0x7fba94071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:08.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.260+0000 7fbabcc00640 1 --2- 172.21.15.86:0/37445718 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fba9406f480 0x7fba94071990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbab8072200 tx=0x7fbaa8009040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:08.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.264+0000 7fbab5600640 1 -- 172.21.15.86:0/37445718 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbaa005bcb0 con 0x7fbab80cdcc0 2024-04-25T14:16:08.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.452+0000 7fbabf600640 1 -- 172.21.15.86:0/37445718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbab80c9010 con 0x7fbab80cdcc0 2024-04-25T14:16:08.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.452+0000 7fbab5600640 1 -- 172.21.15.86:0/37445718 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7fbaa005ffb0 con 0x7fbab80cdcc0 2024-04-25T14:16:08.451 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:08.451 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:08.452 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:16:08.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.456+0000 7fbabf600640 1 -- 172.21.15.86:0/37445718 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fba9406f480 msgr2=0x7fba94071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:08.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.456+0000 7fbabf600640 1 --2- 172.21.15.86:0/37445718 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fba9406f480 0x7fba94071990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbab8072200 tx=0x7fbaa8009040 comp rx=0 tx=0).stop 2024-04-25T14:16:08.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.456+0000 7fbabf600640 1 -- 172.21.15.86:0/37445718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbab80cdcc0 msgr2=0x7fbab80cabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:08.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.456+0000 7fbabf600640 1 --2- 172.21.15.86:0/37445718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbab80cdcc0 0x7fbab80cabd0 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7fbaa000d5c0 tx=0x7fbaa000da80 comp rx=0 tx=0).stop 2024-04-25T14:16:08.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.456+0000 7fbabf600640 1 -- 172.21.15.86:0/37445718 shutdown_connections 2024-04-25T14:16:08.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.456+0000 7fbabf600640 1 --2- 172.21.15.86:0/37445718 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fba9406f480 0x7fba94071990 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:08.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.456+0000 7fbabf600640 1 --2- 172.21.15.86:0/37445718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbab80cb110 0x7fbab80d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:08.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.456+0000 7fbabf600640 1 --2- 172.21.15.86:0/37445718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbab80cdcc0 0x7fbab80cabd0 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:08.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.456+0000 7fbabf600640 1 --2- 172.21.15.86:0/37445718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbab8070330 0x7fbab80c41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:08.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.456+0000 7fbabf600640 1 -- 172.21.15.86:0/37445718 >> 172.21.15.86:0/37445718 conn(0x7fbab80b4650 msgr2=0x7fbab80c4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:08.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.456+0000 7fbabf600640 1 -- 172.21.15.86:0/37445718 shutdown_connections 2024-04-25T14:16:08.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:08.456+0000 7fbabf600640 1 -- 172.21.15.86:0/37445718 wait complete. 2024-04-25T14:16:08.876 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:08.877 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:09.878 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:10.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.040+0000 7f92f6a00640 1 Processor -- start 2024-04-25T14:16:10.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.040+0000 7f92f6a00640 1 -- start start 2024-04-25T14:16:10.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.040+0000 7f92f6a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f92f00cfca0 0x7f92f006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:10.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.040+0000 7f92f6a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92f00730b0 0x7f92f00734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:10.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.040+0000 7f92f6a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92f0072bd0 con 0x7f92f00d16e0 2024-04-25T14:16:10.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.040+0000 7f92f6a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f92f0070470 con 0x7f92f00cfca0 2024-04-25T14:16:10.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.040+0000 7f92f6a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f92f006f930 con 0x7f92f00730b0 2024-04-25T14:16:10.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.040+0000 7f92efe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f92f00cfca0 0x7f92f006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:10.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.040+0000 7f92efe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f92f00cfca0 0x7f92f006f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51060/0 (socket says 172.21.15.86:51060) 2024-04-25T14:16:10.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.040+0000 7f92efe00640 1 -- 172.21.15.86:0/1918407855 learned_addr learned my addr 172.21.15.86:0/1918407855 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:10.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.040+0000 7f92efe00640 1 -- 172.21.15.86:0/1918407855 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92f00730b0 msgr2=0x7f92f00734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:10.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.040+0000 7f92efe00640 1 --2- 172.21.15.86:0/1918407855 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92f00730b0 0x7f92f00734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:10.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.040+0000 7f92efe00640 1 -- 172.21.15.86:0/1918407855 >> v1:172.21.15.86:6789/0 conn(0x7f92f00d16e0 legacy=0x7f92f00d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:10.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.040+0000 7f92efe00640 1 -- 172.21.15.86:0/1918407855 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f006fbd0 con 0x7f92f00cfca0 2024-04-25T14:16:10.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.044+0000 7f92efe00640 1 --2- 172.21.15.86:0/1918407855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f00cfca0 0x7f92f006f3f0 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009870 tx=0x7f92e001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45614eecd98523a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:10.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.044+0000 7f92eea00640 1 -- 172.21.15.86:0/1918407855 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e00a7070 con 0x7f92f00cfca0 2024-04-25T14:16:10.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.044+0000 7f92eea00640 1 -- 172.21.15.86:0/1918407855 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92e00b0070 con 0x7f92f00cfca0 2024-04-25T14:16:10.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.044+0000 7f92eea00640 1 -- 172.21.15.86:0/1918407855 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e00ab890 con 0x7f92f00cfca0 2024-04-25T14:16:10.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.044+0000 7f92f6a00640 1 -- 172.21.15.86:0/1918407855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f00cfca0 msgr2=0x7f92f006f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:10.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.044+0000 7f92f6a00640 1 --2- 172.21.15.86:0/1918407855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f00cfca0 0x7f92f006f3f0 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009870 tx=0x7f92e001c560 comp rx=0 tx=0).stop 2024-04-25T14:16:10.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.044+0000 7f92f6a00640 1 -- 172.21.15.86:0/1918407855 shutdown_connections 2024-04-25T14:16:10.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.044+0000 7f92f6a00640 1 --2- 172.21.15.86:0/1918407855 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92f00730b0 0x7f92f00734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:10.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.044+0000 7f92f6a00640 1 --2- 172.21.15.86:0/1918407855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f00cfca0 0x7f92f006f3f0 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:10.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.044+0000 7f92f6a00640 1 -- 172.21.15.86:0/1918407855 >> 172.21.15.86:0/1918407855 conn(0x7f92f00b4650 msgr2=0x7f92f00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:10.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.044+0000 7f92f6a00640 1 -- 172.21.15.86:0/1918407855 shutdown_connections 2024-04-25T14:16:10.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.044+0000 7f92f6a00640 1 -- 172.21.15.86:0/1918407855 wait complete. 2024-04-25T14:16:10.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.048+0000 7f92f6a00640 1 Processor -- start 2024-04-25T14:16:10.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.048+0000 7f92f6a00640 1 -- start start 2024-04-25T14:16:10.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.048+0000 7f92f6a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92f00730b0 0x7f92f009c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:10.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.048+0000 7f92f6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92f00cfca0 0x7f92f01f7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:10.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.048+0000 7f92f6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f00d16e0 0x7f92f009b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:10.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.048+0000 7f92f6a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0169a70 con 0x7f92f00d16e0 2024-04-25T14:16:10.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.048+0000 7f92f6a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01698f0 con 0x7f92f00cfca0 2024-04-25T14:16:10.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.048+0000 7f92f6a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f92f0169bf0 con 0x7f92f00730b0 2024-04-25T14:16:10.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.048+0000 7f92f4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f00d16e0 0x7f92f009b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:10.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.048+0000 7f92f4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f00d16e0 0x7f92f009b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51066/0 (socket says 172.21.15.86:51066) 2024-04-25T14:16:10.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.048+0000 7f92f4a00640 1 -- 172.21.15.86:0/1766744172 learned_addr learned my addr 172.21.15.86:0/1766744172 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:10.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.048+0000 7f92efe00640 1 --2- 172.21.15.86:0/1766744172 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92f00730b0 0x7f92f009c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:10.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.048+0000 7f92ef400640 1 --2- 172.21.15.86:0/1766744172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92f00cfca0 0x7f92f01f7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:10.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.048+0000 7f92f4a00640 1 -- 172.21.15.86:0/1766744172 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92f00730b0 msgr2=0x7f92f009c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:10.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.048+0000 7f92f4a00640 1 --2- 172.21.15.86:0/1766744172 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92f00730b0 0x7f92f009c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:10.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.048+0000 7f92f4a00640 1 -- 172.21.15.86:0/1766744172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92f00cfca0 msgr2=0x7f92f01f7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:10.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.048+0000 7f92f4a00640 1 --2- 172.21.15.86:0/1766744172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92f00cfca0 0x7f92f01f7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:10.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.048+0000 7f92f4a00640 1 -- 172.21.15.86:0/1766744172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f0157cf0 con 0x7f92f00d16e0 2024-04-25T14:16:10.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.052+0000 7f92f4a00640 1 --2- 172.21.15.86:0/1766744172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f00d16e0 0x7f92f009b2b0 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f92e800b9e0 tx=0x7f92e800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:10.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.052+0000 7f92e7e00640 1 -- 172.21.15.86:0/1766744172 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e8019070 con 0x7f92f00d16e0 2024-04-25T14:16:10.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.052+0000 7f92e7e00640 1 -- 172.21.15.86:0/1766744172 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92e800cb40 con 0x7f92f00d16e0 2024-04-25T14:16:10.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.052+0000 7f92f6a00640 1 -- 172.21.15.86:0/1766744172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f015b5f0 con 0x7f92f00d16e0 2024-04-25T14:16:10.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.052+0000 7f92e7e00640 1 -- 172.21.15.86:0/1766744172 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e8014440 con 0x7f92f00d16e0 2024-04-25T14:16:10.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.052+0000 7f92f6a00640 1 -- 172.21.15.86:0/1766744172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f015bbb0 con 0x7f92f00d16e0 2024-04-25T14:16:10.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.052+0000 7f92f6a00640 1 -- 172.21.15.86:0/1766744172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92f015b360 con 0x7f92f00d16e0 2024-04-25T14:16:10.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.052+0000 7f92e7e00640 1 -- 172.21.15.86:0/1766744172 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f92e801c070 con 0x7f92f00d16e0 2024-04-25T14:16:10.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.052+0000 7f92e7e00640 1 --2- 172.21.15.86:0/1766744172 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f92d006f450 0x7f92d0071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:10.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.052+0000 7f92e7e00640 1 -- 172.21.15.86:0/1766744172 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f92e808f240 con 0x7f92f00d16e0 2024-04-25T14:16:10.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.052+0000 7f92efe00640 1 --2- 172.21.15.86:0/1766744172 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f92d006f450 0x7f92d0071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:10.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.056+0000 7f92efe00640 1 --2- 172.21.15.86:0/1766744172 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f92d006f450 0x7f92d0071960 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f92e00af000 tx=0x7f92e00a7000 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:10.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.056+0000 7f92e7e00640 1 -- 172.21.15.86:0/1766744172 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f92e805b7f0 con 0x7f92f00d16e0 2024-04-25T14:16:10.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.248+0000 7f92f6a00640 1 -- 172.21.15.86:0/1766744172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92f0069d00 con 0x7f92f00d16e0 2024-04-25T14:16:10.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.248+0000 7f92e7e00640 1 -- 172.21.15.86:0/1766744172 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f92e805faf0 con 0x7f92f00d16e0 2024-04-25T14:16:10.244 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:10.245 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:10.245 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:16:10.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.248+0000 7f92f6a00640 1 -- 172.21.15.86:0/1766744172 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f92d006f450 msgr2=0x7f92d0071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:10.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.248+0000 7f92f6a00640 1 --2- 172.21.15.86:0/1766744172 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f92d006f450 0x7f92d0071960 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f92e00af000 tx=0x7f92e00a7000 comp rx=0 tx=0).stop 2024-04-25T14:16:10.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.248+0000 7f92f6a00640 1 -- 172.21.15.86:0/1766744172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f00d16e0 msgr2=0x7f92f009b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:10.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.248+0000 7f92f6a00640 1 --2- 172.21.15.86:0/1766744172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f00d16e0 0x7f92f009b2b0 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f92e800b9e0 tx=0x7f92e800bea0 comp rx=0 tx=0).stop 2024-04-25T14:16:10.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.248+0000 7f92f6a00640 1 -- 172.21.15.86:0/1766744172 shutdown_connections 2024-04-25T14:16:10.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.248+0000 7f92f6a00640 1 --2- 172.21.15.86:0/1766744172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f00d16e0 0x7f92f009b2b0 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:10.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.248+0000 7f92f6a00640 1 --2- 172.21.15.86:0/1766744172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92f00cfca0 0x7f92f01f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:10.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.248+0000 7f92f6a00640 1 --2- 172.21.15.86:0/1766744172 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f92d006f450 0x7f92d0071960 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:10.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.248+0000 7f92f6a00640 1 --2- 172.21.15.86:0/1766744172 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92f00730b0 0x7f92f009c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:10.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.248+0000 7f92f6a00640 1 -- 172.21.15.86:0/1766744172 >> 172.21.15.86:0/1766744172 conn(0x7f92f00b4650 msgr2=0x7f92f015c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:10.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.248+0000 7f92f6a00640 1 -- 172.21.15.86:0/1766744172 shutdown_connections 2024-04-25T14:16:10.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:10.248+0000 7f92f6a00640 1 -- 172.21.15.86:0/1766744172 wait complete. 2024-04-25T14:16:10.670 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:10.671 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:11.672 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:11.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce60a00640 1 Processor -- start 2024-04-25T14:16:11.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce60a00640 1 -- start start 2024-04-25T14:16:11.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce60a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fce5c0cec60 0x7fce5c0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:11.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce60a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce5c0cf5b0 0x7fce5c0ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:11.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce60a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce5c163940 con 0x7fce5c0cf9c0 2024-04-25T14:16:11.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce60a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fce5c0d2f40 con 0x7fce5c0cec60 2024-04-25T14:16:11.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce60a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fce5c0b3720 con 0x7fce5c0cf5b0 2024-04-25T14:16:11.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce5a800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fce5c0cf9c0 0x7fce5c0cd2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47628/0 (socket says 172.21.15.86:47628) 2024-04-25T14:16:11.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce5a800640 1 -- 172.21.15.86:0/2316781777 learned_addr learned my addr 172.21.15.86:0/2316781777 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:16:11.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce59e00640 1 --2- 172.21.15.86:0/2316781777 >> v2:172.21.15.155:3300/0 conn(0x7fce5c0cec60 0x7fce5c0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:11.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce51400640 1 --2- 172.21.15.86:0/2316781777 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce5c0cf5b0 0x7fce5c0ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:11.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce59400640 1 -- 172.21.15.86:0/2316781777 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1385272901 0 0) 0x7fce5c163940 con 0x7fce5c0cf9c0 2024-04-25T14:16:11.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce59400640 1 -- 172.21.15.86:0/2316781777 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce40003530 con 0x7fce5c0cf9c0 2024-04-25T14:16:11.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce59400640 1 -- 172.21.15.86:0/2316781777 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1574723003 0 0) 0x7fce40003530 con 0x7fce5c0cf9c0 2024-04-25T14:16:11.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce59400640 1 -- 172.21.15.86:0/2316781777 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce5c0cf5b0 msgr2=0x7fce5c0ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:11.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce59400640 1 --2- 172.21.15.86:0/2316781777 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce5c0cf5b0 0x7fce5c0ccd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:11.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce59400640 1 -- 172.21.15.86:0/2316781777 >> v2:172.21.15.155:3300/0 conn(0x7fce5c0cec60 msgr2=0x7fce5c0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:11.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce59400640 1 --2- 172.21.15.86:0/2316781777 >> v2:172.21.15.155:3300/0 conn(0x7fce5c0cec60 0x7fce5c0cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:11.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce59400640 1 -- 172.21.15.86:0/2316781777 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5c0b3a00 con 0x7fce5c0cf9c0 2024-04-25T14:16:11.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce59400640 1 -- 172.21.15.86:0/2316781777 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fce54001680 con 0x7fce5c0cf9c0 2024-04-25T14:16:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce59400640 1 -- 172.21.15.86:0/2316781777 >> v1:172.21.15.86:6789/0 conn(0x7fce5c0cf9c0 legacy=0x7fce5c0cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce59400640 1 --2- 172.21.15.86:0/2316781777 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce5c0cf5b0 0x7fce40003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce59400640 1 --2- 172.21.15.86:0/2316781777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce40003fb0 0x7fce40004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce59400640 1 --2- 172.21.15.86:0/2316781777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce40004d50 0x7fce40005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce59400640 1 -- 172.21.15.86:0/2316781777 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c0d2f40 con 0x7fce40003fb0 2024-04-25T14:16:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce59400640 1 -- 172.21.15.86:0/2316781777 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c0b3720 con 0x7fce40004d50 2024-04-25T14:16:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce59400640 1 -- 172.21.15.86:0/2316781777 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fce4000bab0 con 0x7fce5c0cf5b0 2024-04-25T14:16:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce51400640 1 --2- 172.21.15.86:0/2316781777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce40003fb0 0x7fce40004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce59e00640 1 --2- 172.21.15.86:0/2316781777 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce5c0cf5b0 0x7fce40003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce5a800640 1 --2- 172.21.15.86:0/2316781777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce40004d50 0x7fce40005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce51400640 1 -- 172.21.15.86:0/2316781777 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce5c0cf5b0 msgr2=0x7fce40003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce51400640 1 --2- 172.21.15.86:0/2316781777 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce5c0cf5b0 0x7fce40003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce51400640 1 -- 172.21.15.86:0/2316781777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce40004d50 msgr2=0x7fce40005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce51400640 1 --2- 172.21.15.86:0/2316781777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce40004d50 0x7fce40005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:11.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce51400640 1 -- 172.21.15.86:0/2316781777 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce40003530 con 0x7fce40003fb0 2024-04-25T14:16:11.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.836+0000 7fce5a800640 1 --2- 172.21.15.86:0/2316781777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce40004d50 0x7fce40005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:16:11.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.840+0000 7fce51400640 1 --2- 172.21.15.86:0/2316781777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce40003fb0 0x7fce40004810 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7fce44009ae0 tx=0x7fce4401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42dd50c7145079b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:11.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.840+0000 7fce59400640 1 -- 172.21.15.86:0/2316781777 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce440b2070 con 0x7fce40003fb0 2024-04-25T14:16:11.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.840+0000 7fce59400640 1 -- 172.21.15.86:0/2316781777 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce440a5680 con 0x7fce40003fb0 2024-04-25T14:16:11.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.840+0000 7fce59400640 1 -- 172.21.15.86:0/2316781777 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce440096b0 con 0x7fce40003fb0 2024-04-25T14:16:11.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.840+0000 7fce60a00640 1 -- 172.21.15.86:0/2316781777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce40003fb0 msgr2=0x7fce40004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:11.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.840+0000 7fce60a00640 1 --2- 172.21.15.86:0/2316781777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce40003fb0 0x7fce40004810 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7fce44009ae0 tx=0x7fce4401ca90 comp rx=0 tx=0).stop 2024-04-25T14:16:11.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.840+0000 7fce5a800640 1 -- 172.21.15.86:0/2316781777 reap_dead start 2024-04-25T14:16:11.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.840+0000 7fce60a00640 1 -- 172.21.15.86:0/2316781777 shutdown_connections 2024-04-25T14:16:11.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.840+0000 7fce60a00640 1 -- 172.21.15.86:0/2316781777 >> 172.21.15.86:0/2316781777 conn(0x7fce5c09ba30 msgr2=0x7fce5c0b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:11.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.840+0000 7fce60a00640 1 -- 172.21.15.86:0/2316781777 shutdown_connections 2024-04-25T14:16:11.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.844+0000 7fce60a00640 1 -- 172.21.15.86:0/2316781777 wait complete. 2024-04-25T14:16:11.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.844+0000 7fce60a00640 1 Processor -- start 2024-04-25T14:16:11.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.844+0000 7fce60a00640 1 -- start start 2024-04-25T14:16:11.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.844+0000 7fce60a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce40003fb0 0x7fce5c0b0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:11.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.844+0000 7fce60a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce5c1f7620 0x7fce5c1f6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.844+0000 7fce60a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce5c1f9ee0 0x7fce5c1fa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.844+0000 7fce60a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c0ccf10 con 0x7fce5c1f9ee0 2024-04-25T14:16:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.844+0000 7fce60a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c0ccd90 con 0x7fce40003fb0 2024-04-25T14:16:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.844+0000 7fce60a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fce5c0cd090 con 0x7fce5c1f7620 2024-04-25T14:16:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.848+0000 7fce5a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce5c1f9ee0 0x7fce5c1fa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.848+0000 7fce5a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce5c1f9ee0 0x7fce5c1fa2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51090/0 (socket says 172.21.15.86:51090) 2024-04-25T14:16:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.848+0000 7fce5a800640 1 -- 172.21.15.86:0/3766703671 learned_addr learned my addr 172.21.15.86:0/3766703671 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.848+0000 7fce51400640 1 --2- 172.21.15.86:0/3766703671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce5c1f7620 0x7fce5c1f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.848+0000 7fce59e00640 1 --2- 172.21.15.86:0/3766703671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce40003fb0 0x7fce5c0b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.848+0000 7fce5a800640 1 -- 172.21.15.86:0/3766703671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce5c1f7620 msgr2=0x7fce5c1f6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.848+0000 7fce5a800640 1 --2- 172.21.15.86:0/3766703671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce5c1f7620 0x7fce5c1f6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.848+0000 7fce5a800640 1 -- 172.21.15.86:0/3766703671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce40003fb0 msgr2=0x7fce5c0b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.848+0000 7fce5a800640 1 --2- 172.21.15.86:0/3766703671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce40003fb0 0x7fce5c0b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.848+0000 7fce5a800640 1 -- 172.21.15.86:0/3766703671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5c0b7ed0 con 0x7fce5c1f9ee0 2024-04-25T14:16:11.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.848+0000 7fce51400640 1 --2- 172.21.15.86:0/3766703671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce5c1f7620 0x7fce5c1f6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:16:11.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.848+0000 7fce59e00640 1 --2- 172.21.15.86:0/3766703671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce40003fb0 0x7fce5c0b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:16:11.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.848+0000 7fce5a800640 1 --2- 172.21.15.86:0/3766703671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce5c1f9ee0 0x7fce5c1fa2f0 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7fce5c0b8420 tx=0x7fce54010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:11.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.848+0000 7fce52a00640 1 -- 172.21.15.86:0/3766703671 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5401a070 con 0x7fce5c1f9ee0 2024-04-25T14:16:11.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.848+0000 7fce52a00640 1 -- 172.21.15.86:0/3766703671 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce54011040 con 0x7fce5c1f9ee0 2024-04-25T14:16:11.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.848+0000 7fce60a00640 1 -- 172.21.15.86:0/3766703671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce5c1fa830 con 0x7fce5c1f9ee0 2024-04-25T14:16:11.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.848+0000 7fce52a00640 1 -- 172.21.15.86:0/3766703671 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5400ecc0 con 0x7fce5c1f9ee0 2024-04-25T14:16:11.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.848+0000 7fce60a00640 1 -- 172.21.15.86:0/3766703671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce5c0b0de0 con 0x7fce5c1f9ee0 2024-04-25T14:16:11.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.848+0000 7fce60a00640 1 -- 172.21.15.86:0/3766703671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce5c0b0850 con 0x7fce5c1f9ee0 2024-04-25T14:16:11.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.848+0000 7fce52a00640 1 -- 172.21.15.86:0/3766703671 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7fce5401d070 con 0x7fce5c1f9ee0 2024-04-25T14:16:11.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.852+0000 7fce52a00640 1 --2- 172.21.15.86:0/3766703671 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fce3806f450 0x7fce38071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:11.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.852+0000 7fce52a00640 1 -- 172.21.15.86:0/3766703671 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fce54091d90 con 0x7fce5c1f9ee0 2024-04-25T14:16:11.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.852+0000 7fce59e00640 1 --2- 172.21.15.86:0/3766703671 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fce3806f450 0x7fce38071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:11.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.852+0000 7fce59e00640 1 --2- 172.21.15.86:0/3766703671 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fce3806f450 0x7fce38071960 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fce4c007520 tx=0x7fce4c00b040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:11.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:11.856+0000 7fce52a00640 1 -- 172.21.15.86:0/3766703671 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fce5c1fa830 con 0x7fce5c1f9ee0 2024-04-25T14:16:12.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:12.048+0000 7fce60a00640 1 -- 172.21.15.86:0/3766703671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce5c06f250 con 0x7fce5c1f9ee0 2024-04-25T14:16:12.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:12.048+0000 7fce52a00640 1 -- 172.21.15.86:0/3766703671 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7fce5c06f250 con 0x7fce5c1f9ee0 2024-04-25T14:16:12.044 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:12.044 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:12.044 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:16:12.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:12.048+0000 7fce60a00640 1 -- 172.21.15.86:0/3766703671 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fce3806f450 msgr2=0x7fce38071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:12.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:12.048+0000 7fce60a00640 1 --2- 172.21.15.86:0/3766703671 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fce3806f450 0x7fce38071960 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fce4c007520 tx=0x7fce4c00b040 comp rx=0 tx=0).stop 2024-04-25T14:16:12.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:12.048+0000 7fce60a00640 1 -- 172.21.15.86:0/3766703671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce5c1f9ee0 msgr2=0x7fce5c1fa2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:12.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:12.048+0000 7fce60a00640 1 --2- 172.21.15.86:0/3766703671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce5c1f9ee0 0x7fce5c1fa2f0 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7fce5c0b8420 tx=0x7fce54010d70 comp rx=0 tx=0).stop 2024-04-25T14:16:12.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:12.048+0000 7fce60a00640 1 -- 172.21.15.86:0/3766703671 shutdown_connections 2024-04-25T14:16:12.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:12.048+0000 7fce60a00640 1 --2- 172.21.15.86:0/3766703671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce5c1f9ee0 0x7fce5c1fa2f0 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:12.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:12.048+0000 7fce60a00640 1 --2- 172.21.15.86:0/3766703671 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fce3806f450 0x7fce38071960 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:12.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:12.048+0000 7fce60a00640 1 --2- 172.21.15.86:0/3766703671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce5c1f7620 0x7fce5c1f6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:12.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:12.048+0000 7fce60a00640 1 --2- 172.21.15.86:0/3766703671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce40003fb0 0x7fce5c0b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:12.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:12.048+0000 7fce60a00640 1 -- 172.21.15.86:0/3766703671 >> 172.21.15.86:0/3766703671 conn(0x7fce5c09ba30 msgr2=0x7fce5c15ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:12.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:12.048+0000 7fce60a00640 1 -- 172.21.15.86:0/3766703671 shutdown_connections 2024-04-25T14:16:12.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:12.048+0000 7fce60a00640 1 -- 172.21.15.86:0/3766703671 wait complete. 2024-04-25T14:16:12.468 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:12.468 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:13.470 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:13.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.632+0000 7f770fe00640 1 Processor -- start 2024-04-25T14:16:13.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.632+0000 7f770fe00640 1 -- start start 2024-04-25T14:16:13.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.632+0000 7f770fe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f77100ced30 0x7f77100b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:13.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.632+0000 7f770fe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7710161b30 con 0x7f771015dcd0 2024-04-25T14:16:13.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.632+0000 7f770fe00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7710161e10 con 0x7f77100d1b40 2024-04-25T14:16:13.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.632+0000 7f770fe00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f77101618e0 con 0x7f77100ced30 2024-04-25T14:16:13.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.632+0000 7f7706000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f771015dcd0 0x7f77100d1430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47638/0 (socket says 172.21.15.86:47638) 2024-04-25T14:16:13.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.632+0000 7f7706000640 1 -- 172.21.15.86:0/2334324068 learned_addr learned my addr 172.21.15.86:0/2334324068 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:16:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.632+0000 7f770ea00640 1 --2- 172.21.15.86:0/2334324068 >> v2:172.21.15.155:3300/0 conn(0x7f77100ced30 0x7f77100b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.632+0000 7f770e000640 1 -- 172.21.15.86:0/2334324068 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1026343148 0 0) 0x7f7710161b30 con 0x7f771015dcd0 2024-04-25T14:16:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.632+0000 7f770e000640 1 -- 172.21.15.86:0/2334324068 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76f4003530 con 0x7f771015dcd0 2024-04-25T14:16:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.632+0000 7f770e000640 1 -- 172.21.15.86:0/2334324068 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1905975386 0 0) 0x7f76f4003530 con 0x7f771015dcd0 2024-04-25T14:16:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.632+0000 7f770e000640 1 -- 172.21.15.86:0/2334324068 >> v2:172.21.15.155:3300/0 conn(0x7f77100ced30 msgr2=0x7f77100b1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.632+0000 7f770e000640 1 --2- 172.21.15.86:0/2334324068 >> v2:172.21.15.155:3300/0 conn(0x7f77100ced30 0x7f77100b1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.632+0000 7f770e000640 1 -- 172.21.15.86:0/2334324068 >> v1:172.21.15.155:6789/0 conn(0x7f77100d1b40 legacy=0x7f771015e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.632+0000 7f770e000640 1 -- 172.21.15.86:0/2334324068 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77101620f0 con 0x7f771015dcd0 2024-04-25T14:16:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.632+0000 7f770e000640 1 -- 172.21.15.86:0/2334324068 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f76f8001680 con 0x7f771015dcd0 2024-04-25T14:16:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.632+0000 7f770e000640 1 -- 172.21.15.86:0/2334324068 >> v1:172.21.15.86:6789/0 conn(0x7f771015dcd0 legacy=0x7f77100d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:13.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.632+0000 7f770e000640 1 --2- 172.21.15.86:0/2334324068 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76f40039c0 0x7f76f4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:13.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.632+0000 7f770e000640 1 --2- 172.21.15.86:0/2334324068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76f4004810 0x7f76f4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:13.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.632+0000 7f770e000640 1 --2- 172.21.15.86:0/2334324068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76f40051d0 0x7f76f400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:13.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.632+0000 7f770e000640 1 -- 172.21.15.86:0/2334324068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f77101618e0 con 0x7f76f4004810 2024-04-25T14:16:13.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.632+0000 7f770e000640 1 -- 172.21.15.86:0/2334324068 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f76f400c170 con 0x7f76f40051d0 2024-04-25T14:16:13.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.632+0000 7f770e000640 1 -- 172.21.15.86:0/2334324068 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f76f400c370 con 0x7f76f40039c0 2024-04-25T14:16:13.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.636+0000 7f7706000640 1 --2- 172.21.15.86:0/2334324068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76f4004810 0x7f76f4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:13.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.636+0000 7f770ea00640 1 --2- 172.21.15.86:0/2334324068 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76f40039c0 0x7f76f4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:13.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.636+0000 7f770f400640 1 --2- 172.21.15.86:0/2334324068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76f40051d0 0x7f76f400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:13.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.636+0000 7f7706000640 1 -- 172.21.15.86:0/2334324068 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76f40039c0 msgr2=0x7f76f4003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:13.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.636+0000 7f7706000640 1 --2- 172.21.15.86:0/2334324068 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76f40039c0 0x7f76f4003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:13.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.636+0000 7f7706000640 1 -- 172.21.15.86:0/2334324068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76f40051d0 msgr2=0x7f76f400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:13.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.636+0000 7f7706000640 1 --2- 172.21.15.86:0/2334324068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76f40051d0 0x7f76f400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:13.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.636+0000 7f7706000640 1 -- 172.21.15.86:0/2334324068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f4003530 con 0x7f76f4004810 2024-04-25T14:16:13.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.636+0000 7f770f400640 1 --2- 172.21.15.86:0/2334324068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76f40051d0 0x7f76f400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:16:13.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.636+0000 7f770f400640 1 -- 172.21.15.86:0/2334324068 reap_dead start 2024-04-25T14:16:13.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.636+0000 7f7706000640 1 --2- 172.21.15.86:0/2334324068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76f4004810 0x7f76f4004c20 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f76f800c210 tx=0x7f76f801f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3faffc8f77490e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:13.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.636+0000 7f770e000640 1 -- 172.21.15.86:0/2334324068 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f8007d20 con 0x7f76f4004810 2024-04-25T14:16:13.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.636+0000 7f770e000640 1 -- 172.21.15.86:0/2334324068 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76f8005ae0 con 0x7f76f4004810 2024-04-25T14:16:13.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.636+0000 7f770e000640 1 -- 172.21.15.86:0/2334324068 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f8007070 con 0x7f76f4004810 2024-04-25T14:16:13.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.640+0000 7f770fe00640 1 -- 172.21.15.86:0/2334324068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76f4004810 msgr2=0x7f76f4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:13.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.640+0000 7f770fe00640 1 --2- 172.21.15.86:0/2334324068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76f4004810 0x7f76f4004c20 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f76f800c210 tx=0x7f76f801f3b0 comp rx=0 tx=0).stop 2024-04-25T14:16:13.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.640+0000 7f770fe00640 1 -- 172.21.15.86:0/2334324068 shutdown_connections 2024-04-25T14:16:13.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.640+0000 7f770fe00640 1 --2- 172.21.15.86:0/2334324068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76f4004810 0x7f76f4004c20 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:13.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.640+0000 7f770fe00640 1 -- 172.21.15.86:0/2334324068 >> 172.21.15.86:0/2334324068 conn(0x7f771009ba30 msgr2=0x7f77100b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:13.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.640+0000 7f770fe00640 1 -- 172.21.15.86:0/2334324068 shutdown_connections 2024-04-25T14:16:13.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.640+0000 7f770fe00640 1 -- 172.21.15.86:0/2334324068 wait complete. 2024-04-25T14:16:13.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.640+0000 7f770fe00640 1 Processor -- start 2024-04-25T14:16:13.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.640+0000 7f770fe00640 1 -- start start 2024-04-25T14:16:13.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.644+0000 7f770fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76f4004810 0x7f771014fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:13.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.644+0000 7f770fe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f771015f1c0 0x7f771015f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:13.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.644+0000 7f770fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77101602a0 0x7f77101606b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:13.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.644+0000 7f770fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f771015e0e0 con 0x7f76f4004810 2024-04-25T14:16:13.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.644+0000 7f770fe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f77100b7be0 con 0x7f77101602a0 2024-04-25T14:16:13.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.644+0000 7f770fe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f77100b1e20 con 0x7f771015f1c0 2024-04-25T14:16:13.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.644+0000 7f770ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76f4004810 0x7f771014fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:13.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.644+0000 7f770ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76f4004810 0x7f771014fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51114/0 (socket says 172.21.15.86:51114) 2024-04-25T14:16:13.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.644+0000 7f770ea00640 1 -- 172.21.15.86:0/948107135 learned_addr learned my addr 172.21.15.86:0/948107135 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:13.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.644+0000 7f7706000640 1 --2- 172.21.15.86:0/948107135 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f771015f1c0 0x7f771015f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:13.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.644+0000 7f770f400640 1 --2- 172.21.15.86:0/948107135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77101602a0 0x7f77101606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:13.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.644+0000 7f770ea00640 1 -- 172.21.15.86:0/948107135 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f771015f1c0 msgr2=0x7f771015f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:13.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.644+0000 7f770ea00640 1 --2- 172.21.15.86:0/948107135 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f771015f1c0 0x7f771015f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:13.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.644+0000 7f770ea00640 1 -- 172.21.15.86:0/948107135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77101602a0 msgr2=0x7f77101606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:13.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.644+0000 7f770ea00640 1 --2- 172.21.15.86:0/948107135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77101602a0 0x7f77101606b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:13.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.644+0000 7f770ea00640 1 -- 172.21.15.86:0/948107135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7710150910 con 0x7f76f4004810 2024-04-25T14:16:13.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.644+0000 7f770ea00640 1 --2- 172.21.15.86:0/948107135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76f4004810 0x7f771014fe90 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f7700011870 tx=0x7f7700011c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:13.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.644+0000 7f7707e00640 1 -- 172.21.15.86:0/948107135 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7700019070 con 0x7f76f4004810 2024-04-25T14:16:13.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.644+0000 7f7707e00640 1 -- 172.21.15.86:0/948107135 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7700004240 con 0x7f76f4004810 2024-04-25T14:16:13.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.644+0000 7f7707e00640 1 -- 172.21.15.86:0/948107135 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770000f7c0 con 0x7f76f4004810 2024-04-25T14:16:13.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.644+0000 7f770fe00640 1 -- 172.21.15.86:0/948107135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77101503d0 con 0x7f76f4004810 2024-04-25T14:16:13.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.644+0000 7f770fe00640 1 -- 172.21.15.86:0/948107135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77100cb9d0 con 0x7f76f4004810 2024-04-25T14:16:13.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.644+0000 7f770fe00640 1 -- 172.21.15.86:0/948107135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7710076b80 con 0x7f76f4004810 2024-04-25T14:16:13.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.644+0000 7f7707e00640 1 -- 172.21.15.86:0/948107135 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f770001c070 con 0x7f76f4004810 2024-04-25T14:16:13.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.648+0000 7f7707e00640 1 --2- 172.21.15.86:0/948107135 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f76ec06f480 0x7f76ec071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:13.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.648+0000 7f7707e00640 1 -- 172.21.15.86:0/948107135 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f770008f880 con 0x7f76f4004810 2024-04-25T14:16:13.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.648+0000 7f7706000640 1 --2- 172.21.15.86:0/948107135 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f76ec06f480 0x7f76ec071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:13.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.648+0000 7f7706000640 1 --2- 172.21.15.86:0/948107135 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f76ec06f480 0x7f76ec071990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f76f80af000 tx=0x7f76f80aa000 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:13.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.652+0000 7f7707e00640 1 -- 172.21.15.86:0/948107135 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f770005be40 con 0x7f76f4004810 2024-04-25T14:16:13.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.844+0000 7f770fe00640 1 -- 172.21.15.86:0/948107135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77100b79c0 con 0x7f76f4004810 2024-04-25T14:16:13.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.844+0000 7f7707e00640 1 -- 172.21.15.86:0/948107135 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f7700060140 con 0x7f76f4004810 2024-04-25T14:16:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:13.842 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:13.842 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:16:13.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.848+0000 7f770fe00640 1 -- 172.21.15.86:0/948107135 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f76ec06f480 msgr2=0x7f76ec071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:13.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.848+0000 7f770fe00640 1 --2- 172.21.15.86:0/948107135 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f76ec06f480 0x7f76ec071990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f76f80af000 tx=0x7f76f80aa000 comp rx=0 tx=0).stop 2024-04-25T14:16:13.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.848+0000 7f770fe00640 1 -- 172.21.15.86:0/948107135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76f4004810 msgr2=0x7f771014fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:13.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.848+0000 7f770fe00640 1 --2- 172.21.15.86:0/948107135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76f4004810 0x7f771014fe90 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f7700011870 tx=0x7f7700011c70 comp rx=0 tx=0).stop 2024-04-25T14:16:13.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.848+0000 7f770fe00640 1 -- 172.21.15.86:0/948107135 shutdown_connections 2024-04-25T14:16:13.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.848+0000 7f770fe00640 1 --2- 172.21.15.86:0/948107135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77101602a0 0x7f77101606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:13.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.848+0000 7f770fe00640 1 --2- 172.21.15.86:0/948107135 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f76ec06f480 0x7f76ec071990 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:13.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.848+0000 7f770fe00640 1 --2- 172.21.15.86:0/948107135 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f771015f1c0 0x7f771015f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.848+0000 7f770fe00640 1 --2- 172.21.15.86:0/948107135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76f4004810 0x7f771014fe90 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.848+0000 7f770fe00640 1 -- 172.21.15.86:0/948107135 >> 172.21.15.86:0/948107135 conn(0x7f771009ba30 msgr2=0x7f77100d13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.848+0000 7f770fe00640 1 -- 172.21.15.86:0/948107135 shutdown_connections 2024-04-25T14:16:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:13.848+0000 7f770fe00640 1 -- 172.21.15.86:0/948107135 wait complete. 2024-04-25T14:16:14.266 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:14.266 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:15.267 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.432+0000 7fd216200640 1 Processor -- start 2024-04-25T14:16:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.432+0000 7fd216200640 1 -- start start 2024-04-25T14:16:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.432+0000 7fd216200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd2100b36f0 0x7fd2100b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.432+0000 7fd216200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2100b3b00 con 0x7fd21015bec0 2024-04-25T14:16:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.432+0000 7fd216200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2100b1a90 con 0x7fd21006d780 2024-04-25T14:16:15.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.432+0000 7fd216200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd2100d1c40 con 0x7fd2100b36f0 2024-04-25T14:16:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.432+0000 7fd215800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd21015bec0 0x7fd21015c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47646/0 (socket says 172.21.15.86:47646) 2024-04-25T14:16:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.432+0000 7fd215800640 1 -- 172.21.15.86:0/2938507571 learned_addr learned my addr 172.21.15.86:0/2938507571 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:16:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.432+0000 7fd214e00640 1 --2- 172.21.15.86:0/2938507571 >> v2:172.21.15.155:3300/0 conn(0x7fd2100b36f0 0x7fd2100b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.432+0000 7fd20fe00640 1 -- 172.21.15.86:0/2938507571 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3404611923 0 0) 0x7fd2100b3b00 con 0x7fd21015bec0 2024-04-25T14:16:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.432+0000 7fd20fe00640 1 -- 172.21.15.86:0/2938507571 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1f4003530 con 0x7fd21015bec0 2024-04-25T14:16:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.432+0000 7fd20fe00640 1 -- 172.21.15.86:0/2938507571 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4045348057 0 0) 0x7fd2100b1a90 con 0x7fd21006d780 2024-04-25T14:16:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.432+0000 7fd20fe00640 1 -- 172.21.15.86:0/2938507571 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2100b3b00 con 0x7fd21006d780 2024-04-25T14:16:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.432+0000 7fd20fe00640 1 -- 172.21.15.86:0/2938507571 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2235022960 0 0) 0x7fd1f4003530 con 0x7fd21015bec0 2024-04-25T14:16:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.432+0000 7fd214e00640 1 -- 172.21.15.86:0/2938507571 >> v1:172.21.15.155:6789/0 conn(0x7fd21006d780 legacy=0x7fd21006db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.432+0000 7fd214e00640 1 -- 172.21.15.86:0/2938507571 >> v1:172.21.15.86:6789/0 conn(0x7fd21015bec0 legacy=0x7fd21015c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.432+0000 7fd214e00640 1 -- 172.21.15.86:0/2938507571 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2100b2250 con 0x7fd2100b36f0 2024-04-25T14:16:15.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.436+0000 7fd214e00640 1 --2- 172.21.15.86:0/2938507571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2100b36f0 0x7fd2100b77b0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd2000097c0 tx=0x7fd20001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bc7d3efa93bbd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.436+0000 7fd20fe00640 1 -- 172.21.15.86:0/2938507571 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2000b0070 con 0x7fd2100b36f0 2024-04-25T14:16:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.436+0000 7fd20fe00640 1 -- 172.21.15.86:0/2938507571 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2000ab400 con 0x7fd2100b36f0 2024-04-25T14:16:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.436+0000 7fd20fe00640 1 -- 172.21.15.86:0/2938507571 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd208001a00 con 0x7fd2100b36f0 2024-04-25T14:16:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.436+0000 7fd216200640 1 -- 172.21.15.86:0/2938507571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2100b36f0 msgr2=0x7fd2100b77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.436+0000 7fd216200640 1 --2- 172.21.15.86:0/2938507571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2100b36f0 0x7fd2100b77b0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd2000097c0 tx=0x7fd20001c3c0 comp rx=0 tx=0).stop 2024-04-25T14:16:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.436+0000 7fd216200640 1 -- 172.21.15.86:0/2938507571 shutdown_connections 2024-04-25T14:16:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.436+0000 7fd216200640 1 --2- 172.21.15.86:0/2938507571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2100b36f0 0x7fd2100b77b0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.436+0000 7fd216200640 1 -- 172.21.15.86:0/2938507571 >> 172.21.15.86:0/2938507571 conn(0x7fd21009ba30 msgr2=0x7fd2100b70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.436+0000 7fd216200640 1 -- 172.21.15.86:0/2938507571 shutdown_connections 2024-04-25T14:16:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.436+0000 7fd216200640 1 -- 172.21.15.86:0/2938507571 wait complete. 2024-04-25T14:16:15.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.440+0000 7fd216200640 1 Processor -- start 2024-04-25T14:16:15.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.440+0000 7fd216200640 1 -- start start 2024-04-25T14:16:15.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.440+0000 7fd216200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd21006d780 0x7fd21014edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:15.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.440+0000 7fd216200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2100b36f0 0x7fd210155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:15.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.440+0000 7fd216200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd21015bec0 0x7fd2100b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:15.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.440+0000 7fd216200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd2100d0e50 con 0x7fd21015bec0 2024-04-25T14:16:15.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.440+0000 7fd216200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd2100d3140 con 0x7fd21006d780 2024-04-25T14:16:15.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.440+0000 7fd216200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd2100d0fd0 con 0x7fd2100b36f0 2024-04-25T14:16:15.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.440+0000 7fd215800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd21015bec0 0x7fd2100b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:15.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.440+0000 7fd215800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd21015bec0 0x7fd2100b01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51124/0 (socket says 172.21.15.86:51124) 2024-04-25T14:16:15.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.440+0000 7fd215800640 1 -- 172.21.15.86:0/1388006584 learned_addr learned my addr 172.21.15.86:0/1388006584 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:15.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.440+0000 7fd214e00640 1 --2- 172.21.15.86:0/1388006584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd21006d780 0x7fd21014edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:15.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.440+0000 7fd207e00640 1 --2- 172.21.15.86:0/1388006584 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2100b36f0 0x7fd210155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:15.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.444+0000 7fd215800640 1 -- 172.21.15.86:0/1388006584 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2100b36f0 msgr2=0x7fd210155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:15.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.444+0000 7fd215800640 1 --2- 172.21.15.86:0/1388006584 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2100b36f0 0x7fd210155570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:15.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.444+0000 7fd215800640 1 -- 172.21.15.86:0/1388006584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd21006d780 msgr2=0x7fd21014edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:15.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.444+0000 7fd215800640 1 --2- 172.21.15.86:0/1388006584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd21006d780 0x7fd21014edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:15.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.444+0000 7fd215800640 1 -- 172.21.15.86:0/1388006584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21014f330 con 0x7fd21015bec0 2024-04-25T14:16:15.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.444+0000 7fd207e00640 1 --2- 172.21.15.86:0/1388006584 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2100b36f0 0x7fd210155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:16:15.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.444+0000 7fd215800640 1 --2- 172.21.15.86:0/1388006584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd21015bec0 0x7fd2100b01b0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7fd20800cc10 tx=0x7fd20800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:15.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.444+0000 7fd20d600640 1 -- 172.21.15.86:0/1388006584 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20800ce20 con 0x7fd21015bec0 2024-04-25T14:16:15.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.444+0000 7fd20d600640 1 -- 172.21.15.86:0/1388006584 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd208012400 con 0x7fd21015bec0 2024-04-25T14:16:15.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.444+0000 7fd216200640 1 -- 172.21.15.86:0/1388006584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd210152da0 con 0x7fd21015bec0 2024-04-25T14:16:15.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.444+0000 7fd20d600640 1 -- 172.21.15.86:0/1388006584 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2080127d0 con 0x7fd21015bec0 2024-04-25T14:16:15.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.444+0000 7fd216200640 1 -- 172.21.15.86:0/1388006584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd210153360 con 0x7fd21015bec0 2024-04-25T14:16:15.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.444+0000 7fd216200640 1 -- 172.21.15.86:0/1388006584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd210152b10 con 0x7fd21015bec0 2024-04-25T14:16:15.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.444+0000 7fd20d600640 1 -- 172.21.15.86:0/1388006584 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7fd208012970 con 0x7fd21015bec0 2024-04-25T14:16:15.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.444+0000 7fd20d600640 1 --2- 172.21.15.86:0/1388006584 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd1ec06f450 0x7fd1ec071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:15.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.444+0000 7fd20d600640 1 -- 172.21.15.86:0/1388006584 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fd20808fd00 con 0x7fd21015bec0 2024-04-25T14:16:15.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.448+0000 7fd214e00640 1 --2- 172.21.15.86:0/1388006584 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd1ec06f450 0x7fd1ec071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:15.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.448+0000 7fd214e00640 1 --2- 172.21.15.86:0/1388006584 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd1ec06f450 0x7fd1ec071960 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd2000ad000 tx=0x7fd2000a6000 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:15.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.452+0000 7fd20d600640 1 -- 172.21.15.86:0/1388006584 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd20805c900 con 0x7fd21015bec0 2024-04-25T14:16:15.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.640+0000 7fd216200640 1 -- 172.21.15.86:0/1388006584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd210076b80 con 0x7fd21015bec0 2024-04-25T14:16:15.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.644+0000 7fd20d600640 1 -- 172.21.15.86:0/1388006584 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7fd20805c200 con 0x7fd21015bec0 2024-04-25T14:16:15.640 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:15.640 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:15.640 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:16:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.644+0000 7fd216200640 1 -- 172.21.15.86:0/1388006584 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd1ec06f450 msgr2=0x7fd1ec071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.644+0000 7fd216200640 1 --2- 172.21.15.86:0/1388006584 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd1ec06f450 0x7fd1ec071960 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd2000ad000 tx=0x7fd2000a6000 comp rx=0 tx=0).stop 2024-04-25T14:16:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.644+0000 7fd216200640 1 -- 172.21.15.86:0/1388006584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd21015bec0 msgr2=0x7fd2100b01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.644+0000 7fd216200640 1 --2- 172.21.15.86:0/1388006584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd21015bec0 0x7fd2100b01b0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7fd20800cc10 tx=0x7fd20800a430 comp rx=0 tx=0).stop 2024-04-25T14:16:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.644+0000 7fd216200640 1 -- 172.21.15.86:0/1388006584 shutdown_connections 2024-04-25T14:16:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.644+0000 7fd216200640 1 --2- 172.21.15.86:0/1388006584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd21015bec0 0x7fd2100b01b0 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.644+0000 7fd216200640 1 --2- 172.21.15.86:0/1388006584 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd1ec06f450 0x7fd1ec071960 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:15.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.644+0000 7fd216200640 1 --2- 172.21.15.86:0/1388006584 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2100b36f0 0x7fd210155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:15.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.644+0000 7fd216200640 1 --2- 172.21.15.86:0/1388006584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd21006d780 0x7fd21014edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:15.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.644+0000 7fd216200640 1 -- 172.21.15.86:0/1388006584 >> 172.21.15.86:0/1388006584 conn(0x7fd21009ba30 msgr2=0x7fd21006cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:15.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.644+0000 7fd216200640 1 -- 172.21.15.86:0/1388006584 shutdown_connections 2024-04-25T14:16:15.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:15.644+0000 7fd216200640 1 -- 172.21.15.86:0/1388006584 wait complete. 2024-04-25T14:16:16.068 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:16.068 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:17.069 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:17.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.232+0000 7f6f4e400640 1 Processor -- start 2024-04-25T14:16:17.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.232+0000 7f6f4e400640 1 -- start start 2024-04-25T14:16:17.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.232+0000 7f6f4e400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6f4806fd50 0x7f6f48070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:17.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.232+0000 7f6f4e400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f480d16e0 0x7f6f4815feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:17.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.232+0000 7f6f4e400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f480b0a60 con 0x7f6f4815c2a0 2024-04-25T14:16:17.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.232+0000 7f6f4e400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6f480ae840 con 0x7f6f4806fd50 2024-04-25T14:16:17.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.232+0000 7f6f4e400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6f4806ef90 con 0x7f6f480d16e0 2024-04-25T14:16:17.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.232+0000 7f6f46a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6f4806fd50 0x7f6f48070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:17.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.232+0000 7f6f46a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6f4806fd50 0x7f6f48070160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51138/0 (socket says 172.21.15.86:51138) 2024-04-25T14:16:17.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.232+0000 7f6f46a00640 1 -- 172.21.15.86:0/2792122494 learned_addr learned my addr 172.21.15.86:0/2792122494 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:17.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.232+0000 7f6f47e00640 1 --2- 172.21.15.86:0/2792122494 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f480d16e0 0x7f6f4815feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:17.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.232+0000 7f6f46a00640 1 -- 172.21.15.86:0/2792122494 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f480d16e0 msgr2=0x7f6f4815feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:17.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.232+0000 7f6f46a00640 1 --2- 172.21.15.86:0/2792122494 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f480d16e0 0x7f6f4815feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:17.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.232+0000 7f6f46a00640 1 -- 172.21.15.86:0/2792122494 >> v1:172.21.15.86:6789/0 conn(0x7f6f4815c2a0 legacy=0x7f6f480ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:17.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.232+0000 7f6f46a00640 1 -- 172.21.15.86:0/2792122494 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4806f270 con 0x7f6f4806fd50 2024-04-25T14:16:17.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.236+0000 7f6f46a00640 1 --2- 172.21.15.86:0/2792122494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f4806fd50 0x7f6f48070160 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f6f3400bb10 tx=0x7f6f3401eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8525ab254379bbd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:17.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.236+0000 7f6f46000640 1 -- 172.21.15.86:0/2792122494 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f340b4070 con 0x7f6f4806fd50 2024-04-25T14:16:17.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.236+0000 7f6f46000640 1 -- 172.21.15.86:0/2792122494 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f340a7c40 con 0x7f6f4806fd50 2024-04-25T14:16:17.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.236+0000 7f6f46000640 1 -- 172.21.15.86:0/2792122494 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f340af440 con 0x7f6f4806fd50 2024-04-25T14:16:17.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.236+0000 7f6f4e400640 1 -- 172.21.15.86:0/2792122494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f4806fd50 msgr2=0x7f6f48070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:17.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.236+0000 7f6f4e400640 1 --2- 172.21.15.86:0/2792122494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f4806fd50 0x7f6f48070160 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f6f3400bb10 tx=0x7f6f3401eb70 comp rx=0 tx=0).stop 2024-04-25T14:16:17.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.236+0000 7f6f4e400640 1 -- 172.21.15.86:0/2792122494 shutdown_connections 2024-04-25T14:16:17.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.236+0000 7f6f4e400640 1 --2- 172.21.15.86:0/2792122494 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f480d16e0 0x7f6f4815feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:17.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.236+0000 7f6f4e400640 1 --2- 172.21.15.86:0/2792122494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f4806fd50 0x7f6f48070160 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:17.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.236+0000 7f6f4e400640 1 -- 172.21.15.86:0/2792122494 >> 172.21.15.86:0/2792122494 conn(0x7f6f480b4650 msgr2=0x7f6f480b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:17.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.236+0000 7f6f4e400640 1 -- 172.21.15.86:0/2792122494 shutdown_connections 2024-04-25T14:16:17.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.236+0000 7f6f4e400640 1 -- 172.21.15.86:0/2792122494 wait complete. 2024-04-25T14:16:17.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.240+0000 7f6f4e400640 1 Processor -- start 2024-04-25T14:16:17.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.240+0000 7f6f4e400640 1 -- start start 2024-04-25T14:16:17.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.240+0000 7f6f4e400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f4806fd50 0x7f6f480c1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:17.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.240+0000 7f6f4e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f480d16e0 0x7f6f48098680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:17.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.240+0000 7f6f4e400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f4815c2a0 0x7f6f48098bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:17.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.240+0000 7f6f4e400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4815c6b0 con 0x7f6f480d16e0 2024-04-25T14:16:17.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.240+0000 7f6f4e400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6f480d0fc0 con 0x7f6f4815c2a0 2024-04-25T14:16:17.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.240+0000 7f6f4e400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6f4815c830 con 0x7f6f4806fd50 2024-04-25T14:16:17.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.240+0000 7f6f46a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f480d16e0 0x7f6f48098680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:17.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.240+0000 7f6f46a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f480d16e0 0x7f6f48098680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51150/0 (socket says 172.21.15.86:51150) 2024-04-25T14:16:17.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.240+0000 7f6f46a00640 1 -- 172.21.15.86:0/853909248 learned_addr learned my addr 172.21.15.86:0/853909248 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:17.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.240+0000 7f6f46a00640 1 -- 172.21.15.86:0/853909248 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f4806fd50 msgr2=0x7f6f480c1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:17.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.240+0000 7f6f46a00640 1 --2- 172.21.15.86:0/853909248 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f4806fd50 0x7f6f480c1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:17.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.240+0000 7f6f46a00640 1 -- 172.21.15.86:0/853909248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f4815c2a0 msgr2=0x7f6f48098bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:17.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.240+0000 7f6f46a00640 1 --2- 172.21.15.86:0/853909248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f4815c2a0 0x7f6f48098bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:17.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.240+0000 7f6f46a00640 1 -- 172.21.15.86:0/853909248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f48152a00 con 0x7f6f480d16e0 2024-04-25T14:16:17.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.240+0000 7f6f46a00640 1 --2- 172.21.15.86:0/853909248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f480d16e0 0x7f6f48098680 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f6f3401c7a0 tx=0x7f6f3401b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:17.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.240+0000 7f6f3f400640 1 -- 172.21.15.86:0/853909248 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f340be070 con 0x7f6f480d16e0 2024-04-25T14:16:17.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.240+0000 7f6f3f400640 1 -- 172.21.15.86:0/853909248 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f340afce0 con 0x7f6f480d16e0 2024-04-25T14:16:17.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.240+0000 7f6f4e400640 1 -- 172.21.15.86:0/853909248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f4809aeb0 con 0x7f6f480d16e0 2024-04-25T14:16:17.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.240+0000 7f6f3f400640 1 -- 172.21.15.86:0/853909248 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f340b4040 con 0x7f6f480d16e0 2024-04-25T14:16:17.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.240+0000 7f6f4e400640 1 -- 172.21.15.86:0/853909248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f4815f240 con 0x7f6f480d16e0 2024-04-25T14:16:17.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.244+0000 7f6f4e400640 1 -- 172.21.15.86:0/853909248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f4815f030 con 0x7f6f480d16e0 2024-04-25T14:16:17.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.244+0000 7f6f3f400640 1 -- 172.21.15.86:0/853909248 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f6f340c1070 con 0x7f6f480d16e0 2024-04-25T14:16:17.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.244+0000 7f6f3f400640 1 --2- 172.21.15.86:0/853909248 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f6f2806f480 0x7f6f28071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:17.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.244+0000 7f6f3f400640 1 -- 172.21.15.86:0/853909248 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f6f3412cb80 con 0x7f6f480d16e0 2024-04-25T14:16:17.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.244+0000 7f6f47400640 1 --2- 172.21.15.86:0/853909248 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f6f2806f480 0x7f6f28071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:17.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.244+0000 7f6f47400640 1 --2- 172.21.15.86:0/853909248 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f6f2806f480 0x7f6f28071990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6f38000f70 tx=0x7f6f380079e0 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:17.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.248+0000 7f6f3f400640 1 -- 172.21.15.86:0/853909248 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6f34018820 con 0x7f6f480d16e0 2024-04-25T14:16:17.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.436+0000 7f6f4e400640 1 -- 172.21.15.86:0/853909248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f48069d00 con 0x7f6f480d16e0 2024-04-25T14:16:17.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.440+0000 7f6f3f400640 1 -- 172.21.15.86:0/853909248 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f6f340f9900 con 0x7f6f480d16e0 2024-04-25T14:16:17.435 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:17.436 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:17.436 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:16:17.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.440+0000 7f6f4e400640 1 -- 172.21.15.86:0/853909248 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f6f2806f480 msgr2=0x7f6f28071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:17.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.440+0000 7f6f4e400640 1 --2- 172.21.15.86:0/853909248 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f6f2806f480 0x7f6f28071990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6f38000f70 tx=0x7f6f380079e0 comp rx=0 tx=0).stop 2024-04-25T14:16:17.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.440+0000 7f6f4e400640 1 -- 172.21.15.86:0/853909248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f480d16e0 msgr2=0x7f6f48098680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:17.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.440+0000 7f6f4e400640 1 --2- 172.21.15.86:0/853909248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f480d16e0 0x7f6f48098680 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f6f3401c7a0 tx=0x7f6f3401b910 comp rx=0 tx=0).stop 2024-04-25T14:16:17.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.440+0000 7f6f4e400640 1 -- 172.21.15.86:0/853909248 shutdown_connections 2024-04-25T14:16:17.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.440+0000 7f6f4e400640 1 --2- 172.21.15.86:0/853909248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f4815c2a0 0x7f6f48098bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:17.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.440+0000 7f6f4e400640 1 --2- 172.21.15.86:0/853909248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f480d16e0 0x7f6f48098680 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:17.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.440+0000 7f6f4e400640 1 --2- 172.21.15.86:0/853909248 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f6f2806f480 0x7f6f28071990 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:17.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.440+0000 7f6f4e400640 1 --2- 172.21.15.86:0/853909248 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f4806fd50 0x7f6f480c1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:17.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.440+0000 7f6f4e400640 1 -- 172.21.15.86:0/853909248 >> 172.21.15.86:0/853909248 conn(0x7f6f480b4650 msgr2=0x7f6f48159930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:17.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.440+0000 7f6f4e400640 1 -- 172.21.15.86:0/853909248 shutdown_connections 2024-04-25T14:16:17.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:17.440+0000 7f6f4e400640 1 -- 172.21.15.86:0/853909248 wait complete. 2024-04-25T14:16:17.860 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:17.860 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:18.861 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:19.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.020+0000 7f05f5a00640 1 Processor -- start 2024-04-25T14:16:19.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.020+0000 7f05f5a00640 1 -- start start 2024-04-25T14:16:19.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.020+0000 7f05f5a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f05f00d08c0 0x7f05f015c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:19.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.020+0000 7f05f5a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f05f006e830 0x7f05f006ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:19.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.020+0000 7f05f5a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05f006f180 con 0x7f05f00cf380 2024-04-25T14:16:19.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.020+0000 7f05f5a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f05f00af140 con 0x7f05f006e830 2024-04-25T14:16:19.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.020+0000 7f05f5a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f05f006d120 con 0x7f05f00d08c0 2024-04-25T14:16:19.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.020+0000 7f05e7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f05f006e830 0x7f05f006ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:19.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.020+0000 7f05e7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f05f006e830 0x7f05f006ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46458/0 (socket says 172.21.15.86:46458) 2024-04-25T14:16:19.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.020+0000 7f05e7400640 1 -- 172.21.15.86:0/125523250 learned_addr learned my addr 172.21.15.86:0/125523250 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:19.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.024+0000 7f05efe00640 1 --2- 172.21.15.86:0/125523250 >> v2:172.21.15.155:3300/0 conn(0x7f05f00d08c0 0x7f05f015c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:19.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.024+0000 7f05e7400640 1 -- 172.21.15.86:0/125523250 >> v2:172.21.15.155:3300/0 conn(0x7f05f00d08c0 msgr2=0x7f05f015c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:19.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.024+0000 7f05e7400640 1 --2- 172.21.15.86:0/125523250 >> v2:172.21.15.155:3300/0 conn(0x7f05f00d08c0 0x7f05f015c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:19.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.024+0000 7f05e7400640 1 -- 172.21.15.86:0/125523250 >> v1:172.21.15.155:6789/0 conn(0x7f05f00cf380 legacy=0x7f05f00cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:19.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.024+0000 7f05e7400640 1 -- 172.21.15.86:0/125523250 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f006d400 con 0x7f05f006e830 2024-04-25T14:16:19.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.024+0000 7f05e7400640 1 --2- 172.21.15.86:0/125523250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05f006e830 0x7f05f006ec40 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f05f00b3ac0 tx=0x7f05d801c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914cc6b449e22e43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:19.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.024+0000 7f05ef400640 1 -- 172.21.15.86:0/125523250 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d80b0070 con 0x7f05f006e830 2024-04-25T14:16:19.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.024+0000 7f05ef400640 1 -- 172.21.15.86:0/125523250 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05d80ab600 con 0x7f05f006e830 2024-04-25T14:16:19.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.024+0000 7f05ef400640 1 -- 172.21.15.86:0/125523250 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d80ab9d0 con 0x7f05f006e830 2024-04-25T14:16:19.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.028+0000 7f05f5a00640 1 -- 172.21.15.86:0/125523250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05f006e830 msgr2=0x7f05f006ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:19.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.028+0000 7f05f5a00640 1 --2- 172.21.15.86:0/125523250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05f006e830 0x7f05f006ec40 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f05f00b3ac0 tx=0x7f05d801c300 comp rx=0 tx=0).stop 2024-04-25T14:16:19.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.028+0000 7f05f5a00640 1 -- 172.21.15.86:0/125523250 shutdown_connections 2024-04-25T14:16:19.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.028+0000 7f05f5a00640 1 --2- 172.21.15.86:0/125523250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05f006e830 0x7f05f006ec40 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:19.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.028+0000 7f05f5a00640 1 --2- 172.21.15.86:0/125523250 >> v2:172.21.15.155:3300/0 conn(0x7f05f00d08c0 0x7f05f015c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:19.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.028+0000 7f05f5a00640 1 -- 172.21.15.86:0/125523250 >> 172.21.15.86:0/125523250 conn(0x7f05f00b4850 msgr2=0x7f05f00b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:19.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.028+0000 7f05f5a00640 1 -- 172.21.15.86:0/125523250 shutdown_connections 2024-04-25T14:16:19.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.028+0000 7f05f5a00640 1 -- 172.21.15.86:0/125523250 wait complete. 2024-04-25T14:16:19.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.028+0000 7f05f5a00640 1 Processor -- start 2024-04-25T14:16:19.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.028+0000 7f05f5a00640 1 -- start start 2024-04-25T14:16:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.028+0000 7f05f5a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05f006e830 0x7f05f014c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.032+0000 7f05f5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05f00cf380 0x7f05f01569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.032+0000 7f05f5a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05f00d08c0 0x7f05f00b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.032+0000 7f05f5a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05f006f580 con 0x7f05f00cf380 2024-04-25T14:16:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.032+0000 7f05f5a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f05f006dee0 con 0x7f05f00d08c0 2024-04-25T14:16:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.032+0000 7f05f5a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f05f0167d80 con 0x7f05f006e830 2024-04-25T14:16:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.032+0000 7f05e7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05f00cf380 0x7f05f01569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:19.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.032+0000 7f05e7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05f00cf380 0x7f05f01569d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46470/0 (socket says 172.21.15.86:46470) 2024-04-25T14:16:19.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.032+0000 7f05e7400640 1 -- 172.21.15.86:0/4206426131 learned_addr learned my addr 172.21.15.86:0/4206426131 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:19.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.032+0000 7f05efe00640 1 --2- 172.21.15.86:0/4206426131 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05f006e830 0x7f05f014c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:19.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.032+0000 7f05f5000640 1 --2- 172.21.15.86:0/4206426131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05f00d08c0 0x7f05f00b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:19.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.032+0000 7f05e7400640 1 -- 172.21.15.86:0/4206426131 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05f006e830 msgr2=0x7f05f014c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:19.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.032+0000 7f05e7400640 1 --2- 172.21.15.86:0/4206426131 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05f006e830 0x7f05f014c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:19.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.032+0000 7f05e7400640 1 -- 172.21.15.86:0/4206426131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05f00d08c0 msgr2=0x7f05f00b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:19.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.032+0000 7f05e7400640 1 --2- 172.21.15.86:0/4206426131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05f00d08c0 0x7f05f00b1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:19.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.032+0000 7f05e7400640 1 -- 172.21.15.86:0/4206426131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f0156f10 con 0x7f05f00cf380 2024-04-25T14:16:19.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.032+0000 7f05f5000640 1 --2- 172.21.15.86:0/4206426131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05f00d08c0 0x7f05f00b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:16:19.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.032+0000 7f05e7400640 1 --2- 172.21.15.86:0/4206426131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05f00cf380 0x7f05f01569d0 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f05d80ae040 tx=0x7f05d80190a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:19.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.032+0000 7f05ecc00640 1 -- 172.21.15.86:0/4206426131 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d80b0040 con 0x7f05f00cf380 2024-04-25T14:16:19.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.032+0000 7f05ecc00640 1 -- 172.21.15.86:0/4206426131 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05d80b9070 con 0x7f05f00cf380 2024-04-25T14:16:19.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.032+0000 7f05ecc00640 1 -- 172.21.15.86:0/4206426131 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d80b46a0 con 0x7f05f00cf380 2024-04-25T14:16:19.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.032+0000 7f05f5a00640 1 -- 172.21.15.86:0/4206426131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f0157390 con 0x7f05f00cf380 2024-04-25T14:16:19.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.032+0000 7f05f5a00640 1 -- 172.21.15.86:0/4206426131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f0157950 con 0x7f05f00cf380 2024-04-25T14:16:19.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.032+0000 7f05f5a00640 1 -- 172.21.15.86:0/4206426131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05f0157b90 con 0x7f05f00cf380 2024-04-25T14:16:19.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.032+0000 7f05ecc00640 1 -- 172.21.15.86:0/4206426131 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f05d80b4840 con 0x7f05f00cf380 2024-04-25T14:16:19.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.036+0000 7f05ecc00640 1 --2- 172.21.15.86:0/4206426131 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f05cc06f450 0x7f05cc071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:19.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.036+0000 7f05ecc00640 1 -- 172.21.15.86:0/4206426131 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f05d8129290 con 0x7f05f00cf380 2024-04-25T14:16:19.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.036+0000 7f05efe00640 1 --2- 172.21.15.86:0/4206426131 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f05cc06f450 0x7f05cc071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:19.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.036+0000 7f05efe00640 1 --2- 172.21.15.86:0/4206426131 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f05cc06f450 0x7f05cc071960 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f05f00b0d90 tx=0x7f05e0008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:19.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.040+0000 7f05ecc00640 1 -- 172.21.15.86:0/4206426131 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f05f006f180 con 0x7f05f00cf380 2024-04-25T14:16:19.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.228+0000 7f05f5a00640 1 -- 172.21.15.86:0/4206426131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05f0076b80 con 0x7f05f00cf380 2024-04-25T14:16:19.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.228+0000 7f05ecc00640 1 -- 172.21.15.86:0/4206426131 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5220 (secure 0 0 0) 0x7f05d80f5ea0 con 0x7f05f00cf380 2024-04-25T14:16:19.224 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:19.224 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":124,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:14:24.364934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:19.224 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 124 2024-04-25T14:16:19.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.228+0000 7f05f5a00640 1 -- 172.21.15.86:0/4206426131 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f05cc06f450 msgr2=0x7f05cc071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:19.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.228+0000 7f05f5a00640 1 --2- 172.21.15.86:0/4206426131 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f05cc06f450 0x7f05cc071960 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f05f00b0d90 tx=0x7f05e0008040 comp rx=0 tx=0).stop 2024-04-25T14:16:19.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.228+0000 7f05f5a00640 1 -- 172.21.15.86:0/4206426131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05f00cf380 msgr2=0x7f05f01569d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:19.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.228+0000 7f05f5a00640 1 --2- 172.21.15.86:0/4206426131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05f00cf380 0x7f05f01569d0 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f05d80ae040 tx=0x7f05d80190a0 comp rx=0 tx=0).stop 2024-04-25T14:16:19.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.228+0000 7f05f5a00640 1 -- 172.21.15.86:0/4206426131 shutdown_connections 2024-04-25T14:16:19.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.228+0000 7f05f5a00640 1 --2- 172.21.15.86:0/4206426131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05f00d08c0 0x7f05f00b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:19.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.228+0000 7f05f5a00640 1 --2- 172.21.15.86:0/4206426131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05f00cf380 0x7f05f01569d0 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:19.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.228+0000 7f05f5a00640 1 --2- 172.21.15.86:0/4206426131 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f05cc06f450 0x7f05cc071960 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:19.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.228+0000 7f05f5a00640 1 --2- 172.21.15.86:0/4206426131 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05f006e830 0x7f05f014c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:19.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.228+0000 7f05f5a00640 1 -- 172.21.15.86:0/4206426131 >> 172.21.15.86:0/4206426131 conn(0x7f05f00b4850 msgr2=0x7f05f00cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:19.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.228+0000 7f05f5a00640 1 -- 172.21.15.86:0/4206426131 shutdown_connections 2024-04-25T14:16:19.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:19.228+0000 7f05f5a00640 1 -- 172.21.15.86:0/4206426131 wait complete. 2024-04-25T14:16:19.648 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:14:24.364934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:19.649 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:20.578 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:16:20.582 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:16:20.649 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:20.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.807+0000 7feb97000640 1 Processor -- start 2024-04-25T14:16:20.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.807+0000 7feb97000640 1 -- start start 2024-04-25T14:16:20.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.807+0000 7feb97000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb900cfca0 0x7feb9006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:20.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.807+0000 7feb97000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feb900d16e0 0x7feb900d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:20.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.807+0000 7feb97000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7feb9006c670 0x7feb9006ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:20.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.807+0000 7feb97000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7feb90070470 con 0x7feb900d16e0 2024-04-25T14:16:20.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.807+0000 7feb97000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7feb900d2030 con 0x7feb9006c670 2024-04-25T14:16:20.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.807+0000 7feb97000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7feb90072bd0 con 0x7feb900cfca0 2024-04-25T14:16:20.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.807+0000 7feb87400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feb900d16e0 0x7feb900d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:20.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.811+0000 7feb87400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feb900d16e0 0x7feb900d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46480/0 (socket says 172.21.15.86:46480) 2024-04-25T14:16:20.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.811+0000 7feb87400640 1 -- 172.21.15.86:0/2834942508 learned_addr learned my addr 172.21.15.86:0/2834942508 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:20.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.811+0000 7feb8fe00640 1 --2- 172.21.15.86:0/2834942508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb900cfca0 0x7feb9006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:20.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.811+0000 7feb95000640 1 --2- 172.21.15.86:0/2834942508 >> v2:172.21.15.155:3300/0 conn(0x7feb9006c670 0x7feb9006ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:20.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.811+0000 7feb87400640 1 -- 172.21.15.86:0/2834942508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb900cfca0 msgr2=0x7feb9006f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:20.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.811+0000 7feb87400640 1 --2- 172.21.15.86:0/2834942508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb900cfca0 0x7feb9006f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:20.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.811+0000 7feb87400640 1 -- 172.21.15.86:0/2834942508 >> v2:172.21.15.155:3300/0 conn(0x7feb9006c670 msgr2=0x7feb9006ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:20.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.811+0000 7feb87400640 1 --2- 172.21.15.86:0/2834942508 >> v2:172.21.15.155:3300/0 conn(0x7feb9006c670 0x7feb9006ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:20.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.811+0000 7feb87400640 1 -- 172.21.15.86:0/2834942508 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9006f930 con 0x7feb900d16e0 2024-04-25T14:16:20.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.811+0000 7feb95000640 1 --2- 172.21.15.86:0/2834942508 >> v2:172.21.15.155:3300/0 conn(0x7feb9006c670 0x7feb9006ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:16:20.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.811+0000 7feb8fe00640 1 --2- 172.21.15.86:0/2834942508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb900cfca0 0x7feb9006f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:16:20.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.811+0000 7feb87400640 1 --2- 172.21.15.86:0/2834942508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb900d16e0 0x7feb900d1af0 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7feb9006e060 tx=0x7feb7801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d827a982c738772 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:20.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.811+0000 7feb8f400640 1 -- 172.21.15.86:0/2834942508 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb780ab5f0 con 0x7feb900d16e0 2024-04-25T14:16:20.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.811+0000 7feb8f400640 1 -- 172.21.15.86:0/2834942508 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb780b0070 con 0x7feb900d16e0 2024-04-25T14:16:20.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.811+0000 7feb8f400640 1 -- 172.21.15.86:0/2834942508 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb780ab9b0 con 0x7feb900d16e0 2024-04-25T14:16:20.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.815+0000 7feb97000640 1 -- 172.21.15.86:0/2834942508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb900d16e0 msgr2=0x7feb900d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:20.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.815+0000 7feb97000640 1 --2- 172.21.15.86:0/2834942508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb900d16e0 0x7feb900d1af0 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7feb9006e060 tx=0x7feb7801c520 comp rx=0 tx=0).stop 2024-04-25T14:16:20.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.815+0000 7feb97000640 1 -- 172.21.15.86:0/2834942508 shutdown_connections 2024-04-25T14:16:20.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.815+0000 7feb97000640 1 --2- 172.21.15.86:0/2834942508 >> v2:172.21.15.155:3300/0 conn(0x7feb9006c670 0x7feb9006ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:20.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.815+0000 7feb97000640 1 --2- 172.21.15.86:0/2834942508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb900d16e0 0x7feb900d1af0 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:20.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.815+0000 7feb97000640 1 --2- 172.21.15.86:0/2834942508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb900cfca0 0x7feb9006f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:20.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.815+0000 7feb97000640 1 -- 172.21.15.86:0/2834942508 >> 172.21.15.86:0/2834942508 conn(0x7feb900b4650 msgr2=0x7feb900b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:20.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.815+0000 7feb97000640 1 -- 172.21.15.86:0/2834942508 shutdown_connections 2024-04-25T14:16:20.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.815+0000 7feb97000640 1 -- 172.21.15.86:0/2834942508 wait complete. 2024-04-25T14:16:20.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.815+0000 7feb97000640 1 Processor -- start 2024-04-25T14:16:20.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.815+0000 7feb97000640 1 -- start start 2024-04-25T14:16:20.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.819+0000 7feb97000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb9006c670 0x7feb901fc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:20.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.819+0000 7feb97000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb900cfca0 0x7feb9015d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:20.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.819+0000 7feb97000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb900d16e0 0x7feb901fd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:20.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.819+0000 7feb97000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb900738c0 con 0x7feb900cfca0 2024-04-25T14:16:20.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.819+0000 7feb97000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7feb9006d3d0 con 0x7feb9006c670 2024-04-25T14:16:20.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.819+0000 7feb97000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7feb90073a40 con 0x7feb900d16e0 2024-04-25T14:16:20.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.819+0000 7feb87400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb900cfca0 0x7feb9015d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:20.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.819+0000 7feb87400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb900cfca0 0x7feb9015d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46490/0 (socket says 172.21.15.86:46490) 2024-04-25T14:16:20.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.819+0000 7feb87400640 1 -- 172.21.15.86:0/2039423247 learned_addr learned my addr 172.21.15.86:0/2039423247 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:20.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.819+0000 7feb95000640 1 --2- 172.21.15.86:0/2039423247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb900d16e0 0x7feb901fd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:20.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.819+0000 7feb8fe00640 1 --2- 172.21.15.86:0/2039423247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb9006c670 0x7feb901fc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:20.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.819+0000 7feb87400640 1 -- 172.21.15.86:0/2039423247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb900d16e0 msgr2=0x7feb901fd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:20.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.819+0000 7feb87400640 1 --2- 172.21.15.86:0/2039423247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb900d16e0 0x7feb901fd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:20.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.819+0000 7feb87400640 1 -- 172.21.15.86:0/2039423247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb9006c670 msgr2=0x7feb901fc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:20.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.819+0000 7feb87400640 1 --2- 172.21.15.86:0/2039423247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb9006c670 0x7feb901fc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:20.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.819+0000 7feb87400640 1 -- 172.21.15.86:0/2039423247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9015d630 con 0x7feb900cfca0 2024-04-25T14:16:20.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.819+0000 7feb8fe00640 1 --2- 172.21.15.86:0/2039423247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb9006c670 0x7feb901fc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:16:20.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.819+0000 7feb87400640 1 --2- 172.21.15.86:0/2039423247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb900cfca0 0x7feb9015d0f0 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7feb7801ca00 tx=0x7feb7801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:20.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.819+0000 7feb8cc00640 1 -- 172.21.15.86:0/2039423247 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb780be5f0 con 0x7feb900cfca0 2024-04-25T14:16:20.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.819+0000 7feb8cc00640 1 -- 172.21.15.86:0/2039423247 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb780be790 con 0x7feb900cfca0 2024-04-25T14:16:20.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.819+0000 7feb8cc00640 1 -- 172.21.15.86:0/2039423247 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb780beb80 con 0x7feb900cfca0 2024-04-25T14:16:20.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.819+0000 7feb97000640 1 -- 172.21.15.86:0/2039423247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb90155130 con 0x7feb900cfca0 2024-04-25T14:16:20.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.819+0000 7feb97000640 1 -- 172.21.15.86:0/2039423247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb901fd050 con 0x7feb900cfca0 2024-04-25T14:16:20.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.819+0000 7feb97000640 1 -- 172.21.15.86:0/2039423247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb9009f4b0 con 0x7feb900cfca0 2024-04-25T14:16:20.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.823+0000 7feb8cc00640 1 -- 172.21.15.86:0/2039423247 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7feb780b43f0 con 0x7feb900cfca0 2024-04-25T14:16:20.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.823+0000 7feb8cc00640 1 --2- 172.21.15.86:0/2039423247 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7feb6c06f480 0x7feb6c071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:20.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.823+0000 7feb8cc00640 1 -- 172.21.15.86:0/2039423247 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7feb78129a60 con 0x7feb900cfca0 2024-04-25T14:16:20.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.823+0000 7feb8fe00640 1 --2- 172.21.15.86:0/2039423247 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7feb6c06f480 0x7feb6c071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:20.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.823+0000 7feb8fe00640 1 --2- 172.21.15.86:0/2039423247 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7feb6c06f480 0x7feb6c071990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feb80006fd0 tx=0x7feb80008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:20.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:20.827+0000 7feb8cc00640 1 -- 172.21.15.86:0/2039423247 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7feb78016110 con 0x7feb900cfca0 2024-04-25T14:16:21.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:21.019+0000 7feb97000640 1 -- 172.21.15.86:0/2039423247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb90069d00 con 0x7feb900cfca0 2024-04-25T14:16:21.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:21.019+0000 7feb8cc00640 1 -- 172.21.15.86:0/2039423247 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7feb780f6200 con 0x7feb900cfca0 2024-04-25T14:16:21.016 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:21.016 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:21.016 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:16:21.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:21.019+0000 7feb97000640 1 -- 172.21.15.86:0/2039423247 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7feb6c06f480 msgr2=0x7feb6c071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:21.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:21.019+0000 7feb97000640 1 --2- 172.21.15.86:0/2039423247 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7feb6c06f480 0x7feb6c071990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feb80006fd0 tx=0x7feb80008040 comp rx=0 tx=0).stop 2024-04-25T14:16:21.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:21.019+0000 7feb97000640 1 -- 172.21.15.86:0/2039423247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb900cfca0 msgr2=0x7feb9015d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:21.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:21.019+0000 7feb97000640 1 --2- 172.21.15.86:0/2039423247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb900cfca0 0x7feb9015d0f0 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7feb7801ca00 tx=0x7feb7801ad60 comp rx=0 tx=0).stop 2024-04-25T14:16:21.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:21.019+0000 7feb97000640 1 -- 172.21.15.86:0/2039423247 shutdown_connections 2024-04-25T14:16:21.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:21.019+0000 7feb97000640 1 --2- 172.21.15.86:0/2039423247 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7feb6c06f480 0x7feb6c071990 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:21.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:21.019+0000 7feb97000640 1 --2- 172.21.15.86:0/2039423247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb900d16e0 0x7feb901fd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:21.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:21.019+0000 7feb97000640 1 --2- 172.21.15.86:0/2039423247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb900cfca0 0x7feb9015d0f0 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:21.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:21.019+0000 7feb97000640 1 --2- 172.21.15.86:0/2039423247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb9006c670 0x7feb901fc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:21.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:21.019+0000 7feb97000640 1 -- 172.21.15.86:0/2039423247 >> 172.21.15.86:0/2039423247 conn(0x7feb900b4650 msgr2=0x7feb9015c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:21.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:21.019+0000 7feb97000640 1 -- 172.21.15.86:0/2039423247 shutdown_connections 2024-04-25T14:16:21.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:21.019+0000 7feb97000640 1 -- 172.21.15.86:0/2039423247 wait complete. 2024-04-25T14:16:21.441 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:21.441 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:22.442 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:22.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.603+0000 7f3592400640 1 Processor -- start 2024-04-25T14:16:22.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.603+0000 7f3592400640 1 -- start start 2024-04-25T14:16:22.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.603+0000 7f3592400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f358c15c2a0 0x7f358c0ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:22.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.607+0000 7f3592400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f358c06fd50 0x7f358c070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:22.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.607+0000 7f3592400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f358c06c670 0x7f358c06ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:22.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.607+0000 7f3592400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f358c0b0a60 con 0x7f358c06c670 2024-04-25T14:16:22.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.607+0000 7f3592400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f358c06ef90 con 0x7f358c06fd50 2024-04-25T14:16:22.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.607+0000 7f3592400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f358c06f270 con 0x7f358c15c2a0 2024-04-25T14:16:22.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.607+0000 7f358be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f358c06c670 0x7f358c06ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:22.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.607+0000 7f358be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f358c06c670 0x7f358c06ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46502/0 (socket says 172.21.15.86:46502) 2024-04-25T14:16:22.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.607+0000 7f358be00640 1 -- 172.21.15.86:0/3447520214 learned_addr learned my addr 172.21.15.86:0/3447520214 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:22.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.607+0000 7f3582a00640 1 --2- 172.21.15.86:0/3447520214 >> v2:172.21.15.155:3300/0 conn(0x7f358c06fd50 0x7f358c070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:22.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.607+0000 7f358b400640 1 --2- 172.21.15.86:0/3447520214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f358c15c2a0 0x7f358c0ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:22.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.607+0000 7f358be00640 1 -- 172.21.15.86:0/3447520214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f358c15c2a0 msgr2=0x7f358c0ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:22.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.607+0000 7f358be00640 1 --2- 172.21.15.86:0/3447520214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f358c15c2a0 0x7f358c0ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:22.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.607+0000 7f358be00640 1 -- 172.21.15.86:0/3447520214 >> v2:172.21.15.155:3300/0 conn(0x7f358c06fd50 msgr2=0x7f358c070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:22.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.607+0000 7f358be00640 1 --2- 172.21.15.86:0/3447520214 >> v2:172.21.15.155:3300/0 conn(0x7f358c06fd50 0x7f358c070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:22.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.607+0000 7f358be00640 1 -- 172.21.15.86:0/3447520214 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f358c06f550 con 0x7f358c06c670 2024-04-25T14:16:22.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.607+0000 7f3582a00640 1 --2- 172.21.15.86:0/3447520214 >> v2:172.21.15.155:3300/0 conn(0x7f358c06fd50 0x7f358c070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:16:22.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.607+0000 7f358be00640 1 --2- 172.21.15.86:0/3447520214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f358c06c670 0x7f358c06ca80 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f358c070d20 tx=0x7f358401e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745508b32409dc50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:22.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.611+0000 7f358aa00640 1 -- 172.21.15.86:0/3447520214 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35840ad070 con 0x7f358c06c670 2024-04-25T14:16:22.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.611+0000 7f358aa00640 1 -- 172.21.15.86:0/3447520214 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3584004230 con 0x7f358c06c670 2024-04-25T14:16:22.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.611+0000 7f358aa00640 1 -- 172.21.15.86:0/3447520214 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35840a76e0 con 0x7f358c06c670 2024-04-25T14:16:22.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.611+0000 7f3592400640 1 -- 172.21.15.86:0/3447520214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f358c06c670 msgr2=0x7f358c06ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:22.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.611+0000 7f3592400640 1 --2- 172.21.15.86:0/3447520214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f358c06c670 0x7f358c06ca80 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f358c070d20 tx=0x7f358401e910 comp rx=0 tx=0).stop 2024-04-25T14:16:22.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.611+0000 7f3592400640 1 -- 172.21.15.86:0/3447520214 shutdown_connections 2024-04-25T14:16:22.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.611+0000 7f3592400640 1 --2- 172.21.15.86:0/3447520214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f358c06c670 0x7f358c06ca80 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:22.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.611+0000 7f3592400640 1 --2- 172.21.15.86:0/3447520214 >> v2:172.21.15.155:3300/0 conn(0x7f358c06fd50 0x7f358c070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:22.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.611+0000 7f3592400640 1 --2- 172.21.15.86:0/3447520214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f358c15c2a0 0x7f358c0ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:22.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.611+0000 7f3592400640 1 -- 172.21.15.86:0/3447520214 >> 172.21.15.86:0/3447520214 conn(0x7f358c0b4650 msgr2=0x7f358c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:22.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.611+0000 7f3592400640 1 -- 172.21.15.86:0/3447520214 shutdown_connections 2024-04-25T14:16:22.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.611+0000 7f3592400640 1 -- 172.21.15.86:0/3447520214 wait complete. 2024-04-25T14:16:22.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.611+0000 7f3592400640 1 Processor -- start 2024-04-25T14:16:22.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.611+0000 7f3592400640 1 -- start start 2024-04-25T14:16:22.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.615+0000 7f3592400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f358c06c670 0x7f358c09a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:22.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.615+0000 7f3592400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f358c06fd50 0x7f358c1f8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:22.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.615+0000 7f3592400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f358c15c2a0 0x7f358c1f9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:22.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.615+0000 7f3592400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f358c15c6b0 con 0x7f358c06fd50 2024-04-25T14:16:22.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.615+0000 7f3592400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f358c06d3d0 con 0x7f358c06c670 2024-04-25T14:16:22.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.615+0000 7f3592400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f358c15c830 con 0x7f358c15c2a0 2024-04-25T14:16:22.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.615+0000 7f3582a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f358c06fd50 0x7f358c1f8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:22.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.615+0000 7f3582a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f358c06fd50 0x7f358c1f8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46506/0 (socket says 172.21.15.86:46506) 2024-04-25T14:16:22.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.615+0000 7f3582a00640 1 -- 172.21.15.86:0/2256873498 learned_addr learned my addr 172.21.15.86:0/2256873498 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:22.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.615+0000 7f358b400640 1 --2- 172.21.15.86:0/2256873498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f358c06c670 0x7f358c09a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:22.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.615+0000 7f3582a00640 1 -- 172.21.15.86:0/2256873498 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f358c15c2a0 msgr2=0x7f358c1f9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:22.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.615+0000 7f358be00640 1 --2- 172.21.15.86:0/2256873498 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f358c15c2a0 0x7f358c1f9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:22.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.615+0000 7f3582a00640 1 --2- 172.21.15.86:0/2256873498 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f358c15c2a0 0x7f358c1f9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:22.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.615+0000 7f3582a00640 1 -- 172.21.15.86:0/2256873498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f358c06c670 msgr2=0x7f358c09a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:22.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.615+0000 7f3582a00640 1 --2- 172.21.15.86:0/2256873498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f358c06c670 0x7f358c09a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:22.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.615+0000 7f3582a00640 1 -- 172.21.15.86:0/2256873498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f358c166640 con 0x7f358c06fd50 2024-04-25T14:16:22.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.615+0000 7f358b400640 1 --2- 172.21.15.86:0/2256873498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f358c06c670 0x7f358c09a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:16:22.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.615+0000 7f358be00640 1 --2- 172.21.15.86:0/2256873498 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f358c15c2a0 0x7f358c1f9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:16:22.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.615+0000 7f3582a00640 1 --2- 172.21.15.86:0/2256873498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f358c06fd50 0x7f358c1f8ff0 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f357400d830 tx=0x7f357400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:22.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.615+0000 7f3583e00640 1 -- 172.21.15.86:0/2256873498 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3574018070 con 0x7f358c06fd50 2024-04-25T14:16:22.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.615+0000 7f3583e00640 1 -- 172.21.15.86:0/2256873498 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f357400be10 con 0x7f358c06fd50 2024-04-25T14:16:22.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.615+0000 7f3592400640 1 -- 172.21.15.86:0/2256873498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f358c1668b0 con 0x7f358c06fd50 2024-04-25T14:16:22.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.615+0000 7f3583e00640 1 -- 172.21.15.86:0/2256873498 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3574004250 con 0x7f358c06fd50 2024-04-25T14:16:22.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.615+0000 7f3592400640 1 -- 172.21.15.86:0/2256873498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f358c0c42f0 con 0x7f358c06fd50 2024-04-25T14:16:22.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.615+0000 7f3592400640 1 -- 172.21.15.86:0/2256873498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f358c069d00 con 0x7f358c06fd50 2024-04-25T14:16:22.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.619+0000 7f3583e00640 1 -- 172.21.15.86:0/2256873498 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f35740098b0 con 0x7f358c06fd50 2024-04-25T14:16:22.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.619+0000 7f3583e00640 1 --2- 172.21.15.86:0/2256873498 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f356806f480 0x7f3568071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:22.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.619+0000 7f3583e00640 1 -- 172.21.15.86:0/2256873498 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f357408f840 con 0x7f358c06fd50 2024-04-25T14:16:22.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.619+0000 7f358b400640 1 --2- 172.21.15.86:0/2256873498 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f356806f480 0x7f3568071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:22.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.619+0000 7f358b400640 1 --2- 172.21.15.86:0/2256873498 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f356806f480 0x7f3568071990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f358c0b4200 tx=0x7f357c00b040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:22.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.623+0000 7f3583e00640 1 -- 172.21.15.86:0/2256873498 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f357405c440 con 0x7f358c06fd50 2024-04-25T14:16:22.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.815+0000 7f3592400640 1 -- 172.21.15.86:0/2256873498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f358c0c4650 con 0x7f358c06fd50 2024-04-25T14:16:22.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.815+0000 7f3583e00640 1 -- 172.21.15.86:0/2256873498 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f357405bd40 con 0x7f358c06fd50 2024-04-25T14:16:22.812 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:22.812 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:22.813 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:16:22.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.815+0000 7f3592400640 1 -- 172.21.15.86:0/2256873498 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f356806f480 msgr2=0x7f3568071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:22.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.815+0000 7f3592400640 1 --2- 172.21.15.86:0/2256873498 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f356806f480 0x7f3568071990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f358c0b4200 tx=0x7f357c00b040 comp rx=0 tx=0).stop 2024-04-25T14:16:22.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.815+0000 7f3592400640 1 -- 172.21.15.86:0/2256873498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f358c06fd50 msgr2=0x7f358c1f8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:22.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.815+0000 7f3592400640 1 --2- 172.21.15.86:0/2256873498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f358c06fd50 0x7f358c1f8ff0 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f357400d830 tx=0x7f357400dcf0 comp rx=0 tx=0).stop 2024-04-25T14:16:22.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.815+0000 7f3592400640 1 -- 172.21.15.86:0/2256873498 shutdown_connections 2024-04-25T14:16:22.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.815+0000 7f3592400640 1 --2- 172.21.15.86:0/2256873498 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f356806f480 0x7f3568071990 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:22.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.815+0000 7f3592400640 1 --2- 172.21.15.86:0/2256873498 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f358c15c2a0 0x7f358c1f9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:22.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.815+0000 7f3592400640 1 --2- 172.21.15.86:0/2256873498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f358c06fd50 0x7f358c1f8ff0 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:22.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.815+0000 7f3592400640 1 --2- 172.21.15.86:0/2256873498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f358c06c670 0x7f358c09a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:22.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.815+0000 7f3592400640 1 -- 172.21.15.86:0/2256873498 >> 172.21.15.86:0/2256873498 conn(0x7f358c0b4650 msgr2=0x7f358c15e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:22.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.815+0000 7f3592400640 1 -- 172.21.15.86:0/2256873498 shutdown_connections 2024-04-25T14:16:22.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:22.815+0000 7f3592400640 1 -- 172.21.15.86:0/2256873498 wait complete. 2024-04-25T14:16:23.237 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:23.237 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:24.239 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:24.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.399+0000 7f1c16800640 1 Processor -- start 2024-04-25T14:16:24.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.399+0000 7f1c16800640 1 -- start start 2024-04-25T14:16:24.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c16800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c1015c2a0 0x7f1c100ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:24.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c16800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c100b0a60 con 0x7f1c1006fd50 2024-04-25T14:16:24.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c16800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c1006ef90 con 0x7f1c1015feb0 2024-04-25T14:16:24.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c16800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1c1006f270 con 0x7f1c1015c2a0 2024-04-25T14:16:24.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c06a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1c1006fd50 0x7f1c10070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36132/0 (socket says 172.21.15.86:36132) 2024-04-25T14:16:24.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c06a00640 1 -- 172.21.15.86:0/20289541 learned_addr learned my addr 172.21.15.86:0/20289541 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:16:24.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c0f400640 1 --2- 172.21.15.86:0/20289541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c1015c2a0 0x7f1c100ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:24.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c0ea00640 1 -- 172.21.15.86:0/20289541 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 301392475 0 0) 0x7f1c100b0a60 con 0x7f1c1006fd50 2024-04-25T14:16:24.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c0ea00640 1 -- 172.21.15.86:0/20289541 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1bf4003530 con 0x7f1c1006fd50 2024-04-25T14:16:24.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c0ea00640 1 -- 172.21.15.86:0/20289541 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2995272720 0 0) 0x7f1bf4003530 con 0x7f1c1006fd50 2024-04-25T14:16:24.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c0ea00640 1 -- 172.21.15.86:0/20289541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c1015c2a0 msgr2=0x7f1c100ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:24.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c0ea00640 1 --2- 172.21.15.86:0/20289541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c1015c2a0 0x7f1c100ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:24.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c0ea00640 1 -- 172.21.15.86:0/20289541 >> v1:172.21.15.155:6789/0 conn(0x7f1c1015feb0 legacy=0x7f1c101602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:24.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c0ea00640 1 -- 172.21.15.86:0/20289541 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1006f550 con 0x7f1c1006fd50 2024-04-25T14:16:24.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c0ea00640 1 -- 172.21.15.86:0/20289541 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f1c00001680 con 0x7f1c1006fd50 2024-04-25T14:16:24.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c0ea00640 1 -- 172.21.15.86:0/20289541 >> v1:172.21.15.86:6789/0 conn(0x7f1c1006fd50 legacy=0x7f1c10070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:24.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c0ea00640 1 --2- 172.21.15.86:0/20289541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bf4003a60 0x7f1bf4003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:24.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c0ea00640 1 --2- 172.21.15.86:0/20289541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c1015c2a0 0x7f1bf4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:24.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c0ea00640 1 --2- 172.21.15.86:0/20289541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bf4004d50 0x7f1bf4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:24.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c0ea00640 1 -- 172.21.15.86:0/20289541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1006f270 con 0x7f1bf4003a60 2024-04-25T14:16:24.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c0ea00640 1 -- 172.21.15.86:0/20289541 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf400bb90 con 0x7f1bf4004d50 2024-04-25T14:16:24.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c0ea00640 1 -- 172.21.15.86:0/20289541 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1bf400bd90 con 0x7f1c1015c2a0 2024-04-25T14:16:24.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c0f400640 1 --2- 172.21.15.86:0/20289541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bf4003a60 0x7f1bf4003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:24.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c06a00640 1 --2- 172.21.15.86:0/20289541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c1015c2a0 0x7f1bf4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:24.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c0fe00640 1 --2- 172.21.15.86:0/20289541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bf4004d50 0x7f1bf4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:24.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c0f400640 1 -- 172.21.15.86:0/20289541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c1015c2a0 msgr2=0x7f1bf4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:24.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c0f400640 1 --2- 172.21.15.86:0/20289541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c1015c2a0 0x7f1bf4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:24.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c0f400640 1 -- 172.21.15.86:0/20289541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bf4004d50 msgr2=0x7f1bf4005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:24.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c0f400640 1 --2- 172.21.15.86:0/20289541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bf4004d50 0x7f1bf4005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:24.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c0f400640 1 -- 172.21.15.86:0/20289541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf4003530 con 0x7f1bf4003a60 2024-04-25T14:16:24.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.403+0000 7f1c0fe00640 1 --2- 172.21.15.86:0/20289541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bf4004d50 0x7f1bf4005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:16:24.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.407+0000 7f1c0f400640 1 --2- 172.21.15.86:0/20289541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bf4003a60 0x7f1bf4003e90 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f1c10070bb0 tx=0x7f1bf801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36c2b1997563d49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:24.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.407+0000 7f1c0ea00640 1 -- 172.21.15.86:0/20289541 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf80096b0 con 0x7f1bf4003a60 2024-04-25T14:16:24.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.407+0000 7f1c0ea00640 1 -- 172.21.15.86:0/20289541 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bf80a6470 con 0x7f1bf4003a60 2024-04-25T14:16:24.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.407+0000 7f1c0ea00640 1 -- 172.21.15.86:0/20289541 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf80a6820 con 0x7f1bf4003a60 2024-04-25T14:16:24.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.407+0000 7f1c16800640 1 -- 172.21.15.86:0/20289541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bf4003a60 msgr2=0x7f1bf4003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:24.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.407+0000 7f1c16800640 1 --2- 172.21.15.86:0/20289541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bf4003a60 0x7f1bf4003e90 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f1c10070bb0 tx=0x7f1bf801ca00 comp rx=0 tx=0).stop 2024-04-25T14:16:24.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.407+0000 7f1c0fe00640 1 -- 172.21.15.86:0/20289541 reap_dead start 2024-04-25T14:16:24.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.407+0000 7f1c16800640 1 -- 172.21.15.86:0/20289541 shutdown_connections 2024-04-25T14:16:24.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.407+0000 7f1c16800640 1 -- 172.21.15.86:0/20289541 >> 172.21.15.86:0/20289541 conn(0x7f1c100b4650 msgr2=0x7f1c100b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:24.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.407+0000 7f1c16800640 1 -- 172.21.15.86:0/20289541 shutdown_connections 2024-04-25T14:16:24.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.407+0000 7f1c16800640 1 -- 172.21.15.86:0/20289541 wait complete. 2024-04-25T14:16:24.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.411+0000 7f1c16800640 1 Processor -- start 2024-04-25T14:16:24.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.411+0000 7f1c16800640 1 -- start start 2024-04-25T14:16:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.411+0000 7f1c16800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bf4003a60 0x7f1c10074010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.411+0000 7f1c16800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c101579a0 0x7f1c1015e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.411+0000 7f1c16800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c10157db0 0x7f1c1015e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.411+0000 7f1c16800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c100ae2b0 con 0x7f1c101579a0 2024-04-25T14:16:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.411+0000 7f1c16800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1c100ae130 con 0x7f1c10157db0 2024-04-25T14:16:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.411+0000 7f1c16800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1c100ae430 con 0x7f1bf4003a60 2024-04-25T14:16:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.411+0000 7f1c06a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c101579a0 0x7f1c1015e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.411+0000 7f1c06a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c101579a0 0x7f1c1015e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46532/0 (socket says 172.21.15.86:46532) 2024-04-25T14:16:24.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.411+0000 7f1c06a00640 1 -- 172.21.15.86:0/3080114335 learned_addr learned my addr 172.21.15.86:0/3080114335 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:24.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.411+0000 7f1c0fe00640 1 --2- 172.21.15.86:0/3080114335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c10157db0 0x7f1c1015e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:24.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.411+0000 7f1c0f400640 1 --2- 172.21.15.86:0/3080114335 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bf4003a60 0x7f1c10074010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:24.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.411+0000 7f1c06a00640 1 -- 172.21.15.86:0/3080114335 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bf4003a60 msgr2=0x7f1c10074010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:24.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.411+0000 7f1c06a00640 1 --2- 172.21.15.86:0/3080114335 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bf4003a60 0x7f1c10074010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:24.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.411+0000 7f1c06a00640 1 -- 172.21.15.86:0/3080114335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c10157db0 msgr2=0x7f1c1015e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:24.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.411+0000 7f1c06a00640 1 --2- 172.21.15.86:0/3080114335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c10157db0 0x7f1c1015e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:24.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.411+0000 7f1c06a00640 1 -- 172.21.15.86:0/3080114335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c100afba0 con 0x7f1c101579a0 2024-04-25T14:16:24.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.411+0000 7f1c06a00640 1 --2- 172.21.15.86:0/3080114335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c101579a0 0x7f1c1015e3c0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f1c00004eb0 tx=0x7f1c00007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:24.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.411+0000 7f1c07e00640 1 -- 172.21.15.86:0/3080114335 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0000f7b0 con 0x7f1c101579a0 2024-04-25T14:16:24.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.415+0000 7f1c07e00640 1 -- 172.21.15.86:0/3080114335 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c00018070 con 0x7f1c101579a0 2024-04-25T14:16:24.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.415+0000 7f1c07e00640 1 -- 172.21.15.86:0/3080114335 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0000fbd0 con 0x7f1c101579a0 2024-04-25T14:16:24.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.415+0000 7f1c16800640 1 -- 172.21.15.86:0/3080114335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c100b1130 con 0x7f1c101579a0 2024-04-25T14:16:24.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.415+0000 7f1c16800640 1 -- 172.21.15.86:0/3080114335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c100b16f0 con 0x7f1c101579a0 2024-04-25T14:16:24.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.415+0000 7f1c16800640 1 -- 172.21.15.86:0/3080114335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c100b0ea0 con 0x7f1c101579a0 2024-04-25T14:16:24.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.415+0000 7f1c07e00640 1 -- 172.21.15.86:0/3080114335 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f1c0001d460 con 0x7f1c101579a0 2024-04-25T14:16:24.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.415+0000 7f1c07e00640 1 --2- 172.21.15.86:0/3080114335 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1bec06f450 0x7f1bec071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:24.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.415+0000 7f1c07e00640 1 -- 172.21.15.86:0/3080114335 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f1c0008f4b0 con 0x7f1c101579a0 2024-04-25T14:16:24.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.415+0000 7f1c0f400640 1 --2- 172.21.15.86:0/3080114335 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1bec06f450 0x7f1bec071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:24.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.419+0000 7f1c0f400640 1 --2- 172.21.15.86:0/3080114335 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1bec06f450 0x7f1bec071960 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1bf80ad000 tx=0x7f1bf80046d0 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:24.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.419+0000 7f1c07e00640 1 -- 172.21.15.86:0/3080114335 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1c0005c020 con 0x7f1c101579a0 2024-04-25T14:16:24.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.611+0000 7f1c16800640 1 -- 172.21.15.86:0/3080114335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c10069d00 con 0x7f1c101579a0 2024-04-25T14:16:24.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.611+0000 7f1c07e00640 1 -- 172.21.15.86:0/3080114335 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f1c0005b920 con 0x7f1c101579a0 2024-04-25T14:16:24.608 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:24.608 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:24.609 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:16:24.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.611+0000 7f1c16800640 1 -- 172.21.15.86:0/3080114335 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1bec06f450 msgr2=0x7f1bec071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:24.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.611+0000 7f1c16800640 1 --2- 172.21.15.86:0/3080114335 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1bec06f450 0x7f1bec071960 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1bf80ad000 tx=0x7f1bf80046d0 comp rx=0 tx=0).stop 2024-04-25T14:16:24.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.611+0000 7f1c16800640 1 -- 172.21.15.86:0/3080114335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c101579a0 msgr2=0x7f1c1015e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:24.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.611+0000 7f1c16800640 1 --2- 172.21.15.86:0/3080114335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c101579a0 0x7f1c1015e3c0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f1c00004eb0 tx=0x7f1c00007430 comp rx=0 tx=0).stop 2024-04-25T14:16:24.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.611+0000 7f1c16800640 1 -- 172.21.15.86:0/3080114335 shutdown_connections 2024-04-25T14:16:24.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.611+0000 7f1c16800640 1 --2- 172.21.15.86:0/3080114335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c10157db0 0x7f1c1015e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:24.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.611+0000 7f1c16800640 1 --2- 172.21.15.86:0/3080114335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c101579a0 0x7f1c1015e3c0 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:24.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.611+0000 7f1c16800640 1 --2- 172.21.15.86:0/3080114335 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1bec06f450 0x7f1bec071960 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:24.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.611+0000 7f1c16800640 1 --2- 172.21.15.86:0/3080114335 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bf4003a60 0x7f1c10074010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:24.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.611+0000 7f1c16800640 1 -- 172.21.15.86:0/3080114335 >> 172.21.15.86:0/3080114335 conn(0x7f1c100b4650 msgr2=0x7f1c10074710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:24.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.611+0000 7f1c16800640 1 -- 172.21.15.86:0/3080114335 shutdown_connections 2024-04-25T14:16:24.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:24.611+0000 7f1c16800640 1 -- 172.21.15.86:0/3080114335 wait complete. 2024-04-25T14:16:25.033 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:25.033 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:26.034 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:26.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.195+0000 7fb356800640 1 Processor -- start 2024-04-25T14:16:26.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.195+0000 7fb356800640 1 -- start start 2024-04-25T14:16:26.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.195+0000 7fb356800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb3500cfca0 0x7fb35006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:26.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.195+0000 7fb356800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb3500730b0 0x7fb3500734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:26.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.195+0000 7fb356800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb350072bd0 con 0x7fb3500d16e0 2024-04-25T14:16:26.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.195+0000 7fb356800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb350070470 con 0x7fb3500cfca0 2024-04-25T14:16:26.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.195+0000 7fb356800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb35006f930 con 0x7fb3500730b0 2024-04-25T14:16:26.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.195+0000 7fb34f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb3500cfca0 0x7fb35006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:26.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.195+0000 7fb34f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb3500cfca0 0x7fb35006f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46548/0 (socket says 172.21.15.86:46548) 2024-04-25T14:16:26.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.195+0000 7fb34f400640 1 -- 172.21.15.86:0/767287257 learned_addr learned my addr 172.21.15.86:0/767287257 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:26.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.195+0000 7fb34fe00640 1 --2- 172.21.15.86:0/767287257 >> v2:172.21.15.155:3300/0 conn(0x7fb3500730b0 0x7fb3500734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:26.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.199+0000 7fb34f400640 1 -- 172.21.15.86:0/767287257 >> v2:172.21.15.155:3300/0 conn(0x7fb3500730b0 msgr2=0x7fb3500734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:26.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.199+0000 7fb34f400640 1 --2- 172.21.15.86:0/767287257 >> v2:172.21.15.155:3300/0 conn(0x7fb3500730b0 0x7fb3500734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:26.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.199+0000 7fb34f400640 1 -- 172.21.15.86:0/767287257 >> v1:172.21.15.155:6789/0 conn(0x7fb3500d16e0 legacy=0x7fb3500d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:26.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.199+0000 7fb34f400640 1 -- 172.21.15.86:0/767287257 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35006fbd0 con 0x7fb3500cfca0 2024-04-25T14:16:26.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.199+0000 7fb34fe00640 1 --2- 172.21.15.86:0/767287257 >> v2:172.21.15.155:3300/0 conn(0x7fb3500730b0 0x7fb3500734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:16:26.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.199+0000 7fb34f400640 1 --2- 172.21.15.86:0/767287257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3500cfca0 0x7fb35006f3f0 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7fb3500cb8c0 tx=0x7fb34001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a436a8bf710126 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:26.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.199+0000 7fb34ea00640 1 -- 172.21.15.86:0/767287257 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3400a57d0 con 0x7fb3500cfca0 2024-04-25T14:16:26.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.199+0000 7fb34ea00640 1 -- 172.21.15.86:0/767287257 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3400a5970 con 0x7fb3500cfca0 2024-04-25T14:16:26.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.199+0000 7fb34ea00640 1 -- 172.21.15.86:0/767287257 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3400a5d40 con 0x7fb3500cfca0 2024-04-25T14:16:26.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.203+0000 7fb356800640 1 -- 172.21.15.86:0/767287257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3500cfca0 msgr2=0x7fb35006f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:26.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.203+0000 7fb356800640 1 --2- 172.21.15.86:0/767287257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3500cfca0 0x7fb35006f3f0 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7fb3500cb8c0 tx=0x7fb34001c9f0 comp rx=0 tx=0).stop 2024-04-25T14:16:26.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.203+0000 7fb356800640 1 -- 172.21.15.86:0/767287257 shutdown_connections 2024-04-25T14:16:26.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.203+0000 7fb356800640 1 --2- 172.21.15.86:0/767287257 >> v2:172.21.15.155:3300/0 conn(0x7fb3500730b0 0x7fb3500734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:26.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.203+0000 7fb356800640 1 --2- 172.21.15.86:0/767287257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3500cfca0 0x7fb35006f3f0 unknown :-1 s=CLOSED pgs=2285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:26.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.203+0000 7fb356800640 1 -- 172.21.15.86:0/767287257 >> 172.21.15.86:0/767287257 conn(0x7fb3500b4650 msgr2=0x7fb3500b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:26.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.203+0000 7fb356800640 1 -- 172.21.15.86:0/767287257 shutdown_connections 2024-04-25T14:16:26.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.203+0000 7fb356800640 1 -- 172.21.15.86:0/767287257 wait complete. 2024-04-25T14:16:26.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.203+0000 7fb356800640 1 Processor -- start 2024-04-25T14:16:26.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.203+0000 7fb356800640 1 -- start start 2024-04-25T14:16:26.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.203+0000 7fb356800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3500730b0 0x7fb350099f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:26.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.203+0000 7fb356800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3500cfca0 0x7fb3500af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:26.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.203+0000 7fb356800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb3500d16e0 0x7fb3500b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:26.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.203+0000 7fb356800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb350169a70 con 0x7fb3500730b0 2024-04-25T14:16:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.203+0000 7fb356800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb3501698f0 con 0x7fb3500cfca0 2024-04-25T14:16:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.203+0000 7fb356800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb350169bf0 con 0x7fb3500d16e0 2024-04-25T14:16:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.207+0000 7fb34f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3500730b0 0x7fb350099f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.207+0000 7fb34f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3500730b0 0x7fb350099f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46556/0 (socket says 172.21.15.86:46556) 2024-04-25T14:16:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.207+0000 7fb34f400640 1 -- 172.21.15.86:0/3238397464 learned_addr learned my addr 172.21.15.86:0/3238397464 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.207+0000 7fb346a00640 1 --2- 172.21.15.86:0/3238397464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3500cfca0 0x7fb3500af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.207+0000 7fb34fe00640 1 --2- 172.21.15.86:0/3238397464 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb3500d16e0 0x7fb3500b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.207+0000 7fb34f400640 1 -- 172.21.15.86:0/3238397464 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb3500d16e0 msgr2=0x7fb3500b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.207+0000 7fb34f400640 1 --2- 172.21.15.86:0/3238397464 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb3500d16e0 0x7fb3500b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.207+0000 7fb34f400640 1 -- 172.21.15.86:0/3238397464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3500cfca0 msgr2=0x7fb3500af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.207+0000 7fb34f400640 1 --2- 172.21.15.86:0/3238397464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3500cfca0 0x7fb3500af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.207+0000 7fb34f400640 1 -- 172.21.15.86:0/3238397464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3500af140 con 0x7fb3500730b0 2024-04-25T14:16:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.207+0000 7fb346a00640 1 --2- 172.21.15.86:0/3238397464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3500cfca0 0x7fb3500af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:16:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.207+0000 7fb34f400640 1 --2- 172.21.15.86:0/3238397464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3500730b0 0x7fb350099f20 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7fb3400af040 tx=0x7fb340019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:26.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.207+0000 7fb347e00640 1 -- 172.21.15.86:0/3238397464 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3400b8670 con 0x7fb3500730b0 2024-04-25T14:16:26.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.207+0000 7fb347e00640 1 -- 172.21.15.86:0/3238397464 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3400be070 con 0x7fb3500730b0 2024-04-25T14:16:26.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.207+0000 7fb356800640 1 -- 172.21.15.86:0/3238397464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3500ae380 con 0x7fb3500730b0 2024-04-25T14:16:26.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.207+0000 7fb347e00640 1 -- 172.21.15.86:0/3238397464 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3400b8a70 con 0x7fb3500730b0 2024-04-25T14:16:26.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.207+0000 7fb356800640 1 -- 172.21.15.86:0/3238397464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3500ae940 con 0x7fb3500730b0 2024-04-25T14:16:26.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.207+0000 7fb356800640 1 -- 172.21.15.86:0/3238397464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb35015f000 con 0x7fb3500730b0 2024-04-25T14:16:26.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.207+0000 7fb347e00640 1 -- 172.21.15.86:0/3238397464 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7fb3400b8c10 con 0x7fb3500730b0 2024-04-25T14:16:26.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.211+0000 7fb347e00640 1 --2- 172.21.15.86:0/3238397464 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fb32c06f480 0x7fb32c071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:26.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.211+0000 7fb347e00640 1 -- 172.21.15.86:0/3238397464 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fb34012a6d0 con 0x7fb3500730b0 2024-04-25T14:16:26.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.211+0000 7fb346a00640 1 --2- 172.21.15.86:0/3238397464 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fb32c06f480 0x7fb32c071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:26.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.211+0000 7fb346a00640 1 --2- 172.21.15.86:0/3238397464 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fb32c06f480 0x7fb32c071990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb3500ce110 tx=0x7fb338009040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:26.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.215+0000 7fb347e00640 1 -- 172.21.15.86:0/3238397464 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb3400165e0 con 0x7fb3500730b0 2024-04-25T14:16:26.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.403+0000 7fb356800640 1 -- 172.21.15.86:0/3238397464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb35009f4b0 con 0x7fb3500730b0 2024-04-25T14:16:26.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.403+0000 7fb347e00640 1 -- 172.21.15.86:0/3238397464 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7fb3400f7240 con 0x7fb3500730b0 2024-04-25T14:16:26.402 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:26.402 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:26.403 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:16:26.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.407+0000 7fb356800640 1 -- 172.21.15.86:0/3238397464 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fb32c06f480 msgr2=0x7fb32c071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:26.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.407+0000 7fb356800640 1 --2- 172.21.15.86:0/3238397464 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fb32c06f480 0x7fb32c071990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb3500ce110 tx=0x7fb338009040 comp rx=0 tx=0).stop 2024-04-25T14:16:26.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.407+0000 7fb356800640 1 -- 172.21.15.86:0/3238397464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3500730b0 msgr2=0x7fb350099f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:26.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.407+0000 7fb356800640 1 --2- 172.21.15.86:0/3238397464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3500730b0 0x7fb350099f20 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7fb3400af040 tx=0x7fb340019790 comp rx=0 tx=0).stop 2024-04-25T14:16:26.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.407+0000 7fb356800640 1 -- 172.21.15.86:0/3238397464 shutdown_connections 2024-04-25T14:16:26.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.407+0000 7fb356800640 1 --2- 172.21.15.86:0/3238397464 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fb32c06f480 0x7fb32c071990 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:26.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.407+0000 7fb356800640 1 --2- 172.21.15.86:0/3238397464 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb3500d16e0 0x7fb3500b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:26.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.407+0000 7fb356800640 1 --2- 172.21.15.86:0/3238397464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3500cfca0 0x7fb3500af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:26.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.407+0000 7fb356800640 1 --2- 172.21.15.86:0/3238397464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3500730b0 0x7fb350099f20 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:26.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.407+0000 7fb356800640 1 -- 172.21.15.86:0/3238397464 >> 172.21.15.86:0/3238397464 conn(0x7fb3500b4650 msgr2=0x7fb3500b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:26.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.407+0000 7fb356800640 1 -- 172.21.15.86:0/3238397464 shutdown_connections 2024-04-25T14:16:26.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:26.407+0000 7fb356800640 1 -- 172.21.15.86:0/3238397464 wait complete. 2024-04-25T14:16:26.826 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:26.826 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:27.828 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:27.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.991+0000 7f1e58a00640 1 Processor -- start 2024-04-25T14:16:27.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.991+0000 7f1e58a00640 1 -- start start 2024-04-25T14:16:27.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.991+0000 7f1e58a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e540b3950 0x7f1e5415cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:27.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.991+0000 7f1e58a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1e540af8b0 0x7f1e540afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:27.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.991+0000 7f1e58a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e540b3d60 con 0x7f1e541607a0 2024-04-25T14:16:27.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.991+0000 7f1e58a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1e540af140 con 0x7f1e540af8b0 2024-04-25T14:16:27.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.991+0000 7f1e58a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1e540af420 con 0x7f1e540b3950 2024-04-25T14:16:27.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.991+0000 7f1e52800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1e540af8b0 0x7f1e540afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:27.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.991+0000 7f1e52800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1e540af8b0 0x7f1e540afcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46564/0 (socket says 172.21.15.86:46564) 2024-04-25T14:16:27.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.991+0000 7f1e52800640 1 -- 172.21.15.86:0/95661023 learned_addr learned my addr 172.21.15.86:0/95661023 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:27.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.991+0000 7f1e51e00640 1 --2- 172.21.15.86:0/95661023 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e540b3950 0x7f1e5415cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:27.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.991+0000 7f1e52800640 1 -- 172.21.15.86:0/95661023 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e540b3950 msgr2=0x7f1e5415cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:27.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.991+0000 7f1e52800640 1 --2- 172.21.15.86:0/95661023 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e540b3950 0x7f1e5415cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:27.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.991+0000 7f1e52800640 1 -- 172.21.15.86:0/95661023 >> v1:172.21.15.155:6789/0 conn(0x7f1e541607a0 legacy=0x7f1e54160bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:27.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.991+0000 7f1e52800640 1 -- 172.21.15.86:0/95661023 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e540b1d50 con 0x7f1e540af8b0 2024-04-25T14:16:27.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.991+0000 7f1e51e00640 1 --2- 172.21.15.86:0/95661023 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e540b3950 0x7f1e5415cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:16:27.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.995+0000 7f1e52800640 1 --2- 172.21.15.86:0/95661023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e540af8b0 0x7f1e540afcc0 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009870 tx=0x7f1e4c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d244a869ea21693 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:27.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.995+0000 7f1e50a00640 1 -- 172.21.15.86:0/95661023 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c0ab650 con 0x7f1e540af8b0 2024-04-25T14:16:27.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.995+0000 7f1e50a00640 1 -- 172.21.15.86:0/95661023 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e4c0b0070 con 0x7f1e540af8b0 2024-04-25T14:16:27.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.995+0000 7f1e50a00640 1 -- 172.21.15.86:0/95661023 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c0aba10 con 0x7f1e540af8b0 2024-04-25T14:16:27.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.995+0000 7f1e58a00640 1 -- 172.21.15.86:0/95661023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e540af8b0 msgr2=0x7f1e540afcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:27.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.995+0000 7f1e58a00640 1 --2- 172.21.15.86:0/95661023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e540af8b0 0x7f1e540afcc0 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009870 tx=0x7f1e4c01c490 comp rx=0 tx=0).stop 2024-04-25T14:16:27.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.995+0000 7f1e58a00640 1 -- 172.21.15.86:0/95661023 shutdown_connections 2024-04-25T14:16:27.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.995+0000 7f1e58a00640 1 --2- 172.21.15.86:0/95661023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e540af8b0 0x7f1e540afcc0 unknown :-1 s=CLOSED pgs=2287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:27.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.995+0000 7f1e58a00640 1 --2- 172.21.15.86:0/95661023 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e540b3950 0x7f1e5415cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:27.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.995+0000 7f1e58a00640 1 -- 172.21.15.86:0/95661023 >> 172.21.15.86:0/95661023 conn(0x7f1e5409ba30 msgr2=0x7f1e540b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:27.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.995+0000 7f1e58a00640 1 -- 172.21.15.86:0/95661023 shutdown_connections 2024-04-25T14:16:27.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.995+0000 7f1e58a00640 1 -- 172.21.15.86:0/95661023 wait complete. 2024-04-25T14:16:27.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.999+0000 7f1e58a00640 1 Processor -- start 2024-04-25T14:16:27.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.999+0000 7f1e58a00640 1 -- start start 2024-04-25T14:16:27.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.999+0000 7f1e58a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e540af8b0 0x7f1e541f9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:27.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.999+0000 7f1e58a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e540b3950 0x7f1e540d2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.999+0000 7f1e58a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e541607a0 0x7f1e54156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.999+0000 7f1e58a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e54169ae0 con 0x7f1e540b3950 2024-04-25T14:16:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.999+0000 7f1e58a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1e541614d0 con 0x7f1e541607a0 2024-04-25T14:16:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.999+0000 7f1e58a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1e54169c60 con 0x7f1e540af8b0 2024-04-25T14:16:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.999+0000 7f1e51400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e540b3950 0x7f1e540d2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.999+0000 7f1e51400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e540b3950 0x7f1e540d2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46578/0 (socket says 172.21.15.86:46578) 2024-04-25T14:16:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.999+0000 7f1e51400640 1 -- 172.21.15.86:0/3375392363 learned_addr learned my addr 172.21.15.86:0/3375392363 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.999+0000 7f1e51e00640 1 --2- 172.21.15.86:0/3375392363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e540af8b0 0x7f1e541f9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.999+0000 7f1e52800640 1 --2- 172.21.15.86:0/3375392363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e541607a0 0x7f1e54156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.999+0000 7f1e51400640 1 -- 172.21.15.86:0/3375392363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e540af8b0 msgr2=0x7f1e541f9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.999+0000 7f1e51400640 1 --2- 172.21.15.86:0/3375392363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e540af8b0 0x7f1e541f9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.999+0000 7f1e51400640 1 -- 172.21.15.86:0/3375392363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e541607a0 msgr2=0x7f1e54156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.999+0000 7f1e51400640 1 --2- 172.21.15.86:0/3375392363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e541607a0 0x7f1e54156ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:27.999+0000 7f1e51400640 1 -- 172.21.15.86:0/3375392363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e540cca90 con 0x7f1e540b3950 2024-04-25T14:16:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:28.003+0000 7f1e51400640 1 --2- 172.21.15.86:0/3375392363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e540b3950 0x7f1e540d2210 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f1e4000b440 tx=0x7f1e4000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:28.003+0000 7f1e4a000640 1 -- 172.21.15.86:0/3375392363 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40015020 con 0x7f1e540b3950 2024-04-25T14:16:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:28.003+0000 7f1e4a000640 1 -- 172.21.15.86:0/3375392363 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e40011890 con 0x7f1e540b3950 2024-04-25T14:16:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:28.003+0000 7f1e4a000640 1 -- 172.21.15.86:0/3375392363 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40011c80 con 0x7f1e540b3950 2024-04-25T14:16:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:28.003+0000 7f1e58a00640 1 -- 172.21.15.86:0/3375392363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e5415f030 con 0x7f1e540b3950 2024-04-25T14:16:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:28.003+0000 7f1e58a00640 1 -- 172.21.15.86:0/3375392363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e540ca470 con 0x7f1e540b3950 2024-04-25T14:16:28.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:28.003+0000 7f1e58a00640 1 -- 172.21.15.86:0/3375392363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e5415ed80 con 0x7f1e540b3950 2024-04-25T14:16:28.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:28.003+0000 7f1e4a000640 1 -- 172.21.15.86:0/3375392363 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f1e40018020 con 0x7f1e540b3950 2024-04-25T14:16:28.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:28.007+0000 7f1e4a000640 1 --2- 172.21.15.86:0/3375392363 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1e3406f480 0x7f1e34071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:28.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:28.007+0000 7f1e4a000640 1 -- 172.21.15.86:0/3375392363 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f1e4008f3b0 con 0x7f1e540b3950 2024-04-25T14:16:28.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:28.007+0000 7f1e51e00640 1 --2- 172.21.15.86:0/3375392363 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1e3406f480 0x7f1e34071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:28.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:28.007+0000 7f1e51e00640 1 --2- 172.21.15.86:0/3375392363 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1e3406f480 0x7f1e34071990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1e440070b0 tx=0x7f1e44007040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:28.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:28.011+0000 7f1e4a000640 1 -- 172.21.15.86:0/3375392363 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1e4001f030 con 0x7f1e540b3950 2024-04-25T14:16:28.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:28.199+0000 7f1e58a00640 1 -- 172.21.15.86:0/3375392363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e540a04c0 con 0x7f1e540b3950 2024-04-25T14:16:28.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:28.199+0000 7f1e4a000640 1 -- 172.21.15.86:0/3375392363 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f1e4000d040 con 0x7f1e540b3950 2024-04-25T14:16:28.199 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:28.199 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:28.199 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:16:28.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:28.203+0000 7f1e58a00640 1 -- 172.21.15.86:0/3375392363 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1e3406f480 msgr2=0x7f1e34071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:28.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:28.203+0000 7f1e58a00640 1 --2- 172.21.15.86:0/3375392363 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1e3406f480 0x7f1e34071990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1e440070b0 tx=0x7f1e44007040 comp rx=0 tx=0).stop 2024-04-25T14:16:28.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:28.203+0000 7f1e58a00640 1 -- 172.21.15.86:0/3375392363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e540b3950 msgr2=0x7f1e540d2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:28.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:28.203+0000 7f1e58a00640 1 --2- 172.21.15.86:0/3375392363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e540b3950 0x7f1e540d2210 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f1e4000b440 tx=0x7f1e4000b900 comp rx=0 tx=0).stop 2024-04-25T14:16:28.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:28.203+0000 7f1e58a00640 1 -- 172.21.15.86:0/3375392363 shutdown_connections 2024-04-25T14:16:28.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:28.203+0000 7f1e58a00640 1 --2- 172.21.15.86:0/3375392363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e541607a0 0x7f1e54156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:28.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:28.203+0000 7f1e58a00640 1 --2- 172.21.15.86:0/3375392363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e540b3950 0x7f1e540d2210 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:28.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:28.203+0000 7f1e58a00640 1 --2- 172.21.15.86:0/3375392363 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1e3406f480 0x7f1e34071990 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:28.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:28.203+0000 7f1e58a00640 1 --2- 172.21.15.86:0/3375392363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e540af8b0 0x7f1e541f9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:28.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:28.203+0000 7f1e58a00640 1 -- 172.21.15.86:0/3375392363 >> 172.21.15.86:0/3375392363 conn(0x7f1e5409ba30 msgr2=0x7f1e5415ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:28.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:28.203+0000 7f1e58a00640 1 -- 172.21.15.86:0/3375392363 shutdown_connections 2024-04-25T14:16:28.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:28.203+0000 7f1e58a00640 1 -- 172.21.15.86:0/3375392363 wait complete. 2024-04-25T14:16:28.623 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:28.623 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:29.624 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.787+0000 7fdff7200640 1 Processor -- start 2024-04-25T14:16:29.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.787+0000 7fdff7200640 1 -- start start 2024-04-25T14:16:29.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.787+0000 7fdff7200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdff80722a0 0x7fdff80726b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:29.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.787+0000 7fdff7200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdff81614c0 0x7fdff81618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:29.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.787+0000 7fdff7200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdff8072bf0 con 0x7fdff80b6ee0 2024-04-25T14:16:29.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.787+0000 7fdff7200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fdff815df00 con 0x7fdff81614c0 2024-04-25T14:16:29.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.787+0000 7fdff7200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdff815e1e0 con 0x7fdff80722a0 2024-04-25T14:16:29.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.787+0000 7fdff5e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fdff80b6ee0 0x7fdff80d06c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42462/0 (socket says 172.21.15.86:42462) 2024-04-25T14:16:29.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.787+0000 7fdff5e00640 1 -- 172.21.15.86:0/1235493432 learned_addr learned my addr 172.21.15.86:0/1235493432 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:16:29.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.787+0000 7fdff6800640 1 --2- 172.21.15.86:0/1235493432 >> v2:172.21.15.155:3300/0 conn(0x7fdff81614c0 0x7fdff81618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:29.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.787+0000 7fdff5400640 1 --2- 172.21.15.86:0/1235493432 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdff80722a0 0x7fdff80726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:29.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.787+0000 7fdff4a00640 1 -- 172.21.15.86:0/1235493432 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3941620661 0 0) 0x7fdfe8001120 con 0x7fdff80b6ee0 2024-04-25T14:16:29.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.787+0000 7fdff4a00640 1 -- 172.21.15.86:0/1235493432 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdfdc003530 con 0x7fdff80b6ee0 2024-04-25T14:16:29.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.787+0000 7fdff6800640 1 -- 172.21.15.86:0/1235493432 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdff80722a0 msgr2=0x7fdff80726b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:29.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.787+0000 7fdff6800640 1 --2- 172.21.15.86:0/1235493432 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdff80722a0 0x7fdff80726b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:29.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.787+0000 7fdff6800640 1 -- 172.21.15.86:0/1235493432 >> v1:172.21.15.86:6789/0 conn(0x7fdff80b6ee0 legacy=0x7fdff80d06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:29.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.787+0000 7fdff6800640 1 -- 172.21.15.86:0/1235493432 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff815e4c0 con 0x7fdff81614c0 2024-04-25T14:16:29.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.791+0000 7fdff6800640 1 --2- 172.21.15.86:0/1235493432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdff81614c0 0x7fdff81618d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fdff0009d90 tx=0x7fdff001c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97898abe2fc3fd8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:29.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.791+0000 7fdff4a00640 1 -- 172.21.15.86:0/1235493432 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff00b2070 con 0x7fdff81614c0 2024-04-25T14:16:29.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.791+0000 7fdff4a00640 1 -- 172.21.15.86:0/1235493432 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdff00a57a0 con 0x7fdff81614c0 2024-04-25T14:16:29.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.791+0000 7fdff4a00640 1 -- 172.21.15.86:0/1235493432 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff00a5b50 con 0x7fdff81614c0 2024-04-25T14:16:29.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.791+0000 7fdff7200640 1 -- 172.21.15.86:0/1235493432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdff81614c0 msgr2=0x7fdff81618d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:29.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.791+0000 7fdff7200640 1 --2- 172.21.15.86:0/1235493432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdff81614c0 0x7fdff81618d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fdff0009d90 tx=0x7fdff001c8e0 comp rx=0 tx=0).stop 2024-04-25T14:16:29.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.791+0000 7fdff7200640 1 -- 172.21.15.86:0/1235493432 shutdown_connections 2024-04-25T14:16:29.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.791+0000 7fdff7200640 1 --2- 172.21.15.86:0/1235493432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdff81614c0 0x7fdff81618d0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:29.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.791+0000 7fdff7200640 1 --2- 172.21.15.86:0/1235493432 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdff80722a0 0x7fdff80726b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:29.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.791+0000 7fdff7200640 1 -- 172.21.15.86:0/1235493432 >> 172.21.15.86:0/1235493432 conn(0x7fdff80b4f80 msgr2=0x7fdff80b5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:29.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.791+0000 7fdff7200640 1 -- 172.21.15.86:0/1235493432 shutdown_connections 2024-04-25T14:16:29.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.795+0000 7fdff7200640 1 -- 172.21.15.86:0/1235493432 wait complete. 2024-04-25T14:16:29.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.795+0000 7fdff7200640 1 Processor -- start 2024-04-25T14:16:29.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.795+0000 7fdff7200640 1 -- start start 2024-04-25T14:16:29.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.795+0000 7fdff7200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdff80722a0 0x7fdff80cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:29.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.795+0000 7fdff7200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdff80b6ee0 0x7fdff814ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:29.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.795+0000 7fdff7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdff81614c0 0x7fdff8070d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:29.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.795+0000 7fdff7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdff80cfd40 con 0x7fdff81614c0 2024-04-25T14:16:29.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.795+0000 7fdff7200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdff8162220 con 0x7fdff80722a0 2024-04-25T14:16:29.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.795+0000 7fdff7200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdff80cfec0 con 0x7fdff80b6ee0 2024-04-25T14:16:29.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.799+0000 7fdff6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdff81614c0 0x7fdff8070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:29.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.799+0000 7fdff6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdff81614c0 0x7fdff8070d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40094/0 (socket says 172.21.15.86:40094) 2024-04-25T14:16:29.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.799+0000 7fdff6800640 1 -- 172.21.15.86:0/3892560254 learned_addr learned my addr 172.21.15.86:0/3892560254 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:29.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.799+0000 7fdff5400640 1 --2- 172.21.15.86:0/3892560254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdff80b6ee0 0x7fdff814ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:29.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.799+0000 7fdff5e00640 1 --2- 172.21.15.86:0/3892560254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdff80722a0 0x7fdff80cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:29.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.799+0000 7fdff6800640 1 -- 172.21.15.86:0/3892560254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdff80b6ee0 msgr2=0x7fdff814ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:29.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.799+0000 7fdff6800640 1 --2- 172.21.15.86:0/3892560254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdff80b6ee0 0x7fdff814ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:29.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.799+0000 7fdff6800640 1 -- 172.21.15.86:0/3892560254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdff80722a0 msgr2=0x7fdff80cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:29.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.799+0000 7fdff6800640 1 --2- 172.21.15.86:0/3892560254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdff80722a0 0x7fdff80cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:29.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.799+0000 7fdff6800640 1 -- 172.21.15.86:0/3892560254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff80b7460 con 0x7fdff81614c0 2024-04-25T14:16:29.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.799+0000 7fdff6800640 1 --2- 172.21.15.86:0/3892560254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdff81614c0 0x7fdff8070d60 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7fdff0009d90 tx=0x7fdff0019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:29.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.799+0000 7fdfee000640 1 -- 172.21.15.86:0/3892560254 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff00b2040 con 0x7fdff81614c0 2024-04-25T14:16:29.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.799+0000 7fdfee000640 1 -- 172.21.15.86:0/3892560254 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdff00bf070 con 0x7fdff81614c0 2024-04-25T14:16:29.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.799+0000 7fdfee000640 1 -- 172.21.15.86:0/3892560254 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff0009450 con 0x7fdff81614c0 2024-04-25T14:16:29.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.799+0000 7fdff7200640 1 -- 172.21.15.86:0/3892560254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff814da00 con 0x7fdff81614c0 2024-04-25T14:16:29.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.799+0000 7fdff7200640 1 -- 172.21.15.86:0/3892560254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff814b100 con 0x7fdff81614c0 2024-04-25T14:16:29.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.799+0000 7fdff7200640 1 -- 172.21.15.86:0/3892560254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdff8076b80 con 0x7fdff81614c0 2024-04-25T14:16:29.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.799+0000 7fdfee000640 1 -- 172.21.15.86:0/3892560254 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7fdff00b86b0 con 0x7fdff81614c0 2024-04-25T14:16:29.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.803+0000 7fdfee000640 1 --2- 172.21.15.86:0/3892560254 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fdfd806f480 0x7fdfd8071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:29.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.803+0000 7fdfee000640 1 -- 172.21.15.86:0/3892560254 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fdff0129de0 con 0x7fdff81614c0 2024-04-25T14:16:29.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.803+0000 7fdff5e00640 1 --2- 172.21.15.86:0/3892560254 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fdfd806f480 0x7fdfd8071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:29.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.803+0000 7fdff5e00640 1 --2- 172.21.15.86:0/3892560254 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fdfd806f480 0x7fdfd8071990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8005890 tx=0x7fdfe8009040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:29.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.807+0000 7fdfee000640 1 -- 172.21.15.86:0/3892560254 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdff00c8020 con 0x7fdff81614c0 2024-04-25T14:16:29.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.995+0000 7fdff7200640 1 -- 172.21.15.86:0/3892560254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdff8160d00 con 0x7fdff81614c0 2024-04-25T14:16:29.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.999+0000 7fdfee000640 1 -- 172.21.15.86:0/3892560254 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7fdff0016560 con 0x7fdff81614c0 2024-04-25T14:16:29.998 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:29.998 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:29.999 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:16:29.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.999+0000 7fdff7200640 1 -- 172.21.15.86:0/3892560254 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fdfd806f480 msgr2=0x7fdfd8071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:29.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.999+0000 7fdff7200640 1 --2- 172.21.15.86:0/3892560254 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fdfd806f480 0x7fdfd8071990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8005890 tx=0x7fdfe8009040 comp rx=0 tx=0).stop 2024-04-25T14:16:30.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.999+0000 7fdff7200640 1 -- 172.21.15.86:0/3892560254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdff81614c0 msgr2=0x7fdff8070d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:30.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.999+0000 7fdff7200640 1 --2- 172.21.15.86:0/3892560254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdff81614c0 0x7fdff8070d60 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7fdff0009d90 tx=0x7fdff0019680 comp rx=0 tx=0).stop 2024-04-25T14:16:30.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.999+0000 7fdff7200640 1 -- 172.21.15.86:0/3892560254 shutdown_connections 2024-04-25T14:16:30.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.999+0000 7fdff7200640 1 --2- 172.21.15.86:0/3892560254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdff81614c0 0x7fdff8070d60 unknown :-1 s=CLOSED pgs=2290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:30.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.999+0000 7fdff7200640 1 --2- 172.21.15.86:0/3892560254 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fdfd806f480 0x7fdfd8071990 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:30.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.999+0000 7fdff7200640 1 --2- 172.21.15.86:0/3892560254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdff80b6ee0 0x7fdff814ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:30.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.999+0000 7fdff7200640 1 --2- 172.21.15.86:0/3892560254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdff80722a0 0x7fdff80cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:30.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.999+0000 7fdff7200640 1 -- 172.21.15.86:0/3892560254 >> 172.21.15.86:0/3892560254 conn(0x7fdff80b4f80 msgr2=0x7fdff815ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:30.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.999+0000 7fdff7200640 1 -- 172.21.15.86:0/3892560254 shutdown_connections 2024-04-25T14:16:30.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:29.999+0000 7fdff7200640 1 -- 172.21.15.86:0/3892560254 wait complete. 2024-04-25T14:16:30.421 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:30.421 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:31.422 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:31.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.583+0000 7f7350600640 1 Processor -- start 2024-04-25T14:16:31.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.583+0000 7f7350600640 1 -- start start 2024-04-25T14:16:31.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.583+0000 7f7350600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73480cfca0 0x7f734806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:31.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.583+0000 7f7350600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f73480d16e0 0x7f73480d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:31.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.583+0000 7f7350600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f734806c670 0x7f734806ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.583+0000 7f7350600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7348070470 con 0x7f73480d16e0 2024-04-25T14:16:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.583+0000 7f7350600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f73480d2030 con 0x7f734806c670 2024-04-25T14:16:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.583+0000 7f7350600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7348072bd0 con 0x7f73480cfca0 2024-04-25T14:16:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.583+0000 7f734d200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f73480d16e0 0x7f73480d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.583+0000 7f734d200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f73480d16e0 0x7f73480d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40098/0 (socket says 172.21.15.86:40098) 2024-04-25T14:16:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.583+0000 7f734d200640 1 -- 172.21.15.86:0/2228564550 learned_addr learned my addr 172.21.15.86:0/2228564550 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.587+0000 7f734e600640 1 --2- 172.21.15.86:0/2228564550 >> v2:172.21.15.155:3300/0 conn(0x7f734806c670 0x7f734806ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.587+0000 7f734dc00640 1 --2- 172.21.15.86:0/2228564550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73480cfca0 0x7f734806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.587+0000 7f734d200640 1 -- 172.21.15.86:0/2228564550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73480cfca0 msgr2=0x7f734806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.587+0000 7f734d200640 1 --2- 172.21.15.86:0/2228564550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73480cfca0 0x7f734806f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.587+0000 7f734d200640 1 -- 172.21.15.86:0/2228564550 >> v2:172.21.15.155:3300/0 conn(0x7f734806c670 msgr2=0x7f734806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.587+0000 7f734d200640 1 --2- 172.21.15.86:0/2228564550 >> v2:172.21.15.155:3300/0 conn(0x7f734806c670 0x7f734806ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.587+0000 7f734d200640 1 -- 172.21.15.86:0/2228564550 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734806f930 con 0x7f73480d16e0 2024-04-25T14:16:31.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.587+0000 7f734e600640 1 --2- 172.21.15.86:0/2228564550 >> v2:172.21.15.155:3300/0 conn(0x7f734806c670 0x7f734806ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:16:31.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.587+0000 7f734d200640 1 --2- 172.21.15.86:0/2228564550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73480d16e0 0x7f73480d1af0 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f734806e060 tx=0x7f733401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8750773b4a78fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:31.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.587+0000 7f7347e00640 1 -- 172.21.15.86:0/2228564550 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73340ab5f0 con 0x7f73480d16e0 2024-04-25T14:16:31.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.587+0000 7f7347e00640 1 -- 172.21.15.86:0/2228564550 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73340b0070 con 0x7f73480d16e0 2024-04-25T14:16:31.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.587+0000 7f7347e00640 1 -- 172.21.15.86:0/2228564550 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73340ab9b0 con 0x7f73480d16e0 2024-04-25T14:16:31.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.591+0000 7f7350600640 1 -- 172.21.15.86:0/2228564550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73480d16e0 msgr2=0x7f73480d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:31.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.591+0000 7f7350600640 1 --2- 172.21.15.86:0/2228564550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73480d16e0 0x7f73480d1af0 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f734806e060 tx=0x7f733401c520 comp rx=0 tx=0).stop 2024-04-25T14:16:31.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.591+0000 7f7350600640 1 -- 172.21.15.86:0/2228564550 shutdown_connections 2024-04-25T14:16:31.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.591+0000 7f7350600640 1 --2- 172.21.15.86:0/2228564550 >> v2:172.21.15.155:3300/0 conn(0x7f734806c670 0x7f734806ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.591+0000 7f7350600640 1 --2- 172.21.15.86:0/2228564550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73480d16e0 0x7f73480d1af0 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.591+0000 7f7350600640 1 --2- 172.21.15.86:0/2228564550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73480cfca0 0x7f734806f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.591+0000 7f7350600640 1 -- 172.21.15.86:0/2228564550 >> 172.21.15.86:0/2228564550 conn(0x7f73480b4650 msgr2=0x7f73480b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:31.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.591+0000 7f7350600640 1 -- 172.21.15.86:0/2228564550 shutdown_connections 2024-04-25T14:16:31.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.591+0000 7f7350600640 1 -- 172.21.15.86:0/2228564550 wait complete. 2024-04-25T14:16:31.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.591+0000 7f7350600640 1 Processor -- start 2024-04-25T14:16:31.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.591+0000 7f7350600640 1 -- start start 2024-04-25T14:16:31.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.595+0000 7f7350600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f734806c670 0x7f73480b2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:31.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.595+0000 7f7350600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73480cfca0 0x7f73481fce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:31.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.595+0000 7f7350600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73480d16e0 0x7f734809bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:31.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.595+0000 7f7350600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73480738c0 con 0x7f73480cfca0 2024-04-25T14:16:31.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.595+0000 7f7350600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f734806d3d0 con 0x7f734806c670 2024-04-25T14:16:31.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.595+0000 7f7350600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7348073a40 con 0x7f73480d16e0 2024-04-25T14:16:31.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.595+0000 7f734d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73480cfca0 0x7f73481fce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:31.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.595+0000 7f734d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73480cfca0 0x7f73481fce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40114/0 (socket says 172.21.15.86:40114) 2024-04-25T14:16:31.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.595+0000 7f734d200640 1 -- 172.21.15.86:0/2843200910 learned_addr learned my addr 172.21.15.86:0/2843200910 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:31.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.595+0000 7f734e600640 1 --2- 172.21.15.86:0/2843200910 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73480d16e0 0x7f734809bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:31.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.595+0000 7f734dc00640 1 --2- 172.21.15.86:0/2843200910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f734806c670 0x7f73480b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:31.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.595+0000 7f734d200640 1 -- 172.21.15.86:0/2843200910 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73480d16e0 msgr2=0x7f734809bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:31.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.595+0000 7f734d200640 1 --2- 172.21.15.86:0/2843200910 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73480d16e0 0x7f734809bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.595+0000 7f734d200640 1 -- 172.21.15.86:0/2843200910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f734806c670 msgr2=0x7f73480b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:31.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.595+0000 7f734d200640 1 --2- 172.21.15.86:0/2843200910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f734806c670 0x7f73480b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.595+0000 7f734d200640 1 -- 172.21.15.86:0/2843200910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73481fd3d0 con 0x7f73480cfca0 2024-04-25T14:16:31.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.595+0000 7f734dc00640 1 --2- 172.21.15.86:0/2843200910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f734806c670 0x7f73480b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:16:31.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.595+0000 7f734d200640 1 --2- 172.21.15.86:0/2843200910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73480cfca0 0x7f73481fce90 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f7334097ec0 tx=0x7f733401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:31.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.595+0000 7f7345600640 1 -- 172.21.15.86:0/2843200910 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73340be5f0 con 0x7f73480cfca0 2024-04-25T14:16:31.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.595+0000 7f7345600640 1 -- 172.21.15.86:0/2843200910 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73340be790 con 0x7f73480cfca0 2024-04-25T14:16:31.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.595+0000 7f7345600640 1 -- 172.21.15.86:0/2843200910 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73340beb80 con 0x7f73480cfca0 2024-04-25T14:16:31.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.595+0000 7f7350600640 1 -- 172.21.15.86:0/2843200910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73480cc780 con 0x7f73480cfca0 2024-04-25T14:16:31.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.595+0000 7f7350600640 1 -- 172.21.15.86:0/2843200910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73480ccd40 con 0x7f73480cfca0 2024-04-25T14:16:31.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.595+0000 7f7350600640 1 -- 172.21.15.86:0/2843200910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f734809f4b0 con 0x7f73480cfca0 2024-04-25T14:16:31.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.595+0000 7f7345600640 1 -- 172.21.15.86:0/2843200910 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f73340b43f0 con 0x7f73480cfca0 2024-04-25T14:16:31.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.599+0000 7f7345600640 1 --2- 172.21.15.86:0/2843200910 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f732806f480 0x7f7328071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:31.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.599+0000 7f7345600640 1 -- 172.21.15.86:0/2843200910 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f7334129ed0 con 0x7f73480cfca0 2024-04-25T14:16:31.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.599+0000 7f734dc00640 1 --2- 172.21.15.86:0/2843200910 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f732806f480 0x7f7328071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:31.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.599+0000 7f734dc00640 1 --2- 172.21.15.86:0/2843200910 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f732806f480 0x7f7328071990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7348070710 tx=0x7f733c008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:31.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.603+0000 7f7345600640 1 -- 172.21.15.86:0/2843200910 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f73340c4060 con 0x7f73480cfca0 2024-04-25T14:16:31.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.795+0000 7f7350600640 1 -- 172.21.15.86:0/2843200910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7348069d00 con 0x7f73480cfca0 2024-04-25T14:16:31.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.795+0000 7f7345600640 1 -- 172.21.15.86:0/2843200910 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f7334016110 con 0x7f73480cfca0 2024-04-25T14:16:31.794 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:31.794 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:31.794 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:16:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.799+0000 7f7350600640 1 -- 172.21.15.86:0/2843200910 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f732806f480 msgr2=0x7f7328071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.799+0000 7f7350600640 1 --2- 172.21.15.86:0/2843200910 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f732806f480 0x7f7328071990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7348070710 tx=0x7f733c008040 comp rx=0 tx=0).stop 2024-04-25T14:16:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.799+0000 7f7350600640 1 -- 172.21.15.86:0/2843200910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73480cfca0 msgr2=0x7f73481fce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.799+0000 7f7350600640 1 --2- 172.21.15.86:0/2843200910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73480cfca0 0x7f73481fce90 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f7334097ec0 tx=0x7f733401ad60 comp rx=0 tx=0).stop 2024-04-25T14:16:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.799+0000 7f7350600640 1 -- 172.21.15.86:0/2843200910 shutdown_connections 2024-04-25T14:16:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.799+0000 7f7350600640 1 --2- 172.21.15.86:0/2843200910 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f732806f480 0x7f7328071990 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.799+0000 7f7350600640 1 --2- 172.21.15.86:0/2843200910 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73480d16e0 0x7f734809bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.799+0000 7f7350600640 1 --2- 172.21.15.86:0/2843200910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73480cfca0 0x7f73481fce90 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.799+0000 7f7350600640 1 --2- 172.21.15.86:0/2843200910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f734806c670 0x7f73480b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:31.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.799+0000 7f7350600640 1 -- 172.21.15.86:0/2843200910 >> 172.21.15.86:0/2843200910 conn(0x7f73480b4650 msgr2=0x7f734815be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:31.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.799+0000 7f7350600640 1 -- 172.21.15.86:0/2843200910 shutdown_connections 2024-04-25T14:16:31.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:31.799+0000 7f7350600640 1 -- 172.21.15.86:0/2843200910 wait complete. 2024-04-25T14:16:32.218 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:32.218 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:33.219 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:33.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.379+0000 7f89e3200640 1 Processor -- start 2024-04-25T14:16:33.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.379+0000 7f89e3200640 1 -- start start 2024-04-25T14:16:33.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89e3200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89dc0cfca0 0x7f89dc06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:33.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89e3200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89dc072bd0 con 0x7f89dc0730b0 2024-04-25T14:16:33.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89e3200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89dc06f930 con 0x7f89dc0d16e0 2024-04-25T14:16:33.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89e3200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f89dc070470 con 0x7f89dc0cfca0 2024-04-25T14:16:33.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89e1200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f89dc0730b0 0x7f89dc0734c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42466/0 (socket says 172.21.15.86:42466) 2024-04-25T14:16:33.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89e1200640 1 -- 172.21.15.86:0/3463684268 learned_addr learned my addr 172.21.15.86:0/3463684268 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:16:33.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89dbe00640 1 --2- 172.21.15.86:0/3463684268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89dc0cfca0 0x7f89dc06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:33.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89daa00640 1 -- 172.21.15.86:0/3463684268 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2953930489 0 0) 0x7f89dc072bd0 con 0x7f89dc0730b0 2024-04-25T14:16:33.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89daa00640 1 -- 172.21.15.86:0/3463684268 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89c0003530 con 0x7f89dc0730b0 2024-04-25T14:16:33.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89daa00640 1 -- 172.21.15.86:0/3463684268 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3848419290 0 0) 0x7f89c0003530 con 0x7f89dc0730b0 2024-04-25T14:16:33.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89daa00640 1 -- 172.21.15.86:0/3463684268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89dc0cfca0 msgr2=0x7f89dc06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:33.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89daa00640 1 --2- 172.21.15.86:0/3463684268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89dc0cfca0 0x7f89dc06f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89daa00640 1 -- 172.21.15.86:0/3463684268 >> v1:172.21.15.155:6789/0 conn(0x7f89dc0d16e0 legacy=0x7f89dc0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:33.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89daa00640 1 -- 172.21.15.86:0/3463684268 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89dc06fbd0 con 0x7f89dc0730b0 2024-04-25T14:16:33.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89daa00640 1 -- 172.21.15.86:0/3463684268 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f89d4001a00 con 0x7f89dc0730b0 2024-04-25T14:16:33.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89daa00640 1 -- 172.21.15.86:0/3463684268 >> v1:172.21.15.86:6789/0 conn(0x7f89dc0730b0 legacy=0x7f89dc0734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:33.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89daa00640 1 --2- 172.21.15.86:0/3463684268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89c0003a60 0x7f89c0003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:33.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89daa00640 1 --2- 172.21.15.86:0/3463684268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89c0004810 0x7f89c0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:33.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89daa00640 1 --2- 172.21.15.86:0/3463684268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89c00051b0 0x7f89c000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:33.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89daa00640 1 -- 172.21.15.86:0/3463684268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f89dc070470 con 0x7f89c0003a60 2024-04-25T14:16:33.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89daa00640 1 -- 172.21.15.86:0/3463684268 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f89c000c0a0 con 0x7f89c0004810 2024-04-25T14:16:33.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89daa00640 1 -- 172.21.15.86:0/3463684268 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f89c000e4b0 con 0x7f89c00051b0 2024-04-25T14:16:33.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89dbe00640 1 --2- 172.21.15.86:0/3463684268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89dc0cfca0 0x7f89dc06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:16:33.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89db400640 1 --2- 172.21.15.86:0/3463684268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89c0004810 0x7f89c0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:33.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89dbe00640 1 --2- 172.21.15.86:0/3463684268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89c0003a60 0x7f89c0003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:33.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89e1200640 1 --2- 172.21.15.86:0/3463684268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89c00051b0 0x7f89c000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:33.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89dbe00640 1 -- 172.21.15.86:0/3463684268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89c00051b0 msgr2=0x7f89c000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:33.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89dbe00640 1 --2- 172.21.15.86:0/3463684268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89c00051b0 0x7f89c000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89dbe00640 1 -- 172.21.15.86:0/3463684268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89c0004810 msgr2=0x7f89c0004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:33.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89dbe00640 1 --2- 172.21.15.86:0/3463684268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89c0004810 0x7f89c0004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89dbe00640 1 -- 172.21.15.86:0/3463684268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c0003530 con 0x7f89c0003a60 2024-04-25T14:16:33.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.383+0000 7f89db400640 1 --2- 172.21.15.86:0/3463684268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89c0004810 0x7f89c0004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:16:33.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.387+0000 7f89dbe00640 1 --2- 172.21.15.86:0/3463684268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89c0003a60 0x7f89c0003e90 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0d2320 tx=0x7f89cc020bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677904d35675105f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:33.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.387+0000 7f89daa00640 1 -- 172.21.15.86:0/3463684268 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89cc009810 con 0x7f89c0003a60 2024-04-25T14:16:33.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.387+0000 7f89daa00640 1 -- 172.21.15.86:0/3463684268 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89cc004230 con 0x7f89c0003a60 2024-04-25T14:16:33.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.387+0000 7f89daa00640 1 -- 172.21.15.86:0/3463684268 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89cc0a9750 con 0x7f89c0003a60 2024-04-25T14:16:33.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.387+0000 7f89e3200640 1 -- 172.21.15.86:0/3463684268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89c0003a60 msgr2=0x7f89c0003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:33.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.387+0000 7f89e3200640 1 --2- 172.21.15.86:0/3463684268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89c0003a60 0x7f89c0003e90 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0d2320 tx=0x7f89cc020bd0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.387+0000 7f89e1200640 1 -- 172.21.15.86:0/3463684268 reap_dead start 2024-04-25T14:16:33.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.387+0000 7f89e3200640 1 -- 172.21.15.86:0/3463684268 shutdown_connections 2024-04-25T14:16:33.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.387+0000 7f89e3200640 1 -- 172.21.15.86:0/3463684268 >> 172.21.15.86:0/3463684268 conn(0x7f89dc0b4650 msgr2=0x7f89dc0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:33.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.387+0000 7f89e3200640 1 -- 172.21.15.86:0/3463684268 shutdown_connections 2024-04-25T14:16:33.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.387+0000 7f89e3200640 1 -- 172.21.15.86:0/3463684268 wait complete. 2024-04-25T14:16:33.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.391+0000 7f89e3200640 1 Processor -- start 2024-04-25T14:16:33.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.391+0000 7f89e3200640 1 -- start start 2024-04-25T14:16:33.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.391+0000 7f89e3200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89c0003a60 0x7f89dc1f1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:33.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.391+0000 7f89e3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89dc09bca0 0x7f89dc09c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:33.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.391+0000 7f89e3200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89dc0bf170 0x7f89dc0cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:33.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.391+0000 7f89e3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f89dc06f3f0 con 0x7f89dc09bca0 2024-04-25T14:16:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.391+0000 7f89e3200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f89dc070190 con 0x7f89dc0bf170 2024-04-25T14:16:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.391+0000 7f89e3200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f89dc06f570 con 0x7f89c0003a60 2024-04-25T14:16:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.391+0000 7f89db400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89dc09bca0 0x7f89dc09c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.391+0000 7f89db400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89dc09bca0 0x7f89dc09c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40132/0 (socket says 172.21.15.86:40132) 2024-04-25T14:16:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.391+0000 7f89db400640 1 -- 172.21.15.86:0/508403775 learned_addr learned my addr 172.21.15.86:0/508403775 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.391+0000 7f89dbe00640 1 --2- 172.21.15.86:0/508403775 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89c0003a60 0x7f89dc1f1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.391+0000 7f89e1200640 1 --2- 172.21.15.86:0/508403775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89dc0bf170 0x7f89dc0cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.391+0000 7f89db400640 1 -- 172.21.15.86:0/508403775 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89c0003a60 msgr2=0x7f89dc1f1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.391+0000 7f89db400640 1 --2- 172.21.15.86:0/508403775 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89c0003a60 0x7f89dc1f1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.391+0000 7f89db400640 1 -- 172.21.15.86:0/508403775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89dc0bf170 msgr2=0x7f89dc0cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.391+0000 7f89db400640 1 --2- 172.21.15.86:0/508403775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89dc0bf170 0x7f89dc0cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.391+0000 7f89db400640 1 -- 172.21.15.86:0/508403775 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89dc0cccc0 con 0x7f89dc09bca0 2024-04-25T14:16:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.391+0000 7f89db400640 1 --2- 172.21.15.86:0/508403775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89dc09bca0 0x7f89dc09c630 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f89c8012860 tx=0x7f89c8012d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.395+0000 7f89d3e00640 1 -- 172.21.15.86:0/508403775 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8010bd0 con 0x7f89dc09bca0 2024-04-25T14:16:33.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.395+0000 7f89d3e00640 1 -- 172.21.15.86:0/508403775 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89c801b070 con 0x7f89dc09bca0 2024-04-25T14:16:33.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.395+0000 7f89d3e00640 1 -- 172.21.15.86:0/508403775 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8016480 con 0x7f89dc09bca0 2024-04-25T14:16:33.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.395+0000 7f89e3200640 1 -- 172.21.15.86:0/508403775 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89dc0ccfa0 con 0x7f89dc09bca0 2024-04-25T14:16:33.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.395+0000 7f89e3200640 1 -- 172.21.15.86:0/508403775 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89dc0cd560 con 0x7f89dc09bca0 2024-04-25T14:16:33.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.395+0000 7f89e3200640 1 -- 172.21.15.86:0/508403775 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89dc09f4b0 con 0x7f89dc09bca0 2024-04-25T14:16:33.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.395+0000 7f89d3e00640 1 -- 172.21.15.86:0/508403775 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f89c8016620 con 0x7f89dc09bca0 2024-04-25T14:16:33.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.395+0000 7f89d3e00640 1 --2- 172.21.15.86:0/508403775 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f89bc06f450 0x7f89bc071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:33.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.395+0000 7f89d3e00640 1 -- 172.21.15.86:0/508403775 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f89c8092f50 con 0x7f89dc09bca0 2024-04-25T14:16:33.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.395+0000 7f89dbe00640 1 --2- 172.21.15.86:0/508403775 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f89bc06f450 0x7f89bc071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:33.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.399+0000 7f89dbe00640 1 --2- 172.21.15.86:0/508403775 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f89bc06f450 0x7f89bc071960 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0d2320 tx=0x7f89cc0046d0 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:33.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.399+0000 7f89d3e00640 1 -- 172.21.15.86:0/508403775 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f89c805fac0 con 0x7f89dc09bca0 2024-04-25T14:16:33.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.587+0000 7f89e3200640 1 -- 172.21.15.86:0/508403775 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89dc069d00 con 0x7f89dc09bca0 2024-04-25T14:16:33.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.587+0000 7f89d3e00640 1 -- 172.21.15.86:0/508403775 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f89c805f3c0 con 0x7f89dc09bca0 2024-04-25T14:16:33.588 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:33.588 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:33.589 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:16:33.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.591+0000 7f89e3200640 1 -- 172.21.15.86:0/508403775 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f89bc06f450 msgr2=0x7f89bc071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:33.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.591+0000 7f89e3200640 1 --2- 172.21.15.86:0/508403775 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f89bc06f450 0x7f89bc071960 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0d2320 tx=0x7f89cc0046d0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.591+0000 7f89e3200640 1 -- 172.21.15.86:0/508403775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89dc09bca0 msgr2=0x7f89dc09c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:33.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.591+0000 7f89e3200640 1 --2- 172.21.15.86:0/508403775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89dc09bca0 0x7f89dc09c630 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f89c8012860 tx=0x7f89c8012d20 comp rx=0 tx=0).stop 2024-04-25T14:16:33.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.591+0000 7f89e3200640 1 -- 172.21.15.86:0/508403775 shutdown_connections 2024-04-25T14:16:33.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.591+0000 7f89e3200640 1 --2- 172.21.15.86:0/508403775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89dc0bf170 0x7f89dc0cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.591+0000 7f89e3200640 1 --2- 172.21.15.86:0/508403775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89dc09bca0 0x7f89dc09c630 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.591+0000 7f89e3200640 1 --2- 172.21.15.86:0/508403775 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f89bc06f450 0x7f89bc071960 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.591+0000 7f89e3200640 1 --2- 172.21.15.86:0/508403775 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89c0003a60 0x7f89dc1f1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:33.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.591+0000 7f89e3200640 1 -- 172.21.15.86:0/508403775 >> 172.21.15.86:0/508403775 conn(0x7f89dc0b4650 msgr2=0x7f89dc0734c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:33.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.591+0000 7f89e3200640 1 -- 172.21.15.86:0/508403775 shutdown_connections 2024-04-25T14:16:33.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:33.591+0000 7f89e3200640 1 -- 172.21.15.86:0/508403775 wait complete. 2024-04-25T14:16:34.012 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:34.012 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:35.013 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:35.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.175+0000 7fc8ce000640 1 Processor -- start 2024-04-25T14:16:35.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.175+0000 7fc8ce000640 1 -- start start 2024-04-25T14:16:35.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.175+0000 7fc8ce000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8c80d1df0 0x7fc8c80cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:35.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.175+0000 7fc8ce000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc8c80cdcc0 0x7fc8c80ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:35.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.175+0000 7fc8ce000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8c815e5c0 con 0x7fc8c8070330 2024-04-25T14:16:35.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.175+0000 7fc8ce000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc8c806fd40 con 0x7fc8c80cdcc0 2024-04-25T14:16:35.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.175+0000 7fc8ce000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc8c815e8a0 con 0x7fc8c80d1df0 2024-04-25T14:16:35.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.175+0000 7fc8c7400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc8c8070330 0x7fc8c80d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42480/0 (socket says 172.21.15.86:42480) 2024-04-25T14:16:35.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.175+0000 7fc8c7400640 1 -- 172.21.15.86:0/44891356 learned_addr learned my addr 172.21.15.86:0/44891356 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:16:35.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.175+0000 7fc8c7e00640 1 --2- 172.21.15.86:0/44891356 >> v2:172.21.15.155:3300/0 conn(0x7fc8c80cdcc0 0x7fc8c80ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:35.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.175+0000 7fc8bfe00640 1 --2- 172.21.15.86:0/44891356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8c80d1df0 0x7fc8c80cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:35.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.175+0000 7fc8c6a00640 1 -- 172.21.15.86:0/44891356 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 682352939 0 0) 0x7fc8c815e5c0 con 0x7fc8c8070330 2024-04-25T14:16:35.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.175+0000 7fc8c6a00640 1 -- 172.21.15.86:0/44891356 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8ac003530 con 0x7fc8c8070330 2024-04-25T14:16:35.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.175+0000 7fc8c6a00640 1 -- 172.21.15.86:0/44891356 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2931285410 0 0) 0x7fc8ac003530 con 0x7fc8c8070330 2024-04-25T14:16:35.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.175+0000 7fc8c7e00640 1 -- 172.21.15.86:0/44891356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8c80d1df0 msgr2=0x7fc8c80cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:35.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.175+0000 7fc8c7e00640 1 --2- 172.21.15.86:0/44891356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8c80d1df0 0x7fc8c80cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:35.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.175+0000 7fc8c7e00640 1 -- 172.21.15.86:0/44891356 >> v1:172.21.15.86:6789/0 conn(0x7fc8c8070330 legacy=0x7fc8c80d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:35.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.175+0000 7fc8c7e00640 1 -- 172.21.15.86:0/44891356 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c815eb80 con 0x7fc8c80cdcc0 2024-04-25T14:16:35.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.175+0000 7fc8bfe00640 1 --2- 172.21.15.86:0/44891356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8c80d1df0 0x7fc8c80cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:16:35.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.179+0000 7fc8c7e00640 1 --2- 172.21.15.86:0/44891356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8c80cdcc0 0x7fc8c80ce0d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0009d90 tx=0x7fc8c001c7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66633f99623d5e7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:35.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.179+0000 7fc8c6a00640 1 -- 172.21.15.86:0/44891356 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c00b2070 con 0x7fc8c80cdcc0 2024-04-25T14:16:35.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.179+0000 7fc8c6a00640 1 -- 172.21.15.86:0/44891356 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8c00a55b0 con 0x7fc8c80cdcc0 2024-04-25T14:16:35.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.179+0000 7fc8c6a00640 1 -- 172.21.15.86:0/44891356 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8001680 con 0x7fc8c80cdcc0 2024-04-25T14:16:35.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.179+0000 7fc8ce000640 1 -- 172.21.15.86:0/44891356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8c80cdcc0 msgr2=0x7fc8c80ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:35.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.179+0000 7fc8ce000640 1 --2- 172.21.15.86:0/44891356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8c80cdcc0 0x7fc8c80ce0d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0009d90 tx=0x7fc8c001c7d0 comp rx=0 tx=0).stop 2024-04-25T14:16:35.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.183+0000 7fc8ce000640 1 -- 172.21.15.86:0/44891356 shutdown_connections 2024-04-25T14:16:35.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.183+0000 7fc8ce000640 1 --2- 172.21.15.86:0/44891356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8c80cdcc0 0x7fc8c80ce0d0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:35.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.183+0000 7fc8ce000640 1 --2- 172.21.15.86:0/44891356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8c80d1df0 0x7fc8c80cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:35.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.183+0000 7fc8ce000640 1 -- 172.21.15.86:0/44891356 >> 172.21.15.86:0/44891356 conn(0x7fc8c80b4650 msgr2=0x7fc8c80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:35.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.183+0000 7fc8ce000640 1 -- 172.21.15.86:0/44891356 shutdown_connections 2024-04-25T14:16:35.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.183+0000 7fc8ce000640 1 -- 172.21.15.86:0/44891356 wait complete. 2024-04-25T14:16:35.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.183+0000 7fc8ce000640 1 Processor -- start 2024-04-25T14:16:35.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.183+0000 7fc8ce000640 1 -- start start 2024-04-25T14:16:35.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.183+0000 7fc8ce000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c8070330 0x7fc8c80cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:35.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.183+0000 7fc8ce000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8c80cdcc0 0x7fc8c8168160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:35.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.183+0000 7fc8ce000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8c815ad70 0x7fc8c815bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:35.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.183+0000 7fc8ce000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c815f4b0 con 0x7fc8c8070330 2024-04-25T14:16:35.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.183+0000 7fc8ce000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c815f330 con 0x7fc8c815ad70 2024-04-25T14:16:35.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.187+0000 7fc8ce000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc8c8070740 con 0x7fc8c80cdcc0 2024-04-25T14:16:35.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.187+0000 7fc8c7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c8070330 0x7fc8c80cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:35.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.187+0000 7fc8c7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c8070330 0x7fc8c80cb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40146/0 (socket says 172.21.15.86:40146) 2024-04-25T14:16:35.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.187+0000 7fc8c7400640 1 -- 172.21.15.86:0/2009351980 learned_addr learned my addr 172.21.15.86:0/2009351980 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:35.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.187+0000 7fc8bfe00640 1 --2- 172.21.15.86:0/2009351980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8c80cdcc0 0x7fc8c8168160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:35.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.187+0000 7fc8c7400640 1 -- 172.21.15.86:0/2009351980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8c80cdcc0 msgr2=0x7fc8c8168160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:35.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.187+0000 7fc8c7400640 1 --2- 172.21.15.86:0/2009351980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8c80cdcc0 0x7fc8c8168160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:35.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.187+0000 7fc8c7400640 1 -- 172.21.15.86:0/2009351980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8c815ad70 msgr2=0x7fc8c815bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:35.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.187+0000 7fc8c7400640 1 --2- 172.21.15.86:0/2009351980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8c815ad70 0x7fc8c815bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:35.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.187+0000 7fc8c7400640 1 -- 172.21.15.86:0/2009351980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c80c83e0 con 0x7fc8c8070330 2024-04-25T14:16:35.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.187+0000 7fc8bfe00640 1 --2- 172.21.15.86:0/2009351980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8c80cdcc0 0x7fc8c8168160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:16:35.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.187+0000 7fc8c7400640 1 --2- 172.21.15.86:0/2009351980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c8070330 0x7fc8c80cb410 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7fc8b80017f0 tx=0x7fc8b800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:35.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.187+0000 7fc8bf400640 1 -- 172.21.15.86:0/2009351980 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8005ae0 con 0x7fc8c8070330 2024-04-25T14:16:35.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.187+0000 7fc8bf400640 1 -- 172.21.15.86:0/2009351980 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b8005c80 con 0x7fc8c8070330 2024-04-25T14:16:35.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.187+0000 7fc8ce000640 1 -- 172.21.15.86:0/2009351980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c80c8100 con 0x7fc8c8070330 2024-04-25T14:16:35.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.187+0000 7fc8bf400640 1 -- 172.21.15.86:0/2009351980 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8007290 con 0x7fc8c8070330 2024-04-25T14:16:35.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.187+0000 7fc8ce000640 1 -- 172.21.15.86:0/2009351980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c80d0530 con 0x7fc8c8070330 2024-04-25T14:16:35.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.187+0000 7fc8ce000640 1 -- 172.21.15.86:0/2009351980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8c8069d00 con 0x7fc8c8070330 2024-04-25T14:16:35.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.187+0000 7fc8bf400640 1 -- 172.21.15.86:0/2009351980 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7fc8b8007430 con 0x7fc8c8070330 2024-04-25T14:16:35.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.191+0000 7fc8bf400640 1 --2- 172.21.15.86:0/2009351980 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc8a406f480 0x7fc8a4071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:35.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.191+0000 7fc8bf400640 1 -- 172.21.15.86:0/2009351980 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fc8b808f2e0 con 0x7fc8c8070330 2024-04-25T14:16:35.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.191+0000 7fc8bfe00640 1 --2- 172.21.15.86:0/2009351980 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc8a406f480 0x7fc8a4071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:35.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.191+0000 7fc8bfe00640 1 --2- 172.21.15.86:0/2009351980 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc8a406f480 0x7fc8a4071990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc8c81fc4b0 tx=0x7fc8b0008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:35.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.195+0000 7fc8bf400640 1 -- 172.21.15.86:0/2009351980 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc8b8007720 con 0x7fc8c8070330 2024-04-25T14:16:35.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.387+0000 7fc8ce000640 1 -- 172.21.15.86:0/2009351980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8c80c73c0 con 0x7fc8c8070330 2024-04-25T14:16:35.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.387+0000 7fc8bf400640 1 -- 172.21.15.86:0/2009351980 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7fc8b805bf70 con 0x7fc8c8070330 2024-04-25T14:16:35.385 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:35.386 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:35.386 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:16:35.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.391+0000 7fc8ce000640 1 -- 172.21.15.86:0/2009351980 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc8a406f480 msgr2=0x7fc8a4071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:35.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.391+0000 7fc8ce000640 1 --2- 172.21.15.86:0/2009351980 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc8a406f480 0x7fc8a4071990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc8c81fc4b0 tx=0x7fc8b0008040 comp rx=0 tx=0).stop 2024-04-25T14:16:35.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.391+0000 7fc8ce000640 1 -- 172.21.15.86:0/2009351980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c8070330 msgr2=0x7fc8c80cb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:35.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.391+0000 7fc8ce000640 1 --2- 172.21.15.86:0/2009351980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c8070330 0x7fc8c80cb410 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7fc8b80017f0 tx=0x7fc8b800cbe0 comp rx=0 tx=0).stop 2024-04-25T14:16:35.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.391+0000 7fc8ce000640 1 -- 172.21.15.86:0/2009351980 shutdown_connections 2024-04-25T14:16:35.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.391+0000 7fc8ce000640 1 --2- 172.21.15.86:0/2009351980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8c815ad70 0x7fc8c815bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:35.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.391+0000 7fc8ce000640 1 --2- 172.21.15.86:0/2009351980 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc8a406f480 0x7fc8a4071990 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:35.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.391+0000 7fc8ce000640 1 --2- 172.21.15.86:0/2009351980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8c80cdcc0 0x7fc8c8168160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:35.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.391+0000 7fc8ce000640 1 --2- 172.21.15.86:0/2009351980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c8070330 0x7fc8c80cb410 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:35.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.391+0000 7fc8ce000640 1 -- 172.21.15.86:0/2009351980 >> 172.21.15.86:0/2009351980 conn(0x7fc8c80b4650 msgr2=0x7fc8c80afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:35.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.391+0000 7fc8ce000640 1 -- 172.21.15.86:0/2009351980 shutdown_connections 2024-04-25T14:16:35.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:35.391+0000 7fc8ce000640 1 -- 172.21.15.86:0/2009351980 wait complete. 2024-04-25T14:16:35.810 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:35.810 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:36.811 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:36.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.975+0000 7f000f800640 1 Processor -- start 2024-04-25T14:16:36.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.975+0000 7f000f800640 1 -- start start 2024-04-25T14:16:36.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.975+0000 7f000f800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f000806fd50 0x7f0008070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:36.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.975+0000 7f000f800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f000815feb0 0x7f00081602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:36.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.975+0000 7f000f800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00080b0a60 con 0x7f000815c2a0 2024-04-25T14:16:36.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.975+0000 7f000f800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f00080ae840 con 0x7f000815feb0 2024-04-25T14:16:36.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.975+0000 7f000f800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f000806ef90 con 0x7f000806fd50 2024-04-25T14:16:36.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.975+0000 7f000d800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f000815feb0 0x7f00081602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:36.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.975+0000 7f000d800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f000815feb0 0x7f00081602c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40160/0 (socket says 172.21.15.86:40160) 2024-04-25T14:16:36.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.975+0000 7f000d800640 1 -- 172.21.15.86:0/402805977 learned_addr learned my addr 172.21.15.86:0/402805977 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:36.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.975+0000 7effffe00640 1 --2- 172.21.15.86:0/402805977 >> v2:172.21.15.155:3300/0 conn(0x7f000806fd50 0x7f0008070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:36.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.975+0000 7f000d800640 1 -- 172.21.15.86:0/402805977 >> v2:172.21.15.155:3300/0 conn(0x7f000806fd50 msgr2=0x7f0008070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:36.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.975+0000 7f000d800640 1 --2- 172.21.15.86:0/402805977 >> v2:172.21.15.155:3300/0 conn(0x7f000806fd50 0x7f0008070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:36.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.975+0000 7f000d800640 1 -- 172.21.15.86:0/402805977 >> v1:172.21.15.155:6789/0 conn(0x7f000815c2a0 legacy=0x7f00080ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:36.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.975+0000 7f000d800640 1 -- 172.21.15.86:0/402805977 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000806f270 con 0x7f000815feb0 2024-04-25T14:16:36.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.975+0000 7effffe00640 1 --2- 172.21.15.86:0/402805977 >> v2:172.21.15.155:3300/0 conn(0x7f000806fd50 0x7f0008070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:16:36.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.979+0000 7f000d800640 1 --2- 172.21.15.86:0/402805977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000815feb0 0x7f00081602c0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f0008074f40 tx=0x7f000001d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208ce7898339c12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:36.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.979+0000 7f0007e00640 1 -- 172.21.15.86:0/402805977 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000000a2d0 con 0x7f000815feb0 2024-04-25T14:16:36.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.979+0000 7f0007e00640 1 -- 172.21.15.86:0/402805977 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00000a6830 con 0x7f000815feb0 2024-04-25T14:16:36.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.979+0000 7f0007e00640 1 -- 172.21.15.86:0/402805977 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00000a6c00 con 0x7f000815feb0 2024-04-25T14:16:36.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.979+0000 7f000f800640 1 -- 172.21.15.86:0/402805977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000815feb0 msgr2=0x7f00081602c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:36.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.979+0000 7f000f800640 1 --2- 172.21.15.86:0/402805977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000815feb0 0x7f00081602c0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f0008074f40 tx=0x7f000001d9f0 comp rx=0 tx=0).stop 2024-04-25T14:16:36.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.979+0000 7f000f800640 1 -- 172.21.15.86:0/402805977 shutdown_connections 2024-04-25T14:16:36.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.979+0000 7f000f800640 1 --2- 172.21.15.86:0/402805977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000815feb0 0x7f00081602c0 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:36.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.979+0000 7f000f800640 1 --2- 172.21.15.86:0/402805977 >> v2:172.21.15.155:3300/0 conn(0x7f000806fd50 0x7f0008070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:36.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.979+0000 7f000f800640 1 -- 172.21.15.86:0/402805977 >> 172.21.15.86:0/402805977 conn(0x7f00080b4650 msgr2=0x7f00080b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:36.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.979+0000 7f000f800640 1 -- 172.21.15.86:0/402805977 shutdown_connections 2024-04-25T14:16:36.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.979+0000 7f000f800640 1 -- 172.21.15.86:0/402805977 wait complete. 2024-04-25T14:16:36.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.983+0000 7f000f800640 1 Processor -- start 2024-04-25T14:16:36.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.983+0000 7f000f800640 1 -- start start 2024-04-25T14:16:36.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.983+0000 7f000f800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000806fd50 0x7f00080733f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:36.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.983+0000 7f000f800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f000815c2a0 0x7f00081557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:36.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.983+0000 7f000f800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f000815feb0 0x7f00080b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:36.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.983+0000 7f000f800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f000815c6b0 con 0x7f000806fd50 2024-04-25T14:16:36.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.983+0000 7f000f800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f00080d0fc0 con 0x7f000815c2a0 2024-04-25T14:16:36.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.983+0000 7f000f800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f000815c830 con 0x7f000815feb0 2024-04-25T14:16:36.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.983+0000 7f000ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000806fd50 0x7f00080733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:36.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.983+0000 7f000ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000806fd50 0x7f00080733f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40170/0 (socket says 172.21.15.86:40170) 2024-04-25T14:16:36.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.983+0000 7f000ce00640 1 -- 172.21.15.86:0/1645733074 learned_addr learned my addr 172.21.15.86:0/1645733074 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:36.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.983+0000 7effffe00640 1 --2- 172.21.15.86:0/1645733074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f000815c2a0 0x7f00081557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:36.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.983+0000 7f000d800640 1 --2- 172.21.15.86:0/1645733074 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f000815feb0 0x7f00080b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:36.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.983+0000 7f000ce00640 1 -- 172.21.15.86:0/1645733074 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f000815feb0 msgr2=0x7f00080b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:36.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.983+0000 7f000ce00640 1 --2- 172.21.15.86:0/1645733074 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f000815feb0 0x7f00080b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:36.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.983+0000 7f000ce00640 1 -- 172.21.15.86:0/1645733074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f000815c2a0 msgr2=0x7f00081557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:36.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.983+0000 7f000ce00640 1 --2- 172.21.15.86:0/1645733074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f000815c2a0 0x7f00081557b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:36.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.983+0000 7f000ce00640 1 -- 172.21.15.86:0/1645733074 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0008073cf0 con 0x7f000806fd50 2024-04-25T14:16:36.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.983+0000 7effffe00640 1 --2- 172.21.15.86:0/1645733074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f000815c2a0 0x7f00081557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:16:36.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.983+0000 7f000ce00640 1 --2- 172.21.15.86:0/1645733074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000806fd50 0x7f00080733f0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7efff800b3c0 tx=0x7efff800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:36.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.983+0000 7f0005600640 1 -- 172.21.15.86:0/1645733074 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8011830 con 0x7f000806fd50 2024-04-25T14:16:36.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.983+0000 7f0005600640 1 -- 172.21.15.86:0/1645733074 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efff8016070 con 0x7f000806fd50 2024-04-25T14:16:36.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.983+0000 7f000f800640 1 -- 172.21.15.86:0/1645733074 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0008073fd0 con 0x7f000806fd50 2024-04-25T14:16:36.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.983+0000 7f0005600640 1 -- 172.21.15.86:0/1645733074 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8011c30 con 0x7f000806fd50 2024-04-25T14:16:36.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.983+0000 7f000f800640 1 -- 172.21.15.86:0/1645733074 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0008074590 con 0x7f000806fd50 2024-04-25T14:16:36.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.987+0000 7f000f800640 1 -- 172.21.15.86:0/1645733074 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0008069d00 con 0x7f000806fd50 2024-04-25T14:16:36.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.987+0000 7f0005600640 1 -- 172.21.15.86:0/1645733074 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7efff801b460 con 0x7f000806fd50 2024-04-25T14:16:36.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.987+0000 7f0005600640 1 --2- 172.21.15.86:0/1645733074 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7effe406f450 0x7effe4071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:36.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.987+0000 7f0005600640 1 -- 172.21.15.86:0/1645733074 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7efff808f180 con 0x7f000806fd50 2024-04-25T14:16:36.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.987+0000 7effffe00640 1 --2- 172.21.15.86:0/1645733074 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7effe406f450 0x7effe4071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:36.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.987+0000 7effffe00640 1 --2- 172.21.15.86:0/1645733074 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7effe406f450 0x7effe4071960 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f000815de60 tx=0x7efff0008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:36.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:36.991+0000 7f0005600640 1 -- 172.21.15.86:0/1645733074 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efff8020020 con 0x7f000806fd50 2024-04-25T14:16:37.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:37.183+0000 7f000f800640 1 -- 172.21.15.86:0/1645733074 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0008076bf0 con 0x7f000806fd50 2024-04-25T14:16:37.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:37.183+0000 7f0005600640 1 -- 172.21.15.86:0/1645733074 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7efff805bcf0 con 0x7f000806fd50 2024-04-25T14:16:37.181 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:37.181 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:37.181 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:16:37.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:37.183+0000 7f000f800640 1 -- 172.21.15.86:0/1645733074 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7effe406f450 msgr2=0x7effe4071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:37.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:37.183+0000 7f000f800640 1 --2- 172.21.15.86:0/1645733074 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7effe406f450 0x7effe4071960 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f000815de60 tx=0x7efff0008040 comp rx=0 tx=0).stop 2024-04-25T14:16:37.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:37.183+0000 7f000f800640 1 -- 172.21.15.86:0/1645733074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000806fd50 msgr2=0x7f00080733f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:37.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:37.183+0000 7f000f800640 1 --2- 172.21.15.86:0/1645733074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000806fd50 0x7f00080733f0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7efff800b3c0 tx=0x7efff800b880 comp rx=0 tx=0).stop 2024-04-25T14:16:37.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:37.183+0000 7f000f800640 1 -- 172.21.15.86:0/1645733074 shutdown_connections 2024-04-25T14:16:37.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:37.183+0000 7f000f800640 1 --2- 172.21.15.86:0/1645733074 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7effe406f450 0x7effe4071960 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:37.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:37.183+0000 7f000f800640 1 --2- 172.21.15.86:0/1645733074 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f000815feb0 0x7f00080b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:37.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:37.183+0000 7f000f800640 1 --2- 172.21.15.86:0/1645733074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f000815c2a0 0x7f00081557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:37.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:37.183+0000 7f000f800640 1 --2- 172.21.15.86:0/1645733074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000806fd50 0x7f00080733f0 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:37.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:37.183+0000 7f000f800640 1 -- 172.21.15.86:0/1645733074 >> 172.21.15.86:0/1645733074 conn(0x7f00080b4650 msgr2=0x7f000815a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:37.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:37.183+0000 7f000f800640 1 -- 172.21.15.86:0/1645733074 shutdown_connections 2024-04-25T14:16:37.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:37.183+0000 7f000f800640 1 -- 172.21.15.86:0/1645733074 wait complete. 2024-04-25T14:16:37.606 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:37.606 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:38.607 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:38.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.767+0000 7f9859400640 1 Processor -- start 2024-04-25T14:16:38.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.771+0000 7f9859400640 1 -- start start 2024-04-25T14:16:38.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.771+0000 7f9859400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98540cfca0 0x7f985406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:38.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.771+0000 7f9859400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f98540d16e0 0x7f98540d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:38.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.771+0000 7f9859400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f985406c670 0x7f985406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:38.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.771+0000 7f9859400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9854070470 con 0x7f98540d16e0 2024-04-25T14:16:38.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.771+0000 7f9859400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f98540d2030 con 0x7f985406c670 2024-04-25T14:16:38.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.771+0000 7f9859400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9854072bd0 con 0x7f98540cfca0 2024-04-25T14:16:38.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.771+0000 7f9851e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f98540d16e0 0x7f98540d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:38.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.771+0000 7f9851e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f98540d16e0 0x7f98540d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40174/0 (socket says 172.21.15.86:40174) 2024-04-25T14:16:38.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.771+0000 7f9851e00640 1 -- 172.21.15.86:0/2331229142 learned_addr learned my addr 172.21.15.86:0/2331229142 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:38.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.771+0000 7f9853200640 1 --2- 172.21.15.86:0/2331229142 >> v2:172.21.15.155:3300/0 conn(0x7f985406c670 0x7f985406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:38.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.771+0000 7f9852800640 1 --2- 172.21.15.86:0/2331229142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98540cfca0 0x7f985406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:38.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.771+0000 7f9851e00640 1 -- 172.21.15.86:0/2331229142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98540cfca0 msgr2=0x7f985406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:38.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.771+0000 7f9851e00640 1 --2- 172.21.15.86:0/2331229142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98540cfca0 0x7f985406f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:38.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.771+0000 7f9851e00640 1 -- 172.21.15.86:0/2331229142 >> v2:172.21.15.155:3300/0 conn(0x7f985406c670 msgr2=0x7f985406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:38.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.771+0000 7f9851e00640 1 --2- 172.21.15.86:0/2331229142 >> v2:172.21.15.155:3300/0 conn(0x7f985406c670 0x7f985406ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:38.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.771+0000 7f9851e00640 1 -- 172.21.15.86:0/2331229142 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f985406f930 con 0x7f98540d16e0 2024-04-25T14:16:38.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.771+0000 7f9853200640 1 --2- 172.21.15.86:0/2331229142 >> v2:172.21.15.155:3300/0 conn(0x7f985406c670 0x7f985406ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:16:38.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.775+0000 7f9851e00640 1 --2- 172.21.15.86:0/2331229142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98540d16e0 0x7f98540d1af0 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f985406e060 tx=0x7f984001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66dd24bbe5ba841a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:38.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.775+0000 7f9851400640 1 -- 172.21.15.86:0/2331229142 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98400ab5f0 con 0x7f98540d16e0 2024-04-25T14:16:38.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.775+0000 7f9851400640 1 -- 172.21.15.86:0/2331229142 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98400b0070 con 0x7f98540d16e0 2024-04-25T14:16:38.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.775+0000 7f9851400640 1 -- 172.21.15.86:0/2331229142 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98400ab9b0 con 0x7f98540d16e0 2024-04-25T14:16:38.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.775+0000 7f9859400640 1 -- 172.21.15.86:0/2331229142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98540d16e0 msgr2=0x7f98540d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:38.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.775+0000 7f9859400640 1 --2- 172.21.15.86:0/2331229142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98540d16e0 0x7f98540d1af0 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f985406e060 tx=0x7f984001c520 comp rx=0 tx=0).stop 2024-04-25T14:16:38.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.775+0000 7f9859400640 1 -- 172.21.15.86:0/2331229142 shutdown_connections 2024-04-25T14:16:38.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.775+0000 7f9859400640 1 --2- 172.21.15.86:0/2331229142 >> v2:172.21.15.155:3300/0 conn(0x7f985406c670 0x7f985406ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:38.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.775+0000 7f9859400640 1 --2- 172.21.15.86:0/2331229142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98540d16e0 0x7f98540d1af0 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:38.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.775+0000 7f9859400640 1 --2- 172.21.15.86:0/2331229142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98540cfca0 0x7f985406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:38.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.775+0000 7f9859400640 1 -- 172.21.15.86:0/2331229142 >> 172.21.15.86:0/2331229142 conn(0x7f98540b4650 msgr2=0x7f98540b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:38.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.775+0000 7f9859400640 1 -- 172.21.15.86:0/2331229142 shutdown_connections 2024-04-25T14:16:38.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.775+0000 7f9859400640 1 -- 172.21.15.86:0/2331229142 wait complete. 2024-04-25T14:16:38.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.779+0000 7f9859400640 1 Processor -- start 2024-04-25T14:16:38.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.779+0000 7f9859400640 1 -- start start 2024-04-25T14:16:38.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.779+0000 7f9859400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f985406c670 0x7f98540cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:38.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.779+0000 7f9859400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98540cfca0 0x7f9854156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:38.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.779+0000 7f9859400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98540d16e0 0x7f98541f9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:38.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.779+0000 7f9859400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98540738c0 con 0x7f98540cfca0 2024-04-25T14:16:38.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.779+0000 7f9859400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f985406d3d0 con 0x7f985406c670 2024-04-25T14:16:38.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.779+0000 7f9859400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9854073a40 con 0x7f98540d16e0 2024-04-25T14:16:38.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.779+0000 7f9851e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98540cfca0 0x7f9854156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:38.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.779+0000 7f9851e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98540cfca0 0x7f9854156f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58256/0 (socket says 172.21.15.86:58256) 2024-04-25T14:16:38.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.779+0000 7f9851e00640 1 -- 172.21.15.86:0/3521233928 learned_addr learned my addr 172.21.15.86:0/3521233928 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:38.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.779+0000 7f9853200640 1 --2- 172.21.15.86:0/3521233928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98540d16e0 0x7f98541f9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:38.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.779+0000 7f9852800640 1 --2- 172.21.15.86:0/3521233928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f985406c670 0x7f98540cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:38.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.779+0000 7f9851e00640 1 -- 172.21.15.86:0/3521233928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98540d16e0 msgr2=0x7f98541f9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:38.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.779+0000 7f9851e00640 1 --2- 172.21.15.86:0/3521233928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98540d16e0 0x7f98541f9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:38.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.779+0000 7f9851e00640 1 -- 172.21.15.86:0/3521233928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f985406c670 msgr2=0x7f98540cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:38.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.779+0000 7f9851e00640 1 --2- 172.21.15.86:0/3521233928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f985406c670 0x7f98540cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:38.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.779+0000 7f9851e00640 1 -- 172.21.15.86:0/3521233928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f985415ff00 con 0x7f98540cfca0 2024-04-25T14:16:38.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.779+0000 7f9853200640 1 --2- 172.21.15.86:0/3521233928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98540d16e0 0x7f98541f9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:16:38.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.779+0000 7f9851e00640 1 --2- 172.21.15.86:0/3521233928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98540cfca0 0x7f9854156f70 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f98400ad040 tx=0x7f984001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:38.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.779+0000 7f984aa00640 1 -- 172.21.15.86:0/3521233928 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98400abe60 con 0x7f98540cfca0 2024-04-25T14:16:38.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.779+0000 7f984aa00640 1 -- 172.21.15.86:0/3521233928 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98400be5f0 con 0x7f98540cfca0 2024-04-25T14:16:38.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.779+0000 7f984aa00640 1 -- 172.21.15.86:0/3521233928 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98400be9e0 con 0x7f98540cfca0 2024-04-25T14:16:38.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.779+0000 7f9859400640 1 -- 172.21.15.86:0/3521233928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98541574b0 con 0x7f98540cfca0 2024-04-25T14:16:38.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.779+0000 7f9859400640 1 -- 172.21.15.86:0/3521233928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f985415f940 con 0x7f98540cfca0 2024-04-25T14:16:38.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.783+0000 7f9859400640 1 -- 172.21.15.86:0/3521233928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9854160430 con 0x7f98540cfca0 2024-04-25T14:16:38.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.783+0000 7f984aa00640 1 -- 172.21.15.86:0/3521233928 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f98400b43f0 con 0x7f98540cfca0 2024-04-25T14:16:38.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.783+0000 7f984aa00640 1 --2- 172.21.15.86:0/3521233928 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f983406f480 0x7f9834071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:38.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.783+0000 7f984aa00640 1 -- 172.21.15.86:0/3521233928 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f9840129450 con 0x7f98540cfca0 2024-04-25T14:16:38.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.783+0000 7f9852800640 1 --2- 172.21.15.86:0/3521233928 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f983406f480 0x7f9834071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:38.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.783+0000 7f9852800640 1 --2- 172.21.15.86:0/3521233928 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f983406f480 0x7f9834071990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9854070710 tx=0x7f9844008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:38.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.787+0000 7f984aa00640 1 -- 172.21.15.86:0/3521233928 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9840016110 con 0x7f98540cfca0 2024-04-25T14:16:38.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.983+0000 7f9859400640 1 -- 172.21.15.86:0/3521233928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98540ce5f0 con 0x7f98540cfca0 2024-04-25T14:16:38.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.983+0000 7f984aa00640 1 -- 172.21.15.86:0/3521233928 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f98540ce5f0 con 0x7f98540cfca0 2024-04-25T14:16:38.981 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:38.981 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:38.981 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:16:38.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.983+0000 7f9859400640 1 -- 172.21.15.86:0/3521233928 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f983406f480 msgr2=0x7f9834071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:38.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.983+0000 7f9859400640 1 --2- 172.21.15.86:0/3521233928 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f983406f480 0x7f9834071990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9854070710 tx=0x7f9844008040 comp rx=0 tx=0).stop 2024-04-25T14:16:38.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.983+0000 7f9859400640 1 -- 172.21.15.86:0/3521233928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98540cfca0 msgr2=0x7f9854156f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:38.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.983+0000 7f9859400640 1 --2- 172.21.15.86:0/3521233928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98540cfca0 0x7f9854156f70 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f98400ad040 tx=0x7f984001ad60 comp rx=0 tx=0).stop 2024-04-25T14:16:38.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.983+0000 7f9859400640 1 -- 172.21.15.86:0/3521233928 shutdown_connections 2024-04-25T14:16:38.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.983+0000 7f9859400640 1 --2- 172.21.15.86:0/3521233928 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f983406f480 0x7f9834071990 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:38.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.983+0000 7f9859400640 1 --2- 172.21.15.86:0/3521233928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98540d16e0 0x7f98541f9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:38.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.983+0000 7f9859400640 1 --2- 172.21.15.86:0/3521233928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98540cfca0 0x7f9854156f70 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:38.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.983+0000 7f9859400640 1 --2- 172.21.15.86:0/3521233928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f985406c670 0x7f98540cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:38.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.983+0000 7f9859400640 1 -- 172.21.15.86:0/3521233928 >> 172.21.15.86:0/3521233928 conn(0x7f98540b4650 msgr2=0x7f985415b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:38.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.983+0000 7f9859400640 1 -- 172.21.15.86:0/3521233928 shutdown_connections 2024-04-25T14:16:38.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:38.983+0000 7f9859400640 1 -- 172.21.15.86:0/3521233928 wait complete. 2024-04-25T14:16:39.406 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:39.406 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:40.407 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:40.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.563+0000 7f9a4d200640 1 Processor -- start 2024-04-25T14:16:40.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.563+0000 7f9a4d200640 1 -- start start 2024-04-25T14:16:40.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.567+0000 7f9a4d200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a480cfca0 0x7f9a4806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:40.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.567+0000 7f9a4d200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9a480d16e0 0x7f9a480d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:40.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.567+0000 7f9a4d200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a480d2030 con 0x7f9a4806c670 2024-04-25T14:16:40.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.567+0000 7f9a4d200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9a48070470 con 0x7f9a480d16e0 2024-04-25T14:16:40.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.567+0000 7f9a4d200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9a48072bd0 con 0x7f9a480cfca0 2024-04-25T14:16:40.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.567+0000 7f9a3d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9a480d16e0 0x7f9a480d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:40.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.567+0000 7f9a3d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9a480d16e0 0x7f9a480d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58258/0 (socket says 172.21.15.86:58258) 2024-04-25T14:16:40.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.567+0000 7f9a3d400640 1 -- 172.21.15.86:0/1836028957 learned_addr learned my addr 172.21.15.86:0/1836028957 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:40.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.567+0000 7f9a45e00640 1 --2- 172.21.15.86:0/1836028957 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a480cfca0 0x7f9a4806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:40.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.567+0000 7f9a3d400640 1 -- 172.21.15.86:0/1836028957 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a480cfca0 msgr2=0x7f9a4806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:40.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.567+0000 7f9a3d400640 1 --2- 172.21.15.86:0/1836028957 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a480cfca0 0x7f9a4806f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:40.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.567+0000 7f9a3d400640 1 -- 172.21.15.86:0/1836028957 >> v1:172.21.15.155:6789/0 conn(0x7f9a4806c670 legacy=0x7f9a4806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:40.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.567+0000 7f9a3d400640 1 -- 172.21.15.86:0/1836028957 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4806f930 con 0x7f9a480d16e0 2024-04-25T14:16:40.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.567+0000 7f9a45e00640 1 --2- 172.21.15.86:0/1836028957 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a480cfca0 0x7f9a4806f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:16:40.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.567+0000 7f9a3d400640 1 --2- 172.21.15.86:0/1836028957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a480d16e0 0x7f9a480d1af0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f9a30009870 tx=0x7f9a3001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56af79ff5a2861df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:40.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.571+0000 7f9a45400640 1 -- 172.21.15.86:0/1836028957 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a300a7070 con 0x7f9a480d16e0 2024-04-25T14:16:40.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.571+0000 7f9a45400640 1 -- 172.21.15.86:0/1836028957 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a300b0070 con 0x7f9a480d16e0 2024-04-25T14:16:40.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.571+0000 7f9a45400640 1 -- 172.21.15.86:0/1836028957 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a300ab890 con 0x7f9a480d16e0 2024-04-25T14:16:40.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.571+0000 7f9a4d200640 1 -- 172.21.15.86:0/1836028957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a480d16e0 msgr2=0x7f9a480d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:40.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.571+0000 7f9a4d200640 1 --2- 172.21.15.86:0/1836028957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a480d16e0 0x7f9a480d1af0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f9a30009870 tx=0x7f9a3001c560 comp rx=0 tx=0).stop 2024-04-25T14:16:40.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.571+0000 7f9a4d200640 1 -- 172.21.15.86:0/1836028957 shutdown_connections 2024-04-25T14:16:40.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.571+0000 7f9a4d200640 1 --2- 172.21.15.86:0/1836028957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a480d16e0 0x7f9a480d1af0 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:40.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.571+0000 7f9a4d200640 1 --2- 172.21.15.86:0/1836028957 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a480cfca0 0x7f9a4806f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:40.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.571+0000 7f9a4d200640 1 -- 172.21.15.86:0/1836028957 >> 172.21.15.86:0/1836028957 conn(0x7f9a480b4650 msgr2=0x7f9a480b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:40.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.571+0000 7f9a4d200640 1 -- 172.21.15.86:0/1836028957 shutdown_connections 2024-04-25T14:16:40.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.571+0000 7f9a4d200640 1 -- 172.21.15.86:0/1836028957 wait complete. 2024-04-25T14:16:40.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.571+0000 7f9a4d200640 1 Processor -- start 2024-04-25T14:16:40.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.571+0000 7f9a4d200640 1 -- start start 2024-04-25T14:16:40.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.575+0000 7f9a4d200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a4806c670 0x7f9a481f9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:40.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.575+0000 7f9a4d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a480cfca0 0x7f9a4816a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:40.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.575+0000 7f9a4d200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a480d16e0 0x7f9a481590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:40.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.575+0000 7f9a4d200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9a48168060 con 0x7f9a480cfca0 2024-04-25T14:16:40.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.575+0000 7f9a4d200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9a48167ee0 con 0x7f9a4806c670 2024-04-25T14:16:40.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.575+0000 7f9a4d200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9a481681e0 con 0x7f9a480d16e0 2024-04-25T14:16:40.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.575+0000 7f9a3d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a480cfca0 0x7f9a4816a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:40.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.575+0000 7f9a3d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a480cfca0 0x7f9a4816a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58272/0 (socket says 172.21.15.86:58272) 2024-04-25T14:16:40.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.575+0000 7f9a3d400640 1 -- 172.21.15.86:0/696125189 learned_addr learned my addr 172.21.15.86:0/696125189 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:40.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.575+0000 7f9a3d400640 1 -- 172.21.15.86:0/696125189 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a480d16e0 msgr2=0x7f9a481590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:40.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.575+0000 7f9a3d400640 1 --2- 172.21.15.86:0/696125189 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a480d16e0 0x7f9a481590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:40.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.575+0000 7f9a3d400640 1 -- 172.21.15.86:0/696125189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a4806c670 msgr2=0x7f9a481f9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:40.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.575+0000 7f9a3d400640 1 --2- 172.21.15.86:0/696125189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a4806c670 0x7f9a481f9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:40.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.575+0000 7f9a3d400640 1 -- 172.21.15.86:0/696125189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a480b2170 con 0x7f9a480cfca0 2024-04-25T14:16:40.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.575+0000 7f9a3d400640 1 --2- 172.21.15.86:0/696125189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a480cfca0 0x7f9a4816a070 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f9a3001a190 tx=0x7f9a3001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:40.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.575+0000 7f9a3ea00640 1 -- 172.21.15.86:0/696125189 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a300b4800 con 0x7f9a480cfca0 2024-04-25T14:16:40.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.575+0000 7f9a3ea00640 1 -- 172.21.15.86:0/696125189 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a300b49c0 con 0x7f9a480cfca0 2024-04-25T14:16:40.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.575+0000 7f9a3ea00640 1 -- 172.21.15.86:0/696125189 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a300b4db0 con 0x7f9a480cfca0 2024-04-25T14:16:40.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.575+0000 7f9a4d200640 1 -- 172.21.15.86:0/696125189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a480b2450 con 0x7f9a480cfca0 2024-04-25T14:16:40.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.575+0000 7f9a4d200640 1 -- 172.21.15.86:0/696125189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a480b2a10 con 0x7f9a480cfca0 2024-04-25T14:16:40.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.575+0000 7f9a4d200640 1 -- 172.21.15.86:0/696125189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a4809f4b0 con 0x7f9a480cfca0 2024-04-25T14:16:40.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.579+0000 7f9a3ea00640 1 -- 172.21.15.86:0/696125189 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f9a300c7390 con 0x7f9a480cfca0 2024-04-25T14:16:40.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.579+0000 7f9a3ea00640 1 --2- 172.21.15.86:0/696125189 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9a2406f450 0x7f9a24071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:40.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.579+0000 7f9a3ea00640 1 -- 172.21.15.86:0/696125189 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f9a30129c70 con 0x7f9a480cfca0 2024-04-25T14:16:40.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.579+0000 7f9a45e00640 1 --2- 172.21.15.86:0/696125189 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9a2406f450 0x7f9a24071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:40.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.579+0000 7f9a45e00640 1 --2- 172.21.15.86:0/696125189 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9a2406f450 0x7f9a24071960 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9a48070710 tx=0x7f9a38008350 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:40.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.583+0000 7f9a3ea00640 1 -- 172.21.15.86:0/696125189 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9a300c4060 con 0x7f9a480cfca0 2024-04-25T14:16:40.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.771+0000 7f9a4d200640 1 -- 172.21.15.86:0/696125189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a48069d00 con 0x7f9a480cfca0 2024-04-25T14:16:40.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.771+0000 7f9a3ea00640 1 -- 172.21.15.86:0/696125189 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f9a30016150 con 0x7f9a480cfca0 2024-04-25T14:16:40.770 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:40.770 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:40.770 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:16:40.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.775+0000 7f9a4d200640 1 -- 172.21.15.86:0/696125189 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9a2406f450 msgr2=0x7f9a24071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:40.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.775+0000 7f9a4d200640 1 --2- 172.21.15.86:0/696125189 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9a2406f450 0x7f9a24071960 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9a48070710 tx=0x7f9a38008350 comp rx=0 tx=0).stop 2024-04-25T14:16:40.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.775+0000 7f9a4d200640 1 -- 172.21.15.86:0/696125189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a480cfca0 msgr2=0x7f9a4816a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:40.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.775+0000 7f9a4d200640 1 --2- 172.21.15.86:0/696125189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a480cfca0 0x7f9a4816a070 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f9a3001a190 tx=0x7f9a3001ada0 comp rx=0 tx=0).stop 2024-04-25T14:16:40.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.775+0000 7f9a4d200640 1 -- 172.21.15.86:0/696125189 shutdown_connections 2024-04-25T14:16:40.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.775+0000 7f9a4d200640 1 --2- 172.21.15.86:0/696125189 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9a2406f450 0x7f9a24071960 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:40.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.775+0000 7f9a4d200640 1 --2- 172.21.15.86:0/696125189 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a480d16e0 0x7f9a481590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:40.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.775+0000 7f9a4d200640 1 --2- 172.21.15.86:0/696125189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a480cfca0 0x7f9a4816a070 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:40.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.775+0000 7f9a4d200640 1 --2- 172.21.15.86:0/696125189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a4806c670 0x7f9a481f9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:40.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.775+0000 7f9a4d200640 1 -- 172.21.15.86:0/696125189 >> 172.21.15.86:0/696125189 conn(0x7f9a480b4650 msgr2=0x7f9a480afc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:40.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.775+0000 7f9a4d200640 1 -- 172.21.15.86:0/696125189 shutdown_connections 2024-04-25T14:16:40.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:40.775+0000 7f9a4d200640 1 -- 172.21.15.86:0/696125189 wait complete. 2024-04-25T14:16:41.194 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:41.195 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:42.195 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:42.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.355+0000 7f3279200640 1 Processor -- start 2024-04-25T14:16:42.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.355+0000 7f3279200640 1 -- start start 2024-04-25T14:16:42.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.359+0000 7f3279200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f327415c2a0 0x7f32740ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:42.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.359+0000 7f3279200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32740b0a60 con 0x7f327406fd50 2024-04-25T14:16:42.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.359+0000 7f3279200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f327406ef90 con 0x7f327415feb0 2024-04-25T14:16:42.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.359+0000 7f3279200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f327406f270 con 0x7f327415c2a0 2024-04-25T14:16:42.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.359+0000 7f3269400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f327406fd50 0x7f3274070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44194/0 (socket says 172.21.15.86:44194) 2024-04-25T14:16:42.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.359+0000 7f3269400640 1 -- 172.21.15.86:0/3965663344 learned_addr learned my addr 172.21.15.86:0/3965663344 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:16:42.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.359+0000 7f3271e00640 1 --2- 172.21.15.86:0/3965663344 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f327415c2a0 0x7f32740ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:42.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.359+0000 7f3271400640 1 -- 172.21.15.86:0/3965663344 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3613199523 0 0) 0x7f32740b0a60 con 0x7f327406fd50 2024-04-25T14:16:42.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.359+0000 7f3271400640 1 -- 172.21.15.86:0/3965663344 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3258003530 con 0x7f327406fd50 2024-04-25T14:16:42.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.359+0000 7f3271400640 1 -- 172.21.15.86:0/3965663344 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2093577053 0 0) 0x7f327406ef90 con 0x7f327415feb0 2024-04-25T14:16:42.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.359+0000 7f3271400640 1 -- 172.21.15.86:0/3965663344 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32740b0a60 con 0x7f327415feb0 2024-04-25T14:16:42.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.359+0000 7f3271e00640 1 -- 172.21.15.86:0/3965663344 >> v1:172.21.15.155:6789/0 conn(0x7f327415feb0 legacy=0x7f32741602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:42.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.359+0000 7f3271e00640 1 -- 172.21.15.86:0/3965663344 >> v1:172.21.15.86:6789/0 conn(0x7f327406fd50 legacy=0x7f3274070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:42.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.359+0000 7f3271e00640 1 -- 172.21.15.86:0/3965663344 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f327406f550 con 0x7f327415c2a0 2024-04-25T14:16:42.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.363+0000 7f3271e00640 1 --2- 172.21.15.86:0/3965663344 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f327415c2a0 0x7f32740ae130 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f326400aed0 tx=0x7f326401c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a642b69432f6c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:42.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.363+0000 7f3271400640 1 -- 172.21.15.86:0/3965663344 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32640a5620 con 0x7f327415c2a0 2024-04-25T14:16:42.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.363+0000 7f3271400640 1 -- 172.21.15.86:0/3965663344 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32640a57c0 con 0x7f327415c2a0 2024-04-25T14:16:42.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.363+0000 7f3271400640 1 -- 172.21.15.86:0/3965663344 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32640a5b90 con 0x7f327415c2a0 2024-04-25T14:16:42.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.363+0000 7f3279200640 1 -- 172.21.15.86:0/3965663344 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f327415c2a0 msgr2=0x7f32740ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:42.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.363+0000 7f3279200640 1 --2- 172.21.15.86:0/3965663344 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f327415c2a0 0x7f32740ae130 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f326400aed0 tx=0x7f326401c8f0 comp rx=0 tx=0).stop 2024-04-25T14:16:42.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.363+0000 7f3279200640 1 -- 172.21.15.86:0/3965663344 shutdown_connections 2024-04-25T14:16:42.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.363+0000 7f3279200640 1 --2- 172.21.15.86:0/3965663344 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f327415c2a0 0x7f32740ae130 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:42.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.363+0000 7f3279200640 1 -- 172.21.15.86:0/3965663344 >> 172.21.15.86:0/3965663344 conn(0x7f32740b4650 msgr2=0x7f32740b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:42.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.363+0000 7f3279200640 1 -- 172.21.15.86:0/3965663344 shutdown_connections 2024-04-25T14:16:42.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.363+0000 7f3279200640 1 -- 172.21.15.86:0/3965663344 wait complete. 2024-04-25T14:16:42.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.367+0000 7f3279200640 1 Processor -- start 2024-04-25T14:16:42.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.367+0000 7f3279200640 1 -- start start 2024-04-25T14:16:42.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.367+0000 7f3279200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f327406fd50 0x7f32740b1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:42.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.367+0000 7f3279200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f327415c2a0 0x7f3274073ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:42.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.367+0000 7f3279200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f327415feb0 0x7f327415d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:42.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.367+0000 7f3279200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32740ae7f0 con 0x7f327415feb0 2024-04-25T14:16:42.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.367+0000 7f3279200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f32740ae670 con 0x7f327406fd50 2024-04-25T14:16:42.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.367+0000 7f3279200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f327416a200 con 0x7f327415c2a0 2024-04-25T14:16:42.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.367+0000 7f3272800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f327415feb0 0x7f327415d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:42.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.367+0000 7f3272800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f327415feb0 0x7f327415d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58286/0 (socket says 172.21.15.86:58286) 2024-04-25T14:16:42.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.367+0000 7f3272800640 1 -- 172.21.15.86:0/3635451493 learned_addr learned my addr 172.21.15.86:0/3635451493 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:42.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.367+0000 7f3269400640 1 --2- 172.21.15.86:0/3635451493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f327415c2a0 0x7f3274073ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:42.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.367+0000 7f3271e00640 1 --2- 172.21.15.86:0/3635451493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f327406fd50 0x7f32740b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:42.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.367+0000 7f3272800640 1 -- 172.21.15.86:0/3635451493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f327415c2a0 msgr2=0x7f3274073ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:42.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.367+0000 7f3272800640 1 --2- 172.21.15.86:0/3635451493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f327415c2a0 0x7f3274073ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:42.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.367+0000 7f3272800640 1 -- 172.21.15.86:0/3635451493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f327406fd50 msgr2=0x7f32740b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:42.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.367+0000 7f3272800640 1 --2- 172.21.15.86:0/3635451493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f327406fd50 0x7f32740b1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:42.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.367+0000 7f3272800640 1 -- 172.21.15.86:0/3635451493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32740b2060 con 0x7f327415feb0 2024-04-25T14:16:42.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.367+0000 7f3272800640 1 --2- 172.21.15.86:0/3635451493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f327415feb0 0x7f327415d910 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f326c00c3f0 tx=0x7f326c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:42.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.367+0000 7f326aa00640 1 -- 172.21.15.86:0/3635451493 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c017070 con 0x7f327415feb0 2024-04-25T14:16:42.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.367+0000 7f326aa00640 1 -- 172.21.15.86:0/3635451493 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f326c0052d0 con 0x7f327415feb0 2024-04-25T14:16:42.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.367+0000 7f3279200640 1 -- 172.21.15.86:0/3635451493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32740b2340 con 0x7f327415feb0 2024-04-25T14:16:42.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.367+0000 7f326aa00640 1 -- 172.21.15.86:0/3635451493 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c003820 con 0x7f327415feb0 2024-04-25T14:16:42.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.367+0000 7f3279200640 1 -- 172.21.15.86:0/3635451493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32740b2900 con 0x7f327415feb0 2024-04-25T14:16:42.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.371+0000 7f3279200640 1 -- 172.21.15.86:0/3635451493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32740b1dd0 con 0x7f327415feb0 2024-04-25T14:16:42.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.371+0000 7f326aa00640 1 -- 172.21.15.86:0/3635451493 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f326c01a070 con 0x7f327415feb0 2024-04-25T14:16:42.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.371+0000 7f326aa00640 1 --2- 172.21.15.86:0/3635451493 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f325006f480 0x7f3250071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:42.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.371+0000 7f326aa00640 1 -- 172.21.15.86:0/3635451493 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f326c08e930 con 0x7f327415feb0 2024-04-25T14:16:42.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.371+0000 7f3271e00640 1 --2- 172.21.15.86:0/3635451493 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f325006f480 0x7f3250071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:42.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.371+0000 7f3271e00640 1 --2- 172.21.15.86:0/3635451493 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f325006f480 0x7f3250071990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f32640ae000 tx=0x7f32640a7070 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:42.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.375+0000 7f326aa00640 1 -- 172.21.15.86:0/3635451493 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f326c005480 con 0x7f327415feb0 2024-04-25T14:16:42.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.567+0000 7f3279200640 1 -- 172.21.15.86:0/3635451493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3274076bf0 con 0x7f327415feb0 2024-04-25T14:16:42.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.567+0000 7f326aa00640 1 -- 172.21.15.86:0/3635451493 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f326c05ae50 con 0x7f327415feb0 2024-04-25T14:16:42.565 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:42.565 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:42.565 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:16:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.567+0000 7f3279200640 1 -- 172.21.15.86:0/3635451493 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f325006f480 msgr2=0x7f3250071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.567+0000 7f3279200640 1 --2- 172.21.15.86:0/3635451493 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f325006f480 0x7f3250071990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f32640ae000 tx=0x7f32640a7070 comp rx=0 tx=0).stop 2024-04-25T14:16:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.567+0000 7f3279200640 1 -- 172.21.15.86:0/3635451493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f327415feb0 msgr2=0x7f327415d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.567+0000 7f3279200640 1 --2- 172.21.15.86:0/3635451493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f327415feb0 0x7f327415d910 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f326c00c3f0 tx=0x7f326c00c8b0 comp rx=0 tx=0).stop 2024-04-25T14:16:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.567+0000 7f3279200640 1 -- 172.21.15.86:0/3635451493 shutdown_connections 2024-04-25T14:16:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.567+0000 7f3279200640 1 --2- 172.21.15.86:0/3635451493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f327415feb0 0x7f327415d910 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.567+0000 7f3279200640 1 --2- 172.21.15.86:0/3635451493 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f325006f480 0x7f3250071990 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.567+0000 7f3279200640 1 --2- 172.21.15.86:0/3635451493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f327415c2a0 0x7f3274073ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.567+0000 7f3279200640 1 --2- 172.21.15.86:0/3635451493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f327406fd50 0x7f32740b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.567+0000 7f3279200640 1 -- 172.21.15.86:0/3635451493 >> 172.21.15.86:0/3635451493 conn(0x7f32740b4650 msgr2=0x7f3274073060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.567+0000 7f3279200640 1 -- 172.21.15.86:0/3635451493 shutdown_connections 2024-04-25T14:16:42.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:42.567+0000 7f3279200640 1 -- 172.21.15.86:0/3635451493 wait complete. 2024-04-25T14:16:42.989 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:42.989 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:43.990 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:44.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.151+0000 7f55d7600640 1 Processor -- start 2024-04-25T14:16:44.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.151+0000 7f55d7600640 1 -- start start 2024-04-25T14:16:44.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.151+0000 7f55d7600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f55d0070330 0x7f55d00d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:44.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.151+0000 7f55d7600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55d00ce790 0x7f55d006f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:44.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.151+0000 7f55d7600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55d00d2030 con 0x7f55d00d1c20 2024-04-25T14:16:44.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.151+0000 7f55d7600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f55d015e5c0 con 0x7f55d0070330 2024-04-25T14:16:44.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.151+0000 7f55d7600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f55d015e8a0 con 0x7f55d00ce790 2024-04-25T14:16:44.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.151+0000 7f55d4c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f55d0070330 0x7f55d00d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:44.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.151+0000 7f55d4c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f55d0070330 0x7f55d00d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58296/0 (socket says 172.21.15.86:58296) 2024-04-25T14:16:44.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.151+0000 7f55d4c00640 1 -- 172.21.15.86:0/500265370 learned_addr learned my addr 172.21.15.86:0/500265370 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:44.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.155+0000 7f55d5600640 1 --2- 172.21.15.86:0/500265370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55d00ce790 0x7f55d006f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:44.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.155+0000 7f55d4c00640 1 -- 172.21.15.86:0/500265370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55d00ce790 msgr2=0x7f55d006f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:44.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.155+0000 7f55d4c00640 1 --2- 172.21.15.86:0/500265370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55d00ce790 0x7f55d006f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:44.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.155+0000 7f55d4c00640 1 -- 172.21.15.86:0/500265370 >> v1:172.21.15.155:6789/0 conn(0x7f55d00d1c20 legacy=0x7f55d006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:44.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.155+0000 7f55d4c00640 1 -- 172.21.15.86:0/500265370 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d015eb80 con 0x7f55d0070330 2024-04-25T14:16:44.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.155+0000 7f55d5600640 1 --2- 172.21.15.86:0/500265370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55d00ce790 0x7f55d006f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:16:44.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.155+0000 7f55d4c00640 1 --2- 172.21.15.86:0/500265370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55d0070330 0x7f55d00d16e0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009870 tx=0x7f55c401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cbbaf2918e4a6b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:44.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.155+0000 7f55cf400640 1 -- 172.21.15.86:0/500265370 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c40b0070 con 0x7f55d0070330 2024-04-25T14:16:44.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.155+0000 7f55cf400640 1 -- 172.21.15.86:0/500265370 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55c40ab600 con 0x7f55d0070330 2024-04-25T14:16:44.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.155+0000 7f55cf400640 1 -- 172.21.15.86:0/500265370 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c40ab9d0 con 0x7f55d0070330 2024-04-25T14:16:44.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.159+0000 7f55d7600640 1 -- 172.21.15.86:0/500265370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55d0070330 msgr2=0x7f55d00d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:44.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.159+0000 7f55d7600640 1 --2- 172.21.15.86:0/500265370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55d0070330 0x7f55d00d16e0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009870 tx=0x7f55c401c4d0 comp rx=0 tx=0).stop 2024-04-25T14:16:44.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.159+0000 7f55d7600640 1 -- 172.21.15.86:0/500265370 shutdown_connections 2024-04-25T14:16:44.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.159+0000 7f55d7600640 1 --2- 172.21.15.86:0/500265370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55d00ce790 0x7f55d006f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:44.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.159+0000 7f55d7600640 1 --2- 172.21.15.86:0/500265370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55d0070330 0x7f55d00d16e0 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:44.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.159+0000 7f55d7600640 1 -- 172.21.15.86:0/500265370 >> 172.21.15.86:0/500265370 conn(0x7f55d00b4650 msgr2=0x7f55d00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:44.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.159+0000 7f55d7600640 1 -- 172.21.15.86:0/500265370 shutdown_connections 2024-04-25T14:16:44.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.159+0000 7f55d7600640 1 -- 172.21.15.86:0/500265370 wait complete. 2024-04-25T14:16:44.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.159+0000 7f55d7600640 1 Processor -- start 2024-04-25T14:16:44.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.159+0000 7f55d7600640 1 -- start start 2024-04-25T14:16:44.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.163+0000 7f55d7600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55d0070330 0x7f55d015a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:44.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.163+0000 7f55d7600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55d00ce790 0x7f55d00c3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:44.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.163+0000 7f55d7600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55d00d1c20 0x7f55d00c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:44.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.163+0000 7f55d7600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55d00708c0 con 0x7f55d0070330 2024-04-25T14:16:44.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.163+0000 7f55d7600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f55d0070740 con 0x7f55d00d1c20 2024-04-25T14:16:44.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.163+0000 7f55d7600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f55d016afc0 con 0x7f55d00ce790 2024-04-25T14:16:44.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.163+0000 7f55d4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55d0070330 0x7f55d015a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:44.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.163+0000 7f55d4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55d0070330 0x7f55d015a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58312/0 (socket says 172.21.15.86:58312) 2024-04-25T14:16:44.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.163+0000 7f55d4c00640 1 -- 172.21.15.86:0/1761375336 learned_addr learned my addr 172.21.15.86:0/1761375336 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:44.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.163+0000 7f55cfe00640 1 --2- 172.21.15.86:0/1761375336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55d00ce790 0x7f55d00c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:44.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.163+0000 7f55d5600640 1 --2- 172.21.15.86:0/1761375336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55d00d1c20 0x7f55d00c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:44.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.163+0000 7f55d4c00640 1 -- 172.21.15.86:0/1761375336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55d00ce790 msgr2=0x7f55d00c3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:44.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.163+0000 7f55d4c00640 1 --2- 172.21.15.86:0/1761375336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55d00ce790 0x7f55d00c3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:44.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.163+0000 7f55d4c00640 1 -- 172.21.15.86:0/1761375336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55d00d1c20 msgr2=0x7f55d00c3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:44.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.163+0000 7f55d4c00640 1 --2- 172.21.15.86:0/1761375336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55d00d1c20 0x7f55d00c3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:44.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.163+0000 7f55d4c00640 1 -- 172.21.15.86:0/1761375336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d015a850 con 0x7f55d0070330 2024-04-25T14:16:44.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.163+0000 7f55cfe00640 1 --2- 172.21.15.86:0/1761375336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55d00ce790 0x7f55d00c3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:16:44.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.163+0000 7f55d4c00640 1 --2- 172.21.15.86:0/1761375336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55d0070330 0x7f55d015a310 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f55c401a100 tx=0x7f55c401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:44.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.163+0000 7f55ccc00640 1 -- 172.21.15.86:0/1761375336 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c40ba020 con 0x7f55d0070330 2024-04-25T14:16:44.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.163+0000 7f55ccc00640 1 -- 172.21.15.86:0/1761375336 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55c40b6420 con 0x7f55d0070330 2024-04-25T14:16:44.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.163+0000 7f55d7600640 1 -- 172.21.15.86:0/1761375336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d00be780 con 0x7f55d0070330 2024-04-25T14:16:44.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.163+0000 7f55ccc00640 1 -- 172.21.15.86:0/1761375336 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c40b0040 con 0x7f55d0070330 2024-04-25T14:16:44.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.163+0000 7f55d7600640 1 -- 172.21.15.86:0/1761375336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d00c2c70 con 0x7f55d0070330 2024-04-25T14:16:44.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.163+0000 7f55d7600640 1 -- 172.21.15.86:0/1761375336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55d0069d00 con 0x7f55d0070330 2024-04-25T14:16:44.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.163+0000 7f55ccc00640 1 -- 172.21.15.86:0/1761375336 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f55c40bd020 con 0x7f55d0070330 2024-04-25T14:16:44.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.167+0000 7f55ccc00640 1 --2- 172.21.15.86:0/1761375336 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f55b006f480 0x7f55b0071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:44.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.167+0000 7f55ccc00640 1 -- 172.21.15.86:0/1761375336 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f55c4129720 con 0x7f55d0070330 2024-04-25T14:16:44.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.167+0000 7f55cfe00640 1 --2- 172.21.15.86:0/1761375336 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f55b006f480 0x7f55b0071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:44.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.167+0000 7f55cfe00640 1 --2- 172.21.15.86:0/1761375336 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f55b006f480 0x7f55b0071990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f55d00c2750 tx=0x7f55bc009040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:44.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.171+0000 7f55ccc00640 1 -- 172.21.15.86:0/1761375336 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f55c40c4030 con 0x7f55d0070330 2024-04-25T14:16:44.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.363+0000 7f55d7600640 1 -- 172.21.15.86:0/1761375336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55d015b8c0 con 0x7f55d0070330 2024-04-25T14:16:44.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.363+0000 7f55ccc00640 1 -- 172.21.15.86:0/1761375336 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f55d00be780 con 0x7f55d0070330 2024-04-25T14:16:44.362 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:44.363 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:44.363 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:16:44.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.367+0000 7f55d7600640 1 -- 172.21.15.86:0/1761375336 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f55b006f480 msgr2=0x7f55b0071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:44.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.367+0000 7f55d7600640 1 --2- 172.21.15.86:0/1761375336 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f55b006f480 0x7f55b0071990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f55d00c2750 tx=0x7f55bc009040 comp rx=0 tx=0).stop 2024-04-25T14:16:44.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.367+0000 7f55d7600640 1 -- 172.21.15.86:0/1761375336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55d0070330 msgr2=0x7f55d015a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:44.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.367+0000 7f55d7600640 1 --2- 172.21.15.86:0/1761375336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55d0070330 0x7f55d015a310 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f55c401a100 tx=0x7f55c401ad10 comp rx=0 tx=0).stop 2024-04-25T14:16:44.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.367+0000 7f55d7600640 1 -- 172.21.15.86:0/1761375336 shutdown_connections 2024-04-25T14:16:44.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.367+0000 7f55d7600640 1 --2- 172.21.15.86:0/1761375336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55d00d1c20 0x7f55d00c3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:44.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.367+0000 7f55d7600640 1 --2- 172.21.15.86:0/1761375336 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f55b006f480 0x7f55b0071990 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:44.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.367+0000 7f55d7600640 1 --2- 172.21.15.86:0/1761375336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55d00ce790 0x7f55d00c3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:44.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.367+0000 7f55d7600640 1 --2- 172.21.15.86:0/1761375336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55d0070330 0x7f55d015a310 unknown :-1 s=CLOSED pgs=2307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:44.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.367+0000 7f55d7600640 1 -- 172.21.15.86:0/1761375336 >> 172.21.15.86:0/1761375336 conn(0x7f55d00b4650 msgr2=0x7f55d015d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:44.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.367+0000 7f55d7600640 1 -- 172.21.15.86:0/1761375336 shutdown_connections 2024-04-25T14:16:44.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:44.367+0000 7f55d7600640 1 -- 172.21.15.86:0/1761375336 wait complete. 2024-04-25T14:16:44.787 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:44.787 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:45.789 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:45.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.951+0000 7f7bbf000640 1 Processor -- start 2024-04-25T14:16:45.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.951+0000 7f7bbf000640 1 -- start start 2024-04-25T14:16:45.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.951+0000 7f7bbf000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bb80740c0 0x7f7bb81676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:45.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.951+0000 7f7bbf000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bb8072bd0 con 0x7f7bb80d16e0 2024-04-25T14:16:45.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.955+0000 7f7bbf000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bb806fbd0 con 0x7f7bb80cfca0 2024-04-25T14:16:45.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.955+0000 7f7bbf000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7bb8070470 con 0x7f7bb80740c0 2024-04-25T14:16:45.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.955+0000 7f7baf400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7bb80d16e0 0x7f7bb80d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44196/0 (socket says 172.21.15.86:44196) 2024-04-25T14:16:45.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.955+0000 7f7baf400640 1 -- 172.21.15.86:0/4149476644 learned_addr learned my addr 172.21.15.86:0/4149476644 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:16:45.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.955+0000 7f7bbd000640 1 --2- 172.21.15.86:0/4149476644 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bb80740c0 0x7f7bb81676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:45.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.955+0000 7f7bb7400640 1 -- 172.21.15.86:0/4149476644 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1025252673 0 0) 0x7f7bb8072bd0 con 0x7f7bb80d16e0 2024-04-25T14:16:45.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.955+0000 7f7bb7400640 1 -- 172.21.15.86:0/4149476644 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b9c003530 con 0x7f7bb80d16e0 2024-04-25T14:16:45.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.955+0000 7f7bb7400640 1 -- 172.21.15.86:0/4149476644 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 561117088 0 0) 0x7f7bb806fbd0 con 0x7f7bb80cfca0 2024-04-25T14:16:45.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.955+0000 7f7bb7400640 1 -- 172.21.15.86:0/4149476644 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7bb8072bd0 con 0x7f7bb80cfca0 2024-04-25T14:16:45.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.955+0000 7f7bbd000640 1 -- 172.21.15.86:0/4149476644 >> v1:172.21.15.155:6789/0 conn(0x7f7bb80cfca0 legacy=0x7f7bb806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:45.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.955+0000 7f7bbd000640 1 -- 172.21.15.86:0/4149476644 >> v1:172.21.15.86:6789/0 conn(0x7f7bb80d16e0 legacy=0x7f7bb80d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:45.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.955+0000 7f7bb7400640 1 -- 172.21.15.86:0/4149476644 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2881263328 0 0) 0x7f7b9c003530 con 0x7f7bb80d16e0 2024-04-25T14:16:45.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.955+0000 7f7bbd000640 1 -- 172.21.15.86:0/4149476644 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb806feb0 con 0x7f7bb80740c0 2024-04-25T14:16:45.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.959+0000 7f7bbd000640 1 --2- 172.21.15.86:0/4149476644 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bb80740c0 0x7f7bb81676d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7bb00097f0 tx=0x7f7bb001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0bdc6691366ce04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:45.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.959+0000 7f7bb7400640 1 -- 172.21.15.86:0/4149476644 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb00b0070 con 0x7f7bb80740c0 2024-04-25T14:16:45.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.959+0000 7f7bb7400640 1 -- 172.21.15.86:0/4149476644 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bb00ab400 con 0x7f7bb80740c0 2024-04-25T14:16:45.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.959+0000 7f7bb7400640 1 -- 172.21.15.86:0/4149476644 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0001a00 con 0x7f7bb80740c0 2024-04-25T14:16:45.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.959+0000 7f7bbf000640 1 -- 172.21.15.86:0/4149476644 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bb80740c0 msgr2=0x7f7bb81676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:45.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.959+0000 7f7bbf000640 1 --2- 172.21.15.86:0/4149476644 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bb80740c0 0x7f7bb81676d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7bb00097f0 tx=0x7f7bb001c400 comp rx=0 tx=0).stop 2024-04-25T14:16:45.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.959+0000 7f7bbf000640 1 -- 172.21.15.86:0/4149476644 shutdown_connections 2024-04-25T14:16:45.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.959+0000 7f7bbf000640 1 --2- 172.21.15.86:0/4149476644 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bb80740c0 0x7f7bb81676d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:45.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.959+0000 7f7bbf000640 1 -- 172.21.15.86:0/4149476644 >> 172.21.15.86:0/4149476644 conn(0x7f7bb80b4650 msgr2=0x7f7bb80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:45.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.959+0000 7f7bbf000640 1 -- 172.21.15.86:0/4149476644 shutdown_connections 2024-04-25T14:16:45.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.959+0000 7f7bbf000640 1 -- 172.21.15.86:0/4149476644 wait complete. 2024-04-25T14:16:45.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.959+0000 7f7bbf000640 1 Processor -- start 2024-04-25T14:16:45.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.959+0000 7f7bbf000640 1 -- start start 2024-04-25T14:16:45.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.963+0000 7f7bbf000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bb80740c0 0x7f7bb815fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:45.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.963+0000 7f7bbf000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb80cfca0 0x7f7bb80cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:45.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.963+0000 7f7bbf000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bb80d16e0 0x7f7bb81599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:45.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.963+0000 7f7bbf000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb816ade0 con 0x7f7bb80cfca0 2024-04-25T14:16:45.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.963+0000 7f7bbf000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb816ac60 con 0x7f7bb80d16e0 2024-04-25T14:16:45.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.963+0000 7f7bbf000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7bb816af60 con 0x7f7bb80740c0 2024-04-25T14:16:45.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.963+0000 7f7baf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb80cfca0 0x7f7bb80cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:45.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.963+0000 7f7baf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb80cfca0 0x7f7bb80cc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58314/0 (socket says 172.21.15.86:58314) 2024-04-25T14:16:45.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.963+0000 7f7baf400640 1 -- 172.21.15.86:0/2665344897 learned_addr learned my addr 172.21.15.86:0/2665344897 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:45.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.963+0000 7f7bbd000640 1 --2- 172.21.15.86:0/2665344897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bb80d16e0 0x7f7bb81599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:45.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.963+0000 7f7bb7e00640 1 --2- 172.21.15.86:0/2665344897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bb80740c0 0x7f7bb815fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:45.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.963+0000 7f7baf400640 1 -- 172.21.15.86:0/2665344897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bb80740c0 msgr2=0x7f7bb815fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:45.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.963+0000 7f7baf400640 1 --2- 172.21.15.86:0/2665344897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bb80740c0 0x7f7bb815fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:45.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.963+0000 7f7baf400640 1 -- 172.21.15.86:0/2665344897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bb80d16e0 msgr2=0x7f7bb81599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:45.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.963+0000 7f7baf400640 1 --2- 172.21.15.86:0/2665344897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bb80d16e0 0x7f7bb81599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:45.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.963+0000 7f7baf400640 1 -- 172.21.15.86:0/2665344897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb81601a0 con 0x7f7bb80cfca0 2024-04-25T14:16:45.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.963+0000 7f7bbd000640 1 --2- 172.21.15.86:0/2665344897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bb80d16e0 0x7f7bb81599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:16:45.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.963+0000 7f7baf400640 1 --2- 172.21.15.86:0/2665344897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb80cfca0 0x7f7bb80cc6b0 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000cc30 tx=0x7f7ba000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:45.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.963+0000 7f7bb4c00640 1 -- 172.21.15.86:0/2665344897 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0017070 con 0x7f7bb80cfca0 2024-04-25T14:16:45.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.963+0000 7f7bb4c00640 1 -- 172.21.15.86:0/2665344897 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ba000ce40 con 0x7f7bb80cfca0 2024-04-25T14:16:45.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.963+0000 7f7bbf000640 1 -- 172.21.15.86:0/2665344897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bb80cdd60 con 0x7f7bb80cfca0 2024-04-25T14:16:45.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.963+0000 7f7bb4c00640 1 -- 172.21.15.86:0/2665344897 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0012640 con 0x7f7bb80cfca0 2024-04-25T14:16:45.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.963+0000 7f7bbf000640 1 -- 172.21.15.86:0/2665344897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bb80ca310 con 0x7f7bb80cfca0 2024-04-25T14:16:45.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.963+0000 7f7bbf000640 1 -- 172.21.15.86:0/2665344897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bb809f4b0 con 0x7f7bb80cfca0 2024-04-25T14:16:45.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.967+0000 7f7bb4c00640 1 -- 172.21.15.86:0/2665344897 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f7ba001a070 con 0x7f7bb80cfca0 2024-04-25T14:16:45.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.967+0000 7f7bb4c00640 1 --2- 172.21.15.86:0/2665344897 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7b9406f450 0x7f7b94071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:45.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.967+0000 7f7bb4c00640 1 -- 172.21.15.86:0/2665344897 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f7ba008f5a0 con 0x7f7bb80cfca0 2024-04-25T14:16:45.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.967+0000 7f7bb7e00640 1 --2- 172.21.15.86:0/2665344897 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7b9406f450 0x7f7b94071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:45.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.967+0000 7f7bb7e00640 1 --2- 172.21.15.86:0/2665344897 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7b9406f450 0x7f7b94071960 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8000f70 tx=0x7f7ba8009040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:45.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:45.971+0000 7f7bb4c00640 1 -- 172.21.15.86:0/2665344897 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7ba005c1a0 con 0x7f7bb80cfca0 2024-04-25T14:16:46.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:46.159+0000 7f7bbf000640 1 -- 172.21.15.86:0/2665344897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7bb8069d00 con 0x7f7bb80cfca0 2024-04-25T14:16:46.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:46.163+0000 7f7bb4c00640 1 -- 172.21.15.86:0/2665344897 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f7ba005baa0 con 0x7f7bb80cfca0 2024-04-25T14:16:46.160 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:46.160 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:46.160 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:16:46.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:46.163+0000 7f7bbf000640 1 -- 172.21.15.86:0/2665344897 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7b9406f450 msgr2=0x7f7b94071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:46.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:46.163+0000 7f7bbf000640 1 --2- 172.21.15.86:0/2665344897 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7b9406f450 0x7f7b94071960 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8000f70 tx=0x7f7ba8009040 comp rx=0 tx=0).stop 2024-04-25T14:16:46.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:46.163+0000 7f7bbf000640 1 -- 172.21.15.86:0/2665344897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb80cfca0 msgr2=0x7f7bb80cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:46.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:46.163+0000 7f7bbf000640 1 --2- 172.21.15.86:0/2665344897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb80cfca0 0x7f7bb80cc6b0 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000cc30 tx=0x7f7ba000a430 comp rx=0 tx=0).stop 2024-04-25T14:16:46.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:46.163+0000 7f7bbf000640 1 -- 172.21.15.86:0/2665344897 shutdown_connections 2024-04-25T14:16:46.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:46.163+0000 7f7bbf000640 1 --2- 172.21.15.86:0/2665344897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bb80d16e0 0x7f7bb81599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:46.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:46.163+0000 7f7bbf000640 1 --2- 172.21.15.86:0/2665344897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb80cfca0 0x7f7bb80cc6b0 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:46.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:46.163+0000 7f7bbf000640 1 --2- 172.21.15.86:0/2665344897 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7b9406f450 0x7f7b94071960 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:46.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:46.163+0000 7f7bbf000640 1 --2- 172.21.15.86:0/2665344897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bb80740c0 0x7f7bb815fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:46.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:46.163+0000 7f7bbf000640 1 -- 172.21.15.86:0/2665344897 >> 172.21.15.86:0/2665344897 conn(0x7f7bb80b4650 msgr2=0x7f7bb80b0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:46.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:46.163+0000 7f7bbf000640 1 -- 172.21.15.86:0/2665344897 shutdown_connections 2024-04-25T14:16:46.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:46.163+0000 7f7bbf000640 1 -- 172.21.15.86:0/2665344897 wait complete. 2024-04-25T14:16:46.585 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:46.586 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:47.586 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:47.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.747+0000 7fbe26400640 1 Processor -- start 2024-04-25T14:16:47.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.747+0000 7fbe26400640 1 -- start start 2024-04-25T14:16:47.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.747+0000 7fbe26400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbe20072630 0x7fbe20072a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:47.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.747+0000 7fbe26400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbe20163980 0x7fbe20163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:47.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.747+0000 7fbe26400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe2006e340 0x7fbe2006e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:47.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.751+0000 7fbe26400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbe20072f80 con 0x7fbe20072630 2024-04-25T14:16:47.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.751+0000 7fbe26400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fbe201648e0 con 0x7fbe20163980 2024-04-25T14:16:47.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.751+0000 7fbe26400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbe200d1520 con 0x7fbe2006e340 2024-04-25T14:16:47.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.751+0000 7fbe1f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbe20072630 0x7fbe20072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:47.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.751+0000 7fbe1f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbe20072630 0x7fbe20072a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58324/0 (socket says 172.21.15.86:58324) 2024-04-25T14:16:47.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.751+0000 7fbe1f400640 1 -- 172.21.15.86:0/3679884488 learned_addr learned my addr 172.21.15.86:0/3679884488 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.751+0000 7fbe1fe00640 1 --2- 172.21.15.86:0/3679884488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe2006e340 0x7fbe2006e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.751+0000 7fbe17e00640 1 --2- 172.21.15.86:0/3679884488 >> v2:172.21.15.155:3300/0 conn(0x7fbe20163980 0x7fbe20163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.751+0000 7fbe1f400640 1 -- 172.21.15.86:0/3679884488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe2006e340 msgr2=0x7fbe2006e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.751+0000 7fbe1f400640 1 --2- 172.21.15.86:0/3679884488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe2006e340 0x7fbe2006e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.751+0000 7fbe1f400640 1 -- 172.21.15.86:0/3679884488 >> v2:172.21.15.155:3300/0 conn(0x7fbe20163980 msgr2=0x7fbe20163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.751+0000 7fbe1f400640 1 --2- 172.21.15.86:0/3679884488 >> v2:172.21.15.155:3300/0 conn(0x7fbe20163980 0x7fbe20163d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.751+0000 7fbe1f400640 1 -- 172.21.15.86:0/3679884488 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe200d23e0 con 0x7fbe20072630 2024-04-25T14:16:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.751+0000 7fbe17e00640 1 --2- 172.21.15.86:0/3679884488 >> v2:172.21.15.155:3300/0 conn(0x7fbe20163980 0x7fbe20163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:16:47.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.751+0000 7fbe1f400640 1 --2- 172.21.15.86:0/3679884488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe20072630 0x7fbe20072a40 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7fbe20071eb0 tx=0x7fbe1001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c0bbeb5fe29b2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:47.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.755+0000 7fbe1ea00640 1 -- 172.21.15.86:0/3679884488 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe100b0070 con 0x7fbe20072630 2024-04-25T14:16:47.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.755+0000 7fbe1ea00640 1 -- 172.21.15.86:0/3679884488 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe100a4e50 con 0x7fbe20072630 2024-04-25T14:16:47.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.755+0000 7fbe1ea00640 1 -- 172.21.15.86:0/3679884488 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe100ab820 con 0x7fbe20072630 2024-04-25T14:16:47.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.755+0000 7fbe26400640 1 -- 172.21.15.86:0/3679884488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe20072630 msgr2=0x7fbe20072a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:47.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.755+0000 7fbe26400640 1 --2- 172.21.15.86:0/3679884488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe20072630 0x7fbe20072a40 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7fbe20071eb0 tx=0x7fbe1001c400 comp rx=0 tx=0).stop 2024-04-25T14:16:47.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.755+0000 7fbe26400640 1 -- 172.21.15.86:0/3679884488 shutdown_connections 2024-04-25T14:16:47.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.755+0000 7fbe26400640 1 --2- 172.21.15.86:0/3679884488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe2006e340 0x7fbe2006e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:47.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.755+0000 7fbe26400640 1 --2- 172.21.15.86:0/3679884488 >> v2:172.21.15.155:3300/0 conn(0x7fbe20163980 0x7fbe20163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:47.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.755+0000 7fbe26400640 1 --2- 172.21.15.86:0/3679884488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe20072630 0x7fbe20072a40 unknown :-1 s=CLOSED pgs=2310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:47.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.755+0000 7fbe26400640 1 -- 172.21.15.86:0/3679884488 >> 172.21.15.86:0/3679884488 conn(0x7fbe200d3580 msgr2=0x7fbe200d3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:47.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.755+0000 7fbe26400640 1 -- 172.21.15.86:0/3679884488 shutdown_connections 2024-04-25T14:16:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.755+0000 7fbe26400640 1 -- 172.21.15.86:0/3679884488 wait complete. 2024-04-25T14:16:47.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.755+0000 7fbe26400640 1 Processor -- start 2024-04-25T14:16:47.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.755+0000 7fbe26400640 1 -- start start 2024-04-25T14:16:47.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.759+0000 7fbe26400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe2006e340 0x7fbe200ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:47.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.759+0000 7fbe26400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe20072630 0x7fbe200cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:47.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.759+0000 7fbe26400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe20163980 0x7fbe200c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:47.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.759+0000 7fbe26400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbe200cfcb0 con 0x7fbe20163980 2024-04-25T14:16:47.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.759+0000 7fbe26400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2006f0a0 con 0x7fbe2006e340 2024-04-25T14:16:47.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.759+0000 7fbe26400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbe200cfe30 con 0x7fbe20072630 2024-04-25T14:16:47.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.759+0000 7fbe1fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe20163980 0x7fbe200c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:47.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.759+0000 7fbe1fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe20163980 0x7fbe200c7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58332/0 (socket says 172.21.15.86:58332) 2024-04-25T14:16:47.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.759+0000 7fbe1fe00640 1 -- 172.21.15.86:0/1946710309 learned_addr learned my addr 172.21.15.86:0/1946710309 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:47.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.759+0000 7fbe1f400640 1 --2- 172.21.15.86:0/1946710309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe2006e340 0x7fbe200ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:47.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.759+0000 7fbe17e00640 1 --2- 172.21.15.86:0/1946710309 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe20072630 0x7fbe200cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:47.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.759+0000 7fbe1fe00640 1 -- 172.21.15.86:0/1946710309 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe20072630 msgr2=0x7fbe200cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:47.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.759+0000 7fbe1fe00640 1 --2- 172.21.15.86:0/1946710309 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe20072630 0x7fbe200cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:47.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.759+0000 7fbe1fe00640 1 -- 172.21.15.86:0/1946710309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe2006e340 msgr2=0x7fbe200ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:47.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.759+0000 7fbe1fe00640 1 --2- 172.21.15.86:0/1946710309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe2006e340 0x7fbe200ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:47.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.759+0000 7fbe1fe00640 1 -- 172.21.15.86:0/1946710309 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe200caed0 con 0x7fbe20163980 2024-04-25T14:16:47.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.759+0000 7fbe1fe00640 1 --2- 172.21.15.86:0/1946710309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe20163980 0x7fbe200c7fa0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7fbe18007cf0 tx=0x7fbe1800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:47.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.759+0000 7fbe17400640 1 -- 172.21.15.86:0/1946710309 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe18014420 con 0x7fbe20163980 2024-04-25T14:16:47.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.759+0000 7fbe17400640 1 -- 172.21.15.86:0/1946710309 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe18019070 con 0x7fbe20163980 2024-04-25T14:16:47.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.759+0000 7fbe17400640 1 -- 172.21.15.86:0/1946710309 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe18014800 con 0x7fbe20163980 2024-04-25T14:16:47.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.759+0000 7fbe26400640 1 -- 172.21.15.86:0/1946710309 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe200cd6c0 con 0x7fbe20163980 2024-04-25T14:16:47.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.759+0000 7fbe26400640 1 -- 172.21.15.86:0/1946710309 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe200ca760 con 0x7fbe20163980 2024-04-25T14:16:47.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.759+0000 7fbe26400640 1 -- 172.21.15.86:0/1946710309 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe20072170 con 0x7fbe20163980 2024-04-25T14:16:47.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.763+0000 7fbe17400640 1 -- 172.21.15.86:0/1946710309 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7fbe180149a0 con 0x7fbe20163980 2024-04-25T14:16:47.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.763+0000 7fbe17400640 1 --2- 172.21.15.86:0/1946710309 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fbdfc06f480 0x7fbdfc071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:47.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.763+0000 7fbe17400640 1 -- 172.21.15.86:0/1946710309 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fbe1808fbb0 con 0x7fbe20163980 2024-04-25T14:16:47.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.763+0000 7fbe1f400640 1 --2- 172.21.15.86:0/1946710309 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fbdfc06f480 0x7fbdfc071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:47.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.763+0000 7fbe1f400640 1 --2- 172.21.15.86:0/1946710309 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fbdfc06f480 0x7fbdfc071990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbe100ad000 tx=0x7fbe10008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:47.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.767+0000 7fbe17400640 1 -- 172.21.15.86:0/1946710309 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbe1805c7b0 con 0x7fbe20163980 2024-04-25T14:16:47.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.959+0000 7fbe26400640 1 -- 172.21.15.86:0/1946710309 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe200cd890 con 0x7fbe20163980 2024-04-25T14:16:47.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.959+0000 7fbe17400640 1 -- 172.21.15.86:0/1946710309 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7fbe1805c0b0 con 0x7fbe20163980 2024-04-25T14:16:47.958 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:47.958 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:47.958 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:16:47.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.963+0000 7fbe26400640 1 -- 172.21.15.86:0/1946710309 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fbdfc06f480 msgr2=0x7fbdfc071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:47.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.963+0000 7fbe26400640 1 --2- 172.21.15.86:0/1946710309 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fbdfc06f480 0x7fbdfc071990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbe100ad000 tx=0x7fbe10008040 comp rx=0 tx=0).stop 2024-04-25T14:16:47.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.963+0000 7fbe26400640 1 -- 172.21.15.86:0/1946710309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe20163980 msgr2=0x7fbe200c7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:47.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.963+0000 7fbe26400640 1 --2- 172.21.15.86:0/1946710309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe20163980 0x7fbe200c7fa0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7fbe18007cf0 tx=0x7fbe1800e4b0 comp rx=0 tx=0).stop 2024-04-25T14:16:47.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.963+0000 7fbe26400640 1 -- 172.21.15.86:0/1946710309 shutdown_connections 2024-04-25T14:16:47.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.963+0000 7fbe26400640 1 --2- 172.21.15.86:0/1946710309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe20163980 0x7fbe200c7fa0 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:47.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.963+0000 7fbe26400640 1 --2- 172.21.15.86:0/1946710309 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fbdfc06f480 0x7fbdfc071990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:47.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.963+0000 7fbe26400640 1 --2- 172.21.15.86:0/1946710309 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe20072630 0x7fbe200cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:47.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.963+0000 7fbe26400640 1 --2- 172.21.15.86:0/1946710309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe2006e340 0x7fbe200ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:47.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.963+0000 7fbe26400640 1 -- 172.21.15.86:0/1946710309 >> 172.21.15.86:0/1946710309 conn(0x7fbe200d3580 msgr2=0x7fbe201629d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:47.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.963+0000 7fbe26400640 1 -- 172.21.15.86:0/1946710309 shutdown_connections 2024-04-25T14:16:47.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:47.963+0000 7fbe26400640 1 -- 172.21.15.86:0/1946710309 wait complete. 2024-04-25T14:16:48.382 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:48.382 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:49.383 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:49.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.543+0000 7f5a53600640 1 Processor -- start 2024-04-25T14:16:49.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.543+0000 7f5a53600640 1 -- start start 2024-04-25T14:16:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.547+0000 7f5a53600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a4c0740c0 0x7f5a4c1676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.547+0000 7f5a53600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a4c072bd0 con 0x7f5a4c0cfca0 2024-04-25T14:16:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.547+0000 7f5a53600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a4c06fbd0 con 0x7f5a4c0d16e0 2024-04-25T14:16:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.547+0000 7f5a53600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5a4c070470 con 0x7f5a4c0740c0 2024-04-25T14:16:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.547+0000 7f5a50c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5a4c0cfca0 0x7f5a4c06f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52040/0 (socket says 172.21.15.86:52040) 2024-04-25T14:16:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.547+0000 7f5a50c00640 1 -- 172.21.15.86:0/1688278482 learned_addr learned my addr 172.21.15.86:0/1688278482 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:16:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.547+0000 7f5a51600640 1 --2- 172.21.15.86:0/1688278482 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a4c0740c0 0x7f5a4c1676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.547+0000 7f5a4be00640 1 -- 172.21.15.86:0/1688278482 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2094389988 0 0) 0x7f5a4c072bd0 con 0x7f5a4c0cfca0 2024-04-25T14:16:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.547+0000 7f5a4be00640 1 -- 172.21.15.86:0/1688278482 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a30003530 con 0x7f5a4c0cfca0 2024-04-25T14:16:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.547+0000 7f5a4be00640 1 -- 172.21.15.86:0/1688278482 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2945906814 0 0) 0x7f5a4c06fbd0 con 0x7f5a4c0d16e0 2024-04-25T14:16:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.547+0000 7f5a4be00640 1 -- 172.21.15.86:0/1688278482 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a4c072bd0 con 0x7f5a4c0d16e0 2024-04-25T14:16:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.547+0000 7f5a51600640 1 -- 172.21.15.86:0/1688278482 >> v1:172.21.15.155:6789/0 conn(0x7f5a4c0d16e0 legacy=0x7f5a4c0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.547+0000 7f5a51600640 1 -- 172.21.15.86:0/1688278482 >> v1:172.21.15.86:6789/0 conn(0x7f5a4c0cfca0 legacy=0x7f5a4c06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.547+0000 7f5a51600640 1 -- 172.21.15.86:0/1688278482 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4c06feb0 con 0x7f5a4c0740c0 2024-04-25T14:16:49.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.551+0000 7f5a51600640 1 --2- 172.21.15.86:0/1688278482 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a4c0740c0 0x7f5a4c1676d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009dc0 tx=0x7f5a4401c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecf7cc9b4d587ca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:49.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.551+0000 7f5a4be00640 1 -- 172.21.15.86:0/1688278482 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a440b2070 con 0x7f5a4c0740c0 2024-04-25T14:16:49.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.551+0000 7f5a4be00640 1 -- 172.21.15.86:0/1688278482 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a440a5860 con 0x7f5a4c0740c0 2024-04-25T14:16:49.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.551+0000 7f5a4be00640 1 -- 172.21.15.86:0/1688278482 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a440a5c30 con 0x7f5a4c0740c0 2024-04-25T14:16:49.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.551+0000 7f5a53600640 1 -- 172.21.15.86:0/1688278482 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a4c0740c0 msgr2=0x7f5a4c1676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:49.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.551+0000 7f5a53600640 1 --2- 172.21.15.86:0/1688278482 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a4c0740c0 0x7f5a4c1676d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009dc0 tx=0x7f5a4401c990 comp rx=0 tx=0).stop 2024-04-25T14:16:49.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.551+0000 7f5a53600640 1 -- 172.21.15.86:0/1688278482 shutdown_connections 2024-04-25T14:16:49.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.551+0000 7f5a53600640 1 --2- 172.21.15.86:0/1688278482 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a4c0740c0 0x7f5a4c1676d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:49.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.551+0000 7f5a53600640 1 -- 172.21.15.86:0/1688278482 >> 172.21.15.86:0/1688278482 conn(0x7f5a4c0b4650 msgr2=0x7f5a4c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:49.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.551+0000 7f5a53600640 1 -- 172.21.15.86:0/1688278482 shutdown_connections 2024-04-25T14:16:49.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.551+0000 7f5a53600640 1 -- 172.21.15.86:0/1688278482 wait complete. 2024-04-25T14:16:49.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.551+0000 7f5a53600640 1 Processor -- start 2024-04-25T14:16:49.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.551+0000 7f5a53600640 1 -- start start 2024-04-25T14:16:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.555+0000 7f5a53600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a4c0740c0 0x7f5a4c157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.555+0000 7f5a53600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a4c0cfca0 0x7f5a4c15a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.555+0000 7f5a53600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a4c0d16e0 0x7f5a4c1548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.555+0000 7f5a53600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c0ce900 con 0x7f5a4c0d16e0 2024-04-25T14:16:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.555+0000 7f5a53600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c0ce780 con 0x7f5a4c0cfca0 2024-04-25T14:16:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.555+0000 7f5a53600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5a4c0cea80 con 0x7f5a4c0740c0 2024-04-25T14:16:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.555+0000 7f5a51600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a4c0d16e0 0x7f5a4c1548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.555+0000 7f5a51600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a4c0d16e0 0x7f5a4c1548e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40100/0 (socket says 172.21.15.86:40100) 2024-04-25T14:16:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.555+0000 7f5a51600640 1 -- 172.21.15.86:0/1957595796 learned_addr learned my addr 172.21.15.86:0/1957595796 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.555+0000 7f5a50c00640 1 --2- 172.21.15.86:0/1957595796 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a4c0740c0 0x7f5a4c157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.555+0000 7f5a43e00640 1 --2- 172.21.15.86:0/1957595796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a4c0cfca0 0x7f5a4c15a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.555+0000 7f5a51600640 1 -- 172.21.15.86:0/1957595796 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a4c0740c0 msgr2=0x7f5a4c157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.555+0000 7f5a51600640 1 --2- 172.21.15.86:0/1957595796 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a4c0740c0 0x7f5a4c157e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.555+0000 7f5a51600640 1 -- 172.21.15.86:0/1957595796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a4c0cfca0 msgr2=0x7f5a4c15a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.555+0000 7f5a51600640 1 --2- 172.21.15.86:0/1957595796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a4c0cfca0 0x7f5a4c15a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.555+0000 7f5a51600640 1 -- 172.21.15.86:0/1957595796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4c158890 con 0x7f5a4c0d16e0 2024-04-25T14:16:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.555+0000 7f5a51600640 1 --2- 172.21.15.86:0/1957595796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a4c0d16e0 0x7f5a4c1548e0 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f5a4401a5c0 tx=0x7f5a44019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.555+0000 7f5a49600640 1 -- 172.21.15.86:0/1957595796 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a440b2050 con 0x7f5a4c0d16e0 2024-04-25T14:16:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.555+0000 7f5a49600640 1 -- 172.21.15.86:0/1957595796 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a440be070 con 0x7f5a4c0d16e0 2024-04-25T14:16:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.555+0000 7f5a49600640 1 -- 172.21.15.86:0/1957595796 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a44009450 con 0x7f5a4c0d16e0 2024-04-25T14:16:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.555+0000 7f5a53600640 1 -- 172.21.15.86:0/1957595796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4c158b70 con 0x7f5a4c0d16e0 2024-04-25T14:16:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.555+0000 7f5a53600640 1 -- 172.21.15.86:0/1957595796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a4c155c70 con 0x7f5a4c0d16e0 2024-04-25T14:16:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.555+0000 7f5a53600640 1 -- 172.21.15.86:0/1957595796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a4c09f4b0 con 0x7f5a4c0d16e0 2024-04-25T14:16:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.559+0000 7f5a49600640 1 -- 172.21.15.86:0/1957595796 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f5a440adc20 con 0x7f5a4c0d16e0 2024-04-25T14:16:49.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.559+0000 7f5a49600640 1 --2- 172.21.15.86:0/1957595796 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f5a2806f450 0x7f5a28071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:49.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.559+0000 7f5a49600640 1 -- 172.21.15.86:0/1957595796 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f5a4412a5e0 con 0x7f5a4c0d16e0 2024-04-25T14:16:49.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.559+0000 7f5a50c00640 1 --2- 172.21.15.86:0/1957595796 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f5a2806f450 0x7f5a28071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.559+0000 7f5a50c00640 1 --2- 172.21.15.86:0/1957595796 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f5a2806f450 0x7f5a28071960 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c007fd0 tx=0x7f5a3c007d80 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.563+0000 7f5a49600640 1 -- 172.21.15.86:0/1957595796 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5a44016580 con 0x7f5a4c0d16e0 2024-04-25T14:16:49.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.751+0000 7f5a53600640 1 -- 172.21.15.86:0/1957595796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a4c069d00 con 0x7f5a4c0d16e0 2024-04-25T14:16:49.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.751+0000 7f5a49600640 1 -- 172.21.15.86:0/1957595796 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f5a440f6bc0 con 0x7f5a4c0d16e0 2024-04-25T14:16:49.751 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:49.751 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:49.751 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:16:49.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.755+0000 7f5a53600640 1 -- 172.21.15.86:0/1957595796 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f5a2806f450 msgr2=0x7f5a28071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:49.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.755+0000 7f5a53600640 1 --2- 172.21.15.86:0/1957595796 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f5a2806f450 0x7f5a28071960 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c007fd0 tx=0x7f5a3c007d80 comp rx=0 tx=0).stop 2024-04-25T14:16:49.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.755+0000 7f5a53600640 1 -- 172.21.15.86:0/1957595796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a4c0d16e0 msgr2=0x7f5a4c1548e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:49.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.755+0000 7f5a53600640 1 --2- 172.21.15.86:0/1957595796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a4c0d16e0 0x7f5a4c1548e0 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f5a4401a5c0 tx=0x7f5a44019730 comp rx=0 tx=0).stop 2024-04-25T14:16:49.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.755+0000 7f5a53600640 1 -- 172.21.15.86:0/1957595796 shutdown_connections 2024-04-25T14:16:49.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.755+0000 7f5a53600640 1 --2- 172.21.15.86:0/1957595796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a4c0d16e0 0x7f5a4c1548e0 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:49.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.755+0000 7f5a53600640 1 --2- 172.21.15.86:0/1957595796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a4c0cfca0 0x7f5a4c15a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:49.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.755+0000 7f5a53600640 1 --2- 172.21.15.86:0/1957595796 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f5a2806f450 0x7f5a28071960 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:49.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.755+0000 7f5a53600640 1 --2- 172.21.15.86:0/1957595796 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a4c0740c0 0x7f5a4c157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:49.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.755+0000 7f5a53600640 1 -- 172.21.15.86:0/1957595796 >> 172.21.15.86:0/1957595796 conn(0x7f5a4c0b4650 msgr2=0x7f5a4c0b0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:49.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.755+0000 7f5a53600640 1 -- 172.21.15.86:0/1957595796 shutdown_connections 2024-04-25T14:16:49.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:49.755+0000 7f5a53600640 1 -- 172.21.15.86:0/1957595796 wait complete. 2024-04-25T14:16:50.176 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:50.176 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:50.650 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:16:50.654 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:16:51.178 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:51.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.339+0000 7fda48200640 1 Processor -- start 2024-04-25T14:16:51.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.339+0000 7fda48200640 1 -- start start 2024-04-25T14:16:51.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.339+0000 7fda48200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda400d16e0 0x7fda400d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:51.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.339+0000 7fda48200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fda400730b0 0x7fda400734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:51.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.339+0000 7fda48200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda400d2030 con 0x7fda400cfca0 2024-04-25T14:16:51.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.339+0000 7fda48200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fda40070470 con 0x7fda400730b0 2024-04-25T14:16:51.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.339+0000 7fda48200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fda40072bd0 con 0x7fda400d16e0 2024-04-25T14:16:51.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.339+0000 7fda45800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fda400cfca0 0x7fda4006f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52044/0 (socket says 172.21.15.86:52044) 2024-04-25T14:16:51.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.339+0000 7fda45800640 1 -- 172.21.15.86:0/2657308081 learned_addr learned my addr 172.21.15.86:0/2657308081 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:16:51.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.343+0000 7fda44e00640 1 --2- 172.21.15.86:0/2657308081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda400d16e0 0x7fda400d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:51.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.343+0000 7fda46200640 1 --2- 172.21.15.86:0/2657308081 >> v2:172.21.15.155:3300/0 conn(0x7fda400730b0 0x7fda400734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:51.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.343+0000 7fda3fe00640 1 -- 172.21.15.86:0/2657308081 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1913696828 0 0) 0x7fda34001120 con 0x7fda400cfca0 2024-04-25T14:16:51.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.343+0000 7fda3fe00640 1 -- 172.21.15.86:0/2657308081 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda28003530 con 0x7fda400cfca0 2024-04-25T14:16:51.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.343+0000 7fda3fe00640 1 -- 172.21.15.86:0/2657308081 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 455072654 0 0) 0x7fda28003530 con 0x7fda400cfca0 2024-04-25T14:16:51.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.343+0000 7fda3fe00640 1 -- 172.21.15.86:0/2657308081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda400d16e0 msgr2=0x7fda400d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:51.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.343+0000 7fda3fe00640 1 --2- 172.21.15.86:0/2657308081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda400d16e0 0x7fda400d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:51.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.343+0000 7fda3fe00640 1 -- 172.21.15.86:0/2657308081 >> v2:172.21.15.155:3300/0 conn(0x7fda400730b0 msgr2=0x7fda400734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:51.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.343+0000 7fda3fe00640 1 --2- 172.21.15.86:0/2657308081 >> v2:172.21.15.155:3300/0 conn(0x7fda400730b0 0x7fda400734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:51.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.343+0000 7fda3fe00640 1 -- 172.21.15.86:0/2657308081 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda4006fbd0 con 0x7fda400cfca0 2024-04-25T14:16:51.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.343+0000 7fda3fe00640 1 -- 172.21.15.86:0/2657308081 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fda34001850 con 0x7fda400cfca0 2024-04-25T14:16:51.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.343+0000 7fda3fe00640 1 -- 172.21.15.86:0/2657308081 >> v1:172.21.15.86:6789/0 conn(0x7fda400cfca0 legacy=0x7fda4006f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:51.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.343+0000 7fda3fe00640 1 --2- 172.21.15.86:0/2657308081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda28003a30 0x7fda28003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:51.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.343+0000 7fda3fe00640 1 --2- 172.21.15.86:0/2657308081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda28004810 0x7fda28004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:51.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.343+0000 7fda3fe00640 1 --2- 172.21.15.86:0/2657308081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda400d16e0 0x7fda28005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:51.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.343+0000 7fda3fe00640 1 -- 172.21.15.86:0/2657308081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fda40070470 con 0x7fda28003a30 2024-04-25T14:16:51.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.343+0000 7fda3fe00640 1 -- 172.21.15.86:0/2657308081 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fda40072bd0 con 0x7fda28004810 2024-04-25T14:16:51.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.343+0000 7fda3fe00640 1 -- 172.21.15.86:0/2657308081 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fda2800ba80 con 0x7fda400d16e0 2024-04-25T14:16:51.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.343+0000 7fda45800640 1 --2- 172.21.15.86:0/2657308081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda28003a30 0x7fda28003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:51.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.343+0000 7fda44e00640 1 --2- 172.21.15.86:0/2657308081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda28004810 0x7fda28004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:51.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.343+0000 7fda46200640 1 --2- 172.21.15.86:0/2657308081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda400d16e0 0x7fda28005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:51.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.343+0000 7fda45800640 1 -- 172.21.15.86:0/2657308081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda400d16e0 msgr2=0x7fda28005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:51.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.343+0000 7fda45800640 1 --2- 172.21.15.86:0/2657308081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda400d16e0 0x7fda28005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:51.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.343+0000 7fda45800640 1 -- 172.21.15.86:0/2657308081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda28004810 msgr2=0x7fda28004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:51.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.343+0000 7fda45800640 1 --2- 172.21.15.86:0/2657308081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda28004810 0x7fda28004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:51.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.343+0000 7fda45800640 1 -- 172.21.15.86:0/2657308081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda28003530 con 0x7fda28003a30 2024-04-25T14:16:51.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.347+0000 7fda45800640 1 --2- 172.21.15.86:0/2657308081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda28003a30 0x7fda28003ea0 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7fda3400c4c0 tx=0x7fda3401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4cba37eb78aaa7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:51.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.347+0000 7fda3fe00640 1 -- 172.21.15.86:0/2657308081 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3400c0e0 con 0x7fda28003a30 2024-04-25T14:16:51.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.347+0000 7fda3fe00640 1 -- 172.21.15.86:0/2657308081 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda340a9020 con 0x7fda28003a30 2024-04-25T14:16:51.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.347+0000 7fda3fe00640 1 -- 172.21.15.86:0/2657308081 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34005ae0 con 0x7fda28003a30 2024-04-25T14:16:51.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.347+0000 7fda48200640 1 -- 172.21.15.86:0/2657308081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda28003a30 msgr2=0x7fda28003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:51.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.347+0000 7fda48200640 1 --2- 172.21.15.86:0/2657308081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda28003a30 0x7fda28003ea0 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7fda3400c4c0 tx=0x7fda3401cc80 comp rx=0 tx=0).stop 2024-04-25T14:16:51.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.347+0000 7fda46200640 1 -- 172.21.15.86:0/2657308081 reap_dead start 2024-04-25T14:16:51.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.347+0000 7fda48200640 1 -- 172.21.15.86:0/2657308081 shutdown_connections 2024-04-25T14:16:51.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.347+0000 7fda48200640 1 -- 172.21.15.86:0/2657308081 >> 172.21.15.86:0/2657308081 conn(0x7fda400b4650 msgr2=0x7fda400b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:51.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.347+0000 7fda48200640 1 -- 172.21.15.86:0/2657308081 shutdown_connections 2024-04-25T14:16:51.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.347+0000 7fda48200640 1 -- 172.21.15.86:0/2657308081 wait complete. 2024-04-25T14:16:51.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.347+0000 7fda48200640 1 Processor -- start 2024-04-25T14:16:51.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.347+0000 7fda48200640 1 -- start start 2024-04-25T14:16:51.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.351+0000 7fda48200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda28003a30 0x7fda400b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:51.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.351+0000 7fda48200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda400b3970 0x7fda400bfb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:51.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.351+0000 7fda48200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda40156370 0x7fda40156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:51.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.351+0000 7fda48200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fda40167a60 con 0x7fda40156370 2024-04-25T14:16:51.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.351+0000 7fda48200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fda401678e0 con 0x7fda400b3970 2024-04-25T14:16:51.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.351+0000 7fda48200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fda40167be0 con 0x7fda28003a30 2024-04-25T14:16:51.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.351+0000 7fda46200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda40156370 0x7fda40156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:51.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.351+0000 7fda46200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda40156370 0x7fda40156780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40110/0 (socket says 172.21.15.86:40110) 2024-04-25T14:16:51.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.351+0000 7fda46200640 1 -- 172.21.15.86:0/1488311752 learned_addr learned my addr 172.21.15.86:0/1488311752 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:51.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.351+0000 7fda45800640 1 --2- 172.21.15.86:0/1488311752 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda28003a30 0x7fda400b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:51.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.351+0000 7fda44e00640 1 --2- 172.21.15.86:0/1488311752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda400b3970 0x7fda400bfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:51.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.351+0000 7fda46200640 1 -- 172.21.15.86:0/1488311752 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda28003a30 msgr2=0x7fda400b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:51.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.351+0000 7fda46200640 1 --2- 172.21.15.86:0/1488311752 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda28003a30 0x7fda400b0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:51.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.351+0000 7fda46200640 1 -- 172.21.15.86:0/1488311752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda400b3970 msgr2=0x7fda400bfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:51.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.351+0000 7fda46200640 1 --2- 172.21.15.86:0/1488311752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda400b3970 0x7fda400bfb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:51.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.351+0000 7fda46200640 1 -- 172.21.15.86:0/1488311752 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda400b1840 con 0x7fda40156370 2024-04-25T14:16:51.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.351+0000 7fda46200640 1 --2- 172.21.15.86:0/1488311752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda40156370 0x7fda40156780 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7fda400729e0 tx=0x7fda3800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:51.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.351+0000 7fda3d600640 1 -- 172.21.15.86:0/1488311752 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38011e60 con 0x7fda40156370 2024-04-25T14:16:51.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.351+0000 7fda3d600640 1 -- 172.21.15.86:0/1488311752 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda3801c070 con 0x7fda40156370 2024-04-25T14:16:51.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.351+0000 7fda3d600640 1 -- 172.21.15.86:0/1488311752 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38003bc0 con 0x7fda40156370 2024-04-25T14:16:51.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.351+0000 7fda48200640 1 -- 172.21.15.86:0/1488311752 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda400b1b20 con 0x7fda40156370 2024-04-25T14:16:51.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.351+0000 7fda48200640 1 -- 172.21.15.86:0/1488311752 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda400b3170 con 0x7fda40156370 2024-04-25T14:16:51.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.351+0000 7fda48200640 1 -- 172.21.15.86:0/1488311752 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda400b15b0 con 0x7fda40156370 2024-04-25T14:16:51.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.355+0000 7fda3d600640 1 -- 172.21.15.86:0/1488311752 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7fda38004030 con 0x7fda40156370 2024-04-25T14:16:51.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.355+0000 7fda3d600640 1 --2- 172.21.15.86:0/1488311752 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fda2006f480 0x7fda20071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:51.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.355+0000 7fda3d600640 1 -- 172.21.15.86:0/1488311752 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fda3808fd70 con 0x7fda40156370 2024-04-25T14:16:51.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.355+0000 7fda45800640 1 --2- 172.21.15.86:0/1488311752 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fda2006f480 0x7fda20071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:51.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.355+0000 7fda45800640 1 --2- 172.21.15.86:0/1488311752 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fda2006f480 0x7fda20071990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fda3400c490 tx=0x7fda34001c20 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:51.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.359+0000 7fda3d600640 1 -- 172.21.15.86:0/1488311752 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fda38010040 con 0x7fda40156370 2024-04-25T14:16:51.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.555+0000 7fda48200640 1 -- 172.21.15.86:0/1488311752 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda40069d00 con 0x7fda40156370 2024-04-25T14:16:51.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.555+0000 7fda3d600640 1 -- 172.21.15.86:0/1488311752 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7fda3805c970 con 0x7fda40156370 2024-04-25T14:16:51.552 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:51.552 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:51.553 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:16:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.555+0000 7fda48200640 1 -- 172.21.15.86:0/1488311752 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fda2006f480 msgr2=0x7fda20071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.555+0000 7fda48200640 1 --2- 172.21.15.86:0/1488311752 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fda2006f480 0x7fda20071990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fda3400c490 tx=0x7fda34001c20 comp rx=0 tx=0).stop 2024-04-25T14:16:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.555+0000 7fda48200640 1 -- 172.21.15.86:0/1488311752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda40156370 msgr2=0x7fda40156780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.555+0000 7fda48200640 1 --2- 172.21.15.86:0/1488311752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda40156370 0x7fda40156780 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7fda400729e0 tx=0x7fda3800f580 comp rx=0 tx=0).stop 2024-04-25T14:16:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.555+0000 7fda48200640 1 -- 172.21.15.86:0/1488311752 shutdown_connections 2024-04-25T14:16:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.555+0000 7fda48200640 1 --2- 172.21.15.86:0/1488311752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda40156370 0x7fda40156780 unknown :-1 s=CLOSED pgs=2316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.555+0000 7fda48200640 1 --2- 172.21.15.86:0/1488311752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda400b3970 0x7fda400bfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.555+0000 7fda48200640 1 --2- 172.21.15.86:0/1488311752 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fda2006f480 0x7fda20071990 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:51.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.555+0000 7fda48200640 1 --2- 172.21.15.86:0/1488311752 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda28003a30 0x7fda400b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:51.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.555+0000 7fda48200640 1 -- 172.21.15.86:0/1488311752 >> 172.21.15.86:0/1488311752 conn(0x7fda400b4650 msgr2=0x7fda400cb170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:51.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.555+0000 7fda48200640 1 -- 172.21.15.86:0/1488311752 shutdown_connections 2024-04-25T14:16:51.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:51.555+0000 7fda48200640 1 -- 172.21.15.86:0/1488311752 wait complete. 2024-04-25T14:16:51.976 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:51.976 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:52.978 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:53.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.135+0000 7fc258a00640 1 Processor -- start 2024-04-25T14:16:53.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.135+0000 7fc258a00640 1 -- start start 2024-04-25T14:16:53.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc258a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc2540d1df0 0x7fc2540cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:53.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc258a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2540cdcc0 0x7fc2540ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:53.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc258a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc25415e5c0 con 0x7fc254070330 2024-04-25T14:16:53.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc258a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc25406fd40 con 0x7fc2540d1df0 2024-04-25T14:16:53.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc258a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc25415e8a0 con 0x7fc2540cdcc0 2024-04-25T14:16:53.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc251e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc254070330 0x7fc2540d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52054/0 (socket says 172.21.15.86:52054) 2024-04-25T14:16:53.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc251e00640 1 -- 172.21.15.86:0/3871789819 learned_addr learned my addr 172.21.15.86:0/3871789819 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:16:53.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc249400640 1 --2- 172.21.15.86:0/3871789819 >> v2:172.21.15.155:3300/0 conn(0x7fc2540d1df0 0x7fc2540cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:53.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc252800640 1 --2- 172.21.15.86:0/3871789819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2540cdcc0 0x7fc2540ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:53.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc251400640 1 -- 172.21.15.86:0/3871789819 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3734940772 0 0) 0x7fc25415e5c0 con 0x7fc254070330 2024-04-25T14:16:53.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc251400640 1 -- 172.21.15.86:0/3871789819 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc238003530 con 0x7fc254070330 2024-04-25T14:16:53.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc251400640 1 -- 172.21.15.86:0/3871789819 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2225689105 0 0) 0x7fc238003530 con 0x7fc254070330 2024-04-25T14:16:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc251400640 1 -- 172.21.15.86:0/3871789819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2540cdcc0 msgr2=0x7fc2540ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc251400640 1 --2- 172.21.15.86:0/3871789819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2540cdcc0 0x7fc2540ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc251400640 1 -- 172.21.15.86:0/3871789819 >> v2:172.21.15.155:3300/0 conn(0x7fc2540d1df0 msgr2=0x7fc2540cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc251400640 1 --2- 172.21.15.86:0/3871789819 >> v2:172.21.15.155:3300/0 conn(0x7fc2540d1df0 0x7fc2540cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc251400640 1 -- 172.21.15.86:0/3871789819 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc25415eb80 con 0x7fc254070330 2024-04-25T14:16:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc251400640 1 -- 172.21.15.86:0/3871789819 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fc23c001680 con 0x7fc254070330 2024-04-25T14:16:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc251400640 1 -- 172.21.15.86:0/3871789819 >> v1:172.21.15.86:6789/0 conn(0x7fc254070330 legacy=0x7fc2540d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc251400640 1 --2- 172.21.15.86:0/3871789819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc238003a30 0x7fc238003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc251400640 1 --2- 172.21.15.86:0/3871789819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2540cdcc0 0x7fc238004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc251400640 1 --2- 172.21.15.86:0/3871789819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc238004d50 0x7fc238005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc251400640 1 -- 172.21.15.86:0/3871789819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc25415e8a0 con 0x7fc238004d50 2024-04-25T14:16:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc251400640 1 -- 172.21.15.86:0/3871789819 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc23800ba80 con 0x7fc238003a30 2024-04-25T14:16:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc251400640 1 -- 172.21.15.86:0/3871789819 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc23800bc80 con 0x7fc2540cdcc0 2024-04-25T14:16:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc252800640 1 --2- 172.21.15.86:0/3871789819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc238004d50 0x7fc238005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc251e00640 1 --2- 172.21.15.86:0/3871789819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc238003a30 0x7fc238003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc249400640 1 --2- 172.21.15.86:0/3871789819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2540cdcc0 0x7fc238004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc252800640 1 -- 172.21.15.86:0/3871789819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2540cdcc0 msgr2=0x7fc238004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc252800640 1 --2- 172.21.15.86:0/3871789819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2540cdcc0 0x7fc238004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc252800640 1 -- 172.21.15.86:0/3871789819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc238003a30 msgr2=0x7fc238003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc252800640 1 --2- 172.21.15.86:0/3871789819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc238003a30 0x7fc238003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.139+0000 7fc252800640 1 -- 172.21.15.86:0/3871789819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc238003530 con 0x7fc238004d50 2024-04-25T14:16:53.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.143+0000 7fc252800640 1 --2- 172.21.15.86:0/3871789819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc238004d50 0x7fc238005160 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7fc24c009fc0 tx=0x7fc24c01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a584fefee6f0a809 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:53.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.143+0000 7fc251400640 1 -- 172.21.15.86:0/3871789819 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c0b0070 con 0x7fc238004d50 2024-04-25T14:16:53.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.143+0000 7fc251400640 1 -- 172.21.15.86:0/3871789819 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc24c0ab600 con 0x7fc238004d50 2024-04-25T14:16:53.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.143+0000 7fc251400640 1 -- 172.21.15.86:0/3871789819 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c009c10 con 0x7fc238004d50 2024-04-25T14:16:53.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.143+0000 7fc258a00640 1 -- 172.21.15.86:0/3871789819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc238004d50 msgr2=0x7fc238005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:53.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.143+0000 7fc258a00640 1 --2- 172.21.15.86:0/3871789819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc238004d50 0x7fc238005160 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7fc24c009fc0 tx=0x7fc24c01c950 comp rx=0 tx=0).stop 2024-04-25T14:16:53.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.143+0000 7fc252800640 1 -- 172.21.15.86:0/3871789819 reap_dead start 2024-04-25T14:16:53.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.143+0000 7fc258a00640 1 -- 172.21.15.86:0/3871789819 shutdown_connections 2024-04-25T14:16:53.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.143+0000 7fc258a00640 1 -- 172.21.15.86:0/3871789819 >> 172.21.15.86:0/3871789819 conn(0x7fc2540b4650 msgr2=0x7fc2540b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:53.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.143+0000 7fc258a00640 1 -- 172.21.15.86:0/3871789819 shutdown_connections 2024-04-25T14:16:53.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.143+0000 7fc258a00640 1 -- 172.21.15.86:0/3871789819 wait complete. 2024-04-25T14:16:53.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.147+0000 7fc258a00640 1 Processor -- start 2024-04-25T14:16:53.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.147+0000 7fc258a00640 1 -- start start 2024-04-25T14:16:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.147+0000 7fc258a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc25415c250 0x7fc2541e6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.147+0000 7fc258a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc25409a790 0x7fc2541e87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.147+0000 7fc258a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc25409aba0 0x7fc2541e8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.147+0000 7fc258a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc2540708c0 con 0x7fc25415c250 2024-04-25T14:16:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.147+0000 7fc258a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc254070740 con 0x7fc25409a790 2024-04-25T14:16:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.147+0000 7fc258a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc2540ce8f0 con 0x7fc25409aba0 2024-04-25T14:16:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.147+0000 7fc251e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc25415c250 0x7fc2541e6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.147+0000 7fc251e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc25415c250 0x7fc2541e6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40134/0 (socket says 172.21.15.86:40134) 2024-04-25T14:16:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.147+0000 7fc251e00640 1 -- 172.21.15.86:0/3461962268 learned_addr learned my addr 172.21.15.86:0/3461962268 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.147+0000 7fc249400640 1 --2- 172.21.15.86:0/3461962268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc25409a790 0x7fc2541e87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.147+0000 7fc252800640 1 --2- 172.21.15.86:0/3461962268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc25409aba0 0x7fc2541e8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.147+0000 7fc251e00640 1 -- 172.21.15.86:0/3461962268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc25409aba0 msgr2=0x7fc2541e8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.147+0000 7fc251e00640 1 --2- 172.21.15.86:0/3461962268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc25409aba0 0x7fc2541e8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.147+0000 7fc251e00640 1 -- 172.21.15.86:0/3461962268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc25409a790 msgr2=0x7fc2541e87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.147+0000 7fc251e00640 1 --2- 172.21.15.86:0/3461962268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc25409a790 0x7fc2541e87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.147+0000 7fc251e00640 1 -- 172.21.15.86:0/3461962268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2540c9680 con 0x7fc25415c250 2024-04-25T14:16:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.151+0000 7fc251e00640 1 --2- 172.21.15.86:0/3461962268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc25415c250 0x7fc2541e6c60 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7fc23c0017f0 tx=0x7fc23c012770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.151+0000 7fc24aa00640 1 -- 172.21.15.86:0/3461962268 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c01c070 con 0x7fc25415c250 2024-04-25T14:16:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.151+0000 7fc24aa00640 1 -- 172.21.15.86:0/3461962268 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc23c014750 con 0x7fc25415c250 2024-04-25T14:16:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.151+0000 7fc24aa00640 1 -- 172.21.15.86:0/3461962268 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c014b40 con 0x7fc25415c250 2024-04-25T14:16:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.151+0000 7fc258a00640 1 -- 172.21.15.86:0/3461962268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2541e71a0 con 0x7fc25415c250 2024-04-25T14:16:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.151+0000 7fc258a00640 1 -- 172.21.15.86:0/3461962268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2540c9c00 con 0x7fc25415c250 2024-04-25T14:16:53.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.151+0000 7fc258a00640 1 -- 172.21.15.86:0/3461962268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc254069d00 con 0x7fc25415c250 2024-04-25T14:16:53.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.151+0000 7fc24aa00640 1 -- 172.21.15.86:0/3461962268 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7fc23c01f070 con 0x7fc25415c250 2024-04-25T14:16:53.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.155+0000 7fc24aa00640 1 --2- 172.21.15.86:0/3461962268 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc23006f450 0x7fc230071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:53.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.155+0000 7fc24aa00640 1 -- 172.21.15.86:0/3461962268 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fc23c091f10 con 0x7fc25415c250 2024-04-25T14:16:53.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.155+0000 7fc249400640 1 --2- 172.21.15.86:0/3461962268 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc23006f450 0x7fc230071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:53.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.155+0000 7fc249400640 1 --2- 172.21.15.86:0/3461962268 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc23006f450 0x7fc230071960 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc2541fa120 tx=0x7fc244008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:53.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.159+0000 7fc24aa00640 1 -- 172.21.15.86:0/3461962268 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc23c05eba0 con 0x7fc25415c250 2024-04-25T14:16:53.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.351+0000 7fc258a00640 1 -- 172.21.15.86:0/3461962268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc254076bf0 con 0x7fc25415c250 2024-04-25T14:16:53.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.351+0000 7fc24aa00640 1 -- 172.21.15.86:0/3461962268 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7fc23c05e4a0 con 0x7fc25415c250 2024-04-25T14:16:53.349 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:53.350 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:53.350 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:16:53.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.355+0000 7fc258a00640 1 -- 172.21.15.86:0/3461962268 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc23006f450 msgr2=0x7fc230071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:53.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.355+0000 7fc258a00640 1 --2- 172.21.15.86:0/3461962268 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc23006f450 0x7fc230071960 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc2541fa120 tx=0x7fc244008040 comp rx=0 tx=0).stop 2024-04-25T14:16:53.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.355+0000 7fc258a00640 1 -- 172.21.15.86:0/3461962268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc25415c250 msgr2=0x7fc2541e6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:53.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.355+0000 7fc258a00640 1 --2- 172.21.15.86:0/3461962268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc25415c250 0x7fc2541e6c60 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7fc23c0017f0 tx=0x7fc23c012770 comp rx=0 tx=0).stop 2024-04-25T14:16:53.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.355+0000 7fc258a00640 1 -- 172.21.15.86:0/3461962268 shutdown_connections 2024-04-25T14:16:53.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.355+0000 7fc258a00640 1 --2- 172.21.15.86:0/3461962268 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc23006f450 0x7fc230071960 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:53.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.355+0000 7fc258a00640 1 --2- 172.21.15.86:0/3461962268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc25409aba0 0x7fc2541e8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:53.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.355+0000 7fc258a00640 1 --2- 172.21.15.86:0/3461962268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc25409a790 0x7fc2541e87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:53.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.355+0000 7fc258a00640 1 --2- 172.21.15.86:0/3461962268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc25415c250 0x7fc2541e6c60 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:53.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.355+0000 7fc258a00640 1 -- 172.21.15.86:0/3461962268 >> 172.21.15.86:0/3461962268 conn(0x7fc2540b4650 msgr2=0x7fc2540730a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:53.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.355+0000 7fc258a00640 1 -- 172.21.15.86:0/3461962268 shutdown_connections 2024-04-25T14:16:53.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:53.355+0000 7fc258a00640 1 -- 172.21.15.86:0/3461962268 wait complete. 2024-04-25T14:16:53.774 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:53.775 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:54.776 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:54.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.935+0000 7f34a7e00640 1 Processor -- start 2024-04-25T14:16:54.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.935+0000 7f34a7e00640 1 -- start start 2024-04-25T14:16:54.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.939+0000 7f34a7e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34a80d08c0 0x7f34a815c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:54.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.939+0000 7f34a7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f34a806e830 0x7f34a806ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:54.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.939+0000 7f34a7e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34a806f180 con 0x7f34a80cf380 2024-04-25T14:16:54.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.939+0000 7f34a7e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f34a80af140 con 0x7f34a806e830 2024-04-25T14:16:54.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.939+0000 7f34a7e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f34a806d120 con 0x7f34a80d08c0 2024-04-25T14:16:54.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.939+0000 7f34a6000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f34a806e830 0x7f34a806ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:54.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.939+0000 7f34a6000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f34a806e830 0x7f34a806ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40140/0 (socket says 172.21.15.86:40140) 2024-04-25T14:16:54.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.939+0000 7f34a6000640 1 -- 172.21.15.86:0/3367356310 learned_addr learned my addr 172.21.15.86:0/3367356310 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:54.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.939+0000 7f34a6a00640 1 --2- 172.21.15.86:0/3367356310 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34a80d08c0 0x7f34a815c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:54.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.939+0000 7f34a6000640 1 -- 172.21.15.86:0/3367356310 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34a80d08c0 msgr2=0x7f34a815c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:54.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.939+0000 7f34a6000640 1 --2- 172.21.15.86:0/3367356310 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34a80d08c0 0x7f34a815c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:54.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.939+0000 7f34a6000640 1 -- 172.21.15.86:0/3367356310 >> v1:172.21.15.86:6789/0 conn(0x7f34a80cf380 legacy=0x7f34a80cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:54.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.939+0000 7f34a6000640 1 -- 172.21.15.86:0/3367356310 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a806d400 con 0x7f34a806e830 2024-04-25T14:16:54.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.943+0000 7f34a6000640 1 --2- 172.21.15.86:0/3367356310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34a806e830 0x7f34a806ec40 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f3494009870 tx=0x7f349401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201ae48a8ab993c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:54.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.943+0000 7f34a5600640 1 -- 172.21.15.86:0/3367356310 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34940b2070 con 0x7f34a806e830 2024-04-25T14:16:54.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.943+0000 7f34a5600640 1 -- 172.21.15.86:0/3367356310 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34940a9070 con 0x7f34a806e830 2024-04-25T14:16:54.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.943+0000 7f34a5600640 1 -- 172.21.15.86:0/3367356310 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34940ad880 con 0x7f34a806e830 2024-04-25T14:16:54.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.943+0000 7f34a7e00640 1 -- 172.21.15.86:0/3367356310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34a806e830 msgr2=0x7f34a806ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:54.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.943+0000 7f34a7e00640 1 --2- 172.21.15.86:0/3367356310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34a806e830 0x7f34a806ec40 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f3494009870 tx=0x7f349401c560 comp rx=0 tx=0).stop 2024-04-25T14:16:54.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.943+0000 7f34a7e00640 1 -- 172.21.15.86:0/3367356310 shutdown_connections 2024-04-25T14:16:54.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.943+0000 7f34a7e00640 1 --2- 172.21.15.86:0/3367356310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34a806e830 0x7f34a806ec40 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:54.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.943+0000 7f34a7e00640 1 --2- 172.21.15.86:0/3367356310 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34a80d08c0 0x7f34a815c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:54.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.943+0000 7f34a7e00640 1 -- 172.21.15.86:0/3367356310 >> 172.21.15.86:0/3367356310 conn(0x7f34a80b4850 msgr2=0x7f34a80b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:54.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.943+0000 7f34a7e00640 1 -- 172.21.15.86:0/3367356310 shutdown_connections 2024-04-25T14:16:54.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.943+0000 7f34a7e00640 1 -- 172.21.15.86:0/3367356310 wait complete. 2024-04-25T14:16:54.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.947+0000 7f34a7e00640 1 Processor -- start 2024-04-25T14:16:54.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.947+0000 7f34a7e00640 1 -- start start 2024-04-25T14:16:54.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.947+0000 7f34a7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34a806e830 0x7f34a815da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:54.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.947+0000 7f34a7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34a80cf380 0x7f34a8151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:54.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.947+0000 7f34a7e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34a80d08c0 0x7f34a8073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:54.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.947+0000 7f34a7e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34a806f580 con 0x7f34a806e830 2024-04-25T14:16:54.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.947+0000 7f34a7e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f34a806dee0 con 0x7f34a80cf380 2024-04-25T14:16:54.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.947+0000 7f34a7e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f34a8167d80 con 0x7f34a80d08c0 2024-04-25T14:16:54.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.947+0000 7f34a6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34a806e830 0x7f34a815da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:54.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.947+0000 7f34a6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34a806e830 0x7f34a815da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40150/0 (socket says 172.21.15.86:40150) 2024-04-25T14:16:54.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.947+0000 7f34a6a00640 1 -- 172.21.15.86:0/3308428138 learned_addr learned my addr 172.21.15.86:0/3308428138 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:54.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.947+0000 7f34a6000640 1 --2- 172.21.15.86:0/3308428138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34a80cf380 0x7f34a8151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:54.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.947+0000 7f34a7400640 1 --2- 172.21.15.86:0/3308428138 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34a80d08c0 0x7f34a8073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:54.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.947+0000 7f34a6a00640 1 -- 172.21.15.86:0/3308428138 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34a80d08c0 msgr2=0x7f34a8073b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:54.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.947+0000 7f34a6a00640 1 --2- 172.21.15.86:0/3308428138 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34a80d08c0 0x7f34a8073b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:54.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.947+0000 7f34a6a00640 1 -- 172.21.15.86:0/3308428138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34a80cf380 msgr2=0x7f34a8151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:54.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.947+0000 7f34a6a00640 1 --2- 172.21.15.86:0/3308428138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34a80cf380 0x7f34a8151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:54.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.947+0000 7f34a6a00640 1 -- 172.21.15.86:0/3308428138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a815d1b0 con 0x7f34a806e830 2024-04-25T14:16:54.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.947+0000 7f34a6000640 1 --2- 172.21.15.86:0/3308428138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34a80cf380 0x7f34a8151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:16:54.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.947+0000 7f34a7400640 1 --2- 172.21.15.86:0/3308428138 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34a80d08c0 0x7f34a8073b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:16:54.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.947+0000 7f34a6a00640 1 --2- 172.21.15.86:0/3308428138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34a806e830 0x7f34a815da40 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f349800e840 tx=0x7f349800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:54.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.947+0000 7f349ea00640 1 -- 172.21.15.86:0/3308428138 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498017070 con 0x7f34a806e830 2024-04-25T14:16:54.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.947+0000 7f349ea00640 1 -- 172.21.15.86:0/3308428138 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f349800ee90 con 0x7f34a806e830 2024-04-25T14:16:54.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.947+0000 7f34a7e00640 1 -- 172.21.15.86:0/3308428138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a815d450 con 0x7f34a806e830 2024-04-25T14:16:54.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.947+0000 7f349ea00640 1 -- 172.21.15.86:0/3308428138 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349800cb60 con 0x7f34a806e830 2024-04-25T14:16:54.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.947+0000 7f34a7e00640 1 -- 172.21.15.86:0/3308428138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a8168a00 con 0x7f34a806e830 2024-04-25T14:16:54.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.951+0000 7f34a7e00640 1 -- 172.21.15.86:0/3308428138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a8076b80 con 0x7f34a806e830 2024-04-25T14:16:54.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.951+0000 7f349ea00640 1 -- 172.21.15.86:0/3308428138 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f349801a070 con 0x7f34a806e830 2024-04-25T14:16:54.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.951+0000 7f349ea00640 1 --2- 172.21.15.86:0/3308428138 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f348806f480 0x7f3488071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:54.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.951+0000 7f349ea00640 1 -- 172.21.15.86:0/3308428138 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f349808f500 con 0x7f34a806e830 2024-04-25T14:16:54.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.951+0000 7f34a6000640 1 --2- 172.21.15.86:0/3308428138 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f348806f480 0x7f3488071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:54.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.951+0000 7f34a6000640 1 --2- 172.21.15.86:0/3308428138 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f348806f480 0x7f3488071990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f34940af000 tx=0x7f3494008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:54.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:54.955+0000 7f349ea00640 1 -- 172.21.15.86:0/3308428138 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f349805c070 con 0x7f34a806e830 2024-04-25T14:16:55.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:55.143+0000 7f34a7e00640 1 -- 172.21.15.86:0/3308428138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34a80d20f0 con 0x7f34a806e830 2024-04-25T14:16:55.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:55.143+0000 7f349ea00640 1 -- 172.21.15.86:0/3308428138 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f349805b970 con 0x7f34a806e830 2024-04-25T14:16:55.143 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:55.143 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:55.144 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:16:55.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:55.147+0000 7f34a7e00640 1 -- 172.21.15.86:0/3308428138 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f348806f480 msgr2=0x7f3488071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:55.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:55.147+0000 7f34a7e00640 1 --2- 172.21.15.86:0/3308428138 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f348806f480 0x7f3488071990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f34940af000 tx=0x7f3494008040 comp rx=0 tx=0).stop 2024-04-25T14:16:55.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:55.147+0000 7f34a7e00640 1 -- 172.21.15.86:0/3308428138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34a806e830 msgr2=0x7f34a815da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:55.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:55.147+0000 7f34a7e00640 1 --2- 172.21.15.86:0/3308428138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34a806e830 0x7f34a815da40 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f349800e840 tx=0x7f349800ec40 comp rx=0 tx=0).stop 2024-04-25T14:16:55.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:55.147+0000 7f34a7e00640 1 -- 172.21.15.86:0/3308428138 shutdown_connections 2024-04-25T14:16:55.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:55.147+0000 7f34a7e00640 1 --2- 172.21.15.86:0/3308428138 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f348806f480 0x7f3488071990 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:55.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:55.147+0000 7f34a7e00640 1 --2- 172.21.15.86:0/3308428138 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34a80d08c0 0x7f34a8073b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:55.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:55.147+0000 7f34a7e00640 1 --2- 172.21.15.86:0/3308428138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34a80cf380 0x7f34a8151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:55.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:55.147+0000 7f34a7e00640 1 --2- 172.21.15.86:0/3308428138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34a806e830 0x7f34a815da40 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:55.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:55.147+0000 7f34a7e00640 1 -- 172.21.15.86:0/3308428138 >> 172.21.15.86:0/3308428138 conn(0x7f34a80b4850 msgr2=0x7f34a8154e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:55.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:55.147+0000 7f34a7e00640 1 -- 172.21.15.86:0/3308428138 shutdown_connections 2024-04-25T14:16:55.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:55.147+0000 7f34a7e00640 1 -- 172.21.15.86:0/3308428138 wait complete. 2024-04-25T14:16:55.568 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:55.568 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:56.570 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:56.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.731+0000 7f9dd8600640 1 Processor -- start 2024-04-25T14:16:56.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.731+0000 7f9dd8600640 1 -- start start 2024-04-25T14:16:56.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.731+0000 7f9dd8600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9dd00cfca0 0x7f9dd006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:56.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.731+0000 7f9dd8600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9dd00d16e0 0x7f9dd00d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:56.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.731+0000 7f9dd8600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dd00d2030 con 0x7f9dd006c670 2024-04-25T14:16:56.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.731+0000 7f9dd8600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9dd0070470 con 0x7f9dd00cfca0 2024-04-25T14:16:56.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.731+0000 7f9dd8600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9dd0072bd0 con 0x7f9dd00d16e0 2024-04-25T14:16:56.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.735+0000 7f9dd5c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9dd00cfca0 0x7f9dd006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:56.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.735+0000 7f9dd5c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9dd00cfca0 0x7f9dd006f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40154/0 (socket says 172.21.15.86:40154) 2024-04-25T14:16:56.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.735+0000 7f9dd5c00640 1 -- 172.21.15.86:0/1048483428 learned_addr learned my addr 172.21.15.86:0/1048483428 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:56.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.735+0000 7f9dd5200640 1 --2- 172.21.15.86:0/1048483428 >> v2:172.21.15.155:3300/0 conn(0x7f9dd00d16e0 0x7f9dd00d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:56.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.735+0000 7f9dd5c00640 1 -- 172.21.15.86:0/1048483428 >> v2:172.21.15.155:3300/0 conn(0x7f9dd00d16e0 msgr2=0x7f9dd00d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:16:56.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.735+0000 7f9dd5c00640 1 --2- 172.21.15.86:0/1048483428 >> v2:172.21.15.155:3300/0 conn(0x7f9dd00d16e0 0x7f9dd00d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:56.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.735+0000 7f9dd5c00640 1 -- 172.21.15.86:0/1048483428 >> v1:172.21.15.86:6789/0 conn(0x7f9dd006c670 legacy=0x7f9dd006ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:56.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.735+0000 7f9dd5c00640 1 -- 172.21.15.86:0/1048483428 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd006f930 con 0x7f9dd00cfca0 2024-04-25T14:16:56.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.735+0000 7f9dd5c00640 1 --2- 172.21.15.86:0/1048483428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd00cfca0 0x7f9dd006f3f0 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00729c0 tx=0x7f9dc401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328b1d89cc54df09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:56.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.735+0000 7f9dcfe00640 1 -- 172.21.15.86:0/1048483428 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc40ab430 con 0x7f9dd00cfca0 2024-04-25T14:16:56.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.735+0000 7f9dcfe00640 1 -- 172.21.15.86:0/1048483428 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dc40b0070 con 0x7f9dd00cfca0 2024-04-25T14:16:56.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.735+0000 7f9dcfe00640 1 -- 172.21.15.86:0/1048483428 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc40ab7f0 con 0x7f9dd00cfca0 2024-04-25T14:16:56.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.739+0000 7f9dd8600640 1 -- 172.21.15.86:0/1048483428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd00cfca0 msgr2=0x7f9dd006f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:56.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.739+0000 7f9dd8600640 1 --2- 172.21.15.86:0/1048483428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd00cfca0 0x7f9dd006f3f0 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00729c0 tx=0x7f9dc401c520 comp rx=0 tx=0).stop 2024-04-25T14:16:56.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.739+0000 7f9dd8600640 1 -- 172.21.15.86:0/1048483428 shutdown_connections 2024-04-25T14:16:56.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.739+0000 7f9dd8600640 1 --2- 172.21.15.86:0/1048483428 >> v2:172.21.15.155:3300/0 conn(0x7f9dd00d16e0 0x7f9dd00d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:56.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.739+0000 7f9dd8600640 1 --2- 172.21.15.86:0/1048483428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd00cfca0 0x7f9dd006f3f0 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:56.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.739+0000 7f9dd8600640 1 -- 172.21.15.86:0/1048483428 >> 172.21.15.86:0/1048483428 conn(0x7f9dd00b4650 msgr2=0x7f9dd00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:56.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.739+0000 7f9dd8600640 1 -- 172.21.15.86:0/1048483428 shutdown_connections 2024-04-25T14:16:56.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.739+0000 7f9dd8600640 1 -- 172.21.15.86:0/1048483428 wait complete. 2024-04-25T14:16:56.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.739+0000 7f9dd8600640 1 Processor -- start 2024-04-25T14:16:56.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.739+0000 7f9dd8600640 1 -- start start 2024-04-25T14:16:56.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.743+0000 7f9dd8600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9dd006c670 0x7f9dd009a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:56.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.743+0000 7f9dd8600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dd00cfca0 0x7f9dd015ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:56.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.743+0000 7f9dd8600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd00d16e0 0x7f9dd015f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:56.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.743+0000 7f9dd8600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd016a070 con 0x7f9dd00d16e0 2024-04-25T14:16:56.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.743+0000 7f9dd8600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd0169ef0 con 0x7f9dd00cfca0 2024-04-25T14:16:56.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.743+0000 7f9dd8600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9dd016a1f0 con 0x7f9dd006c670 2024-04-25T14:16:56.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.743+0000 7f9dd6600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd00d16e0 0x7f9dd015f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:56.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.743+0000 7f9dd6600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd00d16e0 0x7f9dd015f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40158/0 (socket says 172.21.15.86:40158) 2024-04-25T14:16:56.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.743+0000 7f9dd6600640 1 -- 172.21.15.86:0/3451486697 learned_addr learned my addr 172.21.15.86:0/3451486697 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:56.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.743+0000 7f9dd5200640 1 --2- 172.21.15.86:0/3451486697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dd00cfca0 0x7f9dd015ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:56.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.743+0000 7f9dd5c00640 1 --2- 172.21.15.86:0/3451486697 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9dd006c670 0x7f9dd009a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:56.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.743+0000 7f9dd6600640 1 -- 172.21.15.86:0/3451486697 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9dd006c670 msgr2=0x7f9dd009a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:56.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.743+0000 7f9dd6600640 1 --2- 172.21.15.86:0/3451486697 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9dd006c670 0x7f9dd009a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:56.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.743+0000 7f9dd6600640 1 -- 172.21.15.86:0/3451486697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dd00cfca0 msgr2=0x7f9dd015ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:56.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.743+0000 7f9dd6600640 1 --2- 172.21.15.86:0/3451486697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dd00cfca0 0x7f9dd015ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:56.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.743+0000 7f9dd6600640 1 -- 172.21.15.86:0/3451486697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd009ae90 con 0x7f9dd00d16e0 2024-04-25T14:16:56.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.743+0000 7f9dd6600640 1 --2- 172.21.15.86:0/3451486697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd00d16e0 0x7f9dd015f390 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f9dc800c730 tx=0x7f9dc800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:56.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.743+0000 7f9dcd600640 1 -- 172.21.15.86:0/3451486697 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8003b80 con 0x7f9dd00d16e0 2024-04-25T14:16:56.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.743+0000 7f9dcd600640 1 -- 172.21.15.86:0/3451486697 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dc8003d20 con 0x7f9dd00d16e0 2024-04-25T14:16:56.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.743+0000 7f9dd8600640 1 -- 172.21.15.86:0/3451486697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd015dd00 con 0x7f9dd00d16e0 2024-04-25T14:16:56.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.743+0000 7f9dcd600640 1 -- 172.21.15.86:0/3451486697 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8005310 con 0x7f9dd00d16e0 2024-04-25T14:16:56.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.743+0000 7f9dd8600640 1 -- 172.21.15.86:0/3451486697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd015e2c0 con 0x7f9dd00d16e0 2024-04-25T14:16:56.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.743+0000 7f9dd8600640 1 -- 172.21.15.86:0/3451486697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dd009f4b0 con 0x7f9dd00d16e0 2024-04-25T14:16:56.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.747+0000 7f9dcd600640 1 -- 172.21.15.86:0/3451486697 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f9dc80056c0 con 0x7f9dd00d16e0 2024-04-25T14:16:56.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.747+0000 7f9dcd600640 1 --2- 172.21.15.86:0/3451486697 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9db006f450 0x7f9db0071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:56.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.747+0000 7f9dcd600640 1 -- 172.21.15.86:0/3451486697 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f9dc808f5c0 con 0x7f9dd00d16e0 2024-04-25T14:16:56.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.747+0000 7f9dd5c00640 1 --2- 172.21.15.86:0/3451486697 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9db006f450 0x7f9db0071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:56.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.747+0000 7f9dd5c00640 1 --2- 172.21.15.86:0/3451486697 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9db006f450 0x7f9db0071960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40af000 tx=0x7f9dc40a7000 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:56.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.751+0000 7f9dcd600640 1 -- 172.21.15.86:0/3451486697 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9dd00d2030 con 0x7f9dd00d16e0 2024-04-25T14:16:56.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.939+0000 7f9dd8600640 1 -- 172.21.15.86:0/3451486697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9dd0069d00 con 0x7f9dd00d16e0 2024-04-25T14:16:56.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.939+0000 7f9dcd600640 1 -- 172.21.15.86:0/3451486697 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f9dc805bae0 con 0x7f9dd00d16e0 2024-04-25T14:16:56.939 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:56.939 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:56.939 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:16:56.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.943+0000 7f9dd8600640 1 -- 172.21.15.86:0/3451486697 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9db006f450 msgr2=0x7f9db0071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:56.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.943+0000 7f9dd8600640 1 --2- 172.21.15.86:0/3451486697 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9db006f450 0x7f9db0071960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40af000 tx=0x7f9dc40a7000 comp rx=0 tx=0).stop 2024-04-25T14:16:56.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.943+0000 7f9dd8600640 1 -- 172.21.15.86:0/3451486697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd00d16e0 msgr2=0x7f9dd015f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:56.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.943+0000 7f9dd8600640 1 --2- 172.21.15.86:0/3451486697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd00d16e0 0x7f9dd015f390 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f9dc800c730 tx=0x7f9dc800cbf0 comp rx=0 tx=0).stop 2024-04-25T14:16:56.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.943+0000 7f9dd8600640 1 -- 172.21.15.86:0/3451486697 shutdown_connections 2024-04-25T14:16:56.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.943+0000 7f9dd8600640 1 --2- 172.21.15.86:0/3451486697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd00d16e0 0x7f9dd015f390 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:56.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.943+0000 7f9dd8600640 1 --2- 172.21.15.86:0/3451486697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dd00cfca0 0x7f9dd015ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:56.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.943+0000 7f9dd8600640 1 --2- 172.21.15.86:0/3451486697 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9db006f450 0x7f9db0071960 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:56.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.943+0000 7f9dd8600640 1 --2- 172.21.15.86:0/3451486697 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9dd006c670 0x7f9dd009a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:56.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.943+0000 7f9dd8600640 1 -- 172.21.15.86:0/3451486697 >> 172.21.15.86:0/3451486697 conn(0x7f9dd00b4650 msgr2=0x7f9dd0068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:56.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.943+0000 7f9dd8600640 1 -- 172.21.15.86:0/3451486697 shutdown_connections 2024-04-25T14:16:56.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:56.943+0000 7f9dd8600640 1 -- 172.21.15.86:0/3451486697 wait complete. 2024-04-25T14:16:57.363 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:57.364 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:16:58.364 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:16:58.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.527+0000 7fecd6400640 1 Processor -- start 2024-04-25T14:16:58.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.527+0000 7fecd6400640 1 -- start start 2024-04-25T14:16:58.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.527+0000 7fecd6400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fecd00d2ae0 0x7fecd00ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:58.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.527+0000 7fecd6400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecd00b3bd0 con 0x7fecd00ccab0 2024-04-25T14:16:58.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.527+0000 7fecd6400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecd00b1cf0 con 0x7fecd00b79c0 2024-04-25T14:16:58.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.527+0000 7fecd6400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fecd00b2620 con 0x7fecd00d2ae0 2024-04-25T14:16:58.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.527+0000 7feccf400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fecd00ccab0 0x7fecd00d23d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52078/0 (socket says 172.21.15.86:52078) 2024-04-25T14:16:58.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.527+0000 7feccf400640 1 -- 172.21.15.86:0/23221883 learned_addr learned my addr 172.21.15.86:0/23221883 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:16:58.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.527+0000 7feccea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fecd00d2ae0 0x7fecd00ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:58.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.527+0000 7feccea00640 1 -- 172.21.15.86:0/23221883 >> v1:172.21.15.155:6789/0 conn(0x7fecd00b79c0 legacy=0x7fecd00ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:58.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.527+0000 7feccea00640 1 -- 172.21.15.86:0/23221883 >> v1:172.21.15.86:6789/0 conn(0x7fecd00ccab0 legacy=0x7fecd00d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:58.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.527+0000 7feccea00640 1 -- 172.21.15.86:0/23221883 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd00b1fd0 con 0x7fecd00d2ae0 2024-04-25T14:16:58.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.531+0000 7feccea00640 1 --2- 172.21.15.86:0/23221883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecd00d2ae0 0x7fecd00ce180 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0097c0 tx=0x7fecbc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d5bacb1c5e015c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:58.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.531+0000 7fecce000640 1 -- 172.21.15.86:0/23221883 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc0b1070 con 0x7fecd00d2ae0 2024-04-25T14:16:58.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.531+0000 7fecce000640 1 -- 172.21.15.86:0/23221883 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecbc0ac410 con 0x7fecd00d2ae0 2024-04-25T14:16:58.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.531+0000 7fecce000640 1 -- 172.21.15.86:0/23221883 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc0ac7e0 con 0x7fecd00d2ae0 2024-04-25T14:16:58.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.531+0000 7fecd6400640 1 -- 172.21.15.86:0/23221883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecd00d2ae0 msgr2=0x7fecd00ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:58.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.531+0000 7fecd6400640 1 --2- 172.21.15.86:0/23221883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecd00d2ae0 0x7fecd00ce180 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0097c0 tx=0x7fecbc01c4a0 comp rx=0 tx=0).stop 2024-04-25T14:16:58.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.531+0000 7fecd6400640 1 -- 172.21.15.86:0/23221883 shutdown_connections 2024-04-25T14:16:58.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.531+0000 7fecd6400640 1 --2- 172.21.15.86:0/23221883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecd00d2ae0 0x7fecd00ce180 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:58.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.531+0000 7fecd6400640 1 -- 172.21.15.86:0/23221883 >> 172.21.15.86:0/23221883 conn(0x7fecd009ba30 msgr2=0x7fecd00b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:58.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.531+0000 7fecd6400640 1 -- 172.21.15.86:0/23221883 shutdown_connections 2024-04-25T14:16:58.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.531+0000 7fecd6400640 1 -- 172.21.15.86:0/23221883 wait complete. 2024-04-25T14:16:58.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.535+0000 7fecd6400640 1 Processor -- start 2024-04-25T14:16:58.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.535+0000 7fecd6400640 1 -- start start 2024-04-25T14:16:58.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.535+0000 7fecd6400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fecd00b79c0 0x7fecd00c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:58.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.535+0000 7fecd6400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecd00ccab0 0x7fecd00cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:58.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.535+0000 7fecd6400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecd00cb050 0x7fecd0155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:58.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.535+0000 7fecd6400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fecd016c530 con 0x7fecd00ccab0 2024-04-25T14:16:58.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.535+0000 7fecd6400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fecd016c3b0 con 0x7fecd00cb050 2024-04-25T14:16:58.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.535+0000 7fecd6400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fecd016c6b0 con 0x7fecd00b79c0 2024-04-25T14:16:58.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.535+0000 7feccea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecd00ccab0 0x7fecd00cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:58.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.535+0000 7feccea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecd00ccab0 0x7fecd00cab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40176/0 (socket says 172.21.15.86:40176) 2024-04-25T14:16:58.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.535+0000 7feccea00640 1 -- 172.21.15.86:0/1414146935 learned_addr learned my addr 172.21.15.86:0/1414146935 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:16:58.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.535+0000 7feccf400640 1 --2- 172.21.15.86:0/1414146935 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fecd00b79c0 0x7fecd00c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:58.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.535+0000 7feccfe00640 1 --2- 172.21.15.86:0/1414146935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecd00cb050 0x7fecd0155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:58.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.535+0000 7feccea00640 1 -- 172.21.15.86:0/1414146935 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fecd00b79c0 msgr2=0x7fecd00c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:58.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.535+0000 7feccea00640 1 --2- 172.21.15.86:0/1414146935 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fecd00b79c0 0x7fecd00c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:58.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.535+0000 7feccea00640 1 -- 172.21.15.86:0/1414146935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecd00cb050 msgr2=0x7fecd0155f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:58.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.535+0000 7feccea00640 1 --2- 172.21.15.86:0/1414146935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecd00cb050 0x7fecd0155f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:58.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.535+0000 7feccea00640 1 -- 172.21.15.86:0/1414146935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd00cb840 con 0x7fecd00ccab0 2024-04-25T14:16:58.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.535+0000 7feccf400640 1 --2- 172.21.15.86:0/1414146935 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fecd00b79c0 0x7fecd00c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:16:58.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.535+0000 7feccfe00640 1 --2- 172.21.15.86:0/1414146935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecd00cb050 0x7fecd0155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:16:58.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.539+0000 7feccea00640 1 --2- 172.21.15.86:0/1414146935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecd00ccab0 0x7fecd00cab10 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0af040 tx=0x7fecbc019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:58.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.539+0000 7fecc7400640 1 -- 172.21.15.86:0/1414146935 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc0b9020 con 0x7fecd00ccab0 2024-04-25T14:16:58.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.539+0000 7fecc7400640 1 -- 172.21.15.86:0/1414146935 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecbc0ace40 con 0x7fecd00ccab0 2024-04-25T14:16:58.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.539+0000 7fecc7400640 1 -- 172.21.15.86:0/1414146935 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc0b1040 con 0x7fecd00ccab0 2024-04-25T14:16:58.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.539+0000 7fecd6400640 1 -- 172.21.15.86:0/1414146935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecd0153d10 con 0x7fecd00ccab0 2024-04-25T14:16:58.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.539+0000 7fecd6400640 1 -- 172.21.15.86:0/1414146935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecd014acc0 con 0x7fecd00ccab0 2024-04-25T14:16:58.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.539+0000 7fecd6400640 1 -- 172.21.15.86:0/1414146935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecd00cb5b0 con 0x7fecd00ccab0 2024-04-25T14:16:58.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.539+0000 7fecc7400640 1 -- 172.21.15.86:0/1414146935 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7fecbc0bc020 con 0x7fecd00ccab0 2024-04-25T14:16:58.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.539+0000 7fecc7400640 1 --2- 172.21.15.86:0/1414146935 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fecb006f450 0x7fecb0071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:16:58.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.539+0000 7fecc7400640 1 -- 172.21.15.86:0/1414146935 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fecbc128c10 con 0x7fecd00ccab0 2024-04-25T14:16:58.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.543+0000 7feccf400640 1 --2- 172.21.15.86:0/1414146935 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fecb006f450 0x7fecb0071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:16:58.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.543+0000 7feccf400640 1 --2- 172.21.15.86:0/1414146935 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fecb006f450 0x7fecb0071960 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fecc00010f0 tx=0x7fecc0009040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:16:58.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.547+0000 7fecc7400640 1 -- 172.21.15.86:0/1414146935 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fecbc0c3070 con 0x7fecd00ccab0 2024-04-25T14:16:58.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.735+0000 7fecd6400640 1 -- 172.21.15.86:0/1414146935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fecd0076bf0 con 0x7fecd00ccab0 2024-04-25T14:16:58.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.735+0000 7fecc7400640 1 -- 172.21.15.86:0/1414146935 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7fecbc016090 con 0x7fecd00ccab0 2024-04-25T14:16:58.732 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:16:58.732 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:16:58.732 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:16:58.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.735+0000 7fecd6400640 1 -- 172.21.15.86:0/1414146935 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fecb006f450 msgr2=0x7fecb0071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:58.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.735+0000 7fecd6400640 1 --2- 172.21.15.86:0/1414146935 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fecb006f450 0x7fecb0071960 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fecc00010f0 tx=0x7fecc0009040 comp rx=0 tx=0).stop 2024-04-25T14:16:58.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.735+0000 7fecd6400640 1 -- 172.21.15.86:0/1414146935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecd00ccab0 msgr2=0x7fecd00cab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:16:58.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.735+0000 7fecd6400640 1 --2- 172.21.15.86:0/1414146935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecd00ccab0 0x7fecd00cab10 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0af040 tx=0x7fecbc019240 comp rx=0 tx=0).stop 2024-04-25T14:16:58.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.735+0000 7fecd6400640 1 -- 172.21.15.86:0/1414146935 shutdown_connections 2024-04-25T14:16:58.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.735+0000 7fecd6400640 1 --2- 172.21.15.86:0/1414146935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecd00cb050 0x7fecd0155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:58.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.735+0000 7fecd6400640 1 --2- 172.21.15.86:0/1414146935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecd00ccab0 0x7fecd00cab10 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:58.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.735+0000 7fecd6400640 1 --2- 172.21.15.86:0/1414146935 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fecb006f450 0x7fecb0071960 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:58.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.735+0000 7fecd6400640 1 --2- 172.21.15.86:0/1414146935 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fecd00b79c0 0x7fecd00c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:16:58.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.735+0000 7fecd6400640 1 -- 172.21.15.86:0/1414146935 >> 172.21.15.86:0/1414146935 conn(0x7fecd009ba30 msgr2=0x7fecd006fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:16:58.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.735+0000 7fecd6400640 1 -- 172.21.15.86:0/1414146935 shutdown_connections 2024-04-25T14:16:58.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:16:58.735+0000 7fecd6400640 1 -- 172.21.15.86:0/1414146935 wait complete. 2024-04-25T14:16:59.157 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:16:59.157 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:00.158 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:00.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.319+0000 7fc4da600640 1 Processor -- start 2024-04-25T14:17:00.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.319+0000 7fc4da600640 1 -- start start 2024-04-25T14:17:00.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.323+0000 7fc4da600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc4d40d1df0 0x7fc4d40cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:00.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.323+0000 7fc4da600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4d415e5c0 con 0x7fc4d40cdcc0 2024-04-25T14:17:00.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.323+0000 7fc4da600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4d415e8a0 con 0x7fc4d4070330 2024-04-25T14:17:00.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.323+0000 7fc4da600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc4d406fd40 con 0x7fc4d40d1df0 2024-04-25T14:17:00.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.323+0000 7fc4cbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc4d40d1df0 0x7fc4d40cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:00.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.323+0000 7fc4cbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc4d40d1df0 0x7fc4d40cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42066/0 (socket says 172.21.15.86:42066) 2024-04-25T14:17:00.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.323+0000 7fc4cbe00640 1 -- 172.21.15.86:0/1306997061 learned_addr learned my addr 172.21.15.86:0/1306997061 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:00.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.323+0000 7fc4cbe00640 1 -- 172.21.15.86:0/1306997061 >> v1:172.21.15.155:6789/0 conn(0x7fc4d4070330 legacy=0x7fc4d40d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:00.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.323+0000 7fc4cbe00640 1 -- 172.21.15.86:0/1306997061 >> v1:172.21.15.86:6789/0 conn(0x7fc4d40cdcc0 legacy=0x7fc4d40ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:00.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.323+0000 7fc4cbe00640 1 -- 172.21.15.86:0/1306997061 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d415eb80 con 0x7fc4d40d1df0 2024-04-25T14:17:00.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.323+0000 7fc4cbe00640 1 --2- 172.21.15.86:0/1306997061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4d40d1df0 0x7fc4d40cd780 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0098a0 tx=0x7fc4bc01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2805e990183adffa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:00.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.323+0000 7fc4d2a00640 1 -- 172.21.15.86:0/1306997061 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc0b0070 con 0x7fc4d40d1df0 2024-04-25T14:17:00.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.323+0000 7fc4d2a00640 1 -- 172.21.15.86:0/1306997061 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4bc004230 con 0x7fc4d40d1df0 2024-04-25T14:17:00.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.323+0000 7fc4d2a00640 1 -- 172.21.15.86:0/1306997061 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc003620 con 0x7fc4d40d1df0 2024-04-25T14:17:00.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.327+0000 7fc4da600640 1 -- 172.21.15.86:0/1306997061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4d40d1df0 msgr2=0x7fc4d40cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:00.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.327+0000 7fc4da600640 1 --2- 172.21.15.86:0/1306997061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4d40d1df0 0x7fc4d40cd780 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0098a0 tx=0x7fc4bc01c440 comp rx=0 tx=0).stop 2024-04-25T14:17:00.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.327+0000 7fc4da600640 1 -- 172.21.15.86:0/1306997061 shutdown_connections 2024-04-25T14:17:00.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.327+0000 7fc4da600640 1 --2- 172.21.15.86:0/1306997061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4d40d1df0 0x7fc4d40cd780 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:00.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.327+0000 7fc4da600640 1 -- 172.21.15.86:0/1306997061 >> 172.21.15.86:0/1306997061 conn(0x7fc4d40b4650 msgr2=0x7fc4d40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:00.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.327+0000 7fc4da600640 1 -- 172.21.15.86:0/1306997061 shutdown_connections 2024-04-25T14:17:00.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.327+0000 7fc4da600640 1 -- 172.21.15.86:0/1306997061 wait complete. 2024-04-25T14:17:00.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.327+0000 7fc4da600640 1 Processor -- start 2024-04-25T14:17:00.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.327+0000 7fc4da600640 1 -- start start 2024-04-25T14:17:00.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.331+0000 7fc4da600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4d4070330 0x7fc4d409c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:00.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.331+0000 7fc4da600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4d40cdcc0 0x7fc4d40c4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:00.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.331+0000 7fc4da600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4d4155800 0x7fc4d415a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:00.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.331+0000 7fc4da600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d40708c0 con 0x7fc4d4155800 2024-04-25T14:17:00.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.331+0000 7fc4da600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d4070740 con 0x7fc4d40cdcc0 2024-04-25T14:17:00.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.331+0000 7fc4da600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d416bfd0 con 0x7fc4d4070330 2024-04-25T14:17:00.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.331+0000 7fc4d3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4d4155800 0x7fc4d415a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:00.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.331+0000 7fc4d3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4d4155800 0x7fc4d415a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42078/0 (socket says 172.21.15.86:42078) 2024-04-25T14:17:00.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.331+0000 7fc4d3e00640 1 -- 172.21.15.86:0/2220749533 learned_addr learned my addr 172.21.15.86:0/2220749533 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:00.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.331+0000 7fc4d3400640 1 --2- 172.21.15.86:0/2220749533 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4d4070330 0x7fc4d409c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:00.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.331+0000 7fc4cbe00640 1 --2- 172.21.15.86:0/2220749533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4d40cdcc0 0x7fc4d40c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:00.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.331+0000 7fc4d3e00640 1 -- 172.21.15.86:0/2220749533 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4d4070330 msgr2=0x7fc4d409c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:00.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.331+0000 7fc4d3e00640 1 --2- 172.21.15.86:0/2220749533 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4d4070330 0x7fc4d409c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:00.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.331+0000 7fc4d3e00640 1 -- 172.21.15.86:0/2220749533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4d40cdcc0 msgr2=0x7fc4d40c4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:00.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.331+0000 7fc4d3e00640 1 --2- 172.21.15.86:0/2220749533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4d40cdcc0 0x7fc4d40c4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:00.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.331+0000 7fc4d3e00640 1 -- 172.21.15.86:0/2220749533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d41591a0 con 0x7fc4d4155800 2024-04-25T14:17:00.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.331+0000 7fc4d3e00640 1 --2- 172.21.15.86:0/2220749533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4d4155800 0x7fc4d415a240 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc00c730 tx=0x7fc4cc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:00.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.331+0000 7fc4cb400640 1 -- 172.21.15.86:0/2220749533 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc003b80 con 0x7fc4d4155800 2024-04-25T14:17:00.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.331+0000 7fc4cb400640 1 -- 172.21.15.86:0/2220749533 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4cc003d20 con 0x7fc4d4155800 2024-04-25T14:17:00.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.331+0000 7fc4cb400640 1 -- 172.21.15.86:0/2220749533 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc005310 con 0x7fc4d4155800 2024-04-25T14:17:00.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.331+0000 7fc4da600640 1 -- 172.21.15.86:0/2220749533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d4159480 con 0x7fc4d4155800 2024-04-25T14:17:00.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.331+0000 7fc4da600640 1 -- 172.21.15.86:0/2220749533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d4159a40 con 0x7fc4d4155800 2024-04-25T14:17:00.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.331+0000 7fc4da600640 1 -- 172.21.15.86:0/2220749533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4d4069d00 con 0x7fc4d4155800 2024-04-25T14:17:00.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.335+0000 7fc4cb400640 1 -- 172.21.15.86:0/2220749533 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7fc4cc0056c0 con 0x7fc4d4155800 2024-04-25T14:17:00.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.335+0000 7fc4cb400640 1 --2- 172.21.15.86:0/2220749533 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc4b006f450 0x7fc4b0071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:00.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.335+0000 7fc4cb400640 1 -- 172.21.15.86:0/2220749533 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fc4cc08f6b0 con 0x7fc4d4155800 2024-04-25T14:17:00.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.335+0000 7fc4d3400640 1 --2- 172.21.15.86:0/2220749533 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc4b006f450 0x7fc4b0071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:00.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.335+0000 7fc4d3400640 1 --2- 172.21.15.86:0/2220749533 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc4b006f450 0x7fc4b0071960 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4072200 tx=0x7fc4c4008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:00.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.339+0000 7fc4cb400640 1 -- 172.21.15.86:0/2220749533 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc4cc05c2b0 con 0x7fc4d4155800 2024-04-25T14:17:00.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.527+0000 7fc4da600640 1 -- 172.21.15.86:0/2220749533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4d4076bf0 con 0x7fc4d4155800 2024-04-25T14:17:00.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.531+0000 7fc4cb400640 1 -- 172.21.15.86:0/2220749533 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7fc4cc05bbb0 con 0x7fc4d4155800 2024-04-25T14:17:00.528 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:00.528 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:00.528 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:00.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.531+0000 7fc4da600640 1 -- 172.21.15.86:0/2220749533 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc4b006f450 msgr2=0x7fc4b0071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:00.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.531+0000 7fc4da600640 1 --2- 172.21.15.86:0/2220749533 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc4b006f450 0x7fc4b0071960 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4072200 tx=0x7fc4c4008040 comp rx=0 tx=0).stop 2024-04-25T14:17:00.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.531+0000 7fc4da600640 1 -- 172.21.15.86:0/2220749533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4d4155800 msgr2=0x7fc4d415a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:00.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.531+0000 7fc4da600640 1 --2- 172.21.15.86:0/2220749533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4d4155800 0x7fc4d415a240 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc00c730 tx=0x7fc4cc00cbf0 comp rx=0 tx=0).stop 2024-04-25T14:17:00.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.531+0000 7fc4da600640 1 -- 172.21.15.86:0/2220749533 shutdown_connections 2024-04-25T14:17:00.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.531+0000 7fc4da600640 1 --2- 172.21.15.86:0/2220749533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4d4155800 0x7fc4d415a240 unknown :-1 s=CLOSED pgs=2331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:00.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.531+0000 7fc4da600640 1 --2- 172.21.15.86:0/2220749533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4d40cdcc0 0x7fc4d40c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:00.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.531+0000 7fc4da600640 1 --2- 172.21.15.86:0/2220749533 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc4b006f450 0x7fc4b0071960 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:00.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.531+0000 7fc4da600640 1 --2- 172.21.15.86:0/2220749533 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4d4070330 0x7fc4d409c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:00.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.531+0000 7fc4da600640 1 -- 172.21.15.86:0/2220749533 >> 172.21.15.86:0/2220749533 conn(0x7fc4d40b4650 msgr2=0x7fc4d415c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:00.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.531+0000 7fc4da600640 1 -- 172.21.15.86:0/2220749533 shutdown_connections 2024-04-25T14:17:00.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:00.531+0000 7fc4da600640 1 -- 172.21.15.86:0/2220749533 wait complete. 2024-04-25T14:17:00.952 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:00.952 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:01.953 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:02.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.115+0000 7fd46f800640 1 Processor -- start 2024-04-25T14:17:02.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.115+0000 7fd46f800640 1 -- start start 2024-04-25T14:17:02.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.115+0000 7fd46f800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd4680ccab0 0x7fd4680d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:02.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.115+0000 7fd46f800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd4680cf9d0 0x7fd4680d2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:02.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.115+0000 7fd46f800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4680b3bd0 con 0x7fd4680b79c0 2024-04-25T14:17:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.115+0000 7fd46f800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd4680cf190 con 0x7fd4680ccab0 2024-04-25T14:17:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.115+0000 7fd46f800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd4680b2620 con 0x7fd4680cf9d0 2024-04-25T14:17:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.115+0000 7fd46ce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd4680ccab0 0x7fd4680d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.115+0000 7fd46ce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd4680ccab0 0x7fd4680d23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42086/0 (socket says 172.21.15.86:42086) 2024-04-25T14:17:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.115+0000 7fd46ce00640 1 -- 172.21.15.86:0/634613891 learned_addr learned my addr 172.21.15.86:0/634613891 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.115+0000 7fd46ce00640 1 -- 172.21.15.86:0/634613891 >> v2:172.21.15.155:3300/0 conn(0x7fd4680cf9d0 msgr2=0x7fd4680d2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:02.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.115+0000 7fd46d800640 1 --2- 172.21.15.86:0/634613891 >> v2:172.21.15.155:3300/0 conn(0x7fd4680cf9d0 0x7fd4680d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:02.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.119+0000 7fd46ce00640 1 --2- 172.21.15.86:0/634613891 >> v2:172.21.15.155:3300/0 conn(0x7fd4680cf9d0 0x7fd4680d2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:02.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.119+0000 7fd46ce00640 1 -- 172.21.15.86:0/634613891 >> v1:172.21.15.155:6789/0 conn(0x7fd4680b79c0 legacy=0x7fd4681617f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:02.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.119+0000 7fd46ce00640 1 -- 172.21.15.86:0/634613891 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4680b1cf0 con 0x7fd4680ccab0 2024-04-25T14:17:02.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.119+0000 7fd46d800640 1 --2- 172.21.15.86:0/634613891 >> v2:172.21.15.155:3300/0 conn(0x7fd4680cf9d0 0x7fd4680d2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:17:02.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.119+0000 7fd46ce00640 1 --2- 172.21.15.86:0/634613891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4680ccab0 0x7fd4680d23d0 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7fd46806ea40 tx=0x7fd45c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aaadbf263f271ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:02.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.119+0000 7fd467400640 1 -- 172.21.15.86:0/634613891 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c0b1070 con 0x7fd4680ccab0 2024-04-25T14:17:02.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.119+0000 7fd467400640 1 -- 172.21.15.86:0/634613891 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd45c0ac640 con 0x7fd4680ccab0 2024-04-25T14:17:02.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.119+0000 7fd467400640 1 -- 172.21.15.86:0/634613891 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c0aca10 con 0x7fd4680ccab0 2024-04-25T14:17:02.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.123+0000 7fd46f800640 1 -- 172.21.15.86:0/634613891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4680ccab0 msgr2=0x7fd4680d23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:02.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.123+0000 7fd46f800640 1 --2- 172.21.15.86:0/634613891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4680ccab0 0x7fd4680d23d0 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7fd46806ea40 tx=0x7fd45c01c4e0 comp rx=0 tx=0).stop 2024-04-25T14:17:02.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.123+0000 7fd46f800640 1 -- 172.21.15.86:0/634613891 shutdown_connections 2024-04-25T14:17:02.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.123+0000 7fd46f800640 1 --2- 172.21.15.86:0/634613891 >> v2:172.21.15.155:3300/0 conn(0x7fd4680cf9d0 0x7fd4680d2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:02.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.123+0000 7fd46f800640 1 --2- 172.21.15.86:0/634613891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4680ccab0 0x7fd4680d23d0 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:02.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.123+0000 7fd46f800640 1 -- 172.21.15.86:0/634613891 >> 172.21.15.86:0/634613891 conn(0x7fd46809ba30 msgr2=0x7fd4680b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:02.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.123+0000 7fd46f800640 1 -- 172.21.15.86:0/634613891 shutdown_connections 2024-04-25T14:17:02.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.123+0000 7fd46f800640 1 -- 172.21.15.86:0/634613891 wait complete. 2024-04-25T14:17:02.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.123+0000 7fd46f800640 1 Processor -- start 2024-04-25T14:17:02.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.123+0000 7fd46f800640 1 -- start start 2024-04-25T14:17:02.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.127+0000 7fd46f800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4680b79c0 0x7fd468158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.127+0000 7fd46f800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4680ccab0 0x7fd46806df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.127+0000 7fd46f800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4680cf9d0 0x7fd468167d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.127+0000 7fd46f800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd46816ab00 con 0x7fd4680ccab0 2024-04-25T14:17:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.127+0000 7fd46f800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd46816a980 con 0x7fd4680cf9d0 2024-04-25T14:17:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.127+0000 7fd46f800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd46816ac80 con 0x7fd4680b79c0 2024-04-25T14:17:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.127+0000 7fd467e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4680ccab0 0x7fd46806df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.127+0000 7fd467e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4680ccab0 0x7fd46806df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42102/0 (socket says 172.21.15.86:42102) 2024-04-25T14:17:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.127+0000 7fd467e00640 1 -- 172.21.15.86:0/1719000716 learned_addr learned my addr 172.21.15.86:0/1719000716 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.127+0000 7fd46d800640 1 --2- 172.21.15.86:0/1719000716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4680cf9d0 0x7fd468167d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.127+0000 7fd46ce00640 1 --2- 172.21.15.86:0/1719000716 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4680b79c0 0x7fd468158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.127+0000 7fd467e00640 1 -- 172.21.15.86:0/1719000716 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4680b79c0 msgr2=0x7fd468158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.127+0000 7fd467e00640 1 --2- 172.21.15.86:0/1719000716 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4680b79c0 0x7fd468158650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.127+0000 7fd467e00640 1 -- 172.21.15.86:0/1719000716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4680cf9d0 msgr2=0x7fd468167d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.127+0000 7fd467e00640 1 --2- 172.21.15.86:0/1719000716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4680cf9d0 0x7fd468167d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.127+0000 7fd467e00640 1 -- 172.21.15.86:0/1719000716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4681483e0 con 0x7fd4680ccab0 2024-04-25T14:17:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.127+0000 7fd467e00640 1 --2- 172.21.15.86:0/1719000716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4680ccab0 0x7fd46806df10 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7fd45400b440 tx=0x7fd45400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.127+0000 7fd464c00640 1 -- 172.21.15.86:0/1719000716 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd454015020 con 0x7fd4680ccab0 2024-04-25T14:17:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.127+0000 7fd464c00640 1 -- 172.21.15.86:0/1719000716 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd454011830 con 0x7fd4680ccab0 2024-04-25T14:17:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.127+0000 7fd464c00640 1 -- 172.21.15.86:0/1719000716 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd454011c20 con 0x7fd4680ccab0 2024-04-25T14:17:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.127+0000 7fd46f800640 1 -- 172.21.15.86:0/1719000716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd46815a150 con 0x7fd4680ccab0 2024-04-25T14:17:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.127+0000 7fd46f800640 1 -- 172.21.15.86:0/1719000716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4681fd0f0 con 0x7fd4680ccab0 2024-04-25T14:17:02.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.127+0000 7fd46f800640 1 -- 172.21.15.86:0/1719000716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd46815a3c0 con 0x7fd4680ccab0 2024-04-25T14:17:02.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.127+0000 7fd464c00640 1 -- 172.21.15.86:0/1719000716 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7fd454018020 con 0x7fd4680ccab0 2024-04-25T14:17:02.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.131+0000 7fd464c00640 1 --2- 172.21.15.86:0/1719000716 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd44806f480 0x7fd448071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:02.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.131+0000 7fd464c00640 1 -- 172.21.15.86:0/1719000716 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fd45408f4f0 con 0x7fd4680ccab0 2024-04-25T14:17:02.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.131+0000 7fd46ce00640 1 --2- 172.21.15.86:0/1719000716 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd44806f480 0x7fd448071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:02.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.131+0000 7fd46ce00640 1 --2- 172.21.15.86:0/1719000716 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd44806f480 0x7fd448071990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0ae000 tx=0x7fd45c0a6070 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.135+0000 7fd464c00640 1 -- 172.21.15.86:0/1719000716 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd45401f020 con 0x7fd4680ccab0 2024-04-25T14:17:02.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.331+0000 7fd46f800640 1 -- 172.21.15.86:0/1719000716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4680a04c0 con 0x7fd4680ccab0 2024-04-25T14:17:02.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.331+0000 7fd464c00640 1 -- 172.21.15.86:0/1719000716 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7fd4681483e0 con 0x7fd4680ccab0 2024-04-25T14:17:02.329 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:02.329 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:02.329 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:02.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.331+0000 7fd46f800640 1 -- 172.21.15.86:0/1719000716 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd44806f480 msgr2=0x7fd448071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:02.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.331+0000 7fd46f800640 1 --2- 172.21.15.86:0/1719000716 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd44806f480 0x7fd448071990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0ae000 tx=0x7fd45c0a6070 comp rx=0 tx=0).stop 2024-04-25T14:17:02.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.331+0000 7fd46f800640 1 -- 172.21.15.86:0/1719000716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4680ccab0 msgr2=0x7fd46806df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:02.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.331+0000 7fd46f800640 1 --2- 172.21.15.86:0/1719000716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4680ccab0 0x7fd46806df10 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7fd45400b440 tx=0x7fd45400b900 comp rx=0 tx=0).stop 2024-04-25T14:17:02.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.331+0000 7fd46f800640 1 -- 172.21.15.86:0/1719000716 shutdown_connections 2024-04-25T14:17:02.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.331+0000 7fd46f800640 1 --2- 172.21.15.86:0/1719000716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4680cf9d0 0x7fd468167d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:02.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.331+0000 7fd46f800640 1 --2- 172.21.15.86:0/1719000716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4680ccab0 0x7fd46806df10 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:02.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.331+0000 7fd46f800640 1 --2- 172.21.15.86:0/1719000716 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd44806f480 0x7fd448071990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:02.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.331+0000 7fd46f800640 1 --2- 172.21.15.86:0/1719000716 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4680b79c0 0x7fd468158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:02.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.331+0000 7fd46f800640 1 -- 172.21.15.86:0/1719000716 >> 172.21.15.86:0/1719000716 conn(0x7fd46809ba30 msgr2=0x7fd468157520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:02.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.331+0000 7fd46f800640 1 -- 172.21.15.86:0/1719000716 shutdown_connections 2024-04-25T14:17:02.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:02.331+0000 7fd46f800640 1 -- 172.21.15.86:0/1719000716 wait complete. 2024-04-25T14:17:02.753 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:02.753 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:03.754 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:03.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.919+0000 7ffb6a600640 1 Processor -- start 2024-04-25T14:17:03.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.919+0000 7ffb6a600640 1 -- start start 2024-04-25T14:17:03.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.919+0000 7ffb6a600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffb64071580 0x7ffb640d25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:03.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.919+0000 7ffb6a600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb640ce510 con 0x7ffb6415fc20 2024-04-25T14:17:03.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.919+0000 7ffb6a600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb64164680 con 0x7ffb640b8be0 2024-04-25T14:17:03.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.919+0000 7ffb6a600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ffb64160030 con 0x7ffb64071580 2024-04-25T14:17:03.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.919+0000 7ffb69c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffb64071580 0x7ffb640d25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:03.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.919+0000 7ffb69c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffb64071580 0x7ffb640d25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42106/0 (socket says 172.21.15.86:42106) 2024-04-25T14:17:03.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.919+0000 7ffb69c00640 1 -- 172.21.15.86:0/411298191 learned_addr learned my addr 172.21.15.86:0/411298191 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:03.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.919+0000 7ffb69c00640 1 -- 172.21.15.86:0/411298191 >> v1:172.21.15.155:6789/0 conn(0x7ffb640b8be0 legacy=0x7ffb64070e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:03.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.919+0000 7ffb69c00640 1 -- 172.21.15.86:0/411298191 >> v1:172.21.15.86:6789/0 conn(0x7ffb6415fc20 legacy=0x7ffb640b6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:03.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.919+0000 7ffb69c00640 1 -- 172.21.15.86:0/411298191 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb640d1800 con 0x7ffb64071580 2024-04-25T14:17:03.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.923+0000 7ffb69c00640 1 --2- 172.21.15.86:0/411298191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb64071580 0x7ffb640d25c0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c009e50 tx=0x7ffb5c01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f597cb51ac0dfc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:03.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.923+0000 7ffb63400640 1 -- 172.21.15.86:0/411298191 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c0a58d0 con 0x7ffb64071580 2024-04-25T14:17:03.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.923+0000 7ffb63400640 1 -- 172.21.15.86:0/411298191 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb5c0a5a70 con 0x7ffb64071580 2024-04-25T14:17:03.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.923+0000 7ffb63400640 1 -- 172.21.15.86:0/411298191 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c0a5e40 con 0x7ffb64071580 2024-04-25T14:17:03.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.923+0000 7ffb6a600640 1 -- 172.21.15.86:0/411298191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb64071580 msgr2=0x7ffb640d25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:03.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.923+0000 7ffb6a600640 1 --2- 172.21.15.86:0/411298191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb64071580 0x7ffb640d25c0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c009e50 tx=0x7ffb5c01ca00 comp rx=0 tx=0).stop 2024-04-25T14:17:03.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.923+0000 7ffb6a600640 1 -- 172.21.15.86:0/411298191 shutdown_connections 2024-04-25T14:17:03.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.923+0000 7ffb6a600640 1 --2- 172.21.15.86:0/411298191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb64071580 0x7ffb640d25c0 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:03.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.923+0000 7ffb6a600640 1 -- 172.21.15.86:0/411298191 >> 172.21.15.86:0/411298191 conn(0x7ffb640cfe10 msgr2=0x7ffb640d0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:03.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.923+0000 7ffb6a600640 1 -- 172.21.15.86:0/411298191 shutdown_connections 2024-04-25T14:17:03.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.923+0000 7ffb6a600640 1 -- 172.21.15.86:0/411298191 wait complete. 2024-04-25T14:17:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.927+0000 7ffb6a600640 1 Processor -- start 2024-04-25T14:17:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.927+0000 7ffb6a600640 1 -- start start 2024-04-25T14:17:03.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.927+0000 7ffb6a600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb640b8be0 0x7ffb6414f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:03.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.927+0000 7ffb6a600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb6415fc20 0x7ffb6406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:03.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.927+0000 7ffb6a600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb6406f650 0x7ffb64155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:03.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.927+0000 7ffb6a600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6416db20 con 0x7ffb6406f650 2024-04-25T14:17:03.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.927+0000 7ffb6a600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffb640b8ff0 con 0x7ffb6415fc20 2024-04-25T14:17:03.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.927+0000 7ffb6a600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ffb6416dca0 con 0x7ffb640b8be0 2024-04-25T14:17:03.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.927+0000 7ffb69c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb6406f650 0x7ffb64155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:03.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.927+0000 7ffb69c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb6406f650 0x7ffb64155b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42108/0 (socket says 172.21.15.86:42108) 2024-04-25T14:17:03.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.927+0000 7ffb69c00640 1 -- 172.21.15.86:0/661116875 learned_addr learned my addr 172.21.15.86:0/661116875 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:03.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.927+0000 7ffb63e00640 1 --2- 172.21.15.86:0/661116875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb6415fc20 0x7ffb6406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:03.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.927+0000 7ffb69200640 1 --2- 172.21.15.86:0/661116875 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb640b8be0 0x7ffb6414f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:03.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.927+0000 7ffb69c00640 1 -- 172.21.15.86:0/661116875 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb640b8be0 msgr2=0x7ffb6414f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:03.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.927+0000 7ffb69c00640 1 --2- 172.21.15.86:0/661116875 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb640b8be0 0x7ffb6414f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:03.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.927+0000 7ffb69c00640 1 -- 172.21.15.86:0/661116875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb6415fc20 msgr2=0x7ffb6406f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:03.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.927+0000 7ffb69c00640 1 --2- 172.21.15.86:0/661116875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb6415fc20 0x7ffb6406f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:03.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.927+0000 7ffb69c00640 1 -- 172.21.15.86:0/661116875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb641528a0 con 0x7ffb6406f650 2024-04-25T14:17:03.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.927+0000 7ffb63e00640 1 --2- 172.21.15.86:0/661116875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb6415fc20 0x7ffb6406f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:17:03.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.927+0000 7ffb69c00640 1 --2- 172.21.15.86:0/661116875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb6406f650 0x7ffb64155b80 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c0af040 tx=0x7ffb5c0197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:03.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.927+0000 7ffb60c00640 1 -- 172.21.15.86:0/661116875 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c0a58d0 con 0x7ffb6406f650 2024-04-25T14:17:03.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.927+0000 7ffb60c00640 1 -- 172.21.15.86:0/661116875 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb5c0be070 con 0x7ffb6406f650 2024-04-25T14:17:03.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.927+0000 7ffb6a600640 1 -- 172.21.15.86:0/661116875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb64152b80 con 0x7ffb6406f650 2024-04-25T14:17:03.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.931+0000 7ffb60c00640 1 -- 172.21.15.86:0/661116875 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c009510 con 0x7ffb6406f650 2024-04-25T14:17:03.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.931+0000 7ffb6a600640 1 -- 172.21.15.86:0/661116875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb64153660 con 0x7ffb6406f650 2024-04-25T14:17:03.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.931+0000 7ffb6a600640 1 -- 172.21.15.86:0/661116875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb6406fb50 con 0x7ffb6406f650 2024-04-25T14:17:03.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.931+0000 7ffb60c00640 1 -- 172.21.15.86:0/661116875 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7ffb5c0b86b0 con 0x7ffb6406f650 2024-04-25T14:17:03.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.931+0000 7ffb60c00640 1 --2- 172.21.15.86:0/661116875 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ffb4406f450 0x7ffb44071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:03.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.931+0000 7ffb60c00640 1 -- 172.21.15.86:0/661116875 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7ffb5c12ad70 con 0x7ffb6406f650 2024-04-25T14:17:03.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.931+0000 7ffb69200640 1 --2- 172.21.15.86:0/661116875 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ffb4406f450 0x7ffb44071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:03.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.931+0000 7ffb69200640 1 --2- 172.21.15.86:0/661116875 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ffb4406f450 0x7ffb44071960 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ffb54000f70 tx=0x7ffb54009040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:03.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:03.935+0000 7ffb60c00640 1 -- 172.21.15.86:0/661116875 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffb5c016620 con 0x7ffb6406f650 2024-04-25T14:17:04.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:04.127+0000 7ffb6a600640 1 -- 172.21.15.86:0/661116875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb64076b80 con 0x7ffb6406f650 2024-04-25T14:17:04.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:04.127+0000 7ffb60c00640 1 -- 172.21.15.86:0/661116875 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7ffb5c0f7970 con 0x7ffb6406f650 2024-04-25T14:17:04.126 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:04.126 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:04.126 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:04.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:04.131+0000 7ffb6a600640 1 -- 172.21.15.86:0/661116875 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ffb4406f450 msgr2=0x7ffb44071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:04.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:04.131+0000 7ffb6a600640 1 --2- 172.21.15.86:0/661116875 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ffb4406f450 0x7ffb44071960 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ffb54000f70 tx=0x7ffb54009040 comp rx=0 tx=0).stop 2024-04-25T14:17:04.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:04.131+0000 7ffb6a600640 1 -- 172.21.15.86:0/661116875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb6406f650 msgr2=0x7ffb64155b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:04.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:04.131+0000 7ffb6a600640 1 --2- 172.21.15.86:0/661116875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb6406f650 0x7ffb64155b80 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c0af040 tx=0x7ffb5c0197a0 comp rx=0 tx=0).stop 2024-04-25T14:17:04.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:04.131+0000 7ffb6a600640 1 -- 172.21.15.86:0/661116875 shutdown_connections 2024-04-25T14:17:04.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:04.131+0000 7ffb6a600640 1 --2- 172.21.15.86:0/661116875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb6406f650 0x7ffb64155b80 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:04.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:04.131+0000 7ffb6a600640 1 --2- 172.21.15.86:0/661116875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb6415fc20 0x7ffb6406f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:04.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:04.131+0000 7ffb6a600640 1 --2- 172.21.15.86:0/661116875 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ffb4406f450 0x7ffb44071960 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:04.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:04.131+0000 7ffb6a600640 1 --2- 172.21.15.86:0/661116875 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb640b8be0 0x7ffb6414f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:04.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:04.131+0000 7ffb6a600640 1 -- 172.21.15.86:0/661116875 >> 172.21.15.86:0/661116875 conn(0x7ffb640cfe10 msgr2=0x7ffb640b67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:04.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:04.131+0000 7ffb6a600640 1 -- 172.21.15.86:0/661116875 shutdown_connections 2024-04-25T14:17:04.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:04.131+0000 7ffb6a600640 1 -- 172.21.15.86:0/661116875 wait complete. 2024-04-25T14:17:04.551 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:04.551 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:05.552 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:05.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.707+0000 7f853d600640 1 Processor -- start 2024-04-25T14:17:05.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.707+0000 7f853d600640 1 -- start start 2024-04-25T14:17:05.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.711+0000 7f853d600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8538070330 0x7f85380d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:05.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.711+0000 7f853d600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85380ce790 0x7f853806f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:05.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.711+0000 7f853d600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85380d2030 con 0x7f85380d1c20 2024-04-25T14:17:05.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.711+0000 7f853d600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f853815e5c0 con 0x7f8538070330 2024-04-25T14:17:05.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.711+0000 7f853d600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f853815e8a0 con 0x7f85380ce790 2024-04-25T14:17:05.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.711+0000 7f8537400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85380ce790 0x7f853806f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:05.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.711+0000 7f8536a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8538070330 0x7f85380d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:05.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.711+0000 7f8536a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8538070330 0x7f85380d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:45102/0 (socket says 172.21.15.86:45102) 2024-04-25T14:17:05.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.711+0000 7f8537400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85380ce790 0x7f853806f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:58216/0 (socket says 172.21.15.86:58216) 2024-04-25T14:17:05.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.711+0000 7f8536a00640 1 -- 172.21.15.86:0/328880475 learned_addr learned my addr 172.21.15.86:0/328880475 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:05.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.711+0000 7f8536000640 1 --1- 172.21.15.86:0/328880475 >> v1:172.21.15.155:6789/0 conn(0x7f85380d1c20 0x7f853806ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:51712/0 (socket says 172.21.15.86:51712) 2024-04-25T14:17:05.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.711+0000 7f8537400640 1 -- 172.21.15.86:0/328880475 >> v2:172.21.15.155:3300/0 conn(0x7f8538070330 msgr2=0x7f85380d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:05.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.711+0000 7f8537400640 1 --2- 172.21.15.86:0/328880475 >> v2:172.21.15.155:3300/0 conn(0x7f8538070330 0x7f85380d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:05.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.711+0000 7f8537400640 1 -- 172.21.15.86:0/328880475 >> v1:172.21.15.155:6789/0 conn(0x7f85380d1c20 legacy=0x7f853806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:05.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.711+0000 7f8537400640 1 -- 172.21.15.86:0/328880475 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853815eb80 con 0x7f85380ce790 2024-04-25T14:17:05.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.711+0000 7f8536a00640 1 --2- 172.21.15.86:0/328880475 >> v2:172.21.15.155:3300/0 conn(0x7f8538070330 0x7f85380d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:17:05.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.715+0000 7f8537400640 1 --2- 172.21.15.86:0/328880475 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85380ce790 0x7f853806f440 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f85300097c0 tx=0x7f853001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bb142df3676112a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:05.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.715+0000 7f8535600640 1 -- 172.21.15.86:0/328880475 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85300b0070 con 0x7f85380ce790 2024-04-25T14:17:05.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.715+0000 7f8535600640 1 -- 172.21.15.86:0/328880475 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85300ab600 con 0x7f85380ce790 2024-04-25T14:17:05.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.715+0000 7f8535600640 1 -- 172.21.15.86:0/328880475 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85300ab9d0 con 0x7f85380ce790 2024-04-25T14:17:05.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.715+0000 7f853d600640 1 -- 172.21.15.86:0/328880475 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85380ce790 msgr2=0x7f853806f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:05.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.715+0000 7f853d600640 1 --2- 172.21.15.86:0/328880475 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85380ce790 0x7f853806f440 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f85300097c0 tx=0x7f853001c410 comp rx=0 tx=0).stop 2024-04-25T14:17:05.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.715+0000 7f853d600640 1 -- 172.21.15.86:0/328880475 shutdown_connections 2024-04-25T14:17:05.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.715+0000 7f853d600640 1 --2- 172.21.15.86:0/328880475 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85380ce790 0x7f853806f440 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:05.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.715+0000 7f853d600640 1 --2- 172.21.15.86:0/328880475 >> v2:172.21.15.155:3300/0 conn(0x7f8538070330 0x7f85380d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:05.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.715+0000 7f853d600640 1 -- 172.21.15.86:0/328880475 >> 172.21.15.86:0/328880475 conn(0x7f85380b4650 msgr2=0x7f85380b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:05.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.715+0000 7f853d600640 1 -- 172.21.15.86:0/328880475 shutdown_connections 2024-04-25T14:17:05.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.715+0000 7f853d600640 1 -- 172.21.15.86:0/328880475 wait complete. 2024-04-25T14:17:05.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.719+0000 7f853d600640 1 Processor -- start 2024-04-25T14:17:05.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.719+0000 7f853d600640 1 -- start start 2024-04-25T14:17:05.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.719+0000 7f853d600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8538070330 0x7f85380ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:05.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.719+0000 7f853d600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85380ce790 0x7f85381fcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:05.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.719+0000 7f853d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85380d1c20 0x7f85380af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:05.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.719+0000 7f853d600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85380708c0 con 0x7f85380d1c20 2024-04-25T14:17:05.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.719+0000 7f853d600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8538070740 con 0x7f85380ce790 2024-04-25T14:17:05.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.719+0000 7f853d600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f853816afc0 con 0x7f8538070330 2024-04-25T14:17:05.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.719+0000 7f8537400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85380d1c20 0x7f85380af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:05.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.719+0000 7f8537400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85380d1c20 0x7f85380af7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42112/0 (socket says 172.21.15.86:42112) 2024-04-25T14:17:05.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.719+0000 7f8537400640 1 -- 172.21.15.86:0/1546139588 learned_addr learned my addr 172.21.15.86:0/1546139588 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:05.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.719+0000 7f8536000640 1 --2- 172.21.15.86:0/1546139588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85380ce790 0x7f85381fcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:05.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.719+0000 7f8536a00640 1 --2- 172.21.15.86:0/1546139588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8538070330 0x7f85380ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:05.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.719+0000 7f8537400640 1 -- 172.21.15.86:0/1546139588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8538070330 msgr2=0x7f85380ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:05.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.719+0000 7f8537400640 1 --2- 172.21.15.86:0/1546139588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8538070330 0x7f85380ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:05.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.719+0000 7f8537400640 1 -- 172.21.15.86:0/1546139588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85380ce790 msgr2=0x7f85381fcfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:05.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.719+0000 7f8537400640 1 --2- 172.21.15.86:0/1546139588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85380ce790 0x7f85381fcfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:05.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.719+0000 7f8537400640 1 -- 172.21.15.86:0/1546139588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85380c67f0 con 0x7f85380d1c20 2024-04-25T14:17:05.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.719+0000 7f8537400640 1 --2- 172.21.15.86:0/1546139588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85380d1c20 0x7f85380af7a0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7f85300ae040 tx=0x7f85300191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:05.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.723+0000 7f852ea00640 1 -- 172.21.15.86:0/1546139588 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85300b0040 con 0x7f85380d1c20 2024-04-25T14:17:05.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.723+0000 7f852ea00640 1 -- 172.21.15.86:0/1546139588 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85300ba070 con 0x7f85380d1c20 2024-04-25T14:17:05.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.723+0000 7f852ea00640 1 -- 172.21.15.86:0/1546139588 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530006e20 con 0x7f85380d1c20 2024-04-25T14:17:05.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.723+0000 7f853d600640 1 -- 172.21.15.86:0/1546139588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85380c6ad0 con 0x7f85380d1c20 2024-04-25T14:17:05.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.723+0000 7f853d600640 1 -- 172.21.15.86:0/1546139588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85380c8bb0 con 0x7f85380d1c20 2024-04-25T14:17:05.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.723+0000 7f853d600640 1 -- 172.21.15.86:0/1546139588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8538069d00 con 0x7f85380d1c20 2024-04-25T14:17:05.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.723+0000 7f852ea00640 1 -- 172.21.15.86:0/1546139588 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f85300b4630 con 0x7f85380d1c20 2024-04-25T14:17:05.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.723+0000 7f852ea00640 1 --2- 172.21.15.86:0/1546139588 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f851806f450 0x7f8518071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:05.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.727+0000 7f852ea00640 1 -- 172.21.15.86:0/1546139588 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f85301291a0 con 0x7f85380d1c20 2024-04-25T14:17:05.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.727+0000 7f8536a00640 1 --2- 172.21.15.86:0/1546139588 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f851806f450 0x7f8518071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:05.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.727+0000 7f8536a00640 1 --2- 172.21.15.86:0/1546139588 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f851806f450 0x7f8518071960 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f85280072e0 tx=0x7f852800b040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:05.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.731+0000 7f852ea00640 1 -- 172.21.15.86:0/1546139588 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8530016090 con 0x7f85380d1c20 2024-04-25T14:17:05.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.919+0000 7f853d600640 1 -- 172.21.15.86:0/1546139588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f853806db60 con 0x7f85380d1c20 2024-04-25T14:17:05.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.919+0000 7f852ea00640 1 -- 172.21.15.86:0/1546139588 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f85300f56c0 con 0x7f85380d1c20 2024-04-25T14:17:05.917 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:05.917 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:05.918 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:05.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.923+0000 7f853d600640 1 -- 172.21.15.86:0/1546139588 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f851806f450 msgr2=0x7f8518071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:05.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.923+0000 7f853d600640 1 --2- 172.21.15.86:0/1546139588 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f851806f450 0x7f8518071960 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f85280072e0 tx=0x7f852800b040 comp rx=0 tx=0).stop 2024-04-25T14:17:05.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.923+0000 7f853d600640 1 -- 172.21.15.86:0/1546139588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85380d1c20 msgr2=0x7f85380af7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:05.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.923+0000 7f853d600640 1 --2- 172.21.15.86:0/1546139588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85380d1c20 0x7f85380af7a0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7f85300ae040 tx=0x7f85300191b0 comp rx=0 tx=0).stop 2024-04-25T14:17:05.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.923+0000 7f853d600640 1 -- 172.21.15.86:0/1546139588 shutdown_connections 2024-04-25T14:17:05.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.923+0000 7f853d600640 1 --2- 172.21.15.86:0/1546139588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85380d1c20 0x7f85380af7a0 unknown :-1 s=CLOSED pgs=2337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:05.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.923+0000 7f853d600640 1 --2- 172.21.15.86:0/1546139588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85380ce790 0x7f85381fcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:05.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.923+0000 7f853d600640 1 --2- 172.21.15.86:0/1546139588 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f851806f450 0x7f8518071960 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:05.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.923+0000 7f853d600640 1 --2- 172.21.15.86:0/1546139588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8538070330 0x7f85380ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:05.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.923+0000 7f853d600640 1 -- 172.21.15.86:0/1546139588 >> 172.21.15.86:0/1546139588 conn(0x7f85380b4650 msgr2=0x7f8538167cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:05.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.923+0000 7f853d600640 1 -- 172.21.15.86:0/1546139588 shutdown_connections 2024-04-25T14:17:05.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:05.923+0000 7f853d600640 1 -- 172.21.15.86:0/1546139588 wait complete. 2024-04-25T14:17:06.342 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:06.343 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:07.343 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:07.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.503+0000 7f32b4200640 1 Processor -- start 2024-04-25T14:17:07.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.503+0000 7f32b4200640 1 -- start start 2024-04-25T14:17:07.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32b4200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32ac070330 0x7f32ac0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:07.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32b4200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32ac0d2030 con 0x7f32ac0d1c20 2024-04-25T14:17:07.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32b4200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32ac15e5c0 con 0x7f32ac0ce790 2024-04-25T14:17:07.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32b4200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f32ac15e8a0 con 0x7f32ac070330 2024-04-25T14:17:07.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32b0e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f32ac0d1c20 0x7f32ac06ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34674/0 (socket says 172.21.15.86:34674) 2024-04-25T14:17:07.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32b0e00640 1 -- 172.21.15.86:0/14740247 learned_addr learned my addr 172.21.15.86:0/14740247 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:17:07.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32b1800640 1 --2- 172.21.15.86:0/14740247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32ac070330 0x7f32ac0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32abe00640 1 -- 172.21.15.86:0/14740247 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1858168146 0 0) 0x7f3298001120 con 0x7f32ac0d1c20 2024-04-25T14:17:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32abe00640 1 -- 172.21.15.86:0/14740247 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3294003530 con 0x7f32ac0d1c20 2024-04-25T14:17:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32abe00640 1 -- 172.21.15.86:0/14740247 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1001061616 0 0) 0x7f32ac15e5c0 con 0x7f32ac0ce790 2024-04-25T14:17:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32abe00640 1 -- 172.21.15.86:0/14740247 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3298001120 con 0x7f32ac0ce790 2024-04-25T14:17:07.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32abe00640 1 -- 172.21.15.86:0/14740247 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 993246224 0 0) 0x7f3294003530 con 0x7f32ac0d1c20 2024-04-25T14:17:07.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32abe00640 1 -- 172.21.15.86:0/14740247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32ac070330 msgr2=0x7f32ac0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:07.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32abe00640 1 --2- 172.21.15.86:0/14740247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32ac070330 0x7f32ac0d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:07.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32abe00640 1 -- 172.21.15.86:0/14740247 >> v1:172.21.15.155:6789/0 conn(0x7f32ac0ce790 legacy=0x7f32ac06f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:07.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32abe00640 1 -- 172.21.15.86:0/14740247 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ac15eb80 con 0x7f32ac0d1c20 2024-04-25T14:17:07.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32abe00640 1 -- 172.21.15.86:0/14740247 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f3298001850 con 0x7f32ac0d1c20 2024-04-25T14:17:07.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32b1800640 1 --2- 172.21.15.86:0/14740247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32ac070330 0x7f32ac0d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:17:07.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32abe00640 1 -- 172.21.15.86:0/14740247 >> v1:172.21.15.86:6789/0 conn(0x7f32ac0d1c20 legacy=0x7f32ac06ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:07.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32abe00640 1 --2- 172.21.15.86:0/14740247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3294003bf0 0x7f3294004040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:07.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32abe00640 1 --2- 172.21.15.86:0/14740247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3294004810 0x7f3294004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:07.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32abe00640 1 --2- 172.21.15.86:0/14740247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32ac070330 0x7f32940052b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:07.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32abe00640 1 -- 172.21.15.86:0/14740247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac15e8a0 con 0x7f3294003bf0 2024-04-25T14:17:07.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32abe00640 1 -- 172.21.15.86:0/14740247 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f329400bbb0 con 0x7f3294004810 2024-04-25T14:17:07.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32abe00640 1 -- 172.21.15.86:0/14740247 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f329400bdb0 con 0x7f32ac070330 2024-04-25T14:17:07.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32b1800640 1 --2- 172.21.15.86:0/14740247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3294003bf0 0x7f3294004040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:07.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32b2200640 1 --2- 172.21.15.86:0/14740247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32ac070330 0x7f32940052b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:07.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32b0e00640 1 --2- 172.21.15.86:0/14740247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3294004810 0x7f3294004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:07.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32b1800640 1 -- 172.21.15.86:0/14740247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32ac070330 msgr2=0x7f32940052b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:07.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32b1800640 1 --2- 172.21.15.86:0/14740247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32ac070330 0x7f32940052b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:07.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32b1800640 1 -- 172.21.15.86:0/14740247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3294004810 msgr2=0x7f3294004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:07.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32b1800640 1 --2- 172.21.15.86:0/14740247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3294004810 0x7f3294004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:07.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.507+0000 7f32b1800640 1 -- 172.21.15.86:0/14740247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3294003530 con 0x7f3294003bf0 2024-04-25T14:17:07.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.511+0000 7f32b1800640 1 --2- 172.21.15.86:0/14740247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3294003bf0 0x7f3294004040 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009c80 tx=0x7f32a001cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414959d4e82cfe7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:07.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.511+0000 7f32abe00640 1 -- 172.21.15.86:0/14740247 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0009880 con 0x7f3294003bf0 2024-04-25T14:17:07.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.511+0000 7f32abe00640 1 -- 172.21.15.86:0/14740247 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a0004020 con 0x7f3294003bf0 2024-04-25T14:17:07.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.511+0000 7f32abe00640 1 -- 172.21.15.86:0/14740247 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a00043f0 con 0x7f3294003bf0 2024-04-25T14:17:07.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.511+0000 7f32b4200640 1 -- 172.21.15.86:0/14740247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3294003bf0 msgr2=0x7f3294004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:07.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.511+0000 7f32b4200640 1 --2- 172.21.15.86:0/14740247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3294003bf0 0x7f3294004040 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009c80 tx=0x7f32a001cbf0 comp rx=0 tx=0).stop 2024-04-25T14:17:07.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.511+0000 7f32b2200640 1 -- 172.21.15.86:0/14740247 reap_dead start 2024-04-25T14:17:07.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.511+0000 7f32b4200640 1 -- 172.21.15.86:0/14740247 shutdown_connections 2024-04-25T14:17:07.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.511+0000 7f32b4200640 1 -- 172.21.15.86:0/14740247 >> 172.21.15.86:0/14740247 conn(0x7f32ac0b4650 msgr2=0x7f32ac0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:07.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.511+0000 7f32b4200640 1 -- 172.21.15.86:0/14740247 shutdown_connections 2024-04-25T14:17:07.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.511+0000 7f32b4200640 1 -- 172.21.15.86:0/14740247 wait complete. 2024-04-25T14:17:07.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.515+0000 7f32b4200640 1 Processor -- start 2024-04-25T14:17:07.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.515+0000 7f32b4200640 1 -- start start 2024-04-25T14:17:07.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.515+0000 7f32b4200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3294003bf0 0x7f32ac15ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:07.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.515+0000 7f32b4200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32ac1614d0 0x7f32ac1618e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:07.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.515+0000 7f32b4200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32ac15f540 0x7f32ac15f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:07.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.515+0000 7f32b4200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac0708c0 con 0x7f3294003bf0 2024-04-25T14:17:07.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.515+0000 7f32b4200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac070740 con 0x7f32ac1614d0 2024-04-25T14:17:07.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.515+0000 7f32b4200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f32ac06fb50 con 0x7f32ac15f540 2024-04-25T14:17:07.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.515+0000 7f32b1800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3294003bf0 0x7f32ac15ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:07.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.515+0000 7f32b1800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3294003bf0 0x7f32ac15ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42120/0 (socket says 172.21.15.86:42120) 2024-04-25T14:17:07.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.515+0000 7f32b1800640 1 -- 172.21.15.86:0/704512196 learned_addr learned my addr 172.21.15.86:0/704512196 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:07.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.515+0000 7f32b2200640 1 --2- 172.21.15.86:0/704512196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32ac15f540 0x7f32ac15f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:07.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.515+0000 7f32b0e00640 1 --2- 172.21.15.86:0/704512196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32ac1614d0 0x7f32ac1618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:07.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.515+0000 7f32b1800640 1 -- 172.21.15.86:0/704512196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32ac15f540 msgr2=0x7f32ac15f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:07.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.515+0000 7f32b1800640 1 --2- 172.21.15.86:0/704512196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32ac15f540 0x7f32ac15f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:07.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.515+0000 7f32b1800640 1 -- 172.21.15.86:0/704512196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32ac1614d0 msgr2=0x7f32ac1618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:07.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.515+0000 7f32b1800640 1 --2- 172.21.15.86:0/704512196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32ac1614d0 0x7f32ac1618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:07.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.515+0000 7f32b1800640 1 -- 172.21.15.86:0/704512196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ac15fe90 con 0x7f3294003bf0 2024-04-25T14:17:07.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.515+0000 7f32b0e00640 1 --2- 172.21.15.86:0/704512196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32ac1614d0 0x7f32ac1618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:17:07.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.515+0000 7f32b1800640 1 --2- 172.21.15.86:0/704512196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3294003bf0 0x7f32ac15ad80 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f32a001ca20 tx=0x7f3294005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:07.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.515+0000 7f32a9600640 1 -- 172.21.15.86:0/704512196 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a00059a0 con 0x7f3294003bf0 2024-04-25T14:17:07.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.515+0000 7f32a9600640 1 -- 172.21.15.86:0/704512196 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a00bd070 con 0x7f3294003bf0 2024-04-25T14:17:07.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.515+0000 7f32a9600640 1 -- 172.21.15.86:0/704512196 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0005d40 con 0x7f3294003bf0 2024-04-25T14:17:07.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.515+0000 7f32b4200640 1 -- 172.21.15.86:0/704512196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32ac162200 con 0x7f3294003bf0 2024-04-25T14:17:07.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.515+0000 7f32b4200640 1 -- 172.21.15.86:0/704512196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32ac155c20 con 0x7f3294003bf0 2024-04-25T14:17:07.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.519+0000 7f32b4200640 1 -- 172.21.15.86:0/704512196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32ac069d00 con 0x7f3294003bf0 2024-04-25T14:17:07.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.519+0000 7f32a9600640 1 -- 172.21.15.86:0/704512196 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f32a00a5690 con 0x7f3294003bf0 2024-04-25T14:17:07.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.519+0000 7f32a9600640 1 --2- 172.21.15.86:0/704512196 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f328c06f450 0x7f328c071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:07.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.519+0000 7f32a9600640 1 -- 172.21.15.86:0/704512196 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f32a0129b10 con 0x7f3294003bf0 2024-04-25T14:17:07.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.519+0000 7f32b0e00640 1 --2- 172.21.15.86:0/704512196 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f328c06f450 0x7f328c071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:07.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.519+0000 7f32b0e00640 1 --2- 172.21.15.86:0/704512196 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f328c06f450 0x7f328c071960 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f32ac1fd1d0 tx=0x7f32980010b0 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:07.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.523+0000 7f32a9600640 1 -- 172.21.15.86:0/704512196 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f32ac15fe90 con 0x7f3294003bf0 2024-04-25T14:17:07.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.711+0000 7f32b4200640 1 -- 172.21.15.86:0/704512196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32ac0b4030 con 0x7f3294003bf0 2024-04-25T14:17:07.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.715+0000 7f32a9600640 1 -- 172.21.15.86:0/704512196 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f32a0016900 con 0x7f3294003bf0 2024-04-25T14:17:07.711 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:07.711 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:07.712 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:07.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.715+0000 7f32b4200640 1 -- 172.21.15.86:0/704512196 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f328c06f450 msgr2=0x7f328c071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:07.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.715+0000 7f32b4200640 1 --2- 172.21.15.86:0/704512196 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f328c06f450 0x7f328c071960 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f32ac1fd1d0 tx=0x7f32980010b0 comp rx=0 tx=0).stop 2024-04-25T14:17:07.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.715+0000 7f32b4200640 1 -- 172.21.15.86:0/704512196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3294003bf0 msgr2=0x7f32ac15ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:07.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.715+0000 7f32b4200640 1 --2- 172.21.15.86:0/704512196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3294003bf0 0x7f32ac15ad80 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f32a001ca20 tx=0x7f3294005840 comp rx=0 tx=0).stop 2024-04-25T14:17:07.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.715+0000 7f32b4200640 1 -- 172.21.15.86:0/704512196 shutdown_connections 2024-04-25T14:17:07.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.715+0000 7f32b4200640 1 --2- 172.21.15.86:0/704512196 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f328c06f450 0x7f328c071960 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:07.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.715+0000 7f32b4200640 1 --2- 172.21.15.86:0/704512196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32ac15f540 0x7f32ac15f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:07.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.715+0000 7f32b4200640 1 --2- 172.21.15.86:0/704512196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32ac1614d0 0x7f32ac1618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:07.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.715+0000 7f32b4200640 1 --2- 172.21.15.86:0/704512196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3294003bf0 0x7f32ac15ad80 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:07.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.715+0000 7f32b4200640 1 -- 172.21.15.86:0/704512196 >> 172.21.15.86:0/704512196 conn(0x7f32ac0b4650 msgr2=0x7f32ac0af420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:07.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.715+0000 7f32b4200640 1 -- 172.21.15.86:0/704512196 shutdown_connections 2024-04-25T14:17:07.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:07.715+0000 7f32b4200640 1 -- 172.21.15.86:0/704512196 wait complete. 2024-04-25T14:17:08.135 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:08.136 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:09.137 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:09.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.299+0000 7f8388000640 1 Processor -- start 2024-04-25T14:17:09.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.299+0000 7f8388000640 1 -- start start 2024-04-25T14:17:09.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.303+0000 7f8388000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f83800d1df0 0x7f83800cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:09.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.303+0000 7f8388000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83800cdcc0 0x7f83800ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:09.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.303+0000 7f8388000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f838015e5c0 con 0x7f8380070330 2024-04-25T14:17:09.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.303+0000 7f8388000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f838006fd40 con 0x7f83800d1df0 2024-04-25T14:17:09.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.303+0000 7f8388000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f838015e8a0 con 0x7f83800cdcc0 2024-04-25T14:17:09.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.303+0000 7f8384c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f83800d1df0 0x7f83800cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:09.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.303+0000 7f8384c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f83800d1df0 0x7f83800cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37632/0 (socket says 172.21.15.86:37632) 2024-04-25T14:17:09.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.303+0000 7f8384c00640 1 -- 172.21.15.86:0/4287867154 learned_addr learned my addr 172.21.15.86:0/4287867154 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:09.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.303+0000 7f8386000640 1 --2- 172.21.15.86:0/4287867154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83800cdcc0 0x7f83800ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:09.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.303+0000 7f8384c00640 1 -- 172.21.15.86:0/4287867154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83800cdcc0 msgr2=0x7f83800ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:09.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.303+0000 7f8384c00640 1 --2- 172.21.15.86:0/4287867154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83800cdcc0 0x7f83800ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:09.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.303+0000 7f8384c00640 1 -- 172.21.15.86:0/4287867154 >> v1:172.21.15.86:6789/0 conn(0x7f8380070330 legacy=0x7f83800d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:09.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.303+0000 7f8384c00640 1 -- 172.21.15.86:0/4287867154 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f838015eb80 con 0x7f83800d1df0 2024-04-25T14:17:09.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.303+0000 7f8386000640 1 --2- 172.21.15.86:0/4287867154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83800cdcc0 0x7f83800ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:17:09.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.307+0000 7f8384c00640 1 --2- 172.21.15.86:0/4287867154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83800d1df0 0x7f83800cd780 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f836c009870 tx=0x7f836c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc14d8f7a92a231 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:09.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.307+0000 7f837fe00640 1 -- 172.21.15.86:0/4287867154 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c0ab680 con 0x7f83800d1df0 2024-04-25T14:17:09.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.307+0000 7f837fe00640 1 -- 172.21.15.86:0/4287867154 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f836c0b0070 con 0x7f83800d1df0 2024-04-25T14:17:09.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.307+0000 7f837fe00640 1 -- 172.21.15.86:0/4287867154 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c0aba40 con 0x7f83800d1df0 2024-04-25T14:17:09.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.307+0000 7f8388000640 1 -- 172.21.15.86:0/4287867154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83800d1df0 msgr2=0x7f83800cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:09.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.307+0000 7f8388000640 1 --2- 172.21.15.86:0/4287867154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83800d1df0 0x7f83800cd780 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f836c009870 tx=0x7f836c01c4a0 comp rx=0 tx=0).stop 2024-04-25T14:17:09.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.307+0000 7f8388000640 1 -- 172.21.15.86:0/4287867154 shutdown_connections 2024-04-25T14:17:09.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.307+0000 7f8388000640 1 --2- 172.21.15.86:0/4287867154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83800cdcc0 0x7f83800ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:09.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.307+0000 7f8388000640 1 --2- 172.21.15.86:0/4287867154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83800d1df0 0x7f83800cd780 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:09.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.307+0000 7f8388000640 1 -- 172.21.15.86:0/4287867154 >> 172.21.15.86:0/4287867154 conn(0x7f83800b4650 msgr2=0x7f83800b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:09.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.307+0000 7f8388000640 1 -- 172.21.15.86:0/4287867154 shutdown_connections 2024-04-25T14:17:09.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.307+0000 7f8388000640 1 -- 172.21.15.86:0/4287867154 wait complete. 2024-04-25T14:17:09.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.307+0000 7f8388000640 1 Processor -- start 2024-04-25T14:17:09.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.307+0000 7f8388000640 1 -- start start 2024-04-25T14:17:09.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.311+0000 7f8388000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8380070330 0x7f83801680e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:09.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.311+0000 7f8388000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83800cdcc0 0x7f838015b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:09.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.311+0000 7f8388000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f838015b560 0x7f83800c2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:09.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.311+0000 7f8388000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f838015f4b0 con 0x7f83800cdcc0 2024-04-25T14:17:09.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.311+0000 7f8388000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f838015f330 con 0x7f8380070330 2024-04-25T14:17:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.311+0000 7f8388000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8380070740 con 0x7f838015b560 2024-04-25T14:17:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.311+0000 7f8384c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83800cdcc0 0x7f838015b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.311+0000 7f8384c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83800cdcc0 0x7f838015b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37634/0 (socket says 172.21.15.86:37634) 2024-04-25T14:17:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.311+0000 7f8384c00640 1 -- 172.21.15.86:0/3017748978 learned_addr learned my addr 172.21.15.86:0/3017748978 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.311+0000 7f8386000640 1 --2- 172.21.15.86:0/3017748978 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f838015b560 0x7f83800c2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.311+0000 7f8385600640 1 --2- 172.21.15.86:0/3017748978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8380070330 0x7f83801680e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.311+0000 7f8384c00640 1 -- 172.21.15.86:0/3017748978 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f838015b560 msgr2=0x7f83800c2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.311+0000 7f8384c00640 1 --2- 172.21.15.86:0/3017748978 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f838015b560 0x7f83800c2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.311+0000 7f8384c00640 1 -- 172.21.15.86:0/3017748978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8380070330 msgr2=0x7f83801680e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.311+0000 7f8384c00640 1 --2- 172.21.15.86:0/3017748978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8380070330 0x7f83801680e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.311+0000 7f8384c00640 1 -- 172.21.15.86:0/3017748978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83800af240 con 0x7f83800cdcc0 2024-04-25T14:17:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.311+0000 7f8384c00640 1 --2- 172.21.15.86:0/3017748978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83800cdcc0 0x7f838015b020 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f836c0ad040 tx=0x7f836c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:09.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.311+0000 7f837d600640 1 -- 172.21.15.86:0/3017748978 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c0b5810 con 0x7f83800cdcc0 2024-04-25T14:17:09.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.311+0000 7f837d600640 1 -- 172.21.15.86:0/3017748978 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f836c0a7070 con 0x7f83800cdcc0 2024-04-25T14:17:09.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.311+0000 7f837d600640 1 -- 172.21.15.86:0/3017748978 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c0b5c30 con 0x7f83800cdcc0 2024-04-25T14:17:09.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.311+0000 7f8388000640 1 -- 172.21.15.86:0/3017748978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83800af520 con 0x7f83800cdcc0 2024-04-25T14:17:09.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.311+0000 7f8388000640 1 -- 172.21.15.86:0/3017748978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8380072a20 con 0x7f83800cdcc0 2024-04-25T14:17:09.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.311+0000 7f8388000640 1 -- 172.21.15.86:0/3017748978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83800aefb0 con 0x7f83800cdcc0 2024-04-25T14:17:09.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.315+0000 7f837d600640 1 -- 172.21.15.86:0/3017748978 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f836c0bd020 con 0x7f83800cdcc0 2024-04-25T14:17:09.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.315+0000 7f837d600640 1 --2- 172.21.15.86:0/3017748978 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f836006f450 0x7f8360071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:09.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.315+0000 7f837d600640 1 -- 172.21.15.86:0/3017748978 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f836c129e20 con 0x7f83800cdcc0 2024-04-25T14:17:09.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.315+0000 7f8385600640 1 --2- 172.21.15.86:0/3017748978 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f836006f450 0x7f8360071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:09.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.315+0000 7f8385600640 1 --2- 172.21.15.86:0/3017748978 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f836006f450 0x7f8360071960 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8380072200 tx=0x7f8374009040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:09.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.319+0000 7f837d600640 1 -- 172.21.15.86:0/3017748978 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f836c016150 con 0x7f83800cdcc0 2024-04-25T14:17:09.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.507+0000 7f8388000640 1 -- 172.21.15.86:0/3017748978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8380069d00 con 0x7f83800cdcc0 2024-04-25T14:17:09.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.511+0000 7f837d600640 1 -- 172.21.15.86:0/3017748978 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f836c0f6990 con 0x7f83800cdcc0 2024-04-25T14:17:09.507 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:09.507 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:09.508 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:09.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.511+0000 7f8388000640 1 -- 172.21.15.86:0/3017748978 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f836006f450 msgr2=0x7f8360071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:09.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.511+0000 7f8388000640 1 --2- 172.21.15.86:0/3017748978 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f836006f450 0x7f8360071960 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8380072200 tx=0x7f8374009040 comp rx=0 tx=0).stop 2024-04-25T14:17:09.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.511+0000 7f8388000640 1 -- 172.21.15.86:0/3017748978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83800cdcc0 msgr2=0x7f838015b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:09.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.511+0000 7f8388000640 1 --2- 172.21.15.86:0/3017748978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83800cdcc0 0x7f838015b020 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f836c0ad040 tx=0x7f836c019240 comp rx=0 tx=0).stop 2024-04-25T14:17:09.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.511+0000 7f8388000640 1 -- 172.21.15.86:0/3017748978 shutdown_connections 2024-04-25T14:17:09.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.511+0000 7f8388000640 1 --2- 172.21.15.86:0/3017748978 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f836006f450 0x7f8360071960 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:09.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.511+0000 7f8388000640 1 --2- 172.21.15.86:0/3017748978 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f838015b560 0x7f83800c2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:09.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.511+0000 7f8388000640 1 --2- 172.21.15.86:0/3017748978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83800cdcc0 0x7f838015b020 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:09.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.511+0000 7f8388000640 1 --2- 172.21.15.86:0/3017748978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8380070330 0x7f83801680e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:09.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.511+0000 7f8388000640 1 -- 172.21.15.86:0/3017748978 >> 172.21.15.86:0/3017748978 conn(0x7f83800b4650 msgr2=0x7f83800c46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:09.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.511+0000 7f8388000640 1 -- 172.21.15.86:0/3017748978 shutdown_connections 2024-04-25T14:17:09.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:09.511+0000 7f8388000640 1 -- 172.21.15.86:0/3017748978 wait complete. 2024-04-25T14:17:09.932 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:09.932 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:10.933 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:11.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.091+0000 7f6f79800640 1 Processor -- start 2024-04-25T14:17:11.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f79800640 1 -- start start 2024-04-25T14:17:11.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f79800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6f74070330 0x7f6f740d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:11.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f79800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f740ce790 0x7f6f7406f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:11.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f79800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f740d2030 con 0x7f6f740d1c20 2024-04-25T14:17:11.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f79800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f6f7415e5c0 con 0x7f6f74070330 2024-04-25T14:17:11.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f79800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6f7415e8a0 con 0x7f6f740ce790 2024-04-25T14:17:11.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f6a000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6f740d1c20 0x7f6f7406ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39794/0 (socket says 172.21.15.86:39794) 2024-04-25T14:17:11.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f6a000640 1 -- 172.21.15.86:0/238847300 learned_addr learned my addr 172.21.15.86:0/238847300 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:17:11.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f73400640 1 --2- 172.21.15.86:0/238847300 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f740ce790 0x7f6f7406f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:11.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f72a00640 1 --2- 172.21.15.86:0/238847300 >> v2:172.21.15.155:3300/0 conn(0x7f6f74070330 0x7f6f740d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:11.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f72000640 1 -- 172.21.15.86:0/238847300 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2451569870 0 0) 0x7f6f5c001120 con 0x7f6f740d1c20 2024-04-25T14:17:11.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f72000640 1 -- 172.21.15.86:0/238847300 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f58003530 con 0x7f6f740d1c20 2024-04-25T14:17:11.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f72000640 1 -- 172.21.15.86:0/238847300 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3943496494 0 0) 0x7f6f58003530 con 0x7f6f740d1c20 2024-04-25T14:17:11.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f72000640 1 -- 172.21.15.86:0/238847300 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f740ce790 msgr2=0x7f6f7406f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:11.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f72000640 1 --2- 172.21.15.86:0/238847300 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f740ce790 0x7f6f7406f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:11.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f72000640 1 -- 172.21.15.86:0/238847300 >> v2:172.21.15.155:3300/0 conn(0x7f6f74070330 msgr2=0x7f6f740d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:11.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f72000640 1 --2- 172.21.15.86:0/238847300 >> v2:172.21.15.155:3300/0 conn(0x7f6f74070330 0x7f6f740d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:11.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f72000640 1 -- 172.21.15.86:0/238847300 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f7415eb80 con 0x7f6f740d1c20 2024-04-25T14:17:11.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f72000640 1 -- 172.21.15.86:0/238847300 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f6f5c001850 con 0x7f6f740d1c20 2024-04-25T14:17:11.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f72000640 1 -- 172.21.15.86:0/238847300 >> v1:172.21.15.86:6789/0 conn(0x7f6f740d1c20 legacy=0x7f6f7406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:11.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f72000640 1 --2- 172.21.15.86:0/238847300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f58003a30 0x7f6f58003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:11.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f72000640 1 --2- 172.21.15.86:0/238847300 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f740ce790 0x7f6f58004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:11.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f72000640 1 --2- 172.21.15.86:0/238847300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f58004d50 0x7f6f58005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:11.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f72000640 1 -- 172.21.15.86:0/238847300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7415e5c0 con 0x7f6f58003a30 2024-04-25T14:17:11.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f72000640 1 -- 172.21.15.86:0/238847300 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7415e8a0 con 0x7f6f58004d50 2024-04-25T14:17:11.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f72000640 1 -- 172.21.15.86:0/238847300 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6f5800bb00 con 0x7f6f740ce790 2024-04-25T14:17:11.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f72a00640 1 --2- 172.21.15.86:0/238847300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f58003a30 0x7f6f58003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:11.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f6a000640 1 --2- 172.21.15.86:0/238847300 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f740ce790 0x7f6f58004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:11.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f73400640 1 --2- 172.21.15.86:0/238847300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f58004d50 0x7f6f58005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:11.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f72a00640 1 -- 172.21.15.86:0/238847300 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f740ce790 msgr2=0x7f6f58004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:11.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f72a00640 1 --2- 172.21.15.86:0/238847300 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f740ce790 0x7f6f58004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:11.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f72a00640 1 -- 172.21.15.86:0/238847300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f58004d50 msgr2=0x7f6f58005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:11.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f72a00640 1 --2- 172.21.15.86:0/238847300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f58004d50 0x7f6f58005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:11.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f72a00640 1 -- 172.21.15.86:0/238847300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f58003530 con 0x7f6f58003a30 2024-04-25T14:17:11.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.095+0000 7f6f6a000640 1 --2- 172.21.15.86:0/238847300 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f740ce790 0x7f6f58004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:17:11.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.099+0000 7f6f72a00640 1 --2- 172.21.15.86:0/238847300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f58003a30 0x7f6f58003ea0 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009ab0 tx=0x7f6f6401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc46ad189705a923 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:11.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.099+0000 7f6f72000640 1 -- 172.21.15.86:0/238847300 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f640b2070 con 0x7f6f58003a30 2024-04-25T14:17:11.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.099+0000 7f6f72000640 1 -- 172.21.15.86:0/238847300 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f640a5530 con 0x7f6f58003a30 2024-04-25T14:17:11.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.099+0000 7f6f72000640 1 -- 172.21.15.86:0/238847300 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f640096b0 con 0x7f6f58003a30 2024-04-25T14:17:11.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.099+0000 7f6f79800640 1 -- 172.21.15.86:0/238847300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f58003a30 msgr2=0x7f6f58003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:11.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.099+0000 7f6f79800640 1 --2- 172.21.15.86:0/238847300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f58003a30 0x7f6f58003ea0 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009ab0 tx=0x7f6f6401ca90 comp rx=0 tx=0).stop 2024-04-25T14:17:11.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.099+0000 7f6f73400640 1 -- 172.21.15.86:0/238847300 reap_dead start 2024-04-25T14:17:11.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.099+0000 7f6f79800640 1 -- 172.21.15.86:0/238847300 shutdown_connections 2024-04-25T14:17:11.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.099+0000 7f6f79800640 1 -- 172.21.15.86:0/238847300 >> 172.21.15.86:0/238847300 conn(0x7f6f740b4650 msgr2=0x7f6f740b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:11.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.099+0000 7f6f79800640 1 -- 172.21.15.86:0/238847300 shutdown_connections 2024-04-25T14:17:11.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.099+0000 7f6f79800640 1 -- 172.21.15.86:0/238847300 wait complete. 2024-04-25T14:17:11.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.103+0000 7f6f79800640 1 Processor -- start 2024-04-25T14:17:11.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.103+0000 7f6f79800640 1 -- start start 2024-04-25T14:17:11.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.103+0000 7f6f79800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f58003a30 0x7f6f74159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:11.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.103+0000 7f6f79800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f741fcc40 0x7f6f740ca940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:11.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.103+0000 7f6f79800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f741599d0 0x7f6f740cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:11.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.103+0000 7f6f79800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f740708c0 con 0x7f6f58003a30 2024-04-25T14:17:11.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.103+0000 7f6f79800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6f74070740 con 0x7f6f741599d0 2024-04-25T14:17:11.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.103+0000 7f6f79800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6f741687a0 con 0x7f6f741fcc40 2024-04-25T14:17:11.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.103+0000 7f6f72a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f58003a30 0x7f6f74159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:11.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.103+0000 7f6f72a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f58003a30 0x7f6f74159490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37654/0 (socket says 172.21.15.86:37654) 2024-04-25T14:17:11.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.103+0000 7f6f72a00640 1 -- 172.21.15.86:0/3207136367 learned_addr learned my addr 172.21.15.86:0/3207136367 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:11.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.103+0000 7f6f6a000640 1 --2- 172.21.15.86:0/3207136367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f741fcc40 0x7f6f740ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:11.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.103+0000 7f6f73400640 1 --2- 172.21.15.86:0/3207136367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f741599d0 0x7f6f740cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:11.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.103+0000 7f6f72a00640 1 -- 172.21.15.86:0/3207136367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f741fcc40 msgr2=0x7f6f740ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:11.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.103+0000 7f6f72a00640 1 --2- 172.21.15.86:0/3207136367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f741fcc40 0x7f6f740ca940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:11.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.103+0000 7f6f72a00640 1 -- 172.21.15.86:0/3207136367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f741599d0 msgr2=0x7f6f740cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:11.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.107+0000 7f6f72a00640 1 --2- 172.21.15.86:0/3207136367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f741599d0 0x7f6f740cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:11.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.107+0000 7f6f72a00640 1 -- 172.21.15.86:0/3207136367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f740b2050 con 0x7f6f58003a30 2024-04-25T14:17:11.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.107+0000 7f6f6a000640 1 --2- 172.21.15.86:0/3207136367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f741fcc40 0x7f6f740ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:17:11.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.107+0000 7f6f73400640 1 --2- 172.21.15.86:0/3207136367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f741599d0 0x7f6f740cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:17:11.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.107+0000 7f6f72a00640 1 --2- 172.21.15.86:0/3207136367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f58003a30 0x7f6f74159490 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f6f6401c8c0 tx=0x7f6f58005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:11.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.107+0000 7f6f6b400640 1 -- 172.21.15.86:0/3207136367 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f640bb020 con 0x7f6f58003a30 2024-04-25T14:17:11.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.107+0000 7f6f6b400640 1 -- 172.21.15.86:0/3207136367 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f640ad440 con 0x7f6f58003a30 2024-04-25T14:17:11.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.107+0000 7f6f79800640 1 -- 172.21.15.86:0/3207136367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f740b2330 con 0x7f6f58003a30 2024-04-25T14:17:11.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.107+0000 7f6f6b400640 1 -- 172.21.15.86:0/3207136367 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f640b2040 con 0x7f6f58003a30 2024-04-25T14:17:11.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.107+0000 7f6f79800640 1 -- 172.21.15.86:0/3207136367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f7409bea0 con 0x7f6f58003a30 2024-04-25T14:17:11.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.107+0000 7f6f79800640 1 -- 172.21.15.86:0/3207136367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f74069d00 con 0x7f6f58003a30 2024-04-25T14:17:11.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.107+0000 7f6f6b400640 1 -- 172.21.15.86:0/3207136367 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f6f640be070 con 0x7f6f58003a30 2024-04-25T14:17:11.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.107+0000 7f6f6b400640 1 --2- 172.21.15.86:0/3207136367 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f6f5006f450 0x7f6f50071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:11.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.107+0000 7f6f6b400640 1 -- 172.21.15.86:0/3207136367 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f6f641289f0 con 0x7f6f58003a30 2024-04-25T14:17:11.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.107+0000 7f6f6a000640 1 --2- 172.21.15.86:0/3207136367 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f6f5006f450 0x7f6f50071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:11.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.111+0000 7f6f6a000640 1 --2- 172.21.15.86:0/3207136367 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f6f5006f450 0x7f6f50071960 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c000f70 tx=0x7f6f5c000ee0 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:11.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.115+0000 7f6f6b400640 1 -- 172.21.15.86:0/3207136367 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6f640167a0 con 0x7f6f58003a30 2024-04-25T14:17:11.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.299+0000 7f6f79800640 1 -- 172.21.15.86:0/3207136367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f74076bf0 con 0x7f6f58003a30 2024-04-25T14:17:11.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.303+0000 7f6f6b400640 1 -- 172.21.15.86:0/3207136367 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f6f640f55f0 con 0x7f6f58003a30 2024-04-25T14:17:11.300 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:11.300 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:11.300 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:11.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.303+0000 7f6f79800640 1 -- 172.21.15.86:0/3207136367 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f6f5006f450 msgr2=0x7f6f50071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:11.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.303+0000 7f6f79800640 1 --2- 172.21.15.86:0/3207136367 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f6f5006f450 0x7f6f50071960 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c000f70 tx=0x7f6f5c000ee0 comp rx=0 tx=0).stop 2024-04-25T14:17:11.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.303+0000 7f6f79800640 1 -- 172.21.15.86:0/3207136367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f58003a30 msgr2=0x7f6f74159490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:11.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.303+0000 7f6f79800640 1 --2- 172.21.15.86:0/3207136367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f58003a30 0x7f6f74159490 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f6f6401c8c0 tx=0x7f6f58005840 comp rx=0 tx=0).stop 2024-04-25T14:17:11.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.303+0000 7f6f79800640 1 -- 172.21.15.86:0/3207136367 shutdown_connections 2024-04-25T14:17:11.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.303+0000 7f6f79800640 1 --2- 172.21.15.86:0/3207136367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f741599d0 0x7f6f740cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:11.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.303+0000 7f6f79800640 1 --2- 172.21.15.86:0/3207136367 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f6f5006f450 0x7f6f50071960 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:11.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.303+0000 7f6f79800640 1 --2- 172.21.15.86:0/3207136367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f741fcc40 0x7f6f740ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:11.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.303+0000 7f6f79800640 1 --2- 172.21.15.86:0/3207136367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f58003a30 0x7f6f74159490 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:11.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.303+0000 7f6f79800640 1 -- 172.21.15.86:0/3207136367 >> 172.21.15.86:0/3207136367 conn(0x7f6f740b4650 msgr2=0x7f6f740af720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:11.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.303+0000 7f6f79800640 1 -- 172.21.15.86:0/3207136367 shutdown_connections 2024-04-25T14:17:11.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:11.303+0000 7f6f79800640 1 -- 172.21.15.86:0/3207136367 wait complete. 2024-04-25T14:17:11.725 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:11.725 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:12.726 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:12.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.887+0000 7fe8bda00640 1 Processor -- start 2024-04-25T14:17:12.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.887+0000 7fe8bda00640 1 -- start start 2024-04-25T14:17:12.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8bda00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8b8070330 0x7fe8b80d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:12.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8bda00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe8b80ce790 0x7fe8b806f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:12.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8bda00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8b80d2030 con 0x7fe8b80d1c20 2024-04-25T14:17:12.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8bda00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe8b815e5c0 con 0x7fe8b80ce790 2024-04-25T14:17:12.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8bda00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe8b815e8a0 con 0x7fe8b8070330 2024-04-25T14:17:12.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8ae000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe8b80d1c20 0x7fe8b806ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39804/0 (socket says 172.21.15.86:39804) 2024-04-25T14:17:12.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8ae000640 1 -- 172.21.15.86:0/3919776057 learned_addr learned my addr 172.21.15.86:0/3919776057 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:17:12.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8b7400640 1 --2- 172.21.15.86:0/3919776057 >> v2:172.21.15.155:3300/0 conn(0x7fe8b80ce790 0x7fe8b806f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:12.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8b6a00640 1 --2- 172.21.15.86:0/3919776057 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8b8070330 0x7fe8b80d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:12.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8b6000640 1 -- 172.21.15.86:0/3919776057 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3489581396 0 0) 0x7fe8a0001120 con 0x7fe8b80d1c20 2024-04-25T14:17:12.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8b6000640 1 -- 172.21.15.86:0/3919776057 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe89c003530 con 0x7fe8b80d1c20 2024-04-25T14:17:12.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8b6000640 1 -- 172.21.15.86:0/3919776057 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2252029996 0 0) 0x7fe89c003530 con 0x7fe8b80d1c20 2024-04-25T14:17:12.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8b6000640 1 -- 172.21.15.86:0/3919776057 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8b8070330 msgr2=0x7fe8b80d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:12.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8b6000640 1 --2- 172.21.15.86:0/3919776057 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8b8070330 0x7fe8b80d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:12.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8b6000640 1 -- 172.21.15.86:0/3919776057 >> v2:172.21.15.155:3300/0 conn(0x7fe8b80ce790 msgr2=0x7fe8b806f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:12.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8b6000640 1 --2- 172.21.15.86:0/3919776057 >> v2:172.21.15.155:3300/0 conn(0x7fe8b80ce790 0x7fe8b806f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:12.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8b6000640 1 -- 172.21.15.86:0/3919776057 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b815eb80 con 0x7fe8b80d1c20 2024-04-25T14:17:12.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8b6000640 1 -- 172.21.15.86:0/3919776057 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fe8a0001850 con 0x7fe8b80d1c20 2024-04-25T14:17:12.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8b6000640 1 -- 172.21.15.86:0/3919776057 >> v1:172.21.15.86:6789/0 conn(0x7fe8b80d1c20 legacy=0x7fe8b806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:12.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8b6000640 1 --2- 172.21.15.86:0/3919776057 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8b8070330 0x7fe89c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:12.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8b6000640 1 --2- 172.21.15.86:0/3919776057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe89c003fb0 0x7fe89c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:12.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8b6000640 1 --2- 172.21.15.86:0/3919776057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe89c004d50 0x7fe89c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:12.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8b6000640 1 -- 172.21.15.86:0/3919776057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b815e5c0 con 0x7fe89c003fb0 2024-04-25T14:17:12.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8b6000640 1 -- 172.21.15.86:0/3919776057 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b815e8a0 con 0x7fe89c004d50 2024-04-25T14:17:12.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8b6000640 1 -- 172.21.15.86:0/3919776057 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe89c00bb30 con 0x7fe8b8070330 2024-04-25T14:17:12.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8ae000640 1 --2- 172.21.15.86:0/3919776057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe89c003fb0 0x7fe89c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:12.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8b6a00640 1 --2- 172.21.15.86:0/3919776057 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8b8070330 0x7fe89c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:12.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8b7400640 1 --2- 172.21.15.86:0/3919776057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe89c004d50 0x7fe89c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:12.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8ae000640 1 -- 172.21.15.86:0/3919776057 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8b8070330 msgr2=0x7fe89c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:12.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8ae000640 1 --2- 172.21.15.86:0/3919776057 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8b8070330 0x7fe89c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:12.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8ae000640 1 -- 172.21.15.86:0/3919776057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe89c004d50 msgr2=0x7fe89c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:12.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8ae000640 1 --2- 172.21.15.86:0/3919776057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe89c004d50 0x7fe89c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:12.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.891+0000 7fe8ae000640 1 -- 172.21.15.86:0/3919776057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89c003530 con 0x7fe89c003fb0 2024-04-25T14:17:12.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.895+0000 7fe8ae000640 1 --2- 172.21.15.86:0/3919776057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe89c003fb0 0x7fe89c004810 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000c4c0 tx=0x7fe8a001cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73fdae39577d8858 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:12.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.895+0000 7fe8b6000640 1 -- 172.21.15.86:0/3919776057 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a000c0e0 con 0x7fe89c003fb0 2024-04-25T14:17:12.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.895+0000 7fe8b6000640 1 -- 172.21.15.86:0/3919776057 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8a00a9070 con 0x7fe89c003fb0 2024-04-25T14:17:12.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.895+0000 7fe8b6000640 1 -- 172.21.15.86:0/3919776057 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0005d80 con 0x7fe89c003fb0 2024-04-25T14:17:12.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.895+0000 7fe8bda00640 1 -- 172.21.15.86:0/3919776057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe89c003fb0 msgr2=0x7fe89c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:12.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.895+0000 7fe8bda00640 1 --2- 172.21.15.86:0/3919776057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe89c003fb0 0x7fe89c004810 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000c4c0 tx=0x7fe8a001cca0 comp rx=0 tx=0).stop 2024-04-25T14:17:12.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.895+0000 7fe8b7400640 1 -- 172.21.15.86:0/3919776057 reap_dead start 2024-04-25T14:17:12.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.895+0000 7fe8bda00640 1 -- 172.21.15.86:0/3919776057 shutdown_connections 2024-04-25T14:17:12.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.895+0000 7fe8bda00640 1 -- 172.21.15.86:0/3919776057 >> 172.21.15.86:0/3919776057 conn(0x7fe8b80b4650 msgr2=0x7fe8b80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:12.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.895+0000 7fe8bda00640 1 -- 172.21.15.86:0/3919776057 shutdown_connections 2024-04-25T14:17:12.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.895+0000 7fe8bda00640 1 -- 172.21.15.86:0/3919776057 wait complete. 2024-04-25T14:17:12.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.899+0000 7fe8bda00640 1 Processor -- start 2024-04-25T14:17:12.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.899+0000 7fe8bda00640 1 -- start start 2024-04-25T14:17:12.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.899+0000 7fe8bda00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe89c003fb0 0x7fe8b8160800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:12.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.899+0000 7fe8bda00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8b80b2450 0x7fe8b8165c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:12.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.899+0000 7fe8bda00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8b8166170 0x7fe8b815f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:12.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.899+0000 7fe8bda00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b80708c0 con 0x7fe8b80b2450 2024-04-25T14:17:12.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.899+0000 7fe8bda00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b8070740 con 0x7fe89c003fb0 2024-04-25T14:17:12.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.899+0000 7fe8bda00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe8b8168fb0 con 0x7fe8b8166170 2024-04-25T14:17:12.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.899+0000 7fe8ae000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8b80b2450 0x7fe8b8165c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:12.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.899+0000 7fe8ae000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8b80b2450 0x7fe8b8165c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37662/0 (socket says 172.21.15.86:37662) 2024-04-25T14:17:12.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.899+0000 7fe8ae000640 1 -- 172.21.15.86:0/365508985 learned_addr learned my addr 172.21.15.86:0/365508985 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:12.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.899+0000 7fe8b7400640 1 --2- 172.21.15.86:0/365508985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8b8166170 0x7fe8b815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:12.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.899+0000 7fe8b6a00640 1 --2- 172.21.15.86:0/365508985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe89c003fb0 0x7fe8b8160800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.899+0000 7fe8ae000640 1 -- 172.21.15.86:0/365508985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8b8166170 msgr2=0x7fe8b815f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.899+0000 7fe8ae000640 1 --2- 172.21.15.86:0/365508985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8b8166170 0x7fe8b815f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.899+0000 7fe8ae000640 1 -- 172.21.15.86:0/365508985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe89c003fb0 msgr2=0x7fe8b8160800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.899+0000 7fe8ae000640 1 --2- 172.21.15.86:0/365508985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe89c003fb0 0x7fe8b8160800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.899+0000 7fe8ae000640 1 -- 172.21.15.86:0/365508985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b8166580 con 0x7fe8b80b2450 2024-04-25T14:17:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.899+0000 7fe8b6a00640 1 --2- 172.21.15.86:0/365508985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe89c003fb0 0x7fe8b8160800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:17:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.899+0000 7fe8ae000640 1 --2- 172.21.15.86:0/365508985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8b80b2450 0x7fe8b8165c30 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000c4c0 tx=0x7fe89c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.899+0000 7fe8af400640 1 -- 172.21.15.86:0/365508985 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0001120 con 0x7fe8b80b2450 2024-04-25T14:17:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.899+0000 7fe8af400640 1 -- 172.21.15.86:0/365508985 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8a0005d60 con 0x7fe8b80b2450 2024-04-25T14:17:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.899+0000 7fe8af400640 1 -- 172.21.15.86:0/365508985 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0001810 con 0x7fe8b80b2450 2024-04-25T14:17:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.899+0000 7fe8bda00640 1 -- 172.21.15.86:0/365508985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b8162240 con 0x7fe8b80b2450 2024-04-25T14:17:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.903+0000 7fe8bda00640 1 -- 172.21.15.86:0/365508985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b80b1c50 con 0x7fe8b80b2450 2024-04-25T14:17:12.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.903+0000 7fe8bda00640 1 -- 172.21.15.86:0/365508985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8b8161fb0 con 0x7fe8b80b2450 2024-04-25T14:17:12.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.903+0000 7fe8af400640 1 -- 172.21.15.86:0/365508985 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7fe8a00a5780 con 0x7fe8b80b2450 2024-04-25T14:17:12.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.907+0000 7fe8af400640 1 --2- 172.21.15.86:0/365508985 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe89406f480 0x7fe894071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:12.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.907+0000 7fe8af400640 1 -- 172.21.15.86:0/365508985 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fe8a01289e0 con 0x7fe8b80b2450 2024-04-25T14:17:12.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.907+0000 7fe8b6a00640 1 --2- 172.21.15.86:0/365508985 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe89406f480 0x7fe894071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:12.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.907+0000 7fe8b6a00640 1 --2- 172.21.15.86:0/365508985 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe89406f480 0x7fe894071990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8009160 tx=0x7fe8a8008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:12.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:12.907+0000 7fe8af400640 1 -- 172.21.15.86:0/365508985 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe8a00c0070 con 0x7fe8b80b2450 2024-04-25T14:17:13.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:13.099+0000 7fe8bda00640 1 -- 172.21.15.86:0/365508985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8b8069d00 con 0x7fe8b80b2450 2024-04-25T14:17:13.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:13.099+0000 7fe8af400640 1 -- 172.21.15.86:0/365508985 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7fe8b80d2030 con 0x7fe8b80b2450 2024-04-25T14:17:13.098 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:13.098 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:13.098 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:13.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:13.103+0000 7fe8bda00640 1 -- 172.21.15.86:0/365508985 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe89406f480 msgr2=0x7fe894071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:13.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:13.103+0000 7fe8bda00640 1 --2- 172.21.15.86:0/365508985 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe89406f480 0x7fe894071990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8009160 tx=0x7fe8a8008040 comp rx=0 tx=0).stop 2024-04-25T14:17:13.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:13.103+0000 7fe8bda00640 1 -- 172.21.15.86:0/365508985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8b80b2450 msgr2=0x7fe8b8165c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:13.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:13.103+0000 7fe8bda00640 1 --2- 172.21.15.86:0/365508985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8b80b2450 0x7fe8b8165c30 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000c4c0 tx=0x7fe89c005840 comp rx=0 tx=0).stop 2024-04-25T14:17:13.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:13.103+0000 7fe8bda00640 1 -- 172.21.15.86:0/365508985 shutdown_connections 2024-04-25T14:17:13.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:13.103+0000 7fe8bda00640 1 --2- 172.21.15.86:0/365508985 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fe89406f480 0x7fe894071990 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:13.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:13.103+0000 7fe8bda00640 1 --2- 172.21.15.86:0/365508985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8b8166170 0x7fe8b815f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:13.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:13.103+0000 7fe8bda00640 1 --2- 172.21.15.86:0/365508985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8b80b2450 0x7fe8b8165c30 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:13.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:13.103+0000 7fe8bda00640 1 --2- 172.21.15.86:0/365508985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe89c003fb0 0x7fe8b8160800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:13.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:13.103+0000 7fe8bda00640 1 -- 172.21.15.86:0/365508985 >> 172.21.15.86:0/365508985 conn(0x7fe8b80b4650 msgr2=0x7fe8b815cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:13.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:13.103+0000 7fe8bda00640 1 -- 172.21.15.86:0/365508985 shutdown_connections 2024-04-25T14:17:13.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:13.103+0000 7fe8bda00640 1 -- 172.21.15.86:0/365508985 wait complete. 2024-04-25T14:17:13.523 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:13.523 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:14.524 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:14.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.683+0000 7f7db3400640 1 Processor -- start 2024-04-25T14:17:14.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.683+0000 7f7db3400640 1 -- start start 2024-04-25T14:17:14.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7db3400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7dac0cfca0 0x7f7dac06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:14.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7db3400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dac072bd0 con 0x7f7dac0730b0 2024-04-25T14:17:14.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7db3400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dac06f930 con 0x7f7dac0d16e0 2024-04-25T14:17:14.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7db3400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f7dac070470 con 0x7f7dac0cfca0 2024-04-25T14:17:14.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7db1400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7dac0730b0 0x7f7dac0734c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39816/0 (socket says 172.21.15.86:39816) 2024-04-25T14:17:14.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7db1400640 1 -- 172.21.15.86:0/8427946 learned_addr learned my addr 172.21.15.86:0/8427946 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:17:14.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7db0a00640 1 --2- 172.21.15.86:0/8427946 >> v2:172.21.15.155:3300/0 conn(0x7f7dac0cfca0 0x7f7dac06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:14.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7dabe00640 1 -- 172.21.15.86:0/8427946 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2105316705 0 0) 0x7f7dac072bd0 con 0x7f7dac0730b0 2024-04-25T14:17:14.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7dabe00640 1 -- 172.21.15.86:0/8427946 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d90005740 con 0x7f7dac0730b0 2024-04-25T14:17:14.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7dabe00640 1 -- 172.21.15.86:0/8427946 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2340090389 0 0) 0x7f7dac06f930 con 0x7f7dac0d16e0 2024-04-25T14:17:14.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7dabe00640 1 -- 172.21.15.86:0/8427946 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7dac072bd0 con 0x7f7dac0d16e0 2024-04-25T14:17:14.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7dabe00640 1 -- 172.21.15.86:0/8427946 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 464761922 0 0) 0x7f7d90005740 con 0x7f7dac0730b0 2024-04-25T14:17:14.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7dabe00640 1 -- 172.21.15.86:0/8427946 >> v2:172.21.15.155:3300/0 conn(0x7f7dac0cfca0 msgr2=0x7f7dac06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:14.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7dabe00640 1 --2- 172.21.15.86:0/8427946 >> v2:172.21.15.155:3300/0 conn(0x7f7dac0cfca0 0x7f7dac06f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:14.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7dabe00640 1 -- 172.21.15.86:0/8427946 >> v1:172.21.15.155:6789/0 conn(0x7f7dac0d16e0 legacy=0x7f7dac0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:14.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7dabe00640 1 -- 172.21.15.86:0/8427946 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dac06fbd0 con 0x7f7dac0730b0 2024-04-25T14:17:14.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7dabe00640 1 -- 172.21.15.86:0/8427946 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f7da4001a00 con 0x7f7dac0730b0 2024-04-25T14:17:14.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7dabe00640 1 -- 172.21.15.86:0/8427946 >> v1:172.21.15.86:6789/0 conn(0x7f7dac0730b0 legacy=0x7f7dac0734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:14.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7dabe00640 1 --2- 172.21.15.86:0/8427946 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d90005dd0 0x7f7d90006220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:14.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7dabe00640 1 --2- 172.21.15.86:0/8427946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d90006a20 0x7f7d90006e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:14.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7dabe00640 1 --2- 172.21.15.86:0/8427946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d90007370 0x7f7d9000ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:14.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7dabe00640 1 -- 172.21.15.86:0/8427946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7dac070470 con 0x7f7d90007370 2024-04-25T14:17:14.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7dabe00640 1 -- 172.21.15.86:0/8427946 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7d900105d0 con 0x7f7d90006a20 2024-04-25T14:17:14.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7dabe00640 1 -- 172.21.15.86:0/8427946 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7d900107d0 con 0x7f7d90005dd0 2024-04-25T14:17:14.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7db1400640 1 --2- 172.21.15.86:0/8427946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d90007370 0x7f7d9000ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:14.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7db0a00640 1 --2- 172.21.15.86:0/8427946 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d90005dd0 0x7f7d90006220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:14.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7da3e00640 1 --2- 172.21.15.86:0/8427946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d90006a20 0x7f7d90006e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:14.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7db1400640 1 -- 172.21.15.86:0/8427946 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d90005dd0 msgr2=0x7f7d90006220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:14.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7db1400640 1 --2- 172.21.15.86:0/8427946 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d90005dd0 0x7f7d90006220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:14.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7db1400640 1 -- 172.21.15.86:0/8427946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d90006a20 msgr2=0x7f7d90006e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:14.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7db1400640 1 --2- 172.21.15.86:0/8427946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d90006a20 0x7f7d90006e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:14.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.687+0000 7f7db1400640 1 -- 172.21.15.86:0/8427946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d90005740 con 0x7f7d90007370 2024-04-25T14:17:14.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.691+0000 7f7db1400640 1 -- 172.21.15.86:0/8427946 reap_dead start 2024-04-25T14:17:14.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.691+0000 7f7db1400640 1 --2- 172.21.15.86:0/8427946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d90007370 0x7f7d9000ffe0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7f7da400a7c0 tx=0x7f7da401fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6cdca0325dffecf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:14.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.691+0000 7f7dabe00640 1 -- 172.21.15.86:0/8427946 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da40b0070 con 0x7f7d90007370 2024-04-25T14:17:14.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.691+0000 7f7dabe00640 1 -- 172.21.15.86:0/8427946 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7da40aa670 con 0x7f7d90007370 2024-04-25T14:17:14.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.691+0000 7f7dabe00640 1 -- 172.21.15.86:0/8427946 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da400a410 con 0x7f7d90007370 2024-04-25T14:17:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.691+0000 7f7db3400640 1 -- 172.21.15.86:0/8427946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d90007370 msgr2=0x7f7d9000ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.691+0000 7f7db3400640 1 --2- 172.21.15.86:0/8427946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d90007370 0x7f7d9000ffe0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7f7da400a7c0 tx=0x7f7da401fa40 comp rx=0 tx=0).stop 2024-04-25T14:17:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.691+0000 7f7db3400640 1 -- 172.21.15.86:0/8427946 shutdown_connections 2024-04-25T14:17:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.691+0000 7f7db3400640 1 --2- 172.21.15.86:0/8427946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d90007370 0x7f7d9000ffe0 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.691+0000 7f7db3400640 1 -- 172.21.15.86:0/8427946 >> 172.21.15.86:0/8427946 conn(0x7f7dac0b4650 msgr2=0x7f7dac0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.691+0000 7f7db3400640 1 -- 172.21.15.86:0/8427946 shutdown_connections 2024-04-25T14:17:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.691+0000 7f7db3400640 1 -- 172.21.15.86:0/8427946 wait complete. 2024-04-25T14:17:14.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.695+0000 7f7db3400640 1 Processor -- start 2024-04-25T14:17:14.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.695+0000 7f7db3400640 1 -- start start 2024-04-25T14:17:14.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.695+0000 7f7db3400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d90007370 0x7f7dac158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:14.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.695+0000 7f7db3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dac1f1e90 0x7f7dac15daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:14.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.695+0000 7f7db3400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7dac159c00 0x7f7dac15a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:14.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.695+0000 7f7db3400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7dac06f3f0 con 0x7f7dac1f1e90 2024-04-25T14:17:14.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.695+0000 7f7db3400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7dac070190 con 0x7f7dac159c00 2024-04-25T14:17:14.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.695+0000 7f7db3400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7dac06f570 con 0x7f7d90007370 2024-04-25T14:17:14.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.695+0000 7f7da3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dac1f1e90 0x7f7dac15daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:14.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.695+0000 7f7da3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dac1f1e90 0x7f7dac15daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37680/0 (socket says 172.21.15.86:37680) 2024-04-25T14:17:14.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.695+0000 7f7da3e00640 1 -- 172.21.15.86:0/2784683119 learned_addr learned my addr 172.21.15.86:0/2784683119 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:14.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.695+0000 7f7db0a00640 1 --2- 172.21.15.86:0/2784683119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d90007370 0x7f7dac158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:14.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.695+0000 7f7da3e00640 1 -- 172.21.15.86:0/2784683119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d90007370 msgr2=0x7f7dac158360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:14.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.695+0000 7f7db1400640 1 --2- 172.21.15.86:0/2784683119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7dac159c00 0x7f7dac15a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:14.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.695+0000 7f7da3e00640 1 --2- 172.21.15.86:0/2784683119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d90007370 0x7f7dac158360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:14.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.695+0000 7f7da3e00640 1 -- 172.21.15.86:0/2784683119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7dac159c00 msgr2=0x7f7dac15a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:14.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.695+0000 7f7da3e00640 1 --2- 172.21.15.86:0/2784683119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7dac159c00 0x7f7dac15a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:14.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.695+0000 7f7da3e00640 1 -- 172.21.15.86:0/2784683119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dac1588a0 con 0x7f7dac1f1e90 2024-04-25T14:17:14.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.695+0000 7f7db0a00640 1 --2- 172.21.15.86:0/2784683119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d90007370 0x7f7dac158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:17:14.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.695+0000 7f7db1400640 1 --2- 172.21.15.86:0/2784683119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7dac159c00 0x7f7dac15a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:17:14.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.695+0000 7f7da3e00640 1 --2- 172.21.15.86:0/2784683119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dac1f1e90 0x7f7dac15daf0 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400f7f0 tx=0x7f7d9400fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:14.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.695+0000 7f7da9600640 1 -- 172.21.15.86:0/2784683119 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d940050d0 con 0x7f7dac1f1e90 2024-04-25T14:17:14.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.699+0000 7f7da9600640 1 -- 172.21.15.86:0/2784683119 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d94018070 con 0x7f7dac1f1e90 2024-04-25T14:17:14.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.699+0000 7f7db3400640 1 -- 172.21.15.86:0/2784683119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dac15a550 con 0x7f7dac1f1e90 2024-04-25T14:17:14.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.699+0000 7f7da9600640 1 -- 172.21.15.86:0/2784683119 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d940054b0 con 0x7f7dac1f1e90 2024-04-25T14:17:14.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.699+0000 7f7db3400640 1 -- 172.21.15.86:0/2784683119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dac1f2c00 con 0x7f7dac1f1e90 2024-04-25T14:17:14.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.699+0000 7f7db3400640 1 -- 172.21.15.86:0/2784683119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dac1f2690 con 0x7f7dac1f1e90 2024-04-25T14:17:14.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.699+0000 7f7da9600640 1 -- 172.21.15.86:0/2784683119 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f7d9400d730 con 0x7f7dac1f1e90 2024-04-25T14:17:14.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.699+0000 7f7da9600640 1 --2- 172.21.15.86:0/2784683119 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7d8806f5f0 0x7f7d88071b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:14.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.699+0000 7f7da9600640 1 -- 172.21.15.86:0/2784683119 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f7d9408f3f0 con 0x7f7dac1f1e90 2024-04-25T14:17:14.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.699+0000 7f7db0a00640 1 --2- 172.21.15.86:0/2784683119 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7d8806f5f0 0x7f7d88071b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:14.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.703+0000 7f7db0a00640 1 --2- 172.21.15.86:0/2784683119 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7d8806f5f0 0x7f7d88071b00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7dac070710 tx=0x7f7d9c008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:14.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.703+0000 7f7da9600640 1 -- 172.21.15.86:0/2784683119 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7dac15a550 con 0x7f7dac1f1e90 2024-04-25T14:17:14.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.895+0000 7f7db3400640 1 -- 172.21.15.86:0/2784683119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7dac09f4b0 con 0x7f7dac1f1e90 2024-04-25T14:17:14.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.899+0000 7f7da9600640 1 -- 172.21.15.86:0/2784683119 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f7dac1588a0 con 0x7f7dac1f1e90 2024-04-25T14:17:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:14.895 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:14.896 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:14.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.899+0000 7f7db3400640 1 -- 172.21.15.86:0/2784683119 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7d8806f5f0 msgr2=0x7f7d88071b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:14.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.899+0000 7f7db3400640 1 --2- 172.21.15.86:0/2784683119 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7d8806f5f0 0x7f7d88071b00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7dac070710 tx=0x7f7d9c008040 comp rx=0 tx=0).stop 2024-04-25T14:17:14.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.899+0000 7f7db3400640 1 -- 172.21.15.86:0/2784683119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dac1f1e90 msgr2=0x7f7dac15daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:14.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.899+0000 7f7db3400640 1 --2- 172.21.15.86:0/2784683119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dac1f1e90 0x7f7dac15daf0 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400f7f0 tx=0x7f7d9400fcb0 comp rx=0 tx=0).stop 2024-04-25T14:17:14.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.899+0000 7f7db3400640 1 -- 172.21.15.86:0/2784683119 shutdown_connections 2024-04-25T14:17:14.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.899+0000 7f7db3400640 1 --2- 172.21.15.86:0/2784683119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7dac159c00 0x7f7dac15a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:14.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.899+0000 7f7db3400640 1 --2- 172.21.15.86:0/2784683119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dac1f1e90 0x7f7dac15daf0 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:14.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.899+0000 7f7db3400640 1 --2- 172.21.15.86:0/2784683119 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7d8806f5f0 0x7f7d88071b00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:14.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.899+0000 7f7db3400640 1 --2- 172.21.15.86:0/2784683119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d90007370 0x7f7dac158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:14.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.899+0000 7f7db3400640 1 -- 172.21.15.86:0/2784683119 >> 172.21.15.86:0/2784683119 conn(0x7f7dac0b4650 msgr2=0x7f7dac0b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:14.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.899+0000 7f7db3400640 1 -- 172.21.15.86:0/2784683119 shutdown_connections 2024-04-25T14:17:14.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:14.899+0000 7f7db3400640 1 -- 172.21.15.86:0/2784683119 wait complete. 2024-04-25T14:17:15.319 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:15.320 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:16.320 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:16.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.483+0000 7f0200200640 1 Processor -- start 2024-04-25T14:17:16.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.483+0000 7f0200200640 1 -- start start 2024-04-25T14:17:16.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.483+0000 7f0200200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f01f815c2a0 0x7f01f80ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:16.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.483+0000 7f0200200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01f806fd50 0x7f01f8070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:16.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.483+0000 7f0200200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01f80b0a60 con 0x7f01f806c670 2024-04-25T14:17:16.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.483+0000 7f0200200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f01f806ef90 con 0x7f01f815c2a0 2024-04-25T14:17:16.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.483+0000 7f0200200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f01f806f270 con 0x7f01f806fd50 2024-04-25T14:17:16.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.483+0000 7f01fe200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f01f806c670 0x7f01f806ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39826/0 (socket says 172.21.15.86:39826) 2024-04-25T14:17:16.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.483+0000 7f01fe200640 1 -- 172.21.15.86:0/3472133665 learned_addr learned my addr 172.21.15.86:0/3472133665 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:17:16.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.483+0000 7f01fce00640 1 --2- 172.21.15.86:0/3472133665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01f806fd50 0x7f01f8070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:16.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.483+0000 7f01fd800640 1 --2- 172.21.15.86:0/3472133665 >> v2:172.21.15.155:3300/0 conn(0x7f01f815c2a0 0x7f01f80ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:16.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.483+0000 7f01f7e00640 1 -- 172.21.15.86:0/3472133665 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2097844372 0 0) 0x7f01f80b0a60 con 0x7f01f806c670 2024-04-25T14:17:16.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.483+0000 7f01f7e00640 1 -- 172.21.15.86:0/3472133665 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01e0003530 con 0x7f01f806c670 2024-04-25T14:17:16.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.483+0000 7f01fce00640 1 -- 172.21.15.86:0/3472133665 >> v2:172.21.15.155:3300/0 conn(0x7f01f815c2a0 msgr2=0x7f01f80ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:16.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.483+0000 7f01fce00640 1 --2- 172.21.15.86:0/3472133665 >> v2:172.21.15.155:3300/0 conn(0x7f01f815c2a0 0x7f01f80ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:16.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.483+0000 7f01fce00640 1 -- 172.21.15.86:0/3472133665 >> v1:172.21.15.86:6789/0 conn(0x7f01f806c670 legacy=0x7f01f806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:16.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.483+0000 7f01fce00640 1 -- 172.21.15.86:0/3472133665 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f806f550 con 0x7f01f806fd50 2024-04-25T14:17:16.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.483+0000 7f01fd800640 1 --2- 172.21.15.86:0/3472133665 >> v2:172.21.15.155:3300/0 conn(0x7f01f815c2a0 0x7f01f80ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:17:16.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.487+0000 7f01fce00640 1 --2- 172.21.15.86:0/3472133665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01f806fd50 0x7f01f8070160 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f01e400a960 tx=0x7f01e401d7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1afdd54cb1866c99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:16.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.487+0000 7f01f7e00640 1 -- 172.21.15.86:0/3472133665 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e40b3070 con 0x7f01f806fd50 2024-04-25T14:17:16.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.487+0000 7f01f7e00640 1 -- 172.21.15.86:0/3472133665 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e40a67c0 con 0x7f01f806fd50 2024-04-25T14:17:16.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.487+0000 7f01f7e00640 1 -- 172.21.15.86:0/3472133665 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e400b040 con 0x7f01f806fd50 2024-04-25T14:17:16.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.487+0000 7f0200200640 1 -- 172.21.15.86:0/3472133665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01f806fd50 msgr2=0x7f01f8070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:16.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.487+0000 7f0200200640 1 --2- 172.21.15.86:0/3472133665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01f806fd50 0x7f01f8070160 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f01e400a960 tx=0x7f01e401d7b0 comp rx=0 tx=0).stop 2024-04-25T14:17:16.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.487+0000 7f0200200640 1 -- 172.21.15.86:0/3472133665 shutdown_connections 2024-04-25T14:17:16.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.487+0000 7f0200200640 1 --2- 172.21.15.86:0/3472133665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01f806fd50 0x7f01f8070160 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:16.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.487+0000 7f0200200640 1 --2- 172.21.15.86:0/3472133665 >> v2:172.21.15.155:3300/0 conn(0x7f01f815c2a0 0x7f01f80ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:16.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.487+0000 7f0200200640 1 -- 172.21.15.86:0/3472133665 >> 172.21.15.86:0/3472133665 conn(0x7f01f80b4650 msgr2=0x7f01f80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:16.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.487+0000 7f0200200640 1 -- 172.21.15.86:0/3472133665 shutdown_connections 2024-04-25T14:17:16.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.487+0000 7f0200200640 1 -- 172.21.15.86:0/3472133665 wait complete. 2024-04-25T14:17:16.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.491+0000 7f0200200640 1 Processor -- start 2024-04-25T14:17:16.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.491+0000 7f0200200640 1 -- start start 2024-04-25T14:17:16.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.491+0000 7f0200200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01f806c670 0x7f01f8164000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:16.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.491+0000 7f0200200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01f806fd50 0x7f01f80b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:16.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.491+0000 7f0200200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01f815c2a0 0x7f01f80b3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:16.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.491+0000 7f0200200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01f815c830 con 0x7f01f806fd50 2024-04-25T14:17:16.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.491+0000 7f0200200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f01f815c6b0 con 0x7f01f815c2a0 2024-04-25T14:17:16.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.491+0000 7f0200200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f01f80ae670 con 0x7f01f806c670 2024-04-25T14:17:16.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.491+0000 7f01fce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01f806fd50 0x7f01f80b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:16.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.491+0000 7f01fce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01f806fd50 0x7f01f80b16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37692/0 (socket says 172.21.15.86:37692) 2024-04-25T14:17:16.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.491+0000 7f01fce00640 1 -- 172.21.15.86:0/815465250 learned_addr learned my addr 172.21.15.86:0/815465250 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:16.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.491+0000 7f01fe200640 1 --2- 172.21.15.86:0/815465250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01f815c2a0 0x7f01f80b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:16.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.491+0000 7f01fd800640 1 --2- 172.21.15.86:0/815465250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01f806c670 0x7f01f8164000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:16.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.491+0000 7f01fce00640 1 -- 172.21.15.86:0/815465250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01f806c670 msgr2=0x7f01f8164000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:16.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.491+0000 7f01fce00640 1 --2- 172.21.15.86:0/815465250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01f806c670 0x7f01f8164000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:16.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.491+0000 7f01fce00640 1 -- 172.21.15.86:0/815465250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01f815c2a0 msgr2=0x7f01f80b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:16.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.491+0000 7f01fce00640 1 --2- 172.21.15.86:0/815465250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01f815c2a0 0x7f01f80b3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:16.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.491+0000 7f01fce00640 1 -- 172.21.15.86:0/815465250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f8164540 con 0x7f01f806fd50 2024-04-25T14:17:16.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.495+0000 7f01fce00640 1 --2- 172.21.15.86:0/815465250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01f806fd50 0x7f01f80b16f0 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f01e40b1040 tx=0x7f01e401a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:16.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.495+0000 7f01f5600640 1 -- 172.21.15.86:0/815465250 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e40bc070 con 0x7f01f806fd50 2024-04-25T14:17:16.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.495+0000 7f01f5600640 1 -- 172.21.15.86:0/815465250 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e40ae650 con 0x7f01f806fd50 2024-04-25T14:17:16.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.495+0000 7f0200200640 1 -- 172.21.15.86:0/815465250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f81649c0 con 0x7f01f806fd50 2024-04-25T14:17:16.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.495+0000 7f01f5600640 1 -- 172.21.15.86:0/815465250 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e40b3070 con 0x7f01f806fd50 2024-04-25T14:17:16.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.495+0000 7f0200200640 1 -- 172.21.15.86:0/815465250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f80b0ef0 con 0x7f01f806fd50 2024-04-25T14:17:16.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.495+0000 7f0200200640 1 -- 172.21.15.86:0/815465250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01f8164710 con 0x7f01f806fd50 2024-04-25T14:17:16.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.495+0000 7f01f5600640 1 -- 172.21.15.86:0/815465250 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f01e40bf070 con 0x7f01f806fd50 2024-04-25T14:17:16.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.495+0000 7f01f5600640 1 --2- 172.21.15.86:0/815465250 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f01d806f450 0x7f01d8071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:16.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.495+0000 7f01f5600640 1 -- 172.21.15.86:0/815465250 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f01e4128ef0 con 0x7f01f806fd50 2024-04-25T14:17:16.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.495+0000 7f01fd800640 1 --2- 172.21.15.86:0/815465250 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f01d806f450 0x7f01d8071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:16.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.499+0000 7f01fd800640 1 --2- 172.21.15.86:0/815465250 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f01d806f450 0x7f01d8071960 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f01f80b4200 tx=0x7f01ec00b040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:16.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.503+0000 7f01f5600640 1 -- 172.21.15.86:0/815465250 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f01e4017460 con 0x7f01f806fd50 2024-04-25T14:17:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.691+0000 7f0200200640 1 -- 172.21.15.86:0/815465250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01f8069d00 con 0x7f01f806fd50 2024-04-25T14:17:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.691+0000 7f01f5600640 1 -- 172.21.15.86:0/815465250 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f01e400a030 con 0x7f01f806fd50 2024-04-25T14:17:16.690 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:16.691 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:16.691 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:16.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.695+0000 7f0200200640 1 -- 172.21.15.86:0/815465250 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f01d806f450 msgr2=0x7f01d8071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:16.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.695+0000 7f0200200640 1 --2- 172.21.15.86:0/815465250 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f01d806f450 0x7f01d8071960 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f01f80b4200 tx=0x7f01ec00b040 comp rx=0 tx=0).stop 2024-04-25T14:17:16.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.695+0000 7f0200200640 1 -- 172.21.15.86:0/815465250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01f806fd50 msgr2=0x7f01f80b16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:16.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.695+0000 7f0200200640 1 --2- 172.21.15.86:0/815465250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01f806fd50 0x7f01f80b16f0 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f01e40b1040 tx=0x7f01e401a550 comp rx=0 tx=0).stop 2024-04-25T14:17:16.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.695+0000 7f0200200640 1 -- 172.21.15.86:0/815465250 shutdown_connections 2024-04-25T14:17:16.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.695+0000 7f0200200640 1 --2- 172.21.15.86:0/815465250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01f815c2a0 0x7f01f80b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:16.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.695+0000 7f0200200640 1 --2- 172.21.15.86:0/815465250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01f806fd50 0x7f01f80b16f0 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:16.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.695+0000 7f0200200640 1 --2- 172.21.15.86:0/815465250 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f01d806f450 0x7f01d8071960 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.695+0000 7f0200200640 1 --2- 172.21.15.86:0/815465250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01f806c670 0x7f01f8164000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.695+0000 7f0200200640 1 -- 172.21.15.86:0/815465250 >> 172.21.15.86:0/815465250 conn(0x7f01f80b4650 msgr2=0x7f01f8073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.695+0000 7f0200200640 1 -- 172.21.15.86:0/815465250 shutdown_connections 2024-04-25T14:17:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:16.695+0000 7f0200200640 1 -- 172.21.15.86:0/815465250 wait complete. 2024-04-25T14:17:17.116 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:17.116 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:18.117 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:18.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.283+0000 7f7050a00640 1 Processor -- start 2024-04-25T14:17:18.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.283+0000 7f7050a00640 1 -- start start 2024-04-25T14:17:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.287+0000 7f7050a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f704c0cfca0 0x7f704c06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.287+0000 7f7050a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f704c0730b0 0x7f704c0734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.287+0000 7f7050a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f704c072bd0 con 0x7f704c0d16e0 2024-04-25T14:17:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.287+0000 7f7050a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f704c070470 con 0x7f704c0730b0 2024-04-25T14:17:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.287+0000 7f7050a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f704c06f930 con 0x7f704c0cfca0 2024-04-25T14:17:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.287+0000 7f704a800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f704c0730b0 0x7f704c0734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:18.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.287+0000 7f704a800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f704c0730b0 0x7f704c0734c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37702/0 (socket says 172.21.15.86:37702) 2024-04-25T14:17:18.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.287+0000 7f704a800640 1 -- 172.21.15.86:0/3636926900 learned_addr learned my addr 172.21.15.86:0/3636926900 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:18.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.287+0000 7f7049e00640 1 --2- 172.21.15.86:0/3636926900 >> v2:172.21.15.155:3300/0 conn(0x7f704c0cfca0 0x7f704c06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:18.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.287+0000 7f704a800640 1 -- 172.21.15.86:0/3636926900 >> v2:172.21.15.155:3300/0 conn(0x7f704c0cfca0 msgr2=0x7f704c06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:18.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.287+0000 7f704a800640 1 --2- 172.21.15.86:0/3636926900 >> v2:172.21.15.155:3300/0 conn(0x7f704c0cfca0 0x7f704c06f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:18.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.287+0000 7f704a800640 1 -- 172.21.15.86:0/3636926900 >> v1:172.21.15.86:6789/0 conn(0x7f704c0d16e0 legacy=0x7f704c0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:18.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.287+0000 7f704a800640 1 -- 172.21.15.86:0/3636926900 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704c06fbd0 con 0x7f704c0730b0 2024-04-25T14:17:18.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.287+0000 7f704a800640 1 --2- 172.21.15.86:0/3636926900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f704c0730b0 0x7f704c0734c0 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f704c0d22a0 tx=0x7f704401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea47db287e15d04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:18.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.287+0000 7f7048a00640 1 -- 172.21.15.86:0/3636926900 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70440a57d0 con 0x7f704c0730b0 2024-04-25T14:17:18.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.291+0000 7f7048a00640 1 -- 172.21.15.86:0/3636926900 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70440a5970 con 0x7f704c0730b0 2024-04-25T14:17:18.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.291+0000 7f7048a00640 1 -- 172.21.15.86:0/3636926900 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70440a5d40 con 0x7f704c0730b0 2024-04-25T14:17:18.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.291+0000 7f7050a00640 1 -- 172.21.15.86:0/3636926900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f704c0730b0 msgr2=0x7f704c0734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:18.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.291+0000 7f7050a00640 1 --2- 172.21.15.86:0/3636926900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f704c0730b0 0x7f704c0734c0 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f704c0d22a0 tx=0x7f704401c9f0 comp rx=0 tx=0).stop 2024-04-25T14:17:18.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.291+0000 7f7050a00640 1 -- 172.21.15.86:0/3636926900 shutdown_connections 2024-04-25T14:17:18.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.291+0000 7f7050a00640 1 --2- 172.21.15.86:0/3636926900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f704c0730b0 0x7f704c0734c0 unknown :-1 s=CLOSED pgs=2356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:18.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.291+0000 7f7050a00640 1 --2- 172.21.15.86:0/3636926900 >> v2:172.21.15.155:3300/0 conn(0x7f704c0cfca0 0x7f704c06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:18.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.291+0000 7f7050a00640 1 -- 172.21.15.86:0/3636926900 >> 172.21.15.86:0/3636926900 conn(0x7f704c0b4650 msgr2=0x7f704c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:18.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.291+0000 7f7050a00640 1 -- 172.21.15.86:0/3636926900 shutdown_connections 2024-04-25T14:17:18.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.291+0000 7f7050a00640 1 -- 172.21.15.86:0/3636926900 wait complete. 2024-04-25T14:17:18.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.291+0000 7f7050a00640 1 Processor -- start 2024-04-25T14:17:18.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.291+0000 7f7050a00640 1 -- start start 2024-04-25T14:17:18.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.295+0000 7f7050a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f704c0730b0 0x7f704c0ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:18.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.295+0000 7f7050a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f704c0cfca0 0x7f704c157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:18.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.295+0000 7f7050a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f704c0d16e0 0x7f704c158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:18.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.295+0000 7f7050a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f704c169a70 con 0x7f704c0730b0 2024-04-25T14:17:18.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.295+0000 7f7050a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f704c1698f0 con 0x7f704c0cfca0 2024-04-25T14:17:18.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.295+0000 7f7050a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f704c169bf0 con 0x7f704c0d16e0 2024-04-25T14:17:18.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.295+0000 7f7049e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f704c0730b0 0x7f704c0ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:18.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.295+0000 7f7049e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f704c0730b0 0x7f704c0ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37708/0 (socket says 172.21.15.86:37708) 2024-04-25T14:17:18.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.295+0000 7f7049e00640 1 -- 172.21.15.86:0/3881506746 learned_addr learned my addr 172.21.15.86:0/3881506746 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:18.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.295+0000 7f7049400640 1 --2- 172.21.15.86:0/3881506746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f704c0cfca0 0x7f704c157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:18.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.295+0000 7f704a800640 1 --2- 172.21.15.86:0/3881506746 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f704c0d16e0 0x7f704c158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:18.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.295+0000 7f7049e00640 1 -- 172.21.15.86:0/3881506746 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f704c0d16e0 msgr2=0x7f704c158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:18.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.295+0000 7f7049e00640 1 --2- 172.21.15.86:0/3881506746 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f704c0d16e0 0x7f704c158230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:18.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.295+0000 7f7049e00640 1 -- 172.21.15.86:0/3881506746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f704c0cfca0 msgr2=0x7f704c157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:18.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.295+0000 7f7049e00640 1 --2- 172.21.15.86:0/3881506746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f704c0cfca0 0x7f704c157cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:18.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.295+0000 7f7049e00640 1 -- 172.21.15.86:0/3881506746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704c0ca570 con 0x7f704c0730b0 2024-04-25T14:17:18.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.295+0000 7f7049e00640 1 --2- 172.21.15.86:0/3881506746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f704c0730b0 0x7f704c0ce6e0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f703c00d5d0 tx=0x7f703c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:18.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.295+0000 7f7042000640 1 -- 172.21.15.86:0/3881506746 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c004240 con 0x7f704c0730b0 2024-04-25T14:17:18.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.295+0000 7f7042000640 1 -- 172.21.15.86:0/3881506746 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f703c0043e0 con 0x7f704c0730b0 2024-04-25T14:17:18.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.295+0000 7f7042000640 1 -- 172.21.15.86:0/3881506746 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c003840 con 0x7f704c0730b0 2024-04-25T14:17:18.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.295+0000 7f7050a00640 1 -- 172.21.15.86:0/3881506746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f704c0ca850 con 0x7f704c0730b0 2024-04-25T14:17:18.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.295+0000 7f7050a00640 1 -- 172.21.15.86:0/3881506746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f704c0cae10 con 0x7f704c0730b0 2024-04-25T14:17:18.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.295+0000 7f7050a00640 1 -- 172.21.15.86:0/3881506746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f704c09f4b0 con 0x7f704c0730b0 2024-04-25T14:17:18.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.299+0000 7f7042000640 1 -- 172.21.15.86:0/3881506746 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f703c0039e0 con 0x7f704c0730b0 2024-04-25T14:17:18.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.299+0000 7f7042000640 1 --2- 172.21.15.86:0/3881506746 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f702c06f480 0x7f702c071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:18.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.299+0000 7f7042000640 1 -- 172.21.15.86:0/3881506746 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f703c08f7c0 con 0x7f704c0730b0 2024-04-25T14:17:18.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.299+0000 7f7049400640 1 --2- 172.21.15.86:0/3881506746 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f702c06f480 0x7f702c071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:18.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.299+0000 7f7049400640 1 --2- 172.21.15.86:0/3881506746 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f702c06f480 0x7f702c071990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f704c0b2670 tx=0x7f7038009040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:18.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.303+0000 7f7042000640 1 -- 172.21.15.86:0/3881506746 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f703c05c3c0 con 0x7f704c0730b0 2024-04-25T14:17:18.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.491+0000 7f7050a00640 1 -- 172.21.15.86:0/3881506746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f704c069d00 con 0x7f704c0730b0 2024-04-25T14:17:18.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.491+0000 7f7042000640 1 -- 172.21.15.86:0/3881506746 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f703c05bcc0 con 0x7f704c0730b0 2024-04-25T14:17:18.489 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:18.489 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:18.489 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:18.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.491+0000 7f7050a00640 1 -- 172.21.15.86:0/3881506746 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f702c06f480 msgr2=0x7f702c071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:18.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.491+0000 7f7050a00640 1 --2- 172.21.15.86:0/3881506746 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f702c06f480 0x7f702c071990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f704c0b2670 tx=0x7f7038009040 comp rx=0 tx=0).stop 2024-04-25T14:17:18.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.491+0000 7f7050a00640 1 -- 172.21.15.86:0/3881506746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f704c0730b0 msgr2=0x7f704c0ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:18.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.491+0000 7f7050a00640 1 --2- 172.21.15.86:0/3881506746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f704c0730b0 0x7f704c0ce6e0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f703c00d5d0 tx=0x7f703c00da90 comp rx=0 tx=0).stop 2024-04-25T14:17:18.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.495+0000 7f7050a00640 1 -- 172.21.15.86:0/3881506746 shutdown_connections 2024-04-25T14:17:18.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.495+0000 7f7050a00640 1 --2- 172.21.15.86:0/3881506746 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f702c06f480 0x7f702c071990 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:18.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.495+0000 7f7050a00640 1 --2- 172.21.15.86:0/3881506746 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f704c0d16e0 0x7f704c158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:18.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.495+0000 7f7050a00640 1 --2- 172.21.15.86:0/3881506746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f704c0cfca0 0x7f704c157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:18.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.495+0000 7f7050a00640 1 --2- 172.21.15.86:0/3881506746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f704c0730b0 0x7f704c0ce6e0 unknown :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:18.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.495+0000 7f7050a00640 1 -- 172.21.15.86:0/3881506746 >> 172.21.15.86:0/3881506746 conn(0x7f704c0b4650 msgr2=0x7f704c0af460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:18.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.495+0000 7f7050a00640 1 -- 172.21.15.86:0/3881506746 shutdown_connections 2024-04-25T14:17:18.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:18.495+0000 7f7050a00640 1 -- 172.21.15.86:0/3881506746 wait complete. 2024-04-25T14:17:18.914 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:18.914 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:19.915 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:20.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.075+0000 7f4871200640 1 Processor -- start 2024-04-25T14:17:20.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.075+0000 7f4871200640 1 -- start start 2024-04-25T14:17:20.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.079+0000 7f4871200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f486c0cfca0 0x7f486c06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:20.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.079+0000 7f4871200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f486c0d16e0 0x7f486c0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:20.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.079+0000 7f4871200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f486c0d2030 con 0x7f486c06c670 2024-04-25T14:17:20.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.079+0000 7f4871200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f486c070470 con 0x7f486c0d16e0 2024-04-25T14:17:20.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.079+0000 7f4871200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f486c072bd0 con 0x7f486c0cfca0 2024-04-25T14:17:20.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.079+0000 7f4869e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f486c0cfca0 0x7f486c06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:20.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.079+0000 7f4861400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f486c0d16e0 0x7f486c0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:20.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.079+0000 7f4869e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f486c0cfca0 0x7f486c06f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:42758/0 (socket says 172.21.15.86:42758) 2024-04-25T14:17:20.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.079+0000 7f4869e00640 1 -- 172.21.15.86:0/2089325188 learned_addr learned my addr 172.21.15.86:0/2089325188 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:20.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.079+0000 7f4861400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f486c0d16e0 0x7f486c0d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:52028/0 (socket says 172.21.15.86:52028) 2024-04-25T14:17:20.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.079+0000 7f486a800640 1 --1- 172.21.15.86:0/2089325188 >> v1:172.21.15.155:6789/0 conn(0x7f486c06c670 0x7f486c06ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:36188/0 (socket says 172.21.15.86:36188) 2024-04-25T14:17:20.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.079+0000 7f4861400640 1 -- 172.21.15.86:0/2089325188 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f486c0cfca0 msgr2=0x7f486c06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:20.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.079+0000 7f4861400640 1 --2- 172.21.15.86:0/2089325188 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f486c0cfca0 0x7f486c06f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:20.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.079+0000 7f4861400640 1 -- 172.21.15.86:0/2089325188 >> v1:172.21.15.155:6789/0 conn(0x7f486c06c670 legacy=0x7f486c06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:20.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.079+0000 7f4861400640 1 -- 172.21.15.86:0/2089325188 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486c06f930 con 0x7f486c0d16e0 2024-04-25T14:17:20.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.083+0000 7f4861400640 1 --2- 172.21.15.86:0/2089325188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f486c0d16e0 0x7f486c0d1af0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f485c0097c0 tx=0x7f485c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1624a2b72612ac0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:20.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.083+0000 7f4869400640 1 -- 172.21.15.86:0/2089325188 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c0b1070 con 0x7f486c0d16e0 2024-04-25T14:17:20.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.083+0000 7f4869400640 1 -- 172.21.15.86:0/2089325188 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f485c0ac410 con 0x7f486c0d16e0 2024-04-25T14:17:20.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.083+0000 7f4869400640 1 -- 172.21.15.86:0/2089325188 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c0ac7e0 con 0x7f486c0d16e0 2024-04-25T14:17:20.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.083+0000 7f4871200640 1 -- 172.21.15.86:0/2089325188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f486c0d16e0 msgr2=0x7f486c0d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:20.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.083+0000 7f4871200640 1 --2- 172.21.15.86:0/2089325188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f486c0d16e0 0x7f486c0d1af0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f485c0097c0 tx=0x7f485c01c4a0 comp rx=0 tx=0).stop 2024-04-25T14:17:20.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.083+0000 7f4871200640 1 -- 172.21.15.86:0/2089325188 shutdown_connections 2024-04-25T14:17:20.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.083+0000 7f4871200640 1 --2- 172.21.15.86:0/2089325188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f486c0d16e0 0x7f486c0d1af0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:20.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.083+0000 7f4871200640 1 --2- 172.21.15.86:0/2089325188 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f486c0cfca0 0x7f486c06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:20.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.083+0000 7f4871200640 1 -- 172.21.15.86:0/2089325188 >> 172.21.15.86:0/2089325188 conn(0x7f486c0b4650 msgr2=0x7f486c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:20.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.083+0000 7f4871200640 1 -- 172.21.15.86:0/2089325188 shutdown_connections 2024-04-25T14:17:20.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.083+0000 7f4871200640 1 -- 172.21.15.86:0/2089325188 wait complete. 2024-04-25T14:17:20.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.083+0000 7f4871200640 1 Processor -- start 2024-04-25T14:17:20.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.083+0000 7f4871200640 1 -- start start 2024-04-25T14:17:20.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.087+0000 7f4871200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f486c06c670 0x7f486c099f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:20.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.087+0000 7f4871200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f486c0cfca0 0x7f486c159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:20.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.087+0000 7f4871200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f486c0d16e0 0x7f486c169d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:20.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.087+0000 7f4871200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f486c168060 con 0x7f486c0cfca0 2024-04-25T14:17:20.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.087+0000 7f4871200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f486c167ee0 con 0x7f486c0d16e0 2024-04-25T14:17:20.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.087+0000 7f4871200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f486c1681e0 con 0x7f486c06c670 2024-04-25T14:17:20.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.087+0000 7f4861400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f486c0cfca0 0x7f486c159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:20.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.087+0000 7f4861400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f486c0cfca0 0x7f486c159170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48008/0 (socket says 172.21.15.86:48008) 2024-04-25T14:17:20.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.087+0000 7f4861400640 1 -- 172.21.15.86:0/580816724 learned_addr learned my addr 172.21.15.86:0/580816724 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:20.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.087+0000 7f4869e00640 1 --2- 172.21.15.86:0/580816724 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f486c06c670 0x7f486c099f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:20.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.087+0000 7f486a800640 1 --2- 172.21.15.86:0/580816724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f486c0d16e0 0x7f486c169d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:20.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.087+0000 7f4861400640 1 -- 172.21.15.86:0/580816724 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f486c06c670 msgr2=0x7f486c099f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:20.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.087+0000 7f4861400640 1 --2- 172.21.15.86:0/580816724 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f486c06c670 0x7f486c099f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:20.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.087+0000 7f4861400640 1 -- 172.21.15.86:0/580816724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f486c0d16e0 msgr2=0x7f486c169d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:20.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.087+0000 7f4861400640 1 --2- 172.21.15.86:0/580816724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f486c0d16e0 0x7f486c169d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:20.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.087+0000 7f4861400640 1 -- 172.21.15.86:0/580816724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486c158ae0 con 0x7f486c0cfca0 2024-04-25T14:17:20.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.087+0000 7f4861400640 1 --2- 172.21.15.86:0/580816724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f486c0cfca0 0x7f486c159170 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f485c0af040 tx=0x7f485c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:20.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.087+0000 7f4862a00640 1 -- 172.21.15.86:0/580816724 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c0b1040 con 0x7f486c0cfca0 2024-04-25T14:17:20.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.087+0000 7f4862a00640 1 -- 172.21.15.86:0/580816724 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f485c0bb070 con 0x7f486c0cfca0 2024-04-25T14:17:20.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.087+0000 7f4862a00640 1 -- 172.21.15.86:0/580816724 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c0b5410 con 0x7f486c0cfca0 2024-04-25T14:17:20.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.087+0000 7f4871200640 1 -- 172.21.15.86:0/580816724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f486c158dc0 con 0x7f486c0cfca0 2024-04-25T14:17:20.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.087+0000 7f4871200640 1 -- 172.21.15.86:0/580816724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f486c0ce940 con 0x7f486c0cfca0 2024-04-25T14:17:20.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.087+0000 7f4871200640 1 -- 172.21.15.86:0/580816724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f486c09f4b0 con 0x7f486c0cfca0 2024-04-25T14:17:20.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.091+0000 7f4862a00640 1 -- 172.21.15.86:0/580816724 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f485c0b55b0 con 0x7f486c0cfca0 2024-04-25T14:17:20.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.091+0000 7f4862a00640 1 --2- 172.21.15.86:0/580816724 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f484806f480 0x7f4848071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:20.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.091+0000 7f4862a00640 1 -- 172.21.15.86:0/580816724 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f485c129a90 con 0x7f486c0cfca0 2024-04-25T14:17:20.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.091+0000 7f4869e00640 1 --2- 172.21.15.86:0/580816724 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f484806f480 0x7f4848071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:20.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.091+0000 7f4869e00640 1 --2- 172.21.15.86:0/580816724 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f484806f480 0x7f4848071990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f486c070710 tx=0x7f4854008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:20.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.095+0000 7f4862a00640 1 -- 172.21.15.86:0/580816724 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f485c016090 con 0x7f486c0cfca0 2024-04-25T14:17:20.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.283+0000 7f4871200640 1 -- 172.21.15.86:0/580816724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f486c069d00 con 0x7f486c0cfca0 2024-04-25T14:17:20.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.283+0000 7f4862a00640 1 -- 172.21.15.86:0/580816724 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f485c0f6710 con 0x7f486c0cfca0 2024-04-25T14:17:20.282 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:20.282 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:20.282 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:20.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.287+0000 7f4871200640 1 -- 172.21.15.86:0/580816724 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f484806f480 msgr2=0x7f4848071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:20.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.287+0000 7f4871200640 1 --2- 172.21.15.86:0/580816724 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f484806f480 0x7f4848071990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f486c070710 tx=0x7f4854008040 comp rx=0 tx=0).stop 2024-04-25T14:17:20.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.287+0000 7f4871200640 1 -- 172.21.15.86:0/580816724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f486c0cfca0 msgr2=0x7f486c159170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:20.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.287+0000 7f4871200640 1 --2- 172.21.15.86:0/580816724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f486c0cfca0 0x7f486c159170 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f485c0af040 tx=0x7f485c019240 comp rx=0 tx=0).stop 2024-04-25T14:17:20.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.287+0000 7f4871200640 1 -- 172.21.15.86:0/580816724 shutdown_connections 2024-04-25T14:17:20.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.287+0000 7f4871200640 1 --2- 172.21.15.86:0/580816724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f486c0d16e0 0x7f486c169d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:20.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.287+0000 7f4871200640 1 --2- 172.21.15.86:0/580816724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f486c0cfca0 0x7f486c159170 unknown :-1 s=CLOSED pgs=2358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:20.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.287+0000 7f4871200640 1 --2- 172.21.15.86:0/580816724 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f484806f480 0x7f4848071990 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:20.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.287+0000 7f4871200640 1 --2- 172.21.15.86:0/580816724 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f486c06c670 0x7f486c099f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:20.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.287+0000 7f4871200640 1 -- 172.21.15.86:0/580816724 >> 172.21.15.86:0/580816724 conn(0x7f486c0b4650 msgr2=0x7f486c06ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:20.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.287+0000 7f4871200640 1 -- 172.21.15.86:0/580816724 shutdown_connections 2024-04-25T14:17:20.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:20.287+0000 7f4871200640 1 -- 172.21.15.86:0/580816724 wait complete. 2024-04-25T14:17:20.707 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:20.707 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:20.714 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:17:20.717 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:17:21.708 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:21.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.867+0000 7fa005600640 1 Processor -- start 2024-04-25T14:17:21.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.867+0000 7fa005600640 1 -- start start 2024-04-25T14:17:21.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.871+0000 7fa005600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa00006d680 0x7fa00006da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:21.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.871+0000 7fa005600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0000af700 0x7fa0000afb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:21.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.871+0000 7fa005600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa00006dfd0 con 0x7fa0000b5500 2024-04-25T14:17:21.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.871+0000 7fa005600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa000149a40 con 0x7fa00006d680 2024-04-25T14:17:21.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.871+0000 7fa005600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa0000b5910 con 0x7fa0000af700 2024-04-25T14:17:21.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.871+0000 7f9ff7400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa00006d680 0x7fa00006da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:21.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.871+0000 7fa004c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0000af700 0x7fa0000afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:21.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.871+0000 7f9ff7400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa00006d680 0x7fa00006da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:52048/0 (socket says 172.21.15.86:52048) 2024-04-25T14:17:21.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.871+0000 7fa004c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0000af700 0x7fa0000afb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:42770/0 (socket says 172.21.15.86:42770) 2024-04-25T14:17:21.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.871+0000 7f9ff7400640 1 -- 172.21.15.86:0/2394858303 learned_addr learned my addr 172.21.15.86:0/2394858303 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:21.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.871+0000 7f9ff7400640 1 -- 172.21.15.86:0/2394858303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0000af700 msgr2=0x7fa0000afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:21.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.871+0000 7f9ff7400640 1 --2- 172.21.15.86:0/2394858303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0000af700 0x7fa0000afb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:21.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.871+0000 7f9ff7400640 1 -- 172.21.15.86:0/2394858303 >> v1:172.21.15.155:6789/0 conn(0x7fa0000b5500 legacy=0x7fa0000707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:21.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.871+0000 7f9ff7400640 1 -- 172.21.15.86:0/2394858303 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0000b5bc0 con 0x7fa00006d680 2024-04-25T14:17:21.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.875+0000 7f9ff7400640 1 --2- 172.21.15.86:0/2394858303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa00006d680 0x7fa00006da90 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009870 tx=0x7f9fe801c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=289662f6b01d6baf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:21.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.875+0000 7f9fff400640 1 -- 172.21.15.86:0/2394858303 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe80a95f0 con 0x7fa00006d680 2024-04-25T14:17:21.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.875+0000 7f9fff400640 1 -- 172.21.15.86:0/2394858303 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fe80a9790 con 0x7fa00006d680 2024-04-25T14:17:21.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.875+0000 7f9fff400640 1 -- 172.21.15.86:0/2394858303 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe80a9b60 con 0x7fa00006d680 2024-04-25T14:17:21.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.875+0000 7fa005600640 1 -- 172.21.15.86:0/2394858303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa00006d680 msgr2=0x7fa00006da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:21.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.875+0000 7fa005600640 1 --2- 172.21.15.86:0/2394858303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa00006d680 0x7fa00006da90 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009870 tx=0x7f9fe801c530 comp rx=0 tx=0).stop 2024-04-25T14:17:21.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.875+0000 7fa005600640 1 -- 172.21.15.86:0/2394858303 shutdown_connections 2024-04-25T14:17:21.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.875+0000 7fa005600640 1 --2- 172.21.15.86:0/2394858303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0000af700 0x7fa0000afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:21.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.875+0000 7fa005600640 1 --2- 172.21.15.86:0/2394858303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa00006d680 0x7fa00006da90 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:21.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.875+0000 7fa005600640 1 -- 172.21.15.86:0/2394858303 >> 172.21.15.86:0/2394858303 conn(0x7fa0000730f0 msgr2=0x7fa000073500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:21.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.875+0000 7fa005600640 1 -- 172.21.15.86:0/2394858303 shutdown_connections 2024-04-25T14:17:21.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.875+0000 7fa005600640 1 -- 172.21.15.86:0/2394858303 wait complete. 2024-04-25T14:17:21.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.879+0000 7fa005600640 1 Processor -- start 2024-04-25T14:17:21.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.879+0000 7fa005600640 1 -- start start 2024-04-25T14:17:21.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.879+0000 7fa005600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa00006d680 0x7fa000140240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:21.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.879+0000 7fa005600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0000af700 0x7fa00015f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:21.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.879+0000 7fa005600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0000b5500 0x7fa00015f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:21.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.879+0000 7fa005600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa00016c7d0 con 0x7fa0000af700 2024-04-25T14:17:21.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.879+0000 7fa005600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa0000b6180 con 0x7fa0000b5500 2024-04-25T14:17:21.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.879+0000 7fa005600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa00016c950 con 0x7fa00006d680 2024-04-25T14:17:21.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.879+0000 7f9ff7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0000af700 0x7fa00015f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:21.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.879+0000 7f9ff7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0000af700 0x7fa00015f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48022/0 (socket says 172.21.15.86:48022) 2024-04-25T14:17:21.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.879+0000 7f9ff7400640 1 -- 172.21.15.86:0/677929837 learned_addr learned my addr 172.21.15.86:0/677929837 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:21.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.879+0000 7fa004c00640 1 --2- 172.21.15.86:0/677929837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0000b5500 0x7fa00015f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:21.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.879+0000 7f9fffe00640 1 --2- 172.21.15.86:0/677929837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa00006d680 0x7fa000140240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:21.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.879+0000 7f9ff7400640 1 -- 172.21.15.86:0/677929837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa00006d680 msgr2=0x7fa000140240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:21.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.879+0000 7f9ff7400640 1 --2- 172.21.15.86:0/677929837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa00006d680 0x7fa000140240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:21.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.879+0000 7f9ff7400640 1 -- 172.21.15.86:0/677929837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0000b5500 msgr2=0x7fa00015f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:21.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.879+0000 7f9ff7400640 1 --2- 172.21.15.86:0/677929837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0000b5500 0x7fa00015f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:21.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.879+0000 7f9ff7400640 1 -- 172.21.15.86:0/677929837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0001fcce0 con 0x7fa0000af700 2024-04-25T14:17:21.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.879+0000 7fa004c00640 1 --2- 172.21.15.86:0/677929837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0000b5500 0x7fa00015f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:17:21.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.879+0000 7f9ff7400640 1 --2- 172.21.15.86:0/677929837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0000af700 0x7fa00015f160 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f9fe801a160 tx=0x7f9fe80192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:21.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.879+0000 7f9ffcc00640 1 -- 172.21.15.86:0/677929837 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe80a95f0 con 0x7fa0000af700 2024-04-25T14:17:21.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.883+0000 7f9ffcc00640 1 -- 172.21.15.86:0/677929837 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fe80b1800 con 0x7fa0000af700 2024-04-25T14:17:21.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.883+0000 7fa005600640 1 -- 172.21.15.86:0/677929837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0001fcfc0 con 0x7fa0000af700 2024-04-25T14:17:21.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.883+0000 7f9ffcc00640 1 -- 172.21.15.86:0/677929837 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe80b1bf0 con 0x7fa0000af700 2024-04-25T14:17:21.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.883+0000 7fa005600640 1 -- 172.21.15.86:0/677929837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0001fa0b0 con 0x7fa0000af700 2024-04-25T14:17:21.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.883+0000 7fa005600640 1 -- 172.21.15.86:0/677929837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0001fca50 con 0x7fa0000af700 2024-04-25T14:17:21.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.883+0000 7f9ffcc00640 1 -- 172.21.15.86:0/677929837 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f9fe80ba480 con 0x7fa0000af700 2024-04-25T14:17:21.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.883+0000 7f9ffcc00640 1 --2- 172.21.15.86:0/677929837 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9fdc06f450 0x7f9fdc071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:21.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.883+0000 7f9ffcc00640 1 -- 172.21.15.86:0/677929837 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f9fe8129d90 con 0x7fa0000af700 2024-04-25T14:17:21.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.883+0000 7f9fffe00640 1 --2- 172.21.15.86:0/677929837 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9fdc06f450 0x7f9fdc071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:21.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.887+0000 7f9fffe00640 1 --2- 172.21.15.86:0/677929837 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9fdc06f450 0x7f9fdc071960 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0001cb0 tx=0x7f9ff0009040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:21.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:21.891+0000 7f9ffcc00640 1 -- 172.21.15.86:0/677929837 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9fe80ad020 con 0x7fa0000af700 2024-04-25T14:17:22.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:22.079+0000 7fa005600640 1 -- 172.21.15.86:0/677929837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0001450b0 con 0x7fa0000af700 2024-04-25T14:17:22.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:22.079+0000 7f9ffcc00640 1 -- 172.21.15.86:0/677929837 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f9fe8016150 con 0x7fa0000af700 2024-04-25T14:17:22.077 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:22.077 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:22.078 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:22.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:22.079+0000 7fa005600640 1 -- 172.21.15.86:0/677929837 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9fdc06f450 msgr2=0x7f9fdc071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:22.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:22.079+0000 7fa005600640 1 --2- 172.21.15.86:0/677929837 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9fdc06f450 0x7f9fdc071960 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0001cb0 tx=0x7f9ff0009040 comp rx=0 tx=0).stop 2024-04-25T14:17:22.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:22.079+0000 7fa005600640 1 -- 172.21.15.86:0/677929837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0000af700 msgr2=0x7fa00015f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:22.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:22.079+0000 7fa005600640 1 --2- 172.21.15.86:0/677929837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0000af700 0x7fa00015f160 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f9fe801a160 tx=0x7f9fe80192d0 comp rx=0 tx=0).stop 2024-04-25T14:17:22.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:22.079+0000 7fa005600640 1 -- 172.21.15.86:0/677929837 shutdown_connections 2024-04-25T14:17:22.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:22.079+0000 7fa005600640 1 --2- 172.21.15.86:0/677929837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0000b5500 0x7fa00015f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:22.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:22.079+0000 7fa005600640 1 --2- 172.21.15.86:0/677929837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0000af700 0x7fa00015f160 unknown :-1 s=CLOSED pgs=2359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:22.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:22.079+0000 7fa005600640 1 --2- 172.21.15.86:0/677929837 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9fdc06f450 0x7f9fdc071960 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:22.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:22.079+0000 7fa005600640 1 --2- 172.21.15.86:0/677929837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa00006d680 0x7fa000140240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:22.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:22.079+0000 7fa005600640 1 -- 172.21.15.86:0/677929837 >> 172.21.15.86:0/677929837 conn(0x7fa0000730f0 msgr2=0x7fa0000b1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:22.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:22.079+0000 7fa005600640 1 -- 172.21.15.86:0/677929837 shutdown_connections 2024-04-25T14:17:22.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:22.079+0000 7fa005600640 1 -- 172.21.15.86:0/677929837 wait complete. 2024-04-25T14:17:22.509 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:22.510 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:23.511 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:23.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.671+0000 7f5605200640 1 Processor -- start 2024-04-25T14:17:23.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.671+0000 7f5605200640 1 -- start start 2024-04-25T14:17:23.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.675+0000 7f5605200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f56000cfca0 0x7f560006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:23.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.675+0000 7f5605200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f56000d16e0 0x7f56000d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:23.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.675+0000 7f5605200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f560006c670 0x7f560006ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:23.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.675+0000 7f5605200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5600070470 con 0x7f56000cfca0 2024-04-25T14:17:23.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.675+0000 7f5605200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f56000d2030 con 0x7f56000d16e0 2024-04-25T14:17:23.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.675+0000 7f5605200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5600072bd0 con 0x7f560006c670 2024-04-25T14:17:23.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.675+0000 7f55fde00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f56000cfca0 0x7f560006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:23.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.675+0000 7f55fde00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f56000cfca0 0x7f560006f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48036/0 (socket says 172.21.15.86:48036) 2024-04-25T14:17:23.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.675+0000 7f55fde00640 1 -- 172.21.15.86:0/791623271 learned_addr learned my addr 172.21.15.86:0/791623271 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:23.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.675+0000 7f55fe800640 1 --2- 172.21.15.86:0/791623271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f560006c670 0x7f560006ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:23.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.675+0000 7f55fd400640 1 --2- 172.21.15.86:0/791623271 >> v2:172.21.15.155:3300/0 conn(0x7f56000d16e0 0x7f56000d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:23.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.675+0000 7f55fde00640 1 -- 172.21.15.86:0/791623271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f560006c670 msgr2=0x7f560006ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:23.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.675+0000 7f55fde00640 1 --2- 172.21.15.86:0/791623271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f560006c670 0x7f560006ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:23.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.675+0000 7f55fde00640 1 -- 172.21.15.86:0/791623271 >> v2:172.21.15.155:3300/0 conn(0x7f56000d16e0 msgr2=0x7f56000d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:23.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.675+0000 7f55fde00640 1 --2- 172.21.15.86:0/791623271 >> v2:172.21.15.155:3300/0 conn(0x7f56000d16e0 0x7f56000d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:23.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.675+0000 7f55fde00640 1 -- 172.21.15.86:0/791623271 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560006f930 con 0x7f56000cfca0 2024-04-25T14:17:23.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.679+0000 7f55fde00640 1 --2- 172.21.15.86:0/791623271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56000cfca0 0x7f560006f3f0 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f56000729c0 tx=0x7f55f001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ab2c096904f097 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:23.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.679+0000 7f55fca00640 1 -- 172.21.15.86:0/791623271 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f00ab5f0 con 0x7f56000cfca0 2024-04-25T14:17:23.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.679+0000 7f55fca00640 1 -- 172.21.15.86:0/791623271 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55f00b0070 con 0x7f56000cfca0 2024-04-25T14:17:23.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.679+0000 7f55fca00640 1 -- 172.21.15.86:0/791623271 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f00ab9b0 con 0x7f56000cfca0 2024-04-25T14:17:23.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.679+0000 7f5605200640 1 -- 172.21.15.86:0/791623271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56000cfca0 msgr2=0x7f560006f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:23.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.679+0000 7f5605200640 1 --2- 172.21.15.86:0/791623271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56000cfca0 0x7f560006f3f0 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f56000729c0 tx=0x7f55f001c520 comp rx=0 tx=0).stop 2024-04-25T14:17:23.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.679+0000 7f5605200640 1 -- 172.21.15.86:0/791623271 shutdown_connections 2024-04-25T14:17:23.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.679+0000 7f5605200640 1 --2- 172.21.15.86:0/791623271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f560006c670 0x7f560006ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:23.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.679+0000 7f5605200640 1 --2- 172.21.15.86:0/791623271 >> v2:172.21.15.155:3300/0 conn(0x7f56000d16e0 0x7f56000d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:23.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.679+0000 7f5605200640 1 --2- 172.21.15.86:0/791623271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56000cfca0 0x7f560006f3f0 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:23.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.679+0000 7f5605200640 1 -- 172.21.15.86:0/791623271 >> 172.21.15.86:0/791623271 conn(0x7f56000b4650 msgr2=0x7f56000b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:23.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.679+0000 7f5605200640 1 -- 172.21.15.86:0/791623271 shutdown_connections 2024-04-25T14:17:23.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.679+0000 7f5605200640 1 -- 172.21.15.86:0/791623271 wait complete. 2024-04-25T14:17:23.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.683+0000 7f5605200640 1 Processor -- start 2024-04-25T14:17:23.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.683+0000 7f5605200640 1 -- start start 2024-04-25T14:17:23.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.683+0000 7f5605200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f560006c670 0x7f5600157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:23.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.683+0000 7f5605200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56000cfca0 0x7f56000ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:23.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.683+0000 7f5605200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56000d16e0 0x7f56001fd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:23.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.683+0000 7f5605200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56000738c0 con 0x7f56000cfca0 2024-04-25T14:17:23.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.683+0000 7f5605200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f560006d3d0 con 0x7f56000d16e0 2024-04-25T14:17:23.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.683+0000 7f5605200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5600073a40 con 0x7f560006c670 2024-04-25T14:17:23.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.683+0000 7f55fd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56000cfca0 0x7f56000ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:23.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.683+0000 7f55fd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56000cfca0 0x7f56000ce3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48052/0 (socket says 172.21.15.86:48052) 2024-04-25T14:17:23.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.683+0000 7f55fd400640 1 -- 172.21.15.86:0/3165874375 learned_addr learned my addr 172.21.15.86:0/3165874375 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:23.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.683+0000 7f55fde00640 1 --2- 172.21.15.86:0/3165874375 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f560006c670 0x7f5600157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:23.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.683+0000 7f55fe800640 1 --2- 172.21.15.86:0/3165874375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56000d16e0 0x7f56001fd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:23.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.683+0000 7f55fd400640 1 -- 172.21.15.86:0/3165874375 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f560006c670 msgr2=0x7f5600157f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:23.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.683+0000 7f55fd400640 1 --2- 172.21.15.86:0/3165874375 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f560006c670 0x7f5600157f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:23.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.683+0000 7f55fd400640 1 -- 172.21.15.86:0/3165874375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56000d16e0 msgr2=0x7f56001fd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:23.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.683+0000 7f55fd400640 1 --2- 172.21.15.86:0/3165874375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56000d16e0 0x7f56001fd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:23.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.683+0000 7f55fd400640 1 -- 172.21.15.86:0/3165874375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5600068980 con 0x7f56000cfca0 2024-04-25T14:17:23.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.683+0000 7f55fd400640 1 --2- 172.21.15.86:0/3165874375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56000cfca0 0x7f56000ce3e0 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00d7d0 tx=0x7f55ec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:23.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.683+0000 7f55f6000640 1 -- 172.21.15.86:0/3165874375 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec00b660 con 0x7f56000cfca0 2024-04-25T14:17:23.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.683+0000 7f55f6000640 1 -- 172.21.15.86:0/3165874375 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55ec019070 con 0x7f56000cfca0 2024-04-25T14:17:23.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.683+0000 7f5605200640 1 -- 172.21.15.86:0/3165874375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f560015d300 con 0x7f56000cfca0 2024-04-25T14:17:23.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.683+0000 7f55f6000640 1 -- 172.21.15.86:0/3165874375 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec00ba20 con 0x7f56000cfca0 2024-04-25T14:17:23.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.683+0000 7f5605200640 1 -- 172.21.15.86:0/3165874375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f560015d8c0 con 0x7f56000cfca0 2024-04-25T14:17:23.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.687+0000 7f5605200640 1 -- 172.21.15.86:0/3165874375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f560009f4b0 con 0x7f56000cfca0 2024-04-25T14:17:23.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.687+0000 7f55f6000640 1 -- 172.21.15.86:0/3165874375 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f55ec00bbc0 con 0x7f56000cfca0 2024-04-25T14:17:23.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.687+0000 7f55f6000640 1 --2- 172.21.15.86:0/3165874375 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f55e006f450 0x7f55e0071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:23.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.687+0000 7f55f6000640 1 -- 172.21.15.86:0/3165874375 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f55ec08f820 con 0x7f56000cfca0 2024-04-25T14:17:23.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.687+0000 7f55fde00640 1 --2- 172.21.15.86:0/3165874375 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f55e006f450 0x7f55e0071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:23.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.687+0000 7f55fde00640 1 --2- 172.21.15.86:0/3165874375 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f55e006f450 0x7f55e0071960 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f55f00af000 tx=0x7f55f00a7000 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:23.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.691+0000 7f55f6000640 1 -- 172.21.15.86:0/3165874375 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f55ec05bdd0 con 0x7f56000cfca0 2024-04-25T14:17:23.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.883+0000 7f5605200640 1 -- 172.21.15.86:0/3165874375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5600069d00 con 0x7f56000cfca0 2024-04-25T14:17:23.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.883+0000 7f55f6000640 1 -- 172.21.15.86:0/3165874375 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f55ec0600d0 con 0x7f56000cfca0 2024-04-25T14:17:23.880 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:23.880 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:23.880 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:23.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.883+0000 7f5605200640 1 -- 172.21.15.86:0/3165874375 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f55e006f450 msgr2=0x7f55e0071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:23.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.883+0000 7f5605200640 1 --2- 172.21.15.86:0/3165874375 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f55e006f450 0x7f55e0071960 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f55f00af000 tx=0x7f55f00a7000 comp rx=0 tx=0).stop 2024-04-25T14:17:23.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.883+0000 7f5605200640 1 -- 172.21.15.86:0/3165874375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56000cfca0 msgr2=0x7f56000ce3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:23.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.883+0000 7f5605200640 1 --2- 172.21.15.86:0/3165874375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56000cfca0 0x7f56000ce3e0 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00d7d0 tx=0x7f55ec00dc90 comp rx=0 tx=0).stop 2024-04-25T14:17:23.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.883+0000 7f5605200640 1 -- 172.21.15.86:0/3165874375 shutdown_connections 2024-04-25T14:17:23.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.883+0000 7f5605200640 1 --2- 172.21.15.86:0/3165874375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56000d16e0 0x7f56001fd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:23.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.883+0000 7f5605200640 1 --2- 172.21.15.86:0/3165874375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56000cfca0 0x7f56000ce3e0 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:23.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.883+0000 7f5605200640 1 --2- 172.21.15.86:0/3165874375 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f55e006f450 0x7f55e0071960 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:23.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.883+0000 7f5605200640 1 --2- 172.21.15.86:0/3165874375 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f560006c670 0x7f5600157f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:23.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.883+0000 7f5605200640 1 -- 172.21.15.86:0/3165874375 >> 172.21.15.86:0/3165874375 conn(0x7f56000b4650 msgr2=0x7f56000cbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:23.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.883+0000 7f5605200640 1 -- 172.21.15.86:0/3165874375 shutdown_connections 2024-04-25T14:17:23.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:23.883+0000 7f5605200640 1 -- 172.21.15.86:0/3165874375 wait complete. 2024-04-25T14:17:24.306 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:24.306 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:25.307 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:25.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.466+0000 7f4db8600640 1 Processor -- start 2024-04-25T14:17:25.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.466+0000 7f4db8600640 1 -- start start 2024-04-25T14:17:25.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.470+0000 7f4db8600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4db006fd50 0x7f4db0070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:25.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.470+0000 7f4db8600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4db015feb0 0x7f4db01602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:25.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.470+0000 7f4db8600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4db00b0a60 con 0x7f4db015c2a0 2024-04-25T14:17:25.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.470+0000 7f4db8600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4db00ae840 con 0x7f4db006fd50 2024-04-25T14:17:25.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.470+0000 7f4db8600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4db006ef90 con 0x7f4db015feb0 2024-04-25T14:17:25.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.470+0000 7f4db5200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4db006fd50 0x7f4db0070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:25.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.470+0000 7f4db5200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4db006fd50 0x7f4db0070160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48062/0 (socket says 172.21.15.86:48062) 2024-04-25T14:17:25.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.470+0000 7f4db5200640 1 -- 172.21.15.86:0/4264979043 learned_addr learned my addr 172.21.15.86:0/4264979043 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:25.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.470+0000 7f4db6600640 1 --2- 172.21.15.86:0/4264979043 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4db015feb0 0x7f4db01602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:25.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.470+0000 7f4db5200640 1 -- 172.21.15.86:0/4264979043 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4db015feb0 msgr2=0x7f4db01602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:25.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.470+0000 7f4db5200640 1 --2- 172.21.15.86:0/4264979043 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4db015feb0 0x7f4db01602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:25.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.470+0000 7f4db5200640 1 -- 172.21.15.86:0/4264979043 >> v1:172.21.15.155:6789/0 conn(0x7f4db015c2a0 legacy=0x7f4db00ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:25.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.470+0000 7f4db5200640 1 -- 172.21.15.86:0/4264979043 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db006f270 con 0x7f4db006fd50 2024-04-25T14:17:25.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.470+0000 7f4db6600640 1 --2- 172.21.15.86:0/4264979043 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4db015feb0 0x7f4db01602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:17:25.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.474+0000 7f4db5200640 1 --2- 172.21.15.86:0/4264979043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db006fd50 0x7f4db0070160 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c00aa10 tx=0x7f4d9c01d870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3476880dd20fd60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:25.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.474+0000 7f4dafe00640 1 -- 172.21.15.86:0/4264979043 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c0b3070 con 0x7f4db006fd50 2024-04-25T14:17:25.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.474+0000 7f4dafe00640 1 -- 172.21.15.86:0/4264979043 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d9c0a68a0 con 0x7f4db006fd50 2024-04-25T14:17:25.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.474+0000 7f4dafe00640 1 -- 172.21.15.86:0/4264979043 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c00b040 con 0x7f4db006fd50 2024-04-25T14:17:25.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.474+0000 7f4db8600640 1 -- 172.21.15.86:0/4264979043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db006fd50 msgr2=0x7f4db0070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:25.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.474+0000 7f4db8600640 1 --2- 172.21.15.86:0/4264979043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db006fd50 0x7f4db0070160 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c00aa10 tx=0x7f4d9c01d870 comp rx=0 tx=0).stop 2024-04-25T14:17:25.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.474+0000 7f4db8600640 1 -- 172.21.15.86:0/4264979043 shutdown_connections 2024-04-25T14:17:25.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.474+0000 7f4db8600640 1 --2- 172.21.15.86:0/4264979043 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4db015feb0 0x7f4db01602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:25.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.474+0000 7f4db8600640 1 --2- 172.21.15.86:0/4264979043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db006fd50 0x7f4db0070160 unknown :-1 s=CLOSED pgs=2362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:25.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.474+0000 7f4db8600640 1 -- 172.21.15.86:0/4264979043 >> 172.21.15.86:0/4264979043 conn(0x7f4db00b4650 msgr2=0x7f4db00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:25.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.474+0000 7f4db8600640 1 -- 172.21.15.86:0/4264979043 shutdown_connections 2024-04-25T14:17:25.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.474+0000 7f4db8600640 1 -- 172.21.15.86:0/4264979043 wait complete. 2024-04-25T14:17:25.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.478+0000 7f4db8600640 1 Processor -- start 2024-04-25T14:17:25.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.478+0000 7f4db8600640 1 -- start start 2024-04-25T14:17:25.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.478+0000 7f4db8600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db006fd50 0x7f4db0154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:25.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.478+0000 7f4db8600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4db015c2a0 0x7f4db015e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:25.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.478+0000 7f4db8600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4db015feb0 0x7f4db015e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:25.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.478+0000 7f4db8600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4db015c6b0 con 0x7f4db006fd50 2024-04-25T14:17:25.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.478+0000 7f4db8600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4db00d0fc0 con 0x7f4db015feb0 2024-04-25T14:17:25.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.478+0000 7f4db8600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4db015c830 con 0x7f4db015c2a0 2024-04-25T14:17:25.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.478+0000 7f4db5c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db006fd50 0x7f4db0154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:25.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.478+0000 7f4db5c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db006fd50 0x7f4db0154d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48068/0 (socket says 172.21.15.86:48068) 2024-04-25T14:17:25.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.478+0000 7f4db5c00640 1 -- 172.21.15.86:0/3595453516 learned_addr learned my addr 172.21.15.86:0/3595453516 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:25.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.478+0000 7f4db6600640 1 --2- 172.21.15.86:0/3595453516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4db015feb0 0x7f4db015e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:25.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.478+0000 7f4db5200640 1 --2- 172.21.15.86:0/3595453516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4db015c2a0 0x7f4db015e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:25.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.478+0000 7f4db5c00640 1 -- 172.21.15.86:0/3595453516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4db015c2a0 msgr2=0x7f4db015e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:25.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.478+0000 7f4db5c00640 1 --2- 172.21.15.86:0/3595453516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4db015c2a0 0x7f4db015e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:25.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.478+0000 7f4db5c00640 1 -- 172.21.15.86:0/3595453516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4db015feb0 msgr2=0x7f4db015e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:25.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.478+0000 7f4db5c00640 1 --2- 172.21.15.86:0/3595453516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4db015feb0 0x7f4db015e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:25.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.478+0000 7f4db5c00640 1 -- 172.21.15.86:0/3595453516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db01fda00 con 0x7f4db006fd50 2024-04-25T14:17:25.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.478+0000 7f4db5200640 1 --2- 172.21.15.86:0/3595453516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4db015c2a0 0x7f4db015e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:17:25.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.478+0000 7f4db5c00640 1 --2- 172.21.15.86:0/3595453516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db006fd50 0x7f4db0154d30 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f4da400c340 tx=0x7f4da400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:25.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.478+0000 7f4dad600640 1 -- 172.21.15.86:0/3595453516 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da40052d0 con 0x7f4db006fd50 2024-04-25T14:17:25.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.478+0000 7f4dad600640 1 -- 172.21.15.86:0/3595453516 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4da4005470 con 0x7f4db006fd50 2024-04-25T14:17:25.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.478+0000 7f4db8600640 1 -- 172.21.15.86:0/3595453516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db01fdce0 con 0x7f4db006fd50 2024-04-25T14:17:25.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.478+0000 7f4dad600640 1 -- 172.21.15.86:0/3595453516 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4007260 con 0x7f4db006fd50 2024-04-25T14:17:25.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.478+0000 7f4db8600640 1 -- 172.21.15.86:0/3595453516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db01fe2a0 con 0x7f4db006fd50 2024-04-25T14:17:25.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.482+0000 7f4db8600640 1 -- 172.21.15.86:0/3595453516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4db0069d00 con 0x7f4db006fd50 2024-04-25T14:17:25.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.482+0000 7f4dad600640 1 -- 172.21.15.86:0/3595453516 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f4da4007400 con 0x7f4db006fd50 2024-04-25T14:17:25.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.482+0000 7f4dad600640 1 --2- 172.21.15.86:0/3595453516 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f4d9006f450 0x7f4d90071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:25.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.482+0000 7f4dad600640 1 -- 172.21.15.86:0/3595453516 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f4da408ea30 con 0x7f4db006fd50 2024-04-25T14:17:25.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.482+0000 7f4db5200640 1 --2- 172.21.15.86:0/3595453516 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f4d9006f450 0x7f4d90071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:25.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.482+0000 7f4db5200640 1 --2- 172.21.15.86:0/3595453516 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f4d9006f450 0x7f4d90071960 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c0b0000 tx=0x7f4d9c0a8070 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:25.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.486+0000 7f4dad600640 1 -- 172.21.15.86:0/3595453516 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4da405af50 con 0x7f4db006fd50 2024-04-25T14:17:25.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.678+0000 7f4db8600640 1 -- 172.21.15.86:0/3595453516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4db0076bf0 con 0x7f4db006fd50 2024-04-25T14:17:25.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.678+0000 7f4dad600640 1 -- 172.21.15.86:0/3595453516 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f4da405f250 con 0x7f4db006fd50 2024-04-25T14:17:25.676 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:25.676 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:25.676 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.678+0000 7f4db8600640 1 -- 172.21.15.86:0/3595453516 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f4d9006f450 msgr2=0x7f4d90071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.678+0000 7f4db8600640 1 --2- 172.21.15.86:0/3595453516 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f4d9006f450 0x7f4d90071960 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c0b0000 tx=0x7f4d9c0a8070 comp rx=0 tx=0).stop 2024-04-25T14:17:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.678+0000 7f4db8600640 1 -- 172.21.15.86:0/3595453516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db006fd50 msgr2=0x7f4db0154d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.678+0000 7f4db8600640 1 --2- 172.21.15.86:0/3595453516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db006fd50 0x7f4db0154d30 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f4da400c340 tx=0x7f4da400c800 comp rx=0 tx=0).stop 2024-04-25T14:17:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.678+0000 7f4db8600640 1 -- 172.21.15.86:0/3595453516 shutdown_connections 2024-04-25T14:17:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.678+0000 7f4db8600640 1 --2- 172.21.15.86:0/3595453516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4db015feb0 0x7f4db015e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.678+0000 7f4db8600640 1 --2- 172.21.15.86:0/3595453516 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f4d9006f450 0x7f4d90071960 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.678+0000 7f4db8600640 1 --2- 172.21.15.86:0/3595453516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4db015c2a0 0x7f4db015e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.678+0000 7f4db8600640 1 --2- 172.21.15.86:0/3595453516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db006fd50 0x7f4db0154d30 unknown :-1 s=CLOSED pgs=2363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.678+0000 7f4db8600640 1 -- 172.21.15.86:0/3595453516 >> 172.21.15.86:0/3595453516 conn(0x7f4db00b4650 msgr2=0x7f4db0152ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.678+0000 7f4db8600640 1 -- 172.21.15.86:0/3595453516 shutdown_connections 2024-04-25T14:17:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:25.678+0000 7f4db8600640 1 -- 172.21.15.86:0/3595453516 wait complete. 2024-04-25T14:17:26.100 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:26.101 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:27.102 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:27.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.262+0000 7fa6a7200640 1 Processor -- start 2024-04-25T14:17:27.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.262+0000 7fa6a7200640 1 -- start start 2024-04-25T14:17:27.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.262+0000 7fa6a7200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa6a80ced30 0x7fa6a80b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:27.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.262+0000 7fa6a7200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa6a815dcd0 0x7fa6a80d1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:27.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.262+0000 7fa6a7200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a80d1970 0x7fa6a815e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:27.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.262+0000 7fa6a7200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa6a81618e0 con 0x7fa6a80ced30 2024-04-25T14:17:27.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.262+0000 7fa6a7200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa6a8161b30 con 0x7fa6a815dcd0 2024-04-25T14:17:27.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.262+0000 7fa6a7200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa6a8161e10 con 0x7fa6a80d1970 2024-04-25T14:17:27.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.262+0000 7fa6a5e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa6a80ced30 0x7fa6a80b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:27.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.262+0000 7fa6a5e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa6a80ced30 0x7fa6a80b1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48072/0 (socket says 172.21.15.86:48072) 2024-04-25T14:17:27.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.262+0000 7fa6a5e00640 1 -- 172.21.15.86:0/3613163236 learned_addr learned my addr 172.21.15.86:0/3613163236 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:27.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.262+0000 7fa6a6800640 1 --2- 172.21.15.86:0/3613163236 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a80d1970 0x7fa6a815e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:27.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.262+0000 7fa6a5e00640 1 -- 172.21.15.86:0/3613163236 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a80d1970 msgr2=0x7fa6a815e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:27.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.262+0000 7fa6a5400640 1 --2- 172.21.15.86:0/3613163236 >> v2:172.21.15.155:3300/0 conn(0x7fa6a815dcd0 0x7fa6a80d1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:27.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.262+0000 7fa6a5e00640 1 --2- 172.21.15.86:0/3613163236 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a80d1970 0x7fa6a815e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:27.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.262+0000 7fa6a5e00640 1 -- 172.21.15.86:0/3613163236 >> v2:172.21.15.155:3300/0 conn(0x7fa6a815dcd0 msgr2=0x7fa6a80d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:27.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.262+0000 7fa6a5e00640 1 --2- 172.21.15.86:0/3613163236 >> v2:172.21.15.155:3300/0 conn(0x7fa6a815dcd0 0x7fa6a80d1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:27.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.262+0000 7fa6a5e00640 1 -- 172.21.15.86:0/3613163236 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a81620f0 con 0x7fa6a80ced30 2024-04-25T14:17:27.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.262+0000 7fa6a5400640 1 --2- 172.21.15.86:0/3613163236 >> v2:172.21.15.155:3300/0 conn(0x7fa6a815dcd0 0x7fa6a80d1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:17:27.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.262+0000 7fa6a6800640 1 --2- 172.21.15.86:0/3613163236 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a80d1970 0x7fa6a815e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:17:27.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.266+0000 7fa6a5e00640 1 --2- 172.21.15.86:0/3613163236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a80ced30 0x7fa6a80b1e20 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8073800 tx=0x7fa69801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da607d84802d09e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:27.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.266+0000 7fa6a4a00640 1 -- 172.21.15.86:0/3613163236 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6980090d0 con 0x7fa6a80ced30 2024-04-25T14:17:27.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.266+0000 7fa6a4a00640 1 -- 172.21.15.86:0/3613163236 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6980a5850 con 0x7fa6a80ced30 2024-04-25T14:17:27.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.266+0000 7fa6a4a00640 1 -- 172.21.15.86:0/3613163236 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa698009270 con 0x7fa6a80ced30 2024-04-25T14:17:27.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.266+0000 7fa6a7200640 1 -- 172.21.15.86:0/3613163236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a80ced30 msgr2=0x7fa6a80b1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:27.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.266+0000 7fa6a7200640 1 --2- 172.21.15.86:0/3613163236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a80ced30 0x7fa6a80b1e20 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8073800 tx=0x7fa69801c9f0 comp rx=0 tx=0).stop 2024-04-25T14:17:27.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.266+0000 7fa6a7200640 1 -- 172.21.15.86:0/3613163236 shutdown_connections 2024-04-25T14:17:27.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.266+0000 7fa6a7200640 1 --2- 172.21.15.86:0/3613163236 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a80d1970 0x7fa6a815e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:27.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.266+0000 7fa6a7200640 1 --2- 172.21.15.86:0/3613163236 >> v2:172.21.15.155:3300/0 conn(0x7fa6a815dcd0 0x7fa6a80d1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:27.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.266+0000 7fa6a7200640 1 --2- 172.21.15.86:0/3613163236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a80ced30 0x7fa6a80b1e20 unknown :-1 s=CLOSED pgs=2364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:27.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.266+0000 7fa6a7200640 1 -- 172.21.15.86:0/3613163236 >> 172.21.15.86:0/3613163236 conn(0x7fa6a809ba30 msgr2=0x7fa6a80b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:27.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.266+0000 7fa6a7200640 1 -- 172.21.15.86:0/3613163236 shutdown_connections 2024-04-25T14:17:27.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.266+0000 7fa6a7200640 1 -- 172.21.15.86:0/3613163236 wait complete. 2024-04-25T14:17:27.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.270+0000 7fa6a7200640 1 Processor -- start 2024-04-25T14:17:27.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.270+0000 7fa6a7200640 1 -- start start 2024-04-25T14:17:27.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.270+0000 7fa6a7200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a80ced30 0x7fa6a815f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:27.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.270+0000 7fa6a7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a80d1970 0x7fa6a814edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:27.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.270+0000 7fa6a7200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6a815dcd0 0x7fa6a80b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:27.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.270+0000 7fa6a7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a80d2290 con 0x7fa6a80d1970 2024-04-25T14:17:27.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.270+0000 7fa6a7200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a80d1d80 con 0x7fa6a815dcd0 2024-04-25T14:17:27.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.270+0000 7fa6a7200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa6a80d2410 con 0x7fa6a80ced30 2024-04-25T14:17:27.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.270+0000 7fa6a5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a80d1970 0x7fa6a814edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:27.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.270+0000 7fa6a5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a80d1970 0x7fa6a814edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48086/0 (socket says 172.21.15.86:48086) 2024-04-25T14:17:27.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.270+0000 7fa6a5400640 1 -- 172.21.15.86:0/2273342493 learned_addr learned my addr 172.21.15.86:0/2273342493 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:27.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.270+0000 7fa6a5e00640 1 --2- 172.21.15.86:0/2273342493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a80ced30 0x7fa6a815f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:27.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.270+0000 7fa6a6800640 1 --2- 172.21.15.86:0/2273342493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6a815dcd0 0x7fa6a80b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:27.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.270+0000 7fa6a5400640 1 -- 172.21.15.86:0/2273342493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a80ced30 msgr2=0x7fa6a815f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:27.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.270+0000 7fa6a5400640 1 --2- 172.21.15.86:0/2273342493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a80ced30 0x7fa6a815f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:27.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.270+0000 7fa6a5400640 1 -- 172.21.15.86:0/2273342493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6a815dcd0 msgr2=0x7fa6a80b0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:27.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.270+0000 7fa6a5400640 1 --2- 172.21.15.86:0/2273342493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6a815dcd0 0x7fa6a80b0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:27.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.270+0000 7fa6a5400640 1 -- 172.21.15.86:0/2273342493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a815f7a0 con 0x7fa6a80d1970 2024-04-25T14:17:27.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.270+0000 7fa6a5e00640 1 --2- 172.21.15.86:0/2273342493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a80ced30 0x7fa6a815f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:17:27.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.270+0000 7fa6a6800640 1 --2- 172.21.15.86:0/2273342493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6a815dcd0 0x7fa6a80b0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:17:27.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.274+0000 7fa6a5400640 1 --2- 172.21.15.86:0/2273342493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a80d1970 0x7fa6a814edf0 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7fa69400c7f0 tx=0x7fa69400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:27.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.274+0000 7fa69e000640 1 -- 172.21.15.86:0/2273342493 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa694015070 con 0x7fa6a80d1970 2024-04-25T14:17:27.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.274+0000 7fa69e000640 1 -- 172.21.15.86:0/2273342493 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa694004220 con 0x7fa6a80d1970 2024-04-25T14:17:27.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.274+0000 7fa6a7200640 1 -- 172.21.15.86:0/2273342493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a80b0ef0 con 0x7fa6a80d1970 2024-04-25T14:17:27.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.274+0000 7fa69e000640 1 -- 172.21.15.86:0/2273342493 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa694003620 con 0x7fa6a80d1970 2024-04-25T14:17:27.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.274+0000 7fa6a7200640 1 -- 172.21.15.86:0/2273342493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a8159460 con 0x7fa6a80d1970 2024-04-25T14:17:27.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.274+0000 7fa6a7200640 1 -- 172.21.15.86:0/2273342493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6a80b0c60 con 0x7fa6a80d1970 2024-04-25T14:17:27.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.274+0000 7fa69e000640 1 -- 172.21.15.86:0/2273342493 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7fa6940039d0 con 0x7fa6a80d1970 2024-04-25T14:17:27.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.274+0000 7fa69e000640 1 --2- 172.21.15.86:0/2273342493 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fa68806f480 0x7fa688071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:27.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.274+0000 7fa69e000640 1 -- 172.21.15.86:0/2273342493 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fa69408f610 con 0x7fa6a80d1970 2024-04-25T14:17:27.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.274+0000 7fa6a5e00640 1 --2- 172.21.15.86:0/2273342493 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fa68806f480 0x7fa688071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:27.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.278+0000 7fa6a5e00640 1 --2- 172.21.15.86:0/2273342493 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fa68806f480 0x7fa688071990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa6980ae000 tx=0x7fa6980a7070 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:27.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.282+0000 7fa69e000640 1 -- 172.21.15.86:0/2273342493 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa69405c220 con 0x7fa6a80d1970 2024-04-25T14:17:27.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.470+0000 7fa6a7200640 1 -- 172.21.15.86:0/2273342493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6a8076b80 con 0x7fa6a80d1970 2024-04-25T14:17:27.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.470+0000 7fa69e000640 1 -- 172.21.15.86:0/2273342493 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7fa69405bb20 con 0x7fa6a80d1970 2024-04-25T14:17:27.468 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:27.468 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:27.469 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:27.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.470+0000 7fa6a7200640 1 -- 172.21.15.86:0/2273342493 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fa68806f480 msgr2=0x7fa688071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:27.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.470+0000 7fa6a7200640 1 --2- 172.21.15.86:0/2273342493 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fa68806f480 0x7fa688071990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa6980ae000 tx=0x7fa6980a7070 comp rx=0 tx=0).stop 2024-04-25T14:17:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.470+0000 7fa6a7200640 1 -- 172.21.15.86:0/2273342493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a80d1970 msgr2=0x7fa6a814edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.470+0000 7fa6a7200640 1 --2- 172.21.15.86:0/2273342493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a80d1970 0x7fa6a814edf0 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7fa69400c7f0 tx=0x7fa69400ccb0 comp rx=0 tx=0).stop 2024-04-25T14:17:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.470+0000 7fa6a7200640 1 -- 172.21.15.86:0/2273342493 shutdown_connections 2024-04-25T14:17:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.470+0000 7fa6a7200640 1 --2- 172.21.15.86:0/2273342493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6a815dcd0 0x7fa6a80b0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.470+0000 7fa6a7200640 1 --2- 172.21.15.86:0/2273342493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a80d1970 0x7fa6a814edf0 unknown :-1 s=CLOSED pgs=2365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.470+0000 7fa6a7200640 1 --2- 172.21.15.86:0/2273342493 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fa68806f480 0x7fa688071990 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.470+0000 7fa6a7200640 1 --2- 172.21.15.86:0/2273342493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a80ced30 0x7fa6a815f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.470+0000 7fa6a7200640 1 -- 172.21.15.86:0/2273342493 >> 172.21.15.86:0/2273342493 conn(0x7fa6a809ba30 msgr2=0x7fa6a815b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.470+0000 7fa6a7200640 1 -- 172.21.15.86:0/2273342493 shutdown_connections 2024-04-25T14:17:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:27.470+0000 7fa6a7200640 1 -- 172.21.15.86:0/2273342493 wait complete. 2024-04-25T14:17:27.893 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:27.893 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:28.894 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:29.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.054+0000 7f9b52000640 1 Processor -- start 2024-04-25T14:17:29.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.054+0000 7f9b52000640 1 -- start start 2024-04-25T14:17:29.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.054+0000 7f9b52000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b4c0d2c80 0x7f9b4c15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:29.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.054+0000 7f9b52000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9b4c070e20 0x7f9b4c071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:29.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.054+0000 7f9b52000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b4c071770 con 0x7f9b4c0722f0 2024-04-25T14:17:29.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.054+0000 7f9b52000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9b4c0ce720 con 0x7f9b4c070e20 2024-04-25T14:17:29.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.054+0000 7f9b52000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9b4c0cd8d0 con 0x7f9b4c0d2c80 2024-04-25T14:17:29.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.054+0000 7f9b4b400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b4c0d2c80 0x7f9b4c15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:29.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.054+0000 7f9b4aa00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9b4c070e20 0x7f9b4c071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:29.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.054+0000 7f9b4aa00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9b4c070e20 0x7f9b4c071230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:42300/0 (socket says 172.21.15.86:42300) 2024-04-25T14:17:29.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.054+0000 7f9b4aa00640 1 -- 172.21.15.86:0/2651634873 learned_addr learned my addr 172.21.15.86:0/2651634873 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:29.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.058+0000 7f9b4aa00640 1 -- 172.21.15.86:0/2651634873 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b4c0d2c80 msgr2=0x7f9b4c15dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:29.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.058+0000 7f9b4aa00640 1 --2- 172.21.15.86:0/2651634873 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b4c0d2c80 0x7f9b4c15dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:29.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.058+0000 7f9b4aa00640 1 -- 172.21.15.86:0/2651634873 >> v1:172.21.15.155:6789/0 conn(0x7f9b4c0722f0 legacy=0x7f9b4c072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:29.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.058+0000 7f9b4aa00640 1 -- 172.21.15.86:0/2651634873 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4c0ce970 con 0x7f9b4c070e20 2024-04-25T14:17:29.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.058+0000 7f9b4b400640 1 --2- 172.21.15.86:0/2651634873 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b4c0d2c80 0x7f9b4c15dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:17:29.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.058+0000 7f9b4aa00640 1 --2- 172.21.15.86:0/2651634873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b4c070e20 0x7f9b4c071230 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f9b34006fd0 tx=0x7f9b3401c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99d5792ab4b870c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:29.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.058+0000 7f9b4a000640 1 -- 172.21.15.86:0/2651634873 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b340b1070 con 0x7f9b4c070e20 2024-04-25T14:17:29.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.058+0000 7f9b4a000640 1 -- 172.21.15.86:0/2651634873 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b340ac620 con 0x7f9b4c070e20 2024-04-25T14:17:29.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.058+0000 7f9b4a000640 1 -- 172.21.15.86:0/2651634873 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b340ac9f0 con 0x7f9b4c070e20 2024-04-25T14:17:29.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.062+0000 7f9b52000640 1 -- 172.21.15.86:0/2651634873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b4c070e20 msgr2=0x7f9b4c071230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:29.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.062+0000 7f9b52000640 1 --2- 172.21.15.86:0/2651634873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b4c070e20 0x7f9b4c071230 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f9b34006fd0 tx=0x7f9b3401c460 comp rx=0 tx=0).stop 2024-04-25T14:17:29.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.062+0000 7f9b52000640 1 -- 172.21.15.86:0/2651634873 shutdown_connections 2024-04-25T14:17:29.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.062+0000 7f9b52000640 1 --2- 172.21.15.86:0/2651634873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b4c070e20 0x7f9b4c071230 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:29.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.062+0000 7f9b52000640 1 --2- 172.21.15.86:0/2651634873 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b4c0d2c80 0x7f9b4c15dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:29.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.062+0000 7f9b52000640 1 -- 172.21.15.86:0/2651634873 >> 172.21.15.86:0/2651634873 conn(0x7f9b4c0cffc0 msgr2=0x7f9b4c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:29.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.062+0000 7f9b52000640 1 -- 172.21.15.86:0/2651634873 shutdown_connections 2024-04-25T14:17:29.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.062+0000 7f9b52000640 1 -- 172.21.15.86:0/2651634873 wait complete. 2024-04-25T14:17:29.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.062+0000 7f9b52000640 1 Processor -- start 2024-04-25T14:17:29.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.062+0000 7f9b52000640 1 -- start start 2024-04-25T14:17:29.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.066+0000 7f9b52000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b4c070e20 0x7f9b4c0c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:29.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.066+0000 7f9b52000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b4c0722f0 0x7f9b4c1f7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:29.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.066+0000 7f9b52000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b4c0d2c80 0x7f9b4c0b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:29.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.066+0000 7f9b52000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c0d21a0 con 0x7f9b4c0d2c80 2024-04-25T14:17:29.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.066+0000 7f9b52000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c0d1ec0 con 0x7f9b4c070e20 2024-04-25T14:17:29.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.066+0000 7f9b52000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9b4c0d2480 con 0x7f9b4c0722f0 2024-04-25T14:17:29.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.066+0000 7f9b4be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b4c0d2c80 0x7f9b4c0b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:29.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.066+0000 7f9b4be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b4c0d2c80 0x7f9b4c0b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43696/0 (socket says 172.21.15.86:43696) 2024-04-25T14:17:29.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.066+0000 7f9b4be00640 1 -- 172.21.15.86:0/2673815726 learned_addr learned my addr 172.21.15.86:0/2673815726 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:29.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.066+0000 7f9b4b400640 1 --2- 172.21.15.86:0/2673815726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b4c070e20 0x7f9b4c0c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:29.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.066+0000 7f9b4aa00640 1 --2- 172.21.15.86:0/2673815726 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b4c0722f0 0x7f9b4c1f7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:29.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.066+0000 7f9b4be00640 1 -- 172.21.15.86:0/2673815726 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b4c0722f0 msgr2=0x7f9b4c1f7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:29.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.066+0000 7f9b4be00640 1 --2- 172.21.15.86:0/2673815726 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b4c0722f0 0x7f9b4c1f7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:29.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.066+0000 7f9b4be00640 1 -- 172.21.15.86:0/2673815726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b4c070e20 msgr2=0x7f9b4c0c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:29.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.066+0000 7f9b4be00640 1 --2- 172.21.15.86:0/2673815726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b4c070e20 0x7f9b4c0c98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:29.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.066+0000 7f9b4be00640 1 -- 172.21.15.86:0/2673815726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4c15fcb0 con 0x7f9b4c0d2c80 2024-04-25T14:17:29.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.066+0000 7f9b4be00640 1 --2- 172.21.15.86:0/2673815726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b4c0d2c80 0x7f9b4c0b1ad0 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400faf0 tx=0x7f9b4400ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:29.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.066+0000 7f9b43400640 1 -- 172.21.15.86:0/2673815726 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4401b070 con 0x7f9b4c0d2c80 2024-04-25T14:17:29.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.066+0000 7f9b43400640 1 -- 172.21.15.86:0/2673815726 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b44009800 con 0x7f9b4c0d2c80 2024-04-25T14:17:29.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.066+0000 7f9b52000640 1 -- 172.21.15.86:0/2673815726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b4c15ff90 con 0x7f9b4c0d2c80 2024-04-25T14:17:29.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.066+0000 7f9b43400640 1 -- 172.21.15.86:0/2673815726 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b440084e0 con 0x7f9b4c0d2c80 2024-04-25T14:17:29.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.066+0000 7f9b52000640 1 -- 172.21.15.86:0/2673815726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b4c160550 con 0x7f9b4c0d2c80 2024-04-25T14:17:29.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.066+0000 7f9b52000640 1 -- 172.21.15.86:0/2673815726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b4c0720e0 con 0x7f9b4c0d2c80 2024-04-25T14:17:29.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.070+0000 7f9b43400640 1 -- 172.21.15.86:0/2673815726 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f9b4401e070 con 0x7f9b4c0d2c80 2024-04-25T14:17:29.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.070+0000 7f9b43400640 1 --2- 172.21.15.86:0/2673815726 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9b2806f450 0x7f9b28071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:29.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.070+0000 7f9b43400640 1 -- 172.21.15.86:0/2673815726 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f9b44091dc0 con 0x7f9b4c0d2c80 2024-04-25T14:17:29.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.070+0000 7f9b4b400640 1 --2- 172.21.15.86:0/2673815726 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9b2806f450 0x7f9b28071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:29.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.070+0000 7f9b4b400640 1 --2- 172.21.15.86:0/2673815726 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9b2806f450 0x7f9b28071960 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c009bd0 tx=0x7f9b3c00b040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:29.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.074+0000 7f9b43400640 1 -- 172.21.15.86:0/2673815726 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9b4c071770 con 0x7f9b4c0d2c80 2024-04-25T14:17:29.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.262+0000 7f9b52000640 1 -- 172.21.15.86:0/2673815726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b4c068d70 con 0x7f9b4c0d2c80 2024-04-25T14:17:29.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.262+0000 7f9b43400640 1 -- 172.21.15.86:0/2673815726 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f9b4405e2e0 con 0x7f9b4c0d2c80 2024-04-25T14:17:29.262 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:29.262 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:29.262 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.266+0000 7f9b52000640 1 -- 172.21.15.86:0/2673815726 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9b2806f450 msgr2=0x7f9b28071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.266+0000 7f9b52000640 1 --2- 172.21.15.86:0/2673815726 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9b2806f450 0x7f9b28071960 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c009bd0 tx=0x7f9b3c00b040 comp rx=0 tx=0).stop 2024-04-25T14:17:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.266+0000 7f9b52000640 1 -- 172.21.15.86:0/2673815726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b4c0d2c80 msgr2=0x7f9b4c0b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.266+0000 7f9b52000640 1 --2- 172.21.15.86:0/2673815726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b4c0d2c80 0x7f9b4c0b1ad0 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400faf0 tx=0x7f9b4400ae50 comp rx=0 tx=0).stop 2024-04-25T14:17:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.266+0000 7f9b52000640 1 -- 172.21.15.86:0/2673815726 shutdown_connections 2024-04-25T14:17:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.266+0000 7f9b52000640 1 --2- 172.21.15.86:0/2673815726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b4c0d2c80 0x7f9b4c0b1ad0 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.266+0000 7f9b52000640 1 --2- 172.21.15.86:0/2673815726 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9b2806f450 0x7f9b28071960 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.266+0000 7f9b52000640 1 --2- 172.21.15.86:0/2673815726 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b4c0722f0 0x7f9b4c1f7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.266+0000 7f9b52000640 1 --2- 172.21.15.86:0/2673815726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b4c070e20 0x7f9b4c0c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.266+0000 7f9b52000640 1 -- 172.21.15.86:0/2673815726 >> 172.21.15.86:0/2673815726 conn(0x7f9b4c0cffc0 msgr2=0x7f9b4c0b34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:29.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.266+0000 7f9b52000640 1 -- 172.21.15.86:0/2673815726 shutdown_connections 2024-04-25T14:17:29.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:29.266+0000 7f9b52000640 1 -- 172.21.15.86:0/2673815726 wait complete. 2024-04-25T14:17:29.688 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:29.688 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:30.689 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:30.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.846+0000 7f9dd5800640 1 Processor -- start 2024-04-25T14:17:30.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.846+0000 7f9dd5800640 1 -- start start 2024-04-25T14:17:30.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.850+0000 7f9dd5800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9dd015ecf0 0x7f9dd006f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:30.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.850+0000 7f9dd5800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9dd015f8a0 0x7f9dd006fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:30.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.850+0000 7f9dd5800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dd006d4a0 con 0x7f9dd015e8e0 2024-04-25T14:17:30.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.850+0000 7f9dd5800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9dd015e450 con 0x7f9dd015f8a0 2024-04-25T14:17:30.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.850+0000 7f9dd5800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9dd00d0510 con 0x7f9dd015ecf0 2024-04-25T14:17:30.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.850+0000 7f9dd4e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9dd015f8a0 0x7f9dd006fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:30.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.850+0000 7f9dd4e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9dd015f8a0 0x7f9dd006fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43700/0 (socket says 172.21.15.86:43700) 2024-04-25T14:17:30.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.850+0000 7f9dd4e00640 1 -- 172.21.15.86:0/2641920595 learned_addr learned my addr 172.21.15.86:0/2641920595 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:30.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.850+0000 7f9dc7400640 1 --2- 172.21.15.86:0/2641920595 >> v2:172.21.15.155:3300/0 conn(0x7f9dd015ecf0 0x7f9dd006f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:30.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.850+0000 7f9dd4e00640 1 -- 172.21.15.86:0/2641920595 >> v2:172.21.15.155:3300/0 conn(0x7f9dd015ecf0 msgr2=0x7f9dd006f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:30.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.850+0000 7f9dd4e00640 1 --2- 172.21.15.86:0/2641920595 >> v2:172.21.15.155:3300/0 conn(0x7f9dd015ecf0 0x7f9dd006f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:30.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.850+0000 7f9dd4e00640 1 -- 172.21.15.86:0/2641920595 >> v1:172.21.15.86:6789/0 conn(0x7f9dd015e8e0 legacy=0x7f9dd015f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:30.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.850+0000 7f9dd4e00640 1 -- 172.21.15.86:0/2641920595 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd00d07f0 con 0x7f9dd015f8a0 2024-04-25T14:17:30.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.854+0000 7f9dd4e00640 1 --2- 172.21.15.86:0/2641920595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd015f8a0 0x7f9dd006fd10 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0074ab0 tx=0x7f9dc801c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54de84e46691ff3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:30.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.854+0000 7f9dcf400640 1 -- 172.21.15.86:0/2641920595 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc80a5510 con 0x7f9dd015f8a0 2024-04-25T14:17:30.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.854+0000 7f9dcf400640 1 -- 172.21.15.86:0/2641920595 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dc80a56b0 con 0x7f9dd015f8a0 2024-04-25T14:17:30.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.854+0000 7f9dcf400640 1 -- 172.21.15.86:0/2641920595 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc80a5a80 con 0x7f9dd015f8a0 2024-04-25T14:17:30.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.854+0000 7f9dd5800640 1 -- 172.21.15.86:0/2641920595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd015f8a0 msgr2=0x7f9dd006fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:30.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.854+0000 7f9dd5800640 1 --2- 172.21.15.86:0/2641920595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd015f8a0 0x7f9dd006fd10 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0074ab0 tx=0x7f9dc801c740 comp rx=0 tx=0).stop 2024-04-25T14:17:30.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.854+0000 7f9dd5800640 1 -- 172.21.15.86:0/2641920595 shutdown_connections 2024-04-25T14:17:30.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.854+0000 7f9dd5800640 1 --2- 172.21.15.86:0/2641920595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd015f8a0 0x7f9dd006fd10 unknown :-1 s=CLOSED pgs=2368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:30.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.854+0000 7f9dd5800640 1 --2- 172.21.15.86:0/2641920595 >> v2:172.21.15.155:3300/0 conn(0x7f9dd015ecf0 0x7f9dd006f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:30.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.854+0000 7f9dd5800640 1 -- 172.21.15.86:0/2641920595 >> 172.21.15.86:0/2641920595 conn(0x7f9dd00b4850 msgr2=0x7f9dd00b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:30.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.854+0000 7f9dd5800640 1 -- 172.21.15.86:0/2641920595 shutdown_connections 2024-04-25T14:17:30.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.854+0000 7f9dd5800640 1 -- 172.21.15.86:0/2641920595 wait complete. 2024-04-25T14:17:30.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.854+0000 7f9dd5800640 1 Processor -- start 2024-04-25T14:17:30.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.854+0000 7f9dd5800640 1 -- start start 2024-04-25T14:17:30.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.858+0000 7f9dd5800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9dd015e8e0 0x7f9dd01fd760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:30.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.858+0000 7f9dd5800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dd015ecf0 0x7f9dd0146370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:30.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.858+0000 7f9dd5800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd015f8a0 0x7f9dd014cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:30.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.858+0000 7f9dd5800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd00d0c50 con 0x7f9dd015f8a0 2024-04-25T14:17:30.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.858+0000 7f9dd5800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd00d0ad0 con 0x7f9dd015ecf0 2024-04-25T14:17:30.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.858+0000 7f9dd5800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9dd016ae20 con 0x7f9dd015e8e0 2024-04-25T14:17:30.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.858+0000 7f9dd4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd015f8a0 0x7f9dd014cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:30.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.858+0000 7f9dd4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd015f8a0 0x7f9dd014cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43706/0 (socket says 172.21.15.86:43706) 2024-04-25T14:17:30.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.858+0000 7f9dd4e00640 1 -- 172.21.15.86:0/2382486303 learned_addr learned my addr 172.21.15.86:0/2382486303 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:30.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.858+0000 7f9dc7400640 1 --2- 172.21.15.86:0/2382486303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dd015ecf0 0x7f9dd0146370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:30.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.858+0000 7f9dcfe00640 1 --2- 172.21.15.86:0/2382486303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9dd015e8e0 0x7f9dd01fd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:30.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.858+0000 7f9dd4e00640 1 -- 172.21.15.86:0/2382486303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9dd015e8e0 msgr2=0x7f9dd01fd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:30.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.858+0000 7f9dd4e00640 1 --2- 172.21.15.86:0/2382486303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9dd015e8e0 0x7f9dd01fd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:30.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.858+0000 7f9dd4e00640 1 -- 172.21.15.86:0/2382486303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dd015ecf0 msgr2=0x7f9dd0146370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:30.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.858+0000 7f9dd4e00640 1 --2- 172.21.15.86:0/2382486303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dd015ecf0 0x7f9dd0146370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:30.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.858+0000 7f9dd4e00640 1 -- 172.21.15.86:0/2382486303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd014aac0 con 0x7f9dd015f8a0 2024-04-25T14:17:30.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.858+0000 7f9dd4e00640 1 --2- 172.21.15.86:0/2382486303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd015f8a0 0x7f9dd014cef0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80af040 tx=0x7f9dc80194e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:30.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.858+0000 7f9dccc00640 1 -- 172.21.15.86:0/2382486303 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc80bc020 con 0x7f9dd015f8a0 2024-04-25T14:17:30.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.858+0000 7f9dccc00640 1 -- 172.21.15.86:0/2382486303 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dc80b8670 con 0x7f9dd015f8a0 2024-04-25T14:17:30.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.858+0000 7f9dd5800640 1 -- 172.21.15.86:0/2382486303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd009aab0 con 0x7f9dd015f8a0 2024-04-25T14:17:30.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.858+0000 7f9dccc00640 1 -- 172.21.15.86:0/2382486303 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc80a5510 con 0x7f9dd015f8a0 2024-04-25T14:17:30.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.858+0000 7f9dd5800640 1 -- 172.21.15.86:0/2382486303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd00b0d80 con 0x7f9dd015f8a0 2024-04-25T14:17:30.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.858+0000 7f9dd5800640 1 -- 172.21.15.86:0/2382486303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dd0076b80 con 0x7f9dd015f8a0 2024-04-25T14:17:30.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.862+0000 7f9dccc00640 1 -- 172.21.15.86:0/2382486303 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f9dc80bf020 con 0x7f9dd015f8a0 2024-04-25T14:17:30.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.862+0000 7f9dccc00640 1 --2- 172.21.15.86:0/2382486303 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9dac06f450 0x7f9dac071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:30.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.862+0000 7f9dccc00640 1 -- 172.21.15.86:0/2382486303 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f9dc812a390 con 0x7f9dd015f8a0 2024-04-25T14:17:30.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.862+0000 7f9dcfe00640 1 --2- 172.21.15.86:0/2382486303 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9dac06f450 0x7f9dac071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:30.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.862+0000 7f9dcfe00640 1 --2- 172.21.15.86:0/2382486303 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9dac06f450 0x7f9dac071960 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000b810 tx=0x7f9dc0008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:30.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:30.866+0000 7f9dccc00640 1 -- 172.21.15.86:0/2382486303 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9dc80c6030 con 0x7f9dd015f8a0 2024-04-25T14:17:31.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:31.058+0000 7f9dd5800640 1 -- 172.21.15.86:0/2382486303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9dd015f310 con 0x7f9dd015f8a0 2024-04-25T14:17:31.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:31.058+0000 7f9dccc00640 1 -- 172.21.15.86:0/2382486303 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f9dd014aac0 con 0x7f9dd015f8a0 2024-04-25T14:17:31.275 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:31.275 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:31.277 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:31.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:31.058+0000 7f9dd5800640 1 -- 172.21.15.86:0/2382486303 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9dac06f450 msgr2=0x7f9dac071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:31.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:31.058+0000 7f9dd5800640 1 --2- 172.21.15.86:0/2382486303 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9dac06f450 0x7f9dac071960 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000b810 tx=0x7f9dc0008040 comp rx=0 tx=0).stop 2024-04-25T14:17:31.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:31.058+0000 7f9dd5800640 1 -- 172.21.15.86:0/2382486303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd015f8a0 msgr2=0x7f9dd014cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:31.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:31.058+0000 7f9dd5800640 1 --2- 172.21.15.86:0/2382486303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd015f8a0 0x7f9dd014cef0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80af040 tx=0x7f9dc80194e0 comp rx=0 tx=0).stop 2024-04-25T14:17:31.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:31.058+0000 7f9dd5800640 1 -- 172.21.15.86:0/2382486303 shutdown_connections 2024-04-25T14:17:31.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:31.058+0000 7f9dd5800640 1 --2- 172.21.15.86:0/2382486303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dd015f8a0 0x7f9dd014cef0 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:31.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:31.058+0000 7f9dd5800640 1 --2- 172.21.15.86:0/2382486303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dd015ecf0 0x7f9dd0146370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:31.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:31.058+0000 7f9dd5800640 1 --2- 172.21.15.86:0/2382486303 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f9dac06f450 0x7f9dac071960 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:31.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:31.058+0000 7f9dd5800640 1 --2- 172.21.15.86:0/2382486303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9dd015e8e0 0x7f9dd01fd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:31.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:31.058+0000 7f9dd5800640 1 -- 172.21.15.86:0/2382486303 >> 172.21.15.86:0/2382486303 conn(0x7f9dd00b4850 msgr2=0x7f9dd015caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:31.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:31.058+0000 7f9dd5800640 1 -- 172.21.15.86:0/2382486303 shutdown_connections 2024-04-25T14:17:31.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:31.058+0000 7f9dd5800640 1 -- 172.21.15.86:0/2382486303 wait complete. 2024-04-25T14:17:31.682 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:31.682 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:32.684 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.846+0000 7fc61aa00640 1 Processor -- start 2024-04-25T14:17:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.846+0000 7fc61aa00640 1 -- start start 2024-04-25T14:17:32.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.846+0000 7fc61aa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6140cde90 0x7fc6140ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:32.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.846+0000 7fc61aa00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc61415e5c0 con 0x7fc6140d1df0 2024-04-25T14:17:32.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.846+0000 7fc61aa00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc61415e8a0 con 0x7fc614070330 2024-04-25T14:17:32.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.846+0000 7fc61aa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc61406fd40 con 0x7fc6140cde90 2024-04-25T14:17:32.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.846+0000 7fc613400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc6140d1df0 0x7fc6140cd780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45872/0 (socket says 172.21.15.86:45872) 2024-04-25T14:17:32.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.846+0000 7fc613400640 1 -- 172.21.15.86:0/3988809447 learned_addr learned my addr 172.21.15.86:0/3988809447 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:17:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.850+0000 7fc618a00640 1 --2- 172.21.15.86:0/3988809447 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6140cde90 0x7fc6140ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.850+0000 7fc612a00640 1 -- 172.21.15.86:0/3988809447 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4101087910 0 0) 0x7fc61415e5c0 con 0x7fc6140d1df0 2024-04-25T14:17:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.850+0000 7fc612a00640 1 -- 172.21.15.86:0/3988809447 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5f8003530 con 0x7fc6140d1df0 2024-04-25T14:17:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.850+0000 7fc612a00640 1 -- 172.21.15.86:0/3988809447 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 644847152 0 0) 0x7fc5f8003530 con 0x7fc6140d1df0 2024-04-25T14:17:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.850+0000 7fc612a00640 1 -- 172.21.15.86:0/3988809447 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6140cde90 msgr2=0x7fc6140ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.850+0000 7fc612a00640 1 --2- 172.21.15.86:0/3988809447 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6140cde90 0x7fc6140ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.850+0000 7fc612a00640 1 -- 172.21.15.86:0/3988809447 >> v1:172.21.15.155:6789/0 conn(0x7fc614070330 legacy=0x7fc6140d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.850+0000 7fc612a00640 1 -- 172.21.15.86:0/3988809447 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61415eb80 con 0x7fc6140d1df0 2024-04-25T14:17:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.850+0000 7fc612a00640 1 -- 172.21.15.86:0/3988809447 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7fc600001680 con 0x7fc6140d1df0 2024-04-25T14:17:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.850+0000 7fc612a00640 1 -- 172.21.15.86:0/3988809447 >> v1:172.21.15.86:6789/0 conn(0x7fc6140d1df0 legacy=0x7fc6140cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.850+0000 7fc612a00640 1 --2- 172.21.15.86:0/3988809447 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6140cde90 0x7fc5f8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.850+0000 7fc612a00640 1 --2- 172.21.15.86:0/3988809447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5f8003fe0 0x7fc5f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:32.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.850+0000 7fc612a00640 1 --2- 172.21.15.86:0/3988809447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5f8004d50 0x7fc5f80051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:32.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.850+0000 7fc612a00640 1 -- 172.21.15.86:0/3988809447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f800bc50 con 0x7fc5f8003fe0 2024-04-25T14:17:32.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.850+0000 7fc612a00640 1 -- 172.21.15.86:0/3988809447 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f800be00 con 0x7fc5f8004d50 2024-04-25T14:17:32.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.850+0000 7fc612a00640 1 -- 172.21.15.86:0/3988809447 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc5f800c000 con 0x7fc6140cde90 2024-04-25T14:17:32.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.850+0000 7fc613400640 1 --2- 172.21.15.86:0/3988809447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5f8003fe0 0x7fc5f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:32.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.850+0000 7fc618a00640 1 --2- 172.21.15.86:0/3988809447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5f8004d50 0x7fc5f80051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:32.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.850+0000 7fc613e00640 1 --2- 172.21.15.86:0/3988809447 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6140cde90 0x7fc5f8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:32.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.850+0000 7fc613400640 1 -- 172.21.15.86:0/3988809447 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6140cde90 msgr2=0x7fc5f8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:32.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.850+0000 7fc613400640 1 --2- 172.21.15.86:0/3988809447 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6140cde90 0x7fc5f8003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:32.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.850+0000 7fc613400640 1 -- 172.21.15.86:0/3988809447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5f8004d50 msgr2=0x7fc5f80051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:32.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.850+0000 7fc613400640 1 --2- 172.21.15.86:0/3988809447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5f8004d50 0x7fc5f80051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:32.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.850+0000 7fc613400640 1 -- 172.21.15.86:0/3988809447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f8003530 con 0x7fc5f8003fe0 2024-04-25T14:17:32.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.854+0000 7fc613400640 1 --2- 172.21.15.86:0/3988809447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5f8003fe0 0x7fc5f8004810 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7fc614074ed0 tx=0x7fc60001cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1daabd6672469336 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:32.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.854+0000 7fc612a00640 1 -- 172.21.15.86:0/3988809447 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600007d20 con 0x7fc5f8003fe0 2024-04-25T14:17:32.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.854+0000 7fc612a00640 1 -- 172.21.15.86:0/3988809447 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6000a9070 con 0x7fc5f8003fe0 2024-04-25T14:17:32.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.854+0000 7fc612a00640 1 -- 172.21.15.86:0/3988809447 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600005d80 con 0x7fc5f8003fe0 2024-04-25T14:17:32.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.854+0000 7fc61aa00640 1 -- 172.21.15.86:0/3988809447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5f8003fe0 msgr2=0x7fc5f8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:32.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.854+0000 7fc61aa00640 1 --2- 172.21.15.86:0/3988809447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5f8003fe0 0x7fc5f8004810 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7fc614074ed0 tx=0x7fc60001cbf0 comp rx=0 tx=0).stop 2024-04-25T14:17:32.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.854+0000 7fc618a00640 1 -- 172.21.15.86:0/3988809447 reap_dead start 2024-04-25T14:17:32.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.854+0000 7fc61aa00640 1 -- 172.21.15.86:0/3988809447 shutdown_connections 2024-04-25T14:17:32.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.854+0000 7fc61aa00640 1 -- 172.21.15.86:0/3988809447 >> 172.21.15.86:0/3988809447 conn(0x7fc6140b4650 msgr2=0x7fc6140b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:32.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.854+0000 7fc61aa00640 1 -- 172.21.15.86:0/3988809447 shutdown_connections 2024-04-25T14:17:32.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.854+0000 7fc61aa00640 1 -- 172.21.15.86:0/3988809447 wait complete. 2024-04-25T14:17:32.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.854+0000 7fc61aa00640 1 Processor -- start 2024-04-25T14:17:32.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.854+0000 7fc61aa00640 1 -- start start 2024-04-25T14:17:32.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.858+0000 7fc61aa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc5f8003fe0 0x7fc61409ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:32.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.858+0000 7fc61aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc61409efa0 0x7fc6140ca860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:32.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.858+0000 7fc61aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6140cada0 0x7fc61414a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:32.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.858+0000 7fc61aa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc6140708c0 con 0x7fc61409efa0 2024-04-25T14:17:32.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.858+0000 7fc61aa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc614070740 con 0x7fc6140cada0 2024-04-25T14:17:32.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.858+0000 7fc61aa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc6141697b0 con 0x7fc5f8003fe0 2024-04-25T14:17:32.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.858+0000 7fc613400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc61409efa0 0x7fc6140ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.858+0000 7fc613400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc61409efa0 0x7fc6140ca860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43722/0 (socket says 172.21.15.86:43722) 2024-04-25T14:17:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.858+0000 7fc613400640 1 -- 172.21.15.86:0/87934054 learned_addr learned my addr 172.21.15.86:0/87934054 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.858+0000 7fc613e00640 1 --2- 172.21.15.86:0/87934054 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc5f8003fe0 0x7fc61409ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.858+0000 7fc618a00640 1 --2- 172.21.15.86:0/87934054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6140cada0 0x7fc61414a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.858+0000 7fc613400640 1 -- 172.21.15.86:0/87934054 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc5f8003fe0 msgr2=0x7fc61409ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.858+0000 7fc613400640 1 --2- 172.21.15.86:0/87934054 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc5f8003fe0 0x7fc61409ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.858+0000 7fc613400640 1 -- 172.21.15.86:0/87934054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6140cada0 msgr2=0x7fc61414a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.858+0000 7fc613400640 1 --2- 172.21.15.86:0/87934054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6140cada0 0x7fc61414a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.858+0000 7fc613400640 1 -- 172.21.15.86:0/87934054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6141f9770 con 0x7fc61409efa0 2024-04-25T14:17:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.858+0000 7fc613400640 1 --2- 172.21.15.86:0/87934054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc61409efa0 0x7fc6140ca860 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7fc60001c660 tx=0x7fc5f8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.858+0000 7fc60be00640 1 -- 172.21.15.86:0/87934054 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600007d20 con 0x7fc61409efa0 2024-04-25T14:17:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.858+0000 7fc60be00640 1 -- 172.21.15.86:0/87934054 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6000a58b0 con 0x7fc61409efa0 2024-04-25T14:17:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.858+0000 7fc60be00640 1 -- 172.21.15.86:0/87934054 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6000a5ca0 con 0x7fc61409efa0 2024-04-25T14:17:32.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.858+0000 7fc61aa00640 1 -- 172.21.15.86:0/87934054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc61409dae0 con 0x7fc61409efa0 2024-04-25T14:17:32.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.858+0000 7fc61aa00640 1 -- 172.21.15.86:0/87934054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc61409e0a0 con 0x7fc61409efa0 2024-04-25T14:17:32.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.858+0000 7fc61aa00640 1 -- 172.21.15.86:0/87934054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc614069d00 con 0x7fc61409efa0 2024-04-25T14:17:32.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.862+0000 7fc60be00640 1 -- 172.21.15.86:0/87934054 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7fc6000047f0 con 0x7fc61409efa0 2024-04-25T14:17:32.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.862+0000 7fc60be00640 1 --2- 172.21.15.86:0/87934054 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc5f406f450 0x7fc5f4071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:32.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.862+0000 7fc60be00640 1 -- 172.21.15.86:0/87934054 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fc6001294a0 con 0x7fc61409efa0 2024-04-25T14:17:32.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.862+0000 7fc613e00640 1 --2- 172.21.15.86:0/87934054 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc5f406f450 0x7fc5f4071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:32.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.862+0000 7fc613e00640 1 --2- 172.21.15.86:0/87934054 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc5f406f450 0x7fc5f4071960 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc604000f90 tx=0x7fc604009040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:32.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:32.866+0000 7fc60be00640 1 -- 172.21.15.86:0/87934054 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc6000c0030 con 0x7fc61409efa0 2024-04-25T14:17:33.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:33.054+0000 7fc61aa00640 1 -- 172.21.15.86:0/87934054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc61409d870 con 0x7fc61409efa0 2024-04-25T14:17:33.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:33.058+0000 7fc60be00640 1 -- 172.21.15.86:0/87934054 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7fc600016930 con 0x7fc61409efa0 2024-04-25T14:17:33.055 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:33.055 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:33.055 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:33.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:33.058+0000 7fc61aa00640 1 -- 172.21.15.86:0/87934054 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc5f406f450 msgr2=0x7fc5f4071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:33.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:33.058+0000 7fc61aa00640 1 --2- 172.21.15.86:0/87934054 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc5f406f450 0x7fc5f4071960 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc604000f90 tx=0x7fc604009040 comp rx=0 tx=0).stop 2024-04-25T14:17:33.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:33.058+0000 7fc61aa00640 1 -- 172.21.15.86:0/87934054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc61409efa0 msgr2=0x7fc6140ca860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:33.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:33.058+0000 7fc61aa00640 1 --2- 172.21.15.86:0/87934054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc61409efa0 0x7fc6140ca860 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7fc60001c660 tx=0x7fc5f8005840 comp rx=0 tx=0).stop 2024-04-25T14:17:33.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:33.058+0000 7fc61aa00640 1 -- 172.21.15.86:0/87934054 shutdown_connections 2024-04-25T14:17:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:33.058+0000 7fc61aa00640 1 --2- 172.21.15.86:0/87934054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6140cada0 0x7fc61414a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:33.058+0000 7fc61aa00640 1 --2- 172.21.15.86:0/87934054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc61409efa0 0x7fc6140ca860 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:33.058+0000 7fc61aa00640 1 --2- 172.21.15.86:0/87934054 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc5f406f450 0x7fc5f4071960 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:33.058+0000 7fc61aa00640 1 --2- 172.21.15.86:0/87934054 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc5f8003fe0 0x7fc61409ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:33.058+0000 7fc61aa00640 1 -- 172.21.15.86:0/87934054 >> 172.21.15.86:0/87934054 conn(0x7fc6140b4650 msgr2=0x7fc61415cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:33.058+0000 7fc61aa00640 1 -- 172.21.15.86:0/87934054 shutdown_connections 2024-04-25T14:17:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:33.058+0000 7fc61aa00640 1 -- 172.21.15.86:0/87934054 wait complete. 2024-04-25T14:17:33.479 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:33.479 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:34.481 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:34.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.642+0000 7f7690c00640 1 Processor -- start 2024-04-25T14:17:34.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.642+0000 7f7690c00640 1 -- start start 2024-04-25T14:17:34.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.642+0000 7f7690c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f768c15c2a0 0x7f768c0ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:34.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.642+0000 7f7690c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f768c06fd50 0x7f768c070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:34.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.642+0000 7f7690c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f768c0b0a60 con 0x7f768c06c670 2024-04-25T14:17:34.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.642+0000 7f7690c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f768c06ef90 con 0x7f768c06fd50 2024-04-25T14:17:34.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.642+0000 7f7690c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f768c06f270 con 0x7f768c15c2a0 2024-04-25T14:17:34.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.642+0000 7f7689400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f768c06fd50 0x7f768c070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:34.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.642+0000 7f7689400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f768c06fd50 0x7f768c070160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43724/0 (socket says 172.21.15.86:43724) 2024-04-25T14:17:34.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.642+0000 7f7689400640 1 -- 172.21.15.86:0/3013328094 learned_addr learned my addr 172.21.15.86:0/3013328094 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:34.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.642+0000 7f7689e00640 1 --2- 172.21.15.86:0/3013328094 >> v2:172.21.15.155:3300/0 conn(0x7f768c15c2a0 0x7f768c0ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:34.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.642+0000 7f7689400640 1 -- 172.21.15.86:0/3013328094 >> v2:172.21.15.155:3300/0 conn(0x7f768c15c2a0 msgr2=0x7f768c0ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:34.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.642+0000 7f7689400640 1 --2- 172.21.15.86:0/3013328094 >> v2:172.21.15.155:3300/0 conn(0x7f768c15c2a0 0x7f768c0ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:34.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.642+0000 7f7689400640 1 -- 172.21.15.86:0/3013328094 >> v1:172.21.15.86:6789/0 conn(0x7f768c06c670 legacy=0x7f768c06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:34.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.642+0000 7f7689400640 1 -- 172.21.15.86:0/3013328094 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768c06f550 con 0x7f768c06fd50 2024-04-25T14:17:34.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.642+0000 7f7689e00640 1 --2- 172.21.15.86:0/3013328094 >> v2:172.21.15.155:3300/0 conn(0x7f768c15c2a0 0x7f768c0ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:17:34.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.646+0000 7f7689400640 1 --2- 172.21.15.86:0/3013328094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c06fd50 0x7f768c070160 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f768c072200 tx=0x7f767801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f5b79509faa713 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:34.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.646+0000 7f7688a00640 1 -- 172.21.15.86:0/3013328094 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76780ab620 con 0x7f768c06fd50 2024-04-25T14:17:34.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.646+0000 7f7688a00640 1 -- 172.21.15.86:0/3013328094 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76780b0070 con 0x7f768c06fd50 2024-04-25T14:17:34.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.646+0000 7f7688a00640 1 -- 172.21.15.86:0/3013328094 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76780ab9e0 con 0x7f768c06fd50 2024-04-25T14:17:34.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.646+0000 7f7690c00640 1 -- 172.21.15.86:0/3013328094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c06fd50 msgr2=0x7f768c070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:34.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.646+0000 7f7690c00640 1 --2- 172.21.15.86:0/3013328094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c06fd50 0x7f768c070160 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f768c072200 tx=0x7f767801c460 comp rx=0 tx=0).stop 2024-04-25T14:17:34.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.646+0000 7f7690c00640 1 -- 172.21.15.86:0/3013328094 shutdown_connections 2024-04-25T14:17:34.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.646+0000 7f7690c00640 1 --2- 172.21.15.86:0/3013328094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c06fd50 0x7f768c070160 unknown :-1 s=CLOSED pgs=2374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:34.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.646+0000 7f7690c00640 1 --2- 172.21.15.86:0/3013328094 >> v2:172.21.15.155:3300/0 conn(0x7f768c15c2a0 0x7f768c0ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:34.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.646+0000 7f7690c00640 1 -- 172.21.15.86:0/3013328094 >> 172.21.15.86:0/3013328094 conn(0x7f768c0b4650 msgr2=0x7f768c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:34.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.646+0000 7f7690c00640 1 -- 172.21.15.86:0/3013328094 shutdown_connections 2024-04-25T14:17:34.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.646+0000 7f7690c00640 1 -- 172.21.15.86:0/3013328094 wait complete. 2024-04-25T14:17:34.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.650+0000 7f7690c00640 1 Processor -- start 2024-04-25T14:17:34.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.650+0000 7f7690c00640 1 -- start start 2024-04-25T14:17:34.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.650+0000 7f7690c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f768c06c670 0x7f768c1669a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:34.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.650+0000 7f7690c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c06fd50 0x7f768c09a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:34.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.650+0000 7f7690c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f768c15c2a0 0x7f768c15e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:34.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.650+0000 7f7690c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f768c15c830 con 0x7f768c06fd50 2024-04-25T14:17:34.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.650+0000 7f7690c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f768c15c6b0 con 0x7f768c06c670 2024-04-25T14:17:34.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.650+0000 7f7690c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f768c0ae670 con 0x7f768c15c2a0 2024-04-25T14:17:34.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.650+0000 7f7689400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c06fd50 0x7f768c09a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:34.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.650+0000 7f7689400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c06fd50 0x7f768c09a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43728/0 (socket says 172.21.15.86:43728) 2024-04-25T14:17:34.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.650+0000 7f7689400640 1 -- 172.21.15.86:0/2850756008 learned_addr learned my addr 172.21.15.86:0/2850756008 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:34.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.650+0000 7f7689400640 1 -- 172.21.15.86:0/2850756008 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f768c15c2a0 msgr2=0x7f768c15e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:34.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.650+0000 7f7689400640 1 --2- 172.21.15.86:0/2850756008 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f768c15c2a0 0x7f768c15e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:34.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.650+0000 7f7689400640 1 -- 172.21.15.86:0/2850756008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f768c06c670 msgr2=0x7f768c1669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:34.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.650+0000 7f7689400640 1 --2- 172.21.15.86:0/2850756008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f768c06c670 0x7f768c1669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:34.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.650+0000 7f7689400640 1 -- 172.21.15.86:0/2850756008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768c09ae90 con 0x7f768c06fd50 2024-04-25T14:17:34.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.650+0000 7f7689400640 1 --2- 172.21.15.86:0/2850756008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c06fd50 0x7f768c09a950 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f767801a090 tx=0x7f767801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:34.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.650+0000 7f7682000640 1 -- 172.21.15.86:0/2850756008 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76780abdb0 con 0x7f768c06fd50 2024-04-25T14:17:34.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.650+0000 7f7682000640 1 -- 172.21.15.86:0/2850756008 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76780be3f0 con 0x7f768c06fd50 2024-04-25T14:17:34.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.650+0000 7f7682000640 1 -- 172.21.15.86:0/2850756008 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76780be7e0 con 0x7f768c06fd50 2024-04-25T14:17:34.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.650+0000 7f7690c00640 1 -- 172.21.15.86:0/2850756008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f768c09e950 con 0x7f768c06fd50 2024-04-25T14:17:34.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.650+0000 7f7690c00640 1 -- 172.21.15.86:0/2850756008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f768c09ef10 con 0x7f768c06fd50 2024-04-25T14:17:34.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.654+0000 7f7690c00640 1 -- 172.21.15.86:0/2850756008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f768c09e6c0 con 0x7f768c06fd50 2024-04-25T14:17:34.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.654+0000 7f7682000640 1 -- 172.21.15.86:0/2850756008 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f76780be980 con 0x7f768c06fd50 2024-04-25T14:17:34.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.654+0000 7f7682000640 1 --2- 172.21.15.86:0/2850756008 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f766c06f450 0x7f766c071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:34.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.654+0000 7f7682000640 1 -- 172.21.15.86:0/2850756008 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f7678129530 con 0x7f768c06fd50 2024-04-25T14:17:34.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.654+0000 7f7689e00640 1 --2- 172.21.15.86:0/2850756008 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f766c06f450 0x7f766c071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:34.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.654+0000 7f7689e00640 1 --2- 172.21.15.86:0/2850756008 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f766c06f450 0x7f766c071960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f767c006f10 tx=0x7f767c008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:34.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.658+0000 7f7682000640 1 -- 172.21.15.86:0/2850756008 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7678016110 con 0x7f768c06fd50 2024-04-25T14:17:34.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.846+0000 7f7690c00640 1 -- 172.21.15.86:0/2850756008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f768c069d00 con 0x7f768c06fd50 2024-04-25T14:17:34.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.846+0000 7f7682000640 1 -- 172.21.15.86:0/2850756008 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f76780f5b60 con 0x7f768c06fd50 2024-04-25T14:17:34.846 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:34.847 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:34.847 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:34.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.850+0000 7f7690c00640 1 -- 172.21.15.86:0/2850756008 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f766c06f450 msgr2=0x7f766c071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.850+0000 7f7690c00640 1 --2- 172.21.15.86:0/2850756008 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f766c06f450 0x7f766c071960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f767c006f10 tx=0x7f767c008040 comp rx=0 tx=0).stop 2024-04-25T14:17:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.850+0000 7f7690c00640 1 -- 172.21.15.86:0/2850756008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c06fd50 msgr2=0x7f768c09a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.850+0000 7f7690c00640 1 --2- 172.21.15.86:0/2850756008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c06fd50 0x7f768c09a950 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f767801a090 tx=0x7f767801aca0 comp rx=0 tx=0).stop 2024-04-25T14:17:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.850+0000 7f7690c00640 1 -- 172.21.15.86:0/2850756008 shutdown_connections 2024-04-25T14:17:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.850+0000 7f7690c00640 1 --2- 172.21.15.86:0/2850756008 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f766c06f450 0x7f766c071960 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.850+0000 7f7690c00640 1 --2- 172.21.15.86:0/2850756008 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f768c15c2a0 0x7f768c15e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.850+0000 7f7690c00640 1 --2- 172.21.15.86:0/2850756008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c06fd50 0x7f768c09a950 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.850+0000 7f7690c00640 1 --2- 172.21.15.86:0/2850756008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f768c06c670 0x7f768c1669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.850+0000 7f7690c00640 1 -- 172.21.15.86:0/2850756008 >> 172.21.15.86:0/2850756008 conn(0x7f768c0b4650 msgr2=0x7f768c15b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.850+0000 7f7690c00640 1 -- 172.21.15.86:0/2850756008 shutdown_connections 2024-04-25T14:17:34.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:34.850+0000 7f7690c00640 1 -- 172.21.15.86:0/2850756008 wait complete. 2024-04-25T14:17:35.271 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:35.271 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:36.272 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:36.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.430+0000 7f214f800640 1 Processor -- start 2024-04-25T14:17:36.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.430+0000 7f214f800640 1 -- start start 2024-04-25T14:17:36.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.434+0000 7f214f800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f21480cfca0 0x7f214806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:36.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.434+0000 7f214f800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2148072bd0 con 0x7f21480730b0 2024-04-25T14:17:36.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.434+0000 7f214f800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f214806f930 con 0x7f21480d16e0 2024-04-25T14:17:36.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.434+0000 7f214f800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f2148070470 con 0x7f21480cfca0 2024-04-25T14:17:36.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.434+0000 7f214d800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f21480730b0 0x7f21480734c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45888/0 (socket says 172.21.15.86:45888) 2024-04-25T14:17:36.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.434+0000 7f214d800640 1 -- 172.21.15.86:0/4097184433 learned_addr learned my addr 172.21.15.86:0/4097184433 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:17:36.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.434+0000 7f214ce00640 1 --2- 172.21.15.86:0/4097184433 >> v2:172.21.15.155:3300/0 conn(0x7f21480cfca0 0x7f214806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:36.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.434+0000 7f2147400640 1 -- 172.21.15.86:0/4097184433 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2691633307 0 0) 0x7f2148072bd0 con 0x7f21480730b0 2024-04-25T14:17:36.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.434+0000 7f2147400640 1 -- 172.21.15.86:0/4097184433 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f212c003530 con 0x7f21480730b0 2024-04-25T14:17:36.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.434+0000 7f2147400640 1 -- 172.21.15.86:0/4097184433 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2753096635 0 0) 0x7f214806f930 con 0x7f21480d16e0 2024-04-25T14:17:36.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.434+0000 7f2147400640 1 -- 172.21.15.86:0/4097184433 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2148072bd0 con 0x7f21480d16e0 2024-04-25T14:17:36.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.434+0000 7f214ce00640 1 -- 172.21.15.86:0/4097184433 >> v1:172.21.15.155:6789/0 conn(0x7f21480d16e0 legacy=0x7f21480d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:36.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.434+0000 7f214ce00640 1 -- 172.21.15.86:0/4097184433 >> v1:172.21.15.86:6789/0 conn(0x7f21480730b0 legacy=0x7f21480734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:36.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.434+0000 7f214ce00640 1 -- 172.21.15.86:0/4097184433 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214806fbd0 con 0x7f21480cfca0 2024-04-25T14:17:36.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.438+0000 7f214ce00640 1 --2- 172.21.15.86:0/4097184433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21480cfca0 0x7f214806f3f0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f2138009d90 tx=0x7f213801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e7adece4f8907e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:36.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.438+0000 7f2147400640 1 -- 172.21.15.86:0/4097184433 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21380b2070 con 0x7f21480cfca0 2024-04-25T14:17:36.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.438+0000 7f2147400640 1 -- 172.21.15.86:0/4097184433 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21380a5940 con 0x7f21480cfca0 2024-04-25T14:17:36.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.438+0000 7f2147400640 1 -- 172.21.15.86:0/4097184433 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21480743e0 con 0x7f21480cfca0 2024-04-25T14:17:36.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.438+0000 7f214f800640 1 -- 172.21.15.86:0/4097184433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21480cfca0 msgr2=0x7f214806f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:36.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.438+0000 7f214f800640 1 --2- 172.21.15.86:0/4097184433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21480cfca0 0x7f214806f3f0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f2138009d90 tx=0x7f213801c970 comp rx=0 tx=0).stop 2024-04-25T14:17:36.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.438+0000 7f214f800640 1 -- 172.21.15.86:0/4097184433 shutdown_connections 2024-04-25T14:17:36.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.438+0000 7f214f800640 1 --2- 172.21.15.86:0/4097184433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21480cfca0 0x7f214806f3f0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:36.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.438+0000 7f214f800640 1 -- 172.21.15.86:0/4097184433 >> 172.21.15.86:0/4097184433 conn(0x7f21480b4650 msgr2=0x7f21480b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:36.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.438+0000 7f214f800640 1 -- 172.21.15.86:0/4097184433 shutdown_connections 2024-04-25T14:17:36.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.438+0000 7f214f800640 1 -- 172.21.15.86:0/4097184433 wait complete. 2024-04-25T14:17:36.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.442+0000 7f214f800640 1 Processor -- start 2024-04-25T14:17:36.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.442+0000 7f214f800640 1 -- start start 2024-04-25T14:17:36.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.442+0000 7f214f800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21480730b0 0x7f21480cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:36.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.442+0000 7f214f800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21480cfca0 0x7f21480ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:36.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.442+0000 7f214f800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21480d16e0 0x7f21480cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:36.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.442+0000 7f214f800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f214816aa80 con 0x7f21480cfca0 2024-04-25T14:17:36.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.442+0000 7f214f800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f214816a900 con 0x7f21480d16e0 2024-04-25T14:17:36.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.442+0000 7f214f800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f214816ac00 con 0x7f21480730b0 2024-04-25T14:17:36.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.442+0000 7f2147e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21480cfca0 0x7f21480ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:36.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.442+0000 7f2147e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21480cfca0 0x7f21480ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43742/0 (socket says 172.21.15.86:43742) 2024-04-25T14:17:36.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.442+0000 7f2147e00640 1 -- 172.21.15.86:0/2267329547 learned_addr learned my addr 172.21.15.86:0/2267329547 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:36.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.442+0000 7f214d800640 1 --2- 172.21.15.86:0/2267329547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21480d16e0 0x7f21480cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:36.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.442+0000 7f214ce00640 1 --2- 172.21.15.86:0/2267329547 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21480730b0 0x7f21480cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:36.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.442+0000 7f2147e00640 1 -- 172.21.15.86:0/2267329547 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21480730b0 msgr2=0x7f21480cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:36.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.442+0000 7f2147e00640 1 --2- 172.21.15.86:0/2267329547 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21480730b0 0x7f21480cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:36.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.442+0000 7f2147e00640 1 -- 172.21.15.86:0/2267329547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21480d16e0 msgr2=0x7f21480cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:36.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.442+0000 7f2147e00640 1 --2- 172.21.15.86:0/2267329547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21480d16e0 0x7f21480cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:36.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.442+0000 7f2147e00640 1 -- 172.21.15.86:0/2267329547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21480c9be0 con 0x7f21480cfca0 2024-04-25T14:17:36.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.442+0000 7f214ce00640 1 --2- 172.21.15.86:0/2267329547 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21480730b0 0x7f21480cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:17:36.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.442+0000 7f2147e00640 1 --2- 172.21.15.86:0/2267329547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21480cfca0 0x7f21480ccca0 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f2134001ec0 tx=0x7f213400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:36.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.442+0000 7f2144c00640 1 -- 172.21.15.86:0/2267329547 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2134017070 con 0x7f21480cfca0 2024-04-25T14:17:36.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.442+0000 7f2144c00640 1 -- 172.21.15.86:0/2267329547 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21340052d0 con 0x7f21480cfca0 2024-04-25T14:17:36.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.442+0000 7f2144c00640 1 -- 172.21.15.86:0/2267329547 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21340054c0 con 0x7f21480cfca0 2024-04-25T14:17:36.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.442+0000 7f214f800640 1 -- 172.21.15.86:0/2267329547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21480c9ec0 con 0x7f21480cfca0 2024-04-25T14:17:36.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.442+0000 7f214f800640 1 -- 172.21.15.86:0/2267329547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21480ce910 con 0x7f21480cfca0 2024-04-25T14:17:36.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.446+0000 7f214f800640 1 -- 172.21.15.86:0/2267329547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f214809f4b0 con 0x7f21480cfca0 2024-04-25T14:17:36.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.446+0000 7f2144c00640 1 -- 172.21.15.86:0/2267329547 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f213401a070 con 0x7f21480cfca0 2024-04-25T14:17:36.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.446+0000 7f2144c00640 1 --2- 172.21.15.86:0/2267329547 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f212806f480 0x7f2128071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:36.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.446+0000 7f2144c00640 1 -- 172.21.15.86:0/2267329547 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f213408e7b0 con 0x7f21480cfca0 2024-04-25T14:17:36.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.446+0000 7f214ce00640 1 --2- 172.21.15.86:0/2267329547 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f212806f480 0x7f2128071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:36.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.446+0000 7f214ce00640 1 --2- 172.21.15.86:0/2267329547 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f212806f480 0x7f2128071990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f21380af000 tx=0x7f21380a7070 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:36.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.450+0000 7f2144c00640 1 -- 172.21.15.86:0/2267329547 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f213405ad60 con 0x7f21480cfca0 2024-04-25T14:17:36.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.642+0000 7f214f800640 1 -- 172.21.15.86:0/2267329547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2148069d00 con 0x7f21480cfca0 2024-04-25T14:17:36.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.642+0000 7f2144c00640 1 -- 172.21.15.86:0/2267329547 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f21340633a0 con 0x7f21480cfca0 2024-04-25T14:17:36.643 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:36.643 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:36.643 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:36.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.646+0000 7f214f800640 1 -- 172.21.15.86:0/2267329547 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f212806f480 msgr2=0x7f2128071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:36.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.646+0000 7f214f800640 1 --2- 172.21.15.86:0/2267329547 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f212806f480 0x7f2128071990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f21380af000 tx=0x7f21380a7070 comp rx=0 tx=0).stop 2024-04-25T14:17:36.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.646+0000 7f214f800640 1 -- 172.21.15.86:0/2267329547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21480cfca0 msgr2=0x7f21480ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:36.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.646+0000 7f214f800640 1 --2- 172.21.15.86:0/2267329547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21480cfca0 0x7f21480ccca0 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f2134001ec0 tx=0x7f213400c820 comp rx=0 tx=0).stop 2024-04-25T14:17:36.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.646+0000 7f214f800640 1 -- 172.21.15.86:0/2267329547 shutdown_connections 2024-04-25T14:17:36.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.646+0000 7f214f800640 1 --2- 172.21.15.86:0/2267329547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21480d16e0 0x7f21480cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:36.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.646+0000 7f214f800640 1 --2- 172.21.15.86:0/2267329547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21480cfca0 0x7f21480ccca0 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:36.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.646+0000 7f214f800640 1 --2- 172.21.15.86:0/2267329547 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f212806f480 0x7f2128071990 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:36.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.646+0000 7f214f800640 1 --2- 172.21.15.86:0/2267329547 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21480730b0 0x7f21480cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:36.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.646+0000 7f214f800640 1 -- 172.21.15.86:0/2267329547 >> 172.21.15.86:0/2267329547 conn(0x7f21480b4650 msgr2=0x7f21480af020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:36.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.646+0000 7f214f800640 1 -- 172.21.15.86:0/2267329547 shutdown_connections 2024-04-25T14:17:36.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:36.646+0000 7f214f800640 1 -- 172.21.15.86:0/2267329547 wait complete. 2024-04-25T14:17:37.068 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:37.068 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:38.069 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:38.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.230+0000 7ffbe2e00640 1 Processor -- start 2024-04-25T14:17:38.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.230+0000 7ffbe2e00640 1 -- start start 2024-04-25T14:17:38.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.230+0000 7ffbe2e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffbdc0d2c80 0x7ffbdc15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:38.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.230+0000 7ffbe2e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ffbdc070e20 0x7ffbdc071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:38.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.230+0000 7ffbe2e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffbdc0722f0 0x7ffbdc072700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:38.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.230+0000 7ffbe2e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ffbdc0ce720 con 0x7ffbdc0d2c80 2024-04-25T14:17:38.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.230+0000 7ffbe2e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ffbdc071770 con 0x7ffbdc070e20 2024-04-25T14:17:38.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.230+0000 7ffbe2e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ffbdc0cd8d0 con 0x7ffbdc0722f0 2024-04-25T14:17:38.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.230+0000 7ffbdbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffbdc0d2c80 0x7ffbdc15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:38.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.230+0000 7ffbdbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffbdc0d2c80 0x7ffbdc15dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43754/0 (socket says 172.21.15.86:43754) 2024-04-25T14:17:38.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.230+0000 7ffbdbe00640 1 -- 172.21.15.86:0/3632974864 learned_addr learned my addr 172.21.15.86:0/3632974864 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:38.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.230+0000 7ffbdbe00640 1 -- 172.21.15.86:0/3632974864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffbdc0722f0 msgr2=0x7ffbdc072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:38.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.230+0000 7ffbdbe00640 1 --2- 172.21.15.86:0/3632974864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffbdc0722f0 0x7ffbdc072700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:38.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.230+0000 7ffbdbe00640 1 -- 172.21.15.86:0/3632974864 >> v2:172.21.15.155:3300/0 conn(0x7ffbdc070e20 msgr2=0x7ffbdc071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:38.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.230+0000 7ffbdbe00640 1 --2- 172.21.15.86:0/3632974864 >> v2:172.21.15.155:3300/0 conn(0x7ffbdc070e20 0x7ffbdc071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:38.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.230+0000 7ffbdbe00640 1 -- 172.21.15.86:0/3632974864 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbdc0ce970 con 0x7ffbdc0d2c80 2024-04-25T14:17:38.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.234+0000 7ffbdbe00640 1 --2- 172.21.15.86:0/3632974864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbdc0d2c80 0x7ffbdc15dc90 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc0b6070 tx=0x7ffbcc01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a7db7c9dd9f494 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:38.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.234+0000 7ffbdb400640 1 -- 172.21.15.86:0/3632974864 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0b5070 con 0x7ffbdc0d2c80 2024-04-25T14:17:38.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.234+0000 7ffbdb400640 1 -- 172.21.15.86:0/3632974864 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbcc0a89d0 con 0x7ffbdc0d2c80 2024-04-25T14:17:38.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.234+0000 7ffbdb400640 1 -- 172.21.15.86:0/3632974864 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0a8da0 con 0x7ffbdc0d2c80 2024-04-25T14:17:38.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.234+0000 7ffbe2e00640 1 -- 172.21.15.86:0/3632974864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbdc0d2c80 msgr2=0x7ffbdc15dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:38.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.234+0000 7ffbe2e00640 1 --2- 172.21.15.86:0/3632974864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbdc0d2c80 0x7ffbdc15dc90 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc0b6070 tx=0x7ffbcc01ed10 comp rx=0 tx=0).stop 2024-04-25T14:17:38.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.234+0000 7ffbe2e00640 1 -- 172.21.15.86:0/3632974864 shutdown_connections 2024-04-25T14:17:38.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.234+0000 7ffbe2e00640 1 --2- 172.21.15.86:0/3632974864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffbdc0722f0 0x7ffbdc072700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:38.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.234+0000 7ffbe2e00640 1 --2- 172.21.15.86:0/3632974864 >> v2:172.21.15.155:3300/0 conn(0x7ffbdc070e20 0x7ffbdc071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:38.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.234+0000 7ffbe2e00640 1 --2- 172.21.15.86:0/3632974864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbdc0d2c80 0x7ffbdc15dc90 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:38.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.234+0000 7ffbe2e00640 1 -- 172.21.15.86:0/3632974864 >> 172.21.15.86:0/3632974864 conn(0x7ffbdc0cffc0 msgr2=0x7ffbdc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:38.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.234+0000 7ffbe2e00640 1 -- 172.21.15.86:0/3632974864 shutdown_connections 2024-04-25T14:17:38.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.234+0000 7ffbe2e00640 1 -- 172.21.15.86:0/3632974864 wait complete. 2024-04-25T14:17:38.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.238+0000 7ffbe2e00640 1 Processor -- start 2024-04-25T14:17:38.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.238+0000 7ffbe2e00640 1 -- start start 2024-04-25T14:17:38.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.238+0000 7ffbe2e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffbdc070e20 0x7ffbdc0b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:38.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.238+0000 7ffbe2e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbdc0722f0 0x7ffbdc1f7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:38.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.238+0000 7ffbe2e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbdc0d2c80 0x7ffbdc1542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:38.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.238+0000 7ffbe2e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc0d21a0 con 0x7ffbdc0722f0 2024-04-25T14:17:38.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.238+0000 7ffbe2e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc0d1ec0 con 0x7ffbdc0d2c80 2024-04-25T14:17:38.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.238+0000 7ffbe2e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ffbdc0d2480 con 0x7ffbdc070e20 2024-04-25T14:17:38.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.238+0000 7ffbd3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbdc0722f0 0x7ffbdc1f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:38.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.238+0000 7ffbd3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbdc0722f0 0x7ffbdc1f7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43770/0 (socket says 172.21.15.86:43770) 2024-04-25T14:17:38.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.238+0000 7ffbd3400640 1 -- 172.21.15.86:0/2947122934 learned_addr learned my addr 172.21.15.86:0/2947122934 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:38.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.238+0000 7ffbdbe00640 1 --2- 172.21.15.86:0/2947122934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffbdc070e20 0x7ffbdc0b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:38.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.238+0000 7ffbe0e00640 1 --2- 172.21.15.86:0/2947122934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbdc0d2c80 0x7ffbdc1542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:38.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.238+0000 7ffbd3400640 1 -- 172.21.15.86:0/2947122934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffbdc070e20 msgr2=0x7ffbdc0b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:38.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.238+0000 7ffbd3400640 1 --2- 172.21.15.86:0/2947122934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffbdc070e20 0x7ffbdc0b0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:38.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.238+0000 7ffbd3400640 1 -- 172.21.15.86:0/2947122934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbdc0d2c80 msgr2=0x7ffbdc1542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:38.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.238+0000 7ffbd3400640 1 --2- 172.21.15.86:0/2947122934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbdc0d2c80 0x7ffbdc1542d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:38.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.238+0000 7ffbd3400640 1 -- 172.21.15.86:0/2947122934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbdc1fa100 con 0x7ffbdc0722f0 2024-04-25T14:17:38.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.238+0000 7ffbd3400640 1 --2- 172.21.15.86:0/2947122934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbdc0722f0 0x7ffbdc1f7620 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7ffbc400b410 tx=0x7ffbc400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:38.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.238+0000 7ffbd8c00640 1 -- 172.21.15.86:0/2947122934 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4015030 con 0x7ffbdc0722f0 2024-04-25T14:17:38.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.238+0000 7ffbd8c00640 1 -- 172.21.15.86:0/2947122934 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbc4005980 con 0x7ffbdc0722f0 2024-04-25T14:17:38.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.238+0000 7ffbe2e00640 1 -- 172.21.15.86:0/2947122934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbdc0b16c0 con 0x7ffbdc0722f0 2024-04-25T14:17:38.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.238+0000 7ffbd8c00640 1 -- 172.21.15.86:0/2947122934 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4005d90 con 0x7ffbdc0722f0 2024-04-25T14:17:38.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.238+0000 7ffbe2e00640 1 -- 172.21.15.86:0/2947122934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbdc1fa680 con 0x7ffbdc0722f0 2024-04-25T14:17:38.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.242+0000 7ffbe2e00640 1 -- 172.21.15.86:0/2947122934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbdc0720e0 con 0x7ffbdc0722f0 2024-04-25T14:17:38.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.242+0000 7ffbd8c00640 1 -- 172.21.15.86:0/2947122934 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7ffbc4018070 con 0x7ffbdc0722f0 2024-04-25T14:17:38.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.242+0000 7ffbd8c00640 1 --2- 172.21.15.86:0/2947122934 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ffbb806f480 0x7ffbb8071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:38.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.242+0000 7ffbd8c00640 1 -- 172.21.15.86:0/2947122934 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7ffbc408f090 con 0x7ffbdc0722f0 2024-04-25T14:17:38.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.242+0000 7ffbdbe00640 1 --2- 172.21.15.86:0/2947122934 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ffbb806f480 0x7ffbb8071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:38.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.242+0000 7ffbdbe00640 1 --2- 172.21.15.86:0/2947122934 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ffbb806f480 0x7ffbb8071990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0b2000 tx=0x7ffbcc008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:38.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.246+0000 7ffbd8c00640 1 -- 172.21.15.86:0/2947122934 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffbc405bc90 con 0x7ffbdc0722f0 2024-04-25T14:17:38.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.438+0000 7ffbe2e00640 1 -- 172.21.15.86:0/2947122934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffbdc0b1450 con 0x7ffbdc0722f0 2024-04-25T14:17:38.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.438+0000 7ffbd8c00640 1 -- 172.21.15.86:0/2947122934 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7ffbc405b590 con 0x7ffbdc0722f0 2024-04-25T14:17:38.439 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:38.439 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:38.441 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:38.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.442+0000 7ffbe2e00640 1 -- 172.21.15.86:0/2947122934 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ffbb806f480 msgr2=0x7ffbb8071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:38.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.442+0000 7ffbe2e00640 1 --2- 172.21.15.86:0/2947122934 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ffbb806f480 0x7ffbb8071990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0b2000 tx=0x7ffbcc008040 comp rx=0 tx=0).stop 2024-04-25T14:17:38.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.442+0000 7ffbe2e00640 1 -- 172.21.15.86:0/2947122934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbdc0722f0 msgr2=0x7ffbdc1f7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:38.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.442+0000 7ffbe2e00640 1 --2- 172.21.15.86:0/2947122934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbdc0722f0 0x7ffbdc1f7620 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7ffbc400b410 tx=0x7ffbc400b8d0 comp rx=0 tx=0).stop 2024-04-25T14:17:38.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.442+0000 7ffbe2e00640 1 -- 172.21.15.86:0/2947122934 shutdown_connections 2024-04-25T14:17:38.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.442+0000 7ffbe2e00640 1 --2- 172.21.15.86:0/2947122934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbdc0d2c80 0x7ffbdc1542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:38.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.442+0000 7ffbe2e00640 1 --2- 172.21.15.86:0/2947122934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbdc0722f0 0x7ffbdc1f7620 unknown :-1 s=CLOSED pgs=2379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:38.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.442+0000 7ffbe2e00640 1 --2- 172.21.15.86:0/2947122934 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ffbb806f480 0x7ffbb8071990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:38.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.442+0000 7ffbe2e00640 1 --2- 172.21.15.86:0/2947122934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffbdc070e20 0x7ffbdc0b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:38.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.442+0000 7ffbe2e00640 1 -- 172.21.15.86:0/2947122934 >> 172.21.15.86:0/2947122934 conn(0x7ffbdc0cffc0 msgr2=0x7ffbdc15c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:38.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.442+0000 7ffbe2e00640 1 -- 172.21.15.86:0/2947122934 shutdown_connections 2024-04-25T14:17:38.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:38.442+0000 7ffbe2e00640 1 -- 172.21.15.86:0/2947122934 wait complete. 2024-04-25T14:17:38.863 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:38.863 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:39.864 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:40.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.026+0000 7f3d6d800640 1 Processor -- start 2024-04-25T14:17:40.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.026+0000 7f3d6d800640 1 -- start start 2024-04-25T14:17:40.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.026+0000 7f3d6d800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d680d1df0 0x7f3d680cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:40.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.026+0000 7f3d6d800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3d680cdcc0 0x7f3d680ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:40.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.026+0000 7f3d6d800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d6815e5c0 con 0x7f3d68070330 2024-04-25T14:17:40.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.026+0000 7f3d6d800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3d6806fd40 con 0x7f3d680cdcc0 2024-04-25T14:17:40.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.026+0000 7f3d6d800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3d6815e8a0 con 0x7f3d680d1df0 2024-04-25T14:17:40.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.026+0000 7f3d66a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3d68070330 0x7f3d680d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:46212/0 (socket says 172.21.15.86:46212) 2024-04-25T14:17:40.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.026+0000 7f3d66a00640 1 -- 172.21.15.86:0/411584517 learned_addr learned my addr 172.21.15.86:0/411584517 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:17:40.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.026+0000 7f3d67400640 1 --2- 172.21.15.86:0/411584517 >> v2:172.21.15.155:3300/0 conn(0x7f3d680cdcc0 0x7f3d680ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:40.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.026+0000 7f3d66000640 1 -- 172.21.15.86:0/411584517 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4146788544 0 0) 0x7f3d6815e5c0 con 0x7f3d68070330 2024-04-25T14:17:40.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.026+0000 7f3d66000640 1 -- 172.21.15.86:0/411584517 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d4c003530 con 0x7f3d68070330 2024-04-25T14:17:40.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.026+0000 7f3d5e000640 1 --2- 172.21.15.86:0/411584517 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d680d1df0 0x7f3d680cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:40.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.026+0000 7f3d66000640 1 -- 172.21.15.86:0/411584517 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3839255663 0 0) 0x7f3d4c003530 con 0x7f3d68070330 2024-04-25T14:17:40.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.026+0000 7f3d66000640 1 -- 172.21.15.86:0/411584517 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d680d1df0 msgr2=0x7f3d680cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:40.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.026+0000 7f3d66000640 1 --2- 172.21.15.86:0/411584517 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d680d1df0 0x7f3d680cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:40.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.026+0000 7f3d66000640 1 -- 172.21.15.86:0/411584517 >> v2:172.21.15.155:3300/0 conn(0x7f3d680cdcc0 msgr2=0x7f3d680ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:40.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.026+0000 7f3d66000640 1 --2- 172.21.15.86:0/411584517 >> v2:172.21.15.155:3300/0 conn(0x7f3d680cdcc0 0x7f3d680ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:40.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.026+0000 7f3d66000640 1 -- 172.21.15.86:0/411584517 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6815eb80 con 0x7f3d68070330 2024-04-25T14:17:40.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.026+0000 7f3d66000640 1 -- 172.21.15.86:0/411584517 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f3d58001680 con 0x7f3d68070330 2024-04-25T14:17:40.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.026+0000 7f3d66000640 1 -- 172.21.15.86:0/411584517 >> v1:172.21.15.86:6789/0 conn(0x7f3d68070330 legacy=0x7f3d680d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:40.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.030+0000 7f3d66000640 1 --2- 172.21.15.86:0/411584517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d4c003a30 0x7f3d4c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:40.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.030+0000 7f3d66000640 1 --2- 172.21.15.86:0/411584517 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d4c004810 0x7f3d4c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:40.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.030+0000 7f3d66000640 1 --2- 172.21.15.86:0/411584517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d4c005160 0x7f3d4c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:40.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.030+0000 7f3d66000640 1 -- 172.21.15.86:0/411584517 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6815e8a0 con 0x7f3d4c003a30 2024-04-25T14:17:40.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.030+0000 7f3d66000640 1 -- 172.21.15.86:0/411584517 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4c00bf10 con 0x7f3d4c005160 2024-04-25T14:17:40.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.030+0000 7f3d66000640 1 -- 172.21.15.86:0/411584517 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3d4c00c090 con 0x7f3d4c004810 2024-04-25T14:17:40.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.030+0000 7f3d66a00640 1 --2- 172.21.15.86:0/411584517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d4c003a30 0x7f3d4c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:40.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.030+0000 7f3d5e000640 1 --2- 172.21.15.86:0/411584517 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d4c004810 0x7f3d4c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:40.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.030+0000 7f3d67400640 1 --2- 172.21.15.86:0/411584517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d4c005160 0x7f3d4c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:40.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.030+0000 7f3d66a00640 1 -- 172.21.15.86:0/411584517 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d4c004810 msgr2=0x7f3d4c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:40.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.030+0000 7f3d66a00640 1 --2- 172.21.15.86:0/411584517 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d4c004810 0x7f3d4c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:40.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.030+0000 7f3d66a00640 1 -- 172.21.15.86:0/411584517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d4c005160 msgr2=0x7f3d4c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:40.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.030+0000 7f3d66a00640 1 --2- 172.21.15.86:0/411584517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d4c005160 0x7f3d4c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:40.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.030+0000 7f3d66a00640 1 -- 172.21.15.86:0/411584517 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d4c003530 con 0x7f3d4c003a30 2024-04-25T14:17:40.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.030+0000 7f3d66a00640 1 --2- 172.21.15.86:0/411584517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d4c003a30 0x7f3d4c003ea0 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800c2c0 tx=0x7f3d5801ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72815c674bda7f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:40.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.030+0000 7f3d66000640 1 -- 172.21.15.86:0/411584517 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58007d60 con 0x7f3d4c003a30 2024-04-25T14:17:40.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.030+0000 7f3d66000640 1 -- 172.21.15.86:0/411584517 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d580a9020 con 0x7f3d4c003a30 2024-04-25T14:17:40.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.030+0000 7f3d66000640 1 -- 172.21.15.86:0/411584517 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d580a5b00 con 0x7f3d4c003a30 2024-04-25T14:17:40.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.034+0000 7f3d6d800640 1 -- 172.21.15.86:0/411584517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d4c003a30 msgr2=0x7f3d4c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:40.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.034+0000 7f3d6d800640 1 --2- 172.21.15.86:0/411584517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d4c003a30 0x7f3d4c003ea0 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800c2c0 tx=0x7f3d5801ccd0 comp rx=0 tx=0).stop 2024-04-25T14:17:40.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.034+0000 7f3d67400640 1 -- 172.21.15.86:0/411584517 reap_dead start 2024-04-25T14:17:40.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.034+0000 7f3d6d800640 1 -- 172.21.15.86:0/411584517 shutdown_connections 2024-04-25T14:17:40.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.034+0000 7f3d6d800640 1 -- 172.21.15.86:0/411584517 >> 172.21.15.86:0/411584517 conn(0x7f3d680b4650 msgr2=0x7f3d680b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:40.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.034+0000 7f3d6d800640 1 -- 172.21.15.86:0/411584517 shutdown_connections 2024-04-25T14:17:40.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.034+0000 7f3d6d800640 1 -- 172.21.15.86:0/411584517 wait complete. 2024-04-25T14:17:40.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.034+0000 7f3d6d800640 1 Processor -- start 2024-04-25T14:17:40.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.034+0000 7f3d6d800640 1 -- start start 2024-04-25T14:17:40.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.038+0000 7f3d6d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d4c003a30 0x7f3d680b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:40.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.038+0000 7f3d6d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d680b0090 0x7f3d681f9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:40.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.038+0000 7f3d6d800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d681fb730 0x7f3d681fbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:40.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.038+0000 7f3d6d800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3d680708c0 con 0x7f3d4c003a30 2024-04-25T14:17:40.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.038+0000 7f3d6d800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3d68070740 con 0x7f3d680b0090 2024-04-25T14:17:40.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.038+0000 7f3d6d800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3d680d1df0 con 0x7f3d681fb730 2024-04-25T14:17:40.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.038+0000 7f3d66a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d4c003a30 0x7f3d680b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:40.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.038+0000 7f3d66a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d4c003a30 0x7f3d680b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53398/0 (socket says 172.21.15.86:53398) 2024-04-25T14:17:40.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.038+0000 7f3d66a00640 1 -- 172.21.15.86:0/4294532690 learned_addr learned my addr 172.21.15.86:0/4294532690 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:40.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.038+0000 7f3d66a00640 1 -- 172.21.15.86:0/4294532690 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d681fb730 msgr2=0x7f3d681fbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:40.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.038+0000 7f3d66a00640 1 --2- 172.21.15.86:0/4294532690 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d681fb730 0x7f3d681fbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:40.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.038+0000 7f3d66a00640 1 -- 172.21.15.86:0/4294532690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d680b0090 msgr2=0x7f3d681f9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:40.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.038+0000 7f3d66a00640 1 --2- 172.21.15.86:0/4294532690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d680b0090 0x7f3d681f9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:40.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.038+0000 7f3d66a00640 1 -- 172.21.15.86:0/4294532690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d681fa690 con 0x7f3d4c003a30 2024-04-25T14:17:40.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.038+0000 7f3d66a00640 1 --2- 172.21.15.86:0/4294532690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d4c003a30 0x7f3d680b3520 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7f3d580ab040 tx=0x7f3d4c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:40.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.038+0000 7f3d5f400640 1 -- 172.21.15.86:0/4294532690 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58001c90 con 0x7f3d4c003a30 2024-04-25T14:17:40.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.038+0000 7f3d5f400640 1 -- 172.21.15.86:0/4294532690 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d580a5b00 con 0x7f3d4c003a30 2024-04-25T14:17:40.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.038+0000 7f3d5f400640 1 -- 172.21.15.86:0/4294532690 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d580047f0 con 0x7f3d4c003a30 2024-04-25T14:17:40.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.038+0000 7f3d6d800640 1 -- 172.21.15.86:0/4294532690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d681fa970 con 0x7f3d4c003a30 2024-04-25T14:17:40.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.038+0000 7f3d6d800640 1 -- 172.21.15.86:0/4294532690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d681faf30 con 0x7f3d4c003a30 2024-04-25T14:17:40.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.038+0000 7f3d6d800640 1 -- 172.21.15.86:0/4294532690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d68069d00 con 0x7f3d4c003a30 2024-04-25T14:17:40.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.042+0000 7f3d5f400640 1 -- 172.21.15.86:0/4294532690 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f3d58004990 con 0x7f3d4c003a30 2024-04-25T14:17:40.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.042+0000 7f3d5f400640 1 --2- 172.21.15.86:0/4294532690 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f3d4406f480 0x7f3d44071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:40.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.042+0000 7f3d5f400640 1 -- 172.21.15.86:0/4294532690 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f3d58128ee0 con 0x7f3d4c003a30 2024-04-25T14:17:40.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.042+0000 7f3d5e000640 1 --2- 172.21.15.86:0/4294532690 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f3d4406f480 0x7f3d44071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:40.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.042+0000 7f3d5e000640 1 --2- 172.21.15.86:0/4294532690 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f3d4406f480 0x7f3d44071990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3d68168460 tx=0x7f3d50009340 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:40.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.046+0000 7f3d5f400640 1 -- 172.21.15.86:0/4294532690 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3d580be060 con 0x7f3d4c003a30 2024-04-25T14:17:40.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.238+0000 7f3d6d800640 1 -- 172.21.15.86:0/4294532690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d68076bf0 con 0x7f3d4c003a30 2024-04-25T14:17:40.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.238+0000 7f3d5f400640 1 -- 172.21.15.86:0/4294532690 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f3d58016a10 con 0x7f3d4c003a30 2024-04-25T14:17:40.237 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:40.237 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:40.237 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:40.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.238+0000 7f3d6d800640 1 -- 172.21.15.86:0/4294532690 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f3d4406f480 msgr2=0x7f3d44071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:40.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.238+0000 7f3d6d800640 1 --2- 172.21.15.86:0/4294532690 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f3d4406f480 0x7f3d44071990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3d68168460 tx=0x7f3d50009340 comp rx=0 tx=0).stop 2024-04-25T14:17:40.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.238+0000 7f3d6d800640 1 -- 172.21.15.86:0/4294532690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d4c003a30 msgr2=0x7f3d680b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:40.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.238+0000 7f3d6d800640 1 --2- 172.21.15.86:0/4294532690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d4c003a30 0x7f3d680b3520 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7f3d580ab040 tx=0x7f3d4c005840 comp rx=0 tx=0).stop 2024-04-25T14:17:40.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.238+0000 7f3d6d800640 1 -- 172.21.15.86:0/4294532690 shutdown_connections 2024-04-25T14:17:40.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.238+0000 7f3d6d800640 1 --2- 172.21.15.86:0/4294532690 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f3d4406f480 0x7f3d44071990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:40.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.238+0000 7f3d6d800640 1 --2- 172.21.15.86:0/4294532690 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d681fb730 0x7f3d681fbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:40.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.238+0000 7f3d6d800640 1 --2- 172.21.15.86:0/4294532690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d680b0090 0x7f3d681f9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:40.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.238+0000 7f3d6d800640 1 --2- 172.21.15.86:0/4294532690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d4c003a30 0x7f3d680b3520 unknown :-1 s=CLOSED pgs=2382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:40.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.238+0000 7f3d6d800640 1 -- 172.21.15.86:0/4294532690 >> 172.21.15.86:0/4294532690 conn(0x7f3d680b4650 msgr2=0x7f3d6815be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:40.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.238+0000 7f3d6d800640 1 -- 172.21.15.86:0/4294532690 shutdown_connections 2024-04-25T14:17:40.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:40.238+0000 7f3d6d800640 1 -- 172.21.15.86:0/4294532690 wait complete. 2024-04-25T14:17:40.661 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:40.661 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:41.662 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:41.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.822+0000 7f3b59800640 1 Processor -- start 2024-04-25T14:17:41.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.822+0000 7f3b59800640 1 -- start start 2024-04-25T14:17:41.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.822+0000 7f3b59800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3b540cdd70 0x7f3b540ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:41.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.822+0000 7f3b59800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b54070e20 0x7f3b54071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:41.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.822+0000 7f3b59800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b54071770 con 0x7f3b5406e700 2024-04-25T14:17:41.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.822+0000 7f3b59800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3b5406d940 con 0x7f3b540cdd70 2024-04-25T14:17:41.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.822+0000 7f3b59800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3b5406dc20 con 0x7f3b54070e20 2024-04-25T14:17:41.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.822+0000 7f3b53400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3b5406e700 0x7f3b5406eb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:46228/0 (socket says 172.21.15.86:46228) 2024-04-25T14:17:41.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.822+0000 7f3b53400640 1 -- 172.21.15.86:0/1518192978 learned_addr learned my addr 172.21.15.86:0/1518192978 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:17:41.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.822+0000 7f3b52a00640 1 --2- 172.21.15.86:0/1518192978 >> v2:172.21.15.155:3300/0 conn(0x7f3b540cdd70 0x7f3b540ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:41.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.822+0000 7f3b4a000640 1 --2- 172.21.15.86:0/1518192978 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b54070e20 0x7f3b54071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:41.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.822+0000 7f3b52000640 1 -- 172.21.15.86:0/1518192978 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2712158419 0 0) 0x7f3b4c0014a0 con 0x7f3b5406e700 2024-04-25T14:17:41.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.826+0000 7f3b52000640 1 -- 172.21.15.86:0/1518192978 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b38003530 con 0x7f3b5406e700 2024-04-25T14:17:41.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.826+0000 7f3b52000640 1 -- 172.21.15.86:0/1518192978 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2046011444 0 0) 0x7f3b38003530 con 0x7f3b5406e700 2024-04-25T14:17:41.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.826+0000 7f3b52000640 1 -- 172.21.15.86:0/1518192978 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b54070e20 msgr2=0x7f3b54071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:41.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.826+0000 7f3b52000640 1 --2- 172.21.15.86:0/1518192978 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b54070e20 0x7f3b54071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:41.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.826+0000 7f3b52000640 1 -- 172.21.15.86:0/1518192978 >> v2:172.21.15.155:3300/0 conn(0x7f3b540cdd70 msgr2=0x7f3b540ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:41.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.826+0000 7f3b52000640 1 --2- 172.21.15.86:0/1518192978 >> v2:172.21.15.155:3300/0 conn(0x7f3b540cdd70 0x7f3b540ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:41.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.826+0000 7f3b52000640 1 -- 172.21.15.86:0/1518192978 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b5406df00 con 0x7f3b5406e700 2024-04-25T14:17:41.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.826+0000 7f3b52000640 1 -- 172.21.15.86:0/1518192978 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f3b4c001bd0 con 0x7f3b5406e700 2024-04-25T14:17:41.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.826+0000 7f3b52000640 1 -- 172.21.15.86:0/1518192978 >> v1:172.21.15.86:6789/0 conn(0x7f3b5406e700 legacy=0x7f3b5406eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:41.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.826+0000 7f3b52000640 1 --2- 172.21.15.86:0/1518192978 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b54070e20 0x7f3b38003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:41.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.826+0000 7f3b52000640 1 --2- 172.21.15.86:0/1518192978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b38003fb0 0x7f3b38004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:41.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.826+0000 7f3b52000640 1 --2- 172.21.15.86:0/1518192978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b38004d50 0x7f3b38005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:41.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.826+0000 7f3b52000640 1 -- 172.21.15.86:0/1518192978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5406d940 con 0x7f3b38003fb0 2024-04-25T14:17:41.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.826+0000 7f3b52000640 1 -- 172.21.15.86:0/1518192978 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5406dc20 con 0x7f3b38004d50 2024-04-25T14:17:41.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.826+0000 7f3b52000640 1 -- 172.21.15.86:0/1518192978 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3b3800bb30 con 0x7f3b54070e20 2024-04-25T14:17:41.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.826+0000 7f3b4a000640 1 --2- 172.21.15.86:0/1518192978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b38003fb0 0x7f3b38004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:41.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.826+0000 7f3b52a00640 1 --2- 172.21.15.86:0/1518192978 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b54070e20 0x7f3b38003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:41.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.826+0000 7f3b53400640 1 --2- 172.21.15.86:0/1518192978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b38004d50 0x7f3b38005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:41.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.826+0000 7f3b4a000640 1 -- 172.21.15.86:0/1518192978 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b54070e20 msgr2=0x7f3b38003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:41.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.826+0000 7f3b4a000640 1 --2- 172.21.15.86:0/1518192978 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b54070e20 0x7f3b38003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:41.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.826+0000 7f3b4a000640 1 -- 172.21.15.86:0/1518192978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b38004d50 msgr2=0x7f3b38005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:41.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.826+0000 7f3b4a000640 1 --2- 172.21.15.86:0/1518192978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b38004d50 0x7f3b38005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:41.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.826+0000 7f3b4a000640 1 -- 172.21.15.86:0/1518192978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b38003530 con 0x7f3b38003fb0 2024-04-25T14:17:41.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.830+0000 7f3b4a000640 1 --2- 172.21.15.86:0/1518192978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b38003fb0 0x7f3b38004810 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c009ac0 tx=0x7f3b3c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f093d87f332e3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:41.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.830+0000 7f3b52000640 1 -- 172.21.15.86:0/1518192978 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c0ac070 con 0x7f3b38003fb0 2024-04-25T14:17:41.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.830+0000 7f3b52000640 1 -- 172.21.15.86:0/1518192978 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b3c0059a0 con 0x7f3b38003fb0 2024-04-25T14:17:41.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.830+0000 7f3b52000640 1 -- 172.21.15.86:0/1518192978 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c009690 con 0x7f3b38003fb0 2024-04-25T14:17:41.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.830+0000 7f3b59800640 1 -- 172.21.15.86:0/1518192978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b38003fb0 msgr2=0x7f3b38004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:41.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.830+0000 7f3b59800640 1 --2- 172.21.15.86:0/1518192978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b38003fb0 0x7f3b38004810 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c009ac0 tx=0x7f3b3c01ca70 comp rx=0 tx=0).stop 2024-04-25T14:17:41.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.830+0000 7f3b53400640 1 -- 172.21.15.86:0/1518192978 reap_dead start 2024-04-25T14:17:41.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.830+0000 7f3b59800640 1 -- 172.21.15.86:0/1518192978 shutdown_connections 2024-04-25T14:17:41.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.830+0000 7f3b59800640 1 -- 172.21.15.86:0/1518192978 >> 172.21.15.86:0/1518192978 conn(0x7f3b540cffc0 msgr2=0x7f3b540d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:41.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.830+0000 7f3b59800640 1 -- 172.21.15.86:0/1518192978 shutdown_connections 2024-04-25T14:17:41.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.830+0000 7f3b59800640 1 -- 172.21.15.86:0/1518192978 wait complete. 2024-04-25T14:17:41.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.830+0000 7f3b59800640 1 Processor -- start 2024-04-25T14:17:41.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.830+0000 7f3b59800640 1 -- start start 2024-04-25T14:17:41.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.834+0000 7f3b59800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b38003fb0 0x7f3b541665b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:41.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.834+0000 7f3b59800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b54158940 0x7f3b54158180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:41.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.834+0000 7f3b59800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b5415e7a0 0x7f3b5415ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:41.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.834+0000 7f3b59800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5416abf0 con 0x7f3b5415e7a0 2024-04-25T14:17:41.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.834+0000 7f3b59800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3b54164380 con 0x7f3b54158940 2024-04-25T14:17:41.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.834+0000 7f3b59800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3b5416ad70 con 0x7f3b38003fb0 2024-04-25T14:17:41.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.834+0000 7f3b53400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b5415e7a0 0x7f3b5415ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:41.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.834+0000 7f3b53400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b5415e7a0 0x7f3b5415ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53426/0 (socket says 172.21.15.86:53426) 2024-04-25T14:17:41.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.834+0000 7f3b53400640 1 -- 172.21.15.86:0/3736350479 learned_addr learned my addr 172.21.15.86:0/3736350479 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:41.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.834+0000 7f3b52a00640 1 --2- 172.21.15.86:0/3736350479 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b38003fb0 0x7f3b541665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:41.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.834+0000 7f3b53400640 1 -- 172.21.15.86:0/3736350479 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b38003fb0 msgr2=0x7f3b541665b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:41.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.834+0000 7f3b53400640 1 --2- 172.21.15.86:0/3736350479 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b38003fb0 0x7f3b541665b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:41.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.834+0000 7f3b4a000640 1 --2- 172.21.15.86:0/3736350479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b54158940 0x7f3b54158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:41.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.834+0000 7f3b53400640 1 -- 172.21.15.86:0/3736350479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b54158940 msgr2=0x7f3b54158180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:41.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.834+0000 7f3b53400640 1 --2- 172.21.15.86:0/3736350479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b54158940 0x7f3b54158180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:41.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.834+0000 7f3b53400640 1 -- 172.21.15.86:0/3736350479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b5415f0f0 con 0x7f3b5415e7a0 2024-04-25T14:17:41.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.834+0000 7f3b4a000640 1 --2- 172.21.15.86:0/3736350479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b54158940 0x7f3b54158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:17:41.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.834+0000 7f3b53400640 1 --2- 172.21.15.86:0/3736350479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b5415e7a0 0x7f3b5415ebb0 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f3b540d2e90 tx=0x7f3b4c007af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:41.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.834+0000 7f3b4b400640 1 -- 172.21.15.86:0/3736350479 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c019070 con 0x7f3b5415e7a0 2024-04-25T14:17:41.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.834+0000 7f3b4b400640 1 -- 172.21.15.86:0/3736350479 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b4c004e40 con 0x7f3b5415e7a0 2024-04-25T14:17:41.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.834+0000 7f3b59800640 1 -- 172.21.15.86:0/3736350479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b540b7e80 con 0x7f3b5415e7a0 2024-04-25T14:17:41.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.834+0000 7f3b4b400640 1 -- 172.21.15.86:0/3736350479 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c00ea10 con 0x7f3b5415e7a0 2024-04-25T14:17:41.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.834+0000 7f3b59800640 1 -- 172.21.15.86:0/3736350479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b540b8440 con 0x7f3b5415e7a0 2024-04-25T14:17:41.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.834+0000 7f3b59800640 1 -- 172.21.15.86:0/3736350479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b540720e0 con 0x7f3b5415e7a0 2024-04-25T14:17:41.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.838+0000 7f3b4b400640 1 -- 172.21.15.86:0/3736350479 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f3b4c00ebb0 con 0x7f3b5415e7a0 2024-04-25T14:17:41.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.838+0000 7f3b4b400640 1 --2- 172.21.15.86:0/3736350479 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f3b3006f480 0x7f3b30071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:41.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.838+0000 7f3b4b400640 1 -- 172.21.15.86:0/3736350479 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f3b4c08ef90 con 0x7f3b5415e7a0 2024-04-25T14:17:41.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.838+0000 7f3b52a00640 1 --2- 172.21.15.86:0/3736350479 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f3b3006f480 0x7f3b30071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:41.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.838+0000 7f3b52a00640 1 --2- 172.21.15.86:0/3736350479 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f3b3006f480 0x7f3b30071990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3b44009ba0 tx=0x7f3b44008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:41.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:41.842+0000 7f3b4b400640 1 -- 172.21.15.86:0/3736350479 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3b5415f0f0 con 0x7f3b5415e7a0 2024-04-25T14:17:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:42.034+0000 7f3b59800640 1 -- 172.21.15.86:0/3736350479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b54076bf0 con 0x7f3b5415e7a0 2024-04-25T14:17:42.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:42.034+0000 7f3b4b400640 1 -- 172.21.15.86:0/3736350479 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f3b54071770 con 0x7f3b5415e7a0 2024-04-25T14:17:42.034 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:42.034 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:42.034 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:42.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:42.038+0000 7f3b59800640 1 -- 172.21.15.86:0/3736350479 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f3b3006f480 msgr2=0x7f3b30071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:42.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:42.038+0000 7f3b59800640 1 --2- 172.21.15.86:0/3736350479 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f3b3006f480 0x7f3b30071990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3b44009ba0 tx=0x7f3b44008040 comp rx=0 tx=0).stop 2024-04-25T14:17:42.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:42.038+0000 7f3b59800640 1 -- 172.21.15.86:0/3736350479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b5415e7a0 msgr2=0x7f3b5415ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:42.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:42.038+0000 7f3b59800640 1 --2- 172.21.15.86:0/3736350479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b5415e7a0 0x7f3b5415ebb0 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f3b540d2e90 tx=0x7f3b4c007af0 comp rx=0 tx=0).stop 2024-04-25T14:17:42.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:42.038+0000 7f3b59800640 1 -- 172.21.15.86:0/3736350479 shutdown_connections 2024-04-25T14:17:42.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:42.038+0000 7f3b59800640 1 --2- 172.21.15.86:0/3736350479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b5415e7a0 0x7f3b5415ebb0 unknown :-1 s=CLOSED pgs=2385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:42.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:42.038+0000 7f3b59800640 1 --2- 172.21.15.86:0/3736350479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b54158940 0x7f3b54158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:42.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:42.038+0000 7f3b59800640 1 --2- 172.21.15.86:0/3736350479 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f3b3006f480 0x7f3b30071990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:42.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:42.038+0000 7f3b59800640 1 --2- 172.21.15.86:0/3736350479 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b38003fb0 0x7f3b541665b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:42.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:42.038+0000 7f3b59800640 1 -- 172.21.15.86:0/3736350479 >> 172.21.15.86:0/3736350479 conn(0x7f3b540cffc0 msgr2=0x7f3b5415ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:42.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:42.038+0000 7f3b59800640 1 -- 172.21.15.86:0/3736350479 shutdown_connections 2024-04-25T14:17:42.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:42.038+0000 7f3b59800640 1 -- 172.21.15.86:0/3736350479 wait complete. 2024-04-25T14:17:42.460 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:42.460 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:43.460 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.618+0000 7f7b2f600640 1 Processor -- start 2024-04-25T14:17:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b2f600640 1 -- start start 2024-04-25T14:17:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b2f600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7b28070330 0x7f7b280d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b2f600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b280d2030 con 0x7f7b280d1c20 2024-04-25T14:17:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b2f600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b2815e5c0 con 0x7f7b280ce790 2024-04-25T14:17:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b2f600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f7b2815e8a0 con 0x7f7b28070330 2024-04-25T14:17:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b27e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7b280d1c20 0x7f7b2806ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:46242/0 (socket says 172.21.15.86:46242) 2024-04-25T14:17:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b27e00640 1 -- 172.21.15.86:0/1000137562 learned_addr learned my addr 172.21.15.86:0/1000137562 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:17:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b2cc00640 1 --2- 172.21.15.86:0/1000137562 >> v2:172.21.15.155:3300/0 conn(0x7f7b28070330 0x7f7b280d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b27400640 1 -- 172.21.15.86:0/1000137562 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2145084228 0 0) 0x7f7b14001120 con 0x7f7b280d1c20 2024-04-25T14:17:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b27400640 1 -- 172.21.15.86:0/1000137562 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b10003530 con 0x7f7b280d1c20 2024-04-25T14:17:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b27400640 1 -- 172.21.15.86:0/1000137562 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3625995705 0 0) 0x7f7b2815e5c0 con 0x7f7b280ce790 2024-04-25T14:17:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b27400640 1 -- 172.21.15.86:0/1000137562 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b14001120 con 0x7f7b280ce790 2024-04-25T14:17:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b27400640 1 -- 172.21.15.86:0/1000137562 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 489392254 0 0) 0x7f7b10003530 con 0x7f7b280d1c20 2024-04-25T14:17:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b27400640 1 -- 172.21.15.86:0/1000137562 >> v2:172.21.15.155:3300/0 conn(0x7f7b28070330 msgr2=0x7f7b280d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b27400640 1 --2- 172.21.15.86:0/1000137562 >> v2:172.21.15.155:3300/0 conn(0x7f7b28070330 0x7f7b280d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b27400640 1 -- 172.21.15.86:0/1000137562 >> v1:172.21.15.155:6789/0 conn(0x7f7b280ce790 legacy=0x7f7b2806f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b27400640 1 -- 172.21.15.86:0/1000137562 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b2815eb80 con 0x7f7b280d1c20 2024-04-25T14:17:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b27400640 1 -- 172.21.15.86:0/1000137562 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f7b14001850 con 0x7f7b280d1c20 2024-04-25T14:17:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b2cc00640 1 --2- 172.21.15.86:0/1000137562 >> v2:172.21.15.155:3300/0 conn(0x7f7b28070330 0x7f7b280d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:17:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b27400640 1 -- 172.21.15.86:0/1000137562 >> v1:172.21.15.86:6789/0 conn(0x7f7b280d1c20 legacy=0x7f7b2806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b27400640 1 --2- 172.21.15.86:0/1000137562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b10003bc0 0x7f7b10004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b27400640 1 --2- 172.21.15.86:0/1000137562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b10004810 0x7f7b10004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b27400640 1 --2- 172.21.15.86:0/1000137562 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b10005330 0x7f7b1000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b27400640 1 -- 172.21.15.86:0/1000137562 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2815e8a0 con 0x7f7b10004810 2024-04-25T14:17:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b27400640 1 -- 172.21.15.86:0/1000137562 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1000c260 con 0x7f7b10003bc0 2024-04-25T14:17:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b27400640 1 -- 172.21.15.86:0/1000137562 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7b1000c460 con 0x7f7b10005330 2024-04-25T14:17:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b27e00640 1 --2- 172.21.15.86:0/1000137562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b10004810 0x7f7b10004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b2cc00640 1 --2- 172.21.15.86:0/1000137562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b10003bc0 0x7f7b10004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b2d600640 1 --2- 172.21.15.86:0/1000137562 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b10005330 0x7f7b1000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b27e00640 1 -- 172.21.15.86:0/1000137562 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b10005330 msgr2=0x7f7b1000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b27e00640 1 --2- 172.21.15.86:0/1000137562 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b10005330 0x7f7b1000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b27e00640 1 -- 172.21.15.86:0/1000137562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b10003bc0 msgr2=0x7f7b10004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b27e00640 1 --2- 172.21.15.86:0/1000137562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b10003bc0 0x7f7b10004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b27e00640 1 -- 172.21.15.86:0/1000137562 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b10003530 con 0x7f7b10004810 2024-04-25T14:17:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.622+0000 7f7b2d600640 1 -- 172.21.15.86:0/1000137562 reap_dead start 2024-04-25T14:17:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.626+0000 7f7b27e00640 1 --2- 172.21.15.86:0/1000137562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b10004810 0x7f7b10004c20 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7f7b28074ed0 tx=0x7f7b1401cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188f06184976a76d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:43.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.626+0000 7f7b27400640 1 -- 172.21.15.86:0/1000137562 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1400c0e0 con 0x7f7b10004810 2024-04-25T14:17:43.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.626+0000 7f7b27400640 1 -- 172.21.15.86:0/1000137562 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b140a9020 con 0x7f7b10004810 2024-04-25T14:17:43.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.626+0000 7f7b27400640 1 -- 172.21.15.86:0/1000137562 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b14005b00 con 0x7f7b10004810 2024-04-25T14:17:43.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.626+0000 7f7b2f600640 1 -- 172.21.15.86:0/1000137562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b10004810 msgr2=0x7f7b10004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.626+0000 7f7b2f600640 1 --2- 172.21.15.86:0/1000137562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b10004810 0x7f7b10004c20 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7f7b28074ed0 tx=0x7f7b1401cb80 comp rx=0 tx=0).stop 2024-04-25T14:17:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.626+0000 7f7b2f600640 1 -- 172.21.15.86:0/1000137562 shutdown_connections 2024-04-25T14:17:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.626+0000 7f7b2f600640 1 --2- 172.21.15.86:0/1000137562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b10004810 0x7f7b10004c20 unknown :-1 s=CLOSED pgs=2387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.626+0000 7f7b2f600640 1 -- 172.21.15.86:0/1000137562 >> 172.21.15.86:0/1000137562 conn(0x7f7b280b4650 msgr2=0x7f7b280b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.626+0000 7f7b2f600640 1 -- 172.21.15.86:0/1000137562 shutdown_connections 2024-04-25T14:17:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.626+0000 7f7b2f600640 1 -- 172.21.15.86:0/1000137562 wait complete. 2024-04-25T14:17:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.630+0000 7f7b2f600640 1 Processor -- start 2024-04-25T14:17:43.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.630+0000 7f7b2f600640 1 -- start start 2024-04-25T14:17:43.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.630+0000 7f7b2f600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b10004810 0x7f7b281f74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:43.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.630+0000 7f7b2f600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b28159390 0x7f7b2809c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:43.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.630+0000 7f7b2f600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b2809ba20 0x7f7b2809be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:43.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.630+0000 7f7b2f600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7b280708c0 con 0x7f7b28159390 2024-04-25T14:17:43.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.630+0000 7f7b2f600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7b28070740 con 0x7f7b2809ba20 2024-04-25T14:17:43.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.630+0000 7f7b2f600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7b280cdf90 con 0x7f7b10004810 2024-04-25T14:17:43.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.630+0000 7f7b27e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b28159390 0x7f7b2809c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:43.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.630+0000 7f7b27e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b28159390 0x7f7b2809c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53442/0 (socket says 172.21.15.86:53442) 2024-04-25T14:17:43.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.630+0000 7f7b27e00640 1 -- 172.21.15.86:0/420671677 learned_addr learned my addr 172.21.15.86:0/420671677 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:43.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.634+0000 7f7b27e00640 1 -- 172.21.15.86:0/420671677 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b10004810 msgr2=0x7f7b281f74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:43.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.634+0000 7f7b27e00640 1 --2- 172.21.15.86:0/420671677 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b10004810 0x7f7b281f74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:43.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.634+0000 7f7b27e00640 1 -- 172.21.15.86:0/420671677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b2809ba20 msgr2=0x7f7b2809be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:43.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.634+0000 7f7b2d600640 1 --2- 172.21.15.86:0/420671677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b2809ba20 0x7f7b2809be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:43.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.634+0000 7f7b27e00640 1 --2- 172.21.15.86:0/420671677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b2809ba20 0x7f7b2809be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:43.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.634+0000 7f7b27e00640 1 -- 172.21.15.86:0/420671677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b2809e9e0 con 0x7f7b28159390 2024-04-25T14:17:43.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.634+0000 7f7b2d600640 1 --2- 172.21.15.86:0/420671677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b2809ba20 0x7f7b2809be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:17:43.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.634+0000 7f7b27e00640 1 --2- 172.21.15.86:0/420671677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b28159390 0x7f7b2809c850 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f7b140ab040 tx=0x7f7b10005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:43.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.634+0000 7f7b24c00640 1 -- 172.21.15.86:0/420671677 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b140b4070 con 0x7f7b28159390 2024-04-25T14:17:43.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.634+0000 7f7b24c00640 1 -- 172.21.15.86:0/420671677 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b14005b40 con 0x7f7b28159390 2024-04-25T14:17:43.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.634+0000 7f7b24c00640 1 -- 172.21.15.86:0/420671677 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b140a5660 con 0x7f7b28159390 2024-04-25T14:17:43.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.634+0000 7f7b2f600640 1 -- 172.21.15.86:0/420671677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b2809ecc0 con 0x7f7b28159390 2024-04-25T14:17:43.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.634+0000 7f7b2f600640 1 -- 172.21.15.86:0/420671677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b281597a0 con 0x7f7b28159390 2024-04-25T14:17:43.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.634+0000 7f7b2f600640 1 -- 172.21.15.86:0/420671677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b2809e750 con 0x7f7b28159390 2024-04-25T14:17:43.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.634+0000 7f7b24c00640 1 -- 172.21.15.86:0/420671677 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f7b140bf060 con 0x7f7b28159390 2024-04-25T14:17:43.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.638+0000 7f7b24c00640 1 --2- 172.21.15.86:0/420671677 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7b0806f450 0x7f7b08071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:43.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.638+0000 7f7b24c00640 1 -- 172.21.15.86:0/420671677 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f7b14128710 con 0x7f7b28159390 2024-04-25T14:17:43.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.638+0000 7f7b2cc00640 1 --2- 172.21.15.86:0/420671677 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7b0806f450 0x7f7b08071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:43.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.638+0000 7f7b2cc00640 1 --2- 172.21.15.86:0/420671677 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7b0806f450 0x7f7b08071960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7b280b3f10 tx=0x7f7b1c0112c0 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:43.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.642+0000 7f7b24c00640 1 -- 172.21.15.86:0/420671677 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7b280d2030 con 0x7f7b28159390 2024-04-25T14:17:43.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.830+0000 7f7b2f600640 1 -- 172.21.15.86:0/420671677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b28069d00 con 0x7f7b28159390 2024-04-25T14:17:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.830+0000 7f7b24c00640 1 -- 172.21.15.86:0/420671677 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f7b140f5420 con 0x7f7b28159390 2024-04-25T14:17:43.828 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:43.828 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.830+0000 7f7b2f600640 1 -- 172.21.15.86:0/420671677 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7b0806f450 msgr2=0x7f7b08071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.830+0000 7f7b2f600640 1 --2- 172.21.15.86:0/420671677 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7b0806f450 0x7f7b08071960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7b280b3f10 tx=0x7f7b1c0112c0 comp rx=0 tx=0).stop 2024-04-25T14:17:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.830+0000 7f7b2f600640 1 -- 172.21.15.86:0/420671677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b28159390 msgr2=0x7f7b2809c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.830+0000 7f7b2f600640 1 --2- 172.21.15.86:0/420671677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b28159390 0x7f7b2809c850 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f7b140ab040 tx=0x7f7b10005840 comp rx=0 tx=0).stop 2024-04-25T14:17:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.830+0000 7f7b2f600640 1 -- 172.21.15.86:0/420671677 shutdown_connections 2024-04-25T14:17:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.830+0000 7f7b2f600640 1 --2- 172.21.15.86:0/420671677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b2809ba20 0x7f7b2809be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.830+0000 7f7b2f600640 1 --2- 172.21.15.86:0/420671677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b28159390 0x7f7b2809c850 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.830+0000 7f7b2f600640 1 --2- 172.21.15.86:0/420671677 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f7b0806f450 0x7f7b08071960 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.830+0000 7f7b2f600640 1 --2- 172.21.15.86:0/420671677 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b10004810 0x7f7b281f74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.830+0000 7f7b2f600640 1 -- 172.21.15.86:0/420671677 >> 172.21.15.86:0/420671677 conn(0x7f7b280b4650 msgr2=0x7f7b2815b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.830+0000 7f7b2f600640 1 -- 172.21.15.86:0/420671677 shutdown_connections 2024-04-25T14:17:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:43.830+0000 7f7b2f600640 1 -- 172.21.15.86:0/420671677 wait complete. 2024-04-25T14:17:44.253 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:44.254 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:45.254 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:45.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.414+0000 7f5867800640 1 Processor -- start 2024-04-25T14:17:45.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.414+0000 7f5867800640 1 -- start start 2024-04-25T14:17:45.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.418+0000 7f5867800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f58600740c0 0x7f58601676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:45.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.418+0000 7f5867800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5860072bd0 con 0x7f58600d16e0 2024-04-25T14:17:45.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.418+0000 7f5867800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f586006fbd0 con 0x7f58600cfca0 2024-04-25T14:17:45.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.418+0000 7f5867800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5860070470 con 0x7f58600740c0 2024-04-25T14:17:45.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.418+0000 7f5865800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f58600740c0 0x7f58601676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:45.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.418+0000 7f5865800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f58600740c0 0x7f58601676d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53448/0 (socket says 172.21.15.86:53448) 2024-04-25T14:17:45.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.418+0000 7f5865800640 1 -- 172.21.15.86:0/2868467736 learned_addr learned my addr 172.21.15.86:0/2868467736 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:45.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.418+0000 7f5857e00640 1 --1- 172.21.15.86:0/2868467736 >> v1:172.21.15.86:6789/0 conn(0x7f58600d16e0 0x7f58600d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:46256/0 (socket says 172.21.15.86:46256) 2024-04-25T14:17:45.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.418+0000 7f5865800640 1 -- 172.21.15.86:0/2868467736 >> v1:172.21.15.155:6789/0 conn(0x7f58600cfca0 legacy=0x7f586006f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:45.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.418+0000 7f5865800640 1 -- 172.21.15.86:0/2868467736 >> v1:172.21.15.86:6789/0 conn(0x7f58600d16e0 legacy=0x7f58600d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:45.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.418+0000 7f5865800640 1 -- 172.21.15.86:0/2868467736 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586006feb0 con 0x7f58600740c0 2024-04-25T14:17:45.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.418+0000 7f5865800640 1 --2- 172.21.15.86:0/2868467736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58600740c0 0x7f58601676d0 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f5858009e50 tx=0x7f585801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d8d0fbb19f6ae8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:45.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.418+0000 7f585fe00640 1 -- 172.21.15.86:0/2868467736 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58580a5810 con 0x7f58600740c0 2024-04-25T14:17:45.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.422+0000 7f585fe00640 1 -- 172.21.15.86:0/2868467736 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58580a59b0 con 0x7f58600740c0 2024-04-25T14:17:45.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.422+0000 7f585fe00640 1 -- 172.21.15.86:0/2868467736 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58580a5d80 con 0x7f58600740c0 2024-04-25T14:17:45.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.422+0000 7f5867800640 1 -- 172.21.15.86:0/2868467736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58600740c0 msgr2=0x7f58601676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:45.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.422+0000 7f5867800640 1 --2- 172.21.15.86:0/2868467736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58600740c0 0x7f58601676d0 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f5858009e50 tx=0x7f585801ca30 comp rx=0 tx=0).stop 2024-04-25T14:17:45.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.422+0000 7f5867800640 1 -- 172.21.15.86:0/2868467736 shutdown_connections 2024-04-25T14:17:45.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.422+0000 7f5867800640 1 --2- 172.21.15.86:0/2868467736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58600740c0 0x7f58601676d0 unknown :-1 s=CLOSED pgs=2390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:45.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.422+0000 7f5867800640 1 -- 172.21.15.86:0/2868467736 >> 172.21.15.86:0/2868467736 conn(0x7f58600b4650 msgr2=0x7f58600b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:45.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.422+0000 7f5867800640 1 -- 172.21.15.86:0/2868467736 shutdown_connections 2024-04-25T14:17:45.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.422+0000 7f5867800640 1 -- 172.21.15.86:0/2868467736 wait complete. 2024-04-25T14:17:45.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.422+0000 7f5867800640 1 Processor -- start 2024-04-25T14:17:45.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.422+0000 7f5867800640 1 -- start start 2024-04-25T14:17:45.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.426+0000 7f5867800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58600740c0 0x7f58601f9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:45.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.426+0000 7f5867800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58600cfca0 0x7f586015bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:45.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.426+0000 7f5867800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58600d16e0 0x7f586015c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:45.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.426+0000 7f5867800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f586016ade0 con 0x7f58600740c0 2024-04-25T14:17:45.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.426+0000 7f5867800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f586016ac60 con 0x7f58600d16e0 2024-04-25T14:17:45.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.426+0000 7f5867800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f586016af60 con 0x7f58600cfca0 2024-04-25T14:17:45.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.426+0000 7f5864e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58600740c0 0x7f58601f9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:45.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.426+0000 7f5864e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58600740c0 0x7f58601f9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53454/0 (socket says 172.21.15.86:53454) 2024-04-25T14:17:45.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.426+0000 7f5864e00640 1 -- 172.21.15.86:0/2861963447 learned_addr learned my addr 172.21.15.86:0/2861963447 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:45.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.426+0000 7f5857e00640 1 --2- 172.21.15.86:0/2861963447 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58600cfca0 0x7f586015bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:45.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.426+0000 7f5865800640 1 --2- 172.21.15.86:0/2861963447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58600d16e0 0x7f586015c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:45.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.426+0000 7f5864e00640 1 -- 172.21.15.86:0/2861963447 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58600cfca0 msgr2=0x7f586015bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:45.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.426+0000 7f5864e00640 1 --2- 172.21.15.86:0/2861963447 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58600cfca0 0x7f586015bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:45.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.426+0000 7f5864e00640 1 -- 172.21.15.86:0/2861963447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58600d16e0 msgr2=0x7f586015c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:45.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.426+0000 7f5864e00640 1 --2- 172.21.15.86:0/2861963447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58600d16e0 0x7f586015c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:45.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.426+0000 7f5864e00640 1 -- 172.21.15.86:0/2861963447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58601f9a70 con 0x7f58600740c0 2024-04-25T14:17:45.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.426+0000 7f5865800640 1 --2- 172.21.15.86:0/2861963447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58600d16e0 0x7f586015c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:17:45.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.426+0000 7f5864e00640 1 --2- 172.21.15.86:0/2861963447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58600740c0 0x7f58601f9530 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f585000c370 tx=0x7f585000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:45.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.426+0000 7f585d600640 1 -- 172.21.15.86:0/2861963447 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58500052d0 con 0x7f58600740c0 2024-04-25T14:17:45.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.426+0000 7f585d600640 1 -- 172.21.15.86:0/2861963447 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5850005470 con 0x7f58600740c0 2024-04-25T14:17:45.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.426+0000 7f585d600640 1 -- 172.21.15.86:0/2861963447 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5850007260 con 0x7f58600740c0 2024-04-25T14:17:45.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.426+0000 7f5867800640 1 -- 172.21.15.86:0/2861963447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58600ca4c0 con 0x7f58600740c0 2024-04-25T14:17:45.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.426+0000 7f5867800640 1 -- 172.21.15.86:0/2861963447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58600caa80 con 0x7f58600740c0 2024-04-25T14:17:45.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.426+0000 7f5867800640 1 -- 172.21.15.86:0/2861963447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f586009f4b0 con 0x7f58600740c0 2024-04-25T14:17:45.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.430+0000 7f585d600640 1 -- 172.21.15.86:0/2861963447 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f5850007400 con 0x7f58600740c0 2024-04-25T14:17:45.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.430+0000 7f585d600640 1 --2- 172.21.15.86:0/2861963447 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f583c06f480 0x7f583c071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:45.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.430+0000 7f585d600640 1 -- 172.21.15.86:0/2861963447 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f585008ea60 con 0x7f58600740c0 2024-04-25T14:17:45.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.430+0000 7f5857e00640 1 --2- 172.21.15.86:0/2861963447 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f583c06f480 0x7f583c071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:45.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.430+0000 7f5857e00640 1 --2- 172.21.15.86:0/2861963447 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f583c06f480 0x7f583c071990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f58600c9d00 tx=0x7f5848007c30 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:45.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.434+0000 7f585d600640 1 -- 172.21.15.86:0/2861963447 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f585005b660 con 0x7f58600740c0 2024-04-25T14:17:45.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.622+0000 7f5867800640 1 -- 172.21.15.86:0/2861963447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5860069d00 con 0x7f58600740c0 2024-04-25T14:17:45.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.622+0000 7f585d600640 1 -- 172.21.15.86:0/2861963447 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f585005af60 con 0x7f58600740c0 2024-04-25T14:17:45.622 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:45.622 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:45.622 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:45.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.626+0000 7f5867800640 1 -- 172.21.15.86:0/2861963447 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f583c06f480 msgr2=0x7f583c071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:45.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.626+0000 7f5867800640 1 --2- 172.21.15.86:0/2861963447 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f583c06f480 0x7f583c071990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f58600c9d00 tx=0x7f5848007c30 comp rx=0 tx=0).stop 2024-04-25T14:17:45.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.626+0000 7f5867800640 1 -- 172.21.15.86:0/2861963447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58600740c0 msgr2=0x7f58601f9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:45.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.626+0000 7f5867800640 1 --2- 172.21.15.86:0/2861963447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58600740c0 0x7f58601f9530 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f585000c370 tx=0x7f585000c830 comp rx=0 tx=0).stop 2024-04-25T14:17:45.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.626+0000 7f5867800640 1 -- 172.21.15.86:0/2861963447 shutdown_connections 2024-04-25T14:17:45.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.626+0000 7f5867800640 1 --2- 172.21.15.86:0/2861963447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58600d16e0 0x7f586015c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:45.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.626+0000 7f5867800640 1 --2- 172.21.15.86:0/2861963447 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f583c06f480 0x7f583c071990 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:45.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.626+0000 7f5867800640 1 --2- 172.21.15.86:0/2861963447 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58600cfca0 0x7f586015bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:45.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.626+0000 7f5867800640 1 --2- 172.21.15.86:0/2861963447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58600740c0 0x7f58601f9530 unknown :-1 s=CLOSED pgs=2391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:45.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.626+0000 7f5867800640 1 -- 172.21.15.86:0/2861963447 >> 172.21.15.86:0/2861963447 conn(0x7f58600b4650 msgr2=0x7f58600b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:45.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.626+0000 7f5867800640 1 -- 172.21.15.86:0/2861963447 shutdown_connections 2024-04-25T14:17:45.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:45.626+0000 7f5867800640 1 -- 172.21.15.86:0/2861963447 wait complete. 2024-04-25T14:17:46.046 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:46.046 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:47.047 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:47.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.206+0000 7f79ede00640 1 Processor -- start 2024-04-25T14:17:47.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.206+0000 7f79ede00640 1 -- start start 2024-04-25T14:17:47.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.210+0000 7f79ede00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f79e806fd50 0x7f79e8070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:47.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.210+0000 7f79ede00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f79e815feb0 0x7f79e81602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:47.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.210+0000 7f79ede00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79e80b0a60 con 0x7f79e815c2a0 2024-04-25T14:17:47.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.210+0000 7f79ede00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f79e80ae840 con 0x7f79e806fd50 2024-04-25T14:17:47.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.210+0000 7f79ede00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f79e806ef90 con 0x7f79e815feb0 2024-04-25T14:17:47.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.210+0000 7f79de000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f79e806fd50 0x7f79e8070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:47.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.210+0000 7f79de000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f79e806fd50 0x7f79e8070160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53470/0 (socket says 172.21.15.86:53470) 2024-04-25T14:17:47.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.210+0000 7f79de000640 1 -- 172.21.15.86:0/2358406273 learned_addr learned my addr 172.21.15.86:0/2358406273 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:47.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.210+0000 7f79e7400640 1 --2- 172.21.15.86:0/2358406273 >> v2:172.21.15.155:3300/0 conn(0x7f79e815feb0 0x7f79e81602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:47.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.210+0000 7f79de000640 1 -- 172.21.15.86:0/2358406273 >> v2:172.21.15.155:3300/0 conn(0x7f79e815feb0 msgr2=0x7f79e81602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:47.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.210+0000 7f79de000640 1 --2- 172.21.15.86:0/2358406273 >> v2:172.21.15.155:3300/0 conn(0x7f79e815feb0 0x7f79e81602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:47.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.210+0000 7f79de000640 1 -- 172.21.15.86:0/2358406273 >> v1:172.21.15.86:6789/0 conn(0x7f79e815c2a0 legacy=0x7f79e80ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:47.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.210+0000 7f79de000640 1 -- 172.21.15.86:0/2358406273 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e806f270 con 0x7f79e806fd50 2024-04-25T14:17:47.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.210+0000 7f79de000640 1 --2- 172.21.15.86:0/2358406273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e806fd50 0x7f79e8070160 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7f79e80b0940 tx=0x7f79d001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5319977a1e24046 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:47.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.210+0000 7f79e6000640 1 -- 172.21.15.86:0/2358406273 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d00b1070 con 0x7f79e806fd50 2024-04-25T14:17:47.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.214+0000 7f79e6000640 1 -- 172.21.15.86:0/2358406273 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79d00a8050 con 0x7f79e806fd50 2024-04-25T14:17:47.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.214+0000 7f79e6000640 1 -- 172.21.15.86:0/2358406273 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d00ac860 con 0x7f79e806fd50 2024-04-25T14:17:47.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.214+0000 7f79ede00640 1 -- 172.21.15.86:0/2358406273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e806fd50 msgr2=0x7f79e8070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:47.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.214+0000 7f79ede00640 1 --2- 172.21.15.86:0/2358406273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e806fd50 0x7f79e8070160 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7f79e80b0940 tx=0x7f79d001c460 comp rx=0 tx=0).stop 2024-04-25T14:17:47.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.214+0000 7f79ede00640 1 -- 172.21.15.86:0/2358406273 shutdown_connections 2024-04-25T14:17:47.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.214+0000 7f79ede00640 1 --2- 172.21.15.86:0/2358406273 >> v2:172.21.15.155:3300/0 conn(0x7f79e815feb0 0x7f79e81602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:47.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.214+0000 7f79ede00640 1 --2- 172.21.15.86:0/2358406273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e806fd50 0x7f79e8070160 unknown :-1 s=CLOSED pgs=2393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:47.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.214+0000 7f79ede00640 1 -- 172.21.15.86:0/2358406273 >> 172.21.15.86:0/2358406273 conn(0x7f79e80b4650 msgr2=0x7f79e80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:47.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.214+0000 7f79ede00640 1 -- 172.21.15.86:0/2358406273 shutdown_connections 2024-04-25T14:17:47.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.214+0000 7f79ede00640 1 -- 172.21.15.86:0/2358406273 wait complete. 2024-04-25T14:17:47.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.214+0000 7f79ede00640 1 Processor -- start 2024-04-25T14:17:47.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.214+0000 7f79ede00640 1 -- start start 2024-04-25T14:17:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.218+0000 7f79ede00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e806fd50 0x7f79e81fb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.218+0000 7f79ede00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79e815c2a0 0x7f79e81fbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.218+0000 7f79ede00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79e815feb0 0x7f79e80b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.218+0000 7f79ede00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79e815c6b0 con 0x7f79e806fd50 2024-04-25T14:17:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.218+0000 7f79ede00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f79e80d0fc0 con 0x7f79e815feb0 2024-04-25T14:17:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.218+0000 7f79ede00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f79e815c830 con 0x7f79e815c2a0 2024-04-25T14:17:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.218+0000 7f79e6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e806fd50 0x7f79e81fb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.218+0000 7f79e6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e806fd50 0x7f79e81fb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53484/0 (socket says 172.21.15.86:53484) 2024-04-25T14:17:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.218+0000 7f79e6a00640 1 -- 172.21.15.86:0/3194844145 learned_addr learned my addr 172.21.15.86:0/3194844145 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.218+0000 7f79e7400640 1 --2- 172.21.15.86:0/3194844145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79e815feb0 0x7f79e80b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.218+0000 7f79de000640 1 --2- 172.21.15.86:0/3194844145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79e815c2a0 0x7f79e81fbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.218+0000 7f79e6a00640 1 -- 172.21.15.86:0/3194844145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79e815c2a0 msgr2=0x7f79e81fbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.218+0000 7f79e6a00640 1 --2- 172.21.15.86:0/3194844145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79e815c2a0 0x7f79e81fbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:47.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.218+0000 7f79e6a00640 1 -- 172.21.15.86:0/3194844145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79e815feb0 msgr2=0x7f79e80b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:47.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.218+0000 7f79e6a00640 1 --2- 172.21.15.86:0/3194844145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79e815feb0 0x7f79e80b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:47.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.218+0000 7f79e6a00640 1 -- 172.21.15.86:0/3194844145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e8155f10 con 0x7f79e806fd50 2024-04-25T14:17:47.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.218+0000 7f79e6a00640 1 --2- 172.21.15.86:0/3194844145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e806fd50 0x7f79e81fb890 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f79d800c310 tx=0x7f79d800c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:47.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.218+0000 7f79dfe00640 1 -- 172.21.15.86:0/3194844145 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d80052d0 con 0x7f79e806fd50 2024-04-25T14:17:47.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.218+0000 7f79dfe00640 1 -- 172.21.15.86:0/3194844145 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79d8005470 con 0x7f79e806fd50 2024-04-25T14:17:47.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.218+0000 7f79dfe00640 1 -- 172.21.15.86:0/3194844145 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d8007260 con 0x7f79e806fd50 2024-04-25T14:17:47.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.218+0000 7f79ede00640 1 -- 172.21.15.86:0/3194844145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79e81fc4f0 con 0x7f79e806fd50 2024-04-25T14:17:47.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.218+0000 7f79ede00640 1 -- 172.21.15.86:0/3194844145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79e80b3a70 con 0x7f79e806fd50 2024-04-25T14:17:47.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.218+0000 7f79ede00640 1 -- 172.21.15.86:0/3194844145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79e8069d00 con 0x7f79e806fd50 2024-04-25T14:17:47.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.222+0000 7f79dfe00640 1 -- 172.21.15.86:0/3194844145 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f79d8007400 con 0x7f79e806fd50 2024-04-25T14:17:47.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.222+0000 7f79dfe00640 1 --2- 172.21.15.86:0/3194844145 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f79c406f450 0x7f79c4071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:47.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.222+0000 7f79dfe00640 1 -- 172.21.15.86:0/3194844145 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f79d808eaf0 con 0x7f79e806fd50 2024-04-25T14:17:47.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.222+0000 7f79de000640 1 --2- 172.21.15.86:0/3194844145 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f79c406f450 0x7f79c4071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:47.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.222+0000 7f79de000640 1 --2- 172.21.15.86:0/3194844145 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f79c406f450 0x7f79c4071960 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f79d00ae000 tx=0x7f79d0008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:47.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.226+0000 7f79dfe00640 1 -- 172.21.15.86:0/3194844145 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f79d805b660 con 0x7f79e806fd50 2024-04-25T14:17:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.414+0000 7f79ede00640 1 -- 172.21.15.86:0/3194844145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79e80c47f0 con 0x7f79e806fd50 2024-04-25T14:17:47.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.414+0000 7f79dfe00640 1 -- 172.21.15.86:0/3194844145 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f79d805af60 con 0x7f79e806fd50 2024-04-25T14:17:47.412 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:47.412 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:47.413 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:47.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.414+0000 7f79ede00640 1 -- 172.21.15.86:0/3194844145 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f79c406f450 msgr2=0x7f79c4071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:47.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.414+0000 7f79ede00640 1 --2- 172.21.15.86:0/3194844145 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f79c406f450 0x7f79c4071960 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f79d00ae000 tx=0x7f79d0008040 comp rx=0 tx=0).stop 2024-04-25T14:17:47.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.414+0000 7f79ede00640 1 -- 172.21.15.86:0/3194844145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e806fd50 msgr2=0x7f79e81fb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:47.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.414+0000 7f79ede00640 1 --2- 172.21.15.86:0/3194844145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e806fd50 0x7f79e81fb890 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f79d800c310 tx=0x7f79d800c7d0 comp rx=0 tx=0).stop 2024-04-25T14:17:47.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.414+0000 7f79ede00640 1 -- 172.21.15.86:0/3194844145 shutdown_connections 2024-04-25T14:17:47.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.414+0000 7f79ede00640 1 --2- 172.21.15.86:0/3194844145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79e815feb0 0x7f79e80b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:47.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.414+0000 7f79ede00640 1 --2- 172.21.15.86:0/3194844145 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f79c406f450 0x7f79c4071960 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:47.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.414+0000 7f79ede00640 1 --2- 172.21.15.86:0/3194844145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79e815c2a0 0x7f79e81fbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:47.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.414+0000 7f79ede00640 1 --2- 172.21.15.86:0/3194844145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e806fd50 0x7f79e81fb890 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:47.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.414+0000 7f79ede00640 1 -- 172.21.15.86:0/3194844145 >> 172.21.15.86:0/3194844145 conn(0x7f79e80b4650 msgr2=0x7f79e815ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:47.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.414+0000 7f79ede00640 1 -- 172.21.15.86:0/3194844145 shutdown_connections 2024-04-25T14:17:47.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:47.414+0000 7f79ede00640 1 -- 172.21.15.86:0/3194844145 wait complete. 2024-04-25T14:17:47.836 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:47.836 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:48.837 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:48.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:48.998+0000 7f1d8c800640 1 Processor -- start 2024-04-25T14:17:48.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:48.998+0000 7f1d8c800640 1 -- start start 2024-04-25T14:17:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d8c800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1d840d16e0 0x7f1d840d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d8c800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d840d2030 con 0x7f1d840cfca0 2024-04-25T14:17:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d8c800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d84072bd0 con 0x7f1d840730b0 2024-04-25T14:17:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d8c800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1d84070470 con 0x7f1d840d16e0 2024-04-25T14:17:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d89e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1d840cfca0 0x7f1d8406f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59108/0 (socket says 172.21.15.86:59108) 2024-04-25T14:17:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d89e00640 1 -- 172.21.15.86:0/3822239784 learned_addr learned my addr 172.21.15.86:0/3822239784 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:17:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d89400640 1 --2- 172.21.15.86:0/3822239784 >> v2:172.21.15.155:3300/0 conn(0x7f1d840d16e0 0x7f1d840d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d88a00640 1 -- 172.21.15.86:0/3822239784 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1591990586 0 0) 0x7f1d78001120 con 0x7f1d840cfca0 2024-04-25T14:17:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d88a00640 1 -- 172.21.15.86:0/3822239784 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d6c003530 con 0x7f1d840cfca0 2024-04-25T14:17:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d88a00640 1 -- 172.21.15.86:0/3822239784 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2286932829 0 0) 0x7f1d84072bd0 con 0x7f1d840730b0 2024-04-25T14:17:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d88a00640 1 -- 172.21.15.86:0/3822239784 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d78001120 con 0x7f1d840730b0 2024-04-25T14:17:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d88a00640 1 -- 172.21.15.86:0/3822239784 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3437203881 0 0) 0x7f1d6c003530 con 0x7f1d840cfca0 2024-04-25T14:17:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d88a00640 1 -- 172.21.15.86:0/3822239784 >> v2:172.21.15.155:3300/0 conn(0x7f1d840d16e0 msgr2=0x7f1d840d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d88a00640 1 --2- 172.21.15.86:0/3822239784 >> v2:172.21.15.155:3300/0 conn(0x7f1d840d16e0 0x7f1d840d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d88a00640 1 -- 172.21.15.86:0/3822239784 >> v1:172.21.15.155:6789/0 conn(0x7f1d840730b0 legacy=0x7f1d840734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d88a00640 1 -- 172.21.15.86:0/3822239784 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d8406fbd0 con 0x7f1d840cfca0 2024-04-25T14:17:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d88a00640 1 -- 172.21.15.86:0/3822239784 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f1d78001870 con 0x7f1d840cfca0 2024-04-25T14:17:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d88a00640 1 -- 172.21.15.86:0/3822239784 >> v1:172.21.15.86:6789/0 conn(0x7f1d840cfca0 legacy=0x7f1d8406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d88a00640 1 --2- 172.21.15.86:0/3822239784 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d6c003bc0 0x7f1d6c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d88a00640 1 --2- 172.21.15.86:0/3822239784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d6c004810 0x7f1d6c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d88a00640 1 --2- 172.21.15.86:0/3822239784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d6c005220 0x7f1d6c00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d88a00640 1 -- 172.21.15.86:0/3822239784 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1d84070470 con 0x7f1d6c005220 2024-04-25T14:17:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d88a00640 1 -- 172.21.15.86:0/3822239784 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c00e3d0 con 0x7f1d6c004810 2024-04-25T14:17:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d88a00640 1 -- 172.21.15.86:0/3822239784 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1d6c00e5d0 con 0x7f1d6c003bc0 2024-04-25T14:17:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d8a800640 1 --2- 172.21.15.86:0/3822239784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d6c005220 0x7f1d6c00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d89e00640 1 --2- 172.21.15.86:0/3822239784 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d6c003bc0 0x7f1d6c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d89400640 1 --2- 172.21.15.86:0/3822239784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d6c004810 0x7f1d6c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d8a800640 1 -- 172.21.15.86:0/3822239784 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d6c003bc0 msgr2=0x7f1d6c004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d8a800640 1 --2- 172.21.15.86:0/3822239784 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d6c003bc0 0x7f1d6c004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d8a800640 1 -- 172.21.15.86:0/3822239784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d6c004810 msgr2=0x7f1d6c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d8a800640 1 --2- 172.21.15.86:0/3822239784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d6c004810 0x7f1d6c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.002+0000 7f1d8a800640 1 -- 172.21.15.86:0/3822239784 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6c003530 con 0x7f1d6c005220 2024-04-25T14:17:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.006+0000 7f1d8a800640 1 -- 172.21.15.86:0/3822239784 reap_dead start 2024-04-25T14:17:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.006+0000 7f1d8a800640 1 --2- 172.21.15.86:0/3822239784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d6c005220 0x7f1d6c00dbe0 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f1d840cb8e0 tx=0x7f1d8001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842a2b4dc85b8003 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:49.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.006+0000 7f1d88a00640 1 -- 172.21.15.86:0/3822239784 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d80001c20 con 0x7f1d6c005220 2024-04-25T14:17:49.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.006+0000 7f1d88a00640 1 -- 172.21.15.86:0/3822239784 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d80003ba0 con 0x7f1d6c005220 2024-04-25T14:17:49.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.006+0000 7f1d88a00640 1 -- 172.21.15.86:0/3822239784 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d800050d0 con 0x7f1d6c005220 2024-04-25T14:17:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.006+0000 7f1d8c800640 1 -- 172.21.15.86:0/3822239784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d6c005220 msgr2=0x7f1d6c00dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.006+0000 7f1d8c800640 1 --2- 172.21.15.86:0/3822239784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d6c005220 0x7f1d6c00dbe0 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f1d840cb8e0 tx=0x7f1d8001c8e0 comp rx=0 tx=0).stop 2024-04-25T14:17:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.006+0000 7f1d8c800640 1 -- 172.21.15.86:0/3822239784 shutdown_connections 2024-04-25T14:17:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.006+0000 7f1d8c800640 1 --2- 172.21.15.86:0/3822239784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d6c005220 0x7f1d6c00dbe0 unknown :-1 s=CLOSED pgs=2396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.006+0000 7f1d8c800640 1 -- 172.21.15.86:0/3822239784 >> 172.21.15.86:0/3822239784 conn(0x7f1d840b4650 msgr2=0x7f1d840b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.006+0000 7f1d8c800640 1 -- 172.21.15.86:0/3822239784 shutdown_connections 2024-04-25T14:17:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.006+0000 7f1d8c800640 1 -- 172.21.15.86:0/3822239784 wait complete. 2024-04-25T14:17:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.010+0000 7f1d8c800640 1 Processor -- start 2024-04-25T14:17:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.010+0000 7f1d8c800640 1 -- start start 2024-04-25T14:17:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.010+0000 7f1d8c800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d6c005220 0x7f1d8409da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.010+0000 7f1d8c800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d840afdf0 0x7f1d8415f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.010+0000 7f1d8c800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d8415f800 0x7f1d841f9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.010+0000 7f1d8c800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1d840d1af0 con 0x7f1d6c005220 2024-04-25T14:17:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.010+0000 7f1d8c800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1d84070190 con 0x7f1d840afdf0 2024-04-25T14:17:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.010+0000 7f1d8c800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1d840d1c70 con 0x7f1d8415f800 2024-04-25T14:17:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.010+0000 7f1d89e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d6c005220 0x7f1d8409da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.010+0000 7f1d89e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d6c005220 0x7f1d8409da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58928/0 (socket says 172.21.15.86:58928) 2024-04-25T14:17:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.010+0000 7f1d89e00640 1 -- 172.21.15.86:0/920691408 learned_addr learned my addr 172.21.15.86:0/920691408 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:49.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.010+0000 7f1d89e00640 1 -- 172.21.15.86:0/920691408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d8415f800 msgr2=0x7f1d841f9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:49.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.010+0000 7f1d89e00640 1 --2- 172.21.15.86:0/920691408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d8415f800 0x7f1d841f9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:49.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.010+0000 7f1d89e00640 1 -- 172.21.15.86:0/920691408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d840afdf0 msgr2=0x7f1d8415f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:49.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.010+0000 7f1d89e00640 1 --2- 172.21.15.86:0/920691408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d840afdf0 0x7f1d8415f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:49.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.010+0000 7f1d89e00640 1 -- 172.21.15.86:0/920691408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d8415fc10 con 0x7f1d6c005220 2024-04-25T14:17:49.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.010+0000 7f1d89e00640 1 --2- 172.21.15.86:0/920691408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d6c005220 0x7f1d8409da80 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f1d78005850 tx=0x7f1d780126f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:49.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.010+0000 7f1d7e000640 1 -- 172.21.15.86:0/920691408 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7801c070 con 0x7f1d6c005220 2024-04-25T14:17:49.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.010+0000 7f1d7e000640 1 -- 172.21.15.86:0/920691408 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d78014770 con 0x7f1d6c005220 2024-04-25T14:17:49.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.010+0000 7f1d7e000640 1 -- 172.21.15.86:0/920691408 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d78014b60 con 0x7f1d6c005220 2024-04-25T14:17:49.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.010+0000 7f1d8c800640 1 -- 172.21.15.86:0/920691408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d84151c80 con 0x7f1d6c005220 2024-04-25T14:17:49.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.010+0000 7f1d8c800640 1 -- 172.21.15.86:0/920691408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d84152240 con 0x7f1d6c005220 2024-04-25T14:17:49.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.014+0000 7f1d8c800640 1 -- 172.21.15.86:0/920691408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d8409f4b0 con 0x7f1d6c005220 2024-04-25T14:17:49.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.014+0000 7f1d7e000640 1 -- 172.21.15.86:0/920691408 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f1d78014d00 con 0x7f1d6c005220 2024-04-25T14:17:49.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.014+0000 7f1d7e000640 1 --2- 172.21.15.86:0/920691408 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1d6806f480 0x7f1d68071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:49.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.014+0000 7f1d7e000640 1 -- 172.21.15.86:0/920691408 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f1d78091f30 con 0x7f1d6c005220 2024-04-25T14:17:49.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.014+0000 7f1d89400640 1 --2- 172.21.15.86:0/920691408 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1d6806f480 0x7f1d68071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:49.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.014+0000 7f1d89400640 1 --2- 172.21.15.86:0/920691408 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1d6806f480 0x7f1d68071990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1d841f95d0 tx=0x7f1d74009340 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:49.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.018+0000 7f1d7e000640 1 -- 172.21.15.86:0/920691408 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1d8415fc10 con 0x7f1d6c005220 2024-04-25T14:17:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.206+0000 7f1d8c800640 1 -- 172.21.15.86:0/920691408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d84069d00 con 0x7f1d6c005220 2024-04-25T14:17:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.206+0000 7f1d7e000640 1 -- 172.21.15.86:0/920691408 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f1d840d2030 con 0x7f1d6c005220 2024-04-25T14:17:49.206 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:49.206 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:49.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.210+0000 7f1d8c800640 1 -- 172.21.15.86:0/920691408 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1d6806f480 msgr2=0x7f1d68071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:49.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.210+0000 7f1d8c800640 1 --2- 172.21.15.86:0/920691408 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1d6806f480 0x7f1d68071990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1d841f95d0 tx=0x7f1d74009340 comp rx=0 tx=0).stop 2024-04-25T14:17:49.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.210+0000 7f1d8c800640 1 -- 172.21.15.86:0/920691408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d6c005220 msgr2=0x7f1d8409da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:49.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.210+0000 7f1d8c800640 1 --2- 172.21.15.86:0/920691408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d6c005220 0x7f1d8409da80 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f1d78005850 tx=0x7f1d780126f0 comp rx=0 tx=0).stop 2024-04-25T14:17:49.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.210+0000 7f1d8c800640 1 -- 172.21.15.86:0/920691408 shutdown_connections 2024-04-25T14:17:49.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.210+0000 7f1d8c800640 1 --2- 172.21.15.86:0/920691408 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f1d6806f480 0x7f1d68071990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:49.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.210+0000 7f1d8c800640 1 --2- 172.21.15.86:0/920691408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d8415f800 0x7f1d841f9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:49.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.210+0000 7f1d8c800640 1 --2- 172.21.15.86:0/920691408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d840afdf0 0x7f1d8415f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:49.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.210+0000 7f1d8c800640 1 --2- 172.21.15.86:0/920691408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d6c005220 0x7f1d8409da80 unknown :-1 s=CLOSED pgs=2397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:49.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.210+0000 7f1d8c800640 1 -- 172.21.15.86:0/920691408 >> 172.21.15.86:0/920691408 conn(0x7f1d840b4650 msgr2=0x7f1d840ae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:49.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.210+0000 7f1d8c800640 1 -- 172.21.15.86:0/920691408 shutdown_connections 2024-04-25T14:17:49.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:49.210+0000 7f1d8c800640 1 -- 172.21.15.86:0/920691408 wait complete. 2024-04-25T14:17:49.630 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:49.630 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:50.631 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:50.778 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:17:50.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.782+0000 7f03fb800640 1 Processor -- start 2024-04-25T14:17:50.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.782+0000 7f03fb800640 1 -- start start 2024-04-25T14:17:50.782 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:17:50.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.782+0000 7f03fb800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f03f40cfca0 0x7f03f406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:50.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.782+0000 7f03fb800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f03f40d16e0 0x7f03f40d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:50.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.782+0000 7f03fb800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03f406c670 0x7f03f406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:50.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.782+0000 7f03fb800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f03f4070470 con 0x7f03f40cfca0 2024-04-25T14:17:50.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.782+0000 7f03fb800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f03f40d2030 con 0x7f03f40d16e0 2024-04-25T14:17:50.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.782+0000 7f03fb800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f03f4072bd0 con 0x7f03f406c670 2024-04-25T14:17:50.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.782+0000 7f03f8e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f03f40cfca0 0x7f03f406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:50.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.782+0000 7f03f8e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f03f40cfca0 0x7f03f406f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58936/0 (socket says 172.21.15.86:58936) 2024-04-25T14:17:50.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.782+0000 7f03f8e00640 1 -- 172.21.15.86:0/1137790791 learned_addr learned my addr 172.21.15.86:0/1137790791 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:50.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.782+0000 7f03f9800640 1 --2- 172.21.15.86:0/1137790791 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03f406c670 0x7f03f406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:50.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.782+0000 7f03ebe00640 1 --2- 172.21.15.86:0/1137790791 >> v2:172.21.15.155:3300/0 conn(0x7f03f40d16e0 0x7f03f40d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:50.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.782+0000 7f03f8e00640 1 -- 172.21.15.86:0/1137790791 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03f406c670 msgr2=0x7f03f406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:50.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.782+0000 7f03f8e00640 1 --2- 172.21.15.86:0/1137790791 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03f406c670 0x7f03f406ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:50.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.782+0000 7f03f8e00640 1 -- 172.21.15.86:0/1137790791 >> v2:172.21.15.155:3300/0 conn(0x7f03f40d16e0 msgr2=0x7f03f40d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:50.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.782+0000 7f03f8e00640 1 --2- 172.21.15.86:0/1137790791 >> v2:172.21.15.155:3300/0 conn(0x7f03f40d16e0 0x7f03f40d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:50.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.782+0000 7f03f8e00640 1 -- 172.21.15.86:0/1137790791 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f406f930 con 0x7f03f40cfca0 2024-04-25T14:17:50.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.786+0000 7f03f8e00640 1 --2- 172.21.15.86:0/1137790791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03f40cfca0 0x7f03f406f3f0 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7f03f40729c0 tx=0x7f03dc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85877b4aed755ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:50.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.786+0000 7f03f3e00640 1 -- 172.21.15.86:0/1137790791 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc0ab5f0 con 0x7f03f40cfca0 2024-04-25T14:17:50.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.786+0000 7f03f3e00640 1 -- 172.21.15.86:0/1137790791 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03dc0b0070 con 0x7f03f40cfca0 2024-04-25T14:17:50.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.786+0000 7f03f3e00640 1 -- 172.21.15.86:0/1137790791 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc0ab9b0 con 0x7f03f40cfca0 2024-04-25T14:17:50.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.786+0000 7f03fb800640 1 -- 172.21.15.86:0/1137790791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03f40cfca0 msgr2=0x7f03f406f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:50.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.786+0000 7f03fb800640 1 --2- 172.21.15.86:0/1137790791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03f40cfca0 0x7f03f406f3f0 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7f03f40729c0 tx=0x7f03dc01c520 comp rx=0 tx=0).stop 2024-04-25T14:17:50.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.786+0000 7f03fb800640 1 -- 172.21.15.86:0/1137790791 shutdown_connections 2024-04-25T14:17:50.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.786+0000 7f03fb800640 1 --2- 172.21.15.86:0/1137790791 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03f406c670 0x7f03f406ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:50.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.786+0000 7f03fb800640 1 --2- 172.21.15.86:0/1137790791 >> v2:172.21.15.155:3300/0 conn(0x7f03f40d16e0 0x7f03f40d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:50.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.786+0000 7f03fb800640 1 --2- 172.21.15.86:0/1137790791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03f40cfca0 0x7f03f406f3f0 unknown :-1 s=CLOSED pgs=2398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:50.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.786+0000 7f03fb800640 1 -- 172.21.15.86:0/1137790791 >> 172.21.15.86:0/1137790791 conn(0x7f03f40b4650 msgr2=0x7f03f40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:50.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.786+0000 7f03fb800640 1 -- 172.21.15.86:0/1137790791 shutdown_connections 2024-04-25T14:17:50.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.786+0000 7f03fb800640 1 -- 172.21.15.86:0/1137790791 wait complete. 2024-04-25T14:17:50.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.790+0000 7f03fb800640 1 Processor -- start 2024-04-25T14:17:50.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.790+0000 7f03fb800640 1 -- start start 2024-04-25T14:17:50.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.790+0000 7f03fb800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03f406c670 0x7f03f40c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:50.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.790+0000 7f03fb800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03f40cfca0 0x7f03f40cd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:50.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.790+0000 7f03fb800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03f40d16e0 0x7f03f40cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:50.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.790+0000 7f03fb800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03f40738c0 con 0x7f03f406c670 2024-04-25T14:17:50.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.790+0000 7f03fb800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f03f406d3d0 con 0x7f03f40cfca0 2024-04-25T14:17:50.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.790+0000 7f03fb800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f03f4073a40 con 0x7f03f40d16e0 2024-04-25T14:17:50.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.790+0000 7f03f8e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03f406c670 0x7f03f40c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:50.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.790+0000 7f03f8e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03f406c670 0x7f03f40c9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58942/0 (socket says 172.21.15.86:58942) 2024-04-25T14:17:50.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.790+0000 7f03f8e00640 1 -- 172.21.15.86:0/1482761823 learned_addr learned my addr 172.21.15.86:0/1482761823 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:50.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.794+0000 7f03ebe00640 1 --2- 172.21.15.86:0/1482761823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03f40cfca0 0x7f03f40cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:50.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.794+0000 7f03f9800640 1 --2- 172.21.15.86:0/1482761823 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03f40d16e0 0x7f03f40cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:50.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.794+0000 7f03f8e00640 1 -- 172.21.15.86:0/1482761823 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03f40d16e0 msgr2=0x7f03f40cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:50.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.794+0000 7f03f8e00640 1 --2- 172.21.15.86:0/1482761823 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03f40d16e0 0x7f03f40cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:50.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.794+0000 7f03f8e00640 1 -- 172.21.15.86:0/1482761823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03f40cfca0 msgr2=0x7f03f40cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:50.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.794+0000 7f03f8e00640 1 --2- 172.21.15.86:0/1482761823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03f40cfca0 0x7f03f40cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:50.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.794+0000 7f03f8e00640 1 -- 172.21.15.86:0/1482761823 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f4068980 con 0x7f03f406c670 2024-04-25T14:17:50.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.794+0000 7f03f9800640 1 --2- 172.21.15.86:0/1482761823 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03f40d16e0 0x7f03f40cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:17:50.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.794+0000 7f03f8e00640 1 --2- 172.21.15.86:0/1482761823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03f406c670 0x7f03f40c9290 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7f03dc01ca00 tx=0x7f03dc01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:50.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.794+0000 7f03f1600640 1 -- 172.21.15.86:0/1482761823 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc0be3f0 con 0x7f03f406c670 2024-04-25T14:17:50.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.794+0000 7f03f1600640 1 -- 172.21.15.86:0/1482761823 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03dc0be590 con 0x7f03f406c670 2024-04-25T14:17:50.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.794+0000 7f03fb800640 1 -- 172.21.15.86:0/1482761823 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f40c9d50 con 0x7f03f406c670 2024-04-25T14:17:50.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.794+0000 7f03f1600640 1 -- 172.21.15.86:0/1482761823 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc0be980 con 0x7f03f406c670 2024-04-25T14:17:50.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.794+0000 7f03fb800640 1 -- 172.21.15.86:0/1482761823 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f40ca310 con 0x7f03f406c670 2024-04-25T14:17:50.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.794+0000 7f03fb800640 1 -- 172.21.15.86:0/1482761823 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03f409f4b0 con 0x7f03f406c670 2024-04-25T14:17:50.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.798+0000 7f03f1600640 1 -- 172.21.15.86:0/1482761823 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f03dc0b43f0 con 0x7f03f406c670 2024-04-25T14:17:50.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.798+0000 7f03f1600640 1 --2- 172.21.15.86:0/1482761823 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f03d006f450 0x7f03d0071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:50.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.798+0000 7f03f1600640 1 -- 172.21.15.86:0/1482761823 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f03dc1295e0 con 0x7f03f406c670 2024-04-25T14:17:50.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.798+0000 7f03ebe00640 1 --2- 172.21.15.86:0/1482761823 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f03d006f450 0x7f03d0071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:50.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.798+0000 7f03ebe00640 1 --2- 172.21.15.86:0/1482761823 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f03d006f450 0x7f03d0071960 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f03f4068f30 tx=0x7f03e4009190 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:50.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.802+0000 7f03f1600640 1 -- 172.21.15.86:0/1482761823 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f03dc0c4060 con 0x7f03f406c670 2024-04-25T14:17:50.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.994+0000 7f03fb800640 1 -- 172.21.15.86:0/1482761823 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03f4069d00 con 0x7f03f406c670 2024-04-25T14:17:50.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.994+0000 7f03f1600640 1 -- 172.21.15.86:0/1482761823 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f03dc016110 con 0x7f03f406c670 2024-04-25T14:17:50.994 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:50.994 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:50.995 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:50.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.998+0000 7f03fb800640 1 -- 172.21.15.86:0/1482761823 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f03d006f450 msgr2=0x7f03d0071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:50.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.998+0000 7f03fb800640 1 --2- 172.21.15.86:0/1482761823 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f03d006f450 0x7f03d0071960 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f03f4068f30 tx=0x7f03e4009190 comp rx=0 tx=0).stop 2024-04-25T14:17:50.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.998+0000 7f03fb800640 1 -- 172.21.15.86:0/1482761823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03f406c670 msgr2=0x7f03f40c9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:50.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.998+0000 7f03fb800640 1 --2- 172.21.15.86:0/1482761823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03f406c670 0x7f03f40c9290 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7f03dc01ca00 tx=0x7f03dc01ad60 comp rx=0 tx=0).stop 2024-04-25T14:17:50.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.998+0000 7f03fb800640 1 -- 172.21.15.86:0/1482761823 shutdown_connections 2024-04-25T14:17:50.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.998+0000 7f03fb800640 1 --2- 172.21.15.86:0/1482761823 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f03d006f450 0x7f03d0071960 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:50.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.998+0000 7f03fb800640 1 --2- 172.21.15.86:0/1482761823 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03f40d16e0 0x7f03f40cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:50.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.998+0000 7f03fb800640 1 --2- 172.21.15.86:0/1482761823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03f40cfca0 0x7f03f40cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:50.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.998+0000 7f03fb800640 1 --2- 172.21.15.86:0/1482761823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03f406c670 0x7f03f40c9290 unknown :-1 s=CLOSED pgs=2399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:50.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.998+0000 7f03fb800640 1 -- 172.21.15.86:0/1482761823 >> 172.21.15.86:0/1482761823 conn(0x7f03f40b4650 msgr2=0x7f03f40cbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:50.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.998+0000 7f03fb800640 1 -- 172.21.15.86:0/1482761823 shutdown_connections 2024-04-25T14:17:50.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:50.998+0000 7f03fb800640 1 -- 172.21.15.86:0/1482761823 wait complete. 2024-04-25T14:17:51.418 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T14:13:33.612462+0000', 'modified': '2024-04-25T14:16:19.366674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0, 1], 'up': {'mds_0': 10209, 'mds_1': 8387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8387': {'gid': 8387, 'name': 'b', 'rank': 1, 'incarnation': 119, 'state': 'up:stopping', 'state_seq': 9, 'addr': '172.21.15.155:6835/4231378267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 4231378267}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 4231378267}]}, 'join_fscid': -1, 'export_targets': [0], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10209': {'gid': 10209, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3141015835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 3141015835}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3141015835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10209, 'qdb_cluster': [10209]} 2024-04-25T14:17:51.419 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8387:up:stopping 2024-04-25T14:17:52.420 DEBUG:tasks.cephfs.filesystem:status = { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 11, "epoch": 125, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 11, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-25T14:13:33.612462+0000", "damaged": [], "data_pools": [ 24 ], "enabled": true, "epoch": 125, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_10209": { "addr": "172.21.15.86:6837/3141015835", "addrs": { "addrvec": [ { "addr": "172.21.15.86:6836", "nonce": 3141015835, "type": "v2" }, { "addr": "172.21.15.86:6837", "nonce": 3141015835, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 10209, "incarnation": 113, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 3 }, "gid_8387": { "addr": "172.21.15.155:6835/4231378267", "addrs": { "addrvec": [ { "addr": "172.21.15.155:6834", "nonce": 4231378267, "type": "v2" }, { "addr": "172.21.15.155:6835", "nonce": 4231378267, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0 ], "features": 4540701547738038271, "flags": 0, "gid": 8387, "incarnation": 119, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:stopping", "state_seq": 9 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 23, "modified": "2024-04-25T14:16:19.366674+0000", "qdb_cluster": [ 10209 ], "qdb_leader": 10209, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 10209, "mds_1": 8387 } } } ], "standbys": [ { "addr": "172.21.15.86:6835/3872031627", "addrs": { "addrvec": [ { "addr": "172.21.15.86:6834", "nonce": 3872031627, "type": "v2" }, { "addr": "172.21.15.86:6835", "nonce": 3872031627, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 111, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8378, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.155:6833/3308745791", "addrs": { "addrvec": [ { "addr": "172.21.15.155:6832", "nonce": 3308745791, "type": "v2" }, { "addr": "172.21.15.155:6833", "nonce": 3308745791, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 112, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8381, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-25T14:17:52.420 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable stats 2024-04-25T14:17:52.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.582+0000 7f71d2000640 1 Processor -- start 2024-04-25T14:17:52.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.582+0000 7f71d2000640 1 -- start start 2024-04-25T14:17:52.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.582+0000 7f71d2000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f71cc15c2a0 0x7f71cc0ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:52.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.582+0000 7f71d2000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f71cc15feb0 0x7f71cc1602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:52.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.582+0000 7f71d2000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71cc0b0a60 con 0x7f71cc06fd50 2024-04-25T14:17:52.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.582+0000 7f71d2000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f71cc06ef90 con 0x7f71cc15feb0 2024-04-25T14:17:52.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.582+0000 7f71d2000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f71cc06f270 con 0x7f71cc15c2a0 2024-04-25T14:17:52.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.586+0000 7f71cbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f71cc15feb0 0x7f71cc1602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:52.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.586+0000 7f71cbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f71cc15feb0 0x7f71cc1602c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58952/0 (socket says 172.21.15.86:58952) 2024-04-25T14:17:52.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.586+0000 7f71cbe00640 1 -- 172.21.15.86:0/1830921584 learned_addr learned my addr 172.21.15.86:0/1830921584 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:52.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.586+0000 7f71cb400640 1 --2- 172.21.15.86:0/1830921584 >> v2:172.21.15.155:3300/0 conn(0x7f71cc15c2a0 0x7f71cc0ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:52.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.586+0000 7f71cbe00640 1 -- 172.21.15.86:0/1830921584 >> v2:172.21.15.155:3300/0 conn(0x7f71cc15c2a0 msgr2=0x7f71cc0ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:52.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.586+0000 7f71cbe00640 1 --2- 172.21.15.86:0/1830921584 >> v2:172.21.15.155:3300/0 conn(0x7f71cc15c2a0 0x7f71cc0ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:52.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.586+0000 7f71cbe00640 1 -- 172.21.15.86:0/1830921584 >> v1:172.21.15.155:6789/0 conn(0x7f71cc06fd50 legacy=0x7f71cc070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:52.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.586+0000 7f71cbe00640 1 -- 172.21.15.86:0/1830921584 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71cc06f550 con 0x7f71cc15feb0 2024-04-25T14:17:52.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.586+0000 7f71cb400640 1 --2- 172.21.15.86:0/1830921584 >> v2:172.21.15.155:3300/0 conn(0x7f71cc15c2a0 0x7f71cc0ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:17:52.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.586+0000 7f71cbe00640 1 --2- 172.21.15.86:0/1830921584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71cc15feb0 0x7f71cc1602c0 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7f71cc072200 tx=0x7f71c401d350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c36acb862321e7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:52.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.586+0000 7f71ca000640 1 -- 172.21.15.86:0/1830921584 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c40a5750 con 0x7f71cc15feb0 2024-04-25T14:17:52.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.590+0000 7f71ca000640 1 -- 172.21.15.86:0/1830921584 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71c40a58f0 con 0x7f71cc15feb0 2024-04-25T14:17:52.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.590+0000 7f71ca000640 1 -- 172.21.15.86:0/1830921584 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c40a5cc0 con 0x7f71cc15feb0 2024-04-25T14:17:52.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.590+0000 7f71d2000640 1 -- 172.21.15.86:0/1830921584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71cc15feb0 msgr2=0x7f71cc1602c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:52.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.590+0000 7f71d2000640 1 --2- 172.21.15.86:0/1830921584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71cc15feb0 0x7f71cc1602c0 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7f71cc072200 tx=0x7f71c401d350 comp rx=0 tx=0).stop 2024-04-25T14:17:52.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.590+0000 7f71d2000640 1 -- 172.21.15.86:0/1830921584 shutdown_connections 2024-04-25T14:17:52.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.590+0000 7f71d2000640 1 --2- 172.21.15.86:0/1830921584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71cc15feb0 0x7f71cc1602c0 unknown :-1 s=CLOSED pgs=2400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:52.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.590+0000 7f71d2000640 1 --2- 172.21.15.86:0/1830921584 >> v2:172.21.15.155:3300/0 conn(0x7f71cc15c2a0 0x7f71cc0ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:52.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.590+0000 7f71d2000640 1 -- 172.21.15.86:0/1830921584 >> 172.21.15.86:0/1830921584 conn(0x7f71cc0b4650 msgr2=0x7f71cc0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:52.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.590+0000 7f71d2000640 1 -- 172.21.15.86:0/1830921584 shutdown_connections 2024-04-25T14:17:52.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.590+0000 7f71d2000640 1 -- 172.21.15.86:0/1830921584 wait complete. 2024-04-25T14:17:52.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.590+0000 7f71d2000640 1 Processor -- start 2024-04-25T14:17:52.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.590+0000 7f71d2000640 1 -- start start 2024-04-25T14:17:52.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.594+0000 7f71d2000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71cc06fd50 0x7f71cc15b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:52.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.594+0000 7f71d2000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71cc15c2a0 0x7f71cc0c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:52.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.594+0000 7f71d2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71cc15feb0 0x7f71cc159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:52.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.594+0000 7f71d2000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc160d90 con 0x7f71cc15feb0 2024-04-25T14:17:52.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.594+0000 7f71d2000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc160c10 con 0x7f71cc06fd50 2024-04-25T14:17:52.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.594+0000 7f71d2000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f71cc0ae670 con 0x7f71cc15c2a0 2024-04-25T14:17:52.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.594+0000 7f71cbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71cc15feb0 0x7f71cc159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:52.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.594+0000 7f71cbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71cc15feb0 0x7f71cc159590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58968/0 (socket says 172.21.15.86:58968) 2024-04-25T14:17:52.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.594+0000 7f71cbe00640 1 -- 172.21.15.86:0/4039309272 learned_addr learned my addr 172.21.15.86:0/4039309272 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:52.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.594+0000 7f71cb400640 1 --2- 172.21.15.86:0/4039309272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71cc06fd50 0x7f71cc15b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:52.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.594+0000 7f71caa00640 1 --2- 172.21.15.86:0/4039309272 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71cc15c2a0 0x7f71cc0c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:52.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.594+0000 7f71cbe00640 1 -- 172.21.15.86:0/4039309272 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71cc15c2a0 msgr2=0x7f71cc0c3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:52.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.594+0000 7f71cbe00640 1 --2- 172.21.15.86:0/4039309272 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71cc15c2a0 0x7f71cc0c3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:52.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.594+0000 7f71cbe00640 1 -- 172.21.15.86:0/4039309272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71cc06fd50 msgr2=0x7f71cc15b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.594+0000 7f71cbe00640 1 --2- 172.21.15.86:0/4039309272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71cc06fd50 0x7f71cc15b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.594+0000 7f71cbe00640 1 -- 172.21.15.86:0/4039309272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71cc15a050 con 0x7f71cc15feb0 2024-04-25T14:17:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.594+0000 7f71cbe00640 1 --2- 172.21.15.86:0/4039309272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71cc15feb0 0x7f71cc159590 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f71c401a520 tx=0x7f71c4019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.594+0000 7f71c3400640 1 -- 172.21.15.86:0/4039309272 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c40b8670 con 0x7f71cc15feb0 2024-04-25T14:17:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.594+0000 7f71c3400640 1 -- 172.21.15.86:0/4039309272 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71c40be070 con 0x7f71cc15feb0 2024-04-25T14:17:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.594+0000 7f71c3400640 1 -- 172.21.15.86:0/4039309272 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c40b8a70 con 0x7f71cc15feb0 2024-04-25T14:17:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.594+0000 7f71d2000640 1 -- 172.21.15.86:0/4039309272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71cc15a330 con 0x7f71cc15feb0 2024-04-25T14:17:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.594+0000 7f71d2000640 1 -- 172.21.15.86:0/4039309272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71cc15a8f0 con 0x7f71cc15feb0 2024-04-25T14:17:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.594+0000 7f71d2000640 1 -- 172.21.15.86:0/4039309272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71cc069d00 con 0x7f71cc15feb0 2024-04-25T14:17:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.598+0000 7f71c3400640 1 -- 172.21.15.86:0/4039309272 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 84) ==== 93216+0+0 (secure 0 0 0) 0x7f71c40b8c10 con 0x7f71cc15feb0 2024-04-25T14:17:52.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.598+0000 7f71c3400640 1 --2- 172.21.15.86:0/4039309272 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f71a806f450 0x7f71a8071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:52.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.598+0000 7f71c3400640 1 -- 172.21.15.86:0/4039309272 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f71c412a6f0 con 0x7f71cc15feb0 2024-04-25T14:17:52.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.598+0000 7f71cb400640 1 --2- 172.21.15.86:0/4039309272 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f71a806f450 0x7f71a8071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:52.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.598+0000 7f71cb400640 1 --2- 172.21.15.86:0/4039309272 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f71a806f450 0x7f71a8071960 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f71cc0b4200 tx=0x7f71b4008040 comp rx=0 tx=0).ready entity=mgr.9913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:52.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.602+0000 7f71c3400640 1 -- 172.21.15.86:0/4039309272 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f71c40165a0 con 0x7f71cc15feb0 2024-04-25T14:17:52.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.802+0000 7f71d2000640 1 -- 172.21.15.86:0/4039309272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "stats"} v 0) -- 0x7f71cc1605e0 con 0x7f71cc15feb0 2024-04-25T14:17:52.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.958+0000 7f71c3400640 1 -- 172.21.15.86:0/4039309272 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "stats"}]=0 v85) ==== 85+0+0 (secure 0 0 0) 0x7f71c40f72e0 con 0x7f71cc15feb0 2024-04-25T14:17:52.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.962+0000 7f71d2000640 1 -- 172.21.15.86:0/4039309272 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f71a806f450 msgr2=0x7f71a8071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:52.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.962+0000 7f71d2000640 1 --2- 172.21.15.86:0/4039309272 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f71a806f450 0x7f71a8071960 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f71cc0b4200 tx=0x7f71b4008040 comp rx=0 tx=0).stop 2024-04-25T14:17:52.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.962+0000 7f71d2000640 1 -- 172.21.15.86:0/4039309272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71cc15feb0 msgr2=0x7f71cc159590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:52.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.962+0000 7f71d2000640 1 --2- 172.21.15.86:0/4039309272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71cc15feb0 0x7f71cc159590 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f71c401a520 tx=0x7f71c4019690 comp rx=0 tx=0).stop 2024-04-25T14:17:52.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.962+0000 7f71d2000640 1 -- 172.21.15.86:0/4039309272 shutdown_connections 2024-04-25T14:17:52.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.962+0000 7f71d2000640 1 --2- 172.21.15.86:0/4039309272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71cc15feb0 0x7f71cc159590 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:52.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.962+0000 7f71d2000640 1 --2- 172.21.15.86:0/4039309272 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f71a806f450 0x7f71a8071960 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:52.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.962+0000 7f71d2000640 1 --2- 172.21.15.86:0/4039309272 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71cc15c2a0 0x7f71cc0c3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:52.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.962+0000 7f71d2000640 1 --2- 172.21.15.86:0/4039309272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71cc06fd50 0x7f71cc15b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:52.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.962+0000 7f71d2000640 1 -- 172.21.15.86:0/4039309272 >> 172.21.15.86:0/4039309272 conn(0x7f71cc0b4650 msgr2=0x7f71cc074510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:52.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.962+0000 7f71d2000640 1 -- 172.21.15.86:0/4039309272 shutdown_connections 2024-04-25T14:17:52.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:52.962+0000 7f71d2000640 1 -- 172.21.15.86:0/4039309272 wait complete. 2024-04-25T14:17:52.984 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T14:17:52.984 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:17:53.077 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T14:17:53.162 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T14:17:53.162 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:17:53.162 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:17:53.162 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:17:53.171 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:17:53.171 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:17:53.216 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:17:53.219+0000 7f0fb926d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:17:53.220 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:53.222+0000 7f228a9e8180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:17:53.223 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:17:53.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.414+0000 7f04de600640 1 Processor -- start 2024-04-25T14:17:53.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.414+0000 7f04de600640 1 -- start start 2024-04-25T14:17:53.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.418+0000 7f04de600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04d80d2c80 0x7f04d815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:53.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.418+0000 7f04de600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f04d81616e0 0x7f04d8161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:53.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.418+0000 7f04de600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04d80cd8d0 con 0x7f04d8070e20 2024-04-25T14:17:53.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.418+0000 7f04de600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f04d80ce720 con 0x7f04d81616e0 2024-04-25T14:17:53.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.418+0000 7f04de600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f04d80ce970 con 0x7f04d80d2c80 2024-04-25T14:17:53.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.418+0000 7f04d7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f04d81616e0 0x7f04d8161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:53.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.418+0000 7f04cfe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f04d8070e20 0x7f04d8071230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59132/0 (socket says 172.21.15.86:59132) 2024-04-25T14:17:53.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.418+0000 7f04cfe00640 1 -- 172.21.15.86:0/591135596 learned_addr learned my addr 172.21.15.86:0/591135596 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:17:53.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.418+0000 7f04d7400640 1 --2- 172.21.15.86:0/591135596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04d80d2c80 0x7f04d815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:53.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.418+0000 7f04d7e00640 1 -- 172.21.15.86:0/591135596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04d80d2c80 msgr2=0x7f04d815dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:53.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.418+0000 7f04d7e00640 1 --2- 172.21.15.86:0/591135596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04d80d2c80 0x7f04d815dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:53.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.418+0000 7f04d7e00640 1 -- 172.21.15.86:0/591135596 >> v1:172.21.15.86:6789/0 conn(0x7f04d8070e20 legacy=0x7f04d8071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:53.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.418+0000 7f04d7e00640 1 -- 172.21.15.86:0/591135596 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d80d1be0 con 0x7f04d81616e0 2024-04-25T14:17:53.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.418+0000 7f04d6a00640 1 -- 172.21.15.86:0/591135596 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4037072512 0 0) 0x7f04d80cd8d0 con 0x7f04d8070e20 2024-04-25T14:17:53.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.422+0000 7f04d7e00640 1 --2- 172.21.15.86:0/591135596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04d81616e0 0x7f04d8161af0 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009d90 tx=0x7f04d001f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d5ab464469537a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:53.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.422+0000 7f04d6a00640 1 -- 172.21.15.86:0/591135596 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d00b4070 con 0x7f04d81616e0 2024-04-25T14:17:53.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.422+0000 7f04d6a00640 1 -- 172.21.15.86:0/591135596 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04d00af600 con 0x7f04d81616e0 2024-04-25T14:17:53.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.422+0000 7f04d6a00640 1 -- 172.21.15.86:0/591135596 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d00af9d0 con 0x7f04d81616e0 2024-04-25T14:17:53.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.422+0000 7f04de600640 1 -- 172.21.15.86:0/591135596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04d81616e0 msgr2=0x7f04d8161af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:53.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.422+0000 7f04de600640 1 --2- 172.21.15.86:0/591135596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04d81616e0 0x7f04d8161af0 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009d90 tx=0x7f04d001f150 comp rx=0 tx=0).stop 2024-04-25T14:17:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.422+0000 7f04de600640 1 -- 172.21.15.86:0/591135596 shutdown_connections 2024-04-25T14:17:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.422+0000 7f04de600640 1 --2- 172.21.15.86:0/591135596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04d81616e0 0x7f04d8161af0 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.422+0000 7f04de600640 1 --2- 172.21.15.86:0/591135596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04d80d2c80 0x7f04d815dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.422+0000 7f04de600640 1 -- 172.21.15.86:0/591135596 >> 172.21.15.86:0/591135596 conn(0x7f04d80cffc0 msgr2=0x7f04d80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.422+0000 7f04de600640 1 -- 172.21.15.86:0/591135596 shutdown_connections 2024-04-25T14:17:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.422+0000 7f04de600640 1 -- 172.21.15.86:0/591135596 wait complete. 2024-04-25T14:17:53.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.426+0000 7f04de600640 1 Processor -- start 2024-04-25T14:17:53.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.426+0000 7f04de600640 1 -- start start 2024-04-25T14:17:53.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.426+0000 7f04de600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04d8070e20 0x7f04d81f8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:53.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.426+0000 7f04de600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04d80d2c80 0x7f04d8150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:53.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.426+0000 7f04de600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04d81616e0 0x7f04d814acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:53.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.426+0000 7f04de600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f04d80d2480 con 0x7f04d80d2c80 2024-04-25T14:17:53.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.426+0000 7f04de600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f04d80d21a0 con 0x7f04d8070e20 2024-04-25T14:17:53.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.426+0000 7f04de600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f04d80d29a0 con 0x7f04d81616e0 2024-04-25T14:17:53.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.426+0000 7f04cfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04d80d2c80 0x7f04d8150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:53.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.426+0000 7f04cfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04d80d2c80 0x7f04d8150680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58996/0 (socket says 172.21.15.86:58996) 2024-04-25T14:17:53.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.426+0000 7f04cfe00640 1 -- 172.21.15.86:0/2073013973 learned_addr learned my addr 172.21.15.86:0/2073013973 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:53.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.426+0000 7f04d7e00640 1 --2- 172.21.15.86:0/2073013973 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04d81616e0 0x7f04d814acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:53.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.426+0000 7f04d7400640 1 --2- 172.21.15.86:0/2073013973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04d8070e20 0x7f04d81f8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:53.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.426+0000 7f04cfe00640 1 -- 172.21.15.86:0/2073013973 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04d81616e0 msgr2=0x7f04d814acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:53.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.426+0000 7f04cfe00640 1 --2- 172.21.15.86:0/2073013973 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04d81616e0 0x7f04d814acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:53.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.426+0000 7f04cfe00640 1 -- 172.21.15.86:0/2073013973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04d8070e20 msgr2=0x7f04d81f8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:53.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.426+0000 7f04cfe00640 1 --2- 172.21.15.86:0/2073013973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04d8070e20 0x7f04d81f8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:53.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.426+0000 7f04cfe00640 1 -- 172.21.15.86:0/2073013973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d8150bc0 con 0x7f04d80d2c80 2024-04-25T14:17:53.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.426+0000 7f04cfe00640 1 --2- 172.21.15.86:0/2073013973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04d80d2c80 0x7f04d8150680 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7f04c000c3c0 tx=0x7f04c000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:53.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.426+0000 7f04cf400640 1 -- 172.21.15.86:0/2073013973 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c0017070 con 0x7f04d80d2c80 2024-04-25T14:17:53.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.430+0000 7f04cf400640 1 -- 172.21.15.86:0/2073013973 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04c00074e0 con 0x7f04d80d2c80 2024-04-25T14:17:53.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.430+0000 7f04cf400640 1 -- 172.21.15.86:0/2073013973 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c00059f0 con 0x7f04d80d2c80 2024-04-25T14:17:53.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.430+0000 7f04de600640 1 -- 172.21.15.86:0/2073013973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d815c100 con 0x7f04d80d2c80 2024-04-25T14:17:53.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.430+0000 7f04de600640 1 -- 172.21.15.86:0/2073013973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d814eb20 con 0x7f04d80d2c80 2024-04-25T14:17:53.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.430+0000 7f04de600640 1 -- 172.21.15.86:0/2073013973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04d80720e0 con 0x7f04d80d2c80 2024-04-25T14:17:53.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.430+0000 7f04cf400640 1 -- 172.21.15.86:0/2073013973 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 85) ==== 93207+0+0 (secure 0 0 0) 0x7f04c001a070 con 0x7f04d80d2c80 2024-04-25T14:17:53.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.434+0000 7f04cf400640 1 --2- 172.21.15.86:0/2073013973 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f04b406f400 0x7f04b4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:53.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.434+0000 7f04d7400640 1 -- 172.21.15.86:0/2073013973 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f04b406f400 msgr2=0x7f04b4071910 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2318291413 2024-04-25T14:17:53.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.434+0000 7f04cf400640 1 -- 172.21.15.86:0/2073013973 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f04c008e7a0 con 0x7f04d80d2c80 2024-04-25T14:17:53.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.434+0000 7f04d7400640 1 --2- 172.21.15.86:0/2073013973 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f04b406f400 0x7f04b4071910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:17:53.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.438+0000 7f04cf400640 1 -- 172.21.15.86:0/2073013973 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f04c005adf0 con 0x7f04d80d2c80 2024-04-25T14:17:53.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.598+0000 7f04de600640 1 -- 172.21.15.86:0/2073013973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04d8076bf0 con 0x7f04d80d2c80 2024-04-25T14:17:53.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.598+0000 7f04cf400640 1 -- 172.21.15.86:0/2073013973 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+17382 (secure 0 0 0) 0x7f04c005f0f0 con 0x7f04d80d2c80 2024-04-25T14:17:53.597 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:53.597 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":156,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:13:51.718378+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:13:32.006877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T14:13:32.574088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2043624430":"2024-04-26T14:13:51.718341+0000","172.21.15.86:0/955458502":"2024-04-26T14:13:51.718341+0000","172.21.15.86:0/1302282677":"2024-04-26T14:13:51.718341+0000","172.21.15.86:6833/505730993":"2024-04-26T14:13:51.718341+0000","172.21.15.86:6832/505730993":"2024-04-26T14:13:51.718341+0000","172.21.15.86:0/3455088811":"2024-04-26T14:13:51.718341+0000","172.21.15.86:0/3537592233":"2024-04-26T14:13:51.718341+0000","172.21.15.86:0/2705096450":"2024-04-26T14:13:51.718341+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:17:53.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.602+0000 7f04de600640 1 -- 172.21.15.86:0/2073013973 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f04b406f400 msgr2=0x7f04b4071910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:17:53.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.602+0000 7f04de600640 1 --2- 172.21.15.86:0/2073013973 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f04b406f400 0x7f04b4071910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:53.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.602+0000 7f04de600640 1 -- 172.21.15.86:0/2073013973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04d80d2c80 msgr2=0x7f04d8150680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:53.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.602+0000 7f04de600640 1 --2- 172.21.15.86:0/2073013973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04d80d2c80 0x7f04d8150680 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7f04c000c3c0 tx=0x7f04c000c880 comp rx=0 tx=0).stop 2024-04-25T14:17:53.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.602+0000 7f04de600640 1 -- 172.21.15.86:0/2073013973 shutdown_connections 2024-04-25T14:17:53.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.602+0000 7f04de600640 1 --2- 172.21.15.86:0/2073013973 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f04b406f400 0x7f04b4071910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:53.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.602+0000 7f04de600640 1 --2- 172.21.15.86:0/2073013973 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04d81616e0 0x7f04d814acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:53.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.602+0000 7f04de600640 1 --2- 172.21.15.86:0/2073013973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04d80d2c80 0x7f04d8150680 unknown :-1 s=CLOSED pgs=2406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:53.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.602+0000 7f04de600640 1 --2- 172.21.15.86:0/2073013973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04d8070e20 0x7f04d81f8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:53.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.602+0000 7f04de600640 1 -- 172.21.15.86:0/2073013973 >> 172.21.15.86:0/2073013973 conn(0x7f04d80cffc0 msgr2=0x7f04d80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:53.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.602+0000 7f04de600640 1 -- 172.21.15.86:0/2073013973 shutdown_connections 2024-04-25T14:17:53.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.602+0000 7f04de600640 1 -- 172.21.15.86:0/2073013973 wait complete. 2024-04-25T14:17:53.620 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:13:32.006877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '154', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:13:32.574088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:17:53.620 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:17:53.636 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:17:53.639+0000 7f0fb926d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:17:53.656 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:53.658+0000 7f228a9e8180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:17:53.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.774+0000 7f1404000640 1 Processor -- start 2024-04-25T14:17:53.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.774+0000 7f1404000640 1 -- start start 2024-04-25T14:17:53.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.778+0000 7f1404000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13fc070330 0x7f13fc0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:53.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.778+0000 7f1404000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f13fc0ce790 0x7f13fc06f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:53.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.778+0000 7f1404000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13fc0d2030 con 0x7f13fc0d1c20 2024-04-25T14:17:53.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.778+0000 7f1404000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f13fc15e5c0 con 0x7f13fc0ce790 2024-04-25T14:17:53.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.778+0000 7f1404000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f13fc15e8a0 con 0x7f13fc070330 2024-04-25T14:17:53.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.778+0000 7f1400c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f13fc0d1c20 0x7f13fc06ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59144/0 (socket says 172.21.15.86:59144) 2024-04-25T14:17:53.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.778+0000 7f1400c00640 1 -- 172.21.15.86:0/222022633 learned_addr learned my addr 172.21.15.86:0/222022633 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:17:53.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.778+0000 7f1402000640 1 --2- 172.21.15.86:0/222022633 >> v2:172.21.15.155:3300/0 conn(0x7f13fc0ce790 0x7f13fc06f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:53.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.778+0000 7f1401600640 1 --2- 172.21.15.86:0/222022633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13fc070330 0x7f13fc0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:53.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.778+0000 7f13fbe00640 1 -- 172.21.15.86:0/222022633 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4169612167 0 0) 0x7f13e8001120 con 0x7f13fc0d1c20 2024-04-25T14:17:53.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.778+0000 7f13fbe00640 1 -- 172.21.15.86:0/222022633 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13e4003530 con 0x7f13fc0d1c20 2024-04-25T14:17:53.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.778+0000 7f1401600640 1 -- 172.21.15.86:0/222022633 >> v2:172.21.15.155:3300/0 conn(0x7f13fc0ce790 msgr2=0x7f13fc06f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:53.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.778+0000 7f1401600640 1 --2- 172.21.15.86:0/222022633 >> v2:172.21.15.155:3300/0 conn(0x7f13fc0ce790 0x7f13fc06f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:53.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.778+0000 7f1401600640 1 -- 172.21.15.86:0/222022633 >> v1:172.21.15.86:6789/0 conn(0x7f13fc0d1c20 legacy=0x7f13fc06ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:53.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.778+0000 7f1401600640 1 -- 172.21.15.86:0/222022633 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13fc15eb80 con 0x7f13fc070330 2024-04-25T14:17:53.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.778+0000 7f1402000640 1 --2- 172.21.15.86:0/222022633 >> v2:172.21.15.155:3300/0 conn(0x7f13fc0ce790 0x7f13fc06f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:17:53.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.782+0000 7f1401600640 1 --2- 172.21.15.86:0/222022633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13fc070330 0x7f13fc0d16e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f13f00097f0 tx=0x7f13f001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b4b9787d2662c07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:53.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.782+0000 7f13fbe00640 1 -- 172.21.15.86:0/222022633 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f00b0070 con 0x7f13fc070330 2024-04-25T14:17:53.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.782+0000 7f13fbe00640 1 -- 172.21.15.86:0/222022633 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13f00ab600 con 0x7f13fc070330 2024-04-25T14:17:53.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.782+0000 7f13fbe00640 1 -- 172.21.15.86:0/222022633 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f00ab9b0 con 0x7f13fc070330 2024-04-25T14:17:53.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.782+0000 7f1404000640 1 -- 172.21.15.86:0/222022633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13fc070330 msgr2=0x7f13fc0d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:53.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.782+0000 7f1404000640 1 --2- 172.21.15.86:0/222022633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13fc070330 0x7f13fc0d16e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f13f00097f0 tx=0x7f13f001c410 comp rx=0 tx=0).stop 2024-04-25T14:17:53.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.782+0000 7f1404000640 1 -- 172.21.15.86:0/222022633 shutdown_connections 2024-04-25T14:17:53.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.782+0000 7f1404000640 1 --2- 172.21.15.86:0/222022633 >> v2:172.21.15.155:3300/0 conn(0x7f13fc0ce790 0x7f13fc06f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:53.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.782+0000 7f1404000640 1 --2- 172.21.15.86:0/222022633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13fc070330 0x7f13fc0d16e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:53.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.782+0000 7f1404000640 1 -- 172.21.15.86:0/222022633 >> 172.21.15.86:0/222022633 conn(0x7f13fc0b4650 msgr2=0x7f13fc0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:53.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.782+0000 7f1404000640 1 -- 172.21.15.86:0/222022633 shutdown_connections 2024-04-25T14:17:53.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.782+0000 7f1404000640 1 -- 172.21.15.86:0/222022633 wait complete. 2024-04-25T14:17:53.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.786+0000 7f1404000640 1 Processor -- start 2024-04-25T14:17:53.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.786+0000 7f1404000640 1 -- start start 2024-04-25T14:17:53.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.786+0000 7f1404000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13fc070330 0x7f13fc0b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:53.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.786+0000 7f1404000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13fc0ce790 0x7f13fc15b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:53.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.786+0000 7f1404000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13fc0d1c20 0x7f13fc0c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:53.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.786+0000 7f1404000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc0708c0 con 0x7f13fc0ce790 2024-04-25T14:17:53.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.786+0000 7f1404000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc070740 con 0x7f13fc0d1c20 2024-04-25T14:17:53.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.786+0000 7f1404000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f13fc16afc0 con 0x7f13fc070330 2024-04-25T14:17:53.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.786+0000 7f1400c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13fc0ce790 0x7f13fc15b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:53.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.786+0000 7f1401600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13fc070330 0x7f13fc0b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:53.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.786+0000 7f1400c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13fc0ce790 0x7f13fc15b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59002/0 (socket says 172.21.15.86:59002) 2024-04-25T14:17:53.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.786+0000 7f1400c00640 1 -- 172.21.15.86:0/3455135163 learned_addr learned my addr 172.21.15.86:0/3455135163 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:53.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.786+0000 7f1401600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13fc070330 0x7f13fc0b1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:47088/0 (socket says 172.21.15.86:47088) 2024-04-25T14:17:53.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.786+0000 7f1402000640 1 --2- 172.21.15.86:0/3455135163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13fc0d1c20 0x7f13fc0c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:53.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.786+0000 7f1400c00640 1 -- 172.21.15.86:0/3455135163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13fc070330 msgr2=0x7f13fc0b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:53.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.786+0000 7f1400c00640 1 --2- 172.21.15.86:0/3455135163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13fc070330 0x7f13fc0b1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:53.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.786+0000 7f1400c00640 1 -- 172.21.15.86:0/3455135163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13fc0d1c20 msgr2=0x7f13fc0c59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:53.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.786+0000 7f1400c00640 1 --2- 172.21.15.86:0/3455135163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13fc0d1c20 0x7f13fc0c59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:53.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.786+0000 7f1400c00640 1 -- 172.21.15.86:0/3455135163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13fc15cef0 con 0x7f13fc0ce790 2024-04-25T14:17:53.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.786+0000 7f1401600640 1 --2- 172.21.15.86:0/3455135163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13fc070330 0x7f13fc0b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:17:53.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.786+0000 7f1400c00640 1 --2- 172.21.15.86:0/3455135163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13fc0ce790 0x7f13fc15b390 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f13e800c3f0 tx=0x7f13e800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:53.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.786+0000 7f13f9600640 1 -- 172.21.15.86:0/3455135163 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e8017070 con 0x7f13fc0ce790 2024-04-25T14:17:53.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.786+0000 7f13f9600640 1 -- 172.21.15.86:0/3455135163 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13e8005cf0 con 0x7f13fc0ce790 2024-04-25T14:17:53.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.786+0000 7f13f9600640 1 -- 172.21.15.86:0/3455135163 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e8007310 con 0x7f13fc0ce790 2024-04-25T14:17:53.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.790+0000 7f1404000640 1 -- 172.21.15.86:0/3455135163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13fc15b8d0 con 0x7f13fc0ce790 2024-04-25T14:17:53.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.790+0000 7f1404000640 1 -- 172.21.15.86:0/3455135163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13fc0b14e0 con 0x7f13fc0ce790 2024-04-25T14:17:53.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.790+0000 7f1404000640 1 -- 172.21.15.86:0/3455135163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13fc0c7040 con 0x7f13fc0ce790 2024-04-25T14:17:53.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.790+0000 7f13f9600640 1 -- 172.21.15.86:0/3455135163 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 85) ==== 93207+0+0 (secure 0 0 0) 0x7f13e801a070 con 0x7f13fc0ce790 2024-04-25T14:17:53.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.790+0000 7f13f9600640 1 --2- 172.21.15.86:0/3455135163 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f13dc06f430 0x7f13dc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:53.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.790+0000 7f1401600640 1 -- 172.21.15.86:0/3455135163 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f13dc06f430 msgr2=0x7f13dc071940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2318291413 2024-04-25T14:17:53.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.790+0000 7f13f9600640 1 -- 172.21.15.86:0/3455135163 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f13e808e900 con 0x7f13fc0ce790 2024-04-25T14:17:53.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.790+0000 7f1401600640 1 --2- 172.21.15.86:0/3455135163 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f13dc06f430 0x7f13dc071940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:17:53.790 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:17:53.791+0000 7f0fb926d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:17:53.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.794+0000 7f13f9600640 1 -- 172.21.15.86:0/3455135163 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f13fc0d2030 con 0x7f13fc0ce790 2024-04-25T14:17:53.814 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:53.814+0000 7f228a9e8180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:17:53.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.954+0000 7f1404000640 1 -- 172.21.15.86:0/3455135163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f13fc069d00 con 0x7f13fc0ce790 2024-04-25T14:17:53.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.954+0000 7f13f9600640 1 -- 172.21.15.86:0/3455135163 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v156) ==== 93+0+10 (secure 0 0 0) 0x7f13e805ae20 con 0x7f13fc0ce790 2024-04-25T14:17:53.955 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:17:53.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.958+0000 7f1404000640 1 -- 172.21.15.86:0/3455135163 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f13dc06f430 msgr2=0x7f13dc071940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:17:53.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.958+0000 7f1404000640 1 --2- 172.21.15.86:0/3455135163 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f13dc06f430 0x7f13dc071940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:53.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.958+0000 7f1404000640 1 -- 172.21.15.86:0/3455135163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13fc0ce790 msgr2=0x7f13fc15b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:53.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.958+0000 7f1404000640 1 --2- 172.21.15.86:0/3455135163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13fc0ce790 0x7f13fc15b390 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f13e800c3f0 tx=0x7f13e800c7f0 comp rx=0 tx=0).stop 2024-04-25T14:17:53.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.958+0000 7f1404000640 1 -- 172.21.15.86:0/3455135163 shutdown_connections 2024-04-25T14:17:53.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.958+0000 7f1404000640 1 --2- 172.21.15.86:0/3455135163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13fc0d1c20 0x7f13fc0c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:53.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.958+0000 7f1404000640 1 --2- 172.21.15.86:0/3455135163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13fc0ce790 0x7f13fc15b390 unknown :-1 s=CLOSED pgs=2408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:53.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.958+0000 7f1404000640 1 --2- 172.21.15.86:0/3455135163 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f13dc06f430 0x7f13dc071940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:53.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.958+0000 7f1404000640 1 --2- 172.21.15.86:0/3455135163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13fc070330 0x7f13fc0b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:53.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.958+0000 7f1404000640 1 -- 172.21.15.86:0/3455135163 >> 172.21.15.86:0/3455135163 conn(0x7f13fc0b4650 msgr2=0x7f13fc0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:53.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.958+0000 7f1404000640 1 -- 172.21.15.86:0/3455135163 shutdown_connections 2024-04-25T14:17:53.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:53.958+0000 7f1404000640 1 -- 172.21.15.86:0/3455135163 wait complete. 2024-04-25T14:17:53.977 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:17:54.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.170+0000 7f8c9c600640 1 Processor -- start 2024-04-25T14:17:54.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.170+0000 7f8c9c600640 1 -- start start 2024-04-25T14:17:54.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.174+0000 7f8c9c600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c940d2c80 0x7f8c9415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:54.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.174+0000 7f8c9c600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8c941616e0 0x7f8c94161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:54.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.174+0000 7f8c9c600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c940cd8d0 con 0x7f8c94070e20 2024-04-25T14:17:54.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.174+0000 7f8c9c600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8c940ce720 con 0x7f8c941616e0 2024-04-25T14:17:54.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.174+0000 7f8c9c600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8c940ce970 con 0x7f8c940d2c80 2024-04-25T14:17:54.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.174+0000 7f8c9a600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8c941616e0 0x7f8c94161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:54.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.174+0000 7f8c99200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8c94070e20 0x7f8c94071230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59158/0 (socket says 172.21.15.86:59158) 2024-04-25T14:17:54.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.174+0000 7f8c99200640 1 -- 172.21.15.86:0/1283217635 learned_addr learned my addr 172.21.15.86:0/1283217635 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:17:54.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.174+0000 7f8c99c00640 1 --2- 172.21.15.86:0/1283217635 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c940d2c80 0x7f8c9415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:54.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.174+0000 7f8c9a600640 1 -- 172.21.15.86:0/1283217635 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c940d2c80 msgr2=0x7f8c9415dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:54.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.174+0000 7f8c9a600640 1 --2- 172.21.15.86:0/1283217635 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c940d2c80 0x7f8c9415dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:54.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.174+0000 7f8c9a600640 1 -- 172.21.15.86:0/1283217635 >> v1:172.21.15.86:6789/0 conn(0x7f8c94070e20 legacy=0x7f8c94071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:54.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.174+0000 7f8c9a600640 1 -- 172.21.15.86:0/1283217635 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c940d1be0 con 0x7f8c941616e0 2024-04-25T14:17:54.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.178+0000 7f8c9a600640 1 --2- 172.21.15.86:0/1283217635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c941616e0 0x7f8c94161af0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009d90 tx=0x7f8c8c01f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b2aa2cd4bb936f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:54.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.178+0000 7f8c93e00640 1 -- 172.21.15.86:0/1283217635 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c0b4070 con 0x7f8c941616e0 2024-04-25T14:17:54.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.178+0000 7f8c93e00640 1 -- 172.21.15.86:0/1283217635 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c8c0af600 con 0x7f8c941616e0 2024-04-25T14:17:54.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.178+0000 7f8c93e00640 1 -- 172.21.15.86:0/1283217635 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c0af9d0 con 0x7f8c941616e0 2024-04-25T14:17:54.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.178+0000 7f8c9c600640 1 -- 172.21.15.86:0/1283217635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c941616e0 msgr2=0x7f8c94161af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:54.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.178+0000 7f8c9c600640 1 --2- 172.21.15.86:0/1283217635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c941616e0 0x7f8c94161af0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009d90 tx=0x7f8c8c01f150 comp rx=0 tx=0).stop 2024-04-25T14:17:54.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.178+0000 7f8c9c600640 1 -- 172.21.15.86:0/1283217635 shutdown_connections 2024-04-25T14:17:54.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.178+0000 7f8c9c600640 1 --2- 172.21.15.86:0/1283217635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c941616e0 0x7f8c94161af0 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:54.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.178+0000 7f8c9c600640 1 --2- 172.21.15.86:0/1283217635 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c940d2c80 0x7f8c9415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:54.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.178+0000 7f8c9c600640 1 -- 172.21.15.86:0/1283217635 >> 172.21.15.86:0/1283217635 conn(0x7f8c940cffc0 msgr2=0x7f8c940d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:54.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.178+0000 7f8c9c600640 1 -- 172.21.15.86:0/1283217635 shutdown_connections 2024-04-25T14:17:54.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.178+0000 7f8c9c600640 1 -- 172.21.15.86:0/1283217635 wait complete. 2024-04-25T14:17:54.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.178+0000 7f8c9c600640 1 Processor -- start 2024-04-25T14:17:54.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.182+0000 7f8c9c600640 1 -- start start 2024-04-25T14:17:54.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.182+0000 7f8c9c600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c94070e20 0x7f8c94156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:54.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.182+0000 7f8c9c600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c940d2c80 0x7f8c9415a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:54.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.182+0000 7f8c9c600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c941616e0 0x7f8c940cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:54.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.182+0000 7f8c9c600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c940d2480 con 0x7f8c94070e20 2024-04-25T14:17:54.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.182+0000 7f8c9c600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8c940d21a0 con 0x7f8c940d2c80 2024-04-25T14:17:54.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.182+0000 7f8c9c600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8c940d29a0 con 0x7f8c941616e0 2024-04-25T14:17:54.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.182+0000 7f8c99c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c94070e20 0x7f8c94156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:54.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.182+0000 7f8c99c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c94070e20 0x7f8c94156b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59024/0 (socket says 172.21.15.86:59024) 2024-04-25T14:17:54.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.182+0000 7f8c99c00640 1 -- 172.21.15.86:0/4261617573 learned_addr learned my addr 172.21.15.86:0/4261617573 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:54.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.182+0000 7f8c9a600640 1 --2- 172.21.15.86:0/4261617573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c941616e0 0x7f8c940cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:54.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.182+0000 7f8c99200640 1 --2- 172.21.15.86:0/4261617573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c940d2c80 0x7f8c9415a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:54.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.182+0000 7f8c99c00640 1 -- 172.21.15.86:0/4261617573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c941616e0 msgr2=0x7f8c940cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:54.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.182+0000 7f8c99c00640 1 --2- 172.21.15.86:0/4261617573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c941616e0 0x7f8c940cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:54.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.182+0000 7f8c99c00640 1 -- 172.21.15.86:0/4261617573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c940d2c80 msgr2=0x7f8c9415a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:54.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.182+0000 7f8c99c00640 1 --2- 172.21.15.86:0/4261617573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c940d2c80 0x7f8c9415a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:54.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.182+0000 7f8c99c00640 1 -- 172.21.15.86:0/4261617573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c9415ab60 con 0x7f8c94070e20 2024-04-25T14:17:54.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.182+0000 7f8c99200640 1 --2- 172.21.15.86:0/4261617573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c940d2c80 0x7f8c9415a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:17:54.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.182+0000 7f8c99c00640 1 --2- 172.21.15.86:0/4261617573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c94070e20 0x7f8c94156b70 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f8c8800e8a0 tx=0x7f8c8800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:54.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.182+0000 7f8c91600640 1 -- 172.21.15.86:0/4261617573 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c88017070 con 0x7f8c94070e20 2024-04-25T14:17:54.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.182+0000 7f8c91600640 1 -- 172.21.15.86:0/4261617573 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c8800c940 con 0x7f8c94070e20 2024-04-25T14:17:54.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.182+0000 7f8c91600640 1 -- 172.21.15.86:0/4261617573 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8800ccf0 con 0x7f8c94070e20 2024-04-25T14:17:54.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.182+0000 7f8c9c600640 1 -- 172.21.15.86:0/4261617573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c9415ae40 con 0x7f8c94070e20 2024-04-25T14:17:54.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.182+0000 7f8c9c600640 1 -- 172.21.15.86:0/4261617573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c9415b400 con 0x7f8c94070e20 2024-04-25T14:17:54.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.186+0000 7f8c9c600640 1 -- 172.21.15.86:0/4261617573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c940720e0 con 0x7f8c94070e20 2024-04-25T14:17:54.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.186+0000 7f8c91600640 1 -- 172.21.15.86:0/4261617573 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 85) ==== 93207+0+0 (secure 0 0 0) 0x7f8c8801a070 con 0x7f8c94070e20 2024-04-25T14:17:54.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.186+0000 7f8c91600640 1 --2- 172.21.15.86:0/4261617573 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8c7406f400 0x7f8c74071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:54.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.186+0000 7f8c99200640 1 -- 172.21.15.86:0/4261617573 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8c7406f400 msgr2=0x7f8c74071910 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2318291413 2024-04-25T14:17:54.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.186+0000 7f8c91600640 1 -- 172.21.15.86:0/4261617573 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f8c8808f890 con 0x7f8c94070e20 2024-04-25T14:17:54.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.186+0000 7f8c99200640 1 --2- 172.21.15.86:0/4261617573 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8c7406f400 0x7f8c74071910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:17:54.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.190+0000 7f8c91600640 1 -- 172.21.15.86:0/4261617573 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8c8805c490 con 0x7f8c94070e20 2024-04-25T14:17:54.270 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:17:54.271+0000 7f0fb926d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:17:54.329 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:17:54.331+0000 7f0fb926d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:17:54.339 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:54.342+0000 7f228a9e8180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:17:54.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.390+0000 7f8c99200640 1 -- 172.21.15.86:0/4261617573 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8c7406f400 msgr2=0x7f8c74071910 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2318291413 2024-04-25T14:17:54.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.390+0000 7f8c99200640 1 --2- 172.21.15.86:0/4261617573 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8c7406f400 0x7f8c74071910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:17:54.387 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:17:54.391+0000 7f0fb926d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:17:54.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.390+0000 7f8c9c600640 1 -- 172.21.15.86:0/4261617573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8c94076bf0 con 0x7f8c94070e20 2024-04-25T14:17:54.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.390+0000 7f8c91600640 1 -- 172.21.15.86:0/4261617573 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v156) ==== 104+0+11 (secure 0 0 0) 0x7f8c8805bd90 con 0x7f8c94070e20 2024-04-25T14:17:54.388 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:17:54.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.390+0000 7f8c9c600640 1 -- 172.21.15.86:0/4261617573 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8c7406f400 msgr2=0x7f8c74071910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:17:54.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.390+0000 7f8c9c600640 1 --2- 172.21.15.86:0/4261617573 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8c7406f400 0x7f8c74071910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:54.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.390+0000 7f8c9c600640 1 -- 172.21.15.86:0/4261617573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c94070e20 msgr2=0x7f8c94156b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:54.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.390+0000 7f8c9c600640 1 --2- 172.21.15.86:0/4261617573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c94070e20 0x7f8c94156b70 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f8c8800e8a0 tx=0x7f8c8800eca0 comp rx=0 tx=0).stop 2024-04-25T14:17:54.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.390+0000 7f8c9c600640 1 -- 172.21.15.86:0/4261617573 shutdown_connections 2024-04-25T14:17:54.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.390+0000 7f8c9c600640 1 --2- 172.21.15.86:0/4261617573 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f8c7406f400 0x7f8c74071910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:54.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.390+0000 7f8c9c600640 1 --2- 172.21.15.86:0/4261617573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c941616e0 0x7f8c940cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:54.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.390+0000 7f8c9c600640 1 --2- 172.21.15.86:0/4261617573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c940d2c80 0x7f8c9415a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:54.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.390+0000 7f8c9c600640 1 --2- 172.21.15.86:0/4261617573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c94070e20 0x7f8c94156b70 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:54.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.390+0000 7f8c9c600640 1 -- 172.21.15.86:0/4261617573 >> 172.21.15.86:0/4261617573 conn(0x7f8c940cffc0 msgr2=0x7f8c94158f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:54.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.390+0000 7f8c9c600640 1 -- 172.21.15.86:0/4261617573 shutdown_connections 2024-04-25T14:17:54.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.390+0000 7f8c9c600640 1 -- 172.21.15.86:0/4261617573 wait complete. 2024-04-25T14:17:54.400 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:54.402+0000 7f228a9e8180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:17:54.411 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:17:54.461 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:54.462+0000 7f228a9e8180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:17:54.475 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:17:54.479+0000 7f0fb926d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:17:54.532 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:17:54.535+0000 7f0fb926d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:17:54.551 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:54.554+0000 7f228a9e8180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:17:54.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.566+0000 7fbb40c00640 1 Processor -- start 2024-04-25T14:17:54.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.566+0000 7fbb40c00640 1 -- start start 2024-04-25T14:17:54.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.566+0000 7fbb40c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb3c071530 0x7fbb3c15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:54.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.566+0000 7fbb40c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbb3c1616e0 0x7fbb3c161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:54.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.566+0000 7fbb40c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb3c0cd8d0 con 0x7fbb3c0d2c80 2024-04-25T14:17:54.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.566+0000 7fbb40c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbb3c0ce720 con 0x7fbb3c1616e0 2024-04-25T14:17:54.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.566+0000 7fbb40c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbb3c0ce970 con 0x7fbb3c071530 2024-04-25T14:17:54.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.566+0000 7fbb39e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fbb3c0d2c80 0x7fbb3c070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59160/0 (socket says 172.21.15.86:59160) 2024-04-25T14:17:54.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.566+0000 7fbb39e00640 1 -- 172.21.15.86:0/3754862807 learned_addr learned my addr 172.21.15.86:0/3754862807 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:17:54.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.566+0000 7fbb3a800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbb3c1616e0 0x7fbb3c161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:54.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.566+0000 7fbb39400640 1 --2- 172.21.15.86:0/3754862807 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb3c071530 0x7fbb3c15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:54.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.566+0000 7fbb3a800640 1 -- 172.21.15.86:0/3754862807 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb3c071530 msgr2=0x7fbb3c15dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:54.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.566+0000 7fbb3a800640 1 --2- 172.21.15.86:0/3754862807 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb3c071530 0x7fbb3c15dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:54.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.566+0000 7fbb3a800640 1 -- 172.21.15.86:0/3754862807 >> v1:172.21.15.86:6789/0 conn(0x7fbb3c0d2c80 legacy=0x7fbb3c070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:54.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.566+0000 7fbb3a800640 1 -- 172.21.15.86:0/3754862807 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3c0d1be0 con 0x7fbb3c1616e0 2024-04-25T14:17:54.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.566+0000 7fbb38a00640 1 -- 172.21.15.86:0/3754862807 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 553738084 0 0) 0x7fbb3c0cd8d0 con 0x7fbb3c0d2c80 2024-04-25T14:17:54.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.570+0000 7fbb3a800640 1 --2- 172.21.15.86:0/3754862807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb3c1616e0 0x7fbb3c161af0 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009d90 tx=0x7fbb3401f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80d0589bd9bafd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:54.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.570+0000 7fbb38a00640 1 -- 172.21.15.86:0/3754862807 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb340b4070 con 0x7fbb3c1616e0 2024-04-25T14:17:54.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.570+0000 7fbb38a00640 1 -- 172.21.15.86:0/3754862807 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb340af600 con 0x7fbb3c1616e0 2024-04-25T14:17:54.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.570+0000 7fbb38a00640 1 -- 172.21.15.86:0/3754862807 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb340af9d0 con 0x7fbb3c1616e0 2024-04-25T14:17:54.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.570+0000 7fbb40c00640 1 -- 172.21.15.86:0/3754862807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb3c1616e0 msgr2=0x7fbb3c161af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:54.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.570+0000 7fbb40c00640 1 --2- 172.21.15.86:0/3754862807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb3c1616e0 0x7fbb3c161af0 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009d90 tx=0x7fbb3401f150 comp rx=0 tx=0).stop 2024-04-25T14:17:54.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.570+0000 7fbb40c00640 1 -- 172.21.15.86:0/3754862807 shutdown_connections 2024-04-25T14:17:54.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.570+0000 7fbb40c00640 1 --2- 172.21.15.86:0/3754862807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb3c1616e0 0x7fbb3c161af0 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:54.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.570+0000 7fbb40c00640 1 --2- 172.21.15.86:0/3754862807 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb3c071530 0x7fbb3c15dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:54.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.570+0000 7fbb40c00640 1 -- 172.21.15.86:0/3754862807 >> 172.21.15.86:0/3754862807 conn(0x7fbb3c0cffc0 msgr2=0x7fbb3c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:54.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.570+0000 7fbb40c00640 1 -- 172.21.15.86:0/3754862807 shutdown_connections 2024-04-25T14:17:54.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.570+0000 7fbb40c00640 1 -- 172.21.15.86:0/3754862807 wait complete. 2024-04-25T14:17:54.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.574+0000 7fbb40c00640 1 Processor -- start 2024-04-25T14:17:54.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.574+0000 7fbb40c00640 1 -- start start 2024-04-25T14:17:54.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.574+0000 7fbb40c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb3c0d2c80 0x7fbb3c1504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:54.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.574+0000 7fbb40c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb3c1616e0 0x7fbb3c15f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:54.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.574+0000 7fbb40c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb3c15fd00 0x7fbb3c15bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:54.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.574+0000 7fbb40c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c0d2480 con 0x7fbb3c1616e0 2024-04-25T14:17:54.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.574+0000 7fbb40c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c0d21a0 con 0x7fbb3c0d2c80 2024-04-25T14:17:54.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.574+0000 7fbb40c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbb3c0d29a0 con 0x7fbb3c15fd00 2024-04-25T14:17:54.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.574+0000 7fbb39400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb3c1616e0 0x7fbb3c15f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:54.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.574+0000 7fbb39400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb3c1616e0 0x7fbb3c15f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59038/0 (socket says 172.21.15.86:59038) 2024-04-25T14:17:54.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.574+0000 7fbb39400640 1 -- 172.21.15.86:0/1224433386 learned_addr learned my addr 172.21.15.86:0/1224433386 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:54.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.574+0000 7fbb39e00640 1 --2- 172.21.15.86:0/1224433386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb3c0d2c80 0x7fbb3c1504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:54.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.574+0000 7fbb3a800640 1 --2- 172.21.15.86:0/1224433386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb3c15fd00 0x7fbb3c15bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:54.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.574+0000 7fbb39400640 1 -- 172.21.15.86:0/1224433386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb3c15fd00 msgr2=0x7fbb3c15bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:54.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.574+0000 7fbb39400640 1 --2- 172.21.15.86:0/1224433386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb3c15fd00 0x7fbb3c15bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:54.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.574+0000 7fbb39400640 1 -- 172.21.15.86:0/1224433386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb3c0d2c80 msgr2=0x7fbb3c1504c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:54.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.574+0000 7fbb39400640 1 --2- 172.21.15.86:0/1224433386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb3c0d2c80 0x7fbb3c1504c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:54.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.574+0000 7fbb39400640 1 -- 172.21.15.86:0/1224433386 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3c06db40 con 0x7fbb3c1616e0 2024-04-25T14:17:54.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.578+0000 7fbb39400640 1 --2- 172.21.15.86:0/1224433386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb3c1616e0 0x7fbb3c15f7c0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800e870 tx=0x7fbb2800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:54.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.578+0000 7fbb32000640 1 -- 172.21.15.86:0/1224433386 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb28017070 con 0x7fbb3c1616e0 2024-04-25T14:17:54.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.578+0000 7fbb32000640 1 -- 172.21.15.86:0/1224433386 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb2800c940 con 0x7fbb3c1616e0 2024-04-25T14:17:54.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.578+0000 7fbb40c00640 1 -- 172.21.15.86:0/1224433386 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb3c06de20 con 0x7fbb3c1616e0 2024-04-25T14:17:54.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.578+0000 7fbb32000640 1 -- 172.21.15.86:0/1224433386 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2800ccf0 con 0x7fbb3c1616e0 2024-04-25T14:17:54.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.578+0000 7fbb40c00640 1 -- 172.21.15.86:0/1224433386 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb3c06e3e0 con 0x7fbb3c1616e0 2024-04-25T14:17:54.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.578+0000 7fbb40c00640 1 -- 172.21.15.86:0/1224433386 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb3c076bf0 con 0x7fbb3c1616e0 2024-04-25T14:17:54.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.578+0000 7fbb32000640 1 -- 172.21.15.86:0/1224433386 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 85) ==== 93207+0+0 (secure 0 0 0) 0x7fbb2801a070 con 0x7fbb3c1616e0 2024-04-25T14:17:54.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.582+0000 7fbb32000640 1 --2- 172.21.15.86:0/1224433386 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fbb1c06f400 0x7fbb1c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:54.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.582+0000 7fbb32000640 1 -- 172.21.15.86:0/1224433386 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fbb2808f7a0 con 0x7fbb3c1616e0 2024-04-25T14:17:54.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.582+0000 7fbb39e00640 1 -- 172.21.15.86:0/1224433386 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fbb1c06f400 msgr2=0x7fbb1c071910 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2318291413 2024-04-25T14:17:54.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.582+0000 7fbb39e00640 1 --2- 172.21.15.86:0/1224433386 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fbb1c06f400 0x7fbb1c071910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:17:54.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.586+0000 7fbb32000640 1 -- 172.21.15.86:0/1224433386 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbb2805bf40 con 0x7fbb3c1616e0 2024-04-25T14:17:54.614 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:54.614+0000 7f228a9e8180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:17:54.619 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:17:54.619+0000 7f0fb926d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:17:54.679 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:17:54.679+0000 7f0fb926d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:17:54.701 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:54.702+0000 7f228a9e8180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:17:54.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.746+0000 7fbb40c00640 1 -- 172.21.15.86:0/1224433386 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbb3c0d1470 con 0x7fbb3c1616e0 2024-04-25T14:17:54.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.746+0000 7fbb32000640 1 -- 172.21.15.86:0/1224433386 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v156) ==== 100+0+11 (secure 0 0 0) 0x7fbb28060240 con 0x7fbb3c1616e0 2024-04-25T14:17:54.744 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:17:54.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.746+0000 7fbb40c00640 1 -- 172.21.15.86:0/1224433386 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fbb1c06f400 msgr2=0x7fbb1c071910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:17:54.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.746+0000 7fbb40c00640 1 --2- 172.21.15.86:0/1224433386 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fbb1c06f400 0x7fbb1c071910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:54.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.746+0000 7fbb40c00640 1 -- 172.21.15.86:0/1224433386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb3c1616e0 msgr2=0x7fbb3c15f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:54.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.746+0000 7fbb40c00640 1 --2- 172.21.15.86:0/1224433386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb3c1616e0 0x7fbb3c15f7c0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800e870 tx=0x7fbb2800ed30 comp rx=0 tx=0).stop 2024-04-25T14:17:54.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.746+0000 7fbb40c00640 1 -- 172.21.15.86:0/1224433386 shutdown_connections 2024-04-25T14:17:54.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.746+0000 7fbb40c00640 1 --2- 172.21.15.86:0/1224433386 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fbb1c06f400 0x7fbb1c071910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:54.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.746+0000 7fbb40c00640 1 --2- 172.21.15.86:0/1224433386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbb3c15fd00 0x7fbb3c15bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:54.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.746+0000 7fbb40c00640 1 --2- 172.21.15.86:0/1224433386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb3c1616e0 0x7fbb3c15f7c0 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:54.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.746+0000 7fbb40c00640 1 --2- 172.21.15.86:0/1224433386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb3c0d2c80 0x7fbb3c1504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:54.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.746+0000 7fbb40c00640 1 -- 172.21.15.86:0/1224433386 >> 172.21.15.86:0/1224433386 conn(0x7fbb3c0cffc0 msgr2=0x7fbb3c15a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:54.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.746+0000 7fbb40c00640 1 -- 172.21.15.86:0/1224433386 shutdown_connections 2024-04-25T14:17:54.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.746+0000 7fbb40c00640 1 -- 172.21.15.86:0/1224433386 wait complete. 2024-04-25T14:17:54.763 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:54.766+0000 7f228a9e8180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:17:54.768 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T14:17:54.832 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:17:54.835+0000 7f0fb926d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:17:54.893 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:17:54.895+0000 7f0fb926d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:17:54.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.918+0000 7fca74400640 1 Processor -- start 2024-04-25T14:17:54.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.918+0000 7fca74400640 1 -- start start 2024-04-25T14:17:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.922+0000 7fca74400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fca6c0ccab0 0x7fca6c0d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.922+0000 7fca74400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fca6c0cf9d0 0x7fca6c0d2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.922+0000 7fca74400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca6c0b3bd0 con 0x7fca6c0b79c0 2024-04-25T14:17:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.922+0000 7fca74400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fca6c0cf190 con 0x7fca6c0cf9d0 2024-04-25T14:17:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.922+0000 7fca74400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fca6c0b2620 con 0x7fca6c0ccab0 2024-04-25T14:17:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.922+0000 7fca72400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fca6c0cf9d0 0x7fca6c0d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.922+0000 7fca71000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fca6c0b79c0 0x7fca6c1617f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59162/0 (socket says 172.21.15.86:59162) 2024-04-25T14:17:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.922+0000 7fca71000640 1 -- 172.21.15.86:0/1876954593 learned_addr learned my addr 172.21.15.86:0/1876954593 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:17:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.922+0000 7fca71a00640 1 --2- 172.21.15.86:0/1876954593 >> v2:172.21.15.155:3300/0 conn(0x7fca6c0ccab0 0x7fca6c0d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.922+0000 7fca72400640 1 -- 172.21.15.86:0/1876954593 >> v2:172.21.15.155:3300/0 conn(0x7fca6c0ccab0 msgr2=0x7fca6c0d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.922+0000 7fca72400640 1 --2- 172.21.15.86:0/1876954593 >> v2:172.21.15.155:3300/0 conn(0x7fca6c0ccab0 0x7fca6c0d23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.922+0000 7fca72400640 1 -- 172.21.15.86:0/1876954593 >> v1:172.21.15.86:6789/0 conn(0x7fca6c0b79c0 legacy=0x7fca6c1617f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.922+0000 7fca72400640 1 -- 172.21.15.86:0/1876954593 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6c0b1cf0 con 0x7fca6c0cf9d0 2024-04-25T14:17:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.922+0000 7fca6be00640 1 -- 172.21.15.86:0/1876954593 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2147177885 0 0) 0x7fca6c0b3bd0 con 0x7fca6c0b79c0 2024-04-25T14:17:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.922+0000 7fca71a00640 1 --2- 172.21.15.86:0/1876954593 >> v2:172.21.15.155:3300/0 conn(0x7fca6c0ccab0 0x7fca6c0d23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:17:54.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.926+0000 7fca72400640 1 --2- 172.21.15.86:0/1876954593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca6c0cf9d0 0x7fca6c0d2910 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7fca6c073230 tx=0x7fca6401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ce1f3df5943269 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.926+0000 7fca6be00640 1 -- 172.21.15.86:0/1876954593 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca640b2070 con 0x7fca6c0cf9d0 2024-04-25T14:17:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.926+0000 7fca6be00640 1 -- 172.21.15.86:0/1876954593 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca640a5820 con 0x7fca6c0cf9d0 2024-04-25T14:17:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.926+0000 7fca6be00640 1 -- 172.21.15.86:0/1876954593 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca640a5bf0 con 0x7fca6c0cf9d0 2024-04-25T14:17:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.926+0000 7fca74400640 1 -- 172.21.15.86:0/1876954593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca6c0cf9d0 msgr2=0x7fca6c0d2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.926+0000 7fca74400640 1 --2- 172.21.15.86:0/1876954593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca6c0cf9d0 0x7fca6c0d2910 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7fca6c073230 tx=0x7fca6401c930 comp rx=0 tx=0).stop 2024-04-25T14:17:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.926+0000 7fca74400640 1 -- 172.21.15.86:0/1876954593 shutdown_connections 2024-04-25T14:17:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.926+0000 7fca74400640 1 --2- 172.21.15.86:0/1876954593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca6c0cf9d0 0x7fca6c0d2910 unknown :-1 s=CLOSED pgs=2416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.926+0000 7fca74400640 1 --2- 172.21.15.86:0/1876954593 >> v2:172.21.15.155:3300/0 conn(0x7fca6c0ccab0 0x7fca6c0d23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.926+0000 7fca74400640 1 -- 172.21.15.86:0/1876954593 >> 172.21.15.86:0/1876954593 conn(0x7fca6c09ba30 msgr2=0x7fca6c0b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.926+0000 7fca74400640 1 -- 172.21.15.86:0/1876954593 shutdown_connections 2024-04-25T14:17:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.926+0000 7fca74400640 1 -- 172.21.15.86:0/1876954593 wait complete. 2024-04-25T14:17:54.925 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:54.926+0000 7f228a9e8180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:17:54.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.926+0000 7fca74400640 1 Processor -- start 2024-04-25T14:17:54.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.930+0000 7fca74400640 1 -- start start 2024-04-25T14:17:54.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.930+0000 7fca74400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca6c0b79c0 0x7fca6c15cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:54.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.930+0000 7fca74400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca6c0ccab0 0x7fca6c1532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:54.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.930+0000 7fca74400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca6c0cf9d0 0x7fca6c153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:54.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.930+0000 7fca74400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c16ab00 con 0x7fca6c0b79c0 2024-04-25T14:17:54.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.930+0000 7fca74400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c16a980 con 0x7fca6c0cf9d0 2024-04-25T14:17:54.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.930+0000 7fca74400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fca6c16ac80 con 0x7fca6c0ccab0 2024-04-25T14:17:54.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.930+0000 7fca71a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca6c0b79c0 0x7fca6c15cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:54.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.930+0000 7fca71a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca6c0b79c0 0x7fca6c15cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59062/0 (socket says 172.21.15.86:59062) 2024-04-25T14:17:54.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.930+0000 7fca71a00640 1 -- 172.21.15.86:0/930202382 learned_addr learned my addr 172.21.15.86:0/930202382 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:54.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.930+0000 7fca72400640 1 --2- 172.21.15.86:0/930202382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca6c0cf9d0 0x7fca6c153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:54.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.930+0000 7fca71000640 1 --2- 172.21.15.86:0/930202382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca6c0ccab0 0x7fca6c1532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:54.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.930+0000 7fca71a00640 1 -- 172.21.15.86:0/930202382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca6c0ccab0 msgr2=0x7fca6c1532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:54.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.930+0000 7fca71a00640 1 --2- 172.21.15.86:0/930202382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca6c0ccab0 0x7fca6c1532c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:54.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.930+0000 7fca71a00640 1 -- 172.21.15.86:0/930202382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca6c0cf9d0 msgr2=0x7fca6c153800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:54.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.930+0000 7fca71a00640 1 --2- 172.21.15.86:0/930202382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca6c0cf9d0 0x7fca6c153800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:54.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.930+0000 7fca71a00640 1 -- 172.21.15.86:0/930202382 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6c154b40 con 0x7fca6c0b79c0 2024-04-25T14:17:54.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.930+0000 7fca72400640 1 --2- 172.21.15.86:0/930202382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca6c0cf9d0 0x7fca6c153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:17:54.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.930+0000 7fca71a00640 1 --2- 172.21.15.86:0/930202382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca6c0b79c0 0x7fca6c15cb10 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7fca6000e990 tx=0x7fca6000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:54.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.930+0000 7fca69600640 1 -- 172.21.15.86:0/930202382 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca600059a0 con 0x7fca6c0b79c0 2024-04-25T14:17:54.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.930+0000 7fca69600640 1 -- 172.21.15.86:0/930202382 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca60005b40 con 0x7fca6c0b79c0 2024-04-25T14:17:54.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.930+0000 7fca74400640 1 -- 172.21.15.86:0/930202382 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca6c0c9690 con 0x7fca6c0b79c0 2024-04-25T14:17:54.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.930+0000 7fca69600640 1 -- 172.21.15.86:0/930202382 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6000c9e0 con 0x7fca6c0b79c0 2024-04-25T14:17:54.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.930+0000 7fca74400640 1 -- 172.21.15.86:0/930202382 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca6c0ca380 con 0x7fca6c0b79c0 2024-04-25T14:17:54.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.934+0000 7fca74400640 1 -- 172.21.15.86:0/930202382 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca6c0a04c0 con 0x7fca6c0b79c0 2024-04-25T14:17:54.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.934+0000 7fca69600640 1 -- 172.21.15.86:0/930202382 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 85) ==== 93207+0+0 (secure 0 0 0) 0x7fca6000cb80 con 0x7fca6c0b79c0 2024-04-25T14:17:54.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.934+0000 7fca69600640 1 --2- 172.21.15.86:0/930202382 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fca4c06f430 0x7fca4c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:54.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.934+0000 7fca71000640 1 -- 172.21.15.86:0/930202382 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fca4c06f430 msgr2=0x7fca4c071940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2318291413 2024-04-25T14:17:54.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.934+0000 7fca69600640 1 -- 172.21.15.86:0/930202382 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fca6008f800 con 0x7fca6c0b79c0 2024-04-25T14:17:54.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.934+0000 7fca71000640 1 --2- 172.21.15.86:0/930202382 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fca4c06f430 0x7fca4c071940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:17:54.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:54.938+0000 7fca69600640 1 -- 172.21.15.86:0/930202382 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fca60017020 con 0x7fca6c0b79c0 2024-04-25T14:17:54.989 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:54.990+0000 7f228a9e8180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:17:55.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.110+0000 7fca74400640 1 -- 172.21.15.86:0/930202382 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fca6c0d2f90 con 0x7fca6c0b79c0 2024-04-25T14:17:55.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.110+0000 7fca69600640 1 -- 172.21.15.86:0/930202382 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v156) ==== 80+0+467 (secure 0 0 0) 0x7fca600643f0 con 0x7fca6c0b79c0 2024-04-25T14:17:55.110 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2043624430 2024-04-26T14:13:51.718341+0000 2024-04-25T14:17:55.110 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/955458502 2024-04-26T14:13:51.718341+0000 2024-04-25T14:17:55.110 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1302282677 2024-04-26T14:13:51.718341+0000 2024-04-25T14:17:55.110 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6833/505730993 2024-04-26T14:13:51.718341+0000 2024-04-25T14:17:55.110 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6832/505730993 2024-04-26T14:13:51.718341+0000 2024-04-25T14:17:55.110 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3455088811 2024-04-26T14:13:51.718341+0000 2024-04-25T14:17:55.110 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3537592233 2024-04-26T14:13:51.718341+0000 2024-04-25T14:17:55.111 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2705096450 2024-04-26T14:13:51.718341+0000 2024-04-25T14:17:55.112 INFO:teuthology.orchestra.run.smithi086.stderr:listed 8 entries 2024-04-25T14:17:55.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.114+0000 7fca74400640 1 -- 172.21.15.86:0/930202382 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fca4c06f430 msgr2=0x7fca4c071940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:17:55.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.114+0000 7fca74400640 1 --2- 172.21.15.86:0/930202382 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fca4c06f430 0x7fca4c071940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:55.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.114+0000 7fca74400640 1 -- 172.21.15.86:0/930202382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca6c0b79c0 msgr2=0x7fca6c15cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:55.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.114+0000 7fca74400640 1 --2- 172.21.15.86:0/930202382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca6c0b79c0 0x7fca6c15cb10 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7fca6000e990 tx=0x7fca6000ee50 comp rx=0 tx=0).stop 2024-04-25T14:17:55.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.114+0000 7fca74400640 1 -- 172.21.15.86:0/930202382 shutdown_connections 2024-04-25T14:17:55.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.114+0000 7fca74400640 1 --2- 172.21.15.86:0/930202382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca6c0cf9d0 0x7fca6c153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:55.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.114+0000 7fca74400640 1 --2- 172.21.15.86:0/930202382 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fca4c06f430 0x7fca4c071940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:55.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.114+0000 7fca74400640 1 --2- 172.21.15.86:0/930202382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca6c0ccab0 0x7fca6c1532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:55.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.114+0000 7fca74400640 1 --2- 172.21.15.86:0/930202382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca6c0b79c0 0x7fca6c15cb10 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:55.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.114+0000 7fca74400640 1 -- 172.21.15.86:0/930202382 >> 172.21.15.86:0/930202382 conn(0x7fca6c09ba30 msgr2=0x7fca6c0af140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:55.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.114+0000 7fca74400640 1 -- 172.21.15.86:0/930202382 shutdown_connections 2024-04-25T14:17:55.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.114+0000 7fca74400640 1 -- 172.21.15.86:0/930202382 wait complete. 2024-04-25T14:17:55.135 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T14:17:55.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:17:55.135 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T14:17:55.138 INFO:tasks.ceph.mgr.x.smithi155.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:17:55.138 INFO:tasks.ceph.mgr.x.smithi155.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:17:55.138 INFO:tasks.ceph.mgr.x.smithi155.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:17:55.147 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:17:55.147+0000 7f0fb926d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:17:55.228 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T14:17:55.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:17:55.229 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T14:17:55.260 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:17:55.261 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:17:55.261 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:17:55.269 INFO:tasks.ceph.mgr.y.smithi086.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T14:17:55.269 INFO:tasks.ceph.mgr.y.smithi086.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T14:17:55.269 INFO:tasks.ceph.mgr.y.smithi086.stderr: from numpy import show_config as show_numpy_config 2024-04-25T14:17:55.271 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:17:55.271 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:17:55.278 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:55.282+0000 7f228a9e8180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:17:55.327 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:17:55.335 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:17:55.339+0000 7f0fb926d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:17:55.434 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:55.434+0000 7f228a9e8180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:17:55.462 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:17:55.463+0000 7f0fb926d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:17:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.478+0000 7ff989e00640 1 Processor -- start 2024-04-25T14:17:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.478+0000 7ff989e00640 1 -- start start 2024-04-25T14:17:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.478+0000 7ff989e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff9840d2c80 0x7ff98415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.478+0000 7ff989e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff984070e20 0x7ff984071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.478+0000 7ff989e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff984071770 con 0x7ff9840722f0 2024-04-25T14:17:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.478+0000 7ff989e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff9840ce720 con 0x7ff9840d2c80 2024-04-25T14:17:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.478+0000 7ff989e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff9840cd8d0 con 0x7ff984070e20 2024-04-25T14:17:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.478+0000 7ff983400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff9840722f0 0x7ff984072700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59164/0 (socket says 172.21.15.86:59164) 2024-04-25T14:17:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.478+0000 7ff983400640 1 -- 172.21.15.86:0/467006158 learned_addr learned my addr 172.21.15.86:0/467006158 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:17:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.478+0000 7ff982a00640 1 --2- 172.21.15.86:0/467006158 >> v2:172.21.15.86:3300/0 conn(0x7ff9840d2c80 0x7ff98415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.482+0000 7ff97be00640 1 --2- 172.21.15.86:0/467006158 >> v2:172.21.15.155:3300/0 conn(0x7ff984070e20 0x7ff984071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.482+0000 7ff982a00640 1 -- 172.21.15.86:0/467006158 >> v2:172.21.15.155:3300/0 conn(0x7ff984070e20 msgr2=0x7ff984071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.482+0000 7ff982a00640 1 --2- 172.21.15.86:0/467006158 >> v2:172.21.15.155:3300/0 conn(0x7ff984070e20 0x7ff984071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.482+0000 7ff982a00640 1 -- 172.21.15.86:0/467006158 >> v1:172.21.15.86:6789/0 conn(0x7ff9840722f0 legacy=0x7ff984072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.482+0000 7ff982a00640 1 -- 172.21.15.86:0/467006158 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9840ce970 con 0x7ff9840d2c80 2024-04-25T14:17:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.482+0000 7ff982000640 1 -- 172.21.15.86:0/467006158 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2023701487 0 0) 0x7ff97c0014a0 con 0x7ff9840722f0 2024-04-25T14:17:55.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.482+0000 7ff982a00640 1 --2- 172.21.15.86:0/467006158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9840d2c80 0x7ff98415dc90 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7ff9840b6070 tx=0x7ff97401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c37cb7915421759 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:55.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.482+0000 7ff982000640 1 -- 172.21.15.86:0/467006158 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9740b5070 con 0x7ff9840d2c80 2024-04-25T14:17:55.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.482+0000 7ff982000640 1 -- 172.21.15.86:0/467006158 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9740a8970 con 0x7ff9840d2c80 2024-04-25T14:17:55.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.482+0000 7ff982000640 1 -- 172.21.15.86:0/467006158 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9740a8d40 con 0x7ff9840d2c80 2024-04-25T14:17:55.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.486+0000 7ff989e00640 1 -- 172.21.15.86:0/467006158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9840d2c80 msgr2=0x7ff98415dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:55.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.486+0000 7ff989e00640 1 --2- 172.21.15.86:0/467006158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9840d2c80 0x7ff98415dc90 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7ff9840b6070 tx=0x7ff97401ec70 comp rx=0 tx=0).stop 2024-04-25T14:17:55.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.486+0000 7ff989e00640 1 -- 172.21.15.86:0/467006158 shutdown_connections 2024-04-25T14:17:55.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.486+0000 7ff989e00640 1 --2- 172.21.15.86:0/467006158 >> v2:172.21.15.155:3300/0 conn(0x7ff984070e20 0x7ff984071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:55.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.486+0000 7ff989e00640 1 --2- 172.21.15.86:0/467006158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9840d2c80 0x7ff98415dc90 unknown :-1 s=CLOSED pgs=2419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:55.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.486+0000 7ff989e00640 1 -- 172.21.15.86:0/467006158 >> 172.21.15.86:0/467006158 conn(0x7ff9840cffc0 msgr2=0x7ff9840d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:55.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.486+0000 7ff989e00640 1 -- 172.21.15.86:0/467006158 shutdown_connections 2024-04-25T14:17:55.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.486+0000 7ff989e00640 1 -- 172.21.15.86:0/467006158 wait complete. 2024-04-25T14:17:55.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.486+0000 7ff989e00640 1 Processor -- start 2024-04-25T14:17:55.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.486+0000 7ff989e00640 1 -- start start 2024-04-25T14:17:55.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.490+0000 7ff989e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff984070e20 0x7ff9841e54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:55.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.490+0000 7ff989e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9840722f0 0x7ff9840c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:55.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.490+0000 7ff989e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9840d2c80 0x7ff98415c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:55.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.490+0000 7ff989e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff9840d21a0 con 0x7ff9840722f0 2024-04-25T14:17:55.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.490+0000 7ff989e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff9840d1ec0 con 0x7ff984070e20 2024-04-25T14:17:55.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.490+0000 7ff989e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff9840d2480 con 0x7ff9840d2c80 2024-04-25T14:17:55.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.490+0000 7ff97be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9840722f0 0x7ff9840c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:55.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.490+0000 7ff97be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9840722f0 0x7ff9840c91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59068/0 (socket says 172.21.15.86:59068) 2024-04-25T14:17:55.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.490+0000 7ff97be00640 1 -- 172.21.15.86:0/2562844222 learned_addr learned my addr 172.21.15.86:0/2562844222 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:55.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.490+0000 7ff982a00640 1 --2- 172.21.15.86:0/2562844222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff984070e20 0x7ff9841e54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:55.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.490+0000 7ff983400640 1 --2- 172.21.15.86:0/2562844222 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9840d2c80 0x7ff98415c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:55.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.490+0000 7ff97be00640 1 -- 172.21.15.86:0/2562844222 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9840d2c80 msgr2=0x7ff98415c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:55.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.490+0000 7ff97be00640 1 --2- 172.21.15.86:0/2562844222 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9840d2c80 0x7ff98415c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:55.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.490+0000 7ff97be00640 1 -- 172.21.15.86:0/2562844222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff984070e20 msgr2=0x7ff9841e54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:55.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.490+0000 7ff97be00640 1 --2- 172.21.15.86:0/2562844222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff984070e20 0x7ff9841e54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:55.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.490+0000 7ff97be00640 1 -- 172.21.15.86:0/2562844222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9840c9710 con 0x7ff9840722f0 2024-04-25T14:17:55.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.490+0000 7ff97be00640 1 --2- 172.21.15.86:0/2562844222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9840722f0 0x7ff9840c91d0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00e890 tx=0x7ff96c00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:55.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.490+0000 7ff97b400640 1 -- 172.21.15.86:0/2562844222 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c017070 con 0x7ff9840722f0 2024-04-25T14:17:55.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.490+0000 7ff97b400640 1 -- 172.21.15.86:0/2562844222 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff96c00c940 con 0x7ff9840722f0 2024-04-25T14:17:55.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.490+0000 7ff97b400640 1 -- 172.21.15.86:0/2562844222 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c00cd10 con 0x7ff9840722f0 2024-04-25T14:17:55.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.490+0000 7ff989e00640 1 -- 172.21.15.86:0/2562844222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff984152500 con 0x7ff9840722f0 2024-04-25T14:17:55.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.490+0000 7ff989e00640 1 -- 172.21.15.86:0/2562844222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff984152ac0 con 0x7ff9840722f0 2024-04-25T14:17:55.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.490+0000 7ff989e00640 1 -- 172.21.15.86:0/2562844222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9840720e0 con 0x7ff9840722f0 2024-04-25T14:17:55.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.494+0000 7ff97b400640 1 -- 172.21.15.86:0/2562844222 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 85) ==== 93207+0+0 (secure 0 0 0) 0x7ff96c01a070 con 0x7ff9840722f0 2024-04-25T14:17:55.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.494+0000 7ff97b400640 1 --2- 172.21.15.86:0/2562844222 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ff96006f430 0x7ff960071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:55.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.494+0000 7ff982a00640 1 -- 172.21.15.86:0/2562844222 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ff96006f430 msgr2=0x7ff960071940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2318291413 2024-04-25T14:17:55.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.494+0000 7ff97b400640 1 -- 172.21.15.86:0/2562844222 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7ff96c08f8c0 con 0x7ff9840722f0 2024-04-25T14:17:55.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.494+0000 7ff982a00640 1 --2- 172.21.15.86:0/2562844222 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ff96006f430 0x7ff960071940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:17:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.502+0000 7ff97b400640 1 -- 172.21.15.86:0/2562844222 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff96c05bdf0 con 0x7ff9840722f0 2024-04-25T14:17:55.535 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:17:55.539+0000 7f0fb926d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:17:55.569 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:55.570+0000 7f228a9e8180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:17:55.642 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:55.642+0000 7f228a9e8180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:17:55.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.662+0000 7ff989e00640 1 -- 172.21.15.86:0/2562844222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff984076bf0 con 0x7ff9840722f0 2024-04-25T14:17:55.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.666+0000 7ff97b400640 1 -- 172.21.15.86:0/2562844222 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+17382 (secure 0 0 0) 0x7ff96c064430 con 0x7ff9840722f0 2024-04-25T14:17:55.663 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:55.663 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":156,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:13:51.718378+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:13:32.006877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T14:13:32.574088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/2043624430":"2024-04-26T14:13:51.718341+0000","172.21.15.86:0/955458502":"2024-04-26T14:13:51.718341+0000","172.21.15.86:0/1302282677":"2024-04-26T14:13:51.718341+0000","172.21.15.86:6833/505730993":"2024-04-26T14:13:51.718341+0000","172.21.15.86:6832/505730993":"2024-04-26T14:13:51.718341+0000","172.21.15.86:0/3455088811":"2024-04-26T14:13:51.718341+0000","172.21.15.86:0/3537592233":"2024-04-26T14:13:51.718341+0000","172.21.15.86:0/2705096450":"2024-04-26T14:13:51.718341+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:17:55.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.666+0000 7ff989e00640 1 -- 172.21.15.86:0/2562844222 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ff96006f430 msgr2=0x7ff960071940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:17:55.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.666+0000 7ff989e00640 1 --2- 172.21.15.86:0/2562844222 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ff96006f430 0x7ff960071940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:55.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.666+0000 7ff989e00640 1 -- 172.21.15.86:0/2562844222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9840722f0 msgr2=0x7ff9840c91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:55.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.666+0000 7ff989e00640 1 --2- 172.21.15.86:0/2562844222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9840722f0 0x7ff9840c91d0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00e890 tx=0x7ff96c00ed50 comp rx=0 tx=0).stop 2024-04-25T14:17:55.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.666+0000 7ff989e00640 1 -- 172.21.15.86:0/2562844222 shutdown_connections 2024-04-25T14:17:55.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.666+0000 7ff989e00640 1 --2- 172.21.15.86:0/2562844222 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7ff96006f430 0x7ff960071940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:55.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.666+0000 7ff989e00640 1 --2- 172.21.15.86:0/2562844222 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9840d2c80 0x7ff98415c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:55.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.666+0000 7ff989e00640 1 --2- 172.21.15.86:0/2562844222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9840722f0 0x7ff9840c91d0 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:55.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.666+0000 7ff989e00640 1 --2- 172.21.15.86:0/2562844222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff984070e20 0x7ff9841e54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:55.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.666+0000 7ff989e00640 1 -- 172.21.15.86:0/2562844222 >> 172.21.15.86:0/2562844222 conn(0x7ff9840cffc0 msgr2=0x7ff9840ccaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:55.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.666+0000 7ff989e00640 1 -- 172.21.15.86:0/2562844222 shutdown_connections 2024-04-25T14:17:55.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.666+0000 7ff989e00640 1 -- 172.21.15.86:0/2562844222 wait complete. 2024-04-25T14:17:55.684 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:17:55.687+0000 7f0fb926d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:17:55.686 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:13:32.006877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '154', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:13:32.574088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:17:55.686 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:17:55.800 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:55.802+0000 7f228a9e8180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:17:55.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.842+0000 7fd751a00640 1 Processor -- start 2024-04-25T14:17:55.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.842+0000 7fd751a00640 1 -- start start 2024-04-25T14:17:55.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.842+0000 7fd751a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd74c0cdd70 0x7fd74c0ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:55.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.842+0000 7fd751a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd74c070e20 0x7fd74c071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:55.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.842+0000 7fd751a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd74c06e700 0x7fd74c06eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:55.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.842+0000 7fd751a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd74c071770 con 0x7fd74c0cdd70 2024-04-25T14:17:55.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.842+0000 7fd751a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd74c06f050 con 0x7fd74c06e700 2024-04-25T14:17:55.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.842+0000 7fd751a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd74c06d940 con 0x7fd74c070e20 2024-04-25T14:17:55.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.842+0000 7fd74aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd74c0cdd70 0x7fd74c0ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:55.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.842+0000 7fd74aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd74c0cdd70 0x7fd74c0ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59070/0 (socket says 172.21.15.86:59070) 2024-04-25T14:17:55.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.842+0000 7fd74aa00640 1 -- 172.21.15.86:0/4130591635 learned_addr learned my addr 172.21.15.86:0/4130591635 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:55.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.842+0000 7fd74b400640 1 --2- 172.21.15.86:0/4130591635 >> v2:172.21.15.155:3300/0 conn(0x7fd74c06e700 0x7fd74c06eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:55.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.842+0000 7fd74a000640 1 --2- 172.21.15.86:0/4130591635 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd74c070e20 0x7fd74c071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:55.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.842+0000 7fd74aa00640 1 -- 172.21.15.86:0/4130591635 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd74c070e20 msgr2=0x7fd74c071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:55.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.842+0000 7fd74aa00640 1 --2- 172.21.15.86:0/4130591635 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd74c070e20 0x7fd74c071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:55.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.842+0000 7fd74aa00640 1 -- 172.21.15.86:0/4130591635 >> v2:172.21.15.155:3300/0 conn(0x7fd74c06e700 msgr2=0x7fd74c06eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:55.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.842+0000 7fd74aa00640 1 --2- 172.21.15.86:0/4130591635 >> v2:172.21.15.155:3300/0 conn(0x7fd74c06e700 0x7fd74c06eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:55.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.842+0000 7fd74aa00640 1 -- 172.21.15.86:0/4130591635 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd74c06dc20 con 0x7fd74c0cdd70 2024-04-25T14:17:55.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.842+0000 7fd74b400640 1 --2- 172.21.15.86:0/4130591635 >> v2:172.21.15.155:3300/0 conn(0x7fd74c06e700 0x7fd74c06eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:17:55.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.846+0000 7fd74aa00640 1 --2- 172.21.15.86:0/4130591635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd74c0cdd70 0x7fd74c0ce180 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7fd74c071b00 tx=0x7fd73c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b26dd99bca44735 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:55.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.846+0000 7fd749600640 1 -- 172.21.15.86:0/4130591635 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c0b0070 con 0x7fd74c0cdd70 2024-04-25T14:17:55.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.846+0000 7fd749600640 1 -- 172.21.15.86:0/4130591635 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd73c0ab600 con 0x7fd74c0cdd70 2024-04-25T14:17:55.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.846+0000 7fd749600640 1 -- 172.21.15.86:0/4130591635 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c0ab9d0 con 0x7fd74c0cdd70 2024-04-25T14:17:55.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.846+0000 7fd751a00640 1 -- 172.21.15.86:0/4130591635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd74c0cdd70 msgr2=0x7fd74c0ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:55.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.846+0000 7fd751a00640 1 --2- 172.21.15.86:0/4130591635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd74c0cdd70 0x7fd74c0ce180 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7fd74c071b00 tx=0x7fd73c01c490 comp rx=0 tx=0).stop 2024-04-25T14:17:55.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.846+0000 7fd751a00640 1 -- 172.21.15.86:0/4130591635 shutdown_connections 2024-04-25T14:17:55.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.846+0000 7fd751a00640 1 --2- 172.21.15.86:0/4130591635 >> v2:172.21.15.155:3300/0 conn(0x7fd74c06e700 0x7fd74c06eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:55.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.846+0000 7fd751a00640 1 --2- 172.21.15.86:0/4130591635 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd74c070e20 0x7fd74c071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:55.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.846+0000 7fd751a00640 1 --2- 172.21.15.86:0/4130591635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd74c0cdd70 0x7fd74c0ce180 unknown :-1 s=CLOSED pgs=2421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:55.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.846+0000 7fd751a00640 1 -- 172.21.15.86:0/4130591635 >> 172.21.15.86:0/4130591635 conn(0x7fd74c0cffc0 msgr2=0x7fd74c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:55.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.846+0000 7fd751a00640 1 -- 172.21.15.86:0/4130591635 shutdown_connections 2024-04-25T14:17:55.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.846+0000 7fd751a00640 1 -- 172.21.15.86:0/4130591635 wait complete. 2024-04-25T14:17:55.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.850+0000 7fd751a00640 1 Processor -- start 2024-04-25T14:17:55.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.850+0000 7fd751a00640 1 -- start start 2024-04-25T14:17:55.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.850+0000 7fd751a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd74c06e700 0x7fd74c15a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:55.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.850+0000 7fd751a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd74c070e20 0x7fd74c0c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:55.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.850+0000 7fd751a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd74c0cdd70 0x7fd74c0c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:55.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.850+0000 7fd751a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd74c1642f0 con 0x7fd74c06e700 2024-04-25T14:17:55.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.850+0000 7fd751a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd74c164170 con 0x7fd74c070e20 2024-04-25T14:17:55.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.850+0000 7fd751a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd74c164470 con 0x7fd74c0cdd70 2024-04-25T14:17:55.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.850+0000 7fd74aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd74c06e700 0x7fd74c15a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:55.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.850+0000 7fd74aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd74c06e700 0x7fd74c15a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59086/0 (socket says 172.21.15.86:59086) 2024-04-25T14:17:55.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.850+0000 7fd74aa00640 1 -- 172.21.15.86:0/1999407860 learned_addr learned my addr 172.21.15.86:0/1999407860 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:55.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.850+0000 7fd74b400640 1 --2- 172.21.15.86:0/1999407860 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd74c0cdd70 0x7fd74c0c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:55.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.850+0000 7fd74a000640 1 --2- 172.21.15.86:0/1999407860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd74c070e20 0x7fd74c0c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:55.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.850+0000 7fd74aa00640 1 -- 172.21.15.86:0/1999407860 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd74c0cdd70 msgr2=0x7fd74c0c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:55.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.850+0000 7fd74aa00640 1 --2- 172.21.15.86:0/1999407860 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd74c0cdd70 0x7fd74c0c9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:55.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.850+0000 7fd74aa00640 1 -- 172.21.15.86:0/1999407860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd74c070e20 msgr2=0x7fd74c0c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:55.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.850+0000 7fd74aa00640 1 --2- 172.21.15.86:0/1999407860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd74c070e20 0x7fd74c0c92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:55.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.850+0000 7fd74aa00640 1 -- 172.21.15.86:0/1999407860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd74c0c7390 con 0x7fd74c06e700 2024-04-25T14:17:55.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.854+0000 7fd74aa00640 1 --2- 172.21.15.86:0/1999407860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd74c06e700 0x7fd74c15a080 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7fd73c01cd30 tx=0x7fd73c019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:55.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.854+0000 7fd742a00640 1 -- 172.21.15.86:0/1999407860 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c0b9070 con 0x7fd74c06e700 2024-04-25T14:17:55.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.854+0000 7fd742a00640 1 -- 172.21.15.86:0/1999407860 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd73c0b4420 con 0x7fd74c06e700 2024-04-25T14:17:55.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.854+0000 7fd742a00640 1 -- 172.21.15.86:0/1999407860 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c0b0040 con 0x7fd74c06e700 2024-04-25T14:17:55.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.854+0000 7fd751a00640 1 -- 172.21.15.86:0/1999407860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd74c0c7670 con 0x7fd74c06e700 2024-04-25T14:17:55.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.854+0000 7fd751a00640 1 -- 172.21.15.86:0/1999407860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd74c0c7c30 con 0x7fd74c06e700 2024-04-25T14:17:55.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.854+0000 7fd751a00640 1 -- 172.21.15.86:0/1999407860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd74c0720e0 con 0x7fd74c06e700 2024-04-25T14:17:55.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.854+0000 7fd742a00640 1 -- 172.21.15.86:0/1999407860 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 85) ==== 93207+0+0 (secure 0 0 0) 0x7fd73c0bc070 con 0x7fd74c06e700 2024-04-25T14:17:55.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.858+0000 7fd742a00640 1 --2- 172.21.15.86:0/1999407860 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd72c06f430 0x7fd72c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:55.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.858+0000 7fd74a000640 1 -- 172.21.15.86:0/1999407860 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd72c06f430 msgr2=0x7fd72c071940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2318291413 2024-04-25T14:17:55.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.858+0000 7fd742a00640 1 -- 172.21.15.86:0/1999407860 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fd73c128510 con 0x7fd74c06e700 2024-04-25T14:17:55.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.858+0000 7fd74a000640 1 --2- 172.21.15.86:0/1999407860 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd72c06f430 0x7fd72c071940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:17:55.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:55.862+0000 7fd742a00640 1 -- 172.21.15.86:0/1999407860 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd73c016110 con 0x7fd74c06e700 2024-04-25T14:17:56.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.022+0000 7fd751a00640 1 -- 172.21.15.86:0/1999407860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd74c163690 con 0x7fd74c06e700 2024-04-25T14:17:56.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.022+0000 7fd742a00640 1 -- 172.21.15.86:0/1999407860 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v156) ==== 93+0+10 (secure 0 0 0) 0x7fd73c0f4a30 con 0x7fd74c06e700 2024-04-25T14:17:56.022 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:17:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.026+0000 7fd751a00640 1 -- 172.21.15.86:0/1999407860 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd72c06f430 msgr2=0x7fd72c071940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:17:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.026+0000 7fd751a00640 1 --2- 172.21.15.86:0/1999407860 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd72c06f430 0x7fd72c071940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.026+0000 7fd751a00640 1 -- 172.21.15.86:0/1999407860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd74c06e700 msgr2=0x7fd74c15a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.026+0000 7fd751a00640 1 --2- 172.21.15.86:0/1999407860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd74c06e700 0x7fd74c15a080 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7fd73c01cd30 tx=0x7fd73c019230 comp rx=0 tx=0).stop 2024-04-25T14:17:56.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.026+0000 7fd751a00640 1 -- 172.21.15.86:0/1999407860 shutdown_connections 2024-04-25T14:17:56.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.026+0000 7fd751a00640 1 --2- 172.21.15.86:0/1999407860 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fd72c06f430 0x7fd72c071940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.026+0000 7fd751a00640 1 --2- 172.21.15.86:0/1999407860 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd74c0cdd70 0x7fd74c0c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.026+0000 7fd751a00640 1 --2- 172.21.15.86:0/1999407860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd74c070e20 0x7fd74c0c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.026+0000 7fd751a00640 1 --2- 172.21.15.86:0/1999407860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd74c06e700 0x7fd74c15a080 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.026+0000 7fd751a00640 1 -- 172.21.15.86:0/1999407860 >> 172.21.15.86:0/1999407860 conn(0x7fd74c0cffc0 msgr2=0x7fd74c0b3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:56.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.026+0000 7fd751a00640 1 -- 172.21.15.86:0/1999407860 shutdown_connections 2024-04-25T14:17:56.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.026+0000 7fd751a00640 1 -- 172.21.15.86:0/1999407860 wait complete. 2024-04-25T14:17:56.046 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:17:56.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.206+0000 7fae75800640 1 Processor -- start 2024-04-25T14:17:56.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.206+0000 7fae75800640 1 -- start start 2024-04-25T14:17:56.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.206+0000 7fae75800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fae70070330 0x7fae700d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:56.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.206+0000 7fae75800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae700ce790 0x7fae7006f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:56.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.206+0000 7fae75800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae700d2030 con 0x7fae700d1c20 2024-04-25T14:17:56.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.206+0000 7fae75800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fae7015e5c0 con 0x7fae70070330 2024-04-25T14:17:56.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.206+0000 7fae75800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fae7015e8a0 con 0x7fae700ce790 2024-04-25T14:17:56.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.206+0000 7fae6e000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fae700d1c20 0x7fae7006ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59168/0 (socket says 172.21.15.86:59168) 2024-04-25T14:17:56.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.206+0000 7fae6e000640 1 -- 172.21.15.86:0/2988123444 learned_addr learned my addr 172.21.15.86:0/2988123444 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:17:56.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.206+0000 7fae6ea00640 1 --2- 172.21.15.86:0/2988123444 >> v2:172.21.15.86:3300/0 conn(0x7fae70070330 0x7fae700d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:56.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.206+0000 7fae6f400640 1 --2- 172.21.15.86:0/2988123444 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae700ce790 0x7fae7006f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:56.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.206+0000 7fae6d600640 1 -- 172.21.15.86:0/2988123444 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3408362623 0 0) 0x7fae5c001120 con 0x7fae700d1c20 2024-04-25T14:17:56.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.206+0000 7fae6ea00640 1 -- 172.21.15.86:0/2988123444 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae700ce790 msgr2=0x7fae7006f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:56.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.206+0000 7fae6ea00640 1 --2- 172.21.15.86:0/2988123444 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae700ce790 0x7fae7006f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.206+0000 7fae6ea00640 1 -- 172.21.15.86:0/2988123444 >> v1:172.21.15.86:6789/0 conn(0x7fae700d1c20 legacy=0x7fae7006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:56.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.206+0000 7fae6ea00640 1 -- 172.21.15.86:0/2988123444 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7015eb80 con 0x7fae70070330 2024-04-25T14:17:56.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.206+0000 7fae6ea00640 1 --2- 172.21.15.86:0/2988123444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae70070330 0x7fae700d16e0 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7fae600097c0 tx=0x7fae6001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3042f30436723a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:56.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.206+0000 7fae6d600640 1 -- 172.21.15.86:0/2988123444 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae600b0070 con 0x7fae70070330 2024-04-25T14:17:56.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.206+0000 7fae6d600640 1 -- 172.21.15.86:0/2988123444 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae600ab600 con 0x7fae70070330 2024-04-25T14:17:56.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.206+0000 7fae6d600640 1 -- 172.21.15.86:0/2988123444 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae600ab9d0 con 0x7fae70070330 2024-04-25T14:17:56.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.206+0000 7fae75800640 1 -- 172.21.15.86:0/2988123444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae70070330 msgr2=0x7fae700d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:56.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.206+0000 7fae75800640 1 --2- 172.21.15.86:0/2988123444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae70070330 0x7fae700d16e0 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7fae600097c0 tx=0x7fae6001c410 comp rx=0 tx=0).stop 2024-04-25T14:17:56.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.206+0000 7fae75800640 1 -- 172.21.15.86:0/2988123444 shutdown_connections 2024-04-25T14:17:56.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.206+0000 7fae75800640 1 --2- 172.21.15.86:0/2988123444 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae700ce790 0x7fae7006f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.206+0000 7fae75800640 1 --2- 172.21.15.86:0/2988123444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae70070330 0x7fae700d16e0 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.206+0000 7fae75800640 1 -- 172.21.15.86:0/2988123444 >> 172.21.15.86:0/2988123444 conn(0x7fae700b4650 msgr2=0x7fae700b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:56.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.206+0000 7fae75800640 1 -- 172.21.15.86:0/2988123444 shutdown_connections 2024-04-25T14:17:56.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.206+0000 7fae75800640 1 -- 172.21.15.86:0/2988123444 wait complete. 2024-04-25T14:17:56.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.210+0000 7fae75800640 1 Processor -- start 2024-04-25T14:17:56.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.210+0000 7fae75800640 1 -- start start 2024-04-25T14:17:56.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.210+0000 7fae75800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae70070330 0x7fae700b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:56.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.210+0000 7fae75800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae700ce790 0x7fae70156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:56.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.210+0000 7fae75800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae700d1c20 0x7fae700be1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:56.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.210+0000 7fae75800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae700708c0 con 0x7fae700ce790 2024-04-25T14:17:56.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.210+0000 7fae75800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fae70070740 con 0x7fae700d1c20 2024-04-25T14:17:56.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.210+0000 7fae75800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fae7016afc0 con 0x7fae70070330 2024-04-25T14:17:56.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.210+0000 7fae6e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae700ce790 0x7fae70156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:56.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.210+0000 7fae6e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae700ce790 0x7fae70156ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59106/0 (socket says 172.21.15.86:59106) 2024-04-25T14:17:56.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.210+0000 7fae6e000640 1 -- 172.21.15.86:0/704530329 learned_addr learned my addr 172.21.15.86:0/704530329 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:56.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.210+0000 7fae6ea00640 1 --2- 172.21.15.86:0/704530329 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae70070330 0x7fae700b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:56.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.210+0000 7fae6f400640 1 --2- 172.21.15.86:0/704530329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae700d1c20 0x7fae700be1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:56.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.210+0000 7fae6e000640 1 -- 172.21.15.86:0/704530329 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae70070330 msgr2=0x7fae700b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:56.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.210+0000 7fae6e000640 1 --2- 172.21.15.86:0/704530329 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae70070330 0x7fae700b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.210+0000 7fae6e000640 1 -- 172.21.15.86:0/704530329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae700d1c20 msgr2=0x7fae700be1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:56.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.210+0000 7fae6e000640 1 --2- 172.21.15.86:0/704530329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae700d1c20 0x7fae700be1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.210+0000 7fae6e000640 1 -- 172.21.15.86:0/704530329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae70157310 con 0x7fae700ce790 2024-04-25T14:17:56.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.214+0000 7fae6e000640 1 --2- 172.21.15.86:0/704530329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae700ce790 0x7fae70156ae0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00c3e0 tx=0x7fae5c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:56.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.214+0000 7fae66a00640 1 -- 172.21.15.86:0/704530329 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c0074e0 con 0x7fae700ce790 2024-04-25T14:17:56.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.214+0000 7fae66a00640 1 -- 172.21.15.86:0/704530329 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae5c0047f0 con 0x7fae700ce790 2024-04-25T14:17:56.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.214+0000 7fae75800640 1 -- 172.21.15.86:0/704530329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae701575f0 con 0x7fae700ce790 2024-04-25T14:17:56.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.214+0000 7fae66a00640 1 -- 172.21.15.86:0/704530329 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c004be0 con 0x7fae700ce790 2024-04-25T14:17:56.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.214+0000 7fae75800640 1 -- 172.21.15.86:0/704530329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae70157bb0 con 0x7fae700ce790 2024-04-25T14:17:56.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.214+0000 7fae75800640 1 -- 172.21.15.86:0/704530329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae70069d00 con 0x7fae700ce790 2024-04-25T14:17:56.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.214+0000 7fae66a00640 1 -- 172.21.15.86:0/704530329 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 85) ==== 93207+0+0 (secure 0 0 0) 0x7fae5c011070 con 0x7fae700ce790 2024-04-25T14:17:56.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.218+0000 7fae66a00640 1 --2- 172.21.15.86:0/704530329 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fae5006f400 0x7fae50071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:56.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.218+0000 7fae6ea00640 1 -- 172.21.15.86:0/704530329 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fae5006f400 msgr2=0x7fae50071910 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2318291413 2024-04-25T14:17:56.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.218+0000 7fae66a00640 1 -- 172.21.15.86:0/704530329 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fae5c08f130 con 0x7fae700ce790 2024-04-25T14:17:56.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.218+0000 7fae6ea00640 1 --2- 172.21.15.86:0/704530329 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fae5006f400 0x7fae50071910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:17:56.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.222+0000 7fae66a00640 1 -- 172.21.15.86:0/704530329 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fae700d2030 con 0x7fae700ce790 2024-04-25T14:17:56.239 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:17:56.243+0000 7f0fb926d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:17:56.325 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:17:56.327+0000 7f0fb926d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:17:56.369 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:56.370+0000 7f228a9e8180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:17:56.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.378+0000 7fae75800640 1 -- 172.21.15.86:0/704530329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fae70157050 con 0x7fae700ce790 2024-04-25T14:17:56.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.378+0000 7fae66a00640 1 -- 172.21.15.86:0/704530329 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v156) ==== 104+0+11 (secure 0 0 0) 0x7fae5c05b650 con 0x7fae700ce790 2024-04-25T14:17:56.378 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:17:56.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.382+0000 7fae75800640 1 -- 172.21.15.86:0/704530329 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fae5006f400 msgr2=0x7fae50071910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:17:56.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.382+0000 7fae75800640 1 --2- 172.21.15.86:0/704530329 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fae5006f400 0x7fae50071910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.382+0000 7fae75800640 1 -- 172.21.15.86:0/704530329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae700ce790 msgr2=0x7fae70156ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:56.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.382+0000 7fae75800640 1 --2- 172.21.15.86:0/704530329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae700ce790 0x7fae70156ae0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00c3e0 tx=0x7fae5c00c8a0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.382+0000 7fae75800640 1 -- 172.21.15.86:0/704530329 shutdown_connections 2024-04-25T14:17:56.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.382+0000 7fae75800640 1 --2- 172.21.15.86:0/704530329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae700d1c20 0x7fae700be1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.382+0000 7fae75800640 1 --2- 172.21.15.86:0/704530329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae700ce790 0x7fae70156ae0 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.382+0000 7fae75800640 1 --2- 172.21.15.86:0/704530329 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fae5006f400 0x7fae50071910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.382+0000 7fae75800640 1 --2- 172.21.15.86:0/704530329 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae70070330 0x7fae700b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.382+0000 7fae75800640 1 -- 172.21.15.86:0/704530329 >> 172.21.15.86:0/704530329 conn(0x7fae700b4650 msgr2=0x7fae7015d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:56.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.382+0000 7fae75800640 1 -- 172.21.15.86:0/704530329 shutdown_connections 2024-04-25T14:17:56.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.382+0000 7fae75800640 1 -- 172.21.15.86:0/704530329 wait complete. 2024-04-25T14:17:56.401 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:17:56.458 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:56.458+0000 7f228a9e8180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:17:56.487 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:17:56.491+0000 7f0fb926d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:17:56.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.554+0000 7fc4eb400640 1 Processor -- start 2024-04-25T14:17:56.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.554+0000 7fc4eb400640 1 -- start start 2024-04-25T14:17:56.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.558+0000 7fc4eb400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc4e40d2c80 0x7fc4e415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:56.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.558+0000 7fc4eb400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc4e4070e20 0x7fc4e4071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:56.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.558+0000 7fc4eb400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4e40722f0 0x7fc4e4072700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:56.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.558+0000 7fc4eb400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc4e40ce720 con 0x7fc4e40d2c80 2024-04-25T14:17:56.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.558+0000 7fc4eb400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc4e4071770 con 0x7fc4e4070e20 2024-04-25T14:17:56.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.558+0000 7fc4eb400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc4e40cd8d0 con 0x7fc4e40722f0 2024-04-25T14:17:56.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.558+0000 7fc4e8a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc4e40d2c80 0x7fc4e415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:56.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.558+0000 7fc4e8a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc4e40d2c80 0x7fc4e415dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59114/0 (socket says 172.21.15.86:59114) 2024-04-25T14:17:56.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.558+0000 7fc4e8a00640 1 -- 172.21.15.86:0/1751010575 learned_addr learned my addr 172.21.15.86:0/1751010575 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:56.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.558+0000 7fc4e3e00640 1 --2- 172.21.15.86:0/1751010575 >> v2:172.21.15.155:3300/0 conn(0x7fc4e4070e20 0x7fc4e4071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:56.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.558+0000 7fc4e9400640 1 --2- 172.21.15.86:0/1751010575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4e40722f0 0x7fc4e4072700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:56.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.558+0000 7fc4e8a00640 1 -- 172.21.15.86:0/1751010575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4e40722f0 msgr2=0x7fc4e4072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:56.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.558+0000 7fc4e8a00640 1 --2- 172.21.15.86:0/1751010575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4e40722f0 0x7fc4e4072700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.558+0000 7fc4e8a00640 1 -- 172.21.15.86:0/1751010575 >> v2:172.21.15.155:3300/0 conn(0x7fc4e4070e20 msgr2=0x7fc4e4071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:56.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.558+0000 7fc4e8a00640 1 --2- 172.21.15.86:0/1751010575 >> v2:172.21.15.155:3300/0 conn(0x7fc4e4070e20 0x7fc4e4071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.558+0000 7fc4e8a00640 1 -- 172.21.15.86:0/1751010575 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e40ce970 con 0x7fc4e40d2c80 2024-04-25T14:17:56.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.558+0000 7fc4e3e00640 1 --2- 172.21.15.86:0/1751010575 >> v2:172.21.15.155:3300/0 conn(0x7fc4e4070e20 0x7fc4e4071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:17:56.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.562+0000 7fc4e8a00640 1 --2- 172.21.15.86:0/1751010575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e40d2c80 0x7fc4e415dc90 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7fc4e40b6070 tx=0x7fc4d801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24fc7937e306f29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:56.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.562+0000 7fc4e3400640 1 -- 172.21.15.86:0/1751010575 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d80b5070 con 0x7fc4e40d2c80 2024-04-25T14:17:56.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.562+0000 7fc4e3400640 1 -- 172.21.15.86:0/1751010575 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4d80a89d0 con 0x7fc4e40d2c80 2024-04-25T14:17:56.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.562+0000 7fc4e3400640 1 -- 172.21.15.86:0/1751010575 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d80a8da0 con 0x7fc4e40d2c80 2024-04-25T14:17:56.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.562+0000 7fc4eb400640 1 -- 172.21.15.86:0/1751010575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e40d2c80 msgr2=0x7fc4e415dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:56.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.562+0000 7fc4eb400640 1 --2- 172.21.15.86:0/1751010575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e40d2c80 0x7fc4e415dc90 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7fc4e40b6070 tx=0x7fc4d801ed10 comp rx=0 tx=0).stop 2024-04-25T14:17:56.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.562+0000 7fc4eb400640 1 -- 172.21.15.86:0/1751010575 shutdown_connections 2024-04-25T14:17:56.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.562+0000 7fc4eb400640 1 --2- 172.21.15.86:0/1751010575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4e40722f0 0x7fc4e4072700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.562+0000 7fc4eb400640 1 --2- 172.21.15.86:0/1751010575 >> v2:172.21.15.155:3300/0 conn(0x7fc4e4070e20 0x7fc4e4071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.562+0000 7fc4eb400640 1 --2- 172.21.15.86:0/1751010575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e40d2c80 0x7fc4e415dc90 unknown :-1 s=CLOSED pgs=2426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.562+0000 7fc4eb400640 1 -- 172.21.15.86:0/1751010575 >> 172.21.15.86:0/1751010575 conn(0x7fc4e40cffc0 msgr2=0x7fc4e40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:56.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.562+0000 7fc4eb400640 1 -- 172.21.15.86:0/1751010575 shutdown_connections 2024-04-25T14:17:56.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.562+0000 7fc4eb400640 1 -- 172.21.15.86:0/1751010575 wait complete. 2024-04-25T14:17:56.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.562+0000 7fc4eb400640 1 Processor -- start 2024-04-25T14:17:56.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.562+0000 7fc4eb400640 1 -- start start 2024-04-25T14:17:56.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.566+0000 7fc4eb400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4e4070e20 0x7fc4e4153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:56.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.566+0000 7fc4eb400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4e40722f0 0x7fc4e406e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:56.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.566+0000 7fc4eb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e40d2c80 0x7fc4e415fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:56.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.566+0000 7fc4eb400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e40d21a0 con 0x7fc4e40d2c80 2024-04-25T14:17:56.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.566+0000 7fc4eb400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e40d1ec0 con 0x7fc4e4070e20 2024-04-25T14:17:56.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.566+0000 7fc4eb400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc4e40d2480 con 0x7fc4e40722f0 2024-04-25T14:17:56.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.566+0000 7fc4e9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e40d2c80 0x7fc4e415fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:56.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.566+0000 7fc4e9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e40d2c80 0x7fc4e415fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59122/0 (socket says 172.21.15.86:59122) 2024-04-25T14:17:56.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.566+0000 7fc4e9400640 1 -- 172.21.15.86:0/239038314 learned_addr learned my addr 172.21.15.86:0/239038314 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:56.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.566+0000 7fc4e3e00640 1 --2- 172.21.15.86:0/239038314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4e40722f0 0x7fc4e406e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:56.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.566+0000 7fc4e8a00640 1 --2- 172.21.15.86:0/239038314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4e4070e20 0x7fc4e4153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:56.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.566+0000 7fc4e9400640 1 -- 172.21.15.86:0/239038314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4e40722f0 msgr2=0x7fc4e406e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:56.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.566+0000 7fc4e9400640 1 --2- 172.21.15.86:0/239038314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4e40722f0 0x7fc4e406e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.566+0000 7fc4e9400640 1 -- 172.21.15.86:0/239038314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4e4070e20 msgr2=0x7fc4e4153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:56.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.566+0000 7fc4e9400640 1 --2- 172.21.15.86:0/239038314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4e4070e20 0x7fc4e4153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.566+0000 7fc4e9400640 1 -- 172.21.15.86:0/239038314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e406ec50 con 0x7fc4e40d2c80 2024-04-25T14:17:56.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.566+0000 7fc4e8a00640 1 --2- 172.21.15.86:0/239038314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4e4070e20 0x7fc4e4153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:17:56.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.566+0000 7fc4e9400640 1 --2- 172.21.15.86:0/239038314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e40d2c80 0x7fc4e415fce0 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc007d80 tx=0x7fc4dc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:56.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.566+0000 7fc4e0c00640 1 -- 172.21.15.86:0/239038314 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc014420 con 0x7fc4e40d2c80 2024-04-25T14:17:56.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.566+0000 7fc4e0c00640 1 -- 172.21.15.86:0/239038314 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4dc019070 con 0x7fc4e40d2c80 2024-04-25T14:17:56.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.566+0000 7fc4eb400640 1 -- 172.21.15.86:0/239038314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e406ef30 con 0x7fc4e40d2c80 2024-04-25T14:17:56.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.566+0000 7fc4e0c00640 1 -- 172.21.15.86:0/239038314 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc014800 con 0x7fc4e40d2c80 2024-04-25T14:17:56.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.566+0000 7fc4eb400640 1 -- 172.21.15.86:0/239038314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e406e6d0 con 0x7fc4e40d2c80 2024-04-25T14:17:56.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.570+0000 7fc4eb400640 1 -- 172.21.15.86:0/239038314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4e40720e0 con 0x7fc4e40d2c80 2024-04-25T14:17:56.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.570+0000 7fc4e0c00640 1 -- 172.21.15.86:0/239038314 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 85) ==== 93207+0+0 (secure 0 0 0) 0x7fc4dc0149a0 con 0x7fc4e40d2c80 2024-04-25T14:17:56.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.570+0000 7fc4e0c00640 1 --2- 172.21.15.86:0/239038314 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc4c406f430 0x7fc4c4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:56.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.570+0000 7fc4e0c00640 1 -- 172.21.15.86:0/239038314 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7fc4dc08fc30 con 0x7fc4e40d2c80 2024-04-25T14:17:56.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.570+0000 7fc4e8a00640 1 -- 172.21.15.86:0/239038314 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc4c406f430 msgr2=0x7fc4c4071940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2318291413 2024-04-25T14:17:56.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.570+0000 7fc4e8a00640 1 --2- 172.21.15.86:0/239038314 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc4c406f430 0x7fc4c4071940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:17:56.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.574+0000 7fc4e0c00640 1 -- 172.21.15.86:0/239038314 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc4dc05c1e0 con 0x7fc4e40d2c80 2024-04-25T14:17:56.573 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:17:56.575+0000 7f0fb926d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:17:56.662 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:56.662+0000 7f228a9e8180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:17:56.748 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:56.750+0000 7f228a9e8180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:17:56.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.770+0000 7fc4eb400640 1 -- 172.21.15.86:0/239038314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc4e4076bf0 con 0x7fc4e40d2c80 2024-04-25T14:17:56.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.770+0000 7fc4e0c00640 1 -- 172.21.15.86:0/239038314 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v156) ==== 100+0+11 (secure 0 0 0) 0x7fc4dc0604e0 con 0x7fc4e40d2c80 2024-04-25T14:17:56.770 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:17:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.774+0000 7fc4e8a00640 1 -- 172.21.15.86:0/239038314 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc4c406f430 msgr2=0x7fc4c4071940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2318291413 2024-04-25T14:17:56.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.774+0000 7fc4e8a00640 1 --2- 172.21.15.86:0/239038314 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc4c406f430 0x7fc4c4071940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:17:56.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.774+0000 7fc4eb400640 1 -- 172.21.15.86:0/239038314 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc4c406f430 msgr2=0x7fc4c4071940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:17:56.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.774+0000 7fc4eb400640 1 --2- 172.21.15.86:0/239038314 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc4c406f430 0x7fc4c4071940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.774+0000 7fc4eb400640 1 -- 172.21.15.86:0/239038314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e40d2c80 msgr2=0x7fc4e415fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:56.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.774+0000 7fc4eb400640 1 --2- 172.21.15.86:0/239038314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e40d2c80 0x7fc4e415fce0 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc007d80 tx=0x7fc4dc00e4b0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.774+0000 7fc4eb400640 1 -- 172.21.15.86:0/239038314 shutdown_connections 2024-04-25T14:17:56.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.774+0000 7fc4eb400640 1 --2- 172.21.15.86:0/239038314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e40d2c80 0x7fc4e415fce0 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.774+0000 7fc4eb400640 1 --2- 172.21.15.86:0/239038314 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7fc4c406f430 0x7fc4c4071940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.774+0000 7fc4eb400640 1 --2- 172.21.15.86:0/239038314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4e40722f0 0x7fc4e406e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.774+0000 7fc4eb400640 1 --2- 172.21.15.86:0/239038314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4e4070e20 0x7fc4e4153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.774+0000 7fc4eb400640 1 -- 172.21.15.86:0/239038314 >> 172.21.15.86:0/239038314 conn(0x7fc4e40cffc0 msgr2=0x7fc4e4159530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.774+0000 7fc4eb400640 1 -- 172.21.15.86:0/239038314 shutdown_connections 2024-04-25T14:17:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.774+0000 7fc4eb400640 1 -- 172.21.15.86:0/239038314 wait complete. 2024-04-25T14:17:56.792 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T14:17:56.899 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:17:56.899+0000 7f0fb926d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:17:56.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.942+0000 7f5b80a00640 1 Processor -- start 2024-04-25T14:17:56.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.942+0000 7f5b80a00640 1 -- start start 2024-04-25T14:17:56.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.942+0000 7f5b80a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5b7c160120 0x7f5b7c0af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:56.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.942+0000 7f5b80a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5b7c0d0f50 0x7f5b7c0d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:56.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.942+0000 7f5b80a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b7c0af780 con 0x7f5b7c0d18a0 2024-04-25T14:17:56.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.942+0000 7f5b80a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5b7c0d0240 con 0x7f5b7c160120 2024-04-25T14:17:56.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.942+0000 7f5b80a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f5b7c0b7fd0 con 0x7f5b7c0d0f50 2024-04-25T14:17:56.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.942+0000 7f5b7b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5b7c160120 0x7f5b7c0af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:56.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.946+0000 7f5b7b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5b7c160120 0x7f5b7c0af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59130/0 (socket says 172.21.15.86:59130) 2024-04-25T14:17:56.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.946+0000 7f5b7b400640 1 -- 172.21.15.86:0/2451497498 learned_addr learned my addr 172.21.15.86:0/2451497498 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:56.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.946+0000 7f5b7aa00640 1 --2- 172.21.15.86:0/2451497498 >> v2:172.21.15.155:3300/0 conn(0x7f5b7c0d0f50 0x7f5b7c0d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:56.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.946+0000 7f5b7b400640 1 -- 172.21.15.86:0/2451497498 >> v2:172.21.15.155:3300/0 conn(0x7f5b7c0d0f50 msgr2=0x7f5b7c0d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:56.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.946+0000 7f5b7b400640 1 --2- 172.21.15.86:0/2451497498 >> v2:172.21.15.155:3300/0 conn(0x7f5b7c0d0f50 0x7f5b7c0d1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.946+0000 7f5b7b400640 1 -- 172.21.15.86:0/2451497498 >> v1:172.21.15.155:6789/0 conn(0x7f5b7c0d18a0 legacy=0x7f5b7c0d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:56.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.946+0000 7f5b7b400640 1 -- 172.21.15.86:0/2451497498 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b7c0b7cf0 con 0x7f5b7c160120 2024-04-25T14:17:56.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.946+0000 7f5b7b400640 1 --2- 172.21.15.86:0/2451497498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b7c160120 0x7f5b7c0af240 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c06f8e0 tx=0x7f5b6c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a461c6738e3760 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:56.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.946+0000 7f5b7a000640 1 -- 172.21.15.86:0/2451497498 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c0b0070 con 0x7f5b7c160120 2024-04-25T14:17:56.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.946+0000 7f5b7a000640 1 -- 172.21.15.86:0/2451497498 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b6c0ab600 con 0x7f5b7c160120 2024-04-25T14:17:56.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.946+0000 7f5b7a000640 1 -- 172.21.15.86:0/2451497498 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c0ab9d0 con 0x7f5b7c160120 2024-04-25T14:17:56.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.950+0000 7f5b80a00640 1 -- 172.21.15.86:0/2451497498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b7c160120 msgr2=0x7f5b7c0af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:56.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.950+0000 7f5b80a00640 1 --2- 172.21.15.86:0/2451497498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b7c160120 0x7f5b7c0af240 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c06f8e0 tx=0x7f5b6c01c490 comp rx=0 tx=0).stop 2024-04-25T14:17:56.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.950+0000 7f5b80a00640 1 -- 172.21.15.86:0/2451497498 shutdown_connections 2024-04-25T14:17:56.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.950+0000 7f5b80a00640 1 --2- 172.21.15.86:0/2451497498 >> v2:172.21.15.155:3300/0 conn(0x7f5b7c0d0f50 0x7f5b7c0d1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.950+0000 7f5b80a00640 1 --2- 172.21.15.86:0/2451497498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b7c160120 0x7f5b7c0af240 unknown :-1 s=CLOSED pgs=2428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.950+0000 7f5b80a00640 1 -- 172.21.15.86:0/2451497498 >> 172.21.15.86:0/2451497498 conn(0x7f5b7c09ba30 msgr2=0x7f5b7c0b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:56.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.950+0000 7f5b80a00640 1 -- 172.21.15.86:0/2451497498 shutdown_connections 2024-04-25T14:17:56.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.950+0000 7f5b80a00640 1 -- 172.21.15.86:0/2451497498 wait complete. 2024-04-25T14:17:56.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.950+0000 7f5b80a00640 1 Processor -- start 2024-04-25T14:17:56.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.950+0000 7f5b80a00640 1 -- start start 2024-04-25T14:17:56.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.954+0000 7f5b80a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b7c0d0f50 0x7f5b7c0cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:56.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.954+0000 7f5b80a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b7c0d18a0 0x7f5b7c146d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:56.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.954+0000 7f5b80a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b7c160120 0x7f5b7c06d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:56.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.954+0000 7f5b80a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7c16cd60 con 0x7f5b7c0d0f50 2024-04-25T14:17:56.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.954+0000 7f5b80a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7c16cbe0 con 0x7f5b7c0d18a0 2024-04-25T14:17:56.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.954+0000 7f5b80a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5b7c16cee0 con 0x7f5b7c160120 2024-04-25T14:17:56.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.954+0000 7f5b7b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b7c0d0f50 0x7f5b7c0cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:56.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.954+0000 7f5b7b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b7c0d0f50 0x7f5b7c0cf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59138/0 (socket says 172.21.15.86:59138) 2024-04-25T14:17:56.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.954+0000 7f5b7b400640 1 -- 172.21.15.86:0/650635778 learned_addr learned my addr 172.21.15.86:0/650635778 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:56.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.954+0000 7f5b7aa00640 1 --2- 172.21.15.86:0/650635778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b7c0d18a0 0x7f5b7c146d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:56.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.954+0000 7f5b7be00640 1 --2- 172.21.15.86:0/650635778 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b7c160120 0x7f5b7c06d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:56.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.954+0000 7f5b7b400640 1 -- 172.21.15.86:0/650635778 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b7c160120 msgr2=0x7f5b7c06d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:56.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.954+0000 7f5b7b400640 1 --2- 172.21.15.86:0/650635778 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b7c160120 0x7f5b7c06d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.954+0000 7f5b7b400640 1 -- 172.21.15.86:0/650635778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b7c0d18a0 msgr2=0x7f5b7c146d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:56.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.954+0000 7f5b7b400640 1 --2- 172.21.15.86:0/650635778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b7c0d18a0 0x7f5b7c146d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:56.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.954+0000 7f5b7b400640 1 -- 172.21.15.86:0/650635778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b7c0cf7f0 con 0x7f5b7c0d0f50 2024-04-25T14:17:56.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.954+0000 7f5b7b400640 1 --2- 172.21.15.86:0/650635778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b7c0d0f50 0x7f5b7c0cf2b0 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c0190a0 tx=0x7f5b6c019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:56.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.954+0000 7f5b73400640 1 -- 172.21.15.86:0/650635778 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c0b0040 con 0x7f5b7c0d0f50 2024-04-25T14:17:56.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.954+0000 7f5b73400640 1 -- 172.21.15.86:0/650635778 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b6c0b4420 con 0x7f5b7c0d0f50 2024-04-25T14:17:56.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.954+0000 7f5b80a00640 1 -- 172.21.15.86:0/650635778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b7c1472b0 con 0x7f5b7c0d0f50 2024-04-25T14:17:56.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.954+0000 7f5b73400640 1 -- 172.21.15.86:0/650635778 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c0b47f0 con 0x7f5b7c0d0f50 2024-04-25T14:17:56.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.954+0000 7f5b80a00640 1 -- 172.21.15.86:0/650635778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b7c156080 con 0x7f5b7c0d0f50 2024-04-25T14:17:56.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.954+0000 7f5b80a00640 1 -- 172.21.15.86:0/650635778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b7c076b80 con 0x7f5b7c0d0f50 2024-04-25T14:17:56.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.958+0000 7f5b73400640 1 -- 172.21.15.86:0/650635778 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 85) ==== 93207+0+0 (secure 0 0 0) 0x7f5b6c0b4990 con 0x7f5b7c0d0f50 2024-04-25T14:17:56.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.958+0000 7f5b73400640 1 --2- 172.21.15.86:0/650635778 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f5b5c06f400 0x7f5b5c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:56.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.958+0000 7f5b7aa00640 1 -- 172.21.15.86:0/650635778 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f5b5c06f400 msgr2=0x7f5b5c071910 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6832/2318291413 2024-04-25T14:17:56.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.958+0000 7f5b73400640 1 -- 172.21.15.86:0/650635778 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6445+0+0 (secure 0 0 0) 0x7f5b6c129070 con 0x7f5b7c0d0f50 2024-04-25T14:17:56.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.958+0000 7f5b7aa00640 1 --2- 172.21.15.86:0/650635778 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f5b5c06f400 0x7f5b5c071910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:17:56.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:56.962+0000 7f5b73400640 1 -- 172.21.15.86:0/650635778 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5b6c016110 con 0x7f5b7c0d0f50 2024-04-25T14:17:56.962 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-25T14:17:56.963+0000 7f0fb926d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:17:57.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.026+0000 7f5b73400640 1 -- 172.21.15.86:0/650635778 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 86) ==== 93207+0+0 (secure 0 0 0) 0x7f5b6c0f18b0 con 0x7f5b7c0d0f50 2024-04-25T14:17:57.087 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:57.086+0000 7f228a9e8180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:17:57.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.134+0000 7f5b80a00640 1 -- 172.21.15.86:0/650635778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5b7c0b1c00 con 0x7f5b7c0d0f50 2024-04-25T14:17:57.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.134+0000 7f5b73400640 1 -- 172.21.15.86:0/650635778 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v156) ==== 80+0+467 (secure 0 0 0) 0x7f5b6c126000 con 0x7f5b7c0d0f50 2024-04-25T14:17:57.135 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2043624430 2024-04-26T14:13:51.718341+0000 2024-04-25T14:17:57.135 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/955458502 2024-04-26T14:13:51.718341+0000 2024-04-25T14:17:57.135 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1302282677 2024-04-26T14:13:51.718341+0000 2024-04-25T14:17:57.135 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6833/505730993 2024-04-26T14:13:51.718341+0000 2024-04-25T14:17:57.135 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6832/505730993 2024-04-26T14:13:51.718341+0000 2024-04-25T14:17:57.135 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3455088811 2024-04-26T14:13:51.718341+0000 2024-04-25T14:17:57.135 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3537592233 2024-04-26T14:13:51.718341+0000 2024-04-25T14:17:57.135 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2705096450 2024-04-26T14:13:51.718341+0000 2024-04-25T14:17:57.135 INFO:teuthology.orchestra.run.smithi086.stderr:listed 8 entries 2024-04-25T14:17:57.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.138+0000 7f5b80a00640 1 -- 172.21.15.86:0/650635778 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f5b5c06f400 msgr2=0x7f5b5c071910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:17:57.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.138+0000 7f5b80a00640 1 --2- 172.21.15.86:0/650635778 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f5b5c06f400 0x7f5b5c071910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:57.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.138+0000 7f5b80a00640 1 -- 172.21.15.86:0/650635778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b7c0d0f50 msgr2=0x7f5b7c0cf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:57.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.138+0000 7f5b80a00640 1 --2- 172.21.15.86:0/650635778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b7c0d0f50 0x7f5b7c0cf2b0 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c0190a0 tx=0x7f5b6c019230 comp rx=0 tx=0).stop 2024-04-25T14:17:57.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.138+0000 7f5b80a00640 1 -- 172.21.15.86:0/650635778 shutdown_connections 2024-04-25T14:17:57.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.138+0000 7f5b80a00640 1 --2- 172.21.15.86:0/650635778 >> [v2:172.21.15.86:6832/2318291413,v1:172.21.15.86:6833/2318291413] conn(0x7f5b5c06f400 0x7f5b5c071910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:57.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.138+0000 7f5b80a00640 1 --2- 172.21.15.86:0/650635778 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b7c160120 0x7f5b7c06d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:57.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.138+0000 7f5b80a00640 1 --2- 172.21.15.86:0/650635778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b7c0d18a0 0x7f5b7c146d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:57.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.138+0000 7f5b80a00640 1 --2- 172.21.15.86:0/650635778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b7c0d0f50 0x7f5b7c0cf2b0 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:57.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.138+0000 7f5b80a00640 1 -- 172.21.15.86:0/650635778 >> 172.21.15.86:0/650635778 conn(0x7f5b7c09ba30 msgr2=0x7f5b7c0af950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:57.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.138+0000 7f5b80a00640 1 -- 172.21.15.86:0/650635778 shutdown_connections 2024-04-25T14:17:57.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.138+0000 7f5b80a00640 1 -- 172.21.15.86:0/650635778 wait complete. 2024-04-25T14:17:57.151 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:57.150+0000 7f228a9e8180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:17:57.160 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T14:17:57.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:17:57.161 DEBUG:teuthology.orchestra.run.smithi155:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T14:17:57.192 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T14:17:57.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:17:57.192 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T14:17:57.228 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:57.230+0000 7f2209200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:17:57.229 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:57.230+0000 7f2209200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:17:57.229 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:57.230+0000 7f2209200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:17:57.229 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:57.230+0000 7f2209200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:17:57.229 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:57.230+0000 7f2209200640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:17:57.229 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:57.230+0000 7f220ce00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:17:57.229 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:57.230+0000 7f220ce00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:17:57.229 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:57.230+0000 7f220ce00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:17:57.229 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:57.230+0000 7f220ce00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:17:57.230 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-25T14:17:57.230+0000 7f220ce00640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:17:57.247 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:17:57.247 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:17:57.248 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:17:57.257 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2893 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T14:17:57.257 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T14:17:57.257 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:17:57.257 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:17:57.302 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4517 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T14:17:57.303 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:57.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.458+0000 7f504e600640 1 Processor -- start 2024-04-25T14:17:57.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.462+0000 7f504e600640 1 -- start start 2024-04-25T14:17:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.462+0000 7f504e600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f504815faa0 0x7f504806d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:57.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.462+0000 7f504e600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f50480b59e0 0x7f50480b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:57.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.462+0000 7f504e600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f50480aeea0 0x7f50480af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:57.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.462+0000 7f504e600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f504806d560 con 0x7f50480b59e0 2024-04-25T14:17:57.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.462+0000 7f504e600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f50480b6330 con 0x7f50480aeea0 2024-04-25T14:17:57.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.462+0000 7f504e600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f504806c670 con 0x7f504815faa0 2024-04-25T14:17:57.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.462+0000 7f503ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f50480b59e0 0x7f50480b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:57.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.462+0000 7f503ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f50480b59e0 0x7f50480b5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59232/0 (socket says 172.21.15.86:59232) 2024-04-25T14:17:57.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.462+0000 7f503ea00640 1 -- 172.21.15.86:0/3428776573 learned_addr learned my addr 172.21.15.86:0/3428776573 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:57.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.462+0000 7f5047400640 1 --2- 172.21.15.86:0/3428776573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f504815faa0 0x7f504806d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:57.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.462+0000 7f5047e00640 1 --2- 172.21.15.86:0/3428776573 >> v2:172.21.15.155:3300/0 conn(0x7f50480aeea0 0x7f50480af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:57.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.462+0000 7f503ea00640 1 -- 172.21.15.86:0/3428776573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f504815faa0 msgr2=0x7f504806d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:57.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.462+0000 7f503ea00640 1 --2- 172.21.15.86:0/3428776573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f504815faa0 0x7f504806d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:57.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.462+0000 7f503ea00640 1 -- 172.21.15.86:0/3428776573 >> v2:172.21.15.155:3300/0 conn(0x7f50480aeea0 msgr2=0x7f50480af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:57.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.462+0000 7f503ea00640 1 --2- 172.21.15.86:0/3428776573 >> v2:172.21.15.155:3300/0 conn(0x7f50480aeea0 0x7f50480af2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:57.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.462+0000 7f503ea00640 1 -- 172.21.15.86:0/3428776573 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504806c950 con 0x7f50480b59e0 2024-04-25T14:17:57.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f503ea00640 1 --2- 172.21.15.86:0/3428776573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50480b59e0 0x7f50480b5df0 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f504806d4d0 tx=0x7f503001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46923267b60ed767 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:57.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f5046a00640 1 -- 172.21.15.86:0/3428776573 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50300a6040 con 0x7f50480b59e0 2024-04-25T14:17:57.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f5046a00640 1 -- 172.21.15.86:0/3428776573 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50300af070 con 0x7f50480b59e0 2024-04-25T14:17:57.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f5046a00640 1 -- 172.21.15.86:0/3428776573 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50300aa860 con 0x7f50480b59e0 2024-04-25T14:17:57.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f504e600640 1 -- 172.21.15.86:0/3428776573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50480b59e0 msgr2=0x7f50480b5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:57.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f504e600640 1 --2- 172.21.15.86:0/3428776573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50480b59e0 0x7f50480b5df0 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f504806d4d0 tx=0x7f503001c520 comp rx=0 tx=0).stop 2024-04-25T14:17:57.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f504e600640 1 -- 172.21.15.86:0/3428776573 shutdown_connections 2024-04-25T14:17:57.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f504e600640 1 --2- 172.21.15.86:0/3428776573 >> v2:172.21.15.155:3300/0 conn(0x7f50480aeea0 0x7f50480af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:57.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f504e600640 1 --2- 172.21.15.86:0/3428776573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50480b59e0 0x7f50480b5df0 unknown :-1 s=CLOSED pgs=2440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:57.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f504e600640 1 --2- 172.21.15.86:0/3428776573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f504815faa0 0x7f504806d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:57.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f504e600640 1 -- 172.21.15.86:0/3428776573 >> 172.21.15.86:0/3428776573 conn(0x7f50480d2230 msgr2=0x7f50480d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:57.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f504e600640 1 -- 172.21.15.86:0/3428776573 shutdown_connections 2024-04-25T14:17:57.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f504e600640 1 -- 172.21.15.86:0/3428776573 wait complete. 2024-04-25T14:17:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f504e600640 1 Processor -- start 2024-04-25T14:17:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f504e600640 1 -- start start 2024-04-25T14:17:57.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f504e600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50480aeea0 0x7f5048143760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:57.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f504e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50480b59e0 0x7f50481623a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:57.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f504e600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f504815faa0 0x7f50481628e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:57.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f504e600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f504815feb0 con 0x7f50480b59e0 2024-04-25T14:17:57.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f504e600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f504806df30 con 0x7f504815faa0 2024-04-25T14:17:57.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f504e600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5048160030 con 0x7f50480aeea0 2024-04-25T14:17:57.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f503ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50480b59e0 0x7f50481623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:57.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f503ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50480b59e0 0x7f50481623a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59238/0 (socket says 172.21.15.86:59238) 2024-04-25T14:17:57.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f503ea00640 1 -- 172.21.15.86:0/282618073 learned_addr learned my addr 172.21.15.86:0/282618073 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:57.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f5047400640 1 --2- 172.21.15.86:0/282618073 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50480aeea0 0x7f5048143760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:57.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f5047e00640 1 --2- 172.21.15.86:0/282618073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f504815faa0 0x7f50481628e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:57.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f503ea00640 1 -- 172.21.15.86:0/282618073 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50480aeea0 msgr2=0x7f5048143760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:57.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f503ea00640 1 --2- 172.21.15.86:0/282618073 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50480aeea0 0x7f5048143760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:57.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f503ea00640 1 -- 172.21.15.86:0/282618073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f504815faa0 msgr2=0x7f50481628e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:57.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f503ea00640 1 --2- 172.21.15.86:0/282618073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f504815faa0 0x7f50481628e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:57.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f503ea00640 1 -- 172.21.15.86:0/282618073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50480b0c10 con 0x7f50480b59e0 2024-04-25T14:17:57.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f503ea00640 1 --2- 172.21.15.86:0/282618073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50480b59e0 0x7f50481623a0 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f503001cdc0 tx=0x7f503001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:57.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f503fe00640 1 -- 172.21.15.86:0/282618073 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50300aae10 con 0x7f50480b59e0 2024-04-25T14:17:57.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f503fe00640 1 -- 172.21.15.86:0/282618073 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50300bd3f0 con 0x7f50480b59e0 2024-04-25T14:17:57.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f503fe00640 1 -- 172.21.15.86:0/282618073 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50300bd7e0 con 0x7f50480b59e0 2024-04-25T14:17:57.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.470+0000 7f504e600640 1 -- 172.21.15.86:0/282618073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50480afe50 con 0x7f50480b59e0 2024-04-25T14:17:57.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.474+0000 7f504e600640 1 -- 172.21.15.86:0/282618073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50480b0410 con 0x7f50480b59e0 2024-04-25T14:17:57.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.474+0000 7f504e600640 1 -- 172.21.15.86:0/282618073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5048074cf0 con 0x7f50480b59e0 2024-04-25T14:17:57.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.474+0000 7f503fe00640 1 -- 172.21.15.86:0/282618073 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 87) ==== 92918+0+0 (secure 0 0 0) 0x7f50300b33f0 con 0x7f50480b59e0 2024-04-25T14:17:57.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.474+0000 7f503fe00640 1 -- 172.21.15.86:0/282618073 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6703+0+0 (secure 0 0 0) 0x7f50301292c0 con 0x7f50480b59e0 2024-04-25T14:17:57.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.478+0000 7f503fe00640 1 -- 172.21.15.86:0/282618073 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5030016110 con 0x7f50480b59e0 2024-04-25T14:17:57.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.670+0000 7f504e600640 1 -- 172.21.15.86:0/282618073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50480b55b0 con 0x7f50480b59e0 2024-04-25T14:17:57.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.670+0000 7f503fe00640 1 -- 172.21.15.86:0/282618073 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f50300f5c90 con 0x7f50480b59e0 2024-04-25T14:17:57.668 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:57.668 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:57.668 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:57.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.670+0000 7f504e600640 1 -- 172.21.15.86:0/282618073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50480b59e0 msgr2=0x7f50481623a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:57.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.670+0000 7f504e600640 1 --2- 172.21.15.86:0/282618073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50480b59e0 0x7f50481623a0 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f503001cdc0 tx=0x7f503001ad60 comp rx=0 tx=0).stop 2024-04-25T14:17:57.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.670+0000 7f504e600640 1 -- 172.21.15.86:0/282618073 shutdown_connections 2024-04-25T14:17:57.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.670+0000 7f504e600640 1 --2- 172.21.15.86:0/282618073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f504815faa0 0x7f50481628e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:57.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.670+0000 7f504e600640 1 --2- 172.21.15.86:0/282618073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50480b59e0 0x7f50481623a0 unknown :-1 s=CLOSED pgs=2441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:57.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.670+0000 7f504e600640 1 --2- 172.21.15.86:0/282618073 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50480aeea0 0x7f5048143760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:57.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.670+0000 7f504e600640 1 -- 172.21.15.86:0/282618073 >> 172.21.15.86:0/282618073 conn(0x7f50480d2230 msgr2=0x7f5048158c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:57.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.670+0000 7f504e600640 1 -- 172.21.15.86:0/282618073 shutdown_connections 2024-04-25T14:17:57.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.670+0000 7f504e600640 1 -- 172.21.15.86:0/282618073 wait complete. 2024-04-25T14:17:57.692 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:17:57.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.846+0000 7ff5e5a00640 1 Processor -- start 2024-04-25T14:17:57.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.846+0000 7ff5e5a00640 1 -- start start 2024-04-25T14:17:57.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.850+0000 7ff5e5a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5e00cfca0 0x7ff5e006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:57.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.850+0000 7ff5e5a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff5e00d16e0 0x7ff5e00d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:57.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.850+0000 7ff5e5a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5e00d2030 con 0x7ff5e006c670 2024-04-25T14:17:57.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.850+0000 7ff5e5a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff5e0070470 con 0x7ff5e00d16e0 2024-04-25T14:17:57.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.850+0000 7ff5e5a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e0072bd0 con 0x7ff5e00cfca0 2024-04-25T14:17:57.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.850+0000 7ff5df400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff5e006c670 0x7ff5e006ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59182/0 (socket says 172.21.15.86:59182) 2024-04-25T14:17:57.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.850+0000 7ff5df400640 1 -- 172.21.15.86:0/1735181651 learned_addr learned my addr 172.21.15.86:0/1735181651 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:17:57.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.850+0000 7ff5d6000640 1 --2- 172.21.15.86:0/1735181651 >> v2:172.21.15.86:3300/0 conn(0x7ff5e00d16e0 0x7ff5e00d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:57.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.850+0000 7ff5dea00640 1 --2- 172.21.15.86:0/1735181651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5e00cfca0 0x7ff5e006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:57.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.850+0000 7ff5d6000640 1 -- 172.21.15.86:0/1735181651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5e00cfca0 msgr2=0x7ff5e006f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:57.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.850+0000 7ff5d6000640 1 --2- 172.21.15.86:0/1735181651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5e00cfca0 0x7ff5e006f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:57.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.850+0000 7ff5d6000640 1 -- 172.21.15.86:0/1735181651 >> v1:172.21.15.86:6789/0 conn(0x7ff5e006c670 legacy=0x7ff5e006ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:57.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.850+0000 7ff5d6000640 1 -- 172.21.15.86:0/1735181651 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e006f930 con 0x7ff5e00d16e0 2024-04-25T14:17:57.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.850+0000 7ff5de000640 1 -- 172.21.15.86:0/1735181651 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1270032621 0 0) 0x7ff5d8001120 con 0x7ff5e006c670 2024-04-25T14:17:57.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.854+0000 7ff5d6000640 1 --2- 172.21.15.86:0/1735181651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e00d16e0 0x7ff5e00d1af0 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80097c0 tx=0x7ff5c801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1227d95f23c0c6bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:57.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.854+0000 7ff5de000640 1 -- 172.21.15.86:0/1735181651 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c80b1070 con 0x7ff5e00d16e0 2024-04-25T14:17:57.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.854+0000 7ff5de000640 1 -- 172.21.15.86:0/1735181651 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5c80ac410 con 0x7ff5e00d16e0 2024-04-25T14:17:57.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.854+0000 7ff5de000640 1 -- 172.21.15.86:0/1735181651 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c80ac7e0 con 0x7ff5e00d16e0 2024-04-25T14:17:57.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.854+0000 7ff5e5a00640 1 -- 172.21.15.86:0/1735181651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e00d16e0 msgr2=0x7ff5e00d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:57.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.854+0000 7ff5e5a00640 1 --2- 172.21.15.86:0/1735181651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e00d16e0 0x7ff5e00d1af0 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80097c0 tx=0x7ff5c801c4a0 comp rx=0 tx=0).stop 2024-04-25T14:17:57.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.854+0000 7ff5e5a00640 1 -- 172.21.15.86:0/1735181651 shutdown_connections 2024-04-25T14:17:57.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.854+0000 7ff5e5a00640 1 --2- 172.21.15.86:0/1735181651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e00d16e0 0x7ff5e00d1af0 unknown :-1 s=CLOSED pgs=2443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:57.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.854+0000 7ff5e5a00640 1 --2- 172.21.15.86:0/1735181651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5e00cfca0 0x7ff5e006f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:57.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.854+0000 7ff5e5a00640 1 -- 172.21.15.86:0/1735181651 >> 172.21.15.86:0/1735181651 conn(0x7ff5e00b4650 msgr2=0x7ff5e00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:57.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.854+0000 7ff5e5a00640 1 -- 172.21.15.86:0/1735181651 shutdown_connections 2024-04-25T14:17:57.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.854+0000 7ff5e5a00640 1 -- 172.21.15.86:0/1735181651 wait complete. 2024-04-25T14:17:57.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.858+0000 7ff5e5a00640 1 Processor -- start 2024-04-25T14:17:57.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.858+0000 7ff5e5a00640 1 -- start start 2024-04-25T14:17:57.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.858+0000 7ff5e5a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5e006c670 0x7ff5e00c3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:57.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.858+0000 7ff5e5a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5e00cfca0 0x7ff5e00ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:57.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.858+0000 7ff5e5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e00d16e0 0x7ff5e00cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:57.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.858+0000 7ff5e5a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e016a070 con 0x7ff5e00d16e0 2024-04-25T14:17:57.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.858+0000 7ff5e5a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e0169ef0 con 0x7ff5e006c670 2024-04-25T14:17:57.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.858+0000 7ff5e5a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e016a1f0 con 0x7ff5e00cfca0 2024-04-25T14:17:57.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.858+0000 7ff5df400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e00d16e0 0x7ff5e00cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:57.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.858+0000 7ff5df400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e00d16e0 0x7ff5e00cafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59256/0 (socket says 172.21.15.86:59256) 2024-04-25T14:17:57.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.858+0000 7ff5df400640 1 -- 172.21.15.86:0/2585460227 learned_addr learned my addr 172.21.15.86:0/2585460227 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:57.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.858+0000 7ff5dea00640 1 --2- 172.21.15.86:0/2585460227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5e006c670 0x7ff5e00c3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:57.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.858+0000 7ff5d6000640 1 --2- 172.21.15.86:0/2585460227 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5e00cfca0 0x7ff5e00ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:57.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.858+0000 7ff5df400640 1 -- 172.21.15.86:0/2585460227 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5e00cfca0 msgr2=0x7ff5e00ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:57.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.858+0000 7ff5df400640 1 --2- 172.21.15.86:0/2585460227 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5e00cfca0 0x7ff5e00ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:57.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.858+0000 7ff5df400640 1 -- 172.21.15.86:0/2585460227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5e006c670 msgr2=0x7ff5e00c3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:57.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.858+0000 7ff5df400640 1 --2- 172.21.15.86:0/2585460227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5e006c670 0x7ff5e00c3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:57.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.858+0000 7ff5df400640 1 -- 172.21.15.86:0/2585460227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e00c9f10 con 0x7ff5e00d16e0 2024-04-25T14:17:57.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.858+0000 7ff5df400640 1 --2- 172.21.15.86:0/2585460227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e00d16e0 0x7ff5e00cafb0 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800c440 tx=0x7ff5d800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:57.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.858+0000 7ff5d7400640 1 -- 172.21.15.86:0/2585460227 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d80074e0 con 0x7ff5e00d16e0 2024-04-25T14:17:57.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.858+0000 7ff5d7400640 1 -- 172.21.15.86:0/2585460227 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d80047f0 con 0x7ff5e00d16e0 2024-04-25T14:17:57.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.858+0000 7ff5e5a00640 1 -- 172.21.15.86:0/2585460227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e00ca1f0 con 0x7ff5e00d16e0 2024-04-25T14:17:57.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.858+0000 7ff5d7400640 1 -- 172.21.15.86:0/2585460227 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8004be0 con 0x7ff5e00d16e0 2024-04-25T14:17:57.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.858+0000 7ff5e5a00640 1 -- 172.21.15.86:0/2585460227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e00ca7b0 con 0x7ff5e00d16e0 2024-04-25T14:17:57.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.858+0000 7ff5e5a00640 1 -- 172.21.15.86:0/2585460227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e009f4b0 con 0x7ff5e00d16e0 2024-04-25T14:17:57.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.862+0000 7ff5d7400640 1 -- 172.21.15.86:0/2585460227 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 87) ==== 92918+0+0 (secure 0 0 0) 0x7ff5d8011070 con 0x7ff5e00d16e0 2024-04-25T14:17:57.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.862+0000 7ff5d7400640 1 -- 172.21.15.86:0/2585460227 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6703+0+0 (secure 0 0 0) 0x7ff5d808eb40 con 0x7ff5e00d16e0 2024-04-25T14:17:57.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:57.866+0000 7ff5d7400640 1 -- 172.21.15.86:0/2585460227 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff5e00d2030 con 0x7ff5e00d16e0 2024-04-25T14:17:58.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.030+0000 7ff5e5a00640 1 -- 172.21.15.86:0/2585460227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5e0069d00 con 0x7ff5e00d16e0 2024-04-25T14:17:58.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.030+0000 7ff5d7400640 1 -- 172.21.15.86:0/2585460227 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) ==== 74+0+17760 (secure 0 0 0) 0x7ff5d805b3f0 con 0x7ff5e00d16e0 2024-04-25T14:17:58.025 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:58.025 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":157,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:17:57.159829+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:13:32.006877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T14:13:32.574088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1552836841":"2024-04-26T14:17:57.159798+0000","172.21.15.86:6833/2318291413":"2024-04-26T14:17:57.159798+0000","172.21.15.86:0/3986518615":"2024-04-26T14:17:57.159798+0000","172.21.15.86:0/3859223883":"2024-04-26T14:17:57.159798+0000","172.21.15.86:0/1995365685":"2024-04-26T14:17:57.159798+0000","172.21.15.86:0/2043624430":"2024-04-26T14:13:51.718341+0000","172.21.15.86:0/955458502":"2024-04-26T14:13:51.718341+0000","172.21.15.86:0/1302282677":"2024-04-26T14:13:51.718341+0000","172.21.15.86:6833/505730993":"2024-04-26T14:13:51.718341+0000","172.21.15.86:6832/505730993":"2024-04-26T14:13:51.718341+0000","172.21.15.86:0/3455088811":"2024-04-26T14:13:51.718341+0000","172.21.15.86:0/3537592233":"2024-04-26T14:13:51.718341+0000","172.21.15.86:6832/2318291413":"2024-04-26T14:17:57.159798+0000","172.21.15.86:0/2705096450":"2024-04-26T14:13:51.718341+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:17:58.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.034+0000 7ff5e5a00640 1 -- 172.21.15.86:0/2585460227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e00d16e0 msgr2=0x7ff5e00cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:58.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.034+0000 7ff5e5a00640 1 --2- 172.21.15.86:0/2585460227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e00d16e0 0x7ff5e00cafb0 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800c440 tx=0x7ff5d800c840 comp rx=0 tx=0).stop 2024-04-25T14:17:58.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.034+0000 7ff5e5a00640 1 -- 172.21.15.86:0/2585460227 shutdown_connections 2024-04-25T14:17:58.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.034+0000 7ff5e5a00640 1 --2- 172.21.15.86:0/2585460227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e00d16e0 0x7ff5e00cafb0 unknown :-1 s=CLOSED pgs=2444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:58.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.034+0000 7ff5e5a00640 1 --2- 172.21.15.86:0/2585460227 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5e00cfca0 0x7ff5e00ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:58.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.034+0000 7ff5e5a00640 1 --2- 172.21.15.86:0/2585460227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5e006c670 0x7ff5e00c3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:58.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.034+0000 7ff5e5a00640 1 -- 172.21.15.86:0/2585460227 >> 172.21.15.86:0/2585460227 conn(0x7ff5e00b4650 msgr2=0x7ff5e009a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:58.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.034+0000 7ff5e5a00640 1 -- 172.21.15.86:0/2585460227 shutdown_connections 2024-04-25T14:17:58.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.034+0000 7ff5e5a00640 1 -- 172.21.15.86:0/2585460227 wait complete. 2024-04-25T14:17:58.046 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:03:38.737000+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:13:32.006877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '154', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:13:32.574088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:17:58.047 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:17:58.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.210+0000 7fe5cb400640 1 Processor -- start 2024-04-25T14:17:58.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.210+0000 7fe5cb400640 1 -- start start 2024-04-25T14:17:58.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.214+0000 7fe5cb400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe5c4163980 0x7fe5c4163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:58.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.214+0000 7fe5cb400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe5c406e340 0x7fe5c406e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:58.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.214+0000 7fe5cb400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5c41648e0 con 0x7fe5c4072630 2024-04-25T14:17:58.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.214+0000 7fe5cb400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe5c40d1520 con 0x7fe5c4163980 2024-04-25T14:17:58.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.214+0000 7fe5cb400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe5c40d23e0 con 0x7fe5c406e340 2024-04-25T14:17:58.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.214+0000 7fe5c8a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe5c4072630 0x7fe5c4072a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59186/0 (socket says 172.21.15.86:59186) 2024-04-25T14:17:58.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.214+0000 7fe5bbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe5c4163980 0x7fe5c4163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:58.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.214+0000 7fe5c8a00640 1 -- 172.21.15.86:0/3339054683 learned_addr learned my addr 172.21.15.86:0/3339054683 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:17:58.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.214+0000 7fe5c9400640 1 --2- 172.21.15.86:0/3339054683 >> v2:172.21.15.155:3300/0 conn(0x7fe5c406e340 0x7fe5c406e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:58.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.214+0000 7fe5bbe00640 1 -- 172.21.15.86:0/3339054683 >> v2:172.21.15.155:3300/0 conn(0x7fe5c406e340 msgr2=0x7fe5c406e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:58.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.214+0000 7fe5bbe00640 1 --2- 172.21.15.86:0/3339054683 >> v2:172.21.15.155:3300/0 conn(0x7fe5c406e340 0x7fe5c406e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:58.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.214+0000 7fe5bbe00640 1 -- 172.21.15.86:0/3339054683 >> v1:172.21.15.86:6789/0 conn(0x7fe5c4072630 legacy=0x7fe5c4072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:58.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.214+0000 7fe5bbe00640 1 -- 172.21.15.86:0/3339054683 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c4164600 con 0x7fe5c4163980 2024-04-25T14:17:58.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.214+0000 7fe5c3e00640 1 -- 172.21.15.86:0/3339054683 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2226582162 0 0) 0x7fe5c41648e0 con 0x7fe5c4072630 2024-04-25T14:17:58.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.218+0000 7fe5bbe00640 1 --2- 172.21.15.86:0/3339054683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5c4163980 0x7fe5c4163d90 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40c6b40 tx=0x7fe5ac01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51b6a57415aff4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:58.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.218+0000 7fe5c3e00640 1 -- 172.21.15.86:0/3339054683 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac0b0070 con 0x7fe5c4163980 2024-04-25T14:17:58.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.218+0000 7fe5c3e00640 1 -- 172.21.15.86:0/3339054683 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5ac0ab600 con 0x7fe5c4163980 2024-04-25T14:17:58.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.218+0000 7fe5c3e00640 1 -- 172.21.15.86:0/3339054683 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac0ab9d0 con 0x7fe5c4163980 2024-04-25T14:17:58.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.218+0000 7fe5cb400640 1 -- 172.21.15.86:0/3339054683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5c4163980 msgr2=0x7fe5c4163d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:58.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.218+0000 7fe5cb400640 1 --2- 172.21.15.86:0/3339054683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5c4163980 0x7fe5c4163d90 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40c6b40 tx=0x7fe5ac01c360 comp rx=0 tx=0).stop 2024-04-25T14:17:58.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.218+0000 7fe5cb400640 1 -- 172.21.15.86:0/3339054683 shutdown_connections 2024-04-25T14:17:58.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.218+0000 7fe5cb400640 1 --2- 172.21.15.86:0/3339054683 >> v2:172.21.15.155:3300/0 conn(0x7fe5c406e340 0x7fe5c406e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:58.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.218+0000 7fe5cb400640 1 --2- 172.21.15.86:0/3339054683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5c4163980 0x7fe5c4163d90 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:58.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.218+0000 7fe5cb400640 1 -- 172.21.15.86:0/3339054683 >> 172.21.15.86:0/3339054683 conn(0x7fe5c40d3580 msgr2=0x7fe5c40d3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:58.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.218+0000 7fe5cb400640 1 -- 172.21.15.86:0/3339054683 shutdown_connections 2024-04-25T14:17:58.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.218+0000 7fe5cb400640 1 -- 172.21.15.86:0/3339054683 wait complete. 2024-04-25T14:17:58.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.218+0000 7fe5cb400640 1 Processor -- start 2024-04-25T14:17:58.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.218+0000 7fe5cb400640 1 -- start start 2024-04-25T14:17:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.222+0000 7fe5cb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5c406e340 0x7fe5c40cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.222+0000 7fe5cb400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5c4072630 0x7fe5c40ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.222+0000 7fe5cb400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5c4163980 0x7fe5c40cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.222+0000 7fe5cb400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c416d530 con 0x7fe5c406e340 2024-04-25T14:17:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.222+0000 7fe5cb400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c406f0a0 con 0x7fe5c4163980 2024-04-25T14:17:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.222+0000 7fe5cb400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c416d6b0 con 0x7fe5c4072630 2024-04-25T14:17:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.222+0000 7fe5c8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5c406e340 0x7fe5c40cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:58.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.222+0000 7fe5c8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5c406e340 0x7fe5c40cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59282/0 (socket says 172.21.15.86:59282) 2024-04-25T14:17:58.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.222+0000 7fe5c8a00640 1 -- 172.21.15.86:0/927229168 learned_addr learned my addr 172.21.15.86:0/927229168 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:58.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.222+0000 7fe5c9400640 1 --2- 172.21.15.86:0/927229168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5c4163980 0x7fe5c40cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:58.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.222+0000 7fe5bbe00640 1 --2- 172.21.15.86:0/927229168 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5c4072630 0x7fe5c40ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:58.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.222+0000 7fe5c8a00640 1 -- 172.21.15.86:0/927229168 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5c4072630 msgr2=0x7fe5c40ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:58.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.222+0000 7fe5c8a00640 1 --2- 172.21.15.86:0/927229168 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5c4072630 0x7fe5c40ccb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:58.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.222+0000 7fe5c8a00640 1 -- 172.21.15.86:0/927229168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5c4163980 msgr2=0x7fe5c40cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:58.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.222+0000 7fe5c8a00640 1 --2- 172.21.15.86:0/927229168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5c4163980 0x7fe5c40cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:58.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.222+0000 7fe5c8a00640 1 -- 172.21.15.86:0/927229168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c415d010 con 0x7fe5c406e340 2024-04-25T14:17:58.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.222+0000 7fe5bbe00640 1 --2- 172.21.15.86:0/927229168 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5c4072630 0x7fe5c40ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:17:58.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.222+0000 7fe5c8a00640 1 --2- 172.21.15.86:0/927229168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5c406e340 0x7fe5c40cc640 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400c7c0 tx=0x7fe5b400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:58.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.222+0000 7fe5c1600640 1 -- 172.21.15.86:0/927229168 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4018070 con 0x7fe5c406e340 2024-04-25T14:17:58.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.222+0000 7fe5c1600640 1 -- 172.21.15.86:0/927229168 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5b400a5e0 con 0x7fe5c406e340 2024-04-25T14:17:58.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.222+0000 7fe5cb400640 1 -- 172.21.15.86:0/927229168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c40ca6d0 con 0x7fe5c406e340 2024-04-25T14:17:58.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.222+0000 7fe5c1600640 1 -- 172.21.15.86:0/927229168 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b400a9d0 con 0x7fe5c406e340 2024-04-25T14:17:58.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.222+0000 7fe5cb400640 1 -- 172.21.15.86:0/927229168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c40c9e70 con 0x7fe5c406e340 2024-04-25T14:17:58.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.222+0000 7fe5cb400640 1 -- 172.21.15.86:0/927229168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5c4072170 con 0x7fe5c406e340 2024-04-25T14:17:58.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.226+0000 7fe5c1600640 1 -- 172.21.15.86:0/927229168 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 88) ==== 93045+0+0 (secure 0 0 0) 0x7fe5b401b070 con 0x7fe5c406e340 2024-04-25T14:17:58.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.226+0000 7fe5c1600640 1 --2- 172.21.15.86:0/927229168 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fe5a006f280 0x7fe5a0071790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:58.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.226+0000 7fe5c1600640 1 -- 172.21.15.86:0/927229168 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6703+0+0 (secure 0 0 0) 0x7fe5b408e800 con 0x7fe5c406e340 2024-04-25T14:17:58.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.226+0000 7fe5bbe00640 1 --2- 172.21.15.86:0/927229168 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fe5a006f280 0x7fe5a0071790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:58.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.226+0000 7fe5bbe00640 1 --2- 172.21.15.86:0/927229168 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fe5a006f280 0x7fe5a0071790 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac0ad000 tx=0x7fe5ac008040 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:58.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.230+0000 7fe5c1600640 1 -- 172.21.15.86:0/927229168 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe5b405afc0 con 0x7fe5c406e340 2024-04-25T14:17:58.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.390+0000 7fe5cb400640 1 -- 172.21.15.86:0/927229168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe5c4076bf0 con 0x7fe5c406e340 2024-04-25T14:17:58.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.394+0000 7fe5c1600640 1 -- 172.21.15.86:0/927229168 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v157) ==== 93+0+10 (secure 0 0 0) 0x7fe5b405f2c0 con 0x7fe5c406e340 2024-04-25T14:17:58.387 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-25T14:17:58.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.394+0000 7fe5cb400640 1 -- 172.21.15.86:0/927229168 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fe5a006f280 msgr2=0x7fe5a0071790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:58.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.394+0000 7fe5cb400640 1 --2- 172.21.15.86:0/927229168 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fe5a006f280 0x7fe5a0071790 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac0ad000 tx=0x7fe5ac008040 comp rx=0 tx=0).stop 2024-04-25T14:17:58.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.394+0000 7fe5cb400640 1 -- 172.21.15.86:0/927229168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5c406e340 msgr2=0x7fe5c40cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:58.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.394+0000 7fe5cb400640 1 --2- 172.21.15.86:0/927229168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5c406e340 0x7fe5c40cc640 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400c7c0 tx=0x7fe5b400cc80 comp rx=0 tx=0).stop 2024-04-25T14:17:58.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.394+0000 7fe5cb400640 1 -- 172.21.15.86:0/927229168 shutdown_connections 2024-04-25T14:17:58.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.394+0000 7fe5cb400640 1 --2- 172.21.15.86:0/927229168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5c4163980 0x7fe5c40cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:58.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.394+0000 7fe5cb400640 1 --2- 172.21.15.86:0/927229168 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fe5a006f280 0x7fe5a0071790 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:58.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.394+0000 7fe5cb400640 1 --2- 172.21.15.86:0/927229168 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5c4072630 0x7fe5c40ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.394+0000 7fe5cb400640 1 --2- 172.21.15.86:0/927229168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5c406e340 0x7fe5c40cc640 unknown :-1 s=CLOSED pgs=2447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.394+0000 7fe5cb400640 1 -- 172.21.15.86:0/927229168 >> 172.21.15.86:0/927229168 conn(0x7fe5c40d3580 msgr2=0x7fe5c4161530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.394+0000 7fe5cb400640 1 -- 172.21.15.86:0/927229168 shutdown_connections 2024-04-25T14:17:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.394+0000 7fe5cb400640 1 -- 172.21.15.86:0/927229168 wait complete. 2024-04-25T14:17:58.408 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:17:58.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.562+0000 7f21e4e00640 1 Processor -- start 2024-04-25T14:17:58.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.566+0000 7f21e4e00640 1 -- start start 2024-04-25T14:17:58.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.566+0000 7f21e4e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f21e00b59e0 0x7f21e00b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:58.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.566+0000 7f21e4e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21e00aeea0 0x7f21e00af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:58.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.566+0000 7f21e4e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21e00b6330 con 0x7f21e015faa0 2024-04-25T14:17:58.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.566+0000 7f21e4e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f21e006d560 con 0x7f21e00b59e0 2024-04-25T14:17:58.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.566+0000 7f21e4e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f21e006c670 con 0x7f21e00aeea0 2024-04-25T14:17:58.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.566+0000 7f21dde00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f21e015faa0 0x7f21e006d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59200/0 (socket says 172.21.15.86:59200) 2024-04-25T14:17:58.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.566+0000 7f21dde00640 1 -- 172.21.15.86:0/1628621369 learned_addr learned my addr 172.21.15.86:0/1628621369 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:17:58.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.566+0000 7f21d5400640 1 --2- 172.21.15.86:0/1628621369 >> v2:172.21.15.155:3300/0 conn(0x7f21e00b59e0 0x7f21e00b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:58.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.566+0000 7f21de800640 1 --2- 172.21.15.86:0/1628621369 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21e00aeea0 0x7f21e00af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:58.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.566+0000 7f21dd400640 1 -- 172.21.15.86:0/1628621369 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 694001207 0 0) 0x7f21d0001120 con 0x7f21e015faa0 2024-04-25T14:17:58.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.566+0000 7f21dd400640 1 -- 172.21.15.86:0/1628621369 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21c4003530 con 0x7f21e015faa0 2024-04-25T14:17:58.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.566+0000 7f21de800640 1 -- 172.21.15.86:0/1628621369 >> v2:172.21.15.155:3300/0 conn(0x7f21e00b59e0 msgr2=0x7f21e00b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:58.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.566+0000 7f21de800640 1 --2- 172.21.15.86:0/1628621369 >> v2:172.21.15.155:3300/0 conn(0x7f21e00b59e0 0x7f21e00b5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:58.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.566+0000 7f21de800640 1 -- 172.21.15.86:0/1628621369 >> v1:172.21.15.86:6789/0 conn(0x7f21e015faa0 legacy=0x7f21e006d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:58.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.566+0000 7f21de800640 1 -- 172.21.15.86:0/1628621369 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e006c950 con 0x7f21e00aeea0 2024-04-25T14:17:58.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.566+0000 7f21d5400640 1 --2- 172.21.15.86:0/1628621369 >> v2:172.21.15.155:3300/0 conn(0x7f21e00b59e0 0x7f21e00b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:17:58.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.570+0000 7f21de800640 1 --2- 172.21.15.86:0/1628621369 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21e00aeea0 0x7f21e00af2b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f21d8009d90 tx=0x7f21d801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=656e0e6e23342615 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:58.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.570+0000 7f21dd400640 1 -- 172.21.15.86:0/1628621369 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d80b2070 con 0x7f21e00aeea0 2024-04-25T14:17:58.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.570+0000 7f21dd400640 1 -- 172.21.15.86:0/1628621369 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21d80a5a50 con 0x7f21e00aeea0 2024-04-25T14:17:58.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.570+0000 7f21dd400640 1 -- 172.21.15.86:0/1628621369 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d80a5e20 con 0x7f21e00aeea0 2024-04-25T14:17:58.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.570+0000 7f21e4e00640 1 -- 172.21.15.86:0/1628621369 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21e00aeea0 msgr2=0x7f21e00af2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:58.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.570+0000 7f21e4e00640 1 --2- 172.21.15.86:0/1628621369 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21e00aeea0 0x7f21e00af2b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f21d8009d90 tx=0x7f21d801c970 comp rx=0 tx=0).stop 2024-04-25T14:17:58.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.570+0000 7f21e4e00640 1 -- 172.21.15.86:0/1628621369 shutdown_connections 2024-04-25T14:17:58.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.570+0000 7f21e4e00640 1 --2- 172.21.15.86:0/1628621369 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21e00aeea0 0x7f21e00af2b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:58.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.570+0000 7f21e4e00640 1 --2- 172.21.15.86:0/1628621369 >> v2:172.21.15.155:3300/0 conn(0x7f21e00b59e0 0x7f21e00b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:58.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.570+0000 7f21e4e00640 1 -- 172.21.15.86:0/1628621369 >> 172.21.15.86:0/1628621369 conn(0x7f21e00d2230 msgr2=0x7f21e00d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:58.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.570+0000 7f21e4e00640 1 -- 172.21.15.86:0/1628621369 shutdown_connections 2024-04-25T14:17:58.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.570+0000 7f21e4e00640 1 -- 172.21.15.86:0/1628621369 wait complete. 2024-04-25T14:17:58.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.574+0000 7f21e4e00640 1 Processor -- start 2024-04-25T14:17:58.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.574+0000 7f21e4e00640 1 -- start start 2024-04-25T14:17:58.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.574+0000 7f21e4e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21e00aeea0 0x7f21e01553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:58.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.574+0000 7f21e4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21e00b59e0 0x7f21e0157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:58.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.574+0000 7f21e4e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21e015faa0 0x7f21e014eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:58.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.574+0000 7f21e4e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21e0160030 con 0x7f21e00b59e0 2024-04-25T14:17:58.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.574+0000 7f21e4e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f21e015feb0 con 0x7f21e00aeea0 2024-04-25T14:17:58.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.574+0000 7f21e4e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f21e0070f30 con 0x7f21e015faa0 2024-04-25T14:17:58.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.574+0000 7f21d5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21e00b59e0 0x7f21e0157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:58.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.574+0000 7f21d5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21e00b59e0 0x7f21e0157170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59288/0 (socket says 172.21.15.86:59288) 2024-04-25T14:17:58.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.574+0000 7f21d5400640 1 -- 172.21.15.86:0/3748970557 learned_addr learned my addr 172.21.15.86:0/3748970557 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:58.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.574+0000 7f21dde00640 1 --2- 172.21.15.86:0/3748970557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21e00aeea0 0x7f21e01553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:58.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.574+0000 7f21de800640 1 --2- 172.21.15.86:0/3748970557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21e015faa0 0x7f21e014eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:58.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.574+0000 7f21d5400640 1 -- 172.21.15.86:0/3748970557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21e015faa0 msgr2=0x7f21e014eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:58.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.574+0000 7f21d5400640 1 --2- 172.21.15.86:0/3748970557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21e015faa0 0x7f21e014eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:58.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.574+0000 7f21d5400640 1 -- 172.21.15.86:0/3748970557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21e00aeea0 msgr2=0x7f21e01553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:58.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.574+0000 7f21d5400640 1 --2- 172.21.15.86:0/3748970557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21e00aeea0 0x7f21e01553d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:58.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.574+0000 7f21d5400640 1 -- 172.21.15.86:0/3748970557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e0155910 con 0x7f21e00b59e0 2024-04-25T14:17:58.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.574+0000 7f21dde00640 1 --2- 172.21.15.86:0/3748970557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21e00aeea0 0x7f21e01553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:17:58.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.574+0000 7f21de800640 1 --2- 172.21.15.86:0/3748970557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21e015faa0 0x7f21e014eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:17:58.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.574+0000 7f21d5400640 1 --2- 172.21.15.86:0/3748970557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21e00b59e0 0x7f21e0157170 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f21c800e9b0 tx=0x7f21c800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:58.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.574+0000 7f21d6a00640 1 -- 172.21.15.86:0/3748970557 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c8017070 con 0x7f21e00b59e0 2024-04-25T14:17:58.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.578+0000 7f21d6a00640 1 -- 172.21.15.86:0/3748970557 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21c800ca00 con 0x7f21e00b59e0 2024-04-25T14:17:58.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.578+0000 7f21d6a00640 1 -- 172.21.15.86:0/3748970557 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c800cdd0 con 0x7f21e00b59e0 2024-04-25T14:17:58.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.578+0000 7f21e4e00640 1 -- 172.21.15.86:0/3748970557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e01560d0 con 0x7f21e00b59e0 2024-04-25T14:17:58.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.578+0000 7f21e4e00640 1 -- 172.21.15.86:0/3748970557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e0156690 con 0x7f21e00b59e0 2024-04-25T14:17:58.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.578+0000 7f21e4e00640 1 -- 172.21.15.86:0/3748970557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21e0155e40 con 0x7f21e00b59e0 2024-04-25T14:17:58.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.578+0000 7f21d6a00640 1 -- 172.21.15.86:0/3748970557 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 88) ==== 93045+0+0 (secure 0 0 0) 0x7f21c801a070 con 0x7f21e00b59e0 2024-04-25T14:17:58.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.578+0000 7f21d6a00640 1 --2- 172.21.15.86:0/3748970557 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f21bc06f250 0x7f21bc071760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:58.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.578+0000 7f21d6a00640 1 -- 172.21.15.86:0/3748970557 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6703+0+0 (secure 0 0 0) 0x7f21c808f2b0 con 0x7f21e00b59e0 2024-04-25T14:17:58.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.578+0000 7f21dde00640 1 --2- 172.21.15.86:0/3748970557 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f21bc06f250 0x7f21bc071760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:58.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.582+0000 7f21dde00640 1 --2- 172.21.15.86:0/3748970557 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f21bc06f250 0x7f21bc071760 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f21d00012c0 tx=0x7f21d0009040 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:58.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.582+0000 7f21d6a00640 1 -- 172.21.15.86:0/3748970557 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f21c805c030 con 0x7f21e00b59e0 2024-04-25T14:17:58.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.738+0000 7f21e4e00640 1 -- 172.21.15.86:0/3748970557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f21e0074cf0 con 0x7f21e00b59e0 2024-04-25T14:17:58.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.738+0000 7f21d6a00640 1 -- 172.21.15.86:0/3748970557 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v157) ==== 104+0+11 (secure 0 0 0) 0x7f21c805b930 con 0x7f21e00b59e0 2024-04-25T14:17:58.735 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-25T14:17:58.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.742+0000 7f21e4e00640 1 -- 172.21.15.86:0/3748970557 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f21bc06f250 msgr2=0x7f21bc071760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:58.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.742+0000 7f21e4e00640 1 --2- 172.21.15.86:0/3748970557 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f21bc06f250 0x7f21bc071760 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f21d00012c0 tx=0x7f21d0009040 comp rx=0 tx=0).stop 2024-04-25T14:17:58.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.742+0000 7f21e4e00640 1 -- 172.21.15.86:0/3748970557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21e00b59e0 msgr2=0x7f21e0157170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:58.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.742+0000 7f21e4e00640 1 --2- 172.21.15.86:0/3748970557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21e00b59e0 0x7f21e0157170 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f21c800e9b0 tx=0x7f21c800ee70 comp rx=0 tx=0).stop 2024-04-25T14:17:58.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.742+0000 7f21e4e00640 1 -- 172.21.15.86:0/3748970557 shutdown_connections 2024-04-25T14:17:58.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.742+0000 7f21e4e00640 1 --2- 172.21.15.86:0/3748970557 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f21bc06f250 0x7f21bc071760 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:58.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.742+0000 7f21e4e00640 1 --2- 172.21.15.86:0/3748970557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21e015faa0 0x7f21e014eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:58.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.742+0000 7f21e4e00640 1 --2- 172.21.15.86:0/3748970557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21e00b59e0 0x7f21e0157170 unknown :-1 s=CLOSED pgs=2449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:58.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.742+0000 7f21e4e00640 1 --2- 172.21.15.86:0/3748970557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21e00aeea0 0x7f21e01553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:58.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.742+0000 7f21e4e00640 1 -- 172.21.15.86:0/3748970557 >> 172.21.15.86:0/3748970557 conn(0x7f21e00d2230 msgr2=0x7f21e00b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:58.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.742+0000 7f21e4e00640 1 -- 172.21.15.86:0/3748970557 shutdown_connections 2024-04-25T14:17:58.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.742+0000 7f21e4e00640 1 -- 172.21.15.86:0/3748970557 wait complete. 2024-04-25T14:17:58.756 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:17:58.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.910+0000 7f9138a00640 1 Processor -- start 2024-04-25T14:17:58.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.910+0000 7f9138a00640 1 -- start start 2024-04-25T14:17:58.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.914+0000 7f9138a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f913415faa0 0x7f913406d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:58.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.914+0000 7f9138a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f913406c670 con 0x7f91340b59e0 2024-04-25T14:17:58.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.914+0000 7f9138a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f913406c950 con 0x7f91340aeea0 2024-04-25T14:17:58.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.914+0000 7f9138a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f913406d560 con 0x7f913415faa0 2024-04-25T14:17:58.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.914+0000 7f9131400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f91340b59e0 0x7f91340b5df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60482/0 (socket says 172.21.15.86:60482) 2024-04-25T14:17:58.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.914+0000 7f9131400640 1 -- 172.21.15.86:0/3172681682 learned_addr learned my addr 172.21.15.86:0/3172681682 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:17:58.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.914+0000 7f9131e00640 1 --2- 172.21.15.86:0/3172681682 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f913415faa0 0x7f913406d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.914+0000 7f9130a00640 1 -- 172.21.15.86:0/3172681682 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1296402273 0 0) 0x7f913406c670 con 0x7f91340b59e0 2024-04-25T14:17:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.914+0000 7f9130a00640 1 -- 172.21.15.86:0/3172681682 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9118003530 con 0x7f91340b59e0 2024-04-25T14:17:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.914+0000 7f9130a00640 1 -- 172.21.15.86:0/3172681682 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2350517433 0 0) 0x7f913406c950 con 0x7f91340aeea0 2024-04-25T14:17:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.914+0000 7f9130a00640 1 -- 172.21.15.86:0/3172681682 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f913406c670 con 0x7f91340aeea0 2024-04-25T14:17:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.914+0000 7f9131e00640 1 -- 172.21.15.86:0/3172681682 >> v1:172.21.15.155:6789/0 conn(0x7f91340aeea0 legacy=0x7f91340af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.914+0000 7f9131e00640 1 -- 172.21.15.86:0/3172681682 >> v1:172.21.15.86:6789/0 conn(0x7f91340b59e0 legacy=0x7f91340b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:58.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.914+0000 7f9131e00640 1 -- 172.21.15.86:0/3172681682 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f913406cc30 con 0x7f913415faa0 2024-04-25T14:17:58.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.918+0000 7f9131e00640 1 --2- 172.21.15.86:0/3172681682 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f913415faa0 0x7f913406d9f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9124009dc0 tx=0x7f912401c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f36c181d2d999b35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:58.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.918+0000 7f9130a00640 1 -- 172.21.15.86:0/3172681682 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91240b2070 con 0x7f913415faa0 2024-04-25T14:17:58.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.918+0000 7f9130a00640 1 -- 172.21.15.86:0/3172681682 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91240a5a50 con 0x7f913415faa0 2024-04-25T14:17:58.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.918+0000 7f9130a00640 1 -- 172.21.15.86:0/3172681682 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91240a5e20 con 0x7f913415faa0 2024-04-25T14:17:58.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.918+0000 7f9138a00640 1 -- 172.21.15.86:0/3172681682 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f913415faa0 msgr2=0x7f913406d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:58.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.918+0000 7f9138a00640 1 --2- 172.21.15.86:0/3172681682 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f913415faa0 0x7f913406d9f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9124009dc0 tx=0x7f912401c990 comp rx=0 tx=0).stop 2024-04-25T14:17:58.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.918+0000 7f9138a00640 1 -- 172.21.15.86:0/3172681682 shutdown_connections 2024-04-25T14:17:58.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.918+0000 7f9138a00640 1 --2- 172.21.15.86:0/3172681682 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f913415faa0 0x7f913406d9f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:58.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.918+0000 7f9138a00640 1 -- 172.21.15.86:0/3172681682 >> 172.21.15.86:0/3172681682 conn(0x7f91340d2230 msgr2=0x7f91340d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:58.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.918+0000 7f9138a00640 1 -- 172.21.15.86:0/3172681682 shutdown_connections 2024-04-25T14:17:58.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.918+0000 7f9138a00640 1 -- 172.21.15.86:0/3172681682 wait complete. 2024-04-25T14:17:58.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.918+0000 7f9138a00640 1 Processor -- start 2024-04-25T14:17:58.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.922+0000 7f9138a00640 1 -- start start 2024-04-25T14:17:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.922+0000 7f9138a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91340aeea0 0x7f9134149680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.922+0000 7f9138a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91340b59e0 0x7f913414e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.922+0000 7f9138a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f913415faa0 0x7f913414e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.922+0000 7f9138a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f913415feb0 con 0x7f91340aeea0 2024-04-25T14:17:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.922+0000 7f9138a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f913406df30 con 0x7f913415faa0 2024-04-25T14:17:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.922+0000 7f9138a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9134160030 con 0x7f91340b59e0 2024-04-25T14:17:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.922+0000 7f9131e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91340aeea0 0x7f9134149680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.922+0000 7f9131e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91340aeea0 0x7f9134149680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35500/0 (socket says 172.21.15.86:35500) 2024-04-25T14:17:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.922+0000 7f9131e00640 1 -- 172.21.15.86:0/83063180 learned_addr learned my addr 172.21.15.86:0/83063180 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.922+0000 7f9131400640 1 --2- 172.21.15.86:0/83063180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91340b59e0 0x7f913414e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.922+0000 7f9132800640 1 --2- 172.21.15.86:0/83063180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f913415faa0 0x7f913414e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.922+0000 7f9131e00640 1 -- 172.21.15.86:0/83063180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91340b59e0 msgr2=0x7f913414e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.922+0000 7f9131e00640 1 --2- 172.21.15.86:0/83063180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91340b59e0 0x7f913414e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.922+0000 7f9131e00640 1 -- 172.21.15.86:0/83063180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f913415faa0 msgr2=0x7f913414e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.922+0000 7f9131e00640 1 --2- 172.21.15.86:0/83063180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f913415faa0 0x7f913414e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.922+0000 7f9131e00640 1 -- 172.21.15.86:0/83063180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f913414cd80 con 0x7f91340aeea0 2024-04-25T14:17:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.922+0000 7f9131e00640 1 --2- 172.21.15.86:0/83063180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91340aeea0 0x7f9134149680 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f91240195a0 tx=0x7f9124019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.922+0000 7f912a000640 1 -- 172.21.15.86:0/83063180 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91240b2070 con 0x7f91340aeea0 2024-04-25T14:17:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.922+0000 7f912a000640 1 -- 172.21.15.86:0/83063180 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91240bf070 con 0x7f91340aeea0 2024-04-25T14:17:58.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.922+0000 7f912a000640 1 -- 172.21.15.86:0/83063180 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91240ade50 con 0x7f91340aeea0 2024-04-25T14:17:58.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.922+0000 7f9138a00640 1 -- 172.21.15.86:0/83063180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f913415ef40 con 0x7f91340aeea0 2024-04-25T14:17:58.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.922+0000 7f9138a00640 1 -- 172.21.15.86:0/83063180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f913414f210 con 0x7f91340aeea0 2024-04-25T14:17:58.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.922+0000 7f9138a00640 1 -- 172.21.15.86:0/83063180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f913414ed50 con 0x7f91340aeea0 2024-04-25T14:17:58.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.926+0000 7f912a000640 1 -- 172.21.15.86:0/83063180 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 88) ==== 93045+0+0 (secure 0 0 0) 0x7f91240b9480 con 0x7f91340aeea0 2024-04-25T14:17:58.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.926+0000 7f912a000640 1 --2- 172.21.15.86:0/83063180 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f911406f3f0 0x7f9114071900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:58.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.926+0000 7f912a000640 1 -- 172.21.15.86:0/83063180 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6703+0+0 (secure 0 0 0) 0x7f912412a4c0 con 0x7f91340aeea0 2024-04-25T14:17:58.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.926+0000 7f9131400640 1 --2- 172.21.15.86:0/83063180 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f911406f3f0 0x7f9114071900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:58.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.926+0000 7f9131400640 1 --2- 172.21.15.86:0/83063180 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f911406f3f0 0x7f9114071900 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f91341615d0 tx=0x7f9120009040 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:58.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:58.930+0000 7f912a000640 1 -- 172.21.15.86:0/83063180 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f913414ed50 con 0x7f91340aeea0 2024-04-25T14:17:59.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.090+0000 7f9138a00640 1 -- 172.21.15.86:0/83063180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9134074cf0 con 0x7f91340aeea0 2024-04-25T14:17:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.090+0000 7f912a000640 1 -- 172.21.15.86:0/83063180 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v157) ==== 100+0+11 (secure 0 0 0) 0x7f9124016580 con 0x7f91340aeea0 2024-04-25T14:17:59.085 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-25T14:17:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.090+0000 7f9138a00640 1 -- 172.21.15.86:0/83063180 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f911406f3f0 msgr2=0x7f9114071900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.090+0000 7f9138a00640 1 --2- 172.21.15.86:0/83063180 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f911406f3f0 0x7f9114071900 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f91341615d0 tx=0x7f9120009040 comp rx=0 tx=0).stop 2024-04-25T14:17:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.090+0000 7f9138a00640 1 -- 172.21.15.86:0/83063180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91340aeea0 msgr2=0x7f9134149680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.090+0000 7f9138a00640 1 --2- 172.21.15.86:0/83063180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91340aeea0 0x7f9134149680 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f91240195a0 tx=0x7f9124019730 comp rx=0 tx=0).stop 2024-04-25T14:17:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.090+0000 7f9138a00640 1 -- 172.21.15.86:0/83063180 shutdown_connections 2024-04-25T14:17:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.090+0000 7f9138a00640 1 --2- 172.21.15.86:0/83063180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f913415faa0 0x7f913414e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.090+0000 7f9138a00640 1 --2- 172.21.15.86:0/83063180 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f911406f3f0 0x7f9114071900 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.090+0000 7f9138a00640 1 --2- 172.21.15.86:0/83063180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91340b59e0 0x7f913414e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.090+0000 7f9138a00640 1 --2- 172.21.15.86:0/83063180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91340aeea0 0x7f9134149680 unknown :-1 s=CLOSED pgs=2451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.094+0000 7f9138a00640 1 -- 172.21.15.86:0/83063180 >> 172.21.15.86:0/83063180 conn(0x7f91340d2230 msgr2=0x7f91340b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.094+0000 7f9138a00640 1 -- 172.21.15.86:0/83063180 shutdown_connections 2024-04-25T14:17:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.094+0000 7f9138a00640 1 -- 172.21.15.86:0/83063180 wait complete. 2024-04-25T14:17:59.106 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:59.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.262+0000 7f9810200640 1 Processor -- start 2024-04-25T14:17:59.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.262+0000 7f9810200640 1 -- start start 2024-04-25T14:17:59.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.266+0000 7f9810200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9808070330 0x7f98080d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:59.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.266+0000 7f9810200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98080ce790 0x7f980806f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:59.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.266+0000 7f9810200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98080d2030 con 0x7f98080d1c20 2024-04-25T14:17:59.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.266+0000 7f9810200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f980815e5c0 con 0x7f9808070330 2024-04-25T14:17:59.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.266+0000 7f9810200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f980815e8a0 con 0x7f98080ce790 2024-04-25T14:17:59.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.266+0000 7f980d800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9808070330 0x7f98080d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:59.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.266+0000 7f980ce00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f98080d1c20 0x7f980806ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60486/0 (socket says 172.21.15.86:60486) 2024-04-25T14:17:59.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.266+0000 7f980ce00640 1 -- 172.21.15.86:0/3084191737 learned_addr learned my addr 172.21.15.86:0/3084191737 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:17:59.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.266+0000 7f980e200640 1 --2- 172.21.15.86:0/3084191737 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98080ce790 0x7f980806f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:59.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.266+0000 7f980d800640 1 -- 172.21.15.86:0/3084191737 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98080ce790 msgr2=0x7f980806f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:17:59.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.266+0000 7f980d800640 1 --2- 172.21.15.86:0/3084191737 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98080ce790 0x7f980806f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:59.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.266+0000 7f980d800640 1 -- 172.21.15.86:0/3084191737 >> v1:172.21.15.86:6789/0 conn(0x7f98080d1c20 legacy=0x7f980806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:59.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.266+0000 7f980d800640 1 -- 172.21.15.86:0/3084191737 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f980815eb80 con 0x7f9808070330 2024-04-25T14:17:59.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.266+0000 7f9807e00640 1 -- 172.21.15.86:0/3084191737 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2323995380 0 0) 0x7f97f4001120 con 0x7f98080d1c20 2024-04-25T14:17:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.270+0000 7f980d800640 1 --2- 172.21.15.86:0/3084191737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9808070330 0x7f98080d16e0 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f97fc0097c0 tx=0x7f97fc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=453c8837fd3555ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.270+0000 7f9807e00640 1 -- 172.21.15.86:0/3084191737 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc0b0070 con 0x7f9808070330 2024-04-25T14:17:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.270+0000 7f9807e00640 1 -- 172.21.15.86:0/3084191737 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97fc0ab600 con 0x7f9808070330 2024-04-25T14:17:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.270+0000 7f9807e00640 1 -- 172.21.15.86:0/3084191737 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc0ab9d0 con 0x7f9808070330 2024-04-25T14:17:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.270+0000 7f9810200640 1 -- 172.21.15.86:0/3084191737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9808070330 msgr2=0x7f98080d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.270+0000 7f9810200640 1 --2- 172.21.15.86:0/3084191737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9808070330 0x7f98080d16e0 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f97fc0097c0 tx=0x7f97fc01c410 comp rx=0 tx=0).stop 2024-04-25T14:17:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.270+0000 7f9810200640 1 -- 172.21.15.86:0/3084191737 shutdown_connections 2024-04-25T14:17:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.270+0000 7f9810200640 1 --2- 172.21.15.86:0/3084191737 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98080ce790 0x7f980806f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.270+0000 7f9810200640 1 --2- 172.21.15.86:0/3084191737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9808070330 0x7f98080d16e0 unknown :-1 s=CLOSED pgs=2455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.270+0000 7f9810200640 1 -- 172.21.15.86:0/3084191737 >> 172.21.15.86:0/3084191737 conn(0x7f98080b4650 msgr2=0x7f98080b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.270+0000 7f9810200640 1 -- 172.21.15.86:0/3084191737 shutdown_connections 2024-04-25T14:17:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.270+0000 7f9810200640 1 -- 172.21.15.86:0/3084191737 wait complete. 2024-04-25T14:17:59.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.274+0000 7f9810200640 1 Processor -- start 2024-04-25T14:17:59.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.274+0000 7f9810200640 1 -- start start 2024-04-25T14:17:59.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.274+0000 7f9810200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9808070330 0x7f9808072880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:59.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.274+0000 7f9810200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98080ce790 0x7f9808155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:59.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.274+0000 7f9810200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98080d1c20 0x7f98081595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:59.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.274+0000 7f9810200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98080708c0 con 0x7f9808070330 2024-04-25T14:17:59.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.274+0000 7f9810200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9808070740 con 0x7f98080ce790 2024-04-25T14:17:59.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.274+0000 7f9810200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f980816afc0 con 0x7f98080d1c20 2024-04-25T14:17:59.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.274+0000 7f980d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9808070330 0x7f9808072880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:59.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.274+0000 7f980d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9808070330 0x7f9808072880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35540/0 (socket says 172.21.15.86:35540) 2024-04-25T14:17:59.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.274+0000 7f980d800640 1 -- 172.21.15.86:0/3941675406 learned_addr learned my addr 172.21.15.86:0/3941675406 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:59.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.274+0000 7f980ce00640 1 --2- 172.21.15.86:0/3941675406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98080ce790 0x7f9808155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:59.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.274+0000 7f980e200640 1 --2- 172.21.15.86:0/3941675406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98080d1c20 0x7f98081595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:59.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.274+0000 7f980d800640 1 -- 172.21.15.86:0/3941675406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98080d1c20 msgr2=0x7f98081595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:59.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.274+0000 7f980d800640 1 --2- 172.21.15.86:0/3941675406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98080d1c20 0x7f98081595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:59.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.274+0000 7f980d800640 1 -- 172.21.15.86:0/3941675406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98080ce790 msgr2=0x7f9808155130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:59.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.274+0000 7f980d800640 1 --2- 172.21.15.86:0/3941675406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98080ce790 0x7f9808155130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:59.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.274+0000 7f980d800640 1 -- 172.21.15.86:0/3941675406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9808098110 con 0x7f9808070330 2024-04-25T14:17:59.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.274+0000 7f980ce00640 1 --2- 172.21.15.86:0/3941675406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98080ce790 0x7f9808155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:17:59.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.274+0000 7f980d800640 1 --2- 172.21.15.86:0/3941675406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9808070330 0x7f9808072880 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f97fc01a040 tx=0x7f97fc01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:59.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.274+0000 7f9805600640 1 -- 172.21.15.86:0/3941675406 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc0b0050 con 0x7f9808070330 2024-04-25T14:17:59.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.274+0000 7f9805600640 1 -- 172.21.15.86:0/3941675406 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97fc0bc070 con 0x7f9808070330 2024-04-25T14:17:59.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.274+0000 7f9810200640 1 -- 172.21.15.86:0/3941675406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98081561d0 con 0x7f9808070330 2024-04-25T14:17:59.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.274+0000 7f9805600640 1 -- 172.21.15.86:0/3941675406 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc0b5830 con 0x7f9808070330 2024-04-25T14:17:59.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.274+0000 7f9810200640 1 -- 172.21.15.86:0/3941675406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f980809ae50 con 0x7f9808070330 2024-04-25T14:17:59.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.278+0000 7f9810200640 1 -- 172.21.15.86:0/3941675406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9808069d00 con 0x7f9808070330 2024-04-25T14:17:59.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.278+0000 7f9805600640 1 -- 172.21.15.86:0/3941675406 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 88) ==== 93045+0+0 (secure 0 0 0) 0x7f97fc0b59d0 con 0x7f9808070330 2024-04-25T14:17:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.278+0000 7f9805600640 1 --2- 172.21.15.86:0/3941675406 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f97e806f250 0x7f97e8071760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.278+0000 7f9805600640 1 -- 172.21.15.86:0/3941675406 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6703+0+0 (secure 0 0 0) 0x7f97fc129b50 con 0x7f9808070330 2024-04-25T14:17:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.278+0000 7f980ce00640 1 --2- 172.21.15.86:0/3941675406 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f97e806f250 0x7f97e8071760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:59.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.278+0000 7f980ce00640 1 --2- 172.21.15.86:0/3941675406 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f97e806f250 0x7f97e8071760 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f98080b2d90 tx=0x7f97f4007cd0 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:59.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.282+0000 7f9805600640 1 -- 172.21.15.86:0/3941675406 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f97fc016090 con 0x7f9808070330 2024-04-25T14:17:59.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.474+0000 7f9810200640 1 -- 172.21.15.86:0/3941675406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f980806f5f0 con 0x7f9808070330 2024-04-25T14:17:59.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.474+0000 7f9805600640 1 -- 172.21.15.86:0/3941675406 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f97fc0f6960 con 0x7f9808070330 2024-04-25T14:17:59.468 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:59.468 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:17:59.468 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:17:59.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.474+0000 7f9810200640 1 -- 172.21.15.86:0/3941675406 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f97e806f250 msgr2=0x7f97e8071760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:59.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.474+0000 7f9810200640 1 --2- 172.21.15.86:0/3941675406 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f97e806f250 0x7f97e8071760 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f98080b2d90 tx=0x7f97f4007cd0 comp rx=0 tx=0).stop 2024-04-25T14:17:59.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.474+0000 7f9810200640 1 -- 172.21.15.86:0/3941675406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9808070330 msgr2=0x7f9808072880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:59.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.474+0000 7f9810200640 1 --2- 172.21.15.86:0/3941675406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9808070330 0x7f9808072880 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f97fc01a040 tx=0x7f97fc01ac50 comp rx=0 tx=0).stop 2024-04-25T14:17:59.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.474+0000 7f9810200640 1 -- 172.21.15.86:0/3941675406 shutdown_connections 2024-04-25T14:17:59.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.474+0000 7f9810200640 1 --2- 172.21.15.86:0/3941675406 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f97e806f250 0x7f97e8071760 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:59.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.474+0000 7f9810200640 1 --2- 172.21.15.86:0/3941675406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98080d1c20 0x7f98081595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:59.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.474+0000 7f9810200640 1 --2- 172.21.15.86:0/3941675406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98080ce790 0x7f9808155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:59.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.474+0000 7f9810200640 1 --2- 172.21.15.86:0/3941675406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9808070330 0x7f9808072880 unknown :-1 s=CLOSED pgs=2456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:59.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.474+0000 7f9810200640 1 -- 172.21.15.86:0/3941675406 >> 172.21.15.86:0/3941675406 conn(0x7f98080b4650 msgr2=0x7f980815ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:59.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.474+0000 7f9810200640 1 -- 172.21.15.86:0/3941675406 shutdown_connections 2024-04-25T14:17:59.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.474+0000 7f9810200640 1 -- 172.21.15.86:0/3941675406 wait complete. 2024-04-25T14:17:59.489 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:17:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.642+0000 7f7227600640 1 Processor -- start 2024-04-25T14:17:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.642+0000 7f7227600640 1 -- start start 2024-04-25T14:17:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.642+0000 7f7227600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f72200d1df0 0x7f72200cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:59.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.642+0000 7f7227600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f722015e5c0 con 0x7f7220070330 2024-04-25T14:17:59.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.642+0000 7f7227600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f722015e8a0 con 0x7f72200cdcc0 2024-04-25T14:17:59.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.642+0000 7f7227600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f722006fd40 con 0x7f72200d1df0 2024-04-25T14:17:59.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.642+0000 7f7217e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f72200d1df0 0x7f72200cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:59.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.642+0000 7f7217e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f72200d1df0 0x7f72200cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35554/0 (socket says 172.21.15.86:35554) 2024-04-25T14:17:59.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.642+0000 7f7217e00640 1 -- 172.21.15.86:0/3280195031 learned_addr learned my addr 172.21.15.86:0/3280195031 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:59.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.642+0000 7f7217e00640 1 -- 172.21.15.86:0/3280195031 >> v1:172.21.15.155:6789/0 conn(0x7f72200cdcc0 legacy=0x7f72200ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:59.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.642+0000 7f7217e00640 1 -- 172.21.15.86:0/3280195031 >> v1:172.21.15.86:6789/0 conn(0x7f7220070330 legacy=0x7f72200d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:59.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.642+0000 7f7217e00640 1 -- 172.21.15.86:0/3280195031 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f722015eb80 con 0x7f72200d1df0 2024-04-25T14:17:59.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.646+0000 7f7217e00640 1 --2- 172.21.15.86:0/3280195031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72200d1df0 0x7f72200cd780 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f72080098a0 tx=0x7f720801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5626705292980e50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:59.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.646+0000 7f721fe00640 1 -- 172.21.15.86:0/3280195031 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72080b1070 con 0x7f72200d1df0 2024-04-25T14:17:59.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.646+0000 7f721fe00640 1 -- 172.21.15.86:0/3280195031 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72080ac5f0 con 0x7f72200d1df0 2024-04-25T14:17:59.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.646+0000 7f721fe00640 1 -- 172.21.15.86:0/3280195031 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72080ac9c0 con 0x7f72200d1df0 2024-04-25T14:17:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.646+0000 7f7227600640 1 -- 172.21.15.86:0/3280195031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72200d1df0 msgr2=0x7f72200cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.646+0000 7f7227600640 1 --2- 172.21.15.86:0/3280195031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72200d1df0 0x7f72200cd780 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f72080098a0 tx=0x7f720801c480 comp rx=0 tx=0).stop 2024-04-25T14:17:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.646+0000 7f7227600640 1 -- 172.21.15.86:0/3280195031 shutdown_connections 2024-04-25T14:17:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.646+0000 7f7227600640 1 --2- 172.21.15.86:0/3280195031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72200d1df0 0x7f72200cd780 unknown :-1 s=CLOSED pgs=2458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.646+0000 7f7227600640 1 -- 172.21.15.86:0/3280195031 >> 172.21.15.86:0/3280195031 conn(0x7f72200b4650 msgr2=0x7f72200b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.646+0000 7f7227600640 1 -- 172.21.15.86:0/3280195031 shutdown_connections 2024-04-25T14:17:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.646+0000 7f7227600640 1 -- 172.21.15.86:0/3280195031 wait complete. 2024-04-25T14:17:59.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.650+0000 7f7227600640 1 Processor -- start 2024-04-25T14:17:59.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.650+0000 7f7227600640 1 -- start start 2024-04-25T14:17:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.650+0000 7f7227600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7220070330 0x7f7220155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.650+0000 7f7227600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72200cdcc0 0x7f72201562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.650+0000 7f7227600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f722015c900 0x7f7220157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.650+0000 7f7227600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72200708c0 con 0x7f722015c900 2024-04-25T14:17:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.650+0000 7f7227600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7220070740 con 0x7f72200cdcc0 2024-04-25T14:17:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.650+0000 7f7227600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f722016bfd0 con 0x7f7220070330 2024-04-25T14:17:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.650+0000 7f7225600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f722015c900 0x7f7220157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.650+0000 7f7225600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f722015c900 0x7f7220157d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35560/0 (socket says 172.21.15.86:35560) 2024-04-25T14:17:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.650+0000 7f7225600640 1 -- 172.21.15.86:0/1562659589 learned_addr learned my addr 172.21.15.86:0/1562659589 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.650+0000 7f7217e00640 1 --2- 172.21.15.86:0/1562659589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72200cdcc0 0x7f72201562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:59.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.650+0000 7f7224c00640 1 --2- 172.21.15.86:0/1562659589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7220070330 0x7f7220155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:59.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.650+0000 7f7225600640 1 -- 172.21.15.86:0/1562659589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7220070330 msgr2=0x7f7220155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:59.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.650+0000 7f7225600640 1 --2- 172.21.15.86:0/1562659589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7220070330 0x7f7220155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:59.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.650+0000 7f7225600640 1 -- 172.21.15.86:0/1562659589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72200cdcc0 msgr2=0x7f72201562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:59.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.650+0000 7f7225600640 1 --2- 172.21.15.86:0/1562659589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72200cdcc0 0x7f72201562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:59.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.650+0000 7f7225600640 1 -- 172.21.15.86:0/1562659589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f722009efc0 con 0x7f722015c900 2024-04-25T14:17:59.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.650+0000 7f7224c00640 1 --2- 172.21.15.86:0/1562659589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7220070330 0x7f7220155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:17:59.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.650+0000 7f7225600640 1 --2- 172.21.15.86:0/1562659589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f722015c900 0x7f7220157d80 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f721800c760 tx=0x7f721800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.650+0000 7f721d600640 1 -- 172.21.15.86:0/1562659589 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7218003b80 con 0x7f722015c900 2024-04-25T14:17:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.650+0000 7f721d600640 1 -- 172.21.15.86:0/1562659589 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7218003d20 con 0x7f722015c900 2024-04-25T14:17:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.650+0000 7f7227600640 1 -- 172.21.15.86:0/1562659589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72200cb2a0 con 0x7f722015c900 2024-04-25T14:17:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.650+0000 7f721d600640 1 -- 172.21.15.86:0/1562659589 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721800aa90 con 0x7f722015c900 2024-04-25T14:17:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.654+0000 7f7227600640 1 -- 172.21.15.86:0/1562659589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72200af800 con 0x7f722015c900 2024-04-25T14:17:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.654+0000 7f7227600640 1 -- 172.21.15.86:0/1562659589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72200cb7c0 con 0x7f722015c900 2024-04-25T14:17:59.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.654+0000 7f721d600640 1 -- 172.21.15.86:0/1562659589 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 88) ==== 93045+0+0 (secure 0 0 0) 0x7f721800ac30 con 0x7f722015c900 2024-04-25T14:17:59.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.654+0000 7f721d600640 1 --2- 172.21.15.86:0/1562659589 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f71fc06f250 0x7f71fc071760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:59.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.654+0000 7f721d600640 1 -- 172.21.15.86:0/1562659589 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6703+0+0 (secure 0 0 0) 0x7f721808f0d0 con 0x7f722015c900 2024-04-25T14:17:59.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.654+0000 7f7224c00640 1 --2- 172.21.15.86:0/1562659589 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f71fc06f250 0x7f71fc071760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:59.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.654+0000 7f7224c00640 1 --2- 172.21.15.86:0/1562659589 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f71fc06f250 0x7f71fc071760 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7210000f70 tx=0x7f7210009040 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:59.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.658+0000 7f721d600640 1 -- 172.21.15.86:0/1562659589 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f72200cb7c0 con 0x7f722015c900 2024-04-25T14:17:59.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.818+0000 7f7227600640 1 -- 172.21.15.86:0/1562659589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7220069d00 con 0x7f722015c900 2024-04-25T14:17:59.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.818+0000 7f721d600640 1 -- 172.21.15.86:0/1562659589 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) ==== 74+0+17760 (secure 0 0 0) 0x7f721805bee0 con 0x7f722015c900 2024-04-25T14:17:59.814 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:17:59.814 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":157,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:17:57.159829+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:13:32.006877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T14:13:32.574088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1552836841":"2024-04-26T14:17:57.159798+0000","172.21.15.86:6833/2318291413":"2024-04-26T14:17:57.159798+0000","172.21.15.86:0/3986518615":"2024-04-26T14:17:57.159798+0000","172.21.15.86:0/3859223883":"2024-04-26T14:17:57.159798+0000","172.21.15.86:0/1995365685":"2024-04-26T14:17:57.159798+0000","172.21.15.86:0/2043624430":"2024-04-26T14:13:51.718341+0000","172.21.15.86:0/955458502":"2024-04-26T14:13:51.718341+0000","172.21.15.86:0/1302282677":"2024-04-26T14:13:51.718341+0000","172.21.15.86:6833/505730993":"2024-04-26T14:13:51.718341+0000","172.21.15.86:6832/505730993":"2024-04-26T14:13:51.718341+0000","172.21.15.86:0/3455088811":"2024-04-26T14:13:51.718341+0000","172.21.15.86:0/3537592233":"2024-04-26T14:13:51.718341+0000","172.21.15.86:6832/2318291413":"2024-04-26T14:17:57.159798+0000","172.21.15.86:0/2705096450":"2024-04-26T14:13:51.718341+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:17:59.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.822+0000 7f7227600640 1 -- 172.21.15.86:0/1562659589 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f71fc06f250 msgr2=0x7f71fc071760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:59.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.822+0000 7f7227600640 1 --2- 172.21.15.86:0/1562659589 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f71fc06f250 0x7f71fc071760 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7210000f70 tx=0x7f7210009040 comp rx=0 tx=0).stop 2024-04-25T14:17:59.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.822+0000 7f7227600640 1 -- 172.21.15.86:0/1562659589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f722015c900 msgr2=0x7f7220157d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:59.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.822+0000 7f7227600640 1 --2- 172.21.15.86:0/1562659589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f722015c900 0x7f7220157d80 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f721800c760 tx=0x7f721800cc20 comp rx=0 tx=0).stop 2024-04-25T14:17:59.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.822+0000 7f7227600640 1 -- 172.21.15.86:0/1562659589 shutdown_connections 2024-04-25T14:17:59.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.822+0000 7f7227600640 1 --2- 172.21.15.86:0/1562659589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f722015c900 0x7f7220157d80 unknown :-1 s=CLOSED pgs=2459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:59.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.822+0000 7f7227600640 1 --2- 172.21.15.86:0/1562659589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72200cdcc0 0x7f72201562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:59.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.822+0000 7f7227600640 1 --2- 172.21.15.86:0/1562659589 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f71fc06f250 0x7f71fc071760 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:59.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.822+0000 7f7227600640 1 --2- 172.21.15.86:0/1562659589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7220070330 0x7f7220155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:59.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.822+0000 7f7227600640 1 -- 172.21.15.86:0/1562659589 >> 172.21.15.86:0/1562659589 conn(0x7f72200b4650 msgr2=0x7f72200b17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:59.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.822+0000 7f7227600640 1 -- 172.21.15.86:0/1562659589 shutdown_connections 2024-04-25T14:17:59.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.822+0000 7f7227600640 1 -- 172.21.15.86:0/1562659589 wait complete. 2024-04-25T14:17:59.835 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T14:17:59.836 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:17:59.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.990+0000 7faa03200640 1 Processor -- start 2024-04-25T14:17:59.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.990+0000 7faa03200640 1 -- start start 2024-04-25T14:17:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.994+0000 7faa03200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa9fc06ee10 0x7fa9fc0ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.994+0000 7faa03200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9fc06d940 con 0x7fa9fc071530 2024-04-25T14:17:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.994+0000 7faa03200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9fc06dc20 con 0x7fa9fc0cdd70 2024-04-25T14:17:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.994+0000 7faa03200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa9fc06df00 con 0x7fa9fc06ee10 2024-04-25T14:17:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.994+0000 7faa01200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa9fc06ee10 0x7fa9fc0ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.994+0000 7faa01200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa9fc06ee10 0x7fa9fc0ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35566/0 (socket says 172.21.15.86:35566) 2024-04-25T14:17:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.994+0000 7fa9f3e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa9fc071530 0x7fa9fc06e700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60504/0 (socket says 172.21.15.86:60504) 2024-04-25T14:17:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.994+0000 7faa01200640 1 -- 172.21.15.86:0/3590657620 learned_addr learned my addr 172.21.15.86:0/3590657620 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.994+0000 7faa01200640 1 -- 172.21.15.86:0/3590657620 >> v1:172.21.15.155:6789/0 conn(0x7fa9fc0cdd70 legacy=0x7fa9fc070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:59.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.994+0000 7faa01200640 1 -- 172.21.15.86:0/3590657620 >> v1:172.21.15.86:6789/0 conn(0x7fa9fc071530 legacy=0x7fa9fc06e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:59.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.994+0000 7faa01200640 1 -- 172.21.15.86:0/3590657620 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9fc06e1e0 con 0x7fa9fc06ee10 2024-04-25T14:17:59.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.994+0000 7fa9fb400640 1 -- 172.21.15.86:0/3590657620 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2982384479 0 0) 0x7fa9fc06d940 con 0x7fa9fc071530 2024-04-25T14:17:59.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.998+0000 7faa01200640 1 --2- 172.21.15.86:0/3590657620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9fc06ee10 0x7fa9fc0ce180 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4009e50 tx=0x7fa9f401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae994dc2eb2ef08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:17:59.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.998+0000 7fa9fb400640 1 -- 172.21.15.86:0/3590657620 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f40a5810 con 0x7fa9fc06ee10 2024-04-25T14:17:59.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.998+0000 7fa9fb400640 1 -- 172.21.15.86:0/3590657620 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9f40a59b0 con 0x7fa9fc06ee10 2024-04-25T14:17:59.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.998+0000 7fa9fb400640 1 -- 172.21.15.86:0/3590657620 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f40a5d80 con 0x7fa9fc06ee10 2024-04-25T14:17:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.998+0000 7faa03200640 1 -- 172.21.15.86:0/3590657620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9fc06ee10 msgr2=0x7fa9fc0ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.998+0000 7faa03200640 1 --2- 172.21.15.86:0/3590657620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9fc06ee10 0x7fa9fc0ce180 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4009e50 tx=0x7fa9f401ca30 comp rx=0 tx=0).stop 2024-04-25T14:17:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.998+0000 7faa03200640 1 -- 172.21.15.86:0/3590657620 shutdown_connections 2024-04-25T14:17:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.998+0000 7faa03200640 1 --2- 172.21.15.86:0/3590657620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9fc06ee10 0x7fa9fc0ce180 unknown :-1 s=CLOSED pgs=2461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.998+0000 7faa03200640 1 -- 172.21.15.86:0/3590657620 >> 172.21.15.86:0/3590657620 conn(0x7fa9fc0cffc0 msgr2=0x7fa9fc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:17:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.998+0000 7faa03200640 1 -- 172.21.15.86:0/3590657620 shutdown_connections 2024-04-25T14:17:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:17:59.998+0000 7faa03200640 1 -- 172.21.15.86:0/3590657620 wait complete. 2024-04-25T14:17:59.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.002+0000 7faa03200640 1 Processor -- start 2024-04-25T14:17:59.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.002+0000 7faa03200640 1 -- start start 2024-04-25T14:17:59.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.002+0000 7faa03200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9fc06ee10 0x7fa9fc0b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:59.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.002+0000 7faa03200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9fc0cdd70 0x7fa9fc0ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:59.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.002+0000 7faa03200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9fc1538b0 0x7fa9fc0caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:17:59.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.002+0000 7faa03200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa9fc16dc10 con 0x7fa9fc0cdd70 2024-04-25T14:17:59.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.002+0000 7faa03200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa9fc164380 con 0x7fa9fc1538b0 2024-04-25T14:17:59.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.002+0000 7faa03200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa9fc16dd90 con 0x7fa9fc06ee10 2024-04-25T14:17:59.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.002+0000 7fa9f3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9fc0cdd70 0x7fa9fc0ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:59.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.002+0000 7fa9f3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9fc0cdd70 0x7fa9fc0ca580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35576/0 (socket says 172.21.15.86:35576) 2024-04-25T14:17:59.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.002+0000 7fa9f3e00640 1 -- 172.21.15.86:0/2366640327 learned_addr learned my addr 172.21.15.86:0/2366640327 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:17:59.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.002+0000 7fa9fbe00640 1 --2- 172.21.15.86:0/2366640327 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9fc06ee10 0x7fa9fc0b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:59.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.002+0000 7faa01200640 1 --2- 172.21.15.86:0/2366640327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9fc1538b0 0x7fa9fc0caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:17:59.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.002+0000 7fa9f3e00640 1 -- 172.21.15.86:0/2366640327 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9fc06ee10 msgr2=0x7fa9fc0b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:59.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.002+0000 7fa9f3e00640 1 --2- 172.21.15.86:0/2366640327 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9fc06ee10 0x7fa9fc0b77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:59.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.002+0000 7fa9f3e00640 1 -- 172.21.15.86:0/2366640327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9fc1538b0 msgr2=0x7fa9fc0caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:17:59.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.002+0000 7fa9f3e00640 1 --2- 172.21.15.86:0/2366640327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9fc1538b0 0x7fa9fc0caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:17:59.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.002+0000 7fa9f3e00640 1 -- 172.21.15.86:0/2366640327 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9fc0b6740 con 0x7fa9fc0cdd70 2024-04-25T14:17:59.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.002+0000 7fa9f3e00640 1 --2- 172.21.15.86:0/2366640327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9fc0cdd70 0x7fa9fc0ca580 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400c3c0 tx=0x7fa9e400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:00.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.002+0000 7fa9f8c00640 1 -- 172.21.15.86:0/2366640327 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4017070 con 0x7fa9fc0cdd70 2024-04-25T14:18:00.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.002+0000 7fa9f8c00640 1 -- 172.21.15.86:0/2366640327 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9e4007500 con 0x7fa9fc0cdd70 2024-04-25T14:18:00.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.002+0000 7faa03200640 1 -- 172.21.15.86:0/2366640327 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9fc0b6a20 con 0x7fa9fc0cdd70 2024-04-25T14:18:00.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.002+0000 7fa9f8c00640 1 -- 172.21.15.86:0/2366640327 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e40059f0 con 0x7fa9fc0cdd70 2024-04-25T14:18:00.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.002+0000 7faa03200640 1 -- 172.21.15.86:0/2366640327 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9fc0b6fe0 con 0x7fa9fc0cdd70 2024-04-25T14:18:00.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.006+0000 7faa03200640 1 -- 172.21.15.86:0/2366640327 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9fc0b64b0 con 0x7fa9fc0cdd70 2024-04-25T14:18:00.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.006+0000 7fa9f8c00640 1 -- 172.21.15.86:0/2366640327 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 89) ==== 93151+0+0 (secure 0 0 0) 0x7fa9e401a070 con 0x7fa9fc0cdd70 2024-04-25T14:18:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.006+0000 7fa9f8c00640 1 --2- 172.21.15.86:0/2366640327 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fa9d806f370 0x7fa9d8071880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.006+0000 7fa9f8c00640 1 -- 172.21.15.86:0/2366640327 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6703+0+0 (secure 0 0 0) 0x7fa9e408e920 con 0x7fa9fc0cdd70 2024-04-25T14:18:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.006+0000 7fa9fbe00640 1 --2- 172.21.15.86:0/2366640327 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fa9d806f370 0x7fa9d8071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:00.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.006+0000 7fa9fbe00640 1 --2- 172.21.15.86:0/2366640327 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fa9d806f370 0x7fa9d8071880 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec007fd0 tx=0x7fa9ec009040 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:00.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.010+0000 7fa9f8c00640 1 -- 172.21.15.86:0/2366640327 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa9e405ae20 con 0x7fa9fc0cdd70 2024-04-25T14:18:00.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.206+0000 7faa03200640 1 -- 172.21.15.86:0/2366640327 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9fc0720e0 con 0x7fa9fc0cdd70 2024-04-25T14:18:00.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.206+0000 7fa9f8c00640 1 -- 172.21.15.86:0/2366640327 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7fa9e405f120 con 0x7fa9fc0cdd70 2024-04-25T14:18:00.200 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:18:00.200 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:18:00.200 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:18:00.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.206+0000 7faa03200640 1 -- 172.21.15.86:0/2366640327 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fa9d806f370 msgr2=0x7fa9d8071880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:00.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.206+0000 7faa03200640 1 --2- 172.21.15.86:0/2366640327 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fa9d806f370 0x7fa9d8071880 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec007fd0 tx=0x7fa9ec009040 comp rx=0 tx=0).stop 2024-04-25T14:18:00.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.206+0000 7faa03200640 1 -- 172.21.15.86:0/2366640327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9fc0cdd70 msgr2=0x7fa9fc0ca580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:00.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.206+0000 7faa03200640 1 --2- 172.21.15.86:0/2366640327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9fc0cdd70 0x7fa9fc0ca580 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400c3c0 tx=0x7fa9e400c880 comp rx=0 tx=0).stop 2024-04-25T14:18:00.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.206+0000 7faa03200640 1 -- 172.21.15.86:0/2366640327 shutdown_connections 2024-04-25T14:18:00.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.206+0000 7faa03200640 1 --2- 172.21.15.86:0/2366640327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9fc1538b0 0x7fa9fc0caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:00.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.206+0000 7faa03200640 1 --2- 172.21.15.86:0/2366640327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9fc0cdd70 0x7fa9fc0ca580 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:00.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.206+0000 7faa03200640 1 --2- 172.21.15.86:0/2366640327 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fa9d806f370 0x7fa9d8071880 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:00.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.206+0000 7faa03200640 1 --2- 172.21.15.86:0/2366640327 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9fc06ee10 0x7fa9fc0b77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:00.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.206+0000 7faa03200640 1 -- 172.21.15.86:0/2366640327 >> 172.21.15.86:0/2366640327 conn(0x7fa9fc0cffc0 msgr2=0x7fa9fc15caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:00.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.206+0000 7faa03200640 1 -- 172.21.15.86:0/2366640327 shutdown_connections 2024-04-25T14:18:00.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.206+0000 7faa03200640 1 -- 172.21.15.86:0/2366640327 wait complete. 2024-04-25T14:18:00.222 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:18:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.374+0000 7f2153000640 1 Processor -- start 2024-04-25T14:18:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.374+0000 7f2153000640 1 -- start start 2024-04-25T14:18:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.378+0000 7f2153000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f214c0d1df0 0x7f214c0cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.378+0000 7f2153000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f214c15e5c0 con 0x7f214c070330 2024-04-25T14:18:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.378+0000 7f2153000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f214c15e8a0 con 0x7f214c0cdcc0 2024-04-25T14:18:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.378+0000 7f2153000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f214c06fd40 con 0x7f214c0d1df0 2024-04-25T14:18:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.378+0000 7f2143e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f214c0d1df0 0x7f214c0cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.378+0000 7f2143e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f214c0d1df0 0x7f214c0cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35586/0 (socket says 172.21.15.86:35586) 2024-04-25T14:18:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.378+0000 7f214be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f214c070330 0x7f214c0d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60518/0 (socket says 172.21.15.86:60518) 2024-04-25T14:18:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.378+0000 7f2143e00640 1 -- 172.21.15.86:0/3799221821 learned_addr learned my addr 172.21.15.86:0/3799221821 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.378+0000 7f2143e00640 1 -- 172.21.15.86:0/3799221821 >> v1:172.21.15.155:6789/0 conn(0x7f214c0cdcc0 legacy=0x7f214c0ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.378+0000 7f2143e00640 1 -- 172.21.15.86:0/3799221821 >> v1:172.21.15.86:6789/0 conn(0x7f214c070330 legacy=0x7f214c0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.378+0000 7f2143e00640 1 -- 172.21.15.86:0/3799221821 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214c15eb80 con 0x7f214c0d1df0 2024-04-25T14:18:00.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.378+0000 7f2143e00640 1 --2- 172.21.15.86:0/3799221821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f214c0d1df0 0x7f214c0cd780 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f21340098a0 tx=0x7f213401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98fcd6239ae9819 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:00.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.382+0000 7f214b400640 1 -- 172.21.15.86:0/3799221821 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21340b0070 con 0x7f214c0d1df0 2024-04-25T14:18:00.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.382+0000 7f214b400640 1 -- 172.21.15.86:0/3799221821 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2134004230 con 0x7f214c0d1df0 2024-04-25T14:18:00.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.382+0000 7f214b400640 1 -- 172.21.15.86:0/3799221821 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2134003620 con 0x7f214c0d1df0 2024-04-25T14:18:00.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.382+0000 7f2153000640 1 -- 172.21.15.86:0/3799221821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f214c0d1df0 msgr2=0x7f214c0cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:00.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.382+0000 7f2153000640 1 --2- 172.21.15.86:0/3799221821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f214c0d1df0 0x7f214c0cd780 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f21340098a0 tx=0x7f213401c440 comp rx=0 tx=0).stop 2024-04-25T14:18:00.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.382+0000 7f2153000640 1 -- 172.21.15.86:0/3799221821 shutdown_connections 2024-04-25T14:18:00.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.382+0000 7f2153000640 1 --2- 172.21.15.86:0/3799221821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f214c0d1df0 0x7f214c0cd780 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:00.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.382+0000 7f2153000640 1 -- 172.21.15.86:0/3799221821 >> 172.21.15.86:0/3799221821 conn(0x7f214c0b4650 msgr2=0x7f214c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:00.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.382+0000 7f2153000640 1 -- 172.21.15.86:0/3799221821 shutdown_connections 2024-04-25T14:18:00.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.382+0000 7f2153000640 1 -- 172.21.15.86:0/3799221821 wait complete. 2024-04-25T14:18:00.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.382+0000 7f2153000640 1 Processor -- start 2024-04-25T14:18:00.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.382+0000 7f2153000640 1 -- start start 2024-04-25T14:18:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.386+0000 7f2153000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f214c070330 0x7f214c0ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.386+0000 7f2153000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f214c0cdcc0 0x7f214c0b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.386+0000 7f2153000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f214c0b1440 0x7f214c09da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.386+0000 7f2153000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f214c0708c0 con 0x7f214c0b1440 2024-04-25T14:18:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.386+0000 7f2153000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f214c070740 con 0x7f214c0cdcc0 2024-04-25T14:18:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.386+0000 7f2153000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f214c16bfd0 con 0x7f214c070330 2024-04-25T14:18:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.386+0000 7f2151000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f214c0b1440 0x7f214c09da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.386+0000 7f2151000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f214c0b1440 0x7f214c09da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35588/0 (socket says 172.21.15.86:35588) 2024-04-25T14:18:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.386+0000 7f2151000640 1 -- 172.21.15.86:0/1842258794 learned_addr learned my addr 172.21.15.86:0/1842258794 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.386+0000 7f2143e00640 1 --2- 172.21.15.86:0/1842258794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f214c0cdcc0 0x7f214c0b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.386+0000 7f214be00640 1 --2- 172.21.15.86:0/1842258794 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f214c070330 0x7f214c0ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.386+0000 7f2151000640 1 -- 172.21.15.86:0/1842258794 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f214c070330 msgr2=0x7f214c0ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.386+0000 7f2151000640 1 --2- 172.21.15.86:0/1842258794 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f214c070330 0x7f214c0ca8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.386+0000 7f2151000640 1 -- 172.21.15.86:0/1842258794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f214c0cdcc0 msgr2=0x7f214c0b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.386+0000 7f2151000640 1 --2- 172.21.15.86:0/1842258794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f214c0cdcc0 0x7f214c0b0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:00.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.386+0000 7f2151000640 1 -- 172.21.15.86:0/1842258794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214c0b1c30 con 0x7f214c0b1440 2024-04-25T14:18:00.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.386+0000 7f2151000640 1 --2- 172.21.15.86:0/1842258794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f214c0b1440 0x7f214c09da90 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f214400ba10 tx=0x7f214400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:00.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.386+0000 7f2148c00640 1 -- 172.21.15.86:0/1842258794 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2144019070 con 0x7f214c0b1440 2024-04-25T14:18:00.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.386+0000 7f2148c00640 1 -- 172.21.15.86:0/1842258794 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f214400cba0 con 0x7f214c0b1440 2024-04-25T14:18:00.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.386+0000 7f2153000640 1 -- 172.21.15.86:0/1842258794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f214c0b1f10 con 0x7f214c0b1440 2024-04-25T14:18:00.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.386+0000 7f2148c00640 1 -- 172.21.15.86:0/1842258794 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2144014440 con 0x7f214c0b1440 2024-04-25T14:18:00.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.386+0000 7f2153000640 1 -- 172.21.15.86:0/1842258794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f214c0b24d0 con 0x7f214c0b1440 2024-04-25T14:18:00.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.386+0000 7f2153000640 1 -- 172.21.15.86:0/1842258794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f214c069d00 con 0x7f214c0b1440 2024-04-25T14:18:00.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.390+0000 7f2148c00640 1 -- 172.21.15.86:0/1842258794 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 89) ==== 93151+0+0 (secure 0 0 0) 0x7f21440145e0 con 0x7f214c0b1440 2024-04-25T14:18:00.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.390+0000 7f2148c00640 1 --2- 172.21.15.86:0/1842258794 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f212806f3a0 0x7f21280718b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:00.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.390+0000 7f2148c00640 1 -- 172.21.15.86:0/1842258794 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6703+0+0 (secure 0 0 0) 0x7f214408f270 con 0x7f214c0b1440 2024-04-25T14:18:00.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.390+0000 7f214be00640 1 --2- 172.21.15.86:0/1842258794 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f212806f3a0 0x7f21280718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:00.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.390+0000 7f214be00640 1 --2- 172.21.15.86:0/1842258794 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f212806f3a0 0x7f21280718b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f213c000f70 tx=0x7f213c007cf0 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:00.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.394+0000 7f2148c00640 1 -- 172.21.15.86:0/1842258794 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f214405bdc0 con 0x7f214c0b1440 2024-04-25T14:18:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.582+0000 7f2153000640 1 -- 172.21.15.86:0/1842258794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f214c0b19c0 con 0x7f214c0b1440 2024-04-25T14:18:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.582+0000 7f2148c00640 1 -- 172.21.15.86:0/1842258794 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5219 (secure 0 0 0) 0x7f214405b6c0 con 0x7f214c0b1440 2024-04-25T14:18:00.577 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:18:00.577 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":125,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T14:13:33.612462+0000","modified":"2024-04-25T14:16:19.366674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0,1],"up":{"mds_0":10209,"mds_1":8387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8387":{"gid":8387,"name":"b","rank":1,"incarnation":119,"state":"up:stopping","state_seq":9,"addr":"172.21.15.155:6835/4231378267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4231378267},{"type":"v1","addr":"172.21.15.155:6835","nonce":4231378267}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10209":{"gid":10209,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3141015835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3141015835},{"type":"v1","addr":"172.21.15.86:6837","nonce":3141015835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10209,"qdb_cluster":[10209]},"id":11}]} 2024-04-25T14:18:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 125 2024-04-25T14:18:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.582+0000 7f2153000640 1 -- 172.21.15.86:0/1842258794 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f212806f3a0 msgr2=0x7f21280718b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.582+0000 7f2153000640 1 --2- 172.21.15.86:0/1842258794 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f212806f3a0 0x7f21280718b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f213c000f70 tx=0x7f213c007cf0 comp rx=0 tx=0).stop 2024-04-25T14:18:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.582+0000 7f2153000640 1 -- 172.21.15.86:0/1842258794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f214c0b1440 msgr2=0x7f214c09da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.582+0000 7f2153000640 1 --2- 172.21.15.86:0/1842258794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f214c0b1440 0x7f214c09da90 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f214400ba10 tx=0x7f214400bed0 comp rx=0 tx=0).stop 2024-04-25T14:18:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.582+0000 7f2153000640 1 -- 172.21.15.86:0/1842258794 shutdown_connections 2024-04-25T14:18:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.582+0000 7f2153000640 1 --2- 172.21.15.86:0/1842258794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f214c0b1440 0x7f214c09da90 unknown :-1 s=CLOSED pgs=2465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.582+0000 7f2153000640 1 --2- 172.21.15.86:0/1842258794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f214c0cdcc0 0x7f214c0b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.582+0000 7f2153000640 1 --2- 172.21.15.86:0/1842258794 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f212806f3a0 0x7f21280718b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.582+0000 7f2153000640 1 --2- 172.21.15.86:0/1842258794 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f214c070330 0x7f214c0ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.582+0000 7f2153000640 1 -- 172.21.15.86:0/1842258794 >> 172.21.15.86:0/1842258794 conn(0x7f214c0b4650 msgr2=0x7f214c072880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.586+0000 7f2153000640 1 -- 172.21.15.86:0/1842258794 shutdown_connections 2024-04-25T14:18:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.586+0000 7f2153000640 1 -- 172.21.15.86:0/1842258794 wait complete. 2024-04-25T14:18:00.599 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:18:00.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.754+0000 7f6d55a00640 1 Processor -- start 2024-04-25T14:18:00.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.754+0000 7f6d55a00640 1 -- start start 2024-04-25T14:18:00.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.754+0000 7f6d55a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6d500d16e0 0x7f6d500d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:00.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.754+0000 7f6d55a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d500d2030 con 0x7f6d500cfca0 2024-04-25T14:18:00.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.754+0000 7f6d55a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d50072bd0 con 0x7f6d500730b0 2024-04-25T14:18:00.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.754+0000 7f6d55a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6d50070470 con 0x7f6d500d16e0 2024-04-25T14:18:00.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.754+0000 7f6d4ea00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6d500cfca0 0x7f6d5006f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60522/0 (socket says 172.21.15.86:60522) 2024-04-25T14:18:00.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.754+0000 7f6d4ea00640 1 -- 172.21.15.86:0/3084773753 learned_addr learned my addr 172.21.15.86:0/3084773753 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:18:00.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.754+0000 7f6d46000640 1 --2- 172.21.15.86:0/3084773753 >> v2:172.21.15.86:3300/0 conn(0x7f6d500d16e0 0x7f6d500d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:00.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.754+0000 7f6d4e000640 1 -- 172.21.15.86:0/3084773753 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3060551283 0 0) 0x7f6d40001120 con 0x7f6d500cfca0 2024-04-25T14:18:00.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.754+0000 7f6d4e000640 1 -- 172.21.15.86:0/3084773753 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d34003530 con 0x7f6d500cfca0 2024-04-25T14:18:00.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.754+0000 7f6d4e000640 1 -- 172.21.15.86:0/3084773753 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1942462781 0 0) 0x7f6d50072bd0 con 0x7f6d500730b0 2024-04-25T14:18:00.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.754+0000 7f6d46000640 1 -- 172.21.15.86:0/3084773753 >> v1:172.21.15.155:6789/0 conn(0x7f6d500730b0 legacy=0x7f6d500734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:00.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.754+0000 7f6d46000640 1 -- 172.21.15.86:0/3084773753 >> v1:172.21.15.86:6789/0 conn(0x7f6d500cfca0 legacy=0x7f6d5006f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:00.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.754+0000 7f6d46000640 1 -- 172.21.15.86:0/3084773753 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5006fbd0 con 0x7f6d500d16e0 2024-04-25T14:18:00.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.758+0000 7f6d46000640 1 --2- 172.21.15.86:0/3084773753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d500d16e0 0x7f6d500d1af0 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f6d380097c0 tx=0x7f6d3801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af18392d82459072 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:00.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.758+0000 7f6d4e000640 1 -- 172.21.15.86:0/3084773753 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d380b1070 con 0x7f6d500d16e0 2024-04-25T14:18:00.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.758+0000 7f6d4e000640 1 -- 172.21.15.86:0/3084773753 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d380ac410 con 0x7f6d500d16e0 2024-04-25T14:18:00.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.758+0000 7f6d4e000640 1 -- 172.21.15.86:0/3084773753 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d500743e0 con 0x7f6d500d16e0 2024-04-25T14:18:00.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.758+0000 7f6d55a00640 1 -- 172.21.15.86:0/3084773753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d500d16e0 msgr2=0x7f6d500d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:00.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.758+0000 7f6d55a00640 1 --2- 172.21.15.86:0/3084773753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d500d16e0 0x7f6d500d1af0 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f6d380097c0 tx=0x7f6d3801c4a0 comp rx=0 tx=0).stop 2024-04-25T14:18:00.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.758+0000 7f6d55a00640 1 -- 172.21.15.86:0/3084773753 shutdown_connections 2024-04-25T14:18:00.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.758+0000 7f6d55a00640 1 --2- 172.21.15.86:0/3084773753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d500d16e0 0x7f6d500d1af0 unknown :-1 s=CLOSED pgs=2467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:00.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.758+0000 7f6d55a00640 1 -- 172.21.15.86:0/3084773753 >> 172.21.15.86:0/3084773753 conn(0x7f6d500b4650 msgr2=0x7f6d500b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:00.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.758+0000 7f6d55a00640 1 -- 172.21.15.86:0/3084773753 shutdown_connections 2024-04-25T14:18:00.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.758+0000 7f6d55a00640 1 -- 172.21.15.86:0/3084773753 wait complete. 2024-04-25T14:18:00.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.762+0000 7f6d55a00640 1 Processor -- start 2024-04-25T14:18:00.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.762+0000 7f6d55a00640 1 -- start start 2024-04-25T14:18:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.762+0000 7f6d55a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6d500730b0 0x7f6d500cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.762+0000 7f6d55a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d500cfca0 0x7f6d5015d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.762+0000 7f6d55a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d500d16e0 0x7f6d5015dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.762+0000 7f6d55a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5016aa80 con 0x7f6d500d16e0 2024-04-25T14:18:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.762+0000 7f6d55a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5016a900 con 0x7f6d500cfca0 2024-04-25T14:18:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.762+0000 7f6d55a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6d5016ac00 con 0x7f6d500730b0 2024-04-25T14:18:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.762+0000 7f6d4f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d500d16e0 0x7f6d5015dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:00.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.762+0000 7f6d4f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d500d16e0 0x7f6d5015dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35606/0 (socket says 172.21.15.86:35606) 2024-04-25T14:18:00.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.762+0000 7f6d4f400640 1 -- 172.21.15.86:0/4200203872 learned_addr learned my addr 172.21.15.86:0/4200203872 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:00.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.762+0000 7f6d4ea00640 1 --2- 172.21.15.86:0/4200203872 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6d500730b0 0x7f6d500cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:00.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.762+0000 7f6d46000640 1 --2- 172.21.15.86:0/4200203872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d500cfca0 0x7f6d5015d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:00.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.762+0000 7f6d4f400640 1 -- 172.21.15.86:0/4200203872 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6d500730b0 msgr2=0x7f6d500cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:00.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.762+0000 7f6d4f400640 1 --2- 172.21.15.86:0/4200203872 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6d500730b0 0x7f6d500cdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:00.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.762+0000 7f6d4f400640 1 -- 172.21.15.86:0/4200203872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d500cfca0 msgr2=0x7f6d5015d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:00.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.762+0000 7f6d4f400640 1 --2- 172.21.15.86:0/4200203872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d500cfca0 0x7f6d5015d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.762+0000 7f6d4f400640 1 -- 172.21.15.86:0/4200203872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d50158c60 con 0x7f6d500d16e0 2024-04-25T14:18:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.766+0000 7f6d4f400640 1 --2- 172.21.15.86:0/4200203872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d500d16e0 0x7f6d5015dac0 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f6d4800c7b0 tx=0x7f6d4800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.766+0000 7f6d47400640 1 -- 172.21.15.86:0/4200203872 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4800ce60 con 0x7f6d500d16e0 2024-04-25T14:18:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.766+0000 7f6d47400640 1 -- 172.21.15.86:0/4200203872 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d48003db0 con 0x7f6d500d16e0 2024-04-25T14:18:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.766+0000 7f6d55a00640 1 -- 172.21.15.86:0/4200203872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d50157ea0 con 0x7f6d500d16e0 2024-04-25T14:18:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.766+0000 7f6d47400640 1 -- 172.21.15.86:0/4200203872 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d480052b0 con 0x7f6d500d16e0 2024-04-25T14:18:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.766+0000 7f6d55a00640 1 -- 172.21.15.86:0/4200203872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d50158460 con 0x7f6d500d16e0 2024-04-25T14:18:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.766+0000 7f6d55a00640 1 -- 172.21.15.86:0/4200203872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d500cea30 con 0x7f6d500d16e0 2024-04-25T14:18:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.766+0000 7f6d47400640 1 -- 172.21.15.86:0/4200203872 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 89) ==== 93151+0+0 (secure 0 0 0) 0x7f6d48005450 con 0x7f6d500d16e0 2024-04-25T14:18:00.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.766+0000 7f6d47400640 1 --2- 172.21.15.86:0/4200203872 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f6d2c06f370 0x7f6d2c071880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:00.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.766+0000 7f6d47400640 1 -- 172.21.15.86:0/4200203872 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6703+0+0 (secure 0 0 0) 0x7f6d4808f530 con 0x7f6d500d16e0 2024-04-25T14:18:00.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.766+0000 7f6d4ea00640 1 --2- 172.21.15.86:0/4200203872 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f6d2c06f370 0x7f6d2c071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:00.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.770+0000 7f6d4ea00640 1 --2- 172.21.15.86:0/4200203872 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f6d2c06f370 0x7f6d2c071880 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6d400013e0 tx=0x7f6d40009040 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:00.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.774+0000 7f6d47400640 1 -- 172.21.15.86:0/4200203872 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6d4805c080 con 0x7f6d500d16e0 2024-04-25T14:18:00.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.930+0000 7f6d55a00640 1 -- 172.21.15.86:0/4200203872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d50069d00 con 0x7f6d500d16e0 2024-04-25T14:18:00.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.930+0000 7f6d47400640 1 -- 172.21.15.86:0/4200203872 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) ==== 74+0+17760 (secure 0 0 0) 0x7f6d4805b980 con 0x7f6d500d16e0 2024-04-25T14:18:00.926 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:18:00.926 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":157,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:17:57.159829+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:13:32.006877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T14:13:32.574088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1552836841":"2024-04-26T14:17:57.159798+0000","172.21.15.86:6833/2318291413":"2024-04-26T14:17:57.159798+0000","172.21.15.86:0/3986518615":"2024-04-26T14:17:57.159798+0000","172.21.15.86:0/3859223883":"2024-04-26T14:17:57.159798+0000","172.21.15.86:0/1995365685":"2024-04-26T14:17:57.159798+0000","172.21.15.86:0/2043624430":"2024-04-26T14:13:51.718341+0000","172.21.15.86:0/955458502":"2024-04-26T14:13:51.718341+0000","172.21.15.86:0/1302282677":"2024-04-26T14:13:51.718341+0000","172.21.15.86:6833/505730993":"2024-04-26T14:13:51.718341+0000","172.21.15.86:6832/505730993":"2024-04-26T14:13:51.718341+0000","172.21.15.86:0/3455088811":"2024-04-26T14:13:51.718341+0000","172.21.15.86:0/3537592233":"2024-04-26T14:13:51.718341+0000","172.21.15.86:6832/2318291413":"2024-04-26T14:17:57.159798+0000","172.21.15.86:0/2705096450":"2024-04-26T14:13:51.718341+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:18:00.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.934+0000 7f6d55a00640 1 -- 172.21.15.86:0/4200203872 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f6d2c06f370 msgr2=0x7f6d2c071880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:00.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.934+0000 7f6d55a00640 1 --2- 172.21.15.86:0/4200203872 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f6d2c06f370 0x7f6d2c071880 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6d400013e0 tx=0x7f6d40009040 comp rx=0 tx=0).stop 2024-04-25T14:18:00.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.934+0000 7f6d55a00640 1 -- 172.21.15.86:0/4200203872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d500d16e0 msgr2=0x7f6d5015dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:00.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.934+0000 7f6d55a00640 1 --2- 172.21.15.86:0/4200203872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d500d16e0 0x7f6d5015dac0 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f6d4800c7b0 tx=0x7f6d4800cc70 comp rx=0 tx=0).stop 2024-04-25T14:18:00.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.934+0000 7f6d55a00640 1 -- 172.21.15.86:0/4200203872 shutdown_connections 2024-04-25T14:18:00.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.934+0000 7f6d55a00640 1 --2- 172.21.15.86:0/4200203872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d500d16e0 0x7f6d5015dac0 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:00.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.934+0000 7f6d55a00640 1 --2- 172.21.15.86:0/4200203872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d500cfca0 0x7f6d5015d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:00.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.934+0000 7f6d55a00640 1 --2- 172.21.15.86:0/4200203872 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f6d2c06f370 0x7f6d2c071880 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:00.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.934+0000 7f6d55a00640 1 --2- 172.21.15.86:0/4200203872 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6d500730b0 0x7f6d500cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:00.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.934+0000 7f6d55a00640 1 -- 172.21.15.86:0/4200203872 >> 172.21.15.86:0/4200203872 conn(0x7f6d500b4650 msgr2=0x7f6d500cc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:00.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.934+0000 7f6d55a00640 1 -- 172.21.15.86:0/4200203872 shutdown_connections 2024-04-25T14:18:00.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:00.934+0000 7f6d55a00640 1 -- 172.21.15.86:0/4200203872 wait complete. 2024-04-25T14:18:00.948 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T14:18:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.102+0000 7fc800600640 1 Processor -- start 2024-04-25T14:18:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.102+0000 7fc800600640 1 -- start start 2024-04-25T14:18:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.102+0000 7fc800600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc7f80d1df0 0x7fc7f80cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.102+0000 7fc800600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7f815e5c0 con 0x7fc7f80cdcc0 2024-04-25T14:18:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.102+0000 7fc800600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7f815e8a0 con 0x7fc7f8070330 2024-04-25T14:18:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.102+0000 7fc800600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc7f806fd40 con 0x7fc7f80d1df0 2024-04-25T14:18:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.102+0000 7fc7fd200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc7f80d1df0 0x7fc7f80cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.102+0000 7fc7fe600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc7f80cdcc0 0x7fc7f80ce0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60528/0 (socket says 172.21.15.86:60528) 2024-04-25T14:18:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.106+0000 7fc7fe600640 1 -- 172.21.15.86:0/238802869 learned_addr learned my addr 172.21.15.86:0/238802869 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:18:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.106+0000 7fc7fd200640 1 -- 172.21.15.86:0/238802869 >> v1:172.21.15.155:6789/0 conn(0x7fc7f8070330 legacy=0x7fc7f80d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.106+0000 7fc7fd200640 1 -- 172.21.15.86:0/238802869 >> v1:172.21.15.86:6789/0 conn(0x7fc7f80cdcc0 legacy=0x7fc7f80ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.106+0000 7fc7fd200640 1 -- 172.21.15.86:0/238802869 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f815eb80 con 0x7fc7f80d1df0 2024-04-25T14:18:01.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.106+0000 7fc7fd200640 1 --2- 172.21.15.86:0/238802869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7f80d1df0 0x7fc7f80cd780 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40097f0 tx=0x7fc7e401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914c88f72a385cfb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.106+0000 7fc7f7e00640 1 -- 172.21.15.86:0/238802869 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e40b0070 con 0x7fc7f80d1df0 2024-04-25T14:18:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.106+0000 7fc7f7e00640 1 -- 172.21.15.86:0/238802869 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7e40ab600 con 0x7fc7f80d1df0 2024-04-25T14:18:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.106+0000 7fc7f7e00640 1 -- 172.21.15.86:0/238802869 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e40ab9d0 con 0x7fc7f80d1df0 2024-04-25T14:18:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.110+0000 7fc800600640 1 -- 172.21.15.86:0/238802869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7f80d1df0 msgr2=0x7fc7f80cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.110+0000 7fc800600640 1 --2- 172.21.15.86:0/238802869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7f80d1df0 0x7fc7f80cd780 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40097f0 tx=0x7fc7e401c3a0 comp rx=0 tx=0).stop 2024-04-25T14:18:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.110+0000 7fc800600640 1 -- 172.21.15.86:0/238802869 shutdown_connections 2024-04-25T14:18:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.110+0000 7fc800600640 1 --2- 172.21.15.86:0/238802869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7f80d1df0 0x7fc7f80cd780 unknown :-1 s=CLOSED pgs=2470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.110+0000 7fc800600640 1 -- 172.21.15.86:0/238802869 >> 172.21.15.86:0/238802869 conn(0x7fc7f80b4650 msgr2=0x7fc7f80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.110+0000 7fc800600640 1 -- 172.21.15.86:0/238802869 shutdown_connections 2024-04-25T14:18:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.110+0000 7fc800600640 1 -- 172.21.15.86:0/238802869 wait complete. 2024-04-25T14:18:01.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.110+0000 7fc800600640 1 Processor -- start 2024-04-25T14:18:01.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.110+0000 7fc800600640 1 -- start start 2024-04-25T14:18:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.110+0000 7fc800600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7f8070330 0x7fc7f809aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.110+0000 7fc800600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7f80cdcc0 0x7fc7f8156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.110+0000 7fc800600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7f8157130 0x7fc7f80aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.114+0000 7fc800600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f80708c0 con 0x7fc7f8157130 2024-04-25T14:18:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.114+0000 7fc800600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f8070740 con 0x7fc7f80cdcc0 2024-04-25T14:18:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.114+0000 7fc800600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc7f816bfd0 con 0x7fc7f8070330 2024-04-25T14:18:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.114+0000 7fc7fe600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7f8157130 0x7fc7f80aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.114+0000 7fc7fe600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7f8157130 0x7fc7f80aecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35622/0 (socket says 172.21.15.86:35622) 2024-04-25T14:18:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.114+0000 7fc7fe600640 1 -- 172.21.15.86:0/4233539418 learned_addr learned my addr 172.21.15.86:0/4233539418 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.114+0000 7fc7fdc00640 1 --2- 172.21.15.86:0/4233539418 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7f8070330 0x7fc7f809aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.114+0000 7fc7fd200640 1 --2- 172.21.15.86:0/4233539418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7f80cdcc0 0x7fc7f8156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:01.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.114+0000 7fc7fe600640 1 -- 172.21.15.86:0/4233539418 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7f8070330 msgr2=0x7fc7f809aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:01.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.114+0000 7fc7fe600640 1 --2- 172.21.15.86:0/4233539418 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7f8070330 0x7fc7f809aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:01.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.114+0000 7fc7fe600640 1 -- 172.21.15.86:0/4233539418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7f80cdcc0 msgr2=0x7fc7f8156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:01.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.114+0000 7fc7fe600640 1 --2- 172.21.15.86:0/4233539418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7f80cdcc0 0x7fc7f8156bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:01.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.114+0000 7fc7fe600640 1 -- 172.21.15.86:0/4233539418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f815b5e0 con 0x7fc7f8157130 2024-04-25T14:18:01.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.114+0000 7fc7fe600640 1 --2- 172.21.15.86:0/4233539418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7f8157130 0x7fc7f80aecb0 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7fc7f000caa0 tx=0x7fc7f000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:01.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.114+0000 7fc7f5600640 1 -- 172.21.15.86:0/4233539418 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0017070 con 0x7fc7f8157130 2024-04-25T14:18:01.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.114+0000 7fc7f5600640 1 -- 172.21.15.86:0/4233539418 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7f000ac00 con 0x7fc7f8157130 2024-04-25T14:18:01.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.114+0000 7fc800600640 1 -- 172.21.15.86:0/4233539418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f80aff20 con 0x7fc7f8157130 2024-04-25T14:18:01.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.114+0000 7fc7f5600640 1 -- 172.21.15.86:0/4233539418 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0012400 con 0x7fc7f8157130 2024-04-25T14:18:01.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.114+0000 7fc800600640 1 -- 172.21.15.86:0/4233539418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f80c4ab0 con 0x7fc7f8157130 2024-04-25T14:18:01.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.114+0000 7fc800600640 1 -- 172.21.15.86:0/4233539418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7f80b00f0 con 0x7fc7f8157130 2024-04-25T14:18:01.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.114+0000 7fc7f5600640 1 -- 172.21.15.86:0/4233539418 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 89) ==== 93151+0+0 (secure 0 0 0) 0x7fc7f001a070 con 0x7fc7f8157130 2024-04-25T14:18:01.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.118+0000 7fc7f5600640 1 --2- 172.21.15.86:0/4233539418 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fc7d806f370 0x7fc7d8071880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:01.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.118+0000 7fc7f5600640 1 -- 172.21.15.86:0/4233539418 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6703+0+0 (secure 0 0 0) 0x7fc7f008f1d0 con 0x7fc7f8157130 2024-04-25T14:18:01.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.118+0000 7fc7fdc00640 1 --2- 172.21.15.86:0/4233539418 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fc7d806f370 0x7fc7d8071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:01.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.118+0000 7fc7fdc00640 1 --2- 172.21.15.86:0/4233539418 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fc7d806f370 0x7fc7d8071880 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8072200 tx=0x7fc7ec009060 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:01.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.122+0000 7fc7f5600640 1 -- 172.21.15.86:0/4233539418 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc7f005b770 con 0x7fc7f8157130 2024-04-25T14:18:01.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.310+0000 7fc800600640 1 -- 172.21.15.86:0/4233539418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc7f8069d00 con 0x7fc7f8157130 2024-04-25T14:18:01.307 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-25T14:18:01.314+0000 7fea29c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:18:01.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.314+0000 7fc7f5600640 1 -- 172.21.15.86:0/4233539418 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v126) ==== 161+0+0 (secure 0 0 0) 0x7fc7f005fa70 con 0x7fc7f8157130 2024-04-25T14:18:01.311 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T14:18:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.318+0000 7fc800600640 1 -- 172.21.15.86:0/4233539418 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fc7d806f370 msgr2=0x7fc7d8071880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.318+0000 7fc800600640 1 --2- 172.21.15.86:0/4233539418 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fc7d806f370 0x7fc7d8071880 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8072200 tx=0x7fc7ec009060 comp rx=0 tx=0).stop 2024-04-25T14:18:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.318+0000 7fc800600640 1 -- 172.21.15.86:0/4233539418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7f8157130 msgr2=0x7fc7f80aecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.318+0000 7fc800600640 1 --2- 172.21.15.86:0/4233539418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7f8157130 0x7fc7f80aecb0 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7fc7f000caa0 tx=0x7fc7f000cf60 comp rx=0 tx=0).stop 2024-04-25T14:18:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.318+0000 7fc800600640 1 -- 172.21.15.86:0/4233539418 shutdown_connections 2024-04-25T14:18:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.318+0000 7fc800600640 1 --2- 172.21.15.86:0/4233539418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7f8157130 0x7fc7f80aecb0 unknown :-1 s=CLOSED pgs=2471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.318+0000 7fc800600640 1 --2- 172.21.15.86:0/4233539418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7f80cdcc0 0x7fc7f8156bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.318+0000 7fc800600640 1 --2- 172.21.15.86:0/4233539418 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fc7d806f370 0x7fc7d8071880 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:01.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.318+0000 7fc800600640 1 --2- 172.21.15.86:0/4233539418 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7f8070330 0x7fc7f809aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:01.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.318+0000 7fc800600640 1 -- 172.21.15.86:0/4233539418 >> 172.21.15.86:0/4233539418 conn(0x7fc7f80b4650 msgr2=0x7fc7f80b1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:01.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.318+0000 7fc800600640 1 -- 172.21.15.86:0/4233539418 shutdown_connections 2024-04-25T14:18:01.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.318+0000 7fc800600640 1 -- 172.21.15.86:0/4233539418 wait complete. 2024-04-25T14:18:01.335 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T14:18:01.447 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2024-04-25T14:18:01.448 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-25T14:18:01.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.494+0000 7f259d000640 1 Processor -- start 2024-04-25T14:18:01.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.494+0000 7f259d000640 1 -- start start 2024-04-25T14:18:01.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.494+0000 7f259d000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f25980cfca0 0x7f259806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:01.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.494+0000 7f259d000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25980730b0 0x7f25980734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:01.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.494+0000 7f259d000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2598072bd0 con 0x7f25980d16e0 2024-04-25T14:18:01.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.494+0000 7f259d000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f2598070470 con 0x7f25980cfca0 2024-04-25T14:18:01.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.494+0000 7f259d000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f259806f930 con 0x7f25980730b0 2024-04-25T14:18:01.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.498+0000 7f2595e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f25980cfca0 0x7f259806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:01.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.498+0000 7f2596800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25980730b0 0x7f25980734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:01.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.498+0000 7f2595e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f25980cfca0 0x7f259806f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:59066/0 (socket says 172.21.15.86:59066) 2024-04-25T14:18:01.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.498+0000 7f2596800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25980730b0 0x7f25980734c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:40940/0 (socket says 172.21.15.86:40940) 2024-04-25T14:18:01.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.498+0000 7f2595e00640 1 -- 172.21.15.86:0/3704386570 learned_addr learned my addr 172.21.15.86:0/3704386570 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:01.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.498+0000 7f2595e00640 1 -- 172.21.15.86:0/3704386570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25980730b0 msgr2=0x7f25980734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:01.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.498+0000 7f2595e00640 1 --2- 172.21.15.86:0/3704386570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25980730b0 0x7f25980734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:01.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.498+0000 7f2595e00640 1 -- 172.21.15.86:0/3704386570 >> v1:172.21.15.155:6789/0 conn(0x7f25980d16e0 legacy=0x7f25980d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:01.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.498+0000 7f2595e00640 1 -- 172.21.15.86:0/3704386570 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f259806fbd0 con 0x7f25980cfca0 2024-04-25T14:18:01.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.498+0000 7f2595e00640 1 --2- 172.21.15.86:0/3704386570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25980cfca0 0x7f259806f3f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f2588009e50 tx=0x7f258801ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83ef89dcb6cc021d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:01.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.502+0000 7f2595400640 1 -- 172.21.15.86:0/3704386570 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25880a5810 con 0x7f25980cfca0 2024-04-25T14:18:01.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.502+0000 7f2595400640 1 -- 172.21.15.86:0/3704386570 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25880a59b0 con 0x7f25980cfca0 2024-04-25T14:18:01.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.502+0000 7f2595400640 1 -- 172.21.15.86:0/3704386570 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25880a5d80 con 0x7f25980cfca0 2024-04-25T14:18:01.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.502+0000 7f259d000640 1 -- 172.21.15.86:0/3704386570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25980cfca0 msgr2=0x7f259806f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:01.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.502+0000 7f259d000640 1 --2- 172.21.15.86:0/3704386570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25980cfca0 0x7f259806f3f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f2588009e50 tx=0x7f258801ca30 comp rx=0 tx=0).stop 2024-04-25T14:18:01.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.502+0000 7f259d000640 1 -- 172.21.15.86:0/3704386570 shutdown_connections 2024-04-25T14:18:01.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.502+0000 7f259d000640 1 --2- 172.21.15.86:0/3704386570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25980730b0 0x7f25980734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:01.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.502+0000 7f259d000640 1 --2- 172.21.15.86:0/3704386570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25980cfca0 0x7f259806f3f0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:01.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.502+0000 7f259d000640 1 -- 172.21.15.86:0/3704386570 >> 172.21.15.86:0/3704386570 conn(0x7f25980b4650 msgr2=0x7f25980b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:01.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.502+0000 7f259d000640 1 -- 172.21.15.86:0/3704386570 shutdown_connections 2024-04-25T14:18:01.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.502+0000 7f259d000640 1 -- 172.21.15.86:0/3704386570 wait complete. 2024-04-25T14:18:01.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.502+0000 7f259d000640 1 Processor -- start 2024-04-25T14:18:01.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.502+0000 7f259d000640 1 -- start start 2024-04-25T14:18:01.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.506+0000 7f259d000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25980730b0 0x7f25981f9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:01.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.506+0000 7f259d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25980cfca0 0x7f259809a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:01.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.506+0000 7f259d000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25980d16e0 0x7f25980cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:01.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.506+0000 7f259d000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f25980cdee0 con 0x7f25980cfca0 2024-04-25T14:18:01.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.506+0000 7f259d000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f25980cdd60 con 0x7f25980d16e0 2024-04-25T14:18:01.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.506+0000 7f259d000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f25980ce060 con 0x7f25980730b0 2024-04-25T14:18:01.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.506+0000 7f258d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25980cfca0 0x7f259809a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:01.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.506+0000 7f258d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25980cfca0 0x7f259809a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35650/0 (socket says 172.21.15.86:35650) 2024-04-25T14:18:01.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.506+0000 7f258d400640 1 -- 172.21.15.86:0/962385064 learned_addr learned my addr 172.21.15.86:0/962385064 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:01.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.506+0000 7f2595e00640 1 --2- 172.21.15.86:0/962385064 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25980730b0 0x7f25981f9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:01.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.506+0000 7f2596800640 1 --2- 172.21.15.86:0/962385064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25980d16e0 0x7f25980cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:01.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.506+0000 7f258d400640 1 -- 172.21.15.86:0/962385064 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25980730b0 msgr2=0x7f25981f9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:01.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.506+0000 7f258d400640 1 --2- 172.21.15.86:0/962385064 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25980730b0 0x7f25981f9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:01.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.506+0000 7f258d400640 1 -- 172.21.15.86:0/962385064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25980d16e0 msgr2=0x7f25980cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:01.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.506+0000 7f258d400640 1 --2- 172.21.15.86:0/962385064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25980d16e0 0x7f25980cae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:01.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.506+0000 7f258d400640 1 -- 172.21.15.86:0/962385064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f259815e510 con 0x7f25980cfca0 2024-04-25T14:18:01.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.506+0000 7f2595e00640 1 --2- 172.21.15.86:0/962385064 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25980730b0 0x7f25981f9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:18:01.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.506+0000 7f258d400640 1 --2- 172.21.15.86:0/962385064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25980cfca0 0x7f259809a9e0 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f258000c3f0 tx=0x7f258000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:01.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.506+0000 7f258ea00640 1 -- 172.21.15.86:0/962385064 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580017070 con 0x7f25980cfca0 2024-04-25T14:18:01.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.506+0000 7f258ea00640 1 -- 172.21.15.86:0/962385064 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25800052f0 con 0x7f25980cfca0 2024-04-25T14:18:01.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.506+0000 7f259d000640 1 -- 172.21.15.86:0/962385064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f259815e230 con 0x7f25980cfca0 2024-04-25T14:18:01.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.506+0000 7f258ea00640 1 -- 172.21.15.86:0/962385064 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580003860 con 0x7f25980cfca0 2024-04-25T14:18:01.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.506+0000 7f259d000640 1 -- 172.21.15.86:0/962385064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25980ca660 con 0x7f25980cfca0 2024-04-25T14:18:01.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.506+0000 7f259d000640 1 -- 172.21.15.86:0/962385064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f259809f4b0 con 0x7f25980cfca0 2024-04-25T14:18:01.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.510+0000 7f258ea00640 1 -- 172.21.15.86:0/962385064 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 89) ==== 93151+0+0 (secure 0 0 0) 0x7f258001a070 con 0x7f25980cfca0 2024-04-25T14:18:01.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.510+0000 7f258ea00640 1 --2- 172.21.15.86:0/962385064 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f257406f370 0x7f2574071880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:01.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.510+0000 7f258ea00640 1 -- 172.21.15.86:0/962385064 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 6875+0+0 (secure 0 0 0) 0x7f258008e8d0 con 0x7f25980cfca0 2024-04-25T14:18:01.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.510+0000 7f2595e00640 1 --2- 172.21.15.86:0/962385064 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f257406f370 0x7f2574071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:01.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.510+0000 7f2595e00640 1 --2- 172.21.15.86:0/962385064 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f257406f370 0x7f2574071880 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f25880ae000 tx=0x7f25880a7070 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:01.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.514+0000 7f258ea00640 1 -- 172.21.15.86:0/962385064 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f258005adb0 con 0x7f25980cfca0 2024-04-25T14:18:01.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:01.702+0000 7f259d000640 1 -- 172.21.15.86:0/962385064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2598069d00 con 0x7f25980cfca0 2024-04-25T14:18:01.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.002+0000 7f258ea00640 1 -- 172.21.15.86:0/962385064 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7f2580057040 con 0x7f25980cfca0 2024-04-25T14:18:02.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.318+0000 7f258ea00640 1 -- 172.21.15.86:0/962385064 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v127) ==== 104+0+0 (secure 0 0 0) 0x7f258005b4b0 con 0x7f25980cfca0 2024-04-25T14:18:02.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.322+0000 7f259d000640 1 -- 172.21.15.86:0/962385064 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f257406f370 msgr2=0x7f2574071880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:02.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.322+0000 7f259d000640 1 --2- 172.21.15.86:0/962385064 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f257406f370 0x7f2574071880 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f25880ae000 tx=0x7f25880a7070 comp rx=0 tx=0).stop 2024-04-25T14:18:02.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.322+0000 7f259d000640 1 -- 172.21.15.86:0/962385064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25980cfca0 msgr2=0x7f259809a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:02.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.322+0000 7f259d000640 1 --2- 172.21.15.86:0/962385064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25980cfca0 0x7f259809a9e0 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f258000c3f0 tx=0x7f258000c8b0 comp rx=0 tx=0).stop 2024-04-25T14:18:02.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.322+0000 7f259d000640 1 -- 172.21.15.86:0/962385064 shutdown_connections 2024-04-25T14:18:02.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.322+0000 7f259d000640 1 --2- 172.21.15.86:0/962385064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25980d16e0 0x7f25980cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:02.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.322+0000 7f259d000640 1 --2- 172.21.15.86:0/962385064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25980cfca0 0x7f259809a9e0 unknown :-1 s=CLOSED pgs=2475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:02.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.322+0000 7f259d000640 1 --2- 172.21.15.86:0/962385064 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f257406f370 0x7f2574071880 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:02.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.322+0000 7f259d000640 1 --2- 172.21.15.86:0/962385064 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25980730b0 0x7f25981f9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:02.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.322+0000 7f259d000640 1 -- 172.21.15.86:0/962385064 >> 172.21.15.86:0/962385064 conn(0x7f25980b4650 msgr2=0x7f25980cbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:02.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.322+0000 7f259d000640 1 -- 172.21.15.86:0/962385064 shutdown_connections 2024-04-25T14:18:02.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.322+0000 7f259d000640 1 -- 172.21.15.86:0/962385064 wait complete. 2024-04-25T14:18:02.338 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T14:18:02.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.494+0000 7f2c9bc00640 1 Processor -- start 2024-04-25T14:18:02.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.494+0000 7f2c9bc00640 1 -- start start 2024-04-25T14:18:02.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.498+0000 7f2c9bc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2c940701f0 0x7f2c940729f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:02.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.498+0000 7f2c9bc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c94070600 0x7f2c94072f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:02.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.498+0000 7f2c9bc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c9406f430 con 0x7f2c9406c670 2024-04-25T14:18:02.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.498+0000 7f2c9bc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2c9406f1e0 con 0x7f2c940701f0 2024-04-25T14:18:02.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.498+0000 7f2c9bc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2c9406f710 con 0x7f2c94070600 2024-04-25T14:18:02.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.498+0000 7f2c99200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2c940701f0 0x7f2c940729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:02.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.498+0000 7f2c99200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2c940701f0 0x7f2c940729f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35662/0 (socket says 172.21.15.86:35662) 2024-04-25T14:18:02.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.498+0000 7f2c99200640 1 -- 172.21.15.86:0/3694979877 learned_addr learned my addr 172.21.15.86:0/3694979877 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:02.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.498+0000 7f2c93e00640 1 --2- 172.21.15.86:0/3694979877 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c94070600 0x7f2c94072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:02.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.498+0000 7f2c99200640 1 -- 172.21.15.86:0/3694979877 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c94070600 msgr2=0x7f2c94072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:02.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.498+0000 7f2c99200640 1 --2- 172.21.15.86:0/3694979877 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c94070600 0x7f2c94072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:02.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.498+0000 7f2c99200640 1 -- 172.21.15.86:0/3694979877 >> v1:172.21.15.86:6789/0 conn(0x7f2c9406c670 legacy=0x7f2c9406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:02.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.498+0000 7f2c99200640 1 -- 172.21.15.86:0/3694979877 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9406f9f0 con 0x7f2c940701f0 2024-04-25T14:18:02.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.502+0000 7f2c99200640 1 --2- 172.21.15.86:0/3694979877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c940701f0 0x7f2c940729f0 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f2c8400d150 tx=0x7f2c8401eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4531769453a3d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:02.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.502+0000 7f2c93400640 1 -- 172.21.15.86:0/3694979877 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c840ad070 con 0x7f2c940701f0 2024-04-25T14:18:02.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.502+0000 7f2c93400640 1 -- 172.21.15.86:0/3694979877 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c840a79f0 con 0x7f2c940701f0 2024-04-25T14:18:02.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.502+0000 7f2c93400640 1 -- 172.21.15.86:0/3694979877 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c840a7dc0 con 0x7f2c940701f0 2024-04-25T14:18:02.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.502+0000 7f2c9bc00640 1 -- 172.21.15.86:0/3694979877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c940701f0 msgr2=0x7f2c940729f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:02.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.502+0000 7f2c9bc00640 1 --2- 172.21.15.86:0/3694979877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c940701f0 0x7f2c940729f0 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f2c8400d150 tx=0x7f2c8401eba0 comp rx=0 tx=0).stop 2024-04-25T14:18:02.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.502+0000 7f2c9bc00640 1 -- 172.21.15.86:0/3694979877 shutdown_connections 2024-04-25T14:18:02.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.502+0000 7f2c9bc00640 1 --2- 172.21.15.86:0/3694979877 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c94070600 0x7f2c94072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:02.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.502+0000 7f2c9bc00640 1 --2- 172.21.15.86:0/3694979877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c940701f0 0x7f2c940729f0 unknown :-1 s=CLOSED pgs=2477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:02.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.502+0000 7f2c9bc00640 1 -- 172.21.15.86:0/3694979877 >> 172.21.15.86:0/3694979877 conn(0x7f2c940b4a40 msgr2=0x7f2c940b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:02.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.502+0000 7f2c9bc00640 1 -- 172.21.15.86:0/3694979877 shutdown_connections 2024-04-25T14:18:02.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.502+0000 7f2c9bc00640 1 -- 172.21.15.86:0/3694979877 wait complete. 2024-04-25T14:18:02.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.506+0000 7f2c9bc00640 1 Processor -- start 2024-04-25T14:18:02.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.506+0000 7f2c9bc00640 1 -- start start 2024-04-25T14:18:02.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.506+0000 7f2c9bc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c9406c670 0x7f2c9409eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:02.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.506+0000 7f2c9bc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c940701f0 0x7f2c94145b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:02.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.506+0000 7f2c9bc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c94070600 0x7f2c941460b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:02.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.506+0000 7f2c9bc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c94168f30 con 0x7f2c940701f0 2024-04-25T14:18:02.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.506+0000 7f2c9bc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9415d580 con 0x7f2c9406c670 2024-04-25T14:18:02.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.506+0000 7f2c9bc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2c941690b0 con 0x7f2c94070600 2024-04-25T14:18:02.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.506+0000 7f2c93e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c940701f0 0x7f2c94145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:02.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.506+0000 7f2c93e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c940701f0 0x7f2c94145b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35666/0 (socket says 172.21.15.86:35666) 2024-04-25T14:18:02.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.506+0000 7f2c93e00640 1 -- 172.21.15.86:0/901742701 learned_addr learned my addr 172.21.15.86:0/901742701 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:02.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.506+0000 7f2c99200640 1 --2- 172.21.15.86:0/901742701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c9406c670 0x7f2c9409eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:02.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.506+0000 7f2c99c00640 1 --2- 172.21.15.86:0/901742701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c94070600 0x7f2c941460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:02.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.506+0000 7f2c93e00640 1 -- 172.21.15.86:0/901742701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c94070600 msgr2=0x7f2c941460b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:02.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.506+0000 7f2c93e00640 1 --2- 172.21.15.86:0/901742701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c94070600 0x7f2c941460b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:02.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.506+0000 7f2c93e00640 1 -- 172.21.15.86:0/901742701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c9406c670 msgr2=0x7f2c9409eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:02.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.506+0000 7f2c93e00640 1 --2- 172.21.15.86:0/901742701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c9406c670 0x7f2c9409eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:02.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.506+0000 7f2c93e00640 1 -- 172.21.15.86:0/901742701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9415c360 con 0x7f2c940701f0 2024-04-25T14:18:02.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.506+0000 7f2c93e00640 1 --2- 172.21.15.86:0/901742701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c940701f0 0x7f2c94145b70 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000e8a0 tx=0x7f2c8000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:02.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.506+0000 7f2c90c00640 1 -- 172.21.15.86:0/901742701 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80017070 con 0x7f2c940701f0 2024-04-25T14:18:02.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.506+0000 7f2c90c00640 1 -- 172.21.15.86:0/901742701 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c8000c940 con 0x7f2c940701f0 2024-04-25T14:18:02.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.506+0000 7f2c90c00640 1 -- 172.21.15.86:0/901742701 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8000ccf0 con 0x7f2c940701f0 2024-04-25T14:18:02.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.506+0000 7f2c9bc00640 1 -- 172.21.15.86:0/901742701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c94143a20 con 0x7f2c940701f0 2024-04-25T14:18:02.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.506+0000 7f2c9bc00640 1 -- 172.21.15.86:0/901742701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c94143fe0 con 0x7f2c940701f0 2024-04-25T14:18:02.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.510+0000 7f2c9bc00640 1 -- 172.21.15.86:0/901742701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c94050750 con 0x7f2c940701f0 2024-04-25T14:18:02.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.510+0000 7f2c90c00640 1 -- 172.21.15.86:0/901742701 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7f2c8001a070 con 0x7f2c940701f0 2024-04-25T14:18:02.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.510+0000 7f2c90c00640 1 --2- 172.21.15.86:0/901742701 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f2c7406f430 0x7f2c74071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:02.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.510+0000 7f2c90c00640 1 -- 172.21.15.86:0/901742701 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 6875+0+0 (secure 0 0 0) 0x7f2c8008f8e0 con 0x7f2c940701f0 2024-04-25T14:18:02.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.510+0000 7f2c99200640 1 --2- 172.21.15.86:0/901742701 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f2c7406f430 0x7f2c74071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:02.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.510+0000 7f2c99200640 1 --2- 172.21.15.86:0/901742701 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f2c7406f430 0x7f2c74071940 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2c840b2000 tx=0x7f2c8400a040 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:02.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.514+0000 7f2c90c00640 1 -- 172.21.15.86:0/901742701 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2c8005bcf0 con 0x7f2c940701f0 2024-04-25T14:18:02.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:02.670+0000 7f2c9bc00640 1 -- 172.21.15.86:0/901742701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2c94073060 con 0x7f2c940701f0 2024-04-25T14:18:03.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.322+0000 7f2c90c00640 1 -- 172.21.15.86:0/901742701 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v159) ==== 181+0+0 (secure 0 0 0) 0x7f2c8005fff0 con 0x7f2c940701f0 2024-04-25T14:18:03.317 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-04-25T14:18:03.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.326+0000 7f2c9bc00640 1 -- 172.21.15.86:0/901742701 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f2c7406f430 msgr2=0x7f2c74071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:03.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.326+0000 7f2c9bc00640 1 --2- 172.21.15.86:0/901742701 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f2c7406f430 0x7f2c74071940 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2c840b2000 tx=0x7f2c8400a040 comp rx=0 tx=0).stop 2024-04-25T14:18:03.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.326+0000 7f2c9bc00640 1 -- 172.21.15.86:0/901742701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c940701f0 msgr2=0x7f2c94145b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:03.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.326+0000 7f2c9bc00640 1 --2- 172.21.15.86:0/901742701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c940701f0 0x7f2c94145b70 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000e8a0 tx=0x7f2c8000eca0 comp rx=0 tx=0).stop 2024-04-25T14:18:03.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.326+0000 7f2c9bc00640 1 -- 172.21.15.86:0/901742701 shutdown_connections 2024-04-25T14:18:03.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.326+0000 7f2c9bc00640 1 --2- 172.21.15.86:0/901742701 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f2c7406f430 0x7f2c74071940 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:03.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.326+0000 7f2c9bc00640 1 --2- 172.21.15.86:0/901742701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c94070600 0x7f2c941460b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:03.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.326+0000 7f2c9bc00640 1 --2- 172.21.15.86:0/901742701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c940701f0 0x7f2c94145b70 unknown :-1 s=CLOSED pgs=2478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:03.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.326+0000 7f2c9bc00640 1 --2- 172.21.15.86:0/901742701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c9406c670 0x7f2c9409eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:03.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.326+0000 7f2c9bc00640 1 -- 172.21.15.86:0/901742701 >> 172.21.15.86:0/901742701 conn(0x7f2c940b4a40 msgr2=0x7f2c940aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:03.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.326+0000 7f2c9bc00640 1 -- 172.21.15.86:0/901742701 shutdown_connections 2024-04-25T14:18:03.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.326+0000 7f2c9bc00640 1 -- 172.21.15.86:0/901742701 wait complete. 2024-04-25T14:18:03.345 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T14:18:03.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.502+0000 7f7446200640 1 Processor -- start 2024-04-25T14:18:03.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.502+0000 7f7446200640 1 -- start start 2024-04-25T14:18:03.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.502+0000 7f7446200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f74400b1db0 0x7f74400b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:03.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.502+0000 7f7446200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f74400b0310 0x7f74400b0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:03.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.502+0000 7f7446200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74400b2700 con 0x7f744006ed60 2024-04-25T14:18:03.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.502+0000 7f7446200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f744006f170 con 0x7f74400b1db0 2024-04-25T14:18:03.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.502+0000 7f7446200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f74400d2890 con 0x7f74400b0310 2024-04-25T14:18:03.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.502+0000 7f743ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f74400b1db0 0x7f74400b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:03.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.502+0000 7f743ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f74400b1db0 0x7f74400b21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35674/0 (socket says 172.21.15.86:35674) 2024-04-25T14:18:03.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.502+0000 7f743ea00640 1 -- 172.21.15.86:0/1013202646 learned_addr learned my addr 172.21.15.86:0/1013202646 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:03.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.506+0000 7f743fe00640 1 --2- 172.21.15.86:0/1013202646 >> v2:172.21.15.155:3300/0 conn(0x7f74400b0310 0x7f74400b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:03.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.506+0000 7f743ea00640 1 -- 172.21.15.86:0/1013202646 >> v2:172.21.15.155:3300/0 conn(0x7f74400b0310 msgr2=0x7f74400b0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:03.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.506+0000 7f743ea00640 1 --2- 172.21.15.86:0/1013202646 >> v2:172.21.15.155:3300/0 conn(0x7f74400b0310 0x7f74400b0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:03.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.506+0000 7f743ea00640 1 -- 172.21.15.86:0/1013202646 >> v1:172.21.15.155:6789/0 conn(0x7f744006ed60 legacy=0x7f74400d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:03.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.506+0000 7f743ea00640 1 -- 172.21.15.86:0/1013202646 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74400d2b70 con 0x7f74400b1db0 2024-04-25T14:18:03.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.506+0000 7f743ea00640 1 --2- 172.21.15.86:0/1013202646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74400b1db0 0x7f74400b21c0 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f744009be70 tx=0x7f742c01c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc6a541f81c76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:03.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.506+0000 7f743e000640 1 -- 172.21.15.86:0/1013202646 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c0b0070 con 0x7f74400b1db0 2024-04-25T14:18:03.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.506+0000 7f743e000640 1 -- 172.21.15.86:0/1013202646 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f742c0ab600 con 0x7f74400b1db0 2024-04-25T14:18:03.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.506+0000 7f743e000640 1 -- 172.21.15.86:0/1013202646 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c0ab9d0 con 0x7f74400b1db0 2024-04-25T14:18:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.510+0000 7f7446200640 1 -- 172.21.15.86:0/1013202646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74400b1db0 msgr2=0x7f74400b21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.510+0000 7f7446200640 1 --2- 172.21.15.86:0/1013202646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74400b1db0 0x7f74400b21c0 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f744009be70 tx=0x7f742c01c240 comp rx=0 tx=0).stop 2024-04-25T14:18:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.510+0000 7f7446200640 1 -- 172.21.15.86:0/1013202646 shutdown_connections 2024-04-25T14:18:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.510+0000 7f7446200640 1 --2- 172.21.15.86:0/1013202646 >> v2:172.21.15.155:3300/0 conn(0x7f74400b0310 0x7f74400b0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.510+0000 7f7446200640 1 --2- 172.21.15.86:0/1013202646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74400b1db0 0x7f74400b21c0 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.510+0000 7f7446200640 1 -- 172.21.15.86:0/1013202646 >> 172.21.15.86:0/1013202646 conn(0x7f7440073c90 msgr2=0x7f74400b77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.510+0000 7f7446200640 1 -- 172.21.15.86:0/1013202646 shutdown_connections 2024-04-25T14:18:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.510+0000 7f7446200640 1 -- 172.21.15.86:0/1013202646 wait complete. 2024-04-25T14:18:03.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.510+0000 7f7446200640 1 Processor -- start 2024-04-25T14:18:03.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.510+0000 7f7446200640 1 -- start start 2024-04-25T14:18:03.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.514+0000 7f7446200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f744006ed60 0x7f74401f9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:03.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.514+0000 7f7446200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74400b0310 0x7f744015b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.514+0000 7f7446200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74400b1db0 0x7f744015bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.514+0000 7f7446200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74401698a0 con 0x7f74400b1db0 2024-04-25T14:18:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.514+0000 7f7446200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7440169720 con 0x7f74400b0310 2024-04-25T14:18:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.514+0000 7f7446200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7440169a20 con 0x7f744006ed60 2024-04-25T14:18:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.514+0000 7f743fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74400b1db0 0x7f744015bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.514+0000 7f743fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74400b1db0 0x7f744015bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35682/0 (socket says 172.21.15.86:35682) 2024-04-25T14:18:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.514+0000 7f743fe00640 1 -- 172.21.15.86:0/2675889753 learned_addr learned my addr 172.21.15.86:0/2675889753 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.514+0000 7f743f400640 1 --2- 172.21.15.86:0/2675889753 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f744006ed60 0x7f74401f9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.514+0000 7f743ea00640 1 --2- 172.21.15.86:0/2675889753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74400b0310 0x7f744015b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.514+0000 7f743fe00640 1 -- 172.21.15.86:0/2675889753 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f744006ed60 msgr2=0x7f74401f9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.514+0000 7f743fe00640 1 --2- 172.21.15.86:0/2675889753 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f744006ed60 0x7f74401f9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.514+0000 7f743fe00640 1 -- 172.21.15.86:0/2675889753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74400b0310 msgr2=0x7f744015b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.514+0000 7f743fe00640 1 --2- 172.21.15.86:0/2675889753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74400b0310 0x7f744015b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.514+0000 7f743fe00640 1 -- 172.21.15.86:0/2675889753 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74400d2150 con 0x7f74400b1db0 2024-04-25T14:18:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.514+0000 7f743fe00640 1 --2- 172.21.15.86:0/2675889753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74400b1db0 0x7f744015bdf0 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f7438006e90 tx=0x7f743800d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.514+0000 7f7437400640 1 -- 172.21.15.86:0/2675889753 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74380125f0 con 0x7f74400b1db0 2024-04-25T14:18:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.514+0000 7f7437400640 1 -- 172.21.15.86:0/2675889753 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7438019070 con 0x7f74400b1db0 2024-04-25T14:18:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.514+0000 7f7437400640 1 -- 172.21.15.86:0/2675889753 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74380129d0 con 0x7f74400b1db0 2024-04-25T14:18:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.514+0000 7f7446200640 1 -- 172.21.15.86:0/2675889753 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7440152f40 con 0x7f74400b1db0 2024-04-25T14:18:03.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.514+0000 7f7446200640 1 -- 172.21.15.86:0/2675889753 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f744006d540 con 0x7f74400b1db0 2024-04-25T14:18:03.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.514+0000 7f7446200640 1 -- 172.21.15.86:0/2675889753 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74400b3140 con 0x7f74400b1db0 2024-04-25T14:18:03.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.518+0000 7f7437400640 1 -- 172.21.15.86:0/2675889753 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7f7438012b70 con 0x7f74400b1db0 2024-04-25T14:18:03.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.518+0000 7f7437400640 1 --2- 172.21.15.86:0/2675889753 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f742006f400 0x7f7420071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:03.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.518+0000 7f7437400640 1 -- 172.21.15.86:0/2675889753 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 6437+0+0 (secure 0 0 0) 0x7f743808f7e0 con 0x7f74400b1db0 2024-04-25T14:18:03.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.518+0000 7f743f400640 1 --2- 172.21.15.86:0/2675889753 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f742006f400 0x7f7420071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:03.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.518+0000 7f743f400640 1 --2- 172.21.15.86:0/2675889753 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f742006f400 0x7f7420071910 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7440072c70 tx=0x7f743000d040 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:03.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.522+0000 7f7437400640 1 -- 172.21.15.86:0/2675889753 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f743805be30 con 0x7f74400b1db0 2024-04-25T14:18:03.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:03.678+0000 7f7446200640 1 -- 172.21.15.86:0/2675889753 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f74400008e0 con 0x7f74400b1db0 2024-04-25T14:18:04.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.326+0000 7f7437400640 1 -- 172.21.15.86:0/2675889753 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v160) ==== 169+0+0 (secure 0 0 0) 0x7f7438060130 con 0x7f74400b1db0 2024-04-25T14:18:04.320 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-04-25T14:18:04.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.326+0000 7f7446200640 1 -- 172.21.15.86:0/2675889753 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f742006f400 msgr2=0x7f7420071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:04.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.326+0000 7f7446200640 1 --2- 172.21.15.86:0/2675889753 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f742006f400 0x7f7420071910 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7440072c70 tx=0x7f743000d040 comp rx=0 tx=0).stop 2024-04-25T14:18:04.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.326+0000 7f7446200640 1 -- 172.21.15.86:0/2675889753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74400b1db0 msgr2=0x7f744015bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:04.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.326+0000 7f7446200640 1 --2- 172.21.15.86:0/2675889753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74400b1db0 0x7f744015bdf0 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f7438006e90 tx=0x7f743800d9e0 comp rx=0 tx=0).stop 2024-04-25T14:18:04.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.326+0000 7f7446200640 1 -- 172.21.15.86:0/2675889753 shutdown_connections 2024-04-25T14:18:04.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.326+0000 7f7446200640 1 --2- 172.21.15.86:0/2675889753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74400b1db0 0x7f744015bdf0 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:04.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.326+0000 7f7446200640 1 --2- 172.21.15.86:0/2675889753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74400b0310 0x7f744015b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:04.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.326+0000 7f7446200640 1 --2- 172.21.15.86:0/2675889753 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f742006f400 0x7f7420071910 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:04.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.326+0000 7f7446200640 1 --2- 172.21.15.86:0/2675889753 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f744006ed60 0x7f74401f9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:04.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.326+0000 7f7446200640 1 -- 172.21.15.86:0/2675889753 >> 172.21.15.86:0/2675889753 conn(0x7f7440073c90 msgr2=0x7f7440071400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:04.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.326+0000 7f7446200640 1 -- 172.21.15.86:0/2675889753 shutdown_connections 2024-04-25T14:18:04.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.330+0000 7f7446200640 1 -- 172.21.15.86:0/2675889753 wait complete. 2024-04-25T14:18:04.353 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T14:18:04.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.506+0000 7f1a26800640 1 Processor -- start 2024-04-25T14:18:04.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.506+0000 7f1a26800640 1 -- start start 2024-04-25T14:18:04.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a26800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a2006fd50 0x7f1a20070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:04.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a26800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a200b0a60 con 0x7f1a2015c2a0 2024-04-25T14:18:04.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a26800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a200ae840 con 0x7f1a2015feb0 2024-04-25T14:18:04.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a26800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1a2006ef90 con 0x7f1a2006fd50 2024-04-25T14:18:04.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a1f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1a2015c2a0 0x7f1a200ae130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60562/0 (socket says 172.21.15.86:60562) 2024-04-25T14:18:04.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a1f400640 1 -- 172.21.15.86:0/3853110022 learned_addr learned my addr 172.21.15.86:0/3853110022 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:18:04.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a1ea00640 1 --2- 172.21.15.86:0/3853110022 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a2006fd50 0x7f1a20070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:04.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a1e000640 1 -- 172.21.15.86:0/3853110022 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3108672946 0 0) 0x7f1a200b0a60 con 0x7f1a2015c2a0 2024-04-25T14:18:04.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a1e000640 1 -- 172.21.15.86:0/3853110022 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a04005740 con 0x7f1a2015c2a0 2024-04-25T14:18:04.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a1e000640 1 -- 172.21.15.86:0/3853110022 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 337337573 0 0) 0x7f1a04005740 con 0x7f1a2015c2a0 2024-04-25T14:18:04.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a1e000640 1 -- 172.21.15.86:0/3853110022 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a2006fd50 msgr2=0x7f1a20070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:04.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a1e000640 1 --2- 172.21.15.86:0/3853110022 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a2006fd50 0x7f1a20070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:04.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a1e000640 1 -- 172.21.15.86:0/3853110022 >> v1:172.21.15.155:6789/0 conn(0x7f1a2015feb0 legacy=0x7f1a201602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:04.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a1e000640 1 -- 172.21.15.86:0/3853110022 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a2006f270 con 0x7f1a2015c2a0 2024-04-25T14:18:04.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a1e000640 1 -- 172.21.15.86:0/3853110022 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f1a10001680 con 0x7f1a2015c2a0 2024-04-25T14:18:04.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a1e000640 1 -- 172.21.15.86:0/3853110022 >> v1:172.21.15.86:6789/0 conn(0x7f1a2015c2a0 legacy=0x7f1a200ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:04.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a1e000640 1 --2- 172.21.15.86:0/3853110022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a04005c70 0x7f1a040060e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:04.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a1e000640 1 --2- 172.21.15.86:0/3853110022 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a2006fd50 0x7f1a04006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:04.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a1e000640 1 --2- 172.21.15.86:0/3853110022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a04006f60 0x7f1a040073f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:04.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a1e000640 1 -- 172.21.15.86:0/3853110022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2006ef90 con 0x7f1a04006f60 2024-04-25T14:18:04.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a1e000640 1 -- 172.21.15.86:0/3853110022 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0400de40 con 0x7f1a04005c70 2024-04-25T14:18:04.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a1e000640 1 -- 172.21.15.86:0/3853110022 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1a0400e040 con 0x7f1a2006fd50 2024-04-25T14:18:04.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a1fe00640 1 --2- 172.21.15.86:0/3853110022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a04006f60 0x7f1a040073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:04.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a1ea00640 1 --2- 172.21.15.86:0/3853110022 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a2006fd50 0x7f1a04006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:04.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a1f400640 1 --2- 172.21.15.86:0/3853110022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a04005c70 0x7f1a040060e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:04.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a1fe00640 1 -- 172.21.15.86:0/3853110022 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a2006fd50 msgr2=0x7f1a04006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:04.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a1fe00640 1 --2- 172.21.15.86:0/3853110022 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a2006fd50 0x7f1a04006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:04.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a1fe00640 1 -- 172.21.15.86:0/3853110022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a04005c70 msgr2=0x7f1a040060e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:04.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a1fe00640 1 --2- 172.21.15.86:0/3853110022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a04005c70 0x7f1a040060e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:04.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.510+0000 7f1a1fe00640 1 -- 172.21.15.86:0/3853110022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a04005740 con 0x7f1a04006f60 2024-04-25T14:18:04.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.514+0000 7f1a1fe00640 1 --2- 172.21.15.86:0/3853110022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a04006f60 0x7f1a040073f0 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f1a18001e70 tx=0x7f1a1801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf72a1b2ffe4284f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:04.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.514+0000 7f1a1e000640 1 -- 172.21.15.86:0/3853110022 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18001a90 con 0x7f1a04006f60 2024-04-25T14:18:04.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.514+0000 7f1a1e000640 1 -- 172.21.15.86:0/3853110022 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a18003b80 con 0x7f1a04006f60 2024-04-25T14:18:04.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.514+0000 7f1a1e000640 1 -- 172.21.15.86:0/3853110022 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18005080 con 0x7f1a04006f60 2024-04-25T14:18:04.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.514+0000 7f1a26800640 1 -- 172.21.15.86:0/3853110022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a04006f60 msgr2=0x7f1a040073f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:04.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.514+0000 7f1a26800640 1 --2- 172.21.15.86:0/3853110022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a04006f60 0x7f1a040073f0 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f1a18001e70 tx=0x7f1a1801c780 comp rx=0 tx=0).stop 2024-04-25T14:18:04.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.514+0000 7f1a1fe00640 1 -- 172.21.15.86:0/3853110022 reap_dead start 2024-04-25T14:18:04.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.514+0000 7f1a26800640 1 -- 172.21.15.86:0/3853110022 shutdown_connections 2024-04-25T14:18:04.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.514+0000 7f1a26800640 1 -- 172.21.15.86:0/3853110022 >> 172.21.15.86:0/3853110022 conn(0x7f1a200b4650 msgr2=0x7f1a200b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:04.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.514+0000 7f1a26800640 1 -- 172.21.15.86:0/3853110022 shutdown_connections 2024-04-25T14:18:04.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.514+0000 7f1a26800640 1 -- 172.21.15.86:0/3853110022 wait complete. 2024-04-25T14:18:04.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.518+0000 7f1a26800640 1 Processor -- start 2024-04-25T14:18:04.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.518+0000 7f1a26800640 1 -- start start 2024-04-25T14:18:04.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.518+0000 7f1a26800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a04006f60 0x7f1a2015e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:04.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.518+0000 7f1a26800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a201f2480 0x7f1a200b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:04.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.518+0000 7f1a26800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a200b3900 0x7f1a200b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:04.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.518+0000 7f1a26800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1a200d0fc0 con 0x7f1a201f2480 2024-04-25T14:18:04.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.518+0000 7f1a26800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2006f830 con 0x7f1a200b3900 2024-04-25T14:18:04.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.518+0000 7f1a26800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1a2015c6b0 con 0x7f1a04006f60 2024-04-25T14:18:04.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.518+0000 7f1a1ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a201f2480 0x7f1a200b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:04.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.518+0000 7f1a1ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a201f2480 0x7f1a200b13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35698/0 (socket says 172.21.15.86:35698) 2024-04-25T14:18:04.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.518+0000 7f1a1ea00640 1 -- 172.21.15.86:0/1152203182 learned_addr learned my addr 172.21.15.86:0/1152203182 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:04.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.518+0000 7f1a1fe00640 1 --2- 172.21.15.86:0/1152203182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a200b3900 0x7f1a200b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:04.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.518+0000 7f1a1f400640 1 --2- 172.21.15.86:0/1152203182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a04006f60 0x7f1a2015e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:04.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.518+0000 7f1a1ea00640 1 -- 172.21.15.86:0/1152203182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a04006f60 msgr2=0x7f1a2015e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:04.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.518+0000 7f1a1ea00640 1 --2- 172.21.15.86:0/1152203182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a04006f60 0x7f1a2015e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:04.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.518+0000 7f1a1ea00640 1 -- 172.21.15.86:0/1152203182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a200b3900 msgr2=0x7f1a200b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:04.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.518+0000 7f1a1ea00640 1 --2- 172.21.15.86:0/1152203182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a200b3900 0x7f1a200b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:04.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.518+0000 7f1a1ea00640 1 -- 172.21.15.86:0/1152203182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a2015ecf0 con 0x7f1a201f2480 2024-04-25T14:18:04.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.518+0000 7f1a1f400640 1 --2- 172.21.15.86:0/1152203182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a04006f60 0x7f1a2015e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:18:04.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.518+0000 7f1a1ea00640 1 --2- 172.21.15.86:0/1152203182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a201f2480 0x7f1a200b13a0 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f1a08011a00 tx=0x7f1a08011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:04.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.518+0000 7f1a17400640 1 -- 172.21.15.86:0/1152203182 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0800fb50 con 0x7f1a201f2480 2024-04-25T14:18:04.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.518+0000 7f1a17400640 1 -- 172.21.15.86:0/1152203182 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a0800fcf0 con 0x7f1a201f2480 2024-04-25T14:18:04.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.518+0000 7f1a17400640 1 -- 172.21.15.86:0/1152203182 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0800fea0 con 0x7f1a201f2480 2024-04-25T14:18:04.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.518+0000 7f1a26800640 1 -- 172.21.15.86:0/1152203182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a2015efd0 con 0x7f1a201f2480 2024-04-25T14:18:04.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.522+0000 7f1a26800640 1 -- 172.21.15.86:0/1152203182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a200b2420 con 0x7f1a201f2480 2024-04-25T14:18:04.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.522+0000 7f1a26800640 1 -- 172.21.15.86:0/1152203182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a2015ea60 con 0x7f1a201f2480 2024-04-25T14:18:04.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.522+0000 7f1a17400640 1 -- 172.21.15.86:0/1152203182 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7f1a08012040 con 0x7f1a201f2480 2024-04-25T14:18:04.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.522+0000 7f1a17400640 1 --2- 172.21.15.86:0/1152203182 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f19fc06f400 0x7f19fc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:04.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.522+0000 7f1a17400640 1 -- 172.21.15.86:0/1152203182 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 6023+0+0 (secure 0 0 0) 0x7f1a0808fa80 con 0x7f1a201f2480 2024-04-25T14:18:04.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.522+0000 7f1a1f400640 1 --2- 172.21.15.86:0/1152203182 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f19fc06f400 0x7f19fc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:04.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.522+0000 7f1a1f400640 1 --2- 172.21.15.86:0/1152203182 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f19fc06f400 0x7f19fc071910 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1a10001cd0 tx=0x7f1a10001b00 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:04.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.526+0000 7f1a17400640 1 -- 172.21.15.86:0/1152203182 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1a08019030 con 0x7f1a201f2480 2024-04-25T14:18:04.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.686+0000 7f1a26800640 1 -- 172.21.15.86:0/1152203182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f1a20069d00 con 0x7f1a201f2480 2024-04-25T14:18:04.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.690+0000 7f1a17400640 1 -- 172.21.15.86:0/1152203182 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) ==== 104+0+0 (secure 0 0 0) 0x7f1a0805c980 con 0x7f1a201f2480 2024-04-25T14:18:04.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.690+0000 7f1a26800640 1 -- 172.21.15.86:0/1152203182 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f19fc06f400 msgr2=0x7f19fc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:04.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.690+0000 7f1a26800640 1 --2- 172.21.15.86:0/1152203182 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f19fc06f400 0x7f19fc071910 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1a10001cd0 tx=0x7f1a10001b00 comp rx=0 tx=0).stop 2024-04-25T14:18:04.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.690+0000 7f1a26800640 1 -- 172.21.15.86:0/1152203182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a201f2480 msgr2=0x7f1a200b13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:04.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.690+0000 7f1a26800640 1 --2- 172.21.15.86:0/1152203182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a201f2480 0x7f1a200b13a0 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f1a08011a00 tx=0x7f1a08011ec0 comp rx=0 tx=0).stop 2024-04-25T14:18:04.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.690+0000 7f1a26800640 1 -- 172.21.15.86:0/1152203182 shutdown_connections 2024-04-25T14:18:04.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.690+0000 7f1a26800640 1 --2- 172.21.15.86:0/1152203182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a200b3900 0x7f1a200b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:04.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.690+0000 7f1a26800640 1 --2- 172.21.15.86:0/1152203182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a201f2480 0x7f1a200b13a0 unknown :-1 s=CLOSED pgs=2483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:04.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.690+0000 7f1a26800640 1 --2- 172.21.15.86:0/1152203182 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f19fc06f400 0x7f19fc071910 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:04.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.690+0000 7f1a26800640 1 --2- 172.21.15.86:0/1152203182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a04006f60 0x7f1a2015e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:04.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.690+0000 7f1a26800640 1 -- 172.21.15.86:0/1152203182 >> 172.21.15.86:0/1152203182 conn(0x7f1a200b4650 msgr2=0x7f1a201591a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:04.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.694+0000 7f1a26800640 1 -- 172.21.15.86:0/1152203182 shutdown_connections 2024-04-25T14:18:04.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.694+0000 7f1a26800640 1 -- 172.21.15.86:0/1152203182 wait complete. 2024-04-25T14:18:04.706 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_metrics_mds_grow_and_shrink' 2024-04-25T14:18:04.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.858+0000 7f17e1e00640 1 Processor -- start 2024-04-25T14:18:04.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.858+0000 7f17e1e00640 1 -- start start 2024-04-25T14:18:04.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.858+0000 7f17e1e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17dc0d1df0 0x7f17dc0cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:04.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.858+0000 7f17e1e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17dc15e5c0 con 0x7f17dc070330 2024-04-25T14:18:04.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17e1e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17dc15e8a0 con 0x7f17dc0cdcc0 2024-04-25T14:18:04.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17e1e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f17dc06fd40 con 0x7f17dc0d1df0 2024-04-25T14:18:04.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17daa00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f17dc070330 0x7f17dc0d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60568/0 (socket says 172.21.15.86:60568) 2024-04-25T14:18:04.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17daa00640 1 -- 172.21.15.86:0/1074378368 learned_addr learned my addr 172.21.15.86:0/1074378368 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:18:04.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17da000640 1 --2- 172.21.15.86:0/1074378368 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17dc0d1df0 0x7f17dc0cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:04.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17d9600640 1 -- 172.21.15.86:0/1074378368 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3646543321 0 0) 0x7f17dc15e5c0 con 0x7f17dc070330 2024-04-25T14:18:04.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17d9600640 1 -- 172.21.15.86:0/1074378368 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17c0003530 con 0x7f17dc070330 2024-04-25T14:18:04.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17d9600640 1 -- 172.21.15.86:0/1074378368 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1501669411 0 0) 0x7f17dc15e8a0 con 0x7f17dc0cdcc0 2024-04-25T14:18:04.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17d9600640 1 -- 172.21.15.86:0/1074378368 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17dc15e5c0 con 0x7f17dc0cdcc0 2024-04-25T14:18:04.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17d9600640 1 -- 172.21.15.86:0/1074378368 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3220445885 0 0) 0x7f17c0003530 con 0x7f17dc070330 2024-04-25T14:18:04.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17d9600640 1 -- 172.21.15.86:0/1074378368 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17dc0d1df0 msgr2=0x7f17dc0cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:04.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17d9600640 1 --2- 172.21.15.86:0/1074378368 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17dc0d1df0 0x7f17dc0cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:04.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17d9600640 1 -- 172.21.15.86:0/1074378368 >> v1:172.21.15.155:6789/0 conn(0x7f17dc0cdcc0 legacy=0x7f17dc0ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:04.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17d9600640 1 -- 172.21.15.86:0/1074378368 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17dc15eb80 con 0x7f17dc070330 2024-04-25T14:18:04.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17d9600640 1 -- 172.21.15.86:0/1074378368 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f17cc001a00 con 0x7f17dc070330 2024-04-25T14:18:04.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17da000640 1 --2- 172.21.15.86:0/1074378368 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17dc0d1df0 0x7f17dc0cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:18:04.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17d9600640 1 -- 172.21.15.86:0/1074378368 >> v1:172.21.15.86:6789/0 conn(0x7f17dc070330 legacy=0x7f17dc0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:04.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17d9600640 1 --2- 172.21.15.86:0/1074378368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17c0003bf0 0x7f17c0004040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:04.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17d9600640 1 --2- 172.21.15.86:0/1074378368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17c0004810 0x7f17c0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:04.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17d9600640 1 --2- 172.21.15.86:0/1074378368 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17c0005290 0x7f17c000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:04.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17d9600640 1 -- 172.21.15.86:0/1074378368 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f17c000bfc0 con 0x7f17c0003bf0 2024-04-25T14:18:04.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17d9600640 1 -- 172.21.15.86:0/1074378368 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f17c000c170 con 0x7f17c0004810 2024-04-25T14:18:04.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17d9600640 1 -- 172.21.15.86:0/1074378368 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f17c000c370 con 0x7f17c0005290 2024-04-25T14:18:04.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17daa00640 1 --2- 172.21.15.86:0/1074378368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17c0003bf0 0x7f17c0004040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:04.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17da000640 1 --2- 172.21.15.86:0/1074378368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17c0004810 0x7f17c0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:04.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17db400640 1 --2- 172.21.15.86:0/1074378368 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17c0005290 0x7f17c000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:04.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17daa00640 1 -- 172.21.15.86:0/1074378368 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17c0005290 msgr2=0x7f17c000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:04.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17daa00640 1 --2- 172.21.15.86:0/1074378368 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17c0005290 0x7f17c000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:04.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17daa00640 1 -- 172.21.15.86:0/1074378368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17c0004810 msgr2=0x7f17c0004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:04.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17daa00640 1 --2- 172.21.15.86:0/1074378368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17c0004810 0x7f17c0004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:04.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17daa00640 1 -- 172.21.15.86:0/1074378368 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c0003530 con 0x7f17c0003bf0 2024-04-25T14:18:04.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.862+0000 7f17db400640 1 --2- 172.21.15.86:0/1074378368 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17c0005290 0x7f17c000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:18:04.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.866+0000 7f17daa00640 1 --2- 172.21.15.86:0/1074378368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17c0003bf0 0x7f17c0004040 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f17cc00a7e0 tx=0x7f17cc01d2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5eb59ef051013e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:04.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.866+0000 7f17d9600640 1 -- 172.21.15.86:0/1074378368 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc00a430 con 0x7f17c0003bf0 2024-04-25T14:18:04.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.866+0000 7f17d9600640 1 -- 172.21.15.86:0/1074378368 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17cc0a5e50 con 0x7f17c0003bf0 2024-04-25T14:18:04.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.866+0000 7f17d9600640 1 -- 172.21.15.86:0/1074378368 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc0ad890 con 0x7f17c0003bf0 2024-04-25T14:18:04.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.866+0000 7f17e1e00640 1 -- 172.21.15.86:0/1074378368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17c0003bf0 msgr2=0x7f17c0004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:04.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.866+0000 7f17e1e00640 1 --2- 172.21.15.86:0/1074378368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17c0003bf0 0x7f17c0004040 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f17cc00a7e0 tx=0x7f17cc01d2b0 comp rx=0 tx=0).stop 2024-04-25T14:18:04.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.866+0000 7f17db400640 1 -- 172.21.15.86:0/1074378368 reap_dead start 2024-04-25T14:18:04.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.866+0000 7f17e1e00640 1 -- 172.21.15.86:0/1074378368 shutdown_connections 2024-04-25T14:18:04.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.866+0000 7f17e1e00640 1 -- 172.21.15.86:0/1074378368 >> 172.21.15.86:0/1074378368 conn(0x7f17dc0b4650 msgr2=0x7f17dc0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:04.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.866+0000 7f17e1e00640 1 -- 172.21.15.86:0/1074378368 shutdown_connections 2024-04-25T14:18:04.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.866+0000 7f17e1e00640 1 -- 172.21.15.86:0/1074378368 wait complete. 2024-04-25T14:18:04.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.870+0000 7f17e1e00640 1 Processor -- start 2024-04-25T14:18:04.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.870+0000 7f17e1e00640 1 -- start start 2024-04-25T14:18:04.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.870+0000 7f17e1e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17c0003bf0 0x7f17dc157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:04.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.870+0000 7f17e1e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17dc09d5c0 0x7f17dc15cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:04.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.870+0000 7f17e1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17dc157fd0 0x7f17dc15d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:04.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.870+0000 7f17e1e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc0708c0 con 0x7f17dc157fd0 2024-04-25T14:18:04.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.870+0000 7f17e1e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc070740 con 0x7f17c0003bf0 2024-04-25T14:18:04.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.870+0000 7f17e1e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f17dc0d1df0 con 0x7f17dc09d5c0 2024-04-25T14:18:04.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.870+0000 7f17db400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17dc157fd0 0x7f17dc15d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:04.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.870+0000 7f17db400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17dc157fd0 0x7f17dc15d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35716/0 (socket says 172.21.15.86:35716) 2024-04-25T14:18:04.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.870+0000 7f17db400640 1 -- 172.21.15.86:0/3008202250 learned_addr learned my addr 172.21.15.86:0/3008202250 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:04.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.870+0000 7f17daa00640 1 --2- 172.21.15.86:0/3008202250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17c0003bf0 0x7f17dc157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:04.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.870+0000 7f17da000640 1 --2- 172.21.15.86:0/3008202250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17dc09d5c0 0x7f17dc15cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:04.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.870+0000 7f17db400640 1 -- 172.21.15.86:0/3008202250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17dc09d5c0 msgr2=0x7f17dc15cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:04.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.870+0000 7f17db400640 1 --2- 172.21.15.86:0/3008202250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17dc09d5c0 0x7f17dc15cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:04.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.870+0000 7f17db400640 1 -- 172.21.15.86:0/3008202250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17c0003bf0 msgr2=0x7f17dc157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:04.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.870+0000 7f17db400640 1 --2- 172.21.15.86:0/3008202250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17c0003bf0 0x7f17dc157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:04.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.870+0000 7f17db400640 1 -- 172.21.15.86:0/3008202250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17dc09ce00 con 0x7f17dc157fd0 2024-04-25T14:18:04.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.870+0000 7f17db400640 1 --2- 172.21.15.86:0/3008202250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17dc157fd0 0x7f17dc15d270 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f17d4007af0 tx=0x7f17d400e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:04.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.870+0000 7f17d3400640 1 -- 172.21.15.86:0/3008202250 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4019070 con 0x7f17dc157fd0 2024-04-25T14:18:04.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.870+0000 7f17d3400640 1 -- 172.21.15.86:0/3008202250 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17d40052e0 con 0x7f17dc157fd0 2024-04-25T14:18:04.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.870+0000 7f17e1e00640 1 -- 172.21.15.86:0/3008202250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17dc0b2a40 con 0x7f17dc157fd0 2024-04-25T14:18:04.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.870+0000 7f17d3400640 1 -- 172.21.15.86:0/3008202250 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4003840 con 0x7f17dc157fd0 2024-04-25T14:18:04.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.870+0000 7f17e1e00640 1 -- 172.21.15.86:0/3008202250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17dc1587c0 con 0x7f17dc157fd0 2024-04-25T14:18:04.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.870+0000 7f17e1e00640 1 -- 172.21.15.86:0/3008202250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17dc069d00 con 0x7f17dc157fd0 2024-04-25T14:18:04.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.874+0000 7f17d3400640 1 -- 172.21.15.86:0/3008202250 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7f17d401c070 con 0x7f17dc157fd0 2024-04-25T14:18:04.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.874+0000 7f17d3400640 1 --2- 172.21.15.86:0/3008202250 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f17bc06f400 0x7f17bc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:04.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.874+0000 7f17d3400640 1 -- 172.21.15.86:0/3008202250 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 6023+0+0 (secure 0 0 0) 0x7f17d408eb50 con 0x7f17dc157fd0 2024-04-25T14:18:04.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.874+0000 7f17daa00640 1 --2- 172.21.15.86:0/3008202250 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f17bc06f400 0x7f17bc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:04.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.874+0000 7f17daa00640 1 --2- 172.21.15.86:0/3008202250 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f17bc06f400 0x7f17bc071910 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0ad000 tx=0x7f17cc001d50 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:04.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:04.878+0000 7f17d3400640 1 -- 172.21.15.86:0/3008202250 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f17d4023020 con 0x7f17dc157fd0 2024-04-25T14:18:05.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:05.110+0000 7f17e1e00640 1 -- 172.21.15.86:0/3008202250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_metrics_mds_grow_and_shrink"]} v 0) -- 0x7f17dc0b27d0 con 0x7f17dc157fd0 2024-04-25T14:18:05.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:05.338+0000 7f17d3400640 1 -- 172.21.15.86:0/3008202250 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_mds_metrics.TestMDSMetrics.test_metrics_mds_grow_and_shrink"]}]=0 v743) ==== 156+0+0 (secure 0 0 0) 0x7f17d405b420 con 0x7f17dc157fd0 2024-04-25T14:18:05.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:05.342+0000 7f17e1e00640 1 -- 172.21.15.86:0/3008202250 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f17bc06f400 msgr2=0x7f17bc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:05.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:05.342+0000 7f17e1e00640 1 --2- 172.21.15.86:0/3008202250 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f17bc06f400 0x7f17bc071910 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0ad000 tx=0x7f17cc001d50 comp rx=0 tx=0).stop 2024-04-25T14:18:05.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:05.342+0000 7f17e1e00640 1 -- 172.21.15.86:0/3008202250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17dc157fd0 msgr2=0x7f17dc15d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:05.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:05.342+0000 7f17e1e00640 1 --2- 172.21.15.86:0/3008202250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17dc157fd0 0x7f17dc15d270 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f17d4007af0 tx=0x7f17d400e890 comp rx=0 tx=0).stop 2024-04-25T14:18:05.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:05.342+0000 7f17e1e00640 1 -- 172.21.15.86:0/3008202250 shutdown_connections 2024-04-25T14:18:05.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:05.342+0000 7f17e1e00640 1 --2- 172.21.15.86:0/3008202250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17dc157fd0 0x7f17dc15d270 unknown :-1 s=CLOSED pgs=2486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:05.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:05.342+0000 7f17e1e00640 1 --2- 172.21.15.86:0/3008202250 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f17bc06f400 0x7f17bc071910 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:05.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:05.342+0000 7f17e1e00640 1 --2- 172.21.15.86:0/3008202250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17dc09d5c0 0x7f17dc15cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:05.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:05.342+0000 7f17e1e00640 1 --2- 172.21.15.86:0/3008202250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17c0003bf0 0x7f17dc157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:05.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:05.342+0000 7f17e1e00640 1 -- 172.21.15.86:0/3008202250 >> 172.21.15.86:0/3008202250 conn(0x7f17dc0b4650 msgr2=0x7f17dc15b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:05.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:05.342+0000 7f17e1e00640 1 -- 172.21.15.86:0/3008202250 shutdown_connections 2024-04-25T14:18:05.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:05.342+0000 7f17e1e00640 1 -- 172.21.15.86:0/3008202250 wait complete. 2024-04-25T14:18:05.357 INFO:tasks.cephfs_test_runner:test_metrics_mds_grow_and_shrink (tasks.cephfs.test_mds_metrics.TestMDSMetrics) ... ERROR 2024-04-25T14:18:05.357 INFO:tasks.cephfs_test_runner: 2024-04-25T14:18:05.357 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-25T14:18:05.357 INFO:tasks.cephfs_test_runner:ERROR: test_metrics_mds_grow_and_shrink (tasks.cephfs.test_mds_metrics.TestMDSMetrics) 2024-04-25T14:18:05.357 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T14:18:05.357 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-25T14:18:05.357 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/cephfs/test_mds_metrics.py", line 205, in test_metrics_mds_grow_and_shrink 2024-04-25T14:18:05.357 INFO:tasks.cephfs_test_runner: self.fs.shrink(1) 2024-04-25T14:18:05.358 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/cephfs/filesystem.py", line 1788, in shrink 2024-04-25T14:18:05.358 INFO:tasks.cephfs_test_runner: return self.wait_for_daemons() 2024-04-25T14:18:05.358 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/cephfs/filesystem.py", line 1278, in wait_for_daemons 2024-04-25T14:18:05.358 INFO:tasks.cephfs_test_runner: raise RuntimeError("Timed out waiting for MDS daemons to become healthy") 2024-04-25T14:18:05.358 INFO:tasks.cephfs_test_runner:RuntimeError: Timed out waiting for MDS daemons to become healthy 2024-04-25T14:18:05.358 INFO:tasks.cephfs_test_runner: 2024-04-25T14:18:05.358 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T14:18:05.358 INFO:tasks.cephfs_test_runner:Ran 8 tests in 807.034s 2024-04-25T14:18:05.358 INFO:tasks.cephfs_test_runner: 2024-04-25T14:18:05.358 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-04-25T14:18:05.358 INFO:tasks.cephfs_test_runner: 2024-04-25T14:18:05.358 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-25T14:18:05.358 INFO:tasks.cephfs_test_runner:ERROR: test_metrics_mds_grow_and_shrink (tasks.cephfs.test_mds_metrics.TestMDSMetrics) 2024-04-25T14:18:05.358 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T14:18:05.359 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-25T14:18:05.359 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/cephfs/test_mds_metrics.py", line 205, in test_metrics_mds_grow_and_shrink 2024-04-25T14:18:05.359 INFO:tasks.cephfs_test_runner: self.fs.shrink(1) 2024-04-25T14:18:05.359 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/cephfs/filesystem.py", line 1788, in shrink 2024-04-25T14:18:05.359 INFO:tasks.cephfs_test_runner: return self.wait_for_daemons() 2024-04-25T14:18:05.359 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/cephfs/filesystem.py", line 1278, in wait_for_daemons 2024-04-25T14:18:05.359 INFO:tasks.cephfs_test_runner: raise RuntimeError("Timed out waiting for MDS daemons to become healthy") 2024-04-25T14:18:05.359 INFO:tasks.cephfs_test_runner:RuntimeError: Timed out waiting for MDS daemons to become healthy 2024-04-25T14:18:05.359 INFO:tasks.cephfs_test_runner: 2024-04-25T14:18:05.359 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_metrics_mds_grow_and_shrink (tasks.cephfs.test_mds_metrics.TestMDSMetrics) 2024-04-25T14:18:05.568 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=9cb6ce27042041dcbfe30d7455ebf140 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_metrics_mds_grow_and_shrink (tasks.cephfs.test_mds_metrics.TestMDSMetrics) 2024-04-25T14:18:05.570 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T14:18:05.580 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T14:18:05.587 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T14:18:05.587 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:18:05.588 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:18:05.603 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2893 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T14:18:05.603 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-25T14:18:05.603 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:18:05.654 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2893 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T14:18:05.654 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:18:05.655 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:18:05.665 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4517 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T14:18:05.665 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-25T14:18:05.665 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:18:05.716 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350308k,nr_inodes=4087577,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4517 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274992k,nr_inodes=818748,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T14:18:05.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:18:05.716 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T14:18:05.726 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T14:18:05.726 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T14:18:05.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:18:05.727 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T14:18:05.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:18:05.799 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T14:18:05.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:18:05.867 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-04-25T14:18:06.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:18:06.006 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-25T14:18:06.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:18:06.015 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-04-25T14:18:06.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:18:06.083 INFO:teuthology.orchestra.run.smithi086.stderr:Cannot find device "ceph-brx" 2024-04-25T14:18:06.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:18:06.084 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2024-04-25T14:18:06.094 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T14:18:06.094 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T14:18:06.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:18:06.095 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T14:18:06.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:18:06.163 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T14:18:06.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:18:06.233 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link delete ceph-brx 2024-04-25T14:18:06.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:18:06.418 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2024-04-25T14:18:06.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:18:06.427 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link delete ceph-brx 2024-04-25T14:18:06.496 INFO:teuthology.orchestra.run.smithi155.stderr:Cannot find device "ceph-brx" 2024-04-25T14:18:06.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:18:06.499 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T14:18:06.508 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T14:18:06.508 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:18:06.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.662+0000 7f4730200640 1 Processor -- start 2024-04-25T14:18:06.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.662+0000 7f4730200640 1 -- start start 2024-04-25T14:18:06.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.662+0000 7f4730200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47280d16e0 0x7f47280d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:06.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.662+0000 7f4730200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f47280730b0 0x7f47280734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:06.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.662+0000 7f4730200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47280d2030 con 0x7f47280cfca0 2024-04-25T14:18:06.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.662+0000 7f4730200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f4728070470 con 0x7f47280730b0 2024-04-25T14:18:06.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.662+0000 7f4730200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4728072bd0 con 0x7f47280d16e0 2024-04-25T14:18:06.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.662+0000 7f472d800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f47280cfca0 0x7f472806f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60584/0 (socket says 172.21.15.86:60584) 2024-04-25T14:18:06.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.662+0000 7f472d800640 1 -- 172.21.15.86:0/2883179062 learned_addr learned my addr 172.21.15.86:0/2883179062 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:18:06.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.662+0000 7f472e200640 1 --2- 172.21.15.86:0/2883179062 >> v2:172.21.15.155:3300/0 conn(0x7f47280730b0 0x7f47280734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:06.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.662+0000 7f472ce00640 1 --2- 172.21.15.86:0/2883179062 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47280d16e0 0x7f47280d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:06.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.666+0000 7f4727e00640 1 -- 172.21.15.86:0/2883179062 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2427881567 0 0) 0x7f4718001120 con 0x7f47280cfca0 2024-04-25T14:18:06.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.666+0000 7f4727e00640 1 -- 172.21.15.86:0/2883179062 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4710003530 con 0x7f47280cfca0 2024-04-25T14:18:06.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.666+0000 7f472ce00640 1 -- 172.21.15.86:0/2883179062 >> v2:172.21.15.155:3300/0 conn(0x7f47280730b0 msgr2=0x7f47280734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:06.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.666+0000 7f472ce00640 1 --2- 172.21.15.86:0/2883179062 >> v2:172.21.15.155:3300/0 conn(0x7f47280730b0 0x7f47280734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:06.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.666+0000 7f472ce00640 1 -- 172.21.15.86:0/2883179062 >> v1:172.21.15.86:6789/0 conn(0x7f47280cfca0 legacy=0x7f472806f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:06.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.666+0000 7f472ce00640 1 -- 172.21.15.86:0/2883179062 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472806fbd0 con 0x7f47280d16e0 2024-04-25T14:18:06.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.666+0000 7f472e200640 1 --2- 172.21.15.86:0/2883179062 >> v2:172.21.15.155:3300/0 conn(0x7f47280730b0 0x7f47280734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:18:06.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.666+0000 7f472ce00640 1 --2- 172.21.15.86:0/2883179062 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47280d16e0 0x7f47280d1af0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f471c0097c0 tx=0x7f471c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ad6803de5a325e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:06.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.666+0000 7f4727e00640 1 -- 172.21.15.86:0/2883179062 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c0b1070 con 0x7f47280d16e0 2024-04-25T14:18:06.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.666+0000 7f4727e00640 1 -- 172.21.15.86:0/2883179062 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f471c0ac410 con 0x7f47280d16e0 2024-04-25T14:18:06.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.666+0000 7f4727e00640 1 -- 172.21.15.86:0/2883179062 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c0ac7c0 con 0x7f47280d16e0 2024-04-25T14:18:06.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.670+0000 7f4730200640 1 -- 172.21.15.86:0/2883179062 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47280d16e0 msgr2=0x7f47280d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:06.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.670+0000 7f4730200640 1 --2- 172.21.15.86:0/2883179062 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47280d16e0 0x7f47280d1af0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f471c0097c0 tx=0x7f471c01c4a0 comp rx=0 tx=0).stop 2024-04-25T14:18:06.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.670+0000 7f4730200640 1 -- 172.21.15.86:0/2883179062 shutdown_connections 2024-04-25T14:18:06.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.670+0000 7f4730200640 1 --2- 172.21.15.86:0/2883179062 >> v2:172.21.15.155:3300/0 conn(0x7f47280730b0 0x7f47280734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:06.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.670+0000 7f4730200640 1 --2- 172.21.15.86:0/2883179062 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47280d16e0 0x7f47280d1af0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:06.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.670+0000 7f4730200640 1 -- 172.21.15.86:0/2883179062 >> 172.21.15.86:0/2883179062 conn(0x7f47280b4650 msgr2=0x7f47280b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:06.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.670+0000 7f4730200640 1 -- 172.21.15.86:0/2883179062 shutdown_connections 2024-04-25T14:18:06.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.670+0000 7f4730200640 1 -- 172.21.15.86:0/2883179062 wait complete. 2024-04-25T14:18:06.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.670+0000 7f4730200640 1 Processor -- start 2024-04-25T14:18:06.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.670+0000 7f4730200640 1 -- start start 2024-04-25T14:18:06.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.674+0000 7f4730200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47280730b0 0x7f4728099a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:06.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.674+0000 7f4730200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47280cfca0 0x7f47280c0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:06.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.674+0000 7f4730200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47280d16e0 0x7f47280c0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:06.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.674+0000 7f4730200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4728169a70 con 0x7f47280d16e0 2024-04-25T14:18:06.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.674+0000 7f4730200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f47281698f0 con 0x7f47280730b0 2024-04-25T14:18:06.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.674+0000 7f4730200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4728169bf0 con 0x7f47280cfca0 2024-04-25T14:18:06.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.674+0000 7f472e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47280d16e0 0x7f47280c0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:06.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.674+0000 7f472e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47280d16e0 0x7f47280c0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35730/0 (socket says 172.21.15.86:35730) 2024-04-25T14:18:06.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.674+0000 7f472e200640 1 -- 172.21.15.86:0/3242720014 learned_addr learned my addr 172.21.15.86:0/3242720014 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:06.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.674+0000 7f472d800640 1 --2- 172.21.15.86:0/3242720014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47280730b0 0x7f4728099a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:06.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.674+0000 7f472ce00640 1 --2- 172.21.15.86:0/3242720014 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47280cfca0 0x7f47280c0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:06.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.674+0000 7f472e200640 1 -- 172.21.15.86:0/3242720014 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47280cfca0 msgr2=0x7f47280c0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:06.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.674+0000 7f472e200640 1 --2- 172.21.15.86:0/3242720014 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47280cfca0 0x7f47280c0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:06.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.674+0000 7f472e200640 1 -- 172.21.15.86:0/3242720014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47280730b0 msgr2=0x7f4728099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:06.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.674+0000 7f472e200640 1 --2- 172.21.15.86:0/3242720014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47280730b0 0x7f4728099a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:06.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.674+0000 7f472e200640 1 -- 172.21.15.86:0/3242720014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472809a4c0 con 0x7f47280d16e0 2024-04-25T14:18:06.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.674+0000 7f472ce00640 1 --2- 172.21.15.86:0/3242720014 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47280cfca0 0x7f47280c0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:18:06.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.674+0000 7f472d800640 1 --2- 172.21.15.86:0/3242720014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47280730b0 0x7f4728099a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:18:06.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.674+0000 7f472e200640 1 --2- 172.21.15.86:0/3242720014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47280d16e0 0x7f47280c0e60 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f472000ef90 tx=0x7f472000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:06.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.674+0000 7f4725600640 1 -- 172.21.15.86:0/3242720014 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4720018020 con 0x7f47280d16e0 2024-04-25T14:18:06.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.674+0000 7f4725600640 1 -- 172.21.15.86:0/3242720014 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4720014480 con 0x7f47280d16e0 2024-04-25T14:18:06.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.674+0000 7f4730200640 1 -- 172.21.15.86:0/3242720014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4728152fa0 con 0x7f47280d16e0 2024-04-25T14:18:06.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.674+0000 7f4725600640 1 -- 172.21.15.86:0/3242720014 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4720014850 con 0x7f47280d16e0 2024-04-25T14:18:06.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.674+0000 7f4730200640 1 -- 172.21.15.86:0/3242720014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4728153260 con 0x7f47280d16e0 2024-04-25T14:18:06.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.674+0000 7f4730200640 1 -- 172.21.15.86:0/3242720014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f472809f4b0 con 0x7f47280d16e0 2024-04-25T14:18:06.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.674+0000 7f4725600640 1 -- 172.21.15.86:0/3242720014 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7f472001b020 con 0x7f47280d16e0 2024-04-25T14:18:06.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.678+0000 7f4725600640 1 --2- 172.21.15.86:0/3242720014 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f470c06f400 0x7f470c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:06.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.678+0000 7f4725600640 1 -- 172.21.15.86:0/3242720014 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 6023+0+0 (secure 0 0 0) 0x7f4720090160 con 0x7f47280d16e0 2024-04-25T14:18:06.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.678+0000 7f472d800640 1 --2- 172.21.15.86:0/3242720014 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f470c06f400 0x7f470c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:06.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.678+0000 7f472d800640 1 --2- 172.21.15.86:0/3242720014 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f470c06f400 0x7f470c071910 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4718005890 tx=0x7f4718009040 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:06.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.682+0000 7f4725600640 1 -- 172.21.15.86:0/3242720014 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4720022030 con 0x7f47280d16e0 2024-04-25T14:18:06.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.834+0000 7f4730200640 1 -- 172.21.15.86:0/3242720014 --> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f472809a270 con 0x7f470c06f400 2024-04-25T14:18:06.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.838+0000 7f4725600640 1 -- 172.21.15.86:0/3242720014 <== mgr.11343 v2:172.21.15.86:6832/3174179029 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13520 (secure 0 0 0) 0x7f472809a270 con 0x7f470c06f400 2024-04-25T14:18:06.831 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:18:06.832 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:18:06.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.838+0000 7f4730200640 1 -- 172.21.15.86:0/3242720014 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f470c06f400 msgr2=0x7f470c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:06.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.838+0000 7f4730200640 1 --2- 172.21.15.86:0/3242720014 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f470c06f400 0x7f470c071910 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4718005890 tx=0x7f4718009040 comp rx=0 tx=0).stop 2024-04-25T14:18:06.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.838+0000 7f4730200640 1 -- 172.21.15.86:0/3242720014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47280d16e0 msgr2=0x7f47280c0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:06.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.838+0000 7f4730200640 1 --2- 172.21.15.86:0/3242720014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47280d16e0 0x7f47280c0e60 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f472000ef90 tx=0x7f472000c490 comp rx=0 tx=0).stop 2024-04-25T14:18:06.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.838+0000 7f4730200640 1 -- 172.21.15.86:0/3242720014 shutdown_connections 2024-04-25T14:18:06.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.838+0000 7f4730200640 1 --2- 172.21.15.86:0/3242720014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47280d16e0 0x7f47280c0e60 unknown :-1 s=CLOSED pgs=2488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:06.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.838+0000 7f4730200640 1 --2- 172.21.15.86:0/3242720014 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f470c06f400 0x7f470c071910 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:06.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.838+0000 7f4730200640 1 --2- 172.21.15.86:0/3242720014 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f47280cfca0 0x7f47280c0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:06.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.838+0000 7f4730200640 1 --2- 172.21.15.86:0/3242720014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47280730b0 0x7f4728099a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:06.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.838+0000 7f4730200640 1 -- 172.21.15.86:0/3242720014 >> 172.21.15.86:0/3242720014 conn(0x7f47280b4650 msgr2=0x7f472806e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:06.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.838+0000 7f4730200640 1 -- 172.21.15.86:0/3242720014 shutdown_connections 2024-04-25T14:18:06.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:06.838+0000 7f4730200640 1 -- 172.21.15.86:0/3242720014 wait complete. 2024-04-25T14:18:06.853 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-04-25T14:18:05.178991+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":957,"num_read_kb":2631,"num_write":271,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3305,"ondisk_log_size":3305,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":323700,"kb_used_data":82584,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754651020,"statfs":{"total":773094113280,"available":772762644480,"internally_reserved":0,"allocated":84566016,"data_stored":76578336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-33701767,"num_objects":-73,"num_object_clones":0,"num_object_copies":-146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-73,"num_whiteouts":0,"num_read":60,"num_read_kb":176,"num_write":-162,"num_write_kb":-32966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001369"},"pg_stats":[{"pgid":"1.0","version":"157'3305","reported_seq":3891,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T14:18:01.321074+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:18:01.321074+0000","last_peered":"2024-04-25T14:18:01.321074+0000","last_clean":"2024-04-25T14:18:01.321074+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:18:01.321074+0000","last_undegraded":"2024-04-25T14:18:01.321074+0000","last_fullsized":"2024-04-25T14:18:01.321074+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":3305,"log_dups_size":0,"ondisk_log_size":3305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":957,"num_read_kb":2631,"num_write":271,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":957,"num_read_kb":2631,"num_write":271,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3305,"ondisk_log_size":3305,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575016,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41832,"kb_used_data":14812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94330008,"statfs":{"total":96636764160,"available":96593928192,"internally_reserved":0,"allocated":15167488,"data_stored":14174372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575017,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37220,"kb_used_data":6040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334620,"statfs":{"total":96636764160,"available":96598650880,"internally_reserved":0,"allocated":6184960,"data_stored":5195464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575018,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37228,"kb_used_data":6048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334612,"statfs":{"total":96636764160,"available":96598642688,"internally_reserved":0,"allocated":6193152,"data_stored":5196080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575017,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46452,"kb_used_data":15272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94325388,"statfs":{"total":96636764160,"available":96589197312,"internally_reserved":0,"allocated":15638528,"data_stored":14633904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575017,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42476,"kb_used_data":11296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329364,"statfs":{"total":96636764160,"available":96593268736,"internally_reserved":0,"allocated":11567104,"data_stored":10555382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575016,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41316,"kb_used_data":10136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330524,"statfs":{"total":96636764160,"available":96594456576,"internally_reserved":0,"allocated":10379264,"data_stored":9389768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575018,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40440,"kb_used_data":9260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331400,"statfs":{"total":96636764160,"available":96595353600,"internally_reserved":0,"allocated":9482240,"data_stored":8479698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575017,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36736,"kb_used_data":9720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94335104,"statfs":{"total":96636764160,"available":96599146496,"internally_reserved":0,"allocated":9953280,"data_stored":8953668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:18:06.854 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:18:07.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.006+0000 7f1d60c00640 1 Processor -- start 2024-04-25T14:18:07.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.006+0000 7f1d60c00640 1 -- start start 2024-04-25T14:18:07.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.010+0000 7f1d60c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1d5c0c49c0 0x7f1d5c06e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:07.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.010+0000 7f1d60c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d5c06ef20 0x7f1d5c06f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:07.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.010+0000 7f1d60c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d5c06d0a0 con 0x7f1d5c0ce570 2024-04-25T14:18:07.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.010+0000 7f1d60c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1d5c06dcb0 con 0x7f1d5c0c49c0 2024-04-25T14:18:07.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.010+0000 7f1d60c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1d5c06d380 con 0x7f1d5c06ef20 2024-04-25T14:18:07.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.010+0000 7f1d5a800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d5c06ef20 0x7f1d5c06f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:07.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.010+0000 7f1d51400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1d5c0c49c0 0x7f1d5c06e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:07.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.010+0000 7f1d59e00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f1d5c0ce570 0x7f1d5c0c42b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:44742/0 (socket says 172.21.15.86:44742) 2024-04-25T14:18:07.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.010+0000 7f1d59e00640 1 -- 172.21.15.86:0/3159901119 learned_addr learned my addr 172.21.15.86:0/3159901119 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:18:07.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.010+0000 7f1d59400640 1 -- 172.21.15.86:0/3159901119 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3383585411 0 0) 0x7f1d5c06d0a0 con 0x7f1d5c0ce570 2024-04-25T14:18:07.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.010+0000 7f1d59400640 1 -- 172.21.15.86:0/3159901119 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d40003530 con 0x7f1d5c0ce570 2024-04-25T14:18:07.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.010+0000 7f1d51400640 1 -- 172.21.15.86:0/3159901119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d5c06ef20 msgr2=0x7f1d5c06f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:07.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.010+0000 7f1d51400640 1 --2- 172.21.15.86:0/3159901119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d5c06ef20 0x7f1d5c06f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.010+0000 7f1d51400640 1 -- 172.21.15.86:0/3159901119 >> v1:172.21.15.155:6789/0 conn(0x7f1d5c0ce570 legacy=0x7f1d5c0c42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:07.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.010+0000 7f1d51400640 1 -- 172.21.15.86:0/3159901119 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d5c06d660 con 0x7f1d5c0c49c0 2024-04-25T14:18:07.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.010+0000 7f1d5a800640 1 --2- 172.21.15.86:0/3159901119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d5c06ef20 0x7f1d5c06f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:18:07.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.014+0000 7f1d51400640 1 --2- 172.21.15.86:0/3159901119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d5c0c49c0 0x7f1d5c06e9e0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c0097c0 tx=0x7f1d4c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=921820c313bfa2c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:07.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.014+0000 7f1d59400640 1 -- 172.21.15.86:0/3159901119 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c0b2070 con 0x7f1d5c0c49c0 2024-04-25T14:18:07.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.014+0000 7f1d59400640 1 -- 172.21.15.86:0/3159901119 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d4c0ad640 con 0x7f1d5c0c49c0 2024-04-25T14:18:07.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.014+0000 7f1d59400640 1 -- 172.21.15.86:0/3159901119 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c0ada10 con 0x7f1d5c0c49c0 2024-04-25T14:18:07.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.014+0000 7f1d60c00640 1 -- 172.21.15.86:0/3159901119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d5c0c49c0 msgr2=0x7f1d5c06e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:07.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.014+0000 7f1d60c00640 1 --2- 172.21.15.86:0/3159901119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d5c0c49c0 0x7f1d5c06e9e0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c0097c0 tx=0x7f1d4c01c4a0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.014+0000 7f1d60c00640 1 -- 172.21.15.86:0/3159901119 shutdown_connections 2024-04-25T14:18:07.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.014+0000 7f1d60c00640 1 --2- 172.21.15.86:0/3159901119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d5c06ef20 0x7f1d5c06f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.014+0000 7f1d60c00640 1 --2- 172.21.15.86:0/3159901119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d5c0c49c0 0x7f1d5c06e9e0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.014+0000 7f1d60c00640 1 -- 172.21.15.86:0/3159901119 >> 172.21.15.86:0/3159901119 conn(0x7f1d5c0b4650 msgr2=0x7f1d5c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:07.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.014+0000 7f1d60c00640 1 -- 172.21.15.86:0/3159901119 shutdown_connections 2024-04-25T14:18:07.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.014+0000 7f1d60c00640 1 -- 172.21.15.86:0/3159901119 wait complete. 2024-04-25T14:18:07.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.014+0000 7f1d60c00640 1 Processor -- start 2024-04-25T14:18:07.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.018+0000 7f1d60c00640 1 -- start start 2024-04-25T14:18:07.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.018+0000 7f1d60c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d5c06ef20 0x7f1d5c163250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:07.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.018+0000 7f1d60c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d5c0ce570 0x7f1d5c15eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:07.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.018+0000 7f1d60c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d5c15f0b0 0x7f1d5c155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:07.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.018+0000 7f1d60c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c167b80 con 0x7f1d5c15f0b0 2024-04-25T14:18:07.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.018+0000 7f1d60c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c167a00 con 0x7f1d5c06ef20 2024-04-25T14:18:07.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.018+0000 7f1d60c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1d5c0ce980 con 0x7f1d5c0ce570 2024-04-25T14:18:07.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.018+0000 7f1d5a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d5c15f0b0 0x7f1d5c155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:07.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.018+0000 7f1d5a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d5c15f0b0 0x7f1d5c155d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35744/0 (socket says 172.21.15.86:35744) 2024-04-25T14:18:07.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.018+0000 7f1d5a800640 1 -- 172.21.15.86:0/1273047849 learned_addr learned my addr 172.21.15.86:0/1273047849 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:07.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.018+0000 7f1d51400640 1 --2- 172.21.15.86:0/1273047849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d5c0ce570 0x7f1d5c15eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:07.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.018+0000 7f1d59e00640 1 --2- 172.21.15.86:0/1273047849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d5c06ef20 0x7f1d5c163250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:07.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.018+0000 7f1d5a800640 1 -- 172.21.15.86:0/1273047849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d5c0ce570 msgr2=0x7f1d5c15eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:07.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.018+0000 7f1d5a800640 1 --2- 172.21.15.86:0/1273047849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d5c0ce570 0x7f1d5c15eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.018+0000 7f1d5a800640 1 -- 172.21.15.86:0/1273047849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d5c06ef20 msgr2=0x7f1d5c163250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:07.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.018+0000 7f1d5a800640 1 --2- 172.21.15.86:0/1273047849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d5c06ef20 0x7f1d5c163250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.018+0000 7f1d5a800640 1 -- 172.21.15.86:0/1273047849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d5c15b630 con 0x7f1d5c15f0b0 2024-04-25T14:18:07.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.018+0000 7f1d5a800640 1 --2- 172.21.15.86:0/1273047849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d5c15f0b0 0x7f1d5c155d90 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009e10 tx=0x7f1d5400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:07.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.018+0000 7f1d52a00640 1 -- 172.21.15.86:0/1273047849 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54018020 con 0x7f1d5c15f0b0 2024-04-25T14:18:07.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.018+0000 7f1d52a00640 1 -- 172.21.15.86:0/1273047849 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d54014480 con 0x7f1d5c15f0b0 2024-04-25T14:18:07.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.018+0000 7f1d52a00640 1 -- 172.21.15.86:0/1273047849 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54014850 con 0x7f1d5c15f0b0 2024-04-25T14:18:07.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.018+0000 7f1d60c00640 1 -- 172.21.15.86:0/1273047849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d5c15b910 con 0x7f1d5c15f0b0 2024-04-25T14:18:07.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.018+0000 7f1d60c00640 1 -- 172.21.15.86:0/1273047849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d5c164e10 con 0x7f1d5c15f0b0 2024-04-25T14:18:07.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.022+0000 7f1d60c00640 1 -- 172.21.15.86:0/1273047849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d5c15b3a0 con 0x7f1d5c15f0b0 2024-04-25T14:18:07.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.022+0000 7f1d52a00640 1 -- 172.21.15.86:0/1273047849 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7f1d5401b020 con 0x7f1d5c15f0b0 2024-04-25T14:18:07.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.022+0000 7f1d52a00640 1 --2- 172.21.15.86:0/1273047849 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f1d3806f430 0x7f1d38071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:07.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.022+0000 7f1d52a00640 1 -- 172.21.15.86:0/1273047849 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 6023+0+0 (secure 0 0 0) 0x7f1d54090090 con 0x7f1d5c15f0b0 2024-04-25T14:18:07.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.022+0000 7f1d59e00640 1 --2- 172.21.15.86:0/1273047849 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f1d3806f430 0x7f1d38071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:07.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.022+0000 7f1d59e00640 1 --2- 172.21.15.86:0/1273047849 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f1d3806f430 0x7f1d38071940 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1d44007fd0 tx=0x7f1d44009040 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.026+0000 7f1d52a00640 1 -- 172.21.15.86:0/1273047849 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1d54022030 con 0x7f1d5c15f0b0 2024-04-25T14:18:07.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.182+0000 7f1d60c00640 1 -- 172.21.15.86:0/1273047849 --> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1d5c0b3f70 con 0x7f1d3806f430 2024-04-25T14:18:07.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.182+0000 7f1d52a00640 1 -- 172.21.15.86:0/1273047849 <== mgr.11343 v2:172.21.15.86:6832/3174179029 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13520 (secure 0 0 0) 0x7f1d44001480 con 0x7f1d3806f430 2024-04-25T14:18:07.178 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:18:07.178 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:18:07.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.186+0000 7f1d60c00640 1 -- 172.21.15.86:0/1273047849 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f1d3806f430 msgr2=0x7f1d38071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:07.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.186+0000 7f1d60c00640 1 --2- 172.21.15.86:0/1273047849 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f1d3806f430 0x7f1d38071940 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1d44007fd0 tx=0x7f1d44009040 comp rx=0 tx=0).stop 2024-04-25T14:18:07.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.186+0000 7f1d60c00640 1 -- 172.21.15.86:0/1273047849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d5c15f0b0 msgr2=0x7f1d5c155d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:07.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.186+0000 7f1d60c00640 1 --2- 172.21.15.86:0/1273047849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d5c15f0b0 0x7f1d5c155d90 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009e10 tx=0x7f1d5400c490 comp rx=0 tx=0).stop 2024-04-25T14:18:07.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.186+0000 7f1d60c00640 1 -- 172.21.15.86:0/1273047849 shutdown_connections 2024-04-25T14:18:07.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.186+0000 7f1d60c00640 1 --2- 172.21.15.86:0/1273047849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d5c15f0b0 0x7f1d5c155d90 unknown :-1 s=CLOSED pgs=2489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.186+0000 7f1d60c00640 1 --2- 172.21.15.86:0/1273047849 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f1d3806f430 0x7f1d38071940 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.186+0000 7f1d60c00640 1 --2- 172.21.15.86:0/1273047849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d5c0ce570 0x7f1d5c15eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.186+0000 7f1d60c00640 1 --2- 172.21.15.86:0/1273047849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d5c06ef20 0x7f1d5c163250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.186+0000 7f1d60c00640 1 -- 172.21.15.86:0/1273047849 >> 172.21.15.86:0/1273047849 conn(0x7f1d5c0b4650 msgr2=0x7f1d5c0cf830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:07.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.186+0000 7f1d60c00640 1 -- 172.21.15.86:0/1273047849 shutdown_connections 2024-04-25T14:18:07.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.186+0000 7f1d60c00640 1 -- 172.21.15.86:0/1273047849 wait complete. 2024-04-25T14:18:07.199 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-04-25T14:18:07.179406+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":957,"num_read_kb":2631,"num_write":271,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3305,"ondisk_log_size":3305,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":323700,"kb_used_data":82584,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754651020,"statfs":{"total":773094113280,"available":772762644480,"internally_reserved":0,"allocated":84566016,"data_stored":76578336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-33701767,"num_objects":-73,"num_object_clones":0,"num_object_copies":-146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-73,"num_whiteouts":0,"num_read":60,"num_read_kb":176,"num_write":-162,"num_write_kb":-32966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001410"},"pg_stats":[{"pgid":"1.0","version":"157'3305","reported_seq":3891,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T14:18:01.321074+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:18:01.321074+0000","last_peered":"2024-04-25T14:18:01.321074+0000","last_clean":"2024-04-25T14:18:01.321074+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:18:01.321074+0000","last_undegraded":"2024-04-25T14:18:01.321074+0000","last_fullsized":"2024-04-25T14:18:01.321074+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":3305,"log_dups_size":0,"ondisk_log_size":3305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":957,"num_read_kb":2631,"num_write":271,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":957,"num_read_kb":2631,"num_write":271,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3305,"ondisk_log_size":3305,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575016,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41832,"kb_used_data":14812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94330008,"statfs":{"total":96636764160,"available":96593928192,"internally_reserved":0,"allocated":15167488,"data_stored":14174372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575017,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37220,"kb_used_data":6040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334620,"statfs":{"total":96636764160,"available":96598650880,"internally_reserved":0,"allocated":6184960,"data_stored":5195464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575018,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37228,"kb_used_data":6048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334612,"statfs":{"total":96636764160,"available":96598642688,"internally_reserved":0,"allocated":6193152,"data_stored":5196080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575017,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46452,"kb_used_data":15272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94325388,"statfs":{"total":96636764160,"available":96589197312,"internally_reserved":0,"allocated":15638528,"data_stored":14633904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575017,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42476,"kb_used_data":11296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329364,"statfs":{"total":96636764160,"available":96593268736,"internally_reserved":0,"allocated":11567104,"data_stored":10555382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575016,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41316,"kb_used_data":10136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330524,"statfs":{"total":96636764160,"available":96594456576,"internally_reserved":0,"allocated":10379264,"data_stored":9389768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575018,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40440,"kb_used_data":9260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331400,"statfs":{"total":96636764160,"available":96595353600,"internally_reserved":0,"allocated":9482240,"data_stored":8479698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575017,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36736,"kb_used_data":9720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94335104,"statfs":{"total":96636764160,"available":96599146496,"internally_reserved":0,"allocated":9953280,"data_stored":8953668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:18:07.200 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T14:18:07.200 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:18:07.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.354+0000 7efebbe00640 1 Processor -- start 2024-04-25T14:18:07.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.354+0000 7efebbe00640 1 -- start start 2024-04-25T14:18:07.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.354+0000 7efebbe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7efeb40b3950 0x7efeb415cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:07.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.354+0000 7efebbe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efeb40af8b0 0x7efeb40afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:07.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.354+0000 7efebbe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efeb40b3d60 con 0x7efeb41607a0 2024-04-25T14:18:07.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.354+0000 7efebbe00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7efeb40af140 con 0x7efeb40b3950 2024-04-25T14:18:07.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.354+0000 7efebbe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efeb40af420 con 0x7efeb40af8b0 2024-04-25T14:18:07.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.354+0000 7efeb8a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7efeb41607a0 0x7efeb4160bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60598/0 (socket says 172.21.15.86:60598) 2024-04-25T14:18:07.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.354+0000 7efeb8a00640 1 -- 172.21.15.86:0/4109930874 learned_addr learned my addr 172.21.15.86:0/4109930874 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:18:07.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.354+0000 7efeb9e00640 1 --2- 172.21.15.86:0/4109930874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efeb40af8b0 0x7efeb40afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:07.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.354+0000 7efeb9400640 1 --2- 172.21.15.86:0/4109930874 >> v2:172.21.15.155:3300/0 conn(0x7efeb40b3950 0x7efeb415cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:07.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.358+0000 7efeb3e00640 1 -- 172.21.15.86:0/4109930874 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1930325917 0 0) 0x7efeb40b3d60 con 0x7efeb41607a0 2024-04-25T14:18:07.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.358+0000 7efeb3e00640 1 -- 172.21.15.86:0/4109930874 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe9c003530 con 0x7efeb41607a0 2024-04-25T14:18:07.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.358+0000 7efeb9e00640 1 -- 172.21.15.86:0/4109930874 >> v2:172.21.15.155:3300/0 conn(0x7efeb40b3950 msgr2=0x7efeb415cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:07.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.358+0000 7efeb9e00640 1 --2- 172.21.15.86:0/4109930874 >> v2:172.21.15.155:3300/0 conn(0x7efeb40b3950 0x7efeb415cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.358+0000 7efeb9e00640 1 -- 172.21.15.86:0/4109930874 >> v1:172.21.15.86:6789/0 conn(0x7efeb41607a0 legacy=0x7efeb4160bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:07.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.358+0000 7efeb9e00640 1 -- 172.21.15.86:0/4109930874 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb40b1d50 con 0x7efeb40af8b0 2024-04-25T14:18:07.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.358+0000 7efeb9400640 1 --2- 172.21.15.86:0/4109930874 >> v2:172.21.15.155:3300/0 conn(0x7efeb40b3950 0x7efeb415cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:18:07.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.358+0000 7efeb9e00640 1 --2- 172.21.15.86:0/4109930874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efeb40af8b0 0x7efeb40afcc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efeac0097c0 tx=0x7efeac01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fd7f600f7516aab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:07.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.358+0000 7efeb3e00640 1 -- 172.21.15.86:0/4109930874 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac0ab610 con 0x7efeb40af8b0 2024-04-25T14:18:07.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.358+0000 7efeb3e00640 1 -- 172.21.15.86:0/4109930874 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeac0b0070 con 0x7efeb40af8b0 2024-04-25T14:18:07.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.358+0000 7efeb3e00640 1 -- 172.21.15.86:0/4109930874 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac0ab9d0 con 0x7efeb40af8b0 2024-04-25T14:18:07.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.362+0000 7efebbe00640 1 -- 172.21.15.86:0/4109930874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efeb40af8b0 msgr2=0x7efeb40afcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:07.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.362+0000 7efebbe00640 1 --2- 172.21.15.86:0/4109930874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efeb40af8b0 0x7efeb40afcc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efeac0097c0 tx=0x7efeac01c410 comp rx=0 tx=0).stop 2024-04-25T14:18:07.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.362+0000 7efebbe00640 1 -- 172.21.15.86:0/4109930874 shutdown_connections 2024-04-25T14:18:07.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.362+0000 7efebbe00640 1 --2- 172.21.15.86:0/4109930874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efeb40af8b0 0x7efeb40afcc0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.362+0000 7efebbe00640 1 --2- 172.21.15.86:0/4109930874 >> v2:172.21.15.155:3300/0 conn(0x7efeb40b3950 0x7efeb415cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.362+0000 7efebbe00640 1 -- 172.21.15.86:0/4109930874 >> 172.21.15.86:0/4109930874 conn(0x7efeb409ba30 msgr2=0x7efeb40b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:07.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.362+0000 7efebbe00640 1 -- 172.21.15.86:0/4109930874 shutdown_connections 2024-04-25T14:18:07.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.362+0000 7efebbe00640 1 -- 172.21.15.86:0/4109930874 wait complete. 2024-04-25T14:18:07.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.362+0000 7efebbe00640 1 Processor -- start 2024-04-25T14:18:07.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.362+0000 7efebbe00640 1 -- start start 2024-04-25T14:18:07.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.366+0000 7efebbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efeb40af8b0 0x7efeb414ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:07.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.366+0000 7efebbe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efeb40b3950 0x7efeb40cf250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:07.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.366+0000 7efebbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efeb41607a0 0x7efeb414ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:07.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.366+0000 7efebbe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efeb406c670 con 0x7efeb41607a0 2024-04-25T14:18:07.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.366+0000 7efebbe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efeb41614d0 con 0x7efeb40af8b0 2024-04-25T14:18:07.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.366+0000 7efebbe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efeb406c7f0 con 0x7efeb40b3950 2024-04-25T14:18:07.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.366+0000 7efeb9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efeb41607a0 0x7efeb414ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:07.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.366+0000 7efeb9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efeb41607a0 0x7efeb414ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35746/0 (socket says 172.21.15.86:35746) 2024-04-25T14:18:07.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.366+0000 7efeb9e00640 1 -- 172.21.15.86:0/4110293760 learned_addr learned my addr 172.21.15.86:0/4110293760 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:07.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.366+0000 7efeb8a00640 1 --2- 172.21.15.86:0/4110293760 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efeb40b3950 0x7efeb40cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:07.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.366+0000 7efeb9400640 1 --2- 172.21.15.86:0/4110293760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efeb40af8b0 0x7efeb414ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:07.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.366+0000 7efeb9e00640 1 -- 172.21.15.86:0/4110293760 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efeb40b3950 msgr2=0x7efeb40cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:07.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.366+0000 7efeb9e00640 1 --2- 172.21.15.86:0/4110293760 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efeb40b3950 0x7efeb40cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.366+0000 7efeb9e00640 1 -- 172.21.15.86:0/4110293760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efeb40af8b0 msgr2=0x7efeb414ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:07.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.366+0000 7efeb9e00640 1 --2- 172.21.15.86:0/4110293760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efeb40af8b0 0x7efeb414ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.366+0000 7efeb9e00640 1 -- 172.21.15.86:0/4110293760 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb4165dd0 con 0x7efeb41607a0 2024-04-25T14:18:07.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.366+0000 7efeb9e00640 1 --2- 172.21.15.86:0/4110293760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efeb41607a0 0x7efeb414ffb0 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7efeac01a040 tx=0x7efeac01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:07.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.366+0000 7efeb1600640 1 -- 172.21.15.86:0/4110293760 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac0bf410 con 0x7efeb41607a0 2024-04-25T14:18:07.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.366+0000 7efeb1600640 1 -- 172.21.15.86:0/4110293760 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeac0bf5b0 con 0x7efeb41607a0 2024-04-25T14:18:07.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.366+0000 7efebbe00640 1 -- 172.21.15.86:0/4110293760 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb41660b0 con 0x7efeb41607a0 2024-04-25T14:18:07.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.366+0000 7efeb1600640 1 -- 172.21.15.86:0/4110293760 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac0bf980 con 0x7efeb41607a0 2024-04-25T14:18:07.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.366+0000 7efebbe00640 1 -- 172.21.15.86:0/4110293760 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb4166670 con 0x7efeb41607a0 2024-04-25T14:18:07.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.366+0000 7efebbe00640 1 -- 172.21.15.86:0/4110293760 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeb40a04c0 con 0x7efeb41607a0 2024-04-25T14:18:07.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.366+0000 7efeb1600640 1 -- 172.21.15.86:0/4110293760 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7efeac0bd020 con 0x7efeb41607a0 2024-04-25T14:18:07.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.370+0000 7efeb1600640 1 --2- 172.21.15.86:0/4110293760 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7efe9406f430 0x7efe94071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:07.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.370+0000 7efeb1600640 1 -- 172.21.15.86:0/4110293760 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 6023+0+0 (secure 0 0 0) 0x7efeac1296f0 con 0x7efeb41607a0 2024-04-25T14:18:07.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.370+0000 7efeb9400640 1 --2- 172.21.15.86:0/4110293760 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7efe9406f430 0x7efe94071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:07.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.370+0000 7efeb9400640 1 --2- 172.21.15.86:0/4110293760 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7efe9406f430 0x7efe94071940 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efea8009bd0 tx=0x7efea800b040 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:07.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.374+0000 7efeb1600640 1 -- 172.21.15.86:0/4110293760 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efeac0c4060 con 0x7efeb41607a0 2024-04-25T14:18:07.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.526+0000 7efebbe00640 1 -- 172.21.15.86:0/4110293760 --> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efeb4076bf0 con 0x7efe9406f430 2024-04-25T14:18:07.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.530+0000 7efeb1600640 1 -- 172.21.15.86:0/4110293760 <== mgr.11343 v2:172.21.15.86:6832/3174179029 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13520 (secure 0 0 0) 0x7efeb4076bf0 con 0x7efe9406f430 2024-04-25T14:18:07.523 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:18:07.524 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:18:07.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.530+0000 7efebbe00640 1 -- 172.21.15.86:0/4110293760 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7efe9406f430 msgr2=0x7efe94071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:07.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.530+0000 7efebbe00640 1 --2- 172.21.15.86:0/4110293760 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7efe9406f430 0x7efe94071940 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efea8009bd0 tx=0x7efea800b040 comp rx=0 tx=0).stop 2024-04-25T14:18:07.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.530+0000 7efebbe00640 1 -- 172.21.15.86:0/4110293760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efeb41607a0 msgr2=0x7efeb414ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:07.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.530+0000 7efebbe00640 1 --2- 172.21.15.86:0/4110293760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efeb41607a0 0x7efeb414ffb0 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7efeac01a040 tx=0x7efeac01ac50 comp rx=0 tx=0).stop 2024-04-25T14:18:07.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.530+0000 7efebbe00640 1 -- 172.21.15.86:0/4110293760 shutdown_connections 2024-04-25T14:18:07.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.530+0000 7efebbe00640 1 --2- 172.21.15.86:0/4110293760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efeb41607a0 0x7efeb414ffb0 unknown :-1 s=CLOSED pgs=2491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.530+0000 7efebbe00640 1 --2- 172.21.15.86:0/4110293760 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7efe9406f430 0x7efe94071940 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.530+0000 7efebbe00640 1 --2- 172.21.15.86:0/4110293760 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efeb40b3950 0x7efeb40cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.530+0000 7efebbe00640 1 --2- 172.21.15.86:0/4110293760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efeb40af8b0 0x7efeb414ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.530+0000 7efebbe00640 1 -- 172.21.15.86:0/4110293760 >> 172.21.15.86:0/4110293760 conn(0x7efeb409ba30 msgr2=0x7efeb406ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:07.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.530+0000 7efebbe00640 1 -- 172.21.15.86:0/4110293760 shutdown_connections 2024-04-25T14:18:07.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.530+0000 7efebbe00640 1 -- 172.21.15.86:0/4110293760 wait complete. 2024-04-25T14:18:07.544 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-04-25T14:18:07.179406+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":957,"num_read_kb":2631,"num_write":271,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3305,"ondisk_log_size":3305,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":323700,"kb_used_data":82584,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754651020,"statfs":{"total":773094113280,"available":772762644480,"internally_reserved":0,"allocated":84566016,"data_stored":76578336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-33701767,"num_objects":-73,"num_object_clones":0,"num_object_copies":-146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-73,"num_whiteouts":0,"num_read":60,"num_read_kb":176,"num_write":-162,"num_write_kb":-32966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001410"},"pg_stats":[{"pgid":"1.0","version":"157'3305","reported_seq":3891,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T14:18:01.321074+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:18:01.321074+0000","last_peered":"2024-04-25T14:18:01.321074+0000","last_clean":"2024-04-25T14:18:01.321074+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:18:01.321074+0000","last_undegraded":"2024-04-25T14:18:01.321074+0000","last_fullsized":"2024-04-25T14:18:01.321074+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":3305,"log_dups_size":0,"ondisk_log_size":3305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":957,"num_read_kb":2631,"num_write":271,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":957,"num_read_kb":2631,"num_write":271,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3305,"ondisk_log_size":3305,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575016,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41832,"kb_used_data":14812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94330008,"statfs":{"total":96636764160,"available":96593928192,"internally_reserved":0,"allocated":15167488,"data_stored":14174372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575017,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37220,"kb_used_data":6040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334620,"statfs":{"total":96636764160,"available":96598650880,"internally_reserved":0,"allocated":6184960,"data_stored":5195464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575018,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37228,"kb_used_data":6048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334612,"statfs":{"total":96636764160,"available":96598642688,"internally_reserved":0,"allocated":6193152,"data_stored":5196080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575017,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46452,"kb_used_data":15272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94325388,"statfs":{"total":96636764160,"available":96589197312,"internally_reserved":0,"allocated":15638528,"data_stored":14633904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575017,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42476,"kb_used_data":11296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329364,"statfs":{"total":96636764160,"available":96593268736,"internally_reserved":0,"allocated":11567104,"data_stored":10555382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575016,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41316,"kb_used_data":10136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330524,"statfs":{"total":96636764160,"available":96594456576,"internally_reserved":0,"allocated":10379264,"data_stored":9389768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575018,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40440,"kb_used_data":9260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331400,"statfs":{"total":96636764160,"available":96595353600,"internally_reserved":0,"allocated":9482240,"data_stored":8479698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575017,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36736,"kb_used_data":9720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94335104,"statfs":{"total":96636764160,"available":96599146496,"internally_reserved":0,"allocated":9953280,"data_stored":8953668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:18:07.545 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:18:07.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.698+0000 7f35e9400640 1 Processor -- start 2024-04-25T14:18:07.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.698+0000 7f35e9400640 1 -- start start 2024-04-25T14:18:07.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.698+0000 7f35e9400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f35e40d1df0 0x7f35e40cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:07.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.698+0000 7f35e9400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35e40cdcc0 0x7f35e40ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:07.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.698+0000 7f35e9400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35e415e5c0 con 0x7f35e4070330 2024-04-25T14:18:07.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.698+0000 7f35e9400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f35e406fd40 con 0x7f35e40d1df0 2024-04-25T14:18:07.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.698+0000 7f35e9400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f35e415e8a0 con 0x7f35e40cdcc0 2024-04-25T14:18:07.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.698+0000 7f35e1e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f35e40d1df0 0x7f35e40cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:07.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.698+0000 7f35e1e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f35e40d1df0 0x7f35e40cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35754/0 (socket says 172.21.15.86:35754) 2024-04-25T14:18:07.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.698+0000 7f35e1e00640 1 -- 172.21.15.86:0/3272804528 learned_addr learned my addr 172.21.15.86:0/3272804528 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:07.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.698+0000 7f35e3200640 1 --2- 172.21.15.86:0/3272804528 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35e40cdcc0 0x7f35e40ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:07.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.698+0000 7f35e1e00640 1 -- 172.21.15.86:0/3272804528 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35e40cdcc0 msgr2=0x7f35e40ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:07.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.698+0000 7f35e1e00640 1 --2- 172.21.15.86:0/3272804528 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35e40cdcc0 0x7f35e40ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.698+0000 7f35e1e00640 1 -- 172.21.15.86:0/3272804528 >> v1:172.21.15.155:6789/0 conn(0x7f35e4070330 legacy=0x7f35e40d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:07.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.698+0000 7f35e1e00640 1 -- 172.21.15.86:0/3272804528 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e415eb80 con 0x7f35e40d1df0 2024-04-25T14:18:07.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.702+0000 7f35e1e00640 1 --2- 172.21.15.86:0/3272804528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35e40d1df0 0x7f35e40cd780 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009870 tx=0x7f35d001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb87da3dd9f6312f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:07.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.702+0000 7f35e1400640 1 -- 172.21.15.86:0/3272804528 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d00ab600 con 0x7f35e40d1df0 2024-04-25T14:18:07.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.702+0000 7f35e1400640 1 -- 172.21.15.86:0/3272804528 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35d00b0070 con 0x7f35e40d1df0 2024-04-25T14:18:07.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.702+0000 7f35e1400640 1 -- 172.21.15.86:0/3272804528 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d00ab9c0 con 0x7f35e40d1df0 2024-04-25T14:18:07.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.702+0000 7f35e9400640 1 -- 172.21.15.86:0/3272804528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35e40d1df0 msgr2=0x7f35e40cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:07.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.702+0000 7f35e9400640 1 --2- 172.21.15.86:0/3272804528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35e40d1df0 0x7f35e40cd780 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009870 tx=0x7f35d001c460 comp rx=0 tx=0).stop 2024-04-25T14:18:07.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.702+0000 7f35e9400640 1 -- 172.21.15.86:0/3272804528 shutdown_connections 2024-04-25T14:18:07.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.702+0000 7f35e9400640 1 --2- 172.21.15.86:0/3272804528 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35e40cdcc0 0x7f35e40ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.702+0000 7f35e9400640 1 --2- 172.21.15.86:0/3272804528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35e40d1df0 0x7f35e40cd780 unknown :-1 s=CLOSED pgs=2492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.702+0000 7f35e9400640 1 -- 172.21.15.86:0/3272804528 >> 172.21.15.86:0/3272804528 conn(0x7f35e40b4650 msgr2=0x7f35e40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:07.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.702+0000 7f35e9400640 1 -- 172.21.15.86:0/3272804528 shutdown_connections 2024-04-25T14:18:07.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.702+0000 7f35e9400640 1 -- 172.21.15.86:0/3272804528 wait complete. 2024-04-25T14:18:07.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.706+0000 7f35e9400640 1 Processor -- start 2024-04-25T14:18:07.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.706+0000 7f35e9400640 1 -- start start 2024-04-25T14:18:07.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.706+0000 7f35e9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35e4070330 0x7f35e40c40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:07.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.706+0000 7f35e9400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35e40cdcc0 0x7f35e40c2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:07.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.706+0000 7f35e9400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35e4159dc0 0x7f35e40af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:07.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.706+0000 7f35e9400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f35e415f4b0 con 0x7f35e4070330 2024-04-25T14:18:07.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.706+0000 7f35e9400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f35e415f330 con 0x7f35e40cdcc0 2024-04-25T14:18:07.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.706+0000 7f35e9400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f35e4070740 con 0x7f35e4159dc0 2024-04-25T14:18:07.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.706+0000 7f35e2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35e4070330 0x7f35e40c40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:07.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.706+0000 7f35e2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35e4070330 0x7f35e40c40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35760/0 (socket says 172.21.15.86:35760) 2024-04-25T14:18:07.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.706+0000 7f35e2800640 1 -- 172.21.15.86:0/803796089 learned_addr learned my addr 172.21.15.86:0/803796089 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:07.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.706+0000 7f35e1e00640 1 --2- 172.21.15.86:0/803796089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35e40cdcc0 0x7f35e40c2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:07.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.706+0000 7f35e3200640 1 --2- 172.21.15.86:0/803796089 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35e4159dc0 0x7f35e40af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:07.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.706+0000 7f35e2800640 1 -- 172.21.15.86:0/803796089 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35e4159dc0 msgr2=0x7f35e40af1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:07.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.706+0000 7f35e2800640 1 --2- 172.21.15.86:0/803796089 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35e4159dc0 0x7f35e40af1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.706+0000 7f35e2800640 1 -- 172.21.15.86:0/803796089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35e40cdcc0 msgr2=0x7f35e40c2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:07.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.706+0000 7f35e2800640 1 --2- 172.21.15.86:0/803796089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35e40cdcc0 0x7f35e40c2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.706+0000 7f35e2800640 1 -- 172.21.15.86:0/803796089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e40c2aa0 con 0x7f35e4070330 2024-04-25T14:18:07.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.706+0000 7f35e3200640 1 --2- 172.21.15.86:0/803796089 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35e4159dc0 0x7f35e40af1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:18:07.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.706+0000 7f35e2800640 1 --2- 172.21.15.86:0/803796089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35e4070330 0x7f35e40c40a0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f35d400b410 tx=0x7f35d400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:07.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.706+0000 7f35daa00640 1 -- 172.21.15.86:0/803796089 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4015030 con 0x7f35e4070330 2024-04-25T14:18:07.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.706+0000 7f35daa00640 1 -- 172.21.15.86:0/803796089 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35d4004020 con 0x7f35e4070330 2024-04-25T14:18:07.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.706+0000 7f35daa00640 1 -- 172.21.15.86:0/803796089 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4004410 con 0x7f35e4070330 2024-04-25T14:18:07.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.706+0000 7f35e9400640 1 -- 172.21.15.86:0/803796089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e40c2d80 con 0x7f35e4070330 2024-04-25T14:18:07.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.710+0000 7f35e9400640 1 -- 172.21.15.86:0/803796089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e415a500 con 0x7f35e4070330 2024-04-25T14:18:07.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.710+0000 7f35e9400640 1 -- 172.21.15.86:0/803796089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e4069d00 con 0x7f35e4070330 2024-04-25T14:18:07.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.710+0000 7f35daa00640 1 -- 172.21.15.86:0/803796089 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7f35d4018070 con 0x7f35e4070330 2024-04-25T14:18:07.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.710+0000 7f35daa00640 1 --2- 172.21.15.86:0/803796089 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f35c406f430 0x7f35c4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:07.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.710+0000 7f35daa00640 1 -- 172.21.15.86:0/803796089 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 6023+0+0 (secure 0 0 0) 0x7f35d408f650 con 0x7f35e4070330 2024-04-25T14:18:07.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.710+0000 7f35e1e00640 1 --2- 172.21.15.86:0/803796089 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f35c406f430 0x7f35c4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:07.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.714+0000 7f35e1e00640 1 --2- 172.21.15.86:0/803796089 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f35c406f430 0x7f35c4071940 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f35d00af000 tx=0x7f35d00a7000 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:07.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.714+0000 7f35daa00640 1 -- 172.21.15.86:0/803796089 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f35d405c4e0 con 0x7f35e4070330 2024-04-25T14:18:07.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.870+0000 7f35e9400640 1 -- 172.21.15.86:0/803796089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f35e4076bf0 con 0x7f35e4070330 2024-04-25T14:18:07.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.870+0000 7f35daa00640 1 -- 172.21.15.86:0/803796089 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) ==== 74+0+14312 (secure 0 0 0) 0x7f35d405bde0 con 0x7f35e4070330 2024-04-25T14:18:07.865 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:18:07.866 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":160,"fsid":"8c8ca1ab-5483-4ce5-a20b-ed76e0e847bb","created":"2024-04-25T14:03:25.691680+0000","modified":"2024-04-25T14:18:04.324230+0000","last_up_change":"2024-04-25T14:03:38.147360+0000","last_in_change":"2024-04-25T14:03:28.341654+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:03:38.737000+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf938de9-4676-4229-a12f-aae77996ae9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6824","nonce":2230428952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6827","nonce":2230428952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6831","nonce":2230428952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2230428952},{"type":"v1","addr":"172.21.15.86:6829","nonce":2230428952}]},"public_addr":"172.21.15.86:6824/2230428952","cluster_addr":"172.21.15.86:6827/2230428952","heartbeat_back_addr":"172.21.15.86:6831/2230428952","heartbeat_front_addr":"172.21.15.86:6829/2230428952","state":["exists","up"]},{"osd":1,"uuid":"a3cc1c0b-9570-4687-826a-9a6ffadbeb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6809","nonce":1043507731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6811","nonce":1043507731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6815","nonce":1043507731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":1043507731},{"type":"v1","addr":"172.21.15.86:6813","nonce":1043507731}]},"public_addr":"172.21.15.86:6809/1043507731","cluster_addr":"172.21.15.86:6811/1043507731","heartbeat_back_addr":"172.21.15.86:6815/1043507731","heartbeat_front_addr":"172.21.15.86:6813/1043507731","state":["exists","up"]},{"osd":2,"uuid":"2f65f8e2-ab65-49ce-ae5c-080ca193d8b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6817","nonce":2851379769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6819","nonce":2851379769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6825","nonce":2851379769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2851379769},{"type":"v1","addr":"172.21.15.86:6821","nonce":2851379769}]},"public_addr":"172.21.15.86:6817/2851379769","cluster_addr":"172.21.15.86:6819/2851379769","heartbeat_back_addr":"172.21.15.86:6825/2851379769","heartbeat_front_addr":"172.21.15.86:6821/2851379769","state":["exists","up"]},{"osd":3,"uuid":"23909d38-59a9-41d9-af18-aec4a8cbfa2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6801","nonce":2257199129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6803","nonce":2257199129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6807","nonce":2257199129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2257199129},{"type":"v1","addr":"172.21.15.86:6805","nonce":2257199129}]},"public_addr":"172.21.15.86:6801/2257199129","cluster_addr":"172.21.15.86:6803/2257199129","heartbeat_back_addr":"172.21.15.86:6807/2257199129","heartbeat_front_addr":"172.21.15.86:6805/2257199129","state":["exists","up"]},{"osd":4,"uuid":"57195638-5b04-4bc6-b52b-72d9f8808cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6802","nonce":1655306076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6806","nonce":1655306076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6813","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6815","nonce":1655306076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1655306076},{"type":"v1","addr":"172.21.15.155:6811","nonce":1655306076}]},"public_addr":"172.21.15.155:6802/1655306076","cluster_addr":"172.21.15.155:6806/1655306076","heartbeat_back_addr":"172.21.15.155:6815/1655306076","heartbeat_front_addr":"172.21.15.155:6811/1655306076","state":["exists","up"]},{"osd":5,"uuid":"f888702c-87b9-481e-9dd8-c3a97adea592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6817","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6818","nonce":288803543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6822","nonce":288803543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6830","nonce":288803543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":288803543},{"type":"v1","addr":"172.21.15.155:6826","nonce":288803543}]},"public_addr":"172.21.15.155:6818/288803543","cluster_addr":"172.21.15.155:6822/288803543","heartbeat_back_addr":"172.21.15.155:6830/288803543","heartbeat_front_addr":"172.21.15.155:6826/288803543","state":["exists","up"]},{"osd":6,"uuid":"8f3f9e20-b3b8-4ac3-99cd-6b26e34cbc3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6801","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6803","nonce":2597410484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6805","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6807","nonce":2597410484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6814","nonce":2597410484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6809","nonce":2597410484},{"type":"v1","addr":"172.21.15.155:6810","nonce":2597410484}]},"public_addr":"172.21.15.155:6803/2597410484","cluster_addr":"172.21.15.155:6807/2597410484","heartbeat_back_addr":"172.21.15.155:6814/2597410484","heartbeat_front_addr":"172.21.15.155:6810/2597410484","state":["exists","up"]},{"osd":7,"uuid":"4a769485-72f3-4e7d-a04a-f8f1ae51d98f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6819","nonce":2704073457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6821","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6823","nonce":2704073457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6829","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6831","nonce":2704073457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6825","nonce":2704073457},{"type":"v1","addr":"172.21.15.155:6827","nonce":2704073457}]},"public_addr":"172.21.15.155:6819/2704073457","cluster_addr":"172.21.15.155:6823/2704073457","heartbeat_back_addr":"172.21.15.155:6831/2704073457","heartbeat_front_addr":"172.21.15.155:6827/2704073457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.148581+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.066362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.087259+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:34.094526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.967620+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.927709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.922912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:03:33.975995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.155:6835/4231378267":"2024-04-26T14:18:01.314632+0000","172.21.15.86:6837/3141015835":"2024-04-26T14:18:01.314652+0000","172.21.15.86:0/1552836841":"2024-04-26T14:17:57.159798+0000","172.21.15.86:6833/2318291413":"2024-04-26T14:17:57.159798+0000","172.21.15.86:0/3986518615":"2024-04-26T14:17:57.159798+0000","172.21.15.86:0/3859223883":"2024-04-26T14:17:57.159798+0000","172.21.15.86:0/1995365685":"2024-04-26T14:17:57.159798+0000","172.21.15.86:0/2043624430":"2024-04-26T14:13:51.718341+0000","172.21.15.86:0/955458502":"2024-04-26T14:13:51.718341+0000","172.21.15.86:0/1302282677":"2024-04-26T14:13:51.718341+0000","172.21.15.86:6833/505730993":"2024-04-26T14:13:51.718341+0000","172.21.15.86:6832/505730993":"2024-04-26T14:13:51.718341+0000","172.21.15.86:0/3455088811":"2024-04-26T14:13:51.718341+0000","172.21.15.155:6834/4231378267":"2024-04-26T14:18:01.314632+0000","172.21.15.86:6836/3141015835":"2024-04-26T14:18:01.314652+0000","172.21.15.86:0/3537592233":"2024-04-26T14:13:51.718341+0000","172.21.15.86:6832/2318291413":"2024-04-26T14:17:57.159798+0000","172.21.15.86:0/2705096450":"2024-04-26T14:13:51.718341+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:18:07.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.874+0000 7f35e9400640 1 -- 172.21.15.86:0/803796089 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f35c406f430 msgr2=0x7f35c4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:07.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.874+0000 7f35e9400640 1 --2- 172.21.15.86:0/803796089 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f35c406f430 0x7f35c4071940 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f35d00af000 tx=0x7f35d00a7000 comp rx=0 tx=0).stop 2024-04-25T14:18:07.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.874+0000 7f35e9400640 1 -- 172.21.15.86:0/803796089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35e4070330 msgr2=0x7f35e40c40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:07.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.874+0000 7f35e9400640 1 --2- 172.21.15.86:0/803796089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35e4070330 0x7f35e40c40a0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f35d400b410 tx=0x7f35d400b8d0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.874+0000 7f35e9400640 1 -- 172.21.15.86:0/803796089 shutdown_connections 2024-04-25T14:18:07.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.874+0000 7f35e9400640 1 --2- 172.21.15.86:0/803796089 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f35c406f430 0x7f35c4071940 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.874+0000 7f35e9400640 1 --2- 172.21.15.86:0/803796089 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35e4159dc0 0x7f35e40af1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.874+0000 7f35e9400640 1 --2- 172.21.15.86:0/803796089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35e40cdcc0 0x7f35e40c2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.874+0000 7f35e9400640 1 --2- 172.21.15.86:0/803796089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35e4070330 0x7f35e40c40a0 unknown :-1 s=CLOSED pgs=2493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:07.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.874+0000 7f35e9400640 1 -- 172.21.15.86:0/803796089 >> 172.21.15.86:0/803796089 conn(0x7f35e40b4650 msgr2=0x7f35e40b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:07.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.874+0000 7f35e9400640 1 -- 172.21.15.86:0/803796089 shutdown_connections 2024-04-25T14:18:07.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:07.874+0000 7f35e9400640 1 -- 172.21.15.86:0/803796089 wait complete. 2024-04-25T14:18:08.888 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T14:18:08.889 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T14:18:09.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.058+0000 7fa7d6a00640 1 Processor -- start 2024-04-25T14:18:09.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.058+0000 7fa7d6a00640 1 -- start start 2024-04-25T14:18:09.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.058+0000 7fa7d6a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa7d015ca50 0x7fa7d0073950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:09.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.058+0000 7fa7d6a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7d0160d00 0x7fa7d006e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:09.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.058+0000 7fa7d6a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7d015ce60 con 0x7fa7d00743c0 2024-04-25T14:18:09.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.058+0000 7fa7d6a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa7d0072b90 con 0x7fa7d015ca50 2024-04-25T14:18:09.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.058+0000 7fa7d6a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa7d0072e70 con 0x7fa7d0160d00 2024-04-25T14:18:09.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.058+0000 7fa7d4a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7d0160d00 0x7fa7d006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:09.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.058+0000 7fa7cfe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa7d015ca50 0x7fa7d0073950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:09.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.058+0000 7fa7d4a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7d0160d00 0x7fa7d006e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:40308/0 (socket says 172.21.15.86:40308) 2024-04-25T14:18:09.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.058+0000 7fa7d4a00640 1 -- 172.21.15.86:0/3557278726 learned_addr learned my addr 172.21.15.86:0/3557278726 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:09.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.058+0000 7fa7d4a00640 1 -- 172.21.15.86:0/3557278726 >> v2:172.21.15.155:3300/0 conn(0x7fa7d015ca50 msgr2=0x7fa7d0073950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:09.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.058+0000 7fa7d4a00640 1 --2- 172.21.15.86:0/3557278726 >> v2:172.21.15.155:3300/0 conn(0x7fa7d015ca50 0x7fa7d0073950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.058+0000 7fa7d4a00640 1 -- 172.21.15.86:0/3557278726 >> v1:172.21.15.155:6789/0 conn(0x7fa7d00743c0 legacy=0x7fa7d01605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:09.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.058+0000 7fa7d4a00640 1 -- 172.21.15.86:0/3557278726 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d0073150 con 0x7fa7d0160d00 2024-04-25T14:18:09.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.058+0000 7fa7cf400640 1 -- 172.21.15.86:0/3557278726 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3820556542 0 0) 0x7fa7d015ce60 con 0x7fa7d00743c0 2024-04-25T14:18:09.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.062+0000 7fa7cfe00640 1 --2- 172.21.15.86:0/3557278726 >> v2:172.21.15.155:3300/0 conn(0x7fa7d015ca50 0x7fa7d0073950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:18:09.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.062+0000 7fa7d4a00640 1 --2- 172.21.15.86:0/3557278726 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7d0160d00 0x7fa7d006e770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8009e50 tx=0x7fa7c801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=781cf4ef09b6dd89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.062+0000 7fa7cf400640 1 -- 172.21.15.86:0/3557278726 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c80a5810 con 0x7fa7d0160d00 2024-04-25T14:18:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.062+0000 7fa7cf400640 1 -- 172.21.15.86:0/3557278726 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7c80a59b0 con 0x7fa7d0160d00 2024-04-25T14:18:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.062+0000 7fa7cf400640 1 -- 172.21.15.86:0/3557278726 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c80a5d80 con 0x7fa7d0160d00 2024-04-25T14:18:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.062+0000 7fa7d6a00640 1 -- 172.21.15.86:0/3557278726 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7d0160d00 msgr2=0x7fa7d006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.062+0000 7fa7d6a00640 1 --2- 172.21.15.86:0/3557278726 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7d0160d00 0x7fa7d006e770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8009e50 tx=0x7fa7c801ca30 comp rx=0 tx=0).stop 2024-04-25T14:18:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.062+0000 7fa7d6a00640 1 -- 172.21.15.86:0/3557278726 shutdown_connections 2024-04-25T14:18:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.062+0000 7fa7d6a00640 1 --2- 172.21.15.86:0/3557278726 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7d0160d00 0x7fa7d006e770 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.062+0000 7fa7d6a00640 1 --2- 172.21.15.86:0/3557278726 >> v2:172.21.15.155:3300/0 conn(0x7fa7d015ca50 0x7fa7d0073950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.062+0000 7fa7d6a00640 1 -- 172.21.15.86:0/3557278726 >> 172.21.15.86:0/3557278726 conn(0x7fa7d00b4a40 msgr2=0x7fa7d00b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.066+0000 7fa7d6a00640 1 -- 172.21.15.86:0/3557278726 shutdown_connections 2024-04-25T14:18:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.066+0000 7fa7d6a00640 1 -- 172.21.15.86:0/3557278726 wait complete. 2024-04-25T14:18:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.066+0000 7fa7d6a00640 1 Processor -- start 2024-04-25T14:18:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.066+0000 7fa7d6a00640 1 -- start start 2024-04-25T14:18:09.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.066+0000 7fa7d6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7d00743c0 0x7fa7d0158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:09.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.066+0000 7fa7d6a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7d015ca50 0x7fa7d015d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:09.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.066+0000 7fa7d6a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7d0149820 0x7fa7d015d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:09.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.066+0000 7fa7d6a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d016a530 con 0x7fa7d00743c0 2024-04-25T14:18:09.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.066+0000 7fa7d6a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d016a3b0 con 0x7fa7d0149820 2024-04-25T14:18:09.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.066+0000 7fa7d6a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa7d016a6b0 con 0x7fa7d015ca50 2024-04-25T14:18:09.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.070+0000 7fa7cfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7d00743c0 0x7fa7d0158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:09.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.070+0000 7fa7cfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7d00743c0 0x7fa7d0158980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46738/0 (socket says 172.21.15.86:46738) 2024-04-25T14:18:09.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.070+0000 7fa7cfe00640 1 -- 172.21.15.86:0/1352810240 learned_addr learned my addr 172.21.15.86:0/1352810240 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:09.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.070+0000 7fa7d4a00640 1 --2- 172.21.15.86:0/1352810240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7d0149820 0x7fa7d015d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:09.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.070+0000 7fa7c7e00640 1 --2- 172.21.15.86:0/1352810240 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7d015ca50 0x7fa7d015d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:09.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.070+0000 7fa7cfe00640 1 -- 172.21.15.86:0/1352810240 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7d015ca50 msgr2=0x7fa7d015d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:09.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.070+0000 7fa7cfe00640 1 --2- 172.21.15.86:0/1352810240 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7d015ca50 0x7fa7d015d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.070+0000 7fa7cfe00640 1 -- 172.21.15.86:0/1352810240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7d0149820 msgr2=0x7fa7d015d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:09.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.070+0000 7fa7cfe00640 1 --2- 172.21.15.86:0/1352810240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7d0149820 0x7fa7d015d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.070+0000 7fa7cfe00640 1 -- 172.21.15.86:0/1352810240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d015f330 con 0x7fa7d00743c0 2024-04-25T14:18:09.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.070+0000 7fa7cfe00640 1 --2- 172.21.15.86:0/1352810240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7d00743c0 0x7fa7d0158980 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000e930 tx=0x7fa7c000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:09.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.070+0000 7fa7ccc00640 1 -- 172.21.15.86:0/1352810240 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0017070 con 0x7fa7d00743c0 2024-04-25T14:18:09.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.070+0000 7fa7ccc00640 1 -- 172.21.15.86:0/1352810240 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7c000c9a0 con 0x7fa7d00743c0 2024-04-25T14:18:09.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.070+0000 7fa7ccc00640 1 -- 172.21.15.86:0/1352810240 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c000cd70 con 0x7fa7d00743c0 2024-04-25T14:18:09.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.070+0000 7fa7d6a00640 1 -- 172.21.15.86:0/1352810240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7d015f610 con 0x7fa7d00743c0 2024-04-25T14:18:09.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.070+0000 7fa7d6a00640 1 -- 172.21.15.86:0/1352810240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7d015fbd0 con 0x7fa7d00743c0 2024-04-25T14:18:09.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.070+0000 7fa7d6a00640 1 -- 172.21.15.86:0/1352810240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa7d0050750 con 0x7fa7d00743c0 2024-04-25T14:18:09.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.070+0000 7fa7ccc00640 1 -- 172.21.15.86:0/1352810240 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7fa7c001a070 con 0x7fa7d00743c0 2024-04-25T14:18:09.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.074+0000 7fa7ccc00640 1 --2- 172.21.15.86:0/1352810240 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fa7ac06f400 0x7fa7ac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:09.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.074+0000 7fa7ccc00640 1 -- 172.21.15.86:0/1352810240 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 6023+0+0 (secure 0 0 0) 0x7fa7c008f4b0 con 0x7fa7d00743c0 2024-04-25T14:18:09.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.074+0000 7fa7c7e00640 1 --2- 172.21.15.86:0/1352810240 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fa7ac06f400 0x7fa7ac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:09.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.074+0000 7fa7ccc00640 1 --2- 172.21.15.86:0/1352810240 >> [v2:172.21.15.86:6822/2230428952,v1:172.21.15.86:6824/2230428952] conn(0x7fa7ac075ee0 0x7fa7ac078370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:09.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.074+0000 7fa7ccc00640 1 -- 172.21.15.86:0/1352810240 --> [v2:172.21.15.86:6822/2230428952,v1:172.21.15.86:6824/2230428952] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa7ac078a40 con 0x7fa7ac075ee0 2024-04-25T14:18:09.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.074+0000 7fa7ccc00640 1 -- 172.21.15.86:0/1352810240 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=160) ==== 24+0+0 (secure 0 0 0) 0x7fa7c0096050 con 0x7fa7d00743c0 2024-04-25T14:18:09.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.074+0000 7fa7d4a00640 1 --2- 172.21.15.86:0/1352810240 >> [v2:172.21.15.86:6822/2230428952,v1:172.21.15.86:6824/2230428952] conn(0x7fa7ac075ee0 0x7fa7ac078370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:09.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.074+0000 7fa7c7e00640 1 --2- 172.21.15.86:0/1352810240 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fa7ac06f400 0x7fa7ac071910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa7d015b2c0 tx=0x7fa7b8009040 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:09.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.074+0000 7fa7d4a00640 1 --2- 172.21.15.86:0/1352810240 >> [v2:172.21.15.86:6822/2230428952,v1:172.21.15.86:6824/2230428952] conn(0x7fa7ac075ee0 0x7fa7ac078370 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:09.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.074+0000 7fa7ccc00640 1 -- 172.21.15.86:0/1352810240 <== osd.0 v2:172.21.15.86:6822/2230428952 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa7ac078a40 con 0x7fa7ac075ee0 2024-04-25T14:18:09.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.086+0000 7fa7d6a00640 1 -- 172.21.15.86:0/1352810240 --> [v2:172.21.15.86:6822/2230428952,v1:172.21.15.86:6824/2230428952] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa7d00b3d30 con 0x7fa7ac075ee0 2024-04-25T14:18:09.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.086+0000 7fa7ccc00640 1 -- 172.21.15.86:0/1352810240 <== osd.0 v2:172.21.15.86:6822/2230428952 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa7d00b3d30 con 0x7fa7ac075ee0 2024-04-25T14:18:09.082 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:18:09.083 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T14:18:09.083 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:18:09.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.086+0000 7fa7d6a00640 1 -- 172.21.15.86:0/1352810240 >> [v2:172.21.15.86:6822/2230428952,v1:172.21.15.86:6824/2230428952] conn(0x7fa7ac075ee0 msgr2=0x7fa7ac078370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:09.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.086+0000 7fa7d6a00640 1 --2- 172.21.15.86:0/1352810240 >> [v2:172.21.15.86:6822/2230428952,v1:172.21.15.86:6824/2230428952] conn(0x7fa7ac075ee0 0x7fa7ac078370 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.086+0000 7fa7d6a00640 1 -- 172.21.15.86:0/1352810240 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fa7ac06f400 msgr2=0x7fa7ac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:09.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.086+0000 7fa7d6a00640 1 --2- 172.21.15.86:0/1352810240 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fa7ac06f400 0x7fa7ac071910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa7d015b2c0 tx=0x7fa7b8009040 comp rx=0 tx=0).stop 2024-04-25T14:18:09.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.086+0000 7fa7d6a00640 1 -- 172.21.15.86:0/1352810240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7d00743c0 msgr2=0x7fa7d0158980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:09.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.086+0000 7fa7d6a00640 1 --2- 172.21.15.86:0/1352810240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7d00743c0 0x7fa7d0158980 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000e930 tx=0x7fa7c000edf0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.086+0000 7fa7d4a00640 1 -- 172.21.15.86:0/1352810240 reap_dead start 2024-04-25T14:18:09.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.086+0000 7fa7d6a00640 1 -- 172.21.15.86:0/1352810240 shutdown_connections 2024-04-25T14:18:09.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.086+0000 7fa7d6a00640 1 -- 172.21.15.86:0/1352810240 >> 172.21.15.86:0/1352810240 conn(0x7fa7d00b4a40 msgr2=0x7fa7d006c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:09.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.090+0000 7fa7d6a00640 1 -- 172.21.15.86:0/1352810240 shutdown_connections 2024-04-25T14:18:09.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.090+0000 7fa7d6a00640 1 -- 172.21.15.86:0/1352810240 wait complete. 2024-04-25T14:18:09.102 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T14:18:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.258+0000 7f06aa800640 1 Processor -- start 2024-04-25T14:18:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.258+0000 7f06aa800640 1 -- start start 2024-04-25T14:18:09.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.258+0000 7f06aa800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06a41651d0 0x7f06a41675f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:09.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.258+0000 7f06aa800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06a40b0a60 con 0x7f06a415c2a0 2024-04-25T14:18:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.258+0000 7f06aa800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06a40ae840 con 0x7f06a406fd50 2024-04-25T14:18:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.258+0000 7f06aa800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f06a406ef90 con 0x7f06a41651d0 2024-04-25T14:18:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.258+0000 7f06a3400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f06a415c2a0 0x7f06a40ae130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41850/0 (socket says 172.21.15.86:41850) 2024-04-25T14:18:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.258+0000 7f06a3400640 1 -- 172.21.15.86:0/62072897 learned_addr learned my addr 172.21.15.86:0/62072897 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:18:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.258+0000 7f06a3e00640 1 --2- 172.21.15.86:0/62072897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06a41651d0 0x7f06a41675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.258+0000 7f06a2a00640 1 -- 172.21.15.86:0/62072897 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3919536435 0 0) 0x7f06a40b0a60 con 0x7f06a415c2a0 2024-04-25T14:18:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.258+0000 7f06a2a00640 1 -- 172.21.15.86:0/62072897 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0688003530 con 0x7f06a415c2a0 2024-04-25T14:18:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.258+0000 7f06a2a00640 1 -- 172.21.15.86:0/62072897 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2472967510 0 0) 0x7f06a40ae840 con 0x7f06a406fd50 2024-04-25T14:18:09.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.258+0000 7f06a2a00640 1 -- 172.21.15.86:0/62072897 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06a40b0a60 con 0x7f06a406fd50 2024-04-25T14:18:09.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.258+0000 7f06a3e00640 1 -- 172.21.15.86:0/62072897 >> v1:172.21.15.155:6789/0 conn(0x7f06a406fd50 legacy=0x7f06a4070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:09.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.258+0000 7f06a3e00640 1 -- 172.21.15.86:0/62072897 >> v1:172.21.15.86:6789/0 conn(0x7f06a415c2a0 legacy=0x7f06a40ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:09.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.258+0000 7f06a3e00640 1 -- 172.21.15.86:0/62072897 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a406f270 con 0x7f06a41651d0 2024-04-25T14:18:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.262+0000 7f06a3e00640 1 --2- 172.21.15.86:0/62072897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06a41651d0 0x7f06a41675f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f069c00aed0 tx=0x7f069c01c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25fbff37ee721873 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.262+0000 7f06a2a00640 1 -- 172.21.15.86:0/62072897 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c0a5710 con 0x7f06a41651d0 2024-04-25T14:18:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.262+0000 7f06a2a00640 1 -- 172.21.15.86:0/62072897 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f069c0a58b0 con 0x7f06a41651d0 2024-04-25T14:18:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.262+0000 7f06a2a00640 1 -- 172.21.15.86:0/62072897 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c0a5c80 con 0x7f06a41651d0 2024-04-25T14:18:09.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.262+0000 7f06aa800640 1 -- 172.21.15.86:0/62072897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06a41651d0 msgr2=0x7f06a41675f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:09.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.262+0000 7f06aa800640 1 --2- 172.21.15.86:0/62072897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06a41651d0 0x7f06a41675f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f069c00aed0 tx=0x7f069c01c8f0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.262+0000 7f06aa800640 1 -- 172.21.15.86:0/62072897 shutdown_connections 2024-04-25T14:18:09.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.262+0000 7f06aa800640 1 --2- 172.21.15.86:0/62072897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06a41651d0 0x7f06a41675f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.262+0000 7f06aa800640 1 -- 172.21.15.86:0/62072897 >> 172.21.15.86:0/62072897 conn(0x7f06a40b4650 msgr2=0x7f06a40b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:09.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.262+0000 7f06aa800640 1 -- 172.21.15.86:0/62072897 shutdown_connections 2024-04-25T14:18:09.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.262+0000 7f06aa800640 1 -- 172.21.15.86:0/62072897 wait complete. 2024-04-25T14:18:09.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.266+0000 7f06aa800640 1 Processor -- start 2024-04-25T14:18:09.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.266+0000 7f06aa800640 1 -- start start 2024-04-25T14:18:09.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.266+0000 7f06aa800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06a406fd50 0x7f06a415dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:09.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.266+0000 7f06aa800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06a415c2a0 0x7f06a41554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:09.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.266+0000 7f06aa800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06a41651d0 0x7f06a415d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:09.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.266+0000 7f06aa800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f06a415c830 con 0x7f06a41651d0 2024-04-25T14:18:09.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.266+0000 7f06aa800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f06a415c6b0 con 0x7f06a406fd50 2024-04-25T14:18:09.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.266+0000 7f06aa800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f06a4169b40 con 0x7f06a415c2a0 2024-04-25T14:18:09.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.266+0000 7f06a3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06a41651d0 0x7f06a415d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:09.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.266+0000 7f06a3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06a41651d0 0x7f06a415d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46742/0 (socket says 172.21.15.86:46742) 2024-04-25T14:18:09.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.266+0000 7f06a3e00640 1 -- 172.21.15.86:0/3252138347 learned_addr learned my addr 172.21.15.86:0/3252138347 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:09.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.266+0000 7f06a3400640 1 --2- 172.21.15.86:0/3252138347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06a406fd50 0x7f06a415dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:09.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.266+0000 7f069be00640 1 --2- 172.21.15.86:0/3252138347 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06a415c2a0 0x7f06a41554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:09.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.266+0000 7f06a3e00640 1 -- 172.21.15.86:0/3252138347 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06a415c2a0 msgr2=0x7f06a41554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:09.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.266+0000 7f06a3e00640 1 --2- 172.21.15.86:0/3252138347 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06a415c2a0 0x7f06a41554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.266+0000 7f06a3e00640 1 -- 172.21.15.86:0/3252138347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06a406fd50 msgr2=0x7f06a415dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:09.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.266+0000 7f06a3e00640 1 --2- 172.21.15.86:0/3252138347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06a406fd50 0x7f06a415dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.266+0000 7f06a3e00640 1 -- 172.21.15.86:0/3252138347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a40b1890 con 0x7f06a41651d0 2024-04-25T14:18:09.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.266+0000 7f069be00640 1 --2- 172.21.15.86:0/3252138347 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06a415c2a0 0x7f06a41554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:18:09.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.270+0000 7f06a3e00640 1 --2- 172.21.15.86:0/3252138347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06a41651d0 0x7f06a415d400 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f069c019500 tx=0x7f069c019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:09.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.270+0000 7f069b400640 1 -- 172.21.15.86:0/3252138347 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c0bc020 con 0x7f06a41651d0 2024-04-25T14:18:09.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.270+0000 7f069b400640 1 -- 172.21.15.86:0/3252138347 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f069c0b8670 con 0x7f06a41651d0 2024-04-25T14:18:09.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.270+0000 7f06aa800640 1 -- 172.21.15.86:0/3252138347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a40b1b70 con 0x7f06a41651d0 2024-04-25T14:18:09.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.270+0000 7f069b400640 1 -- 172.21.15.86:0/3252138347 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c00a560 con 0x7f06a41651d0 2024-04-25T14:18:09.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.270+0000 7f06aa800640 1 -- 172.21.15.86:0/3252138347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a415cee0 con 0x7f06a41651d0 2024-04-25T14:18:09.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.270+0000 7f06aa800640 1 -- 172.21.15.86:0/3252138347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a4069d00 con 0x7f06a41651d0 2024-04-25T14:18:09.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.270+0000 7f069b400640 1 -- 172.21.15.86:0/3252138347 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7f069c0bf020 con 0x7f06a41651d0 2024-04-25T14:18:09.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.270+0000 7f069b400640 1 --2- 172.21.15.86:0/3252138347 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f068006f400 0x7f0680071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:09.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.270+0000 7f069b400640 1 -- 172.21.15.86:0/3252138347 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 6023+0+0 (secure 0 0 0) 0x7f069c12a1c0 con 0x7f06a41651d0 2024-04-25T14:18:09.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.270+0000 7f06a3400640 1 --2- 172.21.15.86:0/3252138347 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f068006f400 0x7f0680071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:09.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.274+0000 7f06a3400640 1 --2- 172.21.15.86:0/3252138347 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f068006f400 0x7f0680071910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0694007fd0 tx=0x7f0694007d80 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:09.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.278+0000 7f069b400640 1 -- 172.21.15.86:0/3252138347 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f069c0c6030 con 0x7f06a41651d0 2024-04-25T14:18:09.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.434+0000 7f06aa800640 1 -- 172.21.15.86:0/3252138347 --> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f06a4076bf0 con 0x7f068006f400 2024-04-25T14:18:09.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.434+0000 7f069b400640 1 -- 172.21.15.86:0/3252138347 <== mgr.11343 v2:172.21.15.86:6832/3174179029 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f06a4076bf0 con 0x7f068006f400 2024-04-25T14:18:09.428 INFO:teuthology.orchestra.run.smithi086.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T14:18:09.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.434+0000 7f06aa800640 1 -- 172.21.15.86:0/3252138347 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f068006f400 msgr2=0x7f0680071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.434+0000 7f06aa800640 1 --2- 172.21.15.86:0/3252138347 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f068006f400 0x7f0680071910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0694007fd0 tx=0x7f0694007d80 comp rx=0 tx=0).stop 2024-04-25T14:18:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.434+0000 7f06aa800640 1 -- 172.21.15.86:0/3252138347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06a41651d0 msgr2=0x7f06a415d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.434+0000 7f06aa800640 1 --2- 172.21.15.86:0/3252138347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06a41651d0 0x7f06a415d400 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f069c019500 tx=0x7f069c019690 comp rx=0 tx=0).stop 2024-04-25T14:18:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.434+0000 7f06aa800640 1 -- 172.21.15.86:0/3252138347 shutdown_connections 2024-04-25T14:18:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.434+0000 7f06aa800640 1 --2- 172.21.15.86:0/3252138347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06a41651d0 0x7f06a415d400 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.434+0000 7f06aa800640 1 --2- 172.21.15.86:0/3252138347 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f068006f400 0x7f0680071910 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.434+0000 7f06aa800640 1 --2- 172.21.15.86:0/3252138347 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06a415c2a0 0x7f06a41554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.434+0000 7f06aa800640 1 --2- 172.21.15.86:0/3252138347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06a406fd50 0x7f06a415dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.434+0000 7f06aa800640 1 -- 172.21.15.86:0/3252138347 >> 172.21.15.86:0/3252138347 conn(0x7f06a40b4650 msgr2=0x7f06a4072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:09.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.434+0000 7f06aa800640 1 -- 172.21.15.86:0/3252138347 shutdown_connections 2024-04-25T14:18:09.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.434+0000 7f06aa800640 1 -- 172.21.15.86:0/3252138347 wait complete. 2024-04-25T14:18:09.450 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T14:18:09.451 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T14:18:09.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.606+0000 7fe0cac00640 1 Processor -- start 2024-04-25T14:18:09.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.606+0000 7fe0cac00640 1 -- start start 2024-04-25T14:18:09.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.606+0000 7fe0cac00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0c40701f0 0x7fe0c40729f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:09.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.606+0000 7fe0cac00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe0c4070600 0x7fe0c4072f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:09.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.606+0000 7fe0cac00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe0c406c670 0x7fe0c406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:09.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.606+0000 7fe0cac00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe0c406f1e0 con 0x7fe0c406c670 2024-04-25T14:18:09.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.606+0000 7fe0cac00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe0c406f430 con 0x7fe0c4070600 2024-04-25T14:18:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.606+0000 7fe0cac00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe0c406f710 con 0x7fe0c40701f0 2024-04-25T14:18:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.610+0000 7fe0c8c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe0c406c670 0x7fe0c406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.610+0000 7fe0c8c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe0c406c670 0x7fe0c406ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46754/0 (socket says 172.21.15.86:46754) 2024-04-25T14:18:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.610+0000 7fe0c8c00640 1 -- 172.21.15.86:0/649531889 learned_addr learned my addr 172.21.15.86:0/649531889 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.610+0000 7fe0bb400640 1 --2- 172.21.15.86:0/649531889 >> v2:172.21.15.155:3300/0 conn(0x7fe0c4070600 0x7fe0c4072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.610+0000 7fe0c3e00640 1 --2- 172.21.15.86:0/649531889 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0c40701f0 0x7fe0c40729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.610+0000 7fe0c8c00640 1 -- 172.21.15.86:0/649531889 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0c40701f0 msgr2=0x7fe0c40729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.610+0000 7fe0c8c00640 1 --2- 172.21.15.86:0/649531889 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0c40701f0 0x7fe0c40729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.610+0000 7fe0c8c00640 1 -- 172.21.15.86:0/649531889 >> v2:172.21.15.155:3300/0 conn(0x7fe0c4070600 msgr2=0x7fe0c4072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:09.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.610+0000 7fe0c8c00640 1 --2- 172.21.15.86:0/649531889 >> v2:172.21.15.155:3300/0 conn(0x7fe0c4070600 0x7fe0c4072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.610+0000 7fe0c8c00640 1 -- 172.21.15.86:0/649531889 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c406f9f0 con 0x7fe0c406c670 2024-04-25T14:18:09.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.610+0000 7fe0c8c00640 1 --2- 172.21.15.86:0/649531889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0c406c670 0x7fe0c406ca80 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7fe0c40b5300 tx=0x7fe0bc01eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6404bd1656229932 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:09.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.610+0000 7fe0c3400640 1 -- 172.21.15.86:0/649531889 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc0b4070 con 0x7fe0c406c670 2024-04-25T14:18:09.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.610+0000 7fe0c3400640 1 -- 172.21.15.86:0/649531889 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0bc0a7970 con 0x7fe0c406c670 2024-04-25T14:18:09.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.610+0000 7fe0c3400640 1 -- 172.21.15.86:0/649531889 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc0a7d40 con 0x7fe0c406c670 2024-04-25T14:18:09.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.614+0000 7fe0cac00640 1 -- 172.21.15.86:0/649531889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0c406c670 msgr2=0x7fe0c406ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:09.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.614+0000 7fe0cac00640 1 --2- 172.21.15.86:0/649531889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0c406c670 0x7fe0c406ca80 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7fe0c40b5300 tx=0x7fe0bc01eb60 comp rx=0 tx=0).stop 2024-04-25T14:18:09.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.614+0000 7fe0cac00640 1 -- 172.21.15.86:0/649531889 shutdown_connections 2024-04-25T14:18:09.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.614+0000 7fe0cac00640 1 --2- 172.21.15.86:0/649531889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0c406c670 0x7fe0c406ca80 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.614+0000 7fe0cac00640 1 --2- 172.21.15.86:0/649531889 >> v2:172.21.15.155:3300/0 conn(0x7fe0c4070600 0x7fe0c4072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.614+0000 7fe0cac00640 1 --2- 172.21.15.86:0/649531889 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0c40701f0 0x7fe0c40729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.614+0000 7fe0cac00640 1 -- 172.21.15.86:0/649531889 >> 172.21.15.86:0/649531889 conn(0x7fe0c40b4a40 msgr2=0x7fe0c40b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:09.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.614+0000 7fe0cac00640 1 -- 172.21.15.86:0/649531889 shutdown_connections 2024-04-25T14:18:09.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.614+0000 7fe0cac00640 1 -- 172.21.15.86:0/649531889 wait complete. 2024-04-25T14:18:09.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.614+0000 7fe0cac00640 1 Processor -- start 2024-04-25T14:18:09.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.614+0000 7fe0cac00640 1 -- start start 2024-04-25T14:18:09.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.618+0000 7fe0cac00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0c406c670 0x7fe0c409a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:09.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.618+0000 7fe0cac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0c40701f0 0x7fe0c4143c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:09.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.618+0000 7fe0cac00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0c4070600 0x7fe0c4144150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:09.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.618+0000 7fe0cac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c415fdc0 con 0x7fe0c40701f0 2024-04-25T14:18:09.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.618+0000 7fe0cac00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c415d5c0 con 0x7fe0c4070600 2024-04-25T14:18:09.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.618+0000 7fe0cac00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe0c415ff40 con 0x7fe0c406c670 2024-04-25T14:18:09.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.618+0000 7fe0bb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0c40701f0 0x7fe0c4143c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:09.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.618+0000 7fe0bb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0c40701f0 0x7fe0c4143c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46760/0 (socket says 172.21.15.86:46760) 2024-04-25T14:18:09.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.618+0000 7fe0bb400640 1 -- 172.21.15.86:0/3190660226 learned_addr learned my addr 172.21.15.86:0/3190660226 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:09.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.618+0000 7fe0c3e00640 1 --2- 172.21.15.86:0/3190660226 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0c406c670 0x7fe0c409a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:09.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.618+0000 7fe0c8c00640 1 --2- 172.21.15.86:0/3190660226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0c4070600 0x7fe0c4144150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:09.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.618+0000 7fe0bb400640 1 -- 172.21.15.86:0/3190660226 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0c406c670 msgr2=0x7fe0c409a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:09.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.618+0000 7fe0bb400640 1 --2- 172.21.15.86:0/3190660226 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0c406c670 0x7fe0c409a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.618+0000 7fe0bb400640 1 -- 172.21.15.86:0/3190660226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0c4070600 msgr2=0x7fe0c4144150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:09.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.618+0000 7fe0bb400640 1 --2- 172.21.15.86:0/3190660226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0c4070600 0x7fe0c4144150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.618+0000 7fe0bb400640 1 -- 172.21.15.86:0/3190660226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c4160ca0 con 0x7fe0c40701f0 2024-04-25T14:18:09.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.618+0000 7fe0bb400640 1 --2- 172.21.15.86:0/3190660226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0c40701f0 0x7fe0c4143c10 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00d5a0 tx=0x7fe0ac00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:09.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.618+0000 7fe0c0c00640 1 -- 172.21.15.86:0/3190660226 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac0125f0 con 0x7fe0c40701f0 2024-04-25T14:18:09.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.618+0000 7fe0c0c00640 1 -- 172.21.15.86:0/3190660226 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0ac019070 con 0x7fe0c40701f0 2024-04-25T14:18:09.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.618+0000 7fe0cac00640 1 -- 172.21.15.86:0/3190660226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c415b3c0 con 0x7fe0c40701f0 2024-04-25T14:18:09.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.618+0000 7fe0c0c00640 1 -- 172.21.15.86:0/3190660226 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac0129b0 con 0x7fe0c40701f0 2024-04-25T14:18:09.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.618+0000 7fe0cac00640 1 -- 172.21.15.86:0/3190660226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c415b0e0 con 0x7fe0c40701f0 2024-04-25T14:18:09.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.618+0000 7fe0cac00640 1 -- 172.21.15.86:0/3190660226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe0c4050750 con 0x7fe0c40701f0 2024-04-25T14:18:09.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.622+0000 7fe0c0c00640 1 -- 172.21.15.86:0/3190660226 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7fe0ac012b50 con 0x7fe0c40701f0 2024-04-25T14:18:09.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.622+0000 7fe0c0c00640 1 --2- 172.21.15.86:0/3190660226 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fe0a006f430 0x7fe0a0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.622+0000 7fe0c0c00640 1 -- 172.21.15.86:0/3190660226 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 6023+0+0 (secure 0 0 0) 0x7fe0ac08f590 con 0x7fe0c40701f0 2024-04-25T14:18:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.622+0000 7fe0c3e00640 1 --2- 172.21.15.86:0/3190660226 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fe0a006f430 0x7fe0a0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.622+0000 7fe0c0c00640 1 --2- 172.21.15.86:0/3190660226 >> [v2:172.21.15.86:6808/1043507731,v1:172.21.15.86:6809/1043507731] conn(0x7fe0a0075f10 0x7fe0a00783a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.622+0000 7fe0c0c00640 1 -- 172.21.15.86:0/3190660226 --> [v2:172.21.15.86:6808/1043507731,v1:172.21.15.86:6809/1043507731] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe0a0078a70 con 0x7fe0a0075f10 2024-04-25T14:18:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.622+0000 7fe0c0c00640 1 -- 172.21.15.86:0/3190660226 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=160) ==== 24+0+0 (secure 0 0 0) 0x7fe0ac096050 con 0x7fe0c40701f0 2024-04-25T14:18:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.622+0000 7fe0c8c00640 1 --2- 172.21.15.86:0/3190660226 >> [v2:172.21.15.86:6808/1043507731,v1:172.21.15.86:6809/1043507731] conn(0x7fe0a0075f10 0x7fe0a00783a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.622+0000 7fe0c3e00640 1 --2- 172.21.15.86:0/3190660226 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fe0a006f430 0x7fe0a0071940 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe0b40039f0 tx=0x7fe0b4009790 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.622+0000 7fe0c8c00640 1 --2- 172.21.15.86:0/3190660226 >> [v2:172.21.15.86:6808/1043507731,v1:172.21.15.86:6809/1043507731] conn(0x7fe0a0075f10 0x7fe0a00783a0 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:09.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.622+0000 7fe0c0c00640 1 -- 172.21.15.86:0/3190660226 <== osd.1 v2:172.21.15.86:6808/1043507731 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe0a0078a70 con 0x7fe0a0075f10 2024-04-25T14:18:09.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.634+0000 7fe0cac00640 1 -- 172.21.15.86:0/3190660226 --> [v2:172.21.15.86:6808/1043507731,v1:172.21.15.86:6809/1043507731] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe0c40b0380 con 0x7fe0a0075f10 2024-04-25T14:18:09.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.638+0000 7fe0c0c00640 1 -- 172.21.15.86:0/3190660226 <== osd.1 v2:172.21.15.86:6808/1043507731 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe0c40b0380 con 0x7fe0a0075f10 2024-04-25T14:18:09.631 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:18:09.631 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T14:18:09.631 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:18:09.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.638+0000 7fe0cac00640 1 -- 172.21.15.86:0/3190660226 >> [v2:172.21.15.86:6808/1043507731,v1:172.21.15.86:6809/1043507731] conn(0x7fe0a0075f10 msgr2=0x7fe0a00783a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:09.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.638+0000 7fe0cac00640 1 --2- 172.21.15.86:0/3190660226 >> [v2:172.21.15.86:6808/1043507731,v1:172.21.15.86:6809/1043507731] conn(0x7fe0a0075f10 0x7fe0a00783a0 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.638+0000 7fe0cac00640 1 -- 172.21.15.86:0/3190660226 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fe0a006f430 msgr2=0x7fe0a0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:09.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.638+0000 7fe0cac00640 1 --2- 172.21.15.86:0/3190660226 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fe0a006f430 0x7fe0a0071940 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe0b40039f0 tx=0x7fe0b4009790 comp rx=0 tx=0).stop 2024-04-25T14:18:09.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.638+0000 7fe0cac00640 1 -- 172.21.15.86:0/3190660226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0c40701f0 msgr2=0x7fe0c4143c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:09.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.638+0000 7fe0cac00640 1 --2- 172.21.15.86:0/3190660226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0c40701f0 0x7fe0c4143c10 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00d5a0 tx=0x7fe0ac00da60 comp rx=0 tx=0).stop 2024-04-25T14:18:09.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.638+0000 7fe0c8c00640 1 -- 172.21.15.86:0/3190660226 reap_dead start 2024-04-25T14:18:09.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.638+0000 7fe0cac00640 1 -- 172.21.15.86:0/3190660226 shutdown_connections 2024-04-25T14:18:09.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.638+0000 7fe0cac00640 1 -- 172.21.15.86:0/3190660226 >> 172.21.15.86:0/3190660226 conn(0x7fe0c40b4a40 msgr2=0x7fe0c40af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:09.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.638+0000 7fe0cac00640 1 -- 172.21.15.86:0/3190660226 shutdown_connections 2024-04-25T14:18:09.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.638+0000 7fe0cac00640 1 -- 172.21.15.86:0/3190660226 wait complete. 2024-04-25T14:18:09.650 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T14:18:09.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.802+0000 7f551c600640 1 Processor -- start 2024-04-25T14:18:09.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.802+0000 7f551c600640 1 -- start start 2024-04-25T14:18:09.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.802+0000 7f551c600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f551415e860 0x7f55140ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:09.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.802+0000 7f551c600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55140cf3e0 0x7f55140d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:09.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.802+0000 7f551c600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55140cf8f0 con 0x7f551415ec70 2024-04-25T14:18:09.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.802+0000 7f551c600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f551406cfa0 con 0x7f551415e860 2024-04-25T14:18:09.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.802+0000 7f551c600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f551406c670 con 0x7f55140cf3e0 2024-04-25T14:18:09.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.802+0000 7f5519c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f551415e860 0x7f55140ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:09.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.802+0000 7f5519200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55140cf3e0 0x7f55140d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:09.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.802+0000 7f5519c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f551415e860 0x7f55140ceea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:57280/0 (socket says 172.21.15.86:57280) 2024-04-25T14:18:09.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.802+0000 7f5519c00640 1 -- 172.21.15.86:0/15043472 learned_addr learned my addr 172.21.15.86:0/15043472 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:09.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.802+0000 7f5519c00640 1 -- 172.21.15.86:0/15043472 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55140cf3e0 msgr2=0x7f55140d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:09.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.802+0000 7f5519c00640 1 --2- 172.21.15.86:0/15043472 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55140cf3e0 0x7f55140d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.802+0000 7f5519c00640 1 -- 172.21.15.86:0/15043472 >> v1:172.21.15.155:6789/0 conn(0x7f551415ec70 legacy=0x7f551406d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:09.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.802+0000 7f5519c00640 1 -- 172.21.15.86:0/15043472 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551406c950 con 0x7f551415e860 2024-04-25T14:18:09.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.806+0000 7f5519c00640 1 --2- 172.21.15.86:0/15043472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f551415e860 0x7f55140ceea0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f5508009870 tx=0x7f550801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c219093c6af9b6ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:09.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.806+0000 7f5513e00640 1 -- 172.21.15.86:0/15043472 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55080ab430 con 0x7f551415e860 2024-04-25T14:18:09.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.806+0000 7f5513e00640 1 -- 172.21.15.86:0/15043472 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55080b0070 con 0x7f551415e860 2024-04-25T14:18:09.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.806+0000 7f5513e00640 1 -- 172.21.15.86:0/15043472 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55080ab7f0 con 0x7f551415e860 2024-04-25T14:18:09.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.806+0000 7f551c600640 1 -- 172.21.15.86:0/15043472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f551415e860 msgr2=0x7f55140ceea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:09.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.806+0000 7f551c600640 1 --2- 172.21.15.86:0/15043472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f551415e860 0x7f55140ceea0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f5508009870 tx=0x7f550801c4a0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.806+0000 7f551c600640 1 -- 172.21.15.86:0/15043472 shutdown_connections 2024-04-25T14:18:09.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.806+0000 7f551c600640 1 --2- 172.21.15.86:0/15043472 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55140cf3e0 0x7f55140d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.806+0000 7f551c600640 1 --2- 172.21.15.86:0/15043472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f551415e860 0x7f55140ceea0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.806+0000 7f551c600640 1 -- 172.21.15.86:0/15043472 >> 172.21.15.86:0/15043472 conn(0x7f55140d2230 msgr2=0x7f55140d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:09.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.806+0000 7f551c600640 1 -- 172.21.15.86:0/15043472 shutdown_connections 2024-04-25T14:18:09.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.806+0000 7f551c600640 1 -- 172.21.15.86:0/15043472 wait complete. 2024-04-25T14:18:09.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.810+0000 7f551c600640 1 Processor -- start 2024-04-25T14:18:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.810+0000 7f551c600640 1 -- start start 2024-04-25T14:18:09.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.810+0000 7f551c600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55140cf3e0 0x7f5514157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:09.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.810+0000 7f551c600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f551415e860 0x7f55140b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:09.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.810+0000 7f551c600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f551415ec70 0x7f5514155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:09.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.810+0000 7f551c600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f551416af30 con 0x7f551415e860 2024-04-25T14:18:09.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.810+0000 7f551c600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f55140d18a0 con 0x7f55140cf3e0 2024-04-25T14:18:09.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.810+0000 7f551c600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f551416b0b0 con 0x7f551415ec70 2024-04-25T14:18:09.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.810+0000 7f5519200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f551415e860 0x7f55140b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:09.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.810+0000 7f5519c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55140cf3e0 0x7f5514157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:09.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.810+0000 7f5519c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55140cf3e0 0x7f5514157c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:57286/0 (socket says 172.21.15.86:57286) 2024-04-25T14:18:09.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.810+0000 7f5519c00640 1 -- 172.21.15.86:0/3665849213 learned_addr learned my addr 172.21.15.86:0/3665849213 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:09.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.814+0000 7f551a600640 1 --2- 172.21.15.86:0/3665849213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f551415ec70 0x7f5514155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:09.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.814+0000 7f5519c00640 1 -- 172.21.15.86:0/3665849213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f551415ec70 msgr2=0x7f5514155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:09.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.814+0000 7f5519c00640 1 --2- 172.21.15.86:0/3665849213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f551415ec70 0x7f5514155780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.814+0000 7f5519c00640 1 -- 172.21.15.86:0/3665849213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f551415e860 msgr2=0x7f55140b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:09.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.814+0000 7f5519c00640 1 --2- 172.21.15.86:0/3665849213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f551415e860 0x7f55140b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.814+0000 7f5519c00640 1 -- 172.21.15.86:0/3665849213 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551414dd50 con 0x7f55140cf3e0 2024-04-25T14:18:09.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.814+0000 7f5519c00640 1 --2- 172.21.15.86:0/3665849213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55140cf3e0 0x7f5514157c40 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f550801a0d0 tx=0x7f5508019240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:09.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.814+0000 7f5511600640 1 -- 172.21.15.86:0/3665849213 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55080b9020 con 0x7f55140cf3e0 2024-04-25T14:18:09.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.814+0000 7f5511600640 1 -- 172.21.15.86:0/3665849213 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55080abe40 con 0x7f55140cf3e0 2024-04-25T14:18:09.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.814+0000 7f551c600640 1 -- 172.21.15.86:0/3665849213 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f551414da70 con 0x7f55140cf3e0 2024-04-25T14:18:09.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.814+0000 7f5511600640 1 -- 172.21.15.86:0/3665849213 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55080b4660 con 0x7f55140cf3e0 2024-04-25T14:18:09.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.814+0000 7f551c600640 1 -- 172.21.15.86:0/3665849213 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55140b1be0 con 0x7f55140cf3e0 2024-04-25T14:18:09.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.814+0000 7f551c600640 1 -- 172.21.15.86:0/3665849213 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5514074cf0 con 0x7f55140cf3e0 2024-04-25T14:18:09.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.814+0000 7f5511600640 1 -- 172.21.15.86:0/3665849213 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7f55080bc020 con 0x7f55140cf3e0 2024-04-25T14:18:09.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.818+0000 7f5511600640 1 --2- 172.21.15.86:0/3665849213 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f54f406f400 0x7f54f4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:09.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.818+0000 7f5511600640 1 -- 172.21.15.86:0/3665849213 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 6023+0+0 (secure 0 0 0) 0x7f55081296a0 con 0x7f55140cf3e0 2024-04-25T14:18:09.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.818+0000 7f5519200640 1 --2- 172.21.15.86:0/3665849213 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f54f406f400 0x7f54f4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:09.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.818+0000 7f5519200640 1 --2- 172.21.15.86:0/3665849213 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f54f406f400 0x7f54f4071910 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f55140d1460 tx=0x7f5500009190 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:09.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.822+0000 7f5511600640 1 -- 172.21.15.86:0/3665849213 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f55080c3070 con 0x7f55140cf3e0 2024-04-25T14:18:09.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.978+0000 7f551c600640 1 -- 172.21.15.86:0/3665849213 --> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f5514076bf0 con 0x7f54f406f400 2024-04-25T14:18:09.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.978+0000 7f5511600640 1 -- 172.21.15.86:0/3665849213 <== mgr.11343 v2:172.21.15.86:6832/3174179029 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5514076bf0 con 0x7f54f406f400 2024-04-25T14:18:09.972 INFO:teuthology.orchestra.run.smithi086.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T14:18:09.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.978+0000 7f551c600640 1 -- 172.21.15.86:0/3665849213 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f54f406f400 msgr2=0x7f54f4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:09.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.978+0000 7f551c600640 1 --2- 172.21.15.86:0/3665849213 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f54f406f400 0x7f54f4071910 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f55140d1460 tx=0x7f5500009190 comp rx=0 tx=0).stop 2024-04-25T14:18:09.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.978+0000 7f551c600640 1 -- 172.21.15.86:0/3665849213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55140cf3e0 msgr2=0x7f5514157c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:09.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.978+0000 7f551c600640 1 --2- 172.21.15.86:0/3665849213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55140cf3e0 0x7f5514157c40 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f550801a0d0 tx=0x7f5508019240 comp rx=0 tx=0).stop 2024-04-25T14:18:09.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.978+0000 7f551c600640 1 -- 172.21.15.86:0/3665849213 shutdown_connections 2024-04-25T14:18:09.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.978+0000 7f551c600640 1 --2- 172.21.15.86:0/3665849213 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f54f406f400 0x7f54f4071910 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.978+0000 7f551c600640 1 --2- 172.21.15.86:0/3665849213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f551415ec70 0x7f5514155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.978+0000 7f551c600640 1 --2- 172.21.15.86:0/3665849213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f551415e860 0x7f55140b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.978+0000 7f551c600640 1 --2- 172.21.15.86:0/3665849213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55140cf3e0 0x7f5514157c40 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:09.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.978+0000 7f551c600640 1 -- 172.21.15.86:0/3665849213 >> 172.21.15.86:0/3665849213 conn(0x7f55140d2230 msgr2=0x7f55140b67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:09.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.978+0000 7f551c600640 1 -- 172.21.15.86:0/3665849213 shutdown_connections 2024-04-25T14:18:09.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:09.978+0000 7f551c600640 1 -- 172.21.15.86:0/3665849213 wait complete. 2024-04-25T14:18:09.993 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T14:18:09.993 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T14:18:10.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.150+0000 7f5ea1000640 1 Processor -- start 2024-04-25T14:18:10.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.150+0000 7f5ea1000640 1 -- start start 2024-04-25T14:18:10.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.150+0000 7f5ea1000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5e9c0749b0 0x7f5e9c06c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:10.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.150+0000 7f5ea1000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5e9c0ae4b0 0x7f5e9c0ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:10.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.150+0000 7f5ea1000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e9c0aee00 con 0x7f5e9c0d02d0 2024-04-25T14:18:10.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.150+0000 7f5ea1000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5e9c074dc0 con 0x7f5e9c0ae4b0 2024-04-25T14:18:10.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.150+0000 7f5ea1000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f5e9c06e010 con 0x7f5e9c0749b0 2024-04-25T14:18:10.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.150+0000 7f5e93e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5e9c0ae4b0 0x7f5e9c0ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:10.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.150+0000 7f5e93e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5e9c0ae4b0 0x7f5e9c0ae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46768/0 (socket says 172.21.15.86:46768) 2024-04-25T14:18:10.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.150+0000 7f5e93e00640 1 -- 172.21.15.86:0/1831958072 learned_addr learned my addr 172.21.15.86:0/1831958072 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:10.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.150+0000 7f5e9b400640 1 --2- 172.21.15.86:0/1831958072 >> v2:172.21.15.155:3300/0 conn(0x7f5e9c0749b0 0x7f5e9c06c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:10.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.150+0000 7f5e93e00640 1 -- 172.21.15.86:0/1831958072 >> v2:172.21.15.155:3300/0 conn(0x7f5e9c0749b0 msgr2=0x7f5e9c06c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:10.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.150+0000 7f5e93e00640 1 --2- 172.21.15.86:0/1831958072 >> v2:172.21.15.155:3300/0 conn(0x7f5e9c0749b0 0x7f5e9c06c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:10.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.150+0000 7f5e93e00640 1 -- 172.21.15.86:0/1831958072 >> v1:172.21.15.155:6789/0 conn(0x7f5e9c0d02d0 legacy=0x7f5e9c0d06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:10.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.150+0000 7f5e93e00640 1 -- 172.21.15.86:0/1831958072 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e9c06cf70 con 0x7f5e9c0ae4b0 2024-04-25T14:18:10.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.150+0000 7f5e9aa00640 1 -- 172.21.15.86:0/1831958072 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2220019426 0 0) 0x7f5e940014a0 con 0x7f5e9c0d02d0 2024-04-25T14:18:10.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.154+0000 7f5e93e00640 1 --2- 172.21.15.86:0/1831958072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e9c0ae4b0 0x7f5e9c0ae8c0 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0b5370 tx=0x7f5e8401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe26dd715a96927 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:10.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.154+0000 7f5e9aa00640 1 -- 172.21.15.86:0/1831958072 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e840ab610 con 0x7f5e9c0ae4b0 2024-04-25T14:18:10.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.154+0000 7f5e9aa00640 1 -- 172.21.15.86:0/1831958072 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e840b0070 con 0x7f5e9c0ae4b0 2024-04-25T14:18:10.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.154+0000 7f5e9aa00640 1 -- 172.21.15.86:0/1831958072 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e840ab9d0 con 0x7f5e9c0ae4b0 2024-04-25T14:18:10.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.154+0000 7f5ea1000640 1 -- 172.21.15.86:0/1831958072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e9c0ae4b0 msgr2=0x7f5e9c0ae8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:10.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.154+0000 7f5ea1000640 1 --2- 172.21.15.86:0/1831958072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e9c0ae4b0 0x7f5e9c0ae8c0 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0b5370 tx=0x7f5e8401c520 comp rx=0 tx=0).stop 2024-04-25T14:18:10.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.154+0000 7f5ea1000640 1 -- 172.21.15.86:0/1831958072 shutdown_connections 2024-04-25T14:18:10.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.154+0000 7f5ea1000640 1 --2- 172.21.15.86:0/1831958072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e9c0ae4b0 0x7f5e9c0ae8c0 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:10.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.154+0000 7f5ea1000640 1 --2- 172.21.15.86:0/1831958072 >> v2:172.21.15.155:3300/0 conn(0x7f5e9c0749b0 0x7f5e9c06c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:10.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.154+0000 7f5ea1000640 1 -- 172.21.15.86:0/1831958072 >> 172.21.15.86:0/1831958072 conn(0x7f5e9c0b2fc0 msgr2=0x7f5e9c0b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:10.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.154+0000 7f5ea1000640 1 -- 172.21.15.86:0/1831958072 shutdown_connections 2024-04-25T14:18:10.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.154+0000 7f5ea1000640 1 -- 172.21.15.86:0/1831958072 wait complete. 2024-04-25T14:18:10.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.158+0000 7f5ea1000640 1 Processor -- start 2024-04-25T14:18:10.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.158+0000 7f5ea1000640 1 -- start start 2024-04-25T14:18:10.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.158+0000 7f5ea1000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e9c0749b0 0x7f5e9c153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:10.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.158+0000 7f5ea1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e9c0ae4b0 0x7f5e9c1576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:10.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.158+0000 7f5ea1000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e9c0d02d0 0x7f5e9c167a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:10.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.158+0000 7f5ea1000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9c06cd30 con 0x7f5e9c0ae4b0 2024-04-25T14:18:10.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.158+0000 7f5ea1000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9c06cbb0 con 0x7f5e9c0d02d0 2024-04-25T14:18:10.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.158+0000 7f5ea1000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5e9c073540 con 0x7f5e9c0749b0 2024-04-25T14:18:10.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.158+0000 7f5e93e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e9c0ae4b0 0x7f5e9c1576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:10.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.158+0000 7f5e93e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e9c0ae4b0 0x7f5e9c1576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46776/0 (socket says 172.21.15.86:46776) 2024-04-25T14:18:10.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.158+0000 7f5e93e00640 1 -- 172.21.15.86:0/3667300672 learned_addr learned my addr 172.21.15.86:0/3667300672 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:10.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.158+0000 7f5e9b400640 1 --2- 172.21.15.86:0/3667300672 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e9c0749b0 0x7f5e9c153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:10.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.158+0000 7f5e93e00640 1 -- 172.21.15.86:0/3667300672 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e9c0749b0 msgr2=0x7f5e9c153dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:10.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.158+0000 7f5e9be00640 1 --2- 172.21.15.86:0/3667300672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e9c0d02d0 0x7f5e9c167a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:10.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.158+0000 7f5e93e00640 1 --2- 172.21.15.86:0/3667300672 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e9c0749b0 0x7f5e9c153dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:10.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.158+0000 7f5e93e00640 1 -- 172.21.15.86:0/3667300672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e9c0d02d0 msgr2=0x7f5e9c167a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:10.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.158+0000 7f5e93e00640 1 --2- 172.21.15.86:0/3667300672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e9c0d02d0 0x7f5e9c167a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:10.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.158+0000 7f5e93e00640 1 -- 172.21.15.86:0/3667300672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e9c159db0 con 0x7f5e9c0ae4b0 2024-04-25T14:18:10.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.158+0000 7f5e9b400640 1 --2- 172.21.15.86:0/3667300672 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e9c0749b0 0x7f5e9c153dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:18:10.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.158+0000 7f5e9be00640 1 --2- 172.21.15.86:0/3667300672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e9c0d02d0 0x7f5e9c167a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:18:10.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.162+0000 7f5e93e00640 1 --2- 172.21.15.86:0/3667300672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e9c0ae4b0 0x7f5e9c1576d0 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7f5e84019130 tx=0x7f5e8401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:10.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.162+0000 7f5e93400640 1 -- 172.21.15.86:0/3667300672 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e840be5f0 con 0x7f5e9c0ae4b0 2024-04-25T14:18:10.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.162+0000 7f5e93400640 1 -- 172.21.15.86:0/3667300672 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e840be790 con 0x7f5e9c0ae4b0 2024-04-25T14:18:10.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.162+0000 7f5e93400640 1 -- 172.21.15.86:0/3667300672 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e840beb80 con 0x7f5e9c0ae4b0 2024-04-25T14:18:10.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.162+0000 7f5ea1000640 1 -- 172.21.15.86:0/3667300672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e9c15a090 con 0x7f5e9c0ae4b0 2024-04-25T14:18:10.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.162+0000 7f5ea1000640 1 -- 172.21.15.86:0/3667300672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e9c1fa1c0 con 0x7f5e9c0ae4b0 2024-04-25T14:18:10.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.162+0000 7f5ea1000640 1 -- 172.21.15.86:0/3667300672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5e9c076b80 con 0x7f5e9c0ae4b0 2024-04-25T14:18:10.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.162+0000 7f5e93400640 1 -- 172.21.15.86:0/3667300672 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7f5e840c9430 con 0x7f5e9c0ae4b0 2024-04-25T14:18:10.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.162+0000 7f5e93400640 1 --2- 172.21.15.86:0/3667300672 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f5e7806f430 0x7f5e78071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:10.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.162+0000 7f5e93400640 1 -- 172.21.15.86:0/3667300672 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 6023+0+0 (secure 0 0 0) 0x7f5e84129d90 con 0x7f5e9c0ae4b0 2024-04-25T14:18:10.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.162+0000 7f5e9b400640 1 --2- 172.21.15.86:0/3667300672 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f5e7806f430 0x7f5e78071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:10.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.162+0000 7f5e93400640 1 --2- 172.21.15.86:0/3667300672 >> [v2:172.21.15.86:6816/2851379769,v1:172.21.15.86:6817/2851379769] conn(0x7f5e78075f10 0x7f5e780783a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:10.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.162+0000 7f5e93400640 1 -- 172.21.15.86:0/3667300672 --> [v2:172.21.15.86:6816/2851379769,v1:172.21.15.86:6817/2851379769] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5e78078a70 con 0x7f5e78075f10 2024-04-25T14:18:10.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.162+0000 7f5e93400640 1 -- 172.21.15.86:0/3667300672 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=160) ==== 24+0+0 (secure 0 0 0) 0x7f5e8412a220 con 0x7f5e9c0ae4b0 2024-04-25T14:18:10.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.166+0000 7f5e9be00640 1 --2- 172.21.15.86:0/3667300672 >> [v2:172.21.15.86:6816/2851379769,v1:172.21.15.86:6817/2851379769] conn(0x7f5e78075f10 0x7f5e780783a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:10.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.166+0000 7f5e9b400640 1 --2- 172.21.15.86:0/3667300672 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f5e7806f430 0x7f5e78071940 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c009a40 tx=0x7f5e8c008040 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:10.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.166+0000 7f5e9be00640 1 --2- 172.21.15.86:0/3667300672 >> [v2:172.21.15.86:6816/2851379769,v1:172.21.15.86:6817/2851379769] conn(0x7f5e78075f10 0x7f5e780783a0 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:10.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.166+0000 7f5e93400640 1 -- 172.21.15.86:0/3667300672 <== osd.2 v2:172.21.15.86:6816/2851379769 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5e78078a70 con 0x7f5e78075f10 2024-04-25T14:18:10.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.178+0000 7f5ea1000640 1 -- 172.21.15.86:0/3667300672 --> [v2:172.21.15.86:6816/2851379769,v1:172.21.15.86:6817/2851379769] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5e9c158170 con 0x7f5e78075f10 2024-04-25T14:18:10.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.178+0000 7f5e93400640 1 -- 172.21.15.86:0/3667300672 <== osd.2 v2:172.21.15.86:6816/2851379769 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5e9c158170 con 0x7f5e78075f10 2024-04-25T14:18:10.174 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:18:10.174 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T14:18:10.174 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:18:10.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.178+0000 7f5ea1000640 1 -- 172.21.15.86:0/3667300672 >> [v2:172.21.15.86:6816/2851379769,v1:172.21.15.86:6817/2851379769] conn(0x7f5e78075f10 msgr2=0x7f5e780783a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:10.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.178+0000 7f5ea1000640 1 --2- 172.21.15.86:0/3667300672 >> [v2:172.21.15.86:6816/2851379769,v1:172.21.15.86:6817/2851379769] conn(0x7f5e78075f10 0x7f5e780783a0 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:10.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.178+0000 7f5ea1000640 1 -- 172.21.15.86:0/3667300672 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f5e7806f430 msgr2=0x7f5e78071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:10.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.178+0000 7f5ea1000640 1 --2- 172.21.15.86:0/3667300672 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f5e7806f430 0x7f5e78071940 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c009a40 tx=0x7f5e8c008040 comp rx=0 tx=0).stop 2024-04-25T14:18:10.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.178+0000 7f5ea1000640 1 -- 172.21.15.86:0/3667300672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e9c0ae4b0 msgr2=0x7f5e9c1576d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:10.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.178+0000 7f5ea1000640 1 --2- 172.21.15.86:0/3667300672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e9c0ae4b0 0x7f5e9c1576d0 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7f5e84019130 tx=0x7f5e8401ad60 comp rx=0 tx=0).stop 2024-04-25T14:18:10.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.178+0000 7f5e9be00640 1 -- 172.21.15.86:0/3667300672 reap_dead start 2024-04-25T14:18:10.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.178+0000 7f5ea1000640 1 -- 172.21.15.86:0/3667300672 shutdown_connections 2024-04-25T14:18:10.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.178+0000 7f5ea1000640 1 -- 172.21.15.86:0/3667300672 >> 172.21.15.86:0/3667300672 conn(0x7f5e9c0b2fc0 msgr2=0x7f5e9c06f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:10.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.178+0000 7f5ea1000640 1 -- 172.21.15.86:0/3667300672 shutdown_connections 2024-04-25T14:18:10.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.178+0000 7f5ea1000640 1 -- 172.21.15.86:0/3667300672 wait complete. 2024-04-25T14:18:10.192 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T14:18:10.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.346+0000 7fee5b000640 1 Processor -- start 2024-04-25T14:18:10.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.346+0000 7fee5b000640 1 -- start start 2024-04-25T14:18:10.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.346+0000 7fee5b000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fee54070330 0x7fee540d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:10.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.346+0000 7fee5b000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee540d2030 con 0x7fee540ce790 2024-04-25T14:18:10.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.346+0000 7fee5b000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee5415e5c0 con 0x7fee540d1c20 2024-04-25T14:18:10.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.346+0000 7fee5b000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fee5415e8a0 con 0x7fee54070330 2024-04-25T14:18:10.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.346+0000 7fee59000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fee540ce790 0x7fee5406f440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41852/0 (socket says 172.21.15.86:41852) 2024-04-25T14:18:10.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.346+0000 7fee59000640 1 -- 172.21.15.86:0/821761495 learned_addr learned my addr 172.21.15.86:0/821761495 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:18:10.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.346+0000 7fee53e00640 1 --2- 172.21.15.86:0/821761495 >> v2:172.21.15.155:3300/0 conn(0x7fee54070330 0x7fee540d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:10.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.346+0000 7fee53400640 1 -- 172.21.15.86:0/821761495 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 799291216 0 0) 0x7fee4c0014a0 con 0x7fee540ce790 2024-04-25T14:18:10.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.346+0000 7fee53400640 1 -- 172.21.15.86:0/821761495 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee38003530 con 0x7fee540ce790 2024-04-25T14:18:10.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.346+0000 7fee53e00640 1 -- 172.21.15.86:0/821761495 >> v1:172.21.15.155:6789/0 conn(0x7fee540d1c20 legacy=0x7fee5406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:10.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.346+0000 7fee53400640 1 -- 172.21.15.86:0/821761495 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2346432136 0 0) 0x7fee5415e5c0 con 0x7fee540d1c20 2024-04-25T14:18:10.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.346+0000 7fee53e00640 1 -- 172.21.15.86:0/821761495 >> v1:172.21.15.86:6789/0 conn(0x7fee540ce790 legacy=0x7fee5406f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:10.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.346+0000 7fee53e00640 1 -- 172.21.15.86:0/821761495 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5415eb80 con 0x7fee54070330 2024-04-25T14:18:10.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.350+0000 7fee53e00640 1 --2- 172.21.15.86:0/821761495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee54070330 0x7fee540d16e0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fee440097f0 tx=0x7fee4401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75b4a15374c61d9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:10.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.350+0000 7fee53400640 1 -- 172.21.15.86:0/821761495 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee440b0070 con 0x7fee54070330 2024-04-25T14:18:10.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.350+0000 7fee53400640 1 -- 172.21.15.86:0/821761495 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee440ab600 con 0x7fee54070330 2024-04-25T14:18:10.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.350+0000 7fee53400640 1 -- 172.21.15.86:0/821761495 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee440ab9b0 con 0x7fee54070330 2024-04-25T14:18:10.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.350+0000 7fee5b000640 1 -- 172.21.15.86:0/821761495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee54070330 msgr2=0x7fee540d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:10.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.350+0000 7fee5b000640 1 --2- 172.21.15.86:0/821761495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee54070330 0x7fee540d16e0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fee440097f0 tx=0x7fee4401c410 comp rx=0 tx=0).stop 2024-04-25T14:18:10.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.350+0000 7fee5b000640 1 -- 172.21.15.86:0/821761495 shutdown_connections 2024-04-25T14:18:10.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.350+0000 7fee5b000640 1 --2- 172.21.15.86:0/821761495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee54070330 0x7fee540d16e0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:10.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.350+0000 7fee5b000640 1 -- 172.21.15.86:0/821761495 >> 172.21.15.86:0/821761495 conn(0x7fee540b4650 msgr2=0x7fee540b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:10.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.350+0000 7fee5b000640 1 -- 172.21.15.86:0/821761495 shutdown_connections 2024-04-25T14:18:10.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.350+0000 7fee5b000640 1 -- 172.21.15.86:0/821761495 wait complete. 2024-04-25T14:18:10.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.354+0000 7fee5b000640 1 Processor -- start 2024-04-25T14:18:10.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.354+0000 7fee5b000640 1 -- start start 2024-04-25T14:18:10.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.354+0000 7fee5b000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee54070330 0x7fee541568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:10.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.354+0000 7fee5b000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee540ce790 0x7fee5409e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:10.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.354+0000 7fee5b000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee540d1c20 0x7fee5409e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:10.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.354+0000 7fee5b000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee540708c0 con 0x7fee540ce790 2024-04-25T14:18:10.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.354+0000 7fee5b000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fee54070740 con 0x7fee54070330 2024-04-25T14:18:10.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.354+0000 7fee5b000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fee5406fb50 con 0x7fee540d1c20 2024-04-25T14:18:10.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.354+0000 7fee4b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee540ce790 0x7fee5409e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:10.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.354+0000 7fee4b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee540ce790 0x7fee5409e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46778/0 (socket says 172.21.15.86:46778) 2024-04-25T14:18:10.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.354+0000 7fee4b400640 1 -- 172.21.15.86:0/376178452 learned_addr learned my addr 172.21.15.86:0/376178452 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:10.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.354+0000 7fee53e00640 1 --2- 172.21.15.86:0/376178452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee54070330 0x7fee541568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:10.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.354+0000 7fee59000640 1 --2- 172.21.15.86:0/376178452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee540d1c20 0x7fee5409e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:10.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.358+0000 7fee4b400640 1 -- 172.21.15.86:0/376178452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee540d1c20 msgr2=0x7fee5409e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:10.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.358+0000 7fee4b400640 1 --2- 172.21.15.86:0/376178452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee540d1c20 0x7fee5409e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:10.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.358+0000 7fee4b400640 1 -- 172.21.15.86:0/376178452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee54070330 msgr2=0x7fee541568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:10.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.358+0000 7fee4b400640 1 --2- 172.21.15.86:0/376178452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee54070330 0x7fee541568b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:10.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.358+0000 7fee4b400640 1 -- 172.21.15.86:0/376178452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5409dca0 con 0x7fee540ce790 2024-04-25T14:18:10.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.358+0000 7fee4b400640 1 --2- 172.21.15.86:0/376178452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee540ce790 0x7fee5409e430 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00c420 tx=0x7fee3c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:10.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.358+0000 7fee50c00640 1 -- 172.21.15.86:0/376178452 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c017070 con 0x7fee540ce790 2024-04-25T14:18:10.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.358+0000 7fee50c00640 1 -- 172.21.15.86:0/376178452 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee3c003600 con 0x7fee540ce790 2024-04-25T14:18:10.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.358+0000 7fee5b000640 1 -- 172.21.15.86:0/376178452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee540c4f40 con 0x7fee540ce790 2024-04-25T14:18:10.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.358+0000 7fee50c00640 1 -- 172.21.15.86:0/376178452 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c0039d0 con 0x7fee540ce790 2024-04-25T14:18:10.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.358+0000 7fee5b000640 1 -- 172.21.15.86:0/376178452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee5409df10 con 0x7fee540ce790 2024-04-25T14:18:10.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.358+0000 7fee5b000640 1 -- 172.21.15.86:0/376178452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee54069d00 con 0x7fee540ce790 2024-04-25T14:18:10.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.358+0000 7fee50c00640 1 -- 172.21.15.86:0/376178452 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7fee3c01a070 con 0x7fee540ce790 2024-04-25T14:18:10.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.358+0000 7fee50c00640 1 --2- 172.21.15.86:0/376178452 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fee3006f430 0x7fee30071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:10.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.358+0000 7fee50c00640 1 -- 172.21.15.86:0/376178452 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 6023+0+0 (secure 0 0 0) 0x7fee3c08e770 con 0x7fee540ce790 2024-04-25T14:18:10.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.362+0000 7fee53e00640 1 --2- 172.21.15.86:0/376178452 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fee3006f430 0x7fee30071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:10.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.362+0000 7fee53e00640 1 --2- 172.21.15.86:0/376178452 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fee3006f430 0x7fee30071940 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fee440ad000 tx=0x7fee440a6000 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:10.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.366+0000 7fee50c00640 1 -- 172.21.15.86:0/376178452 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fee3c05aed0 con 0x7fee540ce790 2024-04-25T14:18:10.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.522+0000 7fee5b000640 1 -- 172.21.15.86:0/376178452 --> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fee54074230 con 0x7fee3006f430 2024-04-25T14:18:10.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.522+0000 7fee50c00640 1 -- 172.21.15.86:0/376178452 <== mgr.11343 v2:172.21.15.86:6832/3174179029 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fee54074230 con 0x7fee3006f430 2024-04-25T14:18:10.515 INFO:teuthology.orchestra.run.smithi086.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T14:18:10.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.522+0000 7fee5b000640 1 -- 172.21.15.86:0/376178452 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fee3006f430 msgr2=0x7fee30071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:10.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.522+0000 7fee5b000640 1 --2- 172.21.15.86:0/376178452 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fee3006f430 0x7fee30071940 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fee440ad000 tx=0x7fee440a6000 comp rx=0 tx=0).stop 2024-04-25T14:18:10.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.522+0000 7fee5b000640 1 -- 172.21.15.86:0/376178452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee540ce790 msgr2=0x7fee5409e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:10.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.522+0000 7fee5b000640 1 --2- 172.21.15.86:0/376178452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee540ce790 0x7fee5409e430 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00c420 tx=0x7fee3c00c8e0 comp rx=0 tx=0).stop 2024-04-25T14:18:10.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.522+0000 7fee5b000640 1 -- 172.21.15.86:0/376178452 shutdown_connections 2024-04-25T14:18:10.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.522+0000 7fee5b000640 1 --2- 172.21.15.86:0/376178452 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fee3006f430 0x7fee30071940 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:10.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.522+0000 7fee5b000640 1 --2- 172.21.15.86:0/376178452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee540d1c20 0x7fee5409e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:10.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.522+0000 7fee5b000640 1 --2- 172.21.15.86:0/376178452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee540ce790 0x7fee5409e430 unknown :-1 s=CLOSED pgs=2502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:10.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.522+0000 7fee5b000640 1 --2- 172.21.15.86:0/376178452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee54070330 0x7fee541568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:10.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.522+0000 7fee5b000640 1 -- 172.21.15.86:0/376178452 >> 172.21.15.86:0/376178452 conn(0x7fee540b4650 msgr2=0x7fee54072f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:10.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.522+0000 7fee5b000640 1 -- 172.21.15.86:0/376178452 shutdown_connections 2024-04-25T14:18:10.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.522+0000 7fee5b000640 1 -- 172.21.15.86:0/376178452 wait complete. 2024-04-25T14:18:10.536 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T14:18:10.537 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T14:18:10.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.690+0000 7f6bcca00640 1 Processor -- start 2024-04-25T14:18:10.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.694+0000 7f6bcca00640 1 -- start start 2024-04-25T14:18:10.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.694+0000 7f6bcca00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6bc806dd90 0x7f6bc8160020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:10.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.694+0000 7f6bcca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6bc80b5160 0x7f6bc80b5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:10.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.694+0000 7f6bcca00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bc80705b0 con 0x7f6bc815fc10 2024-04-25T14:18:10.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.694+0000 7f6bcca00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6bc81646d0 con 0x7f6bc80b5160 2024-04-25T14:18:10.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.694+0000 7f6bcca00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6bc8163be0 con 0x7f6bc806dd90 2024-04-25T14:18:10.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.694+0000 7f6bc5e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6bc815fc10 0x7f6bc806d680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41856/0 (socket says 172.21.15.86:41856) 2024-04-25T14:18:10.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.694+0000 7f6bc5e00640 1 -- 172.21.15.86:0/2619990799 learned_addr learned my addr 172.21.15.86:0/2619990799 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:18:10.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.694+0000 7f6bc6800640 1 --2- 172.21.15.86:0/2619990799 >> v2:172.21.15.86:3300/0 conn(0x7f6bc80b5160 0x7f6bc80b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:10.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.694+0000 7f6bc5400640 1 --2- 172.21.15.86:0/2619990799 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6bc806dd90 0x7f6bc8160020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:10.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.694+0000 7f6bc6800640 1 -- 172.21.15.86:0/2619990799 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6bc806dd90 msgr2=0x7f6bc8160020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:10.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.694+0000 7f6bc6800640 1 --2- 172.21.15.86:0/2619990799 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6bc806dd90 0x7f6bc8160020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:10.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.694+0000 7f6bc6800640 1 -- 172.21.15.86:0/2619990799 >> v1:172.21.15.86:6789/0 conn(0x7f6bc815fc10 legacy=0x7f6bc806d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:10.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.694+0000 7f6bc6800640 1 -- 172.21.15.86:0/2619990799 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc8163e30 con 0x7f6bc80b5160 2024-04-25T14:18:10.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.694+0000 7f6bc4a00640 1 -- 172.21.15.86:0/2619990799 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2550977175 0 0) 0x7f6bc80705b0 con 0x7f6bc815fc10 2024-04-25T14:18:10.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.698+0000 7f6bc6800640 1 --2- 172.21.15.86:0/2619990799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bc80b5160 0x7f6bc80b5570 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0009d90 tx=0x7f6bc001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3a80242e7ed75b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:10.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.698+0000 7f6bc4a00640 1 -- 172.21.15.86:0/2619990799 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc00b2070 con 0x7f6bc80b5160 2024-04-25T14:18:10.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.698+0000 7f6bc4a00640 1 -- 172.21.15.86:0/2619990799 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bc00a5890 con 0x7f6bc80b5160 2024-04-25T14:18:10.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.698+0000 7f6bc4a00640 1 -- 172.21.15.86:0/2619990799 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc00a5c60 con 0x7f6bc80b5160 2024-04-25T14:18:10.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.698+0000 7f6bcca00640 1 -- 172.21.15.86:0/2619990799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bc80b5160 msgr2=0x7f6bc80b5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:10.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.698+0000 7f6bcca00640 1 --2- 172.21.15.86:0/2619990799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bc80b5160 0x7f6bc80b5570 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0009d90 tx=0x7f6bc001c8e0 comp rx=0 tx=0).stop 2024-04-25T14:18:10.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.698+0000 7f6bcca00640 1 -- 172.21.15.86:0/2619990799 shutdown_connections 2024-04-25T14:18:10.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.698+0000 7f6bcca00640 1 --2- 172.21.15.86:0/2619990799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bc80b5160 0x7f6bc80b5570 unknown :-1 s=CLOSED pgs=2504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:10.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.698+0000 7f6bcca00640 1 --2- 172.21.15.86:0/2619990799 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6bc806dd90 0x7f6bc8160020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:10.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.698+0000 7f6bcca00640 1 -- 172.21.15.86:0/2619990799 >> 172.21.15.86:0/2619990799 conn(0x7f6bc8073240 msgr2=0x7f6bc8073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:10.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.698+0000 7f6bcca00640 1 -- 172.21.15.86:0/2619990799 shutdown_connections 2024-04-25T14:18:10.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.698+0000 7f6bcca00640 1 -- 172.21.15.86:0/2619990799 wait complete. 2024-04-25T14:18:10.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.702+0000 7f6bcca00640 1 Processor -- start 2024-04-25T14:18:10.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.702+0000 7f6bcca00640 1 -- start start 2024-04-25T14:18:10.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.702+0000 7f6bcca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bc80b5160 0x7f6bc814b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:10.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.702+0000 7f6bcca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bc815fc10 0x7f6bc8162db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:10.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.702+0000 7f6bcca00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6bc80afbc0 0x7f6bc80affd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:10.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.702+0000 7f6bcca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc816cce0 con 0x7f6bc815fc10 2024-04-25T14:18:10.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.702+0000 7f6bcca00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc816cb60 con 0x7f6bc80b5160 2024-04-25T14:18:10.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.702+0000 7f6bcca00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6bc816ce60 con 0x7f6bc80afbc0 2024-04-25T14:18:10.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.702+0000 7f6bc5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bc815fc10 0x7f6bc8162db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:10.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.702+0000 7f6bc5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bc815fc10 0x7f6bc8162db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46786/0 (socket says 172.21.15.86:46786) 2024-04-25T14:18:10.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.702+0000 7f6bc5400640 1 -- 172.21.15.86:0/3934036512 learned_addr learned my addr 172.21.15.86:0/3934036512 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:10.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.702+0000 7f6bc5e00640 1 --2- 172.21.15.86:0/3934036512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bc80b5160 0x7f6bc814b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:10.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.702+0000 7f6bc6800640 1 --2- 172.21.15.86:0/3934036512 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6bc80afbc0 0x7f6bc80affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:10.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.702+0000 7f6bc5400640 1 -- 172.21.15.86:0/3934036512 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6bc80afbc0 msgr2=0x7f6bc80affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:10.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.702+0000 7f6bc5400640 1 --2- 172.21.15.86:0/3934036512 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6bc80afbc0 0x7f6bc80affd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:10.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.702+0000 7f6bc5400640 1 -- 172.21.15.86:0/3934036512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bc80b5160 msgr2=0x7f6bc814b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:10.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.702+0000 7f6bc5400640 1 --2- 172.21.15.86:0/3934036512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bc80b5160 0x7f6bc814b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:10.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.702+0000 7f6bc5400640 1 -- 172.21.15.86:0/3934036512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc80b0510 con 0x7f6bc815fc10 2024-04-25T14:18:10.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.702+0000 7f6bc6800640 1 --2- 172.21.15.86:0/3934036512 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6bc80afbc0 0x7f6bc80affd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:18:10.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.702+0000 7f6bc5e00640 1 --2- 172.21.15.86:0/3934036512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bc80b5160 0x7f6bc814b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:18:10.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.702+0000 7f6bc5400640 1 --2- 172.21.15.86:0/3934036512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bc815fc10 0x7f6bc8162db0 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7f6bb000e870 tx=0x7f6bb000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:10.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.702+0000 7f6bbe000640 1 -- 172.21.15.86:0/3934036512 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0017070 con 0x7f6bc815fc10 2024-04-25T14:18:10.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.702+0000 7f6bbe000640 1 -- 172.21.15.86:0/3934036512 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bb000c940 con 0x7f6bc815fc10 2024-04-25T14:18:10.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.702+0000 7f6bbe000640 1 -- 172.21.15.86:0/3934036512 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb000ccf0 con 0x7f6bc815fc10 2024-04-25T14:18:10.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.702+0000 7f6bcca00640 1 -- 172.21.15.86:0/3934036512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc815e9f0 con 0x7f6bc815fc10 2024-04-25T14:18:10.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.702+0000 7f6bcca00640 1 -- 172.21.15.86:0/3934036512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc815efb0 con 0x7f6bc815fc10 2024-04-25T14:18:10.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.706+0000 7f6bcca00640 1 -- 172.21.15.86:0/3934036512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6bc8141c30 con 0x7f6bc815fc10 2024-04-25T14:18:10.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.706+0000 7f6bbe000640 1 -- 172.21.15.86:0/3934036512 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7f6bb001a070 con 0x7f6bc815fc10 2024-04-25T14:18:10.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.706+0000 7f6bbe000640 1 --2- 172.21.15.86:0/3934036512 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f6ba406f400 0x7f6ba4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:10.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.706+0000 7f6bbe000640 1 -- 172.21.15.86:0/3934036512 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 6023+0+0 (secure 0 0 0) 0x7f6bb008f280 con 0x7f6bc815fc10 2024-04-25T14:18:10.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.706+0000 7f6bc5e00640 1 --2- 172.21.15.86:0/3934036512 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f6ba406f400 0x7f6ba4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:10.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.706+0000 7f6bbe000640 1 --2- 172.21.15.86:0/3934036512 >> [v2:172.21.15.86:6800/2257199129,v1:172.21.15.86:6801/2257199129] conn(0x7f6ba4075ee0 0x7f6ba4078370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:10.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.706+0000 7f6bbe000640 1 -- 172.21.15.86:0/3934036512 --> [v2:172.21.15.86:6800/2257199129,v1:172.21.15.86:6801/2257199129] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6bb001ae70 con 0x7f6ba4075ee0 2024-04-25T14:18:10.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.706+0000 7f6bbe000640 1 -- 172.21.15.86:0/3934036512 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=160) ==== 24+0+0 (secure 0 0 0) 0x7f6bb0096050 con 0x7f6bc815fc10 2024-04-25T14:18:10.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.706+0000 7f6bc6800640 1 --2- 172.21.15.86:0/3934036512 >> [v2:172.21.15.86:6800/2257199129,v1:172.21.15.86:6801/2257199129] conn(0x7f6ba4075ee0 0x7f6ba4078370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:10.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.706+0000 7f6bc5e00640 1 --2- 172.21.15.86:0/3934036512 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f6ba406f400 0x7f6ba4071910 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8007fd0 tx=0x7f6bb8009040 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:10.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.706+0000 7f6bc6800640 1 --2- 172.21.15.86:0/3934036512 >> [v2:172.21.15.86:6800/2257199129,v1:172.21.15.86:6801/2257199129] conn(0x7f6ba4075ee0 0x7f6ba4078370 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:10.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.710+0000 7f6bbe000640 1 -- 172.21.15.86:0/3934036512 <== osd.3 v2:172.21.15.86:6800/2257199129 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6bb001ae70 con 0x7f6ba4075ee0 2024-04-25T14:18:10.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.722+0000 7f6bcca00640 1 -- 172.21.15.86:0/3934036512 --> [v2:172.21.15.86:6800/2257199129,v1:172.21.15.86:6801/2257199129] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6bc815e7a0 con 0x7f6ba4075ee0 2024-04-25T14:18:10.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.722+0000 7f6bbe000640 1 -- 172.21.15.86:0/3934036512 <== osd.3 v2:172.21.15.86:6800/2257199129 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6bc815e7a0 con 0x7f6ba4075ee0 2024-04-25T14:18:10.716 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:18:10.717 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T14:18:10.717 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:18:10.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.722+0000 7f6bcca00640 1 -- 172.21.15.86:0/3934036512 >> [v2:172.21.15.86:6800/2257199129,v1:172.21.15.86:6801/2257199129] conn(0x7f6ba4075ee0 msgr2=0x7f6ba4078370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:10.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.722+0000 7f6bcca00640 1 --2- 172.21.15.86:0/3934036512 >> [v2:172.21.15.86:6800/2257199129,v1:172.21.15.86:6801/2257199129] conn(0x7f6ba4075ee0 0x7f6ba4078370 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:10.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.722+0000 7f6bcca00640 1 -- 172.21.15.86:0/3934036512 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f6ba406f400 msgr2=0x7f6ba4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:10.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.722+0000 7f6bcca00640 1 --2- 172.21.15.86:0/3934036512 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f6ba406f400 0x7f6ba4071910 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8007fd0 tx=0x7f6bb8009040 comp rx=0 tx=0).stop 2024-04-25T14:18:10.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.722+0000 7f6bcca00640 1 -- 172.21.15.86:0/3934036512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bc815fc10 msgr2=0x7f6bc8162db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:10.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.722+0000 7f6bcca00640 1 --2- 172.21.15.86:0/3934036512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bc815fc10 0x7f6bc8162db0 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7f6bb000e870 tx=0x7f6bb000ec70 comp rx=0 tx=0).stop 2024-04-25T14:18:10.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.722+0000 7f6bc6800640 1 -- 172.21.15.86:0/3934036512 reap_dead start 2024-04-25T14:18:10.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.722+0000 7f6bcca00640 1 -- 172.21.15.86:0/3934036512 shutdown_connections 2024-04-25T14:18:10.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.722+0000 7f6bcca00640 1 -- 172.21.15.86:0/3934036512 >> 172.21.15.86:0/3934036512 conn(0x7f6bc8073240 msgr2=0x7f6bc8159520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:10.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.722+0000 7f6bcca00640 1 -- 172.21.15.86:0/3934036512 shutdown_connections 2024-04-25T14:18:10.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.722+0000 7f6bcca00640 1 -- 172.21.15.86:0/3934036512 wait complete. 2024-04-25T14:18:10.735 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T14:18:10.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.886+0000 7fd960e00640 1 Processor -- start 2024-04-25T14:18:10.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.890+0000 7fd960e00640 1 -- start start 2024-04-25T14:18:10.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.890+0000 7fd960e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd95c0cdd70 0x7fd95c0ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:10.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.890+0000 7fd960e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd95c06e700 0x7fd95c06eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:10.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.890+0000 7fd960e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd95c06f050 con 0x7fd95c070e20 2024-04-25T14:18:10.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.890+0000 7fd960e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd95c06d940 con 0x7fd95c06e700 2024-04-25T14:18:10.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.890+0000 7fd960e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd95c06dc20 con 0x7fd95c0cdd70 2024-04-25T14:18:10.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.890+0000 7fd95a800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd95c06e700 0x7fd95c06eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:10.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.890+0000 7fd95a800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd95c06e700 0x7fd95c06eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46800/0 (socket says 172.21.15.86:46800) 2024-04-25T14:18:10.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.890+0000 7fd95a800640 1 -- 172.21.15.86:0/374128540 learned_addr learned my addr 172.21.15.86:0/374128540 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:10.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.890+0000 7fd959e00640 1 --2- 172.21.15.86:0/374128540 >> v2:172.21.15.155:3300/0 conn(0x7fd95c0cdd70 0x7fd95c0ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:10.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.890+0000 7fd95a800640 1 -- 172.21.15.86:0/374128540 >> v2:172.21.15.155:3300/0 conn(0x7fd95c0cdd70 msgr2=0x7fd95c0ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:10.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.890+0000 7fd95a800640 1 --2- 172.21.15.86:0/374128540 >> v2:172.21.15.155:3300/0 conn(0x7fd95c0cdd70 0x7fd95c0ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:10.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.890+0000 7fd95a800640 1 -- 172.21.15.86:0/374128540 >> v1:172.21.15.155:6789/0 conn(0x7fd95c070e20 legacy=0x7fd95c071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:10.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.890+0000 7fd95a800640 1 -- 172.21.15.86:0/374128540 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd95c06df00 con 0x7fd95c06e700 2024-04-25T14:18:10.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.894+0000 7fd95a800640 1 --2- 172.21.15.86:0/374128540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd95c06e700 0x7fd95c06eb10 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7fd95c074610 tx=0x7fd95401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f5af9a1be45ff8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:10.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.894+0000 7fd958a00640 1 -- 172.21.15.86:0/374128540 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9540a5630 con 0x7fd95c06e700 2024-04-25T14:18:10.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.894+0000 7fd958a00640 1 -- 172.21.15.86:0/374128540 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9540a57d0 con 0x7fd95c06e700 2024-04-25T14:18:10.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.894+0000 7fd958a00640 1 -- 172.21.15.86:0/374128540 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9540a5ba0 con 0x7fd95c06e700 2024-04-25T14:18:10.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.894+0000 7fd960e00640 1 -- 172.21.15.86:0/374128540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd95c06e700 msgr2=0x7fd95c06eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:10.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.894+0000 7fd960e00640 1 --2- 172.21.15.86:0/374128540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd95c06e700 0x7fd95c06eb10 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7fd95c074610 tx=0x7fd95401c920 comp rx=0 tx=0).stop 2024-04-25T14:18:10.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.894+0000 7fd960e00640 1 -- 172.21.15.86:0/374128540 shutdown_connections 2024-04-25T14:18:10.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.894+0000 7fd960e00640 1 --2- 172.21.15.86:0/374128540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd95c06e700 0x7fd95c06eb10 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.894+0000 7fd960e00640 1 --2- 172.21.15.86:0/374128540 >> v2:172.21.15.155:3300/0 conn(0x7fd95c0cdd70 0x7fd95c0ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.894+0000 7fd960e00640 1 -- 172.21.15.86:0/374128540 >> 172.21.15.86:0/374128540 conn(0x7fd95c0cffc0 msgr2=0x7fd95c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.894+0000 7fd960e00640 1 -- 172.21.15.86:0/374128540 shutdown_connections 2024-04-25T14:18:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.894+0000 7fd960e00640 1 -- 172.21.15.86:0/374128540 wait complete. 2024-04-25T14:18:10.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.898+0000 7fd960e00640 1 Processor -- start 2024-04-25T14:18:10.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.898+0000 7fd960e00640 1 -- start start 2024-04-25T14:18:10.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.898+0000 7fd960e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd95c06e700 0x7fd95c160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:10.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.898+0000 7fd960e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd95c070e20 0x7fd95c162540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.898+0000 7fd960e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd95c0cdd70 0x7fd95c1fd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.898+0000 7fd960e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd95c16cc00 con 0x7fd95c06e700 2024-04-25T14:18:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.898+0000 7fd960e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd95c1643c0 con 0x7fd95c0cdd70 2024-04-25T14:18:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.898+0000 7fd960e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd95c16cd80 con 0x7fd95c070e20 2024-04-25T14:18:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.898+0000 7fd959e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd95c06e700 0x7fd95c160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.898+0000 7fd959e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd95c06e700 0x7fd95c160cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46814/0 (socket says 172.21.15.86:46814) 2024-04-25T14:18:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.898+0000 7fd959e00640 1 -- 172.21.15.86:0/1020968375 learned_addr learned my addr 172.21.15.86:0/1020968375 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.898+0000 7fd95a800640 1 --2- 172.21.15.86:0/1020968375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd95c0cdd70 0x7fd95c1fd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.898+0000 7fd959400640 1 --2- 172.21.15.86:0/1020968375 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd95c070e20 0x7fd95c162540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.898+0000 7fd959e00640 1 -- 172.21.15.86:0/1020968375 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd95c070e20 msgr2=0x7fd95c162540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.898+0000 7fd959e00640 1 --2- 172.21.15.86:0/1020968375 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd95c070e20 0x7fd95c162540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.898+0000 7fd959e00640 1 -- 172.21.15.86:0/1020968375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd95c0cdd70 msgr2=0x7fd95c1fd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.898+0000 7fd959e00640 1 --2- 172.21.15.86:0/1020968375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd95c0cdd70 0x7fd95c1fd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.898+0000 7fd959e00640 1 -- 172.21.15.86:0/1020968375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd95c161540 con 0x7fd95c06e700 2024-04-25T14:18:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.898+0000 7fd959e00640 1 --2- 172.21.15.86:0/1020968375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd95c06e700 0x7fd95c160cf0 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7fd94c00d620 tx=0x7fd94c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.898+0000 7fd952000640 1 -- 172.21.15.86:0/1020968375 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94c00b620 con 0x7fd95c06e700 2024-04-25T14:18:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.898+0000 7fd952000640 1 -- 172.21.15.86:0/1020968375 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd94c019070 con 0x7fd95c06e700 2024-04-25T14:18:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.898+0000 7fd960e00640 1 -- 172.21.15.86:0/1020968375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd95c161820 con 0x7fd95c06e700 2024-04-25T14:18:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.898+0000 7fd952000640 1 -- 172.21.15.86:0/1020968375 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94c00b9e0 con 0x7fd95c06e700 2024-04-25T14:18:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.898+0000 7fd960e00640 1 -- 172.21.15.86:0/1020968375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd95c152da0 con 0x7fd95c06e700 2024-04-25T14:18:10.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.902+0000 7fd960e00640 1 -- 172.21.15.86:0/1020968375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd95c0720e0 con 0x7fd95c06e700 2024-04-25T14:18:10.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.902+0000 7fd952000640 1 -- 172.21.15.86:0/1020968375 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7fd94c00bb80 con 0x7fd95c06e700 2024-04-25T14:18:10.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.902+0000 7fd952000640 1 --2- 172.21.15.86:0/1020968375 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fd93c06f430 0x7fd93c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:10.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.902+0000 7fd952000640 1 -- 172.21.15.86:0/1020968375 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 6023+0+0 (secure 0 0 0) 0x7fd94c08f780 con 0x7fd95c06e700 2024-04-25T14:18:10.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.902+0000 7fd959400640 1 --2- 172.21.15.86:0/1020968375 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fd93c06f430 0x7fd93c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:10.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.902+0000 7fd959400640 1 --2- 172.21.15.86:0/1020968375 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fd93c06f430 0x7fd93c071940 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd948006fd0 tx=0x7fd948008040 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:10.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:10.906+0000 7fd952000640 1 -- 172.21.15.86:0/1020968375 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd94c05c5c0 con 0x7fd95c06e700 2024-04-25T14:18:11.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.062+0000 7fd960e00640 1 -- 172.21.15.86:0/1020968375 --> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fd95c1612f0 con 0x7fd93c06f430 2024-04-25T14:18:11.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.062+0000 7fd952000640 1 -- 172.21.15.86:0/1020968375 <== mgr.11343 v2:172.21.15.86:6832/3174179029 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd95c1612f0 con 0x7fd93c06f430 2024-04-25T14:18:11.058 INFO:teuthology.orchestra.run.smithi086.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T14:18:11.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.066+0000 7fd960e00640 1 -- 172.21.15.86:0/1020968375 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fd93c06f430 msgr2=0x7fd93c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.066+0000 7fd960e00640 1 --2- 172.21.15.86:0/1020968375 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fd93c06f430 0x7fd93c071940 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd948006fd0 tx=0x7fd948008040 comp rx=0 tx=0).stop 2024-04-25T14:18:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.066+0000 7fd960e00640 1 -- 172.21.15.86:0/1020968375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd95c06e700 msgr2=0x7fd95c160cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.066+0000 7fd960e00640 1 --2- 172.21.15.86:0/1020968375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd95c06e700 0x7fd95c160cf0 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7fd94c00d620 tx=0x7fd94c00dae0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.066+0000 7fd960e00640 1 -- 172.21.15.86:0/1020968375 shutdown_connections 2024-04-25T14:18:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.066+0000 7fd960e00640 1 --2- 172.21.15.86:0/1020968375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd95c0cdd70 0x7fd95c1fd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.066+0000 7fd960e00640 1 --2- 172.21.15.86:0/1020968375 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fd93c06f430 0x7fd93c071940 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.066+0000 7fd960e00640 1 --2- 172.21.15.86:0/1020968375 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd95c070e20 0x7fd95c162540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.066+0000 7fd960e00640 1 --2- 172.21.15.86:0/1020968375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd95c06e700 0x7fd95c160cf0 unknown :-1 s=CLOSED pgs=2507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.066+0000 7fd960e00640 1 -- 172.21.15.86:0/1020968375 >> 172.21.15.86:0/1020968375 conn(0x7fd95c0cffc0 msgr2=0x7fd95c0d1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:11.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.066+0000 7fd960e00640 1 -- 172.21.15.86:0/1020968375 shutdown_connections 2024-04-25T14:18:11.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.066+0000 7fd960e00640 1 -- 172.21.15.86:0/1020968375 wait complete. 2024-04-25T14:18:11.079 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T14:18:11.079 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T14:18:11.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.238+0000 7f3781c00640 1 Processor -- start 2024-04-25T14:18:11.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.238+0000 7f3781c00640 1 -- start start 2024-04-25T14:18:11.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.238+0000 7f3781c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f377c0701f0 0x7f377c0729f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:11.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.238+0000 7f3781c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f377c070600 0x7f377c072f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:11.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.238+0000 7f3781c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f377c06f430 con 0x7f377c06c670 2024-04-25T14:18:11.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.238+0000 7f3781c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f377c06f1e0 con 0x7f377c070600 2024-04-25T14:18:11.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.238+0000 7f3781c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f377c06f710 con 0x7f377c0701f0 2024-04-25T14:18:11.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.238+0000 7f3772000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f377c070600 0x7f377c072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:11.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.238+0000 7f3772000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f377c070600 0x7f377c072f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46816/0 (socket says 172.21.15.86:46816) 2024-04-25T14:18:11.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.238+0000 7f3772000640 1 -- 172.21.15.86:0/1534413053 learned_addr learned my addr 172.21.15.86:0/1534413053 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:11.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.238+0000 7f377aa00640 1 --2- 172.21.15.86:0/1534413053 >> v2:172.21.15.155:3300/0 conn(0x7f377c0701f0 0x7f377c0729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:11.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.238+0000 7f3772000640 1 -- 172.21.15.86:0/1534413053 >> v2:172.21.15.155:3300/0 conn(0x7f377c0701f0 msgr2=0x7f377c0729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:11.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.238+0000 7f3772000640 1 --2- 172.21.15.86:0/1534413053 >> v2:172.21.15.155:3300/0 conn(0x7f377c0701f0 0x7f377c0729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.238+0000 7f3772000640 1 -- 172.21.15.86:0/1534413053 >> v1:172.21.15.155:6789/0 conn(0x7f377c06c670 legacy=0x7f377c06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:11.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.238+0000 7f3772000640 1 -- 172.21.15.86:0/1534413053 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377c06f9f0 con 0x7f377c070600 2024-04-25T14:18:11.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.242+0000 7f3772000640 1 --2- 172.21.15.86:0/1534413053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f377c070600 0x7f377c072f30 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7f377c06f360 tx=0x7f376401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833ba8d4f6a61eba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:11.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.242+0000 7f377a000640 1 -- 172.21.15.86:0/1534413053 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37640b0070 con 0x7f377c070600 2024-04-25T14:18:11.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.242+0000 7f377a000640 1 -- 172.21.15.86:0/1534413053 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37640ab600 con 0x7f377c070600 2024-04-25T14:18:11.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.242+0000 7f377a000640 1 -- 172.21.15.86:0/1534413053 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37640ab9d0 con 0x7f377c070600 2024-04-25T14:18:11.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.242+0000 7f3781c00640 1 -- 172.21.15.86:0/1534413053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f377c070600 msgr2=0x7f377c072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:11.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.242+0000 7f3781c00640 1 --2- 172.21.15.86:0/1534413053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f377c070600 0x7f377c072f30 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7f377c06f360 tx=0x7f376401c490 comp rx=0 tx=0).stop 2024-04-25T14:18:11.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.242+0000 7f3781c00640 1 -- 172.21.15.86:0/1534413053 shutdown_connections 2024-04-25T14:18:11.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.242+0000 7f3781c00640 1 --2- 172.21.15.86:0/1534413053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f377c070600 0x7f377c072f30 unknown :-1 s=CLOSED pgs=2508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.242+0000 7f3781c00640 1 --2- 172.21.15.86:0/1534413053 >> v2:172.21.15.155:3300/0 conn(0x7f377c0701f0 0x7f377c0729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.242+0000 7f3781c00640 1 -- 172.21.15.86:0/1534413053 >> 172.21.15.86:0/1534413053 conn(0x7f377c0b4a40 msgr2=0x7f377c0b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:11.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.242+0000 7f3781c00640 1 -- 172.21.15.86:0/1534413053 shutdown_connections 2024-04-25T14:18:11.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.242+0000 7f3781c00640 1 -- 172.21.15.86:0/1534413053 wait complete. 2024-04-25T14:18:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.246+0000 7f3781c00640 1 Processor -- start 2024-04-25T14:18:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.246+0000 7f3781c00640 1 -- start start 2024-04-25T14:18:11.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.246+0000 7f3781c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f377c06c670 0x7f377c149790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:11.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.246+0000 7f3781c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f377c0701f0 0x7f377c157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:11.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.246+0000 7f3781c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f377c070600 0x7f377c157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:11.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.246+0000 7f3781c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f377c16b250 con 0x7f377c06c670 2024-04-25T14:18:11.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.246+0000 7f3781c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f377c15d5c0 con 0x7f377c0701f0 2024-04-25T14:18:11.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.246+0000 7f3781c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f377c16b3d0 con 0x7f377c070600 2024-04-25T14:18:11.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.246+0000 7f377aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f377c06c670 0x7f377c149790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:11.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.246+0000 7f377aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f377c06c670 0x7f377c149790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46818/0 (socket says 172.21.15.86:46818) 2024-04-25T14:18:11.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.246+0000 7f377aa00640 1 -- 172.21.15.86:0/4253458708 learned_addr learned my addr 172.21.15.86:0/4253458708 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:11.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.246+0000 7f3772000640 1 --2- 172.21.15.86:0/4253458708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f377c0701f0 0x7f377c157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:11.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.246+0000 7f377b400640 1 --2- 172.21.15.86:0/4253458708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f377c070600 0x7f377c157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:11.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.246+0000 7f377aa00640 1 -- 172.21.15.86:0/4253458708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f377c070600 msgr2=0x7f377c157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:11.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.246+0000 7f377aa00640 1 --2- 172.21.15.86:0/4253458708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f377c070600 0x7f377c157790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.246+0000 7f377aa00640 1 -- 172.21.15.86:0/4253458708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f377c0701f0 msgr2=0x7f377c157250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:11.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.246+0000 7f377aa00640 1 --2- 172.21.15.86:0/4253458708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f377c0701f0 0x7f377c157250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.246+0000 7f377aa00640 1 -- 172.21.15.86:0/4253458708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377c15da10 con 0x7f377c06c670 2024-04-25T14:18:11.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.246+0000 7f377aa00640 1 --2- 172.21.15.86:0/4253458708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f377c06c670 0x7f377c149790 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7f376c00e8a0 tx=0x7f376c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:11.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.246+0000 7f3773400640 1 -- 172.21.15.86:0/4253458708 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c017070 con 0x7f377c06c670 2024-04-25T14:18:11.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.246+0000 7f3773400640 1 -- 172.21.15.86:0/4253458708 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f376c00c940 con 0x7f377c06c670 2024-04-25T14:18:11.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.246+0000 7f3781c00640 1 -- 172.21.15.86:0/4253458708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f377c15dcf0 con 0x7f377c06c670 2024-04-25T14:18:11.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.246+0000 7f3773400640 1 -- 172.21.15.86:0/4253458708 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c00ccf0 con 0x7f377c06c670 2024-04-25T14:18:11.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.250+0000 7f3781c00640 1 -- 172.21.15.86:0/4253458708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f377c15e2b0 con 0x7f377c06c670 2024-04-25T14:18:11.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.250+0000 7f3781c00640 1 -- 172.21.15.86:0/4253458708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f377c050750 con 0x7f377c06c670 2024-04-25T14:18:11.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.250+0000 7f3773400640 1 -- 172.21.15.86:0/4253458708 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7f376c01a070 con 0x7f377c06c670 2024-04-25T14:18:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.250+0000 7f3773400640 1 --2- 172.21.15.86:0/4253458708 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f375806f430 0x7f3758071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.250+0000 7f3773400640 1 -- 172.21.15.86:0/4253458708 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 6023+0+0 (secure 0 0 0) 0x7f376c08f5d0 con 0x7f377c06c670 2024-04-25T14:18:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.250+0000 7f3772000640 1 --2- 172.21.15.86:0/4253458708 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f375806f430 0x7f3758071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.250+0000 7f3773400640 1 --2- 172.21.15.86:0/4253458708 >> [v2:172.21.15.155:6800/1655306076,v1:172.21.15.155:6802/1655306076] conn(0x7f3758075f10 0x7f37580783a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.250+0000 7f3773400640 1 -- 172.21.15.86:0/4253458708 --> [v2:172.21.15.155:6800/1655306076,v1:172.21.15.155:6802/1655306076] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f376c00ce90 con 0x7f3758075f10 2024-04-25T14:18:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.250+0000 7f3773400640 1 -- 172.21.15.86:0/4253458708 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=160) ==== 24+0+0 (secure 0 0 0) 0x7f376c096050 con 0x7f377c06c670 2024-04-25T14:18:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.250+0000 7f377b400640 1 --2- 172.21.15.86:0/4253458708 >> [v2:172.21.15.155:6800/1655306076,v1:172.21.15.155:6802/1655306076] conn(0x7f3758075f10 0x7f37580783a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:11.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.250+0000 7f3772000640 1 --2- 172.21.15.86:0/4253458708 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f375806f430 0x7f3758071940 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f37640ad000 tx=0x7f3764008040 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:11.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.254+0000 7f377b400640 1 --2- 172.21.15.86:0/4253458708 >> [v2:172.21.15.155:6800/1655306076,v1:172.21.15.155:6802/1655306076] conn(0x7f3758075f10 0x7f37580783a0 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:11.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.254+0000 7f3773400640 1 -- 172.21.15.86:0/4253458708 <== osd.4 v2:172.21.15.155:6800/1655306076 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f376c00ce90 con 0x7f3758075f10 2024-04-25T14:18:11.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.266+0000 7f3781c00640 1 -- 172.21.15.86:0/4253458708 --> [v2:172.21.15.155:6800/1655306076,v1:172.21.15.155:6802/1655306076] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f377c072ca0 con 0x7f3758075f10 2024-04-25T14:18:11.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.266+0000 7f3773400640 1 -- 172.21.15.86:0/4253458708 <== osd.4 v2:172.21.15.155:6800/1655306076 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f377c072ca0 con 0x7f3758075f10 2024-04-25T14:18:11.261 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:18:11.262 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T14:18:11.262 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:18:11.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.266+0000 7f3781c00640 1 -- 172.21.15.86:0/4253458708 >> [v2:172.21.15.155:6800/1655306076,v1:172.21.15.155:6802/1655306076] conn(0x7f3758075f10 msgr2=0x7f37580783a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:11.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.266+0000 7f3781c00640 1 --2- 172.21.15.86:0/4253458708 >> [v2:172.21.15.155:6800/1655306076,v1:172.21.15.155:6802/1655306076] conn(0x7f3758075f10 0x7f37580783a0 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.266+0000 7f3781c00640 1 -- 172.21.15.86:0/4253458708 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f375806f430 msgr2=0x7f3758071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:11.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.266+0000 7f3781c00640 1 --2- 172.21.15.86:0/4253458708 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f375806f430 0x7f3758071940 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f37640ad000 tx=0x7f3764008040 comp rx=0 tx=0).stop 2024-04-25T14:18:11.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.266+0000 7f3781c00640 1 -- 172.21.15.86:0/4253458708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f377c06c670 msgr2=0x7f377c149790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:11.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.266+0000 7f3781c00640 1 --2- 172.21.15.86:0/4253458708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f377c06c670 0x7f377c149790 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7f376c00e8a0 tx=0x7f376c00eca0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.266+0000 7f377b400640 1 -- 172.21.15.86:0/4253458708 reap_dead start 2024-04-25T14:18:11.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.266+0000 7f3781c00640 1 -- 172.21.15.86:0/4253458708 shutdown_connections 2024-04-25T14:18:11.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.266+0000 7f3781c00640 1 -- 172.21.15.86:0/4253458708 >> 172.21.15.86:0/4253458708 conn(0x7f377c0b4a40 msgr2=0x7f377c0aef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:11.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.266+0000 7f3781c00640 1 -- 172.21.15.86:0/4253458708 shutdown_connections 2024-04-25T14:18:11.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.266+0000 7f3781c00640 1 -- 172.21.15.86:0/4253458708 wait complete. 2024-04-25T14:18:11.280 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T14:18:11.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.434+0000 7fafc5a00640 1 Processor -- start 2024-04-25T14:18:11.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.434+0000 7fafc5a00640 1 -- start start 2024-04-25T14:18:11.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.434+0000 7fafc5a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fafc01615c0 0x7fafc01619d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:11.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.434+0000 7fafc5a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafc00b6530 0x7fafc00b6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:11.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.434+0000 7fafc5a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafc01622f0 con 0x7fafc006f990 2024-04-25T14:18:11.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.434+0000 7fafc5a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fafc00d1730 con 0x7fafc01615c0 2024-04-25T14:18:11.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.434+0000 7fafc5a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fafc00d1a10 con 0x7fafc00b6530 2024-04-25T14:18:11.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.434+0000 7fafbf400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fafc01615c0 0x7fafc01619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:11.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.434+0000 7fafbf400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fafc01615c0 0x7fafc01619d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46824/0 (socket says 172.21.15.86:46824) 2024-04-25T14:18:11.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.434+0000 7fafbf400640 1 -- 172.21.15.86:0/3709783925 learned_addr learned my addr 172.21.15.86:0/3709783925 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:11.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.434+0000 7fafc5000640 1 --2- 172.21.15.86:0/3709783925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafc00b6530 0x7fafc00b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:11.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.434+0000 7fafbf400640 1 -- 172.21.15.86:0/3709783925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafc00b6530 msgr2=0x7fafc00b6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:11.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.434+0000 7fafbf400640 1 --2- 172.21.15.86:0/3709783925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafc00b6530 0x7fafc00b6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.434+0000 7fafbf400640 1 -- 172.21.15.86:0/3709783925 >> v1:172.21.15.155:6789/0 conn(0x7fafc006f990 legacy=0x7fafc006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:11.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.434+0000 7fafbf400640 1 -- 172.21.15.86:0/3709783925 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc00d1cf0 con 0x7fafc01615c0 2024-04-25T14:18:11.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.438+0000 7fafbf400640 1 --2- 172.21.15.86:0/3709783925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafc01615c0 0x7fafc01619d0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7fafac009870 tx=0x7fafac01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd30edc6b9d45fc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:11.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.438+0000 7fafbea00640 1 -- 172.21.15.86:0/3709783925 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac0b0070 con 0x7fafc01615c0 2024-04-25T14:18:11.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.438+0000 7fafbea00640 1 -- 172.21.15.86:0/3709783925 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafac0ab600 con 0x7fafc01615c0 2024-04-25T14:18:11.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.438+0000 7fafbea00640 1 -- 172.21.15.86:0/3709783925 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac0ab9d0 con 0x7fafc01615c0 2024-04-25T14:18:11.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.438+0000 7fafc5a00640 1 -- 172.21.15.86:0/3709783925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafc01615c0 msgr2=0x7fafc01619d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:11.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.438+0000 7fafc5a00640 1 --2- 172.21.15.86:0/3709783925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafc01615c0 0x7fafc01619d0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7fafac009870 tx=0x7fafac01c4d0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.438+0000 7fafc5a00640 1 -- 172.21.15.86:0/3709783925 shutdown_connections 2024-04-25T14:18:11.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.438+0000 7fafc5a00640 1 --2- 172.21.15.86:0/3709783925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafc00b6530 0x7fafc00b6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.438+0000 7fafc5a00640 1 --2- 172.21.15.86:0/3709783925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafc01615c0 0x7fafc01619d0 unknown :-1 s=CLOSED pgs=2510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.438+0000 7fafc5a00640 1 -- 172.21.15.86:0/3709783925 >> 172.21.15.86:0/3709783925 conn(0x7fafc00cfea0 msgr2=0x7fafc00d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:11.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.438+0000 7fafc5a00640 1 -- 172.21.15.86:0/3709783925 shutdown_connections 2024-04-25T14:18:11.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.438+0000 7fafc5a00640 1 -- 172.21.15.86:0/3709783925 wait complete. 2024-04-25T14:18:11.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.442+0000 7fafc5a00640 1 Processor -- start 2024-04-25T14:18:11.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.442+0000 7fafc5a00640 1 -- start start 2024-04-25T14:18:11.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.442+0000 7fafc5a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafc006f990 0x7fafc014e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:11.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.442+0000 7fafc5a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafc00b6530 0x7fafc01541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:11.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.442+0000 7fafc5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafc01615c0 0x7fafc015bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:11.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.442+0000 7fafc5a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fafc00d2880 con 0x7fafc01615c0 2024-04-25T14:18:11.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.442+0000 7fafc5a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fafc015df20 con 0x7fafc00b6530 2024-04-25T14:18:11.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.442+0000 7fafc5a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fafc016b170 con 0x7fafc006f990 2024-04-25T14:18:11.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.442+0000 7fafc5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafc01615c0 0x7fafc015bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:11.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.442+0000 7fafc5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafc01615c0 0x7fafc015bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46836/0 (socket says 172.21.15.86:46836) 2024-04-25T14:18:11.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.442+0000 7fafc5000640 1 -- 172.21.15.86:0/3557873631 learned_addr learned my addr 172.21.15.86:0/3557873631 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:11.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.442+0000 7fafbfe00640 1 --2- 172.21.15.86:0/3557873631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafc006f990 0x7fafc014e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:11.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.442+0000 7fafbf400640 1 --2- 172.21.15.86:0/3557873631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafc00b6530 0x7fafc01541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.446+0000 7fafc5000640 1 -- 172.21.15.86:0/3557873631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafc006f990 msgr2=0x7fafc014e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.446+0000 7fafc5000640 1 --2- 172.21.15.86:0/3557873631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafc006f990 0x7fafc014e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.446+0000 7fafc5000640 1 -- 172.21.15.86:0/3557873631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafc00b6530 msgr2=0x7fafc01541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.446+0000 7fafc5000640 1 --2- 172.21.15.86:0/3557873631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafc00b6530 0x7fafc01541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.446+0000 7fafc5000640 1 -- 172.21.15.86:0/3557873631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc015f850 con 0x7fafc01615c0 2024-04-25T14:18:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.446+0000 7fafc5000640 1 --2- 172.21.15.86:0/3557873631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafc01615c0 0x7fafc015bb10 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7fafb8007d70 tx=0x7fafb800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.446+0000 7fafb7e00640 1 -- 172.21.15.86:0/3557873631 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb8014420 con 0x7fafc01615c0 2024-04-25T14:18:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.446+0000 7fafb7e00640 1 -- 172.21.15.86:0/3557873631 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb8019070 con 0x7fafc01615c0 2024-04-25T14:18:11.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.446+0000 7fafb7e00640 1 -- 172.21.15.86:0/3557873631 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb8014820 con 0x7fafc01615c0 2024-04-25T14:18:11.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.446+0000 7fafc5a00640 1 -- 172.21.15.86:0/3557873631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc015fb30 con 0x7fafc01615c0 2024-04-25T14:18:11.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.446+0000 7fafc5a00640 1 -- 172.21.15.86:0/3557873631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc01600f0 con 0x7fafc01615c0 2024-04-25T14:18:11.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.446+0000 7fafc5a00640 1 -- 172.21.15.86:0/3557873631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc0076b80 con 0x7fafc01615c0 2024-04-25T14:18:11.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.446+0000 7fafb7e00640 1 -- 172.21.15.86:0/3557873631 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7fafb80149c0 con 0x7fafc01615c0 2024-04-25T14:18:11.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.446+0000 7fafb7e00640 1 --2- 172.21.15.86:0/3557873631 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fafa006f400 0x7fafa0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:11.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.446+0000 7fafb7e00640 1 -- 172.21.15.86:0/3557873631 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 6023+0+0 (secure 0 0 0) 0x7fafb808fb50 con 0x7fafc01615c0 2024-04-25T14:18:11.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.450+0000 7fafbfe00640 1 --2- 172.21.15.86:0/3557873631 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fafa006f400 0x7fafa0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:11.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.450+0000 7fafbfe00640 1 --2- 172.21.15.86:0/3557873631 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fafa006f400 0x7fafa0071910 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fafb0001770 tx=0x7fafb0009040 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:11.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.454+0000 7fafb7e00640 1 -- 172.21.15.86:0/3557873631 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fafb805c210 con 0x7fafc01615c0 2024-04-25T14:18:11.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.610+0000 7fafc5a00640 1 -- 172.21.15.86:0/3557873631 --> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fafc00b2c40 con 0x7fafa006f400 2024-04-25T14:18:11.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.610+0000 7fafb7e00640 1 -- 172.21.15.86:0/3557873631 <== mgr.11343 v2:172.21.15.86:6832/3174179029 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fafc00b2c40 con 0x7fafa006f400 2024-04-25T14:18:11.604 INFO:teuthology.orchestra.run.smithi086.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T14:18:11.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.610+0000 7fafc5a00640 1 -- 172.21.15.86:0/3557873631 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fafa006f400 msgr2=0x7fafa0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:11.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.610+0000 7fafc5a00640 1 --2- 172.21.15.86:0/3557873631 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fafa006f400 0x7fafa0071910 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fafb0001770 tx=0x7fafb0009040 comp rx=0 tx=0).stop 2024-04-25T14:18:11.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.610+0000 7fafc5a00640 1 -- 172.21.15.86:0/3557873631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafc01615c0 msgr2=0x7fafc015bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:11.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.610+0000 7fafc5a00640 1 --2- 172.21.15.86:0/3557873631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafc01615c0 0x7fafc015bb10 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7fafb8007d70 tx=0x7fafb800e4b0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.610+0000 7fafc5a00640 1 -- 172.21.15.86:0/3557873631 shutdown_connections 2024-04-25T14:18:11.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.610+0000 7fafc5a00640 1 --2- 172.21.15.86:0/3557873631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafc01615c0 0x7fafc015bb10 unknown :-1 s=CLOSED pgs=2511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.610+0000 7fafc5a00640 1 --2- 172.21.15.86:0/3557873631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafc00b6530 0x7fafc01541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.610+0000 7fafc5a00640 1 --2- 172.21.15.86:0/3557873631 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fafa006f400 0x7fafa0071910 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.610+0000 7fafc5a00640 1 --2- 172.21.15.86:0/3557873631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafc006f990 0x7fafc014e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.610+0000 7fafc5a00640 1 -- 172.21.15.86:0/3557873631 >> 172.21.15.86:0/3557873631 conn(0x7fafc00cfea0 msgr2=0x7fafc015b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:11.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.610+0000 7fafc5a00640 1 -- 172.21.15.86:0/3557873631 shutdown_connections 2024-04-25T14:18:11.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.610+0000 7fafc5a00640 1 -- 172.21.15.86:0/3557873631 wait complete. 2024-04-25T14:18:11.624 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T14:18:11.625 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T14:18:11.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.778+0000 7f93cf800640 1 Processor -- start 2024-04-25T14:18:11.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.782+0000 7f93cf800640 1 -- start start 2024-04-25T14:18:11.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.782+0000 7f93cf800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93c80b0540 0x7f93c80ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:11.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.782+0000 7f93cf800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93c80b0b20 con 0x7f93c8165650 2024-04-25T14:18:11.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.782+0000 7f93cf800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93c806d220 con 0x7f93c815e270 2024-04-25T14:18:11.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.782+0000 7f93cf800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f93c806d870 con 0x7f93c80b0540 2024-04-25T14:18:11.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.782+0000 7f93cd800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f93c8165650 0x7f93c8167a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41866/0 (socket says 172.21.15.86:41866) 2024-04-25T14:18:11.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.782+0000 7f93cd800640 1 -- 172.21.15.86:0/80680004 learned_addr learned my addr 172.21.15.86:0/80680004 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:18:11.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.782+0000 7f93cce00640 1 --2- 172.21.15.86:0/80680004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93c80b0540 0x7f93c80ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:11.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.782+0000 7f93c7e00640 1 -- 172.21.15.86:0/80680004 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3352697770 0 0) 0x7f93c80b0b20 con 0x7f93c8165650 2024-04-25T14:18:11.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.782+0000 7f93c7e00640 1 -- 172.21.15.86:0/80680004 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93ac003530 con 0x7f93c8165650 2024-04-25T14:18:11.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.782+0000 7f93c7e00640 1 -- 172.21.15.86:0/80680004 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2611300986 0 0) 0x7f93c806d220 con 0x7f93c815e270 2024-04-25T14:18:11.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.782+0000 7f93c7e00640 1 -- 172.21.15.86:0/80680004 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93c80b0b20 con 0x7f93c815e270 2024-04-25T14:18:11.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.782+0000 7f93cce00640 1 -- 172.21.15.86:0/80680004 >> v1:172.21.15.155:6789/0 conn(0x7f93c815e270 legacy=0x7f93c815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:11.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.782+0000 7f93cce00640 1 -- 172.21.15.86:0/80680004 >> v1:172.21.15.86:6789/0 conn(0x7f93c8165650 legacy=0x7f93c8167a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:11.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.782+0000 7f93c7e00640 1 -- 172.21.15.86:0/80680004 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3913696150 0 0) 0x7f93ac003530 con 0x7f93c8165650 2024-04-25T14:18:11.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.782+0000 7f93cce00640 1 -- 172.21.15.86:0/80680004 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c806d500 con 0x7f93c80b0540 2024-04-25T14:18:11.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.786+0000 7f93cce00640 1 --2- 172.21.15.86:0/80680004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93c80b0540 0x7f93c80ae2a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f93b80097f0 tx=0x7f93b801c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bca3fd5872218f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:11.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.786+0000 7f93c7e00640 1 -- 172.21.15.86:0/80680004 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b80b0070 con 0x7f93c80b0540 2024-04-25T14:18:11.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.786+0000 7f93c7e00640 1 -- 172.21.15.86:0/80680004 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93b80ab400 con 0x7f93c80b0540 2024-04-25T14:18:11.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.786+0000 7f93c7e00640 1 -- 172.21.15.86:0/80680004 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c0001a00 con 0x7f93c80b0540 2024-04-25T14:18:11.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.786+0000 7f93cf800640 1 -- 172.21.15.86:0/80680004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93c80b0540 msgr2=0x7f93c80ae2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:11.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.786+0000 7f93cf800640 1 --2- 172.21.15.86:0/80680004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93c80b0540 0x7f93c80ae2a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f93b80097f0 tx=0x7f93b801c370 comp rx=0 tx=0).stop 2024-04-25T14:18:11.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.786+0000 7f93cf800640 1 -- 172.21.15.86:0/80680004 shutdown_connections 2024-04-25T14:18:11.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.786+0000 7f93cf800640 1 --2- 172.21.15.86:0/80680004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93c80b0540 0x7f93c80ae2a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.786+0000 7f93cf800640 1 -- 172.21.15.86:0/80680004 >> 172.21.15.86:0/80680004 conn(0x7f93c80b4a40 msgr2=0x7f93c80b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:11.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.786+0000 7f93cf800640 1 -- 172.21.15.86:0/80680004 shutdown_connections 2024-04-25T14:18:11.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.786+0000 7f93cf800640 1 -- 172.21.15.86:0/80680004 wait complete. 2024-04-25T14:18:11.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.790+0000 7f93cf800640 1 Processor -- start 2024-04-25T14:18:11.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.790+0000 7f93cf800640 1 -- start start 2024-04-25T14:18:11.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.790+0000 7f93cf800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93c80b0540 0x7f93c813c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:11.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.790+0000 7f93cf800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93c815e270 0x7f93c815a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:11.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.790+0000 7f93cf800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93c8165650 0x7f93c815ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:11.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.790+0000 7f93cf800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93c816bd40 con 0x7f93c80b0540 2024-04-25T14:18:11.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.790+0000 7f93cf800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f93c816bbc0 con 0x7f93c8165650 2024-04-25T14:18:11.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.790+0000 7f93cf800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f93c816bec0 con 0x7f93c815e270 2024-04-25T14:18:11.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.790+0000 7f93cce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93c80b0540 0x7f93c813c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:11.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.790+0000 7f93cce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93c80b0540 0x7f93c813c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46850/0 (socket says 172.21.15.86:46850) 2024-04-25T14:18:11.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.790+0000 7f93cce00640 1 -- 172.21.15.86:0/1945418174 learned_addr learned my addr 172.21.15.86:0/1945418174 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:11.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.790+0000 7f93cd800640 1 --2- 172.21.15.86:0/1945418174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93c8165650 0x7f93c815ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:11.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.790+0000 7f93bfe00640 1 --2- 172.21.15.86:0/1945418174 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93c815e270 0x7f93c815a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:11.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.790+0000 7f93cce00640 1 -- 172.21.15.86:0/1945418174 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93c815e270 msgr2=0x7f93c815a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:11.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.790+0000 7f93cce00640 1 --2- 172.21.15.86:0/1945418174 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93c815e270 0x7f93c815a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.790+0000 7f93cce00640 1 -- 172.21.15.86:0/1945418174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93c8165650 msgr2=0x7f93c815ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:11.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.790+0000 7f93cce00640 1 --2- 172.21.15.86:0/1945418174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93c8165650 0x7f93c815ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.790+0000 7f93cce00640 1 -- 172.21.15.86:0/1945418174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c8165d40 con 0x7f93c80b0540 2024-04-25T14:18:11.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.794+0000 7f93cce00640 1 --2- 172.21.15.86:0/1945418174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93c80b0540 0x7f93c813c490 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7f93b80097f0 tx=0x7f93b801abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:11.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.794+0000 7f93c5600640 1 -- 172.21.15.86:0/1945418174 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b80bb020 con 0x7f93c80b0540 2024-04-25T14:18:11.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.794+0000 7f93c5600640 1 -- 172.21.15.86:0/1945418174 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93b80b1040 con 0x7f93c80b0540 2024-04-25T14:18:11.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.794+0000 7f93cf800640 1 -- 172.21.15.86:0/1945418174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c8166020 con 0x7f93c80b0540 2024-04-25T14:18:11.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.794+0000 7f93c5600640 1 -- 172.21.15.86:0/1945418174 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b80b0050 con 0x7f93c80b0540 2024-04-25T14:18:11.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.794+0000 7f93cf800640 1 -- 172.21.15.86:0/1945418174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c81665e0 con 0x7f93c80b0540 2024-04-25T14:18:11.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.794+0000 7f93cf800640 1 -- 172.21.15.86:0/1945418174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f93c8050750 con 0x7f93c80b0540 2024-04-25T14:18:11.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.794+0000 7f93c5600640 1 -- 172.21.15.86:0/1945418174 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7f93b80be070 con 0x7f93c80b0540 2024-04-25T14:18:11.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.794+0000 7f93c5600640 1 --2- 172.21.15.86:0/1945418174 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f93a406f400 0x7f93a4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:11.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.794+0000 7f93c5600640 1 -- 172.21.15.86:0/1945418174 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 6023+0+0 (secure 0 0 0) 0x7f93b8128f50 con 0x7f93c80b0540 2024-04-25T14:18:11.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.798+0000 7f93bfe00640 1 --2- 172.21.15.86:0/1945418174 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f93a406f400 0x7f93a4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:11.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.798+0000 7f93c5600640 1 --2- 172.21.15.86:0/1945418174 >> [v2:172.21.15.155:6817/288803543,v1:172.21.15.155:6818/288803543] conn(0x7f93a4075ee0 0x7f93a4078370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:11.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.798+0000 7f93c5600640 1 -- 172.21.15.86:0/1945418174 --> [v2:172.21.15.155:6817/288803543,v1:172.21.15.155:6818/288803543] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f93b800af30 con 0x7f93a4075ee0 2024-04-25T14:18:11.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.798+0000 7f93c5600640 1 -- 172.21.15.86:0/1945418174 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=160) ==== 24+0+0 (secure 0 0 0) 0x7f93b81293e0 con 0x7f93c80b0540 2024-04-25T14:18:11.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.798+0000 7f93cd800640 1 --2- 172.21.15.86:0/1945418174 >> [v2:172.21.15.155:6817/288803543,v1:172.21.15.155:6818/288803543] conn(0x7f93a4075ee0 0x7f93a4078370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:11.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.798+0000 7f93bfe00640 1 --2- 172.21.15.86:0/1945418174 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f93a406f400 0x7f93a4071910 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f93c81fc870 tx=0x7f93b0009040 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:11.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.798+0000 7f93cd800640 1 --2- 172.21.15.86:0/1945418174 >> [v2:172.21.15.155:6817/288803543,v1:172.21.15.155:6818/288803543] conn(0x7f93a4075ee0 0x7f93a4078370 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:11.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.798+0000 7f93c5600640 1 -- 172.21.15.86:0/1945418174 <== osd.5 v2:172.21.15.155:6817/288803543 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f93b800af30 con 0x7f93a4075ee0 2024-04-25T14:18:11.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.810+0000 7f93cf800640 1 -- 172.21.15.86:0/1945418174 --> [v2:172.21.15.155:6817/288803543,v1:172.21.15.155:6818/288803543] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f93c80b3520 con 0x7f93a4075ee0 2024-04-25T14:18:11.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.810+0000 7f93c5600640 1 -- 172.21.15.86:0/1945418174 <== osd.5 v2:172.21.15.155:6817/288803543 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f93c80b3520 con 0x7f93a4075ee0 2024-04-25T14:18:11.806 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:18:11.806 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T14:18:11.807 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:18:11.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.810+0000 7f93cf800640 1 -- 172.21.15.86:0/1945418174 >> [v2:172.21.15.155:6817/288803543,v1:172.21.15.155:6818/288803543] conn(0x7f93a4075ee0 msgr2=0x7f93a4078370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.814+0000 7f93cf800640 1 --2- 172.21.15.86:0/1945418174 >> [v2:172.21.15.155:6817/288803543,v1:172.21.15.155:6818/288803543] conn(0x7f93a4075ee0 0x7f93a4078370 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.814+0000 7f93cf800640 1 -- 172.21.15.86:0/1945418174 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f93a406f400 msgr2=0x7f93a4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.814+0000 7f93cf800640 1 --2- 172.21.15.86:0/1945418174 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f93a406f400 0x7f93a4071910 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f93c81fc870 tx=0x7f93b0009040 comp rx=0 tx=0).stop 2024-04-25T14:18:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.814+0000 7f93cf800640 1 -- 172.21.15.86:0/1945418174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93c80b0540 msgr2=0x7f93c813c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.814+0000 7f93cf800640 1 --2- 172.21.15.86:0/1945418174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93c80b0540 0x7f93c813c490 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7f93b80097f0 tx=0x7f93b801abb0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.814+0000 7f93cd800640 1 -- 172.21.15.86:0/1945418174 reap_dead start 2024-04-25T14:18:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.814+0000 7f93cf800640 1 -- 172.21.15.86:0/1945418174 shutdown_connections 2024-04-25T14:18:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.814+0000 7f93cf800640 1 -- 172.21.15.86:0/1945418174 >> 172.21.15.86:0/1945418174 conn(0x7f93c80b4a40 msgr2=0x7f93c8164e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:11.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.814+0000 7f93cf800640 1 -- 172.21.15.86:0/1945418174 shutdown_connections 2024-04-25T14:18:11.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.814+0000 7f93cf800640 1 -- 172.21.15.86:0/1945418174 wait complete. 2024-04-25T14:18:11.825 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T14:18:11.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.978+0000 7fa5e6c00640 1 Processor -- start 2024-04-25T14:18:11.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.978+0000 7fa5e6c00640 1 -- start start 2024-04-25T14:18:11.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.978+0000 7fa5e6c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa5e0070330 0x7fa5e00d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:11.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.978+0000 7fa5e6c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5e00d2030 con 0x7fa5e00d1c20 2024-04-25T14:18:11.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.978+0000 7fa5e6c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5e015e5c0 con 0x7fa5e00ce790 2024-04-25T14:18:11.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.978+0000 7fa5e6c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa5e015e8a0 con 0x7fa5e0070330 2024-04-25T14:18:11.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.982+0000 7fa5df400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa5e00d1c20 0x7fa5e006ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41878/0 (socket says 172.21.15.86:41878) 2024-04-25T14:18:11.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.982+0000 7fa5df400640 1 -- 172.21.15.86:0/4175599561 learned_addr learned my addr 172.21.15.86:0/4175599561 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:18:11.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.982+0000 7fa5dfe00640 1 --2- 172.21.15.86:0/4175599561 >> v2:172.21.15.155:3300/0 conn(0x7fa5e0070330 0x7fa5e00d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:11.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.982+0000 7fa5dea00640 1 -- 172.21.15.86:0/4175599561 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2425843256 0 0) 0x7fa5cc001120 con 0x7fa5e00d1c20 2024-04-25T14:18:11.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.982+0000 7fa5dea00640 1 -- 172.21.15.86:0/4175599561 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5c4003530 con 0x7fa5e00d1c20 2024-04-25T14:18:11.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.982+0000 7fa5dea00640 1 -- 172.21.15.86:0/4175599561 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 58150397 0 0) 0x7fa5e015e5c0 con 0x7fa5e00ce790 2024-04-25T14:18:11.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.982+0000 7fa5dea00640 1 -- 172.21.15.86:0/4175599561 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5cc001120 con 0x7fa5e00ce790 2024-04-25T14:18:11.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.982+0000 7fa5dfe00640 1 -- 172.21.15.86:0/4175599561 >> v1:172.21.15.155:6789/0 conn(0x7fa5e00ce790 legacy=0x7fa5e006f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:11.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.982+0000 7fa5dfe00640 1 -- 172.21.15.86:0/4175599561 >> v1:172.21.15.86:6789/0 conn(0x7fa5e00d1c20 legacy=0x7fa5e006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:11.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.982+0000 7fa5dfe00640 1 -- 172.21.15.86:0/4175599561 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e015eb80 con 0x7fa5e0070330 2024-04-25T14:18:11.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.982+0000 7fa5dfe00640 1 --2- 172.21.15.86:0/4175599561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5e0070330 0x7fa5e00d16e0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0009dc0 tx=0x7fa5d001c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81fabd23e14627a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:11.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.982+0000 7fa5dea00640 1 -- 172.21.15.86:0/4175599561 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d00b2070 con 0x7fa5e0070330 2024-04-25T14:18:11.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.986+0000 7fa5dea00640 1 -- 172.21.15.86:0/4175599561 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5d00a5850 con 0x7fa5e0070330 2024-04-25T14:18:11.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.986+0000 7fa5dea00640 1 -- 172.21.15.86:0/4175599561 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d00a5c00 con 0x7fa5e0070330 2024-04-25T14:18:11.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.986+0000 7fa5e6c00640 1 -- 172.21.15.86:0/4175599561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5e0070330 msgr2=0x7fa5e00d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:11.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.986+0000 7fa5e6c00640 1 --2- 172.21.15.86:0/4175599561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5e0070330 0x7fa5e00d16e0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0009dc0 tx=0x7fa5d001c8c0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.986+0000 7fa5e6c00640 1 -- 172.21.15.86:0/4175599561 shutdown_connections 2024-04-25T14:18:11.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.986+0000 7fa5e6c00640 1 --2- 172.21.15.86:0/4175599561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5e0070330 0x7fa5e00d16e0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.986+0000 7fa5e6c00640 1 -- 172.21.15.86:0/4175599561 >> 172.21.15.86:0/4175599561 conn(0x7fa5e00b4650 msgr2=0x7fa5e00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:11.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.986+0000 7fa5e6c00640 1 -- 172.21.15.86:0/4175599561 shutdown_connections 2024-04-25T14:18:11.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.986+0000 7fa5e6c00640 1 -- 172.21.15.86:0/4175599561 wait complete. 2024-04-25T14:18:11.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.986+0000 7fa5e6c00640 1 Processor -- start 2024-04-25T14:18:11.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.986+0000 7fa5e6c00640 1 -- start start 2024-04-25T14:18:11.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.990+0000 7fa5e6c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5e0070330 0x7fa5e0099cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:11.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.990+0000 7fa5e6c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5e00ce790 0x7fa5e0154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:11.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.990+0000 7fa5e6c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa5e00d1c20 0x7fa5e0158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:11.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.990+0000 7fa5e6c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e00708c0 con 0x7fa5e00ce790 2024-04-25T14:18:11.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.990+0000 7fa5e6c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e0070740 con 0x7fa5e0070330 2024-04-25T14:18:11.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.990+0000 7fa5e6c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa5e006fb50 con 0x7fa5e00d1c20 2024-04-25T14:18:11.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.990+0000 7fa5df400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5e00ce790 0x7fa5e0154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:11.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.990+0000 7fa5df400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5e00ce790 0x7fa5e0154a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46854/0 (socket says 172.21.15.86:46854) 2024-04-25T14:18:11.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.990+0000 7fa5df400640 1 -- 172.21.15.86:0/1292788859 learned_addr learned my addr 172.21.15.86:0/1292788859 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:11.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.990+0000 7fa5dfe00640 1 --2- 172.21.15.86:0/1292788859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5e0070330 0x7fa5e0099cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:11.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.990+0000 7fa5e4c00640 1 --2- 172.21.15.86:0/1292788859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa5e00d1c20 0x7fa5e0158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:11.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.990+0000 7fa5df400640 1 -- 172.21.15.86:0/1292788859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa5e00d1c20 msgr2=0x7fa5e0158f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:11.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.990+0000 7fa5df400640 1 --2- 172.21.15.86:0/1292788859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa5e00d1c20 0x7fa5e0158f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.990+0000 7fa5df400640 1 -- 172.21.15.86:0/1292788859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5e0070330 msgr2=0x7fa5e0099cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:11.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.990+0000 7fa5df400640 1 --2- 172.21.15.86:0/1292788859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5e0070330 0x7fa5e0099cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:11.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.990+0000 7fa5df400640 1 -- 172.21.15.86:0/1292788859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e014fab0 con 0x7fa5e00ce790 2024-04-25T14:18:11.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.990+0000 7fa5e4c00640 1 --2- 172.21.15.86:0/1292788859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa5e00d1c20 0x7fa5e0158f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:18:11.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.990+0000 7fa5df400640 1 --2- 172.21.15.86:0/1292788859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5e00ce790 0x7fa5e0154a50 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc00c730 tx=0x7fa5cc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:11.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.990+0000 7fa5d7e00640 1 -- 172.21.15.86:0/1292788859 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc017070 con 0x7fa5e00ce790 2024-04-25T14:18:11.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.990+0000 7fa5d7e00640 1 -- 172.21.15.86:0/1292788859 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5cc005aa0 con 0x7fa5e00ce790 2024-04-25T14:18:11.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.990+0000 7fa5d7e00640 1 -- 172.21.15.86:0/1292788859 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc007070 con 0x7fa5e00ce790 2024-04-25T14:18:11.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.990+0000 7fa5e6c00640 1 -- 172.21.15.86:0/1292788859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e009a450 con 0x7fa5e00ce790 2024-04-25T14:18:11.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.990+0000 7fa5e6c00640 1 -- 172.21.15.86:0/1292788859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e009c120 con 0x7fa5e00ce790 2024-04-25T14:18:11.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.990+0000 7fa5e6c00640 1 -- 172.21.15.86:0/1292788859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5e0069d00 con 0x7fa5e00ce790 2024-04-25T14:18:11.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.994+0000 7fa5d7e00640 1 -- 172.21.15.86:0/1292788859 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7fa5cc01a070 con 0x7fa5e00ce790 2024-04-25T14:18:11.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.994+0000 7fa5d7e00640 1 --2- 172.21.15.86:0/1292788859 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fa5c006f430 0x7fa5c0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:11.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.994+0000 7fa5d7e00640 1 -- 172.21.15.86:0/1292788859 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 6023+0+0 (secure 0 0 0) 0x7fa5cc08edd0 con 0x7fa5e00ce790 2024-04-25T14:18:11.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.994+0000 7fa5dfe00640 1 --2- 172.21.15.86:0/1292788859 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fa5c006f430 0x7fa5c0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:11.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.994+0000 7fa5dfe00640 1 --2- 172.21.15.86:0/1292788859 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fa5c006f430 0x7fa5c0071940 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa5d00af000 tx=0x7fa5d00a7070 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:11.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:11.998+0000 7fa5d7e00640 1 -- 172.21.15.86:0/1292788859 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa5cc021020 con 0x7fa5e00ce790 2024-04-25T14:18:12.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.154+0000 7fa5e6c00640 1 -- 172.21.15.86:0/1292788859 --> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fa5e0076bf0 con 0x7fa5c006f430 2024-04-25T14:18:12.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.154+0000 7fa5d7e00640 1 -- 172.21.15.86:0/1292788859 <== mgr.11343 v2:172.21.15.86:6832/3174179029 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa5e0076bf0 con 0x7fa5c006f430 2024-04-25T14:18:12.149 INFO:teuthology.orchestra.run.smithi086.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T14:18:12.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.154+0000 7fa5e6c00640 1 -- 172.21.15.86:0/1292788859 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fa5c006f430 msgr2=0x7fa5c0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:12.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.154+0000 7fa5e6c00640 1 --2- 172.21.15.86:0/1292788859 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fa5c006f430 0x7fa5c0071940 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa5d00af000 tx=0x7fa5d00a7070 comp rx=0 tx=0).stop 2024-04-25T14:18:12.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.154+0000 7fa5e6c00640 1 -- 172.21.15.86:0/1292788859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5e00ce790 msgr2=0x7fa5e0154a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:12.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.154+0000 7fa5e6c00640 1 --2- 172.21.15.86:0/1292788859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5e00ce790 0x7fa5e0154a50 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc00c730 tx=0x7fa5cc00cbf0 comp rx=0 tx=0).stop 2024-04-25T14:18:12.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.158+0000 7fa5e6c00640 1 -- 172.21.15.86:0/1292788859 shutdown_connections 2024-04-25T14:18:12.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.158+0000 7fa5e6c00640 1 --2- 172.21.15.86:0/1292788859 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fa5c006f430 0x7fa5c0071940 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:12.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.158+0000 7fa5e6c00640 1 --2- 172.21.15.86:0/1292788859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa5e00d1c20 0x7fa5e0158f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:12.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.158+0000 7fa5e6c00640 1 --2- 172.21.15.86:0/1292788859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5e00ce790 0x7fa5e0154a50 unknown :-1 s=CLOSED pgs=2515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:12.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.158+0000 7fa5e6c00640 1 --2- 172.21.15.86:0/1292788859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5e0070330 0x7fa5e0099cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:12.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.158+0000 7fa5e6c00640 1 -- 172.21.15.86:0/1292788859 >> 172.21.15.86:0/1292788859 conn(0x7fa5e00b4650 msgr2=0x7fa5e015d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:12.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.158+0000 7fa5e6c00640 1 -- 172.21.15.86:0/1292788859 shutdown_connections 2024-04-25T14:18:12.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.158+0000 7fa5e6c00640 1 -- 172.21.15.86:0/1292788859 wait complete. 2024-04-25T14:18:12.170 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T14:18:12.171 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T14:18:12.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.330+0000 7fd9e2a00640 1 Processor -- start 2024-04-25T14:18:12.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.330+0000 7fd9e2a00640 1 -- start start 2024-04-25T14:18:12.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.330+0000 7fd9e2a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9dc0d1b50 0x7fd9dc15dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:12.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.330+0000 7fd9e2a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd9dc0b1dd0 0x7fd9dc0b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:12.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.330+0000 7fd9e2a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9dc0b19d0 con 0x7fd9dc0d2230 2024-04-25T14:18:12.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.330+0000 7fd9e2a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd9dc06edc0 con 0x7fd9dc0b1dd0 2024-04-25T14:18:12.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.330+0000 7fd9e2a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd9dc06e770 con 0x7fd9dc0d1b50 2024-04-25T14:18:12.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.330+0000 7fd9d3400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd9dc0d2230 0x7fd9dc0d2640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41880/0 (socket says 172.21.15.86:41880) 2024-04-25T14:18:12.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.330+0000 7fd9d3400640 1 -- 172.21.15.86:0/4037840257 learned_addr learned my addr 172.21.15.86:0/4037840257 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:18:12.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.330+0000 7fd9e0a00640 1 --2- 172.21.15.86:0/4037840257 >> v2:172.21.15.155:3300/0 conn(0x7fd9dc0b1dd0 0x7fd9dc0b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:12.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.330+0000 7fd9dbe00640 1 --2- 172.21.15.86:0/4037840257 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9dc0d1b50 0x7fd9dc15dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:12.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.330+0000 7fd9db400640 1 -- 172.21.15.86:0/4037840257 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 712961682 0 0) 0x7fd9dc0b19d0 con 0x7fd9dc0d2230 2024-04-25T14:18:12.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.330+0000 7fd9db400640 1 -- 172.21.15.86:0/4037840257 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9c0003530 con 0x7fd9dc0d2230 2024-04-25T14:18:12.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.330+0000 7fd9e0a00640 1 -- 172.21.15.86:0/4037840257 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9dc0d1b50 msgr2=0x7fd9dc15dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:12.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.330+0000 7fd9e0a00640 1 --2- 172.21.15.86:0/4037840257 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9dc0d1b50 0x7fd9dc15dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:12.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.330+0000 7fd9e0a00640 1 -- 172.21.15.86:0/4037840257 >> v1:172.21.15.86:6789/0 conn(0x7fd9dc0d2230 legacy=0x7fd9dc0d2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:12.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.330+0000 7fd9e0a00640 1 -- 172.21.15.86:0/4037840257 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9dc06ea50 con 0x7fd9dc0b1dd0 2024-04-25T14:18:12.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.334+0000 7fd9e0a00640 1 --2- 172.21.15.86:0/4037840257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9dc0b1dd0 0x7fd9dc0b21e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009d90 tx=0x7fd9d401c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=993e2d189d0f3cf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:12.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.334+0000 7fd9db400640 1 -- 172.21.15.86:0/4037840257 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d40b2070 con 0x7fd9dc0b1dd0 2024-04-25T14:18:12.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.334+0000 7fd9db400640 1 -- 172.21.15.86:0/4037840257 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9d40a58d0 con 0x7fd9dc0b1dd0 2024-04-25T14:18:12.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.334+0000 7fd9db400640 1 -- 172.21.15.86:0/4037840257 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d40a5ca0 con 0x7fd9dc0b1dd0 2024-04-25T14:18:12.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.334+0000 7fd9e2a00640 1 -- 172.21.15.86:0/4037840257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9dc0b1dd0 msgr2=0x7fd9dc0b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:12.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.334+0000 7fd9e2a00640 1 --2- 172.21.15.86:0/4037840257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9dc0b1dd0 0x7fd9dc0b21e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009d90 tx=0x7fd9d401c900 comp rx=0 tx=0).stop 2024-04-25T14:18:12.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.334+0000 7fd9e2a00640 1 -- 172.21.15.86:0/4037840257 shutdown_connections 2024-04-25T14:18:12.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.334+0000 7fd9e2a00640 1 --2- 172.21.15.86:0/4037840257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9dc0b1dd0 0x7fd9dc0b21e0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:12.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.334+0000 7fd9e2a00640 1 --2- 172.21.15.86:0/4037840257 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9dc0d1b50 0x7fd9dc15dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:12.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.334+0000 7fd9e2a00640 1 -- 172.21.15.86:0/4037840257 >> 172.21.15.86:0/4037840257 conn(0x7fd9dc073c90 msgr2=0x7fd9dc0b77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:12.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.334+0000 7fd9e2a00640 1 -- 172.21.15.86:0/4037840257 shutdown_connections 2024-04-25T14:18:12.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.334+0000 7fd9e2a00640 1 -- 172.21.15.86:0/4037840257 wait complete. 2024-04-25T14:18:12.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.338+0000 7fd9e2a00640 1 Processor -- start 2024-04-25T14:18:12.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.338+0000 7fd9e2a00640 1 -- start start 2024-04-25T14:18:12.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.338+0000 7fd9e2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9dc0b1dd0 0x7fd9dc14d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:12.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.338+0000 7fd9e2a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9dc0d1b50 0x7fd9dc15bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:12.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.338+0000 7fd9e2a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9dc0d2230 0x7fd9dc166d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:12.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.338+0000 7fd9e2a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc16ad60 con 0x7fd9dc0b1dd0 2024-04-25T14:18:12.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.338+0000 7fd9e2a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc16abe0 con 0x7fd9dc0d2230 2024-04-25T14:18:12.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.338+0000 7fd9e2a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd9dc16aee0 con 0x7fd9dc0d1b50 2024-04-25T14:18:12.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.338+0000 7fd9dbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9dc0b1dd0 0x7fd9dc14d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:12.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.338+0000 7fd9dbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9dc0b1dd0 0x7fd9dc14d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46870/0 (socket says 172.21.15.86:46870) 2024-04-25T14:18:12.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.338+0000 7fd9dbe00640 1 -- 172.21.15.86:0/2304944699 learned_addr learned my addr 172.21.15.86:0/2304944699 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:12.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.338+0000 7fd9e0a00640 1 --2- 172.21.15.86:0/2304944699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9dc0d2230 0x7fd9dc166d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:12.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.338+0000 7fd9d3400640 1 --2- 172.21.15.86:0/2304944699 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9dc0d1b50 0x7fd9dc15bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:12.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.338+0000 7fd9dbe00640 1 -- 172.21.15.86:0/2304944699 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9dc0d1b50 msgr2=0x7fd9dc15bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:12.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.338+0000 7fd9dbe00640 1 --2- 172.21.15.86:0/2304944699 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9dc0d1b50 0x7fd9dc15bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:12.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.338+0000 7fd9dbe00640 1 -- 172.21.15.86:0/2304944699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9dc0d2230 msgr2=0x7fd9dc166d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:12.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.338+0000 7fd9dbe00640 1 --2- 172.21.15.86:0/2304944699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9dc0d2230 0x7fd9dc166d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:12.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.342+0000 7fd9dbe00640 1 -- 172.21.15.86:0/2304944699 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9dc15c150 con 0x7fd9dc0b1dd0 2024-04-25T14:18:12.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.342+0000 7fd9d3400640 1 --2- 172.21.15.86:0/2304944699 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9dc0d1b50 0x7fd9dc15bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:18:12.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.342+0000 7fd9dbe00640 1 --2- 172.21.15.86:0/2304944699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9dc0b1dd0 0x7fd9dc14d9d0 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00e8d0 tx=0x7fd9cc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:12.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.342+0000 7fd9d8c00640 1 -- 172.21.15.86:0/2304944699 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc017070 con 0x7fd9dc0b1dd0 2024-04-25T14:18:12.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.342+0000 7fd9d8c00640 1 -- 172.21.15.86:0/2304944699 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9cc00c940 con 0x7fd9dc0b1dd0 2024-04-25T14:18:12.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.342+0000 7fd9e2a00640 1 -- 172.21.15.86:0/2304944699 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9dc0af490 con 0x7fd9dc0b1dd0 2024-04-25T14:18:12.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.342+0000 7fd9d8c00640 1 -- 172.21.15.86:0/2304944699 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc00ccf0 con 0x7fd9dc0b1dd0 2024-04-25T14:18:12.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.342+0000 7fd9e2a00640 1 -- 172.21.15.86:0/2304944699 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9dc0afa50 con 0x7fd9dc0b1dd0 2024-04-25T14:18:12.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.342+0000 7fd9e2a00640 1 -- 172.21.15.86:0/2304944699 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd9dc0af240 con 0x7fd9dc0b1dd0 2024-04-25T14:18:12.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.342+0000 7fd9d8c00640 1 -- 172.21.15.86:0/2304944699 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7fd9cc01a070 con 0x7fd9dc0b1dd0 2024-04-25T14:18:12.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.342+0000 7fd9d8c00640 1 --2- 172.21.15.86:0/2304944699 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fd9b806f430 0x7fd9b8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:12.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.342+0000 7fd9d8c00640 1 -- 172.21.15.86:0/2304944699 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 6023+0+0 (secure 0 0 0) 0x7fd9cc08f9c0 con 0x7fd9dc0b1dd0 2024-04-25T14:18:12.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.342+0000 7fd9d3400640 1 --2- 172.21.15.86:0/2304944699 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fd9b806f430 0x7fd9b8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:12.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.346+0000 7fd9d8c00640 1 --2- 172.21.15.86:0/2304944699 >> [v2:172.21.15.155:6801/2597410484,v1:172.21.15.155:6803/2597410484] conn(0x7fd9b8075f10 0x7fd9b80783a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:12.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.346+0000 7fd9d8c00640 1 -- 172.21.15.86:0/2304944699 --> [v2:172.21.15.155:6801/2597410484,v1:172.21.15.155:6803/2597410484] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd9cc016ec0 con 0x7fd9b8075f10 2024-04-25T14:18:12.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.346+0000 7fd9d8c00640 1 -- 172.21.15.86:0/2304944699 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=160) ==== 24+0+0 (secure 0 0 0) 0x7fd9cc08fe50 con 0x7fd9dc0b1dd0 2024-04-25T14:18:12.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.346+0000 7fd9e0a00640 1 --2- 172.21.15.86:0/2304944699 >> [v2:172.21.15.155:6801/2597410484,v1:172.21.15.155:6803/2597410484] conn(0x7fd9b8075f10 0x7fd9b80783a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:12.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.346+0000 7fd9d3400640 1 --2- 172.21.15.86:0/2304944699 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fd9b806f430 0x7fd9b8071940 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc0d1890 tx=0x7fd9c4009040 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:12.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.346+0000 7fd9e0a00640 1 --2- 172.21.15.86:0/2304944699 >> [v2:172.21.15.155:6801/2597410484,v1:172.21.15.155:6803/2597410484] conn(0x7fd9b8075f10 0x7fd9b80783a0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:12.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.346+0000 7fd9d8c00640 1 -- 172.21.15.86:0/2304944699 <== osd.6 v2:172.21.15.155:6801/2597410484 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd9cc016ec0 con 0x7fd9b8075f10 2024-04-25T14:18:12.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.358+0000 7fd9e2a00640 1 -- 172.21.15.86:0/2304944699 --> [v2:172.21.15.155:6801/2597410484,v1:172.21.15.155:6803/2597410484] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd9dc050750 con 0x7fd9b8075f10 2024-04-25T14:18:12.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.358+0000 7fd9d8c00640 1 -- 172.21.15.86:0/2304944699 <== osd.6 v2:172.21.15.155:6801/2597410484 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd9dc050750 con 0x7fd9b8075f10 2024-04-25T14:18:12.354 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:18:12.354 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T14:18:12.355 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:18:12.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.358+0000 7fd9e2a00640 1 -- 172.21.15.86:0/2304944699 >> [v2:172.21.15.155:6801/2597410484,v1:172.21.15.155:6803/2597410484] conn(0x7fd9b8075f10 msgr2=0x7fd9b80783a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:12.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.358+0000 7fd9e2a00640 1 --2- 172.21.15.86:0/2304944699 >> [v2:172.21.15.155:6801/2597410484,v1:172.21.15.155:6803/2597410484] conn(0x7fd9b8075f10 0x7fd9b80783a0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:12.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.362+0000 7fd9e2a00640 1 -- 172.21.15.86:0/2304944699 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fd9b806f430 msgr2=0x7fd9b8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:12.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.362+0000 7fd9e2a00640 1 --2- 172.21.15.86:0/2304944699 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fd9b806f430 0x7fd9b8071940 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc0d1890 tx=0x7fd9c4009040 comp rx=0 tx=0).stop 2024-04-25T14:18:12.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.362+0000 7fd9e2a00640 1 -- 172.21.15.86:0/2304944699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9dc0b1dd0 msgr2=0x7fd9dc14d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:12.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.362+0000 7fd9e2a00640 1 --2- 172.21.15.86:0/2304944699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9dc0b1dd0 0x7fd9dc14d9d0 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00e8d0 tx=0x7fd9cc00ecd0 comp rx=0 tx=0).stop 2024-04-25T14:18:12.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.362+0000 7fd9e0a00640 1 -- 172.21.15.86:0/2304944699 reap_dead start 2024-04-25T14:18:12.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.362+0000 7fd9e2a00640 1 -- 172.21.15.86:0/2304944699 shutdown_connections 2024-04-25T14:18:12.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.362+0000 7fd9e2a00640 1 -- 172.21.15.86:0/2304944699 >> 172.21.15.86:0/2304944699 conn(0x7fd9dc073c90 msgr2=0x7fd9dc068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:12.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.362+0000 7fd9e2a00640 1 -- 172.21.15.86:0/2304944699 shutdown_connections 2024-04-25T14:18:12.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.362+0000 7fd9e2a00640 1 -- 172.21.15.86:0/2304944699 wait complete. 2024-04-25T14:18:12.373 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T14:18:12.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.526+0000 7f1069000640 1 Processor -- start 2024-04-25T14:18:12.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.526+0000 7f1069000640 1 -- start start 2024-04-25T14:18:12.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.526+0000 7f1069000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f10640d2d70 0x7f1064163780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:12.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.526+0000 7f1069000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10640d16b0 0x7f10640d1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:12.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.526+0000 7f1069000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10640cf9e0 con 0x7f1064070e70 2024-04-25T14:18:12.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.526+0000 7f1069000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1064163e80 con 0x7f10640d2d70 2024-04-25T14:18:12.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.526+0000 7f1069000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1064164160 con 0x7f10640d16b0 2024-04-25T14:18:12.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.530+0000 7f105aa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10640d16b0 0x7f10640d1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:12.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.530+0000 7f1063400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f10640d2d70 0x7f1064163780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:12.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.530+0000 7f1063e00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f1064070e70 0x7f1064071280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:54710/0 (socket says 172.21.15.86:54710) 2024-04-25T14:18:12.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.530+0000 7f1063e00640 1 -- 172.21.15.86:0/3610581210 learned_addr learned my addr 172.21.15.86:0/3610581210 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:18:12.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.530+0000 7f105aa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10640d16b0 0x7f10640d1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.86:40504/0 (socket says 172.21.15.86:40504) 2024-04-25T14:18:12.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.530+0000 7f1062a00640 1 -- 172.21.15.86:0/3610581210 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1301465663 0 0) 0x7f105c0014a0 con 0x7f1064070e70 2024-04-25T14:18:12.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.530+0000 7f1062a00640 1 -- 172.21.15.86:0/3610581210 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1048003530 con 0x7f1064070e70 2024-04-25T14:18:12.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.530+0000 7f105aa00640 1 -- 172.21.15.86:0/3610581210 >> v2:172.21.15.155:3300/0 conn(0x7f10640d2d70 msgr2=0x7f1064163780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:12.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.530+0000 7f105aa00640 1 --2- 172.21.15.86:0/3610581210 >> v2:172.21.15.155:3300/0 conn(0x7f10640d2d70 0x7f1064163780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:12.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.530+0000 7f105aa00640 1 -- 172.21.15.86:0/3610581210 >> v1:172.21.15.155:6789/0 conn(0x7f1064070e70 legacy=0x7f1064071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:12.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.530+0000 7f105aa00640 1 -- 172.21.15.86:0/3610581210 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1064164440 con 0x7f10640d16b0 2024-04-25T14:18:12.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.530+0000 7f1063400640 1 --2- 172.21.15.86:0/3610581210 >> v2:172.21.15.155:3300/0 conn(0x7f10640d2d70 0x7f1064163780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:18:12.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.530+0000 7f105aa00640 1 --2- 172.21.15.86:0/3610581210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10640d16b0 0x7f10640d1ac0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f104c0097a0 tx=0x7f104c01c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1ab7dfdfd0e1872 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:12.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.534+0000 7f1062a00640 1 -- 172.21.15.86:0/3610581210 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104c0b0070 con 0x7f10640d16b0 2024-04-25T14:18:12.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.534+0000 7f1062a00640 1 -- 172.21.15.86:0/3610581210 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f104c0ab600 con 0x7f10640d16b0 2024-04-25T14:18:12.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.534+0000 7f1062a00640 1 -- 172.21.15.86:0/3610581210 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104c0ab9d0 con 0x7f10640d16b0 2024-04-25T14:18:12.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.534+0000 7f1069000640 1 -- 172.21.15.86:0/3610581210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10640d16b0 msgr2=0x7f10640d1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:12.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.534+0000 7f1069000640 1 --2- 172.21.15.86:0/3610581210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10640d16b0 0x7f10640d1ac0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f104c0097a0 tx=0x7f104c01c3f0 comp rx=0 tx=0).stop 2024-04-25T14:18:12.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.534+0000 7f1069000640 1 -- 172.21.15.86:0/3610581210 shutdown_connections 2024-04-25T14:18:12.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.534+0000 7f1069000640 1 --2- 172.21.15.86:0/3610581210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10640d16b0 0x7f10640d1ac0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:12.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.534+0000 7f1069000640 1 --2- 172.21.15.86:0/3610581210 >> v2:172.21.15.155:3300/0 conn(0x7f10640d2d70 0x7f1064163780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:12.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.534+0000 7f1069000640 1 -- 172.21.15.86:0/3610581210 >> 172.21.15.86:0/3610581210 conn(0x7f10640cfea0 msgr2=0x7f10640d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:12.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.534+0000 7f1069000640 1 -- 172.21.15.86:0/3610581210 shutdown_connections 2024-04-25T14:18:12.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.534+0000 7f1069000640 1 -- 172.21.15.86:0/3610581210 wait complete. 2024-04-25T14:18:12.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.534+0000 7f1069000640 1 Processor -- start 2024-04-25T14:18:12.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.534+0000 7f1069000640 1 -- start start 2024-04-25T14:18:12.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.538+0000 7f1069000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1064070e70 0x7f1064159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:12.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.538+0000 7f1069000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10640d16b0 0x7f10640cce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:12.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.538+0000 7f1069000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10640d2d70 0x7f1064160c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:12.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.538+0000 7f1069000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f106416d660 con 0x7f10640d16b0 2024-04-25T14:18:12.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.538+0000 7f1069000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f106416d4e0 con 0x7f10640d2d70 2024-04-25T14:18:12.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.538+0000 7f1069000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f106416d7e0 con 0x7f1064070e70 2024-04-25T14:18:12.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.538+0000 7f105aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10640d16b0 0x7f10640cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:12.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.538+0000 7f105aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10640d16b0 0x7f10640cce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46884/0 (socket says 172.21.15.86:46884) 2024-04-25T14:18:12.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.538+0000 7f105aa00640 1 -- 172.21.15.86:0/3062567369 learned_addr learned my addr 172.21.15.86:0/3062567369 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:12.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.538+0000 7f1063400640 1 --2- 172.21.15.86:0/3062567369 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1064070e70 0x7f1064159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:12.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.538+0000 7f1063e00640 1 --2- 172.21.15.86:0/3062567369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10640d2d70 0x7f1064160c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:12.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.538+0000 7f105aa00640 1 -- 172.21.15.86:0/3062567369 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1064070e70 msgr2=0x7f1064159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:12.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.538+0000 7f105aa00640 1 --2- 172.21.15.86:0/3062567369 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1064070e70 0x7f1064159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:12.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.538+0000 7f105aa00640 1 -- 172.21.15.86:0/3062567369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10640d2d70 msgr2=0x7f1064160c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:12.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.538+0000 7f105aa00640 1 --2- 172.21.15.86:0/3062567369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10640d2d70 0x7f1064160c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:12.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.538+0000 7f105aa00640 1 -- 172.21.15.86:0/3062567369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10640cda00 con 0x7f10640d16b0 2024-04-25T14:18:12.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.538+0000 7f1063e00640 1 --2- 172.21.15.86:0/3062567369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10640d2d70 0x7f1064160c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:18:12.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.538+0000 7f105aa00640 1 --2- 172.21.15.86:0/3062567369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10640d16b0 0x7f10640cce20 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f104c0ae040 tx=0x7f104c019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:12.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.538+0000 7f105be00640 1 -- 172.21.15.86:0/3062567369 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104c0b8020 con 0x7f10640d16b0 2024-04-25T14:18:12.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.538+0000 7f105be00640 1 -- 172.21.15.86:0/3062567369 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f104c005bc0 con 0x7f10640d16b0 2024-04-25T14:18:12.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.538+0000 7f105be00640 1 -- 172.21.15.86:0/3062567369 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104c0b0040 con 0x7f10640d16b0 2024-04-25T14:18:12.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.538+0000 7f1069000640 1 -- 172.21.15.86:0/3062567369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10640cdce0 con 0x7f10640d16b0 2024-04-25T14:18:12.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.538+0000 7f1069000640 1 -- 172.21.15.86:0/3062567369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10640c9a90 con 0x7f10640d16b0 2024-04-25T14:18:12.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.538+0000 7f1069000640 1 -- 172.21.15.86:0/3062567369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10640cd770 con 0x7f10640d16b0 2024-04-25T14:18:12.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.542+0000 7f105be00640 1 -- 172.21.15.86:0/3062567369 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7f104c0bb070 con 0x7f10640d16b0 2024-04-25T14:18:12.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.542+0000 7f105be00640 1 --2- 172.21.15.86:0/3062567369 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f104006f400 0x7f1040071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:12.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.542+0000 7f105be00640 1 -- 172.21.15.86:0/3062567369 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 6023+0+0 (secure 0 0 0) 0x7f104c128940 con 0x7f10640d16b0 2024-04-25T14:18:12.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.542+0000 7f1063400640 1 --2- 172.21.15.86:0/3062567369 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f104006f400 0x7f1040071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:12.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.542+0000 7f1063400640 1 --2- 172.21.15.86:0/3062567369 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f104006f400 0x7f1040071910 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1054009580 tx=0x7f1054009190 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:12.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.546+0000 7f105be00640 1 -- 172.21.15.86:0/3062567369 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f104c0c2020 con 0x7f10640d16b0 2024-04-25T14:18:12.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.698+0000 7f1069000640 1 -- 172.21.15.86:0/3062567369 --> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f1064076b80 con 0x7f104006f400 2024-04-25T14:18:12.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.698+0000 7f105be00640 1 -- 172.21.15.86:0/3062567369 <== mgr.11343 v2:172.21.15.86:6832/3174179029 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1064076b80 con 0x7f104006f400 2024-04-25T14:18:12.694 INFO:teuthology.orchestra.run.smithi086.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T14:18:12.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.702+0000 7f1069000640 1 -- 172.21.15.86:0/3062567369 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f104006f400 msgr2=0x7f1040071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:12.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.702+0000 7f1069000640 1 --2- 172.21.15.86:0/3062567369 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f104006f400 0x7f1040071910 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1054009580 tx=0x7f1054009190 comp rx=0 tx=0).stop 2024-04-25T14:18:12.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.702+0000 7f1069000640 1 -- 172.21.15.86:0/3062567369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10640d16b0 msgr2=0x7f10640cce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:12.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.702+0000 7f1069000640 1 --2- 172.21.15.86:0/3062567369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10640d16b0 0x7f10640cce20 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f104c0ae040 tx=0x7f104c019190 comp rx=0 tx=0).stop 2024-04-25T14:18:12.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.702+0000 7f1069000640 1 -- 172.21.15.86:0/3062567369 shutdown_connections 2024-04-25T14:18:12.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.702+0000 7f1069000640 1 --2- 172.21.15.86:0/3062567369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10640d2d70 0x7f1064160c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:12.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.702+0000 7f1069000640 1 --2- 172.21.15.86:0/3062567369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10640d16b0 0x7f10640cce20 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:12.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.702+0000 7f1069000640 1 --2- 172.21.15.86:0/3062567369 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f104006f400 0x7f1040071910 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:12.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.702+0000 7f1069000640 1 --2- 172.21.15.86:0/3062567369 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1064070e70 0x7f1064159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:12.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.702+0000 7f1069000640 1 -- 172.21.15.86:0/3062567369 >> 172.21.15.86:0/3062567369 conn(0x7f10640cfea0 msgr2=0x7f1064071280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:12.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.702+0000 7f1069000640 1 -- 172.21.15.86:0/3062567369 shutdown_connections 2024-04-25T14:18:12.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.702+0000 7f1069000640 1 -- 172.21.15.86:0/3062567369 wait complete. 2024-04-25T14:18:12.715 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T14:18:12.715 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T14:18:12.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.870+0000 7f573c200640 1 Processor -- start 2024-04-25T14:18:12.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.870+0000 7f573c200640 1 -- start start 2024-04-25T14:18:12.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.874+0000 7f573c200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5734166660 0x7f5734168a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:12.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.874+0000 7f573c200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57340b0b20 con 0x7f573415e270 2024-04-25T14:18:12.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.874+0000 7f573c200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f573406d220 con 0x7f57340b0540 2024-04-25T14:18:12.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.874+0000 7f573c200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f573406d870 con 0x7f5734166660 2024-04-25T14:18:12.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.874+0000 7f573a200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5734166660 0x7f5734168a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:12.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.874+0000 7f573a200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5734166660 0x7f5734168a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46896/0 (socket says 172.21.15.86:46896) 2024-04-25T14:18:12.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.874+0000 7f573a200640 1 -- 172.21.15.86:0/344932296 learned_addr learned my addr 172.21.15.86:0/344932296 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:12.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.874+0000 7f573a200640 1 -- 172.21.15.86:0/344932296 >> v1:172.21.15.155:6789/0 conn(0x7f57340b0540 legacy=0x7f57340ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:12.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.874+0000 7f573a200640 1 -- 172.21.15.86:0/344932296 >> v1:172.21.15.86:6789/0 conn(0x7f573415e270 legacy=0x7f573415e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:12.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.874+0000 7f573a200640 1 -- 172.21.15.86:0/344932296 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f573406d500 con 0x7f5734166660 2024-04-25T14:18:12.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.878+0000 7f573a200640 1 --2- 172.21.15.86:0/344932296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5734166660 0x7f5734168a80 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7f572c009870 tx=0x7f572c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66b1d1c8adfcea3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:12.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.878+0000 7f5733e00640 1 -- 172.21.15.86:0/344932296 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c0b0070 con 0x7f5734166660 2024-04-25T14:18:12.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.878+0000 7f5733e00640 1 -- 172.21.15.86:0/344932296 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f572c0ab600 con 0x7f5734166660 2024-04-25T14:18:12.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.878+0000 7f5733e00640 1 -- 172.21.15.86:0/344932296 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c0ab9d0 con 0x7f5734166660 2024-04-25T14:18:12.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.878+0000 7f573c200640 1 -- 172.21.15.86:0/344932296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5734166660 msgr2=0x7f5734168a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:12.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.878+0000 7f573c200640 1 --2- 172.21.15.86:0/344932296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5734166660 0x7f5734168a80 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7f572c009870 tx=0x7f572c01c4d0 comp rx=0 tx=0).stop 2024-04-25T14:18:12.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.878+0000 7f573c200640 1 -- 172.21.15.86:0/344932296 shutdown_connections 2024-04-25T14:18:12.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.878+0000 7f573c200640 1 --2- 172.21.15.86:0/344932296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5734166660 0x7f5734168a80 unknown :-1 s=CLOSED pgs=2520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:12.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.878+0000 7f573c200640 1 -- 172.21.15.86:0/344932296 >> 172.21.15.86:0/344932296 conn(0x7f57340b4a40 msgr2=0x7f57340b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:12.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.878+0000 7f573c200640 1 -- 172.21.15.86:0/344932296 shutdown_connections 2024-04-25T14:18:12.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.878+0000 7f573c200640 1 -- 172.21.15.86:0/344932296 wait complete. 2024-04-25T14:18:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.882+0000 7f573c200640 1 Processor -- start 2024-04-25T14:18:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.882+0000 7f573c200640 1 -- start start 2024-04-25T14:18:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.882+0000 7f573c200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57340b0540 0x7f57340cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.882+0000 7f573c200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f573415e270 0x7f57341502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.882+0000 7f573c200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5734166660 0x7f57340cf480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.882+0000 7f573c200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f573416bb70 con 0x7f5734166660 2024-04-25T14:18:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.882+0000 7f573c200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f573416b9f0 con 0x7f57340b0540 2024-04-25T14:18:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.882+0000 7f573c200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f573416bcf0 con 0x7f573415e270 2024-04-25T14:18:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.882+0000 7f573a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5734166660 0x7f57340cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.882+0000 7f573a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5734166660 0x7f57340cf480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46904/0 (socket says 172.21.15.86:46904) 2024-04-25T14:18:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.882+0000 7f573a200640 1 -- 172.21.15.86:0/3981659464 learned_addr learned my addr 172.21.15.86:0/3981659464 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.882+0000 7f5739800640 1 --2- 172.21.15.86:0/3981659464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57340b0540 0x7f57340cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.882+0000 7f5738e00640 1 --2- 172.21.15.86:0/3981659464 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f573415e270 0x7f57341502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.882+0000 7f573a200640 1 -- 172.21.15.86:0/3981659464 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f573415e270 msgr2=0x7f57341502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.882+0000 7f573a200640 1 --2- 172.21.15.86:0/3981659464 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f573415e270 0x7f57341502b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.882+0000 7f573a200640 1 -- 172.21.15.86:0/3981659464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57340b0540 msgr2=0x7f57340cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.882+0000 7f573a200640 1 --2- 172.21.15.86:0/3981659464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57340b0540 0x7f57340cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.882+0000 7f573a200640 1 -- 172.21.15.86:0/3981659464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57340cde30 con 0x7f5734166660 2024-04-25T14:18:12.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.882+0000 7f5739800640 1 --2- 172.21.15.86:0/3981659464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57340b0540 0x7f57340cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:18:12.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.882+0000 7f573a200640 1 --2- 172.21.15.86:0/3981659464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5734166660 0x7f57340cf480 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f572c0190e0 tx=0x7f572c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:12.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.882+0000 7f5731600640 1 -- 172.21.15.86:0/3981659464 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c0ba020 con 0x7f5734166660 2024-04-25T14:18:12.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.882+0000 7f5731600640 1 -- 172.21.15.86:0/3981659464 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f572c0aa6a0 con 0x7f5734166660 2024-04-25T14:18:12.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.882+0000 7f5731600640 1 -- 172.21.15.86:0/3981659464 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c0b0040 con 0x7f5734166660 2024-04-25T14:18:12.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.882+0000 7f573c200640 1 -- 172.21.15.86:0/3981659464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57340d0570 con 0x7f5734166660 2024-04-25T14:18:12.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.886+0000 7f573c200640 1 -- 172.21.15.86:0/3981659464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f573406fd40 con 0x7f5734166660 2024-04-25T14:18:12.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.886+0000 7f573c200640 1 -- 172.21.15.86:0/3981659464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5734050750 con 0x7f5734166660 2024-04-25T14:18:12.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.886+0000 7f5731600640 1 -- 172.21.15.86:0/3981659464 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7f572c0bd020 con 0x7f5734166660 2024-04-25T14:18:12.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.886+0000 7f5731600640 1 --2- 172.21.15.86:0/3981659464 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f571406f430 0x7f5714071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:12.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.886+0000 7f5731600640 1 -- 172.21.15.86:0/3981659464 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 6023+0+0 (secure 0 0 0) 0x7f572c129cc0 con 0x7f5734166660 2024-04-25T14:18:12.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.886+0000 7f5739800640 1 --2- 172.21.15.86:0/3981659464 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f571406f430 0x7f5714071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:12.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.886+0000 7f5731600640 1 --2- 172.21.15.86:0/3981659464 >> [v2:172.21.15.155:6816/2704073457,v1:172.21.15.155:6819/2704073457] conn(0x7f5714075f10 0x7f57140783a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:12.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.886+0000 7f5731600640 1 -- 172.21.15.86:0/3981659464 --> [v2:172.21.15.155:6816/2704073457,v1:172.21.15.155:6819/2704073457] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5714078a70 con 0x7f5714075f10 2024-04-25T14:18:12.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.886+0000 7f5731600640 1 -- 172.21.15.86:0/3981659464 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=160) ==== 24+0+0 (secure 0 0 0) 0x7f572c12a150 con 0x7f5734166660 2024-04-25T14:18:12.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.886+0000 7f5738e00640 1 --2- 172.21.15.86:0/3981659464 >> [v2:172.21.15.155:6816/2704073457,v1:172.21.15.155:6819/2704073457] conn(0x7f5714075f10 0x7f57140783a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:12.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.886+0000 7f5739800640 1 --2- 172.21.15.86:0/3981659464 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f571406f430 0x7f5714071940 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f57340cd850 tx=0x7f5728009040 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:12.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.890+0000 7f5738e00640 1 --2- 172.21.15.86:0/3981659464 >> [v2:172.21.15.155:6816/2704073457,v1:172.21.15.155:6819/2704073457] conn(0x7f5714075f10 0x7f57140783a0 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:12.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.890+0000 7f5731600640 1 -- 172.21.15.86:0/3981659464 <== osd.7 v2:172.21.15.155:6816/2704073457 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5714078a70 con 0x7f5714075f10 2024-04-25T14:18:12.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.902+0000 7f573c200640 1 -- 172.21.15.86:0/3981659464 --> [v2:172.21.15.155:6816/2704073457,v1:172.21.15.155:6819/2704073457] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f57340d0320 con 0x7f5714075f10 2024-04-25T14:18:12.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.902+0000 7f5731600640 1 -- 172.21.15.86:0/3981659464 <== osd.7 v2:172.21.15.155:6816/2704073457 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f57340d0320 con 0x7f5714075f10 2024-04-25T14:18:12.898 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-25T14:18:12.898 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T14:18:12.898 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-25T14:18:12.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.902+0000 7f573c200640 1 -- 172.21.15.86:0/3981659464 >> [v2:172.21.15.155:6816/2704073457,v1:172.21.15.155:6819/2704073457] conn(0x7f5714075f10 msgr2=0x7f57140783a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:12.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.902+0000 7f573c200640 1 --2- 172.21.15.86:0/3981659464 >> [v2:172.21.15.155:6816/2704073457,v1:172.21.15.155:6819/2704073457] conn(0x7f5714075f10 0x7f57140783a0 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:12.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.902+0000 7f573c200640 1 -- 172.21.15.86:0/3981659464 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f571406f430 msgr2=0x7f5714071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.902+0000 7f573c200640 1 --2- 172.21.15.86:0/3981659464 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f571406f430 0x7f5714071940 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f57340cd850 tx=0x7f5728009040 comp rx=0 tx=0).stop 2024-04-25T14:18:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.902+0000 7f573c200640 1 -- 172.21.15.86:0/3981659464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5734166660 msgr2=0x7f57340cf480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.902+0000 7f573c200640 1 --2- 172.21.15.86:0/3981659464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5734166660 0x7f57340cf480 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f572c0190e0 tx=0x7f572c01ad10 comp rx=0 tx=0).stop 2024-04-25T14:18:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.902+0000 7f573a200640 1 -- 172.21.15.86:0/3981659464 reap_dead start 2024-04-25T14:18:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.902+0000 7f573c200640 1 -- 172.21.15.86:0/3981659464 shutdown_connections 2024-04-25T14:18:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.902+0000 7f573c200640 1 -- 172.21.15.86:0/3981659464 >> 172.21.15.86:0/3981659464 conn(0x7f57340b4a40 msgr2=0x7f573415cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.902+0000 7f573c200640 1 -- 172.21.15.86:0/3981659464 shutdown_connections 2024-04-25T14:18:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:12.902+0000 7f573c200640 1 -- 172.21.15.86:0/3981659464 wait complete. 2024-04-25T14:18:12.917 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T14:18:13.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.070+0000 7fc4a7a00640 1 Processor -- start 2024-04-25T14:18:13.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.070+0000 7fc4a7a00640 1 -- start start 2024-04-25T14:18:13.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.074+0000 7fc4a7a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4a00d16e0 0x7fc4a00d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:13.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.074+0000 7fc4a7a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4a00d2030 con 0x7fc4a00730b0 2024-04-25T14:18:13.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.074+0000 7fc4a7a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4a0072bd0 con 0x7fc4a00cfca0 2024-04-25T14:18:13.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.074+0000 7fc4a7a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc4a0070470 con 0x7fc4a00d16e0 2024-04-25T14:18:13.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.074+0000 7fc4a5a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc4a00730b0 0x7fc4a00734c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41896/0 (socket says 172.21.15.86:41896) 2024-04-25T14:18:13.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.074+0000 7fc4a5a00640 1 -- 172.21.15.86:0/200717383 learned_addr learned my addr 172.21.15.86:0/200717383 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:18:13.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.074+0000 7fc49fe00640 1 --2- 172.21.15.86:0/200717383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4a00d16e0 0x7fc4a00d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:13.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.074+0000 7fc49f400640 1 -- 172.21.15.86:0/200717383 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1854399744 0 0) 0x7fc498001120 con 0x7fc4a00730b0 2024-04-25T14:18:13.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.074+0000 7fc49f400640 1 -- 172.21.15.86:0/200717383 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc484003530 con 0x7fc4a00730b0 2024-04-25T14:18:13.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.074+0000 7fc49f400640 1 -- 172.21.15.86:0/200717383 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1542266066 0 0) 0x7fc4a0072bd0 con 0x7fc4a00cfca0 2024-04-25T14:18:13.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.074+0000 7fc49f400640 1 -- 172.21.15.86:0/200717383 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc498001120 con 0x7fc4a00cfca0 2024-04-25T14:18:13.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.074+0000 7fc49fe00640 1 -- 172.21.15.86:0/200717383 >> v1:172.21.15.155:6789/0 conn(0x7fc4a00cfca0 legacy=0x7fc4a006f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:13.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.074+0000 7fc49fe00640 1 -- 172.21.15.86:0/200717383 >> v1:172.21.15.86:6789/0 conn(0x7fc4a00730b0 legacy=0x7fc4a00734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:13.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.074+0000 7fc49fe00640 1 -- 172.21.15.86:0/200717383 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a006fbd0 con 0x7fc4a00d16e0 2024-04-25T14:18:13.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.078+0000 7fc49fe00640 1 --2- 172.21.15.86:0/200717383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4a00d16e0 0x7fc4a00d1af0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0072da0 tx=0x7fc48c01c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f7ff1d553bceca6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:13.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.078+0000 7fc49f400640 1 -- 172.21.15.86:0/200717383 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c0b1070 con 0x7fc4a00d16e0 2024-04-25T14:18:13.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.078+0000 7fc49f400640 1 -- 172.21.15.86:0/200717383 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc48c0ac630 con 0x7fc4a00d16e0 2024-04-25T14:18:13.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.078+0000 7fc49f400640 1 -- 172.21.15.86:0/200717383 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c0a8050 con 0x7fc4a00d16e0 2024-04-25T14:18:13.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.078+0000 7fc4a7a00640 1 -- 172.21.15.86:0/200717383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4a00d16e0 msgr2=0x7fc4a00d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:13.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.078+0000 7fc4a7a00640 1 --2- 172.21.15.86:0/200717383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4a00d16e0 0x7fc4a00d1af0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0072da0 tx=0x7fc48c01c4c0 comp rx=0 tx=0).stop 2024-04-25T14:18:13.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.078+0000 7fc4a7a00640 1 -- 172.21.15.86:0/200717383 shutdown_connections 2024-04-25T14:18:13.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.078+0000 7fc4a7a00640 1 --2- 172.21.15.86:0/200717383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4a00d16e0 0x7fc4a00d1af0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:13.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.078+0000 7fc4a7a00640 1 -- 172.21.15.86:0/200717383 >> 172.21.15.86:0/200717383 conn(0x7fc4a00b4650 msgr2=0x7fc4a00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:13.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.078+0000 7fc4a7a00640 1 -- 172.21.15.86:0/200717383 shutdown_connections 2024-04-25T14:18:13.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.078+0000 7fc4a7a00640 1 -- 172.21.15.86:0/200717383 wait complete. 2024-04-25T14:18:13.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.078+0000 7fc4a7a00640 1 Processor -- start 2024-04-25T14:18:13.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.078+0000 7fc4a7a00640 1 -- start start 2024-04-25T14:18:13.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.082+0000 7fc4a7a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4a00730b0 0x7fc4a00ae130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:13.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.082+0000 7fc4a7a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4a00cfca0 0x7fc4a009a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:13.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.082+0000 7fc4a7a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4a00d16e0 0x7fc4a015a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:13.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.082+0000 7fc4a7a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a016aa80 con 0x7fc4a00cfca0 2024-04-25T14:18:13.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.082+0000 7fc4a7a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a016a900 con 0x7fc4a00730b0 2024-04-25T14:18:13.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.082+0000 7fc4a7a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc4a016ac00 con 0x7fc4a00d16e0 2024-04-25T14:18:13.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.082+0000 7fc49fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4a00cfca0 0x7fc4a009a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:13.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.082+0000 7fc49fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4a00cfca0 0x7fc4a009a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46914/0 (socket says 172.21.15.86:46914) 2024-04-25T14:18:13.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.082+0000 7fc49fe00640 1 -- 172.21.15.86:0/2879020189 learned_addr learned my addr 172.21.15.86:0/2879020189 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:13.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.082+0000 7fc4a5000640 1 --2- 172.21.15.86:0/2879020189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4a00730b0 0x7fc4a00ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:13.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.082+0000 7fc4a5a00640 1 --2- 172.21.15.86:0/2879020189 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4a00d16e0 0x7fc4a015a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:13.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.082+0000 7fc49fe00640 1 -- 172.21.15.86:0/2879020189 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4a00d16e0 msgr2=0x7fc4a015a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:13.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.082+0000 7fc49fe00640 1 --2- 172.21.15.86:0/2879020189 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4a00d16e0 0x7fc4a015a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:13.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.082+0000 7fc49fe00640 1 -- 172.21.15.86:0/2879020189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4a00730b0 msgr2=0x7fc4a00ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:13.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.082+0000 7fc49fe00640 1 --2- 172.21.15.86:0/2879020189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4a00730b0 0x7fc4a00ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:13.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.082+0000 7fc49fe00640 1 -- 172.21.15.86:0/2879020189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a00aebe0 con 0x7fc4a00cfca0 2024-04-25T14:18:13.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.082+0000 7fc49fe00640 1 --2- 172.21.15.86:0/2879020189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4a00cfca0 0x7fc4a009a740 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0af040 tx=0x7fc48c019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:13.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.082+0000 7fc49cc00640 1 -- 172.21.15.86:0/2879020189 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c0b1040 con 0x7fc4a00cfca0 2024-04-25T14:18:13.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.082+0000 7fc49cc00640 1 -- 172.21.15.86:0/2879020189 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc48c0bb070 con 0x7fc4a00cfca0 2024-04-25T14:18:13.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.082+0000 7fc49cc00640 1 -- 172.21.15.86:0/2879020189 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a00743e0 con 0x7fc4a00cfca0 2024-04-25T14:18:13.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.082+0000 7fc4a7a00640 1 -- 172.21.15.86:0/2879020189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a00aee80 con 0x7fc4a00cfca0 2024-04-25T14:18:13.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.082+0000 7fc4a7a00640 1 -- 172.21.15.86:0/2879020189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a015b7d0 con 0x7fc4a00cfca0 2024-04-25T14:18:13.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.082+0000 7fc4a7a00640 1 -- 172.21.15.86:0/2879020189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4a009f4b0 con 0x7fc4a00cfca0 2024-04-25T14:18:13.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.086+0000 7fc49cc00640 1 -- 172.21.15.86:0/2879020189 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7fc48c0b5630 con 0x7fc4a00cfca0 2024-04-25T14:18:13.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.086+0000 7fc49cc00640 1 --2- 172.21.15.86:0/2879020189 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fc48006f400 0x7fc480071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:13.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.086+0000 7fc49cc00640 1 -- 172.21.15.86:0/2879020189 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 6023+0+0 (secure 0 0 0) 0x7fc48c1293d0 con 0x7fc4a00cfca0 2024-04-25T14:18:13.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.086+0000 7fc4a5000640 1 --2- 172.21.15.86:0/2879020189 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fc48006f400 0x7fc480071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:13.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.086+0000 7fc4a5000640 1 --2- 172.21.15.86:0/2879020189 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fc48006f400 0x7fc480071910 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc490007fd0 tx=0x7fc490009040 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:13.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.090+0000 7fc49cc00640 1 -- 172.21.15.86:0/2879020189 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc48c0160b0 con 0x7fc4a00cfca0 2024-04-25T14:18:13.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.250+0000 7fc4a7a00640 1 -- 172.21.15.86:0/2879020189 --> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fc4a00af050 con 0x7fc48006f400 2024-04-25T14:18:13.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.250+0000 7fc49cc00640 1 -- 172.21.15.86:0/2879020189 <== mgr.11343 v2:172.21.15.86:6832/3174179029 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc4a00af050 con 0x7fc48006f400 2024-04-25T14:18:13.244 INFO:teuthology.orchestra.run.smithi086.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T14:18:13.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.250+0000 7fc4a7a00640 1 -- 172.21.15.86:0/2879020189 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fc48006f400 msgr2=0x7fc480071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:13.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.250+0000 7fc4a7a00640 1 --2- 172.21.15.86:0/2879020189 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fc48006f400 0x7fc480071910 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc490007fd0 tx=0x7fc490009040 comp rx=0 tx=0).stop 2024-04-25T14:18:13.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.250+0000 7fc4a7a00640 1 -- 172.21.15.86:0/2879020189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4a00cfca0 msgr2=0x7fc4a009a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:13.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.250+0000 7fc4a7a00640 1 --2- 172.21.15.86:0/2879020189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4a00cfca0 0x7fc4a009a740 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0af040 tx=0x7fc48c019260 comp rx=0 tx=0).stop 2024-04-25T14:18:13.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.250+0000 7fc4a7a00640 1 -- 172.21.15.86:0/2879020189 shutdown_connections 2024-04-25T14:18:13.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.250+0000 7fc4a7a00640 1 --2- 172.21.15.86:0/2879020189 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fc48006f400 0x7fc480071910 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:13.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.250+0000 7fc4a7a00640 1 --2- 172.21.15.86:0/2879020189 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4a00d16e0 0x7fc4a015a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:13.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.250+0000 7fc4a7a00640 1 --2- 172.21.15.86:0/2879020189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4a00cfca0 0x7fc4a009a740 unknown :-1 s=CLOSED pgs=2523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:13.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.250+0000 7fc4a7a00640 1 --2- 172.21.15.86:0/2879020189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4a00730b0 0x7fc4a00ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:13.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.250+0000 7fc4a7a00640 1 -- 172.21.15.86:0/2879020189 >> 172.21.15.86:0/2879020189 conn(0x7fc4a00b4650 msgr2=0x7fc4a0159130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:13.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.250+0000 7fc4a7a00640 1 -- 172.21.15.86:0/2879020189 shutdown_connections 2024-04-25T14:18:13.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.250+0000 7fc4a7a00640 1 -- 172.21.15.86:0/2879020189 wait complete. 2024-04-25T14:18:13.266 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:18:13.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.418+0000 7ff422600640 1 Processor -- start 2024-04-25T14:18:13.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff422600640 1 -- start start 2024-04-25T14:18:13.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff422600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff41c070330 0x7ff41c0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:13.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff422600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff41c0d2030 con 0x7ff41c0d1c20 2024-04-25T14:18:13.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff422600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff41c15e5c0 con 0x7ff41c0ce790 2024-04-25T14:18:13.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff422600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff41c15e8a0 con 0x7ff41c070330 2024-04-25T14:18:13.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff41aa00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff41c0d1c20 0x7ff41c06ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41908/0 (socket says 172.21.15.86:41908) 2024-04-25T14:18:13.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff41aa00640 1 -- 172.21.15.86:0/4182907029 learned_addr learned my addr 172.21.15.86:0/4182907029 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:18:13.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff41a000640 1 -- 172.21.15.86:0/4182907029 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3400011184 0 0) 0x7ff408001120 con 0x7ff41c0d1c20 2024-04-25T14:18:13.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff41b400640 1 --2- 172.21.15.86:0/4182907029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff41c070330 0x7ff41c0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:13.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff41a000640 1 -- 172.21.15.86:0/4182907029 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff400003530 con 0x7ff41c0d1c20 2024-04-25T14:18:13.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff41a000640 1 -- 172.21.15.86:0/4182907029 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 125960166 0 0) 0x7ff400003530 con 0x7ff41c0d1c20 2024-04-25T14:18:13.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff41a000640 1 -- 172.21.15.86:0/4182907029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff41c070330 msgr2=0x7ff41c0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:13.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff41a000640 1 --2- 172.21.15.86:0/4182907029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff41c070330 0x7ff41c0d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:13.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff41a000640 1 -- 172.21.15.86:0/4182907029 >> v1:172.21.15.155:6789/0 conn(0x7ff41c0ce790 legacy=0x7ff41c06f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:13.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff41a000640 1 -- 172.21.15.86:0/4182907029 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41c15eb80 con 0x7ff41c0d1c20 2024-04-25T14:18:13.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff41a000640 1 -- 172.21.15.86:0/4182907029 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7ff408001850 con 0x7ff41c0d1c20 2024-04-25T14:18:13.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff41a000640 1 -- 172.21.15.86:0/4182907029 >> v1:172.21.15.86:6789/0 conn(0x7ff41c0d1c20 legacy=0x7ff41c06ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:13.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff41a000640 1 --2- 172.21.15.86:0/4182907029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff400003a60 0x7ff400003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:13.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff41a000640 1 --2- 172.21.15.86:0/4182907029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff41c070330 0x7ff400004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:13.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff41a000640 1 --2- 172.21.15.86:0/4182907029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff400004d50 0x7ff4000051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:13.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff41a000640 1 -- 172.21.15.86:0/4182907029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c15e8a0 con 0x7ff400004d50 2024-04-25T14:18:13.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff41a000640 1 -- 172.21.15.86:0/4182907029 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff40000bc30 con 0x7ff400003a60 2024-04-25T14:18:13.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff41a000640 1 -- 172.21.15.86:0/4182907029 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff40000be30 con 0x7ff41c070330 2024-04-25T14:18:13.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff41be00640 1 --2- 172.21.15.86:0/4182907029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff400004d50 0x7ff4000051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:13.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff41aa00640 1 --2- 172.21.15.86:0/4182907029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff41c070330 0x7ff400004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:13.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff41b400640 1 --2- 172.21.15.86:0/4182907029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff400003a60 0x7ff400003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:13.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff41be00640 1 -- 172.21.15.86:0/4182907029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff41c070330 msgr2=0x7ff400004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:13.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff41be00640 1 --2- 172.21.15.86:0/4182907029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff41c070330 0x7ff400004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:13.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff41be00640 1 -- 172.21.15.86:0/4182907029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff400003a60 msgr2=0x7ff400003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:13.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff41be00640 1 --2- 172.21.15.86:0/4182907029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff400003a60 0x7ff400003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:13.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.422+0000 7ff41be00640 1 -- 172.21.15.86:0/4182907029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff400003530 con 0x7ff400004d50 2024-04-25T14:18:13.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.426+0000 7ff41be00640 1 --2- 172.21.15.86:0/4182907029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff400004d50 0x7ff4000051e0 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7ff414001750 tx=0x7ff41401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b978987054a695 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:13.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.426+0000 7ff41a000640 1 -- 172.21.15.86:0/4182907029 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4140013a0 con 0x7ff400004d50 2024-04-25T14:18:13.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.426+0000 7ff41a000640 1 -- 172.21.15.86:0/4182907029 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4140052f0 con 0x7ff400004d50 2024-04-25T14:18:13.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.426+0000 7ff41a000640 1 -- 172.21.15.86:0/4182907029 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414003840 con 0x7ff400004d50 2024-04-25T14:18:13.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.426+0000 7ff422600640 1 -- 172.21.15.86:0/4182907029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff400004d50 msgr2=0x7ff4000051e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:13.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.426+0000 7ff422600640 1 --2- 172.21.15.86:0/4182907029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff400004d50 0x7ff4000051e0 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7ff414001750 tx=0x7ff41401c3e0 comp rx=0 tx=0).stop 2024-04-25T14:18:13.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.426+0000 7ff41be00640 1 -- 172.21.15.86:0/4182907029 reap_dead start 2024-04-25T14:18:13.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.426+0000 7ff422600640 1 -- 172.21.15.86:0/4182907029 shutdown_connections 2024-04-25T14:18:13.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.426+0000 7ff422600640 1 -- 172.21.15.86:0/4182907029 >> 172.21.15.86:0/4182907029 conn(0x7ff41c0b4650 msgr2=0x7ff41c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:13.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.426+0000 7ff422600640 1 -- 172.21.15.86:0/4182907029 shutdown_connections 2024-04-25T14:18:13.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.426+0000 7ff422600640 1 -- 172.21.15.86:0/4182907029 wait complete. 2024-04-25T14:18:13.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.430+0000 7ff422600640 1 Processor -- start 2024-04-25T14:18:13.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.430+0000 7ff422600640 1 -- start start 2024-04-25T14:18:13.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.430+0000 7ff422600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff400004d50 0x7ff41c156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:13.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.430+0000 7ff422600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff41c0cb6b0 0x7ff41c0b1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:13.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.430+0000 7ff422600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff41c1576a0 0x7ff41c0b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:13.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.430+0000 7ff422600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c15f330 con 0x7ff400004d50 2024-04-25T14:18:13.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.430+0000 7ff422600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c06d170 con 0x7ff41c1576a0 2024-04-25T14:18:13.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.430+0000 7ff422600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff41c15f4b0 con 0x7ff41c0cb6b0 2024-04-25T14:18:13.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.430+0000 7ff41b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff400004d50 0x7ff41c156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:13.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.430+0000 7ff41b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff400004d50 0x7ff41c156e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46940/0 (socket says 172.21.15.86:46940) 2024-04-25T14:18:13.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.430+0000 7ff41b400640 1 -- 172.21.15.86:0/2201888789 learned_addr learned my addr 172.21.15.86:0/2201888789 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:13.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.430+0000 7ff41be00640 1 --2- 172.21.15.86:0/2201888789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff41c1576a0 0x7ff41c0b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:13.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.430+0000 7ff41aa00640 1 --2- 172.21.15.86:0/2201888789 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff41c0cb6b0 0x7ff41c0b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:13.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.430+0000 7ff41b400640 1 -- 172.21.15.86:0/2201888789 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff41c0cb6b0 msgr2=0x7ff41c0b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:13.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.430+0000 7ff41b400640 1 --2- 172.21.15.86:0/2201888789 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff41c0cb6b0 0x7ff41c0b1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:13.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.430+0000 7ff41b400640 1 -- 172.21.15.86:0/2201888789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff41c1576a0 msgr2=0x7ff41c0b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:13.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.430+0000 7ff41b400640 1 --2- 172.21.15.86:0/2201888789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff41c1576a0 0x7ff41c0b1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:13.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.430+0000 7ff41b400640 1 -- 172.21.15.86:0/2201888789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41c0ca610 con 0x7ff400004d50 2024-04-25T14:18:13.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.434+0000 7ff41b400640 1 --2- 172.21.15.86:0/2201888789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff400004d50 0x7ff41c156e70 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009fc0 tx=0x7ff40c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:13.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.434+0000 7ff413400640 1 -- 172.21.15.86:0/2201888789 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c00fea0 con 0x7ff400004d50 2024-04-25T14:18:13.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.434+0000 7ff413400640 1 -- 172.21.15.86:0/2201888789 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff40c015630 con 0x7ff400004d50 2024-04-25T14:18:13.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.434+0000 7ff413400640 1 -- 172.21.15.86:0/2201888789 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c015a00 con 0x7ff400004d50 2024-04-25T14:18:13.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.434+0000 7ff422600640 1 -- 172.21.15.86:0/2201888789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff41c0ca8f0 con 0x7ff400004d50 2024-04-25T14:18:13.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.434+0000 7ff422600640 1 -- 172.21.15.86:0/2201888789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff41c0caeb0 con 0x7ff400004d50 2024-04-25T14:18:13.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.434+0000 7ff422600640 1 -- 172.21.15.86:0/2201888789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff41c069d00 con 0x7ff400004d50 2024-04-25T14:18:13.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.434+0000 7ff413400640 1 -- 172.21.15.86:0/2201888789 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7ff40c015ba0 con 0x7ff400004d50 2024-04-25T14:18:13.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.434+0000 7ff413400640 1 --2- 172.21.15.86:0/2201888789 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7ff3fc06f430 0x7ff3fc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:13.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.434+0000 7ff413400640 1 -- 172.21.15.86:0/2201888789 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 6023+0+0 (secure 0 0 0) 0x7ff40c0905b0 con 0x7ff400004d50 2024-04-25T14:18:13.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.434+0000 7ff41aa00640 1 --2- 172.21.15.86:0/2201888789 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7ff3fc06f430 0x7ff3fc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:13.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.438+0000 7ff41aa00640 1 --2- 172.21.15.86:0/2201888789 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7ff3fc06f430 0x7ff3fc071940 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff4080012c0 tx=0x7ff408009040 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:13.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.442+0000 7ff413400640 1 -- 172.21.15.86:0/2201888789 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff40c019030 con 0x7ff400004d50 2024-04-25T14:18:13.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.594+0000 7ff422600640 1 -- 172.21.15.86:0/2201888789 --> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff41c076bf0 con 0x7ff3fc06f430 2024-04-25T14:18:13.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.598+0000 7ff413400640 1 -- 172.21.15.86:0/2201888789 <== mgr.11343 v2:172.21.15.86:6832/3174179029 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13497 (secure 0 0 0) 0x7ff41c076bf0 con 0x7ff3fc06f430 2024-04-25T14:18:13.591 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:18:13.591 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:18:13.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.598+0000 7ff422600640 1 -- 172.21.15.86:0/2201888789 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7ff3fc06f430 msgr2=0x7ff3fc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:13.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.598+0000 7ff422600640 1 --2- 172.21.15.86:0/2201888789 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7ff3fc06f430 0x7ff3fc071940 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff4080012c0 tx=0x7ff408009040 comp rx=0 tx=0).stop 2024-04-25T14:18:13.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.598+0000 7ff422600640 1 -- 172.21.15.86:0/2201888789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff400004d50 msgr2=0x7ff41c156e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:13.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.598+0000 7ff422600640 1 --2- 172.21.15.86:0/2201888789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff400004d50 0x7ff41c156e70 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009fc0 tx=0x7ff40c00f580 comp rx=0 tx=0).stop 2024-04-25T14:18:13.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.598+0000 7ff422600640 1 -- 172.21.15.86:0/2201888789 shutdown_connections 2024-04-25T14:18:13.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.598+0000 7ff422600640 1 --2- 172.21.15.86:0/2201888789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff41c1576a0 0x7ff41c0b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:13.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.598+0000 7ff422600640 1 --2- 172.21.15.86:0/2201888789 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7ff3fc06f430 0x7ff3fc071940 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:13.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.598+0000 7ff422600640 1 --2- 172.21.15.86:0/2201888789 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff41c0cb6b0 0x7ff41c0b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:13.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.598+0000 7ff422600640 1 --2- 172.21.15.86:0/2201888789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff400004d50 0x7ff41c156e70 unknown :-1 s=CLOSED pgs=2526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:13.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.598+0000 7ff422600640 1 -- 172.21.15.86:0/2201888789 >> 172.21.15.86:0/2201888789 conn(0x7ff41c0b4650 msgr2=0x7ff41c073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:13.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.598+0000 7ff422600640 1 -- 172.21.15.86:0/2201888789 shutdown_connections 2024-04-25T14:18:13.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:13.598+0000 7ff422600640 1 -- 172.21.15.86:0/2201888789 wait complete. 2024-04-25T14:18:13.614 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-04-25T14:18:13.180478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":957,"num_read_kb":2631,"num_write":271,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3305,"ondisk_log_size":3305,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":258080,"kb_used_data":16936,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754716640,"statfs":{"total":773094113280,"available":772829839360,"internally_reserved":0,"allocated":17342464,"data_stored":9299120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-33554432,"num_objects":-32,"num_object_clones":0,"num_object_copies":-64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-32,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-32,"num_write_kb":-32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.853526"},"pg_stats":[{"pgid":"1.0","version":"157'3305","reported_seq":3894,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-25T14:18:04.615678+0000","last_change":"2024-04-25T14:03:40.270024+0000","last_active":"2024-04-25T14:18:04.615678+0000","last_peered":"2024-04-25T14:18:04.615678+0000","last_clean":"2024-04-25T14:18:04.615678+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:18:04.615678+0000","last_undegraded":"2024-04-25T14:18:04.615678+0000","last_fullsized":"2024-04-25T14:18:04.615678+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:03:39.149904+0000","last_clean_scrub_stamp":"2024-04-25T14:03:39.149904+0000","objects_scrubbed":0,"log_size":3305,"log_dups_size":0,"ondisk_log_size":3305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:04.680050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":957,"num_read_kb":2631,"num_write":271,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":957,"num_read_kb":2631,"num_write":271,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3305,"ondisk_log_size":3305,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575018,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29572,"kb_used_data":2552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342268,"statfs":{"total":96636764160,"available":96606482432,"internally_reserved":0,"allocated":2613248,"data_stored":1605166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33156,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2019328,"data_stored":1014798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575019,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33156,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2019328,"data_stored":1014798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33156,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2019328,"data_stored":1014798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33152,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2019328,"data_stored":1014798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575017,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33156,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2019328,"data_stored":1014798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575019,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33152,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2019328,"data_stored":1014798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575019,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29580,"kb_used_data":2552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342260,"statfs":{"total":96636764160,"available":96606474240,"internally_reserved":0,"allocated":2613248,"data_stored":1605166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:18:13.616 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T14:03:39.149904+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=14, tm_min=3, tm_sec=39, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=14, tm_min=18, tm_sec=7, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T14:18:13.617 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T14:18:20.842 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:18:20.846 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:18:33.617 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:18:33.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.781+0000 7fb923a00640 1 Processor -- start 2024-04-25T14:18:33.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.781+0000 7fb923a00640 1 -- start start 2024-04-25T14:18:33.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.785+0000 7fb923a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb91c070330 0x7fb91c0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:33.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.785+0000 7fb923a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb91c0d1c20 0x7fb91c06ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:33.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.785+0000 7fb923a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb91c0d2030 con 0x7fb91c06f270 2024-04-25T14:18:33.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.785+0000 7fb923a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb91c15e5c0 con 0x7fb91c0d1c20 2024-04-25T14:18:33.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.785+0000 7fb923a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb91c15e8a0 con 0x7fb91c070330 2024-04-25T14:18:33.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.785+0000 7fb913e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb91c0d1c20 0x7fb91c06ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:33.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.785+0000 7fb921000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb91c070330 0x7fb91c0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:33.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.785+0000 7fb921a00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fb91c06f270 0x7fb91c06f680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.86:40854/0 (socket says 172.21.15.86:40854) 2024-04-25T14:18:33.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.785+0000 7fb913e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb91c0d1c20 0x7fb91c06ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.86:38314/0 (socket says 172.21.15.86:38314) 2024-04-25T14:18:33.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.785+0000 7fb921a00640 1 -- 172.21.15.86:0/1002041626 learned_addr learned my addr 172.21.15.86:0/1002041626 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:18:33.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.785+0000 7fb913e00640 1 -- 172.21.15.86:0/1002041626 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb91c070330 msgr2=0x7fb91c0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:33.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.785+0000 7fb913e00640 1 --2- 172.21.15.86:0/1002041626 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb91c070330 0x7fb91c0d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:33.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.785+0000 7fb913e00640 1 -- 172.21.15.86:0/1002041626 >> v1:172.21.15.155:6789/0 conn(0x7fb91c06f270 legacy=0x7fb91c06f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:33.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.785+0000 7fb913e00640 1 -- 172.21.15.86:0/1002041626 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91c15eb80 con 0x7fb91c0d1c20 2024-04-25T14:18:33.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.789+0000 7fb913e00640 1 --2- 172.21.15.86:0/1002041626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb91c0d1c20 0x7fb91c06ed30 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fb9040097f0 tx=0x7fb90401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96d1df99fe18d6c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:33.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.789+0000 7fb91be00640 1 -- 172.21.15.86:0/1002041626 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9040b0070 con 0x7fb91c0d1c20 2024-04-25T14:18:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.789+0000 7fb91be00640 1 -- 172.21.15.86:0/1002041626 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9040ab600 con 0x7fb91c0d1c20 2024-04-25T14:18:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.789+0000 7fb91be00640 1 -- 172.21.15.86:0/1002041626 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9040ab9d0 con 0x7fb91c0d1c20 2024-04-25T14:18:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.789+0000 7fb923a00640 1 -- 172.21.15.86:0/1002041626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb91c0d1c20 msgr2=0x7fb91c06ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:33.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.789+0000 7fb923a00640 1 --2- 172.21.15.86:0/1002041626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb91c0d1c20 0x7fb91c06ed30 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fb9040097f0 tx=0x7fb90401c410 comp rx=0 tx=0).stop 2024-04-25T14:18:33.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.789+0000 7fb923a00640 1 -- 172.21.15.86:0/1002041626 shutdown_connections 2024-04-25T14:18:33.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.789+0000 7fb923a00640 1 --2- 172.21.15.86:0/1002041626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb91c0d1c20 0x7fb91c06ed30 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:33.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.789+0000 7fb923a00640 1 --2- 172.21.15.86:0/1002041626 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb91c070330 0x7fb91c0d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:33.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.789+0000 7fb923a00640 1 -- 172.21.15.86:0/1002041626 >> 172.21.15.86:0/1002041626 conn(0x7fb91c0b4650 msgr2=0x7fb91c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:33.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.789+0000 7fb923a00640 1 -- 172.21.15.86:0/1002041626 shutdown_connections 2024-04-25T14:18:33.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.789+0000 7fb923a00640 1 -- 172.21.15.86:0/1002041626 wait complete. 2024-04-25T14:18:33.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.793+0000 7fb923a00640 1 Processor -- start 2024-04-25T14:18:33.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.793+0000 7fb923a00640 1 -- start start 2024-04-25T14:18:33.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.793+0000 7fb923a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb91c06f270 0x7fb91c09aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:33.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.793+0000 7fb923a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb91c070330 0x7fb91c0c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:33.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.793+0000 7fb923a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb91c0d1c20 0x7fb91c0af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:33.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.793+0000 7fb923a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb91c0708c0 con 0x7fb91c06f270 2024-04-25T14:18:33.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.793+0000 7fb923a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb91c070740 con 0x7fb91c070330 2024-04-25T14:18:33.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.793+0000 7fb923a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb91c16afc0 con 0x7fb91c0d1c20 2024-04-25T14:18:33.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.793+0000 7fb921000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb91c06f270 0x7fb91c09aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:33.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.793+0000 7fb921000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb91c06f270 0x7fb91c09aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47696/0 (socket says 172.21.15.86:47696) 2024-04-25T14:18:33.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.793+0000 7fb921000640 1 -- 172.21.15.86:0/858454488 learned_addr learned my addr 172.21.15.86:0/858454488 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:33.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.793+0000 7fb921a00640 1 --2- 172.21.15.86:0/858454488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb91c0d1c20 0x7fb91c0af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:33.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.793+0000 7fb913e00640 1 --2- 172.21.15.86:0/858454488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb91c070330 0x7fb91c0c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:33.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.793+0000 7fb921000640 1 -- 172.21.15.86:0/858454488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb91c0d1c20 msgr2=0x7fb91c0af2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:33.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.793+0000 7fb921000640 1 --2- 172.21.15.86:0/858454488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb91c0d1c20 0x7fb91c0af2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:33.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.793+0000 7fb921000640 1 -- 172.21.15.86:0/858454488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb91c070330 msgr2=0x7fb91c0c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:33.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.793+0000 7fb921000640 1 --2- 172.21.15.86:0/858454488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb91c070330 0x7fb91c0c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:33.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.793+0000 7fb921000640 1 -- 172.21.15.86:0/858454488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91c06d840 con 0x7fb91c06f270 2024-04-25T14:18:33.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.793+0000 7fb913e00640 1 --2- 172.21.15.86:0/858454488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb91c070330 0x7fb91c0c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:18:33.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.793+0000 7fb921000640 1 --2- 172.21.15.86:0/858454488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb91c06f270 0x7fb91c09aa70 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7fb90c00e840 tx=0x7fb90c00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:33.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.793+0000 7fb919600640 1 -- 172.21.15.86:0/858454488 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c017070 con 0x7fb91c06f270 2024-04-25T14:18:33.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.793+0000 7fb919600640 1 -- 172.21.15.86:0/858454488 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb90c00ee90 con 0x7fb91c06f270 2024-04-25T14:18:33.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.793+0000 7fb923a00640 1 -- 172.21.15.86:0/858454488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb91c0b2510 con 0x7fb91c06f270 2024-04-25T14:18:33.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.793+0000 7fb919600640 1 -- 172.21.15.86:0/858454488 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c00cb60 con 0x7fb91c06f270 2024-04-25T14:18:33.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.793+0000 7fb923a00640 1 -- 172.21.15.86:0/858454488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb91c0c7ad0 con 0x7fb91c06f270 2024-04-25T14:18:33.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.797+0000 7fb923a00640 1 -- 172.21.15.86:0/858454488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb91c069d00 con 0x7fb91c06f270 2024-04-25T14:18:33.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.797+0000 7fb919600640 1 -- 172.21.15.86:0/858454488 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7fb90c01a070 con 0x7fb91c06f270 2024-04-25T14:18:33.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.797+0000 7fb919600640 1 --2- 172.21.15.86:0/858454488 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fb8f806f400 0x7fb8f8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:33.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.797+0000 7fb919600640 1 -- 172.21.15.86:0/858454488 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 6023+0+0 (secure 0 0 0) 0x7fb90c08f6a0 con 0x7fb91c06f270 2024-04-25T14:18:33.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.797+0000 7fb913e00640 1 --2- 172.21.15.86:0/858454488 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fb8f806f400 0x7fb8f8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:33.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.797+0000 7fb913e00640 1 --2- 172.21.15.86:0/858454488 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fb8f806f400 0x7fb8f8071910 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb9040ad000 tx=0x7fb9040a6000 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:33.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.801+0000 7fb919600640 1 -- 172.21.15.86:0/858454488 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb90c05c450 con 0x7fb91c06f270 2024-04-25T14:18:33.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.953+0000 7fb923a00640 1 -- 172.21.15.86:0/858454488 --> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb91c0b3960 con 0x7fb8f806f400 2024-04-25T14:18:33.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.957+0000 7fb919600640 1 -- 172.21.15.86:0/858454488 <== mgr.11343 v2:172.21.15.86:6832/3174179029 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13397 (secure 0 0 0) 0x7fb91c0b3960 con 0x7fb8f806f400 2024-04-25T14:18:33.951 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:18:33.952 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-25T14:18:33.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.957+0000 7fb923a00640 1 -- 172.21.15.86:0/858454488 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fb8f806f400 msgr2=0x7fb8f8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:33.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.957+0000 7fb923a00640 1 --2- 172.21.15.86:0/858454488 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fb8f806f400 0x7fb8f8071910 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb9040ad000 tx=0x7fb9040a6000 comp rx=0 tx=0).stop 2024-04-25T14:18:33.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.957+0000 7fb923a00640 1 -- 172.21.15.86:0/858454488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb91c06f270 msgr2=0x7fb91c09aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:33.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.957+0000 7fb923a00640 1 --2- 172.21.15.86:0/858454488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb91c06f270 0x7fb91c09aa70 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7fb90c00e840 tx=0x7fb90c00ec40 comp rx=0 tx=0).stop 2024-04-25T14:18:33.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.957+0000 7fb923a00640 1 -- 172.21.15.86:0/858454488 shutdown_connections 2024-04-25T14:18:33.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.957+0000 7fb923a00640 1 --2- 172.21.15.86:0/858454488 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7fb8f806f400 0x7fb8f8071910 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:33.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.957+0000 7fb923a00640 1 --2- 172.21.15.86:0/858454488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb91c0d1c20 0x7fb91c0af2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:33.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.957+0000 7fb923a00640 1 --2- 172.21.15.86:0/858454488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb91c070330 0x7fb91c0c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:33.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.957+0000 7fb923a00640 1 --2- 172.21.15.86:0/858454488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb91c06f270 0x7fb91c09aa70 unknown :-1 s=CLOSED pgs=2527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:33.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.957+0000 7fb923a00640 1 -- 172.21.15.86:0/858454488 >> 172.21.15.86:0/858454488 conn(0x7fb91c0b4650 msgr2=0x7fb91c0c4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:33.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.957+0000 7fb923a00640 1 -- 172.21.15.86:0/858454488 shutdown_connections 2024-04-25T14:18:33.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:33.957+0000 7fb923a00640 1 -- 172.21.15.86:0/858454488 wait complete. 2024-04-25T14:18:33.972 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T14:18:33.184858+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":957,"num_read_kb":2631,"num_write":271,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3306,"ondisk_log_size":3306,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":258064,"kb_used_data":16936,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754716656,"statfs":{"total":773094113280,"available":772829855744,"internally_reserved":0,"allocated":17342464,"data_stored":9299120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002603"},"pg_stats":[{"pgid":"1.0","version":"160'3306","reported_seq":3904,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-25T14:18:13.605095+0000","last_change":"2024-04-25T14:18:13.605131+0000","last_active":"2024-04-25T14:18:13.605095+0000","last_peered":"2024-04-25T14:18:13.605095+0000","last_clean":"2024-04-25T14:18:13.605095+0000","last_became_active":"2024-04-25T14:03:40.268837+0000","last_became_peered":"2024-04-25T14:03:40.268837+0000","last_unstale":"2024-04-25T14:18:13.605095+0000","last_undegraded":"2024-04-25T14:18:13.605095+0000","last_fullsized":"2024-04-25T14:18:13.605095+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'3306","last_scrub_stamp":"2024-04-25T14:18:13.605001+0000","last_deep_scrub":"160'3306","last_deep_scrub_stamp":"2024-04-25T14:18:13.605001+0000","last_clean_scrub_stamp":"2024-04-25T14:18:13.605001+0000","objects_scrubbed":2,"log_size":3306,"log_dups_size":0,"ondisk_log_size":3306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:46.224278+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":957,"num_read_kb":2631,"num_write":271,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":957,"num_read_kb":2631,"num_write":271,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3306,"ondisk_log_size":3306,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575022,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29568,"kb_used_data":2552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342272,"statfs":{"total":96636764160,"available":96606486528,"internally_reserved":0,"allocated":2613248,"data_stored":1605166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33156,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2019328,"data_stored":1014798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575024,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33156,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2019328,"data_stored":1014798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33156,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2019328,"data_stored":1014798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33152,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2019328,"data_stored":1014798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33156,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2019328,"data_stored":1014798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33152,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2019328,"data_stored":1014798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575023,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29568,"kb_used_data":2552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342272,"statfs":{"total":96636764160,"available":96606486528,"internally_reserved":0,"allocated":2613248,"data_stored":1605166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:18:33.974 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T14:18:34.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.121+0000 7feec5000640 1 Processor -- start 2024-04-25T14:18:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.121+0000 7feec5000640 1 -- start start 2024-04-25T14:18:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.125+0000 7feec5000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feec0070330 0x7feec00d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.125+0000 7feec5000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7feec00d1c20 0x7feec006ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.125+0000 7feec5000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feec006f270 0x7feec006f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.125+0000 7feec5000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7feec00d2030 con 0x7feec006f270 2024-04-25T14:18:34.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.125+0000 7feec5000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7feec015e5c0 con 0x7feec00d1c20 2024-04-25T14:18:34.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.125+0000 7feec5000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7feec015e8a0 con 0x7feec0070330 2024-04-25T14:18:34.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.125+0000 7feebe800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feec006f270 0x7feec006f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:34.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.125+0000 7feebe800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feec006f270 0x7feec006f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47706/0 (socket says 172.21.15.86:47706) 2024-04-25T14:18:34.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.125+0000 7feebe800640 1 -- 172.21.15.86:0/3714529913 learned_addr learned my addr 172.21.15.86:0/3714529913 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:34.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.125+0000 7feebde00640 1 --2- 172.21.15.86:0/3714529913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feec0070330 0x7feec00d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:34.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.125+0000 7feebd400640 1 --2- 172.21.15.86:0/3714529913 >> v2:172.21.15.155:3300/0 conn(0x7feec00d1c20 0x7feec006ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.125+0000 7feebe800640 1 -- 172.21.15.86:0/3714529913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feec0070330 msgr2=0x7feec00d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.125+0000 7feebe800640 1 --2- 172.21.15.86:0/3714529913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feec0070330 0x7feec00d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.125+0000 7feebe800640 1 -- 172.21.15.86:0/3714529913 >> v2:172.21.15.155:3300/0 conn(0x7feec00d1c20 msgr2=0x7feec006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.125+0000 7feebe800640 1 --2- 172.21.15.86:0/3714529913 >> v2:172.21.15.155:3300/0 conn(0x7feec00d1c20 0x7feec006ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.125+0000 7feebe800640 1 -- 172.21.15.86:0/3714529913 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec015eb80 con 0x7feec006f270 2024-04-25T14:18:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.125+0000 7feebd400640 1 --2- 172.21.15.86:0/3714529913 >> v2:172.21.15.155:3300/0 conn(0x7feec00d1c20 0x7feec006ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:18:34.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.129+0000 7feebe800640 1 --2- 172.21.15.86:0/3714529913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec006f270 0x7feec006f680 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009870 tx=0x7feeb801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39410c5d5622298e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:34.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.129+0000 7feebca00640 1 -- 172.21.15.86:0/3714529913 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb80ab600 con 0x7feec006f270 2024-04-25T14:18:34.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.129+0000 7feebca00640 1 -- 172.21.15.86:0/3714529913 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb80b0070 con 0x7feec006f270 2024-04-25T14:18:34.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.129+0000 7feebca00640 1 -- 172.21.15.86:0/3714529913 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb80ab9c0 con 0x7feec006f270 2024-04-25T14:18:34.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.129+0000 7feec5000640 1 -- 172.21.15.86:0/3714529913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec006f270 msgr2=0x7feec006f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:34.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.129+0000 7feec5000640 1 --2- 172.21.15.86:0/3714529913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec006f270 0x7feec006f680 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009870 tx=0x7feeb801c460 comp rx=0 tx=0).stop 2024-04-25T14:18:34.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.129+0000 7feec5000640 1 -- 172.21.15.86:0/3714529913 shutdown_connections 2024-04-25T14:18:34.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.129+0000 7feec5000640 1 --2- 172.21.15.86:0/3714529913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec006f270 0x7feec006f680 unknown :-1 s=CLOSED pgs=2528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:34.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.129+0000 7feec5000640 1 --2- 172.21.15.86:0/3714529913 >> v2:172.21.15.155:3300/0 conn(0x7feec00d1c20 0x7feec006ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:34.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.129+0000 7feec5000640 1 --2- 172.21.15.86:0/3714529913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feec0070330 0x7feec00d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:34.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.129+0000 7feec5000640 1 -- 172.21.15.86:0/3714529913 >> 172.21.15.86:0/3714529913 conn(0x7feec00b4650 msgr2=0x7feec00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:34.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.129+0000 7feec5000640 1 -- 172.21.15.86:0/3714529913 shutdown_connections 2024-04-25T14:18:34.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.129+0000 7feec5000640 1 -- 172.21.15.86:0/3714529913 wait complete. 2024-04-25T14:18:34.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.133+0000 7feec5000640 1 Processor -- start 2024-04-25T14:18:34.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.133+0000 7feec5000640 1 -- start start 2024-04-25T14:18:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.133+0000 7feec5000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feec006f270 0x7feec00cf920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.133+0000 7feec5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec0070330 0x7feec0156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.133+0000 7feec5000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feec00d1c20 0x7feec0156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.133+0000 7feec5000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feec015f4b0 con 0x7feec0070330 2024-04-25T14:18:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.133+0000 7feec5000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7feec015f330 con 0x7feec00d1c20 2024-04-25T14:18:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.133+0000 7feec5000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7feec0070740 con 0x7feec006f270 2024-04-25T14:18:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.133+0000 7feebd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec0070330 0x7feec0156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:34.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.133+0000 7feebd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec0070330 0x7feec0156590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47716/0 (socket says 172.21.15.86:47716) 2024-04-25T14:18:34.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.133+0000 7feebd400640 1 -- 172.21.15.86:0/342310013 learned_addr learned my addr 172.21.15.86:0/342310013 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:34.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.133+0000 7feebde00640 1 --2- 172.21.15.86:0/342310013 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feec006f270 0x7feec00cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:34.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.133+0000 7feebe800640 1 --2- 172.21.15.86:0/342310013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feec00d1c20 0x7feec0156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:34.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.133+0000 7feebd400640 1 -- 172.21.15.86:0/342310013 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feec006f270 msgr2=0x7feec00cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:34.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.133+0000 7feebd400640 1 --2- 172.21.15.86:0/342310013 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feec006f270 0x7feec00cf920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:34.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.133+0000 7feebd400640 1 -- 172.21.15.86:0/342310013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feec00d1c20 msgr2=0x7feec0156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.133+0000 7feebd400640 1 --2- 172.21.15.86:0/342310013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feec00d1c20 0x7feec0156ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.133+0000 7feebd400640 1 -- 172.21.15.86:0/342310013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec01f9580 con 0x7feec0070330 2024-04-25T14:18:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.133+0000 7feebd400640 1 --2- 172.21.15.86:0/342310013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec0070330 0x7feec0156590 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7feeac00d800 tx=0x7feeac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.133+0000 7feeb6000640 1 -- 172.21.15.86:0/342310013 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac018020 con 0x7feec0070330 2024-04-25T14:18:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.133+0000 7feeb6000640 1 -- 172.21.15.86:0/342310013 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeac0096c0 con 0x7feec0070330 2024-04-25T14:18:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.133+0000 7feec5000640 1 -- 172.21.15.86:0/342310013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec0157c90 con 0x7feec0070330 2024-04-25T14:18:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.133+0000 7feeb6000640 1 -- 172.21.15.86:0/342310013 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac009ab0 con 0x7feec0070330 2024-04-25T14:18:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.133+0000 7feec5000640 1 -- 172.21.15.86:0/342310013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec01f9b20 con 0x7feec0070330 2024-04-25T14:18:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.137+0000 7feec5000640 1 -- 172.21.15.86:0/342310013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feec01f9390 con 0x7feec0070330 2024-04-25T14:18:34.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.137+0000 7feeb6000640 1 -- 172.21.15.86:0/342310013 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7feeac01b020 con 0x7feec0070330 2024-04-25T14:18:34.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.137+0000 7feeb6000640 1 --2- 172.21.15.86:0/342310013 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7feea006f400 0x7feea0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:34.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.137+0000 7feeb6000640 1 -- 172.21.15.86:0/342310013 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 6023+0+0 (secure 0 0 0) 0x7feeac08f990 con 0x7feec0070330 2024-04-25T14:18:34.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.137+0000 7feebde00640 1 --2- 172.21.15.86:0/342310013 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7feea006f400 0x7feea0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:34.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.137+0000 7feebde00640 1 --2- 172.21.15.86:0/342310013 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7feea006f400 0x7feea0071910 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7feec0072200 tx=0x7feeb0003a40 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:34.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.141+0000 7feeb6000640 1 -- 172.21.15.86:0/342310013 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7feeac022020 con 0x7feec0070330 2024-04-25T14:18:34.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.301+0000 7feec5000640 1 -- 172.21.15.86:0/342310013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7feec0157a20 con 0x7feec0070330 2024-04-25T14:18:34.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.301+0000 7feeb6000640 1 -- 172.21.15.86:0/342310013 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v26)=0 v26) ==== 123+0+0 (secure 0 0 0) 0x7feeac05c730 con 0x7feec0070330 2024-04-25T14:18:34.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.305+0000 7feeb6000640 1 -- 172.21.15.86:0/342310013 <== mon.0 v2:172.21.15.86:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7feeac095080 con 0x7feec0070330 2024-04-25T14:18:34.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.305+0000 7feec5000640 1 -- 172.21.15.86:0/342310013 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7feea006f400 msgr2=0x7feea0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:34.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.305+0000 7feec5000640 1 --2- 172.21.15.86:0/342310013 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7feea006f400 0x7feea0071910 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7feec0072200 tx=0x7feeb0003a40 comp rx=0 tx=0).stop 2024-04-25T14:18:34.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.305+0000 7feec5000640 1 -- 172.21.15.86:0/342310013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec0070330 msgr2=0x7feec0156590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:34.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.305+0000 7feec5000640 1 --2- 172.21.15.86:0/342310013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec0070330 0x7feec0156590 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7feeac00d800 tx=0x7feeac00dcc0 comp rx=0 tx=0).stop 2024-04-25T14:18:34.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.305+0000 7feec5000640 1 -- 172.21.15.86:0/342310013 shutdown_connections 2024-04-25T14:18:34.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.305+0000 7feec5000640 1 --2- 172.21.15.86:0/342310013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feec00d1c20 0x7feec0156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:34.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.305+0000 7feec5000640 1 --2- 172.21.15.86:0/342310013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feec0070330 0x7feec0156590 unknown :-1 s=CLOSED pgs=2529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:34.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.305+0000 7feec5000640 1 --2- 172.21.15.86:0/342310013 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7feea006f400 0x7feea0071910 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:34.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.305+0000 7feec5000640 1 --2- 172.21.15.86:0/342310013 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feec006f270 0x7feec00cf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:34.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.305+0000 7feec5000640 1 -- 172.21.15.86:0/342310013 >> 172.21.15.86:0/342310013 conn(0x7feec00b4650 msgr2=0x7feec015b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:34.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.305+0000 7feec5000640 1 -- 172.21.15.86:0/342310013 shutdown_connections 2024-04-25T14:18:34.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.305+0000 7feec5000640 1 -- 172.21.15.86:0/342310013 wait complete. 2024-04-25T14:18:34.322 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T14:18:34.322 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:18:34.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.473+0000 7f92bbc00640 1 Processor -- start 2024-04-25T14:18:34.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.473+0000 7f92bbc00640 1 -- start start 2024-04-25T14:18:34.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.473+0000 7f92bbc00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f92b415fe00 0x7f92b4070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:34.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.473+0000 7f92bbc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92b4071770 con 0x7f92b4071360 2024-04-25T14:18:34.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.473+0000 7f92bbc00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92b406dc30 con 0x7f92b4163fb0 2024-04-25T14:18:34.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.473+0000 7f92bbc00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f92b4160210 con 0x7f92b415fe00 2024-04-25T14:18:34.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.473+0000 7f92b3e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f92b4071360 0x7f92b41638a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59662/0 (socket says 172.21.15.86:59662) 2024-04-25T14:18:34.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.473+0000 7f92b3e00640 1 -- 172.21.15.86:0/510245664 learned_addr learned my addr 172.21.15.86:0/510245664 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-25T14:18:34.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.473+0000 7f92b9200640 1 --2- 172.21.15.86:0/510245664 >> v2:172.21.15.155:3300/0 conn(0x7f92b415fe00 0x7f92b4070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:34.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.473+0000 7f92b3400640 1 -- 172.21.15.86:0/510245664 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1695181379 0 0) 0x7f929c001120 con 0x7f92b4071360 2024-04-25T14:18:34.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.477+0000 7f92b3400640 1 -- 172.21.15.86:0/510245664 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9298003530 con 0x7f92b4071360 2024-04-25T14:18:34.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.477+0000 7f92b3400640 1 -- 172.21.15.86:0/510245664 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2737147538 0 0) 0x7f92b406dc30 con 0x7f92b4163fb0 2024-04-25T14:18:34.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.477+0000 7f92b3400640 1 -- 172.21.15.86:0/510245664 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f929c001120 con 0x7f92b4163fb0 2024-04-25T14:18:34.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.477+0000 7f92b3400640 1 -- 172.21.15.86:0/510245664 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 977533651 0 0) 0x7f9298003530 con 0x7f92b4071360 2024-04-25T14:18:34.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.477+0000 7f92b3400640 1 -- 172.21.15.86:0/510245664 >> v2:172.21.15.155:3300/0 conn(0x7f92b415fe00 msgr2=0x7f92b4070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:34.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.477+0000 7f92b3400640 1 --2- 172.21.15.86:0/510245664 >> v2:172.21.15.155:3300/0 conn(0x7f92b415fe00 0x7f92b4070e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:34.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.477+0000 7f92b3400640 1 -- 172.21.15.86:0/510245664 >> v1:172.21.15.155:6789/0 conn(0x7f92b4163fb0 legacy=0x7f92b40cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:34.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.477+0000 7f92b3400640 1 -- 172.21.15.86:0/510245664 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b4164480 con 0x7f92b4071360 2024-04-25T14:18:34.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.477+0000 7f92b3400640 1 -- 172.21.15.86:0/510245664 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1357042259 0 0) 0x7f929c001850 con 0x7f92b4071360 2024-04-25T14:18:34.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.477+0000 7f92b3400640 1 -- 172.21.15.86:0/510245664 >> v1:172.21.15.86:6789/0 conn(0x7f92b4071360 legacy=0x7f92b41638a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:34.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.477+0000 7f92b3400640 1 --2- 172.21.15.86:0/510245664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9298003bc0 0x7f9298004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:34.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.477+0000 7f92b3400640 1 --2- 172.21.15.86:0/510245664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9298004810 0x7f9298004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.477+0000 7f92b3400640 1 --2- 172.21.15.86:0/510245664 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9298005220 0x7f929800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.477+0000 7f92b3400640 1 -- 172.21.15.86:0/510245664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f929800c190 con 0x7f9298003bc0 2024-04-25T14:18:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.477+0000 7f92b3400640 1 -- 172.21.15.86:0/510245664 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f929800c340 con 0x7f9298004810 2024-04-25T14:18:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.477+0000 7f92b3400640 1 -- 172.21.15.86:0/510245664 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f929800c540 con 0x7f9298005220 2024-04-25T14:18:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.477+0000 7f92b9200640 1 --2- 172.21.15.86:0/510245664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9298003bc0 0x7f9298004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.477+0000 7f92b3e00640 1 --2- 172.21.15.86:0/510245664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9298004810 0x7f9298004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.477+0000 7f92b9c00640 1 --2- 172.21.15.86:0/510245664 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9298005220 0x7f929800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.477+0000 7f92b9200640 1 -- 172.21.15.86:0/510245664 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9298005220 msgr2=0x7f929800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.477+0000 7f92b9200640 1 --2- 172.21.15.86:0/510245664 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9298005220 0x7f929800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.477+0000 7f92b9200640 1 -- 172.21.15.86:0/510245664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9298004810 msgr2=0x7f9298004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:18:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.477+0000 7f92b9200640 1 --2- 172.21.15.86:0/510245664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9298004810 0x7f9298004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.477+0000 7f92b9200640 1 -- 172.21.15.86:0/510245664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9298003530 con 0x7f9298003bc0 2024-04-25T14:18:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.477+0000 7f92b3e00640 1 --2- 172.21.15.86:0/510245664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9298004810 0x7f9298004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:18:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.477+0000 7f92b9c00640 1 -- 172.21.15.86:0/510245664 reap_dead start 2024-04-25T14:18:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.481+0000 7f92b9200640 1 --2- 172.21.15.86:0/510245664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9298003bc0 0x7f9298004010 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f92b4163800 tx=0x7f92a401cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d02f21b44f74b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.481+0000 7f92b3400640 1 -- 172.21.15.86:0/510245664 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a40b2070 con 0x7f9298003bc0 2024-04-25T14:18:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.481+0000 7f92b3400640 1 -- 172.21.15.86:0/510245664 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f92a40a5bb0 con 0x7f9298003bc0 2024-04-25T14:18:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.481+0000 7f92b3400640 1 -- 172.21.15.86:0/510245664 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a4009c10 con 0x7f9298003bc0 2024-04-25T14:18:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.481+0000 7f92bbc00640 1 -- 172.21.15.86:0/510245664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9298003bc0 msgr2=0x7f9298004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.481+0000 7f92bbc00640 1 --2- 172.21.15.86:0/510245664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9298003bc0 0x7f9298004010 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f92b4163800 tx=0x7f92a401cc20 comp rx=0 tx=0).stop 2024-04-25T14:18:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.481+0000 7f92bbc00640 1 -- 172.21.15.86:0/510245664 shutdown_connections 2024-04-25T14:18:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.481+0000 7f92bbc00640 1 --2- 172.21.15.86:0/510245664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9298003bc0 0x7f9298004010 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.481+0000 7f92bbc00640 1 -- 172.21.15.86:0/510245664 >> 172.21.15.86:0/510245664 conn(0x7f92b40cffc0 msgr2=0x7f92b40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.481+0000 7f92bbc00640 1 -- 172.21.15.86:0/510245664 shutdown_connections 2024-04-25T14:18:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.481+0000 7f92bbc00640 1 -- 172.21.15.86:0/510245664 wait complete. 2024-04-25T14:18:34.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.481+0000 7f92bbc00640 1 Processor -- start 2024-04-25T14:18:34.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.481+0000 7f92bbc00640 1 -- start start 2024-04-25T14:18:34.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.485+0000 7f92bbc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9298003bc0 0x7f92b40b7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:34.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.485+0000 7f92bbc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92b415e880 0x7f92b415ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:34.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.485+0000 7f92bbc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92b41fcfe0 0x7f92b41fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:34.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.485+0000 7f92bbc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92b41638a0 con 0x7f92b41fcfe0 2024-04-25T14:18:34.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.485+0000 7f92bbc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f92b406e910 con 0x7f9298003bc0 2024-04-25T14:18:34.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.485+0000 7f92bbc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f92b4163a20 con 0x7f92b415e880 2024-04-25T14:18:34.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.485+0000 7f92b9c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92b41fcfe0 0x7f92b41fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:34.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.485+0000 7f92b9c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92b41fcfe0 0x7f92b41fd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47740/0 (socket says 172.21.15.86:47740) 2024-04-25T14:18:34.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.485+0000 7f92b9c00640 1 -- 172.21.15.86:0/1743126946 learned_addr learned my addr 172.21.15.86:0/1743126946 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-25T14:18:34.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.485+0000 7f92b9200640 1 --2- 172.21.15.86:0/1743126946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9298003bc0 0x7f92b40b7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:34.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.485+0000 7f92b3e00640 1 --2- 172.21.15.86:0/1743126946 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92b415e880 0x7f92b415ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:34.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.485+0000 7f92b9c00640 1 -- 172.21.15.86:0/1743126946 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92b415e880 msgr2=0x7f92b415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:34.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.485+0000 7f92b9c00640 1 --2- 172.21.15.86:0/1743126946 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92b415e880 0x7f92b415ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:34.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.485+0000 7f92b9c00640 1 -- 172.21.15.86:0/1743126946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9298003bc0 msgr2=0x7f92b40b7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:34.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.485+0000 7f92b9c00640 1 --2- 172.21.15.86:0/1743126946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9298003bc0 0x7f92b40b7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:34.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.485+0000 7f92b9c00640 1 -- 172.21.15.86:0/1743126946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b415f1d0 con 0x7f92b41fcfe0 2024-04-25T14:18:34.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.485+0000 7f92b9c00640 1 --2- 172.21.15.86:0/1743126946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92b41fcfe0 0x7f92b41fd3f0 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f92b40b2f70 tx=0x7f92ac00e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:34.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.485+0000 7f92b0c00640 1 -- 172.21.15.86:0/1743126946 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac019070 con 0x7f92b41fcfe0 2024-04-25T14:18:34.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.485+0000 7f92b0c00640 1 -- 172.21.15.86:0/1743126946 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f92ac0052e0 con 0x7f92b41fcfe0 2024-04-25T14:18:34.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.485+0000 7f92bbc00640 1 -- 172.21.15.86:0/1743126946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b40b7dd0 con 0x7f92b41fcfe0 2024-04-25T14:18:34.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.485+0000 7f92b0c00640 1 -- 172.21.15.86:0/1743126946 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac003860 con 0x7f92b41fcfe0 2024-04-25T14:18:34.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.485+0000 7f92bbc00640 1 -- 172.21.15.86:0/1743126946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b40b8390 con 0x7f92b41fcfe0 2024-04-25T14:18:34.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.485+0000 7f92bbc00640 1 -- 172.21.15.86:0/1743126946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b40720e0 con 0x7f92b41fcfe0 2024-04-25T14:18:34.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.489+0000 7f92b0c00640 1 -- 172.21.15.86:0/1743126946 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 90) ==== 93207+0+0 (secure 0 0 0) 0x7f92ac01c070 con 0x7f92b41fcfe0 2024-04-25T14:18:34.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.489+0000 7f92b0c00640 1 --2- 172.21.15.86:0/1743126946 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f929006f400 0x7f9290071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:18:34.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.489+0000 7f92b0c00640 1 -- 172.21.15.86:0/1743126946 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 6023+0+0 (secure 0 0 0) 0x7f92ac08e950 con 0x7f92b41fcfe0 2024-04-25T14:18:34.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.489+0000 7f92b9200640 1 --2- 172.21.15.86:0/1743126946 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f929006f400 0x7f9290071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:18:34.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.489+0000 7f92b9200640 1 --2- 172.21.15.86:0/1743126946 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f929006f400 0x7f9290071910 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f92a40af000 tx=0x7f92a40a7040 comp rx=0 tx=0).ready entity=mgr.11343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:18:34.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.493+0000 7f92b0c00640 1 -- 172.21.15.86:0/1743126946 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f92b415f1d0 con 0x7f92b41fcfe0 2024-04-25T14:18:34.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.681+0000 7f92bbc00640 1 -- 172.21.15.86:0/1743126946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92b4076bf0 con 0x7f92b41fcfe0 2024-04-25T14:18:34.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.681+0000 7f92b0c00640 1 -- 172.21.15.86:0/1743126946 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 127 v127) ==== 95+0+3881 (secure 0 0 0) 0x7f92ac05b090 con 0x7f92b41fcfe0 2024-04-25T14:18:34.679 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-25T14:18:34.679 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":127,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3872031627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3872031627},{"type":"v1","addr":"172.21.15.86:6835","nonce":3872031627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3308745791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3308745791},{"type":"v1","addr":"172.21.15.155:6833","nonce":3308745791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112},{"gid":9272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2313890707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2313890707},{"type":"v1","addr":"172.21.15.155:6835","nonce":2313890707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":127},{"gid":10861,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2513022356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2513022356},{"type":"v1","addr":"172.21.15.86:6837","nonce":2513022356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":127}],"filesystems":[]} 2024-04-25T14:18:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 127 2024-04-25T14:18:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.685+0000 7f92bbc00640 1 -- 172.21.15.86:0/1743126946 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f929006f400 msgr2=0x7f9290071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.685+0000 7f92bbc00640 1 --2- 172.21.15.86:0/1743126946 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f929006f400 0x7f9290071910 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f92a40af000 tx=0x7f92a40a7040 comp rx=0 tx=0).stop 2024-04-25T14:18:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.685+0000 7f92bbc00640 1 -- 172.21.15.86:0/1743126946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92b41fcfe0 msgr2=0x7f92b41fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:18:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.685+0000 7f92bbc00640 1 --2- 172.21.15.86:0/1743126946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92b41fcfe0 0x7f92b41fd3f0 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f92b40b2f70 tx=0x7f92ac00e820 comp rx=0 tx=0).stop 2024-04-25T14:18:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.685+0000 7f92bbc00640 1 -- 172.21.15.86:0/1743126946 shutdown_connections 2024-04-25T14:18:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.685+0000 7f92bbc00640 1 --2- 172.21.15.86:0/1743126946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92b41fcfe0 0x7f92b41fd3f0 unknown :-1 s=CLOSED pgs=2532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.685+0000 7f92bbc00640 1 --2- 172.21.15.86:0/1743126946 >> [v2:172.21.15.86:6832/3174179029,v1:172.21.15.86:6833/3174179029] conn(0x7f929006f400 0x7f9290071910 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.685+0000 7f92bbc00640 1 --2- 172.21.15.86:0/1743126946 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92b415e880 0x7f92b415ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:34.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.685+0000 7f92bbc00640 1 --2- 172.21.15.86:0/1743126946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9298003bc0 0x7f92b40b7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:18:34.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.685+0000 7f92bbc00640 1 -- 172.21.15.86:0/1743126946 >> 172.21.15.86:0/1743126946 conn(0x7f92b40cffc0 msgr2=0x7f92b40b4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:18:34.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.685+0000 7f92bbc00640 1 -- 172.21.15.86:0/1743126946 shutdown_connections 2024-04-25T14:18:34.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-25T14:18:34.685+0000 7f92bbc00640 1 -- 172.21.15.86:0/1743126946 wait complete. 2024-04-25T14:18:34.699 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T14:18:34.699 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T14:18:34.699 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T14:18:34.699 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:18:34.720 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T14:18:34.720 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T14:18:34.720 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:18:34.773 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T14:18:34.773 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T14:18:34.773 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:18:34.796 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T14:18:34.796 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T14:18:34.796 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:18:34.853 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T14:18:34.853 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T14:18:34.853 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T14:18:34.853 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:18:34.935 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T14:18:34.935 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T14:18:34.935 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:18:35.019 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T14:18:35.019 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T14:18:35.020 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:18:38.454 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-04-25T14:18:41.122 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T14:18:41.122 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T14:18:41.123 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:18:41.187 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T14:18:41.187 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T14:18:41.187 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:18:41.274 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T14:18:41.274 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T14:18:41.275 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:18:44.160 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-04-25T14:18:44.161 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-04-25T14:18:47.377 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T14:18:47.377 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T14:18:47.377 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:18:47.473 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T14:18:47.473 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T14:18:47.473 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:18:49.666 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-04-25T14:18:49.666 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-04-25T14:18:50.899 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:18:50.902 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:18:53.576 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T14:18:53.576 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T14:18:53.577 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T14:18:53.577 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:18:53.653 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T14:18:53.653 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T14:18:53.653 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:18:53.718 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T14:18:53.718 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T14:18:53.719 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T14:18:53.719 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:18:53.741 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T14:18:53.741 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T14:18:53.741 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:18:53.759 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T14:18:53.760 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T14:18:53.760 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T14:18:53.825 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T14:18:53.825 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T14:18:53.826 DEBUG:teuthology.orchestra.run.smithi086:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T14:18:53.858 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T14:18:53.858 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T14:18:54.143 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T14:18:54.143 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T14:18:54.367 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T14:18:54.367 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T14:18:54.553 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T14:18:54.553 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T14:18:54.668 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-04-25T14:18:54.754 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T14:18:54.754 DEBUG:teuthology.orchestra.run.smithi155:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T14:18:54.995 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T14:18:54.995 DEBUG:teuthology.orchestra.run.smithi155:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T14:18:55.183 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T14:18:55.183 DEBUG:teuthology.orchestra.run.smithi155:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T14:18:55.365 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T14:18:55.366 DEBUG:teuthology.orchestra.run.smithi155:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T14:18:55.541 INFO:tasks.ceph:Archiving mon data... 2024-04-25T14:18:55.543 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673185/data/mon.a.tgz 2024-04-25T14:18:55.543 DEBUG:teuthology.orchestra.run.smithi086:> mktemp 2024-04-25T14:18:55.552 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/tmp.4a2LV081FF 2024-04-25T14:18:55.552 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.4a2LV081FF 2024-04-25T14:18:56.035 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0666 /tmp/tmp.4a2LV081FF 2024-04-25T14:18:56.110 DEBUG:teuthology.orchestra.remote:smithi086:/tmp/tmp.4a2LV081FF is 3MB 2024-04-25T14:18:56.206 DEBUG:teuthology.orchestra.run.smithi086:> rm -fr /tmp/tmp.4a2LV081FF 2024-04-25T14:18:56.215 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673185/data/mon.b.tgz 2024-04-25T14:18:56.215 DEBUG:teuthology.orchestra.run.smithi155:> mktemp 2024-04-25T14:18:56.225 INFO:teuthology.orchestra.run.smithi155.stdout:/tmp/tmp.97xFA79tsj 2024-04-25T14:18:56.225 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.97xFA79tsj 2024-04-25T14:18:56.702 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0666 /tmp/tmp.97xFA79tsj 2024-04-25T14:18:56.779 DEBUG:teuthology.orchestra.remote:smithi155:/tmp/tmp.97xFA79tsj is 3MB 2024-04-25T14:18:56.883 DEBUG:teuthology.orchestra.run.smithi155:> rm -fr /tmp/tmp.97xFA79tsj 2024-04-25T14:18:56.892 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673185/data/mon.c.tgz 2024-04-25T14:18:56.893 DEBUG:teuthology.orchestra.run.smithi155:> mktemp 2024-04-25T14:18:56.946 INFO:teuthology.orchestra.run.smithi155.stdout:/tmp/tmp.wYPSAj15Tx 2024-04-25T14:18:56.947 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.wYPSAj15Tx 2024-04-25T14:18:57.418 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0666 /tmp/tmp.wYPSAj15Tx 2024-04-25T14:18:57.491 DEBUG:teuthology.orchestra.remote:smithi155:/tmp/tmp.wYPSAj15Tx is 3MB 2024-04-25T14:18:57.588 DEBUG:teuthology.orchestra.run.smithi155:> rm -fr /tmp/tmp.wYPSAj15Tx 2024-04-25T14:18:57.597 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T14:18:57.598 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T14:18:57.601 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T14:18:57.674 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T14:18:57.711 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T14:18:57.711 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T14:18:57.713 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673185/remote/smithi086/crash 2024-04-25T14:18:57.714 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T14:18:57.743 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673185/remote/smithi155/crash 2024-04-25T14:18:57.744 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T14:18:57.773 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T14:18:57.773 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:18:57.789 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T14:18:57.864 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T14:18:57.885 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T14:18:57.903 INFO:tasks.ceph:Compressing logs... 2024-04-25T14:18:57.904 DEBUG:teuthology.orchestra.run.smithi086:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T14:18:57.932 DEBUG:teuthology.orchestra.run.smithi155:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T14:18:57.966 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62197.log 2024-04-25T14:18:57.967 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57538.log 2024-04-25T14:18:57.967 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44209.log 2024-04-25T14:18:57.968 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62197.log.gz 2024-04-25T14:18:57.968 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.57538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54153.log 2024-04-25T14:18:57.968 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.57538.log.gz 2024-04-25T14:18:57.969 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58286.log 2024-04-25T14:18:57.970 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54153.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44209.log.gz 2024-04-25T14:18:57.970 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44537.log 2024-04-25T14:18:57.970 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54153.log.gz 2024-04-25T14:18:57.971 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49797.log 2024-04-25T14:18:57.971 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58286.log.gz 2024-04-25T14:18:57.972 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42901.log 2024-04-25T14:18:57.972 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44537.log.gz 2024-04-25T14:18:57.972 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50292.log 2024-04-25T14:18:57.973 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49797.log.gz 2024-04-25T14:18:57.974 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40410.log 2024-04-25T14:18:57.974 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42901.log.gz 2024-04-25T14:18:57.974 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34315.log 2024-04-25T14:18:57.975 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48242.log 2024-04-25T14:18:57.975 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50292.log.gz 2024-04-25T14:18:57.975 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45358.log 2024-04-25T14:18:57.976 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40410.log.gz 2024-04-25T14:18:57.978 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44991.log 2024-04-25T14:18:57.978 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48242.log.gz 2024-04-25T14:18:57.978 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35202.log 2024-04-25T14:18:57.979 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45358.log.gz 2024-04-25T14:18:57.979 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55784.log 2024-04-25T14:18:57.979 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44991.log.gz 2024-04-25T14:18:57.980 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27728.log 2024-04-25T14:18:57.980 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.34315.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T14:18:57.980 INFO:teuthology.orchestra.run.smithi155.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34315.log.gz 2024-04-25T14:18:57.980 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.27728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27752.log 2024-04-25T14:18:57.980 INFO:teuthology.orchestra.run.smithi155.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.27728.log.gz 2024-04-25T14:18:57.980 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27848.log 2024-04-25T14:18:57.980 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.27752.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T14:18:57.981 INFO:teuthology.orchestra.run.smithi155.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27752.log.gz 2024-04-25T14:18:57.981 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.27848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33775.log 2024-04-25T14:18:57.981 INFO:teuthology.orchestra.run.smithi155.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.27848.log.gz 2024-04-25T14:18:57.981 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34936.log 2024-04-25T14:18:57.987 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.33775.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33775.log.gz 2024-04-25T14:18:57.987 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33333.log 2024-04-25T14:18:57.987 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.34936.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34936.log.gz 2024-04-25T14:18:57.987 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34931.log 2024-04-25T14:18:57.987 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.33333.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33333.log.gz 2024-04-25T14:18:57.987 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34532.log 2024-04-25T14:18:57.987 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.34931.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34931.log.gz 2024-04-25T14:18:57.987 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27776.log 2024-04-25T14:18:57.988 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.34532.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34532.log.gz 2024-04-25T14:18:57.988 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T14:18:57.988 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.27776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33435.log 2024-04-25T14:18:57.988 INFO:teuthology.orchestra.run.smithi155.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27776.log.gz 2024-04-25T14:18:57.988 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33944.log 2024-04-25T14:18:57.988 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.3.33435.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.33435.log.gz 2024-04-25T14:18:57.988 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27800.log 2024-04-25T14:18:57.991 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45517.log 2024-04-25T14:18:57.991 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35202.log.gz 2024-04-25T14:18:57.991 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60674.log 2024-04-25T14:18:57.991 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55784.log.gz 2024-04-25T14:18:57.991 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37724.log 2024-04-25T14:18:57.992 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45517.log.gz 2024-04-25T14:18:57.992 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55354.log 2024-04-25T14:18:57.992 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60674.log.gz 2024-04-25T14:18:57.992 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53698.log 2024-04-25T14:18:57.992 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37724.log.gz 2024-04-25T14:18:57.992 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52589.log 2024-04-25T14:18:57.992 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55354.log.gz 2024-04-25T14:18:57.992 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35015.log 2024-04-25T14:18:57.993 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53698.log.gz 2024-04-25T14:18:57.993 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42063.log 2024-04-25T14:18:57.993 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52589.log.gz 2024-04-25T14:18:57.993 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36661.log 2024-04-25T14:18:57.993 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35015.log.gz 2024-04-25T14:18:57.993 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54823.log 2024-04-25T14:18:57.993 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42063.log.gz 2024-04-25T14:18:57.993 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51162.log 2024-04-25T14:18:57.993 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36661.log.gz 2024-04-25T14:18:57.994 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34802.log 2024-04-25T14:18:57.994 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54823.log.gz 2024-04-25T14:18:57.994 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45904.log 2024-04-25T14:18:57.994 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51162.log.gz 2024-04-25T14:18:57.994 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38884.log 2024-04-25T14:18:57.994 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34802.log.gz 2024-04-25T14:18:57.994 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55302.log 2024-04-25T14:18:57.994 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45904.log.gz 2024-04-25T14:18:57.995 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42446.log 2024-04-25T14:18:57.995 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38884.log.gz 2024-04-25T14:18:57.995 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42869.log 2024-04-25T14:18:58.001 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.33944.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33944.log.gz 2024-04-25T14:18:58.002 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T14:18:58.002 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.27800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34527.log 2024-04-25T14:18:58.002 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-mgr.x.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27800.log.gz 2024-04-25T14:18:58.002 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T14:18:58.002 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.34527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27824.log 2024-04-25T14:18:58.002 INFO:teuthology.orchestra.run.smithi155.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34527.log.gz 2024-04-25T14:18:58.002 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33351.log 2024-04-25T14:18:58.002 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.27824.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T14:18:58.003 INFO:teuthology.orchestra.run.smithi155.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.27824.log.gz 2024-04-25T14:18:58.003 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.33351.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.1.33351.log.gz -- 2024-04-25T14:18:58.003 INFO:teuthology.orchestra.run.smithi155.stderr: /var/log/ceph/ceph-client.1.33770.log 2024-04-25T14:18:58.003 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33577.log 2024-04-25T14:18:58.003 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.33770.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33770.log.gz 2024-04-25T14:18:58.003 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34741.log 2024-04-25T14:18:58.003 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.33577.log: 92.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T14:18:58.003 INFO:teuthology.orchestra.run.smithi155.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33577.log.gz 2024-04-25T14:18:58.004 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35146.log 2024-04-25T14:18:58.004 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.34741.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34741.log.gz 2024-04-25T14:18:58.004 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34736.log 2024-04-25T14:18:58.004 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.35146.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T14:18:58.004 INFO:teuthology.orchestra.run.smithi155.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.35146.log.gz 2024-04-25T14:18:58.004 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.34736.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34736.log.gz 2024-04-25T14:18:58.004 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33582.log 2024-04-25T14:18:58.006 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T14:18:58.006 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.33582.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33582.log.gz 2024-04-25T14:18:58.008 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55302.log.gz 2024-04-25T14:18:58.009 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39440.log 2024-04-25T14:18:58.009 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42446.log.gz 2024-04-25T14:18:58.009 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36453.log 2024-04-25T14:18:58.009 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42869.log.gz 2024-04-25T14:18:58.009 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39440.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T14:18:58.009 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39440.log.gz 2024-04-25T14:18:58.009 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36453.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36453.log.gz -5 2024-04-25T14:18:58.009 INFO:teuthology.orchestra.run.smithi086.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49102.log 2024-04-25T14:18:58.010 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39877.log 2024-04-25T14:18:58.010 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37386.log 2024-04-25T14:18:58.010 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49102.log.gz 2024-04-25T14:18:58.010 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58364.log 2024-04-25T14:18:58.010 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39877.log.gz 2024-04-25T14:18:58.010 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62481.log 2024-04-25T14:18:58.010 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37386.log.gz 2024-04-25T14:18:58.010 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58364.log: gzip 88.6% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.58364.log.gz -- 2024-04-25T14:18:58.011 INFO:teuthology.orchestra.run.smithi086.stderr: /var/log/ceph/ceph-client.admin.38910.log 2024-04-25T14:18:58.011 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57337.log 2024-04-25T14:18:58.011 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62481.log.gz 2024-04-25T14:18:58.011 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51568.log 2024-04-25T14:18:58.011 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38910.log.gz 2024-04-25T14:18:58.011 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38033.log 2024-04-25T14:18:58.011 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57337.log.gz 2024-04-25T14:18:58.011 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50584.log 2024-04-25T14:18:58.011 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51568.log.gz 2024-04-25T14:18:58.012 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49630.log 2024-04-25T14:18:58.012 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38033.log.gz 2024-04-25T14:18:58.012 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38595.log 2024-04-25T14:18:58.012 INFO:teuthology.orchestra.run.smithi086.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50584.log.gz 2024-04-25T14:18:58.012 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34552.log 2024-04-25T14:18:58.012 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49630.log.gz 2024-04-25T14:18:58.012 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.38595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48269.log 2024-04-25T14:18:58.012 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.38595.log.gz 2024-04-25T14:18:58.013 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52353.log 2024-04-25T14:18:58.013 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34552.log.gz 2024-04-25T14:18:58.013 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56186.log 2024-04-25T14:18:58.013 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48269.log.gz 2024-04-25T14:18:58.013 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38740.log 2024-04-25T14:18:58.013 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52353.log.gz 2024-04-25T14:18:58.015 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27704.log 2024-04-25T14:18:58.016 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33440.log 2024-04-25T14:18:58.016 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.27704.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T14:18:58.016 INFO:teuthology.orchestra.run.smithi155.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.27704.log.gz 2024-04-25T14:18:58.016 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.3.33440.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.33440.log.gz 2024-04-25T14:18:58.016 INFO:teuthology.orchestra.run.smithi155.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T14:18:58.016 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T14:18:58.016 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33949.log 2024-04-25T14:18:58.016 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27680.log 2024-04-25T14:18:58.020 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.33949.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33949.log.gz 2024-04-25T14:18:58.021 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50090.log 2024-04-25T14:18:58.021 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56186.log.gz 2024-04-25T14:18:58.021 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60487.log 2024-04-25T14:18:58.021 INFO:teuthology.orchestra.run.smithi086.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38740.log.gz 2024-04-25T14:18:58.021 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54244.log 2024-04-25T14:18:58.022 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50090.log.gz 2024-04-25T14:18:58.022 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39124.log 2024-04-25T14:18:58.022 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60487.log.gz 2024-04-25T14:18:58.022 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37952.log 2024-04-25T14:18:58.022 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54244.log.gz 2024-04-25T14:18:58.022 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43246.log 2024-04-25T14:18:58.022 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39124.log.gz 2024-04-25T14:18:58.022 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59912.log 2024-04-25T14:18:58.023 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37952.log.gz 2024-04-25T14:18:58.023 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43246.log: gzip 89.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.43246.log.gz --verbose 2024-04-25T14:18:58.023 INFO:teuthology.orchestra.run.smithi086.stderr: -- /var/log/ceph/ceph-client.admin.47400.log 2024-04-25T14:18:58.023 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43508.log 2024-04-25T14:18:58.023 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.59912.log.gz 2024-04-25T14:18:58.023 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58442.log 2024-04-25T14:18:58.023 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47400.log.gz 2024-04-25T14:18:58.023 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54665.log 2024-04-25T14:18:58.023 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43508.log.gz 2024-04-25T14:18:58.024 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54049.log 2024-04-25T14:18:58.024 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58442.log.gz 2024-04-25T14:18:58.024 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54309.log 2024-04-25T14:18:58.024 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54665.log.gz 2024-04-25T14:18:58.024 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58600.log 2024-04-25T14:18:58.024 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54049.log.gz 2024-04-25T14:18:58.025 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T14:18:58.025 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.27680.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27680.log.gz 2024-04-25T14:18:58.028 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56565.log 2024-04-25T14:18:58.028 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54309.log.gz 2024-04-25T14:18:58.028 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41549.log 2024-04-25T14:18:58.028 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58600.log.gz 2024-04-25T14:18:58.028 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61148.log 2024-04-25T14:18:58.028 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56565.log.gz 2024-04-25T14:18:58.028 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38467.log 2024-04-25T14:18:58.029 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41549.log.gz 2024-04-25T14:18:58.029 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39739.log 2024-04-25T14:18:58.029 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61148.log.gz 2024-04-25T14:18:58.029 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56009.log 2024-04-25T14:18:58.029 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38467.log.gz 2024-04-25T14:18:58.029 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33313.log 2024-04-25T14:18:58.029 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39739.log.gz 2024-04-25T14:18:58.029 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45332.log 2024-04-25T14:18:58.030 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56009.log.gz 2024-04-25T14:18:58.030 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34133.log 2024-04-25T14:18:58.032 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47257.log 2024-04-25T14:18:58.032 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33313.log.gz 2024-04-25T14:18:58.032 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39981.log 2024-04-25T14:18:58.032 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45332.log.gz 2024-04-25T14:18:58.032 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57389.log 2024-04-25T14:18:58.033 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47257.log.gz 2024-04-25T14:18:58.033 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39981.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T14:18:58.033 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39981.log.gz 2024-04-25T14:18:58.033 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42732.log 2024-04-25T14:18:58.033 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57389.log.gz 2024-04-25T14:18:58.034 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.34133.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34133.log.gz 2024-04-25T14:18:58.035 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62975.log 2024-04-25T14:18:58.035 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36773.log 2024-04-25T14:18:58.036 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42732.log.gz 2024-04-25T14:18:58.036 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62975.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62975.log.gz 2024-04-25T14:18:58.036 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35961.log 2024-04-25T14:18:58.036 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36401.log 2024-04-25T14:18:58.037 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35151.log 2024-04-25T14:18:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36773.log: /var/log/ceph/ceph-client.2.35961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41121.log 2024-04-25T14:18:58.038 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36773.log.gz 2024-04-25T14:18:58.038 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.35961.log.gz 2024-04-25T14:18:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59623.log 2024-04-25T14:18:58.038 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36401.log.gz 2024-04-25T14:18:58.039 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42037.log 2024-04-25T14:18:58.039 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41121.log.gz 2024-04-25T14:18:58.039 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46550.log 2024-04-25T14:18:58.040 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59623.log.gz 2024-04-25T14:18:58.040 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33737.log 2024-04-25T14:18:58.040 INFO:teuthology.orchestra.run.smithi086.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42037.log.gz 2024-04-25T14:18:58.041 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.35151.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.35151.log.gz 2024-04-25T14:18:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46654.log 2024-04-25T14:18:58.042 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46550.log.gz 2024-04-25T14:18:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41019.log 2024-04-25T14:18:58.043 INFO:teuthology.orchestra.run.smithi086.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.33737.log.gz 2024-04-25T14:18:58.043 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34128.log 2024-04-25T14:18:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45306.log 2024-04-25T14:18:58.044 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46654.log.gz 2024-04-25T14:18:58.045 INFO:teuthology.orchestra.run.smithi155.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T14:18:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62249.log 2024-04-25T14:18:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47153.log 2024-04-25T14:18:58.046 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45306.log.gz 2024-04-25T14:18:58.046 INFO:teuthology.orchestra.run.smithi086.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.41019.log.gz 2024-04-25T14:18:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61702.log 2024-04-25T14:18:58.046 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62249.log.gz 2024-04-25T14:18:58.048 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34320.log 2024-04-25T14:18:58.048 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.34128.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34128.log.gz 2024-04-25T14:18:58.049 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49248.log 2024-04-25T14:18:58.049 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47153.log.gz 2024-04-25T14:18:58.049 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54849.log 2024-04-25T14:18:58.049 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.61702.log.gz 2024-04-25T14:18:58.049 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60752.log 2024-04-25T14:18:58.049 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49248.log.gz 2024-04-25T14:18:58.050 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.34320.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34320.log.gz 2024-04-25T14:18:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41199.log 2024-04-25T14:18:58.051 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54849.log.gz 2024-04-25T14:18:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61818.log 2024-04-25T14:18:58.052 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60752.log.gz 2024-04-25T14:18:58.052 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50610.log 2024-04-25T14:18:58.052 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41199.log.gz 2024-04-25T14:18:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41633.log/var/log/ceph/ceph-client.admin.61818.log: 2024-04-25T14:18:58.053 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61818.log.gz 2024-04-25T14:18:58.054 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36747.log 2024-04-25T14:18:58.054 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50610.log.gz 2024-04-25T14:18:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45956.log 2024-04-25T14:18:58.055 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41633.log.gz 2024-04-25T14:18:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36747.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36747.log.gz 2024-04-25T14:18:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39343.log 2024-04-25T14:18:58.057 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45163.log 2024-04-25T14:18:58.057 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45956.log.gz 2024-04-25T14:18:58.057 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35022.log 2024-04-25T14:18:58.058 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39343.log.gz 2024-04-25T14:18:58.058 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44495.log 2024-04-25T14:18:58.059 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45163.log.gz 2024-04-25T14:18:58.059 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52873.log 2024-04-25T14:18:58.059 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35022.log.gz 2024-04-25T14:18:58.060 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53208.log 2024-04-25T14:18:58.060 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44495.log.gz 2024-04-25T14:18:58.061 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44443.log 2024-04-25T14:18:58.061 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52873.log.gz 2024-04-25T14:18:58.062 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34604.log 2024-04-25T14:18:58.062 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53208.log.gz 2024-04-25T14:18:58.063 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62171.log 2024-04-25T14:18:58.063 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44443.log.gz 2024-04-25T14:18:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42654.log 2024-04-25T14:18:58.064 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34604.log.gz 2024-04-25T14:18:58.065 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61435.log 2024-04-25T14:18:58.065 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62171.log.gz 2024-04-25T14:18:58.066 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49745.log 2024-04-25T14:18:58.066 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42654.log.gz 2024-04-25T14:18:58.067 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34796.log 2024-04-25T14:18:58.067 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61435.log.gz 2024-04-25T14:18:58.067 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47700.log 2024-04-25T14:18:58.068 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49745.log.gz 2024-04-25T14:18:58.068 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60383.log 2024-04-25T14:18:58.068 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34796.log.gz 2024-04-25T14:18:58.069 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57920.log 2024-04-25T14:18:58.069 INFO:teuthology.orchestra.run.smithi086.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.47700.log.gz 2024-04-25T14:18:58.070 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33286.log 2024-04-25T14:18:58.070 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60383.log.gz 2024-04-25T14:18:58.071 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36037.log 2024-04-25T14:18:58.071 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57920.log.gz 2024-04-25T14:18:58.072 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49170.log 2024-04-25T14:18:58.072 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33286.log.gz 2024-04-25T14:18:58.073 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61122.log 2024-04-25T14:18:58.073 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36037.log.gz 2024-04-25T14:18:58.073 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44012.log 2024-04-25T14:18:58.074 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49170.log.gz 2024-04-25T14:18:58.075 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44064.log 2024-04-25T14:18:58.075 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61122.log.gz 2024-04-25T14:18:58.076 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59938.log 2024-04-25T14:18:58.076 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44012.log.gz 2024-04-25T14:18:58.076 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37256.log 2024-04-25T14:18:58.077 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44064.log.gz 2024-04-25T14:18:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60043.log 2024-04-25T14:18:58.077 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59938.log.gz 2024-04-25T14:18:58.078 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37256.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37256.log.gz 2024-04-25T14:18:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55380.log 2024-04-25T14:18:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51542.log 2024-04-25T14:18:58.079 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60043.log.gz 2024-04-25T14:18:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56721.log 2024-04-25T14:18:58.080 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55380.log.gz 2024-04-25T14:18:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49849.log 2024-04-25T14:18:58.081 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51542.log.gz 2024-04-25T14:18:58.082 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52847.log 2024-04-25T14:18:58.082 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56721.log.gz 2024-04-25T14:18:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36271.log 2024-04-25T14:18:58.083 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49849.log.gz 2024-04-25T14:18:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46875.log 2024-04-25T14:18:58.084 INFO:teuthology.orchestra.run.smithi086.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.52847.log.gz 2024-04-25T14:18:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62275.log 2024-04-25T14:18:58.084 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36271.log.gz 2024-04-25T14:18:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62507.log 2024-04-25T14:18:58.085 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46875.log.gz 2024-04-25T14:18:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40302.log 2024-04-25T14:18:58.086 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62275.log.gz 2024-04-25T14:18:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33447.log 2024-04-25T14:18:58.087 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62507.log.gz 2024-04-25T14:18:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62398.log 2024-04-25T14:18:58.088 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40302.log.gz 2024-04-25T14:18:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61068.log 2024-04-25T14:18:58.089 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33447.log.gz 2024-04-25T14:18:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51776.log 2024-04-25T14:18:58.090 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62398.log.gz 2024-04-25T14:18:58.090 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54414.log 2024-04-25T14:18:58.090 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61068.log.gz 2024-04-25T14:18:58.091 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39687.log 2024-04-25T14:18:58.091 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51776.log.gz 2024-04-25T14:18:58.092 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46496.log 2024-04-25T14:18:58.092 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54414.log.gz 2024-04-25T14:18:58.093 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49222.log 2024-04-25T14:18:58.093 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39687.log.gz 2024-04-25T14:18:58.094 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54717.log 2024-04-25T14:18:58.094 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46496.log.gz 2024-04-25T14:18:58.095 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34318.log 2024-04-25T14:18:58.095 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49222.log.gz 2024-04-25T14:18:58.095 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49391.log 2024-04-25T14:18:58.096 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54717.log.gz 2024-04-25T14:18:58.096 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55070.log 2024-04-25T14:18:58.097 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34318.log.gz 2024-04-25T14:18:58.097 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57610.log 2024-04-25T14:18:58.097 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49391.log.gz 2024-04-25T14:18:58.098 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63080.log 2024-04-25T14:18:58.098 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55070.log.gz 2024-04-25T14:18:58.099 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36193.log 2024-04-25T14:18:58.099 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57610.log.gz 2024-04-25T14:18:58.100 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59493.log 2024-04-25T14:18:58.100 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63080.log.gz 2024-04-25T14:18:58.101 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38792.log 2024-04-25T14:18:58.101 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36193.log.gz 2024-04-25T14:18:58.102 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58860.log 2024-04-25T14:18:58.102 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59493.log.gz 2024-04-25T14:18:58.103 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45017.log 2024-04-25T14:18:58.103 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38792.log.gz 2024-04-25T14:18:58.103 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53370.log 2024-04-25T14:18:58.104 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.58860.log.gz 2024-04-25T14:18:58.105 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50936.log 2024-04-25T14:18:58.105 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45017.log.gz 2024-04-25T14:18:58.105 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50662.log 2024-04-25T14:18:58.106 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53370.log.gz 2024-04-25T14:18:58.106 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53344.log 2024-04-25T14:18:58.106 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50936.log.gz 2024-04-25T14:18:58.107 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46576.log 2024-04-25T14:18:58.107 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50662.log.gz 2024-04-25T14:18:58.108 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48507.log 2024-04-25T14:18:58.108 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53344.log.gz 2024-04-25T14:18:58.109 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46732.log 2024-04-25T14:18:58.109 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46576.log.gz 2024-04-25T14:18:58.110 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58000.log 2024-04-25T14:18:58.110 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48507.log.gz 2024-04-25T14:18:58.111 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39929.log 2024-04-25T14:18:58.111 INFO:teuthology.orchestra.run.smithi086.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.46732.log.gz 2024-04-25T14:18:58.111 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47875.log 89.2% 2024-04-25T14:18:58.111 INFO:teuthology.orchestra.run.smithi086.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58000.log.gz 2024-04-25T14:18:58.112 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44159.log 2024-04-25T14:18:58.112 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39929.log.gz 2024-04-25T14:18:58.113 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36063.log 2024-04-25T14:18:58.113 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47875.log.gz 2024-04-25T14:18:58.114 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46901.log 2024-04-25T14:18:58.115 INFO:teuthology.orchestra.run.smithi086.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.44159.log.gz 2024-04-25T14:18:58.115 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58914.log 2024-04-25T14:18:58.115 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36063.log.gz 2024-04-25T14:18:58.116 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50790.log 2024-04-25T14:18:58.116 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46901.log.gz 2024-04-25T14:18:58.117 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45852.log 2024-04-25T14:18:58.117 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58914.log.gz 2024-04-25T14:18:58.118 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59389.log 2024-04-25T14:18:58.118 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50790.log.gz 2024-04-25T14:18:58.119 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36089.log 2024-04-25T14:18:58.119 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45852.log.gz 2024-04-25T14:18:58.120 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59860.log 2024-04-25T14:18:58.120 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59389.log.gz 2024-04-25T14:18:58.120 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35516.log 2024-04-25T14:18:58.121 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36089.log.gz 2024-04-25T14:18:58.121 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35436.log 2024-04-25T14:18:58.122 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59860.log.gz 2024-04-25T14:18:58.122 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41985.log 2024-04-25T14:18:58.122 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35516.log.gz 2024-04-25T14:18:58.123 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43352.log 2024-04-25T14:18:58.123 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35436.log.gz 2024-04-25T14:18:58.124 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40276.log 2024-04-25T14:18:58.124 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41985.log.gz 2024-04-25T14:18:58.125 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48295.log 2024-04-25T14:18:58.125 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43352.log.gz 2024-04-25T14:18:58.126 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54127.log 2024-04-25T14:18:58.126 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40276.log.gz 2024-04-25T14:18:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40664.log 2024-04-25T14:18:58.127 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48295.log.gz 2024-04-25T14:18:58.128 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35870.log 2024-04-25T14:18:58.128 INFO:teuthology.orchestra.run.smithi086.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.54127.log.gz 2024-04-25T14:18:58.129 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40664.log: gzip 88.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.40664.log.gz --verbose 2024-04-25T14:18:58.129 INFO:teuthology.orchestra.run.smithi086.stderr: -- /var/log/ceph/ceph-client.admin.41355.log 2024-04-25T14:18:58.129 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34589.log 2024-04-25T14:18:58.130 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35870.log.gz 2024-04-25T14:18:58.130 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35828.log 2024-04-25T14:18:58.131 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41355.log.gz 2024-04-25T14:18:58.131 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46829.log 2024-04-25T14:18:58.132 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34589.log.gz 2024-04-25T14:18:58.132 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.35828.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.35828.log.gz 2024-04-25T14:18:58.133 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42141.log 2024-04-25T14:18:58.133 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.46829.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46829.log.gz 2024-04-25T14:18:58.133 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37360.log 2024-04-25T14:18:58.134 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53945.log 2024-04-25T14:18:58.134 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42141.log.gz 2024-04-25T14:18:58.135 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60147.log 2024-04-25T14:18:58.135 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37360.log.gz 2024-04-25T14:18:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35306.log 2024-04-25T14:18:58.136 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53945.log.gz 2024-04-25T14:18:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46834.log 2024-04-25T14:18:58.137 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60147.log.gz 2024-04-25T14:18:58.138 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37925.log 2024-04-25T14:18:58.138 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35306.log.gz 2024-04-25T14:18:58.138 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.46834.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46834.log.gz 2024-04-25T14:18:58.139 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T14:18:58.139 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56416.log 2024-04-25T14:18:58.140 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37925.log.gz 2024-04-25T14:18:58.140 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36908.log 2024-04-25T14:18:58.141 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59571.log 2024-04-25T14:18:58.141 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56416.log.gz 2024-04-25T14:18:58.142 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48843.log 2024-04-25T14:18:58.142 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36908.log.gz 2024-04-25T14:18:58.143 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35016.log 2024-04-25T14:18:58.143 INFO:teuthology.orchestra.run.smithi086.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.59571.log.gz 2024-04-25T14:18:58.144 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.48843.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48843.log.gz 2024-04-25T14:18:58.144 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60620.log 2024-04-25T14:18:58.145 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35016.log: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.35016.log.gz 2024-04-25T14:18:58.145 INFO:teuthology.orchestra.run.smithi086.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55867.log 2024-04-25T14:18:58.146 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46036.log 2024-04-25T14:18:58.147 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60620.log: /var/log/ceph/ceph-client.admin.55867.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60620.log.gz 2024-04-25T14:18:58.147 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36011.log 2024-04-25T14:18:58.147 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55867.log.gz 2024-04-25T14:18:58.148 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40992.log 2024-04-25T14:18:58.148 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46036.log.gz 2024-04-25T14:18:58.149 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55539.log 2024-04-25T14:18:58.149 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36011.log.gz/var/log/ceph/ceph-client.admin.40992.log: 2024-04-25T14:18:58.150 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43129.log 2024-04-25T14:18:58.150 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40992.log.gz 2024-04-25T14:18:58.151 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36851.log 2024-04-25T14:18:58.152 INFO:teuthology.orchestra.run.smithi086.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.55539.log.gz 2024-04-25T14:18:58.152 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51594.log 2024-04-25T14:18:58.153 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43129.log.gz 2024-04-25T14:18:58.153 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35020.log 2024-04-25T14:18:58.154 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36851.log.gz 2024-04-25T14:18:58.155 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45930.log 2024-04-25T14:18:58.155 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51594.log.gz 2024-04-25T14:18:58.155 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60912.log 2024-04-25T14:18:58.156 INFO:teuthology.orchestra.run.smithi086.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35020.log.gz 2024-04-25T14:18:58.156 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45930.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45930.log.gz 2024-04-25T14:18:58.157 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61624.log 2024-04-25T14:18:58.157 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5/var/log/ceph/ceph-client.admin.60912.log: --verbose -- /var/log/ceph/ceph-client.admin.51104.log 2024-04-25T14:18:58.158 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60912.log.gz 2024-04-25T14:18:58.159 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56468.log 2024-04-25T14:18:58.159 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61624.log.gz 2024-04-25T14:18:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39265.log 2024-04-25T14:18:58.160 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51104.log.gz 2024-04-25T14:18:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.56468.log: /var/log/ceph/ceph-client.admin.50187.log 2024-04-25T14:18:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39265.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56468.log.gz 2024-04-25T14:18:58.162 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34526.log 2024-04-25T14:18:58.162 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39265.log.gz 2024-04-25T14:18:58.163 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39799.log 2024-04-25T14:18:58.163 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50187.log.gz 2024-04-25T14:18:58.163 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61042.log 2024-04-25T14:18:58.164 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34526.log: /var/log/ceph/ceph-client.admin.39799.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34526.log.gz 2024-04-25T14:18:58.164 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49552.log 2024-04-25T14:18:58.165 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39799.log.gz 2024-04-25T14:18:58.166 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56591.log 2024-04-25T14:18:58.166 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61042.log.gz 2024-04-25T14:18:58.166 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39395.log 2024-04-25T14:18:58.167 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49552.log: /var/log/ceph/ceph-client.admin.56591.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49552.log.gz 2024-04-25T14:18:58.168 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33607.log 2024-04-25T14:18:58.168 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56591.log.gz 2024-04-25T14:18:58.169 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56062.log 2024-04-25T14:18:58.169 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39395.log.gz 2024-04-25T14:18:58.169 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57259.log 2024-04-25T14:18:58.170 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33607.log: /var/log/ceph/ceph-client.admin.56062.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33607.log.gz 2024-04-25T14:18:58.171 INFO:teuthology.orchestra.run.smithi086.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56062.log.gz 2024-04-25T14:18:58.171 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46782.log 2024-04-25T14:18:58.172 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34261.log 2024-04-25T14:18:58.172 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57259.log.gz 2024-04-25T14:18:58.173 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42550.log 2024-04-25T14:18:58.173 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46782.log.gz 2024-04-25T14:18:58.174 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37464.log 2024-04-25T14:18:58.174 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34261.log.gz 2024-04-25T14:18:58.175 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37594.log 2024-04-25T14:18:58.175 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42550.log.gz 2024-04-25T14:18:58.176 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35384.log 2024-04-25T14:18:58.176 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37464.log.gz 2024-04-25T14:18:58.177 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5/var/log/ceph/ceph-client.admin.37594.log: --verbose -- /var/log/ceph/ceph-client.admin.63030.log 2024-04-25T14:18:58.177 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7%/var/log/ceph/ceph-client.admin.35384.log: -- replaced with /var/log/ceph/ceph-client.admin.37594.log.gz 2024-04-25T14:18:58.178 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62533.log 2024-04-25T14:18:58.178 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35384.log.gz 2024-04-25T14:18:58.179 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53234.log 2024-04-25T14:18:58.179 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63030.log.gz 2024-04-25T14:18:58.180 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47347.log 2024-04-25T14:18:58.180 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62533.log.gz 2024-04-25T14:18:58.181 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53971.log 2024-04-25T14:18:58.181 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53234.log.gz 2024-04-25T14:18:58.182 INFO:teuthology.orchestra.run.smithi155.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T14:18:58.183 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57415.log 2024-04-25T14:18:58.183 INFO:teuthology.orchestra.run.smithi086.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47347.log.gz 2024-04-25T14:18:58.183 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47637.log 2024-04-25T14:18:58.184 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57415.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53971.log.gz 2024-04-25T14:18:58.184 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60539.log 2024-04-25T14:18:58.184 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57415.log.gz 2024-04-25T14:18:58.185 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46299.log 2024-04-25T14:18:58.185 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47637.log.gz 2024-04-25T14:18:58.186 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40198.log 2024-04-25T14:18:58.187 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60539.log: /var/log/ceph/ceph-client.admin.46299.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60539.log.gz 2024-04-25T14:18:58.187 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63001.log 2024-04-25T14:18:58.187 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46299.log.gz 2024-04-25T14:18:58.188 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38818.log 2024-04-25T14:18:58.188 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40198.log.gz 2024-04-25T14:18:58.189 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34448.log 2024-04-25T14:18:58.190 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38818.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63001.log.gz 2024-04-25T14:18:58.190 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38818.log.gz 2024-04-25T14:18:58.190 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34235.log 2024-04-25T14:18:58.191 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34448.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T14:18:58.192 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34448.log.gz 2024-04-25T14:18:58.193 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50842.log 2024-04-25T14:18:58.193 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34235.log.gz 2024-04-25T14:18:58.193 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46706.log 2024-04-25T14:18:58.194 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.50842.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50842.log.gz 2024-04-25T14:18:58.195 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54101.log 2024-04-25T14:18:58.196 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42706.log 2024-04-25T14:18:58.196 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46706.log.gz 2024-04-25T14:18:58.197 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37808.log 2024-04-25T14:18:58.197 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54101.log.gz 2024-04-25T14:18:58.197 INFO:teuthology.orchestra.run.smithi155.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T14:18:58.198 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38337.log 2024-04-25T14:18:58.199 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42706.log.gz 2024-04-25T14:18:58.199 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34793.log 2024-04-25T14:18:58.199 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37808.log.gz 2024-04-25T14:18:58.200 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53891.log 2024-04-25T14:18:58.200 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38337.log.gz 2024-04-25T14:18:58.201 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57363.log 2024-04-25T14:18:58.201 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34793.log.gz 2024-04-25T14:18:58.202 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58704.log 2024-04-25T14:18:58.202 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53891.log.gz 2024-04-25T14:18:58.203 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59808.log 2024-04-25T14:18:58.203 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57363.log.gz 2024-04-25T14:18:58.204 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60331.log 2024-04-25T14:18:58.204 INFO:teuthology.orchestra.run.smithi086.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.58704.log.gz 2024-04-25T14:18:58.205 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45189.log 2024-04-25T14:18:58.206 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59808.log: /var/log/ceph/ceph-client.admin.60331.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59808.log.gz 2024-04-25T14:18:58.206 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54769.log 2024-04-25T14:18:58.206 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60331.log.gz 2024-04-25T14:18:58.207 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54613.log 2024-04-25T14:18:58.208 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45189.log.gz 2024-04-25T14:18:58.208 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47979.log 2024-04-25T14:18:58.208 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54769.log.gz 2024-04-25T14:18:58.209 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49338.log 2024-04-25T14:18:58.209 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54613.log.gz 2024-04-25T14:18:58.210 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53839.log 2024-04-25T14:18:58.211 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47979.log.gz 2024-04-25T14:18:58.211 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39492.log 2024-04-25T14:18:58.211 INFO:teuthology.orchestra.run.smithi086.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49338.log.gz 2024-04-25T14:18:58.212 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37197.log 2024-04-25T14:18:58.213 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53839.log.gz 2024-04-25T14:18:58.213 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62741.log 2024-04-25T14:18:58.214 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39492.log.gz 2024-04-25T14:18:58.215 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42498.log 2024-04-25T14:18:58.215 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37197.log.gz 2024-04-25T14:18:58.216 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59044.log 2024-04-25T14:18:58.216 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62741.log.gz 2024-04-25T14:18:58.216 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36687.log 2024-04-25T14:18:58.217 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42498.log.gz 2024-04-25T14:18:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52899.log 2024-04-25T14:18:58.218 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59044.log.gz 2024-04-25T14:18:58.219 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46010.log 2024-04-25T14:18:58.219 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36687.log.gz 2024-04-25T14:18:58.220 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58312.log 2024-04-25T14:18:58.220 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52899.log.gz 2024-04-25T14:18:58.221 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55462.log 2024-04-25T14:18:58.221 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46010.log.gz 2024-04-25T14:18:58.222 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55893.log 2024-04-25T14:18:58.222 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58312.log.gz 2024-04-25T14:18:58.223 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42758.log 2024-04-25T14:18:58.223 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55462.log.gz 2024-04-25T14:18:58.224 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60700.log 2024-04-25T14:18:58.224 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55893.log.gz 2024-04-25T14:18:58.225 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51906.log 2024-04-25T14:18:58.225 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42758.log.gz 2024-04-25T14:18:58.226 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42472.log 2024-04-25T14:18:58.227 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60700.log.gz 2024-04-25T14:18:58.227 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51802.log 2024-04-25T14:18:58.227 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51906.log.gz 2024-04-25T14:18:58.228 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37776.log 2024-04-25T14:18:58.229 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42472.log.gz 2024-04-25T14:18:58.229 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51672.log 2024-04-25T14:18:58.229 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51802.log.gz 2024-04-25T14:18:58.230 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47205.log 2024-04-25T14:18:58.231 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37776.log.gz 2024-04-25T14:18:58.231 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44771.log 2024-04-25T14:18:58.231 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51672.log.gz 2024-04-25T14:18:58.232 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54179.log 2024-04-25T14:18:58.233 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47205.log.gz 2024-04-25T14:18:58.233 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45111.log 2024-04-25T14:18:58.233 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44771.log.gz 2024-04-25T14:18:58.234 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36245.log 2024-04-25T14:18:58.235 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54179.log.gz 2024-04-25T14:18:58.235 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42969.log 2024-04-25T14:18:58.236 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45111.log: /var/log/ceph/ceph-client.admin.36245.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45111.log.gz 2024-04-25T14:18:58.236 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60513.log 2024-04-25T14:18:58.237 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36245.log.gz 2024-04-25T14:18:58.237 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33711.log 2024-04-25T14:18:58.238 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42969.log.gz 2024-04-25T14:18:58.238 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53539.log 2024-04-25T14:18:58.239 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60513.log.gz 2024-04-25T14:18:58.239 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52511.log 2024-04-25T14:18:58.240 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33711.log.gz 2024-04-25T14:18:58.240 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58886.log 2024-04-25T14:18:58.241 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53539.log.gz 2024-04-25T14:18:58.241 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39903.log 2024-04-25T14:18:58.242 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52511.log.gz 2024-04-25T14:18:58.243 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58886.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58886.log.gz 2024-04-25T14:18:58.243 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50349.log 2024-04-25T14:18:58.244 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40122.log 2024-04-25T14:18:58.244 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39903.log.gz 2024-04-25T14:18:58.245 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50349.log: gzip -5 --verbose 89.4% -- -- replaced with /var/log/ceph/ceph-client.admin.50349.log.gz /var/log/ceph/ceph-client.admin.38059.log 2024-04-25T14:18:58.246 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-04-25T14:18:58.246 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51620.log 2024-04-25T14:18:58.247 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8%/var/log/ceph/ceph-client.admin.38059.log: -- replaced with /var/log/ceph/ceph-client.admin.40122.log.gz 2024-04-25T14:18:58.247 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41225.log 2024-04-25T14:18:58.247 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38059.log.gz 2024-04-25T14:18:58.248 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34077.log 2024-04-25T14:18:58.248 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51620.log.gz 2024-04-25T14:18:58.249 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/ceph-client.admin.41225.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43908.log 2024-04-25T14:18:58.250 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41225.log.gz 2024-04-25T14:18:58.251 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34474.log 2024-04-25T14:18:58.251 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34077.log.gz 2024-04-25T14:18:58.251 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61513.log 2024-04-25T14:18:58.251 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43908.log.gz 2024-04-25T14:18:58.252 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34813.log 2024-04-25T14:18:58.253 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34474.log.gz 2024-04-25T14:18:58.254 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43164.log 2024-04-25T14:18:58.254 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61513.log.gz 2024-04-25T14:18:58.254 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51214.log 2024-04-25T14:18:58.255 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34813.log.gz 2024-04-25T14:18:58.256 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60646.log 2024-04-25T14:18:58.256 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43164.log.gz 2024-04-25T14:18:58.257 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39772.log 2024-04-25T14:18:58.257 INFO:teuthology.orchestra.run.smithi086.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51214.log.gz 2024-04-25T14:18:58.258 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38590.log 2024-04-25T14:18:58.258 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60646.log.gz 2024-04-25T14:18:58.259 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63054.log 2024-04-25T14:18:58.259 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39772.log.gz 2024-04-25T14:18:58.260 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.38590.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.38590.log.gz 2024-04-25T14:18:58.260 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37542.log 2024-04-25T14:18:58.261 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58105.log 2024-04-25T14:18:58.262 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63054.log: /var/log/ceph/ceph-client.admin.37542.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.63054.log.gz 2024-04-25T14:18:58.262 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51958.log 2024-04-25T14:18:58.263 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37542.log.gz 2024-04-25T14:18:58.263 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59597.log 2024-04-25T14:18:58.264 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51958.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.58105.log.gz 2024-04-25T14:18:58.264 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59676.log 2024-04-25T14:18:58.264 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51958.log.gz 2024-04-25T14:18:58.265 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54992.log 2024-04-25T14:18:58.265 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59597.log.gz 2024-04-25T14:18:58.266 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59676.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.59676.log.gz 2024-04-25T14:18:58.267 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36557.log 2024-04-25T14:18:58.267 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39596.log 2024-04-25T14:18:58.268 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54992.log.gz 2024-04-25T14:18:58.269 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38194.log 2024-04-25T14:18:58.269 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36557.log.gz 2024-04-25T14:18:58.269 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35542.log 2024-04-25T14:18:58.270 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39596.log.gz 2024-04-25T14:18:58.270 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49274.log 2024-04-25T14:18:58.271 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38194.log.gz 2024-04-25T14:18:58.271 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33419.log 2024-04-25T14:18:58.272 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35542.log.gz 2024-04-25T14:18:58.272 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37981.log 2024-04-25T14:18:58.273 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49274.log.gz 2024-04-25T14:18:58.273 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34994.log 2024-04-25T14:18:58.274 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33419.log.gz 2024-04-25T14:18:58.275 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37981.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.37981.log.gz -5 2024-04-25T14:18:58.275 INFO:teuthology.orchestra.run.smithi086.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43216.log 2024-04-25T14:18:58.276 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56390.log 2024-04-25T14:18:58.276 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34994.log.gz 2024-04-25T14:18:58.277 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27552.log 2024-04-25T14:18:58.277 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43216.log.gz 2024-04-25T14:18:58.278 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59649.log 2024-04-25T14:18:58.278 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56390.log.gz 2024-04-25T14:18:58.279 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.27552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38166.log 2024-04-25T14:18:58.279 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.27552.log.gz 2024-04-25T14:18:58.280 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56035.log 2024-04-25T14:18:58.280 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59649.log.gz 2024-04-25T14:18:58.281 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48351.log 2024-04-25T14:18:58.281 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38166.log.gz 2024-04-25T14:18:58.282 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61676.log 2024-04-25T14:18:58.283 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56035.log.gz 2024-04-25T14:18:58.283 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43729.log 2024-04-25T14:18:58.283 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48351.log.gz 2024-04-25T14:18:58.284 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39851.log 2024-04-25T14:18:58.285 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61676.log.gz 2024-04-25T14:18:58.285 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46444.log 2024-04-25T14:18:58.285 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43729.log.gz 2024-04-25T14:18:58.286 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53130.log 2024-04-25T14:18:58.287 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39851.log.gz 2024-04-25T14:18:58.287 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50453.log 2024-04-25T14:18:58.287 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46444.log.gz 2024-04-25T14:18:58.288 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54972.log 2024-04-25T14:18:58.289 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53130.log.gz 2024-04-25T14:18:58.289 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60594.log 2024-04-25T14:18:58.290 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50453.log.gz 2024-04-25T14:18:58.290 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.54972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49823.log 2024-04-25T14:18:58.291 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54972.log.gz 2024-04-25T14:18:58.292 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48715.log 2024-04-25T14:18:58.292 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60594.log.gz 2024-04-25T14:18:58.292 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60121.log 2024-04-25T14:18:58.293 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49823.log.gz 2024-04-25T14:18:58.294 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57636.log 2024-04-25T14:18:58.294 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48715.log.gz 2024-04-25T14:18:58.295 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46602.log 2024-04-25T14:18:58.295 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60121.log.gz 2024-04-25T14:18:58.296 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57636.log: 90.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.57636.log.gz -5 2024-04-25T14:18:58.296 INFO:teuthology.orchestra.run.smithi086.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50636.log 2024-04-25T14:18:58.297 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5/var/log/ceph/ceph-client.admin.46602.log: --verbose -- /var/log/ceph/ceph-client.admin.47804.log 2024-04-25T14:18:58.297 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46602.log.gz 2024-04-25T14:18:58.298 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43703.log 2024-04-25T14:18:58.298 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50636.log.gz 2024-04-25T14:18:58.299 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61409.log 2024-04-25T14:18:58.299 INFO:teuthology.orchestra.run.smithi086.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47804.log.gz 2024-04-25T14:18:58.300 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35280.log 2024-04-25T14:18:58.300 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43703.log.gz 2024-04-25T14:18:58.301 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36960.log 2024-04-25T14:18:58.301 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61409.log.gz 2024-04-25T14:18:58.302 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37038.log 2024-04-25T14:18:58.302 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35280.log.gz 2024-04-25T14:18:58.303 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40542.log 2024-04-25T14:18:58.304 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36960.log.gz 2024-04-25T14:18:58.304 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53102.log 2024-04-25T14:18:58.305 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37038.log.gz 2024-04-25T14:18:58.305 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55044.log 2024-04-25T14:18:58.306 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40542.log.gz 2024-04-25T14:18:58.307 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57584.log 2024-04-25T14:18:58.307 INFO:teuthology.orchestra.run.smithi086.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53102.log.gz 2024-04-25T14:18:58.308 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50858.log 2024-04-25T14:18:58.308 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55044.log.gz 2024-04-25T14:18:58.309 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57584.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57584.log.gz 2024-04-25T14:18:58.309 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53672.log 2024-04-25T14:18:58.309 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42680.log 2024-04-25T14:18:58.310 INFO:teuthology.orchestra.run.smithi086.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.50858.log.gz 2024-04-25T14:18:58.311 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51188.log 2024-04-25T14:18:58.311 INFO:teuthology.orchestra.run.smithi086.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53672.log.gz 2024-04-25T14:18:58.312 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33999.log 2024-04-25T14:18:58.312 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42680.log.gz 2024-04-25T14:18:58.313 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33095.log 2024-04-25T14:18:58.313 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51188.log.gz 2024-04-25T14:18:58.314 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52192.log 2024-04-25T14:18:58.314 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33999.log.gz 2024-04-25T14:18:58.315 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43651.log 2024-04-25T14:18:58.315 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33095.log.gz 2024-04-25T14:18:58.316 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53292.log 2024-04-25T14:18:58.316 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52192.log.gz 2024-04-25T14:18:58.317 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45788.log 2024-04-25T14:18:58.317 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43651.log.gz 2024-04-25T14:18:58.318 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34811.log 2024-04-25T14:18:58.319 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53292.log.gz 2024-04-25T14:18:58.319 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45788.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45788.log.gz 2024-04-25T14:18:58.319 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47752.log 2024-04-25T14:18:58.320 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/ceph-client.admin.34811.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.61766.log 2024-04-25T14:18:58.321 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34811.log.gz 2024-04-25T14:18:58.321 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47752.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47752.log.gz 2024-04-25T14:18:58.321 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37672.log 2024-04-25T14:18:58.322 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52832.log 2024-04-25T14:18:58.323 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.61766.log.gz 2024-04-25T14:18:58.323 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54557.log 2024-04-25T14:18:58.323 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37672.log.gz 2024-04-25T14:18:58.324 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.52832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33501.log 2024-04-25T14:18:58.324 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52832.log.gz 2024-04-25T14:18:58.325 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62897.log 2024-04-25T14:18:58.325 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54557.log.gz 2024-04-25T14:18:58.326 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40384.log 2024-04-25T14:18:58.327 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62897.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33501.log.gz 2024-04-25T14:18:58.327 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62897.log.gz 2024-04-25T14:18:58.328 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43631.log 2024-04-25T14:18:58.329 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49719.log 2024-04-25T14:18:58.329 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40384.log.gz 2024-04-25T14:18:58.329 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.43631.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43631.log.gz 2024-04-25T14:18:58.329 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44719.log 2024-04-25T14:18:58.330 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62134.log 2024-04-25T14:18:58.331 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49719.log.gz 2024-04-25T14:18:58.331 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33659.log 2024-04-25T14:18:58.331 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44719.log.gz 2024-04-25T14:18:58.332 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59886.log 2024-04-25T14:18:58.332 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62134.log.gz 2024-04-25T14:18:58.333 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50122.log 2024-04-25T14:18:58.334 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33659.log.gz 2024-04-25T14:18:58.334 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52667.log 2024-04-25T14:18:58.335 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59886.log.gz 2024-04-25T14:18:58.335 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47611.log 2024-04-25T14:18:58.336 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50122.log.gz 2024-04-25T14:18:58.336 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52780.log 2024-04-25T14:18:58.337 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52667.log.gz 2024-04-25T14:18:58.337 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37568.log 2024-04-25T14:18:58.338 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47611.log.gz 2024-04-25T14:18:58.338 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53776.log 2024-04-25T14:18:58.339 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52780.log: /var/log/ceph/ceph-client.admin.37568.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52780.log.gz 2024-04-25T14:18:58.340 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48481.log 2024-04-25T14:18:58.340 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37568.log.gz 2024-04-25T14:18:58.341 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40692.log 2024-04-25T14:18:58.341 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53776.log.gz 2024-04-25T14:18:58.342 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52537.log 2024-04-25T14:18:58.342 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48481.log.gz 2024-04-25T14:18:58.343 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40910.log 2024-04-25T14:18:58.343 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40692.log.gz 2024-04-25T14:18:58.344 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43960.log 2024-04-25T14:18:58.344 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52537.log.gz 2024-04-25T14:18:58.345 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38610.log 2024-04-25T14:18:58.346 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43960.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.40910.log.gz 2024-04-25T14:18:58.346 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40351.log 2024-04-25T14:18:58.346 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43960.log.gz 2024-04-25T14:18:58.347 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49683.log 2024-04-25T14:18:58.347 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38610.log.gz 2024-04-25T14:18:58.348 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.40351.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.40351.log.gz 2024-04-25T14:18:58.348 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T14:18:58.349 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/ceph-client.admin.49683.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42271.log 2024-04-25T14:18:58.350 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49683.log.gz 2024-04-25T14:18:58.350 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56877.log 2024-04-25T14:18:58.352 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38662.log 2024-04-25T14:18:58.352 INFO:teuthology.orchestra.run.smithi086.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42271.log.gz 2024-04-25T14:18:58.353 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56877.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56877.log.gz 2024-04-25T14:18:58.353 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41095.log 2024-04-25T14:18:58.354 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38662.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38662.log.gz 2024-04-25T14:18:58.354 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45736.log 2024-04-25T14:18:58.355 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38007.log 2024-04-25T14:18:58.356 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41095.log.gz 2024-04-25T14:18:58.356 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45736.log: 91.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T14:18:58.357 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39150.log 2024-04-25T14:18:58.357 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45736.log.gz 2024-04-25T14:18:58.358 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54361.log 2024-04-25T14:18:58.358 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38007.log.gz 2024-04-25T14:18:58.359 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54691.log 2024-04-25T14:18:58.359 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39150.log.gz 2024-04-25T14:18:58.360 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59099.log 2024-04-25T14:18:58.360 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54361.log.gz 2024-04-25T14:18:58.361 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49953.log 2024-04-25T14:18:58.361 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54691.log.gz 2024-04-25T14:18:58.362 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48791.log 2024-04-25T14:18:58.363 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59099.log.gz 2024-04-25T14:18:58.363 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50884.log 2024-04-25T14:18:58.363 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49953.log.gz 2024-04-25T14:18:58.365 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38937.log 2024-04-25T14:18:58.365 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48791.log.gz 2024-04-25T14:18:58.366 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33339.log 2024-04-25T14:18:58.366 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50884.log.gz 2024-04-25T14:18:58.366 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45710.log 2024-04-25T14:18:58.367 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38937.log.gz 2024-04-25T14:18:58.368 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56088.log 2024-04-25T14:18:58.368 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33339.log.gz 2024-04-25T14:18:58.369 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54388.log 2024-04-25T14:18:58.369 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45710.log.gz 2024-04-25T14:18:58.370 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48663.log 2024-04-25T14:18:58.370 INFO:teuthology.orchestra.run.smithi086.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56088.log.gz 2024-04-25T14:18:58.371 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41329.log 2024-04-25T14:18:58.371 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54388.log.gz 2024-04-25T14:18:58.372 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40798.log 2024-04-25T14:18:58.372 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48663.log.gz 2024-04-25T14:18:58.373 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55746.log 2024-04-25T14:18:58.374 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41329.log.gz 2024-04-25T14:18:58.374 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39624.log 2024-04-25T14:18:58.375 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40798.log.gz 2024-04-25T14:18:58.376 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43456.log 2024-04-25T14:18:58.376 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55746.log.gz 2024-04-25T14:18:58.376 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38858.log 2024-04-25T14:18:58.377 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39624.log.gz 2024-04-25T14:18:58.378 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38085.log 2024-04-25T14:18:58.378 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43456.log.gz 2024-04-25T14:18:58.379 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48059.log 2024-04-25T14:18:58.379 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38858.log.gz 2024-04-25T14:18:58.380 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57181.log 2024-04-25T14:18:58.380 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38085.log.gz 2024-04-25T14:18:58.381 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43534.log 2024-04-25T14:18:58.381 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48059.log.gz 2024-04-25T14:18:58.382 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52218.log 2024-04-25T14:18:58.382 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57181.log.gz 2024-04-25T14:18:58.383 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41959.log 2024-04-25T14:18:58.384 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52218.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.43534.log.gz 2024-04-25T14:18:58.384 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40606.log 2024-04-25T14:18:58.385 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52218.log.gz 2024-04-25T14:18:58.385 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50505.log 2024-04-25T14:18:58.386 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41959.log.gz 2024-04-25T14:18:58.386 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59257.log 2024-04-25T14:18:58.387 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40606.log.gz 2024-04-25T14:18:58.387 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51828.log 2024-04-25T14:18:58.388 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50505.log.gz 2024-04-25T14:18:58.389 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38963.log 2024-04-25T14:18:58.389 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59257.log.gz 2024-04-25T14:18:58.390 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34104.log 2024-04-25T14:18:58.390 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51828.log.gz 2024-04-25T14:18:58.391 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38415.log 2024-04-25T14:18:58.391 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38963.log.gz 2024-04-25T14:18:58.392 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43021.log 2024-04-25T14:18:58.392 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34104.log.gz 2024-04-25T14:18:58.393 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62767.log 2024-04-25T14:18:58.393 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38415.log.gz 2024-04-25T14:18:58.394 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54505.log 2024-04-25T14:18:58.395 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43021.log.gz 2024-04-25T14:18:58.395 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40514.log 2024-04-25T14:18:58.395 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62767.log.gz 2024-04-25T14:18:58.397 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59754.log 2024-04-25T14:18:58.397 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54505.log.gz 2024-04-25T14:18:58.398 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59728.log 2024-04-25T14:18:58.398 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40514.log.gz 2024-04-25T14:18:58.399 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55276.log 2024-04-25T14:18:58.399 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59754.log.gz 2024-04-25T14:18:58.400 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35490.log 2024-04-25T14:18:58.400 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59728.log.gz 2024-04-25T14:18:58.401 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40720.log 2024-04-25T14:18:58.401 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55276.log.gz 2024-04-25T14:18:58.402 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33787.log 2024-04-25T14:18:58.402 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35490.log.gz 2024-04-25T14:18:58.403 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38493.log 2024-04-25T14:18:58.403 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40720.log.gz 2024-04-25T14:18:58.404 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36427.log 2024-04-25T14:18:58.404 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33787.log.gz 2024-04-25T14:18:58.405 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59070.log 2024-04-25T14:18:58.406 INFO:teuthology.orchestra.run.smithi086.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.38493.log.gz 2024-04-25T14:18:58.407 INFO:teuthology.orchestra.run.smithi155.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T14:18:58.407 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58055.log 2024-04-25T14:18:58.408 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36427.log.gz 2024-04-25T14:18:58.408 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58390.log 2024-04-25T14:18:58.408 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59070.log.gz 2024-04-25T14:18:58.409 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56929.log 2024-04-25T14:18:58.409 INFO:teuthology.orchestra.run.smithi086.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.58055.log.gz 2024-04-25T14:18:58.410 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35464.log 2024-04-25T14:18:58.410 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58390.log.gz 2024-04-25T14:18:58.411 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45632.log 2024-04-25T14:18:58.411 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56929.log.gz 2024-04-25T14:18:58.412 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57543.log 2024-04-25T14:18:58.412 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35464.log.gz 2024-04-25T14:18:58.413 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50479.log 2024-04-25T14:18:58.413 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45632.log.gz 2024-04-25T14:18:58.414 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.57543.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.57543.log.gz 2024-04-25T14:18:58.414 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39369.log 2024-04-25T14:18:58.415 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58808.log 2024-04-25T14:18:58.416 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50479.log.gz 2024-04-25T14:18:58.417 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42394.log 2024-04-25T14:18:58.417 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39369.log.gz 2024-04-25T14:18:58.417 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37750.log 2024-04-25T14:18:58.418 INFO:teuthology.orchestra.run.smithi086.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.58808.log.gz 2024-04-25T14:18:58.419 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43298.log 2024-04-25T14:18:58.419 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42394.log.gz 2024-04-25T14:18:58.420 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62845.log 2024-04-25T14:18:58.420 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37750.log.gz 2024-04-25T14:18:58.421 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42089.log 2024-04-25T14:18:58.421 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43298.log.gz 2024-04-25T14:18:58.422 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55589.log 2024-04-25T14:18:58.422 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62845.log.gz 2024-04-25T14:18:58.423 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54797.log 2024-04-25T14:18:58.423 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42089.log.gz 2024-04-25T14:18:58.424 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37067.log 2024-04-25T14:18:58.424 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55589.log.gz 2024-04-25T14:18:58.425 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56273.log 2024-04-25T14:18:58.426 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54797.log.gz 2024-04-25T14:18:58.427 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56903.log 2024-04-25T14:18:58.427 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37067.log.gz 2024-04-25T14:18:58.427 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60173.log 2024-04-25T14:18:58.428 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56273.log.gz 2024-04-25T14:18:58.429 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53486.log 2024-04-25T14:18:58.429 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56903.log.gz 2024-04-25T14:18:58.430 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50239.log 2024-04-25T14:18:58.430 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60173.log.gz 2024-04-25T14:18:58.431 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52036.log 2024-04-25T14:18:58.431 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53486.log.gz 2024-04-25T14:18:58.432 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58652.log 2024-04-25T14:18:58.432 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50239.log.gz 2024-04-25T14:18:58.433 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47231.log 2024-04-25T14:18:58.433 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52036.log.gz 2024-04-25T14:18:58.434 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59363.log 2024-04-25T14:18:58.434 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58652.log.gz 2024-04-25T14:18:58.435 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44887.log 2024-04-25T14:18:58.436 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47231.log.gz 2024-04-25T14:18:58.437 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47095.log 2024-04-25T14:18:58.437 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59363.log.gz 2024-04-25T14:18:58.437 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56956.log 2024-04-25T14:18:58.438 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44887.log.gz 2024-04-25T14:18:58.439 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36167.log 2024-04-25T14:18:58.439 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47095.log.gz 2024-04-25T14:18:58.440 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47901.log 2024-04-25T14:18:58.440 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56956.log.gz 2024-04-25T14:18:58.441 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54531.log 2024-04-25T14:18:58.441 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36167.log.gz 2024-04-25T14:18:58.442 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53434.log 2024-04-25T14:18:58.442 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47901.log.gz 2024-04-25T14:18:58.443 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43404.log 2024-04-25T14:18:58.443 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54531.log.gz 2024-04-25T14:18:58.444 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39317.log 2024-04-25T14:18:58.444 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53434.log.gz 2024-04-25T14:18:58.446 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43404.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T14:18:58.446 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43404.log.gz 2024-04-25T14:18:58.447 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41843.log 2024-04-25T14:18:58.447 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39317.log.gz 2024-04-25T14:18:58.448 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46927.log 2024-04-25T14:18:58.449 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45984.log 2024-04-25T14:18:58.449 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41843.log.gz 2024-04-25T14:18:58.451 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46927.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46927.log.gz 2024-04-25T14:18:58.451 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45984.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45984.log.gz 2024-04-25T14:18:58.453 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50213.log 2024-04-25T14:18:58.454 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55250.log 2024-04-25T14:18:58.455 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53865.log 2024-04-25T14:18:58.456 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50213.log.gz/var/log/ceph/ceph-client.admin.55250.log: 2024-04-25T14:18:58.456 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55250.log.gz 2024-04-25T14:18:58.457 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54977.log 2024-04-25T14:18:58.458 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53865.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53865.log.gz 2024-04-25T14:18:58.458 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60804.log 2024-04-25T14:18:58.459 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.54977.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54977.log.gz 2024-04-25T14:18:58.459 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54587.log 2024-04-25T14:18:58.460 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57155.log 2024-04-25T14:18:58.461 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60804.log.gz 2024-04-25T14:18:58.462 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44835.log 2024-04-25T14:18:58.462 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54587.log.gz 2024-04-25T14:18:58.463 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57155.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57155.log.gz 2024-04-25T14:18:58.464 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44835.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44835.log.gz 2024-04-25T14:18:58.464 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53997.log 2024-04-25T14:18:58.466 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53182.log 2024-04-25T14:18:58.467 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43326.log 2024-04-25T14:18:58.467 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53997.log.gz 2024-04-25T14:18:58.468 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53182.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53182.log.gz 2024-04-25T14:18:58.468 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60964.log 2024-04-25T14:18:58.469 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43326.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43326.log.gz 2024-04-25T14:18:58.470 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57894.log 2024-04-25T14:18:58.471 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52069.log 2024-04-25T14:18:58.471 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60964.log.gz 2024-04-25T14:18:58.472 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57894.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57894.log.gz 2024-04-25T14:18:58.473 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52069.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52069.log.gz 2024-04-25T14:18:58.474 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40091.log 2024-04-25T14:18:58.475 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59415.log 2024-04-25T14:18:58.476 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60886.log 2024-04-25T14:18:58.477 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40091.log.gz 2024-04-25T14:18:58.477 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50837.log 2024-04-25T14:18:58.477 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59415.log.gz 2024-04-25T14:18:58.479 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60886.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60886.log.gz 2024-04-25T14:18:58.479 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.50837.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50837.log.gz 2024-04-25T14:18:58.480 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37145.log 2024-04-25T14:18:58.481 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39518.log 2024-04-25T14:18:58.482 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35008.log 2024-04-25T14:18:58.482 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37145.log.gz 2024-04-25T14:18:58.484 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39518.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39518.log.gz 2024-04-25T14:18:58.484 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35008.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35008.log.gz 2024-04-25T14:18:58.485 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36375.log 2024-04-25T14:18:58.486 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51304.log 2024-04-25T14:18:58.487 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44965.log 2024-04-25T14:18:58.487 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36375.log.gz 2024-04-25T14:18:58.488 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51304.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51304.log.gz 2024-04-25T14:18:58.489 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62637.log 2024-04-25T14:18:58.490 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44965.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44965.log.gz 2024-04-25T14:18:58.490 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49604.log 2024-04-25T14:18:58.491 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62301.log 2024-04-25T14:18:58.491 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62637.log.gz 2024-04-25T14:18:58.493 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49604.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49604.log.gz 2024-04-25T14:18:58.493 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60095.log 2024-04-25T14:18:58.494 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62301.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62301.log.gz 2024-04-25T14:18:58.494 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37171.log 2024-04-25T14:18:58.496 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58756.log 2024-04-25T14:18:58.496 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60095.log.gz 2024-04-25T14:18:58.497 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39007.log 2024-04-25T14:18:58.497 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37171.log.gz 2024-04-25T14:18:58.498 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41412.log 2024-04-25T14:18:58.498 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58756.log.gz 2024-04-25T14:18:58.499 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39007.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39007.log.gz 2024-04-25T14:18:58.500 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41412.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41412.log.gz 2024-04-25T14:18:58.500 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44745.log 2024-04-25T14:18:58.502 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33555.log 2024-04-25T14:18:58.503 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61487.log 2024-04-25T14:18:58.503 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44745.log.gz 2024-04-25T14:18:58.504 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33555.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33555.log.gz 2024-04-25T14:18:58.505 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61650.log 2024-04-25T14:18:58.506 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61487.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61487.log.gz 2024-04-25T14:18:58.506 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62108.log 2024-04-25T14:18:58.508 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50005.log 2024-04-25T14:18:58.508 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61650.log.gz 2024-04-25T14:18:58.508 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41438.log 2024-04-25T14:18:58.508 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62108.log.gz 2024-04-25T14:18:58.509 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34422.log 2024-04-25T14:18:58.510 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50005.log.gz 2024-04-25T14:18:58.511 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41438.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41438.log.gz 2024-04-25T14:18:58.512 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34422.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34422.log.gz 2024-04-25T14:18:58.512 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57099.log 2024-04-25T14:18:58.513 INFO:teuthology.orchestra.run.smithi155.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T14:18:58.514 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62372.log 2024-04-25T14:18:58.515 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34600.log 2024-04-25T14:18:58.515 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57099.log.gz 2024-04-25T14:18:58.516 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62372.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62372.log.gz 2024-04-25T14:18:58.516 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51724.log 2024-04-25T14:18:58.517 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34600.log.gz 2024-04-25T14:18:58.518 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51409.log 2024-04-25T14:18:58.519 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41147.log 2024-04-25T14:18:58.519 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51724.log.gz 2024-04-25T14:18:58.520 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48190.log 2024-04-25T14:18:58.520 INFO:teuthology.orchestra.run.smithi086.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51409.log.gz 2024-04-25T14:18:58.521 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44939.log 2024-04-25T14:18:58.521 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41147.log.gz 2024-04-25T14:18:58.523 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48190.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48190.log.gz 2024-04-25T14:18:58.523 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44939.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44939.log.gz 2024-04-25T14:18:58.524 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51984.log 2024-04-25T14:18:58.525 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49901.log 2024-04-25T14:18:58.527 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60778.log 2024-04-25T14:18:58.527 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51984.log.gz 2024-04-25T14:18:58.528 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49901.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49901.log.gz 2024-04-25T14:18:58.528 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41490.log 2024-04-25T14:18:58.529 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60778.log.gz 2024-04-25T14:18:58.530 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59126.log 2024-04-25T14:18:58.531 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59467.log 2024-04-25T14:18:58.531 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41490.log.gz 2024-04-25T14:18:58.532 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48403.log 2024-04-25T14:18:58.532 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59126.log.gz 2024-04-25T14:18:58.533 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37516.log 2024-04-25T14:18:58.533 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59467.log.gz 2024-04-25T14:18:58.534 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48403.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48403.log.gz 2024-04-25T14:18:58.535 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46101.log 2024-04-25T14:18:58.535 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37516.log.gz 2024-04-25T14:18:58.536 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51240.log 2024-04-25T14:18:58.538 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55328.log 2024-04-25T14:18:58.538 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46101.log.gz 2024-04-25T14:18:58.539 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34814.log 2024-04-25T14:18:58.539 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51240.log.gz 2024-04-25T14:18:58.540 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49771.log 2024-04-25T14:18:58.540 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55328.log.gz 2024-04-25T14:18:58.541 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34814.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34814.log.gz 2024-04-25T14:18:58.541 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44563.log 2024-04-25T14:18:58.542 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49771.log.gz 2024-04-25T14:18:58.543 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33162.log 2024-04-25T14:18:58.544 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52405.log 2024-04-25T14:18:58.545 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44563.log.gz 2024-04-25T14:18:58.545 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41661.log 2024-04-25T14:18:58.546 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33162.log.gz 2024-04-25T14:18:58.547 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53750.log 2024-04-25T14:18:58.547 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52405.log.gz 2024-04-25T14:18:58.548 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41661.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41661.log.gz 2024-04-25T14:18:58.548 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53919.log 2024-04-25T14:18:58.549 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53750.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53750.log.gz 2024-04-25T14:18:58.549 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35985.log 2024-04-25T14:18:58.551 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48216.log 2024-04-25T14:18:58.551 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53919.log.gz 2024-04-25T14:18:58.552 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35985.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35985.log.gz 2024-04-25T14:18:58.553 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48216.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48216.log.gz 2024-04-25T14:18:58.553 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61016.log 2024-04-25T14:18:58.555 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35021.log 2024-04-25T14:18:58.556 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46327.log 2024-04-25T14:18:58.557 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61016.log.gz 2024-04-25T14:18:58.557 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50688.log 2024-04-25T14:18:58.557 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35021.log.gz 2024-04-25T14:18:58.559 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46327.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46327.log.gz 2024-04-25T14:18:58.559 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50688.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50688.log.gz 2024-04-25T14:18:58.560 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51880.log 2024-04-25T14:18:58.561 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39825.log 2024-04-25T14:18:58.562 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40462.log 2024-04-25T14:18:58.563 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51880.log.gz 2024-04-25T14:18:58.563 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39825.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39825.log.gz 2024-04-25T14:18:58.564 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45595.log 2024-04-25T14:18:58.565 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40462.log.gz 2024-04-25T14:18:58.565 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60726.log 2024-04-25T14:18:58.567 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58155.log 2024-04-25T14:18:58.567 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45595.log.gz 2024-04-25T14:18:58.568 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56643.log 2024-04-25T14:18:58.568 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60726.log.gz 2024-04-25T14:18:58.569 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34370.log 2024-04-25T14:18:58.569 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58155.log.gz 2024-04-25T14:18:58.571 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56643.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56643.log.gz 2024-04-25T14:18:58.571 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34370.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34370.log.gz 2024-04-25T14:18:58.572 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61845.log 2024-04-25T14:18:58.573 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35332.log 2024-04-25T14:18:58.575 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42995.log 2024-04-25T14:18:58.575 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.61845.log.gz 2024-04-25T14:18:58.575 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35332.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35332.log.gz 2024-04-25T14:18:58.576 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60305.log 2024-04-25T14:18:58.576 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42995.log.gz 2024-04-25T14:18:58.577 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47927.log 2024-04-25T14:18:58.579 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50531.log 2024-04-25T14:18:58.579 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60305.log.gz 2024-04-25T14:18:58.580 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41069.log 2024-04-25T14:18:58.580 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47927.log.gz 2024-04-25T14:18:58.581 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53460.log 2024-04-25T14:18:58.581 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50531.log.gz 2024-04-25T14:18:58.583 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41069.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41069.log.gz 2024-04-25T14:18:58.583 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53460.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53460.log.gz 2024-04-25T14:18:58.584 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60461.log 2024-04-25T14:18:58.585 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57842.log 2024-04-25T14:18:58.587 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60461.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T14:18:58.587 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60461.log.gz 2024-04-25T14:18:58.587 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57842.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57842.log.gz 2024-04-25T14:18:58.588 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41277.log 2024-04-25T14:18:58.589 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37620.log 2024-04-25T14:18:58.591 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41277.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41277.log.gz 2024-04-25T14:18:58.592 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37620.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37620.log.gz 2024-04-25T14:18:58.592 INFO:teuthology.orchestra.run.smithi086.stderr: 87.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T14:18:58.592 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33475.log 2024-04-25T14:18:58.593 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43482.log 2024-04-25T14:18:58.595 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60069.log 2024-04-25T14:18:58.595 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33475.log.gz 2024-04-25T14:18:58.596 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41173.log 2024-04-25T14:18:58.596 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43482.log.gz 2024-04-25T14:18:58.597 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52431.log 2024-04-25T14:18:58.597 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60069.log.gz 2024-04-25T14:18:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41173.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41173.log.gz 2024-04-25T14:18:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52431.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52431.log.gz 2024-04-25T14:18:58.600 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48858.log 2024-04-25T14:18:58.601 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40746.log 2024-04-25T14:18:58.603 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48637.log 2024-04-25T14:18:58.603 INFO:teuthology.orchestra.run.smithi086.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.48858.log.gz 2024-04-25T14:18:58.603 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40746.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40746.log.gz 2024-04-25T14:18:58.604 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37873.log 2024-04-25T14:18:58.605 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48637.log.gz 2024-04-25T14:18:58.605 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42167.log 2024-04-25T14:18:58.607 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52730.log 2024-04-25T14:18:58.607 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37873.log.gz 2024-04-25T14:18:58.608 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52327.log 2024-04-25T14:18:58.608 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42167.log.gz 2024-04-25T14:18:58.609 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36219.log 2024-04-25T14:18:58.610 INFO:teuthology.orchestra.run.smithi086.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.52730.log.gz/var/log/ceph/ceph-client.admin.52327.log: 2024-04-25T14:18:58.610 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52327.log.gz 2024-04-25T14:18:58.611 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45253.log 2024-04-25T14:18:58.611 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36219.log.gz 2024-04-25T14:18:58.612 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33581.log 2024-04-25T14:18:58.614 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33260.log 2024-04-25T14:18:58.614 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45253.log.gz 2024-04-25T14:18:58.615 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27672.log 2024-04-25T14:18:58.615 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33581.log.gz 2024-04-25T14:18:58.616 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47179.log 2024-04-25T14:18:58.616 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33260.log.gz 2024-04-25T14:18:58.617 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.27672.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.27672.log.gz 2024-04-25T14:18:58.618 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34131.log 2024-04-25T14:18:58.618 INFO:teuthology.orchestra.run.smithi086.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47179.log.gz 2024-04-25T14:18:58.619 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40772.log 2024-04-25T14:18:58.621 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57946.log 2024-04-25T14:18:58.621 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34131.log.gz 2024-04-25T14:18:58.622 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38766.log 2024-04-25T14:18:58.622 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40772.log.gz 2024-04-25T14:18:58.623 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47953.log 2024-04-25T14:18:58.623 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57946.log.gz 2024-04-25T14:18:58.624 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38766.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38766.log.gz 2024-04-25T14:18:58.625 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47953.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47953.log.gz 2024-04-25T14:18:58.626 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56812.log 2024-04-25T14:18:58.627 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45569.log 2024-04-25T14:18:58.628 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59283.log 2024-04-25T14:18:58.628 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56812.log.gz 2024-04-25T14:18:58.630 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45569.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45569.log.gz 2024-04-25T14:18:58.630 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59283.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59283.log.gz 2024-04-25T14:18:58.631 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34602.log 2024-04-25T14:18:58.632 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55841.log 2024-04-25T14:18:58.633 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61278.log 2024-04-25T14:18:58.633 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34602.log.gz 2024-04-25T14:18:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55841.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55841.log.gz 2024-04-25T14:18:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53802.log 2024-04-25T14:18:58.636 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61278.log.gz 2024-04-25T14:18:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62923.log 2024-04-25T14:18:58.638 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44641.log 2024-04-25T14:18:58.638 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53802.log.gz 2024-04-25T14:18:58.639 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62923.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62923.log.gz 2024-04-25T14:18:58.639 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61356.log 2024-04-25T14:18:58.640 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44641.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44641.log.gz 2024-04-25T14:18:58.641 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56520.log 2024-04-25T14:18:58.642 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58548.log 2024-04-25T14:18:58.642 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61356.log.gz 2024-04-25T14:18:58.643 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59702.log 2024-04-25T14:18:58.643 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56520.log.gz 2024-04-25T14:18:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58548.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58548.log.gz 2024-04-25T14:18:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49196.log 2024-04-25T14:18:58.645 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59702.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.59702.log.gz -5 2024-04-25T14:18:58.645 INFO:teuthology.orchestra.run.smithi086.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51383.log 2024-04-25T14:18:58.647 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62056.log 2024-04-25T14:18:58.647 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49196.log.gz 2024-04-25T14:18:58.648 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51383.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51383.log.gz 2024-04-25T14:18:58.649 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62056.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62056.log.gz 2024-04-25T14:18:58.649 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56747.log 2024-04-25T14:18:58.650 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38690.log 2024-04-25T14:18:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59309.log 2024-04-25T14:18:58.652 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56747.log.gz 2024-04-25T14:18:58.653 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38690.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38690.log.gz 2024-04-25T14:18:58.654 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42524.log 2024-04-25T14:18:58.654 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59309.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59309.log.gz 2024-04-25T14:18:58.655 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46414.log 2024-04-25T14:18:58.656 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62424.log 2024-04-25T14:18:58.657 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42524.log.gz 2024-04-25T14:18:58.657 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46414.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46414.log.gz 2024-04-25T14:18:58.658 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57285.log 2024-04-25T14:18:58.659 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62424.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62424.log.gz 2024-04-25T14:18:58.659 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53156.log 2024-04-25T14:18:58.660 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60938.log 2024-04-25T14:18:58.660 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57285.log.gz 2024-04-25T14:18:58.661 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35410.log 2024-04-25T14:18:58.661 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53156.log.gz 2024-04-25T14:18:58.663 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60938.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60938.log.gz 2024-04-25T14:18:58.663 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35410.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35410.log.gz 2024-04-25T14:18:58.664 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48936.log 2024-04-25T14:18:58.665 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61252.log 2024-04-25T14:18:58.667 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57806.log 2024-04-25T14:18:58.667 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48936.log.gz 2024-04-25T14:18:58.668 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61252.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61252.log.gz 2024-04-25T14:18:58.669 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57806.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57806.log.gz 2024-04-25T14:18:58.670 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44259.log 2024-04-25T14:18:58.671 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38281.log 2024-04-25T14:18:58.672 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45878.log 2024-04-25T14:18:58.672 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44259.log.gz 2024-04-25T14:18:58.673 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38281.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38281.log.gz 2024-04-25T14:18:58.674 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61567.log 2024-04-25T14:18:58.674 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57038.log 2024-04-25T14:18:58.675 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45878.log.gz 2024-04-25T14:18:58.676 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33685.log 2024-04-25T14:18:58.676 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61567.log.gz 2024-04-25T14:18:58.677 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57038.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57038.log.gz 2024-04-25T14:18:58.678 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52101.log 2024-04-25T14:18:58.678 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33685.log.gz 2024-04-25T14:18:58.679 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55945.log 2024-04-25T14:18:58.681 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60253.log 2024-04-25T14:18:58.681 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52101.log.gz 2024-04-25T14:18:58.682 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61094.log 2024-04-25T14:18:58.682 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55945.log.gz 2024-04-25T14:18:58.683 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54875.log 2024-04-25T14:18:58.683 INFO:teuthology.orchestra.run.smithi086.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.60253.log.gz 2024-04-25T14:18:58.684 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61094.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61094.log.gz 2024-04-25T14:18:58.685 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55919.log 2024-04-25T14:18:58.686 INFO:teuthology.orchestra.run.smithi086.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.54875.log.gz 2024-04-25T14:18:58.686 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36479.log 2024-04-25T14:18:58.687 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61226.log 2024-04-25T14:18:58.688 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55919.log.gz 2024-04-25T14:18:58.688 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36479.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36479.log.gz 2024-04-25T14:18:58.689 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42219.log 2024-04-25T14:18:58.690 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61226.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61226.log.gz 2024-04-25T14:18:58.690 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47726.log 2024-04-25T14:18:58.691 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42115.log 2024-04-25T14:18:58.691 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42219.log.gz 2024-04-25T14:18:58.692 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47726.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47726.log.gz 2024-04-25T14:18:58.692 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48093.log 2024-04-25T14:18:58.693 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42115.log.gz 2024-04-25T14:18:58.694 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56442.log 2024-04-25T14:18:58.695 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41575.log 2024-04-25T14:18:58.695 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48093.log.gz 2024-04-25T14:18:58.696 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58940.log 2024-04-25T14:18:58.697 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56442.log.gz 2024-04-25T14:18:58.698 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44038.log 2024-04-25T14:18:58.698 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41575.log.gz 2024-04-25T14:18:58.699 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58940.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58940.log.gz 2024-04-25T14:18:58.699 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52563.log 2024-04-25T14:18:58.700 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44038.log.gz 2024-04-25T14:18:58.701 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33527.log 2024-04-25T14:18:58.702 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33391.log 2024-04-25T14:18:58.702 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52563.log.gz 2024-04-25T14:18:58.703 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34598.log 2024-04-25T14:18:58.704 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33527.log.gz 2024-04-25T14:18:58.704 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48455.log 2024-04-25T14:18:58.704 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33391.log.gz 2024-04-25T14:18:58.706 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34598.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34598.log.gz 2024-04-25T14:18:58.707 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48455.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48455.log.gz 2024-04-25T14:18:58.707 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47321.log 2024-04-25T14:18:58.708 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37308.log 2024-04-25T14:18:58.710 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47674.log 2024-04-25T14:18:58.710 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47321.log.gz 2024-04-25T14:18:58.711 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37308.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37308.log.gz 2024-04-25T14:18:58.711 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58574.log 2024-04-25T14:18:58.712 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47674.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47674.log.gz 2024-04-25T14:18:58.712 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58730.log 2024-04-25T14:18:58.714 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35358.log 2024-04-25T14:18:58.714 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.58574.log.gz 2024-04-25T14:18:58.715 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58730.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58730.log.gz 2024-04-25T14:18:58.715 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44132.log 2024-04-25T14:18:58.716 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35358.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35358.log.gz 2024-04-25T14:18:58.716 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50558.log 2024-04-25T14:18:58.718 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62689.log 2024-04-25T14:18:58.718 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44132.log.gz 2024-04-25T14:18:58.719 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50558.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50558.log.gz 2024-04-25T14:18:58.720 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62689.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62689.log.gz 2024-04-25T14:18:58.720 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40356.log 2024-04-25T14:18:58.721 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60199.log 2024-04-25T14:18:58.723 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.40356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46362.log 2024-04-25T14:18:58.723 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.40356.log.gz 2024-04-25T14:18:58.724 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60199.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60199.log.gz 2024-04-25T14:18:58.725 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46362.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46362.log.gz 2024-04-25T14:18:58.725 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45137.log 2024-04-25T14:18:58.726 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42628.log 2024-04-25T14:18:58.728 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59780.log 2024-04-25T14:18:58.728 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45137.log.gz 2024-04-25T14:18:58.729 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52925.log 2024-04-25T14:18:58.729 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42628.log.gz 2024-04-25T14:18:58.730 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59780.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59780.log.gz 2024-04-25T14:18:58.730 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36141.log 2024-04-25T14:18:58.731 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60990.log 2024-04-25T14:18:58.732 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52925.log.gz 2024-04-25T14:18:58.733 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36986.log 2024-04-25T14:18:58.733 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36141.log.gz 2024-04-25T14:18:58.734 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60990.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60990.log.gz 2024-04-25T14:18:58.735 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36986.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36986.log.gz 2024-04-25T14:18:58.735 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52299.log 2024-04-25T14:18:58.737 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61461.log 2024-04-25T14:18:58.738 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33915.log 2024-04-25T14:18:58.738 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52299.log.gz 2024-04-25T14:18:58.739 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61461.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61461.log.gz 2024-04-25T14:18:58.740 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47559.log 2024-04-25T14:18:58.741 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37093.log 2024-04-25T14:18:58.742 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47559.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.33915.log.gz 2024-04-25T14:18:58.743 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47559.log.gz 2024-04-25T14:18:58.744 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37093.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37093.log.gz 2024-04-25T14:18:58.744 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52483.log 2024-04-25T14:18:58.745 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39176.log 2024-04-25T14:18:58.747 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34606.log 2024-04-25T14:18:58.747 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52483.log.gz 2024-04-25T14:18:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40488.log 2024-04-25T14:18:58.748 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39176.log.gz 2024-04-25T14:18:58.749 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35594.log 2024-04-25T14:18:58.749 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34606.log.gz 2024-04-25T14:18:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40488.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40488.log.gz 2024-04-25T14:18:58.751 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39466.log 2024-04-25T14:18:58.751 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35594.log.gz 2024-04-25T14:18:58.752 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58782.log 2024-04-25T14:18:58.753 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48323.log 2024-04-25T14:18:58.754 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39466.log.gz 2024-04-25T14:18:58.755 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58782.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58782.log.gz 2024-04-25T14:18:58.755 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48323.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48323.log.gz 2024-04-25T14:18:58.756 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36115.log 2024-04-25T14:18:58.757 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48585.log 2024-04-25T14:18:58.759 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57491.log 2024-04-25T14:18:58.759 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36115.log.gz 2024-04-25T14:18:58.760 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48585.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48585.log.gz 2024-04-25T14:18:58.761 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57491.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57491.log.gz 2024-04-25T14:18:58.761 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43584.log 2024-04-25T14:18:58.763 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33365.log 2024-04-25T14:18:58.764 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41922.log 2024-04-25T14:18:58.764 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43584.log.gz 2024-04-25T14:18:58.765 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42316.log 2024-04-25T14:18:58.765 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33365.log.gz 2024-04-25T14:18:58.767 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41922.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41922.log.gz 2024-04-25T14:18:58.767 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42316.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42316.log.gz 2024-04-25T14:18:58.768 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37490.log 2024-04-25T14:18:58.769 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55406.log 2024-04-25T14:18:58.770 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58626.log 2024-04-25T14:18:58.770 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37490.log.gz 2024-04-25T14:18:58.772 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55406.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55406.log.gz 2024-04-25T14:18:58.773 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58626.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58626.log.gz 2024-04-25T14:18:58.773 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45215.log 2024-04-25T14:18:58.774 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37119.log 2024-04-25T14:18:58.776 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46217.log 2024-04-25T14:18:58.776 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45215.log.gz 2024-04-25T14:18:58.777 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37119.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37119.log.gz 2024-04-25T14:18:58.777 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59204.log 2024-04-25T14:18:58.778 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46217.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46217.log.gz 2024-04-25T14:18:58.778 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43986.log 2024-04-25T14:18:58.779 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62715.log 2024-04-25T14:18:58.780 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59204.log.gz 2024-04-25T14:18:58.781 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43986.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43986.log.gz 2024-04-25T14:18:58.782 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62715.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.62715.log.gz 2024-04-25T14:18:58.782 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60357.log 2024-04-25T14:18:58.783 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43190.log 2024-04-25T14:18:58.785 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41606.log 2024-04-25T14:18:58.785 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60357.log.gz 2024-04-25T14:18:58.786 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48031.log 2024-04-25T14:18:58.786 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43190.log.gz 2024-04-25T14:18:58.787 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50740.log 2024-04-25T14:18:58.787 INFO:teuthology.orchestra.run.smithi086.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41606.log.gz 2024-04-25T14:18:58.788 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48031.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48031.log.gz 2024-04-25T14:18:58.790 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50740.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.50740.log.gz 2024-04-25T14:18:58.790 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51266.log 2024-04-25T14:18:58.791 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62559.log 2024-04-25T14:18:58.793 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62346.log 2024-04-25T14:18:58.793 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51266.log.gz 2024-04-25T14:18:58.793 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62559.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62559.log.gz 2024-04-25T14:18:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39652.log 2024-04-25T14:18:58.795 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62346.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62346.log.gz 2024-04-25T14:18:58.795 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54335.log 2024-04-25T14:18:58.796 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59152.log 2024-04-25T14:18:58.797 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39652.log.gz 2024-04-25T14:18:58.797 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54335.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54335.log.gz 2024-04-25T14:18:58.798 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58678.log 2024-04-25T14:18:58.799 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59152.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59152.log.gz 2024-04-25T14:18:58.799 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33973.log 2024-04-25T14:18:58.801 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48741.log 2024-04-25T14:18:58.801 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58678.log.gz 2024-04-25T14:18:58.801 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57233.log 2024-04-25T14:18:58.802 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33973.log.gz 2024-04-25T14:18:58.803 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55489.log 2024-04-25T14:18:58.804 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57233.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.48741.log.gz 2024-04-25T14:18:58.804 INFO:teuthology.orchestra.run.smithi086.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57233.log.gz 2024-04-25T14:18:58.805 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56982.log 2024-04-25T14:18:58.806 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62082.log 2024-04-25T14:18:58.806 INFO:teuthology.orchestra.run.smithi086.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.55489.log.gz 2024-04-25T14:18:58.807 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56982.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56982.log.gz 2024-04-25T14:18:58.808 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62082.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62082.log.gz 2024-04-25T14:18:58.808 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56780.log 2024-04-25T14:18:58.810 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39570.log 2024-04-25T14:18:58.811 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56780.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56780.log.gz 2024-04-25T14:18:58.813 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39570.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39570.log.gz 2024-04-25T14:18:58.813 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54743.log 2024-04-25T14:18:58.814 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39202.log 2024-04-25T14:18:58.815 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53724.log 2024-04-25T14:18:58.815 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54743.log.gz 2024-04-25T14:18:58.817 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39202.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39202.log.gz 2024-04-25T14:18:58.817 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38389.log 2024-04-25T14:18:58.817 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53724.log.gz 2024-04-25T14:18:58.818 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42342.log 2024-04-25T14:18:58.819 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43636.log 2024-04-25T14:18:58.819 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38389.log.gz 2024-04-25T14:18:58.821 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42342.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42342.log.gz 2024-04-25T14:18:58.821 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.43636.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43636.log.gz 2024-04-25T14:18:58.822 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54023.log 2024-04-25T14:18:58.823 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40224.log 2024-04-25T14:18:58.824 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34344.log 2024-04-25T14:18:58.824 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54023.log.gz 2024-04-25T14:18:58.825 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38255.log 2024-04-25T14:18:58.826 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40224.log.gz 2024-04-25T14:18:58.827 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34344.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34344.log.gz 2024-04-25T14:18:58.828 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38255.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38255.log.gz 2024-04-25T14:18:58.828 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37412.log 2024-04-25T14:18:58.829 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52615.log 2024-04-25T14:18:58.831 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43677.log 2024-04-25T14:18:58.831 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37412.log.gz 2024-04-25T14:18:58.832 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52615.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52615.log.gz 2024-04-25T14:18:58.832 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43677.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T14:18:58.833 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43677.log.gz 2024-04-25T14:18:58.833 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54925.log 2024-04-25T14:18:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48125.log 2024-04-25T14:18:58.836 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54925.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54925.log.gz 2024-04-25T14:18:58.837 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48125.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48125.log.gz 2024-04-25T14:18:58.838 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T14:18:58.839 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42810.log 2024-04-25T14:18:58.840 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mds.c.log: 89.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T14:18:58.841 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44913.log 2024-04-25T14:18:58.842 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42810.log.gz 2024-04-25T14:18:58.842 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49578.log 2024-04-25T14:18:58.844 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44913.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44913.log.gz 2024-04-25T14:18:58.845 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60568.log 2024-04-25T14:18:58.845 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49578.log.gz 2024-04-25T14:18:58.845 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60833.log 2024-04-25T14:18:58.847 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60568.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60568.log.gz 2024-04-25T14:18:58.848 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36297.log 2024-04-25T14:18:58.848 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60833.log.gz 2024-04-25T14:18:58.849 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60859.log 2024-04-25T14:18:58.850 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36297.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36297.log.gz 2024-04-25T14:18:58.851 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43101.log 2024-04-25T14:18:58.851 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60859.log.gz 2024-04-25T14:18:58.852 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54442.log 2024-04-25T14:18:58.853 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43101.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43101.log.gz 2024-04-25T14:18:58.855 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54442.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T14:18:58.855 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54442.log.gz 2024-04-25T14:18:58.858 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58028.log 2024-04-25T14:18:58.861 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62663.log 2024-04-25T14:18:58.861 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35956.log 2024-04-25T14:18:58.861 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58028.log.gz 2024-04-25T14:18:58.862 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48611.log 2024-04-25T14:18:58.862 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62663.log.gz 2024-04-25T14:18:58.864 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.2.35956.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.35956.log.gz 2024-04-25T14:18:58.865 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36349.log 2024-04-25T14:18:58.868 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36349.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36349.log.gz 2024-04-25T14:18:58.869 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36825.log 2024-04-25T14:18:58.870 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48559.log 2024-04-25T14:18:58.877 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36825.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36825.log.gz 2024-04-25T14:18:58.877 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48559.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48559.log.gz 2024-04-25T14:18:58.877 INFO:teuthology.orchestra.run.smithi086.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48611.log.gz 2024-04-25T14:18:58.877 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37012.log 2024-04-25T14:18:58.877 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37012.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37012.log.gz 2024-04-25T14:18:58.877 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56494.log 2024-04-25T14:18:58.880 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56494.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56494.log.gz 2024-04-25T14:18:58.880 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58992.log 2024-04-25T14:18:58.881 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58522.log 2024-04-25T14:18:58.883 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58992.log: /var/log/ceph/ceph-client.admin.58522.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58992.log.gz 2024-04-25T14:18:58.883 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37646.log 2024-04-25T14:18:58.883 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58522.log.gz 2024-04-25T14:18:58.886 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37646.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37646.log.gz 2024-04-25T14:18:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22745.log 2024-04-25T14:18:58.889 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph.tmp-client.admin.22745.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22745.log.gz 2024-04-25T14:18:58.889 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46524.log 2024-04-25T14:18:58.891 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46524.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46524.log.gz 2024-04-25T14:18:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37899.log 2024-04-25T14:18:58.895 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37899.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37899.log.gz 2024-04-25T14:18:58.895 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51932.log 2024-04-25T14:18:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51932.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51932.log.gz 2024-04-25T14:18:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41303.log 2024-04-25T14:18:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41303.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41303.log.gz 2024-04-25T14:18:58.901 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37698.log 2024-04-25T14:18:58.904 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34051.log 2024-04-25T14:18:58.904 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37698.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37698.log.gz 2024-04-25T14:18:58.907 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34051.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34051.log.gz 2024-04-25T14:18:58.907 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42420.log 2024-04-25T14:18:58.910 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42420.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42420.log.gz 2024-04-25T14:18:58.910 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45658.log 2024-04-25T14:18:58.913 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45658.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45658.log.gz 2024-04-25T14:18:58.913 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61728.log 2024-04-25T14:18:58.914 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61871.log 2024-04-25T14:18:58.916 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61728.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61728.log.gz 2024-04-25T14:18:58.918 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62793.log 2024-04-25T14:18:58.918 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61871.log.gz 2024-04-25T14:18:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56695.log 2024-04-25T14:18:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62793.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62793.log.gz 2024-04-25T14:18:58.924 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56695.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56695.log.gz 2024-04-25T14:18:58.924 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54212.log 2024-04-25T14:18:58.927 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54212.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54212.log.gz 2024-04-25T14:18:58.927 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61330.log 2024-04-25T14:18:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61330.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61330.log.gz 2024-04-25T14:18:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35568.log 2024-04-25T14:18:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61382.log 2024-04-25T14:18:58.933 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35568.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35568.log.gz 2024-04-25T14:18:58.936 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54639.log 2024-04-25T14:18:58.937 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61382.log.gz 2024-04-25T14:18:58.939 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58416.log 2024-04-25T14:18:58.940 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54639.log.gz 2024-04-25T14:18:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46191.log 2024-04-25T14:18:58.943 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58416.log.gz 2024-04-25T14:18:58.946 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46191.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46191.log.gz 2024-04-25T14:18:58.947 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44861.log 2024-04-25T14:18:58.949 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44861.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44861.log.gz 2024-04-25T14:18:58.950 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55434.log 2024-04-25T14:18:58.952 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55434.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55434.log.gz 2024-04-25T14:18:58.953 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61792.log 2024-04-25T14:18:58.955 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61792.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.61792.log.gz 2024-04-25T14:18:58.956 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45684.log 2024-04-25T14:18:58.958 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45491.log 2024-04-25T14:18:58.958 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45684.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45684.log.gz 2024-04-25T14:18:58.961 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45491.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45491.log.gz 2024-04-25T14:18:58.961 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44693.log 2024-04-25T14:18:58.965 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44693.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44693.log.gz 2024-04-25T14:18:58.965 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46680.log 2024-04-25T14:18:58.968 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46680.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46680.log.gz 2024-04-25T14:18:58.968 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62223.log 2024-04-25T14:18:58.971 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40638.log 2024-04-25T14:18:58.971 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62223.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62223.log.gz 2024-04-25T14:18:58.971 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40148.log 2024-04-25T14:18:58.973 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40638.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40638.log.gz 2024-04-25T14:18:58.975 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38543.log 2024-04-25T14:18:58.976 INFO:teuthology.orchestra.run.smithi086.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.40148.log.gz 2024-04-25T14:18:58.978 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39291.log 2024-04-25T14:18:58.980 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38543.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38543.log.gz 2024-04-25T14:18:58.983 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/ceph-client.admin.39291.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39291.log.gz 2024-04-25T14:18:58.983 INFO:teuthology.orchestra.run.smithi086.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.34157.log 2024-04-25T14:18:58.989 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34157.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34157.log.gz 2024-04-25T14:18:58.989 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47533.log 2024-04-25T14:18:58.989 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47533.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47533.log.gz 2024-04-25T14:18:58.990 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36713.log 2024-04-25T14:18:58.992 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36713.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36713.log.gz 2024-04-25T14:18:58.992 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36799.log 2024-04-25T14:18:58.994 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42602.log 2024-04-25T14:18:58.996 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36799.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36799.log.gz 2024-04-25T14:18:58.997 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46388.log 2024-04-25T14:18:58.997 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42602.log.gz 2024-04-25T14:18:58.999 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46388.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46388.log.gz 2024-04-25T14:18:59.000 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50375.log 2024-04-25T14:18:59.001 INFO:teuthology.orchestra.run.smithi086.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T14:18:59.002 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62585.log 2024-04-25T14:18:59.003 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50375.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50375.log.gz 2024-04-25T14:18:59.004 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62585.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62585.log.gz 2024-04-25T14:18:59.005 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37334.log 2024-04-25T14:18:59.006 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34806.log 2024-04-25T14:18:59.007 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37334.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37334.log.gz 2024-04-25T14:18:59.008 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34806.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34806.log.gz 2024-04-25T14:18:59.008 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43378.log 2024-04-25T14:18:59.010 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39955.log 2024-04-25T14:18:59.011 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43378.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43378.log.gz 2024-04-25T14:18:59.013 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39955.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39955.log.gz 2024-04-25T14:18:59.013 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60279.log 2024-04-25T14:18:59.014 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58470.log 2024-04-25T14:18:59.016 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60279.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60279.log.gz 2024-04-25T14:18:59.016 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58470.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58470.log.gz 2024-04-25T14:18:59.017 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48884.log 2024-04-25T14:18:59.018 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43272.log 2024-04-25T14:18:59.020 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48884.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48884.log.gz 2024-04-25T14:18:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43272.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43272.log.gz 2024-04-25T14:18:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34592.log 2024-04-25T14:18:59.022 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62004.log 2024-04-25T14:18:59.024 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34592.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34592.log.gz 2024-04-25T14:18:59.024 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62004.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62004.log.gz 2024-04-25T14:18:59.025 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44589.log 2024-04-25T14:18:59.026 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60227.log 2024-04-25T14:18:59.028 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44589.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44589.log.gz 2024-04-25T14:18:59.028 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60227.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.60227.log.gz 2024-04-25T14:18:59.029 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51646.log 2024-04-25T14:18:59.030 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38229.log 2024-04-25T14:18:59.032 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51646.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51646.log.gz 2024-04-25T14:18:59.032 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38229.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38229.log.gz 2024-04-25T14:18:59.033 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50401.log 2024-04-25T14:18:59.034 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41895.log 2024-04-25T14:18:59.036 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50401.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50401.log.gz 2024-04-25T14:18:59.036 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41895.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41895.log.gz 2024-04-25T14:18:59.037 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36635.log 2024-04-25T14:18:59.038 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42011.log 2024-04-25T14:18:59.040 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36635.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36635.log.gz 2024-04-25T14:18:59.040 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42011.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42011.log.gz 2024-04-25T14:18:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60017.log 2024-04-25T14:18:59.042 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57558.log 2024-04-25T14:18:59.044 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60017.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.60017.log.gz 2024-04-25T14:18:59.045 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57558.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57558.log.gz 2024-04-25T14:18:59.045 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48910.log 2024-04-25T14:18:59.046 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58966.log 2024-04-25T14:18:59.048 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48910.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48910.log.gz 2024-04-25T14:18:59.049 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58966.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.58966.log.gz 2024-04-25T14:18:59.049 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59178.log 2024-04-25T14:18:59.050 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50057.log 2024-04-25T14:18:59.052 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59178.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59178.log.gz 2024-04-25T14:18:59.053 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50057.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50057.log.gz 2024-04-25T14:18:59.053 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36877.log 2024-04-25T14:18:59.055 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56247.log 2024-04-25T14:18:59.056 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36877.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36877.log.gz 2024-04-25T14:18:59.057 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56247.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.56247.log.gz 2024-04-25T14:18:59.057 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56299.log 2024-04-25T14:18:59.059 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49417.log 2024-04-25T14:18:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56299.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56299.log.gz 2024-04-25T14:18:59.061 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49417.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49417.log.gz 2024-04-25T14:18:59.061 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57311.log 2024-04-25T14:18:59.063 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42245.log 2024-04-25T14:18:59.064 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57311.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57311.log.gz 2024-04-25T14:18:59.065 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42245.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42245.log.gz 2024-04-25T14:18:59.066 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34025.log 2024-04-25T14:18:59.067 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58338.log 2024-04-25T14:18:59.069 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34025.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34025.log.gz 2024-04-25T14:18:59.069 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58338.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.58338.log.gz 2024-04-25T14:18:59.070 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48838.log 2024-04-25T14:18:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27771.log 2024-04-25T14:18:59.072 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.48838.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48838.log.gz 2024-04-25T14:18:59.073 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.27771.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.27771.log.gz 2024-04-25T14:18:59.074 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48429.log 2024-04-25T14:18:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35712.log 2024-04-25T14:18:59.077 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48429.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48429.log.gz 2024-04-25T14:18:59.078 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35712.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35712.log.gz 2024-04-25T14:18:59.078 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38363.log 2024-04-25T14:18:59.079 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37228.log 2024-04-25T14:18:59.081 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38363.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38363.log.gz 2024-04-25T14:18:59.081 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37228.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37228.log.gz 2024-04-25T14:18:59.082 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50910.log 2024-04-25T14:18:59.083 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51854.log 2024-04-25T14:18:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50910.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50910.log.gz 2024-04-25T14:18:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51854.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51854.log.gz 2024-04-25T14:18:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56669.log 2024-04-25T14:18:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42784.log 2024-04-25T14:18:59.089 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56669.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56669.log.gz 2024-04-25T14:18:59.090 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42784.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42784.log.gz 2024-04-25T14:18:59.090 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50031.log 2024-04-25T14:18:59.091 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38441.log 2024-04-25T14:18:59.093 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50031.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50031.log.gz 2024-04-25T14:18:59.094 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38441.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38441.log.gz 2024-04-25T14:18:59.094 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55720.log 2024-04-25T14:18:59.096 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41464.log 2024-04-25T14:18:59.097 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55720.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55720.log.gz 2024-04-25T14:18:59.098 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53318.log 2024-04-25T14:18:59.099 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41464.log.gz 2024-04-25T14:18:59.099 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58834.log 2024-04-25T14:18:59.102 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53318.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53318.log.gz 2024-04-25T14:18:59.102 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47374.log 2024-04-25T14:18:59.102 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58834.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58834.log.gz 2024-04-25T14:18:59.104 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47374.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47374.log.gz 2024-04-25T14:18:59.104 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43430.log 2024-04-25T14:18:59.105 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50714.log 2024-04-25T14:18:59.107 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43430.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43430.log.gz 2024-04-25T14:18:59.108 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50714.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50714.log.gz 2024-04-25T14:18:59.108 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45054.log 2024-04-25T14:18:59.109 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44417.log 2024-04-25T14:18:59.111 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45054.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45054.log.gz 2024-04-25T14:18:59.112 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44417.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44417.log.gz 2024-04-25T14:18:59.112 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59441.log 2024-04-25T14:18:59.114 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52827.log 2024-04-25T14:18:59.115 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59441.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59441.log.gz 2024-04-25T14:18:59.116 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.52827.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52827.log.gz 2024-04-25T14:18:59.116 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35254.log 2024-04-25T14:18:59.117 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50427.log 2024-04-25T14:18:59.119 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35254.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35254.log.gz 2024-04-25T14:18:59.120 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50427.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50427.log.gz 2024-04-25T14:18:59.120 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35228.log 2024-04-25T14:18:59.121 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53396.log 2024-04-25T14:18:59.123 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35228.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35228.log.gz 2024-04-25T14:18:59.124 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53396.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53396.log.gz 2024-04-25T14:18:59.124 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41516.log 2024-04-25T14:18:59.126 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38111.log 2024-04-25T14:18:59.127 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41516.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41516.log.gz 2024-04-25T14:18:59.128 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38111.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38111.log.gz 2024-04-25T14:18:59.128 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46069.log 2024-04-25T14:18:59.130 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59018.log 2024-04-25T14:18:59.132 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46069.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46069.log.gz 2024-04-25T14:18:59.132 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59018.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59018.log.gz 2024-04-25T14:18:59.133 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52010.log 2024-04-25T14:18:59.134 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58496.log 2024-04-25T14:18:59.136 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52010.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52010.log.gz 2024-04-25T14:18:59.136 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58496.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58496.log.gz 2024-04-25T14:18:59.137 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38137.log 2024-04-25T14:18:59.138 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56617.log 2024-04-25T14:18:59.140 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38137.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38137.log.gz 2024-04-25T14:18:59.140 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56617.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56617.log.gz 2024-04-25T14:18:59.141 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52379.log 2024-04-25T14:18:59.142 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46849.log 2024-04-25T14:18:59.144 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52379.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52379.log.gz 2024-04-25T14:18:59.144 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46849.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46849.log.gz 2024-04-25T14:18:59.145 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51698.log 2024-04-25T14:18:59.146 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59335.log 2024-04-25T14:18:59.148 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51698.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51698.log.gz 2024-04-25T14:18:59.149 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59335.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59335.log.gz 2024-04-25T14:18:59.149 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44391.log 2024-04-25T14:18:59.151 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48377.log 2024-04-25T14:18:59.152 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44391.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44391.log.gz 2024-04-25T14:18:59.153 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48377.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48377.log.gz 2024-04-25T14:18:59.153 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57868.log 2024-04-25T14:18:59.155 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36323.log 2024-04-25T14:18:59.156 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57868.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57868.log.gz 2024-04-25T14:18:59.157 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36323.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36323.log.gz 2024-04-25T14:18:59.158 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46628.log 2024-04-25T14:18:59.159 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57972.log 2024-04-25T14:18:59.161 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46628.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46628.log.gz 2024-04-25T14:18:59.161 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57972.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57972.log.gz 2024-04-25T14:18:59.162 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49927.log 2024-04-25T14:18:59.163 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57441.log 2024-04-25T14:18:59.165 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49927.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49927.log.gz 2024-04-25T14:18:59.166 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57441.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.57441.log.gz 2024-04-25T14:18:59.166 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46271.log 2024-04-25T14:18:59.167 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52457.log 2024-04-25T14:18:59.169 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46271.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46271.log.gz 2024-04-25T14:18:59.170 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52457.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52457.log.gz 2024-04-25T14:18:59.170 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62819.log 2024-04-25T14:18:59.172 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61200.log 2024-04-25T14:18:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62819.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62819.log.gz 2024-04-25T14:18:59.174 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61200.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.61200.log.gz 2024-04-25T14:18:59.175 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36934.log 2024-04-25T14:18:59.176 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48005.log 2024-04-25T14:18:59.177 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36934.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36934.log.gz 2024-04-25T14:18:59.178 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48005.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48005.log.gz 2024-04-25T14:18:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57207.log 2024-04-25T14:18:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49979.log 2024-04-25T14:18:59.181 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57207.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57207.log.gz 2024-04-25T14:18:59.182 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36505.log 2024-04-25T14:18:59.183 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49979.log.gz 2024-04-25T14:18:59.184 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34292.log 2024-04-25T14:18:59.185 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36505.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36505.log.gz 2024-04-25T14:18:59.186 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36531.log 2024-04-25T14:18:59.188 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34292.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34292.log.gz 2024-04-25T14:18:59.188 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36531.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36531.log.gz 2024-04-25T14:18:59.189 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50266.log 2024-04-25T14:18:59.190 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42368.log 2024-04-25T14:18:59.192 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50266.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50266.log.gz 2024-04-25T14:18:59.193 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42368.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42368.log.gz 2024-04-25T14:18:59.193 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36583.log 2024-04-25T14:18:59.195 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35846.log 2024-04-25T14:18:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36583.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36583.log.gz 2024-04-25T14:18:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.35846.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.35846.log.gz 2024-04-25T14:18:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49656.log 2024-04-25T14:18:59.198 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33195.log 2024-04-25T14:18:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49656.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49656.log.gz 2024-04-25T14:18:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33195.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33195.log.gz 2024-04-25T14:18:59.201 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37282.log 2024-04-25T14:18:59.202 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43047.log 2024-04-25T14:18:59.204 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37282.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37282.log.gz 2024-04-25T14:18:59.205 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43047.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43047.log.gz 2024-04-25T14:18:59.205 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52245.log 2024-04-25T14:18:59.207 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52704.log 2024-04-25T14:18:59.208 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52245.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52245.log.gz 2024-04-25T14:18:59.209 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52704.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52704.log.gz 2024-04-25T14:18:59.209 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59231.log 2024-04-25T14:18:59.211 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36609.log 2024-04-25T14:18:59.212 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59231.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59231.log.gz 2024-04-25T14:18:59.213 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36609.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36609.log.gz 2024-04-25T14:18:59.214 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47295.log 2024-04-25T14:18:59.215 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40013.log 2024-04-25T14:18:59.217 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47295.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47295.log.gz 2024-04-25T14:18:59.217 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40013.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40013.log.gz 2024-04-25T14:18:59.218 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40436.log 2024-04-25T14:18:59.219 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62949.log 2024-04-25T14:18:59.221 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40436.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40436.log.gz 2024-04-25T14:18:59.221 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62949.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62949.log.gz 2024-04-25T14:18:59.222 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60409.log 2024-04-25T14:18:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49364.log 2024-04-25T14:18:59.225 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60409.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60409.log.gz 2024-04-25T14:18:59.225 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49364.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49364.log.gz 2024-04-25T14:18:59.226 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57010.log 2024-04-25T14:18:59.227 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42576.log 2024-04-25T14:18:59.229 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57010.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57010.log.gz 2024-04-25T14:18:59.230 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42576.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42576.log.gz 2024-04-25T14:18:59.230 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34500.log 2024-04-25T14:18:59.231 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62030.log 2024-04-25T14:18:59.233 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34500.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34500.log.gz 2024-04-25T14:18:59.234 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62030.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62030.log.gz 2024-04-25T14:18:59.234 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37438.log 2024-04-25T14:18:59.235 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39713.log 2024-04-25T14:18:59.237 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37438.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37438.log.gz 2024-04-25T14:18:59.238 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39713.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.39713.log.gz 2024-04-25T14:18:59.238 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47830.log 2024-04-25T14:18:59.239 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49875.log 2024-04-25T14:18:59.241 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47830.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47830.log.gz 2024-04-25T14:18:59.242 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49875.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49875.log.gz 2024-04-25T14:18:59.242 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39228.log 2024-04-25T14:18:59.244 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51356.log 2024-04-25T14:18:59.245 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39228.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39228.log.gz 2024-04-25T14:18:59.246 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51356.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51356.log.gz 2024-04-25T14:18:59.246 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54075.log 2024-04-25T14:18:59.248 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62871.log 2024-04-25T14:18:59.249 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54075.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54075.log.gz 2024-04-25T14:18:59.250 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62871.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62871.log.gz 2024-04-25T14:18:59.251 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44469.log 2024-04-25T14:18:59.252 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53513.log 2024-04-25T14:18:59.253 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44469.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44469.log.gz 2024-04-25T14:18:59.254 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53513.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53513.log.gz 2024-04-25T14:18:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51750.log 2024-04-25T14:18:59.256 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41817.log 2024-04-25T14:18:59.257 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51750.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51750.log.gz 2024-04-25T14:18:59.258 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59834.log 2024-04-25T14:18:59.261 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45762.log 2024-04-25T14:18:59.261 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41817.log.gz 2024-04-25T14:18:59.261 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59834.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59834.log.gz 2024-04-25T14:18:59.262 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45762.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45762.log.gz 2024-04-25T14:18:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41251.log 2024-04-25T14:18:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40250.log 2024-04-25T14:18:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41251.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41251.log.gz 2024-04-25T14:18:59.266 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40250.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40250.log.gz 2024-04-25T14:18:59.267 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57125.log 2024-04-25T14:18:59.268 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52166.log 2024-04-25T14:18:59.270 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57125.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57125.log.gz 2024-04-25T14:18:59.270 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33633.log 2024-04-25T14:18:59.271 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52166.log.gz 2024-04-25T14:18:59.272 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46165.log 2024-04-25T14:18:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33633.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33633.log.gz 2024-04-25T14:18:59.274 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61304.log 2024-04-25T14:18:59.276 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46165.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46165.log.gz 2024-04-25T14:18:59.276 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61304.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61304.log.gz 2024-04-25T14:18:59.277 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55983.log 2024-04-25T14:18:59.278 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41869.log 2024-04-25T14:18:59.280 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55983.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55983.log.gz 2024-04-25T14:18:59.280 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41869.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41869.log.gz 2024-04-25T14:18:59.281 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38311.log 2024-04-25T14:18:59.282 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61174.log 2024-04-25T14:18:59.284 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38311.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38311.log.gz 2024-04-25T14:18:59.284 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61174.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61174.log.gz 2024-04-25T14:18:59.285 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40570.log 2024-04-25T14:18:59.286 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46470.log 2024-04-25T14:18:59.288 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40570.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40570.log.gz 2024-04-25T14:18:59.289 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46470.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46470.log.gz 2024-04-25T14:18:59.289 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38636.log 2024-04-25T14:18:59.290 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34183.log 2024-04-25T14:18:59.292 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38636.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38636.log.gz 2024-04-25T14:18:59.293 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34183.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34183.log.gz 2024-04-25T14:18:59.293 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59545.log 2024-04-25T14:18:59.294 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56325.log 2024-04-25T14:18:59.296 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59545.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59545.log.gz 2024-04-25T14:18:59.297 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56325.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56325.log.gz 2024-04-25T14:18:59.297 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34209.log 2024-04-25T14:18:59.299 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39544.log 2024-04-25T14:18:59.300 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34209.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34209.log.gz 2024-04-25T14:18:59.301 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39544.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39544.log.gz 2024-04-25T14:18:59.301 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61539.log 2024-04-25T14:18:59.303 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48689.log 2024-04-25T14:18:59.304 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61539.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61539.log.gz 2024-04-25T14:18:59.305 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48689.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48689.log.gz 2024-04-25T14:18:59.305 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52641.log 2024-04-25T14:18:59.307 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46245.log 2024-04-25T14:18:59.308 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52641.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52641.log.gz 2024-04-25T14:18:59.309 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46245.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46245.log.gz 2024-04-25T14:18:59.310 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60435.log 2024-04-25T14:18:59.311 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55018.log 2024-04-25T14:18:59.312 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60435.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60435.log.gz 2024-04-25T14:18:59.313 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55018.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55018.log.gz 2024-04-25T14:18:59.314 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47778.log 2024-04-25T14:18:59.316 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47778.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47778.log.gz 2024-04-25T14:18:59.317 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35010.log 2024-04-25T14:18:59.318 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34812.log 2024-04-25T14:18:59.319 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35010.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35010.log.gz 2024-04-25T14:18:59.320 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34812.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34812.log.gz 2024-04-25T14:18:59.321 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44101.log 2024-04-25T14:18:59.322 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43073.log 2024-04-25T14:18:59.324 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44101.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44101.log.gz 2024-04-25T14:18:59.324 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43073.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43073.log.gz 2024-04-25T14:18:59.325 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56351.log 2024-04-25T14:18:59.326 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40960.log 2024-04-25T14:18:59.327 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56351.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56351.log.gz 2024-04-25T14:18:59.328 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40960.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40960.log.gz 2024-04-25T14:18:59.329 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35646.log 2024-04-25T14:18:59.330 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50320.log 2024-04-25T14:18:59.333 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35646.log: /var/log/ceph/ceph-client.admin.50320.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.35646.log.gz 2024-04-25T14:18:59.333 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50320.log.gz 2024-04-25T14:18:59.334 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62611.log 2024-04-25T14:18:59.334 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44667.log 2024-04-25T14:18:59.336 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62611.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62611.log.gz 2024-04-25T14:18:59.337 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44667.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44667.log.gz 2024-04-25T14:18:59.337 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45279.log 2024-04-25T14:18:59.338 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48533.log 2024-04-25T14:18:59.340 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45279.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45279.log.gz 2024-04-25T14:18:59.341 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48533.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48533.log.gz 2024-04-25T14:18:59.341 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52271.log 2024-04-25T14:18:59.343 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45543.log 2024-04-25T14:18:59.344 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52271.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52271.log.gz 2024-04-25T14:18:59.345 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45543.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45543.log.gz 2024-04-25T14:18:59.345 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47585.log 2024-04-25T14:18:59.347 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44615.log 2024-04-25T14:18:59.349 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47585.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47585.log.gz 2024-04-25T14:18:59.349 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44615.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44615.log.gz 2024-04-25T14:18:59.350 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57073.log 2024-04-25T14:18:59.351 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54470.log 2024-04-25T14:18:59.353 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57073.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57073.log.gz 2024-04-25T14:18:59.353 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54470.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54470.log.gz 2024-04-25T14:18:59.354 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49312.log 2024-04-25T14:18:59.355 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42193.log 2024-04-25T14:18:59.357 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49312.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49312.log.gz 2024-04-25T14:18:59.357 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42193.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42193.log.gz 2024-04-25T14:18:59.358 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34396.log 2024-04-25T14:18:59.362 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43934.log 2024-04-25T14:18:59.362 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34396.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34396.log.gz 2024-04-25T14:18:59.363 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43934.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43934.log.gz 2024-04-25T14:18:59.363 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59990.log 2024-04-25T14:18:59.363 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44797.log 2024-04-25T14:18:59.365 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59990.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59990.log.gz 2024-04-25T14:18:59.366 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44797.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44797.log.gz 2024-04-25T14:18:59.366 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40830.log 2024-04-25T14:18:59.367 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34591.log 2024-04-25T14:18:59.369 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40830.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40830.log.gz 2024-04-25T14:18:59.370 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34591.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34591.log.gz 2024-04-25T14:18:59.370 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51330.log 2024-04-25T14:18:59.372 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35620.log 2024-04-25T14:18:59.373 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51330.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51330.log.gz 2024-04-25T14:18:59.374 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35620.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35620.log.gz 2024-04-25T14:18:59.374 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59519.log 2024-04-25T14:18:59.376 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59964.log 2024-04-25T14:18:59.377 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59519.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59519.log.gz 2024-04-25T14:18:59.378 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59964.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.59964.log.gz 2024-04-25T14:18:59.379 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42836.log 2024-04-25T14:18:59.381 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42836.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42836.log.gz 2024-04-25T14:18:59.419 INFO:teuthology.orchestra.run.smithi155.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T14:18:59.420 INFO:teuthology.orchestra.run.smithi086.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T14:18:59.439 INFO:teuthology.orchestra.run.smithi086.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T14:18:59.479 INFO:teuthology.orchestra.run.smithi155.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T14:18:59.658 INFO:teuthology.orchestra.run.smithi155.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T14:18:59.861 INFO:teuthology.orchestra.run.smithi086.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T14:19:00.067 INFO:teuthology.orchestra.run.smithi086.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T14:19:00.216 INFO:teuthology.orchestra.run.smithi086.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T14:19:00.244 INFO:teuthology.orchestra.run.smithi086.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T14:19:00.246 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-04-25T14:19:00.246 INFO:teuthology.orchestra.run.smithi086.stderr:real 0m2.310s 2024-04-25T14:19:00.247 INFO:teuthology.orchestra.run.smithi086.stderr:user 0m10.177s 2024-04-25T14:19:00.247 INFO:teuthology.orchestra.run.smithi086.stderr:sys 0m2.335s 2024-04-25T14:19:00.617 INFO:teuthology.orchestra.run.smithi155.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T14:19:00.619 INFO:teuthology.orchestra.run.smithi155.stderr: 2024-04-25T14:19:00.619 INFO:teuthology.orchestra.run.smithi155.stderr:real 0m2.666s 2024-04-25T14:19:00.619 INFO:teuthology.orchestra.run.smithi155.stderr:user 0m7.983s 2024-04-25T14:19:00.619 INFO:teuthology.orchestra.run.smithi155.stderr:sys 0m0.638s 2024-04-25T14:19:00.620 INFO:tasks.ceph:Archiving logs... 2024-04-25T14:19:00.620 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673185/remote/smithi086/log 2024-04-25T14:19:00.621 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T14:19:02.414 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673185/remote/smithi155/log 2024-04-25T14:19:02.415 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T14:19:03.408 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T14:19:03.417 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T14:19:03.418 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T14:19:03.421 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T14:19:03.497 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T14:19:03.507 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T14:19:03.507 DEBUG:teuthology.orchestra.run.smithi086:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T14:19:03.510 DEBUG:teuthology.orchestra.run.smithi155:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T14:19:03.523 INFO:teuthology.orchestra.run.smithi086.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T14:19:03.523 INFO:teuthology.orchestra.run.smithi086.stdout:============================================================================== 2024-04-25T14:19:03.523 INFO:teuthology.orchestra.run.smithi086.stdout:+hv01.front.sepi 67.205.162.81 3 u 6 64 377 0.080 -5.606 1.269 2024-04-25T14:19:03.523 INFO:teuthology.orchestra.run.smithi086.stdout:*hv02.front.sepi 63.231.80.2 3 u 55 64 377 0.084 -3.091 0.995 2024-04-25T14:19:03.523 INFO:teuthology.orchestra.run.smithi086.stdout:+hv03.front.sepi 74.6.168.72 3 u 4 64 377 0.087 -6.798 0.934 2024-04-25T14:19:03.523 INFO:teuthology.orchestra.run.smithi086.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-04-25T14:19:03.525 INFO:teuthology.orchestra.run.smithi155.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T14:19:03.525 INFO:teuthology.orchestra.run.smithi155.stdout:============================================================================== 2024-04-25T14:19:03.525 INFO:teuthology.orchestra.run.smithi155.stdout:*hv01.front.sepi 67.205.162.81 3 u 65 64 377 0.094 -4.776 1.588 2024-04-25T14:19:03.525 INFO:teuthology.orchestra.run.smithi155.stdout:+hv02.front.sepi 63.231.80.2 3 u 53 64 377 0.074 -3.941 0.860 2024-04-25T14:19:03.525 INFO:teuthology.orchestra.run.smithi155.stdout:+hv03.front.sepi 74.6.168.72 3 u 61 64 377 0.083 -6.130 0.880 2024-04-25T14:19:03.526 INFO:teuthology.orchestra.run.smithi155.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-04-25T14:19:03.527 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T14:19:03.536 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T14:19:03.536 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T14:19:03.545 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T14:19:03.555 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T14:19:03.564 INFO:teuthology.task.internal:Duration was 1645.262571 seconds 2024-04-25T14:19:03.564 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T14:19:03.574 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T14:19:03.575 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T14:19:03.578 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T14:19:03.672 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T14:19:03.672 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi086.front.sepia.ceph.com 2024-04-25T14:19:03.672 DEBUG:teuthology.orchestra.run.smithi086:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T14:19:03.692 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi155.front.sepia.ceph.com 2024-04-25T14:19:03.692 DEBUG:teuthology.orchestra.run.smithi155:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T14:19:03.712 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T14:19:03.712 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T14:19:03.740 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T14:19:03.791 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T14:19:03.791 DEBUG:teuthology.orchestra.run.smithi086:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T14:19:03.824 DEBUG:teuthology.orchestra.run.smithi155:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T14:19:03.923 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T14:19:03.934 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T14:19:03.934 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T14:19:03.953 DEBUG:teuthology.orchestra.run.smithi155:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T14:19:03.973 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T14:19:03.983 DEBUG:teuthology.orchestra.run.smithi086:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T14:19:04.004 DEBUG:teuthology.orchestra.run.smithi155:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T14:19:04.018 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = core 2024-04-25T14:19:04.035 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = core 2024-04-25T14:19:04.053 DEBUG:teuthology.orchestra.run.smithi086:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T14:19:04.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:19:04.087 DEBUG:teuthology.orchestra.run.smithi155:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T14:19:04.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:19:04.106 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T14:19:04.116 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T14:19:04.117 DEBUG:teuthology.misc:Transferring archived files from smithi086:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673185/remote/smithi086 2024-04-25T14:19:04.117 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T14:19:04.160 DEBUG:teuthology.misc:Transferring archived files from smithi155:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673185/remote/smithi155 2024-04-25T14:19:04.160 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T14:19:04.184 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T14:19:04.185 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T14:19:04.200 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T14:19:04.230 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T14:19:04.242 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T14:19:04.242 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T14:19:04.252 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T14:19:04.252 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T14:19:04.255 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T14:19:04.261 INFO:teuthology.orchestra.run.smithi086.stdout: 658122 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 14:19 /home/ubuntu/cephtest 2024-04-25T14:19:04.278 INFO:teuthology.orchestra.run.smithi155.stdout: 655441 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 14:19 /home/ubuntu/cephtest 2024-04-25T14:19:04.280 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T14:19:04.291 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T14:19:04.311 INFO:teuthology.nuke:Checking targets against current locks 2024-04-25T14:19:04.338 DEBUG:teuthology.nuke:shortname: smithi086 2024-04-25T14:19:04.338 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T14:19:04.361 DEBUG:teuthology.nuke:shortname: smithi155 2024-04-25T14:19:04.361 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T14:19:04.364 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673185', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 13:39:15.316923', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI/vyPCz35KI9WNWis9mr7nPzhgchQLOuGOS0aRaeuodF47ZVBEp1Nc94q0LWfJLamHLYt/NGpuniB3x65kAv+Y='} 2024-04-25T14:19:04.389 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673185', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 13:39:15.315026', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJBmjCfRPUUkkwsMdWyBgBoqX7RlPMKPwyPeoefSpw0DFoNQnpoLioDSL4biIsefldSwVDMQ0l+7kbfLgBZVYQ4='} 2024-04-25T14:19:04.408 INFO:teuthology.orchestra.console.smithi086:Power off 2024-04-25T14:19:04.408 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T14:19:04.432 DEBUG:teuthology.orchestra.console.smithi086:power off output: Chassis Power Control: Down/Off 2024-04-25T14:19:04.432 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T14:19:04.451 INFO:teuthology.orchestra.console.smithi155:Power off 2024-04-25T14:19:04.451 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T14:19:04.461 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2024-04-25T14:19:04.480 DEBUG:teuthology.orchestra.console.smithi155:power off output: Chassis Power Control: Down/Off 2024-04-25T14:19:04.480 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T14:19:04.513 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2024-04-25T14:19:08.462 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T14:19:08.514 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T14:19:08.574 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2024-04-25T14:19:08.625 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2024-04-25T14:19:12.577 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T14:19:12.626 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T14:19:12.688 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is off 2024-04-25T14:19:12.736 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2024-04-25T14:19:12.789 INFO:teuthology.orchestra.console.smithi086:Power off completed 2024-04-25T14:19:16.739 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T14:19:16.852 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is off 2024-04-25T14:19:16.952 INFO:teuthology.orchestra.console.smithi155:Power off completed 2024-04-25T14:19:17.058 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 client.1-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/metrics} duration: 1645.2625708580017 failure_reason: 'Test failure: test_metrics_mds_grow_and_shrink (tasks.cephfs.test_mds_metrics.TestMDSMetrics)' flavor: default owner: scheduled_pdonnell@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=9cb6ce27042041dcbfe30d7455ebf140 status: fail success: false 2024-04-25T14:19:17.059 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T14:19:17.146 INFO:teuthology.run:FAIL